2024-04-26T17:53:22.334 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T17:53:22.334 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674904 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --name leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi --verbose -- /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674904/orig.config.yaml 2024-04-26T17:53:22.398 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T17:53:22.502 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674904 branch: wip-lusov-quiescer-fixes description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: null first_in_suite: false job_id: '7674904' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer-fixes ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer-fixes sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9496 sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer-fixes suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKc9zfSoxEr0bfslYnlZhDse9dek2xMw3w7Chx7NsG+gKIJzcXkJLVUpKmtoWfOtYdo0uavX48H8l30fhGdaO8= smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnlYeWJ82Gb7ZtY6pFRtwD/gy78lT0IZlqisSwDBvamkWCHCO6Pkp7XfkB4LZnCki9kczkQCze2y9bkPAkHQFU= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFJlZY1qsScv4AikiiIhGMR2DRRBkAuZr+S85KNgUHxozwW3lDG+ygClxEwVToP+1tIIfLyOqQu5jG+9ZGzIVwY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_17:24:23 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T17:53:22.503 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa; will attempt to use it 2024-04-26T17:53:22.503 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks 2024-04-26T17:53:22.504 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T17:53:22.505 INFO:teuthology.task.internal:Checking packages... 2024-04-26T17:53:22.534 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1' 2024-04-26T17:53:22.534 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T17:53:22.534 INFO:teuthology.packaging:ref: None 2024-04-26T17:53:22.534 INFO:teuthology.packaging:tag: None 2024-04-26T17:53:22.535 INFO:teuthology.packaging:branch: wip-lusov-quiescer-fixes 2024-04-26T17:53:22.535 INFO:teuthology.packaging:sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:53:22.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer-fixes 2024-04-26T17:53:22.756 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3267.gbc94a2a9 2024-04-26T17:53:22.757 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T17:53:22.765 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T17:53:22.765 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T17:53:22.778 INFO:teuthology.task.internal:Saving configuration 2024-04-26T17:53:22.801 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T17:53:22.835 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T17:53:22.860 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 17:47:49.027603', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDKc9zfSoxEr0bfslYnlZhDse9dek2xMw3w7Chx7NsG+gKIJzcXkJLVUpKmtoWfOtYdo0uavX48H8l30fhGdaO8='} 2024-04-26T17:53:22.884 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 17:47:49.029442', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEnlYeWJ82Gb7ZtY6pFRtwD/gy78lT0IZlqisSwDBvamkWCHCO6Pkp7XfkB4LZnCki9kczkQCze2y9bkPAkHQFU='} 2024-04-26T17:53:22.907 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674904', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 17:47:49.030976', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFJlZY1qsScv4AikiiIhGMR2DRRBkAuZr+S85KNgUHxozwW3lDG+ygClxEwVToP+1tIIfLyOqQu5jG+9ZGzIVwY='} 2024-04-26T17:53:22.907 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T17:53:22.914 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T17:53:22.915 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T17:53:22.915 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T17:53:22.915 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T17:53:23.059 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcd8357c0d0>, signals=[15]) 2024-04-26T17:53:23.059 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T17:53:23.067 INFO:teuthology.task.internal:Opening connections... 2024-04-26T17:53:23.068 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T17:53:23.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T17:53:23.148 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2024-04-26T17:53:23.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T17:53:23.226 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-04-26T17:53:23.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T17:53:23.303 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T17:53:23.311 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-04-26T17:53:23.329 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-04-26T17:53:23.330 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T17:53:23.387 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-26T17:53:23.388 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-26T17:53:23.388 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T17:53:23.388 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-26T17:53:23.388 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T17:53:23.388 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T17:53:23.388 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T17:53:23.388 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-26T17:53:23.421 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2024-04-26T17:53:23.439 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2024-04-26T17:53:23.439 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T17:53:23.496 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2024-04-26T17:53:23.497 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T17:53:23.497 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T17:53:23.497 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T17:53:23.497 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2024-04-26T17:53:23.521 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-04-26T17:53:23.539 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-04-26T17:53:23.540 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-04-26T17:53:23.597 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-04-26T17:53:23.598 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-04-26T17:53:23.599 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T17:53:23.599 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-04-26T17:53:23.599 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T17:53:23.599 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T17:53:23.599 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T17:53:23.599 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-04-26T17:53:23.621 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T17:53:23.633 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T17:53:23.649 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T17:53:23.649 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-04-26T17:53:23.652 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2024-04-26T17:53:23.655 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-04-26T17:53:23.673 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T17:53:23.723 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T17:53:23.723 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-04-26T17:53:23.726 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2024-04-26T17:53:23.728 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-04-26T17:53:23.825 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T17:53:23.911 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T17:53:23.927 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-26T17:53:23.928 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-26T17:53:23.928 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-26T17:53:23.928 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T17:53:23.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T17:53:23.945 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-04-26T17:53:24.001 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-437.el9.x86_64 2024-04-26T17:53:24.002 INFO:teuthology.task.kernel:Running kernel on smithi090: 5.14.0-437.el9.x86_64 2024-04-26T17:53:24.002 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum install -y kernel 2024-04-26T17:53:26.697 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T17:53:30.665 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 6.6 MB/s | 19 MB 00:02 2024-04-26T17:53:35.483 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.2 MB 00:01 2024-04-26T17:53:37.068 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-26T17:53:39.347 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-04-26T17:53:44.337 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-04-26T17:53:46.219 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T17:53:46.219 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T17:53:46.327 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T17:53:46.328 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T17:53:46.328 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T17:53:46.431 DEBUG:teuthology.orchestra.run.smithi090:> echo no | sudo yum reinstall kernel || true 2024-04-26T17:53:46.930 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 05:53:44 PM UTC. 2024-04-26T17:53:47.053 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalling: 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 255 k 2024-04-26T17:53:47.071 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 0 2024-04-26T17:53:47.072 INFO:teuthology.orchestra.run.smithi090.stderr:Operation aborted. 2024-04-26T17:53:47.158 INFO:teuthology.orchestra.run.smithi090.stdout:Is this ok [y/N]: 2024-04-26T17:53:47.159 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum reinstall -y kernel || true 2024-04-26T17:53:47.646 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 05:53:44 PM UTC. 2024-04-26T17:53:47.761 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T17:53:47.762 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T17:53:47.762 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T17:53:47.762 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T17:53:47.762 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalling: 2024-04-26T17:53:47.762 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T17:53:47.763 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:53:47.763 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T17:53:47.763 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T17:53:47.763 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:53:47.763 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 255 k 2024-04-26T17:53:47.763 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 0 2024-04-26T17:53:47.763 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T17:53:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.8 MB/s | 255 kB 00:00 2024-04-26T17:53:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T17:53:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.5 MB/s | 255 kB 00:00 2024-04-26T17:53:47.953 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T17:53:48.156 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T17:53:48.156 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T17:53:48.161 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T17:53:48.161 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T17:53:48.300 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T17:53:48.392 INFO:teuthology.orchestra.run.smithi090.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T17:53:48.392 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T17:53:49.243 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T17:53:49.244 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T17:53:50.043 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T17:53:50.043 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:53:50.043 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalled: 2024-04-26T17:53:50.043 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T17:53:50.043 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:53:50.044 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T17:53:50.254 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T17:53:50.301 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T17:53:50.302 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T17:53:50.302 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T17:53:50.302 DEBUG:teuthology.parallel:result is None 2024-04-26T17:53:50.302 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T17:53:50.310 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T17:53:50.310 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T17:53:50.312 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T17:53:50.315 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T17:53:50.333 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T17:53:50.340 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T17:53:50.348 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T17:53:50.348 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T17:53:50.367 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T17:53:50.373 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T17:53:50.414 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T17:53:50.422 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T17:53:50.422 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T17:53:50.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T17:53:50.457 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T17:53:50.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T17:53:50.473 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T17:53:50.476 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T17:53:50.500 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T17:53:50.501 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T17:53:50.521 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T17:53:50.530 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T17:53:50.549 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T17:53:50.553 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T17:53:50.572 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T17:53:50.574 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T17:53:50.581 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T17:53:50.581 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T17:53:50.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T17:53:50.593 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T17:53:50.644 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T17:53:50.652 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T17:53:50.652 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T17:53:50.654 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T17:53:50.661 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T17:53:50.701 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T17:53:50.746 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T17:53:50.797 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T17:53:50.878 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T17:53:50.917 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T17:53:50.917 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T17:53:50.981 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T17:53:51.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T17:53:51.075 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T17:53:51.157 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T17:53:51.198 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T17:53:51.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T17:53:51.264 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T17:53:51.306 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T17:53:51.352 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T17:53:51.437 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T17:53:51.476 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T17:53:51.477 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T17:53:51.542 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-04-26T17:53:51.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2024-04-26T17:53:51.547 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-04-26T17:53:51.579 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T17:53:51.583 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T17:53:51.619 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T17:53:52.104 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T17:53:52.111 INFO:teuthology.task.internal:Starting timer... 2024-04-26T17:53:52.111 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T17:53:52.157 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T17:53:52.176 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T17:53:52.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-04-26T17:53:52.258 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-04-26T17:53:52.260 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2024-04-26T17:53:52.321 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2024-04-26T17:53:52.323 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-04-26T17:53:52.397 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-04-26T17:53:52.399 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T17:53:52.399 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-04-26T17:53:52.421 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-04-26T17:53:52.421 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2024-04-26T17:53:52.465 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2024-04-26T17:53:52.466 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-04-26T17:53:52.490 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-04-26T17:53:52.490 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-04-26T17:53:52.490 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T17:53:52.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T17:53:52.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T17:53:52.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T17:53:52.541 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T17:53:52.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T17:53:52.568 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T17:53:52.568 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-04-26T17:53:52.594 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2024-04-26T17:53:52.618 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-04-26T17:53:52.647 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T17:53:52.661 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T17:53:52.667 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T17:53:52.800 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T17:53:52.817 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T17:53:52.819 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi119.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T17:57:43.452 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-04-26T17:57:43.454 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-26T17:57:43.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T17:57:43.532 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-26T17:57:43.609 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-26T17:57:43.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-26T17:57:43.610 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T17:57:43.685 DEBUG:teuthology.orchestra.run.smithi119:> true 2024-04-26T17:57:43.727 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2024-04-26T17:57:43.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-26T17:57:43.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T17:57:43.802 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-04-26T17:57:43.878 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-04-26T17:57:43.879 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T17:57:43.892 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T17:57:43.892 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T17:57:43.893 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T17:57:43.895 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T17:57:43.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T17:57:43.898 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T17:57:43.898 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T17:57:43.923 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T17:57:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T17:57:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T17:57:43.937 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T17:57:43.947 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T17:57:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2024-04-26T17:57:43.961 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-04-26T17:57:43.962 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T17:57:43.969 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2024-04-26T17:57:43.971 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-04-26T17:57:43.982 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T17:57:43.985 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T17:57:43.985 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-04-26T17:57:43.994 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T17:57:43.996 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-04-26T17:57:43.998 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T17:57:44.009 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T17:57:44.022 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T17:57:44.050 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2024-04-26T17:57:44.051 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2024-04-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.053 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.055 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T17:57:44.055 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-26T17:57:44.055 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.055 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.055 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.055 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.079 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-04-26T17:57:44.082 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T17:57:44.083 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-04-26T17:57:44.083 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.083 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.083 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.083 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T17:57:44.084 INFO:teuthology.run_tasks:Running task install... 2024-04-26T17:57:44.093 DEBUG:teuthology.task.install:project ceph 2024-04-26T17:57:44.093 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'}} 2024-04-26T17:57:44.093 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'} 2024-04-26T17:57:44.093 INFO:teuthology.task.install:Using flavor: default 2024-04-26T17:57:44.099 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T17:57:44.099 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T17:57:44.099 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T17:57:44.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:57:44.102 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T17:57:44.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:57:44.103 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T17:57:44.103 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:57:44.320 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T17:57:44.321 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T17:57:44.327 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T17:57:44.328 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T17:57:44.334 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T17:57:44.334 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T17:57:44.433 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T17:57:44.433 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T17:57:44.433 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T17:57:44.458 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T17:57:44.459 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T17:57:44.500 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T17:57:44.500 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T17:57:44.500 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T17:57:44.516 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T17:57:44.516 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T17:57:44.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T17:57:44.524 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T17:57:44.525 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T17:57:44.526 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T17:57:44.541 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T17:57:44.541 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T17:57:44.590 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T17:57:44.597 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T17:57:44.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T17:57:44.663 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-04-26T17:57:44.665 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-04-26T17:57:44.667 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T17:57:44.678 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T17:57:44.729 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-04-26T17:57:44.731 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-04-26T17:57:44.741 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2024-04-26T17:57:44.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2024-04-26T17:57:44.888 INFO:teuthology.orchestra.run.smithi090.stdout:48 files removed 2024-04-26T17:57:44.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T17:57:44.954 INFO:teuthology.orchestra.run.smithi167.stdout:48 files removed 2024-04-26T17:57:44.969 INFO:teuthology.orchestra.run.smithi119.stdout:48 files removed 2024-04-26T17:57:44.981 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T17:57:44.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T17:57:45.651 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 205 kB/s | 84 kB 00:00 2024-04-26T17:57:45.652 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 176 kB/s | 84 kB 00:00 2024-04-26T17:57:45.670 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 202 kB/s | 84 kB 00:00 2024-04-26T17:57:45.970 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 43 kB/s | 12 kB 00:00 2024-04-26T17:57:46.079 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 32 kB/s | 12 kB 00:00 2024-04-26T17:57:46.151 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-26T17:57:46.313 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-04-26T17:57:46.497 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-26T17:57:46.660 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-26T17:57:46.963 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-26T17:57:48.040 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-26T17:57:48.402 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.0 MB 00:01 2024-04-26T17:57:48.940 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-26T17:57:52.267 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 6.0 MB/s | 19 MB 00:03 2024-04-26T17:57:52.670 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-04-26T17:57:54.483 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:01 2024-04-26T17:57:56.051 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-26T17:57:57.178 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-04-26T17:57:57.400 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.2 MB 00:01 2024-04-26T17:57:58.237 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:01 2024-04-26T17:57:59.068 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-26T17:57:59.621 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-26T17:58:00.288 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-04-26T17:58:01.223 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-26T17:58:02.129 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-26T17:58:04.255 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T17:58:04.256 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T17:58:04.267 INFO:teuthology.orchestra.run.smithi090.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T17:58:04.267 INFO:teuthology.orchestra.run.smithi090.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T17:58:04.268 INFO:teuthology.orchestra.run.smithi090.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T17:58:04.269 INFO:teuthology.orchestra.run.smithi090.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T17:58:04.271 INFO:teuthology.orchestra.run.smithi090.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T17:58:04.273 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T17:58:04.274 INFO:teuthology.orchestra.run.smithi090.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T17:58:04.334 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T17:58:04.343 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-26T17:58:04.343 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-26T17:58:04.343 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-26T17:58:04.344 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T17:58:04.344 INFO:teuthology.orchestra.run.smithi090.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T17:58:04.344 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T17:58:04.344 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T17:58:04.344 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T17:58:04.344 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T17:58:04.344 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T17:58:04.345 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T17:58:04.345 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T17:58:04.345 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T17:58:04.346 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T17:58:04.347 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T17:58:04.348 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T17:58:04.348 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T17:58:04.348 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T17:58:04.348 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T17:58:04.348 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T17:58:04.348 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T17:58:04.348 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T17:58:04.349 INFO:teuthology.orchestra.run.smithi090.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T17:58:04.350 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T17:58:04.350 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T17:58:04.350 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T17:58:04.350 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T17:58:04.350 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T17:58:04.350 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T17:58:04.351 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T17:58:04.352 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T17:58:04.352 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T17:58:04.352 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T17:58:04.352 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T17:58:04.352 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T17:58:04.352 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T17:58:04.353 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T17:58:04.354 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T17:58:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T17:58:04.356 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T17:58:04.357 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T17:58:04.358 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T17:58:04.359 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T17:58:04.360 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T17:58:04.361 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout:======================================================================================= 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout:Install 148 Packages 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 4 Packages 2024-04-26T17:58:04.362 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:58:04.363 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 222 M 2024-04-26T17:58:04.363 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T17:58:05.148 INFO:teuthology.orchestra.run.smithi090.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-26T17:58:05.386 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-26T17:58:05.414 INFO:teuthology.orchestra.run.smithi090.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 3.3 MB/s | 890 kB 00:00 2024-04-26T17:58:05.481 INFO:teuthology.orchestra.run.smithi090.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 2.1 MB/s | 145 kB 00:00 2024-04-26T17:58:05.824 INFO:teuthology.orchestra.run.smithi090.stdout:(4/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 6.9 MB/s | 2.4 MB 00:00 2024-04-26T17:58:05.907 INFO:teuthology.orchestra.run.smithi090.stdout:(5/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 5.8 MB/s | 5.4 MB 00:00 2024-04-26T17:58:06.049 INFO:teuthology.orchestra.run.smithi090.stdout:(6/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 6.7 MB/s | 1.5 MB 00:00 2024-04-26T17:58:06.248 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-26T17:58:06.349 INFO:teuthology.orchestra.run.smithi090.stdout:(7/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 11 MB/s | 4.7 MB 00:00 2024-04-26T17:58:07.308 INFO:teuthology.orchestra.run.smithi090.stdout:(8/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el9 11 MB/s | 11 MB 00:00 2024-04-26T17:58:07.367 INFO:teuthology.orchestra.run.smithi090.stdout:(9/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el9 433 kB/s | 25 kB 00:00 2024-04-26T17:58:07.512 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T17:58:07.513 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T17:58:07.524 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T17:58:07.525 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T17:58:07.525 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T17:58:07.526 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T17:58:07.528 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T17:58:07.530 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T17:58:07.531 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T17:58:07.591 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2024-04-26T17:58:07.616 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-26T17:58:07.616 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T17:58:07.617 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T17:58:07.618 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T17:58:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T17:58:07.620 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T17:58:07.621 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T17:58:07.622 INFO:teuthology.orchestra.run.smithi119.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T17:58:07.623 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T17:58:07.624 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T17:58:07.625 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T17:58:07.626 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T17:58:07.627 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T17:58:07.628 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2024-04-26T17:58:07.629 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2024-04-26T17:58:07.630 INFO:teuthology.orchestra.run.smithi119.stdout:Install 148 Packages 2024-04-26T17:58:07.630 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 4 Packages 2024-04-26T17:58:07.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:58:07.630 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 222 M 2024-04-26T17:58:07.630 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2024-04-26T17:58:07.801 INFO:teuthology.orchestra.run.smithi090.stdout:(10/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x8 9.6 MB/s | 17 MB 00:01 2024-04-26T17:58:07.867 INFO:teuthology.orchestra.run.smithi090.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 3.3 MB/s | 224 kB 00:00 2024-04-26T17:58:07.959 INFO:teuthology.orchestra.run.smithi090.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 294 kB/s | 27 kB 00:00 2024-04-26T17:58:08.051 INFO:teuthology.orchestra.run.smithi090.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 346 kB/s | 32 kB 00:00 2024-04-26T17:58:08.418 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T17:58:08.419 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T17:58:08.429 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T17:58:08.430 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T17:58:08.430 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T17:58:08.431 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T17:58:08.433 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T17:58:08.435 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T17:58:08.435 INFO:teuthology.orchestra.run.smithi090.stdout:(14/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 1.9 MB/s | 746 kB 00:00 2024-04-26T17:58:08.436 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T17:58:08.497 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-04-26T17:58:08.504 INFO:teuthology.orchestra.run.smithi119.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-04-26T17:58:08.516 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T17:58:08.517 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T17:58:08.518 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T17:58:08.519 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-04-26T17:58:08.520 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T17:58:08.521 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T17:58:08.522 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T17:58:08.523 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T17:58:08.524 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T17:58:08.525 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T17:58:08.526 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T17:58:08.527 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T17:58:08.528 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T17:58:08.529 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:58:08.530 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-04-26T17:58:08.530 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-04-26T17:58:08.530 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-04-26T17:58:08.530 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 4 Packages 2024-04-26T17:58:08.530 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:58:08.530 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 222 M 2024-04-26T17:58:08.531 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-04-26T17:58:08.562 INFO:teuthology.orchestra.run.smithi090.stdout:(15/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 1.3 MB/s | 163 kB 00:00 2024-04-26T17:58:08.703 INFO:teuthology.orchestra.run.smithi090.stdout:(16/152): librados-devel-19.0.0-3267.gbc94a2a9. 894 kB/s | 127 kB 00:00 2024-04-26T17:58:08.987 INFO:teuthology.orchestra.run.smithi090.stdout:(17/152): libradosstriper1-19.0.0-3267.gbc94a2a 1.7 MB/s | 501 kB 00:00 2024-04-26T17:58:09.879 INFO:teuthology.orchestra.run.smithi090.stdout:(18/152): ceph-common-19.0.0-3267.gbc94a2a9.el9 4.3 MB/s | 21 MB 00:04 2024-04-26T17:58:09.887 INFO:teuthology.orchestra.run.smithi119.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 643 kB/s | 890 kB 00:01 2024-04-26T17:58:09.907 INFO:teuthology.orchestra.run.smithi167.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-26T17:58:09.946 INFO:teuthology.orchestra.run.smithi090.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 680 kB/s | 45 kB 00:00 2024-04-26T17:58:10.063 INFO:teuthology.orchestra.run.smithi119.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 827 kB/s | 145 kB 00:00 2024-04-26T17:58:10.138 INFO:teuthology.orchestra.run.smithi119.stdout:(4/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 2.7 MB/s | 5.4 MB 00:01 2024-04-26T17:58:10.330 INFO:teuthology.orchestra.run.smithi119.stdout:(5/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 7.9 MB/s | 1.5 MB 00:00 2024-04-26T17:58:10.482 INFO:teuthology.orchestra.run.smithi167.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.5 MB/s | 890 kB 00:00 2024-04-26T17:58:10.505 INFO:teuthology.orchestra.run.smithi090.stdout:(20/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 16 MB/s | 49 MB 00:03 2024-04-26T17:58:10.530 INFO:teuthology.orchestra.run.smithi090.stdout:(21/152): python3-ceph-common-19.0.0-3267.gbc94 231 kB/s | 135 kB 00:00 2024-04-26T17:58:10.597 INFO:teuthology.orchestra.run.smithi090.stdout:(22/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 1.7 MB/s | 162 kB 00:00 2024-04-26T17:58:10.649 INFO:teuthology.orchestra.run.smithi167.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 868 kB/s | 145 kB 00:00 2024-04-26T17:58:10.714 INFO:teuthology.orchestra.run.smithi090.stdout:(23/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 3.1 MB/s | 5.4 MB 00:01 2024-04-26T17:58:10.748 INFO:teuthology.orchestra.run.smithi090.stdout:(24/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 1.9 MB/s | 298 kB 00:00 2024-04-26T17:58:10.798 INFO:teuthology.orchestra.run.smithi090.stdout:(25/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T17:58:10.823 INFO:teuthology.orchestra.run.smithi090.stdout:(26/152): python3-rados-19.0.0-3267.gbc94a2a9.e 1.1 MB/s | 320 kB 00:00 2024-04-26T17:58:10.847 INFO:teuthology.orchestra.run.smithi119.stdout:(6/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 9.1 MB/s | 4.7 MB 00:00 2024-04-26T17:58:10.849 INFO:teuthology.orchestra.run.smithi090.stdout:(27/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 842 kB/s | 85 kB 00:00 2024-04-26T17:58:10.891 INFO:teuthology.orchestra.run.smithi090.stdout:(28/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-26T17:58:10.942 INFO:teuthology.orchestra.run.smithi090.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3267.g 371 kB/s | 34 kB 00:00 2024-04-26T17:58:11.009 INFO:teuthology.orchestra.run.smithi090.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 1.2 MB/s | 142 kB 00:00 2024-04-26T17:58:11.389 INFO:teuthology.orchestra.run.smithi119.stdout:(7/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-26T17:58:11.618 INFO:teuthology.orchestra.run.smithi090.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 5.2 MB/s | 3.5 MB 00:00 2024-04-26T17:58:11.693 INFO:teuthology.orchestra.run.smithi090.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 3.2 MB/s | 249 kB 00:00 2024-04-26T17:58:11.768 INFO:teuthology.orchestra.run.smithi090.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 659 kB/s | 49 kB 00:00 2024-04-26T17:58:11.844 INFO:teuthology.orchestra.run.smithi090.stdout:(34/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T17:58:11.866 INFO:teuthology.orchestra.run.smithi167.stdout:(4/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-26T17:58:11.869 INFO:teuthology.orchestra.run.smithi090.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3267.gb 164 kB/s | 16 kB 00:00 2024-04-26T17:58:11.969 INFO:teuthology.orchestra.run.smithi090.stdout:(36/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 2.1 MB/s | 273 kB 00:00 2024-04-26T17:58:12.019 INFO:teuthology.orchestra.run.smithi090.stdout:(37/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-26T17:58:12.045 INFO:teuthology.orchestra.run.smithi090.stdout:(38/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 242 kB/s | 18 kB 00:00 2024-04-26T17:58:12.167 INFO:teuthology.orchestra.run.smithi167.stdout:(5/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 2.0 MB/s | 5.4 MB 00:02 2024-04-26T17:58:12.188 INFO:teuthology.orchestra.run.smithi090.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-04-26T17:58:12.274 INFO:teuthology.orchestra.run.smithi090.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-26T17:58:12.363 INFO:teuthology.orchestra.run.smithi090.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-04-26T17:58:12.497 INFO:teuthology.orchestra.run.smithi090.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-26T17:58:12.517 INFO:teuthology.orchestra.run.smithi167.stdout:(6/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 2.3 MB/s | 1.5 MB 00:00 2024-04-26T17:58:12.581 INFO:teuthology.orchestra.run.smithi090.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-26T17:58:12.664 INFO:teuthology.orchestra.run.smithi090.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-26T17:58:12.706 INFO:teuthology.orchestra.run.smithi090.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 3.6 MB/s | 1.2 MB 00:00 2024-04-26T17:58:12.765 INFO:teuthology.orchestra.run.smithi090.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-26T17:58:12.791 INFO:teuthology.orchestra.run.smithi090.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T17:58:12.999 INFO:teuthology.orchestra.run.smithi090.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.9 MB/s | 689 kB 00:00 2024-04-26T17:58:13.033 INFO:teuthology.orchestra.run.smithi090.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-26T17:58:13.142 INFO:teuthology.orchestra.run.smithi090.stdout:(50/152): ceph-mgr-diskprediction-local-19.0.0- 3.5 MB/s | 7.4 MB 00:02 2024-04-26T17:58:13.167 INFO:teuthology.orchestra.run.smithi090.stdout:(51/152): boost-program-options-1.75.0-8.el9.x8 639 kB/s | 107 kB 00:00 2024-04-26T17:58:13.192 INFO:teuthology.orchestra.run.smithi090.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 135 kB/s | 21 kB 00:00 2024-04-26T17:58:13.301 INFO:teuthology.orchestra.run.smithi090.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-04-26T17:58:13.326 INFO:teuthology.orchestra.run.smithi090.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 206 kB/s | 33 kB 00:00 2024-04-26T17:58:13.351 INFO:teuthology.orchestra.run.smithi090.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 361 kB/s | 18 kB 00:00 2024-04-26T17:58:13.364 INFO:teuthology.orchestra.run.smithi119.stdout:(8/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x86 6.7 MB/s | 17 MB 00:02 2024-04-26T17:58:13.377 INFO:teuthology.orchestra.run.smithi090.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-04-26T17:58:13.417 INFO:teuthology.orchestra.run.smithi167.stdout:(7/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 3.8 MB/s | 4.7 MB 00:01 2024-04-26T17:58:13.427 INFO:teuthology.orchestra.run.smithi090.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 906 kB/s | 45 kB 00:00 2024-04-26T17:58:13.452 INFO:teuthology.orchestra.run.smithi090.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-04-26T17:58:13.581 INFO:teuthology.orchestra.run.smithi119.stdout:(9/152): ceph-common-19.0.0-3267.gbc94a2a9.el9. 3.9 MB/s | 21 MB 00:05 2024-04-26T17:58:13.606 INFO:teuthology.orchestra.run.smithi119.stdout:(10/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el 104 kB/s | 25 kB 00:00 2024-04-26T17:58:13.644 INFO:teuthology.orchestra.run.smithi090.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-26T17:58:13.673 INFO:teuthology.orchestra.run.smithi119.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 3.3 MB/s | 224 kB 00:00 2024-04-26T17:58:13.703 INFO:teuthology.orchestra.run.smithi090.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-04-26T17:58:13.723 INFO:teuthology.orchestra.run.smithi119.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 541 kB/s | 27 kB 00:00 2024-04-26T17:58:13.728 INFO:teuthology.orchestra.run.smithi090.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-04-26T17:58:13.754 INFO:teuthology.orchestra.run.smithi090.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-04-26T17:58:13.773 INFO:teuthology.orchestra.run.smithi119.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 638 kB/s | 32 kB 00:00 2024-04-26T17:58:13.845 INFO:teuthology.orchestra.run.smithi090.stdout:(63/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T17:58:13.871 INFO:teuthology.orchestra.run.smithi090.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-26T17:58:13.898 INFO:teuthology.orchestra.run.smithi119.stdout:(14/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 5.8 MB/s | 746 kB 00:00 2024-04-26T17:58:13.931 INFO:teuthology.orchestra.run.smithi090.stdout:(65/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-04-26T17:58:13.949 INFO:teuthology.orchestra.run.smithi119.stdout:(15/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 3.2 MB/s | 163 kB 00:00 2024-04-26T17:58:13.956 INFO:teuthology.orchestra.run.smithi090.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 320 kB/s | 35 kB 00:00 2024-04-26T17:58:14.007 INFO:teuthology.orchestra.run.smithi119.stdout:(16/152): librados-devel-19.0.0-3267.gbc94a2a9. 2.1 MB/s | 127 kB 00:00 2024-04-26T17:58:14.108 INFO:teuthology.orchestra.run.smithi119.stdout:(17/152): libradosstriper1-19.0.0-3267.gbc94a2a 4.9 MB/s | 501 kB 00:00 2024-04-26T17:58:14.448 INFO:teuthology.orchestra.run.smithi090.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-04-26T17:58:14.740 INFO:teuthology.orchestra.run.smithi090.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2024-04-26T17:58:14.783 INFO:teuthology.orchestra.run.smithi119.stdout:(18/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 8.0 MB/s | 5.4 MB 00:00 2024-04-26T17:58:14.819 INFO:teuthology.orchestra.run.smithi090.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 2.4 MB/s | 911 kB 00:00 2024-04-26T17:58:14.825 INFO:teuthology.orchestra.run.smithi119.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 1.1 MB/s | 45 kB 00:00 2024-04-26T17:58:14.841 INFO:teuthology.orchestra.run.smithi090.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-26T17:58:14.875 INFO:teuthology.orchestra.run.smithi119.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 2.7 MB/s | 135 kB 00:00 2024-04-26T17:58:14.934 INFO:teuthology.orchestra.run.smithi119.stdout:(21/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 2.7 MB/s | 162 kB 00:00 2024-04-26T17:58:15.001 INFO:teuthology.orchestra.run.smithi119.stdout:(22/152): python3-rados-19.0.0-3267.gbc94a2a9.e 4.7 MB/s | 320 kB 00:00 2024-04-26T17:58:15.016 INFO:teuthology.orchestra.run.smithi090.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-26T17:58:15.068 INFO:teuthology.orchestra.run.smithi119.stdout:(23/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 4.3 MB/s | 298 kB 00:00 2024-04-26T17:58:15.118 INFO:teuthology.orchestra.run.smithi119.stdout:(24/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 1.9 MB/s | 99 kB 00:00 2024-04-26T17:58:15.125 INFO:teuthology.orchestra.run.smithi090.stdout:(72/152): openblas-openmp-0.3.21-2.el9.x86_64.r 3.8 MB/s | 4.7 MB 00:01 2024-04-26T17:58:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 655 kB/s | 87 kB 00:00 2024-04-26T17:58:15.178 INFO:teuthology.orchestra.run.smithi119.stdout:(25/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T17:58:15.233 INFO:teuthology.orchestra.run.smithi090.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-26T17:58:15.317 INFO:teuthology.orchestra.run.smithi090.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-26T17:58:15.376 INFO:teuthology.orchestra.run.smithi090.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-04-26T17:58:15.435 INFO:teuthology.orchestra.run.smithi090.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 328 kB/s | 19 kB 00:00 2024-04-26T17:58:15.493 INFO:teuthology.orchestra.run.smithi090.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-26T17:58:15.544 INFO:teuthology.orchestra.run.smithi090.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-04-26T17:58:15.595 INFO:teuthology.orchestra.run.smithi119.stdout:(26/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 7.4 MB/s | 3.1 MB 00:00 2024-04-26T17:58:15.642 INFO:teuthology.orchestra.run.smithi167.stdout:(8/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el9 4.8 MB/s | 11 MB 00:02 2024-04-26T17:58:15.645 INFO:teuthology.orchestra.run.smithi119.stdout:(27/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 3.3 MB/s | 170 kB 00:00 2024-04-26T17:58:15.695 INFO:teuthology.orchestra.run.smithi119.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3267.g 683 kB/s | 34 kB 00:00 2024-04-26T17:58:15.701 INFO:teuthology.orchestra.run.smithi167.stdout:(9/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el9 432 kB/s | 25 kB 00:00 2024-04-26T17:58:15.754 INFO:teuthology.orchestra.run.smithi119.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 2.4 MB/s | 142 kB 00:00 2024-04-26T17:58:16.194 INFO:teuthology.orchestra.run.smithi090.stdout:(80/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.8 MB/s | 5.1 MB 00:00 2024-04-26T17:58:16.229 INFO:teuthology.orchestra.run.smithi119.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 7.5 MB/s | 3.5 MB 00:00 2024-04-26T17:58:16.278 INFO:teuthology.orchestra.run.smithi090.stdout:(81/152): postgresql-server-13.14-1.el9.x86_64. 4.0 MB/s | 5.8 MB 00:01 2024-04-26T17:58:16.303 INFO:teuthology.orchestra.run.smithi090.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-26T17:58:16.353 INFO:teuthology.orchestra.run.smithi090.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-26T17:58:16.371 INFO:teuthology.orchestra.run.smithi119.stdout:(31/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el 2.1 MB/s | 11 MB 00:04 2024-04-26T17:58:16.376 INFO:teuthology.orchestra.run.smithi167.stdout:(10/152): ceph-common-19.0.0-3267.gbc94a2a9.el9 3.1 MB/s | 21 MB 00:06 2024-04-26T17:58:16.378 INFO:teuthology.orchestra.run.smithi090.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-26T17:58:16.404 INFO:teuthology.orchestra.run.smithi090.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-04-26T17:58:16.462 INFO:teuthology.orchestra.run.smithi090.stdout:(86/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-26T17:58:16.496 INFO:teuthology.orchestra.run.smithi090.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-26T17:58:16.518 INFO:teuthology.orchestra.run.smithi167.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 1.6 MB/s | 224 kB 00:00 2024-04-26T17:58:16.521 INFO:teuthology.orchestra.run.smithi119.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 1.6 MB/s | 249 kB 00:00 2024-04-26T17:58:16.530 INFO:teuthology.orchestra.run.smithi090.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-26T17:58:16.555 INFO:teuthology.orchestra.run.smithi090.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-04-26T17:58:16.576 INFO:teuthology.orchestra.run.smithi167.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 462 kB/s | 27 kB 00:00 2024-04-26T17:58:16.582 INFO:teuthology.orchestra.run.smithi090.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-26T17:58:16.588 INFO:teuthology.orchestra.run.smithi119.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 741 kB/s | 49 kB 00:00 2024-04-26T17:58:16.643 INFO:teuthology.orchestra.run.smithi167.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 477 kB/s | 32 kB 00:00 2024-04-26T17:58:16.664 INFO:teuthology.orchestra.run.smithi119.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3267.gb 220 kB/s | 16 kB 00:00 2024-04-26T17:58:16.674 INFO:teuthology.orchestra.run.smithi090.stdout:(91/152): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-04-26T17:58:16.699 INFO:teuthology.orchestra.run.smithi090.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 356 kB/s | 42 kB 00:00 2024-04-26T17:58:16.724 INFO:teuthology.orchestra.run.smithi090.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-04-26T17:58:16.758 INFO:teuthology.orchestra.run.smithi090.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T17:58:16.783 INFO:teuthology.orchestra.run.smithi090.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 611 kB/s | 36 kB 00:00 2024-04-26T17:58:16.797 INFO:teuthology.orchestra.run.smithi119.stdout:(35/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 2.0 MB/s | 273 kB 00:00 2024-04-26T17:58:16.817 INFO:teuthology.orchestra.run.smithi090.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-26T17:58:16.876 INFO:teuthology.orchestra.run.smithi090.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-04-26T17:58:16.918 INFO:teuthology.orchestra.run.smithi090.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 691 kB/s | 58 kB 00:00 2024-04-26T17:58:16.935 INFO:teuthology.orchestra.run.smithi090.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T17:58:16.960 INFO:teuthology.orchestra.run.smithi090.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T17:58:16.977 INFO:teuthology.orchestra.run.smithi167.stdout:(14/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x8 3.8 MB/s | 17 MB 00:04 2024-04-26T17:58:17.003 INFO:teuthology.orchestra.run.smithi167.stdout:(15/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 2.0 MB/s | 746 kB 00:00 2024-04-26T17:58:17.019 INFO:teuthology.orchestra.run.smithi090.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 384 kB/s | 22 kB 00:00 2024-04-26T17:58:17.036 INFO:teuthology.orchestra.run.smithi167.stdout:(16/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 2.7 MB/s | 163 kB 00:00 2024-04-26T17:58:17.064 INFO:teuthology.orchestra.run.smithi119.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 8.9 MB/s | 7.4 MB 00:00 2024-04-26T17:58:17.070 INFO:teuthology.orchestra.run.smithi167.stdout:(17/152): librados-devel-19.0.0-3267.gbc94a2a9. 1.8 MB/s | 127 kB 00:00 2024-04-26T17:58:17.094 INFO:teuthology.orchestra.run.smithi090.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 281 kB/s | 45 kB 00:00 2024-04-26T17:58:17.097 INFO:teuthology.orchestra.run.smithi119.stdout:(37/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 2.5 MB/s | 762 kB 00:00 2024-04-26T17:58:17.123 INFO:teuthology.orchestra.run.smithi119.stdout:(38/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 313 kB/s | 18 kB 00:00 2024-04-26T17:58:17.128 INFO:teuthology.orchestra.run.smithi090.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-04-26T17:58:17.178 INFO:teuthology.orchestra.run.smithi167.stdout:(18/152): libradosstriper1-19.0.0-3267.gbc94a2a 3.5 MB/s | 501 kB 00:00 2024-04-26T17:58:17.254 INFO:teuthology.orchestra.run.smithi167.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 601 kB/s | 45 kB 00:00 2024-04-26T17:58:17.281 INFO:teuthology.orchestra.run.smithi119.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 438 kB/s | 80 kB 00:00 2024-04-26T17:58:17.329 INFO:teuthology.orchestra.run.smithi167.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 1.8 MB/s | 135 kB 00:00 2024-04-26T17:58:17.390 INFO:teuthology.orchestra.run.smithi119.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-26T17:58:17.413 INFO:teuthology.orchestra.run.smithi167.stdout:(21/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 1.9 MB/s | 162 kB 00:00 2024-04-26T17:58:17.555 INFO:teuthology.orchestra.run.smithi167.stdout:(22/152): python3-rados-19.0.0-3267.gbc94a2a9.e 2.2 MB/s | 320 kB 00:00 2024-04-26T17:58:17.574 INFO:teuthology.orchestra.run.smithi119.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.7 MB/s | 803 kB 00:00 2024-04-26T17:58:17.599 INFO:teuthology.orchestra.run.smithi119.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-04-26T17:58:17.674 INFO:teuthology.orchestra.run.smithi119.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-26T17:58:17.705 INFO:teuthology.orchestra.run.smithi167.stdout:(23/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 1.9 MB/s | 298 kB 00:00 2024-04-26T17:58:17.790 INFO:teuthology.orchestra.run.smithi167.stdout:(24/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T17:58:17.791 INFO:teuthology.orchestra.run.smithi119.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-26T17:58:17.850 INFO:teuthology.orchestra.run.smithi119.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T17:58:17.855 INFO:teuthology.orchestra.run.smithi167.stdout:(25/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-26T17:58:17.942 INFO:teuthology.orchestra.run.smithi119.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-26T17:58:17.962 INFO:teuthology.orchestra.run.smithi090.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 370 kB/s | 309 kB 00:00 2024-04-26T17:58:17.995 INFO:teuthology.orchestra.run.smithi090.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 608 kB/s | 548 kB 00:00 2024-04-26T17:58:18.026 INFO:teuthology.orchestra.run.smithi119.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 2.7 MB/s | 1.2 MB 00:00 2024-04-26T17:58:18.068 INFO:teuthology.orchestra.run.smithi119.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-26T17:58:18.071 INFO:teuthology.orchestra.run.smithi090.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 179 kB/s | 19 kB 00:00 2024-04-26T17:58:18.135 INFO:teuthology.orchestra.run.smithi119.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T17:58:18.160 INFO:teuthology.orchestra.run.smithi119.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 6.9 MB/s | 943 kB 00:00 2024-04-26T17:58:18.179 INFO:teuthology.orchestra.run.smithi090.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-26T17:58:18.202 INFO:teuthology.orchestra.run.smithi119.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 320 kB/s | 21 kB 00:00 2024-04-26T17:58:18.252 INFO:teuthology.orchestra.run.smithi119.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-26T17:58:18.273 INFO:teuthology.orchestra.run.smithi167.stdout:(26/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 4.5 MB/s | 5.4 MB 00:01 2024-04-26T17:58:18.278 INFO:teuthology.orchestra.run.smithi119.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-04-26T17:58:18.288 INFO:teuthology.orchestra.run.smithi090.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-26T17:58:18.328 INFO:teuthology.orchestra.run.smithi119.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-04-26T17:58:18.339 INFO:teuthology.orchestra.run.smithi167.stdout:(27/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 2.5 MB/s | 170 kB 00:00 2024-04-26T17:58:18.370 INFO:teuthology.orchestra.run.smithi119.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-04-26T17:58:18.406 INFO:teuthology.orchestra.run.smithi167.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3267.g 514 kB/s | 34 kB 00:00 2024-04-26T17:58:18.447 INFO:teuthology.orchestra.run.smithi090.stdout:(109/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-04-26T17:58:18.454 INFO:teuthology.orchestra.run.smithi119.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-04-26T17:58:18.472 INFO:teuthology.orchestra.run.smithi090.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-26T17:58:18.473 INFO:teuthology.orchestra.run.smithi167.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 2.1 MB/s | 142 kB 00:00 2024-04-26T17:58:18.479 INFO:teuthology.orchestra.run.smithi119.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-26T17:58:18.529 INFO:teuthology.orchestra.run.smithi119.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-04-26T17:58:18.532 INFO:teuthology.orchestra.run.smithi167.stdout:(30/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 4.6 MB/s | 3.1 MB 00:00 2024-04-26T17:58:18.556 INFO:teuthology.orchestra.run.smithi090.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-26T17:58:18.588 INFO:teuthology.orchestra.run.smithi119.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-26T17:58:18.638 INFO:teuthology.orchestra.run.smithi119.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-26T17:58:18.663 INFO:teuthology.orchestra.run.smithi119.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 338 kB/s | 25 kB 00:00 2024-04-26T17:58:18.681 INFO:teuthology.orchestra.run.smithi090.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-26T17:58:18.714 INFO:teuthology.orchestra.run.smithi119.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 630 kB/s | 47 kB 00:00 2024-04-26T17:58:18.756 INFO:teuthology.orchestra.run.smithi090.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.9 MB/s | 839 kB 00:00 2024-04-26T17:58:18.764 INFO:teuthology.orchestra.run.smithi119.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-26T17:58:18.791 INFO:teuthology.orchestra.run.smithi090.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 395 kB/s | 43 kB 00:00 2024-04-26T17:58:18.806 INFO:teuthology.orchestra.run.smithi119.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T17:58:18.831 INFO:teuthology.orchestra.run.smithi119.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-26T17:58:18.883 INFO:teuthology.orchestra.run.smithi090.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 254 kB/s | 32 kB 00:00 2024-04-26T17:58:18.909 INFO:teuthology.orchestra.run.smithi090.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-04-26T17:58:19.009 INFO:teuthology.orchestra.run.smithi090.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-26T17:58:19.043 INFO:teuthology.orchestra.run.smithi090.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-26T17:58:19.057 INFO:teuthology.orchestra.run.smithi119.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-26T17:58:19.115 INFO:teuthology.orchestra.run.smithi119.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-26T17:58:19.135 INFO:teuthology.orchestra.run.smithi090.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-26T17:58:19.160 INFO:teuthology.orchestra.run.smithi090.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-26T17:58:19.191 INFO:teuthology.orchestra.run.smithi167.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 4.9 MB/s | 3.5 MB 00:00 2024-04-26T17:58:19.217 INFO:teuthology.orchestra.run.smithi119.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-26T17:58:19.249 INFO:teuthology.orchestra.run.smithi119.stdout:(69/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-26T17:58:19.252 INFO:teuthology.orchestra.run.smithi090.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-26T17:58:19.299 INFO:teuthology.orchestra.run.smithi167.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 2.3 MB/s | 249 kB 00:00 2024-04-26T17:58:19.308 INFO:teuthology.orchestra.run.smithi119.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-26T17:58:19.319 INFO:teuthology.orchestra.run.smithi090.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-26T17:58:19.366 INFO:teuthology.orchestra.run.smithi167.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 740 kB/s | 49 kB 00:00 2024-04-26T17:58:19.369 INFO:teuthology.orchestra.run.smithi090.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-26T17:58:19.433 INFO:teuthology.orchestra.run.smithi119.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-04-26T17:58:19.445 INFO:teuthology.orchestra.run.smithi090.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-26T17:58:19.466 INFO:teuthology.orchestra.run.smithi167.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3267.gb 164 kB/s | 16 kB 00:00 2024-04-26T17:58:19.495 INFO:teuthology.orchestra.run.smithi090.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-26T17:58:19.517 INFO:teuthology.orchestra.run.smithi119.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-26T17:58:19.550 INFO:teuthology.orchestra.run.smithi119.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 748 kB/s | 87 kB 00:00 2024-04-26T17:58:19.579 INFO:teuthology.orchestra.run.smithi090.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-04-26T17:58:19.642 INFO:teuthology.orchestra.run.smithi119.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-26T17:58:19.645 INFO:teuthology.orchestra.run.smithi090.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 129 kB/s | 19 kB 00:00 2024-04-26T17:58:19.650 INFO:teuthology.orchestra.run.smithi167.stdout:(35/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 1.5 MB/s | 273 kB 00:00 2024-04-26T17:58:19.704 INFO:teuthology.orchestra.run.smithi090.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 157 kB/s | 20 kB 00:00 2024-04-26T17:58:19.726 INFO:teuthology.orchestra.run.smithi119.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-26T17:58:19.818 INFO:teuthology.orchestra.run.smithi119.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 6.0 MB 00:00 2024-04-26T17:58:19.821 INFO:teuthology.orchestra.run.smithi090.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-04-26T17:58:19.835 INFO:teuthology.orchestra.run.smithi119.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-04-26T17:58:19.855 INFO:teuthology.orchestra.run.smithi090.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-26T17:58:19.877 INFO:teuthology.orchestra.run.smithi119.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-04-26T17:58:19.892 INFO:teuthology.orchestra.run.smithi167.stdout:(36/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 3.1 MB/s | 762 kB 00:00 2024-04-26T17:58:19.927 INFO:teuthology.orchestra.run.smithi119.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T17:58:19.939 INFO:teuthology.orchestra.run.smithi090.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 675 kB/s | 79 kB 00:00 2024-04-26T17:58:19.953 INFO:teuthology.orchestra.run.smithi119.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-04-26T17:58:19.976 INFO:teuthology.orchestra.run.smithi167.stdout:(37/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 219 kB/s | 18 kB 00:00 2024-04-26T17:58:19.981 INFO:teuthology.orchestra.run.smithi090.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 460 kB/s | 58 kB 00:00 2024-04-26T17:58:20.028 INFO:teuthology.orchestra.run.smithi119.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T17:58:20.056 INFO:teuthology.orchestra.run.smithi090.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-26T17:58:20.081 INFO:teuthology.orchestra.run.smithi090.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-26T17:58:20.112 INFO:teuthology.orchestra.run.smithi119.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-26T17:58:20.165 INFO:teuthology.orchestra.run.smithi090.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 828 kB/s | 90 kB 00:00 2024-04-26T17:58:20.176 INFO:teuthology.orchestra.run.smithi167.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 4.5 MB/s | 7.4 MB 00:01 2024-04-26T17:58:20.187 INFO:teuthology.orchestra.run.smithi119.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-26T17:58:20.190 INFO:teuthology.orchestra.run.smithi090.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-26T17:58:20.201 INFO:teuthology.orchestra.run.smithi167.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 357 kB/s | 80 kB 00:00 2024-04-26T17:58:20.254 INFO:teuthology.orchestra.run.smithi119.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-26T17:58:20.266 INFO:teuthology.orchestra.run.smithi090.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 307 kB/s | 31 kB 00:00 2024-04-26T17:58:20.302 INFO:teuthology.orchestra.run.smithi167.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T17:58:20.304 INFO:teuthology.orchestra.run.smithi119.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-04-26T17:58:20.309 INFO:teuthology.orchestra.run.smithi090.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-26T17:58:20.354 INFO:teuthology.orchestra.run.smithi119.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-26T17:58:20.376 INFO:teuthology.orchestra.run.smithi090.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 537 kB/s | 59 kB 00:00 2024-04-26T17:58:20.397 INFO:teuthology.orchestra.run.smithi119.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-26T17:58:20.410 INFO:teuthology.orchestra.run.smithi167.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-26T17:58:20.418 INFO:teuthology.orchestra.run.smithi090.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-04-26T17:58:20.455 INFO:teuthology.orchestra.run.smithi119.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-26T17:58:20.481 INFO:teuthology.orchestra.run.smithi119.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 611 kB/s | 51 kB 00:00 2024-04-26T17:58:20.485 INFO:teuthology.orchestra.run.smithi090.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 469 kB/s | 51 kB 00:00 2024-04-26T17:58:20.531 INFO:teuthology.orchestra.run.smithi119.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-04-26T17:58:20.544 INFO:teuthology.orchestra.run.smithi090.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-26T17:58:20.589 INFO:teuthology.orchestra.run.smithi119.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 718 kB/s | 42 kB 00:00 2024-04-26T17:58:20.594 INFO:teuthology.orchestra.run.smithi167.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 803 kB 00:00 2024-04-26T17:58:20.602 INFO:teuthology.orchestra.run.smithi090.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 767 kB/s | 90 kB 00:00 2024-04-26T17:58:20.686 INFO:teuthology.orchestra.run.smithi090.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-04-26T17:58:20.715 INFO:teuthology.orchestra.run.smithi119.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 353 kB/s | 44 kB 00:00 2024-04-26T17:58:20.719 INFO:teuthology.orchestra.run.smithi090.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-26T17:58:20.753 INFO:teuthology.orchestra.run.smithi167.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 673 kB/s | 106 kB 00:00 2024-04-26T17:58:20.828 INFO:teuthology.orchestra.run.smithi167.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-04-26T17:58:20.855 INFO:teuthology.orchestra.run.smithi167.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-26T17:58:20.879 INFO:teuthology.orchestra.run.smithi167.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T17:58:20.945 INFO:teuthology.orchestra.run.smithi090.stdout:(146/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.7 MB/s | 16 MB 00:04 2024-04-26T17:58:20.962 INFO:teuthology.orchestra.run.smithi167.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-26T17:58:20.979 INFO:teuthology.orchestra.run.smithi090.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 655 kB/s | 192 kB 00:00 2024-04-26T17:58:20.996 INFO:teuthology.orchestra.run.smithi167.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.8 MB/s | 689 kB 00:00 2024-04-26T17:58:21.029 INFO:teuthology.orchestra.run.smithi090.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-04-26T17:58:21.063 INFO:teuthology.orchestra.run.smithi167.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T17:58:21.105 INFO:teuthology.orchestra.run.smithi167.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 6.5 MB/s | 943 kB 00:00 2024-04-26T17:58:21.130 INFO:teuthology.orchestra.run.smithi167.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 321 kB/s | 21 kB 00:00 2024-04-26T17:58:21.149 INFO:teuthology.orchestra.run.smithi119.stdout:(93/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 6.5 MB/s | 49 MB 00:07 2024-04-26T17:58:21.172 INFO:teuthology.orchestra.run.smithi167.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 790 kB/s | 33 kB 00:00 2024-04-26T17:58:21.207 INFO:teuthology.orchestra.run.smithi119.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 619 kB/s | 36 kB 00:00 2024-04-26T17:58:21.232 INFO:teuthology.orchestra.run.smithi119.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 426 kB/s | 221 kB 00:00 2024-04-26T17:58:21.281 INFO:teuthology.orchestra.run.smithi167.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-04-26T17:58:21.291 INFO:teuthology.orchestra.run.smithi119.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-26T17:58:21.331 INFO:teuthology.orchestra.run.smithi167.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-26T17:58:21.346 INFO:teuthology.orchestra.run.smithi090.stdout:(149/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 8.6 MB/s | 3.1 MB 00:00 2024-04-26T17:58:21.350 INFO:teuthology.orchestra.run.smithi119.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.5 MB/s | 305 kB 00:00 2024-04-26T17:58:21.375 INFO:teuthology.orchestra.run.smithi119.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 692 kB/s | 58 kB 00:00 2024-04-26T17:58:21.381 INFO:teuthology.orchestra.run.smithi167.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-26T17:58:21.406 INFO:teuthology.orchestra.run.smithi167.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-04-26T17:58:21.442 INFO:teuthology.orchestra.run.smithi119.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-04-26T17:58:21.448 INFO:teuthology.orchestra.run.smithi167.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-26T17:58:21.455 INFO:teuthology.orchestra.run.smithi090.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.2 MB/s | 1.3 MB 00:00 2024-04-26T17:58:21.467 INFO:teuthology.orchestra.run.smithi119.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-04-26T17:58:21.474 INFO:teuthology.orchestra.run.smithi167.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 675 kB/s | 45 kB 00:00 2024-04-26T17:58:21.558 INFO:teuthology.orchestra.run.smithi167.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-26T17:58:21.580 INFO:teuthology.orchestra.run.smithi090.stdout:(151/152): librados2-19.0.0-3267.gbc94a2a9.el9. 5.4 MB/s | 3.4 MB 00:00 2024-04-26T17:58:21.583 INFO:teuthology.orchestra.run.smithi167.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-26T17:58:21.625 INFO:teuthology.orchestra.run.smithi167.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-04-26T17:58:21.650 INFO:teuthology.orchestra.run.smithi167.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-04-26T17:58:21.659 INFO:teuthology.orchestra.run.smithi119.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-26T17:58:21.684 INFO:teuthology.orchestra.run.smithi119.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 185 kB/s | 45 kB 00:00 2024-04-26T17:58:21.710 INFO:teuthology.orchestra.run.smithi119.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-04-26T17:58:21.759 INFO:teuthology.orchestra.run.smithi167.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-26T17:58:21.784 INFO:teuthology.orchestra.run.smithi167.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-26T17:58:21.834 INFO:teuthology.orchestra.run.smithi167.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 468 kB/s | 35 kB 00:00 2024-04-26T17:58:21.851 INFO:teuthology.orchestra.run.smithi119.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.4 MB/s | 863 kB 00:00 2024-04-26T17:58:21.952 INFO:teuthology.orchestra.run.smithi119.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-04-26T17:58:21.977 INFO:teuthology.orchestra.run.smithi119.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-04-26T17:58:22.003 INFO:teuthology.orchestra.run.smithi119.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-04-26T17:58:22.060 INFO:teuthology.orchestra.run.smithi167.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-04-26T17:58:22.172 INFO:teuthology.orchestra.run.smithi090.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.9 MB/s | 2.4 MB 00:00 2024-04-26T17:58:22.177 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T17:58:22.178 INFO:teuthology.orchestra.run.smithi090.stdout:Total 12 MB/s | 222 MB 00:17 2024-04-26T17:58:22.219 INFO:teuthology.orchestra.run.smithi167.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-26T17:58:22.294 INFO:teuthology.orchestra.run.smithi167.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-04-26T17:58:22.320 INFO:teuthology.orchestra.run.smithi119.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 76 kB/s | 26 kB 00:00 2024-04-26T17:58:22.345 INFO:teuthology.orchestra.run.smithi119.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-26T17:58:22.361 INFO:teuthology.orchestra.run.smithi167.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T17:58:22.394 INFO:teuthology.orchestra.run.smithi167.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-04-26T17:58:22.412 INFO:teuthology.orchestra.run.smithi119.stdout:(110/152): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-26T17:58:22.437 INFO:teuthology.orchestra.run.smithi119.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 575 kB/s | 67 kB 00:00 2024-04-26T17:58:22.462 INFO:teuthology.orchestra.run.smithi119.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-26T17:58:22.503 INFO:teuthology.orchestra.run.smithi167.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-26T17:58:22.546 INFO:teuthology.orchestra.run.smithi119.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 839 kB 00:00 2024-04-26T17:58:22.562 INFO:teuthology.orchestra.run.smithi167.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T17:58:22.654 INFO:teuthology.orchestra.run.smithi119.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.5 MB/s | 548 kB 00:00 2024-04-26T17:58:22.680 INFO:teuthology.orchestra.run.smithi119.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 200 kB/s | 43 kB 00:00 2024-04-26T17:58:22.688 INFO:teuthology.orchestra.run.smithi167.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-04-26T17:58:22.705 INFO:teuthology.orchestra.run.smithi119.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 203 kB/s | 32 kB 00:00 2024-04-26T17:58:22.730 INFO:teuthology.orchestra.run.smithi119.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-04-26T17:58:22.770 INFO:teuthology.orchestra.run.smithi167.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-26T17:58:22.781 INFO:teuthology.orchestra.run.smithi119.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-26T17:58:22.806 INFO:teuthology.orchestra.run.smithi119.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-04-26T17:58:22.831 INFO:teuthology.orchestra.run.smithi119.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.9 MB/s | 303 kB 00:00 2024-04-26T17:58:22.846 INFO:teuthology.orchestra.run.smithi167.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-26T17:58:22.856 INFO:teuthology.orchestra.run.smithi119.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 476 kB/s | 35 kB 00:00 2024-04-26T17:58:22.890 INFO:teuthology.orchestra.run.smithi119.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-26T17:58:22.938 INFO:teuthology.orchestra.run.smithi167.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-04-26T17:58:22.963 INFO:teuthology.orchestra.run.smithi167.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-04-26T17:58:23.005 INFO:teuthology.orchestra.run.smithi167.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-04-26T17:58:23.055 INFO:teuthology.orchestra.run.smithi167.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T17:58:23.080 INFO:teuthology.orchestra.run.smithi167.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-04-26T17:58:23.107 INFO:teuthology.orchestra.run.smithi119.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 7.4 MB/s | 2.0 MB 00:00 2024-04-26T17:58:23.132 INFO:teuthology.orchestra.run.smithi119.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 525 kB/s | 144 kB 00:00 2024-04-26T17:58:23.149 INFO:teuthology.orchestra.run.smithi119.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 41 kB/s | 11 kB 00:00 2024-04-26T17:58:23.155 INFO:teuthology.orchestra.run.smithi167.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T17:58:23.166 INFO:teuthology.orchestra.run.smithi119.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-04-26T17:58:23.191 INFO:teuthology.orchestra.run.smithi119.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 556 kB/s | 23 kB 00:00 2024-04-26T17:58:23.208 INFO:teuthology.orchestra.run.smithi119.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-04-26T17:58:23.225 INFO:teuthology.orchestra.run.smithi119.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-26T17:58:23.231 INFO:teuthology.orchestra.run.smithi167.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-04-26T17:58:23.300 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T17:58:23.306 INFO:teuthology.orchestra.run.smithi167.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-26T17:58:23.323 INFO:teuthology.orchestra.run.smithi167.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 839 kB/s | 77 kB 00:00 2024-04-26T17:58:23.373 INFO:teuthology.orchestra.run.smithi167.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-04-26T17:58:23.392 INFO:teuthology.orchestra.run.smithi119.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-04-26T17:58:23.409 INFO:teuthology.orchestra.run.smithi119.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2024-04-26T17:58:23.424 INFO:teuthology.orchestra.run.smithi167.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-26T17:58:23.434 INFO:teuthology.orchestra.run.smithi119.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 378 kB/s | 79 kB 00:00 2024-04-26T17:58:23.445 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T17:58:23.446 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T17:58:23.451 INFO:teuthology.orchestra.run.smithi119.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 990 kB/s | 58 kB 00:00 2024-04-26T17:58:23.466 INFO:teuthology.orchestra.run.smithi167.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-26T17:58:23.516 INFO:teuthology.orchestra.run.smithi167.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-26T17:58:23.518 INFO:teuthology.orchestra.run.smithi119.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T17:58:23.542 INFO:teuthology.orchestra.run.smithi167.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-04-26T17:58:23.543 INFO:teuthology.orchestra.run.smithi119.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-26T17:58:23.568 INFO:teuthology.orchestra.run.smithi119.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 767 kB/s | 90 kB 00:00 2024-04-26T17:58:23.592 INFO:teuthology.orchestra.run.smithi167.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 709 kB/s | 54 kB 00:00 2024-04-26T17:58:23.594 INFO:teuthology.orchestra.run.smithi119.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-26T17:58:23.627 INFO:teuthology.orchestra.run.smithi119.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-26T17:58:23.642 INFO:teuthology.orchestra.run.smithi167.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-04-26T17:58:23.661 INFO:teuthology.orchestra.run.smithi119.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-04-26T17:58:23.686 INFO:teuthology.orchestra.run.smithi119.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 640 kB/s | 59 kB 00:00 2024-04-26T17:58:23.701 INFO:teuthology.orchestra.run.smithi167.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-26T17:58:23.711 INFO:teuthology.orchestra.run.smithi119.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 429 kB/s | 36 kB 00:00 2024-04-26T17:58:23.737 INFO:teuthology.orchestra.run.smithi119.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 680 kB/s | 51 kB 00:00 2024-04-26T17:58:23.776 INFO:teuthology.orchestra.run.smithi167.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-26T17:58:23.779 INFO:teuthology.orchestra.run.smithi119.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-26T17:58:23.804 INFO:teuthology.orchestra.run.smithi119.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 976 kB/s | 90 kB 00:00 2024-04-26T17:58:23.826 INFO:teuthology.orchestra.run.smithi167.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 716 kB/s | 36 kB 00:00 2024-04-26T17:58:23.854 INFO:teuthology.orchestra.run.smithi119.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-26T17:58:23.880 INFO:teuthology.orchestra.run.smithi119.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-26T17:58:23.893 INFO:teuthology.orchestra.run.smithi167.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 713 kB/s | 47 kB 00:00 2024-04-26T17:58:23.913 INFO:teuthology.orchestra.run.smithi119.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-26T17:58:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T17:58:24.077 INFO:teuthology.orchestra.run.smithi167.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-26T17:58:24.239 INFO:teuthology.orchestra.run.smithi119.stdout:(149/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 9.7 MB/s | 3.1 MB 00:00 2024-04-26T17:58:24.364 INFO:teuthology.orchestra.run.smithi119.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 1.3 MB 00:00 2024-04-26T17:58:24.511 INFO:teuthology.orchestra.run.smithi167.stdout:(97/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 5.6 MB/s | 49 MB 00:08 2024-04-26T17:58:24.561 INFO:teuthology.orchestra.run.smithi167.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 457 kB/s | 305 kB 00:00 2024-04-26T17:58:24.587 INFO:teuthology.orchestra.run.smithi167.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 114 kB/s | 58 kB 00:00 2024-04-26T17:58:24.612 INFO:teuthology.orchestra.run.smithi167.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 937 kB/s | 94 kB 00:00 2024-04-26T17:58:24.637 INFO:teuthology.orchestra.run.smithi167.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-04-26T17:58:24.654 INFO:teuthology.orchestra.run.smithi167.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-04-26T17:58:24.671 INFO:teuthology.orchestra.run.smithi167.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 379 kB/s | 22 kB 00:00 2024-04-26T17:58:24.672 INFO:teuthology.orchestra.run.smithi119.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 2.4 MB 00:00 2024-04-26T17:58:24.898 INFO:teuthology.orchestra.run.smithi119.stdout:(152/152): librados2-19.0.0-3267.gbc94a2a9.el9. 3.3 MB/s | 3.4 MB 00:01 2024-04-26T17:58:24.903 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2024-04-26T17:58:24.903 INFO:teuthology.orchestra.run.smithi119.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-26T17:58:24.904 INFO:teuthology.orchestra.run.smithi167.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-26T17:58:24.938 INFO:teuthology.orchestra.run.smithi167.stdout:(105/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-04-26T17:58:24.963 INFO:teuthology.orchestra.run.smithi167.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-04-26T17:58:24.997 INFO:teuthology.orchestra.run.smithi167.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.6 MB/s | 548 kB 00:00 2024-04-26T17:58:25.022 INFO:teuthology.orchestra.run.smithi167.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-04-26T17:58:25.056 INFO:teuthology.orchestra.run.smithi167.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 829 kB/s | 49 kB 00:00 2024-04-26T17:58:25.106 INFO:teuthology.orchestra.run.smithi167.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-04-26T17:58:25.131 INFO:teuthology.orchestra.run.smithi167.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-26T17:58:25.200 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T17:58:25.200 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T17:58:25.215 INFO:teuthology.orchestra.run.smithi167.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-26T17:58:25.240 INFO:teuthology.orchestra.run.smithi167.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-04-26T17:58:25.257 INFO:teuthology.orchestra.run.smithi167.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-26T17:58:25.282 INFO:teuthology.orchestra.run.smithi167.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 766 kB/s | 32 kB 00:00 2024-04-26T17:58:25.299 INFO:teuthology.orchestra.run.smithi167.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 356 kB/s | 15 kB 00:00 2024-04-26T17:58:25.350 INFO:teuthology.orchestra.run.smithi167.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-26T17:58:25.375 INFO:teuthology.orchestra.run.smithi167.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-26T17:58:25.459 INFO:teuthology.orchestra.run.smithi167.stdout:(119/152): libarrow-9.0.0-11.el9.x86_64.rpm 8.5 MB/s | 4.4 MB 00:00 2024-04-26T17:58:25.484 INFO:teuthology.orchestra.run.smithi167.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-04-26T17:58:25.509 INFO:teuthology.orchestra.run.smithi167.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-26T17:58:25.535 INFO:teuthology.orchestra.run.smithi167.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-26T17:58:25.568 INFO:teuthology.orchestra.run.smithi167.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-26T17:58:25.594 INFO:teuthology.orchestra.run.smithi167.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-04-26T17:58:25.619 INFO:teuthology.orchestra.run.smithi167.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-26T17:58:25.669 INFO:teuthology.orchestra.run.smithi167.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-26T17:58:25.686 INFO:teuthology.orchestra.run.smithi167.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-26T17:58:25.703 INFO:teuthology.orchestra.run.smithi167.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-26T17:58:25.728 INFO:teuthology.orchestra.run.smithi167.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 335 kB/s | 20 kB 00:00 2024-04-26T17:58:25.754 INFO:teuthology.orchestra.run.smithi167.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-04-26T17:58:25.871 INFO:teuthology.orchestra.run.smithi167.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-26T17:58:25.963 INFO:teuthology.orchestra.run.smithi167.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 276 kB/s | 58 kB 00:00 2024-04-26T17:58:25.980 INFO:teuthology.orchestra.run.smithi167.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 314 kB/s | 79 kB 00:00 2024-04-26T17:58:26.005 INFO:teuthology.orchestra.run.smithi167.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-26T17:58:26.022 INFO:teuthology.orchestra.run.smithi167.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 279 kB/s | 16 kB 00:00 2024-04-26T17:58:26.033 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2024-04-26T17:58:26.047 INFO:teuthology.orchestra.run.smithi167.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-26T17:58:26.064 INFO:teuthology.orchestra.run.smithi167.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-04-26T17:58:26.081 INFO:teuthology.orchestra.run.smithi167.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 523 kB/s | 31 kB 00:00 2024-04-26T17:58:26.106 INFO:teuthology.orchestra.run.smithi167.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-26T17:58:26.131 INFO:teuthology.orchestra.run.smithi167.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 880 kB/s | 59 kB 00:00 2024-04-26T17:58:26.157 INFO:teuthology.orchestra.run.smithi167.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 473 kB/s | 36 kB 00:00 2024-04-26T17:58:26.174 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2024-04-26T17:58:26.175 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2024-04-26T17:58:26.182 INFO:teuthology.orchestra.run.smithi167.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-26T17:58:26.207 INFO:teuthology.orchestra.run.smithi167.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-26T17:58:26.224 INFO:teuthology.orchestra.run.smithi167.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-26T17:58:26.249 INFO:teuthology.orchestra.run.smithi167.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-26T17:58:26.275 INFO:teuthology.orchestra.run.smithi167.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-04-26T17:58:26.300 INFO:teuthology.orchestra.run.smithi167.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-26T17:58:26.342 INFO:teuthology.orchestra.run.smithi167.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T17:58:26.642 INFO:teuthology.orchestra.run.smithi167.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 1.3 MB 00:00 2024-04-26T17:58:26.926 INFO:teuthology.orchestra.run.smithi167.stdout:(150/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-04-26T17:58:27.118 INFO:teuthology.orchestra.run.smithi167.stdout:(151/152): librados2-19.0.0-3267.gbc94a2a9.el9. 4.0 MB/s | 3.4 MB 00:00 2024-04-26T17:58:27.160 INFO:teuthology.orchestra.run.smithi167.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 4.6 MB/s | 2.4 MB 00:00 2024-04-26T17:58:27.163 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-04-26T17:58:27.163 INFO:teuthology.orchestra.run.smithi167.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-26T17:58:27.924 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2024-04-26T17:58:27.925 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2024-04-26T17:58:28.266 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-04-26T17:58:28.413 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-04-26T17:58:28.413 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-04-26T17:58:30.199 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-04-26T17:58:30.199 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-04-26T17:58:31.119 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T17:58:31.294 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T17:58:31.595 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T17:58:31.657 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T17:58:31.706 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T17:58:31.730 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T17:58:31.775 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T17:58:32.114 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T17:58:32.256 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:58:32.387 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:58:32.416 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:58:32.491 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:58:32.521 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T17:58:32.671 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T17:58:32.727 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T17:58:32.756 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:58:32.859 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:58:32.890 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T17:58:32.953 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T17:58:32.983 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T17:58:33.004 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T17:58:33.199 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T17:58:33.233 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T17:58:33.342 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T17:58:33.437 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T17:58:33.467 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T17:58:33.510 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T17:58:33.583 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T17:58:33.635 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T17:58:33.688 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T17:58:33.728 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T17:58:33.756 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T17:58:33.824 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T17:58:33.887 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T17:58:33.913 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2024-04-26T17:58:33.926 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T17:58:33.970 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T17:58:33.985 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T17:58:34.022 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T17:58:34.046 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T17:58:34.081 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T17:58:34.125 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T17:58:34.152 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T17:58:34.153 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T17:58:34.223 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T17:58:34.277 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T17:58:34.311 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T17:58:34.336 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T17:58:34.351 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T17:58:34.381 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T17:58:34.485 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T17:58:34.539 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T17:58:34.708 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T17:58:34.812 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:58:34.942 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:58:34.980 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:58:35.071 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:58:35.078 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T17:58:35.101 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T17:58:35.219 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T17:58:35.249 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T17:58:35.299 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T17:58:35.306 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T17:58:35.326 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:58:35.349 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T17:58:35.387 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T17:58:35.413 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:58:35.428 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T17:58:35.436 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T17:58:35.498 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T17:58:35.520 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T17:58:35.540 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T17:58:35.635 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T17:58:35.684 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T17:58:35.736 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T17:58:35.747 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T17:58:35.769 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T17:58:35.819 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T17:58:35.859 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T17:58:35.876 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T17:58:35.895 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T17:58:35.943 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T17:58:35.971 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T17:58:35.978 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T17:58:36.010 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T17:58:36.030 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T17:58:36.054 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T17:58:36.081 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T17:58:36.136 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T17:58:36.178 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T17:58:36.222 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T17:58:36.226 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T17:58:36.240 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-04-26T17:58:36.255 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T17:58:36.271 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T17:58:36.283 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T17:58:36.330 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T17:58:36.352 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T17:58:36.369 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T17:58:36.405 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T17:58:36.422 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T17:58:36.436 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T17:58:36.465 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T17:58:36.488 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T17:58:36.490 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T17:58:36.508 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T17:58:36.533 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T17:58:36.572 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T17:58:36.607 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T17:58:36.651 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T17:58:36.679 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T17:58:36.748 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T17:58:36.793 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T17:58:36.794 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T17:58:36.861 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T17:58:36.868 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T17:58:36.910 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T17:58:36.952 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T17:58:36.974 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:58:36.992 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T17:58:37.003 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T17:58:37.026 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T17:58:37.051 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T17:58:37.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T17:58:37.096 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:58:37.125 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:58:37.128 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T17:58:37.216 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:58:37.255 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T17:58:37.263 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T17:58:37.293 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T17:58:37.330 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T17:58:37.364 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T17:58:37.404 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T17:58:37.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T17:58:37.454 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T17:58:37.454 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T17:58:37.480 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:58:37.483 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T17:58:37.518 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T17:58:37.551 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T17:58:37.574 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:58:37.589 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T17:58:37.592 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T17:58:37.597 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T17:58:37.620 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T17:58:37.645 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T17:58:37.667 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T17:58:37.675 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T17:58:37.689 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T17:58:37.698 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T17:58:37.704 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T17:58:37.750 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T17:58:37.755 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T17:58:37.782 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T17:58:37.803 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T17:58:37.828 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T17:58:37.852 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T17:58:37.881 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T17:58:37.923 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T17:58:37.936 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T17:58:37.991 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T17:58:38.025 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T17:58:38.036 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T17:58:38.080 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T17:58:38.129 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T17:58:38.134 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T17:58:38.165 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T17:58:38.198 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T17:58:38.200 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T17:58:38.230 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T17:58:38.267 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T17:58:38.274 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T17:58:38.314 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T17:58:38.316 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T17:58:38.350 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T17:58:38.368 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T17:58:38.392 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T17:58:38.409 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T17:58:38.436 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T17:58:38.444 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T17:58:38.506 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T17:58:38.559 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T17:58:38.577 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T17:58:38.589 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T17:58:38.632 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T17:58:38.633 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T17:58:38.662 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T17:58:38.703 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T17:58:38.710 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T17:58:38.717 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T17:58:38.743 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T17:58:38.773 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T17:58:38.787 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T17:58:38.797 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T17:58:38.807 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T17:58:38.815 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T17:58:38.829 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T17:58:38.869 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T17:58:38.886 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T17:58:38.894 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T17:58:38.906 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T17:58:38.919 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T17:58:38.930 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T17:58:38.966 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T17:58:38.993 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T17:58:39.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T17:58:39.053 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T17:58:39.095 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T17:58:39.120 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T17:58:39.142 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T17:58:39.160 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T17:58:39.188 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T17:58:39.222 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T17:58:39.224 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T17:58:39.262 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T17:58:39.263 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T17:58:39.293 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T17:58:39.293 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T17:58:39.324 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T17:58:39.358 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T17:58:39.379 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T17:58:39.399 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T17:58:39.441 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T17:58:39.476 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T17:58:39.563 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T17:58:39.606 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T17:58:39.721 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T17:58:39.765 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T17:58:39.767 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T17:58:39.854 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T17:58:39.866 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T17:58:39.882 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T17:58:39.923 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T17:58:39.924 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T17:58:39.963 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T17:58:40.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T17:58:40.030 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T17:58:40.168 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T17:58:40.171 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T17:58:40.241 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T17:58:40.296 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T17:58:40.369 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T17:58:40.400 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T17:58:40.436 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T17:58:40.483 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T17:58:40.520 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T17:58:40.562 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T17:58:40.604 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T17:58:40.630 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T17:58:40.669 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T17:58:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T17:58:40.742 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T17:58:40.793 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T17:58:40.823 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T17:58:40.825 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T17:58:40.865 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T17:58:40.878 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T17:58:40.960 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T17:58:40.973 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T17:58:40.998 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T17:58:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T17:58:41.038 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T17:58:41.064 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T17:58:41.089 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T17:58:41.163 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T17:58:41.207 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T17:58:41.237 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T17:58:41.290 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T17:58:41.314 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T17:58:41.321 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T17:58:41.349 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T17:58:41.384 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T17:58:41.398 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T17:58:41.422 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T17:58:41.454 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T17:58:41.484 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T17:58:41.485 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T17:58:41.513 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T17:58:41.518 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T17:58:41.539 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T17:58:41.552 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T17:58:41.593 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T17:58:41.628 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T17:58:41.801 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T17:58:41.856 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T17:58:41.902 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T17:58:41.908 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T17:58:41.950 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T17:58:41.950 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T17:58:41.992 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T17:58:41.995 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T17:58:42.024 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T17:58:42.059 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T17:58:42.068 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T17:58:42.094 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T17:58:42.141 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T17:58:42.550 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T17:58:42.581 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T17:58:42.652 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T17:58:42.683 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T17:58:42.866 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T17:58:42.913 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T17:58:42.964 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T17:58:43.065 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T17:58:43.122 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T17:58:43.209 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T17:58:43.241 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T17:58:43.269 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T17:58:43.358 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T17:58:43.386 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T17:58:43.419 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T17:58:43.420 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T17:58:43.447 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T17:58:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T17:58:43.739 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T17:58:43.791 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T17:58:43.881 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T17:58:43.912 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T17:58:43.948 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T17:58:44.159 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T17:58:44.287 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T17:58:44.430 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T17:58:44.461 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T17:58:44.515 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T17:58:44.537 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T17:58:44.970 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:45.026 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:45.513 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T17:58:45.560 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:45.593 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T17:58:45.603 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T17:58:45.621 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T17:58:45.655 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T17:58:45.681 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T17:58:45.681 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T17:58:45.681 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T17:58:45.681 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:58:45.713 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T17:58:45.966 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T17:58:46.001 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T17:58:46.091 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T17:58:46.122 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T17:58:46.158 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T17:58:46.638 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T17:58:46.671 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T17:58:46.742 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T17:58:46.764 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T17:58:46.827 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:46.875 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:47.422 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:47.462 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T17:58:47.556 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T17:58:47.556 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T17:58:47.556 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T17:58:47.556 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:58:47.589 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T17:58:49.053 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:49.094 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:49.646 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T17:58:49.689 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T17:58:49.775 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T17:58:49.775 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T17:58:49.775 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T17:58:49.775 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:58:49.799 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T17:58:58.894 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T17:58:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2024-04-26T17:58:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2024-04-26T17:58:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2024-04-26T17:58:58.894 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2024-04-26T17:58:58.895 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2024-04-26T17:58:58.895 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2024-04-26T17:58:58.895 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2024-04-26T17:58:58.895 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-04-26T17:58:58.896 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:58:59.065 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T17:58:59.081 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T17:58:59.081 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:58:59.081 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T17:58:59.082 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:58:59.085 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T17:58:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T17:58:59.101 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:58:59.101 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T17:58:59.101 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:00.570 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T17:59:00.588 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T17:59:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T17:59:00.596 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:00.596 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T17:59:00.596 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T17:59:00.596 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T17:59:00.596 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:00.613 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T17:59:00.614 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:00.614 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T17:59:00.614 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T17:59:00.614 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T17:59:00.614 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:00.820 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T17:59:00.839 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T17:59:00.843 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T17:59:00.843 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:00.843 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T17:59:00.843 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T17:59:00.844 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T17:59:00.844 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:00.861 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T17:59:00.861 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:00.861 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T17:59:00.861 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T17:59:00.861 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T17:59:00.861 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:01.280 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T17:59:01.303 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T17:59:01.303 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:01.303 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T17:59:01.303 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T17:59:01.303 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T17:59:01.303 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:01.306 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T17:59:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T17:59:01.329 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:01.329 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T17:59:01.329 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T17:59:01.329 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T17:59:01.329 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:01.410 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T17:59:01.426 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T17:59:01.426 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-04-26T17:59:01.426 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-04-26T17:59:01.426 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-04-26T17:59:01.427 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-04-26T17:59:01.427 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-04-26T17:59:01.427 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-04-26T17:59:01.427 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-04-26T17:59:01.427 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:01.444 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T17:59:01.467 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T17:59:01.492 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T17:59:01.531 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T17:59:01.547 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T17:59:01.596 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T17:59:01.610 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T17:59:01.610 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:01.610 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T17:59:01.610 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:01.725 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T17:59:01.733 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T17:59:01.774 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T17:59:01.781 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T17:59:01.851 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T17:59:01.853 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T17:59:01.938 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T17:59:01.941 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T17:59:02.068 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T17:59:02.080 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T17:59:02.123 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T17:59:02.125 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T17:59:02.156 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T17:59:02.157 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T17:59:02.183 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T17:59:02.185 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T17:59:02.561 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T17:59:02.591 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T17:59:02.623 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T17:59:02.657 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T17:59:02.685 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T17:59:02.693 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T17:59:02.722 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T17:59:02.725 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T17:59:02.795 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T17:59:02.834 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T17:59:03.081 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T17:59:03.107 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T17:59:03.107 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:03.107 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T17:59:03.107 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T17:59:03.107 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T17:59:03.107 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:03.350 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T17:59:03.372 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T17:59:03.373 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:03.373 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T17:59:03.373 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T17:59:03.373 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T17:59:03.373 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:03.759 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T17:59:03.791 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T17:59:03.818 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T17:59:03.841 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T17:59:03.841 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:03.841 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T17:59:03.841 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T17:59:03.842 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T17:59:03.842 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:03.870 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T17:59:03.911 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T17:59:03.957 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T17:59:04.005 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T17:59:04.062 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T17:59:04.246 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T17:59:04.294 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T17:59:04.372 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T17:59:04.470 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T17:59:04.611 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T17:59:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T17:59:04.693 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T17:59:04.721 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T17:59:04.754 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T17:59:04.791 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T17:59:04.876 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T17:59:04.893 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T17:59:04.903 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T17:59:05.004 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T17:59:05.053 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T17:59:05.076 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T17:59:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T17:59:05.100 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:05.100 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T17:59:05.100 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T17:59:05.101 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T17:59:05.101 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:05.144 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T17:59:05.156 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T17:59:05.164 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T17:59:05.166 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T17:59:05.190 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T17:59:05.194 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T17:59:05.213 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T17:59:05.213 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:05.213 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T17:59:05.213 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T17:59:05.213 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T17:59:05.213 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:05.243 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T17:59:05.252 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T17:59:05.258 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T17:59:05.269 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T17:59:05.300 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T17:59:05.313 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T17:59:05.343 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T17:59:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T17:59:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T17:59:06.223 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T17:59:06.255 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T17:59:06.277 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T17:59:06.278 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.278 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T17:59:06.278 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T17:59:06.278 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T17:59:06.278 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:06.322 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T17:59:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T17:59:06.345 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.345 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T17:59:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:06.345 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T17:59:06.377 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T17:59:06.379 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T17:59:06.398 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T17:59:06.401 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T17:59:06.401 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.401 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T17:59:06.401 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T17:59:06.401 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T17:59:06.401 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:06.411 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T17:59:06.420 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T17:59:06.421 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.421 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T17:59:06.421 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T17:59:06.421 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T17:59:06.421 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:06.444 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T17:59:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T17:59:06.467 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.467 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T17:59:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T17:59:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T17:59:06.535 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.535 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T17:59:06.535 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T17:59:06.535 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T17:59:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:06.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T17:59:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T17:59:06.755 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.755 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T17:59:06.755 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T17:59:06.755 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T17:59:06.755 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T17:59:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T17:59:06.860 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:06.860 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T17:59:06.860 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T17:59:06.860 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T17:59:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:07.379 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T17:59:07.411 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T17:59:07.513 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T17:59:07.673 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T17:59:07.705 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T17:59:07.729 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T17:59:07.729 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:07.729 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T17:59:07.730 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T17:59:07.730 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T17:59:07.730 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:07.773 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T17:59:07.784 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T17:59:07.883 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T17:59:07.946 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T17:59:08.878 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T17:59:08.911 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T17:59:08.934 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T17:59:08.934 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:08.934 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T17:59:08.934 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T17:59:08.934 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T17:59:08.934 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:08.983 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T17:59:09.000 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T17:59:09.000 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:09.000 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T17:59:09.000 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:09.045 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T17:59:09.071 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T17:59:09.071 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:09.072 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T17:59:09.072 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T17:59:09.072 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T17:59:09.072 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:09.380 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T17:59:09.403 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T17:59:09.403 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T17:59:09.403 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T17:59:09.403 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T17:59:09.403 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T17:59:09.403 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:11.352 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T17:59:11.394 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T17:59:11.434 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T17:59:11.467 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T17:59:11.469 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T17:59:11.494 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T17:59:11.518 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T17:59:11.559 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T17:59:11.591 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T17:59:11.627 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T17:59:11.678 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T17:59:11.724 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T17:59:11.753 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T17:59:11.811 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T17:59:11.858 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T17:59:11.880 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T17:59:12.341 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:12.385 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:12.405 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:12.474 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:12.522 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:12.542 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:13.228 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T17:59:13.325 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T17:59:13.366 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T17:59:13.373 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T17:59:13.447 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T17:59:13.473 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T17:59:13.479 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T17:59:13.479 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T17:59:13.513 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T17:59:13.564 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T17:59:13.597 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T17:59:13.597 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T17:59:13.603 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T17:59:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T17:59:13.644 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T17:59:13.699 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T17:59:13.745 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T17:59:13.770 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T17:59:13.778 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T17:59:13.873 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T17:59:13.945 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:14.033 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T17:59:14.074 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T17:59:14.115 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T17:59:14.147 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T17:59:14.174 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T17:59:14.357 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T17:59:14.404 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T17:59:14.434 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T17:59:15.061 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:15.107 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:15.126 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T17:59:15.970 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T17:59:16.061 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T17:59:16.103 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T17:59:16.187 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T17:59:16.241 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T17:59:16.241 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T17:59:16.317 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T17:59:16.358 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T17:59:16.358 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T17:59:16.439 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T17:59:16.506 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:19.876 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:59:19.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T17:59:19.878 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T17:59:19.879 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T17:59:19.880 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T17:59:19.881 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T17:59:19.882 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T17:59:19.883 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T17:59:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T17:59:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T17:59:19.886 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T17:59:19.887 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T17:59:19.888 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T17:59:19.889 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T17:59:19.890 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T17:59:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T17:59:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T17:59:19.891 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T17:59:21.922 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:21.922 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T17:59:21.922 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T17:59:21.922 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T17:59:21.923 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T17:59:21.924 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T17:59:21.925 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T17:59:21.926 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T17:59:21.927 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T17:59:21.928 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T17:59:21.929 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T17:59:21.930 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T17:59:21.931 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T17:59:21.932 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T17:59:21.933 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T17:59:21.934 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T17:59:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:21.961 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.962 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.963 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T17:59:21.964 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T17:59:21.965 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T17:59:21.966 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T17:59:21.967 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T17:59:21.968 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T17:59:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T17:59:21.970 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T17:59:21.971 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.972 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout: 2024-04-26T17:59:21.973 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2024-04-26T17:59:22.234 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:22.234 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T17:59:22.234 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T17:59:22.234 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T17:59:22.234 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T17:59:22.234 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T17:59:22.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T17:59:22.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T17:59:22.248 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T17:59:22.248 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T17:59:22.248 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T17:59:22.249 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T17:59:22.250 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T17:59:22.251 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T17:59:22.252 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T17:59:22.253 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T17:59:22.254 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T17:59:22.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T17:59:22.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T17:59:22.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T17:59:22.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T17:59:22.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T17:59:22.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T17:59:22.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T17:59:22.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T17:59:22.348 DEBUG:teuthology.parallel:result is None 2024-04-26T17:59:23.935 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:23.935 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:23.935 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-04-26T17:59:23.935 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.935 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T17:59:23.936 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.937 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.938 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.938 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.938 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.938 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.938 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.938 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.938 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T17:59:23.939 INFO:teuthology.orchestra.run.smithi090.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T17:59:23.940 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T17:59:23.940 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T17:59:23.940 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T17:59:23.940 INFO:teuthology.orchestra.run.smithi090.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T17:59:23.940 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T17:59:23.940 INFO:teuthology.orchestra.run.smithi090.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T17:59:23.940 INFO:teuthology.orchestra.run.smithi090.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T17:59:23.941 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T17:59:23.941 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T17:59:23.941 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.941 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.941 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.941 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T17:59:23.942 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T17:59:23.942 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T17:59:23.942 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T17:59:23.942 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T17:59:23.942 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T17:59:23.943 INFO:teuthology.orchestra.run.smithi090.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T17:59:23.944 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T17:59:23.945 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.946 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.946 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.946 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T17:59:23.946 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T17:59:23.946 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T17:59:23.946 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T17:59:23.946 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T17:59:23.947 INFO:teuthology.orchestra.run.smithi090.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T17:59:23.947 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T17:59:23.947 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T17:59:23.947 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T17:59:23.947 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T17:59:23.947 INFO:teuthology.orchestra.run.smithi090.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T17:59:23.947 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T17:59:23.948 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T17:59:23.948 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T17:59:23.948 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T17:59:23.948 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T17:59:23.948 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T17:59:23.948 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T17:59:23.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T17:59:23.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T17:59:23.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T17:59:23.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T17:59:23.949 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T17:59:23.950 INFO:teuthology.orchestra.run.smithi090.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.951 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T17:59:23.952 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T17:59:23.953 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T17:59:23.954 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T17:59:23.954 INFO:teuthology.orchestra.run.smithi090.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T17:59:23.954 INFO:teuthology.orchestra.run.smithi090.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T17:59:23.954 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.954 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.954 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:23.955 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T17:59:23.955 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T17:59:23.955 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T17:59:23.955 INFO:teuthology.orchestra.run.smithi090.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T17:59:23.955 INFO:teuthology.orchestra.run.smithi090.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T17:59:23.955 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T17:59:23.956 INFO:teuthology.orchestra.run.smithi090.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T17:59:23.956 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T17:59:23.956 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T17:59:24.356 DEBUG:teuthology.parallel:result is None 2024-04-26T17:59:24.421 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T17:59:24.421 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:24.421 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-04-26T17:59:24.422 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.422 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.422 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:24.422 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:24.422 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-04-26T17:59:24.422 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T17:59:24.422 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.423 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.424 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.425 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.425 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.425 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.425 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.425 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.426 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.426 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.426 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.426 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.426 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T17:59:24.427 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T17:59:24.427 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T17:59:24.427 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T17:59:24.427 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T17:59:24.427 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T17:59:24.427 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T17:59:24.428 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T17:59:24.428 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T17:59:24.428 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T17:59:24.428 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T17:59:24.428 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T17:59:24.428 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.428 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T17:59:24.429 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T17:59:24.430 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.431 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T17:59:24.432 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T17:59:24.433 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T17:59:24.434 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T17:59:24.435 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T17:59:24.436 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T17:59:24.437 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T17:59:24.438 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T17:59:24.438 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T17:59:24.438 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T17:59:24.438 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T17:59:24.438 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T17:59:24.438 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T17:59:24.438 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-04-26T17:59:24.831 DEBUG:teuthology.parallel:result is None 2024-04-26T17:59:24.832 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:59:24.975 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T17:59:24.996 INFO:teuthology.orchestra.run.smithi090.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T17:59:24.997 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T17:59:24.997 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T17:59:24.999 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:59:25.213 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T17:59:25.235 INFO:teuthology.orchestra.run.smithi119.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T17:59:25.235 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T17:59:25.235 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T17:59:25.237 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:59:25.453 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T17:59:25.475 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T17:59:25.475 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T17:59:25.475 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T17:59:25.477 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T17:59:25.477 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T17:59:25.477 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T17:59:25.503 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T17:59:25.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T17:59:25.529 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T17:59:25.529 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T17:59:25.556 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T17:59:25.557 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T17:59:25.557 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T17:59:25.580 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T17:59:25.644 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T17:59:25.644 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T17:59:25.668 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T17:59:25.732 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T17:59:25.732 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T17:59:25.755 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T17:59:25.820 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T17:59:25.820 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T17:59:25.820 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T17:59:25.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T17:59:25.907 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T17:59:25.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T17:59:25.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T17:59:25.994 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T17:59:25.994 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T17:59:26.017 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T17:59:26.080 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T17:59:26.080 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T17:59:26.080 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T17:59:26.103 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T17:59:26.167 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T17:59:26.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T17:59:26.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T17:59:26.253 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T17:59:26.253 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T17:59:26.276 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T17:59:26.342 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T17:59:26.449 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-26T17:59:26.449 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:59:26.450 INFO:tasks.cephadm:Cluster fsid is b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T17:59:26.450 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T17:59:26.450 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.90', 'mon.b': '172.21.15.119', 'mon.c': '172.21.15.167'} 2024-04-26T17:59:26.450 INFO:tasks.cephadm:First mon is mon.a on smithi090 2024-04-26T17:59:26.451 INFO:tasks.cephadm:First mgr is x 2024-04-26T17:59:26.451 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T17:59:26.451 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-04-26T17:59:26.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2024-04-26T17:59:26.499 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-04-26T17:59:26.523 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T17:59:26.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:59:26.746 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78913/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3267-gbc94a2a9', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3267.gbc94a2a9'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 08:22:46.917786', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer-fixes', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-26T17:59:26.965 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiescer-fixes, sha1 bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T17:59:26.966 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm 2024-04-26T17:59:26.967 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:27.560 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 17:59 /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:27.561 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:28.122 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 17:59 /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:28.123 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:28.535 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 17:59 /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:28.536 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:28.552 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:28.570 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T17:59:28.606 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 on all hosts... 2024-04-26T17:59:28.606 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T17:59:28.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T17:59:28.613 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T17:59:30.229 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T17:59:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T17:59:31.039 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:00:06.109 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-04-26T18:00:06.109 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:00:06.109 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:00:06.109 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-04-26T18:00:06.109 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:00:06.110 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-04-26T18:00:06.110 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-04-26T18:00:10.926 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2024-04-26T18:00:10.927 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:00:10.927 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:00:10.927 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2024-04-26T18:00:10.927 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:00:10.927 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2024-04-26T18:00:10.927 INFO:teuthology.orchestra.run.smithi119.stdout:} 2024-04-26T18:00:11.303 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:00:11.303 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:00:11.303 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:00:11.303 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-04-26T18:00:11.303 INFO:teuthology.orchestra.run.smithi090.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:00:11.303 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T18:00:11.303 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:00:11.328 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-04-26T18:00:11.373 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2024-04-26T18:00:11.401 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-04-26T18:00:11.432 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-04-26T18:00:11.456 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2024-04-26T18:00:11.480 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-04-26T18:00:11.506 INFO:tasks.cephadm:Writing seed config... 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T18:00:11.507 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T18:00:11.508 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T18:00:11.509 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T18:00:11.510 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T18:00:11.511 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:00:11.511 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T18:00:11.526 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b8878a18-03f6-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T18:00:11.526 DEBUG:teuthology.orchestra.run.smithi090:mon.a> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T18:00:11.568 DEBUG:teuthology.orchestra.run.smithi090:mgr.x> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T18:00:11.611 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T18:00:11.612 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 -v bootstrap --fsid b8878a18-03f6-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:00:11.802 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:00:11.803 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', '-v', 'bootstrap', '--fsid', 'b8878a18-03f6-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-04-26T18:00:11.803 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T18:00:11.803 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-04-26T18:00:11.831 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T18:00:11.831 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-04-26T18:00:11.831 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-04-26T18:00:11.837 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T18:00:11.837 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T18:00:11.843 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T18:00:11.843 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T18:00:11.849 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-26T18:00:11.855 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-26T18:00:11.855 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-26T18:00:11.855 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-04-26T18:00:11.879 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T18:00:11.879 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T18:00:11.879 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-04-26T18:00:11.879 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-04-26T18:00:11.885 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T18:00:11.886 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T18:00:11.891 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T18:00:11.891 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T18:00:11.897 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-26T18:00:11.904 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-26T18:00:11.904 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-26T18:00:11.904 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-04-26T18:00:11.904 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:11.904 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 139708219092128 on /run/cephadm/b8878a18-03f6-11ef-bc93-c7b262605968.lock 2024-04-26T18:00:11.904 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 139708219092128 acquired on /run/cephadm/b8878a18-03f6-11ef-bc93-c7b262605968.lock 2024-04-26T18:00:11.904 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-04-26T18:00:11.905 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-04-26T18:00:11.905 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-26T18:00:11.908 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-04-26T18:00:11.908 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-04-26T18:00:11.910 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T18:00:11.910 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-26T18:00:11.910 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-26T18:00:11.913 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T18:00:11.913 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T18:00:11.913 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T18:00:11.913 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T18:00:11.913 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::8025:e85a:626f:c8e5/64 scope link noprefixroute 2024-04-26T18:00:11.913 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T18:00:11.914 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-26T18:00:11.914 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-26T18:00:11.914 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T18:00:11.914 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T18:00:11.915 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:00:16.556 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 2024-04-26T18:00:16.556 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:00:16.556 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T18:00:16.556 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:449452a284afe5840255d087488a224329a4679c5f5d200bb867a639c75ddfd2 2024-04-26T18:00:16.556 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T18:00:16.556 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 2024-04-26T18:00:16.556 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T18:00:28.824 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:00:28.824 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:00:28.824 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T18:00:30.036 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-04-26T18:00:30.036 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-04-26T18:00:30.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQC+6ytmZYy8FRAA/7JPrv+/5cda6a1PJgOV5w== 2024-04-26T18:00:31.783 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQC/6ytmVKr3DBAA4HPD5TPelvH6tamVznKBLg== 2024-04-26T18:00:32.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQDA6ytmaJ49BhAAOUIe+1yBMTf/NrfXN7iiyQ== 2024-04-26T18:00:32.618 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for a [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = quincy 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T18:00:33.577 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:00:33.578 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-04-26T18:00:35.013 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.a on 2024-04-26T18:00:35.235 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T18:00:35.373 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T18:00:35.541 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b8878a18-03f6-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-b8878a18-03f6-11ef-bc93-c7b262605968.target. 2024-04-26T18:00:35.541 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b8878a18-03f6-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-b8878a18-03f6-11ef-bc93-c7b262605968.target. 2024-04-26T18:00:35.785 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.a 2024-04-26T18:00:35.785 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T18:00:35.945 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b8878a18-03f6-11ef-bc93-c7b262605968.target.wants/ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b8878a18-03f6-11ef-bc93-c7b262605968@.service. 2024-04-26T18:00:36.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 podman[28792]: 2024-04-26 18:00:36.190454016 +0000 UTC m=+0.098235901 container create d7d7f8cca0b7beccdcf362a2b69097b94d8e52fb565b02480a27c8c51f6bc82a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T18:00:36.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 podman[28792]: 2024-04-26 18:00:36.110247963 +0000 UTC m=+0.018029870 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:00:36.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 podman[28792]: 2024-04-26 18:00:36.418564043 +0000 UTC m=+0.326345931 container init d7d7f8cca0b7beccdcf362a2b69097b94d8e52fb565b02480a27c8c51f6bc82a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:00:36.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 podman[28792]: 2024-04-26 18:00:36.421870852 +0000 UTC m=+0.329652745 container start d7d7f8cca0b7beccdcf362a2b69097b94d8e52fb565b02480a27c8c51f6bc82a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T18:00:36.501 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:00:36.501 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T18:00:36.508 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:00:36.508 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T18:00:36.508 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T18:00:36.508 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T18:00:36.508 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-04-26T18:00:36.508 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-04-26T18:00:36.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 ceph-mon[28811]: mkfs b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:36.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 ceph-mon[28811]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T18:00:36.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 bash[28792]: d7d7f8cca0b7beccdcf362a2b69097b94d8e52fb565b02480a27c8c51f6bc82a 2024-04-26T18:00:36.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:36 smithi090 systemd[1]: Started Ceph mon.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:00:37.557 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: monmap epoch 1 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: last_changed 2024-04-26T18:00:32.950419+0000 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: min_mon_release 19 (squid) 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: election_strategy: 1 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: fsmap 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: mgrmap e1: no daemons active 2024-04-26T18:00:37.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:37 smithi090 ceph-mon[28811]: from='client.? 172.21.15.90:0/2667863940' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.517533s) 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:37.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.906+0000 7f2ac2fc7700 1 Processor -- start 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.906+0000 7f2ac2fc7700 1 -- start start 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2ac2fc7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2ac2fc7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc109a40 con 0x7f2abc109050 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2ac0d63700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:37.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2ac0d63700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48238/0 (socket says 172.21.15.90:48238) 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2ac0d63700 1 -- 172.21.15.90:0/2732097013 learned_addr learned my addr 172.21.15.90:0/2732097013 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2ac0d63700 1 -- 172.21.15.90:0/2732097013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc10a250 con 0x7f2abc109050 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2ac0d63700 1 --2- 172.21.15.90:0/2732097013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009e80 tx=0x7f2ab400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc36128e13241d89 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.907+0000 7f2abb7fe700 1 -- 172.21.15.90:0/2732097013 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ab4017070 con 0x7f2abc109050 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2abb7fe700 1 -- 172.21.15.90:0/2732097013 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab4003a20 con 0x7f2abc109050 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2abb7fe700 1 -- 172.21.15.90:0/2732097013 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ab4003d20 con 0x7f2abc109050 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2732097013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 msgr2=0x7f2abc109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2ac2fc7700 1 --2- 172.21.15.90:0/2732097013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4009e80 tx=0x7f2ab400b330 comp rx=0 tx=0).stop 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2732097013 shutdown_connections 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2ac2fc7700 1 --2- 172.21.15.90:0/2732097013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc109470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2732097013 >> 172.21.15.90:0/2732097013 conn(0x7f2abc104610 msgr2=0x7f2abc106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2732097013 shutdown_connections 2024-04-26T18:00:37.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.908+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2732097013 wait complete. 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac2fc7700 1 Processor -- start 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac2fc7700 1 -- start start 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac2fc7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac2fc7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc10a780 con 0x7f2abc109050 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac0d63700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac0d63700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc19c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48254/0 (socket says 172.21.15.90:48254) 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac0d63700 1 -- 172.21.15.90:0/2667863940 learned_addr learned my addr 172.21.15.90:0/2667863940 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac0d63700 1 -- 172.21.15.90:0/2667863940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc19cb50 con 0x7f2abc109050 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.909+0000 7f2ac0d63700 1 --2- 172.21.15.90:0/2667863940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc19c610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4004000 tx=0x7f2ab4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.910+0000 7f2ab9ffb700 1 -- 172.21.15.90:0/2667863940 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ab4022020 con 0x7f2abc109050 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.910+0000 7f2ab9ffb700 1 -- 172.21.15.90:0/2667863940 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab4018040 con 0x7f2abc109050 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.910+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc19cde0 con 0x7f2abc109050 2024-04-26T18:00:37.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.910+0000 7f2ab9ffb700 1 -- 172.21.15.90:0/2667863940 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ab4017070 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.910+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc19fad0 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.910+0000 7f2ab9ffb700 1 -- 172.21.15.90:0/2667863940 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2ab4025020 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.910+0000 7f2ab9ffb700 1 -- 172.21.15.90:0/2667863940 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2ab400f050 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.911+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc064a80 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.913+0000 7f2ab9ffb700 1 -- 172.21.15.90:0/2667863940 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2ab402c030 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.981+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f2abc02e620 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.982+0000 7f2ab9ffb700 1 -- 172.21.15.90:0/2667863940 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f2ab401e480 con 0x7f2abc109050 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.983+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 msgr2=0x7f2abc19c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.983+0000 7f2ac2fc7700 1 --2- 172.21.15.90:0/2667863940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc19c610 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4004000 tx=0x7f2ab4004ad0 comp rx=0 tx=0).stop 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.983+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 shutdown_connections 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.983+0000 7f2ac2fc7700 1 --2- 172.21.15.90:0/2667863940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2abc109050 0x7f2abc19c610 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.983+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 >> 172.21.15.90:0/2667863940 conn(0x7f2abc104610 msgr2=0x7f2abc193310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:37.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.983+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 shutdown_connections 2024-04-26T18:00:37.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:36.983+0000 7f2ac2fc7700 1 -- 172.21.15.90:0/2667863940 wait complete. 2024-04-26T18:00:37.570 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-04-26T18:00:37.570 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T18:00:38.637 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.966+0000 7fc8ba9a0700 1 Processor -- start 2024-04-26T18:00:38.638 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.966+0000 7fc8ba9a0700 1 -- start start 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.966+0000 7fc8ba9a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b4107cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.966+0000 7fc8ba9a0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b407ce80 con 0x7fc8b41078a0 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b4107cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b4107cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48256/0 (socket says 172.21.15.90:48256) 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b3fff700 1 -- 172.21.15.90:0/2852614969 learned_addr learned my addr 172.21.15.90:0/2852614969 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b3fff700 1 -- 172.21.15.90:0/2852614969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b4108200 con 0x7fc8b41078a0 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b3fff700 1 --2- 172.21.15.90:0/2852614969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b4107cc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40098a0 tx=0x7fc8a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fd2c4800da6893 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b2ffd700 1 -- 172.21.15.90:0/2852614969 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a4004160 con 0x7fc8b41078a0 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b2ffd700 1 -- 172.21.15.90:0/2852614969 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a4004300 con 0x7fc8b41078a0 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.967+0000 7fc8b2ffd700 1 -- 172.21.15.90:0/2852614969 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a40036a0 con 0x7fc8b41078a0 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2852614969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 msgr2=0x7fc8b4107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 --2- 172.21.15.90:0/2852614969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b4107cc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40098a0 tx=0x7fc8a4009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:38.639 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2852614969 shutdown_connections 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 --2- 172.21.15.90:0/2852614969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b4107cc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2852614969 >> 172.21.15.90:0/2852614969 conn(0x7fc8b4103500 msgr2=0x7fc8b4105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2852614969 shutdown_connections 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2852614969 wait complete. 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 Processor -- start 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.968+0000 7fc8ba9a0700 1 -- start start 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8ba9a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8ba9a0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc8b4108730 con 0x7fc8b41078a0 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48268/0 (socket says 172.21.15.90:48268) 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b3fff700 1 -- 172.21.15.90:0/2515015192 learned_addr learned my addr 172.21.15.90:0/2515015192 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b3fff700 1 -- 172.21.15.90:0/2515015192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8b419ed00 con 0x7fc8b41078a0 2024-04-26T18:00:38.640 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b3fff700 1 --2- 172.21.15.90:0/2515015192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b419e7c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4000c00 tx=0x7fc8a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a4003db0 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8a4011630 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8b419ef90 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8a4011950 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8b41a1c80 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.969+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc8a4011af0 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.970+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8a401c450 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.970+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8b4064a80 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:37.973+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc8a4004260 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.040+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc8b41a1e50 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.043+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7fc8a401b450 con 0x7fc8b41078a0 2024-04-26T18:00:38.641 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.044+0000 7fc8b17fa700 1 -- 172.21.15.90:0/2515015192 <== mon.0 v2:172.21.15.90:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8a401b630 con 0x7fc8b41078a0 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.045+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 msgr2=0x7fc8b419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.045+0000 7fc8ba9a0700 1 --2- 172.21.15.90:0/2515015192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b419e7c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4000c00 tx=0x7fc8a4004750 comp rx=0 tx=0).stop 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.045+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 shutdown_connections 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.045+0000 7fc8ba9a0700 1 --2- 172.21.15.90:0/2515015192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8b41078a0 0x7fc8b419e7c0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.045+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 >> 172.21.15.90:0/2515015192 conn(0x7fc8b4103500 msgr2=0x7fc8b4194de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.045+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 shutdown_connections 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:38.045+0000 7fc8ba9a0700 1 -- 172.21.15.90:0/2515015192 wait complete. 2024-04-26T18:00:38.642 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-04-26T18:00:38.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:38 smithi090 ceph-mon[28811]: from='client.? 172.21.15.90:0/2515015192' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T18:00:38.745 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:38 smithi090 ceph-mon[28811]: from='client.? 172.21.15.90:0/2515015192' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T18:00:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:39 smithi090 ceph-mon[28811]: from='client.? 172.21.15.90:0/1366302061' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:00:40.778 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.035+0000 7feade333700 1 Processor -- start 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.035+0000 7feade333700 1 -- start start 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.035+0000 7feade333700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead810bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.035+0000 7feade333700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fead807ce20 con 0x7fead8109ae0 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead810bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead810bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48276/0 (socket says 172.21.15.90:48276) 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead7fff700 1 -- 172.21.15.90:0/1581925250 learned_addr learned my addr 172.21.15.90:0/1581925250 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead7fff700 1 -- 172.21.15.90:0/1581925250 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead810c410 con 0x7fead8109ae0 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead7fff700 1 --2- 172.21.15.90:0/1581925250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead810bed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feac80098a0 tx=0x7feac8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fcd015c792cf38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:40.779 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead6ffd700 1 -- 172.21.15.90:0/1581925250 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feac8004160 con 0x7fead8109ae0 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead6ffd700 1 -- 172.21.15.90:0/1581925250 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feac8004300 con 0x7fead8109ae0 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.036+0000 7fead6ffd700 1 -- 172.21.15.90:0/1581925250 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feac8011af0 con 0x7fead8109ae0 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.037+0000 7feade333700 1 -- 172.21.15.90:0/1581925250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 msgr2=0x7fead810bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.037+0000 7feade333700 1 --2- 172.21.15.90:0/1581925250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead810bed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feac80098a0 tx=0x7feac8009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.037+0000 7feade333700 1 -- 172.21.15.90:0/1581925250 shutdown_connections 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.037+0000 7feade333700 1 --2- 172.21.15.90:0/1581925250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead810bed0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.037+0000 7feade333700 1 -- 172.21.15.90:0/1581925250 >> 172.21.15.90:0/1581925250 conn(0x7fead81034f0 msgr2=0x7fead8105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.037+0000 7feade333700 1 -- 172.21.15.90:0/1581925250 shutdown_connections 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.037+0000 7feade333700 1 -- 172.21.15.90:0/1581925250 wait complete. 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7feade333700 1 Processor -- start 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7feade333700 1 -- start start 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7feade333700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:40.780 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7feade333700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fead810d2a0 con 0x7fead8109ae0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7fead7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7fead7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead819f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48286/0 (socket says 172.21.15.90:48286) 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7fead7fff700 1 -- 172.21.15.90:0/1366302061 learned_addr learned my addr 172.21.15.90:0/1366302061 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7fead7fff700 1 -- 172.21.15.90:0/1366302061 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fead819f5b0 con 0x7fead8109ae0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7fead7fff700 1 --2- 172.21.15.90:0/1366302061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead819f070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feac8003f40 tx=0x7feac8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.038+0000 7fead57fa700 1 -- 172.21.15.90:0/1366302061 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feac80038b0 con 0x7fead8109ae0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.039+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fead819f840 con 0x7fead8109ae0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.039+0000 7fead57fa700 1 -- 172.21.15.90:0/1366302061 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feac8003a50 con 0x7fead8109ae0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.039+0000 7fead57fa700 1 -- 172.21.15.90:0/1366302061 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feac801a5c0 con 0x7fead8109ae0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.039+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fead81a2530 con 0x7fead8109ae0 2024-04-26T18:00:40.781 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.039+0000 7fead57fa700 1 -- 172.21.15.90:0/1366302061 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feac801ac50 con 0x7fead8109ae0 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.039+0000 7fead57fa700 1 -- 172.21.15.90:0/1366302061 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feac8019700 con 0x7fead8109ae0 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.040+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fead8064a80 con 0x7fead8109ae0 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.042+0000 7fead57fa700 1 -- 172.21.15.90:0/1366302061 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7feac80199a0 con 0x7fead8109ae0 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.108+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fead81a2700 con 0x7fead8109ae0 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.108+0000 7fead57fa700 1 -- 172.21.15.90:0/1366302061 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7feac8019b80 con 0x7fead8109ae0 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.109+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 msgr2=0x7fead819f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.109+0000 7feade333700 1 --2- 172.21.15.90:0/1366302061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead819f070 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feac8003f40 tx=0x7feac8011890 comp rx=0 tx=0).stop 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.109+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 shutdown_connections 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.109+0000 7feade333700 1 --2- 172.21.15.90:0/1366302061 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fead8109ae0 0x7fead819f070 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.109+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 >> 172.21.15.90:0/1366302061 conn(0x7fead81034f0 msgr2=0x7fead8103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.109+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 shutdown_connections 2024-04-26T18:00:40.782 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:39.109+0000 7feade333700 1 -- 172.21.15.90:0/1366302061 wait complete. 2024-04-26T18:00:40.783 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-04-26T18:00:41.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:40 smithi090 systemd[1]: Stopping Ceph mon.a for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:00:41.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a[28806]: 2024-04-26T18:00:41.050+0000 7f8b03edf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:00:41.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a[28806]: 2024-04-26T18:00:41.050+0000 7f8b03edf700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T18:00:41.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 podman[29054]: 2024-04-26 18:00:41.101333979 +0000 UTC m=+0.149516319 container died d7d7f8cca0b7beccdcf362a2b69097b94d8e52fb565b02480a27c8c51f6bc82a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:00:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 podman[29054]: 2024-04-26 18:00:41.450080181 +0000 UTC m=+0.498262529 container cleanup d7d7f8cca0b7beccdcf362a2b69097b94d8e52fb565b02480a27c8c51f6bc82a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:00:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 bash[29054]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a 2024-04-26T18:00:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 podman[29068]: 2024-04-26 18:00:41.592882965 +0000 UTC m=+0.486388468 container remove d7d7f8cca0b7beccdcf362a2b69097b94d8e52fb565b02480a27c8c51f6bc82a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T18:00:42.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T18:00:42.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 systemd[1]: Stopped Ceph mon.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:00:42.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:41 smithi090 systemd[1]: Starting Ceph mon.a for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:00:42.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 podman[29161]: 2024-04-26 18:00:42.019059868 +0000 UTC m=+0.105774236 container create a34a7841c5bb55300d1da9594e811e5a6515ae798bc7355afaa9670db971449b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:00:42.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 podman[29161]: 2024-04-26 18:00:41.930881679 +0000 UTC m=+0.017596056 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:00:42.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 podman[29161]: 2024-04-26 18:00:42.197850761 +0000 UTC m=+0.284565119 container init a34a7841c5bb55300d1da9594e811e5a6515ae798bc7355afaa9670db971449b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T18:00:42.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 podman[29161]: 2024-04-26 18:00:42.200711391 +0000 UTC m=+0.287425750 container start a34a7841c5bb55300d1da9594e811e5a6515ae798bc7355afaa9670db971449b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev), process ceph-mon, pid 2 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: pidfile_write: ignore empty --pid-file 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: load: jerasure load: lrc 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: RocksDB version: 7.9.2 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Git sha 0 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Compile date 2024-04-26 07:20:26 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: DB SUMMARY 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: DB Session ID: RVKT645YOE5JTL7H9RXX 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: CURRENT file: CURRENT 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T18:00:42.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105442 ; 2024-04-26T18:00:42.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.error_if_exists: 0 2024-04-26T18:00:42.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.create_if_missing: 0 2024-04-26T18:00:42.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.paranoid_checks: 1 2024-04-26T18:00:42.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.env: 0x55d97dc5dcc0 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.info_log: 0x55d97ec07d80 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.statistics: (nil) 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.use_fsync: 0 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_log_file_size: 0 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T18:00:42.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.allow_fallocate: 1 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.use_direct_reads: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.db_log_dir: 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.wal_dir: 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T18:00:42.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.write_buffer_manager: 0x55d97ebd90e0 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.unordered_write: 0 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T18:00:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T18:00:42.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T18:00:42.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T18:00:42.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.row_cache: None 2024-04-26T18:00:42.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.wal_filter: None 2024-04-26T18:00:42.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T18:00:42.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.two_write_queues: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.wal_compression: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.atomic_flush: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.log_readahead_size: 0 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T18:00:42.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_background_jobs: 2 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_background_compactions: -1 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_subcompactions: 1 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T18:00:42.282 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_open_files: -1 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_background_flushes: -1 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Compression algorithms supported: 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kZSTD supported: 0 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kXpressCompression supported: 0 2024-04-26T18:00:42.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kZlibCompression supported: 1 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kSnappyCompression supported: 1 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kLZ4Compression supported: 1 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: kBZip2Compression supported: 0 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.merge_operator: 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_filter: None 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T18:00:42.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d97ec07f40) 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_type: 0 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_index_type: 0 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_shortening: 1 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: checksum: 4 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: no_block_cache: 0 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache: 0x55d97ec0b1f0 2024-04-26T18:00:42.285 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_options: 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: capacity : 536870912 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_shard_bits : 4 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: strict_capacity_limit : 0 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_cache_compressed: (nil) 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: persistent_cache: (nil) 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size: 4096 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_size_deviation: 10 2024-04-26T18:00:42.286 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_restart_interval: 16 2024-04-26T18:00:42.287 INFO:journalctl@ceph.mon.a.smithi090.stdout: index_block_restart_interval: 1 2024-04-26T18:00:42.287 INFO:journalctl@ceph.mon.a.smithi090.stdout: metadata_block_size: 4096 2024-04-26T18:00:42.287 INFO:journalctl@ceph.mon.a.smithi090.stdout: partition_filters: 0 2024-04-26T18:00:42.287 INFO:journalctl@ceph.mon.a.smithi090.stdout: use_delta_encoding: 1 2024-04-26T18:00:42.287 INFO:journalctl@ceph.mon.a.smithi090.stdout: filter_policy: bloomfilter 2024-04-26T18:00:42.287 INFO:journalctl@ceph.mon.a.smithi090.stdout: whole_key_filtering: 1 2024-04-26T18:00:42.287 INFO:journalctl@ceph.mon.a.smithi090.stdout: verify_compression: 0 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: format_version: 5 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_index_compression: 1 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: block_align: 0 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: max_auto_readahead_size: 262144 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: prepopulate_block_cache: 0 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: initial_auto_readahead_size: 8192 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression: NoCompression 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T18:00:42.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.num_levels: 7 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T18:00:42.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T18:00:42.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T18:00:42.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T18:00:42.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.inplace_update_support: 0 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T18:00:42.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.bloom_locality: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.max_successive_merges: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.ttl: 2592000 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.enable_blob_files: false 2024-04-26T18:00:42.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.min_blob_size: 0 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e2121df9-dc3a-41ad-8c9b-3d209d3521d3 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714154442229352, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T18:00:42.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714154442230366, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100554, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98439, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14264, "raw_average_key_size": 49, "raw_value_size": 90693, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714154442, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e2121df9-dc3a-41ad-8c9b-3d209d3521d3", "db_session_id": "RVKT645YOE5JTL7H9RXX", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714154442230427, "job": 1, "event": "recovery_finished"} 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d97ed28000 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: DB pointer 0x55d97ed14000 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** DB Stats ** 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T18:00:42.296 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: L0 2/0 100.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: Sum 2/0 100.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 148.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:00:42.297 INFO:journalctl@ceph.mon.a.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T18:00:42.298 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T18:00:42.298 INFO:journalctl@ceph.mon.a.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 148.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:00:42.298 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-26T18:00:42.298 INFO:journalctl@ceph.mon.a.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T18:00:42.298 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-26T18:00:42.298 INFO:journalctl@ceph.mon.a.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:00:42.298 INFO:journalctl@ceph.mon.a.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T18:00:42.298 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: Cumulative compaction: 0.00 GB write, 32.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: Interval compaction: 0.00 GB write, 32.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache BinnedLRUCache@0x55d97ec0b1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: starting mon.a rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???) e1 preinit fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???).mds e1 new map 2024-04-26T18:00:42.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???).mds e1 print_map 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout: e1 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout: legacy client fscid: -1 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout: 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout: No filesystems configured 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 bash[29161]: a34a7841c5bb55300d1da9594e811e5a6515ae798bc7355afaa9670db971449b 2024-04-26T18:00:42.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 systemd[1]: Started Ceph mon.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: monmap epoch 1 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: last_changed 2024-04-26T18:00:32.950419+0000 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: min_mon_release 19 (squid) 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: election_strategy: 1 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:00:42.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: fsmap 2024-04-26T18:00:42.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T18:00:42.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:42 smithi090 ceph-mon[29180]: mgrmap e1: no daemons active 2024-04-26T18:00:43.371 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.761+0000 7f3f6fa58700 1 Processor -- start 2024-04-26T18:00:43.371 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.761+0000 7f3f6fa58700 1 -- start start 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.761+0000 7f3f6fa58700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f68109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.761+0000 7f3f6fa58700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f68109a40 con 0x7f3f68109050 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.761+0000 7f3f6d7f4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f68109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.762+0000 7f3f6d7f4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f68109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35292/0 (socket says 172.21.15.90:35292) 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.762+0000 7f3f6d7f4700 1 -- 172.21.15.90:0/2902041644 learned_addr learned my addr 172.21.15.90:0/2902041644 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.762+0000 7f3f6d7f4700 1 -- 172.21.15.90:0/2902041644 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6810a250 con 0x7f3f68109050 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.762+0000 7f3f6d7f4700 1 --2- 172.21.15.90:0/2902041644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f68109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f640098a0 tx=0x7f3f64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f25ff9172f8068 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.762+0000 7f3f5ffff700 1 -- 172.21.15.90:0/2902041644 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f64004160 con 0x7f3f68109050 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.762+0000 7f3f5ffff700 1 -- 172.21.15.90:0/2902041644 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f64004300 con 0x7f3f68109050 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.762+0000 7f3f5ffff700 1 -- 172.21.15.90:0/2902041644 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f64011af0 con 0x7f3f68109050 2024-04-26T18:00:43.372 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.763+0000 7f3f6fa58700 1 -- 172.21.15.90:0/2902041644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 msgr2=0x7f3f68109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.763+0000 7f3f6fa58700 1 --2- 172.21.15.90:0/2902041644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f68109470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3f640098a0 tx=0x7f3f64009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.763+0000 7f3f6fa58700 1 -- 172.21.15.90:0/2902041644 shutdown_connections 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.763+0000 7f3f6fa58700 1 --2- 172.21.15.90:0/2902041644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f68109470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.763+0000 7f3f6fa58700 1 -- 172.21.15.90:0/2902041644 >> 172.21.15.90:0/2902041644 conn(0x7f3f68104610 msgr2=0x7f3f68106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.763+0000 7f3f6fa58700 1 -- 172.21.15.90:0/2902041644 shutdown_connections 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.763+0000 7f3f6fa58700 1 -- 172.21.15.90:0/2902041644 wait complete. 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6fa58700 1 Processor -- start 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6fa58700 1 -- start start 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6fa58700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f6807eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6fa58700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6810d480 con 0x7f3f68109050 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6d7f4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f6807eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6d7f4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f6807eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35300/0 (socket says 172.21.15.90:35300) 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6d7f4700 1 -- 172.21.15.90:0/367666283 learned_addr learned my addr 172.21.15.90:0/367666283 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:43.373 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6d7f4700 1 -- 172.21.15.90:0/367666283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6807f080 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6d7f4700 1 --2- 172.21.15.90:0/367666283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f6807eb40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3f640117b0 tx=0x7f3f64011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f5e7fc700 1 -- 172.21.15.90:0/367666283 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f64003af0 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f5e7fc700 1 -- 172.21.15.90:0/367666283 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3f64020070 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f6807d270 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.764+0000 7f3f5e7fc700 1 -- 172.21.15.90:0/367666283 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f64019630 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.765+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6807d7b0 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.765+0000 7f3f5e7fc700 1 -- 172.21.15.90:0/367666283 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3f64019c30 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.765+0000 7f3f5e7fc700 1 -- 172.21.15.90:0/367666283 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3f6400b720 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.765+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f68064a80 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.768+0000 7f3f5e7fc700 1 -- 172.21.15.90:0/367666283 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3f640042a0 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.834+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f3f6807dce0 con 0x7f3f68109050 2024-04-26T18:00:43.374 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.836+0000 7f3f5e7fc700 1 -- 172.21.15.90:0/367666283 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f3f6401a9c0 con 0x7f3f68109050 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.837+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 msgr2=0x7f3f6807eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.837+0000 7f3f6fa58700 1 --2- 172.21.15.90:0/367666283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f6807eb40 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3f640117b0 tx=0x7f3f64011b70 comp rx=0 tx=0).stop 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.837+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 shutdown_connections 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.837+0000 7f3f6fa58700 1 --2- 172.21.15.90:0/367666283 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3f68109050 0x7f3f6807eb40 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.837+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 >> 172.21.15.90:0/367666283 conn(0x7f3f68104610 msgr2=0x7f3f68105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.837+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 shutdown_connections 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:42.837+0000 7f3f6fa58700 1 -- 172.21.15.90:0/367666283 wait complete. 2024-04-26T18:00:43.375 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T18:00:43.389 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:00:43.389 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-04-26T18:00:43.390 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T18:00:43.390 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T18:00:43.635 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mgr.x 2024-04-26T18:00:43.635 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T18:00:43.775 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b8878a18-03f6-11ef-bc93-c7b262605968.target.wants/ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b8878a18-03f6-11ef-bc93-c7b262605968@.service. 2024-04-26T18:00:44.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:43 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/367666283' entity='client.admin' 2024-04-26T18:00:44.259 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:00:44.259 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T18:00:44.266 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:00:44.266 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T18:00:44.266 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T18:00:44.267 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T18:00:44.274 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:00:44.274 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T18:00:44.281 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:00:44.282 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T18:00:44.282 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T18:00:44.282 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T18:00:44.282 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-04-26T18:00:44.282 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-04-26T18:00:44.335 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:44 smithi090 systemd[1]: Started Ceph mgr.x for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:00:44.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:44 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:44.335+0000 7f2c8014c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:00:44.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:44 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:44.522+0000 7f2c8014c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:00:44.948 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:44 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2974842655' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:00:44.949 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:44 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:44.626+0000 7f2c8014c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:00:45.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:44 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:44.948+0000 7f2c8014c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:00:45.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:45 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:45.034+0000 7f2c8014c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:00:45.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:45.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:00:45.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:00:45.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-26T18:00:45.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:00:45.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:45.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:00:45.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:00:45.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:00:45.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:00:45.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:00:45.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:00:45.412 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:00:45.412 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:00:45.412 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:00:45.412 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:45.412 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:00:45.413 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:00:36.466845+0000", 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.733+0000 7fa77ec9a700 1 Processor -- start 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.733+0000 7fa77ec9a700 1 -- start start 2024-04-26T18:00:45.414 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.734+0000 7fa77ec9a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa778108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.734+0000 7fa77ec9a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa778108870 con 0x7fa778107ef0 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.734+0000 7fa77ca36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa778108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.734+0000 7fa77ca36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa778108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35326/0 (socket says 172.21.15.90:35326) 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.734+0000 7fa77ca36700 1 -- 172.21.15.90:0/769558523 learned_addr learned my addr 172.21.15.90:0/769558523 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.734+0000 7fa77ca36700 1 -- 172.21.15.90:0/769558523 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7781090d0 con 0x7fa778107ef0 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.734+0000 7fa77ca36700 1 --2- 172.21.15.90:0/769558523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa778108330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098a0 tx=0x7fa76c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c745abd1c1133bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa7777fe700 1 -- 172.21.15.90:0/769558523 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa76c004160 con 0x7fa778107ef0 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa7777fe700 1 -- 172.21.15.90:0/769558523 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa76c004300 con 0x7fa778107ef0 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa7777fe700 1 -- 172.21.15.90:0/769558523 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa76c011af0 con 0x7fa778107ef0 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa77ec9a700 1 -- 172.21.15.90:0/769558523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 msgr2=0x7fa778108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa77ec9a700 1 --2- 172.21.15.90:0/769558523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa778108330 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa76c0098a0 tx=0x7fa76c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:45.415 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa77ec9a700 1 -- 172.21.15.90:0/769558523 shutdown_connections 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa77ec9a700 1 --2- 172.21.15.90:0/769558523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa778108330 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa77ec9a700 1 -- 172.21.15.90:0/769558523 >> 172.21.15.90:0/769558523 conn(0x7fa778103500 msgr2=0x7fa778105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.735+0000 7fa77ec9a700 1 -- 172.21.15.90:0/769558523 shutdown_connections 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ec9a700 1 -- 172.21.15.90:0/769558523 wait complete. 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ec9a700 1 Processor -- start 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ec9a700 1 -- start start 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ec9a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa77819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ec9a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa778109ef0 con 0x7fa778107ef0 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ca36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa77819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ca36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa77819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35336/0 (socket says 172.21.15.90:35336) 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.736+0000 7fa77ca36700 1 -- 172.21.15.90:0/2974842655 learned_addr learned my addr 172.21.15.90:0/2974842655 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:45.416 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa77ca36700 1 -- 172.21.15.90:0/2974842655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77819d460 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa77ca36700 1 --2- 172.21.15.90:0/2974842655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa77819cf20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009fb0 tx=0x7fa76c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa775ffb700 1 -- 172.21.15.90:0/2974842655 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa76c0036a0 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa775ffb700 1 -- 172.21.15.90:0/2974842655 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa76c01a430 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa775ffb700 1 -- 172.21.15.90:0/2974842655 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa76c019550 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa77819d6f0 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781a03e0 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa775ffb700 1 -- 172.21.15.90:0/2974842655 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa76c019b90 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.737+0000 7fa775ffb700 1 -- 172.21.15.90:0/2974842655 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa76c024730 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.738+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa778064a80 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.740+0000 7fa775ffb700 1 -- 172.21.15.90:0/2974842655 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa76c043e50 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.814+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa77802e690 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.814+0000 7fa775ffb700 1 -- 172.21.15.90:0/2974842655 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa76c032020 con 0x7fa778107ef0 2024-04-26T18:00:45.417 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.815+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 msgr2=0x7fa77819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:45.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.815+0000 7fa77ec9a700 1 --2- 172.21.15.90:0/2974842655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa77819cf20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa76c009fb0 tx=0x7fa76c011940 comp rx=0 tx=0).stop 2024-04-26T18:00:45.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.815+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 shutdown_connections 2024-04-26T18:00:45.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.815+0000 7fa77ec9a700 1 --2- 172.21.15.90:0/2974842655 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa778107ef0 0x7fa77819cf20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:45.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.815+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 >> 172.21.15.90:0/2974842655 conn(0x7fa778103500 msgr2=0x7fa778104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:45.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.815+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 shutdown_connections 2024-04-26T18:00:45.418 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:44.815+0000 7fa77ec9a700 1 -- 172.21.15.90:0/2974842655 wait complete. 2024-04-26T18:00:45.418 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-04-26T18:00:46.575 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:46.239+0000 7f2c8014c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:00:46.575 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:46.407+0000 7f2c8014c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:00:46.575 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:46.575+0000 7f2c8014c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:00:47.125 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:46.860+0000 7f2c8014c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:00:47.125 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:46.951+0000 7f2c8014c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:00:47.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:47 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:47.153+0000 7f2c8014c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:00:47.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:47 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:47.343+0000 7f2c8014c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:00:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:47 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2315344133' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:00:48.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:47 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:47.964+0000 7f2c8014c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:00:48.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:00:48.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:00:48.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:00:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:00:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:00:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:00:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:00:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:00:48.503 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:48.504 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:00:36.466845+0000", 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:48.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.816+0000 7f7b40956700 1 Processor -- start 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.816+0000 7f7b40956700 1 -- start start 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b40956700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c10be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b40956700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c07ce80 con 0x7f7b3c109a80 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c10be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c10be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35342/0 (socket says 172.21.15.90:35342) 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3a59c700 1 -- 172.21.15.90:0/570173364 learned_addr learned my addr 172.21.15.90:0/570173364 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3a59c700 1 -- 172.21.15.90:0/570173364 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c10c3b0 con 0x7f7b3c109a80 2024-04-26T18:00:48.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3a59c700 1 --2- 172.21.15.90:0/570173364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c10be70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0098a0 tx=0x7f7b2c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a878bb757ad60a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3959a700 1 -- 172.21.15.90:0/570173364 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c004160 con 0x7f7b3c109a80 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3959a700 1 -- 172.21.15.90:0/570173364 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b2c004300 con 0x7f7b3c109a80 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.817+0000 7f7b3959a700 1 -- 172.21.15.90:0/570173364 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c011af0 con 0x7f7b3c109a80 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.818+0000 7f7b40956700 1 -- 172.21.15.90:0/570173364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 msgr2=0x7f7b3c10be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.818+0000 7f7b40956700 1 --2- 172.21.15.90:0/570173364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c10be70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c0098a0 tx=0x7f7b2c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.818+0000 7f7b40956700 1 -- 172.21.15.90:0/570173364 shutdown_connections 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.818+0000 7f7b40956700 1 --2- 172.21.15.90:0/570173364 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c10be70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.818+0000 7f7b40956700 1 -- 172.21.15.90:0/570173364 >> 172.21.15.90:0/570173364 conn(0x7f7b3c1034f0 msgr2=0x7f7b3c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.818+0000 7f7b40956700 1 -- 172.21.15.90:0/570173364 shutdown_connections 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.818+0000 7f7b40956700 1 -- 172.21.15.90:0/570173364 wait complete. 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.819+0000 7f7b40956700 1 Processor -- start 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.819+0000 7f7b40956700 1 -- start start 2024-04-26T18:00:48.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.819+0000 7f7b40956700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.819+0000 7f7b40956700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3c10d2a0 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b3a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b3a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35354/0 (socket says 172.21.15.90:35354) 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b3a59c700 1 -- 172.21.15.90:0/2315344133 learned_addr learned my addr 172.21.15.90:0/2315344133 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b3a59c700 1 -- 172.21.15.90:0/2315344133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b3c19d370 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b3a59c700 1 --2- 172.21.15.90:0/2315344133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c19ce30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c003f40 tx=0x7f7b2c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b2b7fe700 1 -- 172.21.15.90:0/2315344133 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c0038b0 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b2b7fe700 1 -- 172.21.15.90:0/2315344133 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7b2c003a50 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b3c19d600 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b2b7fe700 1 -- 172.21.15.90:0/2315344133 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b2c01a5c0 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b3c1a02f0 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b2b7fe700 1 -- 172.21.15.90:0/2315344133 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b2c01ac00 con 0x7f7b3c109a80 2024-04-26T18:00:48.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.820+0000 7f7b2b7fe700 1 -- 172.21.15.90:0/2315344133 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7b2c019700 con 0x7f7b3c109a80 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.821+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b3c064a80 con 0x7f7b3c109a80 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.823+0000 7f7b2b7fe700 1 -- 172.21.15.90:0/2315344133 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7b2c0199a0 con 0x7f7b3c109a80 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.899+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f7b3c1a04c0 con 0x7f7b3c109a80 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.899+0000 7f7b2b7fe700 1 -- 172.21.15.90:0/2315344133 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7b2c019b80 con 0x7f7b3c109a80 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.900+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 msgr2=0x7f7b3c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.900+0000 7f7b40956700 1 --2- 172.21.15.90:0/2315344133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c19ce30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c003f40 tx=0x7f7b2c011890 comp rx=0 tx=0).stop 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.900+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 shutdown_connections 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.900+0000 7f7b40956700 1 --2- 172.21.15.90:0/2315344133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c109a80 0x7f7b3c19ce30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.900+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 >> 172.21.15.90:0/2315344133 conn(0x7f7b3c1034f0 msgr2=0x7f7b3c1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.900+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 shutdown_connections 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:47.900+0000 7f7b40956700 1 -- 172.21.15.90:0/2315344133 wait complete. 2024-04-26T18:00:48.509 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-04-26T18:00:48.685 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:48 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:48.387+0000 7f2c8014c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:00:48.685 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:48 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:48.485+0000 7f2c8014c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:00:48.685 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:48 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:48.685+0000 7f2c8014c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:00:49.541 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:49.210+0000 7f2c8014c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:00:49.542 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:49.301+0000 7f2c8014c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:00:49.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:49.541+0000 7f2c8014c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:00:49.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:49.673+0000 7f2c8014c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:00:49.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:49.767+0000 7f2c8014c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:00:50.184 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:49.883+0000 7f2c8014c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:00:50.184 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:50.184+0000 7f2c8014c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:00:50.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:50.791+0000 7f2c8014c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:00:51.263 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:51 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1742738189' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:00:51.263 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:51 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:51.014+0000 7f2c8014c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:00:51.594 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:51.595 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:00:51.596 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:00:51.597 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:00:51.598 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:00:36.466845+0000", 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:00:51.599 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.883+0000 7f0983cae700 1 Processor -- start 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.883+0000 7f0983cae700 1 -- start start 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.883+0000 7f0983cae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c07d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.883+0000 7f0983cae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f097c07d7f0 con 0x7f097c07ee00 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0981a4a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c07d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0981a4a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c07d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35358/0 (socket says 172.21.15.90:35358) 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0981a4a700 1 -- 172.21.15.90:0/889321555 learned_addr learned my addr 172.21.15.90:0/889321555 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0981a4a700 1 -- 172.21.15.90:0/889321555 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c07d970 con 0x7f097c07ee00 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0981a4a700 1 --2- 172.21.15.90:0/889321555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c07d2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f096c0098a0 tx=0x7f096c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0be0a5e363fcb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:51.600 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0980a48700 1 -- 172.21.15.90:0/889321555 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f096c004160 con 0x7f097c07ee00 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0980a48700 1 -- 172.21.15.90:0/889321555 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f096c004300 con 0x7f097c07ee00 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.884+0000 7f0980a48700 1 -- 172.21.15.90:0/889321555 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f096c011af0 con 0x7f097c07ee00 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.885+0000 7f0983cae700 1 -- 172.21.15.90:0/889321555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 msgr2=0x7f097c07d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.885+0000 7f0983cae700 1 --2- 172.21.15.90:0/889321555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c07d2b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f096c0098a0 tx=0x7f096c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.885+0000 7f0983cae700 1 -- 172.21.15.90:0/889321555 shutdown_connections 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.885+0000 7f0983cae700 1 --2- 172.21.15.90:0/889321555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c07d2b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.885+0000 7f0983cae700 1 -- 172.21.15.90:0/889321555 >> 172.21.15.90:0/889321555 conn(0x7f097c104670 msgr2=0x7f097c106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.885+0000 7f0983cae700 1 -- 172.21.15.90:0/889321555 shutdown_connections 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.885+0000 7f0983cae700 1 -- 172.21.15.90:0/889321555 wait complete. 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0983cae700 1 Processor -- start 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0983cae700 1 -- start start 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0983cae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:51.601 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0983cae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f097c10f4b0 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0981a4a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0981a4a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c19f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35362/0 (socket says 172.21.15.90:35362) 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0981a4a700 1 -- 172.21.15.90:0/1742738189 learned_addr learned my addr 172.21.15.90:0/1742738189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0981a4a700 1 -- 172.21.15.90:0/1742738189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097c19f5b0 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.886+0000 7f0981a4a700 1 --2- 172.21.15.90:0/1742738189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c19f070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f096c011ac0 tx=0x7f096c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.887+0000 7f0972ffd700 1 -- 172.21.15.90:0/1742738189 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f096c003b20 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.887+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097c19f840 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.887+0000 7f0972ffd700 1 -- 172.21.15.90:0/1742738189 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f096c01a8c0 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.887+0000 7f0972ffd700 1 -- 172.21.15.90:0/1742738189 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f096c0248d0 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.887+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097c1a2530 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.887+0000 7f0972ffd700 1 -- 172.21.15.90:0/1742738189 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f096c022020 con 0x7f097c07ee00 2024-04-26T18:00:51.602 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.887+0000 7f0972ffd700 1 -- 172.21.15.90:0/1742738189 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f096c0193c0 con 0x7f097c07ee00 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.888+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097c064a80 con 0x7f097c07ee00 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.890+0000 7f0972ffd700 1 -- 172.21.15.90:0/1742738189 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f096c02ddf0 con 0x7f097c07ee00 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.962+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f097c1a2a80 con 0x7f097c07ee00 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.963+0000 7f0972ffd700 1 -- 172.21.15.90:0/1742738189 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f096c01aa70 con 0x7f097c07ee00 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.964+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 msgr2=0x7f097c19f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.964+0000 7f0983cae700 1 --2- 172.21.15.90:0/1742738189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c19f070 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f096c011ac0 tx=0x7f096c011ba0 comp rx=0 tx=0).stop 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.964+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 shutdown_connections 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.964+0000 7f0983cae700 1 --2- 172.21.15.90:0/1742738189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f097c07ee00 0x7f097c19f070 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.964+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 >> 172.21.15.90:0/1742738189 conn(0x7f097c104670 msgr2=0x7f097c105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.964+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 shutdown_connections 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:50.964+0000 7f0983cae700 1 -- 172.21.15.90:0/1742738189 wait complete. 2024-04-26T18:00:51.603 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (3/15)... 2024-04-26T18:00:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: Activating manager daemon x 2024-04-26T18:00:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: mgrmap e2: x(active, starting, since 0.0032059s) 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: Manager daemon x is now available 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' 2024-04-26T18:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:52 smithi090 ceph-mon[29180]: from='mgr.14100 172.21.15.90:0/1221638261' entity='mgr.x' 2024-04-26T18:00:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:53 smithi090 ceph-mon[29180]: mgrmap e3: x(active, since 1.00817s) 2024-04-26T18:00:54.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:54 smithi090 ceph-mon[29180]: mgrmap e4: x(active, since 2s) 2024-04-26T18:00:54.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:54.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:00:54.905 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:54.906 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:54.907 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:00:54.908 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:54.909 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:00:54.910 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:00:36.466845+0000", 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.025+0000 7fc3f6fb1700 1 Processor -- start 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.025+0000 7fc3f6fb1700 1 -- start start 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.025+0000 7fc3f6fb1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f010be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.026+0000 7fc3f6fb1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f007ce80 con 0x7fc3f0109a80 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.026+0000 7fc3f4d4d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f010be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:54.911 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.026+0000 7fc3f4d4d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f010be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58250/0 (socket says 172.21.15.90:58250) 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.026+0000 7fc3f4d4d700 1 -- 172.21.15.90:0/1438532747 learned_addr learned my addr 172.21.15.90:0/1438532747 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.026+0000 7fc3f4d4d700 1 -- 172.21.15.90:0/1438532747 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f010c3b0 con 0x7fc3f0109a80 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.026+0000 7fc3f4d4d700 1 --2- 172.21.15.90:0/1438532747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f010be70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40098a0 tx=0x7fc3e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175217e83ff73c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.027+0000 7fc3ef7fe700 1 -- 172.21.15.90:0/1438532747 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e4004160 con 0x7fc3f0109a80 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.027+0000 7fc3ef7fe700 1 -- 172.21.15.90:0/1438532747 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc3e4004300 con 0x7fc3f0109a80 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.027+0000 7fc3ef7fe700 1 -- 172.21.15.90:0/1438532747 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e4011af0 con 0x7fc3f0109a80 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.028+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/1438532747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 msgr2=0x7fc3f010be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.028+0000 7fc3f6fb1700 1 --2- 172.21.15.90:0/1438532747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f010be70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc3e40098a0 tx=0x7fc3e4009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.028+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/1438532747 shutdown_connections 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.028+0000 7fc3f6fb1700 1 --2- 172.21.15.90:0/1438532747 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f010be70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:54.912 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.028+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/1438532747 >> 172.21.15.90:0/1438532747 conn(0x7fc3f01034f0 msgr2=0x7fc3f0105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.028+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/1438532747 shutdown_connections 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.028+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/1438532747 wait complete. 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f6fb1700 1 Processor -- start 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f6fb1700 1 -- start start 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f6fb1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f019ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f6fb1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f010d2a0 con 0x7fc3f0109a80 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f4d4d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f019ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f4d4d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f019ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58266/0 (socket says 172.21.15.90:58266) 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f4d4d700 1 -- 172.21.15.90:0/2639231627 learned_addr learned my addr 172.21.15.90:0/2639231627 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f4d4d700 1 -- 172.21.15.90:0/2639231627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f019d370 con 0x7fc3f0109a80 2024-04-26T18:00:54.913 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.029+0000 7fc3f4d4d700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f019ce30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4011ac0 tx=0x7fc3e4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.030+0000 7fc3edffb700 1 -- 172.21.15.90:0/2639231627 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e4003b20 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.030+0000 7fc3edffb700 1 -- 172.21.15.90:0/2639231627 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc3e401a8c0 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.030+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f019d600 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.030+0000 7fc3edffb700 1 -- 172.21.15.90:0/2639231627 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc3e40248d0 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.030+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f01a02f0 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.031+0000 7fc3edffb700 1 -- 172.21.15.90:0/2639231627 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fc3e4022020 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.031+0000 7fc3edffb700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fc3e00395b0 0x7fc3e003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.031+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f0064a80 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.031+0000 7fc3edffb700 1 -- 172.21.15.90:0/2639231627 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc3e404d4b0 con 0x7fc3f0109a80 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.031+0000 7fc3effff700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fc3e00395b0 0x7fc3e003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:54.914 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.031+0000 7fc3effff700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fc3e00395b0 0x7fc3e003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc006fd0 tx=0x7fc3dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.036+0000 7fc3edffb700 1 -- 172.21.15.90:0/2639231627 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3e402cdf0 con 0x7fc3f0109a80 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.313+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc3f01a04c0 con 0x7fc3f0109a80 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.314+0000 7fc3edffb700 1 -- 172.21.15.90:0/2639231627 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fc3e400b5a0 con 0x7fc3f0109a80 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fc3e00395b0 msgr2=0x7fc3e003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fc3e00395b0 0x7fc3e003ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc006fd0 tx=0x7fc3dc006e40 comp rx=0 tx=0).stop 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 msgr2=0x7fc3f019ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f019ce30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4011ac0 tx=0x7fc3e4011ba0 comp rx=0 tx=0).stop 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 shutdown_connections 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fc3e00395b0 0x7fc3e003ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 --2- 172.21.15.90:0/2639231627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3f0109a80 0x7fc3f019ce30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.317+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 >> 172.21.15.90:0/2639231627 conn(0x7fc3f01034f0 msgr2=0x7fc3f01040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:54.915 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.318+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 shutdown_connections 2024-04-26T18:00:54.916 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:54.318+0000 7fc3f6fb1700 1 -- 172.21.15.90:0/2639231627 wait complete. 2024-04-26T18:00:54.916 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-04-26T18:00:55.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:55 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2639231627' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:00:56.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:56 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3322876729' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T18:00:56.134 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.308+0000 7faa95564700 1 Processor -- start 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.308+0000 7faa95564700 1 -- start start 2024-04-26T18:00:56.135 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.308+0000 7faa95564700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa90109490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.308+0000 7faa95564700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa90109a60 con 0x7faa90109070 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa90109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa90109490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58276/0 (socket says 172.21.15.90:58276) 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8effd700 1 -- 172.21.15.90:0/2907640189 learned_addr learned my addr 172.21.15.90:0/2907640189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8effd700 1 -- 172.21.15.90:0/2907640189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9010a220 con 0x7faa90109070 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8effd700 1 --2- 172.21.15.90:0/2907640189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa90109490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa840098a0 tx=0x7faa84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999df4c03bebb78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8dffb700 1 -- 172.21.15.90:0/2907640189 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa84004160 con 0x7faa90109070 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8dffb700 1 -- 172.21.15.90:0/2907640189 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faa84004300 con 0x7faa90109070 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.309+0000 7faa8dffb700 1 -- 172.21.15.90:0/2907640189 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa84011af0 con 0x7faa90109070 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 -- 172.21.15.90:0/2907640189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 msgr2=0x7faa90109490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 --2- 172.21.15.90:0/2907640189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa90109490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7faa840098a0 tx=0x7faa84009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 -- 172.21.15.90:0/2907640189 shutdown_connections 2024-04-26T18:00:56.136 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 --2- 172.21.15.90:0/2907640189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa90109490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 -- 172.21.15.90:0/2907640189 >> 172.21.15.90:0/2907640189 conn(0x7faa90104670 msgr2=0x7faa90106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 -- 172.21.15.90:0/2907640189 shutdown_connections 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 -- 172.21.15.90:0/2907640189 wait complete. 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.310+0000 7faa95564700 1 Processor -- start 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa95564700 1 -- start start 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa95564700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa9019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa95564700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faa9010d450 con 0x7faa90109070 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa8effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa9019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa8effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa9019cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58278/0 (socket says 172.21.15.90:58278) 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa8effd700 1 -- 172.21.15.90:0/3322876729 learned_addr learned my addr 172.21.15.90:0/3322876729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa8effd700 1 -- 172.21.15.90:0/3322876729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa9019d400 con 0x7faa90109070 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa8effd700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa9019cec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faa84003f40 tx=0x7faa84011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:56.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa77fff700 1 -- 172.21.15.90:0/3322876729 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa840038b0 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa9019d690 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa77fff700 1 -- 172.21.15.90:0/3322876729 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faa84003a50 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa901a0380 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.311+0000 7faa77fff700 1 -- 172.21.15.90:0/3322876729 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa8401a5c0 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.312+0000 7faa77fff700 1 -- 172.21.15.90:0/3322876729 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7faa8401a870 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.312+0000 7faa77fff700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7faa78039560 0x7faa7803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.312+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa7c005360 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.312+0000 7faa77fff700 1 -- 172.21.15.90:0/3322876729 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faa8404db00 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.312+0000 7faa8e7fc700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7faa78039560 0x7faa7803ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.313+0000 7faa8e7fc700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7faa78039560 0x7faa7803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa80006fd0 tx=0x7faa80006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.318+0000 7faa77fff700 1 -- 172.21.15.90:0/3322876729 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa84019c70 con 0x7faa90109070 2024-04-26T18:00:56.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.513+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7faa7c002430 con 0x7faa90109070 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.513+0000 7faa77fff700 1 -- 172.21.15.90:0/3322876729 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7faa8402cbc0 con 0x7faa90109070 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7faa78039560 msgr2=0x7faa7803ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7faa78039560 0x7faa7803ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faa80006fd0 tx=0x7faa80006e40 comp rx=0 tx=0).stop 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 msgr2=0x7faa9019cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa9019cec0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faa84003f40 tx=0x7faa84011890 comp rx=0 tx=0).stop 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 shutdown_connections 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7faa78039560 0x7faa7803ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 --2- 172.21.15.90:0/3322876729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faa90109070 0x7faa9019cec0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 >> 172.21.15.90:0/3322876729 conn(0x7faa90104670 msgr2=0x7faa901051b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 shutdown_connections 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:55.516+0000 7faa95564700 1 -- 172.21.15.90:0/3322876729 wait complete. 2024-04-26T18:00:56.139 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-04-26T18:00:57.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/806091809' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T18:00:57.341 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: ignoring --setuser ceph since I am not root 2024-04-26T18:00:57.341 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: ignoring --setgroup ceph since I am not root 2024-04-26T18:00:57.341 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:57.201+0000 7fd1a7d9c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:00:57.600 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:57.384+0000 7fd1a7d9c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:00:57.600 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:57.485+0000 7fd1a7d9c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:00:57.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f8f40700 1 Processor -- start 2024-04-26T18:00:57.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f8f40700 1 -- start start 2024-04-26T18:00:57.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f8f40700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f4109470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:57.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f8f40700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f50f4109a40 con 0x7f50f4109050 2024-04-26T18:00:57.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f4109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:57.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f4109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58284/0 (socket says 172.21.15.90:58284) 2024-04-26T18:00:57.608 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f259c700 1 -- 172.21.15.90:0/723837457 learned_addr learned my addr 172.21.15.90:0/723837457 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.555+0000 7f50f259c700 1 -- 172.21.15.90:0/723837457 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f410a250 con 0x7f50f4109050 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f259c700 1 --2- 172.21.15.90:0/723837457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f4109470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f50e40098a0 tx=0x7f50e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c063d3823219ff69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f159a700 1 -- 172.21.15.90:0/723837457 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4004160 con 0x7f50f4109050 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f159a700 1 -- 172.21.15.90:0/723837457 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50e4004300 con 0x7f50f4109050 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f159a700 1 -- 172.21.15.90:0/723837457 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4011af0 con 0x7f50f4109050 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f8f40700 1 -- 172.21.15.90:0/723837457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 msgr2=0x7f50f4109470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f8f40700 1 --2- 172.21.15.90:0/723837457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f4109470 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f50e40098a0 tx=0x7f50e4009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f8f40700 1 -- 172.21.15.90:0/723837457 shutdown_connections 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f8f40700 1 --2- 172.21.15.90:0/723837457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f4109470 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f8f40700 1 -- 172.21.15.90:0/723837457 >> 172.21.15.90:0/723837457 conn(0x7f50f4104610 msgr2=0x7f50f4106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f8f40700 1 -- 172.21.15.90:0/723837457 shutdown_connections 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.556+0000 7f50f8f40700 1 -- 172.21.15.90:0/723837457 wait complete. 2024-04-26T18:00:57.609 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.557+0000 7f50f8f40700 1 Processor -- start 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.557+0000 7f50f8f40700 1 -- start start 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.557+0000 7f50f8f40700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f407eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.557+0000 7f50f8f40700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f50f410d480 con 0x7f50f4109050 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.557+0000 7f50f259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f407eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.557+0000 7f50f259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f407eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58294/0 (socket says 172.21.15.90:58294) 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.557+0000 7f50f259c700 1 -- 172.21.15.90:0/806091809 learned_addr learned my addr 172.21.15.90:0/806091809 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.558+0000 7f50f259c700 1 -- 172.21.15.90:0/806091809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50f407f080 con 0x7f50f4109050 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.558+0000 7f50f259c700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f407eb40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009fb0 tx=0x7f50e4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.558+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e40036a0 con 0x7f50f4109050 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.558+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f50e401a430 con 0x7f50f4109050 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.558+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50f407d270 con 0x7f50f4109050 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.558+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50e4019550 con 0x7f50f4109050 2024-04-26T18:00:57.610 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.558+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50f407d7b0 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.559+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f50e4019850 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.559+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f4064a80 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.559+0000 7f50e37fe700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f50dc039510 0x7f50dc03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.559+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f50e404d8a0 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.559+0000 7f50f1d9b700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f50dc039510 0x7f50dc03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.559+0000 7f50f1d9b700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f50dc039510 0x7f50dc03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50e8006fd0 tx=0x7f50e8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.564+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50e404d2c0 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:56.816+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f50f407da30 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.030+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f50e400fda0 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.032+0000 7f50e37fe700 1 -- 172.21.15.90:0/806091809 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f50e402d960 con 0x7f50f4109050 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f50dc039510 msgr2=0x7f50dc03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:57.611 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f50dc039510 0x7f50dc03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50e8006fd0 tx=0x7f50e8006e40 comp rx=0 tx=0).stop 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 msgr2=0x7f50f407eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f407eb40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009fb0 tx=0x7f50e4011940 comp rx=0 tx=0).stop 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 shutdown_connections 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f50dc039510 0x7f50dc03b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 --2- 172.21.15.90:0/806091809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50f4109050 0x7f50f407eb40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 >> 172.21.15.90:0/806091809 conn(0x7f50f4104610 msgr2=0x7f50f4105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 shutdown_connections 2024-04-26T18:00:57.612 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:57.033+0000 7f50f8f40700 1 -- 172.21.15.90:0/806091809 wait complete. 2024-04-26T18:00:57.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:57.811+0000 7fd1a7d9c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:00:58.225 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:57.898+0000 7fd1a7d9c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:00:58.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:58 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/806091809' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T18:00:58.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:58 smithi090 ceph-mon[29180]: mgrmap e5: x(active, since 6s) 2024-04-26T18:00:58.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.020+0000 7f72ef17c700 1 Processor -- start 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.020+0000 7f72ef17c700 1 -- start start 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.020+0000 7f72ef17c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e8109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.020+0000 7f72ef17c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8109a50 con 0x7f72e8109060 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.020+0000 7f72ecf18700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e8109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.020+0000 7f72ecf18700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e8109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58328/0 (socket says 172.21.15.90:58328) 2024-04-26T18:00:58.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.020+0000 7f72ecf18700 1 -- 172.21.15.90:0/4256035572 learned_addr learned my addr 172.21.15.90:0/4256035572 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.021+0000 7f72ecf18700 1 -- 172.21.15.90:0/4256035572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e810a210 con 0x7f72e8109060 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.021+0000 7f72ecf18700 1 --2- 172.21.15.90:0/4256035572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e8109480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098a0 tx=0x7f72dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213e62d5089c35f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.021+0000 7f72e77fe700 1 -- 172.21.15.90:0/4256035572 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72dc004160 con 0x7f72e8109060 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.021+0000 7f72e77fe700 1 -- 172.21.15.90:0/4256035572 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f72dc004300 con 0x7f72e8109060 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.021+0000 7f72e77fe700 1 -- 172.21.15.90:0/4256035572 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72dc011af0 con 0x7f72e8109060 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 -- 172.21.15.90:0/4256035572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 msgr2=0x7f72e8109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 --2- 172.21.15.90:0/4256035572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e8109480 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f72dc0098a0 tx=0x7f72dc009bb0 comp rx=0 tx=0).stop 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 -- 172.21.15.90:0/4256035572 shutdown_connections 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 --2- 172.21.15.90:0/4256035572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e8109480 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 -- 172.21.15.90:0/4256035572 >> 172.21.15.90:0/4256035572 conn(0x7f72e8104640 msgr2=0x7f72e8106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 -- 172.21.15.90:0/4256035572 shutdown_connections 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 -- 172.21.15.90:0/4256035572 wait complete. 2024-04-26T18:00:58.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 Processor -- start 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 -- start start 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.022+0000 7f72ef17c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f72e810d440 con 0x7f72e8109060 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72ecf18700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72ecf18700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e819ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58342/0 (socket says 172.21.15.90:58342) 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72ecf18700 1 -- 172.21.15.90:0/2835757976 learned_addr learned my addr 172.21.15.90:0/2835757976 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72ecf18700 1 -- 172.21.15.90:0/2835757976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e819d390 con 0x7f72e8109060 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72ecf18700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e819ce50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72e5ffb700 1 -- 172.21.15.90:0/2835757976 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72dc003b20 con 0x7f72e8109060 2024-04-26T18:00:58.896 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72e5ffb700 1 -- 172.21.15.90:0/2835757976 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f72dc01a8c0 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72e5ffb700 1 -- 172.21.15.90:0/2835757976 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72dc024940 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e819d620 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.023+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a0310 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.024+0000 7f72e5ffb700 1 -- 172.21.15.90:0/2835757976 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f72dc022020 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.024+0000 7f72e5ffb700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 0x7f72d803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.024+0000 7f72e5ffb700 1 -- 172.21.15.90:0/2835757976 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f72dc04deb0 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.024+0000 7f72e7fff700 1 -- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 msgr2=0x7f72d803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4035087696 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.024+0000 7f72e7fff700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 0x7f72d803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.024+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72e8064a80 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.034+0000 7f72e5ffb700 1 -- 172.21.15.90:0/2835757976 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72dc01a650 con 0x7f72e8109060 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.225+0000 7f72e7fff700 1 -- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 msgr2=0x7f72d803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4035087696 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.225+0000 7f72e7fff700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 0x7f72d803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:00:58.897 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.314+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f72e81a04e0 con 0x7f72e8109060 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.315+0000 7f72e5ffb700 1 -- 172.21.15.90:0/2835757976 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f72dc00b5a0 con 0x7f72e8109060 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 msgr2=0x7f72d803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 0x7f72d803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 msgr2=0x7f72e819ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e819ce50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc011b70 comp rx=0 tx=0).stop 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 shutdown_connections 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7f72d8039600 0x7f72d803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 --2- 172.21.15.90:0/2835757976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72e8109060 0x7f72e819ce50 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000c00 tx=0x7f72dc011b70 comp rx=0 tx=0).stop 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 >> 172.21.15.90:0/2835757976 conn(0x7f72e8104640 msgr2=0x7f72e81051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 shutdown_connections 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:58.318+0000 7f72ef17c700 1 -- 172.21.15.90:0/2835757976 wait complete. 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-26T18:00:58.898 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 5... 2024-04-26T18:00:59.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:00:59 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2835757976' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T18:00:59.214 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:59 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:59.126+0000 7fd1a7d9c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:00:59.471 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:59 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:59.300+0000 7fd1a7d9c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:00:59.471 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:59 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:59.471+0000 7fd1a7d9c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:01:00.053 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:59 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:59.759+0000 7fd1a7d9c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:01:00.053 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:00:59 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:00:59.851+0000 7fd1a7d9c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:01:00.053 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:00.053+0000 7fd1a7d9c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:01:00.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:00.246+0000 7fd1a7d9c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:01:01.125 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:00.842+0000 7fd1a7d9c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:01:01.566 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:01.263+0000 7fd1a7d9c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:01:01.567 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:01.361+0000 7fd1a7d9c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:01:01.567 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:01.566+0000 7fd1a7d9c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:01:02.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:02.094+0000 7fd1a7d9c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:01:02.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:02.185+0000 7fd1a7d9c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:01:02.768 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:02.428+0000 7fd1a7d9c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:01:02.768 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:02.559+0000 7fd1a7d9c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:01:02.768 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:02.652+0000 7fd1a7d9c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:01:02.768 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:02.768+0000 7fd1a7d9c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:01:03.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:03.068+0000 7fd1a7d9c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:01:03.934 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:03.665+0000 7fd1a7d9c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:01:03.934 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:03.881+0000 7fd1a7d9c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:01:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: Active manager daemon x restarted 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: Activating manager daemon x 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: mgrmap e6: x(active, starting, since 0.00334742s) 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: Manager daemon x is now available 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:03 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:01:05.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:04 smithi090 ceph-mon[29180]: Found migration_current of "None". Setting to last migration. 2024-04-26T18:01:05.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:04 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:01:05.202 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:04 smithi090 ceph-mon[29180]: mgrmap e7: x(active, since 1.0078s) 2024-04-26T18:01:05.505 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.314+0000 7fed76960700 1 Processor -- start 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.314+0000 7fed76960700 1 -- start start 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.314+0000 7fed76960700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed7010beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.314+0000 7fed76960700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fed7007ce40 con 0x7fed70109ac0 2024-04-26T18:01:05.506 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.314+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed7010beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.315+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed7010beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58354/0 (socket says 172.21.15.90:58354) 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.315+0000 7fed6ffff700 1 -- 172.21.15.90:0/2692124533 learned_addr learned my addr 172.21.15.90:0/2692124533 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.315+0000 7fed6ffff700 1 -- 172.21.15.90:0/2692124533 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7010c3f0 con 0x7fed70109ac0 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.315+0000 7fed6ffff700 1 --2- 172.21.15.90:0/2692124533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed7010beb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed600098a0 tx=0x7fed60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf260c89d96da2dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.315+0000 7fed6effd700 1 -- 172.21.15.90:0/2692124533 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed60004160 con 0x7fed70109ac0 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.315+0000 7fed6effd700 1 -- 172.21.15.90:0/2692124533 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fed60004300 con 0x7fed70109ac0 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.315+0000 7fed6effd700 1 -- 172.21.15.90:0/2692124533 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed60011af0 con 0x7fed70109ac0 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 -- 172.21.15.90:0/2692124533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 msgr2=0x7fed7010beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 --2- 172.21.15.90:0/2692124533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed7010beb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fed600098a0 tx=0x7fed60009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 -- 172.21.15.90:0/2692124533 shutdown_connections 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 --2- 172.21.15.90:0/2692124533 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed7010beb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:05.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 -- 172.21.15.90:0/2692124533 >> 172.21.15.90:0/2692124533 conn(0x7fed701034f0 msgr2=0x7fed70105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 -- 172.21.15.90:0/2692124533 shutdown_connections 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 -- 172.21.15.90:0/2692124533 wait complete. 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 Processor -- start 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.316+0000 7fed76960700 1 -- start start 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed76960700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed70109340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed76960700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fed7010d2d0 con 0x7fed70109ac0 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed70109340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed6ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed70109340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58370/0 (socket says 172.21.15.90:58370) 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed6ffff700 1 -- 172.21.15.90:0/4036844918 learned_addr learned my addr 172.21.15.90:0/4036844918 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed6ffff700 1 -- 172.21.15.90:0/4036844918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed70109880 con 0x7fed70109ac0 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed6ffff700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed70109340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fed60003f40 tx=0x7fed60011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.317+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed600038b0 con 0x7fed70109ac0 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.318+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fed60003a50 con 0x7fed70109ac0 2024-04-26T18:01:05.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.318+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed70107a70 con 0x7fed70109ac0 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.318+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed6001a5c0 con 0x7fed70109ac0 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.318+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed70107fb0 con 0x7fed70109ac0 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.318+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fed6001a8c0 con 0x7fed70109ac0 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.319+0000 7fed6d7fa700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.319+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 --> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed7006a580 con 0x7fed58039560 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.319+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fed6004e640 con 0x7fed70109ac0 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.319+0000 7fed6f7fe700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 msgr2=0x7fed5803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4035087696 2024-04-26T18:01:05.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.319+0000 7fed6f7fe700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.520+0000 7fed6f7fe700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 msgr2=0x7fed5803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4035087696 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.520+0000 7fed6f7fe700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.921+0000 7fed6f7fe700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 msgr2=0x7fed5803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4035087696 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:00:59.921+0000 7fed6f7fe700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:00.722+0000 7fed6f7fe700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 msgr2=0x7fed5803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4035087696 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:00.722+0000 7fed6f7fe700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:02.323+0000 7fed6f7fe700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 msgr2=0x7fed5803ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/4035087696 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:02.323+0000 7fed6f7fe700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:03.885+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fed6002d910 con 0x7fed70109ac0 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:03.885+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 msgr2=0x7fed5803ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:03.885+0000 7fed6d7fa700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:05.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.888+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fed6004fc10 con 0x7fed70109ac0 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.888+0000 7fed6d7fa700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fed5803cc60 0x7fed5803f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.888+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fed60016e70 con 0x7fed5803cc60 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.889+0000 7fed6f7fe700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fed5803cc60 0x7fed5803f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.889+0000 7fed6f7fe700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fed5803cc60 0x7fed5803f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed64003a10 tx=0x7fed64009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.890+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fed60016e70 con 0x7fed5803cc60 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.898+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fed701047b0 con 0x7fed5803cc60 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.898+0000 7fed6d7fa700 1 -- 172.21.15.90:0/4036844918 <== mgr.14120 v2:172.21.15.90:6800/1868991041 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fed701047b0 con 0x7fed5803cc60 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fed5803cc60 msgr2=0x7fed5803f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fed5803cc60 0x7fed5803f050 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fed64003a10 tx=0x7fed64009290 comp rx=0 tx=0).stop 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 msgr2=0x7fed70109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed70109340 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fed60003f40 tx=0x7fed60011890 comp rx=0 tx=0).stop 2024-04-26T18:01:05.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 shutdown_connections 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fed5803cc60 0x7fed5803f050 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed70109ac0 0x7fed70109340 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 --2- 172.21.15.90:0/4036844918 >> [v2:172.21.15.90:6800/4035087696,v1:172.21.15.90:6801/4035087696] conn(0x7fed58039560 0x7fed5803ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 >> 172.21.15.90:0/4036844918 conn(0x7fed701034f0 msgr2=0x7fed701040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 shutdown_connections 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:04.899+0000 7fed76960700 1 -- 172.21.15.90:0/4036844918 wait complete. 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 5 is available 2024-04-26T18:01:05.512 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-04-26T18:01:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:05 smithi090 ceph-mon[29180]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T18:01:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:05 smithi090 ceph-mon[29180]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T18:01:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:05 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:05 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:06.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.957+0000 7f5a9a70b700 1 Processor -- start 2024-04-26T18:01:06.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.957+0000 7f5a9a70b700 1 -- start start 2024-04-26T18:01:06.795 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.957+0000 7f5a9a70b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a94109480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.957+0000 7f5a9a70b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a94109a50 con 0x7f5a94109060 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a94109480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a94109480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50226/0 (socket says 172.21.15.90:50226) 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a93fff700 1 -- 172.21.15.90:0/62158092 learned_addr learned my addr 172.21.15.90:0/62158092 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a93fff700 1 -- 172.21.15.90:0/62158092 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9410a210 con 0x7f5a94109060 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a93fff700 1 --2- 172.21.15.90:0/62158092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a94109480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a840098a0 tx=0x7f5a84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89e55abe39f98e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a92ffd700 1 -- 172.21.15.90:0/62158092 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a84004160 con 0x7f5a94109060 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a92ffd700 1 -- 172.21.15.90:0/62158092 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5a84004300 con 0x7f5a94109060 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.958+0000 7f5a92ffd700 1 -- 172.21.15.90:0/62158092 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a84011af0 con 0x7f5a94109060 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.959+0000 7f5a9a70b700 1 -- 172.21.15.90:0/62158092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 msgr2=0x7f5a94109480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.959+0000 7f5a9a70b700 1 --2- 172.21.15.90:0/62158092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a94109480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a840098a0 tx=0x7f5a84009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.959+0000 7f5a9a70b700 1 -- 172.21.15.90:0/62158092 shutdown_connections 2024-04-26T18:01:06.796 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.959+0000 7f5a9a70b700 1 --2- 172.21.15.90:0/62158092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a94109480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.959+0000 7f5a9a70b700 1 -- 172.21.15.90:0/62158092 >> 172.21.15.90:0/62158092 conn(0x7f5a94104640 msgr2=0x7f5a94106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.959+0000 7f5a9a70b700 1 -- 172.21.15.90:0/62158092 shutdown_connections 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.959+0000 7f5a9a70b700 1 -- 172.21.15.90:0/62158092 wait complete. 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a9a70b700 1 Processor -- start 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a9a70b700 1 -- start start 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a9a70b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a9407eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a9a70b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9410d440 con 0x7f5a94109060 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a9407eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a9407eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50242/0 (socket says 172.21.15.90:50242) 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a93fff700 1 -- 172.21.15.90:0/2909114306 learned_addr learned my addr 172.21.15.90:0/2909114306 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.960+0000 7f5a93fff700 1 -- 172.21.15.90:0/2909114306 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9407f080 con 0x7f5a94109060 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.961+0000 7f5a93fff700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a9407eb40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a84011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.961+0000 7f5a917fa700 1 -- 172.21.15.90:0/2909114306 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a84003b20 con 0x7f5a94109060 2024-04-26T18:01:06.797 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.961+0000 7f5a917fa700 1 -- 172.21.15.90:0/2909114306 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5a8401a8c0 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.961+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9407d270 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.961+0000 7f5a917fa700 1 -- 172.21.15.90:0/2909114306 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a84024940 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.961+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a9407d7b0 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.962+0000 7f5a917fa700 1 -- 172.21.15.90:0/2909114306 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f5a84022020 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.962+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a94064a80 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.962+0000 7f5a917fa700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5a7c0394e0 0x7f5a7c03b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.962+0000 7f5a917fa700 1 -- 172.21.15.90:0/2909114306 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5a8404d600 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.962+0000 7f5a937fe700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5a7c0394e0 0x7f5a7c03b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.962+0000 7f5a937fe700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5a7c0394e0 0x7f5a7c03b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a88006fd0 tx=0x7f5a88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:05.967+0000 7f5a917fa700 1 -- 172.21.15.90:0/2909114306 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a84021d80 con 0x7f5a94109060 2024-04-26T18:01:06.798 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.185+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f5a94105860 con 0x7f5a7c0394e0 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.193+0000 7f5a917fa700 1 -- 172.21.15.90:0/2909114306 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5a94105860 con 0x7f5a7c0394e0 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5a7c0394e0 msgr2=0x7f5a7c03b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5a7c0394e0 0x7f5a7c03b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5a88006fd0 tx=0x7f5a88006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 msgr2=0x7f5a9407eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a9407eb40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a84000c00 tx=0x7f5a84011b70 comp rx=0 tx=0).stop 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 shutdown_connections 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5a7c0394e0 0x7f5a7c03b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 --2- 172.21.15.90:0/2909114306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a94109060 0x7f5a9407eb40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:06.799 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 >> 172.21.15.90:0/2909114306 conn(0x7f5a94104640 msgr2=0x7f5a94105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:06.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 shutdown_connections 2024-04-26T18:01:06.800 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:06.195+0000 7f5a9a70b700 1 -- 172.21.15.90:0/2909114306 wait complete. 2024-04-26T18:01:07.074 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:06 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:05] ENGINE Bus STARTING 2024-04-26T18:01:07.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:06 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:06] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T18:01:07.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:06 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:07.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:06 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:07.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:06 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:07.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:06 smithi090 ceph-mon[29180]: mgrmap e8: x(active, since 2s) 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.173+0000 7f5df259e700 1 Processor -- start 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.173+0000 7f5df259e700 1 -- start start 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5df259e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec10bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5df259e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec07cda0 con 0x7f5dec109b60 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5debfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec10bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5debfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec10bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50246/0 (socket says 172.21.15.90:50246) 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5debfff700 1 -- 172.21.15.90:0/1254171240 learned_addr learned my addr 172.21.15.90:0/1254171240 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:07.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5debfff700 1 -- 172.21.15.90:0/1254171240 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec10c490 con 0x7f5dec109b60 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5debfff700 1 --2- 172.21.15.90:0/1254171240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec10bf50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098a0 tx=0x7f5de0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f29b9dffc62e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5deaffd700 1 -- 172.21.15.90:0/1254171240 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5de0004160 con 0x7f5dec109b60 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5deaffd700 1 -- 172.21.15.90:0/1254171240 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5de0004300 con 0x7f5dec109b60 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.174+0000 7f5deaffd700 1 -- 172.21.15.90:0/1254171240 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5de0011af0 con 0x7f5dec109b60 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 -- 172.21.15.90:0/1254171240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 msgr2=0x7f5dec10bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 --2- 172.21.15.90:0/1254171240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec10bf50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098a0 tx=0x7f5de0009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 -- 172.21.15.90:0/1254171240 shutdown_connections 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 --2- 172.21.15.90:0/1254171240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec10bf50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 -- 172.21.15.90:0/1254171240 >> 172.21.15.90:0/1254171240 conn(0x7f5dec1034f0 msgr2=0x7f5dec105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 -- 172.21.15.90:0/1254171240 shutdown_connections 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 -- 172.21.15.90:0/1254171240 wait complete. 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 Processor -- start 2024-04-26T18:01:07.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.175+0000 7f5df259e700 1 -- start start 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5df259e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5df259e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec10d2b0 con 0x7f5dec109b60 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5debfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5debfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec19ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50252/0 (socket says 172.21.15.90:50252) 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5debfff700 1 -- 172.21.15.90:0/2860214411 learned_addr learned my addr 172.21.15.90:0/2860214411 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5debfff700 1 -- 172.21.15.90:0/2860214411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec19d380 con 0x7f5dec109b60 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5debfff700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec19ce40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5de0011ac0 tx=0x7f5de0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5de97fa700 1 -- 172.21.15.90:0/2860214411 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5de0003b20 con 0x7f5dec109b60 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5de97fa700 1 -- 172.21.15.90:0/2860214411 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5de001a8c0 con 0x7f5dec109b60 2024-04-26T18:01:07.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec19d610 con 0x7f5dec109b60 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5de97fa700 1 -- 172.21.15.90:0/2860214411 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5de00248d0 con 0x7f5dec109b60 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.176+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1a0300 con 0x7f5dec109b60 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.177+0000 7f5de97fa700 1 -- 172.21.15.90:0/2860214411 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5de0022020 con 0x7f5dec109b60 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.177+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec06a580 con 0x7f5dec109b60 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.177+0000 7f5de97fa700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5dd40395b0 0x7f5dd403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.177+0000 7f5de97fa700 1 -- 172.21.15.90:0/2860214411 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5de004d8b0 con 0x7f5dec109b60 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.178+0000 7f5deb7fe700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5dd40395b0 0x7f5dd403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.178+0000 7f5deb7fe700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5dd40395b0 0x7f5dd403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006fd0 tx=0x7f5ddc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.182+0000 7f5de97fa700 1 -- 172.21.15.90:0/2860214411 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5de000b5a0 con 0x7f5dec109b60 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.381+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f5dec1047b0 con 0x7f5dd40395b0 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.382+0000 7f5de97fa700 1 -- 172.21.15.90:0/2860214411 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f5dec1047b0 con 0x7f5dd40395b0 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5dd40395b0 msgr2=0x7f5dd403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:07.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5dd40395b0 0x7f5dd403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006fd0 tx=0x7f5ddc006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:07.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 msgr2=0x7f5dec19ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:07.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec19ce40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5de0011ac0 tx=0x7f5de0011ba0 comp rx=0 tx=0).stop 2024-04-26T18:01:07.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 shutdown_connections 2024-04-26T18:01:07.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f5dd40395b0 0x7f5dd403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:07.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 --2- 172.21.15.90:0/2860214411 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dec109b60 0x7f5dec19ce40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:07.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 >> 172.21.15.90:0/2860214411 conn(0x7f5dec1034f0 msgr2=0x7f5dec1040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:07.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 shutdown_connections 2024-04-26T18:01:07.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:07.384+0000 7f5df259e700 1 -- 172.21.15.90:0/2860214411 wait complete. 2024-04-26T18:01:07.937 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-04-26T18:01:08.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:07 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:06] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T18:01:08.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:07 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:06] ENGINE Bus STARTED 2024-04-26T18:01:08.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:07 smithi090 ceph-mon[29180]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:09.346 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:08 smithi090 ceph-mon[29180]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:09.346 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: Generating public/private rsa key pair. 2024-04-26T18:01:09.346 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: Your identification has been saved in /tmp/tmpsquqiqar/key. 2024-04-26T18:01:09.346 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: Your public key has been saved in /tmp/tmpsquqiqar/key.pub. 2024-04-26T18:01:09.346 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: The key fingerprint is: 2024-04-26T18:01:09.346 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: SHA256:OdI6tH4PyTfxHOCJa3pBZzKI0LsqxN2sr85QQZgWN4Q ceph-b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: The key's randomart image is: 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: +---[RSA 3072]----+ 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: | .** | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: | E+ o | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: |. o o . . | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: | + ..++oo | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: |. ..o.o.S=+ . | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: | o...+ =.+ + . | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: |.. .. + *.o o | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: |. +. . +oo . | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: | ..+o.oo .. | 2024-04-26T18:01:09.347 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: +----[SHA256]-----+ 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.327+0000 7f61bd9b6700 1 Processor -- start 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.327+0000 7f61bd9b6700 1 -- start start 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.327+0000 7f61bd9b6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b8107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.327+0000 7f61bd9b6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61b8107ce0 con 0x7f61b81072f0 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.327+0000 7f61b6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b8107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.327+0000 7f61b6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b8107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50254/0 (socket says 172.21.15.90:50254) 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.327+0000 7f61b6ffd700 1 -- 172.21.15.90:0/3064462293 learned_addr learned my addr 172.21.15.90:0/3064462293 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.328+0000 7f61b6ffd700 1 -- 172.21.15.90:0/3064462293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b8107e60 con 0x7f61b81072f0 2024-04-26T18:01:09.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.328+0000 7f61b6ffd700 1 --2- 172.21.15.90:0/3064462293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b8107710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009e80 tx=0x7f61b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b98f2a25e68b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.328+0000 7f61b5ffb700 1 -- 172.21.15.90:0/3064462293 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0017070 con 0x7f61b81072f0 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.328+0000 7f61b5ffb700 1 -- 172.21.15.90:0/3064462293 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f61b000b8c0 con 0x7f61b81072f0 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.328+0000 7f61b5ffb700 1 -- 172.21.15.90:0/3064462293 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0004490 con 0x7f61b81072f0 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.328+0000 7f61bd9b6700 1 -- 172.21.15.90:0/3064462293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 msgr2=0x7f61b8107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.328+0000 7f61bd9b6700 1 --2- 172.21.15.90:0/3064462293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b8107710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009e80 tx=0x7f61b000b330 comp rx=0 tx=0).stop 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 -- 172.21.15.90:0/3064462293 shutdown_connections 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 --2- 172.21.15.90:0/3064462293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b8107710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 -- 172.21.15.90:0/3064462293 >> 172.21.15.90:0/3064462293 conn(0x7f61b8102cb0 msgr2=0x7f61b81050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 -- 172.21.15.90:0/3064462293 shutdown_connections 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 -- 172.21.15.90:0/3064462293 wait complete. 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 Processor -- start 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 -- start start 2024-04-26T18:01:09.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.329+0000 7f61bd9b6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61b810b0f0 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61b6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61b6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b819ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50264/0 (socket says 172.21.15.90:50264) 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61b6ffd700 1 -- 172.21.15.90:0/1456682845 learned_addr learned my addr 172.21.15.90:0/1456682845 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61b6ffd700 1 -- 172.21.15.90:0/1456682845 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b819d3b0 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61b6ffd700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b819ce70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b7b0 tx=0x7f61b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61bc9b4700 1 -- 172.21.15.90:0/1456682845 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0020020 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61bc9b4700 1 -- 172.21.15.90:0/1456682845 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f61b0012430 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b819d640 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61bc9b4700 1 -- 172.21.15.90:0/1456682845 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61b0017040 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.330+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b81a0330 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.331+0000 7f61bc9b4700 1 -- 172.21.15.90:0/1456682845 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f61b0023020 con 0x7f61b81072f0 2024-04-26T18:01:09.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.331+0000 7f61bc9b4700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f61a4039560 0x7f61a403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.331+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b8064a80 con 0x7f61b81072f0 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.331+0000 7f61bc9b4700 1 -- 172.21.15.90:0/1456682845 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f61b000f050 con 0x7f61b81072f0 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.331+0000 7f61b67fc700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f61a4039560 0x7f61a403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.332+0000 7f61b67fc700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f61a4039560 0x7f61a403ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f61a0006fd0 tx=0x7f61a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.337+0000 7f61bc9b4700 1 -- 172.21.15.90:0/1456682845 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61b002a020 con 0x7f61b81072f0 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:08.540+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f61b8103f30 con 0x7f61a4039560 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.034+0000 7f61bc9b4700 1 -- 172.21.15.90:0/1456682845 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f61b8103f30 con 0x7f61a4039560 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f61a4039560 msgr2=0x7f61a403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f61a4039560 0x7f61a403ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f61a0006fd0 tx=0x7f61a0006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 msgr2=0x7f61b819ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b819ce70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b7b0 tx=0x7f61b000be20 comp rx=0 tx=0).stop 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 shutdown_connections 2024-04-26T18:01:09.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f61a4039560 0x7f61a403ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:09.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 --2- 172.21.15.90:0/1456682845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61b81072f0 0x7f61b819ce70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:09.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.036+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 >> 172.21.15.90:0/1456682845 conn(0x7f61b8102cb0 msgr2=0x7f61b8103820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:09.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.037+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 shutdown_connections 2024-04-26T18:01:09.634 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:09.037+0000 7f61bd9b6700 1 -- 172.21.15.90:0/1456682845 wait complete. 2024-04-26T18:01:10.230 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:10 smithi090 ceph-mon[29180]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:10.230 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:10 smithi090 ceph-mon[29180]: Generating ssh key... 2024-04-26T18:01:10.230 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:10 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:10.230 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:10 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:10.849 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8QMdN4h4gbYRp+CoopOxz3LKv7AxU3854VnI+eo3d683RNuuWHj2Vm+lAZRKDFsoExbQhjOomYMFnrBLCcmhjyV8SqP7aJ2fDCFzUEsA1d8DzEAdLeckenCg0grF282rLF3cfCSYbNHSlBl2l2/3xLB7vHb8A1IFe8eHl9b8LhuQ9yqc8bLHOxYVDPsYrvmayGlEyfgZtHu9E1dJ2Hj3IOdIIz4Wx3OTbV4lQXK3pwtw4chadQMJT0L12DttDCUGdYKnnrJeZhtj5R7haElR1RI7Gt1iKJlHMuaVs10l11WXk3FZYxe/WlvXPUzsP+JpkVfLOuXkYczaugjbbEU/EVko26P0QObqmaA/YGJlhaYXLLFCeiJcU4oaTp6tVS7ibBvaFLiHTJD0puE7VxxUvSSDC5QvVjHO79ZGHr4lmFTkUMmLqpWSDnv2CHwZzRGmFMKSE7IF/VRlnWconGu4I/lW9rGvj3FR6g8XMNRahsOs1mMd/tkG9kUcEu4vy/+U= ceph-b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.017+0000 7f0cc614b700 1 Processor -- start 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.017+0000 7f0cc614b700 1 -- start start 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cc614b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc010bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cc614b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc007ce40 con 0x7f0cc0109af0 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc010bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc010bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50272/0 (socket says 172.21.15.90:50272) 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbf7fe700 1 -- 172.21.15.90:0/525744134 learned_addr learned my addr 172.21.15.90:0/525744134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbf7fe700 1 -- 172.21.15.90:0/525744134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc010c420 con 0x7f0cc0109af0 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbf7fe700 1 --2- 172.21.15.90:0/525744134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc010bee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00098a0 tx=0x7f0cb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e01a0a581c411e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbe7fc700 1 -- 172.21.15.90:0/525744134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0004160 con 0x7f0cc0109af0 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbe7fc700 1 -- 172.21.15.90:0/525744134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0cb0004300 con 0x7f0cc0109af0 2024-04-26T18:01:10.850 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.018+0000 7f0cbe7fc700 1 -- 172.21.15.90:0/525744134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0011af0 con 0x7f0cc0109af0 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.019+0000 7f0cc614b700 1 -- 172.21.15.90:0/525744134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 msgr2=0x7f0cc010bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.019+0000 7f0cc614b700 1 --2- 172.21.15.90:0/525744134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc010bee0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0cb00098a0 tx=0x7f0cb0009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.019+0000 7f0cc614b700 1 -- 172.21.15.90:0/525744134 shutdown_connections 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.019+0000 7f0cc614b700 1 --2- 172.21.15.90:0/525744134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc010bee0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.019+0000 7f0cc614b700 1 -- 172.21.15.90:0/525744134 >> 172.21.15.90:0/525744134 conn(0x7f0cc0103500 msgr2=0x7f0cc0105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.019+0000 7f0cc614b700 1 -- 172.21.15.90:0/525744134 shutdown_connections 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.019+0000 7f0cc614b700 1 -- 172.21.15.90:0/525744134 wait complete. 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cc614b700 1 Processor -- start 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cc614b700 1 -- start start 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cc614b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc0109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cc614b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc010d300 con 0x7f0cc0109af0 2024-04-26T18:01:10.851 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cbf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc0109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cbf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc0109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50288/0 (socket says 172.21.15.90:50288) 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cbf7fe700 1 -- 172.21.15.90:0/695926596 learned_addr learned my addr 172.21.15.90:0/695926596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cbf7fe700 1 -- 172.21.15.90:0/695926596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc01098b0 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.020+0000 7f0cbf7fe700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc0109370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0011ac0 tx=0x7f0cb0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.021+0000 7f0cbcff9700 1 -- 172.21.15.90:0/695926596 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0003970 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.021+0000 7f0cbcff9700 1 -- 172.21.15.90:0/695926596 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0cb0003b10 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.021+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc0107aa0 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.021+0000 7f0cbcff9700 1 -- 172.21.15.90:0/695926596 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cb0024610 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.021+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc0107fe0 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.021+0000 7f0cbcff9700 1 -- 172.21.15.90:0/695926596 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0cb0024890 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.022+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc0064a80 con 0x7f0cc0109af0 2024-04-26T18:01:10.852 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.022+0000 7f0cbcff9700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f0ca80395b0 0x7f0ca803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.022+0000 7f0cbcff9700 1 -- 172.21.15.90:0/695926596 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0cb004da60 con 0x7f0cc0109af0 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.022+0000 7f0cbeffd700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f0ca80395b0 0x7f0ca803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.022+0000 7f0cbeffd700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f0ca80395b0 0x7f0ca803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4006fd0 tx=0x7f0cb4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.027+0000 7f0cbcff9700 1 -- 172.21.15.90:0/695926596 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cb002d9d0 con 0x7f0cc0109af0 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.230+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f0cc0104770 con 0x7f0ca80395b0 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.230+0000 7f0cbcff9700 1 -- 172.21.15.90:0/695926596 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f0cc0104770 con 0x7f0ca80395b0 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f0ca80395b0 msgr2=0x7f0ca803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f0ca80395b0 0x7f0ca803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4006fd0 tx=0x7f0cb4006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 msgr2=0x7f0cc0109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc0109370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0011ac0 tx=0x7f0cb0011ba0 comp rx=0 tx=0).stop 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 shutdown_connections 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f0ca80395b0 0x7f0ca803ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:10.853 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 --2- 172.21.15.90:0/695926596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0cc0109af0 0x7f0cc0109370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:10.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.232+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 >> 172.21.15.90:0/695926596 conn(0x7f0cc0103500 msgr2=0x7f0cc0104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:10.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.233+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 shutdown_connections 2024-04-26T18:01:10.854 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:10.233+0000 7f0cc614b700 1 -- 172.21.15.90:0/695926596 wait complete. 2024-04-26T18:01:10.854 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:01:10.854 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T18:01:10.854 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-04-26T18:01:11.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:11 smithi090 ceph-mon[29180]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:12.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:12 smithi090 ceph-mon[29180]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:12.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:12 smithi090 ceph-mon[29180]: Deploying cephadm binary to smithi090 2024-04-26T18:01:16.200 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488ed0f700 1 Processor -- start 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488ed0f700 1 -- start start 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488ed0f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488ed0f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4888108e20 con 0x7f4888108430 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488caab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488caab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888108850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50302/0 (socket says 172.21.15.90:50302) 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488caab700 1 -- 172.21.15.90:0/4099456950 learned_addr learned my addr 172.21.15.90:0/4099456950 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488caab700 1 -- 172.21.15.90:0/4099456950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4888109150 con 0x7f4888108430 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.282+0000 7f488caab700 1 --2- 172.21.15.90:0/4099456950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888108850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f487c0098a0 tx=0x7f487c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be34d41183cd11da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f48877fe700 1 -- 172.21.15.90:0/4099456950 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c004160 con 0x7f4888108430 2024-04-26T18:01:16.201 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f48877fe700 1 -- 172.21.15.90:0/4099456950 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f487c004300 con 0x7f4888108430 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f48877fe700 1 -- 172.21.15.90:0/4099456950 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c011af0 con 0x7f4888108430 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f488ed0f700 1 -- 172.21.15.90:0/4099456950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 msgr2=0x7f4888108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f488ed0f700 1 --2- 172.21.15.90:0/4099456950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888108850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f487c0098a0 tx=0x7f487c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f488ed0f700 1 -- 172.21.15.90:0/4099456950 shutdown_connections 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f488ed0f700 1 --2- 172.21.15.90:0/4099456950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888108850 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f488ed0f700 1 -- 172.21.15.90:0/4099456950 >> 172.21.15.90:0/4099456950 conn(0x7f4888102cb0 msgr2=0x7f48881050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f488ed0f700 1 -- 172.21.15.90:0/4099456950 shutdown_connections 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.283+0000 7f488ed0f700 1 -- 172.21.15.90:0/4099456950 wait complete. 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488ed0f700 1 Processor -- start 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488ed0f700 1 -- start start 2024-04-26T18:01:16.202 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488ed0f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888107a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488ed0f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f488810c380 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488caab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888107a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488caab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888107a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50306/0 (socket says 172.21.15.90:50306) 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488caab700 1 -- 172.21.15.90:0/2566969647 learned_addr learned my addr 172.21.15.90:0/2566969647 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488caab700 1 -- 172.21.15.90:0/2566969647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488819d9a0 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.284+0000 7f488caab700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888107a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f487c009fb0 tx=0x7f487c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.285+0000 7f4885ffb700 1 -- 172.21.15.90:0/2566969647 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c0036a0 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.285+0000 7f4885ffb700 1 -- 172.21.15.90:0/2566969647 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f487c01a430 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.285+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48881a1fb0 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.285+0000 7f4885ffb700 1 -- 172.21.15.90:0/2566969647 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f487c019550 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.285+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48881a2350 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.286+0000 7f4885ffb700 1 -- 172.21.15.90:0/2566969647 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f487c019800 con 0x7f4888108430 2024-04-26T18:01:16.203 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.286+0000 7f4885ffb700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f48780395b0 0x7f487803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.286+0000 7f4885ffb700 1 -- 172.21.15.90:0/2566969647 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f487c04d8d0 con 0x7f4888108430 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.286+0000 7f4887fff700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f48780395b0 0x7f487803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.286+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4888064a80 con 0x7f4888108430 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.286+0000 7f4887fff700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f48780395b0 0x7f487803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4874006fd0 tx=0x7f4874006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.295+0000 7f4885ffb700 1 -- 172.21.15.90:0/2566969647 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f487c00bc40 con 0x7f4888108430 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:11.505+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}) -- 0x7f4888103f60 con 0x7f48780395b0 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.566+0000 7f4885ffb700 1 -- 172.21.15.90:0/2566969647 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f4888103f60 con 0x7f48780395b0 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f48780395b0 msgr2=0x7f487803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f48780395b0 0x7f487803ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4874006fd0 tx=0x7f4874006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 msgr2=0x7f4888107a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888107a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f487c009fb0 tx=0x7f487c011940 comp rx=0 tx=0).stop 2024-04-26T18:01:16.204 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 shutdown_connections 2024-04-26T18:01:16.205 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f48780395b0 0x7f487803ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:16.205 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 --2- 172.21.15.90:0/2566969647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4888108430 0x7f4888107a10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:16.205 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 >> 172.21.15.90:0/2566969647 conn(0x7f4888102cb0 msgr2=0x7f4888103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:16.205 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 shutdown_connections 2024-04-26T18:01:16.205 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:15.569+0000 7f488ed0f700 1 -- 172.21.15.90:0/2566969647 wait complete. 2024-04-26T18:01:16.205 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mon service... 2024-04-26T18:01:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:16 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:16 smithi090 ceph-mon[29180]: Added host smithi090 2024-04-26T18:01:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:16 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:17 smithi090 ceph-mon[29180]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:17 smithi090 ceph-mon[29180]: Saving service mon spec with placement count:5 2024-04-26T18:01:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:17 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:18.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.692+0000 7fb8411e5700 1 Processor -- start 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb8411e5700 1 -- start start 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb8411e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c07d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb8411e5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c07d8c0 con 0x7fb83c07ee90 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb83ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c07d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb83ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c07d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49302/0 (socket says 172.21.15.90:49302) 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb83ad9d700 1 -- 172.21.15.90:0/122897812 learned_addr learned my addr 172.21.15.90:0/122897812 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb83ad9d700 1 -- 172.21.15.90:0/122897812 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c07da40 con 0x7fb83c07ee90 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb83ad9d700 1 --2- 172.21.15.90:0/122897812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c07d2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8300098a0 tx=0x7fb830009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747701028fec51ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb839d9b700 1 -- 172.21.15.90:0/122897812 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb830004160 con 0x7fb83c07ee90 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb839d9b700 1 -- 172.21.15.90:0/122897812 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb830004300 con 0x7fb83c07ee90 2024-04-26T18:01:18.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.693+0000 7fb839d9b700 1 -- 172.21.15.90:0/122897812 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb830011af0 con 0x7fb83c07ee90 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.694+0000 7fb8411e5700 1 -- 172.21.15.90:0/122897812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 msgr2=0x7fb83c07d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.694+0000 7fb8411e5700 1 --2- 172.21.15.90:0/122897812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c07d2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb8300098a0 tx=0x7fb830009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.694+0000 7fb8411e5700 1 -- 172.21.15.90:0/122897812 shutdown_connections 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.694+0000 7fb8411e5700 1 --2- 172.21.15.90:0/122897812 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c07d2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.694+0000 7fb8411e5700 1 -- 172.21.15.90:0/122897812 >> 172.21.15.90:0/122897812 conn(0x7fb83c104610 msgr2=0x7fb83c106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.694+0000 7fb8411e5700 1 -- 172.21.15.90:0/122897812 shutdown_connections 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.694+0000 7fb8411e5700 1 -- 172.21.15.90:0/122897812 wait complete. 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb8411e5700 1 Processor -- start 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb8411e5700 1 -- start start 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb8411e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb8411e5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c10f510 con 0x7fb83c07ee90 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb83ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:18.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb83ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49308/0 (socket says 172.21.15.90:49308) 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb83ad9d700 1 -- 172.21.15.90:0/2403901692 learned_addr learned my addr 172.21.15.90:0/2403901692 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb83ad9d700 1 -- 172.21.15.90:0/2403901692 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c19f640 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.695+0000 7fb83ad9d700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c19f100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb830009fb0 tx=0x7fb830011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.696+0000 7fb823fff700 1 -- 172.21.15.90:0/2403901692 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8300036a0 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.696+0000 7fb823fff700 1 -- 172.21.15.90:0/2403901692 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb83001a430 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.696+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c19f8d0 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.696+0000 7fb823fff700 1 -- 172.21.15.90:0/2403901692 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb830019550 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.696+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c1a25c0 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.696+0000 7fb823fff700 1 -- 172.21.15.90:0/2403901692 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fb830019850 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.697+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83c064a80 con 0x7fb83c07ee90 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.697+0000 7fb823fff700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fb824039560 0x7fb82403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:18.426 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.697+0000 7fb823fff700 1 -- 172.21.15.90:0/2403901692 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb83004d920 con 0x7fb83c07ee90 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.697+0000 7fb83a59c700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fb824039560 0x7fb82403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.697+0000 7fb83a59c700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fb824039560 0x7fb82403ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb82c006fd0 tx=0x7fb82c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.702+0000 7fb823fff700 1 -- 172.21.15.90:0/2403901692 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb83000fda0 con 0x7fb83c07ee90 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.923+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fb83c105910 con 0x7fb824039560 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.931+0000 7fb823fff700 1 -- 172.21.15.90:0/2403901692 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb83c105910 con 0x7fb824039560 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fb824039560 msgr2=0x7fb82403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fb824039560 0x7fb82403ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb82c006fd0 tx=0x7fb82c006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 msgr2=0x7fb83c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c19f100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb830009fb0 tx=0x7fb830011940 comp rx=0 tx=0).stop 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 shutdown_connections 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fb824039560 0x7fb82403ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:18.427 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 --2- 172.21.15.90:0/2403901692 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb83c07ee90 0x7fb83c19f100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:18.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 >> 172.21.15.90:0/2403901692 conn(0x7fb83c104610 msgr2=0x7fb83c105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:18.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 shutdown_connections 2024-04-26T18:01:18.428 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:16.933+0000 7fb8411e5700 1 -- 172.21.15.90:0/2403901692 wait complete. 2024-04-26T18:01:18.428 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying unmanaged mgr service... 2024-04-26T18:01:19.339 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:19 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:19.339 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:19 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.886+0000 7fdc9f399700 1 Processor -- start 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9f399700 1 -- start start 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9f399700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc98108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9f399700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc98108870 con 0x7fdc98107ef0 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9d135700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc98108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9d135700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc98108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49322/0 (socket says 172.21.15.90:49322) 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9d135700 1 -- 172.21.15.90:0/250288129 learned_addr learned my addr 172.21.15.90:0/250288129 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9d135700 1 -- 172.21.15.90:0/250288129 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc981090d0 con 0x7fdc98107ef0 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.887+0000 7fdc9d135700 1 --2- 172.21.15.90:0/250288129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc98108330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc940098a0 tx=0x7fdc94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a46c912715b4583 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:20.353 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc8ffff700 1 -- 172.21.15.90:0/250288129 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc94004160 con 0x7fdc98107ef0 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc8ffff700 1 -- 172.21.15.90:0/250288129 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc94004300 con 0x7fdc98107ef0 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc8ffff700 1 -- 172.21.15.90:0/250288129 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc94011a20 con 0x7fdc98107ef0 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc9f399700 1 -- 172.21.15.90:0/250288129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 msgr2=0x7fdc98108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc9f399700 1 --2- 172.21.15.90:0/250288129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc98108330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdc940098a0 tx=0x7fdc94009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc9f399700 1 -- 172.21.15.90:0/250288129 shutdown_connections 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc9f399700 1 --2- 172.21.15.90:0/250288129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc98108330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc9f399700 1 -- 172.21.15.90:0/250288129 >> 172.21.15.90:0/250288129 conn(0x7fdc98103500 msgr2=0x7fdc98105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc9f399700 1 -- 172.21.15.90:0/250288129 shutdown_connections 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.888+0000 7fdc9f399700 1 -- 172.21.15.90:0/250288129 wait complete. 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.889+0000 7fdc9f399700 1 Processor -- start 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.889+0000 7fdc9f399700 1 -- start start 2024-04-26T18:01:20.354 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.889+0000 7fdc9f399700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc9819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.889+0000 7fdc9f399700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9810c390 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.889+0000 7fdc9d135700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc9819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.889+0000 7fdc9d135700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc9819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49338/0 (socket says 172.21.15.90:49338) 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.889+0000 7fdc9d135700 1 -- 172.21.15.90:0/1811244397 learned_addr learned my addr 172.21.15.90:0/1811244397 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.890+0000 7fdc9d135700 1 -- 172.21.15.90:0/1811244397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9819d430 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.890+0000 7fdc9d135700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc9819cef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc94011770 tx=0x7fdc94011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.890+0000 7fdc8e7fc700 1 -- 172.21.15.90:0/1811244397 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc940039e0 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.890+0000 7fdc8e7fc700 1 -- 172.21.15.90:0/1811244397 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9401b8f0 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.890+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9819d6c0 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.890+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc981a03b0 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.890+0000 7fdc8e7fc700 1 -- 172.21.15.90:0/1811244397 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc94023760 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.891+0000 7fdc8e7fc700 1 -- 172.21.15.90:0/1811244397 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdc94023a50 con 0x7fdc98107ef0 2024-04-26T18:01:20.355 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.891+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7c005330 con 0x7fdc98107ef0 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.891+0000 7fdc8e7fc700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fdc84039560 0x7fdc8403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.891+0000 7fdc8e7fc700 1 -- 172.21.15.90:0/1811244397 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdc9404dea0 con 0x7fdc98107ef0 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.891+0000 7fdc9c934700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fdc84039560 0x7fdc8403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.891+0000 7fdc9c934700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fdc84039560 0x7fdc8403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006fd0 tx=0x7fdc88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:18.900+0000 7fdc8e7fc700 1 -- 172.21.15.90:0/1811244397 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc9401f030 con 0x7fdc98107ef0 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.150+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdc7c000bc0 con 0x7fdc84039560 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.157+0000 7fdc8e7fc700 1 -- 172.21.15.90:0/1811244397 <== mgr.14120 v2:172.21.15.90:6800/1868991041 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdc7c000bc0 con 0x7fdc84039560 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fdc84039560 msgr2=0x7fdc8403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fdc84039560 0x7fdc8403ba20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdc88006fd0 tx=0x7fdc88006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 msgr2=0x7fdc9819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:20.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc9819cef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc94011770 tx=0x7fdc94011aa0 comp rx=0 tx=0).stop 2024-04-26T18:01:20.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 shutdown_connections 2024-04-26T18:01:20.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fdc84039560 0x7fdc8403ba20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:20.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 --2- 172.21.15.90:0/1811244397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdc98107ef0 0x7fdc9819cef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:20.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.159+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 >> 172.21.15.90:0/1811244397 conn(0x7fdc98103500 msgr2=0x7fdc98104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:20.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.160+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 shutdown_connections 2024-04-26T18:01:20.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:19.160+0000 7fdc9f399700 1 -- 172.21.15.90:0/1811244397 wait complete. 2024-04-26T18:01:20.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:20 smithi090 ceph-mon[29180]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:20.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:20 smithi090 ceph-mon[29180]: Saving service mgr spec with placement count:2 2024-04-26T18:01:20.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:20 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:20.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:20 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:21.629 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.789+0000 7ff3de33a700 1 Processor -- start 2024-04-26T18:01:21.629 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.789+0000 7ff3de33a700 1 -- start start 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.789+0000 7ff3de33a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d8107aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.789+0000 7ff3de33a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d8108070 con 0x7ff3d8107680 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.790+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d8107aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.790+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d8107aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49348/0 (socket says 172.21.15.90:49348) 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.790+0000 7ff3d7fff700 1 -- 172.21.15.90:0/3302913602 learned_addr learned my addr 172.21.15.90:0/3302913602 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.790+0000 7ff3d7fff700 1 -- 172.21.15.90:0/3302913602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d8108880 con 0x7ff3d8107680 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.790+0000 7ff3d7fff700 1 --2- 172.21.15.90:0/3302913602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d8107aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00bb40 tx=0x7ff3cc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99eca1e095725f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3d6ffd700 1 -- 172.21.15.90:0/3302913602 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3cc00dad0 con 0x7ff3d8107680 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3d6ffd700 1 -- 172.21.15.90:0/3302913602 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc0044e0 con 0x7ff3d8107680 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3d6ffd700 1 -- 172.21.15.90:0/3302913602 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3cc003da0 con 0x7ff3d8107680 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3de33a700 1 -- 172.21.15.90:0/3302913602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 msgr2=0x7ff3d8107aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:21.630 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3de33a700 1 --2- 172.21.15.90:0/3302913602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d8107aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00bb40 tx=0x7ff3cc00be50 comp rx=0 tx=0).stop 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3de33a700 1 -- 172.21.15.90:0/3302913602 shutdown_connections 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3de33a700 1 --2- 172.21.15.90:0/3302913602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d8107aa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3de33a700 1 -- 172.21.15.90:0/3302913602 >> 172.21.15.90:0/3302913602 conn(0x7ff3d8102c80 msgr2=0x7ff3d81050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3de33a700 1 -- 172.21.15.90:0/3302913602 shutdown_connections 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.791+0000 7ff3de33a700 1 -- 172.21.15.90:0/3302913602 wait complete. 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.792+0000 7ff3de33a700 1 Processor -- start 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.792+0000 7ff3de33a700 1 -- start start 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.792+0000 7ff3de33a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d819ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.792+0000 7ff3de33a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d8108ff0 con 0x7ff3d8107680 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.792+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d819ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.792+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d819ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49354/0 (socket says 172.21.15.90:49354) 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.792+0000 7ff3d7fff700 1 -- 172.21.15.90:0/712712444 learned_addr learned my addr 172.21.15.90:0/712712444 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:21.631 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.793+0000 7ff3d7fff700 1 -- 172.21.15.90:0/712712444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d819b1d0 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.793+0000 7ff3d7fff700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d819ac90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc019040 tx=0x7ff3cc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.793+0000 7ff3d57fa700 1 -- 172.21.15.90:0/712712444 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3cc025020 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.793+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d819b460 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.793+0000 7ff3d57fa700 1 -- 172.21.15.90:0/712712444 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3cc017960 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.793+0000 7ff3d57fa700 1 -- 172.21.15.90:0/712712444 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3cc0217e0 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.793+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d819e150 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.794+0000 7ff3d57fa700 1 -- 172.21.15.90:0/712712444 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff3cc028020 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.794+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d8064a80 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.794+0000 7ff3d57fa700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7ff3c00395b0 0x7ff3c003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.794+0000 7ff3d57fa700 1 -- 172.21.15.90:0/712712444 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff3cc050040 con 0x7ff3d8107680 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.795+0000 7ff3d77fe700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7ff3c00395b0 0x7ff3c003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.795+0000 7ff3d77fe700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7ff3c00395b0 0x7ff3c003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8006fd0 tx=0x7ff3c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:21.632 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:20.799+0000 7ff3d57fa700 1 -- 172.21.15.90:0/712712444 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3cc02f020 con 0x7ff3d8107680 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.001+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff3d802e6e0 con 0x7ff3d8107680 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.004+0000 7ff3d57fa700 1 -- 172.21.15.90:0/712712444 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7ff3cc012070 con 0x7ff3d8107680 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7ff3c00395b0 msgr2=0x7ff3c003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7ff3c00395b0 0x7ff3c003ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8006fd0 tx=0x7ff3c8006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 msgr2=0x7ff3d819ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d819ac90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc019040 tx=0x7ff3cc0047c0 comp rx=0 tx=0).stop 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 shutdown_connections 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7ff3c00395b0 0x7ff3c003ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 --2- 172.21.15.90:0/712712444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3d8107680 0x7ff3d819ac90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 >> 172.21.15.90:0/712712444 conn(0x7ff3d8102c80 msgr2=0x7ff3d81918f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 shutdown_connections 2024-04-26T18:01:21.633 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:21.006+0000 7ff3de33a700 1 -- 172.21.15.90:0/712712444 wait complete. 2024-04-26T18:01:22.222 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:22 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/712712444' entity='client.admin' 2024-04-26T18:01:22.222 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:22 smithi090 ceph-mon[29180]: from='mgr.14120 172.21.15.90:0/2630140406' entity='mgr.x' 2024-04-26T18:01:23.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.219+0000 7f74e4b6d700 1 Processor -- start 2024-04-26T18:01:23.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.219+0000 7f74e4b6d700 1 -- start start 2024-04-26T18:01:23.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.219+0000 7f74e4b6d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e0109930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:23.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.219+0000 7f74e4b6d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f74e0109f00 con 0x7f74e0109510 2024-04-26T18:01:23.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.219+0000 7f74de59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e0109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:23.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.219+0000 7f74de59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e0109930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54496/0 (socket says 172.21.15.90:54496) 2024-04-26T18:01:23.569 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.219+0000 7f74de59c700 1 -- 172.21.15.90:0/3163123739 learned_addr learned my addr 172.21.15.90:0/3163123739 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.220+0000 7f74de59c700 1 -- 172.21.15.90:0/3163123739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e0108120 con 0x7f74e0109510 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.220+0000 7f74de59c700 1 --2- 172.21.15.90:0/3163123739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e0109930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74d00098a0 tx=0x7f74d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f372011635869fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.220+0000 7f74dd59a700 1 -- 172.21.15.90:0/3163123739 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74d0004160 con 0x7f74e0109510 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.220+0000 7f74dd59a700 1 -- 172.21.15.90:0/3163123739 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d0004300 con 0x7f74e0109510 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.220+0000 7f74dd59a700 1 -- 172.21.15.90:0/3163123739 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74d0011a20 con 0x7f74e0109510 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.220+0000 7f74e4b6d700 1 -- 172.21.15.90:0/3163123739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 msgr2=0x7f74e0109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.220+0000 7f74e4b6d700 1 --2- 172.21.15.90:0/3163123739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e0109930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f74d00098a0 tx=0x7f74d0009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 -- 172.21.15.90:0/3163123739 shutdown_connections 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 --2- 172.21.15.90:0/3163123739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e0109930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 -- 172.21.15.90:0/3163123739 >> 172.21.15.90:0/3163123739 conn(0x7f74e0102cb0 msgr2=0x7f74e01050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 -- 172.21.15.90:0/3163123739 shutdown_connections 2024-04-26T18:01:23.570 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 -- 172.21.15.90:0/3163123739 wait complete. 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 Processor -- start 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 -- start start 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.221+0000 7f74e4b6d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f74e010c4e0 con 0x7f74e0109510 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74de59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74de59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e019cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54512/0 (socket says 172.21.15.90:54512) 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74de59c700 1 -- 172.21.15.90:0/1418661924 learned_addr learned my addr 172.21.15.90:0/1418661924 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74de59c700 1 -- 172.21.15.90:0/1418661924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e019d440 con 0x7f74e0109510 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74de59c700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e019cf00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009520 tx=0x7f74d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74cf7fe700 1 -- 172.21.15.90:0/1418661924 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74d00039e0 con 0x7f74e0109510 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74cf7fe700 1 -- 172.21.15.90:0/1418661924 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d001b8f0 con 0x7f74e0109510 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e019d6d0 con 0x7f74e0109510 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74cf7fe700 1 -- 172.21.15.90:0/1418661924 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74d0019770 con 0x7f74e0109510 2024-04-26T18:01:23.571 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.222+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e01a03c0 con 0x7f74e0109510 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.223+0000 7f74cf7fe700 1 -- 172.21.15.90:0/1418661924 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f74d0019a70 con 0x7f74e0109510 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.223+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e0064a80 con 0x7f74e0109510 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.223+0000 7f74cf7fe700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f74c80395b0 0x7f74c803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.223+0000 7f74cf7fe700 1 -- 172.21.15.90:0/1418661924 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f74d004ded0 con 0x7f74e0109510 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.223+0000 7f74ddd9b700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f74c80395b0 0x7f74c803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.224+0000 7f74ddd9b700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f74c80395b0 0x7f74c803ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f74d4006fd0 tx=0x7f74d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.228+0000 7f74cf7fe700 1 -- 172.21.15.90:0/1418661924 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74d001f030 con 0x7f74e0109510 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.434+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f74e002e7d0 con 0x7f74e0109510 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.437+0000 7f74cf7fe700 1 -- 172.21.15.90:0/1418661924 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f74d000b3e0 con 0x7f74e0109510 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.439+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f74c80395b0 msgr2=0x7f74c803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.439+0000 7f74e4b6d700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f74c80395b0 0x7f74c803ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f74d4006fd0 tx=0x7f74d4006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.439+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 msgr2=0x7f74e019cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:23.572 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.439+0000 7f74e4b6d700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e019cf00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f74d0009520 tx=0x7f74d0011aa0 comp rx=0 tx=0).stop 2024-04-26T18:01:23.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.439+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 shutdown_connections 2024-04-26T18:01:23.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.439+0000 7f74e4b6d700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f74c80395b0 0x7f74c803ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:23.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.439+0000 7f74e4b6d700 1 --2- 172.21.15.90:0/1418661924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74e0109510 0x7f74e019cf00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:23.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.440+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 >> 172.21.15.90:0/1418661924 conn(0x7f74e0102cb0 msgr2=0x7f74e0103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:23.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.440+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 shutdown_connections 2024-04-26T18:01:23.573 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:22.440+0000 7f74e4b6d700 1 -- 172.21.15.90:0/1418661924 wait complete. 2024-04-26T18:01:23.573 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-04-26T18:01:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:23 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1418661924' entity='client.admin' 2024-04-26T18:01:24.938 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:24 smithi090 ceph-mon[29180]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:01:24.938 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:24 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2380561228' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T18:01:25.255 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:24 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: ignoring --setuser ceph since I am not root 2024-04-26T18:01:25.255 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:24 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: ignoring --setgroup ceph since I am not root 2024-04-26T18:01:25.255 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:25 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:25.069+0000 7f0c01726200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:01:25.556 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:25 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:25.255+0000 7f0c01726200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:01:25.556 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:25 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:25.358+0000 7f0c01726200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.420+0000 7fbf7c955700 1 Processor -- start 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.420+0000 7fbf7c955700 1 -- start start 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7c955700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf78109460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7c955700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78109a30 con 0x7fbf78109040 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf78109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf78109460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54518/0 (socket says 172.21.15.90:54518) 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7659c700 1 -- 172.21.15.90:0/2255714963 learned_addr learned my addr 172.21.15.90:0/2255714963 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7659c700 1 -- 172.21.15.90:0/2255714963 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7810a240 con 0x7fbf78109040 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7659c700 1 --2- 172.21.15.90:0/2255714963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf78109460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e80 tx=0x7fbf6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72627a57f5142303 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7559a700 1 -- 172.21.15.90:0/2255714963 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf68017070 con 0x7fbf78109040 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7559a700 1 -- 172.21.15.90:0/2255714963 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf6800b8c0 con 0x7fbf78109040 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.421+0000 7fbf7559a700 1 -- 172.21.15.90:0/2255714963 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf68004490 con 0x7fbf78109040 2024-04-26T18:01:25.564 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.422+0000 7fbf7c955700 1 -- 172.21.15.90:0/2255714963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 msgr2=0x7fbf78109460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.422+0000 7fbf7c955700 1 --2- 172.21.15.90:0/2255714963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf78109460 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e80 tx=0x7fbf6800b330 comp rx=0 tx=0).stop 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.422+0000 7fbf7c955700 1 -- 172.21.15.90:0/2255714963 shutdown_connections 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.422+0000 7fbf7c955700 1 --2- 172.21.15.90:0/2255714963 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf78109460 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.422+0000 7fbf7c955700 1 -- 172.21.15.90:0/2255714963 >> 172.21.15.90:0/2255714963 conn(0x7fbf78104620 msgr2=0x7fbf78106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.422+0000 7fbf7c955700 1 -- 172.21.15.90:0/2255714963 shutdown_connections 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.422+0000 7fbf7c955700 1 -- 172.21.15.90:0/2255714963 wait complete. 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.423+0000 7fbf7c955700 1 Processor -- start 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.423+0000 7fbf7c955700 1 -- start start 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.423+0000 7fbf7c955700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf7819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.423+0000 7fbf7c955700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810d5f0 con 0x7fbf78109040 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.423+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf7819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.423+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf7819cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54530/0 (socket says 172.21.15.90:54530) 2024-04-26T18:01:25.565 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.423+0000 7fbf7659c700 1 -- 172.21.15.90:0/2380561228 learned_addr learned my addr 172.21.15.90:0/2380561228 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.424+0000 7fbf7659c700 1 -- 172.21.15.90:0/2380561228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7819d430 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.424+0000 7fbf7659c700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf7819cef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e50 tx=0x7fbf6800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.424+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf68020020 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.424+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf68003b50 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.424+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7819d6c0 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.424+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf68017040 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.424+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf781a03b0 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.425+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbf68023020 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.425+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf78064a80 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.425+0000 7fbf6f7fe700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fbf640395b0 0x7fbf6403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.425+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbf6800f050 con 0x7fbf78109040 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.425+0000 7fbf75d9b700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fbf640395b0 0x7fbf6403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:25.566 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.426+0000 7fbf75d9b700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fbf640395b0 0x7fbf6403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006fd0 tx=0x7fbf60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.430+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf6802a020 con 0x7fbf78109040 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.687+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fbf781a0580 con 0x7fbf78109040 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.892+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fbf781a0580 con 0x7fbf78109040 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.894+0000 7fbf6f7fe700 1 -- 172.21.15.90:0/2380561228 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fbf6800dc50 con 0x7fbf78109040 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fbf640395b0 msgr2=0x7fbf6403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fbf640395b0 0x7fbf6403ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbf60006fd0 tx=0x7fbf60006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 msgr2=0x7fbf7819cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf7819cef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009e50 tx=0x7fbf6800bd50 comp rx=0 tx=0).stop 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 shutdown_connections 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fbf640395b0 0x7fbf6403ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 --2- 172.21.15.90:0/2380561228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbf78109040 0x7fbf7819cef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 >> 172.21.15.90:0/2380561228 conn(0x7fbf78104620 msgr2=0x7fbf78105130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:25.567 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 shutdown_connections 2024-04-26T18:01:25.568 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:24.895+0000 7fbf7c955700 1 -- 172.21.15.90:0/2380561228 wait complete. 2024-04-26T18:01:25.808 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:25 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:25.680+0000 7f0c01726200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:01:25.808 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:25 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:25.767+0000 7f0c01726200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:01:26.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:25 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2380561228' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T18:01:26.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:25 smithi090 ceph-mon[29180]: mgrmap e9: x(active, since 21s) 2024-04-26T18:01:26.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:01:26.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.995+0000 7fc58dab2700 1 Processor -- start 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.995+0000 7fc58dab2700 1 -- start start 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.995+0000 7fc58dab2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58810bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.995+0000 7fc58dab2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc58807ce20 con 0x7fc588109b10 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58810bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58810bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54562/0 (socket says 172.21.15.90:54562) 2024-04-26T18:01:26.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5877fe700 1 -- 172.21.15.90:0/245529675 learned_addr learned my addr 172.21.15.90:0/245529675 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5877fe700 1 -- 172.21.15.90:0/245529675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58810c440 con 0x7fc588109b10 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5877fe700 1 --2- 172.21.15.90:0/245529675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58810bf00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00f960 tx=0x7fc57c00fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa725322e59ac426 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5867fc700 1 -- 172.21.15.90:0/245529675 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc57c01c070 con 0x7fc588109b10 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5867fc700 1 -- 172.21.15.90:0/245529675 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc57c004030 con 0x7fc588109b10 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.996+0000 7fc5867fc700 1 -- 172.21.15.90:0/245529675 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc57c003c30 con 0x7fc588109b10 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.997+0000 7fc58dab2700 1 -- 172.21.15.90:0/245529675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 msgr2=0x7fc58810bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.997+0000 7fc58dab2700 1 --2- 172.21.15.90:0/245529675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58810bf00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc57c00f960 tx=0x7fc57c00fc70 comp rx=0 tx=0).stop 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.997+0000 7fc58dab2700 1 -- 172.21.15.90:0/245529675 shutdown_connections 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.997+0000 7fc58dab2700 1 --2- 172.21.15.90:0/245529675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58810bf00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.997+0000 7fc58dab2700 1 -- 172.21.15.90:0/245529675 >> 172.21.15.90:0/245529675 conn(0x7fc588103500 msgr2=0x7fc588105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:26.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.997+0000 7fc58dab2700 1 -- 172.21.15.90:0/245529675 shutdown_connections 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.997+0000 7fc58dab2700 1 -- 172.21.15.90:0/245529675 wait complete. 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.998+0000 7fc58dab2700 1 Processor -- start 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.998+0000 7fc58dab2700 1 -- start start 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.998+0000 7fc58dab2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.998+0000 7fc58dab2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881086e0 con 0x7fc588109b10 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.998+0000 7fc5877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.998+0000 7fc5877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54568/0 (socket says 172.21.15.90:54568) 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.998+0000 7fc5877fe700 1 -- 172.21.15.90:0/1560111793 learned_addr learned my addr 172.21.15.90:0/1560111793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.999+0000 7fc5877fe700 1 -- 172.21.15.90:0/1560111793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58819b1f0 con 0x7fc588109b10 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.999+0000 7fc5877fe700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58819acb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc57c003880 tx=0x7fc57c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.999+0000 7fc584ff9700 1 -- 172.21.15.90:0/1560111793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc57c025070 con 0x7fc588109b10 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.999+0000 7fc584ff9700 1 -- 172.21.15.90:0/1560111793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc57c017900 con 0x7fc588109b10 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.999+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58819b480 con 0x7fc588109b10 2024-04-26T18:01:26.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.999+0000 7fc584ff9700 1 -- 172.21.15.90:0/1560111793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc57c01c040 con 0x7fc588109b10 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:25.999+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc58819e170 con 0x7fc588109b10 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.000+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc588064a80 con 0x7fc588109b10 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.000+0000 7fc584ff9700 1 -- 172.21.15.90:0/1560111793 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fc57c028070 con 0x7fc588109b10 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.000+0000 7fc584ff9700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 0x7fc57003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.000+0000 7fc584ff9700 1 -- 172.21.15.90:0/1560111793 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc57c014070 con 0x7fc588109b10 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.000+0000 7fc586ffd700 1 -- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 msgr2=0x7fc57003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1868991041 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.000+0000 7fc586ffd700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 0x7fc57003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.005+0000 7fc584ff9700 1 -- 172.21.15.90:0/1560111793 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc57c017db0 con 0x7fc588109b10 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.201+0000 7fc586ffd700 1 -- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 msgr2=0x7fc57003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1868991041 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.201+0000 7fc586ffd700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 0x7fc57003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.307+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc58802e7c0 con 0x7fc588109b10 2024-04-26T18:01:26.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.307+0000 7fc584ff9700 1 -- 172.21.15.90:0/1560111793 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fc57c028520 con 0x7fc588109b10 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 msgr2=0x7fc57003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 0x7fc57003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 msgr2=0x7fc58819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58819acb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc57c003880 tx=0x7fc57c0044c0 comp rx=0 tx=0).stop 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 shutdown_connections 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7fc5700395b0 0x7fc57003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 --2- 172.21.15.90:0/1560111793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc588109b10 0x7fc58819acb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 >> 172.21.15.90:0/1560111793 conn(0x7fc588103500 msgr2=0x7fc588103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 shutdown_connections 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:26.310+0000 7fc58dab2700 1 -- 172.21.15.90:0/1560111793 wait complete. 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-26T18:01:26.895 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-04-26T18:01:27.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:26 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1560111793' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T18:01:27.166 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:26 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:26.987+0000 7f0c01726200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:01:27.166 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:27.165+0000 7f0c01726200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:01:27.494 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:27.343+0000 7f0c01726200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:01:27.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:27.632+0000 7f0c01726200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:01:27.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:27.725+0000 7f0c01726200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:01:28.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:27.928+0000 7f0c01726200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:01:28.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:28.122+0000 7f0c01726200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:01:29.125 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:28.717+0000 7f0c01726200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:01:29.444 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:29 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:29.144+0000 7f0c01726200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:01:29.444 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:29 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:29.241+0000 7f0c01726200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:01:29.444 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:29 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:29.444+0000 7f0c01726200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:01:30.297 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:29 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:29.960+0000 7f0c01726200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:01:30.298 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:30.053+0000 7f0c01726200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:01:30.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:30.299+0000 7f0c01726200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:01:30.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:30.434+0000 7f0c01726200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:01:30.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:30.530+0000 7f0c01726200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:01:30.947 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:30.648+0000 7f0c01726200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:01:30.947 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:30.947+0000 7f0c01726200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:01:31.817 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:31.548+0000 7f0c01726200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:01:31.818 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:01:31.764+0000 7f0c01726200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:01:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: Active manager daemon x restarted 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: Activating manager daemon x 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: mgrmap e10: x(active, starting, since 0.00337272s) 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: Manager daemon x is now available 2024-04-26T18:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:01:32.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:01:32.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:32 smithi090 ceph-mon[29180]: mgrmap e11: x(active, since 1.00566s) 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.288+0000 7f7688fd9700 1 Processor -- start 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.288+0000 7f7688fd9700 1 -- start start 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.289+0000 7f7688fd9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f768410bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.289+0000 7f7688fd9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f768407ce80 con 0x7f7684109ab0 2024-04-26T18:01:33.613 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.289+0000 7f768259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f768410bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.289+0000 7f768259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f768410bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54580/0 (socket says 172.21.15.90:54580) 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.289+0000 7f768259c700 1 -- 172.21.15.90:0/2411599167 learned_addr learned my addr 172.21.15.90:0/2411599167 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.289+0000 7f768259c700 1 -- 172.21.15.90:0/2411599167 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768410c3e0 con 0x7f7684109ab0 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.289+0000 7f768259c700 1 --2- 172.21.15.90:0/2411599167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f768410bea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76740098a0 tx=0x7f7674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f705c2945a400bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f768159a700 1 -- 172.21.15.90:0/2411599167 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7674004160 con 0x7f7684109ab0 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f768159a700 1 -- 172.21.15.90:0/2411599167 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7674004300 con 0x7f7684109ab0 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f768159a700 1 -- 172.21.15.90:0/2411599167 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7674011a20 con 0x7f7684109ab0 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f7688fd9700 1 -- 172.21.15.90:0/2411599167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 msgr2=0x7f768410bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f7688fd9700 1 --2- 172.21.15.90:0/2411599167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f768410bea0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76740098a0 tx=0x7f7674009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f7688fd9700 1 -- 172.21.15.90:0/2411599167 shutdown_connections 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f7688fd9700 1 --2- 172.21.15.90:0/2411599167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f768410bea0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f7688fd9700 1 -- 172.21.15.90:0/2411599167 >> 172.21.15.90:0/2411599167 conn(0x7f7684103500 msgr2=0x7f7684105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:33.614 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f7688fd9700 1 -- 172.21.15.90:0/2411599167 shutdown_connections 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.290+0000 7f7688fd9700 1 -- 172.21.15.90:0/2411599167 wait complete. 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.291+0000 7f7688fd9700 1 Processor -- start 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.291+0000 7f7688fd9700 1 -- start start 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.291+0000 7f7688fd9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f7684109330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.291+0000 7f7688fd9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f768410d3f0 con 0x7f7684109ab0 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.291+0000 7f768259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f7684109330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.291+0000 7f768259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f7684109330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54590/0 (socket says 172.21.15.90:54590) 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.291+0000 7f768259c700 1 -- 172.21.15.90:0/2689141120 learned_addr learned my addr 172.21.15.90:0/2689141120 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.292+0000 7f768259c700 1 -- 172.21.15.90:0/2689141120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7684109870 con 0x7f7684109ab0 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.292+0000 7f768259c700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f7684109330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7674000c00 tx=0x7f7674011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.292+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7674003aa0 con 0x7f7684109ab0 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.292+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f767401b8f0 con 0x7f7684109ab0 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.292+0000 7f7688fd9700 1 -- 172.21.15.90:0/2689141120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7684107a60 con 0x7f7684109ab0 2024-04-26T18:01:33.615 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.292+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7674023870 con 0x7f7684109ab0 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.292+0000 7f7688fd9700 1 -- 172.21.15.90:0/2689141120 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7684107fa0 con 0x7f7684109ab0 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.293+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f7674023b70 con 0x7f7684109ab0 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.293+0000 7f76737fe700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.293+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 --> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f766c03c1d0 con 0x7f766c039600 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.293+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f767404e060 con 0x7f7684109ab0 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.293+0000 7f7681d9b700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 msgr2=0x7f766c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1868991041 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.293+0000 7f7681d9b700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.494+0000 7f7681d9b700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 msgr2=0x7f766c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1868991041 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.494+0000 7f7681d9b700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.895+0000 7f7681d9b700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 msgr2=0x7f766c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1868991041 2024-04-26T18:01:33.616 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:27.895+0000 7f7681d9b700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:28.696+0000 7f7681d9b700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 msgr2=0x7f766c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1868991041 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:28.696+0000 7f7681d9b700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:30.297+0000 7f7681d9b700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 msgr2=0x7f766c03bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1868991041 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:30.297+0000 7f7681d9b700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:31.767+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f767401b440 con 0x7f7684109ab0 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:31.768+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 msgr2=0x7f766c03bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:31.768+0000 7f76737fe700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.769+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f767402b350 con 0x7f7684109ab0 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.769+0000 7f76737fe700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f766c03cfe0 0x7f766c03f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.769+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f767400ee50 con 0x7f766c03cfe0 2024-04-26T18:01:33.617 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.770+0000 7f7681d9b700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f766c03cfe0 0x7f766c03f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.770+0000 7f7681d9b700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f766c03cfe0 0x7f766c03f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7678003a10 tx=0x7f76780092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.771+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f767400ee50 con 0x7f766c03cfe0 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.786+0000 7f76717fa700 1 -- 172.21.15.90:0/2689141120 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7668002600 con 0x7f766c03cfe0 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76737fe700 1 -- 172.21.15.90:0/2689141120 <== mgr.14152 v2:172.21.15.90:6800/3660618461 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7668002600 con 0x7f766c03cfe0 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f766c03cfe0 msgr2=0x7f766c03f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f766c03cfe0 0x7f766c03f3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f7678003a10 tx=0x7f76780092b0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 -- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 msgr2=0x7f7684109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f7684109330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7674000c00 tx=0x7f7674011aa0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 -- 172.21.15.90:0/2689141120 shutdown_connections 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f766c03cfe0 0x7f766c03f3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7684109ab0 0x7f7684109330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 --2- 172.21.15.90:0/2689141120 >> [v2:172.21.15.90:6800/1868991041,v1:172.21.15.90:6801/1868991041] conn(0x7f766c039600 0x7f766c03bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:33.618 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 -- 172.21.15.90:0/2689141120 >> 172.21.15.90:0/2689141120 conn(0x7f7684103500 msgr2=0x7f7684104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:33.619 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 -- 172.21.15.90:0/2689141120 shutdown_connections 2024-04-26T18:01:33.619 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:32.787+0000 7f76717fa700 1 -- 172.21.15.90:0/2689141120 wait complete. 2024-04-26T18:01:33.619 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-04-26T18:01:33.619 INFO:teuthology.orchestra.run.smithi090.stdout:Generating a dashboard self-signed certificate... 2024-04-26T18:01:33.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:33 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:32] ENGINE Bus STARTING 2024-04-26T18:01:33.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:33 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:32] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T18:01:33.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:33 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:32] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T18:01:33.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:33 smithi090 ceph-mon[29180]: [26/Apr/2024:18:01:32] ENGINE Bus STARTED 2024-04-26T18:01:33.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:33 smithi090 ceph-mon[29180]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T18:01:33.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:33 smithi090 ceph-mon[29180]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T18:01:33.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:34.995 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.067+0000 7f85c8b35700 1 Processor -- start 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.067+0000 7f85c8b35700 1 -- start start 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.068+0000 7f85c8b35700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c410be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.068+0000 7f85c8b35700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85c407ce60 con 0x7f85c4109aa0 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.068+0000 7f85c259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c410be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.068+0000 7f85c259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c410be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42474/0 (socket says 172.21.15.90:42474) 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.068+0000 7f85c259c700 1 -- 172.21.15.90:0/3525114031 learned_addr learned my addr 172.21.15.90:0/3525114031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.068+0000 7f85c259c700 1 -- 172.21.15.90:0/3525114031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c410c3d0 con 0x7f85c4109aa0 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.068+0000 7f85c259c700 1 --2- 172.21.15.90:0/3525114031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c410be90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85b40098a0 tx=0x7f85b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d63a1005a33f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c159a700 1 -- 172.21.15.90:0/3525114031 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b4004160 con 0x7f85c4109aa0 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c159a700 1 -- 172.21.15.90:0/3525114031 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b4004300 con 0x7f85c4109aa0 2024-04-26T18:01:34.996 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c159a700 1 -- 172.21.15.90:0/3525114031 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b4011a20 con 0x7f85c4109aa0 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c8b35700 1 -- 172.21.15.90:0/3525114031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 msgr2=0x7f85c410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c8b35700 1 --2- 172.21.15.90:0/3525114031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c410be90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85b40098a0 tx=0x7f85b4009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c8b35700 1 -- 172.21.15.90:0/3525114031 shutdown_connections 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c8b35700 1 --2- 172.21.15.90:0/3525114031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c410be90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c8b35700 1 -- 172.21.15.90:0/3525114031 >> 172.21.15.90:0/3525114031 conn(0x7f85c41034f0 msgr2=0x7f85c4105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c8b35700 1 -- 172.21.15.90:0/3525114031 shutdown_connections 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.069+0000 7f85c8b35700 1 -- 172.21.15.90:0/3525114031 wait complete. 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.070+0000 7f85c8b35700 1 Processor -- start 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.070+0000 7f85c8b35700 1 -- start start 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c8b35700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c8b35700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f85c410d370 con 0x7f85c4109aa0 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:34.997 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42488/0 (socket says 172.21.15.90:42488) 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c259c700 1 -- 172.21.15.90:0/1935250819 learned_addr learned my addr 172.21.15.90:0/1935250819 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c259c700 1 -- 172.21.15.90:0/1935250819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85c419d330 con 0x7f85c4109aa0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c259c700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c419cdf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85b4011770 tx=0x7f85b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85b37fe700 1 -- 172.21.15.90:0/1935250819 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b4003b20 con 0x7f85c4109aa0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85b37fe700 1 -- 172.21.15.90:0/1935250819 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85b401b8f0 con 0x7f85c4109aa0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85c419d620 con 0x7f85c4109aa0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85b37fe700 1 -- 172.21.15.90:0/1935250819 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85b40238c0 con 0x7f85c4109aa0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.071+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85c41a02b0 con 0x7f85c4109aa0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.072+0000 7f85b37fe700 1 -- 172.21.15.90:0/1935250819 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f85b4023b70 con 0x7f85c4109aa0 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.072+0000 7f85b37fe700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f85ac0394a0 0x7f85ac03b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:34.998 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.072+0000 7f85b37fe700 1 -- 172.21.15.90:0/1935250819 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f85b404df20 con 0x7f85c4109aa0 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.073+0000 7f85c1d9b700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f85ac0394a0 0x7f85ac03b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.073+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85c4064a80 con 0x7f85c4109aa0 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.073+0000 7f85c1d9b700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f85ac0394a0 0x7f85ac03b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006fd0 tx=0x7f85b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.078+0000 7f85b37fe700 1 -- 172.21.15.90:0/1935250819 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85b401f030 con 0x7f85c4109aa0 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.309+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f85c4104730 con 0x7f85ac0394a0 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.402+0000 7f85b37fe700 1 -- 172.21.15.90:0/1935250819 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f85c4104730 con 0x7f85ac0394a0 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.404+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f85ac0394a0 msgr2=0x7f85ac03b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.404+0000 7f85c8b35700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f85ac0394a0 0x7f85ac03b960 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f85b8006fd0 tx=0x7f85b8006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.404+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 msgr2=0x7f85c419cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.404+0000 7f85c8b35700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c419cdf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f85b4011770 tx=0x7f85b4011aa0 comp rx=0 tx=0).stop 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.405+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 shutdown_connections 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.405+0000 7f85c8b35700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f85ac0394a0 0x7f85ac03b960 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:34.999 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.405+0000 7f85c8b35700 1 --2- 172.21.15.90:0/1935250819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f85c4109aa0 0x7f85c419cdf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:35.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.405+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 >> 172.21.15.90:0/1935250819 conn(0x7f85c41034f0 msgr2=0x7f85c4104020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:35.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.405+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 shutdown_connections 2024-04-26T18:01:35.000 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:34.405+0000 7f85c8b35700 1 -- 172.21.15.90:0/1935250819 wait complete. 2024-04-26T18:01:35.000 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-04-26T18:01:35.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:35 smithi090 ceph-mon[29180]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:35 smithi090 ceph-mon[29180]: mgrmap e12: x(active, since 2s) 2024-04-26T18:01:36.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$yyJOdKlNV5eeHLAOmbhYUe9KWMWneXka4chOMFrB.WupQACHLJiq2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714154495, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T18:01:36.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.386+0000 7ff2d88cf700 1 Processor -- start 2024-04-26T18:01:36.421 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.386+0000 7ff2d88cf700 1 -- start start 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.386+0000 7ff2d88cf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d410bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.386+0000 7ff2d88cf700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d407ce40 con 0x7ff2d4109af0 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.386+0000 7ff2d259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d410bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.386+0000 7ff2d259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d410bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42500/0 (socket says 172.21.15.90:42500) 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.386+0000 7ff2d259c700 1 -- 172.21.15.90:0/3785323408 learned_addr learned my addr 172.21.15.90:0/3785323408 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.387+0000 7ff2d259c700 1 -- 172.21.15.90:0/3785323408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d410c420 con 0x7ff2d4109af0 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.387+0000 7ff2d259c700 1 --2- 172.21.15.90:0/3785323408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d410bee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009e80 tx=0x7ff2c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641ff1f1ca0c3341 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.387+0000 7ff2d159a700 1 -- 172.21.15.90:0/3785323408 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c4017070 con 0x7ff2d4109af0 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.387+0000 7ff2d159a700 1 -- 172.21.15.90:0/3785323408 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c400b8c0 con 0x7ff2d4109af0 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.387+0000 7ff2d159a700 1 -- 172.21.15.90:0/3785323408 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c4004490 con 0x7ff2d4109af0 2024-04-26T18:01:36.422 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 -- 172.21.15.90:0/3785323408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 msgr2=0x7ff2d410bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 --2- 172.21.15.90:0/3785323408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d410bee0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4009e80 tx=0x7ff2c400b330 comp rx=0 tx=0).stop 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 -- 172.21.15.90:0/3785323408 shutdown_connections 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 --2- 172.21.15.90:0/3785323408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d410bee0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 -- 172.21.15.90:0/3785323408 >> 172.21.15.90:0/3785323408 conn(0x7ff2d4103500 msgr2=0x7ff2d4105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 -- 172.21.15.90:0/3785323408 shutdown_connections 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 -- 172.21.15.90:0/3785323408 wait complete. 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.388+0000 7ff2d88cf700 1 Processor -- start 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d88cf700 1 -- start start 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d88cf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d4109370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d88cf700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d410d420 con 0x7ff2d4109af0 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d4109370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d4109370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42508/0 (socket says 172.21.15.90:42508) 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d259c700 1 -- 172.21.15.90:0/574455532 learned_addr learned my addr 172.21.15.90:0/574455532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:36.423 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d259c700 1 -- 172.21.15.90:0/574455532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d41098b0 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d259c700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d4109370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40072a0 tx=0x7ff2c400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2cb7fe700 1 -- 172.21.15.90:0/574455532 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c4020020 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2cb7fe700 1 -- 172.21.15.90:0/574455532 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c4012430 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d4107aa0 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2cb7fe700 1 -- 172.21.15.90:0/574455532 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c4017040 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.389+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d4107fe0 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.390+0000 7ff2cb7fe700 1 -- 172.21.15.90:0/574455532 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff2c4023020 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.390+0000 7ff2cb7fe700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff2c0039610 0x7ff2c003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.390+0000 7ff2cb7fe700 1 -- 172.21.15.90:0/574455532 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff2c400f050 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.390+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d4064a80 con 0x7ff2d4109af0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.390+0000 7ff2d1d9b700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff2c0039610 0x7ff2c003bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:36.424 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.391+0000 7ff2d1d9b700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff2c0039610 0x7ff2c003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc006fd0 tx=0x7ff2bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.395+0000 7ff2cb7fe700 1 -- 172.21.15.90:0/574455532 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2c402a020 con 0x7ff2d4109af0 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.622+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff2d41081b0 con 0x7ff2c0039610 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.851+0000 7ff2cb7fe700 1 -- 172.21.15.90:0/574455532 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff2d41081b0 con 0x7ff2c0039610 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff2c0039610 msgr2=0x7ff2c003bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff2c0039610 0x7ff2c003bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc006fd0 tx=0x7ff2bc006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 msgr2=0x7ff2d4109370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d4109370 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40072a0 tx=0x7ff2c400bd50 comp rx=0 tx=0).stop 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 shutdown_connections 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff2c0039610 0x7ff2c003bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 --2- 172.21.15.90:0/574455532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2d4109af0 0x7ff2d4109370 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 >> 172.21.15.90:0/574455532 conn(0x7ff2d4103500 msgr2=0x7ff2d4104010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 shutdown_connections 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:35.853+0000 7ff2d88cf700 1 -- 172.21.15.90:0/574455532 wait complete. 2024-04-26T18:01:36.425 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-04-26T18:01:36.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:36.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:36 smithi090 ceph-mon[29180]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:36.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:37 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/646858345' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T18:01:38.137 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.860+0000 7f5a73dd4700 1 Processor -- start 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.860+0000 7f5a73dd4700 1 -- start start 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a73dd4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a73dd4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c108870 con 0x7f5a6c107f10 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a71b70700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a71b70700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42516/0 (socket says 172.21.15.90:42516) 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a71b70700 1 -- 172.21.15.90:0/900370471 learned_addr learned my addr 172.21.15.90:0/900370471 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:38.138 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a71b70700 1 -- 172.21.15.90:0/900370471 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c109070 con 0x7f5a6c107f10 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a71b70700 1 --2- 172.21.15.90:0/900370471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c108330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098a0 tx=0x7f5a5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242a3baa90be272e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a70b6e700 1 -- 172.21.15.90:0/900370471 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c004160 con 0x7f5a6c107f10 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a70b6e700 1 -- 172.21.15.90:0/900370471 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a5c004300 con 0x7f5a6c107f10 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.861+0000 7f5a70b6e700 1 -- 172.21.15.90:0/900370471 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c011a20 con 0x7f5a6c107f10 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.862+0000 7f5a73dd4700 1 -- 172.21.15.90:0/900370471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 msgr2=0x7f5a6c108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.862+0000 7f5a73dd4700 1 --2- 172.21.15.90:0/900370471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c108330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c0098a0 tx=0x7f5a5c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.862+0000 7f5a73dd4700 1 -- 172.21.15.90:0/900370471 shutdown_connections 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.862+0000 7f5a73dd4700 1 --2- 172.21.15.90:0/900370471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c108330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:38.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.862+0000 7f5a73dd4700 1 -- 172.21.15.90:0/900370471 >> 172.21.15.90:0/900370471 conn(0x7f5a6c1034f0 msgr2=0x7f5a6c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.862+0000 7f5a73dd4700 1 -- 172.21.15.90:0/900370471 shutdown_connections 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.862+0000 7f5a73dd4700 1 -- 172.21.15.90:0/900370471 wait complete. 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a73dd4700 1 Processor -- start 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a73dd4700 1 -- start start 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a73dd4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c105830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a73dd4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c10c4b0 con 0x7f5a6c107f10 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a71b70700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c105830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:38.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a71b70700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c105830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42520/0 (socket says 172.21.15.90:42520) 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a71b70700 1 -- 172.21.15.90:0/646858345 learned_addr learned my addr 172.21.15.90:0/646858345 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.863+0000 7f5a71b70700 1 -- 172.21.15.90:0/646858345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c105d70 con 0x7f5a6c107f10 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.864+0000 7f5a71b70700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c105830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c004660 tx=0x7f5a5c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.864+0000 7f5a62ffd700 1 -- 172.21.15.90:0/646858345 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c003aa0 con 0x7f5a6c107f10 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.864+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c103fb0 con 0x7f5a6c107f10 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.864+0000 7f5a62ffd700 1 -- 172.21.15.90:0/646858345 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a5c01b8f0 con 0x7f5a6c107f10 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.864+0000 7f5a62ffd700 1 -- 172.21.15.90:0/646858345 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a5c023900 con 0x7f5a6c107f10 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.864+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c104490 con 0x7f5a6c107f10 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.865+0000 7f5a62ffd700 1 -- 172.21.15.90:0/646858345 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f5a5c023bc0 con 0x7f5a6c107f10 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.865+0000 7f5a62ffd700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a58039600 0x7f5a5803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:38.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.865+0000 7f5a62ffd700 1 -- 172.21.15.90:0/646858345 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5a5c04e1e0 con 0x7f5a6c107f10 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.865+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a6c064a80 con 0x7f5a6c107f10 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.865+0000 7f5a7136f700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a58039600 0x7f5a5803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.865+0000 7f5a7136f700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a58039600 0x7f5a5803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a68006fd0 tx=0x7f5a68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:36.874+0000 7f5a62ffd700 1 -- 172.21.15.90:0/646858345 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a5c01f030 con 0x7f5a6c107f10 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.098+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f5a6c196570 con 0x7f5a6c107f10 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.098+0000 7f5a62ffd700 1 -- 172.21.15.90:0/646858345 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f5a5c010380 con 0x7f5a6c107f10 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a58039600 msgr2=0x7f5a5803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a58039600 0x7f5a5803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5a68006fd0 tx=0x7f5a68006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 msgr2=0x7f5a6c105830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:38.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c105830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c004660 tx=0x7f5a5c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:01:38.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 shutdown_connections 2024-04-26T18:01:38.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a58039600 0x7f5a5803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:38.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 --2- 172.21.15.90:0/646858345 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a6c107f10 0x7f5a6c105830 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:38.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 >> 172.21.15.90:0/646858345 conn(0x7f5a6c1034f0 msgr2=0x7f5a6c1939b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:38.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 shutdown_connections 2024-04-26T18:01:38.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:37.100+0000 7f5a73dd4700 1 -- 172.21.15.90:0/646858345 wait complete. 2024-04-26T18:01:38.143 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:01:38.144 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T18:01:38.150 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:01:38.151 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T18:01:38.151 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T18:01:38.151 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T18:01:38.152 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-04-26T18:01:38.152 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:38.152 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090:8443/ 2024-04-26T18:01:38.152 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-04-26T18:01:38.152 INFO:teuthology.orchestra.run.smithi090.stdout: Password: 42l2jdalwv 2024-04-26T18:01:38.152 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:38.153 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config directory 2024-04-26T18:01:38.153 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling autotune for osd_memory_target 2024-04-26T18:01:39.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:39 smithi090 ceph-mon[29180]: mgrmap e13: x(active, since 6s) 2024-04-26T18:01:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.990+0000 7f553b15f700 1 Processor -- start 2024-04-26T18:01:39.784 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f553b15f700 1 -- start start 2024-04-26T18:01:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f553b15f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553407d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f553b15f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f553407d8a0 con 0x7f553407ee70 2024-04-26T18:01:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f5538efb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553407d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:39.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f5538efb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553407d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42528/0 (socket says 172.21.15.90:42528) 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f5538efb700 1 -- 172.21.15.90:0/1873883087 learned_addr learned my addr 172.21.15.90:0/1873883087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f5538efb700 1 -- 172.21.15.90:0/1873883087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553407da20 con 0x7f553407ee70 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f5538efb700 1 --2- 172.21.15.90:0/1873883087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553407d2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55280098a0 tx=0x7f5528009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2b82a552713329 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.991+0000 7f55337fe700 1 -- 172.21.15.90:0/1873883087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5528004160 con 0x7f553407ee70 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f55337fe700 1 -- 172.21.15.90:0/1873883087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5528004300 con 0x7f553407ee70 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f55337fe700 1 -- 172.21.15.90:0/1873883087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5528011a20 con 0x7f553407ee70 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f553b15f700 1 -- 172.21.15.90:0/1873883087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 msgr2=0x7f553407d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f553b15f700 1 --2- 172.21.15.90:0/1873883087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553407d2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f55280098a0 tx=0x7f5528009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f553b15f700 1 -- 172.21.15.90:0/1873883087 shutdown_connections 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f553b15f700 1 --2- 172.21.15.90:0/1873883087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553407d2d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f553b15f700 1 -- 172.21.15.90:0/1873883087 >> 172.21.15.90:0/1873883087 conn(0x7f5534104610 msgr2=0x7f5534106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f553b15f700 1 -- 172.21.15.90:0/1873883087 shutdown_connections 2024-04-26T18:01:39.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.992+0000 7f553b15f700 1 -- 172.21.15.90:0/1873883087 wait complete. 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f553b15f700 1 Processor -- start 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f553b15f700 1 -- start start 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f553b15f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f553b15f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f553410f650 con 0x7f553407ee70 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f5538efb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f5538efb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42536/0 (socket says 172.21.15.90:42536) 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f5538efb700 1 -- 172.21.15.90:0/2522920801 learned_addr learned my addr 172.21.15.90:0/2522920801 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f5538efb700 1 -- 172.21.15.90:0/2522920801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553419f620 con 0x7f553407ee70 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f5538efb700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553419f0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5528009520 tx=0x7f5528011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.993+0000 7f5531ffb700 1 -- 172.21.15.90:0/2522920801 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55280039e0 con 0x7f553407ee70 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.994+0000 7f5531ffb700 1 -- 172.21.15.90:0/2522920801 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552801b8f0 con 0x7f553407ee70 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.994+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553419f8b0 con 0x7f553407ee70 2024-04-26T18:01:39.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.994+0000 7f5531ffb700 1 -- 172.21.15.90:0/2522920801 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5528019770 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.994+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55341a25a0 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.995+0000 7f5531ffb700 1 -- 172.21.15.90:0/2522920801 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5528019a70 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.995+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5534064a80 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.995+0000 7f5531ffb700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f55240396a0 0x7f552403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.995+0000 7f5531ffb700 1 -- 172.21.15.90:0/2522920801 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f552804e0c0 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.995+0000 7f5533fff700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f55240396a0 0x7f552403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:38.995+0000 7f5533fff700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f55240396a0 0x7f552403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5520006fd0 tx=0x7f5520006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.000+0000 7f5531ffb700 1 -- 172.21.15.90:0/2522920801 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f552801f030 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.198+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f55341a2c00 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.198+0000 7f5531ffb700 1 -- 172.21.15.90:0/2522920801 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f5528018b80 con 0x7f553407ee70 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f55240396a0 msgr2=0x7f552403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:39.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f55240396a0 0x7f552403bb60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5520006fd0 tx=0x7f5520006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 msgr2=0x7f553419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553419f0e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5528009520 tx=0x7f5528011aa0 comp rx=0 tx=0).stop 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 shutdown_connections 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f55240396a0 0x7f552403bb60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 --2- 172.21.15.90:0/2522920801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f553407ee70 0x7f553419f0e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 >> 172.21.15.90:0/2522920801 conn(0x7f5534104610 msgr2=0x7f5534105100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 shutdown_connections 2024-04-26T18:01:39.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:39.200+0000 7f553b15f700 1 -- 172.21.15.90:0/2522920801 wait complete. 2024-04-26T18:01:41.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c97be700 1 Processor -- start 2024-04-26T18:01:41.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c97be700 1 -- start start 2024-04-26T18:01:41.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c97be700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c4108330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:41.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c97be700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4108870 con 0x7f56c4107ef0 2024-04-26T18:01:41.106 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c4108330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c4108330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42538/0 (socket says 172.21.15.90:42538) 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c2ffd700 1 -- 172.21.15.90:0/336883337 learned_addr learned my addr 172.21.15.90:0/336883337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.177+0000 7f56c2ffd700 1 -- 172.21.15.90:0/336883337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c41090d0 con 0x7f56c4107ef0 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c2ffd700 1 --2- 172.21.15.90:0/336883337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c4108330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f56b40098a0 tx=0x7f56b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed39a66912e6c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c1ffb700 1 -- 172.21.15.90:0/336883337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b4004160 con 0x7f56c4107ef0 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c1ffb700 1 -- 172.21.15.90:0/336883337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b4004300 con 0x7f56c4107ef0 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c1ffb700 1 -- 172.21.15.90:0/336883337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b4011a20 con 0x7f56c4107ef0 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c97be700 1 -- 172.21.15.90:0/336883337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 msgr2=0x7f56c4108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c97be700 1 --2- 172.21.15.90:0/336883337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c4108330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f56b40098a0 tx=0x7f56b4009bb0 comp rx=0 tx=0).stop 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c97be700 1 -- 172.21.15.90:0/336883337 shutdown_connections 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c97be700 1 --2- 172.21.15.90:0/336883337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c4108330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:41.107 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c97be700 1 -- 172.21.15.90:0/336883337 >> 172.21.15.90:0/336883337 conn(0x7f56c4103500 msgr2=0x7f56c4105920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c97be700 1 -- 172.21.15.90:0/336883337 shutdown_connections 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.178+0000 7f56c97be700 1 -- 172.21.15.90:0/336883337 wait complete. 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.179+0000 7f56c97be700 1 Processor -- start 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.179+0000 7f56c97be700 1 -- start start 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.179+0000 7f56c97be700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c407ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.179+0000 7f56c97be700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56c410c390 con 0x7f56c4107ef0 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.179+0000 7f56c2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c407ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.179+0000 7f56c2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c407ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42544/0 (socket says 172.21.15.90:42544) 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.179+0000 7f56c2ffd700 1 -- 172.21.15.90:0/1327137052 learned_addr learned my addr 172.21.15.90:0/1327137052 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.180+0000 7f56c2ffd700 1 -- 172.21.15.90:0/1327137052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c419fc00 con 0x7f56c4107ef0 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.180+0000 7f56c2ffd700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c407ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56b4011770 tx=0x7f56b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:41.108 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.180+0000 7f56abfff700 1 -- 172.21.15.90:0/1327137052 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b40039e0 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.180+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c419fdd0 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.180+0000 7f56abfff700 1 -- 172.21.15.90:0/1327137052 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56b401b8f0 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.180+0000 7f56abfff700 1 -- 172.21.15.90:0/1327137052 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56b4023760 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.180+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c41a01f0 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.181+0000 7f56abfff700 1 -- 172.21.15.90:0/1327137052 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f56b40239d0 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.181+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c4064a80 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.181+0000 7f56abfff700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56ac039650 0x7f56ac03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.181+0000 7f56abfff700 1 -- 172.21.15.90:0/1327137052 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56b404e260 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.181+0000 7f56c27fc700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56ac039650 0x7f56ac03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.182+0000 7f56c27fc700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56ac039650 0x7f56ac03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006fd0 tx=0x7f56b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.186+0000 7f56abfff700 1 -- 172.21.15.90:0/1327137052 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56b401f030 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.490+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f56c41a0600 con 0x7f56c4107ef0 2024-04-26T18:01:41.109 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.493+0000 7f56abfff700 1 -- 172.21.15.90:0/1327137052 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f56b4018c10 con 0x7f56c4107ef0 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56ac039650 msgr2=0x7f56ac03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56ac039650 0x7f56ac03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006fd0 tx=0x7f56b8006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 msgr2=0x7f56c407ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c407ca40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56b4011770 tx=0x7f56b4011aa0 comp rx=0 tx=0).stop 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 shutdown_connections 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56ac039650 0x7f56ac03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 --2- 172.21.15.90:0/1327137052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f56c4107ef0 0x7f56c407ca40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 >> 172.21.15.90:0/1327137052 conn(0x7f56c4103500 msgr2=0x7f56c4103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 shutdown_connections 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T18:01:40.495+0000 7f56c97be700 1 -- 172.21.15.90:0/1327137052 wait complete. 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T18:01:41.110 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T18:01:41.111 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:41.112 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-04-26T18:01:41.148 INFO:tasks.cephadm:Fetching config... 2024-04-26T18:01:41.148 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:01:41.148 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T18:01:41.164 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T18:01:41.164 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:01:41.164 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T18:01:41.219 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T18:01:41.219 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:01:41.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T18:01:41.284 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T18:01:41.284 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:01:41.284 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T18:01:41.339 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T18:01:41.340 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8QMdN4h4gbYRp+CoopOxz3LKv7AxU3854VnI+eo3d683RNuuWHj2Vm+lAZRKDFsoExbQhjOomYMFnrBLCcmhjyV8SqP7aJ2fDCFzUEsA1d8DzEAdLeckenCg0grF282rLF3cfCSYbNHSlBl2l2/3xLB7vHb8A1IFe8eHl9b8LhuQ9yqc8bLHOxYVDPsYrvmayGlEyfgZtHu9E1dJ2Hj3IOdIIz4Wx3OTbV4lQXK3pwtw4chadQMJT0L12DttDCUGdYKnnrJeZhtj5R7haElR1RI7Gt1iKJlHMuaVs10l11WXk3FZYxe/WlvXPUzsP+JpkVfLOuXkYczaugjbbEU/EVko26P0QObqmaA/YGJlhaYXLLFCeiJcU4oaTp6tVS7ibBvaFLiHTJD0puE7VxxUvSSDC5QvVjHO79ZGHr4lmFTkUMmLqpWSDnv2CHwZzRGmFMKSE7IF/VRlnWconGu4I/lW9rGvj3FR6g8XMNRahsOs1mMd/tkG9kUcEu4vy/+U= ceph-b8878a18-03f6-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:01:41.415 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8QMdN4h4gbYRp+CoopOxz3LKv7AxU3854VnI+eo3d683RNuuWHj2Vm+lAZRKDFsoExbQhjOomYMFnrBLCcmhjyV8SqP7aJ2fDCFzUEsA1d8DzEAdLeckenCg0grF282rLF3cfCSYbNHSlBl2l2/3xLB7vHb8A1IFe8eHl9b8LhuQ9yqc8bLHOxYVDPsYrvmayGlEyfgZtHu9E1dJ2Hj3IOdIIz4Wx3OTbV4lQXK3pwtw4chadQMJT0L12DttDCUGdYKnnrJeZhtj5R7haElR1RI7Gt1iKJlHMuaVs10l11WXk3FZYxe/WlvXPUzsP+JpkVfLOuXkYczaugjbbEU/EVko26P0QObqmaA/YGJlhaYXLLFCeiJcU4oaTp6tVS7ibBvaFLiHTJD0puE7VxxUvSSDC5QvVjHO79ZGHr4lmFTkUMmLqpWSDnv2CHwZzRGmFMKSE7IF/VRlnWconGu4I/lW9rGvj3FR6g8XMNRahsOs1mMd/tkG9kUcEu4vy/+U= ceph-b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:01:41.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8QMdN4h4gbYRp+CoopOxz3LKv7AxU3854VnI+eo3d683RNuuWHj2Vm+lAZRKDFsoExbQhjOomYMFnrBLCcmhjyV8SqP7aJ2fDCFzUEsA1d8DzEAdLeckenCg0grF282rLF3cfCSYbNHSlBl2l2/3xLB7vHb8A1IFe8eHl9b8LhuQ9yqc8bLHOxYVDPsYrvmayGlEyfgZtHu9E1dJ2Hj3IOdIIz4Wx3OTbV4lQXK3pwtw4chadQMJT0L12DttDCUGdYKnnrJeZhtj5R7haElR1RI7Gt1iKJlHMuaVs10l11WXk3FZYxe/WlvXPUzsP+JpkVfLOuXkYczaugjbbEU/EVko26P0QObqmaA/YGJlhaYXLLFCeiJcU4oaTp6tVS7ibBvaFLiHTJD0puE7VxxUvSSDC5QvVjHO79ZGHr4lmFTkUMmLqpWSDnv2CHwZzRGmFMKSE7IF/VRlnWconGu4I/lW9rGvj3FR6g8XMNRahsOs1mMd/tkG9kUcEu4vy/+U= ceph-b8878a18-03f6-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:01:41.468 INFO:teuthology.orchestra.run.smithi119.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8QMdN4h4gbYRp+CoopOxz3LKv7AxU3854VnI+eo3d683RNuuWHj2Vm+lAZRKDFsoExbQhjOomYMFnrBLCcmhjyV8SqP7aJ2fDCFzUEsA1d8DzEAdLeckenCg0grF282rLF3cfCSYbNHSlBl2l2/3xLB7vHb8A1IFe8eHl9b8LhuQ9yqc8bLHOxYVDPsYrvmayGlEyfgZtHu9E1dJ2Hj3IOdIIz4Wx3OTbV4lQXK3pwtw4chadQMJT0L12DttDCUGdYKnnrJeZhtj5R7haElR1RI7Gt1iKJlHMuaVs10l11WXk3FZYxe/WlvXPUzsP+JpkVfLOuXkYczaugjbbEU/EVko26P0QObqmaA/YGJlhaYXLLFCeiJcU4oaTp6tVS7ibBvaFLiHTJD0puE7VxxUvSSDC5QvVjHO79ZGHr4lmFTkUMmLqpWSDnv2CHwZzRGmFMKSE7IF/VRlnWconGu4I/lW9rGvj3FR6g8XMNRahsOs1mMd/tkG9kUcEu4vy/+U= ceph-b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:01:41.479 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8QMdN4h4gbYRp+CoopOxz3LKv7AxU3854VnI+eo3d683RNuuWHj2Vm+lAZRKDFsoExbQhjOomYMFnrBLCcmhjyV8SqP7aJ2fDCFzUEsA1d8DzEAdLeckenCg0grF282rLF3cfCSYbNHSlBl2l2/3xLB7vHb8A1IFe8eHl9b8LhuQ9yqc8bLHOxYVDPsYrvmayGlEyfgZtHu9E1dJ2Hj3IOdIIz4Wx3OTbV4lQXK3pwtw4chadQMJT0L12DttDCUGdYKnnrJeZhtj5R7haElR1RI7Gt1iKJlHMuaVs10l11WXk3FZYxe/WlvXPUzsP+JpkVfLOuXkYczaugjbbEU/EVko26P0QObqmaA/YGJlhaYXLLFCeiJcU4oaTp6tVS7ibBvaFLiHTJD0puE7VxxUvSSDC5QvVjHO79ZGHr4lmFTkUMmLqpWSDnv2CHwZzRGmFMKSE7IF/VRlnWconGu4I/lW9rGvj3FR6g8XMNRahsOs1mMd/tkG9kUcEu4vy/+U= ceph-b8878a18-03f6-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:01:41.522 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC8QMdN4h4gbYRp+CoopOxz3LKv7AxU3854VnI+eo3d683RNuuWHj2Vm+lAZRKDFsoExbQhjOomYMFnrBLCcmhjyV8SqP7aJ2fDCFzUEsA1d8DzEAdLeckenCg0grF282rLF3cfCSYbNHSlBl2l2/3xLB7vHb8A1IFe8eHl9b8LhuQ9yqc8bLHOxYVDPsYrvmayGlEyfgZtHu9E1dJ2Hj3IOdIIz4Wx3OTbV4lQXK3pwtw4chadQMJT0L12DttDCUGdYKnnrJeZhtj5R7haElR1RI7Gt1iKJlHMuaVs10l11WXk3FZYxe/WlvXPUzsP+JpkVfLOuXkYczaugjbbEU/EVko26P0QObqmaA/YGJlhaYXLLFCeiJcU4oaTp6tVS7ibBvaFLiHTJD0puE7VxxUvSSDC5QvVjHO79ZGHr4lmFTkUMmLqpWSDnv2CHwZzRGmFMKSE7IF/VRlnWconGu4I/lW9rGvj3FR6g8XMNRahsOs1mMd/tkG9kUcEu4vy/+U= ceph-b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:01:41.534 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T18:01:41.783 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:01:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:41 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1327137052' entity='client.admin' 2024-04-26T18:01:44.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.195+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/441875255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 msgr2=0x7f9a78105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:44.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.195+0000 7f9a7d9a2700 1 --2- 172.21.15.90:0/441875255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78105de0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009910 tx=0x7f9a68009c20 comp rx=0 tx=0).stop 2024-04-26T18:01:44.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.195+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/441875255 shutdown_connections 2024-04-26T18:01:44.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.195+0000 7f9a7d9a2700 1 --2- 172.21.15.90:0/441875255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78105de0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:44.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.195+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/441875255 >> 172.21.15.90:0/441875255 conn(0x7f9a780fd340 msgr2=0x7f9a780ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:44.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.195+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/441875255 shutdown_connections 2024-04-26T18:01:44.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.195+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/441875255 wait complete. 2024-04-26T18:01:44.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.196+0000 7f9a7d9a2700 1 Processor -- start 2024-04-26T18:01:44.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.196+0000 7f9a7d9a2700 1 -- start start 2024-04-26T18:01:44.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.196+0000 7f9a7d9a2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a7d9a2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9a78108e00 con 0x7f9a781039f0 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a76ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37854/0 (socket says 172.21.15.90:37854) 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a76ffd700 1 -- 172.21.15.90:0/4012002496 learned_addr learned my addr 172.21.15.90:0/4012002496 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a76ffd700 1 -- 172.21.15.90:0/4012002496 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a781989b0 con 0x7f9a781039f0 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a76ffd700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78198470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a68005b40 tx=0x7f9a68004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a7c9a0700 1 -- 172.21.15.90:0/4012002496 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a68027070 con 0x7f9a781039f0 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a7c9a0700 1 -- 172.21.15.90:0/4012002496 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a6800be10 con 0x7f9a781039f0 2024-04-26T18:01:44.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a7c9a0700 1 -- 172.21.15.90:0/4012002496 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9a68022de0 con 0x7f9a781039f0 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a78198c40 con 0x7f9a781039f0 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.197+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a7819b930 con 0x7f9a781039f0 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.198+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a78109580 con 0x7f9a781039f0 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.198+0000 7f9a7c9a0700 1 -- 172.21.15.90:0/4012002496 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9a6802a070 con 0x7f9a781039f0 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.198+0000 7f9a7c9a0700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9a600396b0 0x7f9a6003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.199+0000 7f9a7c9a0700 1 -- 172.21.15.90:0/4012002496 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9a6801e030 con 0x7f9a781039f0 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.199+0000 7f9a767fc700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9a600396b0 0x7f9a6003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:44.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.199+0000 7f9a767fc700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9a600396b0 0x7f9a6003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c006fd0 tx=0x7f9a6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:44.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.204+0000 7f9a7c9a0700 1 -- 172.21.15.90:0/4012002496 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a68013070 con 0x7f9a781039f0 2024-04-26T18:01:44.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.406+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9a7802e500 con 0x7f9a781039f0 2024-04-26T18:01:44.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.408+0000 7f9a7c9a0700 1 -- 172.21.15.90:0/4012002496 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9a6800fa20 con 0x7f9a781039f0 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9a600396b0 msgr2=0x7f9a6003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9a600396b0 0x7f9a6003bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c006fd0 tx=0x7f9a6c006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 msgr2=0x7f9a78198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78198470 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9a68005b40 tx=0x7f9a68004f80 comp rx=0 tx=0).stop 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 shutdown_connections 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9a600396b0 0x7f9a6003bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 --2- 172.21.15.90:0/4012002496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9a781039f0 0x7f9a78198470 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 >> 172.21.15.90:0/4012002496 conn(0x7f9a780fd340 msgr2=0x7f9a780fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 shutdown_connections 2024-04-26T18:01:44.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:44.411+0000 7f9a7d9a2700 1 -- 172.21.15.90:0/4012002496 wait complete. 2024-04-26T18:01:45.243 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T18:01:45.243 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T18:01:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:45 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/4012002496' entity='client.admin' 2024-04-26T18:01:45.812 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:01:47.045 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:47.045 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:47.045 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:01:47.045 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:47.045 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:47.045 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:47.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.387+0000 7f9469920700 1 -- 172.21.15.90:0/2775195418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 msgr2=0x7f9464103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.387+0000 7f9469920700 1 --2- 172.21.15.90:0/2775195418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f9464103280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9454009910 tx=0x7f9454009c20 comp rx=0 tx=0).stop 2024-04-26T18:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.387+0000 7f9469920700 1 -- 172.21.15.90:0/2775195418 shutdown_connections 2024-04-26T18:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.387+0000 7f9469920700 1 --2- 172.21.15.90:0/2775195418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f9464103280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:47.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.387+0000 7f9469920700 1 -- 172.21.15.90:0/2775195418 >> 172.21.15.90:0/2775195418 conn(0x7f94640fe470 msgr2=0x7f9464100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:47.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.387+0000 7f9469920700 1 -- 172.21.15.90:0/2775195418 shutdown_connections 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.387+0000 7f9469920700 1 -- 172.21.15.90:0/2775195418 wait complete. 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9469920700 1 Processor -- start 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9469920700 1 -- start start 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9469920700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f94640770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9469920700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9464108d60 con 0x7f9464102e60 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9462ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f94640770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9462ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f94640770e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37884/0 (socket says 172.21.15.90:37884) 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9462ffd700 1 -- 172.21.15.90:0/1514521323 learned_addr learned my addr 172.21.15.90:0/1514521323 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.388+0000 7f9462ffd700 1 -- 172.21.15.90:0/1514521323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9464077620 con 0x7f9464102e60 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.389+0000 7f9462ffd700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f94640770e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9454005800 tx=0x7f9454009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.389+0000 7f946891e700 1 -- 172.21.15.90:0/1514521323 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9454027070 con 0x7f9464102e60 2024-04-26T18:01:47.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.389+0000 7f946891e700 1 -- 172.21.15.90:0/1514521323 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f945400be10 con 0x7f9464102e60 2024-04-26T18:01:47.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.389+0000 7f946891e700 1 -- 172.21.15.90:0/1514521323 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f945400f460 con 0x7f9464102e60 2024-04-26T18:01:47.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.389+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9464075810 con 0x7f9464102e60 2024-04-26T18:01:47.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.389+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9464075d50 con 0x7f9464102e60 2024-04-26T18:01:47.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.390+0000 7f946891e700 1 -- 172.21.15.90:0/1514521323 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f945402a070 con 0x7f9464102e60 2024-04-26T18:01:47.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.390+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94641094e0 con 0x7f9464102e60 2024-04-26T18:01:47.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.390+0000 7f946891e700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f944c0396b0 0x7f944c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:47.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.390+0000 7f946891e700 1 -- 172.21.15.90:0/1514521323 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f945401e030 con 0x7f9464102e60 2024-04-26T18:01:47.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.390+0000 7f94627fc700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f944c0396b0 0x7f944c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:47.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.390+0000 7f94627fc700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f944c0396b0 0x7f944c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9458006fd0 tx=0x7f9458006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:47.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.395+0000 7f946891e700 1 -- 172.21.15.90:0/1514521323 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9454013070 con 0x7f9464102e60 2024-04-26T18:01:47.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.598+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f9464191870 con 0x7f944c0396b0 2024-04-26T18:01:47.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.602+0000 7f946891e700 1 -- 172.21.15.90:0/1514521323 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9464191870 con 0x7f944c0396b0 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f944c0396b0 msgr2=0x7f944c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f944c0396b0 0x7f944c03bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9458006fd0 tx=0x7f9458006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 msgr2=0x7f94640770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f94640770e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9454005800 tx=0x7f9454009f60 comp rx=0 tx=0).stop 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 shutdown_connections 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f944c0396b0 0x7f944c03bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 --2- 172.21.15.90:0/1514521323 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9464102e60 0x7f94640770e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 >> 172.21.15.90:0/1514521323 conn(0x7f94640fe470 msgr2=0x7f94640fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:47.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 shutdown_connections 2024-04-26T18:01:47.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:47.604+0000 7f9469920700 1 -- 172.21.15.90:0/1514521323 wait complete. 2024-04-26T18:01:48.361 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi119 2024-04-26T18:01:48.362 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T18:01:48.362 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.conf 2024-04-26T18:01:48.380 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T18:01:48.380 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:01:48.438 INFO:tasks.cephadm:Adding host smithi119 to orchestrator... 2024-04-26T18:01:48.438 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch host add smithi119 2024-04-26T18:01:48.675 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:01:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:01:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:01:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T18:01:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:01:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:49.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:49 smithi090 ceph-mon[29180]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:01:49.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:49 smithi090 ceph-mon[29180]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:01:50.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 -- 172.21.15.90:0/3035058220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 msgr2=0x7f93581023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 --2- 172.21.15.90:0/3035058220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581023b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9348009910 tx=0x7f9348009c20 comp rx=0 tx=0).stop 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 -- 172.21.15.90:0/3035058220 shutdown_connections 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 --2- 172.21.15.90:0/3035058220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581023b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 -- 172.21.15.90:0/3035058220 >> 172.21.15.90:0/3035058220 conn(0x7f9358077880 msgr2=0x7f9358077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 -- 172.21.15.90:0/3035058220 shutdown_connections 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 -- 172.21.15.90:0/3035058220 wait complete. 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.173+0000 7f935e3c5700 1 Processor -- start 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f935e3c5700 1 -- start start 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f935e3c5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f935e3c5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f935810c200 con 0x7f9358101fd0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f9357fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f9357fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37900/0 (socket says 172.21.15.90:37900) 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f9357fff700 1 -- 172.21.15.90:0/1963974273 learned_addr learned my addr 172.21.15.90:0/1963974273 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f9357fff700 1 -- 172.21.15.90:0/1963974273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9358199bf0 con 0x7f9358101fd0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.174+0000 7f9357fff700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581996b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9348006010 tx=0x7f9348005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.175+0000 7f93557fa700 1 -- 172.21.15.90:0/1963974273 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9348026070 con 0x7f9358101fd0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.175+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9358199ee0 con 0x7f9358101fd0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.175+0000 7f93557fa700 1 -- 172.21.15.90:0/1963974273 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f934800bba0 con 0x7f9358101fd0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.175+0000 7f93557fa700 1 -- 172.21.15.90:0/1963974273 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9348003910 con 0x7f9358101fd0 2024-04-26T18:01:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.175+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935819a3a0 con 0x7f9358101fd0 2024-04-26T18:01:50.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.175+0000 7f93557fa700 1 -- 172.21.15.90:0/1963974273 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9348029070 con 0x7f9358101fd0 2024-04-26T18:01:50.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.176+0000 7f93557fa700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9340039650 0x7f934003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:50.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.176+0000 7f93557fa700 1 -- 172.21.15.90:0/1963974273 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f934801e070 con 0x7f9358101fd0 2024-04-26T18:01:50.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.176+0000 7f93577fe700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9340039650 0x7f934003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:50.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.176+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9358103460 con 0x7f9358101fd0 2024-04-26T18:01:50.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.176+0000 7f93577fe700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9340039650 0x7f934003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:50.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.181+0000 7f93557fa700 1 -- 172.21.15.90:0/1963974273 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9348030070 con 0x7f9358101fd0 2024-04-26T18:01:50.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:50.382+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}) -- 0x7f9358064a80 con 0x7f9340039650 2024-04-26T18:01:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:51 smithi090 ceph-mon[29180]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:01:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:52 smithi090 ceph-mon[29180]: Deploying cephadm binary to smithi119 2024-04-26T18:01:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:52 smithi090 ceph-mon[29180]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:01:54.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.453+0000 7f93557fa700 1 -- 172.21.15.90:0/1963974273 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9358064a80 con 0x7f9340039650 2024-04-26T18:01:54.453 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi119' with addr '172.21.15.119' 2024-04-26T18:01:54.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9340039650 msgr2=0x7f934003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:54.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9340039650 0x7f934003bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:54.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 msgr2=0x7f93581996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:54.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581996b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9348006010 tx=0x7f9348005780 comp rx=0 tx=0).stop 2024-04-26T18:01:54.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 shutdown_connections 2024-04-26T18:01:54.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9340039650 0x7f934003bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:54.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 --2- 172.21.15.90:0/1963974273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9358101fd0 0x7f93581996b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:54.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 >> 172.21.15.90:0/1963974273 conn(0x7f9358077880 msgr2=0x7f9358109af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:54.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 shutdown_connections 2024-04-26T18:01:54.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:54.455+0000 7f935e3c5700 1 -- 172.21.15.90:0/1963974273 wait complete. 2024-04-26T18:01:55.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:54 smithi090 ceph-mon[29180]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:01:55.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:55.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:01:55.111 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T18:01:55.343 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:01:56.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:55 smithi090 ceph-mon[29180]: Added host smithi119 2024-04-26T18:01:56.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:55 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:56.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:55 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:56.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.815+0000 7f80a78c7700 1 -- 172.21.15.90:0/3973083492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 msgr2=0x7f80a0103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.815+0000 7f80a78c7700 1 --2- 172.21.15.90:0/3973083492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0103470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f809c009910 tx=0x7f809c009c20 comp rx=0 tx=0).stop 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.815+0000 7f80a78c7700 1 -- 172.21.15.90:0/3973083492 shutdown_connections 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.815+0000 7f80a78c7700 1 --2- 172.21.15.90:0/3973083492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0103470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.815+0000 7f80a78c7700 1 -- 172.21.15.90:0/3973083492 >> 172.21.15.90:0/3973083492 conn(0x7f80a00fe960 msgr2=0x7f80a0100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.815+0000 7f80a78c7700 1 -- 172.21.15.90:0/3973083492 shutdown_connections 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.815+0000 7f80a78c7700 1 -- 172.21.15.90:0/3973083492 wait complete. 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a78c7700 1 Processor -- start 2024-04-26T18:01:56.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a78c7700 1 -- start start 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a78c7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a78c7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f80a010d310 con 0x7f80a0103090 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a5663700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a5663700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60766/0 (socket says 172.21.15.90:60766) 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a5663700 1 -- 172.21.15.90:0/76418524 learned_addr learned my addr 172.21.15.90:0/76418524 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.816+0000 7f80a5663700 1 -- 172.21.15.90:0/76418524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a0077610 con 0x7f80a0103090 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.817+0000 7f80a5663700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0075720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f809c0056d0 tx=0x7f809c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:56.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.817+0000 7f80967fc700 1 -- 172.21.15.90:0/76418524 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f809c026070 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.817+0000 7f80967fc700 1 -- 172.21.15.90:0/76418524 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809c00bc00 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.817+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a0075cc0 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.817+0000 7f80967fc700 1 -- 172.21.15.90:0/76418524 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f809c003920 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.817+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a0076180 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.818+0000 7f80967fc700 1 -- 172.21.15.90:0/76418524 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f809c029070 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.818+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a0104570 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.818+0000 7f80967fc700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f808c039650 0x7f808c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.818+0000 7f80967fc700 1 -- 172.21.15.90:0/76418524 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f809c01e070 con 0x7f80a0103090 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.818+0000 7f80a4e62700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f808c039650 0x7f808c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:56.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.818+0000 7f80a4e62700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f808c039650 0x7f808c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8090006fd0 tx=0x7f8090006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:56.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:56.823+0000 7f80967fc700 1 -- 172.21.15.90:0/76418524 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f809c013070 con 0x7f80a0103090 2024-04-26T18:01:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:56 smithi090 ceph-mon[29180]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:01:57.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.029+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80a0076410 con 0x7f808c039650 2024-04-26T18:01:57.030 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.030+0000 7f80967fc700 1 -- 172.21.15.90:0/76418524 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f80a0076410 con 0x7f808c039650 2024-04-26T18:01:57.030 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:01:57.030 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}] 2024-04-26T18:01:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f808c039650 msgr2=0x7f808c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f808c039650 0x7f808c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8090006fd0 tx=0x7f8090006e40 comp rx=0 tx=0).stop 2024-04-26T18:01:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 msgr2=0x7f80a0075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:57.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0075720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f809c0056d0 tx=0x7f809c0057b0 comp rx=0 tx=0).stop 2024-04-26T18:01:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 shutdown_connections 2024-04-26T18:01:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f808c039650 0x7f808c03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 --2- 172.21.15.90:0/76418524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80a0103090 0x7f80a0075720 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 >> 172.21.15.90:0/76418524 conn(0x7f80a00fe960 msgr2=0x7f80a0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 shutdown_connections 2024-04-26T18:01:57.035 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:57.032+0000 7f80a78c7700 1 -- 172.21.15.90:0/76418524 wait complete. 2024-04-26T18:01:57.629 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-04-26T18:01:57.629 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T18:01:57.630 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-04-26T18:01:57.648 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T18:01:57.648 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:01:57.705 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-04-26T18:01:57.705 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch host add smithi167 2024-04-26T18:01:57.937 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:01:58.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:58 smithi090 ceph-mon[29180]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:01:58.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:01:58.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:01:58 smithi090 ceph-mon[29180]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:01:59.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.331+0000 7f08f7a16700 1 -- 172.21.15.90:0/1453562046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 msgr2=0x7f08f0105120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:01:59.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.331+0000 7f08f7a16700 1 --2- 172.21.15.90:0/1453562046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0105120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009910 tx=0x7f08e0009c20 comp rx=0 tx=0).stop 2024-04-26T18:01:59.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.331+0000 7f08f7a16700 1 -- 172.21.15.90:0/1453562046 shutdown_connections 2024-04-26T18:01:59.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.331+0000 7f08f7a16700 1 --2- 172.21.15.90:0/1453562046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0105120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:01:59.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.331+0000 7f08f7a16700 1 -- 172.21.15.90:0/1453562046 >> 172.21.15.90:0/1453562046 conn(0x7f08f00fcf90 msgr2=0x7f08f00ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:01:59.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.331+0000 7f08f7a16700 1 -- 172.21.15.90:0/1453562046 shutdown_connections 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.331+0000 7f08f7a16700 1 -- 172.21.15.90:0/1453562046 wait complete. 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f7a16700 1 Processor -- start 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f7a16700 1 -- start start 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f7a16700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0102530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f7a16700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08f010d3b0 con 0x7f08f0104d40 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f57b2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0102530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f57b2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0102530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60774/0 (socket says 172.21.15.90:60774) 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f57b2700 1 -- 172.21.15.90:0/1212619115 learned_addr learned my addr 172.21.15.90:0/1212619115 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:01:59.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.332+0000 7f08f57b2700 1 -- 172.21.15.90:0/1212619115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f0104420 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.333+0000 7f08f57b2700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0102530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08e00056d0 tx=0x7f08e00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.333+0000 7f08e67fc700 1 -- 172.21.15.90:0/1212619115 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08e0026070 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.333+0000 7f08e67fc700 1 -- 172.21.15.90:0/1212619115 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e000bc00 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.333+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f0102ad0 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.333+0000 7f08e67fc700 1 -- 172.21.15.90:0/1212619115 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f08e0003920 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.333+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f0102f90 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.334+0000 7f08e67fc700 1 -- 172.21.15.90:0/1212619115 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f08e0029070 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.334+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f0064a80 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.334+0000 7f08e67fc700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f08dc039650 0x7f08dc03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.334+0000 7f08e67fc700 1 -- 172.21.15.90:0/1212619115 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f08e001e070 con 0x7f08f0104d40 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.334+0000 7f08f4fb1700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f08dc039650 0x7f08dc03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:01:59.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.334+0000 7f08f4fb1700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f08dc039650 0x7f08dc03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006fd0 tx=0x7f08ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:01:59.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.339+0000 7f08e67fc700 1 -- 172.21.15.90:0/1212619115 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08e0013070 con 0x7f08f0104d40 2024-04-26T18:01:59.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:01:59.539+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}) -- 0x7f08f0104650 con 0x7f08dc039650 2024-04-26T18:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:01 smithi090 ceph-mon[29180]: Updating smithi119:/etc/ceph/ceph.conf 2024-04-26T18:02:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:01 smithi090 ceph-mon[29180]: Deploying cephadm binary to smithi167 2024-04-26T18:02:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:01 smithi090 ceph-mon[29180]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:01 smithi090 ceph-mon[29180]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:02:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:01 smithi090 ceph-mon[29180]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:02:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:02 smithi090 ceph-mon[29180]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:03.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.744+0000 7f08e67fc700 1 -- 172.21.15.90:0/1212619115 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f08f0104650 con 0x7f08dc039650 2024-04-26T18:02:03.744 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-04-26T18:02:03.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f08dc039650 msgr2=0x7f08dc03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:03.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f08dc039650 0x7f08dc03bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f08ec006fd0 tx=0x7f08ec006e40 comp rx=0 tx=0).stop 2024-04-26T18:02:03.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 msgr2=0x7f08f0102530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:03.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0102530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f08e00056d0 tx=0x7f08e00057b0 comp rx=0 tx=0).stop 2024-04-26T18:02:03.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 shutdown_connections 2024-04-26T18:02:03.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f08dc039650 0x7f08dc03bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:03.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 --2- 172.21.15.90:0/1212619115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f0104d40 0x7f08f0102530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:03.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 >> 172.21.15.90:0/1212619115 conn(0x7f08f00fcf90 msgr2=0x7f08f010aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:03.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 shutdown_connections 2024-04-26T18:02:03.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:03.746+0000 7f08f7a16700 1 -- 172.21.15.90:0/1212619115 wait complete. 2024-04-26T18:02:04.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:04.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:04.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:04.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:04.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:04 smithi090 ceph-mon[29180]: Added host smithi167 2024-04-26T18:02:04.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:04.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:04 smithi090 ceph-mon[29180]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:04.340 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T18:02:04.572 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:02:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:06.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 -- 172.21.15.90:0/3577266206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 msgr2=0x7fa6181019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:06.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 --2- 172.21.15.90:0/3577266206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa6181019f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa608009910 tx=0x7fa608009c20 comp rx=0 tx=0).stop 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 -- 172.21.15.90:0/3577266206 shutdown_connections 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 --2- 172.21.15.90:0/3577266206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa6181019f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 -- 172.21.15.90:0/3577266206 >> 172.21.15.90:0/3577266206 conn(0x7fa6180784a0 msgr2=0x7fa6180788b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 -- 172.21.15.90:0/3577266206 shutdown_connections 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 -- 172.21.15.90:0/3577266206 wait complete. 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.098+0000 7fa61df5f700 1 Processor -- start 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa61df5f700 1 -- start start 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa61df5f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa61819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa61df5f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa61810d350 con 0x7fa618101610 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa6177fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa61819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa6177fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa61819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59586/0 (socket says 172.21.15.90:59586) 2024-04-26T18:02:06.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa6177fe700 1 -- 172.21.15.90:0/618206229 learned_addr learned my addr 172.21.15.90:0/618206229 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:02:06.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa6177fe700 1 -- 172.21.15.90:0/618206229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61819eb30 con 0x7fa618101610 2024-04-26T18:02:06.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa6177fe700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa61819e5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6080056d0 tx=0x7fa6080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:06.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.099+0000 7fa6157fa700 1 -- 172.21.15.90:0/618206229 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa608026070 con 0x7fa618101610 2024-04-26T18:02:06.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.100+0000 7fa6157fa700 1 -- 172.21.15.90:0/618206229 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa60800bba0 con 0x7fa618101610 2024-04-26T18:02:06.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.100+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61819ee20 con 0x7fa618101610 2024-04-26T18:02:06.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.100+0000 7fa6157fa700 1 -- 172.21.15.90:0/618206229 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa608003920 con 0x7fa618101610 2024-04-26T18:02:06.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.100+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa618198890 con 0x7fa618101610 2024-04-26T18:02:06.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.100+0000 7fa6157fa700 1 -- 172.21.15.90:0/618206229 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa608029070 con 0x7fa618101610 2024-04-26T18:02:06.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.100+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6181046d0 con 0x7fa618101610 2024-04-26T18:02:06.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.101+0000 7fa6157fa700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa604039650 0x7fa60403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:06.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.101+0000 7fa6157fa700 1 -- 172.21.15.90:0/618206229 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa60801e070 con 0x7fa618101610 2024-04-26T18:02:06.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.101+0000 7fa60effd700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa604039650 0x7fa60403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:06.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.101+0000 7fa60effd700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa604039650 0x7fa60403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa600006fd0 tx=0x7fa600006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:06.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.106+0000 7fa6157fa700 1 -- 172.21.15.90:0/618206229 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa608013070 con 0x7fa618101610 2024-04-26T18:02:06.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.308+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa6180008d0 con 0x7fa604039650 2024-04-26T18:02:06.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.309+0000 7fa6157fa700 1 -- 172.21.15.90:0/618206229 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fa6180008d0 con 0x7fa604039650 2024-04-26T18:02:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:02:06.309 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.119", "hostname": "smithi119", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-04-26T18:02:06.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa604039650 msgr2=0x7fa60403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:06.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa604039650 0x7fa60403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa600006fd0 tx=0x7fa600006e40 comp rx=0 tx=0).stop 2024-04-26T18:02:06.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 msgr2=0x7fa61819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:06.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa61819e5f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6080056d0 tx=0x7fa6080057b0 comp rx=0 tx=0).stop 2024-04-26T18:02:06.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 shutdown_connections 2024-04-26T18:02:06.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa604039650 0x7fa60403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:06.311 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 --2- 172.21.15.90:0/618206229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa618101610 0x7fa61819e5f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:06.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 >> 172.21.15.90:0/618206229 conn(0x7fa6180784a0 msgr2=0x7fa618104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:06.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 shutdown_connections 2024-04-26T18:02:06.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:06.311+0000 7fa61df5f700 1 -- 172.21.15.90:0/618206229 wait complete. 2024-04-26T18:02:06.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:06 smithi090 ceph-mon[29180]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:06.941 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T18:02:06.941 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T18:02:07.173 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:02:07.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:07 smithi090 ceph-mon[29180]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:02:07.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:07 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:08.515 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:08 smithi090 ceph-mon[29180]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:08.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.702+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/3684836849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 msgr2=0x7f9c88103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.702+0000 7f9c8e1f4700 1 --2- 172.21.15.90:0/3684836849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c88103470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009910 tx=0x7f9c78009c20 comp rx=0 tx=0).stop 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.702+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/3684836849 shutdown_connections 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.702+0000 7f9c8e1f4700 1 --2- 172.21.15.90:0/3684836849 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c88103470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.702+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/3684836849 >> 172.21.15.90:0/3684836849 conn(0x7f9c880fe960 msgr2=0x7f9c88100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.702+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/3684836849 shutdown_connections 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.702+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/3684836849 wait complete. 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c8e1f4700 1 Processor -- start 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c8e1f4700 1 -- start start 2024-04-26T18:02:08.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c8e1f4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c8819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c8e1f4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8810d310 con 0x7f9c88103090 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c8819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c8819e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59608/0 (socket says 172.21.15.90:59608) 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c877fe700 1 -- 172.21.15.90:0/934350485 learned_addr learned my addr 172.21.15.90:0/934350485 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c877fe700 1 -- 172.21.15.90:0/934350485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8819eaf0 con 0x7f9c88103090 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c877fe700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c8819e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c780056d0 tx=0x7f9c780057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c857fa700 1 -- 172.21.15.90:0/934350485 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78026070 con 0x7f9c88103090 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c857fa700 1 -- 172.21.15.90:0/934350485 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7800bba0 con 0x7f9c88103090 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8819ede0 con 0x7f9c88103090 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c857fa700 1 -- 172.21.15.90:0/934350485 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78003920 con 0x7f9c88103090 2024-04-26T18:02:08.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.703+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c88198850 con 0x7f9c88103090 2024-04-26T18:02:08.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.704+0000 7f9c857fa700 1 -- 172.21.15.90:0/934350485 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9c78029070 con 0x7f9c88103090 2024-04-26T18:02:08.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.704+0000 7f9c857fa700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9c74039650 0x7f9c7403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.704+0000 7f9c857fa700 1 -- 172.21.15.90:0/934350485 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9c7801e070 con 0x7f9c88103090 2024-04-26T18:02:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.704+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c88104570 con 0x7f9c88103090 2024-04-26T18:02:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.705+0000 7f9c7ffff700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9c74039650 0x7f9c7403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:08.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.705+0000 7f9c7ffff700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9c74039650 0x7f9c7403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9c70006fd0 tx=0x7f9c70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:08.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.710+0000 7f9c857fa700 1 -- 172.21.15.90:0/934350485 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9c78030070 con 0x7f9c88103090 2024-04-26T18:02:08.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:08.915+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9c88199bb0 con 0x7f9c88103090 2024-04-26T18:02:09.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.336+0000 7f9c857fa700 1 -- 172.21.15.90:0/934350485 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9c78013070 con 0x7f9c88103090 2024-04-26T18:02:09.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9c74039650 msgr2=0x7f9c7403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:09.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9c74039650 0x7f9c7403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9c70006fd0 tx=0x7f9c70006e40 comp rx=0 tx=0).stop 2024-04-26T18:02:09.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 msgr2=0x7f9c8819e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:09.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c8819e5b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9c780056d0 tx=0x7f9c780057b0 comp rx=0 tx=0).stop 2024-04-26T18:02:09.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 shutdown_connections 2024-04-26T18:02:09.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9c74039650 0x7f9c7403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:09.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 --2- 172.21.15.90:0/934350485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9c88103090 0x7f9c8819e5b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:09.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 >> 172.21.15.90:0/934350485 conn(0x7f9c880fe960 msgr2=0x7f9c88104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:09.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 shutdown_connections 2024-04-26T18:02:09.340 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:09.339+0000 7f9c8e1f4700 1 -- 172.21.15.90:0/934350485 wait complete. 2024-04-26T18:02:09.340 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-04-26T18:02:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/934350485' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T18:02:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:02:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:09.999 INFO:tasks.cephadm:Adding mon.a on smithi090 2024-04-26T18:02:09.999 INFO:tasks.cephadm:Adding mon.b on smithi119 2024-04-26T18:02:09.999 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-04-26T18:02:09.999 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi090:172.21.15.90=a;smithi119:172.21.15.119=b;smithi167:172.21.15.167=c' 2024-04-26T18:02:10.234 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: Updating smithi167:/etc/ceph/ceph.conf 2024-04-26T18:02:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/934350485' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:11.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.588+0000 7f939b3d9700 1 -- 172.21.15.167:0/1981846422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 msgr2=0x7f9394103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:11.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.588+0000 7f939b3d9700 1 --2- 172.21.15.167:0/1981846422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f9394103470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9384009910 tx=0x7f9384009c20 comp rx=0 tx=0).stop 2024-04-26T18:02:11.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.588+0000 7f939b3d9700 1 -- 172.21.15.167:0/1981846422 shutdown_connections 2024-04-26T18:02:11.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.588+0000 7f939b3d9700 1 --2- 172.21.15.167:0/1981846422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f9394103470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:11.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.588+0000 7f939b3d9700 1 -- 172.21.15.167:0/1981846422 >> 172.21.15.167:0/1981846422 conn(0x7f93940fe960 msgr2=0x7f9394100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:11.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.588+0000 7f939b3d9700 1 -- 172.21.15.167:0/1981846422 shutdown_connections 2024-04-26T18:02:11.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.588+0000 7f939b3d9700 1 -- 172.21.15.167:0/1981846422 wait complete. 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f939b3d9700 1 Processor -- start 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f939b3d9700 1 -- start start 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f939b3d9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f939419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f939b3d9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f939410d310 con 0x7f9394103090 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f9399175700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f939419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f9399175700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f939419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.167:57060/0 (socket says 172.21.15.167:57060) 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f9399175700 1 -- 172.21.15.167:0/3608482427 learned_addr learned my addr 172.21.15.167:0/3608482427 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.589+0000 7f9399175700 1 -- 172.21.15.167:0/3608482427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939419eb10 con 0x7f9394103090 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.590+0000 7f9399175700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f939419e5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93840056d0 tx=0x7f93840057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:11.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.590+0000 7f938a7fc700 1 -- 172.21.15.167:0/3608482427 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9384026070 con 0x7f9394103090 2024-04-26T18:02:11.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.590+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939419ee00 con 0x7f9394103090 2024-04-26T18:02:11.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.590+0000 7f938a7fc700 1 -- 172.21.15.167:0/3608482427 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f938400bba0 con 0x7f9394103090 2024-04-26T18:02:11.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.590+0000 7f938a7fc700 1 -- 172.21.15.167:0/3608482427 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9384003920 con 0x7f9394103090 2024-04-26T18:02:11.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.590+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9394198870 con 0x7f9394103090 2024-04-26T18:02:11.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.591+0000 7f938a7fc700 1 -- 172.21.15.167:0/3608482427 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9384029070 con 0x7f9394103090 2024-04-26T18:02:11.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.591+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9394104570 con 0x7f9394103090 2024-04-26T18:02:11.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.591+0000 7f938a7fc700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9380039650 0x7f938003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:11.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.591+0000 7f938a7fc700 1 -- 172.21.15.167:0/3608482427 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f938401e070 con 0x7f9394103090 2024-04-26T18:02:11.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.591+0000 7f9398974700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9380039650 0x7f938003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:11.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.592+0000 7f9398974700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9380039650 0x7f938003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9390006fd0 tx=0x7f9390006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:11.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.596+0000 7f938a7fc700 1 -- 172.21.15.167:0/3608482427 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9384013070 con 0x7f9394103090 2024-04-26T18:02:11.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.804+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi119:172.21.15.119=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}) -- 0x7f93940008d0 con 0x7f9380039650 2024-04-26T18:02:11.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.810+0000 7f938a7fc700 1 -- 172.21.15.167:0/3608482427 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f93940008d0 con 0x7f9380039650 2024-04-26T18:02:11.809 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-04-26T18:02:11.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9380039650 msgr2=0x7f938003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:11.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9380039650 0x7f938003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9390006fd0 tx=0x7f9390006e40 comp rx=0 tx=0).stop 2024-04-26T18:02:11.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 msgr2=0x7f939419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:11.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f939419e5d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93840056d0 tx=0x7f93840057b0 comp rx=0 tx=0).stop 2024-04-26T18:02:11.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 shutdown_connections 2024-04-26T18:02:11.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9380039650 0x7f938003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:11.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 --2- 172.21.15.167:0/3608482427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9394103090 0x7f939419e5d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:11.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 >> 172.21.15.167:0/3608482427 conn(0x7f93940fe960 msgr2=0x7f9394104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:11.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 shutdown_connections 2024-04-26T18:02:11.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:11.812+0000 7f939b3d9700 1 -- 172.21.15.167:0/3608482427 wait complete. 2024-04-26T18:02:12.445 DEBUG:teuthology.orchestra.run.smithi119:mon.b> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T18:02:12.447 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T18:02:12.450 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:02:12.450 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:02:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi119:172.21.15.119=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi119:172.21.15.119=b;smithi167:172.21.15.167=c;count:3 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:12 smithi090 ceph-mon[29180]: Deploying daemon mon.c on smithi167 2024-04-26T18:02:13.196 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:14 smithi090 ceph-mon[29180]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 -- 172.21.15.167:0/1916381002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 msgr2=0x7f5e80103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 --2- 172.21.15.167:0/1916381002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e80103470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009910 tx=0x7f5e74009c20 comp rx=0 tx=0).stop 2024-04-26T18:02:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 -- 172.21.15.167:0/1916381002 shutdown_connections 2024-04-26T18:02:15.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 --2- 172.21.15.167:0/1916381002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e80103470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 -- 172.21.15.167:0/1916381002 >> 172.21.15.167:0/1916381002 conn(0x7f5e800fe960 msgr2=0x7f5e80100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 -- 172.21.15.167:0/1916381002 shutdown_connections 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 -- 172.21.15.167:0/1916381002 wait complete. 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.497+0000 7f5e8708b700 1 Processor -- start 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.498+0000 7f5e8708b700 1 -- start start 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.498+0000 7f5e8708b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e8019e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.498+0000 7f5e8708b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8010d310 con 0x7f5e80103090 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.498+0000 7f5e84e27700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e8019e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.498+0000 7f5e84e27700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e8019e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.167:39162/0 (socket says 172.21.15.167:39162) 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.498+0000 7f5e84e27700 1 -- 172.21.15.167:0/868152753 learned_addr learned my addr 172.21.15.167:0/868152753 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.498+0000 7f5e84e27700 1 -- 172.21.15.167:0/868152753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8019eaf0 con 0x7f5e80103090 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.499+0000 7f5e84e27700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e8019e5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5e740056d0 tx=0x7f5e740057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:15.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.499+0000 7f5e7dffb700 1 -- 172.21.15.167:0/868152753 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e74026070 con 0x7f5e80103090 2024-04-26T18:02:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.499+0000 7f5e7dffb700 1 -- 172.21.15.167:0/868152753 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7400bba0 con 0x7f5e80103090 2024-04-26T18:02:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.499+0000 7f5e7dffb700 1 -- 172.21.15.167:0/868152753 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5e74003920 con 0x7f5e80103090 2024-04-26T18:02:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.499+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8019ede0 con 0x7f5e80103090 2024-04-26T18:02:15.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.499+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e80198850 con 0x7f5e80103090 2024-04-26T18:02:15.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.500+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e80104570 con 0x7f5e80103090 2024-04-26T18:02:15.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.500+0000 7f5e7dffb700 1 -- 172.21.15.167:0/868152753 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5e74029070 con 0x7f5e80103090 2024-04-26T18:02:15.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.500+0000 7f5e7dffb700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e70039650 0x7f5e7003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:15.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.500+0000 7f5e7dffb700 1 -- 172.21.15.167:0/868152753 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5e7401e070 con 0x7f5e80103090 2024-04-26T18:02:15.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.500+0000 7f5e7ffff700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e70039650 0x7f5e7003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:15.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.501+0000 7f5e7ffff700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e70039650 0x7f5e7003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006fd0 tx=0x7f5e6c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:15.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.506+0000 7f5e7dffb700 1 -- 172.21.15.167:0/868152753 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e74013070 con 0x7f5e80103090 2024-04-26T18:02:15.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.790+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5e800008d0 con 0x7f5e80103090 2024-04-26T18:02:15.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.790+0000 7f5e7dffb700 1 -- 172.21.15.167:0/868152753 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f5e740282c0 con 0x7f5e80103090 2024-04-26T18:02:15.790 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T18:02:15.790 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","modified":"2024-04-26T18:00:32.950419Z","created":"2024-04-26T18:00:32.950419Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T18:02:15.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e70039650 msgr2=0x7f5e7003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:15.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e70039650 0x7f5e7003bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c006fd0 tx=0x7f5e6c006e40 comp rx=0 tx=0).stop 2024-04-26T18:02:15.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 msgr2=0x7f5e8019e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e8019e5b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5e740056d0 tx=0x7f5e740057b0 comp rx=0 tx=0).stop 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 shutdown_connections 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e70039650 0x7f5e7003bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 --2- 172.21.15.167:0/868152753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80103090 0x7f5e8019e5b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 >> 172.21.15.167:0/868152753 conn(0x7f5e800fe960 msgr2=0x7f5e80104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 shutdown_connections 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:15.793+0000 7f5e8708b700 1 -- 172.21.15.167:0/868152753 wait complete. 2024-04-26T18:02:15.793 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-04-26T18:02:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:15 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/868152753' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:16 smithi090 ceph-mon[29180]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:17.467 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:02:17.468 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:02:17.720 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:02:22.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.470+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2595507435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 msgr2=0x7f2b40005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:22.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.470+0000 7f2b5d19f700 1 --2- 172.21.15.167:0/2595507435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b40005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2b5806a240 tx=0x7f2b4c013e90 comp rx=0 tx=0).stop 2024-04-26T18:02:22.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.470+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2595507435 shutdown_connections 2024-04-26T18:02:22.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.470+0000 7f2b5d19f700 1 --2- 172.21.15.167:0/2595507435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b40005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:22.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.470+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2595507435 >> 172.21.15.167:0/2595507435 conn(0x7f2b580fcc50 msgr2=0x7f2b580ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:22.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.470+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2595507435 shutdown_connections 2024-04-26T18:02:22.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.470+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2595507435 wait complete. 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.471+0000 7f2b5d19f700 1 Processor -- start 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.471+0000 7f2b5d19f700 1 -- start start 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.471+0000 7f2b5d19f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b5819e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.471+0000 7f2b5d19f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 0x7f2b58198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.471+0000 7f2b5d19f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5810ae60 con 0x7f2b400035c0 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.471+0000 7f2b5d19f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b5810ace0 con 0x7f2b5819ebc0 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b5659c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 0x7f2b58198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b56d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b5819e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b56d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b5819e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.167:58010/0 (socket says 172.21.15.167:58010) 2024-04-26T18:02:22.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b56d9d700 1 -- 172.21.15.167:0/2358634355 learned_addr learned my addr 172.21.15.167:0/2358634355 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b5659c700 1 -- 172.21.15.167:0/2358634355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 msgr2=0x7f2b58198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b5659c700 1 -- 172.21.15.167:0/2358634355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 msgr2=0x7f2b58198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b5659c700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 0x7f2b58198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b5659c700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 0x7f2b58198750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b56d9d700 1 -- 172.21.15.167:0/2358634355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 msgr2=0x7f2b58198750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b56d9d700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 0x7f2b58198750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b56d9d700 1 -- 172.21.15.167:0/2358634355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b5819f140 con 0x7f2b400035c0 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b56d9d700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b5819e680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c005fd0 tx=0x7f2b4c0055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b3ffff700 1 -- 172.21.15.167:0/2358634355 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2b4c00f980 con 0x7f2b400035c0 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b3ffff700 1 -- 172.21.15.167:0/2358634355 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b4c02b070 con 0x7f2b400035c0 2024-04-26T18:02:22.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.472+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b58198cf0 con 0x7f2b400035c0 2024-04-26T18:02:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.473+0000 7f2b3ffff700 1 -- 172.21.15.167:0/2358634355 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2b4c01c410 con 0x7f2b400035c0 2024-04-26T18:02:22.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.473+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b58199280 con 0x7f2b400035c0 2024-04-26T18:02:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.473+0000 7f2b3ffff700 1 -- 172.21.15.167:0/2358634355 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2b4c01c6c0 con 0x7f2b400035c0 2024-04-26T18:02:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.474+0000 7f2b3ffff700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2b44039440 0x7f2b4403b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.474+0000 7f2b3ffff700 1 -- 172.21.15.167:0/2358634355 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2b4c019070 con 0x7f2b400035c0 2024-04-26T18:02:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.474+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b58102470 con 0x7f2b400035c0 2024-04-26T18:02:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.474+0000 7f2b5659c700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2b44039440 0x7f2b4403b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:22.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.474+0000 7f2b5659c700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2b44039440 0x7f2b4403b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009850 tx=0x7f2b48008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:22.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.479+0000 7f2b3ffff700 1 -- 172.21.15.167:0/2358634355 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b4c022ca0 con 0x7f2b400035c0 2024-04-26T18:02:22.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.760+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2b58199d20 con 0x7f2b400035c0 2024-04-26T18:02:22.760 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:22.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.760+0000 7f2b3ffff700 1 -- 172.21.15.167:0/2358634355 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f2b4c024d90 con 0x7f2b400035c0 2024-04-26T18:02:22.760 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T18:02:22.760 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","modified":"2024-04-26T18:02:17.444153Z","created":"2024-04-26T18:00:32.950419Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T18:02:22.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.762+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2b44039440 msgr2=0x7f2b4403b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:22.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2b44039440 0x7f2b4403b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2b48009850 tx=0x7f2b48008040 comp rx=0 tx=0).stop 2024-04-26T18:02:22.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 msgr2=0x7f2b5819e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:22.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b5819e680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c005fd0 tx=0x7f2b4c0055b0 comp rx=0 tx=0).stop 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 shutdown_connections 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2b44039440 0x7f2b4403b900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b400035c0 0x7f2b5819e680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 --2- 172.21.15.167:0/2358634355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b5819ebc0 0x7f2b58198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 >> 172.21.15.167:0/2358634355 conn(0x7f2b580fcc50 msgr2=0x7f2b58190500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 shutdown_connections 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:22.763+0000 7f2b5d19f700 1 -- 172.21.15.167:0/2358634355 wait complete. 2024-04-26T18:02:22.763 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-04-26T18:02:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: mon.a calling monitor election 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: mon.c calling monitor election 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: monmap epoch 2 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: last_changed 2024-04-26T18:02:17.444153+0000 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: min_mon_release 19 (squid) 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: election_strategy: 1 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: fsmap 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: mgrmap e13: x(active, since 50s) 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: overall HEALTH_OK 2024-04-26T18:02:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:23 smithi167 ceph-mon[32751]: Deploying daemon mon.b on smithi119 2024-04-26T18:02:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:23 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/2358634355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:23 smithi090 ceph-mon[29180]: Deploying daemon mon.b on smithi119 2024-04-26T18:02:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:23 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/2358634355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:24.331 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:02:24.332 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:02:24.575 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:02:24.690 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:24 smithi167 ceph-mon[32751]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:24 smithi090 ceph-mon[29180]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.041+0000 7f6ce0d91700 1 -- 172.21.15.167:0/790797767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 msgr2=0x7f6cdc1044d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.041+0000 7f6ce0d91700 1 --2- 172.21.15.167:0/790797767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 0x7f6cdc1044d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009910 tx=0x7f6ccc009c20 comp rx=0 tx=0).stop 2024-04-26T18:02:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.041+0000 7f6ce0d91700 1 -- 172.21.15.167:0/790797767 shutdown_connections 2024-04-26T18:02:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.041+0000 7f6ce0d91700 1 --2- 172.21.15.167:0/790797767 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 0x7f6cdc1044d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.041+0000 7f6ce0d91700 1 -- 172.21.15.167:0/790797767 >> 172.21.15.167:0/790797767 conn(0x7f6cdc0fd920 msgr2=0x7f6cdc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.041+0000 7f6ce0d91700 1 -- 172.21.15.167:0/790797767 shutdown_connections 2024-04-26T18:02:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.041+0000 7f6ce0d91700 1 -- 172.21.15.167:0/790797767 wait complete. 2024-04-26T18:02:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6ce0d91700 1 Processor -- start 2024-04-26T18:02:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6ce0d91700 1 -- start start 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6ce0d91700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 0x7f6cdc101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6ce0d91700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 0x7f6cdc102710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6ce0d91700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc10d490 con 0x7f6cdc1040f0 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6ce0d91700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc10d310 con 0x7f6cdc102290 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6cd1d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 0x7f6cdc102710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6cd1d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 0x7f6cdc102710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:38058/0 (socket says 172.21.15.167:38058) 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6cda59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 0x7f6cdc101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.042+0000 7f6cd1d9b700 1 -- 172.21.15.167:0/3800063568 learned_addr learned my addr 172.21.15.167:0/3800063568 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cd1d9b700 1 -- 172.21.15.167:0/3800063568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 msgr2=0x7f6cdc102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cd1d9b700 1 -- 172.21.15.167:0/3800063568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 msgr2=0x7f6cdc102710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cd1d9b700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 0x7f6cdc102710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cd1d9b700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 0x7f6cdc102710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cda59c700 1 -- 172.21.15.167:0/3800063568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 msgr2=0x7f6cdc102710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cda59c700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 0x7f6cdc102710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cda59c700 1 -- 172.21.15.167:0/3800063568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc103c70 con 0x7f6cdc1040f0 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cda59c700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 0x7f6cdc101d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc014000 tx=0x7f6ccc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6ccc005dc0 con 0x7f6cdc1040f0 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc1a4e10 con 0x7f6cdc1040f0 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ccc00be10 con 0x7f6cdc1040f0 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6ccc022e30 con 0x7f6cdc1040f0 2024-04-26T18:02:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.043+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1a5260 con 0x7f6cdc1040f0 2024-04-26T18:02:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.044+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6ccc02a070 con 0x7f6cdc1040f0 2024-04-26T18:02:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.044+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cdc06bf10 con 0x7f6cdc1040f0 2024-04-26T18:02:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.044+0000 7f6cd3fff700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f6cc80393f0 0x7f6cc803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.044+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6ccc01e030 con 0x7f6cdc1040f0 2024-04-26T18:02:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.045+0000 7f6cd1d9b700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f6cc80393f0 0x7f6cc803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.045+0000 7f6cd1d9b700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f6cc80393f0 0x7f6cc803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009180 tx=0x7f6cc4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:26.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.050+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ccc013070 con 0x7f6cdc1040f0 2024-04-26T18:02:26.080 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:25 smithi119 podman[32734]: 2024-04-26 18:02:25.887831111 +0000 UTC m=+0.018641284 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:02:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.221+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc02a360 con 0x7f6cdc1040f0 2024-04-26T18:02:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:26.335+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6cdc1a5650 con 0x7f6cdc1040f0 2024-04-26T18:02:26.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 podman[32734]: 2024-04-26 18:02:26.164358667 +0000 UTC m=+0.295168825 container init e54181de0e94384942fea202886ee197b4cc3914d3f95fc85c1cbd2ffc927461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:02:26.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 podman[32734]: 2024-04-26 18:02:26.167288904 +0000 UTC m=+0.298099062 container start e54181de0e94384942fea202886ee197b4cc3914d3f95fc85c1cbd2ffc927461 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-26T18:02:26.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T18:02:26.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev), process ceph-mon, pid 2 2024-04-26T18:02:26.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pidfile_write: ignore empty --pid-file 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: load: jerasure load: lrc 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: RocksDB version: 7.9.2 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Git sha 0 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Compile date 2024-04-26 07:20:26 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: DB SUMMARY 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: DB Session ID: X047QABCOBVHI8IBUNV6 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: CURRENT file: CURRENT 2024-04-26T18:02:26.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.error_if_exists: 0 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.create_if_missing: 0 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.paranoid_checks: 1 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T18:02:26.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.env: 0x558a03ad1cc0 2024-04-26T18:02:26.369 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T18:02:26.369 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.info_log: 0x558a047d7d80 2024-04-26T18:02:26.369 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T18:02:26.371 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.statistics: (nil) 2024-04-26T18:02:26.371 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.use_fsync: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_log_file_size: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.allow_fallocate: 1 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.use_direct_reads: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T18:02:26.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.db_log_dir: 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.wal_dir: 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.write_buffer_manager: 0x558a047a90e0 2024-04-26T18:02:26.373 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.unordered_write: 0 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T18:02:26.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.row_cache: None 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.wal_filter: None 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.two_write_queues: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.wal_compression: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.atomic_flush: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.log_readahead_size: 0 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T18:02:26.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_background_jobs: 2 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_background_compactions: -1 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_subcompactions: 1 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T18:02:26.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_open_files: -1 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T18:02:26.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_background_flushes: -1 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Compression algorithms supported: 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kZSTD supported: 0 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kXpressCompression supported: 0 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kZlibCompression supported: 1 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kSnappyCompression supported: 1 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kLZ4Compression supported: 1 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: kBZip2Compression supported: 0 2024-04-26T18:02:26.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.merge_operator: 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_filter: None 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558a047d7f40) 2024-04-26T18:02:26.379 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks: 1 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: pin_top_level_index_and_filter: 1 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_type: 0 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_index_type: 0 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_shortening: 1 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: checksum: 4 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: no_block_cache: 0 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache: 0x558a047db1f0 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_name: BinnedLRUCache 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_options: 2024-04-26T18:02:26.380 INFO:journalctl@ceph.mon.b.smithi119.stdout: capacity : 536870912 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_shard_bits : 4 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: strict_capacity_limit : 0 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: high_pri_pool_ratio: 0.000 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_cache_compressed: (nil) 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: persistent_cache: (nil) 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size: 4096 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_size_deviation: 10 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_restart_interval: 16 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: index_block_restart_interval: 1 2024-04-26T18:02:26.381 INFO:journalctl@ceph.mon.b.smithi119.stdout: metadata_block_size: 4096 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: partition_filters: 0 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: use_delta_encoding: 1 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: filter_policy: bloomfilter 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: whole_key_filtering: 1 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: verify_compression: 0 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: read_amp_bytes_per_bit: 0 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: format_version: 5 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_index_compression: 1 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: block_align: 0 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: max_auto_readahead_size: 262144 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: prepopulate_block_cache: 0 2024-04-26T18:02:26.382 INFO:journalctl@ceph.mon.b.smithi119.stdout: initial_auto_readahead_size: 8192 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression: NoCompression 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.num_levels: 7 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T18:02:26.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T18:02:26.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T18:02:26.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T18:02:26.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.inplace_update_support: 0 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T18:02:26.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.bloom_locality: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.max_successive_merges: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.ttl: 2592000 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.enable_blob_files: false 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.min_blob_size: 0 2024-04-26T18:02:26.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dead0698-9927-4e7b-b24a-df783efddcaa 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714154546197072, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T18:02:26.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714154546197511, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714154546, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dead0698-9927-4e7b-b24a-df783efddcaa", "db_session_id": "X047QABCOBVHI8IBUNV6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714154546197594, "job": 1, "event": "recovery_finished"} 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558a048f8000 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: DB pointer 0x558a048e4000 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** DB Stats ** 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:02:26.390 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-04-26T18:02:26.391 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** Compaction Stats [default] ** 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:02:26.392 INFO:journalctl@ceph.mon.b.smithi119.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache BinnedLRUCache@0x558a047db1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: using public_addr v2:172.21.15.119:0/0 -> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: starting mon.b rank -1 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(???) e0 preinit fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).mds e1 new map 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout: e1 2024-04-26T18:02:26.393 INFO:journalctl@ceph.mon.b.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout: legacy client fscid: -1 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout: 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout: No filesystems configured 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T18:02:26.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Added host smithi119 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi119", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi119:/etc/ceph/ceph.conf 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Deploying cephadm binary to smithi167 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:26.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Added host smithi167 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:02:26.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/934350485' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:02:26.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi167:/etc/ceph/ceph.conf 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/934350485' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.399 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi090:172.21.15.90=a;smithi119:172.21.15.119=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Saving service mon spec with placement smithi090:172.21.15.90=a;smithi119:172.21.15.119=b;smithi167:172.21.15.167=c;count:3 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Deploying daemon mon.c on smithi167 2024-04-26T18:02:26.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/868152753' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.a calling monitor election 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.c calling monitor election 2024-04-26T18:02:26.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: monmap epoch 2 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: last_changed 2024-04-26T18:02:17.444153+0000 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:26.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: min_mon_release 19 (squid) 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: election_strategy: 1 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: fsmap 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mgrmap e13: x(active, since 50s) 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: overall HEALTH_OK 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.403 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: Deploying daemon mon.b on smithi119 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/2358634355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 ceph-mon[32752]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 bash[32734]: e54181de0e94384942fea202886ee197b4cc3914d3f95fc85c1cbd2ffc927461 2024-04-26T18:02:26.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:26 smithi119 systemd[1]: Started Ceph mon.b for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:02:31.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.229+0000 7f6cd3fff700 1 -- 172.21.15.167:0/3800063568 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f6ccc02b3b0 con 0x7f6cdc1040f0 2024-04-26T18:02:31.230 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-04-26T18:02:31.234 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","modified":"2024-04-26T18:02:26.220046Z","created":"2024-04-26T18:00:32.950419Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T18:02:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f6cc80393f0 msgr2=0x7f6cc803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f6cc80393f0 0x7f6cc803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009180 tx=0x7f6cc4008040 comp rx=0 tx=0).stop 2024-04-26T18:02:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 msgr2=0x7f6cdc101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 0x7f6cdc101d50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc014000 tx=0x7f6ccc005030 comp rx=0 tx=0).stop 2024-04-26T18:02:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 shutdown_connections 2024-04-26T18:02:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f6cc80393f0 0x7f6cc803b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6cdc1040f0 0x7f6cdc101d50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:31.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 --2- 172.21.15.167:0/3800063568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6cdc102290 0x7f6cdc102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:31.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.232+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 >> 172.21.15.167:0/3800063568 conn(0x7f6cdc0fd920 msgr2=0x7f6cdc106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:31.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.233+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 shutdown_connections 2024-04-26T18:02:31.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:31.233+0000 7f6ce0d91700 1 -- 172.21.15.167:0/3800063568 wait complete. 2024-04-26T18:02:31.236 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: mon.c calling monitor election 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: mon.a calling monitor election 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/3800063568' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: monmap epoch 3 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: last_changed 2024-04-26T18:02:26.220046+0000 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: min_mon_release 19 (squid) 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: election_strategy: 1 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-04-26T18:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: fsmap 2024-04-26T18:02:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: mgrmap e13: x(active, since 59s) 2024-04-26T18:02:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: overall HEALTH_OK 2024-04-26T18:02:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: mon.c calling monitor election 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: mon.a calling monitor election 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/3800063568' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: monmap epoch 3 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: last_changed 2024-04-26T18:02:26.220046+0000 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: min_mon_release 19 (squid) 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: election_strategy: 1 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: fsmap 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: mgrmap e13: x(active, since 59s) 2024-04-26T18:02:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: overall HEALTH_OK 2024-04-26T18:02:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:31.862 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T18:02:31.863 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T18:02:32.108 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:02:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:32 smithi167 ceph-mon[32751]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:32.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:32.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:32 smithi090 ceph-mon[29180]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:32.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:32.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:02:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.c calling monitor election 2024-04-26T18:02:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.a calling monitor election 2024-04-26T18:02:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:02:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/3800063568' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: monmap epoch 3 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: last_changed 2024-04-26T18:02:26.220046+0000 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: min_mon_release 19 (squid) 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: election_strategy: 1 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: fsmap 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mgrmap e13: x(active, since 59s) 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: overall HEALTH_OK 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:33.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:33.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.785+0000 7f4fe5580700 1 -- 172.21.15.90:0/454076823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0103090 msgr2=0x7f4fe0103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.785+0000 7f4fe5580700 1 --2- 172.21.15.90:0/454076823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0103090 0x7f4fe0103470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0009910 tx=0x7f4fd0009c20 comp rx=0 tx=0).stop 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.785+0000 7f4fe5580700 1 -- 172.21.15.90:0/454076823 shutdown_connections 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.785+0000 7f4fe5580700 1 --2- 172.21.15.90:0/454076823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe0103090 0x7f4fe0103470 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.785+0000 7f4fe5580700 1 -- 172.21.15.90:0/454076823 >> 172.21.15.90:0/454076823 conn(0x7f4fe00fe940 msgr2=0x7f4fe0100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.786+0000 7f4fe5580700 1 -- 172.21.15.90:0/454076823 shutdown_connections 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.786+0000 7f4fe5580700 1 -- 172.21.15.90:0/454076823 wait complete. 2024-04-26T18:02:33.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.786+0000 7f4fe5580700 1 Processor -- start 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.786+0000 7f4fe5580700 1 -- start start 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.786+0000 7f4fe5580700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe0103090 0x7f4fe019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.786+0000 7f4fe5580700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe019ecb0 0x7f4fe0198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.786+0000 7f4fe5580700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe0198e70 0x7f4fe0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fe5580700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe010d310 con 0x7f4fe019ecb0 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fe5580700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe010d190 con 0x7f4fe0103090 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fe5580700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fe010d490 con 0x7f4fe0198e70 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe019ecb0 0x7f4fe0198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe019ecb0 0x7f4fe0198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33108/0 (socket says 172.21.15.90:33108) 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 -- 172.21.15.90:0/3362395821 learned_addr learned my addr 172.21.15.90:0/3362395821 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fdeffd700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe0103090 0x7f4fe019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:33.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fdf7fe700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe0198e70 0x7f4fe0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 -- 172.21.15.90:0/3362395821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe0198e70 msgr2=0x7f4fe0199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe0198e70 0x7f4fe0199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 -- 172.21.15.90:0/3362395821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe0103090 msgr2=0x7f4fe019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe0103090 0x7f4fe019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 -- 172.21.15.90:0/3362395821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fe01a5640 con 0x7f4fe019ecb0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fde7fc700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe019ecb0 0x7f4fe0198930 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b780 tx=0x7f4fd400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fc7fff700 1 -- 172.21.15.90:0/3362395821 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4016070 con 0x7f4fe019ecb0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fc7fff700 1 -- 172.21.15.90:0/3362395821 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fd40036a0 con 0x7f4fe019ecb0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fc7fff700 1 -- 172.21.15.90:0/3362395821 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40119c0 con 0x7f4fe019ecb0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.787+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fe01a5870 con 0x7f4fe019ecb0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.788+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fe01a5e00 con 0x7f4fe019ecb0 2024-04-26T18:02:33.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.788+0000 7f4fc7fff700 1 -- 172.21.15.90:0/3362395821 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4fd4019070 con 0x7f4fe019ecb0 2024-04-26T18:02:33.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.789+0000 7f4fc7fff700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4fc8039ae0 0x7f4fc803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:33.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.789+0000 7f4fc7fff700 1 -- 172.21.15.90:0/3362395821 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4fd404c390 con 0x7f4fe019ecb0 2024-04-26T18:02:33.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.789+0000 7f4fdeffd700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4fc8039ae0 0x7f4fc803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:33.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.789+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fe0064a80 con 0x7f4fe019ecb0 2024-04-26T18:02:33.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.789+0000 7f4fdeffd700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4fc8039ae0 0x7f4fc803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0006010 tx=0x7f4fd001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:33.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:33.794+0000 7f4fc7fff700 1 -- 172.21.15.90:0/3362395821 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fd4003850 con 0x7f4fe019ecb0 2024-04-26T18:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: mon.b calling monitor election 2024-04-26T18:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: mon.b calling monitor election 2024-04-26T18:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: mon.c calling monitor election 2024-04-26T18:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: mon.a calling monitor election 2024-04-26T18:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: monmap epoch 3 2024-04-26T18:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: last_changed 2024-04-26T18:02:26.220046+0000 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: min_mon_release 19 (squid) 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: election_strategy: 1 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: fsmap 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: mgrmap e13: x(active, since 61s) 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: overall HEALTH_OK 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.014+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f4fe01a6090 con 0x7f4fe019ecb0 2024-04-26T18:02:34.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.014+0000 7f4fc7fff700 1 -- 172.21.15.90:0/3362395821 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f4fd401a8d0 con 0x7f4fe019ecb0 2024-04-26T18:02:34.014 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:34.014 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-04-26T18:02:34.015 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:34.015 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.016+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4fc8039ae0 msgr2=0x7f4fc803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.016+0000 7f4fe5580700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4fc8039ae0 0x7f4fc803bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4fd0006010 tx=0x7f4fd001a040 comp rx=0 tx=0).stop 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.016+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe019ecb0 msgr2=0x7f4fe0198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.016+0000 7f4fe5580700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe019ecb0 0x7f4fe0198930 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400b780 tx=0x7f4fd400ba90 comp rx=0 tx=0).stop 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.016+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 shutdown_connections 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.016+0000 7f4fe5580700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4fc8039ae0 0x7f4fc803bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.016+0000 7f4fe5580700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe0198e70 0x7f4fe0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.017+0000 7f4fe5580700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fe0103090 0x7f4fe019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.017+0000 7f4fe5580700 1 --2- 172.21.15.90:0/3362395821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4fe019ecb0 0x7f4fe0198930 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.017+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 >> 172.21.15.90:0/3362395821 conn(0x7f4fe00fe940 msgr2=0x7f4fe0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:34.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.017+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 shutdown_connections 2024-04-26T18:02:34.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:34.017+0000 7f4fe5580700 1 -- 172.21.15.90:0/3362395821 wait complete. 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.b calling monitor election 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.b calling monitor election 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.c calling monitor election 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.a calling monitor election 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: monmap epoch 3 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: last_changed 2024-04-26T18:02:26.220046+0000 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: min_mon_release 19 (squid) 2024-04-26T18:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: election_strategy: 1 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: fsmap 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: mgrmap e13: x(active, since 61s) 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: overall HEALTH_OK 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: mon.b calling monitor election 2024-04-26T18:02:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: mon.b calling monitor election 2024-04-26T18:02:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: mon.c calling monitor election 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: mon.a calling monitor election 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: monmap epoch 3 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: last_changed 2024-04-26T18:02:26.220046+0000 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: created 2024-04-26T18:00:32.950419+0000 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: min_mon_release 19 (squid) 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: election_strategy: 1 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.a 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-04-26T18:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: 2: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: fsmap 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: mgrmap e13: x(active, since 61s) 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: overall HEALTH_OK 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:02:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: Reconfiguring daemon mon.a on smithi090 2024-04-26T18:02:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3362395821' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: Reconfiguring daemon mon.a on smithi090 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3362395821' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:02:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:02:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: Reconfiguring daemon mon.a on smithi090 2024-04-26T18:02:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3362395821' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:02:35.176 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T18:02:35.176 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:02:35.176 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:02:35.202 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:02:35.202 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:02:35.267 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T18:02:35.268 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:02:35.300 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T18:02:35.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:02:35.368 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T18:02:35.368 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:02:35.400 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T18:02:35.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:02:35.467 INFO:tasks.cephadm:Adding mgr.x on smithi090 2024-04-26T18:02:35.467 INFO:tasks.cephadm:Adding mgr.y on smithi119 2024-04-26T18:02:35.468 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-04-26T18:02:35.468 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi090=x;smithi119=y;smithi167=z' 2024-04-26T18:02:35.744 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:02:36.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:36.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:36.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:02:36.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:36.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:36.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:36.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: Reconfiguring daemon mon.b on smithi119 2024-04-26T18:02:36.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:36 smithi167 ceph-mon[32751]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: Reconfiguring daemon mon.b on smithi119 2024-04-26T18:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:36 smithi119 ceph-mon[32752]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:02:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: Reconfiguring daemon mon.b on smithi119 2024-04-26T18:02:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:36 smithi090 ceph-mon[29180]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:37.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.200+0000 7f881cf14700 1 -- 172.21.15.167:0/1457352511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 msgr2=0x7f8818101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:37.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.200+0000 7f881cf14700 1 --2- 172.21.15.167:0/1457352511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f8818101a50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8808009910 tx=0x7f8808009c20 comp rx=0 tx=0).stop 2024-04-26T18:02:37.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.200+0000 7f881cf14700 1 -- 172.21.15.167:0/1457352511 shutdown_connections 2024-04-26T18:02:37.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.200+0000 7f881cf14700 1 --2- 172.21.15.167:0/1457352511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f8818101a50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:37.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.200+0000 7f881cf14700 1 -- 172.21.15.167:0/1457352511 >> 172.21.15.167:0/1457352511 conn(0x7f88180fcbd0 msgr2=0x7f88180ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:37.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.200+0000 7f881cf14700 1 -- 172.21.15.167:0/1457352511 shutdown_connections 2024-04-26T18:02:37.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.200+0000 7f881cf14700 1 -- 172.21.15.167:0/1457352511 wait complete. 2024-04-26T18:02:37.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 Processor -- start 2024-04-26T18:02:37.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 -- start start 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f88180fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f881819b450 0x7f881819d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f881819ddd0 0x7f88181a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8818107570 con 0x7f8818101630 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f88181073f0 con 0x7f881819b450 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881cf14700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f88181076f0 con 0x7f881819ddd0 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f88180fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f8816d9d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f881819ddd0 0x7f88181a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f88180fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.167:46610/0 (socket says 172.21.15.167:46610) 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881659c700 1 -- 172.21.15.167:0/2249319901 learned_addr learned my addr 172.21.15.167:0/2249319901 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f8815d9b700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f881819b450 0x7f881819d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:37.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f8816d9d700 1 -- 172.21.15.167:0/2249319901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f881819b450 msgr2=0x7f881819d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f8816d9d700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f881819b450 0x7f881819d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f8816d9d700 1 -- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 msgr2=0x7f88180fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f8816d9d700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f88180fdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f8816d9d700 1 -- 172.21.15.167:0/2249319901 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88181a0b30 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.201+0000 7f881659c700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f88180fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.202+0000 7f8816d9d700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f881819ddd0 0x7f88181a0260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f880c006fd0 tx=0x7f880c00b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.202+0000 7f88077fe700 1 -- 172.21.15.167:0/2249319901 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c015020 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.202+0000 7f88077fe700 1 -- 172.21.15.167:0/2249319901 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f880c00be30 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.202+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88181a0e20 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.202+0000 7f88077fe700 1 -- 172.21.15.167:0/2249319901 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c010be0 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.202+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88181a1350 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.203+0000 7f88077fe700 1 -- 172.21.15.167:0/2249319901 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f880c018020 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.203+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8818064a80 con 0x7f881819ddd0 2024-04-26T18:02:37.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.203+0000 7f88077fe700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8800039ae0 0x7f880003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:37.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.203+0000 7f88077fe700 1 -- 172.21.15.167:0/2249319901 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f880c04c0d0 con 0x7f881819ddd0 2024-04-26T18:02:37.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.203+0000 7f881659c700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8800039ae0 0x7f880003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:37.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.204+0000 7f881659c700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8800039ae0 0x7f880003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8808005850 tx=0x7f880801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:37.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.208+0000 7f88077fe700 1 -- 172.21.15.167:0/2249319901 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f880c01f020 con 0x7f881819ddd0 2024-04-26T18:02:37.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.419+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi119=y;smithi167=z", "target": ["mon-mgr", ""]}) -- 0x7f88180fe920 con 0x7f8800039ae0 2024-04-26T18:02:37.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.425+0000 7f88077fe700 1 -- 172.21.15.167:0/2249319901 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f88180fe920 con 0x7f8800039ae0 2024-04-26T18:02:37.426 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-04-26T18:02:37.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8800039ae0 msgr2=0x7f880003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:37.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8800039ae0 0x7f880003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8808005850 tx=0x7f880801a040 comp rx=0 tx=0).stop 2024-04-26T18:02:37.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f881819ddd0 msgr2=0x7f88181a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f881819ddd0 0x7f88181a0260 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f880c006fd0 tx=0x7f880c00b730 comp rx=0 tx=0).stop 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 shutdown_connections 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8800039ae0 0x7f880003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8818101630 0x7f88180fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f881819ddd0 0x7f88181a0260 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 --2- 172.21.15.167:0/2249319901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f881819b450 0x7f881819d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 >> 172.21.15.167:0/2249319901 conn(0x7f88180fcbd0 msgr2=0x7f88180fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 shutdown_connections 2024-04-26T18:02:37.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:02:37.428+0000 7f881cf14700 1 -- 172.21.15.167:0/2249319901 wait complete. 2024-04-26T18:02:38.044 DEBUG:teuthology.orchestra.run.smithi119:mgr.y> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T18:02:38.047 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T18:02:38.049 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T18:02:38.049 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:02:38.049 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:02:38.069 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:02:38.069 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-04-26T18:02:38.126 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:02:38.126 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:38.126 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T18:02:38.126 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:38.126 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:38.126 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 18:01:44.506053890 +0000 2024-04-26T18:02:38.127 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 17:57:04.080694802 +0000 2024-04-26T18:02:38.127 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 17:57:04.080694802 +0000 2024-04-26T18:02:38.127 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 17:57:04.080694802 +0000 2024-04-26T18:02:38.127 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:02:38.194 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T18:02:38.194 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T18:02:38.194 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000235602 s, 2.2 MB/s 2024-04-26T18:02:38.195 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:02:38.254 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-04-26T18:02:38.310 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:02:38.310 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:38.310 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T18:02:38.310 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:38.311 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:38.311 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 18:01:44.536053442 +0000 2024-04-26T18:02:38.311 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 17:57:04.418689580 +0000 2024-04-26T18:02:38.311 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 17:57:04.418689580 +0000 2024-04-26T18:02:38.311 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 17:57:04.418689580 +0000 2024-04-26T18:02:38.311 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:02:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T18:02:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T18:02:38.375 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.00027127 s, 1.9 MB/s 2024-04-26T18:02:38.377 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:02:38.434 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 18:01:44.566052993 +0000 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 17:57:04.757684343 +0000 2024-04-26T18:02:38.490 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 17:57:04.757684343 +0000 2024-04-26T18:02:38.491 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 17:57:04.757684343 +0000 2024-04-26T18:02:38.491 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:02:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:02:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:38.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: Reconfiguring daemon mon.c on smithi167 2024-04-26T18:02:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi119=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: Saving service mgr spec with placement smithi090=x;smithi119=y;smithi167=z;count:3 2024-04-26T18:02:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:38 smithi167 ceph-mon[32751]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:38.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: Reconfiguring daemon mon.c on smithi167 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi119=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:38.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: Saving service mgr spec with placement smithi090=x;smithi119=y;smithi167=z;count:3 2024-04-26T18:02:38.553 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.553 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:38 smithi090 ceph-mon[29180]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:38.553 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T18:02:38.553 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T18:02:38.554 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000220328 s, 2.3 MB/s 2024-04-26T18:02:38.555 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:02:38.612 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-04-26T18:02:38.668 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:02:38.668 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:38.668 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T18:02:38.668 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:38.668 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:38.669 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 18:01:44.595052560 +0000 2024-04-26T18:02:38.669 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 17:57:05.092679168 +0000 2024-04-26T18:02:38.669 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 17:57:05.092679168 +0000 2024-04-26T18:02:38.669 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 17:57:05.092679168 +0000 2024-04-26T18:02:38.669 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:02:38.731 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T18:02:38.731 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T18:02:38.731 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000189983 s, 2.7 MB/s 2024-04-26T18:02:38.732 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:02:38.788 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2024-04-26T18:02:38.788 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:02:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:02:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: Reconfiguring daemon mon.c on smithi167 2024-04-26T18:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi090=x;smithi119=y;smithi167=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: Saving service mgr spec with placement smithi090=x;smithi119=y;smithi167=z;count:3 2024-04-26T18:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:38 smithi119 ceph-mon[32752]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:38.806 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:02:38.806 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2024-04-26T18:02:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:02:38.863 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:38.863 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T18:02:38.863 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:38.863 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:38.863 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-26 18:02:25.674146987 +0000 2024-04-26T18:02:38.864 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-26 17:57:25.072982993 +0000 2024-04-26T18:02:38.864 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-26 17:57:25.072982993 +0000 2024-04-26T18:02:38.864 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-26 17:57:25.072982993 +0000 2024-04-26T18:02:38.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:02:38.928 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-26T18:02:38.928 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-26T18:02:38.929 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000279385 s, 1.8 MB/s 2024-04-26T18:02:38.930 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:02:38.987 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-26 18:02:25.675146962 +0000 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-26 17:57:25.414973934 +0000 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-26 17:57:25.414973934 +0000 2024-04-26T18:02:39.043 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-26 17:57:25.414973934 +0000 2024-04-26T18:02:39.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:02:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-26T18:02:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-26T18:02:39.107 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000182077 s, 2.8 MB/s 2024-04-26T18:02:39.108 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:02:39.165 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2024-04-26T18:02:39.221 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:02:39.221 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:39.221 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-26T18:02:39.221 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:39.221 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:39.222 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-26 18:02:25.675146962 +0000 2024-04-26T18:02:39.222 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-26 17:57:25.745965167 +0000 2024-04-26T18:02:39.222 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-26 17:57:25.745965167 +0000 2024-04-26T18:02:39.222 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-26 17:57:25.745965167 +0000 2024-04-26T18:02:39.222 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:02:39.287 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-26T18:02:39.287 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-26T18:02:39.287 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000258733 s, 2.0 MB/s 2024-04-26T18:02:39.288 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:02:39.346 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2024-04-26 18:02:25.675146962 +0000 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2024-04-26 17:57:26.092955975 +0000 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2024-04-26 17:57:26.092955975 +0000 2024-04-26T18:02:39.402 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2024-04-26 17:57:26.092955975 +0000 2024-04-26T18:02:39.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:02:39.465 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2024-04-26T18:02:39.465 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2024-04-26T18:02:39.465 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000224074 s, 2.3 MB/s 2024-04-26T18:02:39.466 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:02:39.523 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-04-26T18:02:39.523 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:02:39.541 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:02:39.541 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-04-26T18:02:39.599 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:02:39.599 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:39.599 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T18:02:39.600 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:39.600 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:39.600 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-26 18:02:16.869321825 +0000 2024-04-26T18:02:39.600 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-26 17:57:24.447045654 +0000 2024-04-26T18:02:39.600 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-26 17:57:24.447045654 +0000 2024-04-26T18:02:39.600 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-04-26 17:57:24.447045654 +0000 2024-04-26T18:02:39.600 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:02:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-26T18:02:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-26T18:02:39.666 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000258106 s, 2.0 MB/s 2024-04-26T18:02:39.667 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:02:39.724 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-04-26T18:02:39.779 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:02:39.779 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:39.779 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T18:02:39.779 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:39.780 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:39.780 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-26 18:02:16.870321807 +0000 2024-04-26T18:02:39.780 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-26 17:57:24.792038861 +0000 2024-04-26T18:02:39.780 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-26 17:57:24.792038861 +0000 2024-04-26T18:02:39.780 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-04-26 17:57:24.792038861 +0000 2024-04-26T18:02:39.780 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:02:39.842 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-26T18:02:39.842 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-26T18:02:39.842 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000206023 s, 2.5 MB/s 2024-04-26T18:02:39.843 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:02:39.899 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-26 18:02:16.870321807 +0000 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-26 17:57:25.153031752 +0000 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-26 17:57:25.153031752 +0000 2024-04-26T18:02:39.955 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-04-26 17:57:25.153031752 +0000 2024-04-26T18:02:39.956 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:02:40.018 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-26T18:02:40.018 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-26T18:02:40.018 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000176106 s, 2.9 MB/s 2024-04-26T18:02:40.019 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:02:40.074 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-04-26T18:02:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:02:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:02:40.130 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-26T18:02:40.131 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:02:40.131 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:02:40.131 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-04-26 18:02:16.870321807 +0000 2024-04-26T18:02:40.131 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-04-26 17:57:25.493025057 +0000 2024-04-26T18:02:40.131 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-04-26 17:57:25.493025057 +0000 2024-04-26T18:02:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-04-26 17:57:25.493025057 +0000 2024-04-26T18:02:40.131 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:02:40.194 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-04-26T18:02:40.194 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-04-26T18:02:40.195 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000194347 s, 2.6 MB/s 2024-04-26T18:02:40.196 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:02:40.251 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-04-26T18:02:40.252 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:02:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: Deploying daemon mgr.z on smithi167 2024-04-26T18:02:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:40 smithi167 ceph-mon[32751]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: Deploying daemon mgr.z on smithi167 2024-04-26T18:02:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:40 smithi090 ceph-mon[29180]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:40.494 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:02:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:02:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: Deploying daemon mgr.z on smithi167 2024-04-26T18:02:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:40 smithi119 ceph-mon[32752]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:41.818 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:41 smithi167 systemd[1]: Starting Ceph mgr.z for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:02:42.154 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:41 smithi167 podman[34581]: 2024-04-26 18:02:41.87730305 +0000 UTC m=+0.107738783 container create 3dfac15081c27d32dfab6435249463e5b7711cdbfc28062e0d85f46408fcbd1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T18:02:42.155 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:41 smithi167 podman[34581]: 2024-04-26 18:02:41.787525759 +0000 UTC m=+0.017961448 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:02:42.155 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 podman[34581]: 2024-04-26 18:02:42.07976727 +0000 UTC m=+0.310202958 container init 3dfac15081c27d32dfab6435249463e5b7711cdbfc28062e0d85f46408fcbd1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T18:02:42.155 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 podman[34581]: 2024-04-26 18:02:42.08226317 +0000 UTC m=+0.312698868 container start 3dfac15081c27d32dfab6435249463e5b7711cdbfc28062e0d85f46408fcbd1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T18:02:42.155 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mgr[34600]: -- 172.21.15.167:0/743387856 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8354dc5a0 con 0x55e8354b0c00 2024-04-26T18:02:42.155 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 bash[34581]: 3dfac15081c27d32dfab6435249463e5b7711cdbfc28062e0d85f46408fcbd1b 2024-04-26T18:02:42.304 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:02:42.304 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:02:42.304 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-26T18:02:42.304 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011561 s, 907 MB/s 2024-04-26T18:02:42.305 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-26T18:02:42.420 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 systemd[1]: Started Ceph mgr.z for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:02:42.420 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:42.234+0000 7f68a1663200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:02:42.421 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:42.419+0000 7f68a1663200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:02:42.817 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:42.521+0000 7f68a1663200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:02:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:02:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:42.941 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-04-26T18:02:43.083 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:43.083 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:43.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:43.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:43.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:43.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:02:43.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:43.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:43.175 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:02:43.236 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:42.826+0000 7f68a1663200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:02:43.236 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:42.911+0000 7f68a1663200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:43.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:43.979 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:43 smithi090 ceph-mon[29180]: Deploying daemon mgr.y on smithi119 2024-04-26T18:02:43.979 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:43 smithi119 ceph-mon[32752]: Deploying daemon mgr.y on smithi119 2024-04-26T18:02:44.153 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:43 smithi167 ceph-mon[32751]: Deploying daemon mgr.y on smithi119 2024-04-26T18:02:44.303 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 systemd[1]: Starting Ceph mgr.y for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:02:44.475 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:44.152+0000 7f68a1663200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:02:44.476 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:44.325+0000 7f68a1663200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:02:44.610 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 podman[34131]: 2024-04-26 18:02:44.280038797 +0000 UTC m=+0.018120710 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:02:44.610 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 podman[34131]: 2024-04-26 18:02:44.416821474 +0000 UTC m=+0.154903375 container create 3b41d9bc66452b2257a532e1973ff7edf26b52720e83da954012faa9c68cb795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:02:44.611 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 podman[34131]: 2024-04-26 18:02:44.611216557 +0000 UTC m=+0.349298463 container init 3b41d9bc66452b2257a532e1973ff7edf26b52720e83da954012faa9c68cb795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:02:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 -- 172.21.15.90:0/3865045600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480104680 msgr2=0x7f3480104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:44.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 --2- 172.21.15.90:0/3865045600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480104680 0x7f3480104ae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f34740099c0 tx=0x7f3474009cd0 comp rx=0 tx=0).stop 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 -- 172.21.15.90:0/3865045600 shutdown_connections 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 --2- 172.21.15.90:0/3865045600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3480105020 0x7f348010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 --2- 172.21.15.90:0/3865045600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3480103480 0x7f34801038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 --2- 172.21.15.90:0/3865045600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480104680 0x7f3480104ae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 -- 172.21.15.90:0/3865045600 >> 172.21.15.90:0/3865045600 conn(0x7f34800fea20 msgr2=0x7f3480100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 -- 172.21.15.90:0/3865045600 shutdown_connections 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 -- 172.21.15.90:0/3865045600 wait complete. 2024-04-26T18:02:44.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.730+0000 7f34854ba700 1 Processor -- start 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f34854ba700 1 -- start start 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f34854ba700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3480103480 0x7f34801995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f34854ba700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3480104680 0x7f3480199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f34854ba700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480105020 0x7f34801a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f34854ba700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f348010ef20 con 0x7f3480105020 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f34854ba700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f348010eda0 con 0x7f3480103480 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f34854ba700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f348010f0a0 con 0x7f3480104680 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480105020 0x7f34801a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3480103480 0x7f34801995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480105020 0x7f34801a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33418/0 (socket says 172.21.15.90:33418) 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 -- 172.21.15.90:0/3381936659 learned_addr learned my addr 172.21.15.90:0/3381936659 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347e7fc700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3480104680 0x7f3480199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:44.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 -- 172.21.15.90:0/3381936659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3480104680 msgr2=0x7f3480199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3480104680 0x7f3480199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 -- 172.21.15.90:0/3381936659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3480103480 msgr2=0x7f34801995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3480103480 0x7f34801995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 -- 172.21.15.90:0/3381936659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34801a1240 con 0x7f3480105020 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f347f7fe700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480105020 0x7f34801a0ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3478007f10 tx=0x7f347800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.731+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347800dd20 con 0x7f3480105020 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.732+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3478004020 con 0x7f3480105020 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.732+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34801a1530 con 0x7f3480105020 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.732+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34801a1a20 con 0x7f3480105020 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.732+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3478015660 con 0x7f3480105020 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.732+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3478015800 con 0x7f3480105020 2024-04-26T18:02:44.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.733+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346c005360 con 0x7f3480105020 2024-04-26T18:02:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.733+0000 7f3467fff700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f3468039890 0x7f346803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:02:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.733+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f347804c280 con 0x7f3480105020 2024-04-26T18:02:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.733+0000 7f347effd700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f3468039890 0x7f346803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:02:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.733+0000 7f347effd700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f3468039890 0x7f346803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f34801044e0 tx=0x7f347000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:02:44.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.738+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3478010070 con 0x7f3480105020 2024-04-26T18:02:44.790 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:44.502+0000 7f68a1663200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:02:44.790 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:44.790+0000 7f68a1663200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:02:44.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:44.954+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f346c000bf0 con 0x7f3468039890 2024-04-26T18:02:44.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:44 smithi119 ceph-mon[32752]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:44.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:44.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:44.958 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 podman[34131]: 2024-04-26 18:02:44.614177637 +0000 UTC m=+0.352259548 container start 3b41d9bc66452b2257a532e1973ff7edf26b52720e83da954012faa9c68cb795 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-04-26T18:02:44.958 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 ceph-mgr[34151]: -- 172.21.15.119:0/3959167495 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a775925a0 con 0x563a77567000 2024-04-26T18:02:44.958 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 bash[34131]: 3b41d9bc66452b2257a532e1973ff7edf26b52720e83da954012faa9c68cb795 2024-04-26T18:02:44.958 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 systemd[1]: Started Ceph mgr.y for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:02:44.958 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:44.768+0000 7fadbecd0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:02:44.958 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:44 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:44.957+0000 7fadbecd0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:02:45.067 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:44.883+0000 7f68a1663200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:02:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-mon[32751]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:44 smithi090 ceph-mon[29180]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:45.303 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:45.067+0000 7fadbecd0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:02:45.567 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:45.097+0000 7f68a1663200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:02:45.567 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:45.296+0000 7f68a1663200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:02:45.724 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:45.399+0000 7fadbecd0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:02:45.724 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:45.492+0000 7fadbecd0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:02:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-mon[32752]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:46.067 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:45.936+0000 7f68a1663200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:02:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-mon[32751]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:02:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:02:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:46.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:46.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:46.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:46.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:45 smithi090 ceph-mon[29180]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:02:46.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:02:46.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:02:46.092 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:46.666 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:46.366+0000 7f68a1663200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:02:46.666 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:46.464+0000 7f68a1663200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:02:46.666 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:46.665+0000 7f68a1663200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:02:47.053 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:46.723+0000 7fadbecd0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:46.891+0000 7fadbecd0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:47.342 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:47.059+0000 7fadbecd0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:02:47.342 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:47.342+0000 7fadbecd0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:02:47.525 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:47.191+0000 7f68a1663200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:02:47.526 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:47.283+0000 7f68a1663200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:02:47.634 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:47.433+0000 7fadbecd0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:02:47.634 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:47.633+0000 7fadbecd0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:02:47.817 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:47.525+0000 7f68a1663200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:02:47.817 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:47.657+0000 7f68a1663200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:02:47.817 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:47.752+0000 7f68a1663200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:02:48.053 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:47.824+0000 7fadbecd0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-mon[32752]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-mon[32752]: Reconfiguring daemon mgr.x on smithi090 2024-04-26T18:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3751941336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d909b27-dec0-46c5-adc0-0eb84d381e2c"}]: dispatch 2024-04-26T18:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3751941336' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d909b27-dec0-46c5-adc0-0eb84d381e2c"}]': finished 2024-04-26T18:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-mon[32752]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:47 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:02:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:47 smithi090 ceph-mon[29180]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:02:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:47 smithi090 ceph-mon[29180]: Reconfiguring daemon mgr.x on smithi090 2024-04-26T18:02:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:47 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3751941336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d909b27-dec0-46c5-adc0-0eb84d381e2c"}]: dispatch 2024-04-26T18:02:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:47 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3751941336' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d909b27-dec0-46c5-adc0-0eb84d381e2c"}]': finished 2024-04-26T18:02:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:47 smithi090 ceph-mon[29180]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:02:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:47 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:02:48.172 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:47.868+0000 7f68a1663200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:02:48.172 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:48 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:48.172+0000 7f68a1663200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:02:48.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-mon[32751]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:02:48.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-mon[32751]: Reconfiguring daemon mgr.x on smithi090 2024-04-26T18:02:48.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3751941336' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7d909b27-dec0-46c5-adc0-0eb84d381e2c"}]: dispatch 2024-04-26T18:02:48.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3751941336' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7d909b27-dec0-46c5-adc0-0eb84d381e2c"}]': finished 2024-04-26T18:02:48.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-mon[32751]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:02:48.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:47 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:02:48.803 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:48 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:48.418+0000 7fadbecd0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:02:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:48 smithi167 ceph-mon[32751]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:48 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/959659642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:02:49.068 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:48 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:48.770+0000 7f68a1663200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:02:49.068 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:02:48 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:02:48.986+0000 7f68a1663200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:02:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:48 smithi090 ceph-mon[29180]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:48 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/959659642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:02:49.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:48 smithi119 ceph-mon[32752]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:49.127 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:48 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/959659642' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:02:49.127 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:48 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:48.831+0000 7fadbecd0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:02:49.127 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:48 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:48.928+0000 7fadbecd0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:02:49.127 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:49.126+0000 7fadbecd0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:02:49.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:49.863+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f34780148c0 con 0x7f3480105020 2024-04-26T18:02:49.971 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:49.642+0000 7fadbecd0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:02:49.971 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:49.732+0000 7fadbecd0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:02:49.971 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-mon[32752]: Standby manager daemon z started 2024-04-26T18:02:49.971 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:02:49.971 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:02:49.971 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:02:49.972 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:02:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:49 smithi090 ceph-mon[29180]: Standby manager daemon z started 2024-04-26T18:02:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:49 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:02:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:49 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:02:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:49 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:02:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:49 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:02:50.303 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:49 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:49.971+0000 7fadbecd0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:02:50.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:50.100+0000 7fadbecd0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:02:50.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:50.193+0000 7fadbecd0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:02:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:49 smithi167 ceph-mon[32751]: Standby manager daemon z started 2024-04-26T18:02:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:49 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:02:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:49 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:02:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:49 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:02:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:49 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4016396704' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:02:50.605 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:50.308+0000 7fadbecd0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:02:50.605 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:50.604+0000 7fadbecd0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:02:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:50 smithi090 ceph-mon[29180]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:50 smithi090 ceph-mon[29180]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-26T18:02:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:50 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:02:51.200 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:50 smithi119 ceph-mon[32752]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:51.200 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:50 smithi119 ceph-mon[32752]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-26T18:02:51.200 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:50 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:02:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:50 smithi167 ceph-mon[32751]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:50 smithi167 ceph-mon[32751]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-26T18:02:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:50 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:02:51.553 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:51.199+0000 7fadbecd0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:02:51.553 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:02:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:02:51.411+0000 7fadbecd0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:02:51.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:02:51.874+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f347801dc10 con 0x7f3480105020 2024-04-26T18:02:52.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:51 smithi090 ceph-mon[29180]: Standby manager daemon y started 2024-04-26T18:02:52.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:51 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:02:52.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:51 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:02:52.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:51 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:02:52.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:51 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:02:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:51 smithi119 ceph-mon[32752]: Standby manager daemon y started 2024-04-26T18:02:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:51 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:02:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:51 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:02:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:51 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:02:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:51 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:02:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:51 smithi167 ceph-mon[32751]: Standby manager daemon y started 2024-04-26T18:02:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:51 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:02:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:51 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:02:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:51 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:02:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:51 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/1275374330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:02:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:52 smithi090 ceph-mon[29180]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:52 smithi090 ceph-mon[29180]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-26T18:02:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:52 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:02:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:52 smithi119 ceph-mon[32752]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:52 smithi119 ceph-mon[32752]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-26T18:02:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:52 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:02:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:52 smithi167 ceph-mon[32751]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:52 smithi167 ceph-mon[32751]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-04-26T18:02:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:52 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:02:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:54 smithi119 ceph-mon[32752]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:54 smithi167 ceph-mon[32751]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:54.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:02:54.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:54.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:02:54.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:02:54.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:54 smithi090 ceph-mon[29180]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:55 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:02:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:55 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:55 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:02:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:55 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:55 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:02:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:55 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:02:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:56 smithi090 ceph-mon[29180]: Deploying daemon osd.0 on smithi090 2024-04-26T18:02:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:56 smithi090 ceph-mon[29180]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:56 smithi119 ceph-mon[32752]: Deploying daemon osd.0 on smithi090 2024-04-26T18:02:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:56 smithi119 ceph-mon[32752]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:56 smithi167 ceph-mon[32751]: Deploying daemon osd.0 on smithi090 2024-04-26T18:02:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:56 smithi167 ceph-mon[32751]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:59.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:02:58 smithi090 ceph-mon[29180]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:02:58 smithi119 ceph-mon[32752]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:02:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:02:58 smithi167 ceph-mon[32751]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:00 smithi090 ceph-mon[29180]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:00 smithi119 ceph-mon[32752]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:00 smithi167 ceph-mon[32751]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:01 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:01 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:01 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:01 smithi119 ceph-mon[32752]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:01 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:01 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:01 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:01 smithi167 ceph-mon[32751]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:01 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:01 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:01 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:01 smithi090 ceph-mon[29180]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:04.260 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-04-26T18:03:04.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.259+0000 7f3467fff700 1 -- 172.21.15.90:0/3381936659 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f346c000bf0 con 0x7f3468039890 2024-04-26T18:03:04.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f3468039890 msgr2=0x7f346803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:04.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f3468039890 0x7f346803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f34801044e0 tx=0x7f347000b410 comp rx=0 tx=0).stop 2024-04-26T18:03:04.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480105020 msgr2=0x7f34801a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480105020 0x7f34801a0ba0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3478007f10 tx=0x7f347800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 shutdown_connections 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f3468039890 0x7f346803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3480105020 0x7f34801a0ba0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3480103480 0x7f34801995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 --2- 172.21.15.90:0/3381936659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3480104680 0x7f3480199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 >> 172.21.15.90:0/3381936659 conn(0x7f34800fea20 msgr2=0x7f3480107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 shutdown_connections 2024-04-26T18:03:04.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:04.262+0000 7f34854ba700 1 -- 172.21.15.90:0/3381936659 wait complete. 2024-04-26T18:03:04.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:04 smithi090 ceph-mon[29180]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:04.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:04.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:05.207 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T18:03:05.209 INFO:tasks.cephadm:Deploying osd.1 on smithi119 with /dev/vg_nvme/lv_4... 2024-04-26T18:03:05.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:03:05.235 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:04 smithi119 ceph-mon[32752]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:05.236 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:04 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:05.236 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:04 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:04 smithi167 ceph-mon[32751]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:04 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:04 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:05.453 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:03:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:05 smithi090 ceph-mon[29180]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:03:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:05 smithi119 ceph-mon[32752]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:03:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:05 smithi167 ceph-mon[32751]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:03:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:06 smithi090 ceph-mon[29180]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:06 smithi090 ceph-mon[29180]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:03:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:06 smithi090 ceph-mon[29180]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:03:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:06 smithi090 ceph-mon[29180]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:03:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:06 smithi119 ceph-mon[32752]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:06 smithi119 ceph-mon[32752]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:03:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:06 smithi119 ceph-mon[32752]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:03:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:06 smithi119 ceph-mon[32752]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:03:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:06 smithi167 ceph-mon[32751]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:06 smithi167 ceph-mon[32751]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:03:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:06 smithi167 ceph-mon[32751]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:03:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:06 smithi167 ceph-mon[32751]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:03:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:07.402 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:03:07.402 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:03:07.402 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-04-26T18:03:07.402 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-04-26T18:03:07.402 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0162472 s, 645 MB/s 2024-04-26T18:03:07.402 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-04-26T18:03:08.105 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi119:vg_nvme/lv_4 2024-04-26T18:03:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-mon[29180]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:03:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-mon[29180]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:03:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-mon[29180]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' 2024-04-26T18:03:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.128 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-0[43781]: 2024-04-26T18:03:07.770+0000 7f4c37454700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T18:03:08.129 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-0[43781]: 2024-04-26T18:03:07.941+0000 7f4c2ea3e700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:03:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:07 smithi119 ceph-mon[32752]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:03:08.168 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:07 smithi119 ceph-mon[32752]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:03:08.168 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:07 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.168 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:07 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.168 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:07 smithi119 ceph-mon[32752]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' 2024-04-26T18:03:08.168 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:07 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:07 smithi167 ceph-mon[32751]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:03:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:07 smithi167 ceph-mon[32751]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:03:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:07 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:07 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:07 smithi167 ceph-mon[32751]: from='osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052]' entity='osd.0' 2024-04-26T18:03:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:07 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:08.387 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:03:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:03:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:03:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 -- 172.21.15.119:0/3863468561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103f10 msgr2=0x7f056c10a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 --2- 172.21.15.119:0/3863468561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103f10 0x7f056c10a7c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f056400b0d0 tx=0x7f056400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 -- 172.21.15.119:0/3863468561 shutdown_connections 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 --2- 172.21.15.119:0/3863468561 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103f10 0x7f056c10a7c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 --2- 172.21.15.119:0/3863468561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f056c102370 0x7f056c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 --2- 172.21.15.119:0/3863468561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f056c103570 0x7f056c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 -- 172.21.15.119:0/3863468561 >> 172.21.15.119:0/3863468561 conn(0x7f056c0fd910 msgr2=0x7f056c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 -- 172.21.15.119:0/3863468561 shutdown_connections 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 -- 172.21.15.119:0/3863468561 wait complete. 2024-04-26T18:03:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 Processor -- start 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.873+0000 7f0573227700 1 -- start start 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0573227700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f056c102370 0x7f056c06c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0573227700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103570 0x7f056c19e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0573227700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f056c103f10 0x7f056c1a0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0573227700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f056c10de60 con 0x7f056c103f10 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0573227700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f056c10dce0 con 0x7f056c103570 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0573227700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f056c10dfe0 con 0x7f056c102370 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103570 0x7f056c19e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103570 0x7f056c19e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54922/0 (socket says 172.21.15.119:54922) 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 -- 172.21.15.119:0/3166329499 learned_addr learned my addr 172.21.15.119:0/3166329499 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0570fc3700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f056c102370 0x7f056c06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:09.875 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f05717c4700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f056c103f10 0x7f056c1a0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 -- 172.21.15.119:0/3166329499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f056c102370 msgr2=0x7f056c06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f056c102370 0x7f056c06c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 -- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f056c103f10 msgr2=0x7f056c1a0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f056c103f10 0x7f056c1a0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 -- 172.21.15.119:0/3166329499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f056c1a11d0 con 0x7f056c103570 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f056bfff700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103570 0x7f056c19e2b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f055800e9a0 tx=0x7f055800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0569ffb700 1 -- 172.21.15.119:0/3166329499 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055800ca10 con 0x7f056c103570 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0569ffb700 1 -- 172.21.15.119:0/3166329499 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f055800cbb0 con 0x7f056c103570 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0569ffb700 1 -- 172.21.15.119:0/3166329499 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558018680 con 0x7f056c103570 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.874+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f056c1a14c0 con 0x7f056c103570 2024-04-26T18:03:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.875+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f056c1a1a50 con 0x7f056c103570 2024-04-26T18:03:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.876+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056c06a110 con 0x7f056c103570 2024-04-26T18:03:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.877+0000 7f0569ffb700 1 -- 172.21.15.119:0/3166329499 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f0558010430 con 0x7f056c103570 2024-04-26T18:03:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.877+0000 7f0569ffb700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f055c0a3c30 0x7f055c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.877+0000 7f0569ffb700 1 -- 172.21.15.119:0/3166329499 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1767+0+0 (secure 0 0 0) 0x7f05580cea80 con 0x7f056c103570 2024-04-26T18:03:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.877+0000 7f0570fc3700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f055c0a3c30 0x7f055c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.878+0000 7f0570fc3700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f055c0a3c30 0x7f055c0a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f056c1033d0 tx=0x7f0560009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:03:09.881 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:09.882+0000 7f0569ffb700 1 -- 172.21.15.119:0/3166329499 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f055809ced0 con 0x7f056c103570 2024-04-26T18:03:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi090 2024-04-26T18:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 17088M 2024-04-26T18:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] boot 2024-04-26T18:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:09 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:10.087 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:10.088+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f056c063860 con 0x7f055c0a3c30 2024-04-26T18:03:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi090 2024-04-26T18:03:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 17088M 2024-04-26T18:03:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] boot 2024-04-26T18:03:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:03:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi090 2024-04-26T18:03:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 17088M 2024-04-26T18:03:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: osd.0 [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] boot 2024-04-26T18:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:09 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:11.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:10 smithi119 ceph-mon[32752]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:11.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:03:11.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:03:11.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:11.014 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:10 smithi119 ceph-mon[32752]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:03:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:10 smithi090 ceph-mon[29180]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:03:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:03:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:10 smithi090 ceph-mon[29180]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:03:11.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:10 smithi167 ceph-mon[32751]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:03:11.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:03:11.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:03:11.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:11.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:10 smithi167 ceph-mon[32751]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:03:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:11 smithi090 ceph-mon[29180]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:03:12.127 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:11 smithi119 ceph-mon[32752]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:03:12.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:11 smithi167 ceph-mon[32751]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:03:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:12 smithi119 ceph-mon[32752]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:12 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/1234499151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]: dispatch 2024-04-26T18:03:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:12 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]: dispatch 2024-04-26T18:03:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:12 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]': finished 2024-04-26T18:03:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:12 smithi119 ceph-mon[32752]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:03:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:12 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:12 smithi167 ceph-mon[32751]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:12 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/1234499151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]: dispatch 2024-04-26T18:03:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:12 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]: dispatch 2024-04-26T18:03:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:12 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]': finished 2024-04-26T18:03:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:12 smithi167 ceph-mon[32751]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:03:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:12 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:12 smithi090 ceph-mon[29180]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:12 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/1234499151' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]: dispatch 2024-04-26T18:03:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:12 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]: dispatch 2024-04-26T18:03:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:12 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9122fc04-ae29-4ed2-a91f-4dcb62208684"}]': finished 2024-04-26T18:03:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:12 smithi090 ceph-mon[29180]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:03:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:13 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/1238577590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:03:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:13 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/1238577590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:03:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:13 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/1238577590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:03:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:14 smithi119 ceph-mon[32752]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:14 smithi167 ceph-mon[32751]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:14 smithi090 ceph-mon[29180]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:17.274 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:16 smithi119 ceph-mon[32752]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:16 smithi167 ceph-mon[32751]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:16 smithi090 ceph-mon[29180]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:19.179 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:18 smithi119 ceph-mon[32752]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:18 smithi167 ceph-mon[32751]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:18 smithi090 ceph-mon[29180]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:21.131 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:20 smithi119 ceph-mon[32752]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:21.131 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:03:21.131 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:20 smithi167 ceph-mon[32751]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:03:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:20 smithi090 ceph-mon[29180]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:03:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:22.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:21 smithi119 ceph-mon[32752]: Deploying daemon osd.1 on smithi119 2024-04-26T18:03:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:21 smithi167 ceph-mon[32751]: Deploying daemon osd.1 on smithi119 2024-04-26T18:03:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:21 smithi090 ceph-mon[29180]: Deploying daemon osd.1 on smithi119 2024-04-26T18:03:23.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:22 smithi119 ceph-mon[32752]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:22 smithi167 ceph-mon[32751]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:22 smithi090 ceph-mon[29180]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:24.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:24 smithi119 ceph-mon[32752]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:24 smithi167 ceph-mon[32751]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:24 smithi090 ceph-mon[29180]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:26 smithi090 ceph-mon[29180]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:26.966 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:26.966 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:26.966 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:26.966 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:26 smithi119 ceph-mon[32752]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:26 smithi167 ceph-mon[32751]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:28 smithi090 ceph-mon[29180]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.149 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:28 smithi119 ceph-mon[32752]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:29.150 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.150 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.150 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:29.150 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:29.150 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:28 smithi167 ceph-mon[32751]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:29.452 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.452+0000 7f0569ffb700 1 -- 172.21.15.119:0/3166329499 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f056c063860 con 0x7f055c0a3c30 2024-04-26T18:03:29.452 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 1 on host 'smithi119' 2024-04-26T18:03:29.454 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f055c0a3c30 msgr2=0x7f055c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f055c0a3c30 0x7f055c0a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f056c1033d0 tx=0x7f0560009260 comp rx=0 tx=0).stop 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103570 msgr2=0x7f056c19e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103570 0x7f056c19e2b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f055800e9a0 tx=0x7f055800ecb0 comp rx=0 tx=0).stop 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 shutdown_connections 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f055c0a3c30 0x7f055c0a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f056c103f10 0x7f056c1a0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f056c102370 0x7f056c06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 --2- 172.21.15.119:0/3166329499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f056c103570 0x7f056c19e2b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 >> 172.21.15.119:0/3166329499 conn(0x7f056c0fd910 msgr2=0x7f056c106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 shutdown_connections 2024-04-26T18:03:29.455 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:03:29.454+0000 7f0573227700 1 -- 172.21.15.119:0/3166329499 wait complete. 2024-04-26T18:03:30.097 DEBUG:teuthology.orchestra.run.smithi119:osd.1> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T18:03:30.139 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-04-26T18:03:30.140 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:03:30.384 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:03:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:30 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:30 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:30 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:30 smithi167 ceph-mon[32751]: from='osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:03:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:30 smithi167 ceph-mon[32751]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:03:30.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:30 smithi167 ceph-mon[32751]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:30 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:30 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:30 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:30 smithi119 ceph-mon[32752]: from='osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:03:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:30 smithi119 ceph-mon[32752]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:03:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:30 smithi119 ceph-mon[32752]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:30 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:30 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:30 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:30 smithi090 ceph-mon[29180]: from='osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:03:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:30 smithi090 ceph-mon[29180]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:03:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:30 smithi090 ceph-mon[29180]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:31.698 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:31 smithi167 ceph-mon[32751]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:03:31.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:31 smithi167 ceph-mon[32751]: from='osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:03:31.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:31 smithi167 ceph-mon[32751]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:03:31.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:31.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:31 smithi167 ceph-mon[32751]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:03:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:31 smithi119 ceph-mon[32752]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:03:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:31 smithi119 ceph-mon[32752]: from='osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:03:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:31 smithi119 ceph-mon[32752]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:03:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:31 smithi119 ceph-mon[32752]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:03:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:31 smithi090 ceph-mon[29180]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:03:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:31 smithi090 ceph-mon[29180]: from='osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:03:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:31 smithi090 ceph-mon[29180]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:03:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:31 smithi090 ceph-mon[29180]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:03:32.293 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:03:32.294 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:03:32.294 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-04-26T18:03:32.294 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-04-26T18:03:32.294 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011364 s, 923 MB/s 2024-04-26T18:03:32.294 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-04-26T18:03:32.643 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:32 smithi167 ceph-mon[32751]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:03:32.643 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:32 smithi167 ceph-mon[32751]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:03:32.643 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:32.643 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:32.643 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:32 smithi167 ceph-mon[32751]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:32.643 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:32 smithi167 ceph-mon[32751]: from='osd.1 ' entity='osd.1' 2024-04-26T18:03:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-mon[32752]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:03:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-mon[32752]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:03:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-mon[32752]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-mon[32752]: from='osd.1 ' entity='osd.1' 2024-04-26T18:03:32.804 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-1[37362]: 2024-04-26T18:03:32.357+0000 7f1b2d561700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T18:03:32.804 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:32 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-1[37362]: 2024-04-26T18:03:32.510+0000 7f1b2534c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:03:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:32 smithi090 ceph-mon[29180]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:03:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:32 smithi090 ceph-mon[29180]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:03:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:32 smithi090 ceph-mon[29180]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:32 smithi090 ceph-mon[29180]: from='osd.1 ' entity='osd.1' 2024-04-26T18:03:32.930 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-04-26T18:03:33.174 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:03:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:03:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:03:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:34.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:34 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi119 2024-04-26T18:03:34.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:34 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 17088M 2024-04-26T18:03:34.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:34 smithi167 ceph-mon[32751]: osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] boot 2024-04-26T18:03:34.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:34 smithi167 ceph-mon[32751]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:03:34.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:34.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:34 smithi167 ceph-mon[32751]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:34 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi119 2024-04-26T18:03:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:34 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 17088M 2024-04-26T18:03:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:34 smithi119 ceph-mon[32752]: osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] boot 2024-04-26T18:03:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:34 smithi119 ceph-mon[32752]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:03:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:34 smithi119 ceph-mon[32752]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 -- 172.21.15.167:0/1104125468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 msgr2=0x7ff394102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 --2- 172.21.15.167:0/1104125468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff394102ab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff3840099c0 tx=0x7ff384009cd0 comp rx=0 tx=0).stop 2024-04-26T18:03:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 -- 172.21.15.167:0/1104125468 shutdown_connections 2024-04-26T18:03:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 --2- 172.21.15.167:0/1104125468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff394102ff0 0x7ff39410a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 --2- 172.21.15.167:0/1104125468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff394103910 0x7ff394103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 --2- 172.21.15.167:0/1104125468 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff394102ab0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 -- 172.21.15.167:0/1104125468 >> 172.21.15.167:0/1104125468 conn(0x7ff3940fd080 msgr2=0x7ff3940ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:03:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 -- 172.21.15.167:0/1104125468 shutdown_connections 2024-04-26T18:03:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.841+0000 7ff399911700 1 -- 172.21.15.167:0/1104125468 wait complete. 2024-04-26T18:03:34.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 Processor -- start 2024-04-26T18:03:34.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 -- start start 2024-04-26T18:03:34.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff3941020c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:34.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff394102ff0 0x7ff39419e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff394103910 0x7ff3941a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410dee0 con 0x7ff394102ff0 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410dd60 con 0x7ff394103910 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff399911700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff39410e060 con 0x7ff394102650 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff392ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff3941020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff392ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff3941020c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:37514/0 (socket says 172.21.15.167:37514) 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff392ffd700 1 -- 172.21.15.167:0/3656401434 learned_addr learned my addr 172.21.15.167:0/3656401434 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.842+0000 7ff3927fc700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff394102ff0 0x7ff39419e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:34.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff3937fe700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff394103910 0x7ff3941a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff392ffd700 1 -- 172.21.15.167:0/3656401434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff394103910 msgr2=0x7ff3941a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff392ffd700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff394103910 0x7ff3941a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff392ffd700 1 -- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff394102ff0 msgr2=0x7ff39419e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff392ffd700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff394102ff0 0x7ff39419e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff392ffd700 1 -- 172.21.15.167:0/3656401434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3941a12b0 con 0x7ff394102650 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff3927fc700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff394102ff0 0x7ff39419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff392ffd700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff3941020c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b500 tx=0x7ff38800b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff39890f700 1 -- 172.21.15.167:0/3656401434 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388015020 con 0x7ff394102650 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff39890f700 1 -- 172.21.15.167:0/3656401434 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff388011ce0 con 0x7ff394102650 2024-04-26T18:03:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3941a15a0 con 0x7ff394102650 2024-04-26T18:03:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff39890f700 1 -- 172.21.15.167:0/3656401434 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff388010d60 con 0x7ff394102650 2024-04-26T18:03:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.843+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941a1ab0 con 0x7ff394102650 2024-04-26T18:03:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.844+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff39406a110 con 0x7ff394102650 2024-04-26T18:03:34.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.845+0000 7ff39890f700 1 -- 172.21.15.167:0/3656401434 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff388018020 con 0x7ff394102650 2024-04-26T18:03:34.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.846+0000 7ff39890f700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff37c0a3c30 0x7ff37c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:34.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.846+0000 7ff39890f700 1 -- 172.21.15.167:0/3656401434 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7ff3880ce8c0 con 0x7ff394102650 2024-04-26T18:03:34.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.846+0000 7ff3927fc700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff37c0a3c30 0x7ff37c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:34.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.846+0000 7ff3927fc700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff37c0a3c30 0x7ff37c0a60f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff384000c00 tx=0x7ff384003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:03:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:34.849+0000 7ff39890f700 1 -- 172.21.15.167:0/3656401434 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff38801f020 con 0x7ff394102650 2024-04-26T18:03:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:34 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi119 2024-04-26T18:03:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:34 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 17088M 2024-04-26T18:03:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:34 smithi090 ceph-mon[29180]: osd.1 [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] boot 2024-04-26T18:03:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:34 smithi090 ceph-mon[29180]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:03:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:03:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:34 smithi090 ceph-mon[29180]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:03:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:35.057+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff394063860 con 0x7ff37c0a3c30 2024-04-26T18:03:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:35 smithi167 ceph-mon[32751]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:03:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:03:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:03:35.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:35 smithi119 ceph-mon[32752]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:03:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:03:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:03:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:35 smithi090 ceph-mon[29180]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:03:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:03:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:03:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:36.756 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:36 smithi167 ceph-mon[32751]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:03:36.756 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:36 smithi167 ceph-mon[32751]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:36 smithi119 ceph-mon[32752]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:03:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:36 smithi119 ceph-mon[32752]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:36 smithi090 ceph-mon[29180]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:03:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:36 smithi090 ceph-mon[29180]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:38.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:37 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/1504275251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]: dispatch 2024-04-26T18:03:38.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:37 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]: dispatch 2024-04-26T18:03:38.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:37 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]': finished 2024-04-26T18:03:38.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:37 smithi167 ceph-mon[32751]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:03:38.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:37 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:37 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/1504275251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]: dispatch 2024-04-26T18:03:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:37 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]: dispatch 2024-04-26T18:03:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:37 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]': finished 2024-04-26T18:03:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:37 smithi119 ceph-mon[32752]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:03:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:37 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:37 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/1504275251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]: dispatch 2024-04-26T18:03:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:37 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]: dispatch 2024-04-26T18:03:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:37 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d283b32e-2f13-4fbf-a75b-957df3b02b70"}]': finished 2024-04-26T18:03:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:37 smithi090 ceph-mon[29180]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:03:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:37 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:38 smithi119 ceph-mon[32752]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:38 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/2394517528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:03:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:38 smithi167 ceph-mon[32751]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:38 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/2394517528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:03:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:38 smithi090 ceph-mon[29180]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:38 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/2394517528' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:03:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:40 smithi090 ceph-mon[29180]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:40 smithi119 ceph-mon[32752]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:40 smithi167 ceph-mon[32751]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:43.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:42 smithi167 ceph-mon[32751]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:43.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:42 smithi119 ceph-mon[32752]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:42 smithi090 ceph-mon[29180]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:44 smithi090 ceph-mon[29180]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:45.250 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:44 smithi167 ceph-mon[32751]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:44 smithi119 ceph-mon[32752]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:46.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:03:46.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:03:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:03:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:47.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:46 smithi167 ceph-mon[32751]: Deploying daemon osd.2 on smithi167 2024-04-26T18:03:47.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:46 smithi167 ceph-mon[32751]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:46 smithi090 ceph-mon[29180]: Deploying daemon osd.2 on smithi167 2024-04-26T18:03:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:46 smithi090 ceph-mon[29180]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:46 smithi119 ceph-mon[32752]: Deploying daemon osd.2 on smithi167 2024-04-26T18:03:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:46 smithi119 ceph-mon[32752]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:48 smithi167 ceph-mon[32751]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:48 smithi090 ceph-mon[29180]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:48 smithi119 ceph-mon[32752]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:50 smithi090 ceph-mon[29180]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:50 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:50 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:50 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:51.139 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:50 smithi167 ceph-mon[32751]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:51.140 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:50 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:51.140 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:50 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:51.140 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:50 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:50 smithi119 ceph-mon[32752]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:50 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:50 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:50 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:52.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:52 smithi167 ceph-mon[32751]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:52 smithi090 ceph-mon[29180]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:52 smithi119 ceph-mon[32752]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:54.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.425+0000 7ff39890f700 1 -- 172.21.15.167:0/3656401434 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff394063860 con 0x7ff37c0a3c30 2024-04-26T18:03:54.426 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-04-26T18:03:54.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff37c0a3c30 msgr2=0x7ff37c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff37c0a3c30 0x7ff37c0a60f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff384000c00 tx=0x7ff384003860 comp rx=0 tx=0).stop 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 msgr2=0x7ff3941020c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff3941020c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b500 tx=0x7ff38800b8c0 comp rx=0 tx=0).stop 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 shutdown_connections 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff37c0a3c30 0x7ff37c0a60f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff394103910 0x7ff3941a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff394102650 0x7ff3941020c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 --2- 172.21.15.167:0/3656401434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff394102ff0 0x7ff39419e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:54.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 >> 172.21.15.167:0/3656401434 conn(0x7ff3940fd080 msgr2=0x7ff3941069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:03:54.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 shutdown_connections 2024-04-26T18:03:54.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:03:54.427+0000 7ff399911700 1 -- 172.21.15.167:0/3656401434 wait complete. 2024-04-26T18:03:54.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:54.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:54.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:54.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:03:55.040 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T18:03:55.043 INFO:tasks.cephadm:Deploying osd.3 on smithi090 with /dev/vg_nvme/lv_3... 2024-04-26T18:03:55.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:03:55.288 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:03:55.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:55 smithi090 ceph-mon[29180]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:03:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:55 smithi119 ceph-mon[32752]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:03:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:55 smithi167 ceph-mon[32751]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:03:56.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:56 smithi090 ceph-mon[29180]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:03:56.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:56 smithi090 ceph-mon[29180]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:03:56.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:56 smithi090 ceph-mon[29180]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:03:56.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:56.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:56 smithi090 ceph-mon[29180]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:56 smithi119 ceph-mon[32752]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:03:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:56 smithi119 ceph-mon[32752]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:03:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:56 smithi119 ceph-mon[32752]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:03:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:56 smithi119 ceph-mon[32752]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:56 smithi167 ceph-mon[32751]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:03:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:56 smithi167 ceph-mon[32751]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:03:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:56 smithi167 ceph-mon[32751]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:03:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:56 smithi167 ceph-mon[32751]: pgmap v78: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:03:56.818 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:56 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-2[37879]: 2024-04-26T18:03:56.525+0000 7ffa97d1a700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T18:03:56.818 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:56 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-2[37879]: 2024-04-26T18:03:56.530+0000 7ffa93b0d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:03:57.165 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:03:57.166 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:03:57.166 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-26T18:03:57.166 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-26T18:03:57.166 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116023 s, 904 MB/s 2024-04-26T18:03:57.166 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-26T18:03:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:57 smithi167 ceph-mon[32751]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:03:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:57 smithi167 ceph-mon[32751]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:03:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:57 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:57 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:57.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:57 smithi090 ceph-mon[29180]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:03:57.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:57 smithi090 ceph-mon[29180]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:03:57.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:57 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:57.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:57 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:57 smithi119 ceph-mon[32752]: from='osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:03:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:57 smithi119 ceph-mon[32752]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:03:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:57 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:57 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:57.827 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-04-26T18:03:58.088 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:03:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] boot 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:58 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:03:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:03:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] boot 2024-04-26T18:03:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:58 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: osd.2 [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] boot 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: pgmap v81: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:03:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:03:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:58 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:03:59.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 -- 172.21.15.90:0/3626548091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668103f10 msgr2=0x7f166810a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:59.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 --2- 172.21.15.90:0/3626548091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668103f10 0x7f166810a7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f166000b0d0 tx=0x7f166000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:03:59.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 -- 172.21.15.90:0/3626548091 shutdown_connections 2024-04-26T18:03:59.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 --2- 172.21.15.90:0/3626548091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668103f10 0x7f166810a7c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:59.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 --2- 172.21.15.90:0/3626548091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668102370 0x7f1668102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:59.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 --2- 172.21.15.90:0/3626548091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1668103570 0x7f16681039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 -- 172.21.15.90:0/3626548091 >> 172.21.15.90:0/3626548091 conn(0x7f16680fd910 msgr2=0x7f16680ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 -- 172.21.15.90:0/3626548091 shutdown_connections 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.653+0000 7f166c87c700 1 -- 172.21.15.90:0/3626548091 wait complete. 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 Processor -- start 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 -- start start 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668102370 0x7f1668199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668103570 0x7f1668199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1668103f10 0x7f16681a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f166810ddf0 con 0x7f1668102370 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f166810dc70 con 0x7f1668103f10 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166c87c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f166810df70 con 0x7f1668103570 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668102370 0x7f1668199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:59.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668102370 0x7f1668199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34166/0 (socket says 172.21.15.90:34166) 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f166659c700 1 -- 172.21.15.90:0/3197649787 learned_addr learned my addr 172.21.15.90:0/3197649787 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.654+0000 7f165dd9b700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668103570 0x7f1668199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166659c700 1 -- 172.21.15.90:0/3197649787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668103570 msgr2=0x7f1668199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f1666d9d700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1668103f10 0x7f16681a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166659c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668103570 0x7f1668199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166659c700 1 -- 172.21.15.90:0/3197649787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1668103f10 msgr2=0x7f16681a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166659c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1668103f10 0x7f16681a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166659c700 1 -- 172.21.15.90:0/3197649787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16681a11b0 con 0x7f1668102370 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f1666d9d700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1668103f10 0x7f16681a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166659c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668102370 0x7f1668199550 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f165800c9a0 tx=0x7f165800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658004220 con 0x7f1668102370 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16681a14a0 con 0x7f1668102370 2024-04-26T18:03:59.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1658007d10 con 0x7f1668102370 2024-04-26T18:03:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1658003e10 con 0x7f1668102370 2024-04-26T18:03:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.655+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16681a1a30 con 0x7f1668102370 2024-04-26T18:03:59.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.656+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166806a110 con 0x7f1668102370 2024-04-26T18:03:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.657+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f1658007860 con 0x7f1668102370 2024-04-26T18:03:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.658+0000 7f165ffff700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f16540a3bb0 0x7f16540a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:03:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.658+0000 7f165dd9b700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f16540a3bb0 0x7f16540a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:03:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.658+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7f16580ceef0 con 0x7f1668102370 2024-04-26T18:03:59.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.658+0000 7f165dd9b700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f16540a3bb0 0x7f16540a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1650005950 tx=0x7f16500058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:03:59.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.663+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1658014030 con 0x7f1668102370 2024-04-26T18:03:59.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi167 2024-04-26T18:03:59.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 17088M 2024-04-26T18:03:59.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:03:59.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 ceph-mon[29180]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:03:59.793 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:03:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:59 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi167 2024-04-26T18:03:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:59 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 17088M 2024-04-26T18:03:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:03:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:59 smithi119 ceph-mon[32752]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:03:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:03:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:03:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:59 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi167 2024-04-26T18:03:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:59 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 17088M 2024-04-26T18:03:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:03:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:59 smithi167 ceph-mon[32751]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:03:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:03:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:03:59.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:03:59.892+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f16680ffac0 con 0x7f16540a3bb0 2024-04-26T18:04:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48150]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:04:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48150]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48150]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48150]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 sudo[48153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:04:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 sudo[48153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 sudo[48153]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 sudo[48153]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.126 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48077]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:04:00.126 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48077]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.126 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48077]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.126 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48077]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.126 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:04:00.126 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.127 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48080]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.127 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 18:03:59 smithi090 sudo[48080]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.303 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41499]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:04:00.304 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41499]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.304 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41499]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.304 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41499]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.304 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41502]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:04:00.304 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41502]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.305 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41502]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.305 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 26 18:03:59 smithi119 sudo[41502]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41505]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41505]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41505]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41508]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41508]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41508]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 sudo[41508]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.317 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41479]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:04:00.317 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41479]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.318 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41479]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.318 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41479]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.318 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41482]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:04:00.318 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41482]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.318 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41482]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.318 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 26 18:03:59 smithi167 sudo[41482]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41485]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:04:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41485]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41485]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41485]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41488]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:04:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41488]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:04:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41488]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:04:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 sudo[41488]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:04:00.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:00.554+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1658098cf0 con 0x7f1668102370 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:00.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.792 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:00 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:00 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:00 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:04:01.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:01 smithi090 ceph-mon[29180]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:04:01.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:01 smithi090 ceph-mon[29180]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T18:04:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:01 smithi119 ceph-mon[32752]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:04:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:01 smithi119 ceph-mon[32752]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T18:04:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:01 smithi167 ceph-mon[32751]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:04:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:01 smithi167 ceph-mon[32751]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T18:04:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:02 smithi167 ceph-mon[32751]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:02 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1888816231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8548cc2a-05b4-4e96-9e1c-22b26116170d"}]: dispatch 2024-04-26T18:04:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:02 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1888816231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8548cc2a-05b4-4e96-9e1c-22b26116170d"}]': finished 2024-04-26T18:04:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:02 smithi167 ceph-mon[32751]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T18:04:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:02 smithi090 ceph-mon[29180]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:02 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1888816231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8548cc2a-05b4-4e96-9e1c-22b26116170d"}]: dispatch 2024-04-26T18:04:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:02 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1888816231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8548cc2a-05b4-4e96-9e1c-22b26116170d"}]': finished 2024-04-26T18:04:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:02 smithi090 ceph-mon[29180]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T18:04:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:02 smithi119 ceph-mon[32752]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:02 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1888816231' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8548cc2a-05b4-4e96-9e1c-22b26116170d"}]: dispatch 2024-04-26T18:04:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:02 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1888816231' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8548cc2a-05b4-4e96-9e1c-22b26116170d"}]': finished 2024-04-26T18:04:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:02 smithi119 ceph-mon[32752]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T18:04:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:03 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2386504679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:03 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2386504679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:03 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2386504679' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:04 smithi090 ceph-mon[29180]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:04 smithi119 ceph-mon[32752]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:04 smithi167 ceph-mon[32751]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:07.060 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:06 smithi090 ceph-mon[29180]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:06 smithi119 ceph-mon[32752]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:06 smithi167 ceph-mon[32751]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:09.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:08 smithi090 ceph-mon[29180]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:08 smithi119 ceph-mon[32752]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:08 smithi167 ceph-mon[32751]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:11.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:10 smithi167 ceph-mon[32751]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:11.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:04:11.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:10 smithi090 ceph-mon[29180]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:04:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:10 smithi119 ceph-mon[32752]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:04:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:12.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:11 smithi090 ceph-mon[29180]: Deploying daemon osd.3 on smithi090 2024-04-26T18:04:12.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:11 smithi167 ceph-mon[32751]: Deploying daemon osd.3 on smithi090 2024-04-26T18:04:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:11 smithi119 ceph-mon[32752]: Deploying daemon osd.3 on smithi090 2024-04-26T18:04:13.040 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:12 smithi090 ceph-mon[29180]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:13.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:12 smithi119 ceph-mon[32752]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:13.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:12 smithi167 ceph-mon[32751]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:15.232 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:14 smithi090 ceph-mon[29180]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:14 smithi119 ceph-mon[32752]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:15.380 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:14 smithi167 ceph-mon[32751]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:16 smithi119 ceph-mon[32752]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:16 smithi167 ceph-mon[32751]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:16 smithi090 ceph-mon[29180]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:19.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:18 smithi090 ceph-mon[29180]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:19.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:19.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:19.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.161+0000 7f165ffff700 1 -- 172.21.15.90:0/3197649787 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f16680ffac0 con 0x7f16540a3bb0 2024-04-26T18:04:19.162 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 3 on host 'smithi090' 2024-04-26T18:04:19.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f16540a3bb0 msgr2=0x7f16540a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:19.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f16540a3bb0 0x7f16540a6070 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1650005950 tx=0x7f16500058e0 comp rx=0 tx=0).stop 2024-04-26T18:04:19.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668102370 msgr2=0x7f1668199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:19.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668102370 0x7f1668199550 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f165800c9a0 tx=0x7f165800cd60 comp rx=0 tx=0).stop 2024-04-26T18:04:19.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 shutdown_connections 2024-04-26T18:04:19.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f16540a3bb0 0x7f16540a6070 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:19.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1668102370 0x7f1668199550 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:19.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1668103f10 0x7f16681a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:19.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 --2- 172.21.15.90:0/3197649787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1668103570 0x7f1668199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:19.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 >> 172.21.15.90:0/3197649787 conn(0x7f16680fd910 msgr2=0x7f16680ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:04:19.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 shutdown_connections 2024-04-26T18:04:19.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:04:19.164+0000 7f166c87c700 1 -- 172.21.15.90:0/3197649787 wait complete. 2024-04-26T18:04:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:18 smithi119 ceph-mon[32752]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:18 smithi167 ceph-mon[32751]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:19.807 DEBUG:teuthology.orchestra.run.smithi090:osd.3> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T18:04:19.810 INFO:tasks.cephadm:Deploying osd.4 on smithi119 with /dev/vg_nvme/lv_3... 2024-04-26T18:04:19.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:04:20.055 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:04:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:20 smithi090 ceph-mon[29180]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:04:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:20 smithi090 ceph-mon[29180]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:20.408 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:20.408 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:20.408 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:20.408 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:20 smithi119 ceph-mon[32752]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:04:20.408 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:20 smithi119 ceph-mon[32752]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:20 smithi167 ceph-mon[32751]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:04:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:20 smithi167 ceph-mon[32751]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:21.415 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:21 smithi119 ceph-mon[32752]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:04:21.415 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:21 smithi119 ceph-mon[32752]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:04:21.416 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:21 smithi119 ceph-mon[32752]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:04:21.416 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:21 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:21 smithi167 ceph-mon[32751]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:04:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:21 smithi167 ceph-mon[32751]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:04:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:21 smithi167 ceph-mon[32751]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:04:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:21 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:21 smithi090 ceph-mon[29180]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:04:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:21 smithi090 ceph-mon[29180]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:04:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:21 smithi090 ceph-mon[29180]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:04:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:21 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:21.628 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 26 18:04:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-3[50346]: 2024-04-26T18:04:21.156+0000 7f6afd08c700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T18:04:21.628 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 26 18:04:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-3[50346]: 2024-04-26T18:04:21.161+0000 7f6af8e7f700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:04:22.123 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:04:22.123 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:04:22.123 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-04-26T18:04:22.123 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-04-26T18:04:22.123 INFO:teuthology.orchestra.run.smithi119.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151981 s, 690 MB/s 2024-04-26T18:04:22.123 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-04-26T18:04:22.268 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:22 smithi090 ceph-mon[29180]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:04:22.268 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:22 smithi090 ceph-mon[29180]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:04:22.268 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:22.269 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:22.269 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:22 smithi090 ceph-mon[29180]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:22.416 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:22 smithi119 ceph-mon[32752]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:04:22.416 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:22 smithi119 ceph-mon[32752]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:04:22.416 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:22.416 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:22.417 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:22 smithi119 ceph-mon[32752]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:22.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:22 smithi167 ceph-mon[32751]: from='osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:04:22.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:22 smithi167 ceph-mon[32751]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:04:22.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:22.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:22.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:22 smithi167 ceph-mon[32751]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:04:22.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi119:vg_nvme/lv_3 2024-04-26T18:04:23.083 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:04:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] boot 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi090 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 8544M 2024-04-26T18:04:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] boot 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi090 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:23.464 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 8544M 2024-04-26T18:04:23.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:23.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:23.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:04:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: osd.3 [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] boot 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: osdmap e25: 4 total, 4 up, 4 in 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi090 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 8544M 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:24.491 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:24 smithi119 ceph-mon[32752]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:04:24.491 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:24 smithi119 ceph-mon[32752]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:24 smithi167 ceph-mon[32751]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:04:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:24 smithi167 ceph-mon[32751]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:24 smithi090 ceph-mon[29180]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:04:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:24 smithi090 ceph-mon[29180]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:24.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 -- 172.21.15.119:0/236744049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 msgr2=0x7fb55c10b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:24.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 --2- 172.21.15.119:0/236744049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c10b5e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b0a0 tx=0x7fb55400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:04:24.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 -- 172.21.15.119:0/236744049 shutdown_connections 2024-04-26T18:04:24.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 --2- 172.21.15.119:0/236744049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55c103f70 0x7fb55c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:24.689 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 --2- 172.21.15.119:0/236744049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c10b5e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:24.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 --2- 172.21.15.119:0/236744049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb55c1068a0 0x7fb55c108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:24.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 -- 172.21.15.119:0/236744049 >> 172.21.15.119:0/236744049 conn(0x7fb55c0fd920 msgr2=0x7fb55c0ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:04:24.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 -- 172.21.15.119:0/236744049 shutdown_connections 2024-04-26T18:04:24.690 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 -- 172.21.15.119:0/236744049 wait complete. 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 Processor -- start 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.690+0000 7fb5645f4700 1 -- start start 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb5645f4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb55c103f70 0x7fb55c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb5645f4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55c1068a0 0x7fb55c199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb5645f4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb5645f4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c10ef90 con 0x7fb55c1068a0 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb5645f4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c10ee10 con 0x7fb55c1091d0 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb5645f4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb55c10f110 con 0x7fb55c103f70 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c1a0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41054/0 (socket says 172.21.15.119:41054) 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 -- 172.21.15.119:0/1335635698 learned_addr learned my addr 172.21.15.119:0/1335635698 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-26T18:04:24.692 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb561b8f700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55c1068a0 0x7fb55c199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562390700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb55c103f70 0x7fb55c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 -- 172.21.15.119:0/1335635698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb55c103f70 msgr2=0x7fb55c1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb55c103f70 0x7fb55c1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 -- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55c1068a0 msgr2=0x7fb55c199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55c1068a0 0x7fb55c199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 -- 172.21.15.119:0/1335635698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55c1a1200 con 0x7fb55c1091d0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb562b91700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c1a0b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d010 tx=0x7fb554004dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.691+0000 7fb5537fe700 1 -- 172.21.15.119:0/1335635698 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554021070 con 0x7fb55c1091d0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.692+0000 7fb5537fe700 1 -- 172.21.15.119:0/1335635698 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb554007b80 con 0x7fb55c1091d0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.692+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55c1a1490 con 0x7fb55c1091d0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.692+0000 7fb5537fe700 1 -- 172.21.15.119:0/1335635698 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554004690 con 0x7fb55c1091d0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.692+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55c1a19d0 con 0x7fb55c1091d0 2024-04-26T18:04:24.693 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.692+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55c06a110 con 0x7fb55c1091d0 2024-04-26T18:04:24.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.694+0000 7fb5537fe700 1 -- 172.21.15.119:0/1335635698 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb554003a00 con 0x7fb55c1091d0 2024-04-26T18:04:24.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.694+0000 7fb5537fe700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb5480a3cc0 0x7fb5480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:24.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.695+0000 7fb5537fe700 1 -- 172.21.15.119:0/1335635698 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fb5540d0140 con 0x7fb55c1091d0 2024-04-26T18:04:24.694 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.695+0000 7fb562390700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb5480a3cc0 0x7fb5480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:24.695 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.695+0000 7fb562390700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb5480a3cc0 0x7fb5480a6180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb558009c60 tx=0x7fb558009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:04:24.697 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.698+0000 7fb5537fe700 1 -- 172.21.15.119:0/1335635698 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb55409ddd0 con 0x7fb55c1091d0 2024-04-26T18:04:24.906 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:24.907+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb55c063860 con 0x7fb5480a3cc0 2024-04-26T18:04:25.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:25 smithi119 ceph-mon[32752]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:04:25.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:25 smithi119 ceph-mon[32752]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:25.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:25.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:25.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:25 smithi167 ceph-mon[32751]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:04:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:25 smithi167 ceph-mon[32751]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:25 smithi090 ceph-mon[29180]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:04:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:25 smithi090 ceph-mon[29180]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:26.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:26 smithi119 ceph-mon[32752]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:04:26.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:26 smithi119 ceph-mon[32752]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-04-26T18:04:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:26 smithi167 ceph-mon[32751]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:04:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:26 smithi167 ceph-mon[32751]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-04-26T18:04:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:26 smithi090 ceph-mon[29180]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:04:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:26 smithi090 ceph-mon[29180]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-04-26T18:04:27.963 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:27 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/1153569658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]: dispatch 2024-04-26T18:04:27.964 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:27 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]: dispatch 2024-04-26T18:04:27.964 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:27 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]': finished 2024-04-26T18:04:27.964 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:27 smithi119 ceph-mon[32752]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T18:04:27.964 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:27 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:27 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/1153569658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]: dispatch 2024-04-26T18:04:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:27 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]: dispatch 2024-04-26T18:04:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:27 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]': finished 2024-04-26T18:04:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:27 smithi167 ceph-mon[32751]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T18:04:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:27 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:27 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/1153569658' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]: dispatch 2024-04-26T18:04:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:27 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]: dispatch 2024-04-26T18:04:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:27 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11b6b750-1a3f-4a50-bacf-145c1586412b"}]': finished 2024-04-26T18:04:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:27 smithi090 ceph-mon[29180]: osdmap e30: 5 total, 4 up, 5 in 2024-04-26T18:04:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:27 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:28 smithi119 ceph-mon[32752]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-26T18:04:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:28 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/4083142584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:28 smithi167 ceph-mon[32751]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-26T18:04:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:28 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/4083142584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:28 smithi090 ceph-mon[29180]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 127 KiB/s, 0 objects/s recovering 2024-04-26T18:04:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:28 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/4083142584' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:30 smithi090 ceph-mon[29180]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:04:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:30 smithi167 ceph-mon[32751]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:04:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:30 smithi119 ceph-mon[32752]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:04:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:32 smithi090 ceph-mon[29180]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T18:04:33.154 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:32 smithi119 ceph-mon[32752]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T18:04:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:32 smithi167 ceph-mon[32751]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T18:04:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:34 smithi090 ceph-mon[29180]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:34 smithi119 ceph-mon[32752]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:34 smithi167 ceph-mon[32751]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:04:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:04:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:04:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:36.918 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:36 smithi119 ceph-mon[32752]: Deploying daemon osd.4 on smithi119 2024-04-26T18:04:36.918 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:36 smithi119 ceph-mon[32752]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:36 smithi090 ceph-mon[29180]: Deploying daemon osd.4 on smithi119 2024-04-26T18:04:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:36 smithi090 ceph-mon[29180]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:36 smithi167 ceph-mon[32751]: Deploying daemon osd.4 on smithi119 2024-04-26T18:04:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:36 smithi167 ceph-mon[32751]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:38.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:38 smithi119 ceph-mon[32752]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:38 smithi090 ceph-mon[29180]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:38 smithi167 ceph-mon[32751]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:41.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:40 smithi167 ceph-mon[32751]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:41.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:41.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:41.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:40 smithi090 ceph-mon[29180]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:41.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:40 smithi119 ceph-mon[32752]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:42 smithi090 ceph-mon[29180]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:43.170 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:42 smithi119 ceph-mon[32752]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:42 smithi167 ceph-mon[32751]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:44.423 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 4 on host 'smithi119' 2024-04-26T18:04:44.423 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.422+0000 7fb5537fe700 1 -- 172.21.15.119:0/1335635698 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb55c063860 con 0x7fb5480a3cc0 2024-04-26T18:04:44.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb5480a3cc0 msgr2=0x7fb5480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:44.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb5480a3cc0 0x7fb5480a6180 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb558009c60 tx=0x7fb558009260 comp rx=0 tx=0).stop 2024-04-26T18:04:44.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 msgr2=0x7fb55c1a0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:44.425 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c1a0b60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d010 tx=0x7fb554004dc0 comp rx=0 tx=0).stop 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 shutdown_connections 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb5480a3cc0 0x7fb5480a6180 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55c1091d0 0x7fb55c1a0b60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb55c103f70 0x7fb55c1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 --2- 172.21.15.119:0/1335635698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb55c1068a0 0x7fb55c199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 >> 172.21.15.119:0/1335635698 conn(0x7fb55c0fd920 msgr2=0x7fb55c1048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 shutdown_connections 2024-04-26T18:04:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:04:44.425+0000 7fb5645f4700 1 -- 172.21.15.119:0/1335635698 wait complete. 2024-04-26T18:04:44.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:44.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:44.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:44.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:44.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:04:44.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:44 smithi119 ceph-mon[32752]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:04:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:04:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:44 smithi167 ceph-mon[32751]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:04:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:44 smithi090 ceph-mon[29180]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:04:45.036 DEBUG:teuthology.orchestra.run.smithi119:osd.4> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T18:04:45.038 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-04-26T18:04:45.039 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:04:45.283 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:04:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:46 smithi119 ceph-mon[32752]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:04:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:46 smithi119 ceph-mon[32752]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:04:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:46 smithi119 ceph-mon[32752]: from='osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:04:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:46 smithi119 ceph-mon[32752]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:04:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:46 smithi119 ceph-mon[32752]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:46.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:46 smithi167 ceph-mon[32751]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:04:46.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:46 smithi167 ceph-mon[32751]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:04:46.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:46.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:46 smithi167 ceph-mon[32751]: from='osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:04:46.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:46 smithi167 ceph-mon[32751]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:04:46.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:46 smithi167 ceph-mon[32751]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:46 smithi090 ceph-mon[29180]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:04:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:46 smithi090 ceph-mon[29180]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:04:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:46 smithi090 ceph-mon[29180]: from='osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:04:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:46 smithi090 ceph-mon[29180]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:04:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:46 smithi090 ceph-mon[29180]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:47.346 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:04:47.347 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:04:47.347 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-04-26T18:04:47.347 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-04-26T18:04:47.347 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115061 s, 911 MB/s 2024-04-26T18:04:47.347 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-04-26T18:04:47.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:47 smithi119 ceph-mon[32752]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:04:47.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:47 smithi119 ceph-mon[32752]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:04:47.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:47 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:47.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:47 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:47.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:47 smithi119 ceph-mon[32752]: from='osd.4 ' entity='osd.4' 2024-04-26T18:04:47.566 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 26 18:04:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-4[44293]: 2024-04-26T18:04:47.305+0000 7f431fad4700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T18:04:47.567 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 26 18:04:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-4[44293]: 2024-04-26T18:04:47.351+0000 7f43170be700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:04:47.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:47 smithi167 ceph-mon[32751]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:04:47.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:47 smithi167 ceph-mon[32751]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:04:47.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:47 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:47.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:47 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:47.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:47 smithi167 ceph-mon[32751]: from='osd.4 ' entity='osd.4' 2024-04-26T18:04:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:47 smithi090 ceph-mon[29180]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:04:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:47 smithi090 ceph-mon[29180]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:04:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:47 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:47 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:47 smithi090 ceph-mon[29180]: from='osd.4 ' entity='osd.4' 2024-04-26T18:04:47.990 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-04-26T18:04:48.225 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] boot 2024-04-26T18:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:04:48.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:04:48.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:48.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:48.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] boot 2024-04-26T18:04:48.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T18:04:48.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:48.667 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: osd.4 [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] boot 2024-04-26T18:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: osdmap e33: 5 total, 5 up, 5 in 2024-04-26T18:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:04:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:04:49.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:49 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi119 2024-04-26T18:04:49.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:49 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 8544M 2024-04-26T18:04:49.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:49 smithi167 ceph-mon[32751]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:04:49.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.774+0000 7f0158ff1700 1 -- 172.21.15.167:0/1918751278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01541036b0 msgr2=0x7f0154109f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:49.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.774+0000 7f0158ff1700 1 --2- 172.21.15.167:0/1918751278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01541036b0 0x7f0154109f60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f014c00c210 tx=0x7f014c00c520 comp rx=0 tx=0).stop 2024-04-26T18:04:49.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 -- 172.21.15.167:0/1918751278 shutdown_connections 2024-04-26T18:04:49.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 --2- 172.21.15.167:0/1918751278 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0154101b10 0x7f0154101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:49.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 --2- 172.21.15.167:0/1918751278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01541036b0 0x7f0154109f60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:49.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 --2- 172.21.15.167:0/1918751278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0154102d10 0x7f0154103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:49.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 -- 172.21.15.167:0/1918751278 >> 172.21.15.167:0/1918751278 conn(0x7f01540fd090 msgr2=0x7f01540ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:04:49.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 -- 172.21.15.167:0/1918751278 shutdown_connections 2024-04-26T18:04:49.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 -- 172.21.15.167:0/1918751278 wait complete. 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 Processor -- start 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.775+0000 7f0158ff1700 1 -- start start 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0158ff1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0154101b10 0x7f0154105aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0158ff1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0154102d10 0x7f01541040f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0158ff1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01541036b0 0x7f0154104630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0158ff1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f015410cd70 con 0x7f01541036b0 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0158ff1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f015410cbf0 con 0x7f0154101b10 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0158ff1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f015410cef0 con 0x7f0154102d10 2024-04-26T18:04:49.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0154102d10 0x7f01541040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0154102d10 0x7f01541040f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51998/0 (socket says 172.21.15.167:51998) 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 -- 172.21.15.167:0/769885360 learned_addr learned my addr 172.21.15.167:0/769885360 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f015259c700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0154101b10 0x7f0154105aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0152d9d700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01541036b0 0x7f0154104630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 -- 172.21.15.167:0/769885360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0154101b10 msgr2=0x7f0154105aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0154101b10 0x7f0154105aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 -- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01541036b0 msgr2=0x7f0154104630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01541036b0 0x7f0154104630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 -- 172.21.15.167:0/769885360 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0154104ee0 con 0x7f0154102d10 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0151d9b700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0154102d10 0x7f01541040f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f014400b530 tx=0x7f014400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f014b7fe700 1 -- 172.21.15.167:0/769885360 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144015020 con 0x7f0154102d10 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f014b7fe700 1 -- 172.21.15.167:0/769885360 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0144011ce0 con 0x7f0154102d10 2024-04-26T18:04:49.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01541a37b0 con 0x7f0154102d10 2024-04-26T18:04:49.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.776+0000 7f014b7fe700 1 -- 172.21.15.167:0/769885360 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144010dd0 con 0x7f0154102d10 2024-04-26T18:04:49.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.777+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01541a3c70 con 0x7f0154102d10 2024-04-26T18:04:49.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.777+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f015406a110 con 0x7f0154102d10 2024-04-26T18:04:49.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.779+0000 7f014b7fe700 1 -- 172.21.15.167:0/769885360 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0144018020 con 0x7f0154102d10 2024-04-26T18:04:49.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.780+0000 7f014b7fe700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f01400a3c40 0x7f01400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:04:49.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.780+0000 7f014b7fe700 1 -- 172.21.15.167:0/769885360 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f01440cf010 con 0x7f0154102d10 2024-04-26T18:04:49.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.780+0000 7f015259c700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f01400a3c40 0x7f01400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:04:49.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.781+0000 7f015259c700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f01400a3c40 0x7f01400a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0154102b70 tx=0x7f013c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:04:49.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.784+0000 7f014b7fe700 1 -- 172.21.15.167:0/769885360 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f014401f020 con 0x7f0154102d10 2024-04-26T18:04:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:49 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi119 2024-04-26T18:04:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:49 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 8544M 2024-04-26T18:04:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:49 smithi119 ceph-mon[32752]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:04:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:49 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi119 2024-04-26T18:04:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:49 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 8544M 2024-04-26T18:04:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:49 smithi090 ceph-mon[29180]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:04:49.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:04:49.989+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f01541a3f90 con 0x7f01400a3c40 2024-04-26T18:04:50.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:50 smithi167 ceph-mon[32751]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:50.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:50 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:50.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:50 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:50.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:50 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:50 smithi119 ceph-mon[32752]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:50 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:50 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:50 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:50 smithi090 ceph-mon[29180]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:50 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:04:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:50 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:04:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:50 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:04:51.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:51 smithi167 ceph-mon[32751]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:51 smithi119 ceph-mon[32752]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:51 smithi090 ceph-mon[29180]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:04:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:52 smithi090 ceph-mon[29180]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:52 smithi119 ceph-mon[32752]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:52 smithi167 ceph-mon[32751]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:53 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/905660860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]: dispatch 2024-04-26T18:04:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:53 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]: dispatch 2024-04-26T18:04:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:53 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]': finished 2024-04-26T18:04:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:53 smithi119 ceph-mon[32752]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T18:04:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:53 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:04:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:53 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/1325632407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:53 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/905660860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]: dispatch 2024-04-26T18:04:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:53 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]: dispatch 2024-04-26T18:04:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:53 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]': finished 2024-04-26T18:04:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:53 smithi167 ceph-mon[32751]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T18:04:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:53 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:04:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:53 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/1325632407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:53 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/905660860' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]: dispatch 2024-04-26T18:04:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:53 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]: dispatch 2024-04-26T18:04:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:53 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "592a3d64-facb-42f0-928d-ab7326e69642"}]': finished 2024-04-26T18:04:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:53 smithi090 ceph-mon[29180]: osdmap e35: 6 total, 5 up, 6 in 2024-04-26T18:04:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:53 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:04:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:53 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/1325632407' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:04:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:54 smithi119 ceph-mon[32752]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:54 smithi167 ceph-mon[32751]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:54 smithi090 ceph-mon[29180]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:56 smithi090 ceph-mon[29180]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:57.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:56 smithi167 ceph-mon[32751]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:56 smithi119 ceph-mon[32752]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:04:58 smithi167 ceph-mon[32751]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:04:58 smithi090 ceph-mon[29180]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:04:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:04:58 smithi119 ceph-mon[32752]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:00 smithi090 ceph-mon[29180]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:05:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:01.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:00 smithi167 ceph-mon[32751]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:01.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:05:01.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:00 smithi119 ceph-mon[32752]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:05:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:01 smithi167 ceph-mon[32751]: Deploying daemon osd.5 on smithi167 2024-04-26T18:05:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:01 smithi090 ceph-mon[29180]: Deploying daemon osd.5 on smithi167 2024-04-26T18:05:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:01 smithi119 ceph-mon[32752]: Deploying daemon osd.5 on smithi167 2024-04-26T18:05:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:02 smithi090 ceph-mon[29180]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:03.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:02 smithi167 ceph-mon[32751]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:02 smithi119 ceph-mon[32752]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:04 smithi090 ceph-mon[29180]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:05.197 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:04 smithi167 ceph-mon[32751]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:04 smithi119 ceph-mon[32752]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:06 smithi119 ceph-mon[32752]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:06 smithi167 ceph-mon[32751]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:06 smithi090 ceph-mon[29180]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:08 smithi090 ceph-mon[29180]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:08 smithi167 ceph-mon[32751]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:09.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:09.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:09.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:08 smithi119 ceph-mon[32752]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:09.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.556+0000 7f014b7fe700 1 -- 172.21.15.167:0/769885360 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f01541a3f90 con 0x7f01400a3c40 2024-04-26T18:05:09.556 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-04-26T18:05:09.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f01400a3c40 msgr2=0x7f01400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f01400a3c40 0x7f01400a6100 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0154102b70 tx=0x7f013c00b410 comp rx=0 tx=0).stop 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0154102d10 msgr2=0x7f01541040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0154102d10 0x7f01541040f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f014400b530 tx=0x7f014400b840 comp rx=0 tx=0).stop 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 shutdown_connections 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f01400a3c40 0x7f01400a6100 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01541036b0 0x7f0154104630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0154101b10 0x7f0154105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 --2- 172.21.15.167:0/769885360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0154102d10 0x7f01541040f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 >> 172.21.15.167:0/769885360 conn(0x7f01540fd090 msgr2=0x7f0154108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 shutdown_connections 2024-04-26T18:05:09.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:05:09.559+0000 7f0158ff1700 1 -- 172.21.15.167:0/769885360 wait complete. 2024-04-26T18:05:10.191 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T18:05:10.203 INFO:tasks.cephadm:Deploying osd.6 on smithi090 with /dev/vg_nvme/lv_2... 2024-04-26T18:05:10.203 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:05:10.450 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:05:10.802 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:10.802 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:10.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:10.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:10 smithi090 ceph-mon[29180]: from='osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:05:10.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:10 smithi090 ceph-mon[29180]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:05:10.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:10 smithi090 ceph-mon[29180]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:10 smithi119 ceph-mon[32752]: from='osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:05:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:10 smithi119 ceph-mon[32752]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:05:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:10 smithi119 ceph-mon[32752]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:10 smithi167 ceph-mon[32751]: from='osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:05:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:10 smithi167 ceph-mon[32751]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:05:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:10 smithi167 ceph-mon[32751]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:11.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:11 smithi167 ceph-mon[32751]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:05:11.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:11 smithi167 ceph-mon[32751]: from='osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:05:11.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:11 smithi167 ceph-mon[32751]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:05:11.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:11 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:11.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:11 smithi167 ceph-mon[32751]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:05:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:11 smithi119 ceph-mon[32752]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:05:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:11 smithi119 ceph-mon[32752]: from='osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:05:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:11 smithi119 ceph-mon[32752]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:05:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:11 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:11 smithi119 ceph-mon[32752]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:05:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:11 smithi090 ceph-mon[29180]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:05:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:11 smithi090 ceph-mon[29180]: from='osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:05:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:11 smithi090 ceph-mon[29180]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:05:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:11 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:11 smithi090 ceph-mon[29180]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:05:12.641 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:05:12.641 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:05:12.641 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-26T18:05:12.641 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-26T18:05:12.641 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151095 s, 694 MB/s 2024-04-26T18:05:12.641 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-26T18:05:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T18:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-mon[32751]: from='osd.5 ' entity='osd.5' 2024-04-26T18:05:12.819 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-26T18:05:12.467+0000 7f61f2aee700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T18:05:12.819 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 26 18:05:12 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-26T18:05:12.661+0000 7f61ea8d9700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:12 smithi090 ceph-mon[29180]: from='osd.5 ' entity='osd.5' 2024-04-26T18:05:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:05:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:05:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:05:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T18:05:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:12 smithi119 ceph-mon[32752]: from='osd.5 ' entity='osd.5' 2024-04-26T18:05:13.279 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-04-26T18:05:13.570 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:05:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] boot 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:13 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:05:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:05:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] boot 2024-04-26T18:05:13.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:13 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:05:13.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:05:13.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: osd.5 [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] boot 2024-04-26T18:05:13.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:13.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:13 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:14 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi167 2024-04-26T18:05:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:14 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 8544M 2024-04-26T18:05:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:14 smithi090 ceph-mon[29180]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:14 smithi090 ceph-mon[29180]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:05:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:14 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi167 2024-04-26T18:05:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:14 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 8544M 2024-04-26T18:05:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:14 smithi167 ceph-mon[32751]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:14 smithi167 ceph-mon[32751]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:05:14.970 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:14 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi167 2024-04-26T18:05:14.970 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:14 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 8544M 2024-04-26T18:05:14.970 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:14 smithi119 ceph-mon[32752]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 582 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:14.971 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:14 smithi119 ceph-mon[32752]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:05:15.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 -- 172.21.15.90:0/1909778784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d8104800 msgr2=0x7fe5d8104c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:15.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 --2- 172.21.15.90:0/1909778784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d8104800 0x7fe5d8104c60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc009990 tx=0x7fe5cc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:05:15.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 -- 172.21.15.90:0/1909778784 shutdown_connections 2024-04-26T18:05:15.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 --2- 172.21.15.90:0/1909778784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d81051a0 0x7fe5d810ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:15.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 --2- 172.21.15.90:0/1909778784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8101820 0x7fe5d8101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:15.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 --2- 172.21.15.90:0/1909778784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d8104800 0x7fe5d8104c60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:15.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 -- 172.21.15.90:0/1909778784 >> 172.21.15.90:0/1909778784 conn(0x7fe5d80fd180 msgr2=0x7fe5d80ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:05:15.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 -- 172.21.15.90:0/1909778784 shutdown_connections 2024-04-26T18:05:15.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 -- 172.21.15.90:0/1909778784 wait complete. 2024-04-26T18:05:15.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 Processor -- start 2024-04-26T18:05:15.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.134+0000 7fe5de486700 1 -- start start 2024-04-26T18:05:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5de486700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d8101820 0x7fe5d8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5de486700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8104800 0x7fe5d8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5de486700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d81051a0 0x7fe5d81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5de486700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d810efc0 con 0x7fe5d81051a0 2024-04-26T18:05:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5de486700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d810ee40 con 0x7fe5d8104800 2024-04-26T18:05:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5de486700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d810f140 con 0x7fe5d8101820 2024-04-26T18:05:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8104800 0x7fe5d8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:15.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5dca23700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d81051a0 0x7fe5d81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8104800 0x7fe5d8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:45710/0 (socket says 172.21.15.90:45710) 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5dca23700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d81051a0 0x7fe5d81a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59142/0 (socket says 172.21.15.90:59142) 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 -- 172.21.15.90:0/3767797363 learned_addr learned my addr 172.21.15.90:0/3767797363 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d7fff700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d8101820 0x7fe5d8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 -- 172.21.15.90:0/3767797363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d8101820 msgr2=0x7fe5d8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d8101820 0x7fe5d8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 -- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d81051a0 msgr2=0x7fe5d81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d81051a0 0x7fe5d81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 -- 172.21.15.90:0/3767797363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d81a1260 con 0x7fe5d8104800 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5dca23700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d81051a0 0x7fe5d81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:05:15.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d7fff700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d8101820 0x7fe5d8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d77fe700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8104800 0x7fe5d8199b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006010 tx=0x7fe5cc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.135+0000 7fe5d57fa700 1 -- 172.21.15.90:0/3767797363 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc027070 con 0x7fe5d8104800 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.136+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d81a14f0 con 0x7fe5d8104800 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.136+0000 7fe5d57fa700 1 -- 172.21.15.90:0/3767797363 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5cc00fbc0 con 0x7fe5d8104800 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.136+0000 7fe5d57fa700 1 -- 172.21.15.90:0/3767797363 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc017650 con 0x7fe5d8104800 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.136+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d81a19a0 con 0x7fe5d8104800 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.136+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d806a110 con 0x7fe5d8104800 2024-04-26T18:05:15.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.138+0000 7fe5d57fa700 1 -- 172.21.15.90:0/3767797363 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe5cc009d10 con 0x7fe5d8104800 2024-04-26T18:05:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.138+0000 7fe5d57fa700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe5c00a3c40 0x7fe5c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.138+0000 7fe5d7fff700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe5c00a3c40 0x7fe5c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.138+0000 7fe5d57fa700 1 -- 172.21.15.90:0/3767797363 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fe5cc0d0530 con 0x7fe5d8104800 2024-04-26T18:05:15.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.139+0000 7fe5d7fff700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe5c00a3c40 0x7fe5c00a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5d806b930 tx=0x7fe5c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:05:15.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.146+0000 7fe5d57fa700 1 -- 172.21.15.90:0/3767797363 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe5cc013070 con 0x7fe5d8104800 2024-04-26T18:05:15.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:15.355+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fe5d8063860 con 0x7fe5c00a3c40 2024-04-26T18:05:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:16 smithi119 ceph-mon[32752]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:05:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:16 smithi119 ceph-mon[32752]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:05:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:05:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:05:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:16 smithi119 ceph-mon[32752]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 586 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:16 smithi167 ceph-mon[32751]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:05:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:16 smithi167 ceph-mon[32751]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:05:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:05:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:05:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:16 smithi167 ceph-mon[32751]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 586 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:16 smithi090 ceph-mon[29180]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:16 smithi090 ceph-mon[29180]: from='client.24242 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:16 smithi090 ceph-mon[29180]: pgmap v143: 1 pgs: 1 remapped+peering; 577 KiB data, 586 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:17 smithi119 ceph-mon[32752]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:05:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:17 smithi167 ceph-mon[32751]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:05:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:17 smithi090 ceph-mon[29180]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:05:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:18 smithi119 ceph-mon[32752]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T18:05:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:18 smithi119 ceph-mon[32752]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 586 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:18 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3938893969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24eb3641-a329-4a5f-b672-06b21c53894b"}]: dispatch 2024-04-26T18:05:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:18 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3938893969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24eb3641-a329-4a5f-b672-06b21c53894b"}]': finished 2024-04-26T18:05:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:18 smithi119 ceph-mon[32752]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:05:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:18 smithi167 ceph-mon[32751]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T18:05:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:18 smithi167 ceph-mon[32751]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 586 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:18 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3938893969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24eb3641-a329-4a5f-b672-06b21c53894b"}]: dispatch 2024-04-26T18:05:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:18 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3938893969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24eb3641-a329-4a5f-b672-06b21c53894b"}]': finished 2024-04-26T18:05:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:18 smithi167 ceph-mon[32751]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:05:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:18 smithi090 ceph-mon[29180]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T18:05:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:18 smithi090 ceph-mon[29180]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 586 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:18 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3938893969' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24eb3641-a329-4a5f-b672-06b21c53894b"}]: dispatch 2024-04-26T18:05:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:18 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3938893969' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24eb3641-a329-4a5f-b672-06b21c53894b"}]': finished 2024-04-26T18:05:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:18 smithi090 ceph-mon[29180]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:05:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:19 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2273788287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:05:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:19 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2273788287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:05:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:19 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2273788287' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:05:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:20 smithi167 ceph-mon[32751]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-26T18:05:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:20 smithi090 ceph-mon[29180]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-26T18:05:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:20 smithi119 ceph-mon[32752]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-04-26T18:05:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:22 smithi090 ceph-mon[29180]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:05:23.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:22 smithi167 ceph-mon[32751]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:05:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:22 smithi119 ceph-mon[32752]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:05:25.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:24 smithi090 ceph-mon[29180]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T18:05:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:24 smithi119 ceph-mon[32752]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T18:05:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:24 smithi167 ceph-mon[32751]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-04-26T18:05:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:05:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:05:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:05:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:26 smithi090 ceph-mon[29180]: Deploying daemon osd.6 on smithi090 2024-04-26T18:05:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:26 smithi090 ceph-mon[29180]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T18:05:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:26 smithi119 ceph-mon[32752]: Deploying daemon osd.6 on smithi090 2024-04-26T18:05:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:26 smithi119 ceph-mon[32752]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T18:05:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:26 smithi167 ceph-mon[32751]: Deploying daemon osd.6 on smithi090 2024-04-26T18:05:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:26 smithi167 ceph-mon[32751]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-04-26T18:05:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:28 smithi090 ceph-mon[29180]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:05:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:28 smithi119 ceph-mon[32752]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:05:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:28 smithi167 ceph-mon[32751]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:05:31.102 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:30 smithi090 ceph-mon[29180]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-26T18:05:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:30 smithi119 ceph-mon[32752]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-26T18:05:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:30 smithi167 ceph-mon[32751]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-04-26T18:05:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:32 smithi090 ceph-mon[29180]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:32 smithi119 ceph-mon[32752]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:32 smithi167 ceph-mon[32751]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:34 smithi090 ceph-mon[29180]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.268+0000 7fe5d57fa700 1 -- 172.21.15.90:0/3767797363 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe5d8063860 con 0x7fe5c00a3c40 2024-04-26T18:05:35.269 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 6 on host 'smithi090' 2024-04-26T18:05:35.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe5c00a3c40 msgr2=0x7fe5c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:35.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe5c00a3c40 0x7fe5c00a6100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5d806b930 tx=0x7fe5c8009260 comp rx=0 tx=0).stop 2024-04-26T18:05:35.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8104800 msgr2=0x7fe5d8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:35.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8104800 0x7fe5d8199b40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc006010 tx=0x7fe5cc0047b0 comp rx=0 tx=0).stop 2024-04-26T18:05:35.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 shutdown_connections 2024-04-26T18:05:35.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe5c00a3c40 0x7fe5c00a6100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe5d81051a0 0x7fe5d81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5d8101820 0x7fe5d8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 --2- 172.21.15.90:0/3767797363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5d8104800 0x7fe5d8199b40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 >> 172.21.15.90:0/3767797363 conn(0x7fe5d80fd180 msgr2=0x7fe5d8107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:05:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 shutdown_connections 2024-04-26T18:05:35.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:05:35.270+0000 7fe5de486700 1 -- 172.21.15.90:0/3767797363 wait complete. 2024-04-26T18:05:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:34 smithi119 ceph-mon[32752]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:34 smithi167 ceph-mon[32751]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:35.935 DEBUG:teuthology.orchestra.run.smithi090:osd.6> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T18:05:35.937 INFO:tasks.cephadm:Deploying osd.7 on smithi119 with /dev/vg_nvme/lv_2... 2024-04-26T18:05:35.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:05:36.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:35 smithi090 ceph-mon[29180]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:05:36.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:36.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:36.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:36.181 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:05:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:35 smithi119 ceph-mon[32752]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:05:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:35 smithi167 ceph-mon[32751]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:05:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:36 smithi090 ceph-mon[29180]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:36 smithi090 ceph-mon[29180]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:05:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:36 smithi090 ceph-mon[29180]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:05:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:36 smithi090 ceph-mon[29180]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:05:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:36 smithi119 ceph-mon[32752]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:36 smithi119 ceph-mon[32752]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:36 smithi119 ceph-mon[32752]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:36 smithi119 ceph-mon[32752]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:36 smithi167 ceph-mon[32751]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:36 smithi167 ceph-mon[32751]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:05:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:36 smithi167 ceph-mon[32751]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:05:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:36 smithi167 ceph-mon[32751]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:05:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:37 smithi090 ceph-mon[29180]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:05:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:37 smithi090 ceph-mon[29180]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T18:05:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:37 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:37 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:37 smithi090 ceph-mon[29180]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' 2024-04-26T18:05:38.126 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 26 18:05:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-6[57169]: 2024-04-26T18:05:37.807+0000 7f7552804700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T18:05:38.127 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 26 18:05:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-6[57169]: 2024-04-26T18:05:37.946+0000 7f754a5ef700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:05:38.202 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:05:38.202 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:05:38.202 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-04-26T18:05:38.202 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-04-26T18:05:38.202 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122392 s, 857 MB/s 2024-04-26T18:05:38.203 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-04-26T18:05:38.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:37 smithi119 ceph-mon[32752]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:05:38.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:37 smithi119 ceph-mon[32752]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T18:05:38.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:37 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:38.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:37 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:38.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:37 smithi119 ceph-mon[32752]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' 2024-04-26T18:05:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:37 smithi167 ceph-mon[32751]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:05:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:37 smithi167 ceph-mon[32751]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T18:05:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:37 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:37 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:37 smithi167 ceph-mon[32751]: from='osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965]' entity='osd.6' 2024-04-26T18:05:38.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi119:vg_nvme/lv_2 2024-04-26T18:05:39.037 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:38 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:05:39.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:38 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:05:39.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:38 smithi090 ceph-mon[29180]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:39.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:39.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:38 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:39.155 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:05:39.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:38 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:05:39.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:38 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:05:39.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:38 smithi119 ceph-mon[32752]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:39.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:39.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:38 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:38 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:05:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:38 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:05:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:38 smithi167 ceph-mon[32751]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:38 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] boot 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.210 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:40.210 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:40.210 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] boot 2024-04-26T18:05:40.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:05:40.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:40.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: osd.6 [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] boot 2024-04-26T18:05:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:05:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:40.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 -- 172.21.15.119:0/2639910602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 msgr2=0x7f340c10b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:40.719 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 --2- 172.21.15.119:0/2639910602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c10b920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f34040099c0 tx=0x7f3404009cd0 comp rx=0 tx=0).stop 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 -- 172.21.15.119:0/2639910602 shutdown_connections 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 --2- 172.21.15.119:0/2639910602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c10b920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 --2- 172.21.15.119:0/2639910602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340c1034d0 0x7f340c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 --2- 172.21.15.119:0/2639910602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f340c1046d0 0x7f340c104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 -- 172.21.15.119:0/2639910602 >> 172.21.15.119:0/2639910602 conn(0x7f340c0fea70 msgr2=0x7f340c100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 -- 172.21.15.119:0/2639910602 shutdown_connections 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.717+0000 7f3413c5a700 1 -- 172.21.15.119:0/2639910602 wait complete. 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 Processor -- start 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 -- start start 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340c1034d0 0x7f340c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f340c1046d0 0x7f340c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:40.720 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10ef70 con 0x7f340c1046d0 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10edf0 con 0x7f340c105070 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f3413c5a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10f0f0 con 0x7f340c1034d0 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c1a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58932/0 (socket says 172.21.15.119:58932) 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 -- 172.21.15.119:0/932525247 learned_addr learned my addr 172.21.15.119:0/932525247 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34119f6700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340c1034d0 0x7f340c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34111f5700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f340c1046d0 0x7f340c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 -- 172.21.15.119:0/932525247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340c1034d0 msgr2=0x7f340c199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340c1034d0 0x7f340c199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 -- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f340c1046d0 msgr2=0x7f340c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f340c1046d0 0x7f340c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.718+0000 7f34121f7700 1 -- 172.21.15.119:0/932525247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c1a11e0 con 0x7f340c105070 2024-04-26T18:05:40.721 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.719+0000 7f34121f7700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c1a0b40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f34040059d0 tx=0x7f34040047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:05:40.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.719+0000 7f3402ffd700 1 -- 172.21.15.119:0/932525247 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404003c30 con 0x7f340c105070 2024-04-26T18:05:40.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.719+0000 7f3402ffd700 1 -- 172.21.15.119:0/932525247 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3404004830 con 0x7f340c105070 2024-04-26T18:05:40.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.719+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c1a1470 con 0x7f340c105070 2024-04-26T18:05:40.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.719+0000 7f3402ffd700 1 -- 172.21.15.119:0/932525247 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404005310 con 0x7f340c105070 2024-04-26T18:05:40.722 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.719+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c1a1860 con 0x7f340c105070 2024-04-26T18:05:40.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.720+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c06a110 con 0x7f340c105070 2024-04-26T18:05:40.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.721+0000 7f3402ffd700 1 -- 172.21.15.119:0/932525247 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f34040054b0 con 0x7f340c105070 2024-04-26T18:05:40.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.721+0000 7f3402ffd700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f33f80a3c40 0x7f33f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:05:40.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.721+0000 7f3402ffd700 1 -- 172.21.15.119:0/932525247 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f34040d0740 con 0x7f340c105070 2024-04-26T18:05:40.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.722+0000 7f34119f6700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f33f80a3c40 0x7f33f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:05:40.723 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.722+0000 7f34119f6700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f33f80a3c40 0x7f33f80a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f340c104530 tx=0x7f3408009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:05:40.726 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.726+0000 7f3402ffd700 1 -- 172.21.15.119:0/932525247 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3404013070 con 0x7f340c105070 2024-04-26T18:05:40.936 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:05:40.935+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f340c063860 con 0x7f33f80a3c40 2024-04-26T18:05:41.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:40 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi090 2024-04-26T18:05:41.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:40 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 5696M 2024-04-26T18:05:41.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:40 smithi167 ceph-mon[32751]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:41.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:40 smithi167 ceph-mon[32751]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T18:05:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:40 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi090 2024-04-26T18:05:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:40 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 5696M 2024-04-26T18:05:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:40 smithi119 ceph-mon[32752]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:40 smithi119 ceph-mon[32752]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T18:05:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:40 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi090 2024-04-26T18:05:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:40 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 5696M 2024-04-26T18:05:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:40 smithi090 ceph-mon[29180]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:05:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:40 smithi090 ceph-mon[29180]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T18:05:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:41 smithi119 ceph-mon[32752]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:05:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:41 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:05:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:41 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:05:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:41 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:41 smithi167 ceph-mon[32751]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:05:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:41 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:05:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:41 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:05:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:41 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:41 smithi090 ceph-mon[29180]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:05:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:41 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:05:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:41 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:05:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:41 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:42 smithi119 ceph-mon[32752]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:42 smithi167 ceph-mon[32751]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:42 smithi090 ceph-mon[29180]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:43 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/3819301010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]: dispatch 2024-04-26T18:05:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:43 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]: dispatch 2024-04-26T18:05:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:43 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]': finished 2024-04-26T18:05:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:43 smithi119 ceph-mon[32752]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:05:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:43 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:05:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:43 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/3819301010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]: dispatch 2024-04-26T18:05:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:43 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]: dispatch 2024-04-26T18:05:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:43 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]': finished 2024-04-26T18:05:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:43 smithi167 ceph-mon[32751]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:05:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:43 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:05:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:43 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/3819301010' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]: dispatch 2024-04-26T18:05:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:43 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]: dispatch 2024-04-26T18:05:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:43 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6aea2198-ac95-4f94-9953-dd093977e34c"}]': finished 2024-04-26T18:05:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:43 smithi090 ceph-mon[29180]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:05:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:43 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:05:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:44 smithi119 ceph-mon[32752]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:44 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/636003004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:05:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:44 smithi167 ceph-mon[32751]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:44 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/636003004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:05:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:44 smithi090 ceph-mon[29180]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:44 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/636003004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:05:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:46 smithi119 ceph-mon[32752]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:46 smithi167 ceph-mon[32751]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:46 smithi090 ceph-mon[29180]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:49.215 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:48 smithi119 ceph-mon[32752]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:48 smithi167 ceph-mon[32751]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:48 smithi090 ceph-mon[29180]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:51.183 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:50 smithi119 ceph-mon[32752]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:50 smithi167 ceph-mon[32751]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:50 smithi090 ceph-mon[29180]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:52.170 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:51 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:05:52.170 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:51 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:51 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:05:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:51 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:51 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:05:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:51 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:05:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:52 smithi119 ceph-mon[32752]: Deploying daemon osd.7 on smithi119 2024-04-26T18:05:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:52 smithi119 ceph-mon[32752]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:52 smithi167 ceph-mon[32751]: Deploying daemon osd.7 on smithi119 2024-04-26T18:05:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:52 smithi167 ceph-mon[32751]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:52 smithi090 ceph-mon[29180]: Deploying daemon osd.7 on smithi119 2024-04-26T18:05:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:52 smithi090 ceph-mon[29180]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:54 smithi119 ceph-mon[32752]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:54 smithi167 ceph-mon[32751]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:54 smithi090 ceph-mon[29180]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:57.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:56 smithi119 ceph-mon[32752]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:57.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:57.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:57.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:56 smithi167 ceph-mon[32751]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:56 smithi090 ceph-mon[29180]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:05:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:05:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:05:58 smithi119 ceph-mon[32752]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:05:58 smithi167 ceph-mon[32751]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:05:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:05:58 smithi090 ceph-mon[29180]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:00.400 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.400+0000 7f3402ffd700 1 -- 172.21.15.119:0/932525247 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f340c063860 con 0x7f33f80a3c40 2024-04-26T18:06:00.400 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 7 on host 'smithi119' 2024-04-26T18:06:00.403 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f33f80a3c40 msgr2=0x7f33f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f33f80a3c40 0x7f33f80a6100 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f340c104530 tx=0x7f3408009180 comp rx=0 tx=0).stop 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 msgr2=0x7f340c1a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c1a0b40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f34040059d0 tx=0x7f34040047b0 comp rx=0 tx=0).stop 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 shutdown_connections 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f33f80a3c40 0x7f33f80a6100 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c105070 0x7f340c1a0b40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f340c1034d0 0x7f340c199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 --2- 172.21.15.119:0/932525247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f340c1046d0 0x7f340c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 >> 172.21.15.119:0/932525247 conn(0x7f340c0fea70 msgr2=0x7f340c107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 shutdown_connections 2024-04-26T18:06:00.404 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:00.403+0000 7f3413c5a700 1 -- 172.21.15.119:0/932525247 wait complete. 2024-04-26T18:06:01.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:00 smithi119 ceph-mon[32752]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:01.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:01.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:01.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:00 smithi119 ceph-mon[32752]: from='osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:06:01.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:00 smithi119 ceph-mon[32752]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:06:01.059 DEBUG:teuthology.orchestra.run.smithi119:osd.7> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T18:06:01.062 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-04-26T18:06:01.062 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:06:01.309 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:06:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:00 smithi167 ceph-mon[32751]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:00 smithi167 ceph-mon[32751]: from='osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:06:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:00 smithi167 ceph-mon[32751]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:06:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:00 smithi090 ceph-mon[29180]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:00 smithi090 ceph-mon[29180]: from='osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:06:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:00 smithi090 ceph-mon[29180]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:06:02.458 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:02.458 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:02.458 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:06:02.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: from='osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:06:02.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:06:02.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:02.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:06:02.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:02 smithi167 ceph-mon[32751]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: from='osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:02 smithi119 ceph-mon[32752]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:06:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: from='osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:06:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:06:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:06:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:02 smithi090 ceph-mon[29180]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:03.173 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:06:03.173 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:06:03.173 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-04-26T18:06:03.173 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-04-26T18:06:03.173 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012864 s, 815 MB/s 2024-04-26T18:06:03.173 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-04-26T18:06:03.405 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 26 18:06:03 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-7[51281]: 2024-04-26T18:06:03.294+0000 7fe431c41700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T18:06:03.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:03 smithi167 ceph-mon[32751]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:06:03.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:03 smithi167 ceph-mon[32751]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T18:06:03.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:03 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:03.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:03 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:03.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:03 smithi167 ceph-mon[32751]: from='osd.7 ' entity='osd.7' 2024-04-26T18:06:03.803 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 26 18:06:03 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-7[51281]: 2024-04-26T18:06:03.465+0000 7fe429a2c700 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:06:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:03 smithi119 ceph-mon[32752]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:06:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:03 smithi119 ceph-mon[32752]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T18:06:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:03 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:03 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:03 smithi119 ceph-mon[32752]: from='osd.7 ' entity='osd.7' 2024-04-26T18:06:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:03 smithi090 ceph-mon[29180]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:06:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:03 smithi090 ceph-mon[29180]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T18:06:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:03 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:03 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:03 smithi090 ceph-mon[29180]: from='osd.7 ' entity='osd.7' 2024-04-26T18:06:03.908 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-04-26T18:06:04.156 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:06:04.411 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:06:04.411 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:06:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:06:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:06:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T18:06:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] boot 2024-04-26T18:06:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:06:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:04 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T18:06:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] boot 2024-04-26T18:06:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:06:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:04 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:06:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:06:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: osdmap e52: 8 total, 7 up, 8 in 2024-04-26T18:06:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:06:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: osd.7 [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] boot 2024-04-26T18:06:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:06:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:04 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi119 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 5696M 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:05.748 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.749 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:05.749 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:05.749 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:05.749 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.749 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:05 smithi167 ceph-mon[32751]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:06:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi119 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 5696M 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:05 smithi090 ceph-mon[29180]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 -- 172.21.15.167:0/503894904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc4103f60 msgr2=0x7fefc410a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 --2- 172.21.15.167:0/503894904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc4103f60 0x7fefc410a810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00b0a0 tx=0x7fefbc00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 -- 172.21.15.167:0/503894904 shutdown_connections 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 --2- 172.21.15.167:0/503894904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc4103f60 0x7fefc410a810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 --2- 172.21.15.167:0/503894904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc41023c0 0x7fefc41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 --2- 172.21.15.167:0/503894904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefc41035c0 0x7fefc4103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 -- 172.21.15.167:0/503894904 >> 172.21.15.167:0/503894904 conn(0x7fefc40fd980 msgr2=0x7fefc40ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 -- 172.21.15.167:0/503894904 shutdown_connections 2024-04-26T18:06:05.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.952+0000 7fefc8c86700 1 -- 172.21.15.167:0/503894904 wait complete. 2024-04-26T18:06:05.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 Processor -- start 2024-04-26T18:06:05.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 -- start start 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc41023c0 0x7fefc4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc41035c0 0x7fefc4199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefc4103f60 0x7fefc41a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410de50 con 0x7fefc4103f60 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410dcd0 con 0x7fefc41035c0 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc8c86700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefc410dfd0 con 0x7fefc41023c0 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc41023c0 0x7fefc4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc41023c0 0x7fefc4199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:44790/0 (socket says 172.21.15.167:44790) 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.953+0000 7fefc259c700 1 -- 172.21.15.167:0/3674088600 learned_addr learned my addr 172.21.15.167:0/3674088600 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc1d9b700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc41035c0 0x7fefc4199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:05.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc2d9d700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefc4103f60 0x7fefc41a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc259c700 1 -- 172.21.15.167:0/3674088600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc41035c0 msgr2=0x7fefc4199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc259c700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc41035c0 0x7fefc4199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc259c700 1 -- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefc4103f60 msgr2=0x7fefc41a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc259c700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefc4103f60 0x7fefc41a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc259c700 1 -- 172.21.15.167:0/3674088600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc41a11e0 con 0x7fefc41023c0 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc1d9b700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc41035c0 0x7fefc4199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc2d9d700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefc4103f60 0x7fefc41a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc259c700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc41023c0 0x7fefc4199580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefb800e980 tx=0x7fefb800ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefb37fe700 1 -- 172.21.15.167:0/3674088600 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb800cbd0 con 0x7fefc41023c0 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefb37fe700 1 -- 172.21.15.167:0/3674088600 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefb80044e0 con 0x7fefc41023c0 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc41a14d0 con 0x7fefc41023c0 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefb37fe700 1 -- 172.21.15.167:0/3674088600 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefb8010430 con 0x7fefc41023c0 2024-04-26T18:06:05.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.954+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc41a19c0 con 0x7fefc41023c0 2024-04-26T18:06:05.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.955+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc406a110 con 0x7fefc41023c0 2024-04-26T18:06:05.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.956+0000 7fefb37fe700 1 -- 172.21.15.167:0/3674088600 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fefb8003f10 con 0x7fefc41023c0 2024-04-26T18:06:05.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.957+0000 7fefb37fe700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefac0a3c40 0x7fefac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:05.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.957+0000 7fefb37fe700 1 -- 172.21.15.167:0/3674088600 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7fefb80cfdb0 con 0x7fefc41023c0 2024-04-26T18:06:05.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.957+0000 7fefc1d9b700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefac0a3c40 0x7fefac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:05.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.958+0000 7fefc1d9b700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefac0a3c40 0x7fefac0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefb4005950 tx=0x7fefb40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:06:05.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:05.961+0000 7fefb37fe700 1 -- 172.21.15.167:0/3674088600 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefb809d270 con 0x7fefc41023c0 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi119 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 5696M 2024-04-26T18:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:05 smithi119 ceph-mon[32752]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:06:06.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:06.168+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fefc406c260 con 0x7fefac0a3c40 2024-04-26T18:06:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:06 smithi167 ceph-mon[32751]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:06 smithi167 ceph-mon[32751]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:06:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:06 smithi090 ceph-mon[29180]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:06 smithi090 ceph-mon[29180]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:06 smithi119 ceph-mon[32752]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:06 smithi119 ceph-mon[32752]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:06:07.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:07 smithi167 ceph-mon[32751]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:07.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:07 smithi167 ceph-mon[32751]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T18:06:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:07 smithi090 ceph-mon[29180]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:07 smithi090 ceph-mon[29180]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T18:06:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:07 smithi119 ceph-mon[32752]: from='client.24304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:07 smithi119 ceph-mon[32752]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T18:06:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:08 smithi090 ceph-mon[29180]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:08 smithi090 ceph-mon[29180]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T18:06:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:08 smithi119 ceph-mon[32752]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:08 smithi119 ceph-mon[32752]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T18:06:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:08 smithi167 ceph-mon[32751]: pgmap v184: 1 pgs: 1 remapped+peering; 577 KiB data, 249 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:08 smithi167 ceph-mon[32751]: osdmap e57: 8 total, 8 up, 8 in 2024-04-26T18:06:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:09 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/1956729973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]: dispatch 2024-04-26T18:06:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:09 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]: dispatch 2024-04-26T18:06:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:09 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]': finished 2024-04-26T18:06:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:09 smithi090 ceph-mon[29180]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:06:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:09 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:09 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/3878908622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:06:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:09 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/1956729973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]: dispatch 2024-04-26T18:06:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:09 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]: dispatch 2024-04-26T18:06:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:09 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]': finished 2024-04-26T18:06:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:09 smithi119 ceph-mon[32752]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:06:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:09 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:09 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/3878908622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:06:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:09 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/1956729973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]: dispatch 2024-04-26T18:06:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:09 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]: dispatch 2024-04-26T18:06:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:09 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b9aca44-5296-4726-b01f-80153f4c806d"}]': finished 2024-04-26T18:06:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:09 smithi167 ceph-mon[32751]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:06:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:09 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:09 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/3878908622' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:06:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:10 smithi090 ceph-mon[29180]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:10 smithi119 ceph-mon[32752]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:10 smithi167 ceph-mon[32751]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:12 smithi167 ceph-mon[32751]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:12 smithi090 ceph-mon[29180]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:12 smithi119 ceph-mon[32752]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:14 smithi090 ceph-mon[29180]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:15.175 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:14 smithi167 ceph-mon[32751]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:14 smithi119 ceph-mon[32752]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:17.198 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:16 smithi167 ceph-mon[32751]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:17.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:06:17.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:16 smithi119 ceph-mon[32752]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:06:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:16 smithi090 ceph-mon[29180]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:06:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:18.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:17 smithi167 ceph-mon[32751]: Deploying daemon osd.8 on smithi167 2024-04-26T18:06:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:17 smithi119 ceph-mon[32752]: Deploying daemon osd.8 on smithi167 2024-04-26T18:06:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:17 smithi090 ceph-mon[29180]: Deploying daemon osd.8 on smithi167 2024-04-26T18:06:19.148 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:18 smithi167 ceph-mon[32751]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:18 smithi119 ceph-mon[32752]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:18 smithi090 ceph-mon[29180]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:21.207 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:20 smithi167 ceph-mon[32751]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:20 smithi119 ceph-mon[32752]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:20 smithi090 ceph-mon[29180]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:23.164 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:22 smithi167 ceph-mon[32751]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:23.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:23.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:23.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:22 smithi119 ceph-mon[32752]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:22 smithi090 ceph-mon[29180]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:25.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:24 smithi167 ceph-mon[32751]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:24 smithi119 ceph-mon[32752]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:24 smithi090 ceph-mon[29180]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:25.987 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-04-26T18:06:25.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.985+0000 7fefb37fe700 1 -- 172.21.15.167:0/3674088600 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fefc406c260 con 0x7fefac0a3c40 2024-04-26T18:06:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.987+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefac0a3c40 msgr2=0x7fefac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.987+0000 7fefc8c86700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefac0a3c40 0x7fefac0a6100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefb4005950 tx=0x7fefb40058e0 comp rx=0 tx=0).stop 2024-04-26T18:06:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.987+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc41023c0 msgr2=0x7fefc4199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.987+0000 7fefc8c86700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc41023c0 0x7fefc4199580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fefb800e980 tx=0x7fefb800ed40 comp rx=0 tx=0).stop 2024-04-26T18:06:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.987+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 shutdown_connections 2024-04-26T18:06:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.987+0000 7fefc8c86700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefac0a3c40 0x7fefac0a6100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.987+0000 7fefc8c86700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefc4103f60 0x7fefc41a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.988+0000 7fefc8c86700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefc41023c0 0x7fefc4199580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.988+0000 7fefc8c86700 1 --2- 172.21.15.167:0/3674088600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefc41035c0 0x7fefc4199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.988+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 >> 172.21.15.167:0/3674088600 conn(0x7fefc40fd980 msgr2=0x7fefc406bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:06:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.988+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 shutdown_connections 2024-04-26T18:06:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:06:25.988+0000 7fefc8c86700 1 -- 172.21.15.167:0/3674088600 wait complete. 2024-04-26T18:06:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:26 smithi167 ceph-mon[32751]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:06:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:26 smithi090 ceph-mon[29180]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:06:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:26 smithi119 ceph-mon[32752]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:06:26.687 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T18:06:26.690 INFO:tasks.cephadm:Deploying osd.9 on smithi090 with /dev/vg_nvme/lv_1... 2024-04-26T18:06:26.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:06:26.937 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:06:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:27 smithi090 ceph-mon[29180]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:06:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:27 smithi090 ceph-mon[29180]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T18:06:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:27 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:27 smithi090 ceph-mon[29180]: from='osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:06:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:27 smithi090 ceph-mon[29180]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:06:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:27 smithi167 ceph-mon[32751]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:06:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:27 smithi167 ceph-mon[32751]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T18:06:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:27 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:27 smithi167 ceph-mon[32751]: from='osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:06:28.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:27 smithi167 ceph-mon[32751]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:06:28.227 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:06:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:06:27.951+0000 7f61c04cb700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T18:06:28.227 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:06:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:06:27.963+0000 7f61bc2be700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:06:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:27 smithi119 ceph-mon[32752]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:06:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:27 smithi119 ceph-mon[32752]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T18:06:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:27 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:27 smithi119 ceph-mon[32752]: from='osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:06:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:27 smithi119 ceph-mon[32752]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:06:29.014 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:06:29.014 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:06:29.015 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-26T18:06:29.015 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-26T18:06:29.015 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0131925 s, 795 MB/s 2024-04-26T18:06:29.015 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 2024-04-26T18:06:29.015 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-26T18:06:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:28 smithi119 ceph-mon[32752]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:28 smithi119 ceph-mon[32752]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:06:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:28 smithi119 ceph-mon[32752]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T18:06:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:29.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:28 smithi090 ceph-mon[29180]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:29.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:28 smithi090 ceph-mon[29180]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:06:29.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:28 smithi090 ceph-mon[29180]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T18:06:29.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:29.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:28 smithi167 ceph-mon[32751]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:28 smithi167 ceph-mon[32751]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:06:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:28 smithi167 ceph-mon[32751]: osdmap e60: 9 total, 8 up, 9 in 2024-04-26T18:06:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:29.690 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_1 2024-04-26T18:06:29.940 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:06:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:06:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:06:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] boot 2024-04-26T18:06:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:06:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:29 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] boot 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:29 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: osd.8 [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] boot 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:30.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:30.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:30.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:30.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:29 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:31.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:30 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi167 2024-04-26T18:06:31.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:30 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 5696M 2024-04-26T18:06:31.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:30 smithi090 ceph-mon[29180]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:31.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:30 smithi090 ceph-mon[29180]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:06:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:30 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi167 2024-04-26T18:06:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:30 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 5696M 2024-04-26T18:06:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:30 smithi119 ceph-mon[32752]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:30 smithi119 ceph-mon[32752]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:06:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:30 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi167 2024-04-26T18:06:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:30 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 5696M 2024-04-26T18:06:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:30 smithi167 ceph-mon[32751]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 250 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:06:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:30 smithi167 ceph-mon[32751]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:06:31.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 -- 172.21.15.90:0/2541697362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20101d10 msgr2=0x7f2d20104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2541697362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20101d10 0x7f2d20104140 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d1800b0a0 tx=0x7f2d1800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 -- 172.21.15.90:0/2541697362 shutdown_connections 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2541697362 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20101d10 0x7f2d20104140 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2541697362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d20107050 0x7f2d201074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2541697362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d20104680 0x7f2d20106b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 -- 172.21.15.90:0/2541697362 >> 172.21.15.90:0/2541697362 conn(0x7f2d200fd950 msgr2=0x7f2d200ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 -- 172.21.15.90:0/2541697362 shutdown_connections 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 -- 172.21.15.90:0/2541697362 wait complete. 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 Processor -- start 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.514+0000 7f2d281ba700 1 -- start start 2024-04-26T18:06:31.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d20101d10 0x7f2d2019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d20104680 0x7f2d201a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20107050 0x7f2d201a7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20115580 con 0x7f2d20107050 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20115400 con 0x7f2d20101d10 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20115700 con 0x7f2d20104680 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20107050 0x7f2d201a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20107050 0x7f2d201a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43394/0 (socket says 172.21.15.90:43394) 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 -- 172.21.15.90:0/2247536741 learned_addr learned my addr 172.21.15.90:0/2247536741 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d25f56700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d20101d10 0x7f2d2019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:31.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d25755700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d20104680 0x7f2d201a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 -- 172.21.15.90:0/2247536741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d20104680 msgr2=0x7f2d201a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d20104680 0x7f2d201a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 -- 172.21.15.90:0/2247536741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d20101d10 msgr2=0x7f2d2019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d20101d10 0x7f2d2019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 -- 172.21.15.90:0/2247536741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d201a7960 con 0x7f2d20107050 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d26757700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20107050 0x7f2d201a7290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b550 tx=0x7f2d1c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d16ffd700 1 -- 172.21.15.90:0/2247536741 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c015020 con 0x7f2d20107050 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d201a7c50 con 0x7f2d20107050 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d16ffd700 1 -- 172.21.15.90:0/2247536741 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d1c011ce0 con 0x7f2d20107050 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d16ffd700 1 -- 172.21.15.90:0/2247536741 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c010dd0 con 0x7f2d20107050 2024-04-26T18:06:31.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.515+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d201a8140 con 0x7f2d20107050 2024-04-26T18:06:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.516+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2006a110 con 0x7f2d20107050 2024-04-26T18:06:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.517+0000 7f2d16ffd700 1 -- 172.21.15.90:0/2247536741 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2d1c018020 con 0x7f2d20107050 2024-04-26T18:06:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.518+0000 7f2d16ffd700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2d0c0a3c40 0x7f2d0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.518+0000 7f2d25f56700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2d0c0a3c40 0x7f2d0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.518+0000 7f2d16ffd700 1 -- 172.21.15.90:0/2247536741 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5858+0+0 (secure 0 0 0) 0x7f2d1c0cf6e0 con 0x7f2d20107050 2024-04-26T18:06:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.518+0000 7f2d25f56700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2d0c0a3c40 0x7f2d0c0a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009fd0 tx=0x7f2d18009c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:06:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.522+0000 7f2d16ffd700 1 -- 172.21.15.90:0/2247536741 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2d1c01f020 con 0x7f2d20107050 2024-04-26T18:06:31.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:31.730+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f2d20063860 con 0x7f2d0c0a3c40 2024-04-26T18:06:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:31 smithi090 ceph-mon[29180]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:06:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:31 smithi119 ceph-mon[32752]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:06:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:31 smithi167 ceph-mon[32751]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:06:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:32 smithi119 ceph-mon[32752]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:32 smithi119 ceph-mon[32752]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:32 smithi119 ceph-mon[32752]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T18:06:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:32 smithi167 ceph-mon[32751]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:32 smithi167 ceph-mon[32751]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:32 smithi167 ceph-mon[32751]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T18:06:33.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:32 smithi090 ceph-mon[29180]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:33.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:32 smithi090 ceph-mon[29180]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:33.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:32 smithi090 ceph-mon[29180]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T18:06:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:33 smithi090 ceph-mon[29180]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T18:06:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:33 smithi119 ceph-mon[32752]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T18:06:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:33 smithi167 ceph-mon[32751]: osdmap e65: 9 total, 9 up, 9 in 2024-04-26T18:06:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:34 smithi119 ceph-mon[32752]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:34 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/753643326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d262c94d-6bc8-4107-851a-36b1db393708"}]: dispatch 2024-04-26T18:06:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:34 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/753643326' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d262c94d-6bc8-4107-851a-36b1db393708"}]': finished 2024-04-26T18:06:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:34 smithi119 ceph-mon[32752]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:06:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:34 smithi167 ceph-mon[32751]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:34 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/753643326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d262c94d-6bc8-4107-851a-36b1db393708"}]: dispatch 2024-04-26T18:06:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:34 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/753643326' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d262c94d-6bc8-4107-851a-36b1db393708"}]': finished 2024-04-26T18:06:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:34 smithi167 ceph-mon[32751]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:06:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:34 smithi090 ceph-mon[29180]: pgmap v206: 1 pgs: 1 remapped+peering; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:34 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/753643326' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d262c94d-6bc8-4107-851a-36b1db393708"}]: dispatch 2024-04-26T18:06:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:34 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/753643326' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d262c94d-6bc8-4107-851a-36b1db393708"}]': finished 2024-04-26T18:06:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:34 smithi090 ceph-mon[29180]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:06:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:35 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1182419348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:06:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:35 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1182419348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:06:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:35 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1182419348' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:06:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:36 smithi119 ceph-mon[32752]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-26T18:06:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:36 smithi167 ceph-mon[32751]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-26T18:06:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:36 smithi090 ceph-mon[29180]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-26T18:06:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:38 smithi119 ceph-mon[32752]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:06:39.316 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:38 smithi090 ceph-mon[29180]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:06:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:38 smithi167 ceph-mon[32751]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 277 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:06:41.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:40 smithi090 ceph-mon[29180]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T18:06:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:40 smithi119 ceph-mon[32752]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T18:06:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:40 smithi167 ceph-mon[32751]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-26T18:06:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:42 smithi119 ceph-mon[32752]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T18:06:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:06:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:42 smithi167 ceph-mon[32751]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T18:06:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:06:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:43.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:42 smithi090 ceph-mon[29180]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-26T18:06:43.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:06:43.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:44.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:44 smithi090 ceph-mon[29180]: Deploying daemon osd.9 on smithi090 2024-04-26T18:06:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:44 smithi119 ceph-mon[32752]: Deploying daemon osd.9 on smithi090 2024-04-26T18:06:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:44 smithi167 ceph-mon[32751]: Deploying daemon osd.9 on smithi090 2024-04-26T18:06:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:45 smithi119 ceph-mon[32752]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:06:45.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:45 smithi090 ceph-mon[29180]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:06:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:45 smithi167 ceph-mon[32751]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:06:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:46 smithi119 ceph-mon[32752]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:46.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:46 smithi090 ceph-mon[29180]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:46 smithi167 ceph-mon[32751]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:48.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:48 smithi090 ceph-mon[29180]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:48.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:48.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:48.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:48 smithi119 ceph-mon[32752]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:48 smithi167 ceph-mon[32751]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:50 smithi119 ceph-mon[32752]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:50 smithi167 ceph-mon[32751]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:51.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:50 smithi090 ceph-mon[29180]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:51.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.738+0000 7f2d16ffd700 1 -- 172.21.15.90:0/2247536741 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2d20063860 con 0x7f2d0c0a3c40 2024-04-26T18:06:51.739 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 9 on host 'smithi090' 2024-04-26T18:06:51.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2d0c0a3c40 msgr2=0x7f2d0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2d0c0a3c40 0x7f2d0c0a6100 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009fd0 tx=0x7f2d18009c50 comp rx=0 tx=0).stop 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20107050 msgr2=0x7f2d201a7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20107050 0x7f2d201a7290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c00b550 tx=0x7f2d1c00b860 comp rx=0 tx=0).stop 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 shutdown_connections 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2d0c0a3c40 0x7f2d0c0a6100 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2d20107050 0x7f2d201a7290 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d20101d10 0x7f2d2019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.740+0000 7f2d281ba700 1 --2- 172.21.15.90:0/2247536741 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d20104680 0x7f2d201a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.741+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 >> 172.21.15.90:0/2247536741 conn(0x7f2d200fd950 msgr2=0x7f2d20105180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.741+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 shutdown_connections 2024-04-26T18:06:51.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:06:51.741+0000 7f2d281ba700 1 -- 172.21.15.90:0/2247536741 wait complete. 2024-04-26T18:06:52.429 DEBUG:teuthology.orchestra.run.smithi090:osd.9> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T18:06:52.432 INFO:tasks.cephadm:Deploying osd.10 on smithi119 with /dev/vg_nvme/lv_1... 2024-04-26T18:06:52.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:06:52.681 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:06:52.999 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:52 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:52 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:52 smithi119 ceph-mon[32752]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:53.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:52 smithi119 ceph-mon[32752]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:06:53.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:52 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:52 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:52 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:52 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:52 smithi167 ceph-mon[32751]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:52 smithi167 ceph-mon[32751]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:06:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:52 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:52 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:52 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:52 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:52 smithi090 ceph-mon[29180]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:52 smithi090 ceph-mon[29180]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:06:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:52 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:52 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:53.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:53 smithi119 ceph-mon[32752]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:06:53.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:53 smithi119 ceph-mon[32752]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T18:06:53.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:53 smithi119 ceph-mon[32752]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:06:53.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:53 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:53 smithi167 ceph-mon[32751]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:06:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:53 smithi167 ceph-mon[32751]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T18:06:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:53 smithi167 ceph-mon[32751]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:06:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:53 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:53 smithi090 ceph-mon[29180]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:06:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:53 smithi090 ceph-mon[29180]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T18:06:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:53 smithi090 ceph-mon[29180]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T18:06:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:53 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:54.126 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 26 18:06:53 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-9[63795]: 2024-04-26T18:06:53.747+0000 7f37df6b3700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T18:06:54.126 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 26 18:06:53 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-9[63795]: 2024-04-26T18:06:53.763+0000 7f37db4a6700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:06:54.807 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:06:54.807 INFO:teuthology.orchestra.run.smithi119.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:06:54.807 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10+0 records in 2024-04-26T18:06:54.807 INFO:teuthology.orchestra.run.smithi119.stderr:10+0 records out 2024-04-26T18:06:54.807 INFO:teuthology.orchestra.run.smithi119.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122245 s, 858 MB/s 2024-04-26T18:06:54.807 INFO:teuthology.orchestra.run.smithi119.stderr:--> Zapping successful for: 2024-04-26T18:06:54.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:54 smithi119 ceph-mon[32752]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:06:54.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:54 smithi119 ceph-mon[32752]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T18:06:54.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:54.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:54 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:54.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:54 smithi119 ceph-mon[32752]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:54 smithi167 ceph-mon[32751]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:06:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:54 smithi167 ceph-mon[32751]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T18:06:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:54 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:54 smithi167 ceph-mon[32751]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:54 smithi090 ceph-mon[29180]: from='osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T18:06:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:54 smithi090 ceph-mon[29180]: osdmap e68: 10 total, 9 up, 10 in 2024-04-26T18:06:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:54 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:54 smithi090 ceph-mon[29180]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:06:55.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi119:vg_nvme/lv_1 2024-04-26T18:06:55.686 INFO:teuthology.orchestra.run.smithi119.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:06:56.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:55 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:06:56.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:55 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:06:56.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:55 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:56.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:55 smithi119 ceph-mon[32752]: osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] boot 2024-04-26T18:06:56.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:55 smithi119 ceph-mon[32752]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T18:06:56.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:55 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:55 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:06:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:55 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:06:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:55 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:55 smithi167 ceph-mon[32751]: osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] boot 2024-04-26T18:06:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:55 smithi167 ceph-mon[32751]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T18:06:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:55 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:56.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:55 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:06:56.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:55 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:06:56.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:55 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:56.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:55 smithi090 ceph-mon[29180]: osd.9 [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] boot 2024-04-26T18:06:56.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:55 smithi090 ceph-mon[29180]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T18:06:56.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:55 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:06:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: osdmap e70: 10 total, 10 up, 10 in 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:06:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:06:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.286+0000 7fbcb5065700 1 -- 172.21.15.119:0/1528139210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 msgr2=0x7fbcb01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.286+0000 7fbcb5065700 1 --2- 172.21.15.119:0/1528139210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb01038d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbca40099c0 tx=0x7fbca4009cd0 comp rx=0 tx=0).stop 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 -- 172.21.15.119:0/1528139210 shutdown_connections 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 --2- 172.21.15.119:0/1528139210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcb0105050 0x7fbcb010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 --2- 172.21.15.119:0/1528139210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb01038d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 --2- 172.21.15.119:0/1528139210 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb01046b0 0x7fbcb0104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 -- 172.21.15.119:0/1528139210 >> 172.21.15.119:0/1528139210 conn(0x7fbcb00fea30 msgr2=0x7fbcb0100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 -- 172.21.15.119:0/1528139210 shutdown_connections 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 -- 172.21.15.119:0/1528139210 wait complete. 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 Processor -- start 2024-04-26T18:06:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.287+0000 7fbcb5065700 1 -- start start 2024-04-26T18:06:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcb5065700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcb5065700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb01046b0 0x7fbcb0199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcb5065700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcb0105050 0x7fbcb01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcb5065700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010f020 con 0x7fbcb0105050 2024-04-26T18:06:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcb5065700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010eea0 con 0x7fbcb01034b0 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcb5065700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb010f1a0 con 0x7fbcb01046b0 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb0199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35600/0 (socket says 172.21.15.119:35600) 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 -- 172.21.15.119:0/3430136148 learned_addr learned my addr 172.21.15.119:0/3430136148 (peer_addr_for_me v2:172.21.15.119:0/0) 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcae59c700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb01046b0 0x7fbcb0199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaf59e700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcb0105050 0x7fbcb01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 -- 172.21.15.119:0/3430136148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb01046b0 msgr2=0x7fbcb0199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb01046b0 0x7fbcb0199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 -- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcb0105050 msgr2=0x7fbcb01a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcb0105050 0x7fbcb01a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 -- 172.21.15.119:0/3430136148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb01a12f0 con 0x7fbcb01034b0 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcae59c700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb01046b0 0x7fbcb0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:06:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcaed9d700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb0199690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009f50 tx=0x7fbca4003a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:06:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbc97fff700 1 -- 172.21.15.119:0/3430136148 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca4027070 con 0x7fbcb01034b0 2024-04-26T18:06:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb01a1580 con 0x7fbcb01034b0 2024-04-26T18:06:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbc97fff700 1 -- 172.21.15.119:0/3430136148 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca4022470 con 0x7fbcb01034b0 2024-04-26T18:06:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.288+0000 7fbc97fff700 1 -- 172.21.15.119:0/3430136148 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca400f6c0 con 0x7fbcb01034b0 2024-04-26T18:06:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.289+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb01a1b30 con 0x7fbcb01034b0 2024-04-26T18:06:57.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.289+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb006a110 con 0x7fbcb01034b0 2024-04-26T18:06:57.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.290+0000 7fbc97fff700 1 -- 172.21.15.119:0/3430136148 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbca402a070 con 0x7fbcb01034b0 2024-04-26T18:06:57.293 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.291+0000 7fbc97fff700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbc980a3c40 0x7fbc980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:06:57.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.291+0000 7fbc97fff700 1 -- 172.21.15.119:0/3430136148 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7fbca40d0f90 con 0x7fbcb01034b0 2024-04-26T18:06:57.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.291+0000 7fbcae59c700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbc980a3c40 0x7fbc980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:06:57.294 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.292+0000 7fbcae59c700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbc980a3c40 0x7fbc980a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbca00099d0 tx=0x7fbca0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:06:57.295 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.295+0000 7fbc97fff700 1 -- 172.21.15.119:0/3430136148 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbca4013070 con 0x7fbcb01034b0 2024-04-26T18:06:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:06:57.500+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbcb0063920 con 0x7fbc980a3c40 2024-04-26T18:06:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:57 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi090 2024-04-26T18:06:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:57 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 4272M 2024-04-26T18:06:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:57 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:57 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:57 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:58.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:57 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi090 2024-04-26T18:06:58.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:57 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 4272M 2024-04-26T18:06:58.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:57 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:58.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:57 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:58.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:57 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:57 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi090 2024-04-26T18:06:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:57 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 4272M 2024-04-26T18:06:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:57 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:06:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:57 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:06:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:57 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:06:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:58 smithi119 ceph-mon[32752]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:06:58 smithi119 ceph-mon[32752]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:06:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:58 smithi167 ceph-mon[32751]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:06:58 smithi167 ceph-mon[32751]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:06:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:58 smithi090 ceph-mon[29180]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi119:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:06:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:06:58 smithi090 ceph-mon[29180]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:00 smithi119 ceph-mon[32752]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:00 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/2264979810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]: dispatch 2024-04-26T18:07:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:00 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]: dispatch 2024-04-26T18:07:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:00 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]': finished 2024-04-26T18:07:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:00 smithi119 ceph-mon[32752]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:07:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:00 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:00 smithi167 ceph-mon[32751]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:00 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/2264979810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]: dispatch 2024-04-26T18:07:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:00 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]: dispatch 2024-04-26T18:07:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:00 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]': finished 2024-04-26T18:07:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:00 smithi167 ceph-mon[32751]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:07:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:00 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:00 smithi090 ceph-mon[29180]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:00 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/2264979810' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]: dispatch 2024-04-26T18:07:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:00 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]: dispatch 2024-04-26T18:07:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:00 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8873f617-32db-4f50-b47e-b1d3f42eaf4b"}]': finished 2024-04-26T18:07:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:00 smithi090 ceph-mon[29180]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:07:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:00 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:01 smithi119 ceph-mon[32752]: from='client.? 172.21.15.119:0/1457737908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:07:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:01 smithi167 ceph-mon[32751]: from='client.? 172.21.15.119:0/1457737908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:07:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:01 smithi090 ceph-mon[29180]: from='client.? 172.21.15.119:0/1457737908' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:07:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:02 smithi119 ceph-mon[32752]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:02 smithi167 ceph-mon[32751]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:02 smithi090 ceph-mon[29180]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:04 smithi119 ceph-mon[32752]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:04 smithi167 ceph-mon[32751]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:04 smithi090 ceph-mon[29180]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:06 smithi119 ceph-mon[32752]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:06 smithi167 ceph-mon[32751]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:06 smithi090 ceph-mon[29180]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:09.152 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:08 smithi119 ceph-mon[32752]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:09.152 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:07:09.152 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:08 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:08 smithi167 ceph-mon[32751]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:07:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:08 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:08 smithi090 ceph-mon[29180]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:07:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:08 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:10.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:09 smithi119 ceph-mon[32752]: Deploying daemon osd.10 on smithi119 2024-04-26T18:07:10.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:09 smithi090 ceph-mon[29180]: Deploying daemon osd.10 on smithi119 2024-04-26T18:07:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:09 smithi167 ceph-mon[32751]: Deploying daemon osd.10 on smithi119 2024-04-26T18:07:11.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:10 smithi119 ceph-mon[32752]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:11.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:10 smithi090 ceph-mon[29180]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:10 smithi167 ceph-mon[32751]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:12 smithi119 ceph-mon[32752]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:12 smithi167 ceph-mon[32751]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:12 smithi090 ceph-mon[29180]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:15.263 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:14 smithi119 ceph-mon[32752]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:15.263 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:15.263 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:15.264 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:14 smithi167 ceph-mon[32751]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:14 smithi090 ceph-mon[29180]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:16 smithi119 ceph-mon[32752]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:16 smithi167 ceph-mon[32751]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:16 smithi090 ceph-mon[29180]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:17.968 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.968+0000 7fbc97fff700 1 -- 172.21.15.119:0/3430136148 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbcb0063920 con 0x7fbc980a3c40 2024-04-26T18:07:17.968 INFO:teuthology.orchestra.run.smithi119.stdout:Created osd(s) 10 on host 'smithi119' 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbc980a3c40 msgr2=0x7fbc980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbc980a3c40 0x7fbc980a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbca00099d0 tx=0x7fbca0008040 comp rx=0 tx=0).stop 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 msgr2=0x7fbcb0199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb0199690 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbca4009f50 tx=0x7fbca4003a10 comp rx=0 tx=0).stop 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 shutdown_connections 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbc980a3c40 0x7fbc980a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcb0105050 0x7fbcb01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcb01034b0 0x7fbcb0199690 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 --2- 172.21.15.119:0/3430136148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb01046b0 0x7fbcb0199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.970+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 >> 172.21.15.119:0/3430136148 conn(0x7fbcb00fea30 msgr2=0x7fbcb0107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:17.971 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.971+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 shutdown_connections 2024-04-26T18:07:17.972 INFO:teuthology.orchestra.run.smithi119.stderr:2024-04-26T18:07:17.971+0000 7fbcb5065700 1 -- 172.21.15.119:0/3430136148 wait complete. 2024-04-26T18:07:18.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:18.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:18.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:18.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:07:18.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:07:18.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:18 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:18 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:18.624 DEBUG:teuthology.orchestra.run.smithi119:osd.10> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T18:07:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:18 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:18.627 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-04-26T18:07:18.627 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:07:18.878 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:07:19.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:19 smithi167 ceph-mon[32751]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:07:19.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:19 smithi167 ceph-mon[32751]: from='osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:07:19.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:19 smithi167 ceph-mon[32751]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T18:07:19.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:19 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:19.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:19 smithi167 ceph-mon[32751]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:07:19.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:19 smithi119 ceph-mon[32752]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:07:19.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:19 smithi119 ceph-mon[32752]: from='osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:07:19.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:19 smithi119 ceph-mon[32752]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T18:07:19.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:19 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:19.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:19 smithi119 ceph-mon[32752]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:07:19.540 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 26 18:07:19 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-10[57861]: 2024-04-26T18:07:19.160+0000 7fb2fb94d700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T18:07:19.540 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 26 18:07:19 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-10[57861]: 2024-04-26T18:07:19.175+0000 7fb2f7740700 -1 osd.10 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:07:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:19 smithi090 ceph-mon[29180]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:07:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:19 smithi090 ceph-mon[29180]: from='osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:07:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:19 smithi090 ceph-mon[29180]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T18:07:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:19 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:19 smithi090 ceph-mon[29180]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]: dispatch 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] boot 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:07:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] boot 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:07:20.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi119", "root=default"]}]': finished 2024-04-26T18:07:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: osdmap e73: 11 total, 10 up, 11 in 2024-04-26T18:07:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: pgmap v237: 1 pgs: 1 peering; 577 KiB data, 305 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:07:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: osd.10 [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] boot 2024-04-26T18:07:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:07:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:07:20.870 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:07:20.871 INFO:teuthology.orchestra.run.smithi167.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:07:20.871 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10+0 records in 2024-04-26T18:07:20.871 INFO:teuthology.orchestra.run.smithi167.stderr:10+0 records out 2024-04-26T18:07:20.871 INFO:teuthology.orchestra.run.smithi167.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114959 s, 912 MB/s 2024-04-26T18:07:20.871 INFO:teuthology.orchestra.run.smithi167.stderr:--> Zapping successful for: 2024-04-26T18:07:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:21 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:07:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:21 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:07:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:21 smithi119 ceph-mon[32752]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:07:21.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:21 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:07:21.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:21 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:07:21.536 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:21 smithi167 ceph-mon[32751]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:07:21.566 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-04-26T18:07:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:21 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:07:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:21 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:07:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:21 smithi090 ceph-mon[29180]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:07:21.813 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi119 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 4272M 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:22 smithi090 ceph-mon[29180]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:07:23.035 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi119 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 4272M 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:23.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:22 smithi167 ceph-mon[32751]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi119 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 4272M 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: pgmap v240: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:22 smithi119 ceph-mon[32752]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 -- 172.21.15.167:0/931257762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 msgr2=0x7fbe4410a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 --2- 172.21.15.167:0/931257762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe4410a7c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00b0a0 tx=0x7fbe3c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 -- 172.21.15.167:0/931257762 shutdown_connections 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 --2- 172.21.15.167:0/931257762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe4410a7c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 --2- 172.21.15.167:0/931257762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe44102370 0x7fbe44102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 --2- 172.21.15.167:0/931257762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44103570 0x7fbe441039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 -- 172.21.15.167:0/931257762 >> 172.21.15.167:0/931257762 conn(0x7fbe440fd910 msgr2=0x7fbe440ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 -- 172.21.15.167:0/931257762 shutdown_connections 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.527+0000 7fbe49608700 1 -- 172.21.15.167:0/931257762 wait complete. 2024-04-26T18:07:23.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 Processor -- start 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 -- start start 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44102370 0x7fbe441995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe44103570 0x7fbe44199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe441a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410de90 con 0x7fbe44102370 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410dd10 con 0x7fbe44103570 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe49608700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410e010 con 0x7fbe44103f10 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe437fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe441a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe437fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe441a0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:49778/0 (socket says 172.21.15.167:49778) 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.528+0000 7fbe437fe700 1 -- 172.21.15.167:0/3750757035 learned_addr learned my addr 172.21.15.167:0/3750757035 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe427fc700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe44103570 0x7fbe44199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:23.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe42ffd700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44102370 0x7fbe441995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe437fe700 1 -- 172.21.15.167:0/3750757035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe44103570 msgr2=0x7fbe44199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe437fe700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe44103570 0x7fbe44199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe437fe700 1 -- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44102370 msgr2=0x7fbe441995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe437fe700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44102370 0x7fbe441995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe437fe700 1 -- 172.21.15.167:0/3750757035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe441a11f0 con 0x7fbe44103f10 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe437fe700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe441a0b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c004e40 tx=0x7fbe3c004e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe2bfff700 1 -- 172.21.15.167:0/3750757035 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c021070 con 0x7fbe44103f10 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe441a1480 con 0x7fbe44103f10 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe2bfff700 1 -- 172.21.15.167:0/3750757035 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe3c007c00 con 0x7fbe44103f10 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe2bfff700 1 -- 172.21.15.167:0/3750757035 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c004730 con 0x7fbe44103f10 2024-04-26T18:07:23.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.529+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe441a1a00 con 0x7fbe44103f10 2024-04-26T18:07:23.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.530+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4406a110 con 0x7fbe44103f10 2024-04-26T18:07:23.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.531+0000 7fbe2bfff700 1 -- 172.21.15.167:0/3750757035 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbe3c003a00 con 0x7fbe44103f10 2024-04-26T18:07:23.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.532+0000 7fbe2bfff700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbe2c0a3c40 0x7fbe2c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:23.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.532+0000 7fbe2bfff700 1 -- 172.21.15.167:0/3750757035 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6694+0+0 (secure 0 0 0) 0x7fbe3c00f020 con 0x7fbe44103f10 2024-04-26T18:07:23.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.532+0000 7fbe42ffd700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbe2c0a3c40 0x7fbe2c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:23.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.532+0000 7fbe42ffd700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbe2c0a3c40 0x7fbe2c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbe441033d0 tx=0x7fbe34009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:23.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.536+0000 7fbe2bfff700 1 -- 172.21.15.167:0/3750757035 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbe3c02a020 con 0x7fbe44103f10 2024-04-26T18:07:23.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:23.741+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbe440ffae0 con 0x7fbe2c0a3c40 2024-04-26T18:07:24.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:24 smithi167 ceph-mon[32751]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T18:07:24.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:24 smithi167 ceph-mon[32751]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:07:24.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:24 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:07:24.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:24 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:07:24.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:24 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:24.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:24 smithi167 ceph-mon[32751]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:24 smithi119 ceph-mon[32752]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T18:07:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:24 smithi119 ceph-mon[32752]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:07:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:24 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:07:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:24 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:07:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:24 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:24 smithi119 ceph-mon[32752]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:24 smithi090 ceph-mon[29180]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T18:07:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:24 smithi090 ceph-mon[29180]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:07:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:24 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:07:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:24 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:07:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:24 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:24 smithi090 ceph-mon[29180]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:25.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:25 smithi167 ceph-mon[32751]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T18:07:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:25 smithi119 ceph-mon[32752]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T18:07:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:25 smithi090 ceph-mon[29180]: osdmap e78: 11 total, 11 up, 11 in 2024-04-26T18:07:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:26 smithi119 ceph-mon[32752]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-26T18:07:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:26 smithi167 ceph-mon[32751]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-26T18:07:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:26 smithi090 ceph-mon[29180]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-26T18:07:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:27 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/2588164593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]: dispatch 2024-04-26T18:07:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:27 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]: dispatch 2024-04-26T18:07:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:27 smithi167 ceph-mon[32751]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]': finished 2024-04-26T18:07:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:27 smithi167 ceph-mon[32751]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:07:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:27 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:27 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/2588164593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]: dispatch 2024-04-26T18:07:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:27 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]: dispatch 2024-04-26T18:07:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:27 smithi119 ceph-mon[32752]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]': finished 2024-04-26T18:07:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:27 smithi119 ceph-mon[32752]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:07:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:27 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:27 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/2588164593' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]: dispatch 2024-04-26T18:07:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:27 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]: dispatch 2024-04-26T18:07:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:27 smithi090 ceph-mon[29180]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "123cd211-4456-4b3d-8ad3-bd47cdda36a3"}]': finished 2024-04-26T18:07:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:27 smithi090 ceph-mon[29180]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:07:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:27 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:28 smithi119 ceph-mon[32752]: from='client.? 172.21.15.167:0/3421047549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:07:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:28 smithi119 ceph-mon[32752]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-26T18:07:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:28 smithi167 ceph-mon[32751]: from='client.? 172.21.15.167:0/3421047549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:07:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:28 smithi167 ceph-mon[32751]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-26T18:07:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:28 smithi090 ceph-mon[29180]: from='client.? 172.21.15.167:0/3421047549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:07:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:28 smithi090 ceph-mon[29180]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-26T18:07:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:30 smithi119 ceph-mon[32752]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:07:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:30 smithi167 ceph-mon[32751]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:07:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:30 smithi090 ceph-mon[29180]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:07:33.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:32 smithi167 ceph-mon[32751]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:07:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:32 smithi119 ceph-mon[32752]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:07:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:32 smithi090 ceph-mon[29180]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:07:35.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:34 smithi167 ceph-mon[32751]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T18:07:35.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:07:35.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:34 smithi119 ceph-mon[32752]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T18:07:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:07:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:34 smithi090 ceph-mon[29180]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-26T18:07:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:07:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:36.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:35 smithi167 ceph-mon[32751]: Deploying daemon osd.11 on smithi167 2024-04-26T18:07:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:35 smithi119 ceph-mon[32752]: Deploying daemon osd.11 on smithi167 2024-04-26T18:07:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:35 smithi090 ceph-mon[29180]: Deploying daemon osd.11 on smithi167 2024-04-26T18:07:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:36 smithi119 ceph-mon[32752]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:36 smithi167 ceph-mon[32751]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:36 smithi090 ceph-mon[29180]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:39.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:38 smithi167 ceph-mon[32751]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:38 smithi119 ceph-mon[32752]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:38 smithi090 ceph-mon[29180]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:40.943 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:40 smithi167 ceph-mon[32751]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:40.944 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:40.944 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:40.944 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:41.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:40 smithi090 ceph-mon[29180]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:41.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:41.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:41.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:40 smithi119 ceph-mon[32752]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:43.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:42 smithi167 ceph-mon[32751]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:42 smithi119 ceph-mon[32752]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:42 smithi090 ceph-mon[29180]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:44.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.313+0000 7fbe2bfff700 1 -- 172.21.15.167:0/3750757035 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbe440ffae0 con 0x7fbe2c0a3c40 2024-04-26T18:07:44.314 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-04-26T18:07:44.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbe2c0a3c40 msgr2=0x7fbe2c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:44.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbe2c0a3c40 0x7fbe2c0a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbe441033d0 tx=0x7fbe34009190 comp rx=0 tx=0).stop 2024-04-26T18:07:44.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 msgr2=0x7fbe441a0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe441a0b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c004e40 tx=0x7fbe3c004e70 comp rx=0 tx=0).stop 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 shutdown_connections 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbe2c0a3c40 0x7fbe2c0a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe44102370 0x7fbe441995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbe44103f10 0x7fbe441a0b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 --2- 172.21.15.167:0/3750757035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe44103570 0x7fbe44199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 >> 172.21.15.167:0/3750757035 conn(0x7fbe440fd910 msgr2=0x7fbe440ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 shutdown_connections 2024-04-26T18:07:44.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:44.316+0000 7fbe49608700 1 -- 172.21.15.167:0/3750757035 wait complete. 2024-04-26T18:07:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:44.966 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T18:07:44.969 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T18:07:44.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T18:07:45.217 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:07:45.461 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:45 smithi167 ceph-mon[32751]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:07:45.461 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:45 smithi167 ceph-mon[32751]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T18:07:45.461 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:45.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:45 smithi167 ceph-mon[32751]: from='osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:07:45.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:45 smithi167 ceph-mon[32751]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:07:45.462 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 26 18:07:45 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-11[57711]: 2024-04-26T18:07:45.386+0000 7fc92b94d700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T18:07:45.462 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 26 18:07:45 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-11[57711]: 2024-04-26T18:07:45.403+0000 7fc927740700 -1 osd.11 81 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:07:45.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:45 smithi090 ceph-mon[29180]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:07:45.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:45 smithi090 ceph-mon[29180]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T18:07:45.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:45.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:45 smithi090 ceph-mon[29180]: from='osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:07:45.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:45 smithi090 ceph-mon[29180]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:07:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:45 smithi119 ceph-mon[32752]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:07:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:45 smithi119 ceph-mon[32752]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T18:07:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:45 smithi119 ceph-mon[32752]: from='osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:07:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:45 smithi119 ceph-mon[32752]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]: dispatch 2024-04-26T18:07:46.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:46 smithi167 ceph-mon[32751]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:07:46.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:46 smithi167 ceph-mon[32751]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T18:07:46.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:46 smithi167 ceph-mon[32751]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:46.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:46 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:46 smithi090 ceph-mon[29180]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:07:46.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:46 smithi090 ceph-mon[29180]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T18:07:46.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:46 smithi090 ceph-mon[29180]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:46.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:46 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:46 smithi119 ceph-mon[32752]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-04-26T18:07:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:46 smithi119 ceph-mon[32752]: osdmap e81: 12 total, 11 up, 12 in 2024-04-26T18:07:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:46 smithi119 ceph-mon[32752]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:07:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:46 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:46.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.863+0000 7f5b7766d700 1 -- 172.21.15.90:0/1513155613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70104680 msgr2=0x7f5b70104ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:46.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.863+0000 7f5b7766d700 1 --2- 172.21.15.90:0/1513155613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70104680 0x7f5b70104ae0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0099c0 tx=0x7f5b6c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.863+0000 7f5b7766d700 1 -- 172.21.15.90:0/1513155613 shutdown_connections 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.863+0000 7f5b7766d700 1 --2- 172.21.15.90:0/1513155613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b70105020 0x7f5b7010b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.863+0000 7f5b7766d700 1 --2- 172.21.15.90:0/1513155613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70103480 0x7f5b701038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.863+0000 7f5b7766d700 1 --2- 172.21.15.90:0/1513155613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70104680 0x7f5b70104ae0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.863+0000 7f5b7766d700 1 -- 172.21.15.90:0/1513155613 >> 172.21.15.90:0/1513155613 conn(0x7f5b700fea20 msgr2=0x7f5b70100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 -- 172.21.15.90:0/1513155613 shutdown_connections 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 -- 172.21.15.90:0/1513155613 wait complete. 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 Processor -- start 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 -- start start 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70103480 0x7f5b70199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70104680 0x7f5b70199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:46.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b70105020 0x7f5b701a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010ef00 con 0x7f5b70103480 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010ed80 con 0x7f5b70105020 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b7766d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b7010f080 con 0x7f5b70104680 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b75409700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70103480 0x7f5b70199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b75409700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70103480 0x7f5b70199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47946/0 (socket says 172.21.15.90:47946) 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b75409700 1 -- 172.21.15.90:0/2631251675 learned_addr learned my addr 172.21.15.90:0/2631251675 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.864+0000 7f5b74c08700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70104680 0x7f5b70199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b75c0a700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b70105020 0x7f5b701a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b75409700 1 -- 172.21.15.90:0/2631251675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70104680 msgr2=0x7f5b70199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b75409700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70104680 0x7f5b70199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b75409700 1 -- 172.21.15.90:0/2631251675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b70105020 msgr2=0x7f5b701a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:46.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b75409700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b70105020 0x7f5b701a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:46.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b75409700 1 -- 172.21.15.90:0/2631251675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b701a11f0 con 0x7f5b70103480 2024-04-26T18:07:46.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b75409700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70103480 0x7f5b70199590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000d9b0 tx=0x7f5b6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:46.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b667fc700 1 -- 172.21.15.90:0/2631251675 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60017030 con 0x7f5b70103480 2024-04-26T18:07:46.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b701a14e0 con 0x7f5b70103480 2024-04-26T18:07:46.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b667fc700 1 -- 172.21.15.90:0/2631251675 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b600044d0 con 0x7f5b70103480 2024-04-26T18:07:46.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b667fc700 1 -- 172.21.15.90:0/2631251675 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600039e0 con 0x7f5b70103480 2024-04-26T18:07:46.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.865+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b701a1a70 con 0x7f5b70103480 2024-04-26T18:07:46.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.866+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7006a110 con 0x7f5b70103480 2024-04-26T18:07:46.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.867+0000 7f5b667fc700 1 -- 172.21.15.90:0/2631251675 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5b6001a070 con 0x7f5b70103480 2024-04-26T18:07:46.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.868+0000 7f5b667fc700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5b5c0a3c40 0x7f5b5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:46.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.868+0000 7f5b74c08700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5b5c0a3c40 0x7f5b5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:46.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.868+0000 7f5b667fc700 1 -- 172.21.15.90:0/2631251675 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f5b600cfce0 con 0x7f5b70103480 2024-04-26T18:07:46.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.868+0000 7f5b74c08700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5b5c0a3c40 0x7f5b5c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009990 tx=0x7f5b6c0096b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:46.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:46.872+0000 7f5b667fc700 1 -- 172.21.15.90:0/2631251675 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b6000e350 con 0x7f5b70103480 2024-04-26T18:07:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.068+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f5b7002e510 con 0x7f5b70103480 2024-04-26T18:07:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.068+0000 7f5b667fc700 1 -- 172.21.15.90:0/2631251675 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7f5b6009cac0 con 0x7f5b70103480 2024-04-26T18:07:47.069 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:07:47.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5b5c0a3c40 msgr2=0x7f5b5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5b5c0a3c40 0x7f5b5c0a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009990 tx=0x7f5b6c0096b0 comp rx=0 tx=0).stop 2024-04-26T18:07:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70103480 msgr2=0x7f5b70199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70103480 0x7f5b70199590 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000d9b0 tx=0x7f5b6000dcc0 comp rx=0 tx=0).stop 2024-04-26T18:07:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 shutdown_connections 2024-04-26T18:07:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5b5c0a3c40 0x7f5b5c0a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:47.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b70103480 0x7f5b70199590 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:47.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b70105020 0x7f5b701a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:47.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 --2- 172.21.15.90:0/2631251675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b70104680 0x7f5b70199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:47.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 >> 172.21.15.90:0/2631251675 conn(0x7f5b700fea20 msgr2=0x7f5b70107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:47.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 shutdown_connections 2024-04-26T18:07:47.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:47.070+0000 7f5b7766d700 1 -- 172.21.15.90:0/2631251675 wait complete. 2024-04-26T18:07:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:47 smithi090 ceph-mon[29180]: purged_snaps scrub starts 2024-04-26T18:07:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:47 smithi090 ceph-mon[29180]: purged_snaps scrub ok 2024-04-26T18:07:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:47 smithi090 ceph-mon[29180]: osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] boot 2024-04-26T18:07:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:47 smithi090 ceph-mon[29180]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:07:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:47 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:47 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2631251675' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:07:47.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:47 smithi167 ceph-mon[32751]: purged_snaps scrub starts 2024-04-26T18:07:47.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:47 smithi167 ceph-mon[32751]: purged_snaps scrub ok 2024-04-26T18:07:47.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:47 smithi167 ceph-mon[32751]: osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] boot 2024-04-26T18:07:47.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:47 smithi167 ceph-mon[32751]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:07:47.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:47 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:47.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:47 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2631251675' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:07:47.766 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714154866,"num_in_osds":12,"osd_in_since":1714154846,"num_remapped_pgs":0} 2024-04-26T18:07:47.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:07:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:47 smithi119 ceph-mon[32752]: purged_snaps scrub starts 2024-04-26T18:07:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:47 smithi119 ceph-mon[32752]: purged_snaps scrub ok 2024-04-26T18:07:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:47 smithi119 ceph-mon[32752]: osd.11 [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] boot 2024-04-26T18:07:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:47 smithi119 ceph-mon[32752]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:07:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:47 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:07:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:47 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2631251675' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:07:48.011 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:07:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:48 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:07:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:48 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:48.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:48 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:49.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:49 smithi090 ceph-mon[29180]: Detected new or changed devices on smithi167 2024-04-26T18:07:49.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:49 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 4272M 2024-04-26T18:07:49.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 -- 172.21.15.90:0/49429022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 msgr2=0x7fd5181102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:49.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 --2- 172.21.15.90:0/49429022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd5181102c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd514009990 tx=0x7fd514009ca0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 -- 172.21.15.90:0/49429022 shutdown_connections 2024-04-26T18:07:49.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 --2- 172.21.15.90:0/49429022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd5181102c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 --2- 172.21.15.90:0/49429022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd518109480 0x7fd518109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 --2- 172.21.15.90:0/49429022 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181034e0 0x7fd518103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 -- 172.21.15.90:0/49429022 >> 172.21.15.90:0/49429022 conn(0x7fd51807a5e0 msgr2=0x7fd51807a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 -- 172.21.15.90:0/49429022 shutdown_connections 2024-04-26T18:07:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 -- 172.21.15.90:0/49429022 wait complete. 2024-04-26T18:07:49.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 Processor -- start 2024-04-26T18:07:49.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.742+0000 7fd52046b700 1 -- start start 2024-04-26T18:07:49.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd52046b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181034e0 0x7fd518075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:49.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd52046b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd518075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:49.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd52046b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd518109480 0x7fd5180761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:49.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd52046b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd5181135c0 con 0x7fd518103ea0 2024-04-26T18:07:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd52046b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd518113440 con 0x7fd518109480 2024-04-26T18:07:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd52046b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd518113740 con 0x7fd5181034e0 2024-04-26T18:07:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd518075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd518075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47962/0 (socket says 172.21.15.90:47962) 2024-04-26T18:07:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 -- 172.21.15.90:0/3399219112 learned_addr learned my addr 172.21.15.90:0/3399219112 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:07:49.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51ea08700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd518109480 0x7fd5180761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51e207700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181034e0 0x7fd518075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 -- 172.21.15.90:0/3399219112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181034e0 msgr2=0x7fd518075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181034e0 0x7fd518075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 -- 172.21.15.90:0/3399219112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd518109480 msgr2=0x7fd5180761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd518109480 0x7fd5180761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 -- 172.21.15.90:0/3399219112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5181a7f90 con 0x7fd518103ea0 2024-04-26T18:07:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd51da06700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd518075c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d980 tx=0x7fd50800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd50f7fe700 1 -- 172.21.15.90:0/3399219112 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508017070 con 0x7fd518103ea0 2024-04-26T18:07:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd50f7fe700 1 -- 172.21.15.90:0/3399219112 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd508005020 con 0x7fd518103ea0 2024-04-26T18:07:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5181a8280 con 0x7fd518103ea0 2024-04-26T18:07:49.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.743+0000 7fd50f7fe700 1 -- 172.21.15.90:0/3399219112 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508009b50 con 0x7fd518103ea0 2024-04-26T18:07:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.744+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5181a8770 con 0x7fd518103ea0 2024-04-26T18:07:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.744+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd518104b30 con 0x7fd518103ea0 2024-04-26T18:07:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.746+0000 7fd50f7fe700 1 -- 172.21.15.90:0/3399219112 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd5080051d0 con 0x7fd518103ea0 2024-04-26T18:07:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.746+0000 7fd50f7fe700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd5040a3c40 0x7fd5040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:49.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.746+0000 7fd51e207700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd5040a3c40 0x7fd5040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.746+0000 7fd50f7fe700 1 -- 172.21.15.90:0/3399219112 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd5080d0210 con 0x7fd518103ea0 2024-04-26T18:07:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.747+0000 7fd51e207700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd5040a3c40 0x7fd5040a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd510007700 tx=0x7fd510007690 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:49.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.750+0000 7fd50f7fe700 1 -- 172.21.15.90:0/3399219112 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd508020020 con 0x7fd518103ea0 2024-04-26T18:07:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:49 smithi119 ceph-mon[32752]: Detected new or changed devices on smithi167 2024-04-26T18:07:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:49 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 4272M 2024-04-26T18:07:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:49 smithi167 ceph-mon[32751]: Detected new or changed devices on smithi167 2024-04-26T18:07:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:49 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 4272M 2024-04-26T18:07:49.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.953+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd51802e490 con 0x7fd518103ea0 2024-04-26T18:07:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.954+0000 7fd50f7fe700 1 -- 172.21.15.90:0/3399219112 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+18454 (secure 0 0 0) 0x7fd50809d090 con 0x7fd518103ea0 2024-04-26T18:07:49.954 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:07:49.954 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":83,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:07:47.399374+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:07:49.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd5040a3c40 msgr2=0x7fd5040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:49.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd5040a3c40 0x7fd5040a6100 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd510007700 tx=0x7fd510007690 comp rx=0 tx=0).stop 2024-04-26T18:07:49.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 msgr2=0x7fd518075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd518075c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd50800d980 tx=0x7fd50800dc90 comp rx=0 tx=0).stop 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 shutdown_connections 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd5040a3c40 0x7fd5040a6100 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd518109480 0x7fd5180761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5181034e0 0x7fd518075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 --2- 172.21.15.90:0/3399219112 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd518103ea0 0x7fd518075c60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 >> 172.21.15.90:0/3399219112 conn(0x7fd51807a5e0 msgr2=0x7fd518101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 shutdown_connections 2024-04-26T18:07:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:49.956+0000 7fd52046b700 1 -- 172.21.15.90:0/3399219112 wait complete. 2024-04-26T18:07:50.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:50 smithi090 ceph-mon[29180]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:50.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:50 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3399219112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:07:50.728 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T18:03:57.856561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T18:07:50.728 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T18:07:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:50 smithi119 ceph-mon[32752]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:50 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3399219112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:07:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:50 smithi167 ceph-mon[32751]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:50 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3399219112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:07:50.963 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:07:52.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 -- 172.21.15.90:0/34293819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3581094c0 msgr2=0x7fd3581098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/34293819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3581094c0 0x7fd3581098a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd348009990 tx=0x7fd348009ca0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 -- 172.21.15.90:0/34293819 shutdown_connections 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/34293819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3581094c0 0x7fd3581098a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/34293819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd358103e30 0x7fd358110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/34293819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd358103470 0x7fd3581038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 -- 172.21.15.90:0/34293819 >> 172.21.15.90:0/34293819 conn(0x7fd35807a580 msgr2=0x7fd35807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 -- 172.21.15.90:0/34293819 shutdown_connections 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.616+0000 7fd35d4ca700 1 -- 172.21.15.90:0/34293819 wait complete. 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 Processor -- start 2024-04-26T18:07:52.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 -- start start 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd358103470 0x7fd35819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd358103e30 0x7fd35819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3581094c0 0x7fd358199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd358113640 con 0x7fd358103470 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581134c0 con 0x7fd358103e30 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd35d4ca700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3581137c0 con 0x7fd3581094c0 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd358103470 0x7fd35819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd358103470 0x7fd35819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52584/0 (socket says 172.21.15.90:52584) 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 -- 172.21.15.90:0/3004732994 learned_addr learned my addr 172.21.15.90:0/3004732994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd3567fc700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd358103e30 0x7fd35819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd3577fe700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3581094c0 0x7fd358199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:52.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 -- 172.21.15.90:0/3004732994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3581094c0 msgr2=0x7fd358199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3581094c0 0x7fd358199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 -- 172.21.15.90:0/3004732994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd358103e30 msgr2=0x7fd35819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd358103e30 0x7fd35819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd356ffd700 1 -- 172.21.15.90:0/3004732994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35819a0e0 con 0x7fd358103470 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd3577fe700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3581094c0 0x7fd358199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.617+0000 7fd3567fc700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd358103e30 0x7fd35819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.618+0000 7fd356ffd700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd358103470 0x7fd35819f7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3480057c0 tx=0x7fd3480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.618+0000 7fd33ffff700 1 -- 172.21.15.90:0/3004732994 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348027070 con 0x7fd358103470 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.618+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35819a370 con 0x7fd358103470 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.618+0000 7fd33ffff700 1 -- 172.21.15.90:0/3004732994 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd348004020 con 0x7fd358103470 2024-04-26T18:07:52.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.618+0000 7fd33ffff700 1 -- 172.21.15.90:0/3004732994 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348022c40 con 0x7fd358103470 2024-04-26T18:07:52.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.618+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3581a6690 con 0x7fd358103470 2024-04-26T18:07:52.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.619+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35806a110 con 0x7fd358103470 2024-04-26T18:07:52.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.620+0000 7fd33ffff700 1 -- 172.21.15.90:0/3004732994 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd34802a070 con 0x7fd358103470 2024-04-26T18:07:52.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.620+0000 7fd33ffff700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd3400a3c40 0x7fd3400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:52.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.621+0000 7fd3567fc700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd3400a3c40 0x7fd3400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:52.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.621+0000 7fd33ffff700 1 -- 172.21.15.90:0/3004732994 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd3480d0eb0 con 0x7fd358103470 2024-04-26T18:07:52.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.621+0000 7fd3567fc700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd3400a3c40 0x7fd3400a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005950 tx=0x7fd34c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:52.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.625+0000 7fd33ffff700 1 -- 172.21.15.90:0/3004732994 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd34805eff0 con 0x7fd358103470 2024-04-26T18:07:52.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.821+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd35802e490 con 0x7fd358103470 2024-04-26T18:07:52.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.822+0000 7fd33ffff700 1 -- 172.21.15.90:0/3004732994 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fd348013070 con 0x7fd358103470 2024-04-26T18:07:52.822 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-26T18:07:52.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd3400a3c40 msgr2=0x7fd3400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:52.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd3400a3c40 0x7fd3400a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005950 tx=0x7fd34c0058e0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd358103470 msgr2=0x7fd35819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd358103470 0x7fd35819f7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd3480057c0 tx=0x7fd3480057f0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 shutdown_connections 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd3400a3c40 0x7fd3400a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd358103470 0x7fd35819f7c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3581094c0 0x7fd358199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 --2- 172.21.15.90:0/3004732994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd358103e30 0x7fd35819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 >> 172.21.15.90:0/3004732994 conn(0x7fd35807a580 msgr2=0x7fd3581022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 shutdown_connections 2024-04-26T18:07:52.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:52.824+0000 7fd35d4ca700 1 -- 172.21.15.90:0/3004732994 wait complete. 2024-04-26T18:07:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:52 smithi090 ceph-mon[29180]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:52 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3004732994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:07:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:52 smithi119 ceph-mon[32752]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:52 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3004732994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:07:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:52 smithi167 ceph-mon[32751]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:52 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3004732994' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:07:53.487 INFO:tasks.cephadm:Adding mds.a on smithi090 2024-04-26T18:07:53.488 INFO:tasks.cephadm:Adding mds.d on smithi090 2024-04-26T18:07:53.488 INFO:tasks.cephadm:Adding mds.g on smithi090 2024-04-26T18:07:53.488 INFO:tasks.cephadm:Adding mds.j on smithi090 2024-04-26T18:07:53.488 INFO:tasks.cephadm:Adding mds.b on smithi119 2024-04-26T18:07:53.488 INFO:tasks.cephadm:Adding mds.e on smithi119 2024-04-26T18:07:53.489 INFO:tasks.cephadm:Adding mds.h on smithi119 2024-04-26T18:07:53.489 INFO:tasks.cephadm:Adding mds.k on smithi119 2024-04-26T18:07:53.489 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-04-26T18:07:53.489 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-04-26T18:07:53.490 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-04-26T18:07:53.490 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-04-26T18:07:53.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-04-26T18:07:53.741 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:07:55.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:54 smithi167 ceph-mon[32751]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:54 smithi119 ceph-mon[32752]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:55.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 -- 172.21.15.167:0/1700143318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268105020 msgr2=0x7fe26810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:55.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/1700143318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268105020 0x7fe26810b8d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe260009990 tx=0x7fe260009ca0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 -- 172.21.15.167:0/1700143318 shutdown_connections 2024-04-26T18:07:55.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/1700143318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268105020 0x7fe26810b8d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/1700143318 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268103480 0x7fe2681038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/1700143318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe268104680 0x7fe268104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 -- 172.21.15.167:0/1700143318 >> 172.21.15.167:0/1700143318 conn(0x7fe2680fea20 msgr2=0x7fe268100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 -- 172.21.15.167:0/1700143318 shutdown_connections 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 -- 172.21.15.167:0/1700143318 wait complete. 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 Processor -- start 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.354+0000 7fe26e0c4700 1 -- start start 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe26e0c4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268103480 0x7fe2681994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe26e0c4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe268104680 0x7fe268199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe26e0c4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268105020 0x7fe2681a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:55.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe26e0c4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810ef20 con 0x7fe268104680 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe26e0c4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810eda0 con 0x7fe268105020 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe26e0c4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe26810f0a0 con 0x7fe268103480 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268103480 0x7fe2681994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268103480 0x7fe2681994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:51732/0 (socket says 172.21.15.167:51732) 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 -- 172.21.15.167:0/2665119564 learned_addr learned my addr 172.21.15.167:0/2665119564 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe267fff700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268105020 0x7fe2681a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe266ffd700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe268104680 0x7fe268199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 -- 172.21.15.167:0/2665119564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268105020 msgr2=0x7fe2681a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268105020 0x7fe2681a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 -- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe268104680 msgr2=0x7fe268199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:55.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe268104680 0x7fe268199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 -- 172.21.15.167:0/2665119564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2681a1130 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe2677fe700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268103480 0x7fe2681994d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe258007f40 tx=0x7fe25800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe264ff9700 1 -- 172.21.15.167:0/2665119564 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25800ddd0 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe264ff9700 1 -- 172.21.15.167:0/2665119564 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe258018070 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2681a1420 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.355+0000 7fe264ff9700 1 -- 172.21.15.167:0/2665119564 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2580125b0 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.356+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2681a1910 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.356+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26806a110 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.358+0000 7fe264ff9700 1 -- 172.21.15.167:0/2665119564 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe2580133f0 con 0x7fe268103480 2024-04-26T18:07:55.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.358+0000 7fe264ff9700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.358+0000 7fe264ff9700 1 -- 172.21.15.167:0/2665119564 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe2580d0120 con 0x7fe268103480 2024-04-26T18:07:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.358+0000 7fe266ffd700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.359+0000 7fe266ffd700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe2500a3c40 0x7fe2500a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe25c005ea0 tx=0x7fe25c005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:55.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.362+0000 7fe264ff9700 1 -- 172.21.15.167:0/2665119564 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe258010070 con 0x7fe268103480 2024-04-26T18:07:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:54 smithi090 ceph-mon[29180]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:55.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.568+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) -- 0x7fe268063b20 con 0x7fe2500a3c40 2024-04-26T18:07:55.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.576+0000 7fe264ff9700 1 -- 172.21.15.167:0/2665119564 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe268063b20 con 0x7fe2500a3c40 2024-04-26T18:07:55.576 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe2500a3c40 msgr2=0x7fe2500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe2500a3c40 0x7fe2500a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe25c005ea0 tx=0x7fe25c005e30 comp rx=0 tx=0).stop 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268103480 msgr2=0x7fe2681994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268103480 0x7fe2681994d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe258007f40 tx=0x7fe25800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 shutdown_connections 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe2500a3c40 0x7fe2500a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268105020 0x7fe2681a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe268103480 0x7fe2681994d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 --2- 172.21.15.167:0/2665119564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe268104680 0x7fe268199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:55.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.578+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 >> 172.21.15.167:0/2665119564 conn(0x7fe2680fea20 msgr2=0x7fe268107a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:55.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.579+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 shutdown_connections 2024-04-26T18:07:55.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-04-26T18:07:55.579+0000 7fe26e0c4700 1 -- 172.21.15.167:0/2665119564 wait complete. 2024-04-26T18:07:56.307 DEBUG:teuthology.orchestra.run.smithi090:mds.a> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T18:07:56.310 DEBUG:teuthology.orchestra.run.smithi090:mds.d> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T18:07:56.353 DEBUG:teuthology.orchestra.run.smithi090:mds.g> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T18:07:56.396 DEBUG:teuthology.orchestra.run.smithi090:mds.j> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T18:07:56.439 DEBUG:teuthology.orchestra.run.smithi119:mds.b> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T18:07:56.441 DEBUG:teuthology.orchestra.run.smithi119:mds.e> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T18:07:56.485 DEBUG:teuthology.orchestra.run.smithi119:mds.h> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T18:07:56.528 DEBUG:teuthology.orchestra.run.smithi119:mds.k> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T18:07:56.572 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T18:07:56.574 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T18:07:56.617 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T18:07:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:07:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:56.661 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T18:07:56.704 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T18:07:56.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:07:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:56.940 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:07:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='client.24400 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:07:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: Saving service mds.all spec with placement smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:07:57.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:57 smithi090 ceph-mon[29180]: Deploying daemon mds.l on smithi167 2024-04-26T18:07:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:57 smithi167 ceph-mon[32751]: Deploying daemon mds.l on smithi167 2024-04-26T18:07:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:57 smithi119 ceph-mon[32752]: Deploying daemon mds.l on smithi167 2024-04-26T18:07:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 -- 172.21.15.90:0/3023584579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c1025b0 msgr2=0x7fb75c102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 --2- 172.21.15.90:0/3023584579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c1025b0 0x7fb75c102990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7580099c0 tx=0x7fb758009cd0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 -- 172.21.15.90:0/3023584579 shutdown_connections 2024-04-26T18:07:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 --2- 172.21.15.90:0/3023584579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c1025b0 0x7fb75c102990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 --2- 172.21.15.90:0/3023584579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c107640 0x7fb75c10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 --2- 172.21.15.90:0/3023584579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75c102f60 0x7fb75c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 -- 172.21.15.90:0/3023584579 >> 172.21.15.90:0/3023584579 conn(0x7fb75c0792d0 msgr2=0x7fb75c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 -- 172.21.15.90:0/3023584579 shutdown_connections 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.620+0000 7fb763891700 1 -- 172.21.15.90:0/3023584579 wait complete. 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 Processor -- start 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 -- start start 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c1025b0 0x7fb75c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75c102f60 0x7fb75c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c107640 0x7fb75c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c111460 con 0x7fb75c107640 2024-04-26T18:07:58.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c1112e0 con 0x7fb75c102f60 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb763891700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c1115e0 con 0x7fb75c1025b0 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c107640 0x7fb75c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c107640 0x7fb75c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52602/0 (socket says 172.21.15.90:52602) 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 -- 172.21.15.90:0/3575880395 learned_addr learned my addr 172.21.15.90:0/3575880395 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb76162d700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c1025b0 0x7fb75c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb760e2c700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75c102f60 0x7fb75c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 -- 172.21.15.90:0/3575880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c1025b0 msgr2=0x7fb75c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c1025b0 0x7fb75c19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 -- 172.21.15.90:0/3575880395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75c102f60 msgr2=0x7fb75c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75c102f60 0x7fb75c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 -- 172.21.15.90:0/3575880395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c198020 con 0x7fb75c107640 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb760e2c700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75c102f60 0x7fb75c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:07:58.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.621+0000 7fb761e2e700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c107640 0x7fb75c197840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb75400ef30 tx=0x7fb75400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.622+0000 7fb7527fc700 1 -- 172.21.15.90:0/3575880395 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7540092e0 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.622+0000 7fb7527fc700 1 -- 172.21.15.90:0/3575880395 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb754004bb0 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.622+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c1982b0 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.622+0000 7fb7527fc700 1 -- 172.21.15.90:0/3575880395 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754004280 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.622+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c1a4520 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.623+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75c103a40 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.624+0000 7fb7527fc700 1 -- 172.21.15.90:0/3575880395 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb75401b030 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.624+0000 7fb7527fc700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7480a3c40 0x7fb7480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.624+0000 7fb76162d700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7480a3c40 0x7fb7480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.624+0000 7fb7527fc700 1 -- 172.21.15.90:0/3575880395 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb7540d0450 con 0x7fb75c107640 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.625+0000 7fb76162d700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7480a3c40 0x7fb7480a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb758000c00 tx=0x7fb758005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:07:58.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.628+0000 7fb7527fc700 1 -- 172.21.15.90:0/3575880395 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb754022020 con 0x7fb75c107640 2024-04-26T18:07:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:58 smithi167 ceph-mon[32751]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:58.818 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:58 smithi167 systemd[1]: Starting Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:07:58.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.867+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb75c198d70 con 0x7fb75c107640 2024-04-26T18:07:58.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:58 smithi090 ceph-mon[29180]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:58.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.868+0000 7fb7527fc700 1 -- 172.21.15.90:0/3575880395 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb754010040 con 0x7fb75c107640 2024-04-26T18:07:58.869 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:07:58.869 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7480a3c40 msgr2=0x7fb7480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7480a3c40 0x7fb7480a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb758000c00 tx=0x7fb758005fb0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c107640 msgr2=0x7fb75c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c107640 0x7fb75c197840 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb75400ef30 tx=0x7fb75400c3a0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 shutdown_connections 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7480a3c40 0x7fb7480a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb75c107640 0x7fb75c197840 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb75c1025b0 0x7fb75c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 --2- 172.21.15.90:0/3575880395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb75c102f60 0x7fb75c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:07:58.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 >> 172.21.15.90:0/3575880395 conn(0x7fb75c0792d0 msgr2=0x7fb75c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:07:58.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 shutdown_connections 2024-04-26T18:07:58.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:07:58.870+0000 7fb763891700 1 -- 172.21.15.90:0/3575880395 wait complete. 2024-04-26T18:07:58.872 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 1 2024-04-26T18:07:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:58 smithi119 ceph-mon[32752]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:07:59.184 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:58 smithi167 podman[62522]: 2024-04-26 18:07:58.808405534 +0000 UTC m=+0.018293144 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:07:59.184 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:58 smithi167 podman[62522]: 2024-04-26 18:07:58.932151141 +0000 UTC m=+0.142038800 container create 02d908f2d3cc5f8a4bb22ba2dd03a342772f68c776cd95838cf2e568135bd603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T18:07:59.567 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:59 smithi167 podman[62522]: 2024-04-26 18:07:59.184717932 +0000 UTC m=+0.394605533 container init 02d908f2d3cc5f8a4bb22ba2dd03a342772f68c776cd95838cf2e568135bd603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:07:59.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:59 smithi167 podman[62522]: 2024-04-26 18:07:59.187529257 +0000 UTC m=+0.397416865 container start 02d908f2d3cc5f8a4bb22ba2dd03a342772f68c776cd95838cf2e568135bd603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-04-26T18:07:59.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mds[62541]: -- 172.21.15.167:0/435909632 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5585d54e1a40 con 0x5585d54b5000 2024-04-26T18:07:59.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: starting mds.l at 2024-04-26T18:07:59.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:59 smithi167 bash[62522]: 02d908f2d3cc5f8a4bb22ba2dd03a342772f68c776cd95838cf2e568135bd603 2024-04-26T18:07:59.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:07:59 smithi167 systemd[1]: Started Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3575880395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:boot 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: fsmap 1 up:standby 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:07:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3575880395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:boot 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: fsmap 1 up:standby 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:00.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:07:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3575880395' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:boot 2024-04-26T18:08:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: fsmap 1 up:standby 2024-04-26T18:08:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:08:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:07:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:00 smithi090 ceph-mon[29180]: Deploying daemon mds.k on smithi119 2024-04-26T18:08:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:00 smithi090 ceph-mon[29180]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:01.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:00 smithi119 ceph-mon[32752]: Deploying daemon mds.k on smithi119 2024-04-26T18:08:01.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:00 smithi119 ceph-mon[32752]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:00 smithi167 ceph-mon[32751]: Deploying daemon mds.k on smithi119 2024-04-26T18:08:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:00 smithi167 ceph-mon[32751]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:01.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:08:01.746 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:01 smithi119 systemd[1]: Starting Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:08:01.835 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:01.999 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:01 smithi119 podman[62570]: 2024-04-26 18:08:01.746747641 +0000 UTC m=+0.110320140 container create f193fc625c27ad74cd6ce3d07bcc119c4c28803a37d0a20f89b822aef20bc685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T18:08:01.999 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:01 smithi119 podman[62570]: 2024-04-26 18:08:01.654357328 +0000 UTC m=+0.017929835 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:08:01.999 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:01 smithi119 podman[62570]: 2024-04-26 18:08:01.974150702 +0000 UTC m=+0.337723200 container init f193fc625c27ad74cd6ce3d07bcc119c4c28803a37d0a20f89b822aef20bc685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T18:08:01.999 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:01 smithi119 podman[62570]: 2024-04-26 18:08:01.976869698 +0000 UTC m=+0.340442196 container start f193fc625c27ad74cd6ce3d07bcc119c4c28803a37d0a20f89b822aef20bc685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:08:02.303 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mds[62589]: -- 172.21.15.119:0/3438440777 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574c78e1a40 con 0x5574c78b5000 2024-04-26T18:08:02.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: starting mds.k at 2024-04-26T18:08:02.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:02 smithi119 bash[62570]: f193fc625c27ad74cd6ce3d07bcc119c4c28803a37d0a20f89b822aef20bc685 2024-04-26T18:08:02.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:08:02 smithi119 systemd[1]: Started Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:02 smithi090 ceph-mon[29180]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:03.155 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mon[32752]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:03.155 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.155 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.156 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.156 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:03.156 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:03.156 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:02 smithi167 ceph-mon[32751]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:04.242 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:03 smithi090 ceph-mon[29180]: Deploying daemon mds.j on smithi090 2024-04-26T18:08:04.242 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:boot 2024-04-26T18:08:04.242 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:03 smithi090 ceph-mon[29180]: fsmap 2 up:standby 2024-04-26T18:08:04.242 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:03 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:08:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:03 smithi119 ceph-mon[32752]: Deploying daemon mds.j on smithi090 2024-04-26T18:08:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:boot 2024-04-26T18:08:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:03 smithi119 ceph-mon[32752]: fsmap 2 up:standby 2024-04-26T18:08:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:03 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:08:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:03 smithi167 ceph-mon[32751]: Deploying daemon mds.j on smithi090 2024-04-26T18:08:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:boot 2024-04-26T18:08:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:03 smithi167 ceph-mon[32751]: fsmap 2 up:standby 2024-04-26T18:08:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:03 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 -- 172.21.15.90:0/3243378152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a8102f60 msgr2=0x7fe9a8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 --2- 172.21.15.90:0/3243378152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a8102f60 0x7fe9a8106f60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe9940099c0 tx=0x7fe994009cd0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 -- 172.21.15.90:0/3243378152 shutdown_connections 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 --2- 172.21.15.90:0/3243378152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a8107640 0x7fe9a810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 --2- 172.21.15.90:0/3243378152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9a81025b0 0x7fe9a8102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 --2- 172.21.15.90:0/3243378152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a8102f60 0x7fe9a8106f60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 -- 172.21.15.90:0/3243378152 >> 172.21.15.90:0/3243378152 conn(0x7fe9a80792d0 msgr2=0x7fe9a80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:04.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 -- 172.21.15.90:0/3243378152 shutdown_connections 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.341+0000 7fe9af243700 1 -- 172.21.15.90:0/3243378152 wait complete. 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 Processor -- start 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 -- start start 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a81025b0 0x7fe9a819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9a8102f60 0x7fe9a819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a8107640 0x7fe9a81a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8112530 con 0x7fe9a81025b0 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81123b0 con 0x7fe9a8102f60 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9af243700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a81126b0 con 0x7fe9a8107640 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a81025b0 0x7fe9a819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:04.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a81025b0 0x7fe9a819aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54200/0 (socket says 172.21.15.90:54200) 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 -- 172.21.15.90:0/2117930485 learned_addr learned my addr 172.21.15.90:0/2117930485 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9a7fff700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9a8102f60 0x7fe9a819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 -- 172.21.15.90:0/2117930485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a8107640 msgr2=0x7fe9a81a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a8107640 0x7fe9a81a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 -- 172.21.15.90:0/2117930485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9a8102f60 msgr2=0x7fe9a819afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9a8102f60 0x7fe9a819afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.342+0000 7fe9acfdf700 1 -- 172.21.15.90:0/2117930485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a81a6020 con 0x7fe9a81025b0 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.343+0000 7fe9acfdf700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a81025b0 0x7fe9a819aa60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00d950 tx=0x7fe99c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.343+0000 7fe9a5ffb700 1 -- 172.21.15.90:0/2117930485 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c016070 con 0x7fe9a81025b0 2024-04-26T18:08:04.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.343+0000 7fe9a5ffb700 1 -- 172.21.15.90:0/2117930485 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe99c009df0 con 0x7fe9a81025b0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.343+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81a6250 con 0x7fe9a81025b0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.343+0000 7fe9a5ffb700 1 -- 172.21.15.90:0/2117930485 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c00f3c0 con 0x7fe9a81025b0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.343+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81a6760 con 0x7fe9a81025b0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.344+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a806a110 con 0x7fe9a81025b0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.345+0000 7fe9a5ffb700 1 -- 172.21.15.90:0/2117930485 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe99c019070 con 0x7fe9a81025b0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.346+0000 7fe9a5ffb700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9980a3c40 0x7fe9980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.346+0000 7fe9a7fff700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9980a3c40 0x7fe9980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.346+0000 7fe9a5ffb700 1 -- 172.21.15.90:0/2117930485 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fe99c0cfb30 con 0x7fe9a81025b0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.346+0000 7fe9a7fff700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9980a3c40 0x7fe9980a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe994006010 tx=0x7fe994005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:04.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.349+0000 7fe9a5ffb700 1 -- 172.21.15.90:0/2117930485 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe99c09c9b0 con 0x7fe9a81025b0 2024-04-26T18:08:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.592+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9a802e650 con 0x7fe9a81025b0 2024-04-26T18:08:04.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.593+0000 7fe9a5ffb700 1 -- 172.21.15.90:0/2117930485 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7fe99c096e20 con 0x7fe9a81025b0 2024-04-26T18:08:04.593 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:04.593 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T18:08:04.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9980a3c40 msgr2=0x7fe9980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:04.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9980a3c40 0x7fe9980a6100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe994006010 tx=0x7fe994005c60 comp rx=0 tx=0).stop 2024-04-26T18:08:04.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a81025b0 msgr2=0x7fe9a819aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:04.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a81025b0 0x7fe9a819aa60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00d950 tx=0x7fe99c00dc60 comp rx=0 tx=0).stop 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 shutdown_connections 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9980a3c40 0x7fe9980a6100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9a81025b0 0x7fe9a819aa60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9a8107640 0x7fe9a81a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 --2- 172.21.15.90:0/2117930485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9a8102f60 0x7fe9a819afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 >> 172.21.15.90:0/2117930485 conn(0x7fe9a80792d0 msgr2=0x7fe9a81004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 shutdown_connections 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:04.595+0000 7fe9af243700 1 -- 172.21.15.90:0/2117930485 wait complete. 2024-04-26T18:08:04.596 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 3 2024-04-26T18:08:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:04 smithi090 ceph-mon[29180]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:04 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2117930485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:04 smithi119 ceph-mon[32752]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:04 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2117930485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:04 smithi167 ceph-mon[32751]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:04 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2117930485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:06.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:08:06 smithi090 systemd[1]: Started Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:07.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:06 smithi090 ceph-mon[29180]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:07.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:07.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:07.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:06 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:06 smithi119 ceph-mon[32752]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:06 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:06 smithi167 ceph-mon[32751]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:06 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:07.696 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:08:08.018 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:08.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:07 smithi090 ceph-mon[29180]: Deploying daemon mds.d on smithi090 2024-04-26T18:08:08.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:boot 2024-04-26T18:08:08.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:07 smithi090 ceph-mon[29180]: fsmap 3 up:standby 2024-04-26T18:08:08.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:07 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:08:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:07 smithi119 ceph-mon[32752]: Deploying daemon mds.d on smithi090 2024-04-26T18:08:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:boot 2024-04-26T18:08:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:07 smithi119 ceph-mon[32752]: fsmap 3 up:standby 2024-04-26T18:08:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:07 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:08:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:07 smithi167 ceph-mon[32751]: Deploying daemon mds.d on smithi090 2024-04-26T18:08:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:boot 2024-04-26T18:08:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:07 smithi167 ceph-mon[32751]: fsmap 3 up:standby 2024-04-26T18:08:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:07 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:08:09.243 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:08 smithi090 ceph-mon[29180]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:08 smithi119 ceph-mon[32752]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:08 smithi167 ceph-mon[32751]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:09.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 -- 172.21.15.90:0/4031371653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 msgr2=0x7fee30103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 --2- 172.21.15.90:0/4031371653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30103930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fee200099c0 tx=0x7fee20009cd0 comp rx=0 tx=0).stop 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 -- 172.21.15.90:0/4031371653 shutdown_connections 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 --2- 172.21.15.90:0/4031371653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee30103e70 0x7fee30110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 --2- 172.21.15.90:0/4031371653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee30109550 0x7fee30109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 --2- 172.21.15.90:0/4031371653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30103930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 -- 172.21.15.90:0/4031371653 >> 172.21.15.90:0/4031371653 conn(0x7fee3007a610 msgr2=0x7fee3007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 -- 172.21.15.90:0/4031371653 shutdown_connections 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.732+0000 7fee3879a700 1 -- 172.21.15.90:0/4031371653 wait complete. 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 Processor -- start 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 -- start start 2024-04-26T18:08:09.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee30103e70 0x7fee30075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee30109550 0x7fee300761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fee30113630 con 0x7fee301034b0 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee301134b0 con 0x7fee30103e70 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee3879a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee301137b0 con 0x7fee30109550 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee36536700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee36536700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54242/0 (socket says 172.21.15.90:54242) 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee36536700 1 -- 172.21.15.90:0/1041541594 learned_addr learned my addr 172.21.15.90:0/1041541594 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.733+0000 7fee36d37700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee30109550 0x7fee300761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee35d35700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee30103e70 0x7fee30075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee36536700 1 -- 172.21.15.90:0/1041541594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee30109550 msgr2=0x7fee300761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee36536700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee30109550 0x7fee300761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:09.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee36536700 1 -- 172.21.15.90:0/1041541594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee30103e70 msgr2=0x7fee30075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee36536700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee30103e70 0x7fee30075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee36536700 1 -- 172.21.15.90:0/1041541594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee301a80b0 con 0x7fee301034b0 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee36d37700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee30109550 0x7fee300761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee36536700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30075720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c920 tx=0x7fee2c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee277fe700 1 -- 172.21.15.90:0/1041541594 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c007b00 con 0x7fee301034b0 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee301a83a0 con 0x7fee301034b0 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee277fe700 1 -- 172.21.15.90:0/1041541594 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee2c004ce0 con 0x7fee301034b0 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee277fe700 1 -- 172.21.15.90:0/1041541594 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c005650 con 0x7fee301034b0 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.734+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301a88b0 con 0x7fee301034b0 2024-04-26T18:08:09.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.735+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3006a110 con 0x7fee301034b0 2024-04-26T18:08:09.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.736+0000 7fee277fe700 1 -- 172.21.15.90:0/1041541594 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fee2c007ca0 con 0x7fee301034b0 2024-04-26T18:08:09.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.736+0000 7fee277fe700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fee1c0a3c40 0x7fee1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:09.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.737+0000 7fee35d35700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fee1c0a3c40 0x7fee1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:09.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.737+0000 7fee277fe700 1 -- 172.21.15.90:0/1041541594 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fee2c0cfea0 con 0x7fee301034b0 2024-04-26T18:08:09.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.737+0000 7fee35d35700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fee1c0a3c40 0x7fee1c0a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee300771b0 tx=0x7fee2001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:09.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.740+0000 7fee277fe700 1 -- 172.21.15.90:0/1041541594 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee2c014030 con 0x7fee301034b0 2024-04-26T18:08:09.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.997+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee30076c50 con 0x7fee301034b0 2024-04-26T18:08:09.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:09.997+0000 7fee277fe700 1 -- 172.21.15.90:0/1041541594 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7fee2c09cd20 con 0x7fee301034b0 2024-04-26T18:08:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:09.998 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T18:08:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fee1c0a3c40 msgr2=0x7fee1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fee1c0a3c40 0x7fee1c0a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fee300771b0 tx=0x7fee2001a040 comp rx=0 tx=0).stop 2024-04-26T18:08:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 msgr2=0x7fee30075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30075720 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c920 tx=0x7fee2c00cce0 comp rx=0 tx=0).stop 2024-04-26T18:08:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 shutdown_connections 2024-04-26T18:08:10.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fee1c0a3c40 0x7fee1c0a6100 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee301034b0 0x7fee30075720 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee30109550 0x7fee300761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 --2- 172.21.15.90:0/1041541594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee30103e70 0x7fee30075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 >> 172.21.15.90:0/1041541594 conn(0x7fee3007a610 msgr2=0x7fee30102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 shutdown_connections 2024-04-26T18:08:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:10.000+0000 7fee3879a700 1 -- 172.21.15.90:0/1041541594 wait complete. 2024-04-26T18:08:10.001 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 4 2024-04-26T18:08:10.708 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:08:10 smithi090 systemd[1]: Started Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1041541594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:11.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:10 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:11.246 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:11.246 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1041541594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:11.246 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.246 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.246 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.246 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:11.247 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:11.247 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:10 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1041541594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:10 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:12.260 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:11 smithi119 ceph-mon[32752]: Deploying daemon mds.e on smithi119 2024-04-26T18:08:12.260 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:boot 2024-04-26T18:08:12.260 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:11 smithi119 ceph-mon[32752]: fsmap 4 up:standby 2024-04-26T18:08:12.261 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:11 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:08:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:11 smithi167 ceph-mon[32751]: Deploying daemon mds.e on smithi119 2024-04-26T18:08:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:boot 2024-04-26T18:08:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:11 smithi167 ceph-mon[32751]: fsmap 4 up:standby 2024-04-26T18:08:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:11 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:08:12.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:11 smithi090 ceph-mon[29180]: Deploying daemon mds.e on smithi119 2024-04-26T18:08:12.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:boot 2024-04-26T18:08:12.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:11 smithi090 ceph-mon[29180]: fsmap 4 up:standby 2024-04-26T18:08:12.320 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:11 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:08:12.933 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:12 smithi119 systemd[1]: Starting Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:08:13.046 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:08:13.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:12 smithi167 ceph-mon[32751]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:13.249 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:12 smithi090 ceph-mon[29180]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:13.268 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:12 smithi119 ceph-mon[32752]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:13.268 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 podman[62888]: 2024-04-26 18:08:13.025548583 +0000 UTC m=+0.113133102 container create a1d026342b5d3d2ae7aa4b6dbf41c6057d7a517f46184dd3feba545931bfa03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T18:08:13.269 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 podman[62888]: 2024-04-26 18:08:12.930562982 +0000 UTC m=+0.018147493 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:08:13.269 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 podman[62888]: 2024-04-26 18:08:13.269652022 +0000 UTC m=+0.357236533 container init a1d026342b5d3d2ae7aa4b6dbf41c6057d7a517f46184dd3feba545931bfa03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T18:08:13.289 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:13.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 podman[62888]: 2024-04-26 18:08:13.272603447 +0000 UTC m=+0.360187967 container start a1d026342b5d3d2ae7aa4b6dbf41c6057d7a517f46184dd3feba545931bfa03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:08:13.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 ceph-mds[62907]: -- 172.21.15.119:0/414418951 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556dd3e39a40 con 0x556dd3e0c800 2024-04-26T18:08:13.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: starting mds.e at 2024-04-26T18:08:13.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 bash[62888]: a1d026342b5d3d2ae7aa4b6dbf41c6057d7a517f46184dd3feba545931bfa03b 2024-04-26T18:08:13.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:08:13 smithi119 systemd[1]: Started Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:14.661 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.661 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.661 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: Deploying daemon mds.f on smithi167 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:boot 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: fsmap 5 up:standby 2024-04-26T18:08:14.662 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:14 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: Deploying daemon mds.f on smithi167 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:14.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:boot 2024-04-26T18:08:14.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: fsmap 5 up:standby 2024-04-26T18:08:14.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:14 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:08:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: Deploying daemon mds.f on smithi167 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:boot 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: fsmap 5 up:standby 2024-04-26T18:08:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:14 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:08:14.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.899+0000 7f1f85fdc700 1 -- 172.21.15.90:0/2290133435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80108340 msgr2=0x7f1f8006bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:14.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.899+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/2290133435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80108340 0x7f1f8006bb70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1f74009990 tx=0x7f1f74009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:14.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.899+0000 7f1f85fdc700 1 -- 172.21.15.90:0/2290133435 shutdown_connections 2024-04-26T18:08:14.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.899+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/2290133435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80108340 0x7f1f8006bb70 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:14.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.899+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/2290133435 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f80105e50 0x7f1f80106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:14.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.899+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/2290133435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f8006c0b0 0x7f1f8006c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.899+0000 7f1f85fdc700 1 -- 172.21.15.90:0/2290133435 >> 172.21.15.90:0/2290133435 conn(0x7f1f800792f0 msgr2=0x7f1f80079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 -- 172.21.15.90:0/2290133435 shutdown_connections 2024-04-26T18:08:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 -- 172.21.15.90:0/2290133435 wait complete. 2024-04-26T18:08:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 Processor -- start 2024-04-26T18:08:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 -- start start 2024-04-26T18:08:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f8006c0b0 0x7f1f801a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:14.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80105e50 0x7f1f801a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:14.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f80108340 0x7f1f801a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:14.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80119ba0 con 0x7f1f80105e50 2024-04-26T18:08:14.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80119a20 con 0x7f1f8006c0b0 2024-04-26T18:08:14.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.900+0000 7f1f85fdc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1f80119d20 con 0x7f1f80108340 2024-04-26T18:08:14.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80105e50 0x7f1f801a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:14.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80105e50 0x7f1f801a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58116/0 (socket says 172.21.15.90:58116) 2024-04-26T18:08:14.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 -- 172.21.15.90:0/3246098789 learned_addr learned my addr 172.21.15.90:0/3246098789 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:14.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7f7fe700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f8006c0b0 0x7f1f801a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:14.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7ffff700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f80108340 0x7f1f801a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:14.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 -- 172.21.15.90:0/3246098789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f80108340 msgr2=0x7f1f801a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:14.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f80108340 0x7f1f801a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:14.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 -- 172.21.15.90:0/3246098789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f8006c0b0 msgr2=0x7f1f801a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:14.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f8006c0b0 0x7f1f801a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:14.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 -- 172.21.15.90:0/3246098789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f801a5180 con 0x7f1f80105e50 2024-04-26T18:08:14.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7f7fe700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f8006c0b0 0x7f1f801a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:08:14.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7effd700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80105e50 0x7f1f801a06b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9a0 tx=0x7f1f7000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:14.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7cff9700 1 -- 172.21.15.90:0/3246098789 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7000cbd0 con 0x7f1f80105e50 2024-04-26T18:08:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7cff9700 1 -- 172.21.15.90:0/3246098789 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7000cd70 con 0x7f1f80105e50 2024-04-26T18:08:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f801a5470 con 0x7f1f80105e50 2024-04-26T18:08:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f7cff9700 1 -- 172.21.15.90:0/3246098789 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f70018680 con 0x7f1f80105e50 2024-04-26T18:08:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.901+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f801accf0 con 0x7f1f80105e50 2024-04-26T18:08:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.902+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f8006a110 con 0x7f1f80105e50 2024-04-26T18:08:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.903+0000 7f1f7cff9700 1 -- 172.21.15.90:0/3246098789 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1f70010430 con 0x7f1f80105e50 2024-04-26T18:08:14.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.904+0000 7f1f7cff9700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1f680a3cc0 0x7f1f680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:14.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.904+0000 7f1f7f7fe700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1f680a3cc0 0x7f1f680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:14.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.904+0000 7f1f7cff9700 1 -- 172.21.15.90:0/3246098789 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f1f700d05e0 con 0x7f1f80105e50 2024-04-26T18:08:14.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.904+0000 7f1f7f7fe700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1f680a3cc0 0x7f1f680a6180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f740098e0 tx=0x7f1f74005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:14.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:14.907+0000 7f1f7cff9700 1 -- 172.21.15.90:0/3246098789 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f7009d460 con 0x7f1f80105e50 2024-04-26T18:08:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.161+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f801a14d0 con 0x7f1f80105e50 2024-04-26T18:08:15.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.161+0000 7f1f7cff9700 1 -- 172.21.15.90:0/3246098789 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f1f7009cd60 con 0x7f1f80105e50 2024-04-26T18:08:15.161 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:15.161 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T18:08:15.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1f680a3cc0 msgr2=0x7f1f680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:15.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1f680a3cc0 0x7f1f680a6180 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f740098e0 tx=0x7f1f74005960 comp rx=0 tx=0).stop 2024-04-26T18:08:15.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80105e50 msgr2=0x7f1f801a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:15.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80105e50 0x7f1f801a06b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000e9a0 tx=0x7f1f7000ecb0 comp rx=0 tx=0).stop 2024-04-26T18:08:15.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 shutdown_connections 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1f680a3cc0 0x7f1f680a6180 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1f80108340 0x7f1f801a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f8006c0b0 0x7f1f801a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 --2- 172.21.15.90:0/3246098789 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1f80105e50 0x7f1f801a06b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 >> 172.21.15.90:0/3246098789 conn(0x7f1f800792f0 msgr2=0x7f1f80101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 shutdown_connections 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:15.163+0000 7f1f85fdc700 1 -- 172.21.15.90:0/3246098789 wait complete. 2024-04-26T18:08:15.164 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 6 2024-04-26T18:08:15.781 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:15 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3246098789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:15 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3246098789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:15.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:15 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3246098789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:16.068 INFO:journalctl@ceph.mds.f.smithi167.stdout:Apr 26 18:08:16 smithi167 podman[62842]: 2024-04-26 18:08:16.010706647 +0000 UTC m=+0.126467542 container create c3d1bb23b8e626c209cc6ae8b3d3477e284aa8334525187ce8562f808264ece1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T18:08:16.456 INFO:journalctl@ceph.mds.f.smithi167.stdout:Apr 26 18:08:16 smithi167 podman[62842]: 2024-04-26 18:08:16.238181987 +0000 UTC m=+0.353942882 container init c3d1bb23b8e626c209cc6ae8b3d3477e284aa8334525187ce8562f808264ece1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-f, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T18:08:16.457 INFO:journalctl@ceph.mds.f.smithi167.stdout:Apr 26 18:08:16 smithi167 podman[62842]: 2024-04-26 18:08:16.240869198 +0000 UTC m=+0.356630103 container start c3d1bb23b8e626c209cc6ae8b3d3477e284aa8334525187ce8562f808264ece1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True) 2024-04-26T18:08:16.457 INFO:journalctl@ceph.mds.f.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mds[62861]: -- 172.21.15.167:0/2967662689 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e3a7df3a40 con 0x55e3a7dc6800 2024-04-26T18:08:16.457 INFO:journalctl@ceph.mds.f.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-f[62857]: starting mds.f at 2024-04-26T18:08:16.457 INFO:journalctl@ceph.mds.f.smithi167.stdout:Apr 26 18:08:16 smithi167 bash[62842]: c3d1bb23b8e626c209cc6ae8b3d3477e284aa8334525187ce8562f808264ece1 2024-04-26T18:08:16.457 INFO:journalctl@ceph.mds.f.smithi167.stdout:Apr 26 18:08:16 smithi167 systemd[1]: Started Ceph mds.f for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:16 smithi119 ceph-mon[32752]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:16 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mon[32751]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:16 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:16 smithi090 ceph-mon[29180]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:16 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:17.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:17 smithi167 ceph-mon[32751]: Deploying daemon mds.c on smithi167 2024-04-26T18:08:17.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:17 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6834/570376817,v1:172.21.15.167:6835/570376817] up:boot 2024-04-26T18:08:17.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:17 smithi167 ceph-mon[32751]: fsmap 6 up:standby 2024-04-26T18:08:17.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:17 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:08:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:17 smithi119 ceph-mon[32752]: Deploying daemon mds.c on smithi167 2024-04-26T18:08:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:17 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6834/570376817,v1:172.21.15.167:6835/570376817] up:boot 2024-04-26T18:08:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:17 smithi119 ceph-mon[32752]: fsmap 6 up:standby 2024-04-26T18:08:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:17 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:08:17.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:08:17.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:17 smithi090 ceph-mon[29180]: Deploying daemon mds.c on smithi167 2024-04-26T18:08:17.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:17 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6834/570376817,v1:172.21.15.167:6835/570376817] up:boot 2024-04-26T18:08:17.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:17 smithi090 ceph-mon[29180]: fsmap 6 up:standby 2024-04-26T18:08:17.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:17 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:08:18.085 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:18.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:18 smithi167 ceph-mon[32751]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:18.728 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:18 smithi167 systemd[1]: Starting Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:08:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:18 smithi119 ceph-mon[32752]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:18 smithi090 ceph-mon[29180]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:19.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:18 smithi167 podman[63149]: 2024-04-26 18:08:18.631812918 +0000 UTC m=+0.017993837 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:08:19.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:18 smithi167 podman[63149]: 2024-04-26 18:08:18.798683195 +0000 UTC m=+0.184864141 container create a0d324860d23eaf098089652289a1fa1e036f1eca55902ac56ac617fdce34b48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:08:19.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:19 smithi167 podman[63149]: 2024-04-26 18:08:19.084743353 +0000 UTC m=+0.470924240 container init a0d324860d23eaf098089652289a1fa1e036f1eca55902ac56ac617fdce34b48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T18:08:19.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:19 smithi167 podman[63149]: 2024-04-26 18:08:19.087383465 +0000 UTC m=+0.473564353 container start a0d324860d23eaf098089652289a1fa1e036f1eca55902ac56ac617fdce34b48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T18:08:19.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:19 smithi167 ceph-mds[63168]: -- 172.21.15.167:0/1083814936 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f9f87cfa40 con 0x55f9f87a3000 2024-04-26T18:08:19.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: starting mds.c at 2024-04-26T18:08:19.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:19 smithi167 bash[63149]: a0d324860d23eaf098089652289a1fa1e036f1eca55902ac56ac617fdce34b48 2024-04-26T18:08:19.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:08:19 smithi167 systemd[1]: Started Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.736+0000 7f8793ac6700 1 -- 172.21.15.90:0/2562198951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c109480 msgr2=0x7f878c109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.736+0000 7f8793ac6700 1 --2- 172.21.15.90:0/2562198951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c109480 0x7f878c109860 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8788009990 tx=0x7f8788009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.736+0000 7f8793ac6700 1 -- 172.21.15.90:0/2562198951 shutdown_connections 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.736+0000 7f8793ac6700 1 --2- 172.21.15.90:0/2562198951 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c109480 0x7f878c109860 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.736+0000 7f8793ac6700 1 --2- 172.21.15.90:0/2562198951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f878c103ea0 0x7f878c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.736+0000 7f8793ac6700 1 --2- 172.21.15.90:0/2562198951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f878c1034e0 0x7f878c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.736+0000 7f8793ac6700 1 -- 172.21.15.90:0/2562198951 >> 172.21.15.90:0/2562198951 conn(0x7f878c07a5e0 msgr2=0x7f878c07a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 -- 172.21.15.90:0/2562198951 shutdown_connections 2024-04-26T18:08:19.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 -- 172.21.15.90:0/2562198951 wait complete. 2024-04-26T18:08:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 Processor -- start 2024-04-26T18:08:19.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 -- start start 2024-04-26T18:08:19.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c1034e0 0x7f878c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:19.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f878c103ea0 0x7f878c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:19.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f878c109480 0x7f878c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f878c113610 con 0x7f878c1034e0 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f878c113490 con 0x7f878c109480 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.737+0000 7f8793ac6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f878c113790 con 0x7f878c103ea0 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c1034e0 0x7f878c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c1034e0 0x7f878c075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58126/0 (socket says 172.21.15.90:58126) 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 -- 172.21.15.90:0/3503222695 learned_addr learned my addr 172.21.15.90:0/3503222695 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8792063700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f878c109480 0x7f878c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791061700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f878c103ea0 0x7f878c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 -- 172.21.15.90:0/3503222695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f878c103ea0 msgr2=0x7f878c075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f878c103ea0 0x7f878c075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 -- 172.21.15.90:0/3503222695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f878c109480 msgr2=0x7f878c0761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f878c109480 0x7f878c0761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:19.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 -- 172.21.15.90:0/3503222695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c1a8070 con 0x7f878c1034e0 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791061700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f878c103ea0 0x7f878c075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8791862700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c1034e0 0x7f878c075720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f878800b5c0 tx=0x7f87880058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8782ffd700 1 -- 172.21.15.90:0/3503222695 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788027070 con 0x7f878c1034e0 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c1a8300 con 0x7f878c1034e0 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8782ffd700 1 -- 172.21.15.90:0/3503222695 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8788003fb0 con 0x7f878c1034e0 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8782ffd700 1 -- 172.21.15.90:0/3503222695 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788022ce0 con 0x7f878c1034e0 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.738+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c1a87b0 con 0x7f878c1034e0 2024-04-26T18:08:19.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.739+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c10b980 con 0x7f878c1034e0 2024-04-26T18:08:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.740+0000 7f8782ffd700 1 -- 172.21.15.90:0/3503222695 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f878802a070 con 0x7f878c1034e0 2024-04-26T18:08:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.741+0000 7f8782ffd700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f87780a3c40 0x7f87780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.741+0000 7f8791061700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f87780a3c40 0x7f87780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.741+0000 7f8782ffd700 1 -- 172.21.15.90:0/3503222695 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f87880d1580 con 0x7f878c1034e0 2024-04-26T18:08:19.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.741+0000 7f8791061700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f87780a3c40 0x7f87780a6100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f878c07acc0 tx=0x7f877c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:19.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.744+0000 7f8782ffd700 1 -- 172.21.15.90:0/3503222695 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8788031070 con 0x7f878c1034e0 2024-04-26T18:08:19.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.998+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f878c104b30 con 0x7f878c1034e0 2024-04-26T18:08:19.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:19.999+0000 7f8782ffd700 1 -- 172.21.15.90:0/3503222695 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f8788013070 con 0x7f878c1034e0 2024-04-26T18:08:19.999 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:19.999 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T18:08:20.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f87780a3c40 msgr2=0x7f87780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:20.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f87780a3c40 0x7f87780a6100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f878c07acc0 tx=0x7f877c008040 comp rx=0 tx=0).stop 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c1034e0 msgr2=0x7f878c075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c1034e0 0x7f878c075720 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f878800b5c0 tx=0x7f87880058e0 comp rx=0 tx=0).stop 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 shutdown_connections 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f87780a3c40 0x7f87780a6100 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f878c1034e0 0x7f878c075720 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f878c109480 0x7f878c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 --2- 172.21.15.90:0/3503222695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f878c103ea0 0x7f878c075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.001+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 >> 172.21.15.90:0/3503222695 conn(0x7f878c07a5e0 msgr2=0x7f878c101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.002+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 shutdown_connections 2024-04-26T18:08:20.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:20.002+0000 7f8793ac6700 1 -- 172.21.15.90:0/3503222695 wait complete. 2024-04-26T18:08:20.003 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 8 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: Deploying daemon mds.h on smithi119 2024-04-26T18:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] up:boot 2024-04-26T18:08:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: fsmap 7 up:standby 2024-04-26T18:08:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:08:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:20 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3503222695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: Deploying daemon mds.h on smithi119 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] up:boot 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: fsmap 7 up:standby 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:20 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3503222695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: Deploying daemon mds.h on smithi119 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] up:boot 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: fsmap 7 up:standby 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:20 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3503222695' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:21.769 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:21 smithi119 systemd[1]: Starting Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:08:22.039 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:21 smithi119 podman[63214]: 2024-04-26 18:08:21.672519691 +0000 UTC m=+0.017810928 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:08:22.040 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:21 smithi119 podman[63214]: 2024-04-26 18:08:21.796220913 +0000 UTC m=+0.141512128 container create c1c36f778f9df7ba738288653f6688bc58a67452edd2a52ce5e30995325f4613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T18:08:22.040 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:22 smithi119 podman[63214]: 2024-04-26 18:08:22.040557588 +0000 UTC m=+0.385848803 container init c1c36f778f9df7ba738288653f6688bc58a67452edd2a52ce5e30995325f4613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T18:08:22.303 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:22 smithi119 podman[63214]: 2024-04-26 18:08:22.043253382 +0000 UTC m=+0.388544603 container start c1c36f778f9df7ba738288653f6688bc58a67452edd2a52ce5e30995325f4613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T18:08:22.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mds[63233]: -- 172.21.15.119:0/1706354621 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56144b72da40 con 0x56144b700c00 2024-04-26T18:08:22.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: starting mds.h at 2024-04-26T18:08:22.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:22 smithi119 bash[63214]: c1c36f778f9df7ba738288653f6688bc58a67452edd2a52ce5e30995325f4613 2024-04-26T18:08:22.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:08:22 smithi119 systemd[1]: Started Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:22.738 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:08:22.991 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:23.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:22 smithi090 ceph-mon[29180]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:22 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:23.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:22 smithi167 ceph-mon[32751]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:23.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:23.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:23.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:22 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mon[32752]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:23.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:23.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:23.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:22 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:24.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:23 smithi167 ceph-mon[32751]: Deploying daemon mds.i on smithi167 2024-04-26T18:08:24.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:boot 2024-04-26T18:08:24.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:23 smithi167 ceph-mon[32751]: fsmap 8 up:standby 2024-04-26T18:08:24.033 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:23 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:08:24.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:23 smithi090 ceph-mon[29180]: Deploying daemon mds.i on smithi167 2024-04-26T18:08:24.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:boot 2024-04-26T18:08:24.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:23 smithi090 ceph-mon[29180]: fsmap 8 up:standby 2024-04-26T18:08:24.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:23 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:08:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:23 smithi119 ceph-mon[32752]: Deploying daemon mds.i on smithi167 2024-04-26T18:08:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:boot 2024-04-26T18:08:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:23 smithi119 ceph-mon[32752]: fsmap 8 up:standby 2024-04-26T18:08:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:23 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:08:24.317 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 systemd[1]: Starting Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 -- 172.21.15.90:0/3137917780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 msgr2=0x7fd050108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 --2- 172.21.15.90:0/3137917780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd050108760 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd0440098f0 tx=0x7fd044009c00 comp rx=0 tx=0).stop 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 -- 172.21.15.90:0/3137917780 shutdown_connections 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 --2- 172.21.15.90:0/3137917780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd050108760 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 --2- 172.21.15.90:0/3137917780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd050102cd0 0x7fd05010f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 --2- 172.21.15.90:0/3137917780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd050102310 0x7fd050102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 -- 172.21.15.90:0/3137917780 >> 172.21.15.90:0/3137917780 conn(0x7fd050079290 msgr2=0x7fd0500796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 -- 172.21.15.90:0/3137917780 shutdown_connections 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.599+0000 7fd054a8c700 1 -- 172.21.15.90:0/3137917780 wait complete. 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 Processor -- start 2024-04-26T18:08:24.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 -- start start 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd050102310 0x7fd0501a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd050102cd0 0x7fd0501a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd05019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd0501124a0 con 0x7fd050108380 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd050112320 con 0x7fd050102cd0 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.600+0000 7fd054a8c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd050112620 con 0x7fd050102310 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd05019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd050102310 0x7fd0501a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd050102310 0x7fd0501a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:39310/0 (socket says 172.21.15.90:39310) 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04e59c700 1 -- 172.21.15.90:0/35835864 learned_addr learned my addr 172.21.15.90:0/35835864 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd05019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34820/0 (socket says 172.21.15.90:34820) 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04dd9b700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd050102cd0 0x7fd0501a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:24.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 -- 172.21.15.90:0/35835864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd050102310 msgr2=0x7fd0501a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd050102310 0x7fd0501a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 -- 172.21.15.90:0/35835864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd050102cd0 msgr2=0x7fd0501a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd050102cd0 0x7fd0501a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 -- 172.21.15.90:0/35835864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05019c450 con 0x7fd050108380 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04dd9b700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd050102cd0 0x7fd0501a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd04ed9d700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd05019bb70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd048007f10 tx=0x7fd04800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd03f7fe700 1 -- 172.21.15.90:0/35835864 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04800ddd0 con 0x7fd050108380 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd03f7fe700 1 -- 172.21.15.90:0/35835864 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd048009e20 con 0x7fd050108380 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05019c740 con 0x7fd050108380 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd03f7fe700 1 -- 172.21.15.90:0/35835864 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048015400 con 0x7fd050108380 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.601+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0501a89d0 con 0x7fd050108380 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.602+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd050103a30 con 0x7fd050108380 2024-04-26T18:08:24.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.603+0000 7fd03f7fe700 1 -- 172.21.15.90:0/35835864 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd048005020 con 0x7fd050108380 2024-04-26T18:08:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.604+0000 7fd03f7fe700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd0380a3c40 0x7fd0380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.604+0000 7fd04e59c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd0380a3c40 0x7fd0380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.604+0000 7fd03f7fe700 1 -- 172.21.15.90:0/35835864 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fd0480cf900 con 0x7fd050108380 2024-04-26T18:08:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.604+0000 7fd04e59c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd0380a3c40 0x7fd0380a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd044009680 tx=0x7fd044019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:24.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.609+0000 7fd03f7fe700 1 -- 172.21.15.90:0/35835864 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd048010070 con 0x7fd050108380 2024-04-26T18:08:24.738 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 podman[63457]: 2024-04-26 18:08:24.357674147 +0000 UTC m=+0.017857503 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:08:24.738 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 podman[63457]: 2024-04-26 18:08:24.461550203 +0000 UTC m=+0.121733540 container create 66d7771cec9fd5e0d21bf457ddad0ec8f4c07848f5745a4533f7820130d689ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:08:24.738 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 podman[63457]: 2024-04-26 18:08:24.739265281 +0000 UTC m=+0.399448618 container init 66d7771cec9fd5e0d21bf457ddad0ec8f4c07848f5745a4533f7820130d689ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T18:08:24.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.860+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0501a8ba0 con 0x7fd050108380 2024-04-26T18:08:24.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.860+0000 7fd03f7fe700 1 -- 172.21.15.90:0/35835864 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fd04809c780 con 0x7fd050108380 2024-04-26T18:08:24.860 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:24.861 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T18:08:24.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd0380a3c40 msgr2=0x7fd0380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:24.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd0380a3c40 0x7fd0380a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd044009680 tx=0x7fd044019040 comp rx=0 tx=0).stop 2024-04-26T18:08:24.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 msgr2=0x7fd05019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:24.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd05019bb70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd048007f10 tx=0x7fd04800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 shutdown_connections 2024-04-26T18:08:24.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd0380a3c40 0x7fd0380a6100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd050108380 0x7fd05019bb70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd050102310 0x7fd0501a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 --2- 172.21.15.90:0/35835864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd050102cd0 0x7fd0501a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:24.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 >> 172.21.15.90:0/35835864 conn(0x7fd050079290 msgr2=0x7fd05006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:24.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 shutdown_connections 2024-04-26T18:08:24.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:24.863+0000 7fd054a8c700 1 -- 172.21.15.90:0/35835864 wait complete. 2024-04-26T18:08:24.864 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 9 2024-04-26T18:08:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:24 smithi167 ceph-mon[32751]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:24 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/35835864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:24 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:24 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 podman[63457]: 2024-04-26 18:08:24.741815065 +0000 UTC m=+0.401998408 container start 66d7771cec9fd5e0d21bf457ddad0ec8f4c07848f5745a4533f7820130d689ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 ceph-mds[63477]: -- 172.21.15.167:0/3807864391 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5632a9e59a40 con 0x5632a9e2cc00 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: starting mds.i at 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 bash[63457]: 66d7771cec9fd5e0d21bf457ddad0ec8f4c07848f5745a4533f7820130d689ed 2024-04-26T18:08:25.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:08:24 smithi167 systemd[1]: Started Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:24 smithi090 ceph-mon[29180]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:24 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/35835864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:24 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:24 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:24 smithi119 ceph-mon[32752]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:24 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/35835864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:24 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:24 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] up:boot 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: fsmap 9 up:standby 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:25 smithi090 ceph-mon[29180]: Deploying daemon mds.g on smithi090 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] up:boot 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: fsmap 9 up:standby 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:25 smithi119 ceph-mon[32752]: Deploying daemon mds.g on smithi090 2024-04-26T18:08:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] up:boot 2024-04-26T18:08:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: fsmap 9 up:standby 2024-04-26T18:08:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:08:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:25 smithi167 ceph-mon[32751]: Deploying daemon mds.g on smithi090 2024-04-26T18:08:27.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:26 smithi090 ceph-mon[29180]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:26 smithi119 ceph-mon[32752]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:26 smithi167 ceph-mon[32751]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:27.375 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 systemd[1]: Starting Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:08:27.631 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:08:27.769 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 podman[70170]: 2024-04-26 18:08:27.467259559 +0000 UTC m=+0.100377781 container create dba63a34a5888de793a9acdfefb10294532e1abcfbe1b4deb2f38c06a029a9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T18:08:27.770 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 podman[70170]: 2024-04-26 18:08:27.38490174 +0000 UTC m=+0.018019973 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:08:27.770 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 podman[70170]: 2024-04-26 18:08:27.686976193 +0000 UTC m=+0.320094425 container init dba63a34a5888de793a9acdfefb10294532e1abcfbe1b4deb2f38c06a029a9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T18:08:27.770 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 podman[70170]: 2024-04-26 18:08:27.690093491 +0000 UTC m=+0.323211721 container start dba63a34a5888de793a9acdfefb10294532e1abcfbe1b4deb2f38c06a029a9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD) 2024-04-26T18:08:27.770 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 ceph-mds[70207]: -- 172.21.15.90:0/647960501 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5570f76e7a40 con 0x5570f76bac00 2024-04-26T18:08:27.770 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: starting mds.g at 2024-04-26T18:08:27.770 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 bash[70170]: dba63a34a5888de793a9acdfefb10294532e1abcfbe1b4deb2f38c06a029a9ae 2024-04-26T18:08:27.887 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:28.020 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:08:27 smithi090 systemd[1]: Started Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:28.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:28.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:28.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:28.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:28.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:28.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:28.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:28.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: Deploying daemon mds.a on smithi090 2024-04-26T18:08:28.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:boot 2024-04-26T18:08:28.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: fsmap 10 up:standby 2024-04-26T18:08:28.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:28 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:08:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: Deploying daemon mds.a on smithi090 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:boot 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: fsmap 10 up:standby 2024-04-26T18:08:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:28 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:08:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: Deploying daemon mds.a on smithi090 2024-04-26T18:08:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:boot 2024-04-26T18:08:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: fsmap 10 up:standby 2024-04-26T18:08:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:28 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:08:30.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.156+0000 7f5f2002e700 1 -- 172.21.15.90:0/2063905788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18108390 msgr2=0x7f5f18108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.156+0000 7f5f2002e700 1 --2- 172.21.15.90:0/2063905788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18108390 0x7f5f18108770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5f14009990 tx=0x7f5f14009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.156+0000 7f5f2002e700 1 -- 172.21.15.90:0/2063905788 shutdown_connections 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.156+0000 7f5f2002e700 1 --2- 172.21.15.90:0/2063905788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18108390 0x7f5f18108770 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.156+0000 7f5f2002e700 1 --2- 172.21.15.90:0/2063905788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f18102cf0 0x7f5f1810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.156+0000 7f5f2002e700 1 --2- 172.21.15.90:0/2063905788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18102350 0x7f5f181027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.156+0000 7f5f2002e700 1 -- 172.21.15.90:0/2063905788 >> 172.21.15.90:0/2063905788 conn(0x7f5f180792c0 msgr2=0x7f5f180796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 -- 172.21.15.90:0/2063905788 shutdown_connections 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 -- 172.21.15.90:0/2063905788 wait complete. 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 Processor -- start 2024-04-26T18:08:30.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 -- start start 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f18102350 0x7f5f1819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18102cf0 0x7f5f1819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18108390 0x7f5f181998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f181124a0 con 0x7f5f18102cf0 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18112320 con 0x7f5f18102350 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f2002e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f18112620 con 0x7f5f18108390 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f1d5c9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18102cf0 0x7f5f1819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f1d5c9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18102cf0 0x7f5f1819fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34846/0 (socket says 172.21.15.90:34846) 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.157+0000 7f5f1d5c9700 1 -- 172.21.15.90:0/3458040087 learned_addr learned my addr 172.21.15.90:0/3458040087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:30.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1e5cb700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18108390 0x7f5f181998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1d5c9700 1 -- 172.21.15.90:0/3458040087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18108390 msgr2=0x7f5f181998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1ddca700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f18102350 0x7f5f1819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1d5c9700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18108390 0x7f5f181998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1d5c9700 1 -- 172.21.15.90:0/3458040087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f18102350 msgr2=0x7f5f1819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1d5c9700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f18102350 0x7f5f1819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1d5c9700 1 -- 172.21.15.90:0/3458040087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1819a030 con 0x7f5f18102cf0 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1e5cb700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18108390 0x7f5f181998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f1d5c9700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18102cf0 0x7f5f1819fc50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800d980 tx=0x7f5f0800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f0effd700 1 -- 172.21.15.90:0/3458040087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08017070 con 0x7f5f18102cf0 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1819a320 con 0x7f5f18102cf0 2024-04-26T18:08:30.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f0effd700 1 -- 172.21.15.90:0/3458040087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f08004830 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f0effd700 1 -- 172.21.15.90:0/3458040087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08005440 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.158+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f181a65e0 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.159+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f18103a40 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.160+0000 7f5f0effd700 1 -- 172.21.15.90:0/3458040087 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5f080055e0 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.161+0000 7f5f0effd700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5f040a3c40 0x7f5f040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.161+0000 7f5f0effd700 1 -- 172.21.15.90:0/3458040087 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f5f080d02b0 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.161+0000 7f5f1ddca700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5f040a3c40 0x7f5f040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.161+0000 7f5f1ddca700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5f040a3c40 0x7f5f040a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b5c0 tx=0x7f5f14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.164+0000 7f5f0effd700 1 -- 172.21.15.90:0/3458040087 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f08020020 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.412+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f1806a110 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.413+0000 7f5f0effd700 1 -- 172.21.15.90:0/3458040087 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f5f1806a110 con 0x7f5f18102cf0 2024-04-26T18:08:30.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5f040a3c40 msgr2=0x7f5f040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5f040a3c40 0x7f5f040a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400b5c0 tx=0x7f5f14005fb0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18102cf0 msgr2=0x7f5f1819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18102cf0 0x7f5f1819fc50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800d980 tx=0x7f5f0800dd40 comp rx=0 tx=0).stop 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 shutdown_connections 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5f040a3c40 0x7f5f040a6100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f18108390 0x7f5f181998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f18102350 0x7f5f1819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 --2- 172.21.15.90:0/3458040087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f18102cf0 0x7f5f1819fc50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 >> 172.21.15.90:0/3458040087 conn(0x7f5f180792c0 msgr2=0x7f5f1806bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 shutdown_connections 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:30.415+0000 7f5f2002e700 1 -- 172.21.15.90:0/3458040087 wait complete. 2024-04-26T18:08:30.684 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 11 2024-04-26T18:08:30.685 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:30.685 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T18:08:30.945 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:30 smithi090 ceph-mon[29180]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:30.945 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:30 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3458040087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:30 smithi119 ceph-mon[32752]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:30 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3458040087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:30 smithi167 ceph-mon[32751]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:30 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3458040087' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:32.125 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:08:31 smithi090 systemd[1]: Started Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:32.973 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:boot 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: fsmap 11 up:standby 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:32.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:32 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:boot 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: fsmap 11 up:standby 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:32 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:boot 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: fsmap 11 up:standby 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:08:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:32 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:33.406 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:08:33.660 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:34.011 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:33 smithi119 ceph-mon[32752]: Deploying daemon mds.b on smithi119 2024-04-26T18:08:34.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:33 smithi090 ceph-mon[29180]: Deploying daemon mds.b on smithi119 2024-04-26T18:08:34.303 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 systemd[1]: Starting Ceph mds.b for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:08:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:33 smithi167 ceph-mon[32751]: Deploying daemon mds.b on smithi119 2024-04-26T18:08:34.650 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 podman[63526]: 2024-04-26 18:08:34.39705391 +0000 UTC m=+0.112192491 container create 98bded1d7c67b97f48b8dfde326d41b7e963ee081c55aaed14262bb6c9cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS) 2024-04-26T18:08:34.650 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 podman[63526]: 2024-04-26 18:08:34.302883898 +0000 UTC m=+0.018022487 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:08:34.650 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 podman[63526]: 2024-04-26 18:08:34.624834231 +0000 UTC m=+0.339972826 container init 98bded1d7c67b97f48b8dfde326d41b7e963ee081c55aaed14262bb6c9cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T18:08:34.650 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 podman[63526]: 2024-04-26 18:08:34.627356442 +0000 UTC m=+0.342495029 container start 98bded1d7c67b97f48b8dfde326d41b7e963ee081c55aaed14262bb6c9cd3a45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:08:34.650 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-mds[63545]: -- 172.21.15.119:0/4051123227 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d95f91fa40 con 0x55d95f8f2800 2024-04-26T18:08:34.919 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: starting mds.b at 2024-04-26T18:08:34.919 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 bash[63526]: 98bded1d7c67b97f48b8dfde326d41b7e963ee081c55aaed14262bb6c9cd3a45 2024-04-26T18:08:34.919 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:34 smithi119 systemd[1]: Started Ceph mds.b for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:08:35.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:34 smithi090 ceph-mon[29180]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:35.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.055 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.055 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.055 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.055 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:34 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 -- 172.21.15.90:0/1743263001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 msgr2=0x7fb6d8109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/1743263001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d8109890 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb6d4009990 tx=0x7fb6d4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 -- 172.21.15.90:0/1743263001 shutdown_connections 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/1743263001 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d8109890 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/1743263001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d81023b0 0x7fb6d81102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/1743263001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d8101a10 0x7fb6d8101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 -- 172.21.15.90:0/1743263001 >> 172.21.15.90:0/1743263001 conn(0x7fb6d8078990 msgr2=0x7fb6d80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:35.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 -- 172.21.15.90:0/1743263001 shutdown_connections 2024-04-26T18:08:35.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 -- 172.21.15.90:0/1743263001 wait complete. 2024-04-26T18:08:35.294 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 Processor -- start 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.292+0000 7fb6dfc69700 1 -- start start 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dfc69700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d8101a10 0x7fb6d81a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dfc69700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d81023b0 0x7fb6d81a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dfc69700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d819bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dfc69700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d81135c0 con 0x7fb6d81094b0 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dfc69700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d8113440 con 0x7fb6d8101a10 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dfc69700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d8113740 con 0x7fb6d81023b0 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d819bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d819bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46220/0 (socket says 172.21.15.90:46220) 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 -- 172.21.15.90:0/2067620936 learned_addr learned my addr 172.21.15.90:0/2067620936 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:35.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dd204700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d81023b0 0x7fb6d81a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dda05700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d8101a10 0x7fb6d81a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 -- 172.21.15.90:0/2067620936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d81023b0 msgr2=0x7fb6d81a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d81023b0 0x7fb6d81a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 -- 172.21.15.90:0/2067620936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d8101a10 msgr2=0x7fb6d81a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d8101a10 0x7fb6d81a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 -- 172.21.15.90:0/2067620936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d819c240 con 0x7fb6d81094b0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6de206700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d819bae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000ef30 tx=0x7fb6d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6ceffd700 1 -- 172.21.15.90:0/2067620936 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d00092e0 con 0x7fb6d81094b0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d819c4d0 con 0x7fb6d81094b0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6ceffd700 1 -- 172.21.15.90:0/2067620936 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6d0004bb0 con 0x7fb6d81094b0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.293+0000 7fb6ceffd700 1 -- 172.21.15.90:0/2067620936 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0004020 con 0x7fb6d81094b0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.294+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d81a87f0 con 0x7fb6d81094b0 2024-04-26T18:08:35.296 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.294+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d806a110 con 0x7fb6d81094b0 2024-04-26T18:08:35.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.296+0000 7fb6ceffd700 1 -- 172.21.15.90:0/2067620936 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb6d001b030 con 0x7fb6d81094b0 2024-04-26T18:08:35.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.296+0000 7fb6ceffd700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb6c40a3c40 0x7fb6c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:35.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.296+0000 7fb6dda05700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb6c40a3c40 0x7fb6c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:35.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.296+0000 7fb6ceffd700 1 -- 172.21.15.90:0/2067620936 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb6d00d0320 con 0x7fb6d81094b0 2024-04-26T18:08:35.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.297+0000 7fb6dda05700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb6c40a3c40 0x7fb6c40a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40098e0 tx=0x7fb6d4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:35.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.300+0000 7fb6ceffd700 1 -- 172.21.15.90:0/2067620936 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6d0022020 con 0x7fb6d81094b0 2024-04-26T18:08:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-mon[32752]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:34 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:34 smithi167 ceph-mon[32751]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:34 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:35.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.581+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6d802e490 con 0x7fb6d81094b0 2024-04-26T18:08:35.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.581+0000 7fb6ceffd700 1 -- 172.21.15.90:0/2067620936 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7fb6d0010040 con 0x7fb6d81094b0 2024-04-26T18:08:35.581 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:35.582 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24412,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T18:08:35.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb6c40a3c40 msgr2=0x7fb6c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:35.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb6c40a3c40 0x7fb6c40a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb6d40098e0 tx=0x7fb6d4005960 comp rx=0 tx=0).stop 2024-04-26T18:08:35.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 msgr2=0x7fb6d819bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:35.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d819bae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000ef30 tx=0x7fb6d000c3a0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 shutdown_connections 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb6c40a3c40 0x7fb6c40a6100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6d81094b0 0x7fb6d819bae0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.583+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d8101a10 0x7fb6d81a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.584+0000 7fb6dfc69700 1 --2- 172.21.15.90:0/2067620936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6d81023b0 0x7fb6d81a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.584+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 >> 172.21.15.90:0/2067620936 conn(0x7fb6d8078990 msgr2=0x7fb6d8106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.584+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 shutdown_connections 2024-04-26T18:08:35.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:35.584+0000 7fb6dfc69700 1 -- 172.21.15.90:0/2067620936 wait complete. 2024-04-26T18:08:35.585 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 13 2024-04-26T18:08:36.018 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:boot 2024-04-26T18:08:36.019 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:35 smithi119 ceph-mon[32752]: fsmap 12 up:standby 2024-04-26T18:08:36.019 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:08:36.019 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:35 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:08:36.019 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:35 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2067620936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:boot 2024-04-26T18:08:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:35 smithi167 ceph-mon[32751]: fsmap 12 up:standby 2024-04-26T18:08:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:08:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:35 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:08:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:35 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2067620936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:36.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:boot 2024-04-26T18:08:36.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:35 smithi090 ceph-mon[29180]: fsmap 12 up:standby 2024-04-26T18:08:36.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:08:36.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:35 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:08:36.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:35 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2067620936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:08:36.558 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-26T18:08:36.994 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:37.261 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:36 smithi090 ceph-mon[29180]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:37.261 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.261 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.261 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.262 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.262 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:36 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:36 smithi119 ceph-mon[32752]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:37.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.269 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.270 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:36 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:36 smithi167 ceph-mon[32751]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:36 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:38.424 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:38 smithi090 ceph-mon[29180]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:38.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 -- 172.21.15.90:0/3830558015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 msgr2=0x7fb4181038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:38.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3830558015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb4181038f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb408009990 tx=0x7fb408009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 -- 172.21.15.90:0/3830558015 shutdown_connections 2024-04-26T18:08:38.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3830558015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb418103e30 0x7fb418110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3830558015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4181094c0 0x7fb4181098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3830558015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb4181038f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 -- 172.21.15.90:0/3830558015 >> 172.21.15.90:0/3830558015 conn(0x7fb41807a580 msgr2=0x7fb41807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:38.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 -- 172.21.15.90:0/3830558015 shutdown_connections 2024-04-26T18:08:38.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 -- 172.21.15.90:0/3830558015 wait complete. 2024-04-26T18:08:38.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 Processor -- start 2024-04-26T18:08:38.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.630+0000 7fb4200ab700 1 -- start start 2024-04-26T18:08:38.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb4200ab700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb418075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:38.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb4200ab700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb418103e30 0x7fb418075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:38.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb4200ab700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4181094c0 0x7fb4180761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb4200ab700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb4181135e0 con 0x7fb418103470 2024-04-26T18:08:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb4200ab700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb418113460 con 0x7fb418103e30 2024-04-26T18:08:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb4200ab700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb418113760 con 0x7fb4181094c0 2024-04-26T18:08:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb418075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb418075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46246/0 (socket says 172.21.15.90:46246) 2024-04-26T18:08:38.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 -- 172.21.15.90:0/3815567537 learned_addr learned my addr 172.21.15.90:0/3815567537 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41d646700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb418103e30 0x7fb418075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 -- 172.21.15.90:0/3815567537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4181094c0 msgr2=0x7fb4180761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41e648700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4181094c0 0x7fb4180761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4181094c0 0x7fb4180761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 -- 172.21.15.90:0/3815567537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb418103e30 msgr2=0x7fb418075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb418103e30 0x7fb418075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 -- 172.21.15.90:0/3815567537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4181a8080 con 0x7fb418103470 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41d646700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb418103e30 0x7fb418075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb41de47700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb418075720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c970 tx=0x7fb41400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:38.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.631+0000 7fb40effd700 1 -- 172.21.15.90:0/3815567537 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb414007aa0 con 0x7fb418103470 2024-04-26T18:08:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.632+0000 7fb40effd700 1 -- 172.21.15.90:0/3815567537 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb414004ce0 con 0x7fb418103470 2024-04-26T18:08:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.632+0000 7fb40effd700 1 -- 172.21.15.90:0/3815567537 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4140055c0 con 0x7fb418103470 2024-04-26T18:08:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.632+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4181a8370 con 0x7fb418103470 2024-04-26T18:08:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.632+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4181a88b0 con 0x7fb418103470 2024-04-26T18:08:38.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.632+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41806a110 con 0x7fb418103470 2024-04-26T18:08:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.634+0000 7fb40effd700 1 -- 172.21.15.90:0/3815567537 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb414007c40 con 0x7fb418103470 2024-04-26T18:08:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.635+0000 7fb40effd700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb4040a3cc0 0x7fb4040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.635+0000 7fb41d646700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb4040a3cc0 0x7fb4040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.635+0000 7fb40effd700 1 -- 172.21.15.90:0/3815567537 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7fb4140cff40 con 0x7fb418103470 2024-04-26T18:08:38.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.635+0000 7fb41d646700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb4040a3cc0 0x7fb4040a6180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b5c0 tx=0x7fb408005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:38.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.639+0000 7fb40effd700 1 -- 172.21.15.90:0/3815567537 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb41409cee0 con 0x7fb418103470 2024-04-26T18:08:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:38 smithi119 ceph-mon[32752]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:38 smithi167 ceph-mon[32751]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:38.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.858+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb41802e490 con 0x7fb418103470 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.859+0000 7fb40effd700 1 -- 172.21.15.90:0/3815567537 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) ==== 84+0+65 (secure 0 0 0) 0x7fb41409c7e0 con 0x7fb418103470 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stdout:[ 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stdout: "poolnum": 1, 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stdout: "poolname": ".mgr" 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:08:38.859 INFO:teuthology.orchestra.run.smithi090.stdout:] 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb4040a3cc0 msgr2=0x7fb4040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb4040a3cc0 0x7fb4040a6180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb40800b5c0 tx=0x7fb408005fb0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 msgr2=0x7fb418075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb418075720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c970 tx=0x7fb41400cd30 comp rx=0 tx=0).stop 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 shutdown_connections 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb4040a3cc0 0x7fb4040a6180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb418103470 0x7fb418075720 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb4181094c0 0x7fb4180761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 --2- 172.21.15.90:0/3815567537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb418103e30 0x7fb418075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 >> 172.21.15.90:0/3815567537 conn(0x7fb41807a580 msgr2=0x7fb418102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 shutdown_connections 2024-04-26T18:08:38.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:38.861+0000 7fb4200ab700 1 -- 172.21.15.90:0/3815567537 wait complete. 2024-04-26T18:08:39.541 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T18:08:39.541 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T18:08:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:08:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:39 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3815567537' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:08:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:39 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:08:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:39 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3815567537' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:39 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:08:40.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:40.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:08:40.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:39 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3815567537' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:08:40.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:39 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:08:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:40 smithi090 ceph-mon[29180]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:40 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:40 smithi167 ceph-mon[32751]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:40 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:40 smithi119 ceph-mon[32752]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:40 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:42.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1070693875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18109480 msgr2=0x7f0c18109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:42.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1070693875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18109480 0x7f0c18109860 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009990 tx=0x7f0c0c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1070693875 shutdown_connections 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1070693875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18109480 0x7f0c18109860 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1070693875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18103ea0 0x7f0c181102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1070693875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c181034e0 0x7f0c18103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1070693875 >> 172.21.15.90:0/1070693875 conn(0x7f0c1807a5e0 msgr2=0x7f0c1807a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1070693875 shutdown_connections 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.153+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1070693875 wait complete. 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 Processor -- start 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 -- start start 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c181034e0 0x7f0c18075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18103ea0 0x7f0c18075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18109480 0x7f0c180761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0c18113660 con 0x7f0c18103ea0 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181134e0 con 0x7f0c181034e0 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1ebc5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c181137e0 con 0x7f0c18109480 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18103ea0 0x7f0c18075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18103ea0 0x7f0c18075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59606/0 (socket says 172.21.15.90:59606) 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 -- 172.21.15.90:0/1890672507 learned_addr learned my addr 172.21.15.90:0/1890672507 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1c961700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c181034e0 0x7f0c18075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c1d162700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18109480 0x7f0c180761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 -- 172.21.15.90:0/1890672507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18109480 msgr2=0x7f0c180761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18109480 0x7f0c180761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 -- 172.21.15.90:0/1890672507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c181034e0 msgr2=0x7f0c18075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c181034e0 0x7f0c18075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.154+0000 7f0c17fff700 1 -- 172.21.15.90:0/1890672507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c181a8090 con 0x7f0c18103ea0 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.155+0000 7f0c17fff700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18103ea0 0x7f0c18075c60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c920 tx=0x7f0c0400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.155+0000 7f0c15ffb700 1 -- 172.21.15.90:0/1890672507 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04007b00 con 0x7f0c18103ea0 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.155+0000 7f0c15ffb700 1 -- 172.21.15.90:0/1890672507 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c04004ce0 con 0x7f0c18103ea0 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.155+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c181a8380 con 0x7f0c18103ea0 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.155+0000 7f0c15ffb700 1 -- 172.21.15.90:0/1890672507 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04005550 con 0x7f0c18103ea0 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.155+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c181a8910 con 0x7f0c18103ea0 2024-04-26T18:08:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.156+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c18104b30 con 0x7f0c18103ea0 2024-04-26T18:08:42.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.157+0000 7f0c15ffb700 1 -- 172.21.15.90:0/1890672507 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0c04007ca0 con 0x7f0c18103ea0 2024-04-26T18:08:42.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.158+0000 7f0c15ffb700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0c080a3cc0 0x7f0c080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:42.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.158+0000 7f0c1c961700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0c080a3cc0 0x7f0c080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:42.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.158+0000 7f0c15ffb700 1 -- 172.21.15.90:0/1890672507 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7127+0+0 (secure 0 0 0) 0x7f0c040cfdb0 con 0x7f0c18103ea0 2024-04-26T18:08:42.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.158+0000 7f0c1c961700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0c080a3cc0 0x7f0c080a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009750 tx=0x7f0c0c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.162+0000 7f0c15ffb700 1 -- 172.21.15.90:0/1890672507 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c04014030 con 0x7f0c18103ea0 2024-04-26T18:08:42.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:42.399+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0c1802e490 con 0x7f0c18103ea0 2024-04-26T18:08:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:42 smithi090 ceph-mon[29180]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:42 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1890672507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:08:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:42 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:43.154 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:42 smithi119 ceph-mon[32752]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:43.154 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:42 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1890672507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:08:43.154 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:43.155 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:42 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:42 smithi167 ceph-mon[32751]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:42 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1890672507' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:08:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:42 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:43.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.324+0000 7f0c15ffb700 1 -- 172.21.15.90:0/1890672507 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) ==== 138+0+0 (secure 0 0 0) 0x7f0c0409cc30 con 0x7f0c18103ea0 2024-04-26T18:08:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0c080a3cc0 msgr2=0x7f0c080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0c080a3cc0 0x7f0c080a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009750 tx=0x7f0c0c005fd0 comp rx=0 tx=0).stop 2024-04-26T18:08:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18103ea0 msgr2=0x7f0c18075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18103ea0 0x7f0c18075c60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c920 tx=0x7f0c0400cc30 comp rx=0 tx=0).stop 2024-04-26T18:08:43.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 shutdown_connections 2024-04-26T18:08:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0c080a3cc0 0x7f0c080a6180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c18109480 0x7f0c180761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c181034e0 0x7f0c18075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 --2- 172.21.15.90:0/1890672507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c18103ea0 0x7f0c18075c60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.328+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 >> 172.21.15.90:0/1890672507 conn(0x7f0c1807a5e0 msgr2=0x7f0c18101570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.329+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 shutdown_connections 2024-04-26T18:08:43.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:43.329+0000 7f0c1ebc5700 1 -- 172.21.15.90:0/1890672507 wait complete. 2024-04-26T18:08:43.331 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_metadata' created 2024-04-26T18:08:43.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:43 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:43.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:43 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:08:43.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:43 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:43.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:43 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1890672507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:08:43.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:43 smithi090 ceph-mon[29180]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:08:43.956 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T18:08:44.193 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:43 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:43 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:08:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:43 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:43 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1890672507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:08:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:43 smithi119 ceph-mon[32752]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:08:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:43 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:08:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:43 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:08:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:43 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:08:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:43 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1890672507' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:08:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:43 smithi167 ceph-mon[32751]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:08:45.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:44 smithi090 ceph-mon[29180]: pgmap v290: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:45.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:44 smithi090 ceph-mon[29180]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:08:45.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:44 smithi090 ceph-mon[29180]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:08:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:44 smithi119 ceph-mon[32752]: pgmap v290: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:44 smithi119 ceph-mon[32752]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:08:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:44 smithi119 ceph-mon[32752]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:08:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:44 smithi167 ceph-mon[32751]: pgmap v290: 33 pgs: 8 creating+peering, 1 active+clean, 24 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:44 smithi167 ceph-mon[32751]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:08:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:44 smithi167 ceph-mon[32751]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:08:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 -- 172.21.15.90:0/1370877742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 msgr2=0x7fb7e010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 --2- 172.21.15.90:0/1370877742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e010f1d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800b0a0 tx=0x7fb7d800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:08:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 -- 172.21.15.90:0/1370877742 shutdown_connections 2024-04-26T18:08:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 --2- 172.21.15.90:0/1370877742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e010f1d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 --2- 172.21.15.90:0/1370877742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7e0108390 0x7fb7e0108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:45.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 --2- 172.21.15.90:0/1370877742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e0102320 0x7fb7e01027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 -- 172.21.15.90:0/1370877742 >> 172.21.15.90:0/1370877742 conn(0x7fb7e00792d0 msgr2=0x7fb7e00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 -- 172.21.15.90:0/1370877742 shutdown_connections 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.793+0000 7fb7e5794700 1 -- 172.21.15.90:0/1370877742 wait complete. 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 Processor -- start 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 -- start start 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e0102320 0x7fb7e0104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e0104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7e0108390 0x7fb7e0104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0112600 con 0x7fb7e0102ce0 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0112480 con 0x7fb7e0102320 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.794+0000 7fb7e5794700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e0112780 con 0x7fb7e0108390 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e0104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:45.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e0104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59612/0 (socket says 172.21.15.90:59612) 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 -- 172.21.15.90:0/2659806770 learned_addr learned my addr 172.21.15.90:0/2659806770 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7df7fe700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7e0108390 0x7fb7e0104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7deffd700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e0102320 0x7fb7e0104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 -- 172.21.15.90:0/2659806770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7e0108390 msgr2=0x7fb7e0104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7e0108390 0x7fb7e0104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 -- 172.21.15.90:0/2659806770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e0102320 msgr2=0x7fb7e0104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e0102320 0x7fb7e0104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 -- 172.21.15.90:0/2659806770 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e01a8110 con 0x7fb7e0102ce0 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7de7fc700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e0104540 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400b530 tx=0x7fb7d400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7c7fff700 1 -- 172.21.15.90:0/2659806770 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4015020 con 0x7fb7e0102ce0 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7c7fff700 1 -- 172.21.15.90:0/2659806770 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d4011ce0 con 0x7fb7e0102ce0 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e01a8400 con 0x7fb7e0102ce0 2024-04-26T18:08:45.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7c7fff700 1 -- 172.21.15.90:0/2659806770 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4010d20 con 0x7fb7e0102ce0 2024-04-26T18:08:45.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.795+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e01a8940 con 0x7fb7e0102ce0 2024-04-26T18:08:45.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.796+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7e006a110 con 0x7fb7e0102ce0 2024-04-26T18:08:45.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.798+0000 7fb7c7fff700 1 -- 172.21.15.90:0/2659806770 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb7d4018020 con 0x7fb7e0102ce0 2024-04-26T18:08:45.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.798+0000 7fb7c7fff700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7c80a3c40 0x7fb7c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:45.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.799+0000 7fb7deffd700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7c80a3c40 0x7fb7c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:45.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.799+0000 7fb7c7fff700 1 -- 172.21.15.90:0/2659806770 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7497+0+0 (secure 0 0 0) 0x7fb7d40cfe50 con 0x7fb7e0102ce0 2024-04-26T18:08:45.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.799+0000 7fb7deffd700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7c80a3c40 0x7fb7c80a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006fd0 tx=0x7fb7d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:45.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:45.802+0000 7fb7c7fff700 1 -- 172.21.15.90:0/2659806770 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7d401f020 con 0x7fb7e0102ce0 2024-04-26T18:08:46.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.004+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb7e0105530 con 0x7fb7e0102ce0 2024-04-26T18:08:46.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.338+0000 7fb7c7fff700 1 -- 172.21.15.90:0/2659806770 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) ==== 170+0+0 (secure 0 0 0) 0x7fb7e0105530 con 0x7fb7e0102ce0 2024-04-26T18:08:46.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.340+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7c80a3c40 msgr2=0x7fb7c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.340+0000 7fb7e5794700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7c80a3c40 0x7fb7c80a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0006fd0 tx=0x7fb7d0008040 comp rx=0 tx=0).stop 2024-04-26T18:08:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.340+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 msgr2=0x7fb7e0104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.340+0000 7fb7e5794700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e0104540 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400b530 tx=0x7fb7d400b8f0 comp rx=0 tx=0).stop 2024-04-26T18:08:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 shutdown_connections 2024-04-26T18:08:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7c80a3c40 0x7fb7c80a6100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:46.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7e0108390 0x7fb7e0104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:46.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7e0102320 0x7fb7e0104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:46.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 --2- 172.21.15.90:0/2659806770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7e0102ce0 0x7fb7e0104540 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:46.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 >> 172.21.15.90:0/2659806770 conn(0x7fb7e00792d0 msgr2=0x7fb7e0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:46.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 shutdown_connections 2024-04-26T18:08:46.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:46.341+0000 7fb7e5794700 1 -- 172.21.15.90:0/2659806770 wait complete. 2024-04-26T18:08:46.344 INFO:teuthology.orchestra.run.smithi090.stderr:pool 'cephfs_data' created 2024-04-26T18:08:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:46 smithi090 ceph-mon[29180]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:08:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:46 smithi090 ceph-mon[29180]: pgmap v293: 33 pgs: 19 active+clean, 1 creating+activating, 8 creating+peering, 5 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:46 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2659806770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:08:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:46 smithi119 ceph-mon[32752]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:08:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:46 smithi119 ceph-mon[32752]: pgmap v293: 33 pgs: 19 active+clean, 1 creating+activating, 8 creating+peering, 5 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:46 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2659806770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:08:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:46 smithi167 ceph-mon[32751]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:08:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:46 smithi167 ceph-mon[32751]: pgmap v293: 33 pgs: 19 active+clean, 1 creating+activating, 8 creating+peering, 5 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:46 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2659806770' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:08:46.987 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T18:08:47.229 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:47 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2659806770' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:08:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:47 smithi090 ceph-mon[29180]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:08:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:47 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2659806770' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:08:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:47 smithi119 ceph-mon[32752]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:08:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:47 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2659806770' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:08:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:47 smithi167 ceph-mon[32751]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:08:48.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:48 smithi090 ceph-mon[29180]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:08:48.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:48 smithi090 ceph-mon[29180]: pgmap v296: 97 pgs: 33 active+clean, 1 creating+activating, 8 creating+peering, 55 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/621206271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c107c20 msgr2=0x7fa19c108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/621206271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c107c20 0x7fa19c108000 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa190009990 tx=0x7fa190009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/621206271 shutdown_connections 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/621206271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c107c20 0x7fa19c108000 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/621206271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa19c1027a0 0x7fa19c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/621206271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa19c101de0 0x7fa19c102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/621206271 >> 172.21.15.90:0/621206271 conn(0x7fa19c077640 msgr2=0x7fa19c077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/621206271 shutdown_connections 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.774+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/621206271 wait complete. 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 Processor -- start 2024-04-26T18:08:48.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 -- start start 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa19c101de0 0x7fa19c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c1027a0 0x7fa19c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa19c107c20 0x7fa19c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c112590 con 0x7fa19c1027a0 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c112410 con 0x7fa19c101de0 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa1a2dfb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c112710 con 0x7fa19c107c20 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.775+0000 7fa19bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c1027a0 0x7fa19c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c1027a0 0x7fa19c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59634/0 (socket says 172.21.15.90:59634) 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 -- 172.21.15.90:0/1702793060 learned_addr learned my addr 172.21.15.90:0/1702793060 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa1a0b97700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa19c101de0 0x7fa19c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa1a1398700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa19c107c20 0x7fa19c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 -- 172.21.15.90:0/1702793060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa19c107c20 msgr2=0x7fa19c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:48.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa19c107c20 0x7fa19c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 -- 172.21.15.90:0/1702793060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa19c101de0 msgr2=0x7fa19c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa19c101de0 0x7fa19c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 -- 172.21.15.90:0/1702793060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c198ff0 con 0x7fa19c1027a0 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa19bfff700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c1027a0 0x7fa19c19ec10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c920 tx=0x7fa18800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa199ffb700 1 -- 172.21.15.90:0/1702793060 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188007b00 con 0x7fa19c1027a0 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa199ffb700 1 -- 172.21.15.90:0/1702793060 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa188004ce0 con 0x7fa19c1027a0 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19c1992e0 con 0x7fa19c1027a0 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa199ffb700 1 -- 172.21.15.90:0/1702793060 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188005650 con 0x7fa19c1027a0 2024-04-26T18:08:48.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.776+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19c1a55a0 con 0x7fa19c1027a0 2024-04-26T18:08:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.777+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa19c06a110 con 0x7fa19c1027a0 2024-04-26T18:08:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.778+0000 7fa199ffb700 1 -- 172.21.15.90:0/1702793060 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa188004750 con 0x7fa19c1027a0 2024-04-26T18:08:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.779+0000 7fa199ffb700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa18c0a3cc0 0x7fa18c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.779+0000 7fa1a0b97700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa18c0a3cc0 0x7fa18c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.779+0000 7fa199ffb700 1 -- 172.21.15.90:0/1702793060 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7879+0+0 (secure 0 0 0) 0x7fa1880cfe30 con 0x7fa19c1027a0 2024-04-26T18:08:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.779+0000 7fa1a0b97700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa18c0a3cc0 0x7fa18c0a6180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa190006010 tx=0x7fa19001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:48.784+0000 7fa199ffb700 1 -- 172.21.15.90:0/1702793060 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa188014030 con 0x7fa19c1027a0 2024-04-26T18:08:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:48 smithi119 ceph-mon[32752]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:08:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:48 smithi119 ceph-mon[32752]: pgmap v296: 97 pgs: 33 active+clean, 1 creating+activating, 8 creating+peering, 55 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:48 smithi167 ceph-mon[32751]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:08:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:48 smithi167 ceph-mon[32751]: pgmap v296: 97 pgs: 33 active+clean, 1 creating+activating, 8 creating+peering, 55 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:49.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.027+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa19c02e490 con 0x7fa19c1027a0 2024-04-26T18:08:49.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.357+0000 7fa199ffb700 1 -- 172.21.15.90:0/1702793060 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa18809c9c0 con 0x7fa19c1027a0 2024-04-26T18:08:49.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.359+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa18c0a3cc0 msgr2=0x7fa18c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.359+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa18c0a3cc0 0x7fa18c0a6180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa190006010 tx=0x7fa19001a040 comp rx=0 tx=0).stop 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.359+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c1027a0 msgr2=0x7fa19c19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.359+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c1027a0 0x7fa19c19ec10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c920 tx=0x7fa18800cce0 comp rx=0 tx=0).stop 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 shutdown_connections 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa18c0a3cc0 0x7fa18c0a6180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa19c107c20 0x7fa19c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa19c101de0 0x7fa19c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 --2- 172.21.15.90:0/1702793060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa19c1027a0 0x7fa19c19ec10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 >> 172.21.15.90:0/1702793060 conn(0x7fa19c077640 msgr2=0x7fa19c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 shutdown_connections 2024-04-26T18:08:49.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:49.360+0000 7fa1a2dfb700 1 -- 172.21.15.90:0/1702793060 wait complete. 2024-04-26T18:08:49.361 INFO:teuthology.orchestra.run.smithi090.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T18:08:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:49 smithi090 ceph-mon[29180]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:08:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:49 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1702793060' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:08:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:49 smithi090 ceph-mon[29180]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:08:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:49 smithi119 ceph-mon[32752]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:08:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:49 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1702793060' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:08:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:49 smithi119 ceph-mon[32752]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:08:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:49 smithi167 ceph-mon[32751]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:08:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:49 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1702793060' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:08:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:49 smithi167 ceph-mon[32751]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:08:49.990 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:08:50.229 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:50 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1702793060' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:50 smithi090 ceph-mon[29180]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:50 smithi090 ceph-mon[29180]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:50 smithi090 ceph-mon[29180]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:50 smithi090 ceph-mon[29180]: pgmap v299: 97 pgs: 73 active+clean, 24 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:50 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1702793060' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:50 smithi119 ceph-mon[32752]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:50 smithi119 ceph-mon[32752]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:50 smithi119 ceph-mon[32752]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:50 smithi119 ceph-mon[32752]: pgmap v299: 97 pgs: 73 active+clean, 24 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:50 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1702793060' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:08:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:50 smithi167 ceph-mon[32751]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:08:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:50 smithi167 ceph-mon[32751]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:08:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:50 smithi167 ceph-mon[32751]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:08:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:50 smithi167 ceph-mon[32751]: pgmap v299: 97 pgs: 73 active+clean, 24 unknown; 577 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:08:51.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:51 smithi090 ceph-mon[29180]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:08:51.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:51 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:08:51.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:51 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:08:51.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:51 smithi090 ceph-mon[29180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:51.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:51 smithi090 ceph-mon[29180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:51 smithi119 ceph-mon[32752]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:08:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:51 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:08:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:51 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:08:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:51 smithi119 ceph-mon[32752]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:51 smithi119 ceph-mon[32752]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:51 smithi167 ceph-mon[32751]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:08:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:51 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:08:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:51 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:08:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:51 smithi167 ceph-mon[32751]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:51 smithi167 ceph-mon[32751]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 -- 172.21.15.90:0/3483333126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 msgr2=0x7f5624102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 --2- 172.21.15.90:0/3483333126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f5624102790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5614009990 tx=0x7f5614009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 -- 172.21.15.90:0/3483333126 shutdown_connections 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 --2- 172.21.15.90:0/3483333126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5624102cd0 0x7f562410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 --2- 172.21.15.90:0/3483333126 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5624108380 0x7f5624108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 --2- 172.21.15.90:0/3483333126 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f5624102790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 -- 172.21.15.90:0/3483333126 >> 172.21.15.90:0/3483333126 conn(0x7f5624079290 msgr2=0x7f56240796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:51.858 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 -- 172.21.15.90:0/3483333126 shutdown_connections 2024-04-26T18:08:51.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 -- 172.21.15.90:0/3483333126 wait complete. 2024-04-26T18:08:51.859 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.856+0000 7f562c41f700 1 Processor -- start 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 -- start start 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f56241987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5624102cd0 0x7f5624198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5624108380 0x7f5624199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112520 con 0x7f5624102310 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56241123a0 con 0x7f5624102cd0 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56241126a0 con 0x7f5624108380 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f56241987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f56241987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57386/0 (socket says 172.21.15.90:57386) 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 -- 172.21.15.90:0/1491695877 learned_addr learned my addr 172.21.15.90:0/1491695877 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a9bc700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5624108380 0x7f5624199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f56299ba700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5624102cd0 0x7f5624198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:51.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 -- 172.21.15.90:0/1491695877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5624108380 msgr2=0x7f5624199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5624108380 0x7f5624199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 -- 172.21.15.90:0/1491695877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5624102cd0 msgr2=0x7f5624198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5624102cd0 0x7f5624198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 -- 172.21.15.90:0/1491695877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56241a8000 con 0x7f5624102310 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562a1bb700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f56241987f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f562000c9b0 tx=0x7f562000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f561b7fe700 1 -- 172.21.15.90:0/1491695877 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620007b00 con 0x7f5624102310 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56241a82f0 con 0x7f5624102310 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f561b7fe700 1 -- 172.21.15.90:0/1491695877 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5620007ca0 con 0x7f5624102310 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.857+0000 7f561b7fe700 1 -- 172.21.15.90:0/1491695877 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56200186f0 con 0x7f5624102310 2024-04-26T18:08:51.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.858+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241a8880 con 0x7f5624102310 2024-04-26T18:08:51.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.859+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f562406a110 con 0x7f5624102310 2024-04-26T18:08:51.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.860+0000 7f561b7fe700 1 -- 172.21.15.90:0/1491695877 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f562000f450 con 0x7f5624102310 2024-04-26T18:08:51.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.860+0000 7f561b7fe700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56100a3c40 0x7f56100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.860+0000 7f56299ba700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56100a3c40 0x7f56100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.860+0000 7f561b7fe700 1 -- 172.21.15.90:0/1491695877 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f56200d0310 con 0x7f5624102310 2024-04-26T18:08:51.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.861+0000 7f56299ba700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56100a3c40 0x7f56100a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5614006010 tx=0x7f561401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:51.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:51.865+0000 7f561b7fe700 1 -- 172.21.15.90:0/1491695877 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5620014030 con 0x7f5624102310 2024-04-26T18:08:52.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.065+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f562406bf30 con 0x7f5624102310 2024-04-26T18:08:52.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.067+0000 7f561b7fe700 1 -- 172.21.15.90:0/1491695877 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22217 (secure 0 0 0) 0x7f562009ce40 con 0x7f5624102310 2024-04-26T18:08:52.067 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:52.067 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:08:49.029325+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:08:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56100a3c40 msgr2=0x7f56100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56100a3c40 0x7f56100a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5614006010 tx=0x7f561401a040 comp rx=0 tx=0).stop 2024-04-26T18:08:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 msgr2=0x7f56241987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f56241987f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f562000c9b0 tx=0x7f562000cd70 comp rx=0 tx=0).stop 2024-04-26T18:08:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 shutdown_connections 2024-04-26T18:08:52.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f56100a3c40 0x7f56100a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5624102310 0x7f56241987f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5624108380 0x7f5624199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 --2- 172.21.15.90:0/1491695877 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5624102cd0 0x7f5624198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 >> 172.21.15.90:0/1491695877 conn(0x7f5624079290 msgr2=0x7f5624100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 shutdown_connections 2024-04-26T18:08:52.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:52.069+0000 7f562c41f700 1 -- 172.21.15.90:0/1491695877 wait complete. 2024-04-26T18:08:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:52 smithi119 ceph-mon[32752]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 929 B/s wr, 5 op/s 2024-04-26T18:08:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:52 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1491695877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:08:52.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:52 smithi090 ceph-mon[29180]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 929 B/s wr, 5 op/s 2024-04-26T18:08:52.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:52 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1491695877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:08:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:52 smithi167 ceph-mon[32751]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 929 B/s wr, 5 op/s 2024-04-26T18:08:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:52 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1491695877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:08:52.836 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:08:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:54.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 -- 172.21.15.90:0/52508189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581034d0 msgr2=0x7fc658103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/52508189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581034d0 0x7fc658103950 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc640009990 tx=0x7fc640009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 -- 172.21.15.90:0/52508189 shutdown_connections 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/52508189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc658103e90 0x7fc6581102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/52508189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6581094f0 0x7fc6581098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/52508189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581034d0 0x7fc658103950 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 -- 172.21.15.90:0/52508189 >> 172.21.15.90:0/52508189 conn(0x7fc65807a4b0 msgr2=0x7fc65807a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 -- 172.21.15.90:0/52508189 shutdown_connections 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.670+0000 7fc65d6aa700 1 -- 172.21.15.90:0/52508189 wait complete. 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 Processor -- start 2024-04-26T18:08:54.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 -- start start 2024-04-26T18:08:54.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6581034d0 0x7fc658075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:54.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc658103e90 0x7fc658075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:54.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581094f0 0x7fc6580761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:54.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc6581135f0 con 0x7fc6581094f0 2024-04-26T18:08:54.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc658113470 con 0x7fc658103e90 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc65d6aa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc658113770 con 0x7fc6581034d0 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc6577fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581094f0 0x7fc6580761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc6577fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581094f0 0x7fc6580761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57402/0 (socket says 172.21.15.90:57402) 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc6577fe700 1 -- 172.21.15.90:0/984597328 learned_addr learned my addr 172.21.15.90:0/984597328 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc6567fc700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc658103e90 0x7fc658075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.671+0000 7fc656ffd700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6581034d0 0x7fc658075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc6577fe700 1 -- 172.21.15.90:0/984597328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6581034d0 msgr2=0x7fc658075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc6577fe700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6581034d0 0x7fc658075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc6577fe700 1 -- 172.21.15.90:0/984597328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc658103e90 msgr2=0x7fc658075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc6577fe700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc658103e90 0x7fc658075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc6577fe700 1 -- 172.21.15.90:0/984597328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6581a7f90 con 0x7fc6581094f0 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc6577fe700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581094f0 0x7fc6580761a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d980 tx=0x7fc65000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:54.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc64ffff700 1 -- 172.21.15.90:0/984597328 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc650017070 con 0x7fc6581094f0 2024-04-26T18:08:54.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6581a8280 con 0x7fc6581094f0 2024-04-26T18:08:54.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc64ffff700 1 -- 172.21.15.90:0/984597328 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc650005020 con 0x7fc6581094f0 2024-04-26T18:08:54.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc64ffff700 1 -- 172.21.15.90:0/984597328 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc65000fa10 con 0x7fc6581094f0 2024-04-26T18:08:54.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.672+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6581a8790 con 0x7fc6581094f0 2024-04-26T18:08:54.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.673+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc65806a110 con 0x7fc6581094f0 2024-04-26T18:08:54.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.674+0000 7fc64ffff700 1 -- 172.21.15.90:0/984597328 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc650004a30 con 0x7fc6581094f0 2024-04-26T18:08:54.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.674+0000 7fc64ffff700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc6440a3c40 0x7fc6440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:54.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.675+0000 7fc656ffd700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc6440a3c40 0x7fc6440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:54.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.675+0000 7fc64ffff700 1 -- 172.21.15.90:0/984597328 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fc6500d00d0 con 0x7fc6581094f0 2024-04-26T18:08:54.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.675+0000 7fc656ffd700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc6440a3c40 0x7fc6440a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc648007fd0 tx=0x7fc6480079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:54.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.679+0000 7fc64ffff700 1 -- 172.21.15.90:0/984597328 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc650020020 con 0x7fc6581094f0 2024-04-26T18:08:54.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.881+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc65802e490 con 0x7fc6581094f0 2024-04-26T18:08:54.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.882+0000 7fc64ffff700 1 -- 172.21.15.90:0/984597328 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22217 (secure 0 0 0) 0x7fc65009cc00 con 0x7fc6581094f0 2024-04-26T18:08:54.882 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:08:54.882 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:08:49.029325+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:08:54.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc6440a3c40 msgr2=0x7fc6440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:54.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc6440a3c40 0x7fc6440a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc648007fd0 tx=0x7fc6480079e0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581094f0 msgr2=0x7fc6580761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:54.884 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581094f0 0x7fc6580761a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d980 tx=0x7fc65000dc90 comp rx=0 tx=0).stop 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 shutdown_connections 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc6440a3c40 0x7fc6440a6100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc6581094f0 0x7fc6580761a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6581034d0 0x7fc658075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 --2- 172.21.15.90:0/984597328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc658103e90 0x7fc658075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 >> 172.21.15.90:0/984597328 conn(0x7fc65807a4b0 msgr2=0x7fc658102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.884+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 shutdown_connections 2024-04-26T18:08:54.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:54.885+0000 7fc65d6aa700 1 -- 172.21.15.90:0/984597328 wait complete. 2024-04-26T18:08:55.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:54 smithi090 ceph-mon[29180]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T18:08:55.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:54 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/984597328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:08:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:54 smithi119 ceph-mon[32752]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T18:08:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:54 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/984597328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:08:55.304 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:08:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:08:54.920+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T18:08:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:54 smithi167 ceph-mon[32751]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T18:08:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:54 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/984597328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:08:55.662 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-26T18:08:55.899 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:57.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:56 smithi090 ceph-mon[29180]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-26T18:08:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:56 smithi119 ceph-mon[32752]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-26T18:08:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:56 smithi167 ceph-mon[32751]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 -- 172.21.15.90:0/339320760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c06b130 msgr2=0x7fd85c06b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 --2- 172.21.15.90:0/339320760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c06b130 0x7fd85c06b590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd850009990 tx=0x7fd850009ca0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 -- 172.21.15.90:0/339320760 shutdown_connections 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 --2- 172.21.15.90:0/339320760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd85c06bad0 0x7fd85c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 --2- 172.21.15.90:0/339320760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85c1094a0 0x7fd85c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 --2- 172.21.15.90:0/339320760 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c06b130 0x7fd85c06b590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 -- 172.21.15.90:0/339320760 >> 172.21.15.90:0/339320760 conn(0x7fd85c0792d0 msgr2=0x7fd85c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 -- 172.21.15.90:0/339320760 shutdown_connections 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.562+0000 7fd861f26700 1 -- 172.21.15.90:0/339320760 wait complete. 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 Processor -- start 2024-04-26T18:08:57.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 -- start start 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd85c06b130 0x7fd85c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85c06bad0 0x7fd85c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c1094a0 0x7fd85c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c113660 con 0x7fd85c1094a0 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1134e0 con 0x7fd85c06bad0 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd861f26700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd85c1137e0 con 0x7fd85c06b130 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c1094a0 0x7fd85c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c1094a0 0x7fd85c102e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57420/0 (socket says 172.21.15.90:57420) 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 -- 172.21.15.90:0/782530927 learned_addr learned my addr 172.21.15.90:0/782530927 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85affd700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85c06bad0 0x7fd85c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85b7fe700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd85c06b130 0x7fd85c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 -- 172.21.15.90:0/782530927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd85c06b130 msgr2=0x7fd85c1023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:57.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd85c06b130 0x7fd85c1023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 -- 172.21.15.90:0/782530927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85c06bad0 msgr2=0x7fd85c102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85c06bad0 0x7fd85c102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85bfff700 1 -- 172.21.15.90:0/782530927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd85c1a8080 con 0x7fd85c1094a0 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.563+0000 7fd85b7fe700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd85c06b130 0x7fd85c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.564+0000 7fd85bfff700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c1094a0 0x7fd85c102e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd854007d70 tx=0x7fd85400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.564+0000 7fd858ff9700 1 -- 172.21.15.90:0/782530927 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd854016020 con 0x7fd85c1094a0 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.564+0000 7fd858ff9700 1 -- 172.21.15.90:0/782530927 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd85400ad40 con 0x7fd85c1094a0 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.564+0000 7fd858ff9700 1 -- 172.21.15.90:0/782530927 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8540115e0 con 0x7fd85c1094a0 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.564+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd85c1a8370 con 0x7fd85c1094a0 2024-04-26T18:08:57.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.564+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd85c1a88b0 con 0x7fd85c1094a0 2024-04-26T18:08:57.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.565+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd85c10ba10 con 0x7fd85c1094a0 2024-04-26T18:08:57.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.566+0000 7fd858ff9700 1 -- 172.21.15.90:0/782530927 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd854019020 con 0x7fd85c1094a0 2024-04-26T18:08:57.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.566+0000 7fd858ff9700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd8440a3c40 0x7fd8440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:08:57.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.566+0000 7fd85b7fe700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd8440a3c40 0x7fd8440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:08:57.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.566+0000 7fd858ff9700 1 -- 172.21.15.90:0/782530927 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fd8540d02a0 con 0x7fd85c1094a0 2024-04-26T18:08:57.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.567+0000 7fd85b7fe700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd8440a3c40 0x7fd8440a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007900 tx=0x7fd84c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:08:57.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.571+0000 7fd858ff9700 1 -- 172.21.15.90:0/782530927 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd854020020 con 0x7fd85c1094a0 2024-04-26T18:08:57.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.817+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd85c10b3f0 con 0x7fd85c1094a0 2024-04-26T18:08:57.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.912+0000 7fd858ff9700 1 -- 172.21.15.90:0/782530927 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fd85409cdd0 con 0x7fd85c1094a0 2024-04-26T18:08:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd8440a3c40 msgr2=0x7fd8440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:57.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd8440a3c40 0x7fd8440a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007900 tx=0x7fd84c008040 comp rx=0 tx=0).stop 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c1094a0 msgr2=0x7fd85c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c1094a0 0x7fd85c102e70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd854007d70 tx=0x7fd85400a340 comp rx=0 tx=0).stop 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 shutdown_connections 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd8440a3c40 0x7fd8440a6100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd85c1094a0 0x7fd85c102e70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd85c06b130 0x7fd85c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 --2- 172.21.15.90:0/782530927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd85c06bad0 0x7fd85c102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.914+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 >> 172.21.15.90:0/782530927 conn(0x7fd85c0792d0 msgr2=0x7fd85c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.915+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 shutdown_connections 2024-04-26T18:08:57.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:08:57.915+0000 7fd861f26700 1 -- 172.21.15.90:0/782530927 wait complete. 2024-04-26T18:08:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:57 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/782530927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:08:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:57 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/782530927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:08:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:57 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/782530927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:08:58.640 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-04-26T18:08:58.641 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-26T18:08:58.876 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:08:59.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:58 smithi090 ceph-mon[29180]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T18:08:59.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:58 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/782530927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:08:59.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:08:58 smithi090 ceph-mon[29180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:58 smithi119 ceph-mon[32752]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T18:08:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:58 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/782530927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:08:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:08:58 smithi119 ceph-mon[32752]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:08:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:58 smithi167 ceph-mon[32751]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T18:08:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:58 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/782530927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:08:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:08:58 smithi167 ceph-mon[32751]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:09:00.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/1263948121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18102320 msgr2=0x7f8b181027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/1263948121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18102320 0x7f8b181027a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009990 tx=0x7f8b0c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/1263948121 shutdown_connections 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/1263948121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b18102ce0 0x7f8b1810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/1263948121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b18108390 0x7f8b18108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/1263948121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18102320 0x7f8b181027a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/1263948121 >> 172.21.15.90:0/1263948121 conn(0x7f8b180792d0 msgr2=0x7f8b180796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/1263948121 shutdown_connections 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/1263948121 wait complete. 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.442+0000 7f8b1d7ae700 1 Processor -- start 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 -- start start 2024-04-26T18:09:00.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b18102320 0x7f8b18104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b18102ce0 0x7f8b18104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18108390 0x7f8b18104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b18112530 con 0x7f8b18108390 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8b181123b0 con 0x7f8b18102320 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b181126b0 con 0x7f8b18102ce0 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18108390 0x7f8b18104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18108390 0x7f8b18104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57428/0 (socket says 172.21.15.90:57428) 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 -- 172.21.15.90:0/708239722 learned_addr learned my addr 172.21.15.90:0/708239722 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b16ffd700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b18102320 0x7f8b18104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b167fc700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b18102ce0 0x7f8b18104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 -- 172.21.15.90:0/708239722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b18102ce0 msgr2=0x7f8b18104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:00.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b18102ce0 0x7f8b18104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 -- 172.21.15.90:0/708239722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b18102320 msgr2=0x7f8b18104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b18102320 0x7f8b18104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 -- 172.21.15.90:0/708239722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b181a8040 con 0x7f8b18108390 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b167fc700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b18102ce0 0x7f8b18104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b177fe700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18108390 0x7f8b18104a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b10007d70 tx=0x7f8b1000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8afffff700 1 -- 172.21.15.90:0/708239722 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10016020 con 0x7f8b18108390 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b181a8330 con 0x7f8b18108390 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.443+0000 7f8afffff700 1 -- 172.21.15.90:0/708239722 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b1000ad40 con 0x7f8b18108390 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.444+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b181a8870 con 0x7f8b18108390 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.444+0000 7f8afffff700 1 -- 172.21.15.90:0/708239722 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b100115e0 con 0x7f8b18108390 2024-04-26T18:09:00.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.444+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b18103a40 con 0x7f8b18108390 2024-04-26T18:09:00.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.446+0000 7f8afffff700 1 -- 172.21.15.90:0/708239722 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8b10019020 con 0x7f8b18108390 2024-04-26T18:09:00.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.447+0000 7f8afffff700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b000a3c40 0x7f8b000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:00.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.447+0000 7f8b16ffd700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b000a3c40 0x7f8b000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:00.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.447+0000 7f8afffff700 1 -- 172.21.15.90:0/708239722 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f8b100d0620 con 0x7f8b18108390 2024-04-26T18:09:00.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.447+0000 7f8b16ffd700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b000a3c40 0x7f8b000a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009a10 tx=0x7f8b08009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:00.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.450+0000 7f8afffff700 1 -- 172.21.15.90:0/708239722 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b10020020 con 0x7f8b18108390 2024-04-26T18:09:00.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.694+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f8b181a8b90 con 0x7f8b18108390 2024-04-26T18:09:00.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.916+0000 7f8afffff700 1 -- 172.21.15.90:0/708239722 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f8b1009d150 con 0x7f8b18108390 2024-04-26T18:09:00.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b000a3c40 msgr2=0x7f8b000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:00.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b000a3c40 0x7f8b000a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b08009a10 tx=0x7f8b08009190 comp rx=0 tx=0).stop 2024-04-26T18:09:00.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18108390 msgr2=0x7f8b18104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:00.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18108390 0x7f8b18104a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8b10007d70 tx=0x7f8b1000a340 comp rx=0 tx=0).stop 2024-04-26T18:09:00.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 shutdown_connections 2024-04-26T18:09:00.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b000a3c40 0x7f8b000a6100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b18108390 0x7f8b18104a80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b18102320 0x7f8b18104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 --2- 172.21.15.90:0/708239722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b18102ce0 0x7f8b18104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:00.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 >> 172.21.15.90:0/708239722 conn(0x7f8b180792d0 msgr2=0x7f8b181015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:00.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 shutdown_connections 2024-04-26T18:09:00.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:00.919+0000 7f8b1d7ae700 1 -- 172.21.15.90:0/708239722 wait complete. 2024-04-26T18:09:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:00 smithi090 ceph-mon[29180]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T18:09:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:00 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/708239722' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T18:09:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:00 smithi119 ceph-mon[32752]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T18:09:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:00 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/708239722' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T18:09:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:00 smithi167 ceph-mon[32751]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-26T18:09:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:00 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/708239722' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-26T18:09:01.587 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-26T18:09:01.826 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:02.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:01 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:09:02.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:01 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/708239722' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T18:09:02.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:01 smithi090 ceph-mon[29180]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:09:02.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:01 smithi090 ceph-mon[29180]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:09:02.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:01 smithi090 ceph-mon[29180]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T18:09:02.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:01 smithi090 ceph-mon[29180]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:09:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:01 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:09:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:01 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/708239722' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T18:09:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:01 smithi119 ceph-mon[32752]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:09:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:01 smithi119 ceph-mon[32752]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:09:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:01 smithi119 ceph-mon[32752]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T18:09:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:01 smithi119 ceph-mon[32752]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:09:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:01 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:09:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:01 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/708239722' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-26T18:09:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:01 smithi167 ceph-mon[32751]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:09:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:01 smithi167 ceph-mon[32751]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:09:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:01 smithi167 ceph-mon[32751]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T18:09:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:01 smithi167 ceph-mon[32751]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:09:03.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:09:03.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] up:active 2024-04-26T18:09:03.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:09:03.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:09:03.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:09:03.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T18:09:03.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T18:09:03.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:02 smithi090 ceph-mon[29180]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:09:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:09:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] up:active 2024-04-26T18:09:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:09:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:09:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:09:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T18:09:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T18:09:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:02 smithi119 ceph-mon[32752]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:09:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:09:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] up:active 2024-04-26T18:09:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:09:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:09:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:09:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T18:09:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2024-04-26T18:09:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:02 smithi167 ceph-mon[32751]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:09:03.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 -- 172.21.15.90:0/2659523027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 msgr2=0x7f9504110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:03.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 --2- 172.21.15.90:0/2659523027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f9504110210 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00b0a0 tx=0x7f94fc00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 -- 172.21.15.90:0/2659523027 shutdown_connections 2024-04-26T18:09:03.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 --2- 172.21.15.90:0/2659523027 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f9504110210 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 --2- 172.21.15.90:0/2659523027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9504108500 0x7f95041088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 --2- 172.21.15.90:0/2659523027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9504101de0 0x7f9504102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 -- 172.21.15.90:0/2659523027 >> 172.21.15.90:0/2659523027 conn(0x7f9504077640 msgr2=0x7f9504077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 -- 172.21.15.90:0/2659523027 shutdown_connections 2024-04-26T18:09:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.447+0000 7f950914f700 1 -- 172.21.15.90:0/2659523027 wait complete. 2024-04-26T18:09:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 Processor -- start 2024-04-26T18:09:03.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 -- start start 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9504101de0 0x7f950419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f950419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9504108500 0x7f95041998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9504113690 con 0x7f95041027a0 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9504113510 con 0x7f9504108500 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950914f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9504113810 con 0x7f9504101de0 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f94fbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f950419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f94fbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f950419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37972/0 (socket says 172.21.15.90:37972) 2024-04-26T18:09:03.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9504108500 0x7f95041998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f94fbfff700 1 -- 172.21.15.90:0/2411560878 learned_addr learned my addr 172.21.15.90:0/2411560878 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f9502d9d700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9504101de0 0x7f950419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950359e700 1 -- 172.21.15.90:0/2411560878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9504101de0 msgr2=0x7f950419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950359e700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9504101de0 0x7f950419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950359e700 1 -- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 msgr2=0x7f950419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950359e700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f950419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950359e700 1 -- 172.21.15.90:0/2411560878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f950419a050 con 0x7f9504108500 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f9502d9d700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9504101de0 0x7f950419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f94fbfff700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f950419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.448+0000 7f950359e700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9504108500 0x7f95041998f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00d010 tx=0x7f94fc004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.449+0000 7f9500d99700 1 -- 172.21.15.90:0/2411560878 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc021070 con 0x7f9504108500 2024-04-26T18:09:03.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.449+0000 7f9500d99700 1 -- 172.21.15.90:0/2411560878 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc007e30 con 0x7f9504108500 2024-04-26T18:09:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.449+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f950419a2e0 con 0x7f9504108500 2024-04-26T18:09:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.449+0000 7f9500d99700 1 -- 172.21.15.90:0/2411560878 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc0046a0 con 0x7f9504108500 2024-04-26T18:09:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.449+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95041a6600 con 0x7f9504108500 2024-04-26T18:09:03.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.449+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950406a110 con 0x7f9504108500 2024-04-26T18:09:03.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.451+0000 7f9500d99700 1 -- 172.21.15.90:0/2411560878 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f94fc003d00 con 0x7f9504108500 2024-04-26T18:09:03.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.451+0000 7f9500d99700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94f00a3cc0 0x7f94f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:03.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.451+0000 7f9502d9d700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94f00a3cc0 0x7f94f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:03.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.451+0000 7f9500d99700 1 -- 172.21.15.90:0/2411560878 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f94fc00f020 con 0x7f9504108500 2024-04-26T18:09:03.453 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.452+0000 7f9502d9d700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94f00a3cc0 0x7f94f00a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f950419aa30 tx=0x7f94f4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:03.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.455+0000 7f9500d99700 1 -- 172.21.15.90:0/2411560878 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94fc02a020 con 0x7f9504108500 2024-04-26T18:09:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.703+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f950402e490 con 0x7f9504108500 2024-04-26T18:09:03.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.941+0000 7f9500d99700 1 -- 172.21.15.90:0/2411560878 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f94fc09e3c0 con 0x7f9504108500 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94f00a3cc0 msgr2=0x7f94f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94f00a3cc0 0x7f94f00a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f950419aa30 tx=0x7f94f4009260 comp rx=0 tx=0).stop 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9504108500 msgr2=0x7f95041998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9504108500 0x7f95041998f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94fc00d010 tx=0x7f94fc004d00 comp rx=0 tx=0).stop 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 shutdown_connections 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94f00a3cc0 0x7f94f00a6180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9504108500 0x7f95041998f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9504101de0 0x7f950419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 --2- 172.21.15.90:0/2411560878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95041027a0 0x7f950419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:03.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 >> 172.21.15.90:0/2411560878 conn(0x7f9504077640 msgr2=0x7f9504100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:03.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 shutdown_connections 2024-04-26T18:09:03.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:03.944+0000 7f950914f700 1 -- 172.21.15.90:0/2411560878 wait complete. 2024-04-26T18:09:03.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] up:active 2024-04-26T18:09:03.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:03 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T18:09:03.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:03 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2411560878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:09:03.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:03 smithi090 ceph-mon[29180]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:09:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] up:active 2024-04-26T18:09:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:03 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T18:09:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:03 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2411560878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:09:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:03 smithi119 ceph-mon[32752]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:09:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] up:active 2024-04-26T18:09:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:03 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T18:09:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:03 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2411560878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:09:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:03 smithi167 ceph-mon[32751]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:09:04.645 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-26T18:09:04.889 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:05.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:04 smithi090 ceph-mon[29180]: pgmap v306: 97 pgs: 97 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-04-26T18:09:05.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:04 smithi090 ceph-mon[29180]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:09:05.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:04 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T18:09:05.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:04 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:09:05.303 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:09:04 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:09:04.922+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:09:05.304 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:09:04 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:09:04.922+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:09:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:04 smithi119 ceph-mon[32752]: pgmap v306: 97 pgs: 97 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-04-26T18:09:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:04 smithi119 ceph-mon[32752]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:09:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:04 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T18:09:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:04 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:09:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:04 smithi167 ceph-mon[32751]: pgmap v306: 97 pgs: 97 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 3 op/s 2024-04-26T18:09:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:04 smithi167 ceph-mon[32751]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:09:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:04 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2024-04-26T18:09:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:04 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 -- 172.21.15.90:0/407352020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c103000 msgr2=0x7fea7c10f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 --2- 172.21.15.90:0/407352020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c103000 0x7fea7c10f180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fea7400b0d0 tx=0x7fea7400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 -- 172.21.15.90:0/407352020 shutdown_connections 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 --2- 172.21.15.90:0/407352020 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c103000 0x7fea7c10f180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 --2- 172.21.15.90:0/407352020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c108530 0x7fea7c108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 --2- 172.21.15.90:0/407352020 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c102640 0x7fea7c102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 -- 172.21.15.90:0/407352020 >> 172.21.15.90:0/407352020 conn(0x7fea7c079080 msgr2=0x7fea7c079490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 -- 172.21.15.90:0/407352020 shutdown_connections 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 -- 172.21.15.90:0/407352020 wait complete. 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 Processor -- start 2024-04-26T18:09:06.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.499+0000 7fea83fce700 1 -- start start 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea83fce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c102640 0x7fea7c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea83fce700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c103000 0x7fea7c19eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea83fce700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c108530 0x7fea7c198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea83fce700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c112580 con 0x7fea7c102640 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea83fce700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c112400 con 0x7fea7c108530 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea83fce700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fea7c112700 con 0x7fea7c103000 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c102640 0x7fea7c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c102640 0x7fea7c19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37984/0 (socket says 172.21.15.90:37984) 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 -- 172.21.15.90:0/1161511654 learned_addr learned my addr 172.21.15.90:0/1161511654 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81569700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c103000 0x7fea7c19eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea8256b700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c108530 0x7fea7c198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:06.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 -- 172.21.15.90:0/1161511654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c103000 msgr2=0x7fea7c19eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c103000 0x7fea7c19eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 -- 172.21.15.90:0/1161511654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c108530 msgr2=0x7fea7c198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c108530 0x7fea7c198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 -- 172.21.15.90:0/1161511654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea7c199000 con 0x7fea7c102640 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81569700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c103000 0x7fea7c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea8256b700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c108530 0x7fea7c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea81d6a700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c102640 0x7fea7c19e650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00d950 tx=0x7fea6c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea72ffd700 1 -- 172.21.15.90:0/1161511654 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c016070 con 0x7fea7c102640 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea72ffd700 1 -- 172.21.15.90:0/1161511654 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea6c010460 con 0x7fea7c102640 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.500+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea7c1992f0 con 0x7fea7c102640 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.501+0000 7fea72ffd700 1 -- 172.21.15.90:0/1161511654 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea6c00f620 con 0x7fea7c102640 2024-04-26T18:09:06.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.501+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea7c1a54b0 con 0x7fea7c102640 2024-04-26T18:09:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.502+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea7c103ab0 con 0x7fea7c102640 2024-04-26T18:09:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.502+0000 7fea72ffd700 1 -- 172.21.15.90:0/1161511654 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fea6c019070 con 0x7fea7c102640 2024-04-26T18:09:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.503+0000 7fea72ffd700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fea680a3c40 0x7fea680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.503+0000 7fea81569700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fea680a3c40 0x7fea680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.503+0000 7fea72ffd700 1 -- 172.21.15.90:0/1161511654 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fea6c0cff20 con 0x7fea7c102640 2024-04-26T18:09:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.504+0000 7fea81569700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fea680a3c40 0x7fea680a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea7c107460 tx=0x7fea7800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:06.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.508+0000 7fea72ffd700 1 -- 172.21.15.90:0/1161511654 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea6c09ca50 con 0x7fea7c102640 2024-04-26T18:09:06.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.758+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fea7c02e490 con 0x7fea7c102640 2024-04-26T18:09:06.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.942+0000 7fea72ffd700 1 -- 172.21.15.90:0/1161511654 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fea6c09c350 con 0x7fea7c102640 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fea680a3c40 msgr2=0x7fea680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fea680a3c40 0x7fea680a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea7c107460 tx=0x7fea7800b2c0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c102640 msgr2=0x7fea7c19e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c102640 0x7fea7c19e650 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fea6c00d950 tx=0x7fea6c00dd10 comp rx=0 tx=0).stop 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 shutdown_connections 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fea680a3c40 0x7fea680a6100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fea7c102640 0x7fea7c19e650 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c108530 0x7fea7c198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 --2- 172.21.15.90:0/1161511654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fea7c103000 0x7fea7c19eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 >> 172.21.15.90:0/1161511654 conn(0x7fea7c079080 msgr2=0x7fea7c101320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 shutdown_connections 2024-04-26T18:09:06.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:06.945+0000 7fea83fce700 1 -- 172.21.15.90:0/1161511654 wait complete. 2024-04-26T18:09:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:06 smithi090 ceph-mon[29180]: pgmap v307: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T18:09:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:06 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1161511654' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:09:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:06 smithi119 ceph-mon[32752]: pgmap v307: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T18:09:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:06 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1161511654' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:09:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:06 smithi167 ceph-mon[32751]: pgmap v307: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-26T18:09:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:06 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1161511654' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:09:07.770 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-04-26T18:09:07.770 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-26T18:09:08.017 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:07 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1161511654' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:09:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:09:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:07 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1161511654' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:09:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:09:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:07 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1161511654' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:09:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:09:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:08 smithi119 ceph-mon[32752]: pgmap v308: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T18:09:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:08 smithi167 ceph-mon[32751]: pgmap v308: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T18:09:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:08 smithi090 ceph-mon[29180]: pgmap v308: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 -- 172.21.15.90:0/2220875728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 msgr2=0x7f0ac01098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 --2- 172.21.15.90:0/2220875728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac01098a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0abc009990 tx=0x7f0abc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 -- 172.21.15.90:0/2220875728 shutdown_connections 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 --2- 172.21.15.90:0/2220875728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac01098a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 --2- 172.21.15.90:0/2220875728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac0103e30 0x7f0ac0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 --2- 172.21.15.90:0/2220875728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac0103470 0x7f0ac01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 -- 172.21.15.90:0/2220875728 >> 172.21.15.90:0/2220875728 conn(0x7f0ac007a580 msgr2=0x7f0ac007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 -- 172.21.15.90:0/2220875728 shutdown_connections 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 -- 172.21.15.90:0/2220875728 wait complete. 2024-04-26T18:09:09.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 Processor -- start 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.904+0000 7f0ac8311700 1 -- start start 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac8311700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac0103470 0x7f0ac019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac8311700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac0103e30 0x7f0ac019fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac8311700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac0199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac8311700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac01136c0 con 0x7f0ac01094c0 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac8311700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0113540 con 0x7f0ac0103e30 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac8311700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac0113840 con 0x7f0ac0103470 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac0199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac58ac700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac0103e30 0x7f0ac019fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac0199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38002/0 (socket says 172.21.15.90:38002) 2024-04-26T18:09:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 -- 172.21.15.90:0/4269514658 learned_addr learned my addr 172.21.15.90:0/4269514658 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac60ad700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac0103470 0x7f0ac019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 -- 172.21.15.90:0/4269514658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac0103470 msgr2=0x7f0ac019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac0103470 0x7f0ac019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 -- 172.21.15.90:0/4269514658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac0103e30 msgr2=0x7f0ac019fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac0103e30 0x7f0ac019fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 -- 172.21.15.90:0/4269514658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac019a200 con 0x7f0ac01094c0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.905+0000 7f0ac68ae700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac0199aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b970 tx=0x7f0ab800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.906+0000 7f0ab77fe700 1 -- 172.21.15.90:0/4269514658 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800d860 con 0x7f0ac01094c0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.906+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac019a4f0 con 0x7f0ac01094c0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.906+0000 7f0ab77fe700 1 -- 172.21.15.90:0/4269514658 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab800e430 con 0x7f0ac01094c0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.906+0000 7f0ab77fe700 1 -- 172.21.15.90:0/4269514658 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8017660 con 0x7f0ac01094c0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.906+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac01a67b0 con 0x7f0ac01094c0 2024-04-26T18:09:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.906+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac006a110 con 0x7f0ac01094c0 2024-04-26T18:09:09.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.908+0000 7f0ab77fe700 1 -- 172.21.15.90:0/4269514658 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0ab800da00 con 0x7f0ac01094c0 2024-04-26T18:09:09.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.908+0000 7f0ab77fe700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0aac0a3c40 0x7f0aac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:09.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.908+0000 7f0ac60ad700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0aac0a3c40 0x7f0aac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.908+0000 7f0ab77fe700 1 -- 172.21.15.90:0/4269514658 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f0ab80d0230 con 0x7f0ac01094c0 2024-04-26T18:09:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.909+0000 7f0ac60ad700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0aac0a3c40 0x7f0aac0a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ac019abe0 tx=0x7f0abc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:09.912+0000 7f0ab77fe700 1 -- 172.21.15.90:0/4269514658 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0ab8013030 con 0x7f0ac01094c0 2024-04-26T18:09:10.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.145+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0ac0063860 con 0x7f0aac0a3c40 2024-04-26T18:09:10.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.174+0000 7f0ab77fe700 1 -- 172.21.15.90:0/4269514658 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0ac0063860 con 0x7f0aac0a3c40 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0aac0a3c40 msgr2=0x7f0aac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0aac0a3c40 0x7f0aac0a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0ac019abe0 tx=0x7f0abc005960 comp rx=0 tx=0).stop 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 msgr2=0x7f0ac0199aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac0199aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800b970 tx=0x7f0ab800bc80 comp rx=0 tx=0).stop 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 shutdown_connections 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f0aac0a3c40 0x7f0aac0a6100 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ac01094c0 0x7f0ac0199aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:10.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0ac0103470 0x7f0ac019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 --2- 172.21.15.90:0/4269514658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ac0103e30 0x7f0ac019fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 >> 172.21.15.90:0/4269514658 conn(0x7f0ac007a580 msgr2=0x7f0ac01022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 shutdown_connections 2024-04-26T18:09:10.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:10.176+0000 7f0ac8311700 1 -- 172.21.15.90:0/4269514658 wait complete. 2024-04-26T18:09:10.486 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:09:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:09:10.150+0000 7f0b63346700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:09:10.486 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:09:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:09:10.150+0000 7f0b63346700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:09:10.486 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:09:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:09:10.150+0000 7f0b63346700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:09:10.486 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:09:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:09:10.150+0000 7f0b63346700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:09:10.486 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:09:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:09:10.150+0000 7f0b63346700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:09:10.924 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T18:09:11.164 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:11.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:10 smithi090 ceph-mon[29180]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:09:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:10 smithi119 ceph-mon[32752]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:09:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:10 smithi167 ceph-mon[32751]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-26T18:09:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:11 smithi119 ceph-mon[32752]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:11 smithi167 ceph-mon[32751]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:12.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:11 smithi090 ceph-mon[29180]: from='client.14661 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:12.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.869+0000 7f04ab5be700 1 -- 172.21.15.90:0/1804946632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a4109490 msgr2=0x7f04a4109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.869+0000 7f04ab5be700 1 --2- 172.21.15.90:0/1804946632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a4109490 0x7f04a4109870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0494009990 tx=0x7f0494009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.869+0000 7f04ab5be700 1 -- 172.21.15.90:0/1804946632 shutdown_connections 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.869+0000 7f04ab5be700 1 --2- 172.21.15.90:0/1804946632 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a4109490 0x7f04a4109870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.869+0000 7f04ab5be700 1 --2- 172.21.15.90:0/1804946632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406c360 0x7f04a41102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.869+0000 7f04ab5be700 1 --2- 172.21.15.90:0/1804946632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a406b9c0 0x7f04a406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.869+0000 7f04ab5be700 1 -- 172.21.15.90:0/1804946632 >> 172.21.15.90:0/1804946632 conn(0x7f04a4079290 msgr2=0x7f04a40796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.870+0000 7f04ab5be700 1 -- 172.21.15.90:0/1804946632 shutdown_connections 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.870+0000 7f04ab5be700 1 -- 172.21.15.90:0/1804946632 wait complete. 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.870+0000 7f04ab5be700 1 Processor -- start 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.870+0000 7f04ab5be700 1 -- start start 2024-04-26T18:09:12.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.870+0000 7f04ab5be700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406b9c0 0x7f04a41023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.870+0000 7f04ab5be700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a406c360 0x7f04a4102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04ab5be700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a4109490 0x7f04a4102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04ab5be700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f04a41135a0 con 0x7f04a406c360 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04ab5be700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f04a4113420 con 0x7f04a406b9c0 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04ab5be700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04a4113720 con 0x7f04a4109490 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a935a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406b9c0 0x7f04a41023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a9b5b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a4109490 0x7f04a4102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a8b59700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a406c360 0x7f04a4102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a9b5b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a4109490 0x7f04a4102e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:59044/0 (socket says 172.21.15.90:59044) 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a935a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406b9c0 0x7f04a41023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:52334/0 (socket says 172.21.15.90:52334) 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a9b5b700 1 -- 172.21.15.90:0/3948633086 learned_addr learned my addr 172.21.15.90:0/3948633086 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:12.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a8b59700 1 -- 172.21.15.90:0/3948633086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a4109490 msgr2=0x7f04a4102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a8b59700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a4109490 0x7f04a4102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a8b59700 1 -- 172.21.15.90:0/3948633086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406b9c0 msgr2=0x7f04a41023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a8b59700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406b9c0 0x7f04a41023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a8b59700 1 -- 172.21.15.90:0/3948633086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04a41a8020 con 0x7f04a406c360 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a9b5b700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a4109490 0x7f04a4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a935a700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406b9c0 0x7f04a41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f04a8b59700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a406c360 0x7f04a4102920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f04a000c970 tx=0x7f04a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.871+0000 7f049a7fc700 1 -- 172.21.15.90:0/3948633086 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a000e880 con 0x7f04a406c360 2024-04-26T18:09:12.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.872+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04a41a8310 con 0x7f04a406c360 2024-04-26T18:09:12.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.872+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04a41a8850 con 0x7f04a406c360 2024-04-26T18:09:12.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.872+0000 7f049a7fc700 1 -- 172.21.15.90:0/3948633086 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04a000f450 con 0x7f04a406c360 2024-04-26T18:09:12.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.872+0000 7f049a7fc700 1 -- 172.21.15.90:0/3948633086 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04a0018660 con 0x7f04a406c360 2024-04-26T18:09:12.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.873+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0488005360 con 0x7f04a406c360 2024-04-26T18:09:12.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.874+0000 7f049a7fc700 1 -- 172.21.15.90:0/3948633086 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f04a000ea20 con 0x7f04a406c360 2024-04-26T18:09:12.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.874+0000 7f049a7fc700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04900a3cd0 0x7f04900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:12.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.875+0000 7f04a935a700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04900a3cd0 0x7f04900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:12.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.875+0000 7f049a7fc700 1 -- 172.21.15.90:0/3948633086 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f04a00d02f0 con 0x7f04a406c360 2024-04-26T18:09:12.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.875+0000 7f04a935a700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04900a3cd0 0x7f04900a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f049400b5c0 tx=0x7f0494005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:12.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:12.878+0000 7f049a7fc700 1 -- 172.21.15.90:0/3948633086 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04a0014030 con 0x7f04a406c360 2024-04-26T18:09:13.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.119+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0488000bf0 con 0x7f04900a3cd0 2024-04-26T18:09:13.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.144+0000 7f049a7fc700 1 -- 172.21.15.90:0/3948633086 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0488000bf0 con 0x7f04900a3cd0 2024-04-26T18:09:13.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04900a3cd0 msgr2=0x7f04900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04900a3cd0 0x7f04900a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f049400b5c0 tx=0x7f0494005960 comp rx=0 tx=0).stop 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a406c360 msgr2=0x7f04a4102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a406c360 0x7f04a4102920 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f04a000c970 tx=0x7f04a000cc80 comp rx=0 tx=0).stop 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 shutdown_connections 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04900a3cd0 0x7f04900a6190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04a4109490 0x7f04a4102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04a406b9c0 0x7f04a41023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 --2- 172.21.15.90:0/3948633086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f04a406c360 0x7f04a4102920 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.146+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 >> 172.21.15.90:0/3948633086 conn(0x7f04a4079290 msgr2=0x7f04a4100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.147+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 shutdown_connections 2024-04-26T18:09:13.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:13.147+0000 7f04ab5be700 1 -- 172.21.15.90:0/3948633086 wait complete. 2024-04-26T18:09:13.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:12 smithi167 ceph-mon[32751]: pgmap v310: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-26T18:09:13.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:12 smithi167 ceph-mon[32751]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T18:09:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:12 smithi119 ceph-mon[32752]: pgmap v310: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-26T18:09:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:12 smithi119 ceph-mon[32752]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T18:09:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:12 smithi090 ceph-mon[29180]: pgmap v310: 97 pgs: 97 active+clean; 588 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-26T18:09:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:12 smithi090 ceph-mon[29180]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T18:09:13.761 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T18:09:14.003 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:14 smithi090 ceph-mon[29180]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:14 smithi090 ceph-mon[29180]: pgmap v311: 97 pgs: 97 active+clean; 589 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-04-26T18:09:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:14 smithi119 ceph-mon[32752]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:14 smithi119 ceph-mon[32752]: pgmap v311: 97 pgs: 97 active+clean; 589 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-04-26T18:09:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:14 smithi167 ceph-mon[32751]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:14 smithi167 ceph-mon[32751]: pgmap v311: 97 pgs: 97 active+clean; 589 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-04-26T18:09:15.651 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 -- 172.21.15.90:0/1650909730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f065406bad0 msgr2=0x7f06541102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 --2- 172.21.15.90:0/1650909730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f065406bad0 0x7f06541102e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b0a0 tx=0x7f064c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 -- 172.21.15.90:0/1650909730 shutdown_connections 2024-04-26T18:09:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 --2- 172.21.15.90:0/1650909730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f065406bad0 0x7f06541102e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 --2- 172.21.15.90:0/1650909730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06541094a0 0x7f0654109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 --2- 172.21.15.90:0/1650909730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065406b130 0x7f065406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 -- 172.21.15.90:0/1650909730 >> 172.21.15.90:0/1650909730 conn(0x7f06540792d0 msgr2=0x7f06540796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:15.652 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 -- 172.21.15.90:0/1650909730 shutdown_connections 2024-04-26T18:09:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.651+0000 7f0659f18700 1 -- 172.21.15.90:0/1650909730 wait complete. 2024-04-26T18:09:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 Processor -- start 2024-04-26T18:09:15.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 -- start start 2024-04-26T18:09:15.654 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065406b130 0x7f065419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f065406bad0 0x7f065419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06541094a0 0x7f0654199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0654113650 con 0x7f06541094a0 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f06541134d0 con 0x7f065406bad0 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0659f18700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06541137d0 con 0x7f065406b130 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0653fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06541094a0 0x7f0654199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0653fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06541094a0 0x7f0654199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41310/0 (socket says 172.21.15.90:41310) 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0653fff700 1 -- 172.21.15.90:0/336570337 learned_addr learned my addr 172.21.15.90:0/336570337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:15.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f06537fe700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065406b130 0x7f065419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.652+0000 7f0652ffd700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f065406bad0 0x7f065419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0653fff700 1 -- 172.21.15.90:0/336570337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065406b130 msgr2=0x7f065419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0653fff700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065406b130 0x7f065419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0653fff700 1 -- 172.21.15.90:0/336570337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f065406bad0 msgr2=0x7f065419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0653fff700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f065406bad0 0x7f065419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0653fff700 1 -- 172.21.15.90:0/336570337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065419a070 con 0x7f06541094a0 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0653fff700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06541094a0 0x7f0654199910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b830 tx=0x7f064c004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:15.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0650ff9700 1 -- 172.21.15.90:0/336570337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c021070 con 0x7f06541094a0 2024-04-26T18:09:15.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0650ff9700 1 -- 172.21.15.90:0/336570337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f064c003c00 con 0x7f06541094a0 2024-04-26T18:09:15.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065419a300 con 0x7f06541094a0 2024-04-26T18:09:15.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0650ff9700 1 -- 172.21.15.90:0/336570337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064c004730 con 0x7f06541094a0 2024-04-26T18:09:15.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.653+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06541a6620 con 0x7f06541094a0 2024-04-26T18:09:15.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.654+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065406a110 con 0x7f06541094a0 2024-04-26T18:09:15.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.655+0000 7f0650ff9700 1 -- 172.21.15.90:0/336570337 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f064c007a00 con 0x7f06541094a0 2024-04-26T18:09:15.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.655+0000 7f0650ff9700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f063c0a3cd0 0x7f063c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:15.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.656+0000 7f06537fe700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f063c0a3cd0 0x7f063c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:15.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.656+0000 7f0650ff9700 1 -- 172.21.15.90:0/336570337 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f064c00f020 con 0x7f06541094a0 2024-04-26T18:09:15.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.656+0000 7f06537fe700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f063c0a3cd0 0x7f063c0a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0644005950 tx=0x7f06440058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:15.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.660+0000 7f0650ff9700 1 -- 172.21.15.90:0/336570337 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f064c02a020 con 0x7f06541094a0 2024-04-26T18:09:15.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.895+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0654063860 con 0x7f063c0a3cd0 2024-04-26T18:09:15.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.900+0000 7f0650ff9700 1 -- 172.21.15.90:0/336570337 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0654063860 con 0x7f063c0a3cd0 2024-04-26T18:09:15.900 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_0/364375e6-c493-40a8-bac7-6b57a4130fc3 2024-04-26T18:09:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f063c0a3cd0 msgr2=0x7f063c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:15.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f063c0a3cd0 0x7f063c0a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0644005950 tx=0x7f06440058e0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06541094a0 msgr2=0x7f0654199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06541094a0 0x7f0654199910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f064c00b830 tx=0x7f064c004d90 comp rx=0 tx=0).stop 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 shutdown_connections 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f063c0a3cd0 0x7f063c0a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06541094a0 0x7f0654199910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f065406b130 0x7f065419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 --2- 172.21.15.90:0/336570337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f065406bad0 0x7f065419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 >> 172.21.15.90:0/336570337 conn(0x7f06540792d0 msgr2=0x7f0654100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 shutdown_connections 2024-04-26T18:09:15.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:15.902+0000 7f0659f18700 1 -- 172.21.15.90:0/336570337 wait complete. 2024-04-26T18:09:16.664 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T18:09:16.902 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:17.239 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:16 smithi090 ceph-mon[29180]: pgmap v312: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-26T18:09:17.239 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:16 smithi090 ceph-mon[29180]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:16 smithi119 ceph-mon[32752]: pgmap v312: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-26T18:09:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:16 smithi119 ceph-mon[32752]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:16 smithi167 ceph-mon[32751]: pgmap v312: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-04-26T18:09:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:16 smithi167 ceph-mon[32751]: from='client.14679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:18.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.411+0000 7fc271a67700 1 -- 172.21.15.90:0/3374310433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 msgr2=0x7fc26c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:18.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.411+0000 7fc271a67700 1 --2- 172.21.15.90:0/3374310433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c102790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc260009990 tx=0x7fc260009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.411+0000 7fc271a67700 1 -- 172.21.15.90:0/3374310433 shutdown_connections 2024-04-26T18:09:18.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.411+0000 7fc271a67700 1 --2- 172.21.15.90:0/3374310433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc26c102cd0 0x7fc26c10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.411+0000 7fc271a67700 1 --2- 172.21.15.90:0/3374310433 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc26c108380 0x7fc26c108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.411+0000 7fc271a67700 1 --2- 172.21.15.90:0/3374310433 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c102790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.411+0000 7fc271a67700 1 -- 172.21.15.90:0/3374310433 >> 172.21.15.90:0/3374310433 conn(0x7fc26c079290 msgr2=0x7fc26c0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 -- 172.21.15.90:0/3374310433 shutdown_connections 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 -- 172.21.15.90:0/3374310433 wait complete. 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 Processor -- start 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 -- start start 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc26c102cd0 0x7fc26c104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc26c108380 0x7fc26c104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c112520 con 0x7fc26c102310 2024-04-26T18:09:18.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1123a0 con 0x7fc26c102cd0 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.412+0000 7fc271a67700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c1126a0 con 0x7fc26c108380 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c103ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41330/0 (socket says 172.21.15.90:41330) 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 -- 172.21.15.90:0/3915596980 learned_addr learned my addr 172.21.15.90:0/3915596980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26affd700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc26c102cd0 0x7fc26c104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26bfff700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc26c108380 0x7fc26c104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 -- 172.21.15.90:0/3915596980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc26c108380 msgr2=0x7fc26c104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc26c108380 0x7fc26c104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 -- 172.21.15.90:0/3915596980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc26c102cd0 msgr2=0x7fc26c104530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc26c102cd0 0x7fc26c104530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 -- 172.21.15.90:0/3915596980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c1a8080 con 0x7fc26c102310 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc26b7fe700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c103ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00b530 tx=0x7fc25c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc268ff9700 1 -- 172.21.15.90:0/3915596980 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c015020 con 0x7fc26c102310 2024-04-26T18:09:18.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc268ff9700 1 -- 172.21.15.90:0/3915596980 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25c011ce0 con 0x7fc26c102310 2024-04-26T18:09:18.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c1a8370 con 0x7fc26c102310 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc268ff9700 1 -- 172.21.15.90:0/3915596980 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c010e40 con 0x7fc26c102310 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.413+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c1a8880 con 0x7fc26c102310 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.414+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c06a110 con 0x7fc26c102310 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.415+0000 7fc268ff9700 1 -- 172.21.15.90:0/3915596980 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc25c018020 con 0x7fc26c102310 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.416+0000 7fc268ff9700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc2540a3cd0 0x7fc2540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.416+0000 7fc268ff9700 1 -- 172.21.15.90:0/3915596980 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fc25c0d00a0 con 0x7fc26c102310 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.416+0000 7fc26affd700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc2540a3cd0 0x7fc2540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:18.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.416+0000 7fc26affd700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc2540a3cd0 0x7fc2540a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b5c0 tx=0x7fc260005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:18.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.420+0000 7fc268ff9700 1 -- 172.21.15.90:0/3915596980 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc25c01f020 con 0x7fc26c102310 2024-04-26T18:09:18.653 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.653+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc26c106130 con 0x7fc2540a3cd0 2024-04-26T18:09:18.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.678+0000 7fc268ff9700 1 -- 172.21.15.90:0/3915596980 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc26c106130 con 0x7fc2540a3cd0 2024-04-26T18:09:18.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc2540a3cd0 msgr2=0x7fc2540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc2540a3cd0 0x7fc2540a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc26000b5c0 tx=0x7fc260005960 comp rx=0 tx=0).stop 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 msgr2=0x7fc26c103ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c103ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc25c00b530 tx=0x7fc25c00b840 comp rx=0 tx=0).stop 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 shutdown_connections 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc2540a3cd0 0x7fc2540a6190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc26c102310 0x7fc26c103ff0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc26c108380 0x7fc26c104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 --2- 172.21.15.90:0/3915596980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc26c102cd0 0x7fc26c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.680+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 >> 172.21.15.90:0/3915596980 conn(0x7fc26c079290 msgr2=0x7fc26c1015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.681+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 shutdown_connections 2024-04-26T18:09:18.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:18.681+0000 7fc271a67700 1 -- 172.21.15.90:0/3915596980 wait complete. 2024-04-26T18:09:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:18 smithi119 ceph-mon[32752]: pgmap v313: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T18:09:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:18 smithi167 ceph-mon[32751]: pgmap v313: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T18:09:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:18 smithi090 ceph-mon[29180]: pgmap v313: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2024-04-26T18:09:19.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T18:09:19.687 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:20.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:19 smithi090 ceph-mon[29180]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:19 smithi119 ceph-mon[32752]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:19 smithi167 ceph-mon[32751]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:20 smithi090 ceph-mon[29180]: pgmap v314: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-26T18:09:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:20 smithi119 ceph-mon[32752]: pgmap v314: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-26T18:09:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:20 smithi167 ceph-mon[32751]: pgmap v314: 97 pgs: 97 active+clean; 607 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-26T18:09:21.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 -- 172.21.15.90:0/422752688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76641083d0 msgr2=0x7f76641087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 --2- 172.21.15.90:0/422752688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76641083d0 0x7f76641087b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7654009990 tx=0x7f7654009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 -- 172.21.15.90:0/422752688 shutdown_connections 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 --2- 172.21.15.90:0/422752688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76641083d0 0x7f76641087b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 --2- 172.21.15.90:0/422752688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7664102d10 0x7f766410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 --2- 172.21.15.90:0/422752688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7664102350 0x7f76641027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 -- 172.21.15.90:0/422752688 >> 172.21.15.90:0/422752688 conn(0x7f7664078390 msgr2=0x7f76640787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 -- 172.21.15.90:0/422752688 shutdown_connections 2024-04-26T18:09:21.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.359+0000 7f766a923700 1 -- 172.21.15.90:0/422752688 wait complete. 2024-04-26T18:09:21.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 Processor -- start 2024-04-26T18:09:21.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 -- start start 2024-04-26T18:09:21.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7664102350 0x7f7664104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7664102d10 0x7f7664104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76641083d0 0x7f7664104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f76641124a0 con 0x7f7664102d10 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7664112320 con 0x7f7664102350 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f766a923700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7664112620 con 0x7f76641083d0 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f76637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7664102d10 0x7f7664104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f76637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7664102d10 0x7f7664104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41360/0 (socket says 172.21.15.90:41360) 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f76637fe700 1 -- 172.21.15.90:0/959027215 learned_addr learned my addr 172.21.15.90:0/959027215 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.360+0000 7f7663fff700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7664102350 0x7f7664104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f7668ec0700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76641083d0 0x7f7664104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:21.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76637fe700 1 -- 172.21.15.90:0/959027215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76641083d0 msgr2=0x7f7664104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76637fe700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76641083d0 0x7f7664104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76637fe700 1 -- 172.21.15.90:0/959027215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7664102350 msgr2=0x7f7664104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76637fe700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7664102350 0x7f7664104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76637fe700 1 -- 172.21.15.90:0/959027215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76641a7ef0 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76637fe700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7664102d10 0x7f7664104540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f765800c9b0 tx=0x7f765800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76617fa700 1 -- 172.21.15.90:0/959027215 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658007b00 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76617fa700 1 -- 172.21.15.90:0/959027215 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7658007ca0 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76641a81e0 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f76617fa700 1 -- 172.21.15.90:0/959027215 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658018660 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.361+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76641a8720 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.362+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7664103a40 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.363+0000 7f76617fa700 1 -- 172.21.15.90:0/959027215 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f765800f450 con 0x7f7664102d10 2024-04-26T18:09:21.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.364+0000 7f76617fa700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f764c0a3cd0 0x7f764c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:21.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.364+0000 7f76617fa700 1 -- 172.21.15.90:0/959027215 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f76580d0640 con 0x7f7664102d10 2024-04-26T18:09:21.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.364+0000 7f7663fff700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f764c0a3cd0 0x7f764c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:21.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.364+0000 7f7663fff700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f764c0a3cd0 0x7f764c0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f765400b5c0 tx=0x7f7654005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:21.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.367+0000 7f76617fa700 1 -- 172.21.15.90:0/959027215 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7658014030 con 0x7f7664102d10 2024-04-26T18:09:21.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.601+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f7664106120 con 0x7f764c0a3cd0 2024-04-26T18:09:21.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.608+0000 7f76617fa700 1 -- 172.21.15.90:0/959027215 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f7664106120 con 0x7f764c0a3cd0 2024-04-26T18:09:21.608 INFO:teuthology.orchestra.run.smithi090.stdout:/volumes/qa/sv_1/85328d59-d769-406c-88c5-4148365dc53c 2024-04-26T18:09:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f764c0a3cd0 msgr2=0x7f764c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f764c0a3cd0 0x7f764c0a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f765400b5c0 tx=0x7f7654005960 comp rx=0 tx=0).stop 2024-04-26T18:09:21.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7664102d10 msgr2=0x7f7664104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7664102d10 0x7f7664104540 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f765800c9b0 tx=0x7f765800ccc0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 shutdown_connections 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f764c0a3cd0 0x7f764c0a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76641083d0 0x7f7664104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7664102350 0x7f7664104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 --2- 172.21.15.90:0/959027215 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7664102d10 0x7f7664104540 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 >> 172.21.15.90:0/959027215 conn(0x7f7664078390 msgr2=0x7f7664101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 shutdown_connections 2024-04-26T18:09:21.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:21.610+0000 7f766a923700 1 -- 172.21.15.90:0/959027215 wait complete. 2024-04-26T18:09:22.361 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:09:22.600 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:22 smithi119 ceph-mon[32752]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:22 smithi119 ceph-mon[32752]: pgmap v315: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-26T18:09:23.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:22 smithi090 ceph-mon[29180]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:23.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:22 smithi090 ceph-mon[29180]: pgmap v315: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-26T18:09:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:22 smithi167 ceph-mon[32751]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:09:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:22 smithi167 ceph-mon[32751]: pgmap v315: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-26T18:09:24.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 -- 172.21.15.90:0/11752620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 msgr2=0x7f8c641038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/11752620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c641038f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009990 tx=0x7f8c58009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 -- 172.21.15.90:0/11752620 shutdown_connections 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/11752620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c64103e30 0x7f8c64110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/11752620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c641094c0 0x7f8c641098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/11752620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c641038f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 -- 172.21.15.90:0/11752620 >> 172.21.15.90:0/11752620 conn(0x7f8c6407a580 msgr2=0x7f8c6407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 -- 172.21.15.90:0/11752620 shutdown_connections 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 -- 172.21.15.90:0/11752620 wait complete. 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.064+0000 7f8c6a05b700 1 Processor -- start 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c6a05b700 1 -- start start 2024-04-26T18:09:24.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c6a05b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c6419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c6a05b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64103e30 0x7f8c6419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c6a05b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c641094c0 0x7f8c64199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c6a05b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641135c0 con 0x7f8c64103470 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c6a05b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64113440 con 0x7f8c64103e30 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c6a05b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64113740 con 0x7f8c641094c0 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c6419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c6419f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40736/0 (socket says 172.21.15.90:40736) 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 -- 172.21.15.90:0/3004043168 learned_addr learned my addr 172.21.15.90:0/3004043168 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c62ffd700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64103e30 0x7f8c6419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c63fff700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c641094c0 0x7f8c64199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 -- 172.21.15.90:0/3004043168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c641094c0 msgr2=0x7f8c64199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c641094c0 0x7f8c64199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 -- 172.21.15.90:0/3004043168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64103e30 msgr2=0x7f8c6419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64103e30 0x7f8c6419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 -- 172.21.15.90:0/3004043168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c6419a0c0 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.065+0000 7f8c637fe700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c6419f7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400d980 tx=0x7f8c5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.066+0000 7f8c60ff9700 1 -- 172.21.15.90:0/3004043168 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c54016070 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.066+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c6419a3b0 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.066+0000 7f8c60ff9700 1 -- 172.21.15.90:0/3004043168 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c54004830 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.066+0000 7f8c60ff9700 1 -- 172.21.15.90:0/3004043168 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c540054c0 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.066+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641a6670 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.066+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c6406a110 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.068+0000 7f8c60ff9700 1 -- 172.21.15.90:0/3004043168 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8c54004220 con 0x7f8c64103470 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.068+0000 7f8c60ff9700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8c4c0a3cd0 0x7f8c4c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.068+0000 7f8c62ffd700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8c4c0a3cd0 0x7f8c4c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:24.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.068+0000 7f8c60ff9700 1 -- 172.21.15.90:0/3004043168 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7f8c540d0640 con 0x7f8c64103470 2024-04-26T18:09:24.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.069+0000 7f8c62ffd700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8c4c0a3cd0 0x7f8c4c0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c580098e0 tx=0x7f8c58005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:24.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.072+0000 7f8c60ff9700 1 -- 172.21.15.90:0/3004043168 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c5401f020 con 0x7f8c64103470 2024-04-26T18:09:24.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.321+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c6402e490 con 0x7f8c64103470 2024-04-26T18:09:24.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.321+0000 7f8c60ff9700 1 -- 172.21.15.90:0/3004043168 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11874 (secure 0 0 0) 0x7f8c5409d070 con 0x7f8c64103470 2024-04-26T18:09:24.321 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:09:24.322 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:09:06.760287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24448,24449,24436]},"id":1}]} 2024-04-26T18:09:24.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8c4c0a3cd0 msgr2=0x7f8c4c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:24.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8c4c0a3cd0 0x7f8c4c0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c580098e0 tx=0x7f8c58005960 comp rx=0 tx=0).stop 2024-04-26T18:09:24.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 msgr2=0x7f8c6419f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c6419f7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8c5400d980 tx=0x7f8c5400dc90 comp rx=0 tx=0).stop 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 shutdown_connections 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8c4c0a3cd0 0x7f8c4c0a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c64103470 0x7f8c6419f7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c641094c0 0x7f8c64199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 --2- 172.21.15.90:0/3004043168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64103e30 0x7f8c6419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.323+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 >> 172.21.15.90:0/3004043168 conn(0x7f8c6407a580 msgr2=0x7f8c64102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.324+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 shutdown_connections 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:24.324+0000 7f8c6a05b700 1 -- 172.21.15.90:0/3004043168 wait complete. 2024-04-26T18:09:24.324 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T18:09:24.948 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:09:25.189 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:24 smithi119 ceph-mon[32752]: pgmap v316: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-26T18:09:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:24 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3004043168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:09:25.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:24 smithi090 ceph-mon[29180]: pgmap v316: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-26T18:09:25.310 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:24 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3004043168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:09:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:24 smithi167 ceph-mon[32751]: pgmap v316: 97 pgs: 97 active+clean; 630 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.3 KiB/s wr, 8 op/s 2024-04-26T18:09:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:24 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3004043168' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:09:26.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 -- 172.21.15.90:0/4142655165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103470 msgr2=0x7fcb0c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:26.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 --2- 172.21.15.90:0/4142655165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103470 0x7fcb0c1038f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009990 tx=0x7fcb00009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:26.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 -- 172.21.15.90:0/4142655165 shutdown_connections 2024-04-26T18:09:26.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 --2- 172.21.15.90:0/4142655165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb0c103e30 0x7fcb0c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:26.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 --2- 172.21.15.90:0/4142655165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c1094c0 0x7fcb0c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:26.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 --2- 172.21.15.90:0/4142655165 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103470 0x7fcb0c1038f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:26.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 -- 172.21.15.90:0/4142655165 >> 172.21.15.90:0/4142655165 conn(0x7fcb0c07a580 msgr2=0x7fcb0c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:26.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 -- 172.21.15.90:0/4142655165 shutdown_connections 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.797+0000 7fcb1131e700 1 -- 172.21.15.90:0/4142655165 wait complete. 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 Processor -- start 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 -- start start 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb0c103470 0x7fcb0c19beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103e30 0x7fcb0c19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c1094c0 0x7fcb0c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c1136c0 con 0x7fcb0c103e30 2024-04-26T18:09:26.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c113540 con 0x7fcb0c1094c0 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb1131e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0c113840 con 0x7fcb0c103470 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb0a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103e30 0x7fcb0c19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb0a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103e30 0x7fcb0c19c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40746/0 (socket says 172.21.15.90:40746) 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.798+0000 7fcb0a7fc700 1 -- 172.21.15.90:0/866523144 learned_addr learned my addr 172.21.15.90:0/866523144 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0b7fe700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c1094c0 0x7fcb0c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0a7fc700 1 -- 172.21.15.90:0/866523144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb0c103470 msgr2=0x7fcb0c19beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0affd700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb0c103470 0x7fcb0c19beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0a7fc700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb0c103470 0x7fcb0c19beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:26.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0a7fc700 1 -- 172.21.15.90:0/866523144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c1094c0 msgr2=0x7fcb0c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0a7fc700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c1094c0 0x7fcb0c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0a7fc700 1 -- 172.21.15.90:0/866523144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0c1a0ec0 con 0x7fcb0c103e30 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0b7fe700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c1094c0 0x7fcb0c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0affd700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb0c103470 0x7fcb0c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb0a7fc700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103e30 0x7fcb0c19c3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcaf3fff700 1 -- 172.21.15.90:0/866523144 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00027070 con 0x7fcb0c103e30 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcaf3fff700 1 -- 172.21.15.90:0/866523144 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb000041d0 con 0x7fcb0c103e30 2024-04-26T18:09:26.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0c1a1150 con 0x7fcb0c103e30 2024-04-26T18:09:26.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcaf3fff700 1 -- 172.21.15.90:0/866523144 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0000f460 con 0x7fcb0c103e30 2024-04-26T18:09:26.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.799+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0c1a8a30 con 0x7fcb0c103e30 2024-04-26T18:09:26.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.800+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0c06a110 con 0x7fcb0c103e30 2024-04-26T18:09:26.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.801+0000 7fcaf3fff700 1 -- 172.21.15.90:0/866523144 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcb0002a070 con 0x7fcb0c103e30 2024-04-26T18:09:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.802+0000 7fcaf3fff700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcaf40a3cd0 0x7fcaf40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.802+0000 7fcb0affd700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcaf40a3cd0 0x7fcaf40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.802+0000 7fcaf3fff700 1 -- 172.21.15.90:0/866523144 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fcb000d17c0 con 0x7fcb0c103e30 2024-04-26T18:09:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.802+0000 7fcb0affd700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcaf40a3cd0 0x7fcaf40a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007900 tx=0x7fcafc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:26.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:26.805+0000 7fcaf3fff700 1 -- 172.21.15.90:0/866523144 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb00031070 con 0x7fcb0c103e30 2024-04-26T18:09:27.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.009+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb0c1a8c00 con 0x7fcb0c103e30 2024-04-26T18:09:27.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.011+0000 7fcaf3fff700 1 -- 172.21.15.90:0/866523144 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22217 (secure 0 0 0) 0x7fcb00013070 con 0x7fcb0c103e30 2024-04-26T18:09:27.011 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:09:27.011 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":90,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:08:49.029325+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcaf40a3cd0 msgr2=0x7fcaf40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcaf40a3cd0 0x7fcaf40a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcafc007900 tx=0x7fcafc008040 comp rx=0 tx=0).stop 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103e30 msgr2=0x7fcb0c19c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103e30 0x7fcb0c19c3f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcb00000c00 tx=0x7fcb000047b0 comp rx=0 tx=0).stop 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 shutdown_connections 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcaf40a3cd0 0x7fcaf40a6190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb0c1094c0 0x7fcb0c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:27.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb0c103470 0x7fcb0c19beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 --2- 172.21.15.90:0/866523144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb0c103e30 0x7fcb0c19c3f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 >> 172.21.15.90:0/866523144 conn(0x7fcb0c07a580 msgr2=0x7fcb0c1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 shutdown_connections 2024-04-26T18:09:27.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:27.013+0000 7fcb1131e700 1 -- 172.21.15.90:0/866523144 wait complete. 2024-04-26T18:09:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:26 smithi119 ceph-mon[32752]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-26T18:09:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:26 smithi167 ceph-mon[32751]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-26T18:09:27.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:26 smithi090 ceph-mon[29180]: pgmap v317: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.6 KiB/s wr, 9 op/s 2024-04-26T18:09:27.709 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:09:27.709 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:09:27.959 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:27 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/866523144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:27 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/866523144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:28.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:27 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/866523144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:29.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:28 smithi090 ceph-mon[29180]: pgmap v318: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-26T18:09:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:28 smithi119 ceph-mon[32752]: pgmap v318: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-26T18:09:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:28 smithi167 ceph-mon[32751]: pgmap v318: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-26T18:09:29.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.569+0000 7fe00f3c8700 1 -- 172.21.15.90:0/2885490590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 msgr2=0x7fe008109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.569+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/2885490590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008109860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe004009990 tx=0x7fe004009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- 172.21.15.90:0/2885490590 shutdown_connections 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/2885490590 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008109860 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/2885490590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe008103ea0 0x7fe0081102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/2885490590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081034e0 0x7fe008103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- 172.21.15.90:0/2885490590 >> 172.21.15.90:0/2885490590 conn(0x7fe00807a5e0 msgr2=0x7fe00807a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- 172.21.15.90:0/2885490590 shutdown_connections 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- 172.21.15.90:0/2885490590 wait complete. 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 Processor -- start 2024-04-26T18:09:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- start start 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081034e0 0x7fe00819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe008103ea0 0x7fe00819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe008113590 con 0x7fe008109480 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe008113410 con 0x7fe0081034e0 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.570+0000 7fe00f3c8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe008113710 con 0x7fe008103ea0 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40774/0 (socket says 172.21.15.90:40774) 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 -- 172.21.15.90:0/1317710341 learned_addr learned my addr 172.21.15.90:0/1317710341 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d164700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081034e0 0x7fe00819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00c963700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe008103ea0 0x7fe00819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 -- 172.21.15.90:0/1317710341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe008103ea0 msgr2=0x7fe00819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:29.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe008103ea0 0x7fe00819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 -- 172.21.15.90:0/1317710341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081034e0 msgr2=0x7fe00819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081034e0 0x7fe00819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 -- 172.21.15.90:0/1317710341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe008199ff0 con 0x7fe008109480 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00c963700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe008103ea0 0x7fe00819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00d965700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008199810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe000007d70 tx=0x7fe00000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fdffe7fc700 1 -- 172.21.15.90:0/1317710341 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000016020 con 0x7fe008109480 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fdffe7fc700 1 -- 172.21.15.90:0/1317710341 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe00000ad40 con 0x7fe008109480 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00819a2e0 con 0x7fe008109480 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fdffe7fc700 1 -- 172.21.15.90:0/1317710341 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0000115e0 con 0x7fe008109480 2024-04-26T18:09:29.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.571+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081a64a0 con 0x7fe008109480 2024-04-26T18:09:29.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.572+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe008104b30 con 0x7fe008109480 2024-04-26T18:09:29.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.573+0000 7fdffe7fc700 1 -- 172.21.15.90:0/1317710341 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe000019020 con 0x7fe008109480 2024-04-26T18:09:29.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.574+0000 7fdffe7fc700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdff40a3d50 0x7fdff40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:29.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.574+0000 7fe00d164700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdff40a3d50 0x7fdff40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:29.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.574+0000 7fdffe7fc700 1 -- 172.21.15.90:0/1317710341 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7979+0+0 (secure 0 0 0) 0x7fe0000d01c0 con 0x7fe008109480 2024-04-26T18:09:29.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.574+0000 7fe00d164700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdff40a3d50 0x7fdff40a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe00400b5c0 tx=0x7fe004005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:29.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.578+0000 7fdffe7fc700 1 -- 172.21.15.90:0/1317710341 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe000020020 con 0x7fe008109480 2024-04-26T18:09:29.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.774+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe008101ba0 con 0x7fdff40a3d50 2024-04-26T18:09:29.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.781+0000 7fdffe7fc700 1 -- 172.21.15.90:0/1317710341 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258691 (secure 0 0 0) 0x7fe008101ba0 con 0x7fdff40a3d50 2024-04-26T18:09:29.786 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdff40a3d50 msgr2=0x7fdff40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdff40a3d50 0x7fdff40a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fe00400b5c0 tx=0x7fe004005960 comp rx=0 tx=0).stop 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 msgr2=0x7fe008199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008199810 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe000007d70 tx=0x7fe00000a340 comp rx=0 tx=0).stop 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 shutdown_connections 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdff40a3d50 0x7fdff40a6210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe008109480 0x7fe008199810 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0081034e0 0x7fe00819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 --2- 172.21.15.90:0/1317710341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe008103ea0 0x7fe00819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:29.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 >> 172.21.15.90:0/1317710341 conn(0x7fe00807a5e0 msgr2=0x7fe008101490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:29.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 shutdown_connections 2024-04-26T18:09:29.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:29.784+0000 7fe00f3c8700 1 -- 172.21.15.90:0/1317710341 wait complete. 2024-04-26T18:09:29.789 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T18:09:30.525 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-04-26T18:09:27.878679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649361,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":291,"num_read_kb":260,"num_write":344,"num_write_kb":3043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375212,"kb_used_data":13580,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566612,"statfs":{"total":1151940427776,"available":1151556210688,"internally_reserved":0,"allocated":13905920,"data_stored":7608510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27539,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":58,"num_write":20,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.021802"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361491+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:08:49.361491+0000","last_peered":"2024-04-26T18:08:49.361491+0000","last_clean":"2024-04-26T18:08:49.361491+0000","last_became_active":"2024-04-26T18:08:47.377408+0000","last_became_peered":"2024-04-26T18:08:47.377408+0000","last_unstale":"2024-04-26T18:08:49.361491+0000","last_undegraded":"2024-04-26T18:08:49.361491+0000","last_fullsized":"2024-04-26T18:08:49.361491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:13.822258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372467+0000","last_change":"2024-04-26T18:08:47.365125+0000","last_active":"2024-04-26T18:08:49.372467+0000","last_peered":"2024-04-26T18:08:49.372467+0000","last_clean":"2024-04-26T18:08:49.372467+0000","last_became_active":"2024-04-26T18:08:47.364847+0000","last_became_peered":"2024-04-26T18:08:47.364847+0000","last_unstale":"2024-04-26T18:08:49.372467+0000","last_undegraded":"2024-04-26T18:08:49.372467+0000","last_fullsized":"2024-04-26T18:08:49.372467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:16:37.890893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372423+0000","last_change":"2024-04-26T18:08:47.377580+0000","last_active":"2024-04-26T18:08:49.372423+0000","last_peered":"2024-04-26T18:08:49.372423+0000","last_clean":"2024-04-26T18:08:49.372423+0000","last_became_active":"2024-04-26T18:08:47.377219+0000","last_became_peered":"2024-04-26T18:08:47.377219+0000","last_unstale":"2024-04-26T18:08:49.372423+0000","last_undegraded":"2024-04-26T18:08:49.372423+0000","last_fullsized":"2024-04-26T18:08:49.372423+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:12.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367582+0000","last_change":"2024-04-26T18:08:47.371628+0000","last_active":"2024-04-26T18:08:49.367582+0000","last_peered":"2024-04-26T18:08:49.367582+0000","last_clean":"2024-04-26T18:08:49.367582+0000","last_became_active":"2024-04-26T18:08:47.370840+0000","last_became_peered":"2024-04-26T18:08:47.370840+0000","last_unstale":"2024-04-26T18:08:49.367582+0000","last_undegraded":"2024-04-26T18:08:49.367582+0000","last_fullsized":"2024-04-26T18:08:49.367582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:11.362664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367277+0000","last_change":"2024-04-26T18:08:47.367632+0000","last_active":"2024-04-26T18:08:49.367277+0000","last_peered":"2024-04-26T18:08:49.367277+0000","last_clean":"2024-04-26T18:08:49.367277+0000","last_became_active":"2024-04-26T18:08:47.367316+0000","last_became_peered":"2024-04-26T18:08:47.367316+0000","last_unstale":"2024-04-26T18:08:49.367277+0000","last_undegraded":"2024-04-26T18:08:49.367277+0000","last_fullsized":"2024-04-26T18:08:49.367277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:31:26.665292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367546+0000","last_change":"2024-04-26T18:08:47.369742+0000","last_active":"2024-04-26T18:08:49.367546+0000","last_peered":"2024-04-26T18:08:49.367546+0000","last_clean":"2024-04-26T18:08:49.367546+0000","last_became_active":"2024-04-26T18:08:47.369230+0000","last_became_peered":"2024-04-26T18:08:47.369230+0000","last_unstale":"2024-04-26T18:08:49.367546+0000","last_undegraded":"2024-04-26T18:08:49.367546+0000","last_fullsized":"2024-04-26T18:08:49.367546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:02:04.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373228+0000","last_change":"2024-04-26T18:08:47.370657+0000","last_active":"2024-04-26T18:08:49.373228+0000","last_peered":"2024-04-26T18:08:49.373228+0000","last_clean":"2024-04-26T18:08:49.373228+0000","last_became_active":"2024-04-26T18:08:47.370133+0000","last_became_peered":"2024-04-26T18:08:47.370133+0000","last_unstale":"2024-04-26T18:08:49.373228+0000","last_undegraded":"2024-04-26T18:08:49.373228+0000","last_fullsized":"2024-04-26T18:08:49.373228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:31.232179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369989+0000","last_change":"2024-04-26T18:08:47.366221+0000","last_active":"2024-04-26T18:08:49.369989+0000","last_peered":"2024-04-26T18:08:49.369989+0000","last_clean":"2024-04-26T18:08:49.369989+0000","last_became_active":"2024-04-26T18:08:47.365990+0000","last_became_peered":"2024-04-26T18:08:47.365990+0000","last_unstale":"2024-04-26T18:08:49.369989+0000","last_undegraded":"2024-04-26T18:08:49.369989+0000","last_fullsized":"2024-04-26T18:08:49.369989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:43.707583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361759+0000","last_change":"2024-04-26T18:08:47.373165+0000","last_active":"2024-04-26T18:08:49.361759+0000","last_peered":"2024-04-26T18:08:49.361759+0000","last_clean":"2024-04-26T18:08:49.361759+0000","last_became_active":"2024-04-26T18:08:47.372688+0000","last_became_peered":"2024-04-26T18:08:47.372688+0000","last_unstale":"2024-04-26T18:08:49.361759+0000","last_undegraded":"2024-04-26T18:08:49.361759+0000","last_fullsized":"2024-04-26T18:08:49.361759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:18.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367900+0000","last_change":"2024-04-26T18:08:47.373963+0000","last_active":"2024-04-26T18:08:49.367900+0000","last_peered":"2024-04-26T18:08:49.367900+0000","last_clean":"2024-04-26T18:08:49.367900+0000","last_became_active":"2024-04-26T18:08:47.372878+0000","last_became_peered":"2024-04-26T18:08:47.372878+0000","last_unstale":"2024-04-26T18:08:49.367900+0000","last_undegraded":"2024-04-26T18:08:49.367900+0000","last_fullsized":"2024-04-26T18:08:49.367900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:25.178809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372276+0000","last_change":"2024-04-26T18:08:47.378643+0000","last_active":"2024-04-26T18:08:49.372276+0000","last_peered":"2024-04-26T18:08:49.372276+0000","last_clean":"2024-04-26T18:08:49.372276+0000","last_became_active":"2024-04-26T18:08:47.378329+0000","last_became_peered":"2024-04-26T18:08:47.378329+0000","last_unstale":"2024-04-26T18:08:49.372276+0000","last_undegraded":"2024-04-26T18:08:49.372276+0000","last_fullsized":"2024-04-26T18:08:49.372276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:59.016025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367405+0000","last_change":"2024-04-26T18:08:47.371663+0000","last_active":"2024-04-26T18:08:49.367405+0000","last_peered":"2024-04-26T18:08:49.367405+0000","last_clean":"2024-04-26T18:08:49.367405+0000","last_became_active":"2024-04-26T18:08:47.371298+0000","last_became_peered":"2024-04-26T18:08:47.371298+0000","last_unstale":"2024-04-26T18:08:49.367405+0000","last_undegraded":"2024-04-26T18:08:49.367405+0000","last_fullsized":"2024-04-26T18:08:49.367405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:04.884698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363040+0000","last_change":"2024-04-26T18:08:47.365179+0000","last_active":"2024-04-26T18:08:49.363040+0000","last_peered":"2024-04-26T18:08:49.363040+0000","last_clean":"2024-04-26T18:08:49.363040+0000","last_became_active":"2024-04-26T18:08:47.364888+0000","last_became_peered":"2024-04-26T18:08:47.364888+0000","last_unstale":"2024-04-26T18:08:49.363040+0000","last_undegraded":"2024-04-26T18:08:49.363040+0000","last_fullsized":"2024-04-26T18:08:49.363040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:34.082855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372196+0000","last_change":"2024-04-26T18:08:47.364828+0000","last_active":"2024-04-26T18:08:49.372196+0000","last_peered":"2024-04-26T18:08:49.372196+0000","last_clean":"2024-04-26T18:08:49.372196+0000","last_became_active":"2024-04-26T18:08:47.364529+0000","last_became_peered":"2024-04-26T18:08:47.364529+0000","last_unstale":"2024-04-26T18:08:49.372196+0000","last_undegraded":"2024-04-26T18:08:49.372196+0000","last_fullsized":"2024-04-26T18:08:49.372196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.480258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371662+0000","last_change":"2024-04-26T18:08:47.373578+0000","last_active":"2024-04-26T18:08:49.371662+0000","last_peered":"2024-04-26T18:08:49.371662+0000","last_clean":"2024-04-26T18:08:49.371662+0000","last_became_active":"2024-04-26T18:08:47.373263+0000","last_became_peered":"2024-04-26T18:08:47.373263+0000","last_unstale":"2024-04-26T18:08:49.371662+0000","last_undegraded":"2024-04-26T18:08:49.371662+0000","last_fullsized":"2024-04-26T18:08:49.371662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:25.340241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.360993+0000","last_change":"2024-04-26T18:08:47.376617+0000","last_active":"2024-04-26T18:08:49.360993+0000","last_peered":"2024-04-26T18:08:49.360993+0000","last_clean":"2024-04-26T18:08:49.360993+0000","last_became_active":"2024-04-26T18:08:47.376290+0000","last_became_peered":"2024-04-26T18:08:47.376290+0000","last_unstale":"2024-04-26T18:08:49.360993+0000","last_undegraded":"2024-04-26T18:08:49.360993+0000","last_fullsized":"2024-04-26T18:08:49.360993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:51.187583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361650+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:08:49.361650+0000","last_peered":"2024-04-26T18:08:49.361650+0000","last_clean":"2024-04-26T18:08:49.361650+0000","last_became_active":"2024-04-26T18:08:47.377427+0000","last_became_peered":"2024-04-26T18:08:47.377427+0000","last_unstale":"2024-04-26T18:08:49.361650+0000","last_undegraded":"2024-04-26T18:08:49.361650+0000","last_fullsized":"2024-04-26T18:08:49.361650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.529500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369299+0000","last_change":"2024-04-26T18:08:47.373314+0000","last_active":"2024-04-26T18:08:49.369299+0000","last_peered":"2024-04-26T18:08:49.369299+0000","last_clean":"2024-04-26T18:08:49.369299+0000","last_became_active":"2024-04-26T18:08:47.372868+0000","last_became_peered":"2024-04-26T18:08:47.372868+0000","last_unstale":"2024-04-26T18:08:49.369299+0000","last_undegraded":"2024-04-26T18:08:49.369299+0000","last_fullsized":"2024-04-26T18:08:49.369299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:19.642655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368630+0000","last_change":"2024-04-26T18:08:47.367593+0000","last_active":"2024-04-26T18:08:49.368630+0000","last_peered":"2024-04-26T18:08:49.368630+0000","last_clean":"2024-04-26T18:08:49.368630+0000","last_became_active":"2024-04-26T18:08:47.367243+0000","last_became_peered":"2024-04-26T18:08:47.367243+0000","last_unstale":"2024-04-26T18:08:49.368630+0000","last_undegraded":"2024-04-26T18:08:49.368630+0000","last_fullsized":"2024-04-26T18:08:49.368630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.380307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.944535+0000","last_change":"2024-04-26T18:08:44.339625+0000","last_active":"2024-04-26T18:09:01.944535+0000","last_peered":"2024-04-26T18:09:01.944535+0000","last_clean":"2024-04-26T18:09:01.944535+0000","last_became_active":"2024-04-26T18:08:44.339209+0000","last_became_peered":"2024-04-26T18:08:44.339209+0000","last_unstale":"2024-04-26T18:09:01.944535+0000","last_undegraded":"2024-04-26T18:09:01.944535+0000","last_fullsized":"2024-04-26T18:09:01.944535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:43.169885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369176+0000","last_change":"2024-04-26T18:08:47.375884+0000","last_active":"2024-04-26T18:08:49.369176+0000","last_peered":"2024-04-26T18:08:49.369176+0000","last_clean":"2024-04-26T18:08:49.369176+0000","last_became_active":"2024-04-26T18:08:47.375557+0000","last_became_peered":"2024-04-26T18:08:47.375557+0000","last_unstale":"2024-04-26T18:08:49.369176+0000","last_undegraded":"2024-04-26T18:08:49.369176+0000","last_fullsized":"2024-04-26T18:08:49.369176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.593910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.376172+0000","last_change":"2024-04-26T18:08:44.557167+0000","last_active":"2024-04-26T18:08:49.376172+0000","last_peered":"2024-04-26T18:08:49.376172+0000","last_clean":"2024-04-26T18:08:49.376172+0000","last_became_active":"2024-04-26T18:08:44.556716+0000","last_became_peered":"2024-04-26T18:08:44.556716+0000","last_unstale":"2024-04-26T18:08:49.376172+0000","last_undegraded":"2024-04-26T18:08:49.376172+0000","last_fullsized":"2024-04-26T18:08:49.376172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372636+0000","last_change":"2024-04-26T18:08:47.377612+0000","last_active":"2024-04-26T18:08:49.372636+0000","last_peered":"2024-04-26T18:08:49.372636+0000","last_clean":"2024-04-26T18:08:49.372636+0000","last_became_active":"2024-04-26T18:08:47.377231+0000","last_became_peered":"2024-04-26T18:08:47.377231+0000","last_unstale":"2024-04-26T18:08:49.372636+0000","last_undegraded":"2024-04-26T18:08:49.372636+0000","last_fullsized":"2024-04-26T18:08:49.372636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:11.594373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.943298+0000","last_change":"2024-04-26T18:08:44.343429+0000","last_active":"2024-04-26T18:09:01.943298+0000","last_peered":"2024-04-26T18:09:01.943298+0000","last_clean":"2024-04-26T18:09:01.943298+0000","last_became_active":"2024-04-26T18:08:44.343103+0000","last_became_peered":"2024-04-26T18:08:44.343103+0000","last_unstale":"2024-04-26T18:09:01.943298+0000","last_undegraded":"2024-04-26T18:09:01.943298+0000","last_fullsized":"2024-04-26T18:09:01.943298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:52.244159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361146+0000","last_change":"2024-04-26T18:08:47.373130+0000","last_active":"2024-04-26T18:08:49.361146+0000","last_peered":"2024-04-26T18:08:49.361146+0000","last_clean":"2024-04-26T18:08:49.361146+0000","last_became_active":"2024-04-26T18:08:47.372726+0000","last_became_peered":"2024-04-26T18:08:47.372726+0000","last_unstale":"2024-04-26T18:08:49.361146+0000","last_undegraded":"2024-04-26T18:08:49.361146+0000","last_fullsized":"2024-04-26T18:08:49.361146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:44.461177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.377157+0000","last_change":"2024-04-26T18:08:44.340152+0000","last_active":"2024-04-26T18:08:49.377157+0000","last_peered":"2024-04-26T18:08:49.377157+0000","last_clean":"2024-04-26T18:08:49.377157+0000","last_became_active":"2024-04-26T18:08:44.339684+0000","last_became_peered":"2024-04-26T18:08:44.339684+0000","last_unstale":"2024-04-26T18:08:49.377157+0000","last_undegraded":"2024-04-26T18:08:49.377157+0000","last_fullsized":"2024-04-26T18:08:49.377157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:49.961201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371953+0000","last_change":"2024-04-26T18:08:47.375387+0000","last_active":"2024-04-26T18:08:49.371953+0000","last_peered":"2024-04-26T18:08:49.371953+0000","last_clean":"2024-04-26T18:08:49.371953+0000","last_became_active":"2024-04-26T18:08:47.374689+0000","last_became_peered":"2024-04-26T18:08:47.374689+0000","last_unstale":"2024-04-26T18:08:49.371953+0000","last_undegraded":"2024-04-26T18:08:49.371953+0000","last_fullsized":"2024-04-26T18:08:49.371953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:05.618804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:21.976034+0000","last_change":"2024-04-26T18:08:44.344167+0000","last_active":"2024-04-26T18:09:21.976034+0000","last_peered":"2024-04-26T18:09:21.976034+0000","last_clean":"2024-04-26T18:09:21.976034+0000","last_became_active":"2024-04-26T18:08:44.343646+0000","last_became_peered":"2024-04-26T18:08:44.343646+0000","last_unstale":"2024-04-26T18:09:21.976034+0000","last_undegraded":"2024-04-26T18:09:21.976034+0000","last_fullsized":"2024-04-26T18:09:21.976034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:53.859483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367042+0000","last_change":"2024-04-26T18:08:47.370546+0000","last_active":"2024-04-26T18:08:49.367042+0000","last_peered":"2024-04-26T18:08:49.367042+0000","last_clean":"2024-04-26T18:08:49.367042+0000","last_became_active":"2024-04-26T18:08:47.370220+0000","last_became_peered":"2024-04-26T18:08:47.370220+0000","last_unstale":"2024-04-26T18:08:49.367042+0000","last_undegraded":"2024-04-26T18:08:49.367042+0000","last_fullsized":"2024-04-26T18:08:49.367042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:05.139824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.943848+0000","last_change":"2024-04-26T18:08:44.347031+0000","last_active":"2024-04-26T18:09:01.943848+0000","last_peered":"2024-04-26T18:09:01.943848+0000","last_clean":"2024-04-26T18:09:01.943848+0000","last_became_active":"2024-04-26T18:08:44.346650+0000","last_became_peered":"2024-04-26T18:08:44.346650+0000","last_unstale":"2024-04-26T18:09:01.943848+0000","last_undegraded":"2024-04-26T18:09:01.943848+0000","last_fullsized":"2024-04-26T18:09:01.943848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:50.946449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373824+0000","last_change":"2024-04-26T18:08:47.374230+0000","last_active":"2024-04-26T18:08:49.373824+0000","last_peered":"2024-04-26T18:08:49.373824+0000","last_clean":"2024-04-26T18:08:49.373824+0000","last_became_active":"2024-04-26T18:08:47.370780+0000","last_became_peered":"2024-04-26T18:08:47.370780+0000","last_unstale":"2024-04-26T18:08:49.373824+0000","last_undegraded":"2024-04-26T18:08:49.373824+0000","last_fullsized":"2024-04-26T18:08:49.373824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:01.171195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.942323+0000","last_change":"2024-04-26T18:08:44.338318+0000","last_active":"2024-04-26T18:09:01.942323+0000","last_peered":"2024-04-26T18:09:01.942323+0000","last_clean":"2024-04-26T18:09:01.942323+0000","last_became_active":"2024-04-26T18:08:44.338009+0000","last_became_peered":"2024-04-26T18:08:44.338009+0000","last_unstale":"2024-04-26T18:09:01.942323+0000","last_undegraded":"2024-04-26T18:09:01.942323+0000","last_fullsized":"2024-04-26T18:09:01.942323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:33.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370158+0000","last_change":"2024-04-26T18:08:47.363924+0000","last_active":"2024-04-26T18:08:49.370158+0000","last_peered":"2024-04-26T18:08:49.370158+0000","last_clean":"2024-04-26T18:08:49.370158+0000","last_became_active":"2024-04-26T18:08:47.363616+0000","last_became_peered":"2024-04-26T18:08:47.363616+0000","last_unstale":"2024-04-26T18:08:49.370158+0000","last_undegraded":"2024-04-26T18:08:49.370158+0000","last_fullsized":"2024-04-26T18:08:49.370158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:27:08.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373675+0000","last_change":"2024-04-26T18:08:44.343350+0000","last_active":"2024-04-26T18:08:49.373675+0000","last_peered":"2024-04-26T18:08:49.373675+0000","last_clean":"2024-04-26T18:08:49.373675+0000","last_became_active":"2024-04-26T18:08:44.343040+0000","last_became_peered":"2024-04-26T18:08:44.343040+0000","last_unstale":"2024-04-26T18:08:49.373675+0000","last_undegraded":"2024-04-26T18:08:49.373675+0000","last_fullsized":"2024-04-26T18:08:49.373675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:09.427785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368108+0000","last_change":"2024-04-26T18:08:47.360180+0000","last_active":"2024-04-26T18:08:49.368108+0000","last_peered":"2024-04-26T18:08:49.368108+0000","last_clean":"2024-04-26T18:08:49.368108+0000","last_became_active":"2024-04-26T18:08:47.359860+0000","last_became_peered":"2024-04-26T18:08:47.359860+0000","last_unstale":"2024-04-26T18:08:49.368108+0000","last_undegraded":"2024-04-26T18:08:49.368108+0000","last_fullsized":"2024-04-26T18:08:49.368108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:00.532589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:23.978267+0000","last_change":"2024-04-26T18:08:44.343942+0000","last_active":"2024-04-26T18:09:23.978267+0000","last_peered":"2024-04-26T18:09:23.978267+0000","last_clean":"2024-04-26T18:09:23.978267+0000","last_became_active":"2024-04-26T18:08:44.343581+0000","last_became_peered":"2024-04-26T18:08:44.343581+0000","last_unstale":"2024-04-26T18:09:23.978267+0000","last_undegraded":"2024-04-26T18:09:23.978267+0000","last_fullsized":"2024-04-26T18:09:23.978267+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:32.044284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361639+0000","last_change":"2024-04-26T18:08:47.372303+0000","last_active":"2024-04-26T18:08:49.361639+0000","last_peered":"2024-04-26T18:08:49.361639+0000","last_clean":"2024-04-26T18:08:49.361639+0000","last_became_active":"2024-04-26T18:08:47.371961+0000","last_became_peered":"2024-04-26T18:08:47.371961+0000","last_unstale":"2024-04-26T18:08:49.361639+0000","last_undegraded":"2024-04-26T18:08:49.361639+0000","last_fullsized":"2024-04-26T18:08:49.361639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:19.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932968+0000","last_change":"2024-04-26T18:08:44.345681+0000","last_active":"2024-04-26T18:09:00.932968+0000","last_peered":"2024-04-26T18:09:00.932968+0000","last_clean":"2024-04-26T18:09:00.932968+0000","last_became_active":"2024-04-26T18:08:44.345329+0000","last_became_peered":"2024-04-26T18:08:44.345329+0000","last_unstale":"2024-04-26T18:09:00.932968+0000","last_undegraded":"2024-04-26T18:09:00.932968+0000","last_fullsized":"2024-04-26T18:09:00.932968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:59.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372811+0000","last_change":"2024-04-26T18:08:47.370773+0000","last_active":"2024-04-26T18:08:49.372811+0000","last_peered":"2024-04-26T18:08:49.372811+0000","last_clean":"2024-04-26T18:08:49.372811+0000","last_became_active":"2024-04-26T18:08:47.370490+0000","last_became_peered":"2024-04-26T18:08:47.370490+0000","last_unstale":"2024-04-26T18:08:49.372811+0000","last_undegraded":"2024-04-26T18:08:49.372811+0000","last_fullsized":"2024-04-26T18:08:49.372811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:01.935335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932773+0000","last_change":"2024-04-26T18:08:44.555714+0000","last_active":"2024-04-26T18:09:00.932773+0000","last_peered":"2024-04-26T18:09:00.932773+0000","last_clean":"2024-04-26T18:09:00.932773+0000","last_became_active":"2024-04-26T18:08:44.555184+0000","last_became_peered":"2024-04-26T18:08:44.555184+0000","last_unstale":"2024-04-26T18:09:00.932773+0000","last_undegraded":"2024-04-26T18:09:00.932773+0000","last_fullsized":"2024-04-26T18:09:00.932773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:01.433764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361146+0000","last_change":"2024-04-26T18:08:47.378495+0000","last_active":"2024-04-26T18:08:49.361146+0000","last_peered":"2024-04-26T18:08:49.361146+0000","last_clean":"2024-04-26T18:08:49.361146+0000","last_became_active":"2024-04-26T18:08:47.377668+0000","last_became_peered":"2024-04-26T18:08:47.377668+0000","last_unstale":"2024-04-26T18:08:49.361146+0000","last_undegraded":"2024-04-26T18:08:49.361146+0000","last_fullsized":"2024-04-26T18:08:49.361146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:08.761695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'26","reported_seq":100,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:24.983735+0000","last_change":"2024-04-26T18:08:44.346346+0000","last_active":"2024-04-26T18:09:24.983735+0000","last_peered":"2024-04-26T18:09:24.983735+0000","last_clean":"2024-04-26T18:09:24.983735+0000","last_became_active":"2024-04-26T18:08:44.345999+0000","last_became_peered":"2024-04-26T18:08:44.345999+0000","last_unstale":"2024-04-26T18:09:24.983735+0000","last_undegraded":"2024-04-26T18:09:24.983735+0000","last_fullsized":"2024-04-26T18:09:24.983735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:27.277880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":52,"num_read_kb":78,"num_write":27,"num_write_kb":61,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367272+0000","last_change":"2024-04-26T18:08:47.366303+0000","last_active":"2024-04-26T18:08:49.367272+0000","last_peered":"2024-04-26T18:08:49.367272+0000","last_clean":"2024-04-26T18:08:49.367272+0000","last_became_active":"2024-04-26T18:08:47.365861+0000","last_became_peered":"2024-04-26T18:08:47.365861+0000","last_unstale":"2024-04-26T18:08:49.367272+0000","last_undegraded":"2024-04-26T18:08:49.367272+0000","last_fullsized":"2024-04-26T18:08:49.367272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:31.413589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.943532+0000","last_change":"2024-04-26T18:08:44.342752+0000","last_active":"2024-04-26T18:09:01.943532+0000","last_peered":"2024-04-26T18:09:01.943532+0000","last_clean":"2024-04-26T18:09:01.943532+0000","last_became_active":"2024-04-26T18:08:44.342432+0000","last_became_peered":"2024-04-26T18:08:44.342432+0000","last_unstale":"2024-04-26T18:09:01.943532+0000","last_undegraded":"2024-04-26T18:09:01.943532+0000","last_fullsized":"2024-04-26T18:09:01.943532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:17.974757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.362831+0000","last_change":"2024-04-26T18:08:47.367835+0000","last_active":"2024-04-26T18:08:49.362831+0000","last_peered":"2024-04-26T18:08:49.362831+0000","last_clean":"2024-04-26T18:08:49.362831+0000","last_became_active":"2024-04-26T18:08:47.367517+0000","last_became_peered":"2024-04-26T18:08:47.367517+0000","last_unstale":"2024-04-26T18:08:49.362831+0000","last_undegraded":"2024-04-26T18:08:49.362831+0000","last_fullsized":"2024-04-26T18:08:49.362831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:54.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932246+0000","last_change":"2024-04-26T18:08:44.344161+0000","last_active":"2024-04-26T18:09:00.932246+0000","last_peered":"2024-04-26T18:09:00.932246+0000","last_clean":"2024-04-26T18:09:00.932246+0000","last_became_active":"2024-04-26T18:08:44.343635+0000","last_became_peered":"2024-04-26T18:08:44.343635+0000","last_unstale":"2024-04-26T18:09:00.932246+0000","last_undegraded":"2024-04-26T18:09:00.932246+0000","last_fullsized":"2024-04-26T18:09:00.932246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.946584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371910+0000","last_change":"2024-04-26T18:08:47.374974+0000","last_active":"2024-04-26T18:08:49.371910+0000","last_peered":"2024-04-26T18:08:49.371910+0000","last_clean":"2024-04-26T18:08:49.371910+0000","last_became_active":"2024-04-26T18:08:47.374663+0000","last_became_peered":"2024-04-26T18:08:47.374663+0000","last_unstale":"2024-04-26T18:08:49.371910+0000","last_undegraded":"2024-04-26T18:08:49.371910+0000","last_fullsized":"2024-04-26T18:08:49.371910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:14:18.399512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:21.976572+0000","last_change":"2024-04-26T18:08:44.345278+0000","last_active":"2024-04-26T18:09:21.976572+0000","last_peered":"2024-04-26T18:09:21.976572+0000","last_clean":"2024-04-26T18:09:21.976572+0000","last_became_active":"2024-04-26T18:08:44.344531+0000","last_became_peered":"2024-04-26T18:08:44.344531+0000","last_unstale":"2024-04-26T18:09:21.976572+0000","last_undegraded":"2024-04-26T18:09:21.976572+0000","last_fullsized":"2024-04-26T18:09:21.976572+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:02.667622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373177+0000","last_change":"2024-04-26T18:08:47.369542+0000","last_active":"2024-04-26T18:08:49.373177+0000","last_peered":"2024-04-26T18:08:49.373177+0000","last_clean":"2024-04-26T18:08:49.373177+0000","last_became_active":"2024-04-26T18:08:47.369181+0000","last_became_peered":"2024-04-26T18:08:47.369181+0000","last_unstale":"2024-04-26T18:08:49.373177+0000","last_undegraded":"2024-04-26T18:08:49.373177+0000","last_fullsized":"2024-04-26T18:08:49.373177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:56.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.944838+0000","last_change":"2024-04-26T18:08:44.555580+0000","last_active":"2024-04-26T18:09:01.944838+0000","last_peered":"2024-04-26T18:09:01.944838+0000","last_clean":"2024-04-26T18:09:01.944838+0000","last_became_active":"2024-04-26T18:08:44.555247+0000","last_became_peered":"2024-04-26T18:08:44.555247+0000","last_unstale":"2024-04-26T18:09:01.944838+0000","last_undegraded":"2024-04-26T18:09:01.944838+0000","last_fullsized":"2024-04-26T18:09:01.944838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:58:55.207355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371041+0000","last_change":"2024-04-26T18:08:47.372300+0000","last_active":"2024-04-26T18:08:49.371041+0000","last_peered":"2024-04-26T18:08:49.371041+0000","last_clean":"2024-04-26T18:08:49.371041+0000","last_became_active":"2024-04-26T18:08:47.371952+0000","last_became_peered":"2024-04-26T18:08:47.371952+0000","last_unstale":"2024-04-26T18:08:49.371041+0000","last_undegraded":"2024-04-26T18:08:49.371041+0000","last_fullsized":"2024-04-26T18:08:49.371041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:58.254691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:02.939397+0000","last_change":"2024-04-26T18:08:44.340926+0000","last_active":"2024-04-26T18:09:02.939397+0000","last_peered":"2024-04-26T18:09:02.939397+0000","last_clean":"2024-04-26T18:09:02.939397+0000","last_became_active":"2024-04-26T18:08:44.340563+0000","last_became_peered":"2024-04-26T18:08:44.340563+0000","last_unstale":"2024-04-26T18:09:02.939397+0000","last_undegraded":"2024-04-26T18:09:02.939397+0000","last_fullsized":"2024-04-26T18:09:02.939397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:57.803017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361370+0000","last_change":"2024-04-26T18:08:47.372186+0000","last_active":"2024-04-26T18:08:49.361370+0000","last_peered":"2024-04-26T18:08:49.361370+0000","last_clean":"2024-04-26T18:08:49.361370+0000","last_became_active":"2024-04-26T18:08:47.371883+0000","last_became_peered":"2024-04-26T18:08:47.371883+0000","last_unstale":"2024-04-26T18:08:49.361370+0000","last_undegraded":"2024-04-26T18:08:49.361370+0000","last_fullsized":"2024-04-26T18:08:49.361370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:40.832127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368323+0000","last_change":"2024-04-26T18:08:44.555076+0000","last_active":"2024-04-26T18:08:49.368323+0000","last_peered":"2024-04-26T18:08:49.368323+0000","last_clean":"2024-04-26T18:08:49.368323+0000","last_became_active":"2024-04-26T18:08:44.554756+0000","last_became_peered":"2024-04-26T18:08:44.554756+0000","last_unstale":"2024-04-26T18:08:49.368323+0000","last_undegraded":"2024-04-26T18:08:49.368323+0000","last_fullsized":"2024-04-26T18:08:49.368323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:21.950644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:50.174999+0000","last_change":"2024-04-26T18:08:47.366370+0000","last_active":"2024-04-26T18:08:50.174999+0000","last_peered":"2024-04-26T18:08:50.174999+0000","last_clean":"2024-04-26T18:08:50.174999+0000","last_became_active":"2024-04-26T18:08:47.365997+0000","last_became_peered":"2024-04-26T18:08:47.365997+0000","last_unstale":"2024-04-26T18:08:50.174999+0000","last_undegraded":"2024-04-26T18:08:50.174999+0000","last_fullsized":"2024-04-26T18:08:50.174999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:51.643438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372919+0000","last_change":"2024-04-26T18:08:44.339299+0000","last_active":"2024-04-26T18:08:49.372919+0000","last_peered":"2024-04-26T18:08:49.372919+0000","last_clean":"2024-04-26T18:08:49.372919+0000","last_became_active":"2024-04-26T18:08:44.338932+0000","last_became_peered":"2024-04-26T18:08:44.338932+0000","last_unstale":"2024-04-26T18:08:49.372919+0000","last_undegraded":"2024-04-26T18:08:49.372919+0000","last_fullsized":"2024-04-26T18:08:49.372919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:18.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367241+0000","last_change":"2024-04-26T18:08:47.369924+0000","last_active":"2024-04-26T18:08:49.367241+0000","last_peered":"2024-04-26T18:08:49.367241+0000","last_clean":"2024-04-26T18:08:49.367241+0000","last_became_active":"2024-04-26T18:08:47.369048+0000","last_became_peered":"2024-04-26T18:08:47.369048+0000","last_unstale":"2024-04-26T18:08:49.367241+0000","last_undegraded":"2024-04-26T18:08:49.367241+0000","last_fullsized":"2024-04-26T18:08:49.367241+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:19.110288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.374101+0000","last_change":"2024-04-26T18:08:47.370836+0000","last_active":"2024-04-26T18:08:49.374101+0000","last_peered":"2024-04-26T18:08:49.374101+0000","last_clean":"2024-04-26T18:08:49.374101+0000","last_became_active":"2024-04-26T18:08:47.370160+0000","last_became_peered":"2024-04-26T18:08:47.370160+0000","last_unstale":"2024-04-26T18:08:49.374101+0000","last_undegraded":"2024-04-26T18:08:49.374101+0000","last_fullsized":"2024-04-26T18:08:49.374101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:11:21.285466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:22.977350+0000","last_change":"2024-04-26T18:08:44.554960+0000","last_active":"2024-04-26T18:09:22.977350+0000","last_peered":"2024-04-26T18:09:22.977350+0000","last_clean":"2024-04-26T18:09:22.977350+0000","last_became_active":"2024-04-26T18:08:44.554607+0000","last_became_peered":"2024-04-26T18:08:44.554607+0000","last_unstale":"2024-04-26T18:09:22.977350+0000","last_undegraded":"2024-04-26T18:09:22.977350+0000","last_fullsized":"2024-04-26T18:09:22.977350+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:22:26.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369255+0000","last_change":"2024-04-26T18:08:47.372528+0000","last_active":"2024-04-26T18:08:49.369255+0000","last_peered":"2024-04-26T18:08:49.369255+0000","last_clean":"2024-04-26T18:08:49.369255+0000","last_became_active":"2024-04-26T18:08:47.372215+0000","last_became_peered":"2024-04-26T18:08:47.372215+0000","last_unstale":"2024-04-26T18:08:49.369255+0000","last_undegraded":"2024-04-26T18:08:49.369255+0000","last_fullsized":"2024-04-26T18:08:49.369255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:39.366917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:03.947228+0000","last_change":"2024-04-26T18:08:44.342667+0000","last_active":"2024-04-26T18:09:03.947228+0000","last_peered":"2024-04-26T18:09:03.947228+0000","last_clean":"2024-04-26T18:09:03.947228+0000","last_became_active":"2024-04-26T18:08:44.342170+0000","last_became_peered":"2024-04-26T18:08:44.342170+0000","last_unstale":"2024-04-26T18:09:03.947228+0000","last_undegraded":"2024-04-26T18:09:03.947228+0000","last_fullsized":"2024-04-26T18:09:03.947228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:50.117177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373006+0000","last_change":"2024-04-26T18:07:24.173301+0000","last_active":"2024-04-26T18:08:49.373006+0000","last_peered":"2024-04-26T18:08:49.373006+0000","last_clean":"2024-04-26T18:08:49.373006+0000","last_became_active":"2024-04-26T18:07:24.172791+0000","last_became_peered":"2024-04-26T18:07:24.172791+0000","last_unstale":"2024-04-26T18:08:49.373006+0000","last_undegraded":"2024-04-26T18:08:49.373006+0000","last_fullsized":"2024-04-26T18:08:49.373006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_clean_scrub_stamp":"2024-04-26T18:03:58.528256+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:32.468624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.362720+0000","last_change":"2024-04-26T18:08:47.370720+0000","last_active":"2024-04-26T18:08:49.362720+0000","last_peered":"2024-04-26T18:08:49.362720+0000","last_clean":"2024-04-26T18:08:49.362720+0000","last_became_active":"2024-04-26T18:08:47.370414+0000","last_became_peered":"2024-04-26T18:08:47.370414+0000","last_unstale":"2024-04-26T18:08:49.362720+0000","last_undegraded":"2024-04-26T18:08:49.362720+0000","last_fullsized":"2024-04-26T18:08:49.362720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:37.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.944020+0000","last_change":"2024-04-26T18:08:44.347077+0000","last_active":"2024-04-26T18:09:01.944020+0000","last_peered":"2024-04-26T18:09:01.944020+0000","last_clean":"2024-04-26T18:09:01.944020+0000","last_became_active":"2024-04-26T18:08:44.346481+0000","last_became_peered":"2024-04-26T18:08:44.346481+0000","last_unstale":"2024-04-26T18:09:01.944020+0000","last_undegraded":"2024-04-26T18:09:01.944020+0000","last_fullsized":"2024-04-26T18:09:01.944020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:08.840814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.931780+0000","last_change":"2024-04-26T18:08:44.555658+0000","last_active":"2024-04-26T18:09:00.931780+0000","last_peered":"2024-04-26T18:09:00.931780+0000","last_clean":"2024-04-26T18:09:00.931780+0000","last_became_active":"2024-04-26T18:08:44.555303+0000","last_became_peered":"2024-04-26T18:08:44.555303+0000","last_unstale":"2024-04-26T18:09:00.931780+0000","last_undegraded":"2024-04-26T18:09:00.931780+0000","last_fullsized":"2024-04-26T18:09:00.931780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:17.781574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363145+0000","last_change":"2024-04-26T18:08:47.366552+0000","last_active":"2024-04-26T18:08:49.363145+0000","last_peered":"2024-04-26T18:08:49.363145+0000","last_clean":"2024-04-26T18:08:49.363145+0000","last_became_active":"2024-04-26T18:08:47.366183+0000","last_became_peered":"2024-04-26T18:08:47.366183+0000","last_unstale":"2024-04-26T18:08:49.363145+0000","last_undegraded":"2024-04-26T18:08:49.363145+0000","last_fullsized":"2024-04-26T18:08:49.363145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:36.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'11","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:21.370406+0000","last_change":"2024-04-26T18:08:44.338078+0000","last_active":"2024-04-26T18:09:21.370406+0000","last_peered":"2024-04-26T18:09:21.370406+0000","last_clean":"2024-04-26T18:09:21.370406+0000","last_became_active":"2024-04-26T18:08:44.337715+0000","last_became_peered":"2024-04-26T18:08:44.337715+0000","last_unstale":"2024-04-26T18:09:21.370406+0000","last_undegraded":"2024-04-26T18:09:21.370406+0000","last_fullsized":"2024-04-26T18:09:21.370406+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.912993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370867+0000","last_change":"2024-04-26T18:08:47.371321+0000","last_active":"2024-04-26T18:08:49.370867+0000","last_peered":"2024-04-26T18:08:49.370867+0000","last_clean":"2024-04-26T18:08:49.370867+0000","last_became_active":"2024-04-26T18:08:47.371015+0000","last_became_peered":"2024-04-26T18:08:47.371015+0000","last_unstale":"2024-04-26T18:08:49.370867+0000","last_undegraded":"2024-04-26T18:08:49.370867+0000","last_fullsized":"2024-04-26T18:08:49.370867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.931804+0000","last_change":"2024-04-26T18:08:44.420101+0000","last_active":"2024-04-26T18:09:00.931804+0000","last_peered":"2024-04-26T18:09:00.931804+0000","last_clean":"2024-04-26T18:09:00.931804+0000","last_became_active":"2024-04-26T18:08:44.419787+0000","last_became_peered":"2024-04-26T18:08:44.419787+0000","last_unstale":"2024-04-26T18:09:00.931804+0000","last_undegraded":"2024-04-26T18:09:00.931804+0000","last_fullsized":"2024-04-26T18:09:00.931804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.485425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372709+0000","last_change":"2024-04-26T18:08:47.372309+0000","last_active":"2024-04-26T18:08:49.372709+0000","last_peered":"2024-04-26T18:08:49.372709+0000","last_clean":"2024-04-26T18:08:49.372709+0000","last_became_active":"2024-04-26T18:08:47.371980+0000","last_became_peered":"2024-04-26T18:08:47.371980+0000","last_unstale":"2024-04-26T18:08:49.372709+0000","last_undegraded":"2024-04-26T18:08:49.372709+0000","last_fullsized":"2024-04-26T18:08:49.372709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:05.011077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.934567+0000","last_change":"2024-04-26T18:08:44.555500+0000","last_active":"2024-04-26T18:09:00.934567+0000","last_peered":"2024-04-26T18:09:00.934567+0000","last_clean":"2024-04-26T18:09:00.934567+0000","last_became_active":"2024-04-26T18:08:44.555137+0000","last_became_peered":"2024-04-26T18:08:44.555137+0000","last_unstale":"2024-04-26T18:09:00.934567+0000","last_undegraded":"2024-04-26T18:09:00.934567+0000","last_fullsized":"2024-04-26T18:09:00.934567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:10:05.723654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367295+0000","last_change":"2024-04-26T18:08:47.364215+0000","last_active":"2024-04-26T18:08:49.367295+0000","last_peered":"2024-04-26T18:08:49.367295+0000","last_clean":"2024-04-26T18:08:49.367295+0000","last_became_active":"2024-04-26T18:08:47.363862+0000","last_became_peered":"2024-04-26T18:08:47.363862+0000","last_unstale":"2024-04-26T18:08:49.367295+0000","last_undegraded":"2024-04-26T18:08:49.367295+0000","last_fullsized":"2024-04-26T18:08:49.367295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:27.020100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932170+0000","last_change":"2024-04-26T18:08:44.343506+0000","last_active":"2024-04-26T18:09:00.932170+0000","last_peered":"2024-04-26T18:09:00.932170+0000","last_clean":"2024-04-26T18:09:00.932170+0000","last_became_active":"2024-04-26T18:08:44.343163+0000","last_became_peered":"2024-04-26T18:08:44.343163+0000","last_unstale":"2024-04-26T18:09:00.932170+0000","last_undegraded":"2024-04-26T18:09:00.932170+0000","last_fullsized":"2024-04-26T18:09:00.932170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:09.941391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361106+0000","last_change":"2024-04-26T18:08:47.365572+0000","last_active":"2024-04-26T18:08:49.361106+0000","last_peered":"2024-04-26T18:08:49.361106+0000","last_clean":"2024-04-26T18:08:49.361106+0000","last_became_active":"2024-04-26T18:08:47.364827+0000","last_became_peered":"2024-04-26T18:08:47.364827+0000","last_unstale":"2024-04-26T18:08:49.361106+0000","last_undegraded":"2024-04-26T18:08:49.361106+0000","last_fullsized":"2024-04-26T18:08:49.361106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:44.492380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367863+0000","last_change":"2024-04-26T18:08:44.339063+0000","last_active":"2024-04-26T18:08:49.367863+0000","last_peered":"2024-04-26T18:08:49.367863+0000","last_clean":"2024-04-26T18:08:49.367863+0000","last_became_active":"2024-04-26T18:08:44.338752+0000","last_became_peered":"2024-04-26T18:08:44.338752+0000","last_unstale":"2024-04-26T18:08:49.367863+0000","last_undegraded":"2024-04-26T18:08:49.367863+0000","last_fullsized":"2024-04-26T18:08:49.367863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:38.201263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370385+0000","last_change":"2024-04-26T18:08:47.367859+0000","last_active":"2024-04-26T18:08:49.370385+0000","last_peered":"2024-04-26T18:08:49.370385+0000","last_clean":"2024-04-26T18:08:49.370385+0000","last_became_active":"2024-04-26T18:08:47.367548+0000","last_became_peered":"2024-04-26T18:08:47.367548+0000","last_unstale":"2024-04-26T18:08:49.370385+0000","last_undegraded":"2024-04-26T18:08:49.370385+0000","last_fullsized":"2024-04-26T18:08:49.370385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:13:27.612151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.934395+0000","last_change":"2024-04-26T18:08:44.345392+0000","last_active":"2024-04-26T18:09:00.934395+0000","last_peered":"2024-04-26T18:09:00.934395+0000","last_clean":"2024-04-26T18:09:00.934395+0000","last_became_active":"2024-04-26T18:08:44.344578+0000","last_became_peered":"2024-04-26T18:08:44.344578+0000","last_unstale":"2024-04-26T18:09:00.934395+0000","last_undegraded":"2024-04-26T18:09:00.934395+0000","last_fullsized":"2024-04-26T18:09:00.934395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:26.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373074+0000","last_change":"2024-04-26T18:08:47.380099+0000","last_active":"2024-04-26T18:08:49.373074+0000","last_peered":"2024-04-26T18:08:49.373074+0000","last_clean":"2024-04-26T18:08:49.373074+0000","last_became_active":"2024-04-26T18:08:47.379576+0000","last_became_peered":"2024-04-26T18:08:47.379576+0000","last_unstale":"2024-04-26T18:08:49.373074+0000","last_undegraded":"2024-04-26T18:08:49.373074+0000","last_fullsized":"2024-04-26T18:08:49.373074+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:28.397886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.942947+0000","last_change":"2024-04-26T18:08:44.555299+0000","last_active":"2024-04-26T18:09:01.942947+0000","last_peered":"2024-04-26T18:09:01.942947+0000","last_clean":"2024-04-26T18:09:01.942947+0000","last_became_active":"2024-04-26T18:08:44.554959+0000","last_became_peered":"2024-04-26T18:08:44.554959+0000","last_unstale":"2024-04-26T18:09:01.942947+0000","last_undegraded":"2024-04-26T18:09:01.942947+0000","last_fullsized":"2024-04-26T18:09:01.942947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.565073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372526+0000","last_change":"2024-04-26T18:08:47.377524+0000","last_active":"2024-04-26T18:08:49.372526+0000","last_peered":"2024-04-26T18:08:49.372526+0000","last_clean":"2024-04-26T18:08:49.372526+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:08:49.372526+0000","last_undegraded":"2024-04-26T18:08:49.372526+0000","last_fullsized":"2024-04-26T18:08:49.372526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:10.466615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.376024+0000","last_change":"2024-04-26T18:08:44.343894+0000","last_active":"2024-04-26T18:08:49.376024+0000","last_peered":"2024-04-26T18:08:49.376024+0000","last_clean":"2024-04-26T18:08:49.376024+0000","last_became_active":"2024-04-26T18:08:44.343153+0000","last_became_peered":"2024-04-26T18:08:44.343153+0000","last_unstale":"2024-04-26T18:08:49.376024+0000","last_undegraded":"2024-04-26T18:08:49.376024+0000","last_fullsized":"2024-04-26T18:08:49.376024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:09.914701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.421579+0000","last_change":"2024-04-26T18:08:47.371962+0000","last_active":"2024-04-26T18:08:49.421579+0000","last_peered":"2024-04-26T18:08:49.421579+0000","last_clean":"2024-04-26T18:08:49.421579+0000","last_became_active":"2024-04-26T18:08:47.371517+0000","last_became_peered":"2024-04-26T18:08:47.371517+0000","last_unstale":"2024-04-26T18:08:49.421579+0000","last_undegraded":"2024-04-26T18:08:49.421579+0000","last_fullsized":"2024-04-26T18:08:49.421579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:23.553409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367893+0000","last_change":"2024-04-26T18:08:44.337853+0000","last_active":"2024-04-26T18:08:49.367893+0000","last_peered":"2024-04-26T18:08:49.367893+0000","last_clean":"2024-04-26T18:08:49.367893+0000","last_became_active":"2024-04-26T18:08:44.337393+0000","last_became_peered":"2024-04-26T18:08:44.337393+0000","last_unstale":"2024-04-26T18:08:49.367893+0000","last_undegraded":"2024-04-26T18:08:49.367893+0000","last_fullsized":"2024-04-26T18:08:49.367893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:26.514821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372700+0000","last_change":"2024-04-26T18:08:47.370999+0000","last_active":"2024-04-26T18:08:49.372700+0000","last_peered":"2024-04-26T18:08:49.372700+0000","last_clean":"2024-04-26T18:08:49.372700+0000","last_became_active":"2024-04-26T18:08:47.370373+0000","last_became_peered":"2024-04-26T18:08:47.370373+0000","last_unstale":"2024-04-26T18:08:49.372700+0000","last_undegraded":"2024-04-26T18:08:49.372700+0000","last_fullsized":"2024-04-26T18:08:49.372700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:19.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361144+0000","last_change":"2024-04-26T18:08:47.365722+0000","last_active":"2024-04-26T18:08:49.361144+0000","last_peered":"2024-04-26T18:08:49.361144+0000","last_clean":"2024-04-26T18:08:49.361144+0000","last_became_active":"2024-04-26T18:08:47.364692+0000","last_became_peered":"2024-04-26T18:08:47.364692+0000","last_unstale":"2024-04-26T18:08:49.361144+0000","last_undegraded":"2024-04-26T18:08:49.361144+0000","last_fullsized":"2024-04-26T18:08:49.361144+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:02.158156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368007+0000","last_change":"2024-04-26T18:08:47.373886+0000","last_active":"2024-04-26T18:08:49.368007+0000","last_peered":"2024-04-26T18:08:49.368007+0000","last_clean":"2024-04-26T18:08:49.368007+0000","last_became_active":"2024-04-26T18:08:47.373391+0000","last_became_peered":"2024-04-26T18:08:47.373391+0000","last_unstale":"2024-04-26T18:08:49.368007+0000","last_undegraded":"2024-04-26T18:08:49.368007+0000","last_fullsized":"2024-04-26T18:08:49.368007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:48:59.280050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367774+0000","last_change":"2024-04-26T18:08:47.366160+0000","last_active":"2024-04-26T18:08:49.367774+0000","last_peered":"2024-04-26T18:08:49.367774+0000","last_clean":"2024-04-26T18:08:49.367774+0000","last_became_active":"2024-04-26T18:08:47.365811+0000","last_became_peered":"2024-04-26T18:08:47.365811+0000","last_unstale":"2024-04-26T18:08:49.367774+0000","last_undegraded":"2024-04-26T18:08:49.367774+0000","last_fullsized":"2024-04-26T18:08:49.367774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:32.124692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372335+0000","last_change":"2024-04-26T18:08:47.366078+0000","last_active":"2024-04-26T18:08:49.372335+0000","last_peered":"2024-04-26T18:08:49.372335+0000","last_clean":"2024-04-26T18:08:49.372335+0000","last_became_active":"2024-04-26T18:08:47.365779+0000","last_became_peered":"2024-04-26T18:08:47.365779+0000","last_unstale":"2024-04-26T18:08:49.372335+0000","last_undegraded":"2024-04-26T18:08:49.372335+0000","last_fullsized":"2024-04-26T18:08:49.372335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.127606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363238+0000","last_change":"2024-04-26T18:08:47.372996+0000","last_active":"2024-04-26T18:08:49.363238+0000","last_peered":"2024-04-26T18:08:49.363238+0000","last_clean":"2024-04-26T18:08:49.363238+0000","last_became_active":"2024-04-26T18:08:47.372687+0000","last_became_peered":"2024-04-26T18:08:47.372687+0000","last_unstale":"2024-04-26T18:08:49.363238+0000","last_undegraded":"2024-04-26T18:08:49.363238+0000","last_fullsized":"2024-04-26T18:08:49.363238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:13:33.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361390+0000","last_change":"2024-04-26T18:08:47.371076+0000","last_active":"2024-04-26T18:08:49.361390+0000","last_peered":"2024-04-26T18:08:49.361390+0000","last_clean":"2024-04-26T18:08:49.361390+0000","last_became_active":"2024-04-26T18:08:47.370688+0000","last_became_peered":"2024-04-26T18:08:47.370688+0000","last_unstale":"2024-04-26T18:08:49.361390+0000","last_undegraded":"2024-04-26T18:08:49.361390+0000","last_fullsized":"2024-04-26T18:08:49.361390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:21.355971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369199+0000","last_change":"2024-04-26T18:08:47.372392+0000","last_active":"2024-04-26T18:08:49.369199+0000","last_peered":"2024-04-26T18:08:49.369199+0000","last_clean":"2024-04-26T18:08:49.369199+0000","last_became_active":"2024-04-26T18:08:47.372020+0000","last_became_peered":"2024-04-26T18:08:47.372020+0000","last_unstale":"2024-04-26T18:08:49.369199+0000","last_undegraded":"2024-04-26T18:08:49.369199+0000","last_fullsized":"2024-04-26T18:08:49.369199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369495+0000","last_change":"2024-04-26T18:08:47.369571+0000","last_active":"2024-04-26T18:08:49.369495+0000","last_peered":"2024-04-26T18:08:49.369495+0000","last_clean":"2024-04-26T18:08:49.369495+0000","last_became_active":"2024-04-26T18:08:47.369165+0000","last_became_peered":"2024-04-26T18:08:47.369165+0000","last_unstale":"2024-04-26T18:08:49.369495+0000","last_undegraded":"2024-04-26T18:08:49.369495+0000","last_fullsized":"2024-04-26T18:08:49.369495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:15.869182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373658+0000","last_change":"2024-04-26T18:08:47.372355+0000","last_active":"2024-04-26T18:08:49.373658+0000","last_peered":"2024-04-26T18:08:49.373658+0000","last_clean":"2024-04-26T18:08:49.373658+0000","last_became_active":"2024-04-26T18:08:47.371810+0000","last_became_peered":"2024-04-26T18:08:47.371810+0000","last_unstale":"2024-04-26T18:08:49.373658+0000","last_undegraded":"2024-04-26T18:08:49.373658+0000","last_fullsized":"2024-04-26T18:08:49.373658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:44:22.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370438+0000","last_change":"2024-04-26T18:08:47.364042+0000","last_active":"2024-04-26T18:08:49.370438+0000","last_peered":"2024-04-26T18:08:49.370438+0000","last_clean":"2024-04-26T18:08:49.370438+0000","last_became_active":"2024-04-26T18:08:47.363727+0000","last_became_peered":"2024-04-26T18:08:47.363727+0000","last_unstale":"2024-04-26T18:08:49.370438+0000","last_undegraded":"2024-04-26T18:08:49.370438+0000","last_fullsized":"2024-04-26T18:08:49.370438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:37.595261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361368+0000","last_change":"2024-04-26T18:08:47.377535+0000","last_active":"2024-04-26T18:08:49.361368+0000","last_peered":"2024-04-26T18:08:49.361368+0000","last_clean":"2024-04-26T18:08:49.361368+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:08:49.361368+0000","last_undegraded":"2024-04-26T18:08:49.361368+0000","last_fullsized":"2024-04-26T18:08:49.361368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:22.895402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363119+0000","last_change":"2024-04-26T18:08:47.367313+0000","last_active":"2024-04-26T18:08:49.363119+0000","last_peered":"2024-04-26T18:08:49.363119+0000","last_clean":"2024-04-26T18:08:49.363119+0000","last_became_active":"2024-04-26T18:08:47.366971+0000","last_became_peered":"2024-04-26T18:08:47.366971+0000","last_unstale":"2024-04-26T18:08:49.363119+0000","last_undegraded":"2024-04-26T18:08:49.363119+0000","last_fullsized":"2024-04-26T18:08:49.363119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:14.518100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361072+0000","last_change":"2024-04-26T18:08:47.373004+0000","last_active":"2024-04-26T18:08:49.361072+0000","last_peered":"2024-04-26T18:08:49.361072+0000","last_clean":"2024-04-26T18:08:49.361072+0000","last_became_active":"2024-04-26T18:08:47.372641+0000","last_became_peered":"2024-04-26T18:08:47.372641+0000","last_unstale":"2024-04-26T18:08:49.361072+0000","last_undegraded":"2024-04-26T18:08:49.361072+0000","last_fullsized":"2024-04-26T18:08:49.361072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:02.018425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58759,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":155,"num_read_kb":141,"num_write":161,"num_write_kb":103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":172020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":472639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579931,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28648,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716504,"statfs":{"total":95995035648,"available":95965700096,"internally_reserved":0,"allocated":1671168,"data_stored":1118878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743457,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":472875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005094,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":472795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266731,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1065116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462961,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":472264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":522167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":473535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":528165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":472575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920826,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":473092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1601536,"data_stored":1064409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":50110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":56336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:09:30.530 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:09:30.771 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:31.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:30 smithi090 ceph-mon[29180]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:09:31.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:30 smithi090 ceph-mon[29180]: pgmap v319: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-26T18:09:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:30 smithi119 ceph-mon[32752]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:09:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:30 smithi119 ceph-mon[32752]: pgmap v319: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-26T18:09:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:30 smithi167 ceph-mon[32751]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:09:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:30 smithi167 ceph-mon[32751]: pgmap v319: 97 pgs: 97 active+clean; 634 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-26T18:09:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-26T18:09:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-26T18:09:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T18:09:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:09:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-26T18:09:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:31 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-26T18:09:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-26T18:09:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-26T18:09:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T18:09:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:09:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-26T18:09:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:31 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-26T18:09:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-26T18:09:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-26T18:09:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-26T18:09:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:09:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-04-26T18:09:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:31 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-26T18:09:32.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 -- 172.21.15.90:0/2698832172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 msgr2=0x7f5900102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:32.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 --2- 172.21.15.90:0/2698832172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f5900102990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f58f40099c0 tx=0x7f58f4009cd0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 -- 172.21.15.90:0/2698832172 shutdown_connections 2024-04-26T18:09:32.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 --2- 172.21.15.90:0/2698832172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f5900102990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 --2- 172.21.15.90:0/2698832172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5900107640 0x7f590010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 --2- 172.21.15.90:0/2698832172 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5900102f60 0x7f5900106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 -- 172.21.15.90:0/2698832172 >> 172.21.15.90:0/2698832172 conn(0x7f59000792d0 msgr2=0x7f59000796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:32.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 -- 172.21.15.90:0/2698832172 shutdown_connections 2024-04-26T18:09:32.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.345+0000 7f5906d84700 1 -- 172.21.15.90:0/2698832172 wait complete. 2024-04-26T18:09:32.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 Processor -- start 2024-04-26T18:09:32.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 -- start start 2024-04-26T18:09:32.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f590019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:32.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5900102f60 0x7f590019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5900107640 0x7f5900198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f59001124f0 con 0x7f59001025b0 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5900112370 con 0x7f5900102f60 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5906d84700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5900112670 con 0x7f5900107640 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f590019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f590019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56428/0 (socket says 172.21.15.90:56428) 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 -- 172.21.15.90:0/3452220337 learned_addr learned my addr 172.21.15.90:0/3452220337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f58fffff700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5900102f60 0x7f590019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5905321700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5900107640 0x7f5900198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 -- 172.21.15.90:0/3452220337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5900107640 msgr2=0x7f5900198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5900107640 0x7f5900198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 -- 172.21.15.90:0/3452220337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5900102f60 msgr2=0x7f590019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5900102f60 0x7f590019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.346+0000 7f5904b20700 1 -- 172.21.15.90:0/3452220337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5900199030 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.347+0000 7f5904b20700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f590019e6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098e0 tx=0x7f58f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.347+0000 7f58fdffb700 1 -- 172.21.15.90:0/3452220337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4009e60 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.347+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59001992c0 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.347+0000 7f58fdffb700 1 -- 172.21.15.90:0/3452220337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f4004810 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.347+0000 7f58fdffb700 1 -- 172.21.15.90:0/3452220337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4005290 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.347+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f590006bb50 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.348+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900199740 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.349+0000 7f58fdffb700 1 -- 172.21.15.90:0/3452220337 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f58f40049c0 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.350+0000 7f58fdffb700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f58f00a3cd0 0x7f58f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.350+0000 7f58fdffb700 1 -- 172.21.15.90:0/3452220337 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8153+0+0 (secure 0 0 0) 0x7f58f40d1400 con 0x7f59001025b0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.350+0000 7f58fffff700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f58f00a3cd0 0x7f58f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:32.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.350+0000 7f58fffff700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f58f00a3cd0 0x7f58f00a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5900101f80 tx=0x7f58ec010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:32.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.353+0000 7f58fdffb700 1 -- 172.21.15.90:0/3452220337 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f58f4013070 con 0x7f59001025b0 2024-04-26T18:09:32.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.552+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f590006be10 con 0x7f58f00a3cd0 2024-04-26T18:09:32.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.560+0000 7f58fdffb700 1 -- 172.21.15.90:0/3452220337 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258691 (secure 0 0 0) 0x7f590006be10 con 0x7f58f00a3cd0 2024-04-26T18:09:32.563 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:09:32.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f58f00a3cd0 msgr2=0x7f58f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:32.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f58f00a3cd0 0x7f58f00a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5900101f80 tx=0x7f58ec010040 comp rx=0 tx=0).stop 2024-04-26T18:09:32.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 msgr2=0x7f590019e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:32.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f590019e6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58f40098e0 tx=0x7f58f40045e0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 shutdown_connections 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f58f00a3cd0 0x7f58f00a6190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f59001025b0 0x7f590019e6d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5900107640 0x7f5900198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 --2- 172.21.15.90:0/3452220337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5900102f60 0x7f590019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 >> 172.21.15.90:0/3452220337 conn(0x7f59000792d0 msgr2=0x7f5900100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 shutdown_connections 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:32.564+0000 7f5906d84700 1 -- 172.21.15.90:0/3452220337 wait complete. 2024-04-26T18:09:32.566 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-26T18:09:33.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:33 smithi090 ceph-mon[29180]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:09:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-26T18:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-26T18:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-26T18:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T18:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-26T18:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-26T18:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:33 smithi119 ceph-mon[32752]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:09:33.311 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-26T18:09:32.002399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654756,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":322,"num_read_kb":281,"num_write":349,"num_write_kb":3051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":280,"ondisk_log_size":280,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":375220,"kb_used_data":13588,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124566604,"statfs":{"total":1151940427776,"available":1151556202496,"internally_reserved":0,"allocated":13914112,"data_stored":7618162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":41,"num_write":10,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.143621"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361491+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:08:49.361491+0000","last_peered":"2024-04-26T18:08:49.361491+0000","last_clean":"2024-04-26T18:08:49.361491+0000","last_became_active":"2024-04-26T18:08:47.377408+0000","last_became_peered":"2024-04-26T18:08:47.377408+0000","last_unstale":"2024-04-26T18:08:49.361491+0000","last_undegraded":"2024-04-26T18:08:49.361491+0000","last_fullsized":"2024-04-26T18:08:49.361491+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:13.822258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372467+0000","last_change":"2024-04-26T18:08:47.365125+0000","last_active":"2024-04-26T18:08:49.372467+0000","last_peered":"2024-04-26T18:08:49.372467+0000","last_clean":"2024-04-26T18:08:49.372467+0000","last_became_active":"2024-04-26T18:08:47.364847+0000","last_became_peered":"2024-04-26T18:08:47.364847+0000","last_unstale":"2024-04-26T18:08:49.372467+0000","last_undegraded":"2024-04-26T18:08:49.372467+0000","last_fullsized":"2024-04-26T18:08:49.372467+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:16:37.890893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372423+0000","last_change":"2024-04-26T18:08:47.377580+0000","last_active":"2024-04-26T18:08:49.372423+0000","last_peered":"2024-04-26T18:08:49.372423+0000","last_clean":"2024-04-26T18:08:49.372423+0000","last_became_active":"2024-04-26T18:08:47.377219+0000","last_became_peered":"2024-04-26T18:08:47.377219+0000","last_unstale":"2024-04-26T18:08:49.372423+0000","last_undegraded":"2024-04-26T18:08:49.372423+0000","last_fullsized":"2024-04-26T18:08:49.372423+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:12.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367582+0000","last_change":"2024-04-26T18:08:47.371628+0000","last_active":"2024-04-26T18:08:49.367582+0000","last_peered":"2024-04-26T18:08:49.367582+0000","last_clean":"2024-04-26T18:08:49.367582+0000","last_became_active":"2024-04-26T18:08:47.370840+0000","last_became_peered":"2024-04-26T18:08:47.370840+0000","last_unstale":"2024-04-26T18:08:49.367582+0000","last_undegraded":"2024-04-26T18:08:49.367582+0000","last_fullsized":"2024-04-26T18:08:49.367582+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:11.362664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367277+0000","last_change":"2024-04-26T18:08:47.367632+0000","last_active":"2024-04-26T18:08:49.367277+0000","last_peered":"2024-04-26T18:08:49.367277+0000","last_clean":"2024-04-26T18:08:49.367277+0000","last_became_active":"2024-04-26T18:08:47.367316+0000","last_became_peered":"2024-04-26T18:08:47.367316+0000","last_unstale":"2024-04-26T18:08:49.367277+0000","last_undegraded":"2024-04-26T18:08:49.367277+0000","last_fullsized":"2024-04-26T18:08:49.367277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:31:26.665292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367546+0000","last_change":"2024-04-26T18:08:47.369742+0000","last_active":"2024-04-26T18:08:49.367546+0000","last_peered":"2024-04-26T18:08:49.367546+0000","last_clean":"2024-04-26T18:08:49.367546+0000","last_became_active":"2024-04-26T18:08:47.369230+0000","last_became_peered":"2024-04-26T18:08:47.369230+0000","last_unstale":"2024-04-26T18:08:49.367546+0000","last_undegraded":"2024-04-26T18:08:49.367546+0000","last_fullsized":"2024-04-26T18:08:49.367546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:02:04.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373228+0000","last_change":"2024-04-26T18:08:47.370657+0000","last_active":"2024-04-26T18:08:49.373228+0000","last_peered":"2024-04-26T18:08:49.373228+0000","last_clean":"2024-04-26T18:08:49.373228+0000","last_became_active":"2024-04-26T18:08:47.370133+0000","last_became_peered":"2024-04-26T18:08:47.370133+0000","last_unstale":"2024-04-26T18:08:49.373228+0000","last_undegraded":"2024-04-26T18:08:49.373228+0000","last_fullsized":"2024-04-26T18:08:49.373228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:31.232179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369989+0000","last_change":"2024-04-26T18:08:47.366221+0000","last_active":"2024-04-26T18:08:49.369989+0000","last_peered":"2024-04-26T18:08:49.369989+0000","last_clean":"2024-04-26T18:08:49.369989+0000","last_became_active":"2024-04-26T18:08:47.365990+0000","last_became_peered":"2024-04-26T18:08:47.365990+0000","last_unstale":"2024-04-26T18:08:49.369989+0000","last_undegraded":"2024-04-26T18:08:49.369989+0000","last_fullsized":"2024-04-26T18:08:49.369989+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:43.707583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361759+0000","last_change":"2024-04-26T18:08:47.373165+0000","last_active":"2024-04-26T18:08:49.361759+0000","last_peered":"2024-04-26T18:08:49.361759+0000","last_clean":"2024-04-26T18:08:49.361759+0000","last_became_active":"2024-04-26T18:08:47.372688+0000","last_became_peered":"2024-04-26T18:08:47.372688+0000","last_unstale":"2024-04-26T18:08:49.361759+0000","last_undegraded":"2024-04-26T18:08:49.361759+0000","last_fullsized":"2024-04-26T18:08:49.361759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:18.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367900+0000","last_change":"2024-04-26T18:08:47.373963+0000","last_active":"2024-04-26T18:08:49.367900+0000","last_peered":"2024-04-26T18:08:49.367900+0000","last_clean":"2024-04-26T18:08:49.367900+0000","last_became_active":"2024-04-26T18:08:47.372878+0000","last_became_peered":"2024-04-26T18:08:47.372878+0000","last_unstale":"2024-04-26T18:08:49.367900+0000","last_undegraded":"2024-04-26T18:08:49.367900+0000","last_fullsized":"2024-04-26T18:08:49.367900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:25.178809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372276+0000","last_change":"2024-04-26T18:08:47.378643+0000","last_active":"2024-04-26T18:08:49.372276+0000","last_peered":"2024-04-26T18:08:49.372276+0000","last_clean":"2024-04-26T18:08:49.372276+0000","last_became_active":"2024-04-26T18:08:47.378329+0000","last_became_peered":"2024-04-26T18:08:47.378329+0000","last_unstale":"2024-04-26T18:08:49.372276+0000","last_undegraded":"2024-04-26T18:08:49.372276+0000","last_fullsized":"2024-04-26T18:08:49.372276+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:59.016025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367405+0000","last_change":"2024-04-26T18:08:47.371663+0000","last_active":"2024-04-26T18:08:49.367405+0000","last_peered":"2024-04-26T18:08:49.367405+0000","last_clean":"2024-04-26T18:08:49.367405+0000","last_became_active":"2024-04-26T18:08:47.371298+0000","last_became_peered":"2024-04-26T18:08:47.371298+0000","last_unstale":"2024-04-26T18:08:49.367405+0000","last_undegraded":"2024-04-26T18:08:49.367405+0000","last_fullsized":"2024-04-26T18:08:49.367405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:04.884698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363040+0000","last_change":"2024-04-26T18:08:47.365179+0000","last_active":"2024-04-26T18:08:49.363040+0000","last_peered":"2024-04-26T18:08:49.363040+0000","last_clean":"2024-04-26T18:08:49.363040+0000","last_became_active":"2024-04-26T18:08:47.364888+0000","last_became_peered":"2024-04-26T18:08:47.364888+0000","last_unstale":"2024-04-26T18:08:49.363040+0000","last_undegraded":"2024-04-26T18:08:49.363040+0000","last_fullsized":"2024-04-26T18:08:49.363040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:34.082855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372196+0000","last_change":"2024-04-26T18:08:47.364828+0000","last_active":"2024-04-26T18:08:49.372196+0000","last_peered":"2024-04-26T18:08:49.372196+0000","last_clean":"2024-04-26T18:08:49.372196+0000","last_became_active":"2024-04-26T18:08:47.364529+0000","last_became_peered":"2024-04-26T18:08:47.364529+0000","last_unstale":"2024-04-26T18:08:49.372196+0000","last_undegraded":"2024-04-26T18:08:49.372196+0000","last_fullsized":"2024-04-26T18:08:49.372196+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.480258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371662+0000","last_change":"2024-04-26T18:08:47.373578+0000","last_active":"2024-04-26T18:08:49.371662+0000","last_peered":"2024-04-26T18:08:49.371662+0000","last_clean":"2024-04-26T18:08:49.371662+0000","last_became_active":"2024-04-26T18:08:47.373263+0000","last_became_peered":"2024-04-26T18:08:47.373263+0000","last_unstale":"2024-04-26T18:08:49.371662+0000","last_undegraded":"2024-04-26T18:08:49.371662+0000","last_fullsized":"2024-04-26T18:08:49.371662+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:25.340241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.360993+0000","last_change":"2024-04-26T18:08:47.376617+0000","last_active":"2024-04-26T18:08:49.360993+0000","last_peered":"2024-04-26T18:08:49.360993+0000","last_clean":"2024-04-26T18:08:49.360993+0000","last_became_active":"2024-04-26T18:08:47.376290+0000","last_became_peered":"2024-04-26T18:08:47.376290+0000","last_unstale":"2024-04-26T18:08:49.360993+0000","last_undegraded":"2024-04-26T18:08:49.360993+0000","last_fullsized":"2024-04-26T18:08:49.360993+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:51.187583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361650+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:08:49.361650+0000","last_peered":"2024-04-26T18:08:49.361650+0000","last_clean":"2024-04-26T18:08:49.361650+0000","last_became_active":"2024-04-26T18:08:47.377427+0000","last_became_peered":"2024-04-26T18:08:47.377427+0000","last_unstale":"2024-04-26T18:08:49.361650+0000","last_undegraded":"2024-04-26T18:08:49.361650+0000","last_fullsized":"2024-04-26T18:08:49.361650+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.529500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369299+0000","last_change":"2024-04-26T18:08:47.373314+0000","last_active":"2024-04-26T18:08:49.369299+0000","last_peered":"2024-04-26T18:08:49.369299+0000","last_clean":"2024-04-26T18:08:49.369299+0000","last_became_active":"2024-04-26T18:08:47.372868+0000","last_became_peered":"2024-04-26T18:08:47.372868+0000","last_unstale":"2024-04-26T18:08:49.369299+0000","last_undegraded":"2024-04-26T18:08:49.369299+0000","last_fullsized":"2024-04-26T18:08:49.369299+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:19.642655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368630+0000","last_change":"2024-04-26T18:08:47.367593+0000","last_active":"2024-04-26T18:08:49.368630+0000","last_peered":"2024-04-26T18:08:49.368630+0000","last_clean":"2024-04-26T18:08:49.368630+0000","last_became_active":"2024-04-26T18:08:47.367243+0000","last_became_peered":"2024-04-26T18:08:47.367243+0000","last_unstale":"2024-04-26T18:08:49.368630+0000","last_undegraded":"2024-04-26T18:08:49.368630+0000","last_fullsized":"2024-04-26T18:08:49.368630+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.380307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.944535+0000","last_change":"2024-04-26T18:08:44.339625+0000","last_active":"2024-04-26T18:09:01.944535+0000","last_peered":"2024-04-26T18:09:01.944535+0000","last_clean":"2024-04-26T18:09:01.944535+0000","last_became_active":"2024-04-26T18:08:44.339209+0000","last_became_peered":"2024-04-26T18:08:44.339209+0000","last_unstale":"2024-04-26T18:09:01.944535+0000","last_undegraded":"2024-04-26T18:09:01.944535+0000","last_fullsized":"2024-04-26T18:09:01.944535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:43.169885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369176+0000","last_change":"2024-04-26T18:08:47.375884+0000","last_active":"2024-04-26T18:08:49.369176+0000","last_peered":"2024-04-26T18:08:49.369176+0000","last_clean":"2024-04-26T18:08:49.369176+0000","last_became_active":"2024-04-26T18:08:47.375557+0000","last_became_peered":"2024-04-26T18:08:47.375557+0000","last_unstale":"2024-04-26T18:08:49.369176+0000","last_undegraded":"2024-04-26T18:08:49.369176+0000","last_fullsized":"2024-04-26T18:08:49.369176+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.593910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.376172+0000","last_change":"2024-04-26T18:08:44.557167+0000","last_active":"2024-04-26T18:08:49.376172+0000","last_peered":"2024-04-26T18:08:49.376172+0000","last_clean":"2024-04-26T18:08:49.376172+0000","last_became_active":"2024-04-26T18:08:44.556716+0000","last_became_peered":"2024-04-26T18:08:44.556716+0000","last_unstale":"2024-04-26T18:08:49.376172+0000","last_undegraded":"2024-04-26T18:08:49.376172+0000","last_fullsized":"2024-04-26T18:08:49.376172+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372636+0000","last_change":"2024-04-26T18:08:47.377612+0000","last_active":"2024-04-26T18:08:49.372636+0000","last_peered":"2024-04-26T18:08:49.372636+0000","last_clean":"2024-04-26T18:08:49.372636+0000","last_became_active":"2024-04-26T18:08:47.377231+0000","last_became_peered":"2024-04-26T18:08:47.377231+0000","last_unstale":"2024-04-26T18:08:49.372636+0000","last_undegraded":"2024-04-26T18:08:49.372636+0000","last_fullsized":"2024-04-26T18:08:49.372636+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:11.594373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.943298+0000","last_change":"2024-04-26T18:08:44.343429+0000","last_active":"2024-04-26T18:09:01.943298+0000","last_peered":"2024-04-26T18:09:01.943298+0000","last_clean":"2024-04-26T18:09:01.943298+0000","last_became_active":"2024-04-26T18:08:44.343103+0000","last_became_peered":"2024-04-26T18:08:44.343103+0000","last_unstale":"2024-04-26T18:09:01.943298+0000","last_undegraded":"2024-04-26T18:09:01.943298+0000","last_fullsized":"2024-04-26T18:09:01.943298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:52.244159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361146+0000","last_change":"2024-04-26T18:08:47.373130+0000","last_active":"2024-04-26T18:08:49.361146+0000","last_peered":"2024-04-26T18:08:49.361146+0000","last_clean":"2024-04-26T18:08:49.361146+0000","last_became_active":"2024-04-26T18:08:47.372726+0000","last_became_peered":"2024-04-26T18:08:47.372726+0000","last_unstale":"2024-04-26T18:08:49.361146+0000","last_undegraded":"2024-04-26T18:08:49.361146+0000","last_fullsized":"2024-04-26T18:08:49.361146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:44.461177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.377157+0000","last_change":"2024-04-26T18:08:44.340152+0000","last_active":"2024-04-26T18:08:49.377157+0000","last_peered":"2024-04-26T18:08:49.377157+0000","last_clean":"2024-04-26T18:08:49.377157+0000","last_became_active":"2024-04-26T18:08:44.339684+0000","last_became_peered":"2024-04-26T18:08:44.339684+0000","last_unstale":"2024-04-26T18:08:49.377157+0000","last_undegraded":"2024-04-26T18:08:49.377157+0000","last_fullsized":"2024-04-26T18:08:49.377157+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:49.961201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371953+0000","last_change":"2024-04-26T18:08:47.375387+0000","last_active":"2024-04-26T18:08:49.371953+0000","last_peered":"2024-04-26T18:08:49.371953+0000","last_clean":"2024-04-26T18:08:49.371953+0000","last_became_active":"2024-04-26T18:08:47.374689+0000","last_became_peered":"2024-04-26T18:08:47.374689+0000","last_unstale":"2024-04-26T18:08:49.371953+0000","last_undegraded":"2024-04-26T18:08:49.371953+0000","last_fullsized":"2024-04-26T18:08:49.371953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:05.618804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:26.983111+0000","last_change":"2024-04-26T18:08:44.344167+0000","last_active":"2024-04-26T18:09:26.983111+0000","last_peered":"2024-04-26T18:09:26.983111+0000","last_clean":"2024-04-26T18:09:26.983111+0000","last_became_active":"2024-04-26T18:08:44.343646+0000","last_became_peered":"2024-04-26T18:08:44.343646+0000","last_unstale":"2024-04-26T18:09:26.983111+0000","last_undegraded":"2024-04-26T18:09:26.983111+0000","last_fullsized":"2024-04-26T18:09:26.983111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:53.859483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367042+0000","last_change":"2024-04-26T18:08:47.370546+0000","last_active":"2024-04-26T18:08:49.367042+0000","last_peered":"2024-04-26T18:08:49.367042+0000","last_clean":"2024-04-26T18:08:49.367042+0000","last_became_active":"2024-04-26T18:08:47.370220+0000","last_became_peered":"2024-04-26T18:08:47.370220+0000","last_unstale":"2024-04-26T18:08:49.367042+0000","last_undegraded":"2024-04-26T18:08:49.367042+0000","last_fullsized":"2024-04-26T18:08:49.367042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:05.139824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.943848+0000","last_change":"2024-04-26T18:08:44.347031+0000","last_active":"2024-04-26T18:09:01.943848+0000","last_peered":"2024-04-26T18:09:01.943848+0000","last_clean":"2024-04-26T18:09:01.943848+0000","last_became_active":"2024-04-26T18:08:44.346650+0000","last_became_peered":"2024-04-26T18:08:44.346650+0000","last_unstale":"2024-04-26T18:09:01.943848+0000","last_undegraded":"2024-04-26T18:09:01.943848+0000","last_fullsized":"2024-04-26T18:09:01.943848+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:50.946449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373824+0000","last_change":"2024-04-26T18:08:47.374230+0000","last_active":"2024-04-26T18:08:49.373824+0000","last_peered":"2024-04-26T18:08:49.373824+0000","last_clean":"2024-04-26T18:08:49.373824+0000","last_became_active":"2024-04-26T18:08:47.370780+0000","last_became_peered":"2024-04-26T18:08:47.370780+0000","last_unstale":"2024-04-26T18:08:49.373824+0000","last_undegraded":"2024-04-26T18:08:49.373824+0000","last_fullsized":"2024-04-26T18:08:49.373824+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:01.171195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.942323+0000","last_change":"2024-04-26T18:08:44.338318+0000","last_active":"2024-04-26T18:09:01.942323+0000","last_peered":"2024-04-26T18:09:01.942323+0000","last_clean":"2024-04-26T18:09:01.942323+0000","last_became_active":"2024-04-26T18:08:44.338009+0000","last_became_peered":"2024-04-26T18:08:44.338009+0000","last_unstale":"2024-04-26T18:09:01.942323+0000","last_undegraded":"2024-04-26T18:09:01.942323+0000","last_fullsized":"2024-04-26T18:09:01.942323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:33.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370158+0000","last_change":"2024-04-26T18:08:47.363924+0000","last_active":"2024-04-26T18:08:49.370158+0000","last_peered":"2024-04-26T18:08:49.370158+0000","last_clean":"2024-04-26T18:08:49.370158+0000","last_became_active":"2024-04-26T18:08:47.363616+0000","last_became_peered":"2024-04-26T18:08:47.363616+0000","last_unstale":"2024-04-26T18:08:49.370158+0000","last_undegraded":"2024-04-26T18:08:49.370158+0000","last_fullsized":"2024-04-26T18:08:49.370158+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:27:08.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373675+0000","last_change":"2024-04-26T18:08:44.343350+0000","last_active":"2024-04-26T18:08:49.373675+0000","last_peered":"2024-04-26T18:08:49.373675+0000","last_clean":"2024-04-26T18:08:49.373675+0000","last_became_active":"2024-04-26T18:08:44.343040+0000","last_became_peered":"2024-04-26T18:08:44.343040+0000","last_unstale":"2024-04-26T18:08:49.373675+0000","last_undegraded":"2024-04-26T18:08:49.373675+0000","last_fullsized":"2024-04-26T18:08:49.373675+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:09.427785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368108+0000","last_change":"2024-04-26T18:08:47.360180+0000","last_active":"2024-04-26T18:08:49.368108+0000","last_peered":"2024-04-26T18:08:49.368108+0000","last_clean":"2024-04-26T18:08:49.368108+0000","last_became_active":"2024-04-26T18:08:47.359860+0000","last_became_peered":"2024-04-26T18:08:47.359860+0000","last_unstale":"2024-04-26T18:08:49.368108+0000","last_undegraded":"2024-04-26T18:08:49.368108+0000","last_fullsized":"2024-04-26T18:08:49.368108+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:00.532589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'3","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:28.985394+0000","last_change":"2024-04-26T18:08:44.343942+0000","last_active":"2024-04-26T18:09:28.985394+0000","last_peered":"2024-04-26T18:09:28.985394+0000","last_clean":"2024-04-26T18:09:28.985394+0000","last_became_active":"2024-04-26T18:08:44.343581+0000","last_became_peered":"2024-04-26T18:08:44.343581+0000","last_unstale":"2024-04-26T18:09:28.985394+0000","last_undegraded":"2024-04-26T18:09:28.985394+0000","last_fullsized":"2024-04-26T18:09:28.985394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:32.044284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361639+0000","last_change":"2024-04-26T18:08:47.372303+0000","last_active":"2024-04-26T18:08:49.361639+0000","last_peered":"2024-04-26T18:08:49.361639+0000","last_clean":"2024-04-26T18:08:49.361639+0000","last_became_active":"2024-04-26T18:08:47.371961+0000","last_became_peered":"2024-04-26T18:08:47.371961+0000","last_unstale":"2024-04-26T18:08:49.361639+0000","last_undegraded":"2024-04-26T18:08:49.361639+0000","last_fullsized":"2024-04-26T18:08:49.361639+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:19.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932968+0000","last_change":"2024-04-26T18:08:44.345681+0000","last_active":"2024-04-26T18:09:00.932968+0000","last_peered":"2024-04-26T18:09:00.932968+0000","last_clean":"2024-04-26T18:09:00.932968+0000","last_became_active":"2024-04-26T18:08:44.345329+0000","last_became_peered":"2024-04-26T18:08:44.345329+0000","last_unstale":"2024-04-26T18:09:00.932968+0000","last_undegraded":"2024-04-26T18:09:00.932968+0000","last_fullsized":"2024-04-26T18:09:00.932968+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:59.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372811+0000","last_change":"2024-04-26T18:08:47.370773+0000","last_active":"2024-04-26T18:08:49.372811+0000","last_peered":"2024-04-26T18:08:49.372811+0000","last_clean":"2024-04-26T18:08:49.372811+0000","last_became_active":"2024-04-26T18:08:47.370490+0000","last_became_peered":"2024-04-26T18:08:47.370490+0000","last_unstale":"2024-04-26T18:08:49.372811+0000","last_undegraded":"2024-04-26T18:08:49.372811+0000","last_fullsized":"2024-04-26T18:08:49.372811+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:01.935335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932773+0000","last_change":"2024-04-26T18:08:44.555714+0000","last_active":"2024-04-26T18:09:00.932773+0000","last_peered":"2024-04-26T18:09:00.932773+0000","last_clean":"2024-04-26T18:09:00.932773+0000","last_became_active":"2024-04-26T18:08:44.555184+0000","last_became_peered":"2024-04-26T18:08:44.555184+0000","last_unstale":"2024-04-26T18:09:00.932773+0000","last_undegraded":"2024-04-26T18:09:00.932773+0000","last_fullsized":"2024-04-26T18:09:00.932773+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:01.433764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361146+0000","last_change":"2024-04-26T18:08:47.378495+0000","last_active":"2024-04-26T18:08:49.361146+0000","last_peered":"2024-04-26T18:08:49.361146+0000","last_clean":"2024-04-26T18:08:49.361146+0000","last_became_active":"2024-04-26T18:08:47.377668+0000","last_became_peered":"2024-04-26T18:08:47.377668+0000","last_unstale":"2024-04-26T18:08:49.361146+0000","last_undegraded":"2024-04-26T18:08:49.361146+0000","last_fullsized":"2024-04-26T18:08:49.361146+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:28:08.761695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'29","reported_seq":114,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:29.991444+0000","last_change":"2024-04-26T18:08:44.346346+0000","last_active":"2024-04-26T18:09:29.991444+0000","last_peered":"2024-04-26T18:09:29.991444+0000","last_clean":"2024-04-26T18:09:29.991444+0000","last_became_active":"2024-04-26T18:08:44.345999+0000","last_became_peered":"2024-04-26T18:08:44.345999+0000","last_unstale":"2024-04-26T18:09:29.991444+0000","last_undegraded":"2024-04-26T18:09:29.991444+0000","last_fullsized":"2024-04-26T18:09:29.991444+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:27.277880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59762,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":63,"num_read_kb":89,"num_write":30,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367272+0000","last_change":"2024-04-26T18:08:47.366303+0000","last_active":"2024-04-26T18:08:49.367272+0000","last_peered":"2024-04-26T18:08:49.367272+0000","last_clean":"2024-04-26T18:08:49.367272+0000","last_became_active":"2024-04-26T18:08:47.365861+0000","last_became_peered":"2024-04-26T18:08:47.365861+0000","last_unstale":"2024-04-26T18:08:49.367272+0000","last_undegraded":"2024-04-26T18:08:49.367272+0000","last_fullsized":"2024-04-26T18:08:49.367272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:31.413589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.943532+0000","last_change":"2024-04-26T18:08:44.342752+0000","last_active":"2024-04-26T18:09:01.943532+0000","last_peered":"2024-04-26T18:09:01.943532+0000","last_clean":"2024-04-26T18:09:01.943532+0000","last_became_active":"2024-04-26T18:08:44.342432+0000","last_became_peered":"2024-04-26T18:08:44.342432+0000","last_unstale":"2024-04-26T18:09:01.943532+0000","last_undegraded":"2024-04-26T18:09:01.943532+0000","last_fullsized":"2024-04-26T18:09:01.943532+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:17.974757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.362831+0000","last_change":"2024-04-26T18:08:47.367835+0000","last_active":"2024-04-26T18:08:49.362831+0000","last_peered":"2024-04-26T18:08:49.362831+0000","last_clean":"2024-04-26T18:08:49.362831+0000","last_became_active":"2024-04-26T18:08:47.367517+0000","last_became_peered":"2024-04-26T18:08:47.367517+0000","last_unstale":"2024-04-26T18:08:49.362831+0000","last_undegraded":"2024-04-26T18:08:49.362831+0000","last_fullsized":"2024-04-26T18:08:49.362831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:54.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932246+0000","last_change":"2024-04-26T18:08:44.344161+0000","last_active":"2024-04-26T18:09:00.932246+0000","last_peered":"2024-04-26T18:09:00.932246+0000","last_clean":"2024-04-26T18:09:00.932246+0000","last_became_active":"2024-04-26T18:08:44.343635+0000","last_became_peered":"2024-04-26T18:08:44.343635+0000","last_unstale":"2024-04-26T18:09:00.932246+0000","last_undegraded":"2024-04-26T18:09:00.932246+0000","last_fullsized":"2024-04-26T18:09:00.932246+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.946584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371910+0000","last_change":"2024-04-26T18:08:47.374974+0000","last_active":"2024-04-26T18:08:49.371910+0000","last_peered":"2024-04-26T18:08:49.371910+0000","last_clean":"2024-04-26T18:08:49.371910+0000","last_became_active":"2024-04-26T18:08:47.374663+0000","last_became_peered":"2024-04-26T18:08:47.374663+0000","last_unstale":"2024-04-26T18:08:49.371910+0000","last_undegraded":"2024-04-26T18:08:49.371910+0000","last_fullsized":"2024-04-26T18:08:49.371910+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:14:18.399512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'4","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:26.983445+0000","last_change":"2024-04-26T18:08:44.345278+0000","last_active":"2024-04-26T18:09:26.983445+0000","last_peered":"2024-04-26T18:09:26.983445+0000","last_clean":"2024-04-26T18:09:26.983445+0000","last_became_active":"2024-04-26T18:08:44.344531+0000","last_became_peered":"2024-04-26T18:08:44.344531+0000","last_unstale":"2024-04-26T18:09:26.983445+0000","last_undegraded":"2024-04-26T18:09:26.983445+0000","last_fullsized":"2024-04-26T18:09:26.983445+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:02.667622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373177+0000","last_change":"2024-04-26T18:08:47.369542+0000","last_active":"2024-04-26T18:08:49.373177+0000","last_peered":"2024-04-26T18:08:49.373177+0000","last_clean":"2024-04-26T18:08:49.373177+0000","last_became_active":"2024-04-26T18:08:47.369181+0000","last_became_peered":"2024-04-26T18:08:47.369181+0000","last_unstale":"2024-04-26T18:08:49.373177+0000","last_undegraded":"2024-04-26T18:08:49.373177+0000","last_fullsized":"2024-04-26T18:08:49.373177+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:56.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.944838+0000","last_change":"2024-04-26T18:08:44.555580+0000","last_active":"2024-04-26T18:09:01.944838+0000","last_peered":"2024-04-26T18:09:01.944838+0000","last_clean":"2024-04-26T18:09:01.944838+0000","last_became_active":"2024-04-26T18:08:44.555247+0000","last_became_peered":"2024-04-26T18:08:44.555247+0000","last_unstale":"2024-04-26T18:09:01.944838+0000","last_undegraded":"2024-04-26T18:09:01.944838+0000","last_fullsized":"2024-04-26T18:09:01.944838+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:58:55.207355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.371041+0000","last_change":"2024-04-26T18:08:47.372300+0000","last_active":"2024-04-26T18:08:49.371041+0000","last_peered":"2024-04-26T18:08:49.371041+0000","last_clean":"2024-04-26T18:08:49.371041+0000","last_became_active":"2024-04-26T18:08:47.371952+0000","last_became_peered":"2024-04-26T18:08:47.371952+0000","last_unstale":"2024-04-26T18:08:49.371041+0000","last_undegraded":"2024-04-26T18:08:49.371041+0000","last_fullsized":"2024-04-26T18:08:49.371041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:58.254691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:02.939397+0000","last_change":"2024-04-26T18:08:44.340926+0000","last_active":"2024-04-26T18:09:02.939397+0000","last_peered":"2024-04-26T18:09:02.939397+0000","last_clean":"2024-04-26T18:09:02.939397+0000","last_became_active":"2024-04-26T18:08:44.340563+0000","last_became_peered":"2024-04-26T18:08:44.340563+0000","last_unstale":"2024-04-26T18:09:02.939397+0000","last_undegraded":"2024-04-26T18:09:02.939397+0000","last_fullsized":"2024-04-26T18:09:02.939397+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:57.803017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361370+0000","last_change":"2024-04-26T18:08:47.372186+0000","last_active":"2024-04-26T18:08:49.361370+0000","last_peered":"2024-04-26T18:08:49.361370+0000","last_clean":"2024-04-26T18:08:49.361370+0000","last_became_active":"2024-04-26T18:08:47.371883+0000","last_became_peered":"2024-04-26T18:08:47.371883+0000","last_unstale":"2024-04-26T18:08:49.361370+0000","last_undegraded":"2024-04-26T18:08:49.361370+0000","last_fullsized":"2024-04-26T18:08:49.361370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:40.832127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368323+0000","last_change":"2024-04-26T18:08:44.555076+0000","last_active":"2024-04-26T18:08:49.368323+0000","last_peered":"2024-04-26T18:08:49.368323+0000","last_clean":"2024-04-26T18:08:49.368323+0000","last_became_active":"2024-04-26T18:08:44.554756+0000","last_became_peered":"2024-04-26T18:08:44.554756+0000","last_unstale":"2024-04-26T18:08:49.368323+0000","last_undegraded":"2024-04-26T18:08:49.368323+0000","last_fullsized":"2024-04-26T18:08:49.368323+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:21.950644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:50.174999+0000","last_change":"2024-04-26T18:08:47.366370+0000","last_active":"2024-04-26T18:08:50.174999+0000","last_peered":"2024-04-26T18:08:50.174999+0000","last_clean":"2024-04-26T18:08:50.174999+0000","last_became_active":"2024-04-26T18:08:47.365997+0000","last_became_peered":"2024-04-26T18:08:47.365997+0000","last_unstale":"2024-04-26T18:08:50.174999+0000","last_undegraded":"2024-04-26T18:08:50.174999+0000","last_fullsized":"2024-04-26T18:08:50.174999+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:51.643438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372919+0000","last_change":"2024-04-26T18:08:44.339299+0000","last_active":"2024-04-26T18:08:49.372919+0000","last_peered":"2024-04-26T18:08:49.372919+0000","last_clean":"2024-04-26T18:08:49.372919+0000","last_became_active":"2024-04-26T18:08:44.338932+0000","last_became_peered":"2024-04-26T18:08:44.338932+0000","last_unstale":"2024-04-26T18:08:49.372919+0000","last_undegraded":"2024-04-26T18:08:49.372919+0000","last_fullsized":"2024-04-26T18:08:49.372919+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:18.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367241+0000","last_change":"2024-04-26T18:08:47.369924+0000","last_active":"2024-04-26T18:08:49.367241+0000","last_peered":"2024-04-26T18:08:49.367241+0000","last_clean":"2024-04-26T18:08:49.367241+0000","last_became_active":"2024-04-26T18:08:47.369048+0000","last_became_peered":"2024-04-26T18:08:47.369048+0000","last_unstale":"2024-04-26T18:08:49.367241+0000","last_undegraded":"2024-04-26T18:08:49.367241+0000","last_fullsized":"2024-04-26T18:08:49.367241+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:19.110288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.374101+0000","last_change":"2024-04-26T18:08:47.370836+0000","last_active":"2024-04-26T18:08:49.374101+0000","last_peered":"2024-04-26T18:08:49.374101+0000","last_clean":"2024-04-26T18:08:49.374101+0000","last_became_active":"2024-04-26T18:08:47.370160+0000","last_became_peered":"2024-04-26T18:08:47.370160+0000","last_unstale":"2024-04-26T18:08:49.374101+0000","last_undegraded":"2024-04-26T18:08:49.374101+0000","last_fullsized":"2024-04-26T18:08:49.374101+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:11:21.285466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'4","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:27.984245+0000","last_change":"2024-04-26T18:08:44.554960+0000","last_active":"2024-04-26T18:09:27.984245+0000","last_peered":"2024-04-26T18:09:27.984245+0000","last_clean":"2024-04-26T18:09:27.984245+0000","last_became_active":"2024-04-26T18:08:44.554607+0000","last_became_peered":"2024-04-26T18:08:44.554607+0000","last_unstale":"2024-04-26T18:09:27.984245+0000","last_undegraded":"2024-04-26T18:09:27.984245+0000","last_fullsized":"2024-04-26T18:09:27.984245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:22:26.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369255+0000","last_change":"2024-04-26T18:08:47.372528+0000","last_active":"2024-04-26T18:08:49.369255+0000","last_peered":"2024-04-26T18:08:49.369255+0000","last_clean":"2024-04-26T18:08:49.369255+0000","last_became_active":"2024-04-26T18:08:47.372215+0000","last_became_peered":"2024-04-26T18:08:47.372215+0000","last_unstale":"2024-04-26T18:08:49.369255+0000","last_undegraded":"2024-04-26T18:08:49.369255+0000","last_fullsized":"2024-04-26T18:08:49.369255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:39.366917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:03.947228+0000","last_change":"2024-04-26T18:08:44.342667+0000","last_active":"2024-04-26T18:09:03.947228+0000","last_peered":"2024-04-26T18:09:03.947228+0000","last_clean":"2024-04-26T18:09:03.947228+0000","last_became_active":"2024-04-26T18:08:44.342170+0000","last_became_peered":"2024-04-26T18:08:44.342170+0000","last_unstale":"2024-04-26T18:09:03.947228+0000","last_undegraded":"2024-04-26T18:09:03.947228+0000","last_fullsized":"2024-04-26T18:09:03.947228+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:50.117177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":357,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373006+0000","last_change":"2024-04-26T18:07:24.173301+0000","last_active":"2024-04-26T18:08:49.373006+0000","last_peered":"2024-04-26T18:08:49.373006+0000","last_clean":"2024-04-26T18:08:49.373006+0000","last_became_active":"2024-04-26T18:07:24.172791+0000","last_became_peered":"2024-04-26T18:07:24.172791+0000","last_unstale":"2024-04-26T18:08:49.373006+0000","last_undegraded":"2024-04-26T18:08:49.373006+0000","last_fullsized":"2024-04-26T18:08:49.373006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_clean_scrub_stamp":"2024-04-26T18:03:58.528256+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:32.468624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.362720+0000","last_change":"2024-04-26T18:08:47.370720+0000","last_active":"2024-04-26T18:08:49.362720+0000","last_peered":"2024-04-26T18:08:49.362720+0000","last_clean":"2024-04-26T18:08:49.362720+0000","last_became_active":"2024-04-26T18:08:47.370414+0000","last_became_peered":"2024-04-26T18:08:47.370414+0000","last_unstale":"2024-04-26T18:08:49.362720+0000","last_undegraded":"2024-04-26T18:08:49.362720+0000","last_fullsized":"2024-04-26T18:08:49.362720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:37.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.944020+0000","last_change":"2024-04-26T18:08:44.347077+0000","last_active":"2024-04-26T18:09:01.944020+0000","last_peered":"2024-04-26T18:09:01.944020+0000","last_clean":"2024-04-26T18:09:01.944020+0000","last_became_active":"2024-04-26T18:08:44.346481+0000","last_became_peered":"2024-04-26T18:08:44.346481+0000","last_unstale":"2024-04-26T18:09:01.944020+0000","last_undegraded":"2024-04-26T18:09:01.944020+0000","last_fullsized":"2024-04-26T18:09:01.944020+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:08.840814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.931780+0000","last_change":"2024-04-26T18:08:44.555658+0000","last_active":"2024-04-26T18:09:00.931780+0000","last_peered":"2024-04-26T18:09:00.931780+0000","last_clean":"2024-04-26T18:09:00.931780+0000","last_became_active":"2024-04-26T18:08:44.555303+0000","last_became_peered":"2024-04-26T18:08:44.555303+0000","last_unstale":"2024-04-26T18:09:00.931780+0000","last_undegraded":"2024-04-26T18:09:00.931780+0000","last_fullsized":"2024-04-26T18:09:00.931780+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:17.781574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363145+0000","last_change":"2024-04-26T18:08:47.366552+0000","last_active":"2024-04-26T18:08:49.363145+0000","last_peered":"2024-04-26T18:08:49.363145+0000","last_clean":"2024-04-26T18:08:49.363145+0000","last_became_active":"2024-04-26T18:08:47.366183+0000","last_became_peered":"2024-04-26T18:08:47.366183+0000","last_unstale":"2024-04-26T18:08:49.363145+0000","last_undegraded":"2024-04-26T18:08:49.363145+0000","last_fullsized":"2024-04-26T18:08:49.363145+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:36.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:24.371105+0000","last_change":"2024-04-26T18:08:44.338078+0000","last_active":"2024-04-26T18:09:24.371105+0000","last_peered":"2024-04-26T18:09:24.371105+0000","last_clean":"2024-04-26T18:09:24.371105+0000","last_became_active":"2024-04-26T18:08:44.337715+0000","last_became_peered":"2024-04-26T18:08:44.337715+0000","last_unstale":"2024-04-26T18:09:24.371105+0000","last_undegraded":"2024-04-26T18:09:24.371105+0000","last_fullsized":"2024-04-26T18:09:24.371105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.912993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370867+0000","last_change":"2024-04-26T18:08:47.371321+0000","last_active":"2024-04-26T18:08:49.370867+0000","last_peered":"2024-04-26T18:08:49.370867+0000","last_clean":"2024-04-26T18:08:49.370867+0000","last_became_active":"2024-04-26T18:08:47.371015+0000","last_became_peered":"2024-04-26T18:08:47.371015+0000","last_unstale":"2024-04-26T18:08:49.370867+0000","last_undegraded":"2024-04-26T18:08:49.370867+0000","last_fullsized":"2024-04-26T18:08:49.370867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.931804+0000","last_change":"2024-04-26T18:08:44.420101+0000","last_active":"2024-04-26T18:09:00.931804+0000","last_peered":"2024-04-26T18:09:00.931804+0000","last_clean":"2024-04-26T18:09:00.931804+0000","last_became_active":"2024-04-26T18:08:44.419787+0000","last_became_peered":"2024-04-26T18:08:44.419787+0000","last_unstale":"2024-04-26T18:09:00.931804+0000","last_undegraded":"2024-04-26T18:09:00.931804+0000","last_fullsized":"2024-04-26T18:09:00.931804+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.485425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372709+0000","last_change":"2024-04-26T18:08:47.372309+0000","last_active":"2024-04-26T18:08:49.372709+0000","last_peered":"2024-04-26T18:08:49.372709+0000","last_clean":"2024-04-26T18:08:49.372709+0000","last_became_active":"2024-04-26T18:08:47.371980+0000","last_became_peered":"2024-04-26T18:08:47.371980+0000","last_unstale":"2024-04-26T18:08:49.372709+0000","last_undegraded":"2024-04-26T18:08:49.372709+0000","last_fullsized":"2024-04-26T18:08:49.372709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:05.011077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.934567+0000","last_change":"2024-04-26T18:08:44.555500+0000","last_active":"2024-04-26T18:09:00.934567+0000","last_peered":"2024-04-26T18:09:00.934567+0000","last_clean":"2024-04-26T18:09:00.934567+0000","last_became_active":"2024-04-26T18:08:44.555137+0000","last_became_peered":"2024-04-26T18:08:44.555137+0000","last_unstale":"2024-04-26T18:09:00.934567+0000","last_undegraded":"2024-04-26T18:09:00.934567+0000","last_fullsized":"2024-04-26T18:09:00.934567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:10:05.723654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367295+0000","last_change":"2024-04-26T18:08:47.364215+0000","last_active":"2024-04-26T18:08:49.367295+0000","last_peered":"2024-04-26T18:08:49.367295+0000","last_clean":"2024-04-26T18:08:49.367295+0000","last_became_active":"2024-04-26T18:08:47.363862+0000","last_became_peered":"2024-04-26T18:08:47.363862+0000","last_unstale":"2024-04-26T18:08:49.367295+0000","last_undegraded":"2024-04-26T18:08:49.367295+0000","last_fullsized":"2024-04-26T18:08:49.367295+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:27.020100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.932170+0000","last_change":"2024-04-26T18:08:44.343506+0000","last_active":"2024-04-26T18:09:00.932170+0000","last_peered":"2024-04-26T18:09:00.932170+0000","last_clean":"2024-04-26T18:09:00.932170+0000","last_became_active":"2024-04-26T18:08:44.343163+0000","last_became_peered":"2024-04-26T18:08:44.343163+0000","last_unstale":"2024-04-26T18:09:00.932170+0000","last_undegraded":"2024-04-26T18:09:00.932170+0000","last_fullsized":"2024-04-26T18:09:00.932170+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:09.941391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361106+0000","last_change":"2024-04-26T18:08:47.365572+0000","last_active":"2024-04-26T18:08:49.361106+0000","last_peered":"2024-04-26T18:08:49.361106+0000","last_clean":"2024-04-26T18:08:49.361106+0000","last_became_active":"2024-04-26T18:08:47.364827+0000","last_became_peered":"2024-04-26T18:08:47.364827+0000","last_unstale":"2024-04-26T18:08:49.361106+0000","last_undegraded":"2024-04-26T18:08:49.361106+0000","last_fullsized":"2024-04-26T18:08:49.361106+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:44.492380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367863+0000","last_change":"2024-04-26T18:08:44.339063+0000","last_active":"2024-04-26T18:08:49.367863+0000","last_peered":"2024-04-26T18:08:49.367863+0000","last_clean":"2024-04-26T18:08:49.367863+0000","last_became_active":"2024-04-26T18:08:44.338752+0000","last_became_peered":"2024-04-26T18:08:44.338752+0000","last_unstale":"2024-04-26T18:08:49.367863+0000","last_undegraded":"2024-04-26T18:08:49.367863+0000","last_fullsized":"2024-04-26T18:08:49.367863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:38.201263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370385+0000","last_change":"2024-04-26T18:08:47.367859+0000","last_active":"2024-04-26T18:08:49.370385+0000","last_peered":"2024-04-26T18:08:49.370385+0000","last_clean":"2024-04-26T18:08:49.370385+0000","last_became_active":"2024-04-26T18:08:47.367548+0000","last_became_peered":"2024-04-26T18:08:47.367548+0000","last_unstale":"2024-04-26T18:08:49.370385+0000","last_undegraded":"2024-04-26T18:08:49.370385+0000","last_fullsized":"2024-04-26T18:08:49.370385+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:13:27.612151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:00.934395+0000","last_change":"2024-04-26T18:08:44.345392+0000","last_active":"2024-04-26T18:09:00.934395+0000","last_peered":"2024-04-26T18:09:00.934395+0000","last_clean":"2024-04-26T18:09:00.934395+0000","last_became_active":"2024-04-26T18:08:44.344578+0000","last_became_peered":"2024-04-26T18:08:44.344578+0000","last_unstale":"2024-04-26T18:09:00.934395+0000","last_undegraded":"2024-04-26T18:09:00.934395+0000","last_fullsized":"2024-04-26T18:09:00.934395+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:26.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373074+0000","last_change":"2024-04-26T18:08:47.380099+0000","last_active":"2024-04-26T18:08:49.373074+0000","last_peered":"2024-04-26T18:08:49.373074+0000","last_clean":"2024-04-26T18:08:49.373074+0000","last_became_active":"2024-04-26T18:08:47.379576+0000","last_became_peered":"2024-04-26T18:08:47.379576+0000","last_unstale":"2024-04-26T18:08:49.373074+0000","last_undegraded":"2024-04-26T18:08:49.373074+0000","last_fullsized":"2024-04-26T18:08:49.373074+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:28.397886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:09:01.942947+0000","last_change":"2024-04-26T18:08:44.555299+0000","last_active":"2024-04-26T18:09:01.942947+0000","last_peered":"2024-04-26T18:09:01.942947+0000","last_clean":"2024-04-26T18:09:01.942947+0000","last_became_active":"2024-04-26T18:08:44.554959+0000","last_became_peered":"2024-04-26T18:08:44.554959+0000","last_unstale":"2024-04-26T18:09:01.942947+0000","last_undegraded":"2024-04-26T18:09:01.942947+0000","last_fullsized":"2024-04-26T18:09:01.942947+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.565073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372526+0000","last_change":"2024-04-26T18:08:47.377524+0000","last_active":"2024-04-26T18:08:49.372526+0000","last_peered":"2024-04-26T18:08:49.372526+0000","last_clean":"2024-04-26T18:08:49.372526+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:08:49.372526+0000","last_undegraded":"2024-04-26T18:08:49.372526+0000","last_fullsized":"2024-04-26T18:08:49.372526+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:10.466615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.376024+0000","last_change":"2024-04-26T18:08:44.343894+0000","last_active":"2024-04-26T18:08:49.376024+0000","last_peered":"2024-04-26T18:08:49.376024+0000","last_clean":"2024-04-26T18:08:49.376024+0000","last_became_active":"2024-04-26T18:08:44.343153+0000","last_became_peered":"2024-04-26T18:08:44.343153+0000","last_unstale":"2024-04-26T18:08:49.376024+0000","last_undegraded":"2024-04-26T18:08:49.376024+0000","last_fullsized":"2024-04-26T18:08:49.376024+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:09.914701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'1","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.421579+0000","last_change":"2024-04-26T18:08:47.371962+0000","last_active":"2024-04-26T18:08:49.421579+0000","last_peered":"2024-04-26T18:08:49.421579+0000","last_clean":"2024-04-26T18:08:49.421579+0000","last_became_active":"2024-04-26T18:08:47.371517+0000","last_became_peered":"2024-04-26T18:08:47.371517+0000","last_unstale":"2024-04-26T18:08:49.421579+0000","last_undegraded":"2024-04-26T18:08:49.421579+0000","last_fullsized":"2024-04-26T18:08:49.421579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:23.553409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":24,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367893+0000","last_change":"2024-04-26T18:08:44.337853+0000","last_active":"2024-04-26T18:08:49.367893+0000","last_peered":"2024-04-26T18:08:49.367893+0000","last_clean":"2024-04-26T18:08:49.367893+0000","last_became_active":"2024-04-26T18:08:44.337393+0000","last_became_peered":"2024-04-26T18:08:44.337393+0000","last_unstale":"2024-04-26T18:08:49.367893+0000","last_undegraded":"2024-04-26T18:08:49.367893+0000","last_fullsized":"2024-04-26T18:08:49.367893+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:26.514821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372700+0000","last_change":"2024-04-26T18:08:47.370999+0000","last_active":"2024-04-26T18:08:49.372700+0000","last_peered":"2024-04-26T18:08:49.372700+0000","last_clean":"2024-04-26T18:08:49.372700+0000","last_became_active":"2024-04-26T18:08:47.370373+0000","last_became_peered":"2024-04-26T18:08:47.370373+0000","last_unstale":"2024-04-26T18:08:49.372700+0000","last_undegraded":"2024-04-26T18:08:49.372700+0000","last_fullsized":"2024-04-26T18:08:49.372700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:19.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361144+0000","last_change":"2024-04-26T18:08:47.365722+0000","last_active":"2024-04-26T18:08:49.361144+0000","last_peered":"2024-04-26T18:08:49.361144+0000","last_clean":"2024-04-26T18:08:49.361144+0000","last_became_active":"2024-04-26T18:08:47.364692+0000","last_became_peered":"2024-04-26T18:08:47.364692+0000","last_unstale":"2024-04-26T18:08:49.361144+0000","last_undegraded":"2024-04-26T18:08:49.361144+0000","last_fullsized":"2024-04-26T18:08:49.361144+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:02.158156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.368007+0000","last_change":"2024-04-26T18:08:47.373886+0000","last_active":"2024-04-26T18:08:49.368007+0000","last_peered":"2024-04-26T18:08:49.368007+0000","last_clean":"2024-04-26T18:08:49.368007+0000","last_became_active":"2024-04-26T18:08:47.373391+0000","last_became_peered":"2024-04-26T18:08:47.373391+0000","last_unstale":"2024-04-26T18:08:49.368007+0000","last_undegraded":"2024-04-26T18:08:49.368007+0000","last_fullsized":"2024-04-26T18:08:49.368007+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:48:59.280050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.367774+0000","last_change":"2024-04-26T18:08:47.366160+0000","last_active":"2024-04-26T18:08:49.367774+0000","last_peered":"2024-04-26T18:08:49.367774+0000","last_clean":"2024-04-26T18:08:49.367774+0000","last_became_active":"2024-04-26T18:08:47.365811+0000","last_became_peered":"2024-04-26T18:08:47.365811+0000","last_unstale":"2024-04-26T18:08:49.367774+0000","last_undegraded":"2024-04-26T18:08:49.367774+0000","last_fullsized":"2024-04-26T18:08:49.367774+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:32.124692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.372335+0000","last_change":"2024-04-26T18:08:47.366078+0000","last_active":"2024-04-26T18:08:49.372335+0000","last_peered":"2024-04-26T18:08:49.372335+0000","last_clean":"2024-04-26T18:08:49.372335+0000","last_became_active":"2024-04-26T18:08:47.365779+0000","last_became_peered":"2024-04-26T18:08:47.365779+0000","last_unstale":"2024-04-26T18:08:49.372335+0000","last_undegraded":"2024-04-26T18:08:49.372335+0000","last_fullsized":"2024-04-26T18:08:49.372335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.127606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363238+0000","last_change":"2024-04-26T18:08:47.372996+0000","last_active":"2024-04-26T18:08:49.363238+0000","last_peered":"2024-04-26T18:08:49.363238+0000","last_clean":"2024-04-26T18:08:49.363238+0000","last_became_active":"2024-04-26T18:08:47.372687+0000","last_became_peered":"2024-04-26T18:08:47.372687+0000","last_unstale":"2024-04-26T18:08:49.363238+0000","last_undegraded":"2024-04-26T18:08:49.363238+0000","last_fullsized":"2024-04-26T18:08:49.363238+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:13:33.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361390+0000","last_change":"2024-04-26T18:08:47.371076+0000","last_active":"2024-04-26T18:08:49.361390+0000","last_peered":"2024-04-26T18:08:49.361390+0000","last_clean":"2024-04-26T18:08:49.361390+0000","last_became_active":"2024-04-26T18:08:47.370688+0000","last_became_peered":"2024-04-26T18:08:47.370688+0000","last_unstale":"2024-04-26T18:08:49.361390+0000","last_undegraded":"2024-04-26T18:08:49.361390+0000","last_fullsized":"2024-04-26T18:08:49.361390+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:21.355971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369199+0000","last_change":"2024-04-26T18:08:47.372392+0000","last_active":"2024-04-26T18:08:49.369199+0000","last_peered":"2024-04-26T18:08:49.369199+0000","last_clean":"2024-04-26T18:08:49.369199+0000","last_became_active":"2024-04-26T18:08:47.372020+0000","last_became_peered":"2024-04-26T18:08:47.372020+0000","last_unstale":"2024-04-26T18:08:49.369199+0000","last_undegraded":"2024-04-26T18:08:49.369199+0000","last_fullsized":"2024-04-26T18:08:49.369199+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.369495+0000","last_change":"2024-04-26T18:08:47.369571+0000","last_active":"2024-04-26T18:08:49.369495+0000","last_peered":"2024-04-26T18:08:49.369495+0000","last_clean":"2024-04-26T18:08:49.369495+0000","last_became_active":"2024-04-26T18:08:47.369165+0000","last_became_peered":"2024-04-26T18:08:47.369165+0000","last_unstale":"2024-04-26T18:08:49.369495+0000","last_undegraded":"2024-04-26T18:08:49.369495+0000","last_fullsized":"2024-04-26T18:08:49.369495+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:15.869182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.373658+0000","last_change":"2024-04-26T18:08:47.372355+0000","last_active":"2024-04-26T18:08:49.373658+0000","last_peered":"2024-04-26T18:08:49.373658+0000","last_clean":"2024-04-26T18:08:49.373658+0000","last_became_active":"2024-04-26T18:08:47.371810+0000","last_became_peered":"2024-04-26T18:08:47.371810+0000","last_unstale":"2024-04-26T18:08:49.373658+0000","last_undegraded":"2024-04-26T18:08:49.373658+0000","last_fullsized":"2024-04-26T18:08:49.373658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:44:22.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.370438+0000","last_change":"2024-04-26T18:08:47.364042+0000","last_active":"2024-04-26T18:08:49.370438+0000","last_peered":"2024-04-26T18:08:49.370438+0000","last_clean":"2024-04-26T18:08:49.370438+0000","last_became_active":"2024-04-26T18:08:47.363727+0000","last_became_peered":"2024-04-26T18:08:47.363727+0000","last_unstale":"2024-04-26T18:08:49.370438+0000","last_undegraded":"2024-04-26T18:08:49.370438+0000","last_fullsized":"2024-04-26T18:08:49.370438+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:37.595261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361368+0000","last_change":"2024-04-26T18:08:47.377535+0000","last_active":"2024-04-26T18:08:49.361368+0000","last_peered":"2024-04-26T18:08:49.361368+0000","last_clean":"2024-04-26T18:08:49.361368+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:08:49.361368+0000","last_undegraded":"2024-04-26T18:08:49.361368+0000","last_fullsized":"2024-04-26T18:08:49.361368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:22.895402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.363119+0000","last_change":"2024-04-26T18:08:47.367313+0000","last_active":"2024-04-26T18:08:49.363119+0000","last_peered":"2024-04-26T18:08:49.363119+0000","last_clean":"2024-04-26T18:08:49.363119+0000","last_became_active":"2024-04-26T18:08:47.366971+0000","last_became_peered":"2024-04-26T18:08:47.366971+0000","last_unstale":"2024-04-26T18:08:49.363119+0000","last_undegraded":"2024-04-26T18:08:49.363119+0000","last_fullsized":"2024-04-26T18:08:49.363119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:14.518100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:08:49.361072+0000","last_change":"2024-04-26T18:08:47.373004+0000","last_active":"2024-04-26T18:08:49.361072+0000","last_peered":"2024-04-26T18:08:49.361072+0000","last_clean":"2024-04-26T18:08:49.361072+0000","last_became_active":"2024-04-26T18:08:47.372641+0000","last_became_peered":"2024-04-26T18:08:47.372641+0000","last_unstale":"2024-04-26T18:08:49.361072+0000","last_undegraded":"2024-04-26T18:08:49.361072+0000","last_fullsized":"2024-04-26T18:08:49.361072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:02.018425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64154,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":186,"num_read_kb":162,"num_write":166,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":181672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":472639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579932,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1675264,"data_stored":1124273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743458,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":472875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005095,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":472795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266732,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36888,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708264,"statfs":{"total":95995035648,"available":95957262336,"internally_reserved":0,"allocated":1589248,"data_stored":1065116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":472264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":526424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":1003520,"data_stored":473535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411396,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":528165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":472575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920827,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":473092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724598,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32740,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712412,"statfs":{"total":95995035648,"available":95961509888,"internally_reserved":0,"allocated":1601536,"data_stored":1064409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":54367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:09:33.315 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:09:33.316 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T18:09:33.316 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T18:09:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: pgmap v320: 97 pgs: 97 active+clean; 639 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-04-26T18:09:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-26T18:09:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-26T18:09:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-26T18:09:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:09:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-04-26T18:09:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-26T18:09:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:33 smithi167 ceph-mon[32751]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:09:33.556 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:34 smithi119 ceph-mon[32752]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:09:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:34 smithi119 ceph-mon[32752]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:09:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:34 smithi167 ceph-mon[32751]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:09:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:34 smithi167 ceph-mon[32751]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:09:34.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:34 smithi090 ceph-mon[29180]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:09:34.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:34 smithi090 ceph-mon[29180]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:09:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 -- 172.21.15.90:0/614309418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c103f40 msgr2=0x7f353c10a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 --2- 172.21.15.90:0/614309418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c103f40 0x7f353c10a7f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f353400b0d0 tx=0x7f353400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 -- 172.21.15.90:0/614309418 shutdown_connections 2024-04-26T18:09:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 --2- 172.21.15.90:0/614309418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c103f40 0x7f353c10a7f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 --2- 172.21.15.90:0/614309418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f353c1023a0 0x7f353c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 --2- 172.21.15.90:0/614309418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f353c1035a0 0x7f353c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 -- 172.21.15.90:0/614309418 >> 172.21.15.90:0/614309418 conn(0x7f353c0fd940 msgr2=0x7f353c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 -- 172.21.15.90:0/614309418 shutdown_connections 2024-04-26T18:09:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 -- 172.21.15.90:0/614309418 wait complete. 2024-04-26T18:09:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.106+0000 7f354280b700 1 Processor -- start 2024-04-26T18:09:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f354280b700 1 -- start start 2024-04-26T18:09:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f354280b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c1023a0 0x7f353c106330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f354280b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f353c1035a0 0x7f353c104980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:35.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f354280b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f353c103f40 0x7f353c104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f354280b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10de10 con 0x7f353c1023a0 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f354280b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10dc90 con 0x7f353c103f40 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f354280b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f353c10df90 con 0x7f353c1035a0 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c1023a0 0x7f353c106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c1023a0 0x7f353c106330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56442/0 (socket says 172.21.15.90:56442) 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 -- 172.21.15.90:0/963183735 learned_addr learned my addr 172.21.15.90:0/963183735 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f3540da8700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f353c103f40 0x7f353c104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353b7fe700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f353c1035a0 0x7f353c104980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 -- 172.21.15.90:0/963183735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f353c1035a0 msgr2=0x7f353c104980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f353c1035a0 0x7f353c104980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 -- 172.21.15.90:0/963183735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f353c103f40 msgr2=0x7f353c104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f353c103f40 0x7f353c104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 -- 172.21.15.90:0/963183735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353c105770 con 0x7f353c1023a0 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.107+0000 7f353bfff700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c1023a0 0x7f353c106330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f352c00c970 tx=0x7f352c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.108+0000 7f35397fa700 1 -- 172.21.15.90:0/963183735 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c004220 con 0x7f353c1023a0 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.108+0000 7f35397fa700 1 -- 172.21.15.90:0/963183735 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f352c0043c0 con 0x7f353c1023a0 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.108+0000 7f35397fa700 1 -- 172.21.15.90:0/963183735 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c003c20 con 0x7f353c1023a0 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.108+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353c1a35f0 con 0x7f353c1023a0 2024-04-26T18:09:35.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.108+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f353c1a3b30 con 0x7f353c1023a0 2024-04-26T18:09:35.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.109+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353c051960 con 0x7f353c1023a0 2024-04-26T18:09:35.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.110+0000 7f35397fa700 1 -- 172.21.15.90:0/963183735 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f352c0076d0 con 0x7f353c1023a0 2024-04-26T18:09:35.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.110+0000 7f35397fa700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f35280a3e00 0x7f35280a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:35.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.110+0000 7f353b7fe700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f35280a3e00 0x7f35280a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:35.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.110+0000 7f35397fa700 1 -- 172.21.15.90:0/963183735 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f352c0cfed0 con 0x7f353c1023a0 2024-04-26T18:09:35.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.111+0000 7f353b7fe700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f35280a3e00 0x7f35280a62c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3524007be0 tx=0x7f35240073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:35.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:35 smithi090 ceph-mon[29180]: pgmap v323: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-26T18:09:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.116+0000 7f35397fa700 1 -- 172.21.15.90:0/963183735 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f352c014030 con 0x7f353c1023a0 2024-04-26T18:09:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:35 smithi119 ceph-mon[32752]: pgmap v323: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-26T18:09:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:35 smithi167 ceph-mon[32751]: pgmap v323: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-26T18:09:35.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.402+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f353c1a3df0 con 0x7f353c1023a0 2024-04-26T18:09:35.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.405+0000 7f35397fa700 1 -- 172.21.15.90:0/963183735 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f352c09c850 con 0x7f353c1023a0 2024-04-26T18:09:35.405 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-04-26T18:09:35.405 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDf7Stm5PMQGBAAHeSsYmRWq/vkBW75kdHOpg== 2024-04-26T18:09:35.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f35280a3e00 msgr2=0x7f35280a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:35.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f35280a3e00 0x7f35280a62c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3524007be0 tx=0x7f35240073d0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c1023a0 msgr2=0x7f353c106330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:35.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c1023a0 0x7f353c106330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f352c00c970 tx=0x7f352c00cc80 comp rx=0 tx=0).stop 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 shutdown_connections 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f35280a3e00 0x7f35280a62c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f353c1023a0 0x7f353c106330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f353c103f40 0x7f353c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 --2- 172.21.15.90:0/963183735 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f353c1035a0 0x7f353c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.407+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 >> 172.21.15.90:0/963183735 conn(0x7f353c0fd940 msgr2=0x7f353c0ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.408+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 shutdown_connections 2024-04-26T18:09:35.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:35.408+0000 7f354280b700 1 -- 172.21.15.90:0/963183735 wait complete. 2024-04-26T18:09:36.055 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:09:36.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T18:09:36.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T18:09:36.091 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T18:09:36.091 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T18:09:36.091 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T18:09:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:36 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/963183735' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:09:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:36 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/963183735' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:09:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:36 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/963183735' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:09:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:36 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/963183735' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:09:36.366 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:36 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/963183735' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:09:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:36 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/963183735' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:09:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:37 smithi090 ceph-mon[29180]: pgmap v324: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:09:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:37 smithi119 ceph-mon[32752]: pgmap v324: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:09:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:37 smithi167 ceph-mon[32751]: pgmap v324: 97 pgs: 3 peering, 94 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:09:37.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 -- 172.21.15.90:0/3828128375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 msgr2=0x7f1994107ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 --2- 172.21.15.90:0/3828128375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f1994107ea0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19840099c0 tx=0x7f1984009cd0 comp rx=0 tx=0).stop 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 -- 172.21.15.90:0/3828128375 shutdown_connections 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 --2- 172.21.15.90:0/3828128375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f1994107ea0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 --2- 172.21.15.90:0/3828128375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19941027f0 0x7f199410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 --2- 172.21.15.90:0/3828128375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1994101e30 0x7f19941022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 -- 172.21.15.90:0/3828128375 >> 172.21.15.90:0/3828128375 conn(0x7f1994078520 msgr2=0x7f19940ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 -- 172.21.15.90:0/3828128375 shutdown_connections 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 -- 172.21.15.90:0/3828128375 wait complete. 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.972+0000 7f199c392700 1 Processor -- start 2024-04-26T18:09:37.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199c392700 1 -- start start 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199c392700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1994101e30 0x7f19941a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199c392700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19941027f0 0x7f19941a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199c392700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f199419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199c392700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1994111c50 con 0x7f1994107ac0 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199c392700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1994111ad0 con 0x7f19941027f0 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199c392700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1994111dd0 con 0x7f1994101e30 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f199419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f199419ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56450/0 (socket says 172.21.15.90:56450) 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 -- 172.21.15.90:0/2597563695 learned_addr learned my addr 172.21.15.90:0/2597563695 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199992d700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19941027f0 0x7f19941a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a12e700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1994101e30 0x7f19941a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:37.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 -- 172.21.15.90:0/2597563695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1994101e30 msgr2=0x7f19941a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1994101e30 0x7f19941a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 -- 172.21.15.90:0/2597563695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19941027f0 msgr2=0x7f19941a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19941027f0 0x7f19941a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 -- 172.21.15.90:0/2597563695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199419c1b0 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.973+0000 7f199a92f700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f199419ba50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f198c007d70 tx=0x7f198c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.974+0000 7f198b7fe700 1 -- 172.21.15.90:0/2597563695 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c016020 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.974+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199419c4a0 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.974+0000 7f198b7fe700 1 -- 172.21.15.90:0/2597563695 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f198c00ad40 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.974+0000 7f198b7fe700 1 -- 172.21.15.90:0/2597563695 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c0115e0 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.974+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19941a8760 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.974+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199406a110 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.976+0000 7f198b7fe700 1 -- 172.21.15.90:0/2597563695 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f198c019020 con 0x7f1994107ac0 2024-04-26T18:09:37.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.976+0000 7f198b7fe700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f19800a3cd0 0x7f19800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.976+0000 7f199a12e700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f19800a3cd0 0x7f19800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.977+0000 7f198b7fe700 1 -- 172.21.15.90:0/2597563695 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f198c0d0710 con 0x7f1994107ac0 2024-04-26T18:09:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.977+0000 7f199a12e700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f19800a3cd0 0x7f19800a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1984009f50 tx=0x7f198401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:37.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:37.980+0000 7f198b7fe700 1 -- 172.21.15.90:0/2597563695 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f198c020020 con 0x7f1994107ac0 2024-04-26T18:09:38.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.238+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f19941a8a50 con 0x7f1994107ac0 2024-04-26T18:09:38.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.243+0000 7f198b7fe700 1 -- 172.21.15.90:0/2597563695 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7f198c09d130 con 0x7f1994107ac0 2024-04-26T18:09:38.247 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f19800a3cd0 msgr2=0x7f19800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f19800a3cd0 0x7f19800a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1984009f50 tx=0x7f198401a040 comp rx=0 tx=0).stop 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 msgr2=0x7f199419ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f199419ba50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f198c007d70 tx=0x7f198c00a340 comp rx=0 tx=0).stop 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 shutdown_connections 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f19800a3cd0 0x7f19800a6190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1994107ac0 0x7f199419ba50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1994101e30 0x7f19941a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 --2- 172.21.15.90:0/2597563695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19941027f0 0x7f19941a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 >> 172.21.15.90:0/2597563695 conn(0x7f1994078520 msgr2=0x7f199410e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 shutdown_connections 2024-04-26T18:09:38.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:38.246+0000 7f199c392700 1 -- 172.21.15.90:0/2597563695 wait complete. 2024-04-26T18:09:38.912 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":3660618461},{"type":"v1","addr":"172.21.15.90:6801","nonce":3660618461}]},"active_addr":"172.21.15.90:6801/3660618461","active_change":"2024-04-26T18:01:31.766831+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":484221164}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1315801302}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2718220414}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1501521801}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1671620802}]}]} 2024-04-26T18:09:38.922 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T18:09:38.922 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T18:09:38.923 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:09:39.163 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:39.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:39 smithi090 ceph-mon[29180]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 6 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:39.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:39 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2597563695' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:09:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:39 smithi119 ceph-mon[32752]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 6 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:39 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2597563695' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:09:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:39 smithi167 ceph-mon[32751]: pgmap v325: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1023 B/s wr, 6 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:39 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2597563695' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:09:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/1256664559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 msgr2=0x7fa9b0102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/1256664559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b0102240 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4009990 tx=0x7fa9a4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/1256664559 shutdown_connections 2024-04-26T18:09:40.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/1256664559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9b0102780 0x7fa9b01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/1256664559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9b01084e0 0x7fa9b01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/1256664559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b0102240 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/1256664559 >> 172.21.15.90:0/1256664559 conn(0x7fa9b0077640 msgr2=0x7fa9b0077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/1256664559 shutdown_connections 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.902+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/1256664559 wait complete. 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 Processor -- start 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 -- start start 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9b0102780 0x7fa9b019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9b01084e0 0x7fa9b0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b0113670 con 0x7fa9b0101dc0 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01134f0 con 0x7fa9b01084e0 2024-04-26T18:09:40.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9b5ec4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa9b01137f0 con 0x7fa9b0102780 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9af7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9af7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56462/0 (socket says 172.21.15.90:56462) 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.903+0000 7fa9af7fe700 1 -- 172.21.15.90:0/3088556348 learned_addr learned my addr 172.21.15.90:0/3088556348 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9aeffd700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9b0102780 0x7fa9b019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9affff700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9b01084e0 0x7fa9b0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9af7fe700 1 -- 172.21.15.90:0/3088556348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9b0102780 msgr2=0x7fa9b019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9af7fe700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9b0102780 0x7fa9b019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9af7fe700 1 -- 172.21.15.90:0/3088556348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9b01084e0 msgr2=0x7fa9b0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9af7fe700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9b01084e0 0x7fa9b0199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9af7fe700 1 -- 172.21.15.90:0/3088556348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9b019a0b0 con 0x7fa9b0101dc0 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9affff700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9b01084e0 0x7fa9b0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:09:40.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9af7fe700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b019f790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000c8f0 tx=0x7fa9a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9acff9700 1 -- 172.21.15.90:0/3088556348 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0004220 con 0x7fa9b0101dc0 2024-04-26T18:09:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9b019a3a0 con 0x7fa9b0101dc0 2024-04-26T18:09:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9acff9700 1 -- 172.21.15.90:0/3088556348 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9a0007db0 con 0x7fa9b0101dc0 2024-04-26T18:09:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9acff9700 1 -- 172.21.15.90:0/3088556348 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9a0003d10 con 0x7fa9b0101dc0 2024-04-26T18:09:40.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.904+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9b01a6660 con 0x7fa9b0101dc0 2024-04-26T18:09:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.905+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9b006a110 con 0x7fa9b0101dc0 2024-04-26T18:09:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.906+0000 7fa9acff9700 1 -- 172.21.15.90:0/3088556348 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa9a0007900 con 0x7fa9b0101dc0 2024-04-26T18:09:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.907+0000 7fa9acff9700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa9980a3cd0 0x7fa9980a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.907+0000 7fa9aeffd700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa9980a3cd0 0x7fa9980a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.907+0000 7fa9acff9700 1 -- 172.21.15.90:0/3088556348 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fa9a00d04b0 con 0x7fa9b0101dc0 2024-04-26T18:09:40.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.907+0000 7fa9aeffd700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa9980a3cd0 0x7fa9980a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006010 tx=0x7fa9a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:40.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:40.910+0000 7fa9acff9700 1 -- 172.21.15.90:0/3088556348 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9a0014030 con 0x7fa9b0101dc0 2024-04-26T18:09:41.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.112+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa9b002e490 con 0x7fa9b0101dc0 2024-04-26T18:09:41.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.113+0000 7fa9acff9700 1 -- 172.21.15.90:0/3088556348 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22557 (secure 0 0 0) 0x7fa9a009ce30 con 0x7fa9b0101dc0 2024-04-26T18:09:41.113 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:09:41.114 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":92,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:09:33.002079+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:09:41.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa9980a3cd0 msgr2=0x7fa9980a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:41.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa9980a3cd0 0x7fa9980a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006010 tx=0x7fa9a401a040 comp rx=0 tx=0).stop 2024-04-26T18:09:41.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 msgr2=0x7fa9b019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:41.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b019f790 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa9a000c8f0 tx=0x7fa9a000ccb0 comp rx=0 tx=0).stop 2024-04-26T18:09:41.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 shutdown_connections 2024-04-26T18:09:41.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa9980a3cd0 0x7fa9980a6190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:41.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9b0101dc0 0x7fa9b019f790 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:41.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9b01084e0 0x7fa9b0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:41.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 --2- 172.21.15.90:0/3088556348 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa9b0102780 0x7fa9b019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:41.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.115+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 >> 172.21.15.90:0/3088556348 conn(0x7fa9b0077640 msgr2=0x7fa9b0100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:41.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.116+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 shutdown_connections 2024-04-26T18:09:41.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:41.116+0000 7fa9b5ec4700 1 -- 172.21.15.90:0/3088556348 wait complete. 2024-04-26T18:09:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:41 smithi119 ceph-mon[32752]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:41 smithi167 ceph-mon[32751]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:41 smithi090 ceph-mon[29180]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 4 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:41.769 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T18:09:41.769 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:09:42.012 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:42.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:42 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3088556348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:42 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3088556348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:42 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3088556348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:43 smithi119 ceph-mon[32752]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:43 smithi167 ceph-mon[32751]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:43.337 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:43 smithi090 ceph-mon[29180]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 5 B/s, 1 keys/s, 0 objects/s recovering 2024-04-26T18:09:43.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.688+0000 7ff10a461700 1 -- 172.21.15.90:0/3510773479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102cd0 msgr2=0x7ff10410f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:43.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 --2- 172.21.15.90:0/3510773479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102cd0 0x7ff10410f1c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00b0a0 tx=0x7ff0fc00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 -- 172.21.15.90:0/3510773479 shutdown_connections 2024-04-26T18:09:43.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 --2- 172.21.15.90:0/3510773479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102cd0 0x7ff10410f1c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 --2- 172.21.15.90:0/3510773479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104108380 0x7ff104108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 --2- 172.21.15.90:0/3510773479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff104102310 0x7ff104102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 -- 172.21.15.90:0/3510773479 >> 172.21.15.90:0/3510773479 conn(0x7ff104079290 msgr2=0x7ff1040796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 -- 172.21.15.90:0/3510773479 shutdown_connections 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 -- 172.21.15.90:0/3510773479 wait complete. 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 Processor -- start 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.689+0000 7ff10a461700 1 -- start start 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff10a461700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102310 0x7ff10419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff10a461700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff104102cd0 0x7ff10419af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff10a461700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104108380 0x7ff1041a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff10a461700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff1041124c0 con 0x7ff104102310 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff10a461700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff104112340 con 0x7ff104102cd0 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff10a461700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff104112640 con 0x7ff104108380 2024-04-26T18:09:43.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102310 0x7ff10419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102310 0x7ff10419a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50526/0 (socket says 172.21.15.90:50526) 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 -- 172.21.15.90:0/1425687702 learned_addr learned my addr 172.21.15.90:0/1425687702 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff1037fe700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff104102cd0 0x7ff10419af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff1089fe700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104108380 0x7ff1041a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 -- 172.21.15.90:0/1425687702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104108380 msgr2=0x7ff1041a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104108380 0x7ff1041a5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 -- 172.21.15.90:0/1425687702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff104102cd0 msgr2=0x7ff10419af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff104102cd0 0x7ff10419af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 -- 172.21.15.90:0/1425687702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1041a5f80 con 0x7ff104102310 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff103fff700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102310 0x7ff10419a9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400e9c0 tx=0x7ff0f400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.690+0000 7ff1017fa700 1 -- 172.21.15.90:0/1425687702 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400cbd0 con 0x7ff104102310 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.691+0000 7ff1017fa700 1 -- 172.21.15.90:0/1425687702 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0f400ee60 con 0x7ff104102310 2024-04-26T18:09:43.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.691+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1041a6150 con 0x7ff104102310 2024-04-26T18:09:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.692+0000 7ff1017fa700 1 -- 172.21.15.90:0/1425687702 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40055c0 con 0x7ff104102310 2024-04-26T18:09:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.692+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1041a6610 con 0x7ff104102310 2024-04-26T18:09:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.693+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff104103a30 con 0x7ff104102310 2024-04-26T18:09:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.694+0000 7ff1017fa700 1 -- 172.21.15.90:0/1425687702 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff0f4004710 con 0x7ff104102310 2024-04-26T18:09:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.694+0000 7ff1017fa700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff0ec0a3bb0 0x7ff0ec0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:43.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.695+0000 7ff1037fe700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff0ec0a3bb0 0x7ff0ec0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:43.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.695+0000 7ff1017fa700 1 -- 172.21.15.90:0/1425687702 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7ff0f40d0780 con 0x7ff104102310 2024-04-26T18:09:43.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.695+0000 7ff1037fe700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff0ec0a3bb0 0x7ff0ec0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff1040798b0 tx=0x7ff0f800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:43.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.699+0000 7ff1017fa700 1 -- 172.21.15.90:0/1425687702 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0f409d000 con 0x7ff104102310 2024-04-26T18:09:43.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.942+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff10406a110 con 0x7ff104102310 2024-04-26T18:09:43.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.944+0000 7ff1017fa700 1 -- 172.21.15.90:0/1425687702 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22557 (secure 0 0 0) 0x7ff0f409c900 con 0x7ff104102310 2024-04-26T18:09:43.944 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:09:43.944 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":92,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:09:33.002079+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:09:43.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff0ec0a3bb0 msgr2=0x7ff0ec0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff0ec0a3bb0 0x7ff0ec0a6070 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff1040798b0 tx=0x7ff0f800b410 comp rx=0 tx=0).stop 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102310 msgr2=0x7ff10419a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102310 0x7ff10419a9c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400e9c0 tx=0x7ff0f400ecd0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 shutdown_connections 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7ff0ec0a3bb0 0x7ff0ec0a6070 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff104102310 0x7ff10419a9c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff104108380 0x7ff1041a5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 --2- 172.21.15.90:0/1425687702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff104102cd0 0x7ff10419af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 >> 172.21.15.90:0/1425687702 conn(0x7ff104079290 msgr2=0x7ff104100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 shutdown_connections 2024-04-26T18:09:43.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:43.946+0000 7ff10a461700 1 -- 172.21.15.90:0/1425687702 wait complete. 2024-04-26T18:09:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:44 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:09:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:44 smithi119 ceph-mon[32752]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:44 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1425687702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:44 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:09:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:44 smithi167 ceph-mon[32751]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:44 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1425687702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:44 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:09:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:44 smithi090 ceph-mon[29180]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:44 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1425687702' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:09:45.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T18:09:45.117 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T18:09:45.117 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T18:09:45.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T18:09:45.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T18:09:45.118 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T18:09:45.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T18:09:45.119 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T18:09:45.120 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T18:09:45.120 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T18:09:45.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T18:09:45.121 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T18:09:45.698 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:45.713 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:45.829 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:45.857 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:46.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:45 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:45 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:45 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:47.068 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:46 smithi090 ceph-mon[29180]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:46 smithi119 ceph-mon[32752]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:46 smithi167 ceph-mon[32751]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:49.066 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:48 smithi090 ceph-mon[29180]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.065+0000 7f96c31cb700 1 -- 172.21.15.90:0/2792864895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 msgr2=0x7f96bc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.065+0000 7f96c31cb700 1 --2- 172.21.15.90:0/2792864895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 0x7f96bc1087b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009990 tx=0x7f96b0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:09:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.065+0000 7f96c31cb700 1 -- 172.21.15.90:0/2792864895 shutdown_connections 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.065+0000 7f96c31cb700 1 --2- 172.21.15.90:0/2792864895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 0x7f96bc1087b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.065+0000 7f96c31cb700 1 --2- 172.21.15.90:0/2792864895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96bc102d10 0x7f96bc10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 --2- 172.21.15.90:0/2792864895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc102350 0x7f96bc1027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 -- 172.21.15.90:0/2792864895 >> 172.21.15.90:0/2792864895 conn(0x7f96bc078390 msgr2=0x7f96bc0787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 -- 172.21.15.90:0/2792864895 shutdown_connections 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 -- 172.21.15.90:0/2792864895 wait complete. 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 Processor -- start 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 -- start start 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96bc102350 0x7f96bc19f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc102d10 0x7f96bc19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 0x7f96bc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc1125a0 con 0x7f96bc1083d0 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc112420 con 0x7f96bc102350 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.066+0000 7f96c31cb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc112720 con 0x7f96bc102d10 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 0x7f96bc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 0x7f96bc199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50536/0 (socket says 172.21.15.90:50536) 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 -- 172.21.15.90:0/1121016752 learned_addr learned my addr 172.21.15.90:0/1121016752 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96bbfff700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc102d10 0x7f96bc19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c0f67700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96bc102350 0x7f96bc19f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 -- 172.21.15.90:0/1121016752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc102d10 msgr2=0x7f96bc19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f96bc102d10 0x7f96bc19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 -- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96bc102350 msgr2=0x7f96bc19f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96bc102350 0x7f96bc19f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 -- 172.21.15.90:0/1121016752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc19a0f0 con 0x7f96bc1083d0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c1768700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 0x7f96bc199990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f96b400b970 tx=0x7f96b400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4019070 con 0x7f96bc1083d0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc19a3e0 con 0x7f96bc1083d0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96b4014440 con 0x7f96bc1083d0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b400ce60 con 0x7f96bc1083d0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.067+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc1a66a0 con 0x7f96bc1083d0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.068+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96bc106450 con 0x7f96bc1083d0 2024-04-26T18:09:49.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.069+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f96b401c070 con 0x7f96bc1083d0 2024-04-26T18:09:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.070+0000 7f96b9ffb700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f96ac0a3cd0 0x7f96ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.070+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f96b40d0010 con 0x7f96bc1083d0 2024-04-26T18:09:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.070+0000 7f96c0f67700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f96ac0a3cd0 0x7f96ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.070+0000 7f96b9ffb700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] conn(0x7f96ac0ad710 0x7f96ac0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.070+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 --> [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96b4017e70 con 0x7f96ac0ad710 2024-04-26T18:09:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.070+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f96b40d04f0 con 0x7f96bc1083d0 2024-04-26T18:09:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.071+0000 7f96c0f67700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f96ac0a3cd0 0x7f96ac0a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006010 tx=0x7f96b000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.071+0000 7f96bbfff700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] conn(0x7f96ac0ad710 0x7f96ac0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.071+0000 7f96bbfff700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] conn(0x7f96ac0ad710 0x7f96ac0afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.072+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== osd.1 v2:172.21.15.119:6800/447582206 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96b4017e70 con 0x7f96ac0ad710 2024-04-26T18:09:49.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.089+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 --> [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f96bc103a40 con 0x7f96ac0ad710 2024-04-26T18:09:49.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96b9ffb700 1 -- 172.21.15.90:0/1121016752 <== osd.1 v2:172.21.15.119:6800/447582206 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f96bc103a40 con 0x7f96ac0ad710 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] conn(0x7f96ac0ad710 msgr2=0x7f96ac0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96c31cb700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] conn(0x7f96ac0ad710 0x7f96ac0afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f96ac0a3cd0 msgr2=0x7f96ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96c31cb700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f96ac0a3cd0 0x7f96ac0a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f96b0006010 tx=0x7f96b000b540 comp rx=0 tx=0).stop 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 msgr2=0x7f96bc199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96c31cb700 1 --2- 172.21.15.90:0/1121016752 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f96bc1083d0 0x7f96bc199990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f96b400b970 tx=0x7f96b400bd30 comp rx=0 tx=0).stop 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.090+0000 7f96c1768700 1 -- 172.21.15.90:0/1121016752 reap_dead start 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.091+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 shutdown_connections 2024-04-26T18:09:49.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.091+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 >> 172.21.15.90:0/1121016752 conn(0x7f96bc078390 msgr2=0x7f96bc1016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:49.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.091+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 shutdown_connections 2024-04-26T18:09:49.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:49.091+0000 7f96c31cb700 1 -- 172.21.15.90:0/1121016752 wait complete. 2024-04-26T18:09:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:48 smithi119 ceph-mon[32752]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:48 smithi167 ceph-mon[32751]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:09:49.418 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:49.418 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:50.037 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:50 smithi090 ceph-mon[29180]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:09:51.236 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:51.237 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:51.237 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:50 smithi119 ceph-mon[32752]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:09:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:50 smithi167 ceph-mon[32751]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:09:52.326 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:52.355 INFO:teuthology.orchestra.run.smithi090.stdout:55834574927 2024-04-26T18:09:52.356 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T18:09:53.055 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:52 smithi090 ceph-mon[29180]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:09:53.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 -- 172.21.15.90:0/437834073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9450103e30 msgr2=0x7f9450110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 --2- 172.21.15.90:0/437834073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9450103e30 0x7f9450110290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f944800b0a0 tx=0x7f944800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 -- 172.21.15.90:0/437834073 shutdown_connections 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 --2- 172.21.15.90:0/437834073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9450103e30 0x7f9450110290 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 --2- 172.21.15.90:0/437834073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f94501094c0 0x7f94501098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 --2- 172.21.15.90:0/437834073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9450103470 0x7f94501038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 -- 172.21.15.90:0/437834073 >> 172.21.15.90:0/437834073 conn(0x7f945007a580 msgr2=0x7f945007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 -- 172.21.15.90:0/437834073 shutdown_connections 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 -- 172.21.15.90:0/437834073 wait complete. 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.055+0000 7f9455f5a700 1 Processor -- start 2024-04-26T18:09:53.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f9455f5a700 1 -- start start 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f9455f5a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450103470 0x7f945019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f9455f5a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9450103e30 0x7f945019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f9455f5a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94501094c0 0x7f9450199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f9455f5a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f94501135c0 con 0x7f94501094c0 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f9455f5a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9450113440 con 0x7f9450103e30 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f9455f5a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9450113740 con 0x7f9450103470 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94501094c0 0x7f9450199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94501094c0 0x7f9450199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47386/0 (socket says 172.21.15.90:47386) 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 -- 172.21.15.90:0/3141873516 learned_addr learned my addr 172.21.15.90:0/3141873516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944f7fe700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450103470 0x7f945019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944effd700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9450103e30 0x7f945019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:53.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 -- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450103470 msgr2=0x7f945019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450103470 0x7f945019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 -- 172.21.15.90:0/3141873516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9450103e30 msgr2=0x7f945019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9450103e30 0x7f945019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.056+0000 7f944ffff700 1 -- 172.21.15.90:0/3141873516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9450199fe0 con 0x7f94501094c0 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.057+0000 7f944ffff700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94501094c0 0x7f9450199880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f944800b830 tx=0x7f9448004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.057+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448021070 con 0x7f94501094c0 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.057+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945019a270 con 0x7f94501094c0 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.057+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9448007e30 con 0x7f94501094c0 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.057+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448004700 con 0x7f94501094c0 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.057+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94501a6590 con 0x7f94501094c0 2024-04-26T18:09:53.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.058+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9450063b30 con 0x7f94501094c0 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.059+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9448003d70 con 0x7f94501094c0 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.059+0000 7f944cff9700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94380a3cd0 0x7f94380a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.060+0000 7f944f7fe700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94380a3cd0 0x7f94380a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.060+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f944800f020 con 0x7f94501094c0 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.060+0000 7f944cff9700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] conn(0x7f94380ad710 0x7f94380afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.060+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 --> [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f944801de50 con 0x7f94380ad710 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.060+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f94480d2000 con 0x7f94501094c0 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.060+0000 7f944f7fe700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94380a3cd0 0x7f94380a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f945019a9c0 tx=0x7f944000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.060+0000 7f944effd700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] conn(0x7f94380ad710 0x7f94380afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:53.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.061+0000 7f944effd700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] conn(0x7f94380ad710 0x7f94380afbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:53.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.062+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== osd.8 v2:172.21.15.167:6816/1376531844 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f944801de50 con 0x7f94380ad710 2024-04-26T18:09:53.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.078+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 --> [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9450063fb0 con 0x7f94380ad710 2024-04-26T18:09:53.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f944cff9700 1 -- 172.21.15.90:0/3141873516 <== osd.8 v2:172.21.15.167:6816/1376531844 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9450063fb0 con 0x7f94380ad710 2024-04-26T18:09:53.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] conn(0x7f94380ad710 msgr2=0x7f94380afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:53.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] conn(0x7f94380ad710 0x7f94380afbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94380a3cd0 msgr2=0x7f94380a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f94380a3cd0 0x7f94380a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f945019a9c0 tx=0x7f944000a380 comp rx=0 tx=0).stop 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94501094c0 msgr2=0x7f9450199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 --2- 172.21.15.90:0/3141873516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94501094c0 0x7f9450199880 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f944800b830 tx=0x7f9448004ce0 comp rx=0 tx=0).stop 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f944ffff700 1 -- 172.21.15.90:0/3141873516 reap_dead start 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 shutdown_connections 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.079+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 >> 172.21.15.90:0/3141873516 conn(0x7f945007a580 msgr2=0x7f94501021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.080+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 shutdown_connections 2024-04-26T18:09:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:53.080+0000 7f9455f5a700 1 -- 172.21.15.90:0/3141873516 wait complete. 2024-04-26T18:09:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:52 smithi119 ceph-mon[32752]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:09:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:52 smithi167 ceph-mon[32751]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:09:54.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.365+0000 7f5997be1700 1 -- 172.21.15.90:0/4033170447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990108390 msgr2=0x7f5990108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:54.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.365+0000 7f5997be1700 1 --2- 172.21.15.90:0/4033170447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990108390 0x7f5990108770 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f598000b0a0 tx=0x7f598000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 -- 172.21.15.90:0/4033170447 shutdown_connections 2024-04-26T18:09:54.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 --2- 172.21.15.90:0/4033170447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990108390 0x7f5990108770 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 --2- 172.21.15.90:0/4033170447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5990102cf0 0x7f599010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 --2- 172.21.15.90:0/4033170447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5990102350 0x7f59901027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 -- 172.21.15.90:0/4033170447 >> 172.21.15.90:0/4033170447 conn(0x7f59900792c0 msgr2=0x7f59900796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 -- 172.21.15.90:0/4033170447 shutdown_connections 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 -- 172.21.15.90:0/4033170447 wait complete. 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 Processor -- start 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 -- start start 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5990102350 0x7f599019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.366+0000 7f5997be1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990102cf0 0x7f599019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f5997be1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5990108390 0x7f5990198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f5997be1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5990112480 con 0x7f5990102cf0 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f5997be1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5990112300 con 0x7f5990102350 2024-04-26T18:09:54.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f5997be1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5990112600 con 0x7f5990108390 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990102cf0 0x7f599019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990102cf0 0x7f599019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47400/0 (socket says 172.21.15.90:47400) 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 -- 172.21.15.90:0/3734133374 learned_addr learned my addr 172.21.15.90:0/3734133374 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599597d700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5990102350 0x7f599019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599617e700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5990108390 0x7f5990198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 -- 172.21.15.90:0/3734133374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5990108390 msgr2=0x7f5990198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5990108390 0x7f5990198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 -- 172.21.15.90:0/3734133374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5990102350 msgr2=0x7f599019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5990102350 0x7f599019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 -- 172.21.15.90:0/3734133374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5990199040 con 0x7f5990102cf0 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f599517c700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990102cf0 0x7f599019eb80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f598400c9b0 tx=0x7f598400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984007b00 con 0x7f5990102cf0 2024-04-26T18:09:54.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5990199330 con 0x7f5990102cf0 2024-04-26T18:09:54.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5984007ca0 con 0x7f5990102cf0 2024-04-26T18:09:54.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984018660 con 0x7f5990102cf0 2024-04-26T18:09:54.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.367+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59901a55c0 con 0x7f5990102cf0 2024-04-26T18:09:54.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.368+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f599006be60 con 0x7f5990102cf0 2024-04-26T18:09:54.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.369+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f598400f450 con 0x7f5990102cf0 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.370+0000 7f598effd700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f59780a3cd0 0x7f59780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.370+0000 7f599597d700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f59780a3cd0 0x7f59780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.370+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f59840d0a60 con 0x7f5990102cf0 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.370+0000 7f598effd700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] conn(0x7f59780ad710 0x7f59780afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.370+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 --> [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f59780b0280 con 0x7f59780ad710 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.370+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f59840d0f40 con 0x7f5990102cf0 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.370+0000 7f599617e700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] conn(0x7f59780ad710 0x7f59780afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.371+0000 7f599597d700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f59780a3cd0 0x7f59780a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5980009fd0 tx=0x7f598000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.371+0000 7f599617e700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] conn(0x7f59780ad710 0x7f59780afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:54.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.371+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== osd.0 v2:172.21.15.90:6802/1246374052 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f59780b0280 con 0x7f59780ad710 2024-04-26T18:09:54.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.393+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 --> [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f599006c260 con 0x7f59780ad710 2024-04-26T18:09:54.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f598effd700 1 -- 172.21.15.90:0/3734133374 <== osd.0 v2:172.21.15.90:6802/1246374052 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f599006c260 con 0x7f59780ad710 2024-04-26T18:09:54.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] conn(0x7f59780ad710 msgr2=0x7f59780afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:54.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] conn(0x7f59780ad710 0x7f59780afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f59780a3cd0 msgr2=0x7f59780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f59780a3cd0 0x7f59780a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5980009fd0 tx=0x7f598000b7b0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990102cf0 msgr2=0x7f599019eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 --2- 172.21.15.90:0/3734133374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5990102cf0 0x7f599019eb80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f598400c9b0 tx=0x7f598400ccc0 comp rx=0 tx=0).stop 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f599617e700 1 -- 172.21.15.90:0/3734133374 reap_dead start 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 shutdown_connections 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 >> 172.21.15.90:0/3734133374 conn(0x7f59900792c0 msgr2=0x7f5990100fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 shutdown_connections 2024-04-26T18:09:54.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:54.394+0000 7f5997be1700 1 -- 172.21.15.90:0/3734133374 wait complete. 2024-04-26T18:09:55.055 INFO:teuthology.orchestra.run.smithi090.stdout:34359738453 2024-04-26T18:09:55.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T18:09:55.250 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:54 smithi090 ceph-mon[29180]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:09:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:54 smithi119 ceph-mon[32752]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:09:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:54 smithi167 ceph-mon[32751]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:09:56.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:56 smithi090 ceph-mon[29180]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:09:56.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:09:56.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:09:56.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:56 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:56 smithi119 ceph-mon[32752]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:09:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:09:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:09:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:56 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:56 smithi167 ceph-mon[32751]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:09:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:09:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:09:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:56 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:09:56.971 INFO:teuthology.orchestra.run.smithi090.stdout:261993005100 2024-04-26T18:09:56.971 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T18:09:58.986 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:09:58 smithi090 ceph-mon[29180]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:09:59.265 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:09:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:09:58 smithi119 ceph-mon[32752]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:09:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:09:58 smithi167 ceph-mon[32751]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:09:59.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.932+0000 7f2a808ea700 1 -- 172.21.15.90:0/3331024614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c101990 msgr2=0x7f2a7c101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:59.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.932+0000 7f2a808ea700 1 --2- 172.21.15.90:0/3331024614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c101990 0x7f2a7c101e10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2a700099c0 tx=0x7f2a70009cd0 comp rx=0 tx=0).stop 2024-04-26T18:09:59.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 -- 172.21.15.90:0/3331024614 shutdown_connections 2024-04-26T18:09:59.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 --2- 172.21.15.90:0/3331024614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7c102350 0x7f2a7c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:59.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 --2- 172.21.15.90:0/3331024614 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a7c109510 0x7f2a7c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:59.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 --2- 172.21.15.90:0/3331024614 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c101990 0x7f2a7c101e10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:59.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 -- 172.21.15.90:0/3331024614 >> 172.21.15.90:0/3331024614 conn(0x7f2a7c078920 msgr2=0x7f2a7c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 -- 172.21.15.90:0/3331024614 shutdown_connections 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 -- 172.21.15.90:0/3331024614 wait complete. 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 Processor -- start 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.933+0000 7f2a808ea700 1 -- start start 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a7c101990 0x7f2a7c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c102350 0x7f2a7c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7c109510 0x7f2a7c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c113670 con 0x7f2a7c102350 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1134f0 con 0x7f2a7c109510 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c1137f0 con 0x7f2a7c101990 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c102350 0x7f2a7c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c102350 0x7f2a7c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47422/0 (socket says 172.21.15.90:47422) 2024-04-26T18:09:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 -- 172.21.15.90:0/2983708736 learned_addr learned my addr 172.21.15.90:0/2983708736 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a7ad9d700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7c109510 0x7f2a7c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 -- 172.21.15.90:0/2983708736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a7c101990 msgr2=0x7f2a7c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a7a59c700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a7c101990 0x7f2a7c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a7c101990 0x7f2a7c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 -- 172.21.15.90:0/2983708736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7c109510 msgr2=0x7f2a7c199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a7c109510 0x7f2a7c199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 -- 172.21.15.90:0/2983708736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c19a090 con 0x7f2a7c102350 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a7a59c700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a7c101990 0x7f2a7c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a79d9b700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c102350 0x7f2a7c19fc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a70004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70026070 con 0x7f2a7c102350 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7000f460 con 0x7f2a7c102350 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c19a320 con 0x7f2a7c102350 2024-04-26T18:09:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70009e60 con 0x7f2a7c102350 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.934+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c1a65f0 con 0x7f2a7c102350 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.935+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2a7c063bb0 con 0x7f2a7c102350 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.936+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2a70029070 con 0x7f2a7c102350 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a6b7fe700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2a640a3cd0 0x7f2a640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a7a59c700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2a640a3cd0 0x7f2a640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f2a700d1160 con 0x7f2a7c102350 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a6b7fe700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] conn(0x7f2a640ad710 0x7f2a640afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 --> [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a640b0280 con 0x7f2a640ad710 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f2a700d1640 con 0x7f2a7c102350 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a7a59c700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2a640a3cd0 0x7f2a640a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c108640 tx=0x7f2a6c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.937+0000 7f2a7ad9d700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] conn(0x7f2a640ad710 0x7f2a640afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:09:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.938+0000 7f2a7ad9d700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] conn(0x7f2a640ad710 0x7f2a640afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:09:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.938+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== osd.3 v2:172.21.15.90:6810/2988562238 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2a640b0280 con 0x7f2a640ad710 2024-04-26T18:09:59.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.956+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 --> [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2a7c064030 con 0x7f2a640ad710 2024-04-26T18:09:59.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a6b7fe700 1 -- 172.21.15.90:0/2983708736 <== osd.3 v2:172.21.15.90:6810/2988562238 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2a7c064030 con 0x7f2a640ad710 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] conn(0x7f2a640ad710 msgr2=0x7f2a640afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] conn(0x7f2a640ad710 0x7f2a640afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2a640a3cd0 msgr2=0x7f2a640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f2a640a3cd0 0x7f2a640a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c108640 tx=0x7f2a6c008040 comp rx=0 tx=0).stop 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c102350 msgr2=0x7f2a7c19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 --2- 172.21.15.90:0/2983708736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a7c102350 0x7f2a7c19fc60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a70006010 tx=0x7f2a70004690 comp rx=0 tx=0).stop 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a7ad9d700 1 -- 172.21.15.90:0/2983708736 reap_dead start 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 shutdown_connections 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.957+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 >> 172.21.15.90:0/2983708736 conn(0x7f2a7c078920 msgr2=0x7f2a7c1000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.958+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 shutdown_connections 2024-04-26T18:09:59.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:09:59.958+0000 7f2a808ea700 1 -- 172.21.15.90:0/2983708736 wait complete. 2024-04-26T18:10:00.588 INFO:teuthology.orchestra.run.smithi090.stdout:107374182471 2024-04-26T18:10:00.588 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T18:10:01.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:00 smithi090 ceph-mon[29180]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:01.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:00 smithi090 ceph-mon[29180]: overall HEALTH_OK 2024-04-26T18:10:01.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 -- 172.21.15.90:0/329430335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b4109490 msgr2=0x7fa4b4109870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:01.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 --2- 172.21.15.90:0/329430335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b4109490 0x7fa4b4109870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00b0d0 tx=0x7fa4ac00b3e0 comp rx=0 tx=0).stop 2024-04-26T18:10:01.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 -- 172.21.15.90:0/329430335 shutdown_connections 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 --2- 172.21.15.90:0/329430335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b4109490 0x7fa4b4109870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 --2- 172.21.15.90:0/329430335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4b406c580 0x7fa4b41102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 --2- 172.21.15.90:0/329430335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b406bbc0 0x7fa4b406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 -- 172.21.15.90:0/329430335 >> 172.21.15.90:0/329430335 conn(0x7fa4b4078390 msgr2=0x7fa4b40787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 -- 172.21.15.90:0/329430335 shutdown_connections 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.218+0000 7fa4bae68700 1 -- 172.21.15.90:0/329430335 wait complete. 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 Processor -- start 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 -- start start 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b406bbc0 0x7fa4b41a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4b406c580 0x7fa4b41a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:01.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b4109490 0x7fa4b419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b41135f0 con 0x7fa4b406bbc0 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4113470 con 0x7fa4b4109490 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4bae68700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b4113770 con 0x7fa4b406c580 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4b8c04700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b406bbc0 0x7fa4b41a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4b8c04700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b406bbc0 0x7fa4b41a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47440/0 (socket says 172.21.15.90:47440) 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.219+0000 7fa4b8c04700 1 -- 172.21.15.90:0/1852074750 learned_addr learned my addr 172.21.15.90:0/1852074750 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b3fff700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4b406c580 0x7fa4b41a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b8c04700 1 -- 172.21.15.90:0/1852074750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4b406c580 msgr2=0x7fa4b41a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b8c04700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa4b406c580 0x7fa4b41a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:01.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b8c04700 1 -- 172.21.15.90:0/1852074750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b4109490 msgr2=0x7fa4b419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b8c04700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4b4109490 0x7fa4b419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b8c04700 1 -- 172.21.15.90:0/1852074750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b419c260 con 0x7fa4b406bbc0 2024-04-26T18:10:01.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b8c04700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b406bbc0 0x7fa4b41a1940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac007a20 tx=0x7fa4ac007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:01.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac01f070 con 0x7fa4b406bbc0 2024-04-26T18:10:01.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4ac005520 con 0x7fa4b406bbc0 2024-04-26T18:10:01.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b419c4f0 con 0x7fa4b406bbc0 2024-04-26T18:10:01.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac01b7f0 con 0x7fa4b406bbc0 2024-04-26T18:10:01.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.220+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b41a8810 con 0x7fa4b406bbc0 2024-04-26T18:10:01.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.221+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa4b410f000 con 0x7fa4b406bbc0 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.222+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa4ac022070 con 0x7fa4b406bbc0 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.223+0000 7fa4b1ffb700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa4a00a3cd0 0x7fa4a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.223+0000 7fa4b3fff700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa4a00a3cd0 0x7fa4a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.223+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fa4ac00f020 con 0x7fa4b406bbc0 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.223+0000 7fa4b1ffb700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] conn(0x7fa4a00ad710 0x7fa4a00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.223+0000 7fa4b3fff700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa4a00a3cd0 0x7fa4a00a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005fd0 tx=0x7fa4a4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.223+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 --> [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa4a00b0280 con 0x7fa4a00ad710 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.223+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac0d1e70 con 0x7fa4b406bbc0 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.224+0000 7fa4b9405700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] conn(0x7fa4a00ad710 0x7fa4a00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:01.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.224+0000 7fa4b9405700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] conn(0x7fa4a00ad710 0x7fa4a00afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:01.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.225+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== osd.7 v2:172.21.15.119:6816/146780393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa4a00b0280 con 0x7fa4a00ad710 2024-04-26T18:10:01.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.242+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 --> [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa4b4104b80 con 0x7fa4a00ad710 2024-04-26T18:10:01.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.242+0000 7fa4b1ffb700 1 -- 172.21.15.90:0/1852074750 <== osd.7 v2:172.21.15.119:6816/146780393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa4b4104b80 con 0x7fa4a00ad710 2024-04-26T18:10:01.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 >> [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] conn(0x7fa4a00ad710 msgr2=0x7fa4a00afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:01.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] conn(0x7fa4a00ad710 0x7fa4a00afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:01.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa4a00a3cd0 msgr2=0x7fa4a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:01.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fa4a00a3cd0 0x7fa4a00a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4005fd0 tx=0x7fa4a4005f60 comp rx=0 tx=0).stop 2024-04-26T18:10:01.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b406bbc0 msgr2=0x7fa4b41a1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:01.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 --2- 172.21.15.90:0/1852074750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4b406bbc0 0x7fa4b41a1940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac007a20 tx=0x7fa4ac007b00 comp rx=0 tx=0).stop 2024-04-26T18:10:01.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4b9405700 1 -- 172.21.15.90:0/1852074750 reap_dead start 2024-04-26T18:10:01.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 shutdown_connections 2024-04-26T18:10:01.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 >> 172.21.15.90:0/1852074750 conn(0x7fa4b4078390 msgr2=0x7fa4b410e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:01.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 shutdown_connections 2024-04-26T18:10:01.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:01.243+0000 7fa4bae68700 1 -- 172.21.15.90:0/1852074750 wait complete. 2024-04-26T18:10:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:00 smithi119 ceph-mon[32752]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:00 smithi119 ceph-mon[32752]: overall HEALTH_OK 2024-04-26T18:10:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:00 smithi167 ceph-mon[32751]: pgmap v336: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:00 smithi167 ceph-mon[32751]: overall HEALTH_OK 2024-04-26T18:10:03.247 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:02 smithi090 ceph-mon[29180]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:02 smithi119 ceph-mon[32752]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:02 smithi167 ceph-mon[32751]: pgmap v337: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:03.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 -- 172.21.15.90:0/4134738258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 msgr2=0x7feb181038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 --2- 172.21.15.90:0/4134738258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 0x7feb181038f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feb08009990 tx=0x7feb08009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 -- 172.21.15.90:0/4134738258 shutdown_connections 2024-04-26T18:10:03.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 --2- 172.21.15.90:0/4134738258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb18103e30 0x7feb18110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 --2- 172.21.15.90:0/4134738258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb181094c0 0x7feb181098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 --2- 172.21.15.90:0/4134738258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 0x7feb181038f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 -- 172.21.15.90:0/4134738258 >> 172.21.15.90:0/4134738258 conn(0x7feb1807a580 msgr2=0x7feb1807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:03.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 -- 172.21.15.90:0/4134738258 shutdown_connections 2024-04-26T18:10:03.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.350+0000 7feb20228700 1 -- 172.21.15.90:0/4134738258 wait complete. 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 Processor -- start 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 -- start start 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 0x7feb1819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb18103e30 0x7feb1819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb181094c0 0x7feb181999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feb18113640 con 0x7feb18103470 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feb181134c0 con 0x7feb18103e30 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb20228700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb181137c0 con 0x7feb181094c0 2024-04-26T18:10:03.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 0x7feb1819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 0x7feb1819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52094/0 (socket says 172.21.15.90:52094) 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 -- 172.21.15.90:0/2806298260 learned_addr learned my addr 172.21.15.90:0/2806298260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1e7c5700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb181094c0 0x7feb181999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 -- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb181094c0 msgr2=0x7feb181999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb181094c0 0x7feb181999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 -- 172.21.15.90:0/2806298260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb18103e30 msgr2=0x7feb1819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb18103e30 0x7feb1819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1dfc4700 1 -- 172.21.15.90:0/2806298260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1819a100 con 0x7feb18103470 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.351+0000 7feb1e7c5700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb181094c0 0x7feb181999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.352+0000 7feb1dfc4700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 0x7feb1819f7e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c9b0 tx=0x7feb1400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.352+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14007b00 con 0x7feb18103470 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.352+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb14007ca0 con 0x7feb18103470 2024-04-26T18:10:03.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.352+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1819a3f0 con 0x7feb18103470 2024-04-26T18:10:03.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.352+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb181a66b0 con 0x7feb18103470 2024-04-26T18:10:03.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.352+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14018730 con 0x7feb18103470 2024-04-26T18:10:03.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.353+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb18063b30 con 0x7feb18103470 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.354+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feb1400f450 con 0x7feb18103470 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.354+0000 7feb0effd700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7feb040a3cd0 0x7feb040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.354+0000 7feb1d7c3700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7feb040a3cd0 0x7feb040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.354+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7feb140d09a0 con 0x7feb18103470 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.355+0000 7feb0effd700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] conn(0x7feb040ad710 0x7feb040afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.355+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 --> [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb14004500 con 0x7feb040ad710 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.355+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7feb140d0e80 con 0x7feb18103470 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.355+0000 7feb1d7c3700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7feb040a3cd0 0x7feb040a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb0800b5c0 tx=0x7feb08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.355+0000 7feb1e7c5700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] conn(0x7feb040ad710 0x7feb040afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.355+0000 7feb1e7c5700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] conn(0x7feb040ad710 0x7feb040afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:03.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.356+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== osd.5 v2:172.21.15.167:6808/2447709216 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb14004500 con 0x7feb040ad710 2024-04-26T18:10:03.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.372+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 --> [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb18063f30 con 0x7feb040ad710 2024-04-26T18:10:03.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb0effd700 1 -- 172.21.15.90:0/2806298260 <== osd.5 v2:172.21.15.167:6808/2447709216 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feb18063f30 con 0x7feb040ad710 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] conn(0x7feb040ad710 msgr2=0x7feb040afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb20228700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] conn(0x7feb040ad710 0x7feb040afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7feb040a3cd0 msgr2=0x7feb040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb20228700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7feb040a3cd0 0x7feb040a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb0800b5c0 tx=0x7feb08005fb0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 msgr2=0x7feb1819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb20228700 1 --2- 172.21.15.90:0/2806298260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb18103470 0x7feb1819f7e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7feb1400c9b0 tx=0x7feb1400cd70 comp rx=0 tx=0).stop 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.373+0000 7feb1e7c5700 1 -- 172.21.15.90:0/2806298260 reap_dead start 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.374+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 shutdown_connections 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.374+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 >> 172.21.15.90:0/2806298260 conn(0x7feb1807a580 msgr2=0x7feb18102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.374+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 shutdown_connections 2024-04-26T18:10:03.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.374+0000 7feb20228700 1 -- 172.21.15.90:0/2806298260 wait complete. 2024-04-26T18:10:03.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 -- 172.21.15.90:0/208605387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 msgr2=0x7fe4981098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 --2- 172.21.15.90:0/208605387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 0x7fe4981098a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe48c009990 tx=0x7fe48c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 -- 172.21.15.90:0/208605387 shutdown_connections 2024-04-26T18:10:03.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 --2- 172.21.15.90:0/208605387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 0x7fe4981098a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 --2- 172.21.15.90:0/208605387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe498103e30 0x7fe498110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 --2- 172.21.15.90:0/208605387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498103470 0x7fe4981038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 -- 172.21.15.90:0/208605387 >> 172.21.15.90:0/208605387 conn(0x7fe49807a580 msgr2=0x7fe49807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.599+0000 7fe49cd58700 1 -- 172.21.15.90:0/208605387 shutdown_connections 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 -- 172.21.15.90:0/208605387 wait complete. 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 Processor -- start 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 -- start start 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498103470 0x7fe49819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe498103e30 0x7fe49819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 0x7fe498199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe4981135f0 con 0x7fe4981094c0 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe498113470 con 0x7fe498103e30 2024-04-26T18:10:03.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.600+0000 7fe49cd58700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe498113770 con 0x7fe498103470 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 0x7fe498199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 0x7fe498199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52126/0 (socket says 172.21.15.90:52126) 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 -- 172.21.15.90:0/2361590731 learned_addr learned my addr 172.21.15.90:0/2361590731 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe495d9b700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe498103e30 0x7fe49819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe49659c700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498103470 0x7fe49819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 -- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498103470 msgr2=0x7fe49819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe498103470 0x7fe49819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 -- 172.21.15.90:0/2361590731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe498103e30 msgr2=0x7fe49819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe498103e30 0x7fe49819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 -- 172.21.15.90:0/2361590731 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49819a0d0 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe495d9b700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe498103e30 0x7fe49819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe496d9d700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 0x7fe498199970 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe490007ee0 tx=0x7fe49000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49000dbd0 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe49000dd70 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49819a3c0 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490005990 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.601+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4981a6680 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.602+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe498063b30 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.603+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe49001a020 con 0x7fe4981094c0 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe4877fe700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe4800a3cd0 0x7fe4800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe49659c700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe4800a3cd0 0x7fe4800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fe4900d0340 con 0x7fe4981094c0 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe4877fe700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] conn(0x7fe4800ad710 0x7fe4800afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 --> [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe49000ae50 con 0x7fe4800ad710 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fe4900d0820 con 0x7fe4981094c0 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe49659c700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe4800a3cd0 0x7fe4800a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe48c000c00 tx=0x7fe48c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.604+0000 7fe495d9b700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] conn(0x7fe4800ad710 0x7fe4800afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.605+0000 7fe495d9b700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] conn(0x7fe4800ad710 0x7fe4800afbb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.606+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== osd.11 v2:172.21.15.167:6824/3743670962 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe49000ae50 con 0x7fe4800ad710 2024-04-26T18:10:03.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.622+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 --> [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe498063fb0 con 0x7fe4800ad710 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe4877fe700 1 -- 172.21.15.90:0/2361590731 <== osd.11 v2:172.21.15.167:6824/3743670962 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe498063fb0 con 0x7fe4800ad710 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] conn(0x7fe4800ad710 msgr2=0x7fe4800afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] conn(0x7fe4800ad710 0x7fe4800afbb0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe4800a3cd0 msgr2=0x7fe4800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe4800a3cd0 0x7fe4800a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe48c000c00 tx=0x7fe48c005fb0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 msgr2=0x7fe498199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 --2- 172.21.15.90:0/2361590731 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4981094c0 0x7fe498199970 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe490007ee0 tx=0x7fe49000d3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe496d9d700 1 -- 172.21.15.90:0/2361590731 reap_dead start 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 shutdown_connections 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 >> 172.21.15.90:0/2361590731 conn(0x7fe49807a580 msgr2=0x7fe4981022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 shutdown_connections 2024-04-26T18:10:03.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:03.623+0000 7fe49cd58700 1 -- 172.21.15.90:0/2361590731 wait complete. 2024-04-26T18:10:04.653 INFO:teuthology.orchestra.run.smithi090.stdout:227633266739 2024-04-26T18:10:04.654 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T18:10:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:04 smithi119 ceph-mon[32752]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:04 smithi167 ceph-mon[32751]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:04 smithi090 ceph-mon[29180]: pgmap v338: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:05.834 INFO:teuthology.orchestra.run.smithi090.stdout:352187318303 2024-04-26T18:10:05.834 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 -- 172.21.15.90:0/3046476133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406bbc0 msgr2=0x7f872406c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 --2- 172.21.15.90:0/3046476133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406bbc0 0x7f872406c040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f87140099c0 tx=0x7f8714009cd0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 -- 172.21.15.90:0/3046476133 shutdown_connections 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 --2- 172.21.15.90:0/3046476133 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f872406c580 0x7f87241102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 --2- 172.21.15.90:0/3046476133 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8724109490 0x7f8724109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 --2- 172.21.15.90:0/3046476133 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406bbc0 0x7f872406c040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 -- 172.21.15.90:0/3046476133 >> 172.21.15.90:0/3046476133 conn(0x7f8724078390 msgr2=0x7f87240787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 -- 172.21.15.90:0/3046476133 shutdown_connections 2024-04-26T18:10:06.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 -- 172.21.15.90:0/3046476133 wait complete. 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.165+0000 7f872ab8a700 1 Processor -- start 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f872ab8a700 1 -- start start 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f872ab8a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f872406bbc0 0x7f872419f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f872ab8a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406c580 0x7f872419fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f872ab8a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8724109490 0x7f8724199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f872ab8a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f87241136c0 con 0x7f872406c580 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f872ab8a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8724113540 con 0x7f872406bbc0 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f872ab8a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8724113840 con 0x7f8724109490 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406c580 0x7f872419fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406c580 0x7f872419fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52160/0 (socket says 172.21.15.90:52160) 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 -- 172.21.15.90:0/1028409908 learned_addr learned my addr 172.21.15.90:0/1028409908 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f8728926700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f872406bbc0 0x7f872419f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f8729127700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8724109490 0x7f8724199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 -- 172.21.15.90:0/1028409908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8724109490 msgr2=0x7f8724199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8724109490 0x7f8724199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 -- 172.21.15.90:0/1028409908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f872406bbc0 msgr2=0x7f872419f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f872406bbc0 0x7f872419f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 -- 172.21.15.90:0/1028409908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872419a180 con 0x7f872406c580 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f871bfff700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406c580 0x7f872419fda0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f871400b580 tx=0x7f87140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.166+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714027070 con 0x7f872406c580 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.167+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f872419a410 con 0x7f872406c580 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.167+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87140040a0 con 0x7f872406c580 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.167+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f871400f460 con 0x7f872406c580 2024-04-26T18:10:06.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.167+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87241a6730 con 0x7f872406c580 2024-04-26T18:10:06.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.168+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8724063bb0 con 0x7f872406c580 2024-04-26T18:10:06.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.169+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f871402a070 con 0x7f872406c580 2024-04-26T18:10:06.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8719ffb700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f870c0a3cd0 0x7f870c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8728926700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f870c0a3cd0 0x7f870c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f87140d1c40 con 0x7f872406c580 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8719ffb700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] conn(0x7f870c0ad710 0x7f870c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 --> [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f870c0b0280 con 0x7f870c0ad710 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f87140d2120 con 0x7f872406c580 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8729127700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] conn(0x7f870c0ad710 0x7f870c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.170+0000 7f8728926700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f870c0a3cd0 0x7f870c0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f872419ab60 tx=0x7f8720009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:06.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.171+0000 7f8729127700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] conn(0x7f870c0ad710 0x7f870c0afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:06.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.171+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== osd.9 v2:172.21.15.90:6826/1702789383 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f870c0b0280 con 0x7f870c0ad710 2024-04-26T18:10:06.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.188+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 --> [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8724063fb0 con 0x7f870c0ad710 2024-04-26T18:10:06.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f8719ffb700 1 -- 172.21.15.90:0/1028409908 <== osd.9 v2:172.21.15.90:6826/1702789383 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8724063fb0 con 0x7f870c0ad710 2024-04-26T18:10:06.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] conn(0x7f870c0ad710 msgr2=0x7f870c0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] conn(0x7f870c0ad710 0x7f870c0afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f870c0a3cd0 msgr2=0x7f870c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f870c0a3cd0 0x7f870c0a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f872419ab60 tx=0x7f8720009190 comp rx=0 tx=0).stop 2024-04-26T18:10:06.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406c580 msgr2=0x7f872419fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 --2- 172.21.15.90:0/1028409908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f872406c580 0x7f872419fda0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f871400b580 tx=0x7f87140047b0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f8729127700 1 -- 172.21.15.90:0/1028409908 reap_dead start 2024-04-26T18:10:06.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 shutdown_connections 2024-04-26T18:10:06.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 >> 172.21.15.90:0/1028409908 conn(0x7f8724078390 msgr2=0x7f8724101090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:06.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 shutdown_connections 2024-04-26T18:10:06.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:06.189+0000 7f872ab8a700 1 -- 172.21.15.90:0/1028409908 wait complete. 2024-04-26T18:10:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 -- 172.21.15.90:0/584451628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994101dc0 msgr2=0x7fc994102240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 --2- 172.21.15.90:0/584451628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994101dc0 0x7fc994102240 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc984009990 tx=0x7fc984009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 -- 172.21.15.90:0/584451628 shutdown_connections 2024-04-26T18:10:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 --2- 172.21.15.90:0/584451628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994102780 0x7fc99410e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 --2- 172.21.15.90:0/584451628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc994107b70 0x7fc994107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 --2- 172.21.15.90:0/584451628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994101dc0 0x7fc994102240 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 -- 172.21.15.90:0/584451628 >> 172.21.15.90:0/584451628 conn(0x7fc994077640 msgr2=0x7fc994077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 -- 172.21.15.90:0/584451628 shutdown_connections 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.601+0000 7fc99b804700 1 -- 172.21.15.90:0/584451628 wait complete. 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 Processor -- start 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 -- start start 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994101dc0 0x7fc99419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc994102780 0x7fc99419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994107b70 0x7fc9941a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc994111c90 con 0x7fc994107b70 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc994111b10 con 0x7fc994101dc0 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc99b804700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc994111e10 con 0x7fc994102780 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc999da1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994107b70 0x7fc9941a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc999da1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994107b70 0x7fc9941a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52140/0 (socket says 172.21.15.90:52140) 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc999da1700 1 -- 172.21.15.90:0/2387551003 learned_addr learned my addr 172.21.15.90:0/2387551003 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc9995a0700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994101dc0 0x7fc99419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc998d9f700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc994102780 0x7fc99419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.602+0000 7fc999da1700 1 -- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc994102780 msgr2=0x7fc99419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc999da1700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc994102780 0x7fc99419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc999da1700 1 -- 172.21.15.90:0/2387551003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994101dc0 msgr2=0x7fc99419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc999da1700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994101dc0 0x7fc99419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc999da1700 1 -- 172.21.15.90:0/2387551003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9941a5e40 con 0x7fc994107b70 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc9995a0700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc994101dc0 0x7fc99419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc999da1700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994107b70 0x7fc9941a5900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ada0 tx=0x7fc98c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c017020 con 0x7fc994107b70 2024-04-26T18:10:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c00ce30 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9941a6070 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c005350 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.603+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941a6530 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.604+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc994105ff0 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.605+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc98c00c980 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.605+0000 7fc98a7fc700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc9800a3cd0 0x7fc9800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.605+0000 7fc9995a0700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc9800a3cd0 0x7fc9800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.605+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc98c0d05b0 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.606+0000 7fc98a7fc700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] conn(0x7fc9800ad710 0x7fc9800afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.606+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 --> [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc98c00fe50 con 0x7fc9800ad710 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.606+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fc98c0d0a90 con 0x7fc994107b70 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.606+0000 7fc9995a0700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc9800a3cd0 0x7fc9800a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc990005fd0 tx=0x7fc99000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.606+0000 7fc998d9f700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] conn(0x7fc9800ad710 0x7fc9800afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.606+0000 7fc998d9f700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] conn(0x7fc9800ad710 0x7fc9800afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.607+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== osd.2 v2:172.21.15.167:6800/1709871603 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc98c00fe50 con 0x7fc9800ad710 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.623+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 --> [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc994103200 con 0x7fc9800ad710 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc98a7fc700 1 -- 172.21.15.90:0/2387551003 <== osd.2 v2:172.21.15.167:6800/1709871603 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc994103200 con 0x7fc9800ad710 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] conn(0x7fc9800ad710 msgr2=0x7fc9800afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] conn(0x7fc9800ad710 0x7fc9800afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc9800a3cd0 msgr2=0x7fc9800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc9800a3cd0 0x7fc9800a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc990005fd0 tx=0x7fc99000a5e0 comp rx=0 tx=0).stop 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994107b70 msgr2=0x7fc9941a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 --2- 172.21.15.90:0/2387551003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc994107b70 0x7fc9941a5900 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ada0 tx=0x7fc98c00c330 comp rx=0 tx=0).stop 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc999da1700 1 -- 172.21.15.90:0/2387551003 reap_dead start 2024-04-26T18:10:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 shutdown_connections 2024-04-26T18:10:06.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 >> 172.21.15.90:0/2387551003 conn(0x7fc994077640 msgr2=0x7fc9941055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:06.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 shutdown_connections 2024-04-26T18:10:06.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:05.624+0000 7fc99b804700 1 -- 172.21.15.90:0/2387551003 wait complete. 2024-04-26T18:10:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:06 smithi119 ceph-mon[32752]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:06 smithi167 ceph-mon[32751]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:07.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:06 smithi090 ceph-mon[29180]: pgmap v339: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:07.389 INFO:teuthology.orchestra.run.smithi090.stdout:77309411404 2024-04-26T18:10:07.389 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T18:10:07.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 -- 172.21.15.90:0/2968938045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01023b0 msgr2=0x7fbad01102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:07.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 --2- 172.21.15.90:0/2968938045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01023b0 0x7fbad01102f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbac800b0a0 tx=0x7fbac800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:07.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 -- 172.21.15.90:0/2968938045 shutdown_connections 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 --2- 172.21.15.90:0/2968938045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01023b0 0x7fbad01102f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 --2- 172.21.15.90:0/2968938045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad01094b0 0x7fbad0109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 --2- 172.21.15.90:0/2968938045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad0101a10 0x7fbad0101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 -- 172.21.15.90:0/2968938045 >> 172.21.15.90:0/2968938045 conn(0x7fbad0078990 msgr2=0x7fbad00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 -- 172.21.15.90:0/2968938045 shutdown_connections 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 -- 172.21.15.90:0/2968938045 wait complete. 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 Processor -- start 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.629+0000 7fbad822a700 1 -- start start 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0101a10 0x7fbad019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad01023b0 0x7fbad019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01094b0 0x7fbad0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbad01135b0 con 0x7fbad01094b0 2024-04-26T18:10:07.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0113430 con 0x7fbad01023b0 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbad0113730 con 0x7fbad0101a10 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01094b0 0x7fbad0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01094b0 0x7fbad0199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52180/0 (socket says 172.21.15.90:52180) 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 -- 172.21.15.90:0/906446273 learned_addr learned my addr 172.21.15.90:0/906446273 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad57c5700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad01023b0 0x7fbad019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 -- 172.21.15.90:0/906446273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0101a10 msgr2=0x7fbad019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbad0101a10 0x7fbad019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 -- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad01023b0 msgr2=0x7fbad019fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbad01023b0 0x7fbad019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 -- 172.21.15.90:0/906446273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad019a050 con 0x7fbad01094b0 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad67c7700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01094b0 0x7fbad0199870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbac800b830 tx=0x7fbac8004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8021070 con 0x7fbad01094b0 2024-04-26T18:10:07.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad019a2e0 con 0x7fbad01094b0 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbac8003c00 con 0x7fbad01094b0 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac8004730 con 0x7fbad01094b0 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.630+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad01a6500 con 0x7fbad01094b0 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.631+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbad0063b70 con 0x7fbad01094b0 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.632+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbac8003db0 con 0x7fbad01094b0 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbac6ffd700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbabc0a3cd0 0x7fbabc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbad5fc6700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbabc0a3cd0 0x7fbabc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:07.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fbac800f020 con 0x7fbad01094b0 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbac6ffd700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] conn(0x7fbabc0ad710 0x7fbabc0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 --> [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbac801de50 con 0x7fbabc0ad710 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbac80d2270 con 0x7fbad01094b0 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbad5fc6700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbabc0a3cd0 0x7fbabc0a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005fd0 tx=0x7fbacc005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.633+0000 7fbad57c5700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] conn(0x7fbabc0ad710 0x7fbabc0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.634+0000 7fbad57c5700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] conn(0x7fbabc0ad710 0x7fbabc0afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.635+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== osd.10 v2:172.21.15.119:6824/249896871 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbac801de50 con 0x7fbabc0ad710 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.650+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 --> [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbad0063f70 con 0x7fbabc0ad710 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbac6ffd700 1 -- 172.21.15.90:0/906446273 <== osd.10 v2:172.21.15.119:6824/249896871 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbad0063f70 con 0x7fbabc0ad710 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] conn(0x7fbabc0ad710 msgr2=0x7fbabc0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] conn(0x7fbabc0ad710 0x7fbabc0afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbabc0a3cd0 msgr2=0x7fbabc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:07.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbabc0a3cd0 0x7fbabc0a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbacc005fd0 tx=0x7fbacc005f00 comp rx=0 tx=0).stop 2024-04-26T18:10:07.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01094b0 msgr2=0x7fbad0199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:07.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 --2- 172.21.15.90:0/906446273 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad01094b0 0x7fbad0199870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbac800b830 tx=0x7fbac8004d90 comp rx=0 tx=0).stop 2024-04-26T18:10:07.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad67c7700 1 -- 172.21.15.90:0/906446273 reap_dead start 2024-04-26T18:10:07.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 shutdown_connections 2024-04-26T18:10:07.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 >> 172.21.15.90:0/906446273 conn(0x7fbad0078990 msgr2=0x7fbad01063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:07.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 shutdown_connections 2024-04-26T18:10:07.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:07.651+0000 7fbad822a700 1 -- 172.21.15.90:0/906446273 wait complete. 2024-04-26T18:10:08.296 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:08.297 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:08 smithi090 ceph-mon[29180]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T18:10:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:08 smithi119 ceph-mon[32752]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T18:10:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:08 smithi167 ceph-mon[32751]: pgmap v340: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-26T18:10:09.608 INFO:teuthology.orchestra.run.smithi090.stdout:167503724605 2024-04-26T18:10:09.608 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T18:10:10.297 INFO:teuthology.orchestra.run.smithi090.stdout:317827579940 2024-04-26T18:10:10.297 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T18:10:11.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:10 smithi090 ceph-mon[29180]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T18:10:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:10 smithi119 ceph-mon[32752]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T18:10:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:10 smithi167 ceph-mon[32751]: pgmap v341: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T18:10:11.747 INFO:teuthology.orchestra.run.smithi090.stdout:296352743466 2024-04-26T18:10:11.747 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 -- 172.21.15.90:0/1768447044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f236c06bad0 msgr2=0x7f236c1102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 --2- 172.21.15.90:0/1768447044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f236c06bad0 0x7f236c1102e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f236400b0a0 tx=0x7f236400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 -- 172.21.15.90:0/1768447044 shutdown_connections 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 --2- 172.21.15.90:0/1768447044 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f236c06bad0 0x7f236c1102e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 --2- 172.21.15.90:0/1768447044 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f236c1094a0 0x7f236c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 --2- 172.21.15.90:0/1768447044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c06b130 0x7f236c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 -- 172.21.15.90:0/1768447044 >> 172.21.15.90:0/1768447044 conn(0x7f236c0792d0 msgr2=0x7f236c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 -- 172.21.15.90:0/1768447044 shutdown_connections 2024-04-26T18:10:11.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.933+0000 7f2371ec6700 1 -- 172.21.15.90:0/1768447044 wait complete. 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.934+0000 7f2371ec6700 1 Processor -- start 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.934+0000 7f2371ec6700 1 -- start start 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f2371ec6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c06b130 0x7f236c1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f2371ec6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f236c06bad0 0x7f236c102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f2371ec6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f236c1094a0 0x7f236c102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f2371ec6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1135a0 con 0x7f236c06bad0 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f2371ec6700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f236c113420 con 0x7f236c06b130 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f2371ec6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f236c113720 con 0x7f236c1094a0 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c06b130 0x7f236c1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c06b130 0x7f236c1023f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:35308/0 (socket says 172.21.15.90:35308) 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 -- 172.21.15.90:0/4150746965 learned_addr learned my addr 172.21.15.90:0/4150746965 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236affd700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f236c06bad0 0x7f236c102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236bfff700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f236c1094a0 0x7f236c102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:11.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 -- 172.21.15.90:0/4150746965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f236c1094a0 msgr2=0x7f236c102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f236c1094a0 0x7f236c102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 -- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f236c06bad0 msgr2=0x7f236c102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f236c06bad0 0x7f236c102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.935+0000 7f236b7fe700 1 -- 172.21.15.90:0/4150746965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c1a7fb0 con 0x7f236c06b130 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.936+0000 7f236b7fe700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c06b130 0x7f236c1023f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f235c00d950 tx=0x7f235c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.936+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c016070 con 0x7f236c06b130 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.936+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c1a82a0 con 0x7f236c06b130 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.936+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f235c009df0 con 0x7f236c06b130 2024-04-26T18:10:11.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.936+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a8830 con 0x7f236c06b130 2024-04-26T18:10:11.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.937+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c00f3c0 con 0x7f236c06b130 2024-04-26T18:10:11.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.937+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f236c063af0 con 0x7f236c06b130 2024-04-26T18:10:11.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.939+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f235c019070 con 0x7f236c06b130 2024-04-26T18:10:11.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f2368ff9700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f23540a3d50 0x7f23540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:11.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f236affd700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f23540a3d50 0x7f23540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:11.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f235c0d03d0 con 0x7f236c06b130 2024-04-26T18:10:11.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f2368ff9700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] conn(0x7f23540ad790 0x7f23540afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:11.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 --> [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f235c014e80 con 0x7f23540ad790 2024-04-26T18:10:11.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7f235c0d08b0 con 0x7f236c06b130 2024-04-26T18:10:11.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f236bfff700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] conn(0x7f23540ad790 0x7f23540afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:11.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.940+0000 7f236affd700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f23540a3d50 0x7f23540a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f236c108aa0 tx=0x7f236000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:11.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.941+0000 7f236bfff700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] conn(0x7f23540ad790 0x7f23540afc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:11.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.942+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== osd.4 v2:172.21.15.119:6808/2105013830 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f235c014e80 con 0x7f23540ad790 2024-04-26T18:10:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.963+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 --> [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f236c063d00 con 0x7f23540ad790 2024-04-26T18:10:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2368ff9700 1 -- 172.21.15.90:0/4150746965 <== osd.4 v2:172.21.15.119:6808/2105013830 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f236c063d00 con 0x7f23540ad790 2024-04-26T18:10:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] conn(0x7f23540ad790 msgr2=0x7f23540afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] conn(0x7f23540ad790 0x7f23540afc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f23540a3d50 msgr2=0x7f23540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f23540a3d50 0x7f23540a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f236c108aa0 tx=0x7f236000b410 comp rx=0 tx=0).stop 2024-04-26T18:10:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c06b130 msgr2=0x7f236c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 --2- 172.21.15.90:0/4150746965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f236c06b130 0x7f236c1023f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f235c00d950 tx=0x7f235c00dc60 comp rx=0 tx=0).stop 2024-04-26T18:10:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f236bfff700 1 -- 172.21.15.90:0/4150746965 reap_dead start 2024-04-26T18:10:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 shutdown_connections 2024-04-26T18:10:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 >> 172.21.15.90:0/4150746965 conn(0x7f236c0792d0 msgr2=0x7f236c100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 shutdown_connections 2024-04-26T18:10:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:11.964+0000 7f2371ec6700 1 -- 172.21.15.90:0/4150746965 wait complete. 2024-04-26T18:10:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:12 smithi119 ceph-mon[32752]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:12 smithi167 ceph-mon[32751]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:12 smithi090 ceph-mon[29180]: pgmap v342: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:13.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 -- 172.21.15.90:0/460631062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd741083c0 msgr2=0x7fbd741087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:13.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 --2- 172.21.15.90:0/460631062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd741083c0 0x7fbd741087a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009990 tx=0x7fbd64009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:13.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 -- 172.21.15.90:0/460631062 shutdown_connections 2024-04-26T18:10:13.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 --2- 172.21.15.90:0/460631062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd741083c0 0x7fbd741087a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:13.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 --2- 172.21.15.90:0/460631062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd74102d10 0x7fbd7410f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 --2- 172.21.15.90:0/460631062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd74102350 0x7fbd741027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 -- 172.21.15.90:0/460631062 >> 172.21.15.90:0/460631062 conn(0x7fbd74079360 msgr2=0x7fbd74079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 -- 172.21.15.90:0/460631062 shutdown_connections 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.490+0000 7fbd7a59e700 1 -- 172.21.15.90:0/460631062 wait complete. 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 Processor -- start 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 -- start start 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd74102350 0x7fbd7419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd74102d10 0x7fbd7419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd741083c0 0x7fbd74198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74112590 con 0x7fbd74102d10 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74112410 con 0x7fbd74102350 2024-04-26T18:10:13.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd7a59e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74112710 con 0x7fbd741083c0 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd74102d10 0x7fbd7419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd74102d10 0x7fbd7419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44344/0 (socket says 172.21.15.90:44344) 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 -- 172.21.15.90:0/413779598 learned_addr learned my addr 172.21.15.90:0/413779598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd78b3b700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd741083c0 0x7fbd74198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd73fff700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd74102350 0x7fbd7419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 -- 172.21.15.90:0/413779598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd741083c0 msgr2=0x7fbd74198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd741083c0 0x7fbd74198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:13.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 -- 172.21.15.90:0/413779598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd74102350 msgr2=0x7fbd7419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd74102350 0x7fbd7419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 -- 172.21.15.90:0/413779598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd74199070 con 0x7fbd74102d10 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.491+0000 7fbd6bfff700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd74102d10 0x7fbd7419ec40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00b530 tx=0x7fbd5c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.492+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c015020 con 0x7fbd74102d10 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.492+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd74199360 con 0x7fbd74102d10 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.492+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd5c011ce0 con 0x7fbd74102d10 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.492+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c010e40 con 0x7fbd74102d10 2024-04-26T18:10:13.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.492+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd741a55d0 con 0x7fbd74102d10 2024-04-26T18:10:13.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.493+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbd74063b70 con 0x7fbd74102d10 2024-04-26T18:10:13.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.494+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd5c018020 con 0x7fbd74102d10 2024-04-26T18:10:13.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.494+0000 7fbd71ffb700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbd600a3cd0 0x7fbd600a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:13.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.494+0000 7fbd73fff700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbd600a3cd0 0x7fbd600a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:13.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.494+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fbd5c0d02a0 con 0x7fbd74102d10 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.495+0000 7fbd71ffb700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] conn(0x7fbd600ad710 0x7fbd600afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.495+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 --> [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd600b0280 con 0x7fbd600ad710 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.495+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fbd5c0d0780 con 0x7fbd74102d10 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.495+0000 7fbd73fff700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbd600a3cd0 0x7fbd600a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006010 tx=0x7fbd6401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.495+0000 7fbd78b3b700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] conn(0x7fbd600ad710 0x7fbd600afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.495+0000 7fbd78b3b700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] conn(0x7fbd600ad710 0x7fbd600afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.496+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== osd.6 v2:172.21.15.90:6818/2940618965 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbd600b0280 con 0x7fbd600ad710 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.512+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 --> [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbd74063f70 con 0x7fbd600ad710 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.512+0000 7fbd71ffb700 1 -- 172.21.15.90:0/413779598 <== osd.6 v2:172.21.15.90:6818/2940618965 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbd74063f70 con 0x7fbd600ad710 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] conn(0x7fbd600ad710 msgr2=0x7fbd600afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] conn(0x7fbd600ad710 0x7fbd600afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbd600a3cd0 msgr2=0x7fbd600a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:13.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fbd600a3cd0 0x7fbd600a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbd64006010 tx=0x7fbd6401a040 comp rx=0 tx=0).stop 2024-04-26T18:10:13.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd74102d10 msgr2=0x7fbd7419ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:13.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 --2- 172.21.15.90:0/413779598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbd74102d10 0x7fbd7419ec40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c00b530 tx=0x7fbd5c00b840 comp rx=0 tx=0).stop 2024-04-26T18:10:13.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd78b3b700 1 -- 172.21.15.90:0/413779598 reap_dead start 2024-04-26T18:10:13.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 shutdown_connections 2024-04-26T18:10:13.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 >> 172.21.15.90:0/413779598 conn(0x7fbd74079360 msgr2=0x7fbd741010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:13.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 shutdown_connections 2024-04-26T18:10:13.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:13.513+0000 7fbd7a59e700 1 -- 172.21.15.90:0/413779598 wait complete. 2024-04-26T18:10:14.767 INFO:teuthology.orchestra.run.smithi090.stdout:201863462971 2024-04-26T18:10:14.767 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T18:10:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:14 smithi090 ceph-mon[29180]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T18:10:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:14 smithi119 ceph-mon[32752]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T18:10:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:14 smithi167 ceph-mon[32751]: pgmap v343: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-26T18:10:15.979 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:15.980 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:16.013 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:16.025 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:16.935 INFO:teuthology.orchestra.run.smithi090.stdout:141733920836 2024-04-26T18:10:16.936 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T18:10:17.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:16 smithi090 ceph-mon[29180]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:16 smithi119 ceph-mon[32752]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:16 smithi167 ceph-mon[32751]: pgmap v344: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:18 smithi119 ceph-mon[32752]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:18 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2662149037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:10:19.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:18 smithi090 ceph-mon[29180]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:19.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:18 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2662149037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:10:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:18 smithi167 ceph-mon[32751]: pgmap v345: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:18 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2662149037' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:10:19.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.587+0000 7fef5cf36700 1 -- 172.21.15.90:0/1536180703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806bdf0 msgr2=0x7fef5806c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:19.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.587+0000 7fef5cf36700 1 --2- 172.21.15.90:0/1536180703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806bdf0 0x7fef5806c1d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fef4c0099c0 tx=0x7fef4c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.587+0000 7fef5cf36700 1 -- 172.21.15.90:0/1536180703 shutdown_connections 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.587+0000 7fef5cf36700 1 --2- 172.21.15.90:0/1536180703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806bdf0 0x7fef5806c1d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.587+0000 7fef5cf36700 1 --2- 172.21.15.90:0/1536180703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef58108740 0x7fef58110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.587+0000 7fef5cf36700 1 --2- 172.21.15.90:0/1536180703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef5806c7a0 0x7fef58108060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.587+0000 7fef5cf36700 1 -- 172.21.15.90:0/1536180703 >> 172.21.15.90:0/1536180703 conn(0x7fef58078370 msgr2=0x7fef58078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 -- 172.21.15.90:0/1536180703 shutdown_connections 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 -- 172.21.15.90:0/1536180703 wait complete. 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 Processor -- start 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 -- start start 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef5806bdf0 0x7fef5819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:19.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806c7a0 0x7fef5819fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef58108740 0x7fef58199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef58113690 con 0x7fef5806c7a0 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef58113510 con 0x7fef5806bdf0 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.588+0000 7fef5cf36700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef58113810 con 0x7fef58108740 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806c7a0 0x7fef5819fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806c7a0 0x7fef5819fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44354/0 (socket says 172.21.15.90:44354) 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 -- 172.21.15.90:0/2662149037 learned_addr learned my addr 172.21.15.90:0/2662149037 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef5659c700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef5806bdf0 0x7fef5819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 -- 172.21.15.90:0/2662149037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef58108740 msgr2=0x7fef58199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef58108740 0x7fef58199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 -- 172.21.15.90:0/2662149037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef5806bdf0 msgr2=0x7fef5819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:19.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef5806bdf0 0x7fef5819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.628 INFO:teuthology.orchestra.run.smithi090.stdout:34359738458 2024-04-26T18:10:19.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 -- 172.21.15.90:0/2662149037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5819a170 con 0x7fef5806c7a0 2024-04-26T18:10:19.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef55d9b700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806c7a0 0x7fef5819fd40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b500 tx=0x7fef4800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:19.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef477fe700 1 -- 172.21.15.90:0/2662149037 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48015020 con 0x7fef5806c7a0 2024-04-26T18:10:19.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5819a460 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef477fe700 1 -- 172.21.15.90:0/2662149037 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef48011ce0 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef477fe700 1 -- 172.21.15.90:0/2662149037 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef48010d40 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.589+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef581a66d0 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.590+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5806a110 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.592+0000 7fef477fe700 1 -- 172.21.15.90:0/2662149037 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef48018020 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.592+0000 7fef477fe700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fef400a3cd0 0x7fef400a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.593+0000 7fef5659c700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fef400a3cd0 0x7fef400a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.593+0000 7fef477fe700 1 -- 172.21.15.90:0/2662149037 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fef480d0050 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.593+0000 7fef5659c700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fef400a3cd0 0x7fef400a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef58108310 tx=0x7fef4c009610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.597+0000 7fef477fe700 1 -- 172.21.15.90:0/2662149037 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef4801f020 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.806+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fef5819aef0 con 0x7fef5806c7a0 2024-04-26T18:10:19.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.807+0000 7fef477fe700 1 -- 172.21.15.90:0/2662149037 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fef4809cae0 con 0x7fef5806c7a0 2024-04-26T18:10:19.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fef400a3cd0 msgr2=0x7fef400a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:19.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fef400a3cd0 0x7fef400a6190 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef58108310 tx=0x7fef4c009610 comp rx=0 tx=0).stop 2024-04-26T18:10:19.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806c7a0 msgr2=0x7fef5819fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:19.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806c7a0 0x7fef5819fd40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fef4800b500 tx=0x7fef4800b810 comp rx=0 tx=0).stop 2024-04-26T18:10:19.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 shutdown_connections 2024-04-26T18:10:19.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fef400a3cd0 0x7fef400a6190 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef58108740 0x7fef58199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef5806bdf0 0x7fef5819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 --2- 172.21.15.90:0/2662149037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef5806c7a0 0x7fef5819fd40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:19.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 >> 172.21.15.90:0/2662149037 conn(0x7fef58078370 msgr2=0x7fef581004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:19.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 shutdown_connections 2024-04-26T18:10:19.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:18.809+0000 7fef5cf36700 1 -- 172.21.15.90:0/2662149037 wait complete. 2024-04-26T18:10:19.831 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:19.832 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:19.834 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:19.834 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:20.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2854781577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806c360 msgr2=0x7f7c081102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:20.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2854781577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806c360 0x7f7c081102d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f7c0000b0a0 tx=0x7f7c0000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2854781577 shutdown_connections 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2854781577 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806c360 0x7f7c081102d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2854781577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c08109490 0x7f7c08109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2854781577 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c0806b9c0 0x7f7c0806be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2854781577 >> 172.21.15.90:0/2854781577 conn(0x7f7c08079290 msgr2=0x7f7c080796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2854781577 shutdown_connections 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.920+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2854781577 wait complete. 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 Processor -- start 2024-04-26T18:10:20.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 -- start start 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806b9c0 0x7f7c0819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c0806c360 0x7f7c0819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c08109490 0x7f7c081999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7c08113640 con 0x7f7c0806b9c0 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081134c0 con 0x7f7c08109490 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0e7c0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c081137c0 con 0x7f7c0806c360 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806b9c0 0x7f7c0819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:20.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806b9c0 0x7f7c0819f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44376/0 (socket says 172.21.15.90:44376) 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 -- 172.21.15.90:0/2187671854 learned_addr learned my addr 172.21.15.90:0/2187671854 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c0cd5d700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c08109490 0x7f7c081999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c077fe700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c0806c360 0x7f7c0819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 -- 172.21.15.90:0/2187671854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c0806c360 msgr2=0x7f7c0819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c0806c360 0x7f7c0819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 -- 172.21.15.90:0/2187671854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c08109490 msgr2=0x7f7c081999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c08109490 0x7f7c081999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.921+0000 7f7c07fff700 1 -- 172.21.15.90:0/2187671854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0819a100 con 0x7f7c0806b9c0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.922+0000 7f7c07fff700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806b9c0 0x7f7c0819f790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d950 tx=0x7f7bf800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.922+0000 7f7c057fa700 1 -- 172.21.15.90:0/2187671854 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8016070 con 0x7f7c0806b9c0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.922+0000 7f7c057fa700 1 -- 172.21.15.90:0/2187671854 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bf8009df0 con 0x7f7c0806b9c0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.922+0000 7f7c057fa700 1 -- 172.21.15.90:0/2187671854 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800f3c0 con 0x7f7c0806b9c0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.922+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0819a3f0 con 0x7f7c0806b9c0 2024-04-26T18:10:20.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.922+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081a6660 con 0x7f7c0806b9c0 2024-04-26T18:10:20.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.923+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0806a110 con 0x7f7c0806b9c0 2024-04-26T18:10:20.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.924+0000 7f7c057fa700 1 -- 172.21.15.90:0/2187671854 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7bf8019070 con 0x7f7c0806b9c0 2024-04-26T18:10:20.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.924+0000 7f7c057fa700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7bf00a3cd0 0x7f7bf00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:20.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.925+0000 7f7c077fe700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7bf00a3cd0 0x7f7bf00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:20.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.925+0000 7f7c057fa700 1 -- 172.21.15.90:0/2187671854 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f7bf80d0060 con 0x7f7c0806b9c0 2024-04-26T18:10:20.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.925+0000 7f7c077fe700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7bf00a3cd0 0x7f7bf00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:20.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:20.929+0000 7f7c057fa700 1 -- 172.21.15.90:0/2187671854 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7bf809c9e0 con 0x7f7c0806b9c0 2024-04-26T18:10:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:20 smithi090 ceph-mon[29180]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-04-26T18:10:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.141+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7c0802e490 con 0x7f7c0806b9c0 2024-04-26T18:10:21.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.141+0000 7f7c057fa700 1 -- 172.21.15.90:0/2187671854 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7bf809c2e0 con 0x7f7c0806b9c0 2024-04-26T18:10:21.141 INFO:teuthology.orchestra.run.smithi090.stdout:227633266743 2024-04-26T18:10:21.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7bf00a3cd0 msgr2=0x7f7bf00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:21.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7bf00a3cd0 0x7f7bf00a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc008040 comp rx=0 tx=0).stop 2024-04-26T18:10:21.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806b9c0 msgr2=0x7f7c0819f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806b9c0 0x7f7c0819f790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800d950 tx=0x7f7bf800dc60 comp rx=0 tx=0).stop 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 shutdown_connections 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7bf00a3cd0 0x7f7bf00a6190 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c0806b9c0 0x7f7c0819f790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c08109490 0x7f7c081999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 --2- 172.21.15.90:0/2187671854 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c0806c360 0x7f7c0819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 >> 172.21.15.90:0/2187671854 conn(0x7f7c08079290 msgr2=0x7f7c08100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 shutdown_connections 2024-04-26T18:10:21.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:21.143+0000 7f7c0e7c0700 1 -- 172.21.15.90:0/2187671854 wait complete. 2024-04-26T18:10:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:20 smithi119 ceph-mon[32752]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-04-26T18:10:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:20 smithi167 ceph-mon[32751]: pgmap v346: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-04-26T18:10:22.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:21 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2187671854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:10:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:21 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2187671854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:10:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:21 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2187671854' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:10:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:22 smithi119 ceph-mon[32752]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-26T18:10:23.317 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:22 smithi090 ceph-mon[29180]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-26T18:10:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:22 smithi167 ceph-mon[32751]: pgmap v347: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 -- 172.21.15.90:0/3319970768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 msgr2=0x7f1b7c109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3319970768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c109860 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009990 tx=0x7f1b78009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 -- 172.21.15.90:0/3319970768 shutdown_connections 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3319970768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c109860 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3319970768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b7c103ea0 0x7f1b7c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3319970768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b7c1034e0 0x7f1b7c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 -- 172.21.15.90:0/3319970768 >> 172.21.15.90:0/3319970768 conn(0x7f1b7c07a5e0 msgr2=0x7f1b7c07a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 -- 172.21.15.90:0/3319970768 shutdown_connections 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 -- 172.21.15.90:0/3319970768 wait complete. 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 Processor -- start 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.159+0000 7f1b842f8700 1 -- start start 2024-04-26T18:10:25.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b842f8700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b7c1034e0 0x7f1b7c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b842f8700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b7c103ea0 0x7f1b7c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b842f8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b842f8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1135b0 con 0x7f1b7c109480 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b842f8700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c113430 con 0x7f1b7c103ea0 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b842f8700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c113730 con 0x7f1b7c1034e0 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52560/0 (socket says 172.21.15.90:52560) 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 -- 172.21.15.90:0/3743164406 learned_addr learned my addr 172.21.15.90:0/3743164406 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82094700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b7c1034e0 0x7f1b7c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b81893700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b7c103ea0 0x7f1b7c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 -- 172.21.15.90:0/3743164406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b7c1034e0 msgr2=0x7f1b7c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b7c1034e0 0x7f1b7c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 -- 172.21.15.90:0/3743164406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b7c103ea0 msgr2=0x7f1b7c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b7c103ea0 0x7f1b7c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 -- 172.21.15.90:0/3743164406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c19a090 con 0x7f1b7c109480 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b82895700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c199930 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b74007ee0 tx=0x7f1b7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.160+0000 7f1b737fe700 1 -- 172.21.15.90:0/3743164406 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74004bb0 con 0x7f1b7c109480 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.161+0000 7f1b737fe700 1 -- 172.21.15.90:0/3743164406 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b74004d50 con 0x7f1b7c109480 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.161+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c19a380 con 0x7f1b7c109480 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.161+0000 7f1b737fe700 1 -- 172.21.15.90:0/3743164406 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74005920 con 0x7f1b7c109480 2024-04-26T18:10:25.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.161+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1a65f0 con 0x7f1b7c109480 2024-04-26T18:10:25.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.162+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c104b30 con 0x7f1b7c109480 2024-04-26T18:10:25.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.163+0000 7f1b737fe700 1 -- 172.21.15.90:0/3743164406 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1b7401a020 con 0x7f1b7c109480 2024-04-26T18:10:25.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.163+0000 7f1b737fe700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1b680a3d50 0x7f1b680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:25.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.164+0000 7f1b82094700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1b680a3d50 0x7f1b680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:25.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.164+0000 7f1b737fe700 1 -- 172.21.15.90:0/3743164406 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f1b740d0050 con 0x7f1b7c109480 2024-04-26T18:10:25.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.164+0000 7f1b82094700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1b680a3d50 0x7f1b680a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800b5c0 tx=0x7f1b78005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:25.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.167+0000 7f1b737fe700 1 -- 172.21.15.90:0/3743164406 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b74010070 con 0x7f1b7c109480 2024-04-26T18:10:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:24 smithi119 ceph-mon[32752]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:24 smithi167 ceph-mon[32751]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:25.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.368+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1b7c02e490 con 0x7f1b7c109480 2024-04-26T18:10:25.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:24 smithi090 ceph-mon[29180]: pgmap v348: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:25.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.369+0000 7f1b737fe700 1 -- 172.21.15.90:0/3743164406 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b7409c9d0 con 0x7f1b7c109480 2024-04-26T18:10:25.369 INFO:teuthology.orchestra.run.smithi090.stdout:55834574934 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1b680a3d50 msgr2=0x7f1b680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1b680a3d50 0x7f1b680a6210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800b5c0 tx=0x7f1b78005960 comp rx=0 tx=0).stop 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 msgr2=0x7f1b7c199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c199930 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b74007ee0 tx=0x7f1b7400d3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 shutdown_connections 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f1b680a3d50 0x7f1b680a6210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b7c109480 0x7f1b7c199930 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b7c1034e0 0x7f1b7c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 --2- 172.21.15.90:0/3743164406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b7c103ea0 0x7f1b7c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 >> 172.21.15.90:0/3743164406 conn(0x7f1b7c07a5e0 msgr2=0x7f1b7c1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:25.371 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 shutdown_connections 2024-04-26T18:10:25.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:25.371+0000 7f1b842f8700 1 -- 172.21.15.90:0/3743164406 wait complete. 2024-04-26T18:10:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:25 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3743164406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:10:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:25 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3743164406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:10:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:25 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3743164406' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:10:26.461 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738458 for osd.0 2024-04-26T18:10:26.461 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:27.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:26 smithi090 ceph-mon[29180]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-26T18:10:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:26 smithi119 ceph-mon[32752]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-26T18:10:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:26 smithi167 ceph-mon[32751]: pgmap v349: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-26T18:10:28.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:28 smithi090 ceph-mon[29180]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:28 smithi119 ceph-mon[32752]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:28 smithi167 ceph-mon[32751]: pgmap v350: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:31.037 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266739 got 227633266743 for osd.7 2024-04-26T18:10:31.037 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:31.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:30 smithi090 ceph-mon[29180]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:30 smithi119 ceph-mon[32752]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:30 smithi167 ceph-mon[32751]: pgmap v351: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 -- 172.21.15.90:0/280410551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c103ea0 msgr2=0x7f177c1102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 --2- 172.21.15.90:0/280410551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c103ea0 0x7f177c1102c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f177400b0a0 tx=0x7f177400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 -- 172.21.15.90:0/280410551 shutdown_connections 2024-04-26T18:10:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 --2- 172.21.15.90:0/280410551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c103ea0 0x7f177c1102c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 --2- 172.21.15.90:0/280410551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177c109480 0x7f177c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 --2- 172.21.15.90:0/280410551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c1034e0 0x7f177c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 -- 172.21.15.90:0/280410551 >> 172.21.15.90:0/280410551 conn(0x7f177c07a5e0 msgr2=0x7f177c07a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 -- 172.21.15.90:0/280410551 shutdown_connections 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.515+0000 7f17823ea700 1 -- 172.21.15.90:0/280410551 wait complete. 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 Processor -- start 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 -- start start 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177c1034e0 0x7f177c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c103ea0 0x7f177c19c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c109480 0x7f177c1a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f177c113610 con 0x7f177c109480 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f177c113490 con 0x7f177c103ea0 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f17823ea700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f177c113790 con 0x7f177c1034e0 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c109480 0x7f177c1a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:31.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c109480 0x7f177c1a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42070/0 (socket says 172.21.15.90:42070) 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 -- 172.21.15.90:0/432953891 learned_addr learned my addr 172.21.15.90:0/432953891 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f177b7fe700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c103ea0 0x7f177c19c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f177bfff700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177c1034e0 0x7f177c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 -- 172.21.15.90:0/432953891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177c1034e0 msgr2=0x7f177c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177c1034e0 0x7f177c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 -- 172.21.15.90:0/432953891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c103ea0 msgr2=0x7f177c19c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c103ea0 0x7f177c19c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.516+0000 7f1780987700 1 -- 172.21.15.90:0/432953891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177c1a0d50 con 0x7f177c109480 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.517+0000 7f1780987700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c109480 0x7f177c1a0610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f17740062a0 tx=0x7f1774004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.517+0000 7f17797fa700 1 -- 172.21.15.90:0/432953891 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774021070 con 0x7f177c109480 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.517+0000 7f17797fa700 1 -- 172.21.15.90:0/432953891 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1774003c00 con 0x7f177c109480 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.517+0000 7f17797fa700 1 -- 172.21.15.90:0/432953891 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1774004880 con 0x7f177c109480 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.517+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177c1a0fe0 con 0x7f177c109480 2024-04-26T18:10:31.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.517+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f177c1a88c0 con 0x7f177c109480 2024-04-26T18:10:31.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.518+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f177c06a110 con 0x7f177c109480 2024-04-26T18:10:31.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.519+0000 7f17797fa700 1 -- 172.21.15.90:0/432953891 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1774004020 con 0x7f177c109480 2024-04-26T18:10:31.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.520+0000 7f17797fa700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f17640a3cd0 0x7f17640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:31.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.520+0000 7f177bfff700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f17640a3cd0 0x7f17640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:31.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.520+0000 7f17797fa700 1 -- 172.21.15.90:0/432953891 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f177400f020 con 0x7f177c109480 2024-04-26T18:10:31.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.520+0000 7f177bfff700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f17640a3cd0 0x7f17640a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f176c009a10 tx=0x7f176c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:31.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.525+0000 7f17797fa700 1 -- 172.21.15.90:0/432953891 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f177402a020 con 0x7f177c109480 2024-04-26T18:10:31.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.736+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f177c1a8a90 con 0x7f177c109480 2024-04-26T18:10:31.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.736+0000 7f17797fa700 1 -- 172.21.15.90:0/432953891 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f177409e440 con 0x7f177c109480 2024-04-26T18:10:31.736 INFO:teuthology.orchestra.run.smithi090.stdout:107374182477 2024-04-26T18:10:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f17640a3cd0 msgr2=0x7f17640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f17640a3cd0 0x7f17640a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f176c009a10 tx=0x7f176c009190 comp rx=0 tx=0).stop 2024-04-26T18:10:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c109480 msgr2=0x7f177c1a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c109480 0x7f177c1a0610 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f17740062a0 tx=0x7f1774004e20 comp rx=0 tx=0).stop 2024-04-26T18:10:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 shutdown_connections 2024-04-26T18:10:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f17640a3cd0 0x7f17640a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f177c109480 0x7f177c1a0610 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f177c1034e0 0x7f177c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 --2- 172.21.15.90:0/432953891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f177c103ea0 0x7f177c19c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:31.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 >> 172.21.15.90:0/432953891 conn(0x7f177c07a5e0 msgr2=0x7f177c10e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:31.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 shutdown_connections 2024-04-26T18:10:31.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:31.738+0000 7f17823ea700 1 -- 172.21.15.90:0/432953891 wait complete. 2024-04-26T18:10:32.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:31 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/432953891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:10:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:31 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/432953891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:10:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:31 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/432953891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:10:32.326 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574934 for osd.1 2024-04-26T18:10:32.327 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:33.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 -- 172.21.15.90:0/4063059206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb20103e90 msgr2=0x7fcb20110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:33.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 --2- 172.21.15.90:0/4063059206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb20103e90 0x7fcb20110300 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb180099c0 tx=0x7fcb18009cd0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 -- 172.21.15.90:0/4063059206 shutdown_connections 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 --2- 172.21.15.90:0/4063059206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb20103e90 0x7fcb20110300 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 --2- 172.21.15.90:0/4063059206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb201083b0 0x7fcb20108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 --2- 172.21.15.90:0/4063059206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb201034d0 0x7fcb20103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 -- 172.21.15.90:0/4063059206 >> 172.21.15.90:0/4063059206 conn(0x7fcb2007a580 msgr2=0x7fcb2007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 -- 172.21.15.90:0/4063059206 shutdown_connections 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.117+0000 7fcb26c59700 1 -- 172.21.15.90:0/4063059206 wait complete. 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 Processor -- start 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 -- start start 2024-04-26T18:10:33.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb201034d0 0x7fcb2019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb20103e90 0x7fcb2019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb201083b0 0x7fcb201998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcb20113640 con 0x7fcb201034d0 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201134c0 con 0x7fcb20103e90 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb26c59700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcb201137c0 con 0x7fcb201083b0 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb201034d0 0x7fcb2019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb201034d0 0x7fcb2019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42090/0 (socket says 172.21.15.90:42090) 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 -- 172.21.15.90:0/2438397549 learned_addr learned my addr 172.21.15.90:0/2438397549 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb251f6700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb201083b0 0x7fcb201998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb1ffff700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb20103e90 0x7fcb2019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 -- 172.21.15.90:0/2438397549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb201083b0 msgr2=0x7fcb201998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:33.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb201083b0 0x7fcb201998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 -- 172.21.15.90:0/2438397549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb20103e90 msgr2=0x7fcb2019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb20103e90 0x7fcb2019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 -- 172.21.15.90:0/2438397549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2019a020 con 0x7fcb201034d0 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb1ffff700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb20103e90 0x7fcb2019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb249f5700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb201034d0 0x7fcb2019f700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb140060b0 tx=0x7fcb14007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.118+0000 7fcb1dffb700 1 -- 172.21.15.90:0/2438397549 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1401a070 con 0x7fcb201034d0 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.119+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2019a310 con 0x7fcb201034d0 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.119+0000 7fcb1dffb700 1 -- 172.21.15.90:0/2438397549 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb1400de20 con 0x7fcb201034d0 2024-04-26T18:10:33.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.119+0000 7fcb1dffb700 1 -- 172.21.15.90:0/2438397549 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14014720 con 0x7fcb201034d0 2024-04-26T18:10:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.119+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb201a65d0 con 0x7fcb201034d0 2024-04-26T18:10:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.120+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2006a110 con 0x7fcb201034d0 2024-04-26T18:10:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.120+0000 7fcb1dffb700 1 -- 172.21.15.90:0/2438397549 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcb1401d070 con 0x7fcb201034d0 2024-04-26T18:10:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.121+0000 7fcb1dffb700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcb100a3c40 0x7fcb100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.121+0000 7fcb1ffff700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcb100a3c40 0x7fcb100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.121+0000 7fcb1dffb700 1 -- 172.21.15.90:0/2438397549 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fcb140cfc00 con 0x7fcb201034d0 2024-04-26T18:10:33.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.121+0000 7fcb1ffff700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcb100a3c40 0x7fcb100a6100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c005fd0 tx=0x7fcb0c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:33.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.126+0000 7fcb1dffb700 1 -- 172.21.15.90:0/2438397549 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb140d6080 con 0x7fcb201034d0 2024-04-26T18:10:33.237 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:32 smithi090 ceph-mon[29180]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:33.237 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:32 smithi090 ceph-mon[29180]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-26T18:10:33.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:32 smithi119 ceph-mon[32752]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:33.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:32 smithi119 ceph-mon[32752]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-26T18:10:33.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.335+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fcb2019ae20 con 0x7fcb201034d0 2024-04-26T18:10:33.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:32 smithi167 ceph-mon[32751]: pgmap v352: 97 pgs: 97 active+clean; 639 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:33.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:32 smithi167 ceph-mon[32751]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-26T18:10:33.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.336+0000 7fcb1dffb700 1 -- 172.21.15.90:0/2438397549 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fcb2019ae20 con 0x7fcb201034d0 2024-04-26T18:10:33.341 INFO:teuthology.orchestra.run.smithi090.stdout:317827579945 2024-04-26T18:10:33.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcb100a3c40 msgr2=0x7fcb100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:33.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcb100a3c40 0x7fcb100a6100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c005fd0 tx=0x7fcb0c005ee0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb201034d0 msgr2=0x7fcb2019f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:33.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb201034d0 0x7fcb2019f700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcb140060b0 tx=0x7fcb14007f90 comp rx=0 tx=0).stop 2024-04-26T18:10:33.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 shutdown_connections 2024-04-26T18:10:33.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fcb100a3c40 0x7fcb100a6100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb201034d0 0x7fcb2019f700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcb201083b0 0x7fcb201998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 --2- 172.21.15.90:0/2438397549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb20103e90 0x7fcb2019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:33.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 >> 172.21.15.90:0/2438397549 conn(0x7fcb2007a580 msgr2=0x7fcb20101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:33.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 shutdown_connections 2024-04-26T18:10:33.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:33.338+0000 7fcb26c59700 1 -- 172.21.15.90:0/2438397549 wait complete. 2024-04-26T18:10:34.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.136+0000 7fde7ed89700 1 -- 172.21.15.90:0/257114704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 msgr2=0x7fde781027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:34.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.136+0000 7fde7ed89700 1 --2- 172.21.15.90:0/257114704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde781027a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fde64009990 tx=0x7fde64009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 -- 172.21.15.90:0/257114704 shutdown_connections 2024-04-26T18:10:34.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 --2- 172.21.15.90:0/257114704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde78102ce0 0x7fde7810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 --2- 172.21.15.90:0/257114704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde78108390 0x7fde78108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 --2- 172.21.15.90:0/257114704 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde781027a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 -- 172.21.15.90:0/257114704 >> 172.21.15.90:0/257114704 conn(0x7fde780792d0 msgr2=0x7fde780796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:34.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 -- 172.21.15.90:0/257114704 shutdown_connections 2024-04-26T18:10:34.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 -- 172.21.15.90:0/257114704 wait complete. 2024-04-26T18:10:34.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 Processor -- start 2024-04-26T18:10:34.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.137+0000 7fde7ed89700 1 -- start start 2024-04-26T18:10:34.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7ed89700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde7819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:34.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7ed89700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde78102ce0 0x7fde7819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:34.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7ed89700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde78108390 0x7fde78198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:34.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7ed89700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fde781125b0 con 0x7fde78102320 2024-04-26T18:10:34.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7ed89700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fde78112430 con 0x7fde78102ce0 2024-04-26T18:10:34.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7ed89700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fde78112730 con 0x7fde78108390 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde7819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde7819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42110/0 (socket says 172.21.15.90:42110) 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 -- 172.21.15.90:0/2530865504 learned_addr learned my addr 172.21.15.90:0/2530865504 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde77fff700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde78102ce0 0x7fde7819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7d326700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde78108390 0x7fde78198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 -- 172.21.15.90:0/2530865504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde78108390 msgr2=0x7fde78198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde78108390 0x7fde78198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 -- 172.21.15.90:0/2530865504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde78102ce0 msgr2=0x7fde7819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde78102ce0 0x7fde7819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 -- 172.21.15.90:0/2530865504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde781990d0 con 0x7fde78102320 2024-04-26T18:10:34.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7d326700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde78108390 0x7fde78198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7cb25700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde7819e7b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00b530 tx=0x7fde6c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde75ffb700 1 -- 172.21.15.90:0/2530865504 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c015020 con 0x7fde78102320 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde75ffb700 1 -- 172.21.15.90:0/2530865504 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde6c011ce0 con 0x7fde78102320 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde781993c0 con 0x7fde78102320 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.138+0000 7fde75ffb700 1 -- 172.21.15.90:0/2530865504 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c010d20 con 0x7fde78102320 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.139+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde781a5680 con 0x7fde78102320 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.139+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7810a900 con 0x7fde78102320 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.140+0000 7fde75ffb700 1 -- 172.21.15.90:0/2530865504 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fde6c018020 con 0x7fde78102320 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.141+0000 7fde75ffb700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fde680a3c40 0x7fde680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:34.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.141+0000 7fde77fff700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fde680a3c40 0x7fde680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:34.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.141+0000 7fde75ffb700 1 -- 172.21.15.90:0/2530865504 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fde6c0cfbf0 con 0x7fde78102320 2024-04-26T18:10:34.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.141+0000 7fde77fff700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fde680a3c40 0x7fde680a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde6401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:34.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.145+0000 7fde75ffb700 1 -- 172.21.15.90:0/2530865504 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde6c01f020 con 0x7fde78102320 2024-04-26T18:10:34.176 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:34.176 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:34.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:33 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2438397549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:10:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:33 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2438397549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:10:34.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.353+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fde78103a40 con 0x7fde78102320 2024-04-26T18:10:34.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:33 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2438397549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:10:34.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.353+0000 7fde75ffb700 1 -- 172.21.15.90:0/2530865504 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fde6c09c670 con 0x7fde78102320 2024-04-26T18:10:34.355 INFO:teuthology.orchestra.run.smithi090.stdout:261993005108 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fde680a3c40 msgr2=0x7fde680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fde680a3c40 0x7fde680a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fde64006010 tx=0x7fde6401a040 comp rx=0 tx=0).stop 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 msgr2=0x7fde7819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde7819e7b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fde6c00b530 tx=0x7fde6c00b8f0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 shutdown_connections 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fde680a3c40 0x7fde680a6100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fde78102320 0x7fde7819e7b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fde78108390 0x7fde78198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 --2- 172.21.15.90:0/2530865504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde78102ce0 0x7fde7819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:34.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.356+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 >> 172.21.15.90:0/2530865504 conn(0x7fde780792d0 msgr2=0x7fde78100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:34.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.357+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 shutdown_connections 2024-04-26T18:10:34.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:34.357+0000 7fde7ed89700 1 -- 172.21.15.90:0/2530865504 wait complete. 2024-04-26T18:10:34.976 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:34 smithi090 ceph-mon[29180]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T18:10:34.976 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:34 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2530865504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:10:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:34 smithi119 ceph-mon[32752]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T18:10:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:34 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2530865504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:10:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:34 smithi167 ceph-mon[32751]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-26T18:10:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:34 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2530865504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:10:35.879 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579940 got 317827579945 for osd.10 2024-04-26T18:10:35.879 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:35.985 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:35 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1895315549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:10:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:35 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1895315549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:10:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:35 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1895315549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:10:37.030 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005100 got 261993005108 for osd.8 2024-04-26T18:10:37.030 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:37.229 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:36 smithi090 ceph-mon[29180]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T18:10:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:36 smithi119 ceph-mon[32752]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T18:10:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:36 smithi167 ceph-mon[32751]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T18:10:37.330 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 -- 172.21.15.90:0/865620722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e101083d0 msgr2=0x7f5e101087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 --2- 172.21.15.90:0/865620722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e101083d0 0x7f5e101087b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5e04009990 tx=0x7f5e04009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 -- 172.21.15.90:0/865620722 shutdown_connections 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 --2- 172.21.15.90:0/865620722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e101083d0 0x7f5e101087b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 --2- 172.21.15.90:0/865620722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e10102d10 0x7f5e1010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 --2- 172.21.15.90:0/865620722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e10102350 0x7f5e101027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 -- 172.21.15.90:0/865620722 >> 172.21.15.90:0/865620722 conn(0x7f5e10078390 msgr2=0x7f5e100787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 -- 172.21.15.90:0/865620722 shutdown_connections 2024-04-26T18:10:37.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 -- 172.21.15.90:0/865620722 wait complete. 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 Processor -- start 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.331+0000 7f5e16a88700 1 -- start start 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e16a88700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e10102350 0x7f5e1019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e16a88700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e10102d10 0x7f5e1019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e16a88700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e101083d0 0x7f5e10199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e16a88700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5e10112570 con 0x7f5e10102350 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e16a88700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5e101123f0 con 0x7f5e10102d10 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e16a88700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e101126f0 con 0x7f5e101083d0 2024-04-26T18:10:37.333 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e10102350 0x7f5e1019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e10102350 0x7f5e1019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42162/0 (socket says 172.21.15.90:42162) 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 -- 172.21.15.90:0/598485063 learned_addr learned my addr 172.21.15.90:0/598485063 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e0ffff700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e10102d10 0x7f5e1019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 -- 172.21.15.90:0/598485063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e101083d0 msgr2=0x7f5e10199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e101083d0 0x7f5e10199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 -- 172.21.15.90:0/598485063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e10102d10 msgr2=0x7f5e1019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e10102d10 0x7f5e1019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 -- 172.21.15.90:0/598485063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1019a0e0 con 0x7f5e10102350 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e0ffff700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e10102d10 0x7f5e1019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e14824700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e10102350 0x7f5e1019f7c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e04000c00 tx=0x7f5e040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e0dffb700 1 -- 172.21.15.90:0/598485063 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e04027070 con 0x7f5e10102350 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e0dffb700 1 -- 172.21.15.90:0/598485063 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e04004150 con 0x7f5e10102350 2024-04-26T18:10:37.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e0dffb700 1 -- 172.21.15.90:0/598485063 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e0400f460 con 0x7f5e10102350 2024-04-26T18:10:37.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.332+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1019a370 con 0x7f5e10102350 2024-04-26T18:10:37.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.333+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e101a6690 con 0x7f5e10102350 2024-04-26T18:10:37.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.334+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e10103a40 con 0x7f5e10102350 2024-04-26T18:10:37.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.335+0000 7f5e0dffb700 1 -- 172.21.15.90:0/598485063 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5e0402a070 con 0x7f5e10102350 2024-04-26T18:10:37.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.335+0000 7f5e0dffb700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e000a3c40 0x7f5e000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.335+0000 7f5e0ffff700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e000a3c40 0x7f5e000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:37.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.335+0000 7f5e0dffb700 1 -- 172.21.15.90:0/598485063 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f5e040d1960 con 0x7f5e10102350 2024-04-26T18:10:37.336 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.336+0000 7f5e0ffff700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e000a3c40 0x7f5e000a6100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc005fd0 tx=0x7f5dfc005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:37.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.339+0000 7f5e0dffb700 1 -- 172.21.15.90:0/598485063 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e04031070 con 0x7f5e10102350 2024-04-26T18:10:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 -- 172.21.15.90:0/3637364230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7781076b0 msgr2=0x7fb77810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 --2- 172.21.15.90:0/3637364230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7781076b0 0x7fb77810f250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb77000b0d0 tx=0x7fb77000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 -- 172.21.15.90:0/3637364230 shutdown_connections 2024-04-26T18:10:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 --2- 172.21.15.90:0/3637364230 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7781076b0 0x7fb77810f250 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 --2- 172.21.15.90:0/3637364230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb778102960 0x7fb778102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 --2- 172.21.15.90:0/3637364230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb778103310 0x7fb778106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 -- 172.21.15.90:0/3637364230 >> 172.21.15.90:0/3637364230 conn(0x7fb7780792d0 msgr2=0x7fb7780796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 -- 172.21.15.90:0/3637364230 shutdown_connections 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.179+0000 7fb77d73d700 1 -- 172.21.15.90:0/3637364230 wait complete. 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 Processor -- start 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 -- start start 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb778102960 0x7fb77819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb778103310 0x7fb77819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7781076b0 0x7fb778199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb778112570 con 0x7fb778103310 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781123f0 con 0x7fb7781076b0 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb77d73d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781126f0 con 0x7fb778102960 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb7767fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb778103310 0x7fb77819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb7767fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb778103310 0x7fb77819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42138/0 (socket says 172.21.15.90:42138) 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb7767fc700 1 -- 172.21.15.90:0/1895315549 learned_addr learned my addr 172.21.15.90:0/1895315549 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb7777fe700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7781076b0 0x7fb778199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb776ffd700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb778102960 0x7fb77819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.180+0000 7fb7767fc700 1 -- 172.21.15.90:0/1895315549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb778102960 msgr2=0x7fb77819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb7767fc700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb778102960 0x7fb77819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb7767fc700 1 -- 172.21.15.90:0/1895315549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7781076b0 msgr2=0x7fb778199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb7767fc700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7781076b0 0x7fb778199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb7767fc700 1 -- 172.21.15.90:0/1895315549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77819a080 con 0x7fb778103310 2024-04-26T18:10:37.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb776ffd700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb778102960 0x7fb77819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stdout:352187318309 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb7767fc700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb778103310 0x7fb77819fca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b4d0 tx=0x7fb76800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb76ffff700 1 -- 172.21.15.90:0/1895315549 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768015020 con 0x7fb778103310 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77819a370 con 0x7fb778103310 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb76ffff700 1 -- 172.21.15.90:0/1895315549 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb768011ce0 con 0x7fb778103310 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb76ffff700 1 -- 172.21.15.90:0/1895315549 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb768010cd0 con 0x7fb778103310 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.181+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7781a6630 con 0x7fb778103310 2024-04-26T18:10:37.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.182+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb778103a60 con 0x7fb778103310 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.183+0000 7fb76ffff700 1 -- 172.21.15.90:0/1895315549 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb768018020 con 0x7fb778103310 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.183+0000 7fb76ffff700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7640a3c40 0x7fb7640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.184+0000 7fb776ffd700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7640a3c40 0x7fb7640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.184+0000 7fb76ffff700 1 -- 172.21.15.90:0/1895315549 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb7680cfab0 con 0x7fb778103310 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.184+0000 7fb776ffd700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7640a3c40 0x7fb7640a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb760005fd0 tx=0x7fb760005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.188+0000 7fb76ffff700 1 -- 172.21.15.90:0/1895315549 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb76801f020 con 0x7fb778103310 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.397+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb77819ae50 con 0x7fb778103310 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.398+0000 7fb76ffff700 1 -- 172.21.15.90:0/1895315549 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb76809c530 con 0x7fb778103310 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7640a3c40 msgr2=0x7fb7640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7640a3c40 0x7fb7640a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb760005fd0 tx=0x7fb760005de0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb778103310 msgr2=0x7fb77819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb778103310 0x7fb77819fca0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b4d0 tx=0x7fb76800b7e0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 shutdown_connections 2024-04-26T18:10:37.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7640a3c40 0x7fb7640a6100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7781076b0 0x7fb778199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb778102960 0x7fb77819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 --2- 172.21.15.90:0/1895315549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb778103310 0x7fb77819fca0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 >> 172.21.15.90:0/1895315549 conn(0x7fb7780792d0 msgr2=0x7fb77806bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:37.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 shutdown_connections 2024-04-26T18:10:37.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:35.400+0000 7fb77d73d700 1 -- 172.21.15.90:0/1895315549 wait complete. 2024-04-26T18:10:37.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.547+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f5e101a6860 con 0x7f5e10102350 2024-04-26T18:10:37.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.548+0000 7f5e0dffb700 1 -- 172.21.15.90:0/598485063 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5e04013070 con 0x7f5e10102350 2024-04-26T18:10:37.548 INFO:teuthology.orchestra.run.smithi090.stdout:296352743472 2024-04-26T18:10:37.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e000a3c40 msgr2=0x7f5e000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e000a3c40 0x7f5e000a6100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5dfc005fd0 tx=0x7f5dfc005dc0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e10102350 msgr2=0x7f5e1019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:37.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e10102350 0x7f5e1019f7c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5e04000c00 tx=0x7f5e040047b0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 shutdown_connections 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5e000a3c40 0x7f5e000a6100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e10102350 0x7f5e1019f7c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e101083d0 0x7f5e10199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 --2- 172.21.15.90:0/598485063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e10102d10 0x7f5e1019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 >> 172.21.15.90:0/598485063 conn(0x7f5e10078390 msgr2=0x7f5e101016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 shutdown_connections 2024-04-26T18:10:37.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:37.550+0000 7f5e16a88700 1 -- 172.21.15.90:0/598485063 wait complete. 2024-04-26T18:10:38.056 INFO:tasks.cephadm.ceph_manager.ceph:need seq 352187318303 got 352187318309 for osd.11 2024-04-26T18:10:38.056 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:38.247 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:37 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/598485063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:10:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:37 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/598485063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:10:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:37 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/598485063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:10:39.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:38 smithi090 ceph-mon[29180]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-26T18:10:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:38 smithi119 ceph-mon[32752]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-26T18:10:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:38 smithi167 ceph-mon[32751]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-26T18:10:40.310 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182471 got 107374182477 for osd.3 2024-04-26T18:10:40.310 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 -- 172.21.15.90:0/2184422724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c06c7a0 msgr2=0x7f7c7c108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2184422724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c06c7a0 0x7f7c7c108060 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7c640099c0 tx=0x7f7c64009cd0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 -- 172.21.15.90:0/2184422724 shutdown_connections 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2184422724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c7c108740 0x7f7c7c110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2184422724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c7c06bdf0 0x7f7c7c06c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2184422724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c06c7a0 0x7f7c7c108060 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 -- 172.21.15.90:0/2184422724 >> 172.21.15.90:0/2184422724 conn(0x7f7c7c078370 msgr2=0x7f7c7c078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:40.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 -- 172.21.15.90:0/2184422724 shutdown_connections 2024-04-26T18:10:40.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.508+0000 7f7c813a7700 1 -- 172.21.15.90:0/2184422724 wait complete. 2024-04-26T18:10:40.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 Processor -- start 2024-04-26T18:10:40.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 -- start start 2024-04-26T18:10:40.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c7c06bdf0 0x7f7c7c102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c7c06c7a0 0x7f7c7c1029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c108740 0x7f7c7c102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c113670 con 0x7f7c7c108740 2024-04-26T18:10:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1134f0 con 0x7f7c7c06c7a0 2024-04-26T18:10:40.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c813a7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c1137f0 con 0x7f7c7c06bdf0 2024-04-26T18:10:40.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c7b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c108740 0x7f7c7c102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c7b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c108740 0x7f7c7c102f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42176/0 (socket says 172.21.15.90:42176) 2024-04-26T18:10:40.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c7b7fe700 1 -- 172.21.15.90:0/2178271779 learned_addr learned my addr 172.21.15.90:0/2178271779 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:40.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c727fc700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c7c06c7a0 0x7f7c7c1029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.509+0000 7f7c7affd700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c7c06bdf0 0x7f7c7c102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c7b7fe700 1 -- 172.21.15.90:0/2178271779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c7c06bdf0 msgr2=0x7f7c7c102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c7b7fe700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c7c06bdf0 0x7f7c7c102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c7b7fe700 1 -- 172.21.15.90:0/2178271779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c7c06c7a0 msgr2=0x7f7c7c1029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c7b7fe700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c7c06c7a0 0x7f7c7c1029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c7b7fe700 1 -- 172.21.15.90:0/2178271779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c1a80c0 con 0x7f7c7c108740 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c7b7fe700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c108740 0x7f7c7c102f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007ee0 tx=0x7f7c7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c78ff9700 1 -- 172.21.15.90:0/2178271779 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7400dbd0 con 0x7f7c7c108740 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7c1a83b0 con 0x7f7c7c108740 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c78ff9700 1 -- 172.21.15.90:0/2178271779 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c7400dd70 con 0x7f7c7c108740 2024-04-26T18:10:40.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.510+0000 7f7c78ff9700 1 -- 172.21.15.90:0/2178271779 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74005990 con 0x7f7c7c108740 2024-04-26T18:10:40.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.511+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7c1a88c0 con 0x7f7c7c108740 2024-04-26T18:10:40.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.512+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7c06a110 con 0x7f7c7c108740 2024-04-26T18:10:40.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.513+0000 7f7c78ff9700 1 -- 172.21.15.90:0/2178271779 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7c7401a020 con 0x7f7c7c108740 2024-04-26T18:10:40.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.513+0000 7f7c78ff9700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7c680a3c40 0x7f7c680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.514+0000 7f7c7affd700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7c680a3c40 0x7f7c680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.514+0000 7f7c78ff9700 1 -- 172.21.15.90:0/2178271779 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f7c740cf4f0 con 0x7f7c7c108740 2024-04-26T18:10:40.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.514+0000 7f7c7affd700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7c680a3c40 0x7f7c680a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c006fd0 tx=0x7f7c6c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:40.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.518+0000 7f7c78ff9700 1 -- 172.21.15.90:0/2178271779 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c74010070 con 0x7f7c7c108740 2024-04-26T18:10:40.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.727+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f7c7c02e490 con 0x7f7c7c108740 2024-04-26T18:10:40.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.727+0000 7f7c78ff9700 1 -- 172.21.15.90:0/2178271779 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7c7409bf70 con 0x7f7c7c108740 2024-04-26T18:10:40.727 INFO:teuthology.orchestra.run.smithi090.stdout:167503724613 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7c680a3c40 msgr2=0x7f7c680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7c680a3c40 0x7f7c680a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c006fd0 tx=0x7f7c6c008040 comp rx=0 tx=0).stop 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c108740 msgr2=0x7f7c7c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c108740 0x7f7c7c102f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7c74007ee0 tx=0x7f7c7400d3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 shutdown_connections 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7c680a3c40 0x7f7c680a6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c7c108740 0x7f7c7c102f00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c7c06bdf0 0x7f7c7c102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 --2- 172.21.15.90:0/2178271779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c7c06c7a0 0x7f7c7c1029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 >> 172.21.15.90:0/2178271779 conn(0x7f7c7c078370 msgr2=0x7f7c7c107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.729+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 shutdown_connections 2024-04-26T18:10:40.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.730+0000 7f7c813a7700 1 -- 172.21.15.90:0/2178271779 wait complete. 2024-04-26T18:10:40.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.762+0000 7f4b18495700 1 -- 172.21.15.90:0/2993571742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10101de0 msgr2=0x7f4b10102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.762+0000 7f4b18495700 1 --2- 172.21.15.90:0/2993571742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10101de0 0x7f4b10102260 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009990 tx=0x7f4b00009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.762+0000 7f4b18495700 1 -- 172.21.15.90:0/2993571742 shutdown_connections 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.762+0000 7f4b18495700 1 --2- 172.21.15.90:0/2993571742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b101027a0 0x7f4b10110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.762+0000 7f4b18495700 1 --2- 172.21.15.90:0/2993571742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b10108500 0x7f4b101088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.762+0000 7f4b18495700 1 --2- 172.21.15.90:0/2993571742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10101de0 0x7f4b10102260 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.762+0000 7f4b18495700 1 -- 172.21.15.90:0/2993571742 >> 172.21.15.90:0/2993571742 conn(0x7f4b10077640 msgr2=0x7f4b10077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 -- 172.21.15.90:0/2993571742 shutdown_connections 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 -- 172.21.15.90:0/2993571742 wait complete. 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 Processor -- start 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 -- start start 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b10101de0 0x7f4b1019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b101027a0 0x7f4b1019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10108500 0x7f4b101999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10113660 con 0x7f4b10108500 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101134e0 con 0x7f4b101027a0 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.763+0000 7f4b18495700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b101137e0 con 0x7f4b10101de0 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b16231700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b10101de0 0x7f4b1019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b16a32700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10108500 0x7f4b101999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b16231700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b10101de0 0x7f4b1019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:54098/0 (socket says 172.21.15.90:54098) 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b16a32700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10108500 0x7f4b101999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42182/0 (socket says 172.21.15.90:42182) 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b16231700 1 -- 172.21.15.90:0/2922813116 learned_addr learned my addr 172.21.15.90:0/2922813116 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b15a30700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b101027a0 0x7f4b1019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b15a30700 1 -- 172.21.15.90:0/2922813116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b10101de0 msgr2=0x7f4b1019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b15a30700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b10101de0 0x7f4b1019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b15a30700 1 -- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10108500 msgr2=0x7f4b101999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b15a30700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10108500 0x7f4b101999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b15a30700 1 -- 172.21.15.90:0/2922813116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1019a120 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b16a32700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10108500 0x7f4b101999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.764+0000 7f4b15a30700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b101027a0 0x7f4b1019fd40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b00000c00 tx=0x7f4b000047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.765+0000 7f4b077fe700 1 -- 172.21.15.90:0/2922813116 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00027070 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.765+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1019a3b0 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.765+0000 7f4b077fe700 1 -- 172.21.15.90:0/2922813116 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b00003f20 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.765+0000 7f4b077fe700 1 -- 172.21.15.90:0/2922813116 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00022c40 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.765+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b10102ef0 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.766+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1006a110 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.767+0000 7f4b077fe700 1 -- 172.21.15.90:0/2922813116 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4b0002a070 con 0x7f4b101027a0 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.768+0000 7f4b077fe700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4afc0a3c40 0x7f4afc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:40.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.768+0000 7f4b16231700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4afc0a3c40 0x7f4afc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.768+0000 7f4b077fe700 1 -- 172.21.15.90:0/2922813116 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f4b000d1b00 con 0x7f4b101027a0 2024-04-26T18:10:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.768+0000 7f4b16231700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4afc0a3c40 0x7f4afc0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b1019ab00 tx=0x7f4b0c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:40.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.772+0000 7f4b077fe700 1 -- 172.21.15.90:0/2922813116 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b00031070 con 0x7f4b101027a0 2024-04-26T18:10:40.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.975+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f4b1002e490 con 0x7f4b101027a0 2024-04-26T18:10:40.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.976+0000 7f4b077fe700 1 -- 172.21.15.90:0/2922813116 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b00022e20 con 0x7f4b101027a0 2024-04-26T18:10:40.976 INFO:teuthology.orchestra.run.smithi090.stdout:77309411411 2024-04-26T18:10:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4afc0a3c40 msgr2=0x7f4afc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4afc0a3c40 0x7f4afc0a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b1019ab00 tx=0x7f4b0c00a3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b101027a0 msgr2=0x7f4b1019fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:40.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b101027a0 0x7f4b1019fd40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4b00000c00 tx=0x7f4b000047b0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 shutdown_connections 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4afc0a3c40 0x7f4afc0a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b10108500 0x7f4b101999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b10101de0 0x7f4b1019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 --2- 172.21.15.90:0/2922813116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b101027a0 0x7f4b1019fd40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 >> 172.21.15.90:0/2922813116 conn(0x7f4b10077640 msgr2=0x7f4b10100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 shutdown_connections 2024-04-26T18:10:40.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:40.978+0000 7f4b18495700 1 -- 172.21.15.90:0/2922813116 wait complete. 2024-04-26T18:10:41.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:40 smithi090 ceph-mon[29180]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T18:10:41.254 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:40 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2178271779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:10:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:40 smithi119 ceph-mon[32752]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T18:10:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:40 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2178271779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:10:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:40 smithi167 ceph-mon[32751]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T18:10:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:40 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2178271779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:10:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:41 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2922813116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:10:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:41 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2922813116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:10:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:41 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2922813116' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:10:42.677 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743466 got 296352743472 for osd.9 2024-04-26T18:10:42.677 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:42.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 -- 172.21.15.90:0/3014334540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39681083d0 msgr2=0x7f39681087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:42.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 --2- 172.21.15.90:0/3014334540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39681083d0 0x7f39681087b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3958009990 tx=0x7f3958009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:42.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 -- 172.21.15.90:0/3014334540 shutdown_connections 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 --2- 172.21.15.90:0/3014334540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39681083d0 0x7f39681087b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 --2- 172.21.15.90:0/3014334540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3968102d10 0x7f396810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 --2- 172.21.15.90:0/3014334540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3968102350 0x7f39681027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 -- 172.21.15.90:0/3014334540 >> 172.21.15.90:0/3014334540 conn(0x7f3968078390 msgr2=0x7f39680787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 -- 172.21.15.90:0/3014334540 shutdown_connections 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.885+0000 7f396d10d700 1 -- 172.21.15.90:0/3014334540 wait complete. 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 Processor -- start 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 -- start start 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3968102350 0x7f3968104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3968102d10 0x7f3968104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681083d0 0x7f3968104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:42.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f39681125a0 con 0x7f3968102350 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3968112420 con 0x7f39681083d0 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.886+0000 7f396d10d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3968112720 con 0x7f3968102d10 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3968102350 0x7f3968104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3968102350 0x7f3968104000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57482/0 (socket says 172.21.15.90:57482) 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 -- 172.21.15.90:0/1035689158 learned_addr learned my addr 172.21.15.90:0/1035689158 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f396759e700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681083d0 0x7f3968104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f396659c700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3968102d10 0x7f3968104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 -- 172.21.15.90:0/1035689158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3968102d10 msgr2=0x7f3968104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3968102d10 0x7f3968104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 -- 172.21.15.90:0/1035689158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681083d0 msgr2=0x7f3968104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681083d0 0x7f3968104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 -- 172.21.15.90:0/1035689158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39681a8110 con 0x7f3968102350 2024-04-26T18:10:42.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f396759e700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681083d0 0x7f3968104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:10:42.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f3966d9d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3968102350 0x7f3968104000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3958000c00 tx=0x7f3958003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:42.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f394ffff700 1 -- 172.21.15.90:0/1035689158 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958027070 con 0x7f3968102350 2024-04-26T18:10:42.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39681a83a0 con 0x7f3968102350 2024-04-26T18:10:42.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f394ffff700 1 -- 172.21.15.90:0/1035689158 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f395800bdc0 con 0x7f3968102350 2024-04-26T18:10:42.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f394ffff700 1 -- 172.21.15.90:0/1035689158 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395800f6c0 con 0x7f3968102350 2024-04-26T18:10:42.889 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.887+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39681a87f0 con 0x7f3968102350 2024-04-26T18:10:42.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.888+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3968103a40 con 0x7f3968102350 2024-04-26T18:10:42.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.889+0000 7f394ffff700 1 -- 172.21.15.90:0/1035689158 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f395802a070 con 0x7f3968102350 2024-04-26T18:10:42.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.890+0000 7f394ffff700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f39500a3c40 0x7f39500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:42.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.890+0000 7f396659c700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f39500a3c40 0x7f39500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:42.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.890+0000 7f394ffff700 1 -- 172.21.15.90:0/1035689158 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f39580d1640 con 0x7f3968102350 2024-04-26T18:10:42.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.890+0000 7f396659c700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f39500a3c40 0x7f39500a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f395c006fd0 tx=0x7f395c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:42.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:42.894+0000 7f394ffff700 1 -- 172.21.15.90:0/1035689158 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f395805eff0 con 0x7f3968102350 2024-04-26T18:10:43.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.103+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f39681a8b10 con 0x7f3968102350 2024-04-26T18:10:43.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.103+0000 7f394ffff700 1 -- 172.21.15.90:0/1035689158 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3958013070 con 0x7f3968102350 2024-04-26T18:10:43.103 INFO:teuthology.orchestra.run.smithi090.stdout:141733920842 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.105+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f39500a3c40 msgr2=0x7f39500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.105+0000 7f396d10d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f39500a3c40 0x7f39500a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f395c006fd0 tx=0x7f395c008040 comp rx=0 tx=0).stop 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.105+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3968102350 msgr2=0x7f3968104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.105+0000 7f396d10d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3968102350 0x7f3968104000 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3958000c00 tx=0x7f3958003b40 comp rx=0 tx=0).stop 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 shutdown_connections 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f39500a3c40 0x7f39500a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3968102350 0x7f3968104000 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39681083d0 0x7f3968104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 --2- 172.21.15.90:0/1035689158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3968102d10 0x7f3968104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 >> 172.21.15.90:0/1035689158 conn(0x7f3968078390 msgr2=0x7f396806bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:43.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 shutdown_connections 2024-04-26T18:10:43.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.106+0000 7f396d10d700 1 -- 172.21.15.90:0/1035689158 wait complete. 2024-04-26T18:10:43.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 -- 172.21.15.90:0/4129884967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80641034c0 msgr2=0x7f8064103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:43.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 --2- 172.21.15.90:0/4129884967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80641034c0 0x7f8064103940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f80580099c0 tx=0x7f8058009cd0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 -- 172.21.15.90:0/4129884967 shutdown_connections 2024-04-26T18:10:43.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 --2- 172.21.15.90:0/4129884967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8064103e80 0x7f8064110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 --2- 172.21.15.90:0/4129884967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8064109560 0x7f8064109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 --2- 172.21.15.90:0/4129884967 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f80641034c0 0x7f8064103940 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 -- 172.21.15.90:0/4129884967 >> 172.21.15.90:0/4129884967 conn(0x7f806407a5d0 msgr2=0x7f806407a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:43.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 -- 172.21.15.90:0/4129884967 shutdown_connections 2024-04-26T18:10:43.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.123+0000 7f806a2ed700 1 -- 172.21.15.90:0/4129884967 wait complete. 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 Processor -- start 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 -- start start 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80641034c0 0x7f806419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8064103e80 0x7f806419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8064109560 0x7f8064199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8064113640 con 0x7f8064103e80 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f80641134c0 con 0x7f80641034c0 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806a2ed700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f80641137c0 con 0x7f8064109560 2024-04-26T18:10:43.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f80637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8064103e80 0x7f806419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f80637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8064103e80 0x7f806419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57494/0 (socket says 172.21.15.90:57494) 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f80637fe700 1 -- 172.21.15.90:0/374439231 learned_addr learned my addr 172.21.15.90:0/374439231 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.124+0000 7f806888a700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8064109560 0x7f8064199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f8063fff700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80641034c0 0x7f806419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80637fe700 1 -- 172.21.15.90:0/374439231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8064109560 msgr2=0x7f8064199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80637fe700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8064109560 0x7f8064199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80637fe700 1 -- 172.21.15.90:0/374439231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80641034c0 msgr2=0x7f806419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80637fe700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80641034c0 0x7f806419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80637fe700 1 -- 172.21.15.90:0/374439231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806419a0c0 con 0x7f8064103e80 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80637fe700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8064103e80 0x7f806419fc90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f805800b580 tx=0x7f80580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80617fa700 1 -- 172.21.15.90:0/374439231 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058027070 con 0x7f8064103e80 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80617fa700 1 -- 172.21.15.90:0/374439231 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8058003f20 con 0x7f8064103e80 2024-04-26T18:10:43.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f80617fa700 1 -- 172.21.15.90:0/374439231 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8058022d70 con 0x7f8064103e80 2024-04-26T18:10:43.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806419a350 con 0x7f8064103e80 2024-04-26T18:10:43.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.125+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80641a6620 con 0x7f8064103e80 2024-04-26T18:10:43.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.126+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8064104bd0 con 0x7f8064103e80 2024-04-26T18:10:43.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.128+0000 7f80617fa700 1 -- 172.21.15.90:0/374439231 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8058022500 con 0x7f8064103e80 2024-04-26T18:10:43.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.129+0000 7f80617fa700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f804c0a3cc0 0x7f804c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:43.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.129+0000 7f8063fff700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f804c0a3cc0 0x7f804c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:43.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.129+0000 7f80617fa700 1 -- 172.21.15.90:0/374439231 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f80580d12e0 con 0x7f8064103e80 2024-04-26T18:10:43.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.130+0000 7f8063fff700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f804c0a3cc0 0x7f804c0a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8054005950 tx=0x7f80540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:43.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.133+0000 7f80617fa700 1 -- 172.21.15.90:0/374439231 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8058013070 con 0x7f8064103e80 2024-04-26T18:10:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:42 smithi119 ceph-mon[32752]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T18:10:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:42 smithi167 ceph-mon[32751]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T18:10:43.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.342+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f806402e490 con 0x7f8064103e80 2024-04-26T18:10:43.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:42 smithi090 ceph-mon[29180]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-26T18:10:43.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.342+0000 7f80617fa700 1 -- 172.21.15.90:0/374439231 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f805809ddf0 con 0x7f8064103e80 2024-04-26T18:10:43.342 INFO:teuthology.orchestra.run.smithi090.stdout:201863462977 2024-04-26T18:10:43.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f804c0a3cc0 msgr2=0x7f804c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f804c0a3cc0 0x7f804c0a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8054005950 tx=0x7f80540058e0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8064103e80 msgr2=0x7f806419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8064103e80 0x7f806419fc90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f805800b580 tx=0x7f80580047b0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 shutdown_connections 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f804c0a3cc0 0x7f804c0a6180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8064109560 0x7f8064199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80641034c0 0x7f806419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 --2- 172.21.15.90:0/374439231 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8064103e80 0x7f806419fc90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 >> 172.21.15.90:0/374439231 conn(0x7f806407a5d0 msgr2=0x7f80641021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 shutdown_connections 2024-04-26T18:10:43.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:43.345+0000 7f806a2ed700 1 -- 172.21.15.90:0/374439231 wait complete. 2024-04-26T18:10:43.754 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920842 for osd.4 2024-04-26T18:10:43.754 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:44.156 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724605 got 167503724613 for osd.5 2024-04-26T18:10:44.156 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:43 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1035689158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:10:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:43 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/374439231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:10:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:43 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1035689158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:10:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:43 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/374439231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:10:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:43 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1035689158' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:10:44.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:43 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/374439231' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:10:44.839 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462971 got 201863462977 for osd.6 2024-04-26T18:10:44.839 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:44 smithi090 ceph-mon[29180]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T18:10:45.247 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411411 for osd.2 2024-04-26T18:10:45.247 DEBUG:teuthology.parallel:result is None 2024-04-26T18:10:45.247 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:10:45.248 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:10:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:44 smithi119 ceph-mon[32752]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T18:10:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:44 smithi167 ceph-mon[32751]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-26T18:10:45.513 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 -- 172.21.15.90:0/3235657427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 msgr2=0x7f5a5c1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3235657427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c1102d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5a5400b0a0 tx=0x7f5a5400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 -- 172.21.15.90:0/3235657427 shutdown_connections 2024-04-26T18:10:47.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3235657427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c1102d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3235657427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c109490 0x7f5a5c109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3235657427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a5c06b9c0 0x7f5a5c06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 -- 172.21.15.90:0/3235657427 >> 172.21.15.90:0/3235657427 conn(0x7f5a5c079290 msgr2=0x7f5a5c0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 -- 172.21.15.90:0/3235657427 shutdown_connections 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.128+0000 7f5a6119b700 1 -- 172.21.15.90:0/3235657427 wait complete. 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 Processor -- start 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 -- start start 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a5c06b9c0 0x7f5a5c1023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c109490 0x7f5a5c102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:47.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c113640 con 0x7f5a5c06c360 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1134c0 con 0x7f5a5c109490 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a6119b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5c1137c0 con 0x7f5a5c06b9c0 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c102920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57506/0 (socket says 172.21.15.90:57506) 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 -- 172.21.15.90:0/3476058599 learned_addr learned my addr 172.21.15.90:0/3476058599 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a5ad9d700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a5c06b9c0 0x7f5a5c1023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a5b59e700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c109490 0x7f5a5c102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 -- 172.21.15.90:0/3476058599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a5c06b9c0 msgr2=0x7f5a5c1023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a5c06b9c0 0x7f5a5c1023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 -- 172.21.15.90:0/3476058599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c109490 msgr2=0x7f5a5c102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c109490 0x7f5a5c102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.129+0000 7f5a53fff700 1 -- 172.21.15.90:0/3476058599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a5c1a80a0 con 0x7f5a5c06c360 2024-04-26T18:10:47.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.130+0000 7f5a53fff700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c102920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b530 tx=0x7f5a4400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:47.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.130+0000 7f5a58d99700 1 -- 172.21.15.90:0/3476058599 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44015020 con 0x7f5a5c06c360 2024-04-26T18:10:47.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.130+0000 7f5a58d99700 1 -- 172.21.15.90:0/3476058599 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a44011ce0 con 0x7f5a5c06c360 2024-04-26T18:10:47.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.130+0000 7f5a58d99700 1 -- 172.21.15.90:0/3476058599 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a44010e40 con 0x7f5a5c06c360 2024-04-26T18:10:47.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.130+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a5c1a8390 con 0x7f5a5c06c360 2024-04-26T18:10:47.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.130+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a5c1a88d0 con 0x7f5a5c06c360 2024-04-26T18:10:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.131+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a5c06a110 con 0x7f5a5c06c360 2024-04-26T18:10:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.132+0000 7f5a58d99700 1 -- 172.21.15.90:0/3476058599 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5a44018020 con 0x7f5a5c06c360 2024-04-26T18:10:47.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.132+0000 7f5a58d99700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a480a3cc0 0x7f5a480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:47.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.133+0000 7f5a5ad9d700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a480a3cc0 0x7f5a480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:47.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.133+0000 7f5a58d99700 1 -- 172.21.15.90:0/3476058599 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f5a440d0100 con 0x7f5a5c06c360 2024-04-26T18:10:47.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.133+0000 7f5a5ad9d700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a480a3cc0 0x7f5a480a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005950 tx=0x7f5a4c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:47.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.136+0000 7f5a58d99700 1 -- 172.21.15.90:0/3476058599 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a4401f020 con 0x7f5a5c06c360 2024-04-26T18:10:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:46 smithi119 ceph-mon[32752]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T18:10:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:46 smithi167 ceph-mon[32751]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T18:10:47.332 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.332+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5a5c103910 con 0x7f5a480a3cc0 2024-04-26T18:10:47.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:46 smithi090 ceph-mon[29180]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-26T18:10:47.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.339+0000 7f5a58d99700 1 -- 172.21.15.90:0/3476058599 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258827 (secure 0 0 0) 0x7f5a5c103910 con 0x7f5a480a3cc0 2024-04-26T18:10:47.344 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a480a3cc0 msgr2=0x7f5a480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a480a3cc0 0x7f5a480a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005950 tx=0x7f5a4c00a300 comp rx=0 tx=0).stop 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 msgr2=0x7f5a5c102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c102920 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5a4400b530 tx=0x7f5a4400b840 comp rx=0 tx=0).stop 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 shutdown_connections 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5a480a3cc0 0x7f5a480a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c109490 0x7f5a5c102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a5c06b9c0 0x7f5a5c1023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 --2- 172.21.15.90:0/3476058599 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5a5c06c360 0x7f5a5c102920 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:47.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 >> 172.21.15.90:0/3476058599 conn(0x7f5a5c079290 msgr2=0x7f5a5c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:47.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 shutdown_connections 2024-04-26T18:10:47.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:47.342+0000 7f5a6119b700 1 -- 172.21.15.90:0/3476058599 wait complete. 2024-04-26T18:10:47.347 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T18:10:48.088 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-04-26T18:10:45.898933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655287,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":774,"num_read_kb":508,"num_write":358,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379676,"kb_used_data":13884,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562148,"statfs":{"total":1151940427776,"available":1151551639552,"internally_reserved":0,"allocated":14217216,"data_stored":7833489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":46,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876561+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:10:31.876561+0000","last_peered":"2024-04-26T18:10:31.876561+0000","last_clean":"2024-04-26T18:10:31.876561+0000","last_became_active":"2024-04-26T18:08:47.377408+0000","last_became_peered":"2024-04-26T18:08:47.377408+0000","last_unstale":"2024-04-26T18:10:31.876561+0000","last_undegraded":"2024-04-26T18:10:31.876561+0000","last_fullsized":"2024-04-26T18:10:31.876561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:13.822258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.866886+0000","last_change":"2024-04-26T18:08:47.365125+0000","last_active":"2024-04-26T18:10:31.866886+0000","last_peered":"2024-04-26T18:10:31.866886+0000","last_clean":"2024-04-26T18:10:31.866886+0000","last_became_active":"2024-04-26T18:08:47.364847+0000","last_became_peered":"2024-04-26T18:08:47.364847+0000","last_unstale":"2024-04-26T18:10:31.866886+0000","last_undegraded":"2024-04-26T18:10:31.866886+0000","last_fullsized":"2024-04-26T18:10:31.866886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:16:37.890893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879920+0000","last_change":"2024-04-26T18:08:47.377580+0000","last_active":"2024-04-26T18:10:31.879920+0000","last_peered":"2024-04-26T18:10:31.879920+0000","last_clean":"2024-04-26T18:10:31.879920+0000","last_became_active":"2024-04-26T18:08:47.377219+0000","last_became_peered":"2024-04-26T18:08:47.377219+0000","last_unstale":"2024-04-26T18:10:31.879920+0000","last_undegraded":"2024-04-26T18:10:31.879920+0000","last_fullsized":"2024-04-26T18:10:31.879920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:12.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.877930+0000","last_change":"2024-04-26T18:08:47.371628+0000","last_active":"2024-04-26T18:10:31.877930+0000","last_peered":"2024-04-26T18:10:31.877930+0000","last_clean":"2024-04-26T18:10:31.877930+0000","last_became_active":"2024-04-26T18:08:47.370840+0000","last_became_peered":"2024-04-26T18:08:47.370840+0000","last_unstale":"2024-04-26T18:10:31.877930+0000","last_undegraded":"2024-04-26T18:10:31.877930+0000","last_fullsized":"2024-04-26T18:10:31.877930+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:11.362664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867410+0000","last_change":"2024-04-26T18:08:47.367632+0000","last_active":"2024-04-26T18:10:31.867410+0000","last_peered":"2024-04-26T18:10:31.867410+0000","last_clean":"2024-04-26T18:10:31.867410+0000","last_became_active":"2024-04-26T18:08:47.367316+0000","last_became_peered":"2024-04-26T18:08:47.367316+0000","last_unstale":"2024-04-26T18:10:31.867410+0000","last_undegraded":"2024-04-26T18:10:31.867410+0000","last_fullsized":"2024-04-26T18:10:31.867410+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:31:26.665292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.865713+0000","last_change":"2024-04-26T18:08:47.369742+0000","last_active":"2024-04-26T18:10:31.865713+0000","last_peered":"2024-04-26T18:10:31.865713+0000","last_clean":"2024-04-26T18:10:31.865713+0000","last_became_active":"2024-04-26T18:08:47.369230+0000","last_became_peered":"2024-04-26T18:08:47.369230+0000","last_unstale":"2024-04-26T18:10:31.865713+0000","last_undegraded":"2024-04-26T18:10:31.865713+0000","last_fullsized":"2024-04-26T18:10:31.865713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:02:04.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879644+0000","last_change":"2024-04-26T18:08:47.370657+0000","last_active":"2024-04-26T18:10:31.879644+0000","last_peered":"2024-04-26T18:10:31.879644+0000","last_clean":"2024-04-26T18:10:31.879644+0000","last_became_active":"2024-04-26T18:08:47.370133+0000","last_became_peered":"2024-04-26T18:08:47.370133+0000","last_unstale":"2024-04-26T18:10:31.879644+0000","last_undegraded":"2024-04-26T18:10:31.879644+0000","last_fullsized":"2024-04-26T18:10:31.879644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:31.232179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879611+0000","last_change":"2024-04-26T18:08:47.366221+0000","last_active":"2024-04-26T18:10:31.879611+0000","last_peered":"2024-04-26T18:10:31.879611+0000","last_clean":"2024-04-26T18:10:31.879611+0000","last_became_active":"2024-04-26T18:08:47.365990+0000","last_became_peered":"2024-04-26T18:08:47.365990+0000","last_unstale":"2024-04-26T18:10:31.879611+0000","last_undegraded":"2024-04-26T18:10:31.879611+0000","last_fullsized":"2024-04-26T18:10:31.879611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:43.707583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.872635+0000","last_change":"2024-04-26T18:08:47.373165+0000","last_active":"2024-04-26T18:10:31.872635+0000","last_peered":"2024-04-26T18:10:31.872635+0000","last_clean":"2024-04-26T18:10:31.872635+0000","last_became_active":"2024-04-26T18:08:47.372688+0000","last_became_peered":"2024-04-26T18:08:47.372688+0000","last_unstale":"2024-04-26T18:10:31.872635+0000","last_undegraded":"2024-04-26T18:10:31.872635+0000","last_fullsized":"2024-04-26T18:10:31.872635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:18.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.866081+0000","last_change":"2024-04-26T18:09:33.016811+0000","last_active":"2024-04-26T18:10:31.866081+0000","last_peered":"2024-04-26T18:10:31.866081+0000","last_clean":"2024-04-26T18:10:31.866081+0000","last_became_active":"2024-04-26T18:09:33.016267+0000","last_became_peered":"2024-04-26T18:09:33.016267+0000","last_unstale":"2024-04-26T18:10:31.866081+0000","last_undegraded":"2024-04-26T18:10:31.866081+0000","last_fullsized":"2024-04-26T18:10:31.866081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.002914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887360+0000","last_change":"2024-04-26T18:08:47.378643+0000","last_active":"2024-04-26T18:10:31.887360+0000","last_peered":"2024-04-26T18:10:31.887360+0000","last_clean":"2024-04-26T18:10:31.887360+0000","last_became_active":"2024-04-26T18:08:47.378329+0000","last_became_peered":"2024-04-26T18:08:47.378329+0000","last_unstale":"2024-04-26T18:10:31.887360+0000","last_undegraded":"2024-04-26T18:10:31.887360+0000","last_fullsized":"2024-04-26T18:10:31.887360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:59.016025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.859036+0000","last_change":"2024-04-26T18:08:47.371663+0000","last_active":"2024-04-26T18:10:31.859036+0000","last_peered":"2024-04-26T18:10:31.859036+0000","last_clean":"2024-04-26T18:10:31.859036+0000","last_became_active":"2024-04-26T18:08:47.371298+0000","last_became_peered":"2024-04-26T18:08:47.371298+0000","last_unstale":"2024-04-26T18:10:31.859036+0000","last_undegraded":"2024-04-26T18:10:31.859036+0000","last_fullsized":"2024-04-26T18:10:31.859036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:04.884698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.875198+0000","last_change":"2024-04-26T18:08:47.365179+0000","last_active":"2024-04-26T18:10:31.875198+0000","last_peered":"2024-04-26T18:10:31.875198+0000","last_clean":"2024-04-26T18:10:31.875198+0000","last_became_active":"2024-04-26T18:08:47.364888+0000","last_became_peered":"2024-04-26T18:08:47.364888+0000","last_unstale":"2024-04-26T18:10:31.875198+0000","last_undegraded":"2024-04-26T18:10:31.875198+0000","last_fullsized":"2024-04-26T18:10:31.875198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:34.082855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.864278+0000","last_change":"2024-04-26T18:08:47.364828+0000","last_active":"2024-04-26T18:10:31.864278+0000","last_peered":"2024-04-26T18:10:31.864278+0000","last_clean":"2024-04-26T18:10:31.864278+0000","last_became_active":"2024-04-26T18:08:47.364529+0000","last_became_peered":"2024-04-26T18:08:47.364529+0000","last_unstale":"2024-04-26T18:10:31.864278+0000","last_undegraded":"2024-04-26T18:10:31.864278+0000","last_fullsized":"2024-04-26T18:10:31.864278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.480258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880160+0000","last_change":"2024-04-26T18:08:47.373578+0000","last_active":"2024-04-26T18:10:31.880160+0000","last_peered":"2024-04-26T18:10:31.880160+0000","last_clean":"2024-04-26T18:10:31.880160+0000","last_became_active":"2024-04-26T18:08:47.373263+0000","last_became_peered":"2024-04-26T18:08:47.373263+0000","last_unstale":"2024-04-26T18:10:31.880160+0000","last_undegraded":"2024-04-26T18:10:31.880160+0000","last_fullsized":"2024-04-26T18:10:31.880160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:25.340241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.886796+0000","last_change":"2024-04-26T18:08:47.376617+0000","last_active":"2024-04-26T18:10:31.886796+0000","last_peered":"2024-04-26T18:10:31.886796+0000","last_clean":"2024-04-26T18:10:31.886796+0000","last_became_active":"2024-04-26T18:08:47.376290+0000","last_became_peered":"2024-04-26T18:08:47.376290+0000","last_unstale":"2024-04-26T18:10:31.886796+0000","last_undegraded":"2024-04-26T18:10:31.886796+0000","last_fullsized":"2024-04-26T18:10:31.886796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:51.187583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876709+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:10:31.876709+0000","last_peered":"2024-04-26T18:10:31.876709+0000","last_clean":"2024-04-26T18:10:31.876709+0000","last_became_active":"2024-04-26T18:08:47.377427+0000","last_became_peered":"2024-04-26T18:08:47.377427+0000","last_unstale":"2024-04-26T18:10:31.876709+0000","last_undegraded":"2024-04-26T18:10:31.876709+0000","last_fullsized":"2024-04-26T18:10:31.876709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.529500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.868655+0000","last_change":"2024-04-26T18:08:47.373314+0000","last_active":"2024-04-26T18:10:31.868655+0000","last_peered":"2024-04-26T18:10:31.868655+0000","last_clean":"2024-04-26T18:10:31.868655+0000","last_became_active":"2024-04-26T18:08:47.372868+0000","last_became_peered":"2024-04-26T18:08:47.372868+0000","last_unstale":"2024-04-26T18:10:31.868655+0000","last_undegraded":"2024-04-26T18:10:31.868655+0000","last_fullsized":"2024-04-26T18:10:31.868655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:19.642655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880658+0000","last_change":"2024-04-26T18:08:47.367593+0000","last_active":"2024-04-26T18:10:31.880658+0000","last_peered":"2024-04-26T18:10:31.880658+0000","last_clean":"2024-04-26T18:10:31.880658+0000","last_became_active":"2024-04-26T18:08:47.367243+0000","last_became_peered":"2024-04-26T18:08:47.367243+0000","last_unstale":"2024-04-26T18:10:31.880658+0000","last_undegraded":"2024-04-26T18:10:31.880658+0000","last_fullsized":"2024-04-26T18:10:31.880658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.380307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.009294+0000","last_change":"2024-04-26T18:08:44.339625+0000","last_active":"2024-04-26T18:09:33.009294+0000","last_peered":"2024-04-26T18:09:33.009294+0000","last_clean":"2024-04-26T18:09:33.009294+0000","last_became_active":"2024-04-26T18:08:44.339209+0000","last_became_peered":"2024-04-26T18:08:44.339209+0000","last_unstale":"2024-04-26T18:09:33.009294+0000","last_undegraded":"2024-04-26T18:09:33.009294+0000","last_fullsized":"2024-04-26T18:09:33.009294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:43.169885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.875738+0000","last_change":"2024-04-26T18:08:47.375884+0000","last_active":"2024-04-26T18:10:31.875738+0000","last_peered":"2024-04-26T18:10:31.875738+0000","last_clean":"2024-04-26T18:10:31.875738+0000","last_became_active":"2024-04-26T18:08:47.375557+0000","last_became_peered":"2024-04-26T18:08:47.375557+0000","last_unstale":"2024-04-26T18:10:31.875738+0000","last_undegraded":"2024-04-26T18:10:31.875738+0000","last_fullsized":"2024-04-26T18:10:31.875738+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.593910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014708+0000","last_change":"2024-04-26T18:08:44.557167+0000","last_active":"2024-04-26T18:09:33.014708+0000","last_peered":"2024-04-26T18:09:33.014708+0000","last_clean":"2024-04-26T18:09:33.014708+0000","last_became_active":"2024-04-26T18:08:44.556716+0000","last_became_peered":"2024-04-26T18:08:44.556716+0000","last_unstale":"2024-04-26T18:09:33.014708+0000","last_undegraded":"2024-04-26T18:09:33.014708+0000","last_fullsized":"2024-04-26T18:09:33.014708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.881072+0000","last_change":"2024-04-26T18:08:47.377612+0000","last_active":"2024-04-26T18:10:31.881072+0000","last_peered":"2024-04-26T18:10:31.881072+0000","last_clean":"2024-04-26T18:10:31.881072+0000","last_became_active":"2024-04-26T18:08:47.377231+0000","last_became_peered":"2024-04-26T18:08:47.377231+0000","last_unstale":"2024-04-26T18:10:31.881072+0000","last_undegraded":"2024-04-26T18:10:31.881072+0000","last_fullsized":"2024-04-26T18:10:31.881072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:11.594373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.015269+0000","last_change":"2024-04-26T18:08:44.343429+0000","last_active":"2024-04-26T18:09:33.015269+0000","last_peered":"2024-04-26T18:09:33.015269+0000","last_clean":"2024-04-26T18:09:33.015269+0000","last_became_active":"2024-04-26T18:08:44.343103+0000","last_became_peered":"2024-04-26T18:08:44.343103+0000","last_unstale":"2024-04-26T18:09:33.015269+0000","last_undegraded":"2024-04-26T18:09:33.015269+0000","last_fullsized":"2024-04-26T18:09:33.015269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:52.244159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882838+0000","last_change":"2024-04-26T18:08:47.373130+0000","last_active":"2024-04-26T18:10:31.882838+0000","last_peered":"2024-04-26T18:10:31.882838+0000","last_clean":"2024-04-26T18:10:31.882838+0000","last_became_active":"2024-04-26T18:08:47.372726+0000","last_became_peered":"2024-04-26T18:08:47.372726+0000","last_unstale":"2024-04-26T18:10:31.882838+0000","last_undegraded":"2024-04-26T18:10:31.882838+0000","last_fullsized":"2024-04-26T18:10:31.882838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:44.461177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014700+0000","last_change":"2024-04-26T18:08:44.340152+0000","last_active":"2024-04-26T18:09:33.014700+0000","last_peered":"2024-04-26T18:09:33.014700+0000","last_clean":"2024-04-26T18:09:33.014700+0000","last_became_active":"2024-04-26T18:08:44.339684+0000","last_became_peered":"2024-04-26T18:08:44.339684+0000","last_unstale":"2024-04-26T18:09:33.014700+0000","last_undegraded":"2024-04-26T18:09:33.014700+0000","last_fullsized":"2024-04-26T18:09:33.014700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:49.961201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880231+0000","last_change":"2024-04-26T18:08:47.375387+0000","last_active":"2024-04-26T18:10:31.880231+0000","last_peered":"2024-04-26T18:10:31.880231+0000","last_clean":"2024-04-26T18:10:31.880231+0000","last_became_active":"2024-04-26T18:08:47.374689+0000","last_became_peered":"2024-04-26T18:08:47.374689+0000","last_unstale":"2024-04-26T18:10:31.880231+0000","last_undegraded":"2024-04-26T18:10:31.880231+0000","last_fullsized":"2024-04-26T18:10:31.880231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:05.618804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":137,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:42.094590+0000","last_change":"2024-04-26T18:08:44.344167+0000","last_active":"2024-04-26T18:10:42.094590+0000","last_peered":"2024-04-26T18:10:42.094590+0000","last_clean":"2024-04-26T18:10:42.094590+0000","last_became_active":"2024-04-26T18:08:44.343646+0000","last_became_peered":"2024-04-26T18:08:44.343646+0000","last_unstale":"2024-04-26T18:10:42.094590+0000","last_undegraded":"2024-04-26T18:10:42.094590+0000","last_fullsized":"2024-04-26T18:10:42.094590+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:53.859483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867790+0000","last_change":"2024-04-26T18:08:47.370546+0000","last_active":"2024-04-26T18:10:31.867790+0000","last_peered":"2024-04-26T18:10:31.867790+0000","last_clean":"2024-04-26T18:10:31.867790+0000","last_became_active":"2024-04-26T18:08:47.370220+0000","last_became_peered":"2024-04-26T18:08:47.370220+0000","last_unstale":"2024-04-26T18:10:31.867790+0000","last_undegraded":"2024-04-26T18:10:31.867790+0000","last_fullsized":"2024-04-26T18:10:31.867790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:05.139824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.010056+0000","last_change":"2024-04-26T18:08:44.347031+0000","last_active":"2024-04-26T18:09:33.010056+0000","last_peered":"2024-04-26T18:09:33.010056+0000","last_clean":"2024-04-26T18:09:33.010056+0000","last_became_active":"2024-04-26T18:08:44.346650+0000","last_became_peered":"2024-04-26T18:08:44.346650+0000","last_unstale":"2024-04-26T18:09:33.010056+0000","last_undegraded":"2024-04-26T18:09:33.010056+0000","last_fullsized":"2024-04-26T18:09:33.010056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:50.946449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879498+0000","last_change":"2024-04-26T18:08:47.374230+0000","last_active":"2024-04-26T18:10:31.879498+0000","last_peered":"2024-04-26T18:10:31.879498+0000","last_clean":"2024-04-26T18:10:31.879498+0000","last_became_active":"2024-04-26T18:08:47.370780+0000","last_became_peered":"2024-04-26T18:08:47.370780+0000","last_unstale":"2024-04-26T18:10:31.879498+0000","last_undegraded":"2024-04-26T18:10:31.879498+0000","last_fullsized":"2024-04-26T18:10:31.879498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:01.171195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.523771+0000","last_change":"2024-04-26T18:08:44.338318+0000","last_active":"2024-04-26T18:09:33.523771+0000","last_peered":"2024-04-26T18:09:33.523771+0000","last_clean":"2024-04-26T18:09:33.523771+0000","last_became_active":"2024-04-26T18:08:44.338009+0000","last_became_peered":"2024-04-26T18:08:44.338009+0000","last_unstale":"2024-04-26T18:09:33.523771+0000","last_undegraded":"2024-04-26T18:09:33.523771+0000","last_fullsized":"2024-04-26T18:09:33.523771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:33.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.872840+0000","last_change":"2024-04-26T18:08:47.363924+0000","last_active":"2024-04-26T18:10:31.872840+0000","last_peered":"2024-04-26T18:10:31.872840+0000","last_clean":"2024-04-26T18:10:31.872840+0000","last_became_active":"2024-04-26T18:08:47.363616+0000","last_became_peered":"2024-04-26T18:08:47.363616+0000","last_unstale":"2024-04-26T18:10:31.872840+0000","last_undegraded":"2024-04-26T18:10:31.872840+0000","last_fullsized":"2024-04-26T18:10:31.872840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:27:08.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.010902+0000","last_change":"2024-04-26T18:08:44.343350+0000","last_active":"2024-04-26T18:09:33.010902+0000","last_peered":"2024-04-26T18:09:33.010902+0000","last_clean":"2024-04-26T18:09:33.010902+0000","last_became_active":"2024-04-26T18:08:44.343040+0000","last_became_peered":"2024-04-26T18:08:44.343040+0000","last_unstale":"2024-04-26T18:09:33.010902+0000","last_undegraded":"2024-04-26T18:09:33.010902+0000","last_fullsized":"2024-04-26T18:09:33.010902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:09.427785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.869466+0000","last_change":"2024-04-26T18:08:47.360180+0000","last_active":"2024-04-26T18:10:31.869466+0000","last_peered":"2024-04-26T18:10:31.869466+0000","last_clean":"2024-04-26T18:10:31.869466+0000","last_became_active":"2024-04-26T18:08:47.359860+0000","last_became_peered":"2024-04-26T18:08:47.359860+0000","last_unstale":"2024-04-26T18:10:31.869466+0000","last_undegraded":"2024-04-26T18:10:31.869466+0000","last_fullsized":"2024-04-26T18:10:31.869466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:00.532589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:44.097224+0000","last_change":"2024-04-26T18:08:44.343942+0000","last_active":"2024-04-26T18:10:44.097224+0000","last_peered":"2024-04-26T18:10:44.097224+0000","last_clean":"2024-04-26T18:10:44.097224+0000","last_became_active":"2024-04-26T18:08:44.343581+0000","last_became_peered":"2024-04-26T18:08:44.343581+0000","last_unstale":"2024-04-26T18:10:44.097224+0000","last_undegraded":"2024-04-26T18:10:44.097224+0000","last_fullsized":"2024-04-26T18:10:44.097224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:32.044284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882269+0000","last_change":"2024-04-26T18:08:47.372303+0000","last_active":"2024-04-26T18:10:31.882269+0000","last_peered":"2024-04-26T18:10:31.882269+0000","last_clean":"2024-04-26T18:10:31.882269+0000","last_became_active":"2024-04-26T18:08:47.371961+0000","last_became_peered":"2024-04-26T18:08:47.371961+0000","last_unstale":"2024-04-26T18:10:31.882269+0000","last_undegraded":"2024-04-26T18:10:31.882269+0000","last_fullsized":"2024-04-26T18:10:31.882269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:19.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012895+0000","last_change":"2024-04-26T18:08:44.345681+0000","last_active":"2024-04-26T18:09:33.012895+0000","last_peered":"2024-04-26T18:09:33.012895+0000","last_clean":"2024-04-26T18:09:33.012895+0000","last_became_active":"2024-04-26T18:08:44.345329+0000","last_became_peered":"2024-04-26T18:08:44.345329+0000","last_unstale":"2024-04-26T18:09:33.012895+0000","last_undegraded":"2024-04-26T18:09:33.012895+0000","last_fullsized":"2024-04-26T18:09:33.012895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:59.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878799+0000","last_change":"2024-04-26T18:08:47.370773+0000","last_active":"2024-04-26T18:10:31.878799+0000","last_peered":"2024-04-26T18:10:31.878799+0000","last_clean":"2024-04-26T18:10:31.878799+0000","last_became_active":"2024-04-26T18:08:47.370490+0000","last_became_peered":"2024-04-26T18:08:47.370490+0000","last_unstale":"2024-04-26T18:10:31.878799+0000","last_undegraded":"2024-04-26T18:10:31.878799+0000","last_fullsized":"2024-04-26T18:10:31.878799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:01.935335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.011664+0000","last_change":"2024-04-26T18:08:44.555714+0000","last_active":"2024-04-26T18:09:33.011664+0000","last_peered":"2024-04-26T18:09:33.011664+0000","last_clean":"2024-04-26T18:09:33.011664+0000","last_became_active":"2024-04-26T18:08:44.555184+0000","last_became_peered":"2024-04-26T18:08:44.555184+0000","last_unstale":"2024-04-26T18:09:33.011664+0000","last_undegraded":"2024-04-26T18:09:33.011664+0000","last_fullsized":"2024-04-26T18:09:33.011664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:01.433764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882986+0000","last_change":"2024-04-26T18:09:33.525026+0000","last_active":"2024-04-26T18:10:31.882986+0000","last_peered":"2024-04-26T18:10:31.882986+0000","last_clean":"2024-04-26T18:10:31.882986+0000","last_became_active":"2024-04-26T18:09:33.524587+0000","last_became_peered":"2024-04-26T18:09:33.524587+0000","last_unstale":"2024-04-26T18:10:31.882986+0000","last_undegraded":"2024-04-26T18:10:31.882986+0000","last_fullsized":"2024-04-26T18:10:31.882986+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:39:48.534737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":272,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:45.104934+0000","last_change":"2024-04-26T18:08:44.346346+0000","last_active":"2024-04-26T18:10:45.104934+0000","last_peered":"2024-04-26T18:10:45.104934+0000","last_clean":"2024-04-26T18:10:45.104934+0000","last_became_active":"2024-04-26T18:08:44.345999+0000","last_became_peered":"2024-04-26T18:08:44.345999+0000","last_unstale":"2024-04-26T18:10:45.104934+0000","last_undegraded":"2024-04-26T18:10:45.104934+0000","last_fullsized":"2024-04-26T18:10:45.104934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:27.277880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":214,"num_read_kb":165,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.860229+0000","last_change":"2024-04-26T18:08:47.366303+0000","last_active":"2024-04-26T18:10:31.860229+0000","last_peered":"2024-04-26T18:10:31.860229+0000","last_clean":"2024-04-26T18:10:31.860229+0000","last_became_active":"2024-04-26T18:08:47.365861+0000","last_became_peered":"2024-04-26T18:08:47.365861+0000","last_unstale":"2024-04-26T18:10:31.860229+0000","last_undegraded":"2024-04-26T18:10:31.860229+0000","last_fullsized":"2024-04-26T18:10:31.860229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:31.413589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.022912+0000","last_change":"2024-04-26T18:09:33.022912+0000","last_active":"2024-04-26T18:09:33.022912+0000","last_peered":"2024-04-26T18:09:33.022912+0000","last_clean":"2024-04-26T18:09:33.022912+0000","last_became_active":"2024-04-26T18:09:33.015390+0000","last_became_peered":"2024-04-26T18:09:33.015390+0000","last_unstale":"2024-04-26T18:09:33.022912+0000","last_undegraded":"2024-04-26T18:09:33.022912+0000","last_fullsized":"2024-04-26T18:09:33.022912+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:53.980531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.874360+0000","last_change":"2024-04-26T18:08:47.367835+0000","last_active":"2024-04-26T18:10:31.874360+0000","last_peered":"2024-04-26T18:10:31.874360+0000","last_clean":"2024-04-26T18:10:31.874360+0000","last_became_active":"2024-04-26T18:08:47.367517+0000","last_became_peered":"2024-04-26T18:08:47.367517+0000","last_unstale":"2024-04-26T18:10:31.874360+0000","last_undegraded":"2024-04-26T18:10:31.874360+0000","last_fullsized":"2024-04-26T18:10:31.874360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:54.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.011273+0000","last_change":"2024-04-26T18:08:44.344161+0000","last_active":"2024-04-26T18:09:33.011273+0000","last_peered":"2024-04-26T18:09:33.011273+0000","last_clean":"2024-04-26T18:09:33.011273+0000","last_became_active":"2024-04-26T18:08:44.343635+0000","last_became_peered":"2024-04-26T18:08:44.343635+0000","last_unstale":"2024-04-26T18:09:33.011273+0000","last_undegraded":"2024-04-26T18:09:33.011273+0000","last_fullsized":"2024-04-26T18:09:33.011273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.946584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.881640+0000","last_change":"2024-04-26T18:09:33.524351+0000","last_active":"2024-04-26T18:10:31.881640+0000","last_peered":"2024-04-26T18:10:31.881640+0000","last_clean":"2024-04-26T18:10:31.881640+0000","last_became_active":"2024-04-26T18:09:33.523953+0000","last_became_peered":"2024-04-26T18:09:33.523953+0000","last_unstale":"2024-04-26T18:10:31.881640+0000","last_undegraded":"2024-04-26T18:10:31.881640+0000","last_fullsized":"2024-04-26T18:10:31.881640+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:22.887498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":135,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:42.095426+0000","last_change":"2024-04-26T18:08:44.345278+0000","last_active":"2024-04-26T18:10:42.095426+0000","last_peered":"2024-04-26T18:10:42.095426+0000","last_clean":"2024-04-26T18:10:42.095426+0000","last_became_active":"2024-04-26T18:08:44.344531+0000","last_became_peered":"2024-04-26T18:08:44.344531+0000","last_unstale":"2024-04-26T18:10:42.095426+0000","last_undegraded":"2024-04-26T18:10:42.095426+0000","last_fullsized":"2024-04-26T18:10:42.095426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:02.667622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.885848+0000","last_change":"2024-04-26T18:08:47.369542+0000","last_active":"2024-04-26T18:10:31.885848+0000","last_peered":"2024-04-26T18:10:31.885848+0000","last_clean":"2024-04-26T18:10:31.885848+0000","last_became_active":"2024-04-26T18:08:47.369181+0000","last_became_peered":"2024-04-26T18:08:47.369181+0000","last_unstale":"2024-04-26T18:10:31.885848+0000","last_undegraded":"2024-04-26T18:10:31.885848+0000","last_fullsized":"2024-04-26T18:10:31.885848+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:56.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014706+0000","last_change":"2024-04-26T18:08:44.555580+0000","last_active":"2024-04-26T18:09:33.014706+0000","last_peered":"2024-04-26T18:09:33.014706+0000","last_clean":"2024-04-26T18:09:33.014706+0000","last_became_active":"2024-04-26T18:08:44.555247+0000","last_became_peered":"2024-04-26T18:08:44.555247+0000","last_unstale":"2024-04-26T18:09:33.014706+0000","last_undegraded":"2024-04-26T18:09:33.014706+0000","last_fullsized":"2024-04-26T18:09:33.014706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:58:55.207355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879529+0000","last_change":"2024-04-26T18:08:47.372300+0000","last_active":"2024-04-26T18:10:31.879529+0000","last_peered":"2024-04-26T18:10:31.879529+0000","last_clean":"2024-04-26T18:10:31.879529+0000","last_became_active":"2024-04-26T18:08:47.371952+0000","last_became_peered":"2024-04-26T18:08:47.371952+0000","last_unstale":"2024-04-26T18:10:31.879529+0000","last_undegraded":"2024-04-26T18:10:31.879529+0000","last_fullsized":"2024-04-26T18:10:31.879529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:58.254691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014666+0000","last_change":"2024-04-26T18:08:44.340926+0000","last_active":"2024-04-26T18:09:33.014666+0000","last_peered":"2024-04-26T18:09:33.014666+0000","last_clean":"2024-04-26T18:09:33.014666+0000","last_became_active":"2024-04-26T18:08:44.340563+0000","last_became_peered":"2024-04-26T18:08:44.340563+0000","last_unstale":"2024-04-26T18:09:33.014666+0000","last_undegraded":"2024-04-26T18:09:33.014666+0000","last_fullsized":"2024-04-26T18:09:33.014666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:57.803017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867616+0000","last_change":"2024-04-26T18:09:33.021890+0000","last_active":"2024-04-26T18:10:31.867616+0000","last_peered":"2024-04-26T18:10:31.867616+0000","last_clean":"2024-04-26T18:10:31.867616+0000","last_became_active":"2024-04-26T18:09:33.021446+0000","last_became_peered":"2024-04-26T18:09:33.021446+0000","last_unstale":"2024-04-26T18:10:31.867616+0000","last_undegraded":"2024-04-26T18:10:31.867616+0000","last_fullsized":"2024-04-26T18:10:31.867616+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:29.158581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.009219+0000","last_change":"2024-04-26T18:08:44.555076+0000","last_active":"2024-04-26T18:09:33.009219+0000","last_peered":"2024-04-26T18:09:33.009219+0000","last_clean":"2024-04-26T18:09:33.009219+0000","last_became_active":"2024-04-26T18:08:44.554756+0000","last_became_peered":"2024-04-26T18:08:44.554756+0000","last_unstale":"2024-04-26T18:09:33.009219+0000","last_undegraded":"2024-04-26T18:09:33.009219+0000","last_fullsized":"2024-04-26T18:09:33.009219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:21.950644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.871347+0000","last_change":"2024-04-26T18:08:47.366370+0000","last_active":"2024-04-26T18:10:31.871347+0000","last_peered":"2024-04-26T18:10:31.871347+0000","last_clean":"2024-04-26T18:10:31.871347+0000","last_became_active":"2024-04-26T18:08:47.365997+0000","last_became_peered":"2024-04-26T18:08:47.365997+0000","last_unstale":"2024-04-26T18:10:31.871347+0000","last_undegraded":"2024-04-26T18:10:31.871347+0000","last_fullsized":"2024-04-26T18:10:31.871347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:51.643438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012712+0000","last_change":"2024-04-26T18:08:44.339299+0000","last_active":"2024-04-26T18:09:33.012712+0000","last_peered":"2024-04-26T18:09:33.012712+0000","last_clean":"2024-04-26T18:09:33.012712+0000","last_became_active":"2024-04-26T18:08:44.338932+0000","last_became_peered":"2024-04-26T18:08:44.338932+0000","last_unstale":"2024-04-26T18:09:33.012712+0000","last_undegraded":"2024-04-26T18:09:33.012712+0000","last_fullsized":"2024-04-26T18:09:33.012712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:18.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867578+0000","last_change":"2024-04-26T18:08:47.369924+0000","last_active":"2024-04-26T18:10:31.867578+0000","last_peered":"2024-04-26T18:10:31.867578+0000","last_clean":"2024-04-26T18:10:31.867578+0000","last_became_active":"2024-04-26T18:08:47.369048+0000","last_became_peered":"2024-04-26T18:08:47.369048+0000","last_unstale":"2024-04-26T18:10:31.867578+0000","last_undegraded":"2024-04-26T18:10:31.867578+0000","last_fullsized":"2024-04-26T18:10:31.867578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:19.110288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878948+0000","last_change":"2024-04-26T18:08:47.370836+0000","last_active":"2024-04-26T18:10:31.878948+0000","last_peered":"2024-04-26T18:10:31.878948+0000","last_clean":"2024-04-26T18:10:31.878948+0000","last_became_active":"2024-04-26T18:08:47.370160+0000","last_became_peered":"2024-04-26T18:08:47.370160+0000","last_unstale":"2024-04-26T18:10:31.878948+0000","last_undegraded":"2024-04-26T18:10:31.878948+0000","last_fullsized":"2024-04-26T18:10:31.878948+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:11:21.285466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:43.096314+0000","last_change":"2024-04-26T18:08:44.554960+0000","last_active":"2024-04-26T18:10:43.096314+0000","last_peered":"2024-04-26T18:10:43.096314+0000","last_clean":"2024-04-26T18:10:43.096314+0000","last_became_active":"2024-04-26T18:08:44.554607+0000","last_became_peered":"2024-04-26T18:08:44.554607+0000","last_unstale":"2024-04-26T18:10:43.096314+0000","last_undegraded":"2024-04-26T18:10:43.096314+0000","last_fullsized":"2024-04-26T18:10:43.096314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:22:26.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882140+0000","last_change":"2024-04-26T18:08:47.372528+0000","last_active":"2024-04-26T18:10:31.882140+0000","last_peered":"2024-04-26T18:10:31.882140+0000","last_clean":"2024-04-26T18:10:31.882140+0000","last_became_active":"2024-04-26T18:08:47.372215+0000","last_became_peered":"2024-04-26T18:08:47.372215+0000","last_unstale":"2024-04-26T18:10:31.882140+0000","last_undegraded":"2024-04-26T18:10:31.882140+0000","last_fullsized":"2024-04-26T18:10:31.882140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:39.366917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.020531+0000","last_change":"2024-04-26T18:08:44.342667+0000","last_active":"2024-04-26T18:09:33.020531+0000","last_peered":"2024-04-26T18:09:33.020531+0000","last_clean":"2024-04-26T18:09:33.020531+0000","last_became_active":"2024-04-26T18:08:44.342170+0000","last_became_peered":"2024-04-26T18:08:44.342170+0000","last_unstale":"2024-04-26T18:09:33.020531+0000","last_undegraded":"2024-04-26T18:09:33.020531+0000","last_fullsized":"2024-04-26T18:09:33.020531+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:50.117177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":362,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012799+0000","last_change":"2024-04-26T18:07:24.173301+0000","last_active":"2024-04-26T18:09:33.012799+0000","last_peered":"2024-04-26T18:09:33.012799+0000","last_clean":"2024-04-26T18:09:33.012799+0000","last_became_active":"2024-04-26T18:07:24.172791+0000","last_became_peered":"2024-04-26T18:07:24.172791+0000","last_unstale":"2024-04-26T18:09:33.012799+0000","last_undegraded":"2024-04-26T18:09:33.012799+0000","last_fullsized":"2024-04-26T18:09:33.012799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_clean_scrub_stamp":"2024-04-26T18:03:58.528256+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:32.468624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876453+0000","last_change":"2024-04-26T18:08:47.370720+0000","last_active":"2024-04-26T18:10:31.876453+0000","last_peered":"2024-04-26T18:10:31.876453+0000","last_clean":"2024-04-26T18:10:31.876453+0000","last_became_active":"2024-04-26T18:08:47.370414+0000","last_became_peered":"2024-04-26T18:08:47.370414+0000","last_unstale":"2024-04-26T18:10:31.876453+0000","last_undegraded":"2024-04-26T18:10:31.876453+0000","last_fullsized":"2024-04-26T18:10:31.876453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:37.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012541+0000","last_change":"2024-04-26T18:08:44.347077+0000","last_active":"2024-04-26T18:09:33.012541+0000","last_peered":"2024-04-26T18:09:33.012541+0000","last_clean":"2024-04-26T18:09:33.012541+0000","last_became_active":"2024-04-26T18:08:44.346481+0000","last_became_peered":"2024-04-26T18:08:44.346481+0000","last_unstale":"2024-04-26T18:09:33.012541+0000","last_undegraded":"2024-04-26T18:09:33.012541+0000","last_fullsized":"2024-04-26T18:09:33.012541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:08.840814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.025243+0000","last_change":"2024-04-26T18:09:33.025243+0000","last_active":"2024-04-26T18:09:33.025243+0000","last_peered":"2024-04-26T18:09:33.025243+0000","last_clean":"2024-04-26T18:09:33.025243+0000","last_became_active":"2024-04-26T18:09:33.015430+0000","last_became_peered":"2024-04-26T18:09:33.015430+0000","last_unstale":"2024-04-26T18:09:33.025243+0000","last_undegraded":"2024-04-26T18:09:33.025243+0000","last_fullsized":"2024-04-26T18:09:33.025243+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:54.271435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.877090+0000","last_change":"2024-04-26T18:08:47.366552+0000","last_active":"2024-04-26T18:10:31.877090+0000","last_peered":"2024-04-26T18:10:31.877090+0000","last_clean":"2024-04-26T18:10:31.877090+0000","last_became_active":"2024-04-26T18:08:47.366183+0000","last_became_peered":"2024-04-26T18:08:47.366183+0000","last_unstale":"2024-04-26T18:10:31.877090+0000","last_undegraded":"2024-04-26T18:10:31.877090+0000","last_fullsized":"2024-04-26T18:10:31.877090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:36.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.009328+0000","last_change":"2024-04-26T18:08:44.338078+0000","last_active":"2024-04-26T18:09:33.009328+0000","last_peered":"2024-04-26T18:09:33.009328+0000","last_clean":"2024-04-26T18:09:33.009328+0000","last_became_active":"2024-04-26T18:08:44.337715+0000","last_became_peered":"2024-04-26T18:08:44.337715+0000","last_unstale":"2024-04-26T18:09:33.009328+0000","last_undegraded":"2024-04-26T18:09:33.009328+0000","last_fullsized":"2024-04-26T18:09:33.009328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.912993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878815+0000","last_change":"2024-04-26T18:08:47.371321+0000","last_active":"2024-04-26T18:10:31.878815+0000","last_peered":"2024-04-26T18:10:31.878815+0000","last_clean":"2024-04-26T18:10:31.878815+0000","last_became_active":"2024-04-26T18:08:47.371015+0000","last_became_peered":"2024-04-26T18:08:47.371015+0000","last_unstale":"2024-04-26T18:10:31.878815+0000","last_undegraded":"2024-04-26T18:10:31.878815+0000","last_fullsized":"2024-04-26T18:10:31.878815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.020792+0000","last_change":"2024-04-26T18:08:44.420101+0000","last_active":"2024-04-26T18:09:33.020792+0000","last_peered":"2024-04-26T18:09:33.020792+0000","last_clean":"2024-04-26T18:09:33.020792+0000","last_became_active":"2024-04-26T18:08:44.419787+0000","last_became_peered":"2024-04-26T18:08:44.419787+0000","last_unstale":"2024-04-26T18:09:33.020792+0000","last_undegraded":"2024-04-26T18:09:33.020792+0000","last_fullsized":"2024-04-26T18:09:33.020792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.485425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878338+0000","last_change":"2024-04-26T18:08:47.372309+0000","last_active":"2024-04-26T18:10:31.878338+0000","last_peered":"2024-04-26T18:10:31.878338+0000","last_clean":"2024-04-26T18:10:31.878338+0000","last_became_active":"2024-04-26T18:08:47.371980+0000","last_became_peered":"2024-04-26T18:08:47.371980+0000","last_unstale":"2024-04-26T18:10:31.878338+0000","last_undegraded":"2024-04-26T18:10:31.878338+0000","last_fullsized":"2024-04-26T18:10:31.878338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:05.011077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012672+0000","last_change":"2024-04-26T18:08:44.555500+0000","last_active":"2024-04-26T18:09:33.012672+0000","last_peered":"2024-04-26T18:09:33.012672+0000","last_clean":"2024-04-26T18:09:33.012672+0000","last_became_active":"2024-04-26T18:08:44.555137+0000","last_became_peered":"2024-04-26T18:08:44.555137+0000","last_unstale":"2024-04-26T18:09:33.012672+0000","last_undegraded":"2024-04-26T18:09:33.012672+0000","last_fullsized":"2024-04-26T18:09:33.012672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:10:05.723654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.839228+0000","last_change":"2024-04-26T18:08:47.364215+0000","last_active":"2024-04-26T18:10:31.839228+0000","last_peered":"2024-04-26T18:10:31.839228+0000","last_clean":"2024-04-26T18:10:31.839228+0000","last_became_active":"2024-04-26T18:08:47.363862+0000","last_became_peered":"2024-04-26T18:08:47.363862+0000","last_unstale":"2024-04-26T18:10:31.839228+0000","last_undegraded":"2024-04-26T18:10:31.839228+0000","last_fullsized":"2024-04-26T18:10:31.839228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:27.020100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.013914+0000","last_change":"2024-04-26T18:08:44.343506+0000","last_active":"2024-04-26T18:09:33.013914+0000","last_peered":"2024-04-26T18:09:33.013914+0000","last_clean":"2024-04-26T18:09:33.013914+0000","last_became_active":"2024-04-26T18:08:44.343163+0000","last_became_peered":"2024-04-26T18:08:44.343163+0000","last_unstale":"2024-04-26T18:09:33.013914+0000","last_undegraded":"2024-04-26T18:09:33.013914+0000","last_fullsized":"2024-04-26T18:09:33.013914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:09.941391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.862510+0000","last_change":"2024-04-26T18:08:47.365572+0000","last_active":"2024-04-26T18:10:31.862510+0000","last_peered":"2024-04-26T18:10:31.862510+0000","last_clean":"2024-04-26T18:10:31.862510+0000","last_became_active":"2024-04-26T18:08:47.364827+0000","last_became_peered":"2024-04-26T18:08:47.364827+0000","last_unstale":"2024-04-26T18:10:31.862510+0000","last_undegraded":"2024-04-26T18:10:31.862510+0000","last_fullsized":"2024-04-26T18:10:31.862510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:44.492380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014614+0000","last_change":"2024-04-26T18:08:44.339063+0000","last_active":"2024-04-26T18:09:33.014614+0000","last_peered":"2024-04-26T18:09:33.014614+0000","last_clean":"2024-04-26T18:09:33.014614+0000","last_became_active":"2024-04-26T18:08:44.338752+0000","last_became_peered":"2024-04-26T18:08:44.338752+0000","last_unstale":"2024-04-26T18:09:33.014614+0000","last_undegraded":"2024-04-26T18:09:33.014614+0000","last_fullsized":"2024-04-26T18:09:33.014614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:38.201263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880808+0000","last_change":"2024-04-26T18:08:47.367859+0000","last_active":"2024-04-26T18:10:31.880808+0000","last_peered":"2024-04-26T18:10:31.880808+0000","last_clean":"2024-04-26T18:10:31.880808+0000","last_became_active":"2024-04-26T18:08:47.367548+0000","last_became_peered":"2024-04-26T18:08:47.367548+0000","last_unstale":"2024-04-26T18:10:31.880808+0000","last_undegraded":"2024-04-26T18:10:31.880808+0000","last_fullsized":"2024-04-26T18:10:31.880808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:13:27.612151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014359+0000","last_change":"2024-04-26T18:08:44.345392+0000","last_active":"2024-04-26T18:09:33.014359+0000","last_peered":"2024-04-26T18:09:33.014359+0000","last_clean":"2024-04-26T18:09:33.014359+0000","last_became_active":"2024-04-26T18:08:44.344578+0000","last_became_peered":"2024-04-26T18:08:44.344578+0000","last_unstale":"2024-04-26T18:09:33.014359+0000","last_undegraded":"2024-04-26T18:09:33.014359+0000","last_fullsized":"2024-04-26T18:09:33.014359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:26.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887184+0000","last_change":"2024-04-26T18:08:47.380099+0000","last_active":"2024-04-26T18:10:31.887184+0000","last_peered":"2024-04-26T18:10:31.887184+0000","last_clean":"2024-04-26T18:10:31.887184+0000","last_became_active":"2024-04-26T18:08:47.379576+0000","last_became_peered":"2024-04-26T18:08:47.379576+0000","last_unstale":"2024-04-26T18:10:31.887184+0000","last_undegraded":"2024-04-26T18:10:31.887184+0000","last_fullsized":"2024-04-26T18:10:31.887184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:28.397886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.015156+0000","last_change":"2024-04-26T18:08:44.555299+0000","last_active":"2024-04-26T18:09:33.015156+0000","last_peered":"2024-04-26T18:09:33.015156+0000","last_clean":"2024-04-26T18:09:33.015156+0000","last_became_active":"2024-04-26T18:08:44.554959+0000","last_became_peered":"2024-04-26T18:08:44.554959+0000","last_unstale":"2024-04-26T18:09:33.015156+0000","last_undegraded":"2024-04-26T18:09:33.015156+0000","last_fullsized":"2024-04-26T18:09:33.015156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.565073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887608+0000","last_change":"2024-04-26T18:08:47.377524+0000","last_active":"2024-04-26T18:10:31.887608+0000","last_peered":"2024-04-26T18:10:31.887608+0000","last_clean":"2024-04-26T18:10:31.887608+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:10:31.887608+0000","last_undegraded":"2024-04-26T18:10:31.887608+0000","last_fullsized":"2024-04-26T18:10:31.887608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:10.466615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.011551+0000","last_change":"2024-04-26T18:08:44.343894+0000","last_active":"2024-04-26T18:09:33.011551+0000","last_peered":"2024-04-26T18:09:33.011551+0000","last_clean":"2024-04-26T18:09:33.011551+0000","last_became_active":"2024-04-26T18:08:44.343153+0000","last_became_peered":"2024-04-26T18:08:44.343153+0000","last_unstale":"2024-04-26T18:09:33.011551+0000","last_undegraded":"2024-04-26T18:09:33.011551+0000","last_fullsized":"2024-04-26T18:09:33.011551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:09.914701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880354+0000","last_change":"2024-04-26T18:08:47.371962+0000","last_active":"2024-04-26T18:10:31.880354+0000","last_peered":"2024-04-26T18:10:31.880354+0000","last_clean":"2024-04-26T18:10:31.880354+0000","last_became_active":"2024-04-26T18:08:47.371517+0000","last_became_peered":"2024-04-26T18:08:47.371517+0000","last_unstale":"2024-04-26T18:10:31.880354+0000","last_undegraded":"2024-04-26T18:10:31.880354+0000","last_fullsized":"2024-04-26T18:10:31.880354+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:23.553409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.523908+0000","last_change":"2024-04-26T18:08:44.337853+0000","last_active":"2024-04-26T18:09:33.523908+0000","last_peered":"2024-04-26T18:09:33.523908+0000","last_clean":"2024-04-26T18:09:33.523908+0000","last_became_active":"2024-04-26T18:08:44.337393+0000","last_became_peered":"2024-04-26T18:08:44.337393+0000","last_unstale":"2024-04-26T18:09:33.523908+0000","last_undegraded":"2024-04-26T18:09:33.523908+0000","last_fullsized":"2024-04-26T18:09:33.523908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:26.514821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887055+0000","last_change":"2024-04-26T18:08:47.370999+0000","last_active":"2024-04-26T18:10:31.887055+0000","last_peered":"2024-04-26T18:10:31.887055+0000","last_clean":"2024-04-26T18:10:31.887055+0000","last_became_active":"2024-04-26T18:08:47.370373+0000","last_became_peered":"2024-04-26T18:08:47.370373+0000","last_unstale":"2024-04-26T18:10:31.887055+0000","last_undegraded":"2024-04-26T18:10:31.887055+0000","last_fullsized":"2024-04-26T18:10:31.887055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:19.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.865070+0000","last_change":"2024-04-26T18:08:47.365722+0000","last_active":"2024-04-26T18:10:31.865070+0000","last_peered":"2024-04-26T18:10:31.865070+0000","last_clean":"2024-04-26T18:10:31.865070+0000","last_became_active":"2024-04-26T18:08:47.364692+0000","last_became_peered":"2024-04-26T18:08:47.364692+0000","last_unstale":"2024-04-26T18:10:31.865070+0000","last_undegraded":"2024-04-26T18:10:31.865070+0000","last_fullsized":"2024-04-26T18:10:31.865070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:02.158156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880953+0000","last_change":"2024-04-26T18:08:47.373886+0000","last_active":"2024-04-26T18:10:31.880953+0000","last_peered":"2024-04-26T18:10:31.880953+0000","last_clean":"2024-04-26T18:10:31.880953+0000","last_became_active":"2024-04-26T18:08:47.373391+0000","last_became_peered":"2024-04-26T18:08:47.373391+0000","last_unstale":"2024-04-26T18:10:31.880953+0000","last_undegraded":"2024-04-26T18:10:31.880953+0000","last_fullsized":"2024-04-26T18:10:31.880953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:48:59.280050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880574+0000","last_change":"2024-04-26T18:08:47.366160+0000","last_active":"2024-04-26T18:10:31.880574+0000","last_peered":"2024-04-26T18:10:31.880574+0000","last_clean":"2024-04-26T18:10:31.880574+0000","last_became_active":"2024-04-26T18:08:47.365811+0000","last_became_peered":"2024-04-26T18:08:47.365811+0000","last_unstale":"2024-04-26T18:10:31.880574+0000","last_undegraded":"2024-04-26T18:10:31.880574+0000","last_fullsized":"2024-04-26T18:10:31.880574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:32.124692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.871898+0000","last_change":"2024-04-26T18:08:47.366078+0000","last_active":"2024-04-26T18:10:31.871898+0000","last_peered":"2024-04-26T18:10:31.871898+0000","last_clean":"2024-04-26T18:10:31.871898+0000","last_became_active":"2024-04-26T18:08:47.365779+0000","last_became_peered":"2024-04-26T18:08:47.365779+0000","last_unstale":"2024-04-26T18:10:31.871898+0000","last_undegraded":"2024-04-26T18:10:31.871898+0000","last_fullsized":"2024-04-26T18:10:31.871898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.127606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.871262+0000","last_change":"2024-04-26T18:08:47.372996+0000","last_active":"2024-04-26T18:10:31.871262+0000","last_peered":"2024-04-26T18:10:31.871262+0000","last_clean":"2024-04-26T18:10:31.871262+0000","last_became_active":"2024-04-26T18:08:47.372687+0000","last_became_peered":"2024-04-26T18:08:47.372687+0000","last_unstale":"2024-04-26T18:10:31.871262+0000","last_undegraded":"2024-04-26T18:10:31.871262+0000","last_fullsized":"2024-04-26T18:10:31.871262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:13:33.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.859831+0000","last_change":"2024-04-26T18:08:47.371076+0000","last_active":"2024-04-26T18:10:31.859831+0000","last_peered":"2024-04-26T18:10:31.859831+0000","last_clean":"2024-04-26T18:10:31.859831+0000","last_became_active":"2024-04-26T18:08:47.370688+0000","last_became_peered":"2024-04-26T18:08:47.370688+0000","last_unstale":"2024-04-26T18:10:31.859831+0000","last_undegraded":"2024-04-26T18:10:31.859831+0000","last_fullsized":"2024-04-26T18:10:31.859831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:21.355971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.884277+0000","last_change":"2024-04-26T18:08:47.372392+0000","last_active":"2024-04-26T18:10:31.884277+0000","last_peered":"2024-04-26T18:10:31.884277+0000","last_clean":"2024-04-26T18:10:31.884277+0000","last_became_active":"2024-04-26T18:08:47.372020+0000","last_became_peered":"2024-04-26T18:08:47.372020+0000","last_unstale":"2024-04-26T18:10:31.884277+0000","last_undegraded":"2024-04-26T18:10:31.884277+0000","last_fullsized":"2024-04-26T18:10:31.884277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878227+0000","last_change":"2024-04-26T18:08:47.369571+0000","last_active":"2024-04-26T18:10:31.878227+0000","last_peered":"2024-04-26T18:10:31.878227+0000","last_clean":"2024-04-26T18:10:31.878227+0000","last_became_active":"2024-04-26T18:08:47.369165+0000","last_became_peered":"2024-04-26T18:08:47.369165+0000","last_unstale":"2024-04-26T18:10:31.878227+0000","last_undegraded":"2024-04-26T18:10:31.878227+0000","last_fullsized":"2024-04-26T18:10:31.878227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:15.869182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.877581+0000","last_change":"2024-04-26T18:08:47.372355+0000","last_active":"2024-04-26T18:10:31.877581+0000","last_peered":"2024-04-26T18:10:31.877581+0000","last_clean":"2024-04-26T18:10:31.877581+0000","last_became_active":"2024-04-26T18:08:47.371810+0000","last_became_peered":"2024-04-26T18:08:47.371810+0000","last_unstale":"2024-04-26T18:10:31.877581+0000","last_undegraded":"2024-04-26T18:10:31.877581+0000","last_fullsized":"2024-04-26T18:10:31.877581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:44:22.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876807+0000","last_change":"2024-04-26T18:08:47.364042+0000","last_active":"2024-04-26T18:10:31.876807+0000","last_peered":"2024-04-26T18:10:31.876807+0000","last_clean":"2024-04-26T18:10:31.876807+0000","last_became_active":"2024-04-26T18:08:47.363727+0000","last_became_peered":"2024-04-26T18:08:47.363727+0000","last_unstale":"2024-04-26T18:10:31.876807+0000","last_undegraded":"2024-04-26T18:10:31.876807+0000","last_fullsized":"2024-04-26T18:10:31.876807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:37.595261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.888285+0000","last_change":"2024-04-26T18:08:47.377535+0000","last_active":"2024-04-26T18:10:31.888285+0000","last_peered":"2024-04-26T18:10:31.888285+0000","last_clean":"2024-04-26T18:10:31.888285+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:10:31.888285+0000","last_undegraded":"2024-04-26T18:10:31.888285+0000","last_fullsized":"2024-04-26T18:10:31.888285+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:22.895402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880770+0000","last_change":"2024-04-26T18:08:47.367313+0000","last_active":"2024-04-26T18:10:31.880770+0000","last_peered":"2024-04-26T18:10:31.880770+0000","last_clean":"2024-04-26T18:10:31.880770+0000","last_became_active":"2024-04-26T18:08:47.366971+0000","last_became_peered":"2024-04-26T18:08:47.366971+0000","last_unstale":"2024-04-26T18:10:31.880770+0000","last_undegraded":"2024-04-26T18:10:31.880770+0000","last_fullsized":"2024-04-26T18:10:31.880770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:14.518100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.860309+0000","last_change":"2024-04-26T18:08:47.373004+0000","last_active":"2024-04-26T18:10:31.860309+0000","last_peered":"2024-04-26T18:10:31.860309+0000","last_clean":"2024-04-26T18:10:31.860309+0000","last_became_active":"2024-04-26T18:08:47.372641+0000","last_became_peered":"2024-04-26T18:08:47.372641+0000","last_unstale":"2024-04-26T18:10:31.860309+0000","last_undegraded":"2024-04-26T18:10:31.860309+0000","last_fullsized":"2024-04-26T18:10:31.860309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:02.018425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64685,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":638,"num_read_kb":389,"num_write":175,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":489607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1699840,"data_stored":1141575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":489787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":489685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266748,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1613824,"data_stored":1082310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":489176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":548980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":490447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":550828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182480,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":489487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920843,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":490145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724614,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1626112,"data_stored":1081462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:10:48.093 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:10:48.336 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:49.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:48 smithi090 ceph-mon[29180]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:10:49.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:48 smithi090 ceph-mon[29180]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:48 smithi119 ceph-mon[32752]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:10:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:48 smithi119 ceph-mon[32752]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:48 smithi167 ceph-mon[32751]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:10:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:48 smithi167 ceph-mon[32751]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:49.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 -- 172.21.15.90:0/1154565190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650108390 msgr2=0x7fd650108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:49.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 --2- 172.21.15.90:0/1154565190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650108390 0x7fd650108770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd644009990 tx=0x7fd644009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:49.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 -- 172.21.15.90:0/1154565190 shutdown_connections 2024-04-26T18:10:49.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 --2- 172.21.15.90:0/1154565190 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650108390 0x7fd650108770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:49.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 --2- 172.21.15.90:0/1154565190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd650102ce0 0x7fd65010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:49.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 --2- 172.21.15.90:0/1154565190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd650102320 0x7fd6501027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:49.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 -- 172.21.15.90:0/1154565190 >> 172.21.15.90:0/1154565190 conn(0x7fd6500792d0 msgr2=0x7fd6500796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:49.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 -- 172.21.15.90:0/1154565190 shutdown_connections 2024-04-26T18:10:49.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.956+0000 7fd65580b700 1 -- 172.21.15.90:0/1154565190 wait complete. 2024-04-26T18:10:49.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 Processor -- start 2024-04-26T18:10:49.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 -- start start 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd650102320 0x7fd65019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650102ce0 0x7fd65019c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd650108390 0x7fd6501a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd650112500 con 0x7fd650102ce0 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd650112380 con 0x7fd650108390 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd65580b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd650112680 con 0x7fd650102320 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.957+0000 7fd64e7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650102ce0 0x7fd65019c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650102ce0 0x7fd65019c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57514/0 (socket says 172.21.15.90:57514) 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 -- 172.21.15.90:0/4207319055 learned_addr learned my addr 172.21.15.90:0/4207319055 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64effd700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd650102320 0x7fd65019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64f7fe700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd650108390 0x7fd6501a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 -- 172.21.15.90:0/4207319055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd650102320 msgr2=0x7fd65019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd650102320 0x7fd65019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 -- 172.21.15.90:0/4207319055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd650108390 msgr2=0x7fd6501a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd650108390 0x7fd6501a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 -- 172.21.15.90:0/4207319055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6501a0d50 con 0x7fd650102ce0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd64e7fc700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650102ce0 0x7fd65019c280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd64000d980 tx=0x7fd64000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd654809700 1 -- 172.21.15.90:0/4207319055 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640016070 con 0x7fd650102ce0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd654809700 1 -- 172.21.15.90:0/4207319055 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6400044d0 con 0x7fd650102ce0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6501a1040 con 0x7fd650102ce0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd654809700 1 -- 172.21.15.90:0/4207319055 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd640004d00 con 0x7fd650102ce0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.958+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6501a88c0 con 0x7fd650102ce0 2024-04-26T18:10:49.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.959+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65006a110 con 0x7fd650102ce0 2024-04-26T18:10:49.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.961+0000 7fd654809700 1 -- 172.21.15.90:0/4207319055 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd640004020 con 0x7fd650102ce0 2024-04-26T18:10:49.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.962+0000 7fd654809700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd6380a3c40 0x7fd6380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:49.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.962+0000 7fd64effd700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd6380a3c40 0x7fd6380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:49.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.962+0000 7fd654809700 1 -- 172.21.15.90:0/4207319055 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fd6400d0790 con 0x7fd650102ce0 2024-04-26T18:10:49.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.962+0000 7fd64effd700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd6380a3c40 0x7fd6380a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd64400b5c0 tx=0x7fd64401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:49.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:49.965+0000 7fd654809700 1 -- 172.21.15.90:0/4207319055 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd64001f020 con 0x7fd650102ce0 2024-04-26T18:10:50.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.158+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd65006b810 con 0x7fd6380a3c40 2024-04-26T18:10:50.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.165+0000 7fd654809700 1 -- 172.21.15.90:0/4207319055 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258825 (secure 0 0 0) 0x7fd65006b810 con 0x7fd6380a3c40 2024-04-26T18:10:50.172 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd6380a3c40 msgr2=0x7fd6380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd6380a3c40 0x7fd6380a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd64400b5c0 tx=0x7fd64401a040 comp rx=0 tx=0).stop 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650102ce0 msgr2=0x7fd65019c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650102ce0 0x7fd65019c280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd64000d980 tx=0x7fd64000dd40 comp rx=0 tx=0).stop 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 shutdown_connections 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fd6380a3c40 0x7fd6380a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd650108390 0x7fd6501a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd650102320 0x7fd65019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 --2- 172.21.15.90:0/4207319055 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd650102ce0 0x7fd65019c280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 >> 172.21.15.90:0/4207319055 conn(0x7fd6500792d0 msgr2=0x7fd65006b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 shutdown_connections 2024-04-26T18:10:50.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:50.168+0000 7fd65580b700 1 -- 172.21.15.90:0/4207319055 wait complete. 2024-04-26T18:10:50.176 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T18:10:50.856 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-04-26T18:10:49.900035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655287,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":784,"num_read_kb":508,"num_write":358,"num_write_kb":3058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":788,"ondisk_log_size":788,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379676,"kb_used_data":13884,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562148,"statfs":{"total":1151940427776,"available":1151551639552,"internally_reserved":0,"allocated":14217216,"data_stored":7833489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003066"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876561+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:10:31.876561+0000","last_peered":"2024-04-26T18:10:31.876561+0000","last_clean":"2024-04-26T18:10:31.876561+0000","last_became_active":"2024-04-26T18:08:47.377408+0000","last_became_peered":"2024-04-26T18:08:47.377408+0000","last_unstale":"2024-04-26T18:10:31.876561+0000","last_undegraded":"2024-04-26T18:10:31.876561+0000","last_fullsized":"2024-04-26T18:10:31.876561+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:13.822258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.866886+0000","last_change":"2024-04-26T18:08:47.365125+0000","last_active":"2024-04-26T18:10:31.866886+0000","last_peered":"2024-04-26T18:10:31.866886+0000","last_clean":"2024-04-26T18:10:31.866886+0000","last_became_active":"2024-04-26T18:08:47.364847+0000","last_became_peered":"2024-04-26T18:08:47.364847+0000","last_unstale":"2024-04-26T18:10:31.866886+0000","last_undegraded":"2024-04-26T18:10:31.866886+0000","last_fullsized":"2024-04-26T18:10:31.866886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:16:37.890893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879920+0000","last_change":"2024-04-26T18:08:47.377580+0000","last_active":"2024-04-26T18:10:31.879920+0000","last_peered":"2024-04-26T18:10:31.879920+0000","last_clean":"2024-04-26T18:10:31.879920+0000","last_became_active":"2024-04-26T18:08:47.377219+0000","last_became_peered":"2024-04-26T18:08:47.377219+0000","last_unstale":"2024-04-26T18:10:31.879920+0000","last_undegraded":"2024-04-26T18:10:31.879920+0000","last_fullsized":"2024-04-26T18:10:31.879920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:12.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878127+0000","last_change":"2024-04-26T18:08:47.371628+0000","last_active":"2024-04-26T18:10:31.878127+0000","last_peered":"2024-04-26T18:10:31.878127+0000","last_clean":"2024-04-26T18:10:31.878127+0000","last_became_active":"2024-04-26T18:08:47.370840+0000","last_became_peered":"2024-04-26T18:08:47.370840+0000","last_unstale":"2024-04-26T18:10:31.878127+0000","last_undegraded":"2024-04-26T18:10:31.878127+0000","last_fullsized":"2024-04-26T18:10:31.878127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:11.362664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867410+0000","last_change":"2024-04-26T18:08:47.367632+0000","last_active":"2024-04-26T18:10:31.867410+0000","last_peered":"2024-04-26T18:10:31.867410+0000","last_clean":"2024-04-26T18:10:31.867410+0000","last_became_active":"2024-04-26T18:08:47.367316+0000","last_became_peered":"2024-04-26T18:08:47.367316+0000","last_unstale":"2024-04-26T18:10:31.867410+0000","last_undegraded":"2024-04-26T18:10:31.867410+0000","last_fullsized":"2024-04-26T18:10:31.867410+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:31:26.665292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.865713+0000","last_change":"2024-04-26T18:08:47.369742+0000","last_active":"2024-04-26T18:10:31.865713+0000","last_peered":"2024-04-26T18:10:31.865713+0000","last_clean":"2024-04-26T18:10:31.865713+0000","last_became_active":"2024-04-26T18:08:47.369230+0000","last_became_peered":"2024-04-26T18:08:47.369230+0000","last_unstale":"2024-04-26T18:10:31.865713+0000","last_undegraded":"2024-04-26T18:10:31.865713+0000","last_fullsized":"2024-04-26T18:10:31.865713+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:02:04.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879644+0000","last_change":"2024-04-26T18:08:47.370657+0000","last_active":"2024-04-26T18:10:31.879644+0000","last_peered":"2024-04-26T18:10:31.879644+0000","last_clean":"2024-04-26T18:10:31.879644+0000","last_became_active":"2024-04-26T18:08:47.370133+0000","last_became_peered":"2024-04-26T18:08:47.370133+0000","last_unstale":"2024-04-26T18:10:31.879644+0000","last_undegraded":"2024-04-26T18:10:31.879644+0000","last_fullsized":"2024-04-26T18:10:31.879644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:31.232179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879611+0000","last_change":"2024-04-26T18:08:47.366221+0000","last_active":"2024-04-26T18:10:31.879611+0000","last_peered":"2024-04-26T18:10:31.879611+0000","last_clean":"2024-04-26T18:10:31.879611+0000","last_became_active":"2024-04-26T18:08:47.365990+0000","last_became_peered":"2024-04-26T18:08:47.365990+0000","last_unstale":"2024-04-26T18:10:31.879611+0000","last_undegraded":"2024-04-26T18:10:31.879611+0000","last_fullsized":"2024-04-26T18:10:31.879611+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:43.707583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.872635+0000","last_change":"2024-04-26T18:08:47.373165+0000","last_active":"2024-04-26T18:10:31.872635+0000","last_peered":"2024-04-26T18:10:31.872635+0000","last_clean":"2024-04-26T18:10:31.872635+0000","last_became_active":"2024-04-26T18:08:47.372688+0000","last_became_peered":"2024-04-26T18:08:47.372688+0000","last_unstale":"2024-04-26T18:10:31.872635+0000","last_undegraded":"2024-04-26T18:10:31.872635+0000","last_fullsized":"2024-04-26T18:10:31.872635+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:18.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.866081+0000","last_change":"2024-04-26T18:09:33.016811+0000","last_active":"2024-04-26T18:10:31.866081+0000","last_peered":"2024-04-26T18:10:31.866081+0000","last_clean":"2024-04-26T18:10:31.866081+0000","last_became_active":"2024-04-26T18:09:33.016267+0000","last_became_peered":"2024-04-26T18:09:33.016267+0000","last_unstale":"2024-04-26T18:10:31.866081+0000","last_undegraded":"2024-04-26T18:10:31.866081+0000","last_fullsized":"2024-04-26T18:10:31.866081+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.002914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887360+0000","last_change":"2024-04-26T18:08:47.378643+0000","last_active":"2024-04-26T18:10:31.887360+0000","last_peered":"2024-04-26T18:10:31.887360+0000","last_clean":"2024-04-26T18:10:31.887360+0000","last_became_active":"2024-04-26T18:08:47.378329+0000","last_became_peered":"2024-04-26T18:08:47.378329+0000","last_unstale":"2024-04-26T18:10:31.887360+0000","last_undegraded":"2024-04-26T18:10:31.887360+0000","last_fullsized":"2024-04-26T18:10:31.887360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:59.016025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.859036+0000","last_change":"2024-04-26T18:08:47.371663+0000","last_active":"2024-04-26T18:10:31.859036+0000","last_peered":"2024-04-26T18:10:31.859036+0000","last_clean":"2024-04-26T18:10:31.859036+0000","last_became_active":"2024-04-26T18:08:47.371298+0000","last_became_peered":"2024-04-26T18:08:47.371298+0000","last_unstale":"2024-04-26T18:10:31.859036+0000","last_undegraded":"2024-04-26T18:10:31.859036+0000","last_fullsized":"2024-04-26T18:10:31.859036+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:04.884698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.875198+0000","last_change":"2024-04-26T18:08:47.365179+0000","last_active":"2024-04-26T18:10:31.875198+0000","last_peered":"2024-04-26T18:10:31.875198+0000","last_clean":"2024-04-26T18:10:31.875198+0000","last_became_active":"2024-04-26T18:08:47.364888+0000","last_became_peered":"2024-04-26T18:08:47.364888+0000","last_unstale":"2024-04-26T18:10:31.875198+0000","last_undegraded":"2024-04-26T18:10:31.875198+0000","last_fullsized":"2024-04-26T18:10:31.875198+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:34.082855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.864278+0000","last_change":"2024-04-26T18:08:47.364828+0000","last_active":"2024-04-26T18:10:31.864278+0000","last_peered":"2024-04-26T18:10:31.864278+0000","last_clean":"2024-04-26T18:10:31.864278+0000","last_became_active":"2024-04-26T18:08:47.364529+0000","last_became_peered":"2024-04-26T18:08:47.364529+0000","last_unstale":"2024-04-26T18:10:31.864278+0000","last_undegraded":"2024-04-26T18:10:31.864278+0000","last_fullsized":"2024-04-26T18:10:31.864278+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.480258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880160+0000","last_change":"2024-04-26T18:08:47.373578+0000","last_active":"2024-04-26T18:10:31.880160+0000","last_peered":"2024-04-26T18:10:31.880160+0000","last_clean":"2024-04-26T18:10:31.880160+0000","last_became_active":"2024-04-26T18:08:47.373263+0000","last_became_peered":"2024-04-26T18:08:47.373263+0000","last_unstale":"2024-04-26T18:10:31.880160+0000","last_undegraded":"2024-04-26T18:10:31.880160+0000","last_fullsized":"2024-04-26T18:10:31.880160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:25.340241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.886796+0000","last_change":"2024-04-26T18:08:47.376617+0000","last_active":"2024-04-26T18:10:31.886796+0000","last_peered":"2024-04-26T18:10:31.886796+0000","last_clean":"2024-04-26T18:10:31.886796+0000","last_became_active":"2024-04-26T18:08:47.376290+0000","last_became_peered":"2024-04-26T18:08:47.376290+0000","last_unstale":"2024-04-26T18:10:31.886796+0000","last_undegraded":"2024-04-26T18:10:31.886796+0000","last_fullsized":"2024-04-26T18:10:31.886796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:51.187583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876709+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:10:31.876709+0000","last_peered":"2024-04-26T18:10:31.876709+0000","last_clean":"2024-04-26T18:10:31.876709+0000","last_became_active":"2024-04-26T18:08:47.377427+0000","last_became_peered":"2024-04-26T18:08:47.377427+0000","last_unstale":"2024-04-26T18:10:31.876709+0000","last_undegraded":"2024-04-26T18:10:31.876709+0000","last_fullsized":"2024-04-26T18:10:31.876709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.529500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.868655+0000","last_change":"2024-04-26T18:08:47.373314+0000","last_active":"2024-04-26T18:10:31.868655+0000","last_peered":"2024-04-26T18:10:31.868655+0000","last_clean":"2024-04-26T18:10:31.868655+0000","last_became_active":"2024-04-26T18:08:47.372868+0000","last_became_peered":"2024-04-26T18:08:47.372868+0000","last_unstale":"2024-04-26T18:10:31.868655+0000","last_undegraded":"2024-04-26T18:10:31.868655+0000","last_fullsized":"2024-04-26T18:10:31.868655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:19.642655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880658+0000","last_change":"2024-04-26T18:08:47.367593+0000","last_active":"2024-04-26T18:10:31.880658+0000","last_peered":"2024-04-26T18:10:31.880658+0000","last_clean":"2024-04-26T18:10:31.880658+0000","last_became_active":"2024-04-26T18:08:47.367243+0000","last_became_peered":"2024-04-26T18:08:47.367243+0000","last_unstale":"2024-04-26T18:10:31.880658+0000","last_undegraded":"2024-04-26T18:10:31.880658+0000","last_fullsized":"2024-04-26T18:10:31.880658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.380307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.009294+0000","last_change":"2024-04-26T18:08:44.339625+0000","last_active":"2024-04-26T18:09:33.009294+0000","last_peered":"2024-04-26T18:09:33.009294+0000","last_clean":"2024-04-26T18:09:33.009294+0000","last_became_active":"2024-04-26T18:08:44.339209+0000","last_became_peered":"2024-04-26T18:08:44.339209+0000","last_unstale":"2024-04-26T18:09:33.009294+0000","last_undegraded":"2024-04-26T18:09:33.009294+0000","last_fullsized":"2024-04-26T18:09:33.009294+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:43.169885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.877654+0000","last_change":"2024-04-26T18:08:47.375884+0000","last_active":"2024-04-26T18:10:31.877654+0000","last_peered":"2024-04-26T18:10:31.877654+0000","last_clean":"2024-04-26T18:10:31.877654+0000","last_became_active":"2024-04-26T18:08:47.375557+0000","last_became_peered":"2024-04-26T18:08:47.375557+0000","last_unstale":"2024-04-26T18:10:31.877654+0000","last_undegraded":"2024-04-26T18:10:31.877654+0000","last_fullsized":"2024-04-26T18:10:31.877654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.593910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014756+0000","last_change":"2024-04-26T18:08:44.557167+0000","last_active":"2024-04-26T18:09:33.014756+0000","last_peered":"2024-04-26T18:09:33.014756+0000","last_clean":"2024-04-26T18:09:33.014756+0000","last_became_active":"2024-04-26T18:08:44.556716+0000","last_became_peered":"2024-04-26T18:08:44.556716+0000","last_unstale":"2024-04-26T18:09:33.014756+0000","last_undegraded":"2024-04-26T18:09:33.014756+0000","last_fullsized":"2024-04-26T18:09:33.014756+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.881072+0000","last_change":"2024-04-26T18:08:47.377612+0000","last_active":"2024-04-26T18:10:31.881072+0000","last_peered":"2024-04-26T18:10:31.881072+0000","last_clean":"2024-04-26T18:10:31.881072+0000","last_became_active":"2024-04-26T18:08:47.377231+0000","last_became_peered":"2024-04-26T18:08:47.377231+0000","last_unstale":"2024-04-26T18:10:31.881072+0000","last_undegraded":"2024-04-26T18:10:31.881072+0000","last_fullsized":"2024-04-26T18:10:31.881072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:11.594373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.015269+0000","last_change":"2024-04-26T18:08:44.343429+0000","last_active":"2024-04-26T18:09:33.015269+0000","last_peered":"2024-04-26T18:09:33.015269+0000","last_clean":"2024-04-26T18:09:33.015269+0000","last_became_active":"2024-04-26T18:08:44.343103+0000","last_became_peered":"2024-04-26T18:08:44.343103+0000","last_unstale":"2024-04-26T18:09:33.015269+0000","last_undegraded":"2024-04-26T18:09:33.015269+0000","last_fullsized":"2024-04-26T18:09:33.015269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:52.244159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882838+0000","last_change":"2024-04-26T18:08:47.373130+0000","last_active":"2024-04-26T18:10:31.882838+0000","last_peered":"2024-04-26T18:10:31.882838+0000","last_clean":"2024-04-26T18:10:31.882838+0000","last_became_active":"2024-04-26T18:08:47.372726+0000","last_became_peered":"2024-04-26T18:08:47.372726+0000","last_unstale":"2024-04-26T18:10:31.882838+0000","last_undegraded":"2024-04-26T18:10:31.882838+0000","last_fullsized":"2024-04-26T18:10:31.882838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:44.461177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014700+0000","last_change":"2024-04-26T18:08:44.340152+0000","last_active":"2024-04-26T18:09:33.014700+0000","last_peered":"2024-04-26T18:09:33.014700+0000","last_clean":"2024-04-26T18:09:33.014700+0000","last_became_active":"2024-04-26T18:08:44.339684+0000","last_became_peered":"2024-04-26T18:08:44.339684+0000","last_unstale":"2024-04-26T18:09:33.014700+0000","last_undegraded":"2024-04-26T18:09:33.014700+0000","last_fullsized":"2024-04-26T18:09:33.014700+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:49.961201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880231+0000","last_change":"2024-04-26T18:08:47.375387+0000","last_active":"2024-04-26T18:10:31.880231+0000","last_peered":"2024-04-26T18:10:31.880231+0000","last_clean":"2024-04-26T18:10:31.880231+0000","last_became_active":"2024-04-26T18:08:47.374689+0000","last_became_peered":"2024-04-26T18:08:47.374689+0000","last_unstale":"2024-04-26T18:10:31.880231+0000","last_undegraded":"2024-04-26T18:10:31.880231+0000","last_fullsized":"2024-04-26T18:10:31.880231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:05.618804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'7","reported_seq":142,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:47.102475+0000","last_change":"2024-04-26T18:08:44.344167+0000","last_active":"2024-04-26T18:10:47.102475+0000","last_peered":"2024-04-26T18:10:47.102475+0000","last_clean":"2024-04-26T18:10:47.102475+0000","last_became_active":"2024-04-26T18:08:44.343646+0000","last_became_peered":"2024-04-26T18:08:44.343646+0000","last_unstale":"2024-04-26T18:10:47.102475+0000","last_undegraded":"2024-04-26T18:10:47.102475+0000","last_fullsized":"2024-04-26T18:10:47.102475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:53.859483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867790+0000","last_change":"2024-04-26T18:08:47.370546+0000","last_active":"2024-04-26T18:10:31.867790+0000","last_peered":"2024-04-26T18:10:31.867790+0000","last_clean":"2024-04-26T18:10:31.867790+0000","last_became_active":"2024-04-26T18:08:47.370220+0000","last_became_peered":"2024-04-26T18:08:47.370220+0000","last_unstale":"2024-04-26T18:10:31.867790+0000","last_undegraded":"2024-04-26T18:10:31.867790+0000","last_fullsized":"2024-04-26T18:10:31.867790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:05.139824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.010056+0000","last_change":"2024-04-26T18:08:44.347031+0000","last_active":"2024-04-26T18:09:33.010056+0000","last_peered":"2024-04-26T18:09:33.010056+0000","last_clean":"2024-04-26T18:09:33.010056+0000","last_became_active":"2024-04-26T18:08:44.346650+0000","last_became_peered":"2024-04-26T18:08:44.346650+0000","last_unstale":"2024-04-26T18:09:33.010056+0000","last_undegraded":"2024-04-26T18:09:33.010056+0000","last_fullsized":"2024-04-26T18:09:33.010056+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:50.946449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879498+0000","last_change":"2024-04-26T18:08:47.374230+0000","last_active":"2024-04-26T18:10:31.879498+0000","last_peered":"2024-04-26T18:10:31.879498+0000","last_clean":"2024-04-26T18:10:31.879498+0000","last_became_active":"2024-04-26T18:08:47.370780+0000","last_became_peered":"2024-04-26T18:08:47.370780+0000","last_unstale":"2024-04-26T18:10:31.879498+0000","last_undegraded":"2024-04-26T18:10:31.879498+0000","last_fullsized":"2024-04-26T18:10:31.879498+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:01.171195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.523771+0000","last_change":"2024-04-26T18:08:44.338318+0000","last_active":"2024-04-26T18:09:33.523771+0000","last_peered":"2024-04-26T18:09:33.523771+0000","last_clean":"2024-04-26T18:09:33.523771+0000","last_became_active":"2024-04-26T18:08:44.338009+0000","last_became_peered":"2024-04-26T18:08:44.338009+0000","last_unstale":"2024-04-26T18:09:33.523771+0000","last_undegraded":"2024-04-26T18:09:33.523771+0000","last_fullsized":"2024-04-26T18:09:33.523771+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:33.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.872840+0000","last_change":"2024-04-26T18:08:47.363924+0000","last_active":"2024-04-26T18:10:31.872840+0000","last_peered":"2024-04-26T18:10:31.872840+0000","last_clean":"2024-04-26T18:10:31.872840+0000","last_became_active":"2024-04-26T18:08:47.363616+0000","last_became_peered":"2024-04-26T18:08:47.363616+0000","last_unstale":"2024-04-26T18:10:31.872840+0000","last_undegraded":"2024-04-26T18:10:31.872840+0000","last_fullsized":"2024-04-26T18:10:31.872840+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:27:08.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.010902+0000","last_change":"2024-04-26T18:08:44.343350+0000","last_active":"2024-04-26T18:09:33.010902+0000","last_peered":"2024-04-26T18:09:33.010902+0000","last_clean":"2024-04-26T18:09:33.010902+0000","last_became_active":"2024-04-26T18:08:44.343040+0000","last_became_peered":"2024-04-26T18:08:44.343040+0000","last_unstale":"2024-04-26T18:09:33.010902+0000","last_undegraded":"2024-04-26T18:09:33.010902+0000","last_fullsized":"2024-04-26T18:09:33.010902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:09.427785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.869466+0000","last_change":"2024-04-26T18:08:47.360180+0000","last_active":"2024-04-26T18:10:31.869466+0000","last_peered":"2024-04-26T18:10:31.869466+0000","last_clean":"2024-04-26T18:10:31.869466+0000","last_became_active":"2024-04-26T18:08:47.359860+0000","last_became_peered":"2024-04-26T18:08:47.359860+0000","last_unstale":"2024-04-26T18:10:31.869466+0000","last_undegraded":"2024-04-26T18:10:31.869466+0000","last_fullsized":"2024-04-26T18:10:31.869466+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:00.532589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'4","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:44.097224+0000","last_change":"2024-04-26T18:08:44.343942+0000","last_active":"2024-04-26T18:10:44.097224+0000","last_peered":"2024-04-26T18:10:44.097224+0000","last_clean":"2024-04-26T18:10:44.097224+0000","last_became_active":"2024-04-26T18:08:44.343581+0000","last_became_peered":"2024-04-26T18:08:44.343581+0000","last_unstale":"2024-04-26T18:10:44.097224+0000","last_undegraded":"2024-04-26T18:10:44.097224+0000","last_fullsized":"2024-04-26T18:10:44.097224+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:32.044284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882269+0000","last_change":"2024-04-26T18:08:47.372303+0000","last_active":"2024-04-26T18:10:31.882269+0000","last_peered":"2024-04-26T18:10:31.882269+0000","last_clean":"2024-04-26T18:10:31.882269+0000","last_became_active":"2024-04-26T18:08:47.371961+0000","last_became_peered":"2024-04-26T18:08:47.371961+0000","last_unstale":"2024-04-26T18:10:31.882269+0000","last_undegraded":"2024-04-26T18:10:31.882269+0000","last_fullsized":"2024-04-26T18:10:31.882269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:19.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012895+0000","last_change":"2024-04-26T18:08:44.345681+0000","last_active":"2024-04-26T18:09:33.012895+0000","last_peered":"2024-04-26T18:09:33.012895+0000","last_clean":"2024-04-26T18:09:33.012895+0000","last_became_active":"2024-04-26T18:08:44.345329+0000","last_became_peered":"2024-04-26T18:08:44.345329+0000","last_unstale":"2024-04-26T18:09:33.012895+0000","last_undegraded":"2024-04-26T18:09:33.012895+0000","last_fullsized":"2024-04-26T18:09:33.012895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:59.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878799+0000","last_change":"2024-04-26T18:08:47.370773+0000","last_active":"2024-04-26T18:10:31.878799+0000","last_peered":"2024-04-26T18:10:31.878799+0000","last_clean":"2024-04-26T18:10:31.878799+0000","last_became_active":"2024-04-26T18:08:47.370490+0000","last_became_peered":"2024-04-26T18:08:47.370490+0000","last_unstale":"2024-04-26T18:10:31.878799+0000","last_undegraded":"2024-04-26T18:10:31.878799+0000","last_fullsized":"2024-04-26T18:10:31.878799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:01.935335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.011664+0000","last_change":"2024-04-26T18:08:44.555714+0000","last_active":"2024-04-26T18:09:33.011664+0000","last_peered":"2024-04-26T18:09:33.011664+0000","last_clean":"2024-04-26T18:09:33.011664+0000","last_became_active":"2024-04-26T18:08:44.555184+0000","last_became_peered":"2024-04-26T18:08:44.555184+0000","last_unstale":"2024-04-26T18:09:33.011664+0000","last_undegraded":"2024-04-26T18:09:33.011664+0000","last_fullsized":"2024-04-26T18:09:33.011664+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:01.433764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882986+0000","last_change":"2024-04-26T18:09:33.525026+0000","last_active":"2024-04-26T18:10:31.882986+0000","last_peered":"2024-04-26T18:10:31.882986+0000","last_clean":"2024-04-26T18:10:31.882986+0000","last_became_active":"2024-04-26T18:09:33.524587+0000","last_became_peered":"2024-04-26T18:09:33.524587+0000","last_unstale":"2024-04-26T18:10:31.882986+0000","last_undegraded":"2024-04-26T18:10:31.882986+0000","last_fullsized":"2024-04-26T18:10:31.882986+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:39:48.534737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'32","reported_seq":272,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:45.104934+0000","last_change":"2024-04-26T18:08:44.346346+0000","last_active":"2024-04-26T18:10:45.104934+0000","last_peered":"2024-04-26T18:10:45.104934+0000","last_clean":"2024-04-26T18:10:45.104934+0000","last_became_active":"2024-04-26T18:08:44.345999+0000","last_became_peered":"2024-04-26T18:08:44.345999+0000","last_unstale":"2024-04-26T18:10:45.104934+0000","last_undegraded":"2024-04-26T18:10:45.104934+0000","last_fullsized":"2024-04-26T18:10:45.104934+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:27.277880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60011,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":214,"num_read_kb":165,"num_write":33,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.860229+0000","last_change":"2024-04-26T18:08:47.366303+0000","last_active":"2024-04-26T18:10:31.860229+0000","last_peered":"2024-04-26T18:10:31.860229+0000","last_clean":"2024-04-26T18:10:31.860229+0000","last_became_active":"2024-04-26T18:08:47.365861+0000","last_became_peered":"2024-04-26T18:08:47.365861+0000","last_unstale":"2024-04-26T18:10:31.860229+0000","last_undegraded":"2024-04-26T18:10:31.860229+0000","last_fullsized":"2024-04-26T18:10:31.860229+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:31.413589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.022912+0000","last_change":"2024-04-26T18:09:33.022912+0000","last_active":"2024-04-26T18:09:33.022912+0000","last_peered":"2024-04-26T18:09:33.022912+0000","last_clean":"2024-04-26T18:09:33.022912+0000","last_became_active":"2024-04-26T18:09:33.015390+0000","last_became_peered":"2024-04-26T18:09:33.015390+0000","last_unstale":"2024-04-26T18:09:33.022912+0000","last_undegraded":"2024-04-26T18:09:33.022912+0000","last_fullsized":"2024-04-26T18:09:33.022912+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:53.980531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.874360+0000","last_change":"2024-04-26T18:08:47.367835+0000","last_active":"2024-04-26T18:10:31.874360+0000","last_peered":"2024-04-26T18:10:31.874360+0000","last_clean":"2024-04-26T18:10:31.874360+0000","last_became_active":"2024-04-26T18:08:47.367517+0000","last_became_peered":"2024-04-26T18:08:47.367517+0000","last_unstale":"2024-04-26T18:10:31.874360+0000","last_undegraded":"2024-04-26T18:10:31.874360+0000","last_fullsized":"2024-04-26T18:10:31.874360+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:54.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.011273+0000","last_change":"2024-04-26T18:08:44.344161+0000","last_active":"2024-04-26T18:09:33.011273+0000","last_peered":"2024-04-26T18:09:33.011273+0000","last_clean":"2024-04-26T18:09:33.011273+0000","last_became_active":"2024-04-26T18:08:44.343635+0000","last_became_peered":"2024-04-26T18:08:44.343635+0000","last_unstale":"2024-04-26T18:09:33.011273+0000","last_undegraded":"2024-04-26T18:09:33.011273+0000","last_fullsized":"2024-04-26T18:09:33.011273+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.946584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.881640+0000","last_change":"2024-04-26T18:09:33.524351+0000","last_active":"2024-04-26T18:10:31.881640+0000","last_peered":"2024-04-26T18:10:31.881640+0000","last_clean":"2024-04-26T18:10:31.881640+0000","last_became_active":"2024-04-26T18:09:33.523953+0000","last_became_peered":"2024-04-26T18:09:33.523953+0000","last_unstale":"2024-04-26T18:10:31.881640+0000","last_undegraded":"2024-04-26T18:10:31.881640+0000","last_fullsized":"2024-04-26T18:10:31.881640+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:22.887498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'5","reported_seq":140,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:47.102648+0000","last_change":"2024-04-26T18:08:44.345278+0000","last_active":"2024-04-26T18:10:47.102648+0000","last_peered":"2024-04-26T18:10:47.102648+0000","last_clean":"2024-04-26T18:10:47.102648+0000","last_became_active":"2024-04-26T18:08:44.344531+0000","last_became_peered":"2024-04-26T18:08:44.344531+0000","last_unstale":"2024-04-26T18:10:47.102648+0000","last_undegraded":"2024-04-26T18:10:47.102648+0000","last_fullsized":"2024-04-26T18:10:47.102648+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:02.667622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.885848+0000","last_change":"2024-04-26T18:08:47.369542+0000","last_active":"2024-04-26T18:10:31.885848+0000","last_peered":"2024-04-26T18:10:31.885848+0000","last_clean":"2024-04-26T18:10:31.885848+0000","last_became_active":"2024-04-26T18:08:47.369181+0000","last_became_peered":"2024-04-26T18:08:47.369181+0000","last_unstale":"2024-04-26T18:10:31.885848+0000","last_undegraded":"2024-04-26T18:10:31.885848+0000","last_fullsized":"2024-04-26T18:10:31.885848+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:56.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014706+0000","last_change":"2024-04-26T18:08:44.555580+0000","last_active":"2024-04-26T18:09:33.014706+0000","last_peered":"2024-04-26T18:09:33.014706+0000","last_clean":"2024-04-26T18:09:33.014706+0000","last_became_active":"2024-04-26T18:08:44.555247+0000","last_became_peered":"2024-04-26T18:08:44.555247+0000","last_unstale":"2024-04-26T18:09:33.014706+0000","last_undegraded":"2024-04-26T18:09:33.014706+0000","last_fullsized":"2024-04-26T18:09:33.014706+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:58:55.207355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879529+0000","last_change":"2024-04-26T18:08:47.372300+0000","last_active":"2024-04-26T18:10:31.879529+0000","last_peered":"2024-04-26T18:10:31.879529+0000","last_clean":"2024-04-26T18:10:31.879529+0000","last_became_active":"2024-04-26T18:08:47.371952+0000","last_became_peered":"2024-04-26T18:08:47.371952+0000","last_unstale":"2024-04-26T18:10:31.879529+0000","last_undegraded":"2024-04-26T18:10:31.879529+0000","last_fullsized":"2024-04-26T18:10:31.879529+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:58.254691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014666+0000","last_change":"2024-04-26T18:08:44.340926+0000","last_active":"2024-04-26T18:09:33.014666+0000","last_peered":"2024-04-26T18:09:33.014666+0000","last_clean":"2024-04-26T18:09:33.014666+0000","last_became_active":"2024-04-26T18:08:44.340563+0000","last_became_peered":"2024-04-26T18:08:44.340563+0000","last_unstale":"2024-04-26T18:09:33.014666+0000","last_undegraded":"2024-04-26T18:09:33.014666+0000","last_fullsized":"2024-04-26T18:09:33.014666+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:57.803017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867616+0000","last_change":"2024-04-26T18:09:33.021890+0000","last_active":"2024-04-26T18:10:31.867616+0000","last_peered":"2024-04-26T18:10:31.867616+0000","last_clean":"2024-04-26T18:10:31.867616+0000","last_became_active":"2024-04-26T18:09:33.021446+0000","last_became_peered":"2024-04-26T18:09:33.021446+0000","last_unstale":"2024-04-26T18:10:31.867616+0000","last_undegraded":"2024-04-26T18:10:31.867616+0000","last_fullsized":"2024-04-26T18:10:31.867616+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:29.158581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.009219+0000","last_change":"2024-04-26T18:08:44.555076+0000","last_active":"2024-04-26T18:09:33.009219+0000","last_peered":"2024-04-26T18:09:33.009219+0000","last_clean":"2024-04-26T18:09:33.009219+0000","last_became_active":"2024-04-26T18:08:44.554756+0000","last_became_peered":"2024-04-26T18:08:44.554756+0000","last_unstale":"2024-04-26T18:09:33.009219+0000","last_undegraded":"2024-04-26T18:09:33.009219+0000","last_fullsized":"2024-04-26T18:09:33.009219+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:21.950644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.871347+0000","last_change":"2024-04-26T18:08:47.366370+0000","last_active":"2024-04-26T18:10:31.871347+0000","last_peered":"2024-04-26T18:10:31.871347+0000","last_clean":"2024-04-26T18:10:31.871347+0000","last_became_active":"2024-04-26T18:08:47.365997+0000","last_became_peered":"2024-04-26T18:08:47.365997+0000","last_unstale":"2024-04-26T18:10:31.871347+0000","last_undegraded":"2024-04-26T18:10:31.871347+0000","last_fullsized":"2024-04-26T18:10:31.871347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:51.643438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.013454+0000","last_change":"2024-04-26T18:08:44.339299+0000","last_active":"2024-04-26T18:09:33.013454+0000","last_peered":"2024-04-26T18:09:33.013454+0000","last_clean":"2024-04-26T18:09:33.013454+0000","last_became_active":"2024-04-26T18:08:44.338932+0000","last_became_peered":"2024-04-26T18:08:44.338932+0000","last_unstale":"2024-04-26T18:09:33.013454+0000","last_undegraded":"2024-04-26T18:09:33.013454+0000","last_fullsized":"2024-04-26T18:09:33.013454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:18.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.867578+0000","last_change":"2024-04-26T18:08:47.369924+0000","last_active":"2024-04-26T18:10:31.867578+0000","last_peered":"2024-04-26T18:10:31.867578+0000","last_clean":"2024-04-26T18:10:31.867578+0000","last_became_active":"2024-04-26T18:08:47.369048+0000","last_became_peered":"2024-04-26T18:08:47.369048+0000","last_unstale":"2024-04-26T18:10:31.867578+0000","last_undegraded":"2024-04-26T18:10:31.867578+0000","last_fullsized":"2024-04-26T18:10:31.867578+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:19.110288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.879156+0000","last_change":"2024-04-26T18:08:47.370836+0000","last_active":"2024-04-26T18:10:31.879156+0000","last_peered":"2024-04-26T18:10:31.879156+0000","last_clean":"2024-04-26T18:10:31.879156+0000","last_became_active":"2024-04-26T18:08:47.370160+0000","last_became_peered":"2024-04-26T18:08:47.370160+0000","last_unstale":"2024-04-26T18:10:31.879156+0000","last_undegraded":"2024-04-26T18:10:31.879156+0000","last_fullsized":"2024-04-26T18:10:31.879156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:11:21.285466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'5","reported_seq":132,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:43.096314+0000","last_change":"2024-04-26T18:08:44.554960+0000","last_active":"2024-04-26T18:10:43.096314+0000","last_peered":"2024-04-26T18:10:43.096314+0000","last_clean":"2024-04-26T18:10:43.096314+0000","last_became_active":"2024-04-26T18:08:44.554607+0000","last_became_peered":"2024-04-26T18:08:44.554607+0000","last_unstale":"2024-04-26T18:10:43.096314+0000","last_undegraded":"2024-04-26T18:10:43.096314+0000","last_fullsized":"2024-04-26T18:10:43.096314+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:22:26.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.882140+0000","last_change":"2024-04-26T18:08:47.372528+0000","last_active":"2024-04-26T18:10:31.882140+0000","last_peered":"2024-04-26T18:10:31.882140+0000","last_clean":"2024-04-26T18:10:31.882140+0000","last_became_active":"2024-04-26T18:08:47.372215+0000","last_became_peered":"2024-04-26T18:08:47.372215+0000","last_unstale":"2024-04-26T18:10:31.882140+0000","last_undegraded":"2024-04-26T18:10:31.882140+0000","last_fullsized":"2024-04-26T18:10:31.882140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:39.366917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.020591+0000","last_change":"2024-04-26T18:08:44.342667+0000","last_active":"2024-04-26T18:09:33.020591+0000","last_peered":"2024-04-26T18:09:33.020591+0000","last_clean":"2024-04-26T18:09:33.020591+0000","last_became_active":"2024-04-26T18:08:44.342170+0000","last_became_peered":"2024-04-26T18:08:44.342170+0000","last_unstale":"2024-04-26T18:09:33.020591+0000","last_undegraded":"2024-04-26T18:09:33.020591+0000","last_fullsized":"2024-04-26T18:09:33.020591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:50.117177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":362,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012799+0000","last_change":"2024-04-26T18:07:24.173301+0000","last_active":"2024-04-26T18:09:33.012799+0000","last_peered":"2024-04-26T18:09:33.012799+0000","last_clean":"2024-04-26T18:09:33.012799+0000","last_became_active":"2024-04-26T18:07:24.172791+0000","last_became_peered":"2024-04-26T18:07:24.172791+0000","last_unstale":"2024-04-26T18:09:33.012799+0000","last_undegraded":"2024-04-26T18:09:33.012799+0000","last_fullsized":"2024-04-26T18:09:33.012799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_clean_scrub_stamp":"2024-04-26T18:03:58.528256+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:32.468624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876453+0000","last_change":"2024-04-26T18:08:47.370720+0000","last_active":"2024-04-26T18:10:31.876453+0000","last_peered":"2024-04-26T18:10:31.876453+0000","last_clean":"2024-04-26T18:10:31.876453+0000","last_became_active":"2024-04-26T18:08:47.370414+0000","last_became_peered":"2024-04-26T18:08:47.370414+0000","last_unstale":"2024-04-26T18:10:31.876453+0000","last_undegraded":"2024-04-26T18:10:31.876453+0000","last_fullsized":"2024-04-26T18:10:31.876453+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:37.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012541+0000","last_change":"2024-04-26T18:08:44.347077+0000","last_active":"2024-04-26T18:09:33.012541+0000","last_peered":"2024-04-26T18:09:33.012541+0000","last_clean":"2024-04-26T18:09:33.012541+0000","last_became_active":"2024-04-26T18:08:44.346481+0000","last_became_peered":"2024-04-26T18:08:44.346481+0000","last_unstale":"2024-04-26T18:09:33.012541+0000","last_undegraded":"2024-04-26T18:09:33.012541+0000","last_fullsized":"2024-04-26T18:09:33.012541+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:08.840814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.025243+0000","last_change":"2024-04-26T18:09:33.025243+0000","last_active":"2024-04-26T18:09:33.025243+0000","last_peered":"2024-04-26T18:09:33.025243+0000","last_clean":"2024-04-26T18:09:33.025243+0000","last_became_active":"2024-04-26T18:09:33.015430+0000","last_became_peered":"2024-04-26T18:09:33.015430+0000","last_unstale":"2024-04-26T18:09:33.025243+0000","last_undegraded":"2024-04-26T18:09:33.025243+0000","last_fullsized":"2024-04-26T18:09:33.025243+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:54.271435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.877090+0000","last_change":"2024-04-26T18:08:47.366552+0000","last_active":"2024-04-26T18:10:31.877090+0000","last_peered":"2024-04-26T18:10:31.877090+0000","last_clean":"2024-04-26T18:10:31.877090+0000","last_became_active":"2024-04-26T18:08:47.366183+0000","last_became_peered":"2024-04-26T18:08:47.366183+0000","last_unstale":"2024-04-26T18:10:31.877090+0000","last_undegraded":"2024-04-26T18:10:31.877090+0000","last_fullsized":"2024-04-26T18:10:31.877090+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:36.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'13","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.009328+0000","last_change":"2024-04-26T18:08:44.338078+0000","last_active":"2024-04-26T18:09:33.009328+0000","last_peered":"2024-04-26T18:09:33.009328+0000","last_clean":"2024-04-26T18:09:33.009328+0000","last_became_active":"2024-04-26T18:08:44.337715+0000","last_became_peered":"2024-04-26T18:08:44.337715+0000","last_unstale":"2024-04-26T18:09:33.009328+0000","last_undegraded":"2024-04-26T18:09:33.009328+0000","last_fullsized":"2024-04-26T18:09:33.009328+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.912993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878815+0000","last_change":"2024-04-26T18:08:47.371321+0000","last_active":"2024-04-26T18:10:31.878815+0000","last_peered":"2024-04-26T18:10:31.878815+0000","last_clean":"2024-04-26T18:10:31.878815+0000","last_became_active":"2024-04-26T18:08:47.371015+0000","last_became_peered":"2024-04-26T18:08:47.371015+0000","last_unstale":"2024-04-26T18:10:31.878815+0000","last_undegraded":"2024-04-26T18:10:31.878815+0000","last_fullsized":"2024-04-26T18:10:31.878815+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.020792+0000","last_change":"2024-04-26T18:08:44.420101+0000","last_active":"2024-04-26T18:09:33.020792+0000","last_peered":"2024-04-26T18:09:33.020792+0000","last_clean":"2024-04-26T18:09:33.020792+0000","last_became_active":"2024-04-26T18:08:44.419787+0000","last_became_peered":"2024-04-26T18:08:44.419787+0000","last_unstale":"2024-04-26T18:09:33.020792+0000","last_undegraded":"2024-04-26T18:09:33.020792+0000","last_fullsized":"2024-04-26T18:09:33.020792+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.485425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878338+0000","last_change":"2024-04-26T18:08:47.372309+0000","last_active":"2024-04-26T18:10:31.878338+0000","last_peered":"2024-04-26T18:10:31.878338+0000","last_clean":"2024-04-26T18:10:31.878338+0000","last_became_active":"2024-04-26T18:08:47.371980+0000","last_became_peered":"2024-04-26T18:08:47.371980+0000","last_unstale":"2024-04-26T18:10:31.878338+0000","last_undegraded":"2024-04-26T18:10:31.878338+0000","last_fullsized":"2024-04-26T18:10:31.878338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:05.011077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.012672+0000","last_change":"2024-04-26T18:08:44.555500+0000","last_active":"2024-04-26T18:09:33.012672+0000","last_peered":"2024-04-26T18:09:33.012672+0000","last_clean":"2024-04-26T18:09:33.012672+0000","last_became_active":"2024-04-26T18:08:44.555137+0000","last_became_peered":"2024-04-26T18:08:44.555137+0000","last_unstale":"2024-04-26T18:09:33.012672+0000","last_undegraded":"2024-04-26T18:09:33.012672+0000","last_fullsized":"2024-04-26T18:09:33.012672+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:10:05.723654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.839228+0000","last_change":"2024-04-26T18:08:47.364215+0000","last_active":"2024-04-26T18:10:31.839228+0000","last_peered":"2024-04-26T18:10:31.839228+0000","last_clean":"2024-04-26T18:10:31.839228+0000","last_became_active":"2024-04-26T18:08:47.363862+0000","last_became_peered":"2024-04-26T18:08:47.363862+0000","last_unstale":"2024-04-26T18:10:31.839228+0000","last_undegraded":"2024-04-26T18:10:31.839228+0000","last_fullsized":"2024-04-26T18:10:31.839228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:27.020100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.013914+0000","last_change":"2024-04-26T18:08:44.343506+0000","last_active":"2024-04-26T18:09:33.013914+0000","last_peered":"2024-04-26T18:09:33.013914+0000","last_clean":"2024-04-26T18:09:33.013914+0000","last_became_active":"2024-04-26T18:08:44.343163+0000","last_became_peered":"2024-04-26T18:08:44.343163+0000","last_unstale":"2024-04-26T18:09:33.013914+0000","last_undegraded":"2024-04-26T18:09:33.013914+0000","last_fullsized":"2024-04-26T18:09:33.013914+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:09.941391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.862510+0000","last_change":"2024-04-26T18:08:47.365572+0000","last_active":"2024-04-26T18:10:31.862510+0000","last_peered":"2024-04-26T18:10:31.862510+0000","last_clean":"2024-04-26T18:10:31.862510+0000","last_became_active":"2024-04-26T18:08:47.364827+0000","last_became_peered":"2024-04-26T18:08:47.364827+0000","last_unstale":"2024-04-26T18:10:31.862510+0000","last_undegraded":"2024-04-26T18:10:31.862510+0000","last_fullsized":"2024-04-26T18:10:31.862510+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:44.492380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014614+0000","last_change":"2024-04-26T18:08:44.339063+0000","last_active":"2024-04-26T18:09:33.014614+0000","last_peered":"2024-04-26T18:09:33.014614+0000","last_clean":"2024-04-26T18:09:33.014614+0000","last_became_active":"2024-04-26T18:08:44.338752+0000","last_became_peered":"2024-04-26T18:08:44.338752+0000","last_unstale":"2024-04-26T18:09:33.014614+0000","last_undegraded":"2024-04-26T18:09:33.014614+0000","last_fullsized":"2024-04-26T18:09:33.014614+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:38.201263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880808+0000","last_change":"2024-04-26T18:08:47.367859+0000","last_active":"2024-04-26T18:10:31.880808+0000","last_peered":"2024-04-26T18:10:31.880808+0000","last_clean":"2024-04-26T18:10:31.880808+0000","last_became_active":"2024-04-26T18:08:47.367548+0000","last_became_peered":"2024-04-26T18:08:47.367548+0000","last_unstale":"2024-04-26T18:10:31.880808+0000","last_undegraded":"2024-04-26T18:10:31.880808+0000","last_fullsized":"2024-04-26T18:10:31.880808+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:13:27.612151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.014359+0000","last_change":"2024-04-26T18:08:44.345392+0000","last_active":"2024-04-26T18:09:33.014359+0000","last_peered":"2024-04-26T18:09:33.014359+0000","last_clean":"2024-04-26T18:09:33.014359+0000","last_became_active":"2024-04-26T18:08:44.344578+0000","last_became_peered":"2024-04-26T18:08:44.344578+0000","last_unstale":"2024-04-26T18:09:33.014359+0000","last_undegraded":"2024-04-26T18:09:33.014359+0000","last_fullsized":"2024-04-26T18:09:33.014359+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:26.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887184+0000","last_change":"2024-04-26T18:08:47.380099+0000","last_active":"2024-04-26T18:10:31.887184+0000","last_peered":"2024-04-26T18:10:31.887184+0000","last_clean":"2024-04-26T18:10:31.887184+0000","last_became_active":"2024-04-26T18:08:47.379576+0000","last_became_peered":"2024-04-26T18:08:47.379576+0000","last_unstale":"2024-04-26T18:10:31.887184+0000","last_undegraded":"2024-04-26T18:10:31.887184+0000","last_fullsized":"2024-04-26T18:10:31.887184+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:28.397886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.015156+0000","last_change":"2024-04-26T18:08:44.555299+0000","last_active":"2024-04-26T18:09:33.015156+0000","last_peered":"2024-04-26T18:09:33.015156+0000","last_clean":"2024-04-26T18:09:33.015156+0000","last_became_active":"2024-04-26T18:08:44.554959+0000","last_became_peered":"2024-04-26T18:08:44.554959+0000","last_unstale":"2024-04-26T18:09:33.015156+0000","last_undegraded":"2024-04-26T18:09:33.015156+0000","last_fullsized":"2024-04-26T18:09:33.015156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.565073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887608+0000","last_change":"2024-04-26T18:08:47.377524+0000","last_active":"2024-04-26T18:10:31.887608+0000","last_peered":"2024-04-26T18:10:31.887608+0000","last_clean":"2024-04-26T18:10:31.887608+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:10:31.887608+0000","last_undegraded":"2024-04-26T18:10:31.887608+0000","last_fullsized":"2024-04-26T18:10:31.887608+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:10.466615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"90'1","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.011551+0000","last_change":"2024-04-26T18:08:44.343894+0000","last_active":"2024-04-26T18:09:33.011551+0000","last_peered":"2024-04-26T18:09:33.011551+0000","last_clean":"2024-04-26T18:09:33.011551+0000","last_became_active":"2024-04-26T18:08:44.343153+0000","last_became_peered":"2024-04-26T18:08:44.343153+0000","last_unstale":"2024-04-26T18:09:33.011551+0000","last_undegraded":"2024-04-26T18:09:33.011551+0000","last_fullsized":"2024-04-26T18:09:33.011551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:09.914701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880354+0000","last_change":"2024-04-26T18:08:47.371962+0000","last_active":"2024-04-26T18:10:31.880354+0000","last_peered":"2024-04-26T18:10:31.880354+0000","last_clean":"2024-04-26T18:10:31.880354+0000","last_became_active":"2024-04-26T18:08:47.371517+0000","last_became_peered":"2024-04-26T18:08:47.371517+0000","last_unstale":"2024-04-26T18:10:31.880354+0000","last_undegraded":"2024-04-26T18:10:31.880354+0000","last_fullsized":"2024-04-26T18:10:31.880354+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:23.553409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:09:33.523908+0000","last_change":"2024-04-26T18:08:44.337853+0000","last_active":"2024-04-26T18:09:33.523908+0000","last_peered":"2024-04-26T18:09:33.523908+0000","last_clean":"2024-04-26T18:09:33.523908+0000","last_became_active":"2024-04-26T18:08:44.337393+0000","last_became_peered":"2024-04-26T18:08:44.337393+0000","last_unstale":"2024-04-26T18:09:33.523908+0000","last_undegraded":"2024-04-26T18:09:33.523908+0000","last_fullsized":"2024-04-26T18:09:33.523908+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:26.514821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.887055+0000","last_change":"2024-04-26T18:08:47.370999+0000","last_active":"2024-04-26T18:10:31.887055+0000","last_peered":"2024-04-26T18:10:31.887055+0000","last_clean":"2024-04-26T18:10:31.887055+0000","last_became_active":"2024-04-26T18:08:47.370373+0000","last_became_peered":"2024-04-26T18:08:47.370373+0000","last_unstale":"2024-04-26T18:10:31.887055+0000","last_undegraded":"2024-04-26T18:10:31.887055+0000","last_fullsized":"2024-04-26T18:10:31.887055+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:19.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.865070+0000","last_change":"2024-04-26T18:08:47.365722+0000","last_active":"2024-04-26T18:10:31.865070+0000","last_peered":"2024-04-26T18:10:31.865070+0000","last_clean":"2024-04-26T18:10:31.865070+0000","last_became_active":"2024-04-26T18:08:47.364692+0000","last_became_peered":"2024-04-26T18:08:47.364692+0000","last_unstale":"2024-04-26T18:10:31.865070+0000","last_undegraded":"2024-04-26T18:10:31.865070+0000","last_fullsized":"2024-04-26T18:10:31.865070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:02.158156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880953+0000","last_change":"2024-04-26T18:08:47.373886+0000","last_active":"2024-04-26T18:10:31.880953+0000","last_peered":"2024-04-26T18:10:31.880953+0000","last_clean":"2024-04-26T18:10:31.880953+0000","last_became_active":"2024-04-26T18:08:47.373391+0000","last_became_peered":"2024-04-26T18:08:47.373391+0000","last_unstale":"2024-04-26T18:10:31.880953+0000","last_undegraded":"2024-04-26T18:10:31.880953+0000","last_fullsized":"2024-04-26T18:10:31.880953+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:48:59.280050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880574+0000","last_change":"2024-04-26T18:08:47.366160+0000","last_active":"2024-04-26T18:10:31.880574+0000","last_peered":"2024-04-26T18:10:31.880574+0000","last_clean":"2024-04-26T18:10:31.880574+0000","last_became_active":"2024-04-26T18:08:47.365811+0000","last_became_peered":"2024-04-26T18:08:47.365811+0000","last_unstale":"2024-04-26T18:10:31.880574+0000","last_undegraded":"2024-04-26T18:10:31.880574+0000","last_fullsized":"2024-04-26T18:10:31.880574+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:32.124692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.871898+0000","last_change":"2024-04-26T18:08:47.366078+0000","last_active":"2024-04-26T18:10:31.871898+0000","last_peered":"2024-04-26T18:10:31.871898+0000","last_clean":"2024-04-26T18:10:31.871898+0000","last_became_active":"2024-04-26T18:08:47.365779+0000","last_became_peered":"2024-04-26T18:08:47.365779+0000","last_unstale":"2024-04-26T18:10:31.871898+0000","last_undegraded":"2024-04-26T18:10:31.871898+0000","last_fullsized":"2024-04-26T18:10:31.871898+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.127606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.871262+0000","last_change":"2024-04-26T18:08:47.372996+0000","last_active":"2024-04-26T18:10:31.871262+0000","last_peered":"2024-04-26T18:10:31.871262+0000","last_clean":"2024-04-26T18:10:31.871262+0000","last_became_active":"2024-04-26T18:08:47.372687+0000","last_became_peered":"2024-04-26T18:08:47.372687+0000","last_unstale":"2024-04-26T18:10:31.871262+0000","last_undegraded":"2024-04-26T18:10:31.871262+0000","last_fullsized":"2024-04-26T18:10:31.871262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:13:33.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.859831+0000","last_change":"2024-04-26T18:08:47.371076+0000","last_active":"2024-04-26T18:10:31.859831+0000","last_peered":"2024-04-26T18:10:31.859831+0000","last_clean":"2024-04-26T18:10:31.859831+0000","last_became_active":"2024-04-26T18:08:47.370688+0000","last_became_peered":"2024-04-26T18:08:47.370688+0000","last_unstale":"2024-04-26T18:10:31.859831+0000","last_undegraded":"2024-04-26T18:10:31.859831+0000","last_fullsized":"2024-04-26T18:10:31.859831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:21.355971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.884277+0000","last_change":"2024-04-26T18:08:47.372392+0000","last_active":"2024-04-26T18:10:31.884277+0000","last_peered":"2024-04-26T18:10:31.884277+0000","last_clean":"2024-04-26T18:10:31.884277+0000","last_became_active":"2024-04-26T18:08:47.372020+0000","last_became_peered":"2024-04-26T18:08:47.372020+0000","last_unstale":"2024-04-26T18:10:31.884277+0000","last_undegraded":"2024-04-26T18:10:31.884277+0000","last_fullsized":"2024-04-26T18:10:31.884277+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.878227+0000","last_change":"2024-04-26T18:08:47.369571+0000","last_active":"2024-04-26T18:10:31.878227+0000","last_peered":"2024-04-26T18:10:31.878227+0000","last_clean":"2024-04-26T18:10:31.878227+0000","last_became_active":"2024-04-26T18:08:47.369165+0000","last_became_peered":"2024-04-26T18:08:47.369165+0000","last_unstale":"2024-04-26T18:10:31.878227+0000","last_undegraded":"2024-04-26T18:10:31.878227+0000","last_fullsized":"2024-04-26T18:10:31.878227+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:15.869182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.877581+0000","last_change":"2024-04-26T18:08:47.372355+0000","last_active":"2024-04-26T18:10:31.877581+0000","last_peered":"2024-04-26T18:10:31.877581+0000","last_clean":"2024-04-26T18:10:31.877581+0000","last_became_active":"2024-04-26T18:08:47.371810+0000","last_became_peered":"2024-04-26T18:08:47.371810+0000","last_unstale":"2024-04-26T18:10:31.877581+0000","last_undegraded":"2024-04-26T18:10:31.877581+0000","last_fullsized":"2024-04-26T18:10:31.877581+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:44:22.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.876807+0000","last_change":"2024-04-26T18:08:47.364042+0000","last_active":"2024-04-26T18:10:31.876807+0000","last_peered":"2024-04-26T18:10:31.876807+0000","last_clean":"2024-04-26T18:10:31.876807+0000","last_became_active":"2024-04-26T18:08:47.363727+0000","last_became_peered":"2024-04-26T18:08:47.363727+0000","last_unstale":"2024-04-26T18:10:31.876807+0000","last_undegraded":"2024-04-26T18:10:31.876807+0000","last_fullsized":"2024-04-26T18:10:31.876807+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:37.595261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.888285+0000","last_change":"2024-04-26T18:08:47.377535+0000","last_active":"2024-04-26T18:10:31.888285+0000","last_peered":"2024-04-26T18:10:31.888285+0000","last_clean":"2024-04-26T18:10:31.888285+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:10:31.888285+0000","last_undegraded":"2024-04-26T18:10:31.888285+0000","last_fullsized":"2024-04-26T18:10:31.888285+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:22.895402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.880770+0000","last_change":"2024-04-26T18:08:47.367313+0000","last_active":"2024-04-26T18:10:31.880770+0000","last_peered":"2024-04-26T18:10:31.880770+0000","last_clean":"2024-04-26T18:10:31.880770+0000","last_became_active":"2024-04-26T18:08:47.366971+0000","last_became_peered":"2024-04-26T18:08:47.366971+0000","last_unstale":"2024-04-26T18:10:31.880770+0000","last_undegraded":"2024-04-26T18:10:31.880770+0000","last_fullsized":"2024-04-26T18:10:31.880770+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:14.518100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T18:10:31.860309+0000","last_change":"2024-04-26T18:08:47.373004+0000","last_active":"2024-04-26T18:10:31.860309+0000","last_peered":"2024-04-26T18:10:31.860309+0000","last_clean":"2024-04-26T18:10:31.860309+0000","last_became_active":"2024-04-26T18:08:47.372641+0000","last_became_peered":"2024-04-26T18:08:47.372641+0000","last_unstale":"2024-04-26T18:10:31.860309+0000","last_undegraded":"2024-04-26T18:10:31.860309+0000","last_fullsized":"2024-04-26T18:10:31.860309+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:02.018425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64685,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":648,"num_read_kb":389,"num_write":175,"num_write_kb":118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":489607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579948,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1699840,"data_stored":1141575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":489787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":489685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266749,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1613824,"data_stored":1082310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":489176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36380,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708772,"statfs":{"total":95995035648,"available":95957782528,"internally_reserved":0,"allocated":1069056,"data_stored":548980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1028096,"data_stored":490447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":550828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":489487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":490145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36924,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708228,"statfs":{"total":95995035648,"available":95957225472,"internally_reserved":0,"allocated":1626112,"data_stored":1081462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":61859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:10:50.860 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:10:50.860 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T18:10:50.860 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T18:10:50.860 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T18:10:51.104 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:51 smithi119 ceph-mon[32752]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:51 smithi167 ceph-mon[32751]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:51.334 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:51 smithi090 ceph-mon[29180]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:52.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:52 smithi090 ceph-mon[29180]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:10:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:52 smithi119 ceph-mon[32752]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:10:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:52 smithi167 ceph-mon[32751]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:10:52.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 -- 172.21.15.90:0/1909630204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8108390 msgr2=0x7f5fb8108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:52.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/1909630204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8108390 0x7f5fb8108770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800b0a0 tx=0x7f5fa800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 -- 172.21.15.90:0/1909630204 shutdown_connections 2024-04-26T18:10:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/1909630204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8108390 0x7f5fb8108770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/1909630204 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fb8102ce0 0x7f5fb810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/1909630204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fb8102320 0x7f5fb81027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 -- 172.21.15.90:0/1909630204 >> 172.21.15.90:0/1909630204 conn(0x7f5fb80792d0 msgr2=0x7f5fb80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 -- 172.21.15.90:0/1909630204 shutdown_connections 2024-04-26T18:10:52.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.738+0000 7f5fbfc35700 1 -- 172.21.15.90:0/1909630204 wait complete. 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.739+0000 7f5fbfc35700 1 Processor -- start 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.739+0000 7f5fbfc35700 1 -- start start 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.739+0000 7f5fbfc35700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8102320 0x7f5fb819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.739+0000 7f5fbfc35700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fb8102ce0 0x7f5fb819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.739+0000 7f5fbfc35700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fb8108390 0x7f5fb8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.739+0000 7f5fbfc35700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8112600 con 0x7f5fb8102320 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.739+0000 7f5fbfc35700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8112480 con 0x7f5fb8102ce0 2024-04-26T18:10:52.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbfc35700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8112780 con 0x7f5fb8108390 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8102320 0x7f5fb819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8102320 0x7f5fb819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46374/0 (socket says 172.21.15.90:46374) 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 -- 172.21.15.90:0/2266833039 learned_addr learned my addr 172.21.15.90:0/2266833039 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd1d0700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fb8102ce0 0x7f5fb819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbe1d2700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fb8108390 0x7f5fb8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 -- 172.21.15.90:0/2266833039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fb8108390 msgr2=0x7f5fb8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fb8108390 0x7f5fb8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 -- 172.21.15.90:0/2266833039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fb8102ce0 msgr2=0x7f5fb819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fb8102ce0 0x7f5fb819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 -- 172.21.15.90:0/2266833039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb81990e0 con 0x7f5fb8102320 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbd9d1700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8102320 0x7f5fb819e7c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800b070 tx=0x7f5fa8004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fb6ffd700 1 -- 172.21.15.90:0/2266833039 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa8021070 con 0x7f5fb8102320 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fb6ffd700 1 -- 172.21.15.90:0/2266833039 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fa8007c80 con 0x7f5fb8102320 2024-04-26T18:10:52.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb8199370 con 0x7f5fb8102320 2024-04-26T18:10:52.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fb6ffd700 1 -- 172.21.15.90:0/2266833039 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fa801d450 con 0x7f5fb8102320 2024-04-26T18:10:52.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.740+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb806b100 con 0x7f5fb8102320 2024-04-26T18:10:52.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.741+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb806a110 con 0x7f5fb8102320 2024-04-26T18:10:52.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.742+0000 7f5fb6ffd700 1 -- 172.21.15.90:0/2266833039 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5fa8004000 con 0x7f5fb8102320 2024-04-26T18:10:52.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.743+0000 7f5fb6ffd700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5fa00a3cc0 0x7f5fa00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:52.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.743+0000 7f5fbd1d0700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5fa00a3cc0 0x7f5fa00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:52.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.743+0000 7f5fb6ffd700 1 -- 172.21.15.90:0/2266833039 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f5fa800f020 con 0x7f5fb8102320 2024-04-26T18:10:52.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.744+0000 7f5fbd1d0700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5fa00a3cc0 0x7f5fa00a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8079a40 tx=0x7f5fac009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:52.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.747+0000 7f5fb6ffd700 1 -- 172.21.15.90:0/2266833039 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fa802a020 con 0x7f5fb8102320 2024-04-26T18:10:52.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.992+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5fb806baa0 con 0x7f5fb8102320 2024-04-26T18:10:52.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.992+0000 7f5fb6ffd700 1 -- 172.21.15.90:0/2266833039 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5fa809e920 con 0x7f5fb8102320 2024-04-26T18:10:52.992 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:10:52.993 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.994+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5fa00a3cc0 msgr2=0x7f5fa00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.994+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5fa00a3cc0 0x7f5fa00a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8079a40 tx=0x7f5fac009190 comp rx=0 tx=0).stop 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.994+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8102320 msgr2=0x7f5fb819e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.994+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8102320 0x7f5fb819e7c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5fa800b070 tx=0x7f5fa8004e40 comp rx=0 tx=0).stop 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 shutdown_connections 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f5fa00a3cc0 0x7f5fa00a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5fb8102320 0x7f5fb819e7c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5fb8108390 0x7f5fb8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 --2- 172.21.15.90:0/2266833039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fb8102ce0 0x7f5fb819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 >> 172.21.15.90:0/2266833039 conn(0x7f5fb80792d0 msgr2=0x7f5fb81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 shutdown_connections 2024-04-26T18:10:52.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:52.995+0000 7f5fbfc35700 1 -- 172.21.15.90:0/2266833039 wait complete. 2024-04-26T18:10:53.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:53 smithi090 ceph-mon[29180]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:53.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:53 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2266833039' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:10:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:53 smithi119 ceph-mon[32752]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:53 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2266833039' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:10:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:53 smithi167 ceph-mon[32751]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:53 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2266833039' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:10:53.615 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T18:10:53.615 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:10:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:55.247 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:55 smithi090 ceph-mon[29180]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:55 smithi119 ceph-mon[32752]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:55 smithi167 ceph-mon[32751]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:10:55.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 -- 172.21.15.90:0/1822187685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc102ce0 msgr2=0x7fb0fc10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 --2- 172.21.15.90:0/1822187685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc102ce0 0x7fb0fc10f1d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec009990 tx=0x7fb0ec009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 -- 172.21.15.90:0/1822187685 shutdown_connections 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 --2- 172.21.15.90:0/1822187685 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc102ce0 0x7fb0fc10f1d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 --2- 172.21.15.90:0/1822187685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc108390 0x7fb0fc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 --2- 172.21.15.90:0/1822187685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc102320 0x7fb0fc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 -- 172.21.15.90:0/1822187685 >> 172.21.15.90:0/1822187685 conn(0x7fb0fc0792d0 msgr2=0x7fb0fc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 -- 172.21.15.90:0/1822187685 shutdown_connections 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.493+0000 7fb102ff9700 1 -- 172.21.15.90:0/1822187685 wait complete. 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 Processor -- start 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 -- start start 2024-04-26T18:10:55.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc102320 0x7fb0fc19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc102ce0 0x7fb0fc19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc108390 0x7fb0fc198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc112530 con 0x7fb0fc108390 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1123b0 con 0x7fb0fc102ce0 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb102ff9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb0fc1126b0 con 0x7fb0fc102320 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc108390 0x7fb0fc198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc108390 0x7fb0fc198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46386/0 (socket says 172.21.15.90:46386) 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 -- 172.21.15.90:0/3275309206 learned_addr learned my addr 172.21.15.90:0/3275309206 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb100d95700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc102320 0x7fb0fc19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb0fbfff700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc102ce0 0x7fb0fc19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 -- 172.21.15.90:0/3275309206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc102320 msgr2=0x7fb0fc19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:55.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc102320 0x7fb0fc19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 -- 172.21.15.90:0/3275309206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc102ce0 msgr2=0x7fb0fc19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc102ce0 0x7fb0fc19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 -- 172.21.15.90:0/3275309206 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0fc199050 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.494+0000 7fb101596700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc108390 0x7fb0fc198770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005910 tx=0x7fb0ec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.495+0000 7fb0f9ffb700 1 -- 172.21.15.90:0/3275309206 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec027070 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.495+0000 7fb0f9ffb700 1 -- 172.21.15.90:0/3275309206 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0ec004020 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.495+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0fc1992e0 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.495+0000 7fb0f9ffb700 1 -- 172.21.15.90:0/3275309206 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ec022d40 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.495+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0fc1a5620 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.496+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0fc199640 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.497+0000 7fb0f9ffb700 1 -- 172.21.15.90:0/3275309206 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb0ec02a070 con 0x7fb0fc108390 2024-04-26T18:10:55.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.497+0000 7fb0f9ffb700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb0e40a3cc0 0x7fb0e40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:55.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.498+0000 7fb100d95700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb0e40a3cc0 0x7fb0e40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:55.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.498+0000 7fb0f9ffb700 1 -- 172.21.15.90:0/3275309206 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb0ec0d1990 con 0x7fb0fc108390 2024-04-26T18:10:55.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.498+0000 7fb100d95700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb0e40a3cc0 0x7fb0e40a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400e270 tx=0x7fb0f4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:55.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.501+0000 7fb0f9ffb700 1 -- 172.21.15.90:0/3275309206 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0ec031070 con 0x7fb0fc108390 2024-04-26T18:10:55.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.746+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0fc02e490 con 0x7fb0fc108390 2024-04-26T18:10:55.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.747+0000 7fb0f9ffb700 1 -- 172.21.15.90:0/3275309206 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11874 (secure 0 0 0) 0x7fb0ec013070 con 0x7fb0fc108390 2024-04-26T18:10:55.747 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:10:55.747 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:09:06.760287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24448,24449,24436]},"id":1}]} 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb0e40a3cc0 msgr2=0x7fb0e40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb0e40a3cc0 0x7fb0e40a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400e270 tx=0x7fb0f4009040 comp rx=0 tx=0).stop 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc108390 msgr2=0x7fb0fc198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc108390 0x7fb0fc198770 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb0ec005910 tx=0x7fb0ec0047b0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 shutdown_connections 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb0e40a3cc0 0x7fb0e40a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0fc108390 0x7fb0fc198770 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb0fc102320 0x7fb0fc19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 --2- 172.21.15.90:0/3275309206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0fc102ce0 0x7fb0fc19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:55.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 >> 172.21.15.90:0/3275309206 conn(0x7fb0fc0792d0 msgr2=0x7fb0fc100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:55.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 shutdown_connections 2024-04-26T18:10:55.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:55.749+0000 7fb102ff9700 1 -- 172.21.15.90:0/3275309206 wait complete. 2024-04-26T18:10:55.750 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T18:10:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:56 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3275309206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:10:56.313 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:56 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3275309206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:10:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:56 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3275309206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:10:56.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:10:56.590 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:10:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:57 smithi167 ceph-mon[32751]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:57.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:57 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:10:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:57 smithi119 ceph-mon[32752]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:57 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:10:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:57 smithi090 ceph-mon[29180]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:10:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:57 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:10:58.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 -- 172.21.15.90:0/3121012595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 msgr2=0x7f4660075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:58.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 --2- 172.21.15.90:0/3121012595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 0x7f4660075650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4650009990 tx=0x7f4650009ca0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 -- 172.21.15.90:0/3121012595 shutdown_connections 2024-04-26T18:10:58.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 --2- 172.21.15.90:0/3121012595 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 0x7f4660075650 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 --2- 172.21.15.90:0/3121012595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660114120 0x7f4660116510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 --2- 172.21.15.90:0/3121012595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4660075b90 0x7f4660076010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 -- 172.21.15.90:0/3121012595 >> 172.21.15.90:0/3121012595 conn(0x7f466007a5e0 msgr2=0x7f466007a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:58.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 -- 172.21.15.90:0/3121012595 shutdown_connections 2024-04-26T18:10:58.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.184+0000 7f466689d700 1 -- 172.21.15.90:0/3121012595 wait complete. 2024-04-26T18:10:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 Processor -- start 2024-04-26T18:10:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 -- start start 2024-04-26T18:10:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660075b90 0x7f46601a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:58.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 0x7f46601a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4660114120 0x7f46601a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4660119c30 con 0x7f4660077400 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4660119ab0 con 0x7f4660075b90 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f466689d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4660119db0 con 0x7f4660114120 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f465ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660075b90 0x7f46601a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f465f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 0x7f46601a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.185+0000 7f465ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660075b90 0x7f46601a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:50232/0 (socket says 172.21.15.90:50232) 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465ffff700 1 -- 172.21.15.90:0/4002219162 learned_addr learned my addr 172.21.15.90:0/4002219162 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f4664e3a700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4660114120 0x7f46601a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465ffff700 1 -- 172.21.15.90:0/4002219162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4660114120 msgr2=0x7f46601a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465ffff700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4660114120 0x7f46601a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465ffff700 1 -- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 msgr2=0x7f46601a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465ffff700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 0x7f46601a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465ffff700 1 -- 172.21.15.90:0/4002219162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46601a5110 con 0x7f4660075b90 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465f7fe700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 0x7f46601a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465ffff700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660075b90 0x7f46601a0100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4650000c00 tx=0x7f46500047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465d7fa700 1 -- 172.21.15.90:0/4002219162 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650027070 con 0x7f4660075b90 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46601a53a0 con 0x7f4660075b90 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465d7fa700 1 -- 172.21.15.90:0/4002219162 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4650003f20 con 0x7f4660075b90 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46601acc80 con 0x7f4660075b90 2024-04-26T18:10:58.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.186+0000 7f465d7fa700 1 -- 172.21.15.90:0/4002219162 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4650022ca0 con 0x7f4660075b90 2024-04-26T18:10:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.188+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466006a110 con 0x7f4660075b90 2024-04-26T18:10:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.189+0000 7f465d7fa700 1 -- 172.21.15.90:0/4002219162 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f465002a070 con 0x7f4660075b90 2024-04-26T18:10:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.189+0000 7f465d7fa700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f46480a3c40 0x7f46480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:10:58.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.189+0000 7f465f7fe700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f46480a3c40 0x7f46480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:10:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.189+0000 7f465d7fa700 1 -- 172.21.15.90:0/4002219162 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f46500d11e0 con 0x7f4660075b90 2024-04-26T18:10:58.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.190+0000 7f465f7fe700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f46480a3c40 0x7f46480a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4660113ee0 tx=0x7f465400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:10:58.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.199+0000 7f465d7fa700 1 -- 172.21.15.90:0/4002219162 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f465005eff0 con 0x7f4660075b90 2024-04-26T18:10:58.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.463+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46601ace50 con 0x7f4660075b90 2024-04-26T18:10:58.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.464+0000 7f465d7fa700 1 -- 172.21.15.90:0/4002219162 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11874 (secure 0 0 0) 0x7f4650013070 con 0x7f4660075b90 2024-04-26T18:10:58.464 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:10:58.465 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:09:06.760287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24448,24449,24436]},"id":1}]} 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f46480a3c40 msgr2=0x7f46480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f46480a3c40 0x7f46480a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4660113ee0 tx=0x7f465400b3c0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660075b90 msgr2=0x7f46601a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660075b90 0x7f46601a0100 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4650000c00 tx=0x7f46500047b0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 shutdown_connections 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f46480a3c40 0x7f46480a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4660114120 0x7f46601a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4660075b90 0x7f46601a0100 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 --2- 172.21.15.90:0/4002219162 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4660077400 0x7f46601a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 >> 172.21.15.90:0/4002219162 conn(0x7f466007a5e0 msgr2=0x7f4660112110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:10:58.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 shutdown_connections 2024-04-26T18:10:58.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:10:58.467+0000 7f466689d700 1 -- 172.21.15.90:0/4002219162 wait complete. 2024-04-26T18:10:58.468 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T18:10:59.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:59 smithi090 ceph-mon[29180]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:59.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:59 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:10:59 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/4002219162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:10:59.144 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:10:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:59 smithi119 ceph-mon[32752]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:59 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:10:59 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/4002219162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:10:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:59 smithi167 ceph-mon[32751]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:10:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:59 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:10:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:10:59 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/4002219162' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:10:59.504 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:01.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 -- 172.21.15.90:0/731854110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 msgr2=0x7f9e941029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 --2- 172.21.15.90:0/731854110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941029f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9e840099c0 tx=0x7f9e84009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 -- 172.21.15.90:0/731854110 shutdown_connections 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 --2- 172.21.15.90:0/731854110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941029f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 --2- 172.21.15.90:0/731854110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e94107660 0x7f9e9410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 --2- 172.21.15.90:0/731854110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e94102fc0 0x7f9e94106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 -- 172.21.15.90:0/731854110 >> 172.21.15.90:0/731854110 conn(0x7f9e94078370 msgr2=0x7f9e94078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 -- 172.21.15.90:0/731854110 shutdown_connections 2024-04-26T18:11:01.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 -- 172.21.15.90:0/731854110 wait complete. 2024-04-26T18:11:01.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:01 smithi090 ceph-mon[29180]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:01.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.151+0000 7f9e99960700 1 Processor -- start 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e99960700 1 -- start start 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e99960700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e99960700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e94102fc0 0x7f9e941a08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e99960700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e94107660 0x7f9e941a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e99960700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9e941124f0 con 0x7f9e94102610 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e99960700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94112370 con 0x7f9e94102fc0 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e99960700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9e94112670 con 0x7f9e94107660 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46420/0 (socket says 172.21.15.90:46420) 2024-04-26T18:11:01.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 -- 172.21.15.90:0/113123766 learned_addr learned my addr 172.21.15.90:0/113123766 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e937fe700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e94107660 0x7f9e941a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e927fc700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e94102fc0 0x7f9e941a08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 -- 172.21.15.90:0/113123766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e94107660 msgr2=0x7f9e941a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e94107660 0x7f9e941a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 -- 172.21.15.90:0/113123766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e94102fc0 msgr2=0x7f9e941a08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e94102fc0 0x7f9e941a08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.152+0000 7f9e92ffd700 1 -- 172.21.15.90:0/113123766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e941a5100 con 0x7f9e94102610 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.153+0000 7f9e92ffd700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941a03a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b580 tx=0x7f9e840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.153+0000 7f9e9895e700 1 -- 172.21.15.90:0/113123766 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84027070 con 0x7f9e94102610 2024-04-26T18:11:01.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.153+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e941a5390 con 0x7f9e94102610 2024-04-26T18:11:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.153+0000 7f9e9895e700 1 -- 172.21.15.90:0/113123766 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e84003f20 con 0x7f9e94102610 2024-04-26T18:11:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.153+0000 7f9e9895e700 1 -- 172.21.15.90:0/113123766 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e84022d10 con 0x7f9e94102610 2024-04-26T18:11:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.153+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e941acc70 con 0x7f9e94102610 2024-04-26T18:11:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.154+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e9406a110 con 0x7f9e94102610 2024-04-26T18:11:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.155+0000 7f9e9895e700 1 -- 172.21.15.90:0/113123766 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9e84022470 con 0x7f9e94102610 2024-04-26T18:11:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.155+0000 7f9e9895e700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9e7c0a3c40 0x7f9e7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.156+0000 7f9e927fc700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9e7c0a3c40 0x7f9e7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.156+0000 7f9e9895e700 1 -- 172.21.15.90:0/113123766 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f9e840d0f50 con 0x7f9e94102610 2024-04-26T18:11:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.156+0000 7f9e927fc700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9e7c0a3c40 0x7f9e7c0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e94100da0 tx=0x7f9e8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:01.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.160+0000 7f9e9895e700 1 -- 172.21.15.90:0/113123766 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e84013070 con 0x7f9e94102610 2024-04-26T18:11:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:01 smithi119 ceph-mon[32752]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:01 smithi167 ceph-mon[32751]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:01.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.398+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e941a1450 con 0x7f9e94102610 2024-04-26T18:11:01.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.400+0000 7f9e9895e700 1 -- 172.21.15.90:0/113123766 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22557 (secure 0 0 0) 0x7f9e8409da60 con 0x7f9e94102610 2024-04-26T18:11:01.400 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:01.400 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":92,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:09:33.002079+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:11:01.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9e7c0a3c40 msgr2=0x7f9e7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9e7c0a3c40 0x7f9e7c0a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9e94100da0 tx=0x7f9e8800b410 comp rx=0 tx=0).stop 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 msgr2=0x7f9e941a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941a03a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9e8400b580 tx=0x7f9e840047b0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 shutdown_connections 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9e7c0a3c40 0x7f9e7c0a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9e94102610 0x7f9e941a03a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9e94107660 0x7f9e941a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 --2- 172.21.15.90:0/113123766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e94102fc0 0x7f9e941a08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.402+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 >> 172.21.15.90:0/113123766 conn(0x7f9e94078370 msgr2=0x7f9e94108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.403+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 shutdown_connections 2024-04-26T18:11:01.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:01.403+0000 7f9e99960700 1 -- 172.21.15.90:0/113123766 wait complete. 2024-04-26T18:11:02.060 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:11:02.342 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:02 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/113123766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:02 smithi090 ceph-mon[29180]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:02 smithi090 ceph-mon[29180]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:02 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/113123766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:11:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:02 smithi119 ceph-mon[32752]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:11:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:11:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:02 smithi119 ceph-mon[32752]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:02 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/113123766' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:11:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:02 smithi167 ceph-mon[32751]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:11:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:11:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:02 smithi167 ceph-mon[32751]: from='mgr.14152 172.21.15.90:0/2653381309' entity='mgr.x' 2024-04-26T18:11:03.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 -- 172.21.15.90:0/3109584777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 msgr2=0x7f4c00106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:03.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 --2- 172.21.15.90:0/3109584777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00106f60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0099c0 tx=0x7f4bec009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:03.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 -- 172.21.15.90:0/3109584777 shutdown_connections 2024-04-26T18:11:03.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 --2- 172.21.15.90:0/3109584777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c00107640 0x7f4c0010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 --2- 172.21.15.90:0/3109584777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c001025b0 0x7f4c00102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 --2- 172.21.15.90:0/3109584777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00106f60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 -- 172.21.15.90:0/3109584777 >> 172.21.15.90:0/3109584777 conn(0x7f4c000792f0 msgr2=0x7f4c00079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 -- 172.21.15.90:0/3109584777 shutdown_connections 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.946+0000 7f4c07063700 1 -- 172.21.15.90:0/3109584777 wait complete. 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 Processor -- start 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 -- start start 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c001025b0 0x7f4c00104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c00107640 0x7f4c00104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4c00112530 con 0x7f4c00102f60 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4c001123b0 con 0x7f4c00107640 2024-04-26T18:11:03.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4c07063700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c001126b0 con 0x7f4c001025b0 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4bfffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4bfffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56522/0 (socket says 172.21.15.90:56522) 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.947+0000 7f4bfffff700 1 -- 172.21.15.90:0/1732256755 learned_addr learned my addr 172.21.15.90:0/1732256755 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4c05600700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c00107640 0x7f4c00104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4c04dff700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c001025b0 0x7f4c00104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfffff700 1 -- 172.21.15.90:0/1732256755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c001025b0 msgr2=0x7f4c00104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfffff700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c001025b0 0x7f4c00104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfffff700 1 -- 172.21.15.90:0/1732256755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c00107640 msgr2=0x7f4c00104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfffff700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c00107640 0x7f4c00104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfffff700 1 -- 172.21.15.90:0/1732256755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c001a80c0 con 0x7f4c00102f60 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4c04dff700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c001025b0 0x7f4c00104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:03.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4c05600700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c00107640 0x7f4c00104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:03.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfffff700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00104540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009f50 tx=0x7f4bec0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:03.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfdffb700 1 -- 172.21.15.90:0/1732256755 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec027070 con 0x7f4c00102f60 2024-04-26T18:11:03.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c001a8350 con 0x7f4c00102f60 2024-04-26T18:11:03.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfdffb700 1 -- 172.21.15.90:0/1732256755 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bec004140 con 0x7f4c00102f60 2024-04-26T18:11:03.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4bfdffb700 1 -- 172.21.15.90:0/1732256755 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00f460 con 0x7f4c00102f60 2024-04-26T18:11:03.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.948+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c001a8800 con 0x7f4c00102f60 2024-04-26T18:11:03.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.949+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c0006a110 con 0x7f4c00102f60 2024-04-26T18:11:03.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.951+0000 7f4bfdffb700 1 -- 172.21.15.90:0/1732256755 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4bec02a070 con 0x7f4c00102f60 2024-04-26T18:11:03.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.952+0000 7f4bfdffb700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4bf00a3c40 0x7f4bf00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:03.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.952+0000 7f4c04dff700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4bf00a3c40 0x7f4bf00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:03.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.952+0000 7f4bfdffb700 1 -- 172.21.15.90:0/1732256755 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f4bec0d1780 con 0x7f4c00102f60 2024-04-26T18:11:03.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.952+0000 7f4c04dff700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4bf00a3c40 0x7f4bf00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4005fd0 tx=0x7f4bf4005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:03.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:03.955+0000 7f4bfdffb700 1 -- 172.21.15.90:0/1732256755 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bec031070 con 0x7f4c00102f60 2024-04-26T18:11:04.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.204+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c0002e490 con 0x7f4c00102f60 2024-04-26T18:11:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.204+0000 7f4bfdffb700 1 -- 172.21.15.90:0/1732256755 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11874 (secure 0 0 0) 0x7f4bec013070 con 0x7f4c00102f60 2024-04-26T18:11:04.208 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:04.210 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:09:06.760287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24448,24449,24436]},"id":1}]} 2024-04-26T18:11:04.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4bf00a3c40 msgr2=0x7f4bf00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:04.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4bf00a3c40 0x7f4bf00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4005fd0 tx=0x7f4bf4005dc0 comp rx=0 tx=0).stop 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 msgr2=0x7f4c00104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00104540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009f50 tx=0x7f4bec0037c0 comp rx=0 tx=0).stop 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 shutdown_connections 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f4bf00a3c40 0x7f4bf00a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c00107640 0x7f4c00104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c001025b0 0x7f4c00104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 --2- 172.21.15.90:0/1732256755 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c00102f60 0x7f4c00104540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:04.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 >> 172.21.15.90:0/1732256755 conn(0x7f4c000792f0 msgr2=0x7f4c0006be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:04.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 shutdown_connections 2024-04-26T18:11:04.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:04.207+0000 7f4c07063700 1 -- 172.21.15.90:0/1732256755 wait complete. 2024-04-26T18:11:04.216 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T18:11:04.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:08:49.029287+0000', 'modified': '2024-04-26T18:09:06.760287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24449, 'mds_1': 24448, 'mds_2': 24436}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/766230459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 766230459}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 766230459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/3534880230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 3534880230}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 3534880230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/570376817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 570376817}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 570376817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.167:6837/200145753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 200145753}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 200145753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.167:6839/4229007587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 4229007587}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 4229007587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.119:6839/1002360318', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1002360318}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1002360318}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24449, 'qdb_cluster': [24448, 24449, 24436]} 2024-04-26T18:11:04.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T18:11:04.881 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T18:11:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:04 smithi090 ceph-mon[29180]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:04 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1732256755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:05.135 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:04 smithi119 ceph-mon[32752]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:04 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1732256755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:04 smithi167 ceph-mon[32751]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:04 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1732256755' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:06.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 -- 172.21.15.90:0/262760175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc06bb50 msgr2=0x7fefdc06bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 --2- 172.21.15.90:0/262760175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc06bb50 0x7fefdc06bfb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fefc80099c0 tx=0x7fefc8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 -- 172.21.15.90:0/262760175 shutdown_connections 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 --2- 172.21.15.90:0/262760175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefdc06c4f0 0x7fefdc1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 --2- 172.21.15.90:0/262760175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc1094a0 0x7fefdc109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 --2- 172.21.15.90:0/262760175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc06bb50 0x7fefdc06bfb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 -- 172.21.15.90:0/262760175 >> 172.21.15.90:0/262760175 conn(0x7fefdc0792c0 msgr2=0x7fefdc0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 -- 172.21.15.90:0/262760175 shutdown_connections 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 -- 172.21.15.90:0/262760175 wait complete. 2024-04-26T18:11:06.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 Processor -- start 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.655+0000 7fefe2f71700 1 -- start start 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe2f71700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc06bb50 0x7fefdc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe2f71700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefdc06c4f0 0x7fefdc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe2f71700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc1094a0 0x7fefdc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe2f71700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc1135b0 con 0x7fefdc1094a0 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe2f71700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc113430 con 0x7fefdc06c4f0 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe2f71700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc113730 con 0x7fefdc06bb50 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc1094a0 0x7fefdc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc1094a0 0x7fefdc1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56540/0 (socket says 172.21.15.90:56540) 2024-04-26T18:11:06.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 -- 172.21.15.90:0/4011562012 learned_addr learned my addr 172.21.15.90:0/4011562012 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefdbfff700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefdc06c4f0 0x7fefdc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc06bb50 0x7fefdc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 -- 172.21.15.90:0/4011562012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc06bb50 msgr2=0x7fefdc19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc06bb50 0x7fefdc19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 -- 172.21.15.90:0/4011562012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefdc06c4f0 msgr2=0x7fefdc19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefdc06c4f0 0x7fefdc19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 -- 172.21.15.90:0/4011562012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc19a140 con 0x7fefdc1094a0 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefdbfff700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefdc06c4f0 0x7fefdc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe150e700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc1094a0 0x7fefdc1999e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fefd4007ee0 tx=0x7fefd400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefd9ffb700 1 -- 172.21.15.90:0/4011562012 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd400dbd0 con 0x7fefdc1094a0 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc19a430 con 0x7fefdc1094a0 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefd9ffb700 1 -- 172.21.15.90:0/4011562012 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd400dd70 con 0x7fefdc1094a0 2024-04-26T18:11:06.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.656+0000 7fefd9ffb700 1 -- 172.21.15.90:0/4011562012 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd4005990 con 0x7fefdc1094a0 2024-04-26T18:11:06.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.657+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc1a66f0 con 0x7fefdc1094a0 2024-04-26T18:11:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.659+0000 7fefd9ffb700 1 -- 172.21.15.90:0/4011562012 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fefd401a020 con 0x7fefdc1094a0 2024-04-26T18:11:06.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.659+0000 7fefd9ffb700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefcc0a3c40 0x7fefcc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.659+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefcc0a3c40 0x7fefcc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.659+0000 7fefd9ffb700 1 -- 172.21.15.90:0/4011562012 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fefd40cfbc0 con 0x7fefdc1094a0 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.660+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefcc0a3c40 0x7fefcc0a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fefdc19ab20 tx=0x7fefd0005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.660+0000 7fefed803b80 1 Processor -- start 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.660+0000 7fefed803b80 1 -- start start 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.660+0000 7fefed803b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace14eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefed803b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace153f0 0x5591ace0ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefed803b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 0x5591ace0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefed803b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5591acd616f0 con 0x5591ace14ad0 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefed803b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5591acd00590 con 0x5591ace153f0 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefed803b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5591acd5d6b0 con 0x5591ace0f2d0 2024-04-26T18:11:06.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace14eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace14eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56546/0 (socket says 172.21.15.90:56546) 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 -- 172.21.15.90:0/2714482624 learned_addr learned my addr 172.21.15.90:0/2714482624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefdbfff700 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace153f0 0x5591ace0ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 -- 172.21.15.90:0/2714482624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 msgr2=0x5591ace0f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 0x5591ace0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 -- 172.21.15.90:0/2714482624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace153f0 msgr2=0x5591ace0ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace153f0 0x5591ace0ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 -- 172.21.15.90:0/2714482624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591accae880 con 0x5591ace14ad0 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace14eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fefd00043e0 tx=0x7fefd0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19179a0b54a6c537 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefc6ffd700 1 -- 172.21.15.90:0/2714482624 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0024070 con 0x5591ace14ad0 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefc6ffd700 1 -- 172.21.15.90:0/2714482624 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd001a050 con 0x5591ace14ad0 2024-04-26T18:11:06.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.661+0000 7fefc6ffd700 1 -- 172.21.15.90:0/2714482624 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd0011970 con 0x5591ace14ad0 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 -- 172.21.15.90:0/2714482624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 msgr2=0x5591ace14eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace14eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fefd00043e0 tx=0x7fefd0011000 comp rx=0 tx=0).stop 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 -- 172.21.15.90:0/2714482624 shutdown_connections 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace14eb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 0x5591ace0f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 --2- 172.21.15.90:0/2714482624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace153f0 0x5591ace0ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 -- 172.21.15.90:0/2714482624 >> 172.21.15.90:0/2714482624 conn(0x5591acd3bcf0 msgr2=0x5591ace0a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 -- 172.21.15.90:0/2714482624 shutdown_connections 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 -- 172.21.15.90:0/2714482624 wait complete. 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 -- start start 2024-04-26T18:11:06.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.662+0000 7fefed803b80 1 Processor -- start 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefed803b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 0x5591ace1b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefed803b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace14ad0 0x5591ace184a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefed803b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace153f0 0x5591ace189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefed803b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5591ace1bf10 con 0x5591ace153f0 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefed803b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5591accc7900 con 0x5591ace14ad0 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefed803b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5591ace1c090 con 0x5591ace0f2d0 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefed803b80 10 client.0 _ll_register_callbacks cb 0x5591accf5530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace153f0 0x5591ace189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace153f0 0x5591ace189e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56548/0 (socket says 172.21.15.90:56548) 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 -- 172.21.15.90:0/882834783 learned_addr learned my addr 172.21.15.90:0/882834783 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:06.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefdbfff700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace14ad0 0x5591ace184a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 0x5591ace1b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 -- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 msgr2=0x5591ace1b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 0x5591ace1b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 -- 172.21.15.90:0/882834783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace14ad0 msgr2=0x5591ace184a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace14ad0 0x5591ace184a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 -- 172.21.15.90:0/882834783 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ace19020 con 0x5591ace153f0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefdbfff700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace14ad0 0x5591ace184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefe150e700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace153f0 0x5591ace189e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fefd40983a0 tx=0x7fefd409c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d378e5a7ffbe65 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.663+0000 7fefbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefbbfff700 1 -- 172.21.15.90:0/882834783 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40b5060 con 0x5591ace153f0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefbbfff700 1 -- 172.21.15.90:0/882834783 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fefd409c3a0 con 0x5591ace153f0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefd001a050 con 0x5591ace153f0 2024-04-26T18:11:06.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefbbfff700 1 -- 172.21.15.90:0/882834783 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefd40a4bb0 con 0x5591ace153f0 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 10 client.14937 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5591ace26a00 con 0x5591ace153f0 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefbbfff700 1 -- 172.21.15.90:0/882834783 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefd001a050 con 0x5591ace153f0 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefbbfff700 1 -- 172.21.15.90:0/882834783 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fefd40cdb80 con 0x5591ace153f0 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 10 client.14937 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 10 client.14937 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 20 client.14937 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace28f90 0x5591ace2a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 4 client.14937 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 --> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591acd6f0d0 con 0x5591ace28f90 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.664+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace28f90 0x5591ace2a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.665+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace28f90 0x5591ace2a400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=120b595505fea5bb server_cookie=5168c9db2796c468 in_seq=0 out_seq=0 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.665+0000 7fefbbfff700 10 client.14937 ms_handle_connect on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefbbfff700 1 -- 172.21.15.90:0/882834783 <== mds.2 v2:172.21.15.167:6836/200145753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591acd6f0d0 con 0x5591ace28f90 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefbbfff700 10 client.14937 handle_command_reply: tid=0 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 client.14937 shutdown 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace153f0 msgr2=0x5591ace189e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace153f0 0x5591ace189e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fefd40983a0 tx=0x7fefd409c770 comp rx=0 tx=0).stop 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 shutdown_connections 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace28f90 0x5591ace2a400 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace153f0 0x5591ace189e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace0f2d0 0x5591ace1b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 --2- 172.21.15.90:0/882834783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace14ad0 0x5591ace184a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 >> 172.21.15.90:0/882834783 conn(0x5591acd3bcf0 msgr2=0x5591ace09da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefbbfff700 0 client.14937 ms_handle_reset on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 shutdown_connections 2024-04-26T18:11:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 1 -- 172.21.15.90:0/882834783 wait complete. 2024-04-26T18:11:06.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.666+0000 7fefed803b80 20 client.14937 trim_cache size 0 max 16384 2024-04-26T18:11:06.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 Processor -- start 2024-04-26T18:11:06.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 -- start start 2024-04-26T18:11:06.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace0ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace09c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 0x5591ace12620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5591ace28740 con 0x5591ace14ad0 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5591ace28e10 con 0x5591ace0f2d0 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.686+0000 7fefed803b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5591ace294e0 con 0x5591ace153f0 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace09c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace09c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56558/0 (socket says 172.21.15.90:56558) 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 -- 172.21.15.90:0/2405321943 learned_addr learned my addr 172.21.15.90:0/2405321943 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace0ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 -- 172.21.15.90:0/2405321943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 msgr2=0x5591ace12620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 0x5591ace12620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 -- 172.21.15.90:0/2405321943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 msgr2=0x5591ace0ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace0ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 -- 172.21.15.90:0/2405321943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591ace0a6b0 con 0x5591ace14ad0 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace0ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefdbfff700 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace09c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fefc80099c0 tx=0x7fefc8005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe2fd0b371d79dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefc67fc700 1 -- 172.21.15.90:0/2405321943 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8027020 con 0x5591ace14ad0 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefc67fc700 1 -- 172.21.15.90:0/2405321943 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefc8005070 con 0x5591ace14ad0 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.687+0000 7fefc67fc700 1 -- 172.21.15.90:0/2405321943 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc800ba80 con 0x5591ace14ad0 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 -- 172.21.15.90:0/2405321943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 msgr2=0x5591ace09c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace09c90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fefc80099c0 tx=0x7fefc8005960 comp rx=0 tx=0).stop 2024-04-26T18:11:06.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 -- 172.21.15.90:0/2405321943 shutdown_connections 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 0x5591ace12620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace0ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 --2- 172.21.15.90:0/2405321943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace09c90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 -- 172.21.15.90:0/2405321943 >> 172.21.15.90:0/2405321943 conn(0x5591acd3bcf0 msgr2=0x5591ace1ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 -- 172.21.15.90:0/2405321943 shutdown_connections 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 -- 172.21.15.90:0/2405321943 wait complete. 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.688+0000 7fefed803b80 1 -- start start 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 1 Processor -- start 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace24380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace248c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 0x5591ace24e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5591ace2a280 con 0x5591ace14ad0 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5591ace29bb0 con 0x5591ace0f2d0 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5591ace2a950 con 0x5591ace153f0 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefed803b80 10 client.0 _ll_register_callbacks cb 0x5591accf5530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace248c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace248c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56562/0 (socket says 172.21.15.90:56562) 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 -- 172.21.15.90:0/932139110 learned_addr learned my addr 172.21.15.90:0/932139110 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:06.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace24380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefe150e700 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 0x5591ace24e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 -- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 msgr2=0x5591ace24e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 0x5591ace24e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 -- 172.21.15.90:0/932139110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 msgr2=0x5591ace24380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace24380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.689+0000 7fefdbfff700 1 -- 172.21.15.90:0/932139110 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591ace930f0 con 0x5591ace14ad0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefdbfff700 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace248c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662522270737854b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefbbfff700 1 -- 172.21.15.90:0/932139110 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8034070 con 0x5591ace14ad0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefbbfff700 1 -- 172.21.15.90:0/932139110 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fefc800b990 con 0x5591ace14ad0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefc8005070 con 0x5591ace14ad0 2024-04-26T18:11:06.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefbbfff700 1 -- 172.21.15.90:0/932139110 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fefc803f050 con 0x5591ace14ad0 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefed803b80 10 client.14943 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5591ace94160 con 0x5591ace14ad0 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefbbfff700 1 -- 172.21.15.90:0/932139110 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefc8005070 con 0x5591ace14ad0 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefbbfff700 1 -- 172.21.15.90:0/932139110 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fefc800f9a0 con 0x5591ace14ad0 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefed803b80 10 client.14943 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefed803b80 10 client.14943 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefed803b80 20 client.14943 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.690+0000 7fefed803b80 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace987d0 0x5591ace974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:06.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefed803b80 4 client.14943 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 --> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] -- command(tid 0: {"prefix": "status"}) -- 0x5591acd69170 con 0x5591ace987d0 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace987d0 0x5591ace974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefbbfff700 10 client.14943 ms_handle_connect on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefe0d0d700 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace987d0 0x5591ace974e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8e50cb0a7b70903 server_cookie=73885adcabead7fd in_seq=0 out_seq=0 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefbbfff700 1 -- 172.21.15.90:0/932139110 <== mds.2 v2:172.21.15.167:6836/200145753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5591acd69170 con 0x5591ace987d0 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefbbfff700 10 client.14943 handle_command_reply: tid=0 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.691+0000 7fefed803b80 1 client.14943 shutdown 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 msgr2=0x5591ace248c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace248c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fefc8000c00 tx=0x7fefc8003a90 comp rx=0 tx=0).stop 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 shutdown_connections 2024-04-26T18:11:06.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x5591ace987d0 0x5591ace974e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5591ace153f0 0x5591ace24e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5591ace0f2d0 0x5591ace24380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefbbfff700 0 client.14943 ms_handle_reset on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 --2- 172.21.15.90:0/932139110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5591ace14ad0 0x5591ace248c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 >> 172.21.15.90:0/932139110 conn(0x5591acd3bcf0 msgr2=0x5591ace09be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 shutdown_connections 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 1 -- 172.21.15.90:0/932139110 wait complete. 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.692+0000 7fefed803b80 20 client.14943 trim_cache size 0 max 16384 2024-04-26T18:11:06.700 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24436, 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 124.75847312400001, 2024-04-26T18:11:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-04-26T18:11:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 92, 2024-04-26T18:11:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T18:11:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 167.579636247 2024-04-26T18:11:06.702 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefcc0a3c40 msgr2=0x7fefcc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefcc0a3c40 0x7fefcc0a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fefdc19ab20 tx=0x7fefd0005ee0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc1094a0 msgr2=0x7fefdc1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc1094a0 0x7fefdc1999e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fefd4007ee0 tx=0x7fefd400d3b0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 shutdown_connections 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fefcc0a3c40 0x7fefcc0a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fefdc1094a0 0x7fefdc1999e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fefdc06bb50 0x7fefdc19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 --2- 172.21.15.90:0/4011562012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefdc06c4f0 0x7fefdc19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 >> 172.21.15.90:0/4011562012 conn(0x7fefdc0792c0 msgr2=0x7fefdc100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 shutdown_connections 2024-04-26T18:11:06.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:06.693+0000 7fefe2f71700 1 -- 172.21.15.90:0/4011562012 wait complete. 2024-04-26T18:11:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:06 smithi119 ceph-mon[32752]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:06 smithi167 ceph-mon[32751]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:07.318 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:06 smithi090 ceph-mon[29180]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:07.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:11:07.624 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:09.214 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:08 smithi090 ceph-mon[29180]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:09.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.223+0000 7fae8b671700 1 -- 172.21.15.90:0/574402366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 msgr2=0x7fae84109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.223+0000 7fae8b671700 1 --2- 172.21.15.90:0/574402366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84109860 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fae80009990 tx=0x7fae80009ca0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- 172.21.15.90:0/574402366 shutdown_connections 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 --2- 172.21.15.90:0/574402366 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84109860 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 --2- 172.21.15.90:0/574402366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae84103ea0 0x7fae841102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 --2- 172.21.15.90:0/574402366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae841034e0 0x7fae84103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- 172.21.15.90:0/574402366 >> 172.21.15.90:0/574402366 conn(0x7fae8407a5e0 msgr2=0x7fae8407a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- 172.21.15.90:0/574402366 shutdown_connections 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- 172.21.15.90:0/574402366 wait complete. 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 Processor -- start 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- start start 2024-04-26T18:11:09.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae841034e0 0x7fae8419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae84103ea0 0x7fae8419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fae84113690 con 0x7fae84109480 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fae84113510 con 0x7fae84103ea0 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.224+0000 7fae8b671700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae84113810 con 0x7fae841034e0 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56584/0 (socket says 172.21.15.90:56584) 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 -- 172.21.15.90:0/65651804 learned_addr learned my addr 172.21.15.90:0/65651804 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae8940d700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae841034e0 0x7fae8419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae88c0c700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae84103ea0 0x7fae8419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 -- 172.21.15.90:0/65651804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae841034e0 msgr2=0x7fae8419f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae841034e0 0x7fae8419f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 -- 172.21.15.90:0/65651804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae84103ea0 msgr2=0x7fae8419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae84103ea0 0x7fae8419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 -- 172.21.15.90:0/65651804 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8419a210 con 0x7fae84109480 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae88c0c700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae84103ea0 0x7fae8419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae89c0e700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84199ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b970 tx=0x7fae7c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae7a7fc700 1 -- 172.21.15.90:0/65651804 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c019070 con 0x7fae84109480 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae7a7fc700 1 -- 172.21.15.90:0/65651804 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c00cde0 con 0x7fae84109480 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8419a500 con 0x7fae84109480 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae7a7fc700 1 -- 172.21.15.90:0/65651804 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c007370 con 0x7fae84109480 2024-04-26T18:11:09.228 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.225+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae841a67c0 con 0x7fae84109480 2024-04-26T18:11:09.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.227+0000 7fae7a7fc700 1 -- 172.21.15.90:0/65651804 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fae7c01c070 con 0x7fae84109480 2024-04-26T18:11:09.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.228+0000 7fae7a7fc700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fae700a3c40 0x7fae700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.228+0000 7fae8940d700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fae700a3c40 0x7fae700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.228+0000 7fae7a7fc700 1 -- 172.21.15.90:0/65651804 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fae7c0cf820 con 0x7fae84109480 2024-04-26T18:11:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.228+0000 7fae8940d700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fae700a3c40 0x7fae700a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae8000b5c0 tx=0x7fae80005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:09.234 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 Processor -- start 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 -- start start 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d84853ad0 0x564d84853eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d8484dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d8484e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564d847a06f0 con 0x564d84853ad0 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564d8473f590 con 0x564d848543f0 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae95f03b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d8479c6b0 con 0x564d8484e2d0 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae8940d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d84853ad0 0x564d84853eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae8940d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d84853ad0 0x564d84853eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56586/0 (socket says 172.21.15.90:56586) 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.229+0000 7fae8940d700 1 -- 172.21.15.90:0/3440295209 learned_addr learned my addr 172.21.15.90:0/3440295209 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae88c0c700 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d8484dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae89c0e700 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d8484e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae8940d700 1 -- 172.21.15.90:0/3440295209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 msgr2=0x564d8484e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae8940d700 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d8484e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae8940d700 1 -- 172.21.15.90:0/3440295209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 msgr2=0x564d8484dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae8940d700 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d8484dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae8940d700 1 -- 172.21.15.90:0/3440295209 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d846ed880 con 0x564d84853ad0 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae88c0c700 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d8484dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae89c0e700 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d8484e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae8940d700 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d84853ad0 0x564d84853eb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fae80009990 tx=0x7fae80020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531bae2d3022bbda server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:09.236 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae6f7fe700 1 -- 172.21.15.90:0/3440295209 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8000f840 con 0x564d84853ad0 2024-04-26T18:11:09.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae6f7fe700 1 -- 172.21.15.90:0/3440295209 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae80024410 con 0x564d84853ad0 2024-04-26T18:11:09.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.230+0000 7fae6f7fe700 1 -- 172.21.15.90:0/3440295209 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8002f640 con 0x564d84853ad0 2024-04-26T18:11:09.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 -- 172.21.15.90:0/3440295209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d84853ad0 msgr2=0x564d84853eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d84853ad0 0x564d84853eb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fae80009990 tx=0x7fae80020000 comp rx=0 tx=0).stop 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 -- 172.21.15.90:0/3440295209 shutdown_connections 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d84853ad0 0x564d84853eb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d8484e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 --2- 172.21.15.90:0/3440295209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d8484dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 -- 172.21.15.90:0/3440295209 >> 172.21.15.90:0/3440295209 conn(0x564d8477acf0 msgr2=0x564d848491b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 -- 172.21.15.90:0/3440295209 shutdown_connections 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 -- 172.21.15.90:0/3440295209 wait complete. 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 -- start start 2024-04-26T18:11:09.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.231+0000 7fae95f03b80 1 Processor -- start 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d8484e2d0 0x564d8485a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 0x564d848573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d84857930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564d8485ae50 con 0x564d848543f0 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564d84706900 con 0x564d8484e2d0 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d8485afd0 con 0x564d84853ad0 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 10 client.0 _ll_register_callbacks cb 0x564d84734530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d84857930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae8940d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d8484e2d0 0x564d8485a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d84857930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56596/0 (socket says 172.21.15.90:56596) 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 -- 172.21.15.90:0/1664751471 learned_addr learned my addr 172.21.15.90:0/1664751471 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae88c0c700 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 0x564d848573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 -- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 msgr2=0x564d848573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 0x564d848573f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 -- 172.21.15.90:0/1664751471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d8484e2d0 msgr2=0x564d8485a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d8484e2d0 0x564d8485a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 -- 172.21.15.90:0/1664751471 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d84857f70 con 0x564d848543f0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae89c0e700 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d84857930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b940 tx=0x7fae7c0bc330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ef62f00ed4b39b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae6cff9700 1 -- 172.21.15.90:0/1664751471 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c09be00 con 0x564d848543f0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae6cff9700 1 -- 172.21.15.90:0/1664751471 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fae7c09bfa0 con 0x564d848543f0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.232+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae80024410 con 0x564d848543f0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae6cff9700 1 -- 172.21.15.90:0/1664751471 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae7c0a4380 con 0x564d848543f0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 10 client.14955 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d84865840 con 0x564d848543f0 2024-04-26T18:11:09.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae6cff9700 1 -- 172.21.15.90:0/1664751471 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae80024410 con 0x564d848543f0 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae6cff9700 1 -- 172.21.15.90:0/1664751471 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fae7c0978f0 con 0x564d848543f0 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 10 client.14955 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 10 client.14955 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 20 client.14955 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d84867e30 0x564d848692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 4 client.14955 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 --> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d847ae0d0 con 0x564d84867e30 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.233+0000 7fae8940d700 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d84867e30 0x564d848692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.234+0000 7fae8940d700 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d84867e30 0x564d848692a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ead65a4e122fc791 server_cookie=ef8e62db1e632 in_seq=0 out_seq=0 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.234+0000 7fae6cff9700 10 client.14955 ms_handle_connect on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.234+0000 7fae6cff9700 1 -- 172.21.15.90:0/1664751471 <== mds.1 v2:172.21.15.167:6838/4229007587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d847ae0d0 con 0x564d84867e30 2024-04-26T18:11:09.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.234+0000 7fae6cff9700 10 client.14955 handle_command_reply: tid=0 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 client.14955 shutdown 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 msgr2=0x564d84857930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d84857930 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b940 tx=0x7fae7c0bc330 comp rx=0 tx=0).stop 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 shutdown_connections 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d84867e30 0x564d848692a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d84857930 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d8484e2d0 0x564d8485a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae6cff9700 0 client.14955 ms_handle_reset on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1664751471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 0x564d848573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 >> 172.21.15.90:0/1664751471 conn(0x564d8477acf0 msgr2=0x564d84848da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 shutdown_connections 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 1 -- 172.21.15.90:0/1664751471 wait complete. 2024-04-26T18:11:09.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.235+0000 7fae95f03b80 20 client.14955 trim_cache size 0 max 16384 2024-04-26T18:11:09.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 Processor -- start 2024-04-26T18:11:09.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 -- start start 2024-04-26T18:11:09.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d8484e2d0 0x564d8484dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 0x564d84867760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d84859d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564d848675e0 con 0x564d8484e2d0 2024-04-26T18:11:09.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564d84867cb0 con 0x564d848543f0 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.255+0000 7fae95f03b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d84868380 con 0x564d84853ad0 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d8484e2d0 0x564d8484dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d8484e2d0 0x564d8484dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56612/0 (socket says 172.21.15.90:56612) 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 -- 172.21.15.90:0/1835760448 learned_addr learned my addr 172.21.15.90:0/1835760448 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 -- 172.21.15.90:0/1835760448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 msgr2=0x564d84867760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 --2- 172.21.15.90:0/1835760448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 0x564d84867760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 -- 172.21.15.90:0/1835760448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 msgr2=0x564d84859d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 --2- 172.21.15.90:0/1835760448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d84859d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 -- 172.21.15.90:0/1835760448 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d84855960 con 0x564d8484e2d0 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae8940d700 1 --2- 172.21.15.90:0/1835760448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d8484e2d0 0x564d8484dcb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae8000bbb0 tx=0x7fae8001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f848dae90e29f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae6effd700 1 -- 172.21.15.90:0/1835760448 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae800249a0 con 0x564d8484e2d0 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae6effd700 1 -- 172.21.15.90:0/1835760448 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae80040070 con 0x564d8484e2d0 2024-04-26T18:11:09.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.256+0000 7fae6effd700 1 -- 172.21.15.90:0/1835760448 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8003b520 con 0x564d8484e2d0 2024-04-26T18:11:09.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- 172.21.15.90:0/1835760448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d8484e2d0 msgr2=0x564d8484dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1835760448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d8484e2d0 0x564d8484dcb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fae8000bbb0 tx=0x7fae8001a040 comp rx=0 tx=0).stop 2024-04-26T18:11:09.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- 172.21.15.90:0/1835760448 shutdown_connections 2024-04-26T18:11:09.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1835760448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d8484e2d0 0x564d8484dcb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1835760448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d848543f0 0x564d84859d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 --2- 172.21.15.90:0/1835760448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d84853ad0 0x564d84867760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- 172.21.15.90:0/1835760448 >> 172.21.15.90:0/1835760448 conn(0x564d8477acf0 msgr2=0x564d84866cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- 172.21.15.90:0/1835760448 shutdown_connections 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- 172.21.15.90:0/1835760448 wait complete. 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- start start 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 Processor -- start 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d84858c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d84853ad0 0x564d84859160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d848596a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564d84869120 con 0x564d848543f0 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564d84868a50 con 0x564d84853ad0 2024-04-26T18:11:09.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.257+0000 7fae95f03b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564d848697f0 con 0x564d8484e2d0 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae95f03b80 10 client.0 _ll_register_callbacks cb 0x564d84734530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d848596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d848596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56618/0 (socket says 172.21.15.90:56618) 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 -- 172.21.15.90:0/2636863658 learned_addr learned my addr 172.21.15.90:0/2636863658 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae8940d700 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d84858c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 -- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 msgr2=0x564d84858c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d84858c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae88c0c700 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d84853ad0 0x564d84859160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 -- 172.21.15.90:0/2636863658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d84853ad0 msgr2=0x564d84859160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d84853ad0 0x564d84859160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 -- 172.21.15.90:0/2636863658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d848719f0 con 0x564d848543f0 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae89c0e700 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d848596a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fae7c09bb30 tx=0x7fae7c09af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113b8002edb4dd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:09.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae6cff9700 1 -- 172.21.15.90:0/2636863658 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c0a4520 con 0x564d848543f0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae6cff9700 1 -- 172.21.15.90:0/2636863658 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fae7c0090a0 con 0x564d848543f0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae80040070 con 0x564d848543f0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae6cff9700 1 -- 172.21.15.90:0/2636863658 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fae7c09be00 con 0x564d848543f0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae95f03b80 10 client.14961 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d84872cd0 con 0x564d848543f0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.258+0000 7fae6cff9700 1 -- 172.21.15.90:0/2636863658 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae80040070 con 0x564d848543f0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae6cff9700 1 -- 172.21.15.90:0/2636863658 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fae7c0a10e0 con 0x564d848543f0 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae95f03b80 10 client.14961 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae95f03b80 10 client.14961 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae95f03b80 20 client.14961 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae95f03b80 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d848d7930 0x564d848d6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:09.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae95f03b80 4 client.14961 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 --> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] -- command(tid 0: {"prefix": "status"}) -- 0x564d847a8170 con 0x564d848d7930 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae8940d700 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d848d7930 0x564d848d6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae6cff9700 10 client.14961 ms_handle_connect on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.259+0000 7fae8940d700 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d848d7930 0x564d848d6640 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=da3b8f538f92576 server_cookie=f8bca983f14b5d6f in_seq=0 out_seq=0 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae6cff9700 1 -- 172.21.15.90:0/2636863658 <== mds.1 v2:172.21.15.167:6838/4229007587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564d847a8170 con 0x564d848d7930 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae6cff9700 10 client.14961 handle_command_reply: tid=0 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 client.14961 shutdown 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 msgr2=0x564d848596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d848596a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fae7c09bb30 tx=0x7fae7c09af40 comp rx=0 tx=0).stop 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 shutdown_connections 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x564d848d7930 0x564d848d6640 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564d848543f0 0x564d848596a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564d8484e2d0 0x564d84858c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae6cff9700 0 client.14961 ms_handle_reset on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 --2- 172.21.15.90:0/2636863658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564d84853ad0 0x564d84859160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 >> 172.21.15.90:0/2636863658 conn(0x564d8477acf0 msgr2=0x564d848573b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 shutdown_connections 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 1 -- 172.21.15.90:0/2636863658 wait complete. 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.260+0000 7fae95f03b80 20 client.14961 trim_cache size 0 max 16384 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-04-26T18:11:09.266 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24448, 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 128.337580683, 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 92, 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 164.49296495300001 2024-04-26T18:11:09.267 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fae700a3c40 msgr2=0x7fae700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fae700a3c40 0x7fae700a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fae8000b5c0 tx=0x7fae80005960 comp rx=0 tx=0).stop 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 msgr2=0x7fae84199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84199ab0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00b970 tx=0x7fae7c00bc80 comp rx=0 tx=0).stop 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 shutdown_connections 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fae700a3c40 0x7fae700a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fae84109480 0x7fae84199ab0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae841034e0 0x7fae8419f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 --2- 172.21.15.90:0/65651804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae84103ea0 0x7fae8419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:09.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 >> 172.21.15.90:0/65651804 conn(0x7fae8407a5e0 msgr2=0x7fae84101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:09.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 shutdown_connections 2024-04-26T18:11:09.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:09.261+0000 7fae8b671700 1 -- 172.21.15.90:0/65651804 wait complete. 2024-04-26T18:11:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:08 smithi119 ceph-mon[32752]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:08 smithi167 ceph-mon[32751]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:09.905 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:11:10.145 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:10 smithi119 ceph-mon[32752]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:10 smithi167 ceph-mon[32751]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:10 smithi090 ceph-mon[29180]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:11.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 -- 172.21.15.90:0/257119613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 msgr2=0x7f40b810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 --2- 172.21.15.90:0/257119613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b810f1d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b0a0 tx=0x7f40b000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 -- 172.21.15.90:0/257119613 shutdown_connections 2024-04-26T18:11:11.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 --2- 172.21.15.90:0/257119613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b810f1d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 --2- 172.21.15.90:0/257119613 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b8108390 0x7f40b8108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 --2- 172.21.15.90:0/257119613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40b8102350 0x7f40b81027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 -- 172.21.15.90:0/257119613 >> 172.21.15.90:0/257119613 conn(0x7f40b80792c0 msgr2=0x7f40b80796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:11.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 -- 172.21.15.90:0/257119613 shutdown_connections 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.945+0000 7f40bff48700 1 -- 172.21.15.90:0/257119613 wait complete. 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 Processor -- start 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 -- start start 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b8102350 0x7f40b819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40b8108390 0x7f40b8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8112540 con 0x7f40b8102cf0 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81123c0 con 0x7f40b8108390 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bff48700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f40b81126c0 con 0x7f40b8102350 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b819fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35940/0 (socket says 172.21.15.90:35940) 2024-04-26T18:11:11.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 -- 172.21.15.90:0/1052042197 learned_addr learned my addr 172.21.15.90:0/1052042197 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40be4e5700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40b8108390 0x7f40b8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bdce4700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b8102350 0x7f40b819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 -- 172.21.15.90:0/1052042197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b8102350 msgr2=0x7f40b819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b8102350 0x7f40b819f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 -- 172.21.15.90:0/1052042197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40b8108390 msgr2=0x7f40b8199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40b8108390 0x7f40b8199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 -- 172.21.15.90:0/1052042197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b819a0d0 con 0x7f40b8102cf0 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40be4e5700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40b8108390 0x7f40b8199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:11.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.946+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b819fcf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f40a800b530 tx=0x7f40a800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:11.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.947+0000 7f40aeffd700 1 -- 172.21.15.90:0/1052042197 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8015020 con 0x7f40b8102cf0 2024-04-26T18:11:11.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.947+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b819a3c0 con 0x7f40b8102cf0 2024-04-26T18:11:11.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.947+0000 7f40aeffd700 1 -- 172.21.15.90:0/1052042197 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8011ce0 con 0x7f40b8102cf0 2024-04-26T18:11:11.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.947+0000 7f40aeffd700 1 -- 172.21.15.90:0/1052042197 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8010d20 con 0x7f40b8102cf0 2024-04-26T18:11:11.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.947+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b81a6680 con 0x7f40b8102cf0 2024-04-26T18:11:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.949+0000 7f40aeffd700 1 -- 172.21.15.90:0/1052042197 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f40a8018020 con 0x7f40b8102cf0 2024-04-26T18:11:11.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.949+0000 7f40aeffd700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f40a40a3c40 0x7f40a40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.949+0000 7f40bdce4700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f40a40a3c40 0x7f40a40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.949+0000 7f40aeffd700 1 -- 172.21.15.90:0/1052042197 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f40a80cfba0 con 0x7f40b8102cf0 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.950+0000 7f40bdce4700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f40a40a3c40 0x7f40a40a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f40b4005950 tx=0x7f40b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.950+0000 7f40ca7dab80 1 Processor -- start 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.950+0000 7f40ca7dab80 1 -- start start 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbc2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbcd50 0x55bafcbbd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbd6f0 0x55bafcbc9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bafcb0f7e0 con 0x55bafcbc2a20 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bafca758e0 con 0x55bafcbbcd50 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bafcb0b7a0 con 0x55bafcbbd6f0 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bdce4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbc2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bdce4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbc2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35944/0 (socket says 172.21.15.90:35944) 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bdce4700 1 -- 172.21.15.90:0/3781166015 learned_addr learned my addr 172.21.15.90:0/3781166015 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbcd50 0x55bafcbbd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40be4e5700 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbd6f0 0x55bafcbc9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bd4e3700 1 -- 172.21.15.90:0/3781166015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbd6f0 msgr2=0x55bafcbc9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbd6f0 0x55bafcbc9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bd4e3700 1 -- 172.21.15.90:0/3781166015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 msgr2=0x55bafcbc2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbc2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bd4e3700 1 -- 172.21.15.90:0/3781166015 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bafca5c880 con 0x55bafcbbcd50 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.951+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbcd50 0x55bafcbbd1b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40a80cd790 tx=0x7f40a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644556d4f77bc7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40a3fff700 1 -- 172.21.15.90:0/3781166015 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80a02d0 con 0x55bafcbbcd50 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40a3fff700 1 -- 172.21.15.90:0/3781166015 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a80a4b90 con 0x55bafcbbcd50 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40a3fff700 1 -- 172.21.15.90:0/3781166015 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a809c3e0 con 0x55bafcbbcd50 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3781166015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbcd50 msgr2=0x55bafcbbd1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbcd50 0x55bafcbbd1b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f40a80cd790 tx=0x7f40a8004750 comp rx=0 tx=0).stop 2024-04-26T18:11:11.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3781166015 shutdown_connections 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbc2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbd6f0 0x55bafcbc9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3781166015 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbcd50 0x55bafcbbd1b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.952+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3781166015 >> 172.21.15.90:0/3781166015 conn(0x55bafcaeb570 msgr2=0x55bafcbb7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3781166015 shutdown_connections 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3781166015 wait complete. 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 -- start start 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 Processor -- start 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbc2a20 0x55bafcbcaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbc9cb0 con 0x55bafcbbd6f0 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bafcac9a30 con 0x55bafcbc2a20 2024-04-26T18:11:11.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.953+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbc9e30 con 0x55bafcbbcd50 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40ca7dab80 10 client.0 _ll_register_callbacks cb 0x55bafcaa7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35956/0 (socket says 172.21.15.90:35956) 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 -- 172.21.15.90:0/984643449 learned_addr learned my addr 172.21.15.90:0/984643449 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40be4e5700 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbc2a20 0x55bafcbcaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bdce4700 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 -- 172.21.15.90:0/984643449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 msgr2=0x55bafcbd0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 -- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbc2a20 msgr2=0x55bafcbcaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbc2a20 0x55bafcbcaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 -- 172.21.15.90:0/984643449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bafcacd430 con 0x55bafcbbd6f0 2024-04-26T18:11:11.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd1030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f40a80988f0 tx=0x7f40a80a02d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b13f890ec0c5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40a17fa700 1 -- 172.21.15.90:0/984643449 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a80b5450 con 0x55bafcbbd6f0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40a17fa700 1 -- 172.21.15.90:0/984643449 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f40a809c180 con 0x55bafcbbd6f0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a80a4b90 con 0x55bafcbbd6f0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40a17fa700 1 -- 172.21.15.90:0/984643449 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40a809c380 con 0x55bafcbbd6f0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40ca7dab80 10 client.14973 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.954+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bafcbc9cb0 con 0x55bafcbbd6f0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40a17fa700 1 -- 172.21.15.90:0/984643449 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a80a4b90 con 0x55bafcbbd6f0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40a17fa700 1 -- 172.21.15.90:0/984643449 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f40a809f030 con 0x55bafcbbd6f0 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40ca7dab80 10 client.14973 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40ca7dab80 10 client.14973 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40ca7dab80 20 client.14973 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:11.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcbda920 0x55bafcbdbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40ca7dab80 4 client.14973 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:11:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.955+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bafcb1d1c0 con 0x55bafcbda920 2024-04-26T18:11:11.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.956+0000 7f40bdce4700 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcbda920 0x55bafcbdbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.956+0000 7f40bdce4700 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcbda920 0x55bafcbdbd90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90fb23b5ebe0d4de server_cookie=1f3a528ddf1ae8bb in_seq=0 out_seq=0 2024-04-26T18:11:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.956+0000 7f40a17fa700 10 client.14973 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40a17fa700 1 -- 172.21.15.90:0/984643449 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bafcb1d1c0 con 0x55bafcbda920 2024-04-26T18:11:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40a17fa700 10 client.14973 handle_command_reply: tid=0 2024-04-26T18:11:11.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 client.14973 shutdown 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbbd6f0 msgr2=0x55bafcbd1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd1030 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f40a80988f0 tx=0x7f40a80a02d0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 shutdown_connections 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcbda920 0x55bafcbdbd90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbc2a20 0x55bafcbcaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40a17fa700 0 client.14973 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/984643449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd1030 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 >> 172.21.15.90:0/984643449 conn(0x55bafcaeb570 msgr2=0x55bafcbc14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 shutdown_connections 2024-04-26T18:11:11.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 1 -- 172.21.15.90:0/984643449 wait complete. 2024-04-26T18:11:11.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.957+0000 7f40ca7dab80 20 client.14973 trim_cache size 0 max 16384 2024-04-26T18:11:11.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 Processor -- start 2024-04-26T18:11:11.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 -- start start 2024-04-26T18:11:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbe51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbe5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbb74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbda0d0 con 0x55bafcbc2a20 2024-04-26T18:11:11.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbda7a0 con 0x55bafcbbd6f0 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbdae70 con 0x55bafcbbcd50 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40be4e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbb74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40be4e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbb74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35962/0 (socket says 172.21.15.90:35962) 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.978+0000 7f40be4e5700 1 -- 172.21.15.90:0/1505340550 learned_addr learned my addr 172.21.15.90:0/1505340550 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40bdce4700 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbe51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40be4e5700 1 -- 172.21.15.90:0/1505340550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 msgr2=0x55bafcbe51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40be4e5700 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbe51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbe5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40be4e5700 1 -- 172.21.15.90:0/1505340550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 msgr2=0x55bafcbe5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40be4e5700 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbe5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40be4e5700 1 -- 172.21.15.90:0/1505340550 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bafcbb2620 con 0x55bafcbc2a20 2024-04-26T18:11:11.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbe5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:11.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40be4e5700 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbb74d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b0a0 tx=0x7f40b00093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67dc7079d37270c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:11.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40a37fe700 1 -- 172.21.15.90:0/1505340550 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0004b50 con 0x55bafcbc2a20 2024-04-26T18:11:11.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40a37fe700 1 -- 172.21.15.90:0/1505340550 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0004cf0 con 0x55bafcbc2a20 2024-04-26T18:11:11.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.979+0000 7f40a37fe700 1 -- 172.21.15.90:0/1505340550 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b00053b0 con 0x55bafcbc2a20 2024-04-26T18:11:11.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 -- 172.21.15.90:0/1505340550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 msgr2=0x55bafcbb74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbb74d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b0a0 tx=0x7f40b00093f0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 -- 172.21.15.90:0/1505340550 shutdown_connections 2024-04-26T18:11:11.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcbb74d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbe51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/1505340550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbe5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 -- 172.21.15.90:0/1505340550 >> 172.21.15.90:0/1505340550 conn(0x55bafcaeb570 msgr2=0x55bafcbd0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 -- 172.21.15.90:0/1505340550 shutdown_connections 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 -- 172.21.15.90:0/1505340550 wait complete. 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 -- start start 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.980+0000 7f40ca7dab80 1 Processor -- start 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40ca7dab80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcc40a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbdbc10 con 0x55bafcbc2a20 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbdb540 con 0x55bafcbbd6f0 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40ca7dab80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bafcbdc2e0 con 0x55bafcbbcd50 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40ca7dab80 10 client.0 _ll_register_callbacks cb 0x55bafcaa7900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcc40a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcc40a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35976/0 (socket says 172.21.15.90:35976) 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 -- 172.21.15.90:0/3688354742 learned_addr learned my addr 172.21.15.90:0/3688354742 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40bdce4700 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40bd4e3700 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 -- 172.21.15.90:0/3688354742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 msgr2=0x55bafcbd2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 -- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 msgr2=0x55bafcbd2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 -- 172.21.15.90:0/3688354742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bafcc40f90 con 0x55bafcbc2a20 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40be4e5700 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcc40a50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f40b0028040 tx=0x7f40b0007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a161d2c735257540 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.981+0000 7f40a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40a17fa700 1 -- 172.21.15.90:0/3688354742 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0035060 con 0x55bafcbc2a20 2024-04-26T18:11:11.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0004cf0 con 0x55bafcbc2a20 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40a17fa700 1 -- 172.21.15.90:0/3688354742 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f40b0023040 con 0x55bafcbc2a20 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40a17fa700 1 -- 172.21.15.90:0/3688354742 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40b0004b50 con 0x55bafcbc2a20 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 10 client.14979 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bafcc42090 con 0x55bafcbc2a20 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40a17fa700 1 -- 172.21.15.90:0/3688354742 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40b0004cf0 con 0x55bafcbc2a20 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40a17fa700 1 -- 172.21.15.90:0/3688354742 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f40b003a070 con 0x55bafcbc2a20 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 10 client.14979 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 10 client.14979 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 20 client.14979 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcc466d0 0x55bafcc453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 4 client.14979 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "status"}) -- 0x55bafcb17260 con 0x55bafcc466d0 2024-04-26T18:11:11.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.982+0000 7f40bdce4700 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcc466d0 0x55bafcc453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40bdce4700 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcc466d0 0x55bafcc453e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60b7934de572e85a server_cookie=335dc641558f8b6c in_seq=0 out_seq=0 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40a17fa700 10 client.14979 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40a17fa700 1 -- 172.21.15.90:0/3688354742 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55bafcb17260 con 0x55bafcc466d0 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40a17fa700 10 client.14979 handle_command_reply: tid=0 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 client.14979 shutdown 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 msgr2=0x55bafcc40a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcc40a50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f40b0028040 tx=0x7f40b0007e20 comp rx=0 tx=0).stop 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 shutdown_connections 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bafcc466d0 0x55bafcc453e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bafcbc2a20 0x55bafcc40a50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bafcbbcd50 0x55bafcbd2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40a17fa700 0 client.14979 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:11.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 --2- 172.21.15.90:0/3688354742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bafcbbd6f0 0x55bafcbd2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 >> 172.21.15.90:0/3688354742 conn(0x55bafcaeb570 msgr2=0x55bafcbb74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 shutdown_connections 2024-04-26T18:11:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.983+0000 7f40ca7dab80 1 -- 172.21.15.90:0/3688354742 wait complete. 2024-04-26T18:11:11.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40ca7dab80 20 client.14979 trim_cache size 0 max 16384 2024-04-26T18:11:11.990 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:11:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:11:11.990 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24449, 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 142.624341168, 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 92, 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 157.331279123 2024-04-26T18:11:11.991 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:11:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f40a40a3c40 msgr2=0x7f40a40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f40a40a3c40 0x7f40a40a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f40b4005950 tx=0x7f40b40058e0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 msgr2=0x7f40b819fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b819fcf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f40a800b530 tx=0x7f40a800b8f0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 shutdown_connections 2024-04-26T18:11:11.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f40a40a3c40 0x7f40a40a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40b8108390 0x7f40b8199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f40b8102350 0x7f40b819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 --2- 172.21.15.90:0/1052042197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f40b8102cf0 0x7f40b819fcf0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 >> 172.21.15.90:0/1052042197 conn(0x7f40b80792c0 msgr2=0x7f40b81016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 shutdown_connections 2024-04-26T18:11:11.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:11.984+0000 7f40bff48700 1 -- 172.21.15.90:0/1052042197 wait complete. 2024-04-26T18:11:12.700 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T18:11:12.701 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T18:11:12.714 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T18:11:12.715 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T18:11:12.962 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:13.247 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:12 smithi090 ceph-mon[29180]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:12 smithi119 ceph-mon[32752]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:12 smithi167 ceph-mon[32751]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:14.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 -- 172.21.15.90:0/2855192360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103590 msgr2=0x7f25981039f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:14.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 --2- 172.21.15.90:0/2855192360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103590 0x7f25981039f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f258c0099c0 tx=0x7f258c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 -- 172.21.15.90:0/2855192360 shutdown_connections 2024-04-26T18:11:14.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 --2- 172.21.15.90:0/2855192360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f30 0x7f259810a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 --2- 172.21.15.90:0/2855192360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598102390 0x7f25981027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 --2- 172.21.15.90:0/2855192360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103590 0x7f25981039f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 -- 172.21.15.90:0/2855192360 >> 172.21.15.90:0/2855192360 conn(0x7f25980fd950 msgr2=0x7f25980ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 -- 172.21.15.90:0/2855192360 shutdown_connections 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.494+0000 7f259d9df700 1 -- 172.21.15.90:0/2855192360 wait complete. 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 Processor -- start 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 -- start start 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598102390 0x7f2598198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103590 0x7f2598198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103f30 0x7f259819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:14.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f259810dea0 con 0x7f2598103f30 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f259810dd20 con 0x7f2598102390 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f259d9df700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f259810e020 con 0x7f2598103590 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f25977fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103f30 0x7f259819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f25977fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103f30 0x7f259819fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35982/0 (socket says 172.21.15.90:35982) 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f25977fe700 1 -- 172.21.15.90:0/1707117986 learned_addr learned my addr 172.21.15.90:0/1707117986 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f25967fc700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103590 0x7f2598198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.495+0000 7f2596ffd700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598102390 0x7f2598198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f25977fe700 1 -- 172.21.15.90:0/1707117986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103590 msgr2=0x7f2598198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f25977fe700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103590 0x7f2598198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f25977fe700 1 -- 172.21.15.90:0/1707117986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598102390 msgr2=0x7f2598198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:14.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f25977fe700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598102390 0x7f2598198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f25977fe700 1 -- 172.21.15.90:0/1707117986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25981a01f0 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f25977fe700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103f30 0x7f259819fb50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f259000b9a0 tx=0x7f259000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f259c9dd700 1 -- 172.21.15.90:0/1707117986 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259000d850 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259806bbd0 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f259c9dd700 1 -- 172.21.15.90:0/1707117986 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f259000e420 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f259c9dd700 1 -- 172.21.15.90:0/1707117986 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2590017640 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.496+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259806c080 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.497+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259806a110 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.498+0000 7f259c9dd700 1 -- 172.21.15.90:0/1707117986 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f259000d9f0 con 0x7f2598103f30 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.498+0000 7f259c9dd700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:14.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.499+0000 7f2596ffd700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:14.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.499+0000 7f259c9dd700 1 -- 172.21.15.90:0/1707117986 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f25900d0030 con 0x7f2598103f30 2024-04-26T18:11:14.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.499+0000 7f2596ffd700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2588005950 tx=0x7f258800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:14.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.502+0000 7f259c9dd700 1 -- 172.21.15.90:0/1707117986 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2590013030 con 0x7f2598103f30 2024-04-26T18:11:14.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.720+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f259806c340 con 0x7f25800a3c40 2024-04-26T18:11:14.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.721+0000 7f259c9dd700 1 -- 172.21.15.90:0/1707117986 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f259806c340 con 0x7f25800a3c40 2024-04-26T18:11:14.721 INFO:teuthology.orchestra.run.smithi090.stdout:Backend: cephadm 2024-04-26T18:11:14.721 INFO:teuthology.orchestra.run.smithi090.stdout:Available: Yes 2024-04-26T18:11:14.722 INFO:teuthology.orchestra.run.smithi090.stdout:Paused: No 2024-04-26T18:11:14.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 msgr2=0x7f25800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:14.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2588005950 tx=0x7f258800a300 comp rx=0 tx=0).stop 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103f30 msgr2=0x7f259819fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103f30 0x7f259819fb50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f259000b9a0 tx=0x7f259000bcb0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 shutdown_connections 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103f30 0x7f259819fb50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598102390 0x7f2598198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 --2- 172.21.15.90:0/1707117986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103590 0x7f2598198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.723+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 >> 172.21.15.90:0/1707117986 conn(0x7f25980fd950 msgr2=0x7f25981069b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.724+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 shutdown_connections 2024-04-26T18:11:14.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:14.724+0000 7f259d9df700 1 -- 172.21.15.90:0/1707117986 wait complete. 2024-04-26T18:11:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:14 smithi119 ceph-mon[32752]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:15.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:14 smithi090 ceph-mon[29180]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:14 smithi167 ceph-mon[32751]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:15.336 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T18:11:15.575 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:15 smithi119 ceph-mon[32752]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:15 smithi167 ceph-mon[32751]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:15 smithi090 ceph-mon[29180]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:17.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 -- 172.21.15.90:0/3327389115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 msgr2=0x7fb220104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:17.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 --2- 172.21.15.90:0/3327389115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220104b10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb2100099c0 tx=0x7fb210009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 -- 172.21.15.90:0/3327389115 shutdown_connections 2024-04-26T18:11:17.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 --2- 172.21.15.90:0/3327389115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb220105050 0x7fb22010b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 --2- 172.21.15.90:0/3327389115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2201034b0 0x7fb2201038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 --2- 172.21.15.90:0/3327389115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220104b10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 -- 172.21.15.90:0/3327389115 >> 172.21.15.90:0/3327389115 conn(0x7fb2200fea70 msgr2=0x7fb220100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 -- 172.21.15.90:0/3327389115 shutdown_connections 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.147+0000 7fb227ef1700 1 -- 172.21.15.90:0/3327389115 wait complete. 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 Processor -- start 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 -- start start 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2201034b0 0x7fb220199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb220105050 0x7fb2201a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb22010ef50 con 0x7fb2201046b0 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb22010edd0 con 0x7fb2201034b0 2024-04-26T18:11:17.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb227ef1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb22010f0d0 con 0x7fb220105050 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb22548c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb22548c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36000/0 (socket says 172.21.15.90:36000) 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb22548c700 1 -- 172.21.15.90:0/1250096037 learned_addr learned my addr 172.21.15.90:0/1250096037 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb225c8d700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2201034b0 0x7fb220199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.148+0000 7fb22648e700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb220105050 0x7fb2201a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb22548c700 1 -- 172.21.15.90:0/1250096037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb220105050 msgr2=0x7fb2201a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb22548c700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb220105050 0x7fb2201a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb22548c700 1 -- 172.21.15.90:0/1250096037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2201034b0 msgr2=0x7fb220199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb22548c700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2201034b0 0x7fb220199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb22548c700 1 -- 172.21.15.90:0/1250096037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2201a11a0 con 0x7fb2201046b0 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb22548c700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220199ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb210005910 tx=0x7fb210005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb216ffd700 1 -- 172.21.15.90:0/1250096037 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210027070 con 0x7fb2201046b0 2024-04-26T18:11:17.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb216ffd700 1 -- 172.21.15.90:0/1250096037 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb210004020 con 0x7fb2201046b0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb216ffd700 1 -- 172.21.15.90:0/1250096037 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb210022c50 con 0x7fb2201046b0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2201a1430 con 0x7fb2201046b0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.149+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2201a1910 con 0x7fb2201046b0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.150+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22006a110 con 0x7fb2201046b0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.151+0000 7fb216ffd700 1 -- 172.21.15.90:0/1250096037 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb21002a070 con 0x7fb2201046b0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.152+0000 7fb216ffd700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb20c0a3cc0 0x7fb20c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.152+0000 7fb225c8d700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb20c0a3cc0 0x7fb20c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.152+0000 7fb216ffd700 1 -- 172.21.15.90:0/1250096037 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb2100d11d0 con 0x7fb2201046b0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.152+0000 7fb225c8d700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb20c0a3cc0 0x7fb20c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb220104510 tx=0x7fb21c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:17.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.156+0000 7fb216ffd700 1 -- 172.21.15.90:0/1250096037 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb21005eff0 con 0x7fb2201046b0 2024-04-26T18:11:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:16 smithi119 ceph-mon[32752]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:16 smithi167 ceph-mon[32751]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:17.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.365+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fb220063860 con 0x7fb20c0a3cc0 2024-04-26T18:11:17.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:16 smithi090 ceph-mon[29180]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:17.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.379+0000 7fb216ffd700 1 -- 172.21.15.90:0/1250096037 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fb220063860 con 0x7fb20c0a3cc0 2024-04-26T18:11:17.379 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T18:11:17.379 INFO:teuthology.orchestra.run.smithi090.stdout:mds.a smithi090 running (2m) 16s ago 2m 16.9M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 62157d4f8505 2024-04-26T18:11:17.379 INFO:teuthology.orchestra.run.smithi090.stdout:mds.b smithi119 running (2m) 19s ago 2m 26.7M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 98bded1d7c67 2024-04-26T18:11:17.379 INFO:teuthology.orchestra.run.smithi090.stdout:mds.c smithi167 running (2m) 19s ago 2m 22.5M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b a0d324860d23 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.d smithi090 running (3m) 16s ago 3m 14.4M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 3862163a35cf 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.e smithi119 running (3m) 19s ago 3m 14.6M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b a1d026342b5d 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.f smithi167 running (3m) 19s ago 3m 23.3M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b c3d1bb23b8e6 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.g smithi090 running (2m) 16s ago 2m 13.8M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b dba63a34a588 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.h smithi119 running (2m) 19s ago 2m 18.7M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b c1c36f778f9d 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.i smithi167 running (2m) 19s ago 2m 20.0M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 66d7771cec9f 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.j smithi090 running (3m) 16s ago 3m 16.9M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b e984b04ed9cb 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.k smithi119 running (3m) 19s ago 3m 14.0M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b f193fc625c27 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mds.l smithi167 running (3m) 19s ago 3m 19.2M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 02d908f2d3cc 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.x smithi090 *:9283,8765 running (10m) 16s ago 10m 515M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b f79488821171 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.y smithi119 *:8443,8765 running (8m) 19s ago 8m 456M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 3b41d9bc6645 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.z smithi167 *:8443,8765 running (8m) 19s ago 8m 452M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 3dfac15081c2 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mon.a smithi090 running (10m) 16s ago 10m 57.0M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b a34a7841c5bb 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mon.b smithi119 running (8m) 19s ago 8m 41.3M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b e54181de0e94 2024-04-26T18:11:17.380 INFO:teuthology.orchestra.run.smithi090.stdout:mon.c smithi167 running (9m) 19s ago 8m 39.6M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 8dabe27af26b 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 16s ago 8m 61.6M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 19cfb3d80091 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi119 running (7m) 19s ago 7m 60.4M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 1495ac0de4d6 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi167 running (7m) 19s ago 7m 50.7M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b ca637430acbc 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi090 running (7m) 16s ago 7m 47.8M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 9eecd2f8c915 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi119 running (6m) 19s ago 6m 53.4M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 5a53dbb71092 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi167 running (6m) 19s ago 6m 57.3M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b b7fc16d1b30d 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 smithi090 running (5m) 16s ago 5m 54.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b a7d85e8ef1bc 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 smithi119 running (5m) 19s ago 5m 58.7M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 9b22619c6116 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 smithi167 running (4m) 19s ago 4m 47.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 901329db75e7 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 smithi090 running (4m) 16s ago 4m 45.3M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b f1275325c2ac 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 smithi119 running (4m) 19s ago 4m 51.0M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b b67f9a39af76 2024-04-26T18:11:17.381 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 smithi167 running (3m) 19s ago 3m 45.5M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 9e9956c036b2 2024-04-26T18:11:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb20c0a3cc0 msgr2=0x7fb20c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb20c0a3cc0 0x7fb20c0a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb220104510 tx=0x7fb21c00b410 comp rx=0 tx=0).stop 2024-04-26T18:11:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 msgr2=0x7fb220199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220199ad0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb210005910 tx=0x7fb210005b00 comp rx=0 tx=0).stop 2024-04-26T18:11:17.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 shutdown_connections 2024-04-26T18:11:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb20c0a3cc0 0x7fb20c0a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb220105050 0x7fb2201a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2201034b0 0x7fb220199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.381+0000 7fb227ef1700 1 --2- 172.21.15.90:0/1250096037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2201046b0 0x7fb220199ad0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.382+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 >> 172.21.15.90:0/1250096037 conn(0x7fb2200fea70 msgr2=0x7fb2201079b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.382+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 shutdown_connections 2024-04-26T18:11:17.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:17.382+0000 7fb227ef1700 1 -- 172.21.15.90:0/1250096037 wait complete. 2024-04-26T18:11:18.049 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T18:11:18.288 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:19.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:18 smithi090 ceph-mon[29180]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:19.075 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:18 smithi090 ceph-mon[29180]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:18 smithi119 ceph-mon[32752]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:18 smithi119 ceph-mon[32752]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:18 smithi167 ceph-mon[32751]: from='client.14991 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:18 smithi167 ceph-mon[32751]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:19.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 -- 172.21.15.90:0/3922246865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 msgr2=0x7fec18103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 --2- 172.21.15.90:0/3922246865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec18103a00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0099c0 tx=0x7fec0c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 -- 172.21.15.90:0/3922246865 shutdown_connections 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 --2- 172.21.15.90:0/3922246865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec18103f40 0x7fec1810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 --2- 172.21.15.90:0/3922246865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec181023a0 0x7fec181027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 --2- 172.21.15.90:0/3922246865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec18103a00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 -- 172.21.15.90:0/3922246865 >> 172.21.15.90:0/3922246865 conn(0x7fec180fd980 msgr2=0x7fec180ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 -- 172.21.15.90:0/3922246865 shutdown_connections 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.813+0000 7fec1db28700 1 -- 172.21.15.90:0/3922246865 wait complete. 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 Processor -- start 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 -- start start 2024-04-26T18:11:19.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec181023a0 0x7fec1819b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec1819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec18103f40 0x7fec181a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fec1810deb0 con 0x7fec181035a0 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fec1810dd30 con 0x7fec18103f40 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec1db28700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec1810e030 con 0x7fec181023a0 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec1819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec16ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec1819be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36008/0 (socket says 172.21.15.90:36008) 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.814+0000 7fec16ffd700 1 -- 172.21.15.90:0/144093668 learned_addr learned my addr 172.21.15.90:0/144093668 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec177fe700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec181023a0 0x7fec1819b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec17fff700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec18103f40 0x7fec181a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec16ffd700 1 -- 172.21.15.90:0/144093668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec181023a0 msgr2=0x7fec1819b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:19.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec16ffd700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec181023a0 0x7fec1819b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec16ffd700 1 -- 172.21.15.90:0/144093668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec18103f40 msgr2=0x7fec181a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec16ffd700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec18103f40 0x7fec181a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec16ffd700 1 -- 172.21.15.90:0/144093668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec181a3530 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec16ffd700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec1819be10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fec0c005910 tx=0x7fec0c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec14ff9700 1 -- 172.21.15.90:0/144093668 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c027070 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec14ff9700 1 -- 172.21.15.90:0/144093668 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec0c0041a0 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec181a37c0 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec14ff9700 1 -- 172.21.15.90:0/144093668 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c022dd0 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.815+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec181a3ca0 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.816+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1806a110 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.817+0000 7fec14ff9700 1 -- 172.21.15.90:0/144093668 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fec0c02a070 con 0x7fec181035a0 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.818+0000 7fec14ff9700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fec000a3c40 0x7fec000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:19.825 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.818+0000 7fec177fe700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fec000a3c40 0x7fec000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:19.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.818+0000 7fec14ff9700 1 -- 172.21.15.90:0/144093668 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fec0c0d14e0 con 0x7fec181035a0 2024-04-26T18:11:19.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.819+0000 7fec177fe700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fec000a3c40 0x7fec000a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec18103400 tx=0x7fec08009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:19.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:19.822+0000 7fec14ff9700 1 -- 172.21.15.90:0/144093668 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec0c013070 con 0x7fec181035a0 2024-04-26T18:11:20.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.033+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fec180ffba0 con 0x7fec000a3c40 2024-04-26T18:11:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.039+0000 7fec14ff9700 1 -- 172.21.15.90:0/144093668 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fec180ffba0 con 0x7fec000a3c40 2024-04-26T18:11:20.039 INFO:teuthology.orchestra.run.smithi090.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T18:11:20.039 INFO:teuthology.orchestra.run.smithi090.stdout:mds.all 12/12 22s ago 3m smithi090=a;smithi090=d;smithi090=g;smithi090=j;smithi119=b;smithi119=e;smithi119=h;smithi119=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-04-26T18:11:20.040 INFO:teuthology.orchestra.run.smithi090.stdout:mgr 3/3 22s ago 8m smithi090=x;smithi119=y;smithi167=z;count:3 2024-04-26T18:11:20.040 INFO:teuthology.orchestra.run.smithi090.stdout:mon 3/3 22s ago 9m smithi090:172.21.15.90=a;smithi119:172.21.15.119=b;smithi167:172.21.15.167=c;count:3 2024-04-26T18:11:20.040 INFO:teuthology.orchestra.run.smithi090.stdout:osd 12 22s ago - 2024-04-26T18:11:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fec000a3c40 msgr2=0x7fec000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fec000a3c40 0x7fec000a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec18103400 tx=0x7fec08009190 comp rx=0 tx=0).stop 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 msgr2=0x7fec1819be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec1819be10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fec0c005910 tx=0x7fec0c005ef0 comp rx=0 tx=0).stop 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 shutdown_connections 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fec000a3c40 0x7fec000a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec18103f40 0x7fec181a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec181023a0 0x7fec1819b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 --2- 172.21.15.90:0/144093668 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fec181035a0 0x7fec1819be10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.041+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 >> 172.21.15.90:0/144093668 conn(0x7fec180fd980 msgr2=0x7fec180ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.042+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 shutdown_connections 2024-04-26T18:11:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:20.042+0000 7fec1db28700 1 -- 172.21.15.90:0/144093668 wait complete. 2024-04-26T18:11:20.663 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T18:11:20.902 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:21.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:20 smithi090 ceph-mon[29180]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:21.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:20 smithi090 ceph-mon[29180]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:20 smithi119 ceph-mon[32752]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:20 smithi119 ceph-mon[32752]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:20 smithi167 ceph-mon[32751]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:20 smithi167 ceph-mon[32751]: from='client.14997 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:22.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 -- 172.21.15.90:0/3392538594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f050810fa60 msgr2=0x7f0508111e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:22.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 --2- 172.21.15.90:0/3392538594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f050810fa60 0x7f0508111e50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f050000b0a0 tx=0x7f050000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 -- 172.21.15.90:0/3392538594 shutdown_connections 2024-04-26T18:11:22.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 --2- 172.21.15.90:0/3392538594 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f050810fa60 0x7f0508111e50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 --2- 172.21.15.90:0/3392538594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0508077450 0x7f05080758b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 --2- 172.21.15.90:0/3392538594 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0508075e80 0x7f0508076300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 -- 172.21.15.90:0/3392538594 >> 172.21.15.90:0/3392538594 conn(0x7f05080fea20 msgr2=0x7f0508100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 -- 172.21.15.90:0/3392538594 shutdown_connections 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 -- 172.21.15.90:0/3392538594 wait complete. 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 Processor -- start 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.576+0000 7f050dd0e700 1 -- start start 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f050dd0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0508075e80 0x7f050819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f050dd0e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0508077450 0x7f05081a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f050dd0e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f050810fa60 0x7f05081a7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f050dd0e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0508115510 con 0x7f0508075e80 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f050dd0e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0508115390 con 0x7f0508077450 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f050dd0e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0508115690 con 0x7f050810fa60 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0508075e80 0x7f050819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:22.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0508075e80 0x7f050819fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36592/0 (socket says 172.21.15.90:36592) 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 -- 172.21.15.90:0/1766094416 learned_addr learned my addr 172.21.15.90:0/1766094416 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f0507fff700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f050810fa60 0x7f05081a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f0506ffd700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0508077450 0x7f05081a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 -- 172.21.15.90:0/1766094416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f050810fa60 msgr2=0x7f05081a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f050810fa60 0x7f05081a7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 -- 172.21.15.90:0/1766094416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0508077450 msgr2=0x7f05081a0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0508077450 0x7f05081a0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 -- 172.21.15.90:0/1766094416 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05081a77a0 con 0x7f0508075e80 2024-04-26T18:11:22.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f0507fff700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f050810fa60 0x7f05081a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:22.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f05077fe700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0508075e80 0x7f050819fb40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c920 tx=0x7f04fc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:22.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f0504ff9700 1 -- 172.21.15.90:0/1766094416 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc007ac0 con 0x7f0508075e80 2024-04-26T18:11:22.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f0504ff9700 1 -- 172.21.15.90:0/1766094416 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04fc007c60 con 0x7f0508075e80 2024-04-26T18:11:22.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f0504ff9700 1 -- 172.21.15.90:0/1766094416 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04fc018660 con 0x7f0508075e80 2024-04-26T18:11:22.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.577+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05081a7a90 con 0x7f0508075e80 2024-04-26T18:11:22.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.578+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05081a7fd0 con 0x7f0508075e80 2024-04-26T18:11:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.579+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050806a110 con 0x7f0508075e80 2024-04-26T18:11:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.580+0000 7f0504ff9700 1 -- 172.21.15.90:0/1766094416 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f04fc018800 con 0x7f0508075e80 2024-04-26T18:11:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.580+0000 7f0504ff9700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04f00a3c40 0x7f04f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.580+0000 7f0506ffd700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04f00a3c40 0x7f04f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.580+0000 7f0504ff9700 1 -- 172.21.15.90:0/1766094416 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f04fc0d0220 con 0x7f0508075e80 2024-04-26T18:11:22.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.581+0000 7f0506ffd700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04f00a3c40 0x7f04f00a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0508077180 tx=0x7f04f800b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:22.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.585+0000 7f0504ff9700 1 -- 172.21.15.90:0/1766094416 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f04fc014030 con 0x7f0508075e80 2024-04-26T18:11:22.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.791+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0508063860 con 0x7f04f00a3c40 2024-04-26T18:11:22.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.793+0000 7f0504ff9700 1 -- 172.21.15.90:0/1766094416 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0508063860 con 0x7f04f00a3c40 2024-04-26T18:11:22.793 INFO:teuthology.orchestra.run.smithi090.stdout:HOST ADDR LABELS STATUS 2024-04-26T18:11:22.793 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 172.21.15.90 2024-04-26T18:11:22.793 INFO:teuthology.orchestra.run.smithi090.stdout:smithi119 172.21.15.119 2024-04-26T18:11:22.793 INFO:teuthology.orchestra.run.smithi090.stdout:smithi167 172.21.15.167 2024-04-26T18:11:22.793 INFO:teuthology.orchestra.run.smithi090.stdout:3 hosts in cluster 2024-04-26T18:11:22.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04f00a3c40 msgr2=0x7f04f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04f00a3c40 0x7f04f00a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0508077180 tx=0x7f04f800b5b0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0508075e80 msgr2=0x7f050819fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0508075e80 0x7f050819fb40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04fc00c920 tx=0x7f04fc00cce0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 shutdown_connections 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f04f00a3c40 0x7f04f00a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0508075e80 0x7f050819fb40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f050810fa60 0x7f05081a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 --2- 172.21.15.90:0/1766094416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0508077450 0x7f05081a0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.795+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 >> 172.21.15.90:0/1766094416 conn(0x7f05080fea20 msgr2=0x7f0508102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.796+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 shutdown_connections 2024-04-26T18:11:22.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:22.796+0000 7f050dd0e700 1 -- 172.21.15.90:0/1766094416 wait complete. 2024-04-26T18:11:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:22 smithi119 ceph-mon[32752]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:23.308 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:22 smithi090 ceph-mon[29180]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:22 smithi167 ceph-mon[32751]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:23.515 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T18:11:23.760 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:24.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:23 smithi090 ceph-mon[29180]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:23 smithi119 ceph-mon[32752]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:23 smithi167 ceph-mon[32751]: from='client.15003 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:25.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:24 smithi090 ceph-mon[29180]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:24 smithi119 ceph-mon[32752]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:25.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 -- 172.21.15.90:0/1559313482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 msgr2=0x7f7df01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:25.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1559313482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df01027a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0099c0 tx=0x7f7dec009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 -- 172.21.15.90:0/1559313482 shutdown_connections 2024-04-26T18:11:25.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1559313482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df01027a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1559313482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df0103f20 0x7f7df010a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1559313482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df0103580 0x7f7df01039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 -- 172.21.15.90:0/1559313482 >> 172.21.15.90:0/1559313482 conn(0x7f7df00fd920 msgr2=0x7f7df00ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:25.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 -- 172.21.15.90:0/1559313482 shutdown_connections 2024-04-26T18:11:25.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.304+0000 7f7df75a4700 1 -- 172.21.15.90:0/1559313482 wait complete. 2024-04-26T18:11:25.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 Processor -- start 2024-04-26T18:11:25.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 -- start start 2024-04-26T18:11:25.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df0198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:25.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df0103580 0x7f7df0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:25.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df0103f20 0x7f7df019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:25.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010def0 con 0x7f7df0102380 2024-04-26T18:11:25.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010dd70 con 0x7f7df0103580 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df75a4700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7df010e070 con 0x7f7df0103f20 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df0198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df0198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36610/0 (socket says 172.21.15.90:36610) 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 -- 172.21.15.90:0/1364917667 learned_addr learned my addr 172.21.15.90:0/1364917667 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df4b3f700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df0103580 0x7f7df0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5b41700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df0103f20 0x7f7df019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 -- 172.21.15.90:0/1364917667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df0103f20 msgr2=0x7f7df019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df0103f20 0x7f7df019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 -- 172.21.15.90:0/1364917667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df0103580 msgr2=0x7f7df0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df0103580 0x7f7df0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 -- 172.21.15.90:0/1364917667 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df01a02a0 con 0x7f7df0102380 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5b41700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df0103f20 0x7f7df019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.305+0000 7f7df5340700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df0198640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00b580 tx=0x7f7dec005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.306+0000 7f7de67fc700 1 -- 172.21.15.90:0/1364917667 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec027070 con 0x7f7df0102380 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.306+0000 7f7de67fc700 1 -- 172.21.15.90:0/1364917667 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dec004120 con 0x7f7df0102380 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.306+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df01a0530 con 0x7f7df0102380 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.306+0000 7f7de67fc700 1 -- 172.21.15.90:0/1364917667 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec022ce0 con 0x7f7df0102380 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.306+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df01a0ae0 con 0x7f7df0102380 2024-04-26T18:11:25.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.307+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df006a110 con 0x7f7df0102380 2024-04-26T18:11:25.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.308+0000 7f7de67fc700 1 -- 172.21.15.90:0/1364917667 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7dec02a070 con 0x7f7df0102380 2024-04-26T18:11:25.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.308+0000 7f7de67fc700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7ddc0a3cc0 0x7f7ddc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:25.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.309+0000 7f7df4b3f700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7ddc0a3cc0 0x7f7ddc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:25.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.309+0000 7f7de67fc700 1 -- 172.21.15.90:0/1364917667 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f7dec0d1980 con 0x7f7df0102380 2024-04-26T18:11:25.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.309+0000 7f7df4b3f700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7ddc0a3cc0 0x7f7ddc0a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009a10 tx=0x7f7de0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:25.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.312+0000 7f7de67fc700 1 -- 172.21.15.90:0/1364917667 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7dec031020 con 0x7f7df0102380 2024-04-26T18:11:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:24 smithi167 ceph-mon[32751]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:25.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.521+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f7df0063860 con 0x7f7ddc0a3cc0 2024-04-26T18:11:25.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.523+0000 7f7de67fc700 1 -- 172.21.15.90:0/1364917667 <== mgr.14152 v2:172.21.15.90:6800/3660618461 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f7df0063860 con 0x7f7ddc0a3cc0 2024-04-26T18:11:25.523 INFO:teuthology.orchestra.run.smithi090.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T18:11:25.523 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T18:11:25.523 INFO:teuthology.orchestra.run.smithi090.stdout:smithi119 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400Y1400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T18:11:25.523 INFO:teuthology.orchestra.run.smithi090.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T18:11:25.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7ddc0a3cc0 msgr2=0x7f7ddc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:25.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7ddc0a3cc0 0x7f7ddc0a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7de0009a10 tx=0x7f7de0009190 comp rx=0 tx=0).stop 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 msgr2=0x7f7df0198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df0198640 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00b580 tx=0x7f7dec005ef0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 shutdown_connections 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7ddc0a3cc0 0x7f7ddc0a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7df0102380 0x7f7df0198640 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7df0103f20 0x7f7df019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 --2- 172.21.15.90:0/1364917667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df0103580 0x7f7df0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 >> 172.21.15.90:0/1364917667 conn(0x7f7df00fd920 msgr2=0x7f7df0106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 shutdown_connections 2024-04-26T18:11:25.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:25.525+0000 7f7df75a4700 1 -- 172.21.15.90:0/1364917667 wait complete. 2024-04-26T18:11:26.255 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T18:11:26.265 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T18:11:26.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T18:11:26.505 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:26 smithi119 ceph-mon[32752]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:26 smithi119 ceph-mon[32752]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:26 smithi167 ceph-mon[32751]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:26 smithi167 ceph-mon[32751]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:26 smithi090 ceph-mon[29180]: from='client.15009 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:26 smithi090 ceph-mon[29180]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:28.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.191+0000 7fc1022ba700 1 -- 172.21.15.90:0/797798099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 msgr2=0x7fc0fc10ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.191+0000 7fc1022ba700 1 --2- 172.21.15.90:0/797798099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc10ba60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400b0d0 tx=0x7fc0f400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 -- 172.21.15.90:0/797798099 shutdown_connections 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 --2- 172.21.15.90:0/797798099 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc10ba60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 --2- 172.21.15.90:0/797798099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0fc101790 0x7fc0fc101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 --2- 172.21.15.90:0/797798099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0fc104810 0x7fc0fc104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 -- 172.21.15.90:0/797798099 >> 172.21.15.90:0/797798099 conn(0x7fc0fc0fd0d0 msgr2=0x7fc0fc0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 -- 172.21.15.90:0/797798099 shutdown_connections 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 -- 172.21.15.90:0/797798099 wait complete. 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 Processor -- start 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 -- start start 2024-04-26T18:11:28.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0fc101790 0x7fc0fc199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0fc104810 0x7fc0fc199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.192+0000 7fc1022ba700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc1022ba700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10ef30 con 0x7fc0fc1051b0 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc1022ba700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10edb0 con 0x7fc0fc101790 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc1022ba700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0fc10f0b0 con 0x7fc0fc104810 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc1a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36620/0 (socket says 172.21.15.90:36620) 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 -- 172.21.15.90:0/4049885081 learned_addr learned my addr 172.21.15.90:0/4049885081 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc0fb7fe700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0fc104810 0x7fc0fc199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc0fbfff700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0fc101790 0x7fc0fc199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 -- 172.21.15.90:0/4049885081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0fc104810 msgr2=0x7fc0fc199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:28.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0fc104810 0x7fc0fc199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 -- 172.21.15.90:0/4049885081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0fc101790 msgr2=0x7fc0fc199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0fc101790 0x7fc0fc199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 -- 172.21.15.90:0/4049885081 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0fc1a10b0 con 0x7fc0fc1051b0 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc100857700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc1a0a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400d010 tx=0x7fc0f4004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc0f97fa700 1 -- 172.21.15.90:0/4049885081 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401f030 con 0x7fc0fc1051b0 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0fc1a1340 con 0x7fc0fc1051b0 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.193+0000 7fc0f97fa700 1 -- 172.21.15.90:0/4049885081 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0f4007be0 con 0x7fc0fc1051b0 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.194+0000 7fc0f97fa700 1 -- 172.21.15.90:0/4049885081 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4004810 con 0x7fc0fc1051b0 2024-04-26T18:11:28.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.194+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0fc1a17f0 con 0x7fc0fc1051b0 2024-04-26T18:11:28.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.195+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0fc06a110 con 0x7fc0fc1051b0 2024-04-26T18:11:28.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.195+0000 7fc0f97fa700 1 -- 172.21.15.90:0/4049885081 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc0f4003a00 con 0x7fc0fc1051b0 2024-04-26T18:11:28.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.196+0000 7fc0f97fa700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc0e40a3c40 0x7fc0e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:28.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.196+0000 7fc0fbfff700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc0e40a3c40 0x7fc0e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:28.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.196+0000 7fc0f97fa700 1 -- 172.21.15.90:0/4049885081 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc0f400f020 con 0x7fc0fc1051b0 2024-04-26T18:11:28.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.196+0000 7fc0fbfff700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc0e40a3c40 0x7fc0e40a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec005fd0 tx=0x7fc0ec005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:28.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.201+0000 7fc0f97fa700 1 -- 172.21.15.90:0/4049885081 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0f4029020 con 0x7fc0fc1051b0 2024-04-26T18:11:28.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.449+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fc0fc02e490 con 0x7fc0fc1051b0 2024-04-26T18:11:28.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.450+0000 7fc0f97fa700 1 -- 172.21.15.90:0/4049885081 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3052 (secure 0 0 0) 0x7fc0f409e2b0 con 0x7fc0fc1051b0 2024-04-26T18:11:28.450 INFO:teuthology.orchestra.run.smithi090.stdout:e25 2024-04-26T18:11:28.450 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:11:28.450 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:11:28.450 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T18:11:28.450 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 25 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T18:08:49.029287+0000 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T18:09:06.760287+0000 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 300 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T18:11:28.451 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 3 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1,2 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=24449,1=24448,2=24436} 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T18:11:28.452 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 0 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 24449 members: 24448,24449,24436 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.b{0:24449} state up:active seq 5 addr [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.f{0:24430} state up:standby-replay seq 1 addr [v2:172.21.15.167:6834/570376817,v1:172.21.15.167:6835/570376817] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.i{1:24448} state up:active seq 11 addr [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.h{1:24428} state up:standby-replay seq 1 addr [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.453 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.c{2:24436} state up:active seq 12 addr [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.l{2:24412} state up:standby-replay seq 1 addr [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.d{-1:14535} state up:standby seq 1 addr [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.g{-1:14574} state up:standby seq 1 addr [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.a{-1:14586} state up:standby seq 1 addr [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.k{-1:24392} state up:standby seq 1 addr [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.454 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.e{-1:24407} state up:standby seq 1 addr [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:11:28.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc0e40a3c40 msgr2=0x7fc0e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:28.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc0e40a3c40 0x7fc0e40a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec005fd0 tx=0x7fc0ec005dd0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 msgr2=0x7fc0fc1a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc1a0a10 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400d010 tx=0x7fc0f4004ef0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 shutdown_connections 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc0e40a3c40 0x7fc0e40a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0fc1051b0 0x7fc0fc1a0a10 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0fc101790 0x7fc0fc199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 --2- 172.21.15.90:0/4049885081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0fc104810 0x7fc0fc199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 >> 172.21.15.90:0/4049885081 conn(0x7fc0fc0fd0d0 msgr2=0x7fc0fc107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 shutdown_connections 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:28.452+0000 7fc1022ba700 1 -- 172.21.15.90:0/4049885081 wait complete. 2024-04-26T18:11:28.456 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T18:11:29.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T18:11:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:29 smithi119 ceph-mon[32752]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:29 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/4049885081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:11:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:29 smithi167 ceph-mon[32751]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:29 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/4049885081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:11:29.364 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:28 smithi090 ceph-mon[29180]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:29 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/4049885081' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:11:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:31 smithi090 ceph-mon[29180]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:31.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.136+0000 7f259de29700 1 -- 172.21.15.90:0/2304557737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103580 msgr2=0x7f25981039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.136+0000 7f259de29700 1 --2- 172.21.15.90:0/2304557737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103580 0x7f25981039e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f25880099c0 tx=0x7f2588009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.136+0000 7f259de29700 1 -- 172.21.15.90:0/2304557737 shutdown_connections 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.136+0000 7f259de29700 1 --2- 172.21.15.90:0/2304557737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f20 0x7f259810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.136+0000 7f259de29700 1 --2- 172.21.15.90:0/2304557737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598102380 0x7f25981027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.136+0000 7f259de29700 1 --2- 172.21.15.90:0/2304557737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598103580 0x7f25981039e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.136+0000 7f259de29700 1 -- 172.21.15.90:0/2304557737 >> 172.21.15.90:0/2304557737 conn(0x7f25980fd920 msgr2=0x7f25980ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.137+0000 7f259de29700 1 -- 172.21.15.90:0/2304557737 shutdown_connections 2024-04-26T18:11:31.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.137+0000 7f259de29700 1 -- 172.21.15.90:0/2304557737 wait complete. 2024-04-26T18:11:31.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.137+0000 7f259de29700 1 Processor -- start 2024-04-26T18:11:31.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.137+0000 7f259de29700 1 -- start start 2024-04-26T18:11:31.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.137+0000 7f259de29700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598102380 0x7f259806bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:31.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.137+0000 7f259de29700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103580 0x7f259819e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:31.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.137+0000 7f259de29700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f20 0x7f25981a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:31.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f259de29700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f259810de70 con 0x7f2598102380 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f259de29700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f259810dcf0 con 0x7f2598103f20 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f259de29700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f259810dff0 con 0x7f2598103580 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598102380 0x7f259806bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598102380 0x7f259806bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36628/0 (socket says 172.21.15.90:36628) 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 -- 172.21.15.90:0/1775729380 learned_addr learned my addr 172.21.15.90:0/1775729380 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f2597fff700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f20 0x7f25981a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 -- 172.21.15.90:0/1775729380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103580 msgr2=0x7f259819e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103580 0x7f259819e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 -- 172.21.15.90:0/1775729380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f20 msgr2=0x7f25981a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f20 0x7f25981a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 -- 172.21.15.90:0/1775729380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25981a1230 con 0x7f2598102380 2024-04-26T18:11:31.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f2597fff700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f20 0x7f25981a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:31.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f25977fe700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598102380 0x7f259806bc60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c990 tx=0x7f258c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:31.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f2594ff9700 1 -- 172.21.15.90:0/1775729380 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c007b00 con 0x7f2598102380 2024-04-26T18:11:31.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f2594ff9700 1 -- 172.21.15.90:0/1775729380 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f258c007ca0 con 0x7f2598102380 2024-04-26T18:11:31.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25981a1520 con 0x7f2598102380 2024-04-26T18:11:31.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f2594ff9700 1 -- 172.21.15.90:0/1775729380 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f258c018660 con 0x7f2598102380 2024-04-26T18:11:31.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.138+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25981a1a60 con 0x7f2598102380 2024-04-26T18:11:31.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.139+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259806a110 con 0x7f2598102380 2024-04-26T18:11:31.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.140+0000 7f2594ff9700 1 -- 172.21.15.90:0/1775729380 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f258c004020 con 0x7f2598102380 2024-04-26T18:11:31.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.141+0000 7f2594ff9700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:31.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.141+0000 7f2596ffd700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:31.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.141+0000 7f2594ff9700 1 -- 172.21.15.90:0/1775729380 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f258c0d04d0 con 0x7f2598102380 2024-04-26T18:11:31.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.141+0000 7f2596ffd700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f25880098e0 tx=0x7f2588009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:31.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.144+0000 7f2594ff9700 1 -- 172.21.15.90:0/1775729380 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f258c014030 con 0x7f2598102380 2024-04-26T18:11:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:31 smithi119 ceph-mon[32752]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:31 smithi167 ceph-mon[32751]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:31.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.341+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f259802e530 con 0x7f2598102380 2024-04-26T18:11:31.342 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.342+0000 7f2594ff9700 1 -- 172.21.15.90:0/1775729380 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5230 (secure 0 0 0) 0x7f258c0a1e20 con 0x7f2598102380 2024-04-26T18:11:31.342 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 92 2024-04-26T18:11:31.342 INFO:teuthology.orchestra.run.smithi090.stdout:fsid b8878a18-03f6-11ef-bc93-c7b262605968 2024-04-26T18:11:31.342 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T18:00:36.466392+0000 2024-04-26T18:11:31.342 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T18:09:33.002079+0000 2024-04-26T18:11:31.342 INFO:teuthology.orchestra.run.smithi090.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:crush_version 26 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:full_ratio 0.95 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:backfillfull_ratio 0.9 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:nearfull_ratio 0.85 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:require_min_compat_client luminous 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:min_compat_client luminous 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:require_osd_release squid 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:stretch_mode_enabled false 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 90 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T18:11:31.343 INFO:teuthology.orchestra.run.smithi090.stdout:max_osd 12 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6802/1246374052,v1:172.21.15.90:6803/1246374052] [v2:172.21.15.90:6804/1246374052,v1:172.21.15.90:6805/1246374052] exists,up 7d909b27-dec0-46c5-adc0-0eb84d381e2c 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 up in weight 1 up_from 13 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6800/447582206,v1:172.21.15.119:6801/447582206] [v2:172.21.15.119:6802/447582206,v1:172.21.15.119:6803/447582206] exists,up 9122fc04-ae29-4ed2-a91f-4dcb62208684 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/1709871603,v1:172.21.15.167:6801/1709871603] [v2:172.21.15.167:6802/1709871603,v1:172.21.15.167:6803/1709871603] exists,up d283b32e-2f13-4fbf-a75b-957df3b02b70 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6810/2988562238,v1:172.21.15.90:6811/2988562238] [v2:172.21.15.90:6812/2988562238,v1:172.21.15.90:6813/2988562238] exists,up 8548cc2a-05b4-4e96-9e1c-22b26116170d 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 up in weight 1 up_from 33 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6808/2105013830,v1:172.21.15.119:6809/2105013830] [v2:172.21.15.119:6810/2105013830,v1:172.21.15.119:6811/2105013830] exists,up 11b6b750-1a3f-4a50-bacf-145c1586412b 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 up in weight 1 up_from 39 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/2447709216,v1:172.21.15.167:6809/2447709216] [v2:172.21.15.167:6810/2447709216,v1:172.21.15.167:6811/2447709216] exists,up 592a3d64-facb-42f0-928d-ab7326e69642 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.6 up in weight 1 up_from 47 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6818/2940618965,v1:172.21.15.90:6819/2940618965] [v2:172.21.15.90:6820/2940618965,v1:172.21.15.90:6821/2940618965] exists,up 24eb3641-a329-4a5f-b672-06b21c53894b 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.7 up in weight 1 up_from 53 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6816/146780393,v1:172.21.15.119:6817/146780393] [v2:172.21.15.119:6818/146780393,v1:172.21.15.119:6819/146780393] exists,up 6aea2198-ac95-4f94-9953-dd093977e34c 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.8 up in weight 1 up_from 61 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/1376531844,v1:172.21.15.167:6817/1376531844] [v2:172.21.15.167:6818/1376531844,v1:172.21.15.167:6819/1376531844] exists,up 0b9aca44-5296-4726-b01f-80153f4c806d 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.9 up in weight 1 up_from 69 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.90:6826/1702789383,v1:172.21.15.90:6827/1702789383] [v2:172.21.15.90:6828/1702789383,v1:172.21.15.90:6829/1702789383] exists,up d262c94d-6bc8-4107-851a-36b1db393708 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.10 up in weight 1 up_from 74 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.119:6824/249896871,v1:172.21.15.119:6825/249896871] [v2:172.21.15.119:6826/249896871,v1:172.21.15.119:6827/249896871] exists,up 8873f617-32db-4f50-b47e-b1d3f42eaf4b 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:osd.11 up in weight 1 up_from 82 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/3743670962,v1:172.21.15.167:6825/3743670962] [v2:172.21.15.167:6826/3743670962,v1:172.21.15.167:6827/3743670962] exists,up 123cd211-4456-4b3d-8ad3-bd47cdda36a3 2024-04-26T18:11:31.344 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.5 [8,11] 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 2.13 [2,11] 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.e [5,9] 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.13 [4,9] 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.14 [4,6] 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:pg_upmap_items 3.36 [4,9] 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/651646115 expires 2024-04-27T18:01:31.766717+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/1868991041 expires 2024-04-27T18:01:31.766717+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6801/4035087696 expires 2024-04-27T18:01:03.883791+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/4035087696 expires 2024-04-27T18:01:03.883791+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/227996061 expires 2024-04-27T18:01:03.883791+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/295280840 expires 2024-04-27T18:01:03.883791+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/2846931038 expires 2024-04-27T18:01:31.766717+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/794009814 expires 2024-04-27T18:01:03.883791+0000 2024-04-26T18:11:31.345 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:0/1492327080 expires 2024-04-27T18:01:31.766717+0000 2024-04-26T18:11:31.346 INFO:teuthology.orchestra.run.smithi090.stdout:blocklist 172.21.15.90:6800/1868991041 expires 2024-04-27T18:01:31.766717+0000 2024-04-26T18:11:31.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 msgr2=0x7f25800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f25880098e0 tx=0x7f2588009f10 comp rx=0 tx=0).stop 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598102380 msgr2=0x7f259806bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598102380 0x7f259806bc60 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f258c00c990 tx=0x7f258c00cd50 comp rx=0 tx=0).stop 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 shutdown_connections 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f25800a3c40 0x7f25800a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2598102380 0x7f259806bc60 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2598103f20 0x7f25981a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 --2- 172.21.15.90:0/1775729380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2598103580 0x7f259819e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 >> 172.21.15.90:0/1775729380 conn(0x7f25980fd920 msgr2=0x7f2598106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 shutdown_connections 2024-04-26T18:11:31.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:31.344+0000 7f259de29700 1 -- 172.21.15.90:0/1775729380 wait complete. 2024-04-26T18:11:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:32 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1775729380' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:11:32.136 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T18:11:32.147 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:11:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:32 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1775729380' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:11:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:32 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1775729380' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:11:32.387 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:33 smithi090 ceph-mon[29180]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:11:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:33 smithi119 ceph-mon[32752]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:11:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:33 smithi167 ceph-mon[32751]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:11:34.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 -- 172.21.15.90:0/2236569425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c1094a0 msgr2=0x7f7e0c109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 --2- 172.21.15.90:0/2236569425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c1094a0 0x7f7e0c109880 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7e00009990 tx=0x7f7e00009ca0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 -- 172.21.15.90:0/2236569425 shutdown_connections 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 --2- 172.21.15.90:0/2236569425 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c1094a0 0x7f7e0c109880 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 --2- 172.21.15.90:0/2236569425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e0c06c510 0x7f7e0c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 --2- 172.21.15.90:0/2236569425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c06bb70 0x7f7e0c06bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 -- 172.21.15.90:0/2236569425 >> 172.21.15.90:0/2236569425 conn(0x7f7e0c0792f0 msgr2=0x7f7e0c079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 -- 172.21.15.90:0/2236569425 shutdown_connections 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 -- 172.21.15.90:0/2236569425 wait complete. 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 Processor -- start 2024-04-26T18:11:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.100+0000 7f7e12de1700 1 -- start start 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c06bb70 0x7f7e0c19f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e0c06c510 0x7f7e0c19fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c1094a0 0x7f7e0c199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c113690 con 0x7f7e0c06bb70 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c113510 con 0x7f7e0c06c510 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7e0c113810 con 0x7f7e0c1094a0 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c06bb70 0x7f7e0c19f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c06bb70 0x7f7e0c19f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34380/0 (socket says 172.21.15.90:34380) 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 -- 172.21.15.90:0/1405754341 learned_addr learned my addr 172.21.15.90:0/1405754341 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e0bfff700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e0c06c510 0x7f7e0c19fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 -- 172.21.15.90:0/1405754341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c1094a0 msgr2=0x7f7e0c199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c1094a0 0x7f7e0c199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 -- 172.21.15.90:0/1405754341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e0c06c510 msgr2=0x7f7e0c19fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e0c06c510 0x7f7e0c19fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 -- 172.21.15.90:0/1405754341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e0c19a1f0 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e0bfff700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e0c06c510 0x7f7e0c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e10b7d700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c06bb70 0x7f7e0c19f8d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e00000c00 tx=0x7f7e000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e09ffb700 1 -- 172.21.15.90:0/1405754341 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e00027070 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e09ffb700 1 -- 172.21.15.90:0/1405754341 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e000041d0 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e0c19a480 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e09ffb700 1 -- 172.21.15.90:0/1405754341 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0000f460 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.101+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e0c1a67a0 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.102+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e0c104b50 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.103+0000 7f7e09ffb700 1 -- 172.21.15.90:0/1405754341 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7e0002a070 con 0x7f7e0c06bb70 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.104+0000 7f7e09ffb700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.104+0000 7f7e0bfff700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.104+0000 7f7e09ffb700 1 -- 172.21.15.90:0/1405754341 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f7e000d1340 con 0x7f7e0c06bb70 2024-04-26T18:11:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.104+0000 7f7e0bfff700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c079a30 tx=0x7f7df8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.108+0000 7f7e09ffb700 1 -- 172.21.15.90:0/1405754341 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e0005eff0 con 0x7f7e0c06bb70 2024-04-26T18:11:34.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.350+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e0c19afc0 con 0x7f7e0c06bb70 2024-04-26T18:11:34.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.351+0000 7f7e09ffb700 1 -- 172.21.15.90:0/1405754341 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11874 (secure 0 0 0) 0x7f7e00013070 con 0x7f7e0c06bb70 2024-04-26T18:11:34.351 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:34.351 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:09:06.760287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24448,24449,24436]},"id":1}]} 2024-04-26T18:11:34.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7dfc0a3c40 msgr2=0x7f7dfc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:34.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c079a30 tx=0x7f7df8008040 comp rx=0 tx=0).stop 2024-04-26T18:11:34.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c06bb70 msgr2=0x7f7e0c19f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:34.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c06bb70 0x7f7e0c19f8d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e00000c00 tx=0x7f7e000047b0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 shutdown_connections 2024-04-26T18:11:34.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f7dfc0a3c40 0x7f7dfc0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7e0c06bb70 0x7f7e0c19f8d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7e0c1094a0 0x7f7e0c199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 --2- 172.21.15.90:0/1405754341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e0c06c510 0x7f7e0c19fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:34.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 >> 172.21.15.90:0/1405754341 conn(0x7f7e0c0792f0 msgr2=0x7f7e0c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:34.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 shutdown_connections 2024-04-26T18:11:34.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:34.353+0000 7f7e12de1700 1 -- 172.21.15.90:0/1405754341 wait complete. 2024-04-26T18:11:34.354 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T18:11:35.035 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:11:35.276 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:35 smithi119 ceph-mon[32752]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:35 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1405754341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:35 smithi167 ceph-mon[32751]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:35 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1405754341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:35 smithi090 ceph-mon[29180]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:35 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1405754341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 -- 172.21.15.90:0/781096496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf201085f0 msgr2=0x7fdf201089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:36.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 --2- 172.21.15.90:0/781096496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf201085f0 0x7fdf201089d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0099c0 tx=0x7fdf1c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 -- 172.21.15.90:0/781096496 shutdown_connections 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 --2- 172.21.15.90:0/781096496 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf201085f0 0x7fdf201089d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 --2- 172.21.15.90:0/781096496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf20103890 0x7fdf2010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 --2- 172.21.15.90:0/781096496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf20102ed0 0x7fdf20103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 -- 172.21.15.90:0/781096496 >> 172.21.15.90:0/781096496 conn(0x7fdf20077640 msgr2=0x7fdf20077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 -- 172.21.15.90:0/781096496 shutdown_connections 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.916+0000 7fdf27760700 1 -- 172.21.15.90:0/781096496 wait complete. 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 Processor -- start 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 -- start start 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf20102ed0 0x7fdf2019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf20103890 0x7fdf2019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:36.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf201085f0 0x7fdf201a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf20112670 con 0x7fdf20102ed0 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201124f0 con 0x7fdf20103890 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf27760700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf201127f0 con 0x7fdf201085f0 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf254fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf20102ed0 0x7fdf2019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf254fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf20102ed0 0x7fdf2019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34400/0 (socket says 172.21.15.90:34400) 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf254fc700 1 -- 172.21.15.90:0/1653317941 learned_addr learned my addr 172.21.15.90:0/1653317941 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.917+0000 7fdf24cfb700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf20103890 0x7fdf2019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf254fc700 1 -- 172.21.15.90:0/1653317941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf201085f0 msgr2=0x7fdf201a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf25cfd700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf201085f0 0x7fdf201a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf254fc700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf201085f0 0x7fdf201a0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf254fc700 1 -- 172.21.15.90:0/1653317941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf20103890 msgr2=0x7fdf2019c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf254fc700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf20103890 0x7fdf2019c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:36.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf254fc700 1 -- 172.21.15.90:0/1653317941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf201a0eb0 con 0x7fdf20102ed0 2024-04-26T18:11:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf254fc700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf20102ed0 0x7fdf2019bea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00b580 tx=0x7fdf1c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf167fc700 1 -- 172.21.15.90:0/1653317941 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c027070 con 0x7fdf20102ed0 2024-04-26T18:11:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf167fc700 1 -- 172.21.15.90:0/1653317941 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf1c004110 con 0x7fdf20102ed0 2024-04-26T18:11:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf201a1140 con 0x7fdf20102ed0 2024-04-26T18:11:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf167fc700 1 -- 172.21.15.90:0/1653317941 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00f460 con 0x7fdf20102ed0 2024-04-26T18:11:36.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.918+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf201a8a20 con 0x7fdf20102ed0 2024-04-26T18:11:36.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.919+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2006a110 con 0x7fdf20102ed0 2024-04-26T18:11:36.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.920+0000 7fdf167fc700 1 -- 172.21.15.90:0/1653317941 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdf1c02a070 con 0x7fdf20102ed0 2024-04-26T18:11:36.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.921+0000 7fdf167fc700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdf0c0a3cc0 0x7fdf0c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:36.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.921+0000 7fdf24cfb700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdf0c0a3cc0 0x7fdf0c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:36.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.921+0000 7fdf167fc700 1 -- 172.21.15.90:0/1653317941 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fdf1c0d1290 con 0x7fdf20102ed0 2024-04-26T18:11:36.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.921+0000 7fdf24cfb700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdf0c0a3cc0 0x7fdf0c0a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:36.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:36.925+0000 7fdf167fc700 1 -- 172.21.15.90:0/1653317941 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf1c05eff0 con 0x7fdf20102ed0 2024-04-26T18:11:37.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.170+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf2019d200 con 0x7fdf20102ed0 2024-04-26T18:11:37.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.171+0000 7fdf167fc700 1 -- 172.21.15.90:0/1653317941 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11874 (secure 0 0 0) 0x7fdf1c013070 con 0x7fdf20102ed0 2024-04-26T18:11:37.171 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:37.171 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:09:06.760287+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24448,24449,24436]},"id":1}]} 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdf0c0a3cc0 msgr2=0x7fdf0c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdf0c0a3cc0 0x7fdf0c0a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006fd0 tx=0x7fdf10008040 comp rx=0 tx=0).stop 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf20102ed0 msgr2=0x7fdf2019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf20102ed0 0x7fdf2019bea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00b580 tx=0x7fdf1c0037c0 comp rx=0 tx=0).stop 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 shutdown_connections 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fdf0c0a3cc0 0x7fdf0c0a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf20102ed0 0x7fdf2019bea0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:37.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf201085f0 0x7fdf201a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 --2- 172.21.15.90:0/1653317941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf20103890 0x7fdf2019c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 >> 172.21.15.90:0/1653317941 conn(0x7fdf20077640 msgr2=0x7fdf2010d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 shutdown_connections 2024-04-26T18:11:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:37.173+0000 7fdf27760700 1 -- 172.21.15.90:0/1653317941 wait complete. 2024-04-26T18:11:37.174 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T18:11:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:37 smithi119 ceph-mon[32752]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:37 smithi167 ceph-mon[32751]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:37 smithi090 ceph-mon[29180]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:37.806 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:11:38.046 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:38.295 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:38 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1653317941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:38 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1653317941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:38 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1653317941' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:11:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:39 smithi090 ceph-mon[29180]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:39 smithi119 ceph-mon[32752]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:39 smithi167 ceph-mon[32751]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 -- 172.21.15.90:0/3354335205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc01094c0 msgr2=0x7f9dc01098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/3354335205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc01094c0 0x7f9dc01098a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009990 tx=0x7f9dbc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 -- 172.21.15.90:0/3354335205 shutdown_connections 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/3354335205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc01094c0 0x7f9dc01098a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/3354335205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc0103e30 0x7f9dc0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/3354335205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc0103470 0x7f9dc01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 -- 172.21.15.90:0/3354335205 >> 172.21.15.90:0/3354335205 conn(0x7f9dc007a580 msgr2=0x7f9dc007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 -- 172.21.15.90:0/3354335205 shutdown_connections 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 -- 172.21.15.90:0/3354335205 wait complete. 2024-04-26T18:11:39.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 Processor -- start 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.733+0000 7f9dc7edb700 1 -- start start 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc0103470 0x7f9dc019f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc0103e30 0x7f9dc019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc01094c0 0x7f9dc0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc0113640 con 0x7f9dc0103e30 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01134c0 con 0x7f9dc0103470 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc01137c0 con 0x7f9dc01094c0 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc0103e30 0x7f9dc019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc0103e30 0x7f9dc019fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34416/0 (socket says 172.21.15.90:34416) 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 -- 172.21.15.90:0/4273681442 learned_addr learned my addr 172.21.15.90:0/4273681442 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5c77700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc0103470 0x7f9dc019f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:39.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc6478700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc01094c0 0x7f9dc0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 -- 172.21.15.90:0/4273681442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc01094c0 msgr2=0x7f9dc0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc01094c0 0x7f9dc0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 -- 172.21.15.90:0/4273681442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc0103470 msgr2=0x7f9dc019f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc0103470 0x7f9dc019f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 -- 172.21.15.90:0/4273681442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc019a160 con 0x7f9dc0103e30 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc6478700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc01094c0 0x7f9dc0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc5476700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc0103e30 0x7f9dc019fd80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9db000b530 tx=0x7f9db000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9db6ffd700 1 -- 172.21.15.90:0/4273681442 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0015020 con 0x7f9dc0103e30 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9db6ffd700 1 -- 172.21.15.90:0/4273681442 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db0011ce0 con 0x7f9dc0103e30 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc019a450 con 0x7f9dc0103e30 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9db6ffd700 1 -- 172.21.15.90:0/4273681442 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db0010d20 con 0x7f9dc0103e30 2024-04-26T18:11:39.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.734+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc01a6710 con 0x7f9dc0103e30 2024-04-26T18:11:39.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.735+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dc006a110 con 0x7f9dc0103e30 2024-04-26T18:11:39.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.737+0000 7f9db6ffd700 1 -- 172.21.15.90:0/4273681442 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9db0018020 con 0x7f9dc0103e30 2024-04-26T18:11:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.737+0000 7f9db6ffd700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.737+0000 7f9dc5c77700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.737+0000 7f9db6ffd700 1 -- 172.21.15.90:0/4273681442 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f9db00d00b0 con 0x7f9dc0103e30 2024-04-26T18:11:39.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.738+0000 7f9dc5c77700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc006010 tx=0x7f9dbc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:39.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.741+0000 7f9db6ffd700 1 -- 172.21.15.90:0/4273681442 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9db001f020 con 0x7f9dc0103e30 2024-04-26T18:11:39.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.946+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9dc019af30 con 0x7f9dc0103e30 2024-04-26T18:11:39.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.947+0000 7f9db6ffd700 1 -- 172.21.15.90:0/4273681442 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22557 (secure 0 0 0) 0x7f9db009cb30 con 0x7f9dc0103e30 2024-04-26T18:11:39.948 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:11:39.948 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":92,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:09:33.002079+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9dac0a3cc0 msgr2=0x7f9dac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc006010 tx=0x7f9dbc00b540 comp rx=0 tx=0).stop 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc0103e30 msgr2=0x7f9dc019fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc0103e30 0x7f9dc019fd80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9db000b530 tx=0x7f9db000b8f0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 shutdown_connections 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f9dac0a3cc0 0x7f9dac0a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9dc01094c0 0x7f9dc0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc0103470 0x7f9dc019f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 --2- 172.21.15.90:0/4273681442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9dc0103e30 0x7f9dc019fd80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 >> 172.21.15.90:0/4273681442 conn(0x7f9dc007a580 msgr2=0x7f9dc0102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 shutdown_connections 2024-04-26T18:11:39.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:39.950+0000 7f9dc7edb700 1 -- 172.21.15.90:0/4273681442 wait complete. 2024-04-26T18:11:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:40 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/4273681442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:11:40.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:40 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/4273681442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:11:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:40 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/4273681442' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:11:40.711 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:08:49.029287+0000', 'modified': '2024-04-26T18:09:06.760287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24449, 'mds_1': 24448, 'mds_2': 24436}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/766230459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 766230459}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 766230459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/3534880230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 3534880230}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 3534880230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/570376817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 570376817}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 570376817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.167:6837/200145753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 200145753}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 200145753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.167:6839/4229007587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 4229007587}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 4229007587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.119:6839/1002360318', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1002360318}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1002360318}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24449, 'qdb_cluster': [24448, 24449, 24436]} 2024-04-26T18:11:40.712 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T18:11:40.712 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T18:11:40.951 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:41.300 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:41 smithi090 ceph-mon[29180]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:41 smithi119 ceph-mon[32752]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:41 smithi167 ceph-mon[32751]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:42.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 -- 172.21.15.90:0/3849333730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14102600 msgr2=0x7faf14102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 --2- 172.21.15.90:0/3849333730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14102600 0x7faf14102a80 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faf04009990 tx=0x7faf04009ca0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 -- 172.21.15.90:0/3849333730 shutdown_connections 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 --2- 172.21.15.90:0/3849333730 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf14102fc0 0x7faf1410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 --2- 172.21.15.90:0/3849333730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf14108470 0x7faf14108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 --2- 172.21.15.90:0/3849333730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14102600 0x7faf14102a80 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 -- 172.21.15.90:0/3849333730 >> 172.21.15.90:0/3849333730 conn(0x7faf14079210 msgr2=0x7faf14079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 -- 172.21.15.90:0/3849333730 shutdown_connections 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 -- 172.21.15.90:0/3849333730 wait complete. 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 Processor -- start 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.742+0000 7faf1c526700 1 -- start start 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1c526700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf14102600 0x7faf14104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1c526700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf14102fc0 0x7faf14104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1c526700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14108470 0x7faf14104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1c526700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112570 con 0x7faf14108470 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1c526700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faf141123f0 con 0x7faf14102600 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1c526700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faf141126f0 con 0x7faf14102fc0 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14108470 0x7faf14104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14108470 0x7faf14104ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59596/0 (socket says 172.21.15.90:59596) 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 -- 172.21.15.90:0/2583796256 learned_addr learned my addr 172.21.15.90:0/2583796256 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf14102600 0x7faf14104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf19ac1700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf14102fc0 0x7faf14104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 -- 172.21.15.90:0/2583796256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf14102fc0 msgr2=0x7faf14104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf14102fc0 0x7faf14104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 -- 172.21.15.90:0/2583796256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf14102600 msgr2=0x7faf14104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf14102600 0x7faf14104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 -- 172.21.15.90:0/2583796256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf141a8050 con 0x7faf14108470 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf14102600 0x7faf14104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf19ac1700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf14102fc0 0x7faf14104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1aac3700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14108470 0x7faf14104ad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00ada0 tx=0x7faf0c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf0b7fe700 1 -- 172.21.15.90:0/2583796256 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c017020 con 0x7faf14108470 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf0b7fe700 1 -- 172.21.15.90:0/2583796256 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c00ce30 con 0x7faf14108470 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf141a8340 con 0x7faf14108470 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.743+0000 7faf0b7fe700 1 -- 172.21.15.90:0/2583796256 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0054a0 con 0x7faf14108470 2024-04-26T18:11:42.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.744+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a8830 con 0x7faf14108470 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.745+0000 7faf0b7fe700 1 -- 172.21.15.90:0/2583796256 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faf0c00c980 con 0x7faf14108470 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.746+0000 7faf0b7fe700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7faf000a3c40 0x7faf000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.746+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7faf000a3c40 0x7faf000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.746+0000 7faf0b7fe700 1 -- 172.21.15.90:0/2583796256 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7faf0c0cfb80 con 0x7faf14108470 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.746+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7faf000a3c40 0x7faf000a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf10009b50 tx=0x7faf10009550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 Processor -- start 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 -- start start 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c20e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1ad50 0x563b24c1b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 0x563b24c27770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563b24b6d7e0 con 0x563b24c20a20 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b24ad38e0 con 0x563b24c1b6f0 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.747+0000 7faf26db8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563b24b697a0 con 0x563b24c1ad50 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c20e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c20e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59600/0 (socket says 172.21.15.90:59600) 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 -- 172.21.15.90:0/1640800725 learned_addr learned my addr 172.21.15.90:0/1640800725 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf19ac1700 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1ad50 0x563b24c1b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 -- 172.21.15.90:0/1640800725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1ad50 msgr2=0x563b24c1b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1ad50 0x563b24c1b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 -- 172.21.15.90:0/1640800725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 msgr2=0x563b24c27770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 0x563b24c27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 -- 172.21.15.90:0/1640800725 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b24aba880 con 0x563b24c20a20 2024-04-26T18:11:42.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf19ac1700 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1ad50 0x563b24c1b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c20e00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf1000ed90 tx=0x7faf10011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a4393bc8c3b147 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf08ff9700 1 -- 172.21.15.90:0/1640800725 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10011c90 con 0x563b24c20a20 2024-04-26T18:11:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf08ff9700 1 -- 172.21.15.90:0/1640800725 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf10023070 con 0x563b24c20a20 2024-04-26T18:11:42.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.748+0000 7faf08ff9700 1 -- 172.21.15.90:0/1640800725 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf10010630 con 0x563b24c20a20 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 -- 172.21.15.90:0/1640800725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 msgr2=0x563b24c20e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c20e00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7faf1000ed90 tx=0x7faf10011000 comp rx=0 tx=0).stop 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 -- 172.21.15.90:0/1640800725 shutdown_connections 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c20e00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 0x563b24c27770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 --2- 172.21.15.90:0/1640800725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1ad50 0x563b24c1b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 -- 172.21.15.90:0/1640800725 >> 172.21.15.90:0/1640800725 conn(0x563b24b49570 msgr2=0x563b24c15900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 -- 172.21.15.90:0/1640800725 shutdown_connections 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 -- 172.21.15.90:0/1640800725 wait complete. 2024-04-26T18:11:42.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 -- start start 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.749+0000 7faf26db8b80 1 Processor -- start 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1ad50 0x563b24c27350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 0x563b24c246f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c24c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563b24c27cb0 con 0x563b24c1ad50 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b24b27a30 con 0x563b24c1b6f0 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563b24c27e30 con 0x563b24c20a20 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 10 client.0 _ll_register_callbacks cb 0x563b24b05900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1ad50 0x563b24c27350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1ad50 0x563b24c27350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59604/0 (socket says 172.21.15.90:59604) 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 -- 172.21.15.90:0/2490674229 learned_addr learned my addr 172.21.15.90:0/2490674229 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf19ac1700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 0x563b24c246f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1aac3700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c24c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 -- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 msgr2=0x563b24c24c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c24c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 -- 172.21.15.90:0/2490674229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 msgr2=0x563b24c246f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 0x563b24c246f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 -- 172.21.15.90:0/2490674229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b24b2b430 con 0x563b24c1ad50 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1aac3700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c24c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1ad50 0x563b24c27350 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faf1000ed60 tx=0x7faf1000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6853c7810750618c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faefdffb700 1 -- 172.21.15.90:0/2490674229 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf100054b0 con 0x563b24c1ad50 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf10023070 con 0x563b24c1ad50 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.750+0000 7faefdffb700 1 -- 172.21.15.90:0/2490674229 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7faf10005650 con 0x563b24c1ad50 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faefdffb700 1 -- 172.21.15.90:0/2490674229 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faf1001e440 con 0x563b24c1ad50 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 10 client.15051 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b24c32400 con 0x563b24c1ad50 2024-04-26T18:11:42.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faefdffb700 1 -- 172.21.15.90:0/2490674229 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf10023070 con 0x563b24c1ad50 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faefdffb700 1 -- 172.21.15.90:0/2490674229 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7faf1000f040 con 0x563b24c1ad50 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 10 client.15051 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 10 client.15051 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 20 client.15051 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24c34640 0x563b24c35ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 4 client.15051 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 --> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b24b7b1c0 con 0x563b24c34640 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.751+0000 7faf19ac1700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24c34640 0x563b24c35ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.752+0000 7faefdffb700 10 client.15051 ms_handle_connect on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:42.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.752+0000 7faf19ac1700 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24c34640 0x563b24c35ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4bece5762c13262 server_cookie=b857e81e45337e99 in_seq=0 out_seq=0 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faefdffb700 1 -- 172.21.15.90:0/2490674229 <== mds.2 v2:172.21.15.167:6836/200145753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b24b7b1c0 con 0x563b24c34640 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faefdffb700 10 client.15051 handle_command_reply: tid=0 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 client.15051 shutdown 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1ad50 msgr2=0x563b24c27350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1ad50 0x563b24c27350 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7faf1000ed60 tx=0x7faf1000eea0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 shutdown_connections 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24c34640 0x563b24c35ab0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1ad50 0x563b24c27350 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c24c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faefdffb700 0 client.15051 ms_handle_reset on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 --2- 172.21.15.90:0/2490674229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1b6f0 0x563b24c246f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 >> 172.21.15.90:0/2490674229 conn(0x563b24b49570 msgr2=0x563b24c19d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 shutdown_connections 2024-04-26T18:11:42.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 1 -- 172.21.15.90:0/2490674229 wait complete. 2024-04-26T18:11:42.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.753+0000 7faf26db8b80 20 client.15051 trim_cache size 0 max 16384 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 Processor -- start 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 -- start start 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c33860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1b6f0 0x563b24c252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563b24c33df0 con 0x563b24c1b6f0 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b24c344c0 con 0x563b24c1ad50 2024-04-26T18:11:42.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf26db8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563b24c34b90 con 0x563b24c20a20 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf19ac1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1b6f0 0x563b24c252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf19ac1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1b6f0 0x563b24c252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59618/0 (socket says 172.21.15.90:59618) 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf19ac1700 1 -- 172.21.15.90:0/566913532 learned_addr learned my addr 172.21.15.90:0/566913532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf19ac1700 1 -- 172.21.15.90:0/566913532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 msgr2=0x563b24c235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf19ac1700 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf19ac1700 1 -- 172.21.15.90:0/566913532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 msgr2=0x563b24c33860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.773+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c33860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf19ac1700 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c33860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf19ac1700 1 -- 172.21.15.90:0/566913532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b24c36a10 con 0x563b24c1b6f0 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf19ac1700 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1b6f0 0x563b24c252c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf14105ae0 tx=0x7faf04005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0838d7bd7701c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faefffff700 1 -- 172.21.15.90:0/566913532 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0402e070 con 0x563b24c1b6f0 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faefffff700 1 -- 172.21.15.90:0/566913532 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf040050b0 con 0x563b24c1b6f0 2024-04-26T18:11:42.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faefffff700 1 -- 172.21.15.90:0/566913532 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04028670 con 0x563b24c1b6f0 2024-04-26T18:11:42.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf26db8b80 1 -- 172.21.15.90:0/566913532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1b6f0 msgr2=0x563b24c252c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf26db8b80 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1b6f0 0x563b24c252c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf14105ae0 tx=0x7faf04005040 comp rx=0 tx=0).stop 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf26db8b80 1 -- 172.21.15.90:0/566913532 shutdown_connections 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf26db8b80 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c20a20 0x563b24c235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf26db8b80 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c33860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf26db8b80 1 --2- 172.21.15.90:0/566913532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c1b6f0 0x563b24c252c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.774+0000 7faf26db8b80 1 -- 172.21.15.90:0/566913532 >> 172.21.15.90:0/566913532 conn(0x563b24b49570 msgr2=0x563b24c22ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 -- 172.21.15.90:0/566913532 shutdown_connections 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 -- 172.21.15.90:0/566913532 wait complete. 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 -- start start 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 Processor -- start 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c3ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1b6f0 0x563b24c3f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c38fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563b24c35930 con 0x563b24c20a20 2024-04-26T18:11:42.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563b24c35260 con 0x563b24c1ad50 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563b24c36000 con 0x563b24c1b6f0 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf26db8b80 10 client.0 _ll_register_callbacks cb 0x563b24b05900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c38fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c38fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59634/0 (socket says 172.21.15.90:59634) 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 -- 172.21.15.90:0/3681532208 learned_addr learned my addr 172.21.15.90:0/3681532208 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf19ac1700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1b6f0 0x563b24c3f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c3ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 -- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1b6f0 msgr2=0x563b24c3f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1b6f0 0x563b24c3f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 -- 172.21.15.90:0/3681532208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 msgr2=0x563b24c3ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c3ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.775+0000 7faf1aac3700 1 -- 172.21.15.90:0/3681532208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b24c39710 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf19ac1700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1b6f0 0x563b24c3f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c3ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf1aac3700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c38fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0cd720 tx=0x7faf0c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9db9ffe3c5e98f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faefdffb700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faefdffb700 1 -- 172.21.15.90:0/3681532208 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c09bf90 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faefdffb700 1 -- 172.21.15.90:0/3681532208 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7faf0c004ad0 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf040050b0 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faefdffb700 1 -- 172.21.15.90:0/3681532208 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faf0c0a0340 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 10 client.15057 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563b24c9c180 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faefdffb700 1 -- 172.21.15.90:0/3681532208 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf040050b0 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faefdffb700 1 -- 172.21.15.90:0/3681532208 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7faf0c012070 con 0x563b24c20a20 2024-04-26T18:11:42.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 10 client.15057 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 10 client.15057 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 20 client.15057 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24ca0de0 0x563b24c9faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 4 client.15057 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.776+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 --> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] -- command(tid 0: {"prefix": "status"}) -- 0x563b24b75260 con 0x563b24ca0de0 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24ca0de0 0x563b24c9faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faefdffb700 10 client.15057 ms_handle_connect on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf1a2c2700 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24ca0de0 0x563b24c9faf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73ead8e62ee223a1 server_cookie=3e6ee7f44cc1ed84 in_seq=0 out_seq=0 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faefdffb700 1 -- 172.21.15.90:0/3681532208 <== mds.2 v2:172.21.15.167:6836/200145753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x563b24b75260 con 0x563b24ca0de0 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faefdffb700 10 client.15057 handle_command_reply: tid=0 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 client.15057 shutdown 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 msgr2=0x563b24c38fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c38fc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0cd720 tx=0x7faf0c007f90 comp rx=0 tx=0).stop 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 shutdown_connections 2024-04-26T18:11:42.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x563b24ca0de0 0x563b24c9faf0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563b24c20a20 0x563b24c38fc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563b24c1ad50 0x563b24c3ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 --2- 172.21.15.90:0/3681532208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563b24c1b6f0 0x563b24c3f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 >> 172.21.15.90:0/3681532208 conn(0x563b24b49570 msgr2=0x563b24c1a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.777+0000 7faefdffb700 0 client.15057 ms_handle_reset on v2:172.21.15.167:6836/200145753 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 shutdown_connections 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf26db8b80 1 -- 172.21.15.90:0/3681532208 wait complete. 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf26db8b80 20 client.15057 trim_cache size 0 max 16384 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:11:42.784 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24436, 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 160.844333818, 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 92, 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 203.66549619200001 2024-04-26T18:11:42.785 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:11:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7faf000a3c40 msgr2=0x7faf000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7faf000a3c40 0x7faf000a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7faf10009b50 tx=0x7faf10009550 comp rx=0 tx=0).stop 2024-04-26T18:11:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14108470 msgr2=0x7faf14104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14108470 0x7faf14104ad0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00ada0 tx=0x7faf0c00c330 comp rx=0 tx=0).stop 2024-04-26T18:11:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 shutdown_connections 2024-04-26T18:11:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7faf000a3c40 0x7faf000a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf14108470 0x7faf14104ad0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf14102600 0x7faf14104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 --2- 172.21.15.90:0/2583796256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf14102fc0 0x7faf14104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 >> 172.21.15.90:0/2583796256 conn(0x7faf14079210 msgr2=0x7faf1406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 shutdown_connections 2024-04-26T18:11:42.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:42.778+0000 7faf1c526700 1 -- 172.21.15.90:0/2583796256 wait complete. 2024-04-26T18:11:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:43 smithi119 ceph-mon[32752]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:11:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:43 smithi167 ceph-mon[32751]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:11:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:43 smithi090 ceph-mon[29180]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-26T18:11:43.563 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:11:43.802 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:45 smithi090 ceph-mon[29180]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:45 smithi119 ceph-mon[32752]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:45 smithi167 ceph-mon[32751]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-26T18:11:45.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.375+0000 7fc20e5d5700 1 -- 172.21.15.90:0/569916448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 msgr2=0x7fc208108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.375+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/569916448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208108770 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009990 tx=0x7fc1f8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 -- 172.21.15.90:0/569916448 shutdown_connections 2024-04-26T18:11:45.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/569916448 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208108770 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/569916448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102ce0 0x7fc20810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/569916448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc208102320 0x7fc2081027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 -- 172.21.15.90:0/569916448 >> 172.21.15.90:0/569916448 conn(0x7fc2080792d0 msgr2=0x7fc2080796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:45.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 -- 172.21.15.90:0/569916448 shutdown_connections 2024-04-26T18:11:45.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 -- 172.21.15.90:0/569916448 wait complete. 2024-04-26T18:11:45.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 Processor -- start 2024-04-26T18:11:45.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.376+0000 7fc20e5d5700 1 -- start start 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102320 0x7fc20819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc208102ce0 0x7fc20819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc208112530 con 0x7fc208108390 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081123b0 con 0x7fc208102320 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081126b0 con 0x7fc208102ce0 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59646/0 (socket says 172.21.15.90:59646) 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 -- 172.21.15.90:0/2058179438 learned_addr learned my addr 172.21.15.90:0/2058179438 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc207fff700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102320 0x7fc20819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc2077fe700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc208102ce0 0x7fc20819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 -- 172.21.15.90:0/2058179438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc208102ce0 msgr2=0x7fc20819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc208102ce0 0x7fc20819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 -- 172.21.15.90:0/2058179438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102320 msgr2=0x7fc20819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102320 0x7fc20819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 -- 172.21.15.90:0/2058179438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20819a0e0 con 0x7fc208108390 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc2077fe700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc208102ce0 0x7fc20819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20cb72700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208199980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ada0 tx=0x7fc20000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc2057fa700 1 -- 172.21.15.90:0/2058179438 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200017020 con 0x7fc208108390 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc20819a3d0 con 0x7fc208108390 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc2057fa700 1 -- 172.21.15.90:0/2058179438 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc20000ce30 con 0x7fc208108390 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc2057fa700 1 -- 172.21.15.90:0/2058179438 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2000054a0 con 0x7fc208108390 2024-04-26T18:11:45.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.377+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2081a6690 con 0x7fc208108390 2024-04-26T18:11:45.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.379+0000 7fc2057fa700 1 -- 172.21.15.90:0/2058179438 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc20000c980 con 0x7fc208108390 2024-04-26T18:11:45.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.380+0000 7fc2057fa700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc1f00a3c40 0x7fc1f00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.380+0000 7fc207fff700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc1f00a3c40 0x7fc1f00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.380+0000 7fc2057fa700 1 -- 172.21.15.90:0/2058179438 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc2000cfbc0 con 0x7fc208108390 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.381+0000 7fc207fff700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc1f00a3c40 0x7fc1f00a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006010 tx=0x7fc1f800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.381+0000 7fc218e67b80 1 Processor -- start 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.381+0000 7fc218e67b80 1 -- start start 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.381+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b960e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.381+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95ad50 0x56104b95b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.381+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95b6f0 0x56104b967770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc218e67b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56104b8ad7e0 con 0x56104b960a20 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc218e67b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56104b8138e0 con 0x56104b95b6f0 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc218e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56104b8a97a0 con 0x56104b95ad50 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b960e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b960e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59652/0 (socket says 172.21.15.90:59652) 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 -- 172.21.15.90:0/1281834851 learned_addr learned my addr 172.21.15.90:0/1281834851 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:45.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 -- 172.21.15.90:0/1281834851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95ad50 msgr2=0x56104b95b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc2077fe700 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95ad50 0x56104b95b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc20cb72700 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95b6f0 0x56104b967770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95ad50 0x56104b95b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 -- 172.21.15.90:0/1281834851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95b6f0 msgr2=0x56104b967770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95b6f0 0x56104b967770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 -- 172.21.15.90:0/1281834851 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104b7fa880 con 0x56104b960a20 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc2077fe700 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95ad50 0x56104b95b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc207fff700 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b960e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8015000 tx=0x7fc1f801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668e9de5a007c648 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc1ee7fc700 1 -- 172.21.15.90:0/1281834851 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800f460 con 0x56104b960a20 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc1ee7fc700 1 -- 172.21.15.90:0/1281834851 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f800fab0 con 0x56104b960a20 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.382+0000 7fc1ee7fc700 1 -- 172.21.15.90:0/1281834851 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8024b70 con 0x56104b960a20 2024-04-26T18:11:45.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 -- 172.21.15.90:0/1281834851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 msgr2=0x56104b960e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b960e00 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8015000 tx=0x7fc1f801f000 comp rx=0 tx=0).stop 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 -- 172.21.15.90:0/1281834851 shutdown_connections 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b960e00 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95b6f0 0x56104b967770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 --2- 172.21.15.90:0/1281834851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95ad50 0x56104b95b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 -- 172.21.15.90:0/1281834851 >> 172.21.15.90:0/1281834851 conn(0x56104b889570 msgr2=0x56104b955900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 -- 172.21.15.90:0/1281834851 shutdown_connections 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 -- 172.21.15.90:0/1281834851 wait complete. 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.383+0000 7fc218e67b80 1 -- start start 2024-04-26T18:11:45.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 1 Processor -- start 2024-04-26T18:11:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95ad50 0x56104b967360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b964700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b964c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56104b967cb0 con 0x56104b960a20 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56104b867a30 con 0x56104b95ad50 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56104b967e30 con 0x56104b95b6f0 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc218e67b80 10 client.0 _ll_register_callbacks cb 0x56104b845900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b964c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b964c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59664/0 (socket says 172.21.15.90:59664) 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 -- 172.21.15.90:0/3557390199 learned_addr learned my addr 172.21.15.90:0/3557390199 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc207fff700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95ad50 0x56104b967360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc2077fe700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b964700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 -- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 msgr2=0x56104b964700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b964700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 -- 172.21.15.90:0/3557390199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95ad50 msgr2=0x56104b967360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95ad50 0x56104b967360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 -- 172.21.15.90:0/3557390199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104b86b430 con 0x56104b960a20 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc207fff700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95ad50 0x56104b967360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc20cb72700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b964c40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc20000a0a0 tx=0x7fc20000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df5c97d0f28ee2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.384+0000 7fc1e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3557390199 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20009bf30 con 0x56104b960a20 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3557390199 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc2000b5220 con 0x56104b960a20 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f800fab0 con 0x56104b960a20 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3557390199 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2000b5420 con 0x56104b960a20 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 10 client.15069 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56104b972360 con 0x56104b960a20 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3557390199 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f800fab0 con 0x56104b960a20 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3557390199 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc200012070 con 0x56104b960a20 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 10 client.15069 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 10 client.15069 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 20 client.15069 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b974960 0x56104b975dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 4 client.15069 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 --> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104b8bb1c0 con 0x56104b974960 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.385+0000 7fc207fff700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b974960 0x56104b975dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.386+0000 7fc1e77fe700 10 client.15069 ms_handle_connect on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.386+0000 7fc207fff700 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b974960 0x56104b975dd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef51da7290758437 server_cookie=9711044ee0221256 in_seq=0 out_seq=0 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.386+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3557390199 <== mds.1 v2:172.21.15.167:6838/4229007587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56104b8bb1c0 con 0x56104b974960 2024-04-26T18:11:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.386+0000 7fc1e77fe700 10 client.15069 handle_command_reply: tid=0 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.386+0000 7fc218e67b80 1 client.15069 shutdown 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 msgr2=0x56104b964c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b964c40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc20000a0a0 tx=0x7fc20000cd20 comp rx=0 tx=0).stop 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 shutdown_connections 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b974960 0x56104b975dd0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b960a20 0x56104b964c40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b95ad50 0x56104b967360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3557390199 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b964700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 >> 172.21.15.90:0/3557390199 conn(0x56104b889570 msgr2=0x56104b959d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc1e77fe700 0 client.15069 ms_handle_reset on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 shutdown_connections 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 1 -- 172.21.15.90:0/3557390199 wait complete. 2024-04-26T18:11:45.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.387+0000 7fc218e67b80 20 client.15069 trim_cache size 0 max 16384 2024-04-26T18:11:45.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 Processor -- start 2024-04-26T18:11:45.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 -- start start 2024-04-26T18:11:45.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b97f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b9554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b9646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56104b974110 con 0x56104b95ad50 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56104b9747e0 con 0x56104b960a20 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.407+0000 7fc218e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56104b974eb0 con 0x56104b95b6f0 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b97f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b97f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59676/0 (socket says 172.21.15.90:59676) 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 -- 172.21.15.90:0/953495182 learned_addr learned my addr 172.21.15.90:0/953495182 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc20cb72700 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b9646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 -- 172.21.15.90:0/953495182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 msgr2=0x56104b9554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b9554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.409 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 -- 172.21.15.90:0/953495182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 msgr2=0x56104b9646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b9646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 -- 172.21.15.90:0/953495182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104b95bcd0 con 0x56104b95ad50 2024-04-26T18:11:45.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc20cb72700 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b9646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:45.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc207fff700 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b97f620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005cd0 tx=0x7fc1f8026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d02696ddead309 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:45.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc1edffb700 1 -- 172.21.15.90:0/953495182 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800f560 con 0x56104b95ad50 2024-04-26T18:11:45.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc1edffb700 1 -- 172.21.15.90:0/953495182 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f803c160 con 0x56104b95ad50 2024-04-26T18:11:45.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.408+0000 7fc1edffb700 1 -- 172.21.15.90:0/953495182 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8024750 con 0x56104b95ad50 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 -- 172.21.15.90:0/953495182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 msgr2=0x56104b97f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b97f620 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8005cd0 tx=0x7fc1f8026040 comp rx=0 tx=0).stop 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 -- 172.21.15.90:0/953495182 shutdown_connections 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b97f620 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b9646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 --2- 172.21.15.90:0/953495182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b9554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 -- 172.21.15.90:0/953495182 >> 172.21.15.90:0/953495182 conn(0x56104b889570 msgr2=0x56104b967360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 -- 172.21.15.90:0/953495182 shutdown_connections 2024-04-26T18:11:45.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.409+0000 7fc218e67b80 1 -- 172.21.15.90:0/953495182 wait complete. 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 -- start start 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 Processor -- start 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b9d8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b9d9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b95d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56104b975c50 con 0x56104b95ad50 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56104b975580 con 0x56104b960a20 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56104b976320 con 0x56104b95b6f0 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc218e67b80 10 client.0 _ll_register_callbacks cb 0x56104b845900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b9d8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b9d8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59692/0 (socket says 172.21.15.90:59692) 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 -- 172.21.15.90:0/3592503739 learned_addr learned my addr 172.21.15.90:0/3592503739 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:45.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc2077fe700 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b9d9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc20cb72700 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b95d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 -- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 msgr2=0x56104b9d9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b9d9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 -- 172.21.15.90:0/3592503739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 msgr2=0x56104b95d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b95d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.410+0000 7fc207fff700 1 -- 172.21.15.90:0/3592503739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104b95d8a0 con 0x56104b95ad50 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc207fff700 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b9d8e20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009350 tx=0x7fc1f803c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdf9cbe05dc5e71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc1e77fe700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3592503739 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8048070 con 0x56104b95ad50 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3592503739 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc1f800f4e0 con 0x56104b95ad50 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1f803c160 con 0x56104b95ad50 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3592503739 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc1f8054050 con 0x56104b95ad50 2024-04-26T18:11:45.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc218e67b80 10 client.15075 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56104b9e04c0 con 0x56104b95ad50 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3592503739 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1f803c160 con 0x56104b95ad50 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3592503739 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc1f8024c10 con 0x56104b95ad50 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc218e67b80 10 client.15075 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc218e67b80 10 client.15075 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc218e67b80 20 client.15075 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.411+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b9e48e0 0x56104b9e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.412+0000 7fc218e67b80 4 client.15075 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.412+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 --> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] -- command(tid 0: {"prefix": "status"}) -- 0x56104b8b5260 con 0x56104b9e48e0 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.412+0000 7fc2077fe700 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b9e48e0 0x56104b9e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.412+0000 7fc2077fe700 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b9e48e0 0x56104b9e35f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9dca3bcf99132786 server_cookie=f5732178dd6c65e2 in_seq=0 out_seq=0 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.412+0000 7fc1e77fe700 10 client.15075 ms_handle_connect on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc1e77fe700 1 -- 172.21.15.90:0/3592503739 <== mds.1 v2:172.21.15.167:6838/4229007587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56104b8b5260 con 0x56104b9e48e0 2024-04-26T18:11:45.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc1e77fe700 10 client.15075 handle_command_reply: tid=0 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 client.15075 shutdown 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 msgr2=0x56104b9d8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b9d8e20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009350 tx=0x7fc1f803c590 comp rx=0 tx=0).stop 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 shutdown_connections 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56104b9e48e0 0x56104b9e35f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc1e77fe700 0 client.15075 ms_handle_reset on v2:172.21.15.167:6838/4229007587 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56104b95ad50 0x56104b9d8e20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56104b960a20 0x56104b95d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 --2- 172.21.15.90:0/3592503739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56104b95b6f0 0x56104b9d9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 >> 172.21.15.90:0/3592503739 conn(0x56104b889570 msgr2=0x56104b9554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 shutdown_connections 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 1 -- 172.21.15.90:0/3592503739 wait complete. 2024-04-26T18:11:45.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.413+0000 7fc218e67b80 20 client.15075 trim_cache size 0 max 16384 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24448, 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 164.49051265099999, 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-04-26T18:11:45.419 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 92, 2024-04-26T18:11:45.420 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T18:11:45.420 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 200.645898203 2024-04-26T18:11:45.420 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:11:45.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc1f00a3c40 msgr2=0x7fc1f00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc1f00a3c40 0x7fc1f00a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8006010 tx=0x7fc1f800b540 comp rx=0 tx=0).stop 2024-04-26T18:11:45.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 msgr2=0x7fc208199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208199980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc20000ada0 tx=0x7fc20000c330 comp rx=0 tx=0).stop 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 shutdown_connections 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc1f00a3c40 0x7fc1f00a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc208108390 0x7fc208199980 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc208102320 0x7fc20819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 --2- 172.21.15.90:0/2058179438 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc208102ce0 0x7fc20819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 >> 172.21.15.90:0/2058179438 conn(0x7fc2080792d0 msgr2=0x7fc20806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 shutdown_connections 2024-04-26T18:11:45.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:45.414+0000 7fc20e5d5700 1 -- 172.21.15.90:0/2058179438 wait complete. 2024-04-26T18:11:46.124 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:11:46.362 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:47 smithi090 ceph-mon[29180]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:47 smithi119 ceph-mon[32752]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:47 smithi167 ceph-mon[32751]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-26T18:11:48.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 -- 172.21.15.90:0/3017237835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7741025b0 msgr2=0x7fc774102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 --2- 172.21.15.90:0/3017237835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7741025b0 0x7fc774102990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc7640099c0 tx=0x7fc764009cd0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 -- 172.21.15.90:0/3017237835 shutdown_connections 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 --2- 172.21.15.90:0/3017237835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7741025b0 0x7fc774102990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 --2- 172.21.15.90:0/3017237835 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc774107640 0x7fc77410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 --2- 172.21.15.90:0/3017237835 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc774102f60 0x7fc774106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 -- 172.21.15.90:0/3017237835 >> 172.21.15.90:0/3017237835 conn(0x7fc7740792f0 msgr2=0x7fc774079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 -- 172.21.15.90:0/3017237835 shutdown_connections 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.055+0000 7fc77929d700 1 -- 172.21.15.90:0/3017237835 wait complete. 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 Processor -- start 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 -- start start 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7741025b0 0x7fc77419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc774102f60 0x7fc77419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc774107640 0x7fc774198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc774112550 con 0x7fc774107640 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741123d0 con 0x7fc774102f60 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc77929d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc7741126d0 con 0x7fc7741025b0 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc7737fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc774107640 0x7fc774198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc7737fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc774107640 0x7fc774198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59708/0 (socket says 172.21.15.90:59708) 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.056+0000 7fc7737fe700 1 -- 172.21.15.90:0/402446033 learned_addr learned my addr 172.21.15.90:0/402446033 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc772ffd700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7741025b0 0x7fc77419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7727fc700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc774102f60 0x7fc77419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7737fe700 1 -- 172.21.15.90:0/402446033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7741025b0 msgr2=0x7fc77419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7737fe700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7741025b0 0x7fc77419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7737fe700 1 -- 172.21.15.90:0/402446033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc774102f60 msgr2=0x7fc77419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7737fe700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc774102f60 0x7fc77419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7737fe700 1 -- 172.21.15.90:0/402446033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc774199070 con 0x7fc774107640 2024-04-26T18:11:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7727fc700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc774102f60 0x7fc77419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:11:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc7737fe700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc774107640 0x7fc774198910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e580 tx=0x7fc76c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc75bfff700 1 -- 172.21.15.90:0/402446033 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c004bb0 con 0x7fc774107640 2024-04-26T18:11:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc75bfff700 1 -- 172.21.15.90:0/402446033 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c004d50 con 0x7fc774107640 2024-04-26T18:11:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc75bfff700 1 -- 172.21.15.90:0/402446033 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c007500 con 0x7fc774107640 2024-04-26T18:11:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc774199360 con 0x7fc774107640 2024-04-26T18:11:48.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.057+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77406bb70 con 0x7fc774107640 2024-04-26T18:11:48.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.059+0000 7fc75bfff700 1 -- 172.21.15.90:0/402446033 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc76c004020 con 0x7fc774107640 2024-04-26T18:11:48.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.060+0000 7fc75bfff700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc75c0a3c40 0x7fc75c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.060+0000 7fc772ffd700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc75c0a3c40 0x7fc75c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.060+0000 7fc75bfff700 1 -- 172.21.15.90:0/402446033 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc76c0d2370 con 0x7fc774107640 2024-04-26T18:11:48.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.060+0000 7fc772ffd700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc75c0a3c40 0x7fc75c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc764000c00 tx=0x7fc764005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 Processor -- start 2024-04-26T18:11:48.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 -- start start 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede15be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede162770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ede0a87e0 con 0x560ede15ba20 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560ede00e8e0 con 0x560ede1566f0 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560ede0a47a0 con 0x560ede155d50 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede15be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59712/0 (socket says 172.21.15.90:59712) 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 -- 172.21.15.90:0/3173943497 learned_addr learned my addr 172.21.15.90:0/3173943497 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc7727fc700 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc7737fe700 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 -- 172.21.15.90:0/3173943497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 msgr2=0x560ede1561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 -- 172.21.15.90:0/3173943497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 msgr2=0x560ede162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede162770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.061+0000 7fc772ffd700 1 -- 172.21.15.90:0/3173943497 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eddff5880 con 0x560ede15ba20 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc772ffd700 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede15be00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc764005f50 tx=0x7fc76401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd842c038abde6cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc7597fa700 1 -- 172.21.15.90:0/3173943497 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7640037c0 con 0x560ede15ba20 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc7597fa700 1 -- 172.21.15.90:0/3173943497 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc764003e10 con 0x560ede15ba20 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc7597fa700 1 -- 172.21.15.90:0/3173943497 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764024670 con 0x560ede15ba20 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3173943497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 msgr2=0x560ede15be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede15be00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc764005f50 tx=0x7fc76401f000 comp rx=0 tx=0).stop 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3173943497 shutdown_connections 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.062+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede15be00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3173943497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3173943497 >> 172.21.15.90:0/3173943497 conn(0x560ede084570 msgr2=0x560ede150900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3173943497 shutdown_connections 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3173943497 wait complete. 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 -- start start 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 Processor -- start 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede162320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede1566f0 0x560ede15f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede15ba20 0x560ede15fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ede162cb0 con 0x560ede1566f0 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560ede062a30 con 0x560ede15ba20 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560ede162e30 con 0x560ede155d50 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc783b2fb80 10 client.0 _ll_register_callbacks cb 0x560ede040900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc7727fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede1566f0 0x560ede15f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc7727fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede1566f0 0x560ede15f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59718/0 (socket says 172.21.15.90:59718) 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc7727fc700 1 -- 172.21.15.90:0/2889301047 learned_addr learned my addr 172.21.15.90:0/2889301047 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc7737fe700 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede15ba20 0x560ede15fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.063+0000 7fc772ffd700 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc7727fc700 1 -- 172.21.15.90:0/2889301047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 msgr2=0x560ede162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc7727fc700 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede162320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc7727fc700 1 -- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede15ba20 msgr2=0x560ede15fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc7727fc700 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede15ba20 0x560ede15fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc7727fc700 1 -- 172.21.15.90:0/2889301047 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ede066430 con 0x560ede1566f0 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc7727fc700 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede1566f0 0x560ede15f6c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc76800df60 tx=0x7fc76800df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a73fce4002ef69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc756ff5700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc756ff5700 1 -- 172.21.15.90:0/2889301047 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc768018070 con 0x560ede1566f0 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc756ff5700 1 -- 172.21.15.90:0/2889301047 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc768014480 con 0x560ede1566f0 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc764003e10 con 0x560ede1566f0 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc756ff5700 1 -- 172.21.15.90:0/2889301047 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc768024050 con 0x560ede1566f0 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc783b2fb80 10 client.15087 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ede171840 con 0x560ede1566f0 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc756ff5700 1 -- 172.21.15.90:0/2889301047 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc764003e10 con 0x560ede1566f0 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc756ff5700 1 -- 172.21.15.90:0/2889301047 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc768013780 con 0x560ede1566f0 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.064+0000 7fc783b2fb80 10 client.15087 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc783b2fb80 10 client.15087 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc783b2fb80 20 client.15087 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede173e90 0x560ede175300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc783b2fb80 4 client.15087 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:11:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ede0b61c0 con 0x560ede173e90 2024-04-26T18:11:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc772ffd700 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede173e90 0x560ede175300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc772ffd700 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede173e90 0x560ede175300 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58cf09049d778c24 server_cookie=1c0b467e60557a7c in_seq=0 out_seq=0 2024-04-26T18:11:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.065+0000 7fc756ff5700 10 client.15087 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc756ff5700 1 -- 172.21.15.90:0/2889301047 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ede0b61c0 con 0x560ede173e90 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc756ff5700 10 client.15087 handle_command_reply: tid=0 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 client.15087 shutdown 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede1566f0 msgr2=0x560ede15f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede1566f0 0x560ede15f6c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc76800df60 tx=0x7fc76800df90 comp rx=0 tx=0).stop 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 shutdown_connections 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede173e90 0x560ede175300 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede15ba20 0x560ede15fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc756ff5700 0 client.15087 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:48.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/2889301047 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede1566f0 0x560ede15f6c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 >> 172.21.15.90:0/2889301047 conn(0x560ede084570 msgr2=0x560ede154d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:48.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 shutdown_connections 2024-04-26T18:11:48.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 1 -- 172.21.15.90:0/2889301047 wait complete. 2024-04-26T18:11:48.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.066+0000 7fc783b2fb80 20 client.15087 trim_cache size 0 max 16384 2024-04-26T18:11:48.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 Processor -- start 2024-04-26T18:11:48.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 -- start start 2024-04-26T18:11:48.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede17e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede17eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ede173640 con 0x560ede15ba20 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560ede173d10 con 0x560ede1566f0 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560ede1743e0 con 0x560ede155d50 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede17eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede17eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59728/0 (socket says 172.21.15.90:59728) 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 -- 172.21.15.90:0/579962052 learned_addr learned my addr 172.21.15.90:0/579962052 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc772ffd700 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7727fc700 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede17e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 -- 172.21.15.90:0/579962052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 msgr2=0x560ede1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 -- 172.21.15.90:0/579962052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 msgr2=0x560ede17e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede17e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.087+0000 7fc7737fe700 1 -- 172.21.15.90:0/579962052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ede14d1f0 con 0x560ede15ba20 2024-04-26T18:11:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.088+0000 7fc7737fe700 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede17eca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc76c09e3f0 tx=0x7fc76c09a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c4b079452d834 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.088+0000 7fc758ff9700 1 -- 172.21.15.90:0/579962052 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c01f040 con 0x560ede15ba20 2024-04-26T18:11:48.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.088+0000 7fc758ff9700 1 -- 172.21.15.90:0/579962052 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c09e6f0 con 0x560ede15ba20 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.088+0000 7fc758ff9700 1 -- 172.21.15.90:0/579962052 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c0a7130 con 0x560ede15ba20 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.088+0000 7fc783b2fb80 1 -- 172.21.15.90:0/579962052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 msgr2=0x560ede17eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.088+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede17eca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc76c09e3f0 tx=0x7fc76c09a350 comp rx=0 tx=0).stop 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- 172.21.15.90:0/579962052 shutdown_connections 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede15ba20 0x560ede17eca0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede155d50 0x560ede1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/579962052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede17e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- 172.21.15.90:0/579962052 >> 172.21.15.90:0/579962052 conn(0x560ede084570 msgr2=0x560ede160260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- 172.21.15.90:0/579962052 shutdown_connections 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- 172.21.15.90:0/579962052 wait complete. 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- start start 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 Processor -- start 2024-04-26T18:11:48.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede155d50 0x560ede158360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede15ba20 0x560ede17cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560ede175180 con 0x560ede155d50 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560ede174ab0 con 0x560ede1566f0 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560ede175850 con 0x560ede15ba20 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc783b2fb80 10 client.0 _ll_register_callbacks cb 0x560ede040900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc772ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede155d50 0x560ede158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc772ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede155d50 0x560ede158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59738/0 (socket says 172.21.15.90:59738) 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.089+0000 7fc772ffd700 1 -- 172.21.15.90:0/3545800390 learned_addr learned my addr 172.21.15.90:0/3545800390 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc7727fc700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc7737fe700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede15ba20 0x560ede17cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc772ffd700 1 -- 172.21.15.90:0/3545800390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede15ba20 msgr2=0x560ede17cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc772ffd700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede15ba20 0x560ede17cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc772ffd700 1 -- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 msgr2=0x560ede1588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc772ffd700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede1588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc772ffd700 1 -- 172.21.15.90:0/3545800390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ede17d2f0 con 0x560ede155d50 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc7737fe700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede15ba20 0x560ede17cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc772ffd700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede155d50 0x560ede158360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc764009350 tx=0x7fc764005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d079b87a8e4c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc756ff5700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc756ff5700 1 -- 172.21.15.90:0/3545800390 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400bb60 con 0x560ede155d50 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc76c09e6f0 con 0x560ede155d50 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc756ff5700 1 -- 172.21.15.90:0/3545800390 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fc7640037c0 con 0x560ede155d50 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc756ff5700 1 -- 172.21.15.90:0/3545800390 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7640039c0 con 0x560ede155d50 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc783b2fb80 10 client.15093 fetch_fsmap learned FSMap version 25 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ede1db780 con 0x560ede155d50 2024-04-26T18:11:48.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.090+0000 7fc756ff5700 1 -- 172.21.15.90:0/3545800390 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c09e6f0 con 0x560ede155d50 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc756ff5700 1 -- 172.21.15.90:0/3545800390 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc764004700 con 0x560ede155d50 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc783b2fb80 10 client.15093 fetch_fsmap finished waiting for FSMap version 25 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc783b2fb80 10 client.15093 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc783b2fb80 20 client.15093 populate_metadata read hostname 'smithi090' 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede1dfd70 0x560ede1dea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc783b2fb80 4 client.15093 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "status"}) -- 0x560ede0b0260 con 0x560ede1dfd70 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc7727fc700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede1dfd70 0x560ede1dea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc7727fc700 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede1dfd70 0x560ede1dea80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ca65a1fe836d97a server_cookie=72e5c8804c0774bf in_seq=0 out_seq=0 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.091+0000 7fc756ff5700 10 client.15093 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc756ff5700 1 -- 172.21.15.90:0/3545800390 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560ede0b0260 con 0x560ede1dfd70 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc756ff5700 10 client.15093 handle_command_reply: tid=0 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 client.15093 shutdown 2024-04-26T18:11:48.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede155d50 msgr2=0x560ede158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede155d50 0x560ede158360 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc764009350 tx=0x7fc764005af0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 shutdown_connections 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560ede1dfd70 0x560ede1dea80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560ede155d50 0x560ede158360 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560ede15ba20 0x560ede17cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 --2- 172.21.15.90:0/3545800390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560ede1566f0 0x560ede1588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc756ff5700 0 client.15093 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 >> 172.21.15.90:0/3545800390 conn(0x560ede084570 msgr2=0x560ede1504d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 shutdown_connections 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 1 -- 172.21.15.90:0/3545800390 wait complete. 2024-04-26T18:11:48.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.092+0000 7fc783b2fb80 20 client.15093 trim_cache size 0 max 16384 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24449, 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 178.732779174, 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 25, 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 92, 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 90, 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 193.43971858099999 2024-04-26T18:11:48.098 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:11:48.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc75c0a3c40 msgr2=0x7fc75c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc75c0a3c40 0x7fc75c0a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc764000c00 tx=0x7fc764005fd0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc774107640 msgr2=0x7fc774198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc774107640 0x7fc774198910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00e580 tx=0x7fc76c00e940 comp rx=0 tx=0).stop 2024-04-26T18:11:48.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 shutdown_connections 2024-04-26T18:11:48.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fc75c0a3c40 0x7fc75c0a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc774107640 0x7fc774198910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc7741025b0 0x7fc77419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 --2- 172.21.15.90:0/402446033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc774102f60 0x7fc77419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 >> 172.21.15.90:0/402446033 conn(0x7fc7740792f0 msgr2=0x7fc7741067d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:48.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 shutdown_connections 2024-04-26T18:11:48.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.093+0000 7fc77929d700 1 -- 172.21.15.90:0/402446033 wait complete. 2024-04-26T18:11:48.712 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T18:11:48.730 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T18:11:48.730 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T18:11:48.731 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T18:11:48.731 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T18:11:48.732 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:11:48.732 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T18:11:48.733 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T18:11:48.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/4243271829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4101930 msgr2=0x7fe9e4101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/4243271829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4101930 0x7fe9e4101d50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe9e0009900 tx=0x7fe9e002a410 comp rx=0 tx=0).stop 2024-04-26T18:11:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/4243271829 shutdown_connections 2024-04-26T18:11:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/4243271829 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e41034e0 0x7fe9e4109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/4243271829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4102b20 0x7fe9e4102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/4243271829 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e4101930 0x7fe9e4101d50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/4243271829 >> 172.21.15.90:0/4243271829 conn(0x7fe9e40fcee0 msgr2=0x7fe9e40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:48.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/4243271829 shutdown_connections 2024-04-26T18:11:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.835+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/4243271829 wait complete. 2024-04-26T18:11:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 Processor -- start 2024-04-26T18:11:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 -- start start 2024-04-26T18:11:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e4101930 0x7fe9e4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4102b20 0x7fe9e4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e41034e0 0x7fe9e41a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e410d3c0 con 0x7fe9e41034e0 2024-04-26T18:11:48.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e410d240 con 0x7fe9e4102b20 2024-04-26T18:11:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9ec4f8640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9e410d540 con 0x7fe9e4101930 2024-04-26T18:11:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e41034e0 0x7fe9e41a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e41034e0 0x7fe9e41a00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59768/0 (socket says 172.21.15.90:59768) 2024-04-26T18:11:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 -- 172.21.15.90:0/431632185 learned_addr learned my addr 172.21.15.90:0/431632185 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9e9a93640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4102b20 0x7fe9e4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 -- 172.21.15.90:0/431632185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e4101930 msgr2=0x7fe9e4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e4101930 0x7fe9e4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 -- 172.21.15.90:0/431632185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4102b20 msgr2=0x7fe9e4199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4102b20 0x7fe9e4199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.836+0000 7fe9eaa95640 1 -- 172.21.15.90:0/431632185 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9e41a0720 con 0x7fe9e41034e0 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.837+0000 7fe9eaa95640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e41034e0 0x7fe9e41a00b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00eaa0 tx=0x7fe9dc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.837+0000 7fe9db7fe640 1 -- 172.21.15.90:0/431632185 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc00cdb0 con 0x7fe9e41034e0 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.837+0000 7fe9db7fe640 1 -- 172.21.15.90:0/431632185 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9dc005020 con 0x7fe9e41034e0 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.837+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9e41a0a10 con 0x7fe9e41034e0 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.837+0000 7fe9db7fe640 1 -- 172.21.15.90:0/431632185 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0116d0 con 0x7fe9e41034e0 2024-04-26T18:11:48.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.837+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9e41a0f10 con 0x7fe9e41034e0 2024-04-26T18:11:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.838+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9e41a11a0 con 0x7fe9e41034e0 2024-04-26T18:11:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.839+0000 7fe9db7fe640 1 -- 172.21.15.90:0/431632185 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe9dc0051d0 con 0x7fe9e41034e0 2024-04-26T18:11:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.839+0000 7fe9db7fe640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9d00a3c50 0x7fe9d00a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.839+0000 7fe9db7fe640 1 -- 172.21.15.90:0/431632185 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fe9dc0d0610 con 0x7fe9e41034e0 2024-04-26T18:11:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.840+0000 7fe9ea294640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9d00a3c50 0x7fe9d00a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:48.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.840+0000 7fe9ea294640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9d00a3c50 0x7fe9d00a6110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00096c0 tx=0x7fe9e0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:48.848 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:48.844+0000 7fe9db7fe640 1 -- 172.21.15.90:0/431632185 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9dc016030 con 0x7fe9e41034e0 2024-04-26T18:11:49.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:49.074+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fe9e41a1410 con 0x7fe9e41034e0 2024-04-26T18:11:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:49 smithi119 ceph-mon[32752]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:49 smithi167 ceph-mon[32751]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:49 smithi090 ceph-mon[29180]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:11:50.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.048+0000 7fe9db7fe640 1 -- 172.21.15.90:0/431632185 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fe9dc09d090 con 0x7fe9e41034e0 2024-04-26T18:11:50.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9d00a3c50 msgr2=0x7fe9d00a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9d00a3c50 0x7fe9d00a6110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe9e00096c0 tx=0x7fe9e0000bc0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e41034e0 msgr2=0x7fe9e41a00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e41034e0 0x7fe9e41a00b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc00eaa0 tx=0x7fe9dc00ef60 comp rx=0 tx=0).stop 2024-04-26T18:11:50.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 shutdown_connections 2024-04-26T18:11:50.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe9d00a3c50 0x7fe9d00a6110 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe9e41034e0 0x7fe9e41a00b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9e4102b20 0x7fe9e4199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 --2- 172.21.15.90:0/431632185 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe9e4101930 0x7fe9e4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.052+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 >> 172.21.15.90:0/431632185 conn(0x7fe9e40fcee0 msgr2=0x7fe9e40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:50.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.053+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 shutdown_connections 2024-04-26T18:11:50.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.053+0000 7fe9ec4f8640 1 -- 172.21.15.90:0/431632185 wait complete. 2024-04-26T18:11:50.069 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T18:11:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- 172.21.15.90:0/2198099045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc108670 msgr2=0x7fb7cc10aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 --2- 172.21.15.90:0/2198099045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc108670 0x7fb7cc10aac0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009ca0 tx=0x7fb7c402a8c0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- 172.21.15.90:0/2198099045 shutdown_connections 2024-04-26T18:11:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 --2- 172.21.15.90:0/2198099045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc108670 0x7fb7cc10aac0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 --2- 172.21.15.90:0/2198099045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7cc105d40 0x7fb7cc108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 --2- 172.21.15.90:0/2198099045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7cc103410 0x7fb7cc105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- 172.21.15.90:0/2198099045 >> 172.21.15.90:0/2198099045 conn(0x7fb7cc0fce00 msgr2=0x7fb7cc0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- 172.21.15.90:0/2198099045 shutdown_connections 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- 172.21.15.90:0/2198099045 wait complete. 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 Processor -- start 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- start start 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc103410 0x7fb7cc102d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7cc105d40 0x7fb7cc1013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7cc108670 0x7fb7cc101900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e4e0 con 0x7fb7cc103410 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e360 con 0x7fb7cc105d40 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.177+0000 7fb7d3339640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc10e660 con 0x7fb7cc108670 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc103410 0x7fb7cc102d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:50.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc103410 0x7fb7cc102d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59786/0 (socket says 172.21.15.90:59786) 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 -- 172.21.15.90:0/1722628280 learned_addr learned my addr 172.21.15.90:0/1722628280 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d08d4640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7cc105d40 0x7fb7cc1013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d18d6640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7cc108670 0x7fb7cc101900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 -- 172.21.15.90:0/1722628280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7cc108670 msgr2=0x7fb7cc101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7cc108670 0x7fb7cc101900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 -- 172.21.15.90:0/1722628280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7cc105d40 msgr2=0x7fb7cc1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7cc105d40 0x7fb7cc1013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 -- 172.21.15.90:0/1722628280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc102190 con 0x7fb7cc103410 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d10d5640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc103410 0x7fb7cc102d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800e9f0 tx=0x7fb7c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7c27fc640 1 -- 172.21.15.90:0/1722628280 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c800cd80 con 0x7fb7cc103410 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7c27fc640 1 -- 172.21.15.90:0/1722628280 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c80108e0 con 0x7fb7cc103410 2024-04-26T18:11:50.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc1a2c10 con 0x7fb7cc103410 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7c27fc640 1 -- 172.21.15.90:0/1722628280 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8018980 con 0x7fb7cc103410 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.178+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc1a31b0 con 0x7fb7cc103410 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.179+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7cc0fe8f0 con 0x7fb7cc103410 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.180+0000 7fb7c27fc640 1 -- 172.21.15.90:0/1722628280 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb7c8010430 con 0x7fb7cc103410 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.181+0000 7fb7c27fc640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7b80a3d20 0x7fb7b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.181+0000 7fb7d08d4640 1 -- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7b80a3d20 msgr2=0x7fb7b80a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.181+0000 7fb7c27fc640 1 -- 172.21.15.90:0/1722628280 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fb7c80d06e0 con 0x7fb7cc103410 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.181+0000 7fb7d08d4640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7b80a3d20 0x7fb7b80a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:11:50.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.185+0000 7fb7c27fc640 1 -- 172.21.15.90:0/1722628280 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7c809d160 con 0x7fb7cc103410 2024-04-26T18:11:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/431632185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:11:50.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: ignoring --setuser ceph since I am not root 2024-04-26T18:11:50.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: ignoring --setgroup ceph since I am not root 2024-04-26T18:11:50.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-mgr[34151]: -- 172.21.15.119:0/3471582536 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564821ca85a0 con 0x564821c7c800 2024-04-26T18:11:50.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:50.231+0000 7fd5b7dda200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:11:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/431632185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:11:50.317 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: ignoring --setuser ceph since I am not root 2024-04-26T18:11:50.318 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: ignoring --setgroup ceph since I am not root 2024-04-26T18:11:50.318 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-mgr[34600]: -- 172.21.15.167:0/3058289982 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647834905a0 con 0x564783464c00 2024-04-26T18:11:50.318 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:50.227+0000 7fb508456200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:11:50.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.368+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb7cc063530 con 0x7fb7cc103410 2024-04-26T18:11:50.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.372+0000 7fb7c27fc640 1 -- 172.21.15.90:0/1722628280 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7fb7c809ca60 con 0x7fb7cc103410 2024-04-26T18:11:50.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7b80a3d20 msgr2=0x7fb7b80a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:11:50.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7b80a3d20 0x7fb7b80a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc103410 msgr2=0x7fb7cc102d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc103410 0x7fb7cc102d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800e9f0 tx=0x7fb7c800eeb0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 shutdown_connections 2024-04-26T18:11:50.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fb7b80a3d20 0x7fb7b80a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb7cc108670 0x7fb7cc101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7cc105d40 0x7fb7cc1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 --2- 172.21.15.90:0/1722628280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb7cc103410 0x7fb7cc102d80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 >> 172.21.15.90:0/1722628280 conn(0x7fb7cc0fce00 msgr2=0x7fb7cc103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:50.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 shutdown_connections 2024-04-26T18:11:50.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.374+0000 7fb7d3339640 1 -- 172.21.15.90:0/1722628280 wait complete. 2024-04-26T18:11:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/431632185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:11:50.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: ignoring --setuser ceph since I am not root 2024-04-26T18:11:50.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: ignoring --setgroup ceph since I am not root 2024-04-26T18:11:50.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:50.235+0000 7f82e7660200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:11:50.384 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T18:11:50.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 -- 172.21.15.90:0/3640469884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714101890 msgr2=0x7f5714101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 --2- 172.21.15.90:0/3640469884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714101890 0x7f5714101cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5704009900 tx=0x7f570402a410 comp rx=0 tx=0).stop 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 -- 172.21.15.90:0/3640469884 shutdown_connections 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 --2- 172.21.15.90:0/3640469884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5714103440 0x7f5714109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 --2- 172.21.15.90:0/3640469884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5714102a80 0x7f5714102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 --2- 172.21.15.90:0/3640469884 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714101890 0x7f5714101cb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 -- 172.21.15.90:0/3640469884 >> 172.21.15.90:0/3640469884 conn(0x7f57140fce00 msgr2=0x7f57140ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 -- 172.21.15.90:0/3640469884 shutdown_connections 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 -- 172.21.15.90:0/3640469884 wait complete. 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 Processor -- start 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 -- start start 2024-04-26T18:11:50.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.487+0000 7f571b36a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5714101890 0x7f571406ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f571b36a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714102a80 0x7f571419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f571b36a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5714103440 0x7f571419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f571b36a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f571410d390 con 0x7f5714102a80 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f571b36a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f571410d210 con 0x7f5714101890 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f571b36a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f571410d510 con 0x7f5714103440 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714102a80 0x7f571419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714102a80 0x7f571419d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59804/0 (socket says 172.21.15.90:59804) 2024-04-26T18:11:50.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 -- 172.21.15.90:0/2855029461 learned_addr learned my addr 172.21.15.90:0/2855029461 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5719106640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5714101890 0x7f571406ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5719907640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5714103440 0x7f571419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 -- 172.21.15.90:0/2855029461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5714103440 msgr2=0x7f571419fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5714103440 0x7f571419fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 -- 172.21.15.90:0/2855029461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5714101890 msgr2=0x7f571406ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5714101890 0x7f571406ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 -- 172.21.15.90:0/2855029461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57141a07a0 con 0x7f5714102a80 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5719106640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5714101890 0x7f571406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f5718905640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714102a80 0x7f571419d910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f571000e9f0 tx=0x7f571000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f570a7fc640 1 -- 172.21.15.90:0/2855029461 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571000cd80 con 0x7f5714102a80 2024-04-26T18:11:50.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f570a7fc640 1 -- 172.21.15.90:0/2855029461 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57100108e0 con 0x7f5714102a80 2024-04-26T18:11:50.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57141a0a90 con 0x7f5714102a80 2024-04-26T18:11:50.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f570a7fc640 1 -- 172.21.15.90:0/2855029461 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710018980 con 0x7f5714102a80 2024-04-26T18:11:50.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.488+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141a0fe0 con 0x7f5714102a80 2024-04-26T18:11:50.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.489+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5714108ca0 con 0x7f5714102a80 2024-04-26T18:11:50.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.490+0000 7f570a7fc640 1 -- 172.21.15.90:0/2855029461 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f5710010430 con 0x7f5714102a80 2024-04-26T18:11:50.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.491+0000 7f570a7fc640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f57000a3ca0 0x7f57000a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:50.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.491+0000 7f5719106640 1 -- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f57000a3ca0 msgr2=0x7f57000a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:50.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.491+0000 7f570a7fc640 1 -- 172.21.15.90:0/2855029461 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f57100d0690 con 0x7f5714102a80 2024-04-26T18:11:50.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.491+0000 7f5719106640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f57000a3ca0 0x7f57000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:11:50.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.495+0000 7f570a7fc640 1 -- 172.21.15.90:0/2855029461 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f571009cac0 con 0x7f5714102a80 2024-04-26T18:11:50.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.678+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f57140635a0 con 0x7f5714102a80 2024-04-26T18:11:50.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.682+0000 7f570a7fc640 1 -- 172.21.15.90:0/2855029461 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f57100a0dc0 con 0x7f5714102a80 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.684+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f57000a3ca0 msgr2=0x7f57000a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.684+0000 7f571b36a640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f57000a3ca0 0x7f57000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.684+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714102a80 msgr2=0x7f571419d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.684+0000 7f571b36a640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714102a80 0x7f571419d910 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f571000e9f0 tx=0x7f571000eeb0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 shutdown_connections 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f57000a3ca0 0x7f57000a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5714103440 0x7f571419fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5714102a80 0x7f571419d910 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 --2- 172.21.15.90:0/2855029461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5714101890 0x7f571406ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 >> 172.21.15.90:0/2855029461 conn(0x7f57140fce00 msgr2=0x7f5714105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:50.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 shutdown_connections 2024-04-26T18:11:50.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:50.685+0000 7f571b36a640 1 -- 172.21.15.90:0/2855029461 wait complete. 2024-04-26T18:11:50.694 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:50.424+0000 7f82e7660200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:11:50.694 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:50.532+0000 7f82e7660200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:11:50.695 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T18:11:50.984 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T18:11:50.984 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T18:11:50.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:50.985 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-26T18:11:50.986 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:50.412+0000 7fb508456200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:11:50.986 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:50.514+0000 7fb508456200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:11:50.986 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:50.414+0000 7fd5b7dda200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:11:50.986 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:50.517+0000 7fd5b7dda200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:11:51.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:51.008 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-04-26T18:11:51.040 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:50.846+0000 7f82e7660200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:11:51.040 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:50 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:50.931+0000 7f82e7660200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:11:51.065 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2024-04-26T18:11:51.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:11:51.067 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-26T18:11:51.067 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T18:11:51.067 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T18:11:51.067 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T18:11:51.303 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:50.825+0000 7fd5b7dda200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:11:51.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:50.910+0000 7fd5b7dda200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:11:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:51 smithi119 ceph-mon[32752]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:51 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/431632185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:11:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:51 smithi119 ceph-mon[32752]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-26T18:11:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:51 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1722628280' entity='client.admin' 2024-04-26T18:11:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:51 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2855029461' entity='client.admin' 2024-04-26T18:11:51.317 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:50.823+0000 7fb508456200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:11:51.317 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:50.908+0000 7fb508456200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:11:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:51 smithi167 ceph-mon[32751]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:51 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/431632185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:11:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:51 smithi167 ceph-mon[32751]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-26T18:11:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:51 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1722628280' entity='client.admin' 2024-04-26T18:11:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:51 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2855029461' entity='client.admin' 2024-04-26T18:11:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:51 smithi090 ceph-mon[29180]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-26T18:11:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:51 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/431632185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:11:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:51 smithi090 ceph-mon[29180]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-26T18:11:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:51 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1722628280' entity='client.admin' 2024-04-26T18:11:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:51 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2855029461' entity='client.admin' 2024-04-26T18:11:51.402 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:11:52.426 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:52.086+0000 7fd5b7dda200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:11:52.426 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:52.256+0000 7fd5b7dda200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:11:52.427 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:52.425+0000 7fd5b7dda200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:11:52.431 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:52.092+0000 7fb508456200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:11:52.431 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:52.261+0000 7fb508456200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:11:52.504 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:52.156+0000 7f82e7660200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:11:52.504 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:52.331+0000 7f82e7660200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:11:52.504 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:52.503+0000 7f82e7660200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:11:52.717 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:52.430+0000 7fb508456200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:11:53.003 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:52.711+0000 7fd5b7dda200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:11:53.003 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:52.803+0000 7fd5b7dda200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:11:53.003 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:53 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:53.003+0000 7fd5b7dda200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:11:53.010 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:52.716+0000 7fb508456200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:11:53.010 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:52.809+0000 7fb508456200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:11:53.103 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:52.799+0000 7f82e7660200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:11:53.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:52.893+0000 7f82e7660200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:11:53.104 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:53 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:53.103+0000 7f82e7660200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:11:53.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 -- 172.21.15.90:0/2061827776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102310 msgr2=0x7f8b1c102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:53.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 --2- 172.21.15.90:0/2061827776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102310 0x7f8b1c102790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c009990 tx=0x7f8b0c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 -- 172.21.15.90:0/2061827776 shutdown_connections 2024-04-26T18:11:53.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 --2- 172.21.15.90:0/2061827776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b1c102cd0 0x7f8b1c10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 --2- 172.21.15.90:0/2061827776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b1c108380 0x7f8b1c108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 --2- 172.21.15.90:0/2061827776 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102310 0x7f8b1c102790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 -- 172.21.15.90:0/2061827776 >> 172.21.15.90:0/2061827776 conn(0x7f8b1c079290 msgr2=0x7f8b1c0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 -- 172.21.15.90:0/2061827776 shutdown_connections 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.146+0000 7f8b220d5700 1 -- 172.21.15.90:0/2061827776 wait complete. 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 Processor -- start 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 -- start start 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b1c102310 0x7f8b1c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102cd0 0x7f8b1c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:53.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b1c108380 0x7f8b1c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c1124d0 con 0x7f8b1c102cd0 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c112350 con 0x7f8b1c102310 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b220d5700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b1c112650 con 0x7f8b1c108380 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b1affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102cd0 0x7f8b1c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b1affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102cd0 0x7f8b1c19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44908/0 (socket says 172.21.15.90:44908) 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b1b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b1c102310 0x7f8b1c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b1affd700 1 -- 172.21.15.90:0/1467784142 learned_addr learned my addr 172.21.15.90:0/1467784142 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.147+0000 7f8b1bfff700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b1c108380 0x7f8b1c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:53.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b1affd700 1 -- 172.21.15.90:0/1467784142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b1c108380 msgr2=0x7f8b1c1a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b1affd700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b1c108380 0x7f8b1c1a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b1affd700 1 -- 172.21.15.90:0/1467784142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b1c102310 msgr2=0x7f8b1c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b1affd700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b1c102310 0x7f8b1c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b1affd700 1 -- 172.21.15.90:0/1467784142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b1c1a5f50 con 0x7f8b1c102cd0 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b1b7fe700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b1c102310 0x7f8b1c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b1affd700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102cd0 0x7f8b1c19aed0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0057f0 tx=0x7f8b0c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b18ff9700 1 -- 172.21.15.90:0/1467784142 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c026070 con 0x7f8b1c102cd0 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b1c1a6120 con 0x7f8b1c102cd0 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b18ff9700 1 -- 172.21.15.90:0/1467784142 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b0c021410 con 0x7f8b1c102cd0 2024-04-26T18:11:53.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b18ff9700 1 -- 172.21.15.90:0/1467784142 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c017650 con 0x7f8b1c102cd0 2024-04-26T18:11:53.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.148+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b1c1a6650 con 0x7f8b1c102cd0 2024-04-26T18:11:53.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.149+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b1c06a110 con 0x7f8b1c102cd0 2024-04-26T18:11:53.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.150+0000 7f8b18ff9700 1 -- 172.21.15.90:0/1467784142 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f8b0c029070 con 0x7f8b1c102cd0 2024-04-26T18:11:53.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.151+0000 7f8b18ff9700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 0x7f8b040a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:53.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.151+0000 7f8b1b7fe700 1 -- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 msgr2=0x7f8b040a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:53.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.151+0000 7f8b18ff9700 1 -- 172.21.15.90:0/1467784142 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f8b0c0d0f20 con 0x7f8b1c102cd0 2024-04-26T18:11:53.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.151+0000 7f8b1b7fe700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 0x7f8b040a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:11:53.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.155+0000 7f8b18ff9700 1 -- 172.21.15.90:0/1467784142 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b0c05dff0 con 0x7f8b1c102cd0 2024-04-26T18:11:53.303 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:53 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:53.196+0000 7fd5b7dda200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:11:53.317 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:53 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:53.010+0000 7fb508456200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:11:53.317 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:53 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:53.202+0000 7fb508456200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:11:53.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.352+0000 7f8b1b7fe700 1 -- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 msgr2=0x7f8b040a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:53.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.352+0000 7f8b1b7fe700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 0x7f8b040a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:11:53.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:53 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:53.328+0000 7f82e7660200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:11:53.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.433+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b1c02e490 con 0x7f8b1c102cd0 2024-04-26T18:11:53.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.434+0000 7f8b18ff9700 1 -- 172.21.15.90:0/1467784142 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f8b0c013070 con 0x7f8b1c102cd0 2024-04-26T18:11:53.434 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 msgr2=0x7f8b040a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 0x7f8b040a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102cd0 msgr2=0x7f8b1c19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102cd0 0x7f8b1c19aed0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c0057f0 tx=0x7f8b0c00bfd0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 shutdown_connections 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f8b040a3d10 0x7f8b040a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b1c108380 0x7f8b1c1a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b1c102310 0x7f8b1c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 --2- 172.21.15.90:0/1467784142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b1c102cd0 0x7f8b1c19aed0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:53.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 >> 172.21.15.90:0/1467784142 conn(0x7f8b1c079290 msgr2=0x7f8b1c101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:53.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 shutdown_connections 2024-04-26T18:11:53.437 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:53.436+0000 7f8b220d5700 1 -- 172.21.15.90:0/1467784142 wait complete. 2024-04-26T18:11:53.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:53 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1467784142' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:11:53.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:53 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1467784142' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:11:53.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:53 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1467784142' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:11:54.053 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:53 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:53.794+0000 7fd5b7dda200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:11:54.067 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:53 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:53.798+0000 7fb508456200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:11:54.098 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:53 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:53.939+0000 7f82e7660200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:11:54.127 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T18:11:54.127 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T18:11:54.127 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-04-26T18:11:54.127 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T18:11:54.127 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T18:11:54.127 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T18:11:54.128 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T18:11:54.128 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T18:11:54.128 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T18:11:54.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:54.128 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-04-26T18:11:54.145 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T18:11:54.145 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T18:11:54.145 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T18:11:54.145 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:11:54.145 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-04-26T18:11:54.145 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:11:54.146 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T18:11:54.147 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42064sec preferred_lft 42064sec 2024-04-26T18:11:54.147 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::8025:e85a:626f:c8e5/64 scope link noprefixroute 2024-04-26T18:11:54.147 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:11:54.147 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T18:11:54.147 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:11:54.148 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T18:11:54.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-04-26T18:11:54.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-04-26T18:11:54.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-04-26T18:11:54.148 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T18:11:54.148 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T18:11:54.235 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:11:54.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:54.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:54.329 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T18:11:54.351 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-04-26T18:11:54.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:54.353 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-04-26T18:11:54.442 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-04-26T18:11:54.442 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T18:11:54.442 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T18:11:54.442 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T18:11:54.442 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T18:11:54.443 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:11:54.443 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T18:11:54.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T18:11:54.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T18:11:54.443 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T18:11:54.443 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T18:11:54.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:11:54.508 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:54.212+0000 7fd5b7dda200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:11:54.508 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:54.308+0000 7fd5b7dda200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:11:54.508 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:54.507+0000 7fd5b7dda200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:11:54.514 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:54 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:54.217+0000 7fb508456200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:11:54.514 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:54 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:54.313+0000 7fb508456200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:11:54.599 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:54 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:54.390+0000 7f82e7660200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:11:54.599 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:54 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:54.494+0000 7f82e7660200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:11:54.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:54.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:54.663 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-26T18:11:54.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:54.679 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-04-26T18:11:54.734 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:11:54.734 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T18:11:54.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T18:11:54.734 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T18:11:54.734 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T18:11:54.817 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:54 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:54.513+0000 7fb508456200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:11:54.822 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:11:54.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:54.851 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T18:11:54.851 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:11:54.851 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T18:11:54.851 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T18:11:54.851 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T18:11:54.851 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T18:11:54.852 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T18:11:54.852 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T18:11:54.852 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T18:11:54.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:54 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:54.712+0000 7f82e7660200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:11:54.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:11:54.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:54.978 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:11:54.978 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T18:11:54.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-04-26T18:11:54.979 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T18:11:54.979 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T18:11:55.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:11:55.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:55.095 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T18:11:55.095 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T18:11:55.109 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T18:11:55.109 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T18:11:55.110 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T18:11:55.164 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T18:11:55.164 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T18:11:55.164 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/85328d59-d769-406c-88c5-4148365dc53c 2024-04-26T18:11:55.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:55.165 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/85328d59-d769-406c-88c5-4148365dc53c /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-26T18:11:55.275 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-26T18:11:55.276 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-26T18:11:55.276 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-04-26T18:11:55.303 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:55.024+0000 7fd5b7dda200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:11:55.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:55.114+0000 7fd5b7dda200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:11:55.317 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:55.034+0000 7fb508456200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:11:55.317 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:55.125+0000 7fb508456200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:11:55.377 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-04-26T18:11:55.377 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-04-26T18:11:55.377 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-04-26T18:11:55.377 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.90:3300,172.21.15.119:3300,172.21.15.167:3300" 2024-04-26T18:11:55.377 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.90:3300,172.21.15.119:3300,172.21.15.167:3300:/volumes/qa/sv_1/85328d59-d769-406c-88c5-4148365dc53c 2024-04-26T18:11:55.377 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=b8878a18-03f6-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T18:11:55.378 INFO:teuthology.orchestra.run.smithi090.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T18:11:55.378 INFO:teuthology.orchestra.run.smithi090.stdout: You just mounted a file system that supports labels which does not 2024-04-26T18:11:55.378 INFO:teuthology.orchestra.run.smithi090.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T18:11:55.378 INFO:teuthology.orchestra.run.smithi090.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T18:11:55.378 INFO:teuthology.orchestra.run.smithi090.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T18:11:55.379 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T18:11:55.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:11:55.380 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T18:11:55.404 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:11:55.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:11:55.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:55.502 INFO:teuthology.orchestra.run.smithi090.stdout:client24733 2024-04-26T18:11:55.502 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:11:55.502 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T18:11:55.519 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:11:55.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:11:55.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:55.616 INFO:teuthology.orchestra.run.smithi090.stdout:client24733 2024-04-26T18:11:55.617 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:11:55.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:55.266+0000 7f82e7660200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:11:55.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:55.365+0000 7f82e7660200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:11:55.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:11:55.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:55.673 INFO:teuthology.orchestra.run.smithi090.stdout:client24733 2024-04-26T18:11:55.673 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b8878a18-03f6-11ef-bc93-c7b262605968.client24733/status') 2024-04-26T18:11:55.704 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:55.357+0000 7fd5b7dda200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:11:55.704 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:55.493+0000 7fd5b7dda200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:11:55.704 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:55.587+0000 7fd5b7dda200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:11:55.704 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:55.703+0000 7fd5b7dda200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:11:55.710 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:55.367+0000 7fb508456200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:11:55.710 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:55.500+0000 7fb508456200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:11:55.710 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:55.594+0000 7fb508456200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:11:55.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:11:55.771 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.24733 (3)172.21.15.90:0/3415564344 2024-04-26T18:11:55.771 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2024-04-26T18:11:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2024-04-26T18:11:55.771 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2024-04-26T18:11:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:71 bytes copied, 2.2332e-05 s, 3.2 MB/s 2024-04-26T18:11:55.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:11:55.779 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T18:11:55.789 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:11:55.790 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T18:11:55.790 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-26T18:11:55.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 -- 172.21.15.90:0/609161591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 msgr2=0x7fe700109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 --2- 172.21.15.90:0/609161591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe700109cd0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009ca0 tx=0x7fe6f802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 -- 172.21.15.90:0/609161591 shutdown_connections 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 --2- 172.21.15.90:0/609161591 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe700109cd0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 --2- 172.21.15.90:0/609161591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700102a80 0x7fe700102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 --2- 172.21.15.90:0/609161591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe700101890 0x7fe700101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 -- 172.21.15.90:0/609161591 >> 172.21.15.90:0/609161591 conn(0x7fe7000fce00 msgr2=0x7fe7000ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 -- 172.21.15.90:0/609161591 shutdown_connections 2024-04-26T18:11:55.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.892+0000 7fe70875f640 1 -- 172.21.15.90:0/609161591 wait complete. 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 Processor -- start 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 -- start start 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700101890 0x7fe700198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe700102a80 0x7fe700199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe7001a00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe70010d3c0 con 0x7fe700103440 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe70010d240 con 0x7fe700101890 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe70875f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe70010d540 con 0x7fe700102a80 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe706cfc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe7001a00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe706cfc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe7001a00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44926/0 (socket says 172.21.15.90:44926) 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe706cfc640 1 -- 172.21.15.90:0/667514241 learned_addr learned my addr 172.21.15.90:0/667514241 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe7064fb640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700101890 0x7fe700198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:55.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe705cfa640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe700102a80 0x7fe700199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe706cfc640 1 -- 172.21.15.90:0/667514241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe700102a80 msgr2=0x7fe700199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe706cfc640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe700102a80 0x7fe700199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.893+0000 7fe706cfc640 1 -- 172.21.15.90:0/667514241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700101890 msgr2=0x7fe700198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe706cfc640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700101890 0x7fe700198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe706cfc640 1 -- 172.21.15.90:0/667514241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7001a0740 con 0x7fe700103440 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe706cfc640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe7001a00d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80097b0 tx=0x7fe6f8031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe6f77fe640 1 -- 172.21.15.90:0/667514241 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8003bf0 con 0x7fe700103440 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe6f77fe640 1 -- 172.21.15.90:0/667514241 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f8003d90 con 0x7fe700103440 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7001a09d0 con 0x7fe700103440 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe6f77fe640 1 -- 172.21.15.90:0/667514241 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f802fc70 con 0x7fe700103440 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.894+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7001a0f20 con 0x7fe700103440 2024-04-26T18:11:55.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.895+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7001a10f0 con 0x7fe700103440 2024-04-26T18:11:55.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.896+0000 7fe6f77fe640 1 -- 172.21.15.90:0/667514241 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fe6f8004040 con 0x7fe700103440 2024-04-26T18:11:55.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.896+0000 7fe6f77fe640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 0x7fe6ec0a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:55.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.896+0000 7fe7064fb640 1 -- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 msgr2=0x7fe6ec0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:55.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.896+0000 7fe6f77fe640 1 -- 172.21.15.90:0/667514241 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7fe6f80f1a00 con 0x7fe700103440 2024-04-26T18:11:55.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.896+0000 7fe7064fb640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 0x7fe6ec0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:11:55.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:55.901+0000 7fe6f77fe640 1 -- 172.21.15.90:0/667514241 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6f80be480 con 0x7fe700103440 2024-04-26T18:11:55.996 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:55.625+0000 7f82e7660200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:11:55.996 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:55.768+0000 7f82e7660200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:11:55.996 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:55.870+0000 7f82e7660200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:11:55.996 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:55.995+0000 7f82e7660200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:11:56.017 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:55.710+0000 7fb508456200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:11:56.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.098+0000 7fe7064fb640 1 -- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 msgr2=0x7fe6ec0a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:56.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.098+0000 7fe7064fb640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 0x7fe6ec0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:11:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.123+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fe7001a1890 con 0x7fe700103440 2024-04-26T18:11:56.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.175+0000 7fe6f77fe640 1 -- 172.21.15.90:0/667514241 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v26) ==== 117+0+0 (secure 0 0 0) 0x7fe6f80bdd80 con 0x7fe700103440 2024-04-26T18:11:56.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 msgr2=0x7fe6ec0a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:11:56.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 0x7fe6ec0a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 msgr2=0x7fe7001a00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:56.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe7001a00d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80097b0 tx=0x7fe6f8031840 comp rx=0 tx=0).stop 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 shutdown_connections 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7fe6ec0a3ca0 0x7fe6ec0a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe700103440 0x7fe7001a00d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe700102a80 0x7fe700199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 --2- 172.21.15.90:0/667514241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700101890 0x7fe700198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 >> 172.21.15.90:0/667514241 conn(0x7fe7000fce00 msgr2=0x7fe7000fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 shutdown_connections 2024-04-26T18:11:56.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.178+0000 7fe70875f640 1 -- 172.21.15.90:0/667514241 wait complete. 2024-04-26T18:11:56.190 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T18:11:56.203 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T18:11:56.211 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:11:56.211 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T18:11:56.212 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T18:11:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:56 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/667514241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T18:11:56.304 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:56 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:56.000+0000 7fd5b7dda200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:11:56.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:56 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/667514241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T18:11:56.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 -- 172.21.15.90:0/1442439846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c102aa0 msgr2=0x7f790c102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:56.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 --2- 172.21.15.90:0/1442439846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c102aa0 0x7f790c102f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009900 tx=0x7f78f802a410 comp rx=0 tx=0).stop 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 -- 172.21.15.90:0/1442439846 shutdown_connections 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 --2- 172.21.15.90:0/1442439846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f790c103460 0x7f790c109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 --2- 172.21.15.90:0/1442439846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c102aa0 0x7f790c102f20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 --2- 172.21.15.90:0/1442439846 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790c1018b0 0x7f790c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 -- 172.21.15.90:0/1442439846 >> 172.21.15.90:0/1442439846 conn(0x7f790c0fce20 msgr2=0x7f790c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 -- 172.21.15.90:0/1442439846 shutdown_connections 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.316+0000 7f791313f640 1 -- 172.21.15.90:0/1442439846 wait complete. 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 Processor -- start 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 -- start start 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f790c1018b0 0x7f790c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790c102aa0 0x7f790c197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:56.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c103460 0x7f790c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10d3e0 con 0x7f790c103460 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10d260 con 0x7f790c1018b0 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f791313f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f790c10d560 con 0x7f790c102aa0 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c103460 0x7f790c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f7910edb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f790c1018b0 0x7f790c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c103460 0x7f790c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44950/0 (socket says 172.21.15.90:44950) 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 -- 172.21.15.90:0/3829685360 learned_addr learned my addr 172.21.15.90:0/3829685360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f790bfff640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790c102aa0 0x7f790c197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790c102aa0 msgr2=0x7f790c197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f790c102aa0 0x7f790c197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f790c1018b0 msgr2=0x7f790c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f790c1018b0 0x7f790c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.317+0000 7f79116dc640 1 -- 172.21.15.90:0/3829685360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f790c19f670 con 0x7f790c103460 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.318+0000 7f79116dc640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c103460 0x7f790c19f000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f790400eb10 tx=0x7f790400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.318+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904004b20 con 0x7f790c103460 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.318+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7904004cc0 con 0x7f790c103460 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.318+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f790c19f960 con 0x7f790c103460 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.318+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7904010670 con 0x7f790c103460 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.318+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f790c19fe80 con 0x7f790c103460 2024-04-26T18:11:56.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.319+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f790c108c50 con 0x7f790c103460 2024-04-26T18:11:56.323 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:56 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:56.016+0000 7fb508456200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:11:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:56 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/667514241' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-26T18:11:56.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.320+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f7904005020 con 0x7f790c103460 2024-04-26T18:11:56.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.321+0000 7f7909ffb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 0x7f78fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:56.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.321+0000 7f7910edb640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 msgr2=0x7f78fc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:56.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.321+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8153+0+0 (secure 0 0 0) 0x7f79040d0940 con 0x7f790c103460 2024-04-26T18:11:56.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.321+0000 7f7910edb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 0x7f78fc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:11:56.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.325+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f790409d320 con 0x7f790c103460 2024-04-26T18:11:56.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.522+0000 7f7910edb640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 msgr2=0x7f78fc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:56.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.522+0000 7f7910edb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 0x7f78fc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:11:56.528 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.528+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 --> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f790c062d70 con 0x7f78fc0a3d20 2024-04-26T18:11:56.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:56.311+0000 7f82e7660200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:11:56.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.923+0000 7f7910edb640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 msgr2=0x7f78fc0a61e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/3660618461 2024-04-26T18:11:56.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:56.923+0000 7f7910edb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 0x7f78fc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:11:57.053 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:56 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:56.605+0000 7fd5b7dda200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:11:57.053 INFO:journalctl@ceph.mgr.y.smithi119.stdout:Apr 26 18:11:56 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-y[34147]: 2024-04-26T18:11:56.818+0000 7fd5b7dda200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:11:57.067 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:56 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:56.618+0000 7fb508456200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:11:57.067 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Apr 26 18:11:56 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-z[34596]: 2024-04-26T18:11:56.835+0000 7fb508456200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:11:57.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:57.164+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 20) ==== 138331+0+0 (secure 0 0 0) 0x7f7904099290 con 0x7f790c103460 2024-04-26T18:11:57.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:57.165+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 msgr2=0x7f78fc0a61e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:11:57.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:57.165+0000 7f7909ffb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/3660618461,v1:172.21.15.90:6801/3660618461] conn(0x7f78fc0a3d20 0x7f78fc0a61e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:57.184 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:56.927+0000 7f82e7660200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:11:57.184 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.151+0000 7f82e7660200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:11:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/667514241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T18:11:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:11:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: Standby manager daemon y restarted 2024-04-26T18:11:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: Standby manager daemon y started 2024-04-26T18:11:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: Standby manager daemon z restarted 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: Standby manager daemon z started 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: Active manager daemon x restarted 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: Activating manager daemon x 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:11:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: mgrmap e20: x(active, starting, since 0.0121438s), standbys: z, y 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:11:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:11:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:11:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:11:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:11:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:11:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:11:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:11:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/667514241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: Standby manager daemon y restarted 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: Standby manager daemon y started 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: Standby manager daemon z restarted 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: Standby manager daemon z started 2024-04-26T18:11:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: Active manager daemon x restarted 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: Activating manager daemon x 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: mgrmap e20: x(active, starting, since 0.0121438s), standbys: z, y 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:11:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:11:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:11:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:11:57.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.399+0000 7f82497c1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.399+0000 7f82497c1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.399+0000 7f82497c1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.399+0000 7f82497c1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.399+0000 7f82497c1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.400+0000 7f82457b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.400+0000 7f82457b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.400+0000 7f82457b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.400+0000 7f82457b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:11:57.400+0000 7f82457b9700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/667514241' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: Standby manager daemon y restarted 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: Standby manager daemon y started 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:11:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.119:0/3319025505' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: Standby manager daemon z restarted 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: Standby manager daemon z started 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.? 172.21.15.167:0/4087292380' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: Active manager daemon x restarted 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: Activating manager daemon x 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: mgrmap e20: x(active, starting, since 0.0121438s), standbys: z, y 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:11:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:11:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:11:57.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:11:57.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:11:57.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:11:57.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:11:57.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:11:58.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.171+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f79040ce4c0 con 0x7f790c103460 2024-04-26T18:11:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.172+0000 7f7909ffb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78fc052a50 0x7f78fc052e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.172+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f78fc05edd0 con 0x7f78fc052a50 2024-04-26T18:11:58.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.176+0000 7f7910edb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78fc052a50 0x7f78fc052e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.180+0000 7f7910edb640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78fc052a50 0x7f78fc052e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f790c102900 tx=0x7f7900005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:58.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.250+0000 7f7909ffb640 1 -- 172.21.15.90:0/3829685360 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f78fc05edd0 con 0x7f78fc052a50 2024-04-26T18:11:58.251 INFO:teuthology.orchestra.run.smithi090.stdout:Schedule set for path / 2024-04-26T18:11:58.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78fc052a50 msgr2=0x7f78fc052e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78fc052a50 0x7f78fc052e30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f790c102900 tx=0x7f7900005f90 comp rx=0 tx=0).stop 2024-04-26T18:11:58.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c103460 msgr2=0x7f790c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 --2- 172.21.15.90:0/3829685360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f790c103460 0x7f790c19f000 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f790400eb10 tx=0x7f790400c2d0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f79116dc640 1 -- 172.21.15.90:0/3829685360 reap_dead start 2024-04-26T18:11:58.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 shutdown_connections 2024-04-26T18:11:58.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 >> 172.21.15.90:0/3829685360 conn(0x7f790c0fce20 msgr2=0x7f790c105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:58.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 shutdown_connections 2024-04-26T18:11:58.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.253+0000 7f791313f640 1 -- 172.21.15.90:0/3829685360 wait complete. 2024-04-26T18:11:58.266 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T18:11:58.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 -- 172.21.15.90:0/3313223482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20103470 msgr2=0x7fef20105860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 --2- 172.21.15.90:0/3313223482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20103470 0x7fef20105860 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fef10009900 tx=0x7fef1002a410 comp rx=0 tx=0).stop 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 -- 172.21.15.90:0/3313223482 shutdown_connections 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 --2- 172.21.15.90:0/3313223482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef201086d0 0x7fef2010aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 --2- 172.21.15.90:0/3313223482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef20105da0 0x7fef20108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 --2- 172.21.15.90:0/3313223482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20103470 0x7fef20105860 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 -- 172.21.15.90:0/3313223482 >> 172.21.15.90:0/3313223482 conn(0x7fef200fce00 msgr2=0x7fef200ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 -- 172.21.15.90:0/3313223482 shutdown_connections 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.370+0000 7fef25a7b640 1 -- 172.21.15.90:0/3313223482 wait complete. 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 Processor -- start 2024-04-26T18:11:58.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 -- start start 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef20103470 0x7fef20198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20105da0 0x7fef20199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef201086d0 0x7fef201a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fef2010e4e0 con 0x7fef20105da0 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fef2010e360 con 0x7fef20103470 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef25a7b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef2010e660 con 0x7fef201086d0 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef1effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20105da0 0x7fef20199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef1effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20105da0 0x7fef20199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45082/0 (socket says 172.21.15.90:45082) 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef1effd640 1 -- 172.21.15.90:0/187105837 learned_addr learned my addr 172.21.15.90:0/187105837 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef1ffff640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef201086d0 0x7fef201a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.371+0000 7fef1f7fe640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef20103470 0x7fef20198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1f7fe640 1 -- 172.21.15.90:0/187105837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef201086d0 msgr2=0x7fef201a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1f7fe640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef201086d0 0x7fef201a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1f7fe640 1 -- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20105da0 msgr2=0x7fef20199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1f7fe640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20105da0 0x7fef20199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1f7fe640 1 -- 172.21.15.90:0/187105837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef201a07e0 con 0x7fef20103470 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1ffff640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef201086d0 0x7fef201a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1f7fe640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef20103470 0x7fef20198bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef10006010 tx=0x7fef100042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1cff9640 1 -- 172.21.15.90:0/187105837 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1003f030 con 0x7fef20103470 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef201a0a70 con 0x7fef20103470 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1cff9640 1 -- 172.21.15.90:0/187105837 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef10031910 con 0x7fef20103470 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef1cff9640 1 -- 172.21.15.90:0/187105837 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10035070 con 0x7fef20103470 2024-04-26T18:11:58.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.372+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef201a0fb0 con 0x7fef20103470 2024-04-26T18:11:58.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.373+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef200fe8f0 con 0x7fef20103470 2024-04-26T18:11:58.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.374+0000 7fef1cff9640 1 -- 172.21.15.90:0/187105837 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fef10042030 con 0x7fef20103470 2024-04-26T18:11:58.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.375+0000 7fef1cff9640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fef080a3a60 0x7fef080a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.375+0000 7fef1effd640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fef080a3a60 0x7fef080a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.375+0000 7fef1cff9640 1 -- 172.21.15.90:0/187105837 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fef100f0740 con 0x7fef20103470 2024-04-26T18:11:58.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.375+0000 7fef1effd640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fef080a3a60 0x7fef080a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef2019a0d0 tx=0x7fef14005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:58.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.383+0000 7fef1cff9640 1 -- 172.21.15.90:0/187105837 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef100bd320 con 0x7fef20103470 2024-04-26T18:11:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: Manager daemon x is now available 2024-04-26T18:11:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:11:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:11:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:11:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:58 smithi119 ceph-mon[32752]: mgrmap e21: x(active, since 1.02134s), standbys: z, y 2024-04-26T18:11:58.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: Manager daemon x is now available 2024-04-26T18:11:58.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:11:58.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:11:58.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:11:58.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:58 smithi167 ceph-mon[32751]: mgrmap e21: x(active, since 1.02134s), standbys: z, y 2024-04-26T18:11:58.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.594+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fef20062ed0 con 0x7fef080a3a60 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: Manager daemon x is now available 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:11:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:58 smithi090 ceph-mon[29180]: mgrmap e21: x(active, since 1.02134s), standbys: z, y 2024-04-26T18:11:58.655 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.655+0000 7fef1cff9640 1 -- 172.21.15.90:0/187105837 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fef20062ed0 con 0x7fef080a3a60 2024-04-26T18:11:58.656 INFO:teuthology.orchestra.run.smithi090.stdout:Retention added to path / 2024-04-26T18:11:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fef080a3a60 msgr2=0x7fef080a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fef080a3a60 0x7fef080a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef2019a0d0 tx=0x7fef14005f90 comp rx=0 tx=0).stop 2024-04-26T18:11:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef20103470 msgr2=0x7fef20198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef20103470 0x7fef20198bc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fef10006010 tx=0x7fef100042f0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.658 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 shutdown_connections 2024-04-26T18:11:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fef080a3a60 0x7fef080a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef201086d0 0x7fef201a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fef20105da0 0x7fef20199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 --2- 172.21.15.90:0/187105837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef20103470 0x7fef20198bc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 >> 172.21.15.90:0/187105837 conn(0x7fef200fce00 msgr2=0x7fef20103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 shutdown_connections 2024-04-26T18:11:58.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.658+0000 7fef25a7b640 1 -- 172.21.15.90:0/187105837 wait complete. 2024-04-26T18:11:58.669 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T18:11:58.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 -- 172.21.15.90:0/1935169369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734102560 msgr2=0x7f77341029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 --2- 172.21.15.90:0/1935169369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734102560 0x7f77341029e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f772c009ca0 tx=0x7f772c02a8c0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 -- 172.21.15.90:0/1935169369 shutdown_connections 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 --2- 172.21.15.90:0/1935169369 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734102560 0x7f77341029e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 --2- 172.21.15.90:0/1935169369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7734101bc0 0x7f7734102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 --2- 172.21.15.90:0/1935169369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734101260 0x7f7734101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 -- 172.21.15.90:0/1935169369 >> 172.21.15.90:0/1935169369 conn(0x7f77340fce00 msgr2=0x7f77340ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 -- 172.21.15.90:0/1935169369 shutdown_connections 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 -- 172.21.15.90:0/1935169369 wait complete. 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 Processor -- start 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.776+0000 7f773a9e4640 1 -- start start 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7734101260 0x7f773419ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734101bc0 0x7f773419b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734102560 0x7f77341a21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7734110690 con 0x7f7734101bc0 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7734110510 con 0x7f7734102560 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7734110810 con 0x7f7734101260 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734101bc0 0x7f773419b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734101bc0 0x7f773419b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45096/0 (socket says 172.21.15.90:45096) 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 -- 172.21.15.90:0/2773968759 learned_addr learned my addr 172.21.15.90:0/2773968759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f7738f81640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734102560 0x7f77341a21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f7733fff640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7734101260 0x7f773419ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 -- 172.21.15.90:0/2773968759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7734101260 msgr2=0x7f773419ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7734101260 0x7f773419ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 -- 172.21.15.90:0/2773968759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734102560 msgr2=0x7f77341a21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734102560 0x7f77341a21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 -- 172.21.15.90:0/2773968759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77341a2880 con 0x7f7734101bc0 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f7738f81640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734102560 0x7f77341a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f7733fff640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7734101260 0x7f773419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77337fe640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734101bc0 0x7f773419b1b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f772800c920 tx=0x7f772800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77317fa640 1 -- 172.21.15.90:0/2773968759 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728007c70 con 0x7f7734101bc0 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77341a2b70 con 0x7f7734101bc0 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77317fa640 1 -- 172.21.15.90:0/2773968759 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7728007e10 con 0x7f7734101bc0 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f77317fa640 1 -- 172.21.15.90:0/2773968759 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7728018820 con 0x7f7734101bc0 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.777+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77341a3110 con 0x7f7734101bc0 2024-04-26T18:11:58.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.778+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f773410bfb0 con 0x7f7734101bc0 2024-04-26T18:11:58.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.779+0000 7f77317fa640 1 -- 172.21.15.90:0/2773968759 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7f772800f450 con 0x7f7734101bc0 2024-04-26T18:11:58.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.780+0000 7f77317fa640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f771c0a3a60 0x7f771c0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:58.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.780+0000 7f7733fff640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f771c0a3a60 0x7f771c0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:58.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.780+0000 7f77317fa640 1 -- 172.21.15.90:0/2773968759 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f77280d0250 con 0x7f7734101bc0 2024-04-26T18:11:58.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.781+0000 7f7733fff640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f771c0a3a60 0x7f771c0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f773419c140 tx=0x7f7724005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:58.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.785+0000 7f77317fa640 1 -- 172.21.15.90:0/2773968759 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7728014030 con 0x7f7734101bc0 2024-04-26T18:11:58.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.988+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7734062d70 con 0x7f771c0a3a60 2024-04-26T18:11:58.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.995+0000 7f77317fa640 1 -- 172.21.15.90:0/2773968759 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f7734062d70 con 0x7f771c0a3a60 2024-04-26T18:11:58.995 INFO:teuthology.orchestra.run.smithi090.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T18:11:58", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T18:11:58.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.997+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f771c0a3a60 msgr2=0x7f771c0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.997+0000 7f773a9e4640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f771c0a3a60 0x7f771c0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f773419c140 tx=0x7f7724005f90 comp rx=0 tx=0).stop 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.997+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734101bc0 msgr2=0x7f773419b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.997+0000 7f773a9e4640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734101bc0 0x7f773419b1b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f772800c920 tx=0x7f772800cde0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 shutdown_connections 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f771c0a3a60 0x7f771c0a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7734102560 0x7f77341a21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7734101bc0 0x7f773419b1b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 --2- 172.21.15.90:0/2773968759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7734101260 0x7f773419ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 >> 172.21.15.90:0/2773968759 conn(0x7f77340fce00 msgr2=0x7f77340fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 shutdown_connections 2024-04-26T18:11:58.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:58.998+0000 7f773a9e4640 1 -- 172.21.15.90:0/2773968759 wait complete. 2024-04-26T18:11:59.009 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T18:11:59.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- 172.21.15.90:0/1904116770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818103440 msgr2=0x7fd818109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 --2- 172.21.15.90:0/1904116770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818103440 0x7fd818109cd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd810009ca0 tx=0x7fd81002a8c0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- 172.21.15.90:0/1904116770 shutdown_connections 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 --2- 172.21.15.90:0/1904116770 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818103440 0x7fd818109cd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 --2- 172.21.15.90:0/1904116770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818102a80 0x7fd818102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 --2- 172.21.15.90:0/1904116770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd818101890 0x7fd818101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- 172.21.15.90:0/1904116770 >> 172.21.15.90:0/1904116770 conn(0x7fd8180fce00 msgr2=0x7fd8180ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- 172.21.15.90:0/1904116770 shutdown_connections 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- 172.21.15.90:0/1904116770 wait complete. 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 Processor -- start 2024-04-26T18:11:59.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- start start 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101890 0x7fd81819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818102a80 0x7fd81819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd818103440 0x7fd8181a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd81810e4e0 con 0x7fd818102a80 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd81810e360 con 0x7fd818103440 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.114+0000 7fd81d766640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd81810e660 con 0x7fd818101890 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818102a80 0x7fd81819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818102a80 0x7fd81819b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45112/0 (socket says 172.21.15.90:45112) 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd816ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101890 0x7fd81819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 -- 172.21.15.90:0/2632814984 learned_addr learned my addr 172.21.15.90:0/2632814984 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd8177fe640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd818103440 0x7fd8181a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 -- 172.21.15.90:0/2632814984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101890 msgr2=0x7fd81819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:59.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101890 0x7fd81819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 -- 172.21.15.90:0/2632814984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd818103440 msgr2=0x7fd8181a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd818103440 0x7fd8181a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 -- 172.21.15.90:0/2632814984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8181a2930 con 0x7fd818102a80 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd80ffff640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818102a80 0x7fd81819b260 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd80000d770 tx=0x7fd80000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800017070 con 0x7fd818102a80 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8181a2c20 con 0x7fd818102a80 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8000054d0 con 0x7fd818102a80 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800004c30 con 0x7fd818102a80 2024-04-26T18:11:59.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.115+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8181a3120 con 0x7fd818102a80 2024-04-26T18:11:59.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.116+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8181a32f0 con 0x7fd818102a80 2024-04-26T18:11:59.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.117+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 21) ==== 138458+0+0 (secure 0 0 0) 0x7fd800004df0 con 0x7fd818102a80 2024-04-26T18:11:59.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.118+0000 7fd814ff9640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd8040a3ae0 0x7fd8040a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:11:59.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.118+0000 7fd816ffd640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd8040a3ae0 0x7fd8040a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:11:59.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.118+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd8000cfb50 con 0x7fd818102a80 2024-04-26T18:11:59.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.118+0000 7fd816ffd640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd8040a3ae0 0x7fd8040a5fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd81819c1f0 tx=0x7fd80800b410 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:11:59.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.122+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd80009c0e0 con 0x7fd818102a80 2024-04-26T18:11:59.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.200+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7fd8000986c0 con 0x7fd818102a80 2024-04-26T18:11:59.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.337+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd81810bdb0 con 0x7fd8040a3ae0 2024-04-26T18:11:59.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.344+0000 7fd814ff9640 1 -- 172.21.15.90:0/2632814984 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd81810bdb0 con 0x7fd8040a3ae0 2024-04-26T18:11:59.344 INFO:teuthology.orchestra.run.smithi090.stdout:/ 1m 6m3h 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.346+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd8040a3ae0 msgr2=0x7fd8040a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.346+0000 7fd81d766640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd8040a3ae0 0x7fd8040a5fa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd81819c1f0 tx=0x7fd80800b410 comp rx=0 tx=0).stop 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.346+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818102a80 msgr2=0x7fd81819b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.346+0000 7fd81d766640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818102a80 0x7fd81819b260 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd80000d770 tx=0x7fd80000dc30 comp rx=0 tx=0).stop 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 shutdown_connections 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd8040a3ae0 0x7fd8040a5fa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd818103440 0x7fd8181a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd818102a80 0x7fd81819b260 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 --2- 172.21.15.90:0/2632814984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd818101890 0x7fd81819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:11:59.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 >> 172.21.15.90:0/2632814984 conn(0x7fd8180fce00 msgr2=0x7fd81810a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:11:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 shutdown_connections 2024-04-26T18:11:59.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:11:59.347+0000 7fd81d766640 1 -- 172.21.15.90:0/2632814984 wait complete. 2024-04-26T18:11:59.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:59 smithi090 ceph-mon[29180]: [26/Apr/2024:18:11:58] ENGINE Bus STARTING 2024-04-26T18:11:59.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:59 smithi090 ceph-mon[29180]: [26/Apr/2024:18:11:58] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T18:11:59.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:59 smithi090 ceph-mon[29180]: [26/Apr/2024:18:11:58] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T18:11:59.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:59 smithi090 ceph-mon[29180]: [26/Apr/2024:18:11:58] ENGINE Bus STARTED 2024-04-26T18:11:59.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:59 smithi090 ceph-mon[29180]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:59.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:11:59 smithi090 ceph-mon[29180]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:59.359 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T18:11:59.387 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T18:11:59.397 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T18:11:59.407 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T18:11:59.407 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T18:11:59.431 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-26T18:11:59.446 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:11:59.447 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:11:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:59 smithi119 ceph-mon[32752]: [26/Apr/2024:18:11:58] ENGINE Bus STARTING 2024-04-26T18:11:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:59 smithi119 ceph-mon[32752]: [26/Apr/2024:18:11:58] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T18:11:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:59 smithi119 ceph-mon[32752]: [26/Apr/2024:18:11:58] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T18:11:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:59 smithi119 ceph-mon[32752]: [26/Apr/2024:18:11:58] ENGINE Bus STARTED 2024-04-26T18:11:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:59 smithi119 ceph-mon[32752]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:11:59 smithi119 ceph-mon[32752]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:59 smithi167 ceph-mon[32751]: [26/Apr/2024:18:11:58] ENGINE Bus STARTING 2024-04-26T18:11:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:59 smithi167 ceph-mon[32751]: [26/Apr/2024:18:11:58] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T18:11:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:59 smithi167 ceph-mon[32751]: [26/Apr/2024:18:11:58] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T18:11:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:59 smithi167 ceph-mon[32751]: [26/Apr/2024:18:11:58] ENGINE Bus STARTED 2024-04-26T18:11:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:59 smithi167 ceph-mon[32751]: from='client.24815 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:11:59 smithi167 ceph-mon[32751]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:11:59.725 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:12:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-26T18:12:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: mgrmap e22: x(active, since 2s), standbys: z, y 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: from='client.15201 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:12:00.665+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:12:01.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:12:00.665+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:12:01.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:12:00.665+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:12:01.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:12:00.665+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:12:01.035 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:12:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:12:00.665+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:12:01.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.571+0000 7f76f8926700 1 -- 172.21.15.90:0/2663136780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4108390 msgr2=0x7f76f4108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:01.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.571+0000 7f76f8926700 1 --2- 172.21.15.90:0/2663136780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4108390 0x7f76f4108770 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009990 tx=0x7f76e4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 -- 172.21.15.90:0/2663136780 shutdown_connections 2024-04-26T18:12:01.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 --2- 172.21.15.90:0/2663136780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4108390 0x7f76f4108770 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 --2- 172.21.15.90:0/2663136780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f4102ce0 0x7f76f410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 --2- 172.21.15.90:0/2663136780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4102320 0x7f76f41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 -- 172.21.15.90:0/2663136780 >> 172.21.15.90:0/2663136780 conn(0x7f76f40792d0 msgr2=0x7f76f40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 -- 172.21.15.90:0/2663136780 shutdown_connections 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 -- 172.21.15.90:0/2663136780 wait complete. 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 Processor -- start 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.572+0000 7f76f8926700 1 -- start start 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f8926700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f4102320 0x7f76f419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f8926700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4102ce0 0x7f76f419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f8926700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4108390 0x7f76f41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f8926700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4112530 con 0x7f76f4102ce0 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f8926700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41123b0 con 0x7f76f4108390 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f8926700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41126b0 con 0x7f76f4102320 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f1d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4102ce0 0x7f76f419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:01.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4108390 0x7f76f41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4108390 0x7f76f41999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:54384/0 (socket says 172.21.15.90:54384) 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 -- 172.21.15.90:0/2507641123 learned_addr learned my addr 172.21.15.90:0/2507641123 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f259c700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f4102320 0x7f76f419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f1d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4102ce0 0x7f76f419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50294/0 (socket says 172.21.15.90:50294) 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 -- 172.21.15.90:0/2507641123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f4102320 msgr2=0x7f76f419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f4102320 0x7f76f419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 -- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4102ce0 msgr2=0x7f76f419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4102ce0 0x7f76f419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 -- 172.21.15.90:0/2507641123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f419a100 con 0x7f76f4108390 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f1d9b700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4102ce0 0x7f76f419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f259c700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f4102320 0x7f76f419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.573+0000 7f76f2d9d700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4108390 0x7f76f41999a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00ef30 tx=0x7f76ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.574+0000 7f76e37fe700 1 -- 172.21.15.90:0/2507641123 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec0092e0 con 0x7f76f4108390 2024-04-26T18:12:01.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.574+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f419a390 con 0x7f76f4108390 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.574+0000 7f76e37fe700 1 -- 172.21.15.90:0/2507641123 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76ec004bb0 con 0x7f76f4108390 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.574+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41a66b0 con 0x7f76f4108390 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.574+0000 7f76e37fe700 1 -- 172.21.15.90:0/2507641123 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec004020 con 0x7f76f4108390 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.575+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f406a110 con 0x7f76f4108390 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.576+0000 7f76e37fe700 1 -- 172.21.15.90:0/2507641123 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) ==== 138766+0+0 (secure 0 0 0) 0x7f76ec01b030 con 0x7f76f4108390 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.576+0000 7f76e37fe700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76dc0a3db0 0x7f76dc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.577+0000 7f76f259c700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76dc0a3db0 0x7f76dc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.577+0000 7f76e37fe700 1 -- 172.21.15.90:0/2507641123 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f76ec0d0b70 con 0x7f76f4108390 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.577+0000 7f76f259c700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76dc0a3db0 0x7f76dc0a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76f419aae0 tx=0x7f76e401a040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:01.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.581+0000 7f76e37fe700 1 -- 172.21.15.90:0/2507641123 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76ec022020 con 0x7f76f4108390 2024-04-26T18:12:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:12:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:12:01.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.791+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f76f4106140 con 0x7f76dc0a3db0 2024-04-26T18:12:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.800+0000 7f76e37fe700 1 -- 172.21.15.90:0/2507641123 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257745 (secure 0 0 0) 0x7f76f4106140 con 0x7f76dc0a3db0 2024-04-26T18:12:01.804 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:01.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76dc0a3db0 msgr2=0x7f76dc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:01.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76dc0a3db0 0x7f76dc0a6270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f76f419aae0 tx=0x7f76e401a040 comp rx=0 tx=0).stop 2024-04-26T18:12:01.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4108390 msgr2=0x7f76f41999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:01.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4108390 0x7f76f41999a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00ef30 tx=0x7f76ec00c3a0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 shutdown_connections 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76dc0a3db0 0x7f76dc0a6270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76f4108390 0x7f76f41999a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f76f4102320 0x7f76f419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 --2- 172.21.15.90:0/2507641123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f76f4102ce0 0x7f76f419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 >> 172.21.15.90:0/2507641123 conn(0x7f76f40792d0 msgr2=0x7f76f4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 shutdown_connections 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:01.803+0000 7f76f8926700 1 -- 172.21.15.90:0/2507641123 wait complete. 2024-04-26T18:12:01.807 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:12:01.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:12:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:12:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:12:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:12:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:12:02.511 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-26T18:12:01.174163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187739,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1304,"num_read_kb":805,"num_write":479,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380984,"kb_used_data":15100,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560840,"statfs":{"total":1151940427776,"available":1151550300160,"internally_reserved":0,"allocated":15462400,"data_stored":8993155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.988626"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347391+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:11:57.347391+0000","last_peered":"2024-04-26T18:11:57.347391+0000","last_clean":"2024-04-26T18:11:57.347391+0000","last_became_active":"2024-04-26T18:08:47.377408+0000","last_became_peered":"2024-04-26T18:08:47.377408+0000","last_unstale":"2024-04-26T18:11:57.347391+0000","last_undegraded":"2024-04-26T18:11:57.347391+0000","last_fullsized":"2024-04-26T18:11:57.347391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:13.822258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347009+0000","last_change":"2024-04-26T18:08:47.365125+0000","last_active":"2024-04-26T18:11:57.347009+0000","last_peered":"2024-04-26T18:11:57.347009+0000","last_clean":"2024-04-26T18:11:57.347009+0000","last_became_active":"2024-04-26T18:08:47.364847+0000","last_became_peered":"2024-04-26T18:08:47.364847+0000","last_unstale":"2024-04-26T18:11:57.347009+0000","last_undegraded":"2024-04-26T18:11:57.347009+0000","last_fullsized":"2024-04-26T18:11:57.347009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:16:37.890893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346817+0000","last_change":"2024-04-26T18:08:47.377580+0000","last_active":"2024-04-26T18:11:57.346817+0000","last_peered":"2024-04-26T18:11:57.346817+0000","last_clean":"2024-04-26T18:11:57.346817+0000","last_became_active":"2024-04-26T18:08:47.377219+0000","last_became_peered":"2024-04-26T18:08:47.377219+0000","last_unstale":"2024-04-26T18:11:57.346817+0000","last_undegraded":"2024-04-26T18:11:57.346817+0000","last_fullsized":"2024-04-26T18:11:57.346817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:12.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161389+0000","last_change":"2024-04-26T18:08:47.371628+0000","last_active":"2024-04-26T18:11:57.161389+0000","last_peered":"2024-04-26T18:11:57.161389+0000","last_clean":"2024-04-26T18:11:57.161389+0000","last_became_active":"2024-04-26T18:08:47.370840+0000","last_became_peered":"2024-04-26T18:08:47.370840+0000","last_unstale":"2024-04-26T18:11:57.161389+0000","last_undegraded":"2024-04-26T18:11:57.161389+0000","last_fullsized":"2024-04-26T18:11:57.161389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:11.362664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254081+0000","last_change":"2024-04-26T18:08:47.367632+0000","last_active":"2024-04-26T18:11:57.254081+0000","last_peered":"2024-04-26T18:11:57.254081+0000","last_clean":"2024-04-26T18:11:57.254081+0000","last_became_active":"2024-04-26T18:08:47.367316+0000","last_became_peered":"2024-04-26T18:08:47.367316+0000","last_unstale":"2024-04-26T18:11:57.254081+0000","last_undegraded":"2024-04-26T18:11:57.254081+0000","last_fullsized":"2024-04-26T18:11:57.254081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:31:26.665292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207571+0000","last_change":"2024-04-26T18:08:47.369742+0000","last_active":"2024-04-26T18:11:57.207571+0000","last_peered":"2024-04-26T18:11:57.207571+0000","last_clean":"2024-04-26T18:11:57.207571+0000","last_became_active":"2024-04-26T18:08:47.369230+0000","last_became_peered":"2024-04-26T18:08:47.369230+0000","last_unstale":"2024-04-26T18:11:57.207571+0000","last_undegraded":"2024-04-26T18:11:57.207571+0000","last_fullsized":"2024-04-26T18:11:57.207571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:02:04.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346750+0000","last_change":"2024-04-26T18:08:47.370657+0000","last_active":"2024-04-26T18:11:57.346750+0000","last_peered":"2024-04-26T18:11:57.346750+0000","last_clean":"2024-04-26T18:11:57.346750+0000","last_became_active":"2024-04-26T18:08:47.370133+0000","last_became_peered":"2024-04-26T18:08:47.370133+0000","last_unstale":"2024-04-26T18:11:57.346750+0000","last_undegraded":"2024-04-26T18:11:57.346750+0000","last_fullsized":"2024-04-26T18:11:57.346750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:31.232179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234654+0000","last_change":"2024-04-26T18:08:47.366221+0000","last_active":"2024-04-26T18:11:57.234654+0000","last_peered":"2024-04-26T18:11:57.234654+0000","last_clean":"2024-04-26T18:11:57.234654+0000","last_became_active":"2024-04-26T18:08:47.365990+0000","last_became_peered":"2024-04-26T18:08:47.365990+0000","last_unstale":"2024-04-26T18:11:57.234654+0000","last_undegraded":"2024-04-26T18:11:57.234654+0000","last_fullsized":"2024-04-26T18:11:57.234654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:43.707583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346856+0000","last_change":"2024-04-26T18:08:47.373165+0000","last_active":"2024-04-26T18:11:57.346856+0000","last_peered":"2024-04-26T18:11:57.346856+0000","last_clean":"2024-04-26T18:11:57.346856+0000","last_became_active":"2024-04-26T18:08:47.372688+0000","last_became_peered":"2024-04-26T18:08:47.372688+0000","last_unstale":"2024-04-26T18:11:57.346856+0000","last_undegraded":"2024-04-26T18:11:57.346856+0000","last_fullsized":"2024-04-26T18:11:57.346856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:18.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207762+0000","last_change":"2024-04-26T18:09:33.016811+0000","last_active":"2024-04-26T18:11:57.207762+0000","last_peered":"2024-04-26T18:11:57.207762+0000","last_clean":"2024-04-26T18:11:57.207762+0000","last_became_active":"2024-04-26T18:09:33.016267+0000","last_became_peered":"2024-04-26T18:09:33.016267+0000","last_unstale":"2024-04-26T18:11:57.207762+0000","last_undegraded":"2024-04-26T18:11:57.207762+0000","last_fullsized":"2024-04-26T18:11:57.207762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.002914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347272+0000","last_change":"2024-04-26T18:08:47.378643+0000","last_active":"2024-04-26T18:11:57.347272+0000","last_peered":"2024-04-26T18:11:57.347272+0000","last_clean":"2024-04-26T18:11:57.347272+0000","last_became_active":"2024-04-26T18:08:47.378329+0000","last_became_peered":"2024-04-26T18:08:47.378329+0000","last_unstale":"2024-04-26T18:11:57.347272+0000","last_undegraded":"2024-04-26T18:11:57.347272+0000","last_fullsized":"2024-04-26T18:11:57.347272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:59.016025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161243+0000","last_change":"2024-04-26T18:08:47.371663+0000","last_active":"2024-04-26T18:11:57.161243+0000","last_peered":"2024-04-26T18:11:57.161243+0000","last_clean":"2024-04-26T18:11:57.161243+0000","last_became_active":"2024-04-26T18:08:47.371298+0000","last_became_peered":"2024-04-26T18:08:47.371298+0000","last_unstale":"2024-04-26T18:11:57.161243+0000","last_undegraded":"2024-04-26T18:11:57.161243+0000","last_fullsized":"2024-04-26T18:11:57.161243+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:04.884698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160404+0000","last_change":"2024-04-26T18:08:47.365179+0000","last_active":"2024-04-26T18:11:57.160404+0000","last_peered":"2024-04-26T18:11:57.160404+0000","last_clean":"2024-04-26T18:11:57.160404+0000","last_became_active":"2024-04-26T18:08:47.364888+0000","last_became_peered":"2024-04-26T18:08:47.364888+0000","last_unstale":"2024-04-26T18:11:57.160404+0000","last_undegraded":"2024-04-26T18:11:57.160404+0000","last_fullsized":"2024-04-26T18:11:57.160404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:34.082855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346532+0000","last_change":"2024-04-26T18:08:47.364828+0000","last_active":"2024-04-26T18:11:57.346532+0000","last_peered":"2024-04-26T18:11:57.346532+0000","last_clean":"2024-04-26T18:11:57.346532+0000","last_became_active":"2024-04-26T18:08:47.364529+0000","last_became_peered":"2024-04-26T18:08:47.364529+0000","last_unstale":"2024-04-26T18:11:57.346532+0000","last_undegraded":"2024-04-26T18:11:57.346532+0000","last_fullsized":"2024-04-26T18:11:57.346532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.480258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346255+0000","last_change":"2024-04-26T18:08:47.373578+0000","last_active":"2024-04-26T18:11:57.346255+0000","last_peered":"2024-04-26T18:11:57.346255+0000","last_clean":"2024-04-26T18:11:57.346255+0000","last_became_active":"2024-04-26T18:08:47.373263+0000","last_became_peered":"2024-04-26T18:08:47.373263+0000","last_unstale":"2024-04-26T18:11:57.346255+0000","last_undegraded":"2024-04-26T18:11:57.346255+0000","last_fullsized":"2024-04-26T18:11:57.346255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:25.340241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346347+0000","last_change":"2024-04-26T18:08:47.376617+0000","last_active":"2024-04-26T18:11:57.346347+0000","last_peered":"2024-04-26T18:11:57.346347+0000","last_clean":"2024-04-26T18:11:57.346347+0000","last_became_active":"2024-04-26T18:08:47.376290+0000","last_became_peered":"2024-04-26T18:08:47.376290+0000","last_unstale":"2024-04-26T18:11:57.346347+0000","last_undegraded":"2024-04-26T18:11:57.346347+0000","last_fullsized":"2024-04-26T18:11:57.346347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:51.187583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347552+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:11:57.347552+0000","last_peered":"2024-04-26T18:11:57.347552+0000","last_clean":"2024-04-26T18:11:57.347552+0000","last_became_active":"2024-04-26T18:08:47.377427+0000","last_became_peered":"2024-04-26T18:08:47.377427+0000","last_unstale":"2024-04-26T18:11:57.347552+0000","last_undegraded":"2024-04-26T18:11:57.347552+0000","last_fullsized":"2024-04-26T18:11:57.347552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.529500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252164+0000","last_change":"2024-04-26T18:08:47.373314+0000","last_active":"2024-04-26T18:11:57.252164+0000","last_peered":"2024-04-26T18:11:57.252164+0000","last_clean":"2024-04-26T18:11:57.252164+0000","last_became_active":"2024-04-26T18:08:47.372868+0000","last_became_peered":"2024-04-26T18:08:47.372868+0000","last_unstale":"2024-04-26T18:11:57.252164+0000","last_undegraded":"2024-04-26T18:11:57.252164+0000","last_fullsized":"2024-04-26T18:11:57.252164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:19.642655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347122+0000","last_change":"2024-04-26T18:08:47.367593+0000","last_active":"2024-04-26T18:11:57.347122+0000","last_peered":"2024-04-26T18:11:57.347122+0000","last_clean":"2024-04-26T18:11:57.347122+0000","last_became_active":"2024-04-26T18:08:47.367243+0000","last_became_peered":"2024-04-26T18:08:47.367243+0000","last_unstale":"2024-04-26T18:11:57.347122+0000","last_undegraded":"2024-04-26T18:11:57.347122+0000","last_fullsized":"2024-04-26T18:11:57.347122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.380307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252032+0000","last_change":"2024-04-26T18:08:44.339625+0000","last_active":"2024-04-26T18:11:57.252032+0000","last_peered":"2024-04-26T18:11:57.252032+0000","last_clean":"2024-04-26T18:11:57.252032+0000","last_became_active":"2024-04-26T18:08:44.339209+0000","last_became_peered":"2024-04-26T18:08:44.339209+0000","last_unstale":"2024-04-26T18:11:57.252032+0000","last_undegraded":"2024-04-26T18:11:57.252032+0000","last_fullsized":"2024-04-26T18:11:57.252032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:43.169885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234859+0000","last_change":"2024-04-26T18:08:47.375884+0000","last_active":"2024-04-26T18:11:57.234859+0000","last_peered":"2024-04-26T18:11:57.234859+0000","last_clean":"2024-04-26T18:11:57.234859+0000","last_became_active":"2024-04-26T18:08:47.375557+0000","last_became_peered":"2024-04-26T18:08:47.375557+0000","last_unstale":"2024-04-26T18:11:57.234859+0000","last_undegraded":"2024-04-26T18:11:57.234859+0000","last_fullsized":"2024-04-26T18:11:57.234859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.593910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235169+0000","last_change":"2024-04-26T18:08:44.557167+0000","last_active":"2024-04-26T18:11:57.235169+0000","last_peered":"2024-04-26T18:11:57.235169+0000","last_clean":"2024-04-26T18:11:57.235169+0000","last_became_active":"2024-04-26T18:08:44.556716+0000","last_became_peered":"2024-04-26T18:08:44.556716+0000","last_unstale":"2024-04-26T18:11:57.235169+0000","last_undegraded":"2024-04-26T18:11:57.235169+0000","last_fullsized":"2024-04-26T18:11:57.235169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347151+0000","last_change":"2024-04-26T18:08:47.377612+0000","last_active":"2024-04-26T18:11:57.347151+0000","last_peered":"2024-04-26T18:11:57.347151+0000","last_clean":"2024-04-26T18:11:57.347151+0000","last_became_active":"2024-04-26T18:08:47.377231+0000","last_became_peered":"2024-04-26T18:08:47.377231+0000","last_unstale":"2024-04-26T18:11:57.347151+0000","last_undegraded":"2024-04-26T18:11:57.347151+0000","last_fullsized":"2024-04-26T18:11:57.347151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:11.594373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254362+0000","last_change":"2024-04-26T18:08:44.343429+0000","last_active":"2024-04-26T18:11:57.254362+0000","last_peered":"2024-04-26T18:11:57.254362+0000","last_clean":"2024-04-26T18:11:57.254362+0000","last_became_active":"2024-04-26T18:08:44.343103+0000","last_became_peered":"2024-04-26T18:08:44.343103+0000","last_unstale":"2024-04-26T18:11:57.254362+0000","last_undegraded":"2024-04-26T18:11:57.254362+0000","last_fullsized":"2024-04-26T18:11:57.254362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:52.244159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346545+0000","last_change":"2024-04-26T18:08:47.373130+0000","last_active":"2024-04-26T18:11:57.346545+0000","last_peered":"2024-04-26T18:11:57.346545+0000","last_clean":"2024-04-26T18:11:57.346545+0000","last_became_active":"2024-04-26T18:08:47.372726+0000","last_became_peered":"2024-04-26T18:08:47.372726+0000","last_unstale":"2024-04-26T18:11:57.346545+0000","last_undegraded":"2024-04-26T18:11:57.346545+0000","last_fullsized":"2024-04-26T18:11:57.346545+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:44.461177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254325+0000","last_change":"2024-04-26T18:08:44.340152+0000","last_active":"2024-04-26T18:11:57.254325+0000","last_peered":"2024-04-26T18:11:57.254325+0000","last_clean":"2024-04-26T18:11:57.254325+0000","last_became_active":"2024-04-26T18:08:44.339684+0000","last_became_peered":"2024-04-26T18:08:44.339684+0000","last_unstale":"2024-04-26T18:11:57.254325+0000","last_undegraded":"2024-04-26T18:11:57.254325+0000","last_fullsized":"2024-04-26T18:11:57.254325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:49.961201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.164063+0000","last_change":"2024-04-26T18:08:47.375387+0000","last_active":"2024-04-26T18:11:57.164063+0000","last_peered":"2024-04-26T18:11:57.164063+0000","last_clean":"2024-04-26T18:11:57.164063+0000","last_became_active":"2024-04-26T18:08:47.374689+0000","last_became_peered":"2024-04-26T18:08:47.374689+0000","last_unstale":"2024-04-26T18:11:57.164063+0000","last_undegraded":"2024-04-26T18:11:57.164063+0000","last_fullsized":"2024-04-26T18:11:57.164063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:05.618804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'9","reported_seq":216,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.405901+0000","last_change":"2024-04-26T18:08:44.344167+0000","last_active":"2024-04-26T18:11:57.405901+0000","last_peered":"2024-04-26T18:11:57.405901+0000","last_clean":"2024-04-26T18:11:57.405901+0000","last_became_active":"2024-04-26T18:08:44.343646+0000","last_became_peered":"2024-04-26T18:08:44.343646+0000","last_unstale":"2024-04-26T18:11:57.405901+0000","last_undegraded":"2024-04-26T18:11:57.405901+0000","last_fullsized":"2024-04-26T18:11:57.405901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:53.859483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254051+0000","last_change":"2024-04-26T18:08:47.370546+0000","last_active":"2024-04-26T18:11:57.254051+0000","last_peered":"2024-04-26T18:11:57.254051+0000","last_clean":"2024-04-26T18:11:57.254051+0000","last_became_active":"2024-04-26T18:08:47.370220+0000","last_became_peered":"2024-04-26T18:08:47.370220+0000","last_unstale":"2024-04-26T18:11:57.254051+0000","last_undegraded":"2024-04-26T18:11:57.254051+0000","last_fullsized":"2024-04-26T18:11:57.254051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:05.139824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207423+0000","last_change":"2024-04-26T18:08:44.347031+0000","last_active":"2024-04-26T18:11:57.207423+0000","last_peered":"2024-04-26T18:11:57.207423+0000","last_clean":"2024-04-26T18:11:57.207423+0000","last_became_active":"2024-04-26T18:08:44.346650+0000","last_became_peered":"2024-04-26T18:08:44.346650+0000","last_unstale":"2024-04-26T18:11:57.207423+0000","last_undegraded":"2024-04-26T18:11:57.207423+0000","last_fullsized":"2024-04-26T18:11:57.207423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:50.946449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163867+0000","last_change":"2024-04-26T18:08:47.374230+0000","last_active":"2024-04-26T18:11:57.163867+0000","last_peered":"2024-04-26T18:11:57.163867+0000","last_clean":"2024-04-26T18:11:57.163867+0000","last_became_active":"2024-04-26T18:08:47.370780+0000","last_became_peered":"2024-04-26T18:08:47.370780+0000","last_unstale":"2024-04-26T18:11:57.163867+0000","last_undegraded":"2024-04-26T18:11:57.163867+0000","last_fullsized":"2024-04-26T18:11:57.163867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:01.171195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160284+0000","last_change":"2024-04-26T18:08:44.338318+0000","last_active":"2024-04-26T18:11:57.160284+0000","last_peered":"2024-04-26T18:11:57.160284+0000","last_clean":"2024-04-26T18:11:57.160284+0000","last_became_active":"2024-04-26T18:08:44.338009+0000","last_became_peered":"2024-04-26T18:08:44.338009+0000","last_unstale":"2024-04-26T18:11:57.160284+0000","last_undegraded":"2024-04-26T18:11:57.160284+0000","last_fullsized":"2024-04-26T18:11:57.160284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:33.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234730+0000","last_change":"2024-04-26T18:08:47.363924+0000","last_active":"2024-04-26T18:11:57.234730+0000","last_peered":"2024-04-26T18:11:57.234730+0000","last_clean":"2024-04-26T18:11:57.234730+0000","last_became_active":"2024-04-26T18:08:47.363616+0000","last_became_peered":"2024-04-26T18:08:47.363616+0000","last_unstale":"2024-04-26T18:11:57.234730+0000","last_undegraded":"2024-04-26T18:11:57.234730+0000","last_fullsized":"2024-04-26T18:11:57.234730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:27:08.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163592+0000","last_change":"2024-04-26T18:08:44.343350+0000","last_active":"2024-04-26T18:11:57.163592+0000","last_peered":"2024-04-26T18:11:57.163592+0000","last_clean":"2024-04-26T18:11:57.163592+0000","last_became_active":"2024-04-26T18:08:44.343040+0000","last_became_peered":"2024-04-26T18:08:44.343040+0000","last_unstale":"2024-04-26T18:11:57.163592+0000","last_undegraded":"2024-04-26T18:11:57.163592+0000","last_fullsized":"2024-04-26T18:11:57.163592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:09.427785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.251855+0000","last_change":"2024-04-26T18:08:47.360180+0000","last_active":"2024-04-26T18:11:57.251855+0000","last_peered":"2024-04-26T18:11:57.251855+0000","last_clean":"2024-04-26T18:11:57.251855+0000","last_became_active":"2024-04-26T18:08:47.359860+0000","last_became_peered":"2024-04-26T18:08:47.359860+0000","last_unstale":"2024-04-26T18:11:57.251855+0000","last_undegraded":"2024-04-26T18:11:57.251855+0000","last_fullsized":"2024-04-26T18:11:57.251855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:00.532589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":208,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207951+0000","last_change":"2024-04-26T18:08:44.343942+0000","last_active":"2024-04-26T18:11:57.207951+0000","last_peered":"2024-04-26T18:11:57.207951+0000","last_clean":"2024-04-26T18:11:57.207951+0000","last_became_active":"2024-04-26T18:08:44.343581+0000","last_became_peered":"2024-04-26T18:08:44.343581+0000","last_unstale":"2024-04-26T18:11:57.207951+0000","last_undegraded":"2024-04-26T18:11:57.207951+0000","last_fullsized":"2024-04-26T18:11:57.207951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:32.044284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346718+0000","last_change":"2024-04-26T18:08:47.372303+0000","last_active":"2024-04-26T18:11:57.346718+0000","last_peered":"2024-04-26T18:11:57.346718+0000","last_clean":"2024-04-26T18:11:57.346718+0000","last_became_active":"2024-04-26T18:08:47.371961+0000","last_became_peered":"2024-04-26T18:08:47.371961+0000","last_unstale":"2024-04-26T18:11:57.346718+0000","last_undegraded":"2024-04-26T18:11:57.346718+0000","last_fullsized":"2024-04-26T18:11:57.346718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:19.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347265+0000","last_change":"2024-04-26T18:08:44.345681+0000","last_active":"2024-04-26T18:11:57.347265+0000","last_peered":"2024-04-26T18:11:57.347265+0000","last_clean":"2024-04-26T18:11:57.347265+0000","last_became_active":"2024-04-26T18:08:44.345329+0000","last_became_peered":"2024-04-26T18:08:44.345329+0000","last_unstale":"2024-04-26T18:11:57.347265+0000","last_undegraded":"2024-04-26T18:11:57.347265+0000","last_fullsized":"2024-04-26T18:11:57.347265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:59.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347192+0000","last_change":"2024-04-26T18:08:47.370773+0000","last_active":"2024-04-26T18:11:57.347192+0000","last_peered":"2024-04-26T18:11:57.347192+0000","last_clean":"2024-04-26T18:11:57.347192+0000","last_became_active":"2024-04-26T18:08:47.370490+0000","last_became_peered":"2024-04-26T18:08:47.370490+0000","last_unstale":"2024-04-26T18:11:57.347192+0000","last_undegraded":"2024-04-26T18:11:57.347192+0000","last_fullsized":"2024-04-26T18:11:57.347192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:01.935335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.164125+0000","last_change":"2024-04-26T18:08:44.555714+0000","last_active":"2024-04-26T18:11:57.164125+0000","last_peered":"2024-04-26T18:11:57.164125+0000","last_clean":"2024-04-26T18:11:57.164125+0000","last_became_active":"2024-04-26T18:08:44.555184+0000","last_became_peered":"2024-04-26T18:08:44.555184+0000","last_unstale":"2024-04-26T18:11:57.164125+0000","last_undegraded":"2024-04-26T18:11:57.164125+0000","last_fullsized":"2024-04-26T18:11:57.164125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:01.433764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346864+0000","last_change":"2024-04-26T18:09:33.525026+0000","last_active":"2024-04-26T18:11:57.346864+0000","last_peered":"2024-04-26T18:11:57.346864+0000","last_clean":"2024-04-26T18:11:57.346864+0000","last_became_active":"2024-04-26T18:09:33.524587+0000","last_became_peered":"2024-04-26T18:09:33.524587+0000","last_unstale":"2024-04-26T18:11:57.346864+0000","last_undegraded":"2024-04-26T18:11:57.346864+0000","last_fullsized":"2024-04-26T18:11:57.346864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:39:48.534737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":424,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.409028+0000","last_change":"2024-04-26T18:08:44.346346+0000","last_active":"2024-04-26T18:11:57.409028+0000","last_peered":"2024-04-26T18:11:57.409028+0000","last_clean":"2024-04-26T18:11:57.409028+0000","last_became_active":"2024-04-26T18:08:44.345999+0000","last_became_peered":"2024-04-26T18:08:44.345999+0000","last_unstale":"2024-04-26T18:11:57.409028+0000","last_undegraded":"2024-04-26T18:11:57.409028+0000","last_fullsized":"2024-04-26T18:11:57.409028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:27.277880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":359,"num_read_kb":238,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254263+0000","last_change":"2024-04-26T18:08:47.366303+0000","last_active":"2024-04-26T18:11:57.254263+0000","last_peered":"2024-04-26T18:11:57.254263+0000","last_clean":"2024-04-26T18:11:57.254263+0000","last_became_active":"2024-04-26T18:08:47.365861+0000","last_became_peered":"2024-04-26T18:08:47.365861+0000","last_unstale":"2024-04-26T18:11:57.254263+0000","last_undegraded":"2024-04-26T18:11:57.254263+0000","last_fullsized":"2024-04-26T18:11:57.254263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:31.413589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346987+0000","last_change":"2024-04-26T18:09:33.022912+0000","last_active":"2024-04-26T18:11:57.346987+0000","last_peered":"2024-04-26T18:11:57.346987+0000","last_clean":"2024-04-26T18:11:57.346987+0000","last_became_active":"2024-04-26T18:09:33.015390+0000","last_became_peered":"2024-04-26T18:09:33.015390+0000","last_unstale":"2024-04-26T18:11:57.346987+0000","last_undegraded":"2024-04-26T18:11:57.346987+0000","last_fullsized":"2024-04-26T18:11:57.346987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:53.980531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160286+0000","last_change":"2024-04-26T18:08:47.367835+0000","last_active":"2024-04-26T18:11:57.160286+0000","last_peered":"2024-04-26T18:11:57.160286+0000","last_clean":"2024-04-26T18:11:57.160286+0000","last_became_active":"2024-04-26T18:08:47.367517+0000","last_became_peered":"2024-04-26T18:08:47.367517+0000","last_unstale":"2024-04-26T18:11:57.160286+0000","last_undegraded":"2024-04-26T18:11:57.160286+0000","last_fullsized":"2024-04-26T18:11:57.160286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:54.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163733+0000","last_change":"2024-04-26T18:08:44.344161+0000","last_active":"2024-04-26T18:11:57.163733+0000","last_peered":"2024-04-26T18:11:57.163733+0000","last_clean":"2024-04-26T18:11:57.163733+0000","last_became_active":"2024-04-26T18:08:44.343635+0000","last_became_peered":"2024-04-26T18:08:44.343635+0000","last_unstale":"2024-04-26T18:11:57.163733+0000","last_undegraded":"2024-04-26T18:11:57.163733+0000","last_fullsized":"2024-04-26T18:11:57.163733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.946584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346903+0000","last_change":"2024-04-26T18:09:33.524351+0000","last_active":"2024-04-26T18:11:57.346903+0000","last_peered":"2024-04-26T18:11:57.346903+0000","last_clean":"2024-04-26T18:11:57.346903+0000","last_became_active":"2024-04-26T18:09:33.523953+0000","last_became_peered":"2024-04-26T18:09:33.523953+0000","last_unstale":"2024-04-26T18:11:57.346903+0000","last_undegraded":"2024-04-26T18:11:57.346903+0000","last_fullsized":"2024-04-26T18:11:57.346903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:22.887498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":213,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.233757+0000","last_change":"2024-04-26T18:08:44.345278+0000","last_active":"2024-04-26T18:11:57.233757+0000","last_peered":"2024-04-26T18:11:57.233757+0000","last_clean":"2024-04-26T18:11:57.233757+0000","last_became_active":"2024-04-26T18:08:44.344531+0000","last_became_peered":"2024-04-26T18:08:44.344531+0000","last_unstale":"2024-04-26T18:11:57.233757+0000","last_undegraded":"2024-04-26T18:11:57.233757+0000","last_fullsized":"2024-04-26T18:11:57.233757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:02.667622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346598+0000","last_change":"2024-04-26T18:08:47.369542+0000","last_active":"2024-04-26T18:11:57.346598+0000","last_peered":"2024-04-26T18:11:57.346598+0000","last_clean":"2024-04-26T18:11:57.346598+0000","last_became_active":"2024-04-26T18:08:47.369181+0000","last_became_peered":"2024-04-26T18:08:47.369181+0000","last_unstale":"2024-04-26T18:11:57.346598+0000","last_undegraded":"2024-04-26T18:11:57.346598+0000","last_fullsized":"2024-04-26T18:11:57.346598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:56.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254034+0000","last_change":"2024-04-26T18:08:44.555580+0000","last_active":"2024-04-26T18:11:57.254034+0000","last_peered":"2024-04-26T18:11:57.254034+0000","last_clean":"2024-04-26T18:11:57.254034+0000","last_became_active":"2024-04-26T18:08:44.555247+0000","last_became_peered":"2024-04-26T18:08:44.555247+0000","last_unstale":"2024-04-26T18:11:57.254034+0000","last_undegraded":"2024-04-26T18:11:57.254034+0000","last_fullsized":"2024-04-26T18:11:57.254034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:58:55.207355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163660+0000","last_change":"2024-04-26T18:08:47.372300+0000","last_active":"2024-04-26T18:11:57.163660+0000","last_peered":"2024-04-26T18:11:57.163660+0000","last_clean":"2024-04-26T18:11:57.163660+0000","last_became_active":"2024-04-26T18:08:47.371952+0000","last_became_peered":"2024-04-26T18:08:47.371952+0000","last_unstale":"2024-04-26T18:11:57.163660+0000","last_undegraded":"2024-04-26T18:11:57.163660+0000","last_fullsized":"2024-04-26T18:11:57.163660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:58.254691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254389+0000","last_change":"2024-04-26T18:08:44.340926+0000","last_active":"2024-04-26T18:11:57.254389+0000","last_peered":"2024-04-26T18:11:57.254389+0000","last_clean":"2024-04-26T18:11:57.254389+0000","last_became_active":"2024-04-26T18:08:44.340563+0000","last_became_peered":"2024-04-26T18:08:44.340563+0000","last_unstale":"2024-04-26T18:11:57.254389+0000","last_undegraded":"2024-04-26T18:11:57.254389+0000","last_fullsized":"2024-04-26T18:11:57.254389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:57.803017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347296+0000","last_change":"2024-04-26T18:09:33.021890+0000","last_active":"2024-04-26T18:11:57.347296+0000","last_peered":"2024-04-26T18:11:57.347296+0000","last_clean":"2024-04-26T18:11:57.347296+0000","last_became_active":"2024-04-26T18:09:33.021446+0000","last_became_peered":"2024-04-26T18:09:33.021446+0000","last_unstale":"2024-04-26T18:11:57.347296+0000","last_undegraded":"2024-04-26T18:11:57.347296+0000","last_fullsized":"2024-04-26T18:11:57.347296+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:29.158581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252002+0000","last_change":"2024-04-26T18:08:44.555076+0000","last_active":"2024-04-26T18:11:57.252002+0000","last_peered":"2024-04-26T18:11:57.252002+0000","last_clean":"2024-04-26T18:11:57.252002+0000","last_became_active":"2024-04-26T18:08:44.554756+0000","last_became_peered":"2024-04-26T18:08:44.554756+0000","last_unstale":"2024-04-26T18:11:57.252002+0000","last_undegraded":"2024-04-26T18:11:57.252002+0000","last_fullsized":"2024-04-26T18:11:57.252002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:21.950644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207518+0000","last_change":"2024-04-26T18:08:47.366370+0000","last_active":"2024-04-26T18:11:57.207518+0000","last_peered":"2024-04-26T18:11:57.207518+0000","last_clean":"2024-04-26T18:11:57.207518+0000","last_became_active":"2024-04-26T18:08:47.365997+0000","last_became_peered":"2024-04-26T18:08:47.365997+0000","last_unstale":"2024-04-26T18:11:57.207518+0000","last_undegraded":"2024-04-26T18:11:57.207518+0000","last_fullsized":"2024-04-26T18:11:57.207518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:51.643438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347119+0000","last_change":"2024-04-26T18:08:44.339299+0000","last_active":"2024-04-26T18:11:57.347119+0000","last_peered":"2024-04-26T18:11:57.347119+0000","last_clean":"2024-04-26T18:11:57.347119+0000","last_became_active":"2024-04-26T18:08:44.338932+0000","last_became_peered":"2024-04-26T18:08:44.338932+0000","last_unstale":"2024-04-26T18:11:57.347119+0000","last_undegraded":"2024-04-26T18:11:57.347119+0000","last_fullsized":"2024-04-26T18:11:57.347119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:18.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207445+0000","last_change":"2024-04-26T18:08:47.369924+0000","last_active":"2024-04-26T18:11:57.207445+0000","last_peered":"2024-04-26T18:11:57.207445+0000","last_clean":"2024-04-26T18:11:57.207445+0000","last_became_active":"2024-04-26T18:08:47.369048+0000","last_became_peered":"2024-04-26T18:08:47.369048+0000","last_unstale":"2024-04-26T18:11:57.207445+0000","last_undegraded":"2024-04-26T18:11:57.207445+0000","last_fullsized":"2024-04-26T18:11:57.207445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:19.110288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163797+0000","last_change":"2024-04-26T18:08:47.370836+0000","last_active":"2024-04-26T18:11:57.163797+0000","last_peered":"2024-04-26T18:11:57.163797+0000","last_clean":"2024-04-26T18:11:57.163797+0000","last_became_active":"2024-04-26T18:08:47.370160+0000","last_became_peered":"2024-04-26T18:08:47.370160+0000","last_unstale":"2024-04-26T18:11:57.163797+0000","last_undegraded":"2024-04-26T18:11:57.163797+0000","last_fullsized":"2024-04-26T18:11:57.163797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:11:21.285466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":209,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.232853+0000","last_change":"2024-04-26T18:08:44.554960+0000","last_active":"2024-04-26T18:11:57.232853+0000","last_peered":"2024-04-26T18:11:57.232853+0000","last_clean":"2024-04-26T18:11:57.232853+0000","last_became_active":"2024-04-26T18:08:44.554607+0000","last_became_peered":"2024-04-26T18:08:44.554607+0000","last_unstale":"2024-04-26T18:11:57.232853+0000","last_undegraded":"2024-04-26T18:11:57.232853+0000","last_fullsized":"2024-04-26T18:11:57.232853+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:22:26.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235031+0000","last_change":"2024-04-26T18:08:47.372528+0000","last_active":"2024-04-26T18:11:57.235031+0000","last_peered":"2024-04-26T18:11:57.235031+0000","last_clean":"2024-04-26T18:11:57.235031+0000","last_became_active":"2024-04-26T18:08:47.372215+0000","last_became_peered":"2024-04-26T18:08:47.372215+0000","last_unstale":"2024-04-26T18:11:57.235031+0000","last_undegraded":"2024-04-26T18:11:57.235031+0000","last_fullsized":"2024-04-26T18:11:57.235031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:39.366917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161184+0000","last_change":"2024-04-26T18:08:44.342667+0000","last_active":"2024-04-26T18:11:57.161184+0000","last_peered":"2024-04-26T18:11:57.161184+0000","last_clean":"2024-04-26T18:11:57.161184+0000","last_became_active":"2024-04-26T18:08:44.342170+0000","last_became_peered":"2024-04-26T18:08:44.342170+0000","last_unstale":"2024-04-26T18:11:57.161184+0000","last_undegraded":"2024-04-26T18:11:57.161184+0000","last_fullsized":"2024-04-26T18:11:57.161184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:50.117177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346590+0000","last_change":"2024-04-26T18:07:24.173301+0000","last_active":"2024-04-26T18:11:57.346590+0000","last_peered":"2024-04-26T18:11:57.346590+0000","last_clean":"2024-04-26T18:11:57.346590+0000","last_became_active":"2024-04-26T18:07:24.172791+0000","last_became_peered":"2024-04-26T18:07:24.172791+0000","last_unstale":"2024-04-26T18:11:57.346590+0000","last_undegraded":"2024-04-26T18:11:57.346590+0000","last_fullsized":"2024-04-26T18:11:57.346590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_clean_scrub_stamp":"2024-04-26T18:03:58.528256+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:32.468624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160143+0000","last_change":"2024-04-26T18:08:47.370720+0000","last_active":"2024-04-26T18:11:57.160143+0000","last_peered":"2024-04-26T18:11:57.160143+0000","last_clean":"2024-04-26T18:11:57.160143+0000","last_became_active":"2024-04-26T18:08:47.370414+0000","last_became_peered":"2024-04-26T18:08:47.370414+0000","last_unstale":"2024-04-26T18:11:57.160143+0000","last_undegraded":"2024-04-26T18:11:57.160143+0000","last_fullsized":"2024-04-26T18:11:57.160143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:37.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346466+0000","last_change":"2024-04-26T18:08:44.347077+0000","last_active":"2024-04-26T18:11:57.346466+0000","last_peered":"2024-04-26T18:11:57.346466+0000","last_clean":"2024-04-26T18:11:57.346466+0000","last_became_active":"2024-04-26T18:08:44.346481+0000","last_became_peered":"2024-04-26T18:08:44.346481+0000","last_unstale":"2024-04-26T18:11:57.346466+0000","last_undegraded":"2024-04-26T18:11:57.346466+0000","last_fullsized":"2024-04-26T18:11:57.346466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:08.840814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346980+0000","last_change":"2024-04-26T18:09:33.025243+0000","last_active":"2024-04-26T18:11:57.346980+0000","last_peered":"2024-04-26T18:11:57.346980+0000","last_clean":"2024-04-26T18:11:57.346980+0000","last_became_active":"2024-04-26T18:09:33.015430+0000","last_became_peered":"2024-04-26T18:09:33.015430+0000","last_unstale":"2024-04-26T18:11:57.346980+0000","last_undegraded":"2024-04-26T18:11:57.346980+0000","last_fullsized":"2024-04-26T18:11:57.346980+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:54.271435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160587+0000","last_change":"2024-04-26T18:08:47.366552+0000","last_active":"2024-04-26T18:11:57.160587+0000","last_peered":"2024-04-26T18:11:57.160587+0000","last_clean":"2024-04-26T18:11:57.160587+0000","last_became_active":"2024-04-26T18:08:47.366183+0000","last_became_peered":"2024-04-26T18:08:47.366183+0000","last_unstale":"2024-04-26T18:11:57.160587+0000","last_undegraded":"2024-04-26T18:11:57.160587+0000","last_fullsized":"2024-04-26T18:11:57.160587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:36.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'48","reported_seq":100,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.367151+0000","last_change":"2024-04-26T18:08:44.338078+0000","last_active":"2024-04-26T18:11:57.367151+0000","last_peered":"2024-04-26T18:11:57.367151+0000","last_clean":"2024-04-26T18:11:57.367151+0000","last_became_active":"2024-04-26T18:08:44.337715+0000","last_became_peered":"2024-04-26T18:08:44.337715+0000","last_unstale":"2024-04-26T18:11:57.367151+0000","last_undegraded":"2024-04-26T18:11:57.367151+0000","last_fullsized":"2024-04-26T18:11:57.367151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.912993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235147+0000","last_change":"2024-04-26T18:08:47.371321+0000","last_active":"2024-04-26T18:11:57.235147+0000","last_peered":"2024-04-26T18:11:57.235147+0000","last_clean":"2024-04-26T18:11:57.235147+0000","last_became_active":"2024-04-26T18:08:47.371015+0000","last_became_peered":"2024-04-26T18:08:47.371015+0000","last_unstale":"2024-04-26T18:11:57.235147+0000","last_undegraded":"2024-04-26T18:11:57.235147+0000","last_fullsized":"2024-04-26T18:11:57.235147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161387+0000","last_change":"2024-04-26T18:08:44.420101+0000","last_active":"2024-04-26T18:11:57.161387+0000","last_peered":"2024-04-26T18:11:57.161387+0000","last_clean":"2024-04-26T18:11:57.161387+0000","last_became_active":"2024-04-26T18:08:44.419787+0000","last_became_peered":"2024-04-26T18:08:44.419787+0000","last_unstale":"2024-04-26T18:11:57.161387+0000","last_undegraded":"2024-04-26T18:11:57.161387+0000","last_fullsized":"2024-04-26T18:11:57.161387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.485425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347044+0000","last_change":"2024-04-26T18:08:47.372309+0000","last_active":"2024-04-26T18:11:57.347044+0000","last_peered":"2024-04-26T18:11:57.347044+0000","last_clean":"2024-04-26T18:11:57.347044+0000","last_became_active":"2024-04-26T18:08:47.371980+0000","last_became_peered":"2024-04-26T18:08:47.371980+0000","last_unstale":"2024-04-26T18:11:57.347044+0000","last_undegraded":"2024-04-26T18:11:57.347044+0000","last_fullsized":"2024-04-26T18:11:57.347044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:05.011077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346322+0000","last_change":"2024-04-26T18:08:44.555500+0000","last_active":"2024-04-26T18:11:57.346322+0000","last_peered":"2024-04-26T18:11:57.346322+0000","last_clean":"2024-04-26T18:11:57.346322+0000","last_became_active":"2024-04-26T18:08:44.555137+0000","last_became_peered":"2024-04-26T18:08:44.555137+0000","last_unstale":"2024-04-26T18:11:57.346322+0000","last_undegraded":"2024-04-26T18:11:57.346322+0000","last_fullsized":"2024-04-26T18:11:57.346322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:10:05.723654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254027+0000","last_change":"2024-04-26T18:08:47.364215+0000","last_active":"2024-04-26T18:11:57.254027+0000","last_peered":"2024-04-26T18:11:57.254027+0000","last_clean":"2024-04-26T18:11:57.254027+0000","last_became_active":"2024-04-26T18:08:47.363862+0000","last_became_peered":"2024-04-26T18:08:47.363862+0000","last_unstale":"2024-04-26T18:11:57.254027+0000","last_undegraded":"2024-04-26T18:11:57.254027+0000","last_fullsized":"2024-04-26T18:11:57.254027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:27.020100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234809+0000","last_change":"2024-04-26T18:08:44.343506+0000","last_active":"2024-04-26T18:11:57.234809+0000","last_peered":"2024-04-26T18:11:57.234809+0000","last_clean":"2024-04-26T18:11:57.234809+0000","last_became_active":"2024-04-26T18:08:44.343163+0000","last_became_peered":"2024-04-26T18:08:44.343163+0000","last_unstale":"2024-04-26T18:11:57.234809+0000","last_undegraded":"2024-04-26T18:11:57.234809+0000","last_fullsized":"2024-04-26T18:11:57.234809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:09.941391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346893+0000","last_change":"2024-04-26T18:08:47.365572+0000","last_active":"2024-04-26T18:11:57.346893+0000","last_peered":"2024-04-26T18:11:57.346893+0000","last_clean":"2024-04-26T18:11:57.346893+0000","last_became_active":"2024-04-26T18:08:47.364827+0000","last_became_peered":"2024-04-26T18:08:47.364827+0000","last_unstale":"2024-04-26T18:11:57.346893+0000","last_undegraded":"2024-04-26T18:11:57.346893+0000","last_fullsized":"2024-04-26T18:11:57.346893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:44.492380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346591+0000","last_change":"2024-04-26T18:08:44.339063+0000","last_active":"2024-04-26T18:11:57.346591+0000","last_peered":"2024-04-26T18:11:57.346591+0000","last_clean":"2024-04-26T18:11:57.346591+0000","last_became_active":"2024-04-26T18:08:44.338752+0000","last_became_peered":"2024-04-26T18:08:44.338752+0000","last_unstale":"2024-04-26T18:11:57.346591+0000","last_undegraded":"2024-04-26T18:11:57.346591+0000","last_fullsized":"2024-04-26T18:11:57.346591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:38.201263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235138+0000","last_change":"2024-04-26T18:08:47.367859+0000","last_active":"2024-04-26T18:11:57.235138+0000","last_peered":"2024-04-26T18:11:57.235138+0000","last_clean":"2024-04-26T18:11:57.235138+0000","last_became_active":"2024-04-26T18:08:47.367548+0000","last_became_peered":"2024-04-26T18:08:47.367548+0000","last_unstale":"2024-04-26T18:11:57.235138+0000","last_undegraded":"2024-04-26T18:11:57.235138+0000","last_fullsized":"2024-04-26T18:11:57.235138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:13:27.612151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254111+0000","last_change":"2024-04-26T18:08:44.345392+0000","last_active":"2024-04-26T18:11:57.254111+0000","last_peered":"2024-04-26T18:11:57.254111+0000","last_clean":"2024-04-26T18:11:57.254111+0000","last_became_active":"2024-04-26T18:08:44.344578+0000","last_became_peered":"2024-04-26T18:08:44.344578+0000","last_unstale":"2024-04-26T18:11:57.254111+0000","last_undegraded":"2024-04-26T18:11:57.254111+0000","last_fullsized":"2024-04-26T18:11:57.254111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:26.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347131+0000","last_change":"2024-04-26T18:08:47.380099+0000","last_active":"2024-04-26T18:11:57.347131+0000","last_peered":"2024-04-26T18:11:57.347131+0000","last_clean":"2024-04-26T18:11:57.347131+0000","last_became_active":"2024-04-26T18:08:47.379576+0000","last_became_peered":"2024-04-26T18:08:47.379576+0000","last_unstale":"2024-04-26T18:11:57.347131+0000","last_undegraded":"2024-04-26T18:11:57.347131+0000","last_fullsized":"2024-04-26T18:11:57.347131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:28.397886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347023+0000","last_change":"2024-04-26T18:08:44.555299+0000","last_active":"2024-04-26T18:11:57.347023+0000","last_peered":"2024-04-26T18:11:57.347023+0000","last_clean":"2024-04-26T18:11:57.347023+0000","last_became_active":"2024-04-26T18:08:44.554959+0000","last_became_peered":"2024-04-26T18:08:44.554959+0000","last_unstale":"2024-04-26T18:11:57.347023+0000","last_undegraded":"2024-04-26T18:11:57.347023+0000","last_fullsized":"2024-04-26T18:11:57.347023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.565073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347338+0000","last_change":"2024-04-26T18:08:47.377524+0000","last_active":"2024-04-26T18:11:57.347338+0000","last_peered":"2024-04-26T18:11:57.347338+0000","last_clean":"2024-04-26T18:11:57.347338+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:11:57.347338+0000","last_undegraded":"2024-04-26T18:11:57.347338+0000","last_fullsized":"2024-04-26T18:11:57.347338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:10.466615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.364671+0000","last_change":"2024-04-26T18:08:44.343894+0000","last_active":"2024-04-26T18:11:57.364671+0000","last_peered":"2024-04-26T18:11:57.364671+0000","last_clean":"2024-04-26T18:11:57.364671+0000","last_became_active":"2024-04-26T18:08:44.343153+0000","last_became_peered":"2024-04-26T18:08:44.343153+0000","last_unstale":"2024-04-26T18:11:57.364671+0000","last_undegraded":"2024-04-26T18:11:57.364671+0000","last_fullsized":"2024-04-26T18:11:57.364671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:09.914701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347161+0000","last_change":"2024-04-26T18:08:47.371962+0000","last_active":"2024-04-26T18:11:57.347161+0000","last_peered":"2024-04-26T18:11:57.347161+0000","last_clean":"2024-04-26T18:11:57.347161+0000","last_became_active":"2024-04-26T18:08:47.371517+0000","last_became_peered":"2024-04-26T18:08:47.371517+0000","last_unstale":"2024-04-26T18:11:57.347161+0000","last_undegraded":"2024-04-26T18:11:57.347161+0000","last_fullsized":"2024-04-26T18:11:57.347161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:23.553409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160624+0000","last_change":"2024-04-26T18:08:44.337853+0000","last_active":"2024-04-26T18:11:57.160624+0000","last_peered":"2024-04-26T18:11:57.160624+0000","last_clean":"2024-04-26T18:11:57.160624+0000","last_became_active":"2024-04-26T18:08:44.337393+0000","last_became_peered":"2024-04-26T18:08:44.337393+0000","last_unstale":"2024-04-26T18:11:57.160624+0000","last_undegraded":"2024-04-26T18:11:57.160624+0000","last_fullsized":"2024-04-26T18:11:57.160624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:26.514821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346960+0000","last_change":"2024-04-26T18:08:47.370999+0000","last_active":"2024-04-26T18:11:57.346960+0000","last_peered":"2024-04-26T18:11:57.346960+0000","last_clean":"2024-04-26T18:11:57.346960+0000","last_became_active":"2024-04-26T18:08:47.370373+0000","last_became_peered":"2024-04-26T18:08:47.370373+0000","last_unstale":"2024-04-26T18:11:57.346960+0000","last_undegraded":"2024-04-26T18:11:57.346960+0000","last_fullsized":"2024-04-26T18:11:57.346960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:19.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346894+0000","last_change":"2024-04-26T18:08:47.365722+0000","last_active":"2024-04-26T18:11:57.346894+0000","last_peered":"2024-04-26T18:11:57.346894+0000","last_clean":"2024-04-26T18:11:57.346894+0000","last_became_active":"2024-04-26T18:08:47.364692+0000","last_became_peered":"2024-04-26T18:08:47.364692+0000","last_unstale":"2024-04-26T18:11:57.346894+0000","last_undegraded":"2024-04-26T18:11:57.346894+0000","last_fullsized":"2024-04-26T18:11:57.346894+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:02.158156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346658+0000","last_change":"2024-04-26T18:08:47.373886+0000","last_active":"2024-04-26T18:11:57.346658+0000","last_peered":"2024-04-26T18:11:57.346658+0000","last_clean":"2024-04-26T18:11:57.346658+0000","last_became_active":"2024-04-26T18:08:47.373391+0000","last_became_peered":"2024-04-26T18:08:47.373391+0000","last_unstale":"2024-04-26T18:11:57.346658+0000","last_undegraded":"2024-04-26T18:11:57.346658+0000","last_fullsized":"2024-04-26T18:11:57.346658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:48:59.280050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346367+0000","last_change":"2024-04-26T18:08:47.366160+0000","last_active":"2024-04-26T18:11:57.346367+0000","last_peered":"2024-04-26T18:11:57.346367+0000","last_clean":"2024-04-26T18:11:57.346367+0000","last_became_active":"2024-04-26T18:08:47.365811+0000","last_became_peered":"2024-04-26T18:08:47.365811+0000","last_unstale":"2024-04-26T18:11:57.346367+0000","last_undegraded":"2024-04-26T18:11:57.346367+0000","last_fullsized":"2024-04-26T18:11:57.346367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:32.124692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346737+0000","last_change":"2024-04-26T18:08:47.366078+0000","last_active":"2024-04-26T18:11:57.346737+0000","last_peered":"2024-04-26T18:11:57.346737+0000","last_clean":"2024-04-26T18:11:57.346737+0000","last_became_active":"2024-04-26T18:08:47.365779+0000","last_became_peered":"2024-04-26T18:08:47.365779+0000","last_unstale":"2024-04-26T18:11:57.346737+0000","last_undegraded":"2024-04-26T18:11:57.346737+0000","last_fullsized":"2024-04-26T18:11:57.346737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.127606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160654+0000","last_change":"2024-04-26T18:08:47.372996+0000","last_active":"2024-04-26T18:11:57.160654+0000","last_peered":"2024-04-26T18:11:57.160654+0000","last_clean":"2024-04-26T18:11:57.160654+0000","last_became_active":"2024-04-26T18:08:47.372687+0000","last_became_peered":"2024-04-26T18:08:47.372687+0000","last_unstale":"2024-04-26T18:11:57.160654+0000","last_undegraded":"2024-04-26T18:11:57.160654+0000","last_fullsized":"2024-04-26T18:11:57.160654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:13:33.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347315+0000","last_change":"2024-04-26T18:08:47.371076+0000","last_active":"2024-04-26T18:11:57.347315+0000","last_peered":"2024-04-26T18:11:57.347315+0000","last_clean":"2024-04-26T18:11:57.347315+0000","last_became_active":"2024-04-26T18:08:47.370688+0000","last_became_peered":"2024-04-26T18:08:47.370688+0000","last_unstale":"2024-04-26T18:11:57.347315+0000","last_undegraded":"2024-04-26T18:11:57.347315+0000","last_fullsized":"2024-04-26T18:11:57.347315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:21.355971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347368+0000","last_change":"2024-04-26T18:08:47.372392+0000","last_active":"2024-04-26T18:11:57.347368+0000","last_peered":"2024-04-26T18:11:57.347368+0000","last_clean":"2024-04-26T18:11:57.347368+0000","last_became_active":"2024-04-26T18:08:47.372020+0000","last_became_peered":"2024-04-26T18:08:47.372020+0000","last_unstale":"2024-04-26T18:11:57.347368+0000","last_undegraded":"2024-04-26T18:11:57.347368+0000","last_fullsized":"2024-04-26T18:11:57.347368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235255+0000","last_change":"2024-04-26T18:08:47.369571+0000","last_active":"2024-04-26T18:11:57.235255+0000","last_peered":"2024-04-26T18:11:57.235255+0000","last_clean":"2024-04-26T18:11:57.235255+0000","last_became_active":"2024-04-26T18:08:47.369165+0000","last_became_peered":"2024-04-26T18:08:47.369165+0000","last_unstale":"2024-04-26T18:11:57.235255+0000","last_undegraded":"2024-04-26T18:11:57.235255+0000","last_fullsized":"2024-04-26T18:11:57.235255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:15.869182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163609+0000","last_change":"2024-04-26T18:08:47.372355+0000","last_active":"2024-04-26T18:11:57.163609+0000","last_peered":"2024-04-26T18:11:57.163609+0000","last_clean":"2024-04-26T18:11:57.163609+0000","last_became_active":"2024-04-26T18:08:47.371810+0000","last_became_peered":"2024-04-26T18:08:47.371810+0000","last_unstale":"2024-04-26T18:11:57.163609+0000","last_undegraded":"2024-04-26T18:11:57.163609+0000","last_fullsized":"2024-04-26T18:11:57.163609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:44:22.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234697+0000","last_change":"2024-04-26T18:08:47.364042+0000","last_active":"2024-04-26T18:11:57.234697+0000","last_peered":"2024-04-26T18:11:57.234697+0000","last_clean":"2024-04-26T18:11:57.234697+0000","last_became_active":"2024-04-26T18:08:47.363727+0000","last_became_peered":"2024-04-26T18:08:47.363727+0000","last_unstale":"2024-04-26T18:11:57.234697+0000","last_undegraded":"2024-04-26T18:11:57.234697+0000","last_fullsized":"2024-04-26T18:11:57.234697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:37.595261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346555+0000","last_change":"2024-04-26T18:08:47.377535+0000","last_active":"2024-04-26T18:11:57.346555+0000","last_peered":"2024-04-26T18:11:57.346555+0000","last_clean":"2024-04-26T18:11:57.346555+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:11:57.346555+0000","last_undegraded":"2024-04-26T18:11:57.346555+0000","last_fullsized":"2024-04-26T18:11:57.346555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:22.895402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160418+0000","last_change":"2024-04-26T18:08:47.367313+0000","last_active":"2024-04-26T18:11:57.160418+0000","last_peered":"2024-04-26T18:11:57.160418+0000","last_clean":"2024-04-26T18:11:57.160418+0000","last_became_active":"2024-04-26T18:08:47.366971+0000","last_became_peered":"2024-04-26T18:08:47.366971+0000","last_unstale":"2024-04-26T18:11:57.160418+0000","last_undegraded":"2024-04-26T18:11:57.160418+0000","last_fullsized":"2024-04-26T18:11:57.160418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:14.518100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347070+0000","last_change":"2024-04-26T18:08:47.373004+0000","last_active":"2024-04-26T18:11:57.347070+0000","last_peered":"2024-04-26T18:11:57.347070+0000","last_clean":"2024-04-26T18:11:57.347070+0000","last_became_active":"2024-04-26T18:08:47.372641+0000","last_became_peered":"2024-04-26T18:08:47.372641+0000","last_unstale":"2024-04-26T18:11:57.347070+0000","last_undegraded":"2024-04-26T18:11:57.347070+0000","last_fullsized":"2024-04-26T18:11:57.347070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:02.018425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597137,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1168,"num_read_kb":686,"num_write":296,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":498476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579961,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28892,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716260,"statfs":{"total":95995035648,"available":95965450240,"internally_reserved":0,"allocated":1859584,"data_stored":1288898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743487,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1429504,"data_stored":891872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005125,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1425408,"data_stored":891770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266762,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1613824,"data_stored":1082310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462992,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":498045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":564312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":499316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":566942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":489487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":499014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724628,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1777664,"data_stored":1222713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:12:02.516 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi119:/etc/ceph/ceph.conf 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi167:/etc/ceph/ceph.conf 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: mgrmap e23: x(active, since 5s), standbys: z, y 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.765 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi119:/etc/ceph/ceph.conf 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi167:/etc/ceph/ceph.conf 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: mgrmap e23: x(active, since 5s), standbys: z, y 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi119:/etc/ceph/ceph.conf 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi167:/etc/ceph/ceph.conf 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi119:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='client.24845 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi119:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi167:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: Updating smithi090:/var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: mgrmap e23: x(active, since 5s), standbys: z, y 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:03.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:12:03.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:12:03.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:12:03.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:12:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:12:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:12:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:12:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:12:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:12:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3881135182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d44102f60 msgr2=0x7f6d44106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3881135182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d44102f60 0x7f6d44106f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6d340099c0 tx=0x7f6d34009cd0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3881135182 shutdown_connections 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3881135182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d44107640 0x7f6d4410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3881135182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d441025b0 0x7f6d44102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3881135182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d44102f60 0x7f6d44106f60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3881135182 >> 172.21.15.90:0/3881135182 conn(0x7f6d440792d0 msgr2=0x7f6d440796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:04.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.357+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3881135182 shutdown_connections 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3881135182 wait complete. 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 Processor -- start 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 -- start start 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d441025b0 0x7f6d44198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44102f60 0x7f6d44198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d44107640 0x7f6d44199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6d441124b0 con 0x7f6d441025b0 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44112330 con 0x7f6d44102f60 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.358+0000 7f6d4b3d1700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44112630 con 0x7f6d44107640 2024-04-26T18:12:04.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d441025b0 0x7f6d44198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d441025b0 0x7f6d44198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50316/0 (socket says 172.21.15.90:50316) 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 -- 172.21.15.90:0/3456684657 learned_addr learned my addr 172.21.15.90:0/3456684657 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4896c700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44102f60 0x7f6d44198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4996e700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d44107640 0x7f6d44199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 -- 172.21.15.90:0/3456684657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d44107640 msgr2=0x7f6d44199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d44107640 0x7f6d44199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 -- 172.21.15.90:0/3456684657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44102f60 msgr2=0x7f6d44198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44102f60 0x7f6d44198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 -- 172.21.15.90:0/3456684657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d441a8080 con 0x7f6d441025b0 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4916d700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d441025b0 0x7f6d44198810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000d950 tx=0x7f6d4000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d3a7fc700 1 -- 172.21.15.90:0/3456684657 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d40016070 con 0x7f6d441025b0 2024-04-26T18:12:04.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d3a7fc700 1 -- 172.21.15.90:0/3456684657 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d4000fd40 con 0x7f6d441025b0 2024-04-26T18:12:04.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d441a8370 con 0x7f6d441025b0 2024-04-26T18:12:04.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d3a7fc700 1 -- 172.21.15.90:0/3456684657 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000e3b0 con 0x7f6d441025b0 2024-04-26T18:12:04.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.359+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a8880 con 0x7f6d441025b0 2024-04-26T18:12:04.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.360+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d44103a40 con 0x7f6d441025b0 2024-04-26T18:12:04.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.361+0000 7f6d3a7fc700 1 -- 172.21.15.90:0/3456684657 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f6d40019070 con 0x7f6d441025b0 2024-04-26T18:12:04.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.362+0000 7f6d3a7fc700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6d300a3ed0 0x7f6d300a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:04.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.362+0000 7f6d4896c700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6d300a3ed0 0x7f6d300a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:04.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.362+0000 7f6d3a7fc700 1 -- 172.21.15.90:0/3456684657 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f6d400d04f0 con 0x7f6d441025b0 2024-04-26T18:12:04.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.362+0000 7f6d4896c700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6d300a3ed0 0x7f6d300a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d34006010 tx=0x7f6d34005c60 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:04.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.367+0000 7f6d3a7fc700 1 -- 172.21.15.90:0/3456684657 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d4009c9b0 con 0x7f6d441025b0 2024-04-26T18:12:04.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.565+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d4406c140 con 0x7f6d300a3ed0 2024-04-26T18:12:04.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.572+0000 7f6d3a7fc700 1 -- 172.21.15.90:0/3456684657 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257745 (secure 0 0 0) 0x7f6d4406c140 con 0x7f6d300a3ed0 2024-04-26T18:12:04.576 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6d300a3ed0 msgr2=0x7f6d300a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:04.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6d300a3ed0 0x7f6d300a6390 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6d34006010 tx=0x7f6d34005c60 comp rx=0 tx=0).stop 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d441025b0 msgr2=0x7f6d44198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d441025b0 0x7f6d44198810 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000d950 tx=0x7f6d4000dc60 comp rx=0 tx=0).stop 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 shutdown_connections 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6d300a3ed0 0x7f6d300a6390 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d441025b0 0x7f6d44198810 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6d44107640 0x7f6d44199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 --2- 172.21.15.90:0/3456684657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44102f60 0x7f6d44198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 >> 172.21.15.90:0/3456684657 conn(0x7f6d440792d0 msgr2=0x7f6d44100520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 shutdown_connections 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:04.575+0000 7f6d4b3d1700 1 -- 172.21.15.90:0/3456684657 wait complete. 2024-04-26T18:12:04.580 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T18:12:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:04 smithi119 ceph-mon[32752]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-26T18:12:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:04 smithi167 ceph-mon[32751]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-26T18:12:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:04 smithi090 ceph-mon[29180]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-26T18:12:05.231 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-26T18:12:03.174478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187739,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1309,"num_read_kb":810,"num_write":479,"num_write_kb":4380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380984,"kb_used_data":15100,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124560840,"statfs":{"total":1151940427776,"available":1151550300160,"internally_reserved":0,"allocated":15462400,"data_stored":8993155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.988940"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347391+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:11:57.347391+0000","last_peered":"2024-04-26T18:11:57.347391+0000","last_clean":"2024-04-26T18:11:57.347391+0000","last_became_active":"2024-04-26T18:08:47.377408+0000","last_became_peered":"2024-04-26T18:08:47.377408+0000","last_unstale":"2024-04-26T18:11:57.347391+0000","last_undegraded":"2024-04-26T18:11:57.347391+0000","last_fullsized":"2024-04-26T18:11:57.347391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:13.822258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347009+0000","last_change":"2024-04-26T18:08:47.365125+0000","last_active":"2024-04-26T18:11:57.347009+0000","last_peered":"2024-04-26T18:11:57.347009+0000","last_clean":"2024-04-26T18:11:57.347009+0000","last_became_active":"2024-04-26T18:08:47.364847+0000","last_became_peered":"2024-04-26T18:08:47.364847+0000","last_unstale":"2024-04-26T18:11:57.347009+0000","last_undegraded":"2024-04-26T18:11:57.347009+0000","last_fullsized":"2024-04-26T18:11:57.347009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:16:37.890893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346817+0000","last_change":"2024-04-26T18:08:47.377580+0000","last_active":"2024-04-26T18:11:57.346817+0000","last_peered":"2024-04-26T18:11:57.346817+0000","last_clean":"2024-04-26T18:11:57.346817+0000","last_became_active":"2024-04-26T18:08:47.377219+0000","last_became_peered":"2024-04-26T18:08:47.377219+0000","last_unstale":"2024-04-26T18:11:57.346817+0000","last_undegraded":"2024-04-26T18:11:57.346817+0000","last_fullsized":"2024-04-26T18:11:57.346817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:12.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161389+0000","last_change":"2024-04-26T18:08:47.371628+0000","last_active":"2024-04-26T18:11:57.161389+0000","last_peered":"2024-04-26T18:11:57.161389+0000","last_clean":"2024-04-26T18:11:57.161389+0000","last_became_active":"2024-04-26T18:08:47.370840+0000","last_became_peered":"2024-04-26T18:08:47.370840+0000","last_unstale":"2024-04-26T18:11:57.161389+0000","last_undegraded":"2024-04-26T18:11:57.161389+0000","last_fullsized":"2024-04-26T18:11:57.161389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:11.362664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254081+0000","last_change":"2024-04-26T18:08:47.367632+0000","last_active":"2024-04-26T18:11:57.254081+0000","last_peered":"2024-04-26T18:11:57.254081+0000","last_clean":"2024-04-26T18:11:57.254081+0000","last_became_active":"2024-04-26T18:08:47.367316+0000","last_became_peered":"2024-04-26T18:08:47.367316+0000","last_unstale":"2024-04-26T18:11:57.254081+0000","last_undegraded":"2024-04-26T18:11:57.254081+0000","last_fullsized":"2024-04-26T18:11:57.254081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:31:26.665292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207571+0000","last_change":"2024-04-26T18:08:47.369742+0000","last_active":"2024-04-26T18:11:57.207571+0000","last_peered":"2024-04-26T18:11:57.207571+0000","last_clean":"2024-04-26T18:11:57.207571+0000","last_became_active":"2024-04-26T18:08:47.369230+0000","last_became_peered":"2024-04-26T18:08:47.369230+0000","last_unstale":"2024-04-26T18:11:57.207571+0000","last_undegraded":"2024-04-26T18:11:57.207571+0000","last_fullsized":"2024-04-26T18:11:57.207571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:02:04.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346750+0000","last_change":"2024-04-26T18:08:47.370657+0000","last_active":"2024-04-26T18:11:57.346750+0000","last_peered":"2024-04-26T18:11:57.346750+0000","last_clean":"2024-04-26T18:11:57.346750+0000","last_became_active":"2024-04-26T18:08:47.370133+0000","last_became_peered":"2024-04-26T18:08:47.370133+0000","last_unstale":"2024-04-26T18:11:57.346750+0000","last_undegraded":"2024-04-26T18:11:57.346750+0000","last_fullsized":"2024-04-26T18:11:57.346750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:31.232179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234654+0000","last_change":"2024-04-26T18:08:47.366221+0000","last_active":"2024-04-26T18:11:57.234654+0000","last_peered":"2024-04-26T18:11:57.234654+0000","last_clean":"2024-04-26T18:11:57.234654+0000","last_became_active":"2024-04-26T18:08:47.365990+0000","last_became_peered":"2024-04-26T18:08:47.365990+0000","last_unstale":"2024-04-26T18:11:57.234654+0000","last_undegraded":"2024-04-26T18:11:57.234654+0000","last_fullsized":"2024-04-26T18:11:57.234654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:43.707583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346856+0000","last_change":"2024-04-26T18:08:47.373165+0000","last_active":"2024-04-26T18:11:57.346856+0000","last_peered":"2024-04-26T18:11:57.346856+0000","last_clean":"2024-04-26T18:11:57.346856+0000","last_became_active":"2024-04-26T18:08:47.372688+0000","last_became_peered":"2024-04-26T18:08:47.372688+0000","last_unstale":"2024-04-26T18:11:57.346856+0000","last_undegraded":"2024-04-26T18:11:57.346856+0000","last_fullsized":"2024-04-26T18:11:57.346856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:18.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207762+0000","last_change":"2024-04-26T18:09:33.016811+0000","last_active":"2024-04-26T18:11:57.207762+0000","last_peered":"2024-04-26T18:11:57.207762+0000","last_clean":"2024-04-26T18:11:57.207762+0000","last_became_active":"2024-04-26T18:09:33.016267+0000","last_became_peered":"2024-04-26T18:09:33.016267+0000","last_unstale":"2024-04-26T18:11:57.207762+0000","last_undegraded":"2024-04-26T18:11:57.207762+0000","last_fullsized":"2024-04-26T18:11:57.207762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.002914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347272+0000","last_change":"2024-04-26T18:08:47.378643+0000","last_active":"2024-04-26T18:11:57.347272+0000","last_peered":"2024-04-26T18:11:57.347272+0000","last_clean":"2024-04-26T18:11:57.347272+0000","last_became_active":"2024-04-26T18:08:47.378329+0000","last_became_peered":"2024-04-26T18:08:47.378329+0000","last_unstale":"2024-04-26T18:11:57.347272+0000","last_undegraded":"2024-04-26T18:11:57.347272+0000","last_fullsized":"2024-04-26T18:11:57.347272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:59.016025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161243+0000","last_change":"2024-04-26T18:08:47.371663+0000","last_active":"2024-04-26T18:11:57.161243+0000","last_peered":"2024-04-26T18:11:57.161243+0000","last_clean":"2024-04-26T18:11:57.161243+0000","last_became_active":"2024-04-26T18:08:47.371298+0000","last_became_peered":"2024-04-26T18:08:47.371298+0000","last_unstale":"2024-04-26T18:11:57.161243+0000","last_undegraded":"2024-04-26T18:11:57.161243+0000","last_fullsized":"2024-04-26T18:11:57.161243+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:04.884698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160404+0000","last_change":"2024-04-26T18:08:47.365179+0000","last_active":"2024-04-26T18:11:57.160404+0000","last_peered":"2024-04-26T18:11:57.160404+0000","last_clean":"2024-04-26T18:11:57.160404+0000","last_became_active":"2024-04-26T18:08:47.364888+0000","last_became_peered":"2024-04-26T18:08:47.364888+0000","last_unstale":"2024-04-26T18:11:57.160404+0000","last_undegraded":"2024-04-26T18:11:57.160404+0000","last_fullsized":"2024-04-26T18:11:57.160404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:34.082855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346532+0000","last_change":"2024-04-26T18:08:47.364828+0000","last_active":"2024-04-26T18:11:57.346532+0000","last_peered":"2024-04-26T18:11:57.346532+0000","last_clean":"2024-04-26T18:11:57.346532+0000","last_became_active":"2024-04-26T18:08:47.364529+0000","last_became_peered":"2024-04-26T18:08:47.364529+0000","last_unstale":"2024-04-26T18:11:57.346532+0000","last_undegraded":"2024-04-26T18:11:57.346532+0000","last_fullsized":"2024-04-26T18:11:57.346532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.480258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346255+0000","last_change":"2024-04-26T18:08:47.373578+0000","last_active":"2024-04-26T18:11:57.346255+0000","last_peered":"2024-04-26T18:11:57.346255+0000","last_clean":"2024-04-26T18:11:57.346255+0000","last_became_active":"2024-04-26T18:08:47.373263+0000","last_became_peered":"2024-04-26T18:08:47.373263+0000","last_unstale":"2024-04-26T18:11:57.346255+0000","last_undegraded":"2024-04-26T18:11:57.346255+0000","last_fullsized":"2024-04-26T18:11:57.346255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:25.340241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346347+0000","last_change":"2024-04-26T18:08:47.376617+0000","last_active":"2024-04-26T18:11:57.346347+0000","last_peered":"2024-04-26T18:11:57.346347+0000","last_clean":"2024-04-26T18:11:57.346347+0000","last_became_active":"2024-04-26T18:08:47.376290+0000","last_became_peered":"2024-04-26T18:08:47.376290+0000","last_unstale":"2024-04-26T18:11:57.346347+0000","last_undegraded":"2024-04-26T18:11:57.346347+0000","last_fullsized":"2024-04-26T18:11:57.346347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:51.187583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347552+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:11:57.347552+0000","last_peered":"2024-04-26T18:11:57.347552+0000","last_clean":"2024-04-26T18:11:57.347552+0000","last_became_active":"2024-04-26T18:08:47.377427+0000","last_became_peered":"2024-04-26T18:08:47.377427+0000","last_unstale":"2024-04-26T18:11:57.347552+0000","last_undegraded":"2024-04-26T18:11:57.347552+0000","last_fullsized":"2024-04-26T18:11:57.347552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.529500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252164+0000","last_change":"2024-04-26T18:08:47.373314+0000","last_active":"2024-04-26T18:11:57.252164+0000","last_peered":"2024-04-26T18:11:57.252164+0000","last_clean":"2024-04-26T18:11:57.252164+0000","last_became_active":"2024-04-26T18:08:47.372868+0000","last_became_peered":"2024-04-26T18:08:47.372868+0000","last_unstale":"2024-04-26T18:11:57.252164+0000","last_undegraded":"2024-04-26T18:11:57.252164+0000","last_fullsized":"2024-04-26T18:11:57.252164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:19.642655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347122+0000","last_change":"2024-04-26T18:08:47.367593+0000","last_active":"2024-04-26T18:11:57.347122+0000","last_peered":"2024-04-26T18:11:57.347122+0000","last_clean":"2024-04-26T18:11:57.347122+0000","last_became_active":"2024-04-26T18:08:47.367243+0000","last_became_peered":"2024-04-26T18:08:47.367243+0000","last_unstale":"2024-04-26T18:11:57.347122+0000","last_undegraded":"2024-04-26T18:11:57.347122+0000","last_fullsized":"2024-04-26T18:11:57.347122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.380307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252032+0000","last_change":"2024-04-26T18:08:44.339625+0000","last_active":"2024-04-26T18:11:57.252032+0000","last_peered":"2024-04-26T18:11:57.252032+0000","last_clean":"2024-04-26T18:11:57.252032+0000","last_became_active":"2024-04-26T18:08:44.339209+0000","last_became_peered":"2024-04-26T18:08:44.339209+0000","last_unstale":"2024-04-26T18:11:57.252032+0000","last_undegraded":"2024-04-26T18:11:57.252032+0000","last_fullsized":"2024-04-26T18:11:57.252032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:43.169885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234859+0000","last_change":"2024-04-26T18:08:47.375884+0000","last_active":"2024-04-26T18:11:57.234859+0000","last_peered":"2024-04-26T18:11:57.234859+0000","last_clean":"2024-04-26T18:11:57.234859+0000","last_became_active":"2024-04-26T18:08:47.375557+0000","last_became_peered":"2024-04-26T18:08:47.375557+0000","last_unstale":"2024-04-26T18:11:57.234859+0000","last_undegraded":"2024-04-26T18:11:57.234859+0000","last_fullsized":"2024-04-26T18:11:57.234859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.593910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235169+0000","last_change":"2024-04-26T18:08:44.557167+0000","last_active":"2024-04-26T18:11:57.235169+0000","last_peered":"2024-04-26T18:11:57.235169+0000","last_clean":"2024-04-26T18:11:57.235169+0000","last_became_active":"2024-04-26T18:08:44.556716+0000","last_became_peered":"2024-04-26T18:08:44.556716+0000","last_unstale":"2024-04-26T18:11:57.235169+0000","last_undegraded":"2024-04-26T18:11:57.235169+0000","last_fullsized":"2024-04-26T18:11:57.235169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347151+0000","last_change":"2024-04-26T18:08:47.377612+0000","last_active":"2024-04-26T18:11:57.347151+0000","last_peered":"2024-04-26T18:11:57.347151+0000","last_clean":"2024-04-26T18:11:57.347151+0000","last_became_active":"2024-04-26T18:08:47.377231+0000","last_became_peered":"2024-04-26T18:08:47.377231+0000","last_unstale":"2024-04-26T18:11:57.347151+0000","last_undegraded":"2024-04-26T18:11:57.347151+0000","last_fullsized":"2024-04-26T18:11:57.347151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:11.594373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254362+0000","last_change":"2024-04-26T18:08:44.343429+0000","last_active":"2024-04-26T18:11:57.254362+0000","last_peered":"2024-04-26T18:11:57.254362+0000","last_clean":"2024-04-26T18:11:57.254362+0000","last_became_active":"2024-04-26T18:08:44.343103+0000","last_became_peered":"2024-04-26T18:08:44.343103+0000","last_unstale":"2024-04-26T18:11:57.254362+0000","last_undegraded":"2024-04-26T18:11:57.254362+0000","last_fullsized":"2024-04-26T18:11:57.254362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:52.244159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346545+0000","last_change":"2024-04-26T18:08:47.373130+0000","last_active":"2024-04-26T18:11:57.346545+0000","last_peered":"2024-04-26T18:11:57.346545+0000","last_clean":"2024-04-26T18:11:57.346545+0000","last_became_active":"2024-04-26T18:08:47.372726+0000","last_became_peered":"2024-04-26T18:08:47.372726+0000","last_unstale":"2024-04-26T18:11:57.346545+0000","last_undegraded":"2024-04-26T18:11:57.346545+0000","last_fullsized":"2024-04-26T18:11:57.346545+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:44.461177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254325+0000","last_change":"2024-04-26T18:08:44.340152+0000","last_active":"2024-04-26T18:11:57.254325+0000","last_peered":"2024-04-26T18:11:57.254325+0000","last_clean":"2024-04-26T18:11:57.254325+0000","last_became_active":"2024-04-26T18:08:44.339684+0000","last_became_peered":"2024-04-26T18:08:44.339684+0000","last_unstale":"2024-04-26T18:11:57.254325+0000","last_undegraded":"2024-04-26T18:11:57.254325+0000","last_fullsized":"2024-04-26T18:11:57.254325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:49.961201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.164063+0000","last_change":"2024-04-26T18:08:47.375387+0000","last_active":"2024-04-26T18:11:57.164063+0000","last_peered":"2024-04-26T18:11:57.164063+0000","last_clean":"2024-04-26T18:11:57.164063+0000","last_became_active":"2024-04-26T18:08:47.374689+0000","last_became_peered":"2024-04-26T18:08:47.374689+0000","last_unstale":"2024-04-26T18:11:57.164063+0000","last_undegraded":"2024-04-26T18:11:57.164063+0000","last_fullsized":"2024-04-26T18:11:57.164063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:05.618804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'9","reported_seq":216,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.405901+0000","last_change":"2024-04-26T18:08:44.344167+0000","last_active":"2024-04-26T18:11:57.405901+0000","last_peered":"2024-04-26T18:11:57.405901+0000","last_clean":"2024-04-26T18:11:57.405901+0000","last_became_active":"2024-04-26T18:08:44.343646+0000","last_became_peered":"2024-04-26T18:08:44.343646+0000","last_unstale":"2024-04-26T18:11:57.405901+0000","last_undegraded":"2024-04-26T18:11:57.405901+0000","last_fullsized":"2024-04-26T18:11:57.405901+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:53.859483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":4,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254051+0000","last_change":"2024-04-26T18:08:47.370546+0000","last_active":"2024-04-26T18:11:57.254051+0000","last_peered":"2024-04-26T18:11:57.254051+0000","last_clean":"2024-04-26T18:11:57.254051+0000","last_became_active":"2024-04-26T18:08:47.370220+0000","last_became_peered":"2024-04-26T18:08:47.370220+0000","last_unstale":"2024-04-26T18:11:57.254051+0000","last_undegraded":"2024-04-26T18:11:57.254051+0000","last_fullsized":"2024-04-26T18:11:57.254051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:05.139824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207423+0000","last_change":"2024-04-26T18:08:44.347031+0000","last_active":"2024-04-26T18:11:57.207423+0000","last_peered":"2024-04-26T18:11:57.207423+0000","last_clean":"2024-04-26T18:11:57.207423+0000","last_became_active":"2024-04-26T18:08:44.346650+0000","last_became_peered":"2024-04-26T18:08:44.346650+0000","last_unstale":"2024-04-26T18:11:57.207423+0000","last_undegraded":"2024-04-26T18:11:57.207423+0000","last_fullsized":"2024-04-26T18:11:57.207423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:50.946449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163867+0000","last_change":"2024-04-26T18:08:47.374230+0000","last_active":"2024-04-26T18:11:57.163867+0000","last_peered":"2024-04-26T18:11:57.163867+0000","last_clean":"2024-04-26T18:11:57.163867+0000","last_became_active":"2024-04-26T18:08:47.370780+0000","last_became_peered":"2024-04-26T18:08:47.370780+0000","last_unstale":"2024-04-26T18:11:57.163867+0000","last_undegraded":"2024-04-26T18:11:57.163867+0000","last_fullsized":"2024-04-26T18:11:57.163867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:01.171195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160284+0000","last_change":"2024-04-26T18:08:44.338318+0000","last_active":"2024-04-26T18:11:57.160284+0000","last_peered":"2024-04-26T18:11:57.160284+0000","last_clean":"2024-04-26T18:11:57.160284+0000","last_became_active":"2024-04-26T18:08:44.338009+0000","last_became_peered":"2024-04-26T18:08:44.338009+0000","last_unstale":"2024-04-26T18:11:57.160284+0000","last_undegraded":"2024-04-26T18:11:57.160284+0000","last_fullsized":"2024-04-26T18:11:57.160284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:33.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234730+0000","last_change":"2024-04-26T18:08:47.363924+0000","last_active":"2024-04-26T18:11:57.234730+0000","last_peered":"2024-04-26T18:11:57.234730+0000","last_clean":"2024-04-26T18:11:57.234730+0000","last_became_active":"2024-04-26T18:08:47.363616+0000","last_became_peered":"2024-04-26T18:08:47.363616+0000","last_unstale":"2024-04-26T18:11:57.234730+0000","last_undegraded":"2024-04-26T18:11:57.234730+0000","last_fullsized":"2024-04-26T18:11:57.234730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:27:08.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163592+0000","last_change":"2024-04-26T18:08:44.343350+0000","last_active":"2024-04-26T18:11:57.163592+0000","last_peered":"2024-04-26T18:11:57.163592+0000","last_clean":"2024-04-26T18:11:57.163592+0000","last_became_active":"2024-04-26T18:08:44.343040+0000","last_became_peered":"2024-04-26T18:08:44.343040+0000","last_unstale":"2024-04-26T18:11:57.163592+0000","last_undegraded":"2024-04-26T18:11:57.163592+0000","last_fullsized":"2024-04-26T18:11:57.163592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:09.427785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.251855+0000","last_change":"2024-04-26T18:08:47.360180+0000","last_active":"2024-04-26T18:11:57.251855+0000","last_peered":"2024-04-26T18:11:57.251855+0000","last_clean":"2024-04-26T18:11:57.251855+0000","last_became_active":"2024-04-26T18:08:47.359860+0000","last_became_peered":"2024-04-26T18:08:47.359860+0000","last_unstale":"2024-04-26T18:11:57.251855+0000","last_undegraded":"2024-04-26T18:11:57.251855+0000","last_fullsized":"2024-04-26T18:11:57.251855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:00.532589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":208,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207951+0000","last_change":"2024-04-26T18:08:44.343942+0000","last_active":"2024-04-26T18:11:57.207951+0000","last_peered":"2024-04-26T18:11:57.207951+0000","last_clean":"2024-04-26T18:11:57.207951+0000","last_became_active":"2024-04-26T18:08:44.343581+0000","last_became_peered":"2024-04-26T18:08:44.343581+0000","last_unstale":"2024-04-26T18:11:57.207951+0000","last_undegraded":"2024-04-26T18:11:57.207951+0000","last_fullsized":"2024-04-26T18:11:57.207951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:32.044284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":174,"num_read_kb":174,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346718+0000","last_change":"2024-04-26T18:08:47.372303+0000","last_active":"2024-04-26T18:11:57.346718+0000","last_peered":"2024-04-26T18:11:57.346718+0000","last_clean":"2024-04-26T18:11:57.346718+0000","last_became_active":"2024-04-26T18:08:47.371961+0000","last_became_peered":"2024-04-26T18:08:47.371961+0000","last_unstale":"2024-04-26T18:11:57.346718+0000","last_undegraded":"2024-04-26T18:11:57.346718+0000","last_fullsized":"2024-04-26T18:11:57.346718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:19.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347265+0000","last_change":"2024-04-26T18:08:44.345681+0000","last_active":"2024-04-26T18:11:57.347265+0000","last_peered":"2024-04-26T18:11:57.347265+0000","last_clean":"2024-04-26T18:11:57.347265+0000","last_became_active":"2024-04-26T18:08:44.345329+0000","last_became_peered":"2024-04-26T18:08:44.345329+0000","last_unstale":"2024-04-26T18:11:57.347265+0000","last_undegraded":"2024-04-26T18:11:57.347265+0000","last_fullsized":"2024-04-26T18:11:57.347265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:59.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347192+0000","last_change":"2024-04-26T18:08:47.370773+0000","last_active":"2024-04-26T18:11:57.347192+0000","last_peered":"2024-04-26T18:11:57.347192+0000","last_clean":"2024-04-26T18:11:57.347192+0000","last_became_active":"2024-04-26T18:08:47.370490+0000","last_became_peered":"2024-04-26T18:08:47.370490+0000","last_unstale":"2024-04-26T18:11:57.347192+0000","last_undegraded":"2024-04-26T18:11:57.347192+0000","last_fullsized":"2024-04-26T18:11:57.347192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:01.935335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.164125+0000","last_change":"2024-04-26T18:08:44.555714+0000","last_active":"2024-04-26T18:11:57.164125+0000","last_peered":"2024-04-26T18:11:57.164125+0000","last_clean":"2024-04-26T18:11:57.164125+0000","last_became_active":"2024-04-26T18:08:44.555184+0000","last_became_peered":"2024-04-26T18:08:44.555184+0000","last_unstale":"2024-04-26T18:11:57.164125+0000","last_undegraded":"2024-04-26T18:11:57.164125+0000","last_fullsized":"2024-04-26T18:11:57.164125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:01.433764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346864+0000","last_change":"2024-04-26T18:09:33.525026+0000","last_active":"2024-04-26T18:11:57.346864+0000","last_peered":"2024-04-26T18:11:57.346864+0000","last_clean":"2024-04-26T18:11:57.346864+0000","last_became_active":"2024-04-26T18:09:33.524587+0000","last_became_peered":"2024-04-26T18:09:33.524587+0000","last_unstale":"2024-04-26T18:11:57.346864+0000","last_undegraded":"2024-04-26T18:11:57.346864+0000","last_fullsized":"2024-04-26T18:11:57.346864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:39:48.534737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'36","reported_seq":424,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.409028+0000","last_change":"2024-04-26T18:08:44.346346+0000","last_active":"2024-04-26T18:11:57.409028+0000","last_peered":"2024-04-26T18:11:57.409028+0000","last_clean":"2024-04-26T18:11:57.409028+0000","last_became_active":"2024-04-26T18:08:44.345999+0000","last_became_peered":"2024-04-26T18:08:44.345999+0000","last_unstale":"2024-04-26T18:11:57.409028+0000","last_undegraded":"2024-04-26T18:11:57.409028+0000","last_fullsized":"2024-04-26T18:11:57.409028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:27.277880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66474,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":359,"num_read_kb":238,"num_write":37,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254263+0000","last_change":"2024-04-26T18:08:47.366303+0000","last_active":"2024-04-26T18:11:57.254263+0000","last_peered":"2024-04-26T18:11:57.254263+0000","last_clean":"2024-04-26T18:11:57.254263+0000","last_became_active":"2024-04-26T18:08:47.365861+0000","last_became_peered":"2024-04-26T18:08:47.365861+0000","last_unstale":"2024-04-26T18:11:57.254263+0000","last_undegraded":"2024-04-26T18:11:57.254263+0000","last_fullsized":"2024-04-26T18:11:57.254263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:31.413589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346987+0000","last_change":"2024-04-26T18:09:33.022912+0000","last_active":"2024-04-26T18:11:57.346987+0000","last_peered":"2024-04-26T18:11:57.346987+0000","last_clean":"2024-04-26T18:11:57.346987+0000","last_became_active":"2024-04-26T18:09:33.015390+0000","last_became_peered":"2024-04-26T18:09:33.015390+0000","last_unstale":"2024-04-26T18:11:57.346987+0000","last_undegraded":"2024-04-26T18:11:57.346987+0000","last_fullsized":"2024-04-26T18:11:57.346987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:53.980531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160286+0000","last_change":"2024-04-26T18:08:47.367835+0000","last_active":"2024-04-26T18:11:57.160286+0000","last_peered":"2024-04-26T18:11:57.160286+0000","last_clean":"2024-04-26T18:11:57.160286+0000","last_became_active":"2024-04-26T18:08:47.367517+0000","last_became_peered":"2024-04-26T18:08:47.367517+0000","last_unstale":"2024-04-26T18:11:57.160286+0000","last_undegraded":"2024-04-26T18:11:57.160286+0000","last_fullsized":"2024-04-26T18:11:57.160286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:54.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163733+0000","last_change":"2024-04-26T18:08:44.344161+0000","last_active":"2024-04-26T18:11:57.163733+0000","last_peered":"2024-04-26T18:11:57.163733+0000","last_clean":"2024-04-26T18:11:57.163733+0000","last_became_active":"2024-04-26T18:08:44.343635+0000","last_became_peered":"2024-04-26T18:08:44.343635+0000","last_unstale":"2024-04-26T18:11:57.163733+0000","last_undegraded":"2024-04-26T18:11:57.163733+0000","last_fullsized":"2024-04-26T18:11:57.163733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.946584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346903+0000","last_change":"2024-04-26T18:09:33.524351+0000","last_active":"2024-04-26T18:11:57.346903+0000","last_peered":"2024-04-26T18:11:57.346903+0000","last_clean":"2024-04-26T18:11:57.346903+0000","last_became_active":"2024-04-26T18:09:33.523953+0000","last_became_peered":"2024-04-26T18:09:33.523953+0000","last_unstale":"2024-04-26T18:11:57.346903+0000","last_undegraded":"2024-04-26T18:11:57.346903+0000","last_fullsized":"2024-04-26T18:11:57.346903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:22.887498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":213,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.233757+0000","last_change":"2024-04-26T18:08:44.345278+0000","last_active":"2024-04-26T18:11:57.233757+0000","last_peered":"2024-04-26T18:11:57.233757+0000","last_clean":"2024-04-26T18:11:57.233757+0000","last_became_active":"2024-04-26T18:08:44.344531+0000","last_became_peered":"2024-04-26T18:08:44.344531+0000","last_unstale":"2024-04-26T18:11:57.233757+0000","last_undegraded":"2024-04-26T18:11:57.233757+0000","last_fullsized":"2024-04-26T18:11:57.233757+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:02.667622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346598+0000","last_change":"2024-04-26T18:08:47.369542+0000","last_active":"2024-04-26T18:11:57.346598+0000","last_peered":"2024-04-26T18:11:57.346598+0000","last_clean":"2024-04-26T18:11:57.346598+0000","last_became_active":"2024-04-26T18:08:47.369181+0000","last_became_peered":"2024-04-26T18:08:47.369181+0000","last_unstale":"2024-04-26T18:11:57.346598+0000","last_undegraded":"2024-04-26T18:11:57.346598+0000","last_fullsized":"2024-04-26T18:11:57.346598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:56.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254034+0000","last_change":"2024-04-26T18:08:44.555580+0000","last_active":"2024-04-26T18:11:57.254034+0000","last_peered":"2024-04-26T18:11:57.254034+0000","last_clean":"2024-04-26T18:11:57.254034+0000","last_became_active":"2024-04-26T18:08:44.555247+0000","last_became_peered":"2024-04-26T18:08:44.555247+0000","last_unstale":"2024-04-26T18:11:57.254034+0000","last_undegraded":"2024-04-26T18:11:57.254034+0000","last_fullsized":"2024-04-26T18:11:57.254034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:58:55.207355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163660+0000","last_change":"2024-04-26T18:08:47.372300+0000","last_active":"2024-04-26T18:11:57.163660+0000","last_peered":"2024-04-26T18:11:57.163660+0000","last_clean":"2024-04-26T18:11:57.163660+0000","last_became_active":"2024-04-26T18:08:47.371952+0000","last_became_peered":"2024-04-26T18:08:47.371952+0000","last_unstale":"2024-04-26T18:11:57.163660+0000","last_undegraded":"2024-04-26T18:11:57.163660+0000","last_fullsized":"2024-04-26T18:11:57.163660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:58.254691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254389+0000","last_change":"2024-04-26T18:08:44.340926+0000","last_active":"2024-04-26T18:11:57.254389+0000","last_peered":"2024-04-26T18:11:57.254389+0000","last_clean":"2024-04-26T18:11:57.254389+0000","last_became_active":"2024-04-26T18:08:44.340563+0000","last_became_peered":"2024-04-26T18:08:44.340563+0000","last_unstale":"2024-04-26T18:11:57.254389+0000","last_undegraded":"2024-04-26T18:11:57.254389+0000","last_fullsized":"2024-04-26T18:11:57.254389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:57.803017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347296+0000","last_change":"2024-04-26T18:09:33.021890+0000","last_active":"2024-04-26T18:11:57.347296+0000","last_peered":"2024-04-26T18:11:57.347296+0000","last_clean":"2024-04-26T18:11:57.347296+0000","last_became_active":"2024-04-26T18:09:33.021446+0000","last_became_peered":"2024-04-26T18:09:33.021446+0000","last_unstale":"2024-04-26T18:11:57.347296+0000","last_undegraded":"2024-04-26T18:11:57.347296+0000","last_fullsized":"2024-04-26T18:11:57.347296+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:29.158581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252002+0000","last_change":"2024-04-26T18:08:44.555076+0000","last_active":"2024-04-26T18:11:57.252002+0000","last_peered":"2024-04-26T18:11:57.252002+0000","last_clean":"2024-04-26T18:11:57.252002+0000","last_became_active":"2024-04-26T18:08:44.554756+0000","last_became_peered":"2024-04-26T18:08:44.554756+0000","last_unstale":"2024-04-26T18:11:57.252002+0000","last_undegraded":"2024-04-26T18:11:57.252002+0000","last_fullsized":"2024-04-26T18:11:57.252002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:21.950644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207518+0000","last_change":"2024-04-26T18:08:47.366370+0000","last_active":"2024-04-26T18:11:57.207518+0000","last_peered":"2024-04-26T18:11:57.207518+0000","last_clean":"2024-04-26T18:11:57.207518+0000","last_became_active":"2024-04-26T18:08:47.365997+0000","last_became_peered":"2024-04-26T18:08:47.365997+0000","last_unstale":"2024-04-26T18:11:57.207518+0000","last_undegraded":"2024-04-26T18:11:57.207518+0000","last_fullsized":"2024-04-26T18:11:57.207518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:51.643438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347119+0000","last_change":"2024-04-26T18:08:44.339299+0000","last_active":"2024-04-26T18:11:57.347119+0000","last_peered":"2024-04-26T18:11:57.347119+0000","last_clean":"2024-04-26T18:11:57.347119+0000","last_became_active":"2024-04-26T18:08:44.338932+0000","last_became_peered":"2024-04-26T18:08:44.338932+0000","last_unstale":"2024-04-26T18:11:57.347119+0000","last_undegraded":"2024-04-26T18:11:57.347119+0000","last_fullsized":"2024-04-26T18:11:57.347119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:18.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207445+0000","last_change":"2024-04-26T18:08:47.369924+0000","last_active":"2024-04-26T18:11:57.207445+0000","last_peered":"2024-04-26T18:11:57.207445+0000","last_clean":"2024-04-26T18:11:57.207445+0000","last_became_active":"2024-04-26T18:08:47.369048+0000","last_became_peered":"2024-04-26T18:08:47.369048+0000","last_unstale":"2024-04-26T18:11:57.207445+0000","last_undegraded":"2024-04-26T18:11:57.207445+0000","last_fullsized":"2024-04-26T18:11:57.207445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:19.110288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163797+0000","last_change":"2024-04-26T18:08:47.370836+0000","last_active":"2024-04-26T18:11:57.163797+0000","last_peered":"2024-04-26T18:11:57.163797+0000","last_clean":"2024-04-26T18:11:57.163797+0000","last_became_active":"2024-04-26T18:08:47.370160+0000","last_became_peered":"2024-04-26T18:08:47.370160+0000","last_unstale":"2024-04-26T18:11:57.163797+0000","last_undegraded":"2024-04-26T18:11:57.163797+0000","last_fullsized":"2024-04-26T18:11:57.163797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:11:21.285466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":214,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:01.239832+0000","last_change":"2024-04-26T18:08:44.554960+0000","last_active":"2024-04-26T18:12:01.239832+0000","last_peered":"2024-04-26T18:12:01.239832+0000","last_clean":"2024-04-26T18:12:01.239832+0000","last_became_active":"2024-04-26T18:08:44.554607+0000","last_became_peered":"2024-04-26T18:08:44.554607+0000","last_unstale":"2024-04-26T18:12:01.239832+0000","last_undegraded":"2024-04-26T18:12:01.239832+0000","last_fullsized":"2024-04-26T18:12:01.239832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:22:26.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235031+0000","last_change":"2024-04-26T18:08:47.372528+0000","last_active":"2024-04-26T18:11:57.235031+0000","last_peered":"2024-04-26T18:11:57.235031+0000","last_clean":"2024-04-26T18:11:57.235031+0000","last_became_active":"2024-04-26T18:08:47.372215+0000","last_became_peered":"2024-04-26T18:08:47.372215+0000","last_unstale":"2024-04-26T18:11:57.235031+0000","last_undegraded":"2024-04-26T18:11:57.235031+0000","last_fullsized":"2024-04-26T18:11:57.235031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:39.366917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161184+0000","last_change":"2024-04-26T18:08:44.342667+0000","last_active":"2024-04-26T18:11:57.161184+0000","last_peered":"2024-04-26T18:11:57.161184+0000","last_clean":"2024-04-26T18:11:57.161184+0000","last_became_active":"2024-04-26T18:08:44.342170+0000","last_became_peered":"2024-04-26T18:08:44.342170+0000","last_unstale":"2024-04-26T18:11:57.161184+0000","last_undegraded":"2024-04-26T18:11:57.161184+0000","last_fullsized":"2024-04-26T18:11:57.161184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:50.117177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":364,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346590+0000","last_change":"2024-04-26T18:07:24.173301+0000","last_active":"2024-04-26T18:11:57.346590+0000","last_peered":"2024-04-26T18:11:57.346590+0000","last_clean":"2024-04-26T18:11:57.346590+0000","last_became_active":"2024-04-26T18:07:24.172791+0000","last_became_peered":"2024-04-26T18:07:24.172791+0000","last_unstale":"2024-04-26T18:11:57.346590+0000","last_undegraded":"2024-04-26T18:11:57.346590+0000","last_fullsized":"2024-04-26T18:11:57.346590+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_clean_scrub_stamp":"2024-04-26T18:03:58.528256+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:32.468624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160143+0000","last_change":"2024-04-26T18:08:47.370720+0000","last_active":"2024-04-26T18:11:57.160143+0000","last_peered":"2024-04-26T18:11:57.160143+0000","last_clean":"2024-04-26T18:11:57.160143+0000","last_became_active":"2024-04-26T18:08:47.370414+0000","last_became_peered":"2024-04-26T18:08:47.370414+0000","last_unstale":"2024-04-26T18:11:57.160143+0000","last_undegraded":"2024-04-26T18:11:57.160143+0000","last_fullsized":"2024-04-26T18:11:57.160143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:37.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346466+0000","last_change":"2024-04-26T18:08:44.347077+0000","last_active":"2024-04-26T18:11:57.346466+0000","last_peered":"2024-04-26T18:11:57.346466+0000","last_clean":"2024-04-26T18:11:57.346466+0000","last_became_active":"2024-04-26T18:08:44.346481+0000","last_became_peered":"2024-04-26T18:08:44.346481+0000","last_unstale":"2024-04-26T18:11:57.346466+0000","last_undegraded":"2024-04-26T18:11:57.346466+0000","last_fullsized":"2024-04-26T18:11:57.346466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:08.840814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346980+0000","last_change":"2024-04-26T18:09:33.025243+0000","last_active":"2024-04-26T18:11:57.346980+0000","last_peered":"2024-04-26T18:11:57.346980+0000","last_clean":"2024-04-26T18:11:57.346980+0000","last_became_active":"2024-04-26T18:09:33.015430+0000","last_became_peered":"2024-04-26T18:09:33.015430+0000","last_unstale":"2024-04-26T18:11:57.346980+0000","last_undegraded":"2024-04-26T18:11:57.346980+0000","last_fullsized":"2024-04-26T18:11:57.346980+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:54.271435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160587+0000","last_change":"2024-04-26T18:08:47.366552+0000","last_active":"2024-04-26T18:11:57.160587+0000","last_peered":"2024-04-26T18:11:57.160587+0000","last_clean":"2024-04-26T18:11:57.160587+0000","last_became_active":"2024-04-26T18:08:47.366183+0000","last_became_peered":"2024-04-26T18:08:47.366183+0000","last_unstale":"2024-04-26T18:11:57.160587+0000","last_undegraded":"2024-04-26T18:11:57.160587+0000","last_fullsized":"2024-04-26T18:11:57.160587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:36.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'48","reported_seq":100,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.367151+0000","last_change":"2024-04-26T18:08:44.338078+0000","last_active":"2024-04-26T18:11:57.367151+0000","last_peered":"2024-04-26T18:11:57.367151+0000","last_clean":"2024-04-26T18:11:57.367151+0000","last_became_active":"2024-04-26T18:08:44.337715+0000","last_became_peered":"2024-04-26T18:08:44.337715+0000","last_unstale":"2024-04-26T18:11:57.367151+0000","last_undegraded":"2024-04-26T18:11:57.367151+0000","last_fullsized":"2024-04-26T18:11:57.367151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.912993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":86,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235147+0000","last_change":"2024-04-26T18:08:47.371321+0000","last_active":"2024-04-26T18:11:57.235147+0000","last_peered":"2024-04-26T18:11:57.235147+0000","last_clean":"2024-04-26T18:11:57.235147+0000","last_became_active":"2024-04-26T18:08:47.371015+0000","last_became_peered":"2024-04-26T18:08:47.371015+0000","last_unstale":"2024-04-26T18:11:57.235147+0000","last_undegraded":"2024-04-26T18:11:57.235147+0000","last_fullsized":"2024-04-26T18:11:57.235147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161387+0000","last_change":"2024-04-26T18:08:44.420101+0000","last_active":"2024-04-26T18:11:57.161387+0000","last_peered":"2024-04-26T18:11:57.161387+0000","last_clean":"2024-04-26T18:11:57.161387+0000","last_became_active":"2024-04-26T18:08:44.419787+0000","last_became_peered":"2024-04-26T18:08:44.419787+0000","last_unstale":"2024-04-26T18:11:57.161387+0000","last_undegraded":"2024-04-26T18:11:57.161387+0000","last_fullsized":"2024-04-26T18:11:57.161387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.485425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347044+0000","last_change":"2024-04-26T18:08:47.372309+0000","last_active":"2024-04-26T18:11:57.347044+0000","last_peered":"2024-04-26T18:11:57.347044+0000","last_clean":"2024-04-26T18:11:57.347044+0000","last_became_active":"2024-04-26T18:08:47.371980+0000","last_became_peered":"2024-04-26T18:08:47.371980+0000","last_unstale":"2024-04-26T18:11:57.347044+0000","last_undegraded":"2024-04-26T18:11:57.347044+0000","last_fullsized":"2024-04-26T18:11:57.347044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:05.011077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346322+0000","last_change":"2024-04-26T18:08:44.555500+0000","last_active":"2024-04-26T18:11:57.346322+0000","last_peered":"2024-04-26T18:11:57.346322+0000","last_clean":"2024-04-26T18:11:57.346322+0000","last_became_active":"2024-04-26T18:08:44.555137+0000","last_became_peered":"2024-04-26T18:08:44.555137+0000","last_unstale":"2024-04-26T18:11:57.346322+0000","last_undegraded":"2024-04-26T18:11:57.346322+0000","last_fullsized":"2024-04-26T18:11:57.346322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:10:05.723654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254027+0000","last_change":"2024-04-26T18:08:47.364215+0000","last_active":"2024-04-26T18:11:57.254027+0000","last_peered":"2024-04-26T18:11:57.254027+0000","last_clean":"2024-04-26T18:11:57.254027+0000","last_became_active":"2024-04-26T18:08:47.363862+0000","last_became_peered":"2024-04-26T18:08:47.363862+0000","last_unstale":"2024-04-26T18:11:57.254027+0000","last_undegraded":"2024-04-26T18:11:57.254027+0000","last_fullsized":"2024-04-26T18:11:57.254027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:27.020100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234809+0000","last_change":"2024-04-26T18:08:44.343506+0000","last_active":"2024-04-26T18:11:57.234809+0000","last_peered":"2024-04-26T18:11:57.234809+0000","last_clean":"2024-04-26T18:11:57.234809+0000","last_became_active":"2024-04-26T18:08:44.343163+0000","last_became_peered":"2024-04-26T18:08:44.343163+0000","last_unstale":"2024-04-26T18:11:57.234809+0000","last_undegraded":"2024-04-26T18:11:57.234809+0000","last_fullsized":"2024-04-26T18:11:57.234809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:09.941391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346893+0000","last_change":"2024-04-26T18:08:47.365572+0000","last_active":"2024-04-26T18:11:57.346893+0000","last_peered":"2024-04-26T18:11:57.346893+0000","last_clean":"2024-04-26T18:11:57.346893+0000","last_became_active":"2024-04-26T18:08:47.364827+0000","last_became_peered":"2024-04-26T18:08:47.364827+0000","last_unstale":"2024-04-26T18:11:57.346893+0000","last_undegraded":"2024-04-26T18:11:57.346893+0000","last_fullsized":"2024-04-26T18:11:57.346893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:44.492380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346591+0000","last_change":"2024-04-26T18:08:44.339063+0000","last_active":"2024-04-26T18:11:57.346591+0000","last_peered":"2024-04-26T18:11:57.346591+0000","last_clean":"2024-04-26T18:11:57.346591+0000","last_became_active":"2024-04-26T18:08:44.338752+0000","last_became_peered":"2024-04-26T18:08:44.338752+0000","last_unstale":"2024-04-26T18:11:57.346591+0000","last_undegraded":"2024-04-26T18:11:57.346591+0000","last_fullsized":"2024-04-26T18:11:57.346591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:38.201263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235138+0000","last_change":"2024-04-26T18:08:47.367859+0000","last_active":"2024-04-26T18:11:57.235138+0000","last_peered":"2024-04-26T18:11:57.235138+0000","last_clean":"2024-04-26T18:11:57.235138+0000","last_became_active":"2024-04-26T18:08:47.367548+0000","last_became_peered":"2024-04-26T18:08:47.367548+0000","last_unstale":"2024-04-26T18:11:57.235138+0000","last_undegraded":"2024-04-26T18:11:57.235138+0000","last_fullsized":"2024-04-26T18:11:57.235138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:13:27.612151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254111+0000","last_change":"2024-04-26T18:08:44.345392+0000","last_active":"2024-04-26T18:11:57.254111+0000","last_peered":"2024-04-26T18:11:57.254111+0000","last_clean":"2024-04-26T18:11:57.254111+0000","last_became_active":"2024-04-26T18:08:44.344578+0000","last_became_peered":"2024-04-26T18:08:44.344578+0000","last_unstale":"2024-04-26T18:11:57.254111+0000","last_undegraded":"2024-04-26T18:11:57.254111+0000","last_fullsized":"2024-04-26T18:11:57.254111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:26.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347131+0000","last_change":"2024-04-26T18:08:47.380099+0000","last_active":"2024-04-26T18:11:57.347131+0000","last_peered":"2024-04-26T18:11:57.347131+0000","last_clean":"2024-04-26T18:11:57.347131+0000","last_became_active":"2024-04-26T18:08:47.379576+0000","last_became_peered":"2024-04-26T18:08:47.379576+0000","last_unstale":"2024-04-26T18:11:57.347131+0000","last_undegraded":"2024-04-26T18:11:57.347131+0000","last_fullsized":"2024-04-26T18:11:57.347131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:28.397886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347023+0000","last_change":"2024-04-26T18:08:44.555299+0000","last_active":"2024-04-26T18:11:57.347023+0000","last_peered":"2024-04-26T18:11:57.347023+0000","last_clean":"2024-04-26T18:11:57.347023+0000","last_became_active":"2024-04-26T18:08:44.554959+0000","last_became_peered":"2024-04-26T18:08:44.554959+0000","last_unstale":"2024-04-26T18:11:57.347023+0000","last_undegraded":"2024-04-26T18:11:57.347023+0000","last_fullsized":"2024-04-26T18:11:57.347023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.565073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347338+0000","last_change":"2024-04-26T18:08:47.377524+0000","last_active":"2024-04-26T18:11:57.347338+0000","last_peered":"2024-04-26T18:11:57.347338+0000","last_clean":"2024-04-26T18:11:57.347338+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:11:57.347338+0000","last_undegraded":"2024-04-26T18:11:57.347338+0000","last_fullsized":"2024-04-26T18:11:57.347338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:10.466615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'39","reported_seq":76,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.364671+0000","last_change":"2024-04-26T18:08:44.343894+0000","last_active":"2024-04-26T18:11:57.364671+0000","last_peered":"2024-04-26T18:11:57.364671+0000","last_clean":"2024-04-26T18:11:57.364671+0000","last_became_active":"2024-04-26T18:08:44.343153+0000","last_became_peered":"2024-04-26T18:08:44.343153+0000","last_unstale":"2024-04-26T18:11:57.364671+0000","last_undegraded":"2024-04-26T18:11:57.364671+0000","last_fullsized":"2024-04-26T18:11:57.364671+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:09.914701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347161+0000","last_change":"2024-04-26T18:08:47.371962+0000","last_active":"2024-04-26T18:11:57.347161+0000","last_peered":"2024-04-26T18:11:57.347161+0000","last_clean":"2024-04-26T18:11:57.347161+0000","last_became_active":"2024-04-26T18:08:47.371517+0000","last_became_peered":"2024-04-26T18:08:47.371517+0000","last_unstale":"2024-04-26T18:11:57.347161+0000","last_undegraded":"2024-04-26T18:11:57.347161+0000","last_fullsized":"2024-04-26T18:11:57.347161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:23.553409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160624+0000","last_change":"2024-04-26T18:08:44.337853+0000","last_active":"2024-04-26T18:11:57.160624+0000","last_peered":"2024-04-26T18:11:57.160624+0000","last_clean":"2024-04-26T18:11:57.160624+0000","last_became_active":"2024-04-26T18:08:44.337393+0000","last_became_peered":"2024-04-26T18:08:44.337393+0000","last_unstale":"2024-04-26T18:11:57.160624+0000","last_undegraded":"2024-04-26T18:11:57.160624+0000","last_fullsized":"2024-04-26T18:11:57.160624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:26.514821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346960+0000","last_change":"2024-04-26T18:08:47.370999+0000","last_active":"2024-04-26T18:11:57.346960+0000","last_peered":"2024-04-26T18:11:57.346960+0000","last_clean":"2024-04-26T18:11:57.346960+0000","last_became_active":"2024-04-26T18:08:47.370373+0000","last_became_peered":"2024-04-26T18:08:47.370373+0000","last_unstale":"2024-04-26T18:11:57.346960+0000","last_undegraded":"2024-04-26T18:11:57.346960+0000","last_fullsized":"2024-04-26T18:11:57.346960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:19.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346894+0000","last_change":"2024-04-26T18:08:47.365722+0000","last_active":"2024-04-26T18:11:57.346894+0000","last_peered":"2024-04-26T18:11:57.346894+0000","last_clean":"2024-04-26T18:11:57.346894+0000","last_became_active":"2024-04-26T18:08:47.364692+0000","last_became_peered":"2024-04-26T18:08:47.364692+0000","last_unstale":"2024-04-26T18:11:57.346894+0000","last_undegraded":"2024-04-26T18:11:57.346894+0000","last_fullsized":"2024-04-26T18:11:57.346894+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:02.158156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346658+0000","last_change":"2024-04-26T18:08:47.373886+0000","last_active":"2024-04-26T18:11:57.346658+0000","last_peered":"2024-04-26T18:11:57.346658+0000","last_clean":"2024-04-26T18:11:57.346658+0000","last_became_active":"2024-04-26T18:08:47.373391+0000","last_became_peered":"2024-04-26T18:08:47.373391+0000","last_unstale":"2024-04-26T18:11:57.346658+0000","last_undegraded":"2024-04-26T18:11:57.346658+0000","last_fullsized":"2024-04-26T18:11:57.346658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:48:59.280050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346367+0000","last_change":"2024-04-26T18:08:47.366160+0000","last_active":"2024-04-26T18:11:57.346367+0000","last_peered":"2024-04-26T18:11:57.346367+0000","last_clean":"2024-04-26T18:11:57.346367+0000","last_became_active":"2024-04-26T18:08:47.365811+0000","last_became_peered":"2024-04-26T18:08:47.365811+0000","last_unstale":"2024-04-26T18:11:57.346367+0000","last_undegraded":"2024-04-26T18:11:57.346367+0000","last_fullsized":"2024-04-26T18:11:57.346367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:32.124692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346737+0000","last_change":"2024-04-26T18:08:47.366078+0000","last_active":"2024-04-26T18:11:57.346737+0000","last_peered":"2024-04-26T18:11:57.346737+0000","last_clean":"2024-04-26T18:11:57.346737+0000","last_became_active":"2024-04-26T18:08:47.365779+0000","last_became_peered":"2024-04-26T18:08:47.365779+0000","last_unstale":"2024-04-26T18:11:57.346737+0000","last_undegraded":"2024-04-26T18:11:57.346737+0000","last_fullsized":"2024-04-26T18:11:57.346737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.127606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160654+0000","last_change":"2024-04-26T18:08:47.372996+0000","last_active":"2024-04-26T18:11:57.160654+0000","last_peered":"2024-04-26T18:11:57.160654+0000","last_clean":"2024-04-26T18:11:57.160654+0000","last_became_active":"2024-04-26T18:08:47.372687+0000","last_became_peered":"2024-04-26T18:08:47.372687+0000","last_unstale":"2024-04-26T18:11:57.160654+0000","last_undegraded":"2024-04-26T18:11:57.160654+0000","last_fullsized":"2024-04-26T18:11:57.160654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:13:33.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347315+0000","last_change":"2024-04-26T18:08:47.371076+0000","last_active":"2024-04-26T18:11:57.347315+0000","last_peered":"2024-04-26T18:11:57.347315+0000","last_clean":"2024-04-26T18:11:57.347315+0000","last_became_active":"2024-04-26T18:08:47.370688+0000","last_became_peered":"2024-04-26T18:08:47.370688+0000","last_unstale":"2024-04-26T18:11:57.347315+0000","last_undegraded":"2024-04-26T18:11:57.347315+0000","last_fullsized":"2024-04-26T18:11:57.347315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:21.355971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347368+0000","last_change":"2024-04-26T18:08:47.372392+0000","last_active":"2024-04-26T18:11:57.347368+0000","last_peered":"2024-04-26T18:11:57.347368+0000","last_clean":"2024-04-26T18:11:57.347368+0000","last_became_active":"2024-04-26T18:08:47.372020+0000","last_became_peered":"2024-04-26T18:08:47.372020+0000","last_unstale":"2024-04-26T18:11:57.347368+0000","last_undegraded":"2024-04-26T18:11:57.347368+0000","last_fullsized":"2024-04-26T18:11:57.347368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235255+0000","last_change":"2024-04-26T18:08:47.369571+0000","last_active":"2024-04-26T18:11:57.235255+0000","last_peered":"2024-04-26T18:11:57.235255+0000","last_clean":"2024-04-26T18:11:57.235255+0000","last_became_active":"2024-04-26T18:08:47.369165+0000","last_became_peered":"2024-04-26T18:08:47.369165+0000","last_unstale":"2024-04-26T18:11:57.235255+0000","last_undegraded":"2024-04-26T18:11:57.235255+0000","last_fullsized":"2024-04-26T18:11:57.235255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:15.869182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163609+0000","last_change":"2024-04-26T18:08:47.372355+0000","last_active":"2024-04-26T18:11:57.163609+0000","last_peered":"2024-04-26T18:11:57.163609+0000","last_clean":"2024-04-26T18:11:57.163609+0000","last_became_active":"2024-04-26T18:08:47.371810+0000","last_became_peered":"2024-04-26T18:08:47.371810+0000","last_unstale":"2024-04-26T18:11:57.163609+0000","last_undegraded":"2024-04-26T18:11:57.163609+0000","last_fullsized":"2024-04-26T18:11:57.163609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:44:22.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234697+0000","last_change":"2024-04-26T18:08:47.364042+0000","last_active":"2024-04-26T18:11:57.234697+0000","last_peered":"2024-04-26T18:11:57.234697+0000","last_clean":"2024-04-26T18:11:57.234697+0000","last_became_active":"2024-04-26T18:08:47.363727+0000","last_became_peered":"2024-04-26T18:08:47.363727+0000","last_unstale":"2024-04-26T18:11:57.234697+0000","last_undegraded":"2024-04-26T18:11:57.234697+0000","last_fullsized":"2024-04-26T18:11:57.234697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:37.595261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346555+0000","last_change":"2024-04-26T18:08:47.377535+0000","last_active":"2024-04-26T18:11:57.346555+0000","last_peered":"2024-04-26T18:11:57.346555+0000","last_clean":"2024-04-26T18:11:57.346555+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:11:57.346555+0000","last_undegraded":"2024-04-26T18:11:57.346555+0000","last_fullsized":"2024-04-26T18:11:57.346555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:22.895402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160418+0000","last_change":"2024-04-26T18:08:47.367313+0000","last_active":"2024-04-26T18:11:57.160418+0000","last_peered":"2024-04-26T18:11:57.160418+0000","last_clean":"2024-04-26T18:11:57.160418+0000","last_became_active":"2024-04-26T18:08:47.366971+0000","last_became_peered":"2024-04-26T18:08:47.366971+0000","last_unstale":"2024-04-26T18:11:57.160418+0000","last_undegraded":"2024-04-26T18:11:57.160418+0000","last_fullsized":"2024-04-26T18:11:57.160418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:14.518100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347070+0000","last_change":"2024-04-26T18:08:47.373004+0000","last_active":"2024-04-26T18:11:57.347070+0000","last_peered":"2024-04-26T18:11:57.347070+0000","last_clean":"2024-04-26T18:11:57.347070+0000","last_became_active":"2024-04-26T18:08:47.372641+0000","last_became_peered":"2024-04-26T18:08:47.372641+0000","last_unstale":"2024-04-26T18:11:57.347070+0000","last_undegraded":"2024-04-26T18:11:57.347070+0000","last_fullsized":"2024-04-26T18:11:57.347070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:02.018425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597137,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1173,"num_read_kb":691,"num_write":296,"num_write_kb":1440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1791411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":498476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579961,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28892,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716260,"statfs":{"total":95995035648,"available":95965450240,"internally_reserved":0,"allocated":1859584,"data_stored":1288898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743487,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28416,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716736,"statfs":{"total":95995035648,"available":95965937664,"internally_reserved":0,"allocated":1429504,"data_stored":891872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005125,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1425408,"data_stored":891770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266762,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708240,"statfs":{"total":95995035648,"available":95957237760,"internally_reserved":0,"allocated":1613824,"data_stored":1082310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462993,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":498045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":564312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":499316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":566942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182494,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":489487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":499014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724628,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37072,"kb_used_data":1736,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708080,"statfs":{"total":95995035648,"available":95957073920,"internally_reserved":0,"allocated":1777664,"data_stored":1222713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":69104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:12:05.235 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:12:05.236 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:12:05.483 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:05.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:05 smithi090 ceph-mon[29180]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:05 smithi119 ceph-mon[32752]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:05 smithi167 ceph-mon[32751]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:06 smithi090 ceph-mon[29180]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 169 KiB/s wr, 102 op/s 2024-04-26T18:12:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:06 smithi119 ceph-mon[32752]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 169 KiB/s wr, 102 op/s 2024-04-26T18:12:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:06 smithi167 ceph-mon[32751]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 169 KiB/s wr, 102 op/s 2024-04-26T18:12:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 -- 172.21.15.90:0/2651737418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c105e60 msgr2=0x7fcd9c1062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 --2- 172.21.15.90:0/2651737418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c105e60 0x7fcd9c1062f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcd9400b0a0 tx=0x7fcd9400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 -- 172.21.15.90:0/2651737418 shutdown_connections 2024-04-26T18:12:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 --2- 172.21.15.90:0/2651737418 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c105e60 0x7fcd9c1062f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 --2- 172.21.15.90:0/2651737418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd9c108350 0x7fcd9c06bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 --2- 172.21.15.90:0/2651737418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9c06c100 0x7fcd9c06c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 -- 172.21.15.90:0/2651737418 >> 172.21.15.90:0/2651737418 conn(0x7fcd9c0792d0 msgr2=0x7fcd9c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 -- 172.21.15.90:0/2651737418 shutdown_connections 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.192+0000 7fcda26d3700 1 -- 172.21.15.90:0/2651737418 wait complete. 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 Processor -- start 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 -- start start 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd9c06c100 0x7fcd9c19ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9c105e60 0x7fcd9c1a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c108350 0x7fcd9c1a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c119b90 con 0x7fcd9c108350 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c119a10 con 0x7fcd9c105e60 2024-04-26T18:12:07.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda26d3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9c119d10 con 0x7fcd9c06c100 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c108350 0x7fcd9c1a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c108350 0x7fcd9c1a4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50326/0 (socket says 172.21.15.90:50326) 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 -- 172.21.15.90:0/1750796714 learned_addr learned my addr 172.21.15.90:0/1750796714 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcd9bfff700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd9c06c100 0x7fcd9c19ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcd9b7fe700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9c105e60 0x7fcd9c1a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 -- 172.21.15.90:0/1750796714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd9c06c100 msgr2=0x7fcd9c19ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd9c06c100 0x7fcd9c19ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 -- 172.21.15.90:0/1750796714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9c105e60 msgr2=0x7fcd9c1a0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9c105e60 0x7fcd9c1a0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.193+0000 7fcda0c70700 1 -- 172.21.15.90:0/1750796714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd9c1a4fd0 con 0x7fcd9c108350 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.194+0000 7fcda0c70700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c108350 0x7fcd9c1a4890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcd940062a0 tx=0x7fcd94004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.194+0000 7fcd997fa700 1 -- 172.21.15.90:0/1750796714 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94021070 con 0x7fcd9c108350 2024-04-26T18:12:07.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.194+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd9c1a5260 con 0x7fcd9c108350 2024-04-26T18:12:07.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.194+0000 7fcd997fa700 1 -- 172.21.15.90:0/1750796714 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd94003c00 con 0x7fcd9c108350 2024-04-26T18:12:07.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.194+0000 7fcd997fa700 1 -- 172.21.15.90:0/1750796714 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd94004880 con 0x7fcd9c108350 2024-04-26T18:12:07.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.194+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd9c1acb40 con 0x7fcd9c108350 2024-04-26T18:12:07.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.195+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9c102940 con 0x7fcd9c108350 2024-04-26T18:12:07.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.197+0000 7fcd997fa700 1 -- 172.21.15.90:0/1750796714 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7fcd94007a00 con 0x7fcd9c108350 2024-04-26T18:12:07.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.197+0000 7fcd997fa700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd840a3ed0 0x7fcd840a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:07.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.197+0000 7fcd9bfff700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd840a3ed0 0x7fcd840a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:07.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.197+0000 7fcd997fa700 1 -- 172.21.15.90:0/1750796714 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fcd940d1af0 con 0x7fcd9c108350 2024-04-26T18:12:07.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.198+0000 7fcd9bfff700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd840a3ed0 0x7fcd840a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00abd0 tx=0x7fcd8c00a380 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:07.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.201+0000 7fcd997fa700 1 -- 172.21.15.90:0/1750796714 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd9409e040 con 0x7fcd9c108350 2024-04-26T18:12:07.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.404+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcd9c105550 con 0x7fcd840a3ed0 2024-04-26T18:12:07.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.411+0000 7fcd997fa700 1 -- 172.21.15.90:0/1750796714 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257775 (secure 0 0 0) 0x7fcd9c105550 con 0x7fcd840a3ed0 2024-04-26T18:12:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd840a3ed0 msgr2=0x7fcd840a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd840a3ed0 0x7fcd840a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00abd0 tx=0x7fcd8c00a380 comp rx=0 tx=0).stop 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c108350 msgr2=0x7fcd9c1a4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c108350 0x7fcd9c1a4890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcd940062a0 tx=0x7fcd94004e20 comp rx=0 tx=0).stop 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 shutdown_connections 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd840a3ed0 0x7fcd840a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcd9c108350 0x7fcd9c1a4890 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcd9c06c100 0x7fcd9c19ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 --2- 172.21.15.90:0/1750796714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd9c105e60 0x7fcd9c1a0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 >> 172.21.15.90:0/1750796714 conn(0x7fcd9c0792d0 msgr2=0x7fcd9c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:07.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 shutdown_connections 2024-04-26T18:12:07.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:07.415+0000 7fcda26d3700 1 -- 172.21.15.90:0/1750796714 wait complete. 2024-04-26T18:12:07.418 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T18:12:08.082 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-26T18:12:07.175404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1386200,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1814,"num_read_kb":1383,"num_write":685,"num_write_kb":5561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":998,"ondisk_log_size":998,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":382332,"kb_used_data":16224,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559492,"statfs":{"total":1151940427776,"available":1151548919808,"internally_reserved":0,"allocated":16613376,"data_stored":10132656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":374453563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":198461,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":510,"num_read_kb":578,"num_write":206,"num_write_kb":1181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.989866"},"pg_stats":[{"pgid":"3.3f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347391+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:11:57.347391+0000","last_peered":"2024-04-26T18:11:57.347391+0000","last_clean":"2024-04-26T18:11:57.347391+0000","last_became_active":"2024-04-26T18:08:47.377408+0000","last_became_peered":"2024-04-26T18:08:47.377408+0000","last_unstale":"2024-04-26T18:11:57.347391+0000","last_undegraded":"2024-04-26T18:11:57.347391+0000","last_fullsized":"2024-04-26T18:11:57.347391+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:13.822258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"92'4","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347009+0000","last_change":"2024-04-26T18:08:47.365125+0000","last_active":"2024-04-26T18:11:57.347009+0000","last_peered":"2024-04-26T18:11:57.347009+0000","last_clean":"2024-04-26T18:11:57.347009+0000","last_became_active":"2024-04-26T18:08:47.364847+0000","last_became_peered":"2024-04-26T18:08:47.364847+0000","last_unstale":"2024-04-26T18:11:57.347009+0000","last_undegraded":"2024-04-26T18:11:57.347009+0000","last_fullsized":"2024-04-26T18:11:57.347009+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:16:37.890893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346817+0000","last_change":"2024-04-26T18:08:47.377580+0000","last_active":"2024-04-26T18:11:57.346817+0000","last_peered":"2024-04-26T18:11:57.346817+0000","last_clean":"2024-04-26T18:11:57.346817+0000","last_became_active":"2024-04-26T18:08:47.377219+0000","last_became_peered":"2024-04-26T18:08:47.377219+0000","last_unstale":"2024-04-26T18:11:57.346817+0000","last_undegraded":"2024-04-26T18:11:57.346817+0000","last_fullsized":"2024-04-26T18:11:57.346817+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:12.398908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161389+0000","last_change":"2024-04-26T18:08:47.371628+0000","last_active":"2024-04-26T18:11:57.161389+0000","last_peered":"2024-04-26T18:11:57.161389+0000","last_clean":"2024-04-26T18:11:57.161389+0000","last_became_active":"2024-04-26T18:08:47.370840+0000","last_became_peered":"2024-04-26T18:08:47.370840+0000","last_unstale":"2024-04-26T18:11:57.161389+0000","last_undegraded":"2024-04-26T18:11:57.161389+0000","last_fullsized":"2024-04-26T18:11:57.161389+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:11.362664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254081+0000","last_change":"2024-04-26T18:08:47.367632+0000","last_active":"2024-04-26T18:11:57.254081+0000","last_peered":"2024-04-26T18:11:57.254081+0000","last_clean":"2024-04-26T18:11:57.254081+0000","last_became_active":"2024-04-26T18:08:47.367316+0000","last_became_peered":"2024-04-26T18:08:47.367316+0000","last_unstale":"2024-04-26T18:11:57.254081+0000","last_undegraded":"2024-04-26T18:11:57.254081+0000","last_fullsized":"2024-04-26T18:11:57.254081+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:31:26.665292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207571+0000","last_change":"2024-04-26T18:08:47.369742+0000","last_active":"2024-04-26T18:11:57.207571+0000","last_peered":"2024-04-26T18:11:57.207571+0000","last_clean":"2024-04-26T18:11:57.207571+0000","last_became_active":"2024-04-26T18:08:47.369230+0000","last_became_peered":"2024-04-26T18:08:47.369230+0000","last_unstale":"2024-04-26T18:11:57.207571+0000","last_undegraded":"2024-04-26T18:11:57.207571+0000","last_fullsized":"2024-04-26T18:11:57.207571+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:02:04.448727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346750+0000","last_change":"2024-04-26T18:08:47.370657+0000","last_active":"2024-04-26T18:11:57.346750+0000","last_peered":"2024-04-26T18:11:57.346750+0000","last_clean":"2024-04-26T18:11:57.346750+0000","last_became_active":"2024-04-26T18:08:47.370133+0000","last_became_peered":"2024-04-26T18:08:47.370133+0000","last_unstale":"2024-04-26T18:11:57.346750+0000","last_undegraded":"2024-04-26T18:11:57.346750+0000","last_fullsized":"2024-04-26T18:11:57.346750+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:31.232179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234654+0000","last_change":"2024-04-26T18:08:47.366221+0000","last_active":"2024-04-26T18:11:57.234654+0000","last_peered":"2024-04-26T18:11:57.234654+0000","last_clean":"2024-04-26T18:11:57.234654+0000","last_became_active":"2024-04-26T18:08:47.365990+0000","last_became_peered":"2024-04-26T18:08:47.365990+0000","last_unstale":"2024-04-26T18:11:57.234654+0000","last_undegraded":"2024-04-26T18:11:57.234654+0000","last_fullsized":"2024-04-26T18:11:57.234654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:43.707583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346856+0000","last_change":"2024-04-26T18:08:47.373165+0000","last_active":"2024-04-26T18:11:57.346856+0000","last_peered":"2024-04-26T18:11:57.346856+0000","last_clean":"2024-04-26T18:11:57.346856+0000","last_became_active":"2024-04-26T18:08:47.372688+0000","last_became_peered":"2024-04-26T18:08:47.372688+0000","last_unstale":"2024-04-26T18:11:57.346856+0000","last_undegraded":"2024-04-26T18:11:57.346856+0000","last_fullsized":"2024-04-26T18:11:57.346856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:03:18.319801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"92'16","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207762+0000","last_change":"2024-04-26T18:09:33.016811+0000","last_active":"2024-04-26T18:11:57.207762+0000","last_peered":"2024-04-26T18:11:57.207762+0000","last_clean":"2024-04-26T18:11:57.207762+0000","last_became_active":"2024-04-26T18:09:33.016267+0000","last_became_peered":"2024-04-26T18:09:33.016267+0000","last_unstale":"2024-04-26T18:11:57.207762+0000","last_undegraded":"2024-04-26T18:11:57.207762+0000","last_fullsized":"2024-04-26T18:11:57.207762+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:20:32.002914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347272+0000","last_change":"2024-04-26T18:08:47.378643+0000","last_active":"2024-04-26T18:11:57.347272+0000","last_peered":"2024-04-26T18:11:57.347272+0000","last_clean":"2024-04-26T18:11:57.347272+0000","last_became_active":"2024-04-26T18:08:47.378329+0000","last_became_peered":"2024-04-26T18:08:47.378329+0000","last_unstale":"2024-04-26T18:11:57.347272+0000","last_undegraded":"2024-04-26T18:11:57.347272+0000","last_fullsized":"2024-04-26T18:11:57.347272+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:36:59.016025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161243+0000","last_change":"2024-04-26T18:08:47.371663+0000","last_active":"2024-04-26T18:11:57.161243+0000","last_peered":"2024-04-26T18:11:57.161243+0000","last_clean":"2024-04-26T18:11:57.161243+0000","last_became_active":"2024-04-26T18:08:47.371298+0000","last_became_peered":"2024-04-26T18:08:47.371298+0000","last_unstale":"2024-04-26T18:11:57.161243+0000","last_undegraded":"2024-04-26T18:11:57.161243+0000","last_fullsized":"2024-04-26T18:11:57.161243+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:04.884698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160404+0000","last_change":"2024-04-26T18:08:47.365179+0000","last_active":"2024-04-26T18:11:57.160404+0000","last_peered":"2024-04-26T18:11:57.160404+0000","last_clean":"2024-04-26T18:11:57.160404+0000","last_became_active":"2024-04-26T18:08:47.364888+0000","last_became_peered":"2024-04-26T18:08:47.364888+0000","last_unstale":"2024-04-26T18:11:57.160404+0000","last_undegraded":"2024-04-26T18:11:57.160404+0000","last_fullsized":"2024-04-26T18:11:57.160404+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:34.082855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346532+0000","last_change":"2024-04-26T18:08:47.364828+0000","last_active":"2024-04-26T18:11:57.346532+0000","last_peered":"2024-04-26T18:11:57.346532+0000","last_clean":"2024-04-26T18:11:57.346532+0000","last_became_active":"2024-04-26T18:08:47.364529+0000","last_became_peered":"2024-04-26T18:08:47.364529+0000","last_unstale":"2024-04-26T18:11:57.346532+0000","last_undegraded":"2024-04-26T18:11:57.346532+0000","last_fullsized":"2024-04-26T18:11:57.346532+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:23.480258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346255+0000","last_change":"2024-04-26T18:08:47.373578+0000","last_active":"2024-04-26T18:11:57.346255+0000","last_peered":"2024-04-26T18:11:57.346255+0000","last_clean":"2024-04-26T18:11:57.346255+0000","last_became_active":"2024-04-26T18:08:47.373263+0000","last_became_peered":"2024-04-26T18:08:47.373263+0000","last_unstale":"2024-04-26T18:11:57.346255+0000","last_undegraded":"2024-04-26T18:11:57.346255+0000","last_fullsized":"2024-04-26T18:11:57.346255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:30:25.340241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346347+0000","last_change":"2024-04-26T18:08:47.376617+0000","last_active":"2024-04-26T18:11:57.346347+0000","last_peered":"2024-04-26T18:11:57.346347+0000","last_clean":"2024-04-26T18:11:57.346347+0000","last_became_active":"2024-04-26T18:08:47.376290+0000","last_became_peered":"2024-04-26T18:08:47.376290+0000","last_unstale":"2024-04-26T18:11:57.346347+0000","last_undegraded":"2024-04-26T18:11:57.346347+0000","last_fullsized":"2024-04-26T18:11:57.346347+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:51.187583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347552+0000","last_change":"2024-04-26T18:08:47.377828+0000","last_active":"2024-04-26T18:11:57.347552+0000","last_peered":"2024-04-26T18:11:57.347552+0000","last_clean":"2024-04-26T18:11:57.347552+0000","last_became_active":"2024-04-26T18:08:47.377427+0000","last_became_peered":"2024-04-26T18:08:47.377427+0000","last_unstale":"2024-04-26T18:11:57.347552+0000","last_undegraded":"2024-04-26T18:11:57.347552+0000","last_fullsized":"2024-04-26T18:11:57.347552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:45.529500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252164+0000","last_change":"2024-04-26T18:08:47.373314+0000","last_active":"2024-04-26T18:11:57.252164+0000","last_peered":"2024-04-26T18:11:57.252164+0000","last_clean":"2024-04-26T18:11:57.252164+0000","last_became_active":"2024-04-26T18:08:47.372868+0000","last_became_peered":"2024-04-26T18:08:47.372868+0000","last_unstale":"2024-04-26T18:11:57.252164+0000","last_undegraded":"2024-04-26T18:11:57.252164+0000","last_fullsized":"2024-04-26T18:11:57.252164+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:07:19.642655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347122+0000","last_change":"2024-04-26T18:08:47.367593+0000","last_active":"2024-04-26T18:11:57.347122+0000","last_peered":"2024-04-26T18:11:57.347122+0000","last_clean":"2024-04-26T18:11:57.347122+0000","last_became_active":"2024-04-26T18:08:47.367243+0000","last_became_peered":"2024-04-26T18:08:47.367243+0000","last_unstale":"2024-04-26T18:11:57.347122+0000","last_undegraded":"2024-04-26T18:11:57.347122+0000","last_fullsized":"2024-04-26T18:11:57.347122+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:23.380307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'9","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252032+0000","last_change":"2024-04-26T18:08:44.339625+0000","last_active":"2024-04-26T18:11:57.252032+0000","last_peered":"2024-04-26T18:11:57.252032+0000","last_clean":"2024-04-26T18:11:57.252032+0000","last_became_active":"2024-04-26T18:08:44.339209+0000","last_became_peered":"2024-04-26T18:08:44.339209+0000","last_unstale":"2024-04-26T18:11:57.252032+0000","last_undegraded":"2024-04-26T18:11:57.252032+0000","last_fullsized":"2024-04-26T18:11:57.252032+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:43.169885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234859+0000","last_change":"2024-04-26T18:08:47.375884+0000","last_active":"2024-04-26T18:11:57.234859+0000","last_peered":"2024-04-26T18:11:57.234859+0000","last_clean":"2024-04-26T18:11:57.234859+0000","last_became_active":"2024-04-26T18:08:47.375557+0000","last_became_peered":"2024-04-26T18:08:47.375557+0000","last_unstale":"2024-04-26T18:11:57.234859+0000","last_undegraded":"2024-04-26T18:11:57.234859+0000","last_fullsized":"2024-04-26T18:11:57.234859+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:19.593910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235169+0000","last_change":"2024-04-26T18:08:44.557167+0000","last_active":"2024-04-26T18:11:57.235169+0000","last_peered":"2024-04-26T18:11:57.235169+0000","last_clean":"2024-04-26T18:11:57.235169+0000","last_became_active":"2024-04-26T18:08:44.556716+0000","last_became_peered":"2024-04-26T18:08:44.556716+0000","last_unstale":"2024-04-26T18:11:57.235169+0000","last_undegraded":"2024-04-26T18:11:57.235169+0000","last_fullsized":"2024-04-26T18:11:57.235169+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:08.108329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"92'6","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347151+0000","last_change":"2024-04-26T18:08:47.377612+0000","last_active":"2024-04-26T18:11:57.347151+0000","last_peered":"2024-04-26T18:11:57.347151+0000","last_clean":"2024-04-26T18:11:57.347151+0000","last_became_active":"2024-04-26T18:08:47.377231+0000","last_became_peered":"2024-04-26T18:08:47.377231+0000","last_unstale":"2024-04-26T18:11:57.347151+0000","last_undegraded":"2024-04-26T18:11:57.347151+0000","last_fullsized":"2024-04-26T18:11:57.347151+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:35:11.594373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254362+0000","last_change":"2024-04-26T18:08:44.343429+0000","last_active":"2024-04-26T18:11:57.254362+0000","last_peered":"2024-04-26T18:11:57.254362+0000","last_clean":"2024-04-26T18:11:57.254362+0000","last_became_active":"2024-04-26T18:08:44.343103+0000","last_became_peered":"2024-04-26T18:08:44.343103+0000","last_unstale":"2024-04-26T18:11:57.254362+0000","last_undegraded":"2024-04-26T18:11:57.254362+0000","last_fullsized":"2024-04-26T18:11:57.254362+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:19:52.244159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346545+0000","last_change":"2024-04-26T18:08:47.373130+0000","last_active":"2024-04-26T18:11:57.346545+0000","last_peered":"2024-04-26T18:11:57.346545+0000","last_clean":"2024-04-26T18:11:57.346545+0000","last_became_active":"2024-04-26T18:08:47.372726+0000","last_became_peered":"2024-04-26T18:08:47.372726+0000","last_unstale":"2024-04-26T18:11:57.346545+0000","last_undegraded":"2024-04-26T18:11:57.346545+0000","last_fullsized":"2024-04-26T18:11:57.346545+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:44.461177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254325+0000","last_change":"2024-04-26T18:08:44.340152+0000","last_active":"2024-04-26T18:11:57.254325+0000","last_peered":"2024-04-26T18:11:57.254325+0000","last_clean":"2024-04-26T18:11:57.254325+0000","last_became_active":"2024-04-26T18:08:44.339684+0000","last_became_peered":"2024-04-26T18:08:44.339684+0000","last_unstale":"2024-04-26T18:11:57.254325+0000","last_undegraded":"2024-04-26T18:11:57.254325+0000","last_fullsized":"2024-04-26T18:11:57.254325+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:49.961201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.164063+0000","last_change":"2024-04-26T18:08:47.375387+0000","last_active":"2024-04-26T18:11:57.164063+0000","last_peered":"2024-04-26T18:11:57.164063+0000","last_clean":"2024-04-26T18:11:57.164063+0000","last_became_active":"2024-04-26T18:08:47.374689+0000","last_became_peered":"2024-04-26T18:08:47.374689+0000","last_unstale":"2024-04-26T18:11:57.164063+0000","last_undegraded":"2024-04-26T18:11:57.164063+0000","last_fullsized":"2024-04-26T18:11:57.164063+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:05.618804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'10","reported_seq":224,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:01.215299+0000","last_change":"2024-04-26T18:08:44.344167+0000","last_active":"2024-04-26T18:12:01.215299+0000","last_peered":"2024-04-26T18:12:01.215299+0000","last_clean":"2024-04-26T18:12:01.215299+0000","last_became_active":"2024-04-26T18:08:44.343646+0000","last_became_peered":"2024-04-26T18:08:44.343646+0000","last_unstale":"2024-04-26T18:12:01.215299+0000","last_undegraded":"2024-04-26T18:12:01.215299+0000","last_fullsized":"2024-04-26T18:12:01.215299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:51:53.859483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2327,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254051+0000","last_change":"2024-04-26T18:08:47.370546+0000","last_active":"2024-04-26T18:11:57.254051+0000","last_peered":"2024-04-26T18:11:57.254051+0000","last_clean":"2024-04-26T18:11:57.254051+0000","last_became_active":"2024-04-26T18:08:47.370220+0000","last_became_peered":"2024-04-26T18:08:47.370220+0000","last_unstale":"2024-04-26T18:11:57.254051+0000","last_undegraded":"2024-04-26T18:11:57.254051+0000","last_fullsized":"2024-04-26T18:11:57.254051+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:23:05.139824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207423+0000","last_change":"2024-04-26T18:08:44.347031+0000","last_active":"2024-04-26T18:11:57.207423+0000","last_peered":"2024-04-26T18:11:57.207423+0000","last_clean":"2024-04-26T18:11:57.207423+0000","last_became_active":"2024-04-26T18:08:44.346650+0000","last_became_peered":"2024-04-26T18:08:44.346650+0000","last_unstale":"2024-04-26T18:11:57.207423+0000","last_undegraded":"2024-04-26T18:11:57.207423+0000","last_fullsized":"2024-04-26T18:11:57.207423+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:50.946449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163867+0000","last_change":"2024-04-26T18:08:47.374230+0000","last_active":"2024-04-26T18:11:57.163867+0000","last_peered":"2024-04-26T18:11:57.163867+0000","last_clean":"2024-04-26T18:11:57.163867+0000","last_became_active":"2024-04-26T18:08:47.370780+0000","last_became_peered":"2024-04-26T18:08:47.370780+0000","last_unstale":"2024-04-26T18:11:57.163867+0000","last_undegraded":"2024-04-26T18:11:57.163867+0000","last_fullsized":"2024-04-26T18:11:57.163867+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:03:01.171195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160284+0000","last_change":"2024-04-26T18:08:44.338318+0000","last_active":"2024-04-26T18:11:57.160284+0000","last_peered":"2024-04-26T18:11:57.160284+0000","last_clean":"2024-04-26T18:11:57.160284+0000","last_became_active":"2024-04-26T18:08:44.338009+0000","last_became_peered":"2024-04-26T18:08:44.338009+0000","last_unstale":"2024-04-26T18:11:57.160284+0000","last_undegraded":"2024-04-26T18:11:57.160284+0000","last_fullsized":"2024-04-26T18:11:57.160284+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:33.558224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234730+0000","last_change":"2024-04-26T18:08:47.363924+0000","last_active":"2024-04-26T18:11:57.234730+0000","last_peered":"2024-04-26T18:11:57.234730+0000","last_clean":"2024-04-26T18:11:57.234730+0000","last_became_active":"2024-04-26T18:08:47.363616+0000","last_became_peered":"2024-04-26T18:08:47.363616+0000","last_unstale":"2024-04-26T18:11:57.234730+0000","last_undegraded":"2024-04-26T18:11:57.234730+0000","last_fullsized":"2024-04-26T18:11:57.234730+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:27:08.226989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163592+0000","last_change":"2024-04-26T18:08:44.343350+0000","last_active":"2024-04-26T18:11:57.163592+0000","last_peered":"2024-04-26T18:11:57.163592+0000","last_clean":"2024-04-26T18:11:57.163592+0000","last_became_active":"2024-04-26T18:08:44.343040+0000","last_became_peered":"2024-04-26T18:08:44.343040+0000","last_unstale":"2024-04-26T18:11:57.163592+0000","last_undegraded":"2024-04-26T18:11:57.163592+0000","last_fullsized":"2024-04-26T18:11:57.163592+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:09.427785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.251855+0000","last_change":"2024-04-26T18:08:47.360180+0000","last_active":"2024-04-26T18:11:57.251855+0000","last_peered":"2024-04-26T18:11:57.251855+0000","last_clean":"2024-04-26T18:11:57.251855+0000","last_became_active":"2024-04-26T18:08:47.359860+0000","last_became_peered":"2024-04-26T18:08:47.359860+0000","last_unstale":"2024-04-26T18:11:57.251855+0000","last_undegraded":"2024-04-26T18:11:57.251855+0000","last_fullsized":"2024-04-26T18:11:57.251855+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:36:00.532589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":213,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:01.213985+0000","last_change":"2024-04-26T18:08:44.343942+0000","last_active":"2024-04-26T18:12:01.213985+0000","last_peered":"2024-04-26T18:12:01.213985+0000","last_clean":"2024-04-26T18:12:01.213985+0000","last_became_active":"2024-04-26T18:08:44.343581+0000","last_became_peered":"2024-04-26T18:08:44.343581+0000","last_unstale":"2024-04-26T18:12:01.213985+0000","last_undegraded":"2024-04-26T18:12:01.213985+0000","last_fullsized":"2024-04-26T18:12:01.213985+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:06:32.044284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346718+0000","last_change":"2024-04-26T18:08:47.372303+0000","last_active":"2024-04-26T18:11:57.346718+0000","last_peered":"2024-04-26T18:11:57.346718+0000","last_clean":"2024-04-26T18:11:57.346718+0000","last_became_active":"2024-04-26T18:08:47.371961+0000","last_became_peered":"2024-04-26T18:08:47.371961+0000","last_unstale":"2024-04-26T18:11:57.346718+0000","last_undegraded":"2024-04-26T18:11:57.346718+0000","last_fullsized":"2024-04-26T18:11:57.346718+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:01:19.839792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347265+0000","last_change":"2024-04-26T18:08:44.345681+0000","last_active":"2024-04-26T18:11:57.347265+0000","last_peered":"2024-04-26T18:11:57.347265+0000","last_clean":"2024-04-26T18:11:57.347265+0000","last_became_active":"2024-04-26T18:08:44.345329+0000","last_became_peered":"2024-04-26T18:08:44.345329+0000","last_unstale":"2024-04-26T18:11:57.347265+0000","last_undegraded":"2024-04-26T18:11:57.347265+0000","last_fullsized":"2024-04-26T18:11:57.347265+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:10:59.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"92'9","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347192+0000","last_change":"2024-04-26T18:08:47.370773+0000","last_active":"2024-04-26T18:11:57.347192+0000","last_peered":"2024-04-26T18:11:57.347192+0000","last_clean":"2024-04-26T18:11:57.347192+0000","last_became_active":"2024-04-26T18:08:47.370490+0000","last_became_peered":"2024-04-26T18:08:47.370490+0000","last_unstale":"2024-04-26T18:11:57.347192+0000","last_undegraded":"2024-04-26T18:11:57.347192+0000","last_fullsized":"2024-04-26T18:11:57.347192+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:01.935335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.164125+0000","last_change":"2024-04-26T18:08:44.555714+0000","last_active":"2024-04-26T18:11:57.164125+0000","last_peered":"2024-04-26T18:11:57.164125+0000","last_clean":"2024-04-26T18:11:57.164125+0000","last_became_active":"2024-04-26T18:08:44.555184+0000","last_became_peered":"2024-04-26T18:08:44.555184+0000","last_unstale":"2024-04-26T18:11:57.164125+0000","last_undegraded":"2024-04-26T18:11:57.164125+0000","last_fullsized":"2024-04-26T18:11:57.164125+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:01.433764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"92'11","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346864+0000","last_change":"2024-04-26T18:09:33.525026+0000","last_active":"2024-04-26T18:11:57.346864+0000","last_peered":"2024-04-26T18:11:57.346864+0000","last_clean":"2024-04-26T18:11:57.346864+0000","last_became_active":"2024-04-26T18:09:33.524587+0000","last_became_peered":"2024-04-26T18:09:33.524587+0000","last_unstale":"2024-04-26T18:11:57.346864+0000","last_undegraded":"2024-04-26T18:11:57.346864+0000","last_fullsized":"2024-04-26T18:11:57.346864+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:39:48.534737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'39","reported_seq":439,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:02.215881+0000","last_change":"2024-04-26T18:08:44.346346+0000","last_active":"2024-04-26T18:12:02.215881+0000","last_peered":"2024-04-26T18:12:02.215881+0000","last_clean":"2024-04-26T18:12:02.215881+0000","last_became_active":"2024-04-26T18:08:44.345999+0000","last_became_peered":"2024-04-26T18:08:44.345999+0000","last_unstale":"2024-04-26T18:12:02.215881+0000","last_undegraded":"2024-04-26T18:12:02.215881+0000","last_fullsized":"2024-04-26T18:12:02.215881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:13:27.277880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67912,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":371,"num_read_kb":252,"num_write":40,"num_write_kb":84,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254263+0000","last_change":"2024-04-26T18:08:47.366303+0000","last_active":"2024-04-26T18:11:57.254263+0000","last_peered":"2024-04-26T18:11:57.254263+0000","last_clean":"2024-04-26T18:11:57.254263+0000","last_became_active":"2024-04-26T18:08:47.365861+0000","last_became_peered":"2024-04-26T18:08:47.365861+0000","last_unstale":"2024-04-26T18:11:57.254263+0000","last_undegraded":"2024-04-26T18:11:57.254263+0000","last_fullsized":"2024-04-26T18:11:57.254263+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:31.413589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"90'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346987+0000","last_change":"2024-04-26T18:09:33.022912+0000","last_active":"2024-04-26T18:11:57.346987+0000","last_peered":"2024-04-26T18:11:57.346987+0000","last_clean":"2024-04-26T18:11:57.346987+0000","last_became_active":"2024-04-26T18:09:33.015390+0000","last_became_peered":"2024-04-26T18:09:33.015390+0000","last_unstale":"2024-04-26T18:11:57.346987+0000","last_undegraded":"2024-04-26T18:11:57.346987+0000","last_fullsized":"2024-04-26T18:11:57.346987+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:53.980531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160286+0000","last_change":"2024-04-26T18:08:47.367835+0000","last_active":"2024-04-26T18:11:57.160286+0000","last_peered":"2024-04-26T18:11:57.160286+0000","last_clean":"2024-04-26T18:11:57.160286+0000","last_became_active":"2024-04-26T18:08:47.367517+0000","last_became_peered":"2024-04-26T18:08:47.367517+0000","last_unstale":"2024-04-26T18:11:57.160286+0000","last_undegraded":"2024-04-26T18:11:57.160286+0000","last_fullsized":"2024-04-26T18:11:57.160286+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:30:54.719628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163733+0000","last_change":"2024-04-26T18:08:44.344161+0000","last_active":"2024-04-26T18:11:57.163733+0000","last_peered":"2024-04-26T18:11:57.163733+0000","last_clean":"2024-04-26T18:11:57.163733+0000","last_became_active":"2024-04-26T18:08:44.343635+0000","last_became_peered":"2024-04-26T18:08:44.343635+0000","last_unstale":"2024-04-26T18:11:57.163733+0000","last_undegraded":"2024-04-26T18:11:57.163733+0000","last_fullsized":"2024-04-26T18:11:57.163733+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:52.946584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"92'15","reported_seq":44,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346903+0000","last_change":"2024-04-26T18:09:33.524351+0000","last_active":"2024-04-26T18:11:57.346903+0000","last_peered":"2024-04-26T18:11:57.346903+0000","last_clean":"2024-04-26T18:11:57.346903+0000","last_became_active":"2024-04-26T18:09:33.523953+0000","last_became_peered":"2024-04-26T18:09:33.523953+0000","last_unstale":"2024-04-26T18:11:57.346903+0000","last_undegraded":"2024-04-26T18:11:57.346903+0000","last_fullsized":"2024-04-26T18:11:57.346903+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:58:22.887498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"93'6","reported_seq":219,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:01.240344+0000","last_change":"2024-04-26T18:08:44.345278+0000","last_active":"2024-04-26T18:12:01.240344+0000","last_peered":"2024-04-26T18:12:01.240344+0000","last_clean":"2024-04-26T18:12:01.240344+0000","last_became_active":"2024-04-26T18:08:44.344531+0000","last_became_peered":"2024-04-26T18:08:44.344531+0000","last_unstale":"2024-04-26T18:12:01.240344+0000","last_undegraded":"2024-04-26T18:12:01.240344+0000","last_fullsized":"2024-04-26T18:12:01.240344+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:45:02.667622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":185,"num_read_kb":6,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346598+0000","last_change":"2024-04-26T18:08:47.369542+0000","last_active":"2024-04-26T18:11:57.346598+0000","last_peered":"2024-04-26T18:11:57.346598+0000","last_clean":"2024-04-26T18:11:57.346598+0000","last_became_active":"2024-04-26T18:08:47.369181+0000","last_became_peered":"2024-04-26T18:08:47.369181+0000","last_unstale":"2024-04-26T18:11:57.346598+0000","last_undegraded":"2024-04-26T18:11:57.346598+0000","last_fullsized":"2024-04-26T18:11:57.346598+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:56.885010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"90'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254034+0000","last_change":"2024-04-26T18:08:44.555580+0000","last_active":"2024-04-26T18:11:57.254034+0000","last_peered":"2024-04-26T18:11:57.254034+0000","last_clean":"2024-04-26T18:11:57.254034+0000","last_became_active":"2024-04-26T18:08:44.555247+0000","last_became_peered":"2024-04-26T18:08:44.555247+0000","last_unstale":"2024-04-26T18:11:57.254034+0000","last_undegraded":"2024-04-26T18:11:57.254034+0000","last_fullsized":"2024-04-26T18:11:57.254034+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:58:55.207355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163660+0000","last_change":"2024-04-26T18:08:47.372300+0000","last_active":"2024-04-26T18:11:57.163660+0000","last_peered":"2024-04-26T18:11:57.163660+0000","last_clean":"2024-04-26T18:11:57.163660+0000","last_became_active":"2024-04-26T18:08:47.371952+0000","last_became_peered":"2024-04-26T18:08:47.371952+0000","last_unstale":"2024-04-26T18:11:57.163660+0000","last_undegraded":"2024-04-26T18:11:57.163660+0000","last_fullsized":"2024-04-26T18:11:57.163660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:54:58.254691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'2","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254389+0000","last_change":"2024-04-26T18:08:44.340926+0000","last_active":"2024-04-26T18:11:57.254389+0000","last_peered":"2024-04-26T18:11:57.254389+0000","last_clean":"2024-04-26T18:11:57.254389+0000","last_became_active":"2024-04-26T18:08:44.340563+0000","last_became_peered":"2024-04-26T18:08:44.340563+0000","last_unstale":"2024-04-26T18:11:57.254389+0000","last_undegraded":"2024-04-26T18:11:57.254389+0000","last_fullsized":"2024-04-26T18:11:57.254389+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:57.803017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"92'5","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347296+0000","last_change":"2024-04-26T18:09:33.021890+0000","last_active":"2024-04-26T18:11:57.347296+0000","last_peered":"2024-04-26T18:11:57.347296+0000","last_clean":"2024-04-26T18:11:57.347296+0000","last_became_active":"2024-04-26T18:09:33.021446+0000","last_became_peered":"2024-04-26T18:09:33.021446+0000","last_unstale":"2024-04-26T18:11:57.347296+0000","last_undegraded":"2024-04-26T18:11:57.347296+0000","last_fullsized":"2024-04-26T18:11:57.347296+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:29.158581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.252002+0000","last_change":"2024-04-26T18:08:44.555076+0000","last_active":"2024-04-26T18:11:57.252002+0000","last_peered":"2024-04-26T18:11:57.252002+0000","last_clean":"2024-04-26T18:11:57.252002+0000","last_became_active":"2024-04-26T18:08:44.554756+0000","last_became_peered":"2024-04-26T18:08:44.554756+0000","last_unstale":"2024-04-26T18:11:57.252002+0000","last_undegraded":"2024-04-26T18:11:57.252002+0000","last_fullsized":"2024-04-26T18:11:57.252002+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:21.950644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207518+0000","last_change":"2024-04-26T18:08:47.366370+0000","last_active":"2024-04-26T18:11:57.207518+0000","last_peered":"2024-04-26T18:11:57.207518+0000","last_clean":"2024-04-26T18:11:57.207518+0000","last_became_active":"2024-04-26T18:08:47.365997+0000","last_became_peered":"2024-04-26T18:08:47.365997+0000","last_unstale":"2024-04-26T18:11:57.207518+0000","last_undegraded":"2024-04-26T18:11:57.207518+0000","last_fullsized":"2024-04-26T18:11:57.207518+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:05:51.643438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'1","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347119+0000","last_change":"2024-04-26T18:08:44.339299+0000","last_active":"2024-04-26T18:11:57.347119+0000","last_peered":"2024-04-26T18:11:57.347119+0000","last_clean":"2024-04-26T18:11:57.347119+0000","last_became_active":"2024-04-26T18:08:44.338932+0000","last_became_peered":"2024-04-26T18:08:44.338932+0000","last_unstale":"2024-04-26T18:11:57.347119+0000","last_undegraded":"2024-04-26T18:11:57.347119+0000","last_fullsized":"2024-04-26T18:11:57.347119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:30:18.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.207445+0000","last_change":"2024-04-26T18:08:47.369924+0000","last_active":"2024-04-26T18:11:57.207445+0000","last_peered":"2024-04-26T18:11:57.207445+0000","last_clean":"2024-04-26T18:11:57.207445+0000","last_became_active":"2024-04-26T18:08:47.369048+0000","last_became_peered":"2024-04-26T18:08:47.369048+0000","last_unstale":"2024-04-26T18:11:57.207445+0000","last_undegraded":"2024-04-26T18:11:57.207445+0000","last_fullsized":"2024-04-26T18:11:57.207445+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:56:19.110288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163797+0000","last_change":"2024-04-26T18:08:47.370836+0000","last_active":"2024-04-26T18:11:57.163797+0000","last_peered":"2024-04-26T18:11:57.163797+0000","last_clean":"2024-04-26T18:11:57.163797+0000","last_became_active":"2024-04-26T18:08:47.370160+0000","last_became_peered":"2024-04-26T18:08:47.370160+0000","last_unstale":"2024-04-26T18:11:57.163797+0000","last_undegraded":"2024-04-26T18:11:57.163797+0000","last_fullsized":"2024-04-26T18:11:57.163797+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:11:21.285466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"93'6","reported_seq":214,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:01.239832+0000","last_change":"2024-04-26T18:08:44.554960+0000","last_active":"2024-04-26T18:12:01.239832+0000","last_peered":"2024-04-26T18:12:01.239832+0000","last_clean":"2024-04-26T18:12:01.239832+0000","last_became_active":"2024-04-26T18:08:44.554607+0000","last_became_peered":"2024-04-26T18:08:44.554607+0000","last_unstale":"2024-04-26T18:12:01.239832+0000","last_undegraded":"2024-04-26T18:12:01.239832+0000","last_fullsized":"2024-04-26T18:12:01.239832+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:22:26.403285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":179,"num_read_kb":179,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235031+0000","last_change":"2024-04-26T18:08:47.372528+0000","last_active":"2024-04-26T18:11:57.235031+0000","last_peered":"2024-04-26T18:11:57.235031+0000","last_clean":"2024-04-26T18:11:57.235031+0000","last_became_active":"2024-04-26T18:08:47.372215+0000","last_became_peered":"2024-04-26T18:08:47.372215+0000","last_unstale":"2024-04-26T18:11:57.235031+0000","last_undegraded":"2024-04-26T18:11:57.235031+0000","last_fullsized":"2024-04-26T18:11:57.235031+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:14:39.366917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"90'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161184+0000","last_change":"2024-04-26T18:08:44.342667+0000","last_active":"2024-04-26T18:11:57.161184+0000","last_peered":"2024-04-26T18:11:57.161184+0000","last_clean":"2024-04-26T18:11:57.161184+0000","last_became_active":"2024-04-26T18:08:44.342170+0000","last_became_peered":"2024-04-26T18:08:44.342170+0000","last_unstale":"2024-04-26T18:11:57.161184+0000","last_undegraded":"2024-04-26T18:11:57.161184+0000","last_fullsized":"2024-04-26T18:11:57.161184+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:50.117177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"93'148","reported_seq":408,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:59.261430+0000","last_change":"2024-04-26T18:07:24.173301+0000","last_active":"2024-04-26T18:11:59.261430+0000","last_peered":"2024-04-26T18:11:59.261430+0000","last_clean":"2024-04-26T18:11:59.261430+0000","last_became_active":"2024-04-26T18:07:24.172791+0000","last_became_peered":"2024-04-26T18:07:24.172791+0000","last_unstale":"2024-04-26T18:11:59.261430+0000","last_undegraded":"2024-04-26T18:11:59.261430+0000","last_fullsized":"2024-04-26T18:11:59.261430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:03:58.528256+0000","last_clean_scrub_stamp":"2024-04-26T18:03:58.528256+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:32.468624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160143+0000","last_change":"2024-04-26T18:08:47.370720+0000","last_active":"2024-04-26T18:11:57.160143+0000","last_peered":"2024-04-26T18:11:57.160143+0000","last_clean":"2024-04-26T18:11:57.160143+0000","last_became_active":"2024-04-26T18:08:47.370414+0000","last_became_peered":"2024-04-26T18:08:47.370414+0000","last_unstale":"2024-04-26T18:11:57.160143+0000","last_undegraded":"2024-04-26T18:11:57.160143+0000","last_fullsized":"2024-04-26T18:11:57.160143+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:37.906234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346466+0000","last_change":"2024-04-26T18:08:44.347077+0000","last_active":"2024-04-26T18:11:57.346466+0000","last_peered":"2024-04-26T18:11:57.346466+0000","last_clean":"2024-04-26T18:11:57.346466+0000","last_became_active":"2024-04-26T18:08:44.346481+0000","last_became_peered":"2024-04-26T18:08:44.346481+0000","last_unstale":"2024-04-26T18:11:57.346466+0000","last_undegraded":"2024-04-26T18:11:57.346466+0000","last_fullsized":"2024-04-26T18:11:57.346466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:49:08.840814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"90'1","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346980+0000","last_change":"2024-04-26T18:09:33.025243+0000","last_active":"2024-04-26T18:11:57.346980+0000","last_peered":"2024-04-26T18:11:57.346980+0000","last_clean":"2024-04-26T18:11:57.346980+0000","last_became_active":"2024-04-26T18:09:33.015430+0000","last_became_peered":"2024-04-26T18:09:33.015430+0000","last_unstale":"2024-04-26T18:11:57.346980+0000","last_undegraded":"2024-04-26T18:11:57.346980+0000","last_fullsized":"2024-04-26T18:11:57.346980+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:54.271435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160587+0000","last_change":"2024-04-26T18:08:47.366552+0000","last_active":"2024-04-26T18:11:57.160587+0000","last_peered":"2024-04-26T18:11:57.160587+0000","last_clean":"2024-04-26T18:11:57.160587+0000","last_became_active":"2024-04-26T18:08:47.366183+0000","last_became_peered":"2024-04-26T18:08:47.366183+0000","last_unstale":"2024-04-26T18:11:57.160587+0000","last_undegraded":"2024-04-26T18:11:57.160587+0000","last_fullsized":"2024-04-26T18:11:57.160587+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:41:36.069102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'126","reported_seq":280,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:00.728133+0000","last_change":"2024-04-26T18:08:44.338078+0000","last_active":"2024-04-26T18:12:00.728133+0000","last_peered":"2024-04-26T18:12:00.728133+0000","last_clean":"2024-04-26T18:12:00.728133+0000","last_became_active":"2024-04-26T18:08:44.337715+0000","last_became_peered":"2024-04-26T18:08:44.337715+0000","last_unstale":"2024-04-26T18:12:00.728133+0000","last_undegraded":"2024-04-26T18:12:00.728133+0000","last_fullsized":"2024-04-26T18:12:00.728133+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.912993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":233,"num_write_kb":1415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235147+0000","last_change":"2024-04-26T18:08:47.371321+0000","last_active":"2024-04-26T18:11:57.235147+0000","last_peered":"2024-04-26T18:11:57.235147+0000","last_clean":"2024-04-26T18:11:57.235147+0000","last_became_active":"2024-04-26T18:08:47.371015+0000","last_became_peered":"2024-04-26T18:08:47.371015+0000","last_unstale":"2024-04-26T18:11:57.235147+0000","last_undegraded":"2024-04-26T18:11:57.235147+0000","last_fullsized":"2024-04-26T18:11:57.235147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:03:21.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"90'4","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.161387+0000","last_change":"2024-04-26T18:08:44.420101+0000","last_active":"2024-04-26T18:11:57.161387+0000","last_peered":"2024-04-26T18:11:57.161387+0000","last_clean":"2024-04-26T18:11:57.161387+0000","last_became_active":"2024-04-26T18:08:44.419787+0000","last_became_peered":"2024-04-26T18:08:44.419787+0000","last_unstale":"2024-04-26T18:11:57.161387+0000","last_undegraded":"2024-04-26T18:11:57.161387+0000","last_fullsized":"2024-04-26T18:11:57.161387+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:49:28.485425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347044+0000","last_change":"2024-04-26T18:08:47.372309+0000","last_active":"2024-04-26T18:11:57.347044+0000","last_peered":"2024-04-26T18:11:57.347044+0000","last_clean":"2024-04-26T18:11:57.347044+0000","last_became_active":"2024-04-26T18:08:47.371980+0000","last_became_peered":"2024-04-26T18:08:47.371980+0000","last_unstale":"2024-04-26T18:11:57.347044+0000","last_undegraded":"2024-04-26T18:11:57.347044+0000","last_fullsized":"2024-04-26T18:11:57.347044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:05.011077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346322+0000","last_change":"2024-04-26T18:08:44.555500+0000","last_active":"2024-04-26T18:11:57.346322+0000","last_peered":"2024-04-26T18:11:57.346322+0000","last_clean":"2024-04-26T18:11:57.346322+0000","last_became_active":"2024-04-26T18:08:44.555137+0000","last_became_peered":"2024-04-26T18:08:44.555137+0000","last_unstale":"2024-04-26T18:11:57.346322+0000","last_undegraded":"2024-04-26T18:11:57.346322+0000","last_fullsized":"2024-04-26T18:11:57.346322+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:10:05.723654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254027+0000","last_change":"2024-04-26T18:08:47.364215+0000","last_active":"2024-04-26T18:11:57.254027+0000","last_peered":"2024-04-26T18:11:57.254027+0000","last_clean":"2024-04-26T18:11:57.254027+0000","last_became_active":"2024-04-26T18:08:47.363862+0000","last_became_peered":"2024-04-26T18:08:47.363862+0000","last_unstale":"2024-04-26T18:11:57.254027+0000","last_undegraded":"2024-04-26T18:11:57.254027+0000","last_fullsized":"2024-04-26T18:11:57.254027+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:27.020100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"90'3","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234809+0000","last_change":"2024-04-26T18:08:44.343506+0000","last_active":"2024-04-26T18:11:57.234809+0000","last_peered":"2024-04-26T18:11:57.234809+0000","last_clean":"2024-04-26T18:11:57.234809+0000","last_became_active":"2024-04-26T18:08:44.343163+0000","last_became_peered":"2024-04-26T18:08:44.343163+0000","last_unstale":"2024-04-26T18:11:57.234809+0000","last_undegraded":"2024-04-26T18:11:57.234809+0000","last_fullsized":"2024-04-26T18:11:57.234809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:59:09.941391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346893+0000","last_change":"2024-04-26T18:08:47.365572+0000","last_active":"2024-04-26T18:11:57.346893+0000","last_peered":"2024-04-26T18:11:57.346893+0000","last_clean":"2024-04-26T18:11:57.346893+0000","last_became_active":"2024-04-26T18:08:47.364827+0000","last_became_peered":"2024-04-26T18:08:47.364827+0000","last_unstale":"2024-04-26T18:11:57.346893+0000","last_undegraded":"2024-04-26T18:11:57.346893+0000","last_fullsized":"2024-04-26T18:11:57.346893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:44.492380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"93'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346591+0000","last_change":"2024-04-26T18:08:44.339063+0000","last_active":"2024-04-26T18:11:57.346591+0000","last_peered":"2024-04-26T18:11:57.346591+0000","last_clean":"2024-04-26T18:11:57.346591+0000","last_became_active":"2024-04-26T18:08:44.338752+0000","last_became_peered":"2024-04-26T18:08:44.338752+0000","last_unstale":"2024-04-26T18:11:57.346591+0000","last_undegraded":"2024-04-26T18:11:57.346591+0000","last_fullsized":"2024-04-26T18:11:57.346591+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:38.201263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235138+0000","last_change":"2024-04-26T18:08:47.367859+0000","last_active":"2024-04-26T18:11:57.235138+0000","last_peered":"2024-04-26T18:11:57.235138+0000","last_clean":"2024-04-26T18:11:57.235138+0000","last_became_active":"2024-04-26T18:08:47.367548+0000","last_became_peered":"2024-04-26T18:08:47.367548+0000","last_unstale":"2024-04-26T18:11:57.235138+0000","last_undegraded":"2024-04-26T18:11:57.235138+0000","last_fullsized":"2024-04-26T18:11:57.235138+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:13:27.612151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"90'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.254111+0000","last_change":"2024-04-26T18:08:44.345392+0000","last_active":"2024-04-26T18:11:57.254111+0000","last_peered":"2024-04-26T18:11:57.254111+0000","last_clean":"2024-04-26T18:11:57.254111+0000","last_became_active":"2024-04-26T18:08:44.344578+0000","last_became_peered":"2024-04-26T18:08:44.344578+0000","last_unstale":"2024-04-26T18:11:57.254111+0000","last_undegraded":"2024-04-26T18:11:57.254111+0000","last_fullsized":"2024-04-26T18:11:57.254111+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:53:26.193264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"92'5","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347131+0000","last_change":"2024-04-26T18:08:47.380099+0000","last_active":"2024-04-26T18:11:57.347131+0000","last_peered":"2024-04-26T18:11:57.347131+0000","last_clean":"2024-04-26T18:11:57.347131+0000","last_became_active":"2024-04-26T18:08:47.379576+0000","last_became_peered":"2024-04-26T18:08:47.379576+0000","last_unstale":"2024-04-26T18:11:57.347131+0000","last_undegraded":"2024-04-26T18:11:57.347131+0000","last_fullsized":"2024-04-26T18:11:57.347131+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:18:28.397886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347023+0000","last_change":"2024-04-26T18:08:44.555299+0000","last_active":"2024-04-26T18:11:57.347023+0000","last_peered":"2024-04-26T18:11:57.347023+0000","last_clean":"2024-04-26T18:11:57.347023+0000","last_became_active":"2024-04-26T18:08:44.554959+0000","last_became_peered":"2024-04-26T18:08:44.554959+0000","last_unstale":"2024-04-26T18:11:57.347023+0000","last_undegraded":"2024-04-26T18:11:57.347023+0000","last_fullsized":"2024-04-26T18:11:57.347023+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:40:29.565073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347338+0000","last_change":"2024-04-26T18:08:47.377524+0000","last_active":"2024-04-26T18:11:57.347338+0000","last_peered":"2024-04-26T18:11:57.347338+0000","last_clean":"2024-04-26T18:11:57.347338+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:11:57.347338+0000","last_undegraded":"2024-04-26T18:11:57.347338+0000","last_fullsized":"2024-04-26T18:11:57.347338+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:10.466615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"93'73","reported_seq":144,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:12:00.723558+0000","last_change":"2024-04-26T18:08:44.343894+0000","last_active":"2024-04-26T18:12:00.723558+0000","last_peered":"2024-04-26T18:12:00.723558+0000","last_clean":"2024-04-26T18:12:00.723558+0000","last_became_active":"2024-04-26T18:08:44.343153+0000","last_became_peered":"2024-04-26T18:08:44.343153+0000","last_unstale":"2024-04-26T18:12:00.723558+0000","last_undegraded":"2024-04-26T18:12:00.723558+0000","last_fullsized":"2024-04-26T18:12:00.723558+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:19:09.914701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'7","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347161+0000","last_change":"2024-04-26T18:08:47.371962+0000","last_active":"2024-04-26T18:11:57.347161+0000","last_peered":"2024-04-26T18:11:57.347161+0000","last_clean":"2024-04-26T18:11:57.347161+0000","last_became_active":"2024-04-26T18:08:47.371517+0000","last_became_peered":"2024-04-26T18:08:47.371517+0000","last_unstale":"2024-04-26T18:11:57.347161+0000","last_undegraded":"2024-04-26T18:11:57.347161+0000","last_fullsized":"2024-04-26T18:11:57.347161+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:27:23.553409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"90'2","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160624+0000","last_change":"2024-04-26T18:08:44.337853+0000","last_active":"2024-04-26T18:11:57.160624+0000","last_peered":"2024-04-26T18:11:57.160624+0000","last_clean":"2024-04-26T18:11:57.160624+0000","last_became_active":"2024-04-26T18:08:44.337393+0000","last_became_peered":"2024-04-26T18:08:44.337393+0000","last_unstale":"2024-04-26T18:11:57.160624+0000","last_undegraded":"2024-04-26T18:11:57.160624+0000","last_fullsized":"2024-04-26T18:11:57.160624+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:43.324165+0000","last_clean_scrub_stamp":"2024-04-26T18:08:43.324165+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:43:26.514821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346960+0000","last_change":"2024-04-26T18:08:47.370999+0000","last_active":"2024-04-26T18:11:57.346960+0000","last_peered":"2024-04-26T18:11:57.346960+0000","last_clean":"2024-04-26T18:11:57.346960+0000","last_became_active":"2024-04-26T18:08:47.370373+0000","last_became_peered":"2024-04-26T18:08:47.370373+0000","last_unstale":"2024-04-26T18:11:57.346960+0000","last_undegraded":"2024-04-26T18:11:57.346960+0000","last_fullsized":"2024-04-26T18:11:57.346960+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:10:19.565357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"92'9","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346894+0000","last_change":"2024-04-26T18:08:47.365722+0000","last_active":"2024-04-26T18:11:57.346894+0000","last_peered":"2024-04-26T18:11:57.346894+0000","last_clean":"2024-04-26T18:11:57.346894+0000","last_became_active":"2024-04-26T18:08:47.364692+0000","last_became_peered":"2024-04-26T18:08:47.364692+0000","last_unstale":"2024-04-26T18:11:57.346894+0000","last_undegraded":"2024-04-26T18:11:57.346894+0000","last_fullsized":"2024-04-26T18:11:57.346894+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:52:02.158156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"92'12","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346658+0000","last_change":"2024-04-26T18:08:47.373886+0000","last_active":"2024-04-26T18:11:57.346658+0000","last_peered":"2024-04-26T18:11:57.346658+0000","last_clean":"2024-04-26T18:11:57.346658+0000","last_became_active":"2024-04-26T18:08:47.373391+0000","last_became_peered":"2024-04-26T18:08:47.373391+0000","last_unstale":"2024-04-26T18:11:57.346658+0000","last_undegraded":"2024-04-26T18:11:57.346658+0000","last_fullsized":"2024-04-26T18:11:57.346658+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:48:59.280050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346367+0000","last_change":"2024-04-26T18:08:47.366160+0000","last_active":"2024-04-26T18:11:57.346367+0000","last_peered":"2024-04-26T18:11:57.346367+0000","last_clean":"2024-04-26T18:11:57.346367+0000","last_became_active":"2024-04-26T18:08:47.365811+0000","last_became_peered":"2024-04-26T18:08:47.365811+0000","last_unstale":"2024-04-26T18:11:57.346367+0000","last_undegraded":"2024-04-26T18:11:57.346367+0000","last_fullsized":"2024-04-26T18:11:57.346367+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:09:32.124692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"92'7","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346737+0000","last_change":"2024-04-26T18:08:47.366078+0000","last_active":"2024-04-26T18:11:57.346737+0000","last_peered":"2024-04-26T18:11:57.346737+0000","last_clean":"2024-04-26T18:11:57.346737+0000","last_became_active":"2024-04-26T18:08:47.365779+0000","last_became_peered":"2024-04-26T18:08:47.365779+0000","last_unstale":"2024-04-26T18:11:57.346737+0000","last_undegraded":"2024-04-26T18:11:57.346737+0000","last_fullsized":"2024-04-26T18:11:57.346737+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:55.127606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"92'4","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160654+0000","last_change":"2024-04-26T18:08:47.372996+0000","last_active":"2024-04-26T18:11:57.160654+0000","last_peered":"2024-04-26T18:11:57.160654+0000","last_clean":"2024-04-26T18:11:57.160654+0000","last_became_active":"2024-04-26T18:08:47.372687+0000","last_became_peered":"2024-04-26T18:08:47.372687+0000","last_unstale":"2024-04-26T18:11:57.160654+0000","last_undegraded":"2024-04-26T18:11:57.160654+0000","last_fullsized":"2024-04-26T18:11:57.160654+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:13:33.780103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347315+0000","last_change":"2024-04-26T18:08:47.371076+0000","last_active":"2024-04-26T18:11:57.347315+0000","last_peered":"2024-04-26T18:11:57.347315+0000","last_clean":"2024-04-26T18:11:57.347315+0000","last_became_active":"2024-04-26T18:08:47.370688+0000","last_became_peered":"2024-04-26T18:08:47.370688+0000","last_unstale":"2024-04-26T18:11:57.347315+0000","last_undegraded":"2024-04-26T18:11:57.347315+0000","last_fullsized":"2024-04-26T18:11:57.347315+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:21.355971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"92'15","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347368+0000","last_change":"2024-04-26T18:08:47.372392+0000","last_active":"2024-04-26T18:11:57.347368+0000","last_peered":"2024-04-26T18:11:57.347368+0000","last_clean":"2024-04-26T18:11:57.347368+0000","last_became_active":"2024-04-26T18:08:47.372020+0000","last_became_peered":"2024-04-26T18:08:47.372020+0000","last_unstale":"2024-04-26T18:11:57.347368+0000","last_undegraded":"2024-04-26T18:11:57.347368+0000","last_fullsized":"2024-04-26T18:11:57.347368+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:45:15.768512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"92'2","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.235255+0000","last_change":"2024-04-26T18:08:47.369571+0000","last_active":"2024-04-26T18:11:57.235255+0000","last_peered":"2024-04-26T18:11:57.235255+0000","last_clean":"2024-04-26T18:11:57.235255+0000","last_became_active":"2024-04-26T18:08:47.369165+0000","last_became_peered":"2024-04-26T18:08:47.369165+0000","last_unstale":"2024-04-26T18:11:57.235255+0000","last_undegraded":"2024-04-26T18:11:57.235255+0000","last_fullsized":"2024-04-26T18:11:57.235255+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:55:15.869182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"92'6","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.163609+0000","last_change":"2024-04-26T18:08:47.372355+0000","last_active":"2024-04-26T18:11:57.163609+0000","last_peered":"2024-04-26T18:11:57.163609+0000","last_clean":"2024-04-26T18:11:57.163609+0000","last_became_active":"2024-04-26T18:08:47.371810+0000","last_became_peered":"2024-04-26T18:08:47.371810+0000","last_unstale":"2024-04-26T18:11:57.163609+0000","last_undegraded":"2024-04-26T18:11:57.163609+0000","last_fullsized":"2024-04-26T18:11:57.163609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:44:22.371945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.234697+0000","last_change":"2024-04-26T18:08:47.364042+0000","last_active":"2024-04-26T18:11:57.234697+0000","last_peered":"2024-04-26T18:11:57.234697+0000","last_clean":"2024-04-26T18:11:57.234697+0000","last_became_active":"2024-04-26T18:08:47.363727+0000","last_became_peered":"2024-04-26T18:08:47.363727+0000","last_unstale":"2024-04-26T18:11:57.234697+0000","last_undegraded":"2024-04-26T18:11:57.234697+0000","last_fullsized":"2024-04-26T18:11:57.234697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:46:37.595261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"92'10","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.346555+0000","last_change":"2024-04-26T18:08:47.377535+0000","last_active":"2024-04-26T18:11:57.346555+0000","last_peered":"2024-04-26T18:11:57.346555+0000","last_clean":"2024-04-26T18:11:57.346555+0000","last_became_active":"2024-04-26T18:08:47.377175+0000","last_became_peered":"2024-04-26T18:08:47.377175+0000","last_unstale":"2024-04-26T18:11:57.346555+0000","last_undegraded":"2024-04-26T18:11:57.346555+0000","last_fullsized":"2024-04-26T18:11:57.346555+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:22.895402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"92'11","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.160418+0000","last_change":"2024-04-26T18:08:47.367313+0000","last_active":"2024-04-26T18:11:57.160418+0000","last_peered":"2024-04-26T18:11:57.160418+0000","last_clean":"2024-04-26T18:11:57.160418+0000","last_became_active":"2024-04-26T18:08:47.366971+0000","last_became_peered":"2024-04-26T18:08:47.366971+0000","last_unstale":"2024-04-26T18:11:57.160418+0000","last_undegraded":"2024-04-26T18:11:57.160418+0000","last_fullsized":"2024-04-26T18:11:57.160418+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:18:14.518100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"92'8","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:11:57.347070+0000","last_change":"2024-04-26T18:08:47.373004+0000","last_active":"2024-04-26T18:11:57.347070+0000","last_peered":"2024-04-26T18:11:57.347070+0000","last_clean":"2024-04-26T18:11:57.347070+0000","last_became_active":"2024-04-26T18:08:47.372641+0000","last_became_peered":"2024-04-26T18:08:47.372641+0000","last_unstale":"2024-04-26T18:11:57.347070+0000","last_undegraded":"2024-04-26T18:11:57.347070+0000","last_fullsized":"2024-04-26T18:11:57.347070+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:08:46.332638+0000","last_clean_scrub_stamp":"2024-04-26T18:08:46.332638+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:02.018425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":795598,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1577,"num_read_kb":1053,"num_write":482,"num_write_kb":2621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2386794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":82,"seq":352187318326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1028096,"data_stored":498476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":74,"seq":317827579962,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29144,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716008,"statfs":{"total":95995035648,"available":95965192192,"internally_reserved":0,"allocated":2056192,"data_stored":1486968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743488,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1429504,"data_stored":891872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005126,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":891770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":53,"seq":227633266763,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36928,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708224,"statfs":{"total":95995035648,"available":95957221376,"internally_reserved":0,"allocated":1630208,"data_stored":1092743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462993,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":498045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1093632,"data_stored":565750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1044480,"data_stored":499316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1114112,"data_stored":568771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28800,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716352,"statfs":{"total":95995035648,"available":95965544448,"internally_reserved":0,"allocated":1765376,"data_stored":1219804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":499405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724629,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37384,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707768,"statfs":{"total":95995035648,"available":95956754432,"internally_reserved":0,"allocated":1974272,"data_stored":1419736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":398645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:12:08.086 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:12:08.330 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:08.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:08 smithi090 ceph-mon[29180]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-04-26T18:12:08.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:08 smithi090 ceph-mon[29180]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:08 smithi119 ceph-mon[32752]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-04-26T18:12:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:08 smithi119 ceph-mon[32752]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:08 smithi167 ceph-mon[32751]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 131 KiB/s wr, 79 op/s 2024-04-26T18:12:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:08 smithi167 ceph-mon[32751]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1867026890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0541027a0 msgr2=0x7fd054110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1867026890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0541027a0 0x7fd054110210 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b0a0 tx=0x7fd04c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1867026890 shutdown_connections 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1867026890 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd0541027a0 0x7fd054110210 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1867026890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd054108500 0x7fd0541088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1867026890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd054101de0 0x7fd054102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1867026890 >> 172.21.15.90:0/1867026890 conn(0x7fd054077640 msgr2=0x7fd054077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1867026890 shutdown_connections 2024-04-26T18:12:09.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.934+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1867026890 wait complete. 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 Processor -- start 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 -- start start 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd054101de0 0x7fd0541a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541027a0 0x7fd0541a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd054108500 0x7fd05419b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541135f0 con 0x7fd054101de0 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd054113470 con 0x7fd054108500 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05c7c6700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd054113770 con 0x7fd0541027a0 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd054101de0 0x7fd0541a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd054101de0 0x7fd0541a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50336/0 (socket says 172.21.15.90:50336) 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 -- 172.21.15.90:0/1212531149 learned_addr learned my addr 172.21.15.90:0/1212531149 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05ad63700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd054108500 0x7fd05419b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:09.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd059d61700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541027a0 0x7fd0541a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 -- 172.21.15.90:0/1212531149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541027a0 msgr2=0x7fd0541a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541027a0 0x7fd0541a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 -- 172.21.15.90:0/1212531149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd054108500 msgr2=0x7fd05419b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd054108500 0x7fd05419b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 -- 172.21.15.90:0/1212531149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05419c150 con 0x7fd054101de0 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd059d61700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541027a0 0x7fd0541a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05ad63700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd054108500 0x7fd05419b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.935+0000 7fd05a562700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd054101de0 0x7fd0541a1830 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c990 tx=0x7fd05000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.936+0000 7fd04b7fe700 1 -- 172.21.15.90:0/1212531149 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050007b00 con 0x7fd054101de0 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.936+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05419c440 con 0x7fd054101de0 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.936+0000 7fd04b7fe700 1 -- 172.21.15.90:0/1212531149 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd050007ca0 con 0x7fd054101de0 2024-04-26T18:12:09.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.936+0000 7fd04b7fe700 1 -- 172.21.15.90:0/1212531149 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd050018660 con 0x7fd054101de0 2024-04-26T18:12:09.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.936+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0541a8700 con 0x7fd054101de0 2024-04-26T18:12:09.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.937+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05406a110 con 0x7fd054101de0 2024-04-26T18:12:09.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.938+0000 7fd04b7fe700 1 -- 172.21.15.90:0/1212531149 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7fd050004020 con 0x7fd054101de0 2024-04-26T18:12:09.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.939+0000 7fd04b7fe700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd0400a3f50 0x7fd0400a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:09.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.939+0000 7fd059d61700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd0400a3f50 0x7fd0400a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:09.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.939+0000 7fd04b7fe700 1 -- 172.21.15.90:0/1212531149 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd0500d0ab0 con 0x7fd054101de0 2024-04-26T18:12:09.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.939+0000 7fd059d61700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd0400a3f50 0x7fd0400a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd044005fd0 tx=0x7fd044005dc0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:09.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:09.943+0000 7fd04b7fe700 1 -- 172.21.15.90:0/1212531149 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd050014030 con 0x7fd054101de0 2024-04-26T18:12:10.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.188+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd05419cf20 con 0x7fd054101de0 2024-04-26T18:12:10.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.189+0000 7fd04b7fe700 1 -- 172.21.15.90:0/1212531149 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11874 (secure 0 0 0) 0x7fd05009d080 con 0x7fd054101de0 2024-04-26T18:12:10.189 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:10.189 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:11:56.125308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd0400a3f50 msgr2=0x7fd0400a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd0400a3f50 0x7fd0400a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd044005fd0 tx=0x7fd044005dc0 comp rx=0 tx=0).stop 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd054101de0 msgr2=0x7fd0541a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd054101de0 0x7fd0541a1830 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c990 tx=0x7fd05000cd50 comp rx=0 tx=0).stop 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 shutdown_connections 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd0400a3f50 0x7fd0400a6410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd054101de0 0x7fd0541a1830 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd054108500 0x7fd05419b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:10.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 --2- 172.21.15.90:0/1212531149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd0541027a0 0x7fd0541a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:10.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 >> 172.21.15.90:0/1212531149 conn(0x7fd054077640 msgr2=0x7fd0541006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:10.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 shutdown_connections 2024-04-26T18:12:10.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:10.191+0000 7fd05c7c6700 1 -- 172.21.15.90:0/1212531149 wait complete. 2024-04-26T18:12:10.192 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T18:12:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:10 smithi090 ceph-mon[29180]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-04-26T18:12:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:10 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1212531149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:10 smithi119 ceph-mon[32752]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-04-26T18:12:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:10 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1212531149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:10.808 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:12:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:10 smithi167 ceph-mon[32751]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 108 KiB/s wr, 68 op/s 2024-04-26T18:12:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:10 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1212531149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:11.045 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2696919751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 msgr2=0x7f6ecc113410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2696919751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc113410 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009900 tx=0x7f6ec002a480 comp rx=0 tx=0).stop 2024-04-26T18:12:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2696919751 shutdown_connections 2024-04-26T18:12:12.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2696919751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc113950 0x7f6ecc115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2696919751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc113410 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2696919751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc074cd0 0x7f6ecc0750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2696919751 >> 172.21.15.90:0/2696919751 conn(0x7f6ecc0fe430 msgr2=0x7f6ecc100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2696919751 shutdown_connections 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.565+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2696919751 wait complete. 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 Processor -- start 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 -- start start 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc074cd0 0x7f6ecc19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc113950 0x7f6ecc1a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1191d0 con 0x7f6ecc0755f0 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc119050 con 0x7f6ecc113950 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ed1d9d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc119350 con 0x7f6ecc074cd0 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc19fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49512/0 (socket says 172.21.15.90:49512) 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2729616519 learned_addr learned my addr 172.21.15.90:0/2729616519 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.566+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc074cd0 0x7f6ecc19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc113950 0x7f6ecc1a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2729616519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc074cd0 msgr2=0x7f6ecc19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc074cd0 0x7f6ecc19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2729616519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc113950 msgr2=0x7f6ecc1a3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc113950 0x7f6ecc1a3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.570 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2729616519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc1a46a0 con 0x7f6ecc0755f0 2024-04-26T18:12:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc19fbf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005940 tx=0x7f6ec00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ec8ff9640 1 -- 172.21.15.90:0/2729616519 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec003f030 con 0x7f6ecc0755f0 2024-04-26T18:12:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ec8ff9640 1 -- 172.21.15.90:0/2729616519 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00044c0 con 0x7f6ecc0755f0 2024-04-26T18:12:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc1a4930 con 0x7f6ecc0755f0 2024-04-26T18:12:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ec8ff9640 1 -- 172.21.15.90:0/2729616519 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0035070 con 0x7f6ecc0755f0 2024-04-26T18:12:12.571 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.567+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1ac220 con 0x7f6ecc0755f0 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.569+0000 7f6ec8ff9640 1 -- 172.21.15.90:0/2729616519 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f6ec0042030 con 0x7f6ecc0755f0 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 Processor -- start 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 -- start start 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a1e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ec8ff9640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb40a3f60 0x7f6eb40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a28ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a2b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f3b5984460 con 0x55f3b5a290e0 2024-04-26T18:12:12.578 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f3b593e590 con 0x55f3b5a1df00 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3b5980420 con 0x55f3b5a1e820 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb40a3f60 0x7f6eb40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ec8ff9640 1 -- 172.21.15.90:0/2729616519 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f6ec00f0bf0 con 0x7f6ecc0755f0 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a1e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a28ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a2b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a28ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:34034/0 (socket says 172.21.15.90:34034) 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecaffd640 1 -- 172.21.15.90:0/3475048427 learned_addr learned my addr 172.21.15.90:0/3475048427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecbfff640 1 -- 172.21.15.90:0/3475048427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 msgr2=0x55f3b5a28ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a28ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.579 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecbfff640 1 -- 172.21.15.90:0/3475048427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 msgr2=0x55f3b5a1e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a1e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecbfff640 1 -- 172.21.15.90:0/3475048427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b594b0d0 con 0x55f3b5a290e0 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.570+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.571+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb40a3f60 0x7f6eb40a6420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009bd0 tx=0x7f6ebc005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.571+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a2b4d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005fd0 tx=0x7f6ec4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3520f315393716 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.571+0000 7f6eb1ffb640 1 -- 172.21.15.90:0/3475048427 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4016070 con 0x55f3b5a290e0 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.571+0000 7f6eb1ffb640 1 -- 172.21.15.90:0/3475048427 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec400fdf0 con 0x55f3b5a290e0 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.571+0000 7f6eb1ffb640 1 -- 172.21.15.90:0/3475048427 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4019040 con 0x55f3b5a290e0 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.571+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3475048427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 msgr2=0x55f3b5a2b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.571+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a2b4d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4005fd0 tx=0x7f6ec4009500 comp rx=0 tx=0).stop 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3475048427 shutdown_connections 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a2b4d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.580 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a28ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3475048427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a1e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3475048427 >> 172.21.15.90:0/3475048427 conn(0x55f3b5a1a410 msgr2=0x55f3b5a1a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3475048427 shutdown_connections 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3475048427 wait complete. 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- start start 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 Processor -- start 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a26670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a26bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f3b5a2bd10 con 0x55f3b5a290e0 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f3b590bb60 con 0x55f3b5a1df00 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3b5a2be90 con 0x55f3b5a1e820 2024-04-26T18:12:12.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ed4b92740 10 client.0 _ll_register_callbacks cb 0x55f3b59302b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.572+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a26bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a26bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49536/0 (socket says 172.21.15.90:49536) 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 -- 172.21.15.90:0/4151273491 learned_addr learned my addr 172.21.15.90:0/4151273491 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a26670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 -- 172.21.15.90:0/4151273491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 msgr2=0x55f3b5a26670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a26670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 -- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 msgr2=0x55f3b5a26130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.582 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a26130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 -- 172.21.15.90:0/4151273491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b5696210 con 0x55f3b5a290e0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a26bb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40041a0 tx=0x7f6ec4003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a83e7d5c9c4b82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6eaaffd640 1 -- 172.21.15.90:0/4151273491 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4016070 con 0x55f3b5a290e0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6eaaffd640 1 -- 172.21.15.90:0/4151273491 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f6ec4002030 con 0x55f3b5a290e0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec400fdf0 con 0x55f3b5a290e0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6eaaffd640 1 -- 172.21.15.90:0/4151273491 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ec4003970 con 0x55f3b5a290e0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ed4b92740 10 client.15249 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3b5762620 con 0x55f3b5a290e0 2024-04-26T18:12:12.583 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.573+0000 7f6eaaffd640 1 -- 172.21.15.90:0/4151273491 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec400fdf0 con 0x55f3b5a290e0 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6eaaffd640 1 -- 172.21.15.90:0/4151273491 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ec4031020 con 0x55f3b5a290e0 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ed4b92740 10 client.15249 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ed4b92740 10 client.15249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ed4b92740 20 client.15249 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a391d0 0x55f3b5a3b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ed4b92740 4 client.15249 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3b5992070 con 0x55f3b5a391d0 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a391d0 0x55f3b5a3b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6eaaffd640 10 client.15249 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.574+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a391d0 0x55f3b5a3b5c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a99d163c6a4f29 server_cookie=265934a7bbbb998d in_seq=0 out_seq=0 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6eaaffd640 1 -- 172.21.15.90:0/4151273491 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3b5992070 con 0x55f3b5a391d0 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6eaaffd640 10 client.15249 handle_command_reply: tid=0 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 client.15249 shutdown 2024-04-26T18:12:12.584 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 msgr2=0x55f3b5a26bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a26bb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6ec40041a0 tx=0x7f6ec4003ed0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 shutdown_connections 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a391d0 0x55f3b5a3b5c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a26bb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1e820 0x55f3b5a26670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/4151273491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1df00 0x55f3b5a26130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6eaaffd640 0 client.15249 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 >> 172.21.15.90:0/4151273491 conn(0x55f3b5a1a410 msgr2=0x55f3b5a20140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 shutdown_connections 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 1 -- 172.21.15.90:0/4151273491 wait complete. 2024-04-26T18:12:12.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.575+0000 7f6ed4b92740 20 client.15249 trim_cache size 0 max 16384 2024-04-26T18:12:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 Processor -- start 2024-04-26T18:12:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 -- start start 2024-04-26T18:12:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a34bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a1e820 0x55f3b5a208c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b7210 con 0x55f3b5a1e820 2024-04-26T18:12:12.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b4ae0 con 0x55f3b5a290e0 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b2930 con 0x55f3b5a1df00 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.590+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a1e820 0x55f3b5a208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a1e820 0x55f3b5a208c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49542/0 (socket says 172.21.15.90:49542) 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2904848598 learned_addr learned my addr 172.21.15.90:0/2904848598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a34bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2904848598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 msgr2=0x55f3b5a34bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a34bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2904848598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a290e0 msgr2=0x55f3b5a3e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 -- 172.21.15.90:0/2904848598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3b5a1b640 con 0x55f3b5a1e820 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a1e820 0x55f3b5a208c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0006010 tx=0x7f6ec003b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7b70997a51529 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6eb17fa640 1 -- 172.21.15.90:0/2904848598 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0049040 con 0x55f3b5a1e820 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6eb17fa640 1 -- 172.21.15.90:0/2904848598 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00c2340 con 0x55f3b5a1e820 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.591+0000 7f6eb17fa640 1 -- 172.21.15.90:0/2904848598 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00c1fc0 con 0x55f3b5a1e820 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 -- 172.21.15.90:0/2904848598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a1e820 msgr2=0x55f3b5a208c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a1e820 0x55f3b5a208c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0006010 tx=0x7f6ec003b460 comp rx=0 tx=0).stop 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 -- 172.21.15.90:0/2904848598 shutdown_connections 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a1e820 0x55f3b5a208c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/2904848598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a34bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 -- 172.21.15.90:0/2904848598 >> 172.21.15.90:0/2904848598 conn(0x55f3b5a1a410 msgr2=0x55f3b5a25160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 -- 172.21.15.90:0/2904848598 shutdown_connections 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 -- 172.21.15.90:0/2904848598 wait complete. 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 -- start start 2024-04-26T18:12:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.592+0000 7f6ed4b92740 1 Processor -- start 2024-04-26T18:12:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a3c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1e820 0x55f3b5a3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b2fa0 con 0x55f3b5a290e0 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b4410 con 0x55f3b5a1e820 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1b3670 con 0x55f3b5a1df00 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 10 client.0 _ll_register_callbacks cb 0x7f6ec40104b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49544/0 (socket says 172.21.15.90:49544) 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 -- 172.21.15.90:0/3286387211 learned_addr learned my addr 172.21.15.90:0/3286387211 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1e820 0x55f3b5a3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 -- 172.21.15.90:0/3286387211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 msgr2=0x55f3b5a3c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a3c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a3c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 -- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1e820 msgr2=0x55f3b5a3c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1e820 0x55f3b5a3c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 -- 172.21.15.90:0/3286387211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3b5a9e4e0 con 0x55f3b5a290e0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a3c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecaffd640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1e820 0x55f3b5a3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ecbfff640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3cbd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009c30 tx=0x7f6ec4004080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef6937c80c0af15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6eaaffd640 1 -- 172.21.15.90:0/3286387211 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec4038060 con 0x55f3b5a290e0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6eaaffd640 1 -- 172.21.15.90:0/3286387211 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f6ec4005070 con 0x55f3b5a290e0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.593+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec00c2340 con 0x55f3b5a290e0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6eaaffd640 1 -- 172.21.15.90:0/3286387211 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6ec4016030 con 0x55f3b5a290e0 2024-04-26T18:12:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3b5a9f7d0 con 0x55f3b5a290e0 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6eaaffd640 1 -- 172.21.15.90:0/3286387211 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec00c2340 con 0x55f3b5a290e0 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6eaaffd640 1 -- 172.21.15.90:0/3286387211 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6ec403b030 con 0x55f3b5a290e0 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 10 client.15255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 20 client.15255 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a9fec0 0x55f3b5aa22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 4 client.15255 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3b598bee0 con 0x55f3b5a9fec0 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.594+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a9fec0 0x55f3b5aa22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.595+0000 7f6eaaffd640 10 client.15255 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.595+0000 7f6ecb7fe640 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a9fec0 0x55f3b5aa22e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e970b0e3251b0c34 server_cookie=a26c47a05bf6480b in_seq=0 out_seq=0 2024-04-26T18:12:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.607+0000 7f6eaaffd640 1 -- 172.21.15.90:0/3286387211 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3b598bee0 con 0x55f3b5a9fec0 2024-04-26T18:12:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.607+0000 7f6eaaffd640 10 client.15255 handle_command_reply: tid=0 2024-04-26T18:12:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.607+0000 7f6ed4b92740 1 client.15255 shutdown 2024-04-26T18:12:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 msgr2=0x55f3b5a3cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3cbd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4009c30 tx=0x7f6ec4004080 comp rx=0 tx=0).stop 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 shutdown_connections 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3b5a9fec0 0x55f3b5aa22e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3b5a290e0 0x55f3b5a3cbd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3b5a1e820 0x55f3b5a3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6eaaffd640 0 client.15255 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 --2- 172.21.15.90:0/3286387211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3b5a1df00 0x55f3b5a3c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 >> 172.21.15.90:0/3286387211 conn(0x55f3b5a1a410 msgr2=0x55f3b5a18d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 shutdown_connections 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 1 -- 172.21.15.90:0/3286387211 wait complete. 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed4b92740 20 client.15255 trim_cache size 0 max 16384 2024-04-26T18:12:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:12:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:12:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:12:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb40a3f60 msgr2=0x7f6eb40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb40a3f60 0x7f6eb40a6420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009bd0 tx=0x7f6ebc005f90 comp rx=0 tx=0).stop 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 msgr2=0x7f6ecc19fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc19fbf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0005940 tx=0x7f6ec00040c0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 shutdown_connections 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb40a3f60 0x7f6eb40a6420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc113950 0x7f6ecc1a3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc0755f0 0x7f6ecc19fbf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 --2- 172.21.15.90:0/2729616519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc074cd0 0x7f6ecc19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.608+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 >> 172.21.15.90:0/2729616519 conn(0x7f6ecc0fe430 msgr2=0x7f6ecc0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.609+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 shutdown_connections 2024-04-26T18:12:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:12.609+0000 7f6ed1d9d640 1 -- 172.21.15.90:0/2729616519 wait complete. 2024-04-26T18:12:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 -- 172.21.15.90:0/128834237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c1094a0 msgr2=0x7f219c109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 --2- 172.21.15.90:0/128834237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c1094a0 0x7f219c109880 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f219400b0a0 tx=0x7f219400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 -- 172.21.15.90:0/128834237 shutdown_connections 2024-04-26T18:12:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 --2- 172.21.15.90:0/128834237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c1094a0 0x7f219c109880 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 --2- 172.21.15.90:0/128834237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f219c06bad0 0x7f219c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 --2- 172.21.15.90:0/128834237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c06b130 0x7f219c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 -- 172.21.15.90:0/128834237 >> 172.21.15.90:0/128834237 conn(0x7f219c0792d0 msgr2=0x7f219c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 -- 172.21.15.90:0/128834237 shutdown_connections 2024-04-26T18:12:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.669+0000 7f21a1564700 1 -- 172.21.15.90:0/128834237 wait complete. 2024-04-26T18:12:12.670 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 Processor -- start 2024-04-26T18:12:12.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 -- start start 2024-04-26T18:12:12.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:12 smithi090 ceph-mon[29180]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 98 KiB/s wr, 62 op/s 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c06b130 0x7f219c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c06bad0 0x7f219c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f219c1094a0 0x7f219c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f219c1135b0 con 0x7f219c06b130 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f219c113430 con 0x7f219c1094a0 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f21a1564700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f219c113730 con 0x7f219c06bad0 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c06b130 0x7f219c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c06b130 0x7f219c19f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49562/0 (socket says 172.21.15.90:49562) 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219affd700 1 -- 172.21.15.90:0/1244814154 learned_addr learned my addr 172.21.15.90:0/1244814154 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219a7fc700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c06bad0 0x7f219c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219b7fe700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f219c1094a0 0x7f219c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219affd700 1 -- 172.21.15.90:0/1244814154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c06bad0 msgr2=0x7f219c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219affd700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c06bad0 0x7f219c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.670+0000 7f219affd700 1 -- 172.21.15.90:0/1244814154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f219c1094a0 msgr2=0x7f219c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f219affd700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f219c1094a0 0x7f219c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f219affd700 1 -- 172.21.15.90:0/1244814154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219c19a0b0 con 0x7f219c06b130 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f219affd700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c06b130 0x7f219c19f790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2194006070 tx=0x7f2194004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f2193fff700 1 -- 172.21.15.90:0/1244814154 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194021070 con 0x7f219c06b130 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219c19a340 con 0x7f219c06b130 2024-04-26T18:12:12.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f2193fff700 1 -- 172.21.15.90:0/1244814154 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2194003c00 con 0x7f219c06b130 2024-04-26T18:12:12.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f2193fff700 1 -- 172.21.15.90:0/1244814154 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21940047b0 con 0x7f219c06b130 2024-04-26T18:12:12.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.671+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219c1a6660 con 0x7f219c06b130 2024-04-26T18:12:12.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.672+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219c06a110 con 0x7f219c06b130 2024-04-26T18:12:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.673+0000 7f2193fff700 1 -- 172.21.15.90:0/1244814154 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f2194003db0 con 0x7f219c06b130 2024-04-26T18:12:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.674+0000 7f2193fff700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f21880a3ed0 0x7f21880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.674+0000 7f219a7fc700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f21880a3ed0 0x7f21880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.674+0000 7f2193fff700 1 -- 172.21.15.90:0/1244814154 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f219400f020 con 0x7f219c06b130 2024-04-26T18:12:12.676 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.674+0000 7f219a7fc700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f21880a3ed0 0x7f21880a6390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f219c0799b0 tx=0x7f218400a300 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:12.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.678+0000 7f2193fff700 1 -- 172.21.15.90:0/1244814154 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f219402a020 con 0x7f219c06b130 2024-04-26T18:12:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:12 smithi119 ceph-mon[32752]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 98 KiB/s wr, 62 op/s 2024-04-26T18:12:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:12 smithi167 ceph-mon[32751]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 98 KiB/s wr, 62 op/s 2024-04-26T18:12:12.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.928+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f219c02e490 con 0x7f219c06b130 2024-04-26T18:12:12.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.929+0000 7f2193fff700 1 -- 172.21.15.90:0/1244814154 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11874 (secure 0 0 0) 0x7f219409e8d0 con 0x7f219c06b130 2024-04-26T18:12:12.929 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:12.929 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:11:56.125308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:12:12.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f21880a3ed0 msgr2=0x7f21880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f21880a3ed0 0x7f21880a6390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f219c0799b0 tx=0x7f218400a300 comp rx=0 tx=0).stop 2024-04-26T18:12:12.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c06b130 msgr2=0x7f219c19f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:12.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c06b130 0x7f219c19f790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2194006070 tx=0x7f2194004e40 comp rx=0 tx=0).stop 2024-04-26T18:12:12.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 shutdown_connections 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f21880a3ed0 0x7f21880a6390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f219c06b130 0x7f219c19f790 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f219c1094a0 0x7f219c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 --2- 172.21.15.90:0/1244814154 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f219c06bad0 0x7f219c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 >> 172.21.15.90:0/1244814154 conn(0x7f219c0792d0 msgr2=0x7f219c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 shutdown_connections 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:12.931+0000 7f21a1564700 1 -- 172.21.15.90:0/1244814154 wait complete. 2024-04-26T18:12:12.932 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T18:12:13.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:13 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1244814154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:13.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:13 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1244814154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:13.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:13 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1244814154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:13.806 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:12:14.046 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:14.717 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:14 smithi119 ceph-mon[32752]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 98 KiB/s wr, 62 op/s 2024-04-26T18:12:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:14 smithi167 ceph-mon[32751]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 98 KiB/s wr, 62 op/s 2024-04-26T18:12:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:14 smithi090 ceph-mon[29180]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 98 KiB/s wr, 62 op/s 2024-04-26T18:12:15.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 -- 172.21.15.90:0/3992087222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c102480 msgr2=0x7ff10c10e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:15.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 --2- 172.21.15.90:0/3992087222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c102480 0x7ff10c10e990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c210 tx=0x7ff10400c520 comp rx=0 tx=0).stop 2024-04-26T18:12:15.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 -- 172.21.15.90:0/3992087222 shutdown_connections 2024-04-26T18:12:15.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 --2- 172.21.15.90:0/3992087222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c102480 0x7ff10c10e990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 --2- 172.21.15.90:0/3992087222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c107ac0 0x7ff10c107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 --2- 172.21.15.90:0/3992087222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c101ac0 0x7ff10c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 -- 172.21.15.90:0/3992087222 >> 172.21.15.90:0/3992087222 conn(0x7ff10c078920 msgr2=0x7ff10c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 -- 172.21.15.90:0/3992087222 shutdown_connections 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.658+0000 7ff110b4d700 1 -- 172.21.15.90:0/3992087222 wait complete. 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 Processor -- start 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 -- start start 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c101ac0 0x7ff10c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c102480 0x7ff10c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c107ac0 0x7ff10c199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c111470 con 0x7ff10c107ac0 2024-04-26T18:12:15.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1112f0 con 0x7ff10c101ac0 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff110b4d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff10c1115f0 con 0x7ff10c102480 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c107ac0 0x7ff10c199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c101ac0 0x7ff10c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c107ac0 0x7ff10c199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49576/0 (socket says 172.21.15.90:49576) 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 -- 172.21.15.90:0/592502137 learned_addr learned my addr 172.21.15.90:0/592502137 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff109d9b700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c102480 0x7ff10c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 -- 172.21.15.90:0/592502137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c102480 msgr2=0x7ff10c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c102480 0x7ff10c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 -- 172.21.15.90:0/592502137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c101ac0 msgr2=0x7ff10c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:15.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c101ac0 0x7ff10c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 -- 172.21.15.90:0/592502137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10c19a0c0 con 0x7ff10c107ac0 2024-04-26T18:12:15.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.659+0000 7ff10ad9d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c107ac0 0x7ff10c199960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff1040062a0 tx=0x7ff1040082f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:15.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.660+0000 7ff0fb7fe700 1 -- 172.21.15.90:0/592502137 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104022020 con 0x7ff10c107ac0 2024-04-26T18:12:15.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.660+0000 7ff0fb7fe700 1 -- 172.21.15.90:0/592502137 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff104004e40 con 0x7ff10c107ac0 2024-04-26T18:12:15.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.660+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10c19a350 con 0x7ff10c107ac0 2024-04-26T18:12:15.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.660+0000 7ff0fb7fe700 1 -- 172.21.15.90:0/592502137 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff104007cd0 con 0x7ff10c107ac0 2024-04-26T18:12:15.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.660+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff10c1a6580 con 0x7ff10c107ac0 2024-04-26T18:12:15.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.661+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff10c06a110 con 0x7ff10c107ac0 2024-04-26T18:12:15.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.662+0000 7ff0fb7fe700 1 -- 172.21.15.90:0/592502137 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7ff104025020 con 0x7ff10c107ac0 2024-04-26T18:12:15.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.663+0000 7ff0fb7fe700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0f40a3ed0 0x7ff0f40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:15.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.663+0000 7ff10a59c700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0f40a3ed0 0x7ff0f40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:15.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.663+0000 7ff0fb7fe700 1 -- 172.21.15.90:0/592502137 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff104010020 con 0x7ff10c107ac0 2024-04-26T18:12:15.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.663+0000 7ff10a59c700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0f40a3ed0 0x7ff0f40a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff10c1062a0 tx=0x7ff0fc009260 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:15.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.667+0000 7ff0fb7fe700 1 -- 172.21.15.90:0/592502137 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff10409e1c0 con 0x7ff10c107ac0 2024-04-26T18:12:15.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.866+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff10c02e490 con 0x7ff10c107ac0 2024-04-26T18:12:15.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.867+0000 7ff0fb7fe700 1 -- 172.21.15.90:0/592502137 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22934 (secure 0 0 0) 0x7ff10409dac0 con 0x7ff10c107ac0 2024-04-26T18:12:15.868 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:15.868 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":93,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:11:57.154966+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6801/3660618461":"2024-04-27T18:11:57.154935+0000","172.21.15.90:0/2718220414":"2024-04-27T18:11:57.154935+0000","172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/3660618461":"2024-04-27T18:11:57.154935+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/1315801302":"2024-04-27T18:11:57.154935+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/484221164":"2024-04-27T18:11:57.154935+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1501521801":"2024-04-27T18:11:57.154935+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0f40a3ed0 msgr2=0x7ff0f40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0f40a3ed0 0x7ff0f40a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff10c1062a0 tx=0x7ff0fc009260 comp rx=0 tx=0).stop 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c107ac0 msgr2=0x7ff10c199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c107ac0 0x7ff10c199960 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff1040062a0 tx=0x7ff1040082f0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 shutdown_connections 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0f40a3ed0 0x7ff0f40a6390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff10c107ac0 0x7ff10c199960 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10c101ac0 0x7ff10c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 --2- 172.21.15.90:0/592502137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff10c102480 0x7ff10c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:15.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 >> 172.21.15.90:0/592502137 conn(0x7ff10c078920 msgr2=0x7ff10c104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:15.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 shutdown_connections 2024-04-26T18:12:15.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:15.870+0000 7ff110b4d700 1 -- 172.21.15.90:0/592502137 wait complete. 2024-04-26T18:12:16.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:16 smithi090 ceph-mon[29180]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 101 KiB/s wr, 68 op/s 2024-04-26T18:12:16.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:16 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/592502137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:12:16.490 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-26T18:12:16.728 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:16 smithi119 ceph-mon[32752]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 101 KiB/s wr, 68 op/s 2024-04-26T18:12:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:16 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/592502137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:12:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:16 smithi167 ceph-mon[32751]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 101 KiB/s wr, 68 op/s 2024-04-26T18:12:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:16 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/592502137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:12:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:18 smithi119 ceph-mon[32752]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-26T18:12:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:18 smithi167 ceph-mon[32751]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-26T18:12:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:18 smithi090 ceph-mon[29180]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-26T18:12:19.001 INFO:teuthology.orchestra.run.smithi090.stdout:{"run_dir":"/var/run/ceph"} 2024-04-26T18:12:19.001 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-26T18:12:19.001 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9496 2024-04-26T18:12:19.002 DEBUG:tasks.quiescer:Yielding 2024-04-26T18:12:19.002 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T18:12:19.017 INFO:tasks.workunit:Pulling workunits from ref bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:12:19.017 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T18:12:19.017 INFO:tasks.workunit:timeout=3h 2024-04-26T18:12:19.017 INFO:tasks.workunit:cleanup=True 2024-04-26T18:12:19.018 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T18:12:19.018 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:12:19.034 INFO:teuthology.orchestra.run.smithi090.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 18:09:18.659195626 +0000 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 18:09:18.659195626 +0000 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 18:11:55.402930249 +0000 2024-04-26T18:12:19.035 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 18:09:18.659195626 +0000 2024-04-26T18:12:19.035 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T18:12:19.035 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T18:12:19.103 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:12:19.158 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T18:12:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:20.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:20 smithi090 ceph-mon[29180]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-26T18:12:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:20 smithi119 ceph-mon[32752]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-26T18:12:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:20 smithi167 ceph-mon[32751]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 -- 172.21.15.90:0/387826479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc108390 msgr2=0x7f6ecc108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 --2- 172.21.15.90:0/387826479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc108390 0x7f6ecc108770 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009990 tx=0x7f6ec8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 -- 172.21.15.90:0/387826479 shutdown_connections 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 --2- 172.21.15.90:0/387826479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc108390 0x7f6ecc108770 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 --2- 172.21.15.90:0/387826479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc102ce0 0x7f6ecc10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 --2- 172.21.15.90:0/387826479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc102320 0x7f6ecc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 -- 172.21.15.90:0/387826479 >> 172.21.15.90:0/387826479 conn(0x7f6ecc0792d0 msgr2=0x7f6ecc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 -- 172.21.15.90:0/387826479 shutdown_connections 2024-04-26T18:12:20.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 -- 172.21.15.90:0/387826479 wait complete. 2024-04-26T18:12:20.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 Processor -- start 2024-04-26T18:12:20.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.905+0000 7f6ed3550700 1 -- start start 2024-04-26T18:12:20.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed3550700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc102320 0x7f6ecc19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed3550700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc102ce0 0x7f6ecc19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed3550700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc108390 0x7f6ecc199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed3550700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc113620 con 0x7f6ecc102ce0 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed3550700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1134a0 con 0x7f6ecc108390 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed3550700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ecc1137a0 con 0x7f6ecc102320 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc102ce0 0x7f6ecc19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc102ce0 0x7f6ecc19fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49602/0 (socket says 172.21.15.90:49602) 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 -- 172.21.15.90:0/138527874 learned_addr learned my addr 172.21.15.90:0/138527874 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed12ec700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc102320 0x7f6ecc19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed1aed700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc108390 0x7f6ecc199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 -- 172.21.15.90:0/138527874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc102320 msgr2=0x7f6ecc19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc102320 0x7f6ecc19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 -- 172.21.15.90:0/138527874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc108390 msgr2=0x7f6ecc199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:20.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc108390 0x7f6ecc199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 -- 172.21.15.90:0/138527874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ecc19a000 con 0x7f6ecc102ce0 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed1aed700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc108390 0x7f6ecc199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ed0aeb700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc102ce0 0x7f6ecc19fb90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c9b0 tx=0x7f6ebc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ec27fc700 1 -- 172.21.15.90:0/138527874 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc007b00 con 0x7f6ecc102ce0 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.906+0000 7f6ec27fc700 1 -- 172.21.15.90:0/138527874 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ebc007ca0 con 0x7f6ecc102ce0 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.907+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ecc19a2f0 con 0x7f6ecc102ce0 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.907+0000 7f6ec27fc700 1 -- 172.21.15.90:0/138527874 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc018660 con 0x7f6ecc102ce0 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.907+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ecc1a64b0 con 0x7f6ecc102ce0 2024-04-26T18:12:20.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.907+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ecc103a40 con 0x7f6ecc102ce0 2024-04-26T18:12:20.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.908+0000 7f6ec27fc700 1 -- 172.21.15.90:0/138527874 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f6ebc00f450 con 0x7f6ecc102ce0 2024-04-26T18:12:20.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.909+0000 7f6ec27fc700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb80a3ed0 0x7f6eb80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:20.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.909+0000 7f6ed12ec700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb80a3ed0 0x7f6eb80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:20.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.909+0000 7f6ec27fc700 1 -- 172.21.15.90:0/138527874 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f6ebc0d0a10 con 0x7f6ecc102ce0 2024-04-26T18:12:20.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.910+0000 7f6ed12ec700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb80a3ed0 0x7f6eb80a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80098e0 tx=0x7f6ec8005960 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:20.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:20.914+0000 7f6ec27fc700 1 -- 172.21.15.90:0/138527874 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ebc014030 con 0x7f6ecc102ce0 2024-04-26T18:12:21.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.163+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ecc02e490 con 0x7f6ecc102ce0 2024-04-26T18:12:21.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.164+0000 7f6ec27fc700 1 -- 172.21.15.90:0/138527874 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11874 (secure 0 0 0) 0x7f6ebc09ced0 con 0x7f6ecc102ce0 2024-04-26T18:12:21.164 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:21.164 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:11:56.125308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:12:21.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb80a3ed0 msgr2=0x7f6eb80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:21.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb80a3ed0 0x7f6eb80a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6ec80098e0 tx=0x7f6ec8005960 comp rx=0 tx=0).stop 2024-04-26T18:12:21.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc102ce0 msgr2=0x7f6ecc19fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:21.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc102ce0 0x7f6ecc19fb90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00c9b0 tx=0x7f6ebc00ccc0 comp rx=0 tx=0).stop 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 shutdown_connections 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6eb80a3ed0 0x7f6eb80a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ecc108390 0x7f6ecc199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ecc102320 0x7f6ecc19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 --2- 172.21.15.90:0/138527874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ecc102ce0 0x7f6ecc19fb90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 >> 172.21.15.90:0/138527874 conn(0x7f6ecc0792d0 msgr2=0x7f6ecc101010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 shutdown_connections 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:21.166+0000 7f6ed3550700 1 -- 172.21.15.90:0/138527874 wait complete. 2024-04-26T18:12:21.167 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T18:12:21.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:21 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/138527874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:21.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:08:49.029287+0000', 'modified': '2024-04-26T18:11:56.125308+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24449, 'mds_1': 24448, 'mds_2': 24436}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24412': {'gid': 24412, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6833/766230459', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6832', 'nonce': 766230459}, {'type': 'v1', 'addr': '172.21.15.167:6833', 'nonce': 766230459}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.119:6837/3534880230', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6836', 'nonce': 3534880230}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 3534880230}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6835/570376817', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6834', 'nonce': 570376817}, {'type': 'v1', 'addr': '172.21.15.167:6835', 'nonce': 570376817}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.167:6837/200145753', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 200145753}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 200145753}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.167:6839/4229007587', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 4229007587}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 4229007587}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.119:6839/1002360318', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6838', 'nonce': 1002360318}, {'type': 'v1', 'addr': '172.21.15.119:6839', 'nonce': 1002360318}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24449, 'qdb_cluster': [24436, 24448, 24449]} 2024-04-26T18:12:21.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-26T18:12:21.804 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T18:12:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:21 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/138527874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:21 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/138527874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:22.082 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:22.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:22 smithi090 ceph-mon[29180]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:22 smithi119 ceph-mon[32752]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:22 smithi167 ceph-mon[32751]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 -- 172.21.15.90:0/3521164602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a8107b30 msgr2=0x7f09a8101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3521164602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a8107b30 0x7f09a8101de0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f099c009990 tx=0x7f099c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 -- 172.21.15.90:0/3521164602 shutdown_connections 2024-04-26T18:12:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3521164602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09a81023b0 0x7f09a8102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3521164602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8107180 0x7f09a8107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3521164602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a8107b30 0x7f09a8101de0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 -- 172.21.15.90:0/3521164602 >> 172.21.15.90:0/3521164602 conn(0x7f09a8077640 msgr2=0x7f09a8077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 -- 172.21.15.90:0/3521164602 shutdown_connections 2024-04-26T18:12:23.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.724+0000 7f09ae63e700 1 -- 172.21.15.90:0/3521164602 wait complete. 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 Processor -- start 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 -- start start 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a81023b0 0x7f09a81a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09a8107180 0x7f09a81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8107b30 0x7f09a81a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8119d00 con 0x7f09a81023b0 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8119b80 con 0x7f09a8107b30 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09ae63e700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8119e80 con 0x7f09a8107180 2024-04-26T18:12:23.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a81023b0 0x7f09a81a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a81023b0 0x7f09a81a0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35596/0 (socket says 172.21.15.90:35596) 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 -- 172.21.15.90:0/3282841244 learned_addr learned my addr 172.21.15.90:0/3282841244 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a77fe700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09a8107180 0x7f09a81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09acbdb700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8107b30 0x7f09a81a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 -- 172.21.15.90:0/3282841244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09a8107180 msgr2=0x7f09a81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09a8107180 0x7f09a81a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 -- 172.21.15.90:0/3282841244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8107b30 msgr2=0x7f09a81a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8107b30 0x7f09a81a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.725+0000 7f09a7fff700 1 -- 172.21.15.90:0/3282841244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a81a5180 con 0x7f09a81023b0 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.726+0000 7f09a7fff700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a81023b0 0x7f09a81a0170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f099800d980 tx=0x7f099800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:23.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.726+0000 7f09a57fa700 1 -- 172.21.15.90:0/3282841244 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998017030 con 0x7f09a81023b0 2024-04-26T18:12:23.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.726+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a81a5470 con 0x7f09a81023b0 2024-04-26T18:12:23.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.726+0000 7f09a57fa700 1 -- 172.21.15.90:0/3282841244 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09980044d0 con 0x7f09a81023b0 2024-04-26T18:12:23.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.726+0000 7f09a57fa700 1 -- 172.21.15.90:0/3282841244 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998003970 con 0x7f09a81023b0 2024-04-26T18:12:23.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.726+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81accf0 con 0x7f09a81023b0 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.728+0000 7f09a57fa700 1 -- 172.21.15.90:0/3282841244 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f099801a070 con 0x7f09a81023b0 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.729+0000 7f09a57fa700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f09900a3ed0 0x7f09900a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.729+0000 7f09a57fa700 1 -- 172.21.15.90:0/3282841244 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f09980d0240 con 0x7f09a81023b0 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.729+0000 7f09a77fe700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f09900a3ed0 0x7f09900a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.729+0000 7f09a77fe700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f09900a3ed0 0x7f09900a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f099c0098e0 tx=0x7f099c005960 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 Processor -- start 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 -- start start 2024-04-26T18:12:23.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c996eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c990cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c991710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56347c8e36f0 con 0x56347c9912d0 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56347c882590 con 0x56347c996ad0 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56347c8df6b0 con 0x56347c9973f0 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c991710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c991710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35598/0 (socket says 172.21.15.90:35598) 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 -- 172.21.15.90:0/708885823 learned_addr learned my addr 172.21.15.90:0/708885823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09a7fff700 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c996eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09a77fe700 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c990cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 -- 172.21.15.90:0/708885823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 msgr2=0x56347c990cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c990cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 -- 172.21.15.90:0/708885823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 msgr2=0x56347c996eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c996eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 -- 172.21.15.90:0/708885823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347c830880 con 0x56347c9912d0 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.730+0000 7f09acbdb700 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c991710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f09a000f5a0 tx=0x7f09a000f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd1dcb0829b908a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f098e7fc700 1 -- 172.21.15.90:0/708885823 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0018030 con 0x56347c9912d0 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f098e7fc700 1 -- 172.21.15.90:0/708885823 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a0004bb0 con 0x56347c9912d0 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f098e7fc700 1 -- 172.21.15.90:0/708885823 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0005020 con 0x56347c9912d0 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/708885823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 msgr2=0x56347c991710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c991710 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f09a000f5a0 tx=0x7f09a000f8b0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/708885823 shutdown_connections 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c991710 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c996eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/708885823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c990cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/708885823 >> 172.21.15.90:0/708885823 conn(0x56347c8bdcf0 msgr2=0x56347c98c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/708885823 shutdown_connections 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.731+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/708885823 wait complete. 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 -- start start 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 Processor -- start 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c99d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c99a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c99a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56347c99df10 con 0x56347c9912d0 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56347c849900 con 0x56347c996ad0 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56347c99e090 con 0x56347c9973f0 2024-04-26T18:12:23.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09b8ed0b80 10 client.0 _ll_register_callbacks cb 0x56347c877530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c99d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c99d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35610/0 (socket says 172.21.15.90:35610) 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.732+0000 7f09a7fff700 1 -- 172.21.15.90:0/2477711579 learned_addr learned my addr 172.21.15.90:0/2477711579 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09a77fe700 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c99a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09acbdb700 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c99a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09a7fff700 1 -- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 msgr2=0x56347c99a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09a7fff700 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c99a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09a7fff700 1 -- 172.21.15.90:0/2477711579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 msgr2=0x56347c99a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09a7fff700 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c99a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09a7fff700 1 -- 172.21.15.90:0/2477711579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347c99afd0 con 0x56347c9912d0 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09a7fff700 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c99d150 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f099809bd00 tx=0x7f09980233f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bcc6182df1dd9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09877fe700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:23.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09877fe700 1 -- 172.21.15.90:0/2477711579 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980b4070 con 0x56347c9912d0 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09877fe700 1 -- 172.21.15.90:0/2477711579 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f099809c460 con 0x56347c9912d0 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a0004bb0 con 0x56347c9912d0 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09877fe700 1 -- 172.21.15.90:0/2477711579 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f09980a03b0 con 0x56347c9912d0 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09b8ed0b80 10 client.15285 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56347c9a8970 con 0x56347c9912d0 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09877fe700 1 -- 172.21.15.90:0/2477711579 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09a0004bb0 con 0x56347c9912d0 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09877fe700 1 -- 172.21.15.90:0/2477711579 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0998024060 con 0x56347c9912d0 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.733+0000 7f09b8ed0b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09b8ed0b80 10 client.15285 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09b8ed0b80 20 client.15285 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9aaf70 0x56347c9ac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09b8ed0b80 4 client.15285 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:23.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 --> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56347c8f10d0 con 0x56347c9aaf70 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09a77fe700 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9aaf70 0x56347c9ac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09a77fe700 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9aaf70 0x56347c9ac3e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ec25aee68b68467e server_cookie=351cb6af0bc9155d in_seq=0 out_seq=0 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.734+0000 7f09877fe700 10 client.15285 ms_handle_connect on v2:172.21.15.167:6836/200145753 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09877fe700 1 -- 172.21.15.90:0/2477711579 <== mds.2 v2:172.21.15.167:6836/200145753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56347c8f10d0 con 0x56347c9aaf70 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09877fe700 10 client.15285 handle_command_reply: tid=0 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 client.15285 shutdown 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 msgr2=0x56347c99d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c99d150 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f099809bd00 tx=0x7f09980233f0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 shutdown_connections 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9aaf70 0x56347c9ac3e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c9912d0 0x56347c99d150 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09877fe700 0 client.15285 ms_handle_reset on v2:172.21.15.167:6836/200145753 2024-04-26T18:12:23.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9973f0 0x56347c99a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/2477711579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c99a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 >> 172.21.15.90:0/2477711579 conn(0x56347c8bdcf0 msgr2=0x56347c98b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 shutdown_connections 2024-04-26T18:12:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.735+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/2477711579 wait complete. 2024-04-26T18:12:23.743 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.736+0000 7f09b8ed0b80 20 client.15285 trim_cache size 0 max 16384 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.755+0000 7f09b8ed0b80 1 Processor -- start 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.755+0000 7f09b8ed0b80 1 -- start start 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c8bdcf0 0x56347c990cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c997fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c998d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56347c9aa720 con 0x56347c8bdcf0 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56347c9aadf0 con 0x56347c996ad0 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56347c9ab4c0 con 0x56347c9912d0 2024-04-26T18:12:23.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c8bdcf0 0x56347c990cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c8bdcf0 0x56347c990cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35626/0 (socket says 172.21.15.90:35626) 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 -- 172.21.15.90:0/1481427759 learned_addr learned my addr 172.21.15.90:0/1481427759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09acbdb700 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c998d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 -- 172.21.15.90:0/1481427759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 msgr2=0x56347c997fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a77fe700 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c997fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c997fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 -- 172.21.15.90:0/1481427759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 msgr2=0x56347c998d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c998d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 -- 172.21.15.90:0/1481427759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56347c99e090 con 0x56347c8bdcf0 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09acbdb700 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c998d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a77fe700 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c997fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.756+0000 7f09a7fff700 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c8bdcf0 0x56347c990cb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f09980ce000 tx=0x7f099809c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fab89d38f1ecc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:23.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f098dffb700 1 -- 172.21.15.90:0/1481427759 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09980b4070 con 0x56347c8bdcf0 2024-04-26T18:12:23.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f098dffb700 1 -- 172.21.15.90:0/1481427759 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09980cd7c0 con 0x56347c8bdcf0 2024-04-26T18:12:23.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f098dffb700 1 -- 172.21.15.90:0/1481427759 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099809c260 con 0x56347c8bdcf0 2024-04-26T18:12:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/1481427759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c8bdcf0 msgr2=0x56347c990cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c8bdcf0 0x56347c990cb0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f09980ce000 tx=0x7f099809c180 comp rx=0 tx=0).stop 2024-04-26T18:12:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/1481427759 shutdown_connections 2024-04-26T18:12:23.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c8bdcf0 0x56347c990cb0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c996ad0 0x56347c998d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/1481427759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c997fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/1481427759 >> 172.21.15.90:0/1481427759 conn(0x56347c991a20 msgr2=0x56347c997890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/1481427759 shutdown_connections 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.757+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/1481427759 wait complete. 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 -- start start 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 Processor -- start 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c8bdcf0 0x56347c9b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c9aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c996ad0 0x56347c9b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56347c9ac260 con 0x56347c996ad0 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56347c9abb90 con 0x56347c8bdcf0 2024-04-26T18:12:23.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56347c9ac930 con 0x56347c9912d0 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09b8ed0b80 10 client.0 _ll_register_callbacks cb 0x56347c877530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c8bdcf0 0x56347c9b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c8bdcf0 0x56347c9b5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:57750/0 (socket says 172.21.15.90:57750) 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09a7fff700 1 -- 172.21.15.90:0/3514679965 learned_addr learned my addr 172.21.15.90:0/3514679965 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.758+0000 7f09acbdb700 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c996ad0 0x56347c9b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09a77fe700 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c9aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09a7fff700 1 -- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 msgr2=0x56347c9aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09a7fff700 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c9aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09a7fff700 1 -- 172.21.15.90:0/3514679965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c996ad0 msgr2=0x56347c9b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09a7fff700 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c996ad0 0x56347c9b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09a7fff700 1 -- 172.21.15.90:0/3514679965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56347c9b0d10 con 0x56347c8bdcf0 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09a7fff700 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c8bdcf0 0x56347c9b5be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f099809bf90 tx=0x7f0998005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb6c3f7fd217b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:23.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09877fe700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09877fe700 1 -- 172.21.15.90:0/3514679965 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099807f060 con 0x56347c8bdcf0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09980cd7c0 con 0x56347c8bdcf0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09877fe700 1 -- 172.21.15.90:0/3514679965 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f099809c460 con 0x56347c8bdcf0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09877fe700 1 -- 172.21.15.90:0/3514679965 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f09980b4050 con 0x56347c8bdcf0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09b8ed0b80 10 client.24899 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.759+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56347c9b0d10 con 0x56347c8bdcf0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09877fe700 1 -- 172.21.15.90:0/3514679965 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09980cd7c0 con 0x56347c8bdcf0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09877fe700 1 -- 172.21.15.90:0/3514679965 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f0998082020 con 0x56347c8bdcf0 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09b8ed0b80 10 client.24899 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09b8ed0b80 10 client.24899 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09b8ed0b80 20 client.24899 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9973f0 0x56347ca162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:23.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09b8ed0b80 4 client.24899 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 --> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] -- command(tid 0: {"prefix": "status"}) -- 0x56347c8eb170 con 0x56347c9973f0 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.760+0000 7f09a77fe700 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9973f0 0x56347ca162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09877fe700 10 client.24899 ms_handle_connect on v2:172.21.15.167:6836/200145753 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09a77fe700 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9973f0 0x56347ca162e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=31a28408028af886 server_cookie=d3287789bee194fa in_seq=0 out_seq=0 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09877fe700 1 -- 172.21.15.90:0/3514679965 <== mds.2 v2:172.21.15.167:6836/200145753 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56347c8eb170 con 0x56347c9973f0 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09877fe700 10 client.24899 handle_command_reply: tid=0 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 client.24899 shutdown 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c8bdcf0 msgr2=0x56347c9b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c8bdcf0 0x56347c9b5be0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f099809bf90 tx=0x7f0998005da0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 shutdown_connections 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:6836/200145753,v1:172.21.15.167:6837/200145753] conn(0x56347c9973f0 0x56347ca162e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56347c996ad0 0x56347c9b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56347c8bdcf0 0x56347c9b5be0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09877fe700 0 client.24899 ms_handle_reset on v2:172.21.15.167:6836/200145753 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 --2- 172.21.15.90:0/3514679965 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56347c9912d0 0x56347c9aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 >> 172.21.15.90:0/3514679965 conn(0x56347c991a20 msgr2=0x56347c99bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 shutdown_connections 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 1 -- 172.21.15.90:0/3514679965 wait complete. 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.761+0000 7f09b8ed0b80 20 client.24899 trim_cache size 0 max 16384 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:12:23.767 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 2, 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24436, 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 201.82781617800001, 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 26, 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 93, 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 244.648979844 2024-04-26T18:12:23.768 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f09900a3ed0 msgr2=0x7f09900a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f09900a3ed0 0x7f09900a6390 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f099c0098e0 tx=0x7f099c005960 comp rx=0 tx=0).stop 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a81023b0 msgr2=0x7f09a81a0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a81023b0 0x7f09a81a0170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f099800d980 tx=0x7f099800dc90 comp rx=0 tx=0).stop 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 shutdown_connections 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f09900a3ed0 0x7f09900a6390 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f09a81023b0 0x7f09a81a0170 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09a8107b30 0x7f09a81a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 --2- 172.21.15.90:0/3282841244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f09a8107180 0x7f09a81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 >> 172.21.15.90:0/3282841244 conn(0x7f09a8077640 msgr2=0x7f09a8102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 shutdown_connections 2024-04-26T18:12:23.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:23.762+0000 7f09ae63e700 1 -- 172.21.15.90:0/3282841244 wait complete. 2024-04-26T18:12:24.384 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:12:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:24 smithi090 ceph-mon[29180]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:24.626 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:24 smithi119 ceph-mon[32752]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:24 smithi167 ceph-mon[32751]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- 172.21.15.90:0/2133122936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828108920 msgr2=0x7f4828108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 --2- 172.21.15.90:0/2133122936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828108920 0x7f4828108d00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4824009930 tx=0x7f482402a420 comp rx=0 tx=0).stop 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- 172.21.15.90:0/2133122936 shutdown_connections 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 --2- 172.21.15.90:0/2133122936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48281032d0 0x7f482810f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 --2- 172.21.15.90:0/2133122936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4828102910 0x7f4828102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 --2- 172.21.15.90:0/2133122936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828108920 0x7f4828108d00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- 172.21.15.90:0/2133122936 >> 172.21.15.90:0/2133122936 conn(0x7f48280fe3e0 msgr2=0x7f4828100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- 172.21.15.90:0/2133122936 shutdown_connections 2024-04-26T18:12:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- 172.21.15.90:0/2133122936 wait complete. 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 Processor -- start 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- start start 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828102910 0x7f4828199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48281032d0 0x7f48281995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4828108920 0x7f482819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4828112be0 con 0x7f4828102910 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4828112a60 con 0x7f4828108920 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.704+0000 7f4830759640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4828112d60 con 0x7f48281032d0 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828102910 0x7f4828199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828102910 0x7f4828199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35656/0 (socket says 172.21.15.90:35656) 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 -- 172.21.15.90:0/8193539 learned_addr learned my addr 172.21.15.90:0/8193539 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482dcf4640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48281032d0 0x7f48281995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482ecf6640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4828108920 0x7f482819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 -- 172.21.15.90:0/8193539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48281032d0 msgr2=0x7f48281995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48281032d0 0x7f48281995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 -- 172.21.15.90:0/8193539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4828108920 msgr2=0x7f482819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4828108920 0x7f482819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 -- 172.21.15.90:0/8193539 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482819e050 con 0x7f4828102910 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482ecf6640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4828108920 0x7f482819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482dcf4640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48281032d0 0x7f48281995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f482e4f5640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828102910 0x7f4828199060 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4824004770 tx=0x7f48240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f481f7fe640 1 -- 172.21.15.90:0/8193539 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482403f070 con 0x7f4828102910 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f481f7fe640 1 -- 172.21.15.90:0/8193539 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f482403b460 con 0x7f4828102910 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f482819e2e0 con 0x7f4828102910 2024-04-26T18:12:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f481f7fe640 1 -- 172.21.15.90:0/8193539 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4824035070 con 0x7f4828102910 2024-04-26T18:12:25.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.705+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48281a5bd0 con 0x7f4828102910 2024-04-26T18:12:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f481f7fe640 1 -- 172.21.15.90:0/8193539 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f482403b8d0 con 0x7f4828102910 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 Processor -- start 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 -- start start 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c5452df00 0x555c5452e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54538ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c545390e0 0x555c5453b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555c54494460 con 0x555c5452df00 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555c5444e590 con 0x555c545390e0 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f4833470740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555c54490420 con 0x555c5452e820 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.708+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c5452df00 0x555c5452e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c5452df00 0x555c5452e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35666/0 (socket says 172.21.15.90:35666) 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 -- 172.21.15.90:0/2648699441 learned_addr learned my addr 172.21.15.90:0/2648699441 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482dcf4640 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c545390e0 0x555c5453b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482e4f5640 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54538ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 -- 172.21.15.90:0/2648699441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 msgr2=0x555c54538ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54538ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 -- 172.21.15.90:0/2648699441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c545390e0 msgr2=0x555c5453b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c545390e0 0x555c5453b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 -- 172.21.15.90:0/2648699441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c5445b0d0 con 0x555c5452df00 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482e4f5640 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54538ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f481f7fe640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f48140a3f60 0x7f48140a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c5452df00 0x555c5452e2e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4820007e60 tx=0x7f482000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37578a8dcb6d5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f482e4f5640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f48140a3f60 0x7f48140a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f481cff9640 1 -- 172.21.15.90:0/2648699441 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820014070 con 0x555c5452df00 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f481f7fe640 1 -- 172.21.15.90:0/8193539 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f48240f0d50 con 0x7f4828102910 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f481cff9640 1 -- 172.21.15.90:0/2648699441 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820004d20 con 0x555c5452df00 2024-04-26T18:12:25.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.709+0000 7f481cff9640 1 -- 172.21.15.90:0/2648699441 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820005440 con 0x555c5452df00 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 -- 172.21.15.90:0/2648699441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c5452df00 msgr2=0x555c5452e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f482e4f5640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f48140a3f60 0x7f48140a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f48240ee390 tx=0x7f482402a420 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c5452df00 0x555c5452e2e0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4820007e60 tx=0x7f482000e720 comp rx=0 tx=0).stop 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 -- 172.21.15.90:0/2648699441 shutdown_connections 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c545390e0 0x555c5453b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54538ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 --2- 172.21.15.90:0/2648699441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c5452df00 0x555c5452e2e0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 -- 172.21.15.90:0/2648699441 >> 172.21.15.90:0/2648699441 conn(0x555c5452a410 msgr2=0x555c5452a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 -- 172.21.15.90:0/2648699441 shutdown_connections 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.710+0000 7f4833470740 1 -- 172.21.15.90:0/2648699441 wait complete. 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 -- start start 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 Processor -- start 2024-04-26T18:12:25.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452df00 0x555c54536070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452e820 0x555c545365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54536af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555c5453bd10 con 0x555c545390e0 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555c5441bb60 con 0x555c5452e820 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555c5453be90 con 0x555c5452df00 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f4833470740 10 client.0 _ll_register_callbacks cb 0x555c544402b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54536af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54536af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35682/0 (socket says 172.21.15.90:35682) 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 -- 172.21.15.90:0/2535167959 learned_addr learned my addr 172.21.15.90:0/2535167959 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482dcf4640 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452e820 0x555c545365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452df00 0x555c54536070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 -- 172.21.15.90:0/2535167959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452df00 msgr2=0x555c54536070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452df00 0x555c54536070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 -- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452e820 msgr2=0x555c545365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452e820 0x555c545365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.711+0000 7f482ecf6640 1 -- 172.21.15.90:0/2535167959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c541a6210 con 0x555c545390e0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54536af0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f48200072d0 tx=0x7f48200050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7493999bba46faeb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4811ffb640 1 -- 172.21.15.90:0/2535167959 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820014070 con 0x555c545390e0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4811ffb640 1 -- 172.21.15.90:0/2535167959 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4820003970 con 0x555c545390e0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4820004d20 con 0x555c545390e0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4811ffb640 1 -- 172.21.15.90:0/2535167959 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4820030050 con 0x555c545390e0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 10 client.15303 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c54272620 con 0x555c545390e0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4811ffb640 1 -- 172.21.15.90:0/2535167959 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820004d20 con 0x555c545390e0 2024-04-26T18:12:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4811ffb640 1 -- 172.21.15.90:0/2535167959 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f48200185a0 con 0x555c545390e0 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 10 client.15303 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 10 client.15303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 20 client.15303 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545490e0 0x555c5454b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 4 client.15303 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.712+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c544a2070 con 0x555c545490e0 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.713+0000 7f482dcf4640 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545490e0 0x555c5454b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.713+0000 7f482dcf4640 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545490e0 0x555c5454b4d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=714aa67972c60813 server_cookie=dee131dc299fb459 in_seq=0 out_seq=0 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.713+0000 7f4811ffb640 10 client.15303 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4811ffb640 1 -- 172.21.15.90:0/2535167959 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c544a2070 con 0x555c545490e0 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4811ffb640 10 client.15303 handle_command_reply: tid=0 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 client.15303 shutdown 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 msgr2=0x555c54536af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54536af0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f48200072d0 tx=0x7f48200050c0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 shutdown_connections 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545490e0 0x555c5454b4d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54536af0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452e820 0x555c545365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4811ffb640 0 client.15303 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 --2- 172.21.15.90:0/2535167959 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452df00 0x555c54536070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 >> 172.21.15.90:0/2535167959 conn(0x555c5452a410 msgr2=0x555c54530140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 shutdown_connections 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 1 -- 172.21.15.90:0/2535167959 wait complete. 2024-04-26T18:12:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.714+0000 7f4833470740 20 client.15303 trim_cache size 0 max 16384 2024-04-26T18:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 Processor -- start 2024-04-26T18:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 -- start start 2024-04-26T18:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c54544bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54550bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54551130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f48281b0c80 con 0x555c545390e0 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ae550 con 0x555c5452df00 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f4833470740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ac3a0 con 0x555c5452e820 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54551130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54551130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35690/0 (socket says 172.21.15.90:35690) 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 -- 172.21.15.90:0/310611948 learned_addr learned my addr 172.21.15.90:0/310611948 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482dcf4640 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54550bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c54544bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 -- 172.21.15.90:0/310611948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 msgr2=0x555c54550bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54550bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 -- 172.21.15.90:0/310611948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 msgr2=0x555c54544bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c54544bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 -- 172.21.15.90:0/310611948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c5452e650 con 0x555c545390e0 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.729+0000 7f482ecf6640 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54551130 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f48200109d0 tx=0x7f482000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5668c3a697aafd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4813fff640 1 -- 172.21.15.90:0/310611948 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820014040 con 0x555c545390e0 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4813fff640 1 -- 172.21.15.90:0/310611948 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820005970 con 0x555c545390e0 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4813fff640 1 -- 172.21.15.90:0/310611948 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820021040 con 0x555c545390e0 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 -- 172.21.15.90:0/310611948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 msgr2=0x555c54551130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54551130 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f48200109d0 tx=0x7f482000e720 comp rx=0 tx=0).stop 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 -- 172.21.15.90:0/310611948 shutdown_connections 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c54551130 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c54550bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 --2- 172.21.15.90:0/310611948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c54544bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 -- 172.21.15.90:0/310611948 >> 172.21.15.90:0/310611948 conn(0x555c5452a410 msgr2=0x555c5452a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 -- 172.21.15.90:0/310611948 shutdown_connections 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.730+0000 7f4833470740 1 -- 172.21.15.90:0/310611948 wait complete. 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 -- start start 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 Processor -- start 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c5453cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c5453d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c5453d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f48281aca10 con 0x555c545390e0 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ade80 con 0x555c5452df00 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48281ad0e0 con 0x555c5452e820 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f4833470740 10 client.0 _ll_register_callbacks cb 0x7f481800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c5453d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c5453d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35706/0 (socket says 172.21.15.90:35706) 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 -- 172.21.15.90:0/2627011860 learned_addr learned my addr 172.21.15.90:0/2627011860 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482dcf4640 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c5453d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c5453cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 -- 172.21.15.90:0/2627011860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 msgr2=0x555c5453d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c5453d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 -- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 msgr2=0x555c5453cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c5453cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.731+0000 7f482ecf6640 1 -- 172.21.15.90:0/2627011860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c545ae530 con 0x555c545390e0 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f482ecf6640 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c5453d650 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4820007fd0 tx=0x7f482002a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f26aa0a2a44551 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4811ffb640 1 -- 172.21.15.90:0/2627011860 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820014070 con 0x555c545390e0 2024-04-26T18:12:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4820005970 con 0x555c545390e0 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4811ffb640 1 -- 172.21.15.90:0/2627011860 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4820010cb0 con 0x555c545390e0 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4811ffb640 1 -- 172.21.15.90:0/2627011860 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4820039040 con 0x555c545390e0 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 10 client.15309 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c545af840 con 0x555c545390e0 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4811ffb640 1 -- 172.21.15.90:0/2627011860 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820005970 con 0x555c545390e0 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4811ffb640 1 -- 172.21.15.90:0/2627011860 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f48200286a0 con 0x555c545390e0 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 10 client.15309 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 10 client.15309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 20 client.15309 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545afe20 0x555c545b2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 4 client.15309 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:12:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.732+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c5449bee0 con 0x555c545afe20 2024-04-26T18:12:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.733+0000 7f482dcf4640 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545afe20 0x555c545b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.733+0000 7f482dcf4640 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545afe20 0x555c545b2240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3ce0e684af31dde server_cookie=91eb3b3c8d517635 in_seq=0 out_seq=0 2024-04-26T18:12:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.733+0000 7f4811ffb640 10 client.15309 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4811ffb640 1 -- 172.21.15.90:0/2627011860 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c5449bee0 con 0x555c545afe20 2024-04-26T18:12:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4811ffb640 10 client.15309 handle_command_reply: tid=0 2024-04-26T18:12:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 client.15309 shutdown 2024-04-26T18:12:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 msgr2=0x555c5453d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c5453d650 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4820007fd0 tx=0x7f482002a040 comp rx=0 tx=0).stop 2024-04-26T18:12:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 shutdown_connections 2024-04-26T18:12:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555c545afe20 0x555c545b2240 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555c545390e0 0x555c5453d650 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555c5452e820 0x555c5453d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4811ffb640 0 client.15309 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 --2- 172.21.15.90:0/2627011860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555c5452df00 0x555c5453cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 >> 172.21.15.90:0/2627011860 conn(0x555c5452a410 msgr2=0x555c54528d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 shutdown_connections 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 1 -- 172.21.15.90:0/2627011860 wait complete. 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.742+0000 7f4833470740 20 client.15309 trim_cache size 0 max 16384 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:12:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:12:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:12:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:12:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f48140a3f60 msgr2=0x7f48140a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f48140a3f60 0x7f48140a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f48240ee390 tx=0x7f482402a420 comp rx=0 tx=0).stop 2024-04-26T18:12:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828102910 msgr2=0x7f4828199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828102910 0x7f4828199060 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4824004770 tx=0x7f48240042f0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 shutdown_connections 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f48140a3f60 0x7f48140a6420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4828108920 0x7f482819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48281032d0 0x7f48281995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 --2- 172.21.15.90:0/8193539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4828102910 0x7f4828199060 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 >> 172.21.15.90:0/8193539 conn(0x7f48280fe3e0 msgr2=0x7f48280feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 shutdown_connections 2024-04-26T18:12:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:25.743+0000 7f4830759640 1 -- 172.21.15.90:0/8193539 wait complete. 2024-04-26T18:12:26.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 -- 172.21.15.90:0/497766513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 msgr2=0x7fee0c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 --2- 172.21.15.90:0/497766513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c1038f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fee08009990 tx=0x7fee08009ca0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 -- 172.21.15.90:0/497766513 shutdown_connections 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 --2- 172.21.15.90:0/497766513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee0c103e30 0x7fee0c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 --2- 172.21.15.90:0/497766513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee0c1094c0 0x7fee0c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 --2- 172.21.15.90:0/497766513 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c1038f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 -- 172.21.15.90:0/497766513 >> 172.21.15.90:0/497766513 conn(0x7fee0c07a580 msgr2=0x7fee0c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 -- 172.21.15.90:0/497766513 shutdown_connections 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 -- 172.21.15.90:0/497766513 wait complete. 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.209+0000 7fee13ae2700 1 Processor -- start 2024-04-26T18:12:26.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee13ae2700 1 -- start start 2024-04-26T18:12:26.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee13ae2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee13ae2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee0c103e30 0x7fee0c19fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee13ae2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee0c1094c0 0x7fee0c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee13ae2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c113630 con 0x7fee0c103470 2024-04-26T18:12:26.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee13ae2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c1134b0 con 0x7fee0c103e30 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee13ae2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c1137b0 con 0x7fee0c1094c0 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35730/0 (socket says 172.21.15.90:35730) 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 -- 172.21.15.90:0/3025797814 learned_addr learned my addr 172.21.15.90:0/3025797814 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1107d700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee0c103e30 0x7fee0c19fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1207f700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee0c1094c0 0x7fee0c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 -- 172.21.15.90:0/3025797814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee0c1094c0 msgr2=0x7fee0c1999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee0c1094c0 0x7fee0c1999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 -- 172.21.15.90:0/3025797814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee0c103e30 msgr2=0x7fee0c19fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee0c103e30 0x7fee0c19fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 -- 172.21.15.90:0/3025797814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c19a150 con 0x7fee0c103470 2024-04-26T18:12:26.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1207f700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee0c1094c0 0x7fee0c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.210+0000 7fee1187e700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c19f830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d980 tx=0x7fedfc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.211+0000 7fee02ffd700 1 -- 172.21.15.90:0/3025797814 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc017070 con 0x7fee0c103470 2024-04-26T18:12:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.211+0000 7fee02ffd700 1 -- 172.21.15.90:0/3025797814 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedfc005020 con 0x7fee0c103470 2024-04-26T18:12:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.211+0000 7fee02ffd700 1 -- 172.21.15.90:0/3025797814 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc009b50 con 0x7fee0c103470 2024-04-26T18:12:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.211+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c19a440 con 0x7fee0c103470 2024-04-26T18:12:26.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.211+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1a6700 con 0x7fee0c103470 2024-04-26T18:12:26.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.213+0000 7fee02ffd700 1 -- 172.21.15.90:0/3025797814 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7fedfc0051d0 con 0x7fee0c103470 2024-04-26T18:12:26.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.213+0000 7fee02ffd700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fedf80a3ed0 0x7fedf80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.213+0000 7fee1107d700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fedf80a3ed0 0x7fedf80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.213+0000 7fee02ffd700 1 -- 172.21.15.90:0/3025797814 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fedfc0d0940 con 0x7fee0c103470 2024-04-26T18:12:26.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.214+0000 7fee1107d700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fedf80a3ed0 0x7fedf80a6390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee0c07abf0 tx=0x7fee08005fd0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:26.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.214+0000 7fee1e374b80 1 Processor -- start 2024-04-26T18:12:26.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.214+0000 7fee1e374b80 1 -- start start 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c16f0 0x56365b2cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1e374b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56365b2137e0 con 0x56365b2c6a20 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1e374b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56365b1798e0 con 0x56365b2c0d50 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1e374b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56365b20f7a0 con 0x56365b2c16f0 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35740/0 (socket says 172.21.15.90:35740) 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 -- 172.21.15.90:0/1359555622 learned_addr learned my addr 172.21.15.90:0/1359555622 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1107d700 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 -- 172.21.15.90:0/1359555622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c16f0 msgr2=0x56365b2cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c16f0 0x56365b2cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 -- 172.21.15.90:0/1359555622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 msgr2=0x56365b2c11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2c11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 -- 172.21.15.90:0/1359555622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365b160880 con 0x56365b2c6a20 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1107d700 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fee1187e700 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2c6e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0ce700 tx=0x7fedfc09c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae830b1d0093cd94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fedf7fff700 1 -- 172.21.15.90:0/1359555622 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0a5370 con 0x56365b2c6a20 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fedf7fff700 1 -- 172.21.15.90:0/1359555622 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedfc09aa00 con 0x56365b2c6a20 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.215+0000 7fedf7fff700 1 -- 172.21.15.90:0/1359555622 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc00ebc0 con 0x56365b2c6a20 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 -- 172.21.15.90:0/1359555622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 msgr2=0x56365b2c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2c6e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0ce700 tx=0x7fedfc09c700 comp rx=0 tx=0).stop 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 -- 172.21.15.90:0/1359555622 shutdown_connections 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2c6e00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c16f0 0x56365b2cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 --2- 172.21.15.90:0/1359555622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2c11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 -- 172.21.15.90:0/1359555622 >> 172.21.15.90:0/1359555622 conn(0x56365b1ef570 msgr2=0x56365b2bb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 -- 172.21.15.90:0/1359555622 shutdown_connections 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.216+0000 7fee1e374b80 1 -- 172.21.15.90:0/1359555622 wait complete. 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 -- start start 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 Processor -- start 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c16f0 0x56365b2ca800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56365b2cdcb0 con 0x56365b2c6a20 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56365b1cda30 con 0x56365b2c16f0 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56365b2cde30 con 0x56365b2c0d50 2024-04-26T18:12:26.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1e374b80 10 client.0 _ll_register_callbacks cb 0x56365b1ab900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1207f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1207f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35754/0 (socket says 172.21.15.90:35754) 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1207f700 1 -- 172.21.15.90:0/3155295522 learned_addr learned my addr 172.21.15.90:0/3155295522 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1187e700 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.217+0000 7fee1107d700 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c16f0 0x56365b2ca800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1207f700 1 -- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 msgr2=0x56365b2cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1207f700 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1207f700 1 -- 172.21.15.90:0/3155295522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c16f0 msgr2=0x56365b2ca800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1207f700 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c16f0 0x56365b2ca800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1207f700 1 -- 172.21.15.90:0/3155295522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365b1d1430 con 0x56365b2c6a20 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1207f700 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2cad40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee040077a0 tx=0x7fee04007ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f06e7f760c7cab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fedf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:26.223 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fedf57fa700 1 -- 172.21.15.90:0/3155295522 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04013790 con 0x56365b2c6a20 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedfc09aa00 con 0x56365b2c6a20 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fedf57fa700 1 -- 172.21.15.90:0/3155295522 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fee04013930 con 0x56365b2c6a20 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fedf57fa700 1 -- 172.21.15.90:0/3155295522 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee04022050 con 0x56365b2c6a20 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1e374b80 10 client.15321 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56365b2d8580 con 0x56365b2c6a20 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fedf57fa700 1 -- 172.21.15.90:0/3155295522 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedfc09aa00 con 0x56365b2c6a20 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.218+0000 7fedf57fa700 1 -- 172.21.15.90:0/3155295522 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fee04014ce0 con 0x56365b2c6a20 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1e374b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1e374b80 10 client.15321 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1e374b80 20 client.15321 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b2daba0 0x56365b2dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1e374b80 4 client.15321 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:26.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 --> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56365b2211c0 con 0x56365b2daba0 2024-04-26T18:12:26.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1187e700 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b2daba0 0x56365b2dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fedf57fa700 10 client.15321 ms_handle_connect on v2:172.21.15.167:6838/4229007587 2024-04-26T18:12:26.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.219+0000 7fee1187e700 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b2daba0 0x56365b2dc010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72d6cea09dd599ef server_cookie=8a8e616ce9054a2d in_seq=0 out_seq=0 2024-04-26T18:12:26.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fedf57fa700 1 -- 172.21.15.90:0/3155295522 <== mds.1 v2:172.21.15.167:6838/4229007587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56365b2211c0 con 0x56365b2daba0 2024-04-26T18:12:26.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fedf57fa700 10 client.15321 handle_command_reply: tid=0 2024-04-26T18:12:26.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 client.15321 shutdown 2024-04-26T18:12:26.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 msgr2=0x56365b2cad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2cad40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fee040077a0 tx=0x7fee04007ab0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 shutdown_connections 2024-04-26T18:12:26.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b2daba0 0x56365b2dc010 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c6a20 0x56365b2cad40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fedf57fa700 0 client.15321 ms_handle_reset on v2:172.21.15.167:6838/4229007587 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3155295522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c16f0 0x56365b2ca800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 >> 172.21.15.90:0/3155295522 conn(0x56365b1ef570 msgr2=0x56365b2bfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 shutdown_connections 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 1 -- 172.21.15.90:0/3155295522 wait complete. 2024-04-26T18:12:26.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.220+0000 7fee1e374b80 20 client.15321 trim_cache size 0 max 16384 2024-04-26T18:12:26.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.240+0000 7fee1e374b80 1 Processor -- start 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.240+0000 7fee1e374b80 1 -- start start 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c6a20 0x56365b2c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1e374b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56365b2da350 con 0x56365b2c16f0 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1e374b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56365b2daa20 con 0x56365b2c6a20 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1e374b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56365b2db0f0 con 0x56365b2c0d50 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35756/0 (socket says 172.21.15.90:35756) 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 -- 172.21.15.90:0/3170787475 learned_addr learned my addr 172.21.15.90:0/3170787475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:26.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1187e700 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 -- 172.21.15.90:0/3170787475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 msgr2=0x56365b2cad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1207f700 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c6a20 0x56365b2c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 -- 172.21.15.90:0/3170787475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c6a20 msgr2=0x56365b2c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c6a20 0x56365b2c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 -- 172.21.15.90:0/3170787475 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56365b2dcf70 con 0x56365b2c16f0 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1207f700 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c6a20 0x56365b2c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.241+0000 7fee1107d700 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2ccca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee08009990 tx=0x7fee08023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ee0d03db7b46f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fedf77fe700 1 -- 172.21.15.90:0/3170787475 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08036070 con 0x56365b2c16f0 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fedf77fe700 1 -- 172.21.15.90:0/3170787475 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee08024bf0 con 0x56365b2c16f0 2024-04-26T18:12:26.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fedf77fe700 1 -- 172.21.15.90:0/3170787475 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08031d00 con 0x56365b2c16f0 2024-04-26T18:12:26.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 -- 172.21.15.90:0/3170787475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 msgr2=0x56365b2ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2ccca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee08009990 tx=0x7fee08023000 comp rx=0 tx=0).stop 2024-04-26T18:12:26.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 -- 172.21.15.90:0/3170787475 shutdown_connections 2024-04-26T18:12:26.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c6a20 0x56365b2c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c0d50 0x56365b2cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 --2- 172.21.15.90:0/3170787475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2ccca0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 -- 172.21.15.90:0/3170787475 >> 172.21.15.90:0/3170787475 conn(0x56365b1ef570 msgr2=0x56365b2bb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 -- 172.21.15.90:0/3170787475 shutdown_connections 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.242+0000 7fee1e374b80 1 -- 172.21.15.90:0/3170787475 wait complete. 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 -- start start 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 Processor -- start 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2e5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c6a20 0x56365b2e0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56365b2dbe90 con 0x56365b2c16f0 2024-04-26T18:12:26.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56365b2db7c0 con 0x56365b2c0d50 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56365b2dc560 con 0x56365b2c6a20 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1e374b80 10 client.0 _ll_register_callbacks cb 0x56365b1ab900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1107d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2e59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1107d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2e59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35766/0 (socket says 172.21.15.90:35766) 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.243+0000 7fee1107d700 1 -- 172.21.15.90:0/794812931 learned_addr learned my addr 172.21.15.90:0/794812931 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1187e700 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1207f700 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c6a20 0x56365b2e0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1107d700 1 -- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c6a20 msgr2=0x56365b2e0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1107d700 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c6a20 0x56365b2e0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1107d700 1 -- 172.21.15.90:0/794812931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 msgr2=0x56365b2e5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1107d700 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2e5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1107d700 1 -- 172.21.15.90:0/794812931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56365b2e0c80 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1107d700 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2e59d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fee08034040 tx=0x7fee0800b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5c1eab8c561192 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fedf57fa700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fedf57fa700 1 -- 172.21.15.90:0/794812931 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08041060 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fedf57fa700 1 -- 172.21.15.90:0/794812931 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fee08031b20 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee08024bf0 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fedf57fa700 1 -- 172.21.15.90:0/794812931 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fee08036070 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1e374b80 10 client.15327 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56365b342680 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.244+0000 7fedf57fa700 1 -- 172.21.15.90:0/794812931 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee08024bf0 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fedf57fa700 1 -- 172.21.15.90:0/794812931 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fee08044030 con 0x56365b2c16f0 2024-04-26T18:12:26.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fee1e374b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fee1e374b80 10 client.15327 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fee1e374b80 20 client.15327 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fee1e374b80 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b346d60 0x56365b345a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fee1e374b80 4 client.15327 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 --> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] -- command(tid 0: {"prefix": "status"}) -- 0x56365b21b260 con 0x56365b346d60 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.245+0000 7fee1187e700 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b346d60 0x56365b345a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fedf57fa700 10 client.15327 ms_handle_connect on v2:172.21.15.167:6838/4229007587 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1187e700 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b346d60 0x56365b345a70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18359153e8da58c4 server_cookie=1c454a1a20f359d0 in_seq=0 out_seq=0 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fedf57fa700 1 -- 172.21.15.90:0/794812931 <== mds.1 v2:172.21.15.167:6838/4229007587 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x56365b21b260 con 0x56365b346d60 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fedf57fa700 10 client.15327 handle_command_reply: tid=0 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 client.15327 shutdown 2024-04-26T18:12:26.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 msgr2=0x56365b2e59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2e59d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fee08034040 tx=0x7fee0800b670 comp rx=0 tx=0).stop 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 shutdown_connections 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56365b2c6a20 0x56365b2e0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56365b2c0d50 0x56365b2e5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.167:6838/4229007587,v1:172.21.15.167:6839/4229007587] conn(0x56365b346d60 0x56365b345a70 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 --2- 172.21.15.90:0/794812931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56365b2c16f0 0x56365b2e59d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fedf57fa700 0 client.15327 ms_handle_reset on v2:172.21.15.167:6838/4229007587 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 >> 172.21.15.90:0/794812931 conn(0x56365b1ef570 msgr2=0x56365b2c4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 shutdown_connections 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 1 -- 172.21.15.90:0/794812931 wait complete. 2024-04-26T18:12:26.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.246+0000 7fee1e374b80 20 client.15327 trim_cache size 0 max 16384 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 1, 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24448, 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 205.323813659, 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 26, 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 93, 2024-04-26T18:12:26.253 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T18:12:26.254 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 241.479196234 2024-04-26T18:12:26.254 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:12:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fedf80a3ed0 msgr2=0x7fedf80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fedf80a3ed0 0x7fedf80a6390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fee0c07abf0 tx=0x7fee08005fd0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 msgr2=0x7fee0c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c19f830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fedfc00d980 tx=0x7fedfc00dc90 comp rx=0 tx=0).stop 2024-04-26T18:12:26.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 shutdown_connections 2024-04-26T18:12:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fedf80a3ed0 0x7fedf80a6390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee0c103470 0x7fee0c19f830 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee0c1094c0 0x7fee0c1999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 --2- 172.21.15.90:0/3025797814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee0c103e30 0x7fee0c19fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 >> 172.21.15.90:0/3025797814 conn(0x7fee0c07a580 msgr2=0x7fee0c102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 shutdown_connections 2024-04-26T18:12:26.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:26.247+0000 7fee13ae2700 1 -- 172.21.15.90:0/3025797814 wait complete. 2024-04-26T18:12:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:26 smithi090 ceph-mon[29180]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-26T18:12:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:26 smithi119 ceph-mon[32752]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-26T18:12:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:26 smithi167 ceph-mon[32751]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-04-26T18:12:26.882 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:12:27.120 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:28.559 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:28 smithi090 ceph-mon[29180]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-26T18:12:28.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 -- 172.21.15.90:0/244333950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8108390 msgr2=0x7f53c8108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 --2- 172.21.15.90:0/244333950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8108390 0x7f53c8108770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53c400b0a0 tx=0x7f53c400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 -- 172.21.15.90:0/244333950 shutdown_connections 2024-04-26T18:12:28.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 --2- 172.21.15.90:0/244333950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8108390 0x7f53c8108770 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 --2- 172.21.15.90:0/244333950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c8102ce0 0x7f53c810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 --2- 172.21.15.90:0/244333950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c8102320 0x7f53c81027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 -- 172.21.15.90:0/244333950 >> 172.21.15.90:0/244333950 conn(0x7f53c80792d0 msgr2=0x7f53c80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 -- 172.21.15.90:0/244333950 shutdown_connections 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.770+0000 7f53cf331700 1 -- 172.21.15.90:0/244333950 wait complete. 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 Processor -- start 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 -- start start 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8102320 0x7f53c819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c8102ce0 0x7f53c819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c8108390 0x7f53c81a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81124b0 con 0x7f53c8102320 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8112330 con 0x7f53c8108390 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cf331700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53c8112630 con 0x7f53c8102ce0 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cd0cd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8102320 0x7f53c819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cd0cd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8102320 0x7f53c819a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35792/0 (socket says 172.21.15.90:35792) 2024-04-26T18:12:28.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2700095235 learned_addr learned my addr 172.21.15.90:0/2700095235 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c8102ce0 0x7f53c819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.771+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c8108390 0x7f53c81a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2700095235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c8102ce0 msgr2=0x7f53c819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c8102ce0 0x7f53c819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2700095235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c8108390 msgr2=0x7f53c81a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c8108390 0x7f53c81a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2700095235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c81a5f70 con 0x7f53c8102320 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c8102ce0 0x7f53c819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c8108390 0x7f53c81a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8102320 0x7f53c819a9b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f53c400b070 tx=0x7f53c4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53be7fc700 1 -- 172.21.15.90:0/2700095235 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c4021070 con 0x7f53c8102320 2024-04-26T18:12:28.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53be7fc700 1 -- 172.21.15.90:0/2700095235 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c4007c80 con 0x7f53c8102320 2024-04-26T18:12:28.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c81a6140 con 0x7f53c8102320 2024-04-26T18:12:28.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53be7fc700 1 -- 172.21.15.90:0/2700095235 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c401d450 con 0x7f53c8102320 2024-04-26T18:12:28.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.772+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c81a65f0 con 0x7f53c8102320 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.774+0000 7f53be7fc700 1 -- 172.21.15.90:0/2700095235 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f53c4004020 con 0x7f53c8102320 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.775+0000 7f53be7fc700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f53b40a3ed0 0x7f53b40a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.775+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f53b40a3ed0 0x7f53b40a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.775+0000 7f53be7fc700 1 -- 172.21.15.90:0/2700095235 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f53c400f020 con 0x7f53c8102320 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.775+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f53b40a3ed0 0x7f53b40a6390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005950 tx=0x7f53b80058e0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 Processor -- start 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 -- start start 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a73feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a739cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a73a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e80a68c6f0 con 0x55e80a73a2d0 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e80a62b590 con 0x55e80a7403f0 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e80a6886b0 con 0x55e80a73fad0 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a73a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a73a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35804/0 (socket says 172.21.15.90:35804) 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1493479057 learned_addr learned my addr 172.21.15.90:0/1493479057 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a73feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a739cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1493479057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 msgr2=0x55e80a73feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a73feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1493479057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 msgr2=0x55e80a739cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a739cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.776+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1493479057 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80a5d9880 con 0x55e80a73a2d0 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.777+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a73a710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53c000bd20 tx=0x7f53c000f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b603cfe890a3793 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.777+0000 7f53b37fe700 1 -- 172.21.15.90:0/1493479057 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0015070 con 0x55e80a73a2d0 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.777+0000 7f53b37fe700 1 -- 172.21.15.90:0/1493479057 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c000fda0 con 0x55e80a73a2d0 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.777+0000 7f53b37fe700 1 -- 172.21.15.90:0/1493479057 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0005590 con 0x55e80a73a2d0 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1493479057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 msgr2=0x55e80a73a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a73a710 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f53c000bd20 tx=0x7f53c000f640 comp rx=0 tx=0).stop 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1493479057 shutdown_connections 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a73a710 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a73feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1493479057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a739cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1493479057 >> 172.21.15.90:0/1493479057 conn(0x55e80a666cf0 msgr2=0x55e80a7351b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1493479057 shutdown_connections 2024-04-26T18:12:28.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1493479057 wait complete. 2024-04-26T18:12:28.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 -- start start 2024-04-26T18:12:28.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 Processor -- start 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a746160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.778+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a743460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a7403f0 0x55e80a7439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e80a746f10 con 0x55e80a7403f0 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e80a5f2900 con 0x55e80a73a2d0 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e80a747090 con 0x55e80a73fad0 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a7403f0 0x55e80a7439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a7403f0 0x55e80a7439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35814/0 (socket says 172.21.15.90:35814) 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 10 client.0 _ll_register_callbacks cb 0x55e80a620530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:28.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1121818536 learned_addr learned my addr 172.21.15.90:0/1121818536 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a743460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a746160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1121818536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 msgr2=0x55e80a743460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a743460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 msgr2=0x55e80a746160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a746160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 -- 172.21.15.90:0/1121818536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80a744010 con 0x55e80a7403f0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a7403f0 0x55e80a7439a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005d80 tx=0x7f53c00054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7274724cac27ed42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53b0ff9700 1 -- 172.21.15.90:0/1121818536 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c0028060 con 0x55e80a7403f0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53b0ff9700 1 -- 172.21.15.90:0/1121818536 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f53c000fba0 con 0x55e80a7403f0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c000fda0 con 0x55e80a7403f0 2024-04-26T18:12:28.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53b0ff9700 1 -- 172.21.15.90:0/1121818536 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53c0015070 con 0x55e80a7403f0 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 10 client.15339 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.779+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e80a751990 con 0x55e80a7403f0 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53b0ff9700 1 -- 172.21.15.90:0/1121818536 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c000fda0 con 0x55e80a7403f0 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53b0ff9700 1 -- 172.21.15.90:0/1121818536 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f53c002b020 con 0x55e80a7403f0 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53d9bc3b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53d9bc3b80 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53d9bc3b80 20 client.15339 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a753bb0 0x55e80a755020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53d9bc3b80 4 client.15339 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e80a69a0d0 con 0x55e80a753bb0 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.780+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a753bb0 0x55e80a755020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.781+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a753bb0 0x55e80a755020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bf1f98226a0a803 server_cookie=536090aa83f4388 in_seq=0 out_seq=0 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.781+0000 7f53b0ff9700 10 client.15339 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.781+0000 7f53b0ff9700 1 -- 172.21.15.90:0/1121818536 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e80a69a0d0 con 0x55e80a753bb0 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.781+0000 7f53b0ff9700 10 client.15339 handle_command_reply: tid=0 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 client.15339 shutdown 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a7403f0 msgr2=0x55e80a7439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a7403f0 0x55e80a7439a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f53c0005d80 tx=0x7f53c00054b0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 shutdown_connections 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a753bb0 0x55e80a755020 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a7403f0 0x55e80a7439a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a746160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53b0ff9700 0 client.15339 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/1121818536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a743460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 >> 172.21.15.90:0/1121818536 conn(0x55e80a666cf0 msgr2=0x55e80a734330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:28.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 shutdown_connections 2024-04-26T18:12:28.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/1121818536 wait complete. 2024-04-26T18:12:28.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.782+0000 7f53d9bc3b80 20 client.15339 trim_cache size 0 max 16384 2024-04-26T18:12:28.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 Processor -- start 2024-04-26T18:12:28.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 -- start start 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a739cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a7439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a740890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e80a753360 con 0x55e80a73a2d0 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e80a753a30 con 0x55e80a7403f0 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e80a754100 con 0x55e80a73fad0 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53cd0cd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a739cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53cd0cd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a739cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35822/0 (socket says 172.21.15.90:35822) 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.802+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2374860688 learned_addr learned my addr 172.21.15.90:0/2374860688 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a740890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a7439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2374860688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 msgr2=0x55e80a7439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a7439f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2374860688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 msgr2=0x55e80a740890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a740890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd0cd700 1 -- 172.21.15.90:0/2374860688 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80a7419f0 con 0x55e80a73a2d0 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a740890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a739cb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f53c409a820 tx=0x7f53c40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e873384a2b554 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53b2ffd700 1 -- 172.21.15.90:0/2374860688 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40a2200 con 0x55e80a73a2d0 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53b2ffd700 1 -- 172.21.15.90:0/2374860688 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c40a0040 con 0x55e80a73a2d0 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.803+0000 7f53b2ffd700 1 -- 172.21.15.90:0/2374860688 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c40b70d0 con 0x55e80a73a2d0 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/2374860688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 msgr2=0x55e80a739cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a739cb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f53c409a820 tx=0x7f53c40099e0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/2374860688 shutdown_connections 2024-04-26T18:12:28.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73a2d0 0x55e80a739cb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a7403f0 0x55e80a740890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/2374860688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a73fad0 0x55e80a7439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/2374860688 >> 172.21.15.90:0/2374860688 conn(0x55e80a666cf0 msgr2=0x55e80a741d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/2374860688 shutdown_connections 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/2374860688 wait complete. 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 -- start start 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 Processor -- start 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.804+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a75e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73fad0 0x55e80a74f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53d9bc3b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a7403f0 0x55e80a74f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e80a754ea0 con 0x55e80a73fad0 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e80a7547d0 con 0x55e80a73a2d0 2024-04-26T18:12:28.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53d9bc3b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e80a755570 con 0x55e80a7403f0 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73fad0 0x55e80a74f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53d9bc3b80 10 client.0 _ll_register_callbacks cb 0x55e80a620530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73fad0 0x55e80a74f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35830/0 (socket says 172.21.15.90:35830) 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 -- 172.21.15.90:0/3276592350 learned_addr learned my addr 172.21.15.90:0/3276592350 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a75e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a7403f0 0x55e80a74f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 -- 172.21.15.90:0/3276592350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a7403f0 msgr2=0x55e80a74f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a7403f0 0x55e80a74f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 -- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 msgr2=0x55e80a75e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a75e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 -- 172.21.15.90:0/3276592350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80a7500c0 con 0x55e80a73fad0 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a75e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:28.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cd8ce700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a7403f0 0x55e80a74f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53cc8cc700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73fad0 0x55e80a74f350 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53b800b5d0 tx=0x7f53b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38bb00e01785356 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53b0ff9700 1 -- 172.21.15.90:0/3276592350 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8026070 con 0x55e80a73fad0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53b0ff9700 1 -- 172.21.15.90:0/3276592350 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f53b8003c50 con 0x55e80a73fad0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.805+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53c40a0040 con 0x55e80a73fad0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53b0ff9700 1 -- 172.21.15.90:0/3276592350 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f53b8032050 con 0x55e80a73fad0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 10 client.15345 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e80a7bedb0 con 0x55e80a73fad0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53b0ff9700 1 -- 172.21.15.90:0/3276592350 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c40a0040 con 0x55e80a73fad0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53b0ff9700 1 -- 172.21.15.90:0/3276592350 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f53b8022430 con 0x55e80a73fad0 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 10 client.15345 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:12:28.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 20 client.15345 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a7c39b0 0x55e80a7c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 4 client.15345 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "status"}) -- 0x55e80a694170 con 0x55e80a7c39b0 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.806+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a7c39b0 0x55e80a7c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53cd0cd700 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a7c39b0 0x55e80a7c26c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379c70a4a0dd7964 server_cookie=93ae0f60c2317811 in_seq=0 out_seq=0 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53b0ff9700 10 client.15345 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53b0ff9700 1 -- 172.21.15.90:0/3276592350 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e80a694170 con 0x55e80a7c39b0 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53b0ff9700 10 client.15345 handle_command_reply: tid=0 2024-04-26T18:12:28.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 client.15345 shutdown 2024-04-26T18:12:28.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:28 smithi119 ceph-mon[32752]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-26T18:12:28.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73fad0 msgr2=0x55e80a74f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73fad0 0x55e80a74f350 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f53b800b5d0 tx=0x7f53b8004750 comp rx=0 tx=0).stop 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 shutdown_connections 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e80a7c39b0 0x55e80a7c26c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e80a7403f0 0x55e80a74f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53b0ff9700 0 client.15345 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e80a73a2d0 0x55e80a75e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 --2- 172.21.15.90:0/3276592350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e80a73fad0 0x55e80a74f350 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 >> 172.21.15.90:0/3276592350 conn(0x55e80a666cf0 msgr2=0x55e80a740870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 shutdown_connections 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 1 -- 172.21.15.90:0/3276592350 wait complete. 2024-04-26T18:12:28.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.807+0000 7f53d9bc3b80 20 client.15345 trim_cache size 0 max 16384 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "cluster_fsid": "b8878a18-03f6-11ef-bc93-c7b262605968", 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "whoami": 0, 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "id": 24449, 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "want_state": "up:active", 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "state": "up:active", 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "fs_name": "cephfs", 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "rank_uptime": 219.44805212399999, 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "mdsmap_epoch": 26, 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch": 93, 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "osdmap_epoch_barrier": 92, 2024-04-26T18:12:28.815 INFO:teuthology.orchestra.run.smithi090.stdout: "uptime": 234.15498948499999 2024-04-26T18:12:28.816 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:12:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f53b40a3ed0 msgr2=0x7f53b40a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f53b40a3ed0 0x7f53b40a6390 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005950 tx=0x7f53b80058e0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8102320 msgr2=0x7f53c819a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8102320 0x7f53c819a9b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f53c400b070 tx=0x7f53c4004db0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 shutdown_connections 2024-04-26T18:12:28.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f53b40a3ed0 0x7f53b40a6390 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53c8102320 0x7f53c819a9b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53c8108390 0x7f53c81a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 --2- 172.21.15.90:0/2700095235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c8102ce0 0x7f53c819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 >> 172.21.15.90:0/2700095235 conn(0x7f53c80792d0 msgr2=0x7f53c8101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 shutdown_connections 2024-04-26T18:12:28.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:28.808+0000 7f53cf331700 1 -- 172.21.15.90:0/2700095235 wait complete. 2024-04-26T18:12:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:28 smithi167 ceph-mon[32751]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-26T18:12:29.607 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-26T18:12:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:30 smithi119 ceph-mon[32752]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:12:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:30 smithi167 ceph-mon[32751]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:12:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:30 smithi090 ceph-mon[29180]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.2 KiB/s wr, 10 op/s 2024-04-26T18:12:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:32 smithi119 ceph-mon[32752]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T18:12:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:32 smithi167 ceph-mon[32751]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T18:12:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:32 smithi090 ceph-mon[29180]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-26T18:12:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:34 smithi119 ceph-mon[32752]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-26T18:12:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:34 smithi167 ceph-mon[32751]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-26T18:12:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:34 smithi090 ceph-mon[29180]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-04-26T18:12:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:36 smithi119 ceph-mon[32752]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:36 smithi167 ceph-mon[32751]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:36 smithi090 ceph-mon[29180]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2024-04-26T18:12:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:38 smithi119 ceph-mon[32752]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-26T18:12:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:38 smithi167 ceph-mon[32751]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-26T18:12:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 -- 172.21.15.90:0/97477080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f0106dd0 msgr2=0x7f84f0107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 --2- 172.21.15.90:0/97477080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f0106dd0 0x7f84f0107280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f84e8009cd0 tx=0x7f84e802a970 comp rx=0 tx=0).stop 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 -- 172.21.15.90:0/97477080 shutdown_connections 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 --2- 172.21.15.90:0/97477080 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f0106dd0 0x7f84f0107280 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 --2- 172.21.15.90:0/97477080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f01022f0 0x7f84f01067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 --2- 172.21.15.90:0/97477080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84f01019d0 0x7f84f0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 -- 172.21.15.90:0/97477080 >> 172.21.15.90:0/97477080 conn(0x7f84f0077bb0 msgr2=0x7f84f0077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 -- 172.21.15.90:0/97477080 shutdown_connections 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.845+0000 7f84f7333640 1 -- 172.21.15.90:0/97477080 wait complete. 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 Processor -- start 2024-04-26T18:12:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 -- start start 2024-04-26T18:12:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f01019d0 0x7f84f019f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84f01022f0 0x7f84f019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0106dd0 0x7f84f01a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01191d0 con 0x7f84f01019d0 2024-04-26T18:12:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0119050 con 0x7f84f0106dd0 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f7333640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0119350 con 0x7f84f01022f0 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f01019d0 0x7f84f019f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f01019d0 0x7f84f019f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38328/0 (socket says 172.21.15.90:38328) 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 -- 172.21.15.90:0/342919068 learned_addr learned my addr 172.21.15.90:0/342919068 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f48ce640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84f01022f0 0x7f84f019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f58d0640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0106dd0 0x7f84f01a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 -- 172.21.15.90:0/342919068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84f01022f0 msgr2=0x7f84f019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84f01022f0 0x7f84f019fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 -- 172.21.15.90:0/342919068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0106dd0 msgr2=0x7f84f01a3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0106dd0 0x7f84f01a3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.846+0000 7f84f50cf640 1 -- 172.21.15.90:0/342919068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f01a45f0 con 0x7f84f01019d0 2024-04-26T18:12:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.847+0000 7f84f50cf640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f01019d0 0x7f84f019f650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00d770 tx=0x7f84ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.847+0000 7f84e67fc640 1 -- 172.21.15.90:0/342919068 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec017070 con 0x7f84f01019d0 2024-04-26T18:12:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.847+0000 7f84e67fc640 1 -- 172.21.15.90:0/342919068 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec0054d0 con 0x7f84f01019d0 2024-04-26T18:12:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.847+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f01a48e0 con 0x7f84f01019d0 2024-04-26T18:12:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.847+0000 7f84e67fc640 1 -- 172.21.15.90:0/342919068 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec004c30 con 0x7f84f01019d0 2024-04-26T18:12:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.847+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f01ac170 con 0x7f84f01019d0 2024-04-26T18:12:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.849+0000 7f84e67fc640 1 -- 172.21.15.90:0/342919068 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f84ec004e10 con 0x7f84f01019d0 2024-04-26T18:12:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 Processor -- start 2024-04-26T18:12:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 -- start start 2024-04-26T18:12:38.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9bd82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9be2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9be30e0 0x55b2d9be54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b2d9b3e460 con 0x55b2d9bd7f00 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b2d9af8590 con 0x55b2d9bd8820 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84fa04c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b2d9b3a420 con 0x55b2d9be30e0 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.850+0000 7f84e67fc640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f84dc0a3ee0 0x7f84dc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f58d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9bd82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f48ce640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9be30e0 0x55b2d9be54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f58d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9bd82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38336/0 (socket says 172.21.15.90:38336) 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f58d0640 1 -- 172.21.15.90:0/1489036637 learned_addr learned my addr 172.21.15.90:0/1489036637 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f58d0640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f84dc0a3ee0 0x7f84dc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f50cf640 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9be2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84e67fc640 1 -- 172.21.15.90:0/342919068 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f84ec0d0510 con 0x7f84f01019d0 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f50cf640 1 -- 172.21.15.90:0/1489036637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9be30e0 msgr2=0x55b2d9be54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f50cf640 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9be30e0 0x55b2d9be54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f50cf640 1 -- 172.21.15.90:0/1489036637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 msgr2=0x55b2d9bd82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f50cf640 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9bd82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f50cf640 1 -- 172.21.15.90:0/1489036637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d9b050d0 con 0x55b2d9bd8820 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f48ce640 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9be30e0 0x55b2d9be54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.851+0000 7f84f58d0640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f84dc0a3ee0 0x7f84dc0a63a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005e30 tx=0x7f84e8009960 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.852+0000 7f84f50cf640 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9be2ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0cdb50 tx=0x7f84ec016310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec4cd6d72029e3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.852+0000 7f84db7fe640 1 -- 172.21.15.90:0/1489036637 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec016170 con 0x55b2d9bd8820 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.852+0000 7f84db7fe640 1 -- 172.21.15.90:0/1489036637 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec0610b0 con 0x55b2d9bd8820 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.852+0000 7f84db7fe640 1 -- 172.21.15.90:0/1489036637 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec098940 con 0x55b2d9bd8820 2024-04-26T18:12:38.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 -- 172.21.15.90:0/1489036637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 msgr2=0x55b2d9be2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9be2ba0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0cdb50 tx=0x7f84ec016310 comp rx=0 tx=0).stop 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 -- 172.21.15.90:0/1489036637 shutdown_connections 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9be30e0 0x55b2d9be54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9be2ba0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 --2- 172.21.15.90:0/1489036637 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9bd82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 -- 172.21.15.90:0/1489036637 >> 172.21.15.90:0/1489036637 conn(0x55b2d9bd4410 msgr2=0x55b2d9bd47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 -- 172.21.15.90:0/1489036637 shutdown_connections 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 -- 172.21.15.90:0/1489036637 wait complete. 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 -- start start 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.853+0000 7f84fa04c740 1 Processor -- start 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9be0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9be05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9be0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84fa04c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b2d9be5d10 con 0x55b2d9bd7f00 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84fa04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b2d9ac5b60 con 0x55b2d9be30e0 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84fa04c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b2d9be5e90 con 0x55b2d9bd8820 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84fa04c740 10 client.0 _ll_register_callbacks cb 0x55b2d9aea2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9be0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9be0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38350/0 (socket says 172.21.15.90:38350) 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 -- 172.21.15.90:0/3278163653 learned_addr learned my addr 172.21.15.90:0/3278163653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f58d0640 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9be0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f48ce640 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9be05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 -- 172.21.15.90:0/3278163653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 msgr2=0x55b2d9be05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9be05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 -- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 msgr2=0x55b2d9be0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9be0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.854+0000 7f84f50cf640 1 -- 172.21.15.90:0/3278163653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d9850210 con 0x55b2d9bd7f00 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84f50cf640 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9be0080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84ec098730 tx=0x7f84ec0a4f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e817bdb93f8ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3278163653 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec020050 con 0x55b2d9bd7f00 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3278163653 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f84ec0a4ab0 con 0x55b2d9bd7f00 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ec0610b0 con 0x55b2d9bd7f00 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3278163653 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84ec0b2e60 con 0x55b2d9bd7f00 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 10 client.15357 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2d991c620 con 0x55b2d9bd7f00 2024-04-26T18:12:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3278163653 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec0610b0 con 0x55b2d9bd7f00 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3278163653 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f84ec0d00d0 con 0x55b2d9bd7f00 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 10 client.15357 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 20 client.15357 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9bf30d0 0x55b2d9bf54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 4 client.15357 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.855+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2d9b4c070 con 0x55b2d9bf30d0 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.856+0000 7f84f48ce640 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9bf30d0 0x55b2d9bf54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.856+0000 7f84d8ff9640 10 client.15357 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.856+0000 7f84f48ce640 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9bf30d0 0x55b2d9bf54c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d026455bc0918698 server_cookie=c15087fc96e0db45 in_seq=0 out_seq=0 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3278163653 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2d9b4c070 con 0x55b2d9bf30d0 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84d8ff9640 10 client.15357 handle_command_reply: tid=0 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 client.15357 shutdown 2024-04-26T18:12:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 msgr2=0x55b2d9be0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9be0080 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f84ec098730 tx=0x7f84ec0a4f60 comp rx=0 tx=0).stop 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 shutdown_connections 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9bf30d0 0x55b2d9bf54c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9be0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9be05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3278163653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9be0080 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 >> 172.21.15.90:0/3278163653 conn(0x55b2d9bd4410 msgr2=0x55b2d9bda190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.857+0000 7f84d8ff9640 0 client.15357 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.858+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 shutdown_connections 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.858+0000 7f84fa04c740 1 -- 172.21.15.90:0/3278163653 wait complete. 2024-04-26T18:12:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.858+0000 7f84fa04c740 20 client.15357 trim_cache size 0 max 16384 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 Processor -- start 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 -- start start 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9bda910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9bd3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01b7240 con 0x55b2d9bd7f00 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01b4b10 con 0x55b2d9be30e0 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84fa04c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01b2960 con 0x55b2d9bd8820 2024-04-26T18:12:38.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84f50cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84f50cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38352/0 (socket says 172.21.15.90:38352) 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84f50cf640 1 -- 172.21.15.90:0/2503640032 learned_addr learned my addr 172.21.15.90:0/2503640032 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84f58d0640 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9bd3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.873+0000 7f84f48ce640 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9bda910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f50cf640 1 -- 172.21.15.90:0/2503640032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 msgr2=0x55b2d9bda910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f50cf640 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9bda910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f50cf640 1 -- 172.21.15.90:0/2503640032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 msgr2=0x55b2d9bd3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f50cf640 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9bd3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f50cf640 1 -- 172.21.15.90:0/2503640032 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d9bdae50 con 0x55b2d9bd7f00 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f48ce640 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9bda910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f58d0640 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9bd3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84f50cf640 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beebd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0ce250 tx=0x7f84ec009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d5ada0319e0d71 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:38.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84daffd640 1 -- 172.21.15.90:0/2503640032 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec020030 con 0x55b2d9bd7f00 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84daffd640 1 -- 172.21.15.90:0/2503640032 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec0610b0 con 0x55b2d9bd7f00 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.874+0000 7f84daffd640 1 -- 172.21.15.90:0/2503640032 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0d00d0 con 0x55b2d9bd7f00 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 -- 172.21.15.90:0/2503640032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 msgr2=0x55b2d9beebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beebd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0ce250 tx=0x7f84ec009e90 comp rx=0 tx=0).stop 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 -- 172.21.15.90:0/2503640032 shutdown_connections 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9be30e0 0x55b2d9bd3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd8820 0x55b2d9bda910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 --2- 172.21.15.90:0/2503640032 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beebd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 -- 172.21.15.90:0/2503640032 >> 172.21.15.90:0/2503640032 conn(0x55b2d9bd4410 msgr2=0x55b2d9bd2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 -- 172.21.15.90:0/2503640032 shutdown_connections 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 -- 172.21.15.90:0/2503640032 wait complete. 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 -- start start 2024-04-26T18:12:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.875+0000 7f84fa04c740 1 Processor -- start 2024-04-26T18:12:38.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:38 smithi090 ceph-mon[29180]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-26T18:12:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9bdc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84fa04c740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9be30e0 0x55b2d9bdce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84fa04c740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01b2fd0 con 0x55b2d9be30e0 2024-04-26T18:12:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84fa04c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01b4440 con 0x55b2d9bd8820 2024-04-26T18:12:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84fa04c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84f01b36a0 con 0x55b2d9bd7f00 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84fa04c740 10 client.0 _ll_register_callbacks cb 0x7f84e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9be30e0 0x55b2d9bdce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9be30e0 0x55b2d9bdce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38368/0 (socket says 172.21.15.90:38368) 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 -- 172.21.15.90:0/3186200182 learned_addr learned my addr 172.21.15.90:0/3186200182 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f48ce640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9bdc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f50cf640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 -- 172.21.15.90:0/3186200182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd7f00 msgr2=0x55b2d9beb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 -- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 msgr2=0x55b2d9bdc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9bdc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 -- 172.21.15.90:0/3186200182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d9bdd3b0 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f50cf640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84f58d0640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9be30e0 0x55b2d9bdce70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84e802ffe0 tx=0x7f84e8048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e85b8243ff4bfc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.876+0000 7f84d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3186200182 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8034070 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3186200182 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f84e8054480 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ec0610b0 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3186200182 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f84e8062050 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 10 client.15363 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b2d9c590b0 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3186200182 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ec0610b0 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3186200182 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f84e80394b0 con 0x55b2d9be30e0 2024-04-26T18:12:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 10 client.15363 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 20 client.15363 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9c598b0 0x55b2d9c5bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 4 client.15363 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.877+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2d9b45ee0 con 0x55b2d9c598b0 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.878+0000 7f84f50cf640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9c598b0 0x55b2d9c5bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.878+0000 7f84f50cf640 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9c598b0 0x55b2d9c5bcd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23aa21a85e675a71 server_cookie=93c897715d7d3e71 in_seq=0 out_seq=0 2024-04-26T18:12:38.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.878+0000 7f84d8ff9640 10 client.15363 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.884+0000 7f84d8ff9640 1 -- 172.21.15.90:0/3186200182 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2d9b45ee0 con 0x55b2d9c598b0 2024-04-26T18:12:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.884+0000 7f84d8ff9640 10 client.15363 handle_command_reply: tid=0 2024-04-26T18:12:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.884+0000 7f84fa04c740 1 client.15363 shutdown 2024-04-26T18:12:38.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.884+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9be30e0 msgr2=0x55b2d9bdce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.884+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9be30e0 0x55b2d9bdce70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f84e802ffe0 tx=0x7f84e8048000 comp rx=0 tx=0).stop 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.884+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 shutdown_connections 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.884+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b2d9c598b0 0x55b2d9c5bcd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b2d9be30e0 0x55b2d9bdce70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b2d9bd8820 0x55b2d9bdc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84d8ff9640 0 client.15363 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84fa04c740 1 --2- 172.21.15.90:0/3186200182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2d9bd7f00 0x55b2d9beb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 >> 172.21.15.90:0/3186200182 conn(0x55b2d9bd4410 msgr2=0x55b2d9be08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 shutdown_connections 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84fa04c740 1 -- 172.21.15.90:0/3186200182 wait complete. 2024-04-26T18:12:38.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84fa04c740 20 client.15363 trim_cache size 0 max 16384 2024-04-26T18:12:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:12:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:12:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:12:38.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f84dc0a3ee0 msgr2=0x7f84dc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f84dc0a3ee0 0x7f84dc0a63a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f84e8005e30 tx=0x7f84e8009960 comp rx=0 tx=0).stop 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f01019d0 msgr2=0x7f84f019f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f01019d0 0x7f84f019f650 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f84ec00d770 tx=0x7f84ec00dc30 comp rx=0 tx=0).stop 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 shutdown_connections 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f84dc0a3ee0 0x7f84dc0a63a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84f0106dd0 0x7f84f01a3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f84f01022f0 0x7f84f019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 --2- 172.21.15.90:0/342919068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f84f01019d0 0x7f84f019f650 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.885+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 >> 172.21.15.90:0/342919068 conn(0x7f84f0077bb0 msgr2=0x7f84f0104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.886+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 shutdown_connections 2024-04-26T18:12:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:38.886+0000 7f84f7333640 1 -- 172.21.15.90:0/342919068 wait complete. 2024-04-26T18:12:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:40 smithi119 ceph-mon[32752]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T18:12:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:40 smithi167 ceph-mon[32751]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T18:12:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:40 smithi090 ceph-mon[29180]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-26T18:12:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:42 smithi119 ceph-mon[32752]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:12:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:42 smithi167 ceph-mon[32751]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:12:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:42 smithi090 ceph-mon[29180]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:12:44.608 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T18:12:44.609 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T18:12:44.609 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:12:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:44 smithi119 ceph-mon[32752]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T18:12:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:44 smithi167 ceph-mon[32751]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T18:12:44.874 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:44 smithi090 ceph-mon[29180]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T18:12:46.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 -- 172.21.15.90:0/988730688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290108390 msgr2=0x7f5290108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:46.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 --2- 172.21.15.90:0/988730688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290108390 0x7f5290108770 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f528c009990 tx=0x7f528c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 -- 172.21.15.90:0/988730688 shutdown_connections 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 --2- 172.21.15.90:0/988730688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290108390 0x7f5290108770 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 --2- 172.21.15.90:0/988730688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5290102cf0 0x7f529010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 --2- 172.21.15.90:0/988730688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5290102350 0x7f52901027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 -- 172.21.15.90:0/988730688 >> 172.21.15.90:0/988730688 conn(0x7f52900792c0 msgr2=0x7f52900796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 -- 172.21.15.90:0/988730688 shutdown_connections 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 -- 172.21.15.90:0/988730688 wait complete. 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 Processor -- start 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.526+0000 7f5298639700 1 -- start start 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5290102350 0x7f5290104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290102cf0 0x7f5290104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5290108390 0x7f5290104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:46.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5290112500 con 0x7f5290102cf0 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5290112380 con 0x7f5290102350 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5290112680 con 0x7f5290108390 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290102cf0 0x7f5290104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290102cf0 0x7f5290104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43192/0 (socket says 172.21.15.90:43192) 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 -- 172.21.15.90:0/2458578477 learned_addr learned my addr 172.21.15.90:0/2458578477 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f52963d5700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5290102350 0x7f5290104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5296bd6700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5290108390 0x7f5290104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 -- 172.21.15.90:0/2458578477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5290108390 msgr2=0x7f5290104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5290108390 0x7f5290104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 -- 172.21.15.90:0/2458578477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5290102350 msgr2=0x7f5290104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5290102350 0x7f5290104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 -- 172.21.15.90:0/2458578477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52901a80d0 con 0x7f5290102cf0 2024-04-26T18:12:46.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5296bd6700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5290108390 0x7f5290104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5295bd4700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290102cf0 0x7f5290104540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f528000c920 tx=0x7f528000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f52877fe700 1 -- 172.21.15.90:0/2458578477 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280007b00 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f52877fe700 1 -- 172.21.15.90:0/2458578477 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5280004ce0 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52901a83c0 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f52877fe700 1 -- 172.21.15.90:0/2458578477 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280005550 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.527+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52901a8900 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.528+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5290103a40 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.530+0000 7f52877fe700 1 -- 172.21.15.90:0/2458578477 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f5280007ca0 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.530+0000 7f52877fe700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f527c0a3ed0 0x7f527c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.531+0000 7f52963d5700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f527c0a3ed0 0x7f527c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.531+0000 7f52877fe700 1 -- 172.21.15.90:0/2458578477 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f52800d0990 con 0x7f5290102cf0 2024-04-26T18:12:46.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.531+0000 7f52963d5700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f527c0a3ed0 0x7f527c0a6390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f528c00b5c0 tx=0x7f528c005960 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:46.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:46 smithi090 ceph-mon[29180]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T18:12:46.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.534+0000 7f52877fe700 1 -- 172.21.15.90:0/2458578477 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5280014030 con 0x7f5290102cf0 2024-04-26T18:12:46.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.783+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f529006a110 con 0x7f5290102cf0 2024-04-26T18:12:46.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.784+0000 7f52877fe700 1 -- 172.21.15.90:0/2458578477 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11874 (secure 0 0 0) 0x7f528009ce50 con 0x7f5290102cf0 2024-04-26T18:12:46.784 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:12:46.784 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:11:56.125308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:12:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f527c0a3ed0 msgr2=0x7f527c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:46.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f527c0a3ed0 0x7f527c0a6390 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f528c00b5c0 tx=0x7f528c005960 comp rx=0 tx=0).stop 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290102cf0 msgr2=0x7f5290104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290102cf0 0x7f5290104540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f528000c920 tx=0x7f528000cc30 comp rx=0 tx=0).stop 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 shutdown_connections 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f527c0a3ed0 0x7f527c0a6390 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5290108390 0x7f5290104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5290102350 0x7f5290104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 --2- 172.21.15.90:0/2458578477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5290102cf0 0x7f5290104540 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 >> 172.21.15.90:0/2458578477 conn(0x7f52900792c0 msgr2=0x7f529006bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 shutdown_connections 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:46.786+0000 7f5298639700 1 -- 172.21.15.90:0/2458578477 wait complete. 2024-04-26T18:12:46.787 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T18:12:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:46 smithi119 ceph-mon[32752]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T18:12:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:46 smithi167 ceph-mon[32751]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-26T18:12:47.511 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T18:12:47.511 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T18:12:47.751 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:12:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:47 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2458578477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:47 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2458578477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:47.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:47 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2458578477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:12:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:48 smithi119 ceph-mon[32752]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T18:12:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:48 smithi167 ceph-mon[32751]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T18:12:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:48 smithi090 ceph-mon[29180]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 -- 172.21.15.90:0/3210536932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f81034d0 msgr2=0x7f97f8103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 --2- 172.21.15.90:0/3210536932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f81034d0 0x7f97f8103950 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f97e40099c0 tx=0x7f97e4009cd0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 -- 172.21.15.90:0/3210536932 shutdown_connections 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 --2- 172.21.15.90:0/3210536932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97f8103e90 0x7f97f81102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 --2- 172.21.15.90:0/3210536932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f81094a0 0x7f97f8109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 --2- 172.21.15.90:0/3210536932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f81034d0 0x7f97f8103950 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 -- 172.21.15.90:0/3210536932 >> 172.21.15.90:0/3210536932 conn(0x7f97f807a4f0 msgr2=0x7f97f807a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.516+0000 7f97fec36700 1 -- 172.21.15.90:0/3210536932 shutdown_connections 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 -- 172.21.15.90:0/3210536932 wait complete. 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 Processor -- start 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 -- start start 2024-04-26T18:12:49.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f81034d0 0x7f97f8075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f8103e90 0x7f97f8075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97f81094a0 0x7f97f80761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97f8113650 con 0x7f97f8103e90 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81134d0 con 0x7f97f81094a0 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.517+0000 7f97fec36700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81137d0 con 0x7f97f81034d0 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f8103e90 0x7f97f8075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f8103e90 0x7f97f8075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43218/0 (socket says 172.21.15.90:43218) 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 -- 172.21.15.90:0/4036996160 learned_addr learned my addr 172.21.15.90:0/4036996160 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97f81094a0 0x7f97f80761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f81034d0 0x7f97f8075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 -- 172.21.15.90:0/4036996160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f81034d0 msgr2=0x7f97f8075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f81034d0 0x7f97f8075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 -- 172.21.15.90:0/4036996160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97f81094a0 msgr2=0x7f97f80761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97f81094a0 0x7f97f80761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 -- 172.21.15.90:0/4036996160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f81a7ff0 con 0x7f97f8103e90 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f81034d0 0x7f97f8075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f7fff700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f8103e90 0x7f97f8075c60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f97e40098e0 tx=0x7f97e40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f5ffb700 1 -- 172.21.15.90:0/4036996160 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4027070 con 0x7f97f8103e90 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f5ffb700 1 -- 172.21.15.90:0/4036996160 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e4003f20 con 0x7f97f8103e90 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f81a8280 con 0x7f97f8103e90 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.518+0000 7f97f5ffb700 1 -- 172.21.15.90:0/4036996160 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4022da0 con 0x7f97f8103e90 2024-04-26T18:12:49.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.519+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f81a86d0 con 0x7f97f8103e90 2024-04-26T18:12:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.521+0000 7f97f5ffb700 1 -- 172.21.15.90:0/4036996160 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f97e402a070 con 0x7f97f8103e90 2024-04-26T18:12:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.522+0000 7f97f5ffb700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f97e80a3f70 0x7f97e80a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.522+0000 7f97f5ffb700 1 -- 172.21.15.90:0/4036996160 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f97e40d1740 con 0x7f97f8103e90 2024-04-26T18:12:49.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.522+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f97e80a3f70 0x7f97e80a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.522+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f97e80a3f70 0x7f97e80a6430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0099d0 tx=0x7f97ec008040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 Processor -- start 2024-04-26T18:12:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 -- start start 2024-04-26T18:12:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677a230 0x55649677a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x5564967854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x556496787df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564966cd8d0 con 0x55649677a230 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55649666c770 con 0x55649677ab50 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f98094c8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5564966c9890 con 0x556496785a00 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677a230 0x55649677a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677a230 0x55649677a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43226/0 (socket says 172.21.15.90:43226) 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 -- 172.21.15.90:0/2440492100 learned_addr learned my addr 172.21.15.90:0/2440492100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97f7fff700 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x5564967854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x556496787df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 -- 172.21.15.90:0/2440492100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 msgr2=0x556496787df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x556496787df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 -- 172.21.15.90:0/2440492100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 msgr2=0x5564967854c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x5564967854c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fc9d2700 1 -- 172.21.15.90:0/2440492100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55649661aa60 con 0x55649677a230 2024-04-26T18:12:49.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.523+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x556496787df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:49.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.524+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677a230 0x55649677a610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00d7c0 tx=0x7f97ec010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67e5a82a8da9f60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:49.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.524+0000 7f97e2ffd700 1 -- 172.21.15.90:0/2440492100 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec024070 con 0x55649677a230 2024-04-26T18:12:49.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.524+0000 7f97e2ffd700 1 -- 172.21.15.90:0/2440492100 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ec014af0 con 0x55649677a230 2024-04-26T18:12:49.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.524+0000 7f97e2ffd700 1 -- 172.21.15.90:0/2440492100 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec01fbf0 con 0x55649677a230 2024-04-26T18:12:49.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.524+0000 7f98094c8b80 1 -- 172.21.15.90:0/2440492100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677a230 msgr2=0x55649677a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.524+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677a230 0x55649677a610 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00d7c0 tx=0x7f97ec010000 comp rx=0 tx=0).stop 2024-04-26T18:12:49.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- 172.21.15.90:0/2440492100 shutdown_connections 2024-04-26T18:12:49.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677a230 0x55649677a610 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x556496787df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2440492100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x5564967854c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- 172.21.15.90:0/2440492100 >> 172.21.15.90:0/2440492100 conn(0x5564966a7ed0 msgr2=0x556496776720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- 172.21.15.90:0/2440492100 shutdown_connections 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- 172.21.15.90:0/2440492100 wait complete. 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- start start 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 Processor -- start 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677a230 0x556496785180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677ab50 0x556496782480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x5564967829c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5564967885d0 con 0x55649677ab50 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556496633ae0 con 0x55649677a230 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.525+0000 7f98094c8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556496788750 con 0x556496785a00 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f98094c8b80 10 client.0 _ll_register_callbacks cb 0x556496661710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677ab50 0x556496782480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.531 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677ab50 0x556496782480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43238/0 (socket says 172.21.15.90:43238) 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 -- 172.21.15.90:0/2099571050 learned_addr learned my addr 172.21.15.90:0/2099571050 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677a230 0x556496785180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 -- 172.21.15.90:0/2099571050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 msgr2=0x5564967829c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x5564967829c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 -- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677a230 msgr2=0x556496785180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677a230 0x556496785180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 -- 172.21.15.90:0/2099571050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556496783070 con 0x55649677ab50 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97f7fff700 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677ab50 0x556496782480 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c010 tx=0x7f97e409d200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c1c92b4bf7ca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97d7fff700 1 -- 172.21.15.90:0/2099571050 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e409dc90 con 0x55649677ab50 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97d7fff700 1 -- 172.21.15.90:0/2099571050 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f97e4005140 con 0x55649677ab50 2024-04-26T18:12:49.532 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ec014af0 con 0x55649677ab50 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f97d7fff700 1 -- 172.21.15.90:0/2099571050 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97e409d590 con 0x55649677ab50 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f98094c8b80 10 client.15381 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.526+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556496792fc0 con 0x55649677ab50 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f97d7fff700 1 -- 172.21.15.90:0/2099571050 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ec014af0 con 0x55649677ab50 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f97d7fff700 1 -- 172.21.15.90:0/2099571050 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f97e40cf060 con 0x55649677ab50 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f98094c8b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f98094c8b80 10 client.15381 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f98094c8b80 20 client.15381 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556496795490 0x556496797880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f98094c8b80 4 client.15381 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564966db2b0 con 0x556496795490 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.527+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556496795490 0x556496797880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556496795490 0x556496797880 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f38ab7766210ee7 server_cookie=5f18801b06a48298 in_seq=0 out_seq=0 2024-04-26T18:12:49.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f97d7fff700 10 client.15381 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f97d7fff700 1 -- 172.21.15.90:0/2099571050 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5564966db2b0 con 0x556496795490 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f97d7fff700 10 client.15381 handle_command_reply: tid=0 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 client.15381 shutdown 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677ab50 msgr2=0x556496782480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677ab50 0x556496782480 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c010 tx=0x7f97e409d200 comp rx=0 tx=0).stop 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 shutdown_connections 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556496795490 0x556496797880 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556496785a00 0x5564967829c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677a230 0x556496785180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 --2- 172.21.15.90:0/2099571050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55649677ab50 0x556496782480 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 >> 172.21.15.90:0/2099571050 conn(0x5564966a7ed0 msgr2=0x556496776350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.528+0000 7f97d7fff700 0 client.15381 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:49.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.529+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 shutdown_connections 2024-04-26T18:12:49.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.529+0000 7f98094c8b80 1 -- 172.21.15.90:0/2099571050 wait complete. 2024-04-26T18:12:49.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.529+0000 7f98094c8b80 20 client.15381 trim_cache size 0 max 16384 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 Processor -- start 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 -- start start 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x5564967768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x556496774e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x556496784cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81b24a0 con 0x556496785a00 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81afd30 con 0x55649677ab50 2024-04-26T18:12:49.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.548+0000 7f98094c8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81adc00 con 0x55649677a230 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x556496784cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x556496784cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43246/0 (socket says 172.21.15.90:43246) 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 -- 172.21.15.90:0/275177934 learned_addr learned my addr 172.21.15.90:0/275177934 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x5564967768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 -- 172.21.15.90:0/275177934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 msgr2=0x5564967768b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x5564967768b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 -- 172.21.15.90:0/275177934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 msgr2=0x556496774e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97f7fff700 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x556496774e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x556496774e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 -- 172.21.15.90:0/275177934 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556496782250 con 0x556496785a00 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x556496784cc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97f0003fc0 tx=0x7f97f0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17731e26afe6843 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97e27fc700 1 -- 172.21.15.90:0/275177934 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0019020 con 0x556496785a00 2024-04-26T18:12:49.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97e27fc700 1 -- 172.21.15.90:0/275177934 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f0007500 con 0x556496785a00 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.549+0000 7f97e27fc700 1 -- 172.21.15.90:0/275177934 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0015650 con 0x556496785a00 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 -- 172.21.15.90:0/275177934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 msgr2=0x556496784cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x556496784cc0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97f0003fc0 tx=0x7f97f0009ba0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 -- 172.21.15.90:0/275177934 shutdown_connections 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x556496784cc0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x5564967768b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 --2- 172.21.15.90:0/275177934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x556496774e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 -- 172.21.15.90:0/275177934 >> 172.21.15.90:0/275177934 conn(0x5564966a7ed0 msgr2=0x5564967761a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 -- 172.21.15.90:0/275177934 shutdown_connections 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 -- 172.21.15.90:0/275177934 wait complete. 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 -- start start 2024-04-26T18:12:49.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.550+0000 7f98094c8b80 1 Processor -- start 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x55649677f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x55649677fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x5564967fa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81ae250 con 0x556496785a00 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81af660 con 0x55649677ab50 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97f81ae8c0 con 0x55649677a230 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 10 client.0 _ll_register_callbacks cb 0x556496661710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x5564967fa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x5564967fa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43256/0 (socket says 172.21.15.90:43256) 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 -- 172.21.15.90:0/1461546652 learned_addr learned my addr 172.21.15.90:0/1461546652 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97f7fff700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x55649677fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x55649677f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 -- 172.21.15.90:0/1461546652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 msgr2=0x55649677f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x55649677f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 -- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 msgr2=0x55649677fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x55649677fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 -- 172.21.15.90:0/1461546652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564967faf80 con 0x556496785a00 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x55649677f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97fd1d3700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x5564967fa9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f97f0004700 tx=0x7f97f0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6421a49029ed05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97d7fff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97d7fff700 1 -- 172.21.15.90:0/1461546652 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007ac0 con 0x556496785a00 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f97d7fff700 1 -- 172.21.15.90:0/1461546652 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f97f0027470 con 0x556496785a00 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.551+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f0007500 con 0x556496785a00 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f97d7fff700 1 -- 172.21.15.90:0/1461546652 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f97f0007c60 con 0x556496785a00 2024-04-26T18:12:49.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 10 client.15387 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5564967fc240 con 0x556496785a00 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f97d7fff700 1 -- 172.21.15.90:0/1461546652 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f0007500 con 0x556496785a00 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f97d7fff700 1 -- 172.21.15.90:0/1461546652 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f97f002f020 con 0x556496785a00 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 10 client.15387 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 20 client.15387 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5564967fc800 0x5564967febf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 4 client.15387 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5564966d5350 con 0x5564967fc800 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.552+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5564967fc800 0x5564967febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.553+0000 7f97d7fff700 10 client.15387 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:49.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.553+0000 7f97fc9d2700 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5564967fc800 0x5564967febf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f5aa93b74f7440 server_cookie=7ec9c085e3d80308 in_seq=0 out_seq=0 2024-04-26T18:12:49.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f97d7fff700 1 -- 172.21.15.90:0/1461546652 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5564966d5350 con 0x5564967fc800 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f97d7fff700 10 client.15387 handle_command_reply: tid=0 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 client.15387 shutdown 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 msgr2=0x5564967fa9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x5564967fa9c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f97f0004700 tx=0x7f97f0004f90 comp rx=0 tx=0).stop 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 shutdown_connections 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5564967fc800 0x5564967febf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556496785a00 0x5564967fa9c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55649677a230 0x55649677f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f97d7fff700 0 client.15387 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 --2- 172.21.15.90:0/1461546652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55649677ab50 0x55649677fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 >> 172.21.15.90:0/1461546652 conn(0x5564966a7ed0 msgr2=0x55649677d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:49.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 shutdown_connections 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 1 -- 172.21.15.90:0/1461546652 wait complete. 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.565+0000 7f98094c8b80 20 client.15387 trim_cache size 0 max 16384 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 26, 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 2, 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stdout: "8454b329": { 2024-04-26T18:12:49.567 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 2, 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 90.0, 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:12:49.568 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:12:49.569 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:12:49.569 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:12:49.569 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:12:49.569 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f97e80a3f70 msgr2=0x7f97e80a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f97e80a3f70 0x7f97e80a6430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0099d0 tx=0x7f97ec008040 comp rx=0 tx=0).stop 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f8103e90 msgr2=0x7f97f8075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f8103e90 0x7f97f8075c60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f97e40098e0 tx=0x7f97e40057c0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 shutdown_connections 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f97e80a3f70 0x7f97e80a6430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97f81094a0 0x7f97f80761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97f81034d0 0x7f97f8075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 --2- 172.21.15.90:0/4036996160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f97f8103e90 0x7f97f8075c60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:49.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 >> 172.21.15.90:0/4036996160 conn(0x7f97f807a4f0 msgr2=0x7f97f8102130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:49.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 shutdown_connections 2024-04-26T18:12:49.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:12:49.566+0000 7f97fec36700 1 -- 172.21.15.90:0/4036996160 wait complete. 2024-04-26T18:12:50.418 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8454b329', quiesce took 5.8 seconds. Will release after: 30.0 2024-04-26T18:12:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:50 smithi119 ceph-mon[32752]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T18:12:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:50 smithi167 ceph-mon[32751]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T18:12:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:50 smithi090 ceph-mon[29180]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-26T18:12:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 -- 172.21.15.90:0/3097852296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c06b160 msgr2=0x7fd96c06b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 --2- 172.21.15.90:0/3097852296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c06b160 0x7fd96c06b5c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009900 tx=0x7fd95c02a410 comp rx=0 tx=0).stop 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 -- 172.21.15.90:0/3097852296 shutdown_connections 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 --2- 172.21.15.90:0/3097852296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c06bb00 0x7fd96c10f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 --2- 172.21.15.90:0/3097852296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c06b160 0x7fd96c06b5c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 --2- 172.21.15.90:0/3097852296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd96c108920 0x7fd96c108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 -- 172.21.15.90:0/3097852296 >> 172.21.15.90:0/3097852296 conn(0x7fd96c077630 msgr2=0x7fd96c077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 -- 172.21.15.90:0/3097852296 shutdown_connections 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.983+0000 7fd9743f0640 1 -- 172.21.15.90:0/3097852296 wait complete. 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 Processor -- start 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 -- start start 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd96c06b160 0x7fd96c101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c06bb00 0x7fd96c102190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c108920 0x7fd96c1026d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c112bc0 con 0x7fd96c108920 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c112a40 con 0x7fd96c06b160 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd9743f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c112d40 con 0x7fd96c06bb00 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd97298d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c108920 0x7fd96c1026d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd97298d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c108920 0x7fd96c1026d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35628/0 (socket says 172.21.15.90:35628) 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd97298d640 1 -- 172.21.15.90:0/1697426719 learned_addr learned my addr 172.21.15.90:0/1697426719 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.984+0000 7fd97218c640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd96c06b160 0x7fd96c101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97198b640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c06bb00 0x7fd96c102190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97298d640 1 -- 172.21.15.90:0/1697426719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c06bb00 msgr2=0x7fd96c102190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97298d640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c06bb00 0x7fd96c102190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97298d640 1 -- 172.21.15.90:0/1697426719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd96c06b160 msgr2=0x7fd96c101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97298d640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd96c06b160 0x7fd96c101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97298d640 1 -- 172.21.15.90:0/1697426719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd96c1a7560 con 0x7fd96c108920 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97218c640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd96c06b160 0x7fd96c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97198b640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c06bb00 0x7fd96c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd97298d640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c108920 0x7fd96c1026d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd964009fd0 tx=0x7fd96400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd9637fe640 1 -- 172.21.15.90:0/1697426719 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964017070 con 0x7fd96c108920 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd9637fe640 1 -- 172.21.15.90:0/1697426719 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd964003e20 con 0x7fd96c108920 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd96c1a7850 con 0x7fd96c108920 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd9637fe640 1 -- 172.21.15.90:0/1697426719 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964010460 con 0x7fd96c108920 2024-04-26T18:12:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.985+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd96c1a7d50 con 0x7fd96c108920 2024-04-26T18:12:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.987+0000 7fd9637fe640 1 -- 172.21.15.90:0/1697426719 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7fd964003970 con 0x7fd96c108920 2024-04-26T18:12:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9771e8740 1 Processor -- start 2024-04-26T18:12:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9771e8740 1 -- start start 2024-04-26T18:12:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9637fe640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd9580a3ee0 0x7fd9580a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e3982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9771e8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd0e2fe460 con 0x55bd0e397f00 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.988+0000 7fd9771e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bd0e2b8590 con 0x55bd0e398820 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd9771e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bd0e2fa420 con 0x55bd0e3a30e0 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97198b640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd9580a3ee0 0x7fd9580a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd9637fe640 1 -- 172.21.15.90:0/1697426719 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd9640d08c0 con 0x7fd96c108920 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e3982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e3982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35634/0 (socket says 172.21.15.90:35634) 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 -- 172.21.15.90:0/2776348681 learned_addr learned my addr 172.21.15.90:0/2776348681 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97198b640 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97298d640 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 -- 172.21.15.90:0/2776348681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 msgr2=0x55bd0e3a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 -- 172.21.15.90:0/2776348681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 msgr2=0x55bd0e3a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 -- 172.21.15.90:0/2776348681 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0e2c50d0 con 0x55bd0e397f00 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97298d640 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97198b640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd9580a3ee0 0x7fd9580a63a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd95c000c00 tx=0x7fd95c032cb0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd97218c640 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e3982e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd968009bd0 tx=0x7fd96800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de471bf123de1ccd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd960bfe640 1 -- 172.21.15.90:0/2776348681 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968016070 con 0x55bd0e397f00 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd960bfe640 1 -- 172.21.15.90:0/2776348681 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9680054d0 con 0x55bd0e397f00 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.989+0000 7fd960bfe640 1 -- 172.21.15.90:0/2776348681 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968011470 con 0x55bd0e397f00 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 -- 172.21.15.90:0/2776348681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 msgr2=0x55bd0e3982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:51.996 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e3982e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd968009bd0 tx=0x7fd96800e6c0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 -- 172.21.15.90:0/2776348681 shutdown_connections 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2776348681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e3982e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 -- 172.21.15.90:0/2776348681 >> 172.21.15.90:0/2776348681 conn(0x55bd0e394410 msgr2=0x55bd0e3947f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 -- 172.21.15.90:0/2776348681 shutdown_connections 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.990+0000 7fd9771e8740 1 -- 172.21.15.90:0/2776348681 wait complete. 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 -- start start 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 Processor -- start 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e397f00 0x55bd0e39ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:51.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd0e3a5d10 con 0x55bd0e3a30e0 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bd0e285b60 con 0x55bd0e398820 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bd0e3a5e90 con 0x55bd0e397f00 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd9771e8740 10 client.0 _ll_register_callbacks cb 0x55bd0e2aa2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35648/0 (socket says 172.21.15.90:35648) 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 -- 172.21.15.90:0/2788895556 learned_addr learned my addr 172.21.15.90:0/2788895556 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97198b640 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97218c640 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e397f00 0x55bd0e39ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 -- 172.21.15.90:0/2788895556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e397f00 msgr2=0x55bd0e39ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e397f00 0x55bd0e39ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 -- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 msgr2=0x55bd0e3a0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:51.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.991+0000 7fd97298d640 1 -- 172.21.15.90:0/2788895556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0e010210 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd97298d640 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a09d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd964098ae0 tx=0x7fd964003da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced99938245156da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd955ffb640 1 -- 172.21.15.90:0/2788895556 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd964020040 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd955ffb640 1 -- 172.21.15.90:0/2788895556 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd9640883e0 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9680054d0 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd955ffb640 1 -- 172.21.15.90:0/2788895556 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9640a52f0 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 10 client.15399 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd0e0dc620 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd955ffb640 1 -- 172.21.15.90:0/2788895556 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9680054d0 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd955ffb640 1 -- 172.21.15.90:0/2788895556 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd9640ce1e0 con 0x55bd0e3a30e0 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 10 client.15399 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:51.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 10 client.15399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 20 client.15399 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e3b3080 0x55bd0e3b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 4 client.15399 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.992+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd0e30c070 con 0x55bd0e3b3080 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.993+0000 7fd97218c640 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e3b3080 0x55bd0e3b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.993+0000 7fd97218c640 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e3b3080 0x55bd0e3b54d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ef6c6a87b29c0b server_cookie=b7c0a9bd348fd2f4 in_seq=0 out_seq=0 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.993+0000 7fd955ffb640 10 client.15399 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd955ffb640 1 -- 172.21.15.90:0/2788895556 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd0e30c070 con 0x55bd0e3b3080 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd955ffb640 10 client.15399 handle_command_reply: tid=0 2024-04-26T18:12:52.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 client.15399 shutdown 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e3a30e0 msgr2=0x55bd0e3a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a09d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd964098ae0 tx=0x7fd964003da0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 shutdown_connections 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e3b3080 0x55bd0e3b54d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3a09d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd955ffb640 0 client.15399 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3a0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 --2- 172.21.15.90:0/2788895556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e397f00 0x55bd0e39ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 >> 172.21.15.90:0/2788895556 conn(0x55bd0e394410 msgr2=0x55bd0e39a190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 shutdown_connections 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 1 -- 172.21.15.90:0/2788895556 wait complete. 2024-04-26T18:12:52.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:51.994+0000 7fd9771e8740 20 client.15399 trim_cache size 0 max 16384 2024-04-26T18:12:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 Processor -- start 2024-04-26T18:12:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 -- start start 2024-04-26T18:12:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e39a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3bb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1b2e80 con 0x55bd0e397f00 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1b0750 con 0x55bd0e398820 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd9771e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1ae5c0 con 0x55bd0e3a30e0 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e39a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e39a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35664/0 (socket says 172.21.15.90:35664) 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 -- 172.21.15.90:0/4016737338 learned_addr learned my addr 172.21.15.90:0/4016737338 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97198b640 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 -- 172.21.15.90:0/4016737338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 msgr2=0x55bd0e3bb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3bb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 -- 172.21.15.90:0/4016737338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 msgr2=0x55bd0e3bade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3bade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 -- 172.21.15.90:0/4016737338 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd0e39e4a0 con 0x55bd0e397f00 2024-04-26T18:12:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd97218c640 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e39a8c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd96800f010 tx=0x7fd968009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5fd1d0c6115bab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd957fff640 1 -- 172.21.15.90:0/4016737338 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968016040 con 0x55bd0e397f00 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd957fff640 1 -- 172.21.15.90:0/4016737338 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd968004880 con 0x55bd0e397f00 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.009+0000 7fd957fff640 1 -- 172.21.15.90:0/4016737338 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd968003cc0 con 0x55bd0e397f00 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 -- 172.21.15.90:0/4016737338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 msgr2=0x55bd0e39a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e39a8c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fd96800f010 tx=0x7fd968009e50 comp rx=0 tx=0).stop 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 -- 172.21.15.90:0/4016737338 shutdown_connections 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e3bb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e398820 0x55bd0e3bade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 --2- 172.21.15.90:0/4016737338 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e397f00 0x55bd0e39a8c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 -- 172.21.15.90:0/4016737338 >> 172.21.15.90:0/4016737338 conn(0x55bd0e394410 msgr2=0x55bd0e3935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 -- 172.21.15.90:0/4016737338 shutdown_connections 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 -- 172.21.15.90:0/4016737338 wait complete. 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 -- start start 2024-04-26T18:12:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.010+0000 7fd9771e8740 1 Processor -- start 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e397f00 0x55bd0e39cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e398820 0x55bd0e39d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e418650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1aec10 con 0x55bd0e398820 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1b0080 con 0x55bd0e397f00 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd96c1af2e0 con 0x55bd0e3a30e0 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 10 client.0 _ll_register_callbacks cb 0x7fd964098260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e398820 0x55bd0e39d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e398820 0x55bd0e39d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35672/0 (socket says 172.21.15.90:35672) 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 -- 172.21.15.90:0/350196242 learned_addr learned my addr 172.21.15.90:0/350196242 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97218c640 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e397f00 0x55bd0e39cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:52.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97298d640 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e418650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 -- 172.21.15.90:0/350196242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 msgr2=0x55bd0e418650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e418650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 -- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e397f00 msgr2=0x55bd0e39cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e397f00 0x55bd0e39cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 -- 172.21.15.90:0/350196242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd0e39dcf0 con 0x55bd0e398820 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd97198b640 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e398820 0x55bd0e39d460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009900 tx=0x7fd95c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812005e536483330 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd955ffb640 1 -- 172.21.15.90:0/350196242 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c035050 con 0x55bd0e398820 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd968004880 con 0x55bd0e398820 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.011+0000 7fd955ffb640 1 -- 172.21.15.90:0/350196242 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd95c005040 con 0x55bd0e398820 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd955ffb640 1 -- 172.21.15.90:0/350196242 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd95c059050 con 0x55bd0e398820 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 10 client.15405 fetch_fsmap learned FSMap version 26 2024-04-26T18:12:52.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bd0e419c70 con 0x55bd0e398820 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd955ffb640 1 -- 172.21.15.90:0/350196242 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd968004880 con 0x55bd0e398820 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd955ffb640 1 -- 172.21.15.90:0/350196242 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd95c049040 con 0x55bd0e398820 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 10 client.15405 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 20 client.15405 populate_metadata read hostname 'smithi090' 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e41a410 0x55bd0e41c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 4 client.15405 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd0e305ee0 con 0x55bd0e41a410 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.012+0000 7fd97218c640 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e41a410 0x55bd0e41c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.013+0000 7fd97218c640 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e41a410 0x55bd0e41c830 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12afe23de79d74be server_cookie=8f1ec01f1b35c44f in_seq=0 out_seq=0 2024-04-26T18:12:52.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.013+0000 7fd955ffb640 10 client.15405 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.019+0000 7fd955ffb640 1 -- 172.21.15.90:0/350196242 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd0e305ee0 con 0x55bd0e41a410 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.019+0000 7fd955ffb640 10 client.15405 handle_command_reply: tid=0 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.019+0000 7fd9771e8740 1 client.15405 shutdown 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e398820 msgr2=0x55bd0e39d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e398820 0x55bd0e39d460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd95c009900 tx=0x7fd95c02a410 comp rx=0 tx=0).stop 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 shutdown_connections 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd0e41a410 0x55bd0e41c830 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd0e3a30e0 0x55bd0e418650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd0e398820 0x55bd0e39d460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd955ffb640 0 client.15405 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 --2- 172.21.15.90:0/350196242 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd0e397f00 0x55bd0e39cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 >> 172.21.15.90:0/350196242 conn(0x55bd0e394410 msgr2=0x55bd0e39ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 shutdown_connections 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 1 -- 172.21.15.90:0/350196242 wait complete. 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9771e8740 20 client.15405 trim_cache size 0 max 16384 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:12:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:12:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:12:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:12:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd9580a3ee0 msgr2=0x7fd9580a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd9580a3ee0 0x7fd9580a63a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd95c000c00 tx=0x7fd95c032cb0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c108920 msgr2=0x7fd96c1026d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c108920 0x7fd96c1026d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd964009fd0 tx=0x7fd96400b260 comp rx=0 tx=0).stop 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 shutdown_connections 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd9580a3ee0 0x7fd9580a63a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd96c108920 0x7fd96c1026d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd96c06bb00 0x7fd96c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 --2- 172.21.15.90:0/1697426719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd96c06b160 0x7fd96c101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 >> 172.21.15.90:0/1697426719 conn(0x7fd96c077630 msgr2=0x7fd96c100830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 shutdown_connections 2024-04-26T18:12:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:12:52.020+0000 7fd9743f0640 1 -- 172.21.15.90:0/1697426719 wait complete. 2024-04-26T18:12:52.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:52 smithi119 ceph-mon[32752]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T18:12:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:52 smithi167 ceph-mon[32751]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T18:12:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:52 smithi090 ceph-mon[29180]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-26T18:12:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:54 smithi119 ceph-mon[32752]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-26T18:12:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:54 smithi167 ceph-mon[32751]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-26T18:12:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:54 smithi090 ceph-mon[29180]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-26T18:12:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:56 smithi119 ceph-mon[32752]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T18:12:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:56 smithi167 ceph-mon[32751]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T18:12:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:56 smithi090 ceph-mon[29180]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T18:12:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:12:58 smithi119 ceph-mon[32752]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T18:12:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:12:58 smithi167 ceph-mon[32751]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T18:12:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:12:58 smithi090 ceph-mon[29180]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T18:13:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:00 smithi119 ceph-mon[32752]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T18:13:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:00 smithi167 ceph-mon[32751]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T18:13:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:00 smithi090 ceph-mon[29180]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-26T18:13:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:02 smithi119 ceph-mon[32752]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T18:13:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:02 smithi167 ceph-mon[32751]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T18:13:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:02 smithi090 ceph-mon[29180]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-26T18:13:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:13:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:04 smithi119 ceph-mon[32752]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T18:13:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:13:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:13:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:04 smithi167 ceph-mon[32751]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T18:13:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:13:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:13:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:04 smithi090 ceph-mon[29180]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-26T18:13:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:13:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:13:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 -- 172.21.15.90:0/3728836003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d01044a0 msgr2=0x7fc0d010c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 --2- 172.21.15.90:0/3728836003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d01044a0 0x7fc0d010c980 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009900 tx=0x7fc0b802a480 comp rx=0 tx=0).stop 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 -- 172.21.15.90:0/3728836003 shutdown_connections 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 --2- 172.21.15.90:0/3728836003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0d010cf50 0x7fc0d010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 --2- 172.21.15.90:0/3728836003 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d01044a0 0x7fc0d010c980 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 --2- 172.21.15.90:0/3728836003 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d0103b80 0x7fc0d0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 -- 172.21.15.90:0/3728836003 >> 172.21.15.90:0/3728836003 conn(0x7fc0d0078a40 msgr2=0x7fc0d0078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 -- 172.21.15.90:0/3728836003 shutdown_connections 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.116+0000 7fc0d5189640 1 -- 172.21.15.90:0/3728836003 wait complete. 2024-04-26T18:13:05.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 Processor -- start 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 -- start start 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0d0103b80 0x7fc0d01990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d01044a0 0x7fc0d01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d010cf50 0x7fc0d019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0112bd0 con 0x7fc0d010cf50 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0112a50 con 0x7fc0d0103b80 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0d5189640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0112d50 con 0x7fc0d01044a0 2024-04-26T18:13:05.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0cf59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d010cf50 0x7fc0d019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0cf59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d010cf50 0x7fc0d019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56180/0 (socket says 172.21.15.90:56180) 2024-04-26T18:13:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.117+0000 7fc0cf59e640 1 -- 172.21.15.90:0/365110094 learned_addr learned my addr 172.21.15.90:0/365110094 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d01044a0 0x7fc0d01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0d0103b80 0x7fc0d01990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0cf59e640 1 -- 172.21.15.90:0/365110094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d01044a0 msgr2=0x7fc0d01995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d01044a0 0x7fc0d01995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0cf59e640 1 -- 172.21.15.90:0/365110094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0d0103b80 msgr2=0x7fc0d01990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0d0103b80 0x7fc0d01990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0cf59e640 1 -- 172.21.15.90:0/365110094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d019e050 con 0x7fc0d010cf50 2024-04-26T18:13:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d010cf50 0x7fc0d019d920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800ea60 tx=0x7fc0c800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0ccd99640 1 -- 172.21.15.90:0/365110094 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c800ce40 con 0x7fc0d010cf50 2024-04-26T18:13:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0ccd99640 1 -- 172.21.15.90:0/365110094 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c8005020 con 0x7fc0d010cf50 2024-04-26T18:13:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d019e340 con 0x7fc0d010cf50 2024-04-26T18:13:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0ccd99640 1 -- 172.21.15.90:0/365110094 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8010670 con 0x7fc0d010cf50 2024-04-26T18:13:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.118+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d01a5bd0 con 0x7fc0d010cf50 2024-04-26T18:13:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.120+0000 7fc0ccd99640 1 -- 172.21.15.90:0/365110094 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7fc0c80054b0 con 0x7fc0d010cf50 2024-04-26T18:13:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 Processor -- start 2024-04-26T18:13:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 -- start start 2024-04-26T18:13:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca108ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0ccd99640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc0bc0a3ee0 0x7fc0bc0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca10b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e1ca064460 con 0x55e1ca0fe820 2024-04-26T18:13:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1ca01e590 con 0x55e1ca0fdf00 2024-04-26T18:13:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e1ca060420 con 0x55e1ca1090e0 2024-04-26T18:13:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0ced9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca108ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc0bc0a3ee0 0x7fc0bc0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0ced9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca108ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56186/0 (socket says 172.21.15.90:56186) 2024-04-26T18:13:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.121+0000 7fc0ced9d640 1 -- 172.21.15.90:0/3691701845 learned_addr learned my addr 172.21.15.90:0/3691701845 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0ccd99640 1 -- 172.21.15.90:0/365110094 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc0c80d0f50 con 0x7fc0d010cf50 2024-04-26T18:13:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca10b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0ced9d640 1 -- 172.21.15.90:0/3691701845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 msgr2=0x55e1ca10b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca10b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0ced9d640 1 -- 172.21.15.90:0/3691701845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 msgr2=0x55e1ca0fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca0fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0ced9d640 1 -- 172.21.15.90:0/3691701845 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ca02b0d0 con 0x55e1ca0fe820 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca108ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000dcb0 tx=0x7fc0c000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36980827141f0b92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc0bc0a3ee0 0x7fc0bc0a63a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8031440 tx=0x7fc0b8032ee0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0c57fa640 1 -- 172.21.15.90:0/3691701845 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0011440 con 0x55e1ca0fe820 2024-04-26T18:13:05.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0c57fa640 1 -- 172.21.15.90:0/3691701845 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c0011a90 con 0x55e1ca0fe820 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.122+0000 7fc0c57fa640 1 -- 172.21.15.90:0/3691701845 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0010990 con 0x55e1ca0fe820 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/3691701845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 msgr2=0x55e1ca108ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca108ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000dcb0 tx=0x7fc0c000f730 comp rx=0 tx=0).stop 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/3691701845 shutdown_connections 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca10b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca108ba0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/3691701845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/3691701845 >> 172.21.15.90:0/3691701845 conn(0x55e1ca0fa410 msgr2=0x55e1ca0fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/3691701845 shutdown_connections 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/3691701845 wait complete. 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.123+0000 7fc0d7f7b740 1 -- start start 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 1 Processor -- start 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fdf00 0x55e1ca106030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fe820 0x55e1ca106570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca106ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e1ca10bd10 con 0x55e1ca0fdf00 2024-04-26T18:13:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e1c9febb60 con 0x55e1ca0fe820 2024-04-26T18:13:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e1ca10be90 con 0x55e1ca1090e0 2024-04-26T18:13:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0d7f7b740 10 client.0 _ll_register_callbacks cb 0x55e1ca0102b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fdf00 0x55e1ca106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fdf00 0x55e1ca106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56188/0 (socket says 172.21.15.90:56188) 2024-04-26T18:13:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 -- 172.21.15.90:0/4238735641 learned_addr learned my addr 172.21.15.90:0/4238735641 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca106ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fe820 0x55e1ca106570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 -- 172.21.15.90:0/4238735641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 msgr2=0x55e1ca106ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca106ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 -- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fe820 msgr2=0x55e1ca106570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fe820 0x55e1ca106570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 -- 172.21.15.90:0/4238735641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c9d76210 con 0x55e1ca0fdf00 2024-04-26T18:13:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:13:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fdf00 0x55e1ca106030 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0006010 tx=0x7fc0c00103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f9556b7cc95bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.124+0000 7fc0b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:13:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0b67fc640 1 -- 172.21.15.90:0/4238735641 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c002b060 con 0x55e1ca0fdf00 2024-04-26T18:13:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0b67fc640 1 -- 172.21.15.90:0/4238735641 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc0c0011c40 con 0x55e1ca0fdf00 2024-04-26T18:13:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c0011a90 con 0x55e1ca0fdf00 2024-04-26T18:13:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0b67fc640 1 -- 172.21.15.90:0/4238735641 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0c0004040 con 0x55e1ca0fdf00 2024-04-26T18:13:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 10 client.15417 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1c9e42620 con 0x55e1ca0fdf00 2024-04-26T18:13:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0b67fc640 1 -- 172.21.15.90:0/4238735641 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c0011a90 con 0x55e1ca0fdf00 2024-04-26T18:13:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0b67fc640 1 -- 172.21.15.90:0/4238735641 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc0c002e020 con 0x55e1ca0fdf00 2024-04-26T18:13:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 10 client.15417 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 20 client.15417 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca119110 0x55e1ca11b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 4 client.15417 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.125+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1ca072070 con 0x55e1ca119110 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.126+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca119110 0x55e1ca11b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.126+0000 7fc0b67fc640 10 client.15417 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.126+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca119110 0x55e1ca11b500 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93d62660518cdc95 server_cookie=2ef5fa9272541951 in_seq=0 out_seq=0 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0b67fc640 1 -- 172.21.15.90:0/4238735641 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1ca072070 con 0x55e1ca119110 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0b67fc640 10 client.15417 handle_command_reply: tid=0 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 client.15417 shutdown 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fdf00 msgr2=0x55e1ca106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fdf00 0x55e1ca106030 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0006010 tx=0x7fc0c00103d0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 shutdown_connections 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca119110 0x55e1ca11b500 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca106ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fe820 0x55e1ca106570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/4238735641 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fdf00 0x55e1ca106030 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0b67fc640 0 client.15417 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 >> 172.21.15.90:0/4238735641 conn(0x55e1ca0fa410 msgr2=0x55e1ca100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 shutdown_connections 2024-04-26T18:13:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/4238735641 wait complete. 2024-04-26T18:13:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.127+0000 7fc0d7f7b740 20 client.15417 trim_cache size 0 max 16384 2024-04-26T18:13:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 Processor -- start 2024-04-26T18:13:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 -- start start 2024-04-26T18:13:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca105970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca1008c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca120aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01b0c30 con 0x55e1ca0fe820 2024-04-26T18:13:05.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01ae500 con 0x55e1ca0fdf00 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01ac350 con 0x55e1ca1090e0 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0c7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca1008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0c7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca1008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56202/0 (socket says 172.21.15.90:56202) 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.142+0000 7fc0c7fff640 1 -- 172.21.15.90:0/967569654 learned_addr learned my addr 172.21.15.90:0/967569654 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca120aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca105970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c7fff640 1 -- 172.21.15.90:0/967569654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 msgr2=0x55e1ca120aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca120aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c7fff640 1 -- 172.21.15.90:0/967569654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 msgr2=0x55e1ca105970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca105970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c7fff640 1 -- 172.21.15.90:0/967569654 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1ca1045e0 con 0x55e1ca0fe820 2024-04-26T18:13:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca1008c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc0b802a9a0 tx=0x7fc0b8049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b5e93afef0fe16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c4ff9640 1 -- 172.21.15.90:0/967569654 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8004300 con 0x55e1ca0fe820 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c4ff9640 1 -- 172.21.15.90:0/967569654 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b8056070 con 0x55e1ca0fe820 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.143+0000 7fc0c4ff9640 1 -- 172.21.15.90:0/967569654 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b80038e0 con 0x55e1ca0fe820 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/967569654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 msgr2=0x55e1ca1008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca1008c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc0b802a9a0 tx=0x7fc0b8049000 comp rx=0 tx=0).stop 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/967569654 shutdown_connections 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca1090e0 0x55e1ca120aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca0fe820 0x55e1ca1008c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/967569654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca105970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/967569654 >> 172.21.15.90:0/967569654 conn(0x55e1ca0fa410 msgr2=0x55e1ca0f8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/967569654 shutdown_connections 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/967569654 wait complete. 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- start start 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 Processor -- start 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca0fe820 0x55e1ca1030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca1090e0 0x55e1ca103620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01ac9c0 con 0x55e1ca1090e0 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01ade30 con 0x55e1ca0fdf00 2024-04-26T18:13:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.144+0000 7fc0d7f7b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01ad090 con 0x55e1ca0fe820 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0d7f7b740 10 client.0 _ll_register_callbacks cb 0x7fc0c0015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca1090e0 0x55e1ca103620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca1090e0 0x55e1ca103620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56204/0 (socket says 172.21.15.90:56204) 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 -- 172.21.15.90:0/765274960 learned_addr learned my addr 172.21.15.90:0/765274960 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 -- 172.21.15.90:0/765274960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca0fe820 msgr2=0x55e1ca1030e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0c7fff640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca0fe820 0x55e1ca1030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca0fe820 0x55e1ca1030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 -- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 msgr2=0x55e1ca102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca102ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 -- 172.21.15.90:0/765274960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1ca17e310 con 0x55e1ca1090e0 2024-04-26T18:13:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0cf59e640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca1090e0 0x55e1ca103620 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80996e0 tx=0x7fc0c809d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35388ac35185b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0b67fc640 1 -- 172.21.15.90:0/765274960 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c80cee60 con 0x55e1ca1090e0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0b67fc640 1 -- 172.21.15.90:0/765274960 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fc0c80c2010 con 0x55e1ca1090e0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0b8056070 con 0x55e1ca1090e0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0b67fc640 1 -- 172.21.15.90:0/765274960 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc0c80a41e0 con 0x55e1ca1090e0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0d7f7b740 10 client.15423 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e1ca17f540 con 0x55e1ca1090e0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.145+0000 7fc0b67fc640 1 -- 172.21.15.90:0/765274960 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0b8056070 con 0x55e1ca1090e0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0b67fc640 1 -- 172.21.15.90:0/765274960 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fc0c80a50f0 con 0x55e1ca1090e0 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0d7f7b740 10 client.15423 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0d7f7b740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0d7f7b740 20 client.15423 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca17fcb0 0x55e1ca1820d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0d7f7b740 4 client.15423 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e1ca06bee0 con 0x55e1ca17fcb0 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca17fcb0 0x55e1ca1820d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0ced9d640 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca17fcb0 0x55e1ca1820d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7c6e13464d34ae server_cookie=a5c8e007b6b74fe9 in_seq=0 out_seq=0 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.146+0000 7fc0b67fc640 10 client.15423 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0b67fc640 1 -- 172.21.15.90:0/765274960 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e1ca06bee0 con 0x55e1ca17fcb0 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0b67fc640 10 client.15423 handle_command_reply: tid=0 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 client.15423 shutdown 2024-04-26T18:13:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca1090e0 msgr2=0x55e1ca103620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca1090e0 0x55e1ca103620 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80996e0 tx=0x7fc0c809d490 comp rx=0 tx=0).stop 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 shutdown_connections 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e1ca17fcb0 0x55e1ca1820d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e1ca1090e0 0x55e1ca103620 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0b67fc640 0 client.15423 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e1ca0fe820 0x55e1ca1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 --2- 172.21.15.90:0/765274960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e1ca0fdf00 0x55e1ca102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.152+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 >> 172.21.15.90:0/765274960 conn(0x55e1ca0fa410 msgr2=0x55e1ca0f95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 shutdown_connections 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d7f7b740 1 -- 172.21.15.90:0/765274960 wait complete. 2024-04-26T18:13:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d7f7b740 20 client.15423 trim_cache size 0 max 16384 2024-04-26T18:13:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:13:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:13:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:13:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:13:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc0bc0a3ee0 msgr2=0x7fc0bc0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc0bc0a3ee0 0x7fc0bc0a63a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8031440 tx=0x7fc0b8032ee0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d010cf50 msgr2=0x7fc0d019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d010cf50 0x7fc0d019d920 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800ea60 tx=0x7fc0c800ef20 comp rx=0 tx=0).stop 2024-04-26T18:13:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 shutdown_connections 2024-04-26T18:13:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc0bc0a3ee0 0x7fc0bc0a63a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0d010cf50 0x7fc0d019d920 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d01044a0 0x7fc0d01995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 --2- 172.21.15.90:0/365110094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0d0103b80 0x7fc0d01990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 >> 172.21.15.90:0/365110094 conn(0x7fc0d0078a40 msgr2=0x7fc0d00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 shutdown_connections 2024-04-26T18:13:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:05.153+0000 7fc0d5189640 1 -- 172.21.15.90:0/365110094 wait complete. 2024-04-26T18:13:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:06 smithi119 ceph-mon[32752]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-26T18:13:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:06 smithi167 ceph-mon[32751]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-26T18:13:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:06 smithi090 ceph-mon[29180]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-04-26T18:13:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:08 smithi119 ceph-mon[32752]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T18:13:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:08 smithi167 ceph-mon[32751]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T18:13:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:08 smithi090 ceph-mon[29180]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-04-26T18:13:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:10 smithi119 ceph-mon[32752]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:13:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:10 smithi167 ceph-mon[32751]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:13:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:10 smithi090 ceph-mon[29180]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:13:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:12 smithi119 ceph-mon[32752]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:13:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:12 smithi167 ceph-mon[32751]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:13:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:12 smithi090 ceph-mon[29180]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:13:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:14 smithi119 ceph-mon[32752]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-26T18:13:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:14 smithi167 ceph-mon[32751]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-26T18:13:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:14 smithi090 ceph-mon[29180]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-26T18:13:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:16 smithi119 ceph-mon[32752]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T18:13:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:16 smithi167 ceph-mon[32751]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T18:13:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:16 smithi090 ceph-mon[29180]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.261+0000 7f2945f19640 1 -- 172.21.15.90:0/1785933950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940101860 msgr2=0x7f2940101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.261+0000 7f2945f19640 1 --2- 172.21.15.90:0/1785933950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940101860 0x7f2940101ce0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2934009930 tx=0x7f293402a420 comp rx=0 tx=0).stop 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 -- 172.21.15.90:0/1785933950 shutdown_connections 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 --2- 172.21.15.90:0/1785933950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2940102220 0x7f294010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 --2- 172.21.15.90:0/1785933950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940101860 0x7f2940101ce0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 --2- 172.21.15.90:0/1785933950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940107820 0x7f2940107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 -- 172.21.15.90:0/1785933950 >> 172.21.15.90:0/1785933950 conn(0x7f29400789b0 msgr2=0x7f2940078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 -- 172.21.15.90:0/1785933950 shutdown_connections 2024-04-26T18:13:18.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 -- 172.21.15.90:0/1785933950 wait complete. 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 Processor -- start 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.262+0000 7f2945f19640 1 -- start start 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f2945f19640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2940101860 0x7f29401990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f2945f19640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940102220 0x7f2940199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f2945f19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940107820 0x7f294019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f2945f19640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2940111ae0 con 0x7f2940102220 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f2945f19640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2940111960 con 0x7f2940107820 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f2945f19640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2940111c60 con 0x7f2940101860 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940102220 0x7f2940199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940102220 0x7f2940199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55910/0 (socket says 172.21.15.90:55910) 2024-04-26T18:13:18.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 -- 172.21.15.90:0/2952185564 learned_addr learned my addr 172.21.15.90:0/2952185564 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2940101860 0x7f29401990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 -- 172.21.15.90:0/2952185564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2940101860 msgr2=0x7f29401990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2940101860 0x7f29401990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 -- 172.21.15.90:0/2952185564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940107820 msgr2=0x7f294019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940107820 0x7f294019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 -- 172.21.15.90:0/2952185564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294019e0c0 con 0x7f2940102220 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.263+0000 7f293effd640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940102220 0x7f2940199610 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2934005940 tx=0x7f29340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:18.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.264+0000 7f293cff9640 1 -- 172.21.15.90:0/2952185564 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293403f030 con 0x7f2940102220 2024-04-26T18:13:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.264+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294019e350 con 0x7f2940102220 2024-04-26T18:13:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.264+0000 7f293cff9640 1 -- 172.21.15.90:0/2952185564 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2934003cb0 con 0x7f2940102220 2024-04-26T18:13:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.264+0000 7f293cff9640 1 -- 172.21.15.90:0/2952185564 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2934035070 con 0x7f2940102220 2024-04-26T18:13:18.267 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.264+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29401a5c40 con 0x7f2940102220 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.266+0000 7f293cff9640 1 -- 172.21.15.90:0/2952185564 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f2934042030 con 0x7f2940102220 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f293cff9640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f29280a3ee0 0x7f29280a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f29280a3ee0 0x7f29280a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f293cff9640 1 -- 172.21.15.90:0/2952185564 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f29340f1170 con 0x7f2940102220 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 Processor -- start 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 -- start start 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f29280a3ee0 0x7f29280a63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2930009b50 tx=0x7f2930005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108ca52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cafba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108cb00e0 0x563108cb24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563108c0b460 con 0x563108cb00e0 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563108bc5590 con 0x563108ca5820 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.267+0000 7f2948c38740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563108c07420 con 0x563108ca4f00 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108cb00e0 0x563108cb24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108cb00e0 0x563108cb24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55912/0 (socket says 172.21.15.90:55912) 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 -- 172.21.15.90:0/3924627887 learned_addr learned my addr 172.21.15.90:0/3924627887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293f7fe640 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108ca52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293effd640 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cafba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 -- 172.21.15.90:0/3924627887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 msgr2=0x563108ca52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108ca52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 -- 172.21.15.90:0/3924627887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 msgr2=0x563108cafba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cafba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 -- 172.21.15.90:0/3924627887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563108bd20d0 con 0x563108cb00e0 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293f7fe640 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108ca52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f293ffff640 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108cb00e0 0x563108cb24d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2938009de0 tx=0x7f293800d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165921b812053fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f2925ffb640 1 -- 172.21.15.90:0/3924627887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2938003970 con 0x563108cb00e0 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f2925ffb640 1 -- 172.21.15.90:0/3924627887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2938004020 con 0x563108cb00e0 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.268+0000 7f2925ffb640 1 -- 172.21.15.90:0/3924627887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29380107c0 con 0x563108cb00e0 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 -- 172.21.15.90:0/3924627887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108cb00e0 msgr2=0x563108cb24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108cb00e0 0x563108cb24d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2938009de0 tx=0x7f293800d6b0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 -- 172.21.15.90:0/3924627887 shutdown_connections 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108cb00e0 0x563108cb24d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cafba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 --2- 172.21.15.90:0/3924627887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108ca52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 -- 172.21.15.90:0/3924627887 >> 172.21.15.90:0/3924627887 conn(0x563108ca1410 msgr2=0x563108ca17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 -- 172.21.15.90:0/3924627887 shutdown_connections 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 -- 172.21.15.90:0/3924627887 wait complete. 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.269+0000 7f2948c38740 1 -- start start 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 1 Processor -- start 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cadaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563108cb2d10 con 0x563108ca4f00 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563108b92b60 con 0x563108ca5820 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563108cb2e90 con 0x563108cb00e0 2024-04-26T18:13:18.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f2948c38740 10 client.0 _ll_register_callbacks cb 0x563108bb72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cad070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55926/0 (socket says 172.21.15.90:55926) 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293f7fe640 1 -- 172.21.15.90:0/2115236166 learned_addr learned my addr 172.21.15.90:0/2115236166 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293effd640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293ffff640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cadaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293f7fe640 1 -- 172.21.15.90:0/2115236166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 msgr2=0x563108cadaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cadaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.270+0000 7f293f7fe640 1 -- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 msgr2=0x563108cad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f293f7fe640 1 -- 172.21.15.90:0/2115236166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56310891d210 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f293effd640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cad070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f29300039c0 tx=0x7f2930004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39253a6991a79ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f291effd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f291effd640 1 -- 172.21.15.90:0/2115236166 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293001eb40 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f291effd640 1 -- 172.21.15.90:0/2115236166 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f293001ece0 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2938004020 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f291effd640 1 -- 172.21.15.90:0/2115236166 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2930023840 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f2948c38740 10 client.15435 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5631089e9620 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f291effd640 1 -- 172.21.15.90:0/2115236166 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2938004020 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.271+0000 7f291effd640 1 -- 172.21.15.90:0/2115236166 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f293002ca00 con 0x563108ca4f00 2024-04-26T18:13:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f2948c38740 10 client.15435 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f2948c38740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f2948c38740 20 client.15435 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f2948c38740 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108cc2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f2948c38740 4 client.15435 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563108c19070 con 0x563108cc0320 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f293effd640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108cc2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f291effd640 10 client.15435 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.272+0000 7f293effd640 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108cc2710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2137aa111621952 server_cookie=f34ab0d10c15db1b in_seq=0 out_seq=0 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f291effd640 1 -- 172.21.15.90:0/2115236166 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563108c19070 con 0x563108cc0320 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f291effd640 10 client.15435 handle_command_reply: tid=0 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 client.15435 shutdown 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 msgr2=0x563108cad070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cad070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f29300039c0 tx=0x7f2930004730 comp rx=0 tx=0).stop 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 shutdown_connections 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108cc2710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f291effd640 0 client.15435 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cadaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108cad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 --2- 172.21.15.90:0/2115236166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cad070 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 >> 172.21.15.90:0/2115236166 conn(0x563108ca1410 msgr2=0x563108ca7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 shutdown_connections 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.273+0000 7f2948c38740 1 -- 172.21.15.90:0/2115236166 wait complete. 2024-04-26T18:13:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.274+0000 7f2948c38740 20 client.15435 trim_cache size 0 max 16384 2024-04-26T18:13:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 Processor -- start 2024-04-26T18:13:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 -- start start 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cbbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108caefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f29401ae9f0 con 0x563108ca4f00 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f29401ac2c0 con 0x563108ca5820 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f2948c38740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29401aa150 con 0x563108cb00e0 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cbbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f293f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cbbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55932/0 (socket says 172.21.15.90:55932) 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.288+0000 7f293f7fe640 1 -- 172.21.15.90:0/2470770739 learned_addr learned my addr 172.21.15.90:0/2470770739 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293effd640 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108caefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293ffff640 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293f7fe640 1 -- 172.21.15.90:0/2470770739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 msgr2=0x563108cc8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cc8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293f7fe640 1 -- 172.21.15.90:0/2470770739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 msgr2=0x563108caefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108caefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293f7fe640 1 -- 172.21.15.90:0/2470770739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563108cbee40 con 0x563108ca4f00 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293effd640 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108caefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f293f7fe640 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cbbbd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2930011970 tx=0x7f2930004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310d787a128bf6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f29257fa640 1 -- 172.21.15.90:0/2470770739 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930039070 con 0x563108ca4f00 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f29257fa640 1 -- 172.21.15.90:0/2470770739 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293001e790 con 0x563108ca4f00 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.289+0000 7f29257fa640 1 -- 172.21.15.90:0/2470770739 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293002c690 con 0x563108ca4f00 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 -- 172.21.15.90:0/2470770739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 msgr2=0x563108cbbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cbbbd0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2930011970 tx=0x7f2930004540 comp rx=0 tx=0).stop 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 -- 172.21.15.90:0/2470770739 shutdown_connections 2024-04-26T18:13:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108cb00e0 0x563108cc8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108ca5820 0x563108caefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 --2- 172.21.15.90:0/2470770739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca4f00 0x563108cbbbd0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 -- 172.21.15.90:0/2470770739 >> 172.21.15.90:0/2470770739 conn(0x563108ca1410 msgr2=0x563108cae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 -- 172.21.15.90:0/2470770739 shutdown_connections 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 -- 172.21.15.90:0/2470770739 wait complete. 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.290+0000 7f2948c38740 1 -- start start 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 1 Processor -- start 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108cb3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca5820 0x563108cb4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108cb00e0 0x563108cb4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f29401aa7a0 con 0x563108ca5820 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f29401abbf0 con 0x563108cb00e0 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29401aae50 con 0x563108ca4f00 2024-04-26T18:13:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f2948c38740 10 client.0 _ll_register_callbacks cb 0x7f2930009960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca5820 0x563108cb4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca5820 0x563108cb4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55948/0 (socket says 172.21.15.90:55948) 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293effd640 1 -- 172.21.15.90:0/1575567616 learned_addr learned my addr 172.21.15.90:0/1575567616 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293f7fe640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108cb3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293ffff640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108cb00e0 0x563108cb4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293f7fe640 1 -- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108cb00e0 msgr2=0x563108cb4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293f7fe640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108cb00e0 0x563108cb4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293f7fe640 1 -- 172.21.15.90:0/1575567616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca5820 msgr2=0x563108cb4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293f7fe640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca5820 0x563108cb4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293f7fe640 1 -- 172.21.15.90:0/1575567616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563108d25a90 con 0x563108ca4f00 2024-04-26T18:13:18.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.291+0000 7f293effd640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca5820 0x563108cb4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f293f7fe640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108cb3cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f293002dff0 tx=0x7f2930023de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cbe70e742d585be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f291effd640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f291effd640 1 -- 172.21.15.90:0/1575567616 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2930039070 con 0x563108ca4f00 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f293001e790 con 0x563108ca4f00 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f291effd640 1 -- 172.21.15.90:0/1575567616 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f293001ed10 con 0x563108ca4f00 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f291effd640 1 -- 172.21.15.90:0/1575567616 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2930005030 con 0x563108ca4f00 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f2948c38740 10 client.24952 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563108d26d80 con 0x563108ca4f00 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.292+0000 7f291effd640 1 -- 172.21.15.90:0/1575567616 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f293001e790 con 0x563108ca4f00 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f291effd640 1 -- 172.21.15.90:0/1575567616 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f2930004cd0 con 0x563108ca4f00 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f2948c38740 10 client.24952 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:18.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f2948c38740 10 client.24952 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f2948c38740 20 client.24952 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f2948c38740 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108d293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f2948c38740 4 client.24952 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:13:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563108c12ee0 con 0x563108cc0320 2024-04-26T18:13:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f293effd640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108d293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f291effd640 10 client.24952 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:18.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.293+0000 7f293effd640 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108d293f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ad2d9bba6656e1 server_cookie=7a4aa5c30c68790d in_seq=0 out_seq=0 2024-04-26T18:13:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.297+0000 7f291effd640 1 -- 172.21.15.90:0/1575567616 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563108c12ee0 con 0x563108cc0320 2024-04-26T18:13:18.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.297+0000 7f291effd640 10 client.24952 handle_command_reply: tid=0 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.297+0000 7f2948c38740 1 client.24952 shutdown 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 msgr2=0x563108cb3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108cb3cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f293002dff0 tx=0x7f2930023de0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 shutdown_connections 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563108cc0320 0x563108d293f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563108cb00e0 0x563108cb4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563108ca5820 0x563108cb4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 --2- 172.21.15.90:0/1575567616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563108ca4f00 0x563108cb3cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 >> 172.21.15.90:0/1575567616 conn(0x563108ca1410 msgr2=0x563108caf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f291effd640 0 client.24952 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 shutdown_connections 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 1 -- 172.21.15.90:0/1575567616 wait complete. 2024-04-26T18:13:18.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.298+0000 7f2948c38740 20 client.24952 trim_cache size 0 max 16384 2024-04-26T18:13:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:13:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:13:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:13:18.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:13:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f29280a3ee0 msgr2=0x7f29280a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f29280a3ee0 0x7f29280a63a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2930009b50 tx=0x7f2930005f90 comp rx=0 tx=0).stop 2024-04-26T18:13:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940102220 msgr2=0x7f2940199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940102220 0x7f2940199610 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2934005940 tx=0x7f29340042f0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 shutdown_connections 2024-04-26T18:13:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f29280a3ee0 0x7f29280a63a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2940107820 0x7f294019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2940102220 0x7f2940199610 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 --2- 172.21.15.90:0/2952185564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2940101860 0x7f29401990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 >> 172.21.15.90:0/2952185564 conn(0x7f29400789b0 msgr2=0x7f294006a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 shutdown_connections 2024-04-26T18:13:18.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:18.299+0000 7f2945f19640 1 -- 172.21.15.90:0/2952185564 wait complete. 2024-04-26T18:13:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:18 smithi119 ceph-mon[32752]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-26T18:13:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:18 smithi167 ceph-mon[32751]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-26T18:13:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:18 smithi090 ceph-mon[29180]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 5 op/s 2024-04-26T18:13:20.419 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:13:20.711 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:13:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:20 smithi119 ceph-mon[32752]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:13:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:20 smithi167 ceph-mon[32751]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:13:20.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:20 smithi090 ceph-mon[29180]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:13:22.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:22 smithi090 ceph-mon[29180]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:13:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 -- 172.21.15.90:0/3598670300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c1094a0 msgr2=0x7f5f8c109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/3598670300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c1094a0 0x7f5f8c109880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009990 tx=0x7f5f7c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 -- 172.21.15.90:0/3598670300 shutdown_connections 2024-04-26T18:13:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/3598670300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c1094a0 0x7f5f8c109880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/3598670300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f8c06bad0 0x7f5f8c1102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/3598670300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f8c06b130 0x7f5f8c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 -- 172.21.15.90:0/3598670300 >> 172.21.15.90:0/3598670300 conn(0x7f5f8c0792d0 msgr2=0x7f5f8c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 -- 172.21.15.90:0/3598670300 shutdown_connections 2024-04-26T18:13:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.662+0000 7f5f91d7d700 1 -- 172.21.15.90:0/3598670300 wait complete. 2024-04-26T18:13:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 Processor -- start 2024-04-26T18:13:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 -- start start 2024-04-26T18:13:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c06b130 0x7f5f8c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f8c06bad0 0x7f5f8c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f8c1094a0 0x7f5f8c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c1135b0 con 0x7f5f8c06b130 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c113430 con 0x7f5f8c1094a0 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f91d7d700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c113730 con 0x7f5f8c06bad0 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.663+0000 7f5f8b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c06b130 0x7f5f8c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c06b130 0x7f5f8c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54982/0 (socket says 172.21.15.90:54982) 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 -- 172.21.15.90:0/892304649 learned_addr learned my addr 172.21.15.90:0/892304649 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8affd700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f8c06bad0 0x7f5f8c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8bfff700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f8c1094a0 0x7f5f8c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 -- 172.21.15.90:0/892304649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f8c06bad0 msgr2=0x7f5f8c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f8c06bad0 0x7f5f8c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 -- 172.21.15.90:0/892304649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f8c1094a0 msgr2=0x7f5f8c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f8c1094a0 0x7f5f8c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:22.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 -- 172.21.15.90:0/892304649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c19a070 con 0x7f5f8c06b130 2024-04-26T18:13:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f8b7fe700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c06b130 0x7f5f8c19f750 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c00 tx=0x7f5f7c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f88ff9700 1 -- 172.21.15.90:0/892304649 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c027070 con 0x7f5f8c06b130 2024-04-26T18:13:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f88ff9700 1 -- 172.21.15.90:0/892304649 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f7c003f20 con 0x7f5f8c06b130 2024-04-26T18:13:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.664+0000 7f5f88ff9700 1 -- 172.21.15.90:0/892304649 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c022cd0 con 0x7f5f8c06b130 2024-04-26T18:13:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.665+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c19a300 con 0x7f5f8c06b130 2024-04-26T18:13:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.665+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c19a6a0 con 0x7f5f8c06b130 2024-04-26T18:13:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.666+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c06a110 con 0x7f5f8c06b130 2024-04-26T18:13:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.667+0000 7f5f88ff9700 1 -- 172.21.15.90:0/892304649 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7f5f7c02a070 con 0x7f5f8c06b130 2024-04-26T18:13:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.668+0000 7f5f88ff9700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5f740a4020 0x7f5f740a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.668+0000 7f5f88ff9700 1 -- 172.21.15.90:0/892304649 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f5f7c0d1f20 con 0x7f5f8c06b130 2024-04-26T18:13:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.668+0000 7f5f8affd700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5f740a4020 0x7f5f740a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.668+0000 7f5f8affd700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5f740a4020 0x7f5f740a64e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c079a10 tx=0x7f5f8000b3c0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:22.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.675+0000 7f5f88ff9700 1 -- 172.21.15.90:0/892304649 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f7c031070 con 0x7f5f8c06b130 2024-04-26T18:13:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:22 smithi119 ceph-mon[32752]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:13:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:22 smithi167 ceph-mon[32751]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-26T18:13:22.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.997+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f8c02e490 con 0x7f5f8c06b130 2024-04-26T18:13:22.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:22.998+0000 7f5f88ff9700 1 -- 172.21.15.90:0/892304649 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11874 (secure 0 0 0) 0x7f5f7c013070 con 0x7f5f8c06b130 2024-04-26T18:13:22.999 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:13:22.999 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:11:56.125308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:13:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.001+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5f740a4020 msgr2=0x7f5f740a64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:23.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.001+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5f740a4020 0x7f5f740a64e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c079a10 tx=0x7f5f8000b3c0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.001+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c06b130 msgr2=0x7f5f8c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.001+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c06b130 0x7f5f8c19f750 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c000c00 tx=0x7f5f7c0047b0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 shutdown_connections 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5f740a4020 0x7f5f740a64e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5f8c06b130 0x7f5f8c19f750 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f8c1094a0 0x7f5f8c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 --2- 172.21.15.90:0/892304649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f8c06bad0 0x7f5f8c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 >> 172.21.15.90:0/892304649 conn(0x7f5f8c0792d0 msgr2=0x7f5f8c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:23.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 shutdown_connections 2024-04-26T18:13:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:23.002+0000 7f5f91d7d700 1 -- 172.21.15.90:0/892304649 wait complete. 2024-04-26T18:13:23.003 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T18:13:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:23 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/892304649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:13:23.791 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db --set-id 8454b329 --release --await' 2024-04-26T18:13:23.791 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db --set-id 8454b329 --release --await 2024-04-26T18:13:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:23 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/892304649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:13:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:23 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/892304649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:13:24.067 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:13:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:24 smithi119 ceph-mon[32752]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:13:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:24 smithi167 ceph-mon[32751]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:13:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:24 smithi090 ceph-mon[29180]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-26T18:13:25.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.945+0000 7fd365024700 1 -- 172.21.15.90:0/3665789156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd360101a10 msgr2=0x7fd360101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:25.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.945+0000 7fd365024700 1 --2- 172.21.15.90:0/3665789156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd360101a10 0x7fd360101e70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd354009990 tx=0x7fd354009ca0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.945+0000 7fd365024700 1 -- 172.21.15.90:0/3665789156 shutdown_connections 2024-04-26T18:13:25.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.945+0000 7fd365024700 1 --2- 172.21.15.90:0/3665789156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3601023b0 0x7fd3601102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.945+0000 7fd365024700 1 --2- 172.21.15.90:0/3665789156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3601094b0 0x7fd360109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.945+0000 7fd365024700 1 --2- 172.21.15.90:0/3665789156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd360101a10 0x7fd360101e70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.945+0000 7fd365024700 1 -- 172.21.15.90:0/3665789156 >> 172.21.15.90:0/3665789156 conn(0x7fd360078990 msgr2=0x7fd3600ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.946+0000 7fd365024700 1 -- 172.21.15.90:0/3665789156 shutdown_connections 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.946+0000 7fd365024700 1 -- 172.21.15.90:0/3665789156 wait complete. 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.946+0000 7fd365024700 1 Processor -- start 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd365024700 1 -- start start 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd365024700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd360101a10 0x7fd3601a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd365024700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3601023b0 0x7fd3601a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd365024700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3601094b0 0x7fd36019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd365024700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601135c0 con 0x7fd3601094b0 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd365024700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd360113440 con 0x7fd360101a10 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd365024700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd360113740 con 0x7fd3601023b0 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd35e59c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3601023b0 0x7fd3601a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3601094b0 0x7fd36019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd360101a10 0x7fd3601a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd35f59e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3601094b0 0x7fd36019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55008/0 (socket says 172.21.15.90:55008) 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.947+0000 7fd35f59e700 1 -- 172.21.15.90:0/491244979 learned_addr learned my addr 172.21.15.90:0/491244979 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35f59e700 1 -- 172.21.15.90:0/491244979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3601023b0 msgr2=0x7fd3601a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35f59e700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3601023b0 0x7fd3601a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35f59e700 1 -- 172.21.15.90:0/491244979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd360101a10 msgr2=0x7fd3601a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35f59e700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd360101a10 0x7fd3601a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35f59e700 1 -- 172.21.15.90:0/491244979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd36019c260 con 0x7fd3601094b0 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35e59c700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3601023b0 0x7fd3601a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd360101a10 0x7fd3601a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd35f59e700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3601094b0 0x7fd36019bb00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd358007ee0 tx=0x7fd35800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.948+0000 7fd347fff700 1 -- 172.21.15.90:0/491244979 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35800dbd0 con 0x7fd3601094b0 2024-04-26T18:13:25.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.949+0000 7fd347fff700 1 -- 172.21.15.90:0/491244979 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd35800dd70 con 0x7fd3601094b0 2024-04-26T18:13:25.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.949+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd36019c550 con 0x7fd3601094b0 2024-04-26T18:13:25.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.949+0000 7fd347fff700 1 -- 172.21.15.90:0/491244979 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358005990 con 0x7fd3601094b0 2024-04-26T18:13:25.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.949+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3601a8810 con 0x7fd3601094b0 2024-04-26T18:13:25.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.952+0000 7fd347fff700 1 -- 172.21.15.90:0/491244979 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 23) ==== 138879+0+0 (secure 0 0 0) 0x7fd35801a020 con 0x7fd3601094b0 2024-04-26T18:13:25.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.953+0000 7fd347fff700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd3480a3f50 0x7fd3480a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.953+0000 7fd347fff700 1 -- 172.21.15.90:0/491244979 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd3580d0160 con 0x7fd3601094b0 2024-04-26T18:13:25.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.953+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd3480a3f50 0x7fd3480a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.953+0000 7fd36f8b6b80 1 Processor -- start 2024-04-26T18:13:25.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.953+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd3480a3f50 0x7fd3480a6410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd350005e10 tx=0x7fd350005d60 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:25.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd36f8b6b80 1 -- start start 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 0x559e261e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559e26127840 con 0x559e261d41a0 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559e260c66e0 con 0x559e261df970 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559e26123800 con 0x559e261d4ac0 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261d4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55012/0 (socket says 172.21.15.90:55012) 2024-04-26T18:13:25.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 -- 172.21.15.90:0/3786042427 learned_addr learned my addr 172.21.15.90:0/3786042427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35e59c700 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 -- 172.21.15.90:0/3786042427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 msgr2=0x559e261df430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261df430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 -- 172.21.15.90:0/3786042427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 msgr2=0x559e261e1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 0x559e261e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.954+0000 7fd35ed9d700 1 -- 172.21.15.90:0/3786042427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e260749d0 con 0x559e261d41a0 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.955+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261d4580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd3500044e0 tx=0x7fd350012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c9a7bb5520c931 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:25.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.955+0000 7fd3457fa700 1 -- 172.21.15.90:0/3786042427 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350003880 con 0x559e261d41a0 2024-04-26T18:13:25.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.955+0000 7fd3457fa700 1 -- 172.21.15.90:0/3786042427 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd350024070 con 0x559e261d41a0 2024-04-26T18:13:25.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.955+0000 7fd3457fa700 1 -- 172.21.15.90:0/3786042427 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35001fe30 con 0x559e261d41a0 2024-04-26T18:13:25.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.956+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3786042427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 msgr2=0x559e261d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:25.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.956+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261d4580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd3500044e0 tx=0x7fd350012000 comp rx=0 tx=0).stop 2024-04-26T18:13:25.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.956+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3786042427 shutdown_connections 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.956+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261d4580 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.956+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 0x559e261e1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.956+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3786042427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.956+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3786042427 >> 172.21.15.90:0/3786042427 conn(0x559e26101e40 msgr2=0x559e261d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.957+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3786042427 shutdown_connections 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.957+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3786042427 wait complete. 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.957+0000 7fd36f8b6b80 1 -- start start 2024-04-26T18:13:25.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.957+0000 7fd36f8b6b80 1 Processor -- start 2024-04-26T18:13:25.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d41a0 0x559e261df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d4ac0 0x559e261dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.960 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559e261e2540 con 0x559e261d4ac0 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559e2608da50 con 0x559e261d41a0 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd35e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d4ac0 0x559e261dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd35e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d4ac0 0x559e261dc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55026/0 (socket says 172.21.15.90:55026) 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd35e59c700 1 -- 172.21.15.90:0/3930188651 learned_addr learned my addr 172.21.15.90:0/3930188651 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d41a0 0x559e261df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559e261e26c0 con 0x559e261df970 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.958+0000 7fd35f59e700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd35f59e700 1 -- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d41a0 msgr2=0x559e261df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd35f59e700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d41a0 0x559e261df190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd35f59e700 1 -- 172.21.15.90:0/3930188651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d4ac0 msgr2=0x559e261dc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd35f59e700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d4ac0 0x559e261dc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd35f59e700 1 -- 172.21.15.90:0/3930188651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e261dd120 con 0x559e261df970 2024-04-26T18:13:25.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd35e59c700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d4ac0 0x559e261dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd35f59e700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261dc9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3580cd7a0 tx=0x7fd3580982d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5f0c996cec6590f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd342ff5700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd36f8b6b80 10 client.0 _ll_register_callbacks cb 0x559e260bb680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd350024070 con 0x559e261df970 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd342ff5700 1 -- 172.21.15.90:0/3930188651 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35809c380 con 0x559e261df970 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd342ff5700 1 -- 172.21.15.90:0/3930188651 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd3580a4700 con 0x559e261df970 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd342ff5700 1 -- 172.21.15.90:0/3930188651 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd35809bf20 con 0x559e261df970 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd36f8b6b80 10 client.24970 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.959+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=26}) -- 0x559e261eceb0 con 0x559e261df970 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd342ff5700 1 -- 172.21.15.90:0/3930188651 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd35809f020 con 0x559e261df970 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd36f8b6b80 10 client.24970 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd36f8b6b80 10 client.24970 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:13:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd36f8b6b80 20 client.24970 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:25.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e261ef3e0 0x559e261f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd36f8b6b80 4 client.24970 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:13:25.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd350024070 con 0x559e261ef3e0 2024-04-26T18:13:25.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e261ef3e0 0x559e261f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:25.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd342ff5700 10 client.24970 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:25.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.960+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e261ef3e0 0x559e261f1870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18bbe8fbce8aa7de server_cookie=3d3d2e998288c6fc in_seq=0 out_seq=0 2024-04-26T18:13:25.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd342ff5700 1 -- 172.21.15.90:0/3930188651 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fd350024070 con 0x559e261ef3e0 2024-04-26T18:13:25.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd342ff5700 10 client.24970 handle_command_reply: tid=0 2024-04-26T18:13:25.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 client.24970 shutdown 2024-04-26T18:13:25.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 msgr2=0x559e261dc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:25.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261dc9d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3580cd7a0 tx=0x7fd3580982d0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 shutdown_connections 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e261ef3e0 0x559e261f1870 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261dc9d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d41a0 0x559e261df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/3930188651 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d4ac0 0x559e261dc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 >> 172.21.15.90:0/3930188651 conn(0x559e26101e40 msgr2=0x559e261d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd342ff5700 0 client.24970 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.961+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 shutdown_connections 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.962+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/3930188651 wait complete. 2024-04-26T18:13:25.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.962+0000 7fd36f8b6b80 20 client.24970 trim_cache size 0 max 16384 2024-04-26T18:13:25.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 Processor -- start 2024-04-26T18:13:25.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 -- start start 2024-04-26T18:13:25.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261eb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:25.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 0x559e261f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:26.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601ae140 con 0x559e261d41a0 2024-04-26T18:13:26.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601ab9d0 con 0x559e261df970 2024-04-26T18:13:26.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd36010e170 con 0x559e261d4ac0 2024-04-26T18:13:26.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:26.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261de3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55032/0 (socket says 172.21.15.90:55032) 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 -- 172.21.15.90:0/1271392280 learned_addr learned my addr 172.21.15.90:0/1271392280 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35e59c700 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 -- 172.21.15.90:0/1271392280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 msgr2=0x559e261eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 -- 172.21.15.90:0/1271392280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 msgr2=0x559e261f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 0x559e261f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:25.999+0000 7fd35ed9d700 1 -- 172.21.15.90:0/1271392280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e261c7f40 con 0x559e261d41a0 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.000+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261de3d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd350010010 tx=0x7fd35000b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08a784b63982a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.000+0000 7fd344ff9700 1 -- 172.21.15.90:0/1271392280 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35001fc40 con 0x559e261d41a0 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.000+0000 7fd344ff9700 1 -- 172.21.15.90:0/1271392280 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd350024050 con 0x559e261d41a0 2024-04-26T18:13:26.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.000+0000 7fd344ff9700 1 -- 172.21.15.90:0/1271392280 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3500362e0 con 0x559e261d41a0 2024-04-26T18:13:26.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1271392280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 msgr2=0x559e261de3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:26.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261de3d0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd350010010 tx=0x7fd35000b9b0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1271392280 shutdown_connections 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261de3d0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261df970 0x559e261f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1271392280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261d4ac0 0x559e261eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1271392280 >> 172.21.15.90:0/1271392280 conn(0x559e26101e40 msgr2=0x559e261ddcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1271392280 shutdown_connections 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.001+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1271392280 wait complete. 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 -- start start 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 Processor -- start 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261e7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d4ac0 0x559e261e4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261e4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601a9eb0 con 0x559e261d41a0 2024-04-26T18:13:26.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601ab300 con 0x559e261d4ac0 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd3601aa560 con 0x559e261df970 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261e7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd35ed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261e7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55034/0 (socket says 172.21.15.90:55034) 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd35ed9d700 1 -- 172.21.15.90:0/1325479852 learned_addr learned my addr 172.21.15.90:0/1325479852 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd35e59c700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d4ac0 0x559e261e4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd36f8b6b80 10 client.0 _ll_register_callbacks cb 0x559e260bb680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.002+0000 7fd35f59e700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261e4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd35e59c700 1 -- 172.21.15.90:0/1325479852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 msgr2=0x559e261e4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd35e59c700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261e4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd35e59c700 1 -- 172.21.15.90:0/1325479852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 msgr2=0x559e261e7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd35e59c700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261e7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd35e59c700 1 -- 172.21.15.90:0/1325479852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e262669d0 con 0x559e261d4ac0 2024-04-26T18:13:26.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261e7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd35e59c700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d4ac0 0x559e261e4920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x559e261d1490 tx=0x7fd35400f9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=903571f388c4ea5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd342ff5700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd342ff5700 1 -- 172.21.15.90:0/1325479852 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35400be90 con 0x559e261d4ac0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd342ff5700 1 -- 172.21.15.90:0/1325479852 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd354003860 con 0x559e261d4ac0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd350024050 con 0x559e261d4ac0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd342ff5700 1 -- 172.21.15.90:0/1325479852 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd354003a80 con 0x559e261d4ac0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd36f8b6b80 10 client.25043 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.003+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559e26267b10 con 0x559e261d4ac0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd342ff5700 1 -- 172.21.15.90:0/1325479852 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd354028070 con 0x559e261d4ac0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd342ff5700 1 -- 172.21.15.90:0/1325479852 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd354034020 con 0x559e261d4ac0 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd36f8b6b80 10 client.25043 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd36f8b6b80 10 client.25043 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:13:26.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd36f8b6b80 20 client.25043 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e26268170 0x559e2626a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd36f8b6b80 4 client.25043 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8454b329", "release": true, "await": true}] 2024-04-26T18:13:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8454b329", "release": true, "await": true}) -- 0x7fd350024050 con 0x559e26268170 2024-04-26T18:13:26.008 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.004+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e26268170 0x559e2626a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.005+0000 7fd342ff5700 10 client.25043 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.005+0000 7fd35ed9d700 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e26268170 0x559e2626a590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=139498b9b31ba981 server_cookie=f2e3920e29892d53 in_seq=0 out_seq=0 2024-04-26T18:13:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd342ff5700 1 -- 172.21.15.90:0/1325479852 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7fd350024050 con 0x559e26268170 2024-04-26T18:13:26.009 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd342ff5700 10 client.25043 handle_command_reply: tid=0 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 client.25043 shutdown 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d4ac0 msgr2=0x559e261e4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d4ac0 0x559e261e4920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x559e261d1490 tx=0x7fd35400f9a0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 shutdown_connections 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559e26268170 0x559e2626a590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559e261d41a0 0x559e261e7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559e261df970 0x559e261e4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 --2- 172.21.15.90:0/1325479852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e261d4ac0 0x559e261e4920 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.007+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 >> 172.21.15.90:0/1325479852 conn(0x559e26101e40 msgr2=0x559e261de550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd342ff5700 0 client.25043 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 shutdown_connections 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd36f8b6b80 1 -- 172.21.15.90:0/1325479852 wait complete. 2024-04-26T18:13:26.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd36f8b6b80 20 client.25043 trim_cache size 0 max 16384 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 26, 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 4, 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "8454b329": { 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 4, 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:13:26.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 90.0, 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:26.012 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd3480a3f50 msgr2=0x7fd3480a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd365024700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd3480a3f50 0x7fd3480a6410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd350005e10 tx=0x7fd350005d60 comp rx=0 tx=0).stop 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3601094b0 msgr2=0x7fd36019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.008+0000 7fd365024700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3601094b0 0x7fd36019bb00 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd358007ee0 tx=0x7fd35800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 shutdown_connections 2024-04-26T18:13:26.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd3480a3f50 0x7fd3480a6410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd3601094b0 0x7fd36019bb00 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd360101a10 0x7fd3601a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 --2- 172.21.15.90:0/491244979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd3601023b0 0x7fd3601a1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:26.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 >> 172.21.15.90:0/491244979 conn(0x7fd360078990 msgr2=0x7fd360106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:26.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 shutdown_connections 2024-04-26T18:13:26.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:26.009+0000 7fd365024700 1 -- 172.21.15.90:0/491244979 wait complete. 2024-04-26T18:13:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:26 smithi119 ceph-mon[32752]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:26 smithi167 ceph-mon[32751]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:26 smithi090 ceph-mon[29180]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:28 smithi119 ceph-mon[32752]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:28 smithi167 ceph-mon[32751]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:28 smithi090 ceph-mon[29180]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:30 smithi167 ceph-mon[32751]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T18:13:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:30 smithi167 ceph-mon[32751]: mgrmap e24: x(active, since 92s), standbys: z, y 2024-04-26T18:13:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:30 smithi090 ceph-mon[29180]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T18:13:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:30 smithi090 ceph-mon[29180]: mgrmap e24: x(active, since 92s), standbys: z, y 2024-04-26T18:13:31.051 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8454b329', seconds elapsed: 46.4 2024-04-26T18:13:31.052 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T18:13:31.052 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:13:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:30 smithi119 ceph-mon[32752]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T18:13:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:30 smithi119 ceph-mon[32752]: mgrmap e24: x(active, since 92s), standbys: z, y 2024-04-26T18:13:31.431 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:13:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.438+0000 7fdfe28d1640 1 -- 172.21.15.90:0/3024672280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc107850 msgr2=0x7fdfdc107c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.438+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/3024672280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc107850 0x7fdfdc107c30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009930 tx=0x7fdfcc02a420 comp rx=0 tx=0).stop 2024-04-26T18:13:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.439+0000 7fdfe28d1640 1 -- 172.21.15.90:0/3024672280 shutdown_connections 2024-04-26T18:13:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.439+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/3024672280 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc102200 0x7fdfdc10e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.439+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/3024672280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfdc101840 0x7fdfdc101cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.439+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/3024672280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc107850 0x7fdfdc107c30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.439+0000 7fdfe28d1640 1 -- 172.21.15.90:0/3024672280 >> 172.21.15.90:0/3024672280 conn(0x7fdfdc078a40 msgr2=0x7fdfdc078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.439+0000 7fdfe28d1640 1 -- 172.21.15.90:0/3024672280 shutdown_connections 2024-04-26T18:13:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.439+0000 7fdfe28d1640 1 -- 172.21.15.90:0/3024672280 wait complete. 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.440+0000 7fdfe28d1640 1 Processor -- start 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.440+0000 7fdfe28d1640 1 -- start start 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.440+0000 7fdfe28d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc101840 0x7fdfdc197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.440+0000 7fdfe28d1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc102200 0x7fdfdc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.440+0000 7fdfe28d1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfdc107850 0x7fdfdc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.440+0000 7fdfe28d1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc111b60 con 0x7fdfdc102200 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe28d1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1119e0 con 0x7fdfdc107850 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe28d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc111ce0 con 0x7fdfdc101840 2024-04-26T18:13:31.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfdbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc101840 0x7fdfdc197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc102200 0x7fdfdc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc102200 0x7fdfdc1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56950/0 (socket says 172.21.15.90:56950) 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/327652571 learned_addr learned my addr 172.21.15.90:0/327652571 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe0e6e640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfdc107850 0x7fdfdc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe0e6e640 1 -- 172.21.15.90:0/327652571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc101840 msgr2=0x7fdfdc197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe0e6e640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc101840 0x7fdfdc197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe0e6e640 1 -- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc102200 msgr2=0x7fdfdc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe0e6e640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc102200 0x7fdfdc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfe0e6e640 1 -- 172.21.15.90:0/327652571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc19cf90 con 0x7fdfdc107850 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.441+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc102200 0x7fdfdc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.442+0000 7fdfe0e6e640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfdc107850 0x7fdfdc19c860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009fd0 tx=0x7fdfd400b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.442+0000 7fdfd97fa640 1 -- 172.21.15.90:0/327652571 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd400be40 con 0x7fdfdc107850 2024-04-26T18:13:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.442+0000 7fdfd97fa640 1 -- 172.21.15.90:0/327652571 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfd4017450 con 0x7fdfdc107850 2024-04-26T18:13:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.442+0000 7fdfd97fa640 1 -- 172.21.15.90:0/327652571 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4005280 con 0x7fdfdc107850 2024-04-26T18:13:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.443+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc19d280 con 0x7fdfdc107850 2024-04-26T18:13:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.443+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc06b100 con 0x7fdfdc107850 2024-04-26T18:13:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.446+0000 7fdfe55ee740 1 Processor -- start 2024-04-26T18:13:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.446+0000 7fdfd97fa640 1 -- 172.21.15.90:0/327652571 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fdfd4004a40 con 0x7fdfdc107850 2024-04-26T18:13:31.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfe55ee740 1 -- start start 2024-04-26T18:13:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 0x561c96fafaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fba3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fba8f0 0x561c96fbcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561c96f0d460 con 0x561c96fb0030 2024-04-26T18:13:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561c96ec7590 con 0x561c96faf710 2024-04-26T18:13:31.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c96f09420 con 0x561c96fba8f0 2024-04-26T18:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfd97fa640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdfc40a3e40 0x7fdfc40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.447+0000 7fdfd97fa640 1 -- 172.21.15.90:0/327652571 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdfd40d0760 con 0x7fdfdc107850 2024-04-26T18:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fba3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fba3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56962/0 (socket says 172.21.15.90:56962) 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 -- 172.21.15.90:0/970297104 learned_addr learned my addr 172.21.15.90:0/970297104 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 0x561c96fafaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fba8f0 0x561c96fbcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 -- 172.21.15.90:0/970297104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fba8f0 msgr2=0x561c96fbcce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fba8f0 0x561c96fbcce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 -- 172.21.15.90:0/970297104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 msgr2=0x561c96fafaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 0x561c96fafaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 -- 172.21.15.90:0/970297104 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c96ed40d0 con 0x561c96fb0030 2024-04-26T18:13:31.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfe0e6e640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdfc40a3e40 0x7fdfc40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.448+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fba3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc006010 tx=0x7fdfcc004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45654cece5a68fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.449+0000 7fdfc27fc640 1 -- 172.21.15.90:0/970297104 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc035040 con 0x561c96fb0030 2024-04-26T18:13:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.449+0000 7fdfc27fc640 1 -- 172.21.15.90:0/970297104 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc0054f0 con 0x561c96fb0030 2024-04-26T18:13:31.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.449+0000 7fdfc27fc640 1 -- 172.21.15.90:0/970297104 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0035f0 con 0x561c96fb0030 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 -- 172.21.15.90:0/970297104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 msgr2=0x561c96fba3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe0e6e640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdfc40a3e40 0x7fdfc40a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40a4eb0 tx=0x7fdfd400bcc0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fba3b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc006010 tx=0x7fdfcc004170 comp rx=0 tx=0).stop 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 -- 172.21.15.90:0/970297104 shutdown_connections 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fba8f0 0x561c96fbcce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fba3b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/970297104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 0x561c96fafaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 -- 172.21.15.90:0/970297104 >> 172.21.15.90:0/970297104 conn(0x561c96fabc20 msgr2=0x561c96fac000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 -- 172.21.15.90:0/970297104 shutdown_connections 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 -- 172.21.15.90:0/970297104 wait complete. 2024-04-26T18:13:31.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.450+0000 7fdfe55ee740 1 -- start start 2024-04-26T18:13:31.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.451+0000 7fdfe55ee740 1 Processor -- start 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.451+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 0x561c96fb7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.451+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fb0030 0x561c96fb7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.451+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fba8f0 0x561c96fb82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.451+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561c96fbd4c0 con 0x561c96fba8f0 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.451+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561c96e94b60 con 0x561c96faf710 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.451+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c96fbd640 con 0x561c96fb0030 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fba8f0 0x561c96fb82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fba8f0 0x561c96fb82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56964/0 (socket says 172.21.15.90:56964) 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/3491833532 learned_addr learned my addr 172.21.15.90:0/3491833532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fb0030 0x561c96fb7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfe55ee740 10 client.0 _ll_register_callbacks cb 0x561c96eb92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/3491833532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fb0030 msgr2=0x561c96fb7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fb0030 0x561c96fb7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/3491833532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 msgr2=0x561c96fb7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 0x561c96fb7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/3491833532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c96c1f210 con 0x561c96fba8f0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fb0030 0x561c96fb7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fba8f0 0x561c96fb82e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0012760 tx=0x7fdfd0012790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6d06e0600d191 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.452+0000 7fdfbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/3491833532 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0004040 con 0x561c96fba8f0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfcc0054f0 con 0x561c96fba8f0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/3491833532 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdfd00041e0 con 0x561c96fba8f0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/3491833532 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfd00043e0 con 0x561c96fba8f0 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfe55ee740 10 client.15477 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:31.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c96ceb620 con 0x561c96fba8f0 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/3491833532 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc0054f0 con 0x561c96fba8f0 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.453+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/3491833532 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdfd0028610 con 0x561c96fba8f0 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfe55ee740 10 client.15477 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfe55ee740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfe55ee740 20 client.15477 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c96fcae30 0x561c96fcd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfe55ee740 4 client.15477 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c96f1b070 con 0x561c96fcae30 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c96fcae30 0x561c96fcd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c96fcae30 0x561c96fcd2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad70137f7c62a073 server_cookie=fc83a3685aaf8adc in_seq=0 out_seq=0 2024-04-26T18:13:31.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.454+0000 7fdfbb7fe640 10 client.15477 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.455+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/3491833532 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c96f1b070 con 0x561c96fcae30 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.455+0000 7fdfbb7fe640 10 client.15477 handle_command_reply: tid=0 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.455+0000 7fdfe55ee740 1 client.15477 shutdown 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fba8f0 msgr2=0x561c96fb82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fba8f0 0x561c96fb82e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0012760 tx=0x7fdfd0012790 comp rx=0 tx=0).stop 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 shutdown_connections 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c96fcae30 0x561c96fcd2b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fba8f0 0x561c96fb82e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96fb0030 0x561c96fb7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3491833532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96faf710 0x561c96fb7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 >> 172.21.15.90:0/3491833532 conn(0x561c96fabc20 msgr2=0x561c96fa25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfbb7fe640 0 client.15477 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 shutdown_connections 2024-04-26T18:13:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3491833532 wait complete. 2024-04-26T18:13:31.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.456+0000 7fdfe55ee740 20 client.15477 trim_cache size 0 max 16384 2024-04-26T18:13:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.485+0000 7fdfe55ee740 1 Processor -- start 2024-04-26T18:13:31.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfe55ee740 1 -- start start 2024-04-26T18:13:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fd2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb3240 0x561c96fd30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1aead0 con 0x561c96fb0030 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1ac3a0 con 0x561c96fb3240 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1aa1f0 con 0x561c96faf710 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fd2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fd2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56966/0 (socket says 172.21.15.90:56966) 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 -- 172.21.15.90:0/3718292560 learned_addr learned my addr 172.21.15.90:0/3718292560 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb3240 0x561c96fd30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 -- 172.21.15.90:0/3718292560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 msgr2=0x561c96fc6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 -- 172.21.15.90:0/3718292560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb3240 msgr2=0x561c96fd30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb3240 0x561c96fd30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.486+0000 7fdfdbfff640 1 -- 172.21.15.90:0/3718292560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c96fc9640 con 0x561c96fb0030 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.487+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fd2b80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc004be0 tx=0x7fdfcc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4221224c129b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.487+0000 7fdfc1ffb640 1 -- 172.21.15.90:0/3718292560 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc035040 con 0x561c96fb0030 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.487+0000 7fdfc1ffb640 1 -- 172.21.15.90:0/3718292560 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc058070 con 0x561c96fb0030 2024-04-26T18:13:31.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.487+0000 7fdfc1ffb640 1 -- 172.21.15.90:0/3718292560 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0531b0 con 0x561c96fb0030 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3718292560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 msgr2=0x561c96fd2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fd2b80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc004be0 tx=0x7fdfcc031400 comp rx=0 tx=0).stop 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3718292560 shutdown_connections 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb3240 0x561c96fd30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb0030 0x561c96fd2b80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/3718292560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3718292560 >> 172.21.15.90:0/3718292560 conn(0x561c96fabc20 msgr2=0x561c96fb2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3718292560 shutdown_connections 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 -- 172.21.15.90:0/3718292560 wait complete. 2024-04-26T18:13:31.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 -- start start 2024-04-26T18:13:31.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.488+0000 7fdfe55ee740 1 Processor -- start 2024-04-26T18:13:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb0030 0x561c96fb41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfe55ee740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb4710 0x561c96fb4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1aa860 con 0x561c96fb4710 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1abcd0 con 0x561c96fb0030 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfe55ee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc1aaf30 con 0x561c96faf710 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb4710 0x561c96fb4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb4710 0x561c96fb4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56980/0 (socket says 172.21.15.90:56980) 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/2798137430 learned_addr learned my addr 172.21.15.90:0/2798137430 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb0030 0x561c96fb41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfe55ee740 10 client.0 _ll_register_callbacks cb 0x7fdfd0012a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/2798137430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 msgr2=0x561c96fc2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb0030 msgr2=0x561c96fb41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:31.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb0030 0x561c96fb41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.489+0000 7fdfdb7fe640 1 -- 172.21.15.90:0/2798137430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c970304c0 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfdb7fe640 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb4710 0x561c96fb4b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x561c96faca60 tx=0x7fdfd0012790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6895e6eaf080b474 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/2798137430 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd002e070 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/2798137430 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdfd0003dd0 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfcc058070 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/2798137430 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fdfd0028450 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfe55ee740 10 client.15483 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561c970317b0 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.490+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/2798137430 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc058070 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/2798137430 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fdfd001f540 con 0x561c96fb4710 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfe55ee740 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:31.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfe55ee740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfe55ee740 20 client.15483 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c97031e10 0x561c97034230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfe55ee740 4 client.15483 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:13:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c96f14ee0 con 0x561c97031e10 2024-04-26T18:13:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.491+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c97031e10 0x561c97034230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.492+0000 7fdfbb7fe640 10 client.15483 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:31.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.492+0000 7fdfdbfff640 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c97031e10 0x561c97034230 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1bb9b0fa0a95db server_cookie=cfd283fbcc678082 in_seq=0 out_seq=0 2024-04-26T18:13:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.497+0000 7fdfbb7fe640 1 -- 172.21.15.90:0/2798137430 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c96f14ee0 con 0x561c97031e10 2024-04-26T18:13:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfbb7fe640 10 client.15483 handle_command_reply: tid=0 2024-04-26T18:13:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfe55ee740 1 client.15483 shutdown 2024-04-26T18:13:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb4710 msgr2=0x561c96fb4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb4710 0x561c96fb4b90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x561c96faca60 tx=0x7fdfd0012790 comp rx=0 tx=0).stop 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 shutdown_connections 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561c97031e10 0x561c97034230 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561c96fb4710 0x561c96fb4b90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561c96fb0030 0x561c96fb41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.498+0000 7fdfbb7fe640 0 client.15483 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.499+0000 7fdfe55ee740 1 --2- 172.21.15.90:0/2798137430 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c96faf710 0x561c96fc2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.499+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 >> 172.21.15.90:0/2798137430 conn(0x561c96fabc20 msgr2=0x561c96fa1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.499+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 shutdown_connections 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.499+0000 7fdfe55ee740 1 -- 172.21.15.90:0/2798137430 wait complete. 2024-04-26T18:13:31.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.499+0000 7fdfe55ee740 20 client.15483 trim_cache size 0 max 16384 2024-04-26T18:13:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:13:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:13:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:13:31.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.500+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdfc40a3e40 msgr2=0x7fdfc40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.500+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdfc40a3e40 0x7fdfc40a6300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40a4eb0 tx=0x7fdfd400bcc0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.500+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfdc107850 msgr2=0x7fdfdc19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.500+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfdc107850 0x7fdfdc19c860 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009fd0 tx=0x7fdfd400b690 comp rx=0 tx=0).stop 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 shutdown_connections 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdfc40a3e40 0x7fdfc40a6300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfdc107850 0x7fdfdc19c860 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdfdc102200 0x7fdfdc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 --2- 172.21.15.90:0/327652571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdfdc101840 0x7fdfdc197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 >> 172.21.15.90:0/327652571 conn(0x7fdfdc078a40 msgr2=0x7fdfdc104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 shutdown_connections 2024-04-26T18:13:31.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:31.501+0000 7fdfe28d1640 1 -- 172.21.15.90:0/327652571 wait complete. 2024-04-26T18:13:32.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:32 smithi090 ceph-mon[29180]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:32 smithi167 ceph-mon[32751]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:32 smithi119 ceph-mon[32752]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:33.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 -- 172.21.15.90:0/2957046134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102ce0 msgr2=0x7fbaf410f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 --2- 172.21.15.90:0/2957046134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102ce0 0x7fbaf410f1d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00b0a0 tx=0x7fbaec00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 -- 172.21.15.90:0/2957046134 shutdown_connections 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 --2- 172.21.15.90:0/2957046134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102ce0 0x7fbaf410f1d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 --2- 172.21.15.90:0/2957046134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbaf4108390 0x7fbaf4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 --2- 172.21.15.90:0/2957046134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaf4102320 0x7fbaf41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 -- 172.21.15.90:0/2957046134 >> 172.21.15.90:0/2957046134 conn(0x7fbaf40792d0 msgr2=0x7fbaf40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 -- 172.21.15.90:0/2957046134 shutdown_connections 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.277+0000 7fbafafe0700 1 -- 172.21.15.90:0/2957046134 wait complete. 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.278+0000 7fbafafe0700 1 Processor -- start 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.278+0000 7fbafafe0700 1 -- start start 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbafafe0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102320 0x7fbaf419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:33.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbafafe0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbaf4102ce0 0x7fbaf419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:33.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbafafe0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaf4108390 0x7fbaf4199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbafafe0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf41124b0 con 0x7fbaf4102320 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbafafe0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4112330 con 0x7fbaf4102ce0 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbafafe0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf4112630 con 0x7fbaf4108390 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102320 0x7fbaf419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102320 0x7fbaf419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56990/0 (socket says 172.21.15.90:56990) 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 -- 172.21.15.90:0/3205264443 learned_addr learned my addr 172.21.15.90:0/3205264443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf3fff700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbaf4102ce0 0x7fbaf419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:33.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf957d700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaf4108390 0x7fbaf4199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 -- 172.21.15.90:0/3205264443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaf4108390 msgr2=0x7fbaf4199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaf4108390 0x7fbaf4199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 -- 172.21.15.90:0/3205264443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbaf4102ce0 msgr2=0x7fbaf419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbaf4102ce0 0x7fbaf419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 -- 172.21.15.90:0/3205264443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf419a010 con 0x7fbaf4102320 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.279+0000 7fbaf8d7c700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102320 0x7fbaf419f660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c9b0 tx=0x7fbae800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.280+0000 7fbaf1ffb700 1 -- 172.21.15.90:0/3205264443 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8007b00 con 0x7fbaf4102320 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.280+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf419a300 con 0x7fbaf4102320 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.280+0000 7fbaf1ffb700 1 -- 172.21.15.90:0/3205264443 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbae8004ce0 con 0x7fbaf4102320 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.280+0000 7fbaf1ffb700 1 -- 172.21.15.90:0/3205264443 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae8005530 con 0x7fbaf4102320 2024-04-26T18:13:33.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.281+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf4103f30 con 0x7fbaf4102320 2024-04-26T18:13:33.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.282+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf4103a40 con 0x7fbaf4102320 2024-04-26T18:13:33.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.282+0000 7fbaf1ffb700 1 -- 172.21.15.90:0/3205264443 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fbae8004750 con 0x7fbaf4102320 2024-04-26T18:13:33.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.283+0000 7fbaf1ffb700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbae40a3db0 0x7fbae40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:33.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.283+0000 7fbaf1ffb700 1 -- 172.21.15.90:0/3205264443 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fbae80d0760 con 0x7fbaf4102320 2024-04-26T18:13:33.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.283+0000 7fbaf3fff700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbae40a3db0 0x7fbae40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:33.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.284+0000 7fbaf3fff700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbae40a3db0 0x7fbae40a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40798b0 tx=0x7fbae000b410 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:33.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.292+0000 7fbaf1ffb700 1 -- 172.21.15.90:0/3205264443 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbae8014030 con 0x7fbaf4102320 2024-04-26T18:13:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.791+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf4064030 con 0x7fbaf4102320 2024-04-26T18:13:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.793+0000 7fbaf1ffb700 1 -- 172.21.15.90:0/3205264443 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11874 (secure 0 0 0) 0x7fbae809ce70 con 0x7fbaf4102320 2024-04-26T18:13:33.794 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:13:33.794 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:11:56.125308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.795+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbae40a3db0 msgr2=0x7fbae40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.795+0000 7fbafafe0700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbae40a3db0 0x7fbae40a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40798b0 tx=0x7fbae000b410 comp rx=0 tx=0).stop 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.795+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102320 msgr2=0x7fbaf419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.795+0000 7fbafafe0700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102320 0x7fbaf419f660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbae800c9b0 tx=0x7fbae800ccc0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 shutdown_connections 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbae40a3db0 0x7fbae40a6270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbaf4102320 0x7fbaf419f660 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaf4108390 0x7fbaf4199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 --2- 172.21.15.90:0/3205264443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbaf4102ce0 0x7fbaf419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 >> 172.21.15.90:0/3205264443 conn(0x7fbaf40792d0 msgr2=0x7fbaf4101560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:33.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 shutdown_connections 2024-04-26T18:13:33.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:33.796+0000 7fbafafe0700 1 -- 172.21.15.90:0/3205264443 wait complete. 2024-04-26T18:13:33.797 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T18:13:34.551 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T18:13:34.551 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T18:13:34.806 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:13:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:34 smithi167 ceph-mon[32751]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T18:13:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:34 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3205264443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:13:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:34 smithi090 ceph-mon[29180]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T18:13:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:34 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3205264443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:34 smithi119 ceph-mon[32752]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-26T18:13:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:34 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3205264443' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:13:36.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.562+0000 7f1693e37700 1 -- 172.21.15.90:0/3061471415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 msgr2=0x7f168c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.562+0000 7f1693e37700 1 --2- 172.21.15.90:0/3061471415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c101e90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.562+0000 7f1693e37700 1 -- 172.21.15.90:0/3061471415 shutdown_connections 2024-04-26T18:13:36.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.562+0000 7f1693e37700 1 --2- 172.21.15.90:0/3061471415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f168c1023d0 0x7f168c110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.562+0000 7f1693e37700 1 --2- 172.21.15.90:0/3061471415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f168c109520 0x7f168c109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.562+0000 7f1693e37700 1 --2- 172.21.15.90:0/3061471415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c101e90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.562+0000 7f1693e37700 1 -- 172.21.15.90:0/3061471415 >> 172.21.15.90:0/3061471415 conn(0x7f168c0789c0 msgr2=0x7f168c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.564+0000 7f1693e37700 1 -- 172.21.15.90:0/3061471415 shutdown_connections 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.564+0000 7f1693e37700 1 -- 172.21.15.90:0/3061471415 wait complete. 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.564+0000 7f1693e37700 1 Processor -- start 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1693e37700 1 -- start start 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1693e37700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1693e37700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f168c1023d0 0x7f168c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1693e37700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f168c109520 0x7f168c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1693e37700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f168c113630 con 0x7f168c101a10 2024-04-26T18:13:36.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1691bd3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f16913d2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f168c1023d0 0x7f168c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1691bd3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57020/0 (socket says 172.21.15.90:57020) 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f16913d2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f168c1023d0 0x7f168c19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:36842/0 (socket says 172.21.15.90:36842) 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1691bd3700 1 -- 172.21.15.90:0/882075985 learned_addr learned my addr 172.21.15.90:0/882075985 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f16923d4700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f168c109520 0x7f168c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1134b0 con 0x7f168c109520 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.565+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1137b0 con 0x7f168c1023d0 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.566+0000 7f1691bd3700 1 -- 172.21.15.90:0/882075985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f168c1023d0 msgr2=0x7f168c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.566+0000 7f1691bd3700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f168c1023d0 0x7f168c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.566+0000 7f1691bd3700 1 -- 172.21.15.90:0/882075985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f168c109520 msgr2=0x7f168c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.566+0000 7f1691bd3700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f168c109520 0x7f168c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.566+0000 7f1691bd3700 1 -- 172.21.15.90:0/882075985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c19a070 con 0x7f168c101a10 2024-04-26T18:13:36.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.566+0000 7f1691bd3700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c19f700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f168800d950 tx=0x7f168800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.569+0000 7f1682ffd700 1 -- 172.21.15.90:0/882075985 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688016070 con 0x7f168c101a10 2024-04-26T18:13:36.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.569+0000 7f1682ffd700 1 -- 172.21.15.90:0/882075985 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168800fd40 con 0x7f168c101a10 2024-04-26T18:13:36.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.570+0000 7f1682ffd700 1 -- 172.21.15.90:0/882075985 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168800e3b0 con 0x7f168c101a10 2024-04-26T18:13:36.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.570+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c19a360 con 0x7f168c101a10 2024-04-26T18:13:36.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.570+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c102b20 con 0x7f168c101a10 2024-04-26T18:13:36.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.572+0000 7f1682ffd700 1 -- 172.21.15.90:0/882075985 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f1688019070 con 0x7f168c101a10 2024-04-26T18:13:36.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.573+0000 7f1682ffd700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f16780a3ed0 0x7f16780a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.573+0000 7f1682ffd700 1 -- 172.21.15.90:0/882075985 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f16880d0170 con 0x7f168c101a10 2024-04-26T18:13:36.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.573+0000 7f16913d2700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f16780a3ed0 0x7f16780a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.573+0000 7f16913d2700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f16780a3ed0 0x7f16780a6390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f167c0098e0 tx=0x7f167c005960 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 Processor -- start 2024-04-26T18:13:36.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 -- start start 2024-04-26T18:13:36.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 0x55f3be495d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f3be3e79c0 con 0x55f3be49b850 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f3be34dac0 con 0x55f3be49af30 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3be3e3980 con 0x55f3be495900 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f16913d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f16913d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57034/0 (socket says 172.21.15.90:57034) 2024-04-26T18:13:36.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.575+0000 7f16913d2700 1 -- 172.21.15.90:0/3854853809 learned_addr learned my addr 172.21.15.90:0/3854853809 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f1691bd3700 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f16913d2700 1 -- 172.21.15.90:0/3854853809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 msgr2=0x55f3be495d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f16913d2700 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 0x55f3be495d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f16913d2700 1 -- 172.21.15.90:0/3854853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 msgr2=0x55f3be49b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f16913d2700 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f16913d2700 1 -- 172.21.15.90:0/3854853809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3be334a60 con 0x55f3be49b850 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f16913d2700 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4952e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f384246d02ee5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f1677fff700 1 -- 172.21.15.90:0/3854853809 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c034070 con 0x55f3be49b850 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f1677fff700 1 -- 172.21.15.90:0/3854853809 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167c024930 con 0x55f3be49b850 2024-04-26T18:13:36.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.576+0000 7f1677fff700 1 -- 172.21.15.90:0/3854853809 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c02f970 con 0x55f3be49b850 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.577+0000 7f169e6c9b80 1 -- 172.21.15.90:0/3854853809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 msgr2=0x55f3be4952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.577+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4952e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f167c009990 tx=0x7f167c01f000 comp rx=0 tx=0).stop 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.577+0000 7f169e6c9b80 1 -- 172.21.15.90:0/3854853809 shutdown_connections 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.577+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 0x55f3be495d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.577+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.577+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/3854853809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4952e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.577+0000 7f169e6c9b80 1 -- 172.21.15.90:0/3854853809 >> 172.21.15.90:0/3854853809 conn(0x55f3be3c3750 msgr2=0x55f3be490090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.578+0000 7f169e6c9b80 1 -- 172.21.15.90:0/3854853809 shutdown_connections 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.578+0000 7f169e6c9b80 1 -- 172.21.15.90:0/3854853809 wait complete. 2024-04-26T18:13:36.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.578+0000 7f169e6c9b80 1 -- start start 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.578+0000 7f169e6c9b80 1 Processor -- start 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be495900 0x55f3be4a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be49e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49b850 0x55f3be49eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f1691bd3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be495900 0x55f3be4a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f1691bd3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be495900 0x55f3be4a15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57042/0 (socket says 172.21.15.90:57042) 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f16913d2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be49e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f3be4a2260 con 0x55f3be495900 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f1691bd3700 1 -- 172.21.15.90:0/64674151 learned_addr learned my addr 172.21.15.90:0/64674151 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f3be3a1c10 con 0x55f3be49b850 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f3be4a23e0 con 0x55f3be49af30 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.579+0000 7f169e6c9b80 10 client.0 _ll_register_callbacks cb 0x55f3be37fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16923d4700 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49b850 0x55f3be49eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16913d2700 1 -- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49b850 msgr2=0x55f3be49eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16913d2700 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49b850 0x55f3be49eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16913d2700 1 -- 172.21.15.90:0/64674151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be495900 msgr2=0x55f3be4a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16913d2700 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be495900 0x55f3be4a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16913d2700 1 -- 172.21.15.90:0/64674151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3be3a5610 con 0x55f3be49af30 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f1691bd3700 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be495900 0x55f3be4a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16913d2700 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be49e970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f167c032040 tx=0x7f167c0246b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6af55ae87872f6aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16757fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:13:36.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16757fa700 1 -- 172.21.15.90:0/64674151 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c034040 con 0x55f3be49af30 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f16757fa700 1 -- 172.21.15.90:0/64674151 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f167c024ae0 con 0x55f3be49af30 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.580+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f167c024930 con 0x55f3be49af30 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f16757fa700 1 -- 172.21.15.90:0/64674151 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f167c024ce0 con 0x55f3be49af30 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 10 client.25009 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3be4a2260 con 0x55f3be49af30 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f16757fa700 1 -- 172.21.15.90:0/64674151 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f167c024930 con 0x55f3be49af30 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f16757fa700 1 -- 172.21.15.90:0/64674151 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f167c0439a0 con 0x55f3be49af30 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 10 client.25009 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 10 client.25009 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 20 client.25009 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:36.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be4b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 4 client.25009 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.581+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3be3f53a0 con 0x55f3be4aedf0 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.582+0000 7f1691bd3700 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be4b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.582+0000 7f16757fa700 10 client.25009 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.582+0000 7f1691bd3700 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be4b11e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c26ff965811ac99 server_cookie=b3e1ab7179f96903 in_seq=0 out_seq=0 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f16757fa700 1 -- 172.21.15.90:0/64674151 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3be3f53a0 con 0x55f3be4aedf0 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f16757fa700 10 client.25009 handle_command_reply: tid=0 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 client.25009 shutdown 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 msgr2=0x55f3be49e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be49e970 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f167c032040 tx=0x7f167c0246b0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 shutdown_connections 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be4b11e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be495900 0x55f3be4a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49b850 0x55f3be49eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f16757fa700 0 client.25009 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/64674151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be49e970 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 >> 172.21.15.90:0/64674151 conn(0x55f3be3c3750 msgr2=0x55f3be4942d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 shutdown_connections 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 1 -- 172.21.15.90:0/64674151 wait complete. 2024-04-26T18:13:36.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.583+0000 7f169e6c9b80 20 client.25009 trim_cache size 0 max 16384 2024-04-26T18:13:36.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 Processor -- start 2024-04-26T18:13:36.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 -- start start 2024-04-26T18:13:36.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 0x55f3be4952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ae120 con 0x55f3be49b850 2024-04-26T18:13:36.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ab9b0 con 0x55f3be49af30 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a9820 con 0x55f3be495900 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f16923d4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f16923d4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57054/0 (socket says 172.21.15.90:57054) 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f16923d4700 1 -- 172.21.15.90:0/2465294073 learned_addr learned my addr 172.21.15.90:0/2465294073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.621+0000 7f16913d2700 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f1691bd3700 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 0x55f3be4952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16923d4700 1 -- 172.21.15.90:0/2465294073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 msgr2=0x55f3be4952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16923d4700 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 0x55f3be4952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16923d4700 1 -- 172.21.15.90:0/2465294073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 msgr2=0x55f3be49daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16923d4700 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16923d4700 1 -- 172.21.15.90:0/2465294073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3be48c880 con 0x55f3be49b850 2024-04-26T18:13:36.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16923d4700 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4ab160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1684009910 tx=0x7f1684009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d009d6f9abdd3681 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16777fe700 1 -- 172.21.15.90:0/2465294073 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168401e070 con 0x55f3be49b850 2024-04-26T18:13:36.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16777fe700 1 -- 172.21.15.90:0/2465294073 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16840047c0 con 0x55f3be49b850 2024-04-26T18:13:36.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.622+0000 7f16777fe700 1 -- 172.21.15.90:0/2465294073 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684007640 con 0x55f3be49b850 2024-04-26T18:13:36.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.623+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2465294073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 msgr2=0x55f3be4ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.623+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4ab160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1684009910 tx=0x7f1684009c20 comp rx=0 tx=0).stop 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.623+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2465294073 shutdown_connections 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.623+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be4ab160 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.623+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be495900 0x55f3be4952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.623+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2465294073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be49af30 0x55f3be49daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.623+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2465294073 >> 172.21.15.90:0/2465294073 conn(0x55f3be3c3750 msgr2=0x55f3be489700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.624+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2465294073 shutdown_connections 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.624+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2465294073 wait complete. 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.624+0000 7f169e6c9b80 1 -- start start 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.624+0000 7f169e6c9b80 1 Processor -- start 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be495900 0x55f3be4a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be4a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f169e6c9b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be515000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1a9e70 con 0x55f3be49b850 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ab2e0 con 0x55f3be495900 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f169e6c9b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1aa540 con 0x55f3be49af30 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f169e6c9b80 10 client.0 _ll_register_callbacks cb 0x55f3be37fae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16923d4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be515000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be4a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be4a4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:36876/0 (socket says 172.21.15.90:36876) 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 -- 172.21.15.90:0/2769884946 learned_addr learned my addr 172.21.15.90:0/2769884946 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 -- 172.21.15.90:0/2769884946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be495900 msgr2=0x55f3be4a4640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:13:36.630 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be495900 0x55f3be4a4640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 -- 172.21.15.90:0/2769884946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 msgr2=0x55f3be515000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be515000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16913d2700 1 -- 172.21.15.90:0/2769884946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3be515540 con 0x55f3be49af30 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f16913d2700 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be4a4b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f167c00ba30 tx=0x7f167c024a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c088bcb9a023bca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f16757fa700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.625+0000 7f16923d4700 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be515000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f16757fa700 1 -- 172.21.15.90:0/2769884946 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c034070 con 0x55f3be49af30 2024-04-26T18:13:36.631 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f16757fa700 1 -- 172.21.15.90:0/2769884946 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f167c03a470 con 0x55f3be49af30 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16840047c0 con 0x55f3be49af30 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f16757fa700 1 -- 172.21.15.90:0/2769884946 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f167c02f790 con 0x55f3be49af30 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f169e6c9b80 10 client.25015 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.626+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f3be5166a0 con 0x55f3be49af30 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f16757fa700 1 -- 172.21.15.90:0/2769884946 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16840047c0 con 0x55f3be49af30 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f16757fa700 1 -- 172.21.15.90:0/2769884946 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f167c03a890 con 0x55f3be49af30 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f169e6c9b80 10 client.25015 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f169e6c9b80 10 client.25015 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f169e6c9b80 20 client.25015 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be518b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:36.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f169e6c9b80 4 client.25015 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T18:13:36.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55f3be3ef440 con 0x55f3be4aedf0 2024-04-26T18:13:36.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.627+0000 7f1691bd3700 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be518b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:36.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.628+0000 7f16757fa700 10 client.25015 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:36.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.628+0000 7f1691bd3700 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be518b70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f7dbcb851f548b server_cookie=117d5188aef2be8c in_seq=0 out_seq=0 2024-04-26T18:13:36.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f16757fa700 1 -- 172.21.15.90:0/2769884946 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x55f3be3ef440 con 0x55f3be4aedf0 2024-04-26T18:13:36.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f16757fa700 10 client.25015 handle_command_reply: tid=0 2024-04-26T18:13:36.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 client.25015 shutdown 2024-04-26T18:13:36.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 msgr2=0x55f3be4a4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be4a4b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f167c00ba30 tx=0x7f167c024a00 comp rx=0 tx=0).stop 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 shutdown_connections 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f3be4aedf0 0x55f3be518b70 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f16757fa700 0 client.25015 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f3be49b850 0x55f3be515000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f3be495900 0x55f3be4a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 --2- 172.21.15.90:0/2769884946 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f3be49af30 0x55f3be4a4b80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.635+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 >> 172.21.15.90:0/2769884946 conn(0x55f3be3c3750 msgr2=0x55f3be489f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.636+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 shutdown_connections 2024-04-26T18:13:36.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.636+0000 7f169e6c9b80 1 -- 172.21.15.90:0/2769884946 wait complete. 2024-04-26T18:13:36.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.636+0000 7f169e6c9b80 20 client.25015 trim_cache size 0 max 16384 2024-04-26T18:13:36.637 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 26, 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 6, 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "e986ca9": { 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 6, 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:13:36.638 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 90.0, 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:13:36.639 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:13:36.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.636+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f16780a3ed0 msgr2=0x7f16780a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.636+0000 7f1693e37700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f16780a3ed0 0x7f16780a6390 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f167c0098e0 tx=0x7f167c005960 comp rx=0 tx=0).stop 2024-04-26T18:13:36.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.636+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 msgr2=0x7f168c19f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:36.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.636+0000 7f1693e37700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c19f700 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f168800d950 tx=0x7f168800dc60 comp rx=0 tx=0).stop 2024-04-26T18:13:36.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 shutdown_connections 2024-04-26T18:13:36.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f16780a3ed0 0x7f16780a6390 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f167c0098e0 tx=0x7f167c005960 comp rx=0 tx=0).stop 2024-04-26T18:13:36.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f168c101a10 0x7f168c19f700 secure :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f168800d950 tx=0x7f168800dc60 comp rx=0 tx=0).stop 2024-04-26T18:13:36.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f168c109520 0x7f168c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 --2- 172.21.15.90:0/882075985 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f168c1023d0 0x7f168c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:36.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 >> 172.21.15.90:0/882075985 conn(0x7f168c0789c0 msgr2=0x7f168c106290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:36.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 shutdown_connections 2024-04-26T18:13:36.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:13:36.637+0000 7f1693e37700 1 -- 172.21.15.90:0/882075985 wait complete. 2024-04-26T18:13:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:36 smithi167 ceph-mon[32751]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:36 smithi090 ceph-mon[29180]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:36 smithi119 ceph-mon[32752]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:37.559 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e986ca9', quiesce took 6.5 seconds. Will release after: 30.0 2024-04-26T18:13:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:38 smithi090 ceph-mon[29180]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:38 smithi119 ceph-mon[32752]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:38 smithi167 ceph-mon[32751]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-26T18:13:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:40 smithi090 ceph-mon[29180]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-26T18:13:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:40 smithi119 ceph-mon[32752]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-26T18:13:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:40 smithi167 ceph-mon[32751]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 938 B/s wr, 9 op/s 2024-04-26T18:13:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:42 smithi090 ceph-mon[29180]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:42 smithi119 ceph-mon[32752]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:42 smithi167 ceph-mon[32751]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:44.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 -- 172.21.15.90:0/4153225113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4107890 msgr2=0x7ff2e4107c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/4153225113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4107890 0x7ff2e4107c70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009900 tx=0x7ff2d802a4a0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 -- 172.21.15.90:0/4153225113 shutdown_connections 2024-04-26T18:13:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/4153225113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e4102960 0x7ff2e410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/4153225113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e4101fa0 0x7ff2e4102420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/4153225113 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4107890 0x7ff2e4107c70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 -- 172.21.15.90:0/4153225113 >> 172.21.15.90:0/4153225113 conn(0x7ff2e4077dc0 msgr2=0x7ff2e40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 -- 172.21.15.90:0/4153225113 shutdown_connections 2024-04-26T18:13:44.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.641+0000 7ff2eac2a640 1 -- 172.21.15.90:0/4153225113 wait complete. 2024-04-26T18:13:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 Processor -- start 2024-04-26T18:13:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 -- start start 2024-04-26T18:13:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e4101fa0 0x7ff2e4198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4102960 0x7ff2e4199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e4107890 0x7ff2e419d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4111b20 con 0x7ff2e4102960 2024-04-26T18:13:44.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41119a0 con 0x7ff2e4107890 2024-04-26T18:13:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2eac2a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e4111ca0 con 0x7ff2e4101fa0 2024-04-26T18:13:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4102960 0x7ff2e4199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2e89c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e4101fa0 0x7ff2e4198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4102960 0x7ff2e4199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44282/0 (socket says 172.21.15.90:44282) 2024-04-26T18:13:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2e3fff640 1 -- 172.21.15.90:0/3029342427 learned_addr learned my addr 172.21.15.90:0/3029342427 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:44.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.642+0000 7ff2e91c7640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e4107890 0x7ff2e419d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e3fff640 1 -- 172.21.15.90:0/3029342427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e4101fa0 msgr2=0x7ff2e4198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e4101fa0 0x7ff2e4198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e3fff640 1 -- 172.21.15.90:0/3029342427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e4107890 msgr2=0x7ff2e419d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e4107890 0x7ff2e419d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e3fff640 1 -- 172.21.15.90:0/3029342427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e419def0 con 0x7ff2e4102960 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e4101fa0 0x7ff2e4198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4102960 0x7ff2e4199490 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000d950 tx=0x7ff2d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:44.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e1ffb640 1 -- 172.21.15.90:0/3029342427 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0017030 con 0x7ff2e4102960 2024-04-26T18:13:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2e1ffb640 1 -- 172.21.15.90:0/3029342427 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d0004020 con 0x7ff2e4102960 2024-04-26T18:13:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.643+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e419e1e0 con 0x7ff2e4102960 2024-04-26T18:13:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.644+0000 7ff2e1ffb640 1 -- 172.21.15.90:0/3029342427 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d00047a0 con 0x7ff2e4102960 2024-04-26T18:13:44.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.644+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e41a5a70 con 0x7ff2e4102960 2024-04-26T18:13:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.647+0000 7ff2e1ffb640 1 -- 172.21.15.90:0/3029342427 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7ff2d0009a30 con 0x7ff2e4102960 2024-04-26T18:13:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 Processor -- start 2024-04-26T18:13:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2e1ffb640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff2d40a3f70 0x7ff2d40a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2e1ffb640 1 -- 172.21.15.90:0/3029342427 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff2d00cfa60 con 0x7ff2e4102960 2024-04-26T18:13:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 -- start start 2024-04-26T18:13:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8a3f00 0x560bcd8a42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8af0e0 0x560bcd8b14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560bcd80a460 con 0x560bcd8a3f00 2024-04-26T18:13:44.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560bcd7c4590 con 0x560bcd8a4820 2024-04-26T18:13:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.648+0000 7ff2eda26740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bcd806420 con 0x560bcd8af0e0 2024-04-26T18:13:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e91c7640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff2d40a3f70 0x7ff2d40a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e89c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8a3f00 0x560bcd8a42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e89c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8a3f00 0x560bcd8a42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44296/0 (socket says 172.21.15.90:44296) 2024-04-26T18:13:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e89c6640 1 -- 172.21.15.90:0/1937079573 learned_addr learned my addr 172.21.15.90:0/1937079573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e3fff640 1 -- 172.21.15.90:0/1937079573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8af0e0 msgr2=0x560bcd8b14d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T18:13:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8af0e0 0x560bcd8b14d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e3fff640 1 -- 172.21.15.90:0/1937079573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8a3f00 msgr2=0x560bcd8a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8a3f00 0x560bcd8a42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e3fff640 1 -- 172.21.15.90:0/1937079573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd7d10d0 con 0x560bcd8a4820 2024-04-26T18:13:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8a3f00 0x560bcd8a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.649+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8aeba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00cd220 tx=0x7ff2d009b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b8187927bfc383 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:44.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.650+0000 7ff2ceffd640 1 -- 172.21.15.90:0/1937079573 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d009fea0 con 0x560bcd8a4820 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.650+0000 7ff2ceffd640 1 -- 172.21.15.90:0/1937079573 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00a4760 con 0x560bcd8a4820 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.650+0000 7ff2ceffd640 1 -- 172.21.15.90:0/1937079573 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0097840 con 0x560bcd8a4820 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 -- 172.21.15.90:0/1937079573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 msgr2=0x560bcd8aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2e91c7640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff2d40a3f70 0x7ff2d40a6430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005fd0 tx=0x7ff2dc009500 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8aeba0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00cd220 tx=0x7ff2d009b9f0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 -- 172.21.15.90:0/1937079573 shutdown_connections 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8af0e0 0x560bcd8b14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8aeba0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1937079573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8a3f00 0x560bcd8a42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 -- 172.21.15.90:0/1937079573 >> 172.21.15.90:0/1937079573 conn(0x560bcd8a0410 msgr2=0x560bcd8a07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 -- 172.21.15.90:0/1937079573 shutdown_connections 2024-04-26T18:13:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.651+0000 7ff2eda26740 1 -- 172.21.15.90:0/1937079573 wait complete. 2024-04-26T18:13:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.652+0000 7ff2eda26740 1 -- start start 2024-04-26T18:13:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.652+0000 7ff2eda26740 1 Processor -- start 2024-04-26T18:13:44.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.652+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8ac680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2eda26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560bcd8b1cb0 con 0x560bcd8af0e0 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2eda26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560bcd791b60 con 0x560bcd8a4820 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2eda26740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560bcd8b1e30 con 0x560bcd8a3f00 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2eda26740 10 client.0 _ll_register_callbacks cb 0x560bcd7b62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e91c7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:43000/0 (socket says 172.21.15.90:43000) 2024-04-26T18:13:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 -- 172.21.15.90:0/1304094715 learned_addr learned my addr 172.21.15.90:0/1304094715 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 -- 172.21.15.90:0/1304094715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a3f00 msgr2=0x560bcd8ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 -- 172.21.15.90:0/1304094715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 msgr2=0x560bcd8acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e3fff640 1 -- 172.21.15.90:0/1304094715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd51c210 con 0x560bcd8a4820 2024-04-26T18:13:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.653+0000 7ff2e91c7640 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8ac680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00cd280 tx=0x7ff2d009ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92f578ce97c8f97a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:44.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:13:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2c3fff640 1 -- 172.21.15.90:0/1304094715 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d009af40 con 0x560bcd8a4820 2024-04-26T18:13:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2c3fff640 1 -- 172.21.15.90:0/1304094715 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff2d009b0e0 con 0x560bcd8a4820 2024-04-26T18:13:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d00a4760 con 0x560bcd8a4820 2024-04-26T18:13:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2c3fff640 1 -- 172.21.15.90:0/1304094715 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2d0082050 con 0x560bcd8a4820 2024-04-26T18:13:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2eda26740 10 client.25088 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560bcd5e8620 con 0x560bcd8a4820 2024-04-26T18:13:44.675 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.654+0000 7ff2c3fff640 1 -- 172.21.15.90:0/1304094715 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00a4760 con 0x560bcd8a4820 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2c3fff640 1 -- 172.21.15.90:0/1304094715 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff2d009aad0 con 0x560bcd8a4820 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2eda26740 10 client.25088 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2eda26740 10 client.25088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2eda26740 20 client.25088 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd8c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2eda26740 4 client.25088 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bcd818070 con 0x560bcd8bef20 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.656+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd8c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.676 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.657+0000 7ff2c3fff640 10 client.25088 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.657+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd8c1310 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6310d32a3eed3600 server_cookie=7c283f0cfbe7a85b in_seq=0 out_seq=0 2024-04-26T18:13:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.657+0000 7ff2c3fff640 1 -- 172.21.15.90:0/1304094715 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bcd818070 con 0x560bcd8bef20 2024-04-26T18:13:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.657+0000 7ff2c3fff640 10 client.25088 handle_command_reply: tid=0 2024-04-26T18:13:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.657+0000 7ff2eda26740 1 client.25088 shutdown 2024-04-26T18:13:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.657+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 msgr2=0x560bcd8ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.657+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8ac680 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00cd280 tx=0x7ff2d009ff70 comp rx=0 tx=0).stop 2024-04-26T18:13:44.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 shutdown_connections 2024-04-26T18:13:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd8c1310 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2c3fff640 0 client.25088 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a4820 0x560bcd8ac680 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1304094715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 >> 172.21.15.90:0/1304094715 conn(0x560bcd8a0410 msgr2=0x560bcd8a6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:44.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 shutdown_connections 2024-04-26T18:13:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 1 -- 172.21.15.90:0/1304094715 wait complete. 2024-04-26T18:13:44.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.658+0000 7ff2eda26740 20 client.25088 trim_cache size 0 max 16384 2024-04-26T18:13:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.686+0000 7ff2eda26740 1 Processor -- start 2024-04-26T18:13:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.686+0000 7ff2eda26740 1 -- start start 2024-04-26T18:13:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8a93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8a9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2eda26740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41ae8c0 con 0x560bcd8af0e0 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2eda26740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41ac190 con 0x560bcd8a3f00 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2eda26740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41aa000 con 0x560bcd8a4820 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8a9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44312/0 (socket says 172.21.15.90:44312) 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 -- 172.21.15.90:0/1802715943 learned_addr learned my addr 172.21.15.90:0/1802715943 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8a93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 -- 172.21.15.90:0/1802715943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 msgr2=0x560bcd8a93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8a93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 -- 172.21.15.90:0/1802715943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 msgr2=0x560bcd8ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e91c7640 1 -- 172.21.15.90:0/1802715943 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bcd8bbb30 con 0x560bcd8af0e0 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.687+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.688+0000 7ff2e91c7640 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8a9920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc003f00 tx=0x7ff2dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bb44c550546297 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.688+0000 7ff2ce7fc640 1 -- 172.21.15.90:0/1802715943 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc02b070 con 0x560bcd8af0e0 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.688+0000 7ff2ce7fc640 1 -- 172.21.15.90:0/1802715943 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc01eb70 con 0x560bcd8af0e0 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.688+0000 7ff2ce7fc640 1 -- 172.21.15.90:0/1802715943 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc026d00 con 0x560bcd8af0e0 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 -- 172.21.15.90:0/1802715943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 msgr2=0x560bcd8a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8a9920 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc003f00 tx=0x7ff2dc017000 comp rx=0 tx=0).stop 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 -- 172.21.15.90:0/1802715943 shutdown_connections 2024-04-26T18:13:44.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd8a9920 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8a93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 --2- 172.21.15.90:0/1802715943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 -- 172.21.15.90:0/1802715943 >> 172.21.15.90:0/1802715943 conn(0x560bcd8a0410 msgr2=0x560bcd8a07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 -- 172.21.15.90:0/1802715943 shutdown_connections 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 -- 172.21.15.90:0/1802715943 wait complete. 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 -- start start 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.689+0000 7ff2eda26740 1 Processor -- start 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2eda26740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd91d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2e91c7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd91d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2e3fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8c1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:48076/0 (socket says 172.21.15.90:48076) 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2e3fff640 1 -- 172.21.15.90:0/2916634225 learned_addr learned my addr 172.21.15.90:0/2916634225 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41aa650 con 0x560bcd8af0e0 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41abac0 con 0x560bcd8a3f00 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e41aad20 con 0x560bcd8a4820 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.690+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2eda26740 10 client.0 _ll_register_callbacks cb 0x7ff2d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2e3fff640 1 -- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 msgr2=0x560bcd8c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2e3fff640 1 -- 172.21.15.90:0/2916634225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 msgr2=0x560bcd91d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd91d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2e3fff640 1 -- 172.21.15.90:0/2916634225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bcd91d770 con 0x560bcd8a4820 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc01eb70 con 0x560bcd8a4820 2024-04-26T18:13:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2e3fff640 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8c1b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2d009ba80 tx=0x7ff2d000df40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45c2c92d57417c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2c3fff640 10 client.25030 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2c3fff640 1 -- 172.21.15.90:0/2916634225 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0097d90 con 0x560bcd8a4820 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2c3fff640 1 -- 172.21.15.90:0/2916634225 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff2d009abb0 con 0x560bcd8a4820 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2c3fff640 1 -- 172.21.15.90:0/2916634225 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff2d00a4760 con 0x560bcd8a4820 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2eda26740 10 client.25030 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.691+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560bcd924970 con 0x560bcd8a4820 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2c3fff640 1 -- 172.21.15.90:0/2916634225 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff2d009af40 con 0x560bcd8a4820 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2eda26740 10 client.25030 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2eda26740 10 client.25030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2eda26740 20 client.25030 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2eda26740 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd926c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2eda26740 4 client.25030 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:13:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bcd811ee0 con 0x560bcd8bef20 2024-04-26T18:13:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.692+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd926c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.693+0000 7ff2c3fff640 10 client.25030 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.693+0000 7ff2e89c6640 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd926c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb0bc3c196bce56 server_cookie=43a135450add00ea in_seq=0 out_seq=0 2024-04-26T18:13:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.697+0000 7ff2c3fff640 1 -- 172.21.15.90:0/2916634225 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bcd811ee0 con 0x560bcd8bef20 2024-04-26T18:13:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.697+0000 7ff2c3fff640 10 client.25030 handle_command_reply: tid=0 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.697+0000 7ff2eda26740 1 client.25030 shutdown 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 msgr2=0x560bcd8c1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8c1b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2d009ba80 tx=0x7ff2d000df40 comp rx=0 tx=0).stop 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 shutdown_connections 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560bcd8bef20 0x560bcd926c80 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560bcd8af0e0 0x560bcd91d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2c3fff640 0 client.25030 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560bcd8a4820 0x560bcd8c1b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 --2- 172.21.15.90:0/2916634225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560bcd8a3f00 0x560bcd8c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 >> 172.21.15.90:0/2916634225 conn(0x560bcd8a0410 msgr2=0x560bcd89ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 shutdown_connections 2024-04-26T18:13:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.698+0000 7ff2eda26740 1 -- 172.21.15.90:0/2916634225 wait complete. 2024-04-26T18:13:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.699+0000 7ff2eda26740 20 client.25030 trim_cache size 0 max 16384 2024-04-26T18:13:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:13:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:13:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:13:44.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.699+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff2d40a3f70 msgr2=0x7ff2d40a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.699+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff2d40a3f70 0x7ff2d40a6430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005fd0 tx=0x7ff2dc009500 comp rx=0 tx=0).stop 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.699+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4102960 msgr2=0x7ff2e4199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.699+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4102960 0x7ff2e4199490 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000d950 tx=0x7ff2d000de10 comp rx=0 tx=0).stop 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.699+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 shutdown_connections 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.700+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff2d40a3f70 0x7ff2d40a6430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.700+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2e4107890 0x7ff2e419d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.700+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff2e4102960 0x7ff2e4199490 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.700+0000 7ff2eac2a640 1 --2- 172.21.15.90:0/3029342427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff2e4101fa0 0x7ff2e4198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.700+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 >> 172.21.15.90:0/3029342427 conn(0x7ff2e4077dc0 msgr2=0x7ff2e40ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.700+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 shutdown_connections 2024-04-26T18:13:44.704 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:44.700+0000 7ff2eac2a640 1 -- 172.21.15.90:0/3029342427 wait complete. 2024-04-26T18:13:44.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:44 smithi119 ceph-mon[32752]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:13:44.869 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:44 smithi167 ceph-mon[32751]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:13:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:44 smithi090 ceph-mon[29180]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:13:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:46 smithi090 ceph-mon[29180]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:13:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:46 smithi119 ceph-mon[32752]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:13:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:46 smithi167 ceph-mon[32751]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:13:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:48 smithi090 ceph-mon[29180]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:13:49.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:48 smithi119 ceph-mon[32752]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:13:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:48 smithi167 ceph-mon[32751]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T18:13:50.032 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'. 2024-04-26T18:13:50.032 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T18:13:50.032 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T18:13:50.032 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T18:13:50.032 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T18:13:50.033 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at bc94a2a924f Merge commit 'refs/pull/57099/head' of github.com:ceph/ceph into quiescer 2024-04-26T18:13:50.037 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T18:13:50.087 INFO:tasks.workunit.client.0.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T18:13:50.089 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T18:13:50.089 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T18:13:50.810 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T18:13:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:50 smithi090 ceph-mon[29180]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:13:50.888 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T18:13:50.927 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T18:13:50.928 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T18:13:50.929 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T18:13:50.968 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T18:13:50.971 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T18:13:50.971 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T18:13:50.986 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-26T18:13:50.987 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-26T18:13:50.988 DEBUG:teuthology.orchestra.run.smithi090:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-26T18:13:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:50 smithi119 ceph-mon[32752]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:13:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:50 smithi167 ceph-mon[32751]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:13:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:52 smithi090 ceph-mon[29180]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:52 smithi119 ceph-mon[32752]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:52 smithi167 ceph-mon[32751]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-26T18:13:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:54 smithi090 ceph-mon[29180]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:13:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:54 smithi119 ceph-mon[32752]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:13:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:54 smithi167 ceph-mon[32751]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 341 B/s wr, 8 op/s 2024-04-26T18:13:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:56 smithi090 ceph-mon[29180]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:13:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:56 smithi119 ceph-mon[32752]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:13:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:56 smithi167 ceph-mon[32751]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 -- 172.21.15.90:0/3651713960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d38101b80 msgr2=0x7f4d38102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 --2- 172.21.15.90:0/3651713960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d38101b80 0x7f4d38102000 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009900 tx=0x7f4d2802a3f0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 -- 172.21.15.90:0/3651713960 shutdown_connections 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 --2- 172.21.15.90:0/3651713960 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38102540 0x7f4d3810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 --2- 172.21.15.90:0/3651713960 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d38101b80 0x7f4d38102000 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 --2- 172.21.15.90:0/3651713960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d381067d0 0x7f4d38106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 -- 172.21.15.90:0/3651713960 >> 172.21.15.90:0/3651713960 conn(0x7f4d38077630 msgr2=0x7f4d38077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 -- 172.21.15.90:0/3651713960 shutdown_connections 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.796+0000 7f4d3faea640 1 -- 172.21.15.90:0/3651713960 wait complete. 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 Processor -- start 2024-04-26T18:13:57.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 -- start start 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38101b80 0x7f4d38198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d38102540 0x7f4d38199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d381067d0 0x7f4d3819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38111ab0 con 0x7f4d381067d0 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38111930 con 0x7f4d38101b80 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3faea640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d38111c30 con 0x7f4d38102540 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3e087640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d381067d0 0x7f4d3819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3e087640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d381067d0 0x7f4d3819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47966/0 (socket says 172.21.15.90:47966) 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.797+0000 7f4d3e087640 1 -- 172.21.15.90:0/1667024451 learned_addr learned my addr 172.21.15.90:0/1667024451 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3d886640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38101b80 0x7f4d38198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3e087640 1 -- 172.21.15.90:0/1667024451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d38102540 msgr2=0x7f4d38199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3e087640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d38102540 0x7f4d38199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3e087640 1 -- 172.21.15.90:0/1667024451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38101b80 msgr2=0x7f4d38198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3e087640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38101b80 0x7f4d38198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3e087640 1 -- 172.21.15.90:0/1667024451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3819def0 con 0x7f4d381067d0 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3e087640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d381067d0 0x7f4d3819d7c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000cce0 tx=0x7f4d3000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d2effd640 1 -- 172.21.15.90:0/1667024451 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000f450 con 0x7f4d381067d0 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d2effd640 1 -- 172.21.15.90:0/1667024451 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d3000faa0 con 0x7f4d381067d0 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d2effd640 1 -- 172.21.15.90:0/1667024451 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30018b70 con 0x7f4d381067d0 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3819e1e0 con 0x7f4d381067d0 2024-04-26T18:13:57.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.798+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d381a5a70 con 0x7f4d381067d0 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.801+0000 7f4d2effd640 1 -- 172.21.15.90:0/1667024451 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f4d30018d10 con 0x7f4d381067d0 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.801+0000 7f4d42802740 1 Processor -- start 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d42802740 1 -- start start 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d42802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x55881417d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d42802740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x55881418bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d42802740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881418c4f0 0x55881418e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d42802740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5588140e3460 con 0x55881418c4f0 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d42802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55881409d590 con 0x55881417ce20 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d42802740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588140df420 con 0x55881417d740 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d2effd640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4d240a3dc0 0x7f4d240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881418c4f0 0x55881418e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x55881417d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x55881417d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:54266/0 (socket says 172.21.15.90:54266) 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 -- 172.21.15.90:0/1318562559 learned_addr learned my addr 172.21.15.90:0/1318562559 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d886640 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x55881418bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3e087640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4d240a3dc0 0x7f4d240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d2effd640 1 -- 172.21.15.90:0/1667024451 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4d300d0c90 con 0x7f4d381067d0 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 -- 172.21.15.90:0/1318562559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 msgr2=0x55881418bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x55881418bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 -- 172.21.15.90:0/1318562559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881418c4f0 msgr2=0x55881418e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881418c4f0 0x55881418e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.802+0000 7f4d3d085640 1 -- 172.21.15.90:0/1318562559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588140aa0d0 con 0x55881417ce20 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.803+0000 7f4d3e087640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4d240a3dc0 0x7f4d240a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d300cea50 tx=0x7f4d30092130 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:57.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.803+0000 7f4d3d085640 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x55881417d200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d2802a910 tx=0x7f4d28031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a69256cb0be9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.803+0000 7f4d23fff640 1 -- 172.21.15.90:0/1318562559 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28044070 con 0x55881417ce20 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.803+0000 7f4d23fff640 1 -- 172.21.15.90:0/1318562559 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d280035f0 con 0x55881417ce20 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.803+0000 7f4d23fff640 1 -- 172.21.15.90:0/1318562559 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28035070 con 0x55881417ce20 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 -- 172.21.15.90:0/1318562559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 msgr2=0x55881417d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x55881417d200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d2802a910 tx=0x7f4d28031400 comp rx=0 tx=0).stop 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 -- 172.21.15.90:0/1318562559 shutdown_connections 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881418c4f0 0x55881418e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x55881418bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 --2- 172.21.15.90:0/1318562559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x55881417d200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 -- 172.21.15.90:0/1318562559 >> 172.21.15.90:0/1318562559 conn(0x558814179410 msgr2=0x5588141797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 -- 172.21.15.90:0/1318562559 shutdown_connections 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 -- 172.21.15.90:0/1318562559 wait complete. 2024-04-26T18:13:57.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 -- start start 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 Processor -- start 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x558814189490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417d740 0x5588141899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.804+0000 7f4d42802740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 0x558814189f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d42802740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55881418f0c0 con 0x55881417d740 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d42802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55881406ab60 con 0x55881417ce20 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d42802740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55881418f240 con 0x55881418c4f0 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d42802740 10 client.0 _ll_register_callbacks cb 0x55881408f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d886640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417d740 0x5588141899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x558814189490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x558814189490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:54282/0 (socket says 172.21.15.90:54282) 2024-04-26T18:13:57.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 -- 172.21.15.90:0/3637559373 learned_addr learned my addr 172.21.15.90:0/3637559373 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 0x558814189f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 -- 172.21.15.90:0/3637559373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 msgr2=0x558814189f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 0x558814189f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 -- 172.21.15.90:0/3637559373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417d740 msgr2=0x5588141899d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417d740 0x5588141899d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 -- 172.21.15.90:0/3637559373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558813df5210 con 0x55881417ce20 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d3d085640 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x558814189490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d28003d40 tx=0x7f4d28003f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc02cb1d06f64fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d217fa640 1 -- 172.21.15.90:0/3637559373 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2804e020 con 0x55881417ce20 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d280035f0 con 0x55881417ce20 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d217fa640 1 -- 172.21.15.90:0/3637559373 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4d28004200 con 0x55881417ce20 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d217fa640 1 -- 172.21.15.90:0/3637559373 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d28044070 con 0x55881417ce20 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d42802740 10 client.25103 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:57.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.805+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558813ec1620 con 0x55881417ce20 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d217fa640 1 -- 172.21.15.90:0/3637559373 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d280035f0 con 0x55881417ce20 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d217fa640 1 -- 172.21.15.90:0/3637559373 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4d2803a040 con 0x55881417ce20 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d42802740 10 client.25103 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d42802740 10 client.25103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d42802740 20 client.25103 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d42802740 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55881419c500 0x55881419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d42802740 4 client.25103 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588140f1070 con 0x55881419c500 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.806+0000 7f4d3d886640 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55881419c500 0x55881419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d217fa640 10 client.25103 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:57.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d3d886640 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55881419c500 0x55881419e8f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f9ea1e7f0cf616 server_cookie=cc4cd0eb5d2048a0 in_seq=0 out_seq=0 2024-04-26T18:13:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d217fa640 1 -- 172.21.15.90:0/3637559373 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588140f1070 con 0x55881419c500 2024-04-26T18:13:57.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d217fa640 10 client.25103 handle_command_reply: tid=0 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 client.25103 shutdown 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 msgr2=0x558814189490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x558814189490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d28003d40 tx=0x7f4d28003f50 comp rx=0 tx=0).stop 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 shutdown_connections 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55881419c500 0x55881419e8f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 0x558814189f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417d740 0x5588141899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 --2- 172.21.15.90:0/3637559373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417ce20 0x558814189490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d217fa640 0 client.25103 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.807+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 >> 172.21.15.90:0/3637559373 conn(0x558814179410 msgr2=0x55881417f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.808+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 shutdown_connections 2024-04-26T18:13:57.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.808+0000 7f4d42802740 1 -- 172.21.15.90:0/3637559373 wait complete. 2024-04-26T18:13:57.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.808+0000 7f4d42802740 20 client.25103 trim_cache size 0 max 16384 2024-04-26T18:13:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.822+0000 7f4d42802740 1 Processor -- start 2024-04-26T18:13:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.822+0000 7f4d42802740 1 -- start start 2024-04-26T18:13:57.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d42802740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x558814197f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d42802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417d740 0x55881417f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d42802740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 0x5588141a3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d42802740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381ae7e0 con 0x55881417ce20 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d42802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381ac0b0 con 0x55881417d740 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d42802740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381a9f20 con 0x55881418c4f0 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x558814197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x558814197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48004/0 (socket says 172.21.15.90:48004) 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 -- 172.21.15.90:0/1562443629 learned_addr learned my addr 172.21.15.90:0/1562443629 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 -- 172.21.15.90:0/1562443629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 msgr2=0x5588141a3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 --2- 172.21.15.90:0/1562443629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 0x5588141a3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 -- 172.21.15.90:0/1562443629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417d740 msgr2=0x55881417f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 --2- 172.21.15.90:0/1562443629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417d740 0x55881417f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 -- 172.21.15.90:0/1562443629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558814188440 con 0x55881417ce20 2024-04-26T18:13:57.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d3d085640 1 --2- 172.21.15.90:0/1562443629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x558814197f80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4d2803bbf0 tx=0x7f4d2804a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9f147883785190 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d237fe640 1 -- 172.21.15.90:0/1562443629 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28000e60 con 0x55881417ce20 2024-04-26T18:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d237fe640 1 -- 172.21.15.90:0/1562443629 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d280035f0 con 0x55881417ce20 2024-04-26T18:13:57.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.823+0000 7f4d237fe640 1 -- 172.21.15.90:0/1562443629 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2804e070 con 0x55881417ce20 2024-04-26T18:13:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 -- 172.21.15.90:0/1562443629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 msgr2=0x558814197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 --2- 172.21.15.90:0/1562443629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x558814197f80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4d2803bbf0 tx=0x7f4d2804a400 comp rx=0 tx=0).stop 2024-04-26T18:13:57.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 -- 172.21.15.90:0/1562443629 shutdown_connections 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 --2- 172.21.15.90:0/1562443629 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881418c4f0 0x5588141a3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 --2- 172.21.15.90:0/1562443629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881417d740 0x55881417f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 --2- 172.21.15.90:0/1562443629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x558814197f80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 -- 172.21.15.90:0/1562443629 >> 172.21.15.90:0/1562443629 conn(0x558814179410 msgr2=0x5588141797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 -- 172.21.15.90:0/1562443629 shutdown_connections 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 -- 172.21.15.90:0/1562443629 wait complete. 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 -- start start 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.824+0000 7f4d42802740 1 Processor -- start 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x55881418fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x5588141903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881418c4f0 0x5588141908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381aa570 con 0x55881417ce20 2024-04-26T18:13:57.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381ab9e0 con 0x55881418c4f0 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4d381aac40 con 0x55881417d740 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 10 client.0 _ll_register_callbacks cb 0x7f4d28031c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x55881418fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x55881418fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48016/0 (socket says 172.21.15.90:48016) 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 -- 172.21.15.90:0/265530887 learned_addr learned my addr 172.21.15.90:0/265530887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881418c4f0 0x5588141908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d886640 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x5588141903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 -- 172.21.15.90:0/265530887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 msgr2=0x5588141903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x5588141903b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 -- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881418c4f0 msgr2=0x5588141908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881418c4f0 0x5588141908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 -- 172.21.15.90:0/265530887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558814201700 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d3d085640 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x55881418fe70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4d28006010 tx=0x7f4d28031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6f2dc648929b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d217fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d217fa640 1 -- 172.21.15.90:0/265530887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2804e050 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d217fa640 1 -- 172.21.15.90:0/265530887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4d2803b400 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.825+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d280035f0 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d217fa640 1 -- 172.21.15.90:0/265530887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4d28035040 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 10 client.15543 fetch_fsmap learned FSMap version 26 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558814202920 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d217fa640 1 -- 172.21.15.90:0/265530887 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d280035f0 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d217fa640 1 -- 172.21.15.90:0/265530887 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4d28065030 con 0x55881417ce20 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 10 client.15543 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:13:57.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 20 client.15543 populate_metadata read hostname 'smithi090' 2024-04-26T18:13:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x558814202d40 0x558814205130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:13:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 4 client.15543 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:13:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588140eaee0 con 0x558814202d40 2024-04-26T18:13:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.826+0000 7f4d3d886640 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x558814202d40 0x558814205130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:13:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.827+0000 7f4d3d886640 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x558814202d40 0x558814205130 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97861881eb50028c server_cookie=2fc8f8452723c742 in_seq=0 out_seq=0 2024-04-26T18:13:57.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.827+0000 7f4d217fa640 10 client.15543 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d217fa640 1 -- 172.21.15.90:0/265530887 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588140eaee0 con 0x558814202d40 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d217fa640 10 client.15543 handle_command_reply: tid=0 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 client.15543 shutdown 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 msgr2=0x55881418fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x55881418fe70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4d28006010 tx=0x7f4d28031400 comp rx=0 tx=0).stop 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 shutdown_connections 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x558814202d40 0x558814205130 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55881418c4f0 0x5588141908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55881417d740 0x5588141903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d217fa640 0 client.15543 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 --2- 172.21.15.90:0/265530887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55881417ce20 0x55881418fe70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 >> 172.21.15.90:0/265530887 conn(0x558814179410 msgr2=0x55881418b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.833+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 shutdown_connections 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d42802740 1 -- 172.21.15.90:0/265530887 wait complete. 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d42802740 20 client.15543 trim_cache size 0 max 16384 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:13:57.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:13:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4d240a3dc0 msgr2=0x7f4d240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4d240a3dc0 0x7f4d240a6280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4d300cea50 tx=0x7f4d30092130 comp rx=0 tx=0).stop 2024-04-26T18:13:57.836 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d381067d0 msgr2=0x7f4d3819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d381067d0 0x7f4d3819d7c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000cce0 tx=0x7f4d3000a620 comp rx=0 tx=0).stop 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 shutdown_connections 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4d240a3dc0 0x7f4d240a6280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4d381067d0 0x7f4d3819d7c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4d38102540 0x7f4d38199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 --2- 172.21.15.90:0/1667024451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d38101b80 0x7f4d38198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 >> 172.21.15.90:0/1667024451 conn(0x7f4d38077630 msgr2=0x7f4d381053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 shutdown_connections 2024-04-26T18:13:57.837 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:13:57.834+0000 7f4d3faea640 1 -- 172.21.15.90:0/1667024451 wait complete. 2024-04-26T18:13:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:13:58 smithi090 ceph-mon[29180]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:13:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:13:58 smithi119 ceph-mon[32752]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:13:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:13:58 smithi167 ceph-mon[32751]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-26T18:14:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:00 smithi090 ceph-mon[29180]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:14:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:00 smithi119 ceph-mon[32752]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:14:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:00 smithi167 ceph-mon[32751]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:14:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:02 smithi090 ceph-mon[29180]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-26T18:14:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:02 smithi119 ceph-mon[32752]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-26T18:14:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:02 smithi167 ceph-mon[32751]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-26T18:14:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:04 smithi090 ceph-mon[29180]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:14:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:04 smithi119 ceph-mon[32752]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:14:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:14:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:04 smithi167 ceph-mon[32751]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:14:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:14:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:14:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:14:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:14:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:06 smithi090 ceph-mon[29180]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:14:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:06 smithi119 ceph-mon[32752]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:14:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:06 smithi167 ceph-mon[32751]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:14:07.562 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:14:07.814 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:08 smithi090 ceph-mon[29180]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:14:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:08 smithi119 ceph-mon[32752]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:14:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:08 smithi167 ceph-mon[32751]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:14:09.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 -- 172.21.15.90:0/3033208312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00581094c0 msgr2=0x7f00581098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 --2- 172.21.15.90:0/3033208312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00581094c0 0x7f00581098a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0054009990 tx=0x7f0054009ca0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 -- 172.21.15.90:0/3033208312 shutdown_connections 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 --2- 172.21.15.90:0/3033208312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f00581094c0 0x7f00581098a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 --2- 172.21.15.90:0/3033208312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0058103e30 0x7f0058110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 --2- 172.21.15.90:0/3033208312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0058103470 0x7f00581038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 -- 172.21.15.90:0/3033208312 >> 172.21.15.90:0/3033208312 conn(0x7f005807a580 msgr2=0x7f005807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 -- 172.21.15.90:0/3033208312 shutdown_connections 2024-04-26T18:14:09.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.562+0000 7f005fe45700 1 -- 172.21.15.90:0/3033208312 wait complete. 2024-04-26T18:14:09.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 Processor -- start 2024-04-26T18:14:09.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 -- start start 2024-04-26T18:14:09.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0058103470 0x7f005819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:09.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058103e30 0x7f005819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:09.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00581094c0 0x7f00581998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:09.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f00581135f0 con 0x7f0058103e30 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0058113470 con 0x7f0058103470 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005fe45700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0058113770 con 0x7f00581094c0 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005d3e0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058103e30 0x7f005819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005d3e0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058103e30 0x7f005819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51798/0 (socket says 172.21.15.90:51798) 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.563+0000 7f005d3e0700 1 -- 172.21.15.90:0/41724415 learned_addr learned my addr 172.21.15.90:0/41724415 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005dbe1700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0058103470 0x7f005819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005e3e2700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00581094c0 0x7f00581998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005d3e0700 1 -- 172.21.15.90:0/41724415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00581094c0 msgr2=0x7f00581998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005d3e0700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00581094c0 0x7f00581998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005d3e0700 1 -- 172.21.15.90:0/41724415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0058103470 msgr2=0x7f005819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005d3e0700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0058103470 0x7f005819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005d3e0700 1 -- 172.21.15.90:0/41724415 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005819a050 con 0x7f0058103e30 2024-04-26T18:14:09.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005e3e2700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00581094c0 0x7f00581998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:14:09.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005d3e0700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058103e30 0x7f005819fc70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f004800d980 tx=0x7f004800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:09.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f004effd700 1 -- 172.21.15.90:0/41724415 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048017070 con 0x7f0058103e30 2024-04-26T18:14:09.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005819a340 con 0x7f0058103e30 2024-04-26T18:14:09.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f004effd700 1 -- 172.21.15.90:0/41724415 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0048005020 con 0x7f0058103e30 2024-04-26T18:14:09.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f004effd700 1 -- 172.21.15.90:0/41724415 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048009b50 con 0x7f0058103e30 2024-04-26T18:14:09.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.564+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0058075650 con 0x7f0058103e30 2024-04-26T18:14:09.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.565+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005810b930 con 0x7f0058103e30 2024-04-26T18:14:09.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.566+0000 7f004effd700 1 -- 172.21.15.90:0/41724415 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f00480051d0 con 0x7f0058103e30 2024-04-26T18:14:09.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.567+0000 7f004effd700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f00440a81c0 0x7f00440aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:09.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.567+0000 7f005dbe1700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f00440a81c0 0x7f00440aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:09.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.567+0000 7f004effd700 1 -- 172.21.15.90:0/41724415 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f00480d06e0 con 0x7f0058103e30 2024-04-26T18:14:09.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.567+0000 7f005dbe1700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f00440a81c0 0x7f00440aa680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f005400b5c0 tx=0x7f0054005960 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:09.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.572+0000 7f004effd700 1 -- 172.21.15.90:0/41724415 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0048020020 con 0x7f0058103e30 2024-04-26T18:14:09.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.820+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f005819ae20 con 0x7f0058103e30 2024-04-26T18:14:09.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.821+0000 7f004effd700 1 -- 172.21.15.90:0/41724415 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11874 (secure 0 0 0) 0x7f004809cdf0 con 0x7f0058103e30 2024-04-26T18:14:09.821 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:14:09.821 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:11:56.125308+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:14:09.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f00440a81c0 msgr2=0x7f00440aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:09.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f00440a81c0 0x7f00440aa680 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f005400b5c0 tx=0x7f0054005960 comp rx=0 tx=0).stop 2024-04-26T18:14:09.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058103e30 msgr2=0x7f005819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058103e30 0x7f005819fc70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f004800d980 tx=0x7f004800dc90 comp rx=0 tx=0).stop 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 shutdown_connections 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f00440a81c0 0x7f00440aa680 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f00581094c0 0x7f00581998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0058103470 0x7f005819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 --2- 172.21.15.90:0/41724415 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0058103e30 0x7f005819fc70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 >> 172.21.15.90:0/41724415 conn(0x7f005807a580 msgr2=0x7f00581021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 shutdown_connections 2024-04-26T18:14:09.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:09.823+0000 7f005fe45700 1 -- 172.21.15.90:0/41724415 wait complete. 2024-04-26T18:14:09.825 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T18:14:10.498 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db --set-id e986ca9 --release --await' 2024-04-26T18:14:10.498 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db --set-id e986ca9 --release --await 2024-04-26T18:14:10.748 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:14:10.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:10 smithi090 ceph-mon[29180]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:14:10.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:10 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/41724415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:14:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 -- 172.21.15.90:0/901418870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c1017f0 msgr2=0x7fd72c10f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 --2- 172.21.15.90:0/901418870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c1017f0 0x7fd72c10f7c0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd724009cd0 tx=0x7fd72402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 -- 172.21.15.90:0/901418870 shutdown_connections 2024-04-26T18:14:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 --2- 172.21.15.90:0/901418870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c1017f0 0x7fd72c10f7c0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 --2- 172.21.15.90:0/901418870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd72c100e30 0x7fd72c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 --2- 172.21.15.90:0/901418870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd72c107910 0x7fd72c107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 -- 172.21.15.90:0/901418870 >> 172.21.15.90:0/901418870 conn(0x7fd72c077dc0 msgr2=0x7fd72c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 -- 172.21.15.90:0/901418870 shutdown_connections 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 -- 172.21.15.90:0/901418870 wait complete. 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 Processor -- start 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.926+0000 7fd733a50640 1 -- start start 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c100e30 0x7fd72c199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd72c1017f0 0x7fd72c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd72c107910 0x7fd72c19d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c112cd0 con 0x7fd72c100e30 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c112b50 con 0x7fd72c107910 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c112e50 con 0x7fd72c1017f0 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c100e30 0x7fd72c199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c100e30 0x7fd72c199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51806/0 (socket says 172.21.15.90:51806) 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 -- 172.21.15.90:0/3027512831 learned_addr learned my addr 172.21.15.90:0/3027512831 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd730feb640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd72c1017f0 0x7fd72c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd731fed640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd72c107910 0x7fd72c19d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 -- 172.21.15.90:0/3027512831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd72c1017f0 msgr2=0x7fd72c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd72c1017f0 0x7fd72c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 -- 172.21.15.90:0/3027512831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd72c107910 msgr2=0x7fd72c19d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd72c107910 0x7fd72c19d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 -- 172.21.15.90:0/3027512831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c19e060 con 0x7fd72c100e30 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd730feb640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd72c1017f0 0x7fd72c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7317ec640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c100e30 0x7fd72c199070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd72800d770 tx=0x7fd72800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7227fc640 1 -- 172.21.15.90:0/3027512831 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728017070 con 0x7fd72c100e30 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c19e350 con 0x7fd72c100e30 2024-04-26T18:14:10.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7227fc640 1 -- 172.21.15.90:0/3027512831 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7280054d0 con 0x7fd72c100e30 2024-04-26T18:14:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd7227fc640 1 -- 172.21.15.90:0/3027512831 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728004c30 con 0x7fd72c100e30 2024-04-26T18:14:10.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.927+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c1a5be0 con 0x7fd72c100e30 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.930+0000 7fd7227fc640 1 -- 172.21.15.90:0/3027512831 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fd728004e10 con 0x7fd72c100e30 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.930+0000 7fd736768740 1 Processor -- start 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.930+0000 7fd736768740 1 -- start start 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd736768740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c45e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd736768740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c468ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd736768740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c4690e0 0x561a5c46b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd736768740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561a5c3c4460 con 0x561a5c4690e0 2024-04-26T18:14:10.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd736768740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561a5c37e590 con 0x561a5c45df00 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd736768740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a5c3c0420 con 0x561a5c45e820 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd7227fc640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd7180a3dc0 0x7fd7180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c4690e0 0x561a5c46b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c4690e0 0x561a5c46b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51814/0 (socket says 172.21.15.90:51814) 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 -- 172.21.15.90:0/1791021626 learned_addr learned my addr 172.21.15.90:0/1791021626 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd731fed640 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c45e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd731fed640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd7180a3dc0 0x7fd7180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd7317ec640 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c468ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd7227fc640 1 -- 172.21.15.90:0/3027512831 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd7280d0370 con 0x7fd72c100e30 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 -- 172.21.15.90:0/1791021626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 msgr2=0x561a5c468ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c468ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 -- 172.21.15.90:0/1791021626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 msgr2=0x561a5c45e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c45e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 -- 172.21.15.90:0/1791021626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5c38b0d0 con 0x561a5c4690e0 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd730feb640 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c4690e0 0x561a5c46b4d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd71c008c10 tx=0x7fd71c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cc82c05207aed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.931+0000 7fd731fed640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd7180a3dc0 0x7fd7180a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd724033040 tx=0x7fd7240095c0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd7177fe640 1 -- 172.21.15.90:0/1791021626 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c014070 con 0x561a5c4690e0 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd7177fe640 1 -- 172.21.15.90:0/1791021626 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd71c0047a0 con 0x561a5c4690e0 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd7177fe640 1 -- 172.21.15.90:0/1791021626 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c010690 con 0x561a5c4690e0 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 -- 172.21.15.90:0/1791021626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c4690e0 msgr2=0x561a5c46b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c4690e0 0x561a5c46b4d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd71c008c10 tx=0x7fd71c00e720 comp rx=0 tx=0).stop 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 -- 172.21.15.90:0/1791021626 shutdown_connections 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c4690e0 0x561a5c46b4d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c468ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 --2- 172.21.15.90:0/1791021626 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c45e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 -- 172.21.15.90:0/1791021626 >> 172.21.15.90:0/1791021626 conn(0x561a5c45a410 msgr2=0x561a5c45a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 -- 172.21.15.90:0/1791021626 shutdown_connections 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.932+0000 7fd736768740 1 -- 172.21.15.90:0/1791021626 wait complete. 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 -- start start 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 Processor -- start 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c466070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45e820 0x561a5c4665b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 0x561a5c466af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561a5c46bd10 con 0x561a5c45e820 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561a5c34bb60 con 0x561a5c45df00 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a5c46be90 con 0x561a5c4690e0 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd736768740 10 client.0 _ll_register_callbacks cb 0x561a5c3702b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd730feb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45e820 0x561a5c4665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd730feb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45e820 0x561a5c4665b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51830/0 (socket says 172.21.15.90:51830) 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.933+0000 7fd730feb640 1 -- 172.21.15.90:0/1084879395 learned_addr learned my addr 172.21.15.90:0/1084879395 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd7317ec640 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c466070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd730feb640 1 -- 172.21.15.90:0/1084879395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 msgr2=0x561a5c466af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd730feb640 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 0x561a5c466af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd730feb640 1 -- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 msgr2=0x561a5c466070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd730feb640 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c466070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd730feb640 1 -- 172.21.15.90:0/1084879395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5c0d6210 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd730feb640 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45e820 0x561a5c4665b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd71c010b40 tx=0x7fd71c010b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12046e4b79f3505 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd714ff9640 1 -- 172.21.15.90:0/1084879395 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c014030 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd714ff9640 1 -- 172.21.15.90:0/1084879395 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd71c004050 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd71c0047a0 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd714ff9640 1 -- 172.21.15.90:0/1084879395 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd71c004250 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd736768740 10 client.15561 fetch_fsmap learned FSMap version 26 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a5c1a2620 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.934+0000 7fd714ff9640 1 -- 172.21.15.90:0/1084879395 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd71c0047a0 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd714ff9640 1 -- 172.21.15.90:0/1084879395 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd71c023420 con 0x561a5c45e820 2024-04-26T18:14:10.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd736768740 10 client.15561 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd736768740 10 client.15561 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd736768740 20 client.15561 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd736768740 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c47f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd736768740 4 client.15561 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a5c3d2070 con 0x561a5c47d510 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd7317ec640 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c47f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd714ff9640 10 client.15561 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.935+0000 7fd7317ec640 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c47f900 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=584eb5cf4dcb1da9 server_cookie=114581cb8f8507f in_seq=0 out_seq=0 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd714ff9640 1 -- 172.21.15.90:0/1084879395 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a5c3d2070 con 0x561a5c47d510 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd714ff9640 10 client.15561 handle_command_reply: tid=0 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 client.15561 shutdown 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45e820 msgr2=0x561a5c4665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45e820 0x561a5c4665b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd71c010b40 tx=0x7fd71c010b70 comp rx=0 tx=0).stop 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 shutdown_connections 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c47f900 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 0x561a5c466af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45e820 0x561a5c4665b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd714ff9640 0 client.15561 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 --2- 172.21.15.90:0/1084879395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45df00 0x561a5c466070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 >> 172.21.15.90:0/1084879395 conn(0x561a5c45a410 msgr2=0x561a5c460140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 shutdown_connections 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.936+0000 7fd736768740 1 -- 172.21.15.90:0/1084879395 wait complete. 2024-04-26T18:14:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.937+0000 7fd736768740 20 client.15561 trim_cache size 0 max 16384 2024-04-26T18:14:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.951+0000 7fd736768740 1 Processor -- start 2024-04-26T18:14:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.951+0000 7fd736768740 1 -- start start 2024-04-26T18:14:10.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.951+0000 7fd736768740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c474bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.951+0000 7fd736768740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c45a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.951+0000 7fd736768740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c4690e0 0x561a5c485100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd736768740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1ae980 con 0x561a5c45df00 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd736768740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1ac250 con 0x561a5c4690e0 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd736768740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1aa0c0 con 0x561a5c45e820 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c474bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c474bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51840/0 (socket says 172.21.15.90:51840) 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 -- 172.21.15.90:0/2723214979 learned_addr learned my addr 172.21.15.90:0/2723214979 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd730feb640 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c45a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd731fed640 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c4690e0 0x561a5c485100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 -- 172.21.15.90:0/2723214979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 msgr2=0x561a5c45a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c45a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 -- 172.21.15.90:0/2723214979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c4690e0 msgr2=0x561a5c485100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c4690e0 0x561a5c485100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 -- 172.21.15.90:0/2723214979 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a5c465040 con 0x561a5c45df00 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd7317ec640 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c474bd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd728098780 tx=0x7fd728016310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d72936f64bc426 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd716ffd640 1 -- 172.21.15.90:0/2723214979 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7280a06d0 con 0x561a5c45df00 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd716ffd640 1 -- 172.21.15.90:0/2723214979 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7280a4f90 con 0x561a5c45df00 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.952+0000 7fd716ffd640 1 -- 172.21.15.90:0/2723214979 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7280cdd50 con 0x561a5c45df00 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 -- 172.21.15.90:0/2723214979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 msgr2=0x561a5c474bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c474bd0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd728098780 tx=0x7fd728016310 comp rx=0 tx=0).stop 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 -- 172.21.15.90:0/2723214979 shutdown_connections 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c4690e0 0x561a5c485100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c45e820 0x561a5c45a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 --2- 172.21.15.90:0/2723214979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c474bd0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 -- 172.21.15.90:0/2723214979 >> 172.21.15.90:0/2723214979 conn(0x561a5c45a410 msgr2=0x561a5c468130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 -- 172.21.15.90:0/2723214979 shutdown_connections 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 -- 172.21.15.90:0/2723214979 wait complete. 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 -- start start 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.953+0000 7fd736768740 1 Processor -- start 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd736768740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c462e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd736768740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45e820 0x561a5c4633c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd736768740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 0x561a5c463900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd736768740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1aa710 con 0x561a5c45df00 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd736768740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1abb80 con 0x561a5c45e820 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd736768740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1aade0 con 0x561a5c4690e0 2024-04-26T18:14:10.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd736768740 10 client.0 _ll_register_callbacks cb 0x7fd724009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c462e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c462e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51848/0 (socket says 172.21.15.90:51848) 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 -- 172.21.15.90:0/600341546 learned_addr learned my addr 172.21.15.90:0/600341546 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd730feb640 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45e820 0x561a5c4633c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd731fed640 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 0x561a5c463900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 -- 172.21.15.90:0/600341546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 msgr2=0x561a5c463900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 0x561a5c463900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 -- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45e820 msgr2=0x561a5c4633c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45e820 0x561a5c4633c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.954+0000 7fd7317ec640 1 -- 172.21.15.90:0/600341546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a5c4e2aa0 con 0x561a5c45df00 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd7317ec640 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c462e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd728021000 tx=0x7fd7280a0740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42eeb6f600a2105d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:10.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd714ff9640 1 -- 172.21.15.90:0/600341546 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728020070 con 0x561a5c45df00 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd714ff9640 1 -- 172.21.15.90:0/600341546 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fd7280a4b00 con 0x561a5c45df00 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7280a4f90 con 0x561a5c45df00 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd714ff9640 1 -- 172.21.15.90:0/600341546 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7280cdf50 con 0x561a5c45df00 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 10 client.15567 fetch_fsmap learned FSMap version 26 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a5c4e3d90 con 0x561a5c45df00 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd714ff9640 1 -- 172.21.15.90:0/600341546 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7280a4f90 con 0x561a5c45df00 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd714ff9640 1 -- 172.21.15.90:0/600341546 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd7280bb130 con 0x561a5c45df00 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 10 client.15567 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 20 client.15567 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c4e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:10.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 4 client.15567 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:14:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.955+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a5c3cbee0 con 0x561a5c47d510 2024-04-26T18:14:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.956+0000 7fd730feb640 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c4e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.956+0000 7fd714ff9640 10 client.15567 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:10.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.956+0000 7fd730feb640 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c4e63f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e2a3f9b96caa51 server_cookie=72fdfb3234f7324c in_seq=0 out_seq=0 2024-04-26T18:14:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd714ff9640 1 -- 172.21.15.90:0/600341546 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a5c3cbee0 con 0x561a5c47d510 2024-04-26T18:14:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd714ff9640 10 client.15567 handle_command_reply: tid=0 2024-04-26T18:14:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 client.15567 shutdown 2024-04-26T18:14:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 msgr2=0x561a5c462e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c462e80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd728021000 tx=0x7fd7280a0740 comp rx=0 tx=0).stop 2024-04-26T18:14:10.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 shutdown_connections 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561a5c47d510 0x561a5c4e63f0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a5c4690e0 0x561a5c463900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd714ff9640 0 client.15567 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561a5c45e820 0x561a5c4633c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 --2- 172.21.15.90:0/600341546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561a5c45df00 0x561a5c462e80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 >> 172.21.15.90:0/600341546 conn(0x561a5c45a410 msgr2=0x561a5c468130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 shutdown_connections 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 1 -- 172.21.15.90:0/600341546 wait complete. 2024-04-26T18:14:10.964 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.962+0000 7fd736768740 20 client.15567 trim_cache size 0 max 16384 2024-04-26T18:14:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:14:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:14:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:14:10.965 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd7180a3dc0 msgr2=0x7fd7180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd7180a3dc0 0x7fd7180a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd724033040 tx=0x7fd7240095c0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c100e30 msgr2=0x7fd72c199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c100e30 0x7fd72c199070 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd72800d770 tx=0x7fd72800dc30 comp rx=0 tx=0).stop 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 shutdown_connections 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd7180a3dc0 0x7fd7180a6280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd72c107910 0x7fd72c19d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd72c1017f0 0x7fd72c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 --2- 172.21.15.90:0/3027512831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd72c100e30 0x7fd72c199070 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 >> 172.21.15.90:0/3027512831 conn(0x7fd72c077dc0 msgr2=0x7fd72c105930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 shutdown_connections 2024-04-26T18:14:10.966 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:10.963+0000 7fd733a50640 1 -- 172.21.15.90:0/3027512831 wait complete. 2024-04-26T18:14:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:10 smithi119 ceph-mon[32752]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:14:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:10 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/41724415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:14:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:10 smithi167 ceph-mon[32751]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:14:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:10 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/41724415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 -- 172.21.15.90:0/2101096934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c1094c0 msgr2=0x7f1b3c1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 --2- 172.21.15.90:0/2101096934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c1094c0 0x7f1b3c1098a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c009990 tx=0x7f1b2c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 -- 172.21.15.90:0/2101096934 shutdown_connections 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 --2- 172.21.15.90:0/2101096934 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c1094c0 0x7f1b3c1098a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 --2- 172.21.15.90:0/2101096934 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c103e30 0x7f1b3c110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 --2- 172.21.15.90:0/2101096934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3c103470 0x7f1b3c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 -- 172.21.15.90:0/2101096934 >> 172.21.15.90:0/2101096934 conn(0x7f1b3c07a580 msgr2=0x7f1b3c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:12.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 -- 172.21.15.90:0/2101096934 shutdown_connections 2024-04-26T18:14:12.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 -- 172.21.15.90:0/2101096934 wait complete. 2024-04-26T18:14:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.508+0000 7f1b421b9700 1 Processor -- start 2024-04-26T18:14:12.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b421b9700 1 -- start start 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b421b9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3c103470 0x7f1b3c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b421b9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c103e30 0x7f1b3c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b421b9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c1094c0 0x7f1b3c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b421b9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1135c0 con 0x7f1b3c103e30 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b421b9700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c113440 con 0x7f1b3c103470 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b421b9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c113740 con 0x7f1b3c1094c0 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c103e30 0x7f1b3c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c103e30 0x7f1b3c19fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36492/0 (socket says 172.21.15.90:36492) 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 -- 172.21.15.90:0/3100937680 learned_addr learned my addr 172.21.15.90:0/3100937680 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3c103470 0x7f1b3c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c1094c0 0x7f1b3c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 -- 172.21.15.90:0/3100937680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c1094c0 msgr2=0x7f1b3c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c1094c0 0x7f1b3c199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 -- 172.21.15.90:0/3100937680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3c103470 msgr2=0x7f1b3c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3c103470 0x7f1b3c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3affd700 1 -- 172.21.15.90:0/3100937680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c19a020 con 0x7f1b3c103e30 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.509+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3c103470 0x7f1b3c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.510+0000 7f1b3affd700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c103e30 0x7f1b3c19fc00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000d980 tx=0x7f1b3000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.510+0000 7f1b38ff9700 1 -- 172.21.15.90:0/3100937680 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30016070 con 0x7f1b3c103e30 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.510+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c19a310 con 0x7f1b3c103e30 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.510+0000 7f1b38ff9700 1 -- 172.21.15.90:0/3100937680 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b300044d0 con 0x7f1b3c103e30 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.510+0000 7f1b38ff9700 1 -- 172.21.15.90:0/3100937680 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30004d00 con 0x7f1b3c103e30 2024-04-26T18:14:12.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.510+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c1a6520 con 0x7f1b3c103e30 2024-04-26T18:14:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.512+0000 7f1b38ff9700 1 -- 172.21.15.90:0/3100937680 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f1b30004020 con 0x7f1b3c103e30 2024-04-26T18:14:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.512+0000 7f1b38ff9700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1b240a3db0 0x7f1b240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.513+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1b240a3db0 0x7f1b240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.513+0000 7f1b38ff9700 1 -- 172.21.15.90:0/3100937680 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1b300d0680 con 0x7f1b3c103e30 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.513+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1b240a3db0 0x7f1b240a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c005960 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.513+0000 7f1b4ca4bb80 1 Processor -- start 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.513+0000 7f1b4ca4bb80 1 -- start start 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a45280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a3f870 0x55a658a3fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a658991930 con 0x55a658a3f870 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a6588f7a30 con 0x55a658a44ea0 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a65898d8f0 con 0x55a658a457c0 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a3f870 0x55a658a3fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a3f870 0x55a658a3fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36508/0 (socket says 172.21.15.90:36508) 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 -- 172.21.15.90:0/1864019553 learned_addr learned my addr 172.21.15.90:0/1864019553 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a45280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3affd700 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a3f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 -- 172.21.15.90:0/1864019553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 msgr2=0x55a658a3f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a3f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 -- 172.21.15.90:0/1864019553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 msgr2=0x55a658a45280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a45280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 -- 172.21.15.90:0/1864019553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6588de9d0 con 0x55a658a3f870 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3affd700 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a3f870 0x55a658a3fcd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400d1d0 tx=0x7f1b3400d4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75272e1569ea1321 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b21ffb700 1 -- 172.21.15.90:0/1864019553 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34016020 con 0x55a658a3f870 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b21ffb700 1 -- 172.21.15.90:0/1864019553 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b3400de20 con 0x55a658a3f870 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.514+0000 7f1b21ffb700 1 -- 172.21.15.90:0/1864019553 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34011850 con 0x55a658a3f870 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1864019553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a3f870 msgr2=0x55a658a3fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a3f870 0x55a658a3fcd0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400d1d0 tx=0x7f1b3400d4e0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1864019553 shutdown_connections 2024-04-26T18:14:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a3f870 0x55a658a3fcd0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a45280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1864019553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1864019553 >> 172.21.15.90:0/1864019553 conn(0x55a65896d6c0 msgr2=0x55a658a3a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1864019553 shutdown_connections 2024-04-26T18:14:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1864019553 wait complete. 2024-04-26T18:14:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.515+0000 7f1b4ca4bb80 1 -- start start 2024-04-26T18:14:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 1 Processor -- start 2024-04-26T18:14:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a3f870 0x55a658a4b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a48800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a48d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a658a4c230 con 0x55a658a457c0 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a65894bb80 con 0x55a658a44ea0 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a658a4c3b0 con 0x55a658a3f870 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b4ca4bb80 10 client.0 _ll_register_callbacks cb 0x55a658929a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a48d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a48d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36512/0 (socket says 172.21.15.90:36512) 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 -- 172.21.15.90:0/3742136257 learned_addr learned my addr 172.21.15.90:0/3742136257 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3affd700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a48800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a3f870 0x55a658a4b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 -- 172.21.15.90:0/3742136257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a3f870 msgr2=0x55a658a4b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a3f870 0x55a658a4b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 -- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 msgr2=0x55a658a48800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a48800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3bfff700 1 -- 172.21.15.90:0/3742136257 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a65894f580 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.516+0000 7f1b3affd700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a48800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a48d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1b34011390 tx=0x7f1b34011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2134ae1d8e915c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b1affd700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b1affd700 1 -- 172.21.15.90:0/3742136257 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34016030 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b3400de20 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b1affd700 1 -- 172.21.15.90:0/3742136257 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1b34011d80 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b1affd700 1 -- 172.21.15.90:0/3742136257 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b34025440 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 10 client.15579 fetch_fsmap learned FSMap version 26 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a658a56960 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b1affd700 1 -- 172.21.15.90:0/3742136257 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b3400de20 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b1affd700 1 -- 172.21.15.90:0/3742136257 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1b34023070 con 0x55a658a457c0 2024-04-26T18:14:12.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 10 client.15579 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 10 client.15579 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 20 client.15579 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658a58e30 0x55a658a5b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 4 client.15579 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.517+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a65899f310 con 0x55a658a58e30 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.518+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658a58e30 0x55a658a5b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.518+0000 7f1b1affd700 10 client.15579 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.518+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658a58e30 0x55a658a5b220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b820c194fc89fe21 server_cookie=e7d80ab391f0b256 in_seq=0 out_seq=0 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b1affd700 1 -- 172.21.15.90:0/3742136257 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a65899f310 con 0x55a658a58e30 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b1affd700 10 client.15579 handle_command_reply: tid=0 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 client.15579 shutdown 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 msgr2=0x55a658a48d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a48d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f1b34011390 tx=0x7f1b34011890 comp rx=0 tx=0).stop 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 shutdown_connections 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658a58e30 0x55a658a5b220 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a48d40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a3f870 0x55a658a4b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/3742136257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a44ea0 0x55a658a48800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 >> 172.21.15.90:0/3742136257 conn(0x55a65896d6c0 msgr2=0x55a658a3e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b1affd700 0 client.15579 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 shutdown_connections 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/3742136257 wait complete. 2024-04-26T18:14:12.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.519+0000 7f1b4ca4bb80 20 client.15579 trim_cache size 0 max 16384 2024-04-26T18:14:12.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 Processor -- start 2024-04-26T18:14:12.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 -- start start 2024-04-26T18:14:12.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a3f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a44ea0 0x55a658a55130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1b03f0 con 0x55a658a44ea0 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1adc80 con 0x55a658a3f870 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1abb10 con 0x55a658a457c0 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a44ea0 0x55a658a55130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b3affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a44ea0 0x55a658a55130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36522/0 (socket says 172.21.15.90:36522) 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.538+0000 7f1b3affd700 1 -- 172.21.15.90:0/1935708905 learned_addr learned my addr 172.21.15.90:0/1935708905 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a3f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3affd700 1 -- 172.21.15.90:0/1935708905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 msgr2=0x55a658a476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3affd700 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3affd700 1 -- 172.21.15.90:0/1935708905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 msgr2=0x55a658a3f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3affd700 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a3f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3affd700 1 -- 172.21.15.90:0/1935708905 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a658a4b0a0 con 0x55a658a44ea0 2024-04-26T18:14:12.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:12.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b3affd700 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a44ea0 0x55a658a55130 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000b540 tx=0x7f1b30004310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1509a09581f542f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:12.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b217fa700 1 -- 172.21.15.90:0/1935708905 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3009c690 con 0x55a658a44ea0 2024-04-26T18:14:12.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b217fa700 1 -- 172.21.15.90:0/1935708905 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b300b3030 con 0x55a658a44ea0 2024-04-26T18:14:12.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.539+0000 7f1b217fa700 1 -- 172.21.15.90:0/1935708905 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3009c8f0 con 0x55a658a44ea0 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1935708905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a44ea0 msgr2=0x55a658a55130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a44ea0 0x55a658a55130 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000b540 tx=0x7f1b30004310 comp rx=0 tx=0).stop 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1935708905 shutdown_connections 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a457c0 0x55a658a476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a3f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/1935708905 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a44ea0 0x55a658a55130 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1935708905 >> 172.21.15.90:0/1935708905 conn(0x55a65896d6c0 msgr2=0x55a658a46fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1935708905 shutdown_connections 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/1935708905 wait complete. 2024-04-26T18:14:12.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 -- start start 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.540+0000 7f1b4ca4bb80 1 Processor -- start 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a4e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a44ea0 0x55a658a4eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a4f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1ac160 con 0x55a658a457c0 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1ad5b0 con 0x55a658a3f870 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c1ac810 con 0x55a658a44ea0 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 10 client.0 _ll_register_callbacks cb 0x55a658929a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a4f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a4f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36526/0 (socket says 172.21.15.90:36526) 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 -- 172.21.15.90:0/2801796224 learned_addr learned my addr 172.21.15.90:0/2801796224 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a4e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3affd700 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a44ea0 0x55a658a4eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 -- 172.21.15.90:0/2801796224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a44ea0 msgr2=0x55a658a4eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a44ea0 0x55a658a4eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 -- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 msgr2=0x55a658a4e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a4e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 -- 172.21.15.90:0/2801796224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a658abf260 con 0x55a658a457c0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a4f090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1b340055a0 tx=0x7f1b34004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80b4619b824ff9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b1affd700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b1affd700 1 -- 172.21.15.90:0/2801796224 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b34016070 con 0x55a658a457c0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b1affd700 1 -- 172.21.15.90:0/2801796224 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1b340050a0 con 0x55a658a457c0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.541+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b300b3030 con 0x55a658a457c0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b1affd700 1 -- 172.21.15.90:0/2801796224 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1b34005c60 con 0x55a658a457c0 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 10 client.15585 fetch_fsmap learned FSMap version 26 2024-04-26T18:14:12.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55a658ac0490 con 0x55a658a457c0 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b1affd700 1 -- 172.21.15.90:0/2801796224 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b300b3030 con 0x55a658a457c0 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b1affd700 1 -- 172.21.15.90:0/2801796224 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1b34023040 con 0x55a658a457c0 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 10 client.15585 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 10 client.15585 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 20 client.15585 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658ac0b10 0x55a658ac2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 4 client.15585 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e986ca9", "release": true, "await": true}] 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e986ca9", "release": true, "await": true}) -- 0x55a6589993b0 con 0x55a658ac0b10 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.542+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658ac0b10 0x55a658ac2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.543+0000 7f1b1affd700 10 client.15585 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:12.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.543+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658ac0b10 0x55a658ac2f30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e82af1d4962d6535 server_cookie=a6c31aca8b1e1648 in_seq=0 out_seq=0 2024-04-26T18:14:12.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b1affd700 1 -- 172.21.15.90:0/2801796224 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x55a6589993b0 con 0x55a658ac0b10 2024-04-26T18:14:12.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b1affd700 10 client.15585 handle_command_reply: tid=0 2024-04-26T18:14:12.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 client.15585 shutdown 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 msgr2=0x55a658a4f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a4f090 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f1b340055a0 tx=0x7f1b34004d60 comp rx=0 tx=0).stop 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 shutdown_connections 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a658ac0b10 0x55a658ac2f30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a658a457c0 0x55a658a4f090 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a658a3f870 0x55a658a4e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 --2- 172.21.15.90:0/2801796224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a658a44ea0 0x55a658a4eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 >> 172.21.15.90:0/2801796224 conn(0x55a65896d6c0 msgr2=0x55a658a47850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b1affd700 0 client.15585 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 shutdown_connections 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 1 -- 172.21.15.90:0/2801796224 wait complete. 2024-04-26T18:14:12.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.546+0000 7f1b4ca4bb80 20 client.15585 trim_cache size 0 max 16384 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 26, 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 8, 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "e986ca9": { 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 8, 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:14:12.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 90.0, 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:14:12.551 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:14:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:14:12.552 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:14:12.552 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1b240a3db0 msgr2=0x7f1b240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1b240a3db0 0x7f1b240a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c006010 tx=0x7f1b2c005960 comp rx=0 tx=0).stop 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c103e30 msgr2=0x7f1b3c19fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c103e30 0x7f1b3c19fc00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000d980 tx=0x7f1b3000dd40 comp rx=0 tx=0).stop 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 shutdown_connections 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1b240a3db0 0x7f1b240a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1b3c1094c0 0x7f1b3c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b3c103470 0x7f1b3c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 --2- 172.21.15.90:0/3100937680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c103e30 0x7f1b3c19fc00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 >> 172.21.15.90:0/3100937680 conn(0x7f1b3c07a580 msgr2=0x7f1b3c102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 shutdown_connections 2024-04-26T18:14:12.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:14:12.547+0000 7f1b421b9700 1 -- 172.21.15.90:0/3100937680 wait complete. 2024-04-26T18:14:12.554 INFO:tasks.workunit.client.0.smithi090.stderr:+ dbench 1 2024-04-26T18:14:12.556 INFO:tasks.workunit.client.0.smithi090.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-26T18:14:12.556 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-04-26T18:14:12.556 INFO:tasks.workunit.client.0.smithi090.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-26T18:14:12.556 INFO:tasks.workunit.client.0.smithi090.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-26T18:14:12.657 INFO:tasks.workunit.client.0.smithi090.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-26T18:14:12.657 INFO:tasks.workunit.client.0.smithi090.stdout:releasing clients 2024-04-26T18:14:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:12 smithi090 ceph-mon[29180]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:14:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:12 smithi119 ceph-mon[32752]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:14:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:12 smithi167 ceph-mon[32751]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-26T18:14:13.426 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e986ca9', seconds elapsed: 42.4 2024-04-26T18:14:13.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 3378 35.46 MB/sec warmup 1 sec latency 7.253 ms 2024-04-26T18:14:14.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 6326 25.54 MB/sec warmup 2 sec latency 12.664 ms 2024-04-26T18:14:14.869 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:14 smithi167 ceph-mon[32751]: pgmap v71: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-04-26T18:14:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:14 smithi090 ceph-mon[29180]: pgmap v71: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-04-26T18:14:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:14 smithi119 ceph-mon[32752]: pgmap v71: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 103 KiB/s wr, 33 op/s 2024-04-26T18:14:15.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 9303 21.67 MB/sec warmup 3 sec latency 13.004 ms 2024-04-26T18:14:16.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 11708 20.06 MB/sec warmup 4 sec latency 20.186 ms 2024-04-26T18:14:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:16 smithi090 ceph-mon[29180]: pgmap v72: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-04-26T18:14:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:16 smithi119 ceph-mon[32752]: pgmap v72: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-04-26T18:14:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:16 smithi167 ceph-mon[32751]: pgmap v72: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-04-26T18:14:17.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 14815 19.49 MB/sec warmup 5 sec latency 7.836 ms 2024-04-26T18:14:18.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 18125 19.42 MB/sec warmup 6 sec latency 20.394 ms 2024-04-26T18:14:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:18 smithi090 ceph-mon[29180]: pgmap v73: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-04-26T18:14:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:18 smithi119 ceph-mon[32752]: pgmap v73: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-04-26T18:14:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:18 smithi167 ceph-mon[32751]: pgmap v73: 97 pgs: 97 active+clean; 2.2 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 81 KiB/s wr, 16 op/s 2024-04-26T18:14:19.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 21388 19.18 MB/sec warmup 7 sec latency 28.458 ms 2024-04-26T18:14:20.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 24099 18.34 MB/sec warmup 8 sec latency 12.304 ms 2024-04-26T18:14:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:20 smithi090 ceph-mon[29180]: pgmap v74: 97 pgs: 97 active+clean; 9.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 844 KiB/s wr, 103 op/s 2024-04-26T18:14:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:20 smithi119 ceph-mon[32752]: pgmap v74: 97 pgs: 97 active+clean; 9.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 844 KiB/s wr, 103 op/s 2024-04-26T18:14:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:20 smithi167 ceph-mon[32751]: pgmap v74: 97 pgs: 97 active+clean; 9.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 844 KiB/s wr, 103 op/s 2024-04-26T18:14:21.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 26765 17.81 MB/sec warmup 9 sec latency 12.337 ms 2024-04-26T18:14:22.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 29475 17.59 MB/sec warmup 10 sec latency 17.087 ms 2024-04-26T18:14:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:22 smithi119 ceph-mon[32752]: pgmap v75: 97 pgs: 97 active+clean; 9.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 843 KiB/s wr, 100 op/s 2024-04-26T18:14:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:22 smithi167 ceph-mon[32751]: pgmap v75: 97 pgs: 97 active+clean; 9.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 843 KiB/s wr, 100 op/s 2024-04-26T18:14:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:22 smithi090 ceph-mon[29180]: pgmap v75: 97 pgs: 97 active+clean; 9.7 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 843 KiB/s wr, 100 op/s 2024-04-26T18:14:23.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 32631 17.56 MB/sec warmup 11 sec latency 10.781 ms 2024-04-26T18:14:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.052+0000 7fabbcdf7640 1 -- 172.21.15.90:0/3338843486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 msgr2=0x7fabb8101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.052+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/3338843486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb8101ce0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faba0009930 tx=0x7faba002a420 comp rx=0 tx=0).stop 2024-04-26T18:14:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 -- 172.21.15.90:0/3338843486 shutdown_connections 2024-04-26T18:14:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/3338843486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb8102220 0x7fabb810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/3338843486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb8101ce0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/3338843486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb8107820 0x7fabb8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 -- 172.21.15.90:0/3338843486 >> 172.21.15.90:0/3338843486 conn(0x7fabb80789b0 msgr2=0x7fabb8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 -- 172.21.15.90:0/3338843486 shutdown_connections 2024-04-26T18:14:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 -- 172.21.15.90:0/3338843486 wait complete. 2024-04-26T18:14:24.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.053+0000 7fabbcdf7640 1 Processor -- start 2024-04-26T18:14:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabbcdf7640 1 -- start start 2024-04-26T18:14:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabbcdf7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb81990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabbcdf7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb8102220 0x7fabb8199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabbcdf7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb8107820 0x7fabb819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabbcdf7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8111ae0 con 0x7fabb8101860 2024-04-26T18:14:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabbcdf7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8111960 con 0x7fabb8102220 2024-04-26T18:14:24.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabbcdf7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabb8111c60 con 0x7fabb8107820 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb81990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb81990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33074/0 (socket says 172.21.15.90:33074) 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 -- 172.21.15.90:0/4193982606 learned_addr learned my addr 172.21.15.90:0/4193982606 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb8107820 0x7fabb819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabadd9b640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb8102220 0x7fabb8199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 -- 172.21.15.90:0/4193982606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb8107820 msgr2=0x7fabb819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb8107820 0x7fabb819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 -- 172.21.15.90:0/4193982606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb8102220 msgr2=0x7fabb8199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb8102220 0x7fabb8199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 -- 172.21.15.90:0/4193982606 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb819e0b0 con 0x7fabb8101860 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabb659c640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb81990c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9f0 tx=0x7faba800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.054+0000 7fabaffff640 1 -- 172.21.15.90:0/4193982606 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba800cd80 con 0x7fabb8101860 2024-04-26T18:14:24.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.055+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb819e3a0 con 0x7fabb8101860 2024-04-26T18:14:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.055+0000 7fabaffff640 1 -- 172.21.15.90:0/4193982606 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba80054d0 con 0x7fabb8101860 2024-04-26T18:14:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.055+0000 7fabaffff640 1 -- 172.21.15.90:0/4193982606 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8010690 con 0x7fabb8101860 2024-04-26T18:14:24.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.055+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb81a5c30 con 0x7fabb8101860 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabaffff640 1 -- 172.21.15.90:0/4193982606 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7faba8004e90 con 0x7fabb8101860 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 Processor -- start 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 -- start start 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa6e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6e820 0x55c06fa78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa7b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c06f9d4460 con 0x55c06fa6e820 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c06f98e590 con 0x55c06fa6df00 2024-04-26T18:14:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabbfbef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c06f9d0420 con 0x55c06fa790e0 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabaffff640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faba40a3dc0 0x7faba40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6e820 0x55c06fa78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.057+0000 7fabadd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa7b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa6e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6e820 0x55c06fa78ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33088/0 (socket says 172.21.15.90:33088) 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 -- 172.21.15.90:0/2938347529 learned_addr learned my addr 172.21.15.90:0/2938347529 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faba40a3dc0 0x7faba40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabaffff640 1 -- 172.21.15.90:0/4193982606 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7faba80d0b10 con 0x7fabb8101860 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 -- 172.21.15.90:0/2938347529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 msgr2=0x55c06fa7b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 --2- 172.21.15.90:0/2938347529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa7b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 -- 172.21.15.90:0/2938347529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 msgr2=0x55c06fa6e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 --2- 172.21.15.90:0/2938347529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa6e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 -- 172.21.15.90:0/2938347529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c06f99b0d0 con 0x55c06fa6e820 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb659c640 1 --2- 172.21.15.90:0/2938347529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6e820 0x55c06fa78ba0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9c0 tx=0x7faba8004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3436a4158dc32792 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faba40a3dc0 0x7faba40a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005c40 tx=0x7fabb0009500 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabacd99640 1 -- 172.21.15.90:0/2938347529 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba809d220 con 0x55c06fa6e820 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabacd99640 1 -- 172.21.15.90:0/2938347529 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba809de50 con 0x55c06fa6e820 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.058+0000 7fabacd99640 1 -- 172.21.15.90:0/2938347529 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80ce550 con 0x55c06fa6e820 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 -- 172.21.15.90:0/2938347529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6e820 msgr2=0x55c06fa78ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2938347529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6e820 0x55c06fa78ba0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9c0 tx=0x7faba8004a90 comp rx=0 tx=0).stop 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 -- 172.21.15.90:0/2938347529 shutdown_connections 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2938347529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa7b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2938347529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6e820 0x55c06fa78ba0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2938347529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa6e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 -- 172.21.15.90:0/2938347529 >> 172.21.15.90:0/2938347529 conn(0x55c06fa6a410 msgr2=0x55c06fa6a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 -- 172.21.15.90:0/2938347529 shutdown_connections 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 -- 172.21.15.90:0/2938347529 wait complete. 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.059+0000 7fabbfbef740 1 -- start start 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 1 Processor -- start 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa76060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa765a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa790e0 0x55c06fa76ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c06fa7bcb0 con 0x55c06fa6df00 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c06f95bb60 con 0x55c06fa790e0 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c06fa7be30 con 0x55c06fa6e820 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabbfbef740 10 client.0 _ll_register_callbacks cb 0x55c06f9802b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa76060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa76060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33094/0 (socket says 172.21.15.90:33094) 2024-04-26T18:14:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 -- 172.21.15.90:0/2226822243 learned_addr learned my addr 172.21.15.90:0/2226822243 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa790e0 0x55c06fa76ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabadd9b640 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa765a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 -- 172.21.15.90:0/2226822243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 msgr2=0x55c06fa765a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa765a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 -- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa790e0 msgr2=0x55c06fa76ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa790e0 0x55c06fa76ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 -- 172.21.15.90:0/2226822243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06f6e6210 con 0x55c06fa6df00 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fabb659c640 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa76060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faba809c820 tx=0x7faba809c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c611a4404841b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fab9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.060+0000 7fab9dffb640 1 -- 172.21.15.90:0/2226822243 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba80a0ed0 con 0x55c06fa6df00 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fab9dffb640 1 -- 172.21.15.90:0/2226822243 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7faba8099190 con 0x55c06fa6df00 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faba809de50 con 0x55c06fa6df00 2024-04-26T18:14:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fab9dffb640 1 -- 172.21.15.90:0/2226822243 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faba80a5790 con 0x55c06fa6df00 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 10 client.15597 fetch_fsmap learned FSMap version 26 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c06f7b2620 con 0x55c06fa6df00 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fab9dffb640 1 -- 172.21.15.90:0/2226822243 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba809de50 con 0x55c06fa6df00 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fab9dffb640 1 -- 172.21.15.90:0/2226822243 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faba80bbce0 con 0x55c06fa6df00 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 10 client.15597 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 20 client.15597 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06fa89030 0x55c06fa8b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 4 client.15597 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c06f9e2070 con 0x55c06fa89030 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.061+0000 7fabadd9b640 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06fa89030 0x55c06fa8b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.062+0000 7fabadd9b640 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06fa89030 0x55c06fa8b420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=309d9640b171254f server_cookie=49039744f1735a70 in_seq=0 out_seq=0 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.062+0000 7fab9dffb640 10 client.15597 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.062+0000 7fab9dffb640 1 -- 172.21.15.90:0/2226822243 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c06f9e2070 con 0x55c06fa89030 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.062+0000 7fab9dffb640 10 client.15597 handle_command_reply: tid=0 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 client.15597 shutdown 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 msgr2=0x55c06fa76060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa76060 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faba809c820 tx=0x7faba809c790 comp rx=0 tx=0).stop 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 shutdown_connections 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06fa89030 0x55c06fa8b420 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa790e0 0x55c06fa76ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa765a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2226822243 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa76060 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 >> 172.21.15.90:0/2226822243 conn(0x55c06fa6a410 msgr2=0x55c06fa70190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fab9dffb640 0 client.15597 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 shutdown_connections 2024-04-26T18:14:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 1 -- 172.21.15.90:0/2226822243 wait complete. 2024-04-26T18:14:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.063+0000 7fabbfbef740 20 client.15597 trim_cache size 0 max 16384 2024-04-26T18:14:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.078+0000 7fabbfbef740 1 Processor -- start 2024-04-26T18:14:24.089 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.078+0000 7fabbfbef740 1 -- start start 2024-04-26T18:14:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa84b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa790e0 0x55c06fa909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabbfbef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81b0c10 con 0x55c06fa790e0 2024-04-26T18:14:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabbfbef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ae4e0 con 0x55c06fa6df00 2024-04-26T18:14:24.090 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabbfbef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ac330 con 0x55c06fa6e820 2024-04-26T18:14:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa790e0 0x55c06fa909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa84b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa790e0 0x55c06fa909c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33102/0 (socket says 172.21.15.90:33102) 2024-04-26T18:14:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 -- 172.21.15.90:0/2474628858 learned_addr learned my addr 172.21.15.90:0/2474628858 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabadd9b640 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 -- 172.21.15.90:0/2474628858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 msgr2=0x55c06fa708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.091 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 -- 172.21.15.90:0/2474628858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 msgr2=0x55c06fa84b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa84b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 -- 172.21.15.90:0/2474628858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c06fa75070 con 0x55c06fa790e0 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa790e0 0x55c06fa909c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fabb0011e20 tx=0x7fabb0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd3063e11359f0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fab9ffff640 1 -- 172.21.15.90:0/2474628858 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0003e00 con 0x55c06fa790e0 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fab9ffff640 1 -- 172.21.15.90:0/2474628858 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb000fcc0 con 0x55c06fa790e0 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.079+0000 7fab9ffff640 1 -- 172.21.15.90:0/2474628858 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb0020c40 con 0x55c06fa790e0 2024-04-26T18:14:24.092 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 -- 172.21.15.90:0/2474628858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa790e0 msgr2=0x55c06fa909c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa790e0 0x55c06fa909c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fabb0011e20 tx=0x7fabb0005bd0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 -- 172.21.15.90:0/2474628858 shutdown_connections 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa790e0 0x55c06fa909c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa6e820 0x55c06fa708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2474628858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6df00 0x55c06fa84b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 -- 172.21.15.90:0/2474628858 >> 172.21.15.90:0/2474628858 conn(0x55c06fa6a410 msgr2=0x55c06fa6a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 -- 172.21.15.90:0/2474628858 shutdown_connections 2024-04-26T18:14:24.093 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 -- 172.21.15.90:0/2474628858 wait complete. 2024-04-26T18:14:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.080+0000 7fabbfbef740 1 -- start start 2024-04-26T18:14:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 Processor -- start 2024-04-26T18:14:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa72f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6e820 0x55c06fa73470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ac9a0 con 0x55c06fa6df00 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ade10 con 0x55c06fa6e820 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fabb81ad070 con 0x55c06fa790e0 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 10 client.0 _ll_register_callbacks cb 0x7faba8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa72f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa72f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33112/0 (socket says 172.21.15.90:33112) 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 -- 172.21.15.90:0/2160720542 learned_addr learned my addr 172.21.15.90:0/2160720542 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabadd9b640 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6e820 0x55c06fa73470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb6d9d640 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 -- 172.21.15.90:0/2160720542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 msgr2=0x55c06fa739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 -- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6e820 msgr2=0x55c06fa73470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6e820 0x55c06fa73470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 -- 172.21.15.90:0/2160720542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06faee430 con 0x55c06fa6df00 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabb659c640 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa72f30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faba80a5950 tx=0x7faba80ce380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9f0b8d26e53f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fab9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fab9dffb640 1 -- 172.21.15.90:0/2160720542 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba809c480 con 0x55c06fa6df00 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fab9dffb640 1 -- 172.21.15.90:0/2160720542 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7faba809bb00 con 0x55c06fa6df00 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.081+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabb000fcc0 con 0x55c06fa6df00 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fab9dffb640 1 -- 172.21.15.90:0/2160720542 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faba809c8f0 con 0x55c06fa6df00 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 10 client.15603 fetch_fsmap learned FSMap version 26 2024-04-26T18:14:24.096 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c06faef6e0 con 0x55c06fa6df00 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fab9dffb640 1 -- 172.21.15.90:0/2160720542 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabb000fcc0 con 0x55c06fa6df00 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fab9dffb640 1 -- 172.21.15.90:0/2160720542 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7faba80bbce0 con 0x55c06fa6df00 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 10 client.15603 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 20 client.15603 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06faefb40 0x55c06faf1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 4 client.15603 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c06f9dbee0 con 0x55c06faefb40 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabadd9b640 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06faefb40 0x55c06faf1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fab9dffb640 10 client.15603 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:24.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.082+0000 7fabadd9b640 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06faefb40 0x55c06faf1f50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a0717b4472fab08 server_cookie=5c73d77680bf512f in_seq=0 out_seq=0 2024-04-26T18:14:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fab9dffb640 1 -- 172.21.15.90:0/2160720542 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c06f9dbee0 con 0x55c06faefb40 2024-04-26T18:14:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fab9dffb640 10 client.15603 handle_command_reply: tid=0 2024-04-26T18:14:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 client.15603 shutdown 2024-04-26T18:14:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 msgr2=0x55c06fa72f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa72f30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7faba80a5950 tx=0x7faba80ce380 comp rx=0 tx=0).stop 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 shutdown_connections 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c06faefb40 0x55c06faf1f50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c06fa790e0 0x55c06fa739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c06fa6e820 0x55c06fa73470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 --2- 172.21.15.90:0/2160720542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c06fa6df00 0x55c06fa72f30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.209+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 >> 172.21.15.90:0/2160720542 conn(0x55c06fa6a410 msgr2=0x55c06fa76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fab9dffb640 0 client.15603 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 shutdown_connections 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbfbef740 1 -- 172.21.15.90:0/2160720542 wait complete. 2024-04-26T18:14:24.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbfbef740 20 client.15603 trim_cache size 0 max 16384 2024-04-26T18:14:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:14:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:14:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:14:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faba40a3dc0 msgr2=0x7faba40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faba40a3dc0 0x7faba40a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fabb0005c40 tx=0x7fabb0009500 comp rx=0 tx=0).stop 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 msgr2=0x7fabb81990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb81990c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faba800e9f0 tx=0x7faba800edf0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 shutdown_connections 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faba40a3dc0 0x7faba40a6280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fabb8107820 0x7fabb819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabb8102220 0x7fabb8199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 --2- 172.21.15.90:0/4193982606 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabb8101860 0x7fabb81990c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 >> 172.21.15.90:0/4193982606 conn(0x7fabb80789b0 msgr2=0x7fabb806a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 shutdown_connections 2024-04-26T18:14:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:24.210+0000 7fabbcdf7640 1 -- 172.21.15.90:0/4193982606 wait complete. 2024-04-26T18:14:24.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 35743 17.66 MB/sec warmup 12 sec latency 81.368 ms 2024-04-26T18:14:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:24 smithi119 ceph-mon[32752]: pgmap v76: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-04-26T18:14:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:24 smithi167 ceph-mon[32751]: pgmap v76: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-04-26T18:14:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:24 smithi090 ceph-mon[29180]: pgmap v76: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-04-26T18:14:25.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 38693 17.35 MB/sec warmup 13 sec latency 12.464 ms 2024-04-26T18:14:26.657 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39059 16.41 MB/sec warmup 14 sec latency 881.758 ms 2024-04-26T18:14:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:26 smithi119 ceph-mon[32752]: pgmap v77: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-26T18:14:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:26 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:14:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:26 smithi167 ceph-mon[32751]: pgmap v77: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-26T18:14:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:26 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:14:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:26 smithi090 ceph-mon[29180]: pgmap v77: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-26T18:14:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:26 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:14:27.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39059 15.31 MB/sec warmup 15 sec latency 1881.865 ms 2024-04-26T18:14:28.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39059 14.35 MB/sec warmup 16 sec latency 2881.962 ms 2024-04-26T18:14:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:28 smithi119 ceph-mon[32752]: pgmap v78: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-26T18:14:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:28 smithi167 ceph-mon[32751]: pgmap v78: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-26T18:14:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:28 smithi090 ceph-mon[29180]: pgmap v78: 97 pgs: 97 active+clean; 16 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-04-26T18:14:29.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39124 13.52 MB/sec warmup 17 sec latency 3356.575 ms 2024-04-26T18:14:30.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39124 12.77 MB/sec warmup 18 sec latency 1509.576 ms 2024-04-26T18:14:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:30 smithi119 ceph-mon[32752]: pgmap v79: 97 pgs: 97 active+clean; 12 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-04-26T18:14:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:30 smithi167 ceph-mon[32751]: pgmap v79: 97 pgs: 97 active+clean; 12 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-04-26T18:14:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:30 smithi090 ceph-mon[29180]: pgmap v79: 97 pgs: 97 active+clean; 12 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 1.9 MiB/s wr, 356 op/s 2024-04-26T18:14:31.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39124 12.10 MB/sec warmup 19 sec latency 2509.635 ms 2024-04-26T18:14:32.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39124 11.49 MB/sec warmup 20 sec latency 3509.736 ms 2024-04-26T18:14:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:32 smithi119 ceph-mon[32752]: pgmap v80: 97 pgs: 97 active+clean; 12 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 269 op/s 2024-04-26T18:14:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:32 smithi167 ceph-mon[32751]: pgmap v80: 97 pgs: 97 active+clean; 12 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 269 op/s 2024-04-26T18:14:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:32 smithi090 ceph-mon[29180]: pgmap v80: 97 pgs: 97 active+clean; 12 MiB data, 453 MiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 1.1 MiB/s wr, 269 op/s 2024-04-26T18:14:33.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39124 10.95 MB/sec warmup 21 sec latency 4509.825 ms 2024-04-26T18:14:34.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39191 10.45 MB/sec warmup 22 sec latency 4984.155 ms 2024-04-26T18:14:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:34 smithi119 ceph-mon[32752]: pgmap v81: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 274 op/s 2024-04-26T18:14:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:34 smithi167 ceph-mon[32751]: pgmap v81: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 274 op/s 2024-04-26T18:14:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:34 smithi090 ceph-mon[29180]: pgmap v81: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 1.1 MiB/s wr, 274 op/s 2024-04-26T18:14:35.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39191 10.00 MB/sec warmup 23 sec latency 1465.737 ms 2024-04-26T18:14:36.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39191 9.58 MB/sec warmup 24 sec latency 2465.857 ms 2024-04-26T18:14:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:36 smithi119 ceph-mon[32752]: pgmap v82: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 426 KiB/s wr, 186 op/s 2024-04-26T18:14:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:36 smithi167 ceph-mon[32751]: pgmap v82: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 426 KiB/s wr, 186 op/s 2024-04-26T18:14:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:36 smithi090 ceph-mon[29180]: pgmap v82: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 426 KiB/s wr, 186 op/s 2024-04-26T18:14:37.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 -- 172.21.15.90:0/3814310586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40101a50 msgr2=0x7fac40107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 --2- 172.21.15.90:0/3814310586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40101a50 0x7fac40107580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac3002a410 comp rx=0 tx=0).stop 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 -- 172.21.15.90:0/3814310586 shutdown_connections 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 --2- 172.21.15.90:0/3814310586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac40107c60 0x7fac4010f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 --2- 172.21.15.90:0/3814310586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40101a50 0x7fac40107580 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 --2- 172.21.15.90:0/3814310586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac401010a0 0x7fac40101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 -- 172.21.15.90:0/3814310586 >> 172.21.15.90:0/3814310586 conn(0x7fac40077dc0 msgr2=0x7fac400ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 -- 172.21.15.90:0/3814310586 shutdown_connections 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.309+0000 7fac467c6640 1 -- 172.21.15.90:0/3814310586 wait complete. 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 Processor -- start 2024-04-26T18:14:37.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 -- start start 2024-04-26T18:14:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac401010a0 0x7fac40102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac40107c60 0x7fac40102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40102a90 0x7fac401a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac40112bc0 con 0x7fac40102a90 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac40112a40 con 0x7fac40107c60 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac467c6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac40112d40 con 0x7fac401010a0 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac44d63640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40102a90 0x7fac401a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac44d63640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40102a90 0x7fac401a72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42166/0 (socket says 172.21.15.90:42166) 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.310+0000 7fac44d63640 1 -- 172.21.15.90:0/3015374896 learned_addr learned my addr 172.21.15.90:0/3015374896 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac40107c60 0x7fac40102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac3ffff640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac401010a0 0x7fac40102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac44d63640 1 -- 172.21.15.90:0/3015374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac401010a0 msgr2=0x7fac40102010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac44d63640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac401010a0 0x7fac40102010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac44d63640 1 -- 172.21.15.90:0/3015374896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac40107c60 msgr2=0x7fac40102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac44d63640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac40107c60 0x7fac40102550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac44d63640 1 -- 172.21.15.90:0/3015374896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac401a7950 con 0x7fac40102a90 2024-04-26T18:14:37.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac44d63640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40102a90 0x7fac401a72b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cad0 tx=0x7fac3800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac3d7fa640 1 -- 172.21.15.90:0/3015374896 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3800ae20 con 0x7fac40102a90 2024-04-26T18:14:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac401a7c40 con 0x7fac40102a90 2024-04-26T18:14:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac3d7fa640 1 -- 172.21.15.90:0/3015374896 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac38005020 con 0x7fac40102a90 2024-04-26T18:14:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac3d7fa640 1 -- 172.21.15.90:0/3015374896 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3800f6b0 con 0x7fac40102a90 2024-04-26T18:14:37.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.311+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac401a8110 con 0x7fac40102a90 2024-04-26T18:14:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac3d7fa640 1 -- 172.21.15.90:0/3015374896 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fac380051d0 con 0x7fac40102a90 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 Processor -- start 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 -- start start 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac3d7fa640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac280a3dc0 0x7fac280a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307dbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307de4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557930737460 con 0x5579307dc0e0 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5579306f1590 con 0x5579307d0f00 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.314+0000 7fac495bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557930733420 con 0x5579307d1820 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307de4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac3ffff640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac280a3dc0 0x7fac280a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307de4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42182/0 (socket says 172.21.15.90:42182) 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 -- 172.21.15.90:0/1820042085 learned_addr learned my addr 172.21.15.90:0/1820042085 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac3d7fa640 1 -- 172.21.15.90:0/3015374896 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fac380d0880 con 0x7fac40102a90 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307dbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 -- 172.21.15.90:0/1820042085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 msgr2=0x5579307dbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307dbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 -- 172.21.15.90:0/1820042085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 msgr2=0x5579307d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 -- 172.21.15.90:0/1820042085 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579306fe0d0 con 0x5579307dc0e0 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307dbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac3ffff640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac280a3dc0 0x7fac280a6280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac34005fd0 tx=0x7fac34005850 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:37.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac44d63640 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307de4d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fac380a4fd0 tx=0x7fac38005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2309738f76de7d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac267fc640 1 -- 172.21.15.90:0/1820042085 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3801d070 con 0x5579307dc0e0 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac267fc640 1 -- 172.21.15.90:0/1820042085 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3809c890 con 0x5579307dc0e0 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.315+0000 7fac267fc640 1 -- 172.21.15.90:0/1820042085 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac380ce2c0 con 0x5579307dc0e0 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 -- 172.21.15.90:0/1820042085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 msgr2=0x5579307de4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307de4d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fac380a4fd0 tx=0x7fac38005890 comp rx=0 tx=0).stop 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 -- 172.21.15.90:0/1820042085 shutdown_connections 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307de4d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307dbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 --2- 172.21.15.90:0/1820042085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 -- 172.21.15.90:0/1820042085 >> 172.21.15.90:0/1820042085 conn(0x5579307cd410 msgr2=0x5579307cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 -- 172.21.15.90:0/1820042085 shutdown_connections 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 -- 172.21.15.90:0/1820042085 wait complete. 2024-04-26T18:14:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.316+0000 7fac495bf740 1 -- start start 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 1 Processor -- start 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307d90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5579307dec50 con 0x5579307d1820 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5579306beb60 con 0x5579307dc0e0 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5579307dedd0 con 0x5579307d0f00 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac495bf740 10 client.0 _ll_register_callbacks cb 0x5579306e32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42194/0 (socket says 172.21.15.90:42194) 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1049887218 learned_addr learned my addr 172.21.15.90:0/1049887218 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac3ffff640 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307d90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.317+0000 7fac44d63640 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1049887218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 msgr2=0x5579307d90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307d90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 msgr2=0x5579307d9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1049887218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557930449210 con 0x5579307d1820 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d9600 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac30032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dd2a02a070d501 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac1f7fe640 1 -- 172.21.15.90:0/1049887218 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30035040 con 0x5579307d1820 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac1f7fe640 1 -- 172.21.15.90:0/1049887218 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fac30004040 con 0x5579307d1820 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac3809c890 con 0x5579307d1820 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac1f7fe640 1 -- 172.21.15.90:0/1049887218 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac30004240 con 0x5579307d1820 2024-04-26T18:14:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac495bf740 10 client.15615 fetch_fsmap learned FSMap version 27 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557930515620 con 0x5579307d1820 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac1f7fe640 1 -- 172.21.15.90:0/1049887218 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3809c890 con 0x5579307d1820 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.318+0000 7fac1f7fe640 1 -- 172.21.15.90:0/1049887218 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fac3004f020 con 0x5579307d1820 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac495bf740 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac495bf740 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac495bf740 20 client.15615 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac495bf740 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5579307ec170 0x5579307ee5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac495bf740 4 client.15615 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557930745070 con 0x5579307ec170 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac3ffff640 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5579307ec170 0x5579307ee5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac1f7fe640 10 client.15615 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.319+0000 7fac3ffff640 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5579307ec170 0x5579307ee5a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333df0ce0221bd68 server_cookie=f32c8206116a734e in_seq=0 out_seq=0 2024-04-26T18:14:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac1f7fe640 1 -- 172.21.15.90:0/1049887218 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557930745070 con 0x5579307ec170 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac1f7fe640 10 client.15615 handle_command_reply: tid=0 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 client.15615 shutdown 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 msgr2=0x5579307d9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d9600 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fac30009900 tx=0x7fac30032f10 comp rx=0 tx=0).stop 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 shutdown_connections 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5579307ec170 0x5579307ee5a0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac1f7fe640 0 client.15615 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d9600 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 --2- 172.21.15.90:0/1049887218 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307d90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 >> 172.21.15.90:0/1049887218 conn(0x5579307cd410 msgr2=0x5579307d3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 shutdown_connections 2024-04-26T18:14:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 1 -- 172.21.15.90:0/1049887218 wait complete. 2024-04-26T18:14:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.320+0000 7fac495bf740 20 client.15615 trim_cache size 0 max 16384 2024-04-26T18:14:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.335+0000 7fac495bf740 1 Processor -- start 2024-04-26T18:14:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.335+0000 7fac495bf740 1 -- start start 2024-04-26T18:14:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac495bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac495bf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac495bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac495bf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac401b2dc0 con 0x5579307d1820 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac495bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac401b0690 con 0x5579307dc0e0 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac495bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac401ae560 con 0x5579307d0f00 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42200/0 (socket says 172.21.15.90:42200) 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1771436045 learned_addr learned my addr 172.21.15.90:0/1771436045 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac44d63640 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3ffff640 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1771436045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 msgr2=0x5579307ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1771436045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 msgr2=0x5579307d9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 -- 172.21.15.90:0/1771436045 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579307d7610 con 0x5579307d1820 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac44d63640 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3ffff640 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d3140 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fac30031440 tx=0x7fac3002aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceebe4cf8649b738 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac25ffb640 1 -- 172.21.15.90:0/1771436045 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30056030 con 0x5579307d1820 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac25ffb640 1 -- 172.21.15.90:0/1771436045 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac30031870 con 0x5579307d1820 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.336+0000 7fac25ffb640 1 -- 172.21.15.90:0/1771436045 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac30035030 con 0x5579307d1820 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 -- 172.21.15.90:0/1771436045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 msgr2=0x5579307d3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d3140 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fac30031440 tx=0x7fac3002aa90 comp rx=0 tx=0).stop 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 -- 172.21.15.90:0/1771436045 shutdown_connections 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307dc0e0 0x5579307d9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307d1820 0x5579307d3140 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 --2- 172.21.15.90:0/1771436045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d0f00 0x5579307ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 -- 172.21.15.90:0/1771436045 >> 172.21.15.90:0/1771436045 conn(0x5579307cd410 msgr2=0x5579307d9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 -- 172.21.15.90:0/1771436045 shutdown_connections 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 -- 172.21.15.90:0/1771436045 wait complete. 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.337+0000 7fac495bf740 1 -- start start 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 1 Processor -- start 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac401aebb0 con 0x5579307dc0e0 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac401affc0 con 0x5579307d0f00 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac401af220 con 0x5579307d1820 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac495bf740 10 client.0 _ll_register_callbacks cb 0x7fac30009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42210/0 (socket says 172.21.15.90:42210) 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 -- 172.21.15.90:0/4040393002 learned_addr learned my addr 172.21.15.90:0/4040393002 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac3f7fe640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac3ffff640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 -- 172.21.15.90:0/4040393002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 msgr2=0x5579307e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 -- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 msgr2=0x5579307e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307e4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 -- 172.21.15.90:0/4040393002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579307e1a70 con 0x5579307dc0e0 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac3ffff640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac44d63640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307e1530 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fac3809c820 tx=0x7fac380b10e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6da477a8daf21a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.338+0000 7fac1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac1f7fe640 1 -- 172.21.15.90:0/4040393002 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3801d070 con 0x5579307dc0e0 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac1f7fe640 1 -- 172.21.15.90:0/4040393002 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fac38098f00 con 0x5579307dc0e0 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac30031870 con 0x5579307dc0e0 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac1f7fe640 1 -- 172.21.15.90:0/4040393002 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fac380a0c40 con 0x5579307dc0e0 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 10 client.15621 fetch_fsmap learned FSMap version 27 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579308532f0 con 0x5579307dc0e0 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac1f7fe640 1 -- 172.21.15.90:0/4040393002 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac30031870 con 0x5579307dc0e0 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac1f7fe640 1 -- 172.21.15.90:0/4040393002 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fac38087070 con 0x5579307dc0e0 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 20 client.15621 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557930853680 0x557930855a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 4 client.15621 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55793073eee0 con 0x557930853680 2024-04-26T18:14:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.339+0000 7fac3ffff640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557930853680 0x557930855a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.340+0000 7fac1f7fe640 10 client.15621 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.340+0000 7fac3ffff640 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557930853680 0x557930855a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37fd2fd13ea8c3ca server_cookie=4610c94957dbd891 in_seq=0 out_seq=0 2024-04-26T18:14:37.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac1f7fe640 1 -- 172.21.15.90:0/4040393002 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55793073eee0 con 0x557930853680 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac1f7fe640 10 client.15621 handle_command_reply: tid=0 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 client.15621 shutdown 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 msgr2=0x5579307e1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307e1530 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fac3809c820 tx=0x7fac380b10e0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 shutdown_connections 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557930853680 0x557930855a70 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5579307dc0e0 0x5579307e1530 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5579307d1820 0x5579307e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 --2- 172.21.15.90:0/4040393002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5579307d0f00 0x5579307e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac1f7fe640 0 client.15621 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 >> 172.21.15.90:0/4040393002 conn(0x5579307cd410 msgr2=0x5579307d2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 shutdown_connections 2024-04-26T18:14:37.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 1 -- 172.21.15.90:0/4040393002 wait complete. 2024-04-26T18:14:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.373+0000 7fac495bf740 20 client.15621 trim_cache size 0 max 16384 2024-04-26T18:14:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:14:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:14:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:14:37.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac280a3dc0 msgr2=0x7fac280a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac280a3dc0 0x7fac280a6280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fac34005fd0 tx=0x7fac34005850 comp rx=0 tx=0).stop 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40102a90 msgr2=0x7fac401a72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40102a90 0x7fac401a72b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fac3800cad0 tx=0x7fac3800cf90 comp rx=0 tx=0).stop 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 shutdown_connections 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac280a3dc0 0x7fac280a6280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac40102a90 0x7fac401a72b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac40107c60 0x7fac40102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 --2- 172.21.15.90:0/3015374896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac401010a0 0x7fac40102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 >> 172.21.15.90:0/3015374896 conn(0x7fac40077dc0 msgr2=0x7fac400ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 shutdown_connections 2024-04-26T18:14:37.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:37.374+0000 7fac467c6640 1 -- 172.21.15.90:0/3015374896 wait complete. 2024-04-26T18:14:37.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39191 9.20 MB/sec warmup 25 sec latency 3465.931 ms 2024-04-26T18:14:38.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39191 8.85 MB/sec warmup 26 sec latency 4466.021 ms 2024-04-26T18:14:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:38 smithi119 ceph-mon[32752]: pgmap v83: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 426 KiB/s wr, 186 op/s 2024-04-26T18:14:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:38 smithi167 ceph-mon[32751]: pgmap v83: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 426 KiB/s wr, 186 op/s 2024-04-26T18:14:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:38 smithi090 ceph-mon[29180]: pgmap v83: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 426 KiB/s wr, 186 op/s 2024-04-26T18:14:39.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 39644 8.57 MB/sec warmup 27 sec latency 4940.962 ms 2024-04-26T18:14:40.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 42327 8.77 MB/sec warmup 28 sec latency 16.156 ms 2024-04-26T18:14:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:40 smithi119 ceph-mon[32752]: pgmap v84: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 442 KiB/s wr, 210 op/s 2024-04-26T18:14:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:40 smithi167 ceph-mon[32751]: pgmap v84: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 442 KiB/s wr, 210 op/s 2024-04-26T18:14:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:40 smithi090 ceph-mon[29180]: pgmap v84: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 442 KiB/s wr, 210 op/s 2024-04-26T18:14:41.658 INFO:tasks.workunit.client.0.smithi090.stdout: 1 43877 8.71 MB/sec warmup 29 sec latency 13.371 ms 2024-04-26T18:14:42.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 46532 8.99 MB/sec warmup 30 sec latency 13.980 ms 2024-04-26T18:14:42.926 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:42 smithi090 ceph-mon[29180]: pgmap v85: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 28 op/s 2024-04-26T18:14:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:42 smithi119 ceph-mon[32752]: pgmap v85: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 28 op/s 2024-04-26T18:14:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:42 smithi167 ceph-mon[32751]: pgmap v85: 97 pgs: 97 active+clean; 12 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 28 op/s 2024-04-26T18:14:43.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 48729 8.95 MB/sec warmup 31 sec latency 13.520 ms 2024-04-26T18:14:44.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 50479 9.03 MB/sec warmup 32 sec latency 17.546 ms 2024-04-26T18:14:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:44 smithi119 ceph-mon[32752]: pgmap v86: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2024-04-26T18:14:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:44 smithi167 ceph-mon[32751]: pgmap v86: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2024-04-26T18:14:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:44 smithi090 ceph-mon[29180]: pgmap v86: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 213 op/s 2024-04-26T18:14:45.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 53446 9.32 MB/sec warmup 33 sec latency 10.747 ms 2024-04-26T18:14:46.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 54904 9.21 MB/sec warmup 34 sec latency 19.072 ms 2024-04-26T18:14:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:46 smithi119 ceph-mon[32752]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 208 op/s 2024-04-26T18:14:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:46 smithi167 ceph-mon[32751]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 208 op/s 2024-04-26T18:14:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:46 smithi090 ceph-mon[29180]: pgmap v87: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 208 op/s 2024-04-26T18:14:47.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 57255 9.34 MB/sec warmup 35 sec latency 10.972 ms 2024-04-26T18:14:48.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 59973 9.45 MB/sec warmup 36 sec latency 12.459 ms 2024-04-26T18:14:49.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:48 smithi119 ceph-mon[32752]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 208 op/s 2024-04-26T18:14:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:48 smithi167 ceph-mon[32751]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 208 op/s 2024-04-26T18:14:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:48 smithi090 ceph-mon[29180]: pgmap v88: 97 pgs: 97 active+clean; 31 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 208 op/s 2024-04-26T18:14:49.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 61515 9.41 MB/sec warmup 37 sec latency 13.656 ms 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 -- 172.21.15.90:0/1464606310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd8102540 msgr2=0x7fcfd810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/1464606310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd8102540 0x7fcfd810e650 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009ca0 tx=0x7fcfd002a8c0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 -- 172.21.15.90:0/1464606310 shutdown_connections 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/1464606310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd8102540 0x7fcfd810e650 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/1464606310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd8101b80 0x7fcfd8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/1464606310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd81067d0 0x7fcfd8106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 -- 172.21.15.90:0/1464606310 >> 172.21.15.90:0/1464606310 conn(0x7fcfd8077630 msgr2=0x7fcfd8077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 -- 172.21.15.90:0/1464606310 shutdown_connections 2024-04-26T18:14:50.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.465+0000 7fcfdd22a640 1 -- 172.21.15.90:0/1464606310 wait complete. 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 Processor -- start 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 -- start start 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd8101b80 0x7fcfd8199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd8102540 0x7fcfd8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd81067d0 0x7fcfd819d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8111ab0 con 0x7fcfd81067d0 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8111930 con 0x7fcfd8102540 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfdd22a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd8111c30 con 0x7fcfd8101b80 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd81067d0 0x7fcfd819d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd81067d0 0x7fcfd819d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35834/0 (socket says 172.21.15.90:35834) 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 -- 172.21.15.90:0/906987332 learned_addr learned my addr 172.21.15.90:0/906987332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd8101b80 0x7fcfd8199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfcffff640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd8102540 0x7fcfd8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 -- 172.21.15.90:0/906987332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd8101b80 msgr2=0x7fcfd8199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd8101b80 0x7fcfd8199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 -- 172.21.15.90:0/906987332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd8102540 msgr2=0x7fcfd8199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd8102540 0x7fcfd8199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.466+0000 7fcfd759e640 1 -- 172.21.15.90:0/906987332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd819e020 con 0x7fcfd81067d0 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.467+0000 7fcfd759e640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd81067d0 0x7fcfd819d8f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0005cc0 tx=0x7fcfd0031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.467+0000 7fcfd4d99640 1 -- 172.21.15.90:0/906987332 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0003b70 con 0x7fcfd81067d0 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.467+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd819e2b0 con 0x7fcfd81067d0 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.467+0000 7fcfd4d99640 1 -- 172.21.15.90:0/906987332 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd00318f0 con 0x7fcfd81067d0 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.467+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd81a5ba0 con 0x7fcfd81067d0 2024-04-26T18:14:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.467+0000 7fcfd4d99640 1 -- 172.21.15.90:0/906987332 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd002f950 con 0x7fcfd81067d0 2024-04-26T18:14:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.469+0000 7fcfd4d99640 1 -- 172.21.15.90:0/906987332 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fcfd002fb50 con 0x7fcfd81067d0 2024-04-26T18:14:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.469+0000 7fcfe0025740 1 Processor -- start 2024-04-26T18:14:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.469+0000 7fcfe0025740 1 -- start start 2024-04-26T18:14:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1bf00 0x55807ad1c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd4d99640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad26ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad270e0 0x55807ad294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfe0025740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55807ac82460 con 0x55807ad1c820 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfe0025740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55807ac3c590 con 0x55807ad270e0 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfe0025740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55807ac7e420 con 0x55807ad1bf00 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad270e0 0x55807ad294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad26ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad26ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35846/0 (socket says 172.21.15.90:35846) 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad270e0 0x55807ad294d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:56294/0 (socket says 172.21.15.90:56294) 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd4d99640 1 -- 172.21.15.90:0/906987332 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fcfd00f19c0 con 0x7fcfd81067d0 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfcffff640 1 -- 172.21.15.90:0/2371489067 learned_addr learned my addr 172.21.15.90:0/2371489067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfcffff640 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1bf00 0x55807ad1c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd759e640 1 -- 172.21.15.90:0/2371489067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1bf00 msgr2=0x55807ad1c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd759e640 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1bf00 0x55807ad1c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd759e640 1 -- 172.21.15.90:0/2371489067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 msgr2=0x55807ad26ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd759e640 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad26ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd759e640 1 -- 172.21.15.90:0/2371489067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807ac490d0 con 0x55807ad270e0 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.470+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009b50 tx=0x7fcfc8005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.471+0000 7fcfd759e640 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad270e0 0x55807ad294d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00ef230 tx=0x7fcfd00b9720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12f6bc7dbdef0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.471+0000 7fcfcd7fa640 1 -- 172.21.15.90:0/2371489067 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00bebb0 con 0x55807ad270e0 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.471+0000 7fcfcd7fa640 1 -- 172.21.15.90:0/2371489067 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd0036030 con 0x55807ad270e0 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.471+0000 7fcfcd7fa640 1 -- 172.21.15.90:0/2371489067 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00be180 con 0x55807ad270e0 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.471+0000 7fcfe0025740 1 -- 172.21.15.90:0/2371489067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad270e0 msgr2=0x55807ad294d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.471+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad270e0 0x55807ad294d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00ef230 tx=0x7fcfd00b9720 comp rx=0 tx=0).stop 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- 172.21.15.90:0/2371489067 shutdown_connections 2024-04-26T18:14:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad270e0 0x55807ad294d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad26ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2371489067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1bf00 0x55807ad1c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- 172.21.15.90:0/2371489067 >> 172.21.15.90:0/2371489067 conn(0x55807ad18410 msgr2=0x55807ad187f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- 172.21.15.90:0/2371489067 shutdown_connections 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- 172.21.15.90:0/2371489067 wait complete. 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- start start 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 Processor -- start 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad24060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad245a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55807ac3c590 con 0x55807ad270e0 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55807ac09b60 con 0x55807ad1bf00 2024-04-26T18:14:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55807ad29ad0 con 0x55807ad1c820 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfe0025740 10 client.0 _ll_register_callbacks cb 0x55807ac2e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfd6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad24060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad24ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35858/0 (socket says 172.21.15.90:35858) 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfd759e640 1 -- 172.21.15.90:0/469126198 learned_addr learned my addr 172.21.15.90:0/469126198 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.472+0000 7fcfcffff640 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfd759e640 1 -- 172.21.15.90:0/469126198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 msgr2=0x55807ad245a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfd759e640 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad245a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfd759e640 1 -- 172.21.15.90:0/469126198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 msgr2=0x55807ad24060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfd759e640 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad24060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfd759e640 1 -- 172.21.15.90:0/469126198 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807a994210 con 0x55807ad270e0 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfd759e640 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad24ae0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00b9ee0 tx=0x7fcfd00be430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae78348ff1b0dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/469126198 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00bda80 con 0x55807ad270e0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/469126198 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fcfd00beeb0 con 0x55807ad270e0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd0036030 con 0x55807ad270e0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/469126198 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfd00be180 con 0x55807ad270e0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 10 client.15633 fetch_fsmap learned FSMap version 27 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55807aa60620 con 0x55807ad270e0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/469126198 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd0036030 con 0x55807ad270e0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/469126198 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcfd00da060 con 0x55807ad270e0 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 10 client.15633 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 20 client.15633 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad36c80 0x55807ad39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 4 client.15633 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.473+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55807ac90070 con 0x55807ad36c80 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.474+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad36c80 0x55807ad39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.474+0000 7fcfbe7fc640 10 client.15633 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.474+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad36c80 0x55807ad39070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f495f82d715cd2ac server_cookie=25072627dc97e732 in_seq=0 out_seq=0 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/469126198 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55807ac90070 con 0x55807ad36c80 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfbe7fc640 10 client.15633 handle_command_reply: tid=0 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 client.15633 shutdown 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 msgr2=0x55807ad24ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad24ae0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00b9ee0 tx=0x7fcfd00be430 comp rx=0 tx=0).stop 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 shutdown_connections 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad36c80 0x55807ad39070 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad24ae0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfbe7fc640 0 client.15633 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad245a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 --2- 172.21.15.90:0/469126198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad24060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 >> 172.21.15.90:0/469126198 conn(0x55807ad18410 msgr2=0x55807ad175b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 shutdown_connections 2024-04-26T18:14:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 1 -- 172.21.15.90:0/469126198 wait complete. 2024-04-26T18:14:50.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.475+0000 7fcfe0025740 20 client.15633 trim_cache size 0 max 16384 2024-04-26T18:14:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 Processor -- start 2024-04-26T18:14:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 -- start start 2024-04-26T18:14:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad32780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad17430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad3e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd81b0b70 con 0x55807ad270e0 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd81ae440 con 0x55807ad1bf00 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfe0025740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd81ac290 con 0x55807ad1c820 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad3e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfd759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad3e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35866/0 (socket says 172.21.15.90:35866) 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad17430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfd759e640 1 -- 172.21.15.90:0/2602523294 learned_addr learned my addr 172.21.15.90:0/2602523294 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.490+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad32780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfd759e640 1 -- 172.21.15.90:0/2602523294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 msgr2=0x55807ad17430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfd759e640 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad17430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfd759e640 1 -- 172.21.15.90:0/2602523294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 msgr2=0x55807ad32780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfd759e640 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad32780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfd759e640 1 -- 172.21.15.90:0/2602523294 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807ad393e0 con 0x55807ad270e0 2024-04-26T18:14:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfd759e640 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad3e980 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0006010 tx=0x7fcfd00b9d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634d620dcd8cee81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfccff9640 1 -- 172.21.15.90:0/2602523294 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00bda80 con 0x55807ad270e0 2024-04-26T18:14:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfccff9640 1 -- 172.21.15.90:0/2602523294 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd0036030 con 0x55807ad270e0 2024-04-26T18:14:50.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.491+0000 7fcfccff9640 1 -- 172.21.15.90:0/2602523294 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00bdce0 con 0x55807ad270e0 2024-04-26T18:14:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- 172.21.15.90:0/2602523294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 msgr2=0x55807ad3e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad3e980 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0006010 tx=0x7fcfd00b9d20 comp rx=0 tx=0).stop 2024-04-26T18:14:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- 172.21.15.90:0/2602523294 shutdown_connections 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad270e0 0x55807ad3e980 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad1c820 0x55807ad17430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 --2- 172.21.15.90:0/2602523294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad32780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- 172.21.15.90:0/2602523294 >> 172.21.15.90:0/2602523294 conn(0x55807ad18410 msgr2=0x55807ad16d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- 172.21.15.90:0/2602523294 shutdown_connections 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- 172.21.15.90:0/2602523294 wait complete. 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- start start 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 Processor -- start 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad2eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad2eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad270e0 0x55807ad2ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd81ac900 con 0x55807ad1c820 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd81add70 con 0x55807ad1bf00 2024-04-26T18:14:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.492+0000 7fcfe0025740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd81acfd0 con 0x55807ad270e0 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfe0025740 10 client.0 _ll_register_callbacks cb 0x7fcfc0009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad2eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad2eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35874/0 (socket says 172.21.15.90:35874) 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 -- 172.21.15.90:0/1083351119 learned_addr learned my addr 172.21.15.90:0/1083351119 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfd6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad2eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfd759e640 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad270e0 0x55807ad2ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 -- 172.21.15.90:0/1083351119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad270e0 msgr2=0x55807ad2ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad270e0 0x55807ad2ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 -- 172.21.15.90:0/1083351119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 msgr2=0x55807ad2eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad2eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 -- 172.21.15.90:0/1083351119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55807ad2bfc0 con 0x55807ad1c820 2024-04-26T18:14:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfcffff640 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad2eff0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00045c0 tx=0x7fcfc0009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c891b30c88bfc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/1083351119 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0022070 con 0x55807ad1c820 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/1083351119 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fcfc0011880 con 0x55807ad1c820 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd0036030 con 0x55807ad1c820 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/1083351119 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfc002d050 con 0x55807ad1c820 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfe0025740 10 client.15639 fetch_fsmap learned FSMap version 27 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55807ad9da60 con 0x55807ad1c820 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.493+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/1083351119 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd0036030 con 0x55807ad1c820 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/1083351119 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcfc001ec40 con 0x55807ad1c820 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfe0025740 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfe0025740 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfe0025740 20 client.15639 populate_metadata read hostname 'smithi090' 2024-04-26T18:14:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfe0025740 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad9e2a0 0x55807ada06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:14:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfe0025740 4 client.15639 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:14:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55807ac89ee0 con 0x55807ad9e2a0 2024-04-26T18:14:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad9e2a0 0x55807ada06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:14:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfd6d9d640 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad9e2a0 0x55807ada06c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39983ca60d0b385c server_cookie=c3a08d25b4a6567e in_seq=0 out_seq=0 2024-04-26T18:14:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.494+0000 7fcfbe7fc640 10 client.15639 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfbe7fc640 1 -- 172.21.15.90:0/1083351119 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55807ac89ee0 con 0x55807ad9e2a0 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfbe7fc640 10 client.15639 handle_command_reply: tid=0 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 client.15639 shutdown 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 msgr2=0x55807ad2eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad2eff0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00045c0 tx=0x7fcfc0009b70 comp rx=0 tx=0).stop 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 shutdown_connections 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55807ad9e2a0 0x55807ada06c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55807ad270e0 0x55807ad2ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfbe7fc640 0 client.15639 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:14:50.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55807ad1c820 0x55807ad2eff0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 --2- 172.21.15.90:0/1083351119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55807ad1bf00 0x55807ad2eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 >> 172.21.15.90:0/1083351119 conn(0x55807ad18410 msgr2=0x55807ad24890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 shutdown_connections 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 1 -- 172.21.15.90:0/1083351119 wait complete. 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.565+0000 7fcfe0025740 20 client.15639 trim_cache size 0 max 16384 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:14:50.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:14:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc40a3dc0 msgr2=0x7fcfc40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8009b50 tx=0x7fcfc8005f90 comp rx=0 tx=0).stop 2024-04-26T18:14:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd81067d0 msgr2=0x7fcfd819d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:14:50.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd81067d0 0x7fcfd819d8f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0005cc0 tx=0x7fcfd0031700 comp rx=0 tx=0).stop 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 shutdown_connections 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd81067d0 0x7fcfd819d8f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd8102540 0x7fcfd8199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 --2- 172.21.15.90:0/906987332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd8101b80 0x7fcfd8199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 >> 172.21.15.90:0/906987332 conn(0x7fcfd8077630 msgr2=0x7fcfd806b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 shutdown_connections 2024-04-26T18:14:50.569 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:14:50.566+0000 7fcfdd22a640 1 -- 172.21.15.90:0/906987332 wait complete. 2024-04-26T18:14:50.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 64118 9.60 MB/sec warmup 38 sec latency 67.374 ms 2024-04-26T18:14:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:50 smithi119 ceph-mon[32752]: pgmap v89: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-04-26T18:14:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:50 smithi167 ceph-mon[32751]: pgmap v89: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-04-26T18:14:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:50 smithi090 ceph-mon[29180]: pgmap v89: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-04-26T18:14:51.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 65716 9.50 MB/sec warmup 39 sec latency 39.065 ms 2024-04-26T18:14:52.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 67990 9.61 MB/sec warmup 40 sec latency 17.131 ms 2024-04-26T18:14:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:52 smithi119 ceph-mon[32752]: pgmap v90: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 288 op/s 2024-04-26T18:14:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:52 smithi167 ceph-mon[32751]: pgmap v90: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 288 op/s 2024-04-26T18:14:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:52 smithi090 ceph-mon[29180]: pgmap v90: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 288 op/s 2024-04-26T18:14:53.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 70960 9.80 MB/sec warmup 41 sec latency 11.368 ms 2024-04-26T18:14:54.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 72388 9.73 MB/sec warmup 42 sec latency 18.425 ms 2024-04-26T18:14:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:54 smithi119 ceph-mon[32752]: pgmap v91: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 447 op/s 2024-04-26T18:14:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:54 smithi167 ceph-mon[32751]: pgmap v91: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 447 op/s 2024-04-26T18:14:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:54 smithi090 ceph-mon[29180]: pgmap v91: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 447 op/s 2024-04-26T18:14:55.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 74916 9.83 MB/sec warmup 43 sec latency 10.966 ms 2024-04-26T18:14:56.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 77139 9.79 MB/sec warmup 44 sec latency 16.228 ms 2024-04-26T18:14:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:56 smithi119 ceph-mon[32752]: pgmap v92: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-04-26T18:14:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:56 smithi167 ceph-mon[32751]: pgmap v92: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-04-26T18:14:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:56 smithi090 ceph-mon[29180]: pgmap v92: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-04-26T18:14:57.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 78989 9.83 MB/sec warmup 45 sec latency 24.104 ms 2024-04-26T18:14:58.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 81890 10.01 MB/sec warmup 46 sec latency 24.937 ms 2024-04-26T18:14:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:14:58 smithi119 ceph-mon[32752]: pgmap v93: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-04-26T18:14:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:14:58 smithi167 ceph-mon[32751]: pgmap v93: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-04-26T18:14:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:14:58 smithi090 ceph-mon[29180]: pgmap v93: 97 pgs: 97 active+clean; 34 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 262 op/s 2024-04-26T18:14:59.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 83896 9.93 MB/sec warmup 47 sec latency 13.003 ms 2024-04-26T18:15:00.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 85772 10.00 MB/sec warmup 48 sec latency 14.277 ms 2024-04-26T18:15:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:00 smithi119 ceph-mon[32752]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 366 op/s 2024-04-26T18:15:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:00 smithi167 ceph-mon[32751]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 366 op/s 2024-04-26T18:15:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:00 smithi090 ceph-mon[29180]: pgmap v94: 97 pgs: 97 active+clean; 39 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 366 op/s 2024-04-26T18:15:01.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 87730 9.96 MB/sec warmup 49 sec latency 22.157 ms 2024-04-26T18:15:02.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 89506 9.99 MB/sec warmup 50 sec latency 52.949 ms 2024-04-26T18:15:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:02 smithi119 ceph-mon[32752]: pgmap v95: 97 pgs: 97 active+clean; 39 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-04-26T18:15:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:02 smithi167 ceph-mon[32751]: pgmap v95: 97 pgs: 97 active+clean; 39 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-04-26T18:15:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:02 smithi090 ceph-mon[29180]: pgmap v95: 97 pgs: 97 active+clean; 39 MiB data, 602 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-04-26T18:15:03.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 92419 10.14 MB/sec warmup 51 sec latency 19.519 ms 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.666+0000 7fcfdbf96640 1 -- 172.21.15.90:0/1902921634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4107820 msgr2=0x7fcfd4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.666+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/1902921634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4107820 0x7fcfd4107c00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009930 tx=0x7fcfd002a420 comp rx=0 tx=0).stop 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.666+0000 7fcfdbf96640 1 -- 172.21.15.90:0/1902921634 shutdown_connections 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.666+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/1902921634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd4102220 0x7fcfd410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.666+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/1902921634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd4101860 0x7fcfd4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.666+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/1902921634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4107820 0x7fcfd4107c00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.666+0000 7fcfdbf96640 1 -- 172.21.15.90:0/1902921634 >> 172.21.15.90:0/1902921634 conn(0x7fcfd40789b0 msgr2=0x7fcfd4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 -- 172.21.15.90:0/1902921634 shutdown_connections 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 -- 172.21.15.90:0/1902921634 wait complete. 2024-04-26T18:15:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 Processor -- start 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 -- start start 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd4101860 0x7fcfd41990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4102220 0x7fcfd41995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd4107820 0x7fcfd419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4111ae0 con 0x7fcfd4102220 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4111960 con 0x7fcfd4101860 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.667+0000 7fcfdbf96640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4111c60 con 0x7fcfd4107820 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4102220 0x7fcfd41995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4102220 0x7fcfd41995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48142/0 (socket says 172.21.15.90:48142) 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9d32640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd4101860 0x7fcfd41990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 -- 172.21.15.90:0/3992727123 learned_addr learned my addr 172.21.15.90:0/3992727123 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfda533640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd4107820 0x7fcfd419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 -- 172.21.15.90:0/3992727123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd4107820 msgr2=0x7fcfd419d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd4107820 0x7fcfd419d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 -- 172.21.15.90:0/3992727123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd4101860 msgr2=0x7fcfd41990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd4101860 0x7fcfd41990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 -- 172.21.15.90:0/3992727123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd419e0a0 con 0x7fcfd4102220 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd4101860 0x7fcfd41990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfd9531640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4102220 0x7fcfd41995f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d770 tx=0x7fcfc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfcaffd640 1 -- 172.21.15.90:0/3992727123 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4017070 con 0x7fcfd4102220 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd419e390 con 0x7fcfd4102220 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd41a5c20 con 0x7fcfd4102220 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfcaffd640 1 -- 172.21.15.90:0/3992727123 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc40054d0 con 0x7fcfd4102220 2024-04-26T18:15:03.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.668+0000 7fcfcaffd640 1 -- 172.21.15.90:0/3992727123 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4020060 con 0x7fcfd4102220 2024-04-26T18:15:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.670+0000 7fcfcaffd640 1 -- 172.21.15.90:0/3992727123 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fcfc4009c30 con 0x7fcfd4102220 2024-04-26T18:15:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.670+0000 7fcfded88740 1 Processor -- start 2024-04-26T18:15:03.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfded88740 1 -- start start 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfded88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfded88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50be820 0x556bd50c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfcaffd640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc00a3dc0 0x7fcfc00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfded88740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfded88740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556bd5024460 con 0x556bd50c90e0 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfded88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556bd4fde590 con 0x556bd50bdf00 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfded88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556bd5020420 con 0x556bd50be820 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfd9531640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc00a3dc0 0x7fcfc00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfd9d32640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50be820 0x556bd50c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48158/0 (socket says 172.21.15.90:48158) 2024-04-26T18:15:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 -- 172.21.15.90:0/2318916713 learned_addr learned my addr 172.21.15.90:0/2318916713 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfcaffd640 1 -- 172.21.15.90:0/3992727123 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fcfc40d0020 con 0x7fcfd4102220 2024-04-26T18:15:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 -- 172.21.15.90:0/2318916713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50be820 msgr2=0x556bd50c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50be820 0x556bd50c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 -- 172.21.15.90:0/2318916713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 msgr2=0x556bd50be2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50be2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfda533640 1 -- 172.21.15.90:0/2318916713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd4feb0d0 con 0x556bd50c90e0 2024-04-26T18:15:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.671+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50be820 0x556bd50c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.672+0000 7fcfd9531640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc00a3dc0 0x7fcfc00a6280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfc409c070 tx=0x7fcfc406db20 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:03.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.672+0000 7fcfda533640 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50cb4d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc005f40 tx=0x7fcfcc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e68e6d0f8700c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.672+0000 7fcfbffff640 1 -- 172.21.15.90:0/2318916713 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011130 con 0x556bd50c90e0 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.672+0000 7fcfbffff640 1 -- 172.21.15.90:0/2318916713 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfcc0112d0 con 0x556bd50c90e0 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.672+0000 7fcfbffff640 1 -- 172.21.15.90:0/2318916713 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00fb70 con 0x556bd50c90e0 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.672+0000 7fcfded88740 1 -- 172.21.15.90:0/2318916713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 msgr2=0x556bd50cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.672+0000 7fcfded88740 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50cb4d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc005f40 tx=0x7fcfcc005bd0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- 172.21.15.90:0/2318916713 shutdown_connections 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50cb4d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50be820 0x556bd50c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 --2- 172.21.15.90:0/2318916713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50be2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- 172.21.15.90:0/2318916713 >> 172.21.15.90:0/2318916713 conn(0x556bd50ba410 msgr2=0x556bd50ba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- 172.21.15.90:0/2318916713 shutdown_connections 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- 172.21.15.90:0/2318916713 wait complete. 2024-04-26T18:15:03.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- start start 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 Processor -- start 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556bd50cbd10 con 0x556bd50c90e0 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556bd4fabb60 con 0x556bd50be820 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.673+0000 7fcfded88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556bd50cbe90 con 0x556bd50bdf00 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfded88740 10 client.0 _ll_register_callbacks cb 0x556bd4fd02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfda533640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfda533640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48162/0 (socket says 172.21.15.90:48162) 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfda533640 1 -- 172.21.15.90:0/3727457298 learned_addr learned my addr 172.21.15.90:0/3727457298 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:03.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfda533640 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfd9d32640 1 -- 172.21.15.90:0/3727457298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 msgr2=0x556bd50c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfd9d32640 1 -- 172.21.15.90:0/3727457298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 msgr2=0x556bd50c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfd9d32640 1 -- 172.21.15.90:0/3727457298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bd4d36210 con 0x556bd50be820 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c6560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0031440 tx=0x7fcfd0031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2547aa9af3054b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/3727457298 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0035040 con 0x556bd50be820 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfcc0112d0 con 0x556bd50be820 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.674+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/3727457298 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fcfd0004450 con 0x556bd50be820 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/3727457298 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfd0031900 con 0x556bd50be820 2024-04-26T18:15:03.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 10 client.25190 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bd4e02620 con 0x556bd50be820 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/3727457298 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfcc0112d0 con 0x556bd50be820 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/3727457298 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcfd0050020 con 0x556bd50be820 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 10 client.25190 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 10 client.25190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 20 client.25190 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd50d9260 0x556bd50db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 4 client.25190 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556bd5032070 con 0x556bd50d9260 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.675+0000 7fcfda533640 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd50d9260 0x556bd50db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.676+0000 7fcfbd7fa640 10 client.25190 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.676+0000 7fcfda533640 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd50d9260 0x556bd50db670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927ff703d466d4f9 server_cookie=5f22d899d4ead2b3 in_seq=0 out_seq=0 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.676+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/3727457298 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556bd5032070 con 0x556bd50d9260 2024-04-26T18:15:03.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.676+0000 7fcfbd7fa640 10 client.25190 handle_command_reply: tid=0 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.676+0000 7fcfded88740 1 client.25190 shutdown 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 msgr2=0x556bd50c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c6560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0031440 tx=0x7fcfd0031810 comp rx=0 tx=0).stop 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 shutdown_connections 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd50d9260 0x556bd50db670 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c6560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 --2- 172.21.15.90:0/3727457298 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 >> 172.21.15.90:0/3727457298 conn(0x556bd50ba410 msgr2=0x556bd50c0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfbd7fa640 0 client.25190 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 shutdown_connections 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 1 -- 172.21.15.90:0/3727457298 wait complete. 2024-04-26T18:15:03.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.677+0000 7fcfded88740 20 client.25190 trim_cache size 0 max 16384 2024-04-26T18:15:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.692+0000 7fcfded88740 1 Processor -- start 2024-04-26T18:15:03.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.692+0000 7fcfded88740 1 -- start start 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.692+0000 7fcfded88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.692+0000 7fcfded88740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50be820 0x556bd50c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.692+0000 7fcfded88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50c90e0 0x556bd50e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfded88740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb8010e90 con 0x556bd50be820 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfded88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb800e760 con 0x556bd50bdf00 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfded88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb800c5b0 con 0x556bd50c90e0 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfd9d32640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50be820 0x556bd50c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfda533640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfda533640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50d4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:37224/0 (socket says 172.21.15.90:37224) 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfda533640 1 -- 172.21.15.90:0/2913495399 learned_addr learned my addr 172.21.15.90:0/2913495399 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfda533640 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50c90e0 0x556bd50e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfd9d32640 1 -- 172.21.15.90:0/2913495399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50c90e0 msgr2=0x556bd50e0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50c90e0 0x556bd50e0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfd9d32640 1 -- 172.21.15.90:0/2913495399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 msgr2=0x556bd50d4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50d4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfd9d32640 1 -- 172.21.15.90:0/2913495399 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bd50c0e00 con 0x556bd50be820 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfda533640 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50be820 0x556bd50c08c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00488e0 tx=0x7fcfd0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88012697e71746fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfbf7fe640 1 -- 172.21.15.90:0/2913495399 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0035030 con 0x556bd50be820 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfbf7fe640 1 -- 172.21.15.90:0/2913495399 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd0005850 con 0x556bd50be820 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.693+0000 7fcfbf7fe640 1 -- 172.21.15.90:0/2913495399 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00044b0 con 0x556bd50be820 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 -- 172.21.15.90:0/2913495399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50be820 msgr2=0x556bd50c08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50be820 0x556bd50c08c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcfd00488e0 tx=0x7fcfd0005fd0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 -- 172.21.15.90:0/2913495399 shutdown_connections 2024-04-26T18:15:03.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50c90e0 0x556bd50e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50be820 0x556bd50c08c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 --2- 172.21.15.90:0/2913495399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50bdf00 0x556bd50d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 -- 172.21.15.90:0/2913495399 >> 172.21.15.90:0/2913495399 conn(0x556bd50ba410 msgr2=0x556bd50c6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 -- 172.21.15.90:0/2913495399 shutdown_connections 2024-04-26T18:15:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 -- 172.21.15.90:0/2913495399 wait complete. 2024-04-26T18:15:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.694+0000 7fcfded88740 1 -- start start 2024-04-26T18:15:03.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 1 Processor -- start 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb800cc20 con 0x556bd50c90e0 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb800e090 con 0x556bd50be820 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb800d2f0 con 0x556bd50bdf00 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfded88740 10 client.0 _ll_register_callbacks cb 0x7fcfd0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48174/0 (socket says 172.21.15.90:48174) 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 -- 172.21.15.90:0/2976207541 learned_addr learned my addr 172.21.15.90:0/2976207541 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 -- 172.21.15.90:0/2976207541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 msgr2=0x556bd50c2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 -- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 msgr2=0x556bd50c3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 -- 172.21.15.90:0/2976207541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bd513e7b0 con 0x556bd50c90e0 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfda533640 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c3650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc006010 tx=0x7fcfcc005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25f7e5b6d461d1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.695+0000 7fcfbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/2976207541 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc031460 con 0x556bd50c90e0 2024-04-26T18:15:03.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd0005850 con 0x556bd50c90e0 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/2976207541 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fcfcc02a040 con 0x556bd50c90e0 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/2976207541 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fcfcc031600 con 0x556bd50c90e0 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 10 client.15657 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556bd513f950 con 0x556bd50c90e0 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/2976207541 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd0005850 con 0x556bd50c90e0 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/2976207541 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fcfcc030040 con 0x556bd50c90e0 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 20 client.15657 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd513ff60 0x556bd5142380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 4 client.15657 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556bd502bee0 con 0x556bd513ff60 2024-04-26T18:15:03.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.696+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd513ff60 0x556bd5142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.697+0000 7fcfbd7fa640 10 client.15657 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:03.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.697+0000 7fcfd9d32640 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd513ff60 0x556bd5142380 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83331d6e14d81deb server_cookie=6d7912f819b61c39 in_seq=0 out_seq=0 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfbd7fa640 1 -- 172.21.15.90:0/2976207541 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556bd502bee0 con 0x556bd513ff60 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfbd7fa640 10 client.15657 handle_command_reply: tid=0 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 client.15657 shutdown 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 msgr2=0x556bd50c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c3650 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc006010 tx=0x7fcfcc005bd0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 shutdown_connections 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556bd513ff60 0x556bd5142380 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556bd50c90e0 0x556bd50c3650 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556bd50be820 0x556bd50c3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 --2- 172.21.15.90:0/2976207541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556bd50bdf00 0x556bd50c2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfbd7fa640 0 client.15657 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 >> 172.21.15.90:0/2976207541 conn(0x556bd50ba410 msgr2=0x556bd50c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 shutdown_connections 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 1 -- 172.21.15.90:0/2976207541 wait complete. 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.751+0000 7fcfded88740 20 client.15657 trim_cache size 0 max 16384 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:15:03.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:15:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc00a3dc0 msgr2=0x7fcfc00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc00a3dc0 0x7fcfc00a6280 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcfc409c070 tx=0x7fcfc406db20 comp rx=0 tx=0).stop 2024-04-26T18:15:03.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4102220 msgr2=0x7fcfd41995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4102220 0x7fcfd41995f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400d770 tx=0x7fcfc400dc30 comp rx=0 tx=0).stop 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 shutdown_connections 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcfc00a3dc0 0x7fcfc00a6280 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcfd4107820 0x7fcfd419d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcfd4102220 0x7fcfd41995f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 --2- 172.21.15.90:0/3992727123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfd4101860 0x7fcfd41990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 >> 172.21.15.90:0/3992727123 conn(0x7fcfd40789b0 msgr2=0x7fcfd4104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 shutdown_connections 2024-04-26T18:15:03.755 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:03.752+0000 7fcfdbf96640 1 -- 172.21.15.90:0/3992727123 wait complete. 2024-04-26T18:15:04.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 93937 10.07 MB/sec warmup 52 sec latency 40.725 ms 2024-04-26T18:15:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:04 smithi119 ceph-mon[32752]: pgmap v96: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 396 op/s 2024-04-26T18:15:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:04 smithi167 ceph-mon[32751]: pgmap v96: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 396 op/s 2024-04-26T18:15:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:04 smithi090 ceph-mon[29180]: pgmap v96: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 396 op/s 2024-04-26T18:15:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:15:05.659 INFO:tasks.workunit.client.0.smithi090.stdout: 1 96272 10.14 MB/sec warmup 53 sec latency 10.098 ms 2024-04-26T18:15:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:15:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:15:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:15:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:15:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:15:06.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 98830 10.18 MB/sec warmup 54 sec latency 14.818 ms 2024-04-26T18:15:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:06 smithi119 ceph-mon[32752]: pgmap v97: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-04-26T18:15:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:06 smithi167 ceph-mon[32751]: pgmap v97: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-04-26T18:15:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:06 smithi090 ceph-mon[29180]: pgmap v97: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-04-26T18:15:07.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 100497 10.15 MB/sec warmup 55 sec latency 11.637 ms 2024-04-26T18:15:08.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 103159 10.27 MB/sec warmup 56 sec latency 23.566 ms 2024-04-26T18:15:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:08 smithi119 ceph-mon[32752]: pgmap v98: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-04-26T18:15:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:08 smithi167 ceph-mon[32751]: pgmap v98: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-04-26T18:15:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:08 smithi090 ceph-mon[29180]: pgmap v98: 97 pgs: 97 active+clean; 44 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.2 MiB/s wr, 237 op/s 2024-04-26T18:15:09.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 105245 10.22 MB/sec warmup 57 sec latency 12.629 ms 2024-04-26T18:15:10.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 107070 10.25 MB/sec warmup 58 sec latency 14.668 ms 2024-04-26T18:15:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:10 smithi119 ceph-mon[32752]: pgmap v99: 97 pgs: 97 active+clean; 46 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 387 op/s 2024-04-26T18:15:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:10 smithi167 ceph-mon[32751]: pgmap v99: 97 pgs: 97 active+clean; 46 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 387 op/s 2024-04-26T18:15:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:10 smithi090 ceph-mon[29180]: pgmap v99: 97 pgs: 97 active+clean; 46 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.9 MiB/s wr, 387 op/s 2024-04-26T18:15:11.663 INFO:tasks.workunit.client.0.smithi090.stdout: 1 110105 10.38 MB/sec warmup 59 sec latency 11.950 ms 2024-04-26T18:15:12.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 111493 10.32 MB/sec warmup 60 sec latency 17.675 ms 2024-04-26T18:15:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:12 smithi119 ceph-mon[32752]: pgmap v100: 97 pgs: 97 active+clean; 46 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-04-26T18:15:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:12 smithi167 ceph-mon[32751]: pgmap v100: 97 pgs: 97 active+clean; 46 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-04-26T18:15:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:12 smithi090 ceph-mon[29180]: pgmap v100: 97 pgs: 97 active+clean; 46 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.3 MiB/s wr, 283 op/s 2024-04-26T18:15:13.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 113932 10.38 MB/sec warmup 61 sec latency 11.244 ms 2024-04-26T18:15:14.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 115817 10.31 MB/sec warmup 62 sec latency 14.642 ms 2024-04-26T18:15:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:14 smithi119 ceph-mon[32752]: pgmap v101: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 393 op/s 2024-04-26T18:15:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:14 smithi167 ceph-mon[32751]: pgmap v101: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 393 op/s 2024-04-26T18:15:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:14 smithi090 ceph-mon[29180]: pgmap v101: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.0 MiB/s wr, 393 op/s 2024-04-26T18:15:15.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 117637 10.35 MB/sec warmup 63 sec latency 63.377 ms 2024-04-26T18:15:16.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 120466 10.41 MB/sec warmup 64 sec latency 11.179 ms 2024-04-26T18:15:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 -- 172.21.15.90:0/3082142522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40102220 msgr2=0x7f2a4010e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 --2- 172.21.15.90:0/3082142522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40102220 0x7f2a4010e6e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009cd0 tx=0x7f2a3802a8e0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 -- 172.21.15.90:0/3082142522 shutdown_connections 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 --2- 172.21.15.90:0/3082142522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40102220 0x7f2a4010e6e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 --2- 172.21.15.90:0/3082142522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a40101860 0x7f2a40101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 --2- 172.21.15.90:0/3082142522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a40107820 0x7f2a40107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 -- 172.21.15.90:0/3082142522 >> 172.21.15.90:0/3082142522 conn(0x7f2a400789b0 msgr2=0x7f2a40078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 -- 172.21.15.90:0/3082142522 shutdown_connections 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.842+0000 7f2a45283640 1 -- 172.21.15.90:0/3082142522 wait complete. 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 Processor -- start 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 -- start start 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40101860 0x7f2a40197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a40102220 0x7f2a40198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a40107820 0x7f2a4019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40111ae0 con 0x7f2a40101860 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40111960 con 0x7f2a40102220 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a45283640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40111c60 con 0x7f2a40107820 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40101860 0x7f2a40197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40101860 0x7f2a40197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35390/0 (socket says 172.21.15.90:35390) 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 -- 172.21.15.90:0/2260747211 learned_addr learned my addr 172.21.15.90:0/2260747211 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a37fff640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a40102220 0x7f2a40198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a40107820 0x7f2a4019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 -- 172.21.15.90:0/2260747211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a40107820 msgr2=0x7f2a4019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a40107820 0x7f2a4019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 -- 172.21.15.90:0/2260747211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a40102220 msgr2=0x7f2a40198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a40102220 0x7f2a40198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 -- 172.21.15.90:0/2260747211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4019cef0 con 0x7f2a40101860 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3effd640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40101860 0x7f2a40197f00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000eb00 tx=0x7f2a3000ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.843+0000 7f2a3cff9640 1 -- 172.21.15.90:0/2260747211 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000bc30 con 0x7f2a40101860 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.844+0000 7f2a3cff9640 1 -- 172.21.15.90:0/2260747211 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a3000bdd0 con 0x7f2a40101860 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.844+0000 7f2a3cff9640 1 -- 172.21.15.90:0/2260747211 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30017410 con 0x7f2a40101860 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.844+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a4019d1e0 con 0x7f2a40101860 2024-04-26T18:15:16.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.844+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a40069fe0 con 0x7f2a40101860 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.846+0000 7f2a3cff9640 1 -- 172.21.15.90:0/2260747211 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f2a30004f70 con 0x7f2a40101860 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.846+0000 7f2a3cff9640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2a2c0a3e40 0x7f2a2c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.846+0000 7f2a47fa6740 1 Processor -- start 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.846+0000 7f2a47fa6740 1 -- start start 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.846+0000 7f2a37fff640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2a2c0a3e40 0x7f2a2c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.846+0000 7f2a3cff9640 1 -- 172.21.15.90:0/2260747211 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2a300d1f30 con 0x7f2a40101860 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e97c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e986ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e9894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.854 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55871e8e2460 con 0x55871e9870e0 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55871e89c590 con 0x55871e97bf00 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55871e8de420 con 0x55871e97c820 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e97c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e97c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:35422/0 (socket says 172.21.15.90:35422) 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 -- 172.21.15.90:0/147704798 learned_addr learned my addr 172.21.15.90:0/147704798 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a37fff640 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e986ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e9894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a37fff640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2a2c0a3e40 0x7f2a2c0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a40078f20 tx=0x7f2a28005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 -- 172.21.15.90:0/147704798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 msgr2=0x55871e986ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e986ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 -- 172.21.15.90:0/147704798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 msgr2=0x55871e9894d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e9894d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 -- 172.21.15.90:0/147704798 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871e8a90d0 con 0x55871e97bf00 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a3effd640 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e97c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a3009a9c0 tx=0x7f2a30005350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66466a03189bdf05 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a357fa640 1 -- 172.21.15.90:0/147704798 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300b7070 con 0x55871e97bf00 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a357fa640 1 -- 172.21.15.90:0/147704798 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a3009c090 con 0x55871e97bf00 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.847+0000 7f2a357fa640 1 -- 172.21.15.90:0/147704798 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300b7210 con 0x55871e97bf00 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 -- 172.21.15.90:0/147704798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 msgr2=0x55871e97c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e97c2e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a3009a9c0 tx=0x7f2a30005350 comp rx=0 tx=0).stop 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 -- 172.21.15.90:0/147704798 shutdown_connections 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e9894d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e986ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/147704798 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e97c2e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 -- 172.21.15.90:0/147704798 >> 172.21.15.90:0/147704798 conn(0x55871e978410 msgr2=0x55871e9787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.848+0000 7f2a47fa6740 1 -- 172.21.15.90:0/147704798 shutdown_connections 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 -- 172.21.15.90:0/147704798 wait complete. 2024-04-26T18:15:16.856 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 -- start start 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 Processor -- start 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e984080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e9845c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e984b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55871e989d10 con 0x55871e9870e0 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55871e869b60 con 0x55871e97bf00 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55871e989e90 con 0x55871e97c820 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a47fa6740 10 client.0 _ll_register_callbacks cb 0x55871e88e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e984b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e984b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35406/0 (socket says 172.21.15.90:35406) 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a37fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e9845c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/3660419578 learned_addr learned my addr 172.21.15.90:0/3660419578 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3effd640 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e984080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/3660419578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 msgr2=0x55871e9845c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e9845c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 msgr2=0x55871e984080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e984080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/3660419578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871e5f4210 con 0x55871e9870e0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.849+0000 7f2a3effd640 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e984080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e984b00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2a38035000 tx=0x7f2a38031a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9971afc307d0004e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a267fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a267fc640 1 -- 172.21.15.90:0/3660419578 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38003b70 con 0x55871e9870e0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a267fc640 1 -- 172.21.15.90:0/3660419578 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2a38003d10 con 0x55871e9870e0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a3009c090 con 0x55871e9870e0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a267fc640 1 -- 172.21.15.90:0/3660419578 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a38004040 con 0x55871e9870e0 2024-04-26T18:15:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 10 client.15669 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55871e6c0620 con 0x55871e9870e0 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a267fc640 1 -- 172.21.15.90:0/3660419578 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a3009c090 con 0x55871e9870e0 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a267fc640 1 -- 172.21.15.90:0/3660419578 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a38030070 con 0x55871e9870e0 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 10 client.15669 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 20 client.15669 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e997380 0x55871e999820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 4 client.15669 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.850+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55871e8f0070 con 0x55871e997380 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.851+0000 7f2a3effd640 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e997380 0x55871e999820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.851+0000 7f2a267fc640 10 client.15669 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:16.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.851+0000 7f2a3effd640 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e997380 0x55871e999820 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2821a2f4d01ce4a3 server_cookie=782ac97ca17af8a4 in_seq=0 out_seq=0 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.851+0000 7f2a267fc640 1 -- 172.21.15.90:0/3660419578 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55871e8f0070 con 0x55871e997380 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a267fc640 10 client.15669 handle_command_reply: tid=0 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 client.15669 shutdown 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 msgr2=0x55871e984b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e984b00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2a38035000 tx=0x7f2a38031a80 comp rx=0 tx=0).stop 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 shutdown_connections 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e997380 0x55871e999820 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e984b00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e9845c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/3660419578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e984080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a267fc640 0 client.15669 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 >> 172.21.15.90:0/3660419578 conn(0x55871e978410 msgr2=0x55871e97e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 shutdown_connections 2024-04-26T18:15:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 1 -- 172.21.15.90:0/3660419578 wait complete. 2024-04-26T18:15:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.852+0000 7f2a47fa6740 20 client.15669 trim_cache size 0 max 16384 2024-04-26T18:15:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.867+0000 7f2a47fa6740 1 Processor -- start 2024-04-26T18:15:16.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.867+0000 7f2a47fa6740 1 -- start start 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e977d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e97e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e99f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401aea10 con 0x55871e9870e0 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401ac2e0 con 0x55871e97bf00 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401aa130 con 0x55871e97c820 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e99f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e99f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35418/0 (socket says 172.21.15.90:35418) 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/627149024 learned_addr learned my addr 172.21.15.90:0/627149024 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a37fff640 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e97e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3effd640 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e977d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/627149024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 msgr2=0x55871e97e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e97e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/627149024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 msgr2=0x55871e977d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e977d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 -- 172.21.15.90:0/627149024 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55871e973d60 con 0x55871e9870e0 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.868+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e99f0f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2a3802ae00 tx=0x7f2a38030f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c0721c2e080854 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a34ff9640 1 -- 172.21.15.90:0/627149024 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380305a0 con 0x55871e9870e0 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a34ff9640 1 -- 172.21.15.90:0/627149024 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a38036040 con 0x55871e9870e0 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a34ff9640 1 -- 172.21.15.90:0/627149024 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38003cd0 con 0x55871e9870e0 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 -- 172.21.15.90:0/627149024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 msgr2=0x55871e99f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e99f0f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f2a3802ae00 tx=0x7f2a38030f40 comp rx=0 tx=0).stop 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 -- 172.21.15.90:0/627149024 shutdown_connections 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e9870e0 0x55871e99f0f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e97e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/627149024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e97bf00 0x55871e977d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 -- 172.21.15.90:0/627149024 >> 172.21.15.90:0/627149024 conn(0x55871e978410 msgr2=0x55871e9848d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 -- 172.21.15.90:0/627149024 shutdown_connections 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.869+0000 7f2a47fa6740 1 -- 172.21.15.90:0/627149024 wait complete. 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 -- start start 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 Processor -- start 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e97bf00 0x55871e98b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e98b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e9870e0 0x55871e9fc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401aa7a0 con 0x55871e97bf00 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401abc10 con 0x55871e9870e0 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401aae70 con 0x55871e97c820 2024-04-26T18:15:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a47fa6740 10 client.0 _ll_register_callbacks cb 0x7f2a38009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e97bf00 0x55871e98b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e97bf00 0x55871e98b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35430/0 (socket says 172.21.15.90:35430) 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 -- 172.21.15.90:0/2168401856 learned_addr learned my addr 172.21.15.90:0/2168401856 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3f7fe640 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e9870e0 0x55871e9fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a37fff640 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e98b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 -- 172.21.15.90:0/2168401856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 msgr2=0x55871e98b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e98b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 -- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e9870e0 msgr2=0x55871e9fc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e9870e0 0x55871e9fc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 -- 172.21.15.90:0/2168401856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55871e9fced0 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a3effd640 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e97bf00 0x55871e98b250 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2a3009a9c0 tx=0x7f2a300a6d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62ea75e8927571 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.870+0000 7f2a267fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a267fc640 1 -- 172.21.15.90:0/2168401856 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300bb020 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a267fc640 1 -- 172.21.15.90:0/2168401856 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f2a300b73e0 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a38036040 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a267fc640 1 -- 172.21.15.90:0/2168401856 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f2a300b7040 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 10 client.15675 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55871e9fe1e0 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a267fc640 1 -- 172.21.15.90:0/2168401856 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a38036040 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a267fc640 1 -- 172.21.15.90:0/2168401856 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f2a30084070 con 0x55871e97bf00 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 10 client.15675 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 20 client.15675 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e9fe4e0 0x55871ea008d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 4 client.15675 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:15:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55871e8e9ee0 con 0x55871e9fe4e0 2024-04-26T18:15:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.871+0000 7f2a37fff640 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e9fe4e0 0x55871ea008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.872+0000 7f2a267fc640 10 client.15675 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.872+0000 7f2a37fff640 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e9fe4e0 0x55871ea008d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ab20a4492cab84 server_cookie=10977da45115d1b9 in_seq=0 out_seq=0 2024-04-26T18:15:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.936+0000 7f2a267fc640 1 -- 172.21.15.90:0/2168401856 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55871e8e9ee0 con 0x55871e9fe4e0 2024-04-26T18:15:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.936+0000 7f2a267fc640 10 client.15675 handle_command_reply: tid=0 2024-04-26T18:15:16.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.936+0000 7f2a47fa6740 1 client.15675 shutdown 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e97bf00 msgr2=0x55871e98b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e97bf00 0x55871e98b250 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2a3009a9c0 tx=0x7f2a300a6d30 comp rx=0 tx=0).stop 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 shutdown_connections 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55871e9fe4e0 0x55871ea008d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55871e9870e0 0x55871e9fc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55871e97c820 0x55871e98b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 --2- 172.21.15.90:0/2168401856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55871e97bf00 0x55871e98b250 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 >> 172.21.15.90:0/2168401856 conn(0x55871e978410 msgr2=0x55871e9858b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a267fc640 0 client.15675 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 shutdown_connections 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 1 -- 172.21.15.90:0/2168401856 wait complete. 2024-04-26T18:15:16.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.937+0000 7f2a47fa6740 20 client.15675 trim_cache size 0 max 16384 2024-04-26T18:15:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:15:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:15:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:15:16.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2a2c0a3e40 msgr2=0x7f2a2c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2a2c0a3e40 0x7f2a2c0a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a40078f20 tx=0x7f2a28005f90 comp rx=0 tx=0).stop 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40101860 msgr2=0x7f2a40197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40101860 0x7f2a40197f00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000eb00 tx=0x7f2a3000ef00 comp rx=0 tx=0).stop 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 shutdown_connections 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2a2c0a3e40 0x7f2a2c0a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a40107820 0x7f2a4019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a40102220 0x7f2a40198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 --2- 172.21.15.90:0/2260747211 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a40101860 0x7f2a40197f00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:16.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 >> 172.21.15.90:0/2260747211 conn(0x7f2a400789b0 msgr2=0x7f2a40104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 shutdown_connections 2024-04-26T18:15:16.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:16.938+0000 7f2a45283640 1 -- 172.21.15.90:0/2260747211 wait complete. 2024-04-26T18:15:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:16 smithi119 ceph-mon[32752]: pgmap v102: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-26T18:15:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:16 smithi167 ceph-mon[32751]: pgmap v102: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-26T18:15:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:16 smithi090 ceph-mon[29180]: pgmap v102: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-26T18:15:17.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 121788 10.36 MB/sec warmup 65 sec latency 63.887 ms 2024-04-26T18:15:18.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 124481 10.45 MB/sec warmup 66 sec latency 14.073 ms 2024-04-26T18:15:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:18 smithi119 ceph-mon[32752]: pgmap v103: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-26T18:15:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:18 smithi167 ceph-mon[32751]: pgmap v103: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-26T18:15:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:18 smithi090 ceph-mon[29180]: pgmap v103: 97 pgs: 97 active+clean; 53 MiB data, 653 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.4 MiB/s wr, 259 op/s 2024-04-26T18:15:19.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 126604 10.41 MB/sec warmup 67 sec latency 16.630 ms 2024-04-26T18:15:20.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 128237 10.43 MB/sec warmup 68 sec latency 51.487 ms 2024-04-26T18:15:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:20 smithi119 ceph-mon[32752]: pgmap v104: 97 pgs: 97 active+clean; 49 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 436 op/s 2024-04-26T18:15:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:20 smithi167 ceph-mon[32751]: pgmap v104: 97 pgs: 97 active+clean; 49 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 436 op/s 2024-04-26T18:15:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:20 smithi090 ceph-mon[29180]: pgmap v104: 97 pgs: 97 active+clean; 49 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 436 op/s 2024-04-26T18:15:21.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 130897 10.47 MB/sec warmup 69 sec latency 25.860 ms 2024-04-26T18:15:22.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 132384 10.44 MB/sec warmup 70 sec latency 14.104 ms 2024-04-26T18:15:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:22 smithi119 ceph-mon[32752]: pgmap v105: 97 pgs: 97 active+clean; 49 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-04-26T18:15:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:22 smithi167 ceph-mon[32751]: pgmap v105: 97 pgs: 97 active+clean; 49 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-04-26T18:15:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:22 smithi090 ceph-mon[29180]: pgmap v105: 97 pgs: 97 active+clean; 49 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.1 MiB/s wr, 286 op/s 2024-04-26T18:15:23.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 135130 10.53 MB/sec warmup 71 sec latency 12.671 ms 2024-04-26T18:15:24.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 137299 10.49 MB/sec warmup 72 sec latency 13.168 ms 2024-04-26T18:15:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:24 smithi119 ceph-mon[32752]: pgmap v106: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-04-26T18:15:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:24 smithi167 ceph-mon[32751]: pgmap v106: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-04-26T18:15:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:24 smithi090 ceph-mon[29180]: pgmap v106: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.6 MiB/s wr, 416 op/s 2024-04-26T18:15:25.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 138961 10.50 MB/sec warmup 73 sec latency 34.001 ms 2024-04-26T18:15:26.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 141731 10.56 MB/sec warmup 74 sec latency 32.823 ms 2024-04-26T18:15:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:26 smithi119 ceph-mon[32752]: pgmap v107: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-26T18:15:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:26 smithi167 ceph-mon[32751]: pgmap v107: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-26T18:15:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:26 smithi090 ceph-mon[29180]: pgmap v107: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-26T18:15:27.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 143078 10.51 MB/sec warmup 75 sec latency 12.780 ms 2024-04-26T18:15:28.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 145727 10.59 MB/sec warmup 76 sec latency 15.325 ms 2024-04-26T18:15:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:28 smithi119 ceph-mon[32752]: pgmap v108: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-26T18:15:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:28 smithi167 ceph-mon[32751]: pgmap v108: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-26T18:15:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:28 smithi090 ceph-mon[29180]: pgmap v108: 97 pgs: 97 active+clean; 54 MiB data, 726 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.9 MiB/s wr, 306 op/s 2024-04-26T18:15:29.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 147661 10.53 MB/sec warmup 77 sec latency 17.989 ms 2024-04-26T18:15:30.029 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 -- 172.21.15.90:0/97878384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8101860 msgr2=0x7f71f8101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/97878384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8101860 0x7f71f8101ce0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009930 tx=0x7f71e002a420 comp rx=0 tx=0).stop 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 -- 172.21.15.90:0/97878384 shutdown_connections 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/97878384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8102220 0x7f71f810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/97878384 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8101860 0x7f71f8101ce0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/97878384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f8107820 0x7f71f8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 -- 172.21.15.90:0/97878384 >> 172.21.15.90:0/97878384 conn(0x7f71f80789b0 msgr2=0x7f71f8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 -- 172.21.15.90:0/97878384 shutdown_connections 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.027+0000 7f71fd5d4640 1 -- 172.21.15.90:0/97878384 wait complete. 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 Processor -- start 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 -- start start 2024-04-26T18:15:30.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8101860 0x7f71f8199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f8102220 0x7f71f819a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8107820 0x7f71f81a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8111ae0 con 0x7f71f8107820 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8111960 con 0x7f71f8101860 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71fd5d4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71f8111c60 con 0x7f71f8102220 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8107820 0x7f71f81a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8101860 0x7f71f8199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8107820 0x7f71f81a4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39080/0 (socket says 172.21.15.90:39080) 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8101860 0x7f71f8199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:43692/0 (socket says 172.21.15.90:43692) 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71effff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f8102220 0x7f71f819a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 -- 172.21.15.90:0/2410859351 learned_addr learned my addr 172.21.15.90:0/2410859351 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 -- 172.21.15.90:0/2410859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f8102220 msgr2=0x7f71f819a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f8102220 0x7f71f819a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 -- 172.21.15.90:0/2410859351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8101860 msgr2=0x7f71f8199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8101860 0x7f71f8199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.028+0000 7f71f77fe640 1 -- 172.21.15.90:0/2410859351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f81a54c0 con 0x7f71f8107820 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.029+0000 7f71f77fe640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8107820 0x7f71f81a4f80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f71f000eb10 tx=0x7f71f000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.029+0000 7f71f4ff9640 1 -- 172.21.15.90:0/2410859351 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004b20 con 0x7f71f8107820 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.029+0000 7f71f4ff9640 1 -- 172.21.15.90:0/2410859351 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f0004cc0 con 0x7f71f8107820 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.029+0000 7f71f4ff9640 1 -- 172.21.15.90:0/2410859351 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f0010670 con 0x7f71f8107820 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.029+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f81a5690 con 0x7f71f8107820 2024-04-26T18:15:30.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.029+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f81a5b60 con 0x7f71f8107820 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f71f4ff9640 1 -- 172.21.15.90:0/2410859351 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f71f0005020 con 0x7f71f8107820 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 Processor -- start 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 -- start start 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cf820 0x55d64c5d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d64c535460 con 0x55d64c5cef00 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d64c4ef590 con 0x55d64c5da0e0 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f72003cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d64c531420 con 0x55d64c5cf820 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f71effff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cf820 0x55d64c5d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.039 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f71effff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39086/0 (socket says 172.21.15.90:39086) 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f71effff640 1 -- 172.21.15.90:0/1472979100 learned_addr learned my addr 172.21.15.90:0/1472979100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f71effff640 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.032+0000 7f71f4ff9640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f71e40a3dc0 0x7f71e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f6ffd640 1 -- 172.21.15.90:0/1472979100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 msgr2=0x55d64c5dc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5dc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f6ffd640 1 -- 172.21.15.90:0/1472979100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 msgr2=0x55d64c5cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f6ffd640 1 -- 172.21.15.90:0/1472979100 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64c4fc0d0 con 0x55d64c5cf820 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71effff640 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f77fe640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f71e40a3dc0 0x7f71e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f4ff9640 1 -- 172.21.15.90:0/2410859351 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f71f00d0d50 con 0x7f71f8107820 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cf820 0x55d64c5d9ba0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009c00 tx=0x7f71e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4290056ef0a18ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:30.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71ed7fa640 1 -- 172.21.15.90:0/1472979100 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004050 con 0x55d64c5cf820 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71ed7fa640 1 -- 172.21.15.90:0/1472979100 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e80054d0 con 0x55d64c5cf820 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.033+0000 7f71ed7fa640 1 -- 172.21.15.90:0/1472979100 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004c40 con 0x55d64c5cf820 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- 172.21.15.90:0/1472979100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cf820 msgr2=0x55d64c5d9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f71f77fe640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f71e40a3dc0 0x7f71e40a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71f00ce5c0 tx=0x7f71f006f850 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cf820 0x55d64c5d9ba0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009c00 tx=0x7f71e8011810 comp rx=0 tx=0).stop 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- 172.21.15.90:0/1472979100 shutdown_connections 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cf820 0x55d64c5d9ba0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 --2- 172.21.15.90:0/1472979100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- 172.21.15.90:0/1472979100 >> 172.21.15.90:0/1472979100 conn(0x55d64c5cb410 msgr2=0x55d64c5cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- 172.21.15.90:0/1472979100 shutdown_connections 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- 172.21.15.90:0/1472979100 wait complete. 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- start start 2024-04-26T18:15:30.041 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 Processor -- start 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cef00 0x55d64c5d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5d7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5d7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d64c5dcd10 con 0x55d64c5cf820 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d64c4bcb60 con 0x55d64c5cef00 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.034+0000 7f72003cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d64c5dce90 con 0x55d64c5da0e0 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f72003cd740 10 client.0 _ll_register_callbacks cb 0x55d64c4e12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cef00 0x55d64c5d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5d7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cef00 0x55d64c5d6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:43712/0 (socket says 172.21.15.90:43712) 2024-04-26T18:15:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 -- 172.21.15.90:0/1854514397 learned_addr learned my addr 172.21.15.90:0/1854514397 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5d7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 -- 172.21.15.90:0/1854514397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 msgr2=0x55d64c5d7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5d7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 -- 172.21.15.90:0/1854514397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 msgr2=0x55d64c5d7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5d7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 -- 172.21.15.90:0/1854514397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64c247210 con 0x55d64c5cef00 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5d7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71effff640 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cef00 0x55d64c5d6fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71e0031440 tx=0x7f71e0040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e91bcbb79b4b8aca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71de7fc640 1 -- 172.21.15.90:0/1854514397 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0035040 con 0x55d64c5cef00 2024-04-26T18:15:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e80054d0 con 0x55d64c5cef00 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71de7fc640 1 -- 172.21.15.90:0/1854514397 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f71e0004260 con 0x55d64c5cef00 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f71de7fc640 1 -- 172.21.15.90:0/1854514397 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71e0004460 con 0x55d64c5cef00 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f72003cd740 10 client.25226 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.035+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d64c313620 con 0x55d64c5cef00 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f71de7fc640 1 -- 172.21.15.90:0/1854514397 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e80054d0 con 0x55d64c5cef00 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f71de7fc640 1 -- 172.21.15.90:0/1854514397 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f71e0004a50 con 0x55d64c5cef00 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f72003cd740 10 client.25226 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f72003cd740 10 client.25226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f72003cd740 20 client.25226 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f72003cd740 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c5ea180 0x55d64c5ec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f72003cd740 4 client.25226 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d64c543070 con 0x55d64c5ea180 2024-04-26T18:15:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c5ea180 0x55d64c5ec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c5ea180 0x55d64c5ec590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5889063335c986cd server_cookie=1650ca154ac3c342 in_seq=0 out_seq=0 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.036+0000 7f71de7fc640 10 client.25226 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f71de7fc640 1 -- 172.21.15.90:0/1854514397 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d64c543070 con 0x55d64c5ea180 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f71de7fc640 10 client.25226 handle_command_reply: tid=0 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 client.25226 shutdown 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cef00 msgr2=0x55d64c5d6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cef00 0x55d64c5d6fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71e0031440 tx=0x7f71e0040040 comp rx=0 tx=0).stop 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 shutdown_connections 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c5ea180 0x55d64c5ec590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5d7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f71de7fc640 0 client.25226 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5d7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 --2- 172.21.15.90:0/1854514397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cef00 0x55d64c5d6fe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 >> 172.21.15.90:0/1854514397 conn(0x55d64c5cb410 msgr2=0x55d64c5d1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 shutdown_connections 2024-04-26T18:15:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 1 -- 172.21.15.90:0/1854514397 wait complete. 2024-04-26T18:15:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.037+0000 7f72003cd740 20 client.25226 trim_cache size 0 max 16384 2024-04-26T18:15:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 Processor -- start 2024-04-26T18:15:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 -- start start 2024-04-26T18:15:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cef00 0x55d64c5e5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81b0bf0 con 0x55d64c5cf820 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81ae4c0 con 0x55d64c5da0e0 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.052+0000 7f72003cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81ac310 con 0x55d64c5cef00 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39106/0 (socket says 172.21.15.90:39106) 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 -- 172.21.15.90:0/319662000 learned_addr learned my addr 172.21.15.90:0/319662000 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71effff640 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cef00 0x55d64c5e5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71effff640 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 -- 172.21.15.90:0/319662000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cef00 msgr2=0x55d64c5e5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cef00 0x55d64c5e5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 -- 172.21.15.90:0/319662000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 msgr2=0x55d64c5f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5f1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 -- 172.21.15.90:0/319662000 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d64c5cbd30 con 0x55d64c5cf820 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71effff640 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5cb7f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55d64c5d04a0 tx=0x7f71e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2206ccab5253c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71ecff9640 1 -- 172.21.15.90:0/319662000 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8009db0 con 0x55d64c5cf820 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71ecff9640 1 -- 172.21.15.90:0/319662000 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e8034070 con 0x55d64c5cf820 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.053+0000 7f71ecff9640 1 -- 172.21.15.90:0/319662000 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8003850 con 0x55d64c5cf820 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 -- 172.21.15.90:0/319662000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 msgr2=0x55d64c5cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5cb7f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x55d64c5d04a0 tx=0x7f71e8005f90 comp rx=0 tx=0).stop 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 -- 172.21.15.90:0/319662000 shutdown_connections 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5da0e0 0x55d64c5f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cf820 0x55d64c5cb7f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 --2- 172.21.15.90:0/319662000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5cef00 0x55d64c5e5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 -- 172.21.15.90:0/319662000 >> 172.21.15.90:0/319662000 conn(0x55d64c5cb410 msgr2=0x55d64c5d6870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 -- 172.21.15.90:0/319662000 shutdown_connections 2024-04-26T18:15:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 -- 172.21.15.90:0/319662000 wait complete. 2024-04-26T18:15:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 -- start start 2024-04-26T18:15:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.054+0000 7f72003cd740 1 Processor -- start 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f72003cd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5e2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f72003cd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cf820 0x55d64c5dd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f72003cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5ddce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f72003cd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81ac980 con 0x55d64c5cef00 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f72003cd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81addf0 con 0x55d64c5cf820 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f72003cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71f81ad050 con 0x55d64c5da0e0 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f72003cd740 10 client.0 _ll_register_callbacks cb 0x7f71e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71effff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5e2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cf820 0x55d64c5dd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cf820 0x55d64c5dd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:43734/0 (socket says 172.21.15.90:43734) 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71effff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5e2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39114/0 (socket says 172.21.15.90:39114) 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 -- 172.21.15.90:0/2130943887 learned_addr learned my addr 172.21.15.90:0/2130943887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71effff640 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5ddce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 -- 172.21.15.90:0/2130943887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 msgr2=0x55d64c5ddce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5ddce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 -- 172.21.15.90:0/2130943887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 msgr2=0x55d64c5e2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5e2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71f6ffd640 1 -- 172.21.15.90:0/2130943887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d64c5de4e0 con 0x55d64c5cf820 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.055+0000 7f71effff640 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f71f6ffd640 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cf820 0x55d64c5dd7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009350 tx=0x7f71e8004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a693624191e4a37d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f71de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f71de7fc640 1 -- 172.21.15.90:0/2130943887 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e803c020 con 0x55d64c5cf820 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e8034070 con 0x55d64c5cf820 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f71de7fc640 1 -- 172.21.15.90:0/2130943887 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f71e802e2e0 con 0x55d64c5cf820 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f71de7fc640 1 -- 172.21.15.90:0/2130943887 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f71e8009e70 con 0x55d64c5cf820 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 10 client.25232 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d64c6506b0 con 0x55d64c5cf820 2024-04-26T18:15:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f71de7fc640 1 -- 172.21.15.90:0/2130943887 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71e8034070 con 0x55d64c5cf820 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f71de7fc640 1 -- 172.21.15.90:0/2130943887 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f71e8003b60 con 0x55d64c5cf820 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 10 client.25232 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 10 client.25232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 20 client.25232 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c650c40 0x55d64c653060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 4 client.25232 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.056+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d64c53cee0 con 0x55d64c650c40 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.057+0000 7f71effff640 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c650c40 0x55d64c653060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.057+0000 7f71effff640 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c650c40 0x55d64c653060 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=784e643ce3f5f448 server_cookie=b8d8b1d451a5efb9 in_seq=0 out_seq=0 2024-04-26T18:15:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.057+0000 7f71de7fc640 10 client.25232 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f71de7fc640 1 -- 172.21.15.90:0/2130943887 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d64c53cee0 con 0x55d64c650c40 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f71de7fc640 10 client.25232 handle_command_reply: tid=0 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 client.25232 shutdown 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cf820 msgr2=0x55d64c5dd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cf820 0x55d64c5dd7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009350 tx=0x7f71e8004450 comp rx=0 tx=0).stop 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 shutdown_connections 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d64c650c40 0x55d64c653060 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d64c5da0e0 0x55d64c5ddce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f71de7fc640 0 client.25232 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d64c5cf820 0x55d64c5dd7a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 --2- 172.21.15.90:0/2130943887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d64c5cef00 0x55d64c5e2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 >> 172.21.15.90:0/2130943887 conn(0x55d64c5cb410 msgr2=0x55d64c5c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 shutdown_connections 2024-04-26T18:15:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 1 -- 172.21.15.90:0/2130943887 wait complete. 2024-04-26T18:15:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.127+0000 7f72003cd740 20 client.25232 trim_cache size 0 max 16384 2024-04-26T18:15:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:15:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:15:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:15:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f71e40a3dc0 msgr2=0x7f71e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f71e40a3dc0 0x7f71e40a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f71f00ce5c0 tx=0x7f71f006f850 comp rx=0 tx=0).stop 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8107820 msgr2=0x7f71f81a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8107820 0x7f71f81a4f80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f71f000eb10 tx=0x7f71f000c2d0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 shutdown_connections 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f71e40a3dc0 0x7f71e40a6280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f71f8107820 0x7f71f81a4f80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f71f8102220 0x7f71f819a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 --2- 172.21.15.90:0/2410859351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8101860 0x7f71f8199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 >> 172.21.15.90:0/2410859351 conn(0x7f71f80789b0 msgr2=0x7f71f81043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 shutdown_connections 2024-04-26T18:15:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:30.128+0000 7f71fd5d4640 1 -- 172.21.15.90:0/2410859351 wait complete. 2024-04-26T18:15:30.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 149360 10.57 MB/sec warmup 78 sec latency 85.489 ms 2024-04-26T18:15:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:30 smithi119 ceph-mon[32752]: pgmap v109: 97 pgs: 97 active+clean; 58 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-04-26T18:15:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:30 smithi167 ceph-mon[32751]: pgmap v109: 97 pgs: 97 active+clean; 58 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-04-26T18:15:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:30 smithi090 ceph-mon[29180]: pgmap v109: 97 pgs: 97 active+clean; 58 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 435 op/s 2024-04-26T18:15:31.660 INFO:tasks.workunit.client.0.smithi090.stdout: 1 151455 10.52 MB/sec warmup 79 sec latency 52.140 ms 2024-04-26T18:15:32.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 153331 10.55 MB/sec warmup 80 sec latency 11.495 ms 2024-04-26T18:15:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:32 smithi119 ceph-mon[32752]: pgmap v110: 97 pgs: 97 active+clean; 58 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-04-26T18:15:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:32 smithi167 ceph-mon[32751]: pgmap v110: 97 pgs: 97 active+clean; 58 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-04-26T18:15:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:32 smithi090 ceph-mon[29180]: pgmap v110: 97 pgs: 97 active+clean; 58 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 259 op/s 2024-04-26T18:15:33.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 156294 10.64 MB/sec warmup 81 sec latency 15.330 ms 2024-04-26T18:15:34.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 157785 10.58 MB/sec warmup 82 sec latency 15.832 ms 2024-04-26T18:15:35.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:34 smithi119 ceph-mon[32752]: pgmap v111: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-04-26T18:15:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:34 smithi167 ceph-mon[32751]: pgmap v111: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-04-26T18:15:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:34 smithi090 ceph-mon[29180]: pgmap v111: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.2 MiB/s wr, 442 op/s 2024-04-26T18:15:35.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 160147 10.62 MB/sec warmup 83 sec latency 9.777 ms 2024-04-26T18:15:36.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 162570 10.64 MB/sec warmup 84 sec latency 30.701 ms 2024-04-26T18:15:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:36 smithi119 ceph-mon[32752]: pgmap v112: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-04-26T18:15:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:36 smithi167 ceph-mon[32751]: pgmap v112: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-04-26T18:15:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:36 smithi090 ceph-mon[29180]: pgmap v112: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-04-26T18:15:37.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 164098 10.60 MB/sec warmup 85 sec latency 23.053 ms 2024-04-26T18:15:38.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 166987 10.69 MB/sec warmup 86 sec latency 11.964 ms 2024-04-26T18:15:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:38 smithi119 ceph-mon[32752]: pgmap v113: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-04-26T18:15:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:38 smithi167 ceph-mon[32751]: pgmap v113: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-04-26T18:15:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:38 smithi090 ceph-mon[29180]: pgmap v113: 97 pgs: 97 active+clean; 58 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 312 op/s 2024-04-26T18:15:39.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 168874 10.64 MB/sec warmup 87 sec latency 13.618 ms 2024-04-26T18:15:40.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 170892 10.67 MB/sec warmup 88 sec latency 10.377 ms 2024-04-26T18:15:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:40 smithi119 ceph-mon[32752]: pgmap v114: 97 pgs: 97 active+clean; 63 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-04-26T18:15:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:40 smithi167 ceph-mon[32751]: pgmap v114: 97 pgs: 97 active+clean; 63 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-04-26T18:15:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:40 smithi090 ceph-mon[29180]: pgmap v114: 97 pgs: 97 active+clean; 63 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 439 op/s 2024-04-26T18:15:41.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 173623 10.71 MB/sec warmup 89 sec latency 31.598 ms 2024-04-26T18:15:42.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 175073 10.67 MB/sec warmup 90 sec latency 22.179 ms 2024-04-26T18:15:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:42 smithi119 ceph-mon[32752]: pgmap v115: 97 pgs: 97 active+clean; 63 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-04-26T18:15:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:42 smithi167 ceph-mon[32751]: pgmap v115: 97 pgs: 97 active+clean; 63 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-04-26T18:15:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:42 smithi090 ceph-mon[29180]: pgmap v115: 97 pgs: 97 active+clean; 63 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-04-26T18:15:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 -- 172.21.15.90:0/3843151475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 msgr2=0x7ff380107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 --2- 172.21.15.90:0/3843151475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff380107c00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff370009930 tx=0x7ff37002a420 comp rx=0 tx=0).stop 2024-04-26T18:15:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 -- 172.21.15.90:0/3843151475 shutdown_connections 2024-04-26T18:15:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 --2- 172.21.15.90:0/3843151475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff380102220 0x7ff38010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 --2- 172.21.15.90:0/3843151475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff380101860 0x7ff380101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 --2- 172.21.15.90:0/3843151475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff380107c00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 -- 172.21.15.90:0/3843151475 >> 172.21.15.90:0/3843151475 conn(0x7ff3800789b0 msgr2=0x7ff380078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 -- 172.21.15.90:0/3843151475 shutdown_connections 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 -- 172.21.15.90:0/3843151475 wait complete. 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 Processor -- start 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.219+0000 7ff38650a640 1 -- start start 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff38650a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff380101860 0x7ff3801990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff38650a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff380102220 0x7ff3801995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff38650a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff38019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff38650a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111ae0 con 0x7ff380107820 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff38650a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111960 con 0x7ff380102220 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff38650a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff380111c60 con 0x7ff380101860 2024-04-26T18:15:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff38019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff38019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51550/0 (socket says 172.21.15.90:51550) 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 -- 172.21.15.90:0/2352890255 learned_addr learned my addr 172.21.15.90:0/2352890255 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff37ffff640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff380101860 0x7ff3801990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff380102220 0x7ff3801995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 -- 172.21.15.90:0/2352890255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff380101860 msgr2=0x7ff3801990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff380101860 0x7ff3801990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 -- 172.21.15.90:0/2352890255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff380102220 msgr2=0x7ff3801995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff380102220 0x7ff3801995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 -- 172.21.15.90:0/2352890255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff38019e0a0 con 0x7ff380107820 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.220+0000 7ff384aa7640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff38019d970 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff37800eb10 tx=0x7ff37800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.221+0000 7ff37d7fa640 1 -- 172.21.15.90:0/2352890255 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378004b20 con 0x7ff380107820 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.221+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff38019e390 con 0x7ff380107820 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.221+0000 7ff37d7fa640 1 -- 172.21.15.90:0/2352890255 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff378004cc0 con 0x7ff380107820 2024-04-26T18:15:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.221+0000 7ff37d7fa640 1 -- 172.21.15.90:0/2352890255 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff378010670 con 0x7ff380107820 2024-04-26T18:15:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.221+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3801a5c20 con 0x7ff380107820 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.223+0000 7ff37d7fa640 1 -- 172.21.15.90:0/2352890255 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7ff378005020 con 0x7ff380107820 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.223+0000 7ff389225740 1 Processor -- start 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.223+0000 7ff389225740 1 -- start start 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff389225740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5bf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff389225740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff389225740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff389225740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b4bf525460 con 0x55b4bf5bee20 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff389225740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b4bf4df590 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff389225740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b4bf521420 con 0x55b4bf5bf740 2024-04-26T18:15:43.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37d7fa640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff3680a3dc0 0x7ff3680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff384aa7640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff3680a3dc0 0x7ff3680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5bf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5bf200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51552/0 (socket says 172.21.15.90:51552) 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37f7fe640 1 -- 172.21.15.90:0/3710183031 learned_addr learned my addr 172.21.15.90:0/3710183031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37d7fa640 1 -- 172.21.15.90:0/2352890255 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff3780d0c70 con 0x7ff380107820 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37ffff640 1 -- 172.21.15.90:0/3710183031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5ce4f0 msgr2=0x55b4bf5d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37ffff640 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37ffff640 1 -- 172.21.15.90:0/3710183031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 msgr2=0x55b4bf5bf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37ffff640 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5bf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37ffff640 1 -- 172.21.15.90:0/3710183031 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4bf4ec0d0 con 0x55b4bf5bf740 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff384aa7640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff3680a3dc0 0x7ff3680a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ce4e0 tx=0x7ff37806f770 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:43.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.224+0000 7ff37ffff640 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cdfb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff370031440 tx=0x7ff370005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a51ac00c6ab0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.225+0000 7ff3667fc640 1 -- 172.21.15.90:0/3710183031 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370040070 con 0x55b4bf5bf740 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.225+0000 7ff3667fc640 1 -- 172.21.15.90:0/3710183031 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff370004030 con 0x55b4bf5bf740 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.225+0000 7ff3667fc640 1 -- 172.21.15.90:0/3710183031 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370035040 con 0x55b4bf5bf740 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.225+0000 7ff389225740 1 -- 172.21.15.90:0/3710183031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bf740 msgr2=0x55b4bf5cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.225+0000 7ff389225740 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cdfb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff370031440 tx=0x7ff370005be0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- 172.21.15.90:0/3710183031 shutdown_connections 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5d08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cdfb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 --2- 172.21.15.90:0/3710183031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5bf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- 172.21.15.90:0/3710183031 >> 172.21.15.90:0/3710183031 conn(0x55b4bf5bb410 msgr2=0x55b4bf5bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- 172.21.15.90:0/3710183031 shutdown_connections 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- 172.21.15.90:0/3710183031 wait complete. 2024-04-26T18:15:43.232 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- start start 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 Processor -- start 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bee20 0x55b4bf5cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b4bf5d10c0 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b4bf4acb60 con 0x55b4bf5bf740 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b4bf5d1240 con 0x55b4bf5bee20 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff389225740 10 client.0 _ll_register_callbacks cb 0x55b4bf4d12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.226+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51554/0 (socket says 172.21.15.90:51554) 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 -- 172.21.15.90:0/3213617746 learned_addr learned my addr 172.21.15.90:0/3213617746 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:43.233 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bee20 0x55b4bf5cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 -- 172.21.15.90:0/3213617746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bee20 msgr2=0x55b4bf5cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bee20 0x55b4bf5cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 -- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 msgr2=0x55b4bf5cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 -- 172.21.15.90:0/3213617746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4bf237210 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37ffff640 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5cbeb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff37400f460 tx=0x7ff374004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa0e70c3ac07da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff35f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff35f7fe640 1 -- 172.21.15.90:0/3213617746 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004250 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff35f7fe640 1 -- 172.21.15.90:0/3213617746 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff37401a040 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff370004030 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff35f7fe640 1 -- 172.21.15.90:0/3213617746 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff3740043f0 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff389225740 10 client.15705 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4bf303620 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.227+0000 7ff35f7fe640 1 -- 172.21.15.90:0/3213617746 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff370004030 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff35f7fe640 1 -- 172.21.15.90:0/3213617746 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff374029030 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff389225740 10 client.15705 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff389225740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff389225740 20 client.15705 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff389225740 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf5e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff389225740 4 client.15705 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4bf533070 con 0x55b4bf5de600 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff37ffff640 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf5e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff37ffff640 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf5e0a80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f76c4d9e710a5b3f server_cookie=e49b7b4ba50d02f4 in_seq=0 out_seq=0 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.228+0000 7ff35f7fe640 10 client.15705 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff35f7fe640 1 -- 172.21.15.90:0/3213617746 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b4bf533070 con 0x55b4bf5de600 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff35f7fe640 10 client.15705 handle_command_reply: tid=0 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 client.15705 shutdown 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5ce4f0 msgr2=0x55b4bf5cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5cbeb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff37400f460 tx=0x7ff374004730 comp rx=0 tx=0).stop 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 shutdown_connections 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf5e0a80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5cbeb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff35f7fe640 0 client.15705 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 --2- 172.21.15.90:0/3213617746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5bee20 0x55b4bf5cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.229+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 >> 172.21.15.90:0/3213617746 conn(0x55b4bf5bb410 msgr2=0x55b4bf5c1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.230+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 shutdown_connections 2024-04-26T18:15:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.230+0000 7ff389225740 1 -- 172.21.15.90:0/3213617746 wait complete. 2024-04-26T18:15:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.230+0000 7ff389225740 20 client.15705 trim_cache size 0 max 16384 2024-04-26T18:15:43.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.245+0000 7ff389225740 1 Processor -- start 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.245+0000 7ff389225740 1 -- start start 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff389225740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff389225740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff389225740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5e6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff389225740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801aead0 con 0x55b4bf5bee20 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff389225740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801ac3a0 con 0x55b4bf5bf740 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff389225740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801aa1f0 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51560/0 (socket says 172.21.15.90:51560) 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 -- 172.21.15.90:0/1813280478 learned_addr learned my addr 172.21.15.90:0/1813280478 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37ffff640 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5e6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 -- 172.21.15.90:0/1813280478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 msgr2=0x55b4bf5e6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5e6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 -- 172.21.15.90:0/1813280478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 msgr2=0x55b4bf5cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 -- 172.21.15.90:0/1813280478 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4bf5bb240 con 0x55b4bf5bee20 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d9f80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff374015040 tx=0x7ff374012db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600ca2c8baf55239 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff365ffb640 1 -- 172.21.15.90:0/1813280478 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004250 con 0x55b4bf5bee20 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff365ffb640 1 -- 172.21.15.90:0/1813280478 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff374005d30 con 0x55b4bf5bee20 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.246+0000 7ff365ffb640 1 -- 172.21.15.90:0/1813280478 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff374004590 con 0x55b4bf5bee20 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 -- 172.21.15.90:0/1813280478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 msgr2=0x55b4bf5d9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d9f80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff374015040 tx=0x7ff374012db0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 -- 172.21.15.90:0/1813280478 shutdown_connections 2024-04-26T18:15:43.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5e6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 --2- 172.21.15.90:0/1813280478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d9f80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 -- 172.21.15.90:0/1813280478 >> 172.21.15.90:0/1813280478 conn(0x55b4bf5bb410 msgr2=0x55b4bf5cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 -- 172.21.15.90:0/1813280478 shutdown_connections 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.247+0000 7ff389225740 1 -- 172.21.15.90:0/1813280478 wait complete. 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 -- start start 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 Processor -- start 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801aa860 con 0x55b4bf5bee20 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801abcd0 con 0x55b4bf5bf740 2024-04-26T18:15:43.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff3801aaf30 con 0x55b4bf5ce4f0 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff389225740 10 client.0 _ll_register_callbacks cb 0x7ff3740126a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51570/0 (socket says 172.21.15.90:51570) 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 -- 172.21.15.90:0/403956768 learned_addr learned my addr 172.21.15.90:0/403956768 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37ffff640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 -- 172.21.15.90:0/403956768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 msgr2=0x55b4bf5c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 -- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 msgr2=0x55b4bf5c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37f7fe640 1 -- 172.21.15.90:0/403956768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4bf5c8720 con 0x55b4bf5bee20 2024-04-26T18:15:43.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.248+0000 7ff37ffff640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff37f7fe640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d6680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff37400f460 tx=0x7ff374005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6649f4f469bb9d58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff35f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff35f7fe640 1 -- 172.21.15.90:0/403956768 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37403f060 con 0x55b4bf5bee20 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff35f7fe640 1 -- 172.21.15.90:0/403956768 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7ff374009ba0 con 0x55b4bf5bee20 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff374005d30 con 0x55b4bf5bee20 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff35f7fe640 1 -- 172.21.15.90:0/403956768 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff374012880 con 0x55b4bf5bee20 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b4bf6448e0 con 0x55b4bf5bee20 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff35f7fe640 1 -- 172.21.15.90:0/403956768 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff374005d30 con 0x55b4bf5bee20 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff35f7fe640 1 -- 172.21.15.90:0/403956768 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff374042030 con 0x55b4bf5bee20 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:43.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 20 client.15711 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf646f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 4 client.15711 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:15:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b4bf52cee0 con 0x55b4bf5de600 2024-04-26T18:15:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.249+0000 7ff37ffff640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf646f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.250+0000 7ff35f7fe640 10 client.15711 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:43.256 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.250+0000 7ff37ffff640 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf646f30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6dd96810014f12d server_cookie=faed314968ccaf45 in_seq=0 out_seq=0 2024-04-26T18:15:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.316+0000 7ff35f7fe640 1 -- 172.21.15.90:0/403956768 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b4bf52cee0 con 0x55b4bf5de600 2024-04-26T18:15:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.316+0000 7ff35f7fe640 10 client.15711 handle_command_reply: tid=0 2024-04-26T18:15:43.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.316+0000 7ff389225740 1 client.15711 shutdown 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 msgr2=0x55b4bf5d6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d6680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff37400f460 tx=0x7ff374005c50 comp rx=0 tx=0).stop 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 shutdown_connections 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b4bf5de600 0x55b4bf646f30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b4bf5ce4f0 0x55b4bf5c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b4bf5bf740 0x55b4bf5c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff35f7fe640 0 client.15711 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 --2- 172.21.15.90:0/403956768 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b4bf5bee20 0x55b4bf5d6680 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 >> 172.21.15.90:0/403956768 conn(0x55b4bf5bb410 msgr2=0x55b4bf5b9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 shutdown_connections 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 1 -- 172.21.15.90:0/403956768 wait complete. 2024-04-26T18:15:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.317+0000 7ff389225740 20 client.15711 trim_cache size 0 max 16384 2024-04-26T18:15:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:15:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:15:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:15:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff3680a3dc0 msgr2=0x7ff3680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff3680a3dc0 0x7ff3680a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff3780ce4e0 tx=0x7ff37806f770 comp rx=0 tx=0).stop 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 msgr2=0x7ff38019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff38019d970 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff37800eb10 tx=0x7ff37800c2d0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 shutdown_connections 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff3680a3dc0 0x7ff3680a6280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff380107820 0x7ff38019d970 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff380102220 0x7ff3801995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 --2- 172.21.15.90:0/2352890255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff380101860 0x7ff3801990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 >> 172.21.15.90:0/2352890255 conn(0x7ff3800789b0 msgr2=0x7ff38006a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 shutdown_connections 2024-04-26T18:15:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:43.318+0000 7ff38650a640 1 -- 172.21.15.90:0/2352890255 wait complete. 2024-04-26T18:15:43.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 177667 10.73 MB/sec warmup 91 sec latency 43.856 ms 2024-04-26T18:15:44.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 179768 10.70 MB/sec warmup 92 sec latency 17.772 ms 2024-04-26T18:15:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:44 smithi119 ceph-mon[32752]: pgmap v116: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-04-26T18:15:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:44 smithi167 ceph-mon[32751]: pgmap v116: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-04-26T18:15:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:44 smithi090 ceph-mon[29180]: pgmap v116: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 442 op/s 2024-04-26T18:15:45.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 181532 10.71 MB/sec warmup 93 sec latency 10.587 ms 2024-04-26T18:15:46.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 184282 10.75 MB/sec warmup 94 sec latency 24.756 ms 2024-04-26T18:15:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:46 smithi119 ceph-mon[32752]: pgmap v117: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-26T18:15:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:46 smithi167 ceph-mon[32751]: pgmap v117: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-26T18:15:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:46 smithi090 ceph-mon[29180]: pgmap v117: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-26T18:15:47.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 185611 10.71 MB/sec warmup 95 sec latency 20.770 ms 2024-04-26T18:15:48.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 188316 10.77 MB/sec warmup 96 sec latency 15.563 ms 2024-04-26T18:15:49.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:48 smithi119 ceph-mon[32752]: pgmap v118: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-26T18:15:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:48 smithi167 ceph-mon[32751]: pgmap v118: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-26T18:15:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:48 smithi090 ceph-mon[29180]: pgmap v118: 97 pgs: 97 active+clean; 67 MiB data, 881 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.8 MiB/s wr, 258 op/s 2024-04-26T18:15:49.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 190366 10.74 MB/sec warmup 97 sec latency 14.710 ms 2024-04-26T18:15:50.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 192142 10.75 MB/sec warmup 98 sec latency 16.356 ms 2024-04-26T18:15:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:50 smithi119 ceph-mon[32752]: pgmap v119: 97 pgs: 97 active+clean; 64 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-04-26T18:15:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:50 smithi167 ceph-mon[32751]: pgmap v119: 97 pgs: 97 active+clean; 64 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-04-26T18:15:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:50 smithi090 ceph-mon[29180]: pgmap v119: 97 pgs: 97 active+clean; 64 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.1 MiB/s wr, 434 op/s 2024-04-26T18:15:51.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 194742 10.78 MB/sec warmup 99 sec latency 57.353 ms 2024-04-26T18:15:52.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 196146 10.75 MB/sec warmup 100 sec latency 16.374 ms 2024-04-26T18:15:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:52 smithi119 ceph-mon[32752]: pgmap v120: 97 pgs: 97 active+clean; 64 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-04-26T18:15:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:52 smithi167 ceph-mon[32751]: pgmap v120: 97 pgs: 97 active+clean; 64 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-04-26T18:15:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:52 smithi090 ceph-mon[29180]: pgmap v120: 97 pgs: 97 active+clean; 64 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-04-26T18:15:53.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 198933 10.81 MB/sec warmup 101 sec latency 13.834 ms 2024-04-26T18:15:54.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 201009 10.77 MB/sec warmup 102 sec latency 13.527 ms 2024-04-26T18:15:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:54 smithi119 ceph-mon[32752]: pgmap v121: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-04-26T18:15:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:54 smithi167 ceph-mon[32751]: pgmap v121: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-04-26T18:15:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:54 smithi090 ceph-mon[29180]: pgmap v121: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 436 op/s 2024-04-26T18:15:55.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 202852 10.79 MB/sec warmup 103 sec latency 12.773 ms 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2641110451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8101860 msgr2=0x7fbab8101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2641110451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8101860 0x7fbab8101ce0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009930 tx=0x7fbaac02a420 comp rx=0 tx=0).stop 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2641110451 shutdown_connections 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2641110451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbab8102220 0x7fbab810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2641110451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8101860 0x7fbab8101ce0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2641110451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab8107820 0x7fbab8107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2641110451 >> 172.21.15.90:0/2641110451 conn(0x7fbab80789b0 msgr2=0x7fbab8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:56.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2641110451 shutdown_connections 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.410+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2641110451 wait complete. 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 Processor -- start 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 -- start start 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab8101860 0x7fbab8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8102220 0x7fbab81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbab8107820 0x7fbab819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8111ae0 con 0x7fbab8102220 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8111960 con 0x7fbab8107820 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbabe9e9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8111c60 con 0x7fbab8101860 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8102220 0x7fbab81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbab7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab8101860 0x7fbab8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8102220 0x7fbab81996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35804/0 (socket says 172.21.15.90:35804) 2024-04-26T18:15:56.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.411+0000 7fbab77fe640 1 -- 172.21.15.90:0/2621942255 learned_addr learned my addr 172.21.15.90:0/2621942255 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbab8107820 0x7fbab819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab77fe640 1 -- 172.21.15.90:0/2621942255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab8101860 msgr2=0x7fbab8199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab77fe640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab8101860 0x7fbab8199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab77fe640 1 -- 172.21.15.90:0/2621942255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbab8107820 msgr2=0x7fbab819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab77fe640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbab8107820 0x7fbab819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab77fe640 1 -- 172.21.15.90:0/2621942255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab819e130 con 0x7fbab8102220 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab7fff640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab8101860 0x7fbab8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbab8107820 0x7fbab819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:56.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab77fe640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8102220 0x7fbab81996d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005940 tx=0x7fbaac0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab57fa640 1 -- 172.21.15.90:0/2621942255 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac03f030 con 0x7fbab8102220 2024-04-26T18:15:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab57fa640 1 -- 172.21.15.90:0/2621942255 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaac0044c0 con 0x7fbab8102220 2024-04-26T18:15:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab819e3c0 con 0x7fbab8102220 2024-04-26T18:15:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbab57fa640 1 -- 172.21.15.90:0/2621942255 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac035070 con 0x7fbab8102220 2024-04-26T18:15:56.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.412+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81a5cb0 con 0x7fbab8102220 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.414+0000 7fbab57fa640 1 -- 172.21.15.90:0/2621942255 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fbaac042030 con 0x7fbab8102220 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 Processor -- start 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 -- start start 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623403b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56234030f460 con 0x5623403a9820 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5623402c9590 con 0x5623403b40e0 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbac17dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56234030b420 con 0x5623403a8f00 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbab57fa640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbaa00a3dc0 0x7fbaa00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbab7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbabcf86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623403b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbabcf86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403a92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:53694/0 (socket says 172.21.15.90:53694) 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbabcf86640 1 -- 172.21.15.90:0/2640041091 learned_addr learned my addr 172.21.15.90:0/2640041091 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbaa00a3dc0 0x7fbaa00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbab57fa640 1 -- 172.21.15.90:0/2621942255 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fbaac0f0ac0 con 0x7fbab8102220 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.415+0000 7fbabcf86640 1 -- 172.21.15.90:0/2640041091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 msgr2=0x5623403b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623403b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fbabcf86640 1 -- 172.21.15.90:0/2640041091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 msgr2=0x5623403b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fbabcf86640 1 -- 172.21.15.90:0/2640041091 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623402d60d0 con 0x5623403a8f00 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fbab7fff640 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:56.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403a92e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbab000f5c0 tx=0x7fbab000fa80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2bb80d25d225cff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbaa00a3dc0 0x7fbaa00a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbab00099e0 tx=0x7fbab001e000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fba9e7fc640 1 -- 172.21.15.90:0/2640041091 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab002b070 con 0x5623403a8f00 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fba9e7fc640 1 -- 172.21.15.90:0/2640041091 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab0011630 con 0x5623403a8f00 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.416+0000 7fba9e7fc640 1 -- 172.21.15.90:0/2640041091 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0022070 con 0x5623403a8f00 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 -- 172.21.15.90:0/2640041091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 msgr2=0x5623403a92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403a92e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbab000f5c0 tx=0x7fbab000fa80 comp rx=0 tx=0).stop 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 -- 172.21.15.90:0/2640041091 shutdown_connections 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623403b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2640041091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403a92e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 -- 172.21.15.90:0/2640041091 >> 172.21.15.90:0/2640041091 conn(0x5623403a5410 msgr2=0x5623403a57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 -- 172.21.15.90:0/2640041091 shutdown_connections 2024-04-26T18:15:56.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 -- 172.21.15.90:0/2640041091 wait complete. 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 -- start start 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.417+0000 7fbac17dd740 1 Processor -- start 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5623403b6cb0 con 0x5623403a9820 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562340296b60 con 0x5623403a8f00 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5623403b6e30 con 0x5623403b40e0 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 10 client.0 _ll_register_callbacks cb 0x5623402bb2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbabcf86640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35822/0 (socket says 172.21.15.90:35822) 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:33356/0 (socket says 172.21.15.90:33356) 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 -- 172.21.15.90:0/1421572753 learned_addr learned my addr 172.21.15.90:0/1421572753 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 -- 172.21.15.90:0/1421572753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 msgr2=0x5623403b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 -- 172.21.15.90:0/1421572753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 msgr2=0x5623403b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 -- 172.21.15.90:0/1421572753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562340021210 con 0x5623403a9820 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbabcf86640 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbab77fe640 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b1630 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0b8fe0 tx=0x7fbaac08e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5970a1027da047 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:56.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fba977fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fba977fe640 1 -- 172.21.15.90:0/1421572753 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac049050 con 0x5623403a9820 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fba977fe640 1 -- 172.21.15.90:0/1421572753 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fbaac0b91f0 con 0x5623403a9820 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbab0011630 con 0x5623403a9820 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fba977fe640 1 -- 172.21.15.90:0/1421572753 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbaac0c0e80 con 0x5623403a9820 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 10 client.15723 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.418+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623400ed620 con 0x5623403a9820 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fba977fe640 1 -- 172.21.15.90:0/1421572753 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab0011630 con 0x5623403a9820 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fba977fe640 1 -- 172.21.15.90:0/1421572753 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbaac08e0b0 con 0x5623403a9820 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbac17dd740 10 client.15723 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbac17dd740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbac17dd740 20 client.15723 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbac17dd740 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5623403c4220 0x5623403c6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbac17dd740 4 client.15723 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:15:56.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56234031d070 con 0x5623403c4220 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbab7fff640 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5623403c4220 0x5623403c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fbab7fff640 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5623403c4220 0x5623403c6640 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de8305a95dc8d43a server_cookie=25d4704124da5fe4 in_seq=0 out_seq=0 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.419+0000 7fba977fe640 10 client.15723 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.420+0000 7fba977fe640 1 -- 172.21.15.90:0/1421572753 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56234031d070 con 0x5623403c4220 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.420+0000 7fba977fe640 10 client.15723 handle_command_reply: tid=0 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.420+0000 7fbac17dd740 1 client.15723 shutdown 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 msgr2=0x5623403b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b1630 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0b8fe0 tx=0x7fbaac08e5c0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 shutdown_connections 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5623403c4220 0x5623403c6640 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.429 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b1630 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fba977fe640 0 client.15723 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 --2- 172.21.15.90:0/1421572753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 >> 172.21.15.90:0/1421572753 conn(0x5623403a5410 msgr2=0x5623403ab140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 shutdown_connections 2024-04-26T18:15:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 1 -- 172.21.15.90:0/1421572753 wait complete. 2024-04-26T18:15:56.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.421+0000 7fbac17dd740 20 client.15723 trim_cache size 0 max 16384 2024-04-26T18:15:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.436+0000 7fbac17dd740 1 Processor -- start 2024-04-26T18:15:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbac17dd740 1 -- start start 2024-04-26T18:15:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403cc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbac17dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81ae9f0 con 0x5623403a9820 2024-04-26T18:15:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbac17dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81ac2c0 con 0x5623403a8f00 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbac17dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81aa130 con 0x5623403b40e0 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35828/0 (socket says 172.21.15.90:35828) 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 -- 172.21.15.90:0/869115493 learned_addr learned my addr 172.21.15.90:0/869115493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab7fff640 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbabcf86640 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 -- 172.21.15.90:0/869115493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 msgr2=0x5623403cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 -- 172.21.15.90:0/869115493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 msgr2=0x5623403b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 -- 172.21.15.90:0/869115493 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623403c6cc0 con 0x5623403a9820 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbabcf86640 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:15:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fbab77fe640 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403cbf50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbaac031440 tx=0x7fbaac0bb090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4dbc7c21ec2ba5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fba9dffb640 1 -- 172.21.15.90:0/869115493 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac049070 con 0x5623403a9820 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.437+0000 7fba9dffb640 1 -- 172.21.15.90:0/869115493 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaac0c5740 con 0x5623403a9820 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fba9dffb640 1 -- 172.21.15.90:0/869115493 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac0b91f0 con 0x5623403a9820 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 -- 172.21.15.90:0/869115493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 msgr2=0x5623403cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403cbf50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fbaac031440 tx=0x7fbaac0bb090 comp rx=0 tx=0).stop 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 -- 172.21.15.90:0/869115493 shutdown_connections 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403b40e0 0x5623403cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403cbf50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 --2- 172.21.15.90:0/869115493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403a8f00 0x5623403b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 -- 172.21.15.90:0/869115493 >> 172.21.15.90:0/869115493 conn(0x5623403a5410 msgr2=0x5623403b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 -- 172.21.15.90:0/869115493 shutdown_connections 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.438+0000 7fbac17dd740 1 -- 172.21.15.90:0/869115493 wait complete. 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 -- start start 2024-04-26T18:15:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 Processor -- start 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403b8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623404297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81aa780 con 0x5623403a9820 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81abbf0 con 0x5623403b40e0 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81aae50 con 0x5623403a8f00 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbac17dd740 10 client.0 _ll_register_callbacks cb 0x7fbaac009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab7fff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35832/0 (socket says 172.21.15.90:35832) 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 -- 172.21.15.90:0/2770449235 learned_addr learned my addr 172.21.15.90:0/2770449235 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbabcf86640 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623404297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 -- 172.21.15.90:0/2770449235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 msgr2=0x5623403b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403b8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 -- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 msgr2=0x5623404297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623404297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.439+0000 7fbab77fe640 1 -- 172.21.15.90:0/2770449235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562340429db0 con 0x5623403a9820 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbab77fe640 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b87c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0b8fe0 tx=0x7fbaac0b8cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741ed4bc57c7b358 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fba977fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fba977fe640 1 -- 172.21.15.90:0/2770449235 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaac049070 con 0x5623403a9820 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fba977fe640 1 -- 172.21.15.90:0/2770449235 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fbaac0c52b0 con 0x5623403a9820 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbaac0c5740 con 0x5623403a9820 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fba977fe640 1 -- 172.21.15.90:0/2770449235 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbaac003cc0 con 0x5623403a9820 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 10 client.15729 fetch_fsmap learned FSMap version 27 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562340429db0 con 0x5623403a9820 2024-04-26T18:15:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fba977fe640 1 -- 172.21.15.90:0/2770449235 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaac0c5740 con 0x5623403a9820 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fba977fe640 1 -- 172.21.15.90:0/2770449235 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fbaac0bba40 con 0x5623403a9820 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 20 client.15729 populate_metadata read hostname 'smithi090' 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56234042b3c0 0x56234042d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 4 client.15729 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562340316ee0 con 0x56234042b3c0 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.440+0000 7fbab7fff640 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56234042b3c0 0x56234042d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.441+0000 7fbab7fff640 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56234042b3c0 0x56234042d7e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51fccdc010988c1 server_cookie=6ae9bcb30321daa7 in_seq=0 out_seq=0 2024-04-26T18:15:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.441+0000 7fba977fe640 10 client.15729 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fba977fe640 1 -- 172.21.15.90:0/2770449235 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562340316ee0 con 0x56234042b3c0 2024-04-26T18:15:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fba977fe640 10 client.15729 handle_command_reply: tid=0 2024-04-26T18:15:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 client.15729 shutdown 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 msgr2=0x5623403b87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b87c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0b8fe0 tx=0x7fbaac0b8cd0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 shutdown_connections 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56234042b3c0 0x56234042d7e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5623403b40e0 0x5623404297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fba977fe640 0 client.15729 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5623403a9820 0x5623403b87c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 --2- 172.21.15.90:0/2770449235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5623403a8f00 0x5623403b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.508+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 >> 172.21.15.90:0/2770449235 conn(0x5623403a5410 msgr2=0x5623403b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 shutdown_connections 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbac17dd740 1 -- 172.21.15.90:0/2770449235 wait complete. 2024-04-26T18:15:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbac17dd740 20 client.15729 trim_cache size 0 max 16384 2024-04-26T18:15:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:15:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:15:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:15:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbaa00a3dc0 msgr2=0x7fbaa00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbaa00a3dc0 0x7fbaa00a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbab00099e0 tx=0x7fbab001e000 comp rx=0 tx=0).stop 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8102220 msgr2=0x7fbab81996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8102220 0x7fbab81996d0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005940 tx=0x7fbaac0040c0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 shutdown_connections 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fbaa00a3dc0 0x7fbaa00a6280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbab8107820 0x7fbab819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbab8102220 0x7fbab81996d0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 --2- 172.21.15.90:0/2621942255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab8101860 0x7fbab8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:15:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 >> 172.21.15.90:0/2621942255 conn(0x7fbab80789b0 msgr2=0x7fbab806a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:15:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 shutdown_connections 2024-04-26T18:15:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:15:56.509+0000 7fbabe9e9640 1 -- 172.21.15.90:0/2621942255 wait complete. 2024-04-26T18:15:56.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 205208 10.80 MB/sec warmup 104 sec latency 65.332 ms 2024-04-26T18:15:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:56 smithi119 ceph-mon[32752]: pgmap v122: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-04-26T18:15:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:56 smithi167 ceph-mon[32751]: pgmap v122: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-04-26T18:15:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:56 smithi090 ceph-mon[29180]: pgmap v122: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-04-26T18:15:57.661 INFO:tasks.workunit.client.0.smithi090.stdout: 1 206643 10.77 MB/sec warmup 105 sec latency 17.652 ms 2024-04-26T18:15:58.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 209503 10.84 MB/sec warmup 106 sec latency 13.327 ms 2024-04-26T18:15:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:15:58 smithi119 ceph-mon[32752]: pgmap v123: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-04-26T18:15:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:15:58 smithi167 ceph-mon[32751]: pgmap v123: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-04-26T18:15:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:15:58 smithi090 ceph-mon[29180]: pgmap v123: 97 pgs: 97 active+clean; 68 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-04-26T18:15:59.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 211104 10.79 MB/sec warmup 107 sec latency 13.069 ms 2024-04-26T18:16:00.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 213366 10.82 MB/sec warmup 108 sec latency 24.714 ms 2024-04-26T18:16:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:00 smithi119 ceph-mon[32752]: pgmap v124: 97 pgs: 97 active+clean; 64 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 486 op/s 2024-04-26T18:16:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:00 smithi167 ceph-mon[32751]: pgmap v124: 97 pgs: 97 active+clean; 64 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 486 op/s 2024-04-26T18:16:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:00 smithi090 ceph-mon[29180]: pgmap v124: 97 pgs: 97 active+clean; 64 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.0 MiB/s wr, 486 op/s 2024-04-26T18:16:01.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 214830 10.77 MB/sec warmup 109 sec latency 55.390 ms 2024-04-26T18:16:02.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 216877 10.80 MB/sec warmup 110 sec latency 54.378 ms 2024-04-26T18:16:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:02 smithi119 ceph-mon[32752]: pgmap v125: 97 pgs: 97 active+clean; 64 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-04-26T18:16:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:02 smithi167 ceph-mon[32751]: pgmap v125: 97 pgs: 97 active+clean; 64 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-04-26T18:16:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:02 smithi090 ceph-mon[29180]: pgmap v125: 97 pgs: 97 active+clean; 64 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-04-26T18:16:03.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 219404 10.81 MB/sec warmup 111 sec latency 12.406 ms 2024-04-26T18:16:04.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 220945 10.78 MB/sec warmup 112 sec latency 12.195 ms 2024-04-26T18:16:04.856 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T18:16:04.856 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.5 2024-04-26T18:16:04.857 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:16:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:04 smithi119 ceph-mon[32752]: pgmap v126: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 464 op/s 2024-04-26T18:16:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:04 smithi167 ceph-mon[32751]: pgmap v126: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 464 op/s 2024-04-26T18:16:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:04 smithi090 ceph-mon[29180]: pgmap v126: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 464 op/s 2024-04-26T18:16:05.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 223770 10.84 MB/sec warmup 113 sec latency 10.695 ms 2024-04-26T18:16:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:06.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 225374 10.80 MB/sec warmup 114 sec latency 38.341 ms 2024-04-26T18:16:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 -- 172.21.15.90:0/342075161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d80103e30 msgr2=0x7f3d80110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 --2- 172.21.15.90:0/342075161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d80103e30 0x7f3d80110290 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800b0a0 tx=0x7f3d7800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:16:06.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 -- 172.21.15.90:0/342075161 shutdown_connections 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 --2- 172.21.15.90:0/342075161 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d80103e30 0x7f3d80110290 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 --2- 172.21.15.90:0/342075161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d801094c0 0x7f3d801098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 --2- 172.21.15.90:0/342075161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103470 0x7f3d801038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 -- 172.21.15.90:0/342075161 >> 172.21.15.90:0/342075161 conn(0x7f3d8007a580 msgr2=0x7f3d8007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 -- 172.21.15.90:0/342075161 shutdown_connections 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 -- 172.21.15.90:0/342075161 wait complete. 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.834+0000 7f3d85d64700 1 Processor -- start 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d85d64700 1 -- start start 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d85d64700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d80103470 0x7f3d80075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d85d64700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103e30 0x7f3d80075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:06.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d85d64700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d801094c0 0x7f3d800761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d85d64700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801135e0 con 0x7f3d801094c0 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d85d64700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80113460 con 0x7f3d80103e30 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d85d64700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80113760 con 0x7f3d80103470 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d801094c0 0x7f3d800761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d80103470 0x7f3d80075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103e30 0x7f3d80075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d801094c0 0x7f3d800761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56084/0 (socket says 172.21.15.90:56084) 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103e30 0x7f3d80075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:60642/0 (socket says 172.21.15.90:60642) 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7f7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d80103470 0x7f3d80075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:34958/0 (socket says 172.21.15.90:34958) 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7ffff700 1 -- 172.21.15.90:0/2289555541 learned_addr learned my addr 172.21.15.90:0/2289555541 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7effd700 1 -- 172.21.15.90:0/2289555541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d80103470 msgr2=0x7f3d80075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7effd700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d80103470 0x7f3d80075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:06.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7effd700 1 -- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d801094c0 msgr2=0x7f3d800761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7effd700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d801094c0 0x7f3d800761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7effd700 1 -- 172.21.15.90:0/2289555541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d801a7fe0 con 0x7f3d80103e30 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.835+0000 7f3d7ffff700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d801094c0 0x7f3d800761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.836+0000 7f3d7effd700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103e30 0x7f3d80075c60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400e9f0 tx=0x7f3d7400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.836+0000 7f3d7cff9700 1 -- 172.21.15.90:0/2289555541 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7400c9c0 con 0x7f3d80103e30 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.836+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d801a82d0 con 0x7f3d80103e30 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.836+0000 7f3d7cff9700 1 -- 172.21.15.90:0/2289555541 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d7400cb60 con 0x7f3d80103e30 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.836+0000 7f3d7cff9700 1 -- 172.21.15.90:0/2289555541 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d74018680 con 0x7f3d80103e30 2024-04-26T18:16:06.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.836+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801a8860 con 0x7f3d80103e30 2024-04-26T18:16:06.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.837+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8006a110 con 0x7f3d80103e30 2024-04-26T18:16:06.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.838+0000 7f3d7cff9700 1 -- 172.21.15.90:0/2289555541 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f3d74010430 con 0x7f3d80103e30 2024-04-26T18:16:06.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.838+0000 7f3d7cff9700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d680a3db0 0x7f3d680a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:06.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.839+0000 7f3d7f7fe700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d680a3db0 0x7f3d680a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:06.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.839+0000 7f3d7cff9700 1 -- 172.21.15.90:0/2289555541 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f3d740d0a00 con 0x7f3d80103e30 2024-04-26T18:16:06.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.839+0000 7f3d7f7fe700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d680a3db0 0x7f3d680a6270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009be0 tx=0x7f3d70009260 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:06.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:06.843+0000 7f3d7cff9700 1 -- 172.21.15.90:0/2289555541 <== mon.2 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d7409d110 con 0x7f3d80103e30 2024-04-26T18:16:06.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:06 smithi090 ceph-mon[29180]: pgmap v127: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-04-26T18:16:06.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:06.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:06.843 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:16:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:06 smithi119 ceph-mon[32752]: pgmap v127: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-04-26T18:16:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:16:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:06 smithi167 ceph-mon[32751]: pgmap v127: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-04-26T18:16:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:16:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:16:07.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.106+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d8002e490 con 0x7f3d80103e30 2024-04-26T18:16:07.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.107+0000 7f3d7cff9700 1 -- 172.21.15.90:0/2289555541 <== mon.2 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11877 (secure 0 0 0) 0x7f3d7409ca10 con 0x7f3d80103e30 2024-04-26T18:16:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:16:07.109 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:14:24.663594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24449,24436,24448]},"id":1}]} 2024-04-26T18:16:07.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d680a3db0 msgr2=0x7f3d680a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:07.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d680a3db0 0x7f3d680a6270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3d70009be0 tx=0x7f3d70009260 comp rx=0 tx=0).stop 2024-04-26T18:16:07.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103e30 msgr2=0x7f3d80075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:07.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103e30 0x7f3d80075c60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3d7400e9f0 tx=0x7f3d7400ed00 comp rx=0 tx=0).stop 2024-04-26T18:16:07.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 shutdown_connections 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d680a3db0 0x7f3d680a6270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d801094c0 0x7f3d800761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d80103470 0x7f3d80075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 --2- 172.21.15.90:0/2289555541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80103e30 0x7f3d80075c60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 >> 172.21.15.90:0/2289555541 conn(0x7f3d8007a580 msgr2=0x7f3d80102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 shutdown_connections 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:07.110+0000 7f3d85d64700 1 -- 172.21.15.90:0/2289555541 wait complete. 2024-04-26T18:16:07.111 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-26T18:16:07.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 227527 10.82 MB/sec warmup 115 sec latency 35.069 ms 2024-04-26T18:16:07.886 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0' 2024-04-26T18:16:07.886 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0 2024-04-26T18:16:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:07 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2289555541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:07 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2289555541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:07 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2289555541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:08.152 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:08.662 INFO:tasks.workunit.client.0.smithi090.stdout: 1 230042 10.84 MB/sec warmup 116 sec latency 12.722 ms 2024-04-26T18:16:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:08 smithi119 ceph-mon[32752]: pgmap v128: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-04-26T18:16:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:08 smithi167 ceph-mon[32751]: pgmap v128: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-04-26T18:16:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:08 smithi090 ceph-mon[29180]: pgmap v128: 97 pgs: 97 active+clean; 69 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 336 op/s 2024-04-26T18:16:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 -- 172.21.15.90:0/769021739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 msgr2=0x7fdac0101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/769021739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0101ce0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009930 tx=0x7fdab402a420 comp rx=0 tx=0).stop 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 -- 172.21.15.90:0/769021739 shutdown_connections 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/769021739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac0102220 0x7fdac010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/769021739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0101ce0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/769021739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac0107820 0x7fdac0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 -- 172.21.15.90:0/769021739 >> 172.21.15.90:0/769021739 conn(0x7fdac00789b0 msgr2=0x7fdac0078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 -- 172.21.15.90:0/769021739 shutdown_connections 2024-04-26T18:16:09.600 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.599+0000 7fdac4f5f640 1 -- 172.21.15.90:0/769021739 wait complete. 2024-04-26T18:16:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 Processor -- start 2024-04-26T18:16:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 -- start start 2024-04-26T18:16:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac0102220 0x7fdac019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac0107820 0x7fdac01a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0111ae0 con 0x7fdac0101860 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0111960 con 0x7fdac0107820 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdac4f5f640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac0111c60 con 0x7fdac0102220 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56106/0 (socket says 172.21.15.90:56106) 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdabe59c640 1 -- 172.21.15.90:0/3399128374 learned_addr learned my addr 172.21.15.90:0/3399128374 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdabed9d640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac0107820 0x7fdac01a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.600+0000 7fdabdd9b640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac0102220 0x7fdac019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdabe59c640 1 -- 172.21.15.90:0/3399128374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac0102220 msgr2=0x7fdac019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdabe59c640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac0102220 0x7fdac019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdabe59c640 1 -- 172.21.15.90:0/3399128374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac0107820 msgr2=0x7fdac01a4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdabe59c640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac0107820 0x7fdac01a4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdabe59c640 1 -- 172.21.15.90:0/3399128374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac01a5500 con 0x7fdac0101860 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdabe59c640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0199f40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdab800eb90 tx=0x7fdab800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdaaf7fe640 1 -- 172.21.15.90:0/3399128374 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8004b20 con 0x7fdac0101860 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdaaf7fe640 1 -- 172.21.15.90:0/3399128374 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab8004cc0 con 0x7fdac0101860 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac01a56d0 con 0x7fdac0101860 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdaaf7fe640 1 -- 172.21.15.90:0/3399128374 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8010690 con 0x7fdac0101860 2024-04-26T18:16:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.601+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac01a5ba0 con 0x7fdac0101860 2024-04-26T18:16:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.603+0000 7fdaaf7fe640 1 -- 172.21.15.90:0/3399128374 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fdab8005020 con 0x7fdac0101860 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.603+0000 7fdac7c78740 1 Processor -- start 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdac7c78740 1 -- start start 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdedf00 0x55a43bdee2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdaaf7fe640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdaa80a3dc0 0x7fdaa80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43bdfb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdac7c78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a43bd54460 con 0x55a43bdf90e0 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdac7c78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a43bd0e590 con 0x55a43bdedf00 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdac7c78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a43bd50420 con 0x55a43bdee820 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdedf00 0x55a43bdee2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdedf00 0x55a43bdee2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:60668/0 (socket says 172.21.15.90:60668) 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdabed9d640 1 -- 172.21.15.90:0/826543573 learned_addr learned my addr 172.21.15.90:0/826543573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdabdd9b640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdaa80a3dc0 0x7fdaa80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdabe59c640 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdabed9d640 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43bdfb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.604+0000 7fdaaf7fe640 1 -- 172.21.15.90:0/3399128374 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdab80d0d10 con 0x7fdac0101860 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdabed9d640 1 -- 172.21.15.90:0/826543573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 msgr2=0x55a43bdf8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdabed9d640 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdabed9d640 1 -- 172.21.15.90:0/826543573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 msgr2=0x55a43bdfb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdabed9d640 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43bdfb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdabed9d640 1 -- 172.21.15.90:0/826543573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43bd1b0d0 con 0x55a43bdedf00 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdabdd9b640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdaa80a3dc0 0x7fdaa80a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab4000bc0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdabed9d640 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdedf00 0x55a43bdee2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdab0011440 tx=0x7fdab00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a39fab5c00efdc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdaacff9640 1 -- 172.21.15.90:0/826543573 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0004050 con 0x55a43bdedf00 2024-04-26T18:16:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdaacff9640 1 -- 172.21.15.90:0/826543573 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab00041f0 con 0x55a43bdedf00 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.605+0000 7fdaacff9640 1 -- 172.21.15.90:0/826543573 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab001d620 con 0x55a43bdedf00 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 -- 172.21.15.90:0/826543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdedf00 msgr2=0x55a43bdee2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdedf00 0x55a43bdee2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdab0011440 tx=0x7fdab00046c0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 -- 172.21.15.90:0/826543573 shutdown_connections 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43bdfb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 --2- 172.21.15.90:0/826543573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdedf00 0x55a43bdee2e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 -- 172.21.15.90:0/826543573 >> 172.21.15.90:0/826543573 conn(0x55a43bdea410 msgr2=0x55a43bdea7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 -- 172.21.15.90:0/826543573 shutdown_connections 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 -- 172.21.15.90:0/826543573 wait complete. 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 -- start start 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.606+0000 7fdac7c78740 1 Processor -- start 2024-04-26T18:16:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdedf00 0x55a43bdf6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43bdf6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdac7c78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a43be00120 con 0x55a43bdedf00 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdac7c78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a43bcdbb60 con 0x55a43bdf90e0 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdac7c78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a43be002a0 con 0x55a43bdee820 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdac7c78740 10 client.0 _ll_register_callbacks cb 0x55a43bd002b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdedf00 0x55a43bdf6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdedf00 0x55a43bdf6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56124/0 (socket says 172.21.15.90:56124) 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabdd9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43bdf6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabe59c640 1 -- 172.21.15.90:0/2843606447 learned_addr learned my addr 172.21.15.90:0/2843606447 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabed9d640 1 -- 172.21.15.90:0/2843606447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 msgr2=0x55a43bdf6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabed9d640 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabed9d640 1 -- 172.21.15.90:0/2843606447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdedf00 msgr2=0x55a43bdf6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabed9d640 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdedf00 0x55a43bdf6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabed9d640 1 -- 172.21.15.90:0/2843606447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43ba66210 con 0x55a43bdf90e0 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabe59c640 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdedf00 0x55a43bdf6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdabed9d640 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43bdf6ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab00094b0 tx=0x7fdab0005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232fa1c88b8f04d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.607+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/2843606447 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0005250 con 0x55a43bdf90e0 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/2843606447 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdab00053f0 con 0x55a43bdf90e0 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab00041f0 con 0x55a43bdf90e0 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/2843606447 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdab0030050 con 0x55a43bdf90e0 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 10 client.25286 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a43bb32620 con 0x55a43bdf90e0 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/2843606447 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab00041f0 con 0x55a43bdf90e0 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/2843606447 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdab001e530 con 0x55a43bdf90e0 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 10 client.25286 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 10 client.25286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 20 client.25286 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be0d4b0 0x55a43be0f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 4 client.25286 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a43bd62070 con 0x55a43be0d4b0 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.608+0000 7fdabe59c640 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be0d4b0 0x55a43be0f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.609+0000 7fdaa5ffb640 10 client.25286 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.609+0000 7fdabe59c640 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be0d4b0 0x55a43be0f8a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9657ff39fd0cda86 server_cookie=31721fec3153b43c in_seq=0 out_seq=0 2024-04-26T18:16:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/2843606447 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a43bd62070 con 0x55a43be0d4b0 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdaa5ffb640 10 client.25286 handle_command_reply: tid=0 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 client.25286 shutdown 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 msgr2=0x55a43bdf6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43bdf6ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab00094b0 tx=0x7fdab0005850 comp rx=0 tx=0).stop 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 shutdown_connections 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be0d4b0 0x55a43be0f8a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43bdf6ab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdaa5ffb640 0 client.25286 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdee820 0x55a43bdf6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2843606447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdedf00 0x55a43bdf6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 >> 172.21.15.90:0/2843606447 conn(0x55a43bdea410 msgr2=0x55a43bdf0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 shutdown_connections 2024-04-26T18:16:09.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 1 -- 172.21.15.90:0/2843606447 wait complete. 2024-04-26T18:16:09.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.610+0000 7fdac7c78740 20 client.25286 trim_cache size 0 max 16384 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.626+0000 7fdac7c78740 1 Processor -- start 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.626+0000 7fdac7c78740 1 -- start start 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be08fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdee820 0x55a43bdf6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43be15060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdac7c78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01aea10 con 0x55a43bdf90e0 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdac7c78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01ac2e0 con 0x55a43bdee820 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdac7c78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01aa150 con 0x55a43bdedf00 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43be15060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabdd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdee820 0x55a43bdf6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be08fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43be15060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56130/0 (socket says 172.21.15.90:56130) 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 -- 172.21.15.90:0/2726552903 learned_addr learned my addr 172.21.15.90:0/2726552903 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 -- 172.21.15.90:0/2726552903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 msgr2=0x55a43be08fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be08fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 -- 172.21.15.90:0/2726552903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdee820 msgr2=0x55a43bdf6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdee820 0x55a43bdf6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 -- 172.21.15.90:0/2726552903 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a43be0fe30 con 0x55a43bdf90e0 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabdd9b640 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdee820 0x55a43bdf6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.627+0000 7fdabed9d640 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43be15060 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdab0003f40 tx=0x7fdab0009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44542e38d3fd9284 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdaa7fff640 1 -- 172.21.15.90:0/2726552903 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab002c070 con 0x55a43bdf90e0 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdaa7fff640 1 -- 172.21.15.90:0/2726552903 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab001eaf0 con 0x55a43bdf90e0 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdaa7fff640 1 -- 172.21.15.90:0/2726552903 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab00052a0 con 0x55a43bdf90e0 2024-04-26T18:16:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 -- 172.21.15.90:0/2726552903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 msgr2=0x55a43be15060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43be15060 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdab0003f40 tx=0x7fdab0009fb0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 -- 172.21.15.90:0/2726552903 shutdown_connections 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdf90e0 0x55a43be15060 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdee820 0x55a43bdf6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 --2- 172.21.15.90:0/2726552903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be08fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 -- 172.21.15.90:0/2726552903 >> 172.21.15.90:0/2726552903 conn(0x55a43bdea410 msgr2=0x55a43bdf5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 -- 172.21.15.90:0/2726552903 shutdown_connections 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.628+0000 7fdac7c78740 1 -- 172.21.15.90:0/2726552903 wait complete. 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 -- start start 2024-04-26T18:16:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 Processor -- start 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be05a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdee820 0x55a43be00d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43be014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01aa7a0 con 0x55a43bdee820 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01abc10 con 0x55a43bdf90e0 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac01aae70 con 0x55a43bdedf00 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdabdd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdee820 0x55a43be00d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be05a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdabe59c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be05a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:35016/0 (socket says 172.21.15.90:35016) 2024-04-26T18:16:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdabdd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdee820 0x55a43be00d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56142/0 (socket says 172.21.15.90:56142) 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdabe59c640 1 -- 172.21.15.90:0/996176705 learned_addr learned my addr 172.21.15.90:0/996176705 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43be014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.629+0000 7fdac7c78740 10 client.0 _ll_register_callbacks cb 0x7fdab0009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdabe59c640 1 -- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 msgr2=0x55a43be014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdabe59c640 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43be014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdabe59c640 1 -- 172.21.15.90:0/996176705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdee820 msgr2=0x55a43be00d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdabe59c640 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdee820 0x55a43be00d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdabe59c640 1 -- 172.21.15.90:0/996176705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a43be728d0 con 0x55a43bdedf00 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdabed9d640 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43be014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdabe59c640 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be05a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdab809ccb0 tx=0x7fdab800a440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=976559a3a02cfa99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdaa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:16:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/996176705 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab809cd20 con 0x55a43bdedf00 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab001eaf0 con 0x55a43bdedf00 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/996176705 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdab80b6120 con 0x55a43bdedf00 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/996176705 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdab80a10d0 con 0x55a43bdedf00 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdac7c78740 10 client.25219 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.630+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a43be73a60 con 0x55a43bdedf00 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/996176705 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab001eaf0 con 0x55a43bdedf00 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/996176705 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdab8099390 con 0x55a43bdedf00 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdac7c78740 10 client.25219 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdac7c78740 10 client.25219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdac7c78740 20 client.25219 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdac7c78740 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be74130 0x55a43be76520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdac7c78740 4 client.25219 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:16:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a43bd5bee0 con 0x55a43be74130 2024-04-26T18:16:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdabdd9b640 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be74130 0x55a43be76520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdaa5ffb640 10 client.25219 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.631+0000 7fdabdd9b640 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be74130 0x55a43be76520 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aa4a177d2382634 server_cookie=eef7d7be670e9fd3 in_seq=0 out_seq=0 2024-04-26T18:16:09.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 231548 10.81 MB/sec warmup 117 sec latency 19.989 ms 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdaa5ffb640 1 -- 172.21.15.90:0/996176705 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a43bd5bee0 con 0x55a43be74130 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdaa5ffb640 10 client.25219 handle_command_reply: tid=0 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 client.25219 shutdown 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 msgr2=0x55a43be05a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be05a00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdab809ccb0 tx=0x7fdab800a440 comp rx=0 tx=0).stop 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 shutdown_connections 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a43be74130 0x55a43be76520 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a43bdf90e0 0x55a43be014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a43bdee820 0x55a43be00d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 --2- 172.21.15.90:0/996176705 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a43bdedf00 0x55a43be05a00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 >> 172.21.15.90:0/996176705 conn(0x55a43bdea410 msgr2=0x55a43bdf0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdaa5ffb640 0 client.25219 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 shutdown_connections 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.692+0000 7fdac7c78740 1 -- 172.21.15.90:0/996176705 wait complete. 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac7c78740 20 client.25219 trim_cache size 0 max 16384 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:16:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:16:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdaa80a3dc0 msgr2=0x7fdaa80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdaa80a3dc0 0x7fdaa80a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009900 tx=0x7fdab4000bc0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 msgr2=0x7fdac0199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0199f40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdab800eb90 tx=0x7fdab800ef90 comp rx=0 tx=0).stop 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 shutdown_connections 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdaa80a3dc0 0x7fdaa80a6280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac0107820 0x7fdac01a4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac0102220 0x7fdac019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 --2- 172.21.15.90:0/3399128374 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac0101860 0x7fdac0199f40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 >> 172.21.15.90:0/3399128374 conn(0x7fdac00789b0 msgr2=0x7fdac0104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 shutdown_connections 2024-04-26T18:16:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:09.693+0000 7fdac4f5f640 1 -- 172.21.15.90:0/3399128374 wait complete. 2024-04-26T18:16:10.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 -- 172.21.15.90:0/1862803639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 msgr2=0x7f5678110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 --2- 172.21.15.90:0/1862803639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 0x7f5678110310 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f56680099c0 tx=0x7f5668009cd0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 -- 172.21.15.90:0/1862803639 shutdown_connections 2024-04-26T18:16:10.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 --2- 172.21.15.90:0/1862803639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 0x7f5678110310 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 --2- 172.21.15.90:0/1862803639 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5678109540 0x7f5678109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 --2- 172.21.15.90:0/1862803639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678103500 0x7f5678103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 -- 172.21.15.90:0/1862803639 >> 172.21.15.90:0/1862803639 conn(0x7f567807a610 msgr2=0x7f567807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 -- 172.21.15.90:0/1862803639 shutdown_connections 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 -- 172.21.15.90:0/1862803639 wait complete. 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.343+0000 7f567dca2700 1 Processor -- start 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f567dca2700 1 -- start start 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f567dca2700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678103500 0x7f567819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f567dca2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 0x7f567819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f567dca2700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5678109540 0x7f56781997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f567dca2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5678113640 con 0x7f5678103ec0 2024-04-26T18:16:10.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f567dca2700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56781134c0 con 0x7f5678103500 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f567dca2700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56781137c0 con 0x7f5678109540 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 0x7f567819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f56777fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678103500 0x7f567819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 0x7f567819fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56166/0 (socket says 172.21.15.90:56166) 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f5676ffd700 1 -- 172.21.15.90:0/1434783539 learned_addr learned my addr 172.21.15.90:0/1434783539 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f5677fff700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5678109540 0x7f56781997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f56777fe700 1 -- 172.21.15.90:0/1434783539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5678109540 msgr2=0x7f56781997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f56777fe700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5678109540 0x7f56781997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f56777fe700 1 -- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 msgr2=0x7f567819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f56777fe700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 0x7f567819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.344+0000 7f56777fe700 1 -- 172.21.15.90:0/1434783539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567819a080 con 0x7f5678103500 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.345+0000 7f56777fe700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678103500 0x7f567819f680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f567000efc0 tx=0x7f567000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.345+0000 7f5674ff9700 1 -- 172.21.15.90:0/1434783539 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700092e0 con 0x7f5678103500 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.345+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567819a310 con 0x7f5678103500 2024-04-26T18:16:10.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.345+0000 7f5674ff9700 1 -- 172.21.15.90:0/1434783539 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5670004bb0 con 0x7f5678103500 2024-04-26T18:16:10.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.345+0000 7f5674ff9700 1 -- 172.21.15.90:0/1434783539 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670004040 con 0x7f5678103500 2024-04-26T18:16:10.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.345+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781a6430 con 0x7f5678103500 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.347+0000 7f5674ff9700 1 -- 172.21.15.90:0/1434783539 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f567001b030 con 0x7f5678103500 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.348+0000 7f5674ff9700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f56640a3e50 0x7f56640a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.348+0000 7f5676ffd700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f56640a3e50 0x7f56640a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.348+0000 7f5674ff9700 1 -- 172.21.15.90:0/1434783539 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f56700d0340 con 0x7f5678103500 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.348+0000 7f5676ffd700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f56640a3e50 0x7f56640a6310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f567819af10 tx=0x7f566000b500 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.348+0000 7f5688534b80 1 Processor -- start 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.348+0000 7f5688534b80 1 -- start start 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5688534b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c3475610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5688534b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c34804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5688534b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c3482df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5688534b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8c33c88d0 con 0x55b8c3480a00 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5688534b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b8c3367770 con 0x55b8c3475230 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5688534b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8c33c4890 con 0x55b8c3475b50 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c3475610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c3475610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:60726/0 (socket says 172.21.15.90:60726) 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 -- 172.21.15.90:0/3835571010 learned_addr learned my addr 172.21.15.90:0/3835571010 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5677fff700 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c3482df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f5676ffd700 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c34804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 -- 172.21.15.90:0/3835571010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 msgr2=0x55b8c34804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c34804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 -- 172.21.15.90:0/3835571010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 msgr2=0x55b8c3482df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c3482df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 -- 172.21.15.90:0/3835571010 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c3315a60 con 0x55b8c3475230 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f56777fe700 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c3475610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56700090a0 tx=0x7f567009bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdfaddf3ea1b684e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:10.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f566dffb700 1 -- 172.21.15.90:0/3835571010 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567009c380 con 0x55b8c3475230 2024-04-26T18:16:10.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f566dffb700 1 -- 172.21.15.90:0/3835571010 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56700a0730 con 0x55b8c3475230 2024-04-26T18:16:10.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.349+0000 7f566dffb700 1 -- 172.21.15.90:0/3835571010 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700980d0 con 0x55b8c3475230 2024-04-26T18:16:10.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 -- 172.21.15.90:0/3835571010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 msgr2=0x55b8c3475610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c3475610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f56700090a0 tx=0x7f567009bdb0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 -- 172.21.15.90:0/3835571010 shutdown_connections 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c3482df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c3475610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 --2- 172.21.15.90:0/3835571010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c34804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 -- 172.21.15.90:0/3835571010 >> 172.21.15.90:0/3835571010 conn(0x55b8c33a2ed0 msgr2=0x55b8c3471720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 -- 172.21.15.90:0/3835571010 shutdown_connections 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.350+0000 7f5688534b80 1 -- 172.21.15.90:0/3835571010 wait complete. 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 -- start start 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 Processor -- start 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c34801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c347d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c347d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55b8c34835d0 con 0x55b8c3475b50 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55b8c332eae0 con 0x55b8c3480a00 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b8c3483750 con 0x55b8c3475230 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5688534b80 10 client.0 _ll_register_callbacks cb 0x55b8c335c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c347d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c347d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56184/0 (socket says 172.21.15.90:56184) 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5676ffd700 1 -- 172.21.15.90:0/1601411084 learned_addr learned my addr 172.21.15.90:0/1601411084 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f5677fff700 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c347d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.351+0000 7f56777fe700 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c34801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5676ffd700 1 -- 172.21.15.90:0/1601411084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 msgr2=0x55b8c34801b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5676ffd700 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c34801b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5676ffd700 1 -- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 msgr2=0x55b8c347d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5676ffd700 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c347d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5676ffd700 1 -- 172.21.15.90:0/1601411084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c347e0f0 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5676ffd700 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c347d4b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56600116f0 tx=0x7f56600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fe65117744405d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5656ffd700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5656ffd700 1 -- 172.21.15.90:0/1601411084 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660004230 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5656ffd700 1 -- 172.21.15.90:0/1601411084 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f56600043d0 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56700a0730 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5656ffd700 1 -- 172.21.15.90:0/1601411084 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5660003680 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5688534b80 10 client.15765 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8c348e0e0 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5656ffd700 1 -- 172.21.15.90:0/1601411084 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56700a0730 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.352+0000 7f5656ffd700 1 -- 172.21.15.90:0/1601411084 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5660030050 con 0x55b8c3475b50 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f5688534b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:10.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f5688534b80 10 client.15765 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f5688534b80 20 client.15765 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f5688534b80 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c3492b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f5688534b80 4 client.15765 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8c33d62b0 con 0x55b8c3490710 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f56777fe700 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c3492b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f5656ffd700 10 client.15765 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.353+0000 7f56777fe700 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c3492b00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2481e10703eeced server_cookie=86ba5ffcd1004a3a in_seq=0 out_seq=0 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5656ffd700 1 -- 172.21.15.90:0/1601411084 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8c33d62b0 con 0x55b8c3490710 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5656ffd700 10 client.15765 handle_command_reply: tid=0 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 client.15765 shutdown 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 msgr2=0x55b8c347d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c347d4b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f56600116f0 tx=0x7f56600047c0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 shutdown_connections 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c3492b00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c347d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c34801b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5688534b80 1 --2- 172.21.15.90:0/1601411084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c347d4b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.354+0000 7f5656ffd700 0 client.15765 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.355+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 >> 172.21.15.90:0/1601411084 conn(0x55b8c33a2ed0 msgr2=0x55b8c3471350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.355+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 shutdown_connections 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.355+0000 7f5688534b80 1 -- 172.21.15.90:0/1601411084 wait complete. 2024-04-26T18:16:10.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.355+0000 7f5688534b80 20 client.15765 trim_cache size 0 max 16384 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.374+0000 7f5688534b80 1 Processor -- start 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.374+0000 7f5688534b80 1 -- start start 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5688534b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c347c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5688534b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c348c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5688534b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c34983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5688534b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b0330 con 0x55b8c3480a00 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5688534b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56781adbc0 con 0x55b8c3475230 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5688534b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56781aba70 con 0x55b8c3475b50 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c347c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c348c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c347c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:60748/0 (socket says 172.21.15.90:60748) 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 -- 172.21.15.90:0/1773180383 learned_addr learned my addr 172.21.15.90:0/1773180383 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f5677fff700 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c34983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 -- 172.21.15.90:0/1773180383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 msgr2=0x55b8c348c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c348c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 -- 172.21.15.90:0/1773180383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 msgr2=0x55b8c34983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c34983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 -- 172.21.15.90:0/1773180383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8c34769f0 con 0x55b8c3475230 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.375+0000 7f56777fe700 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c347c450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f567009c310 tx=0x7f5670009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c386fd3d989f0554 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f566d7fa700 1 -- 172.21.15.90:0/1773180383 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700a4ad0 con 0x55b8c3475230 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f566d7fa700 1 -- 172.21.15.90:0/1773180383 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56700a4c70 con 0x55b8c3475230 2024-04-26T18:16:10.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f566d7fa700 1 -- 172.21.15.90:0/1773180383 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56700983c0 con 0x55b8c3475230 2024-04-26T18:16:10.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 -- 172.21.15.90:0/1773180383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 msgr2=0x55b8c347c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c347c450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f567009c310 tx=0x7f5670009480 comp rx=0 tx=0).stop 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 -- 172.21.15.90:0/1773180383 shutdown_connections 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3480a00 0x55b8c34983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3475230 0x55b8c347c450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 --2- 172.21.15.90:0/1773180383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475b50 0x55b8c348c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 -- 172.21.15.90:0/1773180383 >> 172.21.15.90:0/1773180383 conn(0x55b8c33a2ed0 msgr2=0x55b8c3478bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 -- 172.21.15.90:0/1773180383 shutdown_connections 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.376+0000 7f5688534b80 1 -- 172.21.15.90:0/1773180383 wait complete. 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 -- start start 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 Processor -- start 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c3488d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.381 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c34859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c3485ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f56781ac0c0 con 0x55b8c3475b50 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f56781ad4f0 con 0x55b8c3480a00 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56781ac750 con 0x55b8c3475230 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5688534b80 10 client.0 _ll_register_callbacks cb 0x55b8c335c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f56777fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c3488d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5677fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c3485ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c34859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5677fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c3485ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:60750/0 (socket says 172.21.15.90:60750) 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f56777fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c3488d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:35064/0 (socket says 172.21.15.90:35064) 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5676ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c34859b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56196/0 (socket says 172.21.15.90:56196) 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.377+0000 7f5677fff700 1 -- 172.21.15.90:0/585291493 learned_addr learned my addr 172.21.15.90:0/585291493 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:10.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f56777fe700 1 -- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 msgr2=0x55b8c3485ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f56777fe700 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c3485ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f56777fe700 1 -- 172.21.15.90:0/585291493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 msgr2=0x55b8c34859b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f56777fe700 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c34859b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f56777fe700 1 -- 172.21.15.90:0/585291493 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8c3486730 con 0x55b8c3475230 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5676ffd700 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c34859b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5677fff700 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c3485ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f56777fe700 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c3488d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f567009a630 tx=0x7f567009c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557f7cdf73c1d567 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5656ffd700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5656ffd700 1 -- 172.21.15.90:0/585291493 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567009c5e0 con 0x55b8c3475230 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5656ffd700 1 -- 172.21.15.90:0/585291493 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f567009b7d0 con 0x55b8c3475230 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56700a4c70 con 0x55b8c3475230 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5656ffd700 1 -- 172.21.15.90:0/585291493 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f567009b9d0 con 0x55b8c3475230 2024-04-26T18:16:10.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5688534b80 10 client.25237 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.378+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8c34f7c20 con 0x55b8c3475230 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5656ffd700 1 -- 172.21.15.90:0/585291493 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56700a4c70 con 0x55b8c3475230 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5656ffd700 1 -- 172.21.15.90:0/585291493 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f56700a0160 con 0x55b8c3475230 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5688534b80 10 client.25237 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5688534b80 10 client.25237 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5688534b80 20 client.25237 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5688534b80 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c34f9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5688534b80 4 client.25237 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}] 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}) -- 0x55b8c33d0350 con 0x55b8c3490710 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5676ffd700 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c34f9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5676ffd700 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c34f9e50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11623f1f4cb3b117 server_cookie=b2487c5fa09b962b in_seq=0 out_seq=0 2024-04-26T18:16:10.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.379+0000 7f5656ffd700 10 client.25237 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5656ffd700 1 -- 172.21.15.90:0/585291493 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b8c33d0350 con 0x55b8c3490710 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5656ffd700 10 client.25237 handle_command_reply: tid=0 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 client.25237 shutdown 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 msgr2=0x55b8c3488d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c3488d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f567009a630 tx=0x7f567009c940 comp rx=0 tx=0).stop 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 shutdown_connections 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55b8c3490710 0x55b8c34f9e50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8c3480a00 0x55b8c3485ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5656ffd700 0 client.25237 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b8c3475230 0x55b8c3488d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 --2- 172.21.15.90:0/585291493 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55b8c3475b50 0x55b8c34859b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 >> 172.21.15.90:0/585291493 conn(0x55b8c33a2ed0 msgr2=0x55b8c347ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 shutdown_connections 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 1 -- 172.21.15.90:0/585291493 wait complete. 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.446+0000 7f5688534b80 20 client.25237 trim_cache size 0 max 16384 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:16:10.448 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 27, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 10, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "fb7a5d51": { 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 10, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 79.5, 2024-04-26T18:16:10.449 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:10.450 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f56640a3e50 msgr2=0x7f56640a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f56640a3e50 0x7f56640a6310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f567819af10 tx=0x7f566000b500 comp rx=0 tx=0).stop 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678103500 msgr2=0x7f567819f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678103500 0x7f567819f680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f567000efc0 tx=0x7f567000c3a0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 shutdown_connections 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f56640a3e50 0x7f56640a6310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5678109540 0x7f56781997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5678103500 0x7f567819f680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 --2- 172.21.15.90:0/1434783539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5678103ec0 0x7f567819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:10.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 >> 172.21.15.90:0/1434783539 conn(0x7f567807a610 msgr2=0x7f5678101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:10.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 shutdown_connections 2024-04-26T18:16:10.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:10.447+0000 7f567dca2700 1 -- 172.21.15.90:0/1434783539 wait complete. 2024-04-26T18:16:10.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 10.81 MB/sec warmup 118 sec latency 251.484 ms 2024-04-26T18:16:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:10 smithi119 ceph-mon[32752]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 462 op/s 2024-04-26T18:16:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:10 smithi167 ceph-mon[32751]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 462 op/s 2024-04-26T18:16:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:10 smithi090 ceph-mon[29180]: pgmap v129: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 462 op/s 2024-04-26T18:16:11.203 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fb7a5d51', quiesce took 6.3 seconds. Will release after: 19.5 2024-04-26T18:16:11.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 10.72 MB/sec warmup 119 sec latency 1251.559 ms 2024-04-26T18:16:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:12 smithi119 ceph-mon[32752]: pgmap v130: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-04-26T18:16:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:12 smithi167 ceph-mon[32751]: pgmap v130: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-04-26T18:16:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:12 smithi090 ceph-mon[29180]: pgmap v130: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 280 op/s 2024-04-26T18:16:13.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 1 sec latency 3251.670 ms 2024-04-26T18:16:14.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 2 sec latency 4251.756 ms 2024-04-26T18:16:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:14 smithi119 ceph-mon[32752]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 393 op/s 2024-04-26T18:16:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:14 smithi167 ceph-mon[32751]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 393 op/s 2024-04-26T18:16:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:14 smithi090 ceph-mon[29180]: pgmap v131: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 393 op/s 2024-04-26T18:16:15.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 3 sec latency 5251.855 ms 2024-04-26T18:16:16.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 4 sec latency 6251.909 ms 2024-04-26T18:16:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:16 smithi119 ceph-mon[32752]: pgmap v132: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-04-26T18:16:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:16 smithi167 ceph-mon[32751]: pgmap v132: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-04-26T18:16:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:16 smithi090 ceph-mon[29180]: pgmap v132: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-04-26T18:16:17.664 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 5 sec latency 7251.954 ms 2024-04-26T18:16:18.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 6 sec latency 8252.011 ms 2024-04-26T18:16:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:18 smithi119 ceph-mon[32752]: pgmap v133: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-04-26T18:16:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:18 smithi167 ceph-mon[32751]: pgmap v133: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-04-26T18:16:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:18 smithi090 ceph-mon[29180]: pgmap v133: 97 pgs: 97 active+clean; 78 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 2.5 MiB/s wr, 239 op/s 2024-04-26T18:16:19.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 7 sec latency 9252.118 ms 2024-04-26T18:16:20.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 8 sec latency 10252.215 ms 2024-04-26T18:16:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:20 smithi119 ceph-mon[32752]: pgmap v134: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-04-26T18:16:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:20 smithi167 ceph-mon[32751]: pgmap v134: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-04-26T18:16:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:20 smithi090 ceph-mon[29180]: pgmap v134: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.4 MiB/s wr, 273 op/s 2024-04-26T18:16:21.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 9 sec latency 11252.330 ms 2024-04-26T18:16:22.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 10 sec latency 12252.455 ms 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 -- 172.21.15.90:0/2771292677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 msgr2=0x7efe8c1067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 --2- 172.21.15.90:0/2771292677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c1067b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7efe7c009930 tx=0x7efe7c02a4d0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 -- 172.21.15.90:0/2771292677 shutdown_connections 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 --2- 172.21.15.90:0/2771292677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c106dd0 0x7efe8c107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 --2- 172.21.15.90:0/2771292677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c1067b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 --2- 172.21.15.90:0/2771292677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe8c1019d0 0x7efe8c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 -- 172.21.15.90:0/2771292677 >> 172.21.15.90:0/2771292677 conn(0x7efe8c077bb0 msgr2=0x7efe8c077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.790+0000 7efe940cc640 1 -- 172.21.15.90:0/2771292677 shutdown_connections 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 -- 172.21.15.90:0/2771292677 wait complete. 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 Processor -- start 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 -- start start 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c1019d0 0x7efe8c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe8c106dd0 0x7efe8c1a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1191d0 con 0x7efe8c1022f0 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c119050 con 0x7efe8c1019d0 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe940cc640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c119350 con 0x7efe8c106dd0 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe91667640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.791+0000 7efe91667640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c19fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49462/0 (socket says 172.21.15.90:49462) 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91667640 1 -- 172.21.15.90:0/1276370729 learned_addr learned my addr 172.21.15.90:0/1276370729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe92669640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe8c106dd0 0x7efe8c1a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91e68640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c1019d0 0x7efe8c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91667640 1 -- 172.21.15.90:0/1276370729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe8c106dd0 msgr2=0x7efe8c1a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91667640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe8c106dd0 0x7efe8c1a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91667640 1 -- 172.21.15.90:0/1276370729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c1019d0 msgr2=0x7efe8c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91667640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c1019d0 0x7efe8c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91667640 1 -- 172.21.15.90:0/1276370729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8c1a46c0 con 0x7efe8c1022f0 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe91667640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c19fc10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efe7c004770 tx=0x7efe7c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe82ffd640 1 -- 172.21.15.90:0/1276370729 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c03f070 con 0x7efe8c1022f0 2024-04-26T18:16:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8c1a4950 con 0x7efe8c1022f0 2024-04-26T18:16:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe82ffd640 1 -- 172.21.15.90:0/1276370729 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7c03b460 con 0x7efe8c1022f0 2024-04-26T18:16:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe82ffd640 1 -- 172.21.15.90:0/1276370729 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c035070 con 0x7efe8c1022f0 2024-04-26T18:16:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.792+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8c1ac240 con 0x7efe8c1022f0 2024-04-26T18:16:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.794+0000 7efe82ffd640 1 -- 172.21.15.90:0/1276370729 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7efe7c03b8d0 con 0x7efe8c1022f0 2024-04-26T18:16:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 Processor -- start 2024-04-26T18:16:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 -- start start 2024-04-26T18:16:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a9bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a9c0e0 0x564c22a9e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564c229f7460 con 0x564c22a9c0e0 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564c229b1590 con 0x564c22a91820 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe96ddf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564c229f3420 con 0x564c22a90f00 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.795+0000 7efe82ffd640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe780a3dc0 0x7efe780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a9c0e0 0x564c22a9e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a9c0e0 0x564c22a9e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49476/0 (socket says 172.21.15.90:49476) 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 -- 172.21.15.90:0/1280997649 learned_addr learned my addr 172.21.15.90:0/1280997649 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe92669640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe780a3dc0 0x7efe780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe92669640 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91e68640 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a9bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe82ffd640 1 -- 172.21.15.90:0/1276370729 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7efe7c0f0b30 con 0x7efe8c1022f0 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 -- 172.21.15.90:0/1280997649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 msgr2=0x564c22a912e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a912e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 -- 172.21.15.90:0/1280997649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 msgr2=0x564c22a9bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a9bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 -- 172.21.15.90:0/1280997649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c229be0d0 con 0x564c22a9c0e0 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91e68640 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a9bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe91667640 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a9c0e0 0x564c22a9e4d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0ee200 tx=0x7efe7c049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e611dca225146af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe92669640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe780a3dc0 0x7efe780a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe84009fd0 tx=0x7efe84009d60 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe77fff640 1 -- 172.21.15.90:0/1280997649 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c048070 con 0x564c22a9c0e0 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe77fff640 1 -- 172.21.15.90:0/1280997649 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe7c0dd0d0 con 0x564c22a9c0e0 2024-04-26T18:16:22.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.796+0000 7efe77fff640 1 -- 172.21.15.90:0/1280997649 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0ee510 con 0x564c22a9c0e0 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 -- 172.21.15.90:0/1280997649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a9c0e0 msgr2=0x564c22a9e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a9c0e0 0x564c22a9e4d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0ee200 tx=0x7efe7c049000 comp rx=0 tx=0).stop 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 -- 172.21.15.90:0/1280997649 shutdown_connections 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a9c0e0 0x564c22a9e4d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a9bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1280997649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a912e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 -- 172.21.15.90:0/1280997649 >> 172.21.15.90:0/1280997649 conn(0x564c22a8d410 msgr2=0x564c22a8d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 -- 172.21.15.90:0/1280997649 shutdown_connections 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 -- 172.21.15.90:0/1280997649 wait complete. 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 -- start start 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.797+0000 7efe96ddf740 1 Processor -- start 2024-04-26T18:16:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22a99080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a995c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 0x564c22a99b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe96ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564c22a9ead0 con 0x564c22a90f00 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe96ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564c2297eb60 con 0x564c22a91820 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe96ddf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564c22a9ec50 con 0x564c22a9c0e0 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe96ddf740 10 client.0 _ll_register_callbacks cb 0x564c229a32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22a99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22a99080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49478/0 (socket says 172.21.15.90:49478) 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 -- 172.21.15.90:0/3324561874 learned_addr learned my addr 172.21.15.90:0/3324561874 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe92669640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 0x564c22a99b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 -- 172.21.15.90:0/3324561874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 msgr2=0x564c22a99b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 0x564c22a99b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91667640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 -- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 msgr2=0x564c22a995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91e68640 1 -- 172.21.15.90:0/3324561874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c22709210 con 0x564c22a90f00 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.798+0000 7efe91667640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe91e68640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22a99080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe88010750 tx=0x7efe88010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf32ebe973741305 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe757fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe757fa640 1 -- 172.21.15.90:0/3324561874 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8800cd20 con 0x564c22a90f00 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe757fa640 1 -- 172.21.15.90:0/3324561874 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7efe88004020 con 0x564c22a90f00 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564c22a05070 con 0x564c22a90f00 2024-04-26T18:16:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe757fa640 1 -- 172.21.15.90:0/3324561874 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe88010e60 con 0x564c22a90f00 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c227d5620 con 0x564c22a90f00 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe757fa640 1 -- 172.21.15.90:0/3324561874 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564c22a05070 con 0x564c22a90f00 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe757fa640 1 -- 172.21.15.90:0/3324561874 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efe88023020 con 0x564c22a90f00 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 20 client.15783 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22aac020 0x564c22aae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 4 client.15783 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.799+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c229feee0 con 0x564c22aac020 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.800+0000 7efe91667640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22aac020 0x564c22aae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.800+0000 7efe91667640 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22aac020 0x564c22aae4b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef89daa6c8faa75 server_cookie=54dafc0216433c37 in_seq=0 out_seq=0 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.800+0000 7efe757fa640 10 client.15783 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe757fa640 1 -- 172.21.15.90:0/3324561874 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c229feee0 con 0x564c22aac020 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe757fa640 10 client.15783 handle_command_reply: tid=0 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 client.15783 shutdown 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 msgr2=0x564c22a99080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22a99080 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe88010750 tx=0x7efe88010c10 comp rx=0 tx=0).stop 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 shutdown_connections 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22aac020 0x564c22aae4b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 0x564c22a99b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe757fa640 0 client.15783 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 --2- 172.21.15.90:0/3324561874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22a99080 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 >> 172.21.15.90:0/3324561874 conn(0x564c22a8d410 msgr2=0x564c22a93140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 shutdown_connections 2024-04-26T18:16:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 1 -- 172.21.15.90:0/3324561874 wait complete. 2024-04-26T18:16:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.801+0000 7efe96ddf740 20 client.15783 trim_cache size 0 max 16384 2024-04-26T18:16:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 Processor -- start 2024-04-26T18:16:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 -- start start 2024-04-26T18:16:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22aa7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a8c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 0x564c22ab3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1b71f0 con 0x564c22a90f00 2024-04-26T18:16:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1b4ac0 con 0x564c22a91820 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe96ddf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1b2910 con 0x564c22a9c0e0 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22aa7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22aa7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49488/0 (socket says 172.21.15.90:49488) 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 -- 172.21.15.90:0/1490949564 learned_addr learned my addr 172.21.15.90:0/1490949564 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91667640 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a8c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 -- 172.21.15.90:0/1490949564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 msgr2=0x564c22ab3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 0x564c22ab3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 -- 172.21.15.90:0/1490949564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 msgr2=0x564c22a8c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a8c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91e68640 1 -- 172.21.15.90:0/1490949564 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c22aaab40 con 0x564c22a90f00 2024-04-26T18:16:22.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.816+0000 7efe91667640 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a8c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe91e68640 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22aa7900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efe88003ef0 tx=0x7efe88010650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a6a7777ddfdf23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe777fe640 1 -- 172.21.15.90:0/1490949564 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88025730 con 0x564c22a90f00 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe777fe640 1 -- 172.21.15.90:0/1490949564 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe88004020 con 0x564c22a90f00 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe777fe640 1 -- 172.21.15.90:0/1490949564 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe88025d50 con 0x564c22a90f00 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 -- 172.21.15.90:0/1490949564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 msgr2=0x564c22aa7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22aa7900 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efe88003ef0 tx=0x7efe88010650 comp rx=0 tx=0).stop 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 -- 172.21.15.90:0/1490949564 shutdown_connections 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a9c0e0 0x564c22ab3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a91820 0x564c22a8c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 --2- 172.21.15.90:0/1490949564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a90f00 0x564c22aa7900 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 -- 172.21.15.90:0/1490949564 >> 172.21.15.90:0/1490949564 conn(0x564c22a8d410 msgr2=0x564c22a8bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:22.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 -- 172.21.15.90:0/1490949564 shutdown_connections 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.817+0000 7efe96ddf740 1 -- 172.21.15.90:0/1490949564 wait complete. 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 -- start start 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 Processor -- start 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a9fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a91820 0x564c22aa0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a9c0e0 0x564c22b115f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1b2f80 con 0x564c22a91820 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1b43f0 con 0x564c22a9c0e0 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c1b3650 con 0x564c22a90f00 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe96ddf740 10 client.0 _ll_register_callbacks cb 0x7efe8800e350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:22.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe91667640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a91820 0x564c22aa0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe91667640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a91820 0x564c22aa0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49498/0 (socket says 172.21.15.90:49498) 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe91667640 1 -- 172.21.15.90:0/368408708 learned_addr learned my addr 172.21.15.90:0/368408708 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe91e68640 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a9fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe91667640 1 -- 172.21.15.90:0/368408708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 msgr2=0x564c22a9fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe91667640 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a9fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.818+0000 7efe91667640 1 -- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a9c0e0 msgr2=0x564c22b115f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe92669640 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a9c0e0 0x564c22b115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe91667640 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a9c0e0 0x564c22b115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe91667640 1 -- 172.21.15.90:0/368408708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c22b11b30 con 0x564c22a91820 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe91667640 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a91820 0x564c22aa0430 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0bc7e0 tx=0x7efe7c0bd260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2771b6d497a5e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:22.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe757fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe757fa640 1 -- 172.21.15.90:0/368408708 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c048040 con 0x564c22a91820 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe757fa640 1 -- 172.21.15.90:0/368408708 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7efe7c0b9150 con 0x564c22a91820 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe88004020 con 0x564c22a91820 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe757fa640 1 -- 172.21.15.90:0/368408708 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efe7c0c0e90 con 0x564c22a91820 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 10 client.15789 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564c22b12db0 con 0x564c22a91820 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe757fa640 1 -- 172.21.15.90:0/368408708 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe88004020 con 0x564c22a91820 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe757fa640 1 -- 172.21.15.90:0/368408708 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7efe7c04b060 con 0x564c22a91820 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 10 client.15789 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:22.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 20 client.15789 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22b13320 0x564c22b15740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 4 client.15789 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:16:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.819+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c2297a5f0 con 0x564c22b13320 2024-04-26T18:16:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.820+0000 7efe91e68640 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22b13320 0x564c22b15740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.820+0000 7efe91e68640 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22b13320 0x564c22b15740 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4911e521477000a server_cookie=db54215f7368f592 in_seq=0 out_seq=0 2024-04-26T18:16:22.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.820+0000 7efe757fa640 10 client.15789 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.866+0000 7efe757fa640 1 -- 172.21.15.90:0/368408708 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c2297a5f0 con 0x564c22b13320 2024-04-26T18:16:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.866+0000 7efe757fa640 10 client.15789 handle_command_reply: tid=0 2024-04-26T18:16:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.866+0000 7efe96ddf740 1 client.15789 shutdown 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a91820 msgr2=0x564c22aa0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a91820 0x564c22aa0430 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7efe7c0bc7e0 tx=0x7efe7c0bd260 comp rx=0 tx=0).stop 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 shutdown_connections 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x564c22b13320 0x564c22b15740 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c22a9c0e0 0x564c22b115f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564c22a91820 0x564c22aa0430 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe757fa640 0 client.15789 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 --2- 172.21.15.90:0/368408708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564c22a90f00 0x564c22a9fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 >> 172.21.15.90:0/368408708 conn(0x564c22a8d410 msgr2=0x564c22a8c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:22.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 shutdown_connections 2024-04-26T18:16:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 1 -- 172.21.15.90:0/368408708 wait complete. 2024-04-26T18:16:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe96ddf740 20 client.15789 trim_cache size 0 max 16384 2024-04-26T18:16:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:16:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:16:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:16:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:16:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe780a3dc0 msgr2=0x7efe780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe780a3dc0 0x7efe780a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efe84009fd0 tx=0x7efe84009d60 comp rx=0 tx=0).stop 2024-04-26T18:16:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 msgr2=0x7efe8c19fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c19fc10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efe7c004770 tx=0x7efe7c0042f0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 shutdown_connections 2024-04-26T18:16:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe780a3dc0 0x7efe780a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe8c106dd0 0x7efe8c1a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe8c1022f0 0x7efe8c19fc10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 --2- 172.21.15.90:0/1276370729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c1019d0 0x7efe8c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.867+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 >> 172.21.15.90:0/1276370729 conn(0x7efe8c077bb0 msgr2=0x7efe8c104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.868+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 shutdown_connections 2024-04-26T18:16:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:22.868+0000 7efe940cc640 1 -- 172.21.15.90:0/1276370729 wait complete. 2024-04-26T18:16:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:22 smithi119 ceph-mon[32752]: pgmap v135: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 2.0 MiB/s wr, 147 op/s 2024-04-26T18:16:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:22 smithi167 ceph-mon[32751]: pgmap v135: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 2.0 MiB/s wr, 147 op/s 2024-04-26T18:16:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:22 smithi090 ceph-mon[29180]: pgmap v135: 97 pgs: 97 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 2.0 MiB/s wr, 147 op/s 2024-04-26T18:16:23.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 11 sec latency 13252.553 ms 2024-04-26T18:16:24.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 12 sec latency 14252.647 ms 2024-04-26T18:16:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:24 smithi119 ceph-mon[32752]: pgmap v136: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-04-26T18:16:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:24 smithi167 ceph-mon[32751]: pgmap v136: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-04-26T18:16:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:24 smithi090 ceph-mon[29180]: pgmap v136: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 2.8 MiB/s wr, 189 op/s 2024-04-26T18:16:25.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 13 sec latency 15252.767 ms 2024-04-26T18:16:26.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 14 sec latency 16252.872 ms 2024-04-26T18:16:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:26 smithi119 ceph-mon[32752]: pgmap v137: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-26T18:16:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:26 smithi167 ceph-mon[32751]: pgmap v137: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-26T18:16:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:26 smithi090 ceph-mon[29180]: pgmap v137: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-26T18:16:27.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 15 sec latency 17252.957 ms 2024-04-26T18:16:28.665 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 16 sec latency 18253.033 ms 2024-04-26T18:16:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:28 smithi119 ceph-mon[32752]: pgmap v138: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-26T18:16:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:28 smithi167 ceph-mon[32751]: pgmap v138: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-26T18:16:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:28 smithi090 ceph-mon[29180]: pgmap v138: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 1.6 MiB/s wr, 76 op/s 2024-04-26T18:16:29.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 17 sec latency 19253.131 ms 2024-04-26T18:16:30.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 18 sec latency 20253.275 ms 2024-04-26T18:16:30.704 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:16:30.963 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:30 smithi119 ceph-mon[32752]: pgmap v139: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 2.4 MiB/s wr, 107 op/s 2024-04-26T18:16:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:30 smithi167 ceph-mon[32751]: pgmap v139: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 2.4 MiB/s wr, 107 op/s 2024-04-26T18:16:31.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:30 smithi090 ceph-mon[29180]: pgmap v139: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 2.4 MiB/s wr, 107 op/s 2024-04-26T18:16:31.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 19 sec latency 21253.337 ms 2024-04-26T18:16:32.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 20 sec latency 22253.362 ms 2024-04-26T18:16:32.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1977622097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78102320 msgr2=0x7f8b781027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:32.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1977622097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78102320 0x7f8b781027a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8b60009990 tx=0x7f8b60009ca0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1977622097 shutdown_connections 2024-04-26T18:16:32.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1977622097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b78102ce0 0x7f8b7810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1977622097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b78108390 0x7f8b78108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1977622097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78102320 0x7f8b781027a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1977622097 >> 172.21.15.90:0/1977622097 conn(0x7f8b780792d0 msgr2=0x7f8b780796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1977622097 shutdown_connections 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.846+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1977622097 wait complete. 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 Processor -- start 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 -- start start 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b78102320 0x7f8b7819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b78102ce0 0x7f8b7819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78108390 0x7f8b781999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8b78112530 con 0x7f8b78108390 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781123b0 con 0x7f8b78102ce0 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b7ce56700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b781126b0 con 0x7f8b78102320 2024-04-26T18:16:32.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b76d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78108390 0x7f8b781999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.847+0000 7f8b76d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78108390 0x7f8b781999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58348/0 (socket says 172.21.15.90:58348) 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b76d9d700 1 -- 172.21.15.90:0/1061855508 learned_addr learned my addr 172.21.15.90:0/1061855508 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b75d9b700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b78102ce0 0x7f8b7819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b7659c700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b78102320 0x7f8b7819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b76d9d700 1 -- 172.21.15.90:0/1061855508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b78102320 msgr2=0x7f8b7819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b76d9d700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b78102320 0x7f8b7819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b76d9d700 1 -- 172.21.15.90:0/1061855508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b78102ce0 msgr2=0x7f8b7819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b76d9d700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b78102ce0 0x7f8b7819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b76d9d700 1 -- 172.21.15.90:0/1061855508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b7819a100 con 0x7f8b78108390 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b76d9d700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78108390 0x7f8b781999a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000b550 tx=0x7f8b7000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b6f7fe700 1 -- 172.21.15.90:0/1061855508 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70015020 con 0x7f8b78108390 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b6f7fe700 1 -- 172.21.15.90:0/1061855508 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b70011ce0 con 0x7f8b78108390 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b6f7fe700 1 -- 172.21.15.90:0/1061855508 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b70010c90 con 0x7f8b78108390 2024-04-26T18:16:32.851 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b7819a3f0 con 0x7f8b78108390 2024-04-26T18:16:32.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.848+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b781a6660 con 0x7f8b78108390 2024-04-26T18:16:32.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.849+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b7806a110 con 0x7f8b78108390 2024-04-26T18:16:32.852 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.851+0000 7f8b6f7fe700 1 -- 172.21.15.90:0/1061855508 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f8b70018020 con 0x7f8b78108390 2024-04-26T18:16:32.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.852+0000 7f8b6f7fe700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8b640a3db0 0x7f8b640a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:32.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.852+0000 7f8b7659c700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8b640a3db0 0x7f8b640a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:32.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.852+0000 7f8b6f7fe700 1 -- 172.21.15.90:0/1061855508 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f8b700d02d0 con 0x7f8b78108390 2024-04-26T18:16:32.853 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.852+0000 7f8b7659c700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8b640a3db0 0x7f8b640a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009550 tx=0x7f8b680091b0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:32.855 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:32.855+0000 7f8b6f7fe700 1 -- 172.21.15.90:0/1061855508 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8b7001f020 con 0x7f8b78108390 2024-04-26T18:16:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:32 smithi119 ceph-mon[32752]: pgmap v140: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.5 MiB/s wr, 73 op/s 2024-04-26T18:16:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:32 smithi167 ceph-mon[32751]: pgmap v140: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.5 MiB/s wr, 73 op/s 2024-04-26T18:16:33.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.100+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b781a6830 con 0x7f8b78108390 2024-04-26T18:16:33.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:32 smithi090 ceph-mon[29180]: pgmap v140: 97 pgs: 97 active+clean; 74 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 1.5 MiB/s wr, 73 op/s 2024-04-26T18:16:33.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.101+0000 7f8b6f7fe700 1 -- 172.21.15.90:0/1061855508 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11877 (secure 0 0 0) 0x7f8b7009ca90 con 0x7f8b78108390 2024-04-26T18:16:33.102 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:16:33.102 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:14:24.663594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24449,24436,24448]},"id":1}]} 2024-04-26T18:16:33.103 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8b640a3db0 msgr2=0x7f8b640a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8b640a3db0 0x7f8b640a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009550 tx=0x7f8b680091b0 comp rx=0 tx=0).stop 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78108390 msgr2=0x7f8b781999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78108390 0x7f8b781999a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000b550 tx=0x7f8b7000b860 comp rx=0 tx=0).stop 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 shutdown_connections 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8b640a3db0 0x7f8b640a6270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b78108390 0x7f8b781999a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b78102320 0x7f8b7819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 --2- 172.21.15.90:0/1061855508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b78102ce0 0x7f8b7819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 >> 172.21.15.90:0/1061855508 conn(0x7f8b780792d0 msgr2=0x7f8b78101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:33.104 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 shutdown_connections 2024-04-26T18:16:33.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:33.103+0000 7f8b7ce56700 1 -- 172.21.15.90:0/1061855508 wait complete. 2024-04-26T18:16:33.105 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-26T18:16:33.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 21 sec latency 23253.439 ms 2024-04-26T18:16:33.720 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db --set-id fb7a5d51 --release --await' 2024-04-26T18:16:33.720 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db --set-id fb7a5d51 --release --await 2024-04-26T18:16:33.962 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:33 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1061855508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:33 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1061855508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:34.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:33 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1061855508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:34.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 22 sec latency 24253.487 ms 2024-04-26T18:16:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:34 smithi167 ceph-mon[32751]: pgmap v141: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-04-26T18:16:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:34 smithi090 ceph-mon[29180]: pgmap v141: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-04-26T18:16:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:34 smithi119 ceph-mon[32752]: pgmap v141: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 103 op/s 2024-04-26T18:16:35.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 233505 0.00 MB/sec execute 23 sec latency 25253.567 ms 2024-04-26T18:16:35.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 -- 172.21.15.90:0/2398928193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5214103e30 msgr2=0x7f5214110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 --2- 172.21.15.90:0/2398928193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5214103e30 0x7f5214110290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f520c00b0a0 tx=0x7f520c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 -- 172.21.15.90:0/2398928193 shutdown_connections 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 --2- 172.21.15.90:0/2398928193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5214103e30 0x7f5214110290 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 --2- 172.21.15.90:0/2398928193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52141094c0 0x7f52141098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 --2- 172.21.15.90:0/2398928193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5214103470 0x7f52141038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 -- 172.21.15.90:0/2398928193 >> 172.21.15.90:0/2398928193 conn(0x7f521407a580 msgr2=0x7f521407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 -- 172.21.15.90:0/2398928193 shutdown_connections 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.737+0000 7f521ae5c700 1 -- 172.21.15.90:0/2398928193 wait complete. 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 Processor -- start 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 -- start start 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5214103470 0x7f521419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5214103e30 0x7f521419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.740 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141094c0 0x7f5214199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52141135c0 con 0x7f52141094c0 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5214113440 con 0x7f5214103470 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f521ae5c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5214113740 con 0x7f5214103e30 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f52193f9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141094c0 0x7f5214199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f52193f9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141094c0 0x7f5214199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58374/0 (socket says 172.21.15.90:58374) 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f52193f9700 1 -- 172.21.15.90:0/1895917350 learned_addr learned my addr 172.21.15.90:0/1895917350 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f5213fff700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5214103e30 0x7f521419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.738+0000 7f5218bf8700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5214103470 0x7f521419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f52193f9700 1 -- 172.21.15.90:0/1895917350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5214103e30 msgr2=0x7f521419fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f52193f9700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5214103e30 0x7f521419fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f52193f9700 1 -- 172.21.15.90:0/1895917350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5214103470 msgr2=0x7f521419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.741 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f52193f9700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5214103470 0x7f521419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f52193f9700 1 -- 172.21.15.90:0/1895917350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521419a020 con 0x7f52141094c0 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f5218bf8700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5214103470 0x7f521419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f52193f9700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141094c0 0x7f5214199840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f520c00b830 tx=0x7f520c004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f5211ffb700 1 -- 172.21.15.90:0/1895917350 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c021070 con 0x7f52141094c0 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f5211ffb700 1 -- 172.21.15.90:0/1895917350 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f520c003c00 con 0x7f52141094c0 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521419a2b0 con 0x7f52141094c0 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f5211ffb700 1 -- 172.21.15.90:0/1895917350 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520c004810 con 0x7f52141094c0 2024-04-26T18:16:35.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.739+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52141a6520 con 0x7f52141094c0 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.741+0000 7f5211ffb700 1 -- 172.21.15.90:0/1895917350 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f520c007a00 con 0x7f52141094c0 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.742+0000 7f5211ffb700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f52040a3db0 0x7f52040a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.742+0000 7f5211ffb700 1 -- 172.21.15.90:0/1895917350 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f520c00f020 con 0x7f52141094c0 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.742+0000 7f5218bf8700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f52040a3db0 0x7f52040a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.742+0000 7f5218bf8700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f52040a3db0 0x7f52040a6270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5208005e10 tx=0x7f5208005d60 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 Processor -- start 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 -- start start 2024-04-26T18:16:35.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec80d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec818430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec81ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5637ec760840 con 0x5637ec80d1a0 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5637ec6ff6e0 con 0x5637ec818970 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52256eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5637ec75c800 con 0x5637ec80dac0 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec80d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec80d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58388/0 (socket says 172.21.15.90:58388) 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 -- 172.21.15.90:0/4291283053 learned_addr learned my addr 172.21.15.90:0/4291283053 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5213fff700 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec818430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 -- 172.21.15.90:0/4291283053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 msgr2=0x5637ec818430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52193f9700 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec81ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec818430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 -- 172.21.15.90:0/4291283053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 msgr2=0x5637ec81ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec81ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 -- 172.21.15.90:0/4291283053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ec6ad9d0 con 0x5637ec80d1a0 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f52193f9700 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec81ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.743+0000 7f5218bf8700 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec80d580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f52080044e0 tx=0x7f5208012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f6c187dc5ee159 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f51feffd700 1 -- 172.21.15.90:0/4291283053 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208003880 con 0x5637ec80d1a0 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f51feffd700 1 -- 172.21.15.90:0/4291283053 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5208024070 con 0x5637ec80d1a0 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f51feffd700 1 -- 172.21.15.90:0/4291283053 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520801fc40 con 0x5637ec80d1a0 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f52256eeb80 1 -- 172.21.15.90:0/4291283053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 msgr2=0x5637ec80d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec80d580 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f52080044e0 tx=0x7f5208012000 comp rx=0 tx=0).stop 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f52256eeb80 1 -- 172.21.15.90:0/4291283053 shutdown_connections 2024-04-26T18:16:35.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec80d580 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec81ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4291283053 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec818430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.744+0000 7f52256eeb80 1 -- 172.21.15.90:0/4291283053 >> 172.21.15.90:0/4291283053 conn(0x5637ec73ae40 msgr2=0x5637ec809690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 -- 172.21.15.90:0/4291283053 shutdown_connections 2024-04-26T18:16:35.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 -- 172.21.15.90:0/4291283053 wait complete. 2024-04-26T18:16:35.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 -- start start 2024-04-26T18:16:35.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 Processor -- start 2024-04-26T18:16:35.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec818150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec80dac0 0x5637ec815450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec818970 0x5637ec815990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5637ec81b540 con 0x5637ec80d1a0 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5637ec6c6a50 con 0x5637ec80dac0 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5637ec81b6c0 con 0x5637ec818970 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f52256eeb80 10 client.0 _ll_register_callbacks cb 0x5637ec6f4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f5218bf8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec818150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.745+0000 7f5218bf8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec818150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58392/0 (socket says 172.21.15.90:58392) 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5218bf8700 1 -- 172.21.15.90:0/1518834674 learned_addr learned my addr 172.21.15.90:0/1518834674 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f52193f9700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec818970 0x5637ec815990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5213fff700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec80dac0 0x5637ec815450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5218bf8700 1 -- 172.21.15.90:0/1518834674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec818970 msgr2=0x5637ec815990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5218bf8700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec818970 0x5637ec815990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5218bf8700 1 -- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec80dac0 msgr2=0x5637ec815450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5218bf8700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec80dac0 0x5637ec815450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5218bf8700 1 -- 172.21.15.90:0/1518834674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ec8160a0 con 0x5637ec80d1a0 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f52193f9700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec818970 0x5637ec815990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5213fff700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec80dac0 0x5637ec815450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f5218bf8700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec818150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5208004110 tx=0x7f5208020b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ada4ffb5a0a2494 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f51f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f51f3fff700 1 -- 172.21.15.90:0/1518834674 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208003880 con 0x5637ec80d1a0 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f51f3fff700 1 -- 172.21.15.90:0/1518834674 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f5208003a20 con 0x5637ec80d1a0 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5208024070 con 0x5637ec80d1a0 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f51f3fff700 1 -- 172.21.15.90:0/1518834674 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5208004510 con 0x5637ec80d1a0 2024-04-26T18:16:35.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f52256eeb80 10 client.15807 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637ec825f90 con 0x5637ec80d1a0 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.746+0000 7f51f3fff700 1 -- 172.21.15.90:0/1518834674 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5208024070 con 0x5637ec80d1a0 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f51f3fff700 1 -- 172.21.15.90:0/1518834674 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f520803c020 con 0x5637ec80d1a0 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f52256eeb80 10 client.15807 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f52256eeb80 10 client.15807 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f52256eeb80 20 client.15807 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f52256eeb80 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec828460 0x5637ec82a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f52256eeb80 4 client.15807 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637ec76e220 con 0x5637ec828460 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f5213fff700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec828460 0x5637ec82a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f5213fff700 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec828460 0x5637ec82a850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927c4ff974de2c6c server_cookie=655127aed0c5741c in_seq=0 out_seq=0 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.747+0000 7f51f3fff700 10 client.15807 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:35.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f51f3fff700 1 -- 172.21.15.90:0/1518834674 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5637ec76e220 con 0x5637ec828460 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f51f3fff700 10 client.15807 handle_command_reply: tid=0 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 client.15807 shutdown 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 msgr2=0x5637ec818150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec818150 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5208004110 tx=0x7f5208020b90 comp rx=0 tx=0).stop 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 shutdown_connections 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec828460 0x5637ec82a850 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec818150 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec818970 0x5637ec815990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 --2- 172.21.15.90:0/1518834674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec80dac0 0x5637ec815450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f51f3fff700 0 client.15807 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 >> 172.21.15.90:0/1518834674 conn(0x5637ec73ae40 msgr2=0x5637ec8092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 shutdown_connections 2024-04-26T18:16:35.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 1 -- 172.21.15.90:0/1518834674 wait complete. 2024-04-26T18:16:35.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.748+0000 7f52256eeb80 20 client.15807 trim_cache size 0 max 16384 2024-04-26T18:16:35.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.767+0000 7f52256eeb80 1 Processor -- start 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.767+0000 7f52256eeb80 1 -- start start 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.767+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80d1a0 0x5637ec820d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80dac0 0x5637ec810970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec824440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f52256eeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52141b0340 con 0x5637ec80dac0 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f52256eeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52141adbd0 con 0x5637ec818970 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f52256eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f52141aba80 con 0x5637ec80d1a0 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80dac0 0x5637ec810970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80dac0 0x5637ec810970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58406/0 (socket says 172.21.15.90:58406) 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 -- 172.21.15.90:0/4067444235 learned_addr learned my addr 172.21.15.90:0/4067444235 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5218bf8700 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80d1a0 0x5637ec820d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f52193f9700 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec824440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 -- 172.21.15.90:0/4067444235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80d1a0 msgr2=0x5637ec820d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80d1a0 0x5637ec820d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 -- 172.21.15.90:0/4067444235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 msgr2=0x5637ec824440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec824440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 -- 172.21.15.90:0/4067444235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637ec80a970 con 0x5637ec80dac0 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5218bf8700 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80d1a0 0x5637ec820d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f5213fff700 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80dac0 0x5637ec810970 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5637ec82af80 tx=0x7f5200010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793cf9583f3a21b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f51fe7fc700 1 -- 172.21.15.90:0/4067444235 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5200020020 con 0x5637ec80dac0 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f51fe7fc700 1 -- 172.21.15.90:0/4067444235 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5200010e10 con 0x5637ec80dac0 2024-04-26T18:16:35.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.768+0000 7f51fe7fc700 1 -- 172.21.15.90:0/4067444235 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f520001bca0 con 0x5637ec80dac0 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 -- 172.21.15.90:0/4067444235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80dac0 msgr2=0x5637ec810970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80dac0 0x5637ec810970 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5637ec82af80 tx=0x7f5200010710 comp rx=0 tx=0).stop 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 -- 172.21.15.90:0/4067444235 shutdown_connections 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec824440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80d1a0 0x5637ec820d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 --2- 172.21.15.90:0/4067444235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80dac0 0x5637ec810970 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 -- 172.21.15.90:0/4067444235 >> 172.21.15.90:0/4067444235 conn(0x5637ec73ae40 msgr2=0x5637ec810260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 -- 172.21.15.90:0/4067444235 shutdown_connections 2024-04-26T18:16:35.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 -- 172.21.15.90:0/4067444235 wait complete. 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 -- start start 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.769+0000 7f52256eeb80 1 Processor -- start 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec88bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec88c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52256eeb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec886810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52256eeb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f52141ac0d0 con 0x5637ec80d1a0 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52256eeb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f52141ad500 con 0x5637ec818970 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52256eeb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f52141ac760 con 0x5637ec80dac0 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52256eeb80 10 client.0 _ll_register_callbacks cb 0x5637ec6f4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec88bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec88bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58420/0 (socket says 172.21.15.90:58420) 2024-04-26T18:16:35.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 -- 172.21.15.90:0/924898619 learned_addr learned my addr 172.21.15.90:0/924898619 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5213fff700 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec88c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f52193f9700 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec886810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 -- 172.21.15.90:0/924898619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 msgr2=0x5637ec88c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec88c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 -- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 msgr2=0x5637ec886810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec886810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 -- 172.21.15.90:0/924898619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637ec887050 con 0x5637ec80d1a0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f5218bf8700 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec88bfd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f520803ecd0 tx=0x7f520800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0120b35611aa91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f51f3fff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.770+0000 7f51f3fff700 1 -- 172.21.15.90:0/924898619 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5208020450 con 0x5637ec80d1a0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5200010e10 con 0x5637ec80d1a0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f51f3fff700 1 -- 172.21.15.90:0/924898619 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f52080205f0 con 0x5637ec80d1a0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f51f3fff700 1 -- 172.21.15.90:0/924898619 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5208003680 con 0x5637ec80d1a0 2024-04-26T18:16:35.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 10 client.15813 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5637ec893740 con 0x5637ec80d1a0 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f51f3fff700 1 -- 172.21.15.90:0/924898619 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5200010e10 con 0x5637ec80d1a0 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f51f3fff700 1 -- 172.21.15.90:0/924898619 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f5208046070 con 0x5637ec80d1a0 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 10 client.15813 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 10 client.15813 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 20 client.15813 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec893d40 0x5637ec896160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 4 client.15813 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fb7a5d51", "release": true, "await": true}] 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fb7a5d51", "release": true, "await": true}) -- 0x5637ec7682c0 con 0x5637ec893d40 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.771+0000 7f5213fff700 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec893d40 0x5637ec896160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.772+0000 7f5213fff700 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec893d40 0x5637ec896160 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e1360665c1bb2f server_cookie=748ad1a239688043 in_seq=0 out_seq=0 2024-04-26T18:16:35.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.772+0000 7f51f3fff700 10 client.15813 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f51f3fff700 1 -- 172.21.15.90:0/924898619 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5637ec7682c0 con 0x5637ec893d40 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f51f3fff700 10 client.15813 handle_command_reply: tid=0 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 client.15813 shutdown 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 msgr2=0x5637ec88bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec88bfd0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f520803ecd0 tx=0x7f520800f770 comp rx=0 tx=0).stop 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 shutdown_connections 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5637ec893d40 0x5637ec896160 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5637ec80d1a0 0x5637ec88bfd0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5637ec818970 0x5637ec886810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 --2- 172.21.15.90:0/924898619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5637ec80dac0 0x5637ec88c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 >> 172.21.15.90:0/924898619 conn(0x5637ec73ae40 msgr2=0x5637ec810a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f51f3fff700 0 client.15813 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 shutdown_connections 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.782+0000 7f52256eeb80 1 -- 172.21.15.90:0/924898619 wait complete. 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f52256eeb80 20 client.15813 trim_cache size 0 max 16384 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 27, 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 12, 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:16:35.784 INFO:teuthology.orchestra.run.smithi090.stdout: "fb7a5d51": { 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 12, 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 79.5, 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:16:35.785 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:16:35.786 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:35.786 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:35.786 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:35.786 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:35.786 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:35.786 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f52040a3db0 msgr2=0x7f52040a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f52040a3db0 0x7f52040a6270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5208005e10 tx=0x7f5208005d60 comp rx=0 tx=0).stop 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141094c0 msgr2=0x7f5214199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141094c0 0x7f5214199840 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f520c00b830 tx=0x7f520c004d90 comp rx=0 tx=0).stop 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 shutdown_connections 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f52040a3db0 0x7f52040a6270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52141094c0 0x7f5214199840 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5214103470 0x7f521419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 --2- 172.21.15.90:0/1895917350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5214103e30 0x7f521419fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 >> 172.21.15.90:0/1895917350 conn(0x7f521407a580 msgr2=0x7f5214101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 shutdown_connections 2024-04-26T18:16:35.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:35.783+0000 7f521ae5c700 1 -- 172.21.15.90:0/1895917350 wait complete. 2024-04-26T18:16:35.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 -- 172.21.15.90:0/2519648335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd80104520 msgr2=0x7fdd8010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 --2- 172.21.15.90:0/2519648335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd80104520 0x7fdd8010ca00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009900 tx=0x7fdd6802a480 comp rx=0 tx=0).stop 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 -- 172.21.15.90:0/2519648335 shutdown_connections 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 --2- 172.21.15.90:0/2519648335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd8010cf40 0x7fdd8010f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 --2- 172.21.15.90:0/2519648335 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd80104520 0x7fdd8010ca00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 --2- 172.21.15.90:0/2519648335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd80103c00 0x7fdd80103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 -- 172.21.15.90:0/2519648335 >> 172.21.15.90:0/2519648335 conn(0x7fdd800789b0 msgr2=0x7fdd80078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 -- 172.21.15.90:0/2519648335 shutdown_connections 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.958+0000 7fdd856e3640 1 -- 172.21.15.90:0/2519648335 wait complete. 2024-04-26T18:16:35.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 Processor -- start 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 -- start start 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd80103c00 0x7fdd801990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80104520 0x7fdd80199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd8010cf40 0x7fdd8019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80112ba0 con 0x7fdd8010cf40 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80112a20 con 0x7fdd80104520 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd856e3640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80112d20 con 0x7fdd80103c00 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd8010cf40 0x7fdd8019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd767fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80104520 0x7fdd80199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd8010cf40 0x7fdd8019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58446/0 (socket says 172.21.15.90:58446) 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/3643456764 learned_addr learned my addr 172.21.15.90:0/3643456764 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.959+0000 7fdd7effd640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd80103c00 0x7fdd801990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/3643456764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd80103c00 msgr2=0x7fdd801990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd80103c00 0x7fdd801990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/3643456764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80104520 msgr2=0x7fdd80199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80104520 0x7fdd80199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/3643456764 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8019e0d0 con 0x7fdd8010cf40 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd8010cf40 0x7fdd8019d9a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800e970 tx=0x7fdd7800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7cff9640 1 -- 172.21.15.90:0/3643456764 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800ccc0 con 0x7fdd8010cf40 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8019e3c0 con 0x7fdd8010cf40 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7cff9640 1 -- 172.21.15.90:0/3643456764 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd7800ce60 con 0x7fdd8010cf40 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd7cff9640 1 -- 172.21.15.90:0/3643456764 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78010430 con 0x7fdd8010cf40 2024-04-26T18:16:35.963 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.960+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801a5c50 con 0x7fdd8010cf40 2024-04-26T18:16:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.962+0000 7fdd7cff9640 1 -- 172.21.15.90:0/3643456764 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fdd78004ff0 con 0x7fdd8010cf40 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 Processor -- start 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 -- start start 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd7cff9640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd6c0a3dc0 0x7fdd6c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd7cff9640 1 -- 172.21.15.90:0/3643456764 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdd780d0b40 con 0x7fdd8010cf40 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd767fc640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd6c0a3dc0 0x7fdd6c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eae500b460 con 0x55eae50a4de0 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eae4fc5590 con 0x55eae50a5700 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd883fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eae5007420 con 0x55eae50b44f0 2024-04-26T18:16:35.972 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58462/0 (socket says 172.21.15.90:58462) 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd7effd640 1 -- 172.21.15.90:0/3781704872 learned_addr learned my addr 172.21.15.90:0/3781704872 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.963+0000 7fdd767fc640 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd7effd640 1 -- 172.21.15.90:0/3781704872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 msgr2=0x55eae50b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd7effd640 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd7effd640 1 -- 172.21.15.90:0/3781704872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 msgr2=0x55eae50b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd7effd640 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd7effd640 1 -- 172.21.15.90:0/3781704872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae4fd20d0 con 0x55eae50a4de0 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd767fc640 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd767fc640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd6c0a3dc0 0x7fdd6c0a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd68031440 tx=0x7fdd68032c30 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd7effd640 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50a51c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdd700114c0 tx=0x7fdd70011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648946394e9f26d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd757fa640 1 -- 172.21.15.90:0/3781704872 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd700112f0 con 0x55eae50a4de0 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd757fa640 1 -- 172.21.15.90:0/3781704872 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70003e20 con 0x55eae50a4de0 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.964+0000 7fdd757fa640 1 -- 172.21.15.90:0/3781704872 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000f820 con 0x55eae50a4de0 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 -- 172.21.15.90:0/3781704872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 msgr2=0x55eae50a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50a51c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fdd700114c0 tx=0x7fdd70011980 comp rx=0 tx=0).stop 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 -- 172.21.15.90:0/3781704872 shutdown_connections 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 --2- 172.21.15.90:0/3781704872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50a51c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 -- 172.21.15.90:0/3781704872 >> 172.21.15.90:0/3781704872 conn(0x55eae50a1410 msgr2=0x55eae50a17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 -- 172.21.15.90:0/3781704872 shutdown_connections 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 -- 172.21.15.90:0/3781704872 wait complete. 2024-04-26T18:16:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 -- start start 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 Processor -- start 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a4de0 0x55eae50b1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a5700 0x55eae50b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.965+0000 7fdd883fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55eae50b7120 con 0x55eae50a5700 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd883fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55eae4f92b60 con 0x55eae50a4de0 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd883fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eae50b72a0 con 0x55eae50b44f0 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd883fd740 10 client.0 _ll_register_callbacks cb 0x55eae4fb72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd767fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a5700 0x55eae50b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a4de0 0x55eae50b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd767fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a5700 0x55eae50b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58478/0 (socket says 172.21.15.90:58478) 2024-04-26T18:16:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd767fc640 1 -- 172.21.15.90:0/4219482462 learned_addr learned my addr 172.21.15.90:0/4219482462 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7effd640 1 -- 172.21.15.90:0/4219482462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 msgr2=0x55eae50b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7effd640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7effd640 1 -- 172.21.15.90:0/4219482462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a5700 msgr2=0x55eae50b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7effd640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a5700 0x55eae50b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7effd640 1 -- 172.21.15.90:0/4219482462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae4d1d210 con 0x55eae50a4de0 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd767fc640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a5700 0x55eae50b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd7effd640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a4de0 0x55eae50b1410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd700114c0 tx=0x7fdd70004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6f9806cf0daa591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd667fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd667fc640 1 -- 172.21.15.90:0/4219482462 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70003b80 con 0x55eae50a4de0 2024-04-26T18:16:35.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd667fc640 1 -- 172.21.15.90:0/4219482462 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdd70004050 con 0x55eae50a4de0 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd70003e20 con 0x55eae50a4de0 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd667fc640 1 -- 172.21.15.90:0/4219482462 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd70030050 con 0x55eae50a4de0 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd883fd740 10 client.25346 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.966+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eae4de9620 con 0x55eae50a4de0 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd667fc640 1 -- 172.21.15.90:0/4219482462 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70003e20 con 0x55eae50a4de0 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd667fc640 1 -- 172.21.15.90:0/4219482462 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdd70018070 con 0x55eae50a4de0 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd883fd740 10 client.25346 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd883fd740 10 client.25346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd883fd740 20 client.25346 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd883fd740 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae50c4570 0x55eae50c6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd883fd740 4 client.25346 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eae5019070 con 0x55eae50c4570 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd767fc640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae50c4570 0x55eae50c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd667fc640 10 client.25346 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.967+0000 7fdd767fc640 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae50c4570 0x55eae50c6960 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9e1c3e222a41c1 server_cookie=2388e9f0554a171f in_seq=0 out_seq=0 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd667fc640 1 -- 172.21.15.90:0/4219482462 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eae5019070 con 0x55eae50c4570 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd667fc640 10 client.25346 handle_command_reply: tid=0 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 client.25346 shutdown 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a4de0 msgr2=0x55eae50b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a4de0 0x55eae50b1410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdd700114c0 tx=0x7fdd70004860 comp rx=0 tx=0).stop 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 shutdown_connections 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae50c4570 0x55eae50c6960 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd667fc640 0 client.25346 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a5700 0x55eae50b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 --2- 172.21.15.90:0/4219482462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a4de0 0x55eae50b1410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 >> 172.21.15.90:0/4219482462 conn(0x55eae50a1410 msgr2=0x55eae50a7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 shutdown_connections 2024-04-26T18:16:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 1 -- 172.21.15.90:0/4219482462 wait complete. 2024-04-26T18:16:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.968+0000 7fdd883fd740 20 client.25346 trim_cache size 0 max 16384 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.984+0000 7fdd883fd740 1 Processor -- start 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.984+0000 7fdd883fd740 1 -- start start 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50a0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd883fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801aea50 con 0x55eae50a4de0 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd883fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801ac320 con 0x55eae50a5700 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd883fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801aa190 con 0x55eae50b44f0 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd767fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50c0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58482/0 (socket says 172.21.15.90:58482) 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 -- 172.21.15.90:0/2519815993 learned_addr learned my addr 172.21.15.90:0/2519815993 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 -- 172.21.15.90:0/2519815993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 msgr2=0x55eae50cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50cc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 -- 172.21.15.90:0/2519815993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 msgr2=0x55eae50a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 -- 172.21.15.90:0/2519815993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae50a0e30 con 0x55eae50a4de0 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd767fc640 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd7effd640 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50c0050 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdd70011f00 tx=0x7fdd70005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58fcd03f17d062c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.987 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.985+0000 7fdd74ff9640 1 -- 172.21.15.90:0/2519815993 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70009b30 con 0x55eae50a4de0 2024-04-26T18:16:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd74ff9640 1 -- 172.21.15.90:0/2519815993 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70039070 con 0x55eae50a4de0 2024-04-26T18:16:35.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd74ff9640 1 -- 172.21.15.90:0/2519815993 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd700229c0 con 0x55eae50a4de0 2024-04-26T18:16:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 -- 172.21.15.90:0/2519815993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 msgr2=0x55eae50c0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:35.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50c0050 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdd70011f00 tx=0x7fdd70005960 comp rx=0 tx=0).stop 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 -- 172.21.15.90:0/2519815993 shutdown_connections 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50b44f0 0x55eae50cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 --2- 172.21.15.90:0/2519815993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50a4de0 0x55eae50c0050 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 -- 172.21.15.90:0/2519815993 >> 172.21.15.90:0/2519815993 conn(0x55eae50a1410 msgr2=0x55eae509fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 -- 172.21.15.90:0/2519815993 shutdown_connections 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.986+0000 7fdd883fd740 1 -- 172.21.15.90:0/2519815993 wait complete. 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 -- start start 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 Processor -- start 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50a4de0 0x55eae50bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.992 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50b44f0 0x55eae50b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801aa7e0 con 0x55eae50b44f0 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801abc50 con 0x55eae50a5700 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801aaeb0 con 0x55eae50a4de0 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd883fd740 10 client.0 _ll_register_callbacks cb 0x7fdd78098df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50b44f0 0x55eae50b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50b44f0 0x55eae50b80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58496/0 (socket says 172.21.15.90:58496) 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/1566225676 learned_addr learned my addr 172.21.15.90:0/1566225676 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7effd640 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50a4de0 0x55eae50bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd767fc640 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/1566225676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50a4de0 msgr2=0x55eae50bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50a4de0 0x55eae50bc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 msgr2=0x55eae50b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:35.993 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 -- 172.21.15.90:0/1566225676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae50a1240 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd7f7fe640 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50b44f0 0x55eae50b80c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800e940 tx=0x7fdd780058f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c996b443e3d246ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.987+0000 7fdd667fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd667fc640 1 -- 172.21.15.90:0/1566225676 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd780b5020 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd667fc640 1 -- 172.21.15.90:0/1566225676 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdd780991c0 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd70039070 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd667fc640 1 -- 172.21.15.90:0/1566225676 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd78080050 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 10 client.15831 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eae512a870 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd667fc640 1 -- 172.21.15.90:0/1566225676 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70039070 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd667fc640 1 -- 172.21.15.90:0/1566225676 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fdd780988a0 con 0x55eae50b44f0 2024-04-26T18:16:35.994 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 10 client.15831 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 20 client.15831 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae512aff0 0x55eae512d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 4 client.15831 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eae5012ee0 con 0x55eae512aff0 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.988+0000 7fdd7effd640 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae512aff0 0x55eae512d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.989+0000 7fdd667fc640 10 client.15831 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:35.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:35.989+0000 7fdd7effd640 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae512aff0 0x55eae512d410 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f6150d05e7d7634 server_cookie=925ce673d8ca5b35 in_seq=0 out_seq=0 2024-04-26T18:16:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd667fc640 1 -- 172.21.15.90:0/1566225676 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eae5012ee0 con 0x55eae512aff0 2024-04-26T18:16:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd667fc640 10 client.15831 handle_command_reply: tid=0 2024-04-26T18:16:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 client.15831 shutdown 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50b44f0 msgr2=0x55eae50b80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50b44f0 0x55eae50b80c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800e940 tx=0x7fdd780058f0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 shutdown_connections 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55eae512aff0 0x55eae512d410 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55eae50b44f0 0x55eae50b80c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55eae50a5700 0x55eae50b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 --2- 172.21.15.90:0/1566225676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eae50a4de0 0x55eae50bc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 >> 172.21.15.90:0/1566225676 conn(0x55eae50a1410 msgr2=0x55eae50a05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.014+0000 7fdd667fc640 0 client.15831 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 shutdown_connections 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd883fd740 1 -- 172.21.15.90:0/1566225676 wait complete. 2024-04-26T18:16:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd883fd740 20 client.15831 trim_cache size 0 max 16384 2024-04-26T18:16:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:16:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:16:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:16:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd6c0a3dc0 msgr2=0x7fdd6c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd6c0a3dc0 0x7fdd6c0a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fdd68031440 tx=0x7fdd68032c30 comp rx=0 tx=0).stop 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd8010cf40 msgr2=0x7fdd8019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd8010cf40 0x7fdd8019d9a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800e970 tx=0x7fdd7800ed70 comp rx=0 tx=0).stop 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 shutdown_connections 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd6c0a3dc0 0x7fdd6c0a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd8010cf40 0x7fdd8019d9a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80104520 0x7fdd80199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 --2- 172.21.15.90:0/3643456764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd80103c00 0x7fdd801990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:36.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 >> 172.21.15.90:0/3643456764 conn(0x7fdd800789b0 msgr2=0x7fdd800fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 shutdown_connections 2024-04-26T18:16:36.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:36.015+0000 7fdd856e3640 1 -- 172.21.15.90:0/3643456764 wait complete. 2024-04-26T18:16:36.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 234732 0.30 MB/sec execute 24 sec latency 25374.927 ms 2024-04-26T18:16:36.726 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fb7a5d51', seconds elapsed: 31.9 2024-04-26T18:16:36.726 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.5 2024-04-26T18:16:36.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:16:36.983 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:36 smithi167 ceph-mon[32751]: pgmap v142: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-04-26T18:16:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:36 smithi090 ceph-mon[29180]: pgmap v142: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-04-26T18:16:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:36 smithi119 ceph-mon[32752]: pgmap v142: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-04-26T18:16:37.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 237157 0.79 MB/sec execute 25 sec latency 43.639 ms 2024-04-26T18:16:38.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 238523 1.03 MB/sec execute 26 sec latency 43.053 ms 2024-04-26T18:16:38.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 -- 172.21.15.90:0/1350876258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e30101a10 msgr2=0x7f3e30101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/1350876258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e30101a10 0x7f3e30101e70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009990 tx=0x7f3e24009ca0 comp rx=0 tx=0).stop 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 -- 172.21.15.90:0/1350876258 shutdown_connections 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/1350876258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e301023b0 0x7f3e301102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/1350876258 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e301094b0 0x7f3e30109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/1350876258 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e30101a10 0x7f3e30101e70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 -- 172.21.15.90:0/1350876258 >> 172.21.15.90:0/1350876258 conn(0x7f3e30078990 msgr2=0x7f3e300ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 -- 172.21.15.90:0/1350876258 shutdown_connections 2024-04-26T18:16:38.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.794+0000 7f3e35b1b700 1 -- 172.21.15.90:0/1350876258 wait complete. 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 Processor -- start 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 -- start start 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e30101a10 0x7f3e30102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e301023b0 0x7f3e30103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e301094b0 0x7f3e30103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30113610 con 0x7f3e301094b0 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30113490 con 0x7f3e30101a10 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e35b1b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30113790 con 0x7f3e301023b0 2024-04-26T18:16:38.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e2ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e301094b0 0x7f3e30103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e2ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e301094b0 0x7f3e30103670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58506/0 (socket says 172.21.15.90:58506) 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e2effd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e301023b0 0x7f3e30103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e2ffff700 1 -- 172.21.15.90:0/3517747659 learned_addr learned my addr 172.21.15.90:0/3517747659 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.795+0000 7f3e2f7fe700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e30101a10 0x7f3e30102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2effd700 1 -- 172.21.15.90:0/3517747659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e30101a10 msgr2=0x7f3e30102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2effd700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e30101a10 0x7f3e30102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2effd700 1 -- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e301094b0 msgr2=0x7f3e30103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2effd700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e301094b0 0x7f3e30103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2effd700 1 -- 172.21.15.90:0/3517747659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e301a80b0 con 0x7f3e301023b0 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2effd700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e301023b0 0x7f3e30103130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e24000c00 tx=0x7f3e240047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2cff9700 1 -- 172.21.15.90:0/3517747659 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24026070 con 0x7f3e301023b0 2024-04-26T18:16:38.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e301a8340 con 0x7f3e301023b0 2024-04-26T18:16:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2cff9700 1 -- 172.21.15.90:0/3517747659 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e24021410 con 0x7f3e301023b0 2024-04-26T18:16:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e2cff9700 1 -- 172.21.15.90:0/3517747659 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e24017650 con 0x7f3e301023b0 2024-04-26T18:16:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.796+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e301a87f0 con 0x7f3e301023b0 2024-04-26T18:16:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.797+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e30104d30 con 0x7f3e301023b0 2024-04-26T18:16:38.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.798+0000 7f3e2cff9700 1 -- 172.21.15.90:0/3517747659 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f3e24029070 con 0x7f3e301023b0 2024-04-26T18:16:38.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.799+0000 7f3e2cff9700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3e180a3db0 0x7f3e180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:38.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.799+0000 7f3e2f7fe700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3e180a3db0 0x7f3e180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:38.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.799+0000 7f3e2cff9700 1 -- 172.21.15.90:0/3517747659 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f3e240d1b70 con 0x7f3e301023b0 2024-04-26T18:16:38.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.799+0000 7f3e2f7fe700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3e180a3db0 0x7f3e180a6270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009be0 tx=0x7f3e20009260 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:38.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:38.803+0000 7f3e2cff9700 1 -- 172.21.15.90:0/3517747659 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e24030070 con 0x7f3e301023b0 2024-04-26T18:16:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:38 smithi167 ceph-mon[32751]: pgmap v143: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-04-26T18:16:39.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.076+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e3002e490 con 0x7f3e301023b0 2024-04-26T18:16:39.077 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:38 smithi090 ceph-mon[29180]: pgmap v143: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-04-26T18:16:39.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.077+0000 7f3e2cff9700 1 -- 172.21.15.90:0/3517747659 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11877 (secure 0 0 0) 0x7f3e24013070 con 0x7f3e301023b0 2024-04-26T18:16:39.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:16:39.078 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:14:24.663594+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24449,24436,24448]},"id":1}]} 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.080+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3e180a3db0 msgr2=0x7f3e180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.080+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3e180a3db0 0x7f3e180a6270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3e20009be0 tx=0x7f3e20009260 comp rx=0 tx=0).stop 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.080+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e301023b0 msgr2=0x7f3e30103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.080+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e301023b0 0x7f3e30103130 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3e24000c00 tx=0x7f3e240047b0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 shutdown_connections 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3e180a3db0 0x7f3e180a6270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3e301094b0 0x7f3e30103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e30101a10 0x7f3e30102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 --2- 172.21.15.90:0/3517747659 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e301023b0 0x7f3e30103130 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 >> 172.21.15.90:0/3517747659 conn(0x7f3e30078990 msgr2=0x7f3e30106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 shutdown_connections 2024-04-26T18:16:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:39.081+0000 7f3e35b1b700 1 -- 172.21.15.90:0/3517747659 wait complete. 2024-04-26T18:16:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-26T18:16:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:38 smithi119 ceph-mon[32752]: pgmap v143: 97 pgs: 97 active+clean; 72 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 1.4 MiB/s wr, 61 op/s 2024-04-26T18:16:39.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 241466 1.67 MB/sec execute 27 sec latency 13.015 ms 2024-04-26T18:16:39.877 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0' 2024-04-26T18:16:39.877 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0 2024-04-26T18:16:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:39 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3517747659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:39 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3517747659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:40.137 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:16:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:39 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3517747659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:16:40.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 243066 1.82 MB/sec execute 28 sec latency 15.868 ms 2024-04-26T18:16:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:40 smithi167 ceph-mon[32751]: pgmap v144: 97 pgs: 97 active+clean; 67 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-26T18:16:41.073 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:40 smithi090 ceph-mon[29180]: pgmap v144: 97 pgs: 97 active+clean; 67 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-26T18:16:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:40 smithi119 ceph-mon[32752]: pgmap v144: 97 pgs: 97 active+clean; 67 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 162 op/s 2024-04-26T18:16:41.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 245343 2.23 MB/sec execute 29 sec latency 10.148 ms 2024-04-26T18:16:42.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 -- 172.21.15.90:0/109525183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 msgr2=0x7f9370109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 --2- 172.21.15.90:0/109525183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370109880 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f936c0099c0 tx=0x7f936c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 -- 172.21.15.90:0/109525183 shutdown_connections 2024-04-26T18:16:42.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 --2- 172.21.15.90:0/109525183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370109880 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 --2- 172.21.15.90:0/109525183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9370103e90 0x7f93701102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 --2- 172.21.15.90:0/109525183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93701034d0 0x7f9370103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 -- 172.21.15.90:0/109525183 >> 172.21.15.90:0/109525183 conn(0x7f937007a4f0 msgr2=0x7f937007a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 -- 172.21.15.90:0/109525183 shutdown_connections 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.118+0000 7f937845a700 1 -- 172.21.15.90:0/109525183 wait complete. 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 Processor -- start 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 -- start start 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93701034d0 0x7f937019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9370103e90 0x7f937019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f9370113630 con 0x7f93701094a0 2024-04-26T18:16:42.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93701134b0 con 0x7f9370103e90 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f937845a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93701137b0 con 0x7f93701034d0 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93759f5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9370103e90 0x7f937019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57980/0 (socket says 172.21.15.90:57980) 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93759f5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9370103e90 0x7f937019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38672/0 (socket says 172.21.15.90:38672) 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 -- 172.21.15.90:0/1311490743 learned_addr learned my addr 172.21.15.90:0/1311490743 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93761f6700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93701034d0 0x7f937019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 -- 172.21.15.90:0/1311490743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93701034d0 msgr2=0x7f937019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93701034d0 0x7f937019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 -- 172.21.15.90:0/1311490743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9370103e90 msgr2=0x7f937019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9370103e90 0x7f937019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 -- 172.21.15.90:0/1311490743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937019a070 con 0x7f93701094a0 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93759f5700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9370103e90 0x7f937019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.119+0000 7f93769f7700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370199910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f936800ef30 tx=0x7f936800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.120+0000 7f93677fe700 1 -- 172.21.15.90:0/1311490743 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93680092e0 con 0x7f93701094a0 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.120+0000 7f93677fe700 1 -- 172.21.15.90:0/1311490743 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9368004bb0 con 0x7f93701094a0 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.120+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937019a300 con 0x7f93701094a0 2024-04-26T18:16:42.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.120+0000 7f93677fe700 1 -- 172.21.15.90:0/1311490743 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368004280 con 0x7f93701094a0 2024-04-26T18:16:42.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.120+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93701a6620 con 0x7f93701094a0 2024-04-26T18:16:42.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.122+0000 7f93677fe700 1 -- 172.21.15.90:0/1311490743 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f936801b030 con 0x7f93701094a0 2024-04-26T18:16:42.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.122+0000 7f93677fe700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f935c0a3ed0 0x7f935c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.122+0000 7f93677fe700 1 -- 172.21.15.90:0/1311490743 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f93680d01c0 con 0x7f93701094a0 2024-04-26T18:16:42.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.122+0000 7f93761f6700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f935c0a3ed0 0x7f935c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.123+0000 7f93761f6700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f935c0a3ed0 0x7f935c0a6390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f936c009990 tx=0x7f936c0058e0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:42.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.123+0000 7f9382cecb80 1 Processor -- start 2024-04-26T18:16:42.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f9382cecb80 1 -- start start 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002bf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f9382cecb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a002128d0 con 0x559a002caa00 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f9382cecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a001b1770 con 0x559a002bf230 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f9382cecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a0020e890 con 0x559a002bfb50 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93769f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93761f6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93759f5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93761f6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002bf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38674/0 (socket says 172.21.15.90:38674) 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93761f6700 1 -- 172.21.15.90:0/1887323774 learned_addr learned my addr 172.21.15.90:0/1887323774 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93759f5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:57768/0 (socket says 172.21.15.90:57768) 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93761f6700 1 -- 172.21.15.90:0/1887323774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 msgr2=0x559a002ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93761f6700 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93761f6700 1 -- 172.21.15.90:0/1887323774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 msgr2=0x559a002ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.124+0000 7f93761f6700 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.125+0000 7f93761f6700 1 -- 172.21.15.90:0/1887323774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0015fa60 con 0x559a002bf230 2024-04-26T18:16:42.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.125+0000 7f93769f7700 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.125+0000 7f93761f6700 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002bf610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f936c005950 tx=0x7f936c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c05eaec80f3e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.125+0000 7f9364d70700 1 -- 172.21.15.90:0/1887323774 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c004040 con 0x559a002bf230 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.125+0000 7f9364d70700 1 -- 172.21.15.90:0/1887323774 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c0037c0 con 0x559a002bf230 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.125+0000 7f9364d70700 1 -- 172.21.15.90:0/1887323774 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c004690 con 0x559a002bf230 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 -- 172.21.15.90:0/1887323774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 msgr2=0x559a002bf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002bf610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f936c005950 tx=0x7f936c01f000 comp rx=0 tx=0).stop 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 -- 172.21.15.90:0/1887323774 shutdown_connections 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002bf610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 --2- 172.21.15.90:0/1887323774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 -- 172.21.15.90:0/1887323774 >> 172.21.15.90:0/1887323774 conn(0x559a001eced0 msgr2=0x559a002bb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 -- 172.21.15.90:0/1887323774 shutdown_connections 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 -- 172.21.15.90:0/1887323774 wait complete. 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 -- start start 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.126+0000 7f9382cecb80 1 Processor -- start 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002bfb50 0x559a002c7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002caa00 0x559a002c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9382cecb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a002cd510 con 0x559a002bfb50 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9382cecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a00178ae0 con 0x559a002bf230 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9382cecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a002cd690 con 0x559a002caa00 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9382cecb80 10 client.0 _ll_register_callbacks cb 0x559a001a6710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93759f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002bfb50 0x559a002c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38680/0 (socket says 172.21.15.90:38680) 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 -- 172.21.15.90:0/2303820691 learned_addr learned my addr 172.21.15.90:0/2303820691 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:42.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93769f7700 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002caa00 0x559a002c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 -- 172.21.15.90:0/2303820691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002caa00 msgr2=0x559a002c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002caa00 0x559a002c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 -- 172.21.15.90:0/2303820691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002bfb50 msgr2=0x559a002c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002bfb50 0x559a002c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 -- 172.21.15.90:0/2303820691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a002c8090 con 0x559a002bf230 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f93761f6700 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002ca190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f936c009d10 tx=0x7f936c005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=558ed6b45329cc09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.127+0000 7f9359ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9359ffb700 1 -- 172.21.15.90:0/2303820691 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c045060 con 0x559a002bf230 2024-04-26T18:16:42.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9359ffb700 1 -- 172.21.15.90:0/2303820691 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f936c003b80 con 0x559a002bf230 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c0037c0 con 0x559a002bf230 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9359ffb700 1 -- 172.21.15.90:0/2303820691 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f936c003d80 con 0x559a002bf230 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 10 client.25370 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a002cd510 con 0x559a002bf230 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9359ffb700 1 -- 172.21.15.90:0/2303820691 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c005560 con 0x559a002bf230 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9359ffb700 1 -- 172.21.15.90:0/2303820691 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f936c048020 con 0x559a002bf230 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 10 client.25370 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 10 client.25370 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:16:42.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 20 client.25370 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a002dc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 4 client.25370 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f936c0037c0 con 0x559a002da2a0 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.128+0000 7f93759f5700 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a002dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.129+0000 7f9359ffb700 10 client.25370 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.129+0000 7f93759f5700 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a002dc690 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798a7ab27ed29403 server_cookie=399bd89ae6e4ac32 in_seq=0 out_seq=0 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.129+0000 7f9359ffb700 1 -- 172.21.15.90:0/2303820691 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f936c0037c0 con 0x559a002da2a0 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.129+0000 7f9359ffb700 10 client.25370 handle_command_reply: tid=0 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 client.25370 shutdown 2024-04-26T18:16:42.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 msgr2=0x559a002ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002ca190 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f936c009d10 tx=0x7f936c005a00 comp rx=0 tx=0).stop 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 shutdown_connections 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a002dc690 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9359ffb700 0 client.25370 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002caa00 0x559a002c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002ca190 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 --2- 172.21.15.90:0/2303820691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002bfb50 0x559a002c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 >> 172.21.15.90:0/2303820691 conn(0x559a001eced0 msgr2=0x559a002bb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:42.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 shutdown_connections 2024-04-26T18:16:42.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 1 -- 172.21.15.90:0/2303820691 wait complete. 2024-04-26T18:16:42.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.130+0000 7f9382cecb80 20 client.25370 trim_cache size 0 max 16384 2024-04-26T18:16:42.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 Processor -- start 2024-04-26T18:16:42.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 -- start start 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002e1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f93701ae1e0 con 0x559a002caa00 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93701aba70 con 0x559a002bf230 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f9382cecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93701a9900 con 0x559a002bfb50 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f93769f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002e1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f93759f5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f93769f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002e1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58008/0 (socket says 172.21.15.90:58008) 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f93769f7700 1 -- 172.21.15.90:0/506426303 learned_addr learned my addr 172.21.15.90:0/506426303 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.151+0000 7f93761f6700 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f93759f5700 1 -- 172.21.15.90:0/506426303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 msgr2=0x559a002c29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f93759f5700 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002c29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f93759f5700 1 -- 172.21.15.90:0/506426303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 msgr2=0x559a002e1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f93759f5700 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002e1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f93759f5700 1 -- 172.21.15.90:0/506426303 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a002c5a00 con 0x559a002bfb50 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f93769f7700 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:42.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f93759f5700 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002d6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x559a002dcc00 tx=0x7f9360009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a21a243c3c2101d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f935bfff700 1 -- 172.21.15.90:0/506426303 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360004250 con 0x559a002bfb50 2024-04-26T18:16:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f935bfff700 1 -- 172.21.15.90:0/506426303 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9360025030 con 0x559a002bfb50 2024-04-26T18:16:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.152+0000 7f935bfff700 1 -- 172.21.15.90:0/506426303 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600054f0 con 0x559a002bfb50 2024-04-26T18:16:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 -- 172.21.15.90:0/506426303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 msgr2=0x559a002d6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002d6410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x559a002dcc00 tx=0x7f9360009260 comp rx=0 tx=0).stop 2024-04-26T18:16:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 -- 172.21.15.90:0/506426303 shutdown_connections 2024-04-26T18:16:42.159 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002e1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002c29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 --2- 172.21.15.90:0/506426303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002d6410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 -- 172.21.15.90:0/506426303 >> 172.21.15.90:0/506426303 conn(0x559a001eced0 msgr2=0x559a002c22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:42.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 -- 172.21.15.90:0/506426303 shutdown_connections 2024-04-26T18:16:42.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 -- 172.21.15.90:0/506426303 wait complete. 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 -- start start 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 Processor -- start 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.153+0000 7f9382cecb80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9382cecb80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f93701a9f50 con 0x559a002caa00 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9382cecb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f93701ab3a0 con 0x559a002bf230 2024-04-26T18:16:42.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9382cecb80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93701aa600 con 0x559a002bfb50 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9382cecb80 10 client.0 _ll_register_callbacks cb 0x559a001a6710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93769f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93769f7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58020/0 (socket says 172.21.15.90:58020) 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93759f5700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93769f7700 1 -- 172.21.15.90:0/623417309 learned_addr learned my addr 172.21.15.90:0/623417309 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38700/0 (socket says 172.21.15.90:38700) 2024-04-26T18:16:42.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 -- 172.21.15.90:0/623417309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 msgr2=0x559a002cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 -- 172.21.15.90:0/623417309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 msgr2=0x559a002cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 -- 172.21.15.90:0/623417309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a002d0640 con 0x559a002bf230 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93769f7700 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f93761f6700 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002d2d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f936c006010 tx=0x7f936c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3627b847fac8d7a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9359ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9359ffb700 1 -- 172.21.15.90:0/623417309 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c035060 con 0x559a002bf230 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9359ffb700 1 -- 172.21.15.90:0/623417309 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f936c04a880 con 0x559a002bf230 2024-04-26T18:16:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.154+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9360025030 con 0x559a002bf230 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9359ffb700 1 -- 172.21.15.90:0/623417309 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f936c045040 con 0x559a002bf230 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 10 client.25376 fetch_fsmap learned FSMap version 27 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559a00341500 con 0x559a002bf230 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9359ffb700 1 -- 172.21.15.90:0/623417309 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9360025030 con 0x559a002bf230 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9359ffb700 1 -- 172.21.15.90:0/623417309 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f936c024580 con 0x559a002bf230 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 10 client.25376 fetch_fsmap finished waiting for FSMap version 27 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 10 client.25376 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 20 client.25376 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:42.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a00343c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 4 client.25376 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}] 2024-04-26T18:16:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}) -- 0x559a002202b0 con 0x559a002da2a0 2024-04-26T18:16:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.155+0000 7f93759f5700 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a00343c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.156+0000 7f93759f5700 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a00343c30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fc8dc3bf8c4ccc9 server_cookie=8daef9477e26020b in_seq=0 out_seq=0 2024-04-26T18:16:42.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.156+0000 7f9359ffb700 10 client.25376 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:42.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9359ffb700 1 -- 172.21.15.90:0/623417309 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x559a002202b0 con 0x559a002da2a0 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9359ffb700 10 client.25376 handle_command_reply: tid=0 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 client.25376 shutdown 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 msgr2=0x559a002d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002d2d40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f936c006010 tx=0x7f936c009310 comp rx=0 tx=0).stop 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 shutdown_connections 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a002da2a0 0x559a00343c30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a002caa00 0x559a002cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a002bf230 0x559a002d2d40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 --2- 172.21.15.90:0/623417309 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a002bfb50 0x559a002cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9359ffb700 0 client.25376 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 >> 172.21.15.90:0/623417309 conn(0x559a001eced0 msgr2=0x559a002c19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 shutdown_connections 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 1 -- 172.21.15.90:0/623417309 wait complete. 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.237+0000 7f9382cecb80 20 client.25376 trim_cache size 0 max 16384 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 27, 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 14, 2024-04-26T18:16:42.239 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "692c662e": { 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 14, 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 79.5, 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:16:42.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "QUIESCED", 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:16:42.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f935c0a3ed0 msgr2=0x7f935c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f935c0a3ed0 0x7f935c0a6390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f936c009990 tx=0x7f936c0058e0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 msgr2=0x7f9370199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370199910 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f936800ef30 tx=0x7f936800c3a0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 shutdown_connections 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f935c0a3ed0 0x7f935c0a6390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f93701094a0 0x7f9370199910 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93701034d0 0x7f937019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 --2- 172.21.15.90:0/1311490743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9370103e90 0x7f937019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 >> 172.21.15.90:0/1311490743 conn(0x7f937007a4f0 msgr2=0x7f9370102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 shutdown_connections 2024-04-26T18:16:42.242 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:16:42.238+0000 7f937845a700 1 -- 172.21.15.90:0/1311490743 wait complete. 2024-04-26T18:16:42.666 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 2.32 MB/sec execute 30 sec latency 439.187 ms 2024-04-26T18:16:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:42 smithi167 ceph-mon[32751]: pgmap v145: 97 pgs: 97 active+clean; 67 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 131 op/s 2024-04-26T18:16:43.097 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:42 smithi090 ceph-mon[29180]: pgmap v145: 97 pgs: 97 active+clean; 67 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 131 op/s 2024-04-26T18:16:43.126 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '692c662e', quiesce took 6.4 seconds. Will release after: 19.5 2024-04-26T18:16:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:42 smithi119 ceph-mon[32752]: pgmap v145: 97 pgs: 97 active+clean; 67 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 131 op/s 2024-04-26T18:16:43.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 2.25 MB/sec execute 31 sec latency 1439.282 ms 2024-04-26T18:16:44.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 2.18 MB/sec execute 32 sec latency 2439.327 ms 2024-04-26T18:16:45.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:44 smithi119 ceph-mon[32752]: pgmap v146: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-04-26T18:16:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:44 smithi090 ceph-mon[29180]: pgmap v146: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-04-26T18:16:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:44 smithi167 ceph-mon[32751]: pgmap v146: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-04-26T18:16:45.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 2.11 MB/sec execute 33 sec latency 3439.855 ms 2024-04-26T18:16:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:16:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:16:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:16:46.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 2.05 MB/sec execute 34 sec latency 4439.976 ms 2024-04-26T18:16:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:46 smithi090 ceph-mon[29180]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-04-26T18:16:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:46 smithi119 ceph-mon[32752]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-04-26T18:16:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:46 smithi167 ceph-mon[32751]: pgmap v147: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-04-26T18:16:47.667 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.99 MB/sec execute 35 sec latency 5440.073 ms 2024-04-26T18:16:48.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.94 MB/sec execute 36 sec latency 6440.208 ms 2024-04-26T18:16:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 -- 172.21.15.90:0/909454459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 msgr2=0x7f7a3c10c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 --2- 172.21.15.90:0/909454459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c10c9e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009900 tx=0x7f7a2c02a480 comp rx=0 tx=0).stop 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 -- 172.21.15.90:0/909454459 shutdown_connections 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 --2- 172.21.15.90:0/909454459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c10cfb0 0x7f7a3c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 --2- 172.21.15.90:0/909454459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c10c9e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 --2- 172.21.15.90:0/909454459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a3c103be0 0x7f7a3c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 -- 172.21.15.90:0/909454459 >> 172.21.15.90:0/909454459 conn(0x7f7a3c077bb0 msgr2=0x7f7a3c077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 -- 172.21.15.90:0/909454459 shutdown_connections 2024-04-26T18:16:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.116+0000 7f7a43d12640 1 -- 172.21.15.90:0/909454459 wait complete. 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 Processor -- start 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 -- start start 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a3c103be0 0x7f7a3c101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c10cfb0 0x7f7a3c1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c112bf0 con 0x7f7a3c104500 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c112a70 con 0x7f7a3c10cfb0 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a43d12640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c112d70 con 0x7f7a3c103be0 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a412ad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a412ad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58036/0 (socket says 172.21.15.90:58036) 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.117+0000 7f7a412ad640 1 -- 172.21.15.90:0/435393919 learned_addr learned my addr 172.21.15.90:0/435393919 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a422af640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c10cfb0 0x7f7a3c1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a41aae640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a3c103be0 0x7f7a3c101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a412ad640 1 -- 172.21.15.90:0/435393919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a3c103be0 msgr2=0x7f7a3c101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a412ad640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a3c103be0 0x7f7a3c101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a412ad640 1 -- 172.21.15.90:0/435393919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c10cfb0 msgr2=0x7f7a3c1022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a412ad640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c10cfb0 0x7f7a3c1022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a412ad640 1 -- 172.21.15.90:0/435393919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3c1a7530 con 0x7f7a3c104500 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a412ad640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c101d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c005940 tx=0x7f7a2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a32ffd640 1 -- 172.21.15.90:0/435393919 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c03f030 con 0x7f7a3c104500 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3c1a77c0 con 0x7f7a3c104500 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a32ffd640 1 -- 172.21.15.90:0/435393919 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c003cb0 con 0x7f7a3c104500 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a32ffd640 1 -- 172.21.15.90:0/435393919 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c035070 con 0x7f7a3c104500 2024-04-26T18:16:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.118+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a3c1a7cb0 con 0x7f7a3c104500 2024-04-26T18:16:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.121+0000 7f7a32ffd640 1 -- 172.21.15.90:0/435393919 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f7a2c042030 con 0x7f7a3c104500 2024-04-26T18:16:49.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.122+0000 7f7a46a1f740 1 Processor -- start 2024-04-26T18:16:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.122+0000 7f7a46a1f740 1 -- start start 2024-04-26T18:16:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a32ffd640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7a280a3df0 0x7f7a280a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e022e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e0cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cd36d68460 con 0x55cd36e01f00 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cd36d22590 con 0x55cd36e0d0e0 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd36d64420 con 0x55cd36e02820 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a422af640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7a280a3df0 0x7f7a280a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e022e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a32ffd640 1 -- 172.21.15.90:0/435393919 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7a2c0f0bb0 con 0x7f7a3c104500 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e022e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58042/0 (socket says 172.21.15.90:58042) 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 -- 172.21.15.90:0/3123968888 learned_addr learned my addr 172.21.15.90:0/3123968888 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a422af640 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a412ad640 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e0cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 -- 172.21.15.90:0/3123968888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 msgr2=0x55cd36e0cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e0cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 -- 172.21.15.90:0/3123968888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 msgr2=0x55cd36e0f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a41aae640 1 -- 172.21.15.90:0/3123968888 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd36d2f0d0 con 0x55cd36e01f00 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.123+0000 7f7a422af640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7a280a3df0 0x7f7a280a62b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c06c240 tx=0x7f7a34012ad0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.124+0000 7f7a41aae640 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e022e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800bcd0 tx=0x7f7a380107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe249b4ab2101b81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.124+0000 7f7a27fff640 1 -- 172.21.15.90:0/3123968888 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38005020 con 0x55cd36e01f00 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.124+0000 7f7a27fff640 1 -- 172.21.15.90:0/3123968888 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a380047a0 con 0x55cd36e01f00 2024-04-26T18:16:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.124+0000 7f7a27fff640 1 -- 172.21.15.90:0/3123968888 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a38011680 con 0x55cd36e01f00 2024-04-26T18:16:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 -- 172.21.15.90:0/3123968888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 msgr2=0x55cd36e022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e022e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7a3800bcd0 tx=0x7f7a380107a0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 -- 172.21.15.90:0/3123968888 shutdown_connections 2024-04-26T18:16:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e0cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/3123968888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e022e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 -- 172.21.15.90:0/3123968888 >> 172.21.15.90:0/3123968888 conn(0x55cd36dfe410 msgr2=0x55cd36dfe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 -- 172.21.15.90:0/3123968888 shutdown_connections 2024-04-26T18:16:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 -- 172.21.15.90:0/3123968888 wait complete. 2024-04-26T18:16:49.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:48 smithi090 ceph-mon[29180]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-04-26T18:16:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.125+0000 7f7a46a1f740 1 -- start start 2024-04-26T18:16:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 1 Processor -- start 2024-04-26T18:16:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e09fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e02820 0x55cd36e0a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55cd36e0fd10 con 0x55cd36e01f00 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55cd36cefb60 con 0x55cd36e02820 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd36e0fe90 con 0x55cd36e0d0e0 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a46a1f740 10 client.0 _ll_register_callbacks cb 0x55cd36d142b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a41aae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e09fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a41aae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e09fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58046/0 (socket says 172.21.15.90:58046) 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a41aae640 1 -- 172.21.15.90:0/1311828476 learned_addr learned my addr 172.21.15.90:0/1311828476 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.126+0000 7f7a422af640 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e02820 0x55cd36e0a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a41aae640 1 -- 172.21.15.90:0/1311828476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 msgr2=0x55cd36e0aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a41aae640 1 -- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e02820 msgr2=0x55cd36e0a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e02820 0x55cd36e0a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a41aae640 1 -- 172.21.15.90:0/1311828476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd36a7a210 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e09fc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7a38000c00 tx=0x7f7a38005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26481f3df8fdea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a257fa640 1 -- 172.21.15.90:0/1311828476 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3802b060 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a257fa640 1 -- 172.21.15.90:0/1311828476 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7a38004b50 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a380047a0 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a257fa640 1 -- 172.21.15.90:0/1311828476 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a38005480 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a46a1f740 10 client.15867 fetch_fsmap learned FSMap version 28 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.127+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd36b46620 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a257fa640 1 -- 172.21.15.90:0/1311828476 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a380047a0 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a257fa640 1 -- 172.21.15.90:0/1311828476 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f7a3802e020 con 0x55cd36e01f00 2024-04-26T18:16:49.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a46a1f740 10 client.15867 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a46a1f740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a46a1f740 20 client.15867 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e1d0a0 0x55cd36e1f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a46a1f740 4 client.15867 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd36d76070 con 0x55cd36e1d0a0 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.128+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e1d0a0 0x55cd36e1f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.129+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e1d0a0 0x55cd36e1f490 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76f522b8d3a276be server_cookie=e87b3f00c4e1383f in_seq=0 out_seq=0 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.129+0000 7f7a257fa640 10 client.15867 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a257fa640 1 -- 172.21.15.90:0/1311828476 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cd36d76070 con 0x55cd36e1d0a0 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a257fa640 10 client.15867 handle_command_reply: tid=0 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 client.15867 shutdown 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 msgr2=0x55cd36e09fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e09fc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f7a38000c00 tx=0x7f7a38005080 comp rx=0 tx=0).stop 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 shutdown_connections 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e1d0a0 0x55cd36e1f490 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e0aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e02820 0x55cd36e0a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a257fa640 0 client.15867 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1311828476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e09fc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 >> 172.21.15.90:0/1311828476 conn(0x55cd36dfe410 msgr2=0x55cd36e04140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 shutdown_connections 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1311828476 wait complete. 2024-04-26T18:16:49.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.130+0000 7f7a46a1f740 20 client.15867 trim_cache size 0 max 16384 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 Processor -- start 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 -- start start 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e01f00 0x55cd36e18bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e02820 0x55cd36dfd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e24ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b2db0 con 0x55cd36e02820 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1b0680 con 0x55cd36e01f00 2024-04-26T18:16:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1ae4f0 con 0x55cd36e0d0e0 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.146+0000 7f7a412ad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e02820 0x55cd36dfd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e02820 0x55cd36dfd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58056/0 (socket says 172.21.15.90:58056) 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 -- 172.21.15.90:0/1110236367 learned_addr learned my addr 172.21.15.90:0/1110236367 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e01f00 0x55cd36e18bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a422af640 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e24ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 -- 172.21.15.90:0/1110236367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 msgr2=0x55cd36e24ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e24ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 -- 172.21.15.90:0/1110236367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e01f00 msgr2=0x55cd36e18bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e01f00 0x55cd36e18bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 -- 172.21.15.90:0/1110236367 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd36df9d60 con 0x55cd36e02820 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e01f00 0x55cd36e18bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e02820 0x55cd36dfd430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c004650 tx=0x7f7a2c0dd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b87cbe37297bebf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a277fe640 1 -- 172.21.15.90:0/1110236367 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c049040 con 0x55cd36e02820 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a277fe640 1 -- 172.21.15.90:0/1110236367 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c0c0f70 con 0x55cd36e02820 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.147+0000 7f7a277fe640 1 -- 172.21.15.90:0/1110236367 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c0ee5f0 con 0x55cd36e02820 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1110236367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e02820 msgr2=0x55cd36dfd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e02820 0x55cd36dfd430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c004650 tx=0x7f7a2c0dd040 comp rx=0 tx=0).stop 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1110236367 shutdown_connections 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e0d0e0 0x55cd36e24ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e02820 0x55cd36dfd430 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1110236367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e01f00 0x55cd36e18bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1110236367 >> 172.21.15.90:0/1110236367 conn(0x55cd36dfe410 msgr2=0x55cd36dfcd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1110236367 shutdown_connections 2024-04-26T18:16:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1110236367 wait complete. 2024-04-26T18:16:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.148+0000 7f7a46a1f740 1 -- start start 2024-04-26T18:16:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 1 Processor -- start 2024-04-26T18:16:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e15650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e06b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e07270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1aeb40 con 0x55cd36e01f00 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1affb0 con 0x55cd36e0d0e0 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3c1af210 con 0x55cd36e02820 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a46a1f740 10 client.0 _ll_register_callbacks cb 0x7f7a34009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e15650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e15650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58070/0 (socket says 172.21.15.90:58070) 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 -- 172.21.15.90:0/1048014150 learned_addr learned my addr 172.21.15.90:0/1048014150 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e06b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a422af640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 -- 172.21.15.90:0/1048014150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 msgr2=0x55cd36e06b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e06b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 -- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 msgr2=0x55cd36e07270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e07270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a41aae640 1 -- 172.21.15.90:0/1048014150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd36e82330 con 0x55cd36e01f00 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.149+0000 7f7a422af640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a41aae640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e15650 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7a38000c00 tx=0x7f7a38005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368dfb0fec847de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a257fa640 1 -- 172.21.15.90:0/1048014150 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a3802b050 con 0x55cd36e01f00 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a257fa640 1 -- 172.21.15.90:0/1048014150 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f7a38004660 con 0x55cd36e01f00 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a2c0c0f70 con 0x55cd36e01f00 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a257fa640 1 -- 172.21.15.90:0/1048014150 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a3800b410 con 0x55cd36e01f00 2024-04-26T18:16:49.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55cd36e834e0 con 0x55cd36e01f00 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a257fa640 1 -- 172.21.15.90:0/1048014150 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c0c0f70 con 0x55cd36e01f00 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a257fa640 1 -- 172.21.15.90:0/1048014150 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7f7a38041050 con 0x55cd36e01f00 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 20 client.15873 populate_metadata read hostname 'smithi090' 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e83c00 0x55cd36e86020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 4 client.15873 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:16:49.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.150+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cd36d6fee0 con 0x55cd36e83c00 2024-04-26T18:16:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.151+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e83c00 0x55cd36e86020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.151+0000 7f7a257fa640 10 client.15873 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:49.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.151+0000 7f7a412ad640 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e83c00 0x55cd36e86020 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2951b83442e288b6 server_cookie=ca511021f6b3f33 in_seq=0 out_seq=0 2024-04-26T18:16:49.190 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.189+0000 7f7a257fa640 1 -- 172.21.15.90:0/1048014150 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cd36d6fee0 con 0x55cd36e83c00 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.189+0000 7f7a257fa640 10 client.15873 handle_command_reply: tid=0 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.189+0000 7f7a46a1f740 1 client.15873 shutdown 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.189+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 msgr2=0x55cd36e15650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.189+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e15650 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f7a38000c00 tx=0x7f7a38005ab0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.189+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 shutdown_connections 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55cd36e83c00 0x55cd36e86020 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a257fa640 0 client.15873 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55cd36e0d0e0 0x55cd36e07270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd36e02820 0x55cd36e06b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 1 --2- 172.21.15.90:0/1048014150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55cd36e01f00 0x55cd36e15650 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.191 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 >> 172.21.15.90:0/1048014150 conn(0x55cd36dfe410 msgr2=0x55cd36dfd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 shutdown_connections 2024-04-26T18:16:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 1 -- 172.21.15.90:0/1048014150 wait complete. 2024-04-26T18:16:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a46a1f740 20 client.15873 trim_cache size 0 max 16384 2024-04-26T18:16:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:16:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:16:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:16:49.192 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:16:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7a280a3df0 msgr2=0x7f7a280a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7a280a3df0 0x7f7a280a62b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c06c240 tx=0x7f7a34012ad0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 msgr2=0x7f7a3c101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c101d70 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c005940 tx=0x7f7a2c0042f0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.193 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 shutdown_connections 2024-04-26T18:16:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7a280a3df0 0x7f7a280a62b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a3c10cfb0 0x7f7a3c1022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a3c104500 0x7f7a3c101d70 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 --2- 172.21.15.90:0/435393919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a3c103be0 0x7f7a3c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.190+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 >> 172.21.15.90:0/435393919 conn(0x7f7a3c077bb0 msgr2=0x7f7a3c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.191+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 shutdown_connections 2024-04-26T18:16:49.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:16:49.191+0000 7f7a43d12640 1 -- 172.21.15.90:0/435393919 wait complete. 2024-04-26T18:16:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:48 smithi119 ceph-mon[32752]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-04-26T18:16:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:48 smithi167 ceph-mon[32751]: pgmap v148: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-04-26T18:16:49.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.88 MB/sec execute 37 sec latency 7440.243 ms 2024-04-26T18:16:50.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.83 MB/sec execute 38 sec latency 8440.324 ms 2024-04-26T18:16:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:50 smithi090 ceph-mon[29180]: pgmap v149: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-04-26T18:16:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:50 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:16:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:50 smithi119 ceph-mon[32752]: pgmap v149: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-04-26T18:16:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:50 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:16:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:50 smithi167 ceph-mon[32751]: pgmap v149: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.1 MiB/s wr, 211 op/s 2024-04-26T18:16:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:50 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:16:51.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.79 MB/sec execute 39 sec latency 9440.402 ms 2024-04-26T18:16:52.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.74 MB/sec execute 40 sec latency 10440.500 ms 2024-04-26T18:16:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:52 smithi090 ceph-mon[29180]: pgmap v150: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 993 KiB/s wr, 110 op/s 2024-04-26T18:16:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:52 smithi119 ceph-mon[32752]: pgmap v150: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 993 KiB/s wr, 110 op/s 2024-04-26T18:16:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:52 smithi167 ceph-mon[32751]: pgmap v150: 97 pgs: 97 active+clean; 69 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 993 KiB/s wr, 110 op/s 2024-04-26T18:16:53.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.70 MB/sec execute 41 sec latency 11440.598 ms 2024-04-26T18:16:54.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.66 MB/sec execute 42 sec latency 12440.676 ms 2024-04-26T18:16:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:54 smithi090 ceph-mon[29180]: pgmap v151: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1015 KiB/s wr, 135 op/s 2024-04-26T18:16:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:54 smithi119 ceph-mon[32752]: pgmap v151: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1015 KiB/s wr, 135 op/s 2024-04-26T18:16:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:54 smithi167 ceph-mon[32751]: pgmap v151: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1015 KiB/s wr, 135 op/s 2024-04-26T18:16:55.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.62 MB/sec execute 43 sec latency 13440.778 ms 2024-04-26T18:16:56.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.58 MB/sec execute 44 sec latency 14440.924 ms 2024-04-26T18:16:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:56 smithi090 ceph-mon[29180]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-26T18:16:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:56 smithi119 ceph-mon[32752]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-26T18:16:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:56 smithi167 ceph-mon[32751]: pgmap v152: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-26T18:16:57.668 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.55 MB/sec execute 45 sec latency 15441.025 ms 2024-04-26T18:16:58.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.52 MB/sec execute 46 sec latency 16441.117 ms 2024-04-26T18:16:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:16:58 smithi090 ceph-mon[29180]: pgmap v153: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-26T18:16:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:16:58 smithi119 ceph-mon[32752]: pgmap v153: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-26T18:16:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:16:58 smithi167 ceph-mon[32751]: pgmap v153: 97 pgs: 97 active+clean; 65 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 55 KiB/s wr, 39 op/s 2024-04-26T18:16:59.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.48 MB/sec execute 47 sec latency 17441.216 ms 2024-04-26T18:17:00.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.45 MB/sec execute 48 sec latency 18441.296 ms 2024-04-26T18:17:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:00 smithi090 ceph-mon[29180]: pgmap v154: 97 pgs: 97 active+clean; 66 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 KiB/s wr, 43 op/s 2024-04-26T18:17:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:00 smithi119 ceph-mon[32752]: pgmap v154: 97 pgs: 97 active+clean; 66 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 KiB/s wr, 43 op/s 2024-04-26T18:17:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:00 smithi167 ceph-mon[32751]: pgmap v154: 97 pgs: 97 active+clean; 66 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 59 KiB/s wr, 43 op/s 2024-04-26T18:17:01.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.42 MB/sec execute 49 sec latency 19441.375 ms 2024-04-26T18:17:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 -- 172.21.15.90:0/972482942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 msgr2=0x7f148010f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 --2- 172.21.15.90:0/972482942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f148010f800 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1478009cd0 tx=0x7f147802a8e0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 -- 172.21.15.90:0/972482942 shutdown_connections 2024-04-26T18:17:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 --2- 172.21.15.90:0/972482942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f148010f800 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 --2- 172.21.15.90:0/972482942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1480102930 0x7f1480102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 --2- 172.21.15.90:0/972482942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480108940 0x7f1480108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 -- 172.21.15.90:0/972482942 >> 172.21.15.90:0/972482942 conn(0x7f14800fe400 msgr2=0x7f1480100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 -- 172.21.15.90:0/972482942 shutdown_connections 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.287+0000 7f14852d1640 1 -- 172.21.15.90:0/972482942 wait complete. 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 Processor -- start 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 -- start start 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480102930 0x7f1480074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f1480075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1480108940 0x7f14800756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1480112c00 con 0x7f14801032f0 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1480112a80 con 0x7f1480102930 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f14852d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1480112d80 con 0x7f1480108940 2024-04-26T18:17:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f147e7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f1480075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f147e7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f1480075170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53068/0 (socket says 172.21.15.90:53068) 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.288+0000 7f147e7fc640 1 -- 172.21.15.90:0/2020904953 learned_addr learned my addr 172.21.15.90:0/2020904953 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f147effd640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480102930 0x7f1480074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f147e7fc640 1 -- 172.21.15.90:0/2020904953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1480108940 msgr2=0x7f14800756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f147e7fc640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1480108940 0x7f14800756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f147e7fc640 1 -- 172.21.15.90:0/2020904953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480102930 msgr2=0x7f1480074c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f147e7fc640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480102930 0x7f1480074c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f147e7fc640 1 -- 172.21.15.90:0/2020904953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14801a7480 con 0x7f14801032f0 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f147e7fc640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f1480075170 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f147400c920 tx=0x7f147400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f1467fff640 1 -- 172.21.15.90:0/2020904953 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474007c70 con 0x7f14801032f0 2024-04-26T18:17:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f1467fff640 1 -- 172.21.15.90:0/2020904953 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1474007e10 con 0x7f14801032f0 2024-04-26T18:17:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14801a7770 con 0x7f14801032f0 2024-04-26T18:17:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f1467fff640 1 -- 172.21.15.90:0/2020904953 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474018820 con 0x7f14801032f0 2024-04-26T18:17:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.289+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14801a7cc0 con 0x7f14801032f0 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.291+0000 7f1467fff640 1 -- 172.21.15.90:0/2020904953 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f147400f450 con 0x7f14801032f0 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 Processor -- start 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 -- start start 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2dfe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e08ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 0x5588c2e0b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1467fff640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f14680a3dc0 0x7f14680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5588c2d64460 con 0x5588c2dfdf00 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5588c2d1e590 con 0x5588c2dfe820 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1487fef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588c2d60420 con 0x5588c2e090e0 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2dfe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147e7fc640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f14680a3dc0 0x7f14680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2dfe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53080/0 (socket says 172.21.15.90:53080) 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147f7fe640 1 -- 172.21.15.90:0/3589049357 learned_addr learned my addr 172.21.15.90:0/3589049357 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147effd640 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e08ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 0x5588c2e0b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f1467fff640 1 -- 172.21.15.90:0/2020904953 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f14740d0960 con 0x7f14801032f0 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147f7fe640 1 -- 172.21.15.90:0/3589049357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 msgr2=0x5588c2e0b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.292+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 0x5588c2e0b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f147f7fe640 1 -- 172.21.15.90:0/3589049357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 msgr2=0x5588c2e08ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e08ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f147f7fe640 1 -- 172.21.15.90:0/3589049357 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588c2d2b0d0 con 0x5588c2dfdf00 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2dfe2e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1478004af0 tx=0x7f1478005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f660a098aac87e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f147e7fc640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f14680a3dc0 0x7f14680a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f14740ce720 tx=0x7f147406b320 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f14657fa640 1 -- 172.21.15.90:0/3589049357 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478031a50 con 0x5588c2dfdf00 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f14657fa640 1 -- 172.21.15.90:0/3589049357 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1478036040 con 0x5588c2dfdf00 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.293+0000 7f14657fa640 1 -- 172.21.15.90:0/3589049357 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478004540 con 0x5588c2dfdf00 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- 172.21.15.90:0/3589049357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 msgr2=0x5588c2dfe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2dfe2e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1478004af0 tx=0x7f1478005d80 comp rx=0 tx=0).stop 2024-04-26T18:17:02.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- 172.21.15.90:0/3589049357 shutdown_connections 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 0x5588c2e0b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e08ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 --2- 172.21.15.90:0/3589049357 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2dfe2e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- 172.21.15.90:0/3589049357 >> 172.21.15.90:0/3589049357 conn(0x5588c2dfa410 msgr2=0x5588c2dfa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- 172.21.15.90:0/3589049357 shutdown_connections 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- 172.21.15.90:0/3589049357 wait complete. 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- start start 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 Processor -- start 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfdf00 0x5588c2e06130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfe820 0x5588c2e06670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 0x5588c2e06bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5588c2e0bcb0 con 0x5588c2dfe820 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5588c2cebb60 con 0x5588c2e090e0 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5588c2e0be30 con 0x5588c2dfdf00 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f1487fef740 10 client.0 _ll_register_callbacks cb 0x5588c2d102b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f147effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfe820 0x5588c2e06670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.294+0000 7f147effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfe820 0x5588c2e06670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53082/0 (socket says 172.21.15.90:53082) 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147effd640 1 -- 172.21.15.90:0/3704450653 learned_addr learned my addr 172.21.15.90:0/3704450653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147effd640 1 -- 172.21.15.90:0/3704450653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfdf00 msgr2=0x5588c2e06130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfdf00 0x5588c2e06130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147effd640 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfdf00 0x5588c2e06130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 0x5588c2e06bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147effd640 1 -- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 msgr2=0x5588c2e06bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147effd640 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 0x5588c2e06bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147effd640 1 -- 172.21.15.90:0/3704450653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588c2a76210 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f147effd640 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfe820 0x5588c2e06670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1470012700 tx=0x7f1470012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8548273972f01761 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1462ff5640 1 -- 172.21.15.90:0/3704450653 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470011bb0 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1462ff5640 1 -- 172.21.15.90:0/3704450653 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1470011d50 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1478036040 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1462ff5640 1 -- 172.21.15.90:0/3704450653 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f147000f460 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1487fef740 10 client.15885 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588c2b42620 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.295+0000 7f1462ff5640 1 -- 172.21.15.90:0/3704450653 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1478036040 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1462ff5640 1 -- 172.21.15.90:0/3704450653 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f1470010740 con 0x5588c2dfe820 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1487fef740 10 client.15885 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:02.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1487fef740 10 client.15885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1487fef740 20 client.15885 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1487fef740 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e19380 0x5588c2e1b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1487fef740 4 client.15885 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588c2d72070 con 0x5588c2e19380 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e19380 0x5588c2e1b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f1462ff5640 10 client.15885 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.296+0000 7f147f7fe640 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e19380 0x5588c2e1b7b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f7a43c32a071024 server_cookie=c1b08aebd42201bd in_seq=0 out_seq=0 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1462ff5640 1 -- 172.21.15.90:0/3704450653 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588c2d72070 con 0x5588c2e19380 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1462ff5640 10 client.15885 handle_command_reply: tid=0 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 client.15885 shutdown 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfe820 msgr2=0x5588c2e06670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfe820 0x5588c2e06670 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1470012700 tx=0x7f1470012bc0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 shutdown_connections 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e19380 0x5588c2e1b7b0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 0x5588c2e06bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfe820 0x5588c2e06670 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 --2- 172.21.15.90:0/3704450653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfdf00 0x5588c2e06130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1462ff5640 0 client.15885 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 >> 172.21.15.90:0/3704450653 conn(0x5588c2dfa410 msgr2=0x5588c2e00190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 shutdown_connections 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.297+0000 7f1487fef740 1 -- 172.21.15.90:0/3704450653 wait complete. 2024-04-26T18:17:02.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.298+0000 7f1487fef740 20 client.15885 trim_cache size 0 max 16384 2024-04-26T18:17:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 Processor -- start 2024-04-26T18:17:02.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 -- start start 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2df9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfe820 0x5588c2e008c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 0x5588c2e08070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f14801b2db0 con 0x5588c2dfdf00 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14801b0680 con 0x5588c2e090e0 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f1487fef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14801ae4d0 con 0x5588c2dfe820 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2df9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2df9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53094/0 (socket says 172.21.15.90:53094) 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.312+0000 7f147f7fe640 1 -- 172.21.15.90:0/2466175314 learned_addr learned my addr 172.21.15.90:0/2466175314 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f147f7fe640 1 -- 172.21.15.90:0/2466175314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfe820 msgr2=0x5588c2e008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f147f7fe640 1 --2- 172.21.15.90:0/2466175314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfe820 0x5588c2e008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f147f7fe640 1 -- 172.21.15.90:0/2466175314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 msgr2=0x5588c2e08070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f147f7fe640 1 --2- 172.21.15.90:0/2466175314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 0x5588c2e08070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f147f7fe640 1 -- 172.21.15.90:0/2466175314 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588c2df5b20 con 0x5588c2dfdf00 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f147f7fe640 1 --2- 172.21.15.90:0/2466175314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2df9d30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f14780097b0 tx=0x7f1478005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d1b29aff03a93b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1464ff9640 1 -- 172.21.15.90:0/2466175314 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14780098e0 con 0x5588c2dfdf00 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1464ff9640 1 -- 172.21.15.90:0/2466175314 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147804f070 con 0x5588c2dfdf00 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1464ff9640 1 -- 172.21.15.90:0/2466175314 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1478030800 con 0x5588c2dfdf00 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1487fef740 1 -- 172.21.15.90:0/2466175314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 msgr2=0x5588c2df9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1487fef740 1 --2- 172.21.15.90:0/2466175314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2df9d30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f14780097b0 tx=0x7f1478005df0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1487fef740 1 -- 172.21.15.90:0/2466175314 shutdown_connections 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1487fef740 1 --2- 172.21.15.90:0/2466175314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2e090e0 0x5588c2e08070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1487fef740 1 --2- 172.21.15.90:0/2466175314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2dfe820 0x5588c2e008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1487fef740 1 --2- 172.21.15.90:0/2466175314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2df9d30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.313+0000 7f1487fef740 1 -- 172.21.15.90:0/2466175314 >> 172.21.15.90:0/2466175314 conn(0x5588c2dfa410 msgr2=0x5588c2e07960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 -- 172.21.15.90:0/2466175314 shutdown_connections 2024-04-26T18:17:02.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 -- 172.21.15.90:0/2466175314 wait complete. 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 -- start start 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 Processor -- start 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2e03070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 0x5588c2e03af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f14801aeb40 con 0x5588c2dfdf00 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f14801affb0 con 0x5588c2dfe820 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f14801af210 con 0x5588c2e090e0 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f1487fef740 10 client.0 _ll_register_callbacks cb 0x7f1470009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2e03070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f147f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2e03070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53098/0 (socket says 172.21.15.90:53098) 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f147f7fe640 1 -- 172.21.15.90:0/2460871808 learned_addr learned my addr 172.21.15.90:0/2460871808 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:02.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f147effd640 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f147f7fe640 1 -- 172.21.15.90:0/2460871808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 msgr2=0x5588c2e03af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.314+0000 7f147f7fe640 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 0x5588c2e03af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f147f7fe640 1 -- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 msgr2=0x5588c2e035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f147f7fe640 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f147f7fe640 1 -- 172.21.15.90:0/2460871808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588c2e7ebd0 con 0x5588c2dfdf00 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f147effd640 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f147f7fe640 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2e03070 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1478031590 tx=0x7f1478009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b4f99c700ea34f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1462ff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1462ff5640 1 -- 172.21.15.90:0/2460871808 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147805e060 con 0x5588c2dfdf00 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1462ff5640 1 -- 172.21.15.90:0/2460871808 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f1478031af0 con 0x5588c2dfdf00 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147804f070 con 0x5588c2dfdf00 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1462ff5640 1 -- 172.21.15.90:0/2460871808 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1478031cf0 con 0x5588c2dfdf00 2024-04-26T18:17:02.320 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 10 client.15891 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588c2e7fec0 con 0x5588c2dfdf00 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1462ff5640 1 -- 172.21.15.90:0/2460871808 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f147804f070 con 0x5588c2dfdf00 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1462ff5640 1 -- 172.21.15.90:0/2460871808 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f1478061030 con 0x5588c2dfdf00 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 10 client.15891 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 20 client.15891 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e80140 0x5588c2e82530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 4 client.15891 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.315+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588c2d6bee0 con 0x5588c2e80140 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.316+0000 7f147effd640 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e80140 0x5588c2e82530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:02.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.316+0000 7f147effd640 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e80140 0x5588c2e82530 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d69d4602848fe0 server_cookie=a04534013da03ce0 in_seq=0 out_seq=0 2024-04-26T18:17:02.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.316+0000 7f1462ff5640 10 client.15891 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.334+0000 7f1462ff5640 1 -- 172.21.15.90:0/2460871808 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588c2d6bee0 con 0x5588c2e80140 2024-04-26T18:17:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.334+0000 7f1462ff5640 10 client.15891 handle_command_reply: tid=0 2024-04-26T18:17:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.334+0000 7f1487fef740 1 client.15891 shutdown 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 msgr2=0x5588c2e03070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2e03070 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1478031590 tx=0x7f1478009900 comp rx=0 tx=0).stop 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 shutdown_connections 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5588c2e80140 0x5588c2e82530 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5588c2e090e0 0x5588c2e03af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1462ff5640 0 client.15891 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5588c2dfe820 0x5588c2e035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 --2- 172.21.15.90:0/2460871808 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5588c2dfdf00 0x5588c2e03070 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 >> 172.21.15.90:0/2460871808 conn(0x5588c2dfa410 msgr2=0x5588c2e081f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 shutdown_connections 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 1 -- 172.21.15.90:0/2460871808 wait complete. 2024-04-26T18:17:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f1487fef740 20 client.15891 trim_cache size 0 max 16384 2024-04-26T18:17:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:17:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:17:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:17:02.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f14680a3dc0 msgr2=0x7f14680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f14680a3dc0 0x7f14680a6280 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f14740ce720 tx=0x7f147406b320 comp rx=0 tx=0).stop 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 msgr2=0x7f1480075170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f1480075170 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f147400c920 tx=0x7f147400cde0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 shutdown_connections 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f14680a3dc0 0x7f14680a6280 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1480108940 0x7f14800756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f14801032f0 0x7f1480075170 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 --2- 172.21.15.90:0/2020904953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1480102930 0x7f1480074c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.335+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 >> 172.21.15.90:0/2020904953 conn(0x7f14800fe400 msgr2=0x7f14800fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.336+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 shutdown_connections 2024-04-26T18:17:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:02.336+0000 7f14852d1640 1 -- 172.21.15.90:0/2020904953 wait complete. 2024-04-26T18:17:02.628 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:17:02.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.39 MB/sec execute 50 sec latency 20441.420 ms 2024-04-26T18:17:02.897 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:03.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:02 smithi090 ceph-mon[29180]: pgmap v155: 97 pgs: 97 active+clean; 66 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-04-26T18:17:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:02 smithi119 ceph-mon[32752]: pgmap v155: 97 pgs: 97 active+clean; 66 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-04-26T18:17:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:02 smithi167 ceph-mon[32751]: pgmap v155: 97 pgs: 97 active+clean; 66 MiB data, 1015 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 28 op/s 2024-04-26T18:17:03.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.37 MB/sec execute 51 sec latency 21441.521 ms 2024-04-26T18:17:04.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 -- 172.21.15.90:0/3958923589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 msgr2=0x7f8184109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:04.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 --2- 172.21.15.90:0/3958923589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184109880 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8174009990 tx=0x7f8174009ca0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 -- 172.21.15.90:0/3958923589 shutdown_connections 2024-04-26T18:17:04.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 --2- 172.21.15.90:0/3958923589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184109880 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 --2- 172.21.15.90:0/3958923589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f818406bad0 0x7f81841102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 --2- 172.21.15.90:0/3958923589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f818406b130 0x7f818406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 -- 172.21.15.90:0/3958923589 >> 172.21.15.90:0/3958923589 conn(0x7f81840792d0 msgr2=0x7f81840796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:04.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 -- 172.21.15.90:0/3958923589 shutdown_connections 2024-04-26T18:17:04.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 -- 172.21.15.90:0/3958923589 wait complete. 2024-04-26T18:17:04.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.588+0000 7f81898f3700 1 Processor -- start 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81898f3700 1 -- start start 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81898f3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f818406b130 0x7f818419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81898f3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f818406bad0 0x7f818419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81898f3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81898f3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8184113600 con 0x7f81841094a0 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81898f3700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8184113480 con 0x7f818406b130 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81898f3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8184113780 con 0x7f818406bad0 2024-04-26T18:17:04.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53122/0 (socket says 172.21.15.90:53122) 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 -- 172.21.15.90:0/2028509706 learned_addr learned my addr 172.21.15.90:0/2028509706 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f8182ffd700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f818406b130 0x7f818419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81827fc700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f818406bad0 0x7f818419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 -- 172.21.15.90:0/2028509706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f818406bad0 msgr2=0x7f818419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f818406bad0 0x7f818419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 -- 172.21.15.90:0/2028509706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f818406b130 msgr2=0x7f818419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f818406b130 0x7f818419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 -- 172.21.15.90:0/2028509706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f818419a0c0 con 0x7f81841094a0 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.589+0000 7f81837fe700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184199960 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f817c00ef30 tx=0x7f817c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.590+0000 7f81888f1700 1 -- 172.21.15.90:0/2028509706 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c0092e0 con 0x7f81841094a0 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.590+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f818419a350 con 0x7f81841094a0 2024-04-26T18:17:04.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.590+0000 7f81888f1700 1 -- 172.21.15.90:0/2028509706 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f817c004bb0 con 0x7f81841094a0 2024-04-26T18:17:04.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.590+0000 7f81888f1700 1 -- 172.21.15.90:0/2028509706 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c004020 con 0x7f81841094a0 2024-04-26T18:17:04.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.590+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81841a6620 con 0x7f81841094a0 2024-04-26T18:17:04.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.591+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f818406a110 con 0x7f81841094a0 2024-04-26T18:17:04.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.592+0000 7f81888f1700 1 -- 172.21.15.90:0/2028509706 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f817c01b030 con 0x7f81841094a0 2024-04-26T18:17:04.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.592+0000 7f81888f1700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f816c0a3db0 0x7f816c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:04.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.592+0000 7f8182ffd700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f816c0a3db0 0x7f816c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:04.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.592+0000 7f81888f1700 1 -- 172.21.15.90:0/2028509706 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f817c0d01b0 con 0x7f81841094a0 2024-04-26T18:17:04.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.593+0000 7f8182ffd700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f816c0a3db0 0x7f816c0a6270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81841087d0 tx=0x7f817401a040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:04.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.597+0000 7f81888f1700 1 -- 172.21.15.90:0/2028509706 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f817c022020 con 0x7f81841094a0 2024-04-26T18:17:04.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.34 MB/sec execute 52 sec latency 22441.587 ms 2024-04-26T18:17:04.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.861+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f818402e490 con 0x7f81841094a0 2024-04-26T18:17:04.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.863+0000 7f81888f1700 1 -- 172.21.15.90:0/2028509706 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11874 (secure 0 0 0) 0x7f817c010040 con 0x7f81841094a0 2024-04-26T18:17:04.863 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:17:04.863 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:16:49.669086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:17:04.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f816c0a3db0 msgr2=0x7f816c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:04.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f816c0a3db0 0x7f816c0a6270 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81841087d0 tx=0x7f817401a040 comp rx=0 tx=0).stop 2024-04-26T18:17:04.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 msgr2=0x7f8184199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:04.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184199960 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f817c00ef30 tx=0x7f817c00c3a0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 shutdown_connections 2024-04-26T18:17:04.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f816c0a3db0 0x7f816c0a6270 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81841094a0 0x7f8184199960 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f818406b130 0x7f818419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 --2- 172.21.15.90:0/2028509706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f818406bad0 0x7f818419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 >> 172.21.15.90:0/2028509706 conn(0x7f81840792d0 msgr2=0x7f8184100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 shutdown_connections 2024-04-26T18:17:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:04.865+0000 7f81898f3700 1 -- 172.21.15.90:0/2028509706 wait complete. 2024-04-26T18:17:04.866 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-04-26T18:17:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:04 smithi090 ceph-mon[29180]: pgmap v156: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 966 KiB/s wr, 69 op/s 2024-04-26T18:17:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:04 smithi119 ceph-mon[32752]: pgmap v156: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 966 KiB/s wr, 69 op/s 2024-04-26T18:17:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:04 smithi167 ceph-mon[32751]: pgmap v156: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 966 KiB/s wr, 69 op/s 2024-04-26T18:17:05.623 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db --set-id 692c662e --release --await' 2024-04-26T18:17:05.623 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db --set-id 692c662e --release --await 2024-04-26T18:17:05.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.32 MB/sec execute 53 sec latency 23441.615 ms 2024-04-26T18:17:05.902 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:05 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2028509706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:17:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:05 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2028509706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:17:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:05 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2028509706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:17:06.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246282 1.29 MB/sec execute 54 sec latency 24441.639 ms 2024-04-26T18:17:06.995 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:06 smithi090 ceph-mon[29180]: pgmap v157: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 945 KiB/s wr, 43 op/s 2024-04-26T18:17:06.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:06 smithi119 ceph-mon[32752]: pgmap v157: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 945 KiB/s wr, 43 op/s 2024-04-26T18:17:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:06 smithi167 ceph-mon[32751]: pgmap v157: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 945 KiB/s wr, 43 op/s 2024-04-26T18:17:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:07.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.541+0000 7fdf6acdf700 1 -- 172.21.15.90:0/3552291013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64102780 msgr2=0x7fdf641101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.541+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/3552291013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64102780 0x7fdf641101f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00b0a0 tx=0x7fdf5c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.541+0000 7fdf6acdf700 1 -- 172.21.15.90:0/3552291013 shutdown_connections 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.541+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/3552291013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64102780 0x7fdf641101f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.541+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/3552291013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf641084e0 0x7fdf641088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.541+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/3552291013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf64101dc0 0x7fdf64102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.541+0000 7fdf6acdf700 1 -- 172.21.15.90:0/3552291013 >> 172.21.15.90:0/3552291013 conn(0x7fdf64077640 msgr2=0x7fdf64077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.542+0000 7fdf6acdf700 1 -- 172.21.15.90:0/3552291013 shutdown_connections 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.542+0000 7fdf6acdf700 1 -- 172.21.15.90:0/3552291013 wait complete. 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.542+0000 7fdf6acdf700 1 Processor -- start 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.542+0000 7fdf6acdf700 1 -- start start 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.542+0000 7fdf6acdf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64101dc0 0x7fdf64102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.542+0000 7fdf6acdf700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf64102780 0x7fdf641034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf6acdf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf641084e0 0x7fdf64103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf6acdf700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641135d0 con 0x7fdf64101dc0 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf6acdf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64113450 con 0x7fdf641084e0 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf6acdf700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf64113750 con 0x7fdf64102780 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf64102780 0x7fdf641034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf64102780 0x7fdf641034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:34298/0 (socket says 172.21.15.90:34298) 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 -- 172.21.15.90:0/2307054864 learned_addr learned my addr 172.21.15.90:0/2307054864 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64101dc0 0x7fdf64102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf6927c700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf641084e0 0x7fdf64103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 -- 172.21.15.90:0/2307054864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf641084e0 msgr2=0x7fdf64103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf641084e0 0x7fdf64103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 -- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64101dc0 msgr2=0x7fdf64102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64101dc0 0x7fdf64102fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf63fff700 1 -- 172.21.15.90:0/2307054864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf641a7fb0 con 0x7fdf64102780 2024-04-26T18:17:07.546 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.543+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64101dc0 0x7fdf64102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:17:07.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.544+0000 7fdf6927c700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf641084e0 0x7fdf64103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:17:07.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.544+0000 7fdf63fff700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf64102780 0x7fdf641034e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000d950 tx=0x7fdf5000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:07.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.544+0000 7fdf61ffb700 1 -- 172.21.15.90:0/2307054864 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50016070 con 0x7fdf64102780 2024-04-26T18:17:07.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.544+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf641a82a0 con 0x7fdf64102780 2024-04-26T18:17:07.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.544+0000 7fdf61ffb700 1 -- 172.21.15.90:0/2307054864 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf50010460 con 0x7fdf64102780 2024-04-26T18:17:07.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.544+0000 7fdf61ffb700 1 -- 172.21.15.90:0/2307054864 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000f620 con 0x7fdf64102780 2024-04-26T18:17:07.547 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.544+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf641a87e0 con 0x7fdf64102780 2024-04-26T18:17:07.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.546+0000 7fdf61ffb700 1 -- 172.21.15.90:0/2307054864 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fdf50019070 con 0x7fdf64102780 2024-04-26T18:17:07.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.547+0000 7fdf61ffb700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdf540a3db0 0x7fdf540a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.547+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdf540a3db0 0x7fdf540a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.547+0000 7fdf61ffb700 1 -- 172.21.15.90:0/2307054864 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdf500d0470 con 0x7fdf64102780 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdf540a3db0 0x7fdf540a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf64078bc0 tx=0x7fdf5800b570 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 Processor -- start 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 -- start start 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7c1a0 0x55900af7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7cac0 0x55900af87430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af89d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55900aecf840 con 0x55900af87970 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55900ae6e6e0 con 0x55900af7cac0 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf75571b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55900aecb800 con 0x55900af7c1a0 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf6927c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af89d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf6927c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af89d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53138/0 (socket says 172.21.15.90:53138) 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf6927c700 1 -- 172.21.15.90:0/1010140492 learned_addr learned my addr 172.21.15.90:0/1010140492 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.548+0000 7fdf63fff700 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7cac0 0x55900af87430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7c1a0 0x55900af7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf6927c700 1 -- 172.21.15.90:0/1010140492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7c1a0 msgr2=0x55900af7c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf6927c700 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7c1a0 0x55900af7c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf6927c700 1 -- 172.21.15.90:0/1010140492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7cac0 msgr2=0x55900af87430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf6927c700 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7cac0 0x55900af87430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf6927c700 1 -- 172.21.15.90:0/1010140492 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900ae1c9d0 con 0x55900af87970 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf63fff700 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7cac0 0x55900af87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf6927c700 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af89d60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c006130 tx=0x7fdf5c0090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c572eab36154cbc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf4effd700 1 -- 172.21.15.90:0/1010140492 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c020030 con 0x55900af87970 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf4effd700 1 -- 172.21.15.90:0/1010140492 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5c0092c0 con 0x55900af87970 2024-04-26T18:17:07.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.549+0000 7fdf4effd700 1 -- 172.21.15.90:0/1010140492 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c007e60 con 0x55900af87970 2024-04-26T18:17:07.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 -- 172.21.15.90:0/1010140492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 msgr2=0x55900af89d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af89d60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c006130 tx=0x7fdf5c0090d0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 -- 172.21.15.90:0/1010140492 shutdown_connections 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af89d60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7c1a0 0x55900af7c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 --2- 172.21.15.90:0/1010140492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7cac0 0x55900af87430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 -- 172.21.15.90:0/1010140492 >> 172.21.15.90:0/1010140492 conn(0x55900aea9e40 msgr2=0x55900af78690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 -- 172.21.15.90:0/1010140492 shutdown_connections 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 -- 172.21.15.90:0/1010140492 wait complete. 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 -- start start 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.550+0000 7fdf75571b80 1 Processor -- start 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af87120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af84420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af84960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf75571b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55900af8a4e0 con 0x55900af87970 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf75571b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55900ae35a50 con 0x55900af7c1a0 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf75571b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55900af8a660 con 0x55900af7cac0 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf75571b80 10 client.0 _ll_register_callbacks cb 0x55900ae63680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf6927c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af84960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf6927c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af84960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53140/0 (socket says 172.21.15.90:53140) 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf6927c700 1 -- 172.21.15.90:0/3025157509 learned_addr learned my addr 172.21.15.90:0/3025157509 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf63fff700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af84420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af87120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf63fff700 1 -- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 msgr2=0x55900af87120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf63fff700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af87120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf63fff700 1 -- 172.21.15.90:0/3025157509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 msgr2=0x55900af84960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.557 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf63fff700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af84960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf63fff700 1 -- 172.21.15.90:0/3025157509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900af85060 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.551+0000 7fdf6927c700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af84960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf63fff700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af84420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf500cdbd0 tx=0x7fdf5009cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f7e406cf5772aef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf43fff700 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf43fff700 1 -- 172.21.15.90:0/3025157509 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50020040 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf43fff700 1 -- 172.21.15.90:0/3025157509 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdf5009c680 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf5c0092c0 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf43fff700 1 -- 172.21.15.90:0/3025157509 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf5009c880 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 10 client.25351 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55900af95040 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf43fff700 1 -- 172.21.15.90:0/3025157509 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf5c0092c0 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf43fff700 1 -- 172.21.15.90:0/3025157509 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fdf500b5ba0 con 0x55900af7cac0 2024-04-26T18:17:07.558 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 10 client.25351 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 10 client.25351 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:17:07.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 20 client.25351 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900af97680 0x55900af99a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 4 client.25351 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.552+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900aedd220 con 0x55900af97680 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.553+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900af97680 0x55900af99a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.553+0000 7fdf43fff700 10 client.25351 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.553+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900af97680 0x55900af99a70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae5270a3937af9c3 server_cookie=e2495697f086488a in_seq=0 out_seq=0 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf43fff700 1 -- 172.21.15.90:0/3025157509 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55900aedd220 con 0x55900af97680 2024-04-26T18:17:07.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf43fff700 10 client.25351 handle_command_reply: tid=0 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 client.25351 shutdown 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 msgr2=0x55900af84420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af84420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf500cdbd0 tx=0x7fdf5009cc80 comp rx=0 tx=0).stop 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 shutdown_connections 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900af97680 0x55900af99a70 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af87970 0x55900af84960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af87120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf43fff700 0 client.25351 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3025157509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af84420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 >> 172.21.15.90:0/3025157509 conn(0x55900aea9e40 msgr2=0x55900af782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 shutdown_connections 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 1 -- 172.21.15.90:0/3025157509 wait complete. 2024-04-26T18:17:07.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.554+0000 7fdf75571b80 20 client.25351 trim_cache size 0 max 16384 2024-04-26T18:17:07.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 Processor -- start 2024-04-26T18:17:07.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 -- start start 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af7fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7cac0 0x55900af933e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af87970 0x55900af9f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641b2520 con 0x55900af7cac0 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641afdb0 con 0x55900af7c1a0 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.573+0000 7fdf75571b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641adc20 con 0x55900af87970 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7cac0 0x55900af933e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7cac0 0x55900af933e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53142/0 (socket says 172.21.15.90:53142) 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 -- 172.21.15.90:0/3795588205 learned_addr learned my addr 172.21.15.90:0/3795588205 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af7fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf6927c700 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af87970 0x55900af9f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 -- 172.21.15.90:0/3795588205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af87970 msgr2=0x55900af9f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af87970 0x55900af9f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 -- 172.21.15.90:0/3795588205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 msgr2=0x55900af7fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af7fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 -- 172.21.15.90:0/3795588205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900af829d0 con 0x55900af7cac0 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf63fff700 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7cac0 0x55900af933e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf5009c3b0 tx=0x7fdf500987f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fb06720ee00167 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf4e7fc700 1 -- 172.21.15.90:0/3795588205 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf50020050 con 0x55900af7cac0 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf4e7fc700 1 -- 172.21.15.90:0/3795588205 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf500bb2a0 con 0x55900af7cac0 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.574+0000 7fdf4e7fc700 1 -- 172.21.15.90:0/3795588205 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf500a4c00 con 0x55900af7cac0 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 -- 172.21.15.90:0/3795588205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7cac0 msgr2=0x55900af933e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7cac0 0x55900af933e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf5009c3b0 tx=0x7fdf500987f0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 -- 172.21.15.90:0/3795588205 shutdown_connections 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af87970 0x55900af9f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af7c1a0 0x55900af7fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 --2- 172.21.15.90:0/3795588205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7cac0 0x55900af933e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 -- 172.21.15.90:0/3795588205 >> 172.21.15.90:0/3795588205 conn(0x55900aea9e40 msgr2=0x55900af7f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 -- 172.21.15.90:0/3795588205 shutdown_connections 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 -- 172.21.15.90:0/3795588205 wait complete. 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 -- start start 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.575+0000 7fdf75571b80 1 Processor -- start 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7c1a0 0x55900af813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af87970 0x55900af81e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641ae270 con 0x55900af7c1a0 2024-04-26T18:17:07.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641af6e0 con 0x55900af87970 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf641ae940 con 0x55900af7cac0 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 10 client.0 _ll_register_callbacks cb 0x55900ae63680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7c1a0 0x55900af813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7c1a0 0x55900af813b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53144/0 (socket says 172.21.15.90:53144) 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 -- 172.21.15.90:0/530502095 learned_addr learned my addr 172.21.15.90:0/530502095 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf6927c700 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af87970 0x55900af81e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf63fff700 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 -- 172.21.15.90:0/530502095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 msgr2=0x55900af818f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af818f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 -- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af87970 msgr2=0x55900af81e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af87970 0x55900af81e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 -- 172.21.15.90:0/530502095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900affcdf0 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf68a7b700 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7c1a0 0x55900af813b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800b4b0 tx=0x7fdf58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806f044eef4db979 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf43fff700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf43fff700 1 -- 172.21.15.90:0/530502095 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf58011e20 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf43fff700 1 -- 172.21.15.90:0/530502095 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fdf5802b3f0 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.576+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf500bb2a0 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf43fff700 1 -- 172.21.15.90:0/530502095 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf5802b5f0 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 10 client.15915 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55900affe170 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf43fff700 1 -- 172.21.15.90:0/530502095 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf500bb2a0 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf43fff700 1 -- 172.21.15.90:0/530502095 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fdf5803c020 con 0x55900af7c1a0 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 10 client.15915 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 10 client.15915 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:17:07.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 20 client.15915 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900affe7b0 0x55900b000bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 4 client.15915 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "692c662e", "release": true, "await": true}] 2024-04-26T18:17:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "set_id": "692c662e", "release": true, "await": true}) -- 0x55900aed72c0 con 0x55900affe7b0 2024-04-26T18:17:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.577+0000 7fdf63fff700 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900affe7b0 0x55900b000bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.578+0000 7fdf63fff700 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900affe7b0 0x55900b000bd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d36a832d9ded9d00 server_cookie=16c4113b697ec6b4 in_seq=0 out_seq=0 2024-04-26T18:17:07.583 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.578+0000 7fdf43fff700 10 client.15915 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:07.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf43fff700 1 -- 172.21.15.90:0/530502095 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55900aed72c0 con 0x55900affe7b0 2024-04-26T18:17:07.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf43fff700 10 client.15915 handle_command_reply: tid=0 2024-04-26T18:17:07.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 client.15915 shutdown 2024-04-26T18:17:07.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7c1a0 msgr2=0x55900af813b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7c1a0 0x55900af813b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800b4b0 tx=0x7fdf58004750 comp rx=0 tx=0).stop 2024-04-26T18:17:07.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 shutdown_connections 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55900affe7b0 0x55900b000bd0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55900af7c1a0 0x55900af813b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55900af87970 0x55900af81e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf43fff700 0 client.15915 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 --2- 172.21.15.90:0/530502095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55900af7cac0 0x55900af818f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 >> 172.21.15.90:0/530502095 conn(0x55900aea9e40 msgr2=0x55900af85c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 shutdown_connections 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.585+0000 7fdf75571b80 1 -- 172.21.15.90:0/530502095 wait complete. 2024-04-26T18:17:07.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf75571b80 20 client.15915 trim_cache size 0 max 16384 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 29, 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 16, 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "692c662e": { 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 16, 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:17:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 79.5, 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RELEASED", 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:17:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:17:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:17:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:17:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:17:07.590 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:17:07.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdf540a3db0 msgr2=0x7fdf540a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdf540a3db0 0x7fdf540a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdf64078bc0 tx=0x7fdf5800b570 comp rx=0 tx=0).stop 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf64102780 msgr2=0x7fdf641034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf64102780 0x7fdf641034e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000d950 tx=0x7fdf5000dd10 comp rx=0 tx=0).stop 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 shutdown_connections 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdf540a3db0 0x7fdf540a6270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdf64101dc0 0x7fdf64102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf641084e0 0x7fdf64103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 --2- 172.21.15.90:0/2307054864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf64102780 0x7fdf641034e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 >> 172.21.15.90:0/2307054864 conn(0x7fdf64077640 msgr2=0x7fdf641009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 shutdown_connections 2024-04-26T18:17:07.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:17:07.586+0000 7fdf6acdf700 1 -- 172.21.15.90:0/2307054864 wait complete. 2024-04-26T18:17:07.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 246445 1.27 MB/sec execute 55 sec latency 25361.871 ms 2024-04-26T18:17:08.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:08.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:08.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:17:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:17:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:17:08.453 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '692c662e', seconds elapsed: 31.7 2024-04-26T18:17:08.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 248779 1.49 MB/sec execute 56 sec latency 10.039 ms 2024-04-26T18:17:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:08 smithi119 ceph-mon[32752]: pgmap v158: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 945 KiB/s wr, 43 op/s 2024-04-26T18:17:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:08 smithi167 ceph-mon[32751]: pgmap v158: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 945 KiB/s wr, 43 op/s 2024-04-26T18:17:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:08 smithi090 ceph-mon[29180]: pgmap v158: 97 pgs: 97 active+clean; 75 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 945 KiB/s wr, 43 op/s 2024-04-26T18:17:09.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 251315 1.69 MB/sec execute 57 sec latency 11.768 ms 2024-04-26T18:17:10.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 252793 1.79 MB/sec execute 58 sec latency 11.529 ms 2024-04-26T18:17:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:10 smithi119 ceph-mon[32752]: pgmap v159: 97 pgs: 97 active+clean; 65 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-04-26T18:17:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:10 smithi167 ceph-mon[32751]: pgmap v159: 97 pgs: 97 active+clean; 65 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-04-26T18:17:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:10 smithi090 ceph-mon[29180]: pgmap v159: 97 pgs: 97 active+clean; 65 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.1 MiB/s wr, 75 op/s 2024-04-26T18:17:11.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 255644 2.06 MB/sec execute 59 sec latency 17.120 ms 2024-04-26T18:17:12.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 257030 2.12 MB/sec execute 60 sec latency 16.279 ms 2024-04-26T18:17:13.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:12 smithi090 ceph-mon[29180]: pgmap v160: 97 pgs: 97 active+clean; 65 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-04-26T18:17:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:12 smithi119 ceph-mon[32752]: pgmap v160: 97 pgs: 97 active+clean; 65 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-04-26T18:17:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:12 smithi167 ceph-mon[32751]: pgmap v160: 97 pgs: 97 active+clean; 65 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.1 MiB/s wr, 71 op/s 2024-04-26T18:17:13.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 259362 2.31 MB/sec execute 61 sec latency 9.757 ms 2024-04-26T18:17:14.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 261570 2.41 MB/sec execute 62 sec latency 14.538 ms 2024-04-26T18:17:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:14 smithi119 ceph-mon[32752]: pgmap v161: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-26T18:17:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:14 smithi167 ceph-mon[32751]: pgmap v161: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-26T18:17:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:14 smithi090 ceph-mon[29180]: pgmap v161: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 172 op/s 2024-04-26T18:17:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.426+0000 7fa678eb0640 1 -- 172.21.15.90:0/2888562940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 msgr2=0x7fa67410f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.426+0000 7fa678eb0640 1 --2- 172.21.15.90:0/2888562940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa67410f7c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009ca0 tx=0x7fa66c02a8c0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.426+0000 7fa678eb0640 1 -- 172.21.15.90:0/2888562940 shutdown_connections 2024-04-26T18:17:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.426+0000 7fa678eb0640 1 --2- 172.21.15.90:0/2888562940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa67410f7c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.426+0000 7fa678eb0640 1 --2- 172.21.15.90:0/2888562940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa67406b1a0 0x7fa67406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.426+0000 7fa678eb0640 1 --2- 172.21.15.90:0/2888562940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa674108950 0x7fa674108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.427 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.426+0000 7fa678eb0640 1 -- 172.21.15.90:0/2888562940 >> 172.21.15.90:0/2888562940 conn(0x7fa674078580 msgr2=0x7fa674078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 -- 172.21.15.90:0/2888562940 shutdown_connections 2024-04-26T18:17:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 -- 172.21.15.90:0/2888562940 wait complete. 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 Processor -- start 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 -- start start 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa67406b1a0 0x7fa674198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa674199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa674108950 0x7fa67419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa674112bf0 con 0x7fa67406bb40 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa674112a70 con 0x7fa674108950 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.427+0000 7fa678eb0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa674112d70 con 0x7fa67406b1a0 2024-04-26T18:17:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa674199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa67259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa67406b1a0 0x7fa674198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa67259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa67406b1a0 0x7fa674198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:38540/0 (socket says 172.21.15.90:38540) 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa67259c640 1 -- 172.21.15.90:0/1915702662 learned_addr learned my addr 172.21.15.90:0/1915702662 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa674199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44072/0 (socket says 172.21.15.90:44072) 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa672d9d640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa674108950 0x7fa67419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 -- 172.21.15.90:0/1915702662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa67406b1a0 msgr2=0x7fa674198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa67406b1a0 0x7fa674198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 -- 172.21.15.90:0/1915702662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa674108950 msgr2=0x7fa67419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa674108950 0x7fa67419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 -- 172.21.15.90:0/1915702662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa67419dfe0 con 0x7fa67406bb40 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa67259c640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa67406b1a0 0x7fa674198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa671d9b640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa674199530 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c970 tx=0x7fa66800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa6637fe640 1 -- 172.21.15.90:0/1915702662 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668007bb0 con 0x7fa67406bb40 2024-04-26T18:17:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa67419e2d0 con 0x7fa67406bb40 2024-04-26T18:17:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa6637fe640 1 -- 172.21.15.90:0/1915702662 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa668007d50 con 0x7fa67406bb40 2024-04-26T18:17:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa6637fe640 1 -- 172.21.15.90:0/1915702662 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668004da0 con 0x7fa67406bb40 2024-04-26T18:17:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.428+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6741a5b60 con 0x7fa67406bb40 2024-04-26T18:17:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.431+0000 7fa6637fe640 1 -- 172.21.15.90:0/1915702662 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fa668004f80 con 0x7fa67406bb40 2024-04-26T18:17:15.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.431+0000 7fa67bbc7740 1 Processor -- start 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.431+0000 7fa67bbc7740 1 -- start start 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb4f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4f820 0x55812eb59ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb5c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55812eab5460 con 0x55812eb5a0e0 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55812ea6f590 con 0x55812eb4f820 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55812eab1420 con 0x55812eb4ef00 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb5c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa672d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb4f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4f820 0x55812eb59ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa672d9d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb4f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:38556/0 (socket says 172.21.15.90:38556) 2024-04-26T18:17:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa6637fe640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa65c0a3dc0 0x7fa65c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb5c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44082/0 (socket says 172.21.15.90:44082) 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa672d9d640 1 -- 172.21.15.90:0/230833456 learned_addr learned my addr 172.21.15.90:0/230833456 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4f820 0x55812eb59ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:37066/0 (socket says 172.21.15.90:37066) 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa672d9d640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa65c0a3dc0 0x7fa65c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 -- 172.21.15.90:0/230833456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 msgr2=0x55812eb4f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 --2- 172.21.15.90:0/230833456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb4f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 -- 172.21.15.90:0/230833456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 msgr2=0x55812eb5c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 --2- 172.21.15.90:0/230833456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb5c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 -- 172.21.15.90:0/230833456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55812ea7c0d0 con 0x55812eb4f820 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa6637fe640 1 -- 172.21.15.90:0/1915702662 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa6680d0390 con 0x7fa67406bb40 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.432+0000 7fa67259c640 1 --2- 172.21.15.90:0/230833456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4f820 0x55812eb59ba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6640114f0 tx=0x7fa6640118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2877c2c37b4903d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa672d9d640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa65c0a3dc0 0x7fa65c0a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02f500 tx=0x7fa66c02a8c0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa660ff9640 1 -- 172.21.15.90:0/230833456 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66401e070 con 0x55812eb4f820 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa660ff9640 1 -- 172.21.15.90:0/230833456 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66400fa50 con 0x55812eb4f820 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa660ff9640 1 -- 172.21.15.90:0/230833456 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664019af0 con 0x55812eb4f820 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 -- 172.21.15.90:0/230833456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4f820 msgr2=0x55812eb59ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/230833456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4f820 0x55812eb59ba0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6640114f0 tx=0x7fa6640118f0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 -- 172.21.15.90:0/230833456 shutdown_connections 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/230833456 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb5c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/230833456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4f820 0x55812eb59ba0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/230833456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb4f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 -- 172.21.15.90:0/230833456 >> 172.21.15.90:0/230833456 conn(0x55812eb4b410 msgr2=0x55812eb4b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 -- 172.21.15.90:0/230833456 shutdown_connections 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.433+0000 7fa67bbc7740 1 -- 172.21.15.90:0/230833456 wait complete. 2024-04-26T18:17:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 -- start start 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 Processor -- start 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4ef00 0x55812eb56fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4f820 0x55812eb57510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb57a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55812eb5cd10 con 0x55812eb5a0e0 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55812ea3cb60 con 0x55812eb4ef00 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55812eb5ce90 con 0x55812eb4f820 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67bbc7740 10 client.0 _ll_register_callbacks cb 0x55812ea612b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb57a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4ef00 0x55812eb56fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb57a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44092/0 (socket says 172.21.15.90:44092) 2024-04-26T18:17:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 -- 172.21.15.90:0/726321509 learned_addr learned my addr 172.21.15.90:0/726321509 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa671d9b640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4f820 0x55812eb57510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 -- 172.21.15.90:0/726321509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4f820 msgr2=0x55812eb57510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4f820 0x55812eb57510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 -- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4ef00 msgr2=0x55812eb56fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4ef00 0x55812eb56fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa672d9d640 1 -- 172.21.15.90:0/726321509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812e7c7210 con 0x55812eb5a0e0 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa67259c640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4ef00 0x55812eb56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.434+0000 7fa671d9b640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4f820 0x55812eb57510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa672d9d640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb57a50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa66c003bf0 tx=0x7fa66c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322c15520fa7d677 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa659ffb640 1 -- 172.21.15.90:0/726321509 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66c004740 con 0x55812eb5a0e0 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa659ffb640 1 -- 172.21.15.90:0/726321509 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa66c02f660 con 0x55812eb5a0e0 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66400fa50 con 0x55812eb5a0e0 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa659ffb640 1 -- 172.21.15.90:0/726321509 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa66c05c050 con 0x55812eb5a0e0 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 10 client.15927 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55812e893620 con 0x55812eb5a0e0 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa659ffb640 1 -- 172.21.15.90:0/726321509 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66400fa50 con 0x55812eb5a0e0 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa659ffb640 1 -- 172.21.15.90:0/726321509 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa66c04a050 con 0x55812eb5a0e0 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 10 client.15927 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 10 client.15927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 20 client.15927 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812eb6a1c0 0x55812eb6c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 4 client.15927 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:17:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.435+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55812eac3070 con 0x55812eb6a1c0 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.436+0000 7fa67259c640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812eb6a1c0 0x55812eb6c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.436+0000 7fa659ffb640 10 client.15927 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.436+0000 7fa67259c640 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812eb6a1c0 0x55812eb6c610 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e45d8f85e393838 server_cookie=2adea3d6d232cec6 in_seq=0 out_seq=0 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa659ffb640 1 -- 172.21.15.90:0/726321509 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55812eac3070 con 0x55812eb6a1c0 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa659ffb640 10 client.15927 handle_command_reply: tid=0 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 client.15927 shutdown 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 msgr2=0x55812eb57a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb57a50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa66c003bf0 tx=0x7fa66c0095c0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 shutdown_connections 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812eb6a1c0 0x55812eb6c610 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb5a0e0 0x55812eb57a50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4f820 0x55812eb57510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa659ffb640 0 client.15927 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/726321509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb4ef00 0x55812eb56fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 >> 172.21.15.90:0/726321509 conn(0x55812eb4b410 msgr2=0x55812eb51140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 shutdown_connections 2024-04-26T18:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 1 -- 172.21.15.90:0/726321509 wait complete. 2024-04-26T18:17:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.437+0000 7fa67bbc7740 20 client.15927 trim_cache size 0 max 16384 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 Processor -- start 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 -- start start 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb65bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 0x55812eb71f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa6741b0c20 con 0x55812eb4f820 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6741ae4f0 con 0x55812eb5a0e0 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6741ac360 con 0x55812eb4ef00 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44098/0 (socket says 172.21.15.90:44098) 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa671d9b640 1 -- 172.21.15.90:0/2067013995 learned_addr learned my addr 172.21.15.90:0/2067013995 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa67259c640 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb65bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa671d9b640 1 -- 172.21.15.90:0/2067013995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 msgr2=0x55812eb65bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa671d9b640 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb65bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa671d9b640 1 -- 172.21.15.90:0/2067013995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 msgr2=0x55812eb71f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.452+0000 7fa671d9b640 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 0x55812eb71f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa671d9b640 1 -- 172.21.15.90:0/2067013995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55812eb502d0 con 0x55812eb4f820 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa671d9b640 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb518c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa668000c00 tx=0x7fa6680d5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598d31de2356a846 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa65bfff640 1 -- 172.21.15.90:0/2067013995 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66801d040 con 0x55812eb4f820 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa65bfff640 1 -- 172.21.15.90:0/2067013995 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66809c450 con 0x55812eb4f820 2024-04-26T18:17:15.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa65bfff640 1 -- 172.21.15.90:0/2067013995 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa668005270 con 0x55812eb4f820 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 -- 172.21.15.90:0/2067013995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 msgr2=0x55812eb518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb518c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa668000c00 tx=0x7fa6680d5040 comp rx=0 tx=0).stop 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 -- 172.21.15.90:0/2067013995 shutdown_connections 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 0x55812eb71f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb518c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/2067013995 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb65bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 -- 172.21.15.90:0/2067013995 >> 172.21.15.90:0/2067013995 conn(0x55812eb4b410 msgr2=0x55812eb522c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 -- 172.21.15.90:0/2067013995 shutdown_connections 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.453+0000 7fa67bbc7740 1 -- 172.21.15.90:0/2067013995 wait complete. 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 -- start start 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 Processor -- start 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb53e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 0x55812eb548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa6741ac9b0 con 0x55812eb4f820 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa6741ade20 con 0x55812eb5a0e0 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa6741ad080 con 0x55812eb4ef00 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67bbc7740 10 client.0 _ll_register_callbacks cb 0x7fa66c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa67259c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb53e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44114/0 (socket says 172.21.15.90:44114) 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 -- 172.21.15.90:0/204318671 learned_addr learned my addr 172.21.15.90:0/204318671 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa672d9d640 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 0x55812eb548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 -- 172.21.15.90:0/204318671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 msgr2=0x55812eb53e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb53e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 -- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 msgr2=0x55812eb548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 0x55812eb548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.454+0000 7fa671d9b640 1 -- 172.21.15.90:0/204318671 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55812ebcf990 con 0x55812eb4f820 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa671d9b640 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb543a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa6680988b0 tx=0x7fa6680a5010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa6d2fa442ff080 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa659ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa659ffb640 1 -- 172.21.15.90:0/204318671 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66801d070 con 0x55812eb4f820 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa659ffb640 1 -- 172.21.15.90:0/204318671 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7fa6680bb2a0 con 0x55812eb4f820 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa66809c450 con 0x55812eb4f820 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa659ffb640 1 -- 172.21.15.90:0/204318671 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa6680b5b20 con 0x55812eb4f820 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 10 client.15933 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55812ebd0c40 con 0x55812eb4f820 2024-04-26T18:17:15.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa659ffb640 1 -- 172.21.15.90:0/204318671 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa66809c450 con 0x55812eb4f820 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa659ffb640 1 -- 172.21.15.90:0/204318671 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa668098230 con 0x55812eb4f820 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 10 client.15933 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 10 client.15933 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 20 client.15933 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812ebd1300 0x55812ebd36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 4 client.15933 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55812eabcee0 con 0x55812ebd1300 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.455+0000 7fa67259c640 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812ebd1300 0x55812ebd36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.456+0000 7fa67259c640 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812ebd1300 0x55812ebd36f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6110ea55b6cf2906 server_cookie=3c3eaabdfb7d5604 in_seq=0 out_seq=0 2024-04-26T18:17:15.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.456+0000 7fa659ffb640 10 client.15933 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa659ffb640 1 -- 172.21.15.90:0/204318671 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55812eabcee0 con 0x55812ebd1300 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa659ffb640 10 client.15933 handle_command_reply: tid=0 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 client.15933 shutdown 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 msgr2=0x55812eb543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb543a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa6680988b0 tx=0x7fa6680a5010 comp rx=0 tx=0).stop 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 shutdown_connections 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55812ebd1300 0x55812ebd36f0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55812eb5a0e0 0x55812eb548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55812eb4f820 0x55812eb543a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa659ffb640 0 client.15933 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:15.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 --2- 172.21.15.90:0/204318671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55812eb4ef00 0x55812eb53e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 >> 172.21.15.90:0/204318671 conn(0x55812eb4b410 msgr2=0x55812eb58800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 shutdown_connections 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 1 -- 172.21.15.90:0/204318671 wait complete. 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.521+0000 7fa67bbc7740 20 client.15933 trim_cache size 0 max 16384 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:17:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:17:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa65c0a3dc0 msgr2=0x7fa65c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa65c0a3dc0 0x7fa65c0a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa66c02f500 tx=0x7fa66c02a8c0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 msgr2=0x7fa674199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa674199530 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c970 tx=0x7fa66800ce30 comp rx=0 tx=0).stop 2024-04-26T18:17:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 shutdown_connections 2024-04-26T18:17:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa65c0a3dc0 0x7fa65c0a6280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa674108950 0x7fa67419d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa67406bb40 0x7fa674199530 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 --2- 172.21.15.90:0/1915702662 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa67406b1a0 0x7fa674198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 >> 172.21.15.90:0/1915702662 conn(0x7fa674078580 msgr2=0x7fa674100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 shutdown_connections 2024-04-26T18:17:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:15.522+0000 7fa678eb0640 1 -- 172.21.15.90:0/1915702662 wait complete. 2024-04-26T18:17:15.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 263118 2.54 MB/sec execute 63 sec latency 85.864 ms 2024-04-26T18:17:16.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 266129 2.78 MB/sec execute 64 sec latency 17.705 ms 2024-04-26T18:17:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:16 smithi119 ceph-mon[32752]: pgmap v162: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 879 KiB/s wr, 131 op/s 2024-04-26T18:17:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:16 smithi167 ceph-mon[32751]: pgmap v162: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 879 KiB/s wr, 131 op/s 2024-04-26T18:17:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:16 smithi090 ceph-mon[29180]: pgmap v162: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 879 KiB/s wr, 131 op/s 2024-04-26T18:17:17.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 267548 2.84 MB/sec execute 65 sec latency 16.876 ms 2024-04-26T18:17:18.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 270017 3.01 MB/sec execute 66 sec latency 10.209 ms 2024-04-26T18:17:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:18 smithi119 ceph-mon[32752]: pgmap v163: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 879 KiB/s wr, 131 op/s 2024-04-26T18:17:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:18 smithi167 ceph-mon[32751]: pgmap v163: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 879 KiB/s wr, 131 op/s 2024-04-26T18:17:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:18 smithi090 ceph-mon[29180]: pgmap v163: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 879 KiB/s wr, 131 op/s 2024-04-26T18:17:19.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 272210 3.08 MB/sec execute 67 sec latency 15.527 ms 2024-04-26T18:17:20.669 INFO:tasks.workunit.client.0.smithi090.stdout: 1 273932 3.21 MB/sec execute 68 sec latency 11.183 ms 2024-04-26T18:17:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:20 smithi119 ceph-mon[32752]: pgmap v164: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-04-26T18:17:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:20 smithi167 ceph-mon[32751]: pgmap v164: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-04-26T18:17:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:20 smithi090 ceph-mon[29180]: pgmap v164: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 275 op/s 2024-04-26T18:17:21.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 276812 3.41 MB/sec execute 69 sec latency 18.941 ms 2024-04-26T18:17:22.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 278287 3.46 MB/sec execute 70 sec latency 16.670 ms 2024-04-26T18:17:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:22 smithi119 ceph-mon[32752]: pgmap v165: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-04-26T18:17:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:22 smithi167 ceph-mon[32751]: pgmap v165: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-04-26T18:17:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:22 smithi090 ceph-mon[29180]: pgmap v165: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 244 op/s 2024-04-26T18:17:23.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 280699 3.60 MB/sec execute 71 sec latency 10.536 ms 2024-04-26T18:17:24.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 283251 3.73 MB/sec execute 72 sec latency 13.563 ms 2024-04-26T18:17:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:24 smithi119 ceph-mon[32752]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 347 op/s 2024-04-26T18:17:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:24 smithi167 ceph-mon[32751]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 347 op/s 2024-04-26T18:17:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:24 smithi090 ceph-mon[29180]: pgmap v166: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 347 op/s 2024-04-26T18:17:25.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 284728 3.78 MB/sec execute 73 sec latency 15.608 ms 2024-04-26T18:17:26.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 287601 3.97 MB/sec execute 74 sec latency 16.620 ms 2024-04-26T18:17:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:26 smithi119 ceph-mon[32752]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-04-26T18:17:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:26 smithi167 ceph-mon[32751]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-04-26T18:17:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:26 smithi090 ceph-mon[29180]: pgmap v167: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-04-26T18:17:27.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 289701 4.01 MB/sec execute 75 sec latency 13.575 ms 2024-04-26T18:17:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 -- 172.21.15.90:0/837779571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 msgr2=0x7f051810e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 --2- 172.21.15.90:0/837779571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051810e710 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0510009cd0 tx=0x7f051002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 -- 172.21.15.90:0/837779571 shutdown_connections 2024-04-26T18:17:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 --2- 172.21.15.90:0/837779571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051810e710 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 --2- 172.21.15.90:0/837779571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0518101890 0x7f0518101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 --2- 172.21.15.90:0/837779571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0518107850 0x7f0518107c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 -- 172.21.15.90:0/837779571 >> 172.21.15.90:0/837779571 conn(0x7f0518078980 msgr2=0x7f0518078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 -- 172.21.15.90:0/837779571 shutdown_connections 2024-04-26T18:17:28.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 -- 172.21.15.90:0/837779571 wait complete. 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.614+0000 7f051d020640 1 Processor -- start 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051d020640 1 -- start start 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051d020640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0518101890 0x7f051819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051d020640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051d020640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0518107850 0x7f051819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051d020640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0518111b10 con 0x7f0518102250 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051d020640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0518111990 con 0x7f0518107850 2024-04-26T18:17:28.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051d020640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0518111c90 con 0x7f0518101890 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38554/0 (socket says 172.21.15.90:38554) 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 -- 172.21.15.90:0/204863942 learned_addr learned my addr 172.21.15.90:0/204863942 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f0516d9d640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0518101890 0x7f051819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051759e640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0518107850 0x7f051819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 -- 172.21.15.90:0/204863942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0518101890 msgr2=0x7f051819b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0518101890 0x7f051819b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 -- 172.21.15.90:0/204863942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0518107850 msgr2=0x7f051819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0518107850 0x7f051819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 -- 172.21.15.90:0/204863942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05181a0300 con 0x7f0518102250 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f0516d9d640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0518101890 0x7f051819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.615+0000 7f051659c640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051819b850 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9f0 tx=0x7f050c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.616+0000 7f04fffff640 1 -- 172.21.15.90:0/204863942 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c00cd80 con 0x7f0518102250 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.616+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05181a05f0 con 0x7f0518102250 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.616+0000 7f04fffff640 1 -- 172.21.15.90:0/204863942 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050c0054d0 con 0x7f0518102250 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.616+0000 7f04fffff640 1 -- 172.21.15.90:0/204863942 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c010690 con 0x7f0518102250 2024-04-26T18:17:28.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.616+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05181a7e80 con 0x7f0518102250 2024-04-26T18:17:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.618+0000 7f04fffff640 1 -- 172.21.15.90:0/204863942 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f050c004e90 con 0x7f0518102250 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.618+0000 7f04fffff640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f05000a3dc0 0x7f05000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.618+0000 7f0516d9d640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f05000a3dc0 0x7f05000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.618+0000 7f051fe14740 1 Processor -- start 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.618+0000 7f04fffff640 1 -- 172.21.15.90:0/204863942 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f050c0d0c30 con 0x7f0518102250 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051fe14740 1 -- start start 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051fe14740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2bf00 0x55bd76a2c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051fe14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2c820 0x55bd76a36ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051fe14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a370e0 0x55bd76a394d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051fe14740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd76992460 con 0x55bd76a2bf00 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051fe14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bd7694c590 con 0x55bd76a370e0 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051fe14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bd7698e420 con 0x55bd76a2c820 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f0516d9d640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f05000a3dc0 0x7f05000a6280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f05080058c0 tx=0x7f050800a2d0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f0516d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2bf00 0x55bd76a2c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2c820 0x55bd76a36ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f0516d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2bf00 0x55bd76a2c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38568/0 (socket says 172.21.15.90:38568) 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2c820 0x55bd76a36ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:55632/0 (socket says 172.21.15.90:55632) 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f0516d9d640 1 -- 172.21.15.90:0/2267474399 learned_addr learned my addr 172.21.15.90:0/2267474399 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051759e640 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a370e0 0x55bd76a394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 -- 172.21.15.90:0/2267474399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a370e0 msgr2=0x55bd76a394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a370e0 0x55bd76a394d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 -- 172.21.15.90:0/2267474399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2bf00 msgr2=0x55bd76a2c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2bf00 0x55bd76a2c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 -- 172.21.15.90:0/2267474399 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd769590d0 con 0x55bd76a2c820 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f0516d9d640 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2bf00 0x55bd76a2c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.619+0000 7f051659c640 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2c820 0x55bd76a36ba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f050c0ce3f0 tx=0x7f050c09c5a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=187cf230840d2e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:28.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f04fd7fa640 1 -- 172.21.15.90:0/2267474399 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c09cc40 con 0x55bd76a2c820 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f04fd7fa640 1 -- 172.21.15.90:0/2267474399 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050c087040 con 0x55bd76a2c820 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f04fd7fa640 1 -- 172.21.15.90:0/2267474399 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c0992b0 con 0x55bd76a2c820 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 -- 172.21.15.90:0/2267474399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2c820 msgr2=0x55bd76a36ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2c820 0x55bd76a36ba0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f050c0ce3f0 tx=0x7f050c09c5a0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 -- 172.21.15.90:0/2267474399 shutdown_connections 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a370e0 0x55bd76a394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2c820 0x55bd76a36ba0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 --2- 172.21.15.90:0/2267474399 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2bf00 0x55bd76a2c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 -- 172.21.15.90:0/2267474399 >> 172.21.15.90:0/2267474399 conn(0x55bd76a28410 msgr2=0x55bd76a287f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 -- 172.21.15.90:0/2267474399 shutdown_connections 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.620+0000 7f051fe14740 1 -- 172.21.15.90:0/2267474399 wait complete. 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 -- start start 2024-04-26T18:17:28.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 Processor -- start 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a34100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a34640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a34b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bd76a39d10 con 0x55bd76a2c820 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bd76919b60 con 0x55bd76a2bf00 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bd76a39e90 con 0x55bd76a370e0 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051fe14740 10 client.0 _ll_register_callbacks cb 0x55bd7693e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a34640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a34640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38578/0 (socket says 172.21.15.90:38578) 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 -- 172.21.15.90:0/1055068382 learned_addr learned my addr 172.21.15.90:0/1055068382 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051759e640 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a34b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f0516d9d640 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 -- 172.21.15.90:0/1055068382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 msgr2=0x55bd76a34b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a34b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 -- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 msgr2=0x55bd76a34100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a34100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.621+0000 7f051659c640 1 -- 172.21.15.90:0/1055068382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd766a4210 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051659c640 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a34640 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9c0 tx=0x7f050c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65d94f25a4891f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f04faff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f04faff5640 1 -- 172.21.15.90:0/1055068382 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c09cc40 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f04faff5640 1 -- 172.21.15.90:0/1055068382 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f050c0bd0c0 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050c087040 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f04faff5640 1 -- 172.21.15.90:0/1055068382 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f050c0a0ff0 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 10 client.15945 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd76770620 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f04faff5640 1 -- 172.21.15.90:0/1055068382 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050c087040 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f04faff5640 1 -- 172.21.15.90:0/1055068382 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f050c0a5520 con 0x55bd76a2c820 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 10 client.15945 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 20 client.15945 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76a49530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 4 client.15945 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:17:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd769a0070 con 0x55bd76a47140 2024-04-26T18:17:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.622+0000 7f0516d9d640 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76a49530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.623+0000 7f04faff5640 10 client.15945 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.623+0000 7f0516d9d640 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76a49530 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea34d3b43b9e1067 server_cookie=87e582e4ba551a7d in_seq=0 out_seq=0 2024-04-26T18:17:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f04faff5640 1 -- 172.21.15.90:0/1055068382 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bd769a0070 con 0x55bd76a47140 2024-04-26T18:17:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f04faff5640 10 client.15945 handle_command_reply: tid=0 2024-04-26T18:17:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 client.15945 shutdown 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 msgr2=0x55bd76a34640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a34640 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9c0 tx=0x7f050c0046c0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 shutdown_connections 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76a49530 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a34b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a34640 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f04faff5640 0 client.15945 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 --2- 172.21.15.90:0/1055068382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a34100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 >> 172.21.15.90:0/1055068382 conn(0x55bd76a28410 msgr2=0x55bd76a2e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 shutdown_connections 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 1 -- 172.21.15.90:0/1055068382 wait complete. 2024-04-26T18:17:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.625+0000 7f051fe14740 20 client.15945 trim_cache size 0 max 16384 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 Processor -- start 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 -- start start 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a2e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a4f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f05181b2df0 con 0x55bd76a2c820 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f05181b06c0 con 0x55bd76a2bf00 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051fe14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05181ae510 con 0x55bd76a370e0 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a4eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38580/0 (socket says 172.21.15.90:38580) 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051659c640 1 -- 172.21.15.90:0/301566593 learned_addr learned my addr 172.21.15.90:0/301566593 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f0516d9d640 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a2e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051759e640 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a4f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051659c640 1 -- 172.21.15.90:0/301566593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 msgr2=0x55bd76a4f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051659c640 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a4f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051659c640 1 -- 172.21.15.90:0/301566593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 msgr2=0x55bd76a2e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.640+0000 7f051659c640 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a2e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051659c640 1 -- 172.21.15.90:0/301566593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd76a32650 con 0x55bd76a2c820 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051759e640 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a4f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:17:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051659c640 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a4eb00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f050c09c940 tx=0x7f050c09c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3302754f80be4f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f04fcff9640 1 -- 172.21.15.90:0/301566593 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c005180 con 0x55bd76a2c820 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f04fcff9640 1 -- 172.21.15.90:0/301566593 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050c087070 con 0x55bd76a2c820 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f04fcff9640 1 -- 172.21.15.90:0/301566593 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f050c0a5420 con 0x55bd76a2c820 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 -- 172.21.15.90:0/301566593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 msgr2=0x55bd76a4eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a4eb00 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f050c09c940 tx=0x7f050c09c5a0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 -- 172.21.15.90:0/301566593 shutdown_connections 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a370e0 0x55bd76a4f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a2c820 0x55bd76a4eb00 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 --2- 172.21.15.90:0/301566593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2bf00 0x55bd76a2e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 -- 172.21.15.90:0/301566593 >> 172.21.15.90:0/301566593 conn(0x55bd76a28410 msgr2=0x55bd76a275b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 -- 172.21.15.90:0/301566593 shutdown_connections 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.641+0000 7f051fe14740 1 -- 172.21.15.90:0/301566593 wait complete. 2024-04-26T18:17:28.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 -- start start 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 Processor -- start 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2bf00 0x55bd76a30db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2c820 0x55bd76a312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a370e0 0x55bd76a31830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f05181aeb80 con 0x55bd76a370e0 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f05181afff0 con 0x55bd76a2c820 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05181af250 con 0x55bd76a2bf00 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051fe14740 10 client.0 _ll_register_callbacks cb 0x7f050800ab20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a370e0 0x55bd76a31830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a370e0 0x55bd76a31830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38590/0 (socket says 172.21.15.90:38590) 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 -- 172.21.15.90:0/811289689 learned_addr learned my addr 172.21.15.90:0/811289689 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f0516d9d640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2bf00 0x55bd76a30db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051659c640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2c820 0x55bd76a312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 -- 172.21.15.90:0/811289689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2bf00 msgr2=0x55bd76a30db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2bf00 0x55bd76a30db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 -- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2c820 msgr2=0x55bd76a312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2c820 0x55bd76a312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f051759e640 1 -- 172.21.15.90:0/811289689 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd76aac370 con 0x55bd76a370e0 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.642+0000 7f0516d9d640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2bf00 0x55bd76a30db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051759e640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a370e0 0x55bd76a31830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0510009cd0 tx=0x7f0510031bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1ad795f21566ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f04faff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f04faff5640 1 -- 172.21.15.90:0/811289689 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510003d80 con 0x55bd76a370e0 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f04faff5640 1 -- 172.21.15.90:0/811289689 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f0510004040 con 0x55bd76a370e0 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f050c087070 con 0x55bd76a370e0 2024-04-26T18:17:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f04faff5640 1 -- 172.21.15.90:0/811289689 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0510004240 con 0x55bd76a370e0 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 10 client.15951 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bd76aad660 con 0x55bd76a370e0 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f04faff5640 1 -- 172.21.15.90:0/811289689 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f050c087070 con 0x55bd76a370e0 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f04faff5640 1 -- 172.21.15.90:0/811289689 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0510030070 con 0x55bd76a370e0 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 10 client.15951 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 20 client.15951 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76aafea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 4 client.15951 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bd76999ee0 con 0x55bd76a47140 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.643+0000 7f0516d9d640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76aafea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.644+0000 7f0516d9d640 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76aafea0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2af4ff53c9a2fe2 server_cookie=e006fa73d43137d1 in_seq=0 out_seq=0 2024-04-26T18:17:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.644+0000 7f04faff5640 10 client.15951 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:28.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 291430 4.12 MB/sec execute 76 sec latency 17.425 ms 2024-04-26T18:17:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f04faff5640 1 -- 172.21.15.90:0/811289689 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bd76999ee0 con 0x55bd76a47140 2024-04-26T18:17:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f04faff5640 10 client.15951 handle_command_reply: tid=0 2024-04-26T18:17:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 client.15951 shutdown 2024-04-26T18:17:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a370e0 msgr2=0x55bd76a31830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a370e0 0x55bd76a31830 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0510009cd0 tx=0x7f0510031bc0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 shutdown_connections 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bd76a47140 0x55bd76aafea0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f04faff5640 0 client.15951 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bd76a370e0 0x55bd76a31830 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bd76a2c820 0x55bd76a312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 --2- 172.21.15.90:0/811289689 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bd76a2bf00 0x55bd76a30db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 >> 172.21.15.90:0/811289689 conn(0x55bd76a28410 msgr2=0x55bd76a340a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 shutdown_connections 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 1 -- 172.21.15.90:0/811289689 wait complete. 2024-04-26T18:17:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.713+0000 7f051fe14740 20 client.15951 trim_cache size 0 max 16384 2024-04-26T18:17:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:17:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:17:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:17:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:17:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f05000a3dc0 msgr2=0x7f05000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f05000a3dc0 0x7f05000a6280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f05080058c0 tx=0x7f050800a2d0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 msgr2=0x7f051819b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051819b850 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f050c00e9f0 tx=0x7f050c00edf0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 shutdown_connections 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f05000a3dc0 0x7f05000a6280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0518107850 0x7f051819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0518102250 0x7f051819b850 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 --2- 172.21.15.90:0/204863942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0518101890 0x7f051819b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 >> 172.21.15.90:0/204863942 conn(0x7f0518078980 msgr2=0x7f051810cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 shutdown_connections 2024-04-26T18:17:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:28.714+0000 7f051d020640 1 -- 172.21.15.90:0/204863942 wait complete. 2024-04-26T18:17:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:28 smithi119 ceph-mon[32752]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-04-26T18:17:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:28 smithi167 ceph-mon[32751]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-04-26T18:17:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:28 smithi090 ceph-mon[29180]: pgmap v168: 97 pgs: 97 active+clean; 80 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-04-26T18:17:29.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 293861 4.23 MB/sec execute 77 sec latency 69.519 ms 2024-04-26T18:17:30.670 INFO:tasks.workunit.client.0.smithi090.stdout: 1 295417 4.27 MB/sec execute 78 sec latency 16.233 ms 2024-04-26T18:17:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:30 smithi119 ceph-mon[32752]: pgmap v169: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-04-26T18:17:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:30 smithi167 ceph-mon[32751]: pgmap v169: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-04-26T18:17:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:30 smithi090 ceph-mon[29180]: pgmap v169: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-04-26T18:17:31.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 298214 4.44 MB/sec execute 79 sec latency 11.628 ms 2024-04-26T18:17:32.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 299665 4.46 MB/sec execute 80 sec latency 15.711 ms 2024-04-26T18:17:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:32 smithi119 ceph-mon[32752]: pgmap v170: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-04-26T18:17:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:32 smithi167 ceph-mon[32751]: pgmap v170: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-04-26T18:17:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:32 smithi090 ceph-mon[29180]: pgmap v170: 97 pgs: 97 active+clean; 87 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 207 op/s 2024-04-26T18:17:33.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 301958 4.57 MB/sec execute 81 sec latency 21.491 ms 2024-04-26T18:17:34.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 304087 4.62 MB/sec execute 82 sec latency 16.652 ms 2024-04-26T18:17:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:34 smithi119 ceph-mon[32752]: pgmap v171: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T18:17:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:34 smithi167 ceph-mon[32751]: pgmap v171: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T18:17:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:34 smithi090 ceph-mon[29180]: pgmap v171: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.2 MiB/s wr, 359 op/s 2024-04-26T18:17:35.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 305747 4.70 MB/sec execute 83 sec latency 12.825 ms 2024-04-26T18:17:36.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 308592 4.85 MB/sec execute 84 sec latency 12.779 ms 2024-04-26T18:17:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:36 smithi119 ceph-mon[32752]: pgmap v172: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-26T18:17:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:36 smithi167 ceph-mon[32751]: pgmap v172: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-26T18:17:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:36 smithi090 ceph-mon[29180]: pgmap v172: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-26T18:17:37.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 309865 4.83 MB/sec execute 85 sec latency 16.794 ms 2024-04-26T18:17:38.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 312404 4.97 MB/sec execute 86 sec latency 19.287 ms 2024-04-26T18:17:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:38 smithi119 ceph-mon[32752]: pgmap v173: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-26T18:17:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:38 smithi167 ceph-mon[32751]: pgmap v173: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-26T18:17:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:38 smithi090 ceph-mon[29180]: pgmap v173: 97 pgs: 97 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.5 MiB/s wr, 255 op/s 2024-04-26T18:17:39.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 314150 4.98 MB/sec execute 87 sec latency 15.390 ms 2024-04-26T18:17:40.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 316300 5.08 MB/sec execute 88 sec latency 12.075 ms 2024-04-26T18:17:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:40 smithi119 ceph-mon[32752]: pgmap v174: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 352 op/s 2024-04-26T18:17:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:40 smithi167 ceph-mon[32751]: pgmap v174: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 352 op/s 2024-04-26T18:17:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:40 smithi090 ceph-mon[29180]: pgmap v174: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 352 op/s 2024-04-26T18:17:41.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 319255 5.22 MB/sec execute 89 sec latency 11.959 ms 2024-04-26T18:17:41.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 -- 172.21.15.90:0/2324541979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 msgr2=0x7f32a810f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 --2- 172.21.15.90:0/2324541979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a810f7f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009cd0 tx=0x7f32a002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 -- 172.21.15.90:0/2324541979 shutdown_connections 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 --2- 172.21.15.90:0/2324541979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a810f7f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 --2- 172.21.15.90:0/2324541979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32a8102920 0x7f32a8102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 --2- 172.21.15.90:0/2324541979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a8108930 0x7f32a8108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 -- 172.21.15.90:0/2324541979 >> 172.21.15.90:0/2324541979 conn(0x7f32a80fe3f0 msgr2=0x7f32a8100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 -- 172.21.15.90:0/2324541979 shutdown_connections 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.802+0000 7f32ace47640 1 -- 172.21.15.90:0/2324541979 wait complete. 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 Processor -- start 2024-04-26T18:17:41.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 -- start start 2024-04-26T18:17:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a8102920 0x7f32a8199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32a8108930 0x7f32a819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8112bc0 con 0x7f32a81032e0 2024-04-26T18:17:41.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8112a40 con 0x7f32a8102920 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32ace47640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32a8112d40 con 0x7f32a8108930 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a81995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39700/0 (socket says 172.21.15.90:39700) 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 -- 172.21.15.90:0/3025671578 learned_addr learned my addr 172.21.15.90:0/3025671578 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a659c640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a8102920 0x7f32a8199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32a8108930 0x7f32a819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 -- 172.21.15.90:0/3025671578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32a8108930 msgr2=0x7f32a819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32a8108930 0x7f32a819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 -- 172.21.15.90:0/3025671578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a8102920 msgr2=0x7f32a8199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a8102920 0x7f32a8199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a5d9b640 1 -- 172.21.15.90:0/3025671578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a819e050 con 0x7f32a81032e0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.803+0000 7f32a659c640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a8102920 0x7f32a8199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.804+0000 7f32a5d9b640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a81995a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9f0 tx=0x7f329c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.804+0000 7f32977fe640 1 -- 172.21.15.90:0/3025671578 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c00cd80 con 0x7f32a81032e0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.804+0000 7f32977fe640 1 -- 172.21.15.90:0/3025671578 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c0108e0 con 0x7f32a81032e0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.804+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a819e340 con 0x7f32a81032e0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.804+0000 7f32977fe640 1 -- 172.21.15.90:0/3025671578 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c018980 con 0x7f32a81032e0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.804+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81a5bd0 con 0x7f32a81032e0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.806+0000 7f32977fe640 1 -- 172.21.15.90:0/3025671578 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f329c010430 con 0x7f32a81032e0 2024-04-26T18:17:41.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.806+0000 7f32af7fd740 1 Processor -- start 2024-04-26T18:17:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.806+0000 7f32af7fd740 1 -- start start 2024-04-26T18:17:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766df00 0x55c64766e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32977fe640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f32900a3dc0 0x7f32900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c647678ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c64767b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32af7fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c6475d4460 con 0x55c6476790e0 2024-04-26T18:17:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32af7fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c64758e590 con 0x55c64766e820 2024-04-26T18:17:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32af7fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c6475d0420 con 0x55c64766df00 2024-04-26T18:17:41.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f32900a3dc0 0x7f32900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c647678ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c64767b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766df00 0x55c64766e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c64767b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39708/0 (socket says 172.21.15.90:39708) 2024-04-26T18:17:41.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766df00 0x55c64766e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:46166/0 (socket says 172.21.15.90:46166) 2024-04-26T18:17:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a6d9d640 1 -- 172.21.15.90:0/1958179699 learned_addr learned my addr 172.21.15.90:0/1958179699 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32977fe640 1 -- 172.21.15.90:0/3025671578 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f329c0d0a50 con 0x7f32a81032e0 2024-04-26T18:17:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c647678ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38414/0 (socket says 172.21.15.90:38414) 2024-04-26T18:17:41.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 -- 172.21.15.90:0/1958179699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 msgr2=0x55c647678ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c647678ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 -- 172.21.15.90:0/1958179699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 msgr2=0x55c64767b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c64767b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 -- 172.21.15.90:0/1958179699 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64759b0d0 con 0x55c64766df00 2024-04-26T18:17:41.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c64767b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f32900a3dc0 0x7f32900a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55c64766b830 tx=0x7f32a0004a40 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.807+0000 7f32a659c640 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766df00 0x55c64766e2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3298009f50 tx=0x7f32980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ce55b1797ee064 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f3294ff9640 1 -- 172.21.15.90:0/1958179699 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298004030 con 0x55c64766df00 2024-04-26T18:17:41.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f3294ff9640 1 -- 172.21.15.90:0/1958179699 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3298005020 con 0x55c64766df00 2024-04-26T18:17:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f3294ff9640 1 -- 172.21.15.90:0/1958179699 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298004c20 con 0x55c64766df00 2024-04-26T18:17:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 -- 172.21.15.90:0/1958179699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766df00 msgr2=0x55c64766e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766df00 0x55c64766e2e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3298009f50 tx=0x7f32980118d0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 -- 172.21.15.90:0/1958179699 shutdown_connections 2024-04-26T18:17:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c64767b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c647678ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1958179699 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766df00 0x55c64766e2e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 -- 172.21.15.90:0/1958179699 >> 172.21.15.90:0/1958179699 conn(0x55c64766a410 msgr2=0x55c64766a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:41.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 -- 172.21.15.90:0/1958179699 shutdown_connections 2024-04-26T18:17:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.808+0000 7f32af7fd740 1 -- 172.21.15.90:0/1958179699 wait complete. 2024-04-26T18:17:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 -- start start 2024-04-26T18:17:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 Processor -- start 2024-04-26T18:17:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c647675f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647676450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c647676990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55c64767bd10 con 0x55c6476790e0 2024-04-26T18:17:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55c64755bb60 con 0x55c64766df00 2024-04-26T18:17:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c64767be90 con 0x55c64766e820 2024-04-26T18:17:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32af7fd740 10 client.0 _ll_register_callbacks cb 0x55c6475802b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:41.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c647676990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a5d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647676450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c647676990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39714/0 (socket says 172.21.15.90:39714) 2024-04-26T18:17:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 -- 172.21.15.90:0/1291313075 learned_addr learned my addr 172.21.15.90:0/1291313075 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c647675f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 -- 172.21.15.90:0/1291313075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 msgr2=0x55c647676450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647676450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 -- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 msgr2=0x55c647675f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c647675f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.831 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a6d9d640 1 -- 172.21.15.90:0/1291313075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6472e6210 con 0x55c6476790e0 2024-04-26T18:17:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.809+0000 7f32a659c640 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c647675f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c647676990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f32a0033040 tx=0x7f32a002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d494849772de808 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f328dffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f328dffb640 1 -- 172.21.15.90:0/1291313075 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0044040 con 0x55c6476790e0 2024-04-26T18:17:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f328dffb640 1 -- 172.21.15.90:0/1291313075 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f32a0004510 con 0x55c6476790e0 2024-04-26T18:17:41.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298005020 con 0x55c6476790e0 2024-04-26T18:17:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f328dffb640 1 -- 172.21.15.90:0/1291313075 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32a0004710 con 0x55c6476790e0 2024-04-26T18:17:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 10 client.15963 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6473b2620 con 0x55c6476790e0 2024-04-26T18:17:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f328dffb640 1 -- 172.21.15.90:0/1291313075 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3298005020 con 0x55c6476790e0 2024-04-26T18:17:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f328dffb640 1 -- 172.21.15.90:0/1291313075 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f32a0030070 con 0x55c6476790e0 2024-04-26T18:17:41.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 20 client.15963 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476890b0 0x55c64768b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 4 client.15963 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.810+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6475e2070 con 0x55c6476890b0 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.811+0000 7f32a659c640 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476890b0 0x55c64768b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.811+0000 7f32a659c640 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476890b0 0x55c64768b4a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b165ebac6f122885 server_cookie=2bf950eb4a7a5555 in_seq=0 out_seq=0 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.811+0000 7f328dffb640 10 client.15963 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:41.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.811+0000 7f328dffb640 1 -- 172.21.15.90:0/1291313075 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6475e2070 con 0x55c6476890b0 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.811+0000 7f328dffb640 10 client.15963 handle_command_reply: tid=0 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.811+0000 7f32af7fd740 1 client.15963 shutdown 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 msgr2=0x55c647676990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c647676990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f32a0033040 tx=0x7f32a002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 shutdown_connections 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476890b0 0x55c64768b4a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c647676990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647676450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f328dffb640 0 client.15963 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 --2- 172.21.15.90:0/1291313075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c647675f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 >> 172.21.15.90:0/1291313075 conn(0x55c64766a410 msgr2=0x55c647670140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 shutdown_connections 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 1 -- 172.21.15.90:0/1291313075 wait complete. 2024-04-26T18:17:41.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.812+0000 7f32af7fd740 20 client.15963 trim_cache size 0 max 16384 2024-04-26T18:17:41.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.826+0000 7f32af7fd740 1 Processor -- start 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.826+0000 7f32af7fd740 1 -- start start 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c6476708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647690db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c6476912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32af7fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81b0cc0 con 0x55c6476790e0 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32af7fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81ae590 con 0x55c64766df00 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32af7fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81ac400 con 0x55c64766e820 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c6476912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c6476708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c6476708c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38434/0 (socket says 172.21.15.90:38434) 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c6476912f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39718/0 (socket says 172.21.15.90:39718) 2024-04-26T18:17:41.841 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a659c640 1 -- 172.21.15.90:0/688169082 learned_addr learned my addr 172.21.15.90:0/688169082 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a5d9b640 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647690db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 -- 172.21.15.90:0/688169082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 msgr2=0x55c647690db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647690db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 -- 172.21.15.90:0/688169082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 msgr2=0x55c6476708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c6476708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 -- 172.21.15.90:0/688169082 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c64766e650 con 0x55c6476790e0 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a659c640 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c6476708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c6476912f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f32a00317f0 tx=0x7f32a0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5102ccabd98c876a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f328ffff640 1 -- 172.21.15.90:0/688169082 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0061030 con 0x55c6476790e0 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f328ffff640 1 -- 172.21.15.90:0/688169082 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a0036070 con 0x55c6476790e0 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.827+0000 7f328ffff640 1 -- 172.21.15.90:0/688169082 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00498d0 con 0x55c6476790e0 2024-04-26T18:17:41.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 -- 172.21.15.90:0/688169082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 msgr2=0x55c6476912f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c6476912f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f32a00317f0 tx=0x7f32a0005fd0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 -- 172.21.15.90:0/688169082 shutdown_connections 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c6476790e0 0x55c6476912f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c64766e820 0x55c647690db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 --2- 172.21.15.90:0/688169082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766df00 0x55c6476708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 -- 172.21.15.90:0/688169082 >> 172.21.15.90:0/688169082 conn(0x55c64766a410 msgr2=0x55c6476711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 -- 172.21.15.90:0/688169082 shutdown_connections 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 -- 172.21.15.90:0/688169082 wait complete. 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 -- start start 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.828+0000 7f32af7fd740 1 Processor -- start 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c64766df00 0x55c647679e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c64767a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.843 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6476790e0 0x55c64767a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81aca50 con 0x55c64766df00 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81adec0 con 0x55c64766e820 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f32a81ad120 con 0x55c6476790e0 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 10 client.0 _ll_register_callbacks cb 0x7f32a0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c64766df00 0x55c647679e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c64766df00 0x55c647679e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39734/0 (socket says 172.21.15.90:39734) 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 -- 172.21.15.90:0/3473794674 learned_addr learned my addr 172.21.15.90:0/3473794674 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a5d9b640 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c64767a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a6d9d640 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6476790e0 0x55c64767a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.844 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 -- 172.21.15.90:0/3473794674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6476790e0 msgr2=0x55c64767a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6476790e0 0x55c64767a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 -- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 msgr2=0x55c64767a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c64767a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 -- 172.21.15.90:0/3473794674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c64767f220 con 0x55c64766df00 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32a659c640 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c64766df00 0x55c647679e30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f329801b040 tx=0x7f3298005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d732732f077102 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f328dffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f328dffb640 1 -- 172.21.15.90:0/3473794674 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298004420 con 0x55c64766df00 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f328dffb640 1 -- 172.21.15.90:0/3473794674 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f329801f040 con 0x55c64766df00 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a0036070 con 0x55c64766df00 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f328dffb640 1 -- 172.21.15.90:0/3473794674 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f32980048b0 con 0x55c64766df00 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 10 client.15969 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:41.845 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.829+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6476ef830 con 0x55c64766df00 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f328dffb640 1 -- 172.21.15.90:0/3473794674 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a0036070 con 0x55c64766df00 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f328dffb640 1 -- 172.21.15.90:0/3473794674 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f3298003890 con 0x55c64766df00 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32af7fd740 10 client.15969 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32af7fd740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32af7fd740 20 client.15969 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32af7fd740 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476efd60 0x55c6476f2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32af7fd740 4 client.15969 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6475dbee0 con 0x55c6476efd60 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32a5d9b640 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476efd60 0x55c6476f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f328dffb640 10 client.15969 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:41.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.830+0000 7f32a5d9b640 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476efd60 0x55c6476f2150 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d433d80fdfd3b0 server_cookie=42a23e8e48ea09fe in_seq=0 out_seq=0 2024-04-26T18:17:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f328dffb640 1 -- 172.21.15.90:0/3473794674 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c6475dbee0 con 0x55c6476efd60 2024-04-26T18:17:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f328dffb640 10 client.15969 handle_command_reply: tid=0 2024-04-26T18:17:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 client.15969 shutdown 2024-04-26T18:17:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c64766df00 msgr2=0x55c647679e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c64766df00 0x55c647679e30 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f329801b040 tx=0x7f3298005710 comp rx=0 tx=0).stop 2024-04-26T18:17:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 shutdown_connections 2024-04-26T18:17:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55c6476efd60 0x55c6476f2150 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c6476790e0 0x55c64767a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c64766e820 0x55c64767a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 --2- 172.21.15.90:0/3473794674 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55c64766df00 0x55c647679e30 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f328dffb640 0 client.15969 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 >> 172.21.15.90:0/3473794674 conn(0x55c64766a410 msgr2=0x55c647675ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 shutdown_connections 2024-04-26T18:17:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 1 -- 172.21.15.90:0/3473794674 wait complete. 2024-04-26T18:17:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.896+0000 7f32af7fd740 20 client.15969 trim_cache size 0 max 16384 2024-04-26T18:17:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:17:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:17:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:17:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:17:41.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f32900a3dc0 msgr2=0x7f32900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f32900a3dc0 0x7f32900a6280 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55c64766b830 tx=0x7f32a0004a40 comp rx=0 tx=0).stop 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 msgr2=0x7f32a81995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a81995a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e9f0 tx=0x7f329c00eeb0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 shutdown_connections 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f32900a3dc0 0x7f32900a6280 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f32a8108930 0x7f32a819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f32a81032e0 0x7f32a81995a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 --2- 172.21.15.90:0/3025671578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32a8102920 0x7f32a8199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 >> 172.21.15.90:0/3025671578 conn(0x7f32a80fe3f0 msgr2=0x7f32a80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 shutdown_connections 2024-04-26T18:17:41.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:41.897+0000 7f32ace47640 1 -- 172.21.15.90:0/3025671578 wait complete. 2024-04-26T18:17:42.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 320631 5.24 MB/sec execute 90 sec latency 54.015 ms 2024-04-26T18:17:43.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:42 smithi090 ceph-mon[29180]: pgmap v175: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-04-26T18:17:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:42 smithi119 ceph-mon[32752]: pgmap v175: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-04-26T18:17:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:42 smithi167 ceph-mon[32751]: pgmap v175: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 248 op/s 2024-04-26T18:17:43.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 323167 5.33 MB/sec execute 91 sec latency 24.126 ms 2024-04-26T18:17:44.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 325243 5.35 MB/sec execute 92 sec latency 36.682 ms 2024-04-26T18:17:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:44 smithi119 ceph-mon[32752]: pgmap v176: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-04-26T18:17:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:44 smithi167 ceph-mon[32751]: pgmap v176: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-04-26T18:17:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:44 smithi090 ceph-mon[29180]: pgmap v176: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-04-26T18:17:45.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 327082 5.43 MB/sec execute 93 sec latency 29.454 ms 2024-04-26T18:17:46.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 330022 5.55 MB/sec execute 94 sec latency 14.829 ms 2024-04-26T18:17:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:46 smithi119 ceph-mon[32752]: pgmap v177: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T18:17:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:46 smithi167 ceph-mon[32751]: pgmap v177: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T18:17:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:46 smithi090 ceph-mon[29180]: pgmap v177: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T18:17:47.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 331479 5.56 MB/sec execute 95 sec latency 14.116 ms 2024-04-26T18:17:48.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 333831 5.65 MB/sec execute 96 sec latency 9.352 ms 2024-04-26T18:17:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:48 smithi119 ceph-mon[32752]: pgmap v178: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T18:17:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:48 smithi167 ceph-mon[32751]: pgmap v178: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T18:17:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:48 smithi090 ceph-mon[29180]: pgmap v178: 97 pgs: 97 active+clean; 92 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.7 MiB/s wr, 260 op/s 2024-04-26T18:17:49.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 335805 5.67 MB/sec execute 97 sec latency 54.731 ms 2024-04-26T18:17:50.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 337548 5.73 MB/sec execute 98 sec latency 19.965 ms 2024-04-26T18:17:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:50 smithi119 ceph-mon[32752]: pgmap v179: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 386 op/s 2024-04-26T18:17:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:50 smithi167 ceph-mon[32751]: pgmap v179: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 386 op/s 2024-04-26T18:17:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:50 smithi090 ceph-mon[29180]: pgmap v179: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.2 MiB/s wr, 386 op/s 2024-04-26T18:17:51.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 340186 5.81 MB/sec execute 99 sec latency 12.801 ms 2024-04-26T18:17:52.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 341586 5.83 MB/sec execute 100 sec latency 38.192 ms 2024-04-26T18:17:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:52 smithi119 ceph-mon[32752]: pgmap v180: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 289 op/s 2024-04-26T18:17:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:52 smithi167 ceph-mon[32751]: pgmap v180: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 289 op/s 2024-04-26T18:17:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:52 smithi090 ceph-mon[29180]: pgmap v180: 97 pgs: 97 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.5 MiB/s wr, 289 op/s 2024-04-26T18:17:53.671 INFO:tasks.workunit.client.0.smithi090.stdout: 1 344313 5.94 MB/sec execute 101 sec latency 14.918 ms 2024-04-26T18:17:54.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 345759 5.93 MB/sec execute 102 sec latency 47.868 ms 2024-04-26T18:17:54.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 -- 172.21.15.90:0/1364069127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4102210 msgr2=0x7f4af410e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1364069127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4102210 0x7f4af410e720 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009cd0 tx=0x7f4aec02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 -- 172.21.15.90:0/1364069127 shutdown_connections 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1364069127 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4102210 0x7f4af410e720 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1364069127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af4101850 0x7f4af4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1364069127 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af4107860 0x7f4af4107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 -- 172.21.15.90:0/1364069127 >> 172.21.15.90:0/1364069127 conn(0x7f4af4077bb0 msgr2=0x7f4af4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 -- 172.21.15.90:0/1364069127 shutdown_connections 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.986+0000 7f4af8bda640 1 -- 172.21.15.90:0/1364069127 wait complete. 2024-04-26T18:17:54.989 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 Processor -- start 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 -- start start 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4101850 0x7f4af4198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af4102210 0x7f4af4199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af4107860 0x7f4af419d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4111b20 con 0x7f4af4101850 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41119a0 con 0x7f4af4107860 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af8bda640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4af4111ca0 con 0x7f4af4102210 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4101850 0x7f4af4198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4101850 0x7f4af4198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47046/0 (socket says 172.21.15.90:47046) 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 -- 172.21.15.90:0/1389592596 learned_addr learned my addr 172.21.15.90:0/1389592596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af2d9d640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af4107860 0x7f4af419d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:54.990 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af4102210 0x7f4af4199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 -- 172.21.15.90:0/1389592596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af4102210 msgr2=0x7f4af4199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af4102210 0x7f4af4199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 -- 172.21.15.90:0/1389592596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af4107860 msgr2=0x7f4af419d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af4107860 0x7f4af419d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.987+0000 7f4af259c640 1 -- 172.21.15.90:0/1389592596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4af419dfe0 con 0x7f4af4101850 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.988+0000 7f4af259c640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4101850 0x7f4af4198ff0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400e9f0 tx=0x7f4ae400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.988+0000 7f4ae37fe640 1 -- 172.21.15.90:0/1389592596 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400cd80 con 0x7f4af4101850 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.988+0000 7f4ae37fe640 1 -- 172.21.15.90:0/1389592596 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae40054d0 con 0x7f4af4101850 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.988+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4af419e2d0 con 0x7f4af4101850 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.988+0000 7f4ae37fe640 1 -- 172.21.15.90:0/1389592596 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4010690 con 0x7f4af4101850 2024-04-26T18:17:54.991 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.988+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4af41a5b60 con 0x7f4af4101850 2024-04-26T18:17:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4ae37fe640 1 -- 172.21.15.90:0/1389592596 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f4ae4004e90 con 0x7f4af4101850 2024-04-26T18:17:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 Processor -- start 2024-04-26T18:17:54.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 -- start start 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4ae37fe640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4adc0a3dc0 0x7f4adc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa2311c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa2428e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5638aa197460 con 0x5638aa2404f0 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5638aa151590 con 0x5638aa230de0 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5638aa193420 con 0x5638aa231700 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.991+0000 7f4af2d9d640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4adc0a3dc0 0x7f4adc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4ae37fe640 1 -- 172.21.15.90:0/1389592596 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4ae40d0b10 con 0x7f4af4101850 2024-04-26T18:17:54.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa2311c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa2311c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:59568/0 (socket says 172.21.15.90:59568) 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa2428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 -- 172.21.15.90:0/37673206 learned_addr learned my addr 172.21.15.90:0/37673206 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 -- 172.21.15.90:0/37673206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 msgr2=0x5638aa23ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 -- 172.21.15.90:0/37673206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 msgr2=0x5638aa2428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa2428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 -- 172.21.15.90:0/37673206 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638aa15e0d0 con 0x5638aa230de0 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af2d9d640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4adc0a3dc0 0x7f4adc0a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4aec004af0 tx=0x7f4aec004a40 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:55.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4af259c640 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa2311c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4098f80 tx=0x7f4ae409c660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8164cb8ffb4fd7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4ae0ff9640 1 -- 172.21.15.90:0/37673206 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae409d220 con 0x5638aa230de0 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4ae0ff9640 1 -- 172.21.15.90:0/37673206 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae409de50 con 0x5638aa230de0 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.992+0000 7f4ae0ff9640 1 -- 172.21.15.90:0/37673206 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40a0ed0 con 0x5638aa230de0 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 -- 172.21.15.90:0/37673206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 msgr2=0x5638aa2311c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa2311c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4098f80 tx=0x7f4ae409c660 comp rx=0 tx=0).stop 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 -- 172.21.15.90:0/37673206 shutdown_connections 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa2428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/37673206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa2311c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 -- 172.21.15.90:0/37673206 >> 172.21.15.90:0/37673206 conn(0x5638aa22d410 msgr2=0x5638aa22d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 -- 172.21.15.90:0/37673206 shutdown_connections 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 -- 172.21.15.90:0/37673206 wait complete. 2024-04-26T18:17:55.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 -- start start 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.993+0000 7f4afb9c8740 1 Processor -- start 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa23d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa2404f0 0x5638aa23deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5638aa243120 con 0x5638aa230de0 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5638aa11eb60 con 0x5638aa2404f0 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5638aa2432a0 con 0x5638aa231700 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 10 client.0 _ll_register_callbacks cb 0x5638aa1432b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa23d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa23d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47072/0 (socket says 172.21.15.90:47072) 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 -- 172.21.15.90:0/2622600179 learned_addr learned my addr 172.21.15.90:0/2622600179 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:55.002 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af2d9d640 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa2404f0 0x5638aa23deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 -- 172.21.15.90:0/2622600179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 msgr2=0x5638aa23d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 -- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa2404f0 msgr2=0x5638aa23deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa2404f0 0x5638aa23deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 -- 172.21.15.90:0/2622600179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638a9ea9210 con 0x5638aa230de0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4af259c640 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa23d430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4ae409cab0 tx=0x7f4ae4004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe4a0a0564c581a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4ad9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/2622600179 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40b5030 con 0x5638aa230de0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/2622600179 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4ae409ae20 con 0x5638aa230de0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.994+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae409de50 con 0x5638aa230de0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/2622600179 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ae409d220 con 0x5638aa230de0 2024-04-26T18:17:55.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 10 client.15981 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638a9f75620 con 0x5638aa230de0 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/2622600179 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae409de50 con 0x5638aa230de0 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/2622600179 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f4ae40bbce0 con 0x5638aa230de0 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 10 client.15981 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 20 client.15981 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa250530 0x5638aa252920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 4 client.15981 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638aa1a5070 con 0x5638aa250530 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa250530 0x5638aa252920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4ad9ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.995+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa250530 0x5638aa252920 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc90c85518617e6c server_cookie=68d914e964753bca in_seq=0 out_seq=0 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/2622600179 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5638aa1a5070 con 0x5638aa250530 2024-04-26T18:17:55.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4ad9ffb640 10 client.15981 handle_command_reply: tid=0 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 client.15981 shutdown 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 msgr2=0x5638aa23d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa23d430 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4ae409cab0 tx=0x7f4ae4004a90 comp rx=0 tx=0).stop 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 shutdown_connections 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa250530 0x5638aa252920 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa2404f0 0x5638aa23deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4ad9ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa23d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2622600179 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa23d430 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 >> 172.21.15.90:0/2622600179 conn(0x5638aa22d410 msgr2=0x5638aa233280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 shutdown_connections 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2622600179 wait complete. 2024-04-26T18:17:55.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:54.996+0000 7f4afb9c8740 20 client.15981 trim_cache size 0 max 16384 2024-04-26T18:17:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 Processor -- start 2024-04-26T18:17:55.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 -- start start 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa23dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa22c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa255940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41aea60 con 0x5638aa2404f0 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41ac330 con 0x5638aa230de0 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41aa180 con 0x5638aa231700 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4af259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa23dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4af2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa255940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4af259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa23dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:59586/0 (socket says 172.21.15.90:59586) 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4af259c640 1 -- 172.21.15.90:0/2882032332 learned_addr learned my addr 172.21.15.90:0/2882032332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:55.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.012+0000 7f4af1d9b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa22c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4af259c640 1 -- 172.21.15.90:0/2882032332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 msgr2=0x5638aa22c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4af259c640 1 --2- 172.21.15.90:0/2882032332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa22c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4af259c640 1 -- 172.21.15.90:0/2882032332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 msgr2=0x5638aa255940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4af259c640 1 --2- 172.21.15.90:0/2882032332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa255940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4af259c640 1 -- 172.21.15.90:0/2882032332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638aa231530 con 0x5638aa230de0 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4af259c640 1 --2- 172.21.15.90:0/2882032332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa23dcd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c6a0 tx=0x7f4ae4005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bd5b990a9be9e2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4adbfff640 1 -- 172.21.15.90:0/2882032332 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40b5070 con 0x5638aa230de0 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4adbfff640 1 -- 172.21.15.90:0/2882032332 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae409de50 con 0x5638aa230de0 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.013+0000 7f4adbfff640 1 -- 172.21.15.90:0/2882032332 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40a5790 con 0x5638aa230de0 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2882032332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 msgr2=0x5638aa23dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2882032332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa23dcd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400c6a0 tx=0x7f4ae4005680 comp rx=0 tx=0).stop 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2882032332 shutdown_connections 2024-04-26T18:17:55.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2882032332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa2404f0 0x5638aa255940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2882032332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa231700 0x5638aa22c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/2882032332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa230de0 0x5638aa23dcd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2882032332 >> 172.21.15.90:0/2882032332 conn(0x5638aa22d410 msgr2=0x5638aa22bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2882032332 shutdown_connections 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- 172.21.15.90:0/2882032332 wait complete. 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- start start 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 Processor -- start 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa255c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa231700 0x5638aa253000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa2404f0 0x5638aa253540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41aa7f0 con 0x5638aa230de0 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41abc60 con 0x5638aa231700 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4af41aaec0 con 0x5638aa2404f0 2024-04-26T18:17:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4afb9c8740 10 client.0 _ll_register_callbacks cb 0x7f4ae4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4af259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa255c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4af259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa255c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47100/0 (socket says 172.21.15.90:47100) 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4af259c640 1 -- 172.21.15.90:0/3238675322 learned_addr learned my addr 172.21.15.90:0/3238675322 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa231700 0x5638aa253000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.014+0000 7f4af2d9d640 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa2404f0 0x5638aa253540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4af259c640 1 -- 172.21.15.90:0/3238675322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa2404f0 msgr2=0x5638aa253540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4af259c640 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa2404f0 0x5638aa253540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4af259c640 1 -- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa231700 msgr2=0x5638aa253000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4af259c640 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa231700 0x5638aa253000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4af259c640 1 -- 172.21.15.90:0/3238675322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638aa253d40 con 0x5638aa230de0 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4af259c640 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa255c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4005820 tx=0x7f4ae4004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9592f87df1a6b364 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4ad9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:17:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/3238675322 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40b5070 con 0x5638aa230de0 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/3238675322 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8411+0+0 (secure 0 0 0) 0x7f4ae409bf70 con 0x5638aa230de0 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ae409de50 con 0x5638aa230de0 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/3238675322 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ae40a0ed0 con 0x5638aa230de0 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 10 client.15987 fetch_fsmap learned FSMap version 29 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5638aa253d40 con 0x5638aa230de0 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/3238675322 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ae409de50 con 0x5638aa230de0 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/3238675322 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f4ae40bbce0 con 0x5638aa230de0 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 20 client.15987 populate_metadata read hostname 'smithi090' 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa2b68f0 0x5638aa2b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:55.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 4 client.15987 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:17:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.015+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5638aa19eee0 con 0x5638aa2b68f0 2024-04-26T18:17:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.016+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa2b68f0 0x5638aa2b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.016+0000 7f4af1d9b640 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa2b68f0 0x5638aa2b8ce0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9f4388319ac91b server_cookie=fdfe8fa72f1c1397 in_seq=0 out_seq=0 2024-04-26T18:17:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.016+0000 7f4ad9ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4ad9ffb640 1 -- 172.21.15.90:0/3238675322 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5638aa19eee0 con 0x5638aa2b68f0 2024-04-26T18:17:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4ad9ffb640 10 client.15987 handle_command_reply: tid=0 2024-04-26T18:17:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 client.15987 shutdown 2024-04-26T18:17:55.081 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 msgr2=0x5638aa255c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa255c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4005820 tx=0x7f4ae4004a90 comp rx=0 tx=0).stop 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 shutdown_connections 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5638aa2b68f0 0x5638aa2b8ce0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5638aa2404f0 0x5638aa253540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5638aa231700 0x5638aa253000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 --2- 172.21.15.90:0/3238675322 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5638aa230de0 0x5638aa255c70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 >> 172.21.15.90:0/3238675322 conn(0x5638aa22d410 msgr2=0x5638aa22c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4ad9ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 shutdown_connections 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 1 -- 172.21.15.90:0/3238675322 wait complete. 2024-04-26T18:17:55.082 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.080+0000 7f4afb9c8740 20 client.15987 trim_cache size 0 max 16384 2024-04-26T18:17:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:17:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:17:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:17:55.083 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4adc0a3dc0 msgr2=0x7f4adc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4adc0a3dc0 0x7f4adc0a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4aec004af0 tx=0x7f4aec004a40 comp rx=0 tx=0).stop 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4101850 msgr2=0x7f4af4198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4101850 0x7f4af4198ff0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400e9f0 tx=0x7f4ae400edf0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 shutdown_connections 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f4adc0a3dc0 0x7f4adc0a6280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af4107860 0x7f4af419d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4af4102210 0x7f4af4199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.084 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 --2- 172.21.15.90:0/1389592596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4af4101850 0x7f4af4198ff0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 >> 172.21.15.90:0/1389592596 conn(0x7f4af4077bb0 msgr2=0x7f4af406b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 shutdown_connections 2024-04-26T18:17:55.085 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:17:55.081+0000 7f4af8bda640 1 -- 172.21.15.90:0/1389592596 wait complete. 2024-04-26T18:17:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:54 smithi119 ceph-mon[32752]: pgmap v181: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-04-26T18:17:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:54 smithi167 ceph-mon[32751]: pgmap v181: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-04-26T18:17:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:54 smithi090 ceph-mon[29180]: pgmap v181: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 413 op/s 2024-04-26T18:17:55.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 347897 6.01 MB/sec execute 103 sec latency 152.950 ms 2024-04-26T18:17:56.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 349973 6.02 MB/sec execute 104 sec latency 15.226 ms 2024-04-26T18:17:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:56 smithi119 ceph-mon[32752]: pgmap v182: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-04-26T18:17:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:56 smithi167 ceph-mon[32751]: pgmap v182: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-04-26T18:17:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:56 smithi090 ceph-mon[29180]: pgmap v182: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-04-26T18:17:57.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 351664 6.08 MB/sec execute 105 sec latency 75.946 ms 2024-04-26T18:17:58.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 354163 6.14 MB/sec execute 106 sec latency 13.913 ms 2024-04-26T18:17:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:17:58 smithi119 ceph-mon[32752]: pgmap v183: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-04-26T18:17:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:17:58 smithi167 ceph-mon[32751]: pgmap v183: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-04-26T18:17:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:17:58 smithi090 ceph-mon[29180]: pgmap v183: 97 pgs: 97 active+clean; 103 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 250 op/s 2024-04-26T18:17:59.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 355613 6.15 MB/sec execute 107 sec latency 48.707 ms 2024-04-26T18:18:00.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 357629 6.20 MB/sec execute 108 sec latency 27.211 ms 2024-04-26T18:18:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:00 smithi119 ceph-mon[32752]: pgmap v184: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 411 op/s 2024-04-26T18:18:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:00 smithi167 ceph-mon[32751]: pgmap v184: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 411 op/s 2024-04-26T18:18:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:00 smithi090 ceph-mon[29180]: pgmap v184: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.3 MiB/s wr, 411 op/s 2024-04-26T18:18:01.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 359162 6.21 MB/sec execute 109 sec latency 19.945 ms 2024-04-26T18:18:02.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 362016 6.32 MB/sec execute 110 sec latency 18.852 ms 2024-04-26T18:18:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:02 smithi119 ceph-mon[32752]: pgmap v185: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-04-26T18:18:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:02 smithi167 ceph-mon[32751]: pgmap v185: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-04-26T18:18:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:02 smithi090 ceph-mon[29180]: pgmap v185: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 285 op/s 2024-04-26T18:18:03.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 363484 6.31 MB/sec execute 111 sec latency 15.658 ms 2024-04-26T18:18:04.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 365732 6.38 MB/sec execute 112 sec latency 40.774 ms 2024-04-26T18:18:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:04 smithi119 ceph-mon[32752]: pgmap v186: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-04-26T18:18:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:04 smithi167 ceph-mon[32751]: pgmap v186: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-04-26T18:18:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:04 smithi090 ceph-mon[29180]: pgmap v186: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.4 MiB/s wr, 443 op/s 2024-04-26T18:18:05.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 367201 6.37 MB/sec execute 113 sec latency 25.215 ms 2024-04-26T18:18:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:05 smithi119 ceph-mon[32752]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:18:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:05 smithi167 ceph-mon[32751]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:18:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:05 smithi090 ceph-mon[29180]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:18:06.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 369433 6.44 MB/sec execute 114 sec latency 8.608 ms 2024-04-26T18:18:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:06 smithi119 ceph-mon[32752]: pgmap v188: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-04-26T18:18:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:06 smithi167 ceph-mon[32751]: pgmap v188: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-04-26T18:18:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:06 smithi090 ceph-mon[29180]: pgmap v188: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-04-26T18:18:07.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.42 MB/sec execute 115 sec latency 623.677 ms 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.169+0000 7f2ac8533640 1 -- 172.21.15.90:0/1134228531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 msgr2=0x7f2ac0107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.169+0000 7f2ac8533640 1 --2- 172.21.15.90:0/1134228531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac0107c20 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2abc009930 tx=0x7f2abc02a420 comp rx=0 tx=0).stop 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- 172.21.15.90:0/1134228531 shutdown_connections 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 --2- 172.21.15.90:0/1134228531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac01021f0 0x7f2ac010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 --2- 172.21.15.90:0/1134228531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ac0101830 0x7f2ac0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 --2- 172.21.15.90:0/1134228531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac0107c20 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- 172.21.15.90:0/1134228531 >> 172.21.15.90:0/1134228531 conn(0x7f2ac0078950 msgr2=0x7f2ac0078d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- 172.21.15.90:0/1134228531 shutdown_connections 2024-04-26T18:18:08.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- 172.21.15.90:0/1134228531 wait complete. 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 Processor -- start 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- start start 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ac0101830 0x7f2ac0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac01021f0 0x7f2ac019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac01a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0111ac0 con 0x7f2ac0107840 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0111940 con 0x7f2ac0101830 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac8533640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0111c40 con 0x7f2ac01021f0 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac6ad0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac01a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac6ad0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac01a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34566/0 (socket says 172.21.15.90:34566) 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.170+0000 7f2ac6ad0640 1 -- 172.21.15.90:0/590503036 learned_addr learned my addr 172.21.15.90:0/590503036 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ac0101830 0x7f2ac0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac01021f0 0x7f2ac019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac6ad0640 1 -- 172.21.15.90:0/590503036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac01021f0 msgr2=0x7f2ac019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac6ad0640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac01021f0 0x7f2ac019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac6ad0640 1 -- 172.21.15.90:0/590503036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ac0101830 msgr2=0x7f2ac0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac6ad0640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ac0101830 0x7f2ac0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac6ad0640 1 -- 172.21.15.90:0/590503036 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac01a5540 con 0x7f2ac0107840 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac01021f0 0x7f2ac019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac6ad0640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac01a5000 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800cad0 tx=0x7f2ab800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ab77fe640 1 -- 172.21.15.90:0/590503036 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab800ae20 con 0x7f2ac0107840 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ab77fe640 1 -- 172.21.15.90:0/590503036 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ab8005020 con 0x7f2ac0107840 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ab77fe640 1 -- 172.21.15.90:0/590503036 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab800f6b0 con 0x7f2ac0107840 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac01a5710 con 0x7f2ac0107840 2024-04-26T18:18:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.171+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01a5be0 con 0x7f2ac0107840 2024-04-26T18:18:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2ab77fe640 1 -- 172.21.15.90:0/590503036 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f2ab80051d0 con 0x7f2ac0107840 2024-04-26T18:18:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 Processor -- start 2024-04-26T18:18:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 -- start start 2024-04-26T18:18:08.181 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x5557571532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x55575715dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5557570b9460 con 0x555757152f00 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555757073590 con 0x55575715e0e0 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2acb24b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557570b5420 con 0x555757153820 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2ac5ace640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x5557571532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2ac5ace640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x5557571532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34578/0 (socket says 172.21.15.90:34578) 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2ac5ace640 1 -- 172.21.15.90:0/1876717308 learned_addr learned my addr 172.21.15.90:0/1876717308 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2ab77fe640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2aac0a3dc0 0x7f2aac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.174+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x55575715dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac6ad0640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2aac0a3dc0 0x7f2aac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ab77fe640 1 -- 172.21.15.90:0/590503036 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f2ab80d0830 con 0x7f2ac0107840 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac62cf640 1 -- 172.21.15.90:0/1876717308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 msgr2=0x5557571604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac62cf640 1 -- 172.21.15.90:0/1876717308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 msgr2=0x5557571532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x5557571532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac62cf640 1 -- 172.21.15.90:0/1876717308 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557570800d0 con 0x555757153820 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x5557571532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x55575715dba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2abc031440 tx=0x7f2abc005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a750b0aea7c1e8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ac6ad0640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2aac0a3dc0 0x7f2aac0a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800a6e0 tx=0x7f2ab801baf0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ab4bfe640 1 -- 172.21.15.90:0/1876717308 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc040070 con 0x555757153820 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ab4bfe640 1 -- 172.21.15.90:0/1876717308 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2abc0316a0 con 0x555757153820 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.175+0000 7f2ab4bfe640 1 -- 172.21.15.90:0/1876717308 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc035040 con 0x555757153820 2024-04-26T18:18:08.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 -- 172.21.15.90:0/1876717308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 msgr2=0x55575715dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x55575715dba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2abc031440 tx=0x7f2abc005fd0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 -- 172.21.15.90:0/1876717308 shutdown_connections 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x55575715dba0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1876717308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x5557571532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 -- 172.21.15.90:0/1876717308 >> 172.21.15.90:0/1876717308 conn(0x55575714f410 msgr2=0x55575714f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 -- 172.21.15.90:0/1876717308 shutdown_connections 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 -- 172.21.15.90:0/1876717308 wait complete. 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 -- start start 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.176+0000 7f2acb24b740 1 Processor -- start 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757152f00 0x55575715b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757153820 0x55575715b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x55575715baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555757160cb0 con 0x555757153820 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555757040b60 con 0x55575715e0e0 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555757160e30 con 0x555757152f00 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 10 client.0 _ll_register_callbacks cb 0x5557570652b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757153820 0x55575715b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757153820 0x55575715b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34582/0 (socket says 172.21.15.90:34582) 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 -- 172.21.15.90:0/1340635790 learned_addr learned my addr 172.21.15.90:0/1340635790 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757152f00 0x55575715b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x55575715baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 -- 172.21.15.90:0/1340635790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757152f00 msgr2=0x55575715b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757152f00 0x55575715b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 -- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 msgr2=0x55575715baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x55575715baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 -- 172.21.15.90:0/1340635790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555756dcb210 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x55575715baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757153820 0x55575715b560 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2abc02abd0 tx=0x7f2abc031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6795bc0193eff0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/1340635790 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc040030 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/1340635790 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f2abc005570 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2abc0316a0 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/1340635790 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2abc056050 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 10 client.15999 fetch_fsmap learned FSMap version 29 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.177+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555756e97620 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/1340635790 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2abc0316a0 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/1340635790 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2abc0035f0 con 0x555757153820 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2acb24b740 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:18:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2acb24b740 10 client.15999 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2acb24b740 20 client.15999 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55575716e000 0x5557571703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2acb24b740 4 client.15999 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557570c7070 con 0x55575716e000 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55575716e000 0x5557571703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55575716e000 0x5557571703f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=700bf85b038e167b server_cookie=69e66a530504e9dd in_seq=0 out_seq=0 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.178+0000 7f2aa9ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/1340635790 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557570c7070 con 0x55575716e000 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2aa9ffb640 10 client.15999 handle_command_reply: tid=0 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 client.15999 shutdown 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757153820 msgr2=0x55575715b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757153820 0x55575715b560 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f2abc02abd0 tx=0x7f2abc031b10 comp rx=0 tx=0).stop 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 shutdown_connections 2024-04-26T18:18:08.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55575716e000 0x5557571703f0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x55575715baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2aa9ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757153820 0x55575715b560 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 --2- 172.21.15.90:0/1340635790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757152f00 0x55575715b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 >> 172.21.15.90:0/1340635790 conn(0x55575714f410 msgr2=0x555757155190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 shutdown_connections 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 1 -- 172.21.15.90:0/1340635790 wait complete. 2024-04-26T18:18:08.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.179+0000 7f2acb24b740 20 client.15999 trim_cache size 0 max 16384 2024-04-26T18:18:08.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 Processor -- start 2024-04-26T18:18:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 -- start start 2024-04-26T18:18:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757173410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x555757173950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01b0d40 con 0x555757152f00 2024-04-26T18:18:08.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01ae610 con 0x55575715e0e0 2024-04-26T18:18:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2acb24b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01ac480 con 0x555757153820 2024-04-26T18:18:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757173410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757173410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34584/0 (socket says 172.21.15.90:34584) 2024-04-26T18:18:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 -- 172.21.15.90:0/2466699726 learned_addr learned my addr 172.21.15.90:0/2466699726 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x555757173950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 -- 172.21.15.90:0/2466699726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 msgr2=0x555757173950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x555757173950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 -- 172.21.15.90:0/2466699726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 msgr2=0x5557571582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.194+0000 7f2ac5ace640 1 -- 172.21.15.90:0/2466699726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575714ed30 con 0x555757152f00 2024-04-26T18:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757173410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000c00 tx=0x7f2ab0013480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ac4d83c5795b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2aabfff640 1 -- 172.21.15.90:0/2466699726 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0011e40 con 0x555757152f00 2024-04-26T18:18:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2aabfff640 1 -- 172.21.15.90:0/2466699726 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ab00135f0 con 0x555757152f00 2024-04-26T18:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2aabfff640 1 -- 172.21.15.90:0/2466699726 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0003850 con 0x555757152f00 2024-04-26T18:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 -- 172.21.15.90:0/2466699726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 msgr2=0x555757173410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757173410 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000c00 tx=0x7f2ab0013480 comp rx=0 tx=0).stop 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 -- 172.21.15.90:0/2466699726 shutdown_connections 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x555757173950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 --2- 172.21.15.90:0/2466699726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757173410 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 -- 172.21.15.90:0/2466699726 >> 172.21.15.90:0/2466699726 conn(0x55575714f410 msgr2=0x555757169910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.195+0000 7f2acb24b740 1 -- 172.21.15.90:0/2466699726 shutdown_connections 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 -- 172.21.15.90:0/2466699726 wait complete. 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 -- start start 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 Processor -- start 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757171160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x5557571716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01acad0 con 0x555757152f00 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01adf40 con 0x55575715e0e0 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac01ad1a0 con 0x555757153820 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2acb24b740 10 client.0 _ll_register_callbacks cb 0x7f2ab000b590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757171160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757171160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34600/0 (socket says 172.21.15.90:34600) 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 -- 172.21.15.90:0/188191389 learned_addr learned my addr 172.21.15.90:0/188191389 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x5557571716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 -- 172.21.15.90:0/188191389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 msgr2=0x5557571716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x5557571716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 -- 172.21.15.90:0/188191389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 msgr2=0x5557571d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.196+0000 7f2ac5ace640 1 -- 172.21.15.90:0/188191389 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555757171f70 con 0x555757152f00 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2ac5ace640 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757171160 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b470 tx=0x7f2ab0011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c20c33ea59f34e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2aa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/188191389 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00149f0 con 0x555757152f00 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/188191389 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8499+0+0 (secure 0 0 0) 0x7f2ab0014b90 con 0x555757152f00 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ab00135f0 con 0x555757152f00 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/188191389 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ab003b050 con 0x555757152f00 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 10 client.16005 fetch_fsmap learned FSMap version 29 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557571d4830 con 0x555757152f00 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/188191389 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ab00135f0 con 0x555757152f00 2024-04-26T18:18:08.211 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/188191389 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2ab0003ab0 con 0x555757152f00 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 10 client.16005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 20 client.16005 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5557571d4ef0 0x5557571d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 4 client.16005 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.197+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557570c0ee0 con 0x5557571d4ef0 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.198+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5557571d4ef0 0x5557571d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.198+0000 7f2aa9ffb640 10 client.16005 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.198+0000 7f2ac62cf640 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5557571d4ef0 0x5557571d7310 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c933dd81b35aaace server_cookie=f6e4da9ffa0ae1a3 in_seq=0 out_seq=0 2024-04-26T18:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2aa9ffb640 1 -- 172.21.15.90:0/188191389 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557570c0ee0 con 0x5557571d4ef0 2024-04-26T18:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2aa9ffb640 10 client.16005 handle_command_reply: tid=0 2024-04-26T18:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 client.16005 shutdown 2024-04-26T18:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 msgr2=0x555757171160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757171160 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000b470 tx=0x7f2ab0011fb0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 shutdown_connections 2024-04-26T18:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5557571d4ef0 0x5557571d7310 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55575715e0e0 0x5557571d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2aa9ffb640 0 client.16005 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757153820 0x5557571716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 --2- 172.21.15.90:0/188191389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555757152f00 0x555757171160 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 >> 172.21.15.90:0/188191389 conn(0x55575714f410 msgr2=0x5557571548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 shutdown_connections 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 1 -- 172.21.15.90:0/188191389 wait complete. 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.256+0000 7f2acb24b740 20 client.16005 trim_cache size 0 max 16384 2024-04-26T18:18:08.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:18:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:18:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:18:08.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:18:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2aac0a3dc0 msgr2=0x7f2aac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2aac0a3dc0 0x7f2aac0a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800a6e0 tx=0x7f2ab801baf0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 msgr2=0x7f2ac01a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac01a5000 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800cad0 tx=0x7f2ab800cf90 comp rx=0 tx=0).stop 2024-04-26T18:18:08.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 shutdown_connections 2024-04-26T18:18:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2aac0a3dc0 0x7f2aac0a6280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2ac0107840 0x7f2ac01a5000 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2ac01021f0 0x7f2ac019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 --2- 172.21.15.90:0/590503036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ac0101830 0x7f2ac0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 >> 172.21.15.90:0/590503036 conn(0x7f2ac0078950 msgr2=0x7f2ac0104500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 shutdown_connections 2024-04-26T18:18:08.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:08.257+0000 7f2ac8533640 1 -- 172.21.15.90:0/590503036 wait complete. 2024-04-26T18:18:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:08.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.37 MB/sec execute 116 sec latency 1623.790 ms 2024-04-26T18:18:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:08 smithi119 ceph-mon[32752]: pgmap v189: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-04-26T18:18:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:18:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:08 smithi167 ceph-mon[32751]: pgmap v189: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-04-26T18:18:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:18:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:08 smithi090 ceph-mon[29180]: pgmap v189: 97 pgs: 97 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 383 op/s 2024-04-26T18:18:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:18:09.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.31 MB/sec execute 117 sec latency 2623.835 ms 2024-04-26T18:18:10.672 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.26 MB/sec execute 118 sec latency 3623.911 ms 2024-04-26T18:18:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:10 smithi119 ceph-mon[32752]: pgmap v190: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 316 op/s 2024-04-26T18:18:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:10 smithi167 ceph-mon[32751]: pgmap v190: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 316 op/s 2024-04-26T18:18:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:10 smithi090 ceph-mon[29180]: pgmap v190: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 316 op/s 2024-04-26T18:18:11.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.21 MB/sec execute 119 sec latency 4624.012 ms 2024-04-26T18:18:12.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.16 MB/sec execute 120 sec latency 5624.113 ms 2024-04-26T18:18:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:12 smithi119 ceph-mon[32752]: pgmap v191: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 316 op/s 2024-04-26T18:18:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:12 smithi119 ceph-mon[32752]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:18:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:12 smithi167 ceph-mon[32751]: pgmap v191: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 316 op/s 2024-04-26T18:18:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:12 smithi167 ceph-mon[32751]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:18:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:12 smithi090 ceph-mon[29180]: pgmap v191: 97 pgs: 97 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 316 op/s 2024-04-26T18:18:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:12 smithi090 ceph-mon[29180]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:18:13.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.11 MB/sec execute 121 sec latency 6624.201 ms 2024-04-26T18:18:14.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.06 MB/sec execute 122 sec latency 7624.252 ms 2024-04-26T18:18:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:14 smithi119 ceph-mon[32752]: pgmap v193: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-04-26T18:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:14 smithi167 ceph-mon[32751]: pgmap v193: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-04-26T18:18:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:14 smithi090 ceph-mon[29180]: pgmap v193: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.1 MiB/s wr, 252 op/s 2024-04-26T18:18:15.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 6.01 MB/sec execute 123 sec latency 8624.312 ms 2024-04-26T18:18:16.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.96 MB/sec execute 124 sec latency 9624.422 ms 2024-04-26T18:18:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:16 smithi119 ceph-mon[32752]: pgmap v194: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:18:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:16 smithi167 ceph-mon[32751]: pgmap v194: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:18:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:16 smithi090 ceph-mon[29180]: pgmap v194: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:18:17.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.91 MB/sec execute 125 sec latency 10624.515 ms 2024-04-26T18:18:18.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.86 MB/sec execute 126 sec latency 11624.604 ms 2024-04-26T18:18:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:18 smithi119 ceph-mon[32752]: pgmap v195: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:18:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:18 smithi167 ceph-mon[32751]: pgmap v195: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:18:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:18 smithi090 ceph-mon[29180]: pgmap v195: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:18:19.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.82 MB/sec execute 127 sec latency 12624.649 ms 2024-04-26T18:18:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0d7a933700 time 2024-04-26T18:18:19.793612+0000 2024-04-26T18:18:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0d84f5c3dd] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.069 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.070 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.070 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: clone() 2024-04-26T18:18:20.070 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: *** Caught signal (Aborted) ** 2024-04-26T18:18:20.070 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: in thread 7f0d7a933700 thread_name:ms_dispatch 2024-04-26T18:18:20.070 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:19.793+0000 7f0d7a933700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0d7a933700 time 2024-04-26T18:18:19.793612+0000 2024-04-26T18:18:20.071 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:20.071 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.071 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0d84f5c3dd] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.072 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: clone() 2024-04-26T18:18:20.073 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.073 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.073 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0d83ce4d20] 2024-04-26T18:18:20.073 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: gsignal() 2024-04-26T18:18:20.073 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: abort() 2024-04-26T18:18:20.073 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0d84f5c437] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 16: clone() 2024-04-26T18:18:20.074 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:19.795+0000 7f0d7a933700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: in thread 7f0d7a933700 thread_name:ms_dispatch 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0d83ce4d20] 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: gsignal() 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: abort() 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0d84f5c437] 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.075 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.076 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 16: clone() 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: -1> 2024-04-26T18:18:19.793+0000 7f0d7a933700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0d7a933700 time 2024-04-26T18:18:19.793612+0000 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0d84f5c3dd] 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.077 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: clone() 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 0> 2024-04-26T18:18:19.795+0000 7f0d7a933700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:18:20.078 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: in thread 7f0d7a933700 thread_name:ms_dispatch 2024-04-26T18:18:20.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0d83ce4d20] 2024-04-26T18:18:20.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: gsignal() 2024-04-26T18:18:20.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: abort() 2024-04-26T18:18:20.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0d84f5c437] 2024-04-26T18:18:20.079 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 16: clone() 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:18:20.080 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: -1> 2024-04-26T18:18:19.793+0000 7f0d7a933700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0d7a933700 time 2024-04-26T18:18:19.793612+0000 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0d84f5c3dd] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.081 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: clone() 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 0> 2024-04-26T18:18:19.795+0000 7f0d7a933700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: in thread 7f0d7a933700 thread_name:ms_dispatch 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0d83ce4d20] 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2: gsignal() 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 3: abort() 2024-04-26T18:18:20.082 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0d84f5c437] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0d84f5c5a3] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5632a8047822] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5632a7fb7598] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5632a7fb9c17] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5632a7eb2d0b] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5632a7eb32ac] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5632a7e9bfad] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0d851f0e20] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 13: (DispatchQueue::entry()+0x51f) [0x7f0d851edfdf] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0d852c0fc1] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0d83cda1ca] 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 16: clone() 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:18:20.083 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[63473]: 2024-04-26T18:18:20.084 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:19 smithi167 podman[68106]: 2024-04-26 18:18:19.869070752 +0000 UTC m=+0.025835032 container died 66d7771cec9fd5e0d21bf457ddad0ec8f4c07848f5745a4533f7820130d689ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T18:18:20.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.77 MB/sec execute 128 sec latency 13624.687 ms 2024-04-26T18:18:20.882 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.7 2024-04-26T18:18:20.883 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:18:21.150 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:18:21.271 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:20 smithi090 ceph-mon[29180]: pgmap v196: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-04-26T18:18:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:20 smithi119 ceph-mon[32752]: pgmap v196: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-04-26T18:18:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:20 smithi167 ceph-mon[32751]: pgmap v196: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-04-26T18:18:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 -- 172.21.15.90:0/4057071628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 msgr2=0x7f37e810f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/4057071628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e810f7d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009ca0 tx=0x7f37e002a950 comp rx=0 tx=0).stop 2024-04-26T18:18:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 -- 172.21.15.90:0/4057071628 shutdown_connections 2024-04-26T18:18:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/4057071628 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e810f7d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/4057071628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e806b160 0x7f37e806b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/4057071628 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e8108b30 0x7f37e8108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 -- 172.21.15.90:0/4057071628 >> 172.21.15.90:0/4057071628 conn(0x7f37e8077630 msgr2=0x7f37e8077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 -- 172.21.15.90:0/4057071628 shutdown_connections 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.344+0000 7f37ef6b0640 1 -- 172.21.15.90:0/4057071628 wait complete. 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 Processor -- start 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 -- start start 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e806b160 0x7f37e8198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e81994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e8108b30 0x7f37e819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8112c00 con 0x7f37e806bb20 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8112a80 con 0x7f37e806b160 2024-04-26T18:18:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ef6b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37e8112d80 con 0x7f37e8108b30 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e81994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e81994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36516/0 (socket says 172.21.15.90:36516) 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 -- 172.21.15.90:0/3336293744 learned_addr learned my addr 172.21.15.90:0/3336293744 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37edc4d640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e8108b30 0x7f37e819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 -- 172.21.15.90:0/3336293744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e8108b30 msgr2=0x7f37e819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e8108b30 0x7f37e819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 -- 172.21.15.90:0/3336293744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e806b160 msgr2=0x7f37e8198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e806b160 0x7f37e8198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 -- 172.21.15.90:0/3336293744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e819df90 con 0x7f37e806bb20 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e81994e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c890 tx=0x7f37d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.345+0000 7f37de7fc640 1 -- 172.21.15.90:0/3336293744 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004220 con 0x7f37e806bb20 2024-04-26T18:18:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.346+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e819e280 con 0x7f37e806bb20 2024-04-26T18:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.346+0000 7f37de7fc640 1 -- 172.21.15.90:0/3336293744 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37d80043c0 con 0x7f37e806bb20 2024-04-26T18:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.346+0000 7f37de7fc640 1 -- 172.21.15.90:0/3336293744 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8004a00 con 0x7f37e806bb20 2024-04-26T18:18:21.349 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.346+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e81a5b10 con 0x7f37e806bb20 2024-04-26T18:18:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37de7fc640 1 -- 172.21.15.90:0/3336293744 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f37d80079b0 con 0x7f37e806bb20 2024-04-26T18:18:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 Processor -- start 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 -- start start 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095ef1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095ef720 0x55ec095fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095fe4f0 0x55ec096008e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec09555460 con 0x55ec095eee00 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ec0950f590 con 0x55ec095ef720 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.348+0000 7f37f23ca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec09551420 con 0x55ec095fe4f0 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095ef1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37de7fc640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f37d40a3dc0 0x7f37d40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095ef1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36518/0 (socket says 172.21.15.90:36518) 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 -- 172.21.15.90:0/482628219 learned_addr learned my addr 172.21.15.90:0/482628219 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095fe4f0 0x55ec096008e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37ed44c640 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095ef720 0x55ec095fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f37d40a3dc0 0x7f37d40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37de7fc640 1 -- 172.21.15.90:0/3336293744 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f37d80d05c0 con 0x7f37e806bb20 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 -- 172.21.15.90:0/482628219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095fe4f0 msgr2=0x55ec096008e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095fe4f0 0x55ec096008e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 -- 172.21.15.90:0/482628219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095ef720 msgr2=0x55ec095fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095ef720 0x55ec095fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 -- 172.21.15.90:0/482628219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec0951c0d0 con 0x55ec095eee00 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095ef1e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f37e002f3d0 tx=0x7f37e002f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698575a84bcaaf0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37d37fe640 1 -- 172.21.15.90:0/482628219 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0034040 con 0x55ec095eee00 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37d37fe640 1 -- 172.21.15.90:0/482628219 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e004e070 con 0x55ec095eee00 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37d37fe640 1 -- 172.21.15.90:0/482628219 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00496d0 con 0x55ec095eee00 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.349+0000 7f37edc4d640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f37d40a3dc0 0x7f37d40a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37e004d000 tx=0x7f37e004b000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 -- 172.21.15.90:0/482628219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 msgr2=0x55ec095ef1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095ef1e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f37e002f3d0 tx=0x7f37e002f400 comp rx=0 tx=0).stop 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 -- 172.21.15.90:0/482628219 shutdown_connections 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095fe4f0 0x55ec096008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095ef720 0x55ec095fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 --2- 172.21.15.90:0/482628219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095ef1e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 -- 172.21.15.90:0/482628219 >> 172.21.15.90:0/482628219 conn(0x55ec095eb410 msgr2=0x55ec095eb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 -- 172.21.15.90:0/482628219 shutdown_connections 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 -- 172.21.15.90:0/482628219 wait complete. 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 -- start start 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.350+0000 7f37f23ca740 1 Processor -- start 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095eee00 0x55ec095fb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095ef720 0x55ec095fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec095fbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37f23ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ec09601120 con 0x55ec095ef720 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37f23ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ec094dcb60 con 0x55ec095fe4f0 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37f23ca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec096012a0 con 0x55ec095eee00 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37f23ca740 10 client.0 _ll_register_callbacks cb 0x55ec095012b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095ef720 0x55ec095fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095ef720 0x55ec095fb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36534/0 (socket says 172.21.15.90:36534) 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 -- 172.21.15.90:0/1864554666 learned_addr learned my addr 172.21.15.90:0/1864554666 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37edc4d640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec095fbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ed44c640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095eee00 0x55ec095fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 -- 172.21.15.90:0/1864554666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095eee00 msgr2=0x55ec095fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095eee00 0x55ec095fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 -- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 msgr2=0x55ec095fbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec095fbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 -- 172.21.15.90:0/1864554666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec09267210 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37edc4d640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec095fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095ef720 0x55ec095fb9b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f37d8098a00 tx=0x7f37d806af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1323259a84674371 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.351+0000 7f37d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1864554666 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d801d070 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1864554666 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f37d809b9f0 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e004e070 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1864554666 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37d80670b0 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 10 client.16017 fetch_fsmap learned FSMap version 29 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec09333620 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1864554666 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e004e070 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1864554666 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f37d8085ae0 con 0x55ec095ef720 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 20 client.16017 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec0960e680 0x55ec09610a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 4 client.16017 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec09563070 con 0x55ec0960e680 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.352+0000 7f37ed44c640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec0960e680 0x55ec09610a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.353+0000 7f37ed44c640 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec0960e680 0x55ec09610a70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310483437a7bfd3 server_cookie=21c0f4dc7f144329 in_seq=0 out_seq=0 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.353+0000 7f37d0ff9640 10 client.16017 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.353+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1864554666 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec09563070 con 0x55ec0960e680 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.353+0000 7f37d0ff9640 10 client.16017 handle_command_reply: tid=0 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.353+0000 7f37f23ca740 1 client.16017 shutdown 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095ef720 msgr2=0x55ec095fb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095ef720 0x55ec095fb9b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f37d8098a00 tx=0x7f37d806af50 comp rx=0 tx=0).stop 2024-04-26T18:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 shutdown_connections 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec0960e680 0x55ec09610a70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec095fbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095ef720 0x55ec095fb9b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37d0ff9640 0 client.16017 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1864554666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095eee00 0x55ec095fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 >> 172.21.15.90:0/1864554666 conn(0x55ec095eb410 msgr2=0x55ec095f1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 shutdown_connections 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 1 -- 172.21.15.90:0/1864554666 wait complete. 2024-04-26T18:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.354+0000 7f37f23ca740 20 client.16017 trim_cache size 0 max 16384 2024-04-26T18:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 Processor -- start 2024-04-26T18:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 -- start start 2024-04-26T18:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095eee00 0x55ec09609fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec096162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095fe4f0 0x55ec09616820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81aea80 con 0x55ec095fe4f0 2024-04-26T18:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81ac350 con 0x55ec095eee00 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37f23ca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81aa1e0 con 0x55ec095ef720 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.368+0000 7f37edc4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095fe4f0 0x55ec09616820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095fe4f0 0x55ec09616820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36538/0 (socket says 172.21.15.90:36538) 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 -- 172.21.15.90:0/1903457787 learned_addr learned my addr 172.21.15.90:0/1903457787 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 -- 172.21.15.90:0/1903457787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 msgr2=0x55ec096162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 --2- 172.21.15.90:0/1903457787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec096162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 -- 172.21.15.90:0/1903457787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095eee00 msgr2=0x55ec09609fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 --2- 172.21.15.90:0/1903457787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095eee00 0x55ec09609fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 -- 172.21.15.90:0/1903457787 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec09611050 con 0x55ec095fe4f0 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37edc4d640 1 --2- 172.21.15.90:0/1903457787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095fe4f0 0x55ec09616820 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f37e002f470 tx=0x7f37e002f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1590f68e6a128877 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37d2ffd640 1 -- 172.21.15.90:0/1903457787 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e005a020 con 0x55ec095fe4f0 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37d2ffd640 1 -- 172.21.15.90:0/1903457787 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e004e050 con 0x55ec095fe4f0 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.369+0000 7f37d2ffd640 1 -- 172.21.15.90:0/1903457787 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0004040 con 0x55ec095fe4f0 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- 172.21.15.90:0/1903457787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095fe4f0 msgr2=0x55ec09616820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1903457787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095fe4f0 0x55ec09616820 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f37e002f470 tx=0x7f37e002f740 comp rx=0 tx=0).stop 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- 172.21.15.90:0/1903457787 shutdown_connections 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1903457787 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095fe4f0 0x55ec09616820 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1903457787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec096162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1903457787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095eee00 0x55ec09609fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- 172.21.15.90:0/1903457787 >> 172.21.15.90:0/1903457787 conn(0x55ec095eb410 msgr2=0x55ec095fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- 172.21.15.90:0/1903457787 shutdown_connections 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- 172.21.15.90:0/1903457787 wait complete. 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- start start 2024-04-26T18:18:21.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 Processor -- start 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095f8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec095f89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec09673b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81aa830 con 0x55ec095eee00 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81abc80 con 0x55ec095fe4f0 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37e81aaee0 con 0x55ec095ef720 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.370+0000 7f37f23ca740 10 client.0 _ll_register_callbacks cb 0x7f37d800a520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36546/0 (socket says 172.21.15.90:36546) 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 -- 172.21.15.90:0/1878461310 learned_addr learned my addr 172.21.15.90:0/1878461310 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37edc4d640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec09673b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec095f89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 -- 172.21.15.90:0/1878461310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 msgr2=0x55ec095f89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec095f89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 -- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 msgr2=0x55ec09673b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec09673b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 -- 172.21.15.90:0/1878461310 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec096008e0 con 0x55ec095eee00 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec095f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37ed44c640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095f8490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f37e4005cd0 tx=0x7f37e4003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a65c519d547b4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1878461310 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e401e070 con 0x55ec095eee00 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1878461310 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f37e400baa0 con 0x55ec095eee00 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e004e050 con 0x55ec095eee00 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1878461310 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f37e402a050 con 0x55ec095eee00 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37f23ca740 10 client.16023 fetch_fsmap learned FSMap version 29 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.371+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec09674ff0 con 0x55ec095eee00 2024-04-26T18:18:21.377 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1878461310 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37e004e050 con 0x55ec095eee00 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1878461310 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f37e4011560 con 0x55ec095eee00 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37f23ca740 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37f23ca740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37f23ca740 20 client.16023 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec09675740 0x55ec09677b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37f23ca740 4 client.16023 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec0955cee0 con 0x55ec09675740 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec09675740 0x55ec09677b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37ecc4b640 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec09675740 0x55ec09677b60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a651d6a97483814a server_cookie=34ec08d914c2534f in_seq=0 out_seq=0 2024-04-26T18:18:21.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:21.372+0000 7f37d0ff9640 10 client.16023 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:21.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.73 MB/sec execute 129 sec latency 14624.736 ms 2024-04-26T18:18:21.817 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:21 smithi167 podman[68106]: 2024-04-26 18:18:21.4623054 +0000 UTC m=+1.619069697 container remove 66d7771cec9fd5e0d21bf457ddad0ec8f4c07848f5745a4533f7820130d689ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T18:18:21.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:21 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:18:21.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:21 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-04-26T18:18:21.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:21 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 40.273s CPU time. 2024-04-26T18:18:22.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.68 MB/sec execute 130 sec latency 15624.783 ms 2024-04-26T18:18:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 -- 172.21.15.90:0/3721685887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 msgr2=0x7fcda806be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/3721685887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda806be20 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fcd90009990 tx=0x7fcd90009ca0 comp rx=0 tx=0).stop 2024-04-26T18:18:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 -- 172.21.15.90:0/3721685887 shutdown_connections 2024-04-26T18:18:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/3721685887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda806c360 0x7fcda81102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:22.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/3721685887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda8109490 0x7fcda8109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:22.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/3721685887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda806be20 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:22.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 -- 172.21.15.90:0/3721685887 >> 172.21.15.90:0/3721685887 conn(0x7fcda8079290 msgr2=0x7fcda80796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:22.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 -- 172.21.15.90:0/3721685887 shutdown_connections 2024-04-26T18:18:22.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 -- 172.21.15.90:0/3721685887 wait complete. 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.862+0000 7fcdad3cd700 1 Processor -- start 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcdad3cd700 1 -- start start 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcdad3cd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda81023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcdad3cd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda806c360 0x7fcda8102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcdad3cd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda8109490 0x7fcda8102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcdad3cd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fcda81135c0 con 0x7fcda806b9c0 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcdad3cd700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8113440 con 0x7fcda8109490 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcdad3cd700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcda8113740 con 0x7fcda806c360 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda81023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda81023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33912/0 (socket says 172.21.15.90:33912) 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 -- 172.21.15.90:0/1725455715 learned_addr learned my addr 172.21.15.90:0/1725455715 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:22.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda67fc700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda806c360 0x7fcda8102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 -- 172.21.15.90:0/1725455715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda806c360 msgr2=0x7fcda8102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda806c360 0x7fcda8102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 -- 172.21.15.90:0/1725455715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda8109490 msgr2=0x7fcda8102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda77fe700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda8109490 0x7fcda8102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda8109490 0x7fcda8102e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.863+0000 7fcda6ffd700 1 -- 172.21.15.90:0/1725455715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda81a8020 con 0x7fcda806b9c0 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.864+0000 7fcda6ffd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda81023e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd98007e20 tx=0x7fcd9800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.864+0000 7fcd9ffff700 1 -- 172.21.15.90:0/1725455715 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98016020 con 0x7fcda806b9c0 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.864+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda81a8310 con 0x7fcda806b9c0 2024-04-26T18:18:22.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.864+0000 7fcd9ffff700 1 -- 172.21.15.90:0/1725455715 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd9800ae00 con 0x7fcda806b9c0 2024-04-26T18:18:22.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.864+0000 7fcd9ffff700 1 -- 172.21.15.90:0/1725455715 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd98011700 con 0x7fcda806b9c0 2024-04-26T18:18:22.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.864+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda81a8850 con 0x7fcda806b9c0 2024-04-26T18:18:22.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.865+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda8104b40 con 0x7fcda806b9c0 2024-04-26T18:18:22.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.866+0000 7fcd9ffff700 1 -- 172.21.15.90:0/1725455715 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fcd98019020 con 0x7fcda806b9c0 2024-04-26T18:18:22.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.866+0000 7fcd9ffff700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd940a3db0 0x7fcd940a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:22.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.866+0000 7fcda67fc700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd940a3db0 0x7fcd940a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:22.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.867+0000 7fcd9ffff700 1 -- 172.21.15.90:0/1725455715 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7fcd980d0420 con 0x7fcda806b9c0 2024-04-26T18:18:22.870 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.867+0000 7fcda67fc700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd940a3db0 0x7fcd940a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd90006010 tx=0x7fcd9001a040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:22.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:22.871+0000 7fcd9ffff700 1 -- 172.21.15.90:0/1725455715 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd9809cc10 con 0x7fcda806b9c0 2024-04-26T18:18:23.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.120+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcda8103910 con 0x7fcda806b9c0 2024-04-26T18:18:23.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.121+0000 7fcd9ffff700 1 -- 172.21.15.90:0/1725455715 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11874 (secure 0 0 0) 0x7fcd9809c510 con 0x7fcda806b9c0 2024-04-26T18:18:23.121 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T18:18:23.121 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6835/2911456063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":2911456063},{"type":"v1","addr":"172.21.15.90:6835","nonce":2911456063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14535,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6837/1335338695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":1335338695},{"type":"v1","addr":"172.21.15.90:6837","nonce":1335338695}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14574,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6839/2946680060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6838","nonce":2946680060},{"type":"v1","addr":"172.21.15.90:6839","nonce":2946680060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14586,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6841/3724478516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6840","nonce":3724478516},{"type":"v1","addr":"172.21.15.90:6841","nonce":3724478516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24392,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/137310630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":137310630},{"type":"v1","addr":"172.21.15.119:6833","nonce":137310630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2611460544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2611460544},{"type":"v1","addr":"172.21.15.119:6835","nonce":2611460544}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T18:16:49.669086+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":24448,"mds_2":24436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24412":{"gid":24412,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6833/766230459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":766230459},{"type":"v1","addr":"172.21.15.167:6833","nonce":766230459}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.119:6837/3534880230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":3534880230},{"type":"v1","addr":"172.21.15.119:6837","nonce":3534880230}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.167:6837/200145753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":200145753},{"type":"v1","addr":"172.21.15.167:6837","nonce":200145753}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.167:6839/4229007587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":4229007587},{"type":"v1","addr":"172.21.15.167:6839","nonce":4229007587}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24436,24448,24449]},"id":1}]} 2024-04-26T18:18:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd940a3db0 msgr2=0x7fcd940a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd940a3db0 0x7fcd940a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd90006010 tx=0x7fcd9001a040 comp rx=0 tx=0).stop 2024-04-26T18:18:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 msgr2=0x7fcda81023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda81023e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fcd98007e20 tx=0x7fcd9800a340 comp rx=0 tx=0).stop 2024-04-26T18:18:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 shutdown_connections 2024-04-26T18:18:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fcd940a3db0 0x7fcd940a6270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:23.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcda806b9c0 0x7fcda81023e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:23.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcda8109490 0x7fcda8102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:23.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 --2- 172.21.15.90:0/1725455715 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcda806c360 0x7fcda8102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:23.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 >> 172.21.15.90:0/1725455715 conn(0x7fcda8079290 msgr2=0x7fcda8100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:23.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 shutdown_connections 2024-04-26T18:18:23.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:23.123+0000 7fcdad3cd700 1 -- 172.21.15.90:0/1725455715 wait complete. 2024-04-26T18:18:23.124 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-04-26T18:18:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:22 smithi119 ceph-mon[32752]: pgmap v197: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-04-26T18:18:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:22 smithi167 ceph-mon[32751]: pgmap v197: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-04-26T18:18:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:22 smithi090 ceph-mon[29180]: pgmap v197: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.9 MiB/s wr, 125 op/s 2024-04-26T18:18:23.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.64 MB/sec execute 131 sec latency 16624.855 ms 2024-04-26T18:18:23.821 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db / --timeout 60 --expiration 91.7 --await-for 42.0' 2024-04-26T18:18:23.821 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db / --timeout 60 --expiration 91.7 --await-for 42.0 2024-04-26T18:18:24.059 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:18:24.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:23 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1725455715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:18:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:23 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1725455715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:18:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:23 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1725455715' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:18:24.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.60 MB/sec execute 132 sec latency 17624.898 ms 2024-04-26T18:18:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:24 smithi090 ceph-mon[29180]: pgmap v198: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.6 MiB/s wr, 152 op/s 2024-04-26T18:18:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:24 smithi119 ceph-mon[32752]: pgmap v198: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.6 MiB/s wr, 152 op/s 2024-04-26T18:18:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:24 smithi167 ceph-mon[32751]: pgmap v198: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 2.6 MiB/s wr, 152 op/s 2024-04-26T18:18:25.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.56 MB/sec execute 133 sec latency 18624.950 ms 2024-04-26T18:18:25.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 -- 172.21.15.90:0/2457866560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d481094a0 msgr2=0x7f0d48109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:25.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 --2- 172.21.15.90:0/2457866560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d481094a0 0x7f0d48109880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0d380099c0 tx=0x7f0d38009cd0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 -- 172.21.15.90:0/2457866560 shutdown_connections 2024-04-26T18:18:25.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 --2- 172.21.15.90:0/2457866560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d481094a0 0x7f0d48109880 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 --2- 172.21.15.90:0/2457866560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d48103e90 0x7f0d481102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 --2- 172.21.15.90:0/2457866560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d481034d0 0x7f0d48103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.752 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 -- 172.21.15.90:0/2457866560 >> 172.21.15.90:0/2457866560 conn(0x7f0d4807a4f0 msgr2=0x7f0d4807a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 -- 172.21.15.90:0/2457866560 shutdown_connections 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 -- 172.21.15.90:0/2457866560 wait complete. 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.737+0000 7f0d4d714700 1 Processor -- start 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d4d714700 1 -- start start 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d4d714700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d481034d0 0x7f0d4819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d4d714700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d48103e90 0x7f0d4819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d4d714700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d481094a0 0x7f0d48199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d4d714700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481135d0 con 0x7f0d481094a0 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d4d714700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48113450 con 0x7f0d481034d0 2024-04-26T18:18:25.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d4d714700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48113750 con 0x7f0d48103e90 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d481094a0 0x7f0d48199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d481094a0 0x7f0d48199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33926/0 (socket says 172.21.15.90:33926) 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d46ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d481034d0 0x7f0d4819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 -- 172.21.15.90:0/3012325691 learned_addr learned my addr 172.21.15.90:0/3012325691 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d467fc700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d48103e90 0x7f0d4819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 -- 172.21.15.90:0/3012325691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d48103e90 msgr2=0x7f0d4819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d48103e90 0x7f0d4819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 -- 172.21.15.90:0/3012325691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d481034d0 msgr2=0x7f0d4819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d481034d0 0x7f0d4819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 -- 172.21.15.90:0/3012325691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4819a030 con 0x7f0d481094a0 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d467fc700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d48103e90 0x7f0d4819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d477fe700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0d481094a0 0x7f0d48199850 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0d40007d70 tx=0x7f0d4000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.738+0000 7f0d2ffff700 1 -- 172.21.15.90:0/3012325691 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40016020 con 0x7f0d481094a0 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.739+0000 7f0d4d714700 1 -- 172.21.15.90:0/3012325691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4819a320 con 0x7f0d481094a0 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.739+0000 7f0d2ffff700 1 -- 172.21.15.90:0/3012325691 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d4000ad40 con 0x7f0d481094a0 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.739+0000 7f0d2ffff700 1 -- 172.21.15.90:0/3012325691 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400115e0 con 0x7f0d481094a0 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.739+0000 7f0d4d714700 1 -- 172.21.15.90:0/3012325691 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a6530 con 0x7f0d481094a0 2024-04-26T18:18:25.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.741+0000 7f0d2ffff700 1 -- 172.21.15.90:0/3012325691 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f0d40019020 con 0x7f0d481094a0 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.741+0000 7f0d2ffff700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f0d300a3e50 0x7f0d300a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.741+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f0d300a3e50 0x7f0d300a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.741+0000 7f0d2ffff700 1 -- 172.21.15.90:0/3012325691 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f0d400d0610 con 0x7f0d481094a0 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/3012325691 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f0d300a3e50 0x7f0d300a6310 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009990 tx=0x7f0d38005c40 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 Processor -- start 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 -- start start 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2b50 0x556a7c2bd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556a7c2058d0 con 0x556a7c2b2b50 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556a7c1a4770 con 0x556a7c2b2230 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a7c201890 con 0x556a7c2bda00 2024-04-26T18:18:25.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d467fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2b50 0x556a7c2bd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d467fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2b50 0x556a7c2bd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33928/0 (socket says 172.21.15.90:33928) 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.742+0000 7f0d467fc700 1 -- 172.21.15.90:0/2683764838 learned_addr learned my addr 172.21.15.90:0/2683764838 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d477fe700 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d467fc700 1 -- 172.21.15.90:0/2683764838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 msgr2=0x556a7c2bfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d467fc700 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2bfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d467fc700 1 -- 172.21.15.90:0/2683764838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 msgr2=0x556a7c2b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d467fc700 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d467fc700 1 -- 172.21.15.90:0/2683764838 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7c152a60 con 0x556a7c2b2b50 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d477fe700 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d467fc700 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2b50 0x556a7c2bd4c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009cb0 tx=0x7f0d3c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3889b83551135d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d2d7fa700 1 -- 172.21.15.90:0/2683764838 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c016070 con 0x556a7c2b2b50 2024-04-26T18:18:25.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d2d7fa700 1 -- 172.21.15.90:0/2683764838 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3c004ce0 con 0x556a7c2b2b50 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.743+0000 7f0d2d7fa700 1 -- 172.21.15.90:0/2683764838 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c005550 con 0x556a7c2b2b50 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2683764838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2b50 msgr2=0x556a7c2bd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2b50 0x556a7c2bd4c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009cb0 tx=0x7f0d3c00e2f0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2683764838 shutdown_connections 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2bfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2683764838 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2b50 0x556a7c2bd4c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2683764838 >> 172.21.15.90:0/2683764838 conn(0x556a7c1dfed0 msgr2=0x556a7c2ae720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2683764838 shutdown_connections 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2683764838 wait complete. 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- start start 2024-04-26T18:18:25.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 Processor -- start 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2ba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2ba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x556a7c2c05d0 con 0x556a7c2bda00 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x556a7c16bae0 con 0x556a7c2b2230 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556a7c2c0750 con 0x556a7c2b2b50 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d57fa6b80 10 client.0 _ll_register_callbacks cb 0x556a7c199710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.744+0000 7f0d477fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2ba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2ba9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33932/0 (socket says 172.21.15.90:33932) 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 -- 172.21.15.90:0/2293187360 learned_addr learned my addr 172.21.15.90:0/2293187360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:25.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d467fc700 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2ba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 -- 172.21.15.90:0/2293187360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 msgr2=0x556a7c2ba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2ba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 -- 172.21.15.90:0/2293187360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 msgr2=0x556a7c2bd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 -- 172.21.15.90:0/2293187360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7c2bb090 con 0x556a7c2bda00 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d467fc700 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d477fe700 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2ba9d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d40098b60 tx=0x7f0d4009cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415da6d7d173ca70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d2aff5700 1 -- 172.21.15.90:0/2293187360 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d400870f0 con 0x556a7c2bda00 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d2aff5700 1 -- 172.21.15.90:0/2293187360 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8455+0+0 (secure 0 0 0) 0x7f0d4009c960 con 0x556a7c2bda00 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c004ce0 con 0x556a7c2bda00 2024-04-26T18:18:25.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d2aff5700 1 -- 172.21.15.90:0/2293187360 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d400ba050 con 0x556a7c2bda00 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d57fa6b80 10 client.16041 fetch_fsmap learned FSMap version 29 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a7c2cb030 con 0x556a7c2bda00 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d2aff5700 1 -- 172.21.15.90:0/2293187360 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3c004ce0 con 0x556a7c2bda00 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d2aff5700 1 -- 172.21.15.90:0/2293187360 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f0d40098450 con 0x556a7c2bda00 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d57fa6b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.745+0000 7f0d57fa6b80 10 client.16041 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.746+0000 7f0d57fa6b80 20 client.16041 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.746+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c2cd670 0x556a7c2cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.746+0000 7f0d57fa6b80 4 client.16041 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:18:25.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.746+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a7c2132b0 con 0x556a7c2cd670 2024-04-26T18:18:25.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.746+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c2cd670 0x556a7c2cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:25.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.746+0000 7f0d2aff5700 10 client.16041 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:25.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:25.746+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c2cd670 0x556a7c2cfa60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66e468389f5cbb9e server_cookie=6dd8f05643b333c7 in_seq=0 out_seq=0 2024-04-26T18:18:26.673 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.51 MB/sec execute 134 sec latency 19625.048 ms 2024-04-26T18:18:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:26 smithi119 ceph-mon[32752]: pgmap v199: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-26T18:18:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:26 smithi167 ceph-mon[32751]: pgmap v199: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-26T18:18:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:26 smithi090 ceph-mon[29180]: pgmap v199: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-26T18:18:27.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.47 MB/sec execute 135 sec latency 20625.187 ms 2024-04-26T18:18:28.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.43 MB/sec execute 136 sec latency 21625.310 ms 2024-04-26T18:18:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:28 smithi119 ceph-mon[32752]: pgmap v200: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-26T18:18:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:28 smithi167 ceph-mon[32751]: pgmap v200: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-26T18:18:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:28 smithi090 ceph-mon[29180]: pgmap v200: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 1.5 MiB/s wr, 66 op/s 2024-04-26T18:18:29.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.39 MB/sec execute 137 sec latency 22625.346 ms 2024-04-26T18:18:30.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.35 MB/sec execute 138 sec latency 23625.448 ms 2024-04-26T18:18:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:30 smithi119 ceph-mon[32752]: pgmap v201: 97 pgs: 97 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 90 op/s 2024-04-26T18:18:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:30 smithi167 ceph-mon[32751]: pgmap v201: 97 pgs: 97 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 90 op/s 2024-04-26T18:18:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:30 smithi090 ceph-mon[29180]: pgmap v201: 97 pgs: 97 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 155 KiB/s rd, 2.2 MiB/s wr, 90 op/s 2024-04-26T18:18:31.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.32 MB/sec execute 139 sec latency 24625.496 ms 2024-04-26T18:18:32.121 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:31 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:18:32.121 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:31 smithi167 systemd[1]: Stopped Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:18:32.121 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:31 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 40.273s CPU time. 2024-04-26T18:18:32.121 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:31 smithi167 systemd[1]: Starting Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:18:32.121 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 podman[68211]: 2024-04-26 18:18:31.959829914 +0000 UTC m=+0.018488271 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:18:32.121 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 podman[68211]: 2024-04-26 18:18:32.121773324 +0000 UTC m=+0.180431668 container create a74b35e0ba29feede6f1987a8eea37f858872218d926ade9bb061944f93a1c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T18:18:32.424 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 podman[68211]: 2024-04-26 18:18:32.341397718 +0000 UTC m=+0.400056063 container init a74b35e0ba29feede6f1987a8eea37f858872218d926ade9bb061944f93a1c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T18:18:32.424 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 podman[68211]: 2024-04-26 18:18:32.344099128 +0000 UTC m=+0.402757479 container start a74b35e0ba29feede6f1987a8eea37f858872218d926ade9bb061944f93a1c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:18:32.424 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mds[68229]: -- 172.21.15.167:0/1410038434 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fa4a295a40 con 0x55fa4a268800 2024-04-26T18:18:32.424 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: starting mds.i at 2024-04-26T18:18:32.424 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 bash[68211]: a74b35e0ba29feede6f1987a8eea37f858872218d926ade9bb061944f93a1c1d 2024-04-26T18:18:32.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.28 MB/sec execute 140 sec latency 25625.564 ms 2024-04-26T18:18:32.817 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:18:32 smithi167 systemd[1]: Started Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:18:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: pgmap v202: 97 pgs: 97 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.5 MiB/s wr, 55 op/s 2024-04-26T18:18:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: daemon mds.i restarted 2024-04-26T18:18:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:18:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:18:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:boot 2024-04-26T18:18:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:18:33.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:18:33.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:18:33.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:32 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: pgmap v202: 97 pgs: 97 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.5 MiB/s wr, 55 op/s 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: daemon mds.i restarted 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:boot 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:18:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:18:33.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:32 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: pgmap v202: 97 pgs: 97 active+clean; 98 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.5 MiB/s wr, 55 op/s 2024-04-26T18:18:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: daemon mds.i restarted 2024-04-26T18:18:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:18:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:18:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:boot 2024-04-26T18:18:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:18:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:18:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:18:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:32 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:33.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.24 MB/sec execute 141 sec latency 26625.611 ms 2024-04-26T18:18:34.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.20 MB/sec execute 142 sec latency 27625.703 ms 2024-04-26T18:18:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:34 smithi119 ceph-mon[32752]: pgmap v204: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:resolve 2024-04-26T18:18:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:34 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:34 smithi167 ceph-mon[32751]: pgmap v204: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:resolve 2024-04-26T18:18:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:34 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:34 smithi090 ceph-mon[29180]: pgmap v204: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:resolve 2024-04-26T18:18:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:34 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37d0ff9640 1 -- 172.21.15.90:0/1878461310 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec0955cee0 con 0x55ec09675740 2024-04-26T18:18:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37d0ff9640 10 client.16023 handle_command_reply: tid=0 2024-04-26T18:18:35.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 client.16023 shutdown 2024-04-26T18:18:35.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f0d2aff5700 1 -- 172.21.15.90:0/2293187360 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a7c2132b0 con 0x556a7c2cd670 2024-04-26T18:18:35.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f0d2aff5700 10 client.16041 handle_command_reply: tid=0 2024-04-26T18:18:35.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f0d57fa6b80 1 client.16041 shutdown 2024-04-26T18:18:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 msgr2=0x55ec095f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:35.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095f8490 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f37e4005cd0 tx=0x7f37e4003ca0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 msgr2=0x556a7c2ba9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:35.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2ba9d0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0d40098b60 tx=0x7f0d4009cf70 comp rx=0 tx=0).stop 2024-04-26T18:18:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 shutdown_connections 2024-04-26T18:18:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ec09675740 0x55ec09677b60 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ec095fe4f0 0x55ec09673b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec095ef720 0x55ec095f89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f37f23ca740 1 --2- 172.21.15.90:0/1878461310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ec095eee00 0x55ec095f8490 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 >> 172.21.15.90:0/1878461310 conn(0x55ec095eb410 msgr2=0x55ec095f0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:35.420 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f37d0ff9640 0 client.16023 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.417+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 shutdown_connections 2024-04-26T18:18:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c2cd670 0x556a7c2cfa60 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2ba9d0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2bd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/2293187360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2ba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d2aff5700 0 client.16041 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:35.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 >> 172.21.15.90:0/2293187360 conn(0x556a7c1dfed0 msgr2=0x556a7c2ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 shutdown_connections 2024-04-26T18:18:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f37f23ca740 1 -- 172.21.15.90:0/1878461310 wait complete. 2024-04-26T18:18:35.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 shutdown_connections 2024-04-26T18:18:35.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/2293187360 wait complete. 2024-04-26T18:18:35.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f37f23ca740 20 client.16023 trim_cache size 0 max 16384 2024-04-26T18:18:35.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.418+0000 7f0d57fa6b80 20 client.16041 trim_cache size 0 max 16384 2024-04-26T18:18:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:18:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:18:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:18:35.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f37d40a3dc0 msgr2=0x7f37d40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f37d40a3dc0 0x7f37d40a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f37e004d000 tx=0x7f37e004b000 comp rx=0 tx=0).stop 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 msgr2=0x7f37e81994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e81994e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f37d800c890 tx=0x7f37d800cd50 comp rx=0 tx=0).stop 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 shutdown_connections 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f37d40a3dc0 0x7f37d40a6280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f37e8108b30 0x7f37e819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37e806bb20 0x7f37e81994e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 --2- 172.21.15.90:0/3336293744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37e806b160 0x7f37e8198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 >> 172.21.15.90:0/3336293744 conn(0x7f37e8077630 msgr2=0x7f37e81006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 shutdown_connections 2024-04-26T18:18:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:35.419+0000 7f37ef6b0640 1 -- 172.21.15.90:0/3336293744 wait complete. 2024-04-26T18:18:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.439+0000 7f0d57fa6b80 1 Processor -- start 2024-04-26T18:18:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d57fa6b80 1 -- start start 2024-04-26T18:18:35.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2b5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481b0320 con 0x556a7c2bda00 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481adbb0 con 0x556a7c2b2230 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481aba20 con 0x556a7c2b2b50 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d477fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d46ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d46ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:46074/0 (socket says 172.21.15.90:46074) 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d46ffd700 1 -- 172.21.15.90:0/4245237880 learned_addr learned my addr 172.21.15.90:0/4245237880 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d477fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2bccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46842/0 (socket says 172.21.15.90:46842) 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d467fc700 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2b5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:35.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d477fe700 1 -- 172.21.15.90:0/4245237880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 msgr2=0x556a7c2b5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d477fe700 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2b5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d477fe700 1 -- 172.21.15.90:0/4245237880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 msgr2=0x556a7c2ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d477fe700 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d477fe700 1 -- 172.21.15.90:0/4245237880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a7c2cfd50 con 0x556a7c2bda00 2024-04-26T18:18:35.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.440+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.441+0000 7f0d477fe700 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2bccd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0d4009c400 tx=0x7f0d400b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab568b38d078f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.441+0000 7f0d2cff9700 1 -- 172.21.15.90:0/4245237880 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40065070 con 0x556a7c2bda00 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.441+0000 7f0d2cff9700 1 -- 172.21.15.90:0/4245237880 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d400a5370 con 0x556a7c2bda00 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.441+0000 7f0d2cff9700 1 -- 172.21.15.90:0/4245237880 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4009c960 con 0x556a7c2bda00 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/4245237880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 msgr2=0x556a7c2bccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2bccd0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0d4009c400 tx=0x7f0d400b7000 comp rx=0 tx=0).stop 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/4245237880 shutdown_connections 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2bda00 0x556a7c2bccd0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2230 0x556a7c2ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/4245237880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2b2b50 0x556a7c2b5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/4245237880 >> 172.21.15.90:0/4245237880 conn(0x556a7c1dfed0 msgr2=0x556a7c2b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/4245237880 shutdown_connections 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/4245237880 wait complete. 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 -- start start 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.442+0000 7f0d57fa6b80 1 Processor -- start 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2230 0x556a7c2c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2b50 0x556a7c2c5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d57fa6b80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481ac070 con 0x556a7c2b2230 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481ad4e0 con 0x556a7c2b2b50 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d57fa6b80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d481ac740 con 0x556a7c2bda00 2024-04-26T18:18:35.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d57fa6b80 10 client.0 _ll_register_callbacks cb 0x556a7c199710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2230 0x556a7c2c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d467fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2b50 0x556a7c2c5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2230 0x556a7c2c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46848/0 (socket says 172.21.15.90:46848) 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 -- 172.21.15.90:0/3962416223 learned_addr learned my addr 172.21.15.90:0/3962416223 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d477fe700 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 -- 172.21.15.90:0/3962416223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 msgr2=0x556a7c2c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 -- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2b50 msgr2=0x556a7c2c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2b50 0x556a7c2c5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d46ffd700 1 -- 172.21.15.90:0/3962416223 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a7c2c3210 con 0x556a7c2b2230 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.443+0000 7f0d467fc700 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2b50 0x556a7c2c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d46ffd700 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2230 0x556a7c2c59e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006010 tx=0x7f0d3800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0884d7407212d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:35.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d2aff5700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d2aff5700 1 -- 172.21.15.90:0/3962416223 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38024cf0 con 0x556a7c2b2230 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d2aff5700 1 -- 172.21.15.90:0/3962416223 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8497+0+0 (secure 0 0 0) 0x7f0d38030070 con 0x556a7c2b2230 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d400a5370 con 0x556a7c2b2230 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d2aff5700 1 -- 172.21.15.90:0/3962416223 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0d38040050 con 0x556a7c2b2230 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 10 client.16047 fetch_fsmap learned FSMap version 34 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x556a7c334650 con 0x556a7c2b2230 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d2aff5700 1 -- 172.21.15.90:0/3962416223 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d400a5370 con 0x556a7c2b2230 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d2aff5700 1 -- 172.21.15.90:0/3962416223 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 34) ==== 8342+0+0 (secure 0 0 0) 0x7f0d3802e070 con 0x556a7c2b2230 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 34 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 10 client.16047 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 20 client.16047 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:35.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c334dd0 0x556a7c3371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 4 client.16047 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.7, "await_for": 42.0}] 2024-04-26T18:18:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.444+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.7, "await_for": 42.0}) -- 0x556a7c2ba280 con 0x556a7c334dd0 2024-04-26T18:18:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.445+0000 7f0d467fc700 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c334dd0 0x556a7c3371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.445+0000 7f0d467fc700 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c334dd0 0x556a7c3371f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e600a9d5531cd4a1 server_cookie=764778a04364c177 in_seq=0 out_seq=0 2024-04-26T18:18:35.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:35.445+0000 7f0d2aff5700 10 client.16047 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:35.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370284 5.17 MB/sec execute 143 sec latency 28625.768 ms 2024-04-26T18:18:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:35 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:18:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:35 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:18:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:reconnect 2024-04-26T18:18:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:35 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:rejoin 2024-04-26T18:18:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:35 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:18:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:35 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:18:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:reconnect 2024-04-26T18:18:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:35 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:rejoin 2024-04-26T18:18:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:35 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:18:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:35 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:18:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:reconnect 2024-04-26T18:18:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:35 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:rejoin 2024-04-26T18:18:36.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.379+0000 7f0d2aff5700 1 -- 172.21.15.90:0/3962416223 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: -9 ) ==== 8+0+585 (crc 0 0 0) 0x556a7c2ba280 con 0x556a7c334dd0 2024-04-26T18:18:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.379+0000 7f0d2aff5700 10 client.16047 handle_command_reply: tid=0 2024-04-26T18:18:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 client.16047 shutdown 2024-04-26T18:18:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2230 msgr2=0x556a7c2c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:36.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2230 0x556a7c2c59e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006010 tx=0x7f0d3800bd30 comp rx=0 tx=0).stop 2024-04-26T18:18:36.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 shutdown_connections 2024-04-26T18:18:36.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x556a7c334dd0 0x556a7c3371f0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:36.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x556a7c2b2230 0x556a7c2c59e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:36.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556a7c2bda00 0x556a7c2c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 --2- 172.21.15.90:0/3962416223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556a7c2b2b50 0x556a7c2c5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d2aff5700 0 client.16047 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 >> 172.21.15.90:0/3962416223 conn(0x556a7c1dfed0 msgr2=0x556a7c2b5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 shutdown_connections 2024-04-26T18:18:36.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 1 -- 172.21.15.90:0/3962416223 wait complete. 2024-04-26T18:18:36.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T18:18:36.380+0000 7f0d57fa6b80 20 client.16047 trim_cache size 0 max 16384 2024-04-26T18:18:36.386 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T18:18:36.387 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 34, 2024-04-26T18:18:36.387 INFO:teuthology.orchestra.run.smithi090.stdout: "leader": 24449, 2024-04-26T18:18:36.387 INFO:teuthology.orchestra.run.smithi090.stdout: "set_version": 18, 2024-04-26T18:18:36.387 INFO:teuthology.orchestra.run.smithi090.stdout: "sets": { 2024-04-26T18:18:36.387 INFO:teuthology.orchestra.run.smithi090.stdout: "f82e6e3": { 2024-04-26T18:18:36.387 INFO:teuthology.orchestra.run.smithi090.stdout: "version": 18, 2024-04-26T18:18:36.387 INFO:teuthology.orchestra.run.smithi090.stdout: "age_ref": 0.0, 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FAILED", 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "timeout": 60.0, 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "expiration": 91.7, 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "members": { 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "file:/": { 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "excluded": false, 2024-04-26T18:18:36.388 INFO:teuthology.orchestra.run.smithi090.stdout: "state": { 2024-04-26T18:18:36.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FAILED", 2024-04-26T18:18:36.389 INFO:teuthology.orchestra.run.smithi090.stdout: "age": 0.0 2024-04-26T18:18:36.389 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:18:36.389 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:18:36.389 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:18:36.389 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:18:36.389 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T18:18:36.390 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T18:18:36.390 INFO:teuthology.orchestra.run.smithi090.stderr:Error EBADF: 2024-04-26T18:18:36.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 370532 5.13 MB/sec execute 144 sec latency 29413.338 ms 2024-04-26T18:18:36.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:36 smithi090 ceph-mon[29180]: pgmap v205: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:36.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:36 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:36.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:36 smithi090 ceph-mon[29180]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:18:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:36 smithi119 ceph-mon[32752]: pgmap v205: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:36 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:36 smithi119 ceph-mon[32752]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:18:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:36 smithi167 ceph-mon[32751]: pgmap v205: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:36 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:36 smithi167 ceph-mon[32751]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:18:37.177 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-26T18:18:37.178 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: 9 (EBADF), stdout: { "epoch": 34, "leader": 24449, "set_version": 18, "sets": { "f82e6e3": { "version": 18, "age_ref": 0.0, "state": { "name": "FAILED", "age": 0.0 }, "timeout": 60.0, "expiration": 91.7, "members": { "file:/": { "excluded": false, "state": { "name": "FAILED", "age": 0.0 } } } } } } 2024-04-26T18:18:37.178 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/quiescer.py", line 354, in _run self.do_quiesce(d) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/quiescer.py", line 310, in do_quiesce raise RuntimeError(f"Error quiescing set '{set_id}': {rcinfo(rc)}") RuntimeError: Error quiescing set 'f82e6e3': 9 (EBADF) 2024-04-26T18:18:37.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 372831 5.19 MB/sec execute 145 sec latency 21.521 ms 2024-04-26T18:18:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:37 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:18:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:37 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:18:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:37 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:active 2024-04-26T18:18:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:37 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:37 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:18:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:37 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:18:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:37 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:18:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:37 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:active 2024-04-26T18:18:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:37 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:37 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:18:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:37 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:18:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:37 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:18:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:37 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3534880230,v1:172.21.15.119:6837/3534880230] up:active 2024-04-26T18:18:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:37 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:18:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:37 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:18:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:38 smithi119 ceph-mon[32752]: pgmap v206: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:38.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:18:38 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:18:38.201+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:18:38.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 375040 5.21 MB/sec execute 146 sec latency 15.466 ms 2024-04-26T18:18:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:38 smithi167 ceph-mon[32751]: pgmap v206: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:38 smithi090 ceph-mon[29180]: pgmap v206: 97 pgs: 97 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 1.7 MiB/s wr, 52 op/s 2024-04-26T18:18:39.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 376887 5.26 MB/sec execute 147 sec latency 15.321 ms 2024-04-26T18:18:40.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 379787 5.34 MB/sec execute 148 sec latency 28.754 ms 2024-04-26T18:18:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:40 smithi119 ceph-mon[32752]: pgmap v207: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-04-26T18:18:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:40 smithi167 ceph-mon[32751]: pgmap v207: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-04-26T18:18:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:40 smithi090 ceph-mon[29180]: pgmap v207: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-04-26T18:18:41.674 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.34 MB/sec execute 149 sec latency 128.980 ms 2024-04-26T18:18:42.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.31 MB/sec execute 150 sec latency 1129.011 ms 2024-04-26T18:18:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:42 smithi119 ceph-mon[32752]: pgmap v208: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-04-26T18:18:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:42 smithi167 ceph-mon[32751]: pgmap v208: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-04-26T18:18:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:42 smithi090 ceph-mon[29180]: pgmap v208: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.1 MiB/s wr, 147 op/s 2024-04-26T18:18:43.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.27 MB/sec execute 151 sec latency 2129.063 ms 2024-04-26T18:18:44.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.24 MB/sec execute 152 sec latency 3129.159 ms 2024-04-26T18:18:44.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:44 smithi090 ceph-mon[29180]: pgmap v209: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-26T18:18:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:44 smithi119 ceph-mon[32752]: pgmap v209: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-26T18:18:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:44 smithi167 ceph-mon[32751]: pgmap v209: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.8 MiB/s wr, 224 op/s 2024-04-26T18:18:45.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.20 MB/sec execute 153 sec latency 4129.228 ms 2024-04-26T18:18:46.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.17 MB/sec execute 154 sec latency 5129.309 ms 2024-04-26T18:18:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:46 smithi119 ceph-mon[32752]: pgmap v210: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T18:18:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:46 smithi167 ceph-mon[32751]: pgmap v210: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T18:18:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:46 smithi090 ceph-mon[29180]: pgmap v210: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T18:18:47.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.14 MB/sec execute 155 sec latency 6129.391 ms 2024-04-26T18:18:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 -- 172.21.15.90:0/1786235947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 msgr2=0x7f612410f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 --2- 172.21.15.90:0/1786235947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f612410f810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f611c009cd0 tx=0x7f611c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 -- 172.21.15.90:0/1786235947 shutdown_connections 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 --2- 172.21.15.90:0/1786235947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f612410f810 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 --2- 172.21.15.90:0/1786235947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6124102940 0x7f6124102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 --2- 172.21.15.90:0/1786235947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6124108950 0x7f6124108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 -- 172.21.15.90:0/1786235947 >> 172.21.15.90:0/1786235947 conn(0x7f6124079dc0 msgr2=0x7f612407a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 -- 172.21.15.90:0/1786235947 shutdown_connections 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.517+0000 7f612bb66640 1 -- 172.21.15.90:0/1786235947 wait complete. 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 Processor -- start 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 -- start start 2024-04-26T18:18:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6124102940 0x7f6124199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f6124199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6124108950 0x7f612419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6124112c10 con 0x7f6124103300 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6124112a90 con 0x7f6124108950 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612bb66640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6124112d90 con 0x7f6124102940 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f6124199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f6124199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47978/0 (socket says 172.21.15.90:47978) 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 -- 172.21.15.90:0/3732486901 learned_addr learned my addr 172.21.15.90:0/3732486901 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129902640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6124102940 0x7f6124199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612a103640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6124108950 0x7f612419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 -- 172.21.15.90:0/3732486901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6124102940 msgr2=0x7f6124199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6124102940 0x7f6124199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 -- 172.21.15.90:0/3732486901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6124108950 msgr2=0x7f612419d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6124108950 0x7f612419d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f6129101640 1 -- 172.21.15.90:0/3732486901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612419dfb0 con 0x7f6124103300 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.518+0000 7f612a103640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6124108950 0x7f612419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.519+0000 7f6129101640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f6124199550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f611400c970 tx=0x7f611400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.519+0000 7f611affd640 1 -- 172.21.15.90:0/3732486901 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6114007bb0 con 0x7f6124103300 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.519+0000 7f611affd640 1 -- 172.21.15.90:0/3732486901 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6114007d50 con 0x7f6124103300 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.519+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612419e2a0 con 0x7f6124103300 2024-04-26T18:18:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.519+0000 7f611affd640 1 -- 172.21.15.90:0/3732486901 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611400f450 con 0x7f6124103300 2024-04-26T18:18:48.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.519+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61241a5b30 con 0x7f6124103300 2024-04-26T18:18:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.521+0000 7f611affd640 1 -- 172.21.15.90:0/3732486901 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f6114004f20 con 0x7f6124103300 2024-04-26T18:18:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 Processor -- start 2024-04-26T18:18:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 -- start start 2024-04-26T18:18:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a5f00 0x5575a23a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a6820 0x5575a23b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5575a230c460 con 0x5575a23a5f00 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5575a22c6590 con 0x5575a23b10e0 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.522+0000 7f612e87f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5575a2308420 con 0x5575a23a6820 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f611affd640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f61100a3dc0 0x7f61100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a5f00 0x5575a23a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a5f00 0x5575a23a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47986/0 (socket says 172.21.15.90:47986) 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 -- 172.21.15.90:0/792050109 learned_addr learned my addr 172.21.15.90:0/792050109 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f61100a3dc0 0x7f61100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f6129902640 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a6820 0x5575a23b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f611affd640 1 -- 172.21.15.90:0/3732486901 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8497+0+0 (secure 0 0 0) 0x7f61140d0660 con 0x7f6124103300 2024-04-26T18:18:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f6129101640 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 -- 172.21.15.90:0/792050109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a6820 msgr2=0x5575a23b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a6820 0x5575a23b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 -- 172.21.15.90:0/792050109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 msgr2=0x5575a23b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 -- 172.21.15.90:0/792050109 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a22d30d0 con 0x5575a23a5f00 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f6129101640 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a5f00 0x5575a23a62e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f611c005fe0 tx=0x7f611c031d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447dba1da29631e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f612a103640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f61100a3dc0 0x7f61100a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f611c035000 tx=0x7f611c048000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f610ffff640 1 -- 172.21.15.90:0/792050109 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c04e070 con 0x5575a23a5f00 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f610ffff640 1 -- 172.21.15.90:0/792050109 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c036040 con 0x5575a23a5f00 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.523+0000 7f610ffff640 1 -- 172.21.15.90:0/792050109 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c030800 con 0x5575a23a5f00 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 -- 172.21.15.90:0/792050109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a5f00 msgr2=0x5575a23a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a5f00 0x5575a23a62e0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f611c005fe0 tx=0x7f611c031d80 comp rx=0 tx=0).stop 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 -- 172.21.15.90:0/792050109 shutdown_connections 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a6820 0x5575a23b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 --2- 172.21.15.90:0/792050109 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a5f00 0x5575a23a62e0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 -- 172.21.15.90:0/792050109 >> 172.21.15.90:0/792050109 conn(0x5575a23a2410 msgr2=0x5575a23a27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 -- 172.21.15.90:0/792050109 shutdown_connections 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.524+0000 7f612e87f740 1 -- 172.21.15.90:0/792050109 wait complete. 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 -- start start 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 Processor -- start 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5575a23b3cb0 con 0x5575a23a6820 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5575a2293b60 con 0x5575a23b10e0 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5575a23b3e30 con 0x5575a23a5f00 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612e87f740 10 client.0 _ll_register_callbacks cb 0x5575a22b82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f6129101640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f6129101640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23ae670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47988/0 (socket says 172.21.15.90:47988) 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f6129101640 1 -- 172.21.15.90:0/2110304262 learned_addr learned my addr 172.21.15.90:0/2110304262 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f612a103640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.525+0000 7f6129902640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f6129101640 1 -- 172.21.15.90:0/2110304262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 msgr2=0x5575a23ae130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f6129101640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23ae130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f6129101640 1 -- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 msgr2=0x5575a23aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f6129101640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23aebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f6129101640 1 -- 172.21.15.90:0/2110304262 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a201e210 con 0x5575a23a6820 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612a103640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f6129902640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f6129101640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23ae670 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6114098f80 tx=0x7f611409bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965302f1ee8315b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f610d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f610d7fa640 1 -- 172.21.15.90:0/2110304262 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611401d040 con 0x5575a23a6820 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f610d7fa640 1 -- 172.21.15.90:0/2110304262 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8497+0+0 (secure 0 0 0) 0x7f6114005210 con 0x5575a23a6820 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611c036040 con 0x5575a23a6820 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f610d7fa640 1 -- 172.21.15.90:0/2110304262 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f61140ce1e0 con 0x5575a23a6820 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 10 client.16059 fetch_fsmap learned FSMap version 36 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5575a20ea620 con 0x5575a23a6820 2024-04-26T18:18:48.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f610d7fa640 1 -- 172.21.15.90:0/2110304262 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c036040 con 0x5575a23a6820 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f610d7fa640 1 -- 172.21.15.90:0/2110304262 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f61140bb2b0 con 0x5575a23a6820 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 10 client.16059 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 20 client.16059 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a23c1280 0x5575a23c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.526+0000 7f612e87f740 4 client.16059 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.527+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575a231a070 con 0x5575a23c1280 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.527+0000 7f6129902640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a23c1280 0x5575a23c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.527+0000 7f610d7fa640 10 client.16059 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.527+0000 7f6129902640 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a23c1280 0x5575a23c3670 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de32b5d62dbc1d6 server_cookie=bd871a876ba3d144 in_seq=0 out_seq=0 2024-04-26T18:18:48.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f610d7fa640 1 -- 172.21.15.90:0/2110304262 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575a231a070 con 0x5575a23c1280 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f610d7fa640 10 client.16059 handle_command_reply: tid=0 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 client.16059 shutdown 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 msgr2=0x5575a23ae670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23ae670 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6114098f80 tx=0x7f611409bf10 comp rx=0 tx=0).stop 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 shutdown_connections 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a23c1280 0x5575a23c3670 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23aebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23ae670 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f610d7fa640 0 client.16059 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 --2- 172.21.15.90:0/2110304262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23ae130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 >> 172.21.15.90:0/2110304262 conn(0x5575a23a2410 msgr2=0x5575a23a8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:48.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 shutdown_connections 2024-04-26T18:18:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 1 -- 172.21.15.90:0/2110304262 wait complete. 2024-04-26T18:18:48.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.528+0000 7f612e87f740 20 client.16059 trim_cache size 0 max 16384 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 Processor -- start 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 -- start start 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23a6820 0x5575a23a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23b10e0 0x5575a23b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61241b0c40 con 0x5575a23b10e0 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ae510 con 0x5575a23a6820 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612e87f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ac360 con 0x5575a23a5f00 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612a103640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23b10e0 0x5575a23b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612a103640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23b10e0 0x5575a23b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48000/0 (socket says 172.21.15.90:48000) 2024-04-26T18:18:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.543+0000 7f612a103640 1 -- 172.21.15.90:0/3331275097 learned_addr learned my addr 172.21.15.90:0/3331275097 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f6129902640 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f6129101640 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23a6820 0x5575a23a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f612a103640 1 -- 172.21.15.90:0/3331275097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 msgr2=0x5575a23bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f612a103640 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f612a103640 1 -- 172.21.15.90:0/3331275097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23a6820 msgr2=0x5575a23a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f612a103640 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23a6820 0x5575a23a1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f612a103640 1 -- 172.21.15.90:0/3331275097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575a23c3ca0 con 0x5575a23b10e0 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f6129902640 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f612a103640 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23b10e0 0x5575a23b0070 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f611c009c20 tx=0x7f611c009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307467479280262b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f610f7fe640 1 -- 172.21.15.90:0/3331275097 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c04e040 con 0x5575a23b10e0 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f610f7fe640 1 -- 172.21.15.90:0/3331275097 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c036040 con 0x5575a23b10e0 2024-04-26T18:18:48.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.544+0000 7f610f7fe640 1 -- 172.21.15.90:0/3331275097 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611c030b80 con 0x5575a23b10e0 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- 172.21.15.90:0/3331275097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23b10e0 msgr2=0x5575a23b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23b10e0 0x5575a23b0070 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f611c009c20 tx=0x7f611c009c50 comp rx=0 tx=0).stop 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- 172.21.15.90:0/3331275097 shutdown_connections 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23b10e0 0x5575a23b0070 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23a6820 0x5575a23a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 --2- 172.21.15.90:0/3331275097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- 172.21.15.90:0/3331275097 >> 172.21.15.90:0/3331275097 conn(0x5575a23a2410 msgr2=0x5575a23af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- 172.21.15.90:0/3331275097 shutdown_connections 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- 172.21.15.90:0/3331275097 wait complete. 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- start start 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 Processor -- start 2024-04-26T18:18:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23b9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23b95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ac9d0 con 0x5575a23a6820 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ade40 con 0x5575a23b10e0 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61241ad0a0 con 0x5575a23a5f00 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f612e87f740 10 client.0 _ll_register_callbacks cb 0x7f61140058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.545+0000 7f6129101640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23b95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23b95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48002/0 (socket says 172.21.15.90:48002) 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 -- 172.21.15.90:0/3125326669 learned_addr learned my addr 172.21.15.90:0/3125326669 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 -- 172.21.15.90:0/3125326669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 msgr2=0x5575a23b9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129902640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f612a103640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23b9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 -- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 msgr2=0x5575a23b6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 -- 172.21.15.90:0/3125326669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575a23b6790 con 0x5575a23a6820 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f612a103640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f6129101640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23b95c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6114098a10 tx=0x7f6114098200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134b8600b96794bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f610d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f610d7fa640 1 -- 172.21.15.90:0/3125326669 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f611409c380 con 0x5575a23a6820 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f610d7fa640 1 -- 172.21.15.90:0/3125326669 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8497+0+0 (secure 0 0 0) 0x7f611409ba00 con 0x5575a23a6820 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f611c036040 con 0x5575a23a6820 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f610d7fa640 1 -- 172.21.15.90:0/3125326669 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f61140ce1f0 con 0x5575a23a6820 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f612e87f740 10 client.16065 fetch_fsmap learned FSMap version 36 2024-04-26T18:18:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5575a2428200 con 0x5575a23a6820 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.546+0000 7f610d7fa640 1 -- 172.21.15.90:0/3125326669 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f611c036040 con 0x5575a23a6820 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f610d7fa640 1 -- 172.21.15.90:0/3125326669 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f61140d00d0 con 0x5575a23a6820 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f612e87f740 10 client.16065 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f612e87f740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f612e87f740 20 client.16065 populate_metadata read hostname 'smithi090' 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f612e87f740 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a2428780 0x5575a242aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f612e87f740 4 client.16065 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575a2313ee0 con 0x5575a2428780 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f6129902640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a2428780 0x5575a242aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f610d7fa640 10 client.16065 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.547+0000 7f6129902640 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a2428780 0x5575a242aba0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66468292a5346700 server_cookie=649053faecb4af4 in_seq=0 out_seq=0 2024-04-26T18:18:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.584+0000 7f610d7fa640 1 -- 172.21.15.90:0/3125326669 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575a2313ee0 con 0x5575a2428780 2024-04-26T18:18:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.584+0000 7f610d7fa640 10 client.16065 handle_command_reply: tid=0 2024-04-26T18:18:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.584+0000 7f612e87f740 1 client.16065 shutdown 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.584+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 msgr2=0x5575a23b95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.584+0000 7f612e87f740 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23b95c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6114098a10 tx=0x7f6114098200 comp rx=0 tx=0).stop 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 shutdown_connections 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5575a2428780 0x5575a242aba0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5575a23b10e0 0x5575a23b6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f610d7fa640 0 client.16065 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5575a23a6820 0x5575a23b95c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 --2- 172.21.15.90:0/3125326669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5575a23a5f00 0x5575a23b9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 >> 172.21.15.90:0/3125326669 conn(0x5575a23a2410 msgr2=0x5575a23b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 shutdown_connections 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 1 -- 172.21.15.90:0/3125326669 wait complete. 2024-04-26T18:18:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.585+0000 7f612e87f740 20 client.16065 trim_cache size 0 max 16384 2024-04-26T18:18:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:18:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:18:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:18:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f61100a3dc0 msgr2=0x7f61100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f61100a3dc0 0x7f61100a6280 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f611c035000 tx=0x7f611c048000 comp rx=0 tx=0).stop 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 msgr2=0x7f6124199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f6124199550 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f611400c970 tx=0x7f611400ce30 comp rx=0 tx=0).stop 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 shutdown_connections 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f61100a3dc0 0x7f61100a6280 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6124108950 0x7f612419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6124103300 0x7f6124199550 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 --2- 172.21.15.90:0/3732486901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6124102940 0x7f6124199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 >> 172.21.15.90:0/3732486901 conn(0x7f6124079dc0 msgr2=0x7f6124100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 shutdown_connections 2024-04-26T18:18:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:18:48.586+0000 7f612bb66640 1 -- 172.21.15.90:0/3732486901 wait complete. 2024-04-26T18:18:48.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.10 MB/sec execute 156 sec latency 7129.480 ms 2024-04-26T18:18:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:48 smithi119 ceph-mon[32752]: pgmap v211: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T18:18:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:48 smithi167 ceph-mon[32751]: pgmap v211: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T18:18:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:48 smithi090 ceph-mon[29180]: pgmap v211: 97 pgs: 97 active+clean; 100 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 184 op/s 2024-04-26T18:18:49.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.07 MB/sec execute 157 sec latency 8129.600 ms 2024-04-26T18:18:50.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.04 MB/sec execute 158 sec latency 9129.662 ms 2024-04-26T18:18:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:50 smithi119 ceph-mon[32752]: pgmap v212: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-04-26T18:18:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:50 smithi167 ceph-mon[32751]: pgmap v212: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-04-26T18:18:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:50 smithi090 ceph-mon[29180]: pgmap v212: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-04-26T18:18:51.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 5.01 MB/sec execute 159 sec latency 10129.730 ms 2024-04-26T18:18:52.441 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0e82560700 time 2024-04-26T18:18:52.091214+0000 2024-04-26T18:18:52.441 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:52.441 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.441 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0e8cb893dd] 2024-04-26T18:18:52.442 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.442 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.442 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.442 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.442 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.443 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.443 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.443 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.443 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.443 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.443 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.444 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: clone() 2024-04-26T18:18:52.444 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: *** Caught signal (Aborted) ** 2024-04-26T18:18:52.444 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: in thread 7f0e82560700 thread_name:ms_dispatch 2024-04-26T18:18:52.444 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.091+0000 7f0e82560700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0e82560700 time 2024-04-26T18:18:52.091214+0000 2024-04-26T18:18:52.444 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:52.444 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.445 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.445 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0e8cb893dd] 2024-04-26T18:18:52.445 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.445 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.445 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.446 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.446 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.446 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.446 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.447 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.447 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.447 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.448 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.448 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: clone() 2024-04-26T18:18:52.448 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.448 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.448 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0e8b911d20] 2024-04-26T18:18:52.449 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: gsignal() 2024-04-26T18:18:52.449 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: abort() 2024-04-26T18:18:52.449 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0e8cb89437] 2024-04-26T18:18:52.449 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.450 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.450 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.450 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.450 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.450 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.451 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.451 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.451 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.451 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.451 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.451 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 16: clone() 2024-04-26T18:18:52.451 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.092+0000 7f0e82560700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:18:52.452 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: in thread 7f0e82560700 thread_name:ms_dispatch 2024-04-26T18:18:52.452 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.452 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.452 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0e8b911d20] 2024-04-26T18:18:52.452 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: gsignal() 2024-04-26T18:18:52.452 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: abort() 2024-04-26T18:18:52.452 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0e8cb89437] 2024-04-26T18:18:52.453 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.453 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.453 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.453 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.453 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.453 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.453 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.454 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.454 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.454 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.454 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.454 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 16: clone() 2024-04-26T18:18:52.454 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:18:52.455 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: -1> 2024-04-26T18:18:52.091+0000 7f0e82560700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0e82560700 time 2024-04-26T18:18:52.091214+0000 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0e8cb893dd] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.456 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: clone() 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 0> 2024-04-26T18:18:52.092+0000 7f0e82560700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: in thread 7f0e82560700 thread_name:ms_dispatch 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0e8b911d20] 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: gsignal() 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: abort() 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0e8cb89437] 2024-04-26T18:18:52.457 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.458 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 16: clone() 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: -1> 2024-04-26T18:18:52.091+0000 7f0e82560700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0e82560700 time 2024-04-26T18:18:52.091214+0000 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0e8cb893dd] 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.459 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: clone() 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 0> 2024-04-26T18:18:52.092+0000 7f0e82560700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: in thread 7f0e82560700 thread_name:ms_dispatch 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0e8b911d20] 2024-04-26T18:18:52.460 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2: gsignal() 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 3: abort() 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0e8cb89437] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0e8cb895a3] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56144936f822] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5614492df598] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5614492e1c17] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5614491dad0b] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5614491db2ac] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5614491c3fad] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0e8ce1de20] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 13: (DispatchQueue::entry()+0x51f) [0x7f0e8ce1afdf] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0e8ceedfc1] 2024-04-26T18:18:52.461 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0e8b9071ca] 2024-04-26T18:18:52.462 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 16: clone() 2024-04-26T18:18:52.462 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:18:52.462 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[63229]: 2024-04-26T18:18:52.462 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 podman[67915]: 2024-04-26 18:18:52.158367778 +0000 UTC m=+0.021861109 container died c1c36f778f9df7ba738288653f6688bc58a67452edd2a52ce5e30995325f4613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T18:18:52.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.98 MB/sec execute 160 sec latency 11129.818 ms 2024-04-26T18:18:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:52 smithi119 ceph-mon[32752]: pgmap v213: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 815 KiB/s wr, 88 op/s 2024-04-26T18:18:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:52 smithi167 ceph-mon[32751]: pgmap v213: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 815 KiB/s wr, 88 op/s 2024-04-26T18:18:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:52 smithi090 ceph-mon[29180]: pgmap v213: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 815 KiB/s wr, 88 op/s 2024-04-26T18:18:53.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 podman[67915]: 2024-04-26 18:18:52.895556692 +0000 UTC m=+0.759050005 container remove c1c36f778f9df7ba738288653f6688bc58a67452edd2a52ce5e30995325f4613 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux ) 2024-04-26T18:18:53.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:52 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:18:53.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:53 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-26T18:18:53.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:18:53 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 4.640s CPU time. 2024-04-26T18:18:53.675 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.95 MB/sec execute 161 sec latency 12129.920 ms 2024-04-26T18:18:54.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.92 MB/sec execute 162 sec latency 13130.009 ms 2024-04-26T18:18:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:54 smithi119 ceph-mon[32752]: pgmap v214: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 841 KiB/s wr, 112 op/s 2024-04-26T18:18:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:54 smithi167 ceph-mon[32751]: pgmap v214: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 841 KiB/s wr, 112 op/s 2024-04-26T18:18:54.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:54 smithi090 ceph-mon[29180]: pgmap v214: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 841 KiB/s wr, 112 op/s 2024-04-26T18:18:55.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.88 MB/sec execute 163 sec latency 14130.096 ms 2024-04-26T18:18:56.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.86 MB/sec execute 164 sec latency 15130.161 ms 2024-04-26T18:18:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:56 smithi119 ceph-mon[32752]: pgmap v215: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 31 op/s 2024-04-26T18:18:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:56 smithi167 ceph-mon[32751]: pgmap v215: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 31 op/s 2024-04-26T18:18:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:56 smithi090 ceph-mon[29180]: pgmap v215: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 31 op/s 2024-04-26T18:18:57.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.83 MB/sec execute 165 sec latency 16130.225 ms 2024-04-26T18:18:58.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.80 MB/sec execute 166 sec latency 17130.314 ms 2024-04-26T18:18:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:18:58 smithi119 ceph-mon[32752]: pgmap v216: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 31 op/s 2024-04-26T18:18:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:18:58 smithi167 ceph-mon[32751]: pgmap v216: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 31 op/s 2024-04-26T18:18:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:18:58 smithi090 ceph-mon[29180]: pgmap v216: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 43 KiB/s wr, 31 op/s 2024-04-26T18:18:59.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.77 MB/sec execute 167 sec latency 18130.415 ms 2024-04-26T18:19:00.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.74 MB/sec execute 168 sec latency 19130.469 ms 2024-04-26T18:19:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:00 smithi119 ceph-mon[32752]: pgmap v217: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 33 op/s 2024-04-26T18:19:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:00 smithi167 ceph-mon[32751]: pgmap v217: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 33 op/s 2024-04-26T18:19:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:00 smithi090 ceph-mon[29180]: pgmap v217: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 43 KiB/s wr, 33 op/s 2024-04-26T18:19:01.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.71 MB/sec execute 169 sec latency 20130.553 ms 2024-04-26T18:19:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 -- 172.21.15.90:0/1424393817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230102540 msgr2=0x7f823010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 --2- 172.21.15.90:0/1424393817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230102540 0x7f823010e670 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8228009ca0 tx=0x7f822802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 -- 172.21.15.90:0/1424393817 shutdown_connections 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 --2- 172.21.15.90:0/1424393817 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230102540 0x7f823010e670 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 --2- 172.21.15.90:0/1424393817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8230101b80 0x7f8230102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 --2- 172.21.15.90:0/1424393817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82301067e0 0x7f8230106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 -- 172.21.15.90:0/1424393817 >> 172.21.15.90:0/1424393817 conn(0x7f8230077630 msgr2=0x7f8230077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 -- 172.21.15.90:0/1424393817 shutdown_connections 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.681+0000 7f8237f22640 1 -- 172.21.15.90:0/1424393817 wait complete. 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 Processor -- start 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 -- start start 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230101b80 0x7f8230199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8230102540 0x7f823019a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82301067e0 0x7f82301a4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111ad0 con 0x7f8230101b80 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111950 con 0x7f82301067e0 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8237f22640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111c50 con 0x7f8230102540 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8235cbe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230101b80 0x7f8230199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8235cbe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230101b80 0x7f8230199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34502/0 (socket says 172.21.15.90:34502) 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f8235cbe640 1 -- 172.21.15.90:0/1632264166 learned_addr learned my addr 172.21.15.90:0/1632264166 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f82354bd640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8230102540 0x7f823019a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.682+0000 7f82364bf640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82301067e0 0x7f82301a4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8235cbe640 1 -- 172.21.15.90:0/1632264166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8230102540 msgr2=0x7f823019a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8235cbe640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8230102540 0x7f823019a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8235cbe640 1 -- 172.21.15.90:0/1632264166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82301067e0 msgr2=0x7f82301a4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8235cbe640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82301067e0 0x7f82301a4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8235cbe640 1 -- 172.21.15.90:0/1632264166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82301a5410 con 0x7f8230101b80 2024-04-26T18:19:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8235cbe640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230101b80 0x7f8230199e50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f822c00d950 tx=0x7f822c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8226ffd640 1 -- 172.21.15.90:0/1632264166 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c017070 con 0x7f8230101b80 2024-04-26T18:19:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8226ffd640 1 -- 172.21.15.90:0/1632264166 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822c00de80 con 0x7f8230101b80 2024-04-26T18:19:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82301a55e0 con 0x7f8230101b80 2024-04-26T18:19:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8226ffd640 1 -- 172.21.15.90:0/1632264166 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c00f620 con 0x7f8230101b80 2024-04-26T18:19:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.683+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82301a5af0 con 0x7f8230101b80 2024-04-26T18:19:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.685+0000 7f8226ffd640 1 -- 172.21.15.90:0/1632264166 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f822c009c30 con 0x7f8230101b80 2024-04-26T18:19:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 Processor -- start 2024-04-26T18:19:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 -- start start 2024-04-26T18:19:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb162e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb20ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb234d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f7dda7c460 con 0x55f7ddb16820 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f7dda36590 con 0x55f7ddb15f00 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f823ac34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f7dda78420 con 0x55f7ddb210e0 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f8235cbe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb20ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f8226ffd640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f821c0a3e40 0x7f821c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f8235cbe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb20ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34510/0 (socket says 172.21.15.90:34510) 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f8235cbe640 1 -- 172.21.15.90:0/3192818708 learned_addr learned my addr 172.21.15.90:0/3192818708 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f82354bd640 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb234d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.686+0000 7f82364bf640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f821c0a3e40 0x7f821c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f82364bf640 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f8226ffd640 1 -- 172.21.15.90:0/1632264166 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8497+0+0 (secure 0 0 0) 0x7f822c0d0390 con 0x7f8230101b80 2024-04-26T18:19:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f8235cbe640 1 -- 172.21.15.90:0/3192818708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 msgr2=0x55f7ddb234d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f8235cbe640 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb234d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f8235cbe640 1 -- 172.21.15.90:0/3192818708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 msgr2=0x55f7ddb162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f8235cbe640 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f8235cbe640 1 -- 172.21.15.90:0/3192818708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7dda430d0 con 0x55f7ddb16820 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f8235cbe640 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb20ba0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f822c0cda00 tx=0x7f822c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bb2a18f6a6bb56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f82364bf640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f821c0a3e40 0x7f821c0a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8228006010 tx=0x7f82280098a0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f821bfff640 1 -- 172.21.15.90:0/3192818708 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c020050 con 0x55f7ddb16820 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f821bfff640 1 -- 172.21.15.90:0/3192818708 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822c09c2e0 con 0x55f7ddb16820 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.687+0000 7f821bfff640 1 -- 172.21.15.90:0/3192818708 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c098230 con 0x55f7ddb16820 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 -- 172.21.15.90:0/3192818708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 msgr2=0x55f7ddb20ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb20ba0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f822c0cda00 tx=0x7f822c000bc0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 -- 172.21.15.90:0/3192818708 shutdown_connections 2024-04-26T18:19:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb234d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb20ba0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 --2- 172.21.15.90:0/3192818708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 -- 172.21.15.90:0/3192818708 >> 172.21.15.90:0/3192818708 conn(0x55f7ddb12410 msgr2=0x55f7ddb127f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 -- 172.21.15.90:0/3192818708 shutdown_connections 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 -- 172.21.15.90:0/3192818708 wait complete. 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 -- start start 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.688+0000 7f823ac34740 1 Processor -- start 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f823ac34740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb15f00 0x55f7ddb1e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f823ac34740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb1e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f823ac34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f823ac34740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddb23cb0 con 0x55f7ddb15f00 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f823ac34740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55f7dda03b60 con 0x55f7ddb16820 2024-04-26T18:19:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f823ac34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f7ddb23e30 con 0x55f7ddb210e0 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f823ac34740 10 client.0 _ll_register_callbacks cb 0x55f7dda282b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb15f00 0x55f7ddb1e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb15f00 0x55f7ddb1e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34516/0 (socket says 172.21.15.90:34516) 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 -- 172.21.15.90:0/2651261579 learned_addr learned my addr 172.21.15.90:0/2651261579 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f82354bd640 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb1e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f82364bf640 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 -- 172.21.15.90:0/2651261579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 msgr2=0x55f7ddb1eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 -- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 msgr2=0x55f7ddb1e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb1e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 -- 172.21.15.90:0/2651261579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7dd78e210 con 0x55f7ddb15f00 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f8235cbe640 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb15f00 0x55f7ddb1e120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f822c097fc0 tx=0x7f822c098600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dda154f79f29a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.689+0000 7f82197fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f82197fa640 1 -- 172.21.15.90:0/2651261579 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822c098950 con 0x55f7ddb15f00 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f82197fa640 1 -- 172.21.15.90:0/2651261579 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8497+0+0 (secure 0 0 0) 0x7f822c09b730 con 0x55f7ddb15f00 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f822c09c2e0 con 0x55f7ddb15f00 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f82197fa640 1 -- 172.21.15.90:0/2651261579 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f822c09b970 con 0x55f7ddb15f00 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 10 client.16077 fetch_fsmap learned FSMap version 36 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f7dd85a620 con 0x55f7ddb15f00 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f82197fa640 1 -- 172.21.15.90:0/2651261579 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822c09c2e0 con 0x55f7ddb15f00 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f82197fa640 1 -- 172.21.15.90:0/2651261579 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f822c09b1a0 con 0x55f7ddb15f00 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 10 client.16077 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 20 client.16077 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb31100 0x55f7ddb334f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 4 client.16077 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7dda8a070 con 0x55f7ddb31100 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.690+0000 7f82354bd640 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb31100 0x55f7ddb334f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.691+0000 7f82197fa640 10 client.16077 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.691+0000 7f82354bd640 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb31100 0x55f7ddb334f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b25bf3ac9c5ec8f server_cookie=a8ec3c0d3706bd38 in_seq=0 out_seq=0 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.691+0000 7f82197fa640 1 -- 172.21.15.90:0/2651261579 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7dda8a070 con 0x55f7ddb31100 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.691+0000 7f82197fa640 10 client.16077 handle_command_reply: tid=0 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 client.16077 shutdown 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb15f00 msgr2=0x55f7ddb1e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb15f00 0x55f7ddb1e120 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f822c097fc0 tx=0x7f822c098600 comp rx=0 tx=0).stop 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 shutdown_connections 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb31100 0x55f7ddb334f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb1e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 --2- 172.21.15.90:0/2651261579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb15f00 0x55f7ddb1e120 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f82197fa640 0 client.16077 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 >> 172.21.15.90:0/2651261579 conn(0x55f7ddb12410 msgr2=0x55f7ddb18190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 shutdown_connections 2024-04-26T18:19:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 1 -- 172.21.15.90:0/2651261579 wait complete. 2024-04-26T18:19:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.692+0000 7f823ac34740 20 client.16077 trim_cache size 0 max 16384 2024-04-26T18:19:01.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 Processor -- start 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 -- start start 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb15f00 0x55f7ddb20420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb29460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f82301b0c50 con 0x55f7ddb210e0 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ae520 con 0x55f7ddb16820 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f823ac34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ac370 con 0x55f7ddb15f00 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34532/0 (socket says 172.21.15.90:34532) 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 -- 172.21.15.90:0/2388854156 learned_addr learned my addr 172.21.15.90:0/2388854156 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 -- 172.21.15.90:0/2388854156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb15f00 msgr2=0x55f7ddb20420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb15f00 0x55f7ddb20420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82354bd640 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb29460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 -- 172.21.15.90:0/2388854156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 msgr2=0x55f7ddb29460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb29460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 -- 172.21.15.90:0/2388854156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7ddb1f080 con 0x55f7ddb210e0 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82354bd640 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb29460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f82364bf640 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1d8d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8228009910 tx=0x7f822803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350dddadee72d804 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f821b7fe640 1 -- 172.21.15.90:0/2388854156 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228031510 con 0x55f7ddb210e0 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f821b7fe640 1 -- 172.21.15.90:0/2388854156 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8228003e20 con 0x55f7ddb210e0 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.707+0000 7f821b7fe640 1 -- 172.21.15.90:0/2388854156 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280047f0 con 0x55f7ddb210e0 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 -- 172.21.15.90:0/2388854156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb210e0 msgr2=0x55f7ddb1d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1d8d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8228009910 tx=0x7f822803f040 comp rx=0 tx=0).stop 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 -- 172.21.15.90:0/2388854156 shutdown_connections 2024-04-26T18:19:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb210e0 0x55f7ddb1d8d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb16820 0x55f7ddb29460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 --2- 172.21.15.90:0/2388854156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb15f00 0x55f7ddb20420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 -- 172.21.15.90:0/2388854156 >> 172.21.15.90:0/2388854156 conn(0x55f7ddb12410 msgr2=0x55f7ddb1e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 -- 172.21.15.90:0/2388854156 shutdown_connections 2024-04-26T18:19:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 -- 172.21.15.90:0/2388854156 wait complete. 2024-04-26T18:19:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 -- start start 2024-04-26T18:19:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.708+0000 7f823ac34740 1 Processor -- start 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f823ac34740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb8f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f823ac34740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb8f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f823ac34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb9a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f823ac34740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ac9e0 con 0x55f7ddb16820 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f823ac34740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ade50 con 0x55f7ddb15f00 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f823ac34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82301ad0b0 con 0x55f7ddb210e0 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f823ac34740 10 client.0 _ll_register_callbacks cb 0x7f82200093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb8f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb8f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34544/0 (socket says 172.21.15.90:34544) 2024-04-26T18:19:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 -- 172.21.15.90:0/1380509134 learned_addr learned my addr 172.21.15.90:0/1380509134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f8235cbe640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb8f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82364bf640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb9a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 -- 172.21.15.90:0/1380509134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 msgr2=0x55f7ddb9a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb9a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 -- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 msgr2=0x55f7ddb8f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb8f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 -- 172.21.15.90:0/1380509134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7ddb90150 con 0x55f7ddb16820 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82364bf640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb9a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82354bd640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb8f860 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8220000c00 tx=0x7f822001fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93b585f2963d96c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.709+0000 7f82197fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f82197fa640 1 -- 172.21.15.90:0/1380509134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8220025060 con 0x55f7ddb16820 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f82197fa640 1 -- 172.21.15.90:0/1380509134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8497+0+0 (secure 0 0 0) 0x7f82200170d0 con 0x55f7ddb16820 2024-04-26T18:19:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8228003e20 con 0x55f7ddb16820 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f82197fa640 1 -- 172.21.15.90:0/1380509134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8220005bd0 con 0x55f7ddb16820 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 10 client.16083 fetch_fsmap learned FSMap version 36 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f7ddb9ba20 con 0x55f7ddb16820 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f82197fa640 1 -- 172.21.15.90:0/1380509134 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8228003e20 con 0x55f7ddb16820 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f82197fa640 1 -- 172.21.15.90:0/1380509134 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 36) ==== 8338+0+0 (secure 0 0 0) 0x7f8220006030 con 0x55f7ddb16820 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 10 client.16083 fetch_fsmap finished waiting for FSMap version 36 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 20 client.16083 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb9c180 0x55f7ddb9e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 4 client.16083 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:19:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7dda83ee0 con 0x55f7ddb9c180 2024-04-26T18:19:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.710+0000 7f8235cbe640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb9c180 0x55f7ddb9e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.711+0000 7f82197fa640 10 client.16083 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:01.711+0000 7f8235cbe640 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb9c180 0x55f7ddb9e5a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f53879e0cb34fd server_cookie=f864e948f08f9b07 in_seq=0 out_seq=0 2024-04-26T18:19:02.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.68 MB/sec execute 170 sec latency 21130.636 ms 2024-04-26T18:19:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:02 smithi119 ceph-mon[32752]: pgmap v218: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 26 op/s 2024-04-26T18:19:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:02 smithi167 ceph-mon[32751]: pgmap v218: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 26 op/s 2024-04-26T18:19:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:02 smithi090 ceph-mon[29180]: pgmap v218: 97 pgs: 97 active+clean; 96 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 26 KiB/s wr, 26 op/s 2024-04-26T18:19:03.595 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:19:03.597 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 systemd[1]: Stopped Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:19:03.597 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 4.640s CPU time. 2024-04-26T18:19:03.597 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 systemd[1]: Starting Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:19:03.597 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 podman[68013]: 2024-04-26 18:19:03.445339345 +0000 UTC m=+0.018236367 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:19:03.597 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 podman[68013]: 2024-04-26 18:19:03.596226219 +0000 UTC m=+0.169123231 container create ee12a2d50ea18570bba7642e28d5aa03d35f4b9590984da8726f8a96c4a531f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T18:19:03.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.66 MB/sec execute 171 sec latency 22130.755 ms 2024-04-26T18:19:03.857 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 podman[68013]: 2024-04-26 18:19:03.832371713 +0000 UTC m=+0.405268726 container init ee12a2d50ea18570bba7642e28d5aa03d35f4b9590984da8726f8a96c4a531f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T18:19:03.857 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 podman[68013]: 2024-04-26 18:19:03.834949404 +0000 UTC m=+0.407846417 container start ee12a2d50ea18570bba7642e28d5aa03d35f4b9590984da8726f8a96c4a531f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T18:19:04.123 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 ceph-mds[68032]: -- 172.21.15.119:0/721426985 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df980ada40 con 0x55df98080800 2024-04-26T18:19:04.123 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: starting mds.h at 2024-04-26T18:19:04.123 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 bash[68013]: ee12a2d50ea18570bba7642e28d5aa03d35f4b9590984da8726f8a96c4a531f5 2024-04-26T18:19:04.123 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:03 smithi119 systemd[1]: Started Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:19:04.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.63 MB/sec execute 172 sec latency 23130.848 ms 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: pgmap v219: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 835 KiB/s wr, 55 op/s 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: daemon mds.h restarted 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:boot 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: pgmap v219: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 835 KiB/s wr, 55 op/s 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: daemon mds.h restarted 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:boot 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:19:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: pgmap v219: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 835 KiB/s wr, 55 op/s 2024-04-26T18:19:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: daemon mds.h restarted 2024-04-26T18:19:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:19:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:boot 2024-04-26T18:19:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:19:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:19:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:05.676 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.60 MB/sec execute 173 sec latency 24130.945 ms 2024-04-26T18:19:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:05 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:resolve 2024-04-26T18:19:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:05 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:05 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:resolve 2024-04-26T18:19:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:05 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:05 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:resolve 2024-04-26T18:19:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:05 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:06.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 380964 4.58 MB/sec execute 174 sec latency 25131.100 ms 2024-04-26T18:19:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.901+0000 7f82197fa640 1 -- 172.21.15.90:0/1380509134 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7dda83ee0 con 0x55f7ddb9c180 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f82197fa640 10 client.16083 handle_command_reply: tid=0 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 client.16083 shutdown 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 msgr2=0x55f7ddb8f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb8f860 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8220000c00 tx=0x7f822001fec0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 shutdown_connections 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55f7ddb9c180 0x55f7ddb9e5a0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f7ddb210e0 0x55f7ddb9a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55f7ddb16820 0x55f7ddb8f860 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 --2- 172.21.15.90:0/1380509134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f7ddb15f00 0x55f7ddb8f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f82197fa640 0 client.16083 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 >> 172.21.15.90:0/1380509134 conn(0x55f7ddb12410 msgr2=0x55f7ddb28450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 shutdown_connections 2024-04-26T18:19:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 1 -- 172.21.15.90:0/1380509134 wait complete. 2024-04-26T18:19:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.902+0000 7f823ac34740 20 client.16083 trim_cache size 0 max 16384 2024-04-26T18:19:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:19:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:19:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:19:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:19:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f821c0a3e40 msgr2=0x7f821c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f821c0a3e40 0x7f821c0a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8228006010 tx=0x7f82280098a0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230101b80 msgr2=0x7f8230199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230101b80 0x7f8230199e50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f822c00d950 tx=0x7f822c00dd50 comp rx=0 tx=0).stop 2024-04-26T18:19:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 shutdown_connections 2024-04-26T18:19:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f821c0a3e40 0x7f821c0a6300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82301067e0 0x7f82301a4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8230102540 0x7f823019a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 --2- 172.21.15.90:0/1632264166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8230101b80 0x7f8230199e50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 >> 172.21.15.90:0/1632264166 conn(0x7f8230077630 msgr2=0x7f82300ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 shutdown_connections 2024-04-26T18:19:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:06.903+0000 7f8237f22640 1 -- 172.21.15.90:0/1632264166 wait complete. 2024-04-26T18:19:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:06 smithi119 ceph-mon[32752]: pgmap v221: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 971 KiB/s wr, 38 op/s 2024-04-26T18:19:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:06 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:19:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:reconnect 2024-04-26T18:19:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:06 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:06 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:19:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:06 smithi167 ceph-mon[32751]: pgmap v221: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 971 KiB/s wr, 38 op/s 2024-04-26T18:19:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:06 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:19:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:reconnect 2024-04-26T18:19:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:06 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:06 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:19:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:06 smithi090 ceph-mon[29180]: pgmap v221: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 971 KiB/s wr, 38 op/s 2024-04-26T18:19:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:06 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:19:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:reconnect 2024-04-26T18:19:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:06 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:06 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:19:07.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 381212 4.55 MB/sec execute 175 sec latency 25962.150 ms 2024-04-26T18:19:08.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:rejoin 2024-04-26T18:19:08.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:07 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:08.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:07 smithi119 ceph-mon[32752]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:19:08.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:07 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:19:08.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:07 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:19:08.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:active 2024-04-26T18:19:08.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:19:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:rejoin 2024-04-26T18:19:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:07 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:07 smithi167 ceph-mon[32751]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:19:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:07 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:19:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:07 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:19:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:active 2024-04-26T18:19:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:19:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:rejoin 2024-04-26T18:19:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:07 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:07 smithi090 ceph-mon[29180]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:19:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:07 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:19:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:07 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:19:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/874199594,v1:172.21.15.167:6839/874199594] up:active 2024-04-26T18:19:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:19:08.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:19:08 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:19:08.203+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:19:08.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 383496 4.60 MB/sec execute 176 sec latency 11.157 ms 2024-04-26T18:19:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:08 smithi119 ceph-mon[32752]: pgmap v222: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 971 KiB/s wr, 38 op/s 2024-04-26T18:19:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:08 smithi167 ceph-mon[32751]: pgmap v222: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 971 KiB/s wr, 38 op/s 2024-04-26T18:19:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:08 smithi090 ceph-mon[29180]: pgmap v222: 97 pgs: 97 active+clean; 97 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 971 KiB/s wr, 38 op/s 2024-04-26T18:19:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:09.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 385619 4.62 MB/sec execute 177 sec latency 17.627 ms 2024-04-26T18:19:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:19:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:19:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:19:10.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 387079 4.66 MB/sec execute 178 sec latency 21.219 ms 2024-04-26T18:19:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:10 smithi119 ceph-mon[32752]: pgmap v223: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-26T18:19:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:10 smithi119 ceph-mon[32752]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:19:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:10 smithi167 ceph-mon[32751]: pgmap v223: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-26T18:19:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:10 smithi167 ceph-mon[32751]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:19:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:10 smithi090 ceph-mon[29180]: pgmap v223: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-04-26T18:19:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:10 smithi090 ceph-mon[29180]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:19:11.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 389444 4.70 MB/sec execute 179 sec latency 17.144 ms 2024-04-26T18:19:12.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.71 MB/sec execute 180 sec latency 113.794 ms 2024-04-26T18:19:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:12 smithi119 ceph-mon[32752]: pgmap v225: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 292 KiB/s wr, 71 op/s 2024-04-26T18:19:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:12 smithi167 ceph-mon[32751]: pgmap v225: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 292 KiB/s wr, 71 op/s 2024-04-26T18:19:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:12 smithi090 ceph-mon[29180]: pgmap v225: 97 pgs: 97 active+clean; 93 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 292 KiB/s wr, 71 op/s 2024-04-26T18:19:13.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.69 MB/sec execute 181 sec latency 1113.894 ms 2024-04-26T18:19:14.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.66 MB/sec execute 182 sec latency 2113.946 ms 2024-04-26T18:19:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:14 smithi119 ceph-mon[32752]: pgmap v226: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2024-04-26T18:19:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:14 smithi167 ceph-mon[32751]: pgmap v226: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2024-04-26T18:19:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:14 smithi090 ceph-mon[29180]: pgmap v226: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.1 MiB/s wr, 177 op/s 2024-04-26T18:19:15.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.64 MB/sec execute 183 sec latency 3113.993 ms 2024-04-26T18:19:16.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.61 MB/sec execute 184 sec latency 4114.047 ms 2024-04-26T18:19:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:16 smithi119 ceph-mon[32752]: pgmap v227: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-26T18:19:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:16 smithi167 ceph-mon[32751]: pgmap v227: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-26T18:19:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:16 smithi090 ceph-mon[29180]: pgmap v227: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-26T18:19:17.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.59 MB/sec execute 185 sec latency 5114.168 ms 2024-04-26T18:19:18.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.56 MB/sec execute 186 sec latency 6114.258 ms 2024-04-26T18:19:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:18 smithi119 ceph-mon[32752]: pgmap v228: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-26T18:19:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:18 smithi119 ceph-mon[32752]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:19:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:18 smithi167 ceph-mon[32751]: pgmap v228: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-26T18:19:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:18 smithi167 ceph-mon[32751]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:19:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:18 smithi090 ceph-mon[29180]: pgmap v228: 97 pgs: 97 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.0 MiB/s wr, 167 op/s 2024-04-26T18:19:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:18 smithi090 ceph-mon[29180]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:19:19.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.54 MB/sec execute 187 sec latency 7114.318 ms 2024-04-26T18:19:19.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 -- 172.21.15.90:0/3674450129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0107860 msgr2=0x7feba0107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 --2- 172.21.15.90:0/3674450129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0107860 0x7feba0107c40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feb90009930 tx=0x7feb9002a420 comp rx=0 tx=0).stop 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 -- 172.21.15.90:0/3674450129 shutdown_connections 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 --2- 172.21.15.90:0/3674450129 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0102210 0x7feba010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 --2- 172.21.15.90:0/3674450129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba0101850 0x7feba0101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 --2- 172.21.15.90:0/3674450129 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0107860 0x7feba0107c40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 -- 172.21.15.90:0/3674450129 >> 172.21.15.90:0/3674450129 conn(0x7feba0077bb0 msgr2=0x7feba0077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 -- 172.21.15.90:0/3674450129 shutdown_connections 2024-04-26T18:19:20.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:19.999+0000 7feba50c4640 1 -- 172.21.15.90:0/3674450129 wait complete. 2024-04-26T18:19:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 Processor -- start 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 -- start start 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0101850 0x7feba0199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0102210 0x7feba019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba0107860 0x7feba01a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feba0111b20 con 0x7feba0102210 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feba01119a0 con 0x7feba0107860 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.000+0000 7feba50c4640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feba0111ca0 con 0x7feba0101850 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0102210 0x7feba019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0102210 0x7feba019a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37600/0 (socket says 172.21.15.90:37600) 2024-04-26T18:19:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 -- 172.21.15.90:0/2793807216 learned_addr learned my addr 172.21.15.90:0/2793807216 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0101850 0x7feba0199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9f59e640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba0107860 0x7feba01a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 -- 172.21.15.90:0/2793807216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0101850 msgr2=0x7feba0199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0101850 0x7feba0199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 -- 172.21.15.90:0/2793807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba0107860 msgr2=0x7feba01a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba0107860 0x7feba01a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 -- 172.21.15.90:0/2793807216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba01a54c0 con 0x7feba0102210 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9f59e640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba0107860 0x7feba01a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0101850 0x7feba0199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb9e59c640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0102210 0x7feba019a440 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feb9400d950 tx=0x7feb9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb87fff640 1 -- 172.21.15.90:0/2793807216 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94017070 con 0x7feba0102210 2024-04-26T18:19:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb87fff640 1 -- 172.21.15.90:0/2793807216 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb94009c30 con 0x7feba0102210 2024-04-26T18:19:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba01a5690 con 0x7feba0102210 2024-04-26T18:19:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feb87fff640 1 -- 172.21.15.90:0/2793807216 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9400f620 con 0x7feba0102210 2024-04-26T18:19:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.001+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba01a5be0 con 0x7feba0102210 2024-04-26T18:19:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feb87fff640 1 -- 172.21.15.90:0/2793807216 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7feb94010460 con 0x7feba0102210 2024-04-26T18:19:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 Processor -- start 2024-04-26T18:19:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 -- start start 2024-04-26T18:19:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7faba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a57a756460 con 0x55a57a7eff00 2024-04-26T18:19:20.014 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a57a710590 con 0x55a57a7fb0e0 2024-04-26T18:19:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.004+0000 7feba7ddc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a57a752420 con 0x55a57a7f0820 2024-04-26T18:19:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb87fff640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7feb880a3dc0 0x7feb880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37614/0 (socket says 172.21.15.90:37614) 2024-04-26T18:19:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 -- 172.21.15.90:0/1260844777 learned_addr learned my addr 172.21.15.90:0/1260844777 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:20.015 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7feb880a3dc0 0x7feb880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb87fff640 1 -- 172.21.15.90:0/2793807216 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8627+0+0 (secure 0 0 0) 0x7feb940d05c0 con 0x7feba0102210 2024-04-26T18:19:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9e59c640 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 -- 172.21.15.90:0/1260844777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 msgr2=0x55a57a7faba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7faba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 -- 172.21.15.90:0/1260844777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 msgr2=0x55a57a7fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 -- 172.21.15.90:0/1260844777 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57a71d0d0 con 0x55a57a7eff00 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9e59c640 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f02e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feb98012ce0 tx=0x7feb9800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af18bcbb45bd417f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb9f59e640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7feb880a3dc0 0x7feb880a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb9800fc30 tx=0x7feb9801a000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb857fa640 1 -- 172.21.15.90:0/1260844777 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98020070 con 0x55a57a7eff00 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb857fa640 1 -- 172.21.15.90:0/1260844777 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb98008b20 con 0x55a57a7eff00 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.005+0000 7feb857fa640 1 -- 172.21.15.90:0/1260844777 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800d3d0 con 0x55a57a7eff00 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 -- 172.21.15.90:0/1260844777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 msgr2=0x55a57a7f02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f02e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feb98012ce0 tx=0x7feb9800f730 comp rx=0 tx=0).stop 2024-04-26T18:19:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 -- 172.21.15.90:0/1260844777 shutdown_connections 2024-04-26T18:19:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7faba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 --2- 172.21.15.90:0/1260844777 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f02e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 -- 172.21.15.90:0/1260844777 >> 172.21.15.90:0/1260844777 conn(0x55a57a7ec410 msgr2=0x55a57a7ec7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 -- 172.21.15.90:0/1260844777 shutdown_connections 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 -- 172.21.15.90:0/1260844777 wait complete. 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.006+0000 7feba7ddc740 1 -- start start 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 1 Processor -- start 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7eff00 0x55a57a7f8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7f0820 0x55a57a7f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55a57a7fdcb0 con 0x55a57a7fb0e0 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55a57a6ddb60 con 0x55a57a7f0820 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a57a7fde30 con 0x55a57a7eff00 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feba7ddc740 10 client.0 _ll_register_callbacks cb 0x55a57a7022b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37618/0 (socket says 172.21.15.90:37618) 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 -- 172.21.15.90:0/922082000 learned_addr learned my addr 172.21.15.90:0/922082000 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7eff00 0x55a57a7f8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9e59c640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7f0820 0x55a57a7f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 -- 172.21.15.90:0/922082000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7eff00 msgr2=0x55a57a7f8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7eff00 0x55a57a7f8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 -- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7f0820 msgr2=0x55a57a7f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7f0820 0x55a57a7f86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 -- 172.21.15.90:0/922082000 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57a468210 con 0x55a57a7fb0e0 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9e59c640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7f0820 0x55a57a7f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb9f59e640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f8bf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feb9801cb80 tx=0x7feb9801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc597037b64aed2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.007+0000 7feb82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:19:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feb82ff5640 1 -- 172.21.15.90:0/922082000 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9802b070 con 0x55a57a7fb0e0 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feb82ff5640 1 -- 172.21.15.90:0/922082000 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8627+0+0 (secure 0 0 0) 0x7feb9801c430 con 0x55a57a7fb0e0 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a57a764070 con 0x55a57a7fb0e0 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feb82ff5640 1 -- 172.21.15.90:0/922082000 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7feb98020040 con 0x55a57a7fb0e0 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 10 client.16098 fetch_fsmap learned FSMap version 42 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a57a534620 con 0x55a57a7fb0e0 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feb82ff5640 1 -- 172.21.15.90:0/922082000 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55a57a764070 con 0x55a57a7fb0e0 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feb82ff5640 1 -- 172.21.15.90:0/922082000 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7feb9801b070 con 0x55a57a7fb0e0 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 10 client.16098 fetch_fsmap finished waiting for FSMap version 42 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 10 client.16098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 20 client.16098 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a80b240 0x55a57a80d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 4 client.16098 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:19:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a57a75dee0 con 0x55a57a80b240 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.008+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a80b240 0x55a57a80d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.009+0000 7feb82ff5640 10 client.16098 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.009+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a80b240 0x55a57a80d630 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e672365b0eddebc server_cookie=1bd7ea87896d7ff in_seq=0 out_seq=0 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feb82ff5640 1 -- 172.21.15.90:0/922082000 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a57a75dee0 con 0x55a57a80b240 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feb82ff5640 10 client.16098 handle_command_reply: tid=0 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 client.16098 shutdown 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 msgr2=0x55a57a7f8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f8bf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feb9801cb80 tx=0x7feb9801ce50 comp rx=0 tx=0).stop 2024-04-26T18:19:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 shutdown_connections 2024-04-26T18:19:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a80b240 0x55a57a80d630 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f8bf0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feb82ff5640 0 client.16098 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7f0820 0x55a57a7f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 --2- 172.21.15.90:0/922082000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7eff00 0x55a57a7f8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 >> 172.21.15.90:0/922082000 conn(0x55a57a7ec410 msgr2=0x55a57a7f2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 shutdown_connections 2024-04-26T18:19:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 1 -- 172.21.15.90:0/922082000 wait complete. 2024-04-26T18:19:20.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.010+0000 7feba7ddc740 20 client.16098 trim_cache size 0 max 16384 2024-04-26T18:19:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 Processor -- start 2024-04-26T18:19:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 -- start start 2024-04-26T18:19:20.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7eff00 0x55a57a7f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a812c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a8131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feba01b0ea0 con 0x55a57a7fb0e0 2024-04-26T18:19:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feba01ae770 con 0x55a57a7eff00 2024-04-26T18:19:20.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feba7ddc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feba01ac5e0 con 0x55a57a7f0820 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a8131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a8131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37634/0 (socket says 172.21.15.90:37634) 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 -- 172.21.15.90:0/309640110 learned_addr learned my addr 172.21.15.90:0/309640110 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9e59c640 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a812c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7eff00 0x55a57a7f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 -- 172.21.15.90:0/309640110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 msgr2=0x55a57a812c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a812c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 -- 172.21.15.90:0/309640110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7eff00 msgr2=0x55a57a7f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7eff00 0x55a57a7f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 -- 172.21.15.90:0/309640110 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a57a7fd840 con 0x55a57a7fb0e0 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb9f59e640 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a8131a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7feb98010010 tx=0x7feb9800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2262316e2fe621c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.025+0000 7feb84ff9640 1 -- 172.21.15.90:0/309640110 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9802b070 con 0x55a57a7fb0e0 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feb84ff9640 1 -- 172.21.15.90:0/309640110 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb9801c8e0 con 0x55a57a7fb0e0 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feb84ff9640 1 -- 172.21.15.90:0/309640110 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98037800 con 0x55a57a7fb0e0 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 -- 172.21.15.90:0/309640110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 msgr2=0x55a57a8131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a8131a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7feb98010010 tx=0x7feb9800f730 comp rx=0 tx=0).stop 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 -- 172.21.15.90:0/309640110 shutdown_connections 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7fb0e0 0x55a57a8131a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a812c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 --2- 172.21.15.90:0/309640110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7eff00 0x55a57a7f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 -- 172.21.15.90:0/309640110 >> 172.21.15.90:0/309640110 conn(0x55a57a7ec410 msgr2=0x55a57a7f2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 -- 172.21.15.90:0/309640110 shutdown_connections 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 -- 172.21.15.90:0/309640110 wait complete. 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 -- start start 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.026+0000 7feba7ddc740 1 Processor -- start 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7feba01acc30 con 0x55a57a7eff00 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7feba01ae0a0 con 0x55a57a7fb0e0 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feba01ad300 con 0x55a57a7f0820 2024-04-26T18:19:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 10 client.0 _ll_register_callbacks cb 0x7feb90009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37642/0 (socket says 172.21.15.90:37642) 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 -- 172.21.15.90:0/2701934608 learned_addr learned my addr 172.21.15.90:0/2701934608 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9e59c640 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9f59e640 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 -- 172.21.15.90:0/2701934608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 msgr2=0x55a57a7f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 -- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 msgr2=0x55a57a7f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 -- 172.21.15.90:0/2701934608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a57a878d30 con 0x55a57a7eff00 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb9ed9d640 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f4e80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feb90005940 tx=0x7feb900092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a948fcf3488c3213 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb82ff5640 1 -- 172.21.15.90:0/2701934608 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9004a060 con 0x55a57a7eff00 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feb82ff5640 1 -- 172.21.15.90:0/2701934608 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8627+0+0 (secure 0 0 0) 0x7feb90045bc0 con 0x55a57a7eff00 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.027+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb9801c8e0 con 0x55a57a7eff00 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feb82ff5640 1 -- 172.21.15.90:0/2701934608 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7feb90035040 con 0x55a57a7eff00 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 10 client.16104 fetch_fsmap learned FSMap version 42 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55a57a87a040 con 0x55a57a7eff00 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feb82ff5640 1 -- 172.21.15.90:0/2701934608 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb9801c8e0 con 0x55a57a7eff00 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feb82ff5640 1 -- 172.21.15.90:0/2701934608 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7feb900046c0 con 0x55a57a7eff00 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 10 client.16104 fetch_fsmap finished waiting for FSMap version 42 2024-04-26T18:19:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 20 client.16104 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a87a770 0x55a57a87cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 4 client.16104 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:19:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a57a6d95f0 con 0x55a57a87a770 2024-04-26T18:19:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.028+0000 7feb9e59c640 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a87a770 0x55a57a87cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.029+0000 7feb82ff5640 10 client.16104 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:20.038 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.029+0000 7feb9e59c640 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a87a770 0x55a57a87cb90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d721e4a787d13c14 server_cookie=98f29d5edcab82e4 in_seq=0 out_seq=0 2024-04-26T18:19:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feb82ff5640 1 -- 172.21.15.90:0/2701934608 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a57a6d95f0 con 0x55a57a87a770 2024-04-26T18:19:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feb82ff5640 10 client.16104 handle_command_reply: tid=0 2024-04-26T18:19:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 client.16104 shutdown 2024-04-26T18:19:20.075 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 msgr2=0x55a57a7f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f4e80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feb90005940 tx=0x7feb900092f0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 shutdown_connections 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55a57a87a770 0x55a57a87cb90 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a57a7fb0e0 0x55a57a7f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a57a7f0820 0x55a57a7f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feb82ff5640 0 client.16104 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 --2- 172.21.15.90:0/2701934608 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55a57a7eff00 0x55a57a7f4e80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.074+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 >> 172.21.15.90:0/2701934608 conn(0x55a57a7ec410 msgr2=0x55a57a7fa230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 shutdown_connections 2024-04-26T18:19:20.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba7ddc740 1 -- 172.21.15.90:0/2701934608 wait complete. 2024-04-26T18:19:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba7ddc740 20 client.16104 trim_cache size 0 max 16384 2024-04-26T18:19:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:19:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:19:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:19:20.077 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:19:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7feb880a3dc0 msgr2=0x7feb880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7feb880a3dc0 0x7feb880a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feb9800fc30 tx=0x7feb9801a000 comp rx=0 tx=0).stop 2024-04-26T18:19:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0102210 msgr2=0x7feba019a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0102210 0x7feba019a440 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7feb9400d950 tx=0x7feb9400de10 comp rx=0 tx=0).stop 2024-04-26T18:19:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 shutdown_connections 2024-04-26T18:19:20.079 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7feb880a3dc0 0x7feb880a6280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba0107860 0x7feba01a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feba0102210 0x7feba019a440 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 --2- 172.21.15.90:0/2793807216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feba0101850 0x7feba0199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.075+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 >> 172.21.15.90:0/2793807216 conn(0x7feba0077bb0 msgr2=0x7feba00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.076+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 shutdown_connections 2024-04-26T18:19:20.080 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:20.076+0000 7feba50c4640 1 -- 172.21.15.90:0/2793807216 wait complete. 2024-04-26T18:19:20.677 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.51 MB/sec execute 188 sec latency 8114.408 ms 2024-04-26T18:19:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:20 smithi119 ceph-mon[32752]: pgmap v230: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 890 KiB/s wr, 126 op/s 2024-04-26T18:19:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:20 smithi167 ceph-mon[32751]: pgmap v230: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 890 KiB/s wr, 126 op/s 2024-04-26T18:19:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:20 smithi090 ceph-mon[29180]: pgmap v230: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 890 KiB/s wr, 126 op/s 2024-04-26T18:19:21.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.49 MB/sec execute 189 sec latency 9114.505 ms 2024-04-26T18:19:22.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.47 MB/sec execute 190 sec latency 10114.564 ms 2024-04-26T18:19:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:22 smithi119 ceph-mon[32752]: pgmap v231: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 827 KiB/s wr, 117 op/s 2024-04-26T18:19:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:22 smithi167 ceph-mon[32751]: pgmap v231: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 827 KiB/s wr, 117 op/s 2024-04-26T18:19:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:22 smithi090 ceph-mon[29180]: pgmap v231: 97 pgs: 97 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 827 KiB/s wr, 117 op/s 2024-04-26T18:19:23.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.44 MB/sec execute 191 sec latency 11114.671 ms 2024-04-26T18:19:24.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.42 MB/sec execute 192 sec latency 12114.720 ms 2024-04-26T18:19:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:24 smithi119 ceph-mon[32752]: pgmap v232: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:24 smithi167 ceph-mon[32751]: pgmap v232: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:24 smithi090 ceph-mon[29180]: pgmap v232: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:25.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.40 MB/sec execute 193 sec latency 13114.827 ms 2024-04-26T18:19:26.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.37 MB/sec execute 194 sec latency 14114.887 ms 2024-04-26T18:19:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:26 smithi119 ceph-mon[32752]: pgmap v233: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:26 smithi167 ceph-mon[32751]: pgmap v233: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:26 smithi090 ceph-mon[29180]: pgmap v233: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:27.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.35 MB/sec execute 195 sec latency 15115.011 ms 2024-04-26T18:19:27.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb365b24700 time 2024-04-26T18:19:27.392034+0000 2024-04-26T18:19:27.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:27.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb37014d3dd] 2024-04-26T18:19:27.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: clone() 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.392+0000 7fb365b24700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb365b24700 time 2024-04-26T18:19:27.392034+0000 2024-04-26T18:19:27.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:27.820 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb37014d3dd] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: clone() 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: *** Caught signal (Aborted) ** 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: in thread 7fb365b24700 thread_name:ms_dispatch 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb36eed5d20] 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: gsignal() 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: abort() 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb37014d437] 2024-04-26T18:19:27.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 16: clone() 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.394+0000 7fb365b24700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: in thread 7fb365b24700 thread_name:ms_dispatch 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb36eed5d20] 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: gsignal() 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: abort() 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb37014d437] 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 16: clone() 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: -2> 2024-04-26T18:19:27.392+0000 7fb365b24700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb365b24700 time 2024-04-26T18:19:27.392034+0000 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb37014d3dd] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: clone() 2024-04-26T18:19:27.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 0> 2024-04-26T18:19:27.394+0000 7fb365b24700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: in thread 7fb365b24700 thread_name:ms_dispatch 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb36eed5d20] 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: gsignal() 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: abort() 2024-04-26T18:19:27.827 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb37014d437] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.828 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 16: clone() 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: -2> 2024-04-26T18:19:27.392+0000 7fb365b24700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb365b24700 time 2024-04-26T18:19:27.392034+0000 2024-04-26T18:19:27.829 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb37014d3dd] 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: clone() 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 0> 2024-04-26T18:19:27.394+0000 7fb365b24700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: in thread 7fb365b24700 thread_name:ms_dispatch 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:27.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb36eed5d20] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2: gsignal() 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 3: abort() 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb37014d437] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb37014d5a3] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55fa49290822] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55fa49200598] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55fa49202c17] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55fa490fbd0b] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fa490fc2ac] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55fa490e4fad] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb3703e1e20] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 13: (DispatchQueue::entry()+0x51f) [0x7fb3703defdf] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb3704b1fc1] 2024-04-26T18:19:27.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb36eecb1ca] 2024-04-26T18:19:27.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 16: clone() 2024-04-26T18:19:27.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:19:27.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68225]: 2024-04-26T18:19:27.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:27 smithi167 podman[68460]: 2024-04-26 18:19:27.456934293 +0000 UTC m=+0.021552816 container died a74b35e0ba29feede6f1987a8eea37f858872218d926ade9bb061944f93a1c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-26T18:19:28.410 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:28 smithi167 podman[68460]: 2024-04-26 18:19:28.154927555 +0000 UTC m=+0.719546077 container remove a74b35e0ba29feede6f1987a8eea37f858872218d926ade9bb061944f93a1c1d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:19:28.410 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:28 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:19:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:28 smithi090 ceph-mon[29180]: pgmap v234: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:28.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.33 MB/sec execute 196 sec latency 16115.111 ms 2024-04-26T18:19:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:28 smithi119 ceph-mon[32752]: pgmap v234: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:28 smithi167 ceph-mon[32751]: pgmap v234: 97 pgs: 97 active+clean; 105 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 1019 KiB/s wr, 51 op/s 2024-04-26T18:19:28.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:28 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-04-26T18:19:28.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:28 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.570s CPU time. 2024-04-26T18:19:29.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.31 MB/sec execute 197 sec latency 17115.146 ms 2024-04-26T18:19:30.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.29 MB/sec execute 198 sec latency 18115.194 ms 2024-04-26T18:19:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:30 smithi119 ceph-mon[32752]: pgmap v235: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 894 KiB/s wr, 56 op/s 2024-04-26T18:19:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:30 smithi167 ceph-mon[32751]: pgmap v235: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 894 KiB/s wr, 56 op/s 2024-04-26T18:19:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:30 smithi090 ceph-mon[29180]: pgmap v235: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 894 KiB/s wr, 56 op/s 2024-04-26T18:19:31.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.26 MB/sec execute 199 sec latency 19115.253 ms 2024-04-26T18:19:32.678 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.24 MB/sec execute 200 sec latency 20115.352 ms 2024-04-26T18:19:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:32 smithi119 ceph-mon[32752]: pgmap v236: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 866 KiB/s wr, 49 op/s 2024-04-26T18:19:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:32 smithi167 ceph-mon[32751]: pgmap v236: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 866 KiB/s wr, 49 op/s 2024-04-26T18:19:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:32 smithi090 ceph-mon[29180]: pgmap v236: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 866 KiB/s wr, 49 op/s 2024-04-26T18:19:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 -- 172.21.15.90:0/2887699740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 msgr2=0x7fa0c410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/2887699740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c410e6e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009cd0 tx=0x7fa0bc02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 -- 172.21.15.90:0/2887699740 shutdown_connections 2024-04-26T18:19:33.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/2887699740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c410e6e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/2887699740 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c4101860 0x7fa0c4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/2887699740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c4107820 0x7fa0c4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 -- 172.21.15.90:0/2887699740 >> 172.21.15.90:0/2887699740 conn(0x7fa0c40789b0 msgr2=0x7fa0c4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 -- 172.21.15.90:0/2887699740 shutdown_connections 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 -- 172.21.15.90:0/2887699740 wait complete. 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.170+0000 7fa0cbf14640 1 Processor -- start 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0cbf14640 1 -- start start 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0cbf14640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c4101860 0x7fa0c40fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0cbf14640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c40fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0cbf14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c4107820 0x7fa0c40ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0cbf14640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4111aa0 con 0x7fa0c4102220 2024-04-26T18:19:33.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0cbf14640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4111920 con 0x7fa0c4107820 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0cbf14640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c4111c20 con 0x7fa0c4101860 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c40fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c40fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42584/0 (socket says 172.21.15.90:42584) 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c9cb0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c4101860 0x7fa0c40fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 -- 172.21.15.90:0/334956825 learned_addr learned my addr 172.21.15.90:0/334956825 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0ca4b1640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c4107820 0x7fa0c40ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 -- 172.21.15.90:0/334956825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c4101860 msgr2=0x7fa0c40fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c4101860 0x7fa0c40fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 -- 172.21.15.90:0/334956825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c4107820 msgr2=0x7fa0c40ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c4107820 0x7fa0c40ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 -- 172.21.15.90:0/334956825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c41a74b0 con 0x7fa0c4102220 2024-04-26T18:19:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.171+0000 7fa0c94af640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c40fed30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000d950 tx=0x7fa0c000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.172+0000 7fa0b6ffd640 1 -- 172.21.15.90:0/334956825 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0017070 con 0x7fa0c4102220 2024-04-26T18:19:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.172+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c41a77a0 con 0x7fa0c4102220 2024-04-26T18:19:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.172+0000 7fa0b6ffd640 1 -- 172.21.15.90:0/334956825 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c000de80 con 0x7fa0c4102220 2024-04-26T18:19:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.172+0000 7fa0b6ffd640 1 -- 172.21.15.90:0/334956825 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000f620 con 0x7fa0c4102220 2024-04-26T18:19:33.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.172+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c41a7cf0 con 0x7fa0c4102220 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0b6ffd640 1 -- 172.21.15.90:0/334956825 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fa0c0009c30 con 0x7fa0c4102220 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 Processor -- start 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 -- start start 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f782e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f82ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555754ede460 con 0x555754f78820 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555754e98590 con 0x555754f830e0 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.174+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555754eda420 con 0x555754f77f00 2024-04-26T18:19:33.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f82ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f82ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42594/0 (socket says 172.21.15.90:42594) 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 -- 172.21.15.90:0/404295834 learned_addr learned my addr 172.21.15.90:0/404295834 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0b6ffd640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa0b00a3dc0 0x7fa0b00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c94af640 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0ca4b1640 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f782e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0ca4b1640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa0b00a3dc0 0x7fa0b00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 -- 172.21.15.90:0/404295834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 msgr2=0x555754f782e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f782e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 -- 172.21.15.90:0/404295834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 msgr2=0x555754f854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 -- 172.21.15.90:0/404295834 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555754ea50d0 con 0x555754f78820 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0b6ffd640 1 -- 172.21.15.90:0/334956825 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8627+0+0 (secure 0 0 0) 0x7fa0c00d04a0 con 0x7fa0c4102220 2024-04-26T18:19:33.183 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f82ba0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800ac90 tx=0x7fa0b800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b72622bd77ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0ca4b1640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa0b00a3dc0 0x7fa0b00a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc005fe0 tx=0x7fa0bc005da0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0affff640 1 -- 172.21.15.90:0/404295834 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b801d070 con 0x555754f78820 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0affff640 1 -- 172.21.15.90:0/404295834 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b8018af0 con 0x555754f78820 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.175+0000 7fa0affff640 1 -- 172.21.15.90:0/404295834 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8010ac0 con 0x555754f78820 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 -- 172.21.15.90:0/404295834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 msgr2=0x555754f82ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f82ba0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800ac90 tx=0x7fa0b800f790 comp rx=0 tx=0).stop 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 -- 172.21.15.90:0/404295834 shutdown_connections 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f82ba0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/404295834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f782e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 -- 172.21.15.90:0/404295834 >> 172.21.15.90:0/404295834 conn(0x555754f74410 msgr2=0x555754f747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 -- 172.21.15.90:0/404295834 shutdown_connections 2024-04-26T18:19:33.184 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.176+0000 7fa0cec2d740 1 -- 172.21.15.90:0/404295834 wait complete. 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 -- start start 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 Processor -- start 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f80040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f78820 0x555754f80580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f830e0 0x555754f80ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555754f85d10 con 0x555754f830e0 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555754e65b60 con 0x555754f78820 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555754f85e90 con 0x555754f77f00 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0cec2d740 10 client.0 _ll_register_callbacks cb 0x555754e8a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0ca4b1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f830e0 0x555754f80ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0c94af640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f78820 0x555754f80580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0ca4b1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f830e0 0x555754f80ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42604/0 (socket says 172.21.15.90:42604) 2024-04-26T18:19:33.185 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0ca4b1640 1 -- 172.21.15.90:0/3007484332 learned_addr learned my addr 172.21.15.90:0/3007484332 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.177+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f80040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0c94af640 1 -- 172.21.15.90:0/3007484332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 msgr2=0x555754f80040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f80040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0c94af640 1 -- 172.21.15.90:0/3007484332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f830e0 msgr2=0x555754f80ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f830e0 0x555754f80ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0c94af640 1 -- 172.21.15.90:0/3007484332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555754bf0210 con 0x555754f78820 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0ca4b1640 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f830e0 0x555754f80ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f78820 0x555754f80580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00cdb20 tx=0x7fa0c009bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce7eff5cad8077a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3007484332 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0020040 con 0x555754f78820 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0b8018af0 con 0x555754f78820 2024-04-26T18:19:33.186 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3007484332 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8627+0+0 (secure 0 0 0) 0x7fa0c00cdda0 con 0x555754f78820 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3007484332 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa0c00cdfa0 con 0x555754f78820 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0cec2d740 10 client.25580 fetch_fsmap learned FSMap version 42 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555754cbc620 con 0x555754f78820 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.178+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3007484332 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b8018af0 con 0x555754f78820 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3007484332 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fa0c00982c0 con 0x555754f78820 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0cec2d740 10 client.25580 fetch_fsmap finished waiting for FSMap version 42 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0cec2d740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0cec2d740 20 client.25580 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754f931f0 0x555754f95600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0cec2d740 4 client.25580 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555754eec070 con 0x555754f931f0 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754f931f0 0x555754f95600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0ad7fa640 10 client.25580 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.179+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754f931f0 0x555754f95600 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a6fd456de0cee2 server_cookie=c6085eef870526f in_seq=0 out_seq=0 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3007484332 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555754eec070 con 0x555754f931f0 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0ad7fa640 10 client.25580 handle_command_reply: tid=0 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 client.25580 shutdown 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f78820 msgr2=0x555754f80580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f78820 0x555754f80580 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa0c00cdb20 tx=0x7fa0c009bd90 comp rx=0 tx=0).stop 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 shutdown_connections 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754f931f0 0x555754f95600 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f830e0 0x555754f80ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0ad7fa640 0 client.25580 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f78820 0x555754f80580 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3007484332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f80040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 >> 172.21.15.90:0/3007484332 conn(0x555754f74410 msgr2=0x555754f735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 shutdown_connections 2024-04-26T18:19:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3007484332 wait complete. 2024-04-26T18:19:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.180+0000 7fa0cec2d740 20 client.25580 trim_cache size 0 max 16384 2024-04-26T18:19:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.195+0000 7fa0cec2d740 1 Processor -- start 2024-04-26T18:19:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0cec2d740 1 -- start start 2024-04-26T18:19:33.200 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f9af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f9b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41b2fd0 con 0x555754f78820 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41b08a0 con 0x555754f830e0 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41ae710 con 0x555754f77f00 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f9b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f9b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42620/0 (socket says 172.21.15.90:42620) 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 -- 172.21.15.90:0/3029150925 learned_addr learned my addr 172.21.15.90:0/3029150925 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f9af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 -- 172.21.15.90:0/3029150925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 msgr2=0x555754f9af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f9af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 -- 172.21.15.90:0/3029150925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 msgr2=0x555754f798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 -- 172.21.15.90:0/3029150925 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555754f78650 con 0x555754f78820 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f9b450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0012040 tx=0x7fa0c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b983f9a42c28cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0af7fe640 1 -- 172.21.15.90:0/3029150925 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0066040 con 0x555754f78820 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0af7fe640 1 -- 172.21.15.90:0/3029150925 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c00661e0 con 0x555754f78820 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.196+0000 7fa0af7fe640 1 -- 172.21.15.90:0/3029150925 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c00980c0 con 0x555754f78820 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3029150925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 msgr2=0x555754f9b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f9b450 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa0c0012040 tx=0x7fa0c0005880 comp rx=0 tx=0).stop 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3029150925 shutdown_connections 2024-04-26T18:19:33.202 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f9b450 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3029150925 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f9af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3029150925 >> 172.21.15.90:0/3029150925 conn(0x555754f74410 msgr2=0x555754f747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3029150925 shutdown_connections 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3029150925 wait complete. 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 -- start start 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.197+0000 7fa0cec2d740 1 Processor -- start 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f86900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0cec2d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f86f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41aed60 con 0x555754f78820 2024-04-26T18:19:33.203 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41b01d0 con 0x555754f830e0 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0cec2d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c41af430 con 0x555754f77f00 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0cec2d740 10 client.0 _ll_register_callbacks cb 0x7fa0c009c1f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f86900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f86900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42632/0 (socket says 172.21.15.90:42632) 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 -- 172.21.15.90:0/3376537464 learned_addr learned my addr 172.21.15.90:0/3376537464 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 -- 172.21.15.90:0/3376537464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 msgr2=0x555754f8b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f8b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 -- 172.21.15.90:0/3376537464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 msgr2=0x555754f86f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f86f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 -- 172.21.15.90:0/3376537464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555754f87740 con 0x555754f78820 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0c94af640 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f86900 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa0c009c020 tx=0x7fa0c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5232e8a1cfc37fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:33.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.198+0000 7fa0ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3376537464 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0066040 con 0x555754f78820 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3376537464 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8627+0+0 (secure 0 0 0) 0x7fa0c00980c0 con 0x555754f78820 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0c00661e0 con 0x555754f78820 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3376537464 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fa0c00a4fe0 con 0x555754f78820 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 10 client.16122 fetch_fsmap learned FSMap version 42 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555754ff9890 con 0x555754f78820 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3376537464 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0c00661e0 con 0x555754f78820 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3376537464 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 42) ==== 8338+0+0 (secure 0 0 0) 0x7fa0c00a0120 con 0x555754f78820 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 10 client.16122 fetch_fsmap finished waiting for FSMap version 42 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 20 client.16122 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754ff9cd0 0x555754ffc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:33.205 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 4 client.16122 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:19:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555754ee5ee0 con 0x555754ff9cd0 2024-04-26T18:19:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.199+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754ff9cd0 0x555754ffc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.200+0000 7fa0c9cb0640 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754ff9cd0 0x555754ffc0f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7814d886e0d56f71 server_cookie=1031cdf7aacb824e in_seq=0 out_seq=0 2024-04-26T18:19:33.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:33.200+0000 7fa0ad7fa640 10 client.16122 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:33.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.22 MB/sec execute 201 sec latency 21115.453 ms 2024-04-26T18:19:34.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.20 MB/sec execute 202 sec latency 22115.510 ms 2024-04-26T18:19:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:34 smithi119 ceph-mon[32752]: pgmap v237: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 877 KiB/s wr, 56 op/s 2024-04-26T18:19:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:34 smithi167 ceph-mon[32751]: pgmap v237: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 877 KiB/s wr, 56 op/s 2024-04-26T18:19:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:34 smithi090 ceph-mon[29180]: pgmap v237: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 877 KiB/s wr, 56 op/s 2024-04-26T18:19:35.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.18 MB/sec execute 203 sec latency 23115.608 ms 2024-04-26T18:19:36.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.16 MB/sec execute 204 sec latency 24115.675 ms 2024-04-26T18:19:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:36 smithi119 ceph-mon[32752]: pgmap v238: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-26T18:19:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:36 smithi167 ceph-mon[32751]: pgmap v238: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-26T18:19:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:36 smithi090 ceph-mon[29180]: pgmap v238: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-26T18:19:37.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.14 MB/sec execute 205 sec latency 25115.738 ms 2024-04-26T18:19:38.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.12 MB/sec execute 206 sec latency 26115.829 ms 2024-04-26T18:19:38.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:38 smithi167 ceph-mon[32751]: pgmap v239: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-26T18:19:38.804 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:38 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:19:38.804 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:38 smithi167 systemd[1]: Stopped Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:19:38.804 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:38 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.570s CPU time. 2024-04-26T18:19:38.804 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:38 smithi167 systemd[1]: Starting Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:19:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:38 smithi119 ceph-mon[32752]: pgmap v239: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-26T18:19:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:38 smithi090 ceph-mon[29180]: pgmap v239: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 20 op/s 2024-04-26T18:19:39.067 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:38 smithi167 podman[68559]: 2024-04-26 18:19:38.706815373 +0000 UTC m=+0.018894989 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:19:39.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:38 smithi167 podman[68559]: 2024-04-26 18:19:38.839001687 +0000 UTC m=+0.151081294 container create 1e87c026f61106e99d641646350c01f47f8af3359eaec4318c91de9a64e0d557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:19:39.518 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:39 smithi167 podman[68559]: 2024-04-26 18:19:39.074887101 +0000 UTC m=+0.386966707 container init 1e87c026f61106e99d641646350c01f47f8af3359eaec4318c91de9a64e0d557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T18:19:39.518 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:39 smithi167 podman[68559]: 2024-04-26 18:19:39.077650649 +0000 UTC m=+0.389730260 container start 1e87c026f61106e99d641646350c01f47f8af3359eaec4318c91de9a64e0d557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T18:19:39.518 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mds[68578]: -- 172.21.15.167:0/551981525 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564952a6da40 con 0x564952a41000 2024-04-26T18:19:39.518 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: starting mds.i at 2024-04-26T18:19:39.518 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:39 smithi167 bash[68559]: 1e87c026f61106e99d641646350c01f47f8af3359eaec4318c91de9a64e0d557 2024-04-26T18:19:39.518 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:19:39 smithi167 systemd[1]: Started Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:19:39.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.10 MB/sec execute 207 sec latency 27115.931 ms 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: daemon mds.i restarted 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:boot 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:19:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:39 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: daemon mds.i restarted 2024-04-26T18:19:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:19:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:boot 2024-04-26T18:19:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:19:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:19:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:39 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: daemon mds.i restarted 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:boot 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:19:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:39 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:40.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.08 MB/sec execute 208 sec latency 28116.034 ms 2024-04-26T18:19:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:41 smithi090 ceph-mon[29180]: pgmap v241: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T18:19:42.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:41 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:resolve 2024-04-26T18:19:42.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:41 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:42.784 INFO:tasks.workunit.client.0.smithi090.stdout: 1 390815 4.06 MB/sec execute 209 sec latency 29116.149 ms 2024-04-26T18:19:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:41 smithi167 ceph-mon[32751]: pgmap v241: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T18:19:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:41 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:resolve 2024-04-26T18:19:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:41 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:42 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:19:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:42 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:19:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:42 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:reconnect 2024-04-26T18:19:42.784 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:42 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:41 smithi119 ceph-mon[32752]: pgmap v241: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T18:19:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:41 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:resolve 2024-04-26T18:19:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:41 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:42 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:19:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:42 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:19:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:42 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:reconnect 2024-04-26T18:19:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:42 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0ad7fa640 1 -- 172.21.15.90:0/3376537464 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555754ee5ee0 con 0x555754ff9cd0 2024-04-26T18:19:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0ad7fa640 10 client.16122 handle_command_reply: tid=0 2024-04-26T18:19:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 client.16122 shutdown 2024-04-26T18:19:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 msgr2=0x555754f86900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f86900 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa0c009c020 tx=0x7fa0c0000bc0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 shutdown_connections 2024-04-26T18:19:42.786 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555754ff9cd0 0x555754ffc0f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555754f830e0 0x555754f86f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555754f78820 0x555754f86900 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0ad7fa640 0 client.16122 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 --2- 172.21.15.90:0/3376537464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555754f77f00 0x555754f8b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 >> 172.21.15.90:0/3376537464 conn(0x555754f74410 msgr2=0x555754f80000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 shutdown_connections 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 1 -- 172.21.15.90:0/3376537464 wait complete. 2024-04-26T18:19:42.787 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.147+0000 7fa0cec2d740 20 client.16122 trim_cache size 0 max 16384 2024-04-26T18:19:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:19:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:19:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:19:42.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa0b00a3dc0 msgr2=0x7fa0b00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa0b00a3dc0 0x7fa0b00a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc005fe0 tx=0x7fa0bc005da0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 msgr2=0x7fa0c40fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c40fed30 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000d950 tx=0x7fa0c000dd50 comp rx=0 tx=0).stop 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 shutdown_connections 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa0b00a3dc0 0x7fa0b00a6280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0c4107820 0x7fa0c40ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0c4102220 0x7fa0c40fed30 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 --2- 172.21.15.90:0/334956825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa0c4101860 0x7fa0c40fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 >> 172.21.15.90:0/334956825 conn(0x7fa0c40789b0 msgr2=0x7fa0c406a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:42.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 shutdown_connections 2024-04-26T18:19:42.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:42.148+0000 7fa0cbf14640 1 -- 172.21.15.90:0/334956825 wait complete. 2024-04-26T18:19:42.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:42 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:19:42.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:42 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:19:42.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:42 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:reconnect 2024-04-26T18:19:42.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:42 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:42.790 INFO:tasks.workunit.client.0.smithi090.stdout: 1 391044 4.04 MB/sec execute 210 sec latency 29934.219 ms 2024-04-26T18:19:43.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:19:43.207+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:19:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: pgmap v242: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T18:19:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:rejoin 2024-04-26T18:19:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:19:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:19:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:19:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:active 2024-04-26T18:19:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:19:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: pgmap v242: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T18:19:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:rejoin 2024-04-26T18:19:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:19:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:19:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:19:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:active 2024-04-26T18:19:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: pgmap v242: 97 pgs: 97 active+clean; 96 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:rejoin 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/229859293,v1:172.21.15.119:6837/229859293] up:active 2024-04-26T18:19:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:19:43.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 393836 4.11 MB/sec execute 211 sec latency 17.924 ms 2024-04-26T18:19:44.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 395103 4.11 MB/sec execute 212 sec latency 18.170 ms 2024-04-26T18:19:45.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:45 smithi090 ceph-mon[29180]: pgmap v243: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:45 smithi119 ceph-mon[32752]: pgmap v243: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:45 smithi167 ceph-mon[32751]: pgmap v243: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:45.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 397633 4.16 MB/sec execute 213 sec latency 17.076 ms 2024-04-26T18:19:46.679 INFO:tasks.workunit.client.0.smithi090.stdout: 1 399656 4.18 MB/sec execute 214 sec latency 12.631 ms 2024-04-26T18:19:47.510 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-mon[32752]: pgmap v244: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:47.544 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:47 smithi090 ceph-mon[29180]: pgmap v244: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:47 smithi167 ceph-mon[32751]: pgmap v244: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:47.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.21 MB/sec execute 215 sec latency 167.801 ms 2024-04-26T18:19:47.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe42acf3700 time 2024-04-26T18:19:47.511371+0000 2024-04-26T18:19:47.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:47.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe43531c3dd] 2024-04-26T18:19:47.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: clone() 2024-04-26T18:19:47.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.512+0000 7fe42acf3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe42acf3700 time 2024-04-26T18:19:47.511371+0000 2024-04-26T18:19:47.806 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:47.806 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe43531c3dd] 2024-04-26T18:19:47.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: clone() 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: *** Caught signal (Aborted) ** 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: in thread 7fe42acf3700 thread_name:ms_dispatch 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4340a4d20] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: gsignal() 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: abort() 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe43531c437] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 16: clone() 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.514+0000 7fe42acf3700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: in thread 7fe42acf3700 thread_name:ms_dispatch 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4340a4d20] 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: gsignal() 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: abort() 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe43531c437] 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 16: clone() 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: -1> 2024-04-26T18:19:47.512+0000 7fe42acf3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe42acf3700 time 2024-04-26T18:19:47.511371+0000 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe43531c3dd] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: clone() 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 0> 2024-04-26T18:19:47.514+0000 7fe42acf3700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: in thread 7fe42acf3700 thread_name:ms_dispatch 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4340a4d20] 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: gsignal() 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: abort() 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe43531c437] 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 16: clone() 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: -1> 2024-04-26T18:19:47.512+0000 7fe42acf3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe42acf3700 time 2024-04-26T18:19:47.511371+0000 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe43531c3dd] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: clone() 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 0> 2024-04-26T18:19:47.514+0000 7fe42acf3700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: in thread 7fe42acf3700 thread_name:ms_dispatch 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe4340a4d20] 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2: gsignal() 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 3: abort() 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe43531c437] 2024-04-26T18:19:47.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe43531c5a3] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55df97095822] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55df97005598] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55df97007c17] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55df96f00d0b] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55df96f012ac] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55df96ee9fad] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe4355b0e20] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 13: (DispatchQueue::entry()+0x51f) [0x7fe4355adfdf] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe435680fc1] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe43409a1ca] 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 16: clone() 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:19:47.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68028]: 2024-04-26T18:19:47.818 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:47 smithi119 podman[68218]: 2024-04-26 18:19:47.570132879 +0000 UTC m=+0.019046634 container died ee12a2d50ea18570bba7642e28d5aa03d35f4b9590984da8726f8a96c4a531f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T18:19:48.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:48 smithi119 podman[68218]: 2024-04-26 18:19:48.168306223 +0000 UTC m=+0.617219961 container remove ee12a2d50ea18570bba7642e28d5aa03d35f4b9590984da8726f8a96c4a531f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:19:48.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:48 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:19:48.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:48 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-26T18:19:48.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:48 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.428s CPU time. 2024-04-26T18:19:48.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.19 MB/sec execute 216 sec latency 1167.855 ms 2024-04-26T18:19:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:49 smithi119 ceph-mon[32752]: pgmap v245: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:49 smithi167 ceph-mon[32751]: pgmap v245: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:49.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:49 smithi090 ceph-mon[29180]: pgmap v245: 97 pgs: 97 active+clean; 97 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 164 KiB/s wr, 42 op/s 2024-04-26T18:19:49.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.17 MB/sec execute 217 sec latency 2167.938 ms 2024-04-26T18:19:50.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.16 MB/sec execute 218 sec latency 3168.039 ms 2024-04-26T18:19:51.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:51 smithi090 ceph-mon[29180]: pgmap v246: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1000 KiB/s wr, 141 op/s 2024-04-26T18:19:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:51 smithi119 ceph-mon[32752]: pgmap v246: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1000 KiB/s wr, 141 op/s 2024-04-26T18:19:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:51 smithi167 ceph-mon[32751]: pgmap v246: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1000 KiB/s wr, 141 op/s 2024-04-26T18:19:51.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.14 MB/sec execute 219 sec latency 4168.124 ms 2024-04-26T18:19:52.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.12 MB/sec execute 220 sec latency 5168.192 ms 2024-04-26T18:19:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:53 smithi119 ceph-mon[32752]: pgmap v247: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 848 KiB/s wr, 120 op/s 2024-04-26T18:19:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:53 smithi167 ceph-mon[32751]: pgmap v247: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 848 KiB/s wr, 120 op/s 2024-04-26T18:19:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:53 smithi090 ceph-mon[29180]: pgmap v247: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 848 KiB/s wr, 120 op/s 2024-04-26T18:19:53.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.10 MB/sec execute 221 sec latency 6168.261 ms 2024-04-26T18:19:54.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.08 MB/sec execute 222 sec latency 7168.346 ms 2024-04-26T18:19:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 -- 172.21.15.90:0/2209001819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac1067e0 msgr2=0x7f95ac106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 --2- 172.21.15.90:0/2209001819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac1067e0 0x7f95ac106bc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009900 tx=0x7f95a002a410 comp rx=0 tx=0).stop 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 -- 172.21.15.90:0/2209001819 shutdown_connections 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 --2- 172.21.15.90:0/2209001819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac102540 0x7f95ac10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 --2- 172.21.15.90:0/2209001819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95ac101b80 0x7f95ac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 --2- 172.21.15.90:0/2209001819 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac1067e0 0x7f95ac106bc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 -- 172.21.15.90:0/2209001819 >> 172.21.15.90:0/2209001819 conn(0x7f95ac077630 msgr2=0x7f95ac077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 -- 172.21.15.90:0/2209001819 shutdown_connections 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.245+0000 7f95b2f46640 1 -- 172.21.15.90:0/2209001819 wait complete. 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 Processor -- start 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 -- start start 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac101b80 0x7f95ac199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac102540 0x7f95ac199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95ac1067e0 0x7f95ac19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac111ad0 con 0x7f95ac102540 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac111950 con 0x7f95ac1067e0 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95b2f46640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac111c50 con 0x7f95ac101b80 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95abfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac102540 0x7f95ac199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95abfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac102540 0x7f95ac199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55886/0 (socket says 172.21.15.90:55886) 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.246+0000 7f95abfff640 1 -- 172.21.15.90:0/3903515387 learned_addr learned my addr 172.21.15.90:0/3903515387 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac101b80 0x7f95ac199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95b14e3640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95ac1067e0 0x7f95ac19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95abfff640 1 -- 172.21.15.90:0/3903515387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac101b80 msgr2=0x7f95ac199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95abfff640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac101b80 0x7f95ac199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95abfff640 1 -- 172.21.15.90:0/3903515387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95ac1067e0 msgr2=0x7f95ac19d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95abfff640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95ac1067e0 0x7f95ac19d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95abfff640 1 -- 172.21.15.90:0/3903515387 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac19e010 con 0x7f95ac102540 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95b14e3640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95ac1067e0 0x7f95ac19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95abfff640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac102540 0x7f95ac199560 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f959800e9f0 tx=0x7f959800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95a9ffb640 1 -- 172.21.15.90:0/3903515387 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959800cd80 con 0x7f95ac102540 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac19e300 con 0x7f95ac102540 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95a9ffb640 1 -- 172.21.15.90:0/3903515387 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95980108e0 con 0x7f95ac102540 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95a9ffb640 1 -- 172.21.15.90:0/3903515387 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598018980 con 0x7f95ac102540 2024-04-26T18:19:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.247+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac1a5b90 con 0x7f95ac102540 2024-04-26T18:19:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95a9ffb640 1 -- 172.21.15.90:0/3903515387 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f9598010430 con 0x7f95ac102540 2024-04-26T18:19:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 Processor -- start 2024-04-26T18:19:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 -- start start 2024-04-26T18:19:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95a9ffb640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f959c0a3dc0 0x7f959c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789bf0e0 0x555e789c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555e7891a460 con 0x555e789b3f00 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555e788d4590 con 0x555e789b4820 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b5c64740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555e78916420 con 0x555e789bf0e0 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b0ce2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.250+0000 7f95b14e3640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f959c0a3dc0 0x7f959c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55902/0 (socket says 172.21.15.90:55902) 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1422131202 learned_addr learned my addr 172.21.15.90:0/1422131202 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95abfff640 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95a9ffb640 1 -- 172.21.15.90:0/3903515387 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8669+0+0 (secure 0 0 0) 0x7f95980d0c00 con 0x7f95ac102540 2024-04-26T18:19:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b14e3640 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789bf0e0 0x555e789c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1422131202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789bf0e0 msgr2=0x555e789c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789bf0e0 0x555e789c14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1422131202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 msgr2=0x555e789beba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789beba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1422131202 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e788e10d0 con 0x555e789b3f00 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b14e3640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f959c0a3dc0 0x7f959c0a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009fd0 tx=0x7f95a4013c40 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789b42e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009900 tx=0x7f95a0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af88d7cac3fe02ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f9596ffd640 1 -- 172.21.15.90:0/1422131202 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0040070 con 0x555e789b3f00 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f9596ffd640 1 -- 172.21.15.90:0/1422131202 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a0004030 con 0x555e789b3f00 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.251+0000 7f9596ffd640 1 -- 172.21.15.90:0/1422131202 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a0035040 con 0x555e789b3f00 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 -- 172.21.15.90:0/1422131202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 msgr2=0x555e789b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789b42e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f95a0009900 tx=0x7f95a0005bd0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 -- 172.21.15.90:0/1422131202 shutdown_connections 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789bf0e0 0x555e789c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789beba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1422131202 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789b42e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 -- 172.21.15.90:0/1422131202 >> 172.21.15.90:0/1422131202 conn(0x555e789b0410 msgr2=0x555e789b07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 -- 172.21.15.90:0/1422131202 shutdown_connections 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 -- 172.21.15.90:0/1422131202 wait complete. 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 -- start start 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.252+0000 7f95b5c64740 1 Processor -- start 2024-04-26T18:19:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789bc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b4820 0x555e789bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789bcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b5c64740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x555e789c1cb0 con 0x555e789b3f00 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b5c64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x555e788a1b60 con 0x555e789bf0e0 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b5c64740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555e789c1e30 con 0x555e789b4820 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b5c64740 10 client.0 _ll_register_callbacks cb 0x555e788c62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789bc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789bc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55908/0 (socket says 172.21.15.90:55908) 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1673236236 learned_addr learned my addr 172.21.15.90:0/1673236236 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95abfff640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b4820 0x555e789bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b14e3640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789bcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1673236236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b4820 msgr2=0x555e789bc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b4820 0x555e789bc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 msgr2=0x555e789bcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789bcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 -- 172.21.15.90:0/1673236236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e7862c210 con 0x555e789b3f00 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b14e3640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789bcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789bc020 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f95a0031f80 tx=0x7f95a00041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f8355aa90f0e92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.253+0000 7f958bfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f958bfff640 1 -- 172.21.15.90:0/1673236236 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a004c020 con 0x555e789b3f00 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f958bfff640 1 -- 172.21.15.90:0/1673236236 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8669+0+0 (secure 0 0 0) 0x7f95a0031840 con 0x555e789b3f00 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95a0004030 con 0x555e789b3f00 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f958bfff640 1 -- 172.21.15.90:0/1673236236 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f95a0057050 con 0x555e789b3f00 2024-04-26T18:19:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 10 client.16134 fetch_fsmap learned FSMap version 48 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555e786f8620 con 0x555e789b3f00 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f958bfff640 1 -- 172.21.15.90:0/1673236236 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95a0004030 con 0x555e789b3f00 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f958bfff640 1 -- 172.21.15.90:0/1673236236 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 8338+0+0 (secure 0 0 0) 0x7f95a0004cf0 con 0x555e789b3f00 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 10 client.16134 fetch_fsmap finished waiting for FSMap version 48 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 10 client.16134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 20 client.16134 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e789cf000 0x555e789d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 4 client.16134 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.254+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e78928070 con 0x555e789cf000 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.255+0000 7f95abfff640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e789cf000 0x555e789d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.255+0000 7f95abfff640 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e789cf000 0x555e789d13f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dcc70047a6c7815 server_cookie=ab7e84cbd089426c in_seq=0 out_seq=0 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.255+0000 7f958bfff640 10 client.16134 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f958bfff640 1 -- 172.21.15.90:0/1673236236 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e78928070 con 0x555e789cf000 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f958bfff640 10 client.16134 handle_command_reply: tid=0 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 client.16134 shutdown 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 msgr2=0x555e789bc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789bc020 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f95a0031f80 tx=0x7f95a00041e0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 shutdown_connections 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e789cf000 0x555e789d13f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789bcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b4820 0x555e789bc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f958bfff640 0 client.16134 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 --2- 172.21.15.90:0/1673236236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b3f00 0x555e789bc020 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 >> 172.21.15.90:0/1673236236 conn(0x555e789b0410 msgr2=0x555e789b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 shutdown_connections 2024-04-26T18:19:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 1 -- 172.21.15.90:0/1673236236 wait complete. 2024-04-26T18:19:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.256+0000 7f95b5c64740 20 client.16134 trim_cache size 0 max 16384 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 Processor -- start 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 -- start start 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789cab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b4820 0x555e789b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789d6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1b0e00 con 0x555e789b4820 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1ae6d0 con 0x555e789bf0e0 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b5c64740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1ac540 con 0x555e789b3f00 2024-04-26T18:19:55.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95abfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b4820 0x555e789b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95abfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b4820 0x555e789b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55912/0 (socket says 172.21.15.90:55912) 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95abfff640 1 -- 172.21.15.90:0/930199822 learned_addr learned my addr 172.21.15.90:0/930199822 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b14e3640 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789d6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.271+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95abfff640 1 -- 172.21.15.90:0/930199822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 msgr2=0x555e789cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95abfff640 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95abfff640 1 -- 172.21.15.90:0/930199822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 msgr2=0x555e789d6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95abfff640 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789d6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95abfff640 1 -- 172.21.15.90:0/930199822 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e789ba570 con 0x555e789b4820 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b14e3640 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789d6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95abfff640 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b4820 0x555e789b5fc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f95980ce8c0 tx=0x7f9598018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed2f090da1524c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95967fc640 1 -- 172.21.15.90:0/930199822 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598026040 con 0x555e789b4820 2024-04-26T18:19:55.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95967fc640 1 -- 172.21.15.90:0/930199822 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95980a5780 con 0x555e789b4820 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95967fc640 1 -- 172.21.15.90:0/930199822 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598099180 con 0x555e789b4820 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b5c64740 1 -- 172.21.15.90:0/930199822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b4820 msgr2=0x555e789b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b5c64740 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b4820 0x555e789b5fc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f95980ce8c0 tx=0x7f9598018420 comp rx=0 tx=0).stop 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b5c64740 1 -- 172.21.15.90:0/930199822 shutdown_connections 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b5c64740 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789bf0e0 0x555e789d6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b5c64740 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789b4820 0x555e789b5fc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b5c64740 1 --2- 172.21.15.90:0/930199822 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.272+0000 7f95b5c64740 1 -- 172.21.15.90:0/930199822 >> 172.21.15.90:0/930199822 conn(0x555e789b0410 msgr2=0x555e789b58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 -- 172.21.15.90:0/930199822 shutdown_connections 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 -- 172.21.15.90:0/930199822 wait complete. 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 -- start start 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 Processor -- start 2024-04-26T18:19:55.280 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789bf0e0 0x555e789b9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1acb90 con 0x555e789bf0e0 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1ae000 con 0x555e789b4820 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1ad260 con 0x555e789b3f00 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b5c64740 10 client.0 _ll_register_callbacks cb 0x7f9598009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b14e3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789bf0e0 0x555e789b9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b14e3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789bf0e0 0x555e789b9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55916/0 (socket says 172.21.15.90:55916) 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.273+0000 7f95b14e3640 1 -- 172.21.15.90:0/2601951600 learned_addr learned my addr 172.21.15.90:0/2601951600 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95abfff640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b14e3640 1 -- 172.21.15.90:0/2601951600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 msgr2=0x555e789c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b14e3640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b14e3640 1 -- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 msgr2=0x555e789b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b14e3640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b14e3640 1 -- 172.21.15.90:0/2601951600 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e789b9850 con 0x555e789bf0e0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95abfff640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b14e3640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789bf0e0 0x555e789b9020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009df0 tx=0x7f95a4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6811f7634a4890 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f958bfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f958bfff640 1 -- 172.21.15.90:0/2601951600 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4020d20 con 0x555e789bf0e0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f958bfff640 1 -- 172.21.15.90:0/2601951600 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8669+0+0 (secure 0 0 0) 0x7f95a4021400 con 0x555e789bf0e0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95980a5780 con 0x555e789bf0e0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f958bfff640 1 -- 172.21.15.90:0/2601951600 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f95a4021600 con 0x555e789bf0e0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b5c64740 10 client.16140 fetch_fsmap learned FSMap version 48 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x555e78a35340 con 0x555e789bf0e0 2024-04-26T18:19:55.282 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.274+0000 7f958bfff640 1 -- 172.21.15.90:0/2601951600 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95980a5780 con 0x555e789bf0e0 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f958bfff640 1 -- 172.21.15.90:0/2601951600 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 48) ==== 8338+0+0 (secure 0 0 0) 0x7f95a402c610 con 0x555e789bf0e0 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b5c64740 10 client.16140 fetch_fsmap finished waiting for FSMap version 48 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b5c64740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b5c64740 20 client.16140 populate_metadata read hostname 'smithi090' 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b5c64740 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e78a35af0 0x555e78a37f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b5c64740 4 client.16140 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e78921ee0 con 0x555e78a35af0 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e78a35af0 0x555e78a37f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f95b0ce2640 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e78a35af0 0x555e78a37f10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc751a492650a85f server_cookie=e16377ef8d23f30f in_seq=0 out_seq=0 2024-04-26T18:19:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:19:55.275+0000 7f958bfff640 10 client.16140 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:19:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:55 smithi119 ceph-mon[32752]: pgmap v248: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 853 KiB/s wr, 125 op/s 2024-04-26T18:19:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:55 smithi167 ceph-mon[32751]: pgmap v248: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 853 KiB/s wr, 125 op/s 2024-04-26T18:19:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:55 smithi090 ceph-mon[29180]: pgmap v248: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 853 KiB/s wr, 125 op/s 2024-04-26T18:19:55.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.06 MB/sec execute 223 sec latency 8168.419 ms 2024-04-26T18:19:56.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.04 MB/sec execute 224 sec latency 9168.479 ms 2024-04-26T18:19:57.413 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:57 smithi090 ceph-mon[29180]: pgmap v249: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 717 KiB/s wr, 92 op/s 2024-04-26T18:19:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:57 smithi119 ceph-mon[32752]: pgmap v249: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 717 KiB/s wr, 92 op/s 2024-04-26T18:19:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:57 smithi167 ceph-mon[32751]: pgmap v249: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 717 KiB/s wr, 92 op/s 2024-04-26T18:19:57.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.03 MB/sec execute 225 sec latency 10168.556 ms 2024-04-26T18:19:58.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 4.01 MB/sec execute 226 sec latency 11168.657 ms 2024-04-26T18:19:58.889 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:19:58.889 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 systemd[1]: Stopped Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:19:58.889 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.428s CPU time. 2024-04-26T18:19:58.889 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 systemd[1]: Starting Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:19:58.889 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 podman[68314]: 2024-04-26 18:19:58.785534407 +0000 UTC m=+0.110705859 container create f22de69f48d4db9c4f21f2dc56aa81c54531d4a69868c691f603f38ed0523043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T18:19:58.890 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 podman[68314]: 2024-04-26 18:19:58.692907683 +0000 UTC m=+0.018079139 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:19:59.155 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 podman[68314]: 2024-04-26 18:19:58.995790578 +0000 UTC m=+0.320962032 container init f22de69f48d4db9c4f21f2dc56aa81c54531d4a69868c691f603f38ed0523043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T18:19:59.155 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:58 smithi119 podman[68314]: 2024-04-26 18:19:58.998507367 +0000 UTC m=+0.323678820 container start f22de69f48d4db9c4f21f2dc56aa81c54531d4a69868c691f603f38ed0523043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T18:19:59.155 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mds[68333]: -- 172.21.15.119:0/1745765076 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d5bf9da40 con 0x563d5bf70800 2024-04-26T18:19:59.155 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: starting mds.h at 2024-04-26T18:19:59.155 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:59 smithi119 bash[68314]: f22de69f48d4db9c4f21f2dc56aa81c54531d4a69868c691f603f38ed0523043 2024-04-26T18:19:59.156 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:19:59 smithi119 systemd[1]: Started Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: pgmap v250: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 717 KiB/s wr, 92 op/s 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: daemon mds.h restarted 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:boot 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:19:59 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: pgmap v250: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 717 KiB/s wr, 92 op/s 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: daemon mds.h restarted 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:boot 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:19:59 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: pgmap v250: 97 pgs: 97 active+clean; 104 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 717 KiB/s wr, 92 op/s 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: daemon mds.h restarted 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:boot 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:19:59 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:19:59.680 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 3.99 MB/sec execute 227 sec latency 12168.766 ms 2024-04-26T18:20:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:00 smithi119 ceph-mon[32752]: Health detail: HEALTH_WARN 1 filesystem is degraded 2024-04-26T18:20:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:00 smithi119 ceph-mon[32752]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T18:20:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:00 smithi119 ceph-mon[32752]: fs cephfs is degraded 2024-04-26T18:20:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:00 smithi167 ceph-mon[32751]: Health detail: HEALTH_WARN 1 filesystem is degraded 2024-04-26T18:20:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:00 smithi167 ceph-mon[32751]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T18:20:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:00 smithi167 ceph-mon[32751]: fs cephfs is degraded 2024-04-26T18:20:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:00 smithi090 ceph-mon[29180]: Health detail: HEALTH_WARN 1 filesystem is degraded 2024-04-26T18:20:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:00 smithi090 ceph-mon[29180]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T18:20:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:00 smithi090 ceph-mon[29180]: fs cephfs is degraded 2024-04-26T18:20:00.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 3.97 MB/sec execute 228 sec latency 13168.901 ms 2024-04-26T18:20:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:01 smithi119 ceph-mon[32752]: pgmap v252: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 989 KiB/s wr, 54 op/s 2024-04-26T18:20:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:01 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:resolve 2024-04-26T18:20:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:01 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:01 smithi167 ceph-mon[32751]: pgmap v252: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 989 KiB/s wr, 54 op/s 2024-04-26T18:20:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:01 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:resolve 2024-04-26T18:20:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:01 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:01 smithi090 ceph-mon[29180]: pgmap v252: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 989 KiB/s wr, 54 op/s 2024-04-26T18:20:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:01 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:resolve 2024-04-26T18:20:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:01 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:01.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401289 3.96 MB/sec execute 229 sec latency 14168.935 ms 2024-04-26T18:20:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f958bfff640 1 -- 172.21.15.90:0/2601951600 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e78921ee0 con 0x555e78a35af0 2024-04-26T18:20:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f958bfff640 10 client.16140 handle_command_reply: tid=0 2024-04-26T18:20:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 client.16140 shutdown 2024-04-26T18:20:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789bf0e0 msgr2=0x555e789b9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789bf0e0 0x555e789b9020 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009df0 tx=0x7f95a4004ab0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 shutdown_connections 2024-04-26T18:20:02.194 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x555e78a35af0 0x555e78a37f10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x555e789bf0e0 0x555e789b9020 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555e789b4820 0x555e789b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f958bfff640 0 client.16140 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 --2- 172.21.15.90:0/2601951600 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555e789b3f00 0x555e789c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.193+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 >> 172.21.15.90:0/2601951600 conn(0x555e789b0410 msgr2=0x555e789bc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.194+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 shutdown_connections 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.194+0000 7f95b5c64740 1 -- 172.21.15.90:0/2601951600 wait complete. 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.194+0000 7f95b5c64740 20 client.16140 trim_cache size 0 max 16384 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:20:02.195 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:20:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:20:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:20:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f959c0a3dc0 msgr2=0x7f959c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f959c0a3dc0 0x7f959c0a6280 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009fd0 tx=0x7f95a4013c40 comp rx=0 tx=0).stop 2024-04-26T18:20:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac102540 msgr2=0x7f95ac199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:02.196 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac102540 0x7f95ac199560 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f959800e9f0 tx=0x7f959800eeb0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 shutdown_connections 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f959c0a3dc0 0x7f959c0a6280 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95ac1067e0 0x7f95ac19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95ac102540 0x7f95ac199560 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 --2- 172.21.15.90:0/3903515387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95ac101b80 0x7f95ac199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 >> 172.21.15.90:0/3903515387 conn(0x7f95ac077630 msgr2=0x7f95ac06b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 shutdown_connections 2024-04-26T18:20:02.197 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:02.195+0000 7f95b2f46640 1 -- 172.21.15.90:0/3903515387 wait complete. 2024-04-26T18:20:02.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:02 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:20:02.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:02 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:20:02.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:02 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:reconnect 2024-04-26T18:20:02.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:02 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:02.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:02 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:20:02.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:02 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:20:02.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:02 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:reconnect 2024-04-26T18:20:02.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:02 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:02 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:20:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:02 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:20:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:02 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:reconnect 2024-04-26T18:20:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:02 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:02.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 401648 3.94 MB/sec execute 230 sec latency 14673.967 ms 2024-04-26T18:20:02.817 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 26 18:20:02 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-26T18:20:02.508+0000 7f61d50ae700 -1 bdev(0x557f5274a000 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 2 2024-04-26T18:20:03.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:20:03.207+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:20:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: pgmap v253: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 989 KiB/s wr, 54 op/s 2024-04-26T18:20:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:rejoin 2024-04-26T18:20:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:20:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:20:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:active 2024-04-26T18:20:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:03 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: pgmap v253: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 989 KiB/s wr, 54 op/s 2024-04-26T18:20:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:rejoin 2024-04-26T18:20:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:20:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:20:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:active 2024-04-26T18:20:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:03 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: pgmap v253: 97 pgs: 97 active+clean; 108 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 989 KiB/s wr, 54 op/s 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:rejoin 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/1142719310,v1:172.21.15.167:6839/1142719310] up:active 2024-04-26T18:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:03 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:03.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 404366 4.00 MB/sec execute 231 sec latency 11.265 ms 2024-04-26T18:20:04.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 405659 4.00 MB/sec execute 232 sec latency 14.315 ms 2024-04-26T18:20:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:05 smithi119 ceph-mon[32752]: pgmap v254: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-04-26T18:20:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:05 smithi167 ceph-mon[32751]: pgmap v254: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-04-26T18:20:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:05 smithi090 ceph-mon[29180]: pgmap v254: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.3 MiB/s wr, 113 op/s 2024-04-26T18:20:05.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 407890 4.04 MB/sec execute 233 sec latency 25.202 ms 2024-04-26T18:20:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:06 smithi119 ceph-mon[32752]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:20:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:06 smithi119 ceph-mon[32752]: pgmap v256: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-04-26T18:20:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:06 smithi167 ceph-mon[32751]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:20:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:06 smithi167 ceph-mon[32751]: pgmap v256: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-04-26T18:20:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:06 smithi090 ceph-mon[29180]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:20:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:06 smithi090 ceph-mon[29180]: pgmap v256: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-04-26T18:20:06.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 409133 4.04 MB/sec execute 234 sec latency 16.894 ms 2024-04-26T18:20:07.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 411770 4.10 MB/sec execute 235 sec latency 23.222 ms 2024-04-26T18:20:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:08 smithi090 ceph-mon[29180]: pgmap v257: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 369 KiB/s wr, 78 op/s 2024-04-26T18:20:08.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 4.08 MB/sec execute 236 sec latency 622.391 ms 2024-04-26T18:20:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:08 smithi119 ceph-mon[32752]: pgmap v257: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 369 KiB/s wr, 78 op/s 2024-04-26T18:20:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:08 smithi167 ceph-mon[32751]: pgmap v257: 97 pgs: 97 active+clean; 104 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 369 KiB/s wr, 78 op/s 2024-04-26T18:20:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:09.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 4.06 MB/sec execute 237 sec latency 1622.422 ms 2024-04-26T18:20:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:10 smithi090 ceph-mon[29180]: pgmap v258: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-04-26T18:20:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:20:10.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 4.05 MB/sec execute 238 sec latency 2622.477 ms 2024-04-26T18:20:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:10 smithi119 ceph-mon[32752]: pgmap v258: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-04-26T18:20:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:20:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:10 smithi167 ceph-mon[32751]: pgmap v258: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-04-26T18:20:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:20:11.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 4.03 MB/sec execute 239 sec latency 3622.529 ms 2024-04-26T18:20:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:12 smithi090 ceph-mon[29180]: pgmap v259: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-04-26T18:20:12.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 4.01 MB/sec execute 240 sec latency 4622.656 ms 2024-04-26T18:20:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:12 smithi119 ceph-mon[32752]: pgmap v259: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-04-26T18:20:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:12 smithi167 ceph-mon[32751]: pgmap v259: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.2 MiB/s wr, 181 op/s 2024-04-26T18:20:13.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 4.00 MB/sec execute 241 sec latency 5622.748 ms 2024-04-26T18:20:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:13 smithi119 ceph-mon[32752]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:20:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:13 smithi167 ceph-mon[32751]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:20:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:13 smithi090 ceph-mon[29180]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:20:14.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.98 MB/sec execute 242 sec latency 6622.805 ms 2024-04-26T18:20:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:14 smithi119 ceph-mon[32752]: pgmap v261: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-26T18:20:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:14 smithi167 ceph-mon[32751]: pgmap v261: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-26T18:20:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:14 smithi090 ceph-mon[29180]: pgmap v261: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-04-26T18:20:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.293+0000 7fdd95899640 1 -- 172.21.15.90:0/1276276859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 msgr2=0x7fdd90108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.293+0000 7fdd95899640 1 --2- 172.21.15.90:0/1276276859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd90108d40 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009cd0 tx=0x7fdd8002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.293+0000 7fdd95899640 1 -- 172.21.15.90:0/1276276859 shutdown_connections 2024-04-26T18:20:15.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.293+0000 7fdd95899640 1 --2- 172.21.15.90:0/1276276859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd90103310 0x7fdd9010f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.293+0000 7fdd95899640 1 --2- 172.21.15.90:0/1276276859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd90102950 0x7fdd90102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.293+0000 7fdd95899640 1 --2- 172.21.15.90:0/1276276859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd90108d40 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.293+0000 7fdd95899640 1 -- 172.21.15.90:0/1276276859 >> 172.21.15.90:0/1276276859 conn(0x7fdd900fe3e0 msgr2=0x7fdd90100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.294+0000 7fdd95899640 1 -- 172.21.15.90:0/1276276859 shutdown_connections 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.294+0000 7fdd95899640 1 -- 172.21.15.90:0/1276276859 wait complete. 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.294+0000 7fdd95899640 1 Processor -- start 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.294+0000 7fdd95899640 1 -- start start 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.294+0000 7fdd95899640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd90102950 0x7fdd90199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd95899640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd90103310 0x7fdd90199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd95899640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd9019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd95899640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90112c20 con 0x7fdd90108960 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd95899640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90112aa0 con 0x7fdd90102950 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd95899640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90112da0 con 0x7fdd90103310 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd9019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd9019d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40484/0 (socket says 172.21.15.90:40484) 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 -- 172.21.15.90:0/927478446 learned_addr learned my addr 172.21.15.90:0/927478446 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8effd640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd90102950 0x7fdd90199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd90103310 0x7fdd90199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 -- 172.21.15.90:0/927478446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd90103310 msgr2=0x7fdd90199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd90103310 0x7fdd90199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 -- 172.21.15.90:0/927478446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd90102950 msgr2=0x7fdd90199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd90102950 0x7fdd90199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 -- 172.21.15.90:0/927478446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9019e110 con 0x7fdd90108960 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd90103310 0x7fdd90199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8effd640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd90102950 0x7fdd90199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd8f7fe640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd9019d9e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800d950 tx=0x7fdd8800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd94897640 1 -- 172.21.15.90:0/927478446 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88017030 con 0x7fdd90108960 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9019e400 con 0x7fdd90108960 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd94897640 1 -- 172.21.15.90:0/927478446 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd88004020 con 0x7fdd90108960 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd94897640 1 -- 172.21.15.90:0/927478446 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88003b50 con 0x7fdd90108960 2024-04-26T18:20:15.299 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.295+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd901a5c90 con 0x7fdd90108960 2024-04-26T18:20:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd94897640 1 -- 172.21.15.90:0/927478446 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fdd8801a070 con 0x7fdd90108960 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 Processor -- start 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 -- start start 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c3d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c4bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c4e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd94897640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd780a3dc0 0x7fdd780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e855ba3460 con 0x55e855c3d720 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e855b5d590 con 0x55e855c4c4f0 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd985ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e855b9f420 con 0x55e855c3ce00 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c4bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd8f7fe640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd780a3dc0 0x7fdd780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c4bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40498/0 (socket says 172.21.15.90:40498) 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.298+0000 7fdd8effd640 1 -- 172.21.15.90:0/1096642107 learned_addr learned my addr 172.21.15.90:0/1096642107 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c4e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd94897640 1 -- 172.21.15.90:0/927478446 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8799+0+0 (secure 0 0 0) 0x7fdd880d0400 con 0x7fdd90108960 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c3d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8effd640 1 -- 172.21.15.90:0/1096642107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 msgr2=0x55e855c3d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8effd640 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c3d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8effd640 1 -- 172.21.15.90:0/1096642107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 msgr2=0x55e855c4e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8effd640 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c4e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8effd640 1 -- 172.21.15.90:0/1096642107 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e855b6a0d0 con 0x55e855c3d720 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8f7fe640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd780a3dc0 0x7fdd780a6280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd880986f0 tx=0x7fdd88097e90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd8effd640 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c4bfb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdd80035000 tx=0x7fdd80009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fbf24965d16a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd75ffb640 1 -- 172.21.15.90:0/1096642107 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80031850 con 0x55e855c3d720 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd75ffb640 1 -- 172.21.15.90:0/1096642107 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd80036040 con 0x55e855c3d720 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.299+0000 7fdd75ffb640 1 -- 172.21.15.90:0/1096642107 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80004570 con 0x55e855c3d720 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 -- 172.21.15.90:0/1096642107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 msgr2=0x55e855c4bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c4bfb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdd80035000 tx=0x7fdd80009a20 comp rx=0 tx=0).stop 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 -- 172.21.15.90:0/1096642107 shutdown_connections 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c4e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c4bfb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 --2- 172.21.15.90:0/1096642107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c3d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 -- 172.21.15.90:0/1096642107 >> 172.21.15.90:0/1096642107 conn(0x55e855c39410 msgr2=0x55e855c397f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 -- 172.21.15.90:0/1096642107 shutdown_connections 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 -- 172.21.15.90:0/1096642107 wait complete. 2024-04-26T18:20:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.300+0000 7fdd985ba740 1 -- start start 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 1 Processor -- start 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c49390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c498d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 0x55e855c49e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e855c4f0c0 con 0x55e855c3d720 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e855b2ab60 con 0x55e855c3ce00 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e855c4f240 con 0x55e855c4c4f0 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd985ba740 10 client.0 _ll_register_callbacks cb 0x55e855b4f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40506/0 (socket says 172.21.15.90:40506) 2024-04-26T18:20:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 -- 172.21.15.90:0/2846616580 learned_addr learned my addr 172.21.15.90:0/2846616580 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c49390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 0x55e855c49e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 -- 172.21.15.90:0/2846616580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 msgr2=0x55e855c49e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 0x55e855c49e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 -- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 msgr2=0x55e855c49390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c49390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 -- 172.21.15.90:0/2846616580 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8558b5210 con 0x55e855c3d720 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd8effd640 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c498d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000f80 tx=0x7fdd80004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96f9af51690e4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd6effd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.301+0000 7fdd6effd640 1 -- 172.21.15.90:0/2846616580 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8004e070 con 0x55e855c3d720 2024-04-26T18:20:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd6effd640 1 -- 172.21.15.90:0/2846616580 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8799+0+0 (secure 0 0 0) 0x7fdd800315f0 con 0x55e855c3d720 2024-04-26T18:20:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd80036040 con 0x55e855c3d720 2024-04-26T18:20:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd6effd640 1 -- 172.21.15.90:0/2846616580 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdd8005a050 con 0x55e855c3d720 2024-04-26T18:20:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 10 client.16152 fetch_fsmap learned FSMap version 54 2024-04-26T18:20:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e855981620 con 0x55e855c3d720 2024-04-26T18:20:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd6effd640 1 -- 172.21.15.90:0/2846616580 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd80036040 con 0x55e855c3d720 2024-04-26T18:20:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd6effd640 1 -- 172.21.15.90:0/2846616580 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7fdd80030070 con 0x55e855c3d720 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 10 client.16152 fetch_fsmap finished waiting for FSMap version 54 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 10 client.16152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 20 client.16152 populate_metadata read hostname 'smithi090' 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855c5c470 0x55e855c5e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 4 client.16152 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e855bb1070 con 0x55e855c5c470 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.302+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855c5c470 0x55e855c5e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.303+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855c5c470 0x55e855c5e860 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac8e4898b04c95f server_cookie=56f42d8a95e46953 in_seq=0 out_seq=0 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.303+0000 7fdd6effd640 10 client.16152 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.303+0000 7fdd6effd640 1 -- 172.21.15.90:0/2846616580 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e855bb1070 con 0x55e855c5c470 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.303+0000 7fdd6effd640 10 client.16152 handle_command_reply: tid=0 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 client.16152 shutdown 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 msgr2=0x55e855c498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c498d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000f80 tx=0x7fdd80004520 comp rx=0 tx=0).stop 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 shutdown_connections 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855c5c470 0x55e855c5e860 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 0x55e855c49e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c498d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 --2- 172.21.15.90:0/2846616580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c49390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd6effd640 0 client.16152 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 >> 172.21.15.90:0/2846616580 conn(0x55e855c39410 msgr2=0x55e855c3f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 shutdown_connections 2024-04-26T18:20:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 1 -- 172.21.15.90:0/2846616580 wait complete. 2024-04-26T18:20:15.315 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.304+0000 7fdd985ba740 20 client.16152 trim_cache size 0 max 16384 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 Processor -- start 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 -- start start 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c484f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c57c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 0x55e855c61880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901aedb0 con 0x55e855c3d720 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901ac680 con 0x55e855c3ce00 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd985ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901aa510 con 0x55e855c4c4f0 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c57c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c57c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40510/0 (socket says 172.21.15.90:40510) 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 -- 172.21.15.90:0/3087732713 learned_addr learned my addr 172.21.15.90:0/3087732713 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 -- 172.21.15.90:0/3087732713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 msgr2=0x55e855c61880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 0x55e855c61880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 -- 172.21.15.90:0/3087732713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 msgr2=0x55e855c484f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c484f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 -- 172.21.15.90:0/3087732713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e855c4f140 con 0x55e855c3d720 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd8effd640 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c57c10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd800307c0 tx=0x7fdd80009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1af130673992a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.319+0000 7fdd757fa640 1 -- 172.21.15.90:0/3087732713 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8004e070 con 0x55e855c3d720 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd757fa640 1 -- 172.21.15.90:0/3087732713 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd80036030 con 0x55e855c3d720 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd757fa640 1 -- 172.21.15.90:0/3087732713 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80030ed0 con 0x55e855c3d720 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 -- 172.21.15.90:0/3087732713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 msgr2=0x55e855c57c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c57c10 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdd800307c0 tx=0x7fdd80009a20 comp rx=0 tx=0).stop 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 -- 172.21.15.90:0/3087732713 shutdown_connections 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c4c4f0 0x55e855c61880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c57c10 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 --2- 172.21.15.90:0/3087732713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c3ce00 0x55e855c484f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 -- 172.21.15.90:0/3087732713 >> 172.21.15.90:0/3087732713 conn(0x55e855c39410 msgr2=0x55e855c57500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 -- 172.21.15.90:0/3087732713 shutdown_connections 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 -- 172.21.15.90:0/3087732713 wait complete. 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.320+0000 7fdd985ba740 1 -- start start 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 1 Processor -- start 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c5f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c5f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c5fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901aab60 con 0x55e855c3d720 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901abfb0 con 0x55e855c4c4f0 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd901ab210 con 0x55e855c3ce00 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd985ba740 10 client.0 _ll_register_callbacks cb 0x7fdd80009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c5f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c5f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40518/0 (socket says 172.21.15.90:40518) 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 -- 172.21.15.90:0/315359098 learned_addr learned my addr 172.21.15.90:0/315359098 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c5fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c5f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 -- 172.21.15.90:0/315359098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 msgr2=0x55e855c5f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c5f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 -- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 msgr2=0x55e855c5fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c5fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8effd640 1 -- 172.21.15.90:0/315359098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e855c3dc60 con 0x55e855c3d720 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.321+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c5fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd8effd640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c5f740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd80002230 tx=0x7fdd80009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6648b116c3d93f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd6effd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd6effd640 1 -- 172.21.15.90:0/315359098 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8004e070 con 0x55e855c3d720 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd6effd640 1 -- 172.21.15.90:0/315359098 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8799+0+0 (secure 0 0 0) 0x7fdd800307f0 con 0x55e855c3d720 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd80036030 con 0x55e855c3d720 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd6effd640 1 -- 172.21.15.90:0/315359098 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdd800309f0 con 0x55e855c3d720 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 10 client.16158 fetch_fsmap learned FSMap version 54 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55e855c39240 con 0x55e855c3d720 2024-04-26T18:20:15.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd6effd640 1 -- 172.21.15.90:0/315359098 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd80036030 con 0x55e855c3d720 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd6effd640 1 -- 172.21.15.90:0/315359098 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7fdd80030050 con 0x55e855c3d720 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 10 client.16158 fetch_fsmap finished waiting for FSMap version 54 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 10 client.16158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 20 client.16158 populate_metadata read hostname 'smithi090' 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855cc2b70 0x55e855cc4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 4 client.16158 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e855baaee0 con 0x55e855cc2b70 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.322+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855cc2b70 0x55e855cc4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.323+0000 7fdd8e7fc640 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855cc2b70 0x55e855cc4f90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0b98bbce82792a2 server_cookie=44b86ac4bbf2b763 in_seq=0 out_seq=0 2024-04-26T18:20:15.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.323+0000 7fdd6effd640 10 client.16158 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:15.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd6effd640 1 -- 172.21.15.90:0/315359098 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e855baaee0 con 0x55e855cc2b70 2024-04-26T18:20:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd6effd640 10 client.16158 handle_command_reply: tid=0 2024-04-26T18:20:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 client.16158 shutdown 2024-04-26T18:20:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 msgr2=0x55e855c5f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c5f740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdd80002230 tx=0x7fdd80009800 comp rx=0 tx=0).stop 2024-04-26T18:20:15.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 shutdown_connections 2024-04-26T18:20:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e855cc2b70 0x55e855cc4f90 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e855c4c4f0 0x55e855c5fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e855c3d720 0x55e855c5f740 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd6effd640 0 client.16158 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 --2- 172.21.15.90:0/315359098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e855c3ce00 0x55e855c5f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.372 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 >> 172.21.15.90:0/315359098 conn(0x55e855c39410 msgr2=0x55e855c57500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 shutdown_connections 2024-04-26T18:20:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 1 -- 172.21.15.90:0/315359098 wait complete. 2024-04-26T18:20:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.370+0000 7fdd985ba740 20 client.16158 trim_cache size 0 max 16384 2024-04-26T18:20:15.373 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:20:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:20:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:20:15.374 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:20:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd780a3dc0 msgr2=0x7fdd780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd780a3dc0 0x7fdd780a6280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd880986f0 tx=0x7fdd88097e90 comp rx=0 tx=0).stop 2024-04-26T18:20:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 msgr2=0x7fdd9019d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd9019d9e0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdd8800d950 tx=0x7fdd8800de10 comp rx=0 tx=0).stop 2024-04-26T18:20:15.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 shutdown_connections 2024-04-26T18:20:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdd780a3dc0 0x7fdd780a6280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdd90108960 0x7fdd9019d9e0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd90103310 0x7fdd90199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 --2- 172.21.15.90:0/927478446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd90102950 0x7fdd90199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 >> 172.21.15.90:0/927478446 conn(0x7fdd900fe3e0 msgr2=0x7fdd900fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 shutdown_connections 2024-04-26T18:20:15.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:15.371+0000 7fdd95899640 1 -- 172.21.15.90:0/927478446 wait complete. 2024-04-26T18:20:15.681 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.96 MB/sec execute 243 sec latency 7622.921 ms 2024-04-26T18:20:16.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.95 MB/sec execute 244 sec latency 8623.018 ms 2024-04-26T18:20:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:16 smithi119 ceph-mon[32752]: pgmap v262: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 979 KiB/s wr, 124 op/s 2024-04-26T18:20:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:16 smithi167 ceph-mon[32751]: pgmap v262: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 979 KiB/s wr, 124 op/s 2024-04-26T18:20:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:16 smithi090 ceph-mon[29180]: pgmap v262: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 979 KiB/s wr, 124 op/s 2024-04-26T18:20:17.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.93 MB/sec execute 245 sec latency 9623.094 ms 2024-04-26T18:20:18.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.92 MB/sec execute 246 sec latency 10623.218 ms 2024-04-26T18:20:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:18 smithi119 ceph-mon[32752]: pgmap v263: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 979 KiB/s wr, 124 op/s 2024-04-26T18:20:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:18 smithi167 ceph-mon[32751]: pgmap v263: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 979 KiB/s wr, 124 op/s 2024-04-26T18:20:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:18 smithi090 ceph-mon[29180]: pgmap v263: 97 pgs: 97 active+clean; 109 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 979 KiB/s wr, 124 op/s 2024-04-26T18:20:19.379 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f28aec91700 time 2024-04-26T18:20:19.115854+0000 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f28b92ba3dd] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.380 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.381 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.381 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.381 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: clone() 2024-04-26T18:20:19.381 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.115+0000 7f28aec91700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f28aec91700 time 2024-04-26T18:20:19.115854+0000 2024-04-26T18:20:19.381 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:19.381 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.382 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.382 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f28b92ba3dd] 2024-04-26T18:20:19.382 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.382 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.382 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.382 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: clone() 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: *** Caught signal (Aborted) ** 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: in thread 7f28aec91700 thread_name:ms_dispatch 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f28b8042d20] 2024-04-26T18:20:19.383 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: gsignal() 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: abort() 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f28b92ba437] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.384 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 16: clone() 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.117+0000 7f28aec91700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: in thread 7f28aec91700 thread_name:ms_dispatch 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f28b8042d20] 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: gsignal() 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: abort() 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f28b92ba437] 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.385 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 16: clone() 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: -2> 2024-04-26T18:20:19.115+0000 7f28aec91700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f28aec91700 time 2024-04-26T18:20:19.115854+0000 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.386 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f28b92ba3dd] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: clone() 2024-04-26T18:20:19.387 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 0> 2024-04-26T18:20:19.117+0000 7f28aec91700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: in thread 7f28aec91700 thread_name:ms_dispatch 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f28b8042d20] 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: gsignal() 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: abort() 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f28b92ba437] 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.388 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 16: clone() 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: -2> 2024-04-26T18:20:19.115+0000 7f28aec91700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f28aec91700 time 2024-04-26T18:20:19.115854+0000 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f28b92ba3dd] 2024-04-26T18:20:19.389 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: clone() 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.390 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 0> 2024-04-26T18:20:19.117+0000 7f28aec91700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: in thread 7f28aec91700 thread_name:ms_dispatch 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f28b8042d20] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2: gsignal() 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 3: abort() 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f28b92ba437] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f28b92ba5a3] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56494fe9b822] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56494fe0b598] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56494fe0dc17] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56494fd06d0b] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56494fd072ac] 2024-04-26T18:20:19.391 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56494fceffad] 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f28b954ee20] 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 13: (DispatchQueue::entry()+0x51f) [0x7f28b954bfdf] 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f28b961efc1] 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f28b80381ca] 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 16: clone() 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68574]: 2024-04-26T18:20:19.392 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 podman[68787]: 2024-04-26 18:20:19.18173512 +0000 UTC m=+0.020448557 container died 1e87c026f61106e99d641646350c01f47f8af3359eaec4318c91de9a64e0d557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T18:20:19.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.90 MB/sec execute 247 sec latency 11623.294 ms 2024-04-26T18:20:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 podman[68787]: 2024-04-26 18:20:19.786713451 +0000 UTC m=+0.625426926 container remove 1e87c026f61106e99d641646350c01f47f8af3359eaec4318c91de9a64e0d557 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:20:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:19 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:20:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:20 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-04-26T18:20:20.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:20 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.679s CPU time. 2024-04-26T18:20:20.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.88 MB/sec execute 248 sec latency 12623.344 ms 2024-04-26T18:20:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:20 smithi119 ceph-mon[32752]: pgmap v264: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 40 op/s 2024-04-26T18:20:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:20 smithi167 ceph-mon[32751]: pgmap v264: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 40 op/s 2024-04-26T18:20:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:20 smithi090 ceph-mon[29180]: pgmap v264: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 40 op/s 2024-04-26T18:20:21.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.87 MB/sec execute 249 sec latency 13623.444 ms 2024-04-26T18:20:22.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.85 MB/sec execute 250 sec latency 14623.501 ms 2024-04-26T18:20:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:22 smithi119 ceph-mon[32752]: pgmap v265: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 40 op/s 2024-04-26T18:20:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:22 smithi167 ceph-mon[32751]: pgmap v265: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 40 op/s 2024-04-26T18:20:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:22 smithi090 ceph-mon[29180]: pgmap v265: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 173 KiB/s wr, 40 op/s 2024-04-26T18:20:23.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.84 MB/sec execute 251 sec latency 15623.589 ms 2024-04-26T18:20:24.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.82 MB/sec execute 252 sec latency 16623.650 ms 2024-04-26T18:20:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:24 smithi119 ceph-mon[32752]: pgmap v266: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 164 KiB/s wr, 41 op/s 2024-04-26T18:20:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:24 smithi167 ceph-mon[32751]: pgmap v266: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 164 KiB/s wr, 41 op/s 2024-04-26T18:20:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:24 smithi090 ceph-mon[29180]: pgmap v266: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 164 KiB/s wr, 41 op/s 2024-04-26T18:20:25.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.81 MB/sec execute 253 sec latency 17623.770 ms 2024-04-26T18:20:26.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.79 MB/sec execute 254 sec latency 18623.872 ms 2024-04-26T18:20:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:26 smithi119 ceph-mon[32752]: pgmap v267: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 139 KiB/s wr, 29 op/s 2024-04-26T18:20:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:26 smithi167 ceph-mon[32751]: pgmap v267: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 139 KiB/s wr, 29 op/s 2024-04-26T18:20:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:26 smithi090 ceph-mon[29180]: pgmap v267: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 139 KiB/s wr, 29 op/s 2024-04-26T18:20:27.682 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.78 MB/sec execute 255 sec latency 19623.953 ms 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 -- 172.21.15.90:0/652730245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c107860 msgr2=0x7f578c107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 --2- 172.21.15.90:0/652730245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c107860 0x7f578c107c40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f577c009930 tx=0x7f577c02a420 comp rx=0 tx=0).stop 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 -- 172.21.15.90:0/652730245 shutdown_connections 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 --2- 172.21.15.90:0/652730245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f578c102210 0x7f578c10e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 --2- 172.21.15.90:0/652730245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c101850 0x7f578c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 --2- 172.21.15.90:0/652730245 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c107860 0x7f578c107c40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 -- 172.21.15.90:0/652730245 >> 172.21.15.90:0/652730245 conn(0x7f578c077bb0 msgr2=0x7f578c077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 -- 172.21.15.90:0/652730245 shutdown_connections 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.467+0000 7f5791e08640 1 -- 172.21.15.90:0/652730245 wait complete. 2024-04-26T18:20:28.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 Processor -- start 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 -- start start 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f578c101850 0x7f578c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c102210 0x7f578c19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c107860 0x7f578c19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f578c111b20 con 0x7f578c102210 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1119a0 con 0x7f578c107860 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f5791e08640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f578c111ca0 con 0x7f578c101850 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f578affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c102210 0x7f578c19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f578affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c102210 0x7f578c19b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40784/0 (socket says 172.21.15.90:40784) 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f578affd640 1 -- 172.21.15.90:0/551471208 learned_addr learned my addr 172.21.15.90:0/551471208 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.468+0000 7f578b7fe640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f578c101850 0x7f578c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578bfff640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c107860 0x7f578c19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578affd640 1 -- 172.21.15.90:0/551471208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f578c101850 msgr2=0x7f578c19b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578affd640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f578c101850 0x7f578c19b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578affd640 1 -- 172.21.15.90:0/551471208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c107860 msgr2=0x7f578c19fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578affd640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c107860 0x7f578c19fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578affd640 1 -- 172.21.15.90:0/551471208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578c1a01b0 con 0x7f578c102210 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578bfff640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c107860 0x7f578c19fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578b7fe640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f578c101850 0x7f578c19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f578affd640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c102210 0x7f578c19b700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f578000c920 tx=0x7f578000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f5788ff9640 1 -- 172.21.15.90:0/551471208 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780007c70 con 0x7f578c102210 2024-04-26T18:20:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f5788ff9640 1 -- 172.21.15.90:0/551471208 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5780007e10 con 0x7f578c102210 2024-04-26T18:20:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f5788ff9640 1 -- 172.21.15.90:0/551471208 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780018820 con 0x7f578c102210 2024-04-26T18:20:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578c1a04a0 con 0x7f578c102210 2024-04-26T18:20:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.469+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f578c1a7d30 con 0x7f578c102210 2024-04-26T18:20:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.471+0000 7f5788ff9640 1 -- 172.21.15.90:0/551471208 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f578000f450 con 0x7f578c102210 2024-04-26T18:20:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.471+0000 7f5794bf5740 1 Processor -- start 2024-04-26T18:20:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5794bf5740 1 -- start start 2024-04-26T18:20:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.486 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd424cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5794bf5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561cd41a8460 con 0x561cd4242820 2024-04-26T18:20:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5794bf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561cd4162590 con 0x561cd4241f00 2024-04-26T18:20:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5794bf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561cd41a4420 con 0x561cd424d0e0 2024-04-26T18:20:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd424cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd424cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40786/0 (socket says 172.21.15.90:40786) 2024-04-26T18:20:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 -- 172.21.15.90:0/1948224663 learned_addr learned my addr 172.21.15.90:0/1948224663 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578bfff640 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578affd640 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 -- 172.21.15.90:0/1948224663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 msgr2=0x561cd424f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 -- 172.21.15.90:0/1948224663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 msgr2=0x561cd42422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.488 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5788ff9640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f57740a3dc0 0x7f57740a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 -- 172.21.15.90:0/1948224663 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd416f0d0 con 0x561cd4242820 2024-04-26T18:20:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578bfff640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f57740a3dc0 0x7f57740a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd424cba0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f577c031440 tx=0x7f577c032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce8fe7d75951e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.472+0000 7f5788ff9640 1 -- 172.21.15.90:0/551471208 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8799+0+0 (secure 0 0 0) 0x7f57800d0ae0 con 0x7f578c102210 2024-04-26T18:20:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5771ffb640 1 -- 172.21.15.90:0/1948224663 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c03f070 con 0x561cd4242820 2024-04-26T18:20:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5771ffb640 1 -- 172.21.15.90:0/1948224663 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c031760 con 0x561cd4242820 2024-04-26T18:20:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5771ffb640 1 -- 172.21.15.90:0/1948224663 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c035070 con 0x561cd4242820 2024-04-26T18:20:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 -- 172.21.15.90:0/1948224663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 msgr2=0x561cd424cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd424cba0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f577c031440 tx=0x7f577c032e90 comp rx=0 tx=0).stop 2024-04-26T18:20:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f578bfff640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f57740a3dc0 0x7f57740a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5784005fd0 tx=0x7f5784004a40 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 -- 172.21.15.90:0/1948224663 shutdown_connections 2024-04-26T18:20:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd424cba0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1948224663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 -- 172.21.15.90:0/1948224663 >> 172.21.15.90:0/1948224663 conn(0x561cd423e410 msgr2=0x561cd423e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 -- 172.21.15.90:0/1948224663 shutdown_connections 2024-04-26T18:20:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.473+0000 7f5794bf5740 1 -- 172.21.15.90:0/1948224663 wait complete. 2024-04-26T18:20:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 -- start start 2024-04-26T18:20:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 Processor -- start 2024-04-26T18:20:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4241f00 0x561cd424a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd424a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd424d0e0 0x561cd424aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561cd424fcb0 con 0x561cd4241f00 2024-04-26T18:20:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561cd412fb60 con 0x561cd424d0e0 2024-04-26T18:20:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561cd424fe30 con 0x561cd4242820 2024-04-26T18:20:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f5794bf5740 10 client.0 _ll_register_callbacks cb 0x561cd41542b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:20:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f578b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4241f00 0x561cd424a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f578b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4241f00 0x561cd424a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40788/0 (socket says 172.21.15.90:40788) 2024-04-26T18:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.474+0000 7f578b7fe640 1 -- 172.21.15.90:0/1943875957 learned_addr learned my addr 172.21.15.90:0/1943875957 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578affd640 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd424a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578bfff640 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd424d0e0 0x561cd424aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578b7fe640 1 -- 172.21.15.90:0/1943875957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 msgr2=0x561cd424a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd424a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578b7fe640 1 -- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd424d0e0 msgr2=0x561cd424aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd424d0e0 0x561cd424aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578b7fe640 1 -- 172.21.15.90:0/1943875957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd3eba210 con 0x561cd4241f00 2024-04-26T18:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4241f00 0x561cd424a050 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f577c009900 tx=0x7f577c03c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8937ca3536c7c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f576affd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f576affd640 1 -- 172.21.15.90:0/1943875957 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c03f070 con 0x561cd4241f00 2024-04-26T18:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577c031760 con 0x561cd4241f00 2024-04-26T18:20:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f576affd640 1 -- 172.21.15.90:0/1943875957 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8799+0+0 (secure 0 0 0) 0x7f577c031d20 con 0x561cd4241f00 2024-04-26T18:20:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f576affd640 1 -- 172.21.15.90:0/1943875957 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f577c054050 con 0x561cd4241f00 2024-04-26T18:20:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f5794bf5740 10 client.16170 fetch_fsmap learned FSMap version 54 2024-04-26T18:20:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561cd3f86620 con 0x561cd4241f00 2024-04-26T18:20:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.475+0000 7f576affd640 1 -- 172.21.15.90:0/1943875957 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c031760 con 0x561cd4241f00 2024-04-26T18:20:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f576affd640 1 -- 172.21.15.90:0/1943875957 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f577c0035f0 con 0x561cd4241f00 2024-04-26T18:20:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f5794bf5740 10 client.16170 fetch_fsmap finished waiting for FSMap version 54 2024-04-26T18:20:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f5794bf5740 10 client.16170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:20:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f5794bf5740 20 client.16170 populate_metadata read hostname 'smithi090' 2024-04-26T18:20:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd425d030 0x561cd425f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f5794bf5740 4 client.16170 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:20:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cd41b6070 con 0x561cd425d030 2024-04-26T18:20:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f578affd640 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd425d030 0x561cd425f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f576affd640 10 client.16170 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.476+0000 7f578affd640 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd425d030 0x561cd425f420 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c763ad1ba48e2bd server_cookie=dfe24cc584b6f960 in_seq=0 out_seq=0 2024-04-26T18:20:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f576affd640 1 -- 172.21.15.90:0/1943875957 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cd41b6070 con 0x561cd425d030 2024-04-26T18:20:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f576affd640 10 client.16170 handle_command_reply: tid=0 2024-04-26T18:20:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 client.16170 shutdown 2024-04-26T18:20:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4241f00 msgr2=0x561cd424a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4241f00 0x561cd424a050 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f577c009900 tx=0x7f577c03c6d0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 shutdown_connections 2024-04-26T18:20:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd425d030 0x561cd425f420 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd424d0e0 0x561cd424aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd424a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f576affd640 0 client.16170 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1943875957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4241f00 0x561cd424a050 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 >> 172.21.15.90:0/1943875957 conn(0x561cd423e410 msgr2=0x561cd4244190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 shutdown_connections 2024-04-26T18:20:28.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 1 -- 172.21.15.90:0/1943875957 wait complete. 2024-04-26T18:20:28.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.477+0000 7f5794bf5740 20 client.16170 trim_cache size 0 max 16384 2024-04-26T18:20:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 Processor -- start 2024-04-26T18:20:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 -- start start 2024-04-26T18:20:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd4249930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd42448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1b3030 con 0x561cd4242820 2024-04-26T18:20:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1b0900 con 0x561cd4241f00 2024-04-26T18:20:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f5794bf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1ae770 con 0x561cd424d0e0 2024-04-26T18:20:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f578affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd42448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f578affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd42448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40804/0 (socket says 172.21.15.90:40804) 2024-04-26T18:20:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f578affd640 1 -- 172.21.15.90:0/3139800899 learned_addr learned my addr 172.21.15.90:0/3139800899 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f578b7fe640 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd4249930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.492+0000 7f578bfff640 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f578affd640 1 -- 172.21.15.90:0/3139800899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 msgr2=0x561cd424a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f578affd640 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f578affd640 1 -- 172.21.15.90:0/3139800899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 msgr2=0x561cd4249930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f578affd640 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd4249930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f578affd640 1 -- 172.21.15.90:0/3139800899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd4245c10 con 0x561cd4242820 2024-04-26T18:20:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f578bfff640 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f578affd640 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd42448c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5780000c00 tx=0x7f57800d2ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb03a2eeebbfb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f57717fa640 1 -- 172.21.15.90:0/3139800899 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578009c2d0 con 0x561cd4242820 2024-04-26T18:20:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f57717fa640 1 -- 172.21.15.90:0/3139800899 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f578000de20 con 0x561cd4242820 2024-04-26T18:20:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.493+0000 7f57717fa640 1 -- 172.21.15.90:0/3139800899 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578000fa90 con 0x561cd4242820 2024-04-26T18:20:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- 172.21.15.90:0/3139800899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 msgr2=0x561cd42448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd42448c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f5780000c00 tx=0x7f57800d2ff0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- 172.21.15.90:0/3139800899 shutdown_connections 2024-04-26T18:20:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd424d0e0 0x561cd424a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd4242820 0x561cd42448c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 --2- 172.21.15.90:0/3139800899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd4249930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- 172.21.15.90:0/3139800899 >> 172.21.15.90:0/3139800899 conn(0x561cd423e410 msgr2=0x561cd424a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- 172.21.15.90:0/3139800899 shutdown_connections 2024-04-26T18:20:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- 172.21.15.90:0/3139800899 wait complete. 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- start start 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 Processor -- start 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42553b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd4250720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd424d0e0 0x561cd4250c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1aedc0 con 0x561cd424d0e0 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1b0230 con 0x561cd4241f00 2024-04-26T18:20:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f578c1af490 con 0x561cd4242820 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.494+0000 7f5794bf5740 10 client.0 _ll_register_callbacks cb 0x7f5784009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd424d0e0 0x561cd4250c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd424d0e0 0x561cd4250c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40820/0 (socket says 172.21.15.90:40820) 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 -- 172.21.15.90:0/1252133616 learned_addr learned my addr 172.21.15.90:0/1252133616 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578affd640 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd4250720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 -- 172.21.15.90:0/1252133616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 msgr2=0x561cd4250720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd4250720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 -- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 msgr2=0x561cd42553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 -- 172.21.15.90:0/1252133616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd4251460 con 0x561cd424d0e0 2024-04-26T18:20:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f578bfff640 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd424d0e0 0x561cd4250c60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5784010ac0 tx=0x7f5784004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e46cbdd7acbb46c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f576affd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f576affd640 1 -- 172.21.15.90:0/1252133616 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578402d060 con 0x561cd424d0e0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f576affd640 1 -- 172.21.15.90:0/1252133616 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8799+0+0 (secure 0 0 0) 0x7f578401d860 con 0x561cd424d0e0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f578000de20 con 0x561cd424d0e0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f576affd640 1 -- 172.21.15.90:0/1252133616 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f5784034050 con 0x561cd424d0e0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f5794bf5740 10 client.16176 fetch_fsmap learned FSMap version 54 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561cd42c3210 con 0x561cd424d0e0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.495+0000 7f576affd640 1 -- 172.21.15.90:0/1252133616 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f578000de20 con 0x561cd424d0e0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f576affd640 1 -- 172.21.15.90:0/1252133616 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 54) ==== 8338+0+0 (secure 0 0 0) 0x7f5784027070 con 0x561cd424d0e0 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f5794bf5740 10 client.16176 fetch_fsmap finished waiting for FSMap version 54 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f5794bf5740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:20:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f5794bf5740 20 client.16176 populate_metadata read hostname 'smithi090' 2024-04-26T18:20:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd42c39c0 0x561cd42c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f5794bf5740 4 client.16176 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:20:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cd41afee0 con 0x561cd42c39c0 2024-04-26T18:20:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd42c39c0 0x561cd42c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f576affd640 10 client.16176 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:28.496+0000 7f578b7fe640 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd42c39c0 0x561cd42c5de0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77dca43a0dd2e9f5 server_cookie=b379898b12c09e in_seq=0 out_seq=0 2024-04-26T18:20:28.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.76 MB/sec execute 256 sec latency 20624.079 ms 2024-04-26T18:20:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:28 smithi119 ceph-mon[32752]: pgmap v268: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 139 KiB/s wr, 29 op/s 2024-04-26T18:20:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:28 smithi167 ceph-mon[32751]: pgmap v268: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 139 KiB/s wr, 29 op/s 2024-04-26T18:20:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:28 smithi090 ceph-mon[29180]: pgmap v268: 97 pgs: 97 active+clean; 103 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 139 KiB/s wr, 29 op/s 2024-04-26T18:20:29.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.75 MB/sec execute 257 sec latency 21624.243 ms 2024-04-26T18:20:30.370 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:20:30.370 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 systemd[1]: Stopped Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:20:30.370 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.679s CPU time. 2024-04-26T18:20:30.370 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 systemd[1]: Starting Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:20:30.370 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 podman[68891]: 2024-04-26 18:20:30.215828905 +0000 UTC m=+0.025431896 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:30.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:30 smithi167 ceph-mon[32751]: pgmap v269: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 870 KiB/s wr, 55 op/s 2024-04-26T18:20:30.656 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 podman[68891]: 2024-04-26 18:20:30.370657796 +0000 UTC m=+0.180260807 container create 86d6c16e9312e2fce4974e6f820c04f6599c4f31f23292e9be4690491751985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:20:30.656 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 podman[68891]: 2024-04-26 18:20:30.656391436 +0000 UTC m=+0.465994404 container init 86d6c16e9312e2fce4974e6f820c04f6599c4f31f23292e9be4690491751985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-26T18:20:30.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.73 MB/sec execute 258 sec latency 22624.322 ms 2024-04-26T18:20:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:30 smithi119 ceph-mon[32752]: pgmap v269: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 870 KiB/s wr, 55 op/s 2024-04-26T18:20:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:30 smithi090 ceph-mon[29180]: pgmap v269: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 870 KiB/s wr, 55 op/s 2024-04-26T18:20:30.932 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 podman[68891]: 2024-04-26 18:20:30.659262824 +0000 UTC m=+0.468865799 container start 86d6c16e9312e2fce4974e6f820c04f6599c4f31f23292e9be4690491751985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T18:20:30.932 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 ceph-mds[68911]: -- 172.21.15.167:0/2213801289 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d73c38fa40 con 0x55d73c362800 2024-04-26T18:20:30.932 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: starting mds.i at 2024-04-26T18:20:30.932 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 bash[68891]: 86d6c16e9312e2fce4974e6f820c04f6599c4f31f23292e9be4690491751985f 2024-04-26T18:20:30.933 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:20:30 smithi167 systemd[1]: Started Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:20:31.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.72 MB/sec execute 259 sec latency 23624.443 ms 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: daemon mds.i restarted 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:boot 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:20:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:31 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: daemon mds.i restarted 2024-04-26T18:20:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:20:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:20:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:boot 2024-04-26T18:20:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:20:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:20:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:20:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:31 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: daemon mds.i restarted 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:boot 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:31 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:32.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.71 MB/sec execute 260 sec latency 24624.552 ms 2024-04-26T18:20:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:32 smithi119 ceph-mon[32752]: pgmap v271: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 883 KiB/s wr, 33 op/s 2024-04-26T18:20:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:32 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:resolve 2024-04-26T18:20:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:32 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:32 smithi167 ceph-mon[32751]: pgmap v271: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 883 KiB/s wr, 33 op/s 2024-04-26T18:20:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:32 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:resolve 2024-04-26T18:20:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:32 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:32 smithi090 ceph-mon[29180]: pgmap v271: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 883 KiB/s wr, 33 op/s 2024-04-26T18:20:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:32 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:resolve 2024-04-26T18:20:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:32 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:33.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.69 MB/sec execute 261 sec latency 25624.660 ms 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f576affd640 1 -- 172.21.15.90:0/1252133616 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cd41afee0 con 0x561cd42c39c0 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f576affd640 10 client.16176 handle_command_reply: tid=0 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 client.16176 shutdown 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd424d0e0 msgr2=0x561cd4250c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd424d0e0 0x561cd4250c60 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f5784010ac0 tx=0x7f5784004ab0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 shutdown_connections 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561cd42c39c0 0x561cd42c5de0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561cd424d0e0 0x561cd4250c60 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561cd4242820 0x561cd4250720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f576affd640 0 client.16176 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 --2- 172.21.15.90:0/1252133616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561cd4241f00 0x561cd42553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 >> 172.21.15.90:0/1252133616 conn(0x561cd423e410 msgr2=0x561cd423d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 shutdown_connections 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 1 -- 172.21.15.90:0/1252133616 wait complete. 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.562+0000 7f5794bf5740 20 client.16176 trim_cache size 0 max 16384 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:20:34.564 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:20:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f57740a3dc0 msgr2=0x7f57740a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f57740a3dc0 0x7f57740a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5784005fd0 tx=0x7f5784004a40 comp rx=0 tx=0).stop 2024-04-26T18:20:34.565 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c102210 msgr2=0x7f578c19b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c102210 0x7f578c19b700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f578000c920 tx=0x7f578000cde0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 shutdown_connections 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f57740a3dc0 0x7f57740a6280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f578c107860 0x7f578c19fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f578c102210 0x7f578c19b700 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 --2- 172.21.15.90:0/551471208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f578c101850 0x7f578c19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 >> 172.21.15.90:0/551471208 conn(0x7f578c077bb0 msgr2=0x7f578c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 shutdown_connections 2024-04-26T18:20:34.566 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:34.563+0000 7f5791e08640 1 -- 172.21.15.90:0/551471208 wait complete. 2024-04-26T18:20:34.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412087 3.68 MB/sec execute 262 sec latency 26624.756 ms 2024-04-26T18:20:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:34 smithi119 ceph-mon[32752]: pgmap v272: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:34 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999976 2024-04-26T18:20:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:34 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:20:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:reconnect 2024-04-26T18:20:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:34 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:34 smithi167 ceph-mon[32751]: pgmap v272: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:34 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999976 2024-04-26T18:20:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:34 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:20:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:reconnect 2024-04-26T18:20:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:34 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:34 smithi090 ceph-mon[29180]: pgmap v272: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:34 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999976 2024-04-26T18:20:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:34 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:20:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:reconnect 2024-04-26T18:20:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:34 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:35.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 412214 3.66 MB/sec execute 263 sec latency 27529.991 ms 2024-04-26T18:20:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:rejoin 2024-04-26T18:20:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:35 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:35 smithi119 ceph-mon[32752]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:20:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:rejoin 2024-04-26T18:20:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:35 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:35 smithi167 ceph-mon[32751]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:20:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:rejoin 2024-04-26T18:20:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:35 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:35 smithi090 ceph-mon[29180]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:20:36.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 415038 3.72 MB/sec execute 264 sec latency 12.082 ms 2024-04-26T18:20:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:36 smithi119 ceph-mon[32752]: pgmap v273: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:36 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:36 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:20:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:36 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:active 2024-04-26T18:20:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:36 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:36 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:36 smithi167 ceph-mon[32751]: pgmap v273: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:36 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:36 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:20:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:36 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:active 2024-04-26T18:20:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:36 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:36 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:36 smithi090 ceph-mon[29180]: pgmap v273: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:36 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:36 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:20:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:36 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/845543419,v1:172.21.15.119:6837/845543419] up:active 2024-04-26T18:20:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:36 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:36 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:37.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 416326 3.72 MB/sec execute 265 sec latency 14.748 ms 2024-04-26T18:20:38.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:20:38 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:20:38.210+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:20:38.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 418895 3.76 MB/sec execute 266 sec latency 16.722 ms 2024-04-26T18:20:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:38 smithi119 ceph-mon[32752]: pgmap v274: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:38 smithi167 ceph-mon[32751]: pgmap v274: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:38 smithi090 ceph-mon[29180]: pgmap v274: 97 pgs: 97 active+clean; 102 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 891 KiB/s wr, 38 op/s 2024-04-26T18:20:39.683 INFO:tasks.workunit.client.0.smithi090.stdout: 1 420724 3.77 MB/sec execute 267 sec latency 16.271 ms 2024-04-26T18:20:40.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.81 MB/sec execute 268 sec latency 81.064 ms 2024-04-26T18:20:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:40 smithi167 ceph-mon[32751]: pgmap v275: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 636 KiB/s wr, 97 op/s 2024-04-26T18:20:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:40 smithi090 ceph-mon[29180]: pgmap v275: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 636 KiB/s wr, 97 op/s 2024-04-26T18:20:40.949 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-mon[32752]: pgmap v275: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 636 KiB/s wr, 97 op/s 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff7e5e65700 time 2024-04-26T18:20:40.602532+0000 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff7f048e3dd] 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.950 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: clone() 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.602+0000 7ff7e5e65700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff7e5e65700 time 2024-04-26T18:20:40.602532+0000 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:40.951 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.952 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.952 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff7f048e3dd] 2024-04-26T18:20:40.952 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.952 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.952 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: clone() 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: *** Caught signal (Aborted) ** 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: in thread 7ff7e5e65700 thread_name:ms_dispatch 2024-04-26T18:20:40.953 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7ef216d20] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: gsignal() 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: abort() 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff7f048e437] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.954 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 16: clone() 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.604+0000 7ff7e5e65700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: in thread 7ff7e5e65700 thread_name:ms_dispatch 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7ef216d20] 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: gsignal() 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: abort() 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff7f048e437] 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.955 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 16: clone() 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: -1> 2024-04-26T18:20:40.602+0000 7ff7e5e65700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff7e5e65700 time 2024-04-26T18:20:40.602532+0000 2024-04-26T18:20:40.956 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff7f048e3dd] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.957 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: clone() 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 0> 2024-04-26T18:20:40.604+0000 7ff7e5e65700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: in thread 7ff7e5e65700 thread_name:ms_dispatch 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7ef216d20] 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: gsignal() 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: abort() 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff7f048e437] 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.958 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 16: clone() 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: -1> 2024-04-26T18:20:40.602+0000 7ff7e5e65700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff7e5e65700 time 2024-04-26T18:20:40.602532+0000 2024-04-26T18:20:40.959 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff7f048e3dd] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.960 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: clone() 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 0> 2024-04-26T18:20:40.604+0000 7ff7e5e65700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: in thread 7ff7e5e65700 thread_name:ms_dispatch 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff7ef216d20] 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2: gsignal() 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 3: abort() 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff7f048e437] 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff7f048e5a3] 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563d5a367822] 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563d5a2d7598] 2024-04-26T18:20:40.961 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563d5a2d9c17] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563d5a1d2d0b] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563d5a1d32ac] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563d5a1bbfad] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff7f0722e20] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 13: (DispatchQueue::entry()+0x51f) [0x7ff7f071ffdf] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff7f07f2fc1] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff7ef20c1ca] 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 16: clone() 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68329]: 2024-04-26T18:20:40.962 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:40 smithi119 podman[68518]: 2024-04-26 18:20:40.664558603 +0000 UTC m=+0.020047621 container died f22de69f48d4db9c4f21f2dc56aa81c54531d4a69868c691f603f38ed0523043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T18:20:41.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:41 smithi119 podman[68518]: 2024-04-26 18:20:41.261347138 +0000 UTC m=+0.616836187 container remove f22de69f48d4db9c4f21f2dc56aa81c54531d4a69868c691f603f38ed0523043 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:20:41.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:41 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:20:41.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:41 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-26T18:20:41.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:41 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.990s CPU time. 2024-04-26T18:20:41.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.79 MB/sec execute 269 sec latency 1081.149 ms 2024-04-26T18:20:42.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.78 MB/sec execute 270 sec latency 2081.220 ms 2024-04-26T18:20:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:42 smithi167 ceph-mon[32751]: pgmap v276: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 599 KiB/s wr, 91 op/s 2024-04-26T18:20:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:42 smithi090 ceph-mon[29180]: pgmap v276: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 599 KiB/s wr, 91 op/s 2024-04-26T18:20:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:42 smithi119 ceph-mon[32752]: pgmap v276: 97 pgs: 97 active+clean; 107 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 599 KiB/s wr, 91 op/s 2024-04-26T18:20:43.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.77 MB/sec execute 271 sec latency 3081.329 ms 2024-04-26T18:20:44.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.75 MB/sec execute 272 sec latency 4081.380 ms 2024-04-26T18:20:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:44 smithi167 ceph-mon[32751]: pgmap v277: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 944 KiB/s wr, 134 op/s 2024-04-26T18:20:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:44 smithi090 ceph-mon[29180]: pgmap v277: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 944 KiB/s wr, 134 op/s 2024-04-26T18:20:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:44 smithi119 ceph-mon[32752]: pgmap v277: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 944 KiB/s wr, 134 op/s 2024-04-26T18:20:45.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.74 MB/sec execute 273 sec latency 5081.458 ms 2024-04-26T18:20:46.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.73 MB/sec execute 274 sec latency 6081.570 ms 2024-04-26T18:20:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:46 smithi167 ceph-mon[32751]: pgmap v278: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 933 KiB/s wr, 127 op/s 2024-04-26T18:20:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:46 smithi090 ceph-mon[29180]: pgmap v278: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 933 KiB/s wr, 127 op/s 2024-04-26T18:20:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:46 smithi119 ceph-mon[32752]: pgmap v278: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 933 KiB/s wr, 127 op/s 2024-04-26T18:20:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 -- 172.21.15.90:0/4263187250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f781410cf00 msgr2=0x7f781410f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 --2- 172.21.15.90:0/4263187250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f781410cf00 0x7f781410f3a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f780c009ca0 tx=0x7f780c02a9e0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 -- 172.21.15.90:0/4263187250 shutdown_connections 2024-04-26T18:20:47.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 --2- 172.21.15.90:0/4263187250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f781410cf00 0x7f781410f3a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 --2- 172.21.15.90:0/4263187250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78141044e0 0x7f781410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 --2- 172.21.15.90:0/4263187250 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7814103bc0 0x7f7814103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 -- 172.21.15.90:0/4263187250 >> 172.21.15.90:0/4263187250 conn(0x7f78140789b0 msgr2=0x7f7814078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 -- 172.21.15.90:0/4263187250 shutdown_connections 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.653+0000 7f7818c99640 1 -- 172.21.15.90:0/4263187250 wait complete. 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 Processor -- start 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 -- start start 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7814103bc0 0x7f7814199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78141044e0 0x7f7814199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f781410cf00 0x7f781419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7814112bb0 con 0x7f7814103bc0 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7814112a30 con 0x7f78141044e0 2024-04-26T18:20:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f7818c99640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7814112d30 con 0x7f781410cf00 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f781259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7814103bc0 0x7f7814199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f781259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7814103bc0 0x7f7814199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60530/0 (socket says 172.21.15.90:60530) 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f781259c640 1 -- 172.21.15.90:0/2796773452 learned_addr learned my addr 172.21.15.90:0/2796773452 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.654+0000 7f780bfff640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78141044e0 0x7f7814199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f7812d9d640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f781410cf00 0x7f781419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f781259c640 1 -- 172.21.15.90:0/2796773452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f781410cf00 msgr2=0x7f781419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f781259c640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f781410cf00 0x7f781419d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f781259c640 1 -- 172.21.15.90:0/2796773452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78141044e0 msgr2=0x7f7814199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f781259c640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78141044e0 0x7f7814199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f781259c640 1 -- 172.21.15.90:0/2796773452 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f781419dff0 con 0x7f7814103bc0 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f7812d9d640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f781410cf00 0x7f781419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f781259c640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7814103bc0 0x7f7814199050 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f780400d950 tx=0x7f780400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f780b7fe640 1 -- 172.21.15.90:0/2796773452 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804017030 con 0x7f7814103bc0 2024-04-26T18:20:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f781419e2e0 con 0x7f7814103bc0 2024-04-26T18:20:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f780b7fe640 1 -- 172.21.15.90:0/2796773452 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7804004020 con 0x7f7814103bc0 2024-04-26T18:20:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f780b7fe640 1 -- 172.21.15.90:0/2796773452 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78040047a0 con 0x7f7814103bc0 2024-04-26T18:20:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.655+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78141a5b70 con 0x7f7814103bc0 2024-04-26T18:20:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 Processor -- start 2024-04-26T18:20:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f780b7fe640 1 -- 172.21.15.90:0/2796773452 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f7804009a30 con 0x7f7814103bc0 2024-04-26T18:20:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 -- start start 2024-04-26T18:20:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b0e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b18ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b1b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5603c4a74460 con 0x5603c4b190e0 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5603c4a2e590 con 0x5603c4b0e820 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f781b9b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5603c4a70420 con 0x5603c4b0df00 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f780bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b1b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f780bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b1b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60544/0 (socket says 172.21.15.90:60544) 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.658+0000 7f780bfff640 1 -- 172.21.15.90:0/1573203874 learned_addr learned my addr 172.21.15.90:0/1573203874 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f7812d9d640 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b0e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f781259c640 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b18ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780b7fe640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78000a3dc0 0x7f78000a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780bfff640 1 -- 172.21.15.90:0/1573203874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 msgr2=0x5603c4b0e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780bfff640 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b0e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780bfff640 1 -- 172.21.15.90:0/1573203874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 msgr2=0x5603c4b18ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780bfff640 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b18ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780bfff640 1 -- 172.21.15.90:0/1573203874 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c4a3b0d0 con 0x5603c4b190e0 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f7812d9d640 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b0e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f7812d9d640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78000a3dc0 0x7f78000a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780b7fe640 1 -- 172.21.15.90:0/2796773452 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8841+0+0 (secure 0 0 0) 0x7f78040d03a0 con 0x7f7814103bc0 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f780bfff640 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b1b4d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00bc10 tx=0x7f77fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7a3a189310a793 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f7808ff9640 1 -- 172.21.15.90:0/1573203874 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc005020 con 0x5603c4b190e0 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f7808ff9640 1 -- 172.21.15.90:0/1573203874 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc0047a0 con 0x5603c4b190e0 2024-04-26T18:20:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f7812d9d640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78000a3dc0 0x7f78000a6280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f780c009a60 tx=0x7f780c0095c0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.659+0000 7f7808ff9640 1 -- 172.21.15.90:0/1573203874 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77fc011680 con 0x5603c4b190e0 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 -- 172.21.15.90:0/1573203874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 msgr2=0x5603c4b1b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b1b4d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00bc10 tx=0x7f77fc0107a0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 -- 172.21.15.90:0/1573203874 shutdown_connections 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b1b4d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b18ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1573203874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b0e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 -- 172.21.15.90:0/1573203874 >> 172.21.15.90:0/1573203874 conn(0x5603c4b0a410 msgr2=0x5603c4b0a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 -- 172.21.15.90:0/1573203874 shutdown_connections 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 -- 172.21.15.90:0/1573203874 wait complete. 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.660+0000 7f781b9b8740 1 -- start start 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 1 Processor -- start 2024-04-26T18:20:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b16050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b16ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5603c4b1bd10 con 0x5603c4b190e0 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5603c49fbb60 con 0x5603c4b0e820 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5603c4b1be90 con 0x5603c4b0df00 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781b9b8740 10 client.0 _ll_register_callbacks cb 0x5603c4a202b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b16ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b16ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60554/0 (socket says 172.21.15.90:60554) 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 -- 172.21.15.90:0/812524128 learned_addr learned my addr 172.21.15.90:0/812524128 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f780bfff640 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f781259c640 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b16050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 -- 172.21.15.90:0/812524128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 msgr2=0x5603c4b16050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b16050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 -- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 msgr2=0x5603c4b16590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 -- 172.21.15.90:0/812524128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c4786210 con 0x5603c4b190e0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f7812d9d640 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b16ad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f780c02f040 tx=0x7f780c02a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b17598b27b0d46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f77f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.661+0000 7f77f9ffb640 1 -- 172.21.15.90:0/812524128 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c033040 con 0x5603c4b190e0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f77f9ffb640 1 -- 172.21.15.90:0/812524128 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8841+0+0 (secure 0 0 0) 0x7f780c0047c0 con 0x5603c4b190e0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77fc0047a0 con 0x5603c4b190e0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f77f9ffb640 1 -- 172.21.15.90:0/812524128 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f780c059050 con 0x5603c4b190e0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 10 client.16191 fetch_fsmap learned FSMap version 61 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5603c4852620 con 0x5603c4b190e0 2024-04-26T18:20:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f77f9ffb640 1 -- 172.21.15.90:0/812524128 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77fc0047a0 con 0x5603c4b190e0 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f77f9ffb640 1 -- 172.21.15.90:0/812524128 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 61) ==== 8338+0+0 (secure 0 0 0) 0x7f780c038610 con 0x5603c4b190e0 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 10 client.16191 fetch_fsmap finished waiting for FSMap version 61 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 20 client.16191 populate_metadata read hostname 'smithi090' 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b29250 0x5603c4b2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 4 client.16191 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.662+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603c4a82070 con 0x5603c4b29250 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.663+0000 7f781259c640 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b29250 0x5603c4b2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.663+0000 7f77f9ffb640 10 client.16191 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.663+0000 7f781259c640 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b29250 0x5603c4b2b660 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81d3c5e4e5aeaed server_cookie=5319c3ffb4c003d7 in_seq=0 out_seq=0 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f77f9ffb640 1 -- 172.21.15.90:0/812524128 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603c4a82070 con 0x5603c4b29250 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f77f9ffb640 10 client.16191 handle_command_reply: tid=0 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 client.16191 shutdown 2024-04-26T18:20:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 msgr2=0x5603c4b16ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b16ad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f780c02f040 tx=0x7f780c02a9e0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 shutdown_connections 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b29250 0x5603c4b2b660 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b16ad0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f77f9ffb640 0 client.16191 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 --2- 172.21.15.90:0/812524128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b16050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 >> 172.21.15.90:0/812524128 conn(0x5603c4b0a410 msgr2=0x5603c4b095b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 shutdown_connections 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 1 -- 172.21.15.90:0/812524128 wait complete. 2024-04-26T18:20:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.664+0000 7f781b9b8740 20 client.16191 trim_cache size 0 max 16384 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 Processor -- start 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 -- start start 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b30f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78141b0ef0 con 0x5603c4b190e0 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f78141ae7c0 con 0x5603c4b0e820 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f781b9b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78141ac680 con 0x5603c4b0df00 2024-04-26T18:20:47.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f7812d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b30f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f7812d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b30f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60570/0 (socket says 172.21.15.90:60570) 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.679+0000 7f7812d9d640 1 -- 172.21.15.90:0/2136846062 learned_addr learned my addr 172.21.15.90:0/2136846062 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f780bfff640 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f781259c640 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f7812d9d640 1 -- 172.21.15.90:0/2136846062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 msgr2=0x5603c4b108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f7812d9d640 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f7812d9d640 1 -- 172.21.15.90:0/2136846062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 msgr2=0x5603c4b16720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f7812d9d640 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f7812d9d640 1 -- 172.21.15.90:0/2136846062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c4b1b840 con 0x5603c4b190e0 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f7812d9d640 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b30f70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f780c02ea20 tx=0x7f780c005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4247084981f095c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f77fbfff640 1 -- 172.21.15.90:0/2136846062 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c062070 con 0x5603c4b190e0 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f77fbfff640 1 -- 172.21.15.90:0/2136846062 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780c038650 con 0x5603c4b190e0 2024-04-26T18:20:47.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.680+0000 7f77fbfff640 1 -- 172.21.15.90:0/2136846062 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c033070 con 0x5603c4b190e0 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- 172.21.15.90:0/2136846062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 msgr2=0x5603c4b30f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b30f70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f780c02ea20 tx=0x7f780c005d80 comp rx=0 tx=0).stop 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- 172.21.15.90:0/2136846062 shutdown_connections 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b30f70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b16720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 --2- 172.21.15.90:0/2136846062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- 172.21.15.90:0/2136846062 >> 172.21.15.90:0/2136846062 conn(0x5603c4b0a410 msgr2=0x5603c4b16010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- 172.21.15.90:0/2136846062 shutdown_connections 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- 172.21.15.90:0/2136846062 wait complete. 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- start start 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 Processor -- start 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b13150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.689 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b13690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b8e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f78141accd0 con 0x5603c4b190e0 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f78141ae0f0 con 0x5603c4b0e820 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78141ad350 con 0x5603c4b0df00 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f781b9b8740 10 client.0 _ll_register_callbacks cb 0x7f780c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.681+0000 7f7812d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b8e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b8e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60578/0 (socket says 172.21.15.90:60578) 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 -- 172.21.15.90:0/1342319050 learned_addr learned my addr 172.21.15.90:0/1342319050 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f781259c640 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b13150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.690 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f780bfff640 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b13690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 -- 172.21.15.90:0/1342319050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 msgr2=0x5603c4b13150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b13150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 -- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 msgr2=0x5603c4b13690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b13690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 -- 172.21.15.90:0/1342319050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c4b8ed20 con 0x5603c4b190e0 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f7812d9d640 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b8e7e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f780c009ca0 tx=0x7f780c02f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a5258821f41b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f77f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f77f9ffb640 1 -- 172.21.15.90:0/1342319050 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c062040 con 0x5603c4b190e0 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f77f9ffb640 1 -- 172.21.15.90:0/1342319050 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8841+0+0 (secure 0 0 0) 0x7f780c02fba0 con 0x5603c4b190e0 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780c038650 con 0x5603c4b190e0 2024-04-26T18:20:47.691 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f77f9ffb640 1 -- 172.21.15.90:0/1342319050 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f780c071050 con 0x5603c4b190e0 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f781b9b8740 10 client.16197 fetch_fsmap learned FSMap version 61 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5603c4b8fef0 con 0x5603c4b190e0 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.682+0000 7f77f9ffb640 1 -- 172.21.15.90:0/1342319050 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780c038650 con 0x5603c4b190e0 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f77f9ffb640 1 -- 172.21.15.90:0/1342319050 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 61) ==== 8338+0+0 (secure 0 0 0) 0x7f780c0437c0 con 0x5603c4b190e0 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781b9b8740 10 client.16197 fetch_fsmap finished waiting for FSMap version 61 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781b9b8740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781b9b8740 20 client.16197 populate_metadata read hostname 'smithi090' 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b90570 0x5603c4b92990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781b9b8740 4 client.16197 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603c4a7bee0 con 0x5603c4b90570 2024-04-26T18:20:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781259c640 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b90570 0x5603c4b92990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f77f9ffb640 10 client.16197 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:47.683+0000 7f781259c640 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b90570 0x5603c4b92990 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266697f6196b79cc server_cookie=5cc10fbe9adeaff4 in_seq=0 out_seq=0 2024-04-26T18:20:47.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.71 MB/sec execute 275 sec latency 7081.602 ms 2024-04-26T18:20:48.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.70 MB/sec execute 276 sec latency 8081.675 ms 2024-04-26T18:20:48.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:48 smithi119 ceph-mon[32752]: pgmap v279: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 933 KiB/s wr, 127 op/s 2024-04-26T18:20:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:48 smithi090 ceph-mon[29180]: pgmap v279: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 933 KiB/s wr, 127 op/s 2024-04-26T18:20:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:48 smithi167 ceph-mon[32751]: pgmap v279: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 933 KiB/s wr, 127 op/s 2024-04-26T18:20:49.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.68 MB/sec execute 277 sec latency 9081.775 ms 2024-04-26T18:20:50.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.67 MB/sec execute 278 sec latency 10081.837 ms 2024-04-26T18:20:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:50 smithi090 ceph-mon[29180]: pgmap v280: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 952 KiB/s wr, 150 op/s 2024-04-26T18:20:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:50 smithi119 ceph-mon[32752]: pgmap v280: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 952 KiB/s wr, 150 op/s 2024-04-26T18:20:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:50 smithi167 ceph-mon[32751]: pgmap v280: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 952 KiB/s wr, 150 op/s 2024-04-26T18:20:51.684 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.66 MB/sec execute 279 sec latency 11081.954 ms 2024-04-26T18:20:51.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:51 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:20:51.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:51 smithi119 systemd[1]: Stopped Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:20:51.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:51 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.990s CPU time. 2024-04-26T18:20:51.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:51 smithi119 systemd[1]: Starting Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:20:51.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:51 smithi119 podman[68616]: 2024-04-26 18:20:51.683244427 +0000 UTC m=+0.018008239 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:20:51.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:51 smithi119 podman[68616]: 2024-04-26 18:20:51.811904751 +0000 UTC m=+0.146668547 container create 53c1c4cfa85ffd00debb09dd3976b8665e81b5c4633a1a769f4575a4ac313120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:20:52.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:52 smithi119 podman[68616]: 2024-04-26 18:20:52.074460011 +0000 UTC m=+0.409223810 container init 53c1c4cfa85ffd00debb09dd3976b8665e81b5c4633a1a769f4575a4ac313120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T18:20:52.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:52 smithi119 podman[68616]: 2024-04-26 18:20:52.077136271 +0000 UTC m=+0.411900075 container start 53c1c4cfa85ffd00debb09dd3976b8665e81b5c4633a1a769f4575a4ac313120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T18:20:52.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mds[68634]: -- 172.21.15.119:0/1896409770 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ff282da40 con 0x560ff2800800 2024-04-26T18:20:52.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: starting mds.h at 2024-04-26T18:20:52.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:52 smithi119 bash[68616]: 53c1c4cfa85ffd00debb09dd3976b8665e81b5c4633a1a769f4575a4ac313120 2024-04-26T18:20:52.554 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:20:52 smithi119 systemd[1]: Started Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:20:52.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.65 MB/sec execute 280 sec latency 12082.093 ms 2024-04-26T18:20:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: pgmap v281: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 433 KiB/s wr, 76 op/s 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: daemon mds.h restarted 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:boot 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:20:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:52 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: pgmap v281: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 433 KiB/s wr, 76 op/s 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: daemon mds.h restarted 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:boot 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:20:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:52 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: pgmap v281: 97 pgs: 97 active+clean; 111 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 727 KiB/s rd, 433 KiB/s wr, 76 op/s 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: daemon mds.h restarted 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:boot 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:20:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:52 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:53.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.63 MB/sec execute 281 sec latency 13082.252 ms 2024-04-26T18:20:54.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422581 3.62 MB/sec execute 282 sec latency 14082.309 ms 2024-04-26T18:20:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:54 smithi090 ceph-mon[29180]: pgmap v283: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:54 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:resolve 2024-04-26T18:20:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:54 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:54 smithi119 ceph-mon[32752]: pgmap v283: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:54 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:resolve 2024-04-26T18:20:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:54 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:54 smithi167 ceph-mon[32751]: pgmap v283: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:54 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:resolve 2024-04-26T18:20:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:54 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f77f9ffb640 1 -- 172.21.15.90:0/1342319050 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603c4a7bee0 con 0x5603c4b90570 2024-04-26T18:20:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f77f9ffb640 10 client.16197 handle_command_reply: tid=0 2024-04-26T18:20:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 client.16197 shutdown 2024-04-26T18:20:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 msgr2=0x5603c4b8e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b8e7e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f780c009ca0 tx=0x7f780c02f980 comp rx=0 tx=0).stop 2024-04-26T18:20:55.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 shutdown_connections 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5603c4b90570 0x5603c4b92990 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5603c4b190e0 0x5603c4b8e7e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5603c4b0e820 0x5603c4b13690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f77f9ffb640 0 client.16197 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 --2- 172.21.15.90:0/1342319050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c4b0df00 0x5603c4b13150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 >> 172.21.15.90:0/1342319050 conn(0x5603c4b0a410 msgr2=0x5603c4b168a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 shutdown_connections 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 1 -- 172.21.15.90:0/1342319050 wait complete. 2024-04-26T18:20:55.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.615+0000 7f781b9b8740 20 client.16197 trim_cache size 0 max 16384 2024-04-26T18:20:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:20:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:20:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:20:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:20:55.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78000a3dc0 msgr2=0x7f78000a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78000a3dc0 0x7f78000a6280 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f780c009a60 tx=0x7f780c0095c0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7814103bc0 msgr2=0x7f7814199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7814103bc0 0x7f7814199050 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f780400d950 tx=0x7f780400de10 comp rx=0 tx=0).stop 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 shutdown_connections 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f78000a3dc0 0x7f78000a6280 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f781410cf00 0x7f781419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78141044e0 0x7f7814199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 --2- 172.21.15.90:0/2796773452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7814103bc0 0x7f7814199050 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 >> 172.21.15.90:0/2796773452 conn(0x7f78140789b0 msgr2=0x7f78140fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 shutdown_connections 2024-04-26T18:20:55.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:20:55.616+0000 7f7818c99640 1 -- 172.21.15.90:0/2796773452 wait complete. 2024-04-26T18:20:55.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422656 3.61 MB/sec execute 283 sec latency 15001.906 ms 2024-04-26T18:20:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:55 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:20:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:55 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:20:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:reconnect 2024-04-26T18:20:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:55 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:rejoin 2024-04-26T18:20:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:55 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:55 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:20:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:55 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:20:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:reconnect 2024-04-26T18:20:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:55 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:rejoin 2024-04-26T18:20:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:55 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:55 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:20:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:55 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:20:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:reconnect 2024-04-26T18:20:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:55 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:rejoin 2024-04-26T18:20:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:55 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:56.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 422734 3.59 MB/sec execute 284 sec latency 932.334 ms 2024-04-26T18:20:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:56 smithi090 ceph-mon[29180]: pgmap v284: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:56 smithi090 ceph-mon[29180]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:20:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:56 smithi119 ceph-mon[32752]: pgmap v284: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:56 smithi119 ceph-mon[32752]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:20:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:56 smithi167 ceph-mon[32751]: pgmap v284: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:56 smithi167 ceph-mon[32751]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:20:57.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 425104 3.63 MB/sec execute 285 sec latency 45.961 ms 2024-04-26T18:20:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:57 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:57 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:20:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:57 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:active 2024-04-26T18:20:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:57 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:57 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:57 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:57 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:20:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:57 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:active 2024-04-26T18:20:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:57 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:57 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:57 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:20:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:57 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:20:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:57 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/4026544385,v1:172.21.15.167:6839/4026544385] up:active 2024-04-26T18:20:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:57 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:20:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:57 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:20:58.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:20:58 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:20:58.211+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:20:58.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 426587 3.64 MB/sec execute 286 sec latency 14.201 ms 2024-04-26T18:20:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:20:58 smithi090 ceph-mon[29180]: pgmap v285: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:20:58 smithi119 ceph-mon[32752]: pgmap v285: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:20:58 smithi167 ceph-mon[32751]: pgmap v285: 97 pgs: 97 active+clean; 114 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 987 KiB/s wr, 56 op/s 2024-04-26T18:20:59.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 429455 3.69 MB/sec execute 287 sec latency 17.233 ms 2024-04-26T18:21:00.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 430880 3.69 MB/sec execute 288 sec latency 12.948 ms 2024-04-26T18:21:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:00 smithi090 ceph-mon[29180]: pgmap v286: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-26T18:21:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:00 smithi119 ceph-mon[32752]: pgmap v286: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-26T18:21:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:00 smithi167 ceph-mon[32751]: pgmap v286: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-26T18:21:01.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 432881 3.73 MB/sec execute 289 sec latency 25.746 ms 2024-04-26T18:21:02.685 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.72 MB/sec execute 290 sec latency 436.612 ms 2024-04-26T18:21:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:02 smithi090 ceph-mon[29180]: pgmap v287: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-26T18:21:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:02 smithi119 ceph-mon[32752]: pgmap v287: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-26T18:21:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:02 smithi167 ceph-mon[32751]: pgmap v287: 97 pgs: 97 active+clean; 112 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.3 MiB/s wr, 95 op/s 2024-04-26T18:21:03.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.70 MB/sec execute 291 sec latency 1436.729 ms 2024-04-26T18:21:04.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.69 MB/sec execute 292 sec latency 2436.772 ms 2024-04-26T18:21:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:04 smithi090 ceph-mon[29180]: pgmap v288: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2024-04-26T18:21:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:04 smithi119 ceph-mon[32752]: pgmap v288: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2024-04-26T18:21:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:04 smithi167 ceph-mon[32751]: pgmap v288: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 208 op/s 2024-04-26T18:21:05.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.68 MB/sec execute 293 sec latency 3436.820 ms 2024-04-26T18:21:06.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.67 MB/sec execute 294 sec latency 4436.958 ms 2024-04-26T18:21:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:06 smithi119 ceph-mon[32752]: pgmap v289: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-26T18:21:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:06 smithi119 ceph-mon[32752]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:21:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:06 smithi167 ceph-mon[32751]: pgmap v289: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-26T18:21:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:06 smithi167 ceph-mon[32751]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:21:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:06 smithi090 ceph-mon[29180]: pgmap v289: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-04-26T18:21:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:06 smithi090 ceph-mon[29180]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:21:07.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.65 MB/sec execute 295 sec latency 5437.035 ms 2024-04-26T18:21:08.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.64 MB/sec execute 296 sec latency 6437.107 ms 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.714+0000 7fb2efc6d640 1 -- 172.21.15.90:0/1699160820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e8075680 msgr2=0x7fb2e810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.714+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/1699160820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e8075680 0x7fb2e810ec30 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009900 tx=0x7fb2dc02a480 comp rx=0 tx=0).stop 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.714+0000 7fb2efc6d640 1 -- 172.21.15.90:0/1699160820 shutdown_connections 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.714+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/1699160820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e810f170 0x7fb2e81115c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.714+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/1699160820 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e8075680 0x7fb2e810ec30 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.714+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/1699160820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e8074d60 0x7fb2e8075140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.714+0000 7fb2efc6d640 1 -- 172.21.15.90:0/1699160820 >> 172.21.15.90:0/1699160820 conn(0x7fb2e80fe400 msgr2=0x7fb2e8100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 -- 172.21.15.90:0/1699160820 shutdown_connections 2024-04-26T18:21:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 -- 172.21.15.90:0/1699160820 wait complete. 2024-04-26T18:21:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 Processor -- start 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 -- start start 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e8074d60 0x7fb2e819b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e8075680 0x7fb2e819b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e810f170 0x7fb2e819faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8114d80 con 0x7fb2e810f170 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8114c00 con 0x7fb2e8075680 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2efc6d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8114f00 con 0x7fb2e8074d60 2024-04-26T18:21:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2ee20a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e810f170 0x7fb2e819faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2ee20a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e810f170 0x7fb2e819faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60230/0 (socket says 172.21.15.90:60230) 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.715+0000 7fb2ee20a640 1 -- 172.21.15.90:0/598723488 learned_addr learned my addr 172.21.15.90:0/598723488 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2eda09640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e8074d60 0x7fb2e819b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ed208640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e8075680 0x7fb2e819b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ee20a640 1 -- 172.21.15.90:0/598723488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e8074d60 msgr2=0x7fb2e819b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ee20a640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e8074d60 0x7fb2e819b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ee20a640 1 -- 172.21.15.90:0/598723488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e8075680 msgr2=0x7fb2e819b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ee20a640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e8075680 0x7fb2e819b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ee20a640 1 -- 172.21.15.90:0/598723488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e81a0220 con 0x7fb2e810f170 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ed208640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e8075680 0x7fb2e819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2ee20a640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e810f170 0x7fb2e819faf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009fd0 tx=0x7fb2e000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2daffd640 1 -- 172.21.15.90:0/598723488 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0017070 con 0x7fb2e810f170 2024-04-26T18:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2daffd640 1 -- 172.21.15.90:0/598723488 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2e0003e20 con 0x7fb2e810f170 2024-04-26T18:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e81a0510 con 0x7fb2e810f170 2024-04-26T18:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2daffd640 1 -- 172.21.15.90:0/598723488 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0010460 con 0x7fb2e810f170 2024-04-26T18:21:08.720 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.716+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a7da0 con 0x7fb2e810f170 2024-04-26T18:21:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.719+0000 7fb2daffd640 1 -- 172.21.15.90:0/598723488 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fb2e0004020 con 0x7fb2e810f170 2024-04-26T18:21:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.719+0000 7fb2f2987740 1 Processor -- start 2024-04-26T18:21:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.719+0000 7fb2f2987740 1 -- start start 2024-04-26T18:21:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d36f00 0x559a84d372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d420e0 0x559a84d444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2f2987740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a84c9d460 con 0x559a84d36f00 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2f2987740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a84c57590 con 0x559a84d420e0 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2f2987740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a84c99420 con 0x559a84d37820 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2daffd640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb2d40a3e40 0x7fb2d40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d36f00 0x559a84d372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d36f00 0x559a84d372e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60234/0 (socket says 172.21.15.90:60234) 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 -- 172.21.15.90:0/2293215455 learned_addr learned my addr 172.21.15.90:0/2293215455 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ee20a640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb2d40a3e40 0x7fb2d40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d420e0 0x559a84d444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2daffd640 1 -- 172.21.15.90:0/598723488 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9015+0+0 (secure 0 0 0) 0x7fb2e00d0640 con 0x7fb2e810f170 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 -- 172.21.15.90:0/2293215455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 msgr2=0x559a84d41ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d41ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 -- 172.21.15.90:0/2293215455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d420e0 msgr2=0x559a84d444d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d420e0 0x559a84d444d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 -- 172.21.15.90:0/2293215455 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a84c640d0 con 0x559a84d36f00 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d36f00 0x559a84d372e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0096c0 tx=0x7fb2dc009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e547ca51034babbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.720+0000 7fb2ee20a640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb2d40a3e40 0x7fb2d40a6300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0061d50 tx=0x7fb2e006ada0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2d3fff640 1 -- 172.21.15.90:0/2293215455 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc035070 con 0x559a84d36f00 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2d3fff640 1 -- 172.21.15.90:0/2293215455 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2dc004230 con 0x559a84d36f00 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2d3fff640 1 -- 172.21.15.90:0/2293215455 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc03d480 con 0x559a84d36f00 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2f2987740 1 -- 172.21.15.90:0/2293215455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d36f00 msgr2=0x559a84d372e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d36f00 0x559a84d372e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0096c0 tx=0x7fb2dc009560 comp rx=0 tx=0).stop 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2f2987740 1 -- 172.21.15.90:0/2293215455 shutdown_connections 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d420e0 0x559a84d444d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2293215455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d36f00 0x559a84d372e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.721+0000 7fb2f2987740 1 -- 172.21.15.90:0/2293215455 >> 172.21.15.90:0/2293215455 conn(0x559a84d33410 msgr2=0x559a84d337f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 -- 172.21.15.90:0/2293215455 shutdown_connections 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 -- 172.21.15.90:0/2293215455 wait complete. 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 -- start start 2024-04-26T18:21:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 Processor -- start 2024-04-26T18:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d3fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559a84d44cb0 con 0x559a84d420e0 2024-04-26T18:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559a84c24b60 con 0x559a84d36f00 2024-04-26T18:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.722+0000 7fb2f2987740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559a84d44e30 con 0x559a84d37820 2024-04-26T18:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2f2987740 10 client.0 _ll_register_callbacks cb 0x559a84c492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:21:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d3fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d3fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60246/0 (socket says 172.21.15.90:60246) 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 -- 172.21.15.90:0/2586156125 learned_addr learned my addr 172.21.15.90:0/2586156125 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 -- 172.21.15.90:0/2586156125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 msgr2=0x559a84d3f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 -- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 msgr2=0x559a84d3f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 -- 172.21.15.90:0/2586156125 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a849af210 con 0x559a84d420e0 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:21:08.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d3fb50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0041f0 tx=0x7fb2dc032db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89eb811f6162bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2586156125 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc035070 con 0x559a84d420e0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2586156125 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9015+0+0 (secure 0 0 0) 0x7fb2dc031d10 con 0x559a84d420e0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc004230 con 0x559a84d420e0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2586156125 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc059050 con 0x559a84d420e0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2f2987740 10 client.16209 fetch_fsmap learned FSMap version 68 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.723+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x559a84a7b620 con 0x559a84d420e0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2586156125 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2dc004230 con 0x559a84d420e0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2586156125 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 68) ==== 8338+0+0 (secure 0 0 0) 0x7fb2dc03a040 con 0x559a84d420e0 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2f2987740 10 client.16209 fetch_fsmap finished waiting for FSMap version 68 2024-04-26T18:21:08.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2f2987740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2f2987740 20 client.16209 populate_metadata read hostname 'smithi090' 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84d520b0 0x559a84d544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2f2987740 4 client.16209 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a84cab070 con 0x559a84d520b0 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.724+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84d520b0 0x559a84d544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.725+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84d520b0 0x559a84d544a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b1ed6ce5f0d63f server_cookie=6d13dbcdb1396a08 in_seq=0 out_seq=0 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.725+0000 7fb2d17fa640 10 client.16209 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.725+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2586156125 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a84cab070 con 0x559a84d520b0 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.725+0000 7fb2d17fa640 10 client.16209 handle_command_reply: tid=0 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.725+0000 7fb2f2987740 1 client.16209 shutdown 2024-04-26T18:21:08.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.725+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 msgr2=0x559a84d3fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.725+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d3fb50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0041f0 tx=0x7fb2dc032db0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 shutdown_connections 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84d520b0 0x559a84d544a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d3fb50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2d17fa640 0 client.16209 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2586156125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 >> 172.21.15.90:0/2586156125 conn(0x559a84d33410 msgr2=0x559a84d39190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 shutdown_connections 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 1 -- 172.21.15.90:0/2586156125 wait complete. 2024-04-26T18:21:08.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.726+0000 7fb2f2987740 20 client.16209 trim_cache size 0 max 16384 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.740+0000 7fb2f2987740 1 Processor -- start 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.740+0000 7fb2f2987740 1 -- start start 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d36f00 0x559a84d32d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d37820 0x559a84d398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d59a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2f2987740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81b31b0 con 0x559a84d420e0 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2f2987740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81b0a80 con 0x559a84d37820 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2f2987740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81ae8f0 con 0x559a84d36f00 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d59a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d59a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60252/0 (socket says 172.21.15.90:60252) 2024-04-26T18:21:08.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 -- 172.21.15.90:0/3839800355 learned_addr learned my addr 172.21.15.90:0/3839800355 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2eda09640 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d37820 0x559a84d398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 -- 172.21.15.90:0/3839800355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d36f00 msgr2=0x559a84d32d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d36f00 0x559a84d32d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 -- 172.21.15.90:0/3839800355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d37820 msgr2=0x559a84d398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d37820 0x559a84d398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 -- 172.21.15.90:0/3839800355 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a84d3ff90 con 0x559a84d420e0 2024-04-26T18:21:08.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2eda09640 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d37820 0x559a84d398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2ed208640 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d59a90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0059d0 tx=0x7fb2dc004120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4002aa17dec3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2d37fe640 1 -- 172.21.15.90:0/3839800355 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc035040 con 0x559a84d420e0 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2d37fe640 1 -- 172.21.15.90:0/3839800355 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2dc004230 con 0x559a84d420e0 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.741+0000 7fb2d37fe640 1 -- 172.21.15.90:0/3839800355 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc031da0 con 0x559a84d420e0 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 -- 172.21.15.90:0/3839800355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 msgr2=0x559a84d59a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d59a90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0059d0 tx=0x7fb2dc004120 comp rx=0 tx=0).stop 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 -- 172.21.15.90:0/3839800355 shutdown_connections 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84d59a90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d37820 0x559a84d398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 --2- 172.21.15.90:0/3839800355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d36f00 0x559a84d32d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 -- 172.21.15.90:0/3839800355 >> 172.21.15.90:0/3839800355 conn(0x559a84d33410 msgr2=0x559a84d40900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:08.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 -- 172.21.15.90:0/3839800355 shutdown_connections 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 -- 172.21.15.90:0/3839800355 wait complete. 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 -- start start 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.742+0000 7fb2f2987740 1 Processor -- start 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2f2987740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84dbfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2f2987740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81aef40 con 0x559a84d420e0 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2f2987740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81b03b0 con 0x559a84d36f00 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2f2987740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81af610 con 0x559a84d37820 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2f2987740 10 client.0 _ll_register_callbacks cb 0x7fb2dc0314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84dbfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84dbfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60262/0 (socket says 172.21.15.90:60262) 2024-04-26T18:21:08.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 -- 172.21.15.90:0/2823281321 learned_addr learned my addr 172.21.15.90:0/2823281321 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 -- 172.21.15.90:0/2823281321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 msgr2=0x559a84d3c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 -- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 msgr2=0x559a84d3c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.743+0000 7fb2ed208640 1 -- 172.21.15.90:0/2823281321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a84dc0050 con 0x559a84d420e0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2ed208640 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84dbfb10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x559a84d47450 tx=0x7fb2dc031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2b3343ffbc7274 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2823281321 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc035040 con 0x559a84d420e0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2823281321 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9015+0+0 (secure 0 0 0) 0x7fb2dc0035f0 con 0x559a84d420e0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2dc004230 con 0x559a84d420e0 2024-04-26T18:21:08.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2823281321 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb2dc0037f0 con 0x559a84d420e0 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 10 client.16215 fetch_fsmap learned FSMap version 68 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x559a84dc11a0 con 0x559a84d420e0 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2823281321 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2dc004230 con 0x559a84d420e0 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2823281321 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 68) ==== 8338+0+0 (secure 0 0 0) 0x7fb2dc069040 con 0x559a84d420e0 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 10 client.16215 fetch_fsmap finished waiting for FSMap version 68 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 20 client.16215 populate_metadata read hostname 'smithi090' 2024-04-26T18:21:08.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84dc17a0 0x559a84dc3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 4 client.16215 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.744+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a84ca4ee0 con 0x559a84dc17a0 2024-04-26T18:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.745+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84dc17a0 0x559a84dc3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.745+0000 7fb2eda09640 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84dc17a0 0x559a84dc3bc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d3d35ca264b755 server_cookie=63243a5f32592a66 in_seq=0 out_seq=0 2024-04-26T18:21:08.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.745+0000 7fb2d17fa640 10 client.16215 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:08.788 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2d17fa640 1 -- 172.21.15.90:0/2823281321 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a84ca4ee0 con 0x559a84dc17a0 2024-04-26T18:21:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2d17fa640 10 client.16215 handle_command_reply: tid=0 2024-04-26T18:21:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 client.16215 shutdown 2024-04-26T18:21:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 msgr2=0x559a84dbfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84dbfb10 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x559a84d47450 tx=0x7fb2dc031980 comp rx=0 tx=0).stop 2024-04-26T18:21:08.789 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 shutdown_connections 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559a84dc17a0 0x559a84dc3bc0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2d17fa640 0 client.16215 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559a84d420e0 0x559a84dbfb10 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559a84d37820 0x559a84d3c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 --2- 172.21.15.90:0/2823281321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559a84d36f00 0x559a84d3c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 >> 172.21.15.90:0/2823281321 conn(0x559a84d33410 msgr2=0x559a84d41190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 shutdown_connections 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 1 -- 172.21.15.90:0/2823281321 wait complete. 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.787+0000 7fb2f2987740 20 client.16215 trim_cache size 0 max 16384 2024-04-26T18:21:08.790 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:21:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:21:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:21:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:21:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb2d40a3e40 msgr2=0x7fb2d40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb2d40a3e40 0x7fb2d40a6300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0061d50 tx=0x7fb2e006ada0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e810f170 msgr2=0x7fb2e819faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e810f170 0x7fb2e819faf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009fd0 tx=0x7fb2e000def0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 shutdown_connections 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb2d40a3e40 0x7fb2d40a6300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2e810f170 0x7fb2e819faf0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2e8075680 0x7fb2e819b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 --2- 172.21.15.90:0/598723488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb2e8074d60 0x7fb2e819b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 >> 172.21.15.90:0/598723488 conn(0x7fb2e80fe400 msgr2=0x7fb2e80ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 shutdown_connections 2024-04-26T18:21:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:08.788+0000 7fb2efc6d640 1 -- 172.21.15.90:0/598723488 wait complete. 2024-04-26T18:21:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:08 smithi119 ceph-mon[32752]: pgmap v291: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-04-26T18:21:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:08 smithi167 ceph-mon[32751]: pgmap v291: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-04-26T18:21:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:08 smithi090 ceph-mon[29180]: pgmap v291: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 204 op/s 2024-04-26T18:21:09.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.63 MB/sec execute 297 sec latency 7437.209 ms 2024-04-26T18:21:10.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.62 MB/sec execute 298 sec latency 8437.254 ms 2024-04-26T18:21:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:10 smithi119 ceph-mon[32752]: pgmap v292: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 983 KiB/s wr, 145 op/s 2024-04-26T18:21:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:21:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:10 smithi167 ceph-mon[32751]: pgmap v292: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 983 KiB/s wr, 145 op/s 2024-04-26T18:21:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:21:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:10 smithi090 ceph-mon[29180]: pgmap v292: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 983 KiB/s wr, 145 op/s 2024-04-26T18:21:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:21:11.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.61 MB/sec execute 299 sec latency 9437.324 ms 2024-04-26T18:21:12.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.59 MB/sec execute 300 sec latency 10437.423 ms 2024-04-26T18:21:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:12 smithi119 ceph-mon[32752]: pgmap v293: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 983 KiB/s wr, 145 op/s 2024-04-26T18:21:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:12 smithi119 ceph-mon[32752]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:21:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:12 smithi167 ceph-mon[32751]: pgmap v293: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 983 KiB/s wr, 145 op/s 2024-04-26T18:21:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:12 smithi167 ceph-mon[32751]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:21:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:12 smithi090 ceph-mon[29180]: pgmap v293: 97 pgs: 97 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 983 KiB/s wr, 145 op/s 2024-04-26T18:21:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:12 smithi090 ceph-mon[29180]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:21:13.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.58 MB/sec execute 301 sec latency 11437.529 ms 2024-04-26T18:21:14.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.57 MB/sec execute 302 sec latency 12437.580 ms 2024-04-26T18:21:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:14 smithi119 ceph-mon[32752]: pgmap v295: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 83 KiB/s wr, 44 op/s 2024-04-26T18:21:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:14 smithi167 ceph-mon[32751]: pgmap v295: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 83 KiB/s wr, 44 op/s 2024-04-26T18:21:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:14 smithi090 ceph-mon[29180]: pgmap v295: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 83 KiB/s wr, 44 op/s 2024-04-26T18:21:15.686 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.56 MB/sec execute 303 sec latency 13437.668 ms 2024-04-26T18:21:16.042 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9dd4e1f700 time 2024-04-26T18:21:15.692972+0000 2024-04-26T18:21:16.042 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:16.042 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ddf4483dd] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.043 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.044 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: clone() 2024-04-26T18:21:16.044 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:15.692+0000 7f9dd4e1f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9dd4e1f700 time 2024-04-26T18:21:15.692972+0000 2024-04-26T18:21:16.044 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:16.044 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ddf4483dd] 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.045 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: clone() 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: *** Caught signal (Aborted) ** 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: in thread 7f9dd4e1f700 thread_name:ms_dispatch 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9dde1d0d20] 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: gsignal() 2024-04-26T18:21:16.046 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: abort() 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ddf448437] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 16: clone() 2024-04-26T18:21:16.047 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:15.695+0000 7f9dd4e1f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: in thread 7f9dd4e1f700 thread_name:ms_dispatch 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9dde1d0d20] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: gsignal() 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: abort() 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ddf448437] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.048 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 16: clone() 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: -1> 2024-04-26T18:21:15.692+0000 7f9dd4e1f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9dd4e1f700 time 2024-04-26T18:21:15.692972+0000 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.049 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ddf4483dd] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: clone() 2024-04-26T18:21:16.050 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 0> 2024-04-26T18:21:15.695+0000 7f9dd4e1f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: in thread 7f9dd4e1f700 thread_name:ms_dispatch 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9dde1d0d20] 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: gsignal() 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: abort() 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ddf448437] 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.051 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 16: clone() 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: -2> 2024-04-26T18:21:15.692+0000 7f9dd4e1f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9dd4e1f700 time 2024-04-26T18:21:15.692972+0000 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9ddf4483dd] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: clone() 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.053 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: -1> 2024-04-26T18:21:15.695+0000 7f9dd4e1f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: in thread 7f9dd4e1f700 thread_name:ms_dispatch 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9dde1d0d20] 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2: gsignal() 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 3: abort() 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9ddf448437] 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9ddf4485a3] 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d739bf0822] 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d739b60598] 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d739b62c17] 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d739a5bd0b] 2024-04-26T18:21:16.054 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d739a5c2ac] 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d739a44fad] 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9ddf6dce20] 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 13: (DispatchQueue::entry()+0x51f) [0x7f9ddf6d9fdf] 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9ddf7acfc1] 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9dde1c61ca] 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 16: clone() 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[68907]: 2024-04-26T18:21:16.055 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:15 smithi167 podman[69139]: 2024-04-26 18:21:15.759129793 +0000 UTC m=+0.021192884 container died 86d6c16e9312e2fce4974e6f820c04f6599c4f31f23292e9be4690491751985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T18:21:16.646 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:16 smithi167 podman[69139]: 2024-04-26 18:21:16.384735496 +0000 UTC m=+0.646798580 container remove 86d6c16e9312e2fce4974e6f820c04f6599c4f31f23292e9be4690491751985f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T18:21:16.646 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:16 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:21:16.646 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:16 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-04-26T18:21:16.646 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:16 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.727s CPU time. 2024-04-26T18:21:16.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.55 MB/sec execute 304 sec latency 14437.779 ms 2024-04-26T18:21:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:16 smithi119 ceph-mon[32752]: pgmap v296: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 68 KiB/s wr, 37 op/s 2024-04-26T18:21:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:16 smithi167 ceph-mon[32751]: pgmap v296: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 68 KiB/s wr, 37 op/s 2024-04-26T18:21:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:16 smithi090 ceph-mon[29180]: pgmap v296: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 68 KiB/s wr, 37 op/s 2024-04-26T18:21:17.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.53 MB/sec execute 305 sec latency 15437.896 ms 2024-04-26T18:21:18.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.52 MB/sec execute 306 sec latency 16437.998 ms 2024-04-26T18:21:19.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:18 smithi119 ceph-mon[32752]: pgmap v297: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 66 KiB/s wr, 35 op/s 2024-04-26T18:21:19.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:18 smithi167 ceph-mon[32751]: pgmap v297: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 66 KiB/s wr, 35 op/s 2024-04-26T18:21:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:18 smithi090 ceph-mon[29180]: pgmap v297: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 66 KiB/s wr, 35 op/s 2024-04-26T18:21:19.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.51 MB/sec execute 307 sec latency 17438.104 ms 2024-04-26T18:21:20.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.50 MB/sec execute 308 sec latency 18438.209 ms 2024-04-26T18:21:21.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:20 smithi119 ceph-mon[32752]: pgmap v298: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-26T18:21:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:20 smithi167 ceph-mon[32751]: pgmap v298: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-26T18:21:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:20 smithi090 ceph-mon[29180]: pgmap v298: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-26T18:21:21.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.49 MB/sec execute 309 sec latency 19438.310 ms 2024-04-26T18:21:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/1472855423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4107820 msgr2=0x7fb3c4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/1472855423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4107820 0x7fb3c4107c00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009930 tx=0x7fb3b402a420 comp rx=0 tx=0).stop 2024-04-26T18:21:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/1472855423 shutdown_connections 2024-04-26T18:21:21.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/1472855423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c4102220 0x7fb3c410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/1472855423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c4101860 0x7fb3c4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/1472855423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4107820 0x7fb3c4107c00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/1472855423 >> 172.21.15.90:0/1472855423 conn(0x7fb3c40789b0 msgr2=0x7fb3c4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/1472855423 shutdown_connections 2024-04-26T18:21:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/1472855423 wait complete. 2024-04-26T18:21:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 Processor -- start 2024-04-26T18:21:21.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.883+0000 7fb3ca6f1640 1 -- start start 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3ca6f1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c4101860 0x7fb3c4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3ca6f1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4102220 0x7fb3c4198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3ca6f1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c4107820 0x7fb3c419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3ca6f1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4111aa0 con 0x7fb3c4102220 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3ca6f1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4111920 con 0x7fb3c4107820 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3ca6f1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c4111c20 con 0x7fb3c4101860 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4102220 0x7fb3c4198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4102220 0x7fb3c4198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52048/0 (socket says 172.21.15.90:52048) 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 -- 172.21.15.90:0/4147663465 learned_addr learned my addr 172.21.15.90:0/4147663465 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c4101860 0x7fb3c4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c4107820 0x7fb3c419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 -- 172.21.15.90:0/4147663465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c4101860 msgr2=0x7fb3c4197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c4101860 0x7fb3c4197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 -- 172.21.15.90:0/4147663465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c4107820 msgr2=0x7fb3c419c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c4107820 0x7fb3c419c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 -- 172.21.15.90:0/4147663465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3c419cf30 con 0x7fb3c4102220 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.884+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4102220 0x7fb3c4198480 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800c920 tx=0x7fb3b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.885+0000 7fb3c17fa640 1 -- 172.21.15.90:0/4147663465 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8007c70 con 0x7fb3c4102220 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.885+0000 7fb3c17fa640 1 -- 172.21.15.90:0/4147663465 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b8007e10 con 0x7fb3c4102220 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.885+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3c419d220 con 0x7fb3c4102220 2024-04-26T18:21:21.887 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.885+0000 7fb3c17fa640 1 -- 172.21.15.90:0/4147663465 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8018820 con 0x7fb3c4102220 2024-04-26T18:21:21.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.885+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3c41a4ab0 con 0x7fb3c4102220 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.887+0000 7fb3c17fa640 1 -- 172.21.15.90:0/4147663465 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fb3b800f450 con 0x7fb3c4102220 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.887+0000 7fb3cd407740 1 Processor -- start 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.887+0000 7fb3c17fa640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb3ac0a3e40 0x7fb3ac0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.887+0000 7fb3cd407740 1 -- start start 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb3ac0a3e40 0x7fb3ac0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba732d80 0x5582ba733160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c17fa640 1 -- 172.21.15.90:0/4147663465 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8971+0+0 (secure 0 0 0) 0x7fb3b80d09d0 con 0x7fb3c4102220 2024-04-26T18:21:21.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba7426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3cd407740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5582ba699460 con 0x5582ba732d80 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3cd407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5582ba653590 con 0x5582ba7336a0 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3cd407740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5582ba695420 con 0x5582ba7402f0 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba732d80 0x5582ba733160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba732d80 0x5582ba733160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52060/0 (socket says 172.21.15.90:52060) 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 -- 172.21.15.90:0/4207540175 learned_addr learned my addr 172.21.15.90:0/4207540175 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba7426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 -- 172.21.15.90:0/4207540175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 msgr2=0x5582ba7426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba7426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 -- 172.21.15.90:0/4207540175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 msgr2=0x5582ba73fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb3ac0a3e40 0x7fb3ac0a6300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005fd0 tx=0x7fb3bc010740 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 -- 172.21.15.90:0/4207540175 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ba6600d0 con 0x5582ba732d80 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba732d80 0x5582ba733160 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009930 tx=0x7fb3b4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85b3d29daaea460 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3aa7fc640 1 -- 172.21.15.90:0/4207540175 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4040070 con 0x5582ba732d80 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3aa7fc640 1 -- 172.21.15.90:0/4207540175 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b40316a0 con 0x5582ba732d80 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.888+0000 7fb3aa7fc640 1 -- 172.21.15.90:0/4207540175 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4035040 con 0x5582ba732d80 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 -- 172.21.15.90:0/4207540175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba732d80 msgr2=0x5582ba733160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba732d80 0x5582ba733160 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009930 tx=0x7fb3b4005fd0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 -- 172.21.15.90:0/4207540175 shutdown_connections 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba7426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 --2- 172.21.15.90:0/4207540175 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba732d80 0x5582ba733160 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 -- 172.21.15.90:0/4207540175 >> 172.21.15.90:0/4207540175 conn(0x5582ba72f410 msgr2=0x5582ba72f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 -- 172.21.15.90:0/4207540175 shutdown_connections 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.889+0000 7fb3cd407740 1 -- 172.21.15.90:0/4207540175 wait complete. 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 -- start start 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 Processor -- start 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba732d80 0x5582ba73d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5582ba742e60 con 0x5582ba7402f0 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5582ba620b60 con 0x5582ba7336a0 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5582ba742fe0 con 0x5582ba732d80 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3cd407740 10 client.0 _ll_register_callbacks cb 0x5582ba6452b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52074/0 (socket says 172.21.15.90:52074) 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/738640898 learned_addr learned my addr 172.21.15.90:0/738640898 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba732d80 0x5582ba73d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/738640898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba732d80 msgr2=0x5582ba73d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba732d80 0x5582ba73d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 msgr2=0x5582ba73d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.890+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/738640898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ba3ab210 con 0x5582ba7402f0 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73dc50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc013c20 tx=0x7fb3bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c673bbfd16aa2a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3a37fe640 1 -- 172.21.15.90:0/738640898 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc023070 con 0x5582ba7402f0 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3a37fe640 1 -- 172.21.15.90:0/738640898 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8971+0+0 (secure 0 0 0) 0x7fb3bc003b70 con 0x5582ba7402f0 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b40316a0 con 0x5582ba7402f0 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3a37fe640 1 -- 172.21.15.90:0/738640898 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb3bc013e60 con 0x5582ba7402f0 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 10 client.16227 fetch_fsmap learned FSMap version 68 2024-04-26T18:21:21.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5582ba477620 con 0x5582ba7402f0 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3a37fe640 1 -- 172.21.15.90:0/738640898 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b40316a0 con 0x5582ba7402f0 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3a37fe640 1 -- 172.21.15.90:0/738640898 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 68) ==== 8338+0+0 (secure 0 0 0) 0x7fb3bc00f040 con 0x5582ba7402f0 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 10 client.16227 fetch_fsmap finished waiting for FSMap version 68 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 20 client.16227 populate_metadata read hostname 'smithi090' 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba750370 0x5582ba752780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 4 client.16227 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.891+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582ba6a7070 con 0x5582ba750370 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.892+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba750370 0x5582ba752780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.892+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba750370 0x5582ba752780 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8568aa1b1f4f5131 server_cookie=59c9907c1935a13 in_seq=0 out_seq=0 2024-04-26T18:21:21.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.892+0000 7fb3a37fe640 10 client.16227 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3a37fe640 1 -- 172.21.15.90:0/738640898 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582ba6a7070 con 0x5582ba750370 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3a37fe640 10 client.16227 handle_command_reply: tid=0 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 client.16227 shutdown 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 msgr2=0x5582ba73dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73dc50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc013c20 tx=0x7fb3bc004ab0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 shutdown_connections 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba750370 0x5582ba752780 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73dc50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba7336a0 0x5582ba73d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3a37fe640 0 client.16227 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 --2- 172.21.15.90:0/738640898 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba732d80 0x5582ba73d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 >> 172.21.15.90:0/738640898 conn(0x5582ba72f410 msgr2=0x5582ba7351d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 shutdown_connections 2024-04-26T18:21:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 1 -- 172.21.15.90:0/738640898 wait complete. 2024-04-26T18:21:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.893+0000 7fb3cd407740 20 client.16227 trim_cache size 0 max 16384 2024-04-26T18:21:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 Processor -- start 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 -- start start 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba74bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7336a0 0x5582ba73ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba758090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41aee00 con 0x5582ba7336a0 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41ac6d0 con 0x5582ba732d80 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3cd407740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41aa540 con 0x5582ba7402f0 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7336a0 0x5582ba73ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7336a0 0x5582ba73ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52090/0 (socket says 172.21.15.90:52090) 2024-04-26T18:21:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 -- 172.21.15.90:0/924451809 learned_addr learned my addr 172.21.15.90:0/924451809 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba74bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba758090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 -- 172.21.15.90:0/924451809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 msgr2=0x5582ba758090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba758090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 -- 172.21.15.90:0/924451809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 msgr2=0x5582ba74bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba74bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c37fe640 1 -- 172.21.15.90:0/924451809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ba73b780 con 0x5582ba7336a0 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba758090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.908+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba74bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7336a0 0x5582ba73ca80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80ce010 tx=0x7fb3b809cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef54712ee4578c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3a9ffb640 1 -- 172.21.15.90:0/924451809 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b80a5420 con 0x5582ba7336a0 2024-04-26T18:21:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3a9ffb640 1 -- 172.21.15.90:0/924451809 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b800fa90 con 0x5582ba7336a0 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3a9ffb640 1 -- 172.21.15.90:0/924451809 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8098e20 con 0x5582ba7336a0 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 -- 172.21.15.90:0/924451809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7336a0 msgr2=0x5582ba73ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7336a0 0x5582ba73ca80 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb3b80ce010 tx=0x7fb3b809cf70 comp rx=0 tx=0).stop 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 -- 172.21.15.90:0/924451809 shutdown_connections 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7402f0 0x5582ba758090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7336a0 0x5582ba73ca80 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 --2- 172.21.15.90:0/924451809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba74bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 -- 172.21.15.90:0/924451809 >> 172.21.15.90:0/924451809 conn(0x5582ba72f410 msgr2=0x5582ba72f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 -- 172.21.15.90:0/924451809 shutdown_connections 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.909+0000 7fb3cd407740 1 -- 172.21.15.90:0/924451809 wait complete. 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 -- start start 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 Processor -- start 2024-04-26T18:21:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba7487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7336a0 0x5582ba739d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41aab90 con 0x5582ba7402f0 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41ac000 con 0x5582ba732d80 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3c41ab260 con 0x5582ba7336a0 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3cd407740 10 client.0 _ll_register_callbacks cb 0x7fb3bc010e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52100/0 (socket says 172.21.15.90:52100) 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/3182604434 learned_addr learned my addr 172.21.15.90:0/3182604434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7336a0 0x5582ba739d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/3182604434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7336a0 msgr2=0x5582ba739d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7336a0 0x5582ba739d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 msgr2=0x5582ba7487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba7487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba7487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.910+0000 7fb3c8c8e640 1 -- 172.21.15.90:0/3182604434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582ba7b5900 con 0x5582ba7402f0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3c37fe640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7336a0 0x5582ba739d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3c8c8e640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73a440 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0275a0 tx=0x7fb3bc011fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4048749ba8d73c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3a37fe640 1 -- 172.21.15.90:0/3182604434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc038020 con 0x5582ba7402f0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3a37fe640 1 -- 172.21.15.90:0/3182604434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8971+0+0 (secure 0 0 0) 0x7fb3bc02c840 con 0x5582ba7402f0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b800fa90 con 0x5582ba7402f0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3a37fe640 1 -- 172.21.15.90:0/3182604434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=68) ==== 24+0+0 (secure 0 0 0) 0x7fb3bc023040 con 0x5582ba7402f0 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 10 client.16233 fetch_fsmap learned FSMap version 68 2024-04-26T18:21:21.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=68,monmap=4+}) -- 0x5582ba7b69f0 con 0x5582ba7402f0 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3a37fe640 1 -- 172.21.15.90:0/3182604434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3b800fa90 con 0x5582ba7402f0 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3a37fe640 1 -- 172.21.15.90:0/3182604434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 68) ==== 8338+0+0 (secure 0 0 0) 0x7fb3bc00f040 con 0x5582ba7402f0 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 10 client.16233 fetch_fsmap finished waiting for FSMap version 68 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 20 client.16233 populate_metadata read hostname 'smithi090' 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba7b71c0 0x5582ba7b95e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 4 client.16233 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.911+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582ba6a0ee0 con 0x5582ba7b71c0 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.912+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba7b71c0 0x5582ba7b95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.912+0000 7fb3a37fe640 10 client.16233 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:21.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:21.912+0000 7fb3c3fff640 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba7b71c0 0x5582ba7b95e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8ba7abeb2c2ec3 server_cookie=e18497fd8a1e613d in_seq=0 out_seq=0 2024-04-26T18:21:22.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.48 MB/sec execute 310 sec latency 20438.416 ms 2024-04-26T18:21:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:22 smithi119 ceph-mon[32752]: pgmap v299: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-26T18:21:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:22 smithi167 ceph-mon[32751]: pgmap v299: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-26T18:21:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:22 smithi090 ceph-mon[29180]: pgmap v299: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 1.0 MiB/s wr, 60 op/s 2024-04-26T18:21:23.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.47 MB/sec execute 311 sec latency 21438.525 ms 2024-04-26T18:21:24.687 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.46 MB/sec execute 312 sec latency 22438.629 ms 2024-04-26T18:21:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:24 smithi119 ceph-mon[32752]: pgmap v300: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 976 KiB/s wr, 65 op/s 2024-04-26T18:21:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:24 smithi167 ceph-mon[32751]: pgmap v300: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 976 KiB/s wr, 65 op/s 2024-04-26T18:21:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:24 smithi090 ceph-mon[29180]: pgmap v300: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 976 KiB/s wr, 65 op/s 2024-04-26T18:21:25.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.44 MB/sec execute 313 sec latency 23438.740 ms 2024-04-26T18:21:26.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.43 MB/sec execute 314 sec latency 24438.777 ms 2024-04-26T18:21:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:26 smithi167 ceph-mon[32751]: pgmap v301: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 839 KiB/s wr, 35 op/s 2024-04-26T18:21:27.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:26 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:21:27.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:26 smithi167 systemd[1]: Stopped Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:21:27.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:26 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.727s CPU time. 2024-04-26T18:21:27.052 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:26 smithi167 systemd[1]: Starting Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:21:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:26 smithi119 ceph-mon[32752]: pgmap v301: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 839 KiB/s wr, 35 op/s 2024-04-26T18:21:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:26 smithi090 ceph-mon[29180]: pgmap v301: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 839 KiB/s wr, 35 op/s 2024-04-26T18:21:27.304 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 podman[69254]: 2024-04-26 18:21:26.955502891 +0000 UTC m=+0.018158897 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:21:27.304 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 podman[69254]: 2024-04-26 18:21:27.085735708 +0000 UTC m=+0.148391744 container create efdac7d7c997c7050f4bf903829513707661d1abf33c2b95475d1acb2338e95a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:21:27.568 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 podman[69254]: 2024-04-26 18:21:27.304907704 +0000 UTC m=+0.367563698 container init efdac7d7c997c7050f4bf903829513707661d1abf33c2b95475d1acb2338e95a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:21:27.568 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 podman[69254]: 2024-04-26 18:21:27.307737394 +0000 UTC m=+0.370393397 container start efdac7d7c997c7050f4bf903829513707661d1abf33c2b95475d1acb2338e95a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T18:21:27.568 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mds[69273]: -- 172.21.15.167:0/2308196284 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55abc05a7a40 con 0x55abc057a800 2024-04-26T18:21:27.568 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: starting mds.i at 2024-04-26T18:21:27.568 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 bash[69254]: efdac7d7c997c7050f4bf903829513707661d1abf33c2b95475d1acb2338e95a 2024-04-26T18:21:27.568 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:27 smithi167 systemd[1]: Started Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:21:27.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.42 MB/sec execute 315 sec latency 25438.829 ms 2024-04-26T18:21:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: daemon mds.i restarted 2024-04-26T18:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T18:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:boot 2024-04-26T18:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:27 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: daemon mds.i restarted 2024-04-26T18:21:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:21:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T18:21:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:boot 2024-04-26T18:21:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:21:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:21:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:21:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:27 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: daemon mds.i restarted 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:boot 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:21:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:27 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:28.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.41 MB/sec execute 316 sec latency 26438.930 ms 2024-04-26T18:21:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:28 smithi119 ceph-mon[32752]: pgmap v302: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 839 KiB/s wr, 35 op/s 2024-04-26T18:21:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:28 smithi167 ceph-mon[32751]: pgmap v302: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 839 KiB/s wr, 35 op/s 2024-04-26T18:21:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:28 smithi090 ceph-mon[29180]: pgmap v302: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 839 KiB/s wr, 35 op/s 2024-04-26T18:21:29.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.40 MB/sec execute 317 sec latency 27438.987 ms 2024-04-26T18:21:30.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:29 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:resolve 2024-04-26T18:21:30.459 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:29 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:30.459 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:29 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:resolve 2024-04-26T18:21:30.459 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:29 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:30.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:29 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:resolve 2024-04-26T18:21:30.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:29 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:30.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.39 MB/sec execute 318 sec latency 28439.039 ms 2024-04-26T18:21:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3a37fe640 1 -- 172.21.15.90:0/3182604434 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582ba6a0ee0 con 0x5582ba7b71c0 2024-04-26T18:21:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3a37fe640 10 client.16233 handle_command_reply: tid=0 2024-04-26T18:21:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 client.16233 shutdown 2024-04-26T18:21:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 msgr2=0x5582ba73a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73a440 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0275a0 tx=0x7fb3bc011fd0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 shutdown_connections 2024-04-26T18:21:30.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5582ba7b71c0 0x5582ba7b95e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5582ba7402f0 0x5582ba73a440 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5582ba7336a0 0x5582ba739d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3a37fe640 0 client.16233 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 --2- 172.21.15.90:0/3182604434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5582ba732d80 0x5582ba7487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.698+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 >> 172.21.15.90:0/3182604434 conn(0x5582ba72f410 msgr2=0x5582ba73ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.699+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 shutdown_connections 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.699+0000 7fb3cd407740 1 -- 172.21.15.90:0/3182604434 wait complete. 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.699+0000 7fb3cd407740 20 client.16233 trim_cache size 0 max 16384 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:21:30.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:21:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:21:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:21:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.699+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb3ac0a3e40 msgr2=0x7fb3ac0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:30.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.699+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb3ac0a3e40 0x7fb3ac0a6300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc005fd0 tx=0x7fb3bc010740 comp rx=0 tx=0).stop 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.699+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4102220 msgr2=0x7fb3c4198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.699+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4102220 0x7fb3c4198480 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800c920 tx=0x7fb3b800cde0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 shutdown_connections 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb3ac0a3e40 0x7fb3ac0a6300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3c4107820 0x7fb3c419c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb3c4102220 0x7fb3c4198480 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 --2- 172.21.15.90:0/4147663465 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3c4101860 0x7fb3c4197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 >> 172.21.15.90:0/4147663465 conn(0x7fb3c40789b0 msgr2=0x7fb3c41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 shutdown_connections 2024-04-26T18:21:30.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:30.700+0000 7fb3ca6f1640 1 -- 172.21.15.90:0/4147663465 wait complete. 2024-04-26T18:21:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:30 smithi119 ceph-mon[32752]: pgmap v304: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-26T18:21:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:30 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:21:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:30 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:21:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:30 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:reconnect 2024-04-26T18:21:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:30 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:30 smithi167 ceph-mon[32751]: pgmap v304: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-26T18:21:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:30 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:21:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:30 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:21:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:30 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:reconnect 2024-04-26T18:21:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:30 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:30 smithi090 ceph-mon[29180]: pgmap v304: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-26T18:21:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:30 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:21:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:30 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:21:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:30 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:reconnect 2024-04-26T18:21:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:30 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:31.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 433361 3.38 MB/sec execute 319 sec latency 29439.079 ms 2024-04-26T18:21:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:31 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:rejoin 2024-04-26T18:21:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:31 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:31 smithi119 ceph-mon[32752]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:21:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:31 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:rejoin 2024-04-26T18:21:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:31 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:31 smithi167 ceph-mon[32751]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:21:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:31 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:rejoin 2024-04-26T18:21:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:31 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:31 smithi090 ceph-mon[29180]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:21:32.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 435348 3.39 MB/sec execute 320 sec latency 29487.989 ms 2024-04-26T18:21:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:32 smithi119 ceph-mon[32752]: pgmap v305: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-26T18:21:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:32 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:21:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:32 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:21:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:32 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:active 2024-04-26T18:21:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:32 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:32 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:32 smithi167 ceph-mon[32751]: pgmap v305: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-26T18:21:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:32 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:21:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:32 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:21:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:32 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:active 2024-04-26T18:21:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:32 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:32 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:32 smithi090 ceph-mon[29180]: pgmap v305: 97 pgs: 97 active+clean; 107 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 KiB/s wr, 14 op/s 2024-04-26T18:21:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:32 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:21:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:32 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:21:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:32 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/3690677501,v1:172.21.15.119:6837/3690677501] up:active 2024-04-26T18:21:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:32 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:32 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:33.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:21:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:21:33.214+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:21:33.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 437102 3.42 MB/sec execute 321 sec latency 14.714 ms 2024-04-26T18:21:34.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 439995 3.46 MB/sec execute 322 sec latency 15.548 ms 2024-04-26T18:21:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:34 smithi119 ceph-mon[32752]: pgmap v306: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:34 smithi167 ceph-mon[32751]: pgmap v306: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:34 smithi090 ceph-mon[29180]: pgmap v306: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:35.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 441378 3.47 MB/sec execute 323 sec latency 13.635 ms 2024-04-26T18:21:36.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443783 3.50 MB/sec execute 324 sec latency 10.891 ms 2024-04-26T18:21:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-mon[32752]: pgmap v307: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:37.054 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcd114e7700 time 2024-04-26T18:21:36.752699+0000 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcd1bb103dd] 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.055 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: clone() 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:36.752+0000 7fcd114e7700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcd114e7700 time 2024-04-26T18:21:36.752699+0000 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcd1bb103dd] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.056 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: clone() 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.057 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: *** Caught signal (Aborted) ** 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: in thread 7fcd114e7700 thread_name:ms_dispatch 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcd1a898d20] 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: gsignal() 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: abort() 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcd1bb10437] 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.058 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.059 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.059 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 16: clone() 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:36.754+0000 7fcd114e7700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: in thread 7fcd114e7700 thread_name:ms_dispatch 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcd1a898d20] 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: gsignal() 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: abort() 2024-04-26T18:21:37.060 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcd1bb10437] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.061 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 16: clone() 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: -1> 2024-04-26T18:21:36.752+0000 7fcd114e7700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcd114e7700 time 2024-04-26T18:21:36.752699+0000 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcd1bb103dd] 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.062 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: clone() 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 0> 2024-04-26T18:21:36.754+0000 7fcd114e7700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: in thread 7fcd114e7700 thread_name:ms_dispatch 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.063 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcd1a898d20] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: gsignal() 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: abort() 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcd1bb10437] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 16: clone() 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: -1> 2024-04-26T18:21:36.752+0000 7fcd114e7700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcd114e7700 time 2024-04-26T18:21:36.752699+0000 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fcd1bb103dd] 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: clone() 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 0> 2024-04-26T18:21:36.754+0000 7fcd114e7700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: in thread 7fcd114e7700 thread_name:ms_dispatch 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.066 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fcd1a898d20] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2: gsignal() 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 3: abort() 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fcd1bb10437] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fcd1bb105a3] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560ff104c822] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560ff0fbc598] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560ff0fbec17] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560ff0eb7d0b] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560ff0eb82ac] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560ff0ea0fad] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fcd1bda4e20] 2024-04-26T18:21:37.067 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 13: (DispatchQueue::entry()+0x51f) [0x7fcd1bda1fdf] 2024-04-26T18:21:37.068 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fcd1be74fc1] 2024-04-26T18:21:37.068 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fcd1a88e1ca] 2024-04-26T18:21:37.068 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 16: clone() 2024-04-26T18:21:37.068 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:37.068 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68630]: 2024-04-26T18:21:37.068 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:36 smithi119 podman[68819]: 2024-04-26 18:21:36.812250027 +0000 UTC m=+0.019964990 container died 53c1c4cfa85ffd00debb09dd3976b8665e81b5c4633a1a769f4575a4ac313120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T18:21:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:36 smithi167 ceph-mon[32751]: pgmap v307: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:36 smithi090 ceph-mon[29180]: pgmap v307: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:37.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.49 MB/sec execute 325 sec latency 938.147 ms 2024-04-26T18:21:37.794 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:37 smithi119 podman[68819]: 2024-04-26 18:21:37.539367207 +0000 UTC m=+0.747082177 container remove 53c1c4cfa85ffd00debb09dd3976b8665e81b5c4633a1a769f4575a4ac313120 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T18:21:37.794 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:37 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:21:38.054 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:37 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-26T18:21:38.054 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:37 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 4.137s CPU time. 2024-04-26T18:21:38.688 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.48 MB/sec execute 326 sec latency 1938.220 ms 2024-04-26T18:21:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:38 smithi119 ceph-mon[32752]: pgmap v308: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:38 smithi167 ceph-mon[32751]: pgmap v308: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:38 smithi090 ceph-mon[29180]: pgmap v308: 97 pgs: 97 active+clean; 110 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 372 KiB/s wr, 66 op/s 2024-04-26T18:21:39.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.47 MB/sec execute 327 sec latency 2938.281 ms 2024-04-26T18:21:40.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.46 MB/sec execute 328 sec latency 3938.367 ms 2024-04-26T18:21:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:40 smithi119 ceph-mon[32752]: pgmap v309: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 936 KiB/s wr, 127 op/s 2024-04-26T18:21:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:40 smithi167 ceph-mon[32751]: pgmap v309: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 936 KiB/s wr, 127 op/s 2024-04-26T18:21:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:40 smithi090 ceph-mon[29180]: pgmap v309: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 936 KiB/s wr, 127 op/s 2024-04-26T18:21:41.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.45 MB/sec execute 329 sec latency 4938.489 ms 2024-04-26T18:21:42.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.44 MB/sec execute 330 sec latency 5938.587 ms 2024-04-26T18:21:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:42 smithi119 ceph-mon[32752]: pgmap v310: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 933 KiB/s wr, 124 op/s 2024-04-26T18:21:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:42 smithi167 ceph-mon[32751]: pgmap v310: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 933 KiB/s wr, 124 op/s 2024-04-26T18:21:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:42 smithi090 ceph-mon[29180]: pgmap v310: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 933 KiB/s wr, 124 op/s 2024-04-26T18:21:43.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.43 MB/sec execute 331 sec latency 6938.627 ms 2024-04-26T18:21:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 -- 172.21.15.90:0/3018523536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc10cfb0 msgr2=0x7ff4dc10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 --2- 172.21.15.90:0/3018523536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc10cfb0 0x7ff4dc10f3e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4009ca0 tx=0x7ff4d402a950 comp rx=0 tx=0).stop 2024-04-26T18:21:43.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 -- 172.21.15.90:0/3018523536 shutdown_connections 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 --2- 172.21.15.90:0/3018523536 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc10cfb0 0x7ff4dc10f3e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 --2- 172.21.15.90:0/3018523536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4dc104500 0x7ff4dc10c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 --2- 172.21.15.90:0/3018523536 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc103be0 0x7ff4dc103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 -- 172.21.15.90:0/3018523536 >> 172.21.15.90:0/3018523536 conn(0x7ff4dc077bb0 msgr2=0x7ff4dc077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 -- 172.21.15.90:0/3018523536 shutdown_connections 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.795+0000 7ff4e322c640 1 -- 172.21.15.90:0/3018523536 wait complete. 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 Processor -- start 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 -- start start 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc103be0 0x7ff4dc199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc104500 0x7ff4dc199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4dc10cfb0 0x7ff4dc19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112c80 con 0x7ff4dc104500 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112b00 con 0x7ff4dc103be0 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e322c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc112e00 con 0x7ff4dc10cfb0 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc104500 0x7ff4dc199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc104500 0x7ff4dc199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41712/0 (socket says 172.21.15.90:41712) 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 -- 172.21.15.90:0/990763784 learned_addr learned my addr 172.21.15.90:0/990763784 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc103be0 0x7ff4dc199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4dc10cfb0 0x7ff4dc19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 -- 172.21.15.90:0/990763784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4dc10cfb0 msgr2=0x7ff4dc19d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4dc10cfb0 0x7ff4dc19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 -- 172.21.15.90:0/990763784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc103be0 msgr2=0x7ff4dc199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc103be0 0x7ff4dc199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.796+0000 7ff4dbfff640 1 -- 172.21.15.90:0/990763784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4dc19dff0 con 0x7ff4dc104500 2024-04-26T18:21:43.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.797+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc104500 0x7ff4dc199540 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800b4a0 tx=0x7ff4c800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.797+0000 7ff4d9ffb640 1 -- 172.21.15.90:0/990763784 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8015020 con 0x7ff4dc104500 2024-04-26T18:21:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.797+0000 7ff4d9ffb640 1 -- 172.21.15.90:0/990763784 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c8011ce0 con 0x7ff4dc104500 2024-04-26T18:21:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.797+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4dc19e2e0 con 0x7ff4dc104500 2024-04-26T18:21:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.797+0000 7ff4d9ffb640 1 -- 172.21.15.90:0/990763784 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8010dc0 con 0x7ff4dc104500 2024-04-26T18:21:43.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.797+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4dc1a5b70 con 0x7ff4dc104500 2024-04-26T18:21:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.799+0000 7ff4d9ffb640 1 -- 172.21.15.90:0/990763784 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7ff4c8018020 con 0x7ff4dc104500 2024-04-26T18:21:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.799+0000 7ff4e601f740 1 Processor -- start 2024-04-26T18:21:43.806 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e601f740 1 -- start start 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4d9ffb640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff4cc0a3ec0 0x7ff4cc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf67fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf6a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e601f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5619cdebf460 con 0x5619cdf59720 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e601f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5619cde79590 con 0x5619cdf684f0 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e601f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5619cdebb420 con 0x5619cdf58e00 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff4cc0a3ec0 0x7ff4cc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4d9ffb640 1 -- 172.21.15.90:0/990763784 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9013+0+0 (secure 0 0 0) 0x7ff4c80d01b0 con 0x7ff4dc104500 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf67fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf67fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41724/0 (socket says 172.21.15.90:41724) 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2329760271 learned_addr learned my addr 172.21.15.90:0/2329760271 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:43.807 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf6a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2329760271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 msgr2=0x5619cdf591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2329760271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 msgr2=0x5619cdf6a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf6a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2329760271 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619cde860d0 con 0x5619cdf59720 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff4cc0a3ec0 0x7ff4cc0a6380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff4d402ae70 tx=0x7ff4d402f7a0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf67fb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80989e0 tx=0x7ff4c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4051549e38d0e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.800+0000 7ff4c6ffd640 1 -- 172.21.15.90:0/2329760271 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c808b020 con 0x5619cdf59720 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4c6ffd640 1 -- 172.21.15.90:0/2329760271 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c809c710 con 0x5619cdf59720 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4c6ffd640 1 -- 172.21.15.90:0/2329760271 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c809a310 con 0x5619cdf59720 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 -- 172.21.15.90:0/2329760271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 msgr2=0x5619cdf67fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:43.808 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf67fb0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80989e0 tx=0x7ff4c80046c0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 -- 172.21.15.90:0/2329760271 shutdown_connections 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf6a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf67fb0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2329760271 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 -- 172.21.15.90:0/2329760271 >> 172.21.15.90:0/2329760271 conn(0x5619cdf55410 msgr2=0x5619cdf557f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 -- 172.21.15.90:0/2329760271 shutdown_connections 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.801+0000 7ff4e601f740 1 -- 172.21.15.90:0/2329760271 wait complete. 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 -- start start 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 Processor -- start 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf58e00 0x5619cdf653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf65930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf684f0 0x5619cdf65e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.809 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5619cdf6b120 con 0x5619cdf59720 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5619cde46b60 con 0x5619cdf58e00 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5619cdf6b2a0 con 0x5619cdf684f0 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e601f740 10 client.0 _ll_register_callbacks cb 0x5619cde6b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf65930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf65930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41730/0 (socket says 172.21.15.90:41730) 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4dbfff640 1 -- 172.21.15.90:0/4068374246 learned_addr learned my addr 172.21.15.90:0/4068374246 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf58e00 0x5619cdf653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf684f0 0x5619cdf65e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4dbfff640 1 -- 172.21.15.90:0/4068374246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf684f0 msgr2=0x5619cdf65e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf684f0 0x5619cdf65e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4dbfff640 1 -- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf58e00 msgr2=0x5619cdf653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.802+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf58e00 0x5619cdf653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4dbfff640 1 -- 172.21.15.90:0/4068374246 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619cdbd1210 con 0x5619cdf59720 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf684f0 0x5619cdf65e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf58e00 0x5619cdf653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf65930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff4c809bd10 tx=0x7ff4c8010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0b8bc96de24a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4bbfff640 1 -- 172.21.15.90:0/4068374246 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c80a03c0 con 0x5619cdf59720 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4bbfff640 1 -- 172.21.15.90:0/4068374246 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9013+0+0 (secure 0 0 0) 0x7ff4c80cde00 con 0x5619cdf59720 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c809c710 con 0x5619cdf59720 2024-04-26T18:21:43.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4bbfff640 1 -- 172.21.15.90:0/4068374246 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff4c808b050 con 0x5619cdf59720 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 10 client.16248 fetch_fsmap learned FSMap version 75 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5619cdc9d620 con 0x5619cdf59720 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4bbfff640 1 -- 172.21.15.90:0/4068374246 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c809c710 con 0x5619cdf59720 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4bbfff640 1 -- 172.21.15.90:0/4068374246 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 75) ==== 8338+0+0 (secure 0 0 0) 0x7ff4c80b5290 con 0x5619cdf59720 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 10 client.16248 fetch_fsmap finished waiting for FSMap version 75 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 20 client.16248 populate_metadata read hostname 'smithi090' 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdf784e0 0x5619cdf7a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 4 client.16248 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.803+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619cdecd070 con 0x5619cdf784e0 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.804+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdf784e0 0x5619cdf7a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.804+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdf784e0 0x5619cdf7a8d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ac11a6e0c584ae4 server_cookie=c0ae8ce984938e21 in_seq=0 out_seq=0 2024-04-26T18:21:43.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.804+0000 7ff4bbfff640 10 client.16248 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4bbfff640 1 -- 172.21.15.90:0/4068374246 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619cdecd070 con 0x5619cdf784e0 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4bbfff640 10 client.16248 handle_command_reply: tid=0 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 client.16248 shutdown 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 msgr2=0x5619cdf65930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf65930 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff4c809bd10 tx=0x7ff4c8010640 comp rx=0 tx=0).stop 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 shutdown_connections 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdf784e0 0x5619cdf7a8d0 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf684f0 0x5619cdf65e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf65930 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4bbfff640 0 client.16248 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4068374246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf58e00 0x5619cdf653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 >> 172.21.15.90:0/4068374246 conn(0x5619cdf55410 msgr2=0x5619cdf5b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:43.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 shutdown_connections 2024-04-26T18:21:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 1 -- 172.21.15.90:0/4068374246 wait complete. 2024-04-26T18:21:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.805+0000 7ff4e601f740 20 client.16248 trim_cache size 0 max 16384 2024-04-26T18:21:43.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 Processor -- start 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 -- start start 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf58e00 0x5619cdf5b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf59720 0x5619cdf7d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1aefc0 con 0x5619cdf58e00 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1ac890 con 0x5619cdf684f0 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e601f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1aa720 con 0x5619cdf59720 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e0fc8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf58e00 0x5619cdf5b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e0fc8640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf58e00 0x5619cdf5b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41732/0 (socket says 172.21.15.90:41732) 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e0fc8640 1 -- 172.21.15.90:0/4013905103 learned_addr learned my addr 172.21.15.90:0/4013905103 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.820+0000 7ff4e0fc8640 1 -- 172.21.15.90:0/4013905103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf59720 msgr2=0x5619cdf7d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf59720 0x5619cdf7d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e0fc8640 1 -- 172.21.15.90:0/4013905103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 msgr2=0x5619cdf7de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e0fc8640 1 -- 172.21.15.90:0/4013905103 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619cdf55240 con 0x5619cdf58e00 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf58e00 0x5619cdf5b060 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0000c00 tx=0x7ff4d0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbb35fddabf5704 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4c67fc640 1 -- 172.21.15.90:0/4013905103 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d001d430 con 0x5619cdf58e00 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4c67fc640 1 -- 172.21.15.90:0/4013905103 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0005020 con 0x5619cdf58e00 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4c67fc640 1 -- 172.21.15.90:0/4013905103 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0004b70 con 0x5619cdf58e00 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e601f740 1 -- 172.21.15.90:0/4013905103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf58e00 msgr2=0x5619cdf5b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf58e00 0x5619cdf5b060 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0000c00 tx=0x7ff4d0013830 comp rx=0 tx=0).stop 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e601f740 1 -- 172.21.15.90:0/4013905103 shutdown_connections 2024-04-26T18:21:43.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf59720 0x5619cdf7d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.821+0000 7ff4e601f740 1 --2- 172.21.15.90:0/4013905103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf58e00 0x5619cdf5b060 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 -- 172.21.15.90:0/4013905103 >> 172.21.15.90:0/4013905103 conn(0x5619cdf55410 msgr2=0x5619cdf5a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 -- 172.21.15.90:0/4013905103 shutdown_connections 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 -- 172.21.15.90:0/4013905103 wait complete. 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 -- start start 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 Processor -- start 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf7d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf7db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1aad70 con 0x5619cdf59720 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1ac1c0 con 0x5619cdf684f0 2024-04-26T18:21:43.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4dc1ab420 con 0x5619cdf58e00 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4e601f740 10 client.0 _ll_register_callbacks cb 0x7ff4c8098310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf7db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4dbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf7db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41740/0 (socket says 172.21.15.90:41740) 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.822+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2317072797 learned_addr learned my addr 172.21.15.90:0/2317072797 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf7d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e17c9640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2317072797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 msgr2=0x5619cdf7d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf7d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 msgr2=0x5619cdf7bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4dbfff640 1 -- 172.21.15.90:0/2317072797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619cdf59c60 con 0x5619cdf59720 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf7d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:21:43.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4dbfff640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf7db10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8098470 tx=0x7ff4c809a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1475e97ac08fc532 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4bbfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4bbfff640 1 -- 172.21.15.90:0/2317072797 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c809bb50 con 0x5619cdf59720 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4bbfff640 1 -- 172.21.15.90:0/2317072797 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9013+0+0 (secure 0 0 0) 0x7ff4c80cde00 con 0x5619cdf59720 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d0005020 con 0x5619cdf59720 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4bbfff640 1 -- 172.21.15.90:0/2317072797 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7ff4c80914a0 con 0x5619cdf59720 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 10 client.16254 fetch_fsmap learned FSMap version 75 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=4+}) -- 0x5619cdfdefe0 con 0x5619cdf59720 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4bbfff640 1 -- 172.21.15.90:0/2317072797 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4d0005020 con 0x5619cdf59720 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4bbfff640 1 -- 172.21.15.90:0/2317072797 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 75) ==== 8338+0+0 (secure 0 0 0) 0x7ff4c808e070 con 0x5619cdf59720 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 10 client.16254 fetch_fsmap finished waiting for FSMap version 75 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:21:43.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 20 client.16254 populate_metadata read hostname 'smithi090' 2024-04-26T18:21:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdfdf4b0 0x5619cdfe18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 4 client.16254 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:21:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.823+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619cdec6ee0 con 0x5619cdfdf4b0 2024-04-26T18:21:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.824+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdfdf4b0 0x5619cdfe18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.824+0000 7ff4bbfff640 10 client.16254 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:43.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:43.824+0000 7ff4e0fc8640 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdfdf4b0 0x5619cdfe18a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea24a748f0367869 server_cookie=1aba9b9c37926dcf in_seq=0 out_seq=0 2024-04-26T18:21:44.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.42 MB/sec execute 332 sec latency 7938.664 ms 2024-04-26T18:21:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:44 smithi119 ceph-mon[32752]: pgmap v311: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 938 KiB/s wr, 129 op/s 2024-04-26T18:21:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:44 smithi167 ceph-mon[32751]: pgmap v311: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 938 KiB/s wr, 129 op/s 2024-04-26T18:21:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:44 smithi090 ceph-mon[29180]: pgmap v311: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 938 KiB/s wr, 129 op/s 2024-04-26T18:21:45.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.41 MB/sec execute 333 sec latency 8938.703 ms 2024-04-26T18:21:46.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.40 MB/sec execute 334 sec latency 9938.764 ms 2024-04-26T18:21:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:46 smithi119 ceph-mon[32752]: pgmap v312: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 77 op/s 2024-04-26T18:21:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:46 smithi167 ceph-mon[32751]: pgmap v312: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 77 op/s 2024-04-26T18:21:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:46 smithi090 ceph-mon[29180]: pgmap v312: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 77 op/s 2024-04-26T18:21:47.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.39 MB/sec execute 335 sec latency 10938.807 ms 2024-04-26T18:21:48.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:47 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:21:48.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:47 smithi119 systemd[1]: Stopped Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:21:48.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:47 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 4.137s CPU time. 2024-04-26T18:21:48.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:47 smithi119 systemd[1]: Starting Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:21:48.064 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 podman[68914]: 2024-04-26 18:21:47.940225615 +0000 UTC m=+0.018245523 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:21:48.065 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 podman[68914]: 2024-04-26 18:21:48.065280288 +0000 UTC m=+0.143300183 container create 89c178a3a0f86a2b893e315d8d28ac6fab8c8b425cb4d07aeeda5589ef9648b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:21:48.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.38 MB/sec execute 336 sec latency 11938.882 ms 2024-04-26T18:21:48.717 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 podman[68914]: 2024-04-26 18:21:48.342867063 +0000 UTC m=+0.420886957 container init 89c178a3a0f86a2b893e315d8d28ac6fab8c8b425cb4d07aeeda5589ef9648b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T18:21:48.717 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 podman[68914]: 2024-04-26 18:21:48.346116268 +0000 UTC m=+0.424136164 container start 89c178a3a0f86a2b893e315d8d28ac6fab8c8b425cb4d07aeeda5589ef9648b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T18:21:48.717 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mds[68933]: -- 172.21.15.119:0/1916395897 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d083f5a40 con 0x558d083c8c00 2024-04-26T18:21:48.717 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: starting mds.h at 2024-04-26T18:21:48.717 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 bash[68914]: 89c178a3a0f86a2b893e315d8d28ac6fab8c8b425cb4d07aeeda5589ef9648b8 2024-04-26T18:21:48.717 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:21:48 smithi119 systemd[1]: Started Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:21:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: pgmap v313: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 77 op/s 2024-04-26T18:21:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: daemon mds.h restarted 2024-04-26T18:21:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:21:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T18:21:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:boot 2024-04-26T18:21:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:21:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:21:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:21:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:48 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: pgmap v313: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 77 op/s 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: daemon mds.h restarted 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:boot 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:21:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:48 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: pgmap v313: 97 pgs: 97 active+clean; 116 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 629 KiB/s wr, 77 op/s 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: daemon mds.h restarted 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:boot 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: Standby daemon mds.i assigned to filesystem cephfs as rank 1 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 7 up:standby, 1 failed 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:48 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:replay,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:49.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.37 MB/sec execute 337 sec latency 12938.999 ms 2024-04-26T18:21:50.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.36 MB/sec execute 338 sec latency 13939.132 ms 2024-04-26T18:21:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:50 smithi119 ceph-mon[32752]: pgmap v315: 97 pgs: 97 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 990 KiB/s wr, 55 op/s 2024-04-26T18:21:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:50 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:resolve 2024-04-26T18:21:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:50 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:50 smithi167 ceph-mon[32751]: pgmap v315: 97 pgs: 97 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 990 KiB/s wr, 55 op/s 2024-04-26T18:21:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:50 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:resolve 2024-04-26T18:21:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:50 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:50 smithi090 ceph-mon[29180]: pgmap v315: 97 pgs: 97 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 990 KiB/s wr, 55 op/s 2024-04-26T18:21:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:50 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:resolve 2024-04-26T18:21:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:50 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:resolve,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:51.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 443807 3.35 MB/sec execute 339 sec latency 14939.243 ms 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4bbfff640 1 -- 172.21.15.90:0/2317072797 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619cdec6ee0 con 0x5619cdfdf4b0 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4bbfff640 10 client.16254 handle_command_reply: tid=0 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 client.16254 shutdown 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 msgr2=0x5619cdf7db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf7db10 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8098470 tx=0x7ff4c809a520 comp rx=0 tx=0).stop 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 shutdown_connections 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5619cdfdf4b0 0x5619cdfe18a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5619cdf684f0 0x5619cdf7bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5619cdf59720 0x5619cdf7db10 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 --2- 172.21.15.90:0/2317072797 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619cdf58e00 0x5619cdf7d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4bbfff640 0 client.16254 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:21:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 >> 172.21.15.90:0/2317072797 conn(0x5619cdf55410 msgr2=0x5619cdf65c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 shutdown_connections 2024-04-26T18:21:51.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 1 -- 172.21.15.90:0/2317072797 wait complete. 2024-04-26T18:21:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.766+0000 7ff4e601f740 20 client.16254 trim_cache size 0 max 16384 2024-04-26T18:21:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:21:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:21:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:21:51.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:21:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff4cc0a3ec0 msgr2=0x7ff4cc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff4cc0a3ec0 0x7ff4cc0a6380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff4d402ae70 tx=0x7ff4d402f7a0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc104500 msgr2=0x7ff4dc199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc104500 0x7ff4dc199540 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff4c800b4a0 tx=0x7ff4c800b8a0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 shutdown_connections 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff4cc0a3ec0 0x7ff4cc0a6380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4dc10cfb0 0x7ff4dc19d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4dc104500 0x7ff4dc199540 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 --2- 172.21.15.90:0/990763784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4dc103be0 0x7ff4dc199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 >> 172.21.15.90:0/990763784 conn(0x7ff4dc077bb0 msgr2=0x7ff4dc0fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.767+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 shutdown_connections 2024-04-26T18:21:51.771 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:21:51.768+0000 7ff4e322c640 1 -- 172.21.15.90:0/990763784 wait complete. 2024-04-26T18:21:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:51 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:21:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:51 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:reconnect 2024-04-26T18:21:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:51 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:51 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:21:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:51 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:21:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:51 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:reconnect 2024-04-26T18:21:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:51 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:51 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:21:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:51 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:21:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:51 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:reconnect 2024-04-26T18:21:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:51 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:reconnect,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:51 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:21:52.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 444159 3.34 MB/sec execute 340 sec latency 15005.586 ms 2024-04-26T18:21:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: pgmap v316: 97 pgs: 97 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 990 KiB/s wr, 55 op/s 2024-04-26T18:21:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:rejoin 2024-04-26T18:21:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:21:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:21:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: Cluster is now healthy 2024-04-26T18:21:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:active 2024-04-26T18:21:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:52 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: pgmap v316: 97 pgs: 97 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 990 KiB/s wr, 55 op/s 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:rejoin 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: Cluster is now healthy 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:active 2024-04-26T18:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:52 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: pgmap v316: 97 pgs: 97 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 990 KiB/s wr, 55 op/s 2024-04-26T18:21:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:rejoin 2024-04-26T18:21:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=i=up:rejoin,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-26T18:21:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:21:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:21:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: Cluster is now healthy 2024-04-26T18:21:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6838/2850927327,v1:172.21.15.167:6839/2850927327] up:active 2024-04-26T18:21:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:52 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-26T18:21:53.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:21:53 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:21:53.216+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:21:53.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 446571 3.37 MB/sec execute 341 sec latency 18.331 ms 2024-04-26T18:21:54.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 447820 3.38 MB/sec execute 342 sec latency 18.295 ms 2024-04-26T18:21:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:54 smithi119 ceph-mon[32752]: pgmap v317: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:54 smithi167 ceph-mon[32751]: pgmap v317: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:54 smithi090 ceph-mon[29180]: pgmap v317: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:55.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 450609 3.42 MB/sec execute 343 sec latency 15.520 ms 2024-04-26T18:21:56.689 INFO:tasks.workunit.client.0.smithi090.stdout: 1 451988 3.43 MB/sec execute 344 sec latency 13.345 ms 2024-04-26T18:21:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:56 smithi119 ceph-mon[32752]: pgmap v318: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:56 smithi167 ceph-mon[32751]: pgmap v318: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:56 smithi090 ceph-mon[29180]: pgmap v318: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:57.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.46 MB/sec execute 345 sec latency 129.005 ms 2024-04-26T18:21:57.818 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f8c668e8700 time 2024-04-26T18:21:57.566881+0000 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8c70f113dd] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.819 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.820 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.820 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.820 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: clone() 2024-04-26T18:21:57.820 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.566+0000 7f8c668e8700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f8c668e8700 time 2024-04-26T18:21:57.566881+0000 2024-04-26T18:21:57.820 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:57.820 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8c70f113dd] 2024-04-26T18:21:57.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.821 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: clone() 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: *** Caught signal (Aborted) ** 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: in thread 7f8c668e8700 thread_name:ms_dispatch 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.822 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8c6fc99d20] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: gsignal() 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: abort() 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8c70f11437] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.823 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 16: clone() 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.568+0000 7f8c668e8700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: in thread 7f8c668e8700 thread_name:ms_dispatch 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8c6fc99d20] 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: gsignal() 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: abort() 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8c70f11437] 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.824 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 16: clone() 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: -1> 2024-04-26T18:21:57.566+0000 7f8c668e8700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f8c668e8700 time 2024-04-26T18:21:57.566881+0000 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.825 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8c70f113dd] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.826 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: clone() 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 0> 2024-04-26T18:21:57.568+0000 7f8c668e8700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: in thread 7f8c668e8700 thread_name:ms_dispatch 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8c6fc99d20] 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: gsignal() 2024-04-26T18:21:57.830 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: abort() 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8c70f11437] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 16: clone() 2024-04-26T18:21:57.831 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: -1> 2024-04-26T18:21:57.566+0000 7f8c668e8700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f8c668e8700 time 2024-04-26T18:21:57.566881+0000 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f8c70f113dd] 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.832 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: clone() 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 0> 2024-04-26T18:21:57.568+0000 7f8c668e8700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: in thread 7f8c668e8700 thread_name:ms_dispatch 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8c6fc99d20] 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2: gsignal() 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 3: abort() 2024-04-26T18:21:57.833 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f8c70f11437] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f8c70f115a3] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55abbeac0822] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55abbea30598] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55abbea32c17] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55abbe92bd0b] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55abbe92c2ac] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55abbe914fad] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8c711a5e20] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 13: (DispatchQueue::entry()+0x51f) [0x7f8c711a2fdf] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8c71275fc1] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f8c6fc8f1ca] 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 16: clone() 2024-04-26T18:21:57.834 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:21:57.835 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i[69269]: 2024-04-26T18:21:57.835 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:57 smithi167 podman[69343]: 2024-04-26 18:21:57.624643124 +0000 UTC m=+0.018355051 container died efdac7d7c997c7050f4bf903829513707661d1abf33c2b95475d1acb2338e95a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T18:21:58.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.45 MB/sec execute 346 sec latency 1129.086 ms 2024-04-26T18:21:58.742 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:58 smithi167 podman[69343]: 2024-04-26 18:21:58.324979923 +0000 UTC m=+0.718691841 container remove efdac7d7c997c7050f4bf903829513707661d1abf33c2b95475d1acb2338e95a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:21:58.742 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:58 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:21:58.742 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:58 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-04-26T18:21:58.742 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:21:58 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.857s CPU time. 2024-04-26T18:21:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:21:58 smithi119 ceph-mon[32752]: pgmap v319: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:21:58 smithi167 ceph-mon[32751]: pgmap v319: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:21:58 smithi090 ceph-mon[29180]: pgmap v319: 97 pgs: 97 active+clean; 116 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.2 MiB/s wr, 98 op/s 2024-04-26T18:21:59.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.44 MB/sec execute 347 sec latency 2129.251 ms 2024-04-26T18:22:00.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.43 MB/sec execute 348 sec latency 3129.370 ms 2024-04-26T18:22:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:00 smithi119 ceph-mon[32752]: pgmap v320: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-26T18:22:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:00 smithi167 ceph-mon[32751]: pgmap v320: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-26T18:22:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:00 smithi090 ceph-mon[29180]: pgmap v320: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.0 MiB/s wr, 141 op/s 2024-04-26T18:22:01.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.42 MB/sec execute 349 sec latency 4129.472 ms 2024-04-26T18:22:02.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.41 MB/sec execute 350 sec latency 5129.614 ms 2024-04-26T18:22:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:02 smithi119 ceph-mon[32752]: pgmap v321: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 954 KiB/s wr, 128 op/s 2024-04-26T18:22:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:02 smithi167 ceph-mon[32751]: pgmap v321: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 954 KiB/s wr, 128 op/s 2024-04-26T18:22:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:02 smithi090 ceph-mon[29180]: pgmap v321: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 954 KiB/s wr, 128 op/s 2024-04-26T18:22:03.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.40 MB/sec execute 351 sec latency 6129.731 ms 2024-04-26T18:22:04.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.39 MB/sec execute 352 sec latency 7129.777 ms 2024-04-26T18:22:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 -- 172.21.15.90:0/1141054135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 msgr2=0x7f6f1c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 --2- 172.21.15.90:0/1141054135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c103fa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009900 tx=0x7f6f0402a510 comp rx=0 tx=0).stop 2024-04-26T18:22:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 -- 172.21.15.90:0/1141054135 shutdown_connections 2024-04-26T18:22:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 --2- 172.21.15.90:0/1141054135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f1c10cf00 0x7f6f1c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 --2- 172.21.15.90:0/1141054135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f1c1044e0 0x7f6f1c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 --2- 172.21.15.90:0/1141054135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c103fa0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 -- 172.21.15.90:0/1141054135 >> 172.21.15.90:0/1141054135 conn(0x7f6f1c0789b0 msgr2=0x7f6f1c078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 -- 172.21.15.90:0/1141054135 shutdown_connections 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 -- 172.21.15.90:0/1141054135 wait complete. 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 Processor -- start 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.864+0000 7f6f230a0640 1 -- start start 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f230a0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f230a0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f1c1044e0 0x7f6f1c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f230a0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f1c10cf00 0x7f6f1c19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f230a0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c112bb0 con 0x7f6f1c103bc0 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f230a0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c112a30 con 0x7f6f1c1044e0 2024-04-26T18:22:04.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f230a0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c112d30 con 0x7f6f1c10cf00 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52664/0 (socket says 172.21.15.90:52664) 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 -- 172.21.15.90:0/3736998293 learned_addr learned my addr 172.21.15.90:0/3736998293 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f1c1044e0 0x7f6f1c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f1c10cf00 0x7f6f1c19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 -- 172.21.15.90:0/3736998293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f1c10cf00 msgr2=0x7f6f1c19d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f1c10cf00 0x7f6f1c19d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 -- 172.21.15.90:0/3736998293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f1c1044e0 msgr2=0x7f6f1c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f1c1044e0 0x7f6f1c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 -- 172.21.15.90:0/3736998293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f1c19e0f0 con 0x7f6f1c103bc0 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f1c1044e0 0x7f6f1c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c199150 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005940 tx=0x7f6f040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f19ffb640 1 -- 172.21.15.90:0/3736998293 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0403f030 con 0x7f6f1c103bc0 2024-04-26T18:22:04.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f19ffb640 1 -- 172.21.15.90:0/3736998293 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f04003cb0 con 0x7f6f1c103bc0 2024-04-26T18:22:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f1c19e380 con 0x7f6f1c103bc0 2024-04-26T18:22:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.865+0000 7f6f19ffb640 1 -- 172.21.15.90:0/3736998293 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04035070 con 0x7f6f1c103bc0 2024-04-26T18:22:04.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.866+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f1c1a5c70 con 0x7f6f1c103bc0 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f19ffb640 1 -- 172.21.15.90:0/3736998293 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f6f04042030 con 0x7f6f1c103bc0 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 Processor -- start 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 -- start start 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de182322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de1823cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de1823f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55de18198460 con 0x55de18231f00 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55de18152590 con 0x55de18232820 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55de18194420 con 0x55de1823d0e0 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.868+0000 7f6f19ffb640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6f080a3dc0 0x7f6f080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de182322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de182322e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52678/0 (socket says 172.21.15.90:52678) 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 -- 172.21.15.90:0/3989733091 learned_addr learned my addr 172.21.15.90:0/3989733091 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6f080a3dc0 0x7f6f080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de1823f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f19ffb640 1 -- 172.21.15.90:0/3736998293 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9099+0+0 (secure 0 0 0) 0x7f6f040f13f0 con 0x7f6f1c103bc0 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de1823cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 -- 172.21.15.90:0/3989733091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 msgr2=0x55de1823f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de1823f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 -- 172.21.15.90:0/3989733091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 msgr2=0x55de1823cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de1823cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 -- 172.21.15.90:0/3989733091 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de1815f0d0 con 0x55de18231f00 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de182322e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6f14013a80 tx=0x7f6f14013f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa42bc6a5db7b21f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6f080a3dc0 0x7f6f080a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f140058c0 tx=0x7f6f14015000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f12ffd640 1 -- 172.21.15.90:0/3989733091 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14013250 con 0x55de18231f00 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f12ffd640 1 -- 172.21.15.90:0/3989733091 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f140133f0 con 0x55de18231f00 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.869+0000 7f6f12ffd640 1 -- 172.21.15.90:0/3989733091 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f14004b60 con 0x55de18231f00 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3989733091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 msgr2=0x55de182322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de182322e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f6f14013a80 tx=0x7f6f14013f40 comp rx=0 tx=0).stop 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3989733091 shutdown_connections 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de1823f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de1823cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3989733091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de182322e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3989733091 >> 172.21.15.90:0/3989733091 conn(0x55de1822e410 msgr2=0x55de1822e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:04.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3989733091 shutdown_connections 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3989733091 wait complete. 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 -- start start 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.870+0000 7f6f25dbf740 1 Processor -- start 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de18231f00 0x55de1823a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de1823a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1823d0e0 0x55de1823ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55de1823fcb0 con 0x55de18232820 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55de1811fb60 con 0x55de1823d0e0 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55de1823fe30 con 0x55de18231f00 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f25dbf740 10 client.0 _ll_register_callbacks cb 0x55de181442b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de1823a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de1823a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52692/0 (socket says 172.21.15.90:52692) 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 -- 172.21.15.90:0/3596647571 learned_addr learned my addr 172.21.15.90:0/3596647571 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:04.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1823d0e0 0x55de1823ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de18231f00 0x55de1823a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 -- 172.21.15.90:0/3596647571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de18231f00 msgr2=0x55de1823a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de18231f00 0x55de1823a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 -- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1823d0e0 msgr2=0x55de1823ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1823d0e0 0x55de1823ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f1bfff640 1 -- 172.21.15.90:0/3596647571 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de17eaa210 con 0x55de18232820 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de18231f00 0x55de1823a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.871+0000 7f6f2163d640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1823d0e0 0x55de1823ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de1823a610 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c003c00 tx=0x7f6f0c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f311199d031089d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6efbfff640 1 -- 172.21.15.90:0/3596647571 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c004d20 con 0x55de18232820 2024-04-26T18:22:04.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6efbfff640 1 -- 172.21.15.90:0/3596647571 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9099+0+0 (secure 0 0 0) 0x7f6f0c011710 con 0x55de18232820 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f140133f0 con 0x55de18232820 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6efbfff640 1 -- 172.21.15.90:0/3596647571 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f6f0c023050 con 0x55de18232820 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 10 client.16266 fetch_fsmap learned FSMap version 81 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55de17f76620 con 0x55de18232820 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6efbfff640 1 -- 172.21.15.90:0/3596647571 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f140133f0 con 0x55de18232820 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6efbfff640 1 -- 172.21.15.90:0/3596647571 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 81) ==== 8338+0+0 (secure 0 0 0) 0x7f6f0c019030 con 0x55de18232820 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 10 client.16266 fetch_fsmap finished waiting for FSMap version 81 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 20 client.16266 populate_metadata read hostname 'smithi090' 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de1824d2d0 0x55de1824f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 4 client.16266 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.872+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de181a6070 con 0x55de1824d2d0 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.873+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de1824d2d0 0x55de1824f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.873+0000 7f6efbfff640 10 client.16266 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:04.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.873+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de1824d2d0 0x55de1824f6e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9745f13604a8c252 server_cookie=21d585200ad88141 in_seq=0 out_seq=0 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6efbfff640 1 -- 172.21.15.90:0/3596647571 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de181a6070 con 0x55de1824d2d0 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6efbfff640 10 client.16266 handle_command_reply: tid=0 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 client.16266 shutdown 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 msgr2=0x55de1823a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de1823a610 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c003c00 tx=0x7f6f0c004410 comp rx=0 tx=0).stop 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 shutdown_connections 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de1824d2d0 0x55de1824f6e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de1823d0e0 0x55de1823ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de1823a610 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6efbfff640 0 client.16266 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/3596647571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de18231f00 0x55de1823a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 >> 172.21.15.90:0/3596647571 conn(0x55de1822e410 msgr2=0x55de18234190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:04.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 shutdown_connections 2024-04-26T18:22:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 1 -- 172.21.15.90:0/3596647571 wait complete. 2024-04-26T18:22:04.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.874+0000 7f6f25dbf740 20 client.16266 trim_cache size 0 max 16384 2024-04-26T18:22:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 Processor -- start 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 -- start start 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de18248b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de18254ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de18255530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1b1350 con 0x55de18231f00 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1aec20 con 0x55de18232820 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1aca70 con 0x55de1823d0e0 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de18248b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de18248b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52702/0 (socket says 172.21.15.90:52702) 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 -- 172.21.15.90:0/2997414948 learned_addr learned my addr 172.21.15.90:0/2997414948 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 -- 172.21.15.90:0/2997414948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 msgr2=0x55de18255530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de18255530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de18254ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 -- 172.21.15.90:0/2997414948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 msgr2=0x55de18254ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de18254ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f20e3c640 1 -- 172.21.15.90:0/2997414948 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de18235ad0 con 0x55de18231f00 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.889+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de18254ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de18248b70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6f040ee900 tx=0x7f6f04046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5087d284c7fe42a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f127fc640 1 -- 172.21.15.90:0/2997414948 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f04049040 con 0x55de18231f00 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f127fc640 1 -- 172.21.15.90:0/2997414948 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f040bd0f0 con 0x55de18231f00 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f127fc640 1 -- 172.21.15.90:0/2997414948 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f040c1120 con 0x55de18231f00 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2997414948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 msgr2=0x55de18248b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de18248b70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6f040ee900 tx=0x7f6f04046040 comp rx=0 tx=0).stop 2024-04-26T18:22:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2997414948 shutdown_connections 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de18255530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18232820 0x55de18254ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2997414948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18231f00 0x55de18248b70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2997414948 >> 172.21.15.90:0/2997414948 conn(0x55de1822e410 msgr2=0x55de182353c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2997414948 shutdown_connections 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.890+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2997414948 wait complete. 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 -- start start 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 Processor -- start 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18231f00 0x55de18240c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de182411b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de182416f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1ad0e0 con 0x55de18232820 2024-04-26T18:22:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1ae550 con 0x55de18231f00 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1c1ad7b0 con 0x55de1823d0e0 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f25dbf740 10 client.0 _ll_register_callbacks cb 0x7f6f0c009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de182411b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de182411b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52710/0 (socket says 172.21.15.90:52710) 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 -- 172.21.15.90:0/2676863094 learned_addr learned my addr 172.21.15.90:0/2676863094 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f2163d640 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de182416f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18231f00 0x55de18240c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 -- 172.21.15.90:0/2676863094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 msgr2=0x55de182416f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de182416f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 -- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18231f00 msgr2=0x55de18240c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18231f00 0x55de18240c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.891+0000 7f6f1bfff640 1 -- 172.21.15.90:0/2676863094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de182b2a60 con 0x55de18232820 2024-04-26T18:22:04.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f1bfff640 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de182411b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c01ac30 tx=0x7f6f0c009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3d127f68225ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6efbfff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6efbfff640 1 -- 172.21.15.90:0/2676863094 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c020040 con 0x55de18232820 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6efbfff640 1 -- 172.21.15.90:0/2676863094 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9099+0+0 (secure 0 0 0) 0x7f6f0c01a5f0 con 0x55de18232820 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f040bd0f0 con 0x55de18232820 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6efbfff640 1 -- 172.21.15.90:0/2676863094 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f6f0c037050 con 0x55de18232820 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 10 client.16272 fetch_fsmap learned FSMap version 81 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55de182b3d00 con 0x55de18232820 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6efbfff640 1 -- 172.21.15.90:0/2676863094 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f040bd0f0 con 0x55de18232820 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6efbfff640 1 -- 172.21.15.90:0/2676863094 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 81) ==== 8338+0+0 (secure 0 0 0) 0x7f6f0c019030 con 0x55de18232820 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 10 client.16272 fetch_fsmap finished waiting for FSMap version 81 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 20 client.16272 populate_metadata read hostname 'smithi090' 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de182b44d0 0x55de182b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 4 client.16272 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.892+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de1819fee0 con 0x55de182b44d0 2024-04-26T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.893+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de182b44d0 0x55de182b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.893+0000 7f6efbfff640 10 client.16272 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:04.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:04.893+0000 7f6f20e3c640 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de182b44d0 0x55de182b68f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fee953fd1584d80 server_cookie=4797b5c453df26b0 in_seq=0 out_seq=0 2024-04-26T18:22:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:04 smithi119 ceph-mon[32752]: pgmap v322: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 954 KiB/s wr, 134 op/s 2024-04-26T18:22:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:04 smithi167 ceph-mon[32751]: pgmap v322: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 954 KiB/s wr, 134 op/s 2024-04-26T18:22:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:04 smithi090 ceph-mon[29180]: pgmap v322: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 954 KiB/s wr, 134 op/s 2024-04-26T18:22:05.690 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.38 MB/sec execute 353 sec latency 8129.902 ms 2024-04-26T18:22:06.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.37 MB/sec execute 354 sec latency 9129.995 ms 2024-04-26T18:22:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:06 smithi119 ceph-mon[32752]: pgmap v323: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 749 KiB/s wr, 93 op/s 2024-04-26T18:22:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:06 smithi167 ceph-mon[32751]: pgmap v323: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 749 KiB/s wr, 93 op/s 2024-04-26T18:22:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:06 smithi090 ceph-mon[29180]: pgmap v323: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 749 KiB/s wr, 93 op/s 2024-04-26T18:22:07.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.36 MB/sec execute 355 sec latency 10130.099 ms 2024-04-26T18:22:08.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.35 MB/sec execute 356 sec latency 11130.176 ms 2024-04-26T18:22:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:08 smithi119 ceph-mon[32752]: pgmap v324: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 749 KiB/s wr, 93 op/s 2024-04-26T18:22:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:08 smithi167 ceph-mon[32751]: pgmap v324: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 749 KiB/s wr, 93 op/s 2024-04-26T18:22:09.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:22:08 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:22:09.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:22:08 smithi167 systemd[1]: Stopped Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:22:09.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:22:08 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Consumed 3.857s CPU time. 2024-04-26T18:22:09.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:22:08 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Start request repeated too quickly. 2024-04-26T18:22:09.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:22:08 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.i.service: Failed with result 'exit-code'. 2024-04-26T18:22:09.068 INFO:journalctl@ceph.mds.i.smithi167.stdout:Apr 26 18:22:08 smithi167 systemd[1]: Failed to start Ceph mds.i for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:22:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:08 smithi090 ceph-mon[29180]: pgmap v324: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 749 KiB/s wr, 93 op/s 2024-04-26T18:22:09.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.34 MB/sec execute 357 sec latency 12130.277 ms 2024-04-26T18:22:10.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.33 MB/sec execute 358 sec latency 13130.376 ms 2024-04-26T18:22:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:10 smithi119 ceph-mon[32752]: pgmap v325: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 768 KiB/s wr, 115 op/s 2024-04-26T18:22:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:10 smithi167 ceph-mon[32751]: pgmap v325: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 768 KiB/s wr, 115 op/s 2024-04-26T18:22:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:10 smithi090 ceph-mon[29180]: pgmap v325: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 768 KiB/s wr, 115 op/s 2024-04-26T18:22:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:11.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.32 MB/sec execute 359 sec latency 14130.420 ms 2024-04-26T18:22:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:12 smithi119 ceph-mon[32752]: pgmap v326: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 27 op/s 2024-04-26T18:22:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:12 smithi167 ceph-mon[32751]: pgmap v326: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 27 op/s 2024-04-26T18:22:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:12 smithi090 ceph-mon[29180]: pgmap v326: 97 pgs: 97 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 20 KiB/s wr, 27 op/s 2024-04-26T18:22:12.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.31 MB/sec execute 360 sec latency 15130.453 ms 2024-04-26T18:22:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: Replacing daemon mds.i as rank 1 with standby daemon mds.h 2024-04-26T18:22:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-04-26T18:22:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:22:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T18:22:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:13 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: Replacing daemon mds.i as rank 1 with standby daemon mds.h 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:22:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:13 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: Replacing daemon mds.i as rank 1 with standby daemon mds.h 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: MDS daemon mds.i is removed because it is dead or otherwise unavailable. 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:13 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:22:13.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.30 MB/sec execute 361 sec latency 16130.513 ms 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: pgmap v328: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 971 KiB/s wr, 56 op/s 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:22:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: pgmap v329: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:resolve 2024-04-26T18:22:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:14 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: pgmap v328: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 971 KiB/s wr, 56 op/s 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:22:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:22:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: pgmap v329: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-04-26T18:22:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:resolve 2024-04-26T18:22:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:14 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:14.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: pgmap v328: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 971 KiB/s wr, 56 op/s 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:14.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: pgmap v329: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-04-26T18:22:14.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:22:14.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:resolve 2024-04-26T18:22:14.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:14 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:14.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454277 3.29 MB/sec execute 362 sec latency 17130.596 ms 2024-04-26T18:22:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:15 smithi119 ceph-mon[32752]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:22:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:reconnect 2024-04-26T18:22:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:15 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:15 smithi167 ceph-mon[32751]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:22:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:reconnect 2024-04-26T18:22:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:15 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:15 smithi090 ceph-mon[29180]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:22:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:reconnect 2024-04-26T18:22:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:15 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6efbfff640 1 -- 172.21.15.90:0/2676863094 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de1819fee0 con 0x55de182b44d0 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6efbfff640 10 client.16272 handle_command_reply: tid=0 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 client.16272 shutdown 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 msgr2=0x55de182411b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de182411b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c01ac30 tx=0x7f6f0c009f70 comp rx=0 tx=0).stop 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 shutdown_connections 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55de182b44d0 0x55de182b68f0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55de1823d0e0 0x55de182416f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55de18232820 0x55de182411b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 --2- 172.21.15.90:0/2676863094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55de18231f00 0x55de18240c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6efbfff640 0 client.16272 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 >> 172.21.15.90:0/2676863094 conn(0x55de1822e410 msgr2=0x55de182338b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 shutdown_connections 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.632+0000 7f6f25dbf740 1 -- 172.21.15.90:0/2676863094 wait complete. 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f25dbf740 20 client.16272 trim_cache size 0 max 16384 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:22:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:22:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6f080a3dc0 msgr2=0x7f6f080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6f080a3dc0 0x7f6f080a6280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f140058c0 tx=0x7f6f14015000 comp rx=0 tx=0).stop 2024-04-26T18:22:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 msgr2=0x7f6f1c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c199150 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6f04005940 tx=0x7f6f040042f0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 shutdown_connections 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6f080a3dc0 0x7f6f080a6280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6f1c10cf00 0x7f6f1c19d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f1c1044e0 0x7f6f1c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 --2- 172.21.15.90:0/3736998293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f1c103bc0 0x7f6f1c199150 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 >> 172.21.15.90:0/3736998293 conn(0x7f6f1c0789b0 msgr2=0x7f6f1c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 shutdown_connections 2024-04-26T18:22:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:15.633+0000 7f6f230a0640 1 -- 172.21.15.90:0/3736998293 wait complete. 2024-04-26T18:22:15.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454280 3.28 MB/sec execute 363 sec latency 18064.918 ms 2024-04-26T18:22:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:16 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:22:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:16 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:22:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:16 smithi090 ceph-mon[29180]: pgmap v330: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-04-26T18:22:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:16 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:rejoin 2024-04-26T18:22:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:16 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:16 smithi090 ceph-mon[29180]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:22:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:16 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:22:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:16 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:22:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:16 smithi119 ceph-mon[32752]: pgmap v330: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-04-26T18:22:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:16 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:rejoin 2024-04-26T18:22:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:16 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:16 smithi119 ceph-mon[32752]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:22:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:16 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:22:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:16 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:22:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:16 smithi167 ceph-mon[32751]: pgmap v330: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 1.2 MiB/s wr, 69 op/s 2024-04-26T18:22:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:16 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:rejoin 2024-04-26T18:22:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:16 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:16 smithi167 ceph-mon[32751]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:22:16.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454280 3.28 MB/sec execute 364 sec latency 1065.893 ms 2024-04-26T18:22:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:17 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:22:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:17 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:active 2024-04-26T18:22:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:17 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:17 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:17 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:22:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:17 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:active 2024-04-26T18:22:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:17 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:17 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:17 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:22:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:17 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/4116054973,v1:172.21.15.119:6837/4116054973] up:active 2024-04-26T18:22:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:17 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:17 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:17.691 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454280 3.27 MB/sec execute 365 sec latency 2066.018 ms 2024-04-26T18:22:18.473 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:22:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:22:18.218+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:22:18.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:18 smithi119 ceph-mon[32752]: pgmap v331: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-26T18:22:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:18 smithi167 ceph-mon[32751]: pgmap v331: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-26T18:22:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:18 smithi090 ceph-mon[29180]: pgmap v331: 97 pgs: 97 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-26T18:22:18.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 454324 3.26 MB/sec execute 366 sec latency 2985.273 ms 2024-04-26T18:22:19.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 455864 3.26 MB/sec execute 367 sec latency 19.280 ms 2024-04-26T18:22:20.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 457974 3.29 MB/sec execute 368 sec latency 37.643 ms 2024-04-26T18:22:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:20 smithi167 ceph-mon[32751]: pgmap v332: 97 pgs: 97 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-04-26T18:22:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:20 smithi090 ceph-mon[29180]: pgmap v332: 97 pgs: 97 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-04-26T18:22:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:20 smithi119 ceph-mon[32752]: pgmap v332: 97 pgs: 97 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.2 MiB/s wr, 59 op/s 2024-04-26T18:22:21.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.33 MB/sec execute 369 sec latency 37.745 ms 2024-04-26T18:22:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:21 smithi090 ceph-mon[29180]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T18:22:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:21 smithi119 ceph-mon[32752]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T18:22:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:21 smithi167 ceph-mon[32751]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T18:22:22.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.32 MB/sec execute 370 sec latency 1032.555 ms 2024-04-26T18:22:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:22 smithi090 ceph-mon[29180]: pgmap v334: 97 pgs: 97 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-26T18:22:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:22 smithi119 ceph-mon[32752]: pgmap v334: 97 pgs: 97 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-26T18:22:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:22 smithi167 ceph-mon[32751]: pgmap v334: 97 pgs: 97 active+clean; 118 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 KiB/s wr, 22 op/s 2024-04-26T18:22:23.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.31 MB/sec execute 371 sec latency 2032.597 ms 2024-04-26T18:22:24.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.30 MB/sec execute 372 sec latency 3032.703 ms 2024-04-26T18:22:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:24 smithi090 ceph-mon[29180]: pgmap v335: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:24 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:24 smithi119 ceph-mon[32752]: pgmap v335: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:24 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:24 smithi167 ceph-mon[32751]: pgmap v335: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:24 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:25.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.29 MB/sec execute 373 sec latency 4032.811 ms 2024-04-26T18:22:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:25 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:25 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:25 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:26.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.28 MB/sec execute 374 sec latency 5032.924 ms 2024-04-26T18:22:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:26 smithi090 ceph-mon[29180]: pgmap v336: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:26 smithi119 ceph-mon[32752]: pgmap v336: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:26 smithi167 ceph-mon[32751]: pgmap v336: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:27.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.27 MB/sec execute 375 sec latency 6033.047 ms 2024-04-26T18:22:28.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.26 MB/sec execute 376 sec latency 7033.139 ms 2024-04-26T18:22:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 -- 172.21.15.90:0/3022651392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c104520 msgr2=0x7f409c10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 --2- 172.21.15.90:0/3022651392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c104520 0x7f409c10ca00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4084009900 tx=0x7f408402a480 comp rx=0 tx=0).stop 2024-04-26T18:22:28.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 -- 172.21.15.90:0/3022651392 shutdown_connections 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 --2- 172.21.15.90:0/3022651392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c10cf40 0x7f409c10f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 --2- 172.21.15.90:0/3022651392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c104520 0x7f409c10ca00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 --2- 172.21.15.90:0/3022651392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f409c103c00 0x7f409c103fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 -- 172.21.15.90:0/3022651392 >> 172.21.15.90:0/3022651392 conn(0x7f409c0789b0 msgr2=0x7f409c078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 -- 172.21.15.90:0/3022651392 shutdown_connections 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.731+0000 7f40a1262640 1 -- 172.21.15.90:0/3022651392 wait complete. 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 Processor -- start 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 -- start start 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c103c00 0x7f409c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f409c104520 0x7f409c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c10cf40 0x7f409c19d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f409c112ba0 con 0x7f409c10cf40 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f409c112a20 con 0x7f409c103c00 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f40a1262640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f409c112d20 con 0x7f409c104520 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c10cf40 0x7f409c19d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c10cf40 0x7f409c19d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34164/0 (socket says 172.21.15.90:34164) 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 -- 172.21.15.90:0/766546803 learned_addr learned my addr 172.21.15.90:0/766546803 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f4093fff640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f409c104520 0x7f409c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409affd640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c103c00 0x7f409c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 -- 172.21.15.90:0/766546803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f409c104520 msgr2=0x7f409c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f409c104520 0x7f409c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 -- 172.21.15.90:0/766546803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c103c00 msgr2=0x7f409c199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c103c00 0x7f409c199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.732+0000 7f409b7fe640 1 -- 172.21.15.90:0/766546803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c19e020 con 0x7f409c10cf40 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.733+0000 7f409b7fe640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c10cf40 0x7f409c19d8f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f409400cce0 tx=0x7f409400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.733+0000 7f4098ff9640 1 -- 172.21.15.90:0/766546803 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409400f450 con 0x7f409c10cf40 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.733+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c19e310 con 0x7f409c10cf40 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.733+0000 7f4098ff9640 1 -- 172.21.15.90:0/766546803 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f409400faa0 con 0x7f409c10cf40 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.733+0000 7f4098ff9640 1 -- 172.21.15.90:0/766546803 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094018b70 con 0x7f409c10cf40 2024-04-26T18:22:28.737 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.733+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c1a5ba0 con 0x7f409c10cf40 2024-04-26T18:22:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f4098ff9640 1 -- 172.21.15.90:0/766546803 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f4094018d10 con 0x7f409c10cf40 2024-04-26T18:22:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 Processor -- start 2024-04-26T18:22:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 -- start start 2024-04-26T18:22:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84381f00 0x562b843822e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b8438cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562b842e8460 con 0x562b84382820 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562b842a2590 con 0x562b8438d0e0 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f40a3f76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562b842e4420 con 0x562b84381f00 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f409affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b8438cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f409affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b8438cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34174/0 (socket says 172.21.15.90:34174) 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.736+0000 7f409affd640 1 -- 172.21.15.90:0/1890866758 learned_addr learned my addr 172.21.15.90:0/1890866758 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f4098ff9640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f40880a3dc0 0x7f40880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f4093fff640 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84381f00 0x562b843822e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f4093fff640 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409affd640 1 -- 172.21.15.90:0/1890866758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84381f00 msgr2=0x562b843822e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409affd640 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84381f00 0x562b843822e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409affd640 1 -- 172.21.15.90:0/1890866758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 msgr2=0x562b8438f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409affd640 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409b7fe640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f40880a3dc0 0x7f40880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409affd640 1 -- 172.21.15.90:0/1890866758 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b842af0d0 con 0x562b84382820 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f4093fff640 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f4098ff9640 1 -- 172.21.15.90:0/766546803 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9229+0+0 (secure 0 0 0) 0x7f40940d0e20 con 0x7f409c10cf40 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409affd640 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b8438cba0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f408c00b760 tx=0x7f408c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed582624b70cb50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f40917fa640 1 -- 172.21.15.90:0/1890866758 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c003dd0 con 0x562b84382820 2024-04-26T18:22:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f40917fa640 1 -- 172.21.15.90:0/1890866758 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408c015070 con 0x562b84382820 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f40917fa640 1 -- 172.21.15.90:0/1890866758 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c004e00 con 0x562b84382820 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.737+0000 7f409b7fe640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f40880a3dc0 0x7f40880a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f40940ce360 tx=0x7f409406f5f0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 -- 172.21.15.90:0/1890866758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 msgr2=0x562b8438cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b8438cba0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f408c00b760 tx=0x7f408c00bc20 comp rx=0 tx=0).stop 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 -- 172.21.15.90:0/1890866758 shutdown_connections 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b8438cba0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1890866758 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84381f00 0x562b843822e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 -- 172.21.15.90:0/1890866758 >> 172.21.15.90:0/1890866758 conn(0x562b8437e410 msgr2=0x562b8437e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 -- 172.21.15.90:0/1890866758 shutdown_connections 2024-04-26T18:22:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 -- 172.21.15.90:0/1890866758 wait complete. 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.738+0000 7f40a3f76740 1 -- start start 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 1 Processor -- start 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b8438a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b8438a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x562b8438fd10 con 0x562b84381f00 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x562b8426fb60 con 0x562b8438d0e0 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562b8438fe90 con 0x562b84382820 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40a3f76740 10 client.0 _ll_register_callbacks cb 0x562b842942b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:22:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b8438a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b8438a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34178/0 (socket says 172.21.15.90:34178) 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 -- 172.21.15.90:0/800659191 learned_addr learned my addr 172.21.15.90:0/800659191 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f409affd640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b8438a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 -- 172.21.15.90:0/800659191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 msgr2=0x562b8438a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b8438a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 -- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 msgr2=0x562b8438aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 -- 172.21.15.90:0/800659191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b83ffa210 con 0x562b84381f00 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f409affd640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b8438a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f4093fff640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b8438a040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4084009900 tx=0x7f4084009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961d97eb894dd219 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40827fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.739+0000 7f40827fc640 1 -- 172.21.15.90:0/800659191 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084035040 con 0x562b84381f00 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40827fc640 1 -- 172.21.15.90:0/800659191 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9229+0+0 (secure 0 0 0) 0x7f4084004240 con 0x562b84381f00 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f408c015070 con 0x562b84381f00 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40827fc640 1 -- 172.21.15.90:0/800659191 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f4084004440 con 0x562b84381f00 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 10 client.16284 fetch_fsmap learned FSMap version 89 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x562b840c6620 con 0x562b84381f00 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40827fc640 1 -- 172.21.15.90:0/800659191 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f408c015070 con 0x562b84381f00 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40827fc640 1 -- 172.21.15.90:0/800659191 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 89) ==== 7741+0+0 (secure 0 0 0) 0x7f408403d4c0 con 0x562b84381f00 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 10 client.16284 fetch_fsmap finished waiting for FSMap version 89 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:22:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 20 client.16284 populate_metadata read hostname 'smithi090' 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b8439d1a0 0x562b8439f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 4 client.16284 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b842f6070 con 0x562b8439d1a0 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.740+0000 7f409affd640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b8439d1a0 0x562b8439f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.741+0000 7f40827fc640 10 client.16284 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.741+0000 7f409affd640 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b8439d1a0 0x562b8439f5b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24a0752bb257c9f1 server_cookie=59a1a3eb1f9c471d in_seq=0 out_seq=0 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40827fc640 1 -- 172.21.15.90:0/800659191 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b842f6070 con 0x562b8439d1a0 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40827fc640 10 client.16284 handle_command_reply: tid=0 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 client.16284 shutdown 2024-04-26T18:22:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 msgr2=0x562b8438a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b8438a040 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4084009900 tx=0x7f4084009260 comp rx=0 tx=0).stop 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 shutdown_connections 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b8439d1a0 0x562b8439f5b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b8438aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b8438a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40827fc640 0 client.16284 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 --2- 172.21.15.90:0/800659191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b8438a040 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 >> 172.21.15.90:0/800659191 conn(0x562b8437e410 msgr2=0x562b84384140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 shutdown_connections 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 1 -- 172.21.15.90:0/800659191 wait complete. 2024-04-26T18:22:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.742+0000 7f40a3f76740 20 client.16284 trim_cache size 0 max 16384 2024-04-26T18:22:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 Processor -- start 2024-04-26T18:22:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 -- start start 2024-04-26T18:22:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b84398bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b843848c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b843a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1b05a0 con 0x562b84381f00 2024-04-26T18:22:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1b0c70 con 0x562b8438d0e0 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f40a3f76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ae550 con 0x562b84382820 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b84398bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b84398bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34188/0 (socket says 172.21.15.90:34188) 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 -- 172.21.15.90:0/1015242235 learned_addr learned my addr 172.21.15.90:0/1015242235 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f409affd640 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b843848c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 -- 172.21.15.90:0/1015242235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 msgr2=0x562b843848c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b843848c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 -- 172.21.15.90:0/1015242235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 msgr2=0x562b843a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b843a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f4093fff640 1 -- 172.21.15.90:0/1015242235 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b84384e00 con 0x562b84381f00 2024-04-26T18:22:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.757+0000 7f409affd640 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b843848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.758+0000 7f4093fff640 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b84398bd0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4084006010 tx=0x7f408402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8bbec1c10673a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.758+0000 7f4090ff9640 1 -- 172.21.15.90:0/1015242235 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084035070 con 0x562b84381f00 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.758+0000 7f4090ff9640 1 -- 172.21.15.90:0/1015242235 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40840054e0 con 0x562b84381f00 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.758+0000 7f4090ff9640 1 -- 172.21.15.90:0/1015242235 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4084004920 con 0x562b84381f00 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.758+0000 7f40a3f76740 1 -- 172.21.15.90:0/1015242235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 msgr2=0x562b84398bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.758+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b84398bd0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4084006010 tx=0x7f408402a480 comp rx=0 tx=0).stop 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- 172.21.15.90:0/1015242235 shutdown_connections 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b8438d0e0 0x562b843a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b84382820 0x562b843848c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1015242235 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84381f00 0x562b84398bd0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- 172.21.15.90:0/1015242235 >> 172.21.15.90:0/1015242235 conn(0x562b8437e410 msgr2=0x562b8437e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- 172.21.15.90:0/1015242235 shutdown_connections 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- 172.21.15.90:0/1015242235 wait complete. 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- start start 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 Processor -- start 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b84381f00 0x562b84386d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b843872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b8438d0e0 0x562b84387810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ade80 con 0x562b84382820 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1aca90 con 0x562b84381f00 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f409c1ad0e0 con 0x562b8438d0e0 2024-04-26T18:22:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f40a3f76740 10 client.0 _ll_register_callbacks cb 0x7f4084005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f409affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b843872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f409affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b843872d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34194/0 (socket says 172.21.15.90:34194) 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.759+0000 7f409affd640 1 -- 172.21.15.90:0/1180146875 learned_addr learned my addr 172.21.15.90:0/1180146875 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f4093fff640 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b8438d0e0 0x562b84387810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f409affd640 1 -- 172.21.15.90:0/1180146875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b8438d0e0 msgr2=0x562b84387810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f409affd640 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b8438d0e0 0x562b84387810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f409affd640 1 -- 172.21.15.90:0/1180146875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b84381f00 msgr2=0x562b84386d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f409affd640 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b84381f00 0x562b84386d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f409affd640 1 -- 172.21.15.90:0/1180146875 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b84402780 con 0x562b84382820 2024-04-26T18:22:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f4093fff640 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b8438d0e0 0x562b84387810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f409affd640 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b843872d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f408c0057c0 tx=0x7f408c00b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6337f80e5245450 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40827fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40827fc640 1 -- 172.21.15.90:0/1180146875 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f408c02c020 con 0x562b84382820 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40827fc640 1 -- 172.21.15.90:0/1180146875 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9229+0+0 (secure 0 0 0) 0x7f408c003de0 con 0x562b84382820 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40840054e0 con 0x562b84382820 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40827fc640 1 -- 172.21.15.90:0/1180146875 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f408c037050 con 0x562b84382820 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 10 client.16290 fetch_fsmap learned FSMap version 89 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x562b84403a70 con 0x562b84382820 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40827fc640 1 -- 172.21.15.90:0/1180146875 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40840054e0 con 0x562b84382820 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40827fc640 1 -- 172.21.15.90:0/1180146875 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 89) ==== 7741+0+0 (secure 0 0 0) 0x7f408c011670 con 0x562b84382820 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 10 client.16290 fetch_fsmap finished waiting for FSMap version 89 2024-04-26T18:22:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:22:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 20 client.16290 populate_metadata read hostname 'smithi090' 2024-04-26T18:22:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b84404220 0x562b84406640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 4 client.16290 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:22:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.760+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b842efee0 con 0x562b84404220 2024-04-26T18:22:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.761+0000 7f4093fff640 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b84404220 0x562b84406640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.761+0000 7f40827fc640 10 client.16290 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.761+0000 7f4093fff640 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b84404220 0x562b84406640 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6079ba3b2d7db142 server_cookie=d6a7df5ef3d79b22 in_seq=0 out_seq=0 2024-04-26T18:22:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40827fc640 1 -- 172.21.15.90:0/1180146875 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562b842efee0 con 0x562b84404220 2024-04-26T18:22:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40827fc640 10 client.16290 handle_command_reply: tid=0 2024-04-26T18:22:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 client.16290 shutdown 2024-04-26T18:22:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 msgr2=0x562b843872d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b843872d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f408c0057c0 tx=0x7f408c00b420 comp rx=0 tx=0).stop 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 shutdown_connections 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x562b84404220 0x562b84406640 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562b8438d0e0 0x562b84387810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x562b84382820 0x562b843872d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40827fc640 0 client.16290 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 --2- 172.21.15.90:0/1180146875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562b84381f00 0x562b84386d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.796+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 >> 172.21.15.90:0/1180146875 conn(0x562b8437e410 msgr2=0x562b8437cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 shutdown_connections 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a3f76740 1 -- 172.21.15.90:0/1180146875 wait complete. 2024-04-26T18:22:28.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a3f76740 20 client.16290 trim_cache size 0 max 16384 2024-04-26T18:22:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:22:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:22:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:22:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:22:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f40880a3dc0 msgr2=0x7f40880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f40880a3dc0 0x7f40880a6280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f40940ce360 tx=0x7f409406f5f0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c10cf40 msgr2=0x7f409c19d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c10cf40 0x7f409c19d8f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f409400cce0 tx=0x7f409400a620 comp rx=0 tx=0).stop 2024-04-26T18:22:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 shutdown_connections 2024-04-26T18:22:28.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f40880a3dc0 0x7f40880a6280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f409c10cf40 0x7f409c19d8f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f409c104520 0x7f409c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 --2- 172.21.15.90:0/766546803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f409c103c00 0x7f409c199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 >> 172.21.15.90:0/766546803 conn(0x7f409c0789b0 msgr2=0x7f409c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 shutdown_connections 2024-04-26T18:22:28.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:28.797+0000 7f40a1262640 1 -- 172.21.15.90:0/766546803 wait complete. 2024-04-26T18:22:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:28 smithi119 ceph-mon[32752]: pgmap v337: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:28 smithi119 ceph-mon[32752]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T18:22:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:28 smithi167 ceph-mon[32751]: pgmap v337: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:28 smithi167 ceph-mon[32751]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T18:22:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:28 smithi090 ceph-mon[29180]: pgmap v337: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 740 KiB/s wr, 113 op/s 2024-04-26T18:22:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:28 smithi090 ceph-mon[29180]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T18:22:29.692 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.26 MB/sec execute 377 sec latency 8033.214 ms 2024-04-26T18:22:30.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.25 MB/sec execute 378 sec latency 9033.309 ms 2024-04-26T18:22:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:30 smithi119 ceph-mon[32752]: pgmap v339: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 834 KiB/s wr, 121 op/s 2024-04-26T18:22:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:30 smithi167 ceph-mon[32751]: pgmap v339: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 834 KiB/s wr, 121 op/s 2024-04-26T18:22:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:30 smithi090 ceph-mon[29180]: pgmap v339: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 834 KiB/s wr, 121 op/s 2024-04-26T18:22:31.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.24 MB/sec execute 379 sec latency 10033.343 ms 2024-04-26T18:22:32.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.23 MB/sec execute 380 sec latency 11033.485 ms 2024-04-26T18:22:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:32 smithi119 ceph-mon[32752]: pgmap v340: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 745 KiB/s wr, 108 op/s 2024-04-26T18:22:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:32 smithi167 ceph-mon[32751]: pgmap v340: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 745 KiB/s wr, 108 op/s 2024-04-26T18:22:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:32 smithi090 ceph-mon[29180]: pgmap v340: 97 pgs: 97 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 745 KiB/s wr, 108 op/s 2024-04-26T18:22:33.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.22 MB/sec execute 381 sec latency 12033.574 ms 2024-04-26T18:22:33.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc549a95700 time 2024-04-26T18:22:33.387553+0000 2024-04-26T18:22:33.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:33.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc5540be3dd] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.805 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.806 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: clone() 2024-04-26T18:22:33.806 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: *** Caught signal (Aborted) ** 2024-04-26T18:22:33.806 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: in thread 7fc549a95700 thread_name:ms_dispatch 2024-04-26T18:22:33.806 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.387+0000 7fc549a95700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc549a95700 time 2024-04-26T18:22:33.387553+0000 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc5540be3dd] 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.807 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: clone() 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc552e46d20] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: gsignal() 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: abort() 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc5540be437] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.808 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 16: clone() 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.389+0000 7fc549a95700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: in thread 7fc549a95700 thread_name:ms_dispatch 2024-04-26T18:22:33.809 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc552e46d20] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: gsignal() 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: abort() 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc5540be437] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.810 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 16: clone() 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: -1> 2024-04-26T18:22:33.387+0000 7fc549a95700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc549a95700 time 2024-04-26T18:22:33.387553+0000 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc5540be3dd] 2024-04-26T18:22:33.811 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: clone() 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 0> 2024-04-26T18:22:33.389+0000 7fc549a95700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:22:33.812 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: in thread 7fc549a95700 thread_name:ms_dispatch 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc552e46d20] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: gsignal() 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: abort() 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc5540be437] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.813 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 16: clone() 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: -1> 2024-04-26T18:22:33.387+0000 7fc549a95700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc549a95700 time 2024-04-26T18:22:33.387553+0000 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc5540be3dd] 2024-04-26T18:22:33.814 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.815 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: clone() 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 0> 2024-04-26T18:22:33.389+0000 7fc549a95700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: in thread 7fc549a95700 thread_name:ms_dispatch 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc552e46d20] 2024-04-26T18:22:33.816 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2: gsignal() 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 3: abort() 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc5540be437] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc5540be5a3] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d05c39822] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d05ba9598] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d05babc17] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d05aa4d0b] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d05aa52ac] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d05a8dfad] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc554352e20] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 13: (DispatchQueue::entry()+0x51f) [0x7fc55434ffdf] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc554422fc1] 2024-04-26T18:22:33.817 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc552e3c1ca] 2024-04-26T18:22:33.818 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 16: clone() 2024-04-26T18:22:33.818 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:22:33.818 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[68929]: 2024-04-26T18:22:33.818 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:33 smithi119 podman[69558]: 2024-04-26 18:22:33.454737817 +0000 UTC m=+0.022279700 container died 89c178a3a0f86a2b893e315d8d28ac6fab8c8b425cb4d07aeeda5589ef9648b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:22:34.377 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:34 smithi119 podman[69558]: 2024-04-26 18:22:34.127692597 +0000 UTC m=+0.695234480 container remove 89c178a3a0f86a2b893e315d8d28ac6fab8c8b425cb4d07aeeda5589ef9648b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5) 2024-04-26T18:22:34.377 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:34 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:22:34.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.21 MB/sec execute 382 sec latency 13033.707 ms 2024-04-26T18:22:34.803 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:34 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-26T18:22:34.804 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:34 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.792s CPU time. 2024-04-26T18:22:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:34 smithi119 ceph-mon[32752]: pgmap v341: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:34 smithi167 ceph-mon[32751]: pgmap v341: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:34 smithi090 ceph-mon[29180]: pgmap v341: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:35.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.20 MB/sec execute 383 sec latency 14033.744 ms 2024-04-26T18:22:35.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:35 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:35 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:35 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:36.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.20 MB/sec execute 384 sec latency 15033.830 ms 2024-04-26T18:22:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:36 smithi119 ceph-mon[32752]: pgmap v342: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:36 smithi167 ceph-mon[32751]: pgmap v342: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:36 smithi090 ceph-mon[29180]: pgmap v342: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:37.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.19 MB/sec execute 385 sec latency 16033.932 ms 2024-04-26T18:22:38.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.18 MB/sec execute 386 sec latency 17034.040 ms 2024-04-26T18:22:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:38 smithi119 ceph-mon[32752]: pgmap v343: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:38 smithi167 ceph-mon[32751]: pgmap v343: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:38 smithi090 ceph-mon[29180]: pgmap v343: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 1.9 MiB/s wr, 54 op/s 2024-04-26T18:22:39.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.17 MB/sec execute 387 sec latency 18034.076 ms 2024-04-26T18:22:40.693 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.16 MB/sec execute 388 sec latency 19034.169 ms 2024-04-26T18:22:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:40 smithi119 ceph-mon[32752]: pgmap v344: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.6 MiB/s wr, 48 op/s 2024-04-26T18:22:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:40 smithi167 ceph-mon[32751]: pgmap v344: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.6 MiB/s wr, 48 op/s 2024-04-26T18:22:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:40 smithi090 ceph-mon[29180]: pgmap v344: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.6 MiB/s wr, 48 op/s 2024-04-26T18:22:41.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.15 MB/sec execute 389 sec latency 20034.296 ms 2024-04-26T18:22:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.894+0000 7efd1f724640 1 -- 172.21.15.90:0/1454565490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd181044e0 msgr2=0x7efd1810c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.894+0000 7efd1f724640 1 --2- 172.21.15.90:0/1454565490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd181044e0 0x7efd1810c9c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efd0c009ca0 tx=0x7efd0c02a9e0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.894+0000 7efd1f724640 1 -- 172.21.15.90:0/1454565490 shutdown_connections 2024-04-26T18:22:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.894+0000 7efd1f724640 1 --2- 172.21.15.90:0/1454565490 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1810cf00 0x7efd1810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.894+0000 7efd1f724640 1 --2- 172.21.15.90:0/1454565490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd181044e0 0x7efd1810c9c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.894+0000 7efd1f724640 1 --2- 172.21.15.90:0/1454565490 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd18103bc0 0x7efd18103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.894+0000 7efd1f724640 1 -- 172.21.15.90:0/1454565490 >> 172.21.15.90:0/1454565490 conn(0x7efd180789b0 msgr2=0x7efd18078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 -- 172.21.15.90:0/1454565490 shutdown_connections 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 -- 172.21.15.90:0/1454565490 wait complete. 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 Processor -- start 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 -- start start 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd18103bc0 0x7efd18199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd181044e0 0x7efd18199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1810cf00 0x7efd1819d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd18112bb0 con 0x7efd18103bc0 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd18112a30 con 0x7efd181044e0 2024-04-26T18:22:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1f724640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd18112d30 con 0x7efd1810cf00 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.895+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd18103bc0 0x7efd18199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd18103bc0 0x7efd18199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40990/0 (socket says 172.21.15.90:40990) 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 -- 172.21.15.90:0/411445947 learned_addr learned my addr 172.21.15.90:0/411445947 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd181044e0 0x7efd18199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1810cf00 0x7efd1819d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 -- 172.21.15.90:0/411445947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1810cf00 msgr2=0x7efd1819d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1810cf00 0x7efd1819d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 -- 172.21.15.90:0/411445947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd181044e0 msgr2=0x7efd18199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd181044e0 0x7efd18199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 -- 172.21.15.90:0/411445947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd1819e0f0 con 0x7efd18103bc0 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1810cf00 0x7efd1819d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd181044e0 0x7efd18199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:22:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd18103bc0 0x7efd18199150 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efd1000c920 tx=0x7efd1000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd167fc640 1 -- 172.21.15.90:0/411445947 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10007c70 con 0x7efd18103bc0 2024-04-26T18:22:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd167fc640 1 -- 172.21.15.90:0/411445947 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd10007e10 con 0x7efd18103bc0 2024-04-26T18:22:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd167fc640 1 -- 172.21.15.90:0/411445947 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10018820 con 0x7efd18103bc0 2024-04-26T18:22:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd1819e3e0 con 0x7efd18103bc0 2024-04-26T18:22:41.900 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.896+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd181a5c70 con 0x7efd18103bc0 2024-04-26T18:22:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.898+0000 7efd167fc640 1 -- 172.21.15.90:0/411445947 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7efd1000f450 con 0x7efd18103bc0 2024-04-26T18:22:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 Processor -- start 2024-04-26T18:22:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 -- start start 2024-04-26T18:22:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd167fc640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efd000a3e40 0x7efd000a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f8856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f885c30 0x55e87f88ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f8904f0 0x55e87f8928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e87f7e7460 con 0x55e87f8904f0 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e87f7a1590 con 0x55e87f885c30 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd22443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e87f7e3420 con 0x55e87f885310 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efd000a3e40 0x7efd000a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd167fc640 1 -- 172.21.15.90:0/411445947 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9229+0+0 (secure 0 0 0) 0x7efd100d0c40 con 0x7efd18103bc0 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.899+0000 7efd1dcc1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f8904f0 0x55e87f8928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f8856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f8904f0 0x55e87f8928e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40998/0 (socket says 172.21.15.90:40998) 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1ccbf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f885c30 0x55e87f88ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 -- 172.21.15.90:0/3531944013 learned_addr learned my addr 172.21.15.90:0/3531944013 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f8856f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:41920/0 (socket says 172.21.15.90:41920) 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 -- 172.21.15.90:0/3531944013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 msgr2=0x55e87f8856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f8856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 -- 172.21.15.90:0/3531944013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f885c30 msgr2=0x55e87f88ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f885c30 0x55e87f88ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 -- 172.21.15.90:0/3531944013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87f7ae0d0 con 0x55e87f8904f0 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f8856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efd000a3e40 0x7efd000a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efd080137e0 tx=0x7efd0800ba20 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f8904f0 0x55e87f8928e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efd08004640 tx=0x7efd0801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410080d10c1178b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efcff7fe640 1 -- 172.21.15.90:0/3531944013 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0800e760 con 0x55e87f8904f0 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efcff7fe640 1 -- 172.21.15.90:0/3531944013 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd0800edb0 con 0x55e87f8904f0 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.900+0000 7efcff7fe640 1 -- 172.21.15.90:0/3531944013 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0801fd20 con 0x55e87f8904f0 2024-04-26T18:22:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 -- 172.21.15.90:0/3531944013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f8904f0 msgr2=0x55e87f8928e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f8904f0 0x55e87f8928e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7efd08004640 tx=0x7efd0801a000 comp rx=0 tx=0).stop 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 -- 172.21.15.90:0/3531944013 shutdown_connections 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f8904f0 0x55e87f8928e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f885c30 0x55e87f88ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 --2- 172.21.15.90:0/3531944013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f8856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 -- 172.21.15.90:0/3531944013 >> 172.21.15.90:0/3531944013 conn(0x55e87f881820 msgr2=0x55e87f881c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 -- 172.21.15.90:0/3531944013 shutdown_connections 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 -- 172.21.15.90:0/3531944013 wait complete. 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 -- start start 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.901+0000 7efd22443740 1 Processor -- start 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd22443740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f88d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd22443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f88d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd22443740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd22443740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55e87f8930c0 con 0x55e87f885310 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd22443740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55e87f76eb60 con 0x55e87f8904f0 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd22443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e87f893240 con 0x55e87f885c30 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd22443740 10 client.0 _ll_register_callbacks cb 0x55e87f7932b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f88d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f88d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41006/0 (socket says 172.21.15.90:41006) 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 -- 172.21.15.90:0/3497061957 learned_addr learned my addr 172.21.15.90:0/3497061957 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f88d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 -- 172.21.15.90:0/3497061957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 msgr2=0x55e87f88d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f88d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 -- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 msgr2=0x55e87f88dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.902+0000 7efd1d4c0640 1 -- 172.21.15.90:0/3497061957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87f4f9210 con 0x55e87f885310 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f88d440 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efd10098d30 tx=0x7efd1009c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8cf4028d9fa35c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efcfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efcfcff9640 1 -- 172.21.15.90:0/3497061957 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10085e60 con 0x55e87f885310 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efcfcff9640 1 -- 172.21.15.90:0/3497061957 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9229+0+0 (secure 0 0 0) 0x7efd100be300 con 0x55e87f885310 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd0800edb0 con 0x55e87f885310 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efcfcff9640 1 -- 172.21.15.90:0/3497061957 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7efd1000f740 con 0x55e87f885310 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 10 client.16302 fetch_fsmap learned FSMap version 90 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55e87f5c5620 con 0x55e87f885310 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efcfcff9640 1 -- 172.21.15.90:0/3497061957 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd0800edb0 con 0x55e87f885310 2024-04-26T18:22:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efcfcff9640 1 -- 172.21.15.90:0/3497061957 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 90) ==== 7745+0+0 (secure 0 0 0) 0x7efd100870b0 con 0x55e87f885310 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 10 client.16302 fetch_fsmap finished waiting for FSMap version 90 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 20 client.16302 populate_metadata read hostname 'smithi090' 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f8a06b0 0x55e87f8a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 4 client.16302 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.903+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e87f7f5070 con 0x55e87f8a06b0 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.904+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f8a06b0 0x55e87f8a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.904+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f8a06b0 0x55e87f8a2aa0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72219d6a31467bf4 server_cookie=46707b74ebb1ddac in_seq=0 out_seq=0 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.904+0000 7efcfcff9640 10 client.16302 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efcfcff9640 1 -- 172.21.15.90:0/3497061957 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e87f7f5070 con 0x55e87f8a06b0 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efcfcff9640 10 client.16302 handle_command_reply: tid=0 2024-04-26T18:22:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 client.16302 shutdown 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 msgr2=0x55e87f88d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f88d440 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7efd10098d30 tx=0x7efd1009c9e0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 shutdown_connections 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f8a06b0 0x55e87f8a2aa0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f88d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 --2- 172.21.15.90:0/3497061957 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f88d440 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efcfcff9640 0 client.16302 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 >> 172.21.15.90:0/3497061957 conn(0x55e87f881820 msgr2=0x55e87f87c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 shutdown_connections 2024-04-26T18:22:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 1 -- 172.21.15.90:0/3497061957 wait complete. 2024-04-26T18:22:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.905+0000 7efd22443740 20 client.16302 trim_cache size 0 max 16384 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 Processor -- start 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 -- start start 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f89bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f887d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f8a8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd181ae490 con 0x55e87f885310 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd181aeb60 con 0x55e87f8904f0 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd22443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd181ac410 con 0x55e87f885c30 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f89bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd1d4c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f89bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41010/0 (socket says 172.21.15.90:41010) 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd1d4c0640 1 -- 172.21.15.90:0/1525005148 learned_addr learned my addr 172.21.15.90:0/1525005148 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f887d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.920+0000 7efd1d4c0640 1 -- 172.21.15.90:0/1525005148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 msgr2=0x55e87f887d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f887d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f8a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd1d4c0640 1 -- 172.21.15.90:0/1525005148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 msgr2=0x55e87f8a8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f8a8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd1d4c0640 1 -- 172.21.15.90:0/1525005148 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e87f8a2e10 con 0x55e87f885310 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f89bfd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efd1009c620 tx=0x7efd100a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f9b67a5890ccac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efcfeffd640 1 -- 172.21.15.90:0/1525005148 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10085e60 con 0x55e87f885310 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efcfeffd640 1 -- 172.21.15.90:0/1525005148 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd100a0cd0 con 0x55e87f885310 2024-04-26T18:22:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efcfeffd640 1 -- 172.21.15.90:0/1525005148 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd10098670 con 0x55e87f885310 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 -- 172.21.15.90:0/1525005148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 msgr2=0x55e87f89bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f89bfd0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efd1009c620 tx=0x7efd100a5710 comp rx=0 tx=0).stop 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 -- 172.21.15.90:0/1525005148 shutdown_connections 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f8a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885c30 0x55e87f887d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 --2- 172.21.15.90:0/1525005148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885310 0x55e87f89bfd0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 -- 172.21.15.90:0/1525005148 >> 172.21.15.90:0/1525005148 conn(0x55e87f881820 msgr2=0x55e87f87c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.921+0000 7efd22443740 1 -- 172.21.15.90:0/1525005148 shutdown_connections 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 -- 172.21.15.90:0/1525005148 wait complete. 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 -- start start 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 Processor -- start 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f88a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885c30 0x55e87f88a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efd181abd40 con 0x55e87f885c30 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efd181aa950 con 0x55e87f8904f0 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efd181aafa0 con 0x55e87f885310 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd22443740 10 client.0 _ll_register_callbacks cb 0x7efd1009d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd1ccbf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885c30 0x55e87f88a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd1ccbf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885c30 0x55e87f88a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:41026/0 (socket says 172.21.15.90:41026) 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd1ccbf640 1 -- 172.21.15.90:0/1915481945 learned_addr learned my addr 172.21.15.90:0/1915481945 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.922+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f88a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd1dcc1640 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd1ccbf640 1 -- 172.21.15.90:0/1915481945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 msgr2=0x55e87f88a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f88a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd1ccbf640 1 -- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 msgr2=0x55e87f88ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd1ccbf640 1 -- 172.21.15.90:0/1915481945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e87f905bc0 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd1ccbf640 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885c30 0x55e87f88a610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efd1810ed20 tx=0x7efd0c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7105e88331b1fb0b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efcfcff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efcfcff9640 1 -- 172.21.15.90:0/1915481945 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd0c04c070 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd100a0cd0 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efcfcff9640 1 -- 172.21.15.90:0/1915481945 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9229+0+0 (secure 0 0 0) 0x7efd0c02e590 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efcfcff9640 1 -- 172.21.15.90:0/1915481945 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7efd0c058050 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd22443740 10 client.16308 fetch_fsmap learned FSMap version 90 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x55e87f906f60 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efcfcff9640 1 -- 172.21.15.90:0/1915481945 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd100a0cd0 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efcfcff9640 1 -- 172.21.15.90:0/1915481945 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 90) ==== 7745+0+0 (secure 0 0 0) 0x7efd0c004200 con 0x55e87f885c30 2024-04-26T18:22:41.929 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd22443740 10 client.16308 fetch_fsmap finished waiting for FSMap version 90 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd22443740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd22443740 20 client.16308 populate_metadata read hostname 'smithi090' 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.923+0000 7efd22443740 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f9076c0 0x55e87f909ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.924+0000 7efd22443740 4 client.16308 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.924+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e87f7eeee0 con 0x55e87f9076c0 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.924+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f9076c0 0x55e87f909ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.924+0000 7efcfcff9640 10 client.16308 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:41.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:41.924+0000 7efd1d4c0640 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f9076c0 0x55e87f909ae0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4002c31feaa6cddc server_cookie=a078331be4e8cbf in_seq=0 out_seq=0 2024-04-26T18:22:42.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.15 MB/sec execute 390 sec latency 21034.374 ms 2024-04-26T18:22:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:42 smithi119 ceph-mon[32752]: pgmap v345: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.6 MiB/s wr, 36 op/s 2024-04-26T18:22:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:42 smithi167 ceph-mon[32751]: pgmap v345: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.6 MiB/s wr, 36 op/s 2024-04-26T18:22:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:42 smithi090 ceph-mon[29180]: pgmap v345: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.6 MiB/s wr, 36 op/s 2024-04-26T18:22:43.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.14 MB/sec execute 391 sec latency 22034.433 ms 2024-04-26T18:22:44.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.13 MB/sec execute 392 sec latency 23034.518 ms 2024-04-26T18:22:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:44 smithi167 ceph-mon[32751]: pgmap v346: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.6 MiB/s wr, 44 op/s 2024-04-26T18:22:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:44 smithi090 ceph-mon[29180]: pgmap v346: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.6 MiB/s wr, 44 op/s 2024-04-26T18:22:44.900 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:44 smithi119 ceph-mon[32752]: pgmap v346: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.6 MiB/s wr, 44 op/s 2024-04-26T18:22:44.900 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:44 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:22:44.900 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:44 smithi119 systemd[1]: Stopped Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:22:44.900 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:44 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 3.792s CPU time. 2024-04-26T18:22:44.900 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:44 smithi119 systemd[1]: Starting Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:22:44.901 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:44 smithi119 podman[69656]: 2024-04-26 18:22:44.696051818 +0000 UTC m=+0.018435452 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:22:44.901 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:44 smithi119 podman[69656]: 2024-04-26 18:22:44.79493264 +0000 UTC m=+0.117316265 container create 5756ed1a1238854e21ddb48dbf30b741a01be5e2c475530a37b31ae5745f2b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:22:45.163 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:45 smithi119 podman[69656]: 2024-04-26 18:22:45.013992073 +0000 UTC m=+0.336375697 container init 5756ed1a1238854e21ddb48dbf30b741a01be5e2c475530a37b31ae5745f2b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:22:45.163 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:45 smithi119 podman[69656]: 2024-04-26 18:22:45.017355495 +0000 UTC m=+0.339739129 container start 5756ed1a1238854e21ddb48dbf30b741a01be5e2c475530a37b31ae5745f2b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T18:22:45.163 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mds[69676]: -- 172.21.15.119:0/2102453191 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad6b525a40 con 0x55ad6b4f8800 2024-04-26T18:22:45.163 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: starting mds.h at 2024-04-26T18:22:45.163 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:45 smithi119 bash[69656]: 5756ed1a1238854e21ddb48dbf30b741a01be5e2c475530a37b31ae5745f2b08 2024-04-26T18:22:45.163 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:22:45 smithi119 systemd[1]: Started Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:22:45.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.12 MB/sec execute 393 sec latency 24034.635 ms 2024-04-26T18:22:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: daemon mds.h restarted 2024-04-26T18:22:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:22:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: osdmap e113: 12 total, 12 up, 12 in 2024-04-26T18:22:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:boot 2024-04-26T18:22:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:22:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:22:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:22:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:45 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: daemon mds.h restarted 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: osdmap e113: 12 total, 12 up, 12 in 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:boot 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:22:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:45 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: daemon mds.h restarted 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: osdmap e113: 12 total, 12 up, 12 in 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:boot 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:45 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:46.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.11 MB/sec execute 394 sec latency 25034.683 ms 2024-04-26T18:22:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:47 smithi167 ceph-mon[32751]: pgmap v348: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:22:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:resolve 2024-04-26T18:22:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:47 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:47 smithi090 ceph-mon[29180]: pgmap v348: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:22:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:resolve 2024-04-26T18:22:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:47 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:47 smithi119 ceph-mon[32752]: pgmap v348: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:22:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:resolve 2024-04-26T18:22:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:47 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:47.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.11 MB/sec execute 395 sec latency 26034.766 ms 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efcfcff9640 1 -- 172.21.15.90:0/1915481945 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e87f7eeee0 con 0x55e87f9076c0 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efcfcff9640 10 client.16308 handle_command_reply: tid=0 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 client.16308 shutdown 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885c30 msgr2=0x55e87f88a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885c30 0x55e87f88a610 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7efd1810ed20 tx=0x7efd0c003ba0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 shutdown_connections 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55e87f9076c0 0x55e87f909ae0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e87f8904f0 0x55e87f88ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55e87f885c30 0x55e87f88a610 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 --2- 172.21.15.90:0/1915481945 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e87f885310 0x55e87f88a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efcfcff9640 0 client.16308 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 >> 172.21.15.90:0/1915481945 conn(0x55e87f881820 msgr2=0x55e87f88d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 shutdown_connections 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 1 -- 172.21.15.90:0/1915481945 wait complete. 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.584+0000 7efd22443740 20 client.16308 trim_cache size 0 max 16384 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:22:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efd000a3e40 msgr2=0x7efd000a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efd000a3e40 0x7efd000a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efd080137e0 tx=0x7efd0800ba20 comp rx=0 tx=0).stop 2024-04-26T18:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd18103bc0 msgr2=0x7efd18199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd18103bc0 0x7efd18199150 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7efd1000c920 tx=0x7efd1000cde0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 shutdown_connections 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efd000a3e40 0x7efd000a6300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efd1810cf00 0x7efd1819d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd181044e0 0x7efd18199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 --2- 172.21.15.90:0/411445947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efd18103bc0 0x7efd18199150 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 >> 172.21.15.90:0/411445947 conn(0x7efd180789b0 msgr2=0x7efd180ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 shutdown_connections 2024-04-26T18:22:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:22:48.585+0000 7efd1f724640 1 -- 172.21.15.90:0/411445947 wait complete. 2024-04-26T18:22:48.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458773 3.10 MB/sec execute 396 sec latency 27034.898 ms 2024-04-26T18:22:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:48 smithi090 ceph-mon[29180]: pgmap v349: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:22:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:48 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:22:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:48 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:reconnect 2024-04-26T18:22:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:48 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:48 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:22:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:48 smithi119 ceph-mon[32752]: pgmap v349: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:22:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:48 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:22:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:48 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:reconnect 2024-04-26T18:22:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:48 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:48 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:22:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:48 smithi167 ceph-mon[32751]: pgmap v349: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:22:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:48 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:22:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:48 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:reconnect 2024-04-26T18:22:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:48 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:48 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:22:49.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 458857 3.10 MB/sec execute 397 sec latency 27931.041 ms 2024-04-26T18:22:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:49 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:rejoin 2024-04-26T18:22:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:49 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:49 smithi090 ceph-mon[29180]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:22:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:49 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:rejoin 2024-04-26T18:22:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:49 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:49 smithi119 ceph-mon[32752]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:22:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:49 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:rejoin 2024-04-26T18:22:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:49 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:49 smithi167 ceph-mon[32751]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:22:50.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 461317 3.13 MB/sec execute 398 sec latency 10.780 ms 2024-04-26T18:22:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:50 smithi090 ceph-mon[29180]: pgmap v350: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:22:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:50 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:22:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:50 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:active 2024-04-26T18:22:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:50 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:50 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:50 smithi119 ceph-mon[32752]: pgmap v350: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:22:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:50 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:22:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:50 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:active 2024-04-26T18:22:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:50 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:50 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:50 smithi167 ceph-mon[32751]: pgmap v350: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:22:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:50 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:22:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:50 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2611460544,v1:172.21.15.119:6835/2611460544] up:active 2024-04-26T18:22:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:50 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:22:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:50 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:22:51.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 462704 3.14 MB/sec execute 399 sec latency 13.358 ms 2024-04-26T18:22:52.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 465591 3.17 MB/sec execute 400 sec latency 16.175 ms 2024-04-26T18:22:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:52 smithi090 ceph-mon[29180]: pgmap v351: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:22:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:52 smithi119 ceph-mon[32752]: pgmap v351: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:22:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:52 smithi167 ceph-mon[32751]: pgmap v351: 97 pgs: 97 active+clean; 133 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:22:53.694 INFO:tasks.workunit.client.0.smithi090.stdout: 1 466942 3.18 MB/sec execute 401 sec latency 14.471 ms 2024-04-26T18:22:54.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.21 MB/sec execute 402 sec latency 97.462 ms 2024-04-26T18:22:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:54 smithi090 ceph-mon[29180]: pgmap v352: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 751 KiB/s wr, 99 op/s 2024-04-26T18:22:55.018 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-mon[32752]: pgmap v352: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 751 KiB/s wr, 99 op/s 2024-04-26T18:22:55.018 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3742ed1700 time 2024-04-26T18:22:54.597400+0000 2024-04-26T18:22:55.018 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f374d4fa3dd] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.019 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.020 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: clone() 2024-04-26T18:22:55.020 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:54.597+0000 7f3742ed1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3742ed1700 time 2024-04-26T18:22:54.597400+0000 2024-04-26T18:22:55.020 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:55.020 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f374d4fa3dd] 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.021 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: clone() 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: *** Caught signal (Aborted) ** 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: in thread 7f3742ed1700 thread_name:ms_dispatch 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f374c282d20] 2024-04-26T18:22:55.022 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: gsignal() 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: abort() 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f374d4fa437] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.023 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 16: clone() 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:54.599+0000 7f3742ed1700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: in thread 7f3742ed1700 thread_name:ms_dispatch 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f374c282d20] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: gsignal() 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: abort() 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f374d4fa437] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.024 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 16: clone() 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: -1> 2024-04-26T18:22:54.597+0000 7f3742ed1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3742ed1700 time 2024-04-26T18:22:54.597400+0000 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.025 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f374d4fa3dd] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: clone() 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.026 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 0> 2024-04-26T18:22:54.599+0000 7f3742ed1700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: in thread 7f3742ed1700 thread_name:ms_dispatch 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f374c282d20] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: gsignal() 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: abort() 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f374d4fa437] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.027 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 16: clone() 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: -1> 2024-04-26T18:22:54.597+0000 7f3742ed1700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3742ed1700 time 2024-04-26T18:22:54.597400+0000 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f374d4fa3dd] 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.028 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: clone() 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 0> 2024-04-26T18:22:54.599+0000 7f3742ed1700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: in thread 7f3742ed1700 thread_name:ms_dispatch 2024-04-26T18:22:55.029 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f374c282d20] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2: gsignal() 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 3: abort() 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f374d4fa437] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f374d4fa5a3] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556dd22d0822] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556dd2240598] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556dd2242c17] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556dd213bd0b] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556dd213c2ac] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556dd2124fad] 2024-04-26T18:22:55.030 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f374d78ee20] 2024-04-26T18:22:55.031 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 13: (DispatchQueue::entry()+0x51f) [0x7f374d78bfdf] 2024-04-26T18:22:55.031 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f374d85efc1] 2024-04-26T18:22:55.031 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f374c2781ca] 2024-04-26T18:22:55.031 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 16: clone() 2024-04-26T18:22:55.031 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:22:55.031 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[62903]: 2024-04-26T18:22:55.031 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:54 smithi119 podman[69725]: 2024-04-26 18:22:54.65986226 +0000 UTC m=+0.018987915 container died a1d026342b5d3d2ae7aa4b6dbf41c6057d7a517f46184dd3feba545931bfa03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T18:22:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:54 smithi167 ceph-mon[32751]: pgmap v352: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 751 KiB/s wr, 99 op/s 2024-04-26T18:22:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:55 smithi119 podman[69725]: 2024-04-26 18:22:55.280142879 +0000 UTC m=+0.639268526 container remove a1d026342b5d3d2ae7aa4b6dbf41c6057d7a517f46184dd3feba545931bfa03b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:22:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:55 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:22:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:55 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-26T18:22:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:22:55 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 3.907s CPU time. 2024-04-26T18:22:55.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.20 MB/sec execute 403 sec latency 1097.580 ms 2024-04-26T18:22:56.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.19 MB/sec execute 404 sec latency 2097.669 ms 2024-04-26T18:22:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:56 smithi090 ceph-mon[29180]: pgmap v353: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 717 KiB/s wr, 94 op/s 2024-04-26T18:22:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:56 smithi119 ceph-mon[32752]: pgmap v353: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 717 KiB/s wr, 94 op/s 2024-04-26T18:22:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:56 smithi167 ceph-mon[32751]: pgmap v353: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 717 KiB/s wr, 94 op/s 2024-04-26T18:22:57.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.19 MB/sec execute 405 sec latency 3097.734 ms 2024-04-26T18:22:58.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.18 MB/sec execute 406 sec latency 4097.867 ms 2024-04-26T18:22:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:22:58 smithi090 ceph-mon[29180]: pgmap v354: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 626 KiB/s wr, 82 op/s 2024-04-26T18:22:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:22:58 smithi119 ceph-mon[32752]: pgmap v354: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 626 KiB/s wr, 82 op/s 2024-04-26T18:22:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:22:58 smithi167 ceph-mon[32751]: pgmap v354: 97 pgs: 97 active+clean; 140 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 626 KiB/s wr, 82 op/s 2024-04-26T18:22:59.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.17 MB/sec execute 407 sec latency 5097.971 ms 2024-04-26T18:23:00.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.16 MB/sec execute 408 sec latency 6098.038 ms 2024-04-26T18:23:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:00 smithi090 ceph-mon[29180]: pgmap v355: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 903 KiB/s wr, 117 op/s 2024-04-26T18:23:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:00 smithi119 ceph-mon[32752]: pgmap v355: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 903 KiB/s wr, 117 op/s 2024-04-26T18:23:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:00 smithi167 ceph-mon[32751]: pgmap v355: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 903 KiB/s wr, 117 op/s 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 -- 172.21.15.90:0/528278115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 msgr2=0x7fac48101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 --2- 172.21.15.90:0/528278115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac48101db0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fac44009930 tx=0x7fac4402a4d0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 -- 172.21.15.90:0/528278115 shutdown_connections 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 --2- 172.21.15.90:0/528278115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac48106dd0 0x7fac48107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 --2- 172.21.15.90:0/528278115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481022f0 0x7fac481067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 --2- 172.21.15.90:0/528278115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac48101db0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 -- 172.21.15.90:0/528278115 >> 172.21.15.90:0/528278115 conn(0x7fac48077bb0 msgr2=0x7fac48077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 -- 172.21.15.90:0/528278115 shutdown_connections 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.683+0000 7fac4f637640 1 -- 172.21.15.90:0/528278115 wait complete. 2024-04-26T18:23:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 Processor -- start 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 -- start start 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac4819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481022f0 0x7fac4819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac48106dd0 0x7fac481a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac481191d0 con 0x7fac481019d0 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac48119050 con 0x7fac48106dd0 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4f637640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac48119350 con 0x7fac481022f0 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4d3d3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac4819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4d3d3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac4819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36094/0 (socket says 172.21.15.90:36094) 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4d3d3640 1 -- 172.21.15.90:0/8423647 learned_addr learned my addr 172.21.15.90:0/8423647 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481022f0 0x7fac4819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.684+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac48106dd0 0x7fac481a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4d3d3640 1 -- 172.21.15.90:0/8423647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481022f0 msgr2=0x7fac4819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481022f0 0x7fac4819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4d3d3640 1 -- 172.21.15.90:0/8423647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac48106dd0 msgr2=0x7fac481a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac48106dd0 0x7fac481a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4d3d3640 1 -- 172.21.15.90:0/8423647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac481a4670 con 0x7fac481019d0 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac48106dd0 0x7fac481a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac4819f6d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fac44004770 tx=0x7fac440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac3e7fc640 1 -- 172.21.15.90:0/8423647 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4403f070 con 0x7fac481019d0 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac3e7fc640 1 -- 172.21.15.90:0/8423647 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4403b460 con 0x7fac481019d0 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac481a4900 con 0x7fac481019d0 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac3e7fc640 1 -- 172.21.15.90:0/8423647 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44035070 con 0x7fac481019d0 2024-04-26T18:23:01.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.685+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac481ac1f0 con 0x7fac481019d0 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.687+0000 7fac3e7fc640 1 -- 172.21.15.90:0/8423647 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fac4403b8d0 con 0x7fac481019d0 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 Processor -- start 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 -- start start 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x5646899532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5646898b9460 con 0x564689952f00 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564689873590 con 0x56468995e0e0 2024-04-26T18:23:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac5242d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5646898b5420 con 0x564689953820 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac3e7fc640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac340a3dc0 0x7fac340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x5646899532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x5646899532e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36098/0 (socket says 172.21.15.90:36098) 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 -- 172.21.15.90:0/298600781 learned_addr learned my addr 172.21.15.90:0/298600781 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac340a3dc0 0x7fac340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac3e7fc640 1 -- 172.21.15.90:0/8423647 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9271+0+0 (secure 0 0 0) 0x7fac440f0ee0 con 0x7fac481019d0 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 -- 172.21.15.90:0/298600781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 msgr2=0x56468995dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 -- 172.21.15.90:0/298600781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 msgr2=0x5646899604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4dbd4640 1 -- 172.21.15.90:0/298600781 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646898800d0 con 0x564689952f00 2024-04-26T18:23:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.688+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.689+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x5646899532e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fac400092f0 tx=0x7fac40011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d7e451b44172fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.689+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac340a3dc0 0x7fac340a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fac40011c50 tx=0x7fac40016000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.689+0000 7fac337fe640 1 -- 172.21.15.90:0/298600781 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40022070 con 0x564689952f00 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.689+0000 7fac337fe640 1 -- 172.21.15.90:0/298600781 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac40009bc0 con 0x564689952f00 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.689+0000 7fac337fe640 1 -- 172.21.15.90:0/298600781 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40005310 con 0x564689952f00 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- 172.21.15.90:0/298600781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 msgr2=0x5646899532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x5646899532e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fac400092f0 tx=0x7fac40011750 comp rx=0 tx=0).stop 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- 172.21.15.90:0/298600781 shutdown_connections 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 --2- 172.21.15.90:0/298600781 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x5646899532e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- 172.21.15.90:0/298600781 >> 172.21.15.90:0/298600781 conn(0x56468994f410 msgr2=0x56468994f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- 172.21.15.90:0/298600781 shutdown_connections 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- 172.21.15.90:0/298600781 wait complete. 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- start start 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 Processor -- start 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x56468995b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x56468995bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x564689960cb0 con 0x564689952f00 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x564689840b60 con 0x56468995e0e0 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.690+0000 7fac5242d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564689960e30 con 0x564689953820 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac5242d740 10 client.0 _ll_register_callbacks cb 0x5646898652b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x56468995b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x56468995b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36112/0 (socket says 172.21.15.90:36112) 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 -- 172.21.15.90:0/3820674785 learned_addr learned my addr 172.21.15.90:0/3820674785 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x56468995bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 -- 172.21.15.90:0/3820674785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 msgr2=0x56468995b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 -- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 msgr2=0x56468995bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x56468995bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 -- 172.21.15.90:0/3820674785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646895cb210 con 0x564689952f00 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x56468995b150 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fac440ee940 tx=0x7fac440bca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1316a1e4199af63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac30ff9640 1 -- 172.21.15.90:0/3820674785 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac44048040 con 0x564689952f00 2024-04-26T18:23:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac30ff9640 1 -- 172.21.15.90:0/3820674785 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9271+0+0 (secure 0 0 0) 0x7fac440ee5c0 con 0x564689952f00 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac40009bc0 con 0x564689952f00 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac30ff9640 1 -- 172.21.15.90:0/3820674785 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fac440bd290 con 0x564689952f00 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac5242d740 10 client.16323 fetch_fsmap learned FSMap version 97 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.691+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=4+}) -- 0x564689697620 con 0x564689952f00 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac30ff9640 1 -- 172.21.15.90:0/3820674785 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac40009bc0 con 0x564689952f00 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac30ff9640 1 -- 172.21.15.90:0/3820674785 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 97) ==== 7741+0+0 (secure 0 0 0) 0x7fac4404b060 con 0x564689952f00 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac5242d740 10 client.16323 fetch_fsmap finished waiting for FSMap version 97 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac5242d740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac5242d740 20 client.16323 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac5242d740 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56468996e290 0x564689970700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac5242d740 4 client.16323 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646898c7070 con 0x56468996e290 2024-04-26T18:23:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.692+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56468996e290 0x564689970700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac30ff9640 10 client.16323 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56468996e290 0x564689970700 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faa1d5884310662e server_cookie=88f5413d17496aa9 in_seq=0 out_seq=0 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac30ff9640 1 -- 172.21.15.90:0/3820674785 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646898c7070 con 0x56468996e290 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac30ff9640 10 client.16323 handle_command_reply: tid=0 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 client.16323 shutdown 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 msgr2=0x56468995b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x56468995b150 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fac440ee940 tx=0x7fac440bca20 comp rx=0 tx=0).stop 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 shutdown_connections 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56468996e290 0x564689970700 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x56468995bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689953820 0x56468995b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac30ff9640 0 client.16323 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 --2- 172.21.15.90:0/3820674785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689952f00 0x56468995b150 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.693+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 >> 172.21.15.90:0/3820674785 conn(0x56468994f410 msgr2=0x564689955140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.694+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 shutdown_connections 2024-04-26T18:23:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.694+0000 7fac5242d740 1 -- 172.21.15.90:0/3820674785 wait complete. 2024-04-26T18:23:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.694+0000 7fac5242d740 20 client.16323 trim_cache size 0 max 16384 2024-04-26T18:23:01.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.15 MB/sec execute 409 sec latency 7098.072 ms 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.708+0000 7fac5242d740 1 Processor -- start 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.708+0000 7fac5242d740 1 -- start start 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac5242d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564689952f00 0x56468994e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac5242d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689975fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac5242d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56468995e0e0 0x564689976520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac5242d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac481b6cf0 con 0x564689953820 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac5242d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac481b73c0 con 0x564689952f00 2024-04-26T18:23:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac5242d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac481b4c70 con 0x56468995e0e0 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689975fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689975fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36126/0 (socket says 172.21.15.90:36126) 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 -- 172.21.15.90:0/2955268009 learned_addr learned my addr 172.21.15.90:0/2955268009 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 -- 172.21.15.90:0/2955268009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56468995e0e0 msgr2=0x564689976520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56468995e0e0 0x564689976520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 -- 172.21.15.90:0/2955268009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564689952f00 msgr2=0x56468994e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564689952f00 0x56468994e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564689952f00 0x56468994e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 -- 172.21.15.90:0/2955268009 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564689970d50 con 0x564689953820 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564689952f00 0x56468994e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689975fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x564689950830 tx=0x7fac38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7e959d97288133 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac32ffd640 1 -- 172.21.15.90:0/2955268009 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38021030 con 0x564689953820 2024-04-26T18:23:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac32ffd640 1 -- 172.21.15.90:0/2955268009 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3801d8d0 con 0x564689953820 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.709+0000 7fac32ffd640 1 -- 172.21.15.90:0/2955268009 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38004a00 con 0x564689953820 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 -- 172.21.15.90:0/2955268009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 msgr2=0x564689975fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689975fe0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x564689950830 tx=0x7fac38011810 comp rx=0 tx=0).stop 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 -- 172.21.15.90:0/2955268009 shutdown_connections 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56468995e0e0 0x564689976520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689975fe0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 --2- 172.21.15.90:0/2955268009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564689952f00 0x56468994e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 -- 172.21.15.90:0/2955268009 >> 172.21.15.90:0/2955268009 conn(0x56468994f410 msgr2=0x56468994dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 -- 172.21.15.90:0/2955268009 shutdown_connections 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 -- 172.21.15.90:0/2955268009 wait complete. 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 -- start start 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.710+0000 7fac5242d740 1 Processor -- start 2024-04-26T18:23:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689952f00 0x564689962680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689962bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fac481b45a0 con 0x564689953820 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fac481b3190 con 0x56468995e0e0 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fac481b3800 con 0x564689952f00 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 10 client.0 _ll_register_callbacks cb 0x7fac44009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689962bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689962bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36140/0 (socket says 172.21.15.90:36140) 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 -- 172.21.15.90:0/50324571 learned_addr learned my addr 172.21.15.90:0/50324571 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689952f00 0x564689962680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 -- 172.21.15.90:0/50324571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689952f00 msgr2=0x564689962680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689952f00 0x564689962680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 -- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 msgr2=0x5646899cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 -- 172.21.15.90:0/50324571 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646899cd040 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4dbd4640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac4cbd2640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689962bc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fac38021000 tx=0x7fac3801d790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bfe529f1be7c24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac30ff9640 1 -- 172.21.15.90:0/50324571 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac38005530 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac30ff9640 1 -- 172.21.15.90:0/50324571 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9271+0+0 (secure 0 0 0) 0x7fac380046c0 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.711+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac3801d8d0 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac30ff9640 1 -- 172.21.15.90:0/50324571 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) ==== 24+0+0 (secure 0 0 0) 0x7fac380048c0 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 10 client.16329 fetch_fsmap learned FSMap version 97 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=4+}) -- 0x5646899d5360 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac30ff9640 1 -- 172.21.15.90:0/50324571 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3801d8d0 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac30ff9640 1 -- 172.21.15.90:0/50324571 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 97) ==== 7741+0+0 (secure 0 0 0) 0x7fac38026b70 con 0x564689953820 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 10 client.16329 fetch_fsmap finished waiting for FSMap version 97 2024-04-26T18:23:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 20 client.16329 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5646899d5a90 0x5646899d7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 4 client.16329 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:23:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646898c0ee0 con 0x5646899d5a90 2024-04-26T18:23:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.712+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5646899d5a90 0x5646899d7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.713+0000 7fac30ff9640 10 client.16329 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:01.713+0000 7fac4d3d3640 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5646899d5a90 0x5646899d7eb0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3a490cf63af7410 server_cookie=9c539e2dfa1faf31 in_seq=0 out_seq=0 2024-04-26T18:23:02.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.15 MB/sec execute 410 sec latency 8098.150 ms 2024-04-26T18:23:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:02 smithi090 ceph-mon[29180]: pgmap v356: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 903 KiB/s wr, 114 op/s 2024-04-26T18:23:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:02 smithi119 ceph-mon[32752]: pgmap v356: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 903 KiB/s wr, 114 op/s 2024-04-26T18:23:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:02 smithi167 ceph-mon[32751]: pgmap v356: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 903 KiB/s wr, 114 op/s 2024-04-26T18:23:03.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.14 MB/sec execute 411 sec latency 9098.246 ms 2024-04-26T18:23:04.695 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.13 MB/sec execute 412 sec latency 10098.378 ms 2024-04-26T18:23:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:04 smithi090 ceph-mon[29180]: pgmap v357: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 927 KiB/s wr, 139 op/s 2024-04-26T18:23:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:04 smithi119 ceph-mon[32752]: pgmap v357: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 927 KiB/s wr, 139 op/s 2024-04-26T18:23:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:04 smithi167 ceph-mon[32751]: pgmap v357: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 927 KiB/s wr, 139 op/s 2024-04-26T18:23:05.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.12 MB/sec execute 413 sec latency 11098.447 ms 2024-04-26T18:23:05.897 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:05 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:23:05.897 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:05 smithi119 systemd[1]: Stopped Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:23:05.897 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:05 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 3.907s CPU time. 2024-04-26T18:23:05.898 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:05 smithi119 systemd[1]: Starting Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:23:05.898 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:05 smithi119 podman[69824]: 2024-04-26 18:23:05.687817854 +0000 UTC m=+0.018039213 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:05.898 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:05 smithi119 podman[69824]: 2024-04-26 18:23:05.797562229 +0000 UTC m=+0.127783621 container create 1df1d07e7e0589698a21c37530caf84f384f9a1d85b26573910e4bc18550aa3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T18:23:06.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:06 smithi119 podman[69824]: 2024-04-26 18:23:06.016731726 +0000 UTC m=+0.346953072 container init 1df1d07e7e0589698a21c37530caf84f384f9a1d85b26573910e4bc18550aa3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:23:06.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:06 smithi119 podman[69824]: 2024-04-26 18:23:06.019631516 +0000 UTC m=+0.349852865 container start 1df1d07e7e0589698a21c37530caf84f384f9a1d85b26573910e4bc18550aa3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T18:23:06.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mds[69842]: -- 172.21.15.119:0/4079794890 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e2fe19a40 con 0x556e2fdec800 2024-04-26T18:23:06.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: starting mds.e at 2024-04-26T18:23:06.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:06 smithi119 bash[69824]: 1df1d07e7e0589698a21c37530caf84f384f9a1d85b26573910e4bc18550aa3a 2024-04-26T18:23:06.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:06 smithi119 systemd[1]: Started Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:23:06.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.12 MB/sec execute 414 sec latency 12098.525 ms 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: pgmap v358: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 301 KiB/s wr, 59 op/s 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: daemon mds.e restarted 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: osdmap e114: 12 total, 12 up, 12 in 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:boot 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:06 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: pgmap v358: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 301 KiB/s wr, 59 op/s 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: daemon mds.e restarted 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: osdmap e114: 12 total, 12 up, 12 in 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:boot 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:06 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: pgmap v358: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 301 KiB/s wr, 59 op/s 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: daemon mds.e restarted 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: osdmap e114: 12 total, 12 up, 12 in 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:boot 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: Standby daemon mds.h assigned to filesystem cephfs as rank 1 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:23:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:06 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:replay,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:07.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.11 MB/sec execute 415 sec latency 13098.626 ms 2024-04-26T18:23:08.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469286 3.10 MB/sec execute 416 sec latency 14098.723 ms 2024-04-26T18:23:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:08 smithi090 ceph-mon[29180]: pgmap v360: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 361 KiB/s wr, 71 op/s 2024-04-26T18:23:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:resolve 2024-04-26T18:23:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:08 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:08 smithi119 ceph-mon[32752]: pgmap v360: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 361 KiB/s wr, 71 op/s 2024-04-26T18:23:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:resolve 2024-04-26T18:23:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:08 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:08 smithi167 ceph-mon[32751]: pgmap v360: 97 pgs: 97 active+clean; 142 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 361 KiB/s wr, 71 op/s 2024-04-26T18:23:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:resolve 2024-04-26T18:23:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:08 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:resolve,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac30ff9640 1 -- 172.21.15.90:0/50324571 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646898c0ee0 con 0x5646899d5a90 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac30ff9640 10 client.16329 handle_command_reply: tid=0 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 client.16329 shutdown 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 msgr2=0x564689962bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689962bc0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fac38021000 tx=0x7fac3801d790 comp rx=0 tx=0).stop 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 shutdown_connections 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5646899d5a90 0x5646899d7eb0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56468995e0e0 0x5646899cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac30ff9640 0 client.16329 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x564689953820 0x564689962bc0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 --2- 172.21.15.90:0/50324571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564689952f00 0x564689962680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 >> 172.21.15.90:0/50324571 conn(0x56468994f410 msgr2=0x564689955140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 shutdown_connections 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.632+0000 7fac5242d740 1 -- 172.21.15.90:0/50324571 wait complete. 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.633+0000 7fac5242d740 20 client.16329 trim_cache size 0 max 16384 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:23:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:23:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.633+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac340a3dc0 msgr2=0x7fac340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.633+0000 7fac4f637640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac340a3dc0 0x7fac340a6280 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fac40011c50 tx=0x7fac40016000 comp rx=0 tx=0).stop 2024-04-26T18:23:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.633+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 msgr2=0x7fac4819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.633+0000 7fac4f637640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac4819f6d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fac44004770 tx=0x7fac440042f0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 shutdown_connections 2024-04-26T18:23:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fac340a3dc0 0x7fac340a6280 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac48106dd0 0x7fac481a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fac481022f0 0x7fac4819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 --2- 172.21.15.90:0/8423647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac481019d0 0x7fac4819f6d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 >> 172.21.15.90:0/8423647 conn(0x7fac48077bb0 msgr2=0x7fac48104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 shutdown_connections 2024-04-26T18:23:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:09.634+0000 7fac4f637640 1 -- 172.21.15.90:0/8423647 wait complete. 2024-04-26T18:23:09.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469302 3.09 MB/sec execute 417 sec latency 15029.452 ms 2024-04-26T18:23:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:09 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999976 2024-04-26T18:23:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:09 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:23:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:09 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:reconnect 2024-04-26T18:23:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:09 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:09 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999976 2024-04-26T18:23:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:09 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:23:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:09 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:reconnect 2024-04-26T18:23:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:09 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:09 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999976 2024-04-26T18:23:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:09 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:23:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:09 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:reconnect 2024-04-26T18:23:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:09 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:reconnect,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:10.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 469588 3.09 MB/sec execute 418 sec latency 673.059 ms 2024-04-26T18:23:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:10 smithi090 ceph-mon[29180]: pgmap v361: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-26T18:23:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:10 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:rejoin 2024-04-26T18:23:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:10 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:10 smithi090 ceph-mon[29180]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:23:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:10 smithi119 ceph-mon[32752]: pgmap v361: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-26T18:23:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:10 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:rejoin 2024-04-26T18:23:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:10 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:10 smithi119 ceph-mon[32752]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:23:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:10 smithi167 ceph-mon[32751]: pgmap v361: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-26T18:23:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:10 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:rejoin 2024-04-26T18:23:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:10 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=h=up:rejoin,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:10 smithi167 ceph-mon[32751]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-04-26T18:23:11.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 471890 3.11 MB/sec execute 419 sec latency 20.091 ms 2024-04-26T18:23:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:11 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:23:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:active 2024-04-26T18:23:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:11 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:11 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:23:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:11 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:23:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:active 2024-04-26T18:23:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:11 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:11 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:23:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:11 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:23:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6836/1113931480,v1:172.21.15.119:6837/1113931480] up:active 2024-04-26T18:23:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:11 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 2 up:standby-replay 6 up:standby 2024-04-26T18:23:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:11 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby 2024-04-26T18:23:12.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 473250 3.12 MB/sec execute 420 sec latency 16.294 ms 2024-04-26T18:23:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:12 smithi119 ceph-mon[32752]: pgmap v362: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-26T18:23:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:12 smithi167 ceph-mon[32751]: pgmap v362: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-26T18:23:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:12 smithi090 ceph-mon[29180]: pgmap v362: 97 pgs: 97 active+clean; 151 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.4 MiB/s wr, 57 op/s 2024-04-26T18:23:13.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:23:13 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:23:13.222+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:23:13.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 476036 3.15 MB/sec execute 421 sec latency 17.573 ms 2024-04-26T18:23:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:14.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 477067 3.15 MB/sec execute 422 sec latency 15.117 ms 2024-04-26T18:23:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:15 smithi119 ceph-mon[32752]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-04-26T18:23:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:15 smithi167 ceph-mon[32751]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-04-26T18:23:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:15 smithi090 ceph-mon[29180]: pgmap v363: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-04-26T18:23:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:23:15.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 479488 3.18 MB/sec execute 423 sec latency 22.050 ms 2024-04-26T18:23:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:16 smithi119 ceph-mon[32752]: pgmap v364: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-04-26T18:23:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:16 smithi119 ceph-mon[32752]: pgmap v365: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-04-26T18:23:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:16 smithi119 ceph-mon[32752]: osdmap e115: 12 total, 12 up, 12 in 2024-04-26T18:23:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:16 smithi167 ceph-mon[32751]: pgmap v364: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-04-26T18:23:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:16 smithi167 ceph-mon[32751]: pgmap v365: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-04-26T18:23:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:16 smithi167 ceph-mon[32751]: osdmap e115: 12 total, 12 up, 12 in 2024-04-26T18:23:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:16 smithi090 ceph-mon[29180]: pgmap v364: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.2 MiB/s wr, 150 op/s 2024-04-26T18:23:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:16 smithi090 ceph-mon[29180]: pgmap v365: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.4 MiB/s wr, 159 op/s 2024-04-26T18:23:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:16 smithi090 ceph-mon[29180]: osdmap e115: 12 total, 12 up, 12 in 2024-04-26T18:23:16.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.18 MB/sec execute 424 sec latency 263.671 ms 2024-04-26T18:23:17.696 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.17 MB/sec execute 425 sec latency 1263.760 ms 2024-04-26T18:23:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:18 smithi119 ceph-mon[32752]: pgmap v367: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2024-04-26T18:23:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:18 smithi167 ceph-mon[32751]: pgmap v367: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2024-04-26T18:23:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:18 smithi090 ceph-mon[29180]: pgmap v367: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 1.0 MiB/s wr, 153 op/s 2024-04-26T18:23:18.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.16 MB/sec execute 426 sec latency 2263.860 ms 2024-04-26T18:23:19.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.16 MB/sec execute 427 sec latency 3263.946 ms 2024-04-26T18:23:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:20 smithi119 ceph-mon[32752]: pgmap v368: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-04-26T18:23:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:20 smithi167 ceph-mon[32751]: pgmap v368: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-04-26T18:23:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:20 smithi090 ceph-mon[29180]: pgmap v368: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.9 MiB/s wr, 253 op/s 2024-04-26T18:23:20.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.15 MB/sec execute 428 sec latency 4264.040 ms 2024-04-26T18:23:21.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.14 MB/sec execute 429 sec latency 5264.152 ms 2024-04-26T18:23:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:22 smithi119 ceph-mon[32752]: pgmap v369: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 100 op/s 2024-04-26T18:23:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:22 smithi167 ceph-mon[32751]: pgmap v369: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 100 op/s 2024-04-26T18:23:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:22 smithi090 ceph-mon[29180]: pgmap v369: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 100 op/s 2024-04-26T18:23:22.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.13 MB/sec execute 430 sec latency 6264.201 ms 2024-04-26T18:23:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 -- 172.21.15.90:0/3797829832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8481067d0 msgr2=0x7fc848106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3797829832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8481067d0 0x7fc848106bb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fc838009900 tx=0x7fc83802a410 comp rx=0 tx=0).stop 2024-04-26T18:23:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 -- 172.21.15.90:0/3797829832 shutdown_connections 2024-04-26T18:23:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3797829832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848102540 0x7fc84810e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3797829832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc848101b80 0x7fc848102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3797829832 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8481067d0 0x7fc848106bb0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 -- 172.21.15.90:0/3797829832 >> 172.21.15.90:0/3797829832 conn(0x7fc848077630 msgr2=0x7fc848077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 -- 172.21.15.90:0/3797829832 shutdown_connections 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.731+0000 7fc84d39d640 1 -- 172.21.15.90:0/3797829832 wait complete. 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 Processor -- start 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 -- start start 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848101b80 0x7fc848199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102540 0x7fc848199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8481067d0 0x7fc84819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111ab0 con 0x7fc848102540 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111930 con 0x7fc8481067d0 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc84d39d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc848111c30 con 0x7fc848101b80 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102540 0x7fc848199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102540 0x7fc848199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49180/0 (socket says 172.21.15.90:49180) 2024-04-26T18:23:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc83ffff640 1 -- 172.21.15.90:0/3828974187 learned_addr learned my addr 172.21.15.90:0/3828974187 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848101b80 0x7fc848199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.732+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8481067d0 0x7fc84819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc83ffff640 1 -- 172.21.15.90:0/3828974187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848101b80 msgr2=0x7fc848199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc83ffff640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848101b80 0x7fc848199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc83ffff640 1 -- 172.21.15.90:0/3828974187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8481067d0 msgr2=0x7fc84819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc83ffff640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8481067d0 0x7fc84819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc83ffff640 1 -- 172.21.15.90:0/3828974187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84819e040 con 0x7fc848102540 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc83ffff640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102540 0x7fc848199590 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc83000d950 tx=0x7fc83000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc844ff9640 1 -- 172.21.15.90:0/3828974187 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830017070 con 0x7fc848102540 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc844ff9640 1 -- 172.21.15.90:0/3828974187 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc830009c30 con 0x7fc848102540 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc844ff9640 1 -- 172.21.15.90:0/3828974187 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83000f620 con 0x7fc848102540 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84819e330 con 0x7fc848102540 2024-04-26T18:23:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.733+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a5bc0 con 0x7fc848102540 2024-04-26T18:23:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.736+0000 7fc844ff9640 1 -- 172.21.15.90:0/3828974187 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fc830010460 con 0x7fc848102540 2024-04-26T18:23:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.736+0000 7fc85018e740 1 Processor -- start 2024-04-26T18:23:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.736+0000 7fc85018e740 1 -- start start 2024-04-26T18:23:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.736+0000 7fc85018e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f0f00 0x5578747f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc85018e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f1820 0x5578747fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.743 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc85018e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x5578747fe4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc85018e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557874757460 con 0x5578747fc0e0 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc85018e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557874711590 con 0x5578747f0f00 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc85018e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557874753420 con 0x5578747f1820 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x5578747fe4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc844ff9640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8340a3e40 0x7fc8340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f1820 0x5578747fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x5578747fe4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49182/0 (socket says 172.21.15.90:49182) 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 -- 172.21.15.90:0/3414910821 learned_addr learned my addr 172.21.15.90:0/3414910821 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8340a3e40 0x7fc8340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f0f00 0x5578747f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc844ff9640 1 -- 172.21.15.90:0/3828974187 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9401+0+0 (secure 0 0 0) 0x7fc8300d06a0 con 0x7fc848102540 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 -- 172.21.15.90:0/3414910821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f1820 msgr2=0x5578747fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f1820 0x5578747fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 -- 172.21.15.90:0/3414910821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f0f00 msgr2=0x5578747f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f0f00 0x5578747f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 -- 172.21.15.90:0/3414910821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787471e0d0 con 0x5578747fc0e0 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f0f00 0x5578747f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f1820 0x5578747fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc83ffff640 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x5578747fe4d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc8300cd9f0 tx=0x7fc830010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213524324ae2b1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.737+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8340a3e40 0x7fc8340a6300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc840009840 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc83d7fa640 1 -- 172.21.15.90:0/3414910821 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830020070 con 0x5578747fc0e0 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc83d7fa640 1 -- 172.21.15.90:0/3414910821 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc830009de0 con 0x5578747fc0e0 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc83d7fa640 1 -- 172.21.15.90:0/3414910821 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300a4ab0 con 0x5578747fc0e0 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 -- 172.21.15.90:0/3414910821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 msgr2=0x5578747fe4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x5578747fe4d0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc8300cd9f0 tx=0x7fc830010cb0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 -- 172.21.15.90:0/3414910821 shutdown_connections 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x5578747fe4d0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f1820 0x5578747fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 --2- 172.21.15.90:0/3414910821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f0f00 0x5578747f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 -- 172.21.15.90:0/3414910821 >> 172.21.15.90:0/3414910821 conn(0x5578747ed410 msgr2=0x5578747ed7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 -- 172.21.15.90:0/3414910821 shutdown_connections 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.738+0000 7fc85018e740 1 -- 172.21.15.90:0/3414910821 wait complete. 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 -- start start 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 Processor -- start 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x5578747f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x5578747f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x5578747f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5578747fec50 con 0x5578747f0f00 2024-04-26T18:23:22.746 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5578746deb60 con 0x5578747f1820 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5578747fedd0 con 0x5578747fc0e0 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc85018e740 10 client.0 _ll_register_callbacks cb 0x5578747032b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x5578747f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x5578747f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49190/0 (socket says 172.21.15.90:49190) 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc846ffd640 1 -- 172.21.15.90:0/1316315605 learned_addr learned my addr 172.21.15.90:0/1316315605 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc8477fe640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x5578747f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc83ffff640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x5578747f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc846ffd640 1 -- 172.21.15.90:0/1316315605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 msgr2=0x5578747f9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc846ffd640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x5578747f9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.739+0000 7fc846ffd640 1 -- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 msgr2=0x5578747f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc846ffd640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x5578747f9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc846ffd640 1 -- 172.21.15.90:0/1316315605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557874469210 con 0x5578747f0f00 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc83ffff640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x5578747f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:22.747 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc846ffd640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x5578747f8fe0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc838009900 tx=0x7fc8380314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a834a07f90b428 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc82e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc82e7fc640 1 -- 172.21.15.90:0/1316315605 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838035040 con 0x5578747f0f00 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc82e7fc640 1 -- 172.21.15.90:0/1316315605 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9401+0+0 (secure 0 0 0) 0x7fc838031810 con 0x5578747f0f00 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc830009de0 con 0x5578747f0f00 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc82e7fc640 1 -- 172.21.15.90:0/1316315605 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc838004040 con 0x5578747f0f00 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 10 client.16344 fetch_fsmap learned FSMap version 104 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) -- 0x557874535620 con 0x5578747f0f00 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc82e7fc640 1 -- 172.21.15.90:0/1316315605 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc830009de0 con 0x5578747f0f00 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc82e7fc640 1 -- 172.21.15.90:0/1316315605 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 104) ==== 7741+0+0 (secure 0 0 0) 0x7fc8380047e0 con 0x5578747f0f00 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 10 client.16344 fetch_fsmap finished waiting for FSMap version 104 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 20 client.16344 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:22.748 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55787480c0c0 0x55787480e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 4 client.16344 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.740+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557874765070 con 0x55787480c0c0 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.741+0000 7fc83ffff640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55787480c0c0 0x55787480e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.741+0000 7fc83ffff640 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55787480c0c0 0x55787480e510 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33ea75317bbeed5c server_cookie=d76d675e4188aa48 in_seq=0 out_seq=0 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.741+0000 7fc82e7fc640 10 client.16344 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc82e7fc640 1 -- 172.21.15.90:0/1316315605 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557874765070 con 0x55787480c0c0 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc82e7fc640 10 client.16344 handle_command_reply: tid=0 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 client.16344 shutdown 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 msgr2=0x5578747f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x5578747f8fe0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc838009900 tx=0x7fc8380314a0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 shutdown_connections 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55787480c0c0 0x55787480e510 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x5578747f9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x5578747f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.749 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 --2- 172.21.15.90:0/1316315605 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x5578747f8fe0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc82e7fc640 0 client.16344 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 >> 172.21.15.90:0/1316315605 conn(0x5578747ed410 msgr2=0x5578747f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 shutdown_connections 2024-04-26T18:23:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 1 -- 172.21.15.90:0/1316315605 wait complete. 2024-04-26T18:23:22.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.742+0000 7fc85018e740 20 client.16344 trim_cache size 0 max 16384 2024-04-26T18:23:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 Processor -- start 2024-04-26T18:23:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 -- start start 2024-04-26T18:23:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f0f00 0x557874807b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874813e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x557874814360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b07d0 con 0x5578747fc0e0 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b0ea0 con 0x5578747f1820 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc85018e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ae7b0 con 0x5578747f0f00 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc8477fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x557874814360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc8477fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x557874814360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49194/0 (socket says 172.21.15.90:49194) 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc8477fe640 1 -- 172.21.15.90:0/3349127722 learned_addr learned my addr 172.21.15.90:0/3349127722 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.757+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f0f00 0x557874807b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc83ffff640 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874813e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc8477fe640 1 -- 172.21.15.90:0/3349127722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f0f00 msgr2=0x557874807b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f0f00 0x557874807b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc8477fe640 1 -- 172.21.15.90:0/3349127722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 msgr2=0x557874813e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.759 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874813e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc8477fe640 1 -- 172.21.15.90:0/3349127722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578747f7570 con 0x5578747fc0e0 2024-04-26T18:23:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x557874814360 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc8400125f0 tx=0x7fc840014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199883247131f1f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc83cff9640 1 -- 172.21.15.90:0/3349127722 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84002e070 con 0x5578747fc0e0 2024-04-26T18:23:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc83cff9640 1 -- 172.21.15.90:0/3349127722 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc84001f990 con 0x5578747fc0e0 2024-04-26T18:23:22.760 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.758+0000 7fc83cff9640 1 -- 172.21.15.90:0/3349127722 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840029a50 con 0x5578747fc0e0 2024-04-26T18:23:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- 172.21.15.90:0/3349127722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 msgr2=0x557874814360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x557874814360 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc8400125f0 tx=0x7fc840014040 comp rx=0 tx=0).stop 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- 172.21.15.90:0/3349127722 shutdown_connections 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747fc0e0 0x557874814360 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874813e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 --2- 172.21.15.90:0/3349127722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747f0f00 0x557874807b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- 172.21.15.90:0/3349127722 >> 172.21.15.90:0/3349127722 conn(0x5578747ed410 msgr2=0x5578747fb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- 172.21.15.90:0/3349127722 shutdown_connections 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- 172.21.15.90:0/3349127722 wait complete. 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- start start 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 Processor -- start 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x557874804020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874804560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x557874800ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ae0e0 con 0x5578747f0f00 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481accf0 con 0x5578747f1820 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.759+0000 7fc85018e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ad340 con 0x5578747fc0e0 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc85018e740 10 client.0 _ll_register_callbacks cb 0x7fc838032c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x557874804020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x557874804020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49202/0 (socket says 172.21.15.90:49202) 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 -- 172.21.15.90:0/2501583134 learned_addr learned my addr 172.21.15.90:0/2501583134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc83ffff640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874804560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc8477fe640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x557874800ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 -- 172.21.15.90:0/2501583134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 msgr2=0x557874800ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x557874800ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 -- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 msgr2=0x557874804560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874804560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 -- 172.21.15.90:0/2501583134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557874801820 con 0x5578747f0f00 2024-04-26T18:23:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc8477fe640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x557874800ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc83ffff640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874804560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc846ffd640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x557874804020 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc838004d70 tx=0x7fc838005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1309fc281cebb0be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc82e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc82e7fc640 1 -- 172.21.15.90:0/2501583134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838058060 con 0x5578747f0f00 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc82e7fc640 1 -- 172.21.15.90:0/2501583134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9401+0+0 (secure 0 0 0) 0x7fc838003c40 con 0x5578747f0f00 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc84001f990 con 0x5578747f0f00 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc82e7fc640 1 -- 172.21.15.90:0/2501583134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7fc83805f050 con 0x5578747f0f00 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc85018e740 10 client.16350 fetch_fsmap learned FSMap version 104 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) -- 0x5578748775e0 con 0x5578747f0f00 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.760+0000 7fc82e7fc640 1 -- 172.21.15.90:0/2501583134 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc84001f990 con 0x5578747f0f00 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc82e7fc640 1 -- 172.21.15.90:0/2501583134 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 104) ==== 7741+0+0 (secure 0 0 0) 0x7fc838032070 con 0x5578747f0f00 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc85018e740 10 client.16350 fetch_fsmap finished waiting for FSMap version 104 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc85018e740 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:22.766 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc85018e740 20 client.16350 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc85018e740 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557874877ca0 0x55787487a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc85018e740 4 client.16350 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:23:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55787475eee0 con 0x557874877ca0 2024-04-26T18:23:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc83ffff640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557874877ca0 0x55787487a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc82e7fc640 10 client.16350 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.761+0000 7fc83ffff640 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557874877ca0 0x55787487a0c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6cf1cfe33b713b7 server_cookie=781b2c321f7a1fe5 in_seq=0 out_seq=0 2024-04-26T18:23:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc82e7fc640 1 -- 172.21.15.90:0/2501583134 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55787475eee0 con 0x557874877ca0 2024-04-26T18:23:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc82e7fc640 10 client.16350 handle_command_reply: tid=0 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 client.16350 shutdown 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 msgr2=0x557874804020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x557874804020 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc838004d70 tx=0x7fc838005e80 comp rx=0 tx=0).stop 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 shutdown_connections 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557874877ca0 0x55787487a0c0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5578747fc0e0 0x557874800ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc82e7fc640 0 client.16350 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5578747f1820 0x557874804560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 --2- 172.21.15.90:0/2501583134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5578747f0f00 0x557874804020 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 >> 172.21.15.90:0/2501583134 conn(0x5578747ed410 msgr2=0x5578747f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 shutdown_connections 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 1 -- 172.21.15.90:0/2501583134 wait complete. 2024-04-26T18:23:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.811+0000 7fc85018e740 20 client.16350 trim_cache size 0 max 16384 2024-04-26T18:23:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:23:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:23:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:23:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8340a3e40 msgr2=0x7fc8340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8340a3e40 0x7fc8340a6300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc840009fd0 tx=0x7fc840009840 comp rx=0 tx=0).stop 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102540 msgr2=0x7fc848199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102540 0x7fc848199590 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fc83000d950 tx=0x7fc83000de10 comp rx=0 tx=0).stop 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 shutdown_connections 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8340a3e40 0x7fc8340a6300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8481067d0 0x7fc84819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102540 0x7fc848199590 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 --2- 172.21.15.90:0/3828974187 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848101b80 0x7fc848199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 >> 172.21.15.90:0/3828974187 conn(0x7fc848077630 msgr2=0x7fc84806b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 shutdown_connections 2024-04-26T18:23:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:22.812+0000 7fc84d39d640 1 -- 172.21.15.90:0/3828974187 wait complete. 2024-04-26T18:23:23.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.13 MB/sec execute 431 sec latency 7264.241 ms 2024-04-26T18:23:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:23 smithi119 ceph-mon[32752]: osdmap e116: 12 total, 12 up, 12 in 2024-04-26T18:23:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:23 smithi119 ceph-mon[32752]: pgmap v371: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 KiB/s wr, 91 op/s 2024-04-26T18:23:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:23 smithi167 ceph-mon[32751]: osdmap e116: 12 total, 12 up, 12 in 2024-04-26T18:23:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:23 smithi167 ceph-mon[32751]: pgmap v371: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 KiB/s wr, 91 op/s 2024-04-26T18:23:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:23 smithi090 ceph-mon[29180]: osdmap e116: 12 total, 12 up, 12 in 2024-04-26T18:23:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:23 smithi090 ceph-mon[29180]: pgmap v371: 97 pgs: 97 active+clean; 149 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 827 KiB/s wr, 91 op/s 2024-04-26T18:23:24.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.12 MB/sec execute 432 sec latency 8264.294 ms 2024-04-26T18:23:25.420 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ffbb923f700 time 2024-04-26T18:23:25.056348+0000 2024-04-26T18:23:25.420 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:25.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ffbc38683dd] 2024-04-26T18:23:25.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.421 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.422 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.423 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: clone() 2024-04-26T18:23:25.423 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: *** Caught signal (Aborted) ** 2024-04-26T18:23:25.423 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: in thread 7ffbb923f700 thread_name:ms_dispatch 2024-04-26T18:23:25.423 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.056+0000 7ffbb923f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ffbb923f700 time 2024-04-26T18:23:25.056348+0000 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ffbc38683dd] 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.424 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: clone() 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ffbc25f0d20] 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: gsignal() 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: abort() 2024-04-26T18:23:25.425 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ffbc3868437] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 16: clone() 2024-04-26T18:23:25.426 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.058+0000 7ffbb923f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: in thread 7ffbb923f700 thread_name:ms_dispatch 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ffbc25f0d20] 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: gsignal() 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: abort() 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ffbc3868437] 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.427 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 16: clone() 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: -1> 2024-04-26T18:23:25.056+0000 7ffbb923f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ffbb923f700 time 2024-04-26T18:23:25.056348+0000 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:25.428 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ffbc38683dd] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.429 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: clone() 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 0> 2024-04-26T18:23:25.058+0000 7ffbb923f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: in thread 7ffbb923f700 thread_name:ms_dispatch 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ffbc25f0d20] 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: gsignal() 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: abort() 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ffbc3868437] 2024-04-26T18:23:25.430 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.431 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 16: clone() 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: -1> 2024-04-26T18:23:25.056+0000 7ffbb923f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ffbb923f700 time 2024-04-26T18:23:25.056348+0000 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ffbc38683dd] 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.432 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: clone() 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 0> 2024-04-26T18:23:25.058+0000 7ffbb923f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: in thread 7ffbb923f700 thread_name:ms_dispatch 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.433 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ffbc25f0d20] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2: gsignal() 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 3: abort() 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ffbc3868437] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ffbc38685a3] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ad6a662822] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ad6a5d2598] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ad6a5d4c17] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ad6a4cdd0b] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ad6a4ce2ac] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ad6a4b6fad] 2024-04-26T18:23:25.434 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ffbc3afce20] 2024-04-26T18:23:25.435 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 13: (DispatchQueue::entry()+0x51f) [0x7ffbc3af9fdf] 2024-04-26T18:23:25.435 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ffbc3bccfc1] 2024-04-26T18:23:25.435 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ffbc25e61ca] 2024-04-26T18:23:25.435 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 16: clone() 2024-04-26T18:23:25.435 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:25.435 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h[69671]: 2024-04-26T18:23:25.435 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 podman[70012]: 2024-04-26 18:23:25.122363473 +0000 UTC m=+0.021025740 container died 5756ed1a1238854e21ddb48dbf30b741a01be5e2c475530a37b31ae5745f2b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T18:23:25.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.11 MB/sec execute 433 sec latency 9264.397 ms 2024-04-26T18:23:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:25 smithi119 ceph-mon[32752]: pgmap v372: 97 pgs: 97 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 724 KiB/s wr, 107 op/s 2024-04-26T18:23:26.054 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 podman[70012]: 2024-04-26 18:23:25.771580251 +0000 UTC m=+0.670242509 container remove 5756ed1a1238854e21ddb48dbf30b741a01be5e2c475530a37b31ae5745f2b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:23:26.054 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:25 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:23:26.054 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:26 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-26T18:23:26.054 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:26 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 4.509s CPU time. 2024-04-26T18:23:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:25 smithi167 ceph-mon[32751]: pgmap v372: 97 pgs: 97 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 724 KiB/s wr, 107 op/s 2024-04-26T18:23:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:25 smithi090 ceph-mon[29180]: pgmap v372: 97 pgs: 97 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 724 KiB/s wr, 107 op/s 2024-04-26T18:23:26.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.11 MB/sec execute 434 sec latency 10264.486 ms 2024-04-26T18:23:27.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.10 MB/sec execute 435 sec latency 11264.604 ms 2024-04-26T18:23:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:27 smithi119 ceph-mon[32752]: pgmap v373: 97 pgs: 97 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 701 KiB/s wr, 104 op/s 2024-04-26T18:23:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:27 smithi167 ceph-mon[32751]: pgmap v373: 97 pgs: 97 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 701 KiB/s wr, 104 op/s 2024-04-26T18:23:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:27 smithi090 ceph-mon[29180]: pgmap v373: 97 pgs: 97 active+clean; 145 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 701 KiB/s wr, 104 op/s 2024-04-26T18:23:28.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.09 MB/sec execute 436 sec latency 12264.749 ms 2024-04-26T18:23:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f02e25ff700 time 2024-04-26T18:23:29.145163+0000 2024-04-26T18:23:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f02ecc283dd] 2024-04-26T18:23:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: clone() 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: *** Caught signal (Aborted) ** 2024-04-26T18:23:29.569 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: in thread 7f02e25ff700 thread_name:ms_dispatch 2024-04-26T18:23:29.570 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.146+0000 7f02e25ff700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f02e25ff700 time 2024-04-26T18:23:29.145163+0000 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f02ecc283dd] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.571 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: clone() 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02eb9b0d20] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: gsignal() 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: abort() 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f02ecc28437] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.572 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 16: clone() 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.148+0000 7f02e25ff700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: in thread 7f02e25ff700 thread_name:ms_dispatch 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02eb9b0d20] 2024-04-26T18:23:29.573 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: gsignal() 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: abort() 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f02ecc28437] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.574 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 16: clone() 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: -1> 2024-04-26T18:23:29.146+0000 7f02e25ff700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f02e25ff700 time 2024-04-26T18:23:29.145163+0000 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f02ecc283dd] 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.575 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: clone() 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 0> 2024-04-26T18:23:29.148+0000 7f02e25ff700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: in thread 7f02e25ff700 thread_name:ms_dispatch 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02eb9b0d20] 2024-04-26T18:23:29.576 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: gsignal() 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: abort() 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f02ecc28437] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.577 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 16: clone() 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: -1> 2024-04-26T18:23:29.146+0000 7f02e25ff700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f02e25ff700 time 2024-04-26T18:23:29.145163+0000 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f02ecc283dd] 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.578 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: clone() 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 0> 2024-04-26T18:23:29.148+0000 7f02e25ff700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: in thread 7f02e25ff700 thread_name:ms_dispatch 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:29.579 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f02eb9b0d20] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2: gsignal() 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 3: abort() 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f02ecc28437] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f02ecc285a3] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55f9f61a0822] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55f9f6110598] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55f9f6112c17] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55f9f600bd0b] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55f9f600c2ac] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55f9f5ff4fad] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f02ecebce20] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 13: (DispatchQueue::entry()+0x51f) [0x7f02eceb9fdf] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f02ecf8cfc1] 2024-04-26T18:23:29.580 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f02eb9a61ca] 2024-04-26T18:23:29.581 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 16: clone() 2024-04-26T18:23:29.581 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:29.581 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[63164]: 2024-04-26T18:23:29.581 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:29 smithi167 podman[70165]: 2024-04-26 18:23:29.213668308 +0000 UTC m=+0.022287204 container died a0d324860d23eaf098089652289a1fa1e036f1eca55902ac56ac617fdce34b48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T18:23:29.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.08 MB/sec execute 437 sec latency 13264.856 ms 2024-04-26T18:23:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:29 smithi167 ceph-mon[32751]: pgmap v374: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 KiB/s wr, 37 op/s 2024-04-26T18:23:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:29 smithi119 ceph-mon[32752]: pgmap v374: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 KiB/s wr, 37 op/s 2024-04-26T18:23:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:29 smithi090 ceph-mon[29180]: pgmap v374: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 KiB/s wr, 37 op/s 2024-04-26T18:23:30.697 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.08 MB/sec execute 438 sec latency 14264.931 ms 2024-04-26T18:23:30.818 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:30 smithi167 podman[70165]: 2024-04-26 18:23:30.330387094 +0000 UTC m=+1.139005979 container remove a0d324860d23eaf098089652289a1fa1e036f1eca55902ac56ac617fdce34b48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T18:23:30.818 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:30 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:23:30.818 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:30 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-26T18:23:30.818 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:30 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 1min 48.562s CPU time. 2024-04-26T18:23:31.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.07 MB/sec execute 439 sec latency 15265.053 ms 2024-04-26T18:23:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:31 smithi119 ceph-mon[32752]: pgmap v375: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 KiB/s wr, 37 op/s 2024-04-26T18:23:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:31 smithi167 ceph-mon[32751]: pgmap v375: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 KiB/s wr, 37 op/s 2024-04-26T18:23:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:31 smithi090 ceph-mon[29180]: pgmap v375: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 164 KiB/s wr, 37 op/s 2024-04-26T18:23:32.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.06 MB/sec execute 440 sec latency 16265.143 ms 2024-04-26T18:23:33.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.06 MB/sec execute 441 sec latency 17265.268 ms 2024-04-26T18:23:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:33 smithi119 ceph-mon[32752]: pgmap v376: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 159 KiB/s wr, 36 op/s 2024-04-26T18:23:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:33 smithi167 ceph-mon[32751]: pgmap v376: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 159 KiB/s wr, 36 op/s 2024-04-26T18:23:34.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:33 smithi090 ceph-mon[29180]: pgmap v376: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 159 KiB/s wr, 36 op/s 2024-04-26T18:23:34.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.05 MB/sec execute 442 sec latency 18265.359 ms 2024-04-26T18:23:35.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.04 MB/sec execute 443 sec latency 19265.403 ms 2024-04-26T18:23:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2283459727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64102220 msgr2=0x7efe6410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2283459727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64102220 0x7efe6410e6e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009cd0 tx=0x7efe5c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2283459727 shutdown_connections 2024-04-26T18:23:35.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2283459727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64102220 0x7efe6410e6e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2283459727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe64101860 0x7efe64101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2283459727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64107820 0x7efe64107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2283459727 >> 172.21.15.90:0/2283459727 conn(0x7efe640789b0 msgr2=0x7efe64078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2283459727 shutdown_connections 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2283459727 wait complete. 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 Processor -- start 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.908+0000 7efe6b3f0640 1 -- start start 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64101860 0x7efe640fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe64102220 0x7efe640fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64107820 0x7efe640ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe64111ae0 con 0x7efe64107820 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe64111960 con 0x7efe64101860 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe64111c60 con 0x7efe64102220 2024-04-26T18:23:35.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64107820 0x7efe640ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64107820 0x7efe640ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57888/0 (socket says 172.21.15.90:57888) 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 -- 172.21.15.90:0/2976116908 learned_addr learned my addr 172.21.15.90:0/2976116908 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6918c640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64101860 0x7efe640fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6898b640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe64102220 0x7efe640fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 -- 172.21.15.90:0/2976116908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe64102220 msgr2=0x7efe640fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe64102220 0x7efe640fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 -- 172.21.15.90:0/2976116908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64101860 msgr2=0x7efe640fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64101860 0x7efe640fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 -- 172.21.15.90:0/2976116908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe641a76b0 con 0x7efe64107820 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6918c640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64101860 0x7efe640fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6998d640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64107820 0x7efe640ff270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005cc0 tx=0x7efe5c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe567fc640 1 -- 172.21.15.90:0/2976116908 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c004450 con 0x7efe64107820 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe641a7940 con 0x7efe64107820 2024-04-26T18:23:35.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe567fc640 1 -- 172.21.15.90:0/2976116908 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe5c031c90 con 0x7efe64107820 2024-04-26T18:23:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe567fc640 1 -- 172.21.15.90:0/2976116908 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c03b400 con 0x7efe64107820 2024-04-26T18:23:35.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.909+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe641a7e90 con 0x7efe64107820 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe567fc640 1 -- 172.21.15.90:0/2976116908 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7efe5c004680 con 0x7efe64107820 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 Processor -- start 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 -- start start 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32ed2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629e3253460 con 0x5629e32ecf00 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5629e320d590 con 0x5629e32ed820 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5629e324f420 con 0x5629e32f80e0 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.912+0000 7efe567fc640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe500a3dc0 0x7efe500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.920 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32ed2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32ed2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57904/0 (socket says 172.21.15.90:57904) 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 -- 172.21.15.90:0/3383817620 learned_addr learned my addr 172.21.15.90:0/3383817620 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6918c640 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6998d640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe500a3dc0 0x7efe500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 -- 172.21.15.90:0/3383817620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 msgr2=0x5629e32fa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32fa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 -- 172.21.15.90:0/3383817620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 msgr2=0x5629e32f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 -- 172.21.15.90:0/3383817620 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e321a0d0 con 0x5629e32ecf00 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6918c640 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe567fc640 1 -- 172.21.15.90:0/2976116908 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9401+0+0 (secure 0 0 0) 0x7efe5c02e030 con 0x7efe64107820 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6898b640 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32ed2e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efe60006e10 tx=0x7efe60009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d2e4de5c1665f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe6998d640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe500a3dc0 0x7efe500a6280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0ef210 tx=0x7efe5c0c7680 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe4f7fe640 1 -- 172.21.15.90:0/3383817620 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6001c070 con 0x5629e32ecf00 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe4f7fe640 1 -- 172.21.15.90:0/3383817620 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe60010040 con 0x5629e32ecf00 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.913+0000 7efe4f7fe640 1 -- 172.21.15.90:0/3383817620 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe60014680 con 0x5629e32ecf00 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 -- 172.21.15.90:0/3383817620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 msgr2=0x5629e32ed2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32ed2e0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7efe60006e10 tx=0x7efe60009420 comp rx=0 tx=0).stop 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 -- 172.21.15.90:0/3383817620 shutdown_connections 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32fa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/3383817620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32ed2e0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 -- 172.21.15.90:0/3383817620 >> 172.21.15.90:0/3383817620 conn(0x5629e32e9410 msgr2=0x5629e32e97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:35.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 -- 172.21.15.90:0/3383817620 shutdown_connections 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 -- 172.21.15.90:0/3383817620 wait complete. 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.914+0000 7efe6e1e3740 1 -- start start 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 1 Processor -- start 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32f5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5629e32fad10 con 0x5629e32ecf00 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5629e31dab60 con 0x5629e32ed820 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5629e32fae90 con 0x5629e32f80e0 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6e1e3740 10 client.0 _ll_register_callbacks cb 0x5629e31ff2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57910/0 (socket says 172.21.15.90:57910) 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 -- 172.21.15.90:0/626045052 learned_addr learned my addr 172.21.15.90:0/626045052 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:35.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6918c640 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 -- 172.21.15.90:0/626045052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 msgr2=0x5629e32f5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32f5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 -- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 msgr2=0x5629e32f55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.915+0000 7efe6898b640 1 -- 172.21.15.90:0/626045052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e2f65210 con 0x5629e32ecf00 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6898b640 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32f5070 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efe60006aa0 tx=0x7efe60017f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52174c15a6e3ff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe4cff9640 1 -- 172.21.15.90:0/626045052 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6002b060 con 0x5629e32ecf00 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe4cff9640 1 -- 172.21.15.90:0/626045052 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9401+0+0 (secure 0 0 0) 0x7efe60024640 con 0x5629e32ecf00 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe60010040 con 0x5629e32ecf00 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe4cff9640 1 -- 172.21.15.90:0/626045052 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7efe6001c070 con 0x5629e32ecf00 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 10 client.16362 fetch_fsmap learned FSMap version 104 2024-04-26T18:23:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) -- 0x5629e3031620 con 0x5629e32ecf00 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe4cff9640 1 -- 172.21.15.90:0/626045052 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe60010040 con 0x5629e32ecf00 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe4cff9640 1 -- 172.21.15.90:0/626045052 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 104) ==== 7741+0+0 (secure 0 0 0) 0x7efe60026660 con 0x5629e32ecf00 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 10 client.16362 fetch_fsmap finished waiting for FSMap version 104 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 20 client.16362 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e3308110 0x5629e330a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 4 client.16362 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629e3261070 con 0x5629e3308110 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.916+0000 7efe6918c640 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e3308110 0x5629e330a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.917+0000 7efe4cff9640 10 client.16362 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.917+0000 7efe6918c640 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e3308110 0x5629e330a500 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331805ef8ffbfd70 server_cookie=3503f8b05b68ee33 in_seq=0 out_seq=0 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.917+0000 7efe4cff9640 1 -- 172.21.15.90:0/626045052 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629e3261070 con 0x5629e3308110 2024-04-26T18:23:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.917+0000 7efe4cff9640 10 client.16362 handle_command_reply: tid=0 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 client.16362 shutdown 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 msgr2=0x5629e32f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32f5070 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7efe60006aa0 tx=0x7efe60017f50 comp rx=0 tx=0).stop 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 shutdown_connections 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e3308110 0x5629e330a500 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32f80e0 0x5629e32f5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe4cff9640 0 client.16362 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/626045052 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e32f5070 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 >> 172.21.15.90:0/626045052 conn(0x5629e32e9410 msgr2=0x5629e32ef190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 shutdown_connections 2024-04-26T18:23:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 1 -- 172.21.15.90:0/626045052 wait complete. 2024-04-26T18:23:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.918+0000 7efe6e1e3740 20 client.16362 trim_cache size 0 max 16384 2024-04-26T18:23:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 Processor -- start 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 -- start start 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e3303bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ed820 0x5629e32f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32f80e0 0x5629e330fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe641b2920 con 0x5629e32ecf00 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe641b2ff0 con 0x5629e32f80e0 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe641b0900 con 0x5629e32ed820 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e3303bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e3303bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57912/0 (socket says 172.21.15.90:57912) 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 -- 172.21.15.90:0/2009677649 learned_addr learned my addr 172.21.15.90:0/2009677649 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 -- 172.21.15.90:0/2009677649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ed820 msgr2=0x5629e32f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 --2- 172.21.15.90:0/2009677649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ed820 0x5629e32f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 -- 172.21.15.90:0/2009677649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32f80e0 msgr2=0x5629e330fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 --2- 172.21.15.90:0/2009677649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32f80e0 0x5629e330fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 -- 172.21.15.90:0/2009677649 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629e32ea1a0 con 0x5629e32ecf00 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.933+0000 7efe6898b640 1 --2- 172.21.15.90:0/2009677649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e3303bd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe60006b50 tx=0x7efe60006da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6466dad127b5e2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe4effd640 1 -- 172.21.15.90:0/2009677649 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6000fe60 con 0x5629e32ecf00 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe4effd640 1 -- 172.21.15.90:0/2009677649 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6000f9e0 con 0x5629e32ecf00 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe4effd640 1 -- 172.21.15.90:0/2009677649 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6002b040 con 0x5629e32ecf00 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 -- 172.21.15.90:0/2009677649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 msgr2=0x5629e3303bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/2009677649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e3303bd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe60006b50 tx=0x7efe60006da0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 -- 172.21.15.90:0/2009677649 shutdown_connections 2024-04-26T18:23:35.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/2009677649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32f80e0 0x5629e330fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/2009677649 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ed820 0x5629e32f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/2009677649 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32ecf00 0x5629e3303bd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 -- 172.21.15.90:0/2009677649 >> 172.21.15.90:0/2009677649 conn(0x5629e32e9410 msgr2=0x5629e32f68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 -- 172.21.15.90:0/2009677649 shutdown_connections 2024-04-26T18:23:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 -- 172.21.15.90:0/2009677649 wait complete. 2024-04-26T18:23:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.934+0000 7efe6e1e3740 1 -- start start 2024-04-26T18:23:35.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 1 Processor -- start 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ecf00 0x5629e32f1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32f80e0 0x5629e32f2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efe641b0230 con 0x5629e32f80e0 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efe641aee40 con 0x5629e32ed820 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe641af490 con 0x5629e32ecf00 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6e1e3740 10 client.0 _ll_register_callbacks cb 0x7efe6001a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32f80e0 0x5629e32f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6898b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32f80e0 0x5629e32f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57928/0 (socket says 172.21.15.90:57928) 2024-04-26T18:23:35.940 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6898b640 1 -- 172.21.15.90:0/1412899736 learned_addr learned my addr 172.21.15.90:0/1412899736 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6898b640 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ecf00 0x5629e32f1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.935+0000 7efe6918c640 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6898b640 1 -- 172.21.15.90:0/1412899736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ecf00 msgr2=0x5629e32f1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6898b640 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ecf00 0x5629e32f1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6898b640 1 -- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 msgr2=0x5629e32f2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6898b640 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6898b640 1 -- 172.21.15.90:0/1412899736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629e336d5f0 con 0x5629e32f80e0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6898b640 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32f80e0 0x5629e32f2940 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efe60017ef0 tx=0x7efe60017f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310e75359428f795 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe4cff9640 1 -- 172.21.15.90:0/1412899736 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6000fe60 con 0x5629e32f80e0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6000f9e0 con 0x5629e32f80e0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe4cff9640 1 -- 172.21.15.90:0/1412899736 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9401+0+0 (secure 0 0 0) 0x7efe600263f0 con 0x5629e32f80e0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe4cff9640 1 -- 172.21.15.90:0/1412899736 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) ==== 24+0+0 (secure 0 0 0) 0x7efe6004c050 con 0x5629e32f80e0 2024-04-26T18:23:35.941 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 10 client.16368 fetch_fsmap learned FSMap version 104 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=4+}) -- 0x5629e336e8e0 con 0x5629e32f80e0 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe4cff9640 1 -- 172.21.15.90:0/1412899736 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe6000f9e0 con 0x5629e32f80e0 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe4cff9640 1 -- 172.21.15.90:0/1412899736 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 104) ==== 7741+0+0 (secure 0 0 0) 0x7efe60044020 con 0x5629e32f80e0 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 10 client.16368 fetch_fsmap finished waiting for FSMap version 104 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 20 client.16368 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e336eec0 0x5629e33712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 4 client.16368 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.936+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629e325aee0 con 0x5629e336eec0 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.937+0000 7efe6918c640 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e336eec0 0x5629e33712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.937+0000 7efe4cff9640 10 client.16368 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:35.942 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:35.937+0000 7efe6918c640 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e336eec0 0x5629e33712e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7dd6a0e13492f8c server_cookie=5a669e3ca4a457f6 in_seq=0 out_seq=0 2024-04-26T18:23:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:35 smithi119 ceph-mon[32752]: pgmap v377: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 138 KiB/s wr, 34 op/s 2024-04-26T18:23:36.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:36 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Scheduled restart job, restart counter is at 6. 2024-04-26T18:23:36.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:36 smithi119 systemd[1]: Stopped Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:23:36.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:36 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Consumed 4.509s CPU time. 2024-04-26T18:23:36.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:36 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Start request repeated too quickly. 2024-04-26T18:23:36.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:36 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.h.service: Failed with result 'exit-code'. 2024-04-26T18:23:36.304 INFO:journalctl@ceph.mds.h.smithi119.stdout:Apr 26 18:23:36 smithi119 systemd[1]: Failed to start Ceph mds.h for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:23:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:35 smithi167 ceph-mon[32751]: pgmap v377: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 138 KiB/s wr, 34 op/s 2024-04-26T18:23:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:35 smithi090 ceph-mon[29180]: pgmap v377: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 138 KiB/s wr, 34 op/s 2024-04-26T18:23:36.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.04 MB/sec execute 444 sec latency 20265.452 ms 2024-04-26T18:23:37.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.03 MB/sec execute 445 sec latency 21265.507 ms 2024-04-26T18:23:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:37 smithi119 ceph-mon[32752]: pgmap v378: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 105 KiB/s wr, 8 op/s 2024-04-26T18:23:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:37 smithi119 ceph-mon[32752]: Replacing daemon mds.h as rank 1 with standby daemon mds.e 2024-04-26T18:23:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:37 smithi119 ceph-mon[32752]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-04-26T18:23:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:37 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:23:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:37 smithi119 ceph-mon[32752]: osdmap e117: 12 total, 12 up, 12 in 2024-04-26T18:23:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:37 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:23:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:37 smithi167 ceph-mon[32751]: pgmap v378: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 105 KiB/s wr, 8 op/s 2024-04-26T18:23:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:37 smithi167 ceph-mon[32751]: Replacing daemon mds.h as rank 1 with standby daemon mds.e 2024-04-26T18:23:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:37 smithi167 ceph-mon[32751]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-04-26T18:23:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:37 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:23:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:37 smithi167 ceph-mon[32751]: osdmap e117: 12 total, 12 up, 12 in 2024-04-26T18:23:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:37 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:23:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:37 smithi090 ceph-mon[29180]: pgmap v378: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 105 KiB/s wr, 8 op/s 2024-04-26T18:23:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:37 smithi090 ceph-mon[29180]: Replacing daemon mds.h as rank 1 with standby daemon mds.e 2024-04-26T18:23:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:37 smithi090 ceph-mon[29180]: MDS daemon mds.h is removed because it is dead or otherwise unavailable. 2024-04-26T18:23:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:37 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:23:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:37 smithi090 ceph-mon[29180]: osdmap e117: 12 total, 12 up, 12 in 2024-04-26T18:23:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:37 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:23:38.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.02 MB/sec execute 446 sec latency 22265.605 ms 2024-04-26T18:23:39.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.02 MB/sec execute 447 sec latency 23265.721 ms 2024-04-26T18:23:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:39 smithi119 ceph-mon[32752]: pgmap v380: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 912 KiB/s wr, 37 op/s 2024-04-26T18:23:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:resolve 2024-04-26T18:23:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:39 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:23:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:39 smithi167 ceph-mon[32751]: pgmap v380: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 912 KiB/s wr, 37 op/s 2024-04-26T18:23:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:resolve 2024-04-26T18:23:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:39 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:23:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:39 smithi090 ceph-mon[29180]: pgmap v380: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 912 KiB/s wr, 37 op/s 2024-04-26T18:23:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:resolve 2024-04-26T18:23:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:39 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:23:40.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.01 MB/sec execute 448 sec latency 24265.828 ms 2024-04-26T18:23:41.096 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:40 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:23:41.097 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:40 smithi167 systemd[1]: Stopped Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:23:41.097 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:40 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 1min 48.562s CPU time. 2024-04-26T18:23:41.097 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:40 smithi167 systemd[1]: Starting Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:23:41.097 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 podman[70261]: 2024-04-26 18:23:40.958926527 +0000 UTC m=+0.018648413 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:41.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 podman[70261]: 2024-04-26 18:23:41.097509135 +0000 UTC m=+0.157230998 container create fc4ae0bcf576d0f730a12a609562a2e886d1f884c65aa9d76f8af3e8d50fe870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T18:23:41.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 podman[70261]: 2024-04-26 18:23:41.417042424 +0000 UTC m=+0.476764287 container init fc4ae0bcf576d0f730a12a609562a2e886d1f884c65aa9d76f8af3e8d50fe870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T18:23:41.698 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.00 MB/sec execute 449 sec latency 25265.944 ms 2024-04-26T18:23:41.804 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 podman[70261]: 2024-04-26 18:23:41.419822358 +0000 UTC m=+0.479544221 container start fc4ae0bcf576d0f730a12a609562a2e886d1f884c65aa9d76f8af3e8d50fe870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T18:23:41.804 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mds[70280]: -- 172.21.15.167:0/1206390891 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d5e839da40 con 0x55d5e8371000 2024-04-26T18:23:41.804 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: starting mds.c at 2024-04-26T18:23:41.804 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 bash[70261]: fc4ae0bcf576d0f730a12a609562a2e886d1f884c65aa9d76f8af3e8d50fe870 2024-04-26T18:23:41.805 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:23:41 smithi167 systemd[1]: Started Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: pgmap v381: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 912 KiB/s wr, 37 op/s 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: daemon mds.c restarted 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: osdmap e118: 12 total, 12 up, 12 in 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:boot 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:41 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: pgmap v381: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 912 KiB/s wr, 37 op/s 2024-04-26T18:23:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: daemon mds.c restarted 2024-04-26T18:23:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: osdmap e118: 12 total, 12 up, 12 in 2024-04-26T18:23:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:boot 2024-04-26T18:23:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:23:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:23:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:23:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:41 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: pgmap v381: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 912 KiB/s wr, 37 op/s 2024-04-26T18:23:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: daemon mds.c restarted 2024-04-26T18:23:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: osdmap e118: 12 total, 12 up, 12 in 2024-04-26T18:23:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:boot 2024-04-26T18:23:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:23:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:23:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:23:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:41 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:42.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 3.00 MB/sec execute 450 sec latency 26266.073 ms 2024-04-26T18:23:43.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 2.99 MB/sec execute 451 sec latency 27266.123 ms 2024-04-26T18:23:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:resolve 2024-04-26T18:23:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:43 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:43 smithi119 ceph-mon[32752]: pgmap v383: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 42 op/s 2024-04-26T18:23:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:resolve 2024-04-26T18:23:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:43 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:43 smithi167 ceph-mon[32751]: pgmap v383: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 42 op/s 2024-04-26T18:23:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:resolve 2024-04-26T18:23:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:43 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:43 smithi090 ceph-mon[29180]: pgmap v383: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 42 op/s 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe4cff9640 1 -- 172.21.15.90:0/1412899736 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629e325aee0 con 0x5629e336eec0 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe4cff9640 10 client.16368 handle_command_reply: tid=0 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 client.16368 shutdown 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32f80e0 msgr2=0x5629e32f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32f80e0 0x5629e32f2940 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efe60017ef0 tx=0x7efe60017f50 comp rx=0 tx=0).stop 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 shutdown_connections 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5629e336eec0 0x5629e33712e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5629e32f80e0 0x5629e32f2940 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5629e32ed820 0x5629e32f2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 --2- 172.21.15.90:0/1412899736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5629e32ecf00 0x5629e32f1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe4cff9640 0 client.16368 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 >> 172.21.15.90:0/1412899736 conn(0x5629e32e9410 msgr2=0x5629e32f7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 shutdown_connections 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 1 -- 172.21.15.90:0/1412899736 wait complete. 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.495+0000 7efe6e1e3740 20 client.16368 trim_cache size 0 max 16384 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:23:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:23:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe500a3dc0 msgr2=0x7efe500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe500a3dc0 0x7efe500a6280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efe5c0ef210 tx=0x7efe5c0c7680 comp rx=0 tx=0).stop 2024-04-26T18:23:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64107820 msgr2=0x7efe640ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64107820 0x7efe640ff270 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7efe5c005cc0 tx=0x7efe5c003c20 comp rx=0 tx=0).stop 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 shutdown_connections 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efe500a3dc0 0x7efe500a6280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe64107820 0x7efe640ff270 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe64102220 0x7efe640fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 --2- 172.21.15.90:0/2976116908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe64101860 0x7efe640fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 >> 172.21.15.90:0/2976116908 conn(0x7efe640789b0 msgr2=0x7efe6406a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 shutdown_connections 2024-04-26T18:23:44.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:44.496+0000 7efe6b3f0640 1 -- 172.21.15.90:0/2976116908 wait complete. 2024-04-26T18:23:44.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480264 2.98 MB/sec execute 452 sec latency 28266.219 ms 2024-04-26T18:23:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:44 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:23:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:44 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:23:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:44 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:23:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:reconnect 2024-04-26T18:23:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:reconnect 2024-04-26T18:23:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:44 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:44 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:23:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:44 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:23:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:44 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:23:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:reconnect 2024-04-26T18:23:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:reconnect 2024-04-26T18:23:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:44 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:44 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:23:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:44 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:23:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:44 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:23:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:reconnect 2024-04-26T18:23:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:reconnect 2024-04-26T18:23:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:44 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:45.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 480392 2.98 MB/sec execute 453 sec latency 29104.116 ms 2024-04-26T18:23:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:rejoin 2024-04-26T18:23:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:rejoin 2024-04-26T18:23:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:45 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:45 smithi119 ceph-mon[32752]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:23:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:45 smithi119 ceph-mon[32752]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:23:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:45 smithi119 ceph-mon[32752]: pgmap v384: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 46 op/s 2024-04-26T18:23:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:rejoin 2024-04-26T18:23:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:rejoin 2024-04-26T18:23:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:45 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:45 smithi167 ceph-mon[32751]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:23:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:45 smithi167 ceph-mon[32751]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:23:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:45 smithi167 ceph-mon[32751]: pgmap v384: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 46 op/s 2024-04-26T18:23:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:rejoin 2024-04-26T18:23:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:rejoin 2024-04-26T18:23:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:45 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:45 smithi090 ceph-mon[29180]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:23:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:45 smithi090 ceph-mon[29180]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:23:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:45 smithi090 ceph-mon[29180]: pgmap v384: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 46 op/s 2024-04-26T18:23:46.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 482642 3.00 MB/sec execute 454 sec latency 19.280 ms 2024-04-26T18:23:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:46 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:23:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:46 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:active 2024-04-26T18:23:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:46 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:active 2024-04-26T18:23:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:46 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:46 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:23:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:46 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:23:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:46 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:active 2024-04-26T18:23:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:46 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:active 2024-04-26T18:23:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:46 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:46 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:23:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:46 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:23:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:46 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/2177592042,v1:172.21.15.119:6835/2177592042] up:active 2024-04-26T18:23:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:46 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/766230459,v1:172.21.15.167:6833/766230459] up:active 2024-04-26T18:23:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:46 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:23:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:46 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:23:47.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 484045 3.01 MB/sec execute 455 sec latency 15.941 ms 2024-04-26T18:23:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:47 smithi119 ceph-mon[32752]: pgmap v385: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 978 KiB/s wr, 39 op/s 2024-04-26T18:23:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:47 smithi167 ceph-mon[32751]: pgmap v385: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 978 KiB/s wr, 39 op/s 2024-04-26T18:23:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:47 smithi090 ceph-mon[29180]: pgmap v385: 97 pgs: 97 active+clean; 147 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 978 KiB/s wr, 39 op/s 2024-04-26T18:23:48.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:23:48 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:23:48.226+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:23:48.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:23:48 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:23:48.226+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:23:48.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 486878 3.04 MB/sec execute 456 sec latency 14.183 ms 2024-04-26T18:23:49.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 488230 3.04 MB/sec execute 457 sec latency 14.881 ms 2024-04-26T18:23:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:49 smithi119 ceph-mon[32752]: pgmap v386: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 636 KiB/s wr, 104 op/s 2024-04-26T18:23:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:49 smithi167 ceph-mon[32751]: pgmap v386: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 636 KiB/s wr, 104 op/s 2024-04-26T18:23:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:49 smithi090 ceph-mon[29180]: pgmap v386: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 636 KiB/s wr, 104 op/s 2024-04-26T18:23:50.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fca013de700 time 2024-04-26T18:23:50.554741+0000 2024-04-26T18:23:50.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.07 MB/sec execute 458 sec latency 146.944 ms 2024-04-26T18:23:50.914 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:50.914 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.914 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca0ba073dd] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: clone() 2024-04-26T18:23:50.915 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.554+0000 7fca013de700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fca013de700 time 2024-04-26T18:23:50.554741+0000 2024-04-26T18:23:50.917 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:50.917 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.918 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.918 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca0ba073dd] 2024-04-26T18:23:50.918 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.919 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: clone() 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: *** Caught signal (Aborted) ** 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: in thread 7fca013de700 thread_name:ms_dispatch 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca0a78fd20] 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: gsignal() 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: abort() 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca0ba07437] 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.920 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 16: clone() 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.556+0000 7fca013de700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: in thread 7fca013de700 thread_name:ms_dispatch 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.921 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca0a78fd20] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: gsignal() 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: abort() 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca0ba07437] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.922 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 16: clone() 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: -1> 2024-04-26T18:23:50.554+0000 7fca013de700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fca013de700 time 2024-04-26T18:23:50.554741+0000 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca0ba073dd] 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.923 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: clone() 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 0> 2024-04-26T18:23:50.556+0000 7fca013de700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: in thread 7fca013de700 thread_name:ms_dispatch 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.924 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca0a78fd20] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: gsignal() 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: abort() 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca0ba07437] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.925 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 16: clone() 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: -1> 2024-04-26T18:23:50.554+0000 7fca013de700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fca013de700 time 2024-04-26T18:23:50.554741+0000 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fca0ba073dd] 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.926 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: clone() 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 0> 2024-04-26T18:23:50.556+0000 7fca013de700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: in thread 7fca013de700 thread_name:ms_dispatch 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.927 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fca0a78fd20] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2: gsignal() 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 3: abort() 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fca0ba07437] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fca0ba075a3] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556e2e91a822] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556e2e88a598] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556e2e88cc17] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556e2e785d0b] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556e2e7862ac] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556e2e76efad] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fca0bc9be20] 2024-04-26T18:23:50.928 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 13: (DispatchQueue::entry()+0x51f) [0x7fca0bc98fdf] 2024-04-26T18:23:50.929 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fca0bd6bfc1] 2024-04-26T18:23:50.929 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fca0a7851ca] 2024-04-26T18:23:50.929 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 16: clone() 2024-04-26T18:23:50.929 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:50.929 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[69838]: 2024-04-26T18:23:50.929 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:50 smithi119 podman[70072]: 2024-04-26 18:23:50.615623108 +0000 UTC m=+0.019247187 container died 1df1d07e7e0589698a21c37530caf84f384f9a1d85b26573910e4bc18550aa3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T18:23:51.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:51 smithi119 podman[70072]: 2024-04-26 18:23:51.23252351 +0000 UTC m=+0.636147623 container remove 1df1d07e7e0589698a21c37530caf84f384f9a1d85b26573910e4bc18550aa3a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-26T18:23:51.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:51 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:23:51.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:51 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-26T18:23:51.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:23:51 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.426s CPU time. 2024-04-26T18:23:51.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.06 MB/sec execute 459 sec latency 1147.007 ms 2024-04-26T18:23:52.241 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:51 smithi167 ceph-mon[32751]: pgmap v387: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 636 KiB/s wr, 104 op/s 2024-04-26T18:23:52.241 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe9c8f49700 time 2024-04-26T18:23:52.008314+0000 2024-04-26T18:23:52.241 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe9d35723dd] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.242 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.243 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: clone() 2024-04-26T18:23:52.243 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.008+0000 7fe9c8f49700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe9c8f49700 time 2024-04-26T18:23:52.008314+0000 2024-04-26T18:23:52.243 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:52.243 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe9d35723dd] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.244 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: clone() 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: *** Caught signal (Aborted) ** 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: in thread 7fe9c8f49700 thread_name:ms_dispatch 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe9d22fad20] 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: gsignal() 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: abort() 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe9d3572437] 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.245 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 16: clone() 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.009+0000 7fe9c8f49700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: in thread 7fe9c8f49700 thread_name:ms_dispatch 2024-04-26T18:23:52.246 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe9d22fad20] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: gsignal() 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: abort() 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe9d3572437] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.247 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 16: clone() 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: -1> 2024-04-26T18:23:52.008+0000 7fe9c8f49700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe9c8f49700 time 2024-04-26T18:23:52.008314+0000 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe9d35723dd] 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.248 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: clone() 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 0> 2024-04-26T18:23:52.009+0000 7fe9c8f49700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: in thread 7fe9c8f49700 thread_name:ms_dispatch 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.249 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe9d22fad20] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: gsignal() 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: abort() 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe9d3572437] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.250 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 16: clone() 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: -1> 2024-04-26T18:23:52.008+0000 7fe9c8f49700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe9c8f49700 time 2024-04-26T18:23:52.008314+0000 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe9d35723dd] 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.251 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.252 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: clone() 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 0> 2024-04-26T18:23:52.009+0000 7fe9c8f49700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: in thread 7fe9c8f49700 thread_name:ms_dispatch 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe9d22fad20] 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2: gsignal() 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 3: abort() 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe9d3572437] 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe9d35725a3] 2024-04-26T18:23:52.253 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5585d2a65822] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5585d29d5598] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5585d29d7c17] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5585d28d0d0b] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5585d28d12ac] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5585d28b9fad] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe9d3806e20] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 13: (DispatchQueue::entry()+0x51f) [0x7fe9d3803fdf] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe9d38d6fc1] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe9d22f01ca] 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 16: clone() 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:23:52.254 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 podman[70337]: 2024-04-26 18:23:52.077345471 +0000 UTC m=+0.020946126 container died 02d908f2d3cc5f8a4bb22ba2dd03a342772f68c776cd95838cf2e568135bd603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T18:23:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:51 smithi119 ceph-mon[32752]: pgmap v387: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 636 KiB/s wr, 104 op/s 2024-04-26T18:23:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:51 smithi090 ceph-mon[29180]: pgmap v387: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 636 KiB/s wr, 104 op/s 2024-04-26T18:23:52.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.05 MB/sec execute 460 sec latency 2147.134 ms 2024-04-26T18:23:53.067 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 podman[70337]: 2024-04-26 18:23:52.930580844 +0000 UTC m=+0.874181501 container remove 02d908f2d3cc5f8a4bb22ba2dd03a342772f68c776cd95838cf2e568135bd603 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T18:23:53.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:52 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:23:53.567 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:53 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-04-26T18:23:53.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:23:53 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 9.865s CPU time. 2024-04-26T18:23:53.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.05 MB/sec execute 461 sec latency 3147.225 ms 2024-04-26T18:23:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:53 smithi119 ceph-mon[32752]: pgmap v388: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 556 KiB/s wr, 91 op/s 2024-04-26T18:23:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:53 smithi167 ceph-mon[32751]: pgmap v388: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 556 KiB/s wr, 91 op/s 2024-04-26T18:23:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:53 smithi090 ceph-mon[29180]: pgmap v388: 97 pgs: 97 active+clean; 146 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 556 KiB/s wr, 91 op/s 2024-04-26T18:23:54.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.04 MB/sec execute 462 sec latency 4147.334 ms 2024-04-26T18:23:55.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.03 MB/sec execute 463 sec latency 5147.408 ms 2024-04-26T18:23:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:55 smithi119 ceph-mon[32752]: pgmap v389: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 964 KiB/s wr, 137 op/s 2024-04-26T18:23:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:55 smithi167 ceph-mon[32751]: pgmap v389: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 964 KiB/s wr, 137 op/s 2024-04-26T18:23:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:55 smithi090 ceph-mon[29180]: pgmap v389: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 964 KiB/s wr, 137 op/s 2024-04-26T18:23:56.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.03 MB/sec execute 464 sec latency 6147.530 ms 2024-04-26T18:23:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 -- 172.21.15.90:0/4048330404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102940 msgr2=0x7fc848102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 --2- 172.21.15.90:0/4048330404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102940 0x7fc848102dc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc83c009930 tx=0x7fc83c02a420 comp rx=0 tx=0).stop 2024-04-26T18:23:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 -- 172.21.15.90:0/4048330404 shutdown_connections 2024-04-26T18:23:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 --2- 172.21.15.90:0/4048330404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc848103300 0x7fc84810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 --2- 172.21.15.90:0/4048330404 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848102940 0x7fc848102dc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 --2- 172.21.15.90:0/4048330404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848108950 0x7fc848108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.595 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 -- 172.21.15.90:0/4048330404 >> 172.21.15.90:0/4048330404 conn(0x7fc848079dc0 msgr2=0x7fc84807a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 -- 172.21.15.90:0/4048330404 shutdown_connections 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 -- 172.21.15.90:0/4048330404 wait complete. 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.593+0000 7fc84d338640 1 Processor -- start 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc84d338640 1 -- start start 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc84d338640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc848102940 0x7fc848199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc84d338640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848103300 0x7fc848199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc84d338640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848108950 0x7fc84819d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc84d338640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc848112c10 con 0x7fc848108950 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc84d338640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc848112a90 con 0x7fc848102940 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc84d338640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc848112d90 con 0x7fc848103300 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848108950 0x7fc84819d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.596 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848108950 0x7fc84819d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49298/0 (socket says 172.21.15.90:49298) 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 -- 172.21.15.90:0/2526909389 learned_addr learned my addr 172.21.15.90:0/2526909389 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc846ffd640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc848102940 0x7fc848199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8467fc640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848103300 0x7fc848199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 -- 172.21.15.90:0/2526909389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848103300 msgr2=0x7fc848199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848103300 0x7fc848199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 -- 172.21.15.90:0/2526909389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc848102940 msgr2=0x7fc848199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc848102940 0x7fc848199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 -- 172.21.15.90:0/2526909389 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84819e000 con 0x7fc848108950 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.594+0000 7fc8477fe640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848108950 0x7fc84819d8d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c720 tx=0x7fc84000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.595+0000 7fc82ffff640 1 -- 172.21.15.90:0/2526909389 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840007bb0 con 0x7fc848108950 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.595+0000 7fc82ffff640 1 -- 172.21.15.90:0/2526909389 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc84000ce30 con 0x7fc848108950 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.595+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84819e2f0 con 0x7fc848108950 2024-04-26T18:23:57.597 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.595+0000 7fc82ffff640 1 -- 172.21.15.90:0/2526909389 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc840004db0 con 0x7fc848108950 2024-04-26T18:23:57.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.595+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8481a5b80 con 0x7fc848108950 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc82ffff640 1 -- 172.21.15.90:0/2526909389 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fc840005020 con 0x7fc848108950 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 Processor -- start 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 -- start start 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade173f00 0x559ade1742e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc82ffff640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8300a3dc0 0x7fc8300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade174820 0x559ade17eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade17f0e0 0x559ade1814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559ade0da460 con 0x559ade174820 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559ade094590 con 0x559ade173f00 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc850051740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ade0d6420 con 0x559ade17f0e0 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.597+0000 7fc846ffd640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8300a3dc0 0x7fc8300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade174820 0x559ade17eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade174820 0x559ade17eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49314/0 (socket says 172.21.15.90:49314) 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 -- 172.21.15.90:0/2367270219 learned_addr learned my addr 172.21.15.90:0/2367270219 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8477fe640 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade17f0e0 0x559ade1814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade173f00 0x559ade1742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc82ffff640 1 -- 172.21.15.90:0/2526909389 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9529+0+0 (secure 0 0 0) 0x7fc8400d07e0 con 0x7fc848108950 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 -- 172.21.15.90:0/2367270219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade17f0e0 msgr2=0x559ade1814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade17f0e0 0x559ade1814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 -- 172.21.15.90:0/2367270219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade173f00 msgr2=0x559ade1742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade173f00 0x559ade1742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.605 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 -- 172.21.15.90:0/2367270219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ade0a10d0 con 0x559ade174820 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc846ffd640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8300a3dc0 0x7fc8300a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc838007e40 tx=0x7fc8380075b0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc8467fc640 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade174820 0x559ade17eba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc83c005940 tx=0x7fc83c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2bbd8550a38c72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc82d7fa640 1 -- 172.21.15.90:0/2367270219 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c043030 con 0x559ade174820 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc82d7fa640 1 -- 172.21.15.90:0/2367270219 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc83c031620 con 0x559ade174820 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.598+0000 7fc82d7fa640 1 -- 172.21.15.90:0/2367270219 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83c035040 con 0x559ade174820 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 -- 172.21.15.90:0/2367270219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade174820 msgr2=0x559ade17eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade174820 0x559ade17eba0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc83c005940 tx=0x7fc83c0058b0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 -- 172.21.15.90:0/2367270219 shutdown_connections 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade17f0e0 0x559ade1814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade174820 0x559ade17eba0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 --2- 172.21.15.90:0/2367270219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade173f00 0x559ade1742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 -- 172.21.15.90:0/2367270219 >> 172.21.15.90:0/2367270219 conn(0x559ade170410 msgr2=0x559ade1707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 -- 172.21.15.90:0/2367270219 shutdown_connections 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 -- 172.21.15.90:0/2367270219 wait complete. 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 -- start start 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.599+0000 7fc850051740 1 Processor -- start 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade17c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade17c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559ade181cb0 con 0x559ade173f00 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559ade061b60 con 0x559ade17f0e0 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559ade181e30 con 0x559ade174820 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 10 client.0 _ll_register_callbacks cb 0x559ade0862b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:57.607 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade17c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade17c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49324/0 (socket says 172.21.15.90:49324) 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 -- 172.21.15.90:0/3850915488 learned_addr learned my addr 172.21.15.90:0/3850915488 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc8467fc640 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade17c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 -- 172.21.15.90:0/3850915488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 msgr2=0x559ade17c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade17c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 -- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 msgr2=0x559ade17cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 -- 172.21.15.90:0/3850915488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559adddec210 con 0x559ade173f00 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade17c040 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc838010e40 tx=0x7fc838004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b6b95dfdae0caa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.608 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc82aff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc82aff5640 1 -- 172.21.15.90:0/3850915488 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838026680 con 0x559ade173f00 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc82aff5640 1 -- 172.21.15.90:0/3850915488 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9529+0+0 (secure 0 0 0) 0x7fc838026820 con 0x559ade173f00 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.600+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc83c031620 con 0x559ade173f00 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc82aff5640 1 -- 172.21.15.90:0/3850915488 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fc838026a20 con 0x559ade173f00 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 10 client.16380 fetch_fsmap learned FSMap version 113 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=4+}) -- 0x559addeb8620 con 0x559ade173f00 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc82aff5640 1 -- 172.21.15.90:0/3850915488 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc83c031620 con 0x559ade173f00 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc82aff5640 1 -- 172.21.15.90:0/3850915488 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 113) ==== 7128+0+0 (secure 0 0 0) 0x7fc83801dc00 con 0x559ade173f00 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 10 client.16380 fetch_fsmap finished waiting for FSMap version 113 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 20 client.16380 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade18f1e0 0x559ade1915f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 4 client.16380 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ade0e8070 con 0x559ade18f1e0 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.601+0000 7fc8467fc640 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade18f1e0 0x559ade1915f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.602+0000 7fc82aff5640 10 client.16380 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.602+0000 7fc8467fc640 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade18f1e0 0x559ade1915f0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=728c65bef4a7e3f4 server_cookie=31aee7a2dd1a21a in_seq=0 out_seq=0 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.602+0000 7fc82aff5640 1 -- 172.21.15.90:0/3850915488 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ade0e8070 con 0x559ade18f1e0 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.602+0000 7fc82aff5640 10 client.16380 handle_command_reply: tid=0 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 client.16380 shutdown 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 msgr2=0x559ade17c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade17c040 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc838010e40 tx=0x7fc838004ab0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 shutdown_connections 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade18f1e0 0x559ade1915f0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade17c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc82aff5640 0 client.16380 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 --2- 172.21.15.90:0/3850915488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade17c040 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 >> 172.21.15.90:0/3850915488 conn(0x559ade170410 msgr2=0x559ade176190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 shutdown_connections 2024-04-26T18:23:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 1 -- 172.21.15.90:0/3850915488 wait complete. 2024-04-26T18:23:57.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.603+0000 7fc850051740 20 client.16380 trim_cache size 0 max 16384 2024-04-26T18:23:57.622 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 Processor -- start 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 -- start start 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade18ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade176910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481af9b0 con 0x559ade173f00 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b0080 con 0x559ade17f0e0 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc850051740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481b0750 con 0x559ade174820 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade18ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade18ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49328/0 (socket says 172.21.15.90:49328) 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 -- 172.21.15.90:0/3842423018 learned_addr learned my addr 172.21.15.90:0/3842423018 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc8477fe640 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 -- 172.21.15.90:0/3842423018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 msgr2=0x559ade176910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade176910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 -- 172.21.15.90:0/3842423018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 msgr2=0x559ade17df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 -- 172.21.15.90:0/3842423018 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ade176e50 con 0x559ade173f00 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc846ffd640 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade18ab70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc838010890 tx=0x7fc83801bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336914e822137b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.618+0000 7fc82cff9640 1 -- 172.21.15.90:0/3842423018 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838038070 con 0x559ade173f00 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc82cff9640 1 -- 172.21.15.90:0/3842423018 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc838026dc0 con 0x559ade173f00 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc82cff9640 1 -- 172.21.15.90:0/3842423018 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc83801ce00 con 0x559ade173f00 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 -- 172.21.15.90:0/3842423018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 msgr2=0x559ade18ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade18ab70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc838010890 tx=0x7fc83801bee0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 -- 172.21.15.90:0/3842423018 shutdown_connections 2024-04-26T18:23:57.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade17df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade176910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 --2- 172.21.15.90:0/3842423018 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade18ab70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 -- 172.21.15.90:0/3842423018 >> 172.21.15.90:0/3842423018 conn(0x559ade170410 msgr2=0x559ade17d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 -- 172.21.15.90:0/3842423018 shutdown_connections 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 -- 172.21.15.90:0/3842423018 wait complete. 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 -- start start 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.619+0000 7fc850051740 1 Processor -- start 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade1797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade179d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade1784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481acc90 con 0x559ade173f00 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ae0a0 con 0x559ade17f0e0 2024-04-26T18:23:57.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8481ad300 con 0x559ade174820 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 10 client.0 _ll_register_callbacks cb 0x7fc838007a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade1797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade1797e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49338/0 (socket says 172.21.15.90:49338) 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 -- 172.21.15.90:0/459225238 learned_addr learned my addr 172.21.15.90:0/459225238 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc8477fe640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade1784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 -- 172.21.15.90:0/459225238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 msgr2=0x559ade179d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc8467fc640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade179d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade179d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 -- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 msgr2=0x559ade1784d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade1784d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 -- 172.21.15.90:0/459225238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ade178cd0 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc8477fe640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade1784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc8467fc640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade179d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc846ffd640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade1797e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc83801c010 tx=0x7fc83801ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1152883cbd92ec0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc82aff5640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc82aff5640 1 -- 172.21.15.90:0/459225238 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838042060 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc82aff5640 1 -- 172.21.15.90:0/459225238 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9529+0+0 (secure 0 0 0) 0x7fc838026bc0 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.620+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc838026dc0 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc82aff5640 1 -- 172.21.15.90:0/459225238 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) ==== 24+0+0 (secure 0 0 0) 0x7fc838049050 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 10 client.16386 fetch_fsmap learned FSMap version 113 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=4+}) -- 0x559ade1fa3f0 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc82aff5640 1 -- 172.21.15.90:0/459225238 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc838026dc0 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc82aff5640 1 -- 172.21.15.90:0/459225238 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 113) ==== 7128+0+0 (secure 0 0 0) 0x7fc838005360 con 0x559ade173f00 2024-04-26T18:23:57.627 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 10 client.16386 fetch_fsmap finished waiting for FSMap version 113 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 10 client.16386 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 20 client.16386 populate_metadata read hostname 'smithi090' 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade1fa9a0 0x559ade1fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 4 client.16386 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ade0e1ee0 con 0x559ade1fa9a0 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.621+0000 7fc8467fc640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade1fa9a0 0x559ade1fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.622+0000 7fc82aff5640 10 client.16386 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:23:57.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:23:57.622+0000 7fc8467fc640 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade1fa9a0 0x559ade1fcdc0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91406ebaf54c1397 server_cookie=25f89b63b1617f82 in_seq=0 out_seq=0 2024-04-26T18:23:57.699 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.02 MB/sec execute 465 sec latency 7147.616 ms 2024-04-26T18:23:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:57 smithi119 ceph-mon[32752]: pgmap v390: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 960 KiB/s wr, 135 op/s 2024-04-26T18:23:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:57 smithi167 ceph-mon[32751]: pgmap v390: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 960 KiB/s wr, 135 op/s 2024-04-26T18:23:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:57 smithi090 ceph-mon[29180]: pgmap v390: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 960 KiB/s wr, 135 op/s 2024-04-26T18:23:58.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.01 MB/sec execute 466 sec latency 8147.690 ms 2024-04-26T18:23:59.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.01 MB/sec execute 467 sec latency 9147.768 ms 2024-04-26T18:24:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:23:59 smithi119 ceph-mon[32752]: pgmap v391: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 983 KiB/s wr, 157 op/s 2024-04-26T18:24:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:23:59 smithi167 ceph-mon[32751]: pgmap v391: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 983 KiB/s wr, 157 op/s 2024-04-26T18:24:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:23:59 smithi090 ceph-mon[29180]: pgmap v391: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 983 KiB/s wr, 157 op/s 2024-04-26T18:24:00.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.00 MB/sec execute 468 sec latency 10147.902 ms 2024-04-26T18:24:01.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 3.00 MB/sec execute 469 sec latency 11148.011 ms 2024-04-26T18:24:01.871 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:01 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:24:01.871 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:01 smithi119 systemd[1]: Stopped Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:01.872 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:01 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.426s CPU time. 2024-04-26T18:24:01.872 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:01 smithi119 systemd[1]: Starting Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:24:01.872 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:01 smithi119 podman[70170]: 2024-04-26 18:24:01.685445702 +0000 UTC m=+0.018153698 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:01.872 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:01 smithi119 podman[70170]: 2024-04-26 18:24:01.872366544 +0000 UTC m=+0.205074512 container create e8546b309f2f8f9bbca1fa2a2732748a677790ea9e054c05cf661d2b97d0cc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T18:24:02.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:01 smithi119 ceph-mon[32752]: pgmap v392: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 457 KiB/s wr, 73 op/s 2024-04-26T18:24:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:01 smithi167 ceph-mon[32751]: pgmap v392: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 457 KiB/s wr, 73 op/s 2024-04-26T18:24:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:01 smithi090 ceph-mon[29180]: pgmap v392: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 457 KiB/s wr, 73 op/s 2024-04-26T18:24:02.393 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:02 smithi119 podman[70170]: 2024-04-26 18:24:02.12756461 +0000 UTC m=+0.460272579 container init e8546b309f2f8f9bbca1fa2a2732748a677790ea9e054c05cf661d2b97d0cc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T18:24:02.393 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:02 smithi119 podman[70170]: 2024-04-26 18:24:02.130413304 +0000 UTC m=+0.463121272 container start e8546b309f2f8f9bbca1fa2a2732748a677790ea9e054c05cf661d2b97d0cc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T18:24:02.393 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mds[70189]: -- 172.21.15.119:0/1350969301 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562abcd3da40 con 0x562abcd10800 2024-04-26T18:24:02.393 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: starting mds.e at 2024-04-26T18:24:02.393 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:02 smithi119 bash[70170]: e8546b309f2f8f9bbca1fa2a2732748a677790ea9e054c05cf661d2b97d0cc74 2024-04-26T18:24:02.393 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:24:02 smithi119 systemd[1]: Started Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:02.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 2.99 MB/sec execute 470 sec latency 12148.116 ms 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: daemon mds.e restarted 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: osdmap e119: 12 total, 12 up, 12 in 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:boot 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=l=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:02 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=l=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: daemon mds.e restarted 2024-04-26T18:24:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:24:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: osdmap e119: 12 total, 12 up, 12 in 2024-04-26T18:24:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:boot 2024-04-26T18:24:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-04-26T18:24:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=l=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:24:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:24:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:02 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=l=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: daemon mds.e restarted 2024-04-26T18:24:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:24:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: osdmap e119: 12 total, 12 up, 12 in 2024-04-26T18:24:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:boot 2024-04-26T18:24:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: Standby daemon mds.c assigned to filesystem cephfs as rank 1 2024-04-26T18:24:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=l=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:24:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:24:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:02 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:replay,2=l=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:03.614 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:24:03.614 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 systemd[1]: Stopped Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:03.614 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 9.865s CPU time. 2024-04-26T18:24:03.614 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 systemd[1]: Starting Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:24:03.614 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 podman[70436]: 2024-04-26 18:24:03.469881694 +0000 UTC m=+0.018899638 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:03.615 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 podman[70436]: 2024-04-26 18:24:03.614943575 +0000 UTC m=+0.163961495 container create a2ed6181c689a60f85f4c71d52435acff6c0ff0623fece2ab6a27a7059833a1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T18:24:03.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 2.98 MB/sec execute 471 sec latency 13148.180 ms 2024-04-26T18:24:04.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 podman[70436]: 2024-04-26 18:24:03.867608841 +0000 UTC m=+0.416626771 container init a2ed6181c689a60f85f4c71d52435acff6c0ff0623fece2ab6a27a7059833a1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:24:04.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 podman[70436]: 2024-04-26 18:24:03.870413553 +0000 UTC m=+0.419431472 container start a2ed6181c689a60f85f4c71d52435acff6c0ff0623fece2ab6a27a7059833a1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T18:24:04.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 ceph-mds[70454]: -- 172.21.15.167:0/1246172484 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558c2db43a40 con 0x558c2db16800 2024-04-26T18:24:04.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: starting mds.l at 2024-04-26T18:24:04.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 bash[70436]: a2ed6181c689a60f85f4c71d52435acff6c0ff0623fece2ab6a27a7059833a1a 2024-04-26T18:24:04.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:03 smithi167 systemd[1]: Started Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: pgmap v394: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 548 KiB/s wr, 87 op/s 2024-04-26T18:24:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:resolve 2024-04-26T18:24:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=l=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: daemon mds.l restarted 2024-04-26T18:24:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: osdmap e120: 12 total, 12 up, 12 in 2024-04-26T18:24:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:boot 2024-04-26T18:24:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: Standby daemon mds.k assigned to filesystem cephfs as rank 2 2024-04-26T18:24:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=c=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:24:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:24:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=k=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: pgmap v394: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 548 KiB/s wr, 87 op/s 2024-04-26T18:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:resolve 2024-04-26T18:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=l=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: daemon mds.l restarted 2024-04-26T18:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: osdmap e120: 12 total, 12 up, 12 in 2024-04-26T18:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:boot 2024-04-26T18:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: Standby daemon mds.k assigned to filesystem cephfs as rank 2 2024-04-26T18:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=c=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:24:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:24:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=k=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: pgmap v394: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 548 KiB/s wr, 87 op/s 2024-04-26T18:24:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:resolve 2024-04-26T18:24:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=l=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: daemon mds.l restarted 2024-04-26T18:24:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: osdmap e120: 12 total, 12 up, 12 in 2024-04-26T18:24:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:boot 2024-04-26T18:24:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: Standby daemon mds.k assigned to filesystem cephfs as rank 2 2024-04-26T18:24:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=c=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:24:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:24:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=k=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:04.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 2.98 MB/sec execute 472 sec latency 14148.261 ms 2024-04-26T18:24:05.700 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 2.97 MB/sec execute 473 sec latency 15148.381 ms 2024-04-26T18:24:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:06 smithi119 ceph-mon[32752]: pgmap v396: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-04-26T18:24:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:resolve 2024-04-26T18:24:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:06 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=k=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:06 smithi167 ceph-mon[32751]: pgmap v396: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-04-26T18:24:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:resolve 2024-04-26T18:24:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:06 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=k=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:06 smithi090 ceph-mon[29180]: pgmap v396: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-04-26T18:24:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:resolve 2024-04-26T18:24:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:06 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:resolve,2=k=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:06.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490547 2.96 MB/sec execute 474 sec latency 16148.477 ms 2024-04-26T18:24:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc82aff5640 1 -- 172.21.15.90:0/459225238 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ade0e1ee0 con 0x559ade1fa9a0 2024-04-26T18:24:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc82aff5640 10 client.16386 handle_command_reply: tid=0 2024-04-26T18:24:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 client.16386 shutdown 2024-04-26T18:24:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 msgr2=0x559ade1797e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade1797e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc83801c010 tx=0x7fc83801ce10 comp rx=0 tx=0).stop 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 shutdown_connections 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559ade1fa9a0 0x559ade1fcdc0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559ade17f0e0 0x559ade1784d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559ade174820 0x559ade179d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 --2- 172.21.15.90:0/459225238 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559ade173f00 0x559ade1797e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc82aff5640 0 client.16386 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 >> 172.21.15.90:0/459225238 conn(0x559ade170410 msgr2=0x559ade16f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 shutdown_connections 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 1 -- 172.21.15.90:0/459225238 wait complete. 2024-04-26T18:24:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc850051740 20 client.16386 trim_cache size 0 max 16384 2024-04-26T18:24:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:24:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:24:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:24:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8300a3dc0 msgr2=0x7fc8300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc84d338640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8300a3dc0 0x7fc8300a6280 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc838007e40 tx=0x7fc8380075b0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.215+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848108950 msgr2=0x7fc84819d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848108950 0x7fc84819d8d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c720 tx=0x7fc84000cbe0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 shutdown_connections 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc8300a3dc0 0x7fc8300a6280 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc848108950 0x7fc84819d8d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc848103300 0x7fc848199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 --2- 172.21.15.90:0/2526909389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc848102940 0x7fc848199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 >> 172.21.15.90:0/2526909389 conn(0x7fc848079dc0 msgr2=0x7fc848100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 shutdown_connections 2024-04-26T18:24:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:07.216+0000 7fc84d338640 1 -- 172.21.15.90:0/2526909389 wait complete. 2024-04-26T18:24:07.546 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:24:07.546 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:24:07.546 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:24:07.546 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:reconnect 2024-04-26T18:24:07.546 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:reconnect 2024-04-26T18:24:07.547 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:reconnect,2=k=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:07.547 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:rejoin 2024-04-26T18:24:07.547 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:rejoin 2024-04-26T18:24:07.547 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:07 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:rejoin,2=k=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:reconnect 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:reconnect 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:reconnect,2=k=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:rejoin 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:rejoin 2024-04-26T18:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:07 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:rejoin,2=k=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:reconnect 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:reconnect 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:reconnect,2=k=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:rejoin 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:rejoin 2024-04-26T18:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:07 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=c=up:rejoin,2=k=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:07.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 490582 2.96 MB/sec execute 475 sec latency 17039.866 ms 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:24:08.227+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:24:08.227+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-mon[32752]: pgmap v397: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-mon[32752]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-mon[32752]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:active 2024-04-26T18:24:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:active 2024-04-26T18:24:08.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:08 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:08 smithi167 ceph-mon[32751]: pgmap v397: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-26T18:24:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:08 smithi167 ceph-mon[32751]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-26T18:24:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:08 smithi167 ceph-mon[32751]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-04-26T18:24:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:08 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:24:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:active 2024-04-26T18:24:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:active 2024-04-26T18:24:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:08 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:08 smithi090 ceph-mon[29180]: pgmap v397: 97 pgs: 97 active+clean; 153 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 1.1 MiB/s wr, 36 op/s 2024-04-26T18:24:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:08 smithi090 ceph-mon[29180]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-04-26T18:24:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:08 smithi090 ceph-mon[29180]: daemon mds.k is now active in filesystem cephfs as rank 2 2024-04-26T18:24:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:08 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:24:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1187834156,v1:172.21.15.167:6837/1187834156] up:active 2024-04-26T18:24:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/137310630,v1:172.21.15.119:6833/137310630] up:active 2024-04-26T18:24:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:08 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:08.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 491733 2.96 MB/sec execute 476 sec latency 19.178 ms 2024-04-26T18:24:09.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 494151 2.99 MB/sec execute 477 sec latency 11.563 ms 2024-04-26T18:24:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:10 smithi119 ceph-mon[32752]: pgmap v398: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-26T18:24:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:10 smithi167 ceph-mon[32751]: pgmap v398: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-26T18:24:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:10 smithi090 ceph-mon[29180]: pgmap v398: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-26T18:24:10.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 495520 2.99 MB/sec execute 478 sec latency 28.867 ms 2024-04-26T18:24:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:11 smithi119 ceph-mon[32752]: osdmap e121: 12 total, 12 up, 12 in 2024-04-26T18:24:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:11 smithi167 ceph-mon[32751]: osdmap e121: 12 total, 12 up, 12 in 2024-04-26T18:24:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:11 smithi090 ceph-mon[29180]: osdmap e121: 12 total, 12 up, 12 in 2024-04-26T18:24:11.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 497791 3.01 MB/sec execute 479 sec latency 38.379 ms 2024-04-26T18:24:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:12 smithi119 ceph-mon[32752]: pgmap v400: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-26T18:24:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:12 smithi167 ceph-mon[32751]: pgmap v400: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-26T18:24:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:12 smithi090 ceph-mon[29180]: pgmap v400: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-04-26T18:24:12.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 499857 3.02 MB/sec execute 480 sec latency 19.649 ms 2024-04-26T18:24:13.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 3.04 MB/sec execute 481 sec latency 709.700 ms 2024-04-26T18:24:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:14 smithi119 ceph-mon[32752]: pgmap v401: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 274 KiB/s wr, 59 op/s 2024-04-26T18:24:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:14 smithi167 ceph-mon[32751]: pgmap v401: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 274 KiB/s wr, 59 op/s 2024-04-26T18:24:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:14 smithi090 ceph-mon[29180]: pgmap v401: 97 pgs: 97 active+clean; 151 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 274 KiB/s wr, 59 op/s 2024-04-26T18:24:14.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 3.03 MB/sec execute 482 sec latency 1709.757 ms 2024-04-26T18:24:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:15.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 3.02 MB/sec execute 483 sec latency 2709.800 ms 2024-04-26T18:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:16 smithi119 ceph-mon[32752]: pgmap v402: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-04-26T18:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:16 smithi119 ceph-mon[32752]: pgmap v403: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-04-26T18:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:16 smithi119 ceph-mon[32752]: pgmap v404: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-26T18:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:16 smithi167 ceph-mon[32751]: pgmap v402: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-04-26T18:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:16 smithi167 ceph-mon[32751]: pgmap v403: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-04-26T18:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:16 smithi167 ceph-mon[32751]: pgmap v404: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-26T18:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:24:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:16 smithi090 ceph-mon[29180]: pgmap v402: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-04-26T18:24:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:16 smithi090 ceph-mon[29180]: pgmap v403: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.5 MiB/s wr, 214 op/s 2024-04-26T18:24:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:16 smithi090 ceph-mon[29180]: pgmap v404: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-26T18:24:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:24:16.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 3.02 MB/sec execute 484 sec latency 3709.905 ms 2024-04-26T18:24:17.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 3.01 MB/sec execute 485 sec latency 4710.036 ms 2024-04-26T18:24:18.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 3.01 MB/sec execute 486 sec latency 5710.170 ms 2024-04-26T18:24:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:18 smithi167 ceph-mon[32751]: osdmap e122: 12 total, 12 up, 12 in 2024-04-26T18:24:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:18 smithi167 ceph-mon[32751]: pgmap v406: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-26T18:24:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:18 smithi090 ceph-mon[29180]: osdmap e122: 12 total, 12 up, 12 in 2024-04-26T18:24:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:18 smithi090 ceph-mon[29180]: pgmap v406: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-26T18:24:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:18 smithi119 ceph-mon[32752]: osdmap e122: 12 total, 12 up, 12 in 2024-04-26T18:24:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:18 smithi119 ceph-mon[32752]: pgmap v406: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.6 MiB/s wr, 198 op/s 2024-04-26T18:24:19.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 3.00 MB/sec execute 487 sec latency 6710.252 ms 2024-04-26T18:24:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/4057760481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4107820 msgr2=0x7f6ba4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/4057760481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4107820 0x7f6ba4107c00 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6b94009930 tx=0x7f6b9402a420 comp rx=0 tx=0).stop 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/4057760481 shutdown_connections 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/4057760481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4102220 0x7f6ba410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/4057760481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ba4101860 0x7f6ba4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/4057760481 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4107820 0x7f6ba4107c00 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/4057760481 >> 172.21.15.90:0/4057760481 conn(0x7f6ba40789b0 msgr2=0x7f6ba4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/4057760481 shutdown_connections 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/4057760481 wait complete. 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.313+0000 7f6ba9b6a640 1 Processor -- start 2024-04-26T18:24:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba9b6a640 1 -- start start 2024-04-26T18:24:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba9b6a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101860 0x7f6ba410f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba9b6a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4102220 0x7f6ba410f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba9b6a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ba4107820 0x7f6ba410fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba9b6a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4112be0 con 0x7f6ba4102220 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba9b6a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4112a60 con 0x7f6ba4101860 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba9b6a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4112d60 con 0x7f6ba4107820 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4102220 0x7f6ba410f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4102220 0x7f6ba410f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55294/0 (socket says 172.21.15.90:55294) 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/2844598631 learned_addr learned my addr 172.21.15.90:0/2844598631 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101860 0x7f6ba410f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ba4107820 0x7f6ba410fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/2844598631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ba4107820 msgr2=0x7f6ba410fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ba4107820 0x7f6ba410fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/2844598631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101860 msgr2=0x7f6ba410f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101860 0x7f6ba410f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/2844598631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba41a7660 con 0x7f6ba4102220 2024-04-26T18:24:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.314+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4102220 0x7f6ba410f6b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9f0 tx=0x7f6b9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.315+0000 7f6ba0ff9640 1 -- 172.21.15.90:0/2844598631 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9800cd80 con 0x7f6ba4102220 2024-04-26T18:24:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.315+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba41a7950 con 0x7f6ba4102220 2024-04-26T18:24:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.315+0000 7f6ba0ff9640 1 -- 172.21.15.90:0/2844598631 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b980054d0 con 0x7f6ba4102220 2024-04-26T18:24:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.315+0000 7f6ba0ff9640 1 -- 172.21.15.90:0/2844598631 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98010690 con 0x7f6ba4102220 2024-04-26T18:24:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.315+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41a7e50 con 0x7f6ba4102220 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6ba0ff9640 1 -- 172.21.15.90:0/2844598631 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f6b98004e90 con 0x7f6ba4102220 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 Processor -- start 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 -- start start 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dbd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dc7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6dca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561ea6d23460 con 0x561ea6dbcf00 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561ea6cdd590 con 0x561ea6dc80e0 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.317+0000 7f6bac882740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ea6d1f420 con 0x561ea6dbd820 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dbd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dbd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55298/0 (socket says 172.21.15.90:55298) 2024-04-26T18:24:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 -- 172.21.15.90:0/1903540148 learned_addr learned my addr 172.21.15.90:0/1903540148 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba0ff9640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6b8c0a3dc0 0x7f6b8c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dc7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6dca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6b8c0a3dc0 0x7f6b8c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 -- 172.21.15.90:0/1903540148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 msgr2=0x561ea6dc7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dc7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 -- 172.21.15.90:0/1903540148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 msgr2=0x561ea6dca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba0ff9640 1 -- 172.21.15.90:0/2844598631 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9745+0+0 (secure 0 0 0) 0x7f6b980d11e0 con 0x7f6ba4102220 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6dca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 -- 172.21.15.90:0/1903540148 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea6cea0d0 con 0x561ea6dbcf00 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dc7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:20.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dbd2e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0094d0 tx=0x7f6b9c013dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e2c1a650a53c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6b89ffb640 1 -- 172.21.15.90:0/1903540148 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c013250 con 0x561ea6dbcf00 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6b89ffb640 1 -- 172.21.15.90:0/1903540148 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c0133f0 con 0x561ea6dbcf00 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.318+0000 7f6b89ffb640 1 -- 172.21.15.90:0/1903540148 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c012a00 con 0x561ea6dbcf00 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 -- 172.21.15.90:0/1903540148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 msgr2=0x561ea6dbd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6b8c0a3dc0 0x7f6b8c0a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c022000 tx=0x7f6b9c020000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dbd2e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0094d0 tx=0x7f6b9c013dc0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 -- 172.21.15.90:0/1903540148 shutdown_connections 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6dca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dc7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 --2- 172.21.15.90:0/1903540148 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dbd2e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 -- 172.21.15.90:0/1903540148 >> 172.21.15.90:0/1903540148 conn(0x561ea6db9410 msgr2=0x561ea6db97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 -- 172.21.15.90:0/1903540148 shutdown_connections 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.319+0000 7f6bac882740 1 -- 172.21.15.90:0/1903540148 wait complete. 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 -- start start 2024-04-26T18:24:20.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 Processor -- start 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dc4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbd820 0x561ea6dc5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dc80e0 0x561ea6dc5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x561ea6dcad10 con 0x561ea6dbcf00 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x561ea6caab60 con 0x561ea6dbd820 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561ea6dcae90 con 0x561ea6dc80e0 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6bac882740 10 client.0 _ll_register_callbacks cb 0x561ea6ccf2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dc4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dc4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55304/0 (socket says 172.21.15.90:55304) 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba37fe640 1 -- 172.21.15.90:0/3379935827 learned_addr learned my addr 172.21.15.90:0/3379935827 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dc80e0 0x561ea6dc5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbd820 0x561ea6dc5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.328 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba37fe640 1 -- 172.21.15.90:0/3379935827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dc80e0 msgr2=0x561ea6dc5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dc80e0 0x561ea6dc5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba37fe640 1 -- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbd820 msgr2=0x561ea6dc5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.320+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbd820 0x561ea6dc5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6ba37fe640 1 -- 172.21.15.90:0/3379935827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea6a35210 con 0x561ea6dbcf00 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dc80e0 0x561ea6dc5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dc4fc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6b9402a940 tx=0x7f6b94009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4167d0a307c701ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6b82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3379935827 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b94035040 con 0x561ea6dbcf00 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3379935827 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9745+0+0 (secure 0 0 0) 0x7f6b940316a0 con 0x561ea6dbcf00 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9c0133f0 con 0x561ea6dbcf00 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3379935827 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f6b9404d050 con 0x561ea6dbcf00 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6bac882740 10 client.16404 fetch_fsmap learned FSMap version 123 2024-04-26T18:24:20.329 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) -- 0x561ea6b01620 con 0x561ea6dbcf00 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3379935827 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c0133f0 con 0x561ea6dbcf00 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3379935827 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 123) ==== 7132+0+0 (secure 0 0 0) 0x7f6b94003b10 con 0x561ea6dbcf00 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6bac882740 10 client.16404 fetch_fsmap finished waiting for FSMap version 123 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6bac882740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.321+0000 7f6bac882740 20 client.16404 populate_metadata read hostname 'smithi090' 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.322+0000 7f6bac882740 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6dd8160 0x561ea6dda550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.322+0000 7f6bac882740 4 client.16404 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.322+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ea6d31070 con 0x561ea6dd8160 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.322+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6dd8160 0x561ea6dda550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.322+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6dd8160 0x561ea6dda550 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6290cfe998424358 server_cookie=938660f8687ba17 in_seq=0 out_seq=0 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.322+0000 7f6b82ffd640 10 client.16404 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3379935827 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ea6d31070 con 0x561ea6dd8160 2024-04-26T18:24:20.330 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6b82ffd640 10 client.16404 handle_command_reply: tid=0 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 client.16404 shutdown 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 msgr2=0x561ea6dc4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dc4fc0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f6b9402a940 tx=0x7f6b94009680 comp rx=0 tx=0).stop 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 shutdown_connections 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6dd8160 0x561ea6dda550 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dc80e0 0x561ea6dc5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbd820 0x561ea6dc5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 --2- 172.21.15.90:0/3379935827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbcf00 0x561ea6dc4fc0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6b82ffd640 0 client.16404 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 >> 172.21.15.90:0/3379935827 conn(0x561ea6db9410 msgr2=0x561ea6dbf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 shutdown_connections 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 1 -- 172.21.15.90:0/3379935827 wait complete. 2024-04-26T18:24:20.331 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.323+0000 7f6bac882740 20 client.16404 trim_cache size 0 max 16384 2024-04-26T18:24:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 Processor -- start 2024-04-26T18:24:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 -- start start 2024-04-26T18:24:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbcf00 0x561ea6dd3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dbf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dc80e0 0x561ea6ddfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41b1e40 con 0x561ea6dc80e0 2024-04-26T18:24:20.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41b2510 con 0x561ea6dbcf00 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.338+0000 7f6bac882740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41b2be0 con 0x561ea6dbd820 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dc80e0 0x561ea6ddfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dc80e0 0x561ea6ddfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55306/0 (socket says 172.21.15.90:55306) 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 -- 172.21.15.90:0/3585595314 learned_addr learned my addr 172.21.15.90:0/3585595314 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dbf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbcf00 0x561ea6dd3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 -- 172.21.15.90:0/3585595314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 msgr2=0x561ea6dbf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dbf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 -- 172.21.15.90:0/3585595314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbcf00 msgr2=0x561ea6dd3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbcf00 0x561ea6dd3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 -- 172.21.15.90:0/3585595314 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea6db4d60 con 0x561ea6dc80e0 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbcf00 0x561ea6dd3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dc80e0 0x561ea6ddfe40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009540 tx=0x7f6b9c009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e008112ae6c7e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6b897fa640 1 -- 172.21.15.90:0/3585595314 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c003970 con 0x561ea6dc80e0 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6b897fa640 1 -- 172.21.15.90:0/3585595314 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c013050 con 0x561ea6dc80e0 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.339+0000 7f6b897fa640 1 -- 172.21.15.90:0/3585595314 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c01fd60 con 0x561ea6dc80e0 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 -- 172.21.15.90:0/3585595314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dc80e0 msgr2=0x561ea6ddfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dc80e0 0x561ea6ddfe40 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009540 tx=0x7f6b9c009ae0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 -- 172.21.15.90:0/3585595314 shutdown_connections 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dc80e0 0x561ea6ddfe40 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbd820 0x561ea6dbf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 --2- 172.21.15.90:0/3585595314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dbcf00 0x561ea6dd3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 -- 172.21.15.90:0/3585595314 >> 172.21.15.90:0/3585595314 conn(0x561ea6db9410 msgr2=0x561ea6db85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 -- 172.21.15.90:0/3585595314 shutdown_connections 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 -- 172.21.15.90:0/3585595314 wait complete. 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 -- start start 2024-04-26T18:24:20.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.340+0000 7f6bac882740 1 Processor -- start 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbcf00 0x561ea6dcc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbd820 0x561ea6dcc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6e376f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41af140 con 0x561ea6dbd820 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41b0530 con 0x561ea6dc80e0 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba41af790 con 0x561ea6dbcf00 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 10 client.0 _ll_register_callbacks cb 0x7f6b98005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbd820 0x561ea6dcc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbd820 0x561ea6dcc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55308/0 (socket says 172.21.15.90:55308) 2024-04-26T18:24:20.346 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/3068125932 learned_addr learned my addr 172.21.15.90:0/3068125932 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbcf00 0x561ea6dcc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba3fff640 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6e376f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/3068125932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbcf00 msgr2=0x561ea6dcc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbcf00 0x561ea6dcc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 msgr2=0x561ea6e376f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6e376f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 -- 172.21.15.90:0/3068125932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea6e37cb0 con 0x561ea6dbd820 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6ba2ffd640 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbd820 0x561ea6dcc8c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6b980ce460 tx=0x7f6b9806c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac03429b64099115 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6b82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3068125932 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9809d350 con 0x561ea6dbd820 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3068125932 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9745+0+0 (secure 0 0 0) 0x7f6b980992c0 con 0x561ea6dbd820 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b9c013050 con 0x561ea6dbd820 2024-04-26T18:24:20.347 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3068125932 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=123) ==== 24+0+0 (secure 0 0 0) 0x7f6b9809cc50 con 0x561ea6dbd820 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 10 client.16410 fetch_fsmap learned FSMap version 123 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.341+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=123,monmap=4+}) -- 0x561ea6e3f000 con 0x561ea6dbd820 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3068125932 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b9c013050 con 0x561ea6dbd820 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3068125932 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 123) ==== 7132+0+0 (secure 0 0 0) 0x7f6b980ce680 con 0x561ea6dbd820 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6bac882740 10 client.16410 fetch_fsmap finished waiting for FSMap version 123 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6bac882740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6bac882740 20 client.16410 populate_metadata read hostname 'smithi090' 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6bac882740 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6e3f790 0x561ea6e41bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6bac882740 4 client.16410 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ea6d2aee0 con 0x561ea6e3f790 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6e3f790 0x561ea6e41bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6b82ffd640 10 client.16410 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:20.348 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.342+0000 7f6ba37fe640 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6e3f790 0x561ea6e41bb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef7b9317761a74a server_cookie=31c8cb302405023d in_seq=0 out_seq=0 2024-04-26T18:24:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6b82ffd640 1 -- 172.21.15.90:0/3068125932 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ea6d2aee0 con 0x561ea6e3f790 2024-04-26T18:24:20.402 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6b82ffd640 10 client.16410 handle_command_reply: tid=0 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 client.16410 shutdown 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbd820 msgr2=0x561ea6dcc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbd820 0x561ea6dcc8c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6b980ce460 tx=0x7f6b9806c5f0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 shutdown_connections 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561ea6e3f790 0x561ea6e41bb0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561ea6dc80e0 0x561ea6e376f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6b82ffd640 0 client.16410 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561ea6dbd820 0x561ea6dcc8c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 --2- 172.21.15.90:0/3068125932 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561ea6dbcf00 0x561ea6dcc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 >> 172.21.15.90:0/3068125932 conn(0x561ea6db9410 msgr2=0x561ea6dc4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 shutdown_connections 2024-04-26T18:24:20.403 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 1 -- 172.21.15.90:0/3068125932 wait complete. 2024-04-26T18:24:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.402+0000 7f6bac882740 20 client.16410 trim_cache size 0 max 16384 2024-04-26T18:24:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:24:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:24:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:24:20.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6b8c0a3dc0 msgr2=0x7f6b8c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6b8c0a3dc0 0x7f6b8c0a6280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c022000 tx=0x7f6b9c020000 comp rx=0 tx=0).stop 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4102220 msgr2=0x7f6ba410f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4102220 0x7f6ba410f6b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6b9800e9f0 tx=0x7f6b9800edf0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 shutdown_connections 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f6b8c0a3dc0 0x7f6b8c0a6280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6ba4107820 0x7f6ba410fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6ba4102220 0x7f6ba410f6b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 --2- 172.21.15.90:0/2844598631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ba4101860 0x7f6ba410f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 >> 172.21.15.90:0/2844598631 conn(0x7f6ba40789b0 msgr2=0x7f6ba41044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 shutdown_connections 2024-04-26T18:24:20.406 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:20.403+0000 7f6ba9b6a640 1 -- 172.21.15.90:0/2844598631 wait complete. 2024-04-26T18:24:20.701 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.99 MB/sec execute 488 sec latency 7710.332 ms 2024-04-26T18:24:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:20 smithi119 ceph-mon[32752]: pgmap v407: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-04-26T18:24:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:20 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:20 smithi167 ceph-mon[32751]: pgmap v407: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-04-26T18:24:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:20 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:20 smithi090 ceph-mon[29180]: pgmap v407: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 1.7 MiB/s wr, 212 op/s 2024-04-26T18:24:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:20 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:21.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.99 MB/sec execute 489 sec latency 8710.444 ms 2024-04-26T18:24:22.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.98 MB/sec execute 490 sec latency 9710.561 ms 2024-04-26T18:24:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:22 smithi119 ceph-mon[32752]: pgmap v408: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-26T18:24:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:22 smithi167 ceph-mon[32751]: pgmap v408: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-26T18:24:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:22 smithi090 ceph-mon[29180]: pgmap v408: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 36 KiB/s wr, 13 op/s 2024-04-26T18:24:23.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.98 MB/sec execute 491 sec latency 10710.618 ms 2024-04-26T18:24:24.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.97 MB/sec execute 492 sec latency 11710.727 ms 2024-04-26T18:24:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:24 smithi119 ceph-mon[32752]: pgmap v409: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 240 KiB/s wr, 50 op/s 2024-04-26T18:24:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:24 smithi167 ceph-mon[32751]: pgmap v409: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 240 KiB/s wr, 50 op/s 2024-04-26T18:24:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:24 smithi090 ceph-mon[29180]: pgmap v409: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 240 KiB/s wr, 50 op/s 2024-04-26T18:24:25.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.96 MB/sec execute 493 sec latency 12710.775 ms 2024-04-26T18:24:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:25 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:25 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:25 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=c=up:active,2=k=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:26.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.96 MB/sec execute 494 sec latency 13710.882 ms 2024-04-26T18:24:26.753 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0c7e6bf700 time 2024-04-26T18:24:26.458305+0000 2024-04-26T18:24:26.753 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:26.753 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.753 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0c88ce83dd] 2024-04-26T18:24:26.753 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.753 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: clone() 2024-04-26T18:24:26.754 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.459+0000 7f0c7e6bf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0c7e6bf700 time 2024-04-26T18:24:26.458305+0000 2024-04-26T18:24:26.755 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:26.755 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0c88ce83dd] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.756 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: clone() 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: *** Caught signal (Aborted) ** 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: in thread 7f0c7e6bf700 thread_name:ms_dispatch 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c87a70d20] 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: gsignal() 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: abort() 2024-04-26T18:24:26.757 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0c88ce8437] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 16: clone() 2024-04-26T18:24:26.758 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.460+0000 7f0c7e6bf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: in thread 7f0c7e6bf700 thread_name:ms_dispatch 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c87a70d20] 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: gsignal() 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: abort() 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0c88ce8437] 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.759 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 16: clone() 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: -3> 2024-04-26T18:24:26.459+0000 7f0c7e6bf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0c7e6bf700 time 2024-04-26T18:24:26.458305+0000 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.760 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0c88ce83dd] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: clone() 2024-04-26T18:24:26.761 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 0> 2024-04-26T18:24:26.460+0000 7f0c7e6bf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: in thread 7f0c7e6bf700 thread_name:ms_dispatch 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c87a70d20] 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: gsignal() 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: abort() 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0c88ce8437] 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.762 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 16: clone() 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: -4> 2024-04-26T18:24:26.459+0000 7f0c7e6bf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0c7e6bf700 time 2024-04-26T18:24:26.458305+0000 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.763 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f0c88ce83dd] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: clone() 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: -1> 2024-04-26T18:24:26.460+0000 7f0c7e6bf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: in thread 7f0c7e6bf700 thread_name:ms_dispatch 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f0c87a70d20] 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2: gsignal() 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 3: abort() 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f0c88ce8437] 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f0c88ce85a3] 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d5e6eba822] 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d5e6e2a598] 2024-04-26T18:24:26.765 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d5e6e2cc17] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d5e6d25d0b] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d5e6d262ac] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d5e6d0efad] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f0c88f7ce20] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 13: (DispatchQueue::entry()+0x51f) [0x7f0c88f79fdf] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f0c8904cfc1] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f0c87a661ca] 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 16: clone() 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70276]: 2024-04-26T18:24:26.766 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:26 smithi167 podman[70640]: 2024-04-26 18:24:26.523690737 +0000 UTC m=+0.019754900 container died fc4ae0bcf576d0f730a12a609562a2e886d1f884c65aa9d76f8af3e8d50fe870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424) 2024-04-26T18:24:27.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:26 smithi119 ceph-mon[32752]: pgmap v410: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 192 KiB/s wr, 40 op/s 2024-04-26T18:24:27.067 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:26 smithi090 ceph-mon[29180]: pgmap v410: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 192 KiB/s wr, 40 op/s 2024-04-26T18:24:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:26 smithi167 ceph-mon[32751]: pgmap v410: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 192 KiB/s wr, 40 op/s 2024-04-26T18:24:27.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f79d5f56700 time 2024-04-26T18:24:27.031165+0000 2024-04-26T18:24:27.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:27.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f79e057f3dd] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: clone() 2024-04-26T18:24:27.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: *** Caught signal (Aborted) ** 2024-04-26T18:24:27.306 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: in thread 7f79d5f56700 thread_name:ms_dispatch 2024-04-26T18:24:27.306 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.032+0000 7f79d5f56700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f79d5f56700 time 2024-04-26T18:24:27.031165+0000 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f79e057f3dd] 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: clone() 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f79df307d20] 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: gsignal() 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: abort() 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f79e057f437] 2024-04-26T18:24:27.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 16: clone() 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.033+0000 7f79d5f56700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:27.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: in thread 7f79d5f56700 thread_name:ms_dispatch 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f79df307d20] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: gsignal() 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: abort() 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f79e057f437] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 16: clone() 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: -1> 2024-04-26T18:24:27.032+0000 7f79d5f56700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f79d5f56700 time 2024-04-26T18:24:27.031165+0000 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f79e057f3dd] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: clone() 2024-04-26T18:24:27.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 0> 2024-04-26T18:24:27.033+0000 7f79d5f56700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: in thread 7f79d5f56700 thread_name:ms_dispatch 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f79df307d20] 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: gsignal() 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: abort() 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f79e057f437] 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 16: clone() 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: -1> 2024-04-26T18:24:27.032+0000 7f79d5f56700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f79d5f56700 time 2024-04-26T18:24:27.031165+0000 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f79e057f3dd] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: clone() 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 0> 2024-04-26T18:24:27.033+0000 7f79d5f56700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: in thread 7f79d5f56700 thread_name:ms_dispatch 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f79df307d20] 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2: gsignal() 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 3: abort() 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f79e057f437] 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f79e057f5a3] 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5574c64a9822] 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5574c6419598] 2024-04-26T18:24:27.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5574c641bc17] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5574c6314d0b] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5574c63152ac] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5574c62fdfad] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f79e0813e20] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 13: (DispatchQueue::entry()+0x51f) [0x7f79e0810fdf] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f79e08e3fc1] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f79df2fd1ca] 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 16: clone() 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[62585]: 2024-04-26T18:24:27.317 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 podman[70370]: 2024-04-26 18:24:27.096411271 +0000 UTC m=+0.021566306 container died f193fc625c27ad74cd6ce3d07bcc119c4c28803a37d0a20f89b822aef20bc685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T18:24:27.547 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:27 smithi167 podman[70640]: 2024-04-26 18:24:27.29729335 +0000 UTC m=+0.793357529 container remove fc4ae0bcf576d0f730a12a609562a2e886d1f884c65aa9d76f8af3e8d50fe870 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:24:27.547 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:27 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:24:27.547 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:27 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-26T18:24:27.547 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:27 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 4.337s CPU time. 2024-04-26T18:24:27.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.95 MB/sec execute 495 sec latency 14710.916 ms 2024-04-26T18:24:28.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 podman[70370]: 2024-04-26 18:24:27.798576225 +0000 UTC m=+0.723731248 container remove f193fc625c27ad74cd6ce3d07bcc119c4c28803a37d0a20f89b822aef20bc685 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T18:24:28.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:27 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:24:28.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:28 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-26T18:24:28.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:28 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 9.169s CPU time. 2024-04-26T18:24:28.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.95 MB/sec execute 496 sec latency 15710.967 ms 2024-04-26T18:24:28.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:28 smithi090 ceph-mon[29180]: pgmap v411: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 189 KiB/s wr, 39 op/s 2024-04-26T18:24:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:28 smithi119 ceph-mon[32752]: pgmap v411: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 189 KiB/s wr, 39 op/s 2024-04-26T18:24:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:28 smithi167 ceph-mon[32751]: pgmap v411: 97 pgs: 97 active+clean; 155 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 189 KiB/s wr, 39 op/s 2024-04-26T18:24:29.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.94 MB/sec execute 497 sec latency 16711.002 ms 2024-04-26T18:24:30.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.93 MB/sec execute 498 sec latency 17711.096 ms 2024-04-26T18:24:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:30 smithi119 ceph-mon[32752]: pgmap v412: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 164 KiB/s wr, 36 op/s 2024-04-26T18:24:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:30 smithi167 ceph-mon[32751]: pgmap v412: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 164 KiB/s wr, 36 op/s 2024-04-26T18:24:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:30 smithi090 ceph-mon[29180]: pgmap v412: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 164 KiB/s wr, 36 op/s 2024-04-26T18:24:31.702 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.93 MB/sec execute 499 sec latency 18711.202 ms 2024-04-26T18:24:32.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.92 MB/sec execute 500 sec latency 19711.310 ms 2024-04-26T18:24:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:32 smithi119 ceph-mon[32752]: pgmap v413: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 143 KiB/s wr, 28 op/s 2024-04-26T18:24:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:32 smithi167 ceph-mon[32751]: pgmap v413: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 143 KiB/s wr, 28 op/s 2024-04-26T18:24:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:32 smithi090 ceph-mon[29180]: pgmap v413: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 143 KiB/s wr, 28 op/s 2024-04-26T18:24:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.498+0000 7f7b9aa62640 1 -- 172.21.15.90:0/573765070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b94110430 msgr2=0x7f7b94118000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.498+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/573765070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b94110430 0x7f7b94118000 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c009ca0 tx=0x7f7b8c02a8c0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.498+0000 7f7b9aa62640 1 -- 172.21.15.90:0/573765070 shutdown_connections 2024-04-26T18:24:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.498+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/573765070 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b94110430 0x7f7b94118000 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.498+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/573765070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b9410bd60 0x7f7b9410fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.498+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/573765070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b9410b3b0 0x7f7b9410b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.498+0000 7f7b9aa62640 1 -- 172.21.15.90:0/573765070 >> 172.21.15.90:0/573765070 conn(0x7f7b9406fa70 msgr2=0x7f7b94109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.499+0000 7f7b9aa62640 1 -- 172.21.15.90:0/573765070 shutdown_connections 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.499+0000 7f7b9aa62640 1 -- 172.21.15.90:0/573765070 wait complete. 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.499+0000 7f7b9aa62640 1 Processor -- start 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.499+0000 7f7b9aa62640 1 -- start start 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b9aa62640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b9410b3b0 0x7f7b941a7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b9aa62640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94110430 0x7f7b941a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b9aa62640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b941a1600 0x7f7b941a1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b9aa62640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9411b3a0 con 0x7f7b941a1600 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b9aa62640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9411b220 con 0x7f7b94110430 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b9aa62640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9411b520 con 0x7f7b9410b3b0 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b941a1600 0x7f7b941a1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b941a1600 0x7f7b941a1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42950/0 (socket says 172.21.15.90:42950) 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 -- 172.21.15.90:0/601734994 learned_addr learned my addr 172.21.15.90:0/601734994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b937fe640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94110430 0x7f7b941a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b93fff640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b9410b3b0 0x7f7b941a7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 -- 172.21.15.90:0/601734994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b9410b3b0 msgr2=0x7f7b941a7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b9410b3b0 0x7f7b941a7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 -- 172.21.15.90:0/601734994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94110430 msgr2=0x7f7b941a7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94110430 0x7f7b941a7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 -- 172.21.15.90:0/601734994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b941a2120 con 0x7f7b941a1600 2024-04-26T18:24:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b93fff640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b9410b3b0 0x7f7b941a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b98fff640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b941a1600 0x7f7b941a1ab0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005cc0 tx=0x7f7b8c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b917fa640 1 -- 172.21.15.90:0/601734994 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c003bf0 con 0x7f7b941a1600 2024-04-26T18:24:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b917fa640 1 -- 172.21.15.90:0/601734994 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b8c003d90 con 0x7f7b941a1600 2024-04-26T18:24:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b917fa640 1 -- 172.21.15.90:0/601734994 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c02fbc0 con 0x7f7b941a1600 2024-04-26T18:24:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.500+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b941a23b0 con 0x7f7b941a1600 2024-04-26T18:24:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.501+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b941ae5b0 con 0x7f7b941a1600 2024-04-26T18:24:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b917fa640 1 -- 172.21.15.90:0/601734994 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f7b8c02fd60 con 0x7f7b941a1600 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 Processor -- start 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 -- start start 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432efba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b917fa640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55844324b460 con 0x5584432e5820 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x558443205590 con 0x5584432f00e0 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.503+0000 7f7b9d776740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558443247420 con 0x5584432e4f00 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432efba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42954/0 (socket says 172.21.15.90:42954) 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 -- 172.21.15.90:0/2884886993 learned_addr learned my addr 172.21.15.90:0/2884886993 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b98fff640 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b917fa640 1 -- 172.21.15.90:0/601734994 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9745+0+0 (secure 0 0 0) 0x7f7b8c0f2260 con 0x7f7b941a1600 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 -- 172.21.15.90:0/2884886993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 msgr2=0x5584432e52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 -- 172.21.15.90:0/2884886993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 msgr2=0x5584432f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 -- 172.21.15.90:0/2884886993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584432120d0 con 0x5584432e5820 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b937fe640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.504+0000 7f7b93fff640 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432efba0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009c50 tx=0x7f7b8400e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f81dd96902b7b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b7a7fc640 1 -- 172.21.15.90:0/2884886993 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84016070 con 0x5584432e5820 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b7a7fc640 1 -- 172.21.15.90:0/2884886993 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b84011920 con 0x5584432e5820 2024-04-26T18:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b7a7fc640 1 -- 172.21.15.90:0/2884886993 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84010810 con 0x5584432e5820 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 -- 172.21.15.90:0/2884886993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 msgr2=0x5584432efba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432efba0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009c50 tx=0x7f7b8400e6c0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b937fe640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b9410a920 tx=0x7f7b8800b860 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 -- 172.21.15.90:0/2884886993 shutdown_connections 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432efba0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2884886993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 -- 172.21.15.90:0/2884886993 >> 172.21.15.90:0/2884886993 conn(0x5584432e1410 msgr2=0x5584432e17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.505+0000 7f7b9d776740 1 -- 172.21.15.90:0/2884886993 shutdown_connections 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 -- 172.21.15.90:0/2884886993 wait complete. 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 -- start start 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 Processor -- start 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432e4f00 0x5584432ecfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432ed520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432f00e0 0x5584432eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5584432f2d10 con 0x5584432e5820 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5584431d2b60 con 0x5584432e4f00 2024-04-26T18:24:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5584432f2e90 con 0x5584432f00e0 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b9d776740 10 client.0 _ll_register_callbacks cb 0x5584431f72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.506+0000 7f7b937fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432ed520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432ed520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42964/0 (socket says 172.21.15.90:42964) 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 -- 172.21.15.90:0/1663788094 learned_addr learned my addr 172.21.15.90:0/1663788094 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432e4f00 0x5584432ecfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b98fff640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432f00e0 0x5584432eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 -- 172.21.15.90:0/1663788094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432f00e0 msgr2=0x5584432eda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432f00e0 0x5584432eda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 -- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432e4f00 msgr2=0x5584432ecfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432e4f00 0x5584432ecfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 -- 172.21.15.90:0/1663788094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558442f5d210 con 0x5584432e5820 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432e4f00 0x5584432ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b937fe640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432ed520 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800bea0 tx=0x7f7b8800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf8e931b7480fdd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b737fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b737fe640 1 -- 172.21.15.90:0/1663788094 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88012e60 con 0x5584432e5820 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b737fe640 1 -- 172.21.15.90:0/1663788094 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9745+0+0 (secure 0 0 0) 0x7f7b88005220 con 0x5584432e5820 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b84011920 con 0x5584432e5820 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b737fe640 1 -- 172.21.15.90:0/1663788094 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f7b8802e050 con 0x5584432e5820 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b9d776740 10 client.16422 fetch_fsmap learned FSMap version 124 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.507+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=4+}) -- 0x558443029620 con 0x5584432e5820 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b737fe640 1 -- 172.21.15.90:0/1663788094 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b84011920 con 0x5584432e5820 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b737fe640 1 -- 172.21.15.90:0/1663788094 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 124) ==== 7136+0+0 (secure 0 0 0) 0x7f7b88019030 con 0x5584432e5820 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b9d776740 10 client.16422 fetch_fsmap finished waiting for FSMap version 124 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b9d776740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b9d776740 20 client.16422 populate_metadata read hostname 'smithi090' 2024-04-26T18:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5584433001e0 0x558443302650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b9d776740 4 client.16422 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558443259070 con 0x5584433001e0 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.508+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5584433001e0 0x558443302650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b737fe640 10 client.16422 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5584433001e0 0x558443302650 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a173e9ccfc404b server_cookie=615e3137d86b4ca9 in_seq=0 out_seq=0 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b737fe640 1 -- 172.21.15.90:0/1663788094 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558443259070 con 0x5584433001e0 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b737fe640 10 client.16422 handle_command_reply: tid=0 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 client.16422 shutdown 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 msgr2=0x5584432ed520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432ed520 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800bea0 tx=0x7f7b8800b390 comp rx=0 tx=0).stop 2024-04-26T18:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 shutdown_connections 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5584433001e0 0x558443302650 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432f00e0 0x5584432eda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432ed520 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b737fe640 0 client.16422 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1663788094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432e4f00 0x5584432ecfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.509+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 >> 172.21.15.90:0/1663788094 conn(0x5584432e1410 msgr2=0x5584432e05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.510+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 shutdown_connections 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.510+0000 7f7b9d776740 1 -- 172.21.15.90:0/1663788094 wait complete. 2024-04-26T18:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.510+0000 7f7b9d776740 20 client.16422 trim_cache size 0 max 16384 2024-04-26T18:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.524+0000 7f7b9d776740 1 Processor -- start 2024-04-26T18:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.524+0000 7f7b9d776740 1 -- start start 2024-04-26T18:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e4f00 0x5584432fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e5820 0x5584432f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b9d776740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941b8350 con 0x5584432e4f00 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b9d776740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941b8a20 con 0x5584432f00e0 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b9d776740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941b90f0 con 0x5584432e5820 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e4f00 0x5584432fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e4f00 0x5584432fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42966/0 (socket says 172.21.15.90:42966) 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 -- 172.21.15.90:0/2165224104 learned_addr learned my addr 172.21.15.90:0/2165224104 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b98fff640 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 -- 172.21.15.90:0/2165224104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e5820 msgr2=0x5584432f84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e5820 0x5584432f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 -- 172.21.15.90:0/2165224104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 msgr2=0x5584432e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 -- 172.21.15.90:0/2165224104 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584432e7d70 con 0x5584432e4f00 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b98fff640 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b93fff640 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e4f00 0x5584432fbbd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b84010440 tx=0x7f7b840103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a772cce0261b087 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b79ffb640 1 -- 172.21.15.90:0/2165224104 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b84016040 con 0x5584432e4f00 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b79ffb640 1 -- 172.21.15.90:0/2165224104 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b840054d0 con 0x5584432e4f00 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.525+0000 7f7b79ffb640 1 -- 172.21.15.90:0/2165224104 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b840292d0 con 0x5584432e4f00 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 -- 172.21.15.90:0/2165224104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e4f00 msgr2=0x5584432fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e4f00 0x5584432fbbd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b84010440 tx=0x7f7b840103d0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 -- 172.21.15.90:0/2165224104 shutdown_connections 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e5820 0x5584432f84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 --2- 172.21.15.90:0/2165224104 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e4f00 0x5584432fbbd0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.531 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 -- 172.21.15.90:0/2165224104 >> 172.21.15.90:0/2165224104 conn(0x5584432e1410 msgr2=0x5584432dfd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 -- 172.21.15.90:0/2165224104 shutdown_connections 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 -- 172.21.15.90:0/2165224104 wait complete. 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.526+0000 7f7b9d776740 1 -- start start 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 1 Processor -- start 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432e98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941b5650 con 0x5584432e5820 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941b6a40 con 0x5584432f00e0 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7b941b5ca0 con 0x5584432e4f00 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b9d776740 10 client.0 _ll_register_callbacks cb 0x7f7b88011ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432e98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432e98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42974/0 (socket says 172.21.15.90:42974) 2024-04-26T18:24:33.532 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 -- 172.21.15.90:0/1153088596 learned_addr learned my addr 172.21.15.90:0/1153088596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b98fff640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 -- 172.21.15.90:0/1153088596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 msgr2=0x5584432e93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 -- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 msgr2=0x5584432e9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 -- 172.21.15.90:0/1153088596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55844336a9e0 con 0x5584432e5820 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b937fe640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432e98f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7b8801c9c0 tx=0x7f7b88012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee1ec03c91508f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b737fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.527+0000 7f7b737fe640 1 -- 172.21.15.90:0/1153088596 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8801cc50 con 0x5584432e5820 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b737fe640 1 -- 172.21.15.90:0/1153088596 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9745+0+0 (secure 0 0 0) 0x7f7b88025620 con 0x5584432e5820 2024-04-26T18:24:33.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b840054d0 con 0x5584432e5820 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b737fe640 1 -- 172.21.15.90:0/1153088596 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=124) ==== 24+0+0 (secure 0 0 0) 0x7f7b88041050 con 0x5584432e5820 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 10 client.16428 fetch_fsmap learned FSMap version 124 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=124,monmap=4+}) -- 0x55844336bc60 con 0x5584432e5820 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b737fe640 1 -- 172.21.15.90:0/1153088596 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b840054d0 con 0x5584432e5820 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b737fe640 1 -- 172.21.15.90:0/1153088596 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 124) ==== 7136+0+0 (secure 0 0 0) 0x7f7b88019050 con 0x5584432e5820 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 10 client.16428 fetch_fsmap finished waiting for FSMap version 124 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 20 client.16428 populate_metadata read hostname 'smithi090' 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55844336c350 0x55844336e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 4 client.16428 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558443252ee0 con 0x55844336c350 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.528+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55844336c350 0x55844336e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:33.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.529+0000 7f7b737fe640 10 client.16428 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:33.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:33.529+0000 7f7b93fff640 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55844336c350 0x55844336e770 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b214f8d7b568f8a6 server_cookie=4bf30f39550e27a0 in_seq=0 out_seq=0 2024-04-26T18:24:33.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.92 MB/sec execute 501 sec latency 20711.429 ms 2024-04-26T18:24:34.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.91 MB/sec execute 502 sec latency 21711.517 ms 2024-04-26T18:24:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:34 smithi119 ceph-mon[32752]: pgmap v414: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 859 KiB/s wr, 49 op/s 2024-04-26T18:24:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:34 smithi167 ceph-mon[32751]: pgmap v414: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 859 KiB/s wr, 49 op/s 2024-04-26T18:24:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:34 smithi090 ceph-mon[29180]: pgmap v414: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 859 KiB/s wr, 49 op/s 2024-04-26T18:24:35.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.90 MB/sec execute 503 sec latency 22711.595 ms 2024-04-26T18:24:36.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.90 MB/sec execute 504 sec latency 23711.733 ms 2024-04-26T18:24:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:36 smithi119 ceph-mon[32752]: pgmap v415: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 720 KiB/s wr, 23 op/s 2024-04-26T18:24:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:36 smithi167 ceph-mon[32751]: pgmap v415: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 720 KiB/s wr, 23 op/s 2024-04-26T18:24:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:36 smithi090 ceph-mon[29180]: pgmap v415: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 720 KiB/s wr, 23 op/s 2024-04-26T18:24:37.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.89 MB/sec execute 505 sec latency 24711.856 ms 2024-04-26T18:24:37.822 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:37 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:24:37.822 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:37 smithi167 systemd[1]: Stopped Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:37.822 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:37 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 4.337s CPU time. 2024-04-26T18:24:37.822 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:37 smithi167 systemd[1]: Starting Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:24:37.822 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:37 smithi167 podman[70737]: 2024-04-26 18:24:37.707325651 +0000 UTC m=+0.018801888 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:38.158 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:37 smithi167 podman[70737]: 2024-04-26 18:24:37.822636948 +0000 UTC m=+0.134113174 container create b2de4e490d10d5b20104e30af65e71c21767538e2698707ebcf89935674b31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True) 2024-04-26T18:24:38.158 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:38 smithi167 podman[70737]: 2024-04-26 18:24:38.041585995 +0000 UTC m=+0.353062221 container init b2de4e490d10d5b20104e30af65e71c21767538e2698707ebcf89935674b31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T18:24:38.159 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:38 smithi167 podman[70737]: 2024-04-26 18:24:38.044169247 +0000 UTC m=+0.355645491 container start b2de4e490d10d5b20104e30af65e71c21767538e2698707ebcf89935674b31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T18:24:38.159 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mds[70756]: -- 172.21.15.167:0/127548346 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e382ec1a40 con 0x55e382e94800 2024-04-26T18:24:38.159 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: starting mds.c at 2024-04-26T18:24:38.331 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:24:38.332 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 systemd[1]: Stopped Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:38.332 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 9.169s CPU time. 2024-04-26T18:24:38.332 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 systemd[1]: Starting Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:24:38.332 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 podman[70463]: 2024-04-26 18:24:38.198806897 +0000 UTC m=+0.019475287 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:38.332 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 podman[70463]: 2024-04-26 18:24:38.332489931 +0000 UTC m=+0.153158311 container create 921ec7fa11113251a6046bc42bc34c560d94e247939c1e09842d4e0bf667b02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T18:24:38.422 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:38 smithi167 bash[70737]: b2de4e490d10d5b20104e30af65e71c21767538e2698707ebcf89935674b31bb 2024-04-26T18:24:38.422 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:24:38 smithi167 systemd[1]: Started Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: pgmap v416: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 720 KiB/s wr, 23 op/s 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: daemon mds.c restarted 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: osdmap e123: 12 total, 12 up, 12 in 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:boot 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:24:38.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=k=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:38.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.89 MB/sec execute 506 sec latency 25711.950 ms 2024-04-26T18:24:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: pgmap v416: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 720 KiB/s wr, 23 op/s 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: daemon mds.c restarted 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: osdmap e123: 12 total, 12 up, 12 in 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:boot 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:38 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=k=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: pgmap v416: 97 pgs: 97 active+clean; 158 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 720 KiB/s wr, 23 op/s 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: daemon mds.c restarted 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: osdmap e123: 12 total, 12 up, 12 in 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:boot 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:38 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=k=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:24:39.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 podman[70463]: 2024-04-26 18:24:38.5938321 +0000 UTC m=+0.414500480 container init 921ec7fa11113251a6046bc42bc34c560d94e247939c1e09842d4e0bf667b02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T18:24:39.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 podman[70463]: 2024-04-26 18:24:38.596610883 +0000 UTC m=+0.417279265 container start 921ec7fa11113251a6046bc42bc34c560d94e247939c1e09842d4e0bf667b02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:24:39.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-mds[70482]: -- 172.21.15.119:0/1971940770 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56539b5b3a40 con 0x56539b586800 2024-04-26T18:24:39.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: starting mds.k at 2024-04-26T18:24:39.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 bash[70463]: 921ec7fa11113251a6046bc42bc34c560d94e247939c1e09842d4e0bf667b02e 2024-04-26T18:24:39.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:24:38 smithi119 systemd[1]: Started Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:39.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.88 MB/sec execute 507 sec latency 26712.084 ms 2024-04-26T18:24:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:39 smithi167 ceph-mon[32751]: daemon mds.k restarted 2024-04-26T18:24:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:39 smithi167 ceph-mon[32751]: osdmap e124: 12 total, 12 up, 12 in 2024-04-26T18:24:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:boot 2024-04-26T18:24:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:39 smithi167 ceph-mon[32751]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:24:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:39 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:replay} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:24:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:24:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:39 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:39 smithi090 ceph-mon[29180]: daemon mds.k restarted 2024-04-26T18:24:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:39 smithi090 ceph-mon[29180]: osdmap e124: 12 total, 12 up, 12 in 2024-04-26T18:24:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:boot 2024-04-26T18:24:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:39 smithi090 ceph-mon[29180]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:24:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:39 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:replay} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:24:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:24:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:39 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:39 smithi119 ceph-mon[32752]: daemon mds.k restarted 2024-04-26T18:24:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:39 smithi119 ceph-mon[32752]: osdmap e124: 12 total, 12 up, 12 in 2024-04-26T18:24:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:boot 2024-04-26T18:24:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:39 smithi119 ceph-mon[32752]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:24:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:39 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:replay} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:24:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:24:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:39 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:40.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.88 MB/sec execute 508 sec latency 27712.146 ms 2024-04-26T18:24:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:40 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:resolve 2024-04-26T18:24:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:40 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:resolve 2024-04-26T18:24:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:40 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:40 smithi119 ceph-mon[32752]: pgmap v419: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.1 MiB/s wr, 42 op/s 2024-04-26T18:24:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:40 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:resolve 2024-04-26T18:24:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:40 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:resolve 2024-04-26T18:24:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:40 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:40 smithi167 ceph-mon[32751]: pgmap v419: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.1 MiB/s wr, 42 op/s 2024-04-26T18:24:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:40 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:resolve 2024-04-26T18:24:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:40 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:resolve 2024-04-26T18:24:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:40 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:40 smithi090 ceph-mon[29180]: pgmap v419: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 1.1 MiB/s wr, 42 op/s 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b737fe640 1 -- 172.21.15.90:0/1153088596 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558443252ee0 con 0x55844336c350 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b737fe640 10 client.16428 handle_command_reply: tid=0 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 client.16428 shutdown 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 msgr2=0x5584432e98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432e98f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7b8801c9c0 tx=0x7f7b88012f40 comp rx=0 tx=0).stop 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 shutdown_connections 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55844336c350 0x55844336e770 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5584432f00e0 0x5584432e9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.670 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5584432e5820 0x5584432e98f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 --2- 172.21.15.90:0/1153088596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584432e4f00 0x5584432e93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b737fe640 0 client.16428 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 >> 172.21.15.90:0/1153088596 conn(0x5584432e1410 msgr2=0x5584432e05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 shutdown_connections 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 1 -- 172.21.15.90:0/1153088596 wait complete. 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.669+0000 7f7b9d776740 20 client.16428 trim_cache size 0 max 16384 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:24:41.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:24:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7b7c0a3e40 msgr2=0x7f7b7c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7b9410a920 tx=0x7f7b8800b860 comp rx=0 tx=0).stop 2024-04-26T18:24:41.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b941a1600 msgr2=0x7f7b941a1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b941a1600 0x7f7b941a1ab0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c005cc0 tx=0x7f7b8c02f380 comp rx=0 tx=0).stop 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 shutdown_connections 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7b7c0a3e40 0x7f7b7c0a6300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b941a1600 0x7f7b941a1ab0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b94110430 0x7f7b941a7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 --2- 172.21.15.90:0/601734994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7b9410b3b0 0x7f7b941a7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 >> 172.21.15.90:0/601734994 conn(0x7f7b9406fa70 msgr2=0x7f7b9406ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 shutdown_connections 2024-04-26T18:24:41.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:41.670+0000 7f7b9aa62640 1 -- 172.21.15.90:0/601734994 wait complete. 2024-04-26T18:24:41.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 500888 2.87 MB/sec execute 509 sec latency 28712.258 ms 2024-04-26T18:24:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:41 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:24:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:41 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:reconnect 2024-04-26T18:24:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:41 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:reconnect 2024-04-26T18:24:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:41 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:41 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.0119997 2024-04-26T18:24:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:41 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0119998 2024-04-26T18:24:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:41 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:24:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:41 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:reconnect 2024-04-26T18:24:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:41 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:reconnect 2024-04-26T18:24:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:41 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:41 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.0119997 2024-04-26T18:24:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:41 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0119998 2024-04-26T18:24:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:41 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199995 2024-04-26T18:24:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:41 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:reconnect 2024-04-26T18:24:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:41 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:reconnect 2024-04-26T18:24:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:41 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:41 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.0119997 2024-04-26T18:24:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:41 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0119998 2024-04-26T18:24:42.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 501022 2.86 MB/sec execute 510 sec latency 29589.306 ms 2024-04-26T18:24:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:rejoin 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:rejoin 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: pgmap v420: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:active 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:active 2024-04-26T18:24:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:42 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:rejoin 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:rejoin 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: pgmap v420: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:24:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:active 2024-04-26T18:24:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:active 2024-04-26T18:24:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:42 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:43.067 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:24:42.575+0000 7f61aaa9b700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 1 2024-04-26T18:24:43.067 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:24:42.575+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 3 2024-04-26T18:24:43.068 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:24:42.577+0000 7f61a8a97700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 6 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:rejoin 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:rejoin 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: pgmap v420: 97 pgs: 97 active+clean; 159 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 KiB/s rd, 17 KiB/s wr, 11 op/s 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/958836808,v1:172.21.15.119:6835/958836808] up:active 2024-04-26T18:24:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/3757608812,v1:172.21.15.167:6833/3757608812] up:active 2024-04-26T18:24:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:24:43.069 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 26 18:24:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-26T18:24:42.575+0000 7f61d40ac700 -1 bdev(0x557f5274a000 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 1 2024-04-26T18:24:43.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:24:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:24:43.230+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:24:43.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:24:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:24:43.230+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:24:43.703 INFO:tasks.workunit.client.0.smithi090.stdout: 1 502111 2.87 MB/sec execute 511 sec latency 20.146 ms 2024-04-26T18:24:44.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 504720 2.89 MB/sec execute 512 sec latency 14.151 ms 2024-04-26T18:24:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:44 smithi119 ceph-mon[32752]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 178 KiB/s wr, 59 op/s 2024-04-26T18:24:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:44 smithi167 ceph-mon[32751]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 178 KiB/s wr, 59 op/s 2024-04-26T18:24:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:44 smithi090 ceph-mon[29180]: pgmap v421: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 178 KiB/s wr, 59 op/s 2024-04-26T18:24:45.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 506049 2.90 MB/sec execute 513 sec latency 13.710 ms 2024-04-26T18:24:46.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 508397 2.92 MB/sec execute 514 sec latency 13.604 ms 2024-04-26T18:24:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:46 smithi119 ceph-mon[32752]: pgmap v422: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 178 KiB/s wr, 59 op/s 2024-04-26T18:24:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:46 smithi167 ceph-mon[32751]: pgmap v422: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 178 KiB/s wr, 59 op/s 2024-04-26T18:24:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:46 smithi090 ceph-mon[29180]: pgmap v422: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 178 KiB/s wr, 59 op/s 2024-04-26T18:24:47.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.92 MB/sec execute 515 sec latency 123.465 ms 2024-04-26T18:24:48.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faaa706b700 time 2024-04-26T18:24:47.587528+0000 2024-04-26T18:24:48.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:48.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faab16943dd] 2024-04-26T18:24:48.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: clone() 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:47.587+0000 7faaa706b700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faaa706b700 time 2024-04-26T18:24:47.587528+0000 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:48.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faab16943dd] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: clone() 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: *** Caught signal (Aborted) ** 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: in thread 7faaa706b700 thread_name:ms_dispatch 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faab041cd20] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: gsignal() 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: abort() 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faab1694437] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 16: clone() 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:47.588+0000 7faaa706b700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: in thread 7faaa706b700 thread_name:ms_dispatch 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faab041cd20] 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: gsignal() 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: abort() 2024-04-26T18:24:48.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faab1694437] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 16: clone() 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:48.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: -1> 2024-04-26T18:24:47.587+0000 7faaa706b700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faaa706b700 time 2024-04-26T18:24:47.587528+0000 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faab16943dd] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: clone() 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 0> 2024-04-26T18:24:47.588+0000 7faaa706b700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: in thread 7faaa706b700 thread_name:ms_dispatch 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faab041cd20] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: gsignal() 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: abort() 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faab1694437] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 16: clone() 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: -1> 2024-04-26T18:24:47.587+0000 7faaa706b700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faaa706b700 time 2024-04-26T18:24:47.587528+0000 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7faab16943dd] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: clone() 2024-04-26T18:24:48.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 0> 2024-04-26T18:24:47.588+0000 7faaa706b700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: in thread 7faaa706b700 thread_name:ms_dispatch 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faab041cd20] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2: gsignal() 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 3: abort() 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7faab1694437] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7faab16945a3] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558c2b10b822] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558c2b07b598] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558c2b07dc17] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558c2af76d0b] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558c2af772ac] 2024-04-26T18:24:48.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558c2af5ffad] 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7faab1928e20] 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 13: (DispatchQueue::entry()+0x51f) [0x7faab1925fdf] 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7faab19f8fc1] 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faab04121ca] 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 16: clone() 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70450]: 2024-04-26T18:24:48.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:47 smithi167 podman[70809]: 2024-04-26 18:24:47.646156291 +0000 UTC m=+0.018598004 container died a2ed6181c689a60f85f4c71d52435acff6c0ff0623fece2ab6a27a7059833a1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:24:48.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.92 MB/sec execute 516 sec latency 1123.557 ms 2024-04-26T18:24:48.764 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:48 smithi167 podman[70809]: 2024-04-26 18:24:48.421770444 +0000 UTC m=+0.794212164 container remove a2ed6181c689a60f85f4c71d52435acff6c0ff0623fece2ab6a27a7059833a1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T18:24:48.764 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:48 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:24:48.764 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:48 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-04-26T18:24:48.764 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:48 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 8.954s CPU time. 2024-04-26T18:24:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:48 smithi119 ceph-mon[32752]: pgmap v423: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 148 KiB/s wr, 49 op/s 2024-04-26T18:24:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:48 smithi167 ceph-mon[32751]: pgmap v423: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 148 KiB/s wr, 49 op/s 2024-04-26T18:24:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:48 smithi090 ceph-mon[29180]: pgmap v423: 97 pgs: 97 active+clean; 160 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 148 KiB/s wr, 49 op/s 2024-04-26T18:24:49.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.91 MB/sec execute 517 sec latency 2123.689 ms 2024-04-26T18:24:50.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.91 MB/sec execute 518 sec latency 3123.799 ms 2024-04-26T18:24:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:50 smithi119 ceph-mon[32752]: pgmap v424: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.0 MiB/s wr, 142 op/s 2024-04-26T18:24:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:50 smithi167 ceph-mon[32751]: pgmap v424: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.0 MiB/s wr, 142 op/s 2024-04-26T18:24:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:50 smithi090 ceph-mon[29180]: pgmap v424: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.0 MiB/s wr, 142 op/s 2024-04-26T18:24:51.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.90 MB/sec execute 519 sec latency 4123.905 ms 2024-04-26T18:24:52.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.90 MB/sec execute 520 sec latency 5124.027 ms 2024-04-26T18:24:52.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:52 smithi090 ceph-mon[29180]: pgmap v425: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 949 KiB/s wr, 131 op/s 2024-04-26T18:24:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:52 smithi119 ceph-mon[32752]: pgmap v425: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 949 KiB/s wr, 131 op/s 2024-04-26T18:24:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:52 smithi167 ceph-mon[32751]: pgmap v425: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 949 KiB/s wr, 131 op/s 2024-04-26T18:24:53.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.89 MB/sec execute 521 sec latency 6124.154 ms 2024-04-26T18:24:54.704 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.89 MB/sec execute 522 sec latency 7124.207 ms 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 -- 172.21.15.90:0/3078593848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 msgr2=0x7f8c8410f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/3078593848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8410f3a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009ca0 tx=0x7f8c7c02a9e0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 -- 172.21.15.90:0/3078593848 shutdown_connections 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/3078593848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8410f3a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/3078593848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c841044e0 0x7f8c8410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/3078593848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c84103bc0 0x7f8c84103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 -- 172.21.15.90:0/3078593848 >> 172.21.15.90:0/3078593848 conn(0x7f8c840789b0 msgr2=0x7f8c84078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 -- 172.21.15.90:0/3078593848 shutdown_connections 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.766+0000 7f8c8be7e640 1 -- 172.21.15.90:0/3078593848 wait complete. 2024-04-26T18:24:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 Processor -- start 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 -- start start 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c84103bc0 0x7f8c84198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c841044e0 0x7f8c841994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8419d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112bb0 con 0x7f8c8410cf00 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112a30 con 0x7f8c84103bc0 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8be7e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c84112d30 con 0x7f8c841044e0 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8419d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8419d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57696/0 (socket says 172.21.15.90:57696) 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 -- 172.21.15.90:0/2440507566 learned_addr learned my addr 172.21.15.90:0/2440507566 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c84103bc0 0x7f8c84198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 -- 172.21.15.90:0/2440507566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c841044e0 msgr2=0x7f8c841994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c841044e0 0x7f8c841994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 -- 172.21.15.90:0/2440507566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c84103bc0 msgr2=0x7f8c84198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c84103bc0 0x7f8c84198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 -- 172.21.15.90:0/2440507566 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8419df50 con 0x7f8c8410cf00 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.767+0000 7f8c8a41b640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8419d820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009dc0 tx=0x7f8c7c02e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.768+0000 7f8c7affd640 1 -- 172.21.15.90:0/2440507566 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c033040 con 0x7f8c8410cf00 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.768+0000 7f8c7affd640 1 -- 172.21.15.90:0/2440507566 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7c038b00 con 0x7f8c8410cf00 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.768+0000 7f8c7affd640 1 -- 172.21.15.90:0/2440507566 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c043900 con 0x7f8c8410cf00 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.768+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8419e1e0 con 0x7f8c8410cf00 2024-04-26T18:24:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.768+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c841a5ad0 con 0x7f8c8410cf00 2024-04-26T18:24:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.770+0000 7f8c7affd640 1 -- 172.21.15.90:0/2440507566 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f8c7c036070 con 0x7f8c8410cf00 2024-04-26T18:24:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.770+0000 7f8c8eb98740 1 Processor -- start 2024-04-26T18:24:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8eb98740 1 -- start start 2024-04-26T18:24:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x557725394fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 0x5577253978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c7affd640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8c700a3dc0 0x7f8c700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5577252ec460 con 0x557725385e00 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5577252a6590 con 0x5577253954f0 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577252e8420 con 0x557725386720 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c8a41b640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8c700a3dc0 0x7f8c700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253861e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57698/0 (socket says 172.21.15.90:57698) 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 -- 172.21.15.90:0/26034442 learned_addr learned my addr 172.21.15.90:0/26034442 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x557725394fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 0x5577253978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c7affd640 1 -- 172.21.15.90:0/2440507566 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 9873+0+0 (secure 0 0 0) 0x7f8c7c0f1de0 con 0x7f8c8410cf00 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 -- 172.21.15.90:0/26034442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 msgr2=0x557725394fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x557725394fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 -- 172.21.15.90:0/26034442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 msgr2=0x5577253978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 0x5577253978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89419640 1 -- 172.21.15.90:0/26034442 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577252b30d0 con 0x557725385e00 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.771+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x557725394fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.772+0000 7f8c8a41b640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8c700a3dc0 0x7f8c700a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0b9c90 tx=0x7f8c7c0b93b0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.772+0000 7f8c89419640 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253861e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8c74013790 tx=0x7f8c7400a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530a6d4356ee7bca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c6ffff640 1 -- 172.21.15.90:0/26034442 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74004220 con 0x557725385e00 2024-04-26T18:24:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c6ffff640 1 -- 172.21.15.90:0/26034442 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7400e760 con 0x557725385e00 2024-04-26T18:24:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c6ffff640 1 -- 172.21.15.90:0/26034442 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74004c30 con 0x557725385e00 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c8eb98740 1 -- 172.21.15.90:0/26034442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 msgr2=0x5577253861e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253861e0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8c74013790 tx=0x7f8c7400a250 comp rx=0 tx=0).stop 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c8eb98740 1 -- 172.21.15.90:0/26034442 shutdown_connections 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 0x5577253978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x557725394fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.775+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/26034442 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253861e0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 -- 172.21.15.90:0/26034442 >> 172.21.15.90:0/26034442 conn(0x557725382410 msgr2=0x5577253827f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 -- 172.21.15.90:0/26034442 shutdown_connections 2024-04-26T18:24:54.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 -- 172.21.15.90:0/26034442 wait complete. 2024-04-26T18:24:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 -- start start 2024-04-26T18:24:54.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 Processor -- start 2024-04-26T18:24:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725385e00 0x557725392390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725386720 0x5577253928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577253954f0 0x557725392e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557725398120 con 0x557725386720 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557725273b60 con 0x557725385e00 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.776+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577253982a0 con 0x5577253954f0 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c8eb98740 10 client.0 _ll_register_callbacks cb 0x5577252982b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725386720 0x5577253928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725386720 0x5577253928d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57706/0 (socket says 172.21.15.90:57706) 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 -- 172.21.15.90:0/2914357350 learned_addr learned my addr 172.21.15.90:0/2914357350 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89419640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725385e00 0x557725392390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89419640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577253954f0 0x557725392e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 -- 172.21.15.90:0/2914357350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577253954f0 msgr2=0x557725392e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577253954f0 0x557725392e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 -- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725385e00 msgr2=0x557725392390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725385e00 0x557725392390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 -- 172.21.15.90:0/2914357350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557724ffe210 con 0x557725386720 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89419640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725385e00 0x557725392390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725386720 0x5577253928d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8c80010a50 tx=0x7f8c80010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b91096cefc14780 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2914357350 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8001e060 con 0x557725386720 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c7400e760 con 0x557725386720 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2914357350 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9873+0+0 (secure 0 0 0) 0x7f8c8000fb70 con 0x557725386720 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2914357350 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) ==== 24+0+0 (secure 0 0 0) 0x7f8c80025050 con 0x557725386720 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c8eb98740 10 client.16446 fetch_fsmap learned FSMap version 132 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.777+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) -- 0x5577250ca620 con 0x557725386720 2024-04-26T18:24:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2914357350 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7400e760 con 0x557725386720 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2914357350 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 132) ==== 7136+0+0 (secure 0 0 0) 0x7f8c80018370 con 0x557725386720 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c8eb98740 10 client.16446 fetch_fsmap finished waiting for FSMap version 132 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c8eb98740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c8eb98740 20 client.16446 populate_metadata read hostname 'smithi090' 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5577253a5760 0x5577253a7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c8eb98740 4 client.16446 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577252fa070 con 0x5577253a5760 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c89419640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5577253a5760 0x5577253a7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c6d7fa640 10 client.16446 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.778+0000 7f8c89419640 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5577253a5760 0x5577253a7b70 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10aecce845628779 server_cookie=b31ba4446b0f4e36 in_seq=0 out_seq=0 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2914357350 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577252fa070 con 0x5577253a5760 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c6d7fa640 10 client.16446 handle_command_reply: tid=0 2024-04-26T18:24:54.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 client.16446 shutdown 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725386720 msgr2=0x5577253928d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725386720 0x5577253928d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8c80010a50 tx=0x7f8c80010f10 comp rx=0 tx=0).stop 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 shutdown_connections 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5577253a5760 0x5577253a7b70 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577253954f0 0x557725392e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725386720 0x5577253928d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2914357350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725385e00 0x557725392390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c6d7fa640 0 client.16446 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 >> 172.21.15.90:0/2914357350 conn(0x557725382410 msgr2=0x557725388280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 shutdown_connections 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.779+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2914357350 wait complete. 2024-04-26T18:24:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.780+0000 7f8c8eb98740 20 client.16446 trim_cache size 0 max 16384 2024-04-26T18:24:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 Processor -- start 2024-04-26T18:24:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 -- start start 2024-04-26T18:24:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x5577253914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 0x5577253ad480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ad950 con 0x557725385e00 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ae020 con 0x5577253954f0 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.794+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ae6f0 con 0x557725386720 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57720/0 (socket says 172.21.15.90:57720) 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 -- 172.21.15.90:0/1273562252 learned_addr learned my addr 172.21.15.90:0/1273562252 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 -- 172.21.15.90:0/1273562252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 msgr2=0x5577253914f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x5577253914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x5577253914f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 -- 172.21.15.90:0/1273562252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 msgr2=0x5577253ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 0x5577253ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 -- 172.21.15.90:0/1273562252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557725386550 con 0x557725385e00 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x5577253914f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c89419640 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253a1050 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c74004910 tx=0x7f8c7400e630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702ba18cc0499510 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c6f7fe640 1 -- 172.21.15.90:0/1273562252 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74027040 con 0x557725385e00 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c6f7fe640 1 -- 172.21.15.90:0/1273562252 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c74022070 con 0x557725385e00 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.795+0000 7f8c6f7fe640 1 -- 172.21.15.90:0/1273562252 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74010de0 con 0x557725385e00 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 -- 172.21.15.90:0/1273562252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 msgr2=0x5577253a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253a1050 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c74004910 tx=0x7f8c7400e630 comp rx=0 tx=0).stop 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 -- 172.21.15.90:0/1273562252 shutdown_connections 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5577253954f0 0x5577253ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725386720 0x5577253914f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/1273562252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557725385e00 0x5577253a1050 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 -- 172.21.15.90:0/1273562252 >> 172.21.15.90:0/1273562252 conn(0x557725382410 msgr2=0x5577253827f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 -- 172.21.15.90:0/1273562252 shutdown_connections 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 -- 172.21.15.90:0/1273562252 wait complete. 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 -- start start 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.796+0000 7f8c8eb98740 1 Processor -- start 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725385e00 0x55772539d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725386720 0x55772539daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5577253954f0 0x55772538ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841aac10 con 0x5577253954f0 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841abfe0 con 0x557725386720 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8c841ab260 con 0x557725385e00 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 10 client.0 _ll_register_callbacks cb 0x7f8c8000a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5577253954f0 0x55772538ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5577253954f0 0x55772538ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57724/0 (socket says 172.21.15.90:57724) 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 -- 172.21.15.90:0/2629078698 learned_addr learned my addr 172.21.15.90:0/2629078698 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725386720 0x55772539daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725385e00 0x55772539d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 -- 172.21.15.90:0/2629078698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725385e00 msgr2=0x55772539d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725385e00 0x55772539d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 -- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725386720 msgr2=0x55772539daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725386720 0x55772539daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 -- 172.21.15.90:0/2629078698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55772538f310 con 0x5577253954f0 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725386720 0x55772539daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c89419640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5577253954f0 0x55772538ea90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8c74013760 tx=0x7f8c7401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd96f831cea8c83d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2629078698 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74010c60 con 0x5577253954f0 2024-04-26T18:24:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c74022070 con 0x5577253954f0 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2629078698 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(124..124 src has 1..124) ==== 9873+0+0 (secure 0 0 0) 0x7f8c7402d3f0 con 0x5577253954f0 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2629078698 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=132) ==== 24+0+0 (secure 0 0 0) 0x7f8c7400e7e0 con 0x5577253954f0 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 10 client.16452 fetch_fsmap learned FSMap version 132 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.797+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=132,monmap=4+}) -- 0x55772540b7d0 con 0x5577253954f0 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2629078698 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c74022070 con 0x5577253954f0 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2629078698 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 132) ==== 7136+0+0 (secure 0 0 0) 0x7f8c7400f040 con 0x5577253954f0 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c8eb98740 10 client.16452 fetch_fsmap finished waiting for FSMap version 132 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c8eb98740 10 client.16452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c8eb98740 20 client.16452 populate_metadata read hostname 'smithi090' 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55772540be30 0x55772540e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c8eb98740 4 client.16452 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577252f3ee0 con 0x55772540be30 2024-04-26T18:24:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.798+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55772540be30 0x55772540e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.799+0000 7f8c89c1a640 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55772540be30 0x55772540e250 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c5a80cb0213879a server_cookie=4bf8360af96894b4 in_seq=0 out_seq=0 2024-04-26T18:24:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:24:54.799+0000 7f8c6d7fa640 10 client.16452 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:24:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:54 smithi119 ceph-mon[32752]: pgmap v426: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 953 KiB/s wr, 135 op/s 2024-04-26T18:24:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:54 smithi167 ceph-mon[32751]: pgmap v426: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 953 KiB/s wr, 135 op/s 2024-04-26T18:24:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:54 smithi090 ceph-mon[29180]: pgmap v426: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 953 KiB/s wr, 135 op/s 2024-04-26T18:24:55.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.88 MB/sec execute 523 sec latency 8124.318 ms 2024-04-26T18:24:56.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.87 MB/sec execute 524 sec latency 9124.425 ms 2024-04-26T18:24:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:56 smithi119 ceph-mon[32752]: pgmap v427: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 845 KiB/s wr, 103 op/s 2024-04-26T18:24:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:56 smithi167 ceph-mon[32751]: pgmap v427: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 845 KiB/s wr, 103 op/s 2024-04-26T18:24:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:56 smithi090 ceph-mon[29180]: pgmap v427: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 845 KiB/s wr, 103 op/s 2024-04-26T18:24:57.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.87 MB/sec execute 525 sec latency 10124.472 ms 2024-04-26T18:24:58.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.86 MB/sec execute 526 sec latency 11124.576 ms 2024-04-26T18:24:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:58 smithi119 ceph-mon[32752]: pgmap v428: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 845 KiB/s wr, 103 op/s 2024-04-26T18:24:59.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:58 smithi167 ceph-mon[32751]: pgmap v428: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 845 KiB/s wr, 103 op/s 2024-04-26T18:24:59.057 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:58 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:24:59.057 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:58 smithi167 systemd[1]: Stopped Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:24:59.057 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:58 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 8.954s CPU time. 2024-04-26T18:24:59.057 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:58 smithi167 systemd[1]: Starting Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:24:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:58 smithi090 ceph-mon[29180]: pgmap v428: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 845 KiB/s wr, 103 op/s 2024-04-26T18:24:59.317 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 podman[70910]: 2024-04-26 18:24:58.960530195 +0000 UTC m=+0.018100790 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:59.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 podman[70910]: 2024-04-26 18:24:59.08097129 +0000 UTC m=+0.138541928 container create b63256b929ff5d6180c98ad07605ad9de7e9e388376103a88b404f28e44f6939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T18:24:59.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.86 MB/sec execute 527 sec latency 12124.632 ms 2024-04-26T18:24:59.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 podman[70910]: 2024-04-26 18:24:59.338765993 +0000 UTC m=+0.396336580 container init b63256b929ff5d6180c98ad07605ad9de7e9e388376103a88b404f28e44f6939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:24:59.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 podman[70910]: 2024-04-26 18:24:59.341419404 +0000 UTC m=+0.398989988 container start b63256b929ff5d6180c98ad07605ad9de7e9e388376103a88b404f28e44f6939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T18:24:59.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mds[70930]: -- 172.21.15.167:0/3236646460 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555913f09a40 con 0x555913edcc00 2024-04-26T18:24:59.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: starting mds.l at 2024-04-26T18:24:59.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 bash[70910]: b63256b929ff5d6180c98ad07605ad9de7e9e388376103a88b404f28e44f6939 2024-04-26T18:24:59.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:24:59 smithi167 systemd[1]: Started Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: daemon mds.l restarted 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: osdmap e125: 12 total, 12 up, 12 in 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:boot 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:25:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:24:59 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: daemon mds.l restarted 2024-04-26T18:25:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: osdmap e125: 12 total, 12 up, 12 in 2024-04-26T18:25:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:boot 2024-04-26T18:25:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:25:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:25:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:24:59 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: daemon mds.l restarted 2024-04-26T18:25:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: osdmap e125: 12 total, 12 up, 12 in 2024-04-26T18:25:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:boot 2024-04-26T18:25:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:25:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:25:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:24:59 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:00.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.85 MB/sec execute 528 sec latency 13124.721 ms 2024-04-26T18:25:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:00 smithi119 ceph-mon[32752]: pgmap v430: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:25:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:00 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:resolve 2024-04-26T18:25:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:00 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:00 smithi167 ceph-mon[32751]: pgmap v430: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:25:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:00 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:resolve 2024-04-26T18:25:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:00 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:00 smithi090 ceph-mon[29180]: pgmap v430: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:25:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:00 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:resolve 2024-04-26T18:25:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:00 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:01.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 509683 2.85 MB/sec execute 529 sec latency 14124.844 ms 2024-04-26T18:25:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.420+0000 7f8c6d7fa640 1 -- 172.21.15.90:0/2629078698 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577252f3ee0 con 0x55772540be30 2024-04-26T18:25:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.420+0000 7f8c6d7fa640 10 client.16452 handle_command_reply: tid=0 2024-04-26T18:25:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.420+0000 7f8c8eb98740 1 client.16452 shutdown 2024-04-26T18:25:02.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5577253954f0 msgr2=0x55772538ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5577253954f0 0x55772538ea90 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8c74013760 tx=0x7f8c7401e040 comp rx=0 tx=0).stop 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 shutdown_connections 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55772540be30 0x55772540e250 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c6d7fa640 0 client.16452 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5577253954f0 0x55772538ea90 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557725386720 0x55772539daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 --2- 172.21.15.90:0/2629078698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557725385e00 0x55772539d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 >> 172.21.15.90:0/2629078698 conn(0x557725382410 msgr2=0x557725380d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 shutdown_connections 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 1 -- 172.21.15.90:0/2629078698 wait complete. 2024-04-26T18:25:02.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.421+0000 7f8c8eb98740 20 client.16452 trim_cache size 0 max 16384 2024-04-26T18:25:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:25:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:25:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:25:02.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8c700a3dc0 msgr2=0x7f8c700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8c700a3dc0 0x7f8c700a6280 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0b9c90 tx=0x7f8c7c0b93b0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 msgr2=0x7f8c8419d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8419d820 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009dc0 tx=0x7f8c7c02e5c0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 shutdown_connections 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f8c700a3dc0 0x7f8c700a6280 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c8410cf00 0x7f8c8419d820 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8c841044e0 0x7f8c841994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 --2- 172.21.15.90:0/2440507566 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c84103bc0 0x7f8c84198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 >> 172.21.15.90:0/2440507566 conn(0x7f8c840789b0 msgr2=0x7f8c840fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 shutdown_connections 2024-04-26T18:25:02.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:02.422+0000 7f8c8be7e640 1 -- 172.21.15.90:0/2440507566 wait complete. 2024-04-26T18:25:02.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 510296 2.84 MB/sec execute 530 sec latency 14824.509 ms 2024-04-26T18:25:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:02 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:25:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:02 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:reconnect 2024-04-26T18:25:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:02 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:reconnect} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:02 smithi119 ceph-mon[32752]: pgmap v431: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:25:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:02 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:25:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:02 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:reconnect 2024-04-26T18:25:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:02 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:reconnect} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:02 smithi167 ceph-mon[32751]: pgmap v431: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:25:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:02 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:25:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:02 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:reconnect 2024-04-26T18:25:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:02 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:reconnect} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:02 smithi090 ceph-mon[29180]: pgmap v431: 97 pgs: 97 active+clean; 164 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:25:03.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 510296 2.84 MB/sec execute 531 sec latency 1005.052 ms 2024-04-26T18:25:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:rejoin 2024-04-26T18:25:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:03 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:rejoin} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:rejoin 2024-04-26T18:25:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:03 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:rejoin} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:rejoin 2024-04-26T18:25:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:03 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:rejoin} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:04.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 510912 2.84 MB/sec execute 532 sec latency 1731.733 ms 2024-04-26T18:25:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:clientreplay 2024-04-26T18:25:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:clientreplay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:04 smithi119 ceph-mon[32752]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:25:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:04 smithi119 ceph-mon[32752]: pgmap v432: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-26T18:25:04.818 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:25:04 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:25:04.431+0000 7f61a8296700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 1 2024-04-26T18:25:04.818 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:25:04 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:25:04.431+0000 7f61a8a97700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:25:04.818 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:25:04 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:25:04.431+0000 7f61a6a93700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 3 2024-04-26T18:25:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:clientreplay 2024-04-26T18:25:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:clientreplay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:04 smithi167 ceph-mon[32751]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:25:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:04 smithi167 ceph-mon[32751]: pgmap v432: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-26T18:25:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:clientreplay 2024-04-26T18:25:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:clientreplay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:04 smithi090 ceph-mon[29180]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:25:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:04 smithi090 ceph-mon[29180]: pgmap v432: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 55 KiB/s wr, 30 op/s 2024-04-26T18:25:05.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 511842 2.85 MB/sec execute 533 sec latency 32.419 ms 2024-04-26T18:25:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:05 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:25:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:05 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:active 2024-04-26T18:25:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:05 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:05 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:25:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:05 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:25:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:05 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:active 2024-04-26T18:25:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:05 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:05 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:25:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:05 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:25:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:05 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3309576493,v1:172.21.15.167:6837/3309576493] up:active 2024-04-26T18:25:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:05 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:05 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:25:06.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 513015 2.86 MB/sec execute 534 sec latency 16.646 ms 2024-04-26T18:25:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:06 smithi119 ceph-mon[32752]: osdmap e126: 12 total, 12 up, 12 in 2024-04-26T18:25:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:06 smithi119 ceph-mon[32752]: pgmap v434: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 68 KiB/s wr, 37 op/s 2024-04-26T18:25:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:06 smithi167 ceph-mon[32751]: osdmap e126: 12 total, 12 up, 12 in 2024-04-26T18:25:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:06 smithi167 ceph-mon[32751]: pgmap v434: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 68 KiB/s wr, 37 op/s 2024-04-26T18:25:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:06 smithi090 ceph-mon[29180]: osdmap e126: 12 total, 12 up, 12 in 2024-04-26T18:25:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:06 smithi090 ceph-mon[29180]: pgmap v434: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 68 KiB/s wr, 37 op/s 2024-04-26T18:25:07.705 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515396 2.88 MB/sec execute 535 sec latency 12.859 ms 2024-04-26T18:25:08.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.87 MB/sec execute 536 sec latency 558.816 ms 2024-04-26T18:25:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:08 smithi119 ceph-mon[32752]: pgmap v435: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-04-26T18:25:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:08 smithi167 ceph-mon[32751]: pgmap v435: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-04-26T18:25:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:08 smithi090 ceph-mon[29180]: pgmap v435: 97 pgs: 97 active+clean; 156 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-04-26T18:25:09.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.87 MB/sec execute 537 sec latency 1558.897 ms 2024-04-26T18:25:10.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.86 MB/sec execute 538 sec latency 2558.997 ms 2024-04-26T18:25:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:10 smithi119 ceph-mon[32752]: pgmap v436: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-26T18:25:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:10 smithi167 ceph-mon[32751]: pgmap v436: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-26T18:25:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:10 smithi090 ceph-mon[29180]: pgmap v436: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-26T18:25:11.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.86 MB/sec execute 539 sec latency 3559.101 ms 2024-04-26T18:25:12.173 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fbc9a7ec700 time 2024-04-26T18:25:12.170404+0000 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbca4e153dd] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.174 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: clone() 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.171+0000 7fbc9a7ec700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fbc9a7ec700 time 2024-04-26T18:25:12.170404+0000 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbca4e153dd] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.175 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.176 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.547 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.547 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.547 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.547 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.547 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.547 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: clone() 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: *** Caught signal (Aborted) ** 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: in thread 7fbc9a7ec700 thread_name:ms_dispatch 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbca3b9dd20] 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: gsignal() 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: abort() 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbca4e15437] 2024-04-26T18:25:12.548 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 16: clone() 2024-04-26T18:25:12.549 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.172+0000 7fbc9a7ec700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: in thread 7fbc9a7ec700 thread_name:ms_dispatch 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbca3b9dd20] 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: gsignal() 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: abort() 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbca4e15437] 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.550 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 16: clone() 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: -2> 2024-04-26T18:25:12.171+0000 7fbc9a7ec700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fbc9a7ec700 time 2024-04-26T18:25:12.170404+0000 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:12.551 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.552 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.552 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbca4e153dd] 2024-04-26T18:25:12.552 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: clone() 2024-04-26T18:25:12.553 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 0> 2024-04-26T18:25:12.172+0000 7fbc9a7ec700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: in thread 7fbc9a7ec700 thread_name:ms_dispatch 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbca3b9dd20] 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: gsignal() 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: abort() 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbca4e15437] 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 16: clone() 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: -2> 2024-04-26T18:25:12.171+0000 7fbc9a7ec700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fbc9a7ec700 time 2024-04-26T18:25:12.170404+0000 2024-04-26T18:25:12.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fbca4e153dd] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: clone() 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 0> 2024-04-26T18:25:12.172+0000 7fbc9a7ec700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: in thread 7fbc9a7ec700 thread_name:ms_dispatch 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fbca3b9dd20] 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2: gsignal() 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 3: abort() 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fbca4e15437] 2024-04-26T18:25:12.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fbca4e155a3] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x562aba42d822] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562aba39d598] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562aba39fc17] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562aba298d0b] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x562aba2992ac] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x562aba281fad] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fbca50a9e20] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fbca50a6fdf] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fbca5179fc1] 2024-04-26T18:25:12.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fbca3b931ca] 2024-04-26T18:25:12.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 16: clone() 2024-04-26T18:25:12.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:12.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70185]: 2024-04-26T18:25:12.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 podman[70537]: 2024-04-26 18:25:12.239676087 +0000 UTC m=+0.021068703 container died e8546b309f2f8f9bbca1fa2a2732748a677790ea9e054c05cf661d2b97d0cc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:25:12.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.85 MB/sec execute 540 sec latency 4559.180 ms 2024-04-26T18:25:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-mon[32752]: pgmap v437: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-26T18:25:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:12 smithi119 ceph-mon[32752]: osdmap e127: 12 total, 12 up, 12 in 2024-04-26T18:25:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:12 smithi167 ceph-mon[32751]: pgmap v437: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-26T18:25:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:12 smithi167 ceph-mon[32751]: osdmap e127: 12 total, 12 up, 12 in 2024-04-26T18:25:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:12 smithi090 ceph-mon[29180]: pgmap v437: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-26T18:25:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:12 smithi090 ceph-mon[29180]: osdmap e127: 12 total, 12 up, 12 in 2024-04-26T18:25:13.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 podman[70537]: 2024-04-26 18:25:12.981528019 +0000 UTC m=+0.762920636 container remove e8546b309f2f8f9bbca1fa2a2732748a677790ea9e054c05cf661d2b97d0cc74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T18:25:13.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:12 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:25:13.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:13 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-26T18:25:13.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:13 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.967s CPU time. 2024-04-26T18:25:13.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.84 MB/sec execute 541 sec latency 5559.266 ms 2024-04-26T18:25:14.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.84 MB/sec execute 542 sec latency 6559.356 ms 2024-04-26T18:25:14.981 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:14 smithi090 ceph-mon[29180]: pgmap v439: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-04-26T18:25:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:14 smithi119 ceph-mon[32752]: pgmap v439: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-04-26T18:25:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:14 smithi167 ceph-mon[32751]: pgmap v439: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.0 MiB/s wr, 194 op/s 2024-04-26T18:25:15.523 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 -- 172.21.15.90:0/1650410222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 msgr2=0x7ffabc108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 --2- 172.21.15.90:0/1650410222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc108d10 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009930 tx=0x7ffaac02a420 comp rx=0 tx=0).stop 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 -- 172.21.15.90:0/1650410222 shutdown_connections 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 --2- 172.21.15.90:0/1650410222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc1032e0 0x7ffabc10f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 --2- 172.21.15.90:0/1650410222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc102920 0x7ffabc102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 --2- 172.21.15.90:0/1650410222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc108d10 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 -- 172.21.15.90:0/1650410222 >> 172.21.15.90:0/1650410222 conn(0x7ffabc0fe3f0 msgr2=0x7ffabc100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 -- 172.21.15.90:0/1650410222 shutdown_connections 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 -- 172.21.15.90:0/1650410222 wait complete. 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.521+0000 7ffac1d38640 1 Processor -- start 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffac1d38640 1 -- start start 2024-04-26T18:25:15.524 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffac1d38640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc102920 0x7ffabc199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffac1d38640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc1032e0 0x7ffabc1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffac1d38640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffac1d38640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc112bc0 con 0x7ffabc108930 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffac1d38640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc112a40 con 0x7ffabc1032e0 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffac1d38640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc112d40 con 0x7ffabc102920 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43312/0 (socket says 172.21.15.90:43312) 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 -- 172.21.15.90:0/4236908076 learned_addr learned my addr 172.21.15.90:0/4236908076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabaffd640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc1032e0 0x7ffabc1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc102920 0x7ffabc199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 -- 172.21.15.90:0/4236908076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc102920 msgr2=0x7ffabc199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc102920 0x7ffabc199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 -- 172.21.15.90:0/4236908076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc1032e0 msgr2=0x7ffabc1995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc1032e0 0x7ffabc1995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 -- 172.21.15.90:0/4236908076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc19e050 con 0x7ffabc108930 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.522+0000 7ffabbfff640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc19d920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ffab400cce0 tx=0x7ffab400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.523+0000 7ffab8ff9640 1 -- 172.21.15.90:0/4236908076 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab400f450 con 0x7ffabc108930 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.523+0000 7ffab8ff9640 1 -- 172.21.15.90:0/4236908076 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab400faa0 con 0x7ffabc108930 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.523+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc19e340 con 0x7ffabc108930 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.523+0000 7ffab8ff9640 1 -- 172.21.15.90:0/4236908076 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4018b70 con 0x7ffabc108930 2024-04-26T18:25:15.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.523+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a5bd0 con 0x7ffabc108930 2024-04-26T18:25:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.525+0000 7ffab8ff9640 1 -- 172.21.15.90:0/4236908076 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7ffab4018d10 con 0x7ffabc108930 2024-04-26T18:25:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.525+0000 7ffac4a54740 1 Processor -- start 2024-04-26T18:25:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.525+0000 7ffac4a54740 1 -- start start 2024-04-26T18:25:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.525+0000 7ffab8ff9640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ffaa40a3dc0 0x7ffaa40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.525+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d5b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d69fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffac4a54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5609c9cc1460 con 0x5609c9d6a4f0 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffac4a54740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5609c9c7b590 con 0x5609c9d5ae00 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffac4a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5609c9cbd420 con 0x5609c9d5b720 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabaffd640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ffaa40a3dc0 0x7ffaa40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffab8ff9640 1 -- 172.21.15.90:0/4236908076 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 10003+0+0 (secure 0 0 0) 0x7ffab40d11e0 con 0x7ffabc108930 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43314/0 (socket says 172.21.15.90:43314) 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 -- 172.21.15.90:0/851386184 learned_addr learned my addr 172.21.15.90:0/851386184 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d5b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabaffd640 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d69fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 -- 172.21.15.90:0/851386184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 msgr2=0x5609c9d69fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d69fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 -- 172.21.15.90:0/851386184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 msgr2=0x5609c9d5b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d5b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 -- 172.21.15.90:0/851386184 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c9c880d0 con 0x5609c9d6a4f0 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabaffd640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ffaa40a3dc0 0x7ffaa40a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffab00058a0 tx=0x7ffab000b770 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d5b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffabbfff640 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6c8e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffab40a52f0 tx=0x7ffab409c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706c3cbde260847c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffaa1ffb640 1 -- 172.21.15.90:0/851386184 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab40b6070 con 0x5609c9d6a4f0 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffaa1ffb640 1 -- 172.21.15.90:0/851386184 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab40a0f60 con 0x5609c9d6a4f0 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.526+0000 7ffaa1ffb640 1 -- 172.21.15.90:0/851386184 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab40bd200 con 0x5609c9d6a4f0 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 -- 172.21.15.90:0/851386184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 msgr2=0x5609c9d6c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6c8e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7ffab40a52f0 tx=0x7ffab409c5d0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 -- 172.21.15.90:0/851386184 shutdown_connections 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6c8e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d69fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 --2- 172.21.15.90:0/851386184 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d5b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 -- 172.21.15.90:0/851386184 >> 172.21.15.90:0/851386184 conn(0x5609c9d57410 msgr2=0x5609c9d577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 -- 172.21.15.90:0/851386184 shutdown_connections 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 -- 172.21.15.90:0/851386184 wait complete. 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.527+0000 7ffac4a54740 1 -- start start 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 1 Processor -- start 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5b720 0x5609c9d678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d6a4f0 0x5609c9d67e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5609c9d6d120 con 0x5609c9d5b720 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5609c9c48b60 con 0x5609c9d5ae00 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5609c9d6d2a0 con 0x5609c9d6a4f0 2024-04-26T18:25:15.536 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffac4a54740 10 client.0 _ll_register_callbacks cb 0x5609c9c6d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5b720 0x5609c9d678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5b720 0x5609c9d678e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43330/0 (socket says 172.21.15.90:43330) 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 -- 172.21.15.90:0/109200950 learned_addr learned my addr 172.21.15.90:0/109200950 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabbfff640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d6a4f0 0x5609c9d67e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 -- 172.21.15.90:0/109200950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d6a4f0 msgr2=0x5609c9d67e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d6a4f0 0x5609c9d67e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 -- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 msgr2=0x5609c9d673a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d673a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabaffd640 1 -- 172.21.15.90:0/109200950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c99d3210 con 0x5609c9d5b720 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.528+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d673a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffabaffd640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5b720 0x5609c9d678e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffab001e670 tx=0x7ffab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff94a09cef4f487 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:15.537 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffa9affd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffa9affd640 1 -- 172.21.15.90:0/109200950 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab002b060 con 0x5609c9d5b720 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffa9affd640 1 -- 172.21.15.90:0/109200950 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 10003+0+0 (secure 0 0 0) 0x7ffab001c780 con 0x5609c9d5b720 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffab40a0f60 con 0x5609c9d5b720 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffa9affd640 1 -- 172.21.15.90:0/109200950 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7ffab0032050 con 0x5609c9d5b720 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 10 client.16467 fetch_fsmap learned FSMap version 140 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) -- 0x5609c9a9f620 con 0x5609c9d5b720 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffa9affd640 1 -- 172.21.15.90:0/109200950 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffab40a0f60 con 0x5609c9d5b720 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffa9affd640 1 -- 172.21.15.90:0/109200950 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 140) ==== 7136+0+0 (secure 0 0 0) 0x7ffab0026c60 con 0x5609c9d5b720 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 10 client.16467 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 10 client.16467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 20 client.16467 populate_metadata read hostname 'smithi090' 2024-04-26T18:25:15.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9d7a7d0 0x5609c9d7cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 4 client.16467 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.529+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c9ccf070 con 0x5609c9d7a7d0 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.530+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9d7a7d0 0x5609c9d7cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.530+0000 7ffa9affd640 10 client.16467 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.530+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9d7a7d0 0x5609c9d7cc90 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a982d9b7753e909c server_cookie=2668b901b0f1e9a1 in_seq=0 out_seq=0 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffa9affd640 1 -- 172.21.15.90:0/109200950 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5609c9ccf070 con 0x5609c9d7a7d0 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffa9affd640 10 client.16467 handle_command_reply: tid=0 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 client.16467 shutdown 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5b720 msgr2=0x5609c9d678e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5b720 0x5609c9d678e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffab001e670 tx=0x7ffab0004730 comp rx=0 tx=0).stop 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 shutdown_connections 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9d7a7d0 0x5609c9d7cc90 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.539 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d6a4f0 0x5609c9d67e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5b720 0x5609c9d678e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 --2- 172.21.15.90:0/109200950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5ae00 0x5609c9d673a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 >> 172.21.15.90:0/109200950 conn(0x5609c9d57410 msgr2=0x5609c9d5d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffa9affd640 0 client.16467 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 shutdown_connections 2024-04-26T18:25:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 1 -- 172.21.15.90:0/109200950 wait complete. 2024-04-26T18:25:15.540 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.531+0000 7ffac4a54740 20 client.16467 trim_cache size 0 max 16384 2024-04-26T18:25:15.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 Processor -- start 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 -- start start 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5ae00 0x5609c9d75fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d66500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d6a4f0 0x5609c9d56430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1adb20 con 0x5609c9d5ae00 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1ae1f0 con 0x5609c9d6a4f0 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffac4a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1ae8c0 con 0x5609c9d5b720 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5ae00 0x5609c9d75fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5ae00 0x5609c9d75fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43342/0 (socket says 172.21.15.90:43342) 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 -- 172.21.15.90:0/1978943087 learned_addr learned my addr 172.21.15.90:0/1978943087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabaffd640 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d66500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 -- 172.21.15.90:0/1978943087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 msgr2=0x5609c9d66500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d66500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 -- 172.21.15.90:0/1978943087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d6a4f0 msgr2=0x5609c9d56430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabbfff640 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d6a4f0 0x5609c9d56430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d6a4f0 0x5609c9d56430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabb7fe640 1 -- 172.21.15.90:0/1978943087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c9d57240 con 0x5609c9d5ae00 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.546+0000 7ffabaffd640 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d66500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5ae00 0x5609c9d75fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffaac000c00 tx=0x7ffaac005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e485b270790531b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffaa17fa640 1 -- 172.21.15.90:0/1978943087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac035040 con 0x5609c9d5ae00 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffaa17fa640 1 -- 172.21.15.90:0/1978943087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac047070 con 0x5609c9d5ae00 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffaa17fa640 1 -- 172.21.15.90:0/1978943087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac005570 con 0x5609c9d5ae00 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 -- 172.21.15.90:0/1978943087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5ae00 msgr2=0x5609c9d75fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.552 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5ae00 0x5609c9d75fe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffaac000c00 tx=0x7ffaac005de0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 -- 172.21.15.90:0/1978943087 shutdown_connections 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d6a4f0 0x5609c9d56430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5b720 0x5609c9d66500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 --2- 172.21.15.90:0/1978943087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d5ae00 0x5609c9d75fe0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 -- 172.21.15.90:0/1978943087 >> 172.21.15.90:0/1978943087 conn(0x5609c9d57410 msgr2=0x5609c9d55d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 -- 172.21.15.90:0/1978943087 shutdown_connections 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.547+0000 7ffac4a54740 1 -- 172.21.15.90:0/1978943087 wait complete. 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 -- start start 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 Processor -- start 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5ae00 0x5609c9d72800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5b720 0x5609c9d6db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.553 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1aadc0 con 0x5609c9d6a4f0 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1ac1b0 con 0x5609c9d5b720 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc1ab410 con 0x5609c9d5ae00 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffac4a54740 10 client.0 _ll_register_callbacks cb 0x7ffab000ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffabbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffabbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43344/0 (socket says 172.21.15.90:43344) 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffabbfff640 1 -- 172.21.15.90:0/2844635739 learned_addr learned my addr 172.21.15.90:0/2844635739 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.548+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5ae00 0x5609c9d72800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabaffd640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5b720 0x5609c9d6db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabbfff640 1 -- 172.21.15.90:0/2844635739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5ae00 msgr2=0x5609c9d72800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabbfff640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5ae00 0x5609c9d72800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabbfff640 1 -- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5b720 msgr2=0x5609c9d6db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabbfff640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5b720 0x5609c9d6db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.554 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabbfff640 1 -- 172.21.15.90:0/2844635739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c9d6e8b0 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabaffd640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5b720 0x5609c9d6db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffabbfff640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6e0b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffab400ab20 tx=0x7ffab400ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407ea62c73a5d578 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffa9affd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffa9affd640 1 -- 172.21.15.90:0/2844635739 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab40b6040 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffa9affd640 1 -- 172.21.15.90:0/2844635739 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 10003+0+0 (secure 0 0 0) 0x7ffab4099220 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaac047070 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffa9affd640 1 -- 172.21.15.90:0/2844635739 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=140) ==== 24+0+0 (secure 0 0 0) 0x7ffab40a0f60 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 10 client.16473 fetch_fsmap learned FSMap version 140 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=140,monmap=4+}) -- 0x5609c9de0ce0 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffa9affd640 1 -- 172.21.15.90:0/2844635739 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaac047070 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffa9affd640 1 -- 172.21.15.90:0/2844635739 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 140) ==== 7136+0+0 (secure 0 0 0) 0x7ffab409c000 con 0x5609c9d6a4f0 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 10 client.16473 fetch_fsmap finished waiting for FSMap version 140 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:15.555 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 20 client.16473 populate_metadata read hostname 'smithi090' 2024-04-26T18:25:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9de1170 0x5609c9de3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 4 client.16473 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:25:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.549+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5609c9cc8ee0 con 0x5609c9de1170 2024-04-26T18:25:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.550+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9de1170 0x5609c9de3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.550+0000 7ffabb7fe640 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9de1170 0x5609c9de3590 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4bdde66c7b479e5 server_cookie=c55fca2f922a1b2d in_seq=0 out_seq=0 2024-04-26T18:25:15.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:15.550+0000 7ffa9affd640 10 client.16473 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:15.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.83 MB/sec execute 543 sec latency 7559.464 ms 2024-04-26T18:25:16.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.83 MB/sec execute 544 sec latency 8559.572 ms 2024-04-26T18:25:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:16 smithi119 ceph-mon[32752]: pgmap v440: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-04-26T18:25:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:25:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:16 smithi167 ceph-mon[32751]: pgmap v440: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-04-26T18:25:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:25:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:16 smithi090 ceph-mon[29180]: pgmap v440: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.6 MiB/s wr, 161 op/s 2024-04-26T18:25:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:25:17.706 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.82 MB/sec execute 545 sec latency 9559.657 ms 2024-04-26T18:25:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:17 smithi119 ceph-mon[32752]: pgmap v441: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.8 MiB/s wr, 184 op/s 2024-04-26T18:25:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:17 smithi119 ceph-mon[32752]: pgmap v442: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-04-26T18:25:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:17 smithi167 ceph-mon[32751]: pgmap v441: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.8 MiB/s wr, 184 op/s 2024-04-26T18:25:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:17 smithi167 ceph-mon[32751]: pgmap v442: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-04-26T18:25:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:17 smithi090 ceph-mon[29180]: pgmap v441: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.8 MiB/s wr, 184 op/s 2024-04-26T18:25:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:17 smithi090 ceph-mon[29180]: pgmap v442: 97 pgs: 97 active+clean; 161 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 63 KiB/s wr, 14 op/s 2024-04-26T18:25:18.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.82 MB/sec execute 546 sec latency 10559.773 ms 2024-04-26T18:25:19.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.81 MB/sec execute 547 sec latency 11559.828 ms 2024-04-26T18:25:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:19 smithi119 ceph-mon[32752]: pgmap v443: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 251 KiB/s wr, 44 op/s 2024-04-26T18:25:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:19 smithi167 ceph-mon[32751]: pgmap v443: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 251 KiB/s wr, 44 op/s 2024-04-26T18:25:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:19 smithi090 ceph-mon[29180]: pgmap v443: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 251 KiB/s wr, 44 op/s 2024-04-26T18:25:20.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.81 MB/sec execute 548 sec latency 12559.920 ms 2024-04-26T18:25:21.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.80 MB/sec execute 549 sec latency 13560.027 ms 2024-04-26T18:25:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:21 smithi119 ceph-mon[32752]: pgmap v444: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 214 KiB/s wr, 37 op/s 2024-04-26T18:25:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:21 smithi167 ceph-mon[32751]: pgmap v444: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 214 KiB/s wr, 37 op/s 2024-04-26T18:25:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:21 smithi090 ceph-mon[29180]: pgmap v444: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 214 KiB/s wr, 37 op/s 2024-04-26T18:25:22.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.80 MB/sec execute 550 sec latency 14560.073 ms 2024-04-26T18:25:23.441 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f4522130700 time 2024-04-26T18:25:23.076682+0000 2024-04-26T18:25:23.441 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:23.441 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.441 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f452c7593dd] 2024-04-26T18:25:23.441 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.442 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: clone() 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.077+0000 7f4522130700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f4522130700 time 2024-04-26T18:25:23.076682+0000 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f452c7593dd] 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.443 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: clone() 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: *** Caught signal (Aborted) ** 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: in thread 7f4522130700 thread_name:ms_dispatch 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f452b4e1d20] 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: gsignal() 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: abort() 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f452c759437] 2024-04-26T18:25:23.444 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.445 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.445 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.445 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.445 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 16: clone() 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.079+0000 7f4522130700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: in thread 7f4522130700 thread_name:ms_dispatch 2024-04-26T18:25:23.446 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f452b4e1d20] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: gsignal() 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: abort() 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f452c759437] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.447 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 16: clone() 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: -1> 2024-04-26T18:25:23.077+0000 7f4522130700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f4522130700 time 2024-04-26T18:25:23.076682+0000 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f452c7593dd] 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.448 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: clone() 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 0> 2024-04-26T18:25:23.079+0000 7f4522130700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:23.449 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: in thread 7f4522130700 thread_name:ms_dispatch 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f452b4e1d20] 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: gsignal() 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: abort() 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f452c759437] 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.450 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 16: clone() 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: -2> 2024-04-26T18:25:23.077+0000 7f4522130700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f4522130700 time 2024-04-26T18:25:23.076682+0000 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.451 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f452c7593dd] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.452 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: clone() 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: -1> 2024-04-26T18:25:23.079+0000 7f4522130700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: in thread 7f4522130700 thread_name:ms_dispatch 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f452b4e1d20] 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2: gsignal() 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 3: abort() 2024-04-26T18:25:23.453 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f452c759437] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f452c7595a3] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e38165e822] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e3815ce598] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e3815d0c17] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e3814c9d0b] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e3814ca2ac] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e3814b2fad] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f452c9ede20] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 13: (DispatchQueue::entry()+0x51f) [0x7f452c9eafdf] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f452cabdfc1] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f452b4d71ca] 2024-04-26T18:25:23.454 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 16: clone() 2024-04-26T18:25:23.455 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:23.455 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[70752]: 2024-04-26T18:25:23.455 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 podman[71121]: 2024-04-26 18:25:23.139505057 +0000 UTC m=+0.019228212 container died b2de4e490d10d5b20104e30af65e71c21767538e2698707ebcf89935674b31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:25:23.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:25:23.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 systemd[1]: Stopped Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:23.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.967s CPU time. 2024-04-26T18:25:23.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 systemd[1]: Starting Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:25:23.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 podman[70763]: 2024-04-26 18:25:23.451417457 +0000 UTC m=+0.018574100 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:25:23.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.79 MB/sec execute 551 sec latency 15560.189 ms 2024-04-26T18:25:23.808 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 podman[70763]: 2024-04-26 18:25:23.557035705 +0000 UTC m=+0.124192376 container create 121daece8543862cd3d2bb5608ceb7d47922413d51bf3d38ca9bd08dffc41226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:25:23.809 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 podman[70763]: 2024-04-26 18:25:23.784562187 +0000 UTC m=+0.351718812 container init 121daece8543862cd3d2bb5608ceb7d47922413d51bf3d38ca9bd08dffc41226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:25:23.809 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 podman[70763]: 2024-04-26 18:25:23.787248872 +0000 UTC m=+0.354405498 container start 121daece8543862cd3d2bb5608ceb7d47922413d51bf3d38ca9bd08dffc41226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T18:25:23.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:23 smithi119 ceph-mon[32752]: pgmap v445: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 145 KiB/s wr, 22 op/s 2024-04-26T18:25:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:23 smithi167 ceph-mon[32751]: pgmap v445: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 145 KiB/s wr, 22 op/s 2024-04-26T18:25:24.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 podman[71121]: 2024-04-26 18:25:23.893170927 +0000 UTC m=+0.772894117 container remove b2de4e490d10d5b20104e30af65e71c21767538e2698707ebcf89935674b31bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T18:25:24.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:23 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:25:24.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:23 smithi090 ceph-mon[29180]: pgmap v445: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 145 KiB/s wr, 22 op/s 2024-04-26T18:25:24.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 ceph-mds[70782]: -- 172.21.15.119:0/3171730983 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d692573a40 con 0x55d692546c00 2024-04-26T18:25:24.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: starting mds.e at 2024-04-26T18:25:24.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 bash[70763]: 121daece8543862cd3d2bb5608ceb7d47922413d51bf3d38ca9bd08dffc41226 2024-04-26T18:25:24.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:25:23 smithi119 systemd[1]: Started Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:24.567 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:24 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-26T18:25:24.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:24 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 8.377s CPU time. 2024-04-26T18:25:24.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.79 MB/sec execute 552 sec latency 16560.264 ms 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: daemon mds.e restarted 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: osdmap e128: 12 total, 12 up, 12 in 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:boot 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:25:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:24 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: daemon mds.e restarted 2024-04-26T18:25:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: osdmap e128: 12 total, 12 up, 12 in 2024-04-26T18:25:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:boot 2024-04-26T18:25:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:25:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:25:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:24 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: daemon mds.e restarted 2024-04-26T18:25:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: osdmap e128: 12 total, 12 up, 12 in 2024-04-26T18:25:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:boot 2024-04-26T18:25:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:25:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:25:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:24 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:25.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.78 MB/sec execute 553 sec latency 17560.320 ms 2024-04-26T18:25:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:25 smithi090 ceph-mon[29180]: pgmap v447: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T18:25:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:25 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:resolve 2024-04-26T18:25:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:25 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:25 smithi119 ceph-mon[32752]: pgmap v447: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T18:25:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:25 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:resolve 2024-04-26T18:25:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:25 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:25 smithi167 ceph-mon[32751]: pgmap v447: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 161 KiB/s wr, 29 op/s 2024-04-26T18:25:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:25 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:resolve 2024-04-26T18:25:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:25 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:26.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.78 MB/sec execute 554 sec latency 18560.430 ms 2024-04-26T18:25:27.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.77 MB/sec execute 555 sec latency 19560.512 ms 2024-04-26T18:25:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:27 smithi090 ceph-mon[29180]: pgmap v448: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 129 KiB/s wr, 23 op/s 2024-04-26T18:25:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:27 smithi119 ceph-mon[32752]: pgmap v448: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 129 KiB/s wr, 23 op/s 2024-04-26T18:25:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:27 smithi167 ceph-mon[32751]: pgmap v448: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 129 KiB/s wr, 23 op/s 2024-04-26T18:25:28.707 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.77 MB/sec execute 556 sec latency 20560.622 ms 2024-04-26T18:25:29.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.76 MB/sec execute 557 sec latency 21560.750 ms 2024-04-26T18:25:30.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:29 smithi090 ceph-mon[29180]: pgmap v449: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:29 smithi119 ceph-mon[32752]: pgmap v449: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:29 smithi167 ceph-mon[32751]: pgmap v449: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:30.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.76 MB/sec execute 558 sec latency 22560.842 ms 2024-04-26T18:25:31.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.75 MB/sec execute 559 sec latency 23560.958 ms 2024-04-26T18:25:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:31 smithi090 ceph-mon[29180]: pgmap v450: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:31 smithi119 ceph-mon[32752]: pgmap v450: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:31 smithi167 ceph-mon[32751]: pgmap v450: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:32.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.75 MB/sec execute 560 sec latency 24561.020 ms 2024-04-26T18:25:33.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.74 MB/sec execute 561 sec latency 25561.071 ms 2024-04-26T18:25:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:33 smithi090 ceph-mon[29180]: pgmap v451: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:33 smithi119 ceph-mon[32752]: pgmap v451: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:33 smithi167 ceph-mon[32751]: pgmap v451: 97 pgs: 97 active+clean; 162 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 5.8 KiB/s wr, 7 op/s 2024-04-26T18:25:34.627 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:25:34.627 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 systemd[1]: Stopped Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:34.627 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 8.377s CPU time. 2024-04-26T18:25:34.627 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 systemd[1]: Starting Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:25:34.627 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 podman[71217]: 2024-04-26 18:25:34.466295344 +0000 UTC m=+0.018783356 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:25:34.627 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 podman[71217]: 2024-04-26 18:25:34.627585488 +0000 UTC m=+0.180073535 container create 41d396362e4850f8f571552c2beb6fbd8aabec2e5fc47667df07ba0e40f457e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T18:25:34.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.74 MB/sec execute 562 sec latency 26561.199 ms 2024-04-26T18:25:34.880 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 podman[71217]: 2024-04-26 18:25:34.855666956 +0000 UTC m=+0.408154958 container init 41d396362e4850f8f571552c2beb6fbd8aabec2e5fc47667df07ba0e40f457e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T18:25:34.880 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 podman[71217]: 2024-04-26 18:25:34.858442939 +0000 UTC m=+0.410930943 container start 41d396362e4850f8f571552c2beb6fbd8aabec2e5fc47667df07ba0e40f457e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T18:25:35.317 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 ceph-mds[71236]: -- 172.21.15.167:0/220779583 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56463e591a40 con 0x56463e564c00 2024-04-26T18:25:35.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: starting mds.c at 2024-04-26T18:25:35.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 bash[71217]: 41d396362e4850f8f571552c2beb6fbd8aabec2e5fc47667df07ba0e40f457e4 2024-04-26T18:25:35.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:25:34 smithi167 systemd[1]: Started Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:35.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.73 MB/sec execute 563 sec latency 27561.302 ms 2024-04-26T18:25:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: pgmap v452: 97 pgs: 97 active+clean; 167 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 519 KiB/s wr, 12 op/s 2024-04-26T18:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: daemon mds.c restarted 2024-04-26T18:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: osdmap e129: 12 total, 12 up, 12 in 2024-04-26T18:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:boot 2024-04-26T18:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:35 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: pgmap v452: 97 pgs: 97 active+clean; 167 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 519 KiB/s wr, 12 op/s 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: daemon mds.c restarted 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: osdmap e129: 12 total, 12 up, 12 in 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:boot 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:35 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: pgmap v452: 97 pgs: 97 active+clean; 167 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 519 KiB/s wr, 12 op/s 2024-04-26T18:25:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: daemon mds.c restarted 2024-04-26T18:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: osdmap e129: 12 total, 12 up, 12 in 2024-04-26T18:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:boot 2024-04-26T18:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:35 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:36.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.73 MB/sec execute 564 sec latency 28561.438 ms 2024-04-26T18:25:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:36 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:resolve 2024-04-26T18:25:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:36 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:36 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:resolve 2024-04-26T18:25:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:36 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:36 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:resolve 2024-04-26T18:25:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:36 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:37.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.72 MB/sec execute 565 sec latency 29561.498 ms 2024-04-26T18:25:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.934+0000 7ffa9affd640 1 -- 172.21.15.90:0/2844635739 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5609c9cc8ee0 con 0x5609c9de1170 2024-04-26T18:25:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.934+0000 7ffa9affd640 10 client.16473 handle_command_reply: tid=0 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.934+0000 7ffac4a54740 1 client.16473 shutdown 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 msgr2=0x5609c9d6e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6e0b0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ffab400ab20 tx=0x7ffab400ff50 comp rx=0 tx=0).stop 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 shutdown_connections 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5609c9de1170 0x5609c9de3590 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5609c9d6a4f0 0x5609c9d6e0b0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5609c9d5b720 0x5609c9d6db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 --2- 172.21.15.90:0/2844635739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5609c9d5ae00 0x5609c9d72800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffa9affd640 0 client.16473 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 >> 172.21.15.90:0/2844635739 conn(0x5609c9d57410 msgr2=0x5609c9d67bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 shutdown_connections 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 1 -- 172.21.15.90:0/2844635739 wait complete. 2024-04-26T18:25:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.935+0000 7ffac4a54740 20 client.16473 trim_cache size 0 max 16384 2024-04-26T18:25:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:25:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:25:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:25:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ffaa40a3dc0 msgr2=0x7ffaa40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ffaa40a3dc0 0x7ffaa40a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffab00058a0 tx=0x7ffab000b770 comp rx=0 tx=0).stop 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 msgr2=0x7ffabc19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc19d920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ffab400cce0 tx=0x7ffab400a620 comp rx=0 tx=0).stop 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 shutdown_connections 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ffaa40a3dc0 0x7ffaa40a6280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffabc108930 0x7ffabc19d920 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffabc1032e0 0x7ffabc1995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 --2- 172.21.15.90:0/4236908076 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffabc102920 0x7ffabc199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 >> 172.21.15.90:0/4236908076 conn(0x7ffabc0fe3f0 msgr2=0x7ffabc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 shutdown_connections 2024-04-26T18:25:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:37.936+0000 7ffac1d38640 1 -- 172.21.15.90:0/4236908076 wait complete. 2024-04-26T18:25:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:37 smithi119 ceph-mon[32752]: pgmap v454: 97 pgs: 97 active+clean; 167 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 553 KiB/s wr, 13 op/s 2024-04-26T18:25:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:37 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:25:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:37 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:25:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:37 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:25:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:37 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:reconnect 2024-04-26T18:25:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:37 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:reconnect 2024-04-26T18:25:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:37 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:37 smithi167 ceph-mon[32751]: pgmap v454: 97 pgs: 97 active+clean; 167 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 553 KiB/s wr, 13 op/s 2024-04-26T18:25:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:37 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:25:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:37 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:25:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:37 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:25:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:37 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:reconnect 2024-04-26T18:25:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:37 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:reconnect 2024-04-26T18:25:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:37 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:37 smithi090 ceph-mon[29180]: pgmap v454: 97 pgs: 97 active+clean; 167 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 553 KiB/s wr, 13 op/s 2024-04-26T18:25:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:37 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:25:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:37 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:25:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:37 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:25:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:37 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:reconnect 2024-04-26T18:25:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:37 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:reconnect 2024-04-26T18:25:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:37 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:38.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 515720 2.72 MB/sec execute 566 sec latency 30561.531 ms 2024-04-26T18:25:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:rejoin 2024-04-26T18:25:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:rejoin 2024-04-26T18:25:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:38 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:38 smithi119 ceph-mon[32752]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:25:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:38 smithi119 ceph-mon[32752]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:25:39.317 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:25:38.938+0000 7f61a8a97700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:25:39.317 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:25:38.939+0000 7f61a8296700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 6 2024-04-26T18:25:39.318 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:25:38.939+0000 7f61a6a93700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 6 2024-04-26T18:25:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:rejoin 2024-04-26T18:25:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:rejoin 2024-04-26T18:25:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-mon[32751]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:25:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:38 smithi167 ceph-mon[32751]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:25:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:rejoin 2024-04-26T18:25:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:rejoin 2024-04-26T18:25:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:38 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:38 smithi090 ceph-mon[29180]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:25:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:38 smithi090 ceph-mon[29180]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:25:39.708 INFO:tasks.workunit.client.0.smithi090.stdout: 1 516816 2.72 MB/sec execute 567 sec latency 30805.261 ms 2024-04-26T18:25:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:39 smithi119 ceph-mon[32752]: pgmap v455: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:39 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:25:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:active 2024-04-26T18:25:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:active 2024-04-26T18:25:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:25:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:39 smithi167 ceph-mon[32751]: pgmap v455: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:39 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:25:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:active 2024-04-26T18:25:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:active 2024-04-26T18:25:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:25:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:39 smithi090 ceph-mon[29180]: pgmap v455: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:39 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:25:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1237239761,v1:172.21.15.119:6833/1237239761] up:active 2024-04-26T18:25:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/4236765679,v1:172.21.15.167:6833/4236765679] up:active 2024-04-26T18:25:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:25:40.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 518979 2.74 MB/sec execute 568 sec latency 12.292 ms 2024-04-26T18:25:41.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 520282 2.75 MB/sec execute 569 sec latency 17.572 ms 2024-04-26T18:25:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:41 smithi119 ceph-mon[32752]: pgmap v456: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:41 smithi167 ceph-mon[32751]: pgmap v456: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:41 smithi090 ceph-mon[29180]: pgmap v456: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:42.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 522624 2.77 MB/sec execute 570 sec latency 9.800 ms 2024-04-26T18:25:43.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 524644 2.77 MB/sec execute 571 sec latency 17.232 ms 2024-04-26T18:25:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-mon[32752]: pgmap v457: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:44.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6089956700 time 2024-04-26T18:25:43.957795+0000 2024-04-26T18:25:44.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6093f7f3dd] 2024-04-26T18:25:44.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: clone() 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:43.958+0000 7f6089956700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6089956700 time 2024-04-26T18:25:43.957795+0000 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.305 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.306 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6093f7f3dd] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.307 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: clone() 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: *** Caught signal (Aborted) ** 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: in thread 7f6089956700 thread_name:ms_dispatch 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6092d07d20] 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: gsignal() 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: abort() 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6093f7f437] 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.308 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 16: clone() 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:43.960+0000 7f6089956700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: in thread 7f6089956700 thread_name:ms_dispatch 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6092d07d20] 2024-04-26T18:25:44.309 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: gsignal() 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: abort() 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6093f7f437] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.310 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 16: clone() 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: -1> 2024-04-26T18:25:43.958+0000 7f6089956700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6089956700 time 2024-04-26T18:25:43.957795+0000 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6093f7f3dd] 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.311 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: clone() 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 0> 2024-04-26T18:25:43.960+0000 7f6089956700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: in thread 7f6089956700 thread_name:ms_dispatch 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6092d07d20] 2024-04-26T18:25:44.312 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: gsignal() 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: abort() 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6093f7f437] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 16: clone() 2024-04-26T18:25:44.313 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: -1> 2024-04-26T18:25:43.958+0000 7f6089956700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6089956700 time 2024-04-26T18:25:43.957795+0000 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6093f7f3dd] 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.314 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: clone() 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 0> 2024-04-26T18:25:43.960+0000 7f6089956700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: in thread 7f6089956700 thread_name:ms_dispatch 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6092d07d20] 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2: gsignal() 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 3: abort() 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6093f7f437] 2024-04-26T18:25:44.315 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6093f7f5a3] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x565398fe5822] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x565398f55598] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x565398f57c17] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x565398e50d0b] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x565398e512ac] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x565398e39fad] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6094213e20] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 13: (DispatchQueue::entry()+0x51f) [0x7f6094210fdf] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f60942e3fc1] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6092cfd1ca] 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 16: clone() 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:43 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70478]: 2024-04-26T18:25:44.316 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:44 smithi119 podman[70819]: 2024-04-26 18:25:44.016249601 +0000 UTC m=+0.018866293 container died 921ec7fa11113251a6046bc42bc34c560d94e247939c1e09842d4e0bf667b02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:25:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-mon[32751]: pgmap v457: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff917869700 time 2024-04-26T18:25:43.996528+0000 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff921e923dd] 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: clone() 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:43.996+0000 7ff917869700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff917869700 time 2024-04-26T18:25:43.996528+0000 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff921e923dd] 2024-04-26T18:25:44.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: clone() 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: *** Caught signal (Aborted) ** 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: in thread 7ff917869700 thread_name:ms_dispatch 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff920c1ad20] 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: gsignal() 2024-04-26T18:25:44.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: abort() 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff921e92437] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 16: clone() 2024-04-26T18:25:44.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:43.997+0000 7ff917869700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: in thread 7ff917869700 thread_name:ms_dispatch 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff920c1ad20] 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: gsignal() 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: abort() 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff921e92437] 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 16: clone() 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:43 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: -1> 2024-04-26T18:25:43.996+0000 7ff917869700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff917869700 time 2024-04-26T18:25:43.996528+0000 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff921e923dd] 2024-04-26T18:25:44.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: clone() 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 0> 2024-04-26T18:25:43.997+0000 7ff917869700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: in thread 7ff917869700 thread_name:ms_dispatch 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff920c1ad20] 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: gsignal() 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: abort() 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff921e92437] 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 16: clone() 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: -1> 2024-04-26T18:25:43.996+0000 7ff917869700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7ff917869700 time 2024-04-26T18:25:43.996528+0000 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7ff921e923dd] 2024-04-26T18:25:44.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: clone() 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 0> 2024-04-26T18:25:43.997+0000 7ff917869700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: in thread 7ff917869700 thread_name:ms_dispatch 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7ff920c1ad20] 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2: gsignal() 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 3: abort() 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7ff921e92437] 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7ff921e925a3] 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5559114d9822] 2024-04-26T18:25:44.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555911449598] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55591144bc17] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555911344d0b] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559113452ac] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55591132dfad] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7ff922126e20] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 13: (DispatchQueue::entry()+0x51f) [0x7ff922123fdf] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7ff9221f6fc1] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7ff920c101ca] 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 16: clone() 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[70926]: 2024-04-26T18:25:44.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 podman[71278]: 2024-04-26 18:25:44.055034428 +0000 UTC m=+0.019269176 container died b63256b929ff5d6180c98ad07605ad9de7e9e388376103a88b404f28e44f6939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T18:25:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:43 smithi090 ceph-mon[29180]: pgmap v457: 97 pgs: 97 active+clean; 158 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 570 KiB/s wr, 18 op/s 2024-04-26T18:25:44.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.78 MB/sec execute 572 sec latency 749.378 ms 2024-04-26T18:25:45.067 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 podman[71278]: 2024-04-26 18:25:44.635903999 +0000 UTC m=+0.600138736 container remove b63256b929ff5d6180c98ad07605ad9de7e9e388376103a88b404f28e44f6939 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:25:45.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:25:45.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-04-26T18:25:45.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:44 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 8.838s CPU time. 2024-04-26T18:25:45.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:44 smithi119 podman[70819]: 2024-04-26 18:25:44.813839045 +0000 UTC m=+0.816455768 container remove 921ec7fa11113251a6046bc42bc34c560d94e247939c1e09842d4e0bf667b02e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T18:25:45.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:44 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:25:45.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:45 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-26T18:25:45.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:45 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 4.316s CPU time. 2024-04-26T18:25:45.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.78 MB/sec execute 573 sec latency 1749.508 ms 2024-04-26T18:25:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:45 smithi119 ceph-mon[32752]: pgmap v458: 97 pgs: 97 active+clean; 165 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 933 KiB/s wr, 130 op/s 2024-04-26T18:25:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:45 smithi167 ceph-mon[32751]: pgmap v458: 97 pgs: 97 active+clean; 165 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 933 KiB/s wr, 130 op/s 2024-04-26T18:25:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:45 smithi090 ceph-mon[29180]: pgmap v458: 97 pgs: 97 active+clean; 165 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 933 KiB/s wr, 130 op/s 2024-04-26T18:25:46.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.77 MB/sec execute 574 sec latency 2749.609 ms 2024-04-26T18:25:47.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.77 MB/sec execute 575 sec latency 3749.729 ms 2024-04-26T18:25:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:47 smithi119 ceph-mon[32752]: pgmap v459: 97 pgs: 97 active+clean; 165 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 805 KiB/s wr, 112 op/s 2024-04-26T18:25:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:47 smithi167 ceph-mon[32751]: pgmap v459: 97 pgs: 97 active+clean; 165 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 805 KiB/s wr, 112 op/s 2024-04-26T18:25:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:47 smithi090 ceph-mon[29180]: pgmap v459: 97 pgs: 97 active+clean; 165 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 805 KiB/s wr, 112 op/s 2024-04-26T18:25:48.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.76 MB/sec execute 576 sec latency 4749.862 ms 2024-04-26T18:25:49.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.76 MB/sec execute 577 sec latency 5749.900 ms 2024-04-26T18:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:49 smithi119 ceph-mon[32752]: pgmap v460: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 931 KiB/s wr, 128 op/s 2024-04-26T18:25:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:49 smithi167 ceph-mon[32751]: pgmap v460: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 931 KiB/s wr, 128 op/s 2024-04-26T18:25:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:49 smithi090 ceph-mon[29180]: pgmap v460: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 931 KiB/s wr, 128 op/s 2024-04-26T18:25:50.709 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.75 MB/sec execute 578 sec latency 6749.974 ms 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 -- 172.21.15.90:0/1781144062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560102210 msgr2=0x7f756010e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 --2- 172.21.15.90:0/1781144062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560102210 0x7f756010e720 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7558009cd0 tx=0x7f755802a8e0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 -- 172.21.15.90:0/1781144062 shutdown_connections 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 --2- 172.21.15.90:0/1781144062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560102210 0x7f756010e720 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 --2- 172.21.15.90:0/1781144062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7560101850 0x7f7560101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 --2- 172.21.15.90:0/1781144062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560107860 0x7f7560107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 -- 172.21.15.90:0/1781144062 >> 172.21.15.90:0/1781144062 conn(0x7f7560077bb0 msgr2=0x7f7560077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:51.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 -- 172.21.15.90:0/1781144062 shutdown_connections 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.034+0000 7f75682da640 1 -- 172.21.15.90:0/1781144062 wait complete. 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 Processor -- start 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 -- start start 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560101850 0x7f7560199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7560102210 0x7f756019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560107860 0x7f75601a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7560111b20 con 0x7f7560101850 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f75601119a0 con 0x7f7560107860 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f75682da640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7560111ca0 con 0x7f7560102210 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560101850 0x7f7560199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560101850 0x7f7560199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46184/0 (socket says 172.21.15.90:46184) 2024-04-26T18:25:51.052 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 -- 172.21.15.90:0/318312980 learned_addr learned my addr 172.21.15.90:0/318312980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7565875640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7560102210 0x7f756019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566877640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560107860 0x7f75601a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 -- 172.21.15.90:0/318312980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7560102210 msgr2=0x7f756019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7560102210 0x7f756019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 -- 172.21.15.90:0/318312980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560107860 msgr2=0x7f75601a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560107860 0x7f75601a4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 -- 172.21.15.90:0/318312980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75601a5460 con 0x7f7560101850 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.035+0000 7f7566076640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560101850 0x7f7560199ea0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f755c00e9f0 tx=0x7f755c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.036+0000 7f75577fe640 1 -- 172.21.15.90:0/318312980 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c00cd80 con 0x7f7560101850 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.036+0000 7f75577fe640 1 -- 172.21.15.90:0/318312980 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f755c0108e0 con 0x7f7560101850 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.036+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75601a5630 con 0x7f7560101850 2024-04-26T18:25:51.053 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.036+0000 7f75577fe640 1 -- 172.21.15.90:0/318312980 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c018980 con 0x7f7560101850 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.036+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75601a5b20 con 0x7f7560101850 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.038+0000 7f75577fe640 1 -- 172.21.15.90:0/318312980 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f755c010430 con 0x7f7560101850 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.038+0000 7f756affc740 1 Processor -- start 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f756affc740 1 -- start start 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f756affc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364c1f00 0x55ae364c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f756affc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f756affc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364cd0e0 0x55ae364cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f756affc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ae36428460 con 0x55ae364cd0e0 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f756affc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae363e2590 con 0x55ae364c2820 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f756affc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae36424420 con 0x55ae364c1f00 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f75577fe640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f754c0a3dc0 0x7f754c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.054 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f7565875640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364cd0e0 0x55ae364cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f7565875640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364cd0e0 0x55ae364cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46196/0 (socket says 172.21.15.90:46196) 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f7565875640 1 -- 172.21.15.90:0/355417048 learned_addr learned my addr 172.21.15.90:0/355417048 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f7566877640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f754c0a3dc0 0x7f754c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f75577fe640 1 -- 172.21.15.90:0/318312980 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 10131+0+0 (secure 0 0 0) 0x7f755c0d1180 con 0x7f7560101850 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f7566877640 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364c1f00 0x55ae364c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.039+0000 7f7566076640 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7565875640 1 -- 172.21.15.90:0/355417048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364c1f00 msgr2=0x55ae364c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7565875640 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364c1f00 0x55ae364c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7565875640 1 -- 172.21.15.90:0/355417048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 msgr2=0x55ae364ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7565875640 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7565875640 1 -- 172.21.15.90:0/355417048 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae363ef0d0 con 0x55ae364cd0e0 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7565875640 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364cd0e0 0x55ae364cf4d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f755000b740 tx=0x7f755000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751686fa5f96f442 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.055 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7566877640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f754c0a3dc0 0x7f754c0a6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7558009430 tx=0x7f7558005c30 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7554bfe640 1 -- 172.21.15.90:0/355417048 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550003dd0 con 0x55ae364cd0e0 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7554bfe640 1 -- 172.21.15.90:0/355417048 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550015070 con 0x55ae364cd0e0 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.040+0000 7f7554bfe640 1 -- 172.21.15.90:0/355417048 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7550004e00 con 0x55ae364cd0e0 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- 172.21.15.90:0/355417048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364cd0e0 msgr2=0x55ae364cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364cd0e0 0x55ae364cf4d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f755000b740 tx=0x7f755000bc00 comp rx=0 tx=0).stop 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- 172.21.15.90:0/355417048 shutdown_connections 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364cd0e0 0x55ae364cf4d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 --2- 172.21.15.90:0/355417048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364c1f00 0x55ae364c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- 172.21.15.90:0/355417048 >> 172.21.15.90:0/355417048 conn(0x55ae364be410 msgr2=0x55ae364be7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- 172.21.15.90:0/355417048 shutdown_connections 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- 172.21.15.90:0/355417048 wait complete. 2024-04-26T18:25:51.056 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- start start 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 Processor -- start 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ae364cfd10 con 0x55ae364c1f00 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae363afb60 con 0x55ae364c2820 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae364cfe90 con 0x55ae364cd0e0 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.041+0000 7f756affc740 10 client.0 _ll_register_callbacks cb 0x55ae363d42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46202/0 (socket says 172.21.15.90:46202) 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 -- 172.21.15.90:0/525657627 learned_addr learned my addr 172.21.15.90:0/525657627 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566877640 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.057 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7565875640 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 -- 172.21.15.90:0/525657627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 msgr2=0x55ae364cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 -- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 msgr2=0x55ae364ca5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 -- 172.21.15.90:0/525657627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3613a210 con 0x55ae364c1f00 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7566076640 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364ca0a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f755c0ce280 tx=0x7f755c010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3633ccee2cd80357 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7549ffb640 1 -- 172.21.15.90:0/525657627 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0a0e20 con 0x55ae364c1f00 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7549ffb640 1 -- 172.21.15.90:0/525657627 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 10131+0+0 (secure 0 0 0) 0x7f755c0ce4a0 con 0x55ae364c1f00 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7550015070 con 0x55ae364c1f00 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f7549ffb640 1 -- 172.21.15.90:0/525657627 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7f755c0ce6a0 con 0x55ae364c1f00 2024-04-26T18:25:51.058 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f756affc740 10 client.16488 fetch_fsmap learned FSMap version 150 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.042+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) -- 0x55ae36206620 con 0x55ae364c1f00 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f7549ffb640 1 -- 172.21.15.90:0/525657627 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550015070 con 0x55ae364c1f00 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f7549ffb640 1 -- 172.21.15.90:0/525657627 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 150) ==== 7136+0+0 (secure 0 0 0) 0x7f755c0987c0 con 0x55ae364c1f00 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f756affc740 10 client.16488 fetch_fsmap finished waiting for FSMap version 150 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f756affc740 10 client.16488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f756affc740 20 client.16488 populate_metadata read hostname 'smithi090' 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f756affc740 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae364dd440 0x55ae364df8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f756affc740 4 client.16488 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:25:51.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae36436070 con 0x55ae364dd440 2024-04-26T18:25:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f7565875640 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae364dd440 0x55ae364df8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f7549ffb640 10 client.16488 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.043+0000 7f7565875640 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae364dd440 0x55ae364df8e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1e520490bc5f3e server_cookie=11c322e27d4250d8 in_seq=0 out_seq=0 2024-04-26T18:25:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f7549ffb640 1 -- 172.21.15.90:0/525657627 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae36436070 con 0x55ae364dd440 2024-04-26T18:25:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f7549ffb640 10 client.16488 handle_command_reply: tid=0 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 client.16488 shutdown 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 msgr2=0x55ae364ca0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364ca0a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f755c0ce280 tx=0x7f755c010860 comp rx=0 tx=0).stop 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 shutdown_connections 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae364dd440 0x55ae364df8e0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f7549ffb640 0 client.16488 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364ca5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 --2- 172.21.15.90:0/525657627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364ca0a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 >> 172.21.15.90:0/525657627 conn(0x55ae364be410 msgr2=0x55ae364c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 shutdown_connections 2024-04-26T18:25:51.062 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.044+0000 7f756affc740 1 -- 172.21.15.90:0/525657627 wait complete. 2024-04-26T18:25:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.045+0000 7f756affc740 20 client.16488 trim_cache size 0 max 16384 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 Processor -- start 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 -- start start 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c1f00 0x55ae364d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c2820 0x55ae364c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364e51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75601afe60 con 0x55ae364c2820 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f75601b0530 con 0x55ae364c1f00 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f756affc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75601b0c00 con 0x55ae364cd0e0 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.059+0000 7f7565875640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c2820 0x55ae364c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c2820 0x55ae364c48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46216/0 (socket says 172.21.15.90:46216) 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 -- 172.21.15.90:0/3845590041 learned_addr learned my addr 172.21.15.90:0/3845590041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7566076640 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c1f00 0x55ae364d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7566877640 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364e51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 -- 172.21.15.90:0/3845590041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 msgr2=0x55ae364e51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364e51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 -- 172.21.15.90:0/3845590041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c1f00 msgr2=0x55ae364d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c1f00 0x55ae364d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 -- 172.21.15.90:0/3845590041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae364b89c0 con 0x55ae364c2820 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7566076640 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c1f00 0x55ae364d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f7565875640 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c2820 0x55ae364c48c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f75500039c0 tx=0x7f7550004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375e5de7cb559928 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f754bfff640 1 -- 172.21.15.90:0/3845590041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755002b020 con 0x55ae364c2820 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f754bfff640 1 -- 172.21.15.90:0/3845590041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550015030 con 0x55ae364c2820 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.060+0000 7f754bfff640 1 -- 172.21.15.90:0/3845590041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75500116a0 con 0x55ae364c2820 2024-04-26T18:25:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- 172.21.15.90:0/3845590041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c2820 msgr2=0x55ae364c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c2820 0x55ae364c48c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f75500039c0 tx=0x7f7550004350 comp rx=0 tx=0).stop 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- 172.21.15.90:0/3845590041 shutdown_connections 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364e51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c2820 0x55ae364c48c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 --2- 172.21.15.90:0/3845590041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c1f00 0x55ae364d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- 172.21.15.90:0/3845590041 >> 172.21.15.90:0/3845590041 conn(0x55ae364be410 msgr2=0x55ae364bd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- 172.21.15.90:0/3845590041 shutdown_connections 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- 172.21.15.90:0/3845590041 wait complete. 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- start start 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 Processor -- start 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.070 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ad120 con 0x55ae364c1f00 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ae4f0 con 0x55ae364c2820 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75601ad770 con 0x55ae364cd0e0 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.061+0000 7f756affc740 10 client.0 _ll_register_callbacks cb 0x7f755c098d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364c6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46220/0 (socket says 172.21.15.90:46220) 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 -- 172.21.15.90:0/2786974421 learned_addr learned my addr 172.21.15.90:0/2786974421 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566877640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7565875640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 -- 172.21.15.90:0/2786974421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 msgr2=0x55ae364c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.071 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 -- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 msgr2=0x55ae364c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 -- 172.21.15.90:0/2786974421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae3654b260 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566877640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7566076640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364c6d30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f755c098bb0 tx=0x7f755c09c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=774a02ad5aee5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7549ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7549ffb640 1 -- 172.21.15.90:0/2786974421 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0ce6b0 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7549ffb640 1 -- 172.21.15.90:0/2786974421 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 10131+0+0 (secure 0 0 0) 0x7f755c0a5450 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7550015030 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f7549ffb640 1 -- 172.21.15.90:0/2786974421 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=150) ==== 24+0+0 (secure 0 0 0) 0x7f755c0ce0d0 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f756affc740 10 client.16494 fetch_fsmap learned FSMap version 150 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.062+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=150,monmap=4+}) -- 0x55ae3654c580 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f7549ffb640 1 -- 172.21.15.90:0/2786974421 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7550015030 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f7549ffb640 1 -- 172.21.15.90:0/2786974421 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 150) ==== 7136+0+0 (secure 0 0 0) 0x7f755c0870b0 con 0x55ae364c1f00 2024-04-26T18:25:51.072 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f756affc740 10 client.16494 fetch_fsmap finished waiting for FSMap version 150 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f756affc740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f756affc740 20 client.16494 populate_metadata read hostname 'smithi090' 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f756affc740 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae3654cd10 0x55ae3654f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f756affc740 4 client.16494 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae3642fee0 con 0x55ae3654cd10 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f7565875640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae3654cd10 0x55ae3654f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f7565875640 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae3654cd10 0x55ae3654f130 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=417b609c901b4c81 server_cookie=789ed8d426e1b24 in_seq=0 out_seq=0 2024-04-26T18:25:51.073 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:51.063+0000 7f7549ffb640 10 client.16494 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:51.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.75 MB/sec execute 579 sec latency 7750.104 ms 2024-04-26T18:25:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:51 smithi119 ceph-mon[32752]: pgmap v461: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 913 KiB/s wr, 120 op/s 2024-04-26T18:25:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:51 smithi167 ceph-mon[32751]: pgmap v461: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 913 KiB/s wr, 120 op/s 2024-04-26T18:25:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:51 smithi090 ceph-mon[29180]: pgmap v461: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 913 KiB/s wr, 120 op/s 2024-04-26T18:25:52.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.74 MB/sec execute 580 sec latency 8750.201 ms 2024-04-26T18:25:53.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.74 MB/sec execute 581 sec latency 9750.318 ms 2024-04-26T18:25:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:53 smithi119 ceph-mon[32752]: pgmap v462: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 913 KiB/s wr, 120 op/s 2024-04-26T18:25:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:53 smithi167 ceph-mon[32751]: pgmap v462: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 913 KiB/s wr, 120 op/s 2024-04-26T18:25:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:53 smithi090 ceph-mon[29180]: pgmap v462: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 913 KiB/s wr, 120 op/s 2024-04-26T18:25:54.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.73 MB/sec execute 582 sec latency 10750.352 ms 2024-04-26T18:25:55.322 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:25:55.322 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 systemd[1]: Stopped Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:55.322 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 4.316s CPU time. 2024-04-26T18:25:55.323 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 systemd[1]: Starting Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:25:55.323 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 podman[70919]: 2024-04-26 18:25:55.190581044 +0000 UTC m=+0.018156184 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:25:55.323 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 podman[70919]: 2024-04-26 18:25:55.323294047 +0000 UTC m=+0.150869187 container create a551ac7f8a83746f4de4dd5eb12f71b403284fe650d05dd7542776249c20c720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:25:55.443 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:25:55.444 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 systemd[1]: Stopped Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:55.444 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 8.838s CPU time. 2024-04-26T18:25:55.444 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 systemd[1]: Starting Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:25:55.444 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 podman[71375]: 2024-04-26 18:25:55.312205347 +0000 UTC m=+0.115383667 container create efc0861150d09f9c0263965475c7e56f0f90cbb1c34e885b9e269ec1d00163ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:25:55.444 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 podman[71375]: 2024-04-26 18:25:55.215103016 +0000 UTC m=+0.018281298 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:25:55.582 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 podman[70919]: 2024-04-26 18:25:55.559373928 +0000 UTC m=+0.386949073 container init a551ac7f8a83746f4de4dd5eb12f71b403284fe650d05dd7542776249c20c720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-26T18:25:55.583 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 podman[70919]: 2024-04-26 18:25:55.561912872 +0000 UTC m=+0.389488012 container start a551ac7f8a83746f4de4dd5eb12f71b403284fe650d05dd7542776249c20c720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T18:25:55.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.73 MB/sec execute 583 sec latency 11750.447 ms 2024-04-26T18:25:55.817 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 podman[71375]: 2024-04-26 18:25:55.557260545 +0000 UTC m=+0.360438820 container init efc0861150d09f9c0263965475c7e56f0f90cbb1c34e885b9e269ec1d00163ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:25:55.817 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 podman[71375]: 2024-04-26 18:25:55.559879386 +0000 UTC m=+0.363057665 container start efc0861150d09f9c0263965475c7e56f0f90cbb1c34e885b9e269ec1d00163ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:25:55.818 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mds[71395]: -- 172.21.15.167:0/3683629827 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c003b5a40 con 0x556c00389000 2024-04-26T18:25:55.818 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: starting mds.l at 2024-04-26T18:25:55.818 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 bash[71375]: efc0861150d09f9c0263965475c7e56f0f90cbb1c34e885b9e269ec1d00163ba 2024-04-26T18:25:55.818 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:25:55 smithi167 systemd[1]: Started Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:55.937 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mds[70938]: -- 172.21.15.119:0/4191693037 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dbadfe9a40 con 0x55dbadfbc800 2024-04-26T18:25:55.937 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: starting mds.k at 2024-04-26T18:25:55.937 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 bash[70919]: a551ac7f8a83746f4de4dd5eb12f71b403284fe650d05dd7542776249c20c720 2024-04-26T18:25:55.937 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:25:55 smithi119 systemd[1]: Started Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: pgmap v463: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 931 KiB/s wr, 141 op/s 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: daemon mds.l restarted 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: osdmap e130: 12 total, 12 up, 12 in 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:boot 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: daemon mds.k restarted 2024-04-26T18:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: osdmap e131: 12 total, 12 up, 12 in 2024-04-26T18:25:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:boot 2024-04-26T18:25:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:25:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:replay} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:25:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:25:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:55 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: pgmap v463: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 931 KiB/s wr, 141 op/s 2024-04-26T18:25:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: daemon mds.l restarted 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: osdmap e130: 12 total, 12 up, 12 in 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:boot 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: daemon mds.k restarted 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: osdmap e131: 12 total, 12 up, 12 in 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:boot 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:replay} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:55 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: pgmap v463: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 931 KiB/s wr, 141 op/s 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: daemon mds.l restarted 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: osdmap e130: 12 total, 12 up, 12 in 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:boot 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: daemon mds.k restarted 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: osdmap e131: 12 total, 12 up, 12 in 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:boot 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:replay} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:25:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:25:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:55 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:replay} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:56.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.72 MB/sec execute 584 sec latency 12750.536 ms 2024-04-26T18:25:57.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.72 MB/sec execute 585 sec latency 13750.623 ms 2024-04-26T18:25:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:57 smithi119 ceph-mon[32752]: pgmap v466: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 31 op/s 2024-04-26T18:25:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:57 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:resolve 2024-04-26T18:25:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:57 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:resolve 2024-04-26T18:25:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:57 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:57 smithi167 ceph-mon[32751]: pgmap v466: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 31 op/s 2024-04-26T18:25:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:57 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:resolve 2024-04-26T18:25:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:57 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:resolve 2024-04-26T18:25:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:57 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:57 smithi090 ceph-mon[29180]: pgmap v466: 97 pgs: 97 active+clean; 166 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 27 KiB/s wr, 31 op/s 2024-04-26T18:25:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:57 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:resolve 2024-04-26T18:25:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:57 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:resolve 2024-04-26T18:25:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:57 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:58.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525436 2.71 MB/sec execute 586 sec latency 14750.681 ms 2024-04-26T18:25:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.996+0000 7f7549ffb640 1 -- 172.21.15.90:0/2786974421 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae3642fee0 con 0x55ae3654cd10 2024-04-26T18:25:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.996+0000 7f7549ffb640 10 client.16494 handle_command_reply: tid=0 2024-04-26T18:25:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.996+0000 7f756affc740 1 client.16494 shutdown 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 msgr2=0x55ae364c6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364c6d30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f755c098bb0 tx=0x7f755c09c450 comp rx=0 tx=0).stop 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 shutdown_connections 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae3654cd10 0x55ae3654f130 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae364cd0e0 0x55ae364c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae364c2820 0x55ae364c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 --2- 172.21.15.90:0/2786974421 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae364c1f00 0x55ae364c6d30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f7549ffb640 0 client.16494 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 >> 172.21.15.90:0/2786974421 conn(0x55ae364be410 msgr2=0x55ae364ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 shutdown_connections 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 1 -- 172.21.15.90:0/2786974421 wait complete. 2024-04-26T18:25:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f756affc740 20 client.16494 trim_cache size 0 max 16384 2024-04-26T18:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:25:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f754c0a3dc0 msgr2=0x7f754c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f75682da640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f754c0a3dc0 0x7f754c0a6280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7558009430 tx=0x7f7558005c30 comp rx=0 tx=0).stop 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.997+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560101850 msgr2=0x7f7560199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560101850 0x7f7560199ea0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f755c00e9f0 tx=0x7f755c00eeb0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 shutdown_connections 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f754c0a3dc0 0x7f754c0a6280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7560107860 0x7f75601a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7560102210 0x7f756019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 --2- 172.21.15.90:0/318312980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7560101850 0x7f7560199ea0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 >> 172.21.15.90:0/318312980 conn(0x7f7560077bb0 msgr2=0x7f75600ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 shutdown_connections 2024-04-26T18:25:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:25:58.998+0000 7f75682da640 1 -- 172.21.15.90:0/318312980 wait complete. 2024-04-26T18:25:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:58 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:25:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:58 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:reconnect 2024-04-26T18:25:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:58 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:reconnect 2024-04-26T18:25:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:58 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:58 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:25:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:58 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:25:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:58 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:25:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:58 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:reconnect 2024-04-26T18:25:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:58 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:reconnect 2024-04-26T18:25:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:58 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:58 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:25:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:58 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:25:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:58 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:25:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:58 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:reconnect 2024-04-26T18:25:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:58 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:reconnect 2024-04-26T18:25:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:58 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:25:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:58 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00299993 2024-04-26T18:25:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:58 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199996 2024-04-26T18:25:59.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525444 2.71 MB/sec execute 587 sec latency 15004.986 ms 2024-04-26T18:26:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:59 smithi119 ceph-mon[32752]: pgmap v467: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-26T18:26:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:rejoin 2024-04-26T18:26:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:rejoin 2024-04-26T18:26:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:59 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:25:59 smithi119 ceph-mon[32752]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:26:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:59 smithi167 ceph-mon[32751]: pgmap v467: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-26T18:26:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:rejoin 2024-04-26T18:26:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:rejoin 2024-04-26T18:26:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:59 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:25:59 smithi167 ceph-mon[32751]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:26:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:59 smithi090 ceph-mon[29180]: pgmap v467: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-26T18:26:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:rejoin 2024-04-26T18:26:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:rejoin 2024-04-26T18:26:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:59 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:25:59 smithi090 ceph-mon[29180]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:26:00.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525444 2.71 MB/sec execute 588 sec latency 1745.865 ms 2024-04-26T18:26:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:00 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:active 2024-04-26T18:26:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:00 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:clientreplay 2024-04-26T18:26:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:00 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:clientreplay} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:00 smithi119 ceph-mon[32752]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:26:01.317 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:26:01.003+0000 7f61a8a97700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 8 2024-04-26T18:26:01.318 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:26:01.003+0000 7f61aaa9b700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 9 2024-04-26T18:26:01.318 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:26:01.005+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 16 2024-04-26T18:26:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:00 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:active 2024-04-26T18:26:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:00 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:clientreplay 2024-04-26T18:26:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:00 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:clientreplay} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:00 smithi167 ceph-mon[32751]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:00 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3224484301,v1:172.21.15.119:6835/3224484301] up:active 2024-04-26T18:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:00 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:clientreplay 2024-04-26T18:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:00 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:clientreplay} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:00 smithi090 ceph-mon[29180]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:26:01.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 525978 2.71 MB/sec execute 589 sec latency 2015.658 ms 2024-04-26T18:26:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:01 smithi119 ceph-mon[32752]: pgmap v468: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-26T18:26:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:01 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:01 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:active 2024-04-26T18:26:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:01 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:01 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-mon[32751]: pgmap v468: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-26T18:26:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:active 2024-04-26T18:26:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:01 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:01 smithi090 ceph-mon[29180]: pgmap v468: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 1.1 MiB/s wr, 63 op/s 2024-04-26T18:26:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:01 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:01 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/4250199742,v1:172.21.15.167:6837/4250199742] up:active 2024-04-26T18:26:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:01 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:01 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:02.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 527015 2.71 MB/sec execute 590 sec latency 14.358 ms 2024-04-26T18:26:03.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:26:03 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:26:03.237+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:26:03.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 529517 2.73 MB/sec execute 591 sec latency 16.031 ms 2024-04-26T18:26:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:03 smithi119 ceph-mon[32752]: pgmap v469: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-26T18:26:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:03 smithi167 ceph-mon[32751]: pgmap v469: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-26T18:26:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:03 smithi090 ceph-mon[29180]: pgmap v469: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 1.1 MiB/s wr, 32 op/s 2024-04-26T18:26:04.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 530916 2.74 MB/sec execute 592 sec latency 22.792 ms 2024-04-26T18:26:05.710 INFO:tasks.workunit.client.0.smithi090.stdout: 1 532829 2.75 MB/sec execute 593 sec latency 285.920 ms 2024-04-26T18:26:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:05 smithi119 ceph-mon[32752]: pgmap v470: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-04-26T18:26:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:05 smithi119 ceph-mon[32752]: osdmap e132: 12 total, 12 up, 12 in 2024-04-26T18:26:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:05 smithi167 ceph-mon[32751]: pgmap v470: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-04-26T18:26:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:05 smithi167 ceph-mon[32751]: osdmap e132: 12 total, 12 up, 12 in 2024-04-26T18:26:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:05 smithi090 ceph-mon[29180]: pgmap v470: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 164 op/s 2024-04-26T18:26:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:05 smithi090 ceph-mon[29180]: osdmap e132: 12 total, 12 up, 12 in 2024-04-26T18:26:06.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 532829 2.75 MB/sec execute 594 sec latency 1285.995 ms 2024-04-26T18:26:07.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 532829 2.74 MB/sec execute 595 sec latency 2286.137 ms 2024-04-26T18:26:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:07 smithi119 ceph-mon[32752]: pgmap v472: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-04-26T18:26:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:07 smithi167 ceph-mon[32751]: pgmap v472: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-04-26T18:26:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:07 smithi090 ceph-mon[29180]: pgmap v472: 97 pgs: 97 active+clean; 169 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-04-26T18:26:08.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 532829 2.74 MB/sec execute 596 sec latency 3286.289 ms 2024-04-26T18:26:09.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 532829 2.74 MB/sec execute 597 sec latency 4286.417 ms 2024-04-26T18:26:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:09 smithi119 ceph-mon[32752]: pgmap v473: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:09 smithi167 ceph-mon[32751]: pgmap v473: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:09 smithi090 ceph-mon[29180]: pgmap v473: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:10.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 532829 2.73 MB/sec execute 598 sec latency 5286.552 ms 2024-04-26T18:26:11.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 532829 2.73 MB/sec execute 599 sec latency 6286.632 ms 2024-04-26T18:26:12.094 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/1438889818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74102210 msgr2=0x7f1d7410e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/1438889818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74102210 0x7f1d7410e720 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009930 tx=0x7f1d6c02a420 comp rx=0 tx=0).stop 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/1438889818 shutdown_connections 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/1438889818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74102210 0x7f1d7410e720 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/1438889818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d74101850 0x7f1d74101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/1438889818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d74107860 0x7f1d74107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/1438889818 >> 172.21.15.90:0/1438889818 conn(0x7f1d74077bb0 msgr2=0x7f1d74077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/1438889818 shutdown_connections 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.094+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/1438889818 wait complete. 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 Processor -- start 2024-04-26T18:26:12.095 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 -- start start 2024-04-26T18:26:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74101850 0x7f1d74199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d74102210 0x7f1d74199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d74107860 0x7f1d7419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74112c20 con 0x7f1d74101850 2024-04-26T18:26:12.097 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74112aa0 con 0x7f1d74102210 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d7a1c6640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74112da0 con 0x7f1d74107860 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d737fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74101850 0x7f1d74199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d737fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74101850 0x7f1d74199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39774/0 (socket says 172.21.15.90:39774) 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d737fe640 1 -- 172.21.15.90:0/2574753205 learned_addr learned my addr 172.21.15.90:0/2574753205 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d74102210 0x7f1d74199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.095+0000 7f1d73fff640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d74107860 0x7f1d7419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d737fe640 1 -- 172.21.15.90:0/2574753205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d74107860 msgr2=0x7f1d7419d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d737fe640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d74107860 0x7f1d7419d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d737fe640 1 -- 172.21.15.90:0/2574753205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d74102210 msgr2=0x7f1d74199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d737fe640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d74102210 0x7f1d74199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d737fe640 1 -- 172.21.15.90:0/2574753205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7419e030 con 0x7f1d74101850 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d737fe640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74101850 0x7f1d74199040 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400cce0 tx=0x7f1d6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.098 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d70ff9640 1 -- 172.21.15.90:0/2574753205 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64004b20 con 0x7f1d74101850 2024-04-26T18:26:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7419e320 con 0x7f1d74101850 2024-04-26T18:26:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d70ff9640 1 -- 172.21.15.90:0/2574753205 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d64004cc0 con 0x7f1d74101850 2024-04-26T18:26:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d70ff9640 1 -- 172.21.15.90:0/2574753205 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400f6b0 con 0x7f1d74101850 2024-04-26T18:26:12.099 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.096+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d741a5bb0 con 0x7f1d74101850 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.098+0000 7f1d70ff9640 1 -- 172.21.15.90:0/2574753205 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f1d6400f850 con 0x7f1d74101850 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.098+0000 7f1d7cfbe740 1 Processor -- start 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d7cfbe740 1 -- start start 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1297a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f1297a820 0x563f12984ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f129874d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563f128e0460 con 0x563f129850e0 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563f1289a590 con 0x563f12979f00 2024-04-26T18:26:12.105 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f128dc420 con 0x563f1297a820 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f129874d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d70ff9640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1d5c0a3dc0 0x7f1d5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f129874d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39788/0 (socket says 172.21.15.90:39788) 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2819738337 learned_addr learned my addr 172.21.15.90:0/2819738337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d73fff640 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1297a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d73fff640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1d5c0a3dc0 0x7f1d5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d737fe640 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f1297a820 0x563f12984ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2819738337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f1297a820 msgr2=0x563f12984ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d70ff9640 1 -- 172.21.15.90:0/2574753205 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 10391+0+0 (secure 0 0 0) 0x7f1d640d0ee0 con 0x7f1d74101850 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f1297a820 0x563f12984ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2819738337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 msgr2=0x563f1297a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.106 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1297a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2819738337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f128a70d0 con 0x563f129850e0 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d73fff640 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1297a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.099+0000 7f1d737fe640 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f1297a820 0x563f12984ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f129874d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b740 tx=0x7f1d6800bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57451e7c9895e59a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d73fff640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1d5c0a3dc0 0x7f1d5c0a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c005ab0 tx=0x7f1d6c0058d0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d59ffb640 1 -- 172.21.15.90:0/2819738337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68003dd0 con 0x563f129850e0 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d59ffb640 1 -- 172.21.15.90:0/2819738337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d68015070 con 0x563f129850e0 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d59ffb640 1 -- 172.21.15.90:0/2819738337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68004d90 con 0x563f129850e0 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2819738337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 msgr2=0x563f129874d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f129874d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800b740 tx=0x7f1d6800bc00 comp rx=0 tx=0).stop 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2819738337 shutdown_connections 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f129874d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f1297a820 0x563f12984ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2819738337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1297a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.107 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.100+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2819738337 >> 172.21.15.90:0/2819738337 conn(0x563f12976410 msgr2=0x563f129767f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2819738337 shutdown_connections 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2819738337 wait complete. 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 -- start start 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 Processor -- start 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f129820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f1297a820 0x563f12982600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f12982b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563f12987d10 con 0x563f129850e0 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563f12867b60 con 0x563f1297a820 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563f12987e90 con 0x563f12979f00 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d7cfbe740 10 client.0 _ll_register_callbacks cb 0x563f1288c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.101+0000 7f1d73fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f12982b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f12982b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39792/0 (socket says 172.21.15.90:39792) 2024-04-26T18:26:12.108 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d737fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f129820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 -- 172.21.15.90:0/3462067690 learned_addr learned my addr 172.21.15.90:0/3462067690 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f1297a820 0x563f12982600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 -- 172.21.15.90:0/3462067690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 msgr2=0x563f129820c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f129820c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 -- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f1297a820 msgr2=0x563f12982600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f1297a820 0x563f12982600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 -- 172.21.15.90:0/3462067690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f125f2210 con 0x563f129850e0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d73fff640 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f12982b40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009930 tx=0x7f1d6c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849f25bf4653f9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d52ffd640 1 -- 172.21.15.90:0/3462067690 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c04d030 con 0x563f129850e0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d52ffd640 1 -- 172.21.15.90:0/3462067690 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 10391+0+0 (secure 0 0 0) 0x7f1d6c003980 con 0x563f129850e0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d68015070 con 0x563f129850e0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d52ffd640 1 -- 172.21.15.90:0/3462067690 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f1d6c057050 con 0x563f129850e0 2024-04-26T18:26:12.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d7cfbe740 10 client.16509 fetch_fsmap learned FSMap version 160 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.102+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) -- 0x563f126be620 con 0x563f129850e0 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d52ffd640 1 -- 172.21.15.90:0/3462067690 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d68015070 con 0x563f129850e0 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d52ffd640 1 -- 172.21.15.90:0/3462067690 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 160) ==== 7136+0+0 (secure 0 0 0) 0x7f1d6c052470 con 0x563f129850e0 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d7cfbe740 10 client.16509 fetch_fsmap finished waiting for FSMap version 160 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d7cfbe740 10 client.16509 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d7cfbe740 20 client.16509 populate_metadata read hostname 'smithi090' 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f12995230 0x563f12997680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d7cfbe740 4 client.16509 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f128ee070 con 0x563f12995230 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d737fe640 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f12995230 0x563f12997680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d737fe640 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f12995230 0x563f12997680 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50ffa86b3d170b42 server_cookie=e17e319ec1e22cc0 in_seq=0 out_seq=0 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.103+0000 7f1d52ffd640 10 client.16509 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:12.110 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d52ffd640 1 -- 172.21.15.90:0/3462067690 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563f128ee070 con 0x563f12995230 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d52ffd640 10 client.16509 handle_command_reply: tid=0 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 client.16509 shutdown 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 msgr2=0x563f12982b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f12982b40 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c009930 tx=0x7f1d6c009260 comp rx=0 tx=0).stop 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 shutdown_connections 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f12995230 0x563f12997680 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f129850e0 0x563f12982b40 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f1297a820 0x563f12982600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d52ffd640 0 client.16509 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/3462067690 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f129820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.104+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 >> 172.21.15.90:0/3462067690 conn(0x563f12976410 msgr2=0x563f1297c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.105+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 shutdown_connections 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.105+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/3462067690 wait complete. 2024-04-26T18:26:12.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.105+0000 7f1d7cfbe740 20 client.16509 trim_cache size 0 max 16384 2024-04-26T18:26:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.119+0000 7f1d7cfbe740 1 Processor -- start 2024-04-26T18:26:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.119+0000 7f1d7cfbe740 1 -- start start 2024-04-26T18:26:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.119+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f12990bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.119+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1297c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f129850e0 0x563f12984000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b0580 con 0x563f1297a820 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b0c50 con 0x563f129850e0 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741b1320 con 0x563f12979f00 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1297c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1297c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39808/0 (socket says 172.21.15.90:39808) 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2258735908 learned_addr learned my addr 172.21.15.90:0/2258735908 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d737fe640 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f12990bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2258735908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 msgr2=0x563f12990bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f12990bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2258735908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f129850e0 msgr2=0x563f12984000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f129850e0 0x563f12984000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 -- 172.21.15.90:0/2258735908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f12980650 con 0x563f1297a820 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1297c8c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1d680039c0 tx=0x7f1d68004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340bf288a890ada4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d597fa640 1 -- 172.21.15.90:0/2258735908 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68003dd0 con 0x563f1297a820 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d597fa640 1 -- 172.21.15.90:0/2258735908 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d68029070 con 0x563f1297a820 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.120+0000 7f1d597fa640 1 -- 172.21.15.90:0/2258735908 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d680116a0 con 0x563f1297a820 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2258735908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 msgr2=0x563f1297c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1297c8c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1d680039c0 tx=0x7f1d68004680 comp rx=0 tx=0).stop 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2258735908 shutdown_connections 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f129850e0 0x563f12984000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1297c8c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/2258735908 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f12979f00 0x563f12990bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2258735908 >> 172.21.15.90:0/2258735908 conn(0x563f12976410 msgr2=0x563f129838f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2258735908 shutdown_connections 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/2258735908 wait complete. 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 -- start start 2024-04-26T18:26:12.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.121+0000 7f1d7cfbe740 1 Processor -- start 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1298d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1298d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f129850e0 0x563f1298a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741ad820 con 0x563f1297a820 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741aec10 con 0x563f12979f00 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741ade70 con 0x563f129850e0 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 10 client.0 _ll_register_callbacks cb 0x7f1d6c031720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1298d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1298d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:39816/0 (socket says 172.21.15.90:39816) 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 -- 172.21.15.90:0/14216955 learned_addr learned my addr 172.21.15.90:0/14216955 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d737fe640 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1298d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d73fff640 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f129850e0 0x563f1298a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 -- 172.21.15.90:0/14216955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f129850e0 msgr2=0x563f1298a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f129850e0 0x563f1298a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 -- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 msgr2=0x563f1298d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1298d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 -- 172.21.15.90:0/14216955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f1298a5f0 con 0x563f1297a820 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d72ffd640 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1298d620 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1d68000c00 tx=0x7f1d68003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7cf8fe1b8a91e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d52ffd640 1 -- 172.21.15.90:0/14216955 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d68032070 con 0x563f1297a820 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d52ffd640 1 -- 172.21.15.90:0/14216955 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 10391+0+0 (secure 0 0 0) 0x7f1d68024880 con 0x563f1297a820 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.122+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d68029070 con 0x563f1297a820 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d52ffd640 1 -- 172.21.15.90:0/14216955 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f1d6803e050 con 0x563f1297a820 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 10 client.16515 fetch_fsmap learned FSMap version 160 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) -- 0x563f129fc060 con 0x563f1297a820 2024-04-26T18:26:12.128 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d52ffd640 1 -- 172.21.15.90:0/14216955 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d68029070 con 0x563f1297a820 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d52ffd640 1 -- 172.21.15.90:0/14216955 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 160) ==== 7136+0+0 (secure 0 0 0) 0x7f1d68011b50 con 0x563f1297a820 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 10 client.16515 fetch_fsmap finished waiting for FSMap version 160 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 20 client.16515 populate_metadata read hostname 'smithi090' 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f129fc680 0x563f129feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 4 client.16515 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563f128e7ee0 con 0x563f129fc680 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.123+0000 7f1d737fe640 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f129fc680 0x563f129feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.124+0000 7f1d52ffd640 10 client.16515 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.124+0000 7f1d737fe640 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f129fc680 0x563f129feaa0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb7617cc1e535418 server_cookie=4018113c57c7f7af in_seq=0 out_seq=0 2024-04-26T18:26:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d52ffd640 1 -- 172.21.15.90:0/14216955 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563f128e7ee0 con 0x563f129fc680 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d52ffd640 10 client.16515 handle_command_reply: tid=0 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 client.16515 shutdown 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 msgr2=0x563f1298d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1298d620 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1d68000c00 tx=0x7f1d68003e80 comp rx=0 tx=0).stop 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 shutdown_connections 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563f129fc680 0x563f129feaa0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563f129850e0 0x563f1298a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563f1297a820 0x563f1298d620 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d52ffd640 0 client.16515 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 --2- 172.21.15.90:0/14216955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563f12979f00 0x563f1298d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 >> 172.21.15.90:0/14216955 conn(0x563f12976410 msgr2=0x563f12984180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 shutdown_connections 2024-04-26T18:26:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 1 -- 172.21.15.90:0/14216955 wait complete. 2024-04-26T18:26:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.159+0000 7f1d7cfbe740 20 client.16515 trim_cache size 0 max 16384 2024-04-26T18:26:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:26:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:26:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:26:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:26:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1d5c0a3dc0 msgr2=0x7f1d5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1d5c0a3dc0 0x7f1d5c0a6280 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c005ab0 tx=0x7f1d6c0058d0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74101850 msgr2=0x7f1d74199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74101850 0x7f1d74199040 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400cce0 tx=0x7f1d6400a620 comp rx=0 tx=0).stop 2024-04-26T18:26:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 shutdown_connections 2024-04-26T18:26:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f1d5c0a3dc0 0x7f1d5c0a6280 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1d74107860 0x7f1d7419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d74102210 0x7f1d74199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 --2- 172.21.15.90:0/2574753205 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d74101850 0x7f1d74199040 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 >> 172.21.15.90:0/2574753205 conn(0x7f1d74077bb0 msgr2=0x7f1d740ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 shutdown_connections 2024-04-26T18:26:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:12.160+0000 7f1d7a1c6640 1 -- 172.21.15.90:0/2574753205 wait complete. 2024-04-26T18:26:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:11 smithi119 ceph-mon[32752]: pgmap v474: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:11 smithi167 ceph-mon[32751]: pgmap v474: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:11 smithi090 ceph-mon[29180]: pgmap v474: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:12.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 600 sec 2024-04-26T18:26:13.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 601 sec 2024-04-26T18:26:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:13 smithi090 ceph-mon[29180]: pgmap v475: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:13 smithi090 ceph-mon[29180]: osdmap e133: 12 total, 12 up, 12 in 2024-04-26T18:26:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:13 smithi119 ceph-mon[32752]: pgmap v475: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:13 smithi119 ceph-mon[32752]: osdmap e133: 12 total, 12 up, 12 in 2024-04-26T18:26:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:13 smithi167 ceph-mon[32751]: pgmap v475: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.2 MiB/s wr, 179 op/s 2024-04-26T18:26:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:13 smithi167 ceph-mon[32751]: osdmap e133: 12 total, 12 up, 12 in 2024-04-26T18:26:14.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 602 sec 2024-04-26T18:26:15.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 603 sec 2024-04-26T18:26:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:15 smithi090 ceph-mon[29180]: pgmap v477: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 84 op/s 2024-04-26T18:26:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:15 smithi119 ceph-mon[32752]: pgmap v477: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 84 op/s 2024-04-26T18:26:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:15 smithi167 ceph-mon[32751]: pgmap v477: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 604 KiB/s wr, 84 op/s 2024-04-26T18:26:16.711 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 604 sec 2024-04-26T18:26:17.712 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 605 sec 2024-04-26T18:26:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:17 smithi090 ceph-mon[29180]: pgmap v478: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 575 KiB/s wr, 80 op/s 2024-04-26T18:26:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:26:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:17 smithi119 ceph-mon[32752]: pgmap v478: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 575 KiB/s wr, 80 op/s 2024-04-26T18:26:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:26:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:17 smithi167 ceph-mon[32751]: pgmap v478: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 575 KiB/s wr, 80 op/s 2024-04-26T18:26:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:26:18.712 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 606 sec 2024-04-26T18:26:18.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-mon[32752]: pgmap v479: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-26T18:26:18.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-mon[32752]: pgmap v480: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f61e1fc5700 time 2024-04-26T18:26:18.825521+0000 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f61ec5ee3dd] 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:18.846 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: clone() 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.826+0000 7f61e1fc5700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f61e1fc5700 time 2024-04-26T18:26:18.825521+0000 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:18.847 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.848 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:18.848 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f61ec5ee3dd] 2024-04-26T18:26:18.848 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:18.848 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:18.848 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:18.848 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:18.848 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: clone() 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: *** Caught signal (Aborted) ** 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: in thread 7f61e1fc5700 thread_name:ms_dispatch 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f61eb376d20] 2024-04-26T18:26:18.849 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: gsignal() 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: abort() 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f61ec5ee437] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:18.850 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 16: clone() 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.827+0000 7f61e1fc5700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: in thread 7f61e1fc5700 thread_name:ms_dispatch 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f61eb376d20] 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: gsignal() 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: abort() 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f61ec5ee437] 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:18.851 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 16: clone() 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: -2> 2024-04-26T18:26:18.826+0000 7f61e1fc5700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f61e1fc5700 time 2024-04-26T18:26:18.825521+0000 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:18.852 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f61ec5ee3dd] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:18.853 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: clone() 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 0> 2024-04-26T18:26:18.827+0000 7f61e1fc5700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: in thread 7f61e1fc5700 thread_name:ms_dispatch 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f61eb376d20] 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: gsignal() 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: abort() 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f61ec5ee437] 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:18.854 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 16: clone() 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:18.855 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:18.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:18 smithi090 ceph-mon[29180]: pgmap v479: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-26T18:26:18.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:18 smithi090 ceph-mon[29180]: pgmap v480: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-04-26T18:26:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:18 smithi167 ceph-mon[32751]: pgmap v479: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 24 op/s 2024-04-26T18:26:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:18 smithi167 ceph-mon[32751]: pgmap v480: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-04-26T18:26:19.204 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: -2> 2024-04-26T18:26:18.826+0000 7f61e1fc5700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f61e1fc5700 time 2024-04-26T18:26:18.825521+0000 2024-04-26T18:26:19.204 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f61ec5ee3dd] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:19.205 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: clone() 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 0> 2024-04-26T18:26:18.827+0000 7f61e1fc5700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: in thread 7f61e1fc5700 thread_name:ms_dispatch 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f61eb376d20] 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2: gsignal() 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 3: abort() 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f61ec5ee437] 2024-04-26T18:26:19.206 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f61ec5ee5a3] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d6906a2822] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d690612598] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d690614c17] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d69050dd0b] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d69050e2ac] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d6904f6fad] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f61ec882e20] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 13: (DispatchQueue::entry()+0x51f) [0x7f61ec87ffdf] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f61ec952fc1] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f61eb36c1ca] 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 16: clone() 2024-04-26T18:26:19.207 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:19.208 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[70778]: 2024-04-26T18:26:19.208 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:18 smithi119 podman[71108]: 2024-04-26 18:26:18.892823519 +0000 UTC m=+0.019952543 container died 121daece8543862cd3d2bb5608ceb7d47922413d51bf3d38ca9bd08dffc41226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True) 2024-04-26T18:26:19.712 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 607 sec 2024-04-26T18:26:19.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:19 smithi119 podman[71108]: 2024-04-26 18:26:19.530246445 +0000 UTC m=+0.657375470 container remove 121daece8543862cd3d2bb5608ceb7d47922413d51bf3d38ca9bd08dffc41226 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T18:26:19.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:19 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:26:19.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:19 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-26T18:26:19.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:19 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.289s CPU time. 2024-04-26T18:26:20.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc7dc1cf700 time 2024-04-26T18:26:19.890792+0000 2024-04-26T18:26:20.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:20.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc7e67f83dd] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: clone() 2024-04-26T18:26:20.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:19.891+0000 7fc7dc1cf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc7dc1cf700 time 2024-04-26T18:26:19.890792+0000 2024-04-26T18:26:20.320 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:20.320 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc7e67f83dd] 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.321 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: clone() 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: *** Caught signal (Aborted) ** 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: in thread 7fc7dc1cf700 thread_name:ms_dispatch 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc7e5580d20] 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: gsignal() 2024-04-26T18:26:20.322 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: abort() 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc7e67f8437] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 16: clone() 2024-04-26T18:26:20.323 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:19.893+0000 7fc7dc1cf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: in thread 7fc7dc1cf700 thread_name:ms_dispatch 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc7e5580d20] 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: gsignal() 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: abort() 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc7e67f8437] 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.324 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 16: clone() 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: -1> 2024-04-26T18:26:19.891+0000 7fc7dc1cf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc7dc1cf700 time 2024-04-26T18:26:19.890792+0000 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.325 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc7e67f83dd] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.326 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: clone() 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 0> 2024-04-26T18:26:19.893+0000 7fc7dc1cf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: in thread 7fc7dc1cf700 thread_name:ms_dispatch 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc7e5580d20] 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: gsignal() 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: abort() 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc7e67f8437] 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.327 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 16: clone() 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.328 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: -2> 2024-04-26T18:26:19.891+0000 7fc7dc1cf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc7dc1cf700 time 2024-04-26T18:26:19.890792+0000 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc7e67f83dd] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.329 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: clone() 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: -1> 2024-04-26T18:26:19.893+0000 7fc7dc1cf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: in thread 7fc7dc1cf700 thread_name:ms_dispatch 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc7e5580d20] 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2: gsignal() 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 3: abort() 2024-04-26T18:26:20.330 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc7e67f8437] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc7e67f85a3] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x56463c766822] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x56463c6d6598] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x56463c6d8c17] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56463c5d1d0b] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56463c5d22ac] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x56463c5bafad] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc7e6a8ce20] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 13: (DispatchQueue::entry()+0x51f) [0x7fc7e6a89fdf] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc7e6b5cfc1] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc7e55761ca] 2024-04-26T18:26:20.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 16: clone() 2024-04-26T18:26:20.332 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:20.332 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71232]: 2024-04-26T18:26:20.332 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:19 smithi167 podman[71575]: 2024-04-26 18:26:19.955735468 +0000 UTC m=+0.020686001 container died 41d396362e4850f8f571552c2beb6fbd8aabec2e5fc47667df07ba0e40f457e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:26:20.714 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 608 sec 2024-04-26T18:26:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:20 smithi090 ceph-mon[29180]: pgmap v481: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 221 KiB/s wr, 42 op/s 2024-04-26T18:26:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:20 smithi119 ceph-mon[32752]: pgmap v481: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 221 KiB/s wr, 42 op/s 2024-04-26T18:26:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:20 smithi167 ceph-mon[32751]: pgmap v481: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 221 KiB/s wr, 42 op/s 2024-04-26T18:26:21.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:21 smithi167 podman[71575]: 2024-04-26 18:26:21.09690044 +0000 UTC m=+1.161850975 container remove 41d396362e4850f8f571552c2beb6fbd8aabec2e5fc47667df07ba0e40f457e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T18:26:21.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:21 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:26:21.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:21 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-26T18:26:21.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:21 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 8.742s CPU time. 2024-04-26T18:26:21.712 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 609 sec 2024-04-26T18:26:22.712 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 610 sec 2024-04-26T18:26:22.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:22 smithi119 ceph-mon[32752]: pgmap v482: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 172 KiB/s wr, 33 op/s 2024-04-26T18:26:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:22 smithi090 ceph-mon[29180]: pgmap v482: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 172 KiB/s wr, 33 op/s 2024-04-26T18:26:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:22 smithi167 ceph-mon[32751]: pgmap v482: 97 pgs: 97 active+clean; 168 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 172 KiB/s wr, 33 op/s 2024-04-26T18:26:23.712 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 611 sec 2024-04-26T18:26:24.712 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 612 sec 2024-04-26T18:26:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:24 smithi090 ceph-mon[29180]: pgmap v483: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 644 KiB/s wr, 24 op/s 2024-04-26T18:26:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:24 smithi119 ceph-mon[32752]: pgmap v483: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 644 KiB/s wr, 24 op/s 2024-04-26T18:26:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:24 smithi167 ceph-mon[32751]: pgmap v483: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 644 KiB/s wr, 24 op/s 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.255+0000 7f3d314a9640 1 -- 172.21.15.90:0/4052545511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 msgr2=0x7f3d2c10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.255+0000 7f3d314a9640 1 --2- 172.21.15.90:0/4052545511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c10f3e0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009900 tx=0x7f3d2402a480 comp rx=0 tx=0).stop 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 -- 172.21.15.90:0/4052545511 shutdown_connections 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 --2- 172.21.15.90:0/4052545511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c10f3e0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 --2- 172.21.15.90:0/4052545511 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2c104500 0x7f3d2c10c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 --2- 172.21.15.90:0/4052545511 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d2c103be0 0x7f3d2c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 -- 172.21.15.90:0/4052545511 >> 172.21.15.90:0/4052545511 conn(0x7f3d2c077bb0 msgr2=0x7f3d2c077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 -- 172.21.15.90:0/4052545511 shutdown_connections 2024-04-26T18:26:25.260 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 -- 172.21.15.90:0/4052545511 wait complete. 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 Processor -- start 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.256+0000 7f3d314a9640 1 -- start start 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d314a9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2c103be0 0x7f3d2c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d314a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d2c104500 0x7f3d2c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d314a9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c19faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d314a9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c112bf0 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d314a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c112a70 con 0x7f3d2c104500 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d314a9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c112d70 con 0x7f3d2c103be0 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c19faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c19faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34552/0 (socket says 172.21.15.90:34552) 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 -- 172.21.15.90:0/558245508 learned_addr learned my addr 172.21.15.90:0/558245508 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:25.261 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d2c104500 0x7f3d2c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2affd640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2c103be0 0x7f3d2c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 -- 172.21.15.90:0/558245508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2c103be0 msgr2=0x7f3d2c19b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2c103be0 0x7f3d2c19b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 -- 172.21.15.90:0/558245508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d2c104500 msgr2=0x7f3d2c19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d2c104500 0x7f3d2c19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 -- 172.21.15.90:0/558245508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2c1a0220 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d2b7fe640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c19faf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005940 tx=0x7f3d240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.257+0000 7f3d23fff640 1 -- 172.21.15.90:0/558245508 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2403f030 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.258+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d2c1a04b0 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.258+0000 7f3d23fff640 1 -- 172.21.15.90:0/558245508 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d24003cb0 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.258+0000 7f3d23fff640 1 -- 172.21.15.90:0/558245508 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24035070 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.262 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.258+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d2c1a7da0 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.260+0000 7f3d23fff640 1 -- 172.21.15.90:0/558245508 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f3d24042030 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 Processor -- start 2024-04-26T18:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 -- start start 2024-04-26T18:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x56514503e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145048ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x56514504b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x565144fa4460 con 0x56514503e820 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x565144f5e590 con 0x5651450490e0 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d341c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565144fa0420 con 0x56514503df00 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d23fff640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d180a3dc0 0x7f3d180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d2affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145048ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d2affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145048ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34554/0 (socket says 172.21.15.90:34554) 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d2affd640 1 -- 172.21.15.90:0/3868688864 learned_addr learned my addr 172.21.15.90:0/3868688864 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d2b7fe640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d180a3dc0 0x7f3d180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x56514504b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.261+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x56514503e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d23fff640 1 -- 172.21.15.90:0/558245508 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 10347+0+0 (secure 0 0 0) 0x7f3d240f1720 con 0x7f3d2c10cfb0 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d2affd640 1 -- 172.21.15.90:0/3868688864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 msgr2=0x56514503e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x56514503e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d2affd640 1 -- 172.21.15.90:0/3868688864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 msgr2=0x56514504b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x56514504b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d2affd640 1 -- 172.21.15.90:0/3868688864 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565144f6b0d0 con 0x56514503e820 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145048ba0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0039f0 tx=0x7f3d1c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70058f51b9f3dd35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d2b7fe640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d180a3dc0 0x7f3d180a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d24003fd0 tx=0x7f3d24047000 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d217fa640 1 -- 172.21.15.90:0/3868688864 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c004b20 con 0x56514503e820 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d217fa640 1 -- 172.21.15.90:0/3868688864 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1c004cc0 con 0x56514503e820 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.262+0000 7f3d217fa640 1 -- 172.21.15.90:0/3868688864 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c0126b0 con 0x56514503e820 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 -- 172.21.15.90:0/3868688864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 msgr2=0x565145048ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145048ba0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c0039f0 tx=0x7f3d1c003eb0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 -- 172.21.15.90:0/3868688864 shutdown_connections 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x56514504b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145048ba0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3868688864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x56514503e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 -- 172.21.15.90:0/3868688864 >> 172.21.15.90:0/3868688864 conn(0x56514503a410 msgr2=0x56514503a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 -- 172.21.15.90:0/3868688864 shutdown_connections 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 -- 172.21.15.90:0/3868688864 wait complete. 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 -- start start 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.263+0000 7f3d341c1740 1 Processor -- start 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503df00 0x565145046040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503e820 0x565145046580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x565145046ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56514504bd10 con 0x56514503df00 2024-04-26T18:26:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x565144f2bb60 con 0x5651450490e0 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56514504be90 con 0x56514503e820 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 10 client.0 _ll_register_callbacks cb 0x565144f502b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503df00 0x565145046040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503df00 0x565145046040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34566/0 (socket says 172.21.15.90:34566) 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3760528586 learned_addr learned my addr 172.21.15.90:0/3760528586 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3760528586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503e820 msgr2=0x565145046580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503e820 0x565145046580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3760528586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 msgr2=0x565145046ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x565145046ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3760528586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565144cb6210 con 0x56514503df00 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503df00 0x565145046040 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3d140042f0 tx=0x7f3d140039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e8f598690462c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d127fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d127fc640 1 -- 172.21.15.90:0/3760528586 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14005220 con 0x56514503df00 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d127fc640 1 -- 172.21.15.90:0/3760528586 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 10347+0+0 (secure 0 0 0) 0x7f3d140053c0 con 0x56514503df00 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1c004cc0 con 0x56514503df00 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.264+0000 7f3d127fc640 1 -- 172.21.15.90:0/3760528586 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f3d140055c0 con 0x56514503df00 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 10 client.16527 fetch_fsmap learned FSMap version 160 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) -- 0x565144d82620 con 0x56514503df00 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d127fc640 1 -- 172.21.15.90:0/3760528586 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1c004cc0 con 0x56514503df00 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d127fc640 1 -- 172.21.15.90:0/3760528586 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 160) ==== 7136+0+0 (secure 0 0 0) 0x7f3d1401c030 con 0x56514503df00 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 10 client.16527 fetch_fsmap finished waiting for FSMap version 160 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 10 client.16527 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 20 client.16527 populate_metadata read hostname 'smithi090' 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450593f0 0x56514505b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 4 client.16527 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565144fb2070 con 0x5651450593f0 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450593f0 0x56514505b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d127fc640 10 client.16527 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.265+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450593f0 0x56514505b890 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab3d8e87ffd72d0 server_cookie=83bc4b6e6f096a77 in_seq=0 out_seq=0 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d127fc640 1 -- 172.21.15.90:0/3760528586 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565144fb2070 con 0x5651450593f0 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d127fc640 10 client.16527 handle_command_reply: tid=0 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 client.16527 shutdown 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503df00 msgr2=0x565145046040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503df00 0x565145046040 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3d140042f0 tx=0x7f3d140039c0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 shutdown_connections 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450593f0 0x56514505b890 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x565145046ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503e820 0x565145046580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3760528586 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503df00 0x565145046040 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d127fc640 0 client.16527 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 >> 172.21.15.90:0/3760528586 conn(0x56514503a410 msgr2=0x5651450395b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.266+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 shutdown_connections 2024-04-26T18:26:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.267+0000 7f3d341c1740 1 -- 172.21.15.90:0/3760528586 wait complete. 2024-04-26T18:26:25.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.267+0000 7f3d341c1740 20 client.16527 trim_cache size 0 max 16384 2024-04-26T18:26:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.281+0000 7f3d341c1740 1 Processor -- start 2024-04-26T18:26:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.281+0000 7f3d341c1740 1 -- start start 2024-04-26T18:26:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x565145054bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145046710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x5651450611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d341c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b2700 con 0x56514503e820 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d341c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b2dd0 con 0x5651450490e0 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d341c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b34a0 con 0x56514503df00 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145046710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145046710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34582/0 (socket says 172.21.15.90:34582) 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 -- 172.21.15.90:0/1200283293 learned_addr learned my addr 172.21.15.90:0/1200283293 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x5651450611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 -- 172.21.15.90:0/1200283293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 msgr2=0x565145054bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x565145054bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 -- 172.21.15.90:0/1200283293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 msgr2=0x5651450611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x5651450611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 -- 172.21.15.90:0/1200283293 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56514504b840 con 0x56514503e820 2024-04-26T18:26:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x5651450611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d2affd640 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145046710 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x56514503f4a0 tx=0x7f3d1c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67997f307e764808 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d20ff9640 1 -- 172.21.15.90:0/1200283293 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c02d070 con 0x56514503e820 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d20ff9640 1 -- 172.21.15.90:0/1200283293 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1c0278f0 con 0x56514503e820 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.282+0000 7f3d20ff9640 1 -- 172.21.15.90:0/1200283293 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c012cf0 con 0x56514503e820 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 -- 172.21.15.90:0/1200283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 msgr2=0x565145046710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145046710 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x56514503f4a0 tx=0x7f3d1c009c30 comp rx=0 tx=0).stop 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 -- 172.21.15.90:0/1200283293 shutdown_connections 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5651450490e0 0x5651450611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56514503e820 0x565145046710 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 --2- 172.21.15.90:0/1200283293 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x565145054bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 -- 172.21.15.90:0/1200283293 >> 172.21.15.90:0/1200283293 conn(0x56514503a410 msgr2=0x565145046000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 -- 172.21.15.90:0/1200283293 shutdown_connections 2024-04-26T18:26:25.288 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 -- 172.21.15.90:0/1200283293 wait complete. 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.283+0000 7f3d341c1740 1 -- start start 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 1 Processor -- start 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x5651450510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56514503e820 0x565145051620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651450490e0 0x56514504e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1af9a0 con 0x5651450490e0 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1b0d90 con 0x56514503e820 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2c1afff0 con 0x56514503df00 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d341c1740 10 client.0 _ll_register_callbacks cb 0x7f3d1400bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651450490e0 0x56514504e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651450490e0 0x56514504e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34598/0 (socket says 172.21.15.90:34598) 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3911361927 learned_addr learned my addr 172.21.15.90:0/3911361927 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:25.289 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56514503e820 0x565145051620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3911361927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 msgr2=0x5651450510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x5651450510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56514503e820 msgr2=0x565145051620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56514503e820 0x565145051620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 -- 172.21.15.90:0/3911361927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56514504e9a0 con 0x5651450490e0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d2a7fc640 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651450490e0 0x56514504e1a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3d14005fd0 tx=0x7f3d1400bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2e74bbd72c5c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d127fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d127fc640 1 -- 172.21.15.90:0/3911361927 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d14012a30 con 0x5651450490e0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.284+0000 7f3d127fc640 1 -- 172.21.15.90:0/3911361927 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 10347+0+0 (secure 0 0 0) 0x7f3d14012bd0 con 0x5651450490e0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d1c0278f0 con 0x5651450490e0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d127fc640 1 -- 172.21.15.90:0/3911361927 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=160) ==== 24+0+0 (secure 0 0 0) 0x7f3d1403a050 con 0x5651450490e0 2024-04-26T18:26:25.290 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 10 client.16533 fetch_fsmap learned FSMap version 160 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=160,monmap=4+}) -- 0x5651450c0580 con 0x5651450490e0 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d127fc640 1 -- 172.21.15.90:0/3911361927 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1c0278f0 con 0x5651450490e0 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d127fc640 1 -- 172.21.15.90:0/3911361927 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 160) ==== 7136+0+0 (secure 0 0 0) 0x7f3d1401c040 con 0x5651450490e0 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 10 client.16533 fetch_fsmap finished waiting for FSMap version 160 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 20 client.16533 populate_metadata read hostname 'smithi090' 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450c0d00 0x5651450c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 4 client.16533 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565144fabee0 con 0x5651450c0d00 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.285+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450c0d00 0x5651450c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.286+0000 7f3d2affd640 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450c0d00 0x5651450c3120 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc1dede9d168e581 server_cookie=cef8f6dd3a4dd7ac in_seq=0 out_seq=0 2024-04-26T18:26:25.291 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:25.286+0000 7f3d127fc640 10 client.16533 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:25.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 613 sec 2024-04-26T18:26:26.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 614 sec 2024-04-26T18:26:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:26 smithi090 ceph-mon[29180]: pgmap v484: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 29 op/s 2024-04-26T18:26:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:26 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:26 smithi119 ceph-mon[32752]: pgmap v484: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 29 op/s 2024-04-26T18:26:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:26 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:26 smithi167 ceph-mon[32751]: pgmap v484: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 811 KiB/s wr, 29 op/s 2024-04-26T18:26:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:26 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:27.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 615 sec 2024-04-26T18:26:28.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 616 sec 2024-04-26T18:26:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:28 smithi090 ceph-mon[29180]: pgmap v485: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 711 KiB/s wr, 25 op/s 2024-04-26T18:26:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:28 smithi119 ceph-mon[32752]: pgmap v485: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 711 KiB/s wr, 25 op/s 2024-04-26T18:26:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:28 smithi167 ceph-mon[32751]: pgmap v485: 97 pgs: 97 active+clean; 171 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 711 KiB/s wr, 25 op/s 2024-04-26T18:26:29.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 617 sec 2024-04-26T18:26:30.071 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:29 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:26:30.071 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:29 smithi119 systemd[1]: Stopped Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:30.072 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:29 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.289s CPU time. 2024-04-26T18:26:30.072 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:29 smithi119 systemd[1]: Starting Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:26:30.072 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 podman[71218]: 2024-04-26 18:26:29.945739293 +0000 UTC m=+0.021110658 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:26:30.072 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 podman[71218]: 2024-04-26 18:26:30.072327245 +0000 UTC m=+0.147698597 container create 11cee550334fa1316c49c12d3d101c29c0336e81ba39dcbc5aef8bd41b6601a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T18:26:30.399 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 podman[71218]: 2024-04-26 18:26:30.291606439 +0000 UTC m=+0.366977792 container init 11cee550334fa1316c49c12d3d101c29c0336e81ba39dcbc5aef8bd41b6601a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424) 2024-04-26T18:26:30.399 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 podman[71218]: 2024-04-26 18:26:30.29403204 +0000 UTC m=+0.369403402 container start 11cee550334fa1316c49c12d3d101c29c0336e81ba39dcbc5aef8bd41b6601a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux ) 2024-04-26T18:26:30.399 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mds[71237]: -- 172.21.15.119:0/1441627676 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d13e7eba40 con 0x55d13e7be800 2024-04-26T18:26:30.399 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: starting mds.e at 2024-04-26T18:26:30.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 618 sec 2024-04-26T18:26:30.803 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 bash[71218]: 11cee550334fa1316c49c12d3d101c29c0336e81ba39dcbc5aef8bd41b6601a3 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:26:30 smithi119 systemd[1]: Started Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: pgmap v486: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 607 KiB/s wr, 28 op/s 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: daemon mds.e restarted 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: osdmap e134: 12 total, 12 up, 12 in 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:boot 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:26:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:30 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: pgmap v486: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 607 KiB/s wr, 28 op/s 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: daemon mds.e restarted 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: osdmap e134: 12 total, 12 up, 12 in 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:boot 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:30 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: pgmap v486: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 607 KiB/s wr, 28 op/s 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: daemon mds.e restarted 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: osdmap e134: 12 total, 12 up, 12 in 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:boot 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:30 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:31.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 619 sec 2024-04-26T18:26:32.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:31 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:26:32.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:31 smithi167 systemd[1]: Stopped Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:32.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:31 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 8.742s CPU time. 2024-04-26T18:26:32.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:31 smithi167 systemd[1]: Starting Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:26:32.068 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:31 smithi167 podman[71672]: 2024-04-26 18:26:31.712221398 +0000 UTC m=+0.018357308 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:26:32.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mon[32751]: pgmap v488: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 621 KiB/s wr, 26 op/s 2024-04-26T18:26:32.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:resolve 2024-04-26T18:26:32.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mon[32751]: Replacing daemon mds.c as rank 2 with standby daemon mds.l 2024-04-26T18:26:32.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mon[32751]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-26T18:26:32.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:32.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mon[32751]: osdmap e135: 12 total, 12 up, 12 in 2024-04-26T18:26:32.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:replay} 1 up:standby-replay 5 up:standby 2024-04-26T18:26:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:32 smithi090 ceph-mon[29180]: pgmap v488: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 621 KiB/s wr, 26 op/s 2024-04-26T18:26:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:32 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:resolve 2024-04-26T18:26:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:32 smithi090 ceph-mon[29180]: Replacing daemon mds.c as rank 2 with standby daemon mds.l 2024-04-26T18:26:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:32 smithi090 ceph-mon[29180]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-26T18:26:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:32 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:32 smithi090 ceph-mon[29180]: osdmap e135: 12 total, 12 up, 12 in 2024-04-26T18:26:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:32 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:replay} 1 up:standby-replay 5 up:standby 2024-04-26T18:26:32.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 620 sec 2024-04-26T18:26:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:32 smithi119 ceph-mon[32752]: pgmap v488: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 621 KiB/s wr, 26 op/s 2024-04-26T18:26:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:32 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:resolve 2024-04-26T18:26:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:32 smithi119 ceph-mon[32752]: Replacing daemon mds.c as rank 2 with standby daemon mds.l 2024-04-26T18:26:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:32 smithi119 ceph-mon[32752]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-26T18:26:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:32 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:32 smithi119 ceph-mon[32752]: osdmap e135: 12 total, 12 up, 12 in 2024-04-26T18:26:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:32 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:replay} 1 up:standby-replay 5 up:standby 2024-04-26T18:26:32.887 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:32 smithi167 podman[71672]: 2024-04-26 18:26:32.623593367 +0000 UTC m=+0.929729269 container create fbca98285fcf1237d044cda3df9fbb9baa04e4d60a411ce131992f0eb0517bd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:26:32.887 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:32 smithi167 podman[71672]: 2024-04-26 18:26:32.859985203 +0000 UTC m=+1.166121106 container init fbca98285fcf1237d044cda3df9fbb9baa04e4d60a411ce131992f0eb0517bd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T18:26:32.887 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:32 smithi167 podman[71672]: 2024-04-26 18:26:32.862942044 +0000 UTC m=+1.169077948 container start fbca98285fcf1237d044cda3df9fbb9baa04e4d60a411ce131992f0eb0517bd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T18:26:33.317 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-mds[71690]: -- 172.21.15.167:0/399003328 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b830053a40 con 0x55b830026c00 2024-04-26T18:26:33.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:32 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: starting mds.c at 2024-04-26T18:26:33.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:32 smithi167 bash[71672]: fbca98285fcf1237d044cda3df9fbb9baa04e4d60a411ce131992f0eb0517bd6 2024-04-26T18:26:33.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:26:32 smithi167 systemd[1]: Started Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:33 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:resolve 2024-04-26T18:26:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:33 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:26:33.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 621 sec 2024-04-26T18:26:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:33 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:resolve 2024-04-26T18:26:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:33 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:26:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:33 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:resolve 2024-04-26T18:26:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:33 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=l=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:26:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.376+0000 7f3d127fc640 1 -- 172.21.15.90:0/3911361927 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565144fabee0 con 0x5651450c0d00 2024-04-26T18:26:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.376+0000 7f3d127fc640 10 client.16533 handle_command_reply: tid=0 2024-04-26T18:26:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 client.16533 shutdown 2024-04-26T18:26:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651450490e0 msgr2=0x56514504e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651450490e0 0x56514504e1a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3d14005fd0 tx=0x7f3d1400bb60 comp rx=0 tx=0).stop 2024-04-26T18:26:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 shutdown_connections 2024-04-26T18:26:34.378 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5651450c0d00 0x5651450c3120 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5651450490e0 0x56514504e1a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56514503e820 0x565145051620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d127fc640 0 client.16533 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 --2- 172.21.15.90:0/3911361927 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56514503df00 0x5651450510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 >> 172.21.15.90:0/3911361927 conn(0x56514503a410 msgr2=0x56514503f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 shutdown_connections 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 1 -- 172.21.15.90:0/3911361927 wait complete. 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.377+0000 7f3d341c1740 20 client.16533 trim_cache size 0 max 16384 2024-04-26T18:26:34.379 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:26:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:26:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:26:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:26:34.380 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d180a3dc0 msgr2=0x7f3d180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d180a3dc0 0x7f3d180a6280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d24003fd0 tx=0x7f3d24047000 comp rx=0 tx=0).stop 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 msgr2=0x7f3d2c19faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c19faf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3d24005940 tx=0x7f3d240042f0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 shutdown_connections 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f3d180a3dc0 0x7f3d180a6280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d2c10cfb0 0x7f3d2c19faf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d2c104500 0x7f3d2c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 --2- 172.21.15.90:0/558245508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d2c103be0 0x7f3d2c19b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 >> 172.21.15.90:0/558245508 conn(0x7f3d2c077bb0 msgr2=0x7f3d2c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 shutdown_connections 2024-04-26T18:26:34.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:34.378+0000 7f3d314a9640 1 -- 172.21.15.90:0/558245508 wait complete. 2024-04-26T18:26:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: pgmap v490: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-26T18:26:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:26:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:26:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:26:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:reconnect 2024-04-26T18:26:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:reconnect 2024-04-26T18:26:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:boot 2024-04-26T18:26:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: pgmap v490: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:reconnect 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:reconnect 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:boot 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:26:34.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 622 sec 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: pgmap v490: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999975 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.000999982 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:reconnect 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:reconnect 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:boot 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:26:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:rejoin 2024-04-26T18:26:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:rejoin 2024-04-26T18:26:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:35 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:35 smithi090 ceph-mon[29180]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:26:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:35 smithi090 ceph-mon[29180]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:26:35.713 INFO:tasks.workunit.client.0.smithi090.stdout: 1 cleanup 623 sec 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: 0 cleanup 623 sec 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: Operation Count AvgLat MaxLat 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: ---------------------------------------- 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: NTCreateX 51676 4.951 27931.034 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: Close 37966 0.003 0.090 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: Rename 2189 4.759 28.683 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: Unlink 10430 16.150 30805.245 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: Deltree 2 86.440 135.141 2024-04-26T18:26:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: Mkdir 1 0.007 0.007 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: Qpathinfo 46826 3.585 29973.268 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: Qfileinfo 8236 0.001 0.022 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: Qfsinfo 8602 0.180 11.160 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: Sfileinfo 4204 0.021 3.510 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: Find 18147 0.313 932.323 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: WriteX 25915 0.084 32.409 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: ReadX 81169 0.023 38.369 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: LockX 168 0.823 12.520 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: UnlockX 168 0.719 7.852 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: Flush 3626 2.706 28.860 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-04-26T18:26:35.772 INFO:tasks.workunit.client.0.smithi090.stdout:Throughput 2.7261 MB/sec 1 clients 1 procs max_latency=30805.261 ms 2024-04-26T18:26:35.773 INFO:tasks.workunit.client.0.smithi090.stderr:+ dbench 10 2024-04-26T18:26:35.773 INFO:tasks.workunit.client.0.smithi090.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-26T18:26:35.773 INFO:tasks.workunit.client.0.smithi090.stdout: 2024-04-26T18:26:35.773 INFO:tasks.workunit.client.0.smithi090.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-26T18:26:35.773 INFO:tasks.workunit.client.0.smithi090.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-26T18:26:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:rejoin 2024-04-26T18:26:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:rejoin 2024-04-26T18:26:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:35 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:35 smithi119 ceph-mon[32752]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:26:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:35 smithi119 ceph-mon[32752]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:26:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:rejoin 2024-04-26T18:26:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:rejoin 2024-04-26T18:26:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:35 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:35 smithi167 ceph-mon[32751]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:26:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:35 smithi167 ceph-mon[32751]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:26:35.818 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 26 18:26:35 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-26T18:26:35.390+0000 7f61d38ab700 -1 bdev(0x557f5274a000 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 2 2024-04-26T18:26:35.818 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 26 18:26:35 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-5[44384]: 2024-04-26T18:26:35.392+0000 7f61d80b4700 -1 bdev(0x557f5274a000 /var/lib/ceph/osd/ceph-5/block) aio_submit retries 13 2024-04-26T18:26:35.874 INFO:tasks.workunit.client.0.smithi090.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-26T18:26:35.874 INFO:tasks.workunit.client.0.smithi090.stdout:releasing clients 2024-04-26T18:26:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:36 smithi090 ceph-mon[29180]: pgmap v491: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-26T18:26:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:36 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:36 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:active 2024-04-26T18:26:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:36 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:active 2024-04-26T18:26:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:36 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:36 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:36 smithi119 ceph-mon[32752]: pgmap v491: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-26T18:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:36 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:36 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:active 2024-04-26T18:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:36 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:active 2024-04-26T18:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:36 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:36 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:36 smithi167 ceph-mon[32751]: pgmap v491: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-26T18:26:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:36 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:36 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/925527879,v1:172.21.15.119:6833/925527879] up:active 2024-04-26T18:26:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:36 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2550962231,v1:172.21.15.167:6833/2550962231] up:active 2024-04-26T18:26:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:36 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:36 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:36.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 316 115.37 MB/sec warmup 1 sec latency 62.112 ms 2024-04-26T18:26:37.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 689 103.48 MB/sec warmup 2 sec latency 73.095 ms 2024-04-26T18:26:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:38 smithi090 ceph-mon[29180]: pgmap v492: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4 op/s 2024-04-26T18:26:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:38 smithi119 ceph-mon[32752]: pgmap v492: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4 op/s 2024-04-26T18:26:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:38 smithi167 ceph-mon[32751]: pgmap v492: 97 pgs: 97 active+clean; 172 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 4 op/s 2024-04-26T18:26:38.872 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1141 76.26 MB/sec warmup 3 sec latency 74.154 ms 2024-04-26T18:26:39.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 1728 65.63 MB/sec warmup 4 sec latency 55.160 ms 2024-04-26T18:26:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:40 smithi090 ceph-mon[29180]: pgmap v493: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 901 KiB/s wr, 116 op/s 2024-04-26T18:26:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:40 smithi119 ceph-mon[32752]: pgmap v493: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 901 KiB/s wr, 116 op/s 2024-04-26T18:26:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:40 smithi167 ceph-mon[32751]: pgmap v493: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 901 KiB/s wr, 116 op/s 2024-04-26T18:26:40.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2418 56.77 MB/sec warmup 5 sec latency 483.920 ms 2024-04-26T18:26:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:41 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:41.753 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:41.754 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6c50315700 time 2024-04-26T18:26:41.379266+0000 2024-04-26T18:26:41.754 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.754 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.754 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6c5a93e3dd] 2024-04-26T18:26:41.754 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.755 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: clone() 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.379+0000 7f6c50315700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6c50315700 time 2024-04-26T18:26:41.379266+0000 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6c5a93e3dd] 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.756 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: clone() 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: *** Caught signal (Aborted) ** 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: in thread 7f6c50315700 thread_name:ms_dispatch 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.757 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6c596c6d20] 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: gsignal() 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: abort() 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6c5a93e437] 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.758 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.759 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.759 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.759 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.759 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 16: clone() 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.380+0000 7f6c50315700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: in thread 7f6c50315700 thread_name:ms_dispatch 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6c596c6d20] 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: gsignal() 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: abort() 2024-04-26T18:26:41.760 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6c5a93e437] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 16: clone() 2024-04-26T18:26:41.761 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: -1> 2024-04-26T18:26:41.379+0000 7f6c50315700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6c50315700 time 2024-04-26T18:26:41.379266+0000 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6c5a93e3dd] 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.762 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: clone() 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 0> 2024-04-26T18:26:41.380+0000 7f6c50315700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: in thread 7f6c50315700 thread_name:ms_dispatch 2024-04-26T18:26:41.763 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6c596c6d20] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: gsignal() 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: abort() 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6c5a93e437] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.764 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 16: clone() 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: -1> 2024-04-26T18:26:41.379+0000 7f6c50315700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6c50315700 time 2024-04-26T18:26:41.379266+0000 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6c5a93e3dd] 2024-04-26T18:26:41.765 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.766 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: clone() 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 0> 2024-04-26T18:26:41.380+0000 7f6c50315700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: in thread 7f6c50315700 thread_name:ms_dispatch 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6c596c6d20] 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2: gsignal() 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 3: abort() 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6c5a93e437] 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6c5a93e5a3] 2024-04-26T18:26:41.767 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55dbabeaf822] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55dbabe1f598] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55dbabe21c17] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55dbabd1ad0b] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55dbabd1b2ac] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55dbabd03fad] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6c5abd2e20] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 13: (DispatchQueue::entry()+0x51f) [0x7f6c5abcffdf] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6c5aca2fc1] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6c596bc1ca] 2024-04-26T18:26:41.768 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 16: clone() 2024-04-26T18:26:41.769 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:41.769 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[70934]: 2024-04-26T18:26:41.769 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:41 smithi119 podman[71276]: 2024-04-26 18:26:41.44009672 +0000 UTC m=+0.018981744 container died a551ac7f8a83746f4de4dd5eb12f71b403284fe650d05dd7542776249c20c720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-26T18:26:41.780 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2c53492700 time 2024-04-26T18:26:41.380899+0000 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c5dabb3dd] 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.781 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: clone() 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: *** Caught signal (Aborted) ** 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: in thread 7f2c53492700 thread_name:ms_dispatch 2024-04-26T18:26:41.782 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.380+0000 7f2c53492700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2c53492700 time 2024-04-26T18:26:41.380899+0000 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c5dabb3dd] 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.784 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.785 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.785 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.785 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.785 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.785 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: clone() 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c5c843d20] 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: gsignal() 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: abort() 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c5dabb437] 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.786 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 16: clone() 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.382+0000 7f2c53492700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: in thread 7f2c53492700 thread_name:ms_dispatch 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.787 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c5c843d20] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: gsignal() 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: abort() 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c5dabb437] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.788 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 16: clone() 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: -1> 2024-04-26T18:26:41.380+0000 7f2c53492700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2c53492700 time 2024-04-26T18:26:41.380899+0000 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.789 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c5dabb3dd] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.790 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: clone() 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 0> 2024-04-26T18:26:41.382+0000 7f2c53492700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: in thread 7f2c53492700 thread_name:ms_dispatch 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c5c843d20] 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: gsignal() 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: abort() 2024-04-26T18:26:41.791 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c5dabb437] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.792 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 16: clone() 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: -1> 2024-04-26T18:26:41.380+0000 7f2c53492700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2c53492700 time 2024-04-26T18:26:41.380899+0000 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f2c5dabb3dd] 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.793 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: clone() 2024-04-26T18:26:41.794 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 0> 2024-04-26T18:26:41.382+0000 7f2c53492700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: in thread 7f2c53492700 thread_name:ms_dispatch 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2c5c843d20] 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2: gsignal() 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 3: abort() 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f2c5dabb437] 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f2c5dabb5a3] 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556bff155822] 2024-04-26T18:26:41.795 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556bff0c5598] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556bff0c7c17] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556bfefc0d0b] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556bfefc12ac] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556bfefa9fad] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2c5dd4fe20] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 13: (DispatchQueue::entry()+0x51f) [0x7f2c5dd4cfdf] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f2c5de1ffc1] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f2c5c8391ca] 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 16: clone() 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:26:41.796 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71390]: 2024-04-26T18:26:41.797 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:41 smithi167 podman[71733]: 2024-04-26 18:26:41.438450059 +0000 UTC m=+0.018223714 container died efc0861150d09f9c0263965475c7e56f0f90cbb1c34e885b9e269ec1d00163ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T18:26:41.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 54.15 MB/sec warmup 6 sec latency 1484.019 ms 2024-04-26T18:26:42.399 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:42 smithi167 ceph-mon[32751]: pgmap v494: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 805 KiB/s wr, 103 op/s 2024-04-26T18:26:42.399 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:42 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:42.399 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:42 smithi167 podman[71733]: 2024-04-26 18:26:42.141367895 +0000 UTC m=+0.721141545 container remove efc0861150d09f9c0263965475c7e56f0f90cbb1c34e885b9e269ec1d00163ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T18:26:42.399 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:42 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:26:42.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:42 smithi119 ceph-mon[32752]: pgmap v494: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 805 KiB/s wr, 103 op/s 2024-04-26T18:26:42.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:42 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:42.400 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:42 smithi119 podman[71276]: 2024-04-26 18:26:42.138503813 +0000 UTC m=+0.717388859 container remove a551ac7f8a83746f4de4dd5eb12f71b403284fe650d05dd7542776249c20c720 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2) 2024-04-26T18:26:42.400 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:42 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:26:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:42 smithi090 ceph-mon[29180]: pgmap v494: 97 pgs: 97 active+clean; 173 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 805 KiB/s wr, 103 op/s 2024-04-26T18:26:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:42 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:42.803 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:42 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-26T18:26:42.804 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:42 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 4.347s CPU time. 2024-04-26T18:26:42.817 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:42 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-04-26T18:26:42.817 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:42 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 8.093s CPU time. 2024-04-26T18:26:42.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 46.42 MB/sec warmup 7 sec latency 2484.076 ms 2024-04-26T18:26:43.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 40.62 MB/sec warmup 8 sec latency 3484.170 ms 2024-04-26T18:26:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:44 smithi119 ceph-mon[32752]: pgmap v495: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-26T18:26:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:44 smithi167 ceph-mon[32751]: pgmap v495: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-26T18:26:44.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 36.10 MB/sec warmup 9 sec latency 4484.264 ms 2024-04-26T18:26:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:44 smithi090 ceph-mon[29180]: pgmap v495: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 1.4 MiB/s wr, 135 op/s 2024-04-26T18:26:45.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 32.49 MB/sec warmup 10 sec latency 5484.396 ms 2024-04-26T18:26:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:46 smithi119 ceph-mon[32752]: pgmap v496: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-04-26T18:26:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:46 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:46 smithi167 ceph-mon[32751]: pgmap v496: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-04-26T18:26:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:46 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:46.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 29.54 MB/sec warmup 11 sec latency 6484.520 ms 2024-04-26T18:26:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:46 smithi090 ceph-mon[29180]: pgmap v496: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-04-26T18:26:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:46 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.477+0000 7fa50739e640 1 -- 172.21.15.90:0/2920569839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500101b80 msgr2=0x7fa500102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.477+0000 7fa50739e640 1 --2- 172.21.15.90:0/2920569839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500101b80 0x7fa500102000 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009900 tx=0x7fa4f002a410 comp rx=0 tx=0).stop 2024-04-26T18:26:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.477+0000 7fa50739e640 1 -- 172.21.15.90:0/2920569839 shutdown_connections 2024-04-26T18:26:47.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.477+0000 7fa50739e640 1 --2- 172.21.15.90:0/2920569839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa500102540 0x7fa50010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 --2- 172.21.15.90:0/2920569839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500101b80 0x7fa500102000 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 --2- 172.21.15.90:0/2920569839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5001067d0 0x7fa500106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 -- 172.21.15.90:0/2920569839 >> 172.21.15.90:0/2920569839 conn(0x7fa500077630 msgr2=0x7fa500077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 -- 172.21.15.90:0/2920569839 shutdown_connections 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 -- 172.21.15.90:0/2920569839 wait complete. 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 Processor -- start 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 -- start start 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa500101b80 0x7fa500197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500102540 0x7fa5001981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5001067d0 0x7fa5001986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111a70 con 0x7fa500102540 2024-04-26T18:26:47.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001118f0 con 0x7fa500101b80 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.478+0000 7fa50739e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa500111bf0 con 0x7fa5001067d0 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500102540 0x7fa5001981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500102540 0x7fa5001981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50802/0 (socket says 172.21.15.90:50802) 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 -- 172.21.15.90:0/3510358593 learned_addr learned my addr 172.21.15.90:0/3510358593 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa50513a640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa500101b80 0x7fa500197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa50593b640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5001067d0 0x7fa5001986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 -- 172.21.15.90:0/3510358593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5001067d0 msgr2=0x7fa5001986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5001067d0 0x7fa5001986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 -- 172.21.15.90:0/3510358593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa500101b80 msgr2=0x7fa500197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa500101b80 0x7fa500197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 -- 172.21.15.90:0/3510358593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5001a7470 con 0x7fa500102540 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa50593b640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5001067d0 0x7fa5001986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:47.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa504939640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500102540 0x7fa5001981a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0005940 tx=0x7fa4f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa4f67fc640 1 -- 172.21.15.90:0/3510358593 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f003f030 con 0x7fa500102540 2024-04-26T18:26:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa4f67fc640 1 -- 172.21.15.90:0/3510358593 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f0003cb0 con 0x7fa500102540 2024-04-26T18:26:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa4f67fc640 1 -- 172.21.15.90:0/3510358593 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0035070 con 0x7fa500102540 2024-04-26T18:26:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5001a7700 con 0x7fa500102540 2024-04-26T18:26:47.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.479+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5001a7c40 con 0x7fa500102540 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa4f67fc640 1 -- 172.21.15.90:0/3510358593 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fa4f0042030 con 0x7fa500102540 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa50a18a740 1 Processor -- start 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa50a18a740 1 -- start start 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d30e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 0x55961d318ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa4f67fc640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa4ec0a3e40 0x7fa4ec0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d31b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa50a18a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55961d274460 con 0x55961d30df00 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.482+0000 7fa50a18a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55961d22e590 con 0x55961d3190e0 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50a18a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55961d270420 con 0x55961d30e820 2024-04-26T18:26:47.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d30e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d30e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50812/0 (socket says 172.21.15.90:50812) 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 -- 172.21.15.90:0/3795943073 learned_addr learned my addr 172.21.15.90:0/3795943073 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa504939640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa4ec0a3e40 0x7fa4ec0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa4f67fc640 1 -- 172.21.15.90:0/3510358593 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 10475+0+0 (secure 0 0 0) 0x7fa4f00f17a0 con 0x7fa500102540 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50513a640 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 0x55961d318ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d31b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 -- 172.21.15.90:0/3795943073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 msgr2=0x55961d318ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 0x55961d318ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 -- 172.21.15.90:0/3795943073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 msgr2=0x55961d31b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d31b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 -- 172.21.15.90:0/3795943073 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961d23b0d0 con 0x55961d30df00 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa504939640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa4ec0a3e40 0x7fa4ec0a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00eed50 tx=0x7fa4f00b2430 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa50593b640 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d30e2e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8015930 tx=0x7fa4f8009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd12dd110e974ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa4eb7fe640 1 -- 172.21.15.90:0/3795943073 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f801b070 con 0x55961d30df00 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa4eb7fe640 1 -- 172.21.15.90:0/3795943073 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f8003e20 con 0x55961d30df00 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.483+0000 7fa4eb7fe640 1 -- 172.21.15.90:0/3795943073 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8004030 con 0x55961d30df00 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 -- 172.21.15.90:0/3795943073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 msgr2=0x55961d30e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d30e2e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8015930 tx=0x7fa4f8009990 comp rx=0 tx=0).stop 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 -- 172.21.15.90:0/3795943073 shutdown_connections 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d31b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 0x55961d318ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3795943073 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d30e2e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 -- 172.21.15.90:0/3795943073 >> 172.21.15.90:0/3795943073 conn(0x55961d30a410 msgr2=0x55961d30a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 -- 172.21.15.90:0/3795943073 shutdown_connections 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.484+0000 7fa50a18a740 1 -- 172.21.15.90:0/3795943073 wait complete. 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 -- start start 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 Processor -- start 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d30df00 0x55961d316030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d316570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d3190e0 0x55961d316ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55961d31bd10 con 0x55961d30e820 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55961d1fbb60 con 0x55961d30df00 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55961d31be90 con 0x55961d3190e0 2024-04-26T18:26:47.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50a18a740 10 client.0 _ll_register_callbacks cb 0x55961d2202b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50513a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d316570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50513a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d316570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50822/0 (socket says 172.21.15.90:50822) 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50513a640 1 -- 172.21.15.90:0/851500872 learned_addr learned my addr 172.21.15.90:0/851500872 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50593b640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d3190e0 0x55961d316ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50593b640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d30df00 0x55961d316030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50513a640 1 -- 172.21.15.90:0/851500872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d3190e0 msgr2=0x55961d316ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.485+0000 7fa50513a640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d3190e0 0x55961d316ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50513a640 1 -- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d30df00 msgr2=0x55961d316030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50513a640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d30df00 0x55961d316030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50513a640 1 -- 172.21.15.90:0/851500872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961cf86210 con 0x55961d30e820 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50593b640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d3190e0 0x55961d316ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50513a640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d316570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00be90 tx=0x7fa4fc012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98a3cd05f145bdd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa4e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/851500872 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc004220 con 0x55961d30e820 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/851500872 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 10475+0+0 (secure 0 0 0) 0x7fa4fc0043c0 con 0x55961d30e820 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f8003e20 con 0x55961d30e820 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/851500872 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc005020 con 0x55961d30e820 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 10 client.16557 fetch_fsmap learned FSMap version 173 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) -- 0x55961d052620 con 0x55961d30e820 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/851500872 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f8003e20 con 0x55961d30e820 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/851500872 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 173) ==== 7140+0+0 (secure 0 0 0) 0x7fa4fc010650 con 0x55961d30e820 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 10 client.16557 fetch_fsmap finished waiting for FSMap version 173 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:47.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 20 client.16557 populate_metadata read hostname 'smithi090' 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d329540 0x55961d32ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 4 client.16557 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.486+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55961d282070 con 0x55961d329540 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.487+0000 7fa50593b640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d329540 0x55961d32ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.487+0000 7fa4e8ff9640 10 client.16557 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.487+0000 7fa50593b640 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d329540 0x55961d32ba00 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a16401c6d3d3c99d server_cookie=235adfffb8e525eb in_seq=0 out_seq=0 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/851500872 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55961d282070 con 0x55961d329540 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa4e8ff9640 10 client.16557 handle_command_reply: tid=0 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 client.16557 shutdown 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 msgr2=0x55961d316570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d316570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00be90 tx=0x7fa4fc012820 comp rx=0 tx=0).stop 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 shutdown_connections 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d329540 0x55961d32ba00 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d3190e0 0x55961d316ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d316570 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa4e8ff9640 0 client.16557 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 --2- 172.21.15.90:0/851500872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d30df00 0x55961d316030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 >> 172.21.15.90:0/851500872 conn(0x55961d30a410 msgr2=0x55961d310140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 shutdown_connections 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 1 -- 172.21.15.90:0/851500872 wait complete. 2024-04-26T18:26:47.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.488+0000 7fa50a18a740 20 client.16557 trim_cache size 0 max 16384 2024-04-26T18:26:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 Processor -- start 2024-04-26T18:26:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 -- start start 2024-04-26T18:26:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d309d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 0x55961d324bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d316700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b26e0 con 0x55961d30df00 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b2db0 con 0x55961d3190e0 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50a18a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b34b0 con 0x55961d30e820 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50593b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d309d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50593b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d309d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50832/0 (socket says 172.21.15.90:50832) 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50593b640 1 -- 172.21.15.90:0/436540078 learned_addr learned my addr 172.21.15.90:0/436540078 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50593b640 1 -- 172.21.15.90:0/436540078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 msgr2=0x55961d324bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50593b640 1 --2- 172.21.15.90:0/436540078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 0x55961d324bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50593b640 1 -- 172.21.15.90:0/436540078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 msgr2=0x55961d316700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.503+0000 7fa50593b640 1 --2- 172.21.15.90:0/436540078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d316700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50593b640 1 -- 172.21.15.90:0/436540078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961d31be00 con 0x55961d30df00 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50593b640 1 --2- 172.21.15.90:0/436540078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d309d30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8015430 tx=0x7fa4f8009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bd5c1c9ae29f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa4eaffd640 1 -- 172.21.15.90:0/436540078 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f802b070 con 0x55961d30df00 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa4eaffd640 1 -- 172.21.15.90:0/436540078 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f801c040 con 0x55961d30df00 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa4eaffd640 1 -- 172.21.15.90:0/436540078 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8026e60 con 0x55961d30df00 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 -- 172.21.15.90:0/436540078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 msgr2=0x55961d309d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 --2- 172.21.15.90:0/436540078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d309d30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8015430 tx=0x7fa4f8009990 comp rx=0 tx=0).stop 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 -- 172.21.15.90:0/436540078 shutdown_connections 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 --2- 172.21.15.90:0/436540078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d316700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 --2- 172.21.15.90:0/436540078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30e820 0x55961d324bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 --2- 172.21.15.90:0/436540078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30df00 0x55961d309d30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 -- 172.21.15.90:0/436540078 >> 172.21.15.90:0/436540078 conn(0x55961d30a410 msgr2=0x55961d315ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:47.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 -- 172.21.15.90:0/436540078 shutdown_connections 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.504+0000 7fa50a18a740 1 -- 172.21.15.90:0/436540078 wait complete. 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 -- start start 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 Processor -- start 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30df00 0x55961d312fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d313500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d313a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001af9b0 con 0x55961d30e820 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b0da0 con 0x55961d3190e0 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa5001b0000 con 0x55961d30df00 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50a18a740 10 client.0 _ll_register_callbacks cb 0x7fa4fc00baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50513a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d313500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.510 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50513a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d313500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50848/0 (socket says 172.21.15.90:50848) 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.505+0000 7fa50513a640 1 -- 172.21.15.90:0/3816130622 learned_addr learned my addr 172.21.15.90:0/3816130622 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50593b640 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30df00 0x55961d312fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50513a640 1 -- 172.21.15.90:0/3816130622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30df00 msgr2=0x55961d312fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50513a640 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30df00 0x55961d312fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50513a640 1 -- 172.21.15.90:0/3816130622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 msgr2=0x55961d313a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50513a640 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d313a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50513a640 1 -- 172.21.15.90:0/3816130622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961d38ee10 con 0x55961d30e820 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50513a640 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d313500 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc010ac0 tx=0x7fa4fc010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63747d1f5bf0765 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa4e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/3816130622 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc02e030 con 0x55961d30e820 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/3816130622 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(135..135 src has 1..135) ==== 10475+0+0 (secure 0 0 0) 0x7fa4fc004230 con 0x55961d30e820 2024-04-26T18:26:47.511 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4f801c040 con 0x55961d30e820 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/3816130622 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7fa4fc038050 con 0x55961d30e820 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 10 client.16563 fetch_fsmap learned FSMap version 173 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=173,monmap=4+}) -- 0x55961d390120 con 0x55961d30e820 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/3816130622 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4f801c040 con 0x55961d30e820 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/3816130622 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 173) ==== 7140+0+0 (secure 0 0 0) 0x7fa4fc033890 con 0x55961d30e820 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 10 client.16563 fetch_fsmap finished waiting for FSMap version 173 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 10 client.16563 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 20 client.16563 populate_metadata read hostname 'smithi090' 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d390860 0x55961d392c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 4 client.16563 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.506+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55961d27bee0 con 0x55961d390860 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.507+0000 7fa50593b640 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d390860 0x55961d392c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:47.512 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.507+0000 7fa4e8ff9640 10 client.16563 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:47.513 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:47.507+0000 7fa50593b640 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d390860 0x55961d392c80 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d07de3402fb0d6d9 server_cookie=f19a9a0ff38a8296 in_seq=0 out_seq=0 2024-04-26T18:26:47.873 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 27.08 MB/sec warmup 12 sec latency 7484.641 ms 2024-04-26T18:26:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:48 smithi119 ceph-mon[32752]: pgmap v497: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-04-26T18:26:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:48 smithi167 ceph-mon[32751]: pgmap v497: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-04-26T18:26:48.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 24.99 MB/sec warmup 13 sec latency 8484.731 ms 2024-04-26T18:26:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:48 smithi090 ceph-mon[29180]: pgmap v497: 97 pgs: 97 active+clean; 182 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-04-26T18:26:49.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 23.21 MB/sec warmup 14 sec latency 9484.831 ms 2024-04-26T18:26:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:50 smithi119 ceph-mon[32752]: pgmap v498: 97 pgs: 97 active+clean; 182 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-04-26T18:26:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:50 smithi167 ceph-mon[32751]: pgmap v498: 97 pgs: 97 active+clean; 182 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-04-26T18:26:50.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 21.66 MB/sec warmup 15 sec latency 10484.933 ms 2024-04-26T18:26:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:50 smithi090 ceph-mon[29180]: pgmap v498: 97 pgs: 97 active+clean; 182 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.5 MiB/s wr, 329 op/s 2024-04-26T18:26:51.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 20.31 MB/sec warmup 16 sec latency 11485.014 ms 2024-04-26T18:26:52.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:52 smithi119 ceph-mon[32752]: pgmap v499: 97 pgs: 97 active+clean; 182 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 897 KiB/s wr, 246 op/s 2024-04-26T18:26:52.791 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:52 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:26:52.791 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:52 smithi119 systemd[1]: Stopped Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:52.791 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:52 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 4.347s CPU time. 2024-04-26T18:26:52.791 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:52 smithi119 systemd[1]: Starting Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:26:52.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:52 smithi167 ceph-mon[32751]: pgmap v499: 97 pgs: 97 active+clean; 182 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 897 KiB/s wr, 246 op/s 2024-04-26T18:26:52.800 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:52 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:26:52.800 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:52 smithi167 systemd[1]: Stopped Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:52.800 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:52 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 8.093s CPU time. 2024-04-26T18:26:52.800 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:52 smithi167 systemd[1]: Starting Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:26:52.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 19.11 MB/sec warmup 17 sec latency 12485.095 ms 2024-04-26T18:26:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:52 smithi090 ceph-mon[29180]: pgmap v499: 97 pgs: 97 active+clean; 182 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 897 KiB/s wr, 246 op/s 2024-04-26T18:26:53.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:52 smithi119 podman[71371]: 2024-04-26 18:26:52.694616201 +0000 UTC m=+0.017990605 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:26:53.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:52 smithi167 podman[71830]: 2024-04-26 18:26:52.703531064 +0000 UTC m=+0.018201739 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:26:53.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:52 smithi167 podman[71830]: 2024-04-26 18:26:52.948167683 +0000 UTC m=+0.262838355 container create 6bef2c9637c9cef07fc738b0459e23455bef5d8348c25a5d72456f93218835e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:26:53.396 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:53 smithi167 podman[71830]: 2024-04-26 18:26:53.211370279 +0000 UTC m=+0.526040958 container init 6bef2c9637c9cef07fc738b0459e23455bef5d8348c25a5d72456f93218835e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T18:26:53.396 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:53 smithi167 podman[71830]: 2024-04-26 18:26:53.214058135 +0000 UTC m=+0.528728816 container start 6bef2c9637c9cef07fc738b0459e23455bef5d8348c25a5d72456f93218835e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:26:53.396 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mds[71849]: -- 172.21.15.167:0/1925791509 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557079b5da40 con 0x557079b30c00 2024-04-26T18:26:53.396 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: starting mds.l at 2024-04-26T18:26:53.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: daemon mds.l restarted 2024-04-26T18:26:53.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:26:53.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: osdmap e136: 12 total, 12 up, 12 in 2024-04-26T18:26:53.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:boot 2024-04-26T18:26:53.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:26:53.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:26:53.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:26:53.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:53 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:53.817 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:53 smithi167 bash[71830]: 6bef2c9637c9cef07fc738b0459e23455bef5d8348c25a5d72456f93218835e2 2024-04-26T18:26:53.818 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:26:53 smithi167 systemd[1]: Started Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: daemon mds.l restarted 2024-04-26T18:26:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:26:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: osdmap e136: 12 total, 12 up, 12 in 2024-04-26T18:26:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:boot 2024-04-26T18:26:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:26:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:26:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:26:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:53 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:53.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 18.05 MB/sec warmup 18 sec latency 13485.204 ms 2024-04-26T18:26:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: daemon mds.l restarted 2024-04-26T18:26:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:26:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: osdmap e136: 12 total, 12 up, 12 in 2024-04-26T18:26:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:boot 2024-04-26T18:26:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:26:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:26:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:26:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:53 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:26:54.054 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:53 smithi119 podman[71371]: 2024-04-26 18:26:53.782242631 +0000 UTC m=+1.105617026 container create 8a6d37d0cefedd1f77685ba01fc0b032bd19b4e80ffb9bdc0a8ad5ba68c42dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T18:26:54.398 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:54 smithi119 podman[71371]: 2024-04-26 18:26:54.094044001 +0000 UTC m=+1.417418395 container init 8a6d37d0cefedd1f77685ba01fc0b032bd19b4e80ffb9bdc0a8ad5ba68c42dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T18:26:54.398 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:54 smithi119 podman[71371]: 2024-04-26 18:26:54.096916907 +0000 UTC m=+1.420291303 container start 8a6d37d0cefedd1f77685ba01fc0b032bd19b4e80ffb9bdc0a8ad5ba68c42dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T18:26:54.398 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mds[71391]: -- 172.21.15.119:0/1824880691 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635e9a85a40 con 0x5635e9a58c00 2024-04-26T18:26:54.398 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: starting mds.k at 2024-04-26T18:26:54.398 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:54 smithi119 bash[71371]: 8a6d37d0cefedd1f77685ba01fc0b032bd19b4e80ffb9bdc0a8ad5ba68c42dbc 2024-04-26T18:26:54.398 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:26:54 smithi119 systemd[1]: Started Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:26:54.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: pgmap v501: 97 pgs: 97 active+clean; 320 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 283 op/s 2024-04-26T18:26:54.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: daemon mds.k restarted 2024-04-26T18:26:54.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: osdmap e137: 12 total, 12 up, 12 in 2024-04-26T18:26:54.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:resolve 2024-04-26T18:26:54.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:boot 2024-04-26T18:26:54.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:26:54.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:26:54.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:26:54.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:54 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: pgmap v501: 97 pgs: 97 active+clean; 320 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 283 op/s 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: daemon mds.k restarted 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: osdmap e137: 12 total, 12 up, 12 in 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:resolve 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:boot 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:54 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: pgmap v501: 97 pgs: 97 active+clean; 320 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 14 MiB/s wr, 283 op/s 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: daemon mds.k restarted 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: osdmap e137: 12 total, 12 up, 12 in 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:resolve 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:boot 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:26:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:54 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:54.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 17.10 MB/sec warmup 19 sec latency 14485.285 ms 2024-04-26T18:26:55.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 16.25 MB/sec warmup 20 sec latency 15485.370 ms 2024-04-26T18:26:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:56 smithi119 ceph-mon[32752]: pgmap v503: 97 pgs: 97 active+clean; 360 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 364 op/s 2024-04-26T18:26:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:56 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:resolve 2024-04-26T18:26:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:56 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:56 smithi167 ceph-mon[32751]: pgmap v503: 97 pgs: 97 active+clean; 360 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 364 op/s 2024-04-26T18:26:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:56 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:resolve 2024-04-26T18:26:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:56 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:56.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2793 15.47 MB/sec warmup 21 sec latency 16485.468 ms 2024-04-26T18:26:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:56 smithi090 ceph-mon[29180]: pgmap v503: 97 pgs: 97 active+clean; 360 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 364 op/s 2024-04-26T18:26:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:56 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:resolve 2024-04-26T18:26:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:56 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=c=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:57.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.498+0000 7fa4e8ff9640 1 -- 172.21.15.90:0/3816130622 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55961d27bee0 con 0x55961d390860 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.498+0000 7fa4e8ff9640 10 client.16563 handle_command_reply: tid=0 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 client.16563 shutdown 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 msgr2=0x55961d313500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d313500 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc010ac0 tx=0x7fa4fc010900 comp rx=0 tx=0).stop 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 shutdown_connections 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55961d390860 0x55961d392c80 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55961d3190e0 0x55961d313a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55961d30e820 0x55961d313500 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 --2- 172.21.15.90:0/3816130622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55961d30df00 0x55961d312fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa4e8ff9640 0 client.16563 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:26:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 >> 172.21.15.90:0/3816130622 conn(0x55961d30a410 msgr2=0x55961d316830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 shutdown_connections 2024-04-26T18:26:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 1 -- 172.21.15.90:0/3816130622 wait complete. 2024-04-26T18:26:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.499+0000 7fa50a18a740 20 client.16563 trim_cache size 0 max 16384 2024-04-26T18:26:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:26:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:26:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:26:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:26:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa4ec0a3e40 msgr2=0x7fa4ec0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa4ec0a3e40 0x7fa4ec0a6300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00eed50 tx=0x7fa4f00b2430 comp rx=0 tx=0).stop 2024-04-26T18:26:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500102540 msgr2=0x7fa5001981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500102540 0x7fa5001981a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0005940 tx=0x7fa4f00042f0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 shutdown_connections 2024-04-26T18:26:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fa4ec0a3e40 0x7fa4ec0a6300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5001067d0 0x7fa5001986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa500102540 0x7fa5001981a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 --2- 172.21.15.90:0/3510358593 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa500101b80 0x7fa500197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 >> 172.21.15.90:0/3510358593 conn(0x7fa500077630 msgr2=0x7fa5000ff8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 shutdown_connections 2024-04-26T18:26:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:26:57.500+0000 7fa50739e640 1 -- 172.21.15.90:0/3510358593 wait complete. 2024-04-26T18:26:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:57 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:26:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:57 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:reconnect 2024-04-26T18:26:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:57 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:reconnect 2024-04-26T18:26:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:57 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:57 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109997 2024-04-26T18:26:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:57 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:26:57.817 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:26:57.600+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:26:57.817 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:26:57.601+0000 7f61a8a97700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 6 2024-04-26T18:26:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:26:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:reconnect 2024-04-26T18:26:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:reconnect 2024-04-26T18:26:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109997 2024-04-26T18:26:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:57 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:26:57.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 2948 15.02 MB/sec warmup 22 sec latency 17248.107 ms 2024-04-26T18:26:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:57 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999975 2024-04-26T18:26:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:57 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:reconnect 2024-04-26T18:26:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:57 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:reconnect 2024-04-26T18:26:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:57 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:57 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109997 2024-04-26T18:26:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:57 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:26:58.243+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:26:58.243+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: pgmap v504: 97 pgs: 97 active+clean; 360 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 22 MiB/s wr, 71 op/s 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:rejoin 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:rejoin 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:26:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:active 2024-04-26T18:26:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:active 2024-04-26T18:26:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:26:58 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: pgmap v504: 97 pgs: 97 active+clean; 360 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 22 MiB/s wr, 71 op/s 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:rejoin 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:rejoin 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:active 2024-04-26T18:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:active 2024-04-26T18:26:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:26:58 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:58.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3397 15.99 MB/sec warmup 23 sec latency 68.952 ms 2024-04-26T18:26:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: pgmap v504: 97 pgs: 97 active+clean; 360 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 22 MiB/s wr, 71 op/s 2024-04-26T18:26:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:rejoin 2024-04-26T18:26:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:rejoin 2024-04-26T18:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/114254184,v1:172.21.15.119:6835/114254184] up:active 2024-04-26T18:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/1881734029,v1:172.21.15.167:6837/1881734029] up:active 2024-04-26T18:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:26:58 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:26:59.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3688 16.31 MB/sec warmup 24 sec latency 58.686 ms 2024-04-26T18:27:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:00 smithi119 ceph-mon[32752]: pgmap v505: 97 pgs: 97 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 23 MiB/s wr, 125 op/s 2024-04-26T18:27:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:00 smithi167 ceph-mon[32751]: pgmap v505: 97 pgs: 97 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 23 MiB/s wr, 125 op/s 2024-04-26T18:27:00.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 3818 15.72 MB/sec warmup 25 sec latency 262.764 ms 2024-04-26T18:27:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:00 smithi090 ceph-mon[29180]: pgmap v505: 97 pgs: 97 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 23 MiB/s wr, 125 op/s 2024-04-26T18:27:01.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4028 15.36 MB/sec warmup 26 sec latency 77.859 ms 2024-04-26T18:27:02.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:02 smithi090 ceph-mon[29180]: pgmap v506: 97 pgs: 97 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 125 op/s 2024-04-26T18:27:02.688 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-mon[32752]: pgmap v506: 97 pgs: 97 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 125 op/s 2024-04-26T18:27:02.688 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9f9d0f6700 time 2024-04-26T18:27:02.617121+0000 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9fa771f3dd] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.690 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.690 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.690 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.690 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: clone() 2024-04-26T18:27:02.690 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: *** Caught signal (Aborted) ** 2024-04-26T18:27:02.690 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: in thread 7f9f9d0f6700 thread_name:ms_dispatch 2024-04-26T18:27:02.690 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.617+0000 7f9f9d0f6700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9f9d0f6700 time 2024-04-26T18:27:02.617121+0000 2024-04-26T18:27:02.691 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:02.691 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.691 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.691 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9fa771f3dd] 2024-04-26T18:27:02.691 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.691 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.691 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: clone() 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9fa64a7d20] 2024-04-26T18:27:02.692 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: gsignal() 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: abort() 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9fa771f437] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.693 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 16: clone() 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.620+0000 7f9f9d0f6700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: in thread 7f9f9d0f6700 thread_name:ms_dispatch 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9fa64a7d20] 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: gsignal() 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: abort() 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9fa771f437] 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.694 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 16: clone() 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: -1> 2024-04-26T18:27:02.617+0000 7f9f9d0f6700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9f9d0f6700 time 2024-04-26T18:27:02.617121+0000 2024-04-26T18:27:02.695 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9fa771f3dd] 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.696 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: clone() 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 0> 2024-04-26T18:27:02.620+0000 7f9f9d0f6700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: in thread 7f9f9d0f6700 thread_name:ms_dispatch 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.697 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9fa64a7d20] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: gsignal() 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: abort() 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9fa771f437] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.698 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.699 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.699 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 16: clone() 2024-04-26T18:27:02.699 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:02.699 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.699 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: -1> 2024-04-26T18:27:02.617+0000 7f9f9d0f6700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9f9d0f6700 time 2024-04-26T18:27:02.617121+0000 2024-04-26T18:27:02.699 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:02.700 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.700 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.700 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9fa771f3dd] 2024-04-26T18:27:02.700 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.700 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.701 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: clone() 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 0> 2024-04-26T18:27:02.620+0000 7f9f9d0f6700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: in thread 7f9f9d0f6700 thread_name:ms_dispatch 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9fa64a7d20] 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2: gsignal() 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 3: abort() 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9fa771f437] 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9fa771f5a3] 2024-04-26T18:27:02.702 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55d13c820822] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55d13c790598] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55d13c792c17] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55d13c68bd0b] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55d13c68c2ac] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55d13c674fad] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9fa79b3e20] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 13: (DispatchQueue::entry()+0x51f) [0x7f9fa79b0fdf] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9fa7a83fc1] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9fa649d1ca] 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 16: clone() 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71233]: 2024-04-26T18:27:02.703 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:02 smithi119 podman[71427]: 2024-04-26 18:27:02.689100484 +0000 UTC m=+0.023521038 container died 11cee550334fa1316c49c12d3d101c29c0336e81ba39dcbc5aef8bd41b6601a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:27:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:02 smithi167 ceph-mon[32751]: pgmap v506: 97 pgs: 97 active+clean; 355 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 22 MiB/s wr, 125 op/s 2024-04-26T18:27:02.874 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 14.84 MB/sec warmup 27 sec latency 264.887 ms 2024-04-26T18:27:03.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:03 smithi119 podman[71427]: 2024-04-26 18:27:03.406222618 +0000 UTC m=+0.740643173 container remove 11cee550334fa1316c49c12d3d101c29c0336e81ba39dcbc5aef8bd41b6601a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:27:03.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:03 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:27:03.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:03 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-26T18:27:03.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:03 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.370s CPU time. 2024-04-26T18:27:03.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 14.31 MB/sec warmup 28 sec latency 1264.965 ms 2024-04-26T18:27:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:04 smithi119 ceph-mon[32752]: pgmap v507: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 5.4 MiB/s wr, 177 op/s 2024-04-26T18:27:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:04 smithi167 ceph-mon[32751]: pgmap v507: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 5.4 MiB/s wr, 177 op/s 2024-04-26T18:27:04.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 13.82 MB/sec warmup 29 sec latency 2265.133 ms 2024-04-26T18:27:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:04 smithi090 ceph-mon[29180]: pgmap v507: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 5.4 MiB/s wr, 177 op/s 2024-04-26T18:27:05.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 13.36 MB/sec warmup 30 sec latency 3265.166 ms 2024-04-26T18:27:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:06 smithi119 ceph-mon[32752]: pgmap v508: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-04-26T18:27:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:06 smithi119 ceph-mon[32752]: osdmap e138: 12 total, 12 up, 12 in 2024-04-26T18:27:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:06 smithi167 ceph-mon[32751]: pgmap v508: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-04-26T18:27:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:06 smithi167 ceph-mon[32751]: osdmap e138: 12 total, 12 up, 12 in 2024-04-26T18:27:06.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 12.92 MB/sec warmup 31 sec latency 4265.273 ms 2024-04-26T18:27:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:06 smithi090 ceph-mon[29180]: pgmap v508: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-04-26T18:27:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:06 smithi090 ceph-mon[29180]: osdmap e138: 12 total, 12 up, 12 in 2024-04-26T18:27:07.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 12.52 MB/sec warmup 32 sec latency 5265.375 ms 2024-04-26T18:27:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:08 smithi119 ceph-mon[32752]: pgmap v510: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-26T18:27:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:08 smithi167 ceph-mon[32751]: pgmap v510: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-26T18:27:08.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 12.14 MB/sec warmup 33 sec latency 6265.476 ms 2024-04-26T18:27:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:08 smithi090 ceph-mon[29180]: pgmap v510: 97 pgs: 97 active+clean; 364 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-04-26T18:27:09.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 11.78 MB/sec warmup 34 sec latency 7265.561 ms 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 -- 172.21.15.90:0/1126027316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 msgr2=0x7f136010f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 --2- 172.21.15.90:0/1126027316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f136010f7f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f1358009cd0 tx=0x7f135802a8e0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 -- 172.21.15.90:0/1126027316 shutdown_connections 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 --2- 172.21.15.90:0/1126027316 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f136010f7f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 --2- 172.21.15.90:0/1126027316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136006b2a0 0x7f136006b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 --2- 172.21.15.90:0/1126027316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13601089e0 0x7f1360108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 -- 172.21.15.90:0/1126027316 >> 172.21.15.90:0/1126027316 conn(0x7f1360077b70 msgr2=0x7f1360077f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 -- 172.21.15.90:0/1126027316 shutdown_connections 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.598+0000 7f136521d640 1 -- 172.21.15.90:0/1126027316 wait complete. 2024-04-26T18:27:10.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 Processor -- start 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 -- start start 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136006b2a0 0x7f1360199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f1360199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13601089e0 0x7f136019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1360112be0 con 0x7f136006bc60 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1360112a60 con 0x7f13601089e0 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f136521d640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1360112d60 con 0x7f136006b2a0 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f1357fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f1360199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f1357fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f1360199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38312/0 (socket says 172.21.15.90:38312) 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.599+0000 7f1357fff640 1 -- 172.21.15.90:0/1150020624 learned_addr learned my addr 172.21.15.90:0/1150020624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f135ed9d640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136006b2a0 0x7f1360199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f135f59e640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13601089e0 0x7f136019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f1357fff640 1 -- 172.21.15.90:0/1150020624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136006b2a0 msgr2=0x7f1360199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f1357fff640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136006b2a0 0x7f1360199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f1357fff640 1 -- 172.21.15.90:0/1150020624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13601089e0 msgr2=0x7f136019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f1357fff640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13601089e0 0x7f136019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f1357fff640 1 -- 172.21.15.90:0/1150020624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136019dff0 con 0x7f136006bc60 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f135f59e640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13601089e0 0x7f136019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f1357fff640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f1360199540 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f134800d950 tx=0x7f134800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f135cd99640 1 -- 172.21.15.90:0/1150020624 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348017030 con 0x7f136006bc60 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136019e2e0 con 0x7f136006bc60 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f135cd99640 1 -- 172.21.15.90:0/1150020624 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1348004020 con 0x7f136006bc60 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f135cd99640 1 -- 172.21.15.90:0/1150020624 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480047a0 con 0x7f136006bc60 2024-04-26T18:27:10.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.600+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13601a5b70 con 0x7f136006bc60 2024-04-26T18:27:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.602+0000 7f135cd99640 1 -- 172.21.15.90:0/1150020624 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f1348009a30 con 0x7f136006bc60 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 Processor -- start 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 -- start start 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f135cd99640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f134c0a3dc0 0x7f134c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f135f59e640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f134c0a3dc0 0x7f134c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f135cd99640 1 -- 172.21.15.90:0/1150020624 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 10735+0+0 (secure 0 0 0) 0x7f13480d0b70 con 0x7f136006bc60 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d22825d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22825d820 0x55d228267ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2282680e0 0x55d22826a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d2281c3460 con 0x55d22825cf00 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d22817d590 con 0x55d22825d820 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f136800f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d2281bf420 con 0x55d2282680e0 2024-04-26T18:27:10.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f135ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d22825d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f135ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d22825d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38320/0 (socket says 172.21.15.90:38320) 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.603+0000 7f135ed9d640 1 -- 172.21.15.90:0/456546999 learned_addr learned my addr 172.21.15.90:0/456546999 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f1357fff640 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22825d820 0x55d228267ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135f59e640 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2282680e0 0x55d22826a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135ed9d640 1 -- 172.21.15.90:0/456546999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2282680e0 msgr2=0x55d22826a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135ed9d640 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2282680e0 0x55d22826a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135ed9d640 1 -- 172.21.15.90:0/456546999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22825d820 msgr2=0x55d228267ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135ed9d640 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22825d820 0x55d228267ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135ed9d640 1 -- 172.21.15.90:0/456546999 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d22818a0d0 con 0x55d22825cf00 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135f59e640 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2282680e0 0x55d22826a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:10.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135f59e640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f134c0a3dc0 0x7f134c0a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1358009ca0 tx=0x7f1358009360 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f135ed9d640 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d22825d2e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1350009c70 tx=0x7f1350011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea23b791f5627c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f13557fa640 1 -- 172.21.15.90:0/456546999 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350004230 con 0x55d22825cf00 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f13557fa640 1 -- 172.21.15.90:0/456546999 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13500043d0 con 0x55d22825cf00 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.604+0000 7f13557fa640 1 -- 172.21.15.90:0/456546999 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500196d0 con 0x55d22825cf00 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 -- 172.21.15.90:0/456546999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 msgr2=0x55d22825d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d22825d2e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1350009c70 tx=0x7f1350011810 comp rx=0 tx=0).stop 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 -- 172.21.15.90:0/456546999 shutdown_connections 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d2282680e0 0x55d22826a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d22825d820 0x55d228267ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 --2- 172.21.15.90:0/456546999 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d22825d2e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 -- 172.21.15.90:0/456546999 >> 172.21.15.90:0/456546999 conn(0x55d228259410 msgr2=0x55d2282597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 -- 172.21.15.90:0/456546999 shutdown_connections 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 -- 172.21.15.90:0/456546999 wait complete. 2024-04-26T18:27:10.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 -- start start 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.605+0000 7f136800f740 1 Processor -- start 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228264fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d228265510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228265a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d22826ad10 con 0x55d22825d820 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d22814ab60 con 0x55d2282680e0 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d22826ae90 con 0x55d22825cf00 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 10 client.0 _ll_register_callbacks cb 0x55d22816f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d228265510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d228265510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38322/0 (socket says 172.21.15.90:38322) 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 -- 172.21.15.90:0/2597258988 learned_addr learned my addr 172.21.15.90:0/2597258988 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:10.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f135ed9d640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228264fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f135f59e640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228265a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 -- 172.21.15.90:0/2597258988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 msgr2=0x55d228264fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228264fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 -- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 msgr2=0x55d228265a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228265a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 -- 172.21.15.90:0/2597258988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d227ed5210 con 0x55d22825d820 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f135f59e640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228265a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f1357fff640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d228265510 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f13480cda80 tx=0x7f134809cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1c233d7a2bfeff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f13467fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f13467fc640 1 -- 172.21.15.90:0/2597258988 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b4020 con 0x55d22825d820 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f13467fc640 1 -- 172.21.15.90:0/2597258988 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 10735+0+0 (secure 0 0 0) 0x7f13480041d0 con 0x55d22825d820 2024-04-26T18:27:10.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.606+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13500043d0 con 0x55d22825d820 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f13467fc640 1 -- 172.21.15.90:0/2597258988 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) ==== 24+0+0 (secure 0 0 0) 0x7f13480cf040 con 0x55d22825d820 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 10 client.16575 fetch_fsmap learned FSMap version 181 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=4+}) -- 0x55d227fa1620 con 0x55d22825d820 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f13467fc640 1 -- 172.21.15.90:0/2597258988 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13500043d0 con 0x55d22825d820 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f13467fc640 1 -- 172.21.15.90:0/2597258988 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 181) ==== 7132+0+0 (secure 0 0 0) 0x7f1348064020 con 0x55d22825d820 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 10 client.16575 fetch_fsmap finished waiting for FSMap version 181 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 20 client.16575 populate_metadata read hostname 'smithi090' 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282782e0 0x55d22827a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 4 client.16575 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2281d1070 con 0x55d2282782e0 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.607+0000 7f135ed9d640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282782e0 0x55d22827a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.608+0000 7f13467fc640 10 client.16575 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.608+0000 7f135ed9d640 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282782e0 0x55d22827a720 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ccf1fa99113e6d1 server_cookie=d90f53ca16520587 in_seq=0 out_seq=0 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.608+0000 7f13467fc640 1 -- 172.21.15.90:0/2597258988 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2281d1070 con 0x55d2282782e0 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.608+0000 7f13467fc640 10 client.16575 handle_command_reply: tid=0 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 client.16575 shutdown 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 msgr2=0x55d228265510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d228265510 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f13480cda80 tx=0x7f134809cab0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 shutdown_connections 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282782e0 0x55d22827a720 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228265a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d228265510 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f13467fc640 0 client.16575 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:10.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 --2- 172.21.15.90:0/2597258988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228264fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 >> 172.21.15.90:0/2597258988 conn(0x55d228259410 msgr2=0x55d22825f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 shutdown_connections 2024-04-26T18:27:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 1 -- 172.21.15.90:0/2597258988 wait complete. 2024-04-26T18:27:10.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.609+0000 7f136800f740 20 client.16575 trim_cache size 0 max 16384 2024-04-26T18:27:10.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.623+0000 7f136800f740 1 Processor -- start 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f136800f740 1 -- start start 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f136800f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228273bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f136800f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d22827fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f136800f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228280530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f136800f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f13601b0720 con 0x55d22825d820 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f136800f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f13601b0df0 con 0x55d2282680e0 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f136800f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13601b14c0 con 0x55d22825cf00 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d22827fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d22827fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38330/0 (socket says 172.21.15.90:38330) 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 -- 172.21.15.90:0/300706545 learned_addr learned my addr 172.21.15.90:0/300706545 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:10.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f135ed9d640 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228273bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 -- 172.21.15.90:0/300706545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 msgr2=0x55d228273bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228273bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f135f59e640 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228280530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 -- 172.21.15.90:0/300706545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 msgr2=0x55d228280530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228280530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f1357fff640 1 -- 172.21.15.90:0/300706545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d22826a840 con 0x55d22825d820 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f135ed9d640 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228273bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.624+0000 7f135f59e640 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228280530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:10.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f1357fff640 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d22827fff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f13480ce020 tx=0x7f13480a5060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5296f3231e0e8c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f1354ff9640 1 -- 172.21.15.90:0/300706545 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1348014070 con 0x55d22825d820 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f1354ff9640 1 -- 172.21.15.90:0/300706545 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13480a0620 con 0x55d22825d820 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f1354ff9640 1 -- 172.21.15.90:0/300706545 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b4070 con 0x55d22825d820 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 -- 172.21.15.90:0/300706545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 msgr2=0x55d22827fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d22827fff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f13480ce020 tx=0x7f13480a5060 comp rx=0 tx=0).stop 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 -- 172.21.15.90:0/300706545 shutdown_connections 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228280530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825d820 0x55d22827fff0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 --2- 172.21.15.90:0/300706545 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825cf00 0x55d228273bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 -- 172.21.15.90:0/300706545 >> 172.21.15.90:0/300706545 conn(0x55d228259410 msgr2=0x55d228265820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 -- 172.21.15.90:0/300706545 shutdown_connections 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.625+0000 7f136800f740 1 -- 172.21.15.90:0/300706545 wait complete. 2024-04-26T18:27:10.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 -- start start 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 Processor -- start 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d228261dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825d820 0x55d228262310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228262850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ad9f0 con 0x55d22825cf00 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f13601aede0 con 0x55d2282680e0 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ae040 con 0x55d22825d820 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f136800f740 10 client.0 _ll_register_callbacks cb 0x7f1348004550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d228261dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d228261dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38336/0 (socket says 172.21.15.90:38336) 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 -- 172.21.15.90:0/2764344729 learned_addr learned my addr 172.21.15.90:0/2764344729 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f1357fff640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825d820 0x55d228262310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135f59e640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228262850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 -- 172.21.15.90:0/2764344729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825d820 msgr2=0x55d228262310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825d820 0x55d228262310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 -- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 msgr2=0x55d228262850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228262850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135ed9d640 1 -- 172.21.15.90:0/2764344729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2282dd930 con 0x55d22825cf00 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.626+0000 7f135f59e640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228262850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f135ed9d640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d228261dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1350009c70 tx=0x7f1350005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621aec27a2a226cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f13467fc640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:27:10.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f13467fc640 1 -- 172.21.15.90:0/2764344729 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135002f060 con 0x55d22825cf00 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f13467fc640 1 -- 172.21.15.90:0/2764344729 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 10735+0+0 (secure 0 0 0) 0x7f1350005040 con 0x55d22825cf00 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13480a0620 con 0x55d22825cf00 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f13467fc640 1 -- 172.21.15.90:0/2764344729 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=181) ==== 24+0+0 (secure 0 0 0) 0x7f1350036050 con 0x55d22825cf00 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 10 client.16581 fetch_fsmap learned FSMap version 181 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=181,monmap=4+}) -- 0x55d2282debb0 con 0x55d22825cf00 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f13467fc640 1 -- 172.21.15.90:0/2764344729 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13480a0620 con 0x55d22825cf00 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f13467fc640 1 -- 172.21.15.90:0/2764344729 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 181) ==== 7132+0+0 (secure 0 0 0) 0x7f1350018070 con 0x55d22825cf00 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 10 client.16581 fetch_fsmap finished waiting for FSMap version 181 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 20 client.16581 populate_metadata read hostname 'smithi090' 2024-04-26T18:27:10.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282df2e0 0x55d2282e1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 4 client.16581 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:27:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2281caee0 con 0x55d2282df2e0 2024-04-26T18:27:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.627+0000 7f1357fff640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282df2e0 0x55d2282e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.628+0000 7f1357fff640 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282df2e0 0x55d2282e1700 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b2bb00cb5b8eee0 server_cookie=f749165261907b23 in_seq=0 out_seq=0 2024-04-26T18:27:10.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:10.628+0000 7f13467fc640 10 client.16581 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:10 smithi119 ceph-mon[32752]: pgmap v511: 97 pgs: 97 active+clean; 363 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-26T18:27:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:10 smithi167 ceph-mon[32751]: pgmap v511: 97 pgs: 97 active+clean; 363 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-26T18:27:10.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 11.45 MB/sec warmup 35 sec latency 8265.615 ms 2024-04-26T18:27:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:10 smithi090 ceph-mon[29180]: pgmap v511: 97 pgs: 97 active+clean; 363 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-26T18:27:11.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 11.13 MB/sec warmup 36 sec latency 9265.705 ms 2024-04-26T18:27:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:12 smithi119 ceph-mon[32752]: pgmap v512: 97 pgs: 97 active+clean; 363 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-26T18:27:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:12 smithi167 ceph-mon[32751]: pgmap v512: 97 pgs: 97 active+clean; 363 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-26T18:27:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:12 smithi090 ceph-mon[29180]: pgmap v512: 97 pgs: 97 active+clean; 363 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-26T18:27:12.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 10.83 MB/sec warmup 37 sec latency 10265.799 ms 2024-04-26T18:27:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:13 smithi090 ceph-mon[29180]: osdmap e139: 12 total, 12 up, 12 in 2024-04-26T18:27:13.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 10.54 MB/sec warmup 38 sec latency 11265.839 ms 2024-04-26T18:27:14.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:13 smithi119 ceph-mon[32752]: osdmap e139: 12 total, 12 up, 12 in 2024-04-26T18:27:14.039 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:13 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:27:14.039 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:13 smithi119 systemd[1]: Stopped Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:27:14.039 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:13 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.370s CPU time. 2024-04-26T18:27:14.039 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:13 smithi119 systemd[1]: Starting Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:27:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:13 smithi167 ceph-mon[32751]: osdmap e139: 12 total, 12 up, 12 in 2024-04-26T18:27:14.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 podman[71523]: 2024-04-26 18:27:13.942557895 +0000 UTC m=+0.018445207 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:27:14.304 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 podman[71523]: 2024-04-26 18:27:14.065027994 +0000 UTC m=+0.140915287 container create 17f1726c4621020ac494aaf9962222a93c17112c8a93036a2dbb780aa8ea0f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:27:14.596 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 podman[71523]: 2024-04-26 18:27:14.37573602 +0000 UTC m=+0.451623314 container init 17f1726c4621020ac494aaf9962222a93c17112c8a93036a2dbb780aa8ea0f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:27:14.596 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 podman[71523]: 2024-04-26 18:27:14.378416569 +0000 UTC m=+0.454303873 container start 17f1726c4621020ac494aaf9962222a93c17112c8a93036a2dbb780aa8ea0f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T18:27:14.596 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mds[71542]: -- 172.21.15.119:0/789603310 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c9c9c23a40 con 0x55c9c9bf7000 2024-04-26T18:27:14.597 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: starting mds.e at 2024-04-26T18:27:14.597 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 bash[71523]: 17f1726c4621020ac494aaf9962222a93c17112c8a93036a2dbb780aa8ea0f4f 2024-04-26T18:27:14.597 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:14 smithi119 systemd[1]: Started Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: pgmap v514: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 182 op/s 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: daemon mds.e restarted 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: osdmap e140: 12 total, 12 up, 12 in 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:boot 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:14 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:14.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 10.27 MB/sec warmup 39 sec latency 12265.884 ms 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: pgmap v514: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 182 op/s 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: daemon mds.e restarted 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: osdmap e140: 12 total, 12 up, 12 in 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:boot 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:14 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: pgmap v514: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 182 op/s 2024-04-26T18:27:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: daemon mds.e restarted 2024-04-26T18:27:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:27:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: osdmap e140: 12 total, 12 up, 12 in 2024-04-26T18:27:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:boot 2024-04-26T18:27:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: Standby daemon mds.k assigned to filesystem cephfs as rank 1 2024-04-26T18:27:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=c=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:27:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:27:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:14 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:15.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 10.02 MB/sec warmup 40 sec latency 13265.985 ms 2024-04-26T18:27:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:16 smithi090 ceph-mon[29180]: pgmap v516: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 182 op/s 2024-04-26T18:27:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:16 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:resolve 2024-04-26T18:27:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:16 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:16.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4135 9.77 MB/sec warmup 41 sec latency 14266.032 ms 2024-04-26T18:27:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:16 smithi119 ceph-mon[32752]: pgmap v516: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 182 op/s 2024-04-26T18:27:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:16 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:resolve 2024-04-26T18:27:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:16 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:16 smithi167 ceph-mon[32751]: pgmap v516: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 4.9 MiB/s wr, 182 op/s 2024-04-26T18:27:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:16 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:resolve 2024-04-26T18:27:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:16 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f13467fc640 1 -- 172.21.15.90:0/2764344729 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2281caee0 con 0x55d2282df2e0 2024-04-26T18:27:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f13467fc640 10 client.16581 handle_command_reply: tid=0 2024-04-26T18:27:17.846 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 client.16581 shutdown 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 msgr2=0x55d228261dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d228261dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1350009c70 tx=0x7f1350005d30 comp rx=0 tx=0).stop 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 shutdown_connections 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d2282df2e0 0x55d2282e1700 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2282680e0 0x55d228262850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f13467fc640 0 client.16581 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d22825d820 0x55d228262310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 --2- 172.21.15.90:0/2764344729 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d22825cf00 0x55d228261dd0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.845+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 >> 172.21.15.90:0/2764344729 conn(0x55d228259410 msgr2=0x55d22825e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.846+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 shutdown_connections 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.846+0000 7f136800f740 1 -- 172.21.15.90:0/2764344729 wait complete. 2024-04-26T18:27:17.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.846+0000 7f136800f740 20 client.16581 trim_cache size 0 max 16384 2024-04-26T18:27:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:27:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:27:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:27:17.848 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:27:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.846+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f134c0a3dc0 msgr2=0x7f134c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.846+0000 7f136521d640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f134c0a3dc0 0x7f134c0a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1358009ca0 tx=0x7f1358009360 comp rx=0 tx=0).stop 2024-04-26T18:27:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.846+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 msgr2=0x7f1360199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.846+0000 7f136521d640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f1360199540 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f134800d950 tx=0x7f134800de10 comp rx=0 tx=0).stop 2024-04-26T18:27:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 shutdown_connections 2024-04-26T18:27:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f134c0a3dc0 0x7f134c0a6280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13601089e0 0x7f136019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f136006bc60 0x7f1360199540 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 --2- 172.21.15.90:0/1150020624 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f136006b2a0 0x7f1360199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 >> 172.21.15.90:0/1150020624 conn(0x7f1360077b70 msgr2=0x7f13600fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 shutdown_connections 2024-04-26T18:27:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:17.847+0000 7f136521d640 1 -- 172.21.15.90:0/1150020624 wait complete. 2024-04-26T18:27:17.875 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4182 9.56 MB/sec warmup 42 sec latency 15232.923 ms 2024-04-26T18:27:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:17 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:27:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:17 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:reconnect 2024-04-26T18:27:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:17 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:17 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00499987 2024-04-26T18:27:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:17 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:27:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:17 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:reconnect 2024-04-26T18:27:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:17 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:17 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00499987 2024-04-26T18:27:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:17 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:27:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:17 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:reconnect 2024-04-26T18:27:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:17 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:17 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00499987 2024-04-26T18:27:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:18.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4465 9.50 MB/sec warmup 43 sec latency 16050.504 ms 2024-04-26T18:27:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: pgmap v517: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.9 MiB/s wr, 172 op/s 2024-04-26T18:27:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:rejoin 2024-04-26T18:27:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:27:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: pgmap v518: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 202 op/s 2024-04-26T18:27:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: pgmap v519: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.0 MiB/s wr, 250 op/s 2024-04-26T18:27:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:27:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: pgmap v517: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.9 MiB/s wr, 172 op/s 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:rejoin 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: pgmap v518: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 202 op/s 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: pgmap v519: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.0 MiB/s wr, 250 op/s 2024-04-26T18:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:27:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: pgmap v517: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 4.9 MiB/s wr, 172 op/s 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:rejoin 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: pgmap v518: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 5.7 MiB/s wr, 202 op/s 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: pgmap v519: 97 pgs: 97 active+clean; 400 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 7.0 MiB/s wr, 250 op/s 2024-04-26T18:27:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:27:19.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 4753 9.67 MB/sec warmup 44 sec latency 72.973 ms 2024-04-26T18:27:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:19 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:27:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:19 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:active 2024-04-26T18:27:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:19 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:19 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:27:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:19 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:27:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:19 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:active 2024-04-26T18:27:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:19 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:19 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:27:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:19 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:27:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:19 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/4202449285,v1:172.21.15.119:6833/4202449285] up:active 2024-04-26T18:27:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:19 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:19 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:27:20.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5221 9.98 MB/sec warmup 45 sec latency 70.772 ms 2024-04-26T18:27:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:20 smithi119 ceph-mon[32752]: pgmap v520: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-04-26T18:27:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:20 smithi167 ceph-mon[32751]: pgmap v520: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-04-26T18:27:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:20 smithi090 ceph-mon[29180]: pgmap v520: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-04-26T18:27:21.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 5856 10.17 MB/sec warmup 46 sec latency 112.896 ms 2024-04-26T18:27:22.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6334 10.88 MB/sec warmup 47 sec latency 89.674 ms 2024-04-26T18:27:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:23 smithi119 ceph-mon[32752]: pgmap v521: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-26T18:27:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:23 smithi167 ceph-mon[32751]: pgmap v521: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-26T18:27:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:23 smithi090 ceph-mon[29180]: pgmap v521: 97 pgs: 97 active+clean; 397 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-04-26T18:27:23.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 6886 11.44 MB/sec warmup 48 sec latency 59.452 ms 2024-04-26T18:27:24.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7125 11.64 MB/sec warmup 49 sec latency 96.533 ms 2024-04-26T18:27:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:25 smithi119 ceph-mon[32752]: pgmap v522: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.5 MiB/s wr, 382 op/s 2024-04-26T18:27:25.410 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-mon[32751]: pgmap v522: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.5 MiB/s wr, 382 op/s 2024-04-26T18:27:25.410 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9c6f8bd700 time 2024-04-26T18:27:25.380187+0000 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9c79ee63dd] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.411 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.412 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.412 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: clone() 2024-04-26T18:27:25.412 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.381+0000 7f9c6f8bd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9c6f8bd700 time 2024-04-26T18:27:25.380187+0000 2024-04-26T18:27:25.412 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:25.412 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9c79ee63dd] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.413 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: clone() 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: *** Caught signal (Aborted) ** 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: in thread 7f9c6f8bd700 thread_name:ms_dispatch 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9c78c6ed20] 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: gsignal() 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: abort() 2024-04-26T18:27:25.414 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9c79ee6437] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 16: clone() 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.382+0000 7f9c6f8bd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:25.415 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: in thread 7f9c6f8bd700 thread_name:ms_dispatch 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9c78c6ed20] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: gsignal() 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: abort() 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9c79ee6437] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.416 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 16: clone() 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: -1> 2024-04-26T18:27:25.381+0000 7f9c6f8bd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9c6f8bd700 time 2024-04-26T18:27:25.380187+0000 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9c79ee63dd] 2024-04-26T18:27:25.417 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.418 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.418 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.418 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.418 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.418 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.418 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: clone() 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 0> 2024-04-26T18:27:25.382+0000 7f9c6f8bd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: in thread 7f9c6f8bd700 thread_name:ms_dispatch 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.419 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9c78c6ed20] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: gsignal() 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: abort() 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9c79ee6437] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.420 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 16: clone() 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: -1> 2024-04-26T18:27:25.381+0000 7f9c6f8bd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9c6f8bd700 time 2024-04-26T18:27:25.380187+0000 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f9c79ee63dd] 2024-04-26T18:27:25.421 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.422 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.422 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:25 smithi090 ceph-mon[29180]: pgmap v522: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.5 MiB/s wr, 382 op/s 2024-04-26T18:27:25.422 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: clone() 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 0> 2024-04-26T18:27:25.382+0000 7f9c6f8bd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: in thread 7f9c6f8bd700 thread_name:ms_dispatch 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.423 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f9c78c6ed20] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2: gsignal() 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 3: abort() 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f9c79ee6437] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f9c79ee65a3] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b82e66f822] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b82e5df598] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b82e5e1c17] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b82e4dad0b] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b82e4db2ac] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b82e4c3fad] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f9c7a17ae20] 2024-04-26T18:27:25.424 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 13: (DispatchQueue::entry()+0x51f) [0x7f9c7a177fdf] 2024-04-26T18:27:25.425 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f9c7a24afc1] 2024-04-26T18:27:25.425 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f9c78c641ca] 2024-04-26T18:27:25.425 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 16: clone() 2024-04-26T18:27:25.425 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:25.425 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[71686]: 2024-04-26T18:27:25.764 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:25 smithi167 podman[72043]: 2024-04-26 18:27:25.444244971 +0000 UTC m=+0.018598582 container died fbca98285fcf1237d044cda3df9fbb9baa04e4d60a411ce131992f0eb0517bd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:27:25.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 11.63 MB/sec warmup 50 sec latency 517.580 ms 2024-04-26T18:27:26.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:26 smithi167 podman[72043]: 2024-04-26 18:27:26.158500902 +0000 UTC m=+0.732854558 container remove fbca98285fcf1237d044cda3df9fbb9baa04e4d60a411ce131992f0eb0517bd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T18:27:26.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:26 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:27:26.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:26 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-26T18:27:26.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:26 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 8.786s CPU time. 2024-04-26T18:27:26.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 11.40 MB/sec warmup 51 sec latency 1517.646 ms 2024-04-26T18:27:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:27 smithi119 ceph-mon[32752]: pgmap v523: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.5 MiB/s wr, 382 op/s 2024-04-26T18:27:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:27 smithi167 ceph-mon[32751]: pgmap v523: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.5 MiB/s wr, 382 op/s 2024-04-26T18:27:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:27 smithi090 ceph-mon[29180]: pgmap v523: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.5 MiB/s wr, 382 op/s 2024-04-26T18:27:27.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 11.18 MB/sec warmup 52 sec latency 2517.692 ms 2024-04-26T18:27:28.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 10.97 MB/sec warmup 53 sec latency 3517.786 ms 2024-04-26T18:27:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:29 smithi119 ceph-mon[32752]: pgmap v524: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-26T18:27:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:29 smithi167 ceph-mon[32751]: pgmap v524: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-26T18:27:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:29 smithi090 ceph-mon[29180]: pgmap v524: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.1 MiB/s wr, 337 op/s 2024-04-26T18:27:29.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 10.77 MB/sec warmup 54 sec latency 4517.851 ms 2024-04-26T18:27:30.876 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 10.57 MB/sec warmup 55 sec latency 5517.916 ms 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 -- 172.21.15.90:0/1080843818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 msgr2=0x7fe938102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/1080843818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe938102dc0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe924009930 tx=0x7fe92402a420 comp rx=0 tx=0).stop 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 -- 172.21.15.90:0/1080843818 shutdown_connections 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/1080843818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe938103300 0x7fe93810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/1080843818 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe938102dc0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/1080843818 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe938108950 0x7fe938108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 -- 172.21.15.90:0/1080843818 >> 172.21.15.90:0/1080843818 conn(0x7fe938079dc0 msgr2=0x7fe93807a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:30.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 -- 172.21.15.90:0/1080843818 shutdown_connections 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.943+0000 7fe93ebaf640 1 -- 172.21.15.90:0/1080843818 wait complete. 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 Processor -- start 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 -- start start 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe93819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe938103300 0x7fe93819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe938108950 0x7fe93819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe938112be0 con 0x7fe938102940 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe938112a60 con 0x7fe938108950 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93ebaf640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe938112d60 con 0x7fe938103300 2024-04-26T18:27:30.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe93819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe93819b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44864/0 (socket says 172.21.15.90:44864) 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 -- 172.21.15.90:0/418708393 learned_addr learned my addr 172.21.15.90:0/418708393 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93d14c640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe938108950 0x7fe93819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe937fff640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe938103300 0x7fe93819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 -- 172.21.15.90:0/418708393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe938103300 msgr2=0x7fe93819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe938103300 0x7fe93819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 -- 172.21.15.90:0/418708393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe938108950 msgr2=0x7fe93819fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe938108950 0x7fe93819fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 -- 172.21.15.90:0/418708393 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9381a0260 con 0x7fe938102940 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.944+0000 7fe93c94b640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe93819b270 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00d950 tx=0x7fe92c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.945+0000 7fe935ffb640 1 -- 172.21.15.90:0/418708393 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c017070 con 0x7fe938102940 2024-04-26T18:27:30.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.945+0000 7fe935ffb640 1 -- 172.21.15.90:0/418708393 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c009c30 con 0x7fe938102940 2024-04-26T18:27:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.945+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9381a0550 con 0x7fe938102940 2024-04-26T18:27:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.945+0000 7fe935ffb640 1 -- 172.21.15.90:0/418708393 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00f620 con 0x7fe938102940 2024-04-26T18:27:30.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.945+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9381a7de0 con 0x7fe938102940 2024-04-26T18:27:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.947+0000 7fe935ffb640 1 -- 172.21.15.90:0/418708393 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fe92c010460 con 0x7fe938102940 2024-04-26T18:27:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 Processor -- start 2024-04-26T18:27:30.955 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 -- start start 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe935ffb640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe9280a3dc0 0x7fe9280a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f75f00 0x563c06f762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f80ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563c06edc460 con 0x563c06f75f00 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563c06e96590 con 0x563c06f810e0 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe9419a3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c06ed8420 con 0x563c06f76820 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93d14c640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe9280a3dc0 0x7fe9280a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe935ffb640 1 -- 172.21.15.90:0/418708393 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 10733+0+0 (secure 0 0 0) 0x7fe92c0d0c00 con 0x7fe938102940 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f75f00 0x563c06f762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f75f00 0x563c06f762e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44866/0 (socket says 172.21.15.90:44866) 2024-04-26T18:27:30.956 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 -- 172.21.15.90:0/1236180128 learned_addr learned my addr 172.21.15.90:0/1236180128 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93d14c640 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe937fff640 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f80ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 -- 172.21.15.90:0/1236180128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 msgr2=0x563c06f80ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f80ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 -- 172.21.15.90:0/1236180128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 msgr2=0x563c06f834d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f834d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe93c94b640 1 -- 172.21.15.90:0/1236180128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c06ea30d0 con 0x563c06f75f00 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.948+0000 7fe937fff640 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f80ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe93d14c640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe9280a3dc0 0x7fe9280a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe930005fd0 tx=0x7fe930013b60 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe93c94b640 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f75f00 0x563c06f762e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0cdab0 tx=0x7fe92c09bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cac44cc7101ee45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe922ffd640 1 -- 172.21.15.90:0/1236180128 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c020070 con 0x563c06f75f00 2024-04-26T18:27:30.957 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe922ffd640 1 -- 172.21.15.90:0/1236180128 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c09c350 con 0x563c06f75f00 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe922ffd640 1 -- 172.21.15.90:0/1236180128 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c080210 con 0x563c06f75f00 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe9419a3740 1 -- 172.21.15.90:0/1236180128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f75f00 msgr2=0x563c06f762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe9419a3740 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f75f00 0x563c06f762e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0cdab0 tx=0x7fe92c09bfc0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe9419a3740 1 -- 172.21.15.90:0/1236180128 shutdown_connections 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe9419a3740 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f834d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe9419a3740 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f80ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.949+0000 7fe9419a3740 1 --2- 172.21.15.90:0/1236180128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f75f00 0x563c06f762e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 -- 172.21.15.90:0/1236180128 >> 172.21.15.90:0/1236180128 conn(0x563c06f72410 msgr2=0x563c06f727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 -- 172.21.15.90:0/1236180128 shutdown_connections 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 -- 172.21.15.90:0/1236180128 wait complete. 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 -- start start 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 Processor -- start 2024-04-26T18:27:30.958 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f7e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f810e0 0x563c06f7eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563c06f83cb0 con 0x563c06f76820 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563c06e63b60 con 0x563c06f75f00 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.950+0000 7fe9419a3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c06f83e30 con 0x563c06f810e0 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe9419a3740 10 client.0 _ll_register_callbacks cb 0x563c06e882b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f7e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f7e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44882/0 (socket says 172.21.15.90:44882) 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 -- 172.21.15.90:0/2626991360 learned_addr learned my addr 172.21.15.90:0/2626991360 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe93c94b640 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe93d14c640 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f810e0 0x563c06f7eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 -- 172.21.15.90:0/2626991360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f810e0 msgr2=0x563c06f7eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.959 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f810e0 0x563c06f7eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 -- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 msgr2=0x563c06f7e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 -- 172.21.15.90:0/2626991360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c06bee210 con 0x563c06f76820 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe937fff640 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f7e610 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe92402a940 tx=0x7fe924031e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b516697ed3dd91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe917fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe917fff640 1 -- 172.21.15.90:0/2626991360 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035040 con 0x563c06f76820 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe917fff640 1 -- 172.21.15.90:0/2626991360 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 10733+0+0 (secure 0 0 0) 0x7fe924031760 con 0x563c06f76820 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92c09c350 con 0x563c06f76820 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe917fff640 1 -- 172.21.15.90:0/2626991360 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) ==== 24+0+0 (secure 0 0 0) 0x7fe92404e050 con 0x563c06f76820 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe9419a3740 10 client.16593 fetch_fsmap learned FSMap version 188 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.951+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=4+}) -- 0x563c06cba620 con 0x563c06f76820 2024-04-26T18:27:30.960 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe917fff640 1 -- 172.21.15.90:0/2626991360 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c09c350 con 0x563c06f76820 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe917fff640 1 -- 172.21.15.90:0/2626991360 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 188) ==== 7132+0+0 (secure 0 0 0) 0x7fe924003b10 con 0x563c06f76820 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe9419a3740 10 client.16593 fetch_fsmap finished waiting for FSMap version 188 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe9419a3740 10 client.16593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe9419a3740 20 client.16593 populate_metadata read hostname 'smithi090' 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe9419a3740 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06f91210 0x563c06f93620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe9419a3740 4 client.16593 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c06eea070 con 0x563c06f91210 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe93c94b640 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06f91210 0x563c06f93620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe93c94b640 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06f91210 0x563c06f93620 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebf99905ef7811ec server_cookie=ebb8b9bfaa7673c8 in_seq=0 out_seq=0 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.952+0000 7fe917fff640 10 client.16593 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe917fff640 1 -- 172.21.15.90:0/2626991360 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c06eea070 con 0x563c06f91210 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe917fff640 10 client.16593 handle_command_reply: tid=0 2024-04-26T18:27:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 client.16593 shutdown 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 msgr2=0x563c06f7e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f7e610 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe92402a940 tx=0x7fe924031e10 comp rx=0 tx=0).stop 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 shutdown_connections 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06f91210 0x563c06f93620 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f810e0 0x563c06f7eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f7e610 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe917fff640 0 client.16593 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 --2- 172.21.15.90:0/2626991360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 >> 172.21.15.90:0/2626991360 conn(0x563c06f72410 msgr2=0x563c06f78190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 shutdown_connections 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.953+0000 7fe9419a3740 1 -- 172.21.15.90:0/2626991360 wait complete. 2024-04-26T18:27:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.954+0000 7fe9419a3740 20 client.16593 trim_cache size 0 max 16384 2024-04-26T18:27:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 Processor -- start 2024-04-26T18:27:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 -- start start 2024-04-26T18:27:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f75f00 0x563c06f8cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f788c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f71430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe9381b2970 con 0x563c06f76820 2024-04-26T18:27:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9381b3040 con 0x563c06f810e0 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.968+0000 7fe9419a3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9381b3710 con 0x563c06f75f00 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f788c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f788c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44896/0 (socket says 172.21.15.90:44896) 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 -- 172.21.15.90:0/854470895 learned_addr learned my addr 172.21.15.90:0/854470895 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe93d14c640 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f71430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe93c94b640 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f75f00 0x563c06f8cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 -- 172.21.15.90:0/854470895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f75f00 msgr2=0x563c06f8cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f75f00 0x563c06f8cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 -- 172.21.15.90:0/854470895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 msgr2=0x563c06f71430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f71430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 -- 172.21.15.90:0/854470895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c06f771b0 con 0x563c06f76820 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe937fff640 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f788c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe924001eb0 tx=0x7fe924009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd697384f6e4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe9227fc640 1 -- 172.21.15.90:0/854470895 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035030 con 0x563c06f76820 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe9227fc640 1 -- 172.21.15.90:0/854470895 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9240455a0 con 0x563c06f76820 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.969+0000 7fe9227fc640 1 -- 172.21.15.90:0/854470895 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924003fe0 con 0x563c06f76820 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 -- 172.21.15.90:0/854470895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 msgr2=0x563c06f788c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f788c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe924001eb0 tx=0x7fe924009680 comp rx=0 tx=0).stop 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 -- 172.21.15.90:0/854470895 shutdown_connections 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f810e0 0x563c06f71430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f76820 0x563c06f788c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 --2- 172.21.15.90:0/854470895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f75f00 0x563c06f8cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 -- 172.21.15.90:0/854470895 >> 172.21.15.90:0/854470895 conn(0x563c06f72410 msgr2=0x563c06f70d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 -- 172.21.15.90:0/854470895 shutdown_connections 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 -- 172.21.15.90:0/854470895 wait complete. 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 -- start start 2024-04-26T18:27:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.970+0000 7fe9419a3740 1 Processor -- start 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f7b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f810e0 0x563c06f7b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe9381afc20 con 0x563c06f810e0 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe9381b1030 con 0x563c06f75f00 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe9381b0290 con 0x563c06f76820 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 10 client.0 _ll_register_callbacks cb 0x7fe924009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f810e0 0x563c06f7b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f810e0 0x563c06f7b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44906/0 (socket says 172.21.15.90:44906) 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 -- 172.21.15.90:0/3459799872 learned_addr learned my addr 172.21.15.90:0/3459799872 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93c94b640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 -- 172.21.15.90:0/3459799872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 msgr2=0x563c06f7b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe937fff640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f7b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f7b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 -- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 msgr2=0x563c06f7acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 -- 172.21.15.90:0/3459799872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c06ff67a0 con 0x563c06f810e0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93c94b640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe93d14c640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f810e0 0x563c06f7b730 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe930003c50 tx=0x7fe930004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42fe97b9a53d1370 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe917fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe917fff640 1 -- 172.21.15.90:0/3459799872 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93002a430 con 0x563c06f810e0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe917fff640 1 -- 172.21.15.90:0/3459799872 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 10733+0+0 (secure 0 0 0) 0x7fe93002a5d0 con 0x563c06f810e0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.971+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9240455a0 con 0x563c06f810e0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe917fff640 1 -- 172.21.15.90:0/3459799872 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=188) ==== 24+0+0 (secure 0 0 0) 0x7fe930032050 con 0x563c06f810e0 2024-04-26T18:27:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 10 client.16599 fetch_fsmap learned FSMap version 188 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=188,monmap=4+}) -- 0x563c06ff79a0 con 0x563c06f810e0 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe917fff640 1 -- 172.21.15.90:0/3459799872 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9240455a0 con 0x563c06f810e0 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe917fff640 1 -- 172.21.15.90:0/3459799872 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 188) ==== 7132+0+0 (secure 0 0 0) 0x7fe93001f070 con 0x563c06f810e0 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 10 client.16599 fetch_fsmap finished waiting for FSMap version 188 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 20 client.16599 populate_metadata read hostname 'smithi090' 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06ff81b0 0x563c06ffa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 4 client.16599 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:27:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c06ee3ee0 con 0x563c06ff81b0 2024-04-26T18:27:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.972+0000 7fe93c94b640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06ff81b0 0x563c06ffa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.973+0000 7fe93c94b640 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06ff81b0 0x563c06ffa5f0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c82b2b0ff3a95d6 server_cookie=64d7e93022bea78f in_seq=0 out_seq=0 2024-04-26T18:27:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:30.973+0000 7fe917fff640 10 client.16599 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:31 smithi119 ceph-mon[32752]: pgmap v525: 97 pgs: 97 active+clean; 416 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-04-26T18:27:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:31 smithi167 ceph-mon[32751]: pgmap v525: 97 pgs: 97 active+clean; 416 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-04-26T18:27:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:31 smithi090 ceph-mon[29180]: pgmap v525: 97 pgs: 97 active+clean; 416 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.9 MiB/s wr, 325 op/s 2024-04-26T18:27:31.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 10.38 MB/sec warmup 56 sec latency 6518.026 ms 2024-04-26T18:27:32.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 10.20 MB/sec warmup 57 sec latency 7518.079 ms 2024-04-26T18:27:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:33 smithi119 ceph-mon[32752]: pgmap v526: 97 pgs: 97 active+clean; 416 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-04-26T18:27:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:33 smithi167 ceph-mon[32751]: pgmap v526: 97 pgs: 97 active+clean; 416 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-04-26T18:27:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:33 smithi090 ceph-mon[29180]: pgmap v526: 97 pgs: 97 active+clean; 416 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.0 MiB/s wr, 256 op/s 2024-04-26T18:27:33.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 10.03 MB/sec warmup 58 sec latency 8518.210 ms 2024-04-26T18:27:34.409 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fd6c24cd700 time 2024-04-26T18:27:34.133399+0000 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd6ccaf63dd] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.410 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.411 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.411 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.411 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.411 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: clone() 2024-04-26T18:27:34.411 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.134+0000 7fd6c24cd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fd6c24cd700 time 2024-04-26T18:27:34.133399+0000 2024-04-26T18:27:34.411 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:34.411 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.412 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.412 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd6ccaf63dd] 2024-04-26T18:27:34.412 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.412 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: clone() 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.413 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: *** Caught signal (Aborted) ** 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: in thread 7fd6c24cd700 thread_name:ms_dispatch 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd6cb87ed20] 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: gsignal() 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: abort() 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd6ccaf6437] 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.414 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 16: clone() 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.135+0000 7fd6c24cd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: in thread 7fd6c24cd700 thread_name:ms_dispatch 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd6cb87ed20] 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: gsignal() 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: abort() 2024-04-26T18:27:34.415 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd6ccaf6437] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 16: clone() 2024-04-26T18:27:34.416 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: -1> 2024-04-26T18:27:34.134+0000 7fd6c24cd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fd6c24cd700 time 2024-04-26T18:27:34.133399+0000 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd6ccaf63dd] 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.417 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: clone() 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 0> 2024-04-26T18:27:34.135+0000 7fd6c24cd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: in thread 7fd6c24cd700 thread_name:ms_dispatch 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.418 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd6cb87ed20] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: gsignal() 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: abort() 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd6ccaf6437] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.419 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 16: clone() 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: -2> 2024-04-26T18:27:34.134+0000 7fd6c24cd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fd6c24cd700 time 2024-04-26T18:27:34.133399+0000 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fd6ccaf63dd] 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.420 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: clone() 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: -1> 2024-04-26T18:27:34.135+0000 7fd6c24cd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: in thread 7fd6c24cd700 thread_name:ms_dispatch 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:34.421 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fd6cb87ed20] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2: gsignal() 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 3: abort() 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fd6ccaf6437] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fd6ccaf65a3] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5635e6edc822] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5635e6e4c598] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5635e6e4ec17] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5635e6d47d0b] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5635e6d482ac] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5635e6d30fad] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fd6ccd8ae20] 2024-04-26T18:27:34.422 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 13: (DispatchQueue::entry()+0x51f) [0x7fd6ccd87fdf] 2024-04-26T18:27:34.423 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fd6cce5afc1] 2024-04-26T18:27:34.423 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fd6cb8741ca] 2024-04-26T18:27:34.423 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 16: clone() 2024-04-26T18:27:34.423 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:34.423 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71387]: 2024-04-26T18:27:34.423 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:34 smithi119 podman[71720]: 2024-04-26 18:27:34.197693753 +0000 UTC m=+0.020092387 container died 8a6d37d0cefedd1f77685ba01fc0b032bd19b4e80ffb9bdc0a8ad5ba68c42dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:27:34.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 9.86 MB/sec warmup 59 sec latency 9518.260 ms 2024-04-26T18:27:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:35 smithi119 ceph-mon[32752]: pgmap v527: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-04-26T18:27:35.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:35 smithi119 podman[71720]: 2024-04-26 18:27:35.091245224 +0000 UTC m=+0.913643855 container remove 8a6d37d0cefedd1f77685ba01fc0b032bd19b4e80ffb9bdc0a8ad5ba68c42dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T18:27:35.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:35 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:27:35.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:35 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-26T18:27:35.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:35 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 4.783s CPU time. 2024-04-26T18:27:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:35 smithi167 ceph-mon[32751]: pgmap v527: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-04-26T18:27:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:35 smithi090 ceph-mon[29180]: pgmap v527: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 399 op/s 2024-04-26T18:27:35.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 9.69 MB/sec warmup 60 sec latency 10518.375 ms 2024-04-26T18:27:36.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 9.53 MB/sec warmup 61 sec latency 11518.479 ms 2024-04-26T18:27:37.028 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:36 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:27:37.028 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:36 smithi167 systemd[1]: Stopped Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:27:37.028 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:36 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 8.786s CPU time. 2024-04-26T18:27:37.028 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:36 smithi167 systemd[1]: Starting Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:27:37.028 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:36 smithi167 podman[72145]: 2024-04-26 18:27:36.708307989 +0000 UTC m=+0.018807259 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:27:37.028 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:36 smithi167 podman[72145]: 2024-04-26 18:27:36.817548794 +0000 UTC m=+0.128048096 container create 819c6d46945e06e87bf21d4900cfd0dd36d68c86cb88ba09ddb0845f281dd202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD) 2024-04-26T18:27:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:37 smithi167 ceph-mon[32751]: pgmap v528: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 187 op/s 2024-04-26T18:27:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:37 smithi167 ceph-mon[32751]: daemon mds.c restarted 2024-04-26T18:27:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:37 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:27:37.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:37 smithi167 podman[72145]: 2024-04-26 18:27:37.128649096 +0000 UTC m=+0.439148354 container init 819c6d46945e06e87bf21d4900cfd0dd36d68c86cb88ba09ddb0845f281dd202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T18:27:37.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:37 smithi167 podman[72145]: 2024-04-26 18:27:37.131404901 +0000 UTC m=+0.441904167 container start 819c6d46945e06e87bf21d4900cfd0dd36d68c86cb88ba09ddb0845f281dd202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:27:37.318 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:37 smithi167 ceph-mds[72164]: -- 172.21.15.167:0/4125701280 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55599b263a40 con 0x55599b236800 2024-04-26T18:27:37.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: starting mds.c at 2024-04-26T18:27:37.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:37 smithi167 bash[72145]: 819c6d46945e06e87bf21d4900cfd0dd36d68c86cb88ba09ddb0845f281dd202 2024-04-26T18:27:37.319 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:27:37 smithi167 systemd[1]: Started Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:27:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:37 smithi119 ceph-mon[32752]: pgmap v528: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 187 op/s 2024-04-26T18:27:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:37 smithi119 ceph-mon[32752]: daemon mds.c restarted 2024-04-26T18:27:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:37 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:27:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:37 smithi090 ceph-mon[29180]: pgmap v528: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.2 MiB/s wr, 187 op/s 2024-04-26T18:27:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:37 smithi090 ceph-mon[29180]: daemon mds.c restarted 2024-04-26T18:27:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:37 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:27:37.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 9.38 MB/sec warmup 62 sec latency 12518.541 ms 2024-04-26T18:27:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:38 smithi119 ceph-mon[32752]: osdmap e141: 12 total, 12 up, 12 in 2024-04-26T18:27:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:boot 2024-04-26T18:27:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:38 smithi119 ceph-mon[32752]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:27:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:38 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:27:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:27:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:38 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=l=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:38 smithi167 ceph-mon[32751]: osdmap e141: 12 total, 12 up, 12 in 2024-04-26T18:27:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:boot 2024-04-26T18:27:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:38 smithi167 ceph-mon[32751]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:27:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:38 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:27:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:27:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:38 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=l=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:38.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:38 smithi090 ceph-mon[29180]: osdmap e141: 12 total, 12 up, 12 in 2024-04-26T18:27:38.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:boot 2024-04-26T18:27:38.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:38 smithi090 ceph-mon[29180]: Standby daemon mds.l assigned to filesystem cephfs as rank 2 2024-04-26T18:27:38.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:38 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:27:38.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:27:38.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:38 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=l=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:38.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 9.23 MB/sec warmup 63 sec latency 13518.629 ms 2024-04-26T18:27:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:39 smithi119 ceph-mon[32752]: pgmap v530: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-04-26T18:27:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:resolve 2024-04-26T18:27:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:39 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=l=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:39 smithi167 ceph-mon[32751]: pgmap v530: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-04-26T18:27:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:resolve 2024-04-26T18:27:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:39 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=l=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:39 smithi090 ceph-mon[29180]: pgmap v530: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 224 op/s 2024-04-26T18:27:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:resolve 2024-04-26T18:27:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:39 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=l=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:27:39.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 9.09 MB/sec warmup 64 sec latency 14518.715 ms 2024-04-26T18:27:40.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.95 MB/sec warmup 65 sec latency 15518.830 ms 2024-04-26T18:27:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:41 smithi119 ceph-mon[32752]: pgmap v531: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-26T18:27:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:41 smithi167 ceph-mon[32751]: pgmap v531: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-26T18:27:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:41 smithi090 ceph-mon[29180]: pgmap v531: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-26T18:27:41.877 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.81 MB/sec warmup 66 sec latency 16518.945 ms 2024-04-26T18:27:42.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.68 MB/sec warmup 67 sec latency 17519.037 ms 2024-04-26T18:27:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:43 smithi119 ceph-mon[32752]: pgmap v532: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-26T18:27:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:43 smithi167 ceph-mon[32751]: pgmap v532: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-26T18:27:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:43 smithi090 ceph-mon[29180]: pgmap v532: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 1.1 MiB/s wr, 193 op/s 2024-04-26T18:27:43.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.55 MB/sec warmup 68 sec latency 18519.131 ms 2024-04-26T18:27:44.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.43 MB/sec warmup 69 sec latency 19519.250 ms 2024-04-26T18:27:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:45 smithi119 ceph-mon[32752]: pgmap v533: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:45 smithi167 ceph-mon[32751]: pgmap v533: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:45 smithi090 ceph-mon[29180]: pgmap v533: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:45.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.31 MB/sec warmup 70 sec latency 20519.295 ms 2024-04-26T18:27:45.906 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:45 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:27:45.906 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:45 smithi119 systemd[1]: Stopped Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:27:45.907 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:45 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 4.783s CPU time. 2024-04-26T18:27:45.907 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:45 smithi119 systemd[1]: Starting Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:27:45.907 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:45 smithi119 podman[71820]: 2024-04-26 18:27:45.792583088 +0000 UTC m=+0.110282472 container create 079d1ed4e668fab0d071da2ace3f95f0869cdf020181dbba891474f5850582d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T18:27:45.907 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:45 smithi119 podman[71820]: 2024-04-26 18:27:45.701181499 +0000 UTC m=+0.018880897 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:27:46.199 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:46 smithi119 podman[71820]: 2024-04-26 18:27:46.104010592 +0000 UTC m=+0.421709978 container init 079d1ed4e668fab0d071da2ace3f95f0869cdf020181dbba891474f5850582d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T18:27:46.199 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:46 smithi119 podman[71820]: 2024-04-26 18:27:46.106523456 +0000 UTC m=+0.424222835 container start 079d1ed4e668fab0d071da2ace3f95f0869cdf020181dbba891474f5850582d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-26T18:27:46.199 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mds[71839]: -- 172.21.15.119:0/3076484547 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca444c3a40 con 0x55ca44496c00 2024-04-26T18:27:46.199 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: starting mds.k at 2024-04-26T18:27:46.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:46 smithi119 bash[71820]: 079d1ed4e668fab0d071da2ace3f95f0869cdf020181dbba891474f5850582d3 2024-04-26T18:27:46.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:27:46 smithi119 systemd[1]: Started Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:27:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mon[32752]: daemon mds.k restarted 2024-04-26T18:27:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mon[32752]: osdmap e142: 12 total, 12 up, 12 in 2024-04-26T18:27:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:boot 2024-04-26T18:27:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mon[32752]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:27:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=l=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:27:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:27:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:46 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:46 smithi167 ceph-mon[32751]: daemon mds.k restarted 2024-04-26T18:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:46 smithi167 ceph-mon[32751]: osdmap e142: 12 total, 12 up, 12 in 2024-04-26T18:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:46 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:boot 2024-04-26T18:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:46 smithi167 ceph-mon[32751]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:46 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=l=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:46 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:46 smithi090 ceph-mon[29180]: daemon mds.k restarted 2024-04-26T18:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:46 smithi090 ceph-mon[29180]: osdmap e142: 12 total, 12 up, 12 in 2024-04-26T18:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:46 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:boot 2024-04-26T18:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:46 smithi090 ceph-mon[29180]: Standby daemon mds.e assigned to filesystem cephfs as rank 1 2024-04-26T18:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:46 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=l=up:resolve} 2 up:standby-replay 6 up:standby, 1 failed 2024-04-26T18:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:46 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:replay,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:46.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.19 MB/sec warmup 71 sec latency 21519.449 ms 2024-04-26T18:27:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:47 smithi119 ceph-mon[32752]: pgmap v534: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:47 smithi167 ceph-mon[32751]: pgmap v534: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:47 smithi090 ceph-mon[29180]: pgmap v534: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:47.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 8.08 MB/sec warmup 72 sec latency 22519.567 ms 2024-04-26T18:27:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:48 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:resolve 2024-04-26T18:27:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:48 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:48 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:resolve 2024-04-26T18:27:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:48 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:48 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:resolve 2024-04-26T18:27:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:48 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:resolve,2=l=up:resolve} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:48.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7249 7.97 MB/sec warmup 73 sec latency 23519.694 ms 2024-04-26T18:27:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.341+0000 7fe917fff640 1 -- 172.21.15.90:0/3459799872 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c06ee3ee0 con 0x563c06ff81b0 2024-04-26T18:27:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.341+0000 7fe917fff640 10 client.16599 handle_command_reply: tid=0 2024-04-26T18:27:49.342 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.341+0000 7fe9419a3740 1 client.16599 shutdown 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f810e0 msgr2=0x563c06f7b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f810e0 0x563c06f7b730 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe930003c50 tx=0x7fe930004ab0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 shutdown_connections 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x563c06ff81b0 0x563c06ffa5f0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563c06f810e0 0x563c06f7b730 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe917fff640 0 client.16599 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c06f76820 0x563c06f7b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 --2- 172.21.15.90:0/3459799872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563c06f75f00 0x563c06f7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 >> 172.21.15.90:0/3459799872 conn(0x563c06f72410 msgr2=0x563c06f715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 shutdown_connections 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 1 -- 172.21.15.90:0/3459799872 wait complete. 2024-04-26T18:27:49.343 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.342+0000 7fe9419a3740 20 client.16599 trim_cache size 0 max 16384 2024-04-26T18:27:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:27:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:27:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:27:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe9280a3dc0 msgr2=0x7fe9280a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe9280a3dc0 0x7fe9280a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe930005fd0 tx=0x7fe930013b60 comp rx=0 tx=0).stop 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 msgr2=0x7fe93819b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe93819b270 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00d950 tx=0x7fe92c00de10 comp rx=0 tx=0).stop 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 shutdown_connections 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe9280a3dc0 0x7fe9280a6280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe938108950 0x7fe93819fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe938103300 0x7fe93819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 --2- 172.21.15.90:0/418708393 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe938102940 0x7fe93819b270 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 >> 172.21.15.90:0/418708393 conn(0x7fe938079dc0 msgr2=0x7fe93810dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 shutdown_connections 2024-04-26T18:27:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:27:49.343+0000 7fe93ebaf640 1 -- 172.21.15.90:0/418708393 wait complete. 2024-04-26T18:27:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:49 smithi119 ceph-mon[32752]: pgmap v536: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:49 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:reconnect 2024-04-26T18:27:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:49 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:reconnect 2024-04-26T18:27:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:49 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:49 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.0039999 2024-04-26T18:27:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:49 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599985 2024-04-26T18:27:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:49 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599989 2024-04-26T18:27:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:49 smithi167 ceph-mon[32751]: pgmap v536: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:49 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:reconnect 2024-04-26T18:27:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:49 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:reconnect 2024-04-26T18:27:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:49 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:49 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.0039999 2024-04-26T18:27:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:49 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599985 2024-04-26T18:27:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:49 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599989 2024-04-26T18:27:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:49 smithi090 ceph-mon[29180]: pgmap v536: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 230 KiB/s wr, 23 op/s 2024-04-26T18:27:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:49 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:reconnect 2024-04-26T18:27:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:49 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:reconnect 2024-04-26T18:27:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:49 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:reconnect,2=l=up:reconnect} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:49 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.0039999 2024-04-26T18:27:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:49 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599985 2024-04-26T18:27:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:49 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599989 2024-04-26T18:27:49.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7374 7.90 MB/sec warmup 74 sec latency 24486.613 ms 2024-04-26T18:27:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:50 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:rejoin 2024-04-26T18:27:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:50 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:rejoin 2024-04-26T18:27:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:50 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:50 smithi119 ceph-mon[32752]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:27:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:50 smithi119 ceph-mon[32752]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:27:50.567 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:27:50.240+0000 7f61a6a93700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 1 2024-04-26T18:27:50.567 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:27:50.241+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 3 2024-04-26T18:27:50.567 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:27:50.241+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 4 2024-04-26T18:27:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:rejoin 2024-04-26T18:27:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:rejoin 2024-04-26T18:27:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-mon[32751]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:27:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:50 smithi167 ceph-mon[32751]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:27:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:50 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:rejoin 2024-04-26T18:27:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:50 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:rejoin 2024-04-26T18:27:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:50 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:rejoin,2=l=up:rejoin} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:50 smithi090 ceph-mon[29180]: daemon mds.e is now active in filesystem cephfs as rank 1 2024-04-26T18:27:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:50 smithi090 ceph-mon[29180]: daemon mds.l is now active in filesystem cephfs as rank 2 2024-04-26T18:27:50.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7534 7.83 MB/sec warmup 75 sec latency 24921.722 ms 2024-04-26T18:27:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:51 smithi119 ceph-mon[32752]: pgmap v537: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 125 KiB/s wr, 4 op/s 2024-04-26T18:27:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:51 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:27:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:51 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:active 2024-04-26T18:27:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:51 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:active 2024-04-26T18:27:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:51 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:51 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:27:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:51 smithi167 ceph-mon[32751]: pgmap v537: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 125 KiB/s wr, 4 op/s 2024-04-26T18:27:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:51 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:27:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:51 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:active 2024-04-26T18:27:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:51 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:active 2024-04-26T18:27:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:51 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:51 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:27:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:51 smithi090 ceph-mon[29180]: pgmap v537: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 125 KiB/s wr, 4 op/s 2024-04-26T18:27:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:51 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:27:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:51 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6834/3561315595,v1:172.21.15.119:6835/3561315595] up:active 2024-04-26T18:27:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:51 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/2849164668,v1:172.21.15.167:6833/2849164668] up:active 2024-04-26T18:27:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:51 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 1 up:standby-replay 6 up:standby 2024-04-26T18:27:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:51 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:27:51.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 7786 7.84 MB/sec warmup 76 sec latency 78.908 ms 2024-04-26T18:27:52.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8084 7.91 MB/sec warmup 77 sec latency 99.922 ms 2024-04-26T18:27:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:53 smithi119 ceph-mon[32752]: pgmap v538: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 125 KiB/s wr, 4 op/s 2024-04-26T18:27:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:53 smithi167 ceph-mon[32751]: pgmap v538: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 125 KiB/s wr, 4 op/s 2024-04-26T18:27:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:53 smithi090 ceph-mon[29180]: pgmap v538: 97 pgs: 97 active+clean; 424 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 125 KiB/s wr, 4 op/s 2024-04-26T18:27:53.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8350 7.97 MB/sec warmup 78 sec latency 254.420 ms 2024-04-26T18:27:54.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 8835 8.07 MB/sec warmup 79 sec latency 108.866 ms 2024-04-26T18:27:55.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:55 smithi090 ceph-mon[29180]: pgmap v539: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-04-26T18:27:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-mon[32752]: pgmap v539: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6d68b8d700 time 2024-04-26T18:27:55.260444+0000 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6d731b63dd] 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.554 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: clone() 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.260+0000 7f6d68b8d700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6d68b8d700 time 2024-04-26T18:27:55.260444+0000 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:55.555 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.556 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6d731b63dd] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.557 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: clone() 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: *** Caught signal (Aborted) ** 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: in thread 7f6d68b8d700 thread_name:ms_dispatch 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6d71f3ed20] 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: gsignal() 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: abort() 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6d731b6437] 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.558 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 16: clone() 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.262+0000 7f6d68b8d700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: in thread 7f6d68b8d700 thread_name:ms_dispatch 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.559 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6d71f3ed20] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: gsignal() 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: abort() 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6d731b6437] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.560 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 16: clone() 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: -1> 2024-04-26T18:27:55.260+0000 7f6d68b8d700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6d68b8d700 time 2024-04-26T18:27:55.260444+0000 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6d731b63dd] 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.561 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: clone() 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 0> 2024-04-26T18:27:55.262+0000 7f6d68b8d700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: in thread 7f6d68b8d700 thread_name:ms_dispatch 2024-04-26T18:27:55.562 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6d71f3ed20] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: gsignal() 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: abort() 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6d731b6437] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.563 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 16: clone() 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: -1> 2024-04-26T18:27:55.260+0000 7f6d68b8d700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6d68b8d700 time 2024-04-26T18:27:55.260444+0000 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6d731b63dd] 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.564 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: clone() 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 0> 2024-04-26T18:27:55.262+0000 7f6d68b8d700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: in thread 7f6d68b8d700 thread_name:ms_dispatch 2024-04-26T18:27:55.565 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6d71f3ed20] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2: gsignal() 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 3: abort() 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6d731b6437] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6d731b65a3] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55c9c8738822] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55c9c86a8598] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55c9c86aac17] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55c9c85a3d0b] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55c9c85a42ac] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55c9c858cfad] 2024-04-26T18:27:55.566 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6d7344ae20] 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 13: (DispatchQueue::entry()+0x51f) [0x7f6d73447fdf] 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6d7351afc1] 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6d71f341ca] 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 16: clone() 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e[71538]: 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:55 smithi119 podman[71874]: 2024-04-26 18:27:55.320524772 +0000 UTC m=+0.019177231 container died 17f1726c4621020ac494aaf9962222a93c17112c8a93036a2dbb780aa8ea0f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T18:27:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-mon[32751]: pgmap v539: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-04-26T18:27:55.878 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 8.12 MB/sec warmup 80 sec latency 645.510 ms 2024-04-26T18:27:56.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3be3a34700 time 2024-04-26T18:27:55.958247+0000 2024-04-26T18:27:56.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:56.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3bee05d3dd] 2024-04-26T18:27:56.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.318 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: clone() 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:55.958+0000 7f3be3a34700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3be3a34700 time 2024-04-26T18:27:55.958247+0000 2024-04-26T18:27:56.319 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:56.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3bee05d3dd] 2024-04-26T18:27:56.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.320 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: clone() 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: *** Caught signal (Aborted) ** 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: in thread 7f3be3a34700 thread_name:ms_dispatch 2024-04-26T18:27:56.321 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3becde5d20] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: gsignal() 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: abort() 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3bee05d437] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.322 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 16: clone() 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:55.959+0000 7f3be3a34700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: in thread 7f3be3a34700 thread_name:ms_dispatch 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3becde5d20] 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: gsignal() 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: abort() 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3bee05d437] 2024-04-26T18:27:56.323 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 16: clone() 2024-04-26T18:27:56.324 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:56.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: -1> 2024-04-26T18:27:55.958+0000 7f3be3a34700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3be3a34700 time 2024-04-26T18:27:55.958247+0000 2024-04-26T18:27:56.325 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3bee05d3dd] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.326 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: clone() 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 0> 2024-04-26T18:27:55.959+0000 7f3be3a34700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: in thread 7f3be3a34700 thread_name:ms_dispatch 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3becde5d20] 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: gsignal() 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: abort() 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3bee05d437] 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.327 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 16: clone() 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:56.328 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: -1> 2024-04-26T18:27:55.958+0000 7f3be3a34700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3be3a34700 time 2024-04-26T18:27:55.958247+0000 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3bee05d3dd] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.329 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: clone() 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 0> 2024-04-26T18:27:55.959+0000 7f3be3a34700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: in thread 7f3be3a34700 thread_name:ms_dispatch 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3becde5d20] 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2: gsignal() 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 3: abort() 2024-04-26T18:27:56.330 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3bee05d437] 2024-04-26T18:27:56.331 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3bee05d5a3] 2024-04-26T18:27:56.331 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557078c39822] 2024-04-26T18:27:56.331 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557078ba9598] 2024-04-26T18:27:56.331 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557078babc17] 2024-04-26T18:27:56.331 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557078aa4d0b] 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557078aa52ac] 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557078a8dfad] 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3bee2f1e20] 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 13: (DispatchQueue::entry()+0x51f) [0x7f3bee2eefdf] 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3bee3c1fc1] 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3becddb1ca] 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 16: clone() 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:55 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[71845]: 2024-04-26T18:27:56.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:56 smithi167 podman[72207]: 2024-04-26 18:27:56.017252278 +0000 UTC m=+0.018757258 container died 6bef2c9637c9cef07fc738b0459e23455bef5d8348c25a5d72456f93218835e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T18:27:56.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:56 smithi119 podman[71874]: 2024-04-26 18:27:56.527161394 +0000 UTC m=+1.225813895 container remove 17f1726c4621020ac494aaf9962222a93c17112c8a93036a2dbb780aa8ea0f4f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:27:56.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:56 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:27:56.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:56 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-26T18:27:56.804 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:27:56 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.734s CPU time. 2024-04-26T18:27:56.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 8.02 MB/sec warmup 81 sec latency 1645.597 ms 2024-04-26T18:27:57.067 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:56 smithi167 podman[72207]: 2024-04-26 18:27:56.666872499 +0000 UTC m=+0.668377511 container remove 6bef2c9637c9cef07fc738b0459e23455bef5d8348c25a5d72456f93218835e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T18:27:57.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:56 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:27:57.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:56 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-04-26T18:27:57.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:27:56 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 7.693s CPU time. 2024-04-26T18:27:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:57 smithi119 ceph-mon[32752]: pgmap v540: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-04-26T18:27:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:57 smithi167 ceph-mon[32751]: pgmap v540: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-04-26T18:27:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:57 smithi090 ceph-mon[29180]: pgmap v540: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-04-26T18:27:57.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.92 MB/sec warmup 82 sec latency 2645.692 ms 2024-04-26T18:27:58.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.82 MB/sec warmup 83 sec latency 3645.814 ms 2024-04-26T18:27:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:27:59 smithi119 ceph-mon[32752]: pgmap v541: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 128 op/s 2024-04-26T18:27:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:27:59 smithi167 ceph-mon[32751]: pgmap v541: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 128 op/s 2024-04-26T18:27:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:27:59 smithi090 ceph-mon[29180]: pgmap v541: 97 pgs: 97 active+clean; 437 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.7 MiB/s wr, 128 op/s 2024-04-26T18:27:59.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.73 MB/sec warmup 84 sec latency 4645.855 ms 2024-04-26T18:28:00.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.64 MB/sec warmup 85 sec latency 5645.903 ms 2024-04-26T18:28:01.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:01 smithi167 ceph-mon[32751]: pgmap v542: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-04-26T18:28:01.492 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:01 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:28:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:01 smithi119 ceph-mon[32752]: pgmap v542: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-04-26T18:28:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:01 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:28:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:01 smithi090 ceph-mon[29180]: pgmap v542: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-04-26T18:28:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:01 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=e=up:active,2=l=up:active} 3 up:standby-replay 4 up:standby 2024-04-26T18:28:01.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.55 MB/sec warmup 86 sec latency 6646.040 ms 2024-04-26T18:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3336953633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a410cf90 msgr2=0x7ff5a410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.445 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3336953633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a410cf90 0x7ff5a410f3c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff594009ca0 tx=0x7ff59402a950 comp rx=0 tx=0).stop 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3336953633 shutdown_connections 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3336953633 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a410cf90 0x7ff5a410f3c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3336953633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a41044e0 0x7ff5a410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3336953633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5a4103bc0 0x7ff5a4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3336953633 >> 172.21.15.90:0/3336953633 conn(0x7ff5a4078940 msgr2=0x7ff5a4078d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3336953633 shutdown_connections 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3336953633 wait complete. 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.443+0000 7ff5a9a2c640 1 Processor -- start 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a9a2c640 1 -- start start 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a9a2c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a4103bc0 0x7ff5a4199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.446 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a9a2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5a41044e0 0x7ff5a41995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a9a2c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a410cf90 0x7ff5a419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a9a2c640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4112bd0 con 0x7ff5a4103bc0 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a9a2c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4112a50 con 0x7ff5a41044e0 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a9a2c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a4112d50 con 0x7ff5a410cf90 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a4103bc0 0x7ff5a4199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a4103bc0 0x7ff5a4199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55784/0 (socket says 172.21.15.90:55784) 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3080357077 learned_addr learned my addr 172.21.15.90:0/3080357077 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a37fe640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a410cf90 0x7ff5a419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5a41044e0 0x7ff5a41995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3080357077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a410cf90 msgr2=0x7ff5a419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.447 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a410cf90 0x7ff5a419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3080357077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5a41044e0 msgr2=0x7ff5a41995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5a41044e0 0x7ff5a41995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.444+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3080357077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5a419e080 con 0x7ff5a4103bc0 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.445+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a4103bc0 0x7ff5a4199090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00d950 tx=0x7ff59c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.445+0000 7ff5a8a2a640 1 -- 172.21.15.90:0/3080357077 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c017070 con 0x7ff5a4103bc0 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.445+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5a419e370 con 0x7ff5a4103bc0 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.445+0000 7ff5a8a2a640 1 -- 172.21.15.90:0/3080357077 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c009c30 con 0x7ff5a4103bc0 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.445+0000 7ff5a8a2a640 1 -- 172.21.15.90:0/3080357077 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00f620 con 0x7ff5a4103bc0 2024-04-26T18:28:02.448 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.445+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5a41a5c00 con 0x7ff5a4103bc0 2024-04-26T18:28:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 Processor -- start 2024-04-26T18:28:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a8a2a640 1 -- 172.21.15.90:0/3080357077 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7ff59c010460 con 0x7ff5a4103bc0 2024-04-26T18:28:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 -- start start 2024-04-26T18:28:02.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004912e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x56130049bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x56130049e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5613003f7460 con 0x561300491820 2024-04-26T18:28:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5613003b1590 con 0x56130049c0e0 2024-04-26T18:28:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5ac820740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5613003f3420 con 0x561300490f00 2024-04-26T18:28:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x56130049bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x56130049bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55790/0 (socket says 172.21.15.90:55790) 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3063039865 learned_addr learned my addr 172.21.15.90:0/3063039865 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a37fe640 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004912e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x56130049e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3063039865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 msgr2=0x5613004912e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.448+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004912e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3063039865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 msgr2=0x56130049e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x56130049e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff5a8a2a640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff58c0a3dc0 0x7ff58c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/3063039865 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613003be0d0 con 0x561300491820 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff5a37fe640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff58c0a3dc0 0x7ff58c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x56130049bba0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0cda70 tx=0x7ff59c0d4f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd215ebcbf22e8d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff5a8a2a640 1 -- 172.21.15.90:0/3080357077 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 10905+0+0 (secure 0 0 0) 0x7ff59c0d0d00 con 0x7ff5a4103bc0 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff589ffb640 1 -- 172.21.15.90:0/3063039865 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c020030 con 0x561300491820 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff589ffb640 1 -- 172.21.15.90:0/3063039865 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c009de0 con 0x561300491820 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.449+0000 7ff589ffb640 1 -- 172.21.15.90:0/3063039865 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c09bd60 con 0x561300491820 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 -- 172.21.15.90:0/3063039865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 msgr2=0x56130049bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x56130049bba0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0cda70 tx=0x7ff59c0d4f50 comp rx=0 tx=0).stop 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5a37fe640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff58c0a3dc0 0x7ff58c0a6280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff594005e30 tx=0x7ff594005c50 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 -- 172.21.15.90:0/3063039865 shutdown_connections 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x56130049e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x56130049bba0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 --2- 172.21.15.90:0/3063039865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004912e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.458 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 -- 172.21.15.90:0/3063039865 >> 172.21.15.90:0/3063039865 conn(0x56130048d410 msgr2=0x56130048d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 -- 172.21.15.90:0/3063039865 shutdown_connections 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 -- 172.21.15.90:0/3063039865 wait complete. 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 -- start start 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.450+0000 7ff5ac820740 1 Processor -- start 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x561300499600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 0x561300499b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56130049ed10 con 0x561300490f00 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x56130037eb60 con 0x561300491820 2024-04-26T18:28:02.459 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56130049ee90 con 0x56130049c0e0 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 10 client.0 _ll_register_callbacks cb 0x5613003a32b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55806/0 (socket says 172.21.15.90:55806) 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/2291654516 learned_addr learned my addr 172.21.15.90:0/2291654516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a37fe640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 0x561300499b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x561300499600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/2291654516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 msgr2=0x561300499b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 0x561300499b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 msgr2=0x561300499600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x561300499600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.460 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/2291654516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561300109210 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x561300499600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004990c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff59c012040 tx=0x7ff59c0d4c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3efe33be4a27b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff582ffd640 1 -- 172.21.15.90:0/2291654516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c020040 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff582ffd640 1 -- 172.21.15.90:0/2291654516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 10905+0+0 (secure 0 0 0) 0x7ff59c098a70 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.451+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59c009de0 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff582ffd640 1 -- 172.21.15.90:0/2291654516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=199) ==== 24+0+0 (secure 0 0 0) 0x7ff59c09bd60 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 10 client.16611 fetch_fsmap learned FSMap version 199 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=199,monmap=4+}) -- 0x5613001d5620 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff582ffd640 1 -- 172.21.15.90:0/2291654516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59c009de0 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff582ffd640 1 -- 172.21.15.90:0/2291654516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 199) ==== 7136+0+0 (secure 0 0 0) 0x7ff59c0bb120 con 0x561300490f00 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 10 client.16611 fetch_fsmap finished waiting for FSMap version 199 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:02.461 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 20 client.16611 populate_metadata read hostname 'smithi090' 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5613004ac100 0x5613004ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 4 client.16611 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561300405070 con 0x5613004ac100 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.452+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5613004ac100 0x5613004ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.453+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5613004ac100 0x5613004ae4f0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473404eeb390c813 server_cookie=ee2be68e4faf5722 in_seq=0 out_seq=0 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.453+0000 7ff582ffd640 10 client.16611 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.453+0000 7ff582ffd640 1 -- 172.21.15.90:0/2291654516 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561300405070 con 0x5613004ac100 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.453+0000 7ff582ffd640 10 client.16611 handle_command_reply: tid=0 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.453+0000 7ff5ac820740 1 client.16611 shutdown 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 msgr2=0x5613004990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004990c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff59c012040 tx=0x7ff59c0d4c60 comp rx=0 tx=0).stop 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 shutdown_connections 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5613004ac100 0x5613004ae4f0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 0x561300499b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x561300499600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 --2- 172.21.15.90:0/2291654516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004990c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff582ffd640 0 client.16611 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 >> 172.21.15.90:0/2291654516 conn(0x56130048d410 msgr2=0x561300493140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 shutdown_connections 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 1 -- 172.21.15.90:0/2291654516 wait complete. 2024-04-26T18:28:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.454+0000 7ff5ac820740 20 client.16611 trim_cache size 0 max 16384 2024-04-26T18:28:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.468+0000 7ff5ac820740 1 Processor -- start 2024-04-26T18:28:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.468+0000 7ff5ac820740 1 -- start start 2024-04-26T18:28:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004a7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x5613004b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x5613004b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5ac820740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b0920 con 0x561300491820 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5ac820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b0ff0 con 0x56130049c0e0 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5ac820740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41b16c0 con 0x561300490f00 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x5613004b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x5613004b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55822/0 (socket says 172.21.15.90:55822) 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 -- 172.21.15.90:0/1406745598 learned_addr learned my addr 172.21.15.90:0/1406745598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004a7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 -- 172.21.15.90:0/1406745598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 msgr2=0x5613004a7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004a7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 -- 172.21.15.90:0/1406745598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 msgr2=0x5613004b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a37fe640 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x5613004b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x5613004b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 -- 172.21.15.90:0/1406745598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561300493eb0 con 0x561300491820 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x5613004b3ca0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x56130048e5a0 tx=0x7ff59800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6812da19bb0fcade server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5897fa640 1 -- 172.21.15.90:0/1406745598 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598021030 con 0x561300491820 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5897fa640 1 -- 172.21.15.90:0/1406745598 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59801d420 con 0x561300491820 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.469+0000 7ff5897fa640 1 -- 172.21.15.90:0/1406745598 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5980055f0 con 0x561300491820 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 -- 172.21.15.90:0/1406745598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 msgr2=0x5613004b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x5613004b3ca0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x56130048e5a0 tx=0x7ff59800bd50 comp rx=0 tx=0).stop 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 -- 172.21.15.90:0/1406745598 shutdown_connections 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56130049c0e0 0x5613004b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300491820 0x5613004b3ca0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1406745598 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561300490f00 0x5613004a7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 -- 172.21.15.90:0/1406745598 >> 172.21.15.90:0/1406745598 conn(0x56130048d410 msgr2=0x56130048d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 -- 172.21.15.90:0/1406745598 shutdown_connections 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 -- 172.21.15.90:0/1406745598 wait complete. 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 -- start start 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.470+0000 7ff5ac820740 1 Processor -- start 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x5613004a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 0x5613004a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41adba0 con 0x561300490f00 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41aefb0 con 0x561300491820 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff5a41ae210 con 0x56130049c0e0 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 10 client.0 _ll_register_callbacks cb 0x7ff594009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55834/0 (socket says 172.21.15.90:55834) 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/1098601409 learned_addr learned my addr 172.21.15.90:0/1098601409 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x5613004a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/1098601409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 msgr2=0x5613004a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 0x5613004a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 msgr2=0x5613004a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x5613004a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 -- 172.21.15.90:0/1098601409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613004a15f0 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5a2ffd640 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004a40e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0d4cd0 tx=0x7ff59c0d1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27586f3ea07dcf39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff582ffd640 1 -- 172.21.15.90:0/1098601409 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff59c04f030 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff582ffd640 1 -- 172.21.15.90:0/1098601409 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 10905+0+0 (secure 0 0 0) 0x7ff59c09a700 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.471+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff59801d420 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff582ffd640 1 -- 172.21.15.90:0/1098601409 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=199) ==== 24+0+0 (secure 0 0 0) 0x7ff59c020070 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 10 client.16617 fetch_fsmap learned FSMap version 199 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=199,monmap=4+}) -- 0x561300512cc0 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff582ffd640 1 -- 172.21.15.90:0/1098601409 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff59801d420 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff582ffd640 1 -- 172.21.15.90:0/1098601409 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 199) ==== 7136+0+0 (secure 0 0 0) 0x7ff59c098a70 con 0x561300490f00 2024-04-26T18:28:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 10 client.16617 fetch_fsmap finished waiting for FSMap version 199 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 20 client.16617 populate_metadata read hostname 'smithi090' 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561300513250 0x5613005156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 4 client.16617 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5613003feee0 con 0x561300513250 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.472+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561300513250 0x5613005156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.473+0000 7ff5a27fc640 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561300513250 0x5613005156b0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f84aecd21cdbbb86 server_cookie=29e494c59434bfdc in_seq=0 out_seq=0 2024-04-26T18:28:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:02.473+0000 7ff582ffd640 10 client.16617 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:02.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.47 MB/sec warmup 87 sec latency 7646.071 ms 2024-04-26T18:28:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:03 smithi119 ceph-mon[32752]: pgmap v543: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.1 MiB/s wr, 237 op/s 2024-04-26T18:28:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:03 smithi167 ceph-mon[32751]: pgmap v543: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.1 MiB/s wr, 237 op/s 2024-04-26T18:28:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:03 smithi090 ceph-mon[29180]: pgmap v543: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.1 MiB/s wr, 237 op/s 2024-04-26T18:28:03.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.38 MB/sec warmup 88 sec latency 8646.156 ms 2024-04-26T18:28:04.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.30 MB/sec warmup 89 sec latency 9646.212 ms 2024-04-26T18:28:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:05 smithi119 ceph-mon[32752]: pgmap v544: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.2 MiB/s wr, 303 op/s 2024-04-26T18:28:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:05 smithi167 ceph-mon[32751]: pgmap v544: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.2 MiB/s wr, 303 op/s 2024-04-26T18:28:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:05 smithi090 ceph-mon[29180]: pgmap v544: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 4.2 MiB/s wr, 303 op/s 2024-04-26T18:28:05.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.22 MB/sec warmup 90 sec latency 10646.288 ms 2024-04-26T18:28:06.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.14 MB/sec warmup 91 sec latency 11646.356 ms 2024-04-26T18:28:07.247 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:28:06 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 6. 2024-04-26T18:28:07.247 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:28:06 smithi119 systemd[1]: Stopped Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:28:07.247 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:28:06 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Consumed 4.734s CPU time. 2024-04-26T18:28:07.247 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:28:06 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Start request repeated too quickly. 2024-04-26T18:28:07.247 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:28:06 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-04-26T18:28:07.247 INFO:journalctl@ceph.mds.e.smithi119.stdout:Apr 26 18:28:06 smithi119 systemd[1]: Failed to start Ceph mds.e for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:28:07.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:07 smithi167 ceph-mon[32751]: pgmap v545: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-26T18:28:07.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:28:07.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 systemd[1]: Stopped Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:28:07.334 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 7.693s CPU time. 2024-04-26T18:28:07.335 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 systemd[1]: Starting Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:28:07.335 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 podman[72314]: 2024-04-26 18:28:07.212462795 +0000 UTC m=+0.018973051 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:28:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:07 smithi119 ceph-mon[32752]: pgmap v545: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-26T18:28:07.586 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 podman[72314]: 2024-04-26 18:28:07.334984163 +0000 UTC m=+0.141494418 container create 6c28c32abdbcceabbe04a8b75e14b05e25de5b15e415bbd2dde3a815430a2f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T18:28:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:07 smithi090 ceph-mon[29180]: pgmap v545: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-26T18:28:07.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 7.06 MB/sec warmup 92 sec latency 12646.444 ms 2024-04-26T18:28:08.067 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 podman[72314]: 2024-04-26 18:28:07.587696609 +0000 UTC m=+0.394206866 container init 6c28c32abdbcceabbe04a8b75e14b05e25de5b15e415bbd2dde3a815430a2f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T18:28:08.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 podman[72314]: 2024-04-26 18:28:07.590325235 +0000 UTC m=+0.396835500 container start 6c28c32abdbcceabbe04a8b75e14b05e25de5b15e415bbd2dde3a815430a2f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS) 2024-04-26T18:28:08.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 ceph-mds[72334]: -- 172.21.15.167:0/1930997614 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1f2a75a40 con 0x55a1f2a48800 2024-04-26T18:28:08.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: starting mds.l at 2024-04-26T18:28:08.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 bash[72314]: 6c28c32abdbcceabbe04a8b75e14b05e25de5b15e415bbd2dde3a815430a2f22 2024-04-26T18:28:08.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:28:07 smithi167 systemd[1]: Started Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: daemon mds.l restarted 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: osdmap e143: 12 total, 12 up, 12 in 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:boot 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:28:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:08 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: daemon mds.l restarted 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: osdmap e143: 12 total, 12 up, 12 in 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:boot 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:28:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:08 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: daemon mds.l restarted 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: osdmap e143: 12 total, 12 up, 12 in 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:boot 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: Standby daemon mds.c assigned to filesystem cephfs as rank 2 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=e=up:active} 3 up:standby-replay 5 up:standby, 1 failed 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:28:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:08 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:replay} 2 up:standby-replay 5 up:standby 2024-04-26T18:28:08.879 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 6.98 MB/sec warmup 93 sec latency 13646.570 ms 2024-04-26T18:28:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:09 smithi119 ceph-mon[32752]: pgmap v547: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-04-26T18:28:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:09 smithi167 ceph-mon[32751]: pgmap v547: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-04-26T18:28:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:09 smithi090 ceph-mon[29180]: pgmap v547: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.7 MiB/s wr, 213 op/s 2024-04-26T18:28:09.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 6.91 MB/sec warmup 94 sec latency 14646.624 ms 2024-04-26T18:28:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:10 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:resolve 2024-04-26T18:28:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:10 smithi119 ceph-mon[32752]: Replacing daemon mds.e as rank 1 with standby daemon mds.k 2024-04-26T18:28:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:10 smithi119 ceph-mon[32752]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-04-26T18:28:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:10 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:28:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:10 smithi119 ceph-mon[32752]: osdmap e144: 12 total, 12 up, 12 in 2024-04-26T18:28:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:10 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:10 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:resolve 2024-04-26T18:28:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:10 smithi167 ceph-mon[32751]: Replacing daemon mds.e as rank 1 with standby daemon mds.k 2024-04-26T18:28:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:10 smithi167 ceph-mon[32751]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-04-26T18:28:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:10 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:28:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:10 smithi167 ceph-mon[32751]: osdmap e144: 12 total, 12 up, 12 in 2024-04-26T18:28:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:10 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:10 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:resolve 2024-04-26T18:28:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:10 smithi090 ceph-mon[29180]: Replacing daemon mds.e as rank 1 with standby daemon mds.k 2024-04-26T18:28:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:10 smithi090 ceph-mon[29180]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2024-04-26T18:28:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:10 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=e=up:active,2=c=up:resolve} 2 up:standby-replay 5 up:standby 2024-04-26T18:28:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:10 smithi090 ceph-mon[29180]: osdmap e144: 12 total, 12 up, 12 in 2024-04-26T18:28:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:10 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=c=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:10.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 6.84 MB/sec warmup 95 sec latency 15646.686 ms 2024-04-26T18:28:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:11 smithi119 ceph-mon[32752]: pgmap v549: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 119 KiB/s wr, 118 op/s 2024-04-26T18:28:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:resolve 2024-04-26T18:28:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:11 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:11 smithi167 ceph-mon[32751]: pgmap v549: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 119 KiB/s wr, 118 op/s 2024-04-26T18:28:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:resolve 2024-04-26T18:28:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:11 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:11 smithi090 ceph-mon[29180]: pgmap v549: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 119 KiB/s wr, 118 op/s 2024-04-26T18:28:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:resolve 2024-04-26T18:28:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:11 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=c=up:resolve} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:11.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9070 6.77 MB/sec warmup 96 sec latency 16646.787 ms 2024-04-26T18:28:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.352+0000 7ff582ffd640 1 -- 172.21.15.90:0/1098601409 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5613003feee0 con 0x561300513250 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.352+0000 7ff582ffd640 10 client.16617 handle_command_reply: tid=0 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.352+0000 7ff5ac820740 1 client.16617 shutdown 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 msgr2=0x5613004a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004a40e0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0d4cd0 tx=0x7ff59c0d1000 comp rx=0 tx=0).stop 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 shutdown_connections 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x561300513250 0x5613005156b0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56130049c0e0 0x5613004a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff582ffd640 0 client.16617 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561300491820 0x5613004a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 --2- 172.21.15.90:0/1098601409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x561300490f00 0x5613004a40e0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 >> 172.21.15.90:0/1098601409 conn(0x56130048d410 msgr2=0x561300492880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 shutdown_connections 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 1 -- 172.21.15.90:0/1098601409 wait complete. 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.353+0000 7ff5ac820740 20 client.16617 trim_cache size 0 max 16384 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:28:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff58c0a3dc0 msgr2=0x7ff58c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff58c0a3dc0 0x7ff58c0a6280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff594005e30 tx=0x7ff594005c50 comp rx=0 tx=0).stop 2024-04-26T18:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a4103bc0 msgr2=0x7ff5a4199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a4103bc0 0x7ff5a4199090 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00d950 tx=0x7ff59c00de10 comp rx=0 tx=0).stop 2024-04-26T18:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 shutdown_connections 2024-04-26T18:28:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff58c0a3dc0 0x7ff58c0a6280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff5a410cf90 0x7ff5a419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5a41044e0 0x7ff5a41995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 --2- 172.21.15.90:0/3080357077 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff5a4103bc0 0x7ff5a4199090 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 >> 172.21.15.90:0/3080357077 conn(0x7ff5a4078940 msgr2=0x7ff5a40fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 shutdown_connections 2024-04-26T18:28:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:12.354+0000 7ff5a9a2c640 1 -- 172.21.15.90:0/3080357077 wait complete. 2024-04-26T18:28:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:12 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:28:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:reconnect 2024-04-26T18:28:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:reconnect 2024-04-26T18:28:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:12 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:12 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00399993 2024-04-26T18:28:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:12 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0039999 2024-04-26T18:28:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:12 smithi119 ceph-mon[32752]: pgmap v550: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-26T18:28:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:12 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:28:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:reconnect 2024-04-26T18:28:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:reconnect 2024-04-26T18:28:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:12 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:12 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00399993 2024-04-26T18:28:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:12 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0039999 2024-04-26T18:28:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:12 smithi167 ceph-mon[32751]: pgmap v550: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-26T18:28:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:12 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:28:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:reconnect 2024-04-26T18:28:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:reconnect 2024-04-26T18:28:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:12 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=c=up:reconnect} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:12 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00399993 2024-04-26T18:28:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:12 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0039999 2024-04-26T18:28:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:12 smithi090 ceph-mon[29180]: pgmap v550: 97 pgs: 97 active+clean; 423 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 19 op/s 2024-04-26T18:28:12.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9384 6.88 MB/sec warmup 97 sec latency 17646.825 ms 2024-04-26T18:28:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:13 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:rejoin 2024-04-26T18:28:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:13 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:rejoin 2024-04-26T18:28:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:13 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:13 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:rejoin 2024-04-26T18:28:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:13 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:rejoin 2024-04-26T18:28:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:13 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:13 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:rejoin 2024-04-26T18:28:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:13 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:rejoin 2024-04-26T18:28:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:13 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=c=up:rejoin} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:13.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 9930 7.21 MB/sec warmup 98 sec latency 18646.863 ms 2024-04-26T18:28:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:clientreplay 2024-04-26T18:28:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:clientreplay 2024-04-26T18:28:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:14 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=c=up:clientreplay} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:14 smithi119 ceph-mon[32752]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:28:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:14 smithi119 ceph-mon[32752]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:28:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:14 smithi119 ceph-mon[32752]: pgmap v551: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 392 KiB/s wr, 73 op/s 2024-04-26T18:28:14.568 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:28:14.316+0000 7f61a8296700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 15 2024-04-26T18:28:14.568 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:28:14.317+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 15 2024-04-26T18:28:14.568 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:28:14.317+0000 7f61a6a93700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 18 2024-04-26T18:28:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:clientreplay 2024-04-26T18:28:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:clientreplay 2024-04-26T18:28:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=c=up:clientreplay} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-mon[32751]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:28:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-mon[32751]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:28:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:14 smithi167 ceph-mon[32751]: pgmap v551: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 392 KiB/s wr, 73 op/s 2024-04-26T18:28:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:clientreplay 2024-04-26T18:28:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:clientreplay 2024-04-26T18:28:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:14 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=c=up:clientreplay} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:14 smithi090 ceph-mon[29180]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-26T18:28:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:14 smithi090 ceph-mon[29180]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:28:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:14 smithi090 ceph-mon[29180]: pgmap v551: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 392 KiB/s wr, 73 op/s 2024-04-26T18:28:14.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10387 7.48 MB/sec warmup 99 sec latency 19174.508 ms 2024-04-26T18:28:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:15 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:28:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:active 2024-04-26T18:28:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:active 2024-04-26T18:28:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 3 up:standby-replay 3 up:standby 2024-04-26T18:28:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:15 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:28:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:active 2024-04-26T18:28:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:active 2024-04-26T18:28:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 3 up:standby-replay 3 up:standby 2024-04-26T18:28:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:15 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:28:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6836/3885172962,v1:172.21.15.167:6837/3885172962] up:active 2024-04-26T18:28:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/117919007,v1:172.21.15.119:6833/117919007] up:active 2024-04-26T18:28:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 1 up:standby-replay 5 up:standby 2024-04-26T18:28:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=c=up:active} 3 up:standby-replay 3 up:standby 2024-04-26T18:28:15.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10574 7.48 MB/sec warmup 100 sec latency 216.947 ms 2024-04-26T18:28:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:16 smithi167 ceph-mon[32751]: osdmap e145: 12 total, 12 up, 12 in 2024-04-26T18:28:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:16 smithi167 ceph-mon[32751]: pgmap v553: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 392 KiB/s wr, 73 op/s 2024-04-26T18:28:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:16 smithi090 ceph-mon[29180]: osdmap e145: 12 total, 12 up, 12 in 2024-04-26T18:28:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:16 smithi090 ceph-mon[29180]: pgmap v553: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 392 KiB/s wr, 73 op/s 2024-04-26T18:28:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:16 smithi119 ceph-mon[32752]: osdmap e145: 12 total, 12 up, 12 in 2024-04-26T18:28:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:16 smithi119 ceph-mon[32752]: pgmap v553: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 392 KiB/s wr, 73 op/s 2024-04-26T18:28:16.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 10903 7.61 MB/sec warmup 101 sec latency 164.165 ms 2024-04-26T18:28:17.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11245 7.69 MB/sec warmup 102 sec latency 82.417 ms 2024-04-26T18:28:18.683 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f80a0378700 time 2024-04-26T18:28:18.323052+0000 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f80aa9a13dd] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.684 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: clone() 2024-04-26T18:28:18.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.324+0000 7f80a0378700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f80a0378700 time 2024-04-26T18:28:18.323052+0000 2024-04-26T18:28:18.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:18.685 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.686 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.686 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f80aa9a13dd] 2024-04-26T18:28:18.686 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.686 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.686 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: clone() 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: *** Caught signal (Aborted) ** 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: in thread 7f80a0378700 thread_name:ms_dispatch 2024-04-26T18:28:18.687 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f80a9729d20] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: gsignal() 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: abort() 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f80aa9a1437] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.688 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 16: clone() 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.326+0000 7f80a0378700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: in thread 7f80a0378700 thread_name:ms_dispatch 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f80a9729d20] 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: gsignal() 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: abort() 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f80aa9a1437] 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.689 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 16: clone() 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: -1> 2024-04-26T18:28:18.324+0000 7f80a0378700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f80a0378700 time 2024-04-26T18:28:18.323052+0000 2024-04-26T18:28:18.690 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f80aa9a13dd] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.691 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: clone() 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 0> 2024-04-26T18:28:18.326+0000 7f80a0378700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: in thread 7f80a0378700 thread_name:ms_dispatch 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f80a9729d20] 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: gsignal() 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: abort() 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f80aa9a1437] 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.692 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 16: clone() 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: -1> 2024-04-26T18:28:18.324+0000 7f80a0378700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f80a0378700 time 2024-04-26T18:28:18.323052+0000 2024-04-26T18:28:18.693 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f80aa9a13dd] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.694 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: clone() 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 0> 2024-04-26T18:28:18.326+0000 7f80a0378700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: in thread 7f80a0378700 thread_name:ms_dispatch 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f80a9729d20] 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2: gsignal() 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 3: abort() 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f80aa9a1437] 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f80aa9a15a3] 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555999826822] 2024-04-26T18:28:18.695 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555999796598] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555999798c17] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555999691d0b] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5559996922ac] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55599967afad] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f80aac35e20] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 13: (DispatchQueue::entry()+0x51f) [0x7f80aac32fdf] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f80aad05fc1] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f80a971f1ca] 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 16: clone() 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c[72160]: 2024-04-26T18:28:18.696 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:18 smithi167 podman[72424]: 2024-04-26 18:28:18.399143037 +0000 UTC m=+0.018996945 container died 819c6d46945e06e87bf21d4900cfd0dd36d68c86cb88ba09ddb0845f281dd202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T18:28:18.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.65 MB/sec warmup 103 sec latency 579.280 ms 2024-04-26T18:28:19.273 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:19 smithi167 ceph-mon[32751]: pgmap v554: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 342 KiB/s wr, 48 op/s 2024-04-26T18:28:19.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:19.274 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:19 smithi167 podman[72424]: 2024-04-26 18:28:19.009641271 +0000 UTC m=+0.629495171 container remove 819c6d46945e06e87bf21d4900cfd0dd36d68c86cb88ba09ddb0845f281dd202 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:28:19.274 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:19 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:28:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:19 smithi119 ceph-mon[32752]: pgmap v554: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 342 KiB/s wr, 48 op/s 2024-04-26T18:28:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:19.567 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:19 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-26T18:28:19.567 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:19 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 7.446s CPU time. 2024-04-26T18:28:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:19 smithi090 ceph-mon[29180]: pgmap v554: 97 pgs: 97 active+clean; 414 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 342 KiB/s wr, 48 op/s 2024-04-26T18:28:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:19.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.57 MB/sec warmup 104 sec latency 1579.381 ms 2024-04-26T18:28:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:28:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:28:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:28:20.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.50 MB/sec warmup 105 sec latency 2579.448 ms 2024-04-26T18:28:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:21 smithi119 ceph-mon[32752]: pgmap v555: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.4 MiB/s wr, 507 op/s 2024-04-26T18:28:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:21 smithi119 ceph-mon[32752]: pgmap v556: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 4.4 MiB/s wr, 649 op/s 2024-04-26T18:28:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:21 smithi167 ceph-mon[32751]: pgmap v555: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.4 MiB/s wr, 507 op/s 2024-04-26T18:28:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:21 smithi167 ceph-mon[32751]: pgmap v556: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 4.4 MiB/s wr, 649 op/s 2024-04-26T18:28:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:21 smithi090 ceph-mon[29180]: pgmap v555: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 3.4 MiB/s wr, 507 op/s 2024-04-26T18:28:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:21 smithi090 ceph-mon[29180]: pgmap v556: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 4.4 MiB/s wr, 649 op/s 2024-04-26T18:28:21.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.43 MB/sec warmup 106 sec latency 3579.512 ms 2024-04-26T18:28:22.880 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.36 MB/sec warmup 107 sec latency 4579.631 ms 2024-04-26T18:28:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:23 smithi119 ceph-mon[32752]: pgmap v557: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.0 MiB/s wr, 589 op/s 2024-04-26T18:28:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:23 smithi119 ceph-mon[32752]: osdmap e146: 12 total, 12 up, 12 in 2024-04-26T18:28:23.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:23 smithi167 ceph-mon[32751]: pgmap v557: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.0 MiB/s wr, 589 op/s 2024-04-26T18:28:23.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:23 smithi167 ceph-mon[32751]: osdmap e146: 12 total, 12 up, 12 in 2024-04-26T18:28:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:23 smithi090 ceph-mon[29180]: pgmap v557: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.0 MiB/s wr, 589 op/s 2024-04-26T18:28:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:23 smithi090 ceph-mon[29180]: osdmap e146: 12 total, 12 up, 12 in 2024-04-26T18:28:23.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.29 MB/sec warmup 108 sec latency 5579.729 ms 2024-04-26T18:28:24.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.23 MB/sec warmup 109 sec latency 6579.846 ms 2024-04-26T18:28:25.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:25 smithi090 ceph-mon[29180]: pgmap v559: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.1 MiB/s wr, 603 op/s 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 -- 172.21.15.90:0/1989337780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 msgr2=0x7fb838103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 --2- 172.21.15.90:0/1989337780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb838103fa0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fb834009900 tx=0x7fb83402a510 comp rx=0 tx=0).stop 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 -- 172.21.15.90:0/1989337780 shutdown_connections 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 --2- 172.21.15.90:0/1989337780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83810cf00 0x7fb83810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 --2- 172.21.15.90:0/1989337780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8381044e0 0x7fb83810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 --2- 172.21.15.90:0/1989337780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb838103fa0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 -- 172.21.15.90:0/1989337780 >> 172.21.15.90:0/1989337780 conn(0x7fb8380789b0 msgr2=0x7fb838078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 -- 172.21.15.90:0/1989337780 shutdown_connections 2024-04-26T18:28:25.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.464+0000 7fb840433640 1 -- 172.21.15.90:0/1989337780 wait complete. 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 Processor -- start 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 -- start start 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb8381991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8381044e0 0x7fb8381996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83810cf00 0x7fb83819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb838112bb0 con 0x7fb838103bc0 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb838112a30 con 0x7fb83810cf00 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb840433640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb838112d30 con 0x7fb8381044e0 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb83e1cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb8381991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb83e1cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb8381991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52528/0 (socket says 172.21.15.90:52528) 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.465+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1932721516 learned_addr learned my addr 172.21.15.90:0/1932721516 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:25.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8381044e0 0x7fb8381996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83810cf00 0x7fb83819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1932721516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8381044e0 msgr2=0x7fb8381996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8381044e0 0x7fb8381996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1932721516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83810cf00 msgr2=0x7fb83819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83810cf00 0x7fb83819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1932721516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83819e1a0 con 0x7fb838103bc0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83810cf00 0x7fb83819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb8381991b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb834005940 tx=0x7fb8340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb82f7fe640 1 -- 172.21.15.90:0/1932721516 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83403f030 con 0x7fb838103bc0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83819e430 con 0x7fb838103bc0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb82f7fe640 1 -- 172.21.15.90:0/1932721516 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb834003cb0 con 0x7fb838103bc0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.466+0000 7fb82f7fe640 1 -- 172.21.15.90:0/1932721516 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834035070 con 0x7fb838103bc0 2024-04-26T18:28:25.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.467+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8381a5d20 con 0x7fb838103bc0 2024-04-26T18:28:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.469+0000 7fb82f7fe640 1 -- 172.21.15.90:0/1932721516 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fb834042030 con 0x7fb838103bc0 2024-04-26T18:28:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.469+0000 7fb84322a740 1 Processor -- start 2024-04-26T18:28:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb84322a740 1 -- start start 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb84322a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee6513d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb84322a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb82f7fe640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb8240a3dc0 0x7fb8240a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb84322a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee6514a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb84322a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ee650a3460 con 0x55ee6513cf00 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb84322a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ee6505d590 con 0x55ee651480e0 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb84322a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee6509f420 con 0x55ee6513d820 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee6513d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb8240a3dc0 0x7fb8240a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee6513d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52534/0 (socket says 172.21.15.90:52534) 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 -- 172.21.15.90:0/1901987375 learned_addr learned my addr 172.21.15.90:0/1901987375 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb82f7fe640 1 -- 172.21.15.90:0/1932721516 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 11121+0+0 (secure 0 0 0) 0x7fb8340f1ab0 con 0x7fb838103bc0 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee6514a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 -- 172.21.15.90:0/1901987375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 msgr2=0x55ee65147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 -- 172.21.15.90:0/1901987375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 msgr2=0x55ee6514a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee6514a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.470+0000 7fb83e9d0640 1 -- 172.21.15.90:0/1901987375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6506a0d0 con 0x55ee6513cf00 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee6513d2e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb830009fd0 tx=0x7fb830009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc14b66d375a3664 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb8240a3dc0 0x7fb8240a6280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb83810ed30 tx=0x7fb82800b390 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb82cbfe640 1 -- 172.21.15.90:0/1901987375 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830004d20 con 0x55ee6513cf00 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb82cbfe640 1 -- 172.21.15.90:0/1901987375 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83000db90 con 0x55ee6513cf00 2024-04-26T18:28:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb82cbfe640 1 -- 172.21.15.90:0/1901987375 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8300055b0 con 0x55ee6513cf00 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb84322a740 1 -- 172.21.15.90:0/1901987375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 msgr2=0x55ee6513d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb84322a740 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee6513d2e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fb830009fd0 tx=0x7fb830009670 comp rx=0 tx=0).stop 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb84322a740 1 -- 172.21.15.90:0/1901987375 shutdown_connections 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb84322a740 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee6514a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.471+0000 7fb84322a740 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 --2- 172.21.15.90:0/1901987375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee6513d2e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 -- 172.21.15.90:0/1901987375 >> 172.21.15.90:0/1901987375 conn(0x55ee65139410 msgr2=0x55ee651397f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 -- 172.21.15.90:0/1901987375 shutdown_connections 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 -- 172.21.15.90:0/1901987375 wait complete. 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 -- start start 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 Processor -- start 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee6513cf00 0x55ee65144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513d820 0x55ee65145530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee651480e0 0x55ee65145a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ee6514ad10 con 0x55ee6513d820 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ee6502ab60 con 0x55ee6513cf00 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ee6514ae90 con 0x55ee651480e0 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb84322a740 10 client.0 _ll_register_callbacks cb 0x55ee6504f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb83d9ce640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513d820 0x55ee65145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb83d9ce640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513d820 0x55ee65145530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52540/0 (socket says 172.21.15.90:52540) 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb83d9ce640 1 -- 172.21.15.90:0/2062741919 learned_addr learned my addr 172.21.15.90:0/2062741919 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee651480e0 0x55ee65145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.472+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee6513cf00 0x55ee65144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb83d9ce640 1 -- 172.21.15.90:0/2062741919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee651480e0 msgr2=0x55ee65145a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee651480e0 0x55ee65145a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb83d9ce640 1 -- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee6513cf00 msgr2=0x55ee65144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee6513cf00 0x55ee65144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb83d9ce640 1 -- 172.21.15.90:0/2062741919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee64db5210 con 0x55ee6513d820 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee6513cf00 0x55ee65144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513d820 0x55ee65145530 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb828003ea0 tx=0x7fb828004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ddb025d739bdb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb821ffb640 1 -- 172.21.15.90:0/2062741919 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb828005220 con 0x55ee6513d820 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb821ffb640 1 -- 172.21.15.90:0/2062741919 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 11121+0+0 (secure 0 0 0) 0x7fb8280053c0 con 0x55ee6513d820 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb83000db90 con 0x55ee6513d820 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb821ffb640 1 -- 172.21.15.90:0/2062741919 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) ==== 24+0+0 (secure 0 0 0) 0x7fb82802e050 con 0x55ee6513d820 2024-04-26T18:28:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb84322a740 10 client.16629 fetch_fsmap learned FSMap version 209 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=4+}) -- 0x55ee64e81620 con 0x55ee6513d820 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb821ffb640 1 -- 172.21.15.90:0/2062741919 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb83000db90 con 0x55ee6513d820 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.473+0000 7fb821ffb640 1 -- 172.21.15.90:0/2062741919 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 209) ==== 6531+0+0 (secure 0 0 0) 0x7fb8280294f0 con 0x55ee6513d820 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb84322a740 10 client.16629 fetch_fsmap finished waiting for FSMap version 209 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb84322a740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb84322a740 20 client.16629 populate_metadata read hostname 'smithi090' 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb84322a740 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651583f0 0x55ee6515a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb84322a740 4 client.16629 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee650b1070 con 0x55ee651583f0 2024-04-26T18:28:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651583f0 0x55ee6515a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651583f0 0x55ee6515a890 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2956d3b4616f3d61 server_cookie=5b6392e5def58cd3 in_seq=0 out_seq=0 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.474+0000 7fb821ffb640 10 client.16629 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb821ffb640 1 -- 172.21.15.90:0/2062741919 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee650b1070 con 0x55ee651583f0 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb821ffb640 10 client.16629 handle_command_reply: tid=0 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 client.16629 shutdown 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513d820 msgr2=0x55ee65145530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513d820 0x55ee65145530 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fb828003ea0 tx=0x7fb828004730 comp rx=0 tx=0).stop 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 shutdown_connections 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651583f0 0x55ee6515a890 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee651480e0 0x55ee65145a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb821ffb640 0 client.16629 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513d820 0x55ee65145530 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 --2- 172.21.15.90:0/2062741919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee6513cf00 0x55ee65144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 >> 172.21.15.90:0/2062741919 conn(0x55ee65139410 msgr2=0x55ee6513f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 shutdown_connections 2024-04-26T18:28:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 1 -- 172.21.15.90:0/2062741919 wait complete. 2024-04-26T18:28:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.475+0000 7fb84322a740 20 client.16629 trim_cache size 0 max 16384 2024-04-26T18:28:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 Processor -- start 2024-04-26T18:28:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 -- start start 2024-04-26T18:28:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee65153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee6513f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee65160160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381b1710 con 0x55ee6513cf00 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381b02a0 con 0x55ee651480e0 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb84322a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381b0970 con 0x55ee6513d820 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb83e1cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee65153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb83e1cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee65153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52544/0 (socket says 172.21.15.90:52544) 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb83e1cf640 1 -- 172.21.15.90:0/3185369178 learned_addr learned my addr 172.21.15.90:0/3185369178 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee6513f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb83e1cf640 1 -- 172.21.15.90:0/3185369178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 msgr2=0x55ee6513f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.490+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee6513f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb83e1cf640 1 -- 172.21.15.90:0/3185369178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 msgr2=0x55ee65160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee65160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb83e1cf640 1 -- 172.21.15.90:0/3185369178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee65143fa0 con 0x55ee6513cf00 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee65153bd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb8340bcd20 tx=0x7fb83407cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956c29d18aacafdf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb823fff640 1 -- 172.21.15.90:0/3185369178 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834049040 con 0x55ee6513cf00 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb823fff640 1 -- 172.21.15.90:0/3185369178 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8340c13d0 con 0x55ee6513cf00 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb823fff640 1 -- 172.21.15.90:0/3185369178 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8340b9690 con 0x55ee6513cf00 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 -- 172.21.15.90:0/3185369178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 msgr2=0x55ee65153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee65153bd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fb8340bcd20 tx=0x7fb83407cb70 comp rx=0 tx=0).stop 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 -- 172.21.15.90:0/3185369178 shutdown_connections 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee65160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee6513f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 --2- 172.21.15.90:0/3185369178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee65153bd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 -- 172.21.15.90:0/3185369178 >> 172.21.15.90:0/3185369178 conn(0x55ee65139410 msgr2=0x55ee651470b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 -- 172.21.15.90:0/3185369178 shutdown_connections 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.491+0000 7fb84322a740 1 -- 172.21.15.90:0/3185369178 wait complete. 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 -- start start 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 Processor -- start 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee651420e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65142620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee651bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381ae9d0 con 0x55ee6513cf00 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381b1040 con 0x55ee651480e0 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8381adcb0 con 0x55ee6513d820 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb84322a740 10 client.0 _ll_register_callbacks cb 0x7fb828004360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee651420e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee651420e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52548/0 (socket says 172.21.15.90:52548) 2024-04-26T18:28:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1591628132 learned_addr learned my addr 172.21.15.90:0/1591628132 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65142620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e9d0640 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee651bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1591628132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 msgr2=0x55ee65142620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65142620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 msgr2=0x55ee651bd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee651bd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.492+0000 7fb83e1cf640 1 -- 172.21.15.90:0/1591628132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee651bdf10 con 0x55ee6513cf00 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb83e1cf640 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee651420e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb8340ee880 tx=0x7fb834032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd5ba02d1f9193d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb821ffb640 1 -- 172.21.15.90:0/1591628132 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834049070 con 0x55ee6513cf00 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb821ffb640 1 -- 172.21.15.90:0/1591628132 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 11121+0+0 (secure 0 0 0) 0x7fb834081cf0 con 0x55ee6513cf00 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8340c13d0 con 0x55ee6513cf00 2024-04-26T18:28:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb821ffb640 1 -- 172.21.15.90:0/1591628132 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=209) ==== 24+0+0 (secure 0 0 0) 0x7fb8340eea50 con 0x55ee6513cf00 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 10 client.16635 fetch_fsmap learned FSMap version 209 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=209,monmap=4+}) -- 0x55ee651bf0e0 con 0x55ee6513cf00 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb821ffb640 1 -- 172.21.15.90:0/1591628132 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8340c13d0 con 0x55ee6513cf00 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb821ffb640 1 -- 172.21.15.90:0/1591628132 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 209) ==== 6531+0+0 (secure 0 0 0) 0x7fb83407cbe0 con 0x55ee6513cf00 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 10 client.16635 fetch_fsmap finished waiting for FSMap version 209 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 20 client.16635 populate_metadata read hostname 'smithi090' 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651bf750 0x55ee651c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 4 client.16635 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.493+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee650aaee0 con 0x55ee651bf750 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.494+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651bf750 0x55ee651c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.494+0000 7fb83d9ce640 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651bf750 0x55ee651c1b70 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=682f4cc851f83cec server_cookie=e7013893e651d5df in_seq=0 out_seq=0 2024-04-26T18:28:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:25.494+0000 7fb821ffb640 10 client.16635 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:25 smithi119 ceph-mon[32752]: pgmap v559: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.1 MiB/s wr, 603 op/s 2024-04-26T18:28:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:25 smithi167 ceph-mon[32751]: pgmap v559: 97 pgs: 97 active+clean; 409 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.1 MiB/s wr, 603 op/s 2024-04-26T18:28:25.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.16 MB/sec warmup 110 sec latency 7579.893 ms 2024-04-26T18:28:26.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.10 MB/sec warmup 111 sec latency 8579.927 ms 2024-04-26T18:28:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:27 smithi119 ceph-mon[32752]: pgmap v560: 97 pgs: 97 active+clean; 410 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.1 MiB/s wr, 612 op/s 2024-04-26T18:28:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:27 smithi167 ceph-mon[32751]: pgmap v560: 97 pgs: 97 active+clean; 410 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.1 MiB/s wr, 612 op/s 2024-04-26T18:28:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:27 smithi090 ceph-mon[29180]: pgmap v560: 97 pgs: 97 active+clean; 410 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 4.1 MiB/s wr, 612 op/s 2024-04-26T18:28:27.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 7.03 MB/sec warmup 112 sec latency 9580.017 ms 2024-04-26T18:28:28.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 6.97 MB/sec warmup 113 sec latency 10580.103 ms 2024-04-26T18:28:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:29 smithi119 ceph-mon[32752]: pgmap v561: 97 pgs: 97 active+clean; 410 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 150 KiB/s wr, 20 op/s 2024-04-26T18:28:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:29 smithi167 ceph-mon[32751]: pgmap v561: 97 pgs: 97 active+clean; 410 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 150 KiB/s wr, 20 op/s 2024-04-26T18:28:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:29 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Scheduled restart job, restart counter is at 6. 2024-04-26T18:28:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:29 smithi167 systemd[1]: Stopped Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:28:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:29 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Consumed 7.446s CPU time. 2024-04-26T18:28:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:29 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Start request repeated too quickly. 2024-04-26T18:28:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:29 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.c.service: Failed with result 'exit-code'. 2024-04-26T18:28:29.568 INFO:journalctl@ceph.mds.c.smithi167.stdout:Apr 26 18:28:29 smithi167 systemd[1]: Failed to start Ceph mds.c for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:28:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:29 smithi090 ceph-mon[29180]: pgmap v561: 97 pgs: 97 active+clean; 410 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 150 KiB/s wr, 20 op/s 2024-04-26T18:28:29.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 6.91 MB/sec warmup 114 sec latency 11580.189 ms 2024-04-26T18:28:30.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 6.85 MB/sec warmup 115 sec latency 12580.275 ms 2024-04-26T18:28:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:31 smithi119 ceph-mon[32752]: pgmap v562: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-04-26T18:28:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:31 smithi167 ceph-mon[32751]: pgmap v562: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-04-26T18:28:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:31 smithi090 ceph-mon[29180]: pgmap v562: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-04-26T18:28:31.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 6.79 MB/sec warmup 116 sec latency 13580.311 ms 2024-04-26T18:28:32.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 6.73 MB/sec warmup 117 sec latency 14580.361 ms 2024-04-26T18:28:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:33 smithi119 ceph-mon[32752]: pgmap v563: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-04-26T18:28:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:33 smithi119 ceph-mon[32752]: Replacing daemon mds.c as rank 2 with standby daemon mds.a 2024-04-26T18:28:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:33 smithi119 ceph-mon[32752]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-26T18:28:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:33 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:28:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:33 smithi119 ceph-mon[32752]: osdmap e147: 12 total, 12 up, 12 in 2024-04-26T18:28:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:33 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:replay} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:33 smithi167 ceph-mon[32751]: pgmap v563: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-04-26T18:28:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:33 smithi167 ceph-mon[32751]: Replacing daemon mds.c as rank 2 with standby daemon mds.a 2024-04-26T18:28:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:33 smithi167 ceph-mon[32751]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-26T18:28:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:33 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:28:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:33 smithi167 ceph-mon[32751]: osdmap e147: 12 total, 12 up, 12 in 2024-04-26T18:28:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:33 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:replay} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:33 smithi090 ceph-mon[29180]: pgmap v563: 97 pgs: 97 active+clean; 413 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-04-26T18:28:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:33 smithi090 ceph-mon[29180]: Replacing daemon mds.c as rank 2 with standby daemon mds.a 2024-04-26T18:28:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:33 smithi090 ceph-mon[29180]: MDS daemon mds.c is removed because it is dead or otherwise unavailable. 2024-04-26T18:28:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:33 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:28:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:33 smithi090 ceph-mon[29180]: osdmap e147: 12 total, 12 up, 12 in 2024-04-26T18:28:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:33 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:replay} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:33.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 6.68 MB/sec warmup 118 sec latency 15580.487 ms 2024-04-26T18:28:34.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11458 6.62 MB/sec warmup 119 sec latency 16580.593 ms 2024-04-26T18:28:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:35 smithi119 ceph-mon[32752]: pgmap v565: 97 pgs: 97 active+clean; 424 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-04-26T18:28:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:resolve 2024-04-26T18:28:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:35 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:resolve} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:35 smithi167 ceph-mon[32751]: pgmap v565: 97 pgs: 97 active+clean; 424 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-04-26T18:28:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:resolve 2024-04-26T18:28:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:35 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:resolve} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:35 smithi090 ceph-mon[29180]: pgmap v565: 97 pgs: 97 active+clean; 424 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-04-26T18:28:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:resolve 2024-04-26T18:28:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:35 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:resolve} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb821ffb640 1 -- 172.21.15.90:0/1591628132 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee650aaee0 con 0x55ee651bf750 2024-04-26T18:28:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb821ffb640 10 client.16635 handle_command_reply: tid=0 2024-04-26T18:28:36.275 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 client.16635 shutdown 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 msgr2=0x55ee651420e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee651420e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb8340ee880 tx=0x7fb834032f60 comp rx=0 tx=0).stop 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 shutdown_connections 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ee651bf750 0x55ee651c1b70 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ee651480e0 0x55ee651bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb821ffb640 0 client.16635 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ee6513d820 0x55ee65142620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 --2- 172.21.15.90:0/1591628132 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ee6513cf00 0x55ee651420e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 >> 172.21.15.90:0/1591628132 conn(0x55ee65139410 msgr2=0x55ee65137d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 shutdown_connections 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 1 -- 172.21.15.90:0/1591628132 wait complete. 2024-04-26T18:28:36.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.273+0000 7fb84322a740 20 client.16635 trim_cache size 0 max 16384 2024-04-26T18:28:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:28:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:28:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:28:36.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb8240a3dc0 msgr2=0x7fb8240a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb8240a3dc0 0x7fb8240a6280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb83810ed30 tx=0x7fb82800b390 comp rx=0 tx=0).stop 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 msgr2=0x7fb8381991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb8381991b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb834005940 tx=0x7fb8340042f0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 shutdown_connections 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb8240a3dc0 0x7fb8240a6280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83810cf00 0x7fb83819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8381044e0 0x7fb8381996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 --2- 172.21.15.90:0/1932721516 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb838103bc0 0x7fb8381991b0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 >> 172.21.15.90:0/1932721516 conn(0x7fb8380789b0 msgr2=0x7fb8380ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:36.278 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 shutdown_connections 2024-04-26T18:28:36.279 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:36.274+0000 7fb840433640 1 -- 172.21.15.90:0/1932721516 wait complete. 2024-04-26T18:28:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:36 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:reconnect 2024-04-26T18:28:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:36 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:reconnect} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:36 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:28:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:36 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:reconnect 2024-04-26T18:28:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:36 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:reconnect} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:36 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:28:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:36 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:reconnect 2024-04-26T18:28:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:36 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:reconnect} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:36 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:28:36.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 11824 15.94 MB/sec execute 1 sec latency 18555.731 ms 2024-04-26T18:28:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:37 smithi119 ceph-mon[32752]: pgmap v566: 97 pgs: 97 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-04-26T18:28:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:37 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:rejoin 2024-04-26T18:28:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:37 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:rejoin} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:37 smithi119 ceph-mon[32752]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:28:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:37 smithi167 ceph-mon[32751]: pgmap v566: 97 pgs: 97 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-04-26T18:28:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:37 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:rejoin 2024-04-26T18:28:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:37 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:rejoin} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:37 smithi167 ceph-mon[32751]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:28:37.568 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:28:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:28:37.229+0000 7f61aaa9b700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:28:37.568 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:28:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:28:37.229+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:28:37.568 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:28:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:28:37.230+0000 7f61a8296700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 6 2024-04-26T18:28:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:37 smithi090 ceph-mon[29180]: pgmap v566: 97 pgs: 97 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-04-26T18:28:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:37 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:rejoin 2024-04-26T18:28:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:37 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=a=up:rejoin} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:37 smithi090 ceph-mon[29180]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:28:37.881 INFO:tasks.workunit.client.0.smithi090.stdout: 10 12574 24.93 MB/sec execute 2 sec latency 18935.463 ms 2024-04-26T18:28:38.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:28:38 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:28:38.253+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:28:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:38 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:28:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:active 2024-04-26T18:28:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:38 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:28:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:active 2024-04-26T18:28:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:38 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:28:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3724478516,v1:172.21.15.90:6841/3724478516] up:active 2024-04-26T18:28:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:28:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:38.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13233 33.50 MB/sec execute 3 sec latency 57.493 ms 2024-04-26T18:28:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:39 smithi119 ceph-mon[32752]: pgmap v567: 97 pgs: 97 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-04-26T18:28:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:39 smithi167 ceph-mon[32751]: pgmap v567: 97 pgs: 97 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-04-26T18:28:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:39 smithi090 ceph-mon[29180]: pgmap v567: 97 pgs: 97 active+clean; 434 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 3.2 MiB/s wr, 183 op/s 2024-04-26T18:28:39.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 13857 34.83 MB/sec execute 4 sec latency 58.302 ms 2024-04-26T18:28:40.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14272 32.66 MB/sec execute 5 sec latency 91.494 ms 2024-04-26T18:28:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:41 smithi119 ceph-mon[32752]: pgmap v568: 97 pgs: 97 active+clean; 436 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 176 op/s 2024-04-26T18:28:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:41 smithi167 ceph-mon[32751]: pgmap v568: 97 pgs: 97 active+clean; 436 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 176 op/s 2024-04-26T18:28:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:41 smithi090 ceph-mon[29180]: pgmap v568: 97 pgs: 97 active+clean; 436 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 176 op/s 2024-04-26T18:28:41.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 14785 30.83 MB/sec execute 6 sec latency 60.373 ms 2024-04-26T18:28:42.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15316 30.67 MB/sec execute 7 sec latency 591.528 ms 2024-04-26T18:28:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:43 smithi119 ceph-mon[32752]: pgmap v569: 97 pgs: 97 active+clean; 436 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 176 op/s 2024-04-26T18:28:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:43 smithi167 ceph-mon[32751]: pgmap v569: 97 pgs: 97 active+clean; 436 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 176 op/s 2024-04-26T18:28:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:43 smithi090 ceph-mon[29180]: pgmap v569: 97 pgs: 97 active+clean; 436 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.5 MiB/s wr, 176 op/s 2024-04-26T18:28:43.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 15798 29.68 MB/sec execute 8 sec latency 1591.565 ms 2024-04-26T18:28:44.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 16306 29.53 MB/sec execute 9 sec latency 2591.624 ms 2024-04-26T18:28:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:45 smithi119 ceph-mon[32752]: pgmap v570: 97 pgs: 97 active+clean; 449 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 283 op/s 2024-04-26T18:28:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:45 smithi167 ceph-mon[32751]: pgmap v570: 97 pgs: 97 active+clean; 449 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 283 op/s 2024-04-26T18:28:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:45 smithi090 ceph-mon[29180]: pgmap v570: 97 pgs: 97 active+clean; 449 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 4.7 MiB/s wr, 283 op/s 2024-04-26T18:28:45.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17061 30.03 MB/sec execute 10 sec latency 3591.655 ms 2024-04-26T18:28:46.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 17771 31.60 MB/sec execute 11 sec latency 4591.680 ms 2024-04-26T18:28:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:47 smithi119 ceph-mon[32752]: pgmap v571: 97 pgs: 97 active+clean; 458 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.9 MiB/s wr, 285 op/s 2024-04-26T18:28:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:47 smithi167 ceph-mon[32751]: pgmap v571: 97 pgs: 97 active+clean; 458 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.9 MiB/s wr, 285 op/s 2024-04-26T18:28:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:47 smithi090 ceph-mon[29180]: pgmap v571: 97 pgs: 97 active+clean; 458 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.9 MiB/s wr, 285 op/s 2024-04-26T18:28:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:47.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18162 30.98 MB/sec execute 12 sec latency 5591.707 ms 2024-04-26T18:28:48.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18522 29.61 MB/sec execute 13 sec latency 6591.738 ms 2024-04-26T18:28:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 -- 172.21.15.90:0/2584029564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 msgr2=0x7f35ec107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 --2- 172.21.15.90:0/2584029564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec107c00 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009930 tx=0x7f35e802a420 comp rx=0 tx=0).stop 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 -- 172.21.15.90:0/2584029564 shutdown_connections 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 --2- 172.21.15.90:0/2584029564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35ec102220 0x7f35ec10e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 --2- 172.21.15.90:0/2584029564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35ec101860 0x7f35ec101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 --2- 172.21.15.90:0/2584029564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec107c00 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 -- 172.21.15.90:0/2584029564 >> 172.21.15.90:0/2584029564 conn(0x7f35ec0789b0 msgr2=0x7f35ec078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 -- 172.21.15.90:0/2584029564 shutdown_connections 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 -- 172.21.15.90:0/2584029564 wait complete. 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 Processor -- start 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.382+0000 7f35f46b0640 1 -- start start 2024-04-26T18:28:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f46b0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35ec101860 0x7f35ec0fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f46b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35ec102220 0x7f35ec0fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f46b0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec0ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f46b0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec111b30 con 0x7f35ec107820 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f46b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1119b0 con 0x7f35ec102220 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f46b0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec111cb0 con 0x7f35ec101860 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec0ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec0ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48798/0 (socket says 172.21.15.90:48798) 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1922576 learned_addr learned my addr 172.21.15.90:0/1922576 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35ec102220 0x7f35ec0fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f244c640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35ec101860 0x7f35ec0fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1922576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35ec101860 msgr2=0x7f35ec0fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35ec101860 0x7f35ec0fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1922576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35ec102220 msgr2=0x7f35ec0fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35ec102220 0x7f35ec0fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1922576 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec1a7580 con 0x7f35ec107820 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec0ff270 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f35e400cad0 tx=0x7f35e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35e37fe640 1 -- 172.21.15.90:0/1922576 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e400ae20 con 0x7f35ec107820 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35e37fe640 1 -- 172.21.15.90:0/1922576 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e4005020 con 0x7f35ec107820 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.383+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec1a7870 con 0x7f35ec107820 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.384+0000 7f35e37fe640 1 -- 172.21.15.90:0/1922576 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e400f6b0 con 0x7f35ec107820 2024-04-26T18:28:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.384+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec1a7dc0 con 0x7f35ec107820 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35e37fe640 1 -- 172.21.15.90:0/1922576 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f35e40051d0 con 0x7f35ec107820 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 Processor -- start 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 -- start start 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeeb6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35e37fe640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f35d80a3dc0 0x7f35d80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ebbee12460 con 0x55ebbeeac820 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ebbedcc590 con 0x55ebbeeb70e0 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.386+0000 7f35f73c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ebbee0e420 con 0x55ebbeeabf00 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f244c640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f35d80a3dc0 0x7f35d80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeeb6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35e37fe640 1 -- 172.21.15.90:0/1922576 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 11163+0+0 (secure 0 0 0) 0x7f35e40d1180 con 0x7f35ec107820 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f2c4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeeb6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48804/0 (socket says 172.21.15.90:48804) 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 -- 172.21.15.90:0/4114119574 learned_addr learned my addr 172.21.15.90:0/4114119574 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 -- 172.21.15.90:0/4114119574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 msgr2=0x55ebbeeac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 -- 172.21.15.90:0/4114119574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 msgr2=0x55ebbeeb94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 -- 172.21.15.90:0/4114119574 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbedd90d0 con 0x55ebbeeac820 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f244c640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f35d80a3dc0 0x7f35d80a6280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35e8031440 tx=0x7f35e8005a20 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeeb6ba0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0058c0 tx=0x7f35dc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d73675c5141ff2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35e0ff9640 1 -- 172.21.15.90:0/4114119574 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc004030 con 0x55ebbeeac820 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35e0ff9640 1 -- 172.21.15.90:0/4114119574 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc005020 con 0x55ebbeeac820 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.387+0000 7f35e0ff9640 1 -- 172.21.15.90:0/4114119574 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc004b20 con 0x55ebbeeac820 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 -- 172.21.15.90:0/4114119574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 msgr2=0x55ebbeeb6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeeb6ba0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0058c0 tx=0x7f35dc005850 comp rx=0 tx=0).stop 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 -- 172.21.15.90:0/4114119574 shutdown_connections 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeeb6ba0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.396 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 --2- 172.21.15.90:0/4114119574 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 -- 172.21.15.90:0/4114119574 >> 172.21.15.90:0/4114119574 conn(0x55ebbeea8410 msgr2=0x55ebbeea87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 -- 172.21.15.90:0/4114119574 shutdown_connections 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.388+0000 7f35f73c9740 1 -- 172.21.15.90:0/4114119574 wait complete. 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 -- start start 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 Processor -- start 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeac820 0x55ebbeeb45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ebbeeb9d10 con 0x55ebbeeb70e0 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ebbed99b60 con 0x55ebbeeac820 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ebbeeb9e90 con 0x55ebbeeabf00 2024-04-26T18:28:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f73c9740 10 client.0 _ll_register_callbacks cb 0x55ebbedbe2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f2c4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f2c4d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48810/0 (socket says 172.21.15.90:48810) 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1990895611 learned_addr learned my addr 172.21.15.90:0/1990895611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeac820 0x55ebbeeb45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.389+0000 7f35f244c640 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1990895611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 msgr2=0x55ebbeeb4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeac820 msgr2=0x55ebbeeb45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeac820 0x55ebbeeb45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f2c4d640 1 -- 172.21.15.90:0/1990895611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbeb24210 con 0x55ebbeeb70e0 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb4b10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f35e409c710 tx=0x7f35e409cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8b147bdfb0b702 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:28:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1990895611 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e401d070 con 0x55ebbeeb70e0 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1990895611 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 11163+0+0 (secure 0 0 0) 0x7f35e40054f0 con 0x55ebbeeb70e0 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35dc005020 con 0x55ebbeeb70e0 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1990895611 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7f35e40a0a80 con 0x55ebbeeb70e0 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 10 client.16647 fetch_fsmap learned FSMap version 216 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=216,monmap=4+}) -- 0x55ebbebf0620 con 0x55ebbeeb70e0 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1990895611 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc005020 con 0x55ebbeeb70e0 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1990895611 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 216) ==== 5930+0+0 (secure 0 0 0) 0x7f35e4098040 con 0x55ebbeeb70e0 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 10 client.16647 fetch_fsmap finished waiting for FSMap version 216 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 20 client.16647 populate_metadata read hostname 'smithi090' 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbeec75d0 0x55ebbeec9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 4 client.16647 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.390+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebbee20070 con 0x55ebbeec75d0 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.391+0000 7f35f244c640 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbeec75d0 0x55ebbeec9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.391+0000 7f35d5ffb640 10 client.16647 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.391+0000 7f35f244c640 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbeec75d0 0x55ebbeec9a10 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d814edbb2c2ac1 server_cookie=13549232dd42b139 in_seq=0 out_seq=0 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1990895611 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ebbee20070 con 0x55ebbeec75d0 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35d5ffb640 10 client.16647 handle_command_reply: tid=0 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 client.16647 shutdown 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeb70e0 msgr2=0x55ebbeeb4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb4b10 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f35e409c710 tx=0x7f35e409cf70 comp rx=0 tx=0).stop 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 shutdown_connections 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbeec75d0 0x55ebbeec9a10 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb4b10 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeac820 0x55ebbeeb45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1990895611 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.392+0000 7f35d5ffb640 0 client.16647 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.393+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 >> 172.21.15.90:0/1990895611 conn(0x55ebbeea8410 msgr2=0x55ebbeeae190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.393+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 shutdown_connections 2024-04-26T18:28:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.393+0000 7f35f73c9740 1 -- 172.21.15.90:0/1990895611 wait complete. 2024-04-26T18:28:49.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.393+0000 7f35f73c9740 20 client.16647 trim_cache size 0 max 16384 2024-04-26T18:28:49.410 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 Processor -- start 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 -- start start 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeec2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeecf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeecf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1b16e0 con 0x55ebbeeac820 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1b32b0 con 0x55ebbeeb70e0 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f73c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1b1db0 con 0x55ebbeeabf00 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.409+0000 7f35f1c4b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeecf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeec2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f2c4d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeecf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeec2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:40038/0 (socket says 172.21.15.90:40038) 2024-04-26T18:28:49.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 -- 172.21.15.90:0/204272721 learned_addr learned my addr 172.21.15.90:0/204272721 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 -- 172.21.15.90:0/204272721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 msgr2=0x55ebbeecf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 --2- 172.21.15.90:0/204272721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeecf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 -- 172.21.15.90:0/204272721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 msgr2=0x55ebbeecf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 --2- 172.21.15.90:0/204272721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeecf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 -- 172.21.15.90:0/204272721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbeeafc70 con 0x55ebbeeabf00 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35f244c640 1 --2- 172.21.15.90:0/204272721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeec2bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009930 tx=0x7f35e8040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca22ac82ac50050d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35d7fff640 1 -- 172.21.15.90:0/204272721 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8057070 con 0x55ebbeeabf00 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35d7fff640 1 -- 172.21.15.90:0/204272721 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e8005550 con 0x55ebbeeabf00 2024-04-26T18:28:49.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.410+0000 7f35d7fff640 1 -- 172.21.15.90:0/204272721 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e804e030 con 0x55ebbeeabf00 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 -- 172.21.15.90:0/204272721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 msgr2=0x55ebbeec2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 --2- 172.21.15.90:0/204272721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeec2bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f35e8009930 tx=0x7f35e8040000 comp rx=0 tx=0).stop 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 -- 172.21.15.90:0/204272721 shutdown_connections 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 --2- 172.21.15.90:0/204272721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeecf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 --2- 172.21.15.90:0/204272721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeac820 0x55ebbeecf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 --2- 172.21.15.90:0/204272721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeabf00 0x55ebbeec2bd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 -- 172.21.15.90:0/204272721 >> 172.21.15.90:0/204272721 conn(0x55ebbeea8410 msgr2=0x55ebbeea87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:49.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 -- 172.21.15.90:0/204272721 shutdown_connections 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 -- 172.21.15.90:0/204272721 wait complete. 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 -- start start 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.411+0000 7f35f73c9740 1 Processor -- start 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeac820 0x55ebbeeb11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1b2bb0 con 0x55ebbeeabf00 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1b24e0 con 0x55ebbeeb70e0 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec1afed0 con 0x55ebbeeac820 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 10 client.0 _ll_register_callbacks cb 0x7f35e400ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48816/0 (socket says 172.21.15.90:48816) 2024-04-26T18:28:49.417 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 -- 172.21.15.90:0/1311699997 learned_addr learned my addr 172.21.15.90:0/1311699997 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f2c4d640 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeac820 0x55ebbeeb11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 -- 172.21.15.90:0/1311699997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeac820 msgr2=0x55ebbeeb11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeac820 0x55ebbeeb11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 -- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 msgr2=0x55ebbeeb1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 -- 172.21.15.90:0/1311699997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbef2cb50 con 0x55ebbeeabf00 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f244c640 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb0c80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f35e802a940 tx=0x7f35e80042d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7195b34e72edd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1311699997 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8060060 con 0x55ebbeeabf00 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1311699997 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(147..147 src has 1..147) ==== 11163+0+0 (secure 0 0 0) 0x7f35e8004c10 con 0x55ebbeeabf00 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.412+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35e8005550 con 0x55ebbeeabf00 2024-04-26T18:28:49.418 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1311699997 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=216) ==== 24+0+0 (secure 0 0 0) 0x7f35e804e030 con 0x55ebbeeabf00 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 10 client.16653 fetch_fsmap learned FSMap version 216 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=216,monmap=4+}) -- 0x55ebbef2de10 con 0x55ebbeeabf00 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1311699997 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35e8005550 con 0x55ebbeeabf00 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1311699997 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 216) ==== 5930+0+0 (secure 0 0 0) 0x7f35e805cbe0 con 0x55ebbeeabf00 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 10 client.16653 fetch_fsmap finished waiting for FSMap version 216 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 20 client.16653 populate_metadata read hostname 'smithi090' 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbef2e510 0x55ebbef30930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 4 client.16653 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ebbee19ee0 con 0x55ebbef2e510 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbef2e510 0x55ebbef30930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35f1c4b640 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbef2e510 0x55ebbef30930 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15b9d2f85eac7d45 server_cookie=5656324f10aa56fe in_seq=0 out_seq=0 2024-04-26T18:28:49.419 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:28:49.413+0000 7f35d5ffb640 10 client.16653 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:28:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:49 smithi119 ceph-mon[32752]: pgmap v572: 97 pgs: 97 active+clean; 458 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-04-26T18:28:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:49 smithi167 ceph-mon[32751]: pgmap v572: 97 pgs: 97 active+clean; 458 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-04-26T18:28:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:49 smithi090 ceph-mon[29180]: pgmap v572: 97 pgs: 97 active+clean; 458 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.0 MiB/s wr, 265 op/s 2024-04-26T18:28:49.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 18942 28.98 MB/sec execute 14 sec latency 7591.768 ms 2024-04-26T18:28:50.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19740 29.76 MB/sec execute 15 sec latency 8591.807 ms 2024-04-26T18:28:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-mon[32752]: pgmap v573: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 445 op/s 2024-04-26T18:28:51.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1224d14700 time 2024-04-26T18:28:51.183511+0000 2024-04-26T18:28:51.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:51.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f122f33d3dd] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: clone() 2024-04-26T18:28:51.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.183+0000 7f1224d14700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1224d14700 time 2024-04-26T18:28:51.183511+0000 2024-04-26T18:28:51.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:51.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f122f33d3dd] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: clone() 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: *** Caught signal (Aborted) ** 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: in thread 7f1224d14700 thread_name:ms_dispatch 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f122e0c5d20] 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: gsignal() 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: abort() 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f122f33d437] 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 16: clone() 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.185+0000 7f1224d14700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: in thread 7f1224d14700 thread_name:ms_dispatch 2024-04-26T18:28:51.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f122e0c5d20] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: gsignal() 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: abort() 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f122f33d437] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 16: clone() 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: -1> 2024-04-26T18:28:51.183+0000 7f1224d14700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1224d14700 time 2024-04-26T18:28:51.183511+0000 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f122f33d3dd] 2024-04-26T18:28:51.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: clone() 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 0> 2024-04-26T18:28:51.185+0000 7f1224d14700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: in thread 7f1224d14700 thread_name:ms_dispatch 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f122e0c5d20] 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: gsignal() 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: abort() 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f122f33d437] 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 16: clone() 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: -1> 2024-04-26T18:28:51.183+0000 7f1224d14700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1224d14700 time 2024-04-26T18:28:51.183511+0000 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f122f33d3dd] 2024-04-26T18:28:51.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: clone() 2024-04-26T18:28:51.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 0> 2024-04-26T18:28:51.185+0000 7f1224d14700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: in thread 7f1224d14700 thread_name:ms_dispatch 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f122e0c5d20] 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2: gsignal() 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 3: abort() 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f122f33d437] 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f122f33d5a3] 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55ca41e38822] 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55ca41da8598] 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55ca41daac17] 2024-04-26T18:28:51.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55ca41ca3d0b] 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55ca41ca42ac] 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55ca41c8cfad] 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f122f5d1e20] 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 13: (DispatchQueue::entry()+0x51f) [0x7f122f5cefdf] 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f122f6a1fc1] 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f122e0bb1ca] 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 16: clone() 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[71835]: 2024-04-26T18:28:51.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 podman[72064]: 2024-04-26 18:28:51.245410433 +0000 UTC m=+0.019079101 container died 079d1ed4e668fab0d071da2ace3f95f0869cdf020181dbba891474f5850582d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T18:28:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:51 smithi167 ceph-mon[32751]: pgmap v573: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 445 op/s 2024-04-26T18:28:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:51 smithi090 ceph-mon[29180]: pgmap v573: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 445 op/s 2024-04-26T18:28:51.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 29.00 MB/sec execute 16 sec latency 9591.943 ms 2024-04-26T18:28:52.253 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 podman[72064]: 2024-04-26 18:28:51.940751467 +0000 UTC m=+0.714420152 container remove 079d1ed4e668fab0d071da2ace3f95f0869cdf020181dbba891474f5850582d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424) 2024-04-26T18:28:52.253 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:51 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:28:52.253 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:52 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-26T18:28:52.253 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:28:52 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 32.182s CPU time. 2024-04-26T18:28:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:52 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:52 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:52 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:28:52.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 27.30 MB/sec execute 17 sec latency 10591.986 ms 2024-04-26T18:28:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:53 smithi119 ceph-mon[32752]: pgmap v574: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.1 MiB/s wr, 348 op/s 2024-04-26T18:28:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:53 smithi167 ceph-mon[32751]: pgmap v574: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.1 MiB/s wr, 348 op/s 2024-04-26T18:28:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:53 smithi090 ceph-mon[29180]: pgmap v574: 97 pgs: 97 active+clean; 479 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.1 MiB/s wr, 348 op/s 2024-04-26T18:28:53.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 25.78 MB/sec execute 18 sec latency 11592.092 ms 2024-04-26T18:28:54.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 24.42 MB/sec execute 19 sec latency 12592.161 ms 2024-04-26T18:28:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:55 smithi119 ceph-mon[32752]: pgmap v575: 97 pgs: 97 active+clean; 487 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.7 MiB/s wr, 537 op/s 2024-04-26T18:28:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:55 smithi167 ceph-mon[32751]: pgmap v575: 97 pgs: 97 active+clean; 487 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.7 MiB/s wr, 537 op/s 2024-04-26T18:28:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:55 smithi090 ceph-mon[29180]: pgmap v575: 97 pgs: 97 active+clean; 487 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.7 MiB/s wr, 537 op/s 2024-04-26T18:28:55.882 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 23.20 MB/sec execute 20 sec latency 13592.250 ms 2024-04-26T18:28:56.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 22.10 MB/sec execute 21 sec latency 14592.407 ms 2024-04-26T18:28:57.182 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa930085700 time 2024-04-26T18:28:56.896749+0000 2024-04-26T18:28:57.182 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:57.182 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.182 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa93a6ae3dd] 2024-04-26T18:28:57.182 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.182 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.182 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: clone() 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: *** Caught signal (Aborted) ** 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: in thread 7fa930085700 thread_name:ms_dispatch 2024-04-26T18:28:57.183 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:56.897+0000 7fa930085700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa930085700 time 2024-04-26T18:28:56.896749+0000 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa93a6ae3dd] 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.185 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: clone() 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa939436d20] 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: gsignal() 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: abort() 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa93a6ae437] 2024-04-26T18:28:57.186 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 16: clone() 2024-04-26T18:28:57.187 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:56.899+0000 7fa930085700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: in thread 7fa930085700 thread_name:ms_dispatch 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa939436d20] 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: gsignal() 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: abort() 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa93a6ae437] 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.188 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 16: clone() 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: -1> 2024-04-26T18:28:56.897+0000 7fa930085700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa930085700 time 2024-04-26T18:28:56.896749+0000 2024-04-26T18:28:57.189 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa93a6ae3dd] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.190 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: clone() 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 0> 2024-04-26T18:28:56.899+0000 7fa930085700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: in thread 7fa930085700 thread_name:ms_dispatch 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa939436d20] 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: gsignal() 2024-04-26T18:28:57.191 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: abort() 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa93a6ae437] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.192 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 16: clone() 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: -1> 2024-04-26T18:28:56.897+0000 7fa930085700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa930085700 time 2024-04-26T18:28:56.896749+0000 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa93a6ae3dd] 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.193 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: clone() 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 0> 2024-04-26T18:28:56.899+0000 7fa930085700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:28:57.194 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: in thread 7fa930085700 thread_name:ms_dispatch 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa939436d20] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2: gsignal() 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 3: abort() 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa93a6ae437] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa93a6ae5a3] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557c44747822] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557c446b7598] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557c446b9c17] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557c445b2d0b] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557c445b32ac] 2024-04-26T18:28:57.195 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557c4459bfad] 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa93a942e20] 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 13: (DispatchQueue::entry()+0x51f) [0x7fa93a93ffdf] 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa93aa12fc1] 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa93942c1ca] 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 16: clone() 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[70646]: 2024-04-26T18:28:57.196 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:56 smithi090 podman[97198]: 2024-04-26 18:28:56.973921899 +0000 UTC m=+0.022824799 container died 62157d4f85050d3da84e07ba3df6685335dc23cb3310b2d3ca973c0a4ec123a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True) 2024-04-26T18:28:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:57 smithi119 ceph-mon[32752]: pgmap v576: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 481 op/s 2024-04-26T18:28:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:57 smithi167 ceph-mon[32751]: pgmap v576: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 481 op/s 2024-04-26T18:28:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:57 smithi090 ceph-mon[29180]: pgmap v576: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 481 op/s 2024-04-26T18:28:57.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 21.09 MB/sec execute 22 sec latency 15592.479 ms 2024-04-26T18:28:57.960 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:57 smithi090 podman[97198]: 2024-04-26 18:28:57.709988933 +0000 UTC m=+0.758891820 container remove 62157d4f85050d3da84e07ba3df6685335dc23cb3310b2d3ca973c0a4ec123a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T18:28:57.960 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:57 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:28:58.376 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:57 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-04-26T18:28:58.376 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:28:57 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 12.132s CPU time. 2024-04-26T18:28:58.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 20.18 MB/sec execute 23 sec latency 16592.805 ms 2024-04-26T18:28:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:28:59 smithi119 ceph-mon[32752]: pgmap v577: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-04-26T18:28:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:28:59 smithi167 ceph-mon[32751]: pgmap v577: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-04-26T18:28:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:28:59 smithi090 ceph-mon[29180]: pgmap v577: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 418 op/s 2024-04-26T18:28:59.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 19.34 MB/sec execute 24 sec latency 17592.849 ms 2024-04-26T18:29:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:00 smithi167 ceph-mon[32751]: pgmap v578: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2024-04-26T18:29:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:00 smithi090 ceph-mon[29180]: pgmap v578: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2024-04-26T18:29:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:00 smithi119 ceph-mon[32752]: pgmap v578: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2024-04-26T18:29:00.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 18.56 MB/sec execute 25 sec latency 18592.970 ms 2024-04-26T18:29:01.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 17.85 MB/sec execute 26 sec latency 19593.098 ms 2024-04-26T18:29:02.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:02 smithi119 ceph-mon[32752]: pgmap v579: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-04-26T18:29:02.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:29:02.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 systemd[1]: Stopped Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:02.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 32.182s CPU time. 2024-04-26T18:29:02.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 systemd[1]: Starting Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:29:02.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 podman[72161]: 2024-04-26 18:29:02.44497685 +0000 UTC m=+0.018303440 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:29:02.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 podman[72161]: 2024-04-26 18:29:02.566527172 +0000 UTC m=+0.139853756 container create 1e4bed44865d29600f853c38e0af92e9269818c821c5d6df8f7a80566390816f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:29:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:02 smithi090 ceph-mon[29180]: pgmap v579: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-04-26T18:29:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:02 smithi167 ceph-mon[32751]: pgmap v579: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.1 MiB/s wr, 245 op/s 2024-04-26T18:29:02.869 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 podman[72161]: 2024-04-26 18:29:02.786959013 +0000 UTC m=+0.360285604 container init 1e4bed44865d29600f853c38e0af92e9269818c821c5d6df8f7a80566390816f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T18:29:02.869 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 podman[72161]: 2024-04-26 18:29:02.789714286 +0000 UTC m=+0.363040868 container start 1e4bed44865d29600f853c38e0af92e9269818c821c5d6df8f7a80566390816f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T18:29:02.869 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 ceph-mds[72180]: -- 172.21.15.119:0/496661648 <== mon.2 v2:172.21.15.119:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b67d867a40 con 0x55b67d83ac00 2024-04-26T18:29:02.869 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: starting mds.k at 2024-04-26T18:29:02.883 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 17.19 MB/sec execute 27 sec latency 20593.194 ms 2024-04-26T18:29:03.303 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 bash[72161]: 1e4bed44865d29600f853c38e0af92e9269818c821c5d6df8f7a80566390816f 2024-04-26T18:29:03.304 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:29:02 smithi119 systemd[1]: Started Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: daemon mds.k restarted 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: osdmap e148: 12 total, 12 up, 12 in 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:boot 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=a=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:03 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:replay,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: daemon mds.k restarted 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: osdmap e148: 12 total, 12 up, 12 in 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:boot 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=a=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:29:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:03 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:replay,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: daemon mds.k restarted 2024-04-26T18:29:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: osdmap e148: 12 total, 12 up, 12 in 2024-04-26T18:29:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:boot 2024-04-26T18:29:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: Standby daemon mds.l assigned to filesystem cephfs as rank 1 2024-04-26T18:29:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=a=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:29:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:03 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:replay,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:03.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 16.57 MB/sec execute 28 sec latency 21593.322 ms 2024-04-26T18:29:04.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 16.00 MB/sec execute 29 sec latency 22593.439 ms 2024-04-26T18:29:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:04 smithi090 ceph-mon[29180]: pgmap v581: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 619 KiB/s wr, 73 op/s 2024-04-26T18:29:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:resolve 2024-04-26T18:29:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:04 smithi119 ceph-mon[32752]: pgmap v581: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 619 KiB/s wr, 73 op/s 2024-04-26T18:29:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:resolve 2024-04-26T18:29:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:04 smithi167 ceph-mon[32751]: pgmap v581: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 619 KiB/s wr, 73 op/s 2024-04-26T18:29:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:resolve 2024-04-26T18:29:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:05.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 15.47 MB/sec execute 30 sec latency 23593.503 ms 2024-04-26T18:29:06.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 14.97 MB/sec execute 31 sec latency 24593.566 ms 2024-04-26T18:29:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:06 smithi090 ceph-mon[29180]: pgmap v582: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-26T18:29:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:06 smithi119 ceph-mon[32752]: pgmap v582: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-26T18:29:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:06 smithi167 ceph-mon[32751]: pgmap v582: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-26T18:29:07.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 14.50 MB/sec execute 32 sec latency 25593.609 ms 2024-04-26T18:29:08.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:29:08.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 systemd[1]: Stopped Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:08.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 12.132s CPU time. 2024-04-26T18:29:08.353 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 systemd[1]: Starting Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:29:08.606 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 podman[97308]: 2024-04-26 18:29:08.256683327 +0000 UTC m=+0.017900166 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:29:08.606 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 podman[97308]: 2024-04-26 18:29:08.378875431 +0000 UTC m=+0.140092246 container create 46532ba4a1d0d26286a77390018c69c3434bf33e8e44c206ad42eada58a8d9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T18:29:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35d5ffb640 1 -- 172.21.15.90:0/1311699997 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ebbee19ee0 con 0x55ebbef2e510 2024-04-26T18:29:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35d5ffb640 10 client.16653 handle_command_reply: tid=0 2024-04-26T18:29:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35f73c9740 1 client.16653 shutdown 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeabf00 msgr2=0x55ebbeeb0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb0c80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f35e802a940 tx=0x7f35e80042d0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 shutdown_connections 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ebbef2e510 0x55ebbef30930 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ebbeeb70e0 0x55ebbeeb1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.676+0000 7f35d5ffb640 0 client.16653 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.677+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ebbeeac820 0x55ebbeeb11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.677+0000 7f35f73c9740 1 --2- 172.21.15.90:0/1311699997 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ebbeeabf00 0x55ebbeeb0c80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.677+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 >> 172.21.15.90:0/1311699997 conn(0x55ebbeea8410 msgr2=0x55ebbeea6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.677+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 shutdown_connections 2024-04-26T18:29:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.677+0000 7f35f73c9740 1 -- 172.21.15.90:0/1311699997 wait complete. 2024-04-26T18:29:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.677+0000 7f35f73c9740 20 client.16653 trim_cache size 0 max 16384 2024-04-26T18:29:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:29:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:29:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:29:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f35d80a3dc0 msgr2=0x7f35d80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f35d80a3dc0 0x7f35d80a6280 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f35e8031440 tx=0x7f35e8005a20 comp rx=0 tx=0).stop 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 msgr2=0x7f35ec0ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec0ff270 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f35e400cad0 tx=0x7f35e400cf90 comp rx=0 tx=0).stop 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 shutdown_connections 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f35d80a3dc0 0x7f35d80a6280 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35ec107820 0x7f35ec0ff270 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35ec102220 0x7f35ec0fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 --2- 172.21.15.90:0/1922576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f35ec101860 0x7f35ec0fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 >> 172.21.15.90:0/1922576 conn(0x7f35ec0789b0 msgr2=0x7f35ec06a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 shutdown_connections 2024-04-26T18:29:08.681 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:08.678+0000 7f35f46b0640 1 -- 172.21.15.90:0/1922576 wait complete. 2024-04-26T18:29:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: pgmap v583: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: daemon mds.a restarted 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: osdmap e149: 12 total, 12 up, 12 in 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:boot 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: Standby daemon mds.g assigned to filesystem cephfs as rank 2 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:resolve} 2 up:standby-replay 4 up:standby, 1 failed 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=g=up:replay} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:08.876 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 podman[97308]: 2024-04-26 18:29:08.606845691 +0000 UTC m=+0.368062519 container init 46532ba4a1d0d26286a77390018c69c3434bf33e8e44c206ad42eada58a8d9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T18:29:08.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 podman[97308]: 2024-04-26 18:29:08.609596382 +0000 UTC m=+0.370813206 container start 46532ba4a1d0d26286a77390018c69c3434bf33e8e44c206ad42eada58a8d9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:29:08.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-mds[97327]: -- 172.21.15.90:0/3687868265 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55838f387a40 con 0x55838f35a800 2024-04-26T18:29:08.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: starting mds.a at 2024-04-26T18:29:08.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 bash[97308]: 46532ba4a1d0d26286a77390018c69c3434bf33e8e44c206ad42eada58a8d9f9 2024-04-26T18:29:08.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:08 smithi090 systemd[1]: Started Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:08.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 14.06 MB/sec execute 33 sec latency 26593.651 ms 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: pgmap v583: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: daemon mds.a restarted 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: osdmap e149: 12 total, 12 up, 12 in 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:boot 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: Standby daemon mds.g assigned to filesystem cephfs as rank 2 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:resolve} 2 up:standby-replay 4 up:standby, 1 failed 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:29:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:08 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=g=up:replay} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: pgmap v583: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 14 op/s 2024-04-26T18:29:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: daemon mds.a restarted 2024-04-26T18:29:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: osdmap e149: 12 total, 12 up, 12 in 2024-04-26T18:29:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:boot 2024-04-26T18:29:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: Standby daemon mds.g assigned to filesystem cephfs as rank 2 2024-04-26T18:29:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:resolve} 2 up:standby-replay 4 up:standby, 1 failed 2024-04-26T18:29:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:29:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:08 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=g=up:replay} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:09.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 13.65 MB/sec execute 34 sec latency 27593.701 ms 2024-04-26T18:29:10.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19964 13.26 MB/sec execute 35 sec latency 28593.809 ms 2024-04-26T18:29:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:10 smithi090 ceph-mon[29180]: pgmap v585: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T18:29:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:10 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:resolve 2024-04-26T18:29:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:10 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=g=up:resolve} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:10 smithi119 ceph-mon[32752]: pgmap v585: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T18:29:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:10 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:resolve 2024-04-26T18:29:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:10 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=g=up:resolve} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:10 smithi167 ceph-mon[32751]: pgmap v585: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T18:29:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:10 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:resolve 2024-04-26T18:29:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:10 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:resolve,2=g=up:resolve} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:11.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 19965 12.89 MB/sec execute 36 sec latency 29593.855 ms 2024-04-26T18:29:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:11 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:reconnect 2024-04-26T18:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:reconnect 2024-04-26T18:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:11 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:reconnect,2=g=up:reconnect} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:11 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:29:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:11 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799986 2024-04-26T18:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:11 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:reconnect 2024-04-26T18:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:reconnect 2024-04-26T18:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:11 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:reconnect,2=g=up:reconnect} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:11 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:11 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799986 2024-04-26T18:29:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:11 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999982 2024-04-26T18:29:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:reconnect 2024-04-26T18:29:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:reconnect 2024-04-26T18:29:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:11 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:reconnect,2=g=up:reconnect} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:11 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:29:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:11 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799986 2024-04-26T18:29:12.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20306 12.99 MB/sec execute 37 sec latency 30525.259 ms 2024-04-26T18:29:13.067 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:29:12.641+0000 7f61a6a93700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 1 2024-04-26T18:29:13.068 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:29:12.644+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 5 2024-04-26T18:29:13.068 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:29:12.645+0000 7f61a8296700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 8 2024-04-26T18:29:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: pgmap v586: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:29:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:rejoin 2024-04-26T18:29:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:rejoin 2024-04-26T18:29:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:rejoin,2=g=up:rejoin} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-04-26T18:29:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T18:29:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:active 2024-04-26T18:29:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:active 2024-04-26T18:29:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=g=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: pgmap v586: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:rejoin 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:rejoin 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:rejoin,2=g=up:rejoin} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:active 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:active 2024-04-26T18:29:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=g=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:29:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: pgmap v586: 97 pgs: 97 active+clean; 489 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T18:29:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:rejoin 2024-04-26T18:29:13.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:rejoin 2024-04-26T18:29:13.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:rejoin,2=g=up:rejoin} 1 up:standby-replay 4 up:standby 2024-04-26T18:29:13.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: daemon mds.l is now active in filesystem cephfs as rank 1 2024-04-26T18:29:13.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T18:29:13.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:13.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.167:6832/37569032,v1:172.21.15.167:6833/37569032] up:active 2024-04-26T18:29:13.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2946680060,v1:172.21.15.90:6839/2946680060] up:active 2024-04-26T18:29:13.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=g=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:29:13.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:29:13 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:29:13.256+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:29:13.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:29:13 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:29:13.256+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:29:13.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 20731 13.33 MB/sec execute 38 sec latency 24663.320 ms 2024-04-26T18:29:14.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21173 13.65 MB/sec execute 39 sec latency 158.867 ms 2024-04-26T18:29:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:14 smithi090 ceph-mon[29180]: pgmap v587: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 352 op/s 2024-04-26T18:29:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:14 smithi119 ceph-mon[32752]: pgmap v587: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 352 op/s 2024-04-26T18:29:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:14 smithi167 ceph-mon[32751]: pgmap v587: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.7 MiB/s wr, 352 op/s 2024-04-26T18:29:15.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 21556 13.70 MB/sec execute 40 sec latency 180.328 ms 2024-04-26T18:29:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:15 smithi119 ceph-mon[32752]: osdmap e150: 12 total, 12 up, 12 in 2024-04-26T18:29:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:15 smithi167 ceph-mon[32751]: osdmap e150: 12 total, 12 up, 12 in 2024-04-26T18:29:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:15 smithi090 ceph-mon[29180]: osdmap e150: 12 total, 12 up, 12 in 2024-04-26T18:29:16.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22225 14.24 MB/sec execute 41 sec latency 59.139 ms 2024-04-26T18:29:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:16 smithi119 ceph-mon[32752]: pgmap v589: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-04-26T18:29:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:16 smithi167 ceph-mon[32751]: pgmap v589: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-04-26T18:29:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:16 smithi090 ceph-mon[29180]: pgmap v589: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 11 MiB/s wr, 440 op/s 2024-04-26T18:29:17.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc1e4fd4700 time 2024-04-26T18:29:17.720330+0000 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc1ef5fd3dd] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: clone() 2024-04-26T18:29:17.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.720+0000 7fc1e4fd4700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc1e4fd4700 time 2024-04-26T18:29:17.720330+0000 2024-04-26T18:29:17.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:17.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc1ef5fd3dd] 2024-04-26T18:29:17.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: clone() 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: *** Caught signal (Aborted) ** 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: in thread 7fc1e4fd4700 thread_name:ms_dispatch 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc1ee385d20] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: gsignal() 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: abort() 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc1ef5fd437] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 16: clone() 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.722+0000 7fc1e4fd4700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: in thread 7fc1e4fd4700 thread_name:ms_dispatch 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc1ee385d20] 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: gsignal() 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: abort() 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc1ef5fd437] 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 16: clone() 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: -1> 2024-04-26T18:29:17.720+0000 7fc1e4fd4700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc1e4fd4700 time 2024-04-26T18:29:17.720330+0000 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:17.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc1ef5fd3dd] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: clone() 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 0> 2024-04-26T18:29:17.722+0000 7fc1e4fd4700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: in thread 7fc1e4fd4700 thread_name:ms_dispatch 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc1ee385d20] 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: gsignal() 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: abort() 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc1ef5fd437] 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 16: clone() 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: -1> 2024-04-26T18:29:17.720+0000 7fc1e4fd4700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fc1e4fd4700 time 2024-04-26T18:29:17.720330+0000 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:17.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc1ef5fd3dd] 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: clone() 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 0> 2024-04-26T18:29:17.722+0000 7fc1e4fd4700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: in thread 7fc1e4fd4700 thread_name:ms_dispatch 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fc1ee385d20] 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2: gsignal() 2024-04-26T18:29:17.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 3: abort() 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc1ef5fd437] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fc1ef5fd5a3] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5570f68bd822] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5570f682d598] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5570f682fc17] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5570f6728d0b] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5570f67292ac] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5570f6711fad] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fc1ef891e20] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 13: (DispatchQueue::entry()+0x51f) [0x7fc1ef88efdf] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fc1ef961fc1] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fc1ee37b1ca] 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 16: clone() 2024-04-26T18:29:17.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:17.890 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[70185]: 2024-04-26T18:29:17.890 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:17 smithi090 podman[97367]: 2024-04-26 18:29:17.781038351 +0000 UTC m=+0.019026532 container died dba63a34a5888de793a9acdfefb10294532e1abcfbe1b4deb2f38c06a029a9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T18:29:17.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 14.58 MB/sec execute 42 sec latency 194.774 ms 2024-04-26T18:29:18.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:18 smithi090 podman[97367]: 2024-04-26 18:29:18.515465811 +0000 UTC m=+0.753454037 container remove dba63a34a5888de793a9acdfefb10294532e1abcfbe1b4deb2f38c06a029a9ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T18:29:18.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:18 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:29:18.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:18 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Failed with result 'exit-code'. 2024-04-26T18:29:18.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:18 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 12.301s CPU time. 2024-04-26T18:29:18.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 14.24 MB/sec execute 43 sec latency 1194.824 ms 2024-04-26T18:29:19.197 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:18 smithi090 ceph-mon[29180]: pgmap v590: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-04-26T18:29:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:18 smithi119 ceph-mon[32752]: pgmap v590: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-04-26T18:29:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:18 smithi167 ceph-mon[32751]: pgmap v590: 97 pgs: 97 active+clean; 497 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 10 MiB/s wr, 404 op/s 2024-04-26T18:29:19.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 13.91 MB/sec execute 44 sec latency 2194.850 ms 2024-04-26T18:29:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:20.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 13.60 MB/sec execute 45 sec latency 3194.954 ms 2024-04-26T18:29:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:20 smithi119 ceph-mon[32752]: pgmap v591: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 12 MiB/s wr, 736 op/s 2024-04-26T18:29:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:20 smithi119 ceph-mon[32752]: pgmap v592: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 14 MiB/s wr, 839 op/s 2024-04-26T18:29:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:20 smithi119 ceph-mon[32752]: pgmap v593: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 4.9 MiB/s wr, 570 op/s 2024-04-26T18:29:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:29:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:20 smithi167 ceph-mon[32751]: pgmap v591: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 12 MiB/s wr, 736 op/s 2024-04-26T18:29:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:20 smithi167 ceph-mon[32751]: pgmap v592: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 14 MiB/s wr, 839 op/s 2024-04-26T18:29:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:20 smithi167 ceph-mon[32751]: pgmap v593: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 4.9 MiB/s wr, 570 op/s 2024-04-26T18:29:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:29:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:20 smithi090 ceph-mon[29180]: pgmap v591: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 12 MiB/s wr, 736 op/s 2024-04-26T18:29:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:20 smithi090 ceph-mon[29180]: pgmap v592: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 MiB/s rd, 14 MiB/s wr, 839 op/s 2024-04-26T18:29:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:20 smithi090 ceph-mon[29180]: pgmap v593: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 4.9 MiB/s wr, 570 op/s 2024-04-26T18:29:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:29:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.778+0000 7fdacc939640 1 -- 172.21.15.90:0/3777794878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac810cfb0 msgr2=0x7fdac810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.778+0000 7fdacc939640 1 --2- 172.21.15.90:0/3777794878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac810cfb0 0x7fdac810f3e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fdac0009ca0 tx=0x7fdac002a950 comp rx=0 tx=0).stop 2024-04-26T18:29:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.778+0000 7fdacc939640 1 -- 172.21.15.90:0/3777794878 shutdown_connections 2024-04-26T18:29:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.778+0000 7fdacc939640 1 --2- 172.21.15.90:0/3777794878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac810cfb0 0x7fdac810f3e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.778+0000 7fdacc939640 1 --2- 172.21.15.90:0/3777794878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac8104500 0x7fdac810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.778+0000 7fdacc939640 1 --2- 172.21.15.90:0/3777794878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac8103be0 0x7fdac8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.778+0000 7fdacc939640 1 -- 172.21.15.90:0/3777794878 >> 172.21.15.90:0/3777794878 conn(0x7fdac8077bb0 msgr2=0x7fdac8077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.779+0000 7fdacc939640 1 -- 172.21.15.90:0/3777794878 shutdown_connections 2024-04-26T18:29:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.779+0000 7fdacc939640 1 -- 172.21.15.90:0/3777794878 wait complete. 2024-04-26T18:29:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.779+0000 7fdacc939640 1 Processor -- start 2024-04-26T18:29:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.779+0000 7fdacc939640 1 -- start start 2024-04-26T18:29:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdacc939640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac8103be0 0x7fdac8198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdacc939640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac8104500 0x7fdac8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdacc939640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac810cfb0 0x7fdac819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.782 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdacc939640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8112bf0 con 0x7fdac8103be0 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdacc939640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8112a70 con 0x7fdac8104500 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdacc939640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac8112d70 con 0x7fdac810cfb0 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac8103be0 0x7fdac8198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac8103be0 0x7fdac8198ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60290/0 (socket says 172.21.15.90:60290) 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 -- 172.21.15.90:0/1955654050 learned_addr learned my addr 172.21.15.90:0/1955654050 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac6d9d640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac810cfb0 0x7fdac819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdabffff640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac8104500 0x7fdac8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 -- 172.21.15.90:0/1955654050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac810cfb0 msgr2=0x7fdac819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac810cfb0 0x7fdac819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 -- 172.21.15.90:0/1955654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac8104500 msgr2=0x7fdac8199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac8104500 0x7fdac8199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 -- 172.21.15.90:0/1955654050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdac819dfe0 con 0x7fdac8103be0 2024-04-26T18:29:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.780+0000 7fdac659c640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac8103be0 0x7fdac8198ff0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d950 tx=0x7fdab800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.781+0000 7fdabf7fe640 1 -- 172.21.15.90:0/1955654050 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8017070 con 0x7fdac8103be0 2024-04-26T18:29:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.781+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdac819e2d0 con 0x7fdac8103be0 2024-04-26T18:29:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.781+0000 7fdabf7fe640 1 -- 172.21.15.90:0/1955654050 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab8009c30 con 0x7fdac8103be0 2024-04-26T18:29:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.781+0000 7fdabf7fe640 1 -- 172.21.15.90:0/1955654050 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab800f620 con 0x7fdac8103be0 2024-04-26T18:29:21.784 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.781+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdac81a5b60 con 0x7fdac8103be0 2024-04-26T18:29:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.783+0000 7fdabf7fe640 1 -- 172.21.15.90:0/1955654050 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fdab8010460 con 0x7fdac8103be0 2024-04-26T18:29:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.783+0000 7fdacf64f740 1 Processor -- start 2024-04-26T18:29:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdacf64f740 1 -- start start 2024-04-26T18:29:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9bf00 0x55889be9c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9c820 0x55889bea6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdabf7fe640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdab40a3dc0 0x7fdab40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889bea70e0 0x55889bea94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdacf64f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55889be02460 con 0x55889bea70e0 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdacf64f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55889bdbc590 con 0x55889be9bf00 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdacf64f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55889bdfe420 con 0x55889be9c820 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdabffff640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdab40a3dc0 0x7fdab40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdac659c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9c820 0x55889bea6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889bea70e0 0x55889bea94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889bea70e0 0x55889bea94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60306/0 (socket says 172.21.15.90:60306) 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdac6d9d640 1 -- 172.21.15.90:0/2575086475 learned_addr learned my addr 172.21.15.90:0/2575086475 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdac6d9d640 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9bf00 0x55889be9c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.784+0000 7fdabf7fe640 1 -- 172.21.15.90:0/1955654050 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 11423+0+0 (secure 0 0 0) 0x7fdab80d0fb0 con 0x7fdac8103be0 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdac659c640 1 -- 172.21.15.90:0/2575086475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9bf00 msgr2=0x55889be9c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdac659c640 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9bf00 0x55889be9c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdac659c640 1 -- 172.21.15.90:0/2575086475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889bea70e0 msgr2=0x55889bea94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdac659c640 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889bea70e0 0x55889bea94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdac659c640 1 -- 172.21.15.90:0/2575086475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889bdc90d0 con 0x55889be9c820 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdac6d9d640 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889bea70e0 0x55889bea94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdabffff640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdab40a3dc0 0x7fdab40a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005fd0 tx=0x7fdab0005850 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdac659c640 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9c820 0x55889bea6ba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdab80cdbb0 tx=0x7fdab8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1716d8c85d899ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdabcff9640 1 -- 172.21.15.90:0/2575086475 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8020070 con 0x55889be9c820 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdabcff9640 1 -- 172.21.15.90:0/2575086475 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab809cb00 con 0x55889be9c820 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.785+0000 7fdabcff9640 1 -- 172.21.15.90:0/2575086475 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab8051e60 con 0x55889be9c820 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 -- 172.21.15.90:0/2575086475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9c820 msgr2=0x55889bea6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9c820 0x55889bea6ba0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdab80cdbb0 tx=0x7fdab8005880 comp rx=0 tx=0).stop 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 -- 172.21.15.90:0/2575086475 shutdown_connections 2024-04-26T18:29:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889bea70e0 0x55889bea94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9c820 0x55889bea6ba0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2575086475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9bf00 0x55889be9c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 -- 172.21.15.90:0/2575086475 >> 172.21.15.90:0/2575086475 conn(0x55889be98410 msgr2=0x55889be987f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 -- 172.21.15.90:0/2575086475 shutdown_connections 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 -- 172.21.15.90:0/2575086475 wait complete. 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 -- start start 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.786+0000 7fdacf64f740 1 Processor -- start 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889bea4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bea45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.794 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bea4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55889bea9cb0 con 0x55889be9bf00 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55889bd89b60 con 0x55889be9c820 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55889bea9e30 con 0x55889bea70e0 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 10 client.0 _ll_register_callbacks cb 0x55889bdae2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889bea4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889bea4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60320/0 (socket says 172.21.15.90:60320) 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 -- 172.21.15.90:0/2766748239 learned_addr learned my addr 172.21.15.90:0/2766748239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac6d9d640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bea4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdabffff640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bea45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 -- 172.21.15.90:0/2766748239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 msgr2=0x55889bea4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bea4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 -- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 msgr2=0x55889bea45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bea45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 -- 172.21.15.90:0/2766748239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889bb14210 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdabffff640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bea45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdac659c640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889bea4060 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdab80ce1b0 tx=0x7fdab80ce060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729b2c0908651367 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdaadffb640 1 -- 172.21.15.90:0/2766748239 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab80b5060 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdaadffb640 1 -- 172.21.15.90:0/2766748239 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 11423+0+0 (secure 0 0 0) 0x7fdab80bfdb0 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.787+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab809cb00 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdaadffb640 1 -- 172.21.15.90:0/2766748239 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=226) ==== 24+0+0 (secure 0 0 0) 0x7fdab8020070 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 10 client.16674 fetch_fsmap learned FSMap version 226 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=226,monmap=4+}) -- 0x55889bbe0620 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdaadffb640 1 -- 172.21.15.90:0/2766748239 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab809cb00 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdaadffb640 1 -- 172.21.15.90:0/2766748239 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 226) ==== 5926+0+0 (secure 0 0 0) 0x7fdab807e020 con 0x55889be9bf00 2024-04-26T18:29:21.796 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 10 client.16674 fetch_fsmap finished waiting for FSMap version 226 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 20 client.16674 populate_metadata read hostname 'smithi090' 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889beb7030 0x55889beb9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 4 client.16674 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55889be10070 con 0x55889beb7030 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.788+0000 7fdabffff640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889beb7030 0x55889beb9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.789+0000 7fdaadffb640 10 client.16674 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.789+0000 7fdabffff640 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889beb7030 0x55889beb9490 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f18fda8870791e0 server_cookie=645c2c1d2d394692 in_seq=0 out_seq=0 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.789+0000 7fdaadffb640 1 -- 172.21.15.90:0/2766748239 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55889be10070 con 0x55889beb7030 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdaadffb640 10 client.16674 handle_command_reply: tid=0 2024-04-26T18:29:21.797 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 client.16674 shutdown 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 msgr2=0x55889bea4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889bea4060 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fdab80ce1b0 tx=0x7fdab80ce060 comp rx=0 tx=0).stop 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 shutdown_connections 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889beb7030 0x55889beb9490 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bea4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bea45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdaadffb640 0 client.16674 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2766748239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889bea4060 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 >> 172.21.15.90:0/2766748239 conn(0x55889be98410 msgr2=0x55889be9e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 shutdown_connections 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 1 -- 172.21.15.90:0/2766748239 wait complete. 2024-04-26T18:29:21.798 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.790+0000 7fdacf64f740 20 client.16674 trim_cache size 0 max 16384 2024-04-26T18:29:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 Processor -- start 2024-04-26T18:29:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 -- start start 2024-04-26T18:29:21.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889beb2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bebeda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bebf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81af740 con 0x55889be9bf00 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81b12e0 con 0x55889be9c820 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdacf64f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81afe10 con 0x55889bea70e0 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdac659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889beb2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdac659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889beb2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60332/0 (socket says 172.21.15.90:60332) 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdac659c640 1 -- 172.21.15.90:0/2160545878 learned_addr learned my addr 172.21.15.90:0/2160545878 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.805+0000 7fdac6d9d640 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bebf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdabffff640 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bebeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdac659c640 1 -- 172.21.15.90:0/2160545878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 msgr2=0x55889bebf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdac659c640 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bebf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.811 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdac659c640 1 -- 172.21.15.90:0/2160545878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 msgr2=0x55889bebeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdac659c640 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bebeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdac659c640 1 -- 172.21.15.90:0/2160545878 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55889bea9840 con 0x55889be9bf00 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdabffff640 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bebeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdac659c640 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889beb2b70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdab80cdc60 tx=0x7fdab8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501948c5978d7259 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdaaffff640 1 -- 172.21.15.90:0/2160545878 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab80b5070 con 0x55889be9bf00 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdaaffff640 1 -- 172.21.15.90:0/2160545878 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab809cb00 con 0x55889be9bf00 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdaaffff640 1 -- 172.21.15.90:0/2160545878 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab80982e0 con 0x55889be9bf00 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdacf64f740 1 -- 172.21.15.90:0/2160545878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 msgr2=0x55889beb2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.806+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889beb2b70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdab80cdc60 tx=0x7fdab8005880 comp rx=0 tx=0).stop 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- 172.21.15.90:0/2160545878 shutdown_connections 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889bea70e0 0x55889bebf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889be9c820 0x55889bebeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.812 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2160545878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9bf00 0x55889beb2b70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- 172.21.15.90:0/2160545878 >> 172.21.15.90:0/2160545878 conn(0x55889be98410 msgr2=0x55889be975b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- 172.21.15.90:0/2160545878 shutdown_connections 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- 172.21.15.90:0/2160545878 wait complete. 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- start start 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 Processor -- start 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9bf00 0x55889bea0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9c820 0x55889bea1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889bea70e0 0x55889bea1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81b0c10 con 0x55889be9c820 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81b0540 con 0x55889bea70e0 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdac81adf30 con 0x55889be9bf00 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdacf64f740 10 client.0 _ll_register_callbacks cb 0x7fdab800df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:21.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdabffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9c820 0x55889bea1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdabffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9c820 0x55889bea1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60344/0 (socket says 172.21.15.90:60344) 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdabffff640 1 -- 172.21.15.90:0/2418083529 learned_addr learned my addr 172.21.15.90:0/2418083529 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.807+0000 7fdac659c640 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9bf00 0x55889bea0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdac6d9d640 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889bea70e0 0x55889bea1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdabffff640 1 -- 172.21.15.90:0/2418083529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9bf00 msgr2=0x55889bea0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdabffff640 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9bf00 0x55889bea0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdabffff640 1 -- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889bea70e0 msgr2=0x55889bea1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdabffff640 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889bea70e0 0x55889bea1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdabffff640 1 -- 172.21.15.90:0/2418083529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55889bf1c660 con 0x55889be9c820 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdabffff640 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9c820 0x55889bea1310 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdab00058c0 tx=0x7fdab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9e5aba24351e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdaadffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdaadffb640 1 -- 172.21.15.90:0/2418083529 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdab0026d50 con 0x55889be9c820 2024-04-26T18:29:21.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdaadffb640 1 -- 172.21.15.90:0/2418083529 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(150..150 src has 1..150) ==== 11423+0+0 (secure 0 0 0) 0x7fdab00261c0 con 0x55889be9c820 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdab809cb00 con 0x55889be9c820 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdaadffb640 1 -- 172.21.15.90:0/2418083529 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=226) ==== 24+0+0 (secure 0 0 0) 0x7fdab00263c0 con 0x55889be9c820 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdacf64f740 10 client.16680 fetch_fsmap learned FSMap version 226 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=226,monmap=4+}) -- 0x55889bf1d920 con 0x55889be9c820 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdaadffb640 1 -- 172.21.15.90:0/2418083529 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdab809cb00 con 0x55889be9c820 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdaadffb640 1 -- 172.21.15.90:0/2418083529 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 226) ==== 5926+0+0 (secure 0 0 0) 0x7fdab00328a0 con 0x55889be9c820 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdacf64f740 10 client.16680 fetch_fsmap finished waiting for FSMap version 226 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdacf64f740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdacf64f740 20 client.16680 populate_metadata read hostname 'smithi090' 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.808+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889bf1e090 0x55889bf204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.809+0000 7fdacf64f740 4 client.16680 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.809+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55889be09ee0 con 0x55889bf1e090 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.809+0000 7fdac659c640 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889bf1e090 0x55889bf204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:21.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.809+0000 7fdac659c640 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889bf1e090 0x55889bf204b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3ea26bab3845d54 server_cookie=26203517807b09b3 in_seq=0 out_seq=0 2024-04-26T18:29:21.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:21.809+0000 7fdaadffb640 10 client.16680 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:21.884 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 13.31 MB/sec execute 46 sec latency 4195.006 ms 2024-04-26T18:29:22.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 13.02 MB/sec execute 47 sec latency 5195.164 ms 2024-04-26T18:29:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:23 smithi090 ceph-mon[29180]: pgmap v594: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 4.7 MiB/s wr, 537 op/s 2024-04-26T18:29:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:23 smithi090 ceph-mon[29180]: osdmap e151: 12 total, 12 up, 12 in 2024-04-26T18:29:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:23 smithi119 ceph-mon[32752]: pgmap v594: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 4.7 MiB/s wr, 537 op/s 2024-04-26T18:29:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:23 smithi119 ceph-mon[32752]: osdmap e151: 12 total, 12 up, 12 in 2024-04-26T18:29:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:23 smithi167 ceph-mon[32751]: pgmap v594: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 4.7 MiB/s wr, 537 op/s 2024-04-26T18:29:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:23 smithi167 ceph-mon[32751]: osdmap e151: 12 total, 12 up, 12 in 2024-04-26T18:29:23.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 12.75 MB/sec execute 48 sec latency 6195.271 ms 2024-04-26T18:29:24.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 12.49 MB/sec execute 49 sec latency 7195.359 ms 2024-04-26T18:29:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:25 smithi090 ceph-mon[29180]: pgmap v596: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 685 op/s 2024-04-26T18:29:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:25 smithi119 ceph-mon[32752]: pgmap v596: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 685 op/s 2024-04-26T18:29:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:25 smithi167 ceph-mon[32751]: pgmap v596: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 685 op/s 2024-04-26T18:29:25.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 12.24 MB/sec execute 50 sec latency 8195.393 ms 2024-04-26T18:29:26.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 12.00 MB/sec execute 51 sec latency 9195.699 ms 2024-04-26T18:29:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:27 smithi090 ceph-mon[29180]: pgmap v597: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 167 KiB/s wr, 114 op/s 2024-04-26T18:29:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:27 smithi119 ceph-mon[32752]: pgmap v597: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 167 KiB/s wr, 114 op/s 2024-04-26T18:29:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:27 smithi167 ceph-mon[32751]: pgmap v597: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 627 KiB/s rd, 167 KiB/s wr, 114 op/s 2024-04-26T18:29:27.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 11.77 MB/sec execute 52 sec latency 10195.789 ms 2024-04-26T18:29:28.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 11.55 MB/sec execute 53 sec latency 11195.909 ms 2024-04-26T18:29:29.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:29 smithi090 ceph-mon[29180]: pgmap v598: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 141 KiB/s wr, 96 op/s 2024-04-26T18:29:29.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:28 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:29:29.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:28 smithi090 systemd[1]: Stopped Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:29.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:28 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 12.301s CPU time. 2024-04-26T18:29:29.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:28 smithi090 systemd[1]: Starting Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:29:29.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 podman[97646]: 2024-04-26 18:29:29.020462918 +0000 UTC m=+0.018401705 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:29:29.397 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 podman[97646]: 2024-04-26 18:29:29.135998833 +0000 UTC m=+0.133937666 container create bfecc0ee8d439bad838dd890e69e6b99df37fcbfe5322a83acf44030e368abe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True) 2024-04-26T18:29:29.397 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 podman[97646]: 2024-04-26 18:29:29.372133555 +0000 UTC m=+0.370072336 container init bfecc0ee8d439bad838dd890e69e6b99df37fcbfe5322a83acf44030e368abe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T18:29:29.397 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 podman[97646]: 2024-04-26 18:29:29.37475684 +0000 UTC m=+0.372695624 container start bfecc0ee8d439bad838dd890e69e6b99df37fcbfe5322a83acf44030e368abe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T18:29:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:29 smithi119 ceph-mon[32752]: pgmap v598: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 141 KiB/s wr, 96 op/s 2024-04-26T18:29:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:29 smithi167 ceph-mon[32751]: pgmap v598: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 141 KiB/s wr, 96 op/s 2024-04-26T18:29:29.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 ceph-mds[97666]: -- 172.21.15.90:0/139533938 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564f4ba43a40 con 0x564f4ba16800 2024-04-26T18:29:29.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: starting mds.g at 2024-04-26T18:29:29.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 bash[97646]: bfecc0ee8d439bad838dd890e69e6b99df37fcbfe5322a83acf44030e368abe9 2024-04-26T18:29:29.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:29:29 smithi090 systemd[1]: Started Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:29.885 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 11.34 MB/sec execute 54 sec latency 12195.998 ms 2024-04-26T18:29:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: daemon mds.g restarted 2024-04-26T18:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: osdmap e152: 12 total, 12 up, 12 in 2024-04-26T18:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:boot 2024-04-26T18:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:30 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:replay} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: daemon mds.g restarted 2024-04-26T18:29:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: osdmap e152: 12 total, 12 up, 12 in 2024-04-26T18:29:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:boot 2024-04-26T18:29:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:29:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:29:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:30 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:replay} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: daemon mds.g restarted 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: osdmap e152: 12 total, 12 up, 12 in 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:boot 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:30 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:replay} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:30.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 11.13 MB/sec execute 55 sec latency 13196.137 ms 2024-04-26T18:29:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:31 smithi119 ceph-mon[32752]: pgmap v600: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 282 KiB/s wr, 113 op/s 2024-04-26T18:29:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:31 smithi167 ceph-mon[32751]: pgmap v600: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 282 KiB/s wr, 113 op/s 2024-04-26T18:29:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:31 smithi090 ceph-mon[29180]: pgmap v600: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 282 KiB/s wr, 113 op/s 2024-04-26T18:29:31.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 10.93 MB/sec execute 56 sec latency 14196.256 ms 2024-04-26T18:29:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:32 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:resolve 2024-04-26T18:29:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:32 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:resolve} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:32 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:resolve 2024-04-26T18:29:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:32 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:resolve} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:32 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:resolve 2024-04-26T18:29:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:32 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:resolve} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:32.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 22841 10.74 MB/sec execute 57 sec latency 15196.388 ms 2024-04-26T18:29:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.250+0000 7fdaadffb640 1 -- 172.21.15.90:0/2418083529 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55889be09ee0 con 0x55889bf1e090 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.250+0000 7fdaadffb640 10 client.16680 handle_command_reply: tid=0 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.250+0000 7fdacf64f740 1 client.16680 shutdown 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.250+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9c820 msgr2=0x55889bea1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9c820 0x55889bea1310 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdab00058c0 tx=0x7fdab0004730 comp rx=0 tx=0).stop 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 shutdown_connections 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55889bf1e090 0x55889bf204b0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55889bea70e0 0x55889bea1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdaadffb640 0 client.16680 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55889be9c820 0x55889bea1310 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 --2- 172.21.15.90:0/2418083529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55889be9bf00 0x55889bea0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 >> 172.21.15.90:0/2418083529 conn(0x55889be98410 msgr2=0x55889be975b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 shutdown_connections 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 1 -- 172.21.15.90:0/2418083529 wait complete. 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.251+0000 7fdacf64f740 20 client.16680 trim_cache size 0 max 16384 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:29:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:29:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdab40a3dc0 msgr2=0x7fdab40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdab40a3dc0 0x7fdab40a6280 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005fd0 tx=0x7fdab0005850 comp rx=0 tx=0).stop 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac8103be0 msgr2=0x7fdac8198ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac8103be0 0x7fdac8198ff0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fdab800d950 tx=0x7fdab800de10 comp rx=0 tx=0).stop 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 shutdown_connections 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fdab40a3dc0 0x7fdab40a6280 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdac810cfb0 0x7fdac819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac8104500 0x7fdac8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 --2- 172.21.15.90:0/1955654050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdac8103be0 0x7fdac8198ff0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.252+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 >> 172.21.15.90:0/1955654050 conn(0x7fdac8077bb0 msgr2=0x7fdac80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.253+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 shutdown_connections 2024-04-26T18:29:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:33.253+0000 7fdacc939640 1 -- 172.21.15.90:0/1955654050 wait complete. 2024-04-26T18:29:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:33 smithi119 ceph-mon[32752]: pgmap v601: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 239 KiB/s wr, 96 op/s 2024-04-26T18:29:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:33 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:reconnect 2024-04-26T18:29:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:33 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:reconnect} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:33 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00499992 2024-04-26T18:29:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:33 smithi167 ceph-mon[32751]: pgmap v601: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 239 KiB/s wr, 96 op/s 2024-04-26T18:29:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:33 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:reconnect 2024-04-26T18:29:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:33 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:reconnect} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:33 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00499992 2024-04-26T18:29:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:33 smithi090 ceph-mon[29180]: pgmap v601: 97 pgs: 97 active+clean; 502 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 239 KiB/s wr, 96 op/s 2024-04-26T18:29:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:33 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:reconnect 2024-04-26T18:29:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:33 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:reconnect} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:33 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00499992 2024-04-26T18:29:33.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 23261 10.86 MB/sec execute 58 sec latency 16186.481 ms 2024-04-26T18:29:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:rejoin 2024-04-26T18:29:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:34 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:rejoin} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:34 smithi119 ceph-mon[32752]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:29:34.567 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:29:34 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:29:34.168+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:29:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:rejoin 2024-04-26T18:29:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:34 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:rejoin} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:34 smithi167 ceph-mon[32751]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:29:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:rejoin 2024-04-26T18:29:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:34 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=l=up:active,2=a=up:rejoin} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:34 smithi090 ceph-mon[29180]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:29:34.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24004 11.44 MB/sec execute 59 sec latency 16608.089 ms 2024-04-26T18:29:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:35 smithi119 ceph-mon[32752]: pgmap v602: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:35 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:active 2024-04-26T18:29:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:35 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:35 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:29:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:35 smithi167 ceph-mon[32751]: pgmap v602: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:35 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:active 2024-04-26T18:29:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:35 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:35 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:29:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:35 smithi090 ceph-mon[29180]: pgmap v602: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:35 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/3362405108,v1:172.21.15.90:6841/3362405108] up:active 2024-04-26T18:29:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:35 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=a=up:active} 2 up:standby-replay 3 up:standby 2024-04-26T18:29:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:35 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=l=up:active,2=a=up:active} 3 up:standby-replay 2 up:standby 2024-04-26T18:29:35.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 24764 12.16 MB/sec execute 60 sec latency 49.006 ms 2024-04-26T18:29:36.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25366 12.43 MB/sec execute 61 sec latency 60.436 ms 2024-04-26T18:29:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:37 smithi119 ceph-mon[32752]: pgmap v603: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-mon[32751]: pgmap v603: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:37 smithi090 ceph-mon[29180]: pgmap v603: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:37.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 12.64 MB/sec execute 62 sec latency 179.390 ms 2024-04-26T18:29:38.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa5d77dd700 time 2024-04-26T18:29:37.726953+0000 2024-04-26T18:29:38.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa5e1e063dd] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.069 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: clone() 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:37.726+0000 7fa5d77dd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa5d77dd700 time 2024-04-26T18:29:37.726953+0000 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa5e1e063dd] 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.070 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: clone() 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: *** Caught signal (Aborted) ** 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: in thread 7fa5d77dd700 thread_name:ms_dispatch 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5e0b8ed20] 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: gsignal() 2024-04-26T18:29:38.071 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: abort() 2024-04-26T18:29:38.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa5e1e06437] 2024-04-26T18:29:38.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.072 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 16: clone() 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:37.728+0000 7fa5d77dd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:38.073 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: in thread 7fa5d77dd700 thread_name:ms_dispatch 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5e0b8ed20] 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: gsignal() 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: abort() 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa5e1e06437] 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.074 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 16: clone() 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: -1> 2024-04-26T18:29:37.726+0000 7fa5d77dd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa5d77dd700 time 2024-04-26T18:29:37.726953+0000 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa5e1e063dd] 2024-04-26T18:29:38.075 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: clone() 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.076 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 0> 2024-04-26T18:29:37.728+0000 7fa5d77dd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: in thread 7fa5d77dd700 thread_name:ms_dispatch 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5e0b8ed20] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: gsignal() 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: abort() 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa5e1e06437] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.077 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 16: clone() 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: -1> 2024-04-26T18:29:37.726+0000 7fa5d77dd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa5d77dd700 time 2024-04-26T18:29:37.726953+0000 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.078 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa5e1e063dd] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: clone() 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.079 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 0> 2024-04-26T18:29:37.728+0000 7fa5d77dd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: in thread 7fa5d77dd700 thread_name:ms_dispatch 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa5e0b8ed20] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2: gsignal() 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 3: abort() 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa5e1e06437] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa5e1e065a3] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a1f1200822] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a1f1170598] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a1f1172c17] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a1f106bd0b] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a1f106c2ac] 2024-04-26T18:29:38.080 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a1f1054fad] 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa5e209ae20] 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 13: (DispatchQueue::entry()+0x51f) [0x7fa5e2097fdf] 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa5e216afc1] 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa5e0b841ca] 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 16: clone() 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l[72330]: 2024-04-26T18:29:38.081 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:37 smithi167 podman[72766]: 2024-04-26 18:29:37.786757182 +0000 UTC m=+0.019340835 container died 6c28c32abdbcceabbe04a8b75e14b05e25de5b15e415bbd2dde3a815430a2f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T18:29:38.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 12.44 MB/sec execute 63 sec latency 1179.510 ms 2024-04-26T18:29:39.067 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:38 smithi167 podman[72766]: 2024-04-26 18:29:38.69676898 +0000 UTC m=+0.929352613 container remove 6c28c32abdbcceabbe04a8b75e14b05e25de5b15e415bbd2dde3a815430a2f22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T18:29:39.067 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:38 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:29:39.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:38 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-04-26T18:29:39.068 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:38 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 19.044s CPU time. 2024-04-26T18:29:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:39 smithi119 ceph-mon[32752]: pgmap v604: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:39 smithi167 ceph-mon[32751]: pgmap v604: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-mon[29180]: pgmap v604: 97 pgs: 97 active+clean; 502 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 535 KiB/s wr, 30 op/s 2024-04-26T18:29:39.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f355e009700 time 2024-04-26T18:29:39.290720+0000 2024-04-26T18:29:39.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:39.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f35686323dd] 2024-04-26T18:29:39.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: clone() 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.291+0000 7f355e009700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f355e009700 time 2024-04-26T18:29:39.290720+0000 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:39.627 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f35686323dd] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.629 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: clone() 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: *** Caught signal (Aborted) ** 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: in thread 7f355e009700 thread_name:ms_dispatch 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f35673bad20] 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: gsignal() 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: abort() 2024-04-26T18:29:39.630 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3568632437] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 16: clone() 2024-04-26T18:29:39.631 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.292+0000 7f355e009700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: in thread 7f355e009700 thread_name:ms_dispatch 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f35673bad20] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: gsignal() 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: abort() 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3568632437] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.632 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 16: clone() 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: -1> 2024-04-26T18:29:39.291+0000 7f355e009700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f355e009700 time 2024-04-26T18:29:39.290720+0000 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.633 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f35686323dd] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: clone() 2024-04-26T18:29:39.634 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 0> 2024-04-26T18:29:39.292+0000 7f355e009700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: in thread 7f355e009700 thread_name:ms_dispatch 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f35673bad20] 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: gsignal() 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: abort() 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3568632437] 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.635 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 16: clone() 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: -1> 2024-04-26T18:29:39.291+0000 7f355e009700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f355e009700 time 2024-04-26T18:29:39.290720+0000 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.636 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f35686323dd] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: clone() 2024-04-26T18:29:39.637 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 0> 2024-04-26T18:29:39.292+0000 7f355e009700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: in thread 7f355e009700 thread_name:ms_dispatch 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f35673bad20] 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2: gsignal() 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 3: abort() 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3568632437] 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f35686325a3] 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55838e1cb822] 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55838e13b598] 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55838e13dc17] 2024-04-26T18:29:39.638 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55838e036d0b] 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55838e0372ac] 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55838e01ffad] 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f35688c6e20] 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 13: (DispatchQueue::entry()+0x51f) [0x7f35688c3fdf] 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3568996fc1] 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f35673b01ca] 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 16: clone() 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97323]: 2024-04-26T18:29:39.639 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:39 smithi090 podman[97703]: 2024-04-26 18:29:39.36372347 +0000 UTC m=+0.022263800 container died 46532ba4a1d0d26286a77390018c69c3434bf33e8e44c206ad42eada58a8d9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T18:29:39.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 12.24 MB/sec execute 64 sec latency 2179.626 ms 2024-04-26T18:29:40.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:40 smithi090 podman[97703]: 2024-04-26 18:29:40.371729381 +0000 UTC m=+1.030269691 container remove 46532ba4a1d0d26286a77390018c69c3434bf33e8e44c206ad42eada58a8d9f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T18:29:40.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:40 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:29:40.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:40 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-04-26T18:29:40.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:40 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 12.666s CPU time. 2024-04-26T18:29:40.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 12.05 MB/sec execute 65 sec latency 3179.734 ms 2024-04-26T18:29:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:41 smithi119 ceph-mon[32752]: pgmap v605: 97 pgs: 97 active+clean; 521 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-04-26T18:29:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:41 smithi167 ceph-mon[32751]: pgmap v605: 97 pgs: 97 active+clean; 521 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-04-26T18:29:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:41 smithi090 ceph-mon[29180]: pgmap v605: 97 pgs: 97 active+clean; 521 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 3.3 MiB/s wr, 243 op/s 2024-04-26T18:29:41.886 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 11.87 MB/sec execute 66 sec latency 4179.826 ms 2024-04-26T18:29:42.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 11.69 MB/sec execute 67 sec latency 5179.915 ms 2024-04-26T18:29:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:43 smithi119 ceph-mon[32752]: pgmap v606: 97 pgs: 97 active+clean; 521 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-04-26T18:29:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:43 smithi167 ceph-mon[32751]: pgmap v606: 97 pgs: 97 active+clean; 521 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-04-26T18:29:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:43 smithi090 ceph-mon[29180]: pgmap v606: 97 pgs: 97 active+clean; 521 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 216 op/s 2024-04-26T18:29:43.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 11.52 MB/sec execute 68 sec latency 6180.032 ms 2024-04-26T18:29:44.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 11.36 MB/sec execute 69 sec latency 7180.126 ms 2024-04-26T18:29:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:45 smithi119 ceph-mon[32752]: pgmap v607: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-04-26T18:29:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:45 smithi167 ceph-mon[32751]: pgmap v607: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-04-26T18:29:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:45 smithi090 ceph-mon[29180]: pgmap v607: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 3.3 MiB/s wr, 228 op/s 2024-04-26T18:29:45.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 11.19 MB/sec execute 70 sec latency 8180.174 ms 2024-04-26T18:29:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.350+0000 7fb473ec5640 1 -- 172.21.15.90:0/1667847305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c102930 msgr2=0x7fb46c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.350+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1667847305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c102930 0x7fb46c102db0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb468009930 tx=0x7fb46802a420 comp rx=0 tx=0).stop 2024-04-26T18:29:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.350+0000 7fb473ec5640 1 -- 172.21.15.90:0/1667847305 shutdown_connections 2024-04-26T18:29:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.350+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1667847305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb46c1032f0 0x7fb46c10f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.350+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1667847305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c102930 0x7fb46c102db0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.350+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1667847305 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46c108940 0x7fb46c108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.350+0000 7fb473ec5640 1 -- 172.21.15.90:0/1667847305 >> 172.21.15.90:0/1667847305 conn(0x7fb46c0fe400 msgr2=0x7fb46c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 -- 172.21.15.90:0/1667847305 shutdown_connections 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 -- 172.21.15.90:0/1667847305 wait complete. 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 Processor -- start 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 -- start start 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46c102930 0x7fb46c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb46c1032f0 0x7fb46c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c108940 0x7fb46c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c112c00 con 0x7fb46c108940 2024-04-26T18:29:46.354 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c112a80 con 0x7fb46c1032f0 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb473ec5640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c112d80 con 0x7fb46c102930 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.351+0000 7fb472462640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c108940 0x7fb46c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c108940 0x7fb46c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45868/0 (socket says 172.21.15.90:45868) 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 -- 172.21.15.90:0/1622278328 learned_addr learned my addr 172.21.15.90:0/1622278328 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb471c61640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46c102930 0x7fb46c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb471460640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb46c1032f0 0x7fb46c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 -- 172.21.15.90:0/1622278328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46c102930 msgr2=0x7fb46c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46c102930 0x7fb46c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 -- 172.21.15.90:0/1622278328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb46c1032f0 msgr2=0x7fb46c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb46c1032f0 0x7fb46c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 -- 172.21.15.90:0/1622278328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb46c19e110 con 0x7fb46c108940 2024-04-26T18:29:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb471460640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb46c1032f0 0x7fb46c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:29:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb472462640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c108940 0x7fb46c19d9e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb464009fd0 tx=0x7fb46400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb462ffd640 1 -- 172.21.15.90:0/1622278328 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464017070 con 0x7fb46c108940 2024-04-26T18:29:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb462ffd640 1 -- 172.21.15.90:0/1622278328 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb464003e20 con 0x7fb46c108940 2024-04-26T18:29:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb46c19e400 con 0x7fb46c108940 2024-04-26T18:29:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb462ffd640 1 -- 172.21.15.90:0/1622278328 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb464010460 con 0x7fb46c108940 2024-04-26T18:29:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.352+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb46c1a5c90 con 0x7fb46c108940 2024-04-26T18:29:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.355+0000 7fb462ffd640 1 -- 172.21.15.90:0/1622278328 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fb464003970 con 0x7fb46c108940 2024-04-26T18:29:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.355+0000 7fb476bd7740 1 Processor -- start 2024-04-26T18:29:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.355+0000 7fb476bd7740 1 -- start start 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb476bd7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560512730460 con 0x5605127d50e0 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb476bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5605126ea590 con 0x5605127ca820 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb476bd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56051272c420 con 0x5605127c9f00 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb462ffd640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb4580a3dc0 0x7fb4580a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45872/0 (socket says 172.21.15.90:45872) 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 -- 172.21.15.90:0/3553988765 learned_addr learned my addr 172.21.15.90:0/3553988765 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb472462640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb4580a3dc0 0x7fb4580a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471c61640 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb462ffd640 1 -- 172.21.15.90:0/1622278328 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 11421+0+0 (secure 0 0 0) 0x7fb4640d1170 con 0x7fb46c108940 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 -- 172.21.15.90:0/3553988765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 msgr2=0x5605127ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 -- 172.21.15.90:0/3553988765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 msgr2=0x5605127d4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 -- 172.21.15.90:0/3553988765 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605126f70d0 con 0x5605127d50e0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb472462640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb4580a3dc0 0x7fb4580a6280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4640989c0 tx=0x7fb464004480 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.356+0000 7fb471460640 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d74d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb468033000 tx=0x7fb468031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57b41f5686426c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb457fff640 1 -- 172.21.15.90:0/3553988765 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468045070 con 0x5605127d50e0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb457fff640 1 -- 172.21.15.90:0/3553988765 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680035f0 con 0x5605127d50e0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb457fff640 1 -- 172.21.15.90:0/3553988765 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468035040 con 0x5605127d50e0 2024-04-26T18:29:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb476bd7740 1 -- 172.21.15.90:0/3553988765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 msgr2=0x5605127d74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d74d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fb468033000 tx=0x7fb468031400 comp rx=0 tx=0).stop 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb476bd7740 1 -- 172.21.15.90:0/3553988765 shutdown_connections 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d74d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3553988765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.357+0000 7fb476bd7740 1 -- 172.21.15.90:0/3553988765 >> 172.21.15.90:0/3553988765 conn(0x5605127c6410 msgr2=0x5605127c67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 -- 172.21.15.90:0/3553988765 shutdown_connections 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 -- 172.21.15.90:0/3553988765 wait complete. 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 -- start start 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 Processor -- start 2024-04-26T18:29:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5605127d7cb0 con 0x5605127d50e0 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5605126b7b60 con 0x5605127ca820 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5605127d7e30 con 0x5605127c9f00 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.358+0000 7fb476bd7740 10 client.0 _ll_register_callbacks cb 0x5605126dc2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45878/0 (socket says 172.21.15.90:45878) 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 -- 172.21.15.90:0/1215724809 learned_addr learned my addr 172.21.15.90:0/1215724809 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471c61640 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 -- 172.21.15.90:0/1215724809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 msgr2=0x5605127d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 -- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 msgr2=0x5605127d2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 -- 172.21.15.90:0/1215724809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560512442210 con 0x5605127d50e0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb471460640 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d2b80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb468031a60 tx=0x7fb468003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8b6795224ddf02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb4557fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb4557fa640 1 -- 172.21.15.90:0/1215724809 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46804e020 con 0x5605127d50e0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb4557fa640 1 -- 172.21.15.90:0/1215724809 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 11421+0+0 (secure 0 0 0) 0x7fb4680041b0 con 0x5605127d50e0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4680035f0 con 0x5605127d50e0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb4557fa640 1 -- 172.21.15.90:0/1215724809 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=233) ==== 24+0+0 (secure 0 0 0) 0x7fb468035070 con 0x5605127d50e0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb476bd7740 10 client.16698 fetch_fsmap learned FSMap version 233 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.359+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=233,monmap=4+}) -- 0x56051250e620 con 0x5605127d50e0 2024-04-26T18:29:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb4557fa640 1 -- 172.21.15.90:0/1215724809 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4680035f0 con 0x5605127d50e0 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb4557fa640 1 -- 172.21.15.90:0/1215724809 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 233) ==== 5926+0+0 (secure 0 0 0) 0x7fb46803a040 con 0x5605127d50e0 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb476bd7740 10 client.16698 fetch_fsmap finished waiting for FSMap version 233 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb476bd7740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb476bd7740 20 client.16698 populate_metadata read hostname 'smithi090' 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb476bd7740 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5605127e5150 0x5605127e7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb476bd7740 4 client.16698 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56051273e070 con 0x5605127e5150 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb471c61640 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5605127e5150 0x5605127e7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb4557fa640 10 client.16698 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.360+0000 7fb471c61640 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5605127e5150 0x5605127e7560 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7132fecbd14ea7d server_cookie=ab4c6e69209a6240 in_seq=0 out_seq=0 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb4557fa640 1 -- 172.21.15.90:0/1215724809 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56051273e070 con 0x5605127e5150 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb4557fa640 10 client.16698 handle_command_reply: tid=0 2024-04-26T18:29:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 client.16698 shutdown 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 msgr2=0x5605127d2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d2b80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fb468031a60 tx=0x7fb468003e60 comp rx=0 tx=0).stop 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 shutdown_connections 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5605127e5150 0x5605127e7560 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127d50e0 0x5605127d2b80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127ca820 0x5605127d2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb4557fa640 0 client.16698 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 --2- 172.21.15.90:0/1215724809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127c9f00 0x5605127d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 >> 172.21.15.90:0/1215724809 conn(0x5605127c6410 msgr2=0x5605127cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 shutdown_connections 2024-04-26T18:29:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.361+0000 7fb476bd7740 1 -- 172.21.15.90:0/1215724809 wait complete. 2024-04-26T18:29:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.362+0000 7fb476bd7740 20 client.16698 trim_cache size 0 max 16384 2024-04-26T18:29:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 Processor -- start 2024-04-26T18:29:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 -- start start 2024-04-26T18:29:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127c9f00 0x5605127e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127ca820 0x5605127c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127d50e0 0x5605127cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1af6b0 con 0x5605127c9f00 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b1250 con 0x5605127d50e0 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.376+0000 7fb476bd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1afd80 con 0x5605127ca820 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127c9f00 0x5605127e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127c9f00 0x5605127e0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45890/0 (socket says 172.21.15.90:45890) 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 -- 172.21.15.90:0/3588145392 learned_addr learned my addr 172.21.15.90:0/3588145392 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 -- 172.21.15.90:0/3588145392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127ca820 msgr2=0x5605127c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 --2- 172.21.15.90:0/3588145392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127ca820 0x5605127c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 -- 172.21.15.90:0/3588145392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127d50e0 msgr2=0x5605127cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 --2- 172.21.15.90:0/3588145392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127d50e0 0x5605127cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 -- 172.21.15.90:0/3588145392 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605127e7be0 con 0x5605127c9f00 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb471460640 1 --2- 172.21.15.90:0/3588145392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127c9f00 0x5605127e0b70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb468009a50 tx=0x7fb468005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f36f18c2383c4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb4577fe640 1 -- 172.21.15.90:0/3588145392 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46804e040 con 0x5605127c9f00 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb4577fe640 1 -- 172.21.15.90:0/3588145392 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb468042030 con 0x5605127c9f00 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.377+0000 7fb4577fe640 1 -- 172.21.15.90:0/3588145392 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468045070 con 0x5605127c9f00 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 -- 172.21.15.90:0/3588145392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127c9f00 msgr2=0x5605127e0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3588145392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127c9f00 0x5605127e0b70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fb468009a50 tx=0x7fb468005a20 comp rx=0 tx=0).stop 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 -- 172.21.15.90:0/3588145392 shutdown_connections 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3588145392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127d50e0 0x5605127cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3588145392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127ca820 0x5605127c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3588145392 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127c9f00 0x5605127e0b70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 -- 172.21.15.90:0/3588145392 >> 172.21.15.90:0/3588145392 conn(0x5605127c6410 msgr2=0x5605127cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 -- 172.21.15.90:0/3588145392 shutdown_connections 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 -- 172.21.15.90:0/3588145392 wait complete. 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 -- start start 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.378+0000 7fb476bd7740 1 Processor -- start 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127c9f00 0x5605127cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127ca820 0x5605127cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127d50e0 0x5605127cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b0b80 con 0x5605127ca820 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1b04b0 con 0x5605127c9f00 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb46c1ade80 con 0x5605127d50e0 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 10 client.0 _ll_register_callbacks cb 0x7fb45c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127ca820 0x5605127cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127ca820 0x5605127cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45892/0 (socket says 172.21.15.90:45892) 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 -- 172.21.15.90:0/3635958810 learned_addr learned my addr 172.21.15.90:0/3635958810 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471460640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127c9f00 0x5605127cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471460640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127d50e0 0x5605127cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 -- 172.21.15.90:0/3635958810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127d50e0 msgr2=0x5605127cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127d50e0 0x5605127cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 -- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127c9f00 msgr2=0x5605127cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127c9f00 0x5605127cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 -- 172.21.15.90:0/3635958810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605127d74d0 con 0x5605127ca820 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471460640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127c9f00 0x5605127cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb471c61640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127ca820 0x5605127cf580 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0041e0 tx=0x7fb45c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cae95f0013af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb4557fa640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb4557fa640 1 -- 172.21.15.90:0/3635958810 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45c005220 con 0x5605127ca820 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb468042030 con 0x5605127ca820 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.379+0000 7fb4557fa640 1 -- 172.21.15.90:0/3635958810 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(152..152 src has 1..152) ==== 11421+0+0 (secure 0 0 0) 0x7fb45c0053c0 con 0x5605127ca820 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb4557fa640 1 -- 172.21.15.90:0/3635958810 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=233) ==== 24+0+0 (secure 0 0 0) 0x7fb45c003e90 con 0x5605127ca820 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 10 client.16704 fetch_fsmap learned FSMap version 233 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=233,monmap=4+}) -- 0x56051284bac0 con 0x5605127ca820 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb4557fa640 1 -- 172.21.15.90:0/3635958810 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb468042030 con 0x5605127ca820 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb4557fa640 1 -- 172.21.15.90:0/3635958810 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 233) ==== 5926+0+0 (secure 0 0 0) 0x7fb45c02aba0 con 0x5605127ca820 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 10 client.16704 fetch_fsmap finished waiting for FSMap version 233 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 10 client.16704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 20 client.16704 populate_metadata read hostname 'smithi090' 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56051284c2d0 0x56051284e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 4 client.16704 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560512737ee0 con 0x56051284c2d0 2024-04-26T18:29:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.380+0000 7fb471460640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56051284c2d0 0x56051284e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.381+0000 7fb4557fa640 10 client.16704 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:46.381+0000 7fb471460640 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56051284c2d0 0x56051284e6f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec92436b3acac12 server_cookie=1f54f7cd895e6d57 in_seq=0 out_seq=0 2024-04-26T18:29:46.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 11.04 MB/sec execute 71 sec latency 9180.309 ms 2024-04-26T18:29:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:47 smithi119 ceph-mon[32752]: pgmap v608: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-04-26T18:29:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:47 smithi167 ceph-mon[32751]: pgmap v608: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-04-26T18:29:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:47 smithi090 ceph-mon[29180]: pgmap v608: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-04-26T18:29:47.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 10.88 MB/sec execute 72 sec latency 10180.366 ms 2024-04-26T18:29:48.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 10.73 MB/sec execute 73 sec latency 11180.458 ms 2024-04-26T18:29:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:49 smithi119 ceph-mon[32752]: pgmap v609: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-04-26T18:29:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:49 smithi167 ceph-mon[32751]: pgmap v609: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-04-26T18:29:49.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:49 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Scheduled restart job, restart counter is at 6. 2024-04-26T18:29:49.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:49 smithi167 systemd[1]: Stopped Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:49.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:49 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Consumed 19.044s CPU time. 2024-04-26T18:29:49.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:49 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Start request repeated too quickly. 2024-04-26T18:29:49.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:49 smithi167 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.l.service: Failed with result 'exit-code'. 2024-04-26T18:29:49.568 INFO:journalctl@ceph.mds.l.smithi167.stdout:Apr 26 18:29:49 smithi167 systemd[1]: Failed to start Ceph mds.l for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:49 smithi090 ceph-mon[29180]: pgmap v609: 97 pgs: 97 active+clean; 525 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-04-26T18:29:49.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 10.59 MB/sec execute 74 sec latency 12180.574 ms 2024-04-26T18:29:50.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 10.45 MB/sec execute 75 sec latency 13180.654 ms 2024-04-26T18:29:50.994 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:50 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:29:50.994 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:50 smithi090 systemd[1]: Stopped Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:50.994 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:50 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 12.666s CPU time. 2024-04-26T18:29:50.994 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:50 smithi090 systemd[1]: Starting Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:29:50.994 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:50 smithi090 podman[97843]: 2024-04-26 18:29:50.762333166 +0000 UTC m=+0.018791009 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:29:50.994 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:50 smithi090 podman[97843]: 2024-04-26 18:29:50.873934782 +0000 UTC m=+0.130392649 container create 3b4faade63458f04b9af805c5bd829b8fd1a72060edaea80ce4e995d9e64ad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T18:29:51.325 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:51 smithi090 podman[97843]: 2024-04-26 18:29:51.110210288 +0000 UTC m=+0.366668113 container init 3b4faade63458f04b9af805c5bd829b8fd1a72060edaea80ce4e995d9e64ad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:51 smithi090 podman[97843]: 2024-04-26 18:29:51.112805193 +0000 UTC m=+0.369263019 container start 3b4faade63458f04b9af805c5bd829b8fd1a72060edaea80ce4e995d9e64ad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mds[97862]: -- 172.21.15.90:0/1636152308 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56398a119a40 con 0x56398a0ecc00 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: starting mds.a at 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: pgmap v610: 97 pgs: 97 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: daemon mds.a restarted 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: osdmap e153: 12 total, 12 up, 12 in 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:boot 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: Replacing daemon mds.l as rank 1 with standby daemon mds.k 2024-04-26T18:29:51.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-04-26T18:29:51.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: Standby daemon mds.g assigned to filesystem cephfs as rank 2 2024-04-26T18:29:51.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:51.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:29:51.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: osdmap e154: 12 total, 12 up, 12 in 2024-04-26T18:29:51.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:51 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=g=up:replay} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: pgmap v610: 97 pgs: 97 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: daemon mds.a restarted 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: osdmap e153: 12 total, 12 up, 12 in 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:boot 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: Replacing daemon mds.l as rank 1 with standby daemon mds.k 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: Standby daemon mds.g assigned to filesystem cephfs as rank 2 2024-04-26T18:29:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:29:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: osdmap e154: 12 total, 12 up, 12 in 2024-04-26T18:29:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:51 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=g=up:replay} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: pgmap v610: 97 pgs: 97 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 322 op/s 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: daemon mds.a restarted 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: osdmap e153: 12 total, 12 up, 12 in 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:boot 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: Replacing daemon mds.l as rank 1 with standby daemon mds.k 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: MDS daemon mds.l is removed because it is dead or otherwise unavailable. 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: Standby daemon mds.g assigned to filesystem cephfs as rank 2 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=l=up:active} 3 up:standby-replay 3 up:standby, 1 failed 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: osdmap e154: 12 total, 12 up, 12 in 2024-04-26T18:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:51 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:replay,2=g=up:replay} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:51.625 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:51 smithi090 bash[97843]: 3b4faade63458f04b9af805c5bd829b8fd1a72060edaea80ce4e995d9e64ad19 2024-04-26T18:29:51.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:29:51 smithi090 systemd[1]: Started Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:29:51.887 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 10.31 MB/sec execute 76 sec latency 14180.781 ms 2024-04-26T18:29:52.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 10.18 MB/sec execute 77 sec latency 15180.873 ms 2024-04-26T18:29:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:53 smithi119 ceph-mon[32752]: pgmap v613: 97 pgs: 97 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.9 MiB/s wr, 160 op/s 2024-04-26T18:29:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:53 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:resolve 2024-04-26T18:29:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:53 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:resolve 2024-04-26T18:29:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:53 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=g=up:resolve} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:53 smithi167 ceph-mon[32751]: pgmap v613: 97 pgs: 97 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.9 MiB/s wr, 160 op/s 2024-04-26T18:29:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:53 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:resolve 2024-04-26T18:29:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:53 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:resolve 2024-04-26T18:29:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:53 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=g=up:resolve} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:53 smithi090 ceph-mon[29180]: pgmap v613: 97 pgs: 97 active+clean; 547 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 2.9 MiB/s wr, 160 op/s 2024-04-26T18:29:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:53 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:resolve 2024-04-26T18:29:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:53 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:resolve 2024-04-26T18:29:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:53 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:resolve,2=g=up:resolve} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:53.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 25904 10.04 MB/sec execute 78 sec latency 16180.963 ms 2024-04-26T18:29:54.302 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.301+0000 7fb4557fa640 1 -- 172.21.15.90:0/3635958810 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560512737ee0 con 0x56051284c2d0 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.301+0000 7fb4557fa640 10 client.16704 handle_command_reply: tid=0 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.301+0000 7fb476bd7740 1 client.16704 shutdown 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.301+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127ca820 msgr2=0x5605127cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.301+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127ca820 0x5605127cf580 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0041e0 tx=0x7fb45c00be50 comp rx=0 tx=0).stop 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 shutdown_connections 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56051284c2d0 0x56051284e6f0 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5605127d50e0 0x5605127cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5605127ca820 0x5605127cf580 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 --2- 172.21.15.90:0/3635958810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5605127c9f00 0x5605127cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb4557fa640 0 client.16704 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 >> 172.21.15.90:0/3635958810 conn(0x5605127c6410 msgr2=0x5605127cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 shutdown_connections 2024-04-26T18:29:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 1 -- 172.21.15.90:0/3635958810 wait complete. 2024-04-26T18:29:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.302+0000 7fb476bd7740 20 client.16704 trim_cache size 0 max 16384 2024-04-26T18:29:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:29:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:29:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:29:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:29:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb4580a3dc0 msgr2=0x7fb4580a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb4580a3dc0 0x7fb4580a6280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb4640989c0 tx=0x7fb464004480 comp rx=0 tx=0).stop 2024-04-26T18:29:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c108940 msgr2=0x7fb46c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c108940 0x7fb46c19d9e0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb464009fd0 tx=0x7fb46400b260 comp rx=0 tx=0).stop 2024-04-26T18:29:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 shutdown_connections 2024-04-26T18:29:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fb4580a3dc0 0x7fb4580a6280 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb46c108940 0x7fb46c19d9e0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb46c1032f0 0x7fb46c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 --2- 172.21.15.90:0/1622278328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb46c102930 0x7fb46c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 >> 172.21.15.90:0/1622278328 conn(0x7fb46c0fe400 msgr2=0x7fb46c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 shutdown_connections 2024-04-26T18:29:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:29:54.303+0000 7fb473ec5640 1 -- 172.21.15.90:0/1622278328 wait complete. 2024-04-26T18:29:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:54 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:29:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:54 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:reconnect 2024-04-26T18:29:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:54 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:reconnect 2024-04-26T18:29:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:54 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=g=up:reconnect} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:54 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:29:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:54 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699983 2024-04-26T18:29:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:54 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:29:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:54 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:reconnect 2024-04-26T18:29:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:54 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:reconnect 2024-04-26T18:29:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:54 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=g=up:reconnect} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:54 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:29:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:54 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699983 2024-04-26T18:29:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:54 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:29:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:54 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:reconnect 2024-04-26T18:29:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:54 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:reconnect 2024-04-26T18:29:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:54 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:reconnect,2=g=up:reconnect} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:54 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:29:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:54 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699983 2024-04-26T18:29:54.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26091 10.02 MB/sec execute 79 sec latency 17170.302 ms 2024-04-26T18:29:55.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:55 smithi090 ceph-mon[29180]: pgmap v614: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-04-26T18:29:55.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:rejoin 2024-04-26T18:29:55.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:rejoin 2024-04-26T18:29:55.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:55 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=g=up:rejoin} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:55.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:55 smithi090 ceph-mon[29180]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T18:29:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:55 smithi119 ceph-mon[32752]: pgmap v614: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-04-26T18:29:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:rejoin 2024-04-26T18:29:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:rejoin 2024-04-26T18:29:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:55 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=g=up:rejoin} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:55 smithi119 ceph-mon[32752]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T18:29:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:55 smithi167 ceph-mon[32751]: pgmap v614: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-04-26T18:29:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:rejoin 2024-04-26T18:29:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:rejoin 2024-04-26T18:29:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:55 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:rejoin,2=g=up:rejoin} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:55 smithi167 ceph-mon[32751]: daemon mds.g is now active in filesystem cephfs as rank 2 2024-04-26T18:29:55.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 26616 10.17 MB/sec execute 80 sec latency 18170.332 ms 2024-04-26T18:29:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:56 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:clientreplay 2024-04-26T18:29:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:56 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:active 2024-04-26T18:29:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:56 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=g=up:active} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:56 smithi119 ceph-mon[32752]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:29:56.567 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:29:56 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:29:56.246+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 14 2024-04-26T18:29:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:56 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:clientreplay 2024-04-26T18:29:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:56 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:active 2024-04-26T18:29:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:56 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=g=up:active} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:56 smithi167 ceph-mon[32751]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:29:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:56 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:clientreplay 2024-04-26T18:29:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:56 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/862440561,v1:172.21.15.90:6839/862440561] up:active 2024-04-26T18:29:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:56 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:clientreplay,2=g=up:active} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:56 smithi090 ceph-mon[29180]: daemon mds.k is now active in filesystem cephfs as rank 1 2024-04-26T18:29:56.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27250 10.56 MB/sec execute 81 sec latency 18635.668 ms 2024-04-26T18:29:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:57 smithi119 ceph-mon[32752]: pgmap v615: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-04-26T18:29:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:57 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:57 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:active 2024-04-26T18:29:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:57 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=g=up:active} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:57 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=g=up:active} 3 up:standby-replay 1 up:standby 2024-04-26T18:29:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:57 smithi167 ceph-mon[32751]: pgmap v615: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-04-26T18:29:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:57 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:57 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:active 2024-04-26T18:29:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:57 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=g=up:active} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:57 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=g=up:active} 3 up:standby-replay 1 up:standby 2024-04-26T18:29:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:57 smithi090 ceph-mon[29180]: pgmap v615: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.9 MiB/s wr, 184 op/s 2024-04-26T18:29:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:57 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:29:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:57 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6832/1677033493,v1:172.21.15.119:6833/1677033493] up:active 2024-04-26T18:29:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:57 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=g=up:active} 1 up:standby-replay 3 up:standby 2024-04-26T18:29:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:57 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=k=up:active,2=g=up:active} 3 up:standby-replay 1 up:standby 2024-04-26T18:29:57.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 27830 10.83 MB/sec execute 82 sec latency 93.896 ms 2024-04-26T18:29:58.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:29:58 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:29:58.261+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:29:58.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28432 11.13 MB/sec execute 83 sec latency 140.138 ms 2024-04-26T18:29:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:29:59 smithi119 ceph-mon[32752]: pgmap v616: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-26T18:29:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:29:59 smithi167 ceph-mon[32751]: pgmap v616: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-26T18:29:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:29:59 smithi090 ceph-mon[29180]: pgmap v616: 97 pgs: 97 active+clean; 540 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-26T18:29:59.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 28853 11.24 MB/sec execute 84 sec latency 173.901 ms 2024-04-26T18:30:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-mon[32752]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-26T18:30:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-mon[32752]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T18:30:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-mon[32752]: daemon mds.i on smithi167 is in error state 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fec8f1a3700 time 2024-04-26T18:30:00.355123+0000 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec997cc3dd] 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.555 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: clone() 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.355+0000 7fec8f1a3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fec8f1a3700 time 2024-04-26T18:30:00.355123+0000 2024-04-26T18:30:00.556 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec997cc3dd] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.557 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: clone() 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: *** Caught signal (Aborted) ** 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: in thread 7fec8f1a3700 thread_name:ms_dispatch 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec98554d20] 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: gsignal() 2024-04-26T18:30:00.558 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: abort() 2024-04-26T18:30:00.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec997cc437] 2024-04-26T18:30:00.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.559 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.560 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 16: clone() 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.356+0000 7fec8f1a3700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: in thread 7fec8f1a3700 thread_name:ms_dispatch 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec98554d20] 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: gsignal() 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: abort() 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec997cc437] 2024-04-26T18:30:00.561 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.562 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 16: clone() 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: -3> 2024-04-26T18:30:00.355+0000 7fec8f1a3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fec8f1a3700 time 2024-04-26T18:30:00.355123+0000 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec997cc3dd] 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.563 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: clone() 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.564 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 0> 2024-04-26T18:30:00.356+0000 7fec8f1a3700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: in thread 7fec8f1a3700 thread_name:ms_dispatch 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec98554d20] 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: gsignal() 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: abort() 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec997cc437] 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.565 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 16: clone() 2024-04-26T18:30:00.566 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: -3> 2024-04-26T18:30:00.355+0000 7fec8f1a3700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fec8f1a3700 time 2024-04-26T18:30:00.355123+0000 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fec997cc3dd] 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.567 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: clone() 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.568 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 0> 2024-04-26T18:30:00.356+0000 7fec8f1a3700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: in thread 7fec8f1a3700 thread_name:ms_dispatch 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fec98554d20] 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2: gsignal() 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 3: abort() 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fec997cc437] 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fec997cc5a3] 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55b67c665822] 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55b67c5d5598] 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55b67c5d7c17] 2024-04-26T18:30:00.569 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55b67c4d0d0b] 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55b67c4d12ac] 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55b67c4b9fad] 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fec99a60e20] 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 13: (DispatchQueue::entry()+0x51f) [0x7fec99a5dfdf] 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fec99b30fc1] 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fec9854a1ca] 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 16: clone() 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k[72176]: 2024-04-26T18:30:00.570 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:00 smithi119 podman[72353]: 2024-04-26 18:30:00.41625919 +0000 UTC m=+0.019677103 container died 1e4bed44865d29600f853c38e0af92e9269818c821c5d6df8f7a80566390816f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T18:30:00.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:00 smithi167 ceph-mon[32751]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-26T18:30:00.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:00 smithi167 ceph-mon[32751]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T18:30:00.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:00 smithi167 ceph-mon[32751]: daemon mds.i on smithi167 is in error state 2024-04-26T18:30:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-mon[29180]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2024-04-26T18:30:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-mon[29180]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2024-04-26T18:30:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-mon[29180]: daemon mds.i on smithi167 is in error state 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f214bf43700 time 2024-04-26T18:30:00.413898+0000 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f215656c3dd] 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: clone() 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.414+0000 7f214bf43700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f214bf43700 time 2024-04-26T18:30:00.413898+0000 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.628 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f215656c3dd] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.629 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: clone() 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: *** Caught signal (Aborted) ** 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: in thread 7f214bf43700 thread_name:ms_dispatch 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21552f4d20] 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: gsignal() 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: abort() 2024-04-26T18:30:00.630 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f215656c437] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.631 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 16: clone() 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.415+0000 7f214bf43700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: in thread 7f214bf43700 thread_name:ms_dispatch 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21552f4d20] 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: gsignal() 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: abort() 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f215656c437] 2024-04-26T18:30:00.632 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 16: clone() 2024-04-26T18:30:00.633 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: -1> 2024-04-26T18:30:00.414+0000 7f214bf43700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f214bf43700 time 2024-04-26T18:30:00.413898+0000 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f215656c3dd] 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.634 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: clone() 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 0> 2024-04-26T18:30:00.415+0000 7f214bf43700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:00.635 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: in thread 7f214bf43700 thread_name:ms_dispatch 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21552f4d20] 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: gsignal() 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: abort() 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f215656c437] 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.636 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 16: clone() 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: -3> 2024-04-26T18:30:00.414+0000 7f214bf43700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f214bf43700 time 2024-04-26T18:30:00.413898+0000 2024-04-26T18:30:00.637 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f215656c3dd] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.638 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: clone() 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: -2> 2024-04-26T18:30:00.415+0000 7f214bf43700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: in thread 7f214bf43700 thread_name:ms_dispatch 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f21552f4d20] 2024-04-26T18:30:00.639 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2: gsignal() 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 3: abort() 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f215656c437] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f215656c5a3] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f49c85822] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f49bf5598] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f49bf7c17] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f49af0d0b] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f49af12ac] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f49ad9fad] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f2156800e20] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 13: (DispatchQueue::entry()+0x51f) [0x7f21567fdfdf] 2024-04-26T18:30:00.640 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f21568d0fc1] 2024-04-26T18:30:00.641 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f21552ea1ca] 2024-04-26T18:30:00.641 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 16: clone() 2024-04-26T18:30:00.641 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:00.641 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[97662]: 2024-04-26T18:30:00.641 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:00 smithi090 podman[97909]: 2024-04-26 18:30:00.473330031 +0000 UTC m=+0.019477624 container died bfecc0ee8d439bad838dd890e69e6b99df37fcbfe5322a83acf44030e368abe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T18:30:00.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 11.23 MB/sec execute 85 sec latency 550.655 ms 2024-04-26T18:30:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:01 smithi119 ceph-mon[32752]: pgmap v617: 97 pgs: 97 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-04-26T18:30:01.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:01 smithi090 ceph-mon[29180]: pgmap v617: 97 pgs: 97 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-04-26T18:30:01.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:01 smithi090 podman[97909]: 2024-04-26 18:30:01.345932854 +0000 UTC m=+0.892080492 container remove bfecc0ee8d439bad838dd890e69e6b99df37fcbfe5322a83acf44030e368abe9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T18:30:01.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:01 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:30:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:01 smithi167 ceph-mon[32751]: pgmap v617: 97 pgs: 97 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-04-26T18:30:01.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:01 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Failed with result 'exit-code'. 2024-04-26T18:30:01.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:01 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 13.520s CPU time. 2024-04-26T18:30:01.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 11.10 MB/sec execute 86 sec latency 1550.735 ms 2024-04-26T18:30:02.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.97 MB/sec execute 87 sec latency 2550.790 ms 2024-04-26T18:30:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:03 smithi119 ceph-mon[32752]: pgmap v618: 97 pgs: 97 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-04-26T18:30:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:03 smithi167 ceph-mon[32751]: pgmap v618: 97 pgs: 97 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-04-26T18:30:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:03 smithi090 ceph-mon[29180]: pgmap v618: 97 pgs: 97 active+clean; 536 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-04-26T18:30:03.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.84 MB/sec execute 88 sec latency 3550.888 ms 2024-04-26T18:30:04.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.72 MB/sec execute 89 sec latency 4551.014 ms 2024-04-26T18:30:05.235 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:04 smithi119 podman[72353]: 2024-04-26 18:30:04.81237273 +0000 UTC m=+4.415790624 container remove 1e4bed44865d29600f853c38e0af92e9269818c821c5d6df8f7a80566390816f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T18:30:05.235 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:04 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:30:05.235 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:05 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-26T18:30:05.235 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:05 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 12.433s CPU time. 2024-04-26T18:30:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:05 smithi119 ceph-mon[32752]: pgmap v619: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.2 MiB/s wr, 327 op/s 2024-04-26T18:30:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:05 smithi167 ceph-mon[32751]: pgmap v619: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.2 MiB/s wr, 327 op/s 2024-04-26T18:30:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:05 smithi090 ceph-mon[29180]: pgmap v619: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.2 MiB/s wr, 327 op/s 2024-04-26T18:30:05.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.60 MB/sec execute 90 sec latency 5551.091 ms 2024-04-26T18:30:06.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.49 MB/sec execute 91 sec latency 6551.189 ms 2024-04-26T18:30:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 -- 172.21.15.90:0/1082714308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 msgr2=0x7faafc10c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 --2- 172.21.15.90:0/1082714308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc10c9c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7faae4009900 tx=0x7faae402a510 comp rx=0 tx=0).stop 2024-04-26T18:30:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 -- 172.21.15.90:0/1082714308 shutdown_connections 2024-04-26T18:30:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 --2- 172.21.15.90:0/1082714308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc10cf00 0x7faafc10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 --2- 172.21.15.90:0/1082714308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc10c9c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 --2- 172.21.15.90:0/1082714308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc103bc0 0x7faafc103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 -- 172.21.15.90:0/1082714308 >> 172.21.15.90:0/1082714308 conn(0x7faafc0789b0 msgr2=0x7faafc078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 -- 172.21.15.90:0/1082714308 shutdown_connections 2024-04-26T18:30:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.403+0000 7fab02186640 1 -- 172.21.15.90:0/1082714308 wait complete. 2024-04-26T18:30:07.405 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.404+0000 7fab02186640 1 Processor -- start 2024-04-26T18:30:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.404+0000 7fab02186640 1 -- start start 2024-04-26T18:30:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7fab02186640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc103bc0 0x7faafc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.411 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7fab02186640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7fab02186640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc10cf00 0x7faafc19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7fab02186640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faafc112c00 con 0x7faafc1044e0 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7fab02186640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faafc112a80 con 0x7faafc103bc0 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7fab02186640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faafc112d80 con 0x7faafc10cf00 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc103bc0 0x7faafc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7faafaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7faafaffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc19b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46690/0 (socket says 172.21.15.90:46690) 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7faafbfff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc10cf00 0x7faafc19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.405+0000 7faafaffd640 1 -- 172.21.15.90:0/3835206349 learned_addr learned my addr 172.21.15.90:0/3835206349 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faafaffd640 1 -- 172.21.15.90:0/3835206349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc10cf00 msgr2=0x7faafc19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:07.412 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faafaffd640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc10cf00 0x7faafc19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faafaffd640 1 -- 172.21.15.90:0/3835206349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc103bc0 msgr2=0x7faafc19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faafaffd640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc103bc0 0x7faafc19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faafaffd640 1 -- 172.21.15.90:0/3835206349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc1a0370 con 0x7faafc1044e0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faafaffd640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc19b8c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7faae4005ab0 tx=0x7faae4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faaf8ff9640 1 -- 172.21.15.90:0/3835206349 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae403f070 con 0x7faafc1044e0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faaf8ff9640 1 -- 172.21.15.90:0/3835206349 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae4031d60 con 0x7faafc1044e0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc1a0600 con 0x7faafc1044e0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7faaf8ff9640 1 -- 172.21.15.90:0/3835206349 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4035030 con 0x7faafc1044e0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.406+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc1a7ef0 con 0x7faafc1044e0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.409+0000 7faaf8ff9640 1 -- 172.21.15.90:0/3835206349 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7faae4042070 con 0x7faafc1044e0 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.409+0000 7fab04f78740 1 Processor -- start 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.409+0000 7fab04f78740 1 -- start start 2024-04-26T18:30:07.413 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7fab04f78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995cf00 0x55d69995d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7fab04f78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995d820 0x55d699967ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7fab04f78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d69996a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7fab04f78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d6998c3460 con 0x55d69995d820 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7fab04f78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d69987d590 con 0x55d6999680e0 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7fab04f78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d6998bf420 con 0x55d69995cf00 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995d820 0x55d699967ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faaf8ff9640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faae80a3dc0 0x7faae80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995d820 0x55d699967ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46692/0 (socket says 172.21.15.90:46692) 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 -- 172.21.15.90:0/2648069635 learned_addr learned my addr 172.21.15.90:0/2648069635 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafaffd640 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d69996a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.414 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafbfff640 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995cf00 0x55d69995d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafbfff640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faae80a3dc0 0x7faae80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faaf8ff9640 1 -- 172.21.15.90:0/3835206349 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 11593+0+0 (secure 0 0 0) 0x7faae40f1cd0 con 0x7faafc1044e0 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 -- 172.21.15.90:0/2648069635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995cf00 msgr2=0x55d69995d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995cf00 0x55d69995d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 -- 172.21.15.90:0/2648069635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 msgr2=0x55d69996a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d69996a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafb7fe640 1 -- 172.21.15.90:0/2648069635 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d69988a0d0 con 0x55d69995d820 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.410+0000 7faafaffd640 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d69996a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.411+0000 7faafb7fe640 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995d820 0x55d699967ba0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faaec007f90 tx=0x7faaec00f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfd6c5fafb9f755 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.411+0000 7faafbfff640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faae80a3dc0 0x7faae80a6280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005fd0 tx=0x7faaf4005f60 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.411+0000 7faaf1ffb640 1 -- 172.21.15.90:0/2648069635 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec004d20 con 0x55d69995d820 2024-04-26T18:30:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.411+0000 7faaf1ffb640 1 -- 172.21.15.90:0/2648069635 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec010bb0 con 0x55d69995d820 2024-04-26T18:30:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.411+0000 7faaf1ffb640 1 -- 172.21.15.90:0/2648069635 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec0056b0 con 0x55d69995d820 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 -- 172.21.15.90:0/2648069635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995d820 msgr2=0x55d699967ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995d820 0x55d699967ba0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faaec007f90 tx=0x7faaec00f850 comp rx=0 tx=0).stop 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 -- 172.21.15.90:0/2648069635 shutdown_connections 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d69996a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995d820 0x55d699967ba0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 --2- 172.21.15.90:0/2648069635 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995cf00 0x55d69995d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 -- 172.21.15.90:0/2648069635 >> 172.21.15.90:0/2648069635 conn(0x55d699959410 msgr2=0x55d6999597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 -- 172.21.15.90:0/2648069635 shutdown_connections 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.412+0000 7fab04f78740 1 -- 172.21.15.90:0/2648069635 wait complete. 2024-04-26T18:30:07.421 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 -- start start 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 Processor -- start 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699964ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d699965530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d699965a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d69996ad60 con 0x55d69995cf00 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d69984ab60 con 0x55d6999680e0 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d69996aee0 con 0x55d69995d820 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7fab04f78740 10 client.0 _ll_register_callbacks cb 0x55d69986f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699964ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699964ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46706/0 (socket says 172.21.15.90:46706) 2024-04-26T18:30:07.422 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 -- 172.21.15.90:0/115022840 learned_addr learned my addr 172.21.15.90:0/115022840 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafbfff640 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d699965a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafaffd640 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d699965530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 -- 172.21.15.90:0/115022840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 msgr2=0x55d699965530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d699965530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 -- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 msgr2=0x55d699965a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d699965a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.413+0000 7faafb7fe640 1 -- 172.21.15.90:0/115022840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6995d5210 con 0x55d69995cf00 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7faafb7fe640 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699964ff0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7faaec010f90 tx=0x7faaec007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8cd18da3f2071a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7faadaffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7faadaffd640 1 -- 172.21.15.90:0/115022840 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec026020 con 0x55d69995cf00 2024-04-26T18:30:07.423 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7faadaffd640 1 -- 172.21.15.90:0/115022840 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 11593+0+0 (secure 0 0 0) 0x7faaec004d00 con 0x55d69995cf00 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec010bb0 con 0x55d69995cf00 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7faadaffd640 1 -- 172.21.15.90:0/115022840 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=241) ==== 24+0+0 (secure 0 0 0) 0x7faaec01d040 con 0x55d69995cf00 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 10 client.16722 fetch_fsmap learned FSMap version 241 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=241,monmap=4+}) -- 0x55d6996a1620 con 0x55d69995cf00 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7faadaffd640 1 -- 172.21.15.90:0/115022840 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec010bb0 con 0x55d69995cf00 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7faadaffd640 1 -- 172.21.15.90:0/115022840 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 241) ==== 5321+0+0 (secure 0 0 0) 0x7faaec02c940 con 0x55d69995cf00 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 10 client.16722 fetch_fsmap finished waiting for FSMap version 241 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 20 client.16722 populate_metadata read hostname 'smithi090' 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d699978510 0x55d69997a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 4 client.16722 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:07.424 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.414+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6998d1070 con 0x55d699978510 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.415+0000 7faafaffd640 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d699978510 0x55d69997a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.415+0000 7faadaffd640 10 client.16722 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.415+0000 7faafaffd640 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d699978510 0x55d69997a9b0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea42682bb2b4dac server_cookie=d911ea9aea3e90f4 in_seq=0 out_seq=0 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7faadaffd640 1 -- 172.21.15.90:0/115022840 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6998d1070 con 0x55d699978510 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7faadaffd640 10 client.16722 handle_command_reply: tid=0 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 client.16722 shutdown 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 msgr2=0x55d699964ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699964ff0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7faaec010f90 tx=0x7faaec007410 comp rx=0 tx=0).stop 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 shutdown_connections 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d699978510 0x55d69997a9b0 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d6999680e0 0x55d699965a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d699965530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7faadaffd640 0 client.16722 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 --2- 172.21.15.90:0/115022840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699964ff0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 >> 172.21.15.90:0/115022840 conn(0x55d699959410 msgr2=0x55d6999585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 shutdown_connections 2024-04-26T18:30:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 1 -- 172.21.15.90:0/115022840 wait complete. 2024-04-26T18:30:07.426 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.416+0000 7fab04f78740 20 client.16722 trim_cache size 0 max 16384 2024-04-26T18:30:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 Processor -- start 2024-04-26T18:30:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 -- start start 2024-04-26T18:30:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699973c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995d820 0x55d6999802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.436 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6999680e0 0x55d699980800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b34f0 con 0x55d69995cf00 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b1280 con 0x55d69995d820 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7fab04f78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b2e20 con 0x55d6999680e0 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699973c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699973c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46712/0 (socket says 172.21.15.90:46712) 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 -- 172.21.15.90:0/3241623423 learned_addr learned my addr 172.21.15.90:0/3241623423 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafaffd640 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995d820 0x55d6999802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafbfff640 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6999680e0 0x55d699980800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 -- 172.21.15.90:0/3241623423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6999680e0 msgr2=0x55d699980800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6999680e0 0x55d699980800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 -- 172.21.15.90:0/3241623423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995d820 msgr2=0x55d6999802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.437 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995d820 0x55d6999802c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafb7fe640 1 -- 172.21.15.90:0/3241623423 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d699964880 con 0x55d69995cf00 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.431+0000 7faafbfff640 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6999680e0 0x55d699980800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7faafb7fe640 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699973c20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7faaec000f80 tx=0x7faaec0075b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db915a9c5965b873 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7faaf17fa640 1 -- 172.21.15.90:0/3241623423 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec026070 con 0x55d69995cf00 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7faaf17fa640 1 -- 172.21.15.90:0/3241623423 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec010bb0 con 0x55d69995cf00 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7faaf17fa640 1 -- 172.21.15.90:0/3241623423 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaec022670 con 0x55d69995cf00 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 -- 172.21.15.90:0/3241623423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 msgr2=0x55d699973c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699973c20 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7faaec000f80 tx=0x7faaec0075b0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 -- 172.21.15.90:0/3241623423 shutdown_connections 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d6999680e0 0x55d699980800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995d820 0x55d6999802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.438 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 --2- 172.21.15.90:0/3241623423 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d69995cf00 0x55d699973c20 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 -- 172.21.15.90:0/3241623423 >> 172.21.15.90:0/3241623423 conn(0x55d699959410 msgr2=0x55d699960280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 -- 172.21.15.90:0/3241623423 shutdown_connections 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.432+0000 7fab04f78740 1 -- 172.21.15.90:0/3241623423 wait complete. 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 -- start start 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 Processor -- start 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995cf00 0x55d69996bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d69996c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6999680e0 0x55d69996c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b2020 con 0x55d6999680e0 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b1950 con 0x55d69995cf00 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1b2750 con 0x55d69995d820 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7fab04f78740 10 client.0 _ll_register_callbacks cb 0x7faaf400aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:07.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7faafbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6999680e0 0x55d69996c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7faafbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6999680e0 0x55d69996c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46720/0 (socket says 172.21.15.90:46720) 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7faafbfff640 1 -- 172.21.15.90:0/3515004788 learned_addr learned my addr 172.21.15.90:0/3515004788 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7faafb7fe640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995cf00 0x55d69996bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.433+0000 7faafaffd640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d69996c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafbfff640 1 -- 172.21.15.90:0/3515004788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 msgr2=0x55d69996c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafbfff640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d69996c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafbfff640 1 -- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995cf00 msgr2=0x55d69996bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafbfff640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995cf00 0x55d69996bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafbfff640 1 -- 172.21.15.90:0/3515004788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6999ddc30 con 0x55d6999680e0 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafb7fe640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995cf00 0x55d69996bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:07.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafaffd640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d69996c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faafbfff640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6999680e0 0x55d69996c6c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7faaf400ab60 tx=0x7faaf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137e7ae2db720444 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faadaffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faadaffd640 1 -- 172.21.15.90:0/3515004788 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf4011d70 con 0x55d6999680e0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faadaffd640 1 -- 172.21.15.90:0/3515004788 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 11593+0+0 (secure 0 0 0) 0x7faaf4005020 con 0x55d6999680e0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaec010bb0 con 0x55d6999680e0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faadaffd640 1 -- 172.21.15.90:0/3515004788 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=241) ==== 24+0+0 (secure 0 0 0) 0x7faaf4033050 con 0x55d6999680e0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 10 client.16728 fetch_fsmap learned FSMap version 241 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=241,monmap=4+}) -- 0x55d6999dee90 con 0x55d6999680e0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faadaffd640 1 -- 172.21.15.90:0/3515004788 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec010bb0 con 0x55d6999680e0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7faadaffd640 1 -- 172.21.15.90:0/3515004788 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 241) ==== 5321+0+0 (secure 0 0 0) 0x7faaf401e400 con 0x55d6999680e0 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 10 client.16728 fetch_fsmap finished waiting for FSMap version 241 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 20 client.16728 populate_metadata read hostname 'smithi090' 2024-04-26T18:30:07.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d6999df6f0 0x55d6999e1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 4 client.16728 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.434+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6998caee0 con 0x55d6999df6f0 2024-04-26T18:30:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.435+0000 7faafb7fe640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d6999df6f0 0x55d6999e1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.435+0000 7faafb7fe640 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d6999df6f0 0x55d6999e1b10 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8674eb2ab2250f3f server_cookie=e6384e4fdc5d0fc3 in_seq=0 out_seq=0 2024-04-26T18:30:07.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:07.435+0000 7faadaffd640 10 client.16728 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:07 smithi119 ceph-mon[32752]: pgmap v620: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-26T18:30:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:07 smithi167 ceph-mon[32751]: pgmap v620: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-26T18:30:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:07 smithi090 ceph-mon[29180]: pgmap v620: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-26T18:30:07.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.37 MB/sec execute 92 sec latency 7551.294 ms 2024-04-26T18:30:08.888 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.26 MB/sec execute 93 sec latency 8551.354 ms 2024-04-26T18:30:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:09 smithi119 ceph-mon[32752]: pgmap v621: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-26T18:30:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:09 smithi167 ceph-mon[32751]: pgmap v621: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-26T18:30:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:09 smithi090 ceph-mon[29180]: pgmap v621: 97 pgs: 97 active+clean; 547 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.1 MiB/s wr, 311 op/s 2024-04-26T18:30:09.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.15 MB/sec execute 94 sec latency 9551.433 ms 2024-04-26T18:30:10.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 10.05 MB/sec execute 95 sec latency 10551.515 ms 2024-04-26T18:30:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:11 smithi119 ceph-mon[32752]: pgmap v622: 97 pgs: 97 active+clean; 584 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.9 MiB/s wr, 444 op/s 2024-04-26T18:30:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:11 smithi167 ceph-mon[32751]: pgmap v622: 97 pgs: 97 active+clean; 584 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.9 MiB/s wr, 444 op/s 2024-04-26T18:30:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:11 smithi090 ceph-mon[29180]: pgmap v622: 97 pgs: 97 active+clean; 584 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.9 MiB/s wr, 444 op/s 2024-04-26T18:30:11.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:11 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:30:11.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:11 smithi090 systemd[1]: Stopped Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:11.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:11 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 13.520s CPU time. 2024-04-26T18:30:11.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:11 smithi090 systemd[1]: Starting Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:30:11.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:11 smithi090 podman[98062]: 2024-04-26 18:30:11.759635933 +0000 UTC m=+0.021889588 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:30:11.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:11 smithi090 podman[98062]: 2024-04-26 18:30:11.881743625 +0000 UTC m=+0.143997294 container create 5567dc654d3e70507b69040a911441c9422223824681d05c06e0bfed79976208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:30:11.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.94 MB/sec execute 96 sec latency 11551.553 ms 2024-04-26T18:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: daemon mds.g restarted 2024-04-26T18:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: osdmap e155: 12 total, 12 up, 12 in 2024-04-26T18:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:boot 2024-04-26T18:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:30:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:12 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=d=up:replay} 2 up:standby-replay 2 up:standby 2024-04-26T18:30:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: daemon mds.g restarted 2024-04-26T18:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: osdmap e155: 12 total, 12 up, 12 in 2024-04-26T18:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:boot 2024-04-26T18:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:12 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=d=up:replay} 2 up:standby-replay 2 up:standby 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: daemon mds.g restarted 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: osdmap e155: 12 total, 12 up, 12 in 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:boot 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=k=up:active} 3 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=d=up:replay} 2 up:standby-replay 2 up:standby 2024-04-26T18:30:12.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:12 smithi090 podman[98062]: 2024-04-26 18:30:12.158734793 +0000 UTC m=+0.420988429 container init 5567dc654d3e70507b69040a911441c9422223824681d05c06e0bfed79976208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T18:30:12.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:12 smithi090 podman[98062]: 2024-04-26 18:30:12.161252296 +0000 UTC m=+0.423505924 container start 5567dc654d3e70507b69040a911441c9422223824681d05c06e0bfed79976208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:30:12.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-mds[98082]: -- 172.21.15.90:0/4288228304 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fa05b5a40 con 0x564fa0589000 2024-04-26T18:30:12.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:12 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: starting mds.g at 2024-04-26T18:30:12.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:12 smithi090 bash[98062]: 5567dc654d3e70507b69040a911441c9422223824681d05c06e0bfed79976208 2024-04-26T18:30:12.627 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:30:12 smithi090 systemd[1]: Started Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:12.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.84 MB/sec execute 97 sec latency 12551.623 ms 2024-04-26T18:30:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:13 smithi119 ceph-mon[32752]: pgmap v623: 97 pgs: 97 active+clean; 584 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.9 MiB/s wr, 229 op/s 2024-04-26T18:30:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:13 smithi167 ceph-mon[32751]: pgmap v623: 97 pgs: 97 active+clean; 584 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.9 MiB/s wr, 229 op/s 2024-04-26T18:30:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:13 smithi090 ceph-mon[29180]: pgmap v623: 97 pgs: 97 active+clean; 584 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.9 MiB/s wr, 229 op/s 2024-04-26T18:30:13.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.74 MB/sec execute 98 sec latency 13551.700 ms 2024-04-26T18:30:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:resolve 2024-04-26T18:30:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:14 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=d=up:resolve} 2 up:standby-replay 2 up:standby 2024-04-26T18:30:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:resolve 2024-04-26T18:30:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:14 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=d=up:resolve} 2 up:standby-replay 2 up:standby 2024-04-26T18:30:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:resolve 2024-04-26T18:30:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:14 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=k=up:active,2=d=up:resolve} 2 up:standby-replay 2 up:standby 2024-04-26T18:30:14.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.64 MB/sec execute 99 sec latency 14551.798 ms 2024-04-26T18:30:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:15 smithi119 ceph-mon[32752]: pgmap v625: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.2 MiB/s wr, 236 op/s 2024-04-26T18:30:15.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:15 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Scheduled restart job, restart counter is at 6. 2024-04-26T18:30:15.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:15 smithi119 systemd[1]: Stopped Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:15.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:15 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Consumed 12.433s CPU time. 2024-04-26T18:30:15.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:15 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Start request repeated too quickly. 2024-04-26T18:30:15.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:15 smithi119 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.k.service: Failed with result 'exit-code'. 2024-04-26T18:30:15.554 INFO:journalctl@ceph.mds.k.smithi119.stdout:Apr 26 18:30:15 smithi119 systemd[1]: Failed to start Ceph mds.k for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:15 smithi167 ceph-mon[32751]: pgmap v625: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.2 MiB/s wr, 236 op/s 2024-04-26T18:30:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:15 smithi090 ceph-mon[29180]: pgmap v625: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.2 MiB/s wr, 236 op/s 2024-04-26T18:30:15.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.54 MB/sec execute 100 sec latency 15551.896 ms 2024-04-26T18:30:16.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.45 MB/sec execute 101 sec latency 16551.990 ms 2024-04-26T18:30:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:17 smithi119 ceph-mon[32752]: pgmap v626: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.2 MiB/s wr, 236 op/s 2024-04-26T18:30:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:17 smithi167 ceph-mon[32751]: pgmap v626: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.2 MiB/s wr, 236 op/s 2024-04-26T18:30:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:17 smithi090 ceph-mon[29180]: pgmap v626: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 139 KiB/s rd, 6.2 MiB/s wr, 236 op/s 2024-04-26T18:30:17.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.36 MB/sec execute 102 sec latency 17552.133 ms 2024-04-26T18:30:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:18 smithi119 ceph-mon[32752]: Replacing daemon mds.k as rank 1 with standby daemon mds.a 2024-04-26T18:30:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:18 smithi119 ceph-mon[32752]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-04-26T18:30:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:18 smithi119 ceph-mon[32752]: osdmap e156: 12 total, 12 up, 12 in 2024-04-26T18:30:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:18 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:replay,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:18 smithi167 ceph-mon[32751]: Replacing daemon mds.k as rank 1 with standby daemon mds.a 2024-04-26T18:30:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:18 smithi167 ceph-mon[32751]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-04-26T18:30:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:18 smithi167 ceph-mon[32751]: osdmap e156: 12 total, 12 up, 12 in 2024-04-26T18:30:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:18 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:replay,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:18 smithi090 ceph-mon[29180]: Replacing daemon mds.k as rank 1 with standby daemon mds.a 2024-04-26T18:30:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:18 smithi090 ceph-mon[29180]: MDS daemon mds.k is removed because it is dead or otherwise unavailable. 2024-04-26T18:30:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:18 smithi090 ceph-mon[29180]: osdmap e156: 12 total, 12 up, 12 in 2024-04-26T18:30:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:18 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:replay,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:18.889 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.26 MB/sec execute 103 sec latency 18552.233 ms 2024-04-26T18:30:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:19 smithi119 ceph-mon[32752]: pgmap v628: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 2.1 MiB/s wr, 95 op/s 2024-04-26T18:30:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:19 smithi167 ceph-mon[32751]: pgmap v628: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 2.1 MiB/s wr, 95 op/s 2024-04-26T18:30:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:19 smithi090 ceph-mon[29180]: pgmap v628: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 2.1 MiB/s wr, 95 op/s 2024-04-26T18:30:19.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.18 MB/sec execute 104 sec latency 19552.337 ms 2024-04-26T18:30:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:20 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:resolve 2024-04-26T18:30:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:20 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:20 smithi119 ceph-mon[32752]: pgmap v629: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-04-26T18:30:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:20 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:resolve 2024-04-26T18:30:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:20 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:20 smithi167 ceph-mon[32751]: pgmap v629: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-04-26T18:30:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:20 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:resolve 2024-04-26T18:30:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:20 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:20 smithi090 ceph-mon[29180]: pgmap v629: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 2.1 MiB/s wr, 99 op/s 2024-04-26T18:30:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:20.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29129 9.09 MB/sec execute 105 sec latency 20552.392 ms 2024-04-26T18:30:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.335+0000 7faadaffd640 1 -- 172.21.15.90:0/3515004788 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6998caee0 con 0x55d6999df6f0 2024-04-26T18:30:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.335+0000 7faadaffd640 10 client.16728 handle_command_reply: tid=0 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.335+0000 7fab04f78740 1 client.16728 shutdown 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.335+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6999680e0 msgr2=0x55d69996c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.335+0000 7fab04f78740 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6999680e0 0x55d69996c6c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7faaf400ab60 tx=0x7faaf40046c0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 shutdown_connections 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d6999df6f0 0x55d6999e1b10 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d6999680e0 0x55d69996c6c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d69995d820 0x55d69996c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 --2- 172.21.15.90:0/3515004788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d69995cf00 0x55d69996bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 >> 172.21.15.90:0/3515004788 conn(0x55d699959410 msgr2=0x55d699957d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7faadaffd640 0 client.16728 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:21.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 shutdown_connections 2024-04-26T18:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 1 -- 172.21.15.90:0/3515004788 wait complete. 2024-04-26T18:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.336+0000 7fab04f78740 20 client.16728 trim_cache size 0 max 16384 2024-04-26T18:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:30:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:30:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faae80a3dc0 msgr2=0x7faae80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faae80a3dc0 0x7faae80a6280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faaf4005fd0 tx=0x7faaf4005f60 comp rx=0 tx=0).stop 2024-04-26T18:30:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 msgr2=0x7faafc19b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc19b8c0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7faae4005ab0 tx=0x7faae4003d50 comp rx=0 tx=0).stop 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 shutdown_connections 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7faae80a3dc0 0x7faae80a6280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faafc10cf00 0x7faafc19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faafc1044e0 0x7faafc19b8c0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 --2- 172.21.15.90:0/3835206349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc103bc0 0x7faafc19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 >> 172.21.15.90:0/3835206349 conn(0x7faafc0789b0 msgr2=0x7faafc108f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.337+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 shutdown_connections 2024-04-26T18:30:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:21.338+0000 7fab02186640 1 -- 172.21.15.90:0/3835206349 wait complete. 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:reconnect 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:reconnect 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: pgmap v630: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: pgmap v631: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 5 op/s 2024-04-26T18:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:30:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:30:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:reconnect 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:reconnect 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: pgmap v630: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: pgmap v631: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 5 op/s 2024-04-26T18:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:reconnect 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:reconnect 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: pgmap v630: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.9 MiB/s wr, 92 op/s 2024-04-26T18:30:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: pgmap v631: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 5 op/s 2024-04-26T18:30:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:30:21.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29306 9.10 MB/sec execute 106 sec latency 21552.440 ms 2024-04-26T18:30:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:22 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:rejoin 2024-04-26T18:30:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:22 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:rejoin 2024-04-26T18:30:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:22 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:22 smithi119 ceph-mon[32752]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:30:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:22 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:rejoin 2024-04-26T18:30:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:22 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:rejoin 2024-04-26T18:30:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:22 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:22 smithi167 ceph-mon[32751]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:30:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:22 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:rejoin 2024-04-26T18:30:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:22 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:rejoin 2024-04-26T18:30:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:22 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:22 smithi090 ceph-mon[29180]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:30:22.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29446 9.08 MB/sec execute 107 sec latency 22552.472 ms 2024-04-26T18:30:23.067 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:30:22 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:30:22.649+0000 7f61aaa9b700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 10 2024-04-26T18:30:23.068 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:30:22 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:30:22.649+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 11 2024-04-26T18:30:23.068 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:30:22 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:30:22.650+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 13 2024-04-26T18:30:23.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:30:23.264+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:30:23.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:30:23.264+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:30:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:active 2024-04-26T18:30:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:clientreplay 2024-04-26T18:30:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:clientreplay,2=d=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T18:30:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:30:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:active 2024-04-26T18:30:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:30:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:23 smithi119 ceph-mon[32752]: pgmap v632: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 5 op/s 2024-04-26T18:30:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:active 2024-04-26T18:30:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:clientreplay 2024-04-26T18:30:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:clientreplay,2=d=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T18:30:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:30:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:active 2024-04-26T18:30:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:30:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:23 smithi167 ceph-mon[32751]: pgmap v632: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 5 op/s 2024-04-26T18:30:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1335338695,v1:172.21.15.90:6837/1335338695] up:active 2024-04-26T18:30:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:clientreplay 2024-04-26T18:30:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:clientreplay,2=d=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:30:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-26T18:30:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:30:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/720001018,v1:172.21.15.90:6841/720001018] up:active 2024-04-26T18:30:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:30:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:23 smithi090 ceph-mon[29180]: pgmap v632: 97 pgs: 97 active+clean; 571 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 5 op/s 2024-04-26T18:30:23.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29446 9.00 MB/sec execute 108 sec latency 23552.525 ms 2024-04-26T18:30:24.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29446 8.92 MB/sec execute 109 sec latency 24552.559 ms 2024-04-26T18:30:25.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 29642 8.91 MB/sec execute 110 sec latency 25221.237 ms 2024-04-26T18:30:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:25 smithi119 ceph-mon[32752]: pgmap v633: 97 pgs: 97 active+clean; 559 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-04-26T18:30:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:25 smithi167 ceph-mon[32751]: pgmap v633: 97 pgs: 97 active+clean; 559 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-04-26T18:30:26.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:25 smithi090 ceph-mon[29180]: pgmap v633: 97 pgs: 97 active+clean; 559 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 1.1 MiB/s wr, 186 op/s 2024-04-26T18:30:26.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30335 9.18 MB/sec execute 111 sec latency 52.494 ms 2024-04-26T18:30:27.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9572b2e700 time 2024-04-26T18:30:27.500951+0000 2024-04-26T18:30:27.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f957d1573dd] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.878 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.878 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.878 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.878 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: clone() 2024-04-26T18:30:27.878 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.501+0000 7f9572b2e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9572b2e700 time 2024-04-26T18:30:27.500951+0000 2024-04-26T18:30:27.878 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:27.878 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.879 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f957d1573dd] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.880 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: clone() 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: *** Caught signal (Aborted) ** 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: in thread 7f9572b2e700 thread_name:ms_dispatch 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f957bedfd20] 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: gsignal() 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: abort() 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f957d157437] 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.881 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.882 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 16: clone() 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.502+0000 7f9572b2e700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: in thread 7f9572b2e700 thread_name:ms_dispatch 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f957bedfd20] 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: gsignal() 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: abort() 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f957d157437] 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.883 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 16: clone() 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: -1> 2024-04-26T18:30:27.501+0000 7f9572b2e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9572b2e700 time 2024-04-26T18:30:27.500951+0000 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f957d1573dd] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: clone() 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 0> 2024-04-26T18:30:27.502+0000 7f9572b2e700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: in thread 7f9572b2e700 thread_name:ms_dispatch 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f957bedfd20] 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: gsignal() 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: abort() 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f957d157437] 2024-04-26T18:30:27.886 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 16: clone() 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:27.887 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: -1> 2024-04-26T18:30:27.501+0000 7f9572b2e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f9572b2e700 time 2024-04-26T18:30:27.500951+0000 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f957d1573dd] 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.888 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: clone() 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 0> 2024-04-26T18:30:27.502+0000 7f9572b2e700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: in thread 7f9572b2e700 thread_name:ms_dispatch 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f957bedfd20] 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2: gsignal() 2024-04-26T18:30:27.889 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 3: abort() 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f957d157437] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f957d1575a3] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555780a10822] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555780980598] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555780982c17] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55578087bd0b] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55578087c2ac] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555780864fad] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f957d3ebe20] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 13: (DispatchQueue::entry()+0x51f) [0x7f957d3e8fdf] 2024-04-26T18:30:27.890 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f957d4bbfc1] 2024-04-26T18:30:27.891 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f957bed51ca] 2024-04-26T18:30:27.891 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 16: clone() 2024-04-26T18:30:27.891 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:27.891 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[69390]: 2024-04-26T18:30:27.891 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:27 smithi090 podman[98263]: 2024-04-26 18:30:27.571047458 +0000 UTC m=+0.019755320 container died 3862163a35cff07d232b283d33e3c2ba4f0a969796dbe29d875e0309eacce57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T18:30:27.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 9.30 MB/sec execute 112 sec latency 415.727 ms 2024-04-26T18:30:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:27 smithi119 ceph-mon[32752]: pgmap v634: 97 pgs: 97 active+clean; 559 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 955 KiB/s wr, 153 op/s 2024-04-26T18:30:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:27 smithi167 ceph-mon[32751]: pgmap v634: 97 pgs: 97 active+clean; 559 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 955 KiB/s wr, 153 op/s 2024-04-26T18:30:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:27 smithi090 ceph-mon[29180]: pgmap v634: 97 pgs: 97 active+clean; 559 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 955 KiB/s wr, 153 op/s 2024-04-26T18:30:28.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 9.22 MB/sec execute 113 sec latency 1415.767 ms 2024-04-26T18:30:29.125 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:28 smithi090 podman[98263]: 2024-04-26 18:30:28.793876713 +0000 UTC m=+1.242584573 container remove 3862163a35cff07d232b283d33e3c2ba4f0a969796dbe29d875e0309eacce57f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:30:29.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:28 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:30:29.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:29 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-26T18:30:29.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:29 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 12.879s CPU time. 2024-04-26T18:30:29.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 9.14 MB/sec execute 114 sec latency 2415.844 ms 2024-04-26T18:30:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:29 smithi119 ceph-mon[32752]: pgmap v635: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-26T18:30:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:29 smithi167 ceph-mon[32751]: pgmap v635: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-26T18:30:30.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:29 smithi090 ceph-mon[29180]: pgmap v635: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 3.1 MiB/s wr, 390 op/s 2024-04-26T18:30:30.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 9.06 MB/sec execute 115 sec latency 3415.993 ms 2024-04-26T18:30:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:30 smithi119 ceph-mon[32752]: osdmap e157: 12 total, 12 up, 12 in 2024-04-26T18:30:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:30 smithi167 ceph-mon[32751]: osdmap e157: 12 total, 12 up, 12 in 2024-04-26T18:30:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:30 smithi090 ceph-mon[29180]: osdmap e157: 12 total, 12 up, 12 in 2024-04-26T18:30:31.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.98 MB/sec execute 116 sec latency 4416.098 ms 2024-04-26T18:30:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:31 smithi119 ceph-mon[32752]: pgmap v637: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-26T18:30:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:31 smithi167 ceph-mon[32751]: pgmap v637: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-26T18:30:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:31 smithi090 ceph-mon[29180]: pgmap v637: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-26T18:30:32.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.91 MB/sec execute 117 sec latency 5416.203 ms 2024-04-26T18:30:33.890 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.83 MB/sec execute 118 sec latency 6416.265 ms 2024-04-26T18:30:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:33 smithi119 ceph-mon[32752]: pgmap v638: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-26T18:30:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:33 smithi167 ceph-mon[32751]: pgmap v638: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-26T18:30:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:33 smithi090 ceph-mon[29180]: pgmap v638: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.8 MiB/s wr, 342 op/s 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 -- 172.21.15.90:0/1813104734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c1010a0 msgr2=0x7f906c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 --2- 172.21.15.90:0/1813104734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c1010a0 0x7f906c101480 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f905c009900 tx=0x7f905c02a410 comp rx=0 tx=0).stop 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 -- 172.21.15.90:0/1813104734 shutdown_connections 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 --2- 172.21.15.90:0/1813104734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f906c107c60 0x7f906c10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 --2- 172.21.15.90:0/1813104734 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f906c101a50 0x7f906c107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 --2- 172.21.15.90:0/1813104734 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c1010a0 0x7f906c101480 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 -- 172.21.15.90:0/1813104734 >> 172.21.15.90:0/1813104734 conn(0x7f906c077dc0 msgr2=0x7f906c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:34.439 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 -- 172.21.15.90:0/1813104734 shutdown_connections 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 -- 172.21.15.90:0/1813104734 wait complete. 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.437+0000 7f9071aa9640 1 Processor -- start 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 -- start start 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f906c1010a0 0x7f906c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c107c60 0x7f906c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f906c19d970 0x7f906c19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f906c112bc0 con 0x7f906c107c60 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f906c112a40 con 0x7f906c19d970 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f906c112d40 con 0x7f906c1010a0 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c107c60 0x7f906c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c107c60 0x7f906c199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46824/0 (socket says 172.21.15.90:46824) 2024-04-26T18:30:34.440 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 -- 172.21.15.90:0/849869823 learned_addr learned my addr 172.21.15.90:0/849869823 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f906bfff640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f906c19d970 0x7f906c19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f906b7fe640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f906c1010a0 0x7f906c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 -- 172.21.15.90:0/849869823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f906c1010a0 msgr2=0x7f906c199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f906c1010a0 0x7f906c199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 -- 172.21.15.90:0/849869823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f906c19d970 msgr2=0x7f906c19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f906c19d970 0x7f906c19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 -- 172.21.15.90:0/849869823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c19e460 con 0x7f906c107c60 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f906b7fe640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f906c1010a0 0x7f906c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f906bfff640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f906c19d970 0x7f906c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9063fff640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c107c60 0x7f906c199640 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9f0 tx=0x7f905400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f90697fa640 1 -- 172.21.15.90:0/849869823 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905400ccc0 con 0x7f906c107c60 2024-04-26T18:30:34.441 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.438+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c19e750 con 0x7f906c107c60 2024-04-26T18:30:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.439+0000 7f90697fa640 1 -- 172.21.15.90:0/849869823 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90540054d0 con 0x7f906c107c60 2024-04-26T18:30:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.439+0000 7f90697fa640 1 -- 172.21.15.90:0/849869823 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9054010430 con 0x7f906c107c60 2024-04-26T18:30:34.442 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.439+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c1a5f40 con 0x7f906c107c60 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90697fa640 1 -- 172.21.15.90:0/849869823 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f90540046c0 con 0x7f906c107c60 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 Processor -- start 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 -- start start 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b6f00 0x559beb4b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4c20e0 0x559beb4c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559beb41d460 con 0x559beb4c20e0 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559beb3d7590 con 0x559beb4b7820 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90747c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559beb419420 con 0x559beb4b6f00 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.441+0000 7f90697fa640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f90580a3dc0 0x7f90580a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4c20e0 0x559beb4c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f906bfff640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f90580a3dc0 0x7f90580a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4c20e0 0x559beb4c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46830/0 (socket says 172.21.15.90:46830) 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 -- 172.21.15.90:0/4133442189 learned_addr learned my addr 172.21.15.90:0/4133442189 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f906bfff640 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b6f00 0x559beb4b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f90697fa640 1 -- 172.21.15.90:0/849869823 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 11853+0+0 (secure 0 0 0) 0x7f90540d19e0 con 0x7f906c107c60 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 -- 172.21.15.90:0/4133442189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b6f00 msgr2=0x559beb4b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b6f00 0x559beb4b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 -- 172.21.15.90:0/4133442189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 msgr2=0x559beb4c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 -- 172.21.15.90:0/4133442189 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559beb3e40d0 con 0x559beb4c20e0 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f906bfff640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f90580a3dc0 0x7f90580a6280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9064005c40 tx=0x7f9064009f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:34.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9063fff640 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4c20e0 0x559beb4c44d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9054098960 tx=0x7f9054004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abec6989ac986a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9061ffb640 1 -- 172.21.15.90:0/4133442189 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905409d380 con 0x559beb4c20e0 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9061ffb640 1 -- 172.21.15.90:0/4133442189 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905409cc80 con 0x559beb4c20e0 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.442+0000 7f9061ffb640 1 -- 172.21.15.90:0/4133442189 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90540992f0 con 0x559beb4c20e0 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 -- 172.21.15.90:0/4133442189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4c20e0 msgr2=0x559beb4c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4c20e0 0x559beb4c44d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9054098960 tx=0x7f9054004a60 comp rx=0 tx=0).stop 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 -- 172.21.15.90:0/4133442189 shutdown_connections 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4c20e0 0x559beb4c44d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 --2- 172.21.15.90:0/4133442189 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b6f00 0x559beb4b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 -- 172.21.15.90:0/4133442189 >> 172.21.15.90:0/4133442189 conn(0x559beb4b3410 msgr2=0x559beb4b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 -- 172.21.15.90:0/4133442189 shutdown_connections 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 -- 172.21.15.90:0/4133442189 wait complete. 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.443+0000 7f90747c0740 1 -- start start 2024-04-26T18:30:34.451 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 1 Processor -- start 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b7820 0x559beb4bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4c20e0 0x559beb4bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559beb4c4d10 con 0x559beb4b6f00 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559beb3a4b60 con 0x559beb4c20e0 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559beb4c4e90 con 0x559beb4b7820 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f90747c0740 10 client.0 _ll_register_callbacks cb 0x559beb3c92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46844/0 (socket says 172.21.15.90:46844) 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 -- 172.21.15.90:0/3660710531 learned_addr learned my addr 172.21.15.90:0/3660710531 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f9063fff640 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b7820 0x559beb4bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906bfff640 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4c20e0 0x559beb4bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.452 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 -- 172.21.15.90:0/3660710531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b7820 msgr2=0x559beb4bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b7820 0x559beb4bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 -- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4c20e0 msgr2=0x559beb4bfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4c20e0 0x559beb4bfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.444+0000 7f906b7fe640 1 -- 172.21.15.90:0/3660710531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559beb12f210 con 0x559beb4b6f00 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f906b7fe640 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bf050 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f905c009900 tx=0x7f905c032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc13589ae635de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f904affd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f904affd640 1 -- 172.21.15.90:0/3660710531 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c040070 con 0x559beb4b6f00 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f904affd640 1 -- 172.21.15.90:0/3660710531 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(157..157 src has 1..157) ==== 11853+0+0 (secure 0 0 0) 0x7f905c004240 con 0x559beb4b6f00 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f905409cc80 con 0x559beb4b6f00 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f904affd640 1 -- 172.21.15.90:0/3660710531 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7f905c035040 con 0x559beb4b6f00 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 10 client.16746 fetch_fsmap learned FSMap version 250 2024-04-26T18:30:34.453 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=4+}) -- 0x559beb1fb620 con 0x559beb4b6f00 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f904affd640 1 -- 172.21.15.90:0/3660710531 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905409cc80 con 0x559beb4b6f00 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f904affd640 1 -- 172.21.15.90:0/3660710531 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 250) ==== 4716+0+0 (secure 0 0 0) 0x7f905c0047c0 con 0x559beb4b6f00 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 10 client.16746 fetch_fsmap finished waiting for FSMap version 250 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 20 client.16746 populate_metadata read hostname 'smithi090' 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb4d4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 4 client.16746 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559beb42b070 con 0x559beb4d25f0 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.445+0000 7f9063fff640 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb4d4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.446+0000 7f9063fff640 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb4d4a90 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8ecc4d57d2de746 server_cookie=2d2e55f3c5612487 in_seq=0 out_seq=0 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.446+0000 7f904affd640 10 client.16746 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f904affd640 1 -- 172.21.15.90:0/3660710531 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559beb42b070 con 0x559beb4d25f0 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f904affd640 10 client.16746 handle_command_reply: tid=0 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 client.16746 shutdown 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 msgr2=0x559beb4bf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bf050 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f905c009900 tx=0x7f905c032f20 comp rx=0 tx=0).stop 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 shutdown_connections 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb4d4a90 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4c20e0 0x559beb4bfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4b7820 0x559beb4bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f904affd640 0 client.16746 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 --2- 172.21.15.90:0/3660710531 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bf050 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 >> 172.21.15.90:0/3660710531 conn(0x559beb4b3410 msgr2=0x559beb4b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 shutdown_connections 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 1 -- 172.21.15.90:0/3660710531 wait complete. 2024-04-26T18:30:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.447+0000 7f90747c0740 20 client.16746 trim_cache size 0 max 16384 2024-04-26T18:30:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 Processor -- start 2024-04-26T18:30:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 -- start start 2024-04-26T18:30:34.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 0x559beb4c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1b0ed0 con 0x559beb4b6f00 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1aec60 con 0x559beb4b7820 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90747c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1b0800 con 0x559beb4c20e0 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4cdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46852/0 (socket says 172.21.15.90:46852) 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 -- 172.21.15.90:0/1943314527 learned_addr learned my addr 172.21.15.90:0/1943314527 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 -- 172.21.15.90:0/1943314527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 msgr2=0x559beb4c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 --2- 172.21.15.90:0/1943314527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 0x559beb4c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 -- 172.21.15.90:0/1943314527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 msgr2=0x559beb4b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 --2- 172.21.15.90:0/1943314527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 -- 172.21.15.90:0/1943314527 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559beb4ad9c0 con 0x559beb4b6f00 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f906b7fe640 1 --2- 172.21.15.90:0/1943314527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4cdbd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f905c0096c0 tx=0x7f905c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8149f7d2b934aa82 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90617fa640 1 -- 172.21.15.90:0/1943314527 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c056070 con 0x559beb4b6f00 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90617fa640 1 -- 172.21.15.90:0/1943314527 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c0318f0 con 0x559beb4b6f00 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.462+0000 7f90617fa640 1 -- 172.21.15.90:0/1943314527 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c040040 con 0x559beb4b6f00 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 -- 172.21.15.90:0/1943314527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 msgr2=0x559beb4cdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 --2- 172.21.15.90:0/1943314527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4cdbd0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f905c0096c0 tx=0x7f905c02a410 comp rx=0 tx=0).stop 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 -- 172.21.15.90:0/1943314527 shutdown_connections 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 --2- 172.21.15.90:0/1943314527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 0x559beb4c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 --2- 172.21.15.90:0/1943314527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 --2- 172.21.15.90:0/1943314527 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4cdbd0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 -- 172.21.15.90:0/1943314527 >> 172.21.15.90:0/1943314527 conn(0x559beb4b3410 msgr2=0x559beb4c0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:34.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 -- 172.21.15.90:0/1943314527 shutdown_connections 2024-04-26T18:30:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 -- 172.21.15.90:0/1943314527 wait complete. 2024-04-26T18:30:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.463+0000 7f90747c0740 1 -- start start 2024-04-26T18:30:34.466 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 1 Processor -- start 2024-04-26T18:30:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4bc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 0x559beb4bc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1afa00 con 0x559beb4b6f00 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1af330 con 0x559beb4b7820 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f906c1b00d0 con 0x559beb4c20e0 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f90747c0740 10 client.0 _ll_register_callbacks cb 0x7f905c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46858/0 (socket says 172.21.15.90:46858) 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 -- 172.21.15.90:0/4222046841 learned_addr learned my addr 172.21.15.90:0/4222046841 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f9063fff640 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906bfff640 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 0x559beb4bc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 -- 172.21.15.90:0/4222046841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 msgr2=0x559beb4bc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 0x559beb4bc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 -- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 msgr2=0x559beb4bc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4bc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 -- 172.21.15.90:0/4222046841 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559beb537c80 con 0x559beb4b6f00 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f906b7fe640 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bbe50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f905c004bc0 tx=0x7f905c004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5228f5d835a5f34f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.464+0000 7f904affd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f904affd640 1 -- 172.21.15.90:0/4222046841 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c040070 con 0x559beb4b6f00 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f904affd640 1 -- 172.21.15.90:0/4222046841 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(157..157 src has 1..157) ==== 11853+0+0 (secure 0 0 0) 0x7f905c031d10 con 0x559beb4b6f00 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f905c0318f0 con 0x559beb4b6f00 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f904affd640 1 -- 172.21.15.90:0/4222046841 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=250) ==== 24+0+0 (secure 0 0 0) 0x7f905c056040 con 0x559beb4b6f00 2024-04-26T18:30:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 10 client.16752 fetch_fsmap learned FSMap version 250 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=250,monmap=4+}) -- 0x559beb538f70 con 0x559beb4b6f00 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f904affd640 1 -- 172.21.15.90:0/4222046841 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f905c0318f0 con 0x559beb4b6f00 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f904affd640 1 -- 172.21.15.90:0/4222046841 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 250) ==== 4716+0+0 (secure 0 0 0) 0x7f905c005110 con 0x559beb4b6f00 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 10 client.16752 fetch_fsmap finished waiting for FSMap version 250 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 20 client.16752 populate_metadata read hostname 'smithi090' 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb53b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 4 client.16752 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559beb424ee0 con 0x559beb4d25f0 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.465+0000 7f9063fff640 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb53b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.466+0000 7f9063fff640 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb53b2d0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d82d252597152dc server_cookie=f1f237f262a90de9 in_seq=0 out_seq=0 2024-04-26T18:30:34.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:34.466+0000 7f904affd640 10 client.16752 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:34.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.76 MB/sec execute 119 sec latency 7416.352 ms 2024-04-26T18:30:35.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.68 MB/sec execute 120 sec latency 8416.445 ms 2024-04-26T18:30:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:35 smithi119 ceph-mon[32752]: pgmap v639: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-04-26T18:30:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:35 smithi167 ceph-mon[32751]: pgmap v639: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-04-26T18:30:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:35 smithi090 ceph-mon[29180]: pgmap v639: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-04-26T18:30:36.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.61 MB/sec execute 121 sec latency 9416.493 ms 2024-04-26T18:30:37.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.54 MB/sec execute 122 sec latency 10416.561 ms 2024-04-26T18:30:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:37 smithi119 ceph-mon[32752]: pgmap v640: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-04-26T18:30:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:37 smithi119 ceph-mon[32752]: osdmap e158: 12 total, 12 up, 12 in 2024-04-26T18:30:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:37 smithi167 ceph-mon[32751]: pgmap v640: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-04-26T18:30:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:37 smithi167 ceph-mon[32751]: osdmap e158: 12 total, 12 up, 12 in 2024-04-26T18:30:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:37 smithi090 ceph-mon[29180]: pgmap v640: 97 pgs: 97 active+clean; 558 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.0 MiB/s wr, 222 op/s 2024-04-26T18:30:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:37 smithi090 ceph-mon[29180]: osdmap e158: 12 total, 12 up, 12 in 2024-04-26T18:30:38.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.47 MB/sec execute 123 sec latency 11416.668 ms 2024-04-26T18:30:39.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:39 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:30:39.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:39 smithi090 systemd[1]: Stopped Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:39.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:39 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 12.879s CPU time. 2024-04-26T18:30:39.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:39 smithi090 systemd[1]: Starting Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:30:39.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:39 smithi090 podman[98404]: 2024-04-26 18:30:39.263819352 +0000 UTC m=+0.019045261 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:30:39.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.40 MB/sec execute 124 sec latency 12416.726 ms 2024-04-26T18:30:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:39 smithi090 ceph-mon[29180]: pgmap v642: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 469 KiB/s wr, 78 op/s 2024-04-26T18:30:40.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:39 smithi090 podman[98404]: 2024-04-26 18:30:39.709406046 +0000 UTC m=+0.464632007 container create 49f8fdc018cf3a22971f5d9f743446ceb0197a5700c4c1cae989890d63d4a588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T18:30:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:39 smithi119 ceph-mon[32752]: pgmap v642: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 469 KiB/s wr, 78 op/s 2024-04-26T18:30:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:39 smithi167 ceph-mon[32751]: pgmap v642: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 469 KiB/s wr, 78 op/s 2024-04-26T18:30:40.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:40 smithi090 podman[98404]: 2024-04-26 18:30:40.710251888 +0000 UTC m=+1.465477780 container init 49f8fdc018cf3a22971f5d9f743446ceb0197a5700c4c1cae989890d63d4a588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:30:40.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:40 smithi090 podman[98404]: 2024-04-26 18:30:40.713903766 +0000 UTC m=+1.469129662 container start 49f8fdc018cf3a22971f5d9f743446ceb0197a5700c4c1cae989890d63d4a588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:30:40.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mds[98422]: -- 172.21.15.90:0/3041056270 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556eddabba40 con 0x556edda8f000 2024-04-26T18:30:40.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: starting mds.d at 2024-04-26T18:30:40.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:40 smithi090 bash[98404]: 49f8fdc018cf3a22971f5d9f743446ceb0197a5700c4c1cae989890d63d4a588 2024-04-26T18:30:40.877 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:30:40 smithi090 systemd[1]: Started Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:40.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.34 MB/sec execute 125 sec latency 13416.779 ms 2024-04-26T18:30:41.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: daemon mds.d restarted 2024-04-26T18:30:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: osdmap e159: 12 total, 12 up, 12 in 2024-04-26T18:30:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:boot 2024-04-26T18:30:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: Standby daemon mds.j assigned to filesystem cephfs as rank 2 2024-04-26T18:30:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=a=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:30:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:30:41.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:40 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: daemon mds.d restarted 2024-04-26T18:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: osdmap e159: 12 total, 12 up, 12 in 2024-04-26T18:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:boot 2024-04-26T18:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: Standby daemon mds.j assigned to filesystem cephfs as rank 2 2024-04-26T18:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=a=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:40 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: daemon mds.d restarted 2024-04-26T18:30:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: osdmap e159: 12 total, 12 up, 12 in 2024-04-26T18:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:boot 2024-04-26T18:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: Standby daemon mds.j assigned to filesystem cephfs as rank 2 2024-04-26T18:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=a=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:40 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:41.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.27 MB/sec execute 126 sec latency 14416.825 ms 2024-04-26T18:30:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:41 smithi119 ceph-mon[32752]: pgmap v644: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 524 KiB/s wr, 87 op/s 2024-04-26T18:30:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:41 smithi167 ceph-mon[32751]: pgmap v644: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 524 KiB/s wr, 87 op/s 2024-04-26T18:30:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:41 smithi090 ceph-mon[29180]: pgmap v644: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 524 KiB/s wr, 87 op/s 2024-04-26T18:30:42.891 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.21 MB/sec execute 127 sec latency 15417.173 ms 2024-04-26T18:30:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:42 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:resolve 2024-04-26T18:30:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:42 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:42 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:resolve 2024-04-26T18:30:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:42 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:42 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:resolve 2024-04-26T18:30:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:42 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:43.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 30906 8.14 MB/sec execute 128 sec latency 16417.304 ms 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.018+0000 7f904affd640 1 -- 172.21.15.90:0/4222046841 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559beb424ee0 con 0x559beb4d25f0 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.018+0000 7f904affd640 10 client.16752 handle_command_reply: tid=0 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.018+0000 7f90747c0740 1 client.16752 shutdown 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 msgr2=0x559beb4bbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bbe50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f905c004bc0 tx=0x7f905c004970 comp rx=0 tx=0).stop 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 shutdown_connections 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559beb4d25f0 0x559beb53b2d0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559beb4c20e0 0x559beb4bc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559beb4b7820 0x559beb4bc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.020 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 --2- 172.21.15.90:0/4222046841 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559beb4b6f00 0x559beb4bbe50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 >> 172.21.15.90:0/4222046841 conn(0x559beb4b3410 msgr2=0x559beb4c1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f904affd640 0 client.16752 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 shutdown_connections 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 1 -- 172.21.15.90:0/4222046841 wait complete. 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.019+0000 7f90747c0740 20 client.16752 trim_cache size 0 max 16384 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:30:44.021 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:30:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f90580a3dc0 msgr2=0x7f90580a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f90580a3dc0 0x7f90580a6280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9064005c40 tx=0x7f9064009f90 comp rx=0 tx=0).stop 2024-04-26T18:30:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c107c60 msgr2=0x7f906c199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c107c60 0x7f906c199640 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f905400e9f0 tx=0x7f905400eeb0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 shutdown_connections 2024-04-26T18:30:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f90580a3dc0 0x7f90580a6280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f906c19d970 0x7f906c19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f906c107c60 0x7f906c199640 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 --2- 172.21.15.90:0/849869823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f906c1010a0 0x7f906c199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 >> 172.21.15.90:0/849869823 conn(0x7f906c077dc0 msgr2=0x7f906c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 shutdown_connections 2024-04-26T18:30:44.023 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:44.020+0000 7f9071aa9640 1 -- 172.21.15.90:0/849869823 wait complete. 2024-04-26T18:30:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:43 smithi119 ceph-mon[32752]: pgmap v645: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 426 KiB/s wr, 73 op/s 2024-04-26T18:30:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:reconnect 2024-04-26T18:30:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:43 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:reconnect} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:43 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:30:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:43 smithi167 ceph-mon[32751]: pgmap v645: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 426 KiB/s wr, 73 op/s 2024-04-26T18:30:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:reconnect 2024-04-26T18:30:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:43 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:reconnect} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:43 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:30:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:43 smithi090 ceph-mon[29180]: pgmap v645: 97 pgs: 97 active+clean; 561 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 426 KiB/s wr, 73 op/s 2024-04-26T18:30:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:reconnect 2024-04-26T18:30:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:43 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:reconnect} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:43 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:30:44.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 31516 8.42 MB/sec execute 129 sec latency 17417.367 ms 2024-04-26T18:30:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:rejoin 2024-04-26T18:30:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:44 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:rejoin} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:44 smithi119 ceph-mon[32752]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:30:45.317 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:30:44 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:30:44.987+0000 7f61a8a97700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 5 2024-04-26T18:30:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:rejoin 2024-04-26T18:30:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:44 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:rejoin} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:44 smithi167 ceph-mon[32751]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:30:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:rejoin 2024-04-26T18:30:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:44 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=a=up:active,2=j=up:rejoin} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:44 smithi090 ceph-mon[29180]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:30:45.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32019 8.61 MB/sec execute 130 sec latency 17618.429 ms 2024-04-26T18:30:46.224 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:45 smithi090 ceph-mon[29180]: pgmap v646: 97 pgs: 97 active+clean; 565 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 886 KiB/s wr, 93 op/s 2024-04-26T18:30:46.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:45 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:30:46.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:active 2024-04-26T18:30:46.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=j=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:46.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=j=up:active} 3 up:standby-replay 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcead85f700 time 2024-04-26T18:30:46.189808+0000 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fceb7e883dd] 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.226 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: clone() 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: *** Caught signal (Aborted) ** 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: in thread 7fcead85f700 thread_name:ms_dispatch 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.190+0000 7fcead85f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcead85f700 time 2024-04-26T18:30:46.189808+0000 2024-04-26T18:30:46.227 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fceb7e883dd] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.228 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: clone() 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fceb6c10d20] 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: gsignal() 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: abort() 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fceb7e88437] 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.229 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 16: clone() 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.192+0000 7fcead85f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: in thread 7fcead85f700 thread_name:ms_dispatch 2024-04-26T18:30:46.230 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fceb6c10d20] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: gsignal() 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: abort() 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fceb7e88437] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.231 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.232 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.232 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.232 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 16: clone() 2024-04-26T18:30:46.232 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: -4> 2024-04-26T18:30:46.190+0000 7fcead85f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcead85f700 time 2024-04-26T18:30:46.189808+0000 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fceb7e883dd] 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.233 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: clone() 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 0> 2024-04-26T18:30:46.192+0000 7fcead85f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: in thread 7fcead85f700 thread_name:ms_dispatch 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.234 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fceb6c10d20] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: gsignal() 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: abort() 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fceb7e88437] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.235 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 16: clone() 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: -4> 2024-04-26T18:30:46.190+0000 7fcead85f700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fcead85f700 time 2024-04-26T18:30:46.189808+0000 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fceb7e883dd] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.236 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: clone() 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 0> 2024-04-26T18:30:46.192+0000 7fcead85f700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: in thread 7fcead85f700 thread_name:ms_dispatch 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.237 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fceb6c10d20] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2: gsignal() 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 3: abort() 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fceb7e88437] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fceb7e885a3] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563988920822] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563988890598] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563988892c17] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x56398878bd0b] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x56398878c2ac] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563988774fad] 2024-04-26T18:30:46.238 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fceb811ce20] 2024-04-26T18:30:46.239 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 13: (DispatchQueue::entry()+0x51f) [0x7fceb8119fdf] 2024-04-26T18:30:46.239 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fceb81ecfc1] 2024-04-26T18:30:46.239 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fceb6c061ca] 2024-04-26T18:30:46.239 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 16: clone() 2024-04-26T18:30:46.239 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:46.239 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[97858]: 2024-04-26T18:30:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:45 smithi119 ceph-mon[32752]: pgmap v646: 97 pgs: 97 active+clean; 565 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 886 KiB/s wr, 93 op/s 2024-04-26T18:30:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:45 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:30:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:active 2024-04-26T18:30:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=j=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=j=up:active} 3 up:standby-replay 2024-04-26T18:30:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:45 smithi167 ceph-mon[32751]: pgmap v646: 97 pgs: 97 active+clean; 565 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 886 KiB/s wr, 93 op/s 2024-04-26T18:30:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:45 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:30:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/2911456063,v1:172.21.15.90:6835/2911456063] up:active 2024-04-26T18:30:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=j=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:30:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=j=up:active} 3 up:standby-replay 2024-04-26T18:30:46.501 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 podman[98473]: 2024-04-26 18:30:46.265616512 +0000 UTC m=+0.026230404 container died 3b4faade63458f04b9af805c5bd829b8fd1a72060edaea80ce4e995d9e64ad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:30:46.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.62 MB/sec execute 131 sec latency 722.025 ms 2024-04-26T18:30:47.117 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 podman[98473]: 2024-04-26 18:30:46.866857498 +0000 UTC m=+0.627471398 container remove 3b4faade63458f04b9af805c5bd829b8fd1a72060edaea80ce4e995d9e64ad19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T18:30:47.118 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:46 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:30:47.625 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:47 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-04-26T18:30:47.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:47 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 15.220s CPU time. 2024-04-26T18:30:47.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.55 MB/sec execute 132 sec latency 1722.058 ms 2024-04-26T18:30:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:47 smithi119 ceph-mon[32752]: pgmap v647: 97 pgs: 97 active+clean; 565 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 769 KiB/s wr, 81 op/s 2024-04-26T18:30:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:47 smithi167 ceph-mon[32751]: pgmap v647: 97 pgs: 97 active+clean; 565 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 769 KiB/s wr, 81 op/s 2024-04-26T18:30:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:47 smithi090 ceph-mon[29180]: pgmap v647: 97 pgs: 97 active+clean; 565 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 769 KiB/s wr, 81 op/s 2024-04-26T18:30:48.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.49 MB/sec execute 133 sec latency 2722.090 ms 2024-04-26T18:30:49.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.42 MB/sec execute 134 sec latency 3722.125 ms 2024-04-26T18:30:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:49 smithi119 ceph-mon[32752]: pgmap v648: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-04-26T18:30:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:49 smithi167 ceph-mon[32751]: pgmap v648: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-04-26T18:30:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:49 smithi090 ceph-mon[29180]: pgmap v648: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-04-26T18:30:50.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.36 MB/sec execute 135 sec latency 4722.218 ms 2024-04-26T18:30:51.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.30 MB/sec execute 136 sec latency 5722.330 ms 2024-04-26T18:30:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:51 smithi119 ceph-mon[32752]: pgmap v649: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-04-26T18:30:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:51 smithi167 ceph-mon[32751]: pgmap v649: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-04-26T18:30:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:51 smithi090 ceph-mon[29180]: pgmap v649: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.0 MiB/s wr, 175 op/s 2024-04-26T18:30:52.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.24 MB/sec execute 137 sec latency 6722.423 ms 2024-04-26T18:30:53.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.18 MB/sec execute 138 sec latency 7722.493 ms 2024-04-26T18:30:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:53 smithi119 ceph-mon[32752]: pgmap v650: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-04-26T18:30:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:53 smithi167 ceph-mon[32751]: pgmap v650: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-04-26T18:30:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:53 smithi090 ceph-mon[29180]: pgmap v650: 97 pgs: 97 active+clean; 572 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.7 MiB/s wr, 147 op/s 2024-04-26T18:30:54.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.12 MB/sec execute 139 sec latency 8722.605 ms 2024-04-26T18:30:55.892 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.06 MB/sec execute 140 sec latency 9722.715 ms 2024-04-26T18:30:56.231 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:55 smithi090 ceph-mon[29180]: pgmap v651: 97 pgs: 97 active+clean; 576 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fead0e5a700 time 2024-04-26T18:30:56.198256+0000 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feadb4833dd] 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.232 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: clone() 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.198+0000 7fead0e5a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fead0e5a700 time 2024-04-26T18:30:56.198256+0000 2024-04-26T18:30:56.233 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:56.234 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.235 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.235 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feadb4833dd] 2024-04-26T18:30:56.235 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.235 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: clone() 2024-04-26T18:30:56.236 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: *** Caught signal (Aborted) ** 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: in thread 7fead0e5a700 thread_name:ms_dispatch 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feada20bd20] 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: gsignal() 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: abort() 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feadb483437] 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.237 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 16: clone() 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.200+0000 7fead0e5a700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:56.238 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: in thread 7fead0e5a700 thread_name:ms_dispatch 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feada20bd20] 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: gsignal() 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: abort() 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feadb483437] 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.239 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 16: clone() 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: -1> 2024-04-26T18:30:56.198+0000 7fead0e5a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fead0e5a700 time 2024-04-26T18:30:56.198256+0000 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:56.240 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feadb4833dd] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.241 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: clone() 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 0> 2024-04-26T18:30:56.200+0000 7fead0e5a700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: in thread 7fead0e5a700 thread_name:ms_dispatch 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feada20bd20] 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: gsignal() 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: abort() 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feadb483437] 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.242 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 16: clone() 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: -1> 2024-04-26T18:30:56.198+0000 7fead0e5a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fead0e5a700 time 2024-04-26T18:30:56.198256+0000 2024-04-26T18:30:56.243 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7feadb4833dd] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.244 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: clone() 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 0> 2024-04-26T18:30:56.200+0000 7fead0e5a700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: in thread 7fead0e5a700 thread_name:ms_dispatch 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7feada20bd20] 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2: gsignal() 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 3: abort() 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7feadb483437] 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7feadb4835a3] 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559918b32822] 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559918aa2598] 2024-04-26T18:30:56.245 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559918aa4c17] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55991899dd0b] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55991899e2ac] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559918986fad] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7feadb717e20] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 13: (DispatchQueue::entry()+0x51f) [0x7feadb714fdf] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7feadb7e7fc1] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7feada2011ca] 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 16: clone() 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:30:56.246 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[68927]: 2024-04-26T18:30:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:55 smithi119 ceph-mon[32752]: pgmap v651: 97 pgs: 97 active+clean; 576 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-04-26T18:30:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:55 smithi167 ceph-mon[32751]: pgmap v651: 97 pgs: 97 active+clean; 576 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 158 op/s 2024-04-26T18:30:56.625 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:56 smithi090 podman[98518]: 2024-04-26 18:30:56.271501506 +0000 UTC m=+0.019021432 container died e984b04ed9cbbf71e4bd2d29860e1a5c32f78d8b062a5ac47dd9a6c5e6e63244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T18:30:56.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 8.00 MB/sec execute 141 sec latency 10722.848 ms 2024-04-26T18:30:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 -- 172.21.15.90:0/3153500387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c104500 msgr2=0x7f801c10c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:57.122 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 --2- 172.21.15.90:0/3153500387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c104500 0x7f801c10c9e0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f8004009900 tx=0x7f800402a480 comp rx=0 tx=0).stop 2024-04-26T18:30:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 -- 172.21.15.90:0/3153500387 shutdown_connections 2024-04-26T18:30:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 --2- 172.21.15.90:0/3153500387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c10cfb0 0x7f801c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 --2- 172.21.15.90:0/3153500387 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c104500 0x7f801c10c9e0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 --2- 172.21.15.90:0/3153500387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c103be0 0x7f801c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 -- 172.21.15.90:0/3153500387 >> 172.21.15.90:0/3153500387 conn(0x7f801c077bb0 msgr2=0x7f801c077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 -- 172.21.15.90:0/3153500387 shutdown_connections 2024-04-26T18:30:57.123 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.122+0000 7f8020e07640 1 -- 172.21.15.90:0/3153500387 wait complete. 2024-04-26T18:30:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 Processor -- start 2024-04-26T18:30:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 -- start start 2024-04-26T18:30:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c103be0 0x7f801c101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c104500 0x7f801c101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c10cfb0 0x7f801c1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f801c112bf0 con 0x7f801c10cfb0 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f801c112a70 con 0x7f801c103be0 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8020e07640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f801c112d70 con 0x7f801c104500 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f801ad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c10cfb0 0x7f801c1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f801ad9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c10cfb0 0x7f801c1022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45222/0 (socket says 172.21.15.90:45222) 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f801ad9d640 1 -- 172.21.15.90:0/2358062936 learned_addr learned my addr 172.21.15.90:0/2358062936 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c104500 0x7f801c101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.123+0000 7f801a59c640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c103be0 0x7f801c101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f801ad9d640 1 -- 172.21.15.90:0/2358062936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c104500 msgr2=0x7f801c101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f801ad9d640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c104500 0x7f801c101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.126 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f801ad9d640 1 -- 172.21.15.90:0/2358062936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c103be0 msgr2=0x7f801c101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f801ad9d640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c103be0 0x7f801c101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f801ad9d640 1 -- 172.21.15.90:0/2358062936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801c1a75d0 con 0x7f801c10cfb0 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f801ad9d640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c10cfb0 0x7f801c1022b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9d0 tx=0x7f801400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f80137fe640 1 -- 172.21.15.90:0/2358062936 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400cd80 con 0x7f801c10cfb0 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f80137fe640 1 -- 172.21.15.90:0/2358062936 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8014005020 con 0x7f801c10cfb0 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f801c1a78c0 con 0x7f801c10cfb0 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f80137fe640 1 -- 172.21.15.90:0/2358062936 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80140116d0 con 0x7f801c10cfb0 2024-04-26T18:30:57.127 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.124+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f801c1a7d90 con 0x7f801c10cfb0 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.126+0000 7f80137fe640 1 -- 172.21.15.90:0/2358062936 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f80140051d0 con 0x7f801c10cfb0 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 Processor -- start 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 -- start start 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efad52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad5820 0x5592efadfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efae00e0 0x5592efae24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5592efa3b460 con 0x5592efad4f00 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5592ef9f5590 con 0x5592efae00e0 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8023bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5592efa37420 con 0x5592efad5820 2024-04-26T18:30:57.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f80137fe640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f80080a3dc0 0x7f80080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8019d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efad52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8019d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efad52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45238/0 (socket says 172.21.15.90:45238) 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f8019d9b640 1 -- 172.21.15.90:0/638248652 learned_addr learned my addr 172.21.15.90:0/638248652 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.127+0000 7f801ad9d640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f80080a3dc0 0x7f80080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f801a59c640 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad5820 0x5592efadfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8019d9b640 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efae00e0 0x5592efae24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f80137fe640 1 -- 172.21.15.90:0/2358062936 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 11851+0+0 (secure 0 0 0) 0x7f80140d16c0 con 0x7f801c10cfb0 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8019d9b640 1 -- 172.21.15.90:0/638248652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad5820 msgr2=0x5592efadfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8019d9b640 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad5820 0x5592efadfba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8019d9b640 1 -- 172.21.15.90:0/638248652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efae00e0 msgr2=0x5592efae24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8019d9b640 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efae00e0 0x5592efae24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.135 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8019d9b640 1 -- 172.21.15.90:0/638248652 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592efa020d0 con 0x5592efad4f00 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8019d9b640 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efad52e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5592efad25a0 tx=0x7f8004005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ce9a7190c38203 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f801ad9d640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f80080a3dc0 0x7f80080a6280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f801409c660 tx=0x7f80140bd040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8010ff9640 1 -- 172.21.15.90:0/638248652 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004044020 con 0x5592efad4f00 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8010ff9640 1 -- 172.21.15.90:0/638248652 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8004031870 con 0x5592efad4f00 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.128+0000 7f8010ff9640 1 -- 172.21.15.90:0/638248652 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004035040 con 0x5592efad4f00 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 -- 172.21.15.90:0/638248652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 msgr2=0x5592efad52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efad52e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x5592efad25a0 tx=0x7f8004005e60 comp rx=0 tx=0).stop 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 -- 172.21.15.90:0/638248652 shutdown_connections 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efae00e0 0x5592efae24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad5820 0x5592efadfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 --2- 172.21.15.90:0/638248652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efad52e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 -- 172.21.15.90:0/638248652 >> 172.21.15.90:0/638248652 conn(0x5592efad1410 msgr2=0x5592efad17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:57.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 -- 172.21.15.90:0/638248652 shutdown_connections 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 -- 172.21.15.90:0/638248652 wait complete. 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.129+0000 7f8023bfa740 1 -- start start 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 1 Processor -- start 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efadd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efadd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efaddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5592efae2cb0 con 0x5592efad4f00 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x5592ef9c2b60 con 0x5592efad5820 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5592efae2e30 con 0x5592efae00e0 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8023bfa740 10 client.0 _ll_register_callbacks cb 0x5592ef9e72b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efadd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efadd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45250/0 (socket says 172.21.15.90:45250) 2024-04-26T18:30:57.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 -- 172.21.15.90:0/2197621156 learned_addr learned my addr 172.21.15.90:0/2197621156 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f801a59c640 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efadd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efaddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 -- 172.21.15.90:0/2197621156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 msgr2=0x5592efaddac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efaddac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 -- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 msgr2=0x5592efadd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efadd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 -- 172.21.15.90:0/2197621156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592ef74d210 con 0x5592efad4f00 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efadd040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8004032d70 tx=0x7f8004005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a0cdbe1fc60682 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.130+0000 7f8001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8001ffb640 1 -- 172.21.15.90:0/2197621156 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004044030 con 0x5592efad4f00 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8001ffb640 1 -- 172.21.15.90:0/2197621156 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(159..159 src has 1..159) ==== 11851+0+0 (secure 0 0 0) 0x7f80040035f0 con 0x5592efad4f00 2024-04-26T18:30:57.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8004031870 con 0x5592efad4f00 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8001ffb640 1 -- 172.21.15.90:0/2197621156 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7f8004035070 con 0x5592efad4f00 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 10 client.16770 fetch_fsmap learned FSMap version 257 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=4+}) -- 0x5592ef819620 con 0x5592efad4f00 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8001ffb640 1 -- 172.21.15.90:0/2197621156 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8004031870 con 0x5592efad4f00 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8001ffb640 1 -- 172.21.15.90:0/2197621156 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 257) ==== 4716+0+0 (secure 0 0 0) 0x7f800404a760 con 0x5592efad4f00 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 10 client.16770 fetch_fsmap finished waiting for FSMap version 257 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 20 client.16770 populate_metadata read hostname 'smithi090' 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efaf0370 0x5592efaf27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 4 client.16770 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592efa49070 con 0x5592efaf0370 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.131+0000 7f801a59c640 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efaf0370 0x5592efaf27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.132+0000 7f801a59c640 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efaf0370 0x5592efaf27b0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cdcea6223cdf04c server_cookie=adaedadeae8ddf29 in_seq=0 out_seq=0 2024-04-26T18:30:57.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.132+0000 7f8001ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8001ffb640 1 -- 172.21.15.90:0/2197621156 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5592efa49070 con 0x5592efaf0370 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8001ffb640 10 client.16770 handle_command_reply: tid=0 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 client.16770 shutdown 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 msgr2=0x5592efadd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efadd040 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f8004032d70 tx=0x7f8004005910 comp rx=0 tx=0).stop 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 shutdown_connections 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efaf0370 0x5592efaf27b0 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efaddac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8001ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efadd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2197621156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad4f00 0x5592efadd040 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 >> 172.21.15.90:0/2197621156 conn(0x5592efad1410 msgr2=0x5592efad7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:57.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 shutdown_connections 2024-04-26T18:30:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 1 -- 172.21.15.90:0/2197621156 wait complete. 2024-04-26T18:30:57.141 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.133+0000 7f8023bfa740 20 client.16770 trim_cache size 0 max 16384 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 Processor -- start 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 -- start start 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad4f00 0x5592efaebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efaf8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efae00e0 0x5592efaf85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1b2e90 con 0x5592efae00e0 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1b0c20 con 0x5592efad5820 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8023bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1b27c0 con 0x5592efad4f00 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efae00e0 0x5592efaf85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efae00e0 0x5592efaf85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45254/0 (socket says 172.21.15.90:45254) 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 -- 172.21.15.90:0/2380205261 learned_addr learned my addr 172.21.15.90:0/2380205261 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f801a59c640 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efaf8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad4f00 0x5592efaebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 -- 172.21.15.90:0/2380205261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad4f00 msgr2=0x5592efaebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad4f00 0x5592efaebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 -- 172.21.15.90:0/2380205261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 msgr2=0x5592efaf8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efaf8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 -- 172.21.15.90:0/2380205261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592efae2840 con 0x5592efae00e0 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f801a59c640 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efaf8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.148+0000 7f8019d9b640 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efae00e0 0x5592efaf85c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8004033000 tx=0x7f8004005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66acd6441fc42cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8003fff640 1 -- 172.21.15.90:0/2380205261 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004044040 con 0x5592efae00e0 2024-04-26T18:30:57.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8003fff640 1 -- 172.21.15.90:0/2380205261 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80040317b0 con 0x5592efae00e0 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8003fff640 1 -- 172.21.15.90:0/2380205261 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800404a430 con 0x5592efae00e0 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 -- 172.21.15.90:0/2380205261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efae00e0 msgr2=0x5592efaf85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efae00e0 0x5592efaf85c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8004033000 tx=0x7f8004005fd0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 -- 172.21.15.90:0/2380205261 shutdown_connections 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efae00e0 0x5592efaf85c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad5820 0x5592efaf8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 --2- 172.21.15.90:0/2380205261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efad4f00 0x5592efaebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 -- 172.21.15.90:0/2380205261 >> 172.21.15.90:0/2380205261 conn(0x5592efad1410 msgr2=0x5592efad7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 -- 172.21.15.90:0/2380205261 shutdown_connections 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 -- 172.21.15.90:0/2380205261 wait complete. 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.149+0000 7f8023bfa740 1 -- start start 2024-04-26T18:30:57.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 1 Processor -- start 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad4f00 0x5592efad9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad5820 0x5592efada360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efada8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1b19c0 con 0x5592efad5820 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1b12f0 con 0x5592efad4f00 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f801c1b2090 con 0x5592efae00e0 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8023bfa740 10 client.0 _ll_register_callbacks cb 0x7f800c00c2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad5820 0x5592efada360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad5820 0x5592efada360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45262/0 (socket says 172.21.15.90:45262) 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 -- 172.21.15.90:0/1849847128 learned_addr learned my addr 172.21.15.90:0/1849847128 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8019d9b640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad4f00 0x5592efad9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8019d9b640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efada8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 -- 172.21.15.90:0/1849847128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 msgr2=0x5592efada8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efada8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 -- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad4f00 msgr2=0x5592efad9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad4f00 0x5592efad9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 -- 172.21.15.90:0/1849847128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592efb558f0 con 0x5592efad5820 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8019d9b640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efada8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f801a59c640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad5820 0x5592efada360 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f800c005d70 tx=0x7f800c0108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e623f3f0ec948b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8001ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.150+0000 7f8001ffb640 1 -- 172.21.15.90:0/1849847128 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c020070 con 0x5592efad5820 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8001ffb640 1 -- 172.21.15.90:0/1849847128 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(159..159 src has 1..159) ==== 11851+0+0 (secure 0 0 0) 0x7f800c01c850 con 0x5592efad5820 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80040317b0 con 0x5592efad5820 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8001ffb640 1 -- 172.21.15.90:0/1849847128 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=257) ==== 24+0+0 (secure 0 0 0) 0x7f800c02e050 con 0x5592efad5820 2024-04-26T18:30:57.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 10 client.16776 fetch_fsmap learned FSMap version 257 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=257,monmap=4+}) -- 0x5592efb56ab0 con 0x5592efad5820 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8001ffb640 1 -- 172.21.15.90:0/1849847128 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80040317b0 con 0x5592efad5820 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8001ffb640 1 -- 172.21.15.90:0/1849847128 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 257) ==== 4716+0+0 (secure 0 0 0) 0x7f800c00f4e0 con 0x5592efad5820 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 10 client.16776 fetch_fsmap finished waiting for FSMap version 257 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 10 client.16776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 20 client.16776 populate_metadata read hostname 'smithi090' 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efb57270 0x5592efb59690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 4 client.16776 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592efa42ee0 con 0x5592efb57270 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.151+0000 7f8019d9b640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efb57270 0x5592efb59690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.152+0000 7f8001ffb640 10 client.16776 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:30:57.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:30:57.152+0000 7f8019d9b640 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efb57270 0x5592efb59690 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b3b4f9e18e53808 server_cookie=5f6d96aca9c97d7e in_seq=0 out_seq=0 2024-04-26T18:30:57.642 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:30:57.642 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 systemd[1]: Stopped Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:57.642 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 15.220s CPU time. 2024-04-26T18:30:57.642 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 systemd[1]: Starting Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:30:57.642 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:57 smithi090 podman[98518]: 2024-04-26 18:30:57.441830349 +0000 UTC m=+1.189350268 container remove e984b04ed9cbbf71e4bd2d29860e1a5c32f78d8b062a5ac47dd9a6c5e6e63244 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424) 2024-04-26T18:30:57.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:57 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:30:57.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.95 MB/sec execute 142 sec latency 11722.907 ms 2024-04-26T18:30:57.896 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 podman[98641]: 2024-04-26 18:30:57.642905796 +0000 UTC m=+0.103967505 container create a8156d426adfc1bffeac73c09deeb1c526615cbe667533be86947c167b91ce91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T18:30:57.896 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 podman[98641]: 2024-04-26 18:30:57.557210059 +0000 UTC m=+0.018271781 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:30:57.896 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 podman[98641]: 2024-04-26 18:30:57.872247915 +0000 UTC m=+0.333309629 container init a8156d426adfc1bffeac73c09deeb1c526615cbe667533be86947c167b91ce91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS) 2024-04-26T18:30:57.897 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 podman[98641]: 2024-04-26 18:30:57.874987538 +0000 UTC m=+0.336049250 container start a8156d426adfc1bffeac73c09deeb1c526615cbe667533be86947c167b91ce91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T18:30:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:57 smithi119 ceph-mon[32752]: pgmap v652: 97 pgs: 97 active+clean; 576 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-04-26T18:30:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:57 smithi119 ceph-mon[32752]: Replacing daemon mds.a as rank 1 with standby daemon mds.g 2024-04-26T18:30:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:57 smithi119 ceph-mon[32752]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:30:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:57 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:57 smithi119 ceph-mon[32752]: osdmap e160: 12 total, 12 up, 12 in 2024-04-26T18:30:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:57 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=j=up:active} 2 up:standby-replay 2024-04-26T18:30:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:57 smithi167 ceph-mon[32751]: pgmap v652: 97 pgs: 97 active+clean; 576 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-04-26T18:30:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:57 smithi167 ceph-mon[32751]: Replacing daemon mds.a as rank 1 with standby daemon mds.g 2024-04-26T18:30:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:57 smithi167 ceph-mon[32751]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:30:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:57 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:57 smithi167 ceph-mon[32751]: osdmap e160: 12 total, 12 up, 12 in 2024-04-26T18:30:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:57 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=j=up:active} 2 up:standby-replay 2024-04-26T18:30:58.375 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:58 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-26T18:30:58.376 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:30:58 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 10.845s CPU time. 2024-04-26T18:30:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-mon[29180]: pgmap v652: 97 pgs: 97 active+clean; 576 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-04-26T18:30:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-mon[29180]: Replacing daemon mds.a as rank 1 with standby daemon mds.g 2024-04-26T18:30:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-mon[29180]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:30:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:30:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-mon[29180]: osdmap e160: 12 total, 12 up, 12 in 2024-04-26T18:30:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=j=up:active} 2 up:standby-replay 2024-04-26T18:30:58.377 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-mds[98673]: -- 172.21.15.90:0/1229961083 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559133803a40 con 0x5591337d7000 2024-04-26T18:30:58.377 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: starting mds.a at 2024-04-26T18:30:58.377 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 bash[98641]: a8156d426adfc1bffeac73c09deeb1c526615cbe667533be86947c167b91ce91 2024-04-26T18:30:58.377 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:30:57 smithi090 systemd[1]: Started Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:30:58.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.89 MB/sec execute 143 sec latency 12722.982 ms 2024-04-26T18:30:59.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.84 MB/sec execute 144 sec latency 13723.073 ms 2024-04-26T18:31:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:boot 2024-04-26T18:31:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:resolve 2024-04-26T18:31:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:59 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:31:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:30:59 smithi119 ceph-mon[32752]: pgmap v654: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:boot 2024-04-26T18:31:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:resolve 2024-04-26T18:31:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:59 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:31:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:30:59 smithi167 ceph-mon[32751]: pgmap v654: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:boot 2024-04-26T18:31:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:resolve 2024-04-26T18:31:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:59 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:31:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:30:59 smithi090 ceph-mon[29180]: pgmap v654: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:00.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.78 MB/sec execute 145 sec latency 14723.177 ms 2024-04-26T18:31:01.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.73 MB/sec execute 146 sec latency 15723.273 ms 2024-04-26T18:31:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:01 smithi119 ceph-mon[32752]: pgmap v655: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:01 smithi167 ceph-mon[32751]: pgmap v655: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:02.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:01 smithi090 ceph-mon[29180]: pgmap v655: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:02.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.68 MB/sec execute 147 sec latency 16723.359 ms 2024-04-26T18:31:03.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.63 MB/sec execute 148 sec latency 17723.441 ms 2024-04-26T18:31:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:03 smithi119 ceph-mon[32752]: pgmap v656: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:03 smithi167 ceph-mon[32751]: pgmap v656: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:03 smithi090 ceph-mon[29180]: pgmap v656: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-04-26T18:31:04.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.57 MB/sec execute 149 sec latency 18723.539 ms 2024-04-26T18:31:05.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.52 MB/sec execute 150 sec latency 19723.619 ms 2024-04-26T18:31:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:05 smithi119 ceph-mon[32752]: pgmap v657: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-26T18:31:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:05 smithi167 ceph-mon[32751]: pgmap v657: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-26T18:31:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:05 smithi090 ceph-mon[29180]: pgmap v657: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-26T18:31:06.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.47 MB/sec execute 151 sec latency 20723.699 ms 2024-04-26T18:31:07.893 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.43 MB/sec execute 152 sec latency 21723.779 ms 2024-04-26T18:31:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:07 smithi119 ceph-mon[32752]: pgmap v658: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-26T18:31:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:07 smithi167 ceph-mon[32751]: pgmap v658: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-26T18:31:08.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:07 smithi090 ceph-mon[29180]: pgmap v658: 97 pgs: 97 active+clean; 580 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.0 MiB/s wr, 87 op/s 2024-04-26T18:31:08.363 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 1. 2024-04-26T18:31:08.363 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 systemd[1]: Stopped Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:31:08.363 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 10.845s CPU time. 2024-04-26T18:31:08.363 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 systemd[1]: Starting Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:31:08.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 podman[98745]: 2024-04-26 18:31:08.265992389 +0000 UTC m=+0.018653348 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:31:08.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 podman[98745]: 2024-04-26 18:31:08.366663066 +0000 UTC m=+0.119323994 container create c2366e377dcb0615fc82b9d16d74d2f414f0a577537ee8b60c157b0c505ebf11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:31:08.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.38 MB/sec execute 153 sec latency 22723.899 ms 2024-04-26T18:31:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mon[29180]: daemon mds.j restarted 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mon[29180]: osdmap e161: 12 total, 12 up, 12 in 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:boot 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mon[29180]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:resolve} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:replay} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 podman[98745]: 2024-04-26 18:31:08.837932861 +0000 UTC m=+0.590593796 container init c2366e377dcb0615fc82b9d16d74d2f414f0a577537ee8b60c157b0c505ebf11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T18:31:09.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 podman[98745]: 2024-04-26 18:31:08.840862691 +0000 UTC m=+0.593523627 container start c2366e377dcb0615fc82b9d16d74d2f414f0a577537ee8b60c157b0c505ebf11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T18:31:09.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-mds[98764]: -- 172.21.15.90:0/21462989 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5628072d5a40 con 0x5628072a9000 2024-04-26T18:31:09.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: starting mds.j at 2024-04-26T18:31:09.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 bash[98745]: c2366e377dcb0615fc82b9d16d74d2f414f0a577537ee8b60c157b0c505ebf11 2024-04-26T18:31:09.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:31:08 smithi090 systemd[1]: Started Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:08 smithi119 ceph-mon[32752]: daemon mds.j restarted 2024-04-26T18:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:08 smithi119 ceph-mon[32752]: osdmap e161: 12 total, 12 up, 12 in 2024-04-26T18:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:boot 2024-04-26T18:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:08 smithi119 ceph-mon[32752]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:08 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:resolve} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:08 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:replay} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:08 smithi167 ceph-mon[32751]: daemon mds.j restarted 2024-04-26T18:31:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:08 smithi167 ceph-mon[32751]: osdmap e161: 12 total, 12 up, 12 in 2024-04-26T18:31:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:boot 2024-04-26T18:31:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:08 smithi167 ceph-mon[32751]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:31:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:08 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:resolve} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:31:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:31:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:08 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:replay} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:09.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.33 MB/sec execute 154 sec latency 23724.011 ms 2024-04-26T18:31:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:09 smithi119 ceph-mon[32752]: pgmap v659: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 963 KiB/s wr, 81 op/s 2024-04-26T18:31:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:09 smithi167 ceph-mon[32751]: pgmap v659: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 963 KiB/s wr, 81 op/s 2024-04-26T18:31:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:09 smithi090 ceph-mon[29180]: pgmap v659: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 963 KiB/s wr, 81 op/s 2024-04-26T18:31:10.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.28 MB/sec execute 155 sec latency 24724.111 ms 2024-04-26T18:31:11.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:10 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:resolve 2024-04-26T18:31:11.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:10 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:10 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:resolve 2024-04-26T18:31:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:10 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:10 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:resolve 2024-04-26T18:31:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:10 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:11.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32211 7.23 MB/sec execute 156 sec latency 25724.171 ms 2024-04-26T18:31:12.030 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8001ffb640 1 -- 172.21.15.90:0/1849847128 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592efa42ee0 con 0x5592efb57270 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8001ffb640 10 client.16776 handle_command_reply: tid=0 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 client.16776 shutdown 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad5820 msgr2=0x5592efada360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad5820 0x5592efada360 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f800c005d70 tx=0x7f800c0108c0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 shutdown_connections 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5592efb57270 0x5592efb59690 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5592efae00e0 0x5592efada8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x5592efad5820 0x5592efada360 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8001ffb640 0 client.16776 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 --2- 172.21.15.90:0/1849847128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5592efad4f00 0x5592efad9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.031 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 >> 172.21.15.90:0/1849847128 conn(0x5592efad1410 msgr2=0x5592efad68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 shutdown_connections 2024-04-26T18:31:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 1 -- 172.21.15.90:0/1849847128 wait complete. 2024-04-26T18:31:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.030+0000 7f8023bfa740 20 client.16776 trim_cache size 0 max 16384 2024-04-26T18:31:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:31:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:31:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:31:12.032 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:31:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f80080a3dc0 msgr2=0x7f80080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f80080a3dc0 0x7f80080a6280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f801409c660 tx=0x7f80140bd040 comp rx=0 tx=0).stop 2024-04-26T18:31:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c10cfb0 msgr2=0x7f801c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c10cfb0 0x7f801c1022b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f801400e9d0 tx=0x7f801400ee90 comp rx=0 tx=0).stop 2024-04-26T18:31:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 shutdown_connections 2024-04-26T18:31:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f80080a3dc0 0x7f80080a6280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.033 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f801c10cfb0 0x7f801c1022b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f801c104500 0x7f801c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 --2- 172.21.15.90:0/2358062936 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f801c103be0 0x7f801c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 >> 172.21.15.90:0/2358062936 conn(0x7f801c077bb0 msgr2=0x7f801c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 shutdown_connections 2024-04-26T18:31:12.034 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:12.031+0000 7f8020e07640 1 -- 172.21.15.90:0/2358062936 wait complete. 2024-04-26T18:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:11 smithi119 ceph-mon[32752]: pgmap v661: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 236 KiB/s wr, 21 op/s 2024-04-26T18:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:11 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:11 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199997 2024-04-26T18:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:11 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199997 2024-04-26T18:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:reconnect 2024-04-26T18:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:reconnect 2024-04-26T18:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:11 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:11 smithi167 ceph-mon[32751]: pgmap v661: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 236 KiB/s wr, 21 op/s 2024-04-26T18:31:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:11 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:31:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:11 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199997 2024-04-26T18:31:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:11 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199997 2024-04-26T18:31:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:reconnect 2024-04-26T18:31:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:reconnect 2024-04-26T18:31:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:11 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:11 smithi090 ceph-mon[29180]: pgmap v661: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 236 KiB/s wr, 21 op/s 2024-04-26T18:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:11 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:11 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199997 2024-04-26T18:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:11 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00199997 2024-04-26T18:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:reconnect 2024-04-26T18:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:reconnect 2024-04-26T18:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:11 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:12.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32447 7.26 MB/sec execute 157 sec latency 26717.874 ms 2024-04-26T18:31:12.952 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:31:12.686+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:31:13.271 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:rejoin 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:rejoin 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:active 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:active 2024-04-26T18:31:13.272 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:31:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:rejoin 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:rejoin 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:active 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:active 2024-04-26T18:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:rejoin 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:rejoin 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2673484677,v1:172.21.15.90:6839/2673484677] up:active 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2470728764,v1:172.21.15.90:6837/2470728764] up:active 2024-04-26T18:31:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:31:13.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:31:13 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:31:13.270+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:31:13.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:31:13 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:31:13.270+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:31:13.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32507 7.24 MB/sec execute 158 sec latency 27717.915 ms 2024-04-26T18:31:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:13 smithi119 ceph-mon[32752]: pgmap v662: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 236 KiB/s wr, 21 op/s 2024-04-26T18:31:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:13 smithi167 ceph-mon[32751]: pgmap v662: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 236 KiB/s wr, 21 op/s 2024-04-26T18:31:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:13 smithi090 ceph-mon[29180]: pgmap v662: 97 pgs: 97 active+clean; 578 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 236 KiB/s wr, 21 op/s 2024-04-26T18:31:14.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32507 7.19 MB/sec execute 159 sec latency 28717.956 ms 2024-04-26T18:31:15.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32580 7.17 MB/sec execute 160 sec latency 29613.549 ms 2024-04-26T18:31:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:15 smithi119 ceph-mon[32752]: pgmap v663: 97 pgs: 97 active+clean; 577 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 665 KiB/s wr, 94 op/s 2024-04-26T18:31:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:15 smithi167 ceph-mon[32751]: pgmap v663: 97 pgs: 97 active+clean; 577 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 665 KiB/s wr, 94 op/s 2024-04-26T18:31:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:15 smithi090 ceph-mon[29180]: pgmap v663: 97 pgs: 97 active+clean; 577 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 665 KiB/s wr, 94 op/s 2024-04-26T18:31:16.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 32964 7.19 MB/sec execute 161 sec latency 107.436 ms 2024-04-26T18:31:17.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33516 7.34 MB/sec execute 162 sec latency 152.613 ms 2024-04-26T18:31:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:17 smithi119 ceph-mon[32752]: pgmap v664: 97 pgs: 97 active+clean; 577 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 665 KiB/s wr, 94 op/s 2024-04-26T18:31:18.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-mon[29180]: pgmap v664: 97 pgs: 97 active+clean; 577 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 665 KiB/s wr, 94 op/s 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1b9cff0700 time 2024-04-26T18:31:17.885600+0000 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1ba76193dd] 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.310 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: clone() 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:17.886+0000 7f1b9cff0700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1b9cff0700 time 2024-04-26T18:31:17.885600+0000 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:18.311 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1ba76193dd] 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.312 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: clone() 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: *** Caught signal (Aborted) ** 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: in thread 7f1b9cff0700 thread_name:ms_dispatch 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1ba63a1d20] 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: gsignal() 2024-04-26T18:31:18.313 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: abort() 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1ba7619437] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.314 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 16: clone() 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:17.888+0000 7f1b9cff0700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: in thread 7f1b9cff0700 thread_name:ms_dispatch 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1ba63a1d20] 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: gsignal() 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: abort() 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1ba7619437] 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.315 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 16: clone() 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: -1> 2024-04-26T18:31:17.886+0000 7f1b9cff0700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1b9cff0700 time 2024-04-26T18:31:17.885600+0000 2024-04-26T18:31:18.316 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1ba76193dd] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.317 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: clone() 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 0> 2024-04-26T18:31:17.888+0000 7f1b9cff0700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: in thread 7f1b9cff0700 thread_name:ms_dispatch 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1ba63a1d20] 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: gsignal() 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: abort() 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1ba7619437] 2024-04-26T18:31:18.318 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 16: clone() 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:18.319 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.320 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: -1> 2024-04-26T18:31:17.886+0000 7f1b9cff0700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f1b9cff0700 time 2024-04-26T18:31:17.885600+0000 2024-04-26T18:31:18.320 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:18.320 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.320 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.320 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f1ba76193dd] 2024-04-26T18:31:18.320 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.320 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:17 smithi167 ceph-mon[32751]: pgmap v664: 97 pgs: 97 active+clean; 577 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 665 KiB/s wr, 94 op/s 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.321 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: clone() 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 0> 2024-04-26T18:31:17.888+0000 7f1b9cff0700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: in thread 7f1b9cff0700 thread_name:ms_dispatch 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f1ba63a1d20] 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2: gsignal() 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 3: abort() 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f1ba7619437] 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f1ba76195a3] 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x556edbab4822] 2024-04-26T18:31:18.322 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x556edba24598] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x556edba26c17] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x556edb91fd0b] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x556edb9202ac] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x556edb908fad] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f1ba78ade20] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 13: (DispatchQueue::entry()+0x51f) [0x7f1ba78aafdf] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f1ba797dfc1] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f1ba63971ca] 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 16: clone() 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[98418]: 2024-04-26T18:31:18.323 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:17 smithi090 podman[98817]: 2024-04-26 18:31:17.979328345 +0000 UTC m=+0.026838686 container died 49f8fdc018cf3a22971f5d9f743446ceb0197a5700c4c1cae989890d63d4a588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T18:31:18.874 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:18 smithi090 podman[98817]: 2024-04-26 18:31:18.604716435 +0000 UTC m=+0.652226756 container remove 49f8fdc018cf3a22971f5d9f743446ceb0197a5700c4c1cae989890d63d4a588 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T18:31:18.874 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:18 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:31:18.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33662 7.37 MB/sec execute 163 sec latency 1074.148 ms 2024-04-26T18:31:19.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:18 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-26T18:31:19.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:18 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 11.292s CPU time. 2024-04-26T18:31:19.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.33 MB/sec execute 164 sec latency 2074.239 ms 2024-04-26T18:31:20.038 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:19 smithi090 ceph-mon[29180]: pgmap v665: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 5.1 MiB/s wr, 376 op/s 2024-04-26T18:31:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:19 smithi119 ceph-mon[32752]: pgmap v665: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 5.1 MiB/s wr, 376 op/s 2024-04-26T18:31:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:19 smithi167 ceph-mon[32751]: pgmap v665: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 5.1 MiB/s wr, 376 op/s 2024-04-26T18:31:20.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.29 MB/sec execute 165 sec latency 3074.364 ms 2024-04-26T18:31:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:20 smithi119 ceph-mon[32752]: osdmap e162: 12 total, 12 up, 12 in 2024-04-26T18:31:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:20 smithi167 ceph-mon[32751]: osdmap e162: 12 total, 12 up, 12 in 2024-04-26T18:31:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:20 smithi090 ceph-mon[29180]: osdmap e162: 12 total, 12 up, 12 in 2024-04-26T18:31:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:21.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.24 MB/sec execute 166 sec latency 4074.482 ms 2024-04-26T18:31:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:21 smithi119 ceph-mon[32752]: pgmap v667: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 5.1 MiB/s wr, 376 op/s 2024-04-26T18:31:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:31:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:21 smithi167 ceph-mon[32751]: pgmap v667: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 5.1 MiB/s wr, 376 op/s 2024-04-26T18:31:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:31:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:21 smithi090 ceph-mon[29180]: pgmap v667: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 5.1 MiB/s wr, 376 op/s 2024-04-26T18:31:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:31:22.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.20 MB/sec execute 167 sec latency 5074.633 ms 2024-04-26T18:31:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:22 smithi119 ceph-mon[32752]: pgmap v668: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2024-04-26T18:31:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:22 smithi119 ceph-mon[32752]: pgmap v669: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-04-26T18:31:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:22 smithi167 ceph-mon[32751]: pgmap v668: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2024-04-26T18:31:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:22 smithi167 ceph-mon[32751]: pgmap v669: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-04-26T18:31:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:22 smithi090 ceph-mon[29180]: pgmap v668: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 5.8 MiB/s wr, 429 op/s 2024-04-26T18:31:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:22 smithi090 ceph-mon[29180]: pgmap v669: 97 pgs: 97 active+clean; 613 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 6.5 MiB/s wr, 422 op/s 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.16 MB/sec execute 168 sec latency 6074.666 ms 2024-04-26T18:31:24.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.12 MB/sec execute 169 sec latency 7074.757 ms 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 -- 172.21.15.90:0/2384515060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006b160 msgr2=0x7f120006b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 --2- 172.21.15.90:0/2384515060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006b160 0x7f120006b5c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009930 tx=0x7f11f002a4d0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 -- 172.21.15.90:0/2384515060 shutdown_connections 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 --2- 172.21.15.90:0/2384515060 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f120006bb00 0x7f120010f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 --2- 172.21.15.90:0/2384515060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006b160 0x7f120006b5c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 --2- 172.21.15.90:0/2384515060 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1200108980 0x7f1200108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 -- 172.21.15.90:0/2384515060 >> 172.21.15.90:0/2384515060 conn(0x7f1200077630 msgr2=0x7f1200077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:25.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 -- 172.21.15.90:0/2384515060 shutdown_connections 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.130+0000 7f120862a640 1 -- 172.21.15.90:0/2384515060 wait complete. 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 Processor -- start 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 -- start start 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f120006b160 0x7f1200199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006bb00 0x7f12001995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1200108980 0x7f120019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112bd0 con 0x7f120006b160 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112a50 con 0x7f120006bb00 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f120862a640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1200112d50 con 0x7f1200108980 2024-04-26T18:31:25.134 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f120006b160 0x7f1200199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f120006b160 0x7f1200199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52528/0 (socket says 172.21.15.90:52528) 2024-04-26T18:31:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 -- 172.21.15.90:0/39467450 learned_addr learned my addr 172.21.15.90:0/39467450 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f1205bc5640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006bb00 0x7f12001995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f1206bc7640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1200108980 0x7f120019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 -- 172.21.15.90:0/39467450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1200108980 msgr2=0x7f120019d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1200108980 0x7f120019d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 -- 172.21.15.90:0/39467450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006bb00 msgr2=0x7f12001995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006bb00 0x7f12001995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 -- 172.21.15.90:0/39467450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120019e080 con 0x7f120006b160 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f1205bc5640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006bb00 0x7f12001995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f1206bc7640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1200108980 0x7f120019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.131+0000 7f12063c6640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f120006b160 0x7f1200199090 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9f0 tx=0x7f11fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.132+0000 7f11f77fe640 1 -- 172.21.15.90:0/39467450 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00cd80 con 0x7f120006b160 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.132+0000 7f11f77fe640 1 -- 172.21.15.90:0/39467450 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11fc0108e0 con 0x7f120006b160 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.132+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120019e370 con 0x7f120006b160 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.132+0000 7f11f77fe640 1 -- 172.21.15.90:0/39467450 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc018980 con 0x7f120006b160 2024-04-26T18:31:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.132+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12001a5c00 con 0x7f120006b160 2024-04-26T18:31:25.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.134+0000 7f11f77fe640 1 -- 172.21.15.90:0/39467450 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f11fc010430 con 0x7f120006b160 2024-04-26T18:31:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 Processor -- start 2024-04-26T18:31:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 -- start start 2024-04-26T18:31:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1be41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f11f77fe640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11ec0a3dc0 0x7f11ec0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.149 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d9c1b4a460 con 0x55d9c1bf34f0 2024-04-26T18:31:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d9c1b04590 con 0x55d9c1be3de0 2024-04-26T18:31:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.135+0000 7f120b422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9c1b46420 con 0x55d9c1be4700 2024-04-26T18:31:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1205bc5640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11ec0a3dc0 0x7f11ec0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f12063c6640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.150 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52534/0 (socket says 172.21.15.90:52534) 2024-04-26T18:31:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 -- 172.21.15.90:0/1560311673 learned_addr learned my addr 172.21.15.90:0/1560311673 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1be41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f11f77fe640 1 -- 172.21.15.90:0/39467450 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 12111+0+0 (secure 0 0 0) 0x7f11fc0d19c0 con 0x7f120006b160 2024-04-26T18:31:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 -- 172.21.15.90:0/1560311673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 msgr2=0x55d9c1bf2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 -- 172.21.15.90:0/1560311673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 msgr2=0x55d9c1be41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1be41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 -- 172.21.15.90:0/1560311673 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c1b110d0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f12063c6640 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1205bc5640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11ec0a3dc0 0x7f11ec0a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11f0000c00 tx=0x7f11f0009610 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f1206bc7640 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf58e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f11f8012b80 tx=0x7f11f8009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a18e006c5da57c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f11f4bfe640 1 -- 172.21.15.90:0/1560311673 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8012c70 con 0x55d9c1bf34f0 2024-04-26T18:31:25.152 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f11f4bfe640 1 -- 172.21.15.90:0/1560311673 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f8012e10 con 0x55d9c1bf34f0 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.136+0000 7f11f4bfe640 1 -- 172.21.15.90:0/1560311673 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f800e3b0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 -- 172.21.15.90:0/1560311673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 msgr2=0x55d9c1bf58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf58e0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f11f8012b80 tx=0x7f11f8009ae0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 -- 172.21.15.90:0/1560311673 shutdown_connections 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf58e0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 --2- 172.21.15.90:0/1560311673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1be41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 -- 172.21.15.90:0/1560311673 >> 172.21.15.90:0/1560311673 conn(0x55d9c1be0410 msgr2=0x55d9c1be07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:25.153 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 -- 172.21.15.90:0/1560311673 shutdown_connections 2024-04-26T18:31:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.137+0000 7f120b422740 1 -- 172.21.15.90:0/1560311673 wait complete. 2024-04-26T18:31:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 -- start start 2024-04-26T18:31:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 Processor -- start 2024-04-26T18:31:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1bf04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.154 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d9c1bf6120 con 0x55d9c1bf34f0 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d9c1ad1b60 con 0x55d9c1be3de0 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d9c1bf62a0 con 0x55d9c1be4700 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f120b422740 10 client.0 _ll_register_callbacks cb 0x55d9c1af62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52536/0 (socket says 172.21.15.90:52536) 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 -- 172.21.15.90:0/1912604100 learned_addr learned my addr 172.21.15.90:0/1912604100 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1205bc5640 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.155 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f12063c6640 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1bf04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 -- 172.21.15.90:0/1912604100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 msgr2=0x55d9c1bf09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 -- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 msgr2=0x55d9c1bf04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1bf04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.138+0000 7f1206bc7640 1 -- 172.21.15.90:0/1912604100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c185c210 con 0x55d9c1bf34f0 2024-04-26T18:31:25.156 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f1206bc7640 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf0f20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f11f801ef50 tx=0x7f11f801ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfdd2e184489424a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f11e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:31:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f11e9ffb640 1 -- 172.21.15.90:0/1912604100 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8012c70 con 0x55d9c1bf34f0 2024-04-26T18:31:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f11e9ffb640 1 -- 172.21.15.90:0/1912604100 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 12111+0+0 (secure 0 0 0) 0x7f11f801e5c0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f8012e10 con 0x55d9c1bf34f0 2024-04-26T18:31:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f11e9ffb640 1 -- 172.21.15.90:0/1912604100 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f11f8032070 con 0x55d9c1bf34f0 2024-04-26T18:31:25.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 10 client.16800 fetch_fsmap learned FSMap version 265 2024-04-26T18:31:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=4+}) -- 0x55d9c1928620 con 0x55d9c1bf34f0 2024-04-26T18:31:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f11e9ffb640 1 -- 172.21.15.90:0/1912604100 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f8012e10 con 0x55d9c1bf34f0 2024-04-26T18:31:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f11e9ffb640 1 -- 172.21.15.90:0/1912604100 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 265) ==== 4716+0+0 (secure 0 0 0) 0x7f11f80055b0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 10 client.16800 fetch_fsmap finished waiting for FSMap version 265 2024-04-26T18:31:25.158 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 20 client.16800 populate_metadata read hostname 'smithi090' 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c03ac0 0x55d9c1c05f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 4 client.16800 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.139+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c1b58070 con 0x55d9c1c03ac0 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.140+0000 7f12063c6640 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c03ac0 0x55d9c1c05f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.140+0000 7f11e9ffb640 10 client.16800 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.140+0000 7f12063c6640 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c03ac0 0x55d9c1c05f20 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21c9f2001b71952 server_cookie=60edd3ea3831ab4f in_seq=0 out_seq=0 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f11e9ffb640 1 -- 172.21.15.90:0/1912604100 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9c1b58070 con 0x55d9c1c03ac0 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f11e9ffb640 10 client.16800 handle_command_reply: tid=0 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 client.16800 shutdown 2024-04-26T18:31:25.159 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 msgr2=0x55d9c1bf0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf0f20 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f11f801ef50 tx=0x7f11f801ef80 comp rx=0 tx=0).stop 2024-04-26T18:31:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 shutdown_connections 2024-04-26T18:31:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c03ac0 0x55d9c1c05f20 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bf0f20 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be4700 0x55d9c1bf09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f11e9ffb640 0 client.16800 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:25.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 --2- 172.21.15.90:0/1912604100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be3de0 0x55d9c1bf04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 >> 172.21.15.90:0/1912604100 conn(0x55d9c1be0410 msgr2=0x55d9c1be6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 shutdown_connections 2024-04-26T18:31:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 1 -- 172.21.15.90:0/1912604100 wait complete. 2024-04-26T18:31:25.161 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.141+0000 7f120b422740 20 client.16800 trim_cache size 0 max 16384 2024-04-26T18:31:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 Processor -- start 2024-04-26T18:31:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 -- start start 2024-04-26T18:31:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bff050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1bf0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1c0b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f12001aee20 con 0x55d9c1bf34f0 2024-04-26T18:31:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f12001acbb0 con 0x55d9c1be4700 2024-04-26T18:31:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f120b422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ae750 con 0x55d9c1be3de0 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1c0b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1c0b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52546/0 (socket says 172.21.15.90:52546) 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.156+0000 7f1206bc7640 1 -- 172.21.15.90:0/4081041432 learned_addr learned my addr 172.21.15.90:0/4081041432 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f12063c6640 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bff050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f1205bc5640 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1bf0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f1206bc7640 1 -- 172.21.15.90:0/4081041432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 msgr2=0x55d9c1bff050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f1206bc7640 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bff050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f1206bc7640 1 -- 172.21.15.90:0/4081041432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 msgr2=0x55d9c1bf0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f1206bc7640 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1bf0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f1206bc7640 1 -- 172.21.15.90:0/4081041432 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c1beb2c0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f12063c6640 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f1206bc7640 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1c0b830 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f11f800a010 tx=0x7f11f801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a45a9c8fe05ae2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f11ebfff640 1 -- 172.21.15.90:0/4081041432 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8009e10 con 0x55d9c1bf34f0 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f11ebfff640 1 -- 172.21.15.90:0/4081041432 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f80099c0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.157+0000 7f11ebfff640 1 -- 172.21.15.90:0/4081041432 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f801e410 con 0x55d9c1bf34f0 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- 172.21.15.90:0/4081041432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 msgr2=0x55d9c1c0b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1c0b830 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f11f800a010 tx=0x7f11f801ec90 comp rx=0 tx=0).stop 2024-04-26T18:31:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- 172.21.15.90:0/4081041432 shutdown_connections 2024-04-26T18:31:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1c0b830 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1bf0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 --2- 172.21.15.90:0/4081041432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bff050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- 172.21.15.90:0/4081041432 >> 172.21.15.90:0/4081041432 conn(0x55d9c1be0410 msgr2=0x55d9c1bf0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- 172.21.15.90:0/4081041432 shutdown_connections 2024-04-26T18:31:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- 172.21.15.90:0/4081041432 wait complete. 2024-04-26T18:31:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- start start 2024-04-26T18:31:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 Processor -- start 2024-04-26T18:31:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bfb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1beced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ad950 con 0x55d9c1bf34f0 2024-04-26T18:31:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ad280 con 0x55d9c1be4700 2024-04-26T18:31:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12001ae020 con 0x55d9c1be3de0 2024-04-26T18:31:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f120b422740 10 client.0 _ll_register_callbacks cb 0x7f11f8012650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f1206bc7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52560/0 (socket says 172.21.15.90:52560) 2024-04-26T18:31:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f1206bc7640 1 -- 172.21.15.90:0/2351963821 learned_addr learned my addr 172.21.15.90:0/2351963821 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.158+0000 7f1205bc5640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1beced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f12063c6640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bfb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.176 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f1206bc7640 1 -- 172.21.15.90:0/2351963821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 msgr2=0x55d9c1bfb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f1206bc7640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bfb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f1206bc7640 1 -- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 msgr2=0x55d9c1beced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f1206bc7640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1beced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f1206bc7640 1 -- 172.21.15.90:0/2351963821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c1bedc90 con 0x55d9c1bf34f0 2024-04-26T18:31:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f12063c6640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bfb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:25.177 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f1206bc7640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bed410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f11f8012070 tx=0x7f11f8009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d09328f19897203 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f11e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:31:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f11e9ffb640 1 -- 172.21.15.90:0/2351963821 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8042020 con 0x55d9c1bf34f0 2024-04-26T18:31:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f11e9ffb640 1 -- 172.21.15.90:0/2351963821 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 12111+0+0 (secure 0 0 0) 0x7f11f801e410 con 0x55d9c1bf34f0 2024-04-26T18:31:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f80099c0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f11e9ffb640 1 -- 172.21.15.90:0/2351963821 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=265) ==== 24+0+0 (secure 0 0 0) 0x7f11f801e610 con 0x55d9c1bf34f0 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 10 client.16806 fetch_fsmap learned FSMap version 265 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=265,monmap=4+}) -- 0x55d9c1c6a100 con 0x55d9c1bf34f0 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f11e9ffb640 1 -- 172.21.15.90:0/2351963821 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f80099c0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f11e9ffb640 1 -- 172.21.15.90:0/2351963821 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 265) ==== 4716+0+0 (secure 0 0 0) 0x7f11f803f6e0 con 0x55d9c1bf34f0 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 10 client.16806 fetch_fsmap finished waiting for FSMap version 265 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 20 client.16806 populate_metadata read hostname 'smithi090' 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c6a730 0x55d9c1c6cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 4 client.16806 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:31:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.159+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9c1b51ee0 con 0x55d9c1c6a730 2024-04-26T18:31:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.160+0000 7f12063c6640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c6a730 0x55d9c1c6cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.160+0000 7f12063c6640 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c6a730 0x55d9c1c6cb50 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d88665295e36db server_cookie=ab487f4e1c1c736e in_seq=0 out_seq=0 2024-04-26T18:31:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:25.160+0000 7f11e9ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:24 smithi119 ceph-mon[32752]: pgmap v670: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.9 MiB/s wr, 480 op/s 2024-04-26T18:31:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:24 smithi167 ceph-mon[32751]: pgmap v670: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.9 MiB/s wr, 480 op/s 2024-04-26T18:31:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:24 smithi090 ceph-mon[29180]: pgmap v670: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.9 MiB/s wr, 480 op/s 2024-04-26T18:31:25.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.07 MB/sec execute 170 sec latency 8074.819 ms 2024-04-26T18:31:26.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 7.03 MB/sec execute 171 sec latency 9074.938 ms 2024-04-26T18:31:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:26 smithi119 ceph-mon[32752]: pgmap v671: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 352 KiB/s wr, 57 op/s 2024-04-26T18:31:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:26 smithi167 ceph-mon[32751]: pgmap v671: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 352 KiB/s wr, 57 op/s 2024-04-26T18:31:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:26 smithi090 ceph-mon[29180]: pgmap v671: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 352 KiB/s wr, 57 op/s 2024-04-26T18:31:27.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 6.99 MB/sec execute 172 sec latency 10075.018 ms 2024-04-26T18:31:28.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 6.95 MB/sec execute 173 sec latency 11075.108 ms 2024-04-26T18:31:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:28 smithi119 ceph-mon[32752]: pgmap v672: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 311 KiB/s wr, 50 op/s 2024-04-26T18:31:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:28 smithi119 ceph-mon[32752]: osdmap e163: 12 total, 12 up, 12 in 2024-04-26T18:31:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:28 smithi167 ceph-mon[32751]: pgmap v672: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 311 KiB/s wr, 50 op/s 2024-04-26T18:31:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:28 smithi167 ceph-mon[32751]: osdmap e163: 12 total, 12 up, 12 in 2024-04-26T18:31:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:28 smithi090 ceph-mon[29180]: pgmap v672: 97 pgs: 97 active+clean; 606 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 311 KiB/s wr, 50 op/s 2024-04-26T18:31:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:28 smithi090 ceph-mon[29180]: osdmap e163: 12 total, 12 up, 12 in 2024-04-26T18:31:29.415 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:31:29.415 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 systemd[1]: Stopped Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:31:29.415 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 11.292s CPU time. 2024-04-26T18:31:29.415 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 systemd[1]: Starting Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:31:29.415 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 podman[99092]: 2024-04-26 18:31:29.273297349 +0000 UTC m=+0.018253172 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:31:29.416 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 podman[99092]: 2024-04-26 18:31:29.415994142 +0000 UTC m=+0.160949967 container create f18336076f7d7b0ca8322d55481fe559816242be1ad9ee410f4ab3303515ef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424) 2024-04-26T18:31:29.726 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 podman[99092]: 2024-04-26 18:31:29.627001565 +0000 UTC m=+0.371957390 container init f18336076f7d7b0ca8322d55481fe559816242be1ad9ee410f4ab3303515ef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T18:31:29.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 podman[99092]: 2024-04-26 18:31:29.629576682 +0000 UTC m=+0.374532505 container start f18336076f7d7b0ca8322d55481fe559816242be1ad9ee410f4ab3303515ef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T18:31:29.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 ceph-mds[99111]: -- 172.21.15.90:0/2655550689 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561093be1a40 con 0x561093bb4c00 2024-04-26T18:31:29.727 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: starting mds.d at 2024-04-26T18:31:29.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 6.91 MB/sec execute 174 sec latency 12075.238 ms 2024-04-26T18:31:30.125 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 bash[99092]: f18336076f7d7b0ca8322d55481fe559816242be1ad9ee410f4ab3303515ef83 2024-04-26T18:31:30.125 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:31:29 smithi090 systemd[1]: Started Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:31:30.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 6.87 MB/sec execute 175 sec latency 13075.385 ms 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: pgmap v674: 97 pgs: 97 active+clean; 607 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 428 KiB/s wr, 119 op/s 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: daemon mds.d restarted 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: osdmap e164: 12 total, 12 up, 12 in 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:boot 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:30 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: pgmap v674: 97 pgs: 97 active+clean; 607 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 428 KiB/s wr, 119 op/s 2024-04-26T18:31:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: daemon mds.d restarted 2024-04-26T18:31:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:31:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: osdmap e164: 12 total, 12 up, 12 in 2024-04-26T18:31:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:boot 2024-04-26T18:31:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:31:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:31:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:31:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:30 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: pgmap v674: 97 pgs: 97 active+clean; 607 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 428 KiB/s wr, 119 op/s 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: daemon mds.d restarted 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: osdmap e164: 12 total, 12 up, 12 in 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:boot 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:30 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:31.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33705 6.83 MB/sec execute 176 sec latency 14075.438 ms 2024-04-26T18:31:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:31 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:resolve 2024-04-26T18:31:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:31 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:31 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:resolve 2024-04-26T18:31:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:31 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:31 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:resolve 2024-04-26T18:31:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:31 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f11e9ffb640 1 -- 172.21.15.90:0/2351963821 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9c1b51ee0 con 0x55d9c1c6a730 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f11e9ffb640 10 client.16806 handle_command_reply: tid=0 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 client.16806 shutdown 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 msgr2=0x55d9c1bed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bed410 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f11f8012070 tx=0x7f11f8009b70 comp rx=0 tx=0).stop 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 shutdown_connections 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d9c1c6a730 0x55d9c1c6cb50 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d9c1bf34f0 0x55d9c1bed410 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d9c1be4700 0x55d9c1beced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 --2- 172.21.15.90:0/2351963821 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d9c1be3de0 0x55d9c1bfb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 >> 172.21.15.90:0/2351963821 conn(0x55d9c1be0410 msgr2=0x55d9c1bdf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f11e9ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 shutdown_connections 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 1 -- 172.21.15.90:0/2351963821 wait complete. 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.772+0000 7f120b422740 20 client.16806 trim_cache size 0 max 16384 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:31:32.774 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:31:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11ec0a3dc0 msgr2=0x7f11ec0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11ec0a3dc0 0x7f11ec0a6280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11f0000c00 tx=0x7f11f0009610 comp rx=0 tx=0).stop 2024-04-26T18:31:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f120006b160 msgr2=0x7f1200199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f120006b160 0x7f1200199090 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00e9f0 tx=0x7f11fc00eeb0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 shutdown_connections 2024-04-26T18:31:32.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11ec0a3dc0 0x7f11ec0a6280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1200108980 0x7f120019d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f120006bb00 0x7f12001995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 --2- 172.21.15.90:0/39467450 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f120006b160 0x7f1200199090 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 >> 172.21.15.90:0/39467450 conn(0x7f1200077630 msgr2=0x7f12000ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 shutdown_connections 2024-04-26T18:31:32.776 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:32.773+0000 7f120862a640 1 -- 172.21.15.90:0/39467450 wait complete. 2024-04-26T18:31:32.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 33774 6.82 MB/sec execute 177 sec latency 15069.254 ms 2024-04-26T18:31:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:32 smithi119 ceph-mon[32752]: pgmap v676: 97 pgs: 97 active+clean; 607 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 70 op/s 2024-04-26T18:31:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:32 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:31:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:32 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:reconnect 2024-04-26T18:31:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:32 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:reconnect} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:32 smithi167 ceph-mon[32751]: pgmap v676: 97 pgs: 97 active+clean; 607 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 70 op/s 2024-04-26T18:31:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:32 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:31:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:32 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:reconnect 2024-04-26T18:31:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:32 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:reconnect} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:32 smithi090 ceph-mon[29180]: pgmap v676: 97 pgs: 97 active+clean; 607 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 131 KiB/s wr, 70 op/s 2024-04-26T18:31:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:32 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:31:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:32 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:reconnect 2024-04-26T18:31:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:32 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:reconnect} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:33.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34291 6.94 MB/sec execute 178 sec latency 16000.556 ms 2024-04-26T18:31:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:33 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:rejoin 2024-04-26T18:31:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:33 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:rejoin} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:33 smithi119 ceph-mon[32752]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:31:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:33 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:rejoin 2024-04-26T18:31:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:33 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:rejoin} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:33 smithi167 ceph-mon[32751]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:31:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:33 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:rejoin 2024-04-26T18:31:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:33 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=a=up:rejoin} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:33 smithi090 ceph-mon[29180]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:31:34.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 34764 7.02 MB/sec execute 179 sec latency 142.688 ms 2024-04-26T18:31:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:34 smithi119 ceph-mon[32752]: pgmap v677: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 826 KiB/s wr, 127 op/s 2024-04-26T18:31:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:34 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:31:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:active 2024-04-26T18:31:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:34 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:34 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 2024-04-26T18:31:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:34 smithi167 ceph-mon[32751]: pgmap v677: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 826 KiB/s wr, 127 op/s 2024-04-26T18:31:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:34 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:31:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:active 2024-04-26T18:31:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:34 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:34 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 2024-04-26T18:31:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:34 smithi090 ceph-mon[29180]: pgmap v677: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 826 KiB/s wr, 127 op/s 2024-04-26T18:31:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:34 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:31:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/4072394290,v1:172.21.15.90:6835/4072394290] up:active 2024-04-26T18:31:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:34 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:34 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 2024-04-26T18:31:35.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35244 7.11 MB/sec execute 180 sec latency 778.220 ms 2024-04-26T18:31:36.895 INFO:tasks.workunit.client.0.smithi090.stdout: 10 35875 7.30 MB/sec execute 181 sec latency 1778.265 ms 2024-04-26T18:31:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:36 smithi119 ceph-mon[32752]: pgmap v678: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 826 KiB/s wr, 127 op/s 2024-04-26T18:31:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:36 smithi167 ceph-mon[32751]: pgmap v678: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 826 KiB/s wr, 127 op/s 2024-04-26T18:31:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:36 smithi090 ceph-mon[29180]: pgmap v678: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 826 KiB/s wr, 127 op/s 2024-04-26T18:31:37.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36285 7.40 MB/sec execute 182 sec latency 2778.305 ms 2024-04-26T18:31:38.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.45 MB/sec execute 183 sec latency 3778.339 ms 2024-04-26T18:31:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:38 smithi119 ceph-mon[32752]: pgmap v679: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 662 KiB/s wr, 101 op/s 2024-04-26T18:31:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:38 smithi167 ceph-mon[32751]: pgmap v679: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 662 KiB/s wr, 101 op/s 2024-04-26T18:31:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-mon[29180]: pgmap v679: 97 pgs: 97 active+clean; 604 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 662 KiB/s wr, 101 op/s 2024-04-26T18:31:39.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f520f008700 time 2024-04-26T18:31:38.840598+0000 2024-04-26T18:31:39.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:39.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52196313dd] 2024-04-26T18:31:39.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.126 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: clone() 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:38.841+0000 7f520f008700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f520f008700 time 2024-04-26T18:31:38.840598+0000 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52196313dd] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.129 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: clone() 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: *** Caught signal (Aborted) ** 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: in thread 7f520f008700 thread_name:ms_dispatch 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52183b9d20] 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: gsignal() 2024-04-26T18:31:39.130 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: abort() 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5219631437] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 16: clone() 2024-04-26T18:31:39.131 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:38.844+0000 7f520f008700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: in thread 7f520f008700 thread_name:ms_dispatch 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52183b9d20] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: gsignal() 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: abort() 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5219631437] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.132 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 16: clone() 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: -1> 2024-04-26T18:31:38.841+0000 7f520f008700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f520f008700 time 2024-04-26T18:31:38.840598+0000 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52196313dd] 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.133 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: clone() 2024-04-26T18:31:39.134 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 0> 2024-04-26T18:31:38.844+0000 7f520f008700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: in thread 7f520f008700 thread_name:ms_dispatch 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52183b9d20] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: gsignal() 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: abort() 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5219631437] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.135 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 16: clone() 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: -1> 2024-04-26T18:31:38.841+0000 7f520f008700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f520f008700 time 2024-04-26T18:31:38.840598+0000 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f52196313dd] 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.136 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: clone() 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.137 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 0> 2024-04-26T18:31:38.844+0000 7f520f008700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: in thread 7f520f008700 thread_name:ms_dispatch 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f52183b9d20] 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2: gsignal() 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 3: abort() 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f5219631437] 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f52196315a3] 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559130c07822] 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559130b77598] 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559130b79c17] 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559130a72d0b] 2024-04-26T18:31:39.138 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559130a732ac] 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559130a5bfad] 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f52198c5e20] 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 13: (DispatchQueue::entry()+0x51f) [0x7f52198c2fdf] 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f5219995fc1] 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f52183af1ca] 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 16: clone() 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[98656]: 2024-04-26T18:31:39.139 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:38 smithi090 podman[99153]: 2024-04-26 18:31:38.926609866 +0000 UTC m=+0.022971022 container died a8156d426adfc1bffeac73c09deeb1c526615cbe667533be86947c167b91ce91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-04-26T18:31:39.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.41 MB/sec execute 184 sec latency 4778.410 ms 2024-04-26T18:31:40.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.37 MB/sec execute 185 sec latency 5778.532 ms 2024-04-26T18:31:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:40 smithi119 ceph-mon[32752]: pgmap v680: 97 pgs: 97 active+clean; 633 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.7 MiB/s wr, 281 op/s 2024-04-26T18:31:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:40 smithi167 ceph-mon[32751]: pgmap v680: 97 pgs: 97 active+clean; 633 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.7 MiB/s wr, 281 op/s 2024-04-26T18:31:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:40 smithi090 ceph-mon[29180]: pgmap v680: 97 pgs: 97 active+clean; 633 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.7 MiB/s wr, 281 op/s 2024-04-26T18:31:41.886 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:41 smithi090 podman[99153]: 2024-04-26 18:31:41.628286682 +0000 UTC m=+2.724647870 container remove a8156d426adfc1bffeac73c09deeb1c526615cbe667533be86947c167b91ce91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-26T18:31:41.886 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:41 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:31:41.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.33 MB/sec execute 186 sec latency 6778.578 ms 2024-04-26T18:31:42.375 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:41 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-04-26T18:31:42.376 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:41 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 16.021s CPU time. 2024-04-26T18:31:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:42 smithi090 ceph-mon[29180]: pgmap v681: 97 pgs: 97 active+clean; 633 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-26T18:31:42.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.29 MB/sec execute 187 sec latency 7778.664 ms 2024-04-26T18:31:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:42 smithi119 ceph-mon[32752]: pgmap v681: 97 pgs: 97 active+clean; 633 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-26T18:31:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:42 smithi167 ceph-mon[32751]: pgmap v681: 97 pgs: 97 active+clean; 633 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-04-26T18:31:43.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.25 MB/sec execute 188 sec latency 8778.750 ms 2024-04-26T18:31:44.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.21 MB/sec execute 189 sec latency 9778.799 ms 2024-04-26T18:31:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:44 smithi119 ceph-mon[32752]: pgmap v682: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-04-26T18:31:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:44 smithi167 ceph-mon[32751]: pgmap v682: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-04-26T18:31:45.085 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:44 smithi090 ceph-mon[29180]: pgmap v682: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.4 MiB/s wr, 260 op/s 2024-04-26T18:31:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 -- 172.21.15.90:0/1864249028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 msgr2=0x7fd12810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/1864249028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12810f3e0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd120009ca0 tx=0x7fd12002a950 comp rx=0 tx=0).stop 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 -- 172.21.15.90:0/1864249028 shutdown_connections 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/1864249028 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12810f3e0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/1864249028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd128104500 0x7fd12810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/1864249028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd128103be0 0x7fd128103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 -- 172.21.15.90:0/1864249028 >> 172.21.15.90:0/1864249028 conn(0x7fd128077bb0 msgr2=0x7fd128077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 -- 172.21.15.90:0/1864249028 shutdown_connections 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 -- 172.21.15.90:0/1864249028 wait complete. 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 Processor -- start 2024-04-26T18:31:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.862+0000 7fd12d8d1640 1 -- start start 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd12d8d1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd128103be0 0x7fd128198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd12d8d1640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd128104500 0x7fd1281994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd12d8d1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd12d8d1640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd128112bf0 con 0x7fd12810cfb0 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd12d8d1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd128112a70 con 0x7fd128103be0 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd12d8d1640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd128112d70 con 0x7fd128104500 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd128103be0 0x7fd128198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33254/0 (socket says 172.21.15.90:33254) 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 -- 172.21.15.90:0/401368039 learned_addr learned my addr 172.21.15.90:0/401368039 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd128104500 0x7fd1281994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 -- 172.21.15.90:0/401368039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd128104500 msgr2=0x7fd1281994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd128104500 0x7fd1281994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 -- 172.21.15.90:0/401368039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd128103be0 msgr2=0x7fd128198f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd128103be0 0x7fd128198f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 -- 172.21.15.90:0/401368039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12819df60 con 0x7fd12810cfb0 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd128104500 0x7fd1281994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd126ffd640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd128103be0 0x7fd128198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd1277fe640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12819d830 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd120004af0 tx=0x7fd12002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.863+0000 7fd12c8cf640 1 -- 172.21.15.90:0/401368039 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120034040 con 0x7fd12810cfb0 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.864+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12819e1f0 con 0x7fd12810cfb0 2024-04-26T18:31:45.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.864+0000 7fd12c8cf640 1 -- 172.21.15.90:0/401368039 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd12002fe80 con 0x7fd12810cfb0 2024-04-26T18:31:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.864+0000 7fd12c8cf640 1 -- 172.21.15.90:0/401368039 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd120004670 con 0x7fd12810cfb0 2024-04-26T18:31:45.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.864+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a5ae0 con 0x7fd12810cfb0 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd12c8cf640 1 -- 172.21.15.90:0/401368039 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fd120037020 con 0x7fd12810cfb0 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 Processor -- start 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd12c8cf640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd1140a3dc0 0x7fd1140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 -- start start 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb69fd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd1140a3dc0 0x7fd1140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 0x55acb6a07ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55acb6963460 con 0x55acb69fcf00 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55acb691d590 con 0x55acb6a080e0 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd1305ec740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55acb695f420 con 0x55acb69fd820 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd12c8cf640 1 -- 172.21.15.90:0/401368039 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 12109+0+0 (secure 0 0 0) 0x7fd1200f2360 con 0x7fd12810cfb0 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb69fd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb69fd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33260/0 (socket says 172.21.15.90:33260) 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.866+0000 7fd126ffd640 1 -- 172.21.15.90:0/2853323074 learned_addr learned my addr 172.21.15.90:0/2853323074 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd1277fe640 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd126ffd640 1 -- 172.21.15.90:0/2853323074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 msgr2=0x55acb6a07ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd126ffd640 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 0x55acb6a07ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd126ffd640 1 -- 172.21.15.90:0/2853323074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 msgr2=0x55acb6a0a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd1140a3dc0 0x7fd1140a6280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd128100f50 tx=0x7fd11000be80 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd126ffd640 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd126ffd640 1 -- 172.21.15.90:0/2853323074 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb692a0d0 con 0x55acb69fcf00 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd126ffd640 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb69fd2e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd1180115f0 tx=0x7fd118011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8c2932399c9549 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd11dffb640 1 -- 172.21.15.90:0/2853323074 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11801e070 con 0x55acb69fcf00 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd11dffb640 1 -- 172.21.15.90:0/2853323074 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11800faf0 con 0x55acb69fcf00 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.867+0000 7fd11dffb640 1 -- 172.21.15.90:0/2853323074 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118019ac0 con 0x55acb69fcf00 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 -- 172.21.15.90:0/2853323074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 msgr2=0x55acb69fd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb69fd2e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fd1180115f0 tx=0x7fd118011ab0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 -- 172.21.15.90:0/2853323074 shutdown_connections 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 0x55acb6a07ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2853323074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb69fd2e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 -- 172.21.15.90:0/2853323074 >> 172.21.15.90:0/2853323074 conn(0x55acb69f9410 msgr2=0x55acb69f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 -- 172.21.15.90:0/2853323074 shutdown_connections 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 -- 172.21.15.90:0/2853323074 wait complete. 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 -- start start 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 Processor -- start 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.868+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb69fcf00 0x55acb6a05010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb6a080e0 0x55acb6a05a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55acb6a0ad10 con 0x55acb69fd820 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55acb68eab60 con 0x55acb69fcf00 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55acb6a0ae90 con 0x55acb6a080e0 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 10 client.0 _ll_register_callbacks cb 0x55acb690f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a05550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33262/0 (socket says 172.21.15.90:33262) 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 -- 172.21.15.90:0/2410895506 learned_addr learned my addr 172.21.15.90:0/2410895506 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd126ffd640 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb69fcf00 0x55acb6a05010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 -- 172.21.15.90:0/2410895506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb6a080e0 msgr2=0x55acb6a05a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb6a080e0 0x55acb6a05a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 -- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb69fcf00 msgr2=0x55acb6a05010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb69fcf00 0x55acb6a05010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 -- 172.21.15.90:0/2410895506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb6675210 con 0x55acb69fd820 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a05550 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd110011990 tx=0x7fd110004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee46827af52a6f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd106ffd640 1 -- 172.21.15.90:0/2410895506 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110021070 con 0x55acb69fd820 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd11800faf0 con 0x55acb69fd820 2024-04-26T18:31:45.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd106ffd640 1 -- 172.21.15.90:0/2410895506 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 12109+0+0 (secure 0 0 0) 0x7fd110010c00 con 0x55acb69fd820 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd106ffd640 1 -- 172.21.15.90:0/2410895506 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=272) ==== 24+0+0 (secure 0 0 0) 0x7fd110010e00 con 0x55acb69fd820 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 10 client.16824 fetch_fsmap learned FSMap version 272 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.869+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=272,monmap=4+}) -- 0x55acb6741620 con 0x55acb69fd820 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd106ffd640 1 -- 172.21.15.90:0/2410895506 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd11800faf0 con 0x55acb69fd820 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd106ffd640 1 -- 172.21.15.90:0/2410895506 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 272) ==== 4716+0+0 (secure 0 0 0) 0x7fd11002b6d0 con 0x55acb69fd820 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd1305ec740 10 client.16824 fetch_fsmap finished waiting for FSMap version 272 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd1305ec740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd1305ec740 20 client.16824 populate_metadata read hostname 'smithi090' 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a18740 0x55acb6a1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd1305ec740 4 client.16824 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb6971070 con 0x55acb6a18740 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd126ffd640 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a18740 0x55acb6a1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd106ffd640 10 client.16824 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.870+0000 7fd126ffd640 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a18740 0x55acb6a1ac00 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2bb303693cb2b7d server_cookie=6b092cef22a7001f in_seq=0 out_seq=0 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd106ffd640 1 -- 172.21.15.90:0/2410895506 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acb6971070 con 0x55acb6a18740 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd106ffd640 10 client.16824 handle_command_reply: tid=0 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 client.16824 shutdown 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 msgr2=0x55acb6a05550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a05550 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd110011990 tx=0x7fd110004730 comp rx=0 tx=0).stop 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 shutdown_connections 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a18740 0x55acb6a1ac00 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb6a080e0 0x55acb6a05a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd106ffd640 0 client.16824 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a05550 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 --2- 172.21.15.90:0/2410895506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb69fcf00 0x55acb6a05010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 >> 172.21.15.90:0/2410895506 conn(0x55acb69f9410 msgr2=0x55acb69ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 shutdown_connections 2024-04-26T18:31:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 1 -- 172.21.15.90:0/2410895506 wait complete. 2024-04-26T18:31:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.871+0000 7fd1305ec740 20 client.16824 trim_cache size 0 max 16384 2024-04-26T18:31:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 Processor -- start 2024-04-26T18:31:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 -- start start 2024-04-26T18:31:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb6a13bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 0x55acb6a204d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281b0e00 con 0x55acb69fcf00 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281aeb90 con 0x55acb6a080e0 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1305ec740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281b0730 con 0x55acb69fd820 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb6a13bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb6a13bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33270/0 (socket says 172.21.15.90:33270) 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 -- 172.21.15.90:0/645212339 learned_addr learned my addr 172.21.15.90:0/645212339 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd1277fe640 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a20a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 0x55acb6a204d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 -- 172.21.15.90:0/645212339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 msgr2=0x55acb6a204d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 0x55acb6a204d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 -- 172.21.15.90:0/645212339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 msgr2=0x55acb6a20a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a20a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 -- 172.21.15.90:0/645212339 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb6a16d90 con 0x55acb69fcf00 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.886+0000 7fd126ffd640 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb6a13bd0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd118019710 tx=0x7fd118019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9758b9a1a6a5f59e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd11d7fa640 1 -- 172.21.15.90:0/645212339 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11802f070 con 0x55acb69fcf00 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd11d7fa640 1 -- 172.21.15.90:0/645212339 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd118005560 con 0x55acb69fcf00 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd11d7fa640 1 -- 172.21.15.90:0/645212339 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11801e040 con 0x55acb69fcf00 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 -- 172.21.15.90:0/645212339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 msgr2=0x55acb6a13bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb6a13bd0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd118019710 tx=0x7fd118019600 comp rx=0 tx=0).stop 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 -- 172.21.15.90:0/645212339 shutdown_connections 2024-04-26T18:31:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a20a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fd820 0x55acb6a204d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 --2- 172.21.15.90:0/645212339 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fcf00 0x55acb6a13bd0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 -- 172.21.15.90:0/645212339 >> 172.21.15.90:0/645212339 conn(0x55acb69f9410 msgr2=0x55acb69f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 -- 172.21.15.90:0/645212339 shutdown_connections 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.887+0000 7fd1305ec740 1 -- 172.21.15.90:0/645212339 wait complete. 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 -- start start 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 Processor -- start 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fcf00 0x55acb6a101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a0b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281af930 con 0x55acb69fd820 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281af260 con 0x55acb6a080e0 2024-04-26T18:31:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd1281b0000 con 0x55acb69fcf00 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1305ec740 10 client.0 _ll_register_callbacks cb 0x7fd110015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a0b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a0b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33284/0 (socket says 172.21.15.90:33284) 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 -- 172.21.15.90:0/703815302 learned_addr learned my addr 172.21.15.90:0/703815302 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd126ffd640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fcf00 0x55acb6a101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd1277fe640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 -- 172.21.15.90:0/703815302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fcf00 msgr2=0x55acb6a101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fcf00 0x55acb6a101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 -- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 msgr2=0x55acb6a0bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 -- 172.21.15.90:0/703815302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb6a0c120 con 0x55acb69fd820 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd126ffd640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fcf00 0x55acb6a101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd11e7fc640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a0b6a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd11002bd30 tx=0x7fd11000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36690827798d25d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.888+0000 7fd106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd106ffd640 1 -- 172.21.15.90:0/703815302 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110021040 con 0x55acb69fd820 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd106ffd640 1 -- 172.21.15.90:0/703815302 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(164..164 src has 1..164) ==== 12109+0+0 (secure 0 0 0) 0x7fd110004bb0 con 0x55acb69fd820 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd118005560 con 0x55acb69fd820 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd106ffd640 1 -- 172.21.15.90:0/703815302 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=272) ==== 24+0+0 (secure 0 0 0) 0x7fd110029780 con 0x55acb69fd820 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 10 client.16830 fetch_fsmap learned FSMap version 272 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=272,monmap=4+}) -- 0x55acb6a7eaa0 con 0x55acb69fd820 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd106ffd640 1 -- 172.21.15.90:0/703815302 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd118005560 con 0x55acb69fd820 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd106ffd640 1 -- 172.21.15.90:0/703815302 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 272) ==== 4716+0+0 (secure 0 0 0) 0x7fd11003a030 con 0x55acb69fd820 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 10 client.16830 fetch_fsmap finished waiting for FSMap version 272 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 20 client.16830 populate_metadata read hostname 'smithi090' 2024-04-26T18:31:45.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a7f220 0x55acb6a81640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 4 client.16830 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:31:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acb696aee0 con 0x55acb6a7f220 2024-04-26T18:31:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.889+0000 7fd126ffd640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a7f220 0x55acb6a81640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.890+0000 7fd126ffd640 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a7f220 0x55acb6a81640 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9ac7b88cf4db4e server_cookie=667c54103ef62dfe in_seq=0 out_seq=0 2024-04-26T18:31:45.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:31:45.890+0000 7fd106ffd640 10 client.16830 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:31:45.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.17 MB/sec execute 190 sec latency 10778.835 ms 2024-04-26T18:31:46.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.13 MB/sec execute 191 sec latency 11778.922 ms 2024-04-26T18:31:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:46 smithi119 ceph-mon[32752]: pgmap v683: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-04-26T18:31:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:46 smithi167 ceph-mon[32751]: pgmap v683: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-04-26T18:31:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:46 smithi090 ceph-mon[29180]: pgmap v683: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-04-26T18:31:47.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.10 MB/sec execute 192 sec latency 12778.989 ms 2024-04-26T18:31:48.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.06 MB/sec execute 193 sec latency 13779.100 ms 2024-04-26T18:31:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:48 smithi119 ceph-mon[32752]: pgmap v684: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-04-26T18:31:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:48 smithi167 ceph-mon[32751]: pgmap v684: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-04-26T18:31:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:48 smithi090 ceph-mon[29180]: pgmap v684: 97 pgs: 97 active+clean; 636 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.9 MiB/s wr, 223 op/s 2024-04-26T18:31:49.896 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 7.02 MB/sec execute 194 sec latency 14779.180 ms 2024-04-26T18:31:50.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.99 MB/sec execute 195 sec latency 15779.317 ms 2024-04-26T18:31:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:50 smithi119 ceph-mon[32752]: pgmap v685: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-04-26T18:31:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:50 smithi167 ceph-mon[32751]: pgmap v685: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-04-26T18:31:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:50 smithi090 ceph-mon[29180]: pgmap v685: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 6.6 MiB/s wr, 351 op/s 2024-04-26T18:31:51.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.95 MB/sec execute 196 sec latency 16779.422 ms 2024-04-26T18:31:52.549 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:52 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:31:52.549 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:52 smithi090 systemd[1]: Stopped Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:31:52.549 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:52 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 16.021s CPU time. 2024-04-26T18:31:52.549 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:52 smithi090 systemd[1]: Starting Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:31:52.550 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:52 smithi090 podman[99308]: 2024-04-26 18:31:52.276082599 +0000 UTC m=+0.019630775 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:31:52.550 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f877ce98700 time 2024-04-26T18:31:52.205864+0000 2024-04-26T18:31:52.550 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:52.550 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.550 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f87874c13dd] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: clone() 2024-04-26T18:31:52.551 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: *** Caught signal (Aborted) ** 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: in thread 7f877ce98700 thread_name:ms_dispatch 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.206+0000 7f877ce98700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f877ce98700 time 2024-04-26T18:31:52.205864+0000 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f87874c13dd] 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.552 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: clone() 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8786249d20] 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: gsignal() 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: abort() 2024-04-26T18:31:52.553 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f87874c1437] 2024-04-26T18:31:52.554 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.554 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.554 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.554 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.554 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.555 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 16: clone() 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.207+0000 7f877ce98700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: in thread 7f877ce98700 thread_name:ms_dispatch 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8786249d20] 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: gsignal() 2024-04-26T18:31:52.556 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: abort() 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f87874c1437] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.557 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 16: clone() 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: -1> 2024-04-26T18:31:52.206+0000 7f877ce98700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f877ce98700 time 2024-04-26T18:31:52.205864+0000 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f87874c13dd] 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.558 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: clone() 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 0> 2024-04-26T18:31:52.207+0000 7f877ce98700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: in thread 7f877ce98700 thread_name:ms_dispatch 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.559 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8786249d20] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: gsignal() 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: abort() 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f87874c1437] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.561 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 16: clone() 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: -1> 2024-04-26T18:31:52.206+0000 7f877ce98700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f877ce98700 time 2024-04-26T18:31:52.205864+0000 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f87874c13dd] 2024-04-26T18:31:52.562 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: clone() 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 0> 2024-04-26T18:31:52.207+0000 7f877ce98700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:31:52.563 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: in thread 7f877ce98700 thread_name:ms_dispatch 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f8786249d20] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2: gsignal() 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 3: abort() 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f87874c1437] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f87874c15a3] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x564f9dc0f822] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x564f9db7f598] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x564f9db81c17] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x564f9da7ad0b] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x564f9da7b2ac] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x564f9da63fad] 2024-04-26T18:31:52.564 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f8787755e20] 2024-04-26T18:31:52.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 13: (DispatchQueue::entry()+0x51f) [0x7f8787752fdf] 2024-04-26T18:31:52.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f8787825fc1] 2024-04-26T18:31:52.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f878623f1ca] 2024-04-26T18:31:52.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 16: clone() 2024-04-26T18:31:52.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:31:52.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[98078]: 2024-04-26T18:31:52.565 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 podman[99309]: 2024-04-26 18:31:52.278159956 +0000 UTC m=+0.019616769 container died 5567dc654d3e70507b69040a911441c9422223824681d05c06e0bfed79976208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux ) 2024-04-26T18:31:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-mon[29180]: pgmap v686: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-04-26T18:31:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-mon[29180]: Replacing daemon mds.a as rank 2 with standby daemon mds.d 2024-04-26T18:31:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-mon[29180]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:31:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:31:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-mon[29180]: osdmap e165: 12 total, 12 up, 12 in 2024-04-26T18:31:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:52 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:31:52.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.92 MB/sec execute 197 sec latency 17779.503 ms 2024-04-26T18:31:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:52 smithi119 ceph-mon[32752]: pgmap v686: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-04-26T18:31:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:52 smithi119 ceph-mon[32752]: Replacing daemon mds.a as rank 2 with standby daemon mds.d 2024-04-26T18:31:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:52 smithi119 ceph-mon[32752]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:31:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:52 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:31:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:52 smithi119 ceph-mon[32752]: osdmap e165: 12 total, 12 up, 12 in 2024-04-26T18:31:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:52 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:31:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:52 smithi167 ceph-mon[32751]: pgmap v686: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 154 op/s 2024-04-26T18:31:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:52 smithi167 ceph-mon[32751]: Replacing daemon mds.a as rank 2 with standby daemon mds.d 2024-04-26T18:31:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:52 smithi167 ceph-mon[32751]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:31:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:52 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:31:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:52 smithi167 ceph-mon[32751]: osdmap e165: 12 total, 12 up, 12 in 2024-04-26T18:31:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:52 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:31:53.268 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:53 smithi090 podman[99308]: 2024-04-26 18:31:53.146588694 +0000 UTC m=+0.890136849 container create a612cacab2d8088069a242a7dc5d887ce3796896983e26ca077b022b975da7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T18:31:53.268 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 podman[99309]: 2024-04-26 18:31:52.970919172 +0000 UTC m=+0.712375965 container remove 5567dc654d3e70507b69040a911441c9422223824681d05c06e0bfed79976208 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:31:53.268 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:52 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:31:53.607 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:53 smithi090 podman[99308]: 2024-04-26 18:31:53.417885454 +0000 UTC m=+1.161433612 container init a612cacab2d8088069a242a7dc5d887ce3796896983e26ca077b022b975da7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-26T18:31:53.608 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:53 smithi090 podman[99308]: 2024-04-26 18:31:53.420524153 +0000 UTC m=+1.164072310 container start a612cacab2d8088069a242a7dc5d887ce3796896983e26ca077b022b975da7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:31:53.608 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:53 smithi090 ceph-mds[99368]: -- 172.21.15.90:0/3672560578 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a309f9a40 con 0x557a309cc800 2024-04-26T18:31:53.608 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:53 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: starting mds.a at 2024-04-26T18:31:53.608 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:53 smithi090 bash[99308]: a612cacab2d8088069a242a7dc5d887ce3796896983e26ca077b022b975da7bb 2024-04-26T18:31:53.608 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:31:53 smithi090 systemd[1]: Started Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:31:53.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.88 MB/sec execute 198 sec latency 18779.561 ms 2024-04-26T18:31:54.125 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:53 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Failed with result 'exit-code'. 2024-04-26T18:31:54.126 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:31:53 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 20.698s CPU time. 2024-04-26T18:31:54.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.85 MB/sec execute 199 sec latency 19779.626 ms 2024-04-26T18:31:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:54 smithi119 ceph-mon[32752]: pgmap v688: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:54 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:resolve 2024-04-26T18:31:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:54 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:boot 2024-04-26T18:31:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:54 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:31:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:54 smithi167 ceph-mon[32751]: pgmap v688: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:54 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:resolve 2024-04-26T18:31:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:54 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:boot 2024-04-26T18:31:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:54 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:31:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:54 smithi090 ceph-mon[29180]: pgmap v688: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:54 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:resolve 2024-04-26T18:31:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:54 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:boot 2024-04-26T18:31:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:54 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:resolve} 2 up:standby-replay 1 up:standby 2024-04-26T18:31:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:31:55.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.81 MB/sec execute 200 sec latency 20779.720 ms 2024-04-26T18:31:56.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.78 MB/sec execute 201 sec latency 21779.861 ms 2024-04-26T18:31:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:56 smithi119 ceph-mon[32752]: pgmap v689: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:56 smithi167 ceph-mon[32751]: pgmap v689: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:56 smithi090 ceph-mon[29180]: pgmap v689: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:57.897 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.75 MB/sec execute 202 sec latency 22779.917 ms 2024-04-26T18:31:58.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.71 MB/sec execute 203 sec latency 23780.034 ms 2024-04-26T18:31:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:31:58 smithi119 ceph-mon[32752]: pgmap v690: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:31:58 smithi167 ceph-mon[32751]: pgmap v690: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:31:58 smithi090 ceph-mon[29180]: pgmap v690: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 396 KiB/s rd, 4.5 MiB/s wr, 161 op/s 2024-04-26T18:31:59.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.68 MB/sec execute 204 sec latency 24780.134 ms 2024-04-26T18:32:00.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.65 MB/sec execute 205 sec latency 25780.216 ms 2024-04-26T18:32:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:00 smithi119 ceph-mon[32752]: pgmap v691: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-26T18:32:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:00 smithi167 ceph-mon[32751]: pgmap v691: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-26T18:32:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:00 smithi090 ceph-mon[29180]: pgmap v691: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-26T18:32:01.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.61 MB/sec execute 206 sec latency 26780.288 ms 2024-04-26T18:32:02.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.58 MB/sec execute 207 sec latency 27780.393 ms 2024-04-26T18:32:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:02 smithi119 ceph-mon[32752]: pgmap v692: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-26T18:32:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:02 smithi167 ceph-mon[32751]: pgmap v692: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-26T18:32:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:02 smithi090 ceph-mon[29180]: pgmap v692: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 9 op/s 2024-04-26T18:32:03.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.55 MB/sec execute 208 sec latency 28780.444 ms 2024-04-26T18:32:04.127 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:03 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:32:04.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:03 smithi090 systemd[1]: Stopped Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:32:04.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:03 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 20.698s CPU time. 2024-04-26T18:32:04.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:03 smithi090 systemd[1]: Starting Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:32:04.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 podman[99447]: 2024-04-26 18:32:04.015944433 +0000 UTC m=+0.018230908 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:32:04.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 podman[99447]: 2024-04-26 18:32:04.128493051 +0000 UTC m=+0.130779528 container create 139db8389c6f668d8f1f60932454a0092b472369b35b2388f20c29b4f8a2ee0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T18:32:04.390 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 podman[99447]: 2024-04-26 18:32:04.364447462 +0000 UTC m=+0.366733938 container init 139db8389c6f668d8f1f60932454a0092b472369b35b2388f20c29b4f8a2ee0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T18:32:04.390 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 podman[99447]: 2024-04-26 18:32:04.367664757 +0000 UTC m=+0.369951237 container start 139db8389c6f668d8f1f60932454a0092b472369b35b2388f20c29b4f8a2ee0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T18:32:04.734 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mds[99466]: -- 172.21.15.90:0/2891566663 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560e01c8ba40 con 0x560e01c5f000 2024-04-26T18:32:04.734 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: starting mds.g at 2024-04-26T18:32:04.734 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 bash[99447]: 139db8389c6f668d8f1f60932454a0092b472369b35b2388f20c29b4f8a2ee0f 2024-04-26T18:32:04.734 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:04 smithi090 systemd[1]: Started Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:32:04.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.52 MB/sec execute 209 sec latency 29780.559 ms 2024-04-26T18:32:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: pgmap v693: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-26T18:32:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: daemon mds.g restarted 2024-04-26T18:32:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: osdmap e166: 12 total, 12 up, 12 in 2024-04-26T18:32:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:boot 2024-04-26T18:32:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-04-26T18:32:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:resolve} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:32:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:32:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: pgmap v693: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: daemon mds.g restarted 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: osdmap e166: 12 total, 12 up, 12 in 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:boot 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:resolve} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:32:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: pgmap v693: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 29 KiB/s wr, 12 op/s 2024-04-26T18:32:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: daemon mds.g restarted 2024-04-26T18:32:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: osdmap e166: 12 total, 12 up, 12 in 2024-04-26T18:32:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:boot 2024-04-26T18:32:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-04-26T18:32:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:resolve} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:32:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:32:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:05.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.49 MB/sec execute 210 sec latency 30780.655 ms 2024-04-26T18:32:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:resolve 2024-04-26T18:32:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:06 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:06 smithi119 ceph-mon[32752]: pgmap v695: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T18:32:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:resolve 2024-04-26T18:32:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:06 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:06 smithi167 ceph-mon[32751]: pgmap v695: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T18:32:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:resolve 2024-04-26T18:32:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:06 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:06 smithi090 ceph-mon[29180]: pgmap v695: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T18:32:06.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36664 6.46 MB/sec execute 211 sec latency 31780.771 ms 2024-04-26T18:32:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd106ffd640 1 -- 172.21.15.90:0/703815302 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acb696aee0 con 0x55acb6a7f220 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd106ffd640 10 client.16830 handle_command_reply: tid=0 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 client.16830 shutdown 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 msgr2=0x55acb6a0b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a0b6a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd11002bd30 tx=0x7fd11000da30 comp rx=0 tx=0).stop 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 shutdown_connections 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55acb6a7f220 0x55acb6a81640 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55acb6a080e0 0x55acb6a0bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55acb69fd820 0x55acb6a0b6a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd106ffd640 0 client.16830 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 --2- 172.21.15.90:0/703815302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55acb69fcf00 0x55acb6a101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 >> 172.21.15.90:0/703815302 conn(0x55acb69f9410 msgr2=0x55acb6a04fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 shutdown_connections 2024-04-26T18:32:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 1 -- 172.21.15.90:0/703815302 wait complete. 2024-04-26T18:32:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.500+0000 7fd1305ec740 20 client.16830 trim_cache size 0 max 16384 2024-04-26T18:32:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:32:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:32:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:32:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd1140a3dc0 msgr2=0x7fd1140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd1140a3dc0 0x7fd1140a6280 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd128100f50 tx=0x7fd11000be80 comp rx=0 tx=0).stop 2024-04-26T18:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 msgr2=0x7fd12819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12819d830 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd120004af0 tx=0x7fd12002f5d0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 shutdown_connections 2024-04-26T18:32:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fd1140a3dc0 0x7fd1140a6280 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd12810cfb0 0x7fd12819d830 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd128104500 0x7fd1281994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 --2- 172.21.15.90:0/401368039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd128103be0 0x7fd128198f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 >> 172.21.15.90:0/401368039 conn(0x7fd128077bb0 msgr2=0x7fd1280fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 shutdown_connections 2024-04-26T18:32:07.504 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:07.501+0000 7fd12d8d1640 1 -- 172.21.15.90:0/401368039 wait complete. 2024-04-26T18:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:07 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:reconnect 2024-04-26T18:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:reconnect 2024-04-26T18:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:07 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:07 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:07 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:32:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:07 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:32:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:reconnect 2024-04-26T18:32:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:reconnect 2024-04-26T18:32:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:07 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:07 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:32:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:07 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:07 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:reconnect 2024-04-26T18:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:reconnect 2024-04-26T18:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:07 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:07 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:07 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:32:07.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 36808 6.44 MB/sec execute 212 sec latency 32321.284 ms 2024-04-26T18:32:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:rejoin 2024-04-26T18:32:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:rejoin 2024-04-26T18:32:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:08 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:08 smithi119 ceph-mon[32752]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:32:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:08 smithi119 ceph-mon[32752]: pgmap v696: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T18:32:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:rejoin 2024-04-26T18:32:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:rejoin 2024-04-26T18:32:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:08 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:08 smithi167 ceph-mon[32751]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:32:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:08 smithi167 ceph-mon[32751]: pgmap v696: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T18:32:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:rejoin 2024-04-26T18:32:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:rejoin 2024-04-26T18:32:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:08 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:08 smithi090 ceph-mon[29180]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:32:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:08 smithi090 ceph-mon[29180]: pgmap v696: 97 pgs: 97 active+clean; 651 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-26T18:32:08.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37201 6.52 MB/sec execute 213 sec latency 30795.857 ms 2024-04-26T18:32:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:09 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:clientreplay 2024-04-26T18:32:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:09 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:active 2024-04-26T18:32:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:09 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:clientreplay} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:09 smithi119 ceph-mon[32752]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:32:09.817 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:32:09 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:32:09.444+0000 7f61a8296700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 2 2024-04-26T18:32:09.817 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:32:09 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:32:09.447+0000 7f61a4a8f700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 11 2024-04-26T18:32:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:09 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:clientreplay 2024-04-26T18:32:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:09 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:active 2024-04-26T18:32:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:09 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:clientreplay} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:09 smithi167 ceph-mon[32751]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:32:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:09 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:clientreplay 2024-04-26T18:32:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:09 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/349095443,v1:172.21.15.90:6841/349095443] up:active 2024-04-26T18:32:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:09 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:clientreplay} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:09 smithi090 ceph-mon[29180]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:32:09.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.64 MB/sec execute 214 sec latency 31379.903 ms 2024-04-26T18:32:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:10 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:32:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:10 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:active 2024-04-26T18:32:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:10 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:10 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:32:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:10 smithi119 ceph-mon[32752]: pgmap v697: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 221 KiB/s wr, 32 op/s 2024-04-26T18:32:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:10 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:32:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:10 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:active 2024-04-26T18:32:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:10 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:10 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:32:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:10 smithi167 ceph-mon[32751]: pgmap v697: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 221 KiB/s wr, 32 op/s 2024-04-26T18:32:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:10 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:32:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:10 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/3651305274,v1:172.21.15.90:6837/3651305274] up:active 2024-04-26T18:32:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:10 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:10 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 3 up:standby-replay 2024-04-26T18:32:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:10 smithi090 ceph-mon[29180]: pgmap v697: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 221 KiB/s wr, 32 op/s 2024-04-26T18:32:10.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.61 MB/sec execute 215 sec latency 1241.028 ms 2024-04-26T18:32:11.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.58 MB/sec execute 216 sec latency 2241.062 ms 2024-04-26T18:32:12.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.55 MB/sec execute 217 sec latency 3241.096 ms 2024-04-26T18:32:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:12 smithi119 ceph-mon[32752]: pgmap v698: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 221 KiB/s wr, 32 op/s 2024-04-26T18:32:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:12 smithi167 ceph-mon[32751]: pgmap v698: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 221 KiB/s wr, 32 op/s 2024-04-26T18:32:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:12 smithi090 ceph-mon[29180]: pgmap v698: 97 pgs: 97 active+clean; 645 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 221 KiB/s wr, 32 op/s 2024-04-26T18:32:13.621 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe86b6ca700 time 2024-04-26T18:32:13.617905+0000 2024-04-26T18:32:13.621 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.621 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.621 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe875cf33dd] 2024-04-26T18:32:13.622 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.622 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.622 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.622 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.622 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.622 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.622 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: clone() 2024-04-26T18:32:13.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.618+0000 7fe86b6ca700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe86b6ca700 time 2024-04-26T18:32:13.617905+0000 2024-04-26T18:32:13.623 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.624 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.624 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa017986700 time 2024-04-26T18:32:13.486594+0000 2024-04-26T18:32:13.624 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.624 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.625 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa021faf3dd] 2024-04-26T18:32:13.625 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.625 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.625 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.625 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.625 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: clone() 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.486+0000 7fa017986700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa017986700 time 2024-04-26T18:32:13.486594+0000 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.627 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.627 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe875cf33dd] 2024-04-26T18:32:13.627 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.627 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.627 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.627 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: clone() 2024-04-26T18:32:13.628 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.629 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: *** Caught signal (Aborted) ** 2024-04-26T18:32:13.629 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: in thread 7fe86b6ca700 thread_name:ms_dispatch 2024-04-26T18:32:13.629 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.629 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe874a7bd20] 2024-04-26T18:32:13.629 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: gsignal() 2024-04-26T18:32:13.629 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: abort() 2024-04-26T18:32:13.629 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe875cf3437] 2024-04-26T18:32:13.630 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.630 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.630 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.630 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.630 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.631 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.631 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa021faf3dd] 2024-04-26T18:32:13.631 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.631 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.632 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: clone() 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: *** Caught signal (Aborted) ** 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: in thread 7fa017986700 thread_name:ms_dispatch 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa020d37d20] 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: gsignal() 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: abort() 2024-04-26T18:32:13.633 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa021faf437] 2024-04-26T18:32:13.634 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.634 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.634 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.634 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.634 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.635 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.635 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.635 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.635 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.635 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.636 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.636 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 16: clone() 2024-04-26T18:32:13.636 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.619+0000 7fe86b6ca700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:13.636 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: in thread 7fe86b6ca700 thread_name:ms_dispatch 2024-04-26T18:32:13.636 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.636 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.637 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe874a7bd20] 2024-04-26T18:32:13.637 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: gsignal() 2024-04-26T18:32:13.637 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: abort() 2024-04-26T18:32:13.637 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe875cf3437] 2024-04-26T18:32:13.637 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.637 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.638 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 16: clone() 2024-04-26T18:32:13.639 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:13.639 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.640 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.640 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.640 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.640 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.640 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.640 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.641 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 16: clone() 2024-04-26T18:32:13.641 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.488+0000 7fa017986700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:13.641 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: in thread 7fa017986700 thread_name:ms_dispatch 2024-04-26T18:32:13.641 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.641 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.641 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa020d37d20] 2024-04-26T18:32:13.642 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: gsignal() 2024-04-26T18:32:13.642 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: abort() 2024-04-26T18:32:13.642 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa021faf437] 2024-04-26T18:32:13.642 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.642 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.643 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 16: clone() 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: -1> 2024-04-26T18:32:13.486+0000 7fa017986700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa017986700 time 2024-04-26T18:32:13.486594+0000 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.644 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa021faf3dd] 2024-04-26T18:32:13.645 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.645 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.645 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.645 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.645 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.645 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.646 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.646 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.646 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.646 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.646 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.646 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: clone() 2024-04-26T18:32:13.646 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.647 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 0> 2024-04-26T18:32:13.488+0000 7fa017986700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:13.647 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: in thread 7fa017986700 thread_name:ms_dispatch 2024-04-26T18:32:13.647 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.647 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.647 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa020d37d20] 2024-04-26T18:32:13.647 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: gsignal() 2024-04-26T18:32:13.647 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: abort() 2024-04-26T18:32:13.648 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa021faf437] 2024-04-26T18:32:13.648 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.648 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.648 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.648 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.649 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.649 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.649 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 16: clone() 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: -1> 2024-04-26T18:32:13.486+0000 7fa017986700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fa017986700 time 2024-04-26T18:32:13.486594+0000 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.650 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.651 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.651 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fa021faf3dd] 2024-04-26T18:32:13.651 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.651 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.651 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.651 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: clone() 2024-04-26T18:32:13.653 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.653 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 0> 2024-04-26T18:32:13.488+0000 7fa017986700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:13.653 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: in thread 7fa017986700 thread_name:ms_dispatch 2024-04-26T18:32:13.653 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.653 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.654 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fa020d37d20] 2024-04-26T18:32:13.654 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2: gsignal() 2024-04-26T18:32:13.654 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 3: abort() 2024-04-26T18:32:13.654 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fa021faf437] 2024-04-26T18:32:13.655 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fa021faf5a3] 2024-04-26T18:32:13.655 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5628053a7822] 2024-04-26T18:32:13.655 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x562805317598] 2024-04-26T18:32:13.655 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x562805319c17] 2024-04-26T18:32:13.655 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x562805212d0b] 2024-04-26T18:32:13.655 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5628052132ac] 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5628051fbfad] 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fa022243e20] 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 13: (DispatchQueue::entry()+0x51f) [0x7fa022240fdf] 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fa022313fc1] 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fa020d2d1ca] 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 16: clone() 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[98760]: 2024-04-26T18:32:13.656 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:13 smithi090 podman[99517]: 2024-04-26 18:32:13.559245967 +0000 UTC m=+0.025709096 container died c2366e377dcb0615fc82b9d16d74d2f414f0a577537ee8b60c157b0c505ebf11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-26T18:32:13.751 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: -1> 2024-04-26T18:32:13.618+0000 7fe86b6ca700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe86b6ca700 time 2024-04-26T18:32:13.617905+0000 2024-04-26T18:32:13.751 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.752 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.752 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.752 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe875cf33dd] 2024-04-26T18:32:13.752 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.752 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.752 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.752 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: clone() 2024-04-26T18:32:13.753 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.754 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 0> 2024-04-26T18:32:13.619+0000 7fe86b6ca700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:13.754 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: in thread 7fe86b6ca700 thread_name:ms_dispatch 2024-04-26T18:32:13.754 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.754 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.754 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe874a7bd20] 2024-04-26T18:32:13.754 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: gsignal() 2024-04-26T18:32:13.754 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: abort() 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe875cf3437] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.755 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.756 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.756 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.756 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.756 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 16: clone() 2024-04-26T18:32:13.756 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:13.756 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.757 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: -1> 2024-04-26T18:32:13.618+0000 7fe86b6ca700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fe86b6ca700 time 2024-04-26T18:32:13.617905+0000 2024-04-26T18:32:13.757 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:13.757 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.757 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.757 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fe875cf33dd] 2024-04-26T18:32:13.757 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.758 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.759 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: clone() 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 0> 2024-04-26T18:32:13.619+0000 7fe86b6ca700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: in thread 7fe86b6ca700 thread_name:ms_dispatch 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fe874a7bd20] 2024-04-26T18:32:13.760 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2: gsignal() 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 3: abort() 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fe875cf3437] 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fe875cf35a3] 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561091c6b822] 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561091bdb598] 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561091bddc17] 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561091ad6d0b] 2024-04-26T18:32:13.761 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561091ad72ac] 2024-04-26T18:32:13.762 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561091abffad] 2024-04-26T18:32:13.762 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fe875f87e20] 2024-04-26T18:32:13.762 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 13: (DispatchQueue::entry()+0x51f) [0x7fe875f84fdf] 2024-04-26T18:32:13.762 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fe876057fc1] 2024-04-26T18:32:13.762 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fe874a711ca] 2024-04-26T18:32:13.762 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 16: clone() 2024-04-26T18:32:13.763 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:13.763 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:13 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[99107]: 2024-04-26T18:32:13.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.52 MB/sec execute 218 sec latency 4241.123 ms 2024-04-26T18:32:14.698 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:14 smithi090 podman[99529]: 2024-04-26 18:32:14.415118754 +0000 UTC m=+0.755170176 container died f18336076f7d7b0ca8322d55481fe559816242be1ad9ee410f4ab3303515ef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD) 2024-04-26T18:32:14.699 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:14 smithi090 podman[99517]: 2024-04-26 18:32:14.408533334 +0000 UTC m=+0.874996462 container remove c2366e377dcb0615fc82b9d16d74d2f414f0a577537ee8b60c157b0c505ebf11 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:32:14.699 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:14 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:32:14.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.49 MB/sec execute 219 sec latency 5241.160 ms 2024-04-26T18:32:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:14 smithi119 ceph-mon[32752]: pgmap v699: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-04-26T18:32:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:14 smithi167 ceph-mon[32751]: pgmap v699: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-04-26T18:32:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:14 smithi090 ceph-mon[29180]: pgmap v699: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-04-26T18:32:15.580 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:15 smithi090 podman[99529]: 2024-04-26 18:32:15.297225534 +0000 UTC m=+1.637276956 container remove f18336076f7d7b0ca8322d55481fe559816242be1ad9ee410f4ab3303515ef83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T18:32:15.580 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:15 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:32:15.875 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:15 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-26T18:32:15.875 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:15 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 9.193s CPU time. 2024-04-26T18:32:15.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:15 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-26T18:32:15.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:15 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 11.471s CPU time. 2024-04-26T18:32:15.898 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.46 MB/sec execute 220 sec latency 6241.254 ms 2024-04-26T18:32:16.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.43 MB/sec execute 221 sec latency 7241.356 ms 2024-04-26T18:32:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:16 smithi119 ceph-mon[32752]: pgmap v700: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-26T18:32:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:16 smithi167 ceph-mon[32751]: pgmap v700: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-26T18:32:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:16 smithi090 ceph-mon[29180]: pgmap v700: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-04-26T18:32:17.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.41 MB/sec execute 222 sec latency 8241.436 ms 2024-04-26T18:32:18.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.38 MB/sec execute 223 sec latency 9241.485 ms 2024-04-26T18:32:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:18 smithi119 ceph-mon[32752]: pgmap v701: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-26T18:32:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:18 smithi167 ceph-mon[32751]: pgmap v701: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-26T18:32:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:18 smithi090 ceph-mon[29180]: pgmap v701: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-04-26T18:32:19.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.35 MB/sec execute 224 sec latency 10241.552 ms 2024-04-26T18:32:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 -- 172.21.15.90:0/1773567979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c101860 msgr2=0x7ff08c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 --2- 172.21.15.90:0/1773567979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c101860 0x7ff08c101ce0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff088009930 tx=0x7ff08802a420 comp rx=0 tx=0).stop 2024-04-26T18:32:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 -- 172.21.15.90:0/1773567979 shutdown_connections 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 --2- 172.21.15.90:0/1773567979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff08c102220 0x7ff08c10e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 --2- 172.21.15.90:0/1773567979 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c101860 0x7ff08c101ce0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 --2- 172.21.15.90:0/1773567979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c107820 0x7ff08c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 -- 172.21.15.90:0/1773567979 >> 172.21.15.90:0/1773567979 conn(0x7ff08c0789b0 msgr2=0x7ff08c078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 -- 172.21.15.90:0/1773567979 shutdown_connections 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.598+0000 7ff09343e640 1 -- 172.21.15.90:0/1773567979 wait complete. 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 Processor -- start 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 -- start start 2024-04-26T18:32:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff08c101860 0x7ff08c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c102220 0x7ff08c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c107820 0x7ff08c19d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c111ae0 con 0x7ff08c107820 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c111960 con 0x7ff08c101860 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff09343e640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff08c111c60 con 0x7ff08c102220 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff0919db640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c107820 0x7ff08c19d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff0919db640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c107820 0x7ff08c19d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54596/0 (socket says 172.21.15.90:54596) 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff0919db640 1 -- 172.21.15.90:0/2019220802 learned_addr learned my addr 172.21.15.90:0/2019220802 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff0909d9640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c102220 0x7ff08c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.599+0000 7ff0911da640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff08c101860 0x7ff08c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0919db640 1 -- 172.21.15.90:0/2019220802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c102220 msgr2=0x7ff08c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0919db640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c102220 0x7ff08c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0919db640 1 -- 172.21.15.90:0/2019220802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff08c101860 msgr2=0x7ff08c199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0919db640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff08c101860 0x7ff08c199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0919db640 1 -- 172.21.15.90:0/2019220802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08c19e120 con 0x7ff08c107820 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0911da640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff08c101860 0x7ff08c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0919db640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c107820 0x7ff08c19d9f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff08400eaa0 tx=0x7ff08400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0827fc640 1 -- 172.21.15.90:0/2019220802 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08400cdb0 con 0x7ff08c107820 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08c19e410 con 0x7ff08c107820 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff08c1a5ca0 con 0x7ff08c107820 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0827fc640 1 -- 172.21.15.90:0/2019220802 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff084011920 con 0x7ff08c107820 2024-04-26T18:32:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.600+0000 7ff0827fc640 1 -- 172.21.15.90:0/2019220802 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08401f060 con 0x7ff08c107820 2024-04-26T18:32:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff0827fc640 1 -- 172.21.15.90:0/2019220802 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7ff084011470 con 0x7ff08c107820 2024-04-26T18:32:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 Processor -- start 2024-04-26T18:32:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 -- start start 2024-04-26T18:32:20.611 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff0827fc640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0780a3e40 0x7ff0780a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad742e00 0x557fad7431e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad743720 0x557fad751fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557fad6a9460 con 0x557fad743720 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557fad663590 con 0x557fad742e00 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.603+0000 7ff096232740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557fad6a5420 con 0x557fad7524f0 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0827fc640 1 -- 172.21.15.90:0/2019220802 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 12281+0+0 (secure 0 0 0) 0x7ff0840d17f0 con 0x7ff08c107820 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0780a3e40 0x7ff0780a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad743720 0x557fad751fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0911da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad742e00 0x557fad7431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.612 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad743720 0x557fad751fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54602/0 (socket says 172.21.15.90:54602) 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 -- 172.21.15.90:0/4043043718 learned_addr learned my addr 172.21.15.90:0/4043043718 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0919db640 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 -- 172.21.15.90:0/4043043718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 msgr2=0x557fad7548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 -- 172.21.15.90:0/4043043718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad742e00 msgr2=0x557fad7431e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad742e00 0x557fad7431e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 -- 172.21.15.90:0/4043043718 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fad6700d0 con 0x557fad743720 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0919db640 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0780a3e40 0x7ff0780a6300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff088032fd0 tx=0x7ff088032b20 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0909d9640 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad743720 0x557fad751fb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff088009930 tx=0x7ff088046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd0361755e17dba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.604+0000 7ff0777fe640 1 -- 172.21.15.90:0/4043043718 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0880039c0 con 0x557fad743720 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff0777fe640 1 -- 172.21.15.90:0/4043043718 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08803d450 con 0x557fad743720 2024-04-26T18:32:20.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff0777fe640 1 -- 172.21.15.90:0/4043043718 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08803c670 con 0x557fad743720 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 -- 172.21.15.90:0/4043043718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad743720 msgr2=0x557fad751fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad743720 0x557fad751fb0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7ff088009930 tx=0x7ff088046000 comp rx=0 tx=0).stop 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 -- 172.21.15.90:0/4043043718 shutdown_connections 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad743720 0x557fad751fb0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 --2- 172.21.15.90:0/4043043718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad742e00 0x557fad7431e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 -- 172.21.15.90:0/4043043718 >> 172.21.15.90:0/4043043718 conn(0x557fad73f410 msgr2=0x557fad73f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 -- 172.21.15.90:0/4043043718 shutdown_connections 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.605+0000 7ff096232740 1 -- 172.21.15.90:0/4043043718 wait complete. 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 -- start start 2024-04-26T18:32:20.614 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 Processor -- start 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad74f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad74fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557fad7550c0 con 0x557fad7524f0 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557fad630b60 con 0x557fad743720 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557fad755240 con 0x557fad742e00 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff096232740 10 client.0 _ll_register_callbacks cb 0x557fad6552b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff0919db640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad74fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff0919db640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad74fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54616/0 (socket says 172.21.15.90:54616) 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff0919db640 1 -- 172.21.15.90:0/1405234121 learned_addr learned my addr 172.21.15.90:0/1405234121 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:20.615 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff0911da640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad74f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.606+0000 7ff0909d9640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff0919db640 1 -- 172.21.15.90:0/1405234121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 msgr2=0x557fad74f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff0919db640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad74f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff0919db640 1 -- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 msgr2=0x557fad74f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff0919db640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff0919db640 1 -- 172.21.15.90:0/1405234121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fad3bb210 con 0x557fad7524f0 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff0909d9640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff0919db640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad74fec0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff084010520 tx=0x7ff084068390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5ecbe63fbfacf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff074ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff074ff9640 1 -- 172.21.15.90:0/1405234121 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840965e0 con 0x557fad7524f0 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff074ff9640 1 -- 172.21.15.90:0/1405234121 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 12281+0+0 (secure 0 0 0) 0x7ff0840a50c0 con 0x557fad7524f0 2024-04-26T18:32:20.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff08803d450 con 0x557fad7524f0 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff074ff9640 1 -- 172.21.15.90:0/1405234121 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=282) ==== 24+0+0 (secure 0 0 0) 0x7ff084096780 con 0x557fad7524f0 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 10 client.16854 fetch_fsmap learned FSMap version 282 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=282,monmap=4+}) -- 0x557fad487620 con 0x557fad7524f0 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff074ff9640 1 -- 172.21.15.90:0/1405234121 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff08803d450 con 0x557fad7524f0 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff074ff9640 1 -- 172.21.15.90:0/1405234121 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 282) ==== 4716+0+0 (secure 0 0 0) 0x7ff08400f040 con 0x557fad7524f0 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 10 client.16854 fetch_fsmap finished waiting for FSMap version 282 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 20 client.16854 populate_metadata read hostname 'smithi090' 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad762bc0 0x557fad765020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 4 client.16854 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.607+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fad6b7070 con 0x557fad762bc0 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.608+0000 7ff0911da640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad762bc0 0x557fad765020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.617 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.608+0000 7ff0911da640 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad762bc0 0x557fad765020 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cd4ec26ca659e18 server_cookie=be8980dc70b2e5df in_seq=0 out_seq=0 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.608+0000 7ff074ff9640 10 client.16854 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff074ff9640 1 -- 172.21.15.90:0/1405234121 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557fad6b7070 con 0x557fad762bc0 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff074ff9640 10 client.16854 handle_command_reply: tid=0 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 client.16854 shutdown 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 msgr2=0x557fad74fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad74fec0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ff084010520 tx=0x7ff084068390 comp rx=0 tx=0).stop 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 shutdown_connections 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad762bc0 0x557fad765020 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad74fec0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 --2- 172.21.15.90:0/1405234121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad74f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff074ff9640 0 client.16854 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 >> 172.21.15.90:0/1405234121 conn(0x557fad73f410 msgr2=0x557fad745280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 shutdown_connections 2024-04-26T18:32:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 1 -- 172.21.15.90:0/1405234121 wait complete. 2024-04-26T18:32:20.619 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.609+0000 7ff096232740 20 client.16854 trim_cache size 0 max 16384 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 Processor -- start 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 -- start start 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad75e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad76a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad76ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff070011260 con 0x557fad7524f0 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff07000eff0 con 0x557fad743720 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.624+0000 7ff096232740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff070010b90 con 0x557fad742e00 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad76ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad76ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54630/0 (socket says 172.21.15.90:54630) 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 -- 172.21.15.90:0/1005928711 learned_addr learned my addr 172.21.15.90:0/1005928711 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0909d9640 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad76a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0911da640 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad75e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 -- 172.21.15.90:0/1005928711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 msgr2=0x557fad75e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad75e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 -- 172.21.15.90:0/1005928711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 msgr2=0x557fad76a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad76a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 -- 172.21.15.90:0/1005928711 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fad7656a0 con 0x557fad7524f0 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0911da640 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad75e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff0919db640 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad76ae70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff0840992b0 tx=0x7ff0840a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9d2b597e9fae19 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff076ffd640 1 -- 172.21.15.90:0/1005928711 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840681f0 con 0x557fad7524f0 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff076ffd640 1 -- 172.21.15.90:0/1005928711 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff084098630 con 0x557fad7524f0 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.625+0000 7ff076ffd640 1 -- 172.21.15.90:0/1005928711 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0840a50c0 con 0x557fad7524f0 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 -- 172.21.15.90:0/1005928711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 msgr2=0x557fad76ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad76ae70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff0840992b0 tx=0x7ff0840a56d0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 -- 172.21.15.90:0/1005928711 shutdown_connections 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad7524f0 0x557fad76ae70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad76a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 --2- 172.21.15.90:0/1005928711 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad742e00 0x557fad75e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 -- 172.21.15.90:0/1005928711 >> 172.21.15.90:0/1005928711 conn(0x557fad73f410 msgr2=0x557fad73f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 -- 172.21.15.90:0/1005928711 shutdown_connections 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 -- 172.21.15.90:0/1005928711 wait complete. 2024-04-26T18:32:20.632 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 -- start start 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.626+0000 7ff096232740 1 Processor -- start 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad742e00 0x557fad74c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7ff07000fd90 con 0x557fad742e00 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7ff07000f6c0 con 0x557fad743720 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff070010460 con 0x557fad7524f0 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 10 client.0 _ll_register_callbacks cb 0x7ff084011840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad742e00 0x557fad74c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad742e00 0x557fad74c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54632/0 (socket says 172.21.15.90:54632) 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 -- 172.21.15.90:0/433609067 learned_addr learned my addr 172.21.15.90:0/433609067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:20.633 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0909d9640 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0919db640 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 -- 172.21.15.90:0/433609067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 msgr2=0x557fad7c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 -- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 msgr2=0x557fad74c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 -- 172.21.15.90:0/433609067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fad73f240 con 0x557fad742e00 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff0911da640 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad742e00 0x557fad74c440 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x557fad765870 tx=0x7ff07c01af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bcf65248ae2dd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff074ff9640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff074ff9640 1 -- 172.21.15.90:0/433609067 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07c013e60 con 0x557fad742e00 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff074ff9640 1 -- 172.21.15.90:0/433609067 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(166..166 src has 1..166) ==== 12281+0+0 (secure 0 0 0) 0x7ff07c01a2c0 con 0x557fad742e00 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff084098630 con 0x557fad742e00 2024-04-26T18:32:20.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.627+0000 7ff074ff9640 1 -- 172.21.15.90:0/433609067 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=282) ==== 24+0+0 (secure 0 0 0) 0x7ff07c02b070 con 0x557fad742e00 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 10 client.16860 fetch_fsmap learned FSMap version 282 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=282,monmap=4+}) -- 0x557fad7c95f0 con 0x557fad742e00 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff074ff9640 1 -- 172.21.15.90:0/433609067 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff084098630 con 0x557fad742e00 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff074ff9640 1 -- 172.21.15.90:0/433609067 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 282) ==== 4716+0+0 (secure 0 0 0) 0x7ff07c00f040 con 0x557fad742e00 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 10 client.16860 fetch_fsmap finished waiting for FSMap version 282 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 20 client.16860 populate_metadata read hostname 'smithi090' 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad7c9db0 0x557fad7cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 4 client.16860 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557fad6b0ee0 con 0x557fad7c9db0 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.628+0000 7ff0909d9640 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad7c9db0 0x557fad7cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:20.635 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.629+0000 7ff074ff9640 10 client.16860 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:20.636 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:20.629+0000 7ff0909d9640 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad7c9db0 0x557fad7cc1d0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4d9dadd21fc17c3 server_cookie=85991ed3790b7838 in_seq=0 out_seq=0 2024-04-26T18:32:20.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.32 MB/sec execute 225 sec latency 11241.674 ms 2024-04-26T18:32:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:20 smithi119 ceph-mon[32752]: pgmap v702: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-26T18:32:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:20 smithi167 ceph-mon[32751]: pgmap v702: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-26T18:32:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:20 smithi090 ceph-mon[29180]: pgmap v702: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-04-26T18:32:21.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.29 MB/sec execute 226 sec latency 12241.716 ms 2024-04-26T18:32:22.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:22 smithi167 ceph-mon[32751]: pgmap v703: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-04-26T18:32:22.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:22.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:22.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:22.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:22.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:22.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.26 MB/sec execute 227 sec latency 13241.818 ms 2024-04-26T18:32:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:22 smithi119 ceph-mon[32752]: pgmap v703: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-04-26T18:32:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:22 smithi090 ceph-mon[29180]: pgmap v703: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 1.1 MiB/s wr, 124 op/s 2024-04-26T18:32:23.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:23.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:23.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.24 MB/sec execute 228 sec latency 14241.901 ms 2024-04-26T18:32:24.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.21 MB/sec execute 229 sec latency 15241.947 ms 2024-04-26T18:32:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: pgmap v704: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: pgmap v704: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: pgmap v704: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.7 MiB/s wr, 182 op/s 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:25.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 2. 2024-04-26T18:32:25.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: Stopped Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:32:25.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 9.193s CPU time. 2024-04-26T18:32:25.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: Starting Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:32:25.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:32:25.884 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: Stopped Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:32:25.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 11.471s CPU time. 2024-04-26T18:32:25.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:25 smithi090 systemd[1]: Starting Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:32:25.885 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:25 smithi090 podman[100266]: 2024-04-26 18:32:25.776820443 +0000 UTC m=+0.019068678 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:32:25.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37670 6.18 MB/sec execute 230 sec latency 16242.004 ms 2024-04-26T18:32:26.156 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:25 smithi090 podman[100269]: 2024-04-26 18:32:25.788576645 +0000 UTC m=+0.026841860 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:32:26.156 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:25 smithi090 podman[100266]: 2024-04-26 18:32:25.908585124 +0000 UTC m=+0.150833381 container create e02271592453b69a35d5ee24ec1a11cbf2ebadd9f9874f8cb72001f7b41188df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T18:32:26.156 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:26 smithi090 podman[100266]: 2024-04-26 18:32:26.130254291 +0000 UTC m=+0.372502532 container init e02271592453b69a35d5ee24ec1a11cbf2ebadd9f9874f8cb72001f7b41188df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-26T18:32:26.156 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:26 smithi090 podman[100266]: 2024-04-26 18:32:26.133517286 +0000 UTC m=+0.375765519 container start e02271592453b69a35d5ee24ec1a11cbf2ebadd9f9874f8cb72001f7b41188df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T18:32:26.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:32:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:32:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:32:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:32:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:32:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:32:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:26.458 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:26 smithi090 podman[100269]: 2024-04-26 18:32:26.247266714 +0000 UTC m=+0.485531904 container create fd56d7301a1a4df662621faf619ea2ddf3f518967b12f90b50ea3d6462066ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T18:32:26.458 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mds[100332]: -- 172.21.15.90:0/3958905691 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561bdc45ba40 con 0x561bdc42ec00 2024-04-26T18:32:26.459 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: starting mds.d at 2024-04-26T18:32:26.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:26 smithi090 bash[100266]: e02271592453b69a35d5ee24ec1a11cbf2ebadd9f9874f8cb72001f7b41188df 2024-04-26T18:32:26.876 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:32:26 smithi090 systemd[1]: Started Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:32:26.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:26 smithi090 podman[100269]: 2024-04-26 18:32:26.551402427 +0000 UTC m=+0.789667616 container init fd56d7301a1a4df662621faf619ea2ddf3f518967b12f90b50ea3d6462066ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T18:32:26.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:26 smithi090 podman[100269]: 2024-04-26 18:32:26.554072063 +0000 UTC m=+0.792337257 container start fd56d7301a1a4df662621faf619ea2ddf3f518967b12f90b50ea3d6462066ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:32:26.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-mds[100390]: -- 172.21.15.90:0/2728396590 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bd5b5fa40 con 0x563bd5b33000 2024-04-26T18:32:26.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:26 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: starting mds.j at 2024-04-26T18:32:26.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:26 smithi090 bash[100269]: fd56d7301a1a4df662621faf619ea2ddf3f518967b12f90b50ea3d6462066ccf 2024-04-26T18:32:26.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:32:26 smithi090 systemd[1]: Started Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:32:26.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37672 6.16 MB/sec execute 231 sec latency 17242.031 ms 2024-04-26T18:32:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: pgmap v705: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-04-26T18:32:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: daemon mds.d restarted 2024-04-26T18:32:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: osdmap e167: 12 total, 12 up, 12 in 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:boot 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=a=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:32:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: pgmap v707: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.3 MiB/s wr, 92 op/s 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: pgmap v708: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 2.8 MiB/s wr, 107 op/s 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: Health check update: 8 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: daemon mds.j restarted 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: osdmap e168: 12 total, 12 up, 12 in 2024-04-26T18:32:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:boot 2024-04-26T18:32:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:32:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=a=up:replay} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:32:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:32:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:27 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=a=up:replay} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: pgmap v705: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-04-26T18:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: daemon mds.d restarted 2024-04-26T18:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: osdmap e167: 12 total, 12 up, 12 in 2024-04-26T18:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:boot 2024-04-26T18:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=a=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: pgmap v707: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.3 MiB/s wr, 92 op/s 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: pgmap v708: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 2.8 MiB/s wr, 107 op/s 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: Health check update: 8 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: daemon mds.j restarted 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: osdmap e168: 12 total, 12 up, 12 in 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:boot 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=a=up:replay} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:32:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:32:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:27 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=a=up:replay} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: pgmap v705: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.6 MiB/s wr, 66 op/s 2024-04-26T18:32:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: daemon mds.d restarted 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: osdmap e167: 12 total, 12 up, 12 in 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:boot 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: Standby daemon mds.a assigned to filesystem cephfs as rank 2 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:active} 3 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=a=up:replay} 2 up:standby-replay 1 up:standby 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:32:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: pgmap v707: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 330 KiB/s rd, 2.3 MiB/s wr, 92 op/s 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: pgmap v708: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 2.8 MiB/s wr, 107 op/s 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: Health check update: 8 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: daemon mds.j restarted 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: osdmap e168: 12 total, 12 up, 12 in 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:boot 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=a=up:replay} 2 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:32:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:27 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=a=up:replay} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:27.899 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37672 6.13 MB/sec execute 232 sec latency 18242.096 ms 2024-04-26T18:32:28.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:28 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:resolve 2024-04-26T18:32:28.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:28 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:resolve 2024-04-26T18:32:28.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:28 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=a=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:28.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:28 smithi090 ceph-mon[29180]: pgmap v710: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:32:28.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37672 6.10 MB/sec execute 233 sec latency 19242.150 ms 2024-04-26T18:32:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:28 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:resolve 2024-04-26T18:32:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:28 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:resolve 2024-04-26T18:32:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:28 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=a=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:28 smithi119 ceph-mon[32752]: pgmap v710: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:32:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:28 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:resolve 2024-04-26T18:32:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:28 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:resolve 2024-04-26T18:32:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:28 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=a=up:resolve} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:28 smithi167 ceph-mon[32751]: pgmap v710: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff074ff9640 1 -- 172.21.15.90:0/433609067 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557fad6b0ee0 con 0x557fad7c9db0 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff074ff9640 10 client.16860 handle_command_reply: tid=0 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 client.16860 shutdown 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad742e00 msgr2=0x557fad74c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad742e00 0x557fad74c440 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x557fad765870 tx=0x7ff07c01af90 comp rx=0 tx=0).stop 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 shutdown_connections 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557fad7c9db0 0x557fad7cc1d0 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557fad7524f0 0x557fad7c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557fad743720 0x557fad74c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 --2- 172.21.15.90:0/433609067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557fad742e00 0x557fad74c440 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 >> 172.21.15.90:0/433609067 conn(0x557fad73f410 msgr2=0x557fad73dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff074ff9640 0 client.16860 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 shutdown_connections 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.691+0000 7ff096232740 1 -- 172.21.15.90:0/433609067 wait complete. 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff096232740 20 client.16860 trim_cache size 0 max 16384 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:32:29.693 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:32:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0780a3e40 msgr2=0x7ff0780a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0780a3e40 0x7ff0780a6300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff088032fd0 tx=0x7ff088032b20 comp rx=0 tx=0).stop 2024-04-26T18:32:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c107820 msgr2=0x7ff08c19d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c107820 0x7ff08c19d9f0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff08400eaa0 tx=0x7ff08400ef60 comp rx=0 tx=0).stop 2024-04-26T18:32:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 shutdown_connections 2024-04-26T18:32:29.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7ff0780a3e40 0x7ff0780a6300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff08c107820 0x7ff08c19d9f0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff08c102220 0x7ff08c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 --2- 172.21.15.90:0/2019220802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff08c101860 0x7ff08c199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 >> 172.21.15.90:0/2019220802 conn(0x7ff08c0789b0 msgr2=0x7ff08c06a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.692+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 shutdown_connections 2024-04-26T18:32:29.695 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:29.693+0000 7ff09343e640 1 -- 172.21.15.90:0/2019220802 wait complete. 2024-04-26T18:32:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:29 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:32:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:29 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:reconnect 2024-04-26T18:32:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:29 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:reconnect 2024-04-26T18:32:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:29 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=a=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:29 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:32:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:29 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:32:29.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37743 6.10 MB/sec execute 234 sec latency 20242.484 ms 2024-04-26T18:32:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:29 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:32:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:29 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:reconnect 2024-04-26T18:32:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:29 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:reconnect 2024-04-26T18:32:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:29 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=a=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:29 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:32:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:29 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:32:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:29 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:32:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:29 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:reconnect 2024-04-26T18:32:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:29 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:reconnect 2024-04-26T18:32:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:29 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=a=up:reconnect} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:29 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00599991 2024-04-26T18:32:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:29 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:32:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:30 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:rejoin 2024-04-26T18:32:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:30 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:rejoin 2024-04-26T18:32:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:30 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=a=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:30 smithi090 ceph-mon[29180]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:32:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:30 smithi090 ceph-mon[29180]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:32:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:30 smithi090 ceph-mon[29180]: pgmap v711: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 20 op/s 2024-04-26T18:32:30.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 37944 6.11 MB/sec execute 235 sec latency 21242.515 ms 2024-04-26T18:32:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:30 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:rejoin 2024-04-26T18:32:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:30 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:rejoin 2024-04-26T18:32:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:30 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=a=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:30 smithi119 ceph-mon[32752]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:32:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:30 smithi119 ceph-mon[32752]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:32:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:30 smithi119 ceph-mon[32752]: pgmap v711: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 20 op/s 2024-04-26T18:32:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:30 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:rejoin 2024-04-26T18:32:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:30 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:rejoin 2024-04-26T18:32:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:30 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=a=up:rejoin} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:30 smithi167 ceph-mon[32751]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:32:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:30 smithi167 ceph-mon[32751]: daemon mds.a is now active in filesystem cephfs as rank 2 2024-04-26T18:32:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:30 smithi167 ceph-mon[32751]: pgmap v711: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 74 KiB/s wr, 20 op/s 2024-04-26T18:32:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:31 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:32:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:31 smithi090 ceph-mon[29180]: osdmap e169: 12 total, 12 up, 12 in 2024-04-26T18:32:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:31 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:active 2024-04-26T18:32:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:31 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:active 2024-04-26T18:32:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:31 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:31 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 2024-04-26T18:32:31.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38237 6.13 MB/sec execute 236 sec latency 21410.058 ms 2024-04-26T18:32:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:31 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:32:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:31 smithi119 ceph-mon[32752]: osdmap e169: 12 total, 12 up, 12 in 2024-04-26T18:32:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:31 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:active 2024-04-26T18:32:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:31 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:active 2024-04-26T18:32:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:31 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:31 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 2024-04-26T18:32:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:31 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:32:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:31 smithi167 ceph-mon[32751]: osdmap e169: 12 total, 12 up, 12 in 2024-04-26T18:32:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:31 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/1000850976,v1:172.21.15.90:6839/1000850976] up:active 2024-04-26T18:32:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:31 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/1044509479,v1:172.21.15.90:6835/1044509479] up:active 2024-04-26T18:32:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:31 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 1 up:standby-replay 2 up:standby 2024-04-26T18:32:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:31 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=a=up:active} 3 up:standby-replay 2024-04-26T18:32:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:32 smithi090 ceph-mon[29180]: pgmap v713: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-04-26T18:32:32.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 38770 6.23 MB/sec execute 237 sec latency 152.331 ms 2024-04-26T18:32:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:32 smithi119 ceph-mon[32752]: pgmap v713: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-04-26T18:32:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:32 smithi167 ceph-mon[32751]: pgmap v713: 97 pgs: 97 active+clean; 659 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 56 KiB/s wr, 15 op/s 2024-04-26T18:32:33.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:32:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:32:33.280+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:32:33.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:32:33 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:32:33.280+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:32:33.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39288 6.30 MB/sec execute 238 sec latency 116.457 ms 2024-04-26T18:32:34.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 39860 6.38 MB/sec execute 239 sec latency 136.372 ms 2024-04-26T18:32:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:35 smithi119 ceph-mon[32752]: pgmap v714: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-04-26T18:32:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:35 smithi167 ceph-mon[32751]: pgmap v714: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-04-26T18:32:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:35 smithi090 ceph-mon[29180]: pgmap v714: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 2.7 MiB/s wr, 222 op/s 2024-04-26T18:32:35.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40460 6.54 MB/sec execute 240 sec latency 164.977 ms 2024-04-26T18:32:36.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 40888 6.62 MB/sec execute 241 sec latency 113.596 ms 2024-04-26T18:32:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:37 smithi119 ceph-mon[32752]: pgmap v715: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-04-26T18:32:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:37 smithi167 ceph-mon[32751]: pgmap v715: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-04-26T18:32:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:37 smithi090 ceph-mon[29180]: pgmap v715: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.2 MiB/s wr, 184 op/s 2024-04-26T18:32:37.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41339 6.67 MB/sec execute 242 sec latency 638.111 ms 2024-04-26T18:32:38.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 41843 6.76 MB/sec execute 243 sec latency 1638.140 ms 2024-04-26T18:32:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:38 smithi119 ceph-mon[32752]: osdmap e170: 12 total, 12 up, 12 in 2024-04-26T18:32:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:38 smithi119 ceph-mon[32752]: pgmap v717: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:32:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:38 smithi167 ceph-mon[32751]: osdmap e170: 12 total, 12 up, 12 in 2024-04-26T18:32:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:38 smithi167 ceph-mon[32751]: pgmap v717: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:32:39.090 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:38 smithi090 ceph-mon[29180]: osdmap e170: 12 total, 12 up, 12 in 2024-04-26T18:32:39.090 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:38 smithi090 ceph-mon[29180]: pgmap v717: 97 pgs: 97 active+clean; 648 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-04-26T18:32:39.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42307 6.82 MB/sec execute 244 sec latency 2638.177 ms 2024-04-26T18:32:40.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 42921 6.94 MB/sec execute 245 sec latency 3638.216 ms 2024-04-26T18:32:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:41 smithi119 ceph-mon[32752]: pgmap v718: 97 pgs: 97 active+clean; 680 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 5.8 MiB/s wr, 407 op/s 2024-04-26T18:32:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:41 smithi167 ceph-mon[32751]: pgmap v718: 97 pgs: 97 active+clean; 680 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 5.8 MiB/s wr, 407 op/s 2024-04-26T18:32:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:41 smithi090 ceph-mon[29180]: pgmap v718: 97 pgs: 97 active+clean; 680 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 5.8 MiB/s wr, 407 op/s 2024-04-26T18:32:41.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 43362 7.00 MB/sec execute 246 sec latency 4638.251 ms 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.812+0000 7f5125322640 1 -- 172.21.15.90:0/2820712539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 msgr2=0x7f5120107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.812+0000 7f5125322640 1 --2- 172.21.15.90:0/2820712539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 0x7f5120107c20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5110009930 tx=0x7f511002a420 comp rx=0 tx=0).stop 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.812+0000 7f5125322640 1 -- 172.21.15.90:0/2820712539 shutdown_connections 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.812+0000 7f5125322640 1 --2- 172.21.15.90:0/2820712539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51201021f0 0x7f512010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.812+0000 7f5125322640 1 --2- 172.21.15.90:0/2820712539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5120101830 0x7f5120101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.812+0000 7f5125322640 1 --2- 172.21.15.90:0/2820712539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 0x7f5120107c20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.812+0000 7f5125322640 1 -- 172.21.15.90:0/2820712539 >> 172.21.15.90:0/2820712539 conn(0x7f5120078980 msgr2=0x7f5120078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.813+0000 7f5125322640 1 -- 172.21.15.90:0/2820712539 shutdown_connections 2024-04-26T18:32:42.813 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.813+0000 7f5125322640 1 -- 172.21.15.90:0/2820712539 wait complete. 2024-04-26T18:32:42.815 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.813+0000 7f5125322640 1 Processor -- start 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.813+0000 7f5125322640 1 -- start start 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f5125322640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5120101830 0x7f5120196d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f5125322640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51201021f0 0x7f5120197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f5125322640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 0x7f512019b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f5125322640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5120110a40 con 0x7f5120107840 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f5125322640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51201108c0 con 0x7f5120101830 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f5125322640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5120110bc0 con 0x7f51201021f0 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5120101830 0x7f5120196d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5120101830 0x7f5120196d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38502/0 (socket says 172.21.15.90:38502) 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 -- 172.21.15.90:0/2828307473 learned_addr learned my addr 172.21.15.90:0/2828307473 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:42.816 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511f7fe640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 0x7f512019b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f5117fff640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51201021f0 0x7f5120197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 -- 172.21.15.90:0/2828307473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51201021f0 msgr2=0x7f5120197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51201021f0 0x7f5120197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 -- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 msgr2=0x7f512019b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 0x7f512019b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 -- 172.21.15.90:0/2828307473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512019bd30 con 0x7f5120101830 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511f7fe640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 0x7f512019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.814+0000 7f511effd640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5120101830 0x7f5120196d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f511002a940 tx=0x7f5110030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.815+0000 7f511cff9640 1 -- 172.21.15.90:0/2828307473 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511003f070 con 0x7f5120101830 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.815+0000 7f511cff9640 1 -- 172.21.15.90:0/2828307473 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5110004460 con 0x7f5120101830 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.815+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512019bfc0 con 0x7f5120101830 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.815+0000 7f511cff9640 1 -- 172.21.15.90:0/2828307473 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110035030 con 0x7f5120101830 2024-04-26T18:32:42.817 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.815+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51201a38b0 con 0x7f5120101830 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.817+0000 7f511cff9640 1 -- 172.21.15.90:0/2828307473 <== mon.2 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f5110042070 con 0x7f5120101830 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 Processor -- start 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 -- start start 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac65f00 0x55bb7ac662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac66820 0x55bb7ac70ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bb7abcc460 con 0x55bb7ac65f00 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bb7ab86590 con 0x55bb7ac66820 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.818+0000 7f512803d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bb7abc8420 con 0x55bb7ac710e0 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac66820 0x55bb7ac70ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511cff9640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f510c0a5fd0 0x7f510c0a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.820 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac66820 0x55bb7ac70ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38510/0 (socket says 172.21.15.90:38510) 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 -- 172.21.15.90:0/2913305038 learned_addr learned my addr 172.21.15.90:0/2913305038 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511f7fe640 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac65f00 0x55bb7ac662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511cff9640 1 -- 172.21.15.90:0/2828307473 <== mon.2 v2:172.21.15.119:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 12497+0+0 (secure 0 0 0) 0x7f51100f22b0 con 0x7f5120101830 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511f7fe640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f510c0a5fd0 0x7f510c0a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f5117fff640 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 -- 172.21.15.90:0/2913305038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 msgr2=0x55bb7ac734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac734d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 -- 172.21.15.90:0/2913305038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac65f00 msgr2=0x55bb7ac662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac65f00 0x55bb7ac662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 -- 172.21.15.90:0/2913305038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7ab930d0 con 0x55bb7ac66820 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.819+0000 7f511effd640 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac66820 0x55bb7ac70ba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51100c5a00 tx=0x7f51100037d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4c99b16ccbf717 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.820+0000 7f51157fa640 1 -- 172.21.15.90:0/2913305038 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110049070 con 0x55bb7ac66820 2024-04-26T18:32:42.821 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.820+0000 7f51157fa640 1 -- 172.21.15.90:0/2913305038 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5110030e10 con 0x55bb7ac66820 2024-04-26T18:32:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.820+0000 7f51157fa640 1 -- 172.21.15.90:0/2913305038 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100eecf0 con 0x55bb7ac66820 2024-04-26T18:32:42.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.820+0000 7f511f7fe640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f510c0a5fd0 0x7f510c0a8490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5118005fd0 tx=0x7f5118004ab0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 -- 172.21.15.90:0/2913305038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac66820 msgr2=0x55bb7ac70ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac66820 0x55bb7ac70ba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51100c5a00 tx=0x7f51100037d0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 -- 172.21.15.90:0/2913305038 shutdown_connections 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac66820 0x55bb7ac70ba0 secure :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51100c5a00 tx=0x7f51100037d0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 --2- 172.21.15.90:0/2913305038 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac65f00 0x55bb7ac662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 -- 172.21.15.90:0/2913305038 >> 172.21.15.90:0/2913305038 conn(0x55bb7ac62410 msgr2=0x55bb7ac627f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 -- 172.21.15.90:0/2913305038 shutdown_connections 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.821+0000 7f512803d740 1 -- 172.21.15.90:0/2913305038 wait complete. 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.822+0000 7f512803d740 1 -- start start 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.822+0000 7f512803d740 1 Processor -- start 2024-04-26T18:32:42.824 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.822+0000 7f512803d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac65f00 0x55bb7ac6e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.822+0000 7f512803d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.822+0000 7f512803d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac7cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.822+0000 7f512803d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55bb7ab86590 con 0x55bb7ac6ed20 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f511effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac65f00 0x55bb7ac6e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f511effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac65f00 0x55bb7ac6e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:38520/0 (socket says 172.21.15.90:38520) 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f511effd640 1 -- 172.21.15.90:0/1451108523 learned_addr learned my addr 172.21.15.90:0/1451108523 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55bb7ab53b60 con 0x55bb7ac65f00 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bb7ac73ad0 con 0x55bb7ac710e0 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f5117fff640 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f511f7fe640 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac7cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.823+0000 7f512803d740 10 client.0 _ll_register_callbacks cb 0x55bb7ab782b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.824+0000 7f5117fff640 1 -- 172.21.15.90:0/1451108523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac65f00 msgr2=0x55bb7ac6e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.824+0000 7f5117fff640 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac65f00 0x55bb7ac6e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.824+0000 7f5117fff640 1 -- 172.21.15.90:0/1451108523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 msgr2=0x55bb7ac7cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.824+0000 7f5117fff640 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac7cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.824+0000 7f5117fff640 1 -- 172.21.15.90:0/1451108523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7a8de210 con 0x55bb7ac710e0 2024-04-26T18:32:42.826 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.825+0000 7f5117fff640 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6e7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55bb7ac635a0 tx=0x7f51080128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c0195a6c80bd0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.825+0000 7f51067fc640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:32:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.825+0000 7f51067fc640 1 -- 172.21.15.90:0/1451108523 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108004460 con 0x55bb7ac710e0 2024-04-26T18:32:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.825+0000 7f51067fc640 1 -- 172.21.15.90:0/1451108523 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 12497+0+0 (secure 0 0 0) 0x7f510801b970 con 0x55bb7ac710e0 2024-04-26T18:32:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.826+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb7abda070 con 0x55bb7ac710e0 2024-04-26T18:32:42.827 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.826+0000 7f51067fc640 1 -- 172.21.15.90:0/1451108523 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f510801bbb0 con 0x55bb7ac710e0 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.827+0000 7f512803d740 10 client.26107 fetch_fsmap learned FSMap version 291 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.827+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=4+}) -- 0x55bb7a9aa620 con 0x55bb7ac710e0 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.827+0000 7f51067fc640 1 -- 172.21.15.90:0/1451108523 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f510801f070 con 0x55bb7ac710e0 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.827+0000 7f51067fc640 1 -- 172.21.15.90:0/1451108523 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 291) ==== 4716+0+0 (secure 0 0 0) 0x7f5108004b10 con 0x55bb7ac710e0 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.828+0000 7f512803d740 10 client.26107 fetch_fsmap finished waiting for FSMap version 291 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.828+0000 7f512803d740 10 client.26107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.828+0000 7f512803d740 20 client.26107 populate_metadata read hostname 'smithi090' 2024-04-26T18:32:42.828 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.828+0000 7f512803d740 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7ac83db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.828+0000 7f512803d740 4 client.26107 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.828+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7abda070 con 0x55bb7ac81930 2024-04-26T18:32:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.828+0000 7f511effd640 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7ac83db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.829+0000 7f511effd640 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7ac83db0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e795812d72555f56 server_cookie=5b07176771081e63 in_seq=0 out_seq=0 2024-04-26T18:32:42.829 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.829+0000 7f51067fc640 10 client.26107 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f51067fc640 1 -- 172.21.15.90:0/1451108523 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f51100ccae0 con 0x55bb7ac81930 2024-04-26T18:32:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f51067fc640 10 client.26107 handle_command_reply: tid=0 2024-04-26T18:32:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 client.26107 shutdown 2024-04-26T18:32:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 msgr2=0x55bb7ac6e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6e7e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55bb7ac635a0 tx=0x7f51080128e0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 shutdown_connections 2024-04-26T18:32:42.833 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7ac83db0 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f51067fc640 0 client.26107 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac7cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6e7e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 --2- 172.21.15.90:0/1451108523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac65f00 0x55bb7ac6e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 >> 172.21.15.90:0/1451108523 conn(0x55bb7ac62410 msgr2=0x55bb7ac68190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 shutdown_connections 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.832+0000 7f512803d740 1 -- 172.21.15.90:0/1451108523 wait complete. 2024-04-26T18:32:42.834 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.833+0000 7f512803d740 20 client.26107 trim_cache size 0 max 16384 2024-04-26T18:32:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 Processor -- start 2024-04-26T18:32:42.858 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 -- start start 2024-04-26T18:32:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac68910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac89680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac710e0 0x55bb7ac89bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f51201aef10 con 0x55bb7ac6ed20 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51201acca0 con 0x55bb7ac710e0 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f512803d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51201ae840 con 0x55bb7ac65f00 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f5117fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac89680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f5117fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac89680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42740/0 (socket says 172.21.15.90:42740) 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f5117fff640 1 -- 172.21.15.90:0/1658150584 learned_addr learned my addr 172.21.15.90:0/1658150584 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f511effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac68910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.858+0000 7f511f7fe640 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac710e0 0x55bb7ac89bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f511effd640 1 -- 172.21.15.90:0/1658150584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac710e0 msgr2=0x55bb7ac89bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.860 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f511effd640 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac710e0 0x55bb7ac89bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f511effd640 1 -- 172.21.15.90:0/1658150584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 msgr2=0x55bb7ac89680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f511effd640 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac89680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f511effd640 1 -- 172.21.15.90:0/1658150584 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7ac66640 con 0x55bb7ac65f00 2024-04-26T18:32:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f511effd640 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac68910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5110009850 tx=0x7f51100bcf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59b6d1f3fc49ef27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f5114ff9640 1 -- 172.21.15.90:0/1658150584 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51100d5020 con 0x55bb7ac65f00 2024-04-26T18:32:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f5114ff9640 1 -- 172.21.15.90:0/1658150584 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bb7abda070 con 0x55bb7ac65f00 2024-04-26T18:32:42.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.859+0000 7f5114ff9640 1 -- 172.21.15.90:0/1658150584 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5110049040 con 0x55bb7ac65f00 2024-04-26T18:32:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.860+0000 7f512803d740 1 -- 172.21.15.90:0/1658150584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 msgr2=0x55bb7ac68910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.860+0000 7f512803d740 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac68910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5110009850 tx=0x7f51100bcf00 comp rx=0 tx=0).stop 2024-04-26T18:32:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.860+0000 7f512803d740 1 -- 172.21.15.90:0/1658150584 shutdown_connections 2024-04-26T18:32:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac710e0 0x55bb7ac89bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac89680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 --2- 172.21.15.90:0/1658150584 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac68910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 -- 172.21.15.90:0/1658150584 >> 172.21.15.90:0/1658150584 conn(0x55bb7ac62410 msgr2=0x55bb7ac69500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 -- 172.21.15.90:0/1658150584 shutdown_connections 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 -- 172.21.15.90:0/1658150584 wait complete. 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 -- start start 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 Processor -- start 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.861+0000 7f512803d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac79290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f512803d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac6ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f511effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac79290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f511effd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac79290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:58736/0 (socket says 172.21.15.90:58736) 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f511effd640 1 -- 172.21.15.90:0/4159939269 learned_addr learned my addr 172.21.15.90:0/4159939269 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f512803d740 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f51201ada40 con 0x55bb7ac710e0 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f5117fff640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac6ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f51201ad370 con 0x55bb7ac6ed20 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f511f7fe640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51201ae110 con 0x55bb7ac65f00 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.862+0000 7f512803d740 10 client.0 _ll_register_callbacks cb 0x7f5108009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f5117fff640 1 -- 172.21.15.90:0/4159939269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 msgr2=0x55bb7ac79290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f5117fff640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac79290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f5117fff640 1 -- 172.21.15.90:0/4159939269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac710e0 msgr2=0x55bb7ac6b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f5117fff640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f5117fff640 1 -- 172.21.15.90:0/4159939269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7ace6ef0 con 0x55bb7ac6ed20 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f511f7fe640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f5117fff640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac6ae60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5108000c00 tx=0x7f510800e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e1b1f6834e9fea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:42.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f51067fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f51067fc640 1 -- 172.21.15.90:0/4159939269 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108003da0 con 0x55bb7ac6ed20 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb7abda070 con 0x55bb7ac6ed20 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f51067fc640 1 -- 172.21.15.90:0/4159939269 <== mon.2 v2:172.21.15.119:3300/0 2 ==== osd_map(170..170 src has 1..170) ==== 12497+0+0 (secure 0 0 0) 0x7f5108005020 con 0x55bb7ac6ed20 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f51067fc640 1 -- 172.21.15.90:0/4159939269 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=291) ==== 24+0+0 (secure 0 0 0) 0x7f5108005240 con 0x55bb7ac6ed20 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f512803d740 10 client.26246 fetch_fsmap learned FSMap version 291 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.863+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=291,monmap=4+}) -- 0x55bb7ace8080 con 0x55bb7ac6ed20 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f51067fc640 1 -- 172.21.15.90:0/4159939269 <== mon.2 v2:172.21.15.119:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bb7abda070 con 0x55bb7ac6ed20 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f51067fc640 1 -- 172.21.15.90:0/4159939269 <== mon.2 v2:172.21.15.119:3300/0 5 ==== fsmap(e 291) ==== 4716+0+0 (secure 0 0 0) 0x7f510800e940 con 0x55bb7ac6ed20 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f512803d740 10 client.26246 fetch_fsmap finished waiting for FSMap version 291 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f512803d740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f512803d740 20 client.26246 populate_metadata read hostname 'smithi090' 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f512803d740 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7acea870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:42.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f512803d740 4 client.26246 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:32:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb7abd3ee0 con 0x55bb7ac81930 2024-04-26T18:32:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.864+0000 7f511effd640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7acea870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.865+0000 7f51067fc640 10 client.26246 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:42.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:42.865+0000 7f511effd640 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7acea870 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70aab6526ca6a729 server_cookie=3df02b0acd73d503 in_seq=0 out_seq=0 2024-04-26T18:32:42.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44026 7.15 MB/sec execute 247 sec latency 5638.288 ms 2024-04-26T18:32:43.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-mon[29180]: pgmap v719: 97 pgs: 97 active+clean; 680 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 5.6 MiB/s wr, 390 op/s 2024-04-26T18:32:43.513 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6dba952700 time 2024-04-26T18:32:43.487760+0000 2024-04-26T18:32:43.513 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:43.513 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6dc4f7b3dd] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.514 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: clone() 2024-04-26T18:32:43.515 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.488+0000 7f6dba952700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6dba952700 time 2024-04-26T18:32:43.487760+0000 2024-04-26T18:32:43.515 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:43.515 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6dc4f7b3dd] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.516 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: clone() 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: *** Caught signal (Aborted) ** 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: in thread 7f6dba952700 thread_name:ms_dispatch 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6dc3d03d20] 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: gsignal() 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: abort() 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6dc4f7b437] 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.517 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 16: clone() 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.490+0000 7f6dba952700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: in thread 7f6dba952700 thread_name:ms_dispatch 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.518 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6dc3d03d20] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: gsignal() 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: abort() 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6dc4f7b437] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.519 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 16: clone() 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: -1> 2024-04-26T18:32:43.488+0000 7f6dba952700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6dba952700 time 2024-04-26T18:32:43.487760+0000 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6dc4f7b3dd] 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.520 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: clone() 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 0> 2024-04-26T18:32:43.490+0000 7f6dba952700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: in thread 7f6dba952700 thread_name:ms_dispatch 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.521 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6dc3d03d20] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: gsignal() 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: abort() 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6dc4f7b437] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.522 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.523 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.523 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 16: clone() 2024-04-26T18:32:43.523 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:43.523 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:43 smithi119 ceph-mon[32752]: pgmap v719: 97 pgs: 97 active+clean; 680 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 5.6 MiB/s wr, 390 op/s 2024-04-26T18:32:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:43 smithi167 ceph-mon[32751]: pgmap v719: 97 pgs: 97 active+clean; 680 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 5.6 MiB/s wr, 390 op/s 2024-04-26T18:32:43.876 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: -1> 2024-04-26T18:32:43.488+0000 7f6dba952700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f6dba952700 time 2024-04-26T18:32:43.487760+0000 2024-04-26T18:32:43.876 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f6dc4f7b3dd] 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.877 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.878 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.878 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.878 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.878 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.878 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: clone() 2024-04-26T18:32:43.878 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.879 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 0> 2024-04-26T18:32:43.490+0000 7f6dba952700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:43.879 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: in thread 7f6dba952700 thread_name:ms_dispatch 2024-04-26T18:32:43.879 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.879 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:43.879 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f6dc3d03d20] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2: gsignal() 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 3: abort() 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f6dc4f7b437] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f6dc4f7b5a3] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x557a2faf2822] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x557a2fa62598] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x557a2fa64c17] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x557a2f95dd0b] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x557a2f95e2ac] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x557a2f946fad] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f6dc520fe20] 2024-04-26T18:32:43.880 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 13: (DispatchQueue::entry()+0x51f) [0x7f6dc520cfdf] 2024-04-26T18:32:43.881 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f6dc52dffc1] 2024-04-26T18:32:43.881 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f6dc3cf91ca] 2024-04-26T18:32:43.881 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 16: clone() 2024-04-26T18:32:43.881 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:43.881 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a[99353]: 2024-04-26T18:32:43.881 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:43 smithi090 podman[100506]: 2024-04-26 18:32:43.56616537 +0000 UTC m=+0.019584672 container died a612cacab2d8088069a242a7dc5d887ce3796896983e26ca077b022b975da7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:32:43.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 7.17 MB/sec execute 248 sec latency 6638.419 ms 2024-04-26T18:32:44.900 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 7.15 MB/sec execute 249 sec latency 7638.484 ms 2024-04-26T18:32:45.539 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:45 smithi090 ceph-mon[29180]: pgmap v720: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-26T18:32:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:45 smithi119 ceph-mon[32752]: pgmap v720: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-26T18:32:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:45 smithi167 ceph-mon[32751]: pgmap v720: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-26T18:32:45.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 7.12 MB/sec execute 250 sec latency 8638.530 ms 2024-04-26T18:32:46.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 7.09 MB/sec execute 251 sec latency 9638.650 ms 2024-04-26T18:32:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:47 smithi119 ceph-mon[32752]: pgmap v721: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-26T18:32:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:47 smithi167 ceph-mon[32751]: pgmap v721: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-26T18:32:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:47 smithi090 ceph-mon[29180]: pgmap v721: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 6.8 MiB/s wr, 549 op/s 2024-04-26T18:32:47.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 7.06 MB/sec execute 252 sec latency 10638.694 ms 2024-04-26T18:32:48.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 7.03 MB/sec execute 253 sec latency 11638.736 ms 2024-04-26T18:32:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-mon[29180]: pgmap v722: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 6.5 MiB/s wr, 520 op/s 2024-04-26T18:32:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:49 smithi119 ceph-mon[32752]: pgmap v722: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 6.5 MiB/s wr, 520 op/s 2024-04-26T18:32:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:49 smithi167 ceph-mon[32751]: pgmap v722: 97 pgs: 97 active+clean; 710 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 6.5 MiB/s wr, 520 op/s 2024-04-26T18:32:49.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f7f54d28700 time 2024-04-26T18:32:49.555974+0000 2024-04-26T18:32:49.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:49.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7f5f3513dd] 2024-04-26T18:32:49.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.876 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: clone() 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.556+0000 7f7f54d28700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f7f54d28700 time 2024-04-26T18:32:49.555974+0000 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:49.877 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7f5f3513dd] 2024-04-26T18:32:49.878 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: clone() 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.879 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: *** Caught signal (Aborted) ** 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: in thread 7f7f54d28700 thread_name:ms_dispatch 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7f5e0d9d20] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: gsignal() 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: abort() 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7f5f351437] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.880 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 16: clone() 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.557+0000 7f7f54d28700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: in thread 7f7f54d28700 thread_name:ms_dispatch 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7f5e0d9d20] 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: gsignal() 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: abort() 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7f5f351437] 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.881 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 16: clone() 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.882 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: -1> 2024-04-26T18:32:49.556+0000 7f7f54d28700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f7f54d28700 time 2024-04-26T18:32:49.555974+0000 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7f5f3513dd] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.883 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: clone() 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 0> 2024-04-26T18:32:49.557+0000 7f7f54d28700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: in thread 7f7f54d28700 thread_name:ms_dispatch 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7f5e0d9d20] 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: gsignal() 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: abort() 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7f5f351437] 2024-04-26T18:32:49.884 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 16: clone() 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:49.885 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: -1> 2024-04-26T18:32:49.556+0000 7f7f54d28700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f7f54d28700 time 2024-04-26T18:32:49.555974+0000 2024-04-26T18:32:49.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:32:49.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f7f5f3513dd] 2024-04-26T18:32:49.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.886 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: clone() 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 0> 2024-04-26T18:32:49.557+0000 7f7f54d28700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:32:49.887 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: in thread 7f7f54d28700 thread_name:ms_dispatch 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f7f5e0d9d20] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2: gsignal() 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 3: abort() 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f7f5f351437] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f7f5f3515a3] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x560dff685822] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x560dff5f5598] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x560dff5f7c17] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x560dff4f0d0b] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x560dff4f12ac] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x560dff4d9fad] 2024-04-26T18:32:49.888 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f7f5f5e5e20] 2024-04-26T18:32:49.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 13: (DispatchQueue::entry()+0x51f) [0x7f7f5f5e2fdf] 2024-04-26T18:32:49.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f7f5f6b5fc1] 2024-04-26T18:32:49.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f7f5e0cf1ca] 2024-04-26T18:32:49.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 16: clone() 2024-04-26T18:32:49.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:32:49.889 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:49 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[99462]: 2024-04-26T18:32:49.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 7.01 MB/sec execute 254 sec latency 12638.797 ms 2024-04-26T18:32:50.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.98 MB/sec execute 255 sec latency 13638.842 ms 2024-04-26T18:32:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:51 smithi119 ceph-mon[32752]: pgmap v723: 97 pgs: 97 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 11 MiB/s wr, 571 op/s 2024-04-26T18:32:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:51 smithi167 ceph-mon[32751]: pgmap v723: 97 pgs: 97 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 11 MiB/s wr, 571 op/s 2024-04-26T18:32:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:51 smithi090 ceph-mon[29180]: pgmap v723: 97 pgs: 97 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 11 MiB/s wr, 571 op/s 2024-04-26T18:32:51.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.95 MB/sec execute 256 sec latency 14638.947 ms 2024-04-26T18:32:52.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.92 MB/sec execute 257 sec latency 15639.001 ms 2024-04-26T18:32:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:53 smithi119 ceph-mon[32752]: pgmap v724: 97 pgs: 97 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 385 op/s 2024-04-26T18:32:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:53 smithi167 ceph-mon[32751]: pgmap v724: 97 pgs: 97 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 385 op/s 2024-04-26T18:32:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:53 smithi090 ceph-mon[29180]: pgmap v724: 97 pgs: 97 active+clean; 769 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 7.8 MiB/s wr, 385 op/s 2024-04-26T18:32:53.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.90 MB/sec execute 258 sec latency 16639.081 ms 2024-04-26T18:32:54.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.87 MB/sec execute 259 sec latency 17639.141 ms 2024-04-26T18:32:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:55 smithi119 ceph-mon[32752]: pgmap v725: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 424 op/s 2024-04-26T18:32:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:55 smithi167 ceph-mon[32751]: pgmap v725: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 424 op/s 2024-04-26T18:32:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:55 smithi090 ceph-mon[29180]: pgmap v725: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 424 op/s 2024-04-26T18:32:55.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:55 smithi090 podman[100524]: 2024-04-26 18:32:55.331014378 +0000 UTC m=+5.733880427 container died 139db8389c6f668d8f1f60932454a0092b472369b35b2388f20c29b4f8a2ee0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T18:32:55.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:55 smithi090 podman[100506]: 2024-04-26 18:32:55.254663241 +0000 UTC m=+11.708082592 container remove a612cacab2d8088069a242a7dc5d887ce3796896983e26ca077b022b975da7bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS) 2024-04-26T18:32:55.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:55 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:32:55.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.84 MB/sec execute 260 sec latency 18639.234 ms 2024-04-26T18:32:56.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.82 MB/sec execute 261 sec latency 19639.343 ms 2024-04-26T18:32:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:57 smithi119 ceph-mon[32752]: pgmap v726: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 5.5 MiB/s wr, 152 op/s 2024-04-26T18:32:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:57 smithi167 ceph-mon[32751]: pgmap v726: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 5.5 MiB/s wr, 152 op/s 2024-04-26T18:32:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:57 smithi090 ceph-mon[29180]: pgmap v726: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 5.5 MiB/s wr, 152 op/s 2024-04-26T18:32:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f51067fc640 1 -- 172.21.15.90:0/4159939269 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb7abd3ee0 con 0x55bb7ac81930 2024-04-26T18:32:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f51067fc640 10 client.26246 handle_command_reply: tid=0 2024-04-26T18:32:57.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 client.26246 shutdown 2024-04-26T18:32:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac6ed20 msgr2=0x55bb7ac6ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac6ae60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5108000c00 tx=0x7f510800e3b0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 shutdown_connections 2024-04-26T18:32:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55bb7ac81930 0x55bb7acea870 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.706 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55bb7ac710e0 0x55bb7ac6b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f51067fc640 0 client.26246 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:32:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55bb7ac6ed20 0x55bb7ac6ae60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 --2- 172.21.15.90:0/4159939269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bb7ac65f00 0x55bb7ac79290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 >> 172.21.15.90:0/4159939269 conn(0x55bb7ac62410 msgr2=0x55bb7ac60d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 shutdown_connections 2024-04-26T18:32:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 1 -- 172.21.15.90:0/4159939269 wait complete. 2024-04-26T18:32:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f512803d740 20 client.26246 trim_cache size 0 max 16384 2024-04-26T18:32:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:32:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:32:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:32:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:32:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f510c0a5fd0 msgr2=0x7f510c0a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f510c0a5fd0 0x7f510c0a8490 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5118005fd0 tx=0x7f5118004ab0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5120101830 msgr2=0x7f5120196d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5120101830 0x7f5120196d40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f511002a940 tx=0x7f5110030650 comp rx=0 tx=0).stop 2024-04-26T18:32:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 shutdown_connections 2024-04-26T18:32:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f510c0a5fd0 0x7f510c0a8490 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5120107840 0x7f512019b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51201021f0 0x7f5120197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 --2- 172.21.15.90:0/2828307473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5120101830 0x7f5120196d40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 >> 172.21.15.90:0/2828307473 conn(0x7f5120078980 msgr2=0x7f5120104220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 shutdown_connections 2024-04-26T18:32:57.712 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:32:57.705+0000 7f5125322640 1 -- 172.21.15.90:0/2828307473 wait complete. 2024-04-26T18:32:57.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.79 MB/sec execute 262 sec latency 20639.467 ms 2024-04-26T18:32:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:58 smithi119 ceph-mon[32752]: Replacing daemon mds.a as rank 2 with standby daemon mds.d 2024-04-26T18:32:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:58 smithi119 ceph-mon[32752]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:32:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:58 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:32:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:58 smithi119 ceph-mon[32752]: osdmap e171: 12 total, 12 up, 12 in 2024-04-26T18:32:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:58 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:32:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:58 smithi167 ceph-mon[32751]: Replacing daemon mds.a as rank 2 with standby daemon mds.d 2024-04-26T18:32:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:58 smithi167 ceph-mon[32751]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:32:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:58 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:32:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:58 smithi167 ceph-mon[32751]: osdmap e171: 12 total, 12 up, 12 in 2024-04-26T18:32:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:58 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:32:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:58 smithi090 ceph-mon[29180]: Replacing daemon mds.a as rank 2 with standby daemon mds.d 2024-04-26T18:32:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:58 smithi090 ceph-mon[29180]: MDS daemon mds.a is removed because it is dead or otherwise unavailable. 2024-04-26T18:32:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:58 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:32:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:58 smithi090 ceph-mon[29180]: osdmap e171: 12 total, 12 up, 12 in 2024-04-26T18:32:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:58 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:32:58.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.77 MB/sec execute 263 sec latency 21639.536 ms 2024-04-26T18:32:58.938 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:58 smithi090 podman[100524]: 2024-04-26 18:32:58.923617927 +0000 UTC m=+9.326483991 container remove 139db8389c6f668d8f1f60932454a0092b472369b35b2388f20c29b4f8a2ee0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T18:32:58.938 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:58 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:32:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:59 smithi119 ceph-mon[32752]: pgmap v728: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.6 MiB/s wr, 183 op/s 2024-04-26T18:32:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:resolve 2024-04-26T18:32:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:32:59 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:resolve} 2 up:standby-replay 2024-04-26T18:32:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:59 smithi167 ceph-mon[32751]: pgmap v728: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.6 MiB/s wr, 183 op/s 2024-04-26T18:32:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:resolve 2024-04-26T18:32:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:32:59 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:resolve} 2 up:standby-replay 2024-04-26T18:32:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:59 smithi090 ceph-mon[29180]: pgmap v728: 97 pgs: 97 active+clean; 753 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.6 MiB/s wr, 183 op/s 2024-04-26T18:32:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:resolve 2024-04-26T18:32:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:32:59 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:resolve} 2 up:standby-replay 2024-04-26T18:32:59.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:59 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-04-26T18:32:59.626 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:32:59 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 23.208s CPU time. 2024-04-26T18:32:59.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:59 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Failed with result 'exit-code'. 2024-04-26T18:32:59.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:32:59 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 26.314s CPU time. 2024-04-26T18:32:59.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.74 MB/sec execute 264 sec latency 22639.619 ms 2024-04-26T18:33:00.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.71 MB/sec execute 265 sec latency 23639.715 ms 2024-04-26T18:33:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:01 smithi119 ceph-mon[32752]: pgmap v729: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 582 KiB/s wr, 55 op/s 2024-04-26T18:33:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:01 smithi167 ceph-mon[32751]: pgmap v729: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 582 KiB/s wr, 55 op/s 2024-04-26T18:33:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:01 smithi090 ceph-mon[29180]: pgmap v729: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 582 KiB/s wr, 55 op/s 2024-04-26T18:33:01.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.69 MB/sec execute 266 sec latency 24639.835 ms 2024-04-26T18:33:02.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.66 MB/sec execute 267 sec latency 25639.911 ms 2024-04-26T18:33:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:03 smithi119 ceph-mon[32752]: pgmap v730: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 582 KiB/s wr, 55 op/s 2024-04-26T18:33:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:03 smithi119 ceph-mon[32752]: Replacing daemon mds.g as rank 1 with standby daemon mds.j 2024-04-26T18:33:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:03 smithi119 ceph-mon[32752]: MDS daemon mds.g is removed because it is dead or otherwise unavailable. 2024-04-26T18:33:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:03 smithi119 ceph-mon[32752]: osdmap e172: 12 total, 12 up, 12 in 2024-04-26T18:33:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:03 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:resolve} 1 up:standby-replay 2024-04-26T18:33:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:03 smithi167 ceph-mon[32751]: pgmap v730: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 582 KiB/s wr, 55 op/s 2024-04-26T18:33:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:03 smithi167 ceph-mon[32751]: Replacing daemon mds.g as rank 1 with standby daemon mds.j 2024-04-26T18:33:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:03 smithi167 ceph-mon[32751]: MDS daemon mds.g is removed because it is dead or otherwise unavailable. 2024-04-26T18:33:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:03 smithi167 ceph-mon[32751]: osdmap e172: 12 total, 12 up, 12 in 2024-04-26T18:33:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:03 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:resolve} 1 up:standby-replay 2024-04-26T18:33:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:03 smithi090 ceph-mon[29180]: pgmap v730: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 584 KiB/s rd, 582 KiB/s wr, 55 op/s 2024-04-26T18:33:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:03 smithi090 ceph-mon[29180]: Replacing daemon mds.g as rank 1 with standby daemon mds.j 2024-04-26T18:33:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:03 smithi090 ceph-mon[29180]: MDS daemon mds.g is removed because it is dead or otherwise unavailable. 2024-04-26T18:33:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:03 smithi090 ceph-mon[29180]: osdmap e172: 12 total, 12 up, 12 in 2024-04-26T18:33:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:03 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:resolve} 1 up:standby-replay 2024-04-26T18:33:03.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.64 MB/sec execute 268 sec latency 26640.008 ms 2024-04-26T18:33:04.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44252 6.61 MB/sec execute 269 sec latency 27640.117 ms 2024-04-26T18:33:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:resolve 2024-04-26T18:33:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 2024-04-26T18:33:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:04 smithi119 ceph-mon[32752]: pgmap v732: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-26T18:33:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:resolve 2024-04-26T18:33:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 2024-04-26T18:33:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:04 smithi167 ceph-mon[32751]: pgmap v732: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-26T18:33:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:resolve 2024-04-26T18:33:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 2024-04-26T18:33:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:04 smithi090 ceph-mon[29180]: pgmap v732: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-26T18:33:05.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44296 6.60 MB/sec execute 270 sec latency 28640.163 ms 2024-04-26T18:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:05 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:05 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:reconnect 2024-04-26T18:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:05 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:reconnect 2024-04-26T18:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:05 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2024-04-26T18:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:05 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:05 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:33:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:05 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:33:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:05 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:reconnect 2024-04-26T18:33:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:05 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:reconnect 2024-04-26T18:33:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:05 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2024-04-26T18:33:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:05 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:33:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:05 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:33:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:05 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:05 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:reconnect 2024-04-26T18:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:05 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:reconnect 2024-04-26T18:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:05 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 2024-04-26T18:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:05 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:05 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:33:06.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44497 6.59 MB/sec execute 271 sec latency 29640.200 ms 2024-04-26T18:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:rejoin 2024-04-26T18:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:rejoin 2024-04-26T18:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:06 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2024-04-26T18:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:06 smithi119 ceph-mon[32752]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:06 smithi119 ceph-mon[32752]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:06 smithi119 ceph-mon[32752]: pgmap v733: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:33:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:rejoin 2024-04-26T18:33:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:rejoin 2024-04-26T18:33:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:06 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2024-04-26T18:33:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:06 smithi167 ceph-mon[32751]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:33:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:06 smithi167 ceph-mon[32751]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:33:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:06 smithi167 ceph-mon[32751]: pgmap v733: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:33:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:rejoin 2024-04-26T18:33:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:rejoin 2024-04-26T18:33:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:06 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 2024-04-26T18:33:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:06 smithi090 ceph-mon[29180]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:33:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:06 smithi090 ceph-mon[29180]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:33:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:06 smithi090 ceph-mon[29180]: pgmap v733: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:33:07.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 44856 6.62 MB/sec execute 272 sec latency 29733.463 ms 2024-04-26T18:33:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:07 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:33:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:active 2024-04-26T18:33:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:active 2024-04-26T18:33:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 2024-04-26T18:33:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:07 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:33:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:active 2024-04-26T18:33:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:active 2024-04-26T18:33:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 2024-04-26T18:33:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:07 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:33:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6840/2568313147,v1:172.21.15.90:6841/2568313147] up:active 2024-04-26T18:33:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/1881511598,v1:172.21.15.90:6837/1881511598] up:active 2024-04-26T18:33:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 2024-04-26T18:33:08.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:33:08 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:33:08.285+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:33:08.554 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:33:08 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:33:08.285+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:33:08.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 45537 6.71 MB/sec execute 273 sec latency 100.781 ms 2024-04-26T18:33:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:08 smithi119 ceph-mon[32752]: pgmap v734: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-04-26T18:33:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:08 smithi167 ceph-mon[32751]: pgmap v734: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-04-26T18:33:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:08 smithi090 ceph-mon[29180]: pgmap v734: 97 pgs: 97 active+clean; 734 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 33 KiB/s wr, 14 op/s 2024-04-26T18:33:09.655 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: Stopped Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 26.314s CPU time. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: Starting Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: Stopped Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Consumed 23.208s CPU time. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Start request repeated too quickly. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.a.service: Failed with result 'exit-code'. 2024-04-26T18:33:09.656 INFO:journalctl@ceph.mds.a.smithi090.stdout:Apr 26 18:33:09 smithi090 systemd[1]: Failed to start Ceph mds.a for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:09.902 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46244 6.89 MB/sec execute 274 sec latency 162.229 ms 2024-04-26T18:33:10.073 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:09 smithi090 podman[100659]: 2024-04-26 18:33:09.561687135 +0000 UTC m=+0.033176702 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:33:10.073 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:09 smithi090 podman[100659]: 2024-04-26 18:33:09.686337498 +0000 UTC m=+0.157827064 container create 5229c8d86f2b7035ae9fc129eb76edf594e178e534bb10d9f70fcbc19c5a9819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T18:33:10.376 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:10 smithi090 podman[100659]: 2024-04-26 18:33:10.073800803 +0000 UTC m=+0.545290368 container init 5229c8d86f2b7035ae9fc129eb76edf594e178e534bb10d9f70fcbc19c5a9819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-26T18:33:10.376 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:10 smithi090 podman[100659]: 2024-04-26 18:33:10.095497922 +0000 UTC m=+0.566987484 container start 5229c8d86f2b7035ae9fc129eb76edf594e178e534bb10d9f70fcbc19c5a9819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:33:10.376 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: starting mds.g at 2024-04-26T18:33:10.376 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:10 smithi090 bash[100659]: 5229c8d86f2b7035ae9fc129eb76edf594e178e534bb10d9f70fcbc19c5a9819 2024-04-26T18:33:10.376 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:10 smithi090 systemd[1]: Started Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.861+0000 7fe826389640 1 -- 172.21.15.90:0/815575874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe820101bc0 msgr2=0x7fe820105c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.861+0000 7fe826389640 1 --2- 172.21.15.90:0/815575874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe820101bc0 0x7fe820105c00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe80800b9f0 tx=0x7fe80802c0d0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.862+0000 7fe826389640 1 -- 172.21.15.90:0/815575874 shutdown_connections 2024-04-26T18:33:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.862+0000 7fe826389640 1 --2- 172.21.15.90:0/815575874 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe8201062e0 0x7fe82010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.862+0000 7fe826389640 1 --2- 172.21.15.90:0/815575874 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe820101bc0 0x7fe820105c00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.862 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.862+0000 7fe826389640 1 --2- 172.21.15.90:0/815575874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe820101210 0x7fe8201015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.862+0000 7fe826389640 1 -- 172.21.15.90:0/815575874 >> 172.21.15.90:0/815575874 conn(0x7fe820077dc0 msgr2=0x7fe8200ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.862+0000 7fe826389640 1 -- 172.21.15.90:0/815575874 shutdown_connections 2024-04-26T18:33:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.862+0000 7fe826389640 1 -- 172.21.15.90:0/815575874 wait complete. 2024-04-26T18:33:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.863+0000 7fe826389640 1 Processor -- start 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe826389640 1 -- start start 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe826389640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe820101210 0x7fe82019dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe826389640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8201062e0 0x7fe820196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe826389640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82019e360 0x7fe8201973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe826389640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe820110a70 con 0x7fe820101210 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe826389640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe8201108f0 con 0x7fe8201062e0 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe826389640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe820110bf0 con 0x7fe82019e360 2024-04-26T18:33:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.864+0000 7fe824926640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82019e360 0x7fe8201973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.865+0000 7fe824926640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82019e360 0x7fe8201973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:48528/0 (socket says 172.21.15.90:48528) 2024-04-26T18:33:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.865+0000 7fe824926640 1 -- 172.21.15.90:0/1387501596 learned_addr learned my addr 172.21.15.90:0/1387501596 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.865+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8201062e0 0x7fe820196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.866 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.865+0000 7fe81ffff640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe820101210 0x7fe82019dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.866+0000 7fe824926640 1 -- 172.21.15.90:0/1387501596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8201062e0 msgr2=0x7fe820196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.866+0000 7fe824926640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8201062e0 0x7fe820196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.866+0000 7fe824926640 1 -- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe820101210 msgr2=0x7fe82019dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.866+0000 7fe824926640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe820101210 0x7fe82019dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.866+0000 7fe824926640 1 -- 172.21.15.90:0/1387501596 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82019b830 con 0x7fe82019e360 2024-04-26T18:33:10.867 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.866+0000 7fe824926640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82019e360 0x7fe8201973a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe81000db10 tx=0x7fe81000a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.867+0000 7fe81d7fa640 1 -- 172.21.15.90:0/1387501596 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81000ae60 con 0x7fe82019e360 2024-04-26T18:33:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.867+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82019bb20 con 0x7fe82019e360 2024-04-26T18:33:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.867+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82019c040 con 0x7fe82019e360 2024-04-26T18:33:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.868+0000 7fe81d7fa640 1 -- 172.21.15.90:0/1387501596 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8100054d0 con 0x7fe82019e360 2024-04-26T18:33:10.868 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.868+0000 7fe81d7fa640 1 -- 172.21.15.90:0/1387501596 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81000e690 con 0x7fe82019e360 2024-04-26T18:33:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.871+0000 7fe82917b740 1 Processor -- start 2024-04-26T18:33:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.872+0000 7fe81d7fa640 1 -- 172.21.15.90:0/1387501596 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fe81000e830 con 0x7fe82019e360 2024-04-26T18:33:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.874+0000 7fe82917b740 1 -- start start 2024-04-26T18:33:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.874+0000 7fe82917b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d36f00 0x557349d372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe82917b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe82917b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d420e0 0x557349d444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe82917b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557349c9d460 con 0x557349d37820 2024-04-26T18:33:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe82917b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557349c57590 con 0x557349d36f00 2024-04-26T18:33:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe82917b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557349c99420 con 0x557349d420e0 2024-04-26T18:33:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe81f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d420e0 0x557349d444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe81f7fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d420e0 0x557349d444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:48542/0 (socket says 172.21.15.90:48542) 2024-04-26T18:33:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe81f7fe640 1 -- 172.21.15.90:0/353201087 learned_addr learned my addr 172.21.15.90:0/353201087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d36f00 0x557349d372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.875+0000 7fe81ffff640 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.878+0000 7fe81f7fe640 1 -- 172.21.15.90:0/353201087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d36f00 msgr2=0x557349d372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.878+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d36f00 0x557349d372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.878+0000 7fe81f7fe640 1 -- 172.21.15.90:0/353201087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 msgr2=0x557349d41ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.878+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d41ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.879 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.878+0000 7fe81f7fe640 1 -- 172.21.15.90:0/353201087 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557349c640d0 con 0x557349d420e0 2024-04-26T18:33:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.879+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d420e0 0x557349d444d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe8080057c0 tx=0x7fe808005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d9c73f55e970f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.879+0000 7fe8167fc640 1 -- 172.21.15.90:0/353201087 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe808039070 con 0x557349d420e0 2024-04-26T18:33:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.879+0000 7fe8167fc640 1 -- 172.21.15.90:0/353201087 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe80803a040 con 0x557349d420e0 2024-04-26T18:33:10.880 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.880+0000 7fe81d7fa640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe80c0a3ff0 0x7fe80c0a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.880+0000 7fe81d7fa640 1 -- 172.21.15.90:0/1387501596 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 12713+0+0 (secure 0 0 0) 0x7fe8100d1500 con 0x7fe82019e360 2024-04-26T18:33:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.880+0000 7fe82917b740 1 -- 172.21.15.90:0/353201087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d420e0 msgr2=0x557349d444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.880+0000 7fe82917b740 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d420e0 0x557349d444d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe8080057c0 tx=0x7fe808005730 comp rx=0 tx=0).stop 2024-04-26T18:33:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.881+0000 7fe82917b740 1 -- 172.21.15.90:0/353201087 shutdown_connections 2024-04-26T18:33:10.881 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.881+0000 7fe82917b740 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d420e0 0x557349d444d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.881+0000 7fe82917b740 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.881+0000 7fe82917b740 1 --2- 172.21.15.90:0/353201087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d36f00 0x557349d372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.881+0000 7fe82917b740 1 -- 172.21.15.90:0/353201087 >> 172.21.15.90:0/353201087 conn(0x557349d33410 msgr2=0x557349d337f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.883+0000 7fe82917b740 1 -- 172.21.15.90:0/353201087 shutdown_connections 2024-04-26T18:33:10.883 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.883+0000 7fe81ffff640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe80c0a3ff0 0x7fe80c0a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.884+0000 7fe82917b740 1 -- 172.21.15.90:0/353201087 wait complete. 2024-04-26T18:33:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.884+0000 7fe82917b740 1 -- start start 2024-04-26T18:33:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.884+0000 7fe82917b740 1 Processor -- start 2024-04-26T18:33:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.885+0000 7fe82917b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d36f00 0x557349d3efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.885+0000 7fe82917b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d37820 0x557349d3f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.885+0000 7fe82917b740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d3fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.885+0000 7fe82917b740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x557349d44d00 con 0x557349d36f00 2024-04-26T18:33:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.885+0000 7fe82917b740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x557349c24b60 con 0x557349d420e0 2024-04-26T18:33:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.885+0000 7fe82917b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557349d44e80 con 0x557349d37820 2024-04-26T18:33:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.885+0000 7fe82917b740 10 client.0 _ll_register_callbacks cb 0x557349c492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.886+0000 7fe81ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d36f00 0x557349d3efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.886+0000 7fe81ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d36f00 0x557349d3efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49238/0 (socket says 172.21.15.90:49238) 2024-04-26T18:33:10.886 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.886+0000 7fe81ffff640 1 -- 172.21.15.90:0/1520695563 learned_addr learned my addr 172.21.15.90:0/1520695563 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.888+0000 7fe81ffff640 1 -- 172.21.15.90:0/1520695563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d37820 msgr2=0x557349d3f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.888+0000 7fe81ffff640 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d37820 0x557349d3f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.888+0000 7fe81ffff640 1 -- 172.21.15.90:0/1520695563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 msgr2=0x557349d3fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.888+0000 7fe81ffff640 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d3fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.888+0000 7fe81ffff640 1 -- 172.21.15.90:0/1520695563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573499af210 con 0x557349d36f00 2024-04-26T18:33:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.889+0000 7fe81ffff640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe80c0a3ff0 0x7fe80c0a64b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe818009fd0 tx=0x7fe818009f40 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.890+0000 7fe81ffff640 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d36f00 0x557349d3efc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe818007680 tx=0x7fe818004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdcd1248cdd9410 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.890+0000 7fe7ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:33:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.890+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/1520695563 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818020070 con 0x557349d36f00 2024-04-26T18:33:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.890+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/1520695563 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 12713+0+0 (secure 0 0 0) 0x7fe81801b660 con 0x557349d36f00 2024-04-26T18:33:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.890+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe80803a040 con 0x557349d36f00 2024-04-26T18:33:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.890+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/1520695563 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=300) ==== 24+0+0 (secure 0 0 0) 0x7fe818015040 con 0x557349d36f00 2024-04-26T18:33:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 10 client.16908 fetch_fsmap learned FSMap version 300 2024-04-26T18:33:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=300,monmap=4+}) -- 0x557349a7b620 con 0x557349d36f00 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/1520695563 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe80803a040 con 0x557349d36f00 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/1520695563 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 300) ==== 4123+0+0 (secure 0 0 0) 0x7fe81801c950 con 0x557349d36f00 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 10 client.16908 fetch_fsmap finished waiting for FSMap version 300 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 20 client.16908 populate_metadata read hostname 'smithi090' 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349d528d0 0x557349d54d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 4 client.16908 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:10.892 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.891+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557349cab070 con 0x557349d528d0 2024-04-26T18:33:10.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.893+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349d528d0 0x557349d54d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.895+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349d528d0 0x557349d54d90 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a3e743b6b8a650c server_cookie=890b0760c1f8db82 in_seq=0 out_seq=0 2024-04-26T18:33:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.895+0000 7fe7ff7fe640 10 client.16908 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.897+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/1520695563 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557349cab070 con 0x557349d528d0 2024-04-26T18:33:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.897+0000 7fe7ff7fe640 10 client.16908 handle_command_reply: tid=0 2024-04-26T18:33:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.898+0000 7fe82917b740 1 client.16908 shutdown 2024-04-26T18:33:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.898+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d36f00 msgr2=0x557349d3efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.898+0000 7fe82917b740 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d36f00 0x557349d3efc0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe818007680 tx=0x7fe818004ad0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.903+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 shutdown_connections 2024-04-26T18:33:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.903+0000 7fe82917b740 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349d528d0 0x557349d54d90 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.903+0000 7fe82917b740 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d3fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.903+0000 7fe82917b740 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d37820 0x557349d3f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.903+0000 7fe82917b740 1 --2- 172.21.15.90:0/1520695563 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d36f00 0x557349d3efc0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.903+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 >> 172.21.15.90:0/1520695563 conn(0x557349d33410 msgr2=0x557349d325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.904+0000 7fe7ff7fe640 0 client.16908 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:10.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46524 6.89 MB/sec execute 275 sec latency 191.482 ms 2024-04-26T18:33:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.907+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 shutdown_connections 2024-04-26T18:33:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.907+0000 7fe82917b740 1 -- 172.21.15.90:0/1520695563 wait complete. 2024-04-26T18:33:10.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.908+0000 7fe82917b740 20 client.16908 trim_cache size 0 max 16384 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.941+0000 7fe82917b740 1 Processor -- start 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.942+0000 7fe82917b740 1 -- start start 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.942+0000 7fe82917b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.942+0000 7fe82917b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d4dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe81ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe81ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:48566/0 (socket says 172.21.15.90:48566) 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe81ffff640 1 -- 172.21.15.90:0/1137122459 learned_addr learned my addr 172.21.15.90:0/1137122459 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe82917b740 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d5a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe800010f90 con 0x557349d37820 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe80000f3f0 con 0x557349d420e0 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8000108c0 con 0x557349d36f00 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe824926640 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d5a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.943+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d4dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.944+0000 7fe824926640 1 -- 172.21.15.90:0/1137122459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 msgr2=0x557349d4e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.944+0000 7fe824926640 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.944+0000 7fe824926640 1 -- 172.21.15.90:0/1137122459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 msgr2=0x557349d4dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.944+0000 7fe824926640 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d4dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.944+0000 7fe824926640 1 -- 172.21.15.90:0/1137122459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557349d3df70 con 0x557349d420e0 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.945+0000 7fe824926640 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d5a6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8100cdca0 tx=0x7fe81009bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c172df4a337ac04f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.945+0000 7fe815ffb640 1 -- 172.21.15.90:0/1137122459 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a5160 con 0x557349d420e0 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.945+0000 7fe815ffb640 1 -- 172.21.15.90:0/1137122459 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe81009a7f0 con 0x557349d420e0 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.945+0000 7fe815ffb640 1 -- 172.21.15.90:0/1137122459 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a4f30 con 0x557349d420e0 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 msgr2=0x557349d5a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d5a6a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8100cdca0 tx=0x7fe81009bf10 comp rx=0 tx=0).stop 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 shutdown_connections 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d5a6a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d4dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 --2- 172.21.15.90:0/1137122459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 >> 172.21.15.90:0/1137122459 conn(0x557349d33410 msgr2=0x557349d337f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.946+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 shutdown_connections 2024-04-26T18:33:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.947+0000 7fe82917b740 1 -- 172.21.15.90:0/1137122459 wait complete. 2024-04-26T18:33:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.947+0000 7fe82917b740 1 -- start start 2024-04-26T18:33:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.947+0000 7fe82917b740 1 Processor -- start 2024-04-26T18:33:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.948+0000 7fe82917b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.948+0000 7fe82917b740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d45770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.948+0000 7fe81ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.948+0000 7fe81ffff640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.90:48574/0 (socket says 172.21.15.90:48574) 2024-04-26T18:33:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.948+0000 7fe81ffff640 1 -- 172.21.15.90:0/2902092229 learned_addr learned my addr 172.21.15.90:0/2902092229 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe82917b740 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d45cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fe8000101f0 con 0x557349d37820 2024-04-26T18:33:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe824926640 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d45cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fe80000fac0 con 0x557349d420e0 2024-04-26T18:33:10.950 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557349d400d0 con 0x557349d36f00 2024-04-26T18:33:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe82917b740 10 client.0 _ll_register_callbacks cb 0x7fe81800a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe81ffff640 1 -- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 msgr2=0x557349d45cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe81ffff640 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d45cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.951 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe81ffff640 1 -- 172.21.15.90:0/2902092229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 msgr2=0x557349d45770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe81ffff640 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d45770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.949+0000 7fe81ffff640 1 -- 172.21.15.90:0/2902092229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557349d46220 con 0x557349d36f00 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.950+0000 7fe81ffff640 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4a2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe818007800 tx=0x7fe818007830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25efef47becd6b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.950+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe81009a7f0 con 0x557349d36f00 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.950+0000 7fe7ff7fe640 10 client.26134 ms_handle_connect on v2:172.21.15.167:3300/0 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.950+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/2902092229 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe818020040 con 0x557349d36f00 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.951+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/2902092229 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(173..173 src has 1..173) ==== 12713+0+0 (secure 0 0 0) 0x7fe81801a0d0 con 0x557349d36f00 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.951+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/2902092229 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=300) ==== 24+0+0 (secure 0 0 0) 0x7fe81801a2d0 con 0x557349d36f00 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.951+0000 7fe82917b740 10 client.26134 fetch_fsmap learned FSMap version 300 2024-04-26T18:33:10.952 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.951+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({fsmap=300}) -- 0x557349db8ba0 con 0x557349d36f00 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/2902092229 <== mon.1 v2:172.21.15.167:3300/0 4 ==== fsmap(e 300) ==== 4123+0+0 (secure 0 0 0) 0x7fe81801c420 con 0x557349d36f00 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe82917b740 10 client.26134 fetch_fsmap finished waiting for FSMap version 300 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe82917b740 10 client.26134 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe82917b740 20 client.26134 populate_metadata read hostname 'smithi090' 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe82917b740 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349db9280 0x557349dbb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe82917b740 4 client.26134 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349db9280 0x557349dbb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.952+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe81009a7f0 con 0x557349db9280 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.953+0000 7fe81f7fe640 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349db9280 0x557349dbb6a0 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6041e22089c3a9e0 server_cookie=35960233407f834 in_seq=0 out_seq=0 2024-04-26T18:33:10.954 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:10.953+0000 7fe7ff7fe640 10 client.26134 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:10 smithi119 ceph-mon[32752]: osdmap e173: 12 total, 12 up, 12 in 2024-04-26T18:33:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:10 smithi119 ceph-mon[32752]: pgmap v736: 97 pgs: 97 active+clean; 727 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-04-26T18:33:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:10 smithi167 ceph-mon[32751]: osdmap e173: 12 total, 12 up, 12 in 2024-04-26T18:33:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:10 smithi167 ceph-mon[32751]: pgmap v736: 97 pgs: 97 active+clean; 727 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-04-26T18:33:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.110+0000 7fe7ff7fe640 1 -- 172.21.15.90:0/2902092229 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe81009a7f0 con 0x557349db9280 2024-04-26T18:33:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.110+0000 7fe7ff7fe640 10 client.26134 handle_command_reply: tid=0 2024-04-26T18:33:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.110+0000 7fe82917b740 1 client.26134 shutdown 2024-04-26T18:33:11.111 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.110+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 msgr2=0x557349d4a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.110+0000 7fe82917b740 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4a2d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe818007800 tx=0x7fe818007830 comp rx=0 tx=0).stop 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.110+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 shutdown_connections 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.110+0000 7fe82917b740 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x557349db9280 0x557349dbb6a0 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe82917b740 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557349d420e0 0x557349d45cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe82917b740 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x557349d37820 0x557349d45770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe82917b740 1 --2- 172.21.15.90:0/2902092229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557349d36f00 0x557349d4a2d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 >> 172.21.15.90:0/2902092229 conn(0x557349d33410 msgr2=0x557349d31d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe7ff7fe640 0 client.26134 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 shutdown_connections 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe82917b740 1 -- 172.21.15.90:0/2902092229 wait complete. 2024-04-26T18:33:11.112 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.111+0000 7fe82917b740 20 client.26134 trim_cache size 0 max 16384 2024-04-26T18:33:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:33:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:33:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:33:11.113 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:33:11.114 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.112+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe80c0a3ff0 msgr2=0x7fe80c0a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.112+0000 7fe826389640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe80c0a3ff0 0x7fe80c0a64b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe818009fd0 tx=0x7fe818009f40 comp rx=0 tx=0).stop 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.112+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82019e360 msgr2=0x7fe8201973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.112+0000 7fe826389640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82019e360 0x7fe8201973a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe81000db10 tx=0x7fe81000a250 comp rx=0 tx=0).stop 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 shutdown_connections 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fe80c0a3ff0 0x7fe80c0a64b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe82019e360 0x7fe8201973a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8201062e0 0x7fe820196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 --2- 172.21.15.90:0/1387501596 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe820101210 0x7fe82019dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 >> 172.21.15.90:0/1387501596 conn(0x7fe820077dc0 msgr2=0x7fe8200ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 shutdown_connections 2024-04-26T18:33:11.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:11.113+0000 7fe826389640 1 -- 172.21.15.90:0/1387501596 wait complete. 2024-04-26T18:33:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:10 smithi090 ceph-mon[29180]: osdmap e173: 12 total, 12 up, 12 in 2024-04-26T18:33:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:10 smithi090 ceph-mon[29180]: pgmap v736: 97 pgs: 97 active+clean; 727 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 2.7 MiB/s wr, 286 op/s 2024-04-26T18:33:11.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46933 6.93 MB/sec execute 276 sec latency 274.060 ms 2024-04-26T18:33:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:11 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:boot 2024-04-26T18:33:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:11 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:33:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:11 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:33:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:11 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:boot 2024-04-26T18:33:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:11 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:33:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:11 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:33:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:boot 2024-04-26T18:33:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:33:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:33:12.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f38d989e700 time 2024-04-26T18:33:11.907672+0000 2024-04-26T18:33:12.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:12.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f38e3ec73dd] 2024-04-26T18:33:12.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: clone() 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:11.908+0000 7f38d989e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f38d989e700 time 2024-04-26T18:33:11.907672+0000 2024-04-26T18:33:12.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:12.128 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.128 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.128 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f38e3ec73dd] 2024-04-26T18:33:12.128 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.128 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: clone() 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: *** Caught signal (Aborted) ** 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: in thread 7f38d989e700 thread_name:ms_dispatch 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f38e2c4fd20] 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: gsignal() 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: abort() 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f38e3ec7437] 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 16: clone() 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:11.910+0000 7f38d989e700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: in thread 7f38d989e700 thread_name:ms_dispatch 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f38e2c4fd20] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: gsignal() 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: abort() 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f38e3ec7437] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.132 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.133 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.133 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.133 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.133 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 16: clone() 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: -1> 2024-04-26T18:33:11.908+0000 7f38d989e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f38d989e700 time 2024-04-26T18:33:11.907672+0000 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f38e3ec73dd] 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.134 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: clone() 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.135 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 0> 2024-04-26T18:33:11.910+0000 7f38d989e700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: in thread 7f38d989e700 thread_name:ms_dispatch 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f38e2c4fd20] 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: gsignal() 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: abort() 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f38e3ec7437] 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 16: clone() 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: -1> 2024-04-26T18:33:11.908+0000 7f38d989e700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f38d989e700 time 2024-04-26T18:33:11.907672+0000 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:12.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f38e3ec73dd] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.138 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: clone() 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 0> 2024-04-26T18:33:11.910+0000 7f38d989e700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: in thread 7f38d989e700 thread_name:ms_dispatch 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f38e2c4fd20] 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2: gsignal() 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 3: abort() 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f38e3ec7437] 2024-04-26T18:33:12.139 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f38e3ec75a3] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x561bd9bfe822] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x561bd9b6e598] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x561bd9b70c17] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x561bd9a69d0b] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x561bd9a6a2ac] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x561bd9a52fad] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f38e415be20] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 13: (DispatchQueue::entry()+0x51f) [0x7f38e4158fdf] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f38e422bfc1] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f38e2c451ca] 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 16: clone() 2024-04-26T18:33:12.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:12.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100309]: 2024-04-26T18:33:12.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:11 smithi090 podman[100754]: 2024-04-26 18:33:11.977756174 +0000 UTC m=+0.026322643 container died e02271592453b69a35d5ee24ec1a11cbf2ebadd9f9874f8cb72001f7b41188df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T18:33:12.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.91 MB/sec execute 277 sec latency 1092.473 ms 2024-04-26T18:33:12.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:12 smithi090 ceph-mon[29180]: pgmap v737: 97 pgs: 97 active+clean; 727 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-04-26T18:33:12.960 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:12 smithi090 podman[100754]: 2024-04-26 18:33:12.697223079 +0000 UTC m=+0.745789553 container remove e02271592453b69a35d5ee24ec1a11cbf2ebadd9f9874f8cb72001f7b41188df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T18:33:12.960 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:12 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:33:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:12 smithi119 ceph-mon[32752]: pgmap v737: 97 pgs: 97 active+clean; 727 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-04-26T18:33:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:12 smithi167 ceph-mon[32751]: pgmap v737: 97 pgs: 97 active+clean; 727 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.2 MiB/s wr, 240 op/s 2024-04-26T18:33:13.375 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:12 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-26T18:33:13.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:12 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 19.022s CPU time. 2024-04-26T18:33:13.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.88 MB/sec execute 278 sec latency 2092.514 ms 2024-04-26T18:33:14.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.86 MB/sec execute 279 sec latency 3092.609 ms 2024-04-26T18:33:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:15 smithi119 ceph-mon[32752]: pgmap v738: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 512 op/s 2024-04-26T18:33:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:15 smithi167 ceph-mon[32751]: pgmap v738: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 512 op/s 2024-04-26T18:33:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:15 smithi090 ceph-mon[29180]: pgmap v738: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 512 op/s 2024-04-26T18:33:15.905 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.83 MB/sec execute 280 sec latency 4092.670 ms 2024-04-26T18:33:16.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.81 MB/sec execute 281 sec latency 5092.707 ms 2024-04-26T18:33:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:17 smithi119 ceph-mon[32752]: pgmap v739: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 512 op/s 2024-04-26T18:33:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:17 smithi167 ceph-mon[32751]: pgmap v739: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 512 op/s 2024-04-26T18:33:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:17 smithi090 ceph-mon[29180]: pgmap v739: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.1 MiB/s wr, 512 op/s 2024-04-26T18:33:17.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.79 MB/sec execute 282 sec latency 6092.840 ms 2024-04-26T18:33:18.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.76 MB/sec execute 283 sec latency 7092.877 ms 2024-04-26T18:33:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:18 smithi119 ceph-mon[32752]: osdmap e174: 12 total, 12 up, 12 in 2024-04-26T18:33:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:18 smithi119 ceph-mon[32752]: pgmap v741: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-04-26T18:33:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:18 smithi167 ceph-mon[32751]: osdmap e174: 12 total, 12 up, 12 in 2024-04-26T18:33:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:18 smithi167 ceph-mon[32751]: pgmap v741: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-04-26T18:33:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:18 smithi090 ceph-mon[29180]: osdmap e174: 12 total, 12 up, 12 in 2024-04-26T18:33:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:18 smithi090 ceph-mon[29180]: pgmap v741: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.5 MiB/s wr, 341 op/s 2024-04-26T18:33:19.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.74 MB/sec execute 284 sec latency 8092.944 ms 2024-04-26T18:33:20.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.71 MB/sec execute 285 sec latency 9092.994 ms 2024-04-26T18:33:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:21 smithi119 ceph-mon[32752]: pgmap v742: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-04-26T18:33:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:21 smithi167 ceph-mon[32751]: pgmap v742: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-04-26T18:33:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-mon[29180]: pgmap v742: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-04-26T18:33:21.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.69 MB/sec execute 286 sec latency 10093.043 ms 2024-04-26T18:33:22.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3e6e038700 time 2024-04-26T18:33:21.729325+0000 2024-04-26T18:33:22.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:22.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e786613dd] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.127 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: clone() 2024-04-26T18:33:22.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:21.729+0000 7f3e6e038700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3e6e038700 time 2024-04-26T18:33:21.729325+0000 2024-04-26T18:33:22.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:22.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e786613dd] 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: clone() 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: *** Caught signal (Aborted) ** 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: in thread 7f3e6e038700 thread_name:ms_dispatch 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e773e9d20] 2024-04-26T18:33:22.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: gsignal() 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: abort() 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e78661437] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 16: clone() 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:21.731+0000 7f3e6e038700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: in thread 7f3e6e038700 thread_name:ms_dispatch 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e773e9d20] 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: gsignal() 2024-04-26T18:33:22.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: abort() 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e78661437] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 16: clone() 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: -2> 2024-04-26T18:33:21.729+0000 7f3e6e038700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3e6e038700 time 2024-04-26T18:33:21.729325+0000 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e786613dd] 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: clone() 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 0> 2024-04-26T18:33:21.731+0000 7f3e6e038700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: in thread 7f3e6e038700 thread_name:ms_dispatch 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e773e9d20] 2024-04-26T18:33:22.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: gsignal() 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: abort() 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e78661437] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 16: clone() 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: -9> 2024-04-26T18:33:21.729+0000 7f3e6e038700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f3e6e038700 time 2024-04-26T18:33:21.729325+0000 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f3e786613dd] 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: clone() 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: -7> 2024-04-26T18:33:21.731+0000 7f3e6e038700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: in thread 7f3e6e038700 thread_name:ms_dispatch 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:22.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f3e773e9d20] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2: gsignal() 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 3: abort() 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f3e78661437] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f3e786615a3] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x563bd443b822] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x563bd43ab598] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x563bd43adc17] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x563bd42a6d0b] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x563bd42a72ac] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x563bd428ffad] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f3e788f5e20] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 13: (DispatchQueue::entry()+0x51f) [0x7f3e788f2fdf] 2024-04-26T18:33:22.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f3e789c5fc1] 2024-04-26T18:33:22.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f3e773df1ca] 2024-04-26T18:33:22.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 16: clone() 2024-04-26T18:33:22.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:22.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[100386]: 2024-04-26T18:33:22.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:21 smithi090 podman[100799]: 2024-04-26 18:33:21.789675417 +0000 UTC m=+0.020253889 container died fd56d7301a1a4df662621faf619ea2ddf3f518967b12f90b50ea3d6462066ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:33:22.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.67 MB/sec execute 287 sec latency 11093.072 ms 2024-04-26T18:33:23.038 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:33:23.039 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 systemd[1]: Stopped Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:23.039 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 19.022s CPU time. 2024-04-26T18:33:23.039 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 systemd[1]: Starting Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:33:23.040 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:22 smithi090 podman[100799]: 2024-04-26 18:33:22.788748064 +0000 UTC m=+1.019326519 container remove fd56d7301a1a4df662621faf619ea2ddf3f518967b12f90b50ea3d6462066ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-26T18:33:23.040 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:22 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:33:23.040 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:23 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-26T18:33:23.040 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:23 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 18.374s CPU time. 2024-04-26T18:33:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:23 smithi090 ceph-mon[29180]: pgmap v743: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-04-26T18:33:23.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 podman[100896]: 2024-04-26 18:33:23.155576561 +0000 UTC m=+0.018050597 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:33:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:23 smithi119 ceph-mon[32752]: pgmap v743: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-04-26T18:33:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:23 smithi167 ceph-mon[32751]: pgmap v743: 97 pgs: 97 active+clean; 701 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 295 op/s 2024-04-26T18:33:23.669 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 podman[100896]: 2024-04-26 18:33:23.417285206 +0000 UTC m=+0.279759242 container create 5e27c13e983031e3a3589a79ec84e76b3a7fe4621e5800efe89c36d1a6d9e09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T18:33:23.669 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 podman[100896]: 2024-04-26 18:33:23.644924572 +0000 UTC m=+0.507398611 container init 5e27c13e983031e3a3589a79ec84e76b3a7fe4621e5800efe89c36d1a6d9e09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T18:33:23.669 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 podman[100896]: 2024-04-26 18:33:23.647518592 +0000 UTC m=+0.509992629 container start 5e27c13e983031e3a3589a79ec84e76b3a7fe4621e5800efe89c36d1a6d9e09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424) 2024-04-26T18:33:23.669 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 ceph-mds[100915]: -- 172.21.15.90:0/2758067927 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593bb889a40 con 0x5593bb85d000 2024-04-26T18:33:23.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.64 MB/sec execute 288 sec latency 12093.105 ms 2024-04-26T18:33:24.067 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:33:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:33:23.699+0000 7f61aaa9b700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 8 2024-04-26T18:33:24.068 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:33:23 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:33:23.700+0000 7f61a8a97700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 9 2024-04-26T18:33:24.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: starting mds.d at 2024-04-26T18:33:24.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 bash[100896]: 5e27c13e983031e3a3589a79ec84e76b3a7fe4621e5800efe89c36d1a6d9e09f 2024-04-26T18:33:24.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:23 smithi090 systemd[1]: Started Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 -- 172.21.15.90:0/4134744455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94101760 msgr2=0x7fee94101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 --2- 172.21.15.90:0/4134744455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94101760 0x7fee94101b40 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fee84009900 tx=0x7fee8402a3f0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 -- 172.21.15.90:0/4134744455 shutdown_connections 2024-04-26T18:33:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 --2- 172.21.15.90:0/4134744455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee9410f310 0x7fee94111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 --2- 172.21.15.90:0/4134744455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee94102080 0x7fee9410edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.206 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 --2- 172.21.15.90:0/4134744455 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94101760 0x7fee94101b40 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 -- 172.21.15.90:0/4134744455 >> 172.21.15.90:0/4134744455 conn(0x7fee940789b0 msgr2=0x7fee94078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 -- 172.21.15.90:0/4134744455 shutdown_connections 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 -- 172.21.15.90:0/4134744455 wait complete. 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.203+0000 7fee98d04640 1 Processor -- start 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee98d04640 1 -- start start 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee98d04640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee94101760 0x7fee9419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee98d04640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94102080 0x7fee9419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee98d04640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee9410f310 0x7fee9419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee98d04640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fee94114dd0 con 0x7fee94102080 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee98d04640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee94114c50 con 0x7fee94101760 2024-04-26T18:33:24.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee98d04640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee94114f50 con 0x7fee9410f310 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94102080 0x7fee9419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94102080 0x7fee9419b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54604/0 (socket says 172.21.15.90:54604) 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 -- 172.21.15.90:0/3823212887 learned_addr learned my addr 172.21.15.90:0/3823212887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee9259c640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee94101760 0x7fee9419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee92d9d640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee9410f310 0x7fee9419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 -- 172.21.15.90:0/3823212887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee9410f310 msgr2=0x7fee9419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee9410f310 0x7fee9419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 -- 172.21.15.90:0/3823212887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee94101760 msgr2=0x7fee9419b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:24.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee94101760 0x7fee9419b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 -- 172.21.15.90:0/3823212887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee941a01b0 con 0x7fee94102080 2024-04-26T18:33:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee92d9d640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee9410f310 0x7fee9419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.204+0000 7fee91d9b640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94102080 0x7fee9419b750 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c920 tx=0x7fee8800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.205+0000 7fee837fe640 1 -- 172.21.15.90:0/3823212887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88007c70 con 0x7fee94102080 2024-04-26T18:33:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.205+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee941a04a0 con 0x7fee94102080 2024-04-26T18:33:24.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.205+0000 7fee837fe640 1 -- 172.21.15.90:0/3823212887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee88007e10 con 0x7fee94102080 2024-04-26T18:33:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.205+0000 7fee837fe640 1 -- 172.21.15.90:0/3823212887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88018820 con 0x7fee94102080 2024-04-26T18:33:24.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.205+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee941a7d30 con 0x7fee94102080 2024-04-26T18:33:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.207+0000 7fee837fe640 1 -- 172.21.15.90:0/3823212887 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fee8800f450 con 0x7fee94102080 2024-04-26T18:33:24.217 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee837fe640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fee7c0a3dc0 0x7fee7c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee9259c640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fee7c0a3dc0 0x7fee7c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee837fe640 1 -- 172.21.15.90:0/3823212887 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 12711+0+0 (secure 0 0 0) 0x7fee880d1a30 con 0x7fee94102080 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee9ba16740 1 Processor -- start 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee9ba16740 1 -- start start 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee9259c640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fee7c0a3dc0 0x7fee7c0a6280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fee84031440 tx=0x7fee840095f0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e943ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.218 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.208+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e9464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee9ba16740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559b3e89f460 con 0x559b3e9440e0 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee9ba16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559b3e859590 con 0x559b3e938f00 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee9ba16740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b3e89b420 con 0x559b3e939820 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e9464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e9464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54614/0 (socket says 172.21.15.90:54614) 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 -- 172.21.15.90:0/3263032300 learned_addr learned my addr 172.21.15.90:0/3263032300 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee91d9b640 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e943ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee9259c640 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 -- 172.21.15.90:0/3263032300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 msgr2=0x559b3e943ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e943ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 -- 172.21.15.90:0/3263032300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 msgr2=0x559b3e9392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.219 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 -- 172.21.15.90:0/3263032300 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b3e8660d0 con 0x559b3e9440e0 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee92d9d640 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e9464d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fee8c008db0 tx=0x7fee8c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ac5b693e9adbfd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee80ff9640 1 -- 172.21.15.90:0/3263032300 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00f830 con 0x559b3e9440e0 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee80ff9640 1 -- 172.21.15.90:0/3263032300 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c00de20 con 0x559b3e9440e0 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.209+0000 7fee80ff9640 1 -- 172.21.15.90:0/3263032300 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00ed40 con 0x559b3e9440e0 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 -- 172.21.15.90:0/3263032300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 msgr2=0x559b3e9464d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e9464d0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fee8c008db0 tx=0x7fee8c00d710 comp rx=0 tx=0).stop 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 -- 172.21.15.90:0/3263032300 shutdown_connections 2024-04-26T18:33:24.220 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e9464d0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e943ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 --2- 172.21.15.90:0/3263032300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 -- 172.21.15.90:0/3263032300 >> 172.21.15.90:0/3263032300 conn(0x559b3e935410 msgr2=0x559b3e9357f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 -- 172.21.15.90:0/3263032300 shutdown_connections 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.210+0000 7fee9ba16740 1 -- 172.21.15.90:0/3263032300 wait complete. 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 -- start start 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 Processor -- start 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e941000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e941540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e941a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559b3e946d10 con 0x559b3e9440e0 2024-04-26T18:33:24.221 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559b3e826b60 con 0x559b3e938f00 2024-04-26T18:33:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b3e946e90 con 0x559b3e939820 2024-04-26T18:33:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9ba16740 10 client.0 _ll_register_callbacks cb 0x559b3e84b2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e941a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e941a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54618/0 (socket says 172.21.15.90:54618) 2024-04-26T18:33:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 -- 172.21.15.90:0/2378304540 learned_addr learned my addr 172.21.15.90:0/2378304540 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:24.222 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee9259c640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e941000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee91d9b640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e941540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 -- 172.21.15.90:0/2378304540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 msgr2=0x559b3e941540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e941540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 -- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 msgr2=0x559b3e941000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e941000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee92d9d640 1 -- 172.21.15.90:0/2378304540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b3e5b1210 con 0x559b3e9440e0 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.211+0000 7fee91d9b640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e941540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee92d9d640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e941a80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000f80 tx=0x7fee8c008af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541fe02ea0eff5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee79ffb640 1 -- 172.21.15.90:0/2378304540 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c028060 con 0x559b3e9440e0 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8c00de20 con 0x559b3e9440e0 2024-04-26T18:33:24.223 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee79ffb640 1 -- 172.21.15.90:0/2378304540 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 12711+0+0 (secure 0 0 0) 0x7fee8c004a40 con 0x559b3e9440e0 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee79ffb640 1 -- 172.21.15.90:0/2378304540 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=302) ==== 24+0+0 (secure 0 0 0) 0x7fee8c004c40 con 0x559b3e9440e0 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 10 client.16929 fetch_fsmap learned FSMap version 302 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=302,monmap=4+}) -- 0x559b3e67d620 con 0x559b3e9440e0 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee79ffb640 1 -- 172.21.15.90:0/2378304540 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c00de20 con 0x559b3e9440e0 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee79ffb640 1 -- 172.21.15.90:0/2378304540 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 302) ==== 4115+0+0 (secure 0 0 0) 0x7fee8c005040 con 0x559b3e9440e0 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 10 client.16929 fetch_fsmap finished waiting for FSMap version 302 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 20 client.16929 populate_metadata read hostname 'smithi090' 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e954550 0x559b3e956990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.224 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 4 client.16929 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.212+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b3e8ad070 con 0x559b3e954550 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.213+0000 7fee9259c640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e954550 0x559b3e956990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.213+0000 7fee79ffb640 10 client.16929 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.213+0000 7fee9259c640 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e954550 0x559b3e956990 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c76f48d01870d92f server_cookie=36f18eb77ddbe333 in_seq=0 out_seq=0 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee79ffb640 1 -- 172.21.15.90:0/2378304540 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b3e8ad070 con 0x559b3e954550 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee79ffb640 10 client.16929 handle_command_reply: tid=0 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 client.16929 shutdown 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 msgr2=0x559b3e941a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e941a80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fee8c000f80 tx=0x7fee8c008af0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.225 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 shutdown_connections 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e954550 0x559b3e956990 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee79ffb640 0 client.16929 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e941a80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e939820 0x559b3e941540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 --2- 172.21.15.90:0/2378304540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e941000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 >> 172.21.15.90:0/2378304540 conn(0x559b3e935410 msgr2=0x559b3e93b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 shutdown_connections 2024-04-26T18:33:24.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 1 -- 172.21.15.90:0/2378304540 wait complete. 2024-04-26T18:33:24.227 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.214+0000 7fee9ba16740 20 client.16929 trim_cache size 0 max 16384 2024-04-26T18:33:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 Processor -- start 2024-04-26T18:33:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 -- start start 2024-04-26T18:33:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e938f00 0x559b3e94fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e939820 0x559b3e95c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e95c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fee941b2e10 con 0x559b3e9440e0 2024-04-26T18:33:24.234 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee941b1270 con 0x559b3e939820 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.229+0000 7fee9ba16740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fee941b2740 con 0x559b3e938f00 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e95c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e95c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54620/0 (socket says 172.21.15.90:54620) 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 -- 172.21.15.90:0/496886346 learned_addr learned my addr 172.21.15.90:0/496886346 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee91d9b640 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e939820 0x559b3e95c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee9259c640 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e938f00 0x559b3e94fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 -- 172.21.15.90:0/496886346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e938f00 msgr2=0x559b3e94fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e938f00 0x559b3e94fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 -- 172.21.15.90:0/496886346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e939820 msgr2=0x559b3e95c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e939820 0x559b3e95c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 -- 172.21.15.90:0/496886346 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b3e930d60 con 0x559b3e9440e0 2024-04-26T18:33:24.235 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee92d9d640 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e95c7a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fee8c005650 tx=0x7fee8c008330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60096d1495c4979 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee7bfff640 1 -- 172.21.15.90:0/496886346 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c01c020 con 0x559b3e9440e0 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee7bfff640 1 -- 172.21.15.90:0/496886346 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c00de20 con 0x559b3e9440e0 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.230+0000 7fee7bfff640 1 -- 172.21.15.90:0/496886346 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c028040 con 0x559b3e9440e0 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 -- 172.21.15.90:0/496886346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 msgr2=0x559b3e95c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e95c7a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fee8c005650 tx=0x7fee8c008330 comp rx=0 tx=0).stop 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 -- 172.21.15.90:0/496886346 shutdown_connections 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e9440e0 0x559b3e95c7a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e939820 0x559b3e95c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 --2- 172.21.15.90:0/496886346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e938f00 0x559b3e94fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 -- 172.21.15.90:0/496886346 >> 172.21.15.90:0/496886346 conn(0x559b3e935410 msgr2=0x559b3e9357f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:24.236 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 -- 172.21.15.90:0/496886346 shutdown_connections 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 -- 172.21.15.90:0/496886346 wait complete. 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 -- start start 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.231+0000 7fee9ba16740 1 Processor -- start 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e939820 0x559b3e947f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e9440e0 0x559b3e948470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fee941b2070 con 0x559b3e939820 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fee941b1940 con 0x559b3e938f00 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559b3e859590 con 0x559b3e9440e0 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 10 client.0 _ll_register_callbacks cb 0x7fee88098c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e939820 0x559b3e947f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e939820 0x559b3e947f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54622/0 (socket says 172.21.15.90:54622) 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 -- 172.21.15.90:0/112597115 learned_addr learned my addr 172.21.15.90:0/112597115 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9259c640 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 -- 172.21.15.90:0/112597115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e9440e0 msgr2=0x559b3e948470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e9440e0 0x559b3e948470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 -- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 msgr2=0x559b3e9479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 -- 172.21.15.90:0/112597115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b3e9b9ad0 con 0x559b3e939820 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee91d9b640 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e939820 0x559b3e947f30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fee880ce1d0 tx=0x7fee8800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c169136ead548052 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee79ffb640 1 -- 172.21.15.90:0/112597115 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee880a55e0 con 0x559b3e939820 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee79ffb640 1 -- 172.21.15.90:0/112597115 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(175..175 src has 1..175) ==== 12711+0+0 (secure 0 0 0) 0x7fee88098fe0 con 0x559b3e939820 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8c00de20 con 0x559b3e939820 2024-04-26T18:33:24.238 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee79ffb640 1 -- 172.21.15.90:0/112597115 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=302) ==== 24+0+0 (secure 0 0 0) 0x7fee880672d0 con 0x559b3e939820 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 10 client.16935 fetch_fsmap learned FSMap version 302 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.232+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=302,monmap=4+}) -- 0x559b3e9bad20 con 0x559b3e939820 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee79ffb640 1 -- 172.21.15.90:0/112597115 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee8c00de20 con 0x559b3e939820 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee79ffb640 1 -- 172.21.15.90:0/112597115 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 302) ==== 4115+0+0 (secure 0 0 0) 0x7fee880bb020 con 0x559b3e939820 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9ba16740 10 client.16935 fetch_fsmap finished waiting for FSMap version 302 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9ba16740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9ba16740 20 client.16935 populate_metadata read hostname 'smithi090' 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9ba16740 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e9bb500 0x559b3e9bd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9ba16740 4 client.16935 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b3e8a6ee0 con 0x559b3e9bb500 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9259c640 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e9bb500 0x559b3e9bd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:24.239 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee79ffb640 10 client.16935 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:24.240 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:24.233+0000 7fee9259c640 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e9bb500 0x559b3e9bd920 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f9f4e6de92fba5b server_cookie=7e0d72f751b9ab96 in_seq=0 out_seq=0 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: daemon mds.d restarted 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: osdmap e175: 12 total, 12 up, 12 in 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:boot 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:active} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:33:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:24 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:33:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: daemon mds.d restarted 2024-04-26T18:33:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:33:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: osdmap e175: 12 total, 12 up, 12 in 2024-04-26T18:33:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:boot 2024-04-26T18:33:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:33:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:active} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:33:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:24 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:33:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: daemon mds.d restarted 2024-04-26T18:33:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:33:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: osdmap e175: 12 total, 12 up, 12 in 2024-04-26T18:33:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:boot 2024-04-26T18:33:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:33:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:active} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:33:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:24 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:replay} 2 up:standby-replay 2024-04-26T18:33:24.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.62 MB/sec execute 289 sec latency 13093.143 ms 2024-04-26T18:33:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:25 smithi119 ceph-mon[32752]: pgmap v745: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.8 MiB/s wr, 107 op/s 2024-04-26T18:33:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:25 smithi167 ceph-mon[32751]: pgmap v745: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.8 MiB/s wr, 107 op/s 2024-04-26T18:33:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:25 smithi090 ceph-mon[29180]: pgmap v745: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.8 MiB/s wr, 107 op/s 2024-04-26T18:33:25.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.60 MB/sec execute 290 sec latency 14093.175 ms 2024-04-26T18:33:26.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.58 MB/sec execute 291 sec latency 15093.209 ms 2024-04-26T18:33:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:27 smithi119 ceph-mon[32752]: pgmap v746: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 100 op/s 2024-04-26T18:33:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:27 smithi167 ceph-mon[32751]: pgmap v746: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 100 op/s 2024-04-26T18:33:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:27 smithi090 ceph-mon[29180]: pgmap v746: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 100 op/s 2024-04-26T18:33:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:27.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.55 MB/sec execute 292 sec latency 16093.266 ms 2024-04-26T18:33:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:33:28.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.53 MB/sec execute 293 sec latency 17093.350 ms 2024-04-26T18:33:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:29 smithi119 ceph-mon[32752]: pgmap v747: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 91 op/s 2024-04-26T18:33:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:29 smithi119 ceph-mon[32752]: pgmap v748: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 4.1 MiB/s wr, 106 op/s 2024-04-26T18:33:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:29 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:resolve 2024-04-26T18:33:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:29 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:resolve} 2 up:standby-replay 2024-04-26T18:33:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:29 smithi167 ceph-mon[32751]: pgmap v747: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 91 op/s 2024-04-26T18:33:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:29 smithi167 ceph-mon[32751]: pgmap v748: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 4.1 MiB/s wr, 106 op/s 2024-04-26T18:33:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:29 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:resolve 2024-04-26T18:33:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:29 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:resolve} 2 up:standby-replay 2024-04-26T18:33:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:29 smithi090 ceph-mon[29180]: pgmap v747: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.3 MiB/s wr, 91 op/s 2024-04-26T18:33:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:29 smithi090 ceph-mon[29180]: pgmap v748: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 4.1 MiB/s wr, 106 op/s 2024-04-26T18:33:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:29 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:resolve 2024-04-26T18:33:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:29 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:active,2=d=up:resolve} 2 up:standby-replay 2024-04-26T18:33:29.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.51 MB/sec execute 294 sec latency 18093.462 ms 2024-04-26T18:33:30.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.49 MB/sec execute 295 sec latency 19093.578 ms 2024-04-26T18:33:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:31 smithi119 ceph-mon[32752]: pgmap v749: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.2 MiB/s wr, 152 op/s 2024-04-26T18:33:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:31 smithi167 ceph-mon[32751]: pgmap v749: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.2 MiB/s wr, 152 op/s 2024-04-26T18:33:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:31 smithi090 ceph-mon[29180]: pgmap v749: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.2 MiB/s wr, 152 op/s 2024-04-26T18:33:31.906 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.47 MB/sec execute 296 sec latency 20093.701 ms 2024-04-26T18:33:32.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.44 MB/sec execute 297 sec latency 21093.787 ms 2024-04-26T18:33:33.389 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:33 smithi090 ceph-mon[29180]: pgmap v750: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 72 KiB/s wr, 43 op/s 2024-04-26T18:33:33.389 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 3. 2024-04-26T18:33:33.390 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 systemd[1]: Stopped Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:33.390 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 18.374s CPU time. 2024-04-26T18:33:33.390 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 systemd[1]: Starting Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:33:33.390 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 podman[101186]: 2024-04-26 18:33:33.268137333 +0000 UTC m=+0.018688706 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:33:33.390 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 podman[101186]: 2024-04-26 18:33:33.389875728 +0000 UTC m=+0.140427067 container create ca0879c6923e23ca19f30189212e7341a3965a8a8d4e50bfedec75a357cee598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:33:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:33 smithi119 ceph-mon[32752]: pgmap v750: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 72 KiB/s wr, 43 op/s 2024-04-26T18:33:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:33 smithi167 ceph-mon[32751]: pgmap v750: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 72 KiB/s wr, 43 op/s 2024-04-26T18:33:33.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 podman[101186]: 2024-04-26 18:33:33.627483491 +0000 UTC m=+0.378034832 container init ca0879c6923e23ca19f30189212e7341a3965a8a8d4e50bfedec75a357cee598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD) 2024-04-26T18:33:33.652 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 podman[101186]: 2024-04-26 18:33:33.6301436 +0000 UTC m=+0.380694931 container start ca0879c6923e23ca19f30189212e7341a3965a8a8d4e50bfedec75a357cee598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T18:33:33.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.42 MB/sec execute 298 sec latency 22093.848 ms 2024-04-26T18:33:34.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 ceph-mds[101205]: -- 172.21.15.90:0/1158446010 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d1906ba40 con 0x558d1903ec00 2024-04-26T18:33:34.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: starting mds.j at 2024-04-26T18:33:34.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 bash[101186]: ca0879c6923e23ca19f30189212e7341a3965a8a8d4e50bfedec75a357cee598 2024-04-26T18:33:34.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:33:33 smithi090 systemd[1]: Started Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:34 smithi119 ceph-mon[32752]: daemon mds.j restarted 2024-04-26T18:33:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:34 smithi119 ceph-mon[32752]: osdmap e176: 12 total, 12 up, 12 in 2024-04-26T18:33:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:34 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:boot 2024-04-26T18:33:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:34 smithi119 ceph-mon[32752]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:33:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:34 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:resolve} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:33:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:34 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:34 smithi167 ceph-mon[32751]: daemon mds.j restarted 2024-04-26T18:33:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:34 smithi167 ceph-mon[32751]: osdmap e176: 12 total, 12 up, 12 in 2024-04-26T18:33:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:34 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:boot 2024-04-26T18:33:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:34 smithi167 ceph-mon[32751]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:33:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:34 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:resolve} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:33:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:34 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:34 smithi090 ceph-mon[29180]: daemon mds.j restarted 2024-04-26T18:33:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:34 smithi090 ceph-mon[29180]: osdmap e176: 12 total, 12 up, 12 in 2024-04-26T18:33:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:34 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:boot 2024-04-26T18:33:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:34 smithi090 ceph-mon[29180]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:33:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:34 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:resolve} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:33:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:34 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:34.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.40 MB/sec execute 299 sec latency 23093.976 ms 2024-04-26T18:33:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:35 smithi119 ceph-mon[32752]: pgmap v751: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 71 KiB/s wr, 38 op/s 2024-04-26T18:33:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:resolve 2024-04-26T18:33:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:35 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:35 smithi167 ceph-mon[32751]: pgmap v751: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 71 KiB/s wr, 38 op/s 2024-04-26T18:33:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:resolve 2024-04-26T18:33:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:35 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:35 smithi090 ceph-mon[29180]: pgmap v751: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 71 KiB/s wr, 38 op/s 2024-04-26T18:33:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:resolve 2024-04-26T18:33:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:35 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:35.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 46940 6.38 MB/sec execute 300 sec latency 24094.086 ms 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.765+0000 7fee79ffb640 1 -- 172.21.15.90:0/112597115 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b3e8a6ee0 con 0x559b3e9bb500 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.765+0000 7fee79ffb640 10 client.16935 handle_command_reply: tid=0 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 client.16935 shutdown 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e939820 msgr2=0x559b3e947f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e939820 0x559b3e947f30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fee880ce1d0 tx=0x7fee8800df70 comp rx=0 tx=0).stop 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 shutdown_connections 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559b3e9bb500 0x559b3e9bd920 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559b3e9440e0 0x559b3e948470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559b3e939820 0x559b3e947f30 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee79ffb640 0 client.16935 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 --2- 172.21.15.90:0/112597115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559b3e938f00 0x559b3e9479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 >> 172.21.15.90:0/112597115 conn(0x559b3e935410 msgr2=0x559b3e942830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 shutdown_connections 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 1 -- 172.21.15.90:0/112597115 wait complete. 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.766+0000 7fee9ba16740 20 client.16935 trim_cache size 0 max 16384 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:33:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:33:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:33:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:33:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fee7c0a3dc0 msgr2=0x7fee7c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fee7c0a3dc0 0x7fee7c0a6280 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fee84031440 tx=0x7fee840095f0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94102080 msgr2=0x7fee9419b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94102080 0x7fee9419b750 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fee8800c920 tx=0x7fee8800cde0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 shutdown_connections 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fee7c0a3dc0 0x7fee7c0a6280 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee9410f310 0x7fee9419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fee94102080 0x7fee9419b750 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 --2- 172.21.15.90:0/3823212887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee94101760 0x7fee9419b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 >> 172.21.15.90:0/3823212887 conn(0x7fee940789b0 msgr2=0x7fee941106c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 shutdown_connections 2024-04-26T18:33:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:36.767+0000 7fee98d04640 1 -- 172.21.15.90:0/3823212887 wait complete. 2024-04-26T18:33:36.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47002 6.37 MB/sec execute 301 sec latency 25094.130 ms 2024-04-26T18:33:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:36 smithi119 ceph-mon[32752]: pgmap v753: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 83 KiB/s wr, 45 op/s 2024-04-26T18:33:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:36 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:33:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:36 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:reconnect 2024-04-26T18:33:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:36 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:reconnect 2024-04-26T18:33:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:36 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:36 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:33:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:36 smithi167 ceph-mon[32751]: pgmap v753: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 83 KiB/s wr, 45 op/s 2024-04-26T18:33:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:36 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:33:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:36 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:reconnect 2024-04-26T18:33:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:36 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:reconnect 2024-04-26T18:33:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:36 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:36 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:33:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:36 smithi090 ceph-mon[29180]: pgmap v753: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 83 KiB/s wr, 45 op/s 2024-04-26T18:33:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:36 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0 2024-04-26T18:33:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:36 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:reconnect 2024-04-26T18:33:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:36 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:reconnect 2024-04-26T18:33:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:36 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:36 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:33:37.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47292 6.39 MB/sec execute 302 sec latency 26019.095 ms 2024-04-26T18:33:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:37 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00899986 2024-04-26T18:33:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:37 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:rejoin 2024-04-26T18:33:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:37 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:rejoin 2024-04-26T18:33:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:37 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:37 smithi119 ceph-mon[32752]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:33:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:37 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00899986 2024-04-26T18:33:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:37 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:rejoin 2024-04-26T18:33:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:37 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:rejoin 2024-04-26T18:33:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:37 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:37 smithi167 ceph-mon[32751]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:33:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:37 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00899986 2024-04-26T18:33:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:37 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:rejoin 2024-04-26T18:33:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:37 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:rejoin 2024-04-26T18:33:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:37 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:37 smithi090 ceph-mon[29180]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:33:38.553 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:33:38 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:33:38.288+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-26T18:33:38.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 47884 6.47 MB/sec execute 303 sec latency 26894.488 ms 2024-04-26T18:33:38.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:38 smithi090 ceph-mon[29180]: pgmap v754: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-26T18:33:38.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:clientreplay 2024-04-26T18:33:38.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:active 2024-04-26T18:33:38.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:38 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:clientreplay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:38.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:38 smithi090 ceph-mon[29180]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:33:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:38 smithi119 ceph-mon[32752]: pgmap v754: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-26T18:33:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:clientreplay 2024-04-26T18:33:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:active 2024-04-26T18:33:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:38 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:clientreplay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:38 smithi119 ceph-mon[32752]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:33:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:38 smithi167 ceph-mon[32751]: pgmap v754: 97 pgs: 97 active+clean; 723 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-26T18:33:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:clientreplay 2024-04-26T18:33:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/641065147,v1:172.21.15.90:6835/641065147] up:active 2024-04-26T18:33:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:38 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=d=up:clientreplay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:38 smithi167 ceph-mon[32751]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:33:39.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48490 6.59 MB/sec execute 304 sec latency 179.206 ms 2024-04-26T18:33:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:39 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:33:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:active 2024-04-26T18:33:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:33:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:39 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:33:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:active 2024-04-26T18:33:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:33:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:39 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:33:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2850537869,v1:172.21.15.90:6837/2850537869] up:active 2024-04-26T18:33:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:33:40.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 48912 6.63 MB/sec execute 305 sec latency 1179.250 ms 2024-04-26T18:33:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:40 smithi119 ceph-mon[32752]: pgmap v755: 97 pgs: 97 active+clean; 728 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1019 KiB/s wr, 100 op/s 2024-04-26T18:33:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:40 smithi167 ceph-mon[32751]: pgmap v755: 97 pgs: 97 active+clean; 728 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1019 KiB/s wr, 100 op/s 2024-04-26T18:33:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:40 smithi090 ceph-mon[29180]: pgmap v755: 97 pgs: 97 active+clean; 728 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1019 KiB/s wr, 100 op/s 2024-04-26T18:33:41.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49396 6.69 MB/sec execute 306 sec latency 2179.288 ms 2024-04-26T18:33:42.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.76 MB/sec execute 307 sec latency 3179.321 ms 2024-04-26T18:33:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:42 smithi119 ceph-mon[32752]: pgmap v756: 97 pgs: 97 active+clean; 728 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1019 KiB/s wr, 100 op/s 2024-04-26T18:33:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:42 smithi167 ceph-mon[32751]: pgmap v756: 97 pgs: 97 active+clean; 728 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1019 KiB/s wr, 100 op/s 2024-04-26T18:33:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-mon[29180]: pgmap v756: 97 pgs: 97 active+clean; 728 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1019 KiB/s wr, 100 op/s 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb450c78700 time 2024-04-26T18:33:42.829658+0000 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb45b2a13dd] 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: clone() 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: *** Caught signal (Aborted) ** 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: in thread 7fb450c78700 thread_name:ms_dispatch 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:42.830+0000 7fb450c78700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb450c78700 time 2024-04-26T18:33:42.829658+0000 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f95b192a700 time 2024-04-26T18:33:42.745543+0000 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f95bbf533dd] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.128 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: clone() 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: *** Caught signal (Aborted) ** 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: in thread 7f95b192a700 thread_name:ms_dispatch 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:42.746+0000 7f95b192a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f95b192a700 time 2024-04-26T18:33:42.745543+0000 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb45b2a13dd] 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.129 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: clone() 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb45a029d20] 2024-04-26T18:33:43.130 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: gsignal() 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: abort() 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb45b2a1437] 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.131 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f95bbf533dd] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.132 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: clone() 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f95bacdbd20] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: gsignal() 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: abort() 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f95bbf53437] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.133 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 16: clone() 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:42.747+0000 7f95b192a700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: in thread 7f95b192a700 thread_name:ms_dispatch 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f95bacdbd20] 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: gsignal() 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: abort() 2024-04-26T18:33:43.134 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f95bbf53437] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.135 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 16: clone() 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 16: clone() 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:42.833+0000 7fb450c78700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: in thread 7fb450c78700 thread_name:ms_dispatch 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb45a029d20] 2024-04-26T18:33:43.136 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: gsignal() 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: abort() 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb45b2a1437] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.137 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 16: clone() 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: -1> 2024-04-26T18:33:42.746+0000 7f95b192a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f95b192a700 time 2024-04-26T18:33:42.745543+0000 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f95bbf533dd] 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.138 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: clone() 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 0> 2024-04-26T18:33:42.747+0000 7f95b192a700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: in thread 7f95b192a700 thread_name:ms_dispatch 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.139 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: -107> 2024-04-26T18:33:42.830+0000 7fb450c78700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb450c78700 time 2024-04-26T18:33:42.829658+0000 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb45b2a13dd] 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.140 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: clone() 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 0> 2024-04-26T18:33:42.833+0000 7fb450c78700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: in thread 7fb450c78700 thread_name:ms_dispatch 2024-04-26T18:33:43.141 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb45a029d20] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: gsignal() 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: abort() 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb45b2a1437] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.142 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 16: clone() 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: -484> 2024-04-26T18:33:42.830+0000 7fb450c78700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb450c78700 time 2024-04-26T18:33:42.829658+0000 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb45b2a13dd] 2024-04-26T18:33:43.143 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f95bacdbd20] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: gsignal() 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: abort() 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f95bbf53437] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.144 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 16: clone() 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: -1> 2024-04-26T18:33:42.746+0000 7f95b192a700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f95b192a700 time 2024-04-26T18:33:42.745543+0000 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f95bbf533dd] 2024-04-26T18:33:43.145 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: clone() 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 0> 2024-04-26T18:33:42.747+0000 7f95b192a700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:43.146 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: in thread 7f95b192a700 thread_name:ms_dispatch 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f95bacdbd20] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2: gsignal() 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 3: abort() 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f95bbf53437] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f95bbf535a3] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55e286155822] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55e2860c5598] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55e2860c7c17] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55e285fc0d0b] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55e285fc12ac] 2024-04-26T18:33:43.147 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55e285fa9fad] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.148 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: clone() 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: -377> 2024-04-26T18:33:42.833+0000 7fb450c78700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: in thread 7fb450c78700 thread_name:ms_dispatch 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb45a029d20] 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2: gsignal() 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 3: abort() 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb45b2a1437] 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb45b2a15a3] 2024-04-26T18:33:43.149 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x5593b8de1822] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x5593b8d51598] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x5593b8d53c17] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x5593b8c4cd0b] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5593b8c4d2ac] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x5593b8c35fad] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb45b535e20] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 13: (DispatchQueue::entry()+0x51f) [0x7fb45b532fdf] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb45b605fc1] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb45a01f1ca] 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 16: clone() 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:43.150 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[100911]: 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f95bc1e7e20] 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 13: (DispatchQueue::entry()+0x51f) [0x7f95bc1e4fdf] 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f95bc2b7fc1] 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f95bacd11ca] 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 16: clone() 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[100672]: 2024-04-26T18:33:43.151 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:42 smithi090 podman[101244]: 2024-04-26 18:33:42.855763366 +0000 UTC m=+0.050011764 container died 5229c8d86f2b7035ae9fc129eb76edf594e178e534bb10d9f70fcbc19c5a9819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T18:33:43.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.74 MB/sec execute 308 sec latency 4179.378 ms 2024-04-26T18:33:44.126 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:43 smithi090 podman[101255]: 2024-04-26 18:33:43.85092179 +0000 UTC m=+0.954527440 container died 5e27c13e983031e3a3589a79ec84e76b3a7fe4621e5800efe89c36d1a6d9e09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T18:33:44.126 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:43 smithi090 podman[101244]: 2024-04-26 18:33:43.763068166 +0000 UTC m=+0.957316554 container remove 5229c8d86f2b7035ae9fc129eb76edf594e178e534bb10d9f70fcbc19c5a9819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS) 2024-04-26T18:33:44.126 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:43 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:33:44.899 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:44 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Failed with result 'exit-code'. 2024-04-26T18:33:44.899 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:44 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 15.981s CPU time. 2024-04-26T18:33:44.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:44 smithi090 ceph-mon[29180]: pgmap v757: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-04-26T18:33:44.900 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:44 smithi090 podman[101255]: 2024-04-26 18:33:44.644262666 +0000 UTC m=+1.747868298 container remove 5e27c13e983031e3a3589a79ec84e76b3a7fe4621e5800efe89c36d1a6d9e09f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:33:44.900 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:44 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:33:44.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.71 MB/sec execute 309 sec latency 5179.435 ms 2024-04-26T18:33:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:44 smithi119 ceph-mon[32752]: pgmap v757: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-04-26T18:33:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:44 smithi167 ceph-mon[32751]: pgmap v757: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 4.2 MiB/s wr, 315 op/s 2024-04-26T18:33:45.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:44 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-26T18:33:45.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:44 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 14.966s CPU time. 2024-04-26T18:33:45.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.69 MB/sec execute 310 sec latency 6179.510 ms 2024-04-26T18:33:46.907 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.67 MB/sec execute 311 sec latency 7179.609 ms 2024-04-26T18:33:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:46 smithi119 ceph-mon[32752]: pgmap v758: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-04-26T18:33:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:46 smithi167 ceph-mon[32751]: pgmap v758: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-04-26T18:33:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:46 smithi090 ceph-mon[29180]: pgmap v758: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-04-26T18:33:47.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.65 MB/sec execute 312 sec latency 8179.670 ms 2024-04-26T18:33:48.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.63 MB/sec execute 313 sec latency 9179.784 ms 2024-04-26T18:33:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:48 smithi119 ceph-mon[32752]: pgmap v759: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-04-26T18:33:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:48 smithi167 ceph-mon[32751]: pgmap v759: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-04-26T18:33:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:48 smithi090 ceph-mon[29180]: pgmap v759: 97 pgs: 97 active+clean; 750 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.5 MiB/s wr, 262 op/s 2024-04-26T18:33:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.890+0000 7fadbeea0640 1 -- 172.21.15.90:0/152963306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8107a70 msgr2=0x7fadb8107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.890+0000 7fadbeea0640 1 --2- 172.21.15.90:0/152963306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8107a70 0x7fadb8107e50 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fadac009930 tx=0x7fadac02a420 comp rx=0 tx=0).stop 2024-04-26T18:33:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.890+0000 7fadbeea0640 1 -- 172.21.15.90:0/152963306 shutdown_connections 2024-04-26T18:33:49.893 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.890+0000 7fadbeea0640 1 --2- 172.21.15.90:0/152963306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb8102940 0x7fadb810e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.890+0000 7fadbeea0640 1 --2- 172.21.15.90:0/152963306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8101f80 0x7fadb8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.890+0000 7fadbeea0640 1 --2- 172.21.15.90:0/152963306 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8107a70 0x7fadb8107e50 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.890+0000 7fadbeea0640 1 -- 172.21.15.90:0/152963306 >> 172.21.15.90:0/152963306 conn(0x7fadb8077d90 msgr2=0x7fadb80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 -- 172.21.15.90:0/152963306 shutdown_connections 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 -- 172.21.15.90:0/152963306 wait complete. 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 Processor -- start 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 -- start start 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8101f80 0x7fadb80fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8102940 0x7fadb80fd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb8107a70 0x7fadb80fdc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.894 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8111d30 con 0x7fadb8102940 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8111bb0 con 0x7fadb8107a70 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbeea0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadb8111eb0 con 0x7fadb8101f80 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadb7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8102940 0x7fadb80fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbcc3c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8101f80 0x7fadb80fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadb7fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8102940 0x7fadb80fd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52638/0 (socket says 172.21.15.90:52638) 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadbd43d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb8107a70 0x7fadb80fdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.891+0000 7fadb7fff640 1 -- 172.21.15.90:0/3442280601 learned_addr learned my addr 172.21.15.90:0/3442280601 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbcc3c640 1 -- 172.21.15.90:0/3442280601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb8107a70 msgr2=0x7fadb80fdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb8107a70 0x7fadb80fdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbcc3c640 1 -- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8102940 msgr2=0x7fadb80fd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8102940 0x7fadb80fd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.895 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbcc3c640 1 -- 172.21.15.90:0/3442280601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb81a77c0 con 0x7fadb8101f80 2024-04-26T18:33:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadb7fff640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8102940 0x7fadb80fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:33:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8101f80 0x7fadb80fd200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fadac031440 tx=0x7fadac004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadb5ffb640 1 -- 172.21.15.90:0/3442280601 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac03f070 con 0x7fadb8101f80 2024-04-26T18:33:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb81a7a50 con 0x7fadb8101f80 2024-04-26T18:33:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadb5ffb640 1 -- 172.21.15.90:0/3442280601 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac003aa0 con 0x7fadb8101f80 2024-04-26T18:33:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadb5ffb640 1 -- 172.21.15.90:0/3442280601 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac035070 con 0x7fadb8101f80 2024-04-26T18:33:49.896 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.892+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb81a7f60 con 0x7fadb8101f80 2024-04-26T18:33:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.895+0000 7fadb5ffb640 1 -- 172.21.15.90:0/3442280601 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fadac03b460 con 0x7fadb8101f80 2024-04-26T18:33:49.906 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.895+0000 7fadc1bbf740 1 Processor -- start 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadc1bbf740 1 -- start start 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aad82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d51aa3e460 con 0x55d51aad7f00 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d51a9f8590 con 0x55d51aad8820 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d51aa3a420 con 0x55d51aae30e0 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbd43d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aad82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:48250/0 (socket says 172.21.15.90:48250) 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbd43d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aad82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52648/0 (socket says 172.21.15.90:52648) 2024-04-26T18:33:49.907 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1861781224 learned_addr learned my addr 172.21.15.90:0/1861781224 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadb7fff640 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadb5ffb640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fada80a3dc0 0x7fada80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1861781224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 msgr2=0x55d51aae54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1861781224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 msgr2=0x55d51aad82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aad82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1861781224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d51aa050d0 con 0x55d51aad8820 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbd43d640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fada80a3dc0 0x7fada80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.896+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae2ba0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fadac0ee2f0 tx=0x7fadac0b87f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2cbf2612bff0b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadb5ffb640 1 -- 172.21.15.90:0/3442280601 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 12797+0+0 (secure 0 0 0) 0x7fadac0f1ce0 con 0x7fadb8101f80 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fada2ffd640 1 -- 172.21.15.90:0/1861781224 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac0bc7c0 con 0x55d51aad8820 2024-04-26T18:33:49.908 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadbd43d640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fada80a3dc0 0x7fada80a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadb0005c40 tx=0x7fadb0004a40 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fada2ffd640 1 -- 172.21.15.90:0/1861781224 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac0b8b70 con 0x55d51aad8820 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fada2ffd640 1 -- 172.21.15.90:0/1861781224 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac0bc5e0 con 0x55d51aad8820 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1861781224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 msgr2=0x55d51aae2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae2ba0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fadac0ee2f0 tx=0x7fadac0b87f0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1861781224 shutdown_connections 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae2ba0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1861781224 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aad82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1861781224 >> 172.21.15.90:0/1861781224 conn(0x55d51aad4410 msgr2=0x55d51aad47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1861781224 shutdown_connections 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.897+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1861781224 wait complete. 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 -- start start 2024-04-26T18:33:49.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 Processor -- start 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aad7f00 0x55d51aae0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aae30e0 0x55d51aae0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55d51aae5ad0 con 0x55d51aae30e0 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55d51a9c5b60 con 0x55d51aad8820 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d51aae5c50 con 0x55d51aad7f00 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadc1bbf740 10 client.0 _ll_register_callbacks cb 0x55d51a9ea2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadbd43d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aae30e0 0x55d51aae0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadbd43d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aae30e0 0x55d51aae0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52660/0 (socket says 172.21.15.90:52660) 2024-04-26T18:33:49.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadbd43d640 1 -- 172.21.15.90:0/607960134 learned_addr learned my addr 172.21.15.90:0/607960134 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.898+0000 7fadb7fff640 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aad7f00 0x55d51aae0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadbd43d640 1 -- 172.21.15.90:0/607960134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aad7f00 msgr2=0x55d51aae0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadbd43d640 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aad7f00 0x55d51aae0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadbd43d640 1 -- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 msgr2=0x55d51aae05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadbd43d640 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadbd43d640 1 -- 172.21.15.90:0/607960134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d51a750210 con 0x55d51aae30e0 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadbd43d640 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aae30e0 0x55d51aae0ae0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fadb000fac0 tx=0x7fadb0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8ee51060891aff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fad97fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fad97fff640 1 -- 172.21.15.90:0/607960134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb002b060 con 0x55d51aae30e0 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadac0b8b70 con 0x55d51aae30e0 2024-04-26T18:33:49.911 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fad97fff640 1 -- 172.21.15.90:0/607960134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(176..176 src has 1..176) ==== 12797+0+0 (secure 0 0 0) 0x7fadb001d5f0 con 0x55d51aae30e0 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fad97fff640 1 -- 172.21.15.90:0/607960134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=311) ==== 24+0+0 (secure 0 0 0) 0x7fadb0033050 con 0x55d51aae30e0 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadc1bbf740 10 client.16953 fetch_fsmap learned FSMap version 311 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=311,monmap=4+}) -- 0x55d51a81c620 con 0x55d51aae30e0 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.899+0000 7fad97fff640 1 -- 172.21.15.90:0/607960134 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac0b8b70 con 0x55d51aae30e0 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fad97fff640 1 -- 172.21.15.90:0/607960134 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 311) ==== 4123+0+0 (secure 0 0 0) 0x7fadb001d7f0 con 0x55d51aae30e0 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fadc1bbf740 10 client.16953 fetch_fsmap finished waiting for FSMap version 311 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fadc1bbf740 10 client.16953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fadc1bbf740 20 client.16953 populate_metadata read hostname 'smithi090' 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51aaf3440 0x55d51aaf58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fadc1bbf740 4 client.16953 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d51aa4c070 con 0x55d51aaf3440 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.900+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51aaf3440 0x55d51aaf58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.901+0000 7fad97fff640 10 client.16953 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.901+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51aaf3440 0x55d51aaf58a0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffd77cdcb9330773 server_cookie=53e06889945df0e1 in_seq=0 out_seq=0 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.901+0000 7fad97fff640 1 -- 172.21.15.90:0/607960134 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d51aa4c070 con 0x55d51aaf3440 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.901+0000 7fad97fff640 10 client.16953 handle_command_reply: tid=0 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.901+0000 7fadc1bbf740 1 client.16953 shutdown 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aae30e0 msgr2=0x55d51aae0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aae30e0 0x55d51aae0ae0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fadb000fac0 tx=0x7fadb0004ab0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 shutdown_connections 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51aaf3440 0x55d51aaf58a0 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aae30e0 0x55d51aae0ae0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aae05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/607960134 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aad7f00 0x55d51aae0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.913 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fad97fff640 0 client.16953 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 >> 172.21.15.90:0/607960134 conn(0x55d51aad4410 msgr2=0x55d51aada190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 shutdown_connections 2024-04-26T18:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 1 -- 172.21.15.90:0/607960134 wait complete. 2024-04-26T18:33:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.902+0000 7fadc1bbf740 20 client.16953 trim_cache size 0 max 16384 2024-04-26T18:33:49.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.61 MB/sec execute 314 sec latency 10179.853 ms 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 Processor -- start 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 -- start start 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aada8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aaee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b2f30 con 0x55d51aad7f00 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b1390 con 0x55d51aad8820 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b2860 con 0x55d51aae30e0 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aada8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aada8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52674/0 (socket says 172.21.15.90:52674) 2024-04-26T18:33:49.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1382052452 learned_addr learned my addr 172.21.15.90:0/1382052452 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbd43d640 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadb7fff640 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aaee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1382052452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 msgr2=0x55d51aae0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1382052452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 msgr2=0x55d51aaee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aaee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1382052452 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d51aadae00 con 0x55d51aad7f00 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadb7fff640 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aaee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.917+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aada8c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fadac0bcc10 tx=0x7fadac0f1fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0648a831004d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fada27fc640 1 -- 172.21.15.90:0/1382052452 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac048040 con 0x55d51aad7f00 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fada27fc640 1 -- 172.21.15.90:0/1382052452 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac0b8b70 con 0x55d51aad7f00 2024-04-26T18:33:49.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fada27fc640 1 -- 172.21.15.90:0/1382052452 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac003c50 con 0x55d51aad7f00 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1382052452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 msgr2=0x55d51aada8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aada8c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fadac0bcc10 tx=0x7fadac0f1fe0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1382052452 shutdown_connections 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aae0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aaee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1382052452 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aada8c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1382052452 >> 172.21.15.90:0/1382052452 conn(0x55d51aad4410 msgr2=0x55d51aae0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1382052452 shutdown_connections 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.918+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1382052452 wait complete. 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 -- start start 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 Processor -- start 2024-04-26T18:33:49.924 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aadcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aadd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aadd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b2190 con 0x55d51aad7f00 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fadb81b1a60 con 0x55d51aad8820 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d51ab58a20 con 0x55d51aae30e0 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadc1bbf740 10 client.0 _ll_register_callbacks cb 0x7fadb0009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadbcc3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aadcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadbcc3c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aadcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52680/0 (socket says 172.21.15.90:52680) 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1563029816 learned_addr learned my addr 172.21.15.90:0/1563029816 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:33:49.925 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.919+0000 7fadbd43d640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aadd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadb7fff640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aadd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1563029816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 msgr2=0x55d51aadd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aadd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 msgr2=0x55d51aadd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aadd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadbcc3c640 1 -- 172.21.15.90:0/1563029816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d51ab58bf0 con 0x55d51aad7f00 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadbd43d640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aadd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadbcc3c640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aadcd70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fadac0ccea0 tx=0x7fadac02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6afaef2b567c1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fad97fff640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fad97fff640 1 -- 172.21.15.90:0/1563029816 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac048030 con 0x55d51aad7f00 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fad97fff640 1 -- 172.21.15.90:0/1563029816 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(176..176 src has 1..176) ==== 12797+0+0 (secure 0 0 0) 0x7fadac003c50 con 0x55d51aad7f00 2024-04-26T18:33:49.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadac0b8b70 con 0x55d51aad7f00 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fad97fff640 1 -- 172.21.15.90:0/1563029816 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=311) ==== 24+0+0 (secure 0 0 0) 0x7fadac0bc5e0 con 0x55d51aad7f00 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadc1bbf740 10 client.16959 fetch_fsmap learned FSMap version 311 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.920+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=311,monmap=4+}) -- 0x55d51ab59e70 con 0x55d51aad7f00 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fad97fff640 1 -- 172.21.15.90:0/1563029816 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadac0b8b70 con 0x55d51aad7f00 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fad97fff640 1 -- 172.21.15.90:0/1563029816 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 311) ==== 4123+0+0 (secure 0 0 0) 0x7fadac04b060 con 0x55d51aad7f00 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadc1bbf740 10 client.16959 fetch_fsmap finished waiting for FSMap version 311 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadc1bbf740 10 client.16959 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadc1bbf740 20 client.16959 populate_metadata read hostname 'smithi090' 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51ab5a110 0x55d51ab5c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadc1bbf740 4 client.16959 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d51aa45ee0 con 0x55d51ab5a110 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadb7fff640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51ab5a110 0x55d51ab5c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:49.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fad97fff640 10 client.16959 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:33:49.928 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:33:49.921+0000 7fadb7fff640 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51ab5a110 0x55d51ab5c500 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64521072fe2691b server_cookie=d3013ff2a218fc86 in_seq=0 out_seq=0 2024-04-26T18:33:50.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.59 MB/sec execute 315 sec latency 11179.962 ms 2024-04-26T18:33:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:50 smithi119 ceph-mon[32752]: pgmap v760: 97 pgs: 97 active+clean; 776 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.5 MiB/s wr, 317 op/s 2024-04-26T18:33:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:50 smithi167 ceph-mon[32751]: pgmap v760: 97 pgs: 97 active+clean; 776 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.5 MiB/s wr, 317 op/s 2024-04-26T18:33:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:50 smithi090 ceph-mon[29180]: pgmap v760: 97 pgs: 97 active+clean; 776 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.5 MiB/s wr, 317 op/s 2024-04-26T18:33:51.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.57 MB/sec execute 316 sec latency 12180.053 ms 2024-04-26T18:33:52.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.54 MB/sec execute 317 sec latency 13180.147 ms 2024-04-26T18:33:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:52 smithi119 ceph-mon[32752]: pgmap v761: 97 pgs: 97 active+clean; 776 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.7 MiB/s wr, 235 op/s 2024-04-26T18:33:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:52 smithi167 ceph-mon[32751]: pgmap v761: 97 pgs: 97 active+clean; 776 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.7 MiB/s wr, 235 op/s 2024-04-26T18:33:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:52 smithi090 ceph-mon[29180]: pgmap v761: 97 pgs: 97 active+clean; 776 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.7 MiB/s wr, 235 op/s 2024-04-26T18:33:53.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.52 MB/sec execute 318 sec latency 14180.234 ms 2024-04-26T18:33:54.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.50 MB/sec execute 319 sec latency 15180.287 ms 2024-04-26T18:33:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:54 smithi119 ceph-mon[32752]: pgmap v762: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.8 MiB/s wr, 283 op/s 2024-04-26T18:33:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:54 smithi167 ceph-mon[32751]: pgmap v762: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.8 MiB/s wr, 283 op/s 2024-04-26T18:33:55.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:54 smithi090 ceph-mon[29180]: pgmap v762: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 5.8 MiB/s wr, 283 op/s 2024-04-26T18:33:55.115 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:54 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:33:55.115 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:54 smithi090 systemd[1]: Stopped Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:55.115 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:54 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.g.service: Consumed 15.981s CPU time. 2024-04-26T18:33:55.115 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:54 smithi090 systemd[1]: Starting Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:33:55.365 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:55 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:33:55.365 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:55 smithi090 systemd[1]: Stopped Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:55.365 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:55 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 14.966s CPU time. 2024-04-26T18:33:55.365 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:55 smithi090 systemd[1]: Starting Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:33:55.366 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 podman[101422]: 2024-04-26 18:33:55.017472166 +0000 UTC m=+0.018412037 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:33:55.366 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 podman[101422]: 2024-04-26 18:33:55.131166051 +0000 UTC m=+0.132105955 container create 501db6498d4dc32a5b28854912f17e8a2223ee21058510925570d46cb0b45276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:33:55.366 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 podman[101422]: 2024-04-26 18:33:55.350493728 +0000 UTC m=+0.351433583 container init 501db6498d4dc32a5b28854912f17e8a2223ee21058510925570d46cb0b45276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:33:55.366 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 podman[101422]: 2024-04-26 18:33:55.353345919 +0000 UTC m=+0.354285785 container start 501db6498d4dc32a5b28854912f17e8a2223ee21058510925570d46cb0b45276 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T18:33:55.625 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mds[101466]: -- 172.21.15.90:0/2835005763 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ddce69a40 con 0x558ddce3cc00 2024-04-26T18:33:55.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-g[101451]: starting mds.g at 2024-04-26T18:33:55.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 bash[101422]: 501db6498d4dc32a5b28854912f17e8a2223ee21058510925570d46cb0b45276 2024-04-26T18:33:55.626 INFO:journalctl@ceph.mds.g.smithi090.stdout:Apr 26 18:33:55 smithi090 systemd[1]: Started Ceph mds.g for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:55.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.48 MB/sec execute 320 sec latency 16180.400 ms 2024-04-26T18:33:56.047 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: daemon mds.g restarted 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: osdmap e177: 12 total, 12 up, 12 in 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:boot 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:active} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:55 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:55 smithi090 podman[101512]: 2024-04-26 18:33:55.685438384 +0000 UTC m=+0.018104506 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:33:56.048 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:55 smithi090 podman[101512]: 2024-04-26 18:33:55.795719205 +0000 UTC m=+0.128385300 container create 5744d11e9736748ada884b3318a95781763e62234ed9e482ed07e30f3f8ac37e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T18:33:56.049 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:56 smithi090 podman[101512]: 2024-04-26 18:33:56.048485832 +0000 UTC m=+0.381151928 container init 5744d11e9736748ada884b3318a95781763e62234ed9e482ed07e30f3f8ac37e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: daemon mds.g restarted 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: osdmap e177: 12 total, 12 up, 12 in 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:boot 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:active} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:55 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: daemon mds.g restarted 2024-04-26T18:33:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:33:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: osdmap e177: 12 total, 12 up, 12 in 2024-04-26T18:33:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:boot 2024-04-26T18:33:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: Standby daemon mds.j assigned to filesystem cephfs as rank 1 2024-04-26T18:33:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:active} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:33:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:33:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:55 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:56.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:56 smithi090 podman[101512]: 2024-04-26 18:33:56.051042664 +0000 UTC m=+0.383708760 container start 5744d11e9736748ada884b3318a95781763e62234ed9e482ed07e30f3f8ac37e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T18:33:56.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mds[101531]: -- 172.21.15.90:0/1620500566 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594386c3a40 con 0x559438696800 2024-04-26T18:33:56.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: starting mds.d at 2024-04-26T18:33:56.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:56 smithi090 bash[101512]: 5744d11e9736748ada884b3318a95781763e62234ed9e482ed07e30f3f8ac37e 2024-04-26T18:33:56.376 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:33:56 smithi090 systemd[1]: Started Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:33:56.567 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:33:56.108+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 1 2024-04-26T18:33:56.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.46 MB/sec execute 321 sec latency 17180.445 ms 2024-04-26T18:33:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: pgmap v764: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 122 op/s 2024-04-26T18:33:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: daemon mds.d restarted 2024-04-26T18:33:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: osdmap e178: 12 total, 12 up, 12 in 2024-04-26T18:33:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:boot 2024-04-26T18:33:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:33:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:replay} 1 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:33:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:33:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:56 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:replay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: pgmap v764: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 122 op/s 2024-04-26T18:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: daemon mds.d restarted 2024-04-26T18:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: osdmap e178: 12 total, 12 up, 12 in 2024-04-26T18:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:boot 2024-04-26T18:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:replay} 1 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:56 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:replay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: pgmap v764: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.8 MiB/s wr, 122 op/s 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: daemon mds.d restarted 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: osdmap e178: 12 total, 12 up, 12 in 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:boot 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: Standby daemon mds.d assigned to filesystem cephfs as rank 2 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=j=up:replay} 1 up:standby-replay 2 up:standby, 1 failed 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:56 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:replay,2=d=up:replay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:57.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.44 MB/sec execute 322 sec latency 18180.482 ms 2024-04-26T18:33:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:58 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:resolve 2024-04-26T18:33:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:58 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:replay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:58 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:resolve 2024-04-26T18:33:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:58 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:replay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:58 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:resolve 2024-04-26T18:33:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:58 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:replay} 1 up:standby-replay 1 up:standby 2024-04-26T18:33:58.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.42 MB/sec execute 323 sec latency 19180.541 ms 2024-04-26T18:33:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:33:59 smithi090 ceph-mon[29180]: pgmap v766: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 201 KiB/s wr, 71 op/s 2024-04-26T18:33:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:33:59 smithi119 ceph-mon[32752]: pgmap v766: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 201 KiB/s wr, 71 op/s 2024-04-26T18:33:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:33:59 smithi167 ceph-mon[32751]: pgmap v766: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 201 KiB/s wr, 71 op/s 2024-04-26T18:33:59.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.40 MB/sec execute 324 sec latency 20180.572 ms 2024-04-26T18:34:00.908 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.38 MB/sec execute 325 sec latency 21180.684 ms 2024-04-26T18:34:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:01 smithi090 ceph-mon[29180]: pgmap v767: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 202 KiB/s wr, 82 op/s 2024-04-26T18:34:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:01 smithi119 ceph-mon[32752]: pgmap v767: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 202 KiB/s wr, 82 op/s 2024-04-26T18:34:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:01 smithi167 ceph-mon[32751]: pgmap v767: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 202 KiB/s wr, 82 op/s 2024-04-26T18:34:01.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.36 MB/sec execute 326 sec latency 22180.839 ms 2024-04-26T18:34:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:02 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:resolve 2024-04-26T18:34:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:02 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:02 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:resolve 2024-04-26T18:34:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:02 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:02 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:resolve 2024-04-26T18:34:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:02 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:resolve,2=d=up:resolve} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:02.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49839 6.34 MB/sec execute 327 sec latency 23180.965 ms 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fad97fff640 1 -- 172.21.15.90:0/1563029816 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d51aa45ee0 con 0x55d51ab5a110 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fad97fff640 10 client.16959 handle_command_reply: tid=0 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 client.16959 shutdown 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 msgr2=0x55d51aadcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aadcd70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fadac0ccea0 tx=0x7fadac02a420 comp rx=0 tx=0).stop 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 shutdown_connections 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55d51ab5a110 0x55d51ab5c500 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d51aae30e0 0x55d51aadd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.207 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d51aad8820 0x55d51aadd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fad97fff640 0 client.16959 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 --2- 172.21.15.90:0/1563029816 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55d51aad7f00 0x55d51aadcd70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 >> 172.21.15.90:0/1563029816 conn(0x55d51aad4410 msgr2=0x55d51aae08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 shutdown_connections 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 1 -- 172.21.15.90:0/1563029816 wait complete. 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.206+0000 7fadc1bbf740 20 client.16959 trim_cache size 0 max 16384 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:34:03.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:34:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:34:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fada80a3dc0 msgr2=0x7fada80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:03.209 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fada80a3dc0 0x7fada80a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadb0005c40 tx=0x7fadb0004a40 comp rx=0 tx=0).stop 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8101f80 msgr2=0x7fadb80fd200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8101f80 0x7fadb80fd200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fadac031440 tx=0x7fadac004270 comp rx=0 tx=0).stop 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 shutdown_connections 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fada80a3dc0 0x7fada80a6280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb8107a70 0x7fadb80fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadb8102940 0x7fadb80fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 --2- 172.21.15.90:0/3442280601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadb8101f80 0x7fadb80fd200 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 >> 172.21.15.90:0/3442280601 conn(0x7fadb8077d90 msgr2=0x7fadb8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 shutdown_connections 2024-04-26T18:34:03.210 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:03.207+0000 7fadbeea0640 1 -- 172.21.15.90:0/3442280601 wait complete. 2024-04-26T18:34:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:03 smithi090 ceph-mon[29180]: pgmap v768: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 255 B/s wr, 10 op/s 2024-04-26T18:34:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:03 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:34:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:reconnect 2024-04-26T18:34:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:reconnect 2024-04-26T18:34:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:03 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:03 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:34:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:03 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:34:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:03 smithi119 ceph-mon[32752]: pgmap v768: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 255 B/s wr, 10 op/s 2024-04-26T18:34:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:03 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:34:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:reconnect 2024-04-26T18:34:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:reconnect 2024-04-26T18:34:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:03 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:03 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:34:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:03 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:34:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:03 smithi167 ceph-mon[32751]: pgmap v768: 97 pgs: 97 active+clean; 770 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 255 B/s wr, 10 op/s 2024-04-26T18:34:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:03 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:34:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:reconnect 2024-04-26T18:34:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:reconnect 2024-04-26T18:34:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:03 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:reconnect,2=d=up:reconnect} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:03 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00699989 2024-04-26T18:34:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:03 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00799988 2024-04-26T18:34:03.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 49974 6.35 MB/sec execute 328 sec latency 23408.003 ms 2024-04-26T18:34:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:rejoin 2024-04-26T18:34:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:04 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:rejoin 2024-04-26T18:34:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:04 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:rejoin 2024-04-26T18:34:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:04 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:rejoin 2024-04-26T18:34:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:04 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:rejoin 2024-04-26T18:34:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:04 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:rejoin 2024-04-26T18:34:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:04 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:rejoin,2=d=up:rejoin} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:04.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50222 6.35 MB/sec execute 329 sec latency 22191.844 ms 2024-04-26T18:34:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:05 smithi119 ceph-mon[32752]: pgmap v769: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 107 KiB/s wr, 24 op/s 2024-04-26T18:34:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:05 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:clientreplay 2024-04-26T18:34:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:05 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:clientreplay 2024-04-26T18:34:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:05 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:clientreplay,2=d=up:clientreplay} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:05 smithi119 ceph-mon[32752]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:34:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:05 smithi119 ceph-mon[32752]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:34:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:05 smithi167 ceph-mon[32751]: pgmap v769: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 107 KiB/s wr, 24 op/s 2024-04-26T18:34:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:05 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:clientreplay 2024-04-26T18:34:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:05 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:clientreplay 2024-04-26T18:34:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:05 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:clientreplay,2=d=up:clientreplay} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:05 smithi167 ceph-mon[32751]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:34:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:05 smithi167 ceph-mon[32751]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:34:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:05 smithi090 ceph-mon[29180]: pgmap v769: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 107 KiB/s wr, 24 op/s 2024-04-26T18:34:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:05 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:clientreplay 2024-04-26T18:34:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:05 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:clientreplay 2024-04-26T18:34:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:05 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=j=up:clientreplay,2=d=up:clientreplay} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:05 smithi090 ceph-mon[29180]: daemon mds.d is now active in filesystem cephfs as rank 2 2024-04-26T18:34:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:05 smithi090 ceph-mon[29180]: daemon mds.j is now active in filesystem cephfs as rank 1 2024-04-26T18:34:05.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50412 6.38 MB/sec execute 330 sec latency 22511.457 ms 2024-04-26T18:34:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:06 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:34:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:active 2024-04-26T18:34:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:active 2024-04-26T18:34:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:06 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:06 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:34:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:06 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:34:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:active 2024-04-26T18:34:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:active 2024-04-26T18:34:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:06 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:06 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:34:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:06 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:34:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2196548810,v1:172.21.15.90:6837/2196548810] up:active 2024-04-26T18:34:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6838/2261571250,v1:172.21.15.90:6839/2261571250] up:active 2024-04-26T18:34:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:06 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 1 up:standby-replay 1 up:standby 2024-04-26T18:34:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:06 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:34:06.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 50785 6.43 MB/sec execute 331 sec latency 784.634 ms 2024-04-26T18:34:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:07 smithi119 ceph-mon[32752]: pgmap v770: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 88 KiB/s wr, 19 op/s 2024-04-26T18:34:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:07 smithi167 ceph-mon[32751]: pgmap v770: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 88 KiB/s wr, 19 op/s 2024-04-26T18:34:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:07 smithi090 ceph-mon[29180]: pgmap v770: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 88 KiB/s wr, 19 op/s 2024-04-26T18:34:07.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51214 6.47 MB/sec execute 332 sec latency 111.595 ms 2024-04-26T18:34:08.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51589 6.51 MB/sec execute 333 sec latency 144.841 ms 2024-04-26T18:34:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:09 smithi119 ceph-mon[32752]: pgmap v771: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 76 KiB/s wr, 17 op/s 2024-04-26T18:34:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:09 smithi167 ceph-mon[32751]: pgmap v771: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 76 KiB/s wr, 17 op/s 2024-04-26T18:34:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:09 smithi090 ceph-mon[29180]: pgmap v771: 97 pgs: 97 active+clean; 759 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 76 KiB/s wr, 17 op/s 2024-04-26T18:34:09.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 51966 6.54 MB/sec execute 334 sec latency 954.026 ms 2024-04-26T18:34:10.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52231 6.56 MB/sec execute 335 sec latency 1954.065 ms 2024-04-26T18:34:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:11 smithi119 ceph-mon[32752]: pgmap v772: 97 pgs: 97 active+clean; 774 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-04-26T18:34:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:11 smithi119 ceph-mon[32752]: osdmap e179: 12 total, 12 up, 12 in 2024-04-26T18:34:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:11 smithi167 ceph-mon[32751]: pgmap v772: 97 pgs: 97 active+clean; 774 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-04-26T18:34:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:11 smithi167 ceph-mon[32751]: osdmap e179: 12 total, 12 up, 12 in 2024-04-26T18:34:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:11 smithi090 ceph-mon[29180]: pgmap v772: 97 pgs: 97 active+clean; 774 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 2.7 MiB/s wr, 277 op/s 2024-04-26T18:34:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:11 smithi090 ceph-mon[29180]: osdmap e179: 12 total, 12 up, 12 in 2024-04-26T18:34:11.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 52692 6.63 MB/sec execute 336 sec latency 2954.101 ms 2024-04-26T18:34:12.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53100 6.68 MB/sec execute 337 sec latency 3954.130 ms 2024-04-26T18:34:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:13 smithi119 ceph-mon[32752]: pgmap v774: 97 pgs: 97 active+clean; 774 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-04-26T18:34:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:13 smithi167 ceph-mon[32751]: pgmap v774: 97 pgs: 97 active+clean; 774 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-04-26T18:34:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:13 smithi090 ceph-mon[29180]: pgmap v774: 97 pgs: 97 active+clean; 774 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.2 MiB/s wr, 323 op/s 2024-04-26T18:34:13.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 53641 6.73 MB/sec execute 338 sec latency 4954.161 ms 2024-04-26T18:34:14.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54272 6.84 MB/sec execute 339 sec latency 5954.197 ms 2024-04-26T18:34:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:15 smithi119 ceph-mon[32752]: pgmap v775: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:15 smithi167 ceph-mon[32751]: pgmap v775: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:15 smithi090 ceph-mon[29180]: pgmap v775: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:15.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54795 6.89 MB/sec execute 340 sec latency 6954.233 ms 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 -- 172.21.15.90:0/2566820603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4107860 msgr2=0x7f08f4107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 --2- 172.21.15.90:0/2566820603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4107860 0x7f08f4107c40 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f08e4009930 tx=0x7f08e402a420 comp rx=0 tx=0).stop 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 -- 172.21.15.90:0/2566820603 shutdown_connections 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 --2- 172.21.15.90:0/2566820603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f4102210 0x7f08f410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 --2- 172.21.15.90:0/2566820603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08f4101850 0x7f08f4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 --2- 172.21.15.90:0/2566820603 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4107860 0x7f08f4107c40 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 -- 172.21.15.90:0/2566820603 >> 172.21.15.90:0/2566820603 conn(0x7f08f4077bb0 msgr2=0x7f08f4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 -- 172.21.15.90:0/2566820603 shutdown_connections 2024-04-26T18:34:16.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.293+0000 7f08fa546640 1 -- 172.21.15.90:0/2566820603 wait complete. 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 Processor -- start 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 -- start start 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4101850 0x7f08f41990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f4102210 0x7f08f41995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08f4107860 0x7f08f419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4112c20 con 0x7f08f4101850 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4112aa0 con 0x7f08f4102210 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08fa546640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08f4112da0 con 0x7f08f4107860 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4101850 0x7f08f41990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4101850 0x7f08f41990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53454/0 (socket says 172.21.15.90:53454) 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 -- 172.21.15.90:0/982317473 learned_addr learned my addr 172.21.15.90:0/982317473 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08f4107860 0x7f08f419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.297 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f37fe640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f4102210 0x7f08f41995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 -- 172.21.15.90:0/982317473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08f4107860 msgr2=0x7f08f419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08f4107860 0x7f08f419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 -- 172.21.15.90:0/982317473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f4102210 msgr2=0x7f08f41995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f4102210 0x7f08f41995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 -- 172.21.15.90:0/982317473 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f419e090 con 0x7f08f4101850 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08f4107860 0x7f08f419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.294+0000 7f08f3fff640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4101850 0x7f08f41990a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005940 tx=0x7f08e40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.295+0000 7f08f17fa640 1 -- 172.21.15.90:0/982317473 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e403f030 con 0x7f08f4101850 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.295+0000 7f08f17fa640 1 -- 172.21.15.90:0/982317473 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e40044c0 con 0x7f08f4101850 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.295+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f419e320 con 0x7f08f4101850 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.295+0000 7f08f17fa640 1 -- 172.21.15.90:0/982317473 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e4035070 con 0x7f08f4101850 2024-04-26T18:34:16.298 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.295+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f41a5c10 con 0x7f08f4101850 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08f17fa640 1 -- 172.21.15.90:0/982317473 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f08e4042030 con 0x7f08f4101850 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 Processor -- start 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 -- start start 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980cf00 0x559f1980d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980d820 0x559f19817ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08f17fa640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f08dc0a3dc0 0x7f08dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f1981a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.305 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559f19773460 con 0x559f1980d820 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559f1972d590 con 0x559f1980cf00 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08fd264740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559f1976f420 con 0x559f198180e0 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980d820 0x559f19817ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08f3fff640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f08dc0a3dc0 0x7f08dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08f37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980d820 0x559f19817ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53456/0 (socket says 172.21.15.90:53456) 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.297+0000 7f08f37fe640 1 -- 172.21.15.90:0/3673139888 learned_addr learned my addr 172.21.15.90:0/3673139888 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f1981a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f17fa640 1 -- 172.21.15.90:0/982317473 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 13057+0+0 (secure 0 0 0) 0x7f08e40f1e50 con 0x7f08f4101850 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980cf00 0x559f1980d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f37fe640 1 -- 172.21.15.90:0/3673139888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 msgr2=0x559f1981a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f37fe640 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f1981a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f37fe640 1 -- 172.21.15.90:0/3673139888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980cf00 msgr2=0x559f1980d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f37fe640 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980cf00 0x559f1980d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f37fe640 1 -- 172.21.15.90:0/3673139888 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f1973a0d0 con 0x559f1980d820 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980cf00 0x559f1980d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f3fff640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f08dc0a3dc0 0x7f08dc0a6280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08e40ee490 tx=0x7f08e403b460 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08f37fe640 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980d820 0x559f19817ba0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009c00 tx=0x7f08e8010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4fafaa15ed75e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08da7fc640 1 -- 172.21.15.90:0/3673139888 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e801b480 con 0x559f1980d820 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08da7fc640 1 -- 172.21.15.90:0/3673139888 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e801bad0 con 0x559f1980d820 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.298+0000 7f08da7fc640 1 -- 172.21.15.90:0/3673139888 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08e8012a80 con 0x559f1980d820 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 -- 172.21.15.90:0/3673139888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980d820 msgr2=0x559f19817ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980d820 0x559f19817ba0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009c00 tx=0x7f08e8010740 comp rx=0 tx=0).stop 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 -- 172.21.15.90:0/3673139888 shutdown_connections 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f1981a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980d820 0x559f19817ba0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.307 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 --2- 172.21.15.90:0/3673139888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980cf00 0x559f1980d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 -- 172.21.15.90:0/3673139888 >> 172.21.15.90:0/3673139888 conn(0x559f19809410 msgr2=0x559f198097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 -- 172.21.15.90:0/3673139888 shutdown_connections 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 -- 172.21.15.90:0/3673139888 wait complete. 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 -- start start 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.299+0000 7f08fd264740 1 Processor -- start 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19815090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f1980d820 0x559f198155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f198180e0 0x559f19815b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x559f1981ad10 con 0x559f1980cf00 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x559f196fab60 con 0x559f198180e0 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559f1981ae90 con 0x559f1980d820 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 10 client.0 _ll_register_callbacks cb 0x559f1971f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:16.308 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19815090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19815090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53466/0 (socket says 172.21.15.90:53466) 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 -- 172.21.15.90:0/2649842067 learned_addr learned my addr 172.21.15.90:0/2649842067 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f37fe640 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f1980d820 0x559f198155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f198180e0 0x559f19815b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 -- 172.21.15.90:0/2649842067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f1980d820 msgr2=0x559f198155d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f1980d820 0x559f198155d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 -- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f198180e0 msgr2=0x559f19815b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f198180e0 0x559f19815b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 -- 172.21.15.90:0/2649842067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f19485210 con 0x559f1980cf00 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19815090 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f08ec014f10 tx=0x7f08ec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110182d3b0625ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08d37fe640 1 -- 172.21.15.90:0/2649842067 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec011e90 con 0x559f1980cf00 2024-04-26T18:34:16.309 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08e801bad0 con 0x559f1980cf00 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08d37fe640 1 -- 172.21.15.90:0/2649842067 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(179..179 src has 1..179) ==== 13057+0+0 (secure 0 0 0) 0x7f08ec014110 con 0x559f1980cf00 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08d37fe640 1 -- 172.21.15.90:0/2649842067 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=322) ==== 24+0+0 (secure 0 0 0) 0x7f08ec00f630 con 0x559f1980cf00 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 10 client.16983 fetch_fsmap learned FSMap version 322 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.300+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=322,monmap=4+}) -- 0x559f19551620 con 0x559f1980cf00 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08d37fe640 1 -- 172.21.15.90:0/2649842067 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08e801bad0 con 0x559f1980cf00 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08d37fe640 1 -- 172.21.15.90:0/2649842067 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 322) ==== 4123+0+0 (secure 0 0 0) 0x7f08ec0127f0 con 0x559f1980cf00 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08fd264740 10 client.16983 fetch_fsmap finished waiting for FSMap version 322 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08fd264740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08fd264740 20 client.16983 populate_metadata read hostname 'smithi090' 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08fd264740 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f198286c0 0x559f1982ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08fd264740 4 client.16983 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f19781070 con 0x559f198286c0 2024-04-26T18:34:16.310 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08f37fe640 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f198286c0 0x559f1982ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08d37fe640 10 client.16983 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.301+0000 7f08f37fe640 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f198286c0 0x559f1982ab40 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958ea979bbf8d038 server_cookie=532112bef4288414 in_seq=0 out_seq=0 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08d37fe640 1 -- 172.21.15.90:0/2649842067 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f19781070 con 0x559f198286c0 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08d37fe640 10 client.16983 handle_command_reply: tid=0 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 client.16983 shutdown 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 msgr2=0x559f19815090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19815090 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f08ec014f10 tx=0x7f08ec004ab0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 shutdown_connections 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f198286c0 0x559f1982ab40 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f198180e0 0x559f19815b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f1980d820 0x559f198155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08d37fe640 0 client.16983 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:34:16.311 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 --2- 172.21.15.90:0/2649842067 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19815090 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 >> 172.21.15.90:0/2649842067 conn(0x559f19809410 msgr2=0x559f1980f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 shutdown_connections 2024-04-26T18:34:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 1 -- 172.21.15.90:0/2649842067 wait complete. 2024-04-26T18:34:16.312 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.302+0000 7f08fd264740 20 client.16983 trim_cache size 0 max 16384 2024-04-26T18:34:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 Processor -- start 2024-04-26T18:34:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 -- start start 2024-04-26T18:34:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f198047b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19823bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.321 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f19808d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41b0f60 con 0x559f1980cf00 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41af3c0 con 0x559f1980d820 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08fd264740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41b0890 con 0x559f198180e0 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f198047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f198047b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53482/0 (socket says 172.21.15.90:53482) 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 -- 172.21.15.90:0/478917446 learned_addr learned my addr 172.21.15.90:0/478917446 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f37fe640 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19823bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 -- 172.21.15.90:0/478917446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 msgr2=0x559f19808d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f19808d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 -- 172.21.15.90:0/478917446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 msgr2=0x559f19823bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19823bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.322 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 -- 172.21.15.90:0/478917446 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f198140a0 con 0x559f1980cf00 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f37fe640 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19823bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.317+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f198047b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00a010 tx=0x7f08ec005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf34dbe071ced56d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08d9ffb640 1 -- 172.21.15.90:0/478917446 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec033070 con 0x559f1980cf00 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08d9ffb640 1 -- 172.21.15.90:0/478917446 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ec011e90 con 0x559f1980cf00 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08d9ffb640 1 -- 172.21.15.90:0/478917446 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec00fa10 con 0x559f1980cf00 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 -- 172.21.15.90:0/478917446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 msgr2=0x559f198047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f198047b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f08ec00a010 tx=0x7f08ec005c50 comp rx=0 tx=0).stop 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 -- 172.21.15.90:0/478917446 shutdown_connections 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f19808d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19823bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 --2- 172.21.15.90:0/478917446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f198047b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 -- 172.21.15.90:0/478917446 >> 172.21.15.90:0/478917446 conn(0x559f19809410 msgr2=0x559f198097f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:16.323 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 -- 172.21.15.90:0/478917446 shutdown_connections 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.318+0000 7f08fd264740 1 -- 172.21.15.90:0/478917446 wait complete. 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 -- start start 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 Processor -- start 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19811cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19812200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f19812740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41b0160 con 0x559f1980cf00 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f08f41afa90 con 0x559f1980d820 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559f19781070 con 0x559f198180e0 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08fd264740 10 client.0 _ll_register_callbacks cb 0x7f08ec0091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19811cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19811cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53490/0 (socket says 172.21.15.90:53490) 2024-04-26T18:34:16.324 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 -- 172.21.15.90:0/3914511498 learned_addr learned my addr 172.21.15.90:0/3914511498 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f37fe640 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19812200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f19812740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 -- 172.21.15.90:0/3914511498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 msgr2=0x559f19812740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f19812740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 -- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 msgr2=0x559f19812200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19812200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.319+0000 7f08f8ae3640 1 -- 172.21.15.90:0/3914511498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f1988dc80 con 0x559f1980cf00 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08f8ae3640 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19811cc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009600 tx=0x7f08ec011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2307adef109d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08d37fe640 1 -- 172.21.15.90:0/3914511498 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec033040 con 0x559f1980cf00 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08d37fe640 1 -- 172.21.15.90:0/3914511498 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(179..179 src has 1..179) ==== 13057+0+0 (secure 0 0 0) 0x7f08ec009d70 con 0x559f1980cf00 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08ec011e90 con 0x559f1980cf00 2024-04-26T18:34:16.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08d37fe640 1 -- 172.21.15.90:0/3914511498 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=322) ==== 24+0+0 (secure 0 0 0) 0x7f08ec004a00 con 0x559f1980cf00 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 10 client.16989 fetch_fsmap learned FSMap version 322 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=322,monmap=4+}) -- 0x559f1988ee90 con 0x559f1980cf00 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08d37fe640 1 -- 172.21.15.90:0/3914511498 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ec011e90 con 0x559f1980cf00 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08d37fe640 1 -- 172.21.15.90:0/3914511498 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 322) ==== 4123+0+0 (secure 0 0 0) 0x7f08ec005070 con 0x559f1980cf00 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 10 client.16989 fetch_fsmap finished waiting for FSMap version 322 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 20 client.16989 populate_metadata read hostname 'smithi090' 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f1988f550 0x559f19891970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 4 client.16989 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f1977aee0 con 0x559f1988f550 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.320+0000 7f08f37fe640 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f1988f550 0x559f19891970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:16.326 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.321+0000 7f08d37fe640 10 client.16989 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:34:16.327 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:16.321+0000 7f08f37fe640 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f1988f550 0x559f19891970 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=934c8ac93f279d62 server_cookie=3669a932cc20bcdb in_seq=0 out_seq=0 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faff5420700 time 2024-04-26T18:34:16.103886+0000 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fafffa493dd] 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: clone() 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.104+0000 7faff5420700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faff5420700 time 2024-04-26T18:34:16.103886+0000 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:16.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fafffa493dd] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.380 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: clone() 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: *** Caught signal (Aborted) ** 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: in thread 7faff5420700 thread_name:ms_dispatch 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faffe7d1d20] 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: gsignal() 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: abort() 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fafffa49437] 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.381 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.382 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 16: clone() 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.105+0000 7faff5420700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: in thread 7faff5420700 thread_name:ms_dispatch 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faffe7d1d20] 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: gsignal() 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: abort() 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fafffa49437] 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.383 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.384 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 16: clone() 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: -1> 2024-04-26T18:34:16.104+0000 7faff5420700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faff5420700 time 2024-04-26T18:34:16.103886+0000 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fafffa493dd] 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.385 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: clone() 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 0> 2024-04-26T18:34:16.105+0000 7faff5420700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: in thread 7faff5420700 thread_name:ms_dispatch 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.386 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faffe7d1d20] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: gsignal() 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: abort() 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fafffa49437] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.387 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 16: clone() 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: -1> 2024-04-26T18:34:16.104+0000 7faff5420700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7faff5420700 time 2024-04-26T18:34:16.103886+0000 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fafffa493dd] 2024-04-26T18:34:16.388 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.389 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: clone() 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 0> 2024-04-26T18:34:16.105+0000 7faff5420700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: in thread 7faff5420700 thread_name:ms_dispatch 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7faffe7d1d20] 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2: gsignal() 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 3: abort() 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fafffa49437] 2024-04-26T18:34:16.390 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fafffa495a3] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x559436f17822] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x559436e87598] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x559436e89c17] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x559436d82d0b] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x559436d832ac] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x559436d6bfad] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fafffcdde20] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 13: (DispatchQueue::entry()+0x51f) [0x7fafffcdafdf] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fafffdadfc1] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7faffe7c71ca] 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 16: clone() 2024-04-26T18:34:16.391 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:16.392 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d[101527]: 2024-04-26T18:34:16.392 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:16 smithi090 podman[101586]: 2024-04-26 18:34:16.1634512 +0000 UTC m=+0.018348513 container died 5744d11e9736748ada884b3318a95781763e62234ed9e482ed07e30f3f8ac37e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD) 2024-04-26T18:34:16.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.89 MB/sec execute 341 sec latency 7954.277 ms 2024-04-26T18:34:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:17 smithi090 ceph-mon[29180]: pgmap v776: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:17.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:17 smithi090 podman[101586]: 2024-04-26 18:34:17.025405249 +0000 UTC m=+0.880302562 container remove 5744d11e9736748ada884b3318a95781763e62234ed9e482ed07e30f3f8ac37e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T18:34:17.378 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:17 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:34:17.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:17 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-26T18:34:17.379 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:17 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 22.800s CPU time. 2024-04-26T18:34:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:17 smithi119 ceph-mon[32752]: pgmap v776: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:17 smithi167 ceph-mon[32751]: pgmap v776: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:17.909 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.87 MB/sec execute 342 sec latency 8954.359 ms 2024-04-26T18:34:18.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.85 MB/sec execute 343 sec latency 9954.431 ms 2024-04-26T18:34:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:18 smithi119 ceph-mon[32752]: pgmap v777: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:18 smithi119 ceph-mon[32752]: osdmap e180: 12 total, 12 up, 12 in 2024-04-26T18:34:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:18 smithi167 ceph-mon[32751]: pgmap v777: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:18 smithi167 ceph-mon[32751]: osdmap e180: 12 total, 12 up, 12 in 2024-04-26T18:34:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:18 smithi090 ceph-mon[29180]: pgmap v777: 97 pgs: 97 active+clean; 779 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.5 MiB/s wr, 542 op/s 2024-04-26T18:34:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:18 smithi090 ceph-mon[29180]: osdmap e180: 12 total, 12 up, 12 in 2024-04-26T18:34:19.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.83 MB/sec execute 344 sec latency 10954.551 ms 2024-04-26T18:34:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:20 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:34:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:20 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:34:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:20 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active} 2 up:standby-replay 2024-04-26T18:34:20.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.81 MB/sec execute 345 sec latency 11954.660 ms 2024-04-26T18:34:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:21 smithi119 ceph-mon[32752]: pgmap v779: 97 pgs: 97 active+clean; 798 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 5.9 MiB/s wr, 470 op/s 2024-04-26T18:34:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:21 smithi167 ceph-mon[32751]: pgmap v779: 97 pgs: 97 active+clean; 798 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 5.9 MiB/s wr, 470 op/s 2024-04-26T18:34:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:21 smithi090 ceph-mon[29180]: pgmap v779: 97 pgs: 97 active+clean; 798 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 5.9 MiB/s wr, 470 op/s 2024-04-26T18:34:21.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.79 MB/sec execute 346 sec latency 12954.778 ms 2024-04-26T18:34:22.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.77 MB/sec execute 347 sec latency 13954.871 ms 2024-04-26T18:34:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:23 smithi119 ceph-mon[32752]: pgmap v780: 97 pgs: 97 active+clean; 798 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 5.6 MiB/s wr, 444 op/s 2024-04-26T18:34:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:23 smithi167 ceph-mon[32751]: pgmap v780: 97 pgs: 97 active+clean; 798 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 5.6 MiB/s wr, 444 op/s 2024-04-26T18:34:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:23 smithi090 ceph-mon[29180]: pgmap v780: 97 pgs: 97 active+clean; 798 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 5.6 MiB/s wr, 444 op/s 2024-04-26T18:34:23.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.75 MB/sec execute 348 sec latency 14954.922 ms 2024-04-26T18:34:24.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.73 MB/sec execute 349 sec latency 15954.951 ms 2024-04-26T18:34:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:25 smithi119 ceph-mon[32752]: pgmap v781: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:25 smithi167 ceph-mon[32751]: pgmap v781: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:25 smithi090 ceph-mon[29180]: pgmap v781: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:25.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.71 MB/sec execute 350 sec latency 16954.994 ms 2024-04-26T18:34:26.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.69 MB/sec execute 351 sec latency 17955.134 ms 2024-04-26T18:34:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:27 smithi119 ceph-mon[32752]: pgmap v782: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:27 smithi167 ceph-mon[32751]: pgmap v782: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:27 smithi090 ceph-mon[29180]: pgmap v782: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:27.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:27 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Scheduled restart job, restart counter is at 6. 2024-04-26T18:34:27.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:27 smithi090 systemd[1]: Stopped Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:34:27.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:27 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Consumed 22.800s CPU time. 2024-04-26T18:34:27.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:27 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Start request repeated too quickly. 2024-04-26T18:34:27.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:27 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.d.service: Failed with result 'exit-code'. 2024-04-26T18:34:27.626 INFO:journalctl@ceph.mds.d.smithi090.stdout:Apr 26 18:34:27 smithi090 systemd[1]: Failed to start Ceph mds.d for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:34:27.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.67 MB/sec execute 352 sec latency 18955.167 ms 2024-04-26T18:34:28.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.65 MB/sec execute 353 sec latency 19955.246 ms 2024-04-26T18:34:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:28 smithi119 ceph-mon[32752]: pgmap v783: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:28 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active(laggy or crashed)} 2 up:standby-replay 2024-04-26T18:34:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:34:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:28 smithi167 ceph-mon[32751]: pgmap v783: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:28 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active(laggy or crashed)} 2 up:standby-replay 2024-04-26T18:34:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:34:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-mon[29180]: pgmap v783: 97 pgs: 97 active+clean; 803 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.9 MiB/s wr, 225 op/s 2024-04-26T18:34:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=j=up:active,2=d=up:active(laggy or crashed)} 2 up:standby-replay 2024-04-26T18:34:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb4daedf700 time 2024-04-26T18:34:28.714215+0000 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb4e55083dd] 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.128 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.129 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: clone() 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:28.714+0000 7fb4daedf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb4daedf700 time 2024-04-26T18:34:28.714215+0000 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb4e55083dd] 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.130 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: clone() 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.131 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: *** Caught signal (Aborted) ** 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: in thread 7fb4daedf700 thread_name:ms_dispatch 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb4e4290d20] 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: gsignal() 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: abort() 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb4e5508437] 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.132 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 16: clone() 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:28.716+0000 7fb4daedf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: in thread 7fb4daedf700 thread_name:ms_dispatch 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb4e4290d20] 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: gsignal() 2024-04-26T18:34:29.133 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: abort() 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb4e5508437] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.134 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 16: clone() 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: -1> 2024-04-26T18:34:28.714+0000 7fb4daedf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb4daedf700 time 2024-04-26T18:34:28.714215+0000 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb4e55083dd] 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.135 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: clone() 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 0> 2024-04-26T18:34:28.716+0000 7fb4daedf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: in thread 7fb4daedf700 thread_name:ms_dispatch 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.136 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb4e4290d20] 2024-04-26T18:34:29.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: gsignal() 2024-04-26T18:34:29.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: abort() 2024-04-26T18:34:29.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb4e5508437] 2024-04-26T18:34:29.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.137 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 16: clone() 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: -1> 2024-04-26T18:34:28.714+0000 7fb4daedf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7fb4daedf700 time 2024-04-26T18:34:28.714215+0000 2024-04-26T18:34:29.138 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fb4e55083dd] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.139 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: clone() 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 0> 2024-04-26T18:34:28.716+0000 7fb4daedf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: in thread 7fb4daedf700 thread_name:ms_dispatch 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7fb4e4290d20] 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2: gsignal() 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 3: abort() 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fb4e5508437] 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7fb4e55085a3] 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x558d182c6822] 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x558d18236598] 2024-04-26T18:34:29.140 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x558d18238c17] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x558d18131d0b] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x558d181322ac] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x558d1811afad] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7fb4e579ce20] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 13: (DispatchQueue::entry()+0x51f) [0x7fb4e5799fdf] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7fb4e586cfc1] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7fb4e42861ca] 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 16: clone() 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101201]: 2024-04-26T18:34:29.141 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:28 smithi090 podman[101811]: 2024-04-26 18:34:28.773554148 +0000 UTC m=+0.018719471 container died ca0879c6923e23ca19f30189212e7341a3965a8a8d4e50bfedec75a357cee598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:34:29.910 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.64 MB/sec execute 354 sec latency 20955.302 ms 2024-04-26T18:34:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:30 smithi119 ceph-mon[32752]: pgmap v784: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 317 op/s 2024-04-26T18:34:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:30 smithi119 ceph-mon[32752]: pgmap v785: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 5.9 MiB/s wr, 145 op/s 2024-04-26T18:34:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:30 smithi167 ceph-mon[32751]: pgmap v784: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 317 op/s 2024-04-26T18:34:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:30 smithi167 ceph-mon[32751]: pgmap v785: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 5.9 MiB/s wr, 145 op/s 2024-04-26T18:34:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:30 smithi090 ceph-mon[29180]: pgmap v784: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 317 op/s 2024-04-26T18:34:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:30 smithi090 ceph-mon[29180]: pgmap v785: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 5.9 MiB/s wr, 145 op/s 2024-04-26T18:34:30.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.62 MB/sec execute 355 sec latency 21955.413 ms 2024-04-26T18:34:31.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:31 smithi090 podman[101811]: 2024-04-26 18:34:31.603644856 +0000 UTC m=+2.848810258 container remove ca0879c6923e23ca19f30189212e7341a3965a8a8d4e50bfedec75a357cee598 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:34:31.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:31 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:34:31.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:31 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-26T18:34:31.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:31 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 27.321s CPU time. 2024-04-26T18:34:31.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.60 MB/sec execute 356 sec latency 22955.508 ms 2024-04-26T18:34:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:32 smithi119 ceph-mon[32752]: pgmap v786: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 5.9 MiB/s wr, 145 op/s 2024-04-26T18:34:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:32 smithi167 ceph-mon[32751]: pgmap v786: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 5.9 MiB/s wr, 145 op/s 2024-04-26T18:34:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:32 smithi090 ceph-mon[29180]: pgmap v786: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 5.9 MiB/s wr, 145 op/s 2024-04-26T18:34:32.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.58 MB/sec execute 357 sec latency 23955.579 ms 2024-04-26T18:34:33.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.56 MB/sec execute 358 sec latency 24955.663 ms 2024-04-26T18:34:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:34 smithi119 ceph-mon[32752]: pgmap v787: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.2 MiB/s wr, 131 op/s 2024-04-26T18:34:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:34 smithi167 ceph-mon[32751]: pgmap v787: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.2 MiB/s wr, 131 op/s 2024-04-26T18:34:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:34 smithi090 ceph-mon[29180]: pgmap v787: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 5.2 MiB/s wr, 131 op/s 2024-04-26T18:34:34.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.54 MB/sec execute 359 sec latency 25955.755 ms 2024-04-26T18:34:35.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.53 MB/sec execute 360 sec latency 26955.864 ms 2024-04-26T18:34:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:36 smithi119 ceph-mon[32752]: pgmap v788: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 5.4 MiB/s wr, 135 op/s 2024-04-26T18:34:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:36 smithi167 ceph-mon[32751]: pgmap v788: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 5.4 MiB/s wr, 135 op/s 2024-04-26T18:34:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:36 smithi090 ceph-mon[29180]: pgmap v788: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 5.4 MiB/s wr, 135 op/s 2024-04-26T18:34:36.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.51 MB/sec execute 361 sec latency 27955.907 ms 2024-04-26T18:34:37.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.49 MB/sec execute 362 sec latency 28956.011 ms 2024-04-26T18:34:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:38 smithi119 ceph-mon[32752]: pgmap v789: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 5.4 MiB/s wr, 135 op/s 2024-04-26T18:34:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:38 smithi167 ceph-mon[32751]: pgmap v789: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 5.4 MiB/s wr, 135 op/s 2024-04-26T18:34:38.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:38 smithi090 ceph-mon[29180]: pgmap v789: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 5.4 MiB/s wr, 135 op/s 2024-04-26T18:34:38.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.47 MB/sec execute 363 sec latency 29956.110 ms 2024-04-26T18:34:39.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.45 MB/sec execute 364 sec latency 30956.169 ms 2024-04-26T18:34:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:40 smithi119 ceph-mon[32752]: pgmap v790: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 6 op/s 2024-04-26T18:34:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:40 smithi167 ceph-mon[32751]: pgmap v790: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 6 op/s 2024-04-26T18:34:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:40 smithi090 ceph-mon[29180]: pgmap v790: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 231 KiB/s wr, 6 op/s 2024-04-26T18:34:40.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.44 MB/sec execute 365 sec latency 31956.278 ms 2024-04-26T18:34:41.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.42 MB/sec execute 366 sec latency 32956.323 ms 2024-04-26T18:34:42.221 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:41 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 4. 2024-04-26T18:34:42.221 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:41 smithi090 systemd[1]: Stopped Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:34:42.221 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:41 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 27.321s CPU time. 2024-04-26T18:34:42.221 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:41 smithi090 systemd[1]: Starting Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:34:42.222 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 podman[101909]: 2024-04-26 18:34:42.016170308 +0000 UTC m=+0.018138317 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:34:42.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:42 smithi167 ceph-mon[32751]: pgmap v791: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 5 op/s 2024-04-26T18:34:42.535 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:42 smithi090 ceph-mon[29180]: pgmap v791: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 5 op/s 2024-04-26T18:34:42.535 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 podman[101909]: 2024-04-26 18:34:42.23198229 +0000 UTC m=+0.233950286 container create 2d7a0b714d2433d0934dfa7135afab55d5940613f29dad2a009a5aa791cd5907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T18:34:42.535 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 podman[101909]: 2024-04-26 18:34:42.434807817 +0000 UTC m=+0.436775813 container init 2d7a0b714d2433d0934dfa7135afab55d5940613f29dad2a009a5aa791cd5907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:34:42.535 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 podman[101909]: 2024-04-26 18:34:42.437370005 +0000 UTC m=+0.439338011 container start 2d7a0b714d2433d0934dfa7135afab55d5940613f29dad2a009a5aa791cd5907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T18:34:42.535 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 ceph-mds[101928]: -- 172.21.15.90:0/1202995360 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a96807fa40 con 0x55a968052800 2024-04-26T18:34:42.535 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: starting mds.j at 2024-04-26T18:34:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:42 smithi119 ceph-mon[32752]: pgmap v791: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 192 KiB/s wr, 5 op/s 2024-04-26T18:34:42.818 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:34:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:34:42.504+0000 7f61a428e700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 8 2024-04-26T18:34:42.818 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:34:42 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:34:42.504+0000 7f61aaa9b700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 7 2024-04-26T18:34:42.875 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 bash[101909]: 2d7a0b714d2433d0934dfa7135afab55d5940613f29dad2a009a5aa791cd5907 2024-04-26T18:34:42.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:42 smithi090 systemd[1]: Started Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:34:42.911 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.40 MB/sec execute 367 sec latency 33956.361 ms 2024-04-26T18:34:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: daemon mds.j restarted 2024-04-26T18:34:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:34:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: osdmap e181: 12 total, 12 up, 12 in 2024-04-26T18:34:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:boot 2024-04-26T18:34:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: Replacing daemon mds.d as rank 2 with standby daemon mds.j 2024-04-26T18:34:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-04-26T18:34:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:34:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:active(laggy or crashed)} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:34:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:34:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: osdmap e182: 12 total, 12 up, 12 in 2024-04-26T18:34:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:43 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: daemon mds.j restarted 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: osdmap e181: 12 total, 12 up, 12 in 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:boot 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: Replacing daemon mds.d as rank 2 with standby daemon mds.j 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:active(laggy or crashed)} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:34:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:34:43.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: osdmap e182: 12 total, 12 up, 12 in 2024-04-26T18:34:43.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:43 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: daemon mds.j restarted 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: osdmap e181: 12 total, 12 up, 12 in 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:boot 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: Replacing daemon mds.d as rank 2 with standby daemon mds.j 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: MDS daemon mds.d is removed because it is dead or otherwise unavailable. 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: Standby daemon mds.g assigned to filesystem cephfs as rank 1 2024-04-26T18:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,2=d=up:active(laggy or crashed)} 2 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:34:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:34:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: osdmap e182: 12 total, 12 up, 12 in 2024-04-26T18:34:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:43 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:replay,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:34:43.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.38 MB/sec execute 368 sec latency 34956.400 ms 2024-04-26T18:34:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:44 smithi119 ceph-mon[32752]: pgmap v794: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3 op/s 2024-04-26T18:34:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:resolve 2024-04-26T18:34:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:44 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:34:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:44 smithi167 ceph-mon[32751]: pgmap v794: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3 op/s 2024-04-26T18:34:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:resolve 2024-04-26T18:34:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:44 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:34:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:44 smithi090 ceph-mon[29180]: pgmap v794: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3 op/s 2024-04-26T18:34:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:resolve 2024-04-26T18:34:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:44 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:34:44.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.37 MB/sec execute 369 sec latency 35956.435 ms 2024-04-26T18:34:45.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.35 MB/sec execute 370 sec latency 36956.493 ms 2024-04-26T18:34:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:46 smithi119 ceph-mon[32752]: pgmap v795: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-04-26T18:34:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:46 smithi167 ceph-mon[32751]: pgmap v795: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-04-26T18:34:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:46 smithi090 ceph-mon[29180]: pgmap v795: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-04-26T18:34:46.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.33 MB/sec execute 371 sec latency 37956.548 ms 2024-04-26T18:34:47.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.31 MB/sec execute 372 sec latency 38956.620 ms 2024-04-26T18:34:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:48 smithi119 ceph-mon[32752]: pgmap v796: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-26T18:34:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:48 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:resolve 2024-04-26T18:34:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:48 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:resolve} 1 up:standby-replay 2024-04-26T18:34:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:48 smithi167 ceph-mon[32751]: pgmap v796: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-26T18:34:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:48 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:resolve 2024-04-26T18:34:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:48 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:resolve} 1 up:standby-replay 2024-04-26T18:34:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:48 smithi090 ceph-mon[29180]: pgmap v796: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-26T18:34:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:48 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:resolve 2024-04-26T18:34:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:48 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:resolve,2=j=up:resolve} 1 up:standby-replay 2024-04-26T18:34:48.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 54917 6.30 MB/sec execute 373 sec latency 39956.668 ms 2024-04-26T18:34:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08d37fe640 1 -- 172.21.15.90:0/3914511498 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f1977aee0 con 0x559f1988f550 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08d37fe640 10 client.16989 handle_command_reply: tid=0 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 client.16989 shutdown 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 msgr2=0x559f19811cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19811cc0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f08ec009600 tx=0x7f08ec011d20 comp rx=0 tx=0).stop 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 shutdown_connections 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x559f1988f550 0x559f19891970 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f198180e0 0x559f19812740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559f1980d820 0x559f19812200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 --2- 172.21.15.90:0/3914511498 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x559f1980cf00 0x559f19811cc0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 >> 172.21.15.90:0/3914511498 conn(0x559f19809410 msgr2=0x559f198158e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08d37fe640 0 client.16989 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 shutdown_connections 2024-04-26T18:34:49.366 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 1 -- 172.21.15.90:0/3914511498 wait complete. 2024-04-26T18:34:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.365+0000 7f08fd264740 20 client.16989 trim_cache size 0 max 16384 2024-04-26T18:34:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:34:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:34:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:34:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f08dc0a3dc0 msgr2=0x7f08dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f08dc0a3dc0 0x7f08dc0a6280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08e40ee490 tx=0x7f08e403b460 comp rx=0 tx=0).stop 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4101850 msgr2=0x7f08f41990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4101850 0x7f08f41990a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f08e4005940 tx=0x7f08e40040c0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 shutdown_connections 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f08dc0a3dc0 0x7f08dc0a6280 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f08f4107860 0x7f08f419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08f4102210 0x7f08f41995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 --2- 172.21.15.90:0/982317473 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08f4101850 0x7f08f41990a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 >> 172.21.15.90:0/982317473 conn(0x7f08f4077bb0 msgr2=0x7f08f40ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 shutdown_connections 2024-04-26T18:34:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:34:49.366+0000 7f08fa546640 1 -- 172.21.15.90:0/982317473 wait complete. 2024-04-26T18:34:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:49 smithi119 ceph-mon[32752]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:34:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:49 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:reconnect 2024-04-26T18:34:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:49 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:reconnect 2024-04-26T18:34:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:49 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=j=up:reconnect} 1 up:standby-replay 2024-04-26T18:34:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:49 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:34:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:49 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:34:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:49 smithi167 ceph-mon[32751]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:34:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:49 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:reconnect 2024-04-26T18:34:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:49 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:reconnect 2024-04-26T18:34:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:49 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=j=up:reconnect} 1 up:standby-replay 2024-04-26T18:34:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:49 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:34:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:49 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:34:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:49 smithi090 ceph-mon[29180]: reconnect by client.15213 172.21.15.90:0/2929304563 after 0.000999985 2024-04-26T18:34:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:49 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:reconnect 2024-04-26T18:34:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:49 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:reconnect 2024-04-26T18:34:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:49 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:reconnect,2=j=up:reconnect} 1 up:standby-replay 2024-04-26T18:34:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:49 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:34:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:49 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.00999985 2024-04-26T18:34:49.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55015 6.29 MB/sec execute 374 sec latency 40309.902 ms 2024-04-26T18:34:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:50 smithi119 ceph-mon[32752]: pgmap v797: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:34:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:50 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:rejoin 2024-04-26T18:34:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:50 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:rejoin 2024-04-26T18:34:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:50 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=j=up:rejoin} 1 up:standby-replay 2024-04-26T18:34:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:50 smithi119 ceph-mon[32752]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:34:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:50 smithi119 ceph-mon[32752]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:34:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:50 smithi167 ceph-mon[32751]: pgmap v797: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:34:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:50 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:rejoin 2024-04-26T18:34:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:50 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:rejoin 2024-04-26T18:34:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:50 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=j=up:rejoin} 1 up:standby-replay 2024-04-26T18:34:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:50 smithi167 ceph-mon[32751]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:34:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:50 smithi167 ceph-mon[32751]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:34:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:50 smithi090 ceph-mon[29180]: pgmap v797: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:34:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:50 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:rejoin 2024-04-26T18:34:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:50 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:rejoin 2024-04-26T18:34:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:50 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:rejoin,2=j=up:rejoin} 1 up:standby-replay 2024-04-26T18:34:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:50 smithi090 ceph-mon[29180]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-04-26T18:34:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:50 smithi090 ceph-mon[29180]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:34:50.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55346 6.33 MB/sec execute 375 sec latency 35547.228 ms 2024-04-26T18:34:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:51 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:34:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:51 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:34:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:51 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:active 2024-04-26T18:34:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:51 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:34:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:51 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:34:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:51 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:34:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:51 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:active 2024-04-26T18:34:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:51 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:34:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:51 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:34:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:51 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:34:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:51 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/2797511346,v1:172.21.15.90:6837/2797511346] up:active 2024-04-26T18:34:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:51 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:34:51.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 55977 6.41 MB/sec execute 376 sec latency 142.467 ms 2024-04-26T18:34:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:52 smithi119 ceph-mon[32752]: pgmap v798: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:34:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:52 smithi167 ceph-mon[32751]: pgmap v798: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:34:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:52 smithi090 ceph-mon[29180]: pgmap v798: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:34:52.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56473 6.46 MB/sec execute 377 sec latency 135.492 ms 2024-04-26T18:34:53.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 56987 6.50 MB/sec execute 378 sec latency 114.903 ms 2024-04-26T18:34:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:54 smithi119 ceph-mon[32752]: pgmap v799: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-04-26T18:34:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:54 smithi167 ceph-mon[32751]: pgmap v799: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-04-26T18:34:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:54 smithi090 ceph-mon[29180]: pgmap v799: 97 pgs: 97 active+clean; 838 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-04-26T18:34:54.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57572 6.56 MB/sec execute 379 sec latency 112.598 ms 2024-04-26T18:34:55.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0863ecd700 time 2024-04-26T18:34:55.447416+0000 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f086e4f63dd] 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: clone() 2024-04-26T18:34:55.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.447+0000 7f0863ecd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0863ecd700 time 2024-04-26T18:34:55.447416+0000 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f086e4f63dd] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: clone() 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: *** Caught signal (Aborted) ** 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: in thread 7f0863ecd700 thread_name:ms_dispatch 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f086d27ed20] 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: gsignal() 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: abort() 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f086e4f6437] 2024-04-26T18:34:55.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.882 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.882 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.882 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 16: clone() 2024-04-26T18:34:55.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.449+0000 7f0863ecd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:55.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: in thread 7f0863ecd700 thread_name:ms_dispatch 2024-04-26T18:34:55.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f086d27ed20] 2024-04-26T18:34:55.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: gsignal() 2024-04-26T18:34:55.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: abort() 2024-04-26T18:34:55.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f086e4f6437] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 16: clone() 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: -1> 2024-04-26T18:34:55.447+0000 7f0863ecd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0863ecd700 time 2024-04-26T18:34:55.447416+0000 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f086e4f63dd] 2024-04-26T18:34:55.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: clone() 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 0> 2024-04-26T18:34:55.449+0000 7f0863ecd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: in thread 7f0863ecd700 thread_name:ms_dispatch 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f086d27ed20] 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: gsignal() 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: abort() 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f086e4f6437] 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 16: clone() 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: -4> 2024-04-26T18:34:55.447+0000 7f0863ecd700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f0863ecd700 time 2024-04-26T18:34:55.447416+0000 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f086e4f63dd] 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: clone() 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: -3> 2024-04-26T18:34:55.449+0000 7f0863ecd700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: in thread 7f0863ecd700 thread_name:ms_dispatch 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:34:55.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f086d27ed20] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2: gsignal() 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 3: abort() 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f086e4f6437] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f086e4f65a3] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x55a966cd1822] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x55a966c41598] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x55a966c43c17] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x55a966b3cd0b] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55a966b3d2ac] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x55a966b25fad] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f086e78ae20] 2024-04-26T18:34:55.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 13: (DispatchQueue::entry()+0x51f) [0x7f086e787fdf] 2024-04-26T18:34:55.893 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f086e85afc1] 2024-04-26T18:34:55.893 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f086d2741ca] 2024-04-26T18:34:55.893 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 16: clone() 2024-04-26T18:34:55.893 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:34:55.893 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[101924]: 2024-04-26T18:34:55.893 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:55 smithi090 podman[101979]: 2024-04-26 18:34:55.518403059 +0000 UTC m=+0.019144121 container died 2d7a0b714d2433d0934dfa7135afab55d5940613f29dad2a009a5aa791cd5907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T18:34:55.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.57 MB/sec execute 380 sec latency 534.188 ms 2024-04-26T18:34:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:56 smithi090 ceph-mon[29180]: pgmap v800: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-04-26T18:34:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:56 smithi119 ceph-mon[32752]: pgmap v800: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-04-26T18:34:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:56 smithi167 ceph-mon[32751]: pgmap v800: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 3.0 MiB/s wr, 235 op/s 2024-04-26T18:34:56.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:56 smithi090 podman[101979]: 2024-04-26 18:34:56.403536712 +0000 UTC m=+0.904277782 container remove 2d7a0b714d2433d0934dfa7135afab55d5940613f29dad2a009a5aa791cd5907 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T18:34:56.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:56 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:34:56.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:56 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-26T18:34:56.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:34:56 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 16.740s CPU time. 2024-04-26T18:34:56.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.56 MB/sec execute 381 sec latency 1534.271 ms 2024-04-26T18:34:57.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.54 MB/sec execute 382 sec latency 2534.316 ms 2024-04-26T18:34:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:34:58 smithi119 ceph-mon[32752]: pgmap v801: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-04-26T18:34:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:34:58 smithi167 ceph-mon[32751]: pgmap v801: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-04-26T18:34:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:34:58 smithi090 ceph-mon[29180]: pgmap v801: 97 pgs: 97 active+clean; 827 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 229 op/s 2024-04-26T18:34:58.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.52 MB/sec execute 383 sec latency 3534.402 ms 2024-04-26T18:34:59.912 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.51 MB/sec execute 384 sec latency 4534.498 ms 2024-04-26T18:35:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:00 smithi119 ceph-mon[32752]: pgmap v802: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.4 MiB/s wr, 336 op/s 2024-04-26T18:35:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:00 smithi167 ceph-mon[32751]: pgmap v802: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.4 MiB/s wr, 336 op/s 2024-04-26T18:35:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:00 smithi090 ceph-mon[29180]: pgmap v802: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 4.4 MiB/s wr, 336 op/s 2024-04-26T18:35:00.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.49 MB/sec execute 385 sec latency 5534.566 ms 2024-04-26T18:35:01.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.47 MB/sec execute 386 sec latency 6534.640 ms 2024-04-26T18:35:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 -- 172.21.15.90:0/61941983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c103300 msgr2=0x7f727c10f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 --2- 172.21.15.90:0/61941983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c103300 0x7f727c10f810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7274009cd0 tx=0x7f727402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 -- 172.21.15.90:0/61941983 shutdown_connections 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 --2- 172.21.15.90:0/61941983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c103300 0x7f727c10f810 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 --2- 172.21.15.90:0/61941983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c102940 0x7f727c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 --2- 172.21.15.90:0/61941983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727c108950 0x7f727c108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 -- 172.21.15.90:0/61941983 >> 172.21.15.90:0/61941983 conn(0x7f727c079dc0 msgr2=0x7f727c07a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 -- 172.21.15.90:0/61941983 shutdown_connections 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.465+0000 7f7280df0640 1 -- 172.21.15.90:0/61941983 wait complete. 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 Processor -- start 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 -- start start 2024-04-26T18:35:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c102940 0x7f727c199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727c103300 0x7f727c1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c108950 0x7f727c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f727c112c10 con 0x7f727c102940 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f727c112a90 con 0x7f727c103300 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7280df0640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f727c112d90 con 0x7f727c108950 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c102940 0x7f727c199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c102940 0x7f727c199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51012/0 (socket says 172.21.15.90:51012) 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 -- 172.21.15.90:0/3148342135 learned_addr learned my addr 172.21.15.90:0/3148342135 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f7273fff640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727c103300 0x7f727c1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727ad9d640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c108950 0x7f727c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 -- 172.21.15.90:0/3148342135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c108950 msgr2=0x7f727c19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c108950 0x7f727c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 -- 172.21.15.90:0/3148342135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727c103300 msgr2=0x7f727c1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727c103300 0x7f727c1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 -- 172.21.15.90:0/3148342135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c19e030 con 0x7f727c102940 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.466+0000 7f727a59c640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c102940 0x7f727c199090 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f726c00e9f0 tx=0x7f726c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.467+0000 7f72737fe640 1 -- 172.21.15.90:0/3148342135 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c00cd80 con 0x7f727c102940 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.467+0000 7f72737fe640 1 -- 172.21.15.90:0/3148342135 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c0108e0 con 0x7f727c102940 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.467+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c19e320 con 0x7f727c102940 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.467+0000 7f72737fe640 1 -- 172.21.15.90:0/3148342135 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c018980 con 0x7f727c102940 2024-04-26T18:35:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.467+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a5bb0 con 0x7f727c102940 2024-04-26T18:35:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.469+0000 7f72737fe640 1 -- 172.21.15.90:0/3148342135 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f726c010430 con 0x7f727c102940 2024-04-26T18:35:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.469+0000 7f7283be1740 1 Processor -- start 2024-04-26T18:35:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7283be1740 1 -- start start 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7283be1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7283be1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a04720 0x55ae92a12fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7283be1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a134f0 0x55ae92a158e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7283be1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ae9296a460 con 0x55ae92a134f0 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7283be1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae92924590 con 0x55ae92a04720 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7283be1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae92966420 con 0x55ae92a03e00 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f72737fe640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f72680a3dc0 0x7f72680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a134f0 0x55ae92a158e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a134f0 0x55ae92a158e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51016/0 (socket says 172.21.15.90:51016) 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 -- 172.21.15.90:0/3912858786 learned_addr learned my addr 172.21.15.90:0/3912858786 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f727a59c640 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a04720 0x55ae92a12fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f727ad9d640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f72680a3dc0 0x7f72680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f727ad9d640 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f72737fe640 1 -- 172.21.15.90:0/3148342135 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(182..182 src has 1..182) ==== 13141+0+0 (secure 0 0 0) 0x7f726c0d1d20 con 0x7f727c102940 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 -- 172.21.15.90:0/3912858786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 msgr2=0x55ae92a041e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a041e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 -- 172.21.15.90:0/3912858786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a04720 msgr2=0x55ae92a12fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a04720 0x55ae92a12fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 -- 172.21.15.90:0/3912858786 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae929310d0 con 0x55ae92a134f0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f7273fff640 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a134f0 0x55ae92a158e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f726400b740 tx=0x7f726400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27ad5ce92edd12b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.470+0000 7f727ad9d640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f72680a3dc0 0x7f72680a6280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7274033040 tx=0x7f7274032fb0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7270ff9640 1 -- 172.21.15.90:0/3912858786 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264003dd0 con 0x55ae92a134f0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7270ff9640 1 -- 172.21.15.90:0/3912858786 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264015070 con 0x55ae92a134f0 2024-04-26T18:35:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7270ff9640 1 -- 172.21.15.90:0/3912858786 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264004d90 con 0x55ae92a134f0 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7283be1740 1 -- 172.21.15.90:0/3912858786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a134f0 msgr2=0x55ae92a158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7283be1740 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a134f0 0x55ae92a158e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f726400b740 tx=0x7f726400bc00 comp rx=0 tx=0).stop 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7283be1740 1 -- 172.21.15.90:0/3912858786 shutdown_connections 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7283be1740 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a134f0 0x55ae92a158e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7283be1740 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a04720 0x55ae92a12fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7283be1740 1 --2- 172.21.15.90:0/3912858786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a041e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.471+0000 7f7283be1740 1 -- 172.21.15.90:0/3912858786 >> 172.21.15.90:0/3912858786 conn(0x55ae92a00410 msgr2=0x55ae92a007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 -- 172.21.15.90:0/3912858786 shutdown_connections 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 -- 172.21.15.90:0/3912858786 wait complete. 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 -- start start 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 Processor -- start 2024-04-26T18:35:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a04720 0x55ae92a108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a10e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ae92a16120 con 0x55ae92a04720 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x55ae928f1b60 con 0x55ae92a134f0 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae92a162a0 con 0x55ae92a03e00 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7283be1740 10 client.0 _ll_register_callbacks cb 0x55ae929162b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7273fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a04720 0x55ae92a108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7273fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a04720 0x55ae92a108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51032/0 (socket says 172.21.15.90:51032) 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f7273fff640 1 -- 172.21.15.90:0/1405214509 learned_addr learned my addr 172.21.15.90:0/1405214509 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f727a59c640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.472+0000 7f727ad9d640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a10e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7273fff640 1 -- 172.21.15.90:0/1405214509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 msgr2=0x55ae92a103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7273fff640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7273fff640 1 -- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 msgr2=0x55ae92a10e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7273fff640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a10e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7273fff640 1 -- 172.21.15.90:0/1405214509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae9267c210 con 0x55ae92a04720 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f727ad9d640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a10e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7273fff640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a04720 0x55ae92a108f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7264006010 tx=0x7f7264003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9c9df07e1a2742 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7261ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7261ffb640 1 -- 172.21.15.90:0/1405214509 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264028060 con 0x55ae92a04720 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7261ffb640 1 -- 172.21.15.90:0/1405214509 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(182..182 src has 1..182) ==== 13141+0+0 (secure 0 0 0) 0x7f7264011850 con 0x55ae92a04720 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7264015070 con 0x55ae92a04720 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7261ffb640 1 -- 172.21.15.90:0/1405214509 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=331) ==== 24+0+0 (secure 0 0 0) 0x7f7264030050 con 0x55ae92a04720 2024-04-26T18:35:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7283be1740 10 client.17007 fetch_fsmap learned FSMap version 331 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=331,monmap=4+}) -- 0x55ae92748620 con 0x55ae92a04720 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7261ffb640 1 -- 172.21.15.90:0/1405214509 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264015070 con 0x55ae92a04720 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7261ffb640 1 -- 172.21.15.90:0/1405214509 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 331) ==== 3530+0+0 (secure 0 0 0) 0x7f7264022070 con 0x55ae92a04720 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7283be1740 10 client.17007 fetch_fsmap finished waiting for FSMap version 331 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7283be1740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7283be1740 20 client.17007 populate_metadata read hostname 'smithi090' 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.473+0000 7f7283be1740 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a23960 0x55ae92a25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.474+0000 7f7283be1740 4 client.17007 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.474+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae92978070 con 0x55ae92a23960 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.474+0000 7f727a59c640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a23960 0x55ae92a25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.474+0000 7f727a59c640 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a23960 0x55ae92a25de0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4c881d13c673639 server_cookie=6c3d108c092023bd in_seq=0 out_seq=0 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.474+0000 7f7261ffb640 10 client.17007 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:35:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7261ffb640 1 -- 172.21.15.90:0/1405214509 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae92978070 con 0x55ae92a23960 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7261ffb640 10 client.17007 handle_command_reply: tid=0 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 client.17007 shutdown 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a04720 msgr2=0x55ae92a108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a04720 0x55ae92a108f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7264006010 tx=0x7f7264003e00 comp rx=0 tx=0).stop 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 shutdown_connections 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a23960 0x55ae92a25de0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a10e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a04720 0x55ae92a108f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7261ffb640 0 client.17007 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 --2- 172.21.15.90:0/1405214509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a03e00 0x55ae92a103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 >> 172.21.15.90:0/1405214509 conn(0x55ae92a00410 msgr2=0x55ae92a06280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 shutdown_connections 2024-04-26T18:35:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 1 -- 172.21.15.90:0/1405214509 wait complete. 2024-04-26T18:35:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.475+0000 7f7283be1740 20 client.17007 trim_cache size 0 max 16384 2024-04-26T18:35:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 Processor -- start 2024-04-26T18:35:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 -- start start 2024-04-26T18:35:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 0x55ae929ff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a2b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1ae640 con 0x55ae92a03e00 2024-04-26T18:35:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1ad170 con 0x55ae92a134f0 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f7283be1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1ad840 con 0x55ae92a04720 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f727a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f727a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51048/0 (socket says 172.21.15.90:51048) 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.490+0000 7f727a59c640 1 -- 172.21.15.90:0/2679213291 learned_addr learned my addr 172.21.15.90:0/2679213291 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f727ad9d640 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a2b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f727a59c640 1 -- 172.21.15.90:0/2679213291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 msgr2=0x55ae929ff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f727a59c640 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 0x55ae929ff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f727a59c640 1 -- 172.21.15.90:0/2679213291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 msgr2=0x55ae92a2b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f727a59c640 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a2b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f727a59c640 1 -- 172.21.15.90:0/2679213291 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae92a04c60 con 0x55ae92a03e00 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f727a59c640 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1efe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f726c099480 tx=0x7f726c0bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e553568fef7e4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f7263fff640 1 -- 172.21.15.90:0/2679213291 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c09ab50 con 0x55ae92a03e00 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f7263fff640 1 -- 172.21.15.90:0/2679213291 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c09adb0 con 0x55ae92a03e00 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f7263fff640 1 -- 172.21.15.90:0/2679213291 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c018b20 con 0x55ae92a03e00 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f7283be1740 1 -- 172.21.15.90:0/2679213291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 msgr2=0x55ae92a1efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.491+0000 7f7283be1740 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1efe0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f726c099480 tx=0x7f726c0bd040 comp rx=0 tx=0).stop 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- 172.21.15.90:0/2679213291 shutdown_connections 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a2b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 0x55ae929ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 --2- 172.21.15.90:0/2679213291 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1efe0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- 172.21.15.90:0/2679213291 >> 172.21.15.90:0/2679213291 conn(0x55ae92a00410 msgr2=0x55ae929fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- 172.21.15.90:0/2679213291 shutdown_connections 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- 172.21.15.90:0/2679213291 wait complete. 2024-04-26T18:35:02.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- start start 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 Processor -- start 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 0x55ae92a1ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x55ae92a101a0 con 0x55ae92a03e00 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1adf10 con 0x55ae92a134f0 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ae92a16230 con 0x55ae92a04720 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7283be1740 10 client.0 _ll_register_callbacks cb 0x7f726c09c5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f727a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f727a59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51064/0 (socket says 172.21.15.90:51064) 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f727a59c640 1 -- 172.21.15.90:0/948411208 learned_addr learned my addr 172.21.15.90:0/948411208 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:02.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f7273fff640 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 0x55ae92a1ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f727a59c640 1 -- 172.21.15.90:0/948411208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 msgr2=0x55ae92a1ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.492+0000 7f727a59c640 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 0x55ae92a1ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f727a59c640 1 -- 172.21.15.90:0/948411208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 msgr2=0x55ae92a184c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f727a59c640 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f727a59c640 1 -- 172.21.15.90:0/948411208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae92a00240 con 0x55ae92a03e00 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f727a59c640 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1b4f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f726c0ce860 tx=0x7f726c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbcd338eeaeac63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7261ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7261ffb640 1 -- 172.21.15.90:0/948411208 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c09cab0 con 0x55ae92a03e00 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7261ffb640 1 -- 172.21.15.90:0/948411208 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(182..182 src has 1..182) ==== 13141+0+0 (secure 0 0 0) 0x7f726c099120 con 0x55ae92a03e00 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726c09adb0 con 0x55ae92a03e00 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7261ffb640 1 -- 172.21.15.90:0/948411208 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=331) ==== 24+0+0 (secure 0 0 0) 0x7f726c09bf00 con 0x55ae92a03e00 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 10 client.17013 fetch_fsmap learned FSMap version 331 2024-04-26T18:35:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=331,monmap=4+}) -- 0x55ae92a8a620 con 0x55ae92a03e00 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7261ffb640 1 -- 172.21.15.90:0/948411208 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c09adb0 con 0x55ae92a03e00 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7261ffb640 1 -- 172.21.15.90:0/948411208 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 331) ==== 3530+0+0 (secure 0 0 0) 0x7f726c09b970 con 0x55ae92a03e00 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 10 client.17013 fetch_fsmap finished waiting for FSMap version 331 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 10 client.17013 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 20 client.17013 populate_metadata read hostname 'smithi090' 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a8acb0 0x55ae92a8d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 4 client.17013 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae92971ee0 con 0x55ae92a8acb0 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.493+0000 7f7273fff640 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a8acb0 0x55ae92a8d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.494+0000 7f7261ffb640 10 client.17013 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:35:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:02.494+0000 7f7273fff640 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a8acb0 0x55ae92a8d0d0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce26e45e7e1dc2c4 server_cookie=1e8964ac223bc057 in_seq=0 out_seq=0 2024-04-26T18:35:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:02 smithi119 ceph-mon[32752]: pgmap v803: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-26T18:35:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:02 smithi167 ceph-mon[32751]: pgmap v803: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-26T18:35:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:02 smithi090 ceph-mon[29180]: pgmap v803: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-26T18:35:02.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.46 MB/sec execute 387 sec latency 7534.730 ms 2024-04-26T18:35:03.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.44 MB/sec execute 388 sec latency 8534.838 ms 2024-04-26T18:35:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:04 smithi119 ceph-mon[32752]: pgmap v804: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-26T18:35:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:04 smithi167 ceph-mon[32751]: pgmap v804: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-26T18:35:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:04 smithi090 ceph-mon[29180]: pgmap v804: 97 pgs: 97 active+clean; 836 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.4 MiB/s wr, 333 op/s 2024-04-26T18:35:04.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.42 MB/sec execute 389 sec latency 9534.891 ms 2024-04-26T18:35:05.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.41 MB/sec execute 390 sec latency 10534.925 ms 2024-04-26T18:35:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:06 smithi119 ceph-mon[32752]: pgmap v805: 97 pgs: 97 active+clean; 837 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 394 op/s 2024-04-26T18:35:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:06 smithi167 ceph-mon[32751]: pgmap v805: 97 pgs: 97 active+clean; 837 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 394 op/s 2024-04-26T18:35:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:06 smithi090 ceph-mon[29180]: pgmap v805: 97 pgs: 97 active+clean; 837 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 394 op/s 2024-04-26T18:35:06.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.39 MB/sec execute 391 sec latency 11534.957 ms 2024-04-26T18:35:07.223 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:06 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 5. 2024-04-26T18:35:07.223 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:06 smithi090 systemd[1]: Stopped Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:35:07.223 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:06 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 16.740s CPU time. 2024-04-26T18:35:07.223 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:06 smithi090 systemd[1]: Starting Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968... 2024-04-26T18:35:07.223 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 podman[102123]: 2024-04-26 18:35:07.022950328 +0000 UTC m=+0.018762204 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:35:07.535 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 podman[102123]: 2024-04-26 18:35:07.223727503 +0000 UTC m=+0.219539415 container create d6ccb1d1c5b37cb5ec13757869ce9f687527bf24769e8de05c9d6d9b8b55699d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:35:07.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 podman[102123]: 2024-04-26 18:35:07.535949164 +0000 UTC m=+0.531761031 container init d6ccb1d1c5b37cb5ec13757869ce9f687527bf24769e8de05c9d6d9b8b55699d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux ) 2024-04-26T18:35:07.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 podman[102123]: 2024-04-26 18:35:07.53856753 +0000 UTC m=+0.534379402 container start d6ccb1d1c5b37cb5ec13757869ce9f687527bf24769e8de05c9d6d9b8b55699d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T18:35:07.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 ceph-mds[102143]: -- 172.21.15.90:0/2527035391 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555adca3ba40 con 0x555adca0f000 2024-04-26T18:35:07.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: starting mds.j at 2024-04-26T18:35:07.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 bash[102123]: d6ccb1d1c5b37cb5ec13757869ce9f687527bf24769e8de05c9d6d9b8b55699d 2024-04-26T18:35:07.876 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:07 smithi090 systemd[1]: Started Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:35:07.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.37 MB/sec execute 392 sec latency 12534.995 ms 2024-04-26T18:35:08.067 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 26 18:35:07 smithi167 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-osd-8[50979]: 2024-04-26T18:35:07.595+0000 7f61a6a93700 -1 bdev(0x564976b28700 /var/lib/ceph/osd/ceph-8/block) aio_submit retries 3 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: pgmap v806: 97 pgs: 97 active+clean; 837 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: daemon mds.j restarted 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: osdmap e183: 12 total, 12 up, 12 in 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:boot 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: Standby daemon mds.j assigned to filesystem cephfs as rank 2 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:active} 1 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:35:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:08 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: pgmap v806: 97 pgs: 97 active+clean; 837 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: daemon mds.j restarted 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: osdmap e183: 12 total, 12 up, 12 in 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:boot 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: Standby daemon mds.j assigned to filesystem cephfs as rank 2 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:active} 1 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:35:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:08 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: pgmap v806: 97 pgs: 97 active+clean; 837 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.5 MiB/s wr, 169 op/s 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: daemon mds.j restarted 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: osdmap e183: 12 total, 12 up, 12 in 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:boot 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: Standby daemon mds.j assigned to filesystem cephfs as rank 2 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: fsmap cephfs:2/3 {0=b=up:active,1=g=up:active} 1 up:standby-replay 1 up:standby, 1 failed 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:35:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:08 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:replay} 1 up:standby-replay 2024-04-26T18:35:08.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.36 MB/sec execute 393 sec latency 13535.031 ms 2024-04-26T18:35:09.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.34 MB/sec execute 394 sec latency 14535.067 ms 2024-04-26T18:35:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:10 smithi119 ceph-mon[32752]: pgmap v808: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:10 smithi167 ceph-mon[32751]: pgmap v808: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:10 smithi090 ceph-mon[29180]: pgmap v808: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:10.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.32 MB/sec execute 395 sec latency 15535.108 ms 2024-04-26T18:35:11.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.31 MB/sec execute 396 sec latency 16535.164 ms 2024-04-26T18:35:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:12 smithi119 ceph-mon[32752]: pgmap v809: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:12 smithi167 ceph-mon[32751]: pgmap v809: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:12 smithi090 ceph-mon[29180]: pgmap v809: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:12.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.29 MB/sec execute 397 sec latency 17535.265 ms 2024-04-26T18:35:13.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.28 MB/sec execute 398 sec latency 18535.362 ms 2024-04-26T18:35:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:14 smithi119 ceph-mon[32752]: pgmap v810: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:resolve 2024-04-26T18:35:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:14 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:resolve} 1 up:standby-replay 2024-04-26T18:35:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:14 smithi167 ceph-mon[32751]: pgmap v810: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:resolve 2024-04-26T18:35:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:14 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:resolve} 1 up:standby-replay 2024-04-26T18:35:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:14 smithi090 ceph-mon[29180]: pgmap v810: 97 pgs: 97 active+clean; 851 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.9 MiB/s wr, 145 op/s 2024-04-26T18:35:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:resolve 2024-04-26T18:35:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:14 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:resolve} 1 up:standby-replay 2024-04-26T18:35:14.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 57819 6.26 MB/sec execute 399 sec latency 19535.432 ms 2024-04-26T18:35:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.358+0000 7f7261ffb640 1 -- 172.21.15.90:0/948411208 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae92971ee0 con 0x55ae92a8acb0 2024-04-26T18:35:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.358+0000 7f7261ffb640 10 client.17013 handle_command_reply: tid=0 2024-04-26T18:35:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.358+0000 7f7283be1740 1 client.17013 shutdown 2024-04-26T18:35:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 msgr2=0x55ae92a1b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:15.359 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1b4f0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f726c0ce860 tx=0x7f726c018420 comp rx=0 tx=0).stop 2024-04-26T18:35:15.360 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 shutdown_connections 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x55ae92a8acb0 0x55ae92a8d0d0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7261ffb640 0 client.17013 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ae92a134f0 0x55ae92a184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ae92a04720 0x55ae92a1ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 --2- 172.21.15.90:0/948411208 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x55ae92a03e00 0x55ae92a1b4f0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.359+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 >> 172.21.15.90:0/948411208 conn(0x55ae92a00410 msgr2=0x55ae92a10bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.360+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 shutdown_connections 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.360+0000 7f7283be1740 1 -- 172.21.15.90:0/948411208 wait complete. 2024-04-26T18:35:15.361 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.360+0000 7f7283be1740 20 client.17013 trim_cache size 0 max 16384 2024-04-26T18:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:{ 2024-04-26T18:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "message": "", 2024-04-26T18:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout: "return_code": 0 2024-04-26T18:35:15.362 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stdout:} 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f72680a3dc0 msgr2=0x7f72680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f72680a3dc0 0x7f72680a6280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7274033040 tx=0x7f7274032fb0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c102940 msgr2=0x7f727c199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c102940 0x7f727c199090 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f726c00e9f0 tx=0x7f726c00eeb0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 shutdown_connections 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f72680a3dc0 0x7f72680a6280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f727c108950 0x7f727c19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.363 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f727c103300 0x7f727c1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 --2- 172.21.15.90:0/3148342135 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f727c102940 0x7f727c199090 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 >> 172.21.15.90:0/3148342135 conn(0x7f727c079dc0 msgr2=0x7f727c100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 shutdown_connections 2024-04-26T18:35:15.364 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:15.361+0000 7f7280df0640 1 -- 172.21.15.90:0/3148342135 wait complete. 2024-04-26T18:35:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:reconnect 2024-04-26T18:35:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:15 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:reconnect} 1 up:standby-replay 2024-04-26T18:35:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:15 smithi167 ceph-mon[32751]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:35:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:reconnect 2024-04-26T18:35:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:15 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:reconnect} 1 up:standby-replay 2024-04-26T18:35:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:15 smithi090 ceph-mon[29180]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:35:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:reconnect 2024-04-26T18:35:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:15 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:reconnect} 1 up:standby-replay 2024-04-26T18:35:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:15 smithi119 ceph-mon[32752]: reconnect by client.24733 172.21.15.90:0/3415564344 after 0.0109998 2024-04-26T18:35:15.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58063 6.26 MB/sec execute 400 sec latency 20481.680 ms 2024-04-26T18:35:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:16 smithi167 ceph-mon[32751]: pgmap v811: 97 pgs: 97 active+clean; 843 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 92 op/s 2024-04-26T18:35:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:16 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:rejoin 2024-04-26T18:35:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:16 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:rejoin} 1 up:standby-replay 2024-04-26T18:35:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:16 smithi090 ceph-mon[29180]: pgmap v811: 97 pgs: 97 active+clean; 843 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 92 op/s 2024-04-26T18:35:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:16 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:rejoin 2024-04-26T18:35:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:16 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:rejoin} 1 up:standby-replay 2024-04-26T18:35:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:16 smithi119 ceph-mon[32752]: pgmap v811: 97 pgs: 97 active+clean; 843 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 92 op/s 2024-04-26T18:35:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:16 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:rejoin 2024-04-26T18:35:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:16 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:rejoin} 1 up:standby-replay 2024-04-26T18:35:16.913 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58615 6.35 MB/sec execute 401 sec latency 21481.738 ms 2024-04-26T18:35:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:17 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:clientreplay 2024-04-26T18:35:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:17 smithi090 ceph-mon[29180]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:clientreplay} 1 up:standby-replay 2024-04-26T18:35:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:17 smithi090 ceph-mon[29180]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:35:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:17 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:clientreplay 2024-04-26T18:35:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:17 smithi119 ceph-mon[32752]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:clientreplay} 1 up:standby-replay 2024-04-26T18:35:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:17 smithi119 ceph-mon[32752]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:35:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:17 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:clientreplay 2024-04-26T18:35:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:17 smithi167 ceph-mon[32751]: fsmap cephfs:3/3 {0=b=up:active,1=g=up:active,2=j=up:clientreplay} 1 up:standby-replay 2024-04-26T18:35:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:17 smithi167 ceph-mon[32751]: daemon mds.j is now active in filesystem cephfs as rank 2 2024-04-26T18:35:17.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.35 MB/sec execute 402 sec latency 22226.745 ms 2024-04-26T18:35:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:18 smithi090 ceph-mon[29180]: pgmap v812: 97 pgs: 97 active+clean; 843 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 92 op/s 2024-04-26T18:35:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:18 smithi090 ceph-mon[29180]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:35:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:18 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:active 2024-04-26T18:35:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:18 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:35:18.804 INFO:journalctl@ceph.mds.b.smithi119.stdout:Apr 26 18:35:18 smithi119 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-b[63541]: 2024-04-26T18:35:18.304+0000 7fdb72b8b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-26T18:35:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:18 smithi119 ceph-mon[32752]: pgmap v812: 97 pgs: 97 active+clean; 843 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 92 op/s 2024-04-26T18:35:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:18 smithi119 ceph-mon[32752]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:35:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:18 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:active 2024-04-26T18:35:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:18 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:35:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:18 smithi167 ceph-mon[32751]: pgmap v812: 97 pgs: 97 active+clean; 843 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 92 op/s 2024-04-26T18:35:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:18 smithi167 ceph-mon[32751]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:35:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:18 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6836/507519675,v1:172.21.15.90:6837/507519675] up:active 2024-04-26T18:35:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:18 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:35:18.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.34 MB/sec execute 403 sec latency 1219.296 ms 2024-04-26T18:35:19.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.32 MB/sec execute 404 sec latency 2219.425 ms 2024-04-26T18:35:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:20 smithi090 ceph-mon[29180]: pgmap v813: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-04-26T18:35:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:20 smithi119 ceph-mon[32752]: pgmap v813: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-04-26T18:35:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:20 smithi167 ceph-mon[32751]: pgmap v813: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 4.1 MiB/s wr, 224 op/s 2024-04-26T18:35:20.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.31 MB/sec execute 405 sec latency 3219.531 ms 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2223abf700 time 2024-04-26T18:35:21.449478+0000 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f222e0e83dd] 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: clone() 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.449+0000 7f2223abf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2223abf700 time 2024-04-26T18:35:21.449478+0000 2024-04-26T18:35:21.878 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f222e0e83dd] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.879 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: clone() 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: *** Caught signal (Aborted) ** 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: in thread 7f2223abf700 thread_name:ms_dispatch 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f222ce70d20] 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: gsignal() 2024-04-26T18:35:21.880 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: abort() 2024-04-26T18:35:21.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f222e0e8437] 2024-04-26T18:35:21.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.881 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.882 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.882 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.882 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.882 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 16: clone() 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.451+0000 7f2223abf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: in thread 7f2223abf700 thread_name:ms_dispatch 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f222ce70d20] 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: gsignal() 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: abort() 2024-04-26T18:35:21.883 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f222e0e8437] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.884 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 16: clone() 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: -1> 2024-04-26T18:35:21.449+0000 7f2223abf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2223abf700 time 2024-04-26T18:35:21.449478+0000 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f222e0e83dd] 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.885 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: clone() 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 0> 2024-04-26T18:35:21.451+0000 7f2223abf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:35:21.886 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: in thread 7f2223abf700 thread_name:ms_dispatch 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f222ce70d20] 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: gsignal() 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: abort() 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f222e0e8437] 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.887 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 16: clone() 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: -1> 2024-04-26T18:35:21.449+0000 7f2223abf700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: In function 'MDRequestRef MDCache::request_get(metareqid_t)' thread 7f2223abf700 time 2024-04-26T18:35:21.449478+0000 2024-04-26T18:35:21.888 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/19.0.0-3267-gbc94a2a9/rpm/el8/BUILD/ceph-19.0.0-3267-gbc94a2a9/src/mds/MDCache.cc: 9728: FAILED ceph_assert(p != active_requests.end()) 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7f222e0e83dd] 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.889 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: clone() 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 0> 2024-04-26T18:35:21.451+0000 7f2223abf700 -1 *** Caught signal (Aborted) ** 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: in thread 7f2223abf700 thread_name:ms_dispatch 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f222ce70d20] 2024-04-26T18:35:21.890 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2: gsignal() 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 3: abort() 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7f222e0e8437] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x2d15a3) [0x7f222e0e85a3] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 6: (MDCache::request_get(metareqid_t)+0x1c2) [0x555adb8a5822] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 7: (Server::handle_peer_request_reply(boost::intrusive_ptr const&)+0xa8) [0x555adb815598] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 8: (Server::dispatch(boost::intrusive_ptr const&)+0x97) [0x555adb817c17] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x5fb) [0x555adb710d0b] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x555adb7112ac] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x1bd) [0x555adb6f9fad] 2024-04-26T18:35:21.891 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 12: (Messenger::ms_deliver_dispatch(boost::intrusive_ptr const&)+0x470) [0x7f222e37ce20] 2024-04-26T18:35:21.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 13: (DispatchQueue::entry()+0x51f) [0x7f222e379fdf] 2024-04-26T18:35:21.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 14: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f222e44cfc1] 2024-04-26T18:35:21.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 15: /lib64/libpthread.so.0(+0x81ca) [0x7f222ce661ca] 2024-04-26T18:35:21.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 16: clone() 2024-04-26T18:35:21.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-04-26T18:35:21.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j[102139]: 2024-04-26T18:35:21.892 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:21 smithi090 podman[102180]: 2024-04-26 18:35:21.5223504 +0000 UTC m=+0.019752898 container died d6ccb1d1c5b37cb5ec13757869ce9f687527bf24769e8de05c9d6d9b8b55699d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True) 2024-04-26T18:35:21.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.29 MB/sec execute 406 sec latency 4219.617 ms 2024-04-26T18:35:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:22 smithi090 ceph-mon[29180]: pgmap v814: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-04-26T18:35:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:22 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:35:22.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:22 smithi090 podman[102180]: 2024-04-26 18:35:22.303709416 +0000 UTC m=+0.801111903 container remove d6ccb1d1c5b37cb5ec13757869ce9f687527bf24769e8de05c9d6d9b8b55699d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T18:35:22.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:22 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Main process exited, code=exited, status=134/n/a 2024-04-26T18:35:22.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:22 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-26T18:35:22.626 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:22 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 10.122s CPU time. 2024-04-26T18:35:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:22 smithi119 ceph-mon[32752]: pgmap v814: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-04-26T18:35:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:22 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:35:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:22 smithi167 ceph-mon[32751]: pgmap v814: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-04-26T18:35:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:22 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active} 1 up:standby-replay 2024-04-26T18:35:22.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.28 MB/sec execute 407 sec latency 5219.657 ms 2024-04-26T18:35:23.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.26 MB/sec execute 408 sec latency 6219.728 ms 2024-04-26T18:35:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:24 smithi090 ceph-mon[29180]: pgmap v815: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-04-26T18:35:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:24 smithi119 ceph-mon[32752]: pgmap v815: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-04-26T18:35:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:24 smithi167 ceph-mon[32751]: pgmap v815: 97 pgs: 97 active+clean; 837 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-04-26T18:35:24.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.24 MB/sec execute 409 sec latency 7219.826 ms 2024-04-26T18:35:25.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.23 MB/sec execute 410 sec latency 8219.926 ms 2024-04-26T18:35:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:26 smithi090 ceph-mon[29180]: pgmap v816: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-04-26T18:35:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:26 smithi119 ceph-mon[32752]: pgmap v816: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-04-26T18:35:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:26 smithi167 ceph-mon[32751]: pgmap v816: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-04-26T18:35:26.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.21 MB/sec execute 411 sec latency 9219.983 ms 2024-04-26T18:35:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:27 smithi119 ceph-mon[32752]: pgmap v817: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-04-26T18:35:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:27 smithi167 ceph-mon[32751]: pgmap v817: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-04-26T18:35:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:27 smithi090 ceph-mon[29180]: pgmap v817: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.3 MiB/s wr, 132 op/s 2024-04-26T18:35:27.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.20 MB/sec execute 412 sec latency 10220.101 ms 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 -- 172.21.15.90:0/2019491576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd4104500 msgr2=0x7f2cd410c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 --2- 172.21.15.90:0/2019491576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd4104500 0x7f2cd410c9e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009900 tx=0x7f2cbc02a480 comp rx=0 tx=0).stop 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 -- 172.21.15.90:0/2019491576 shutdown_connections 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 --2- 172.21.15.90:0/2019491576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd410cfb0 0x7f2cd410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 --2- 172.21.15.90:0/2019491576 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd4104500 0x7f2cd410c9e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 --2- 172.21.15.90:0/2019491576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cd4103be0 0x7f2cd4103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 -- 172.21.15.90:0/2019491576 >> 172.21.15.90:0/2019491576 conn(0x7f2cd4077bb0 msgr2=0x7f2cd4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 -- 172.21.15.90:0/2019491576 shutdown_connections 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.460+0000 7f2cd8852640 1 -- 172.21.15.90:0/2019491576 wait complete. 2024-04-26T18:35:28.463 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 Processor -- start 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 -- start start 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd4103be0 0x7f2cd419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd4104500 0x7f2cd419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cd410cfb0 0x7f2cd419fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4112bf0 con 0x7f2cd4104500 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4112a70 con 0x7f2cd4103be0 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.461+0000 7f2cd8852640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd4112d70 con 0x7f2cd410cfb0 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd4104500 0x7f2cd419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd4104500 0x7f2cd419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37536/0 (socket says 172.21.15.90:37536) 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 -- 172.21.15.90:0/3143758137 learned_addr learned my addr 172.21.15.90:0/3143758137 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd4103be0 0x7f2cd419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cd410cfb0 0x7f2cd419fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.464 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cd410cfb0 msgr2=0x7f2cd419fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cd410cfb0 0x7f2cd419fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd4103be0 msgr2=0x7f2cd419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd4103be0 0x7f2cd419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd41a01f0 con 0x7f2cd4104500 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2cd410cfb0 0x7f2cd419fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd4103be0 0x7f2cd419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2cd4104500 0x7f2cd419b740 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc005940 tx=0x7f2cbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc03f030 con 0x7f2cd4104500 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cbc003aa0 con 0x7f2cd4104500 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2cd8852640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd41a0480 con 0x7f2cd4104500 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc035070 con 0x7f2cd4104500 2024-04-26T18:35:28.465 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.462+0000 7f2cd8852640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd41a7d70 con 0x7f2cd4104500 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.464+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f2cbc042030 con 0x7f2cd4104500 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 Processor -- start 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 -- start start 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x5630289342e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x56302893eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56302893f0e0 0x5630289414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x56302889a460 con 0x563028933f00 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563028854590 con 0x563028934820 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.465+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563028896420 con 0x56302893f0e0 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cd2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x5630289342e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cd2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x5630289342e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37544/0 (socket says 172.21.15.90:37544) 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/4172385486 learned_addr learned my addr 172.21.15.90:0/4172385486 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x56302893eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f2cbc0f1f30 con 0x7f2cd4104500 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56302893f0e0 0x5630289414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccbfff640 1 -- 172.21.15.90:0/4172385486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56302893f0e0 msgr2=0x5630289414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56302893f0e0 0x5630289414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccbfff640 1 -- 172.21.15.90:0/4172385486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 msgr2=0x5630289342e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x5630289342e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccbfff640 1 -- 172.21.15.90:0/4172385486 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630288610d0 con 0x563028934820 2024-04-26T18:35:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x5630289342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400afb0 tx=0x7f2cc400a2d0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x56302893eba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0ee420 tx=0x7f2cbc03b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70c6cdd254cbf246 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cc8ff9640 1 -- 172.21.15.90:0/4172385486 <== mon.2 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc049070 con 0x563028934820 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cc8ff9640 1 -- 172.21.15.90:0/4172385486 <== mon.2 v2:172.21.15.119:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cbc0bd370 con 0x563028934820 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.466+0000 7f2cc8ff9640 1 -- 172.21.15.90:0/4172385486 <== mon.2 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc0c2530 con 0x563028934820 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 -- 172.21.15.90:0/4172385486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 msgr2=0x56302893eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x56302893eba0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc0ee420 tx=0x7f2cbc03b460 comp rx=0 tx=0).stop 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 -- 172.21.15.90:0/4172385486 shutdown_connections 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56302893f0e0 0x5630289414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x56302893eba0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/4172385486 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x5630289342e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 -- 172.21.15.90:0/4172385486 >> 172.21.15.90:0/4172385486 conn(0x563028930410 msgr2=0x5630289307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 -- 172.21.15.90:0/4172385486 shutdown_connections 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 -- 172.21.15.90:0/4172385486 wait complete. 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.467+0000 7f2cdb56f740 1 -- start start 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 1 Processor -- start 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x56302893c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028934820 0x56302893c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56302893f0e0 0x56302893cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x563028854590 con 0x563028933f00 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x563028821b60 con 0x56302893f0e0 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563028941ad0 con 0x563028934820 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x56302893c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cdb56f740 10 client.0 _ll_register_callbacks cb 0x5630288462b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x56302893c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37550/0 (socket says 172.21.15.90:37550) 2024-04-26T18:35:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/2569570163 learned_addr learned my addr 172.21.15.90:0/2569570163 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56302893f0e0 0x56302893cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028934820 0x56302893c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/2569570163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028934820 msgr2=0x56302893c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028934820 0x56302893c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56302893f0e0 msgr2=0x56302893cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56302893f0e0 0x56302893cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/2569570163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630285ac210 con 0x563028933f00 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x56302893c0e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010fc0 tx=0x7f2cc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0a2e4f1ca72f91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.468+0000 7f2cb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/2569570163 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4016070 con 0x563028933f00 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/2569570163 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f2cc4027680 con 0x563028933f00 2024-04-26T18:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cbc0bd370 con 0x563028933f00 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/2569570163 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=339) ==== 24+0+0 (secure 0 0 0) 0x7f2cc4027880 con 0x563028933f00 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 10 client.17031 fetch_fsmap learned FSMap version 339 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=339,monmap=4+}) -- 0x563028678620 con 0x563028933f00 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/2569570163 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cbc0bd370 con 0x563028933f00 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/2569570163 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 339) ==== 3534+0+0 (secure 0 0 0) 0x7f2cc401cb50 con 0x563028933f00 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 10 client.17031 fetch_fsmap finished waiting for FSMap version 339 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 10 client.17031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 20 client.17031 populate_metadata read hostname 'smithi090' 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56302894f3d0 0x563028951850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 4 client.17031 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630288a8070 con 0x56302894f3d0 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.469+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56302894f3d0 0x563028951850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56302894f3d0 0x563028951850 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8349846d9206a25d server_cookie=a0d86fcc3fc86725 in_seq=0 out_seq=0 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cb9ffb640 10 client.17031 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/2569570163 <== mds.0 v2:172.21.15.119:6838/1002360318 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630288a8070 con 0x56302894f3d0 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cb9ffb640 10 client.17031 handle_command_reply: tid=0 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 client.17031 shutdown 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 msgr2=0x56302893c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x56302893c0e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010fc0 tx=0x7f2cc4004730 comp rx=0 tx=0).stop 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 shutdown_connections 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x56302894f3d0 0x563028951850 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56302893f0e0 0x56302893cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cb9ffb640 0 client.17031 ms_handle_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:35:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028934820 0x56302893c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/2569570163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x563028933f00 0x56302893c0e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.470+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 >> 172.21.15.90:0/2569570163 conn(0x563028930410 msgr2=0x563028936190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.471+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 shutdown_connections 2024-04-26T18:35:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.471+0000 7f2cdb56f740 1 -- 172.21.15.90:0/2569570163 wait complete. 2024-04-26T18:35:28.480 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.471+0000 7f2cdb56f740 20 client.17031 trim_cache size 0 max 16384 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 Processor -- start 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 -- start start 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 0x56302894a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028957120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x563028957660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41b3320 con 0x56302893f0e0 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41b1e50 con 0x563028934820 2024-04-26T18:35:28.490 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.485+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41b2520 con 0x563028933f00 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x563028957660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x563028957660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37560/0 (socket says 172.21.15.90:37560) 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3397918750 learned_addr learned my addr 172.21.15.90:0/3397918750 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2ccbfff640 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028957120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 0x56302894a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3397918750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 msgr2=0x56302894a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 0x56302894a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3397918750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 msgr2=0x563028957120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028957120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3397918750 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563028934650 con 0x56302893f0e0 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x563028957660 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0127f0 tx=0x7f2ccc009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6455c119629b4b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cbbfff640 1 -- 172.21.15.90:0/3397918750 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc022070 con 0x56302893f0e0 2024-04-26T18:35:28.491 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cbbfff640 1 -- 172.21.15.90:0/3397918750 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc011d70 con 0x56302893f0e0 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.486+0000 7f2cbbfff640 1 -- 172.21.15.90:0/3397918750 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc004430 con 0x56302893f0e0 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3397918750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 msgr2=0x563028957660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x563028957660 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0127f0 tx=0x7f2ccc009a30 comp rx=0 tx=0).stop 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3397918750 shutdown_connections 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x563028957660 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028957120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/3397918750 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 0x56302894a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3397918750 >> 172.21.15.90:0/3397918750 conn(0x563028930410 msgr2=0x563028934d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3397918750 shutdown_connections 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3397918750 wait complete. 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- start start 2024-04-26T18:35:28.492 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 Processor -- start 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 0x563028942a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028942fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x5630289b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x5630288a8070 con 0x56302893f0e0 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd41b2bf0 con 0x563028934820 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56302892bd60 con 0x563028933f00 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cdb56f740 10 client.0 _ll_register_callbacks cb 0x7f2cc4010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2cd2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x5630289b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.487+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028942fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x5630289b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37568/0 (socket says 172.21.15.90:37568) 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028942fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.90:43286/0 (socket says 172.21.15.90:43286) 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3906338670 learned_addr learned my addr 172.21.15.90:0/3906338670 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T18:35:28.493 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 0x563028942a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 msgr2=0x563028942a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563028933f00 0x563028942a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 msgr2=0x563028942fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563028934820 0x563028942fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630289438c0 con 0x56302893f0e0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cd2d9d640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x56302893f0e0 0x5630289b4990 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc006010 tx=0x7f2ccc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9637c1dff7c980 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc02e060 con 0x56302893f0e0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f2ccc004a40 con 0x56302893f0e0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ccc011d70 con 0x56302893f0e0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=339) ==== 24+0+0 (secure 0 0 0) 0x7f2ccc036050 con 0x56302893f0e0 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cdb56f740 10 client.17037 fetch_fsmap learned FSMap version 339 2024-04-26T18:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=339,monmap=4+}) -- 0x5630289b5fe0 con 0x56302893f0e0 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ccc011d70 con 0x56302893f0e0 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 339) ==== 3534+0+0 (secure 0 0 0) 0x7f2ccc01c070 con 0x56302893f0e0 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cdb56f740 10 client.17037 fetch_fsmap finished waiting for FSMap version 339 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cdb56f740 10 client.17037 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.488+0000 7f2cdb56f740 20 client.17037 populate_metadata read hostname 'smithi090' 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.489+0000 7f2cdb56f740 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.489+0000 7f2cdb56f740 4 client.17037 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.489+0000 7f2cdb56f740 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5630288a1ee0 con 0x5630289b6650 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.489+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.489+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:35:28.489+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4074875e42f583f server_cookie=29cfa9ced9ecb106 in_seq=0 out_seq=0 2024-04-26T18:35:28.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.18 MB/sec execute 413 sec latency 11220.144 ms 2024-04-26T18:35:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:29 smithi167 ceph-mon[32751]: pgmap v818: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-04-26T18:35:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:35:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:35:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:35:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:29 smithi167 ceph-mon[32751]: pgmap v819: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 74 KiB/s wr, 9 op/s 2024-04-26T18:35:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:29 smithi167 ceph-mon[32751]: pgmap v820: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 11 op/s 2024-04-26T18:35:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:35:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:29 smithi090 ceph-mon[29180]: pgmap v818: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-04-26T18:35:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:35:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:35:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:35:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:29 smithi090 ceph-mon[29180]: pgmap v819: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 74 KiB/s wr, 9 op/s 2024-04-26T18:35:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:29 smithi090 ceph-mon[29180]: pgmap v820: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 11 op/s 2024-04-26T18:35:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:35:29.914 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.17 MB/sec execute 414 sec latency 12220.199 ms 2024-04-26T18:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:29 smithi119 ceph-mon[32752]: pgmap v818: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.4 MiB/s wr, 134 op/s 2024-04-26T18:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:29 smithi119 ceph-mon[32752]: pgmap v819: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 74 KiB/s wr, 9 op/s 2024-04-26T18:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:29 smithi119 ceph-mon[32752]: pgmap v820: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 11 op/s 2024-04-26T18:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:35:30.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.15 MB/sec execute 415 sec latency 13220.253 ms 2024-04-26T18:35:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:31 smithi119 ceph-mon[32752]: pgmap v821: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 11 op/s 2024-04-26T18:35:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:31 smithi167 ceph-mon[32751]: pgmap v821: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 11 op/s 2024-04-26T18:35:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:31 smithi090 ceph-mon[29180]: pgmap v821: 97 pgs: 97 active+clean; 835 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 91 KiB/s wr, 11 op/s 2024-04-26T18:35:31.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.14 MB/sec execute 416 sec latency 14220.613 ms 2024-04-26T18:35:32.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.12 MB/sec execute 417 sec latency 15220.743 ms 2024-04-26T18:35:33.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:32 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Scheduled restart job, restart counter is at 6. 2024-04-26T18:35:33.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:32 smithi090 systemd[1]: Stopped Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:35:33.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:32 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Consumed 10.122s CPU time. 2024-04-26T18:35:33.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:32 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Start request repeated too quickly. 2024-04-26T18:35:33.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:32 smithi090 systemd[1]: ceph-b8878a18-03f6-11ef-bc93-c7b262605968@mds.j.service: Failed with result 'exit-code'. 2024-04-26T18:35:33.126 INFO:journalctl@ceph.mds.j.smithi090.stdout:Apr 26 18:35:32 smithi090 systemd[1]: Failed to start Ceph mds.j for b8878a18-03f6-11ef-bc93-c7b262605968. 2024-04-26T18:35:33.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.11 MB/sec execute 418 sec latency 16220.779 ms 2024-04-26T18:35:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:33 smithi119 ceph-mon[32752]: pgmap v822: 97 pgs: 97 active+clean; 836 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 77 KiB/s wr, 7 op/s 2024-04-26T18:35:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:33 smithi167 ceph-mon[32751]: pgmap v822: 97 pgs: 97 active+clean; 836 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 77 KiB/s wr, 7 op/s 2024-04-26T18:35:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:33 smithi090 ceph-mon[29180]: pgmap v822: 97 pgs: 97 active+clean; 836 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 77 KiB/s wr, 7 op/s 2024-04-26T18:35:34.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.10 MB/sec execute 419 sec latency 17220.821 ms 2024-04-26T18:35:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:35 smithi119 ceph-mon[32752]: pgmap v823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.1 MiB/s wr, 84 op/s 2024-04-26T18:35:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:35 smithi167 ceph-mon[32751]: pgmap v823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.1 MiB/s wr, 84 op/s 2024-04-26T18:35:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:35 smithi090 ceph-mon[29180]: pgmap v823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 2.1 MiB/s wr, 84 op/s 2024-04-26T18:35:35.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.08 MB/sec execute 420 sec latency 18220.911 ms 2024-04-26T18:35:36.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.07 MB/sec execute 421 sec latency 19221.015 ms 2024-04-26T18:35:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:37 smithi119 ceph-mon[32752]: pgmap v824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.0 MiB/s wr, 80 op/s 2024-04-26T18:35:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:37 smithi167 ceph-mon[32751]: pgmap v824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.0 MiB/s wr, 80 op/s 2024-04-26T18:35:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:37 smithi090 ceph-mon[29180]: pgmap v824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 2.0 MiB/s wr, 80 op/s 2024-04-26T18:35:37.915 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.05 MB/sec execute 422 sec latency 20221.117 ms 2024-04-26T18:35:38.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.04 MB/sec execute 423 sec latency 21221.230 ms 2024-04-26T18:35:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:35:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:35:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:35:39.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.02 MB/sec execute 424 sec latency 22221.318 ms 2024-04-26T18:35:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:39 smithi119 ceph-mon[32752]: pgmap v825: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-04-26T18:35:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:39 smithi167 ceph-mon[32751]: pgmap v825: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-04-26T18:35:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:39 smithi090 ceph-mon[29180]: pgmap v825: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-04-26T18:35:40.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.01 MB/sec execute 425 sec latency 23221.377 ms 2024-04-26T18:35:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:41 smithi119 ceph-mon[32752]: pgmap v826: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-04-26T18:35:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:41 smithi167 ceph-mon[32751]: pgmap v826: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-04-26T18:35:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:41 smithi090 ceph-mon[29180]: pgmap v826: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-04-26T18:35:41.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 6.00 MB/sec execute 426 sec latency 24221.490 ms 2024-04-26T18:35:42.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.98 MB/sec execute 427 sec latency 25221.581 ms 2024-04-26T18:35:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:43 smithi119 ceph-mon[32752]: pgmap v827: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-04-26T18:35:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:43 smithi167 ceph-mon[32751]: pgmap v827: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-04-26T18:35:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:43 smithi090 ceph-mon[29180]: pgmap v827: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 1.6 MiB/s wr, 63 op/s 2024-04-26T18:35:43.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.97 MB/sec execute 428 sec latency 26221.683 ms 2024-04-26T18:35:44.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.95 MB/sec execute 429 sec latency 27221.777 ms 2024-04-26T18:35:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:45 smithi119 ceph-mon[32752]: pgmap v828: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-04-26T18:35:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:45 smithi167 ceph-mon[32751]: pgmap v828: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-04-26T18:35:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:45 smithi090 ceph-mon[29180]: pgmap v828: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 1.5 MiB/s wr, 62 op/s 2024-04-26T18:35:45.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.94 MB/sec execute 430 sec latency 28221.892 ms 2024-04-26T18:35:46.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.93 MB/sec execute 431 sec latency 29221.969 ms 2024-04-26T18:35:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:47 smithi119 ceph-mon[32752]: pgmap v829: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-04-26T18:35:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:47 smithi167 ceph-mon[32751]: pgmap v829: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-04-26T18:35:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:47 smithi090 ceph-mon[29180]: pgmap v829: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-04-26T18:35:47.916 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.91 MB/sec execute 432 sec latency 30222.067 ms 2024-04-26T18:35:48.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.90 MB/sec execute 433 sec latency 31222.174 ms 2024-04-26T18:35:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:49 smithi119 ceph-mon[32752]: pgmap v830: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 7 op/s 2024-04-26T18:35:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:49 smithi167 ceph-mon[32751]: pgmap v830: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 7 op/s 2024-04-26T18:35:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:49 smithi090 ceph-mon[29180]: pgmap v830: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 67 KiB/s wr, 7 op/s 2024-04-26T18:35:49.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.89 MB/sec execute 434 sec latency 32222.309 ms 2024-04-26T18:35:50.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.87 MB/sec execute 435 sec latency 33222.421 ms 2024-04-26T18:35:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:51 smithi119 ceph-mon[32752]: pgmap v831: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 3 op/s 2024-04-26T18:35:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:51 smithi167 ceph-mon[32751]: pgmap v831: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 3 op/s 2024-04-26T18:35:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:51 smithi090 ceph-mon[29180]: pgmap v831: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 3 op/s 2024-04-26T18:35:51.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.86 MB/sec execute 436 sec latency 34222.522 ms 2024-04-26T18:35:52.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.84 MB/sec execute 437 sec latency 35222.580 ms 2024-04-26T18:35:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:53 smithi119 ceph-mon[32752]: pgmap v832: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 3 op/s 2024-04-26T18:35:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:53 smithi167 ceph-mon[32751]: pgmap v832: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 3 op/s 2024-04-26T18:35:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:53 smithi090 ceph-mon[29180]: pgmap v832: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9.3 KiB/s wr, 3 op/s 2024-04-26T18:35:53.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.83 MB/sec execute 438 sec latency 36222.707 ms 2024-04-26T18:35:54.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.82 MB/sec execute 439 sec latency 37222.802 ms 2024-04-26T18:35:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:55 smithi119 ceph-mon[32752]: pgmap v833: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-26T18:35:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:55 smithi167 ceph-mon[32751]: pgmap v833: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-26T18:35:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:55 smithi090 ceph-mon[29180]: pgmap v833: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 11 KiB/s wr, 4 op/s 2024-04-26T18:35:55.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.80 MB/sec execute 440 sec latency 38222.922 ms 2024-04-26T18:35:56.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.79 MB/sec execute 441 sec latency 39222.965 ms 2024-04-26T18:35:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:57 smithi119 ceph-mon[32752]: pgmap v834: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-04-26T18:35:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:57 smithi167 ceph-mon[32751]: pgmap v834: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-04-26T18:35:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:57 smithi090 ceph-mon[29180]: pgmap v834: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.0 KiB/s wr, 2 op/s 2024-04-26T18:35:57.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.78 MB/sec execute 442 sec latency 40223.021 ms 2024-04-26T18:35:58.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.77 MB/sec execute 443 sec latency 41223.105 ms 2024-04-26T18:35:59.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:35:59 smithi090 ceph-mon[29180]: pgmap v835: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-26T18:35:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:35:59 smithi119 ceph-mon[32752]: pgmap v835: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-26T18:35:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:35:59 smithi167 ceph-mon[32751]: pgmap v835: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-26T18:35:59.917 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.75 MB/sec execute 444 sec latency 42223.207 ms 2024-04-26T18:36:00.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.74 MB/sec execute 445 sec latency 43223.304 ms 2024-04-26T18:36:01.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:36:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:36:00.751+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:36:01.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:36:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:36:00.751+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:36:01.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:36:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:36:00.751+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:36:01.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:36:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:36:00.751+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:36:01.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:36:00 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:36:00.751+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:36:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:01 smithi119 ceph-mon[32752]: pgmap v836: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-26T18:36:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:01 smithi167 ceph-mon[32751]: pgmap v836: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-26T18:36:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:01 smithi090 ceph-mon[29180]: pgmap v836: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-04-26T18:36:01.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.73 MB/sec execute 446 sec latency 44223.440 ms 2024-04-26T18:36:02.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.71 MB/sec execute 447 sec latency 45223.549 ms 2024-04-26T18:36:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:03 smithi119 ceph-mon[32752]: pgmap v837: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T18:36:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:03 smithi167 ceph-mon[32751]: pgmap v837: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T18:36:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:03 smithi090 ceph-mon[29180]: pgmap v837: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T18:36:03.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.70 MB/sec execute 448 sec latency 46223.664 ms 2024-04-26T18:36:04.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.69 MB/sec execute 449 sec latency 47223.791 ms 2024-04-26T18:36:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:05 smithi119 ceph-mon[32752]: pgmap v838: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-04-26T18:36:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:05 smithi167 ceph-mon[32751]: pgmap v838: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-04-26T18:36:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:05 smithi090 ceph-mon[29180]: pgmap v838: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-04-26T18:36:05.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.68 MB/sec execute 450 sec latency 48223.868 ms 2024-04-26T18:36:06.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.66 MB/sec execute 451 sec latency 49223.989 ms 2024-04-26T18:36:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:07 smithi119 ceph-mon[32752]: pgmap v839: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T18:36:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:07 smithi167 ceph-mon[32751]: pgmap v839: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T18:36:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:07 smithi090 ceph-mon[29180]: pgmap v839: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T18:36:07.918 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.65 MB/sec execute 452 sec latency 50224.071 ms 2024-04-26T18:36:08.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.64 MB/sec execute 453 sec latency 51224.172 ms 2024-04-26T18:36:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:09 smithi119 ceph-mon[32752]: pgmap v840: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-04-26T18:36:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:09 smithi167 ceph-mon[32751]: pgmap v840: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-04-26T18:36:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:09 smithi090 ceph-mon[29180]: pgmap v840: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-04-26T18:36:09.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.63 MB/sec execute 454 sec latency 52224.296 ms 2024-04-26T18:36:10.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.61 MB/sec execute 455 sec latency 53224.360 ms 2024-04-26T18:36:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:11 smithi119 ceph-mon[32752]: pgmap v841: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T18:36:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:11 smithi167 ceph-mon[32751]: pgmap v841: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T18:36:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:11 smithi090 ceph-mon[29180]: pgmap v841: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T18:36:11.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.60 MB/sec execute 456 sec latency 54224.476 ms 2024-04-26T18:36:12.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.59 MB/sec execute 457 sec latency 55224.571 ms 2024-04-26T18:36:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:13 smithi119 ceph-mon[32752]: pgmap v842: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T18:36:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:13 smithi167 ceph-mon[32751]: pgmap v842: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T18:36:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:13 smithi090 ceph-mon[29180]: pgmap v842: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-04-26T18:36:13.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.58 MB/sec execute 458 sec latency 56224.650 ms 2024-04-26T18:36:14.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.56 MB/sec execute 459 sec latency 57224.718 ms 2024-04-26T18:36:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:15 smithi119 ceph-mon[32752]: pgmap v843: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-26T18:36:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:15 smithi167 ceph-mon[32751]: pgmap v843: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-26T18:36:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:15 smithi090 ceph-mon[29180]: pgmap v843: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.7 KiB/s wr, 4 op/s 2024-04-26T18:36:15.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.55 MB/sec execute 460 sec latency 58224.824 ms 2024-04-26T18:36:16.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.54 MB/sec execute 461 sec latency 59224.941 ms 2024-04-26T18:36:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:17 smithi119 ceph-mon[32752]: pgmap v844: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-04-26T18:36:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:17 smithi167 ceph-mon[32751]: pgmap v844: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-04-26T18:36:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:17 smithi090 ceph-mon[29180]: pgmap v844: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-04-26T18:36:17.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.53 MB/sec execute 462 sec latency 60225.037 ms 2024-04-26T18:36:18.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.52 MB/sec execute 463 sec latency 61225.133 ms 2024-04-26T18:36:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:19 smithi119 ceph-mon[32752]: pgmap v845: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-26T18:36:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:19 smithi167 ceph-mon[32751]: pgmap v845: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-26T18:36:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:19 smithi090 ceph-mon[29180]: pgmap v845: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 4.0 KiB/s wr, 3 op/s 2024-04-26T18:36:19.919 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.50 MB/sec execute 464 sec latency 62225.272 ms 2024-04-26T18:36:20.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.49 MB/sec execute 465 sec latency 63225.343 ms 2024-04-26T18:36:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:21 smithi119 ceph-mon[32752]: pgmap v846: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-04-26T18:36:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:21 smithi167 ceph-mon[32751]: pgmap v846: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-04-26T18:36:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:21 smithi090 ceph-mon[29180]: pgmap v846: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-04-26T18:36:21.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.48 MB/sec execute 466 sec latency 64225.452 ms 2024-04-26T18:36:22.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.47 MB/sec execute 467 sec latency 65225.499 ms 2024-04-26T18:36:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:23 smithi119 ceph-mon[32752]: pgmap v847: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-04-26T18:36:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:23 smithi167 ceph-mon[32751]: pgmap v847: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-04-26T18:36:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:23 smithi090 ceph-mon[29180]: pgmap v847: 97 pgs: 97 active+clean; 846 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 2.6 KiB/s wr, 2 op/s 2024-04-26T18:36:23.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.46 MB/sec execute 468 sec latency 66225.570 ms 2024-04-26T18:36:24.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.45 MB/sec execute 469 sec latency 67225.704 ms 2024-04-26T18:36:25.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:25 smithi090 ceph-mon[29180]: pgmap v848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 15 KiB/s wr, 3 op/s 2024-04-26T18:36:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:25 smithi119 ceph-mon[32752]: pgmap v848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 15 KiB/s wr, 3 op/s 2024-04-26T18:36:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:25 smithi167 ceph-mon[32751]: pgmap v848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 15 KiB/s wr, 3 op/s 2024-04-26T18:36:25.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.43 MB/sec execute 470 sec latency 68225.811 ms 2024-04-26T18:36:26.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.42 MB/sec execute 471 sec latency 69225.905 ms 2024-04-26T18:36:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:27 smithi119 ceph-mon[32752]: pgmap v849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-04-26T18:36:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:27 smithi167 ceph-mon[32751]: pgmap v849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-04-26T18:36:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:27 smithi090 ceph-mon[29180]: pgmap v849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-04-26T18:36:27.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.41 MB/sec execute 472 sec latency 70226.006 ms 2024-04-26T18:36:28.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.40 MB/sec execute 473 sec latency 71226.083 ms 2024-04-26T18:36:29.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:29 smithi090 ceph-mon[29180]: pgmap v850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 50 KiB/s wr, 3 op/s 2024-04-26T18:36:29.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:36:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:29 smithi119 ceph-mon[32752]: pgmap v850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 50 KiB/s wr, 3 op/s 2024-04-26T18:36:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:36:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:29 smithi167 ceph-mon[32751]: pgmap v850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 50 KiB/s wr, 3 op/s 2024-04-26T18:36:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:36:29.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.39 MB/sec execute 474 sec latency 72226.131 ms 2024-04-26T18:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:30 smithi119 ceph-mon[32752]: pgmap v851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 2 op/s 2024-04-26T18:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:30 smithi119 ceph-mon[32752]: pgmap v852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 3 op/s 2024-04-26T18:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:36:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:36:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:36:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:30 smithi167 ceph-mon[32751]: pgmap v851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 2 op/s 2024-04-26T18:36:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:30 smithi167 ceph-mon[32751]: pgmap v852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 3 op/s 2024-04-26T18:36:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:36:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:36:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:36:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:30 smithi090 ceph-mon[29180]: pgmap v851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 55 KiB/s wr, 2 op/s 2024-04-26T18:36:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:30 smithi090 ceph-mon[29180]: pgmap v852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 3 op/s 2024-04-26T18:36:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:36:30.920 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.38 MB/sec execute 475 sec latency 73226.205 ms 2024-04-26T18:36:31.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.37 MB/sec execute 476 sec latency 74226.299 ms 2024-04-26T18:36:32.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.35 MB/sec execute 477 sec latency 75226.417 ms 2024-04-26T18:36:33.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:33 smithi119 ceph-mon[32752]: pgmap v853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 3 op/s 2024-04-26T18:36:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:33 smithi167 ceph-mon[32751]: pgmap v853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 3 op/s 2024-04-26T18:36:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:33 smithi090 ceph-mon[29180]: pgmap v853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 67 KiB/s wr, 3 op/s 2024-04-26T18:36:33.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.34 MB/sec execute 478 sec latency 76226.531 ms 2024-04-26T18:36:34.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.33 MB/sec execute 479 sec latency 77226.649 ms 2024-04-26T18:36:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:35 smithi119 ceph-mon[32752]: pgmap v854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 4 op/s 2024-04-26T18:36:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:35 smithi167 ceph-mon[32751]: pgmap v854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 4 op/s 2024-04-26T18:36:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:35 smithi090 ceph-mon[29180]: pgmap v854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 4 op/s 2024-04-26T18:36:35.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.32 MB/sec execute 480 sec latency 78226.754 ms 2024-04-26T18:36:36.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.31 MB/sec execute 481 sec latency 79226.856 ms 2024-04-26T18:36:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:37 smithi119 ceph-mon[32752]: pgmap v855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 4 op/s 2024-04-26T18:36:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:37 smithi167 ceph-mon[32751]: pgmap v855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 4 op/s 2024-04-26T18:36:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:37 smithi090 ceph-mon[29180]: pgmap v855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 58 KiB/s wr, 4 op/s 2024-04-26T18:36:37.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.30 MB/sec execute 482 sec latency 80226.996 ms 2024-04-26T18:36:38.921 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.29 MB/sec execute 483 sec latency 81227.076 ms 2024-04-26T18:36:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:38 smithi119 ceph-mon[32752]: pgmap v856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.8 KiB/s wr, 2 op/s 2024-04-26T18:36:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:38 smithi167 ceph-mon[32751]: pgmap v856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.8 KiB/s wr, 2 op/s 2024-04-26T18:36:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:38 smithi090 ceph-mon[29180]: pgmap v856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 6.8 KiB/s wr, 2 op/s 2024-04-26T18:36:39.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.28 MB/sec execute 484 sec latency 82227.195 ms 2024-04-26T18:36:40.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.27 MB/sec execute 485 sec latency 83227.299 ms 2024-04-26T18:36:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:41 smithi119 ceph-mon[32752]: pgmap v857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8.0 KiB/s wr, 3 op/s 2024-04-26T18:36:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:41 smithi167 ceph-mon[32751]: pgmap v857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8.0 KiB/s wr, 3 op/s 2024-04-26T18:36:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:41 smithi090 ceph-mon[29180]: pgmap v857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8.0 KiB/s wr, 3 op/s 2024-04-26T18:36:41.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.26 MB/sec execute 486 sec latency 84227.392 ms 2024-04-26T18:36:42.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.24 MB/sec execute 487 sec latency 85227.469 ms 2024-04-26T18:36:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:43 smithi119 ceph-mon[32752]: pgmap v858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-04-26T18:36:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:43 smithi167 ceph-mon[32751]: pgmap v858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-04-26T18:36:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:43 smithi090 ceph-mon[29180]: pgmap v858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.7 KiB/s wr, 2 op/s 2024-04-26T18:36:43.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.23 MB/sec execute 488 sec latency 86227.560 ms 2024-04-26T18:36:44.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.22 MB/sec execute 489 sec latency 87227.669 ms 2024-04-26T18:36:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:45 smithi119 ceph-mon[32752]: pgmap v859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.9 KiB/s wr, 4 op/s 2024-04-26T18:36:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:45 smithi167 ceph-mon[32751]: pgmap v859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.9 KiB/s wr, 4 op/s 2024-04-26T18:36:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:45 smithi090 ceph-mon[29180]: pgmap v859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 7.9 KiB/s wr, 4 op/s 2024-04-26T18:36:45.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.21 MB/sec execute 490 sec latency 88227.772 ms 2024-04-26T18:36:46.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.20 MB/sec execute 491 sec latency 89227.808 ms 2024-04-26T18:36:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:47 smithi119 ceph-mon[32752]: pgmap v860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.9 KiB/s wr, 2 op/s 2024-04-26T18:36:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:47 smithi167 ceph-mon[32751]: pgmap v860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.9 KiB/s wr, 2 op/s 2024-04-26T18:36:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:47 smithi090 ceph-mon[29180]: pgmap v860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.9 KiB/s wr, 2 op/s 2024-04-26T18:36:47.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.19 MB/sec execute 492 sec latency 90227.912 ms 2024-04-26T18:36:48.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.18 MB/sec execute 493 sec latency 91228.014 ms 2024-04-26T18:36:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:49 smithi119 ceph-mon[32752]: pgmap v861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.9 KiB/s wr, 2 op/s 2024-04-26T18:36:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:49 smithi167 ceph-mon[32751]: pgmap v861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.9 KiB/s wr, 2 op/s 2024-04-26T18:36:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:49 smithi090 ceph-mon[29180]: pgmap v861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.9 KiB/s wr, 2 op/s 2024-04-26T18:36:49.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.17 MB/sec execute 494 sec latency 92228.110 ms 2024-04-26T18:36:50.922 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.16 MB/sec execute 495 sec latency 93228.187 ms 2024-04-26T18:36:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:51 smithi119 ceph-mon[32752]: pgmap v862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-26T18:36:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:51 smithi167 ceph-mon[32751]: pgmap v862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-26T18:36:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:51 smithi090 ceph-mon[29180]: pgmap v862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 3 op/s 2024-04-26T18:36:51.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.15 MB/sec execute 496 sec latency 94228.258 ms 2024-04-26T18:36:52.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.14 MB/sec execute 497 sec latency 95228.349 ms 2024-04-26T18:36:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:53 smithi119 ceph-mon[32752]: pgmap v863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-26T18:36:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:53 smithi167 ceph-mon[32751]: pgmap v863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-26T18:36:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:53 smithi090 ceph-mon[29180]: pgmap v863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-26T18:36:53.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.13 MB/sec execute 498 sec latency 96228.457 ms 2024-04-26T18:36:54.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.12 MB/sec execute 499 sec latency 97228.549 ms 2024-04-26T18:36:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:55 smithi119 ceph-mon[32752]: pgmap v864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-26T18:36:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:55 smithi167 ceph-mon[32751]: pgmap v864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-26T18:36:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:55 smithi090 ceph-mon[29180]: pgmap v864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-04-26T18:36:55.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.11 MB/sec execute 500 sec latency 98228.679 ms 2024-04-26T18:36:56.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.10 MB/sec execute 501 sec latency 99228.802 ms 2024-04-26T18:36:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:57 smithi119 ceph-mon[32752]: pgmap v865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:36:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:57 smithi167 ceph-mon[32751]: pgmap v865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:36:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:57 smithi090 ceph-mon[29180]: pgmap v865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:36:57.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.09 MB/sec execute 502 sec latency 100228.883 ms 2024-04-26T18:36:58.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.08 MB/sec execute 503 sec latency 101229.002 ms 2024-04-26T18:36:59.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:36:59 smithi090 ceph-mon[29180]: pgmap v866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:36:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:36:59 smithi119 ceph-mon[32752]: pgmap v866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:36:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:36:59 smithi167 ceph-mon[32751]: pgmap v866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:36:59.923 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.07 MB/sec execute 504 sec latency 102229.095 ms 2024-04-26T18:37:00.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.06 MB/sec execute 505 sec latency 103229.177 ms 2024-04-26T18:37:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:01 smithi119 ceph-mon[32752]: pgmap v867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:37:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:01 smithi167 ceph-mon[32751]: pgmap v867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:37:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:01 smithi090 ceph-mon[29180]: pgmap v867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:37:01.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.05 MB/sec execute 506 sec latency 104229.272 ms 2024-04-26T18:37:02.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.04 MB/sec execute 507 sec latency 105229.355 ms 2024-04-26T18:37:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:03 smithi119 ceph-mon[32752]: pgmap v868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:03 smithi167 ceph-mon[32751]: pgmap v868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:03 smithi090 ceph-mon[29180]: pgmap v868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:03.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.03 MB/sec execute 508 sec latency 106229.439 ms 2024-04-26T18:37:04.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.02 MB/sec execute 509 sec latency 107229.519 ms 2024-04-26T18:37:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:05 smithi119 ceph-mon[32752]: pgmap v869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:05 smithi167 ceph-mon[32751]: pgmap v869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:05 smithi090 ceph-mon[29180]: pgmap v869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:05.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.01 MB/sec execute 510 sec latency 108229.599 ms 2024-04-26T18:37:06.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 5.00 MB/sec execute 511 sec latency 109229.716 ms 2024-04-26T18:37:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:07 smithi119 ceph-mon[32752]: pgmap v870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:07 smithi167 ceph-mon[32751]: pgmap v870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:07 smithi090 ceph-mon[29180]: pgmap v870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:07.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.99 MB/sec execute 512 sec latency 110229.809 ms 2024-04-26T18:37:08.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.98 MB/sec execute 513 sec latency 111229.887 ms 2024-04-26T18:37:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:09 smithi119 ceph-mon[32752]: pgmap v871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:09 smithi167 ceph-mon[32751]: pgmap v871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:09 smithi090 ceph-mon[29180]: pgmap v871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:09.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.97 MB/sec execute 514 sec latency 112229.994 ms 2024-04-26T18:37:10.924 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.96 MB/sec execute 515 sec latency 113230.086 ms 2024-04-26T18:37:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:11 smithi119 ceph-mon[32752]: pgmap v872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:11 smithi167 ceph-mon[32751]: pgmap v872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:11 smithi090 ceph-mon[29180]: pgmap v872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:11.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.95 MB/sec execute 516 sec latency 114230.223 ms 2024-04-26T18:37:12.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.94 MB/sec execute 517 sec latency 115230.351 ms 2024-04-26T18:37:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:13 smithi119 ceph-mon[32752]: pgmap v873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:13 smithi167 ceph-mon[32751]: pgmap v873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:13 smithi090 ceph-mon[29180]: pgmap v873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:13.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.93 MB/sec execute 518 sec latency 116230.442 ms 2024-04-26T18:37:14.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.92 MB/sec execute 519 sec latency 117230.540 ms 2024-04-26T18:37:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:15 smithi119 ceph-mon[32752]: pgmap v874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:15 smithi167 ceph-mon[32751]: pgmap v874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:15 smithi090 ceph-mon[29180]: pgmap v874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:15.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.91 MB/sec execute 520 sec latency 118230.642 ms 2024-04-26T18:37:16.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.90 MB/sec execute 521 sec latency 119230.753 ms 2024-04-26T18:37:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:17 smithi119 ceph-mon[32752]: pgmap v875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:17 smithi167 ceph-mon[32751]: pgmap v875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:17 smithi090 ceph-mon[29180]: pgmap v875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:17.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.89 MB/sec execute 522 sec latency 120230.833 ms 2024-04-26T18:37:18.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.88 MB/sec execute 523 sec latency 121230.950 ms 2024-04-26T18:37:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:19 smithi119 ceph-mon[32752]: pgmap v876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:19 smithi167 ceph-mon[32751]: pgmap v876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:19 smithi090 ceph-mon[29180]: pgmap v876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:19.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.87 MB/sec execute 524 sec latency 122231.038 ms 2024-04-26T18:37:20.925 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.86 MB/sec execute 525 sec latency 123231.072 ms 2024-04-26T18:37:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:21 smithi119 ceph-mon[32752]: pgmap v877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:21 smithi167 ceph-mon[32751]: pgmap v877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:21 smithi090 ceph-mon[29180]: pgmap v877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:21.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.86 MB/sec execute 526 sec latency 124231.182 ms 2024-04-26T18:37:22.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.85 MB/sec execute 527 sec latency 125231.291 ms 2024-04-26T18:37:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:23 smithi119 ceph-mon[32752]: pgmap v878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:23 smithi167 ceph-mon[32751]: pgmap v878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:23 smithi090 ceph-mon[29180]: pgmap v878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:23.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.84 MB/sec execute 528 sec latency 126231.424 ms 2024-04-26T18:37:24.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.83 MB/sec execute 529 sec latency 127231.535 ms 2024-04-26T18:37:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:25 smithi119 ceph-mon[32752]: pgmap v879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:25 smithi167 ceph-mon[32751]: pgmap v879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:25 smithi090 ceph-mon[29180]: pgmap v879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:25.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.82 MB/sec execute 530 sec latency 128231.655 ms 2024-04-26T18:37:26.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.81 MB/sec execute 531 sec latency 129231.702 ms 2024-04-26T18:37:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:27 smithi119 ceph-mon[32752]: pgmap v880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:27 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:27 smithi167 ceph-mon[32751]: pgmap v880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:27 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:27 smithi090 ceph-mon[29180]: pgmap v880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:27 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:27.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.80 MB/sec execute 532 sec latency 130231.789 ms 2024-04-26T18:37:28.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.79 MB/sec execute 533 sec latency 131231.889 ms 2024-04-26T18:37:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:29 smithi119 ceph-mon[32752]: pgmap v881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:29 smithi167 ceph-mon[32751]: pgmap v881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:29 smithi090 ceph-mon[29180]: pgmap v881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:29.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.78 MB/sec execute 534 sec latency 132231.974 ms 2024-04-26T18:37:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:37:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:37:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:37:30.926 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.77 MB/sec execute 535 sec latency 133232.040 ms 2024-04-26T18:37:31.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:31 smithi090 ceph-mon[29180]: pgmap v882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:31.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:31 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:31.478 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:31 smithi119 ceph-mon[32752]: pgmap v882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:31.478 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:31 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:31 smithi167 ceph-mon[32751]: pgmap v882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:31 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:31.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.77 MB/sec execute 536 sec latency 134232.109 ms 2024-04-26T18:37:32.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.76 MB/sec execute 537 sec latency 135232.220 ms 2024-04-26T18:37:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:33 smithi090 ceph-mon[29180]: pgmap v883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:33 smithi119 ceph-mon[32752]: pgmap v883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:33 smithi167 ceph-mon[32751]: pgmap v883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:33.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.75 MB/sec execute 538 sec latency 136232.285 ms 2024-04-26T18:37:34.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.74 MB/sec execute 539 sec latency 137232.358 ms 2024-04-26T18:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: pgmap v884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:37:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: pgmap v884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: pgmap v884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:37:35.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.73 MB/sec execute 540 sec latency 138232.449 ms 2024-04-26T18:37:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:36 smithi119 ceph-mon[32752]: pgmap v885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T18:37:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:36 smithi119 ceph-mon[32752]: pgmap v886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:36 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:36 smithi167 ceph-mon[32751]: pgmap v885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T18:37:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:36 smithi167 ceph-mon[32751]: pgmap v886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:36 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:36 smithi090 ceph-mon[29180]: pgmap v885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T18:37:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:36 smithi090 ceph-mon[29180]: pgmap v886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:36 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:37:36.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.72 MB/sec execute 541 sec latency 139232.532 ms 2024-04-26T18:37:37.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.71 MB/sec execute 542 sec latency 140232.607 ms 2024-04-26T18:37:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:38 smithi119 ceph-mon[32752]: pgmap v887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:38 smithi167 ceph-mon[32751]: pgmap v887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:38 smithi090 ceph-mon[29180]: pgmap v887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:38.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.70 MB/sec execute 543 sec latency 141232.677 ms 2024-04-26T18:37:39.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.70 MB/sec execute 544 sec latency 142232.769 ms 2024-04-26T18:37:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:40 smithi119 ceph-mon[32752]: pgmap v888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:40 smithi167 ceph-mon[32751]: pgmap v888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:40 smithi090 ceph-mon[29180]: pgmap v888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:40.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.69 MB/sec execute 545 sec latency 143232.814 ms 2024-04-26T18:37:41.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.68 MB/sec execute 546 sec latency 144232.918 ms 2024-04-26T18:37:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:42 smithi119 ceph-mon[32752]: pgmap v889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:42 smithi167 ceph-mon[32751]: pgmap v889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:42 smithi090 ceph-mon[29180]: pgmap v889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:42.927 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.67 MB/sec execute 547 sec latency 145233.012 ms 2024-04-26T18:37:43.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.66 MB/sec execute 548 sec latency 146233.109 ms 2024-04-26T18:37:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:44 smithi119 ceph-mon[32752]: pgmap v890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T18:37:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:44 smithi167 ceph-mon[32751]: pgmap v890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T18:37:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:44 smithi090 ceph-mon[29180]: pgmap v890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T18:37:44.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.65 MB/sec execute 549 sec latency 147233.227 ms 2024-04-26T18:37:45.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.64 MB/sec execute 550 sec latency 148233.313 ms 2024-04-26T18:37:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:46 smithi119 ceph-mon[32752]: pgmap v891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:37:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:46 smithi167 ceph-mon[32751]: pgmap v891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:37:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:46 smithi090 ceph-mon[29180]: pgmap v891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:37:46.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.64 MB/sec execute 551 sec latency 149233.418 ms 2024-04-26T18:37:47.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.63 MB/sec execute 552 sec latency 150233.504 ms 2024-04-26T18:37:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:48 smithi119 ceph-mon[32752]: pgmap v892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:48 smithi167 ceph-mon[32751]: pgmap v892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:48 smithi090 ceph-mon[29180]: pgmap v892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:48.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.62 MB/sec execute 553 sec latency 151233.575 ms 2024-04-26T18:37:49.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.61 MB/sec execute 554 sec latency 152233.613 ms 2024-04-26T18:37:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:50 smithi119 ceph-mon[32752]: pgmap v893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:50 smithi167 ceph-mon[32751]: pgmap v893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:50 smithi090 ceph-mon[29180]: pgmap v893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:50.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.60 MB/sec execute 555 sec latency 153233.693 ms 2024-04-26T18:37:51.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.59 MB/sec execute 556 sec latency 154233.797 ms 2024-04-26T18:37:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:52 smithi119 ceph-mon[32752]: pgmap v894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:52 smithi167 ceph-mon[32751]: pgmap v894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:52 smithi090 ceph-mon[29180]: pgmap v894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:52.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.59 MB/sec execute 557 sec latency 155233.903 ms 2024-04-26T18:37:53.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.58 MB/sec execute 558 sec latency 156234.002 ms 2024-04-26T18:37:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:54 smithi119 ceph-mon[32752]: pgmap v895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:54 smithi167 ceph-mon[32751]: pgmap v895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:54 smithi090 ceph-mon[29180]: pgmap v895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:54.928 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.57 MB/sec execute 559 sec latency 157234.083 ms 2024-04-26T18:37:55.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.56 MB/sec execute 560 sec latency 158234.163 ms 2024-04-26T18:37:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:56 smithi119 ceph-mon[32752]: pgmap v896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:56 smithi167 ceph-mon[32751]: pgmap v896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:56 smithi090 ceph-mon[29180]: pgmap v896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:37:56.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.55 MB/sec execute 561 sec latency 159234.259 ms 2024-04-26T18:37:57.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.54 MB/sec execute 562 sec latency 160234.391 ms 2024-04-26T18:37:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:37:58 smithi119 ceph-mon[32752]: pgmap v897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:37:58 smithi167 ceph-mon[32751]: pgmap v897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:37:58 smithi090 ceph-mon[29180]: pgmap v897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:37:58.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.54 MB/sec execute 563 sec latency 161234.474 ms 2024-04-26T18:37:59.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.53 MB/sec execute 564 sec latency 162234.520 ms 2024-04-26T18:38:00.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:00 smithi090 ceph-mon[29180]: pgmap v898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:00 smithi119 ceph-mon[32752]: pgmap v898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:00 smithi167 ceph-mon[32751]: pgmap v898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:00.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.52 MB/sec execute 565 sec latency 163234.566 ms 2024-04-26T18:38:01.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.51 MB/sec execute 566 sec latency 164234.622 ms 2024-04-26T18:38:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:02 smithi119 ceph-mon[32752]: pgmap v899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:02 smithi167 ceph-mon[32751]: pgmap v899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:02 smithi090 ceph-mon[29180]: pgmap v899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:02.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.50 MB/sec execute 567 sec latency 165234.695 ms 2024-04-26T18:38:03.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.50 MB/sec execute 568 sec latency 166234.739 ms 2024-04-26T18:38:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:04 smithi119 ceph-mon[32752]: pgmap v900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:04 smithi167 ceph-mon[32751]: pgmap v900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:04 smithi090 ceph-mon[29180]: pgmap v900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:04.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.49 MB/sec execute 569 sec latency 167234.862 ms 2024-04-26T18:38:05.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.48 MB/sec execute 570 sec latency 168234.988 ms 2024-04-26T18:38:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:06 smithi119 ceph-mon[32752]: pgmap v901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:06 smithi167 ceph-mon[32751]: pgmap v901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:06 smithi090 ceph-mon[29180]: pgmap v901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:06.929 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.47 MB/sec execute 571 sec latency 169235.117 ms 2024-04-26T18:38:07.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.47 MB/sec execute 572 sec latency 170235.197 ms 2024-04-26T18:38:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:08 smithi119 ceph-mon[32752]: pgmap v902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:08 smithi167 ceph-mon[32751]: pgmap v902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:08 smithi090 ceph-mon[29180]: pgmap v902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:08.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.46 MB/sec execute 573 sec latency 171235.289 ms 2024-04-26T18:38:09.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.45 MB/sec execute 574 sec latency 172235.398 ms 2024-04-26T18:38:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:10 smithi119 ceph-mon[32752]: pgmap v903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:10 smithi167 ceph-mon[32751]: pgmap v903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:10 smithi090 ceph-mon[29180]: pgmap v903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:10.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.44 MB/sec execute 575 sec latency 173235.496 ms 2024-04-26T18:38:11.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.43 MB/sec execute 576 sec latency 174235.619 ms 2024-04-26T18:38:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:12 smithi119 ceph-mon[32752]: pgmap v904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:12 smithi167 ceph-mon[32751]: pgmap v904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:12 smithi090 ceph-mon[29180]: pgmap v904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:12.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.43 MB/sec execute 577 sec latency 175235.743 ms 2024-04-26T18:38:13.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.42 MB/sec execute 578 sec latency 176235.835 ms 2024-04-26T18:38:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:14 smithi119 ceph-mon[32752]: pgmap v905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:14 smithi167 ceph-mon[32751]: pgmap v905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:14 smithi090 ceph-mon[29180]: pgmap v905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:14.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.41 MB/sec execute 579 sec latency 177235.960 ms 2024-04-26T18:38:15.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.40 MB/sec execute 580 sec latency 178236.059 ms 2024-04-26T18:38:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:16 smithi119 ceph-mon[32752]: pgmap v906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:16 smithi167 ceph-mon[32751]: pgmap v906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:16 smithi090 ceph-mon[29180]: pgmap v906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:16.930 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.40 MB/sec execute 581 sec latency 179236.123 ms 2024-04-26T18:38:17.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.39 MB/sec execute 582 sec latency 180236.202 ms 2024-04-26T18:38:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:18 smithi119 ceph-mon[32752]: pgmap v907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:18 smithi167 ceph-mon[32751]: pgmap v907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:18 smithi090 ceph-mon[29180]: pgmap v907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:18.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.38 MB/sec execute 583 sec latency 181236.288 ms 2024-04-26T18:38:19.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.37 MB/sec execute 584 sec latency 182236.376 ms 2024-04-26T18:38:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:20 smithi119 ceph-mon[32752]: pgmap v908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:20 smithi167 ceph-mon[32751]: pgmap v908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:20 smithi090 ceph-mon[29180]: pgmap v908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:20.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.37 MB/sec execute 585 sec latency 183236.426 ms 2024-04-26T18:38:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:21 smithi119 ceph-mon[32752]: 5 slow requests, 5 included below; oldest blocked for > 182.637654 secs 2024-04-26T18:38:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:21 smithi119 ceph-mon[32752]: slow request 182.637653 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:21 smithi119 ceph-mon[32752]: slow request 182.636092 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:21 smithi119 ceph-mon[32752]: slow request 182.634623 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:21 smithi119 ceph-mon[32752]: slow request 182.634153 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:21 smithi119 ceph-mon[32752]: slow request 182.621888 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:21 smithi167 ceph-mon[32751]: 5 slow requests, 5 included below; oldest blocked for > 182.637654 secs 2024-04-26T18:38:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:21 smithi167 ceph-mon[32751]: slow request 182.637653 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:21 smithi167 ceph-mon[32751]: slow request 182.636092 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:21 smithi167 ceph-mon[32751]: slow request 182.634623 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:21 smithi167 ceph-mon[32751]: slow request 182.634153 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:21 smithi167 ceph-mon[32751]: slow request 182.621888 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:21 smithi090 ceph-mon[29180]: 5 slow requests, 5 included below; oldest blocked for > 182.637654 secs 2024-04-26T18:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:21 smithi090 ceph-mon[29180]: slow request 182.637653 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:21 smithi090 ceph-mon[29180]: slow request 182.636092 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:21 smithi090 ceph-mon[29180]: slow request 182.634623 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:21 smithi090 ceph-mon[29180]: slow request 182.634153 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:21 smithi090 ceph-mon[29180]: slow request 182.621888 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:38:21.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.36 MB/sec execute 586 sec latency 184236.505 ms 2024-04-26T18:38:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:22 smithi119 ceph-mon[32752]: pgmap v909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:22 smithi167 ceph-mon[32751]: pgmap v909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:22 smithi090 ceph-mon[29180]: pgmap v909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:22.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.35 MB/sec execute 587 sec latency 185236.620 ms 2024-04-26T18:38:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:23 smithi119 ceph-mon[32752]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-04-26T18:38:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:38:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:23 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:38:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:23 smithi167 ceph-mon[32751]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-04-26T18:38:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:38:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:23 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:38:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:23 smithi090 ceph-mon[29180]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-04-26T18:38:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:38:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:23 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:38:23.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.34 MB/sec execute 588 sec latency 186236.708 ms 2024-04-26T18:38:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:24 smithi119 ceph-mon[32752]: pgmap v910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:24 smithi167 ceph-mon[32751]: pgmap v910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:24 smithi090 ceph-mon[29180]: pgmap v910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:24.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.34 MB/sec execute 589 sec latency 187236.753 ms 2024-04-26T18:38:25.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.33 MB/sec execute 590 sec latency 188236.851 ms 2024-04-26T18:38:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:26 smithi119 ceph-mon[32752]: pgmap v911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:26 smithi119 ceph-mon[32752]: 9 slow requests, 4 included below; oldest blocked for > 187.637770 secs 2024-04-26T18:38:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:26 smithi119 ceph-mon[32752]: slow request 184.053133 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:38:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:26 smithi119 ceph-mon[32752]: slow request 184.052797 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:38:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:26 smithi119 ceph-mon[32752]: slow request 183.943962 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:38:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:26 smithi119 ceph-mon[32752]: slow request 183.939674 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:38:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:26 smithi167 ceph-mon[32751]: pgmap v911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:26 smithi167 ceph-mon[32751]: 9 slow requests, 4 included below; oldest blocked for > 187.637770 secs 2024-04-26T18:38:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:26 smithi167 ceph-mon[32751]: slow request 184.053133 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:38:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:26 smithi167 ceph-mon[32751]: slow request 184.052797 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:38:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:26 smithi167 ceph-mon[32751]: slow request 183.943962 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:38:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:26 smithi167 ceph-mon[32751]: slow request 183.939674 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:26 smithi090 ceph-mon[29180]: pgmap v911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:26 smithi090 ceph-mon[29180]: 9 slow requests, 4 included below; oldest blocked for > 187.637770 secs 2024-04-26T18:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:26 smithi090 ceph-mon[29180]: slow request 184.053133 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:26 smithi090 ceph-mon[29180]: slow request 184.052797 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:26 smithi090 ceph-mon[29180]: slow request 183.943962 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:26 smithi090 ceph-mon[29180]: slow request 183.939674 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:38:26.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.32 MB/sec execute 591 sec latency 189236.901 ms 2024-04-26T18:38:27.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.31 MB/sec execute 592 sec latency 190236.985 ms 2024-04-26T18:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:38:28.485+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80034f0 con 0x7f2cd4104500 2024-04-26T18:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:38:28.485+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 6 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80034f0 con 0x7f2cd4104500 2024-04-26T18:38:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:38:28.507+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac0034c0 con 0x56302893f0e0 2024-04-26T18:38:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:38:28.507+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 6 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac0034c0 con 0x56302893f0e0 2024-04-26T18:38:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:28 smithi167 ceph-mon[32751]: pgmap v912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:28 smithi090 ceph-mon[29180]: pgmap v912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:28 smithi119 ceph-mon[32752]: pgmap v912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:28.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.31 MB/sec execute 593 sec latency 191237.076 ms 2024-04-26T18:38:29.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.30 MB/sec execute 594 sec latency 192237.157 ms 2024-04-26T18:38:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:30 smithi167 ceph-mon[32751]: pgmap v913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:30 smithi090 ceph-mon[29180]: pgmap v913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:30 smithi119 ceph-mon[32752]: pgmap v913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:30.931 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.29 MB/sec execute 595 sec latency 193237.223 ms 2024-04-26T18:38:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:31 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:38:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:31 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:38:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 192.637887 secs 2024-04-26T18:38:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:31 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:38:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:31 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:38:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 192.637887 secs 2024-04-26T18:38:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:31 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:38:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:31 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:38:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 192.637887 secs 2024-04-26T18:38:31.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.29 MB/sec execute 596 sec latency 194237.347 ms 2024-04-26T18:38:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:32 smithi090 ceph-mon[29180]: pgmap v914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:32 smithi119 ceph-mon[32752]: pgmap v914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:32 smithi167 ceph-mon[32751]: pgmap v914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:32.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.28 MB/sec execute 597 sec latency 195237.413 ms 2024-04-26T18:38:33.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.27 MB/sec execute 598 sec latency 196237.494 ms 2024-04-26T18:38:34.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:34 smithi119 ceph-mon[32752]: pgmap v915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:34 smithi090 ceph-mon[29180]: pgmap v915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:34 smithi167 ceph-mon[32751]: pgmap v915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:34.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 58812 4.26 MB/sec execute 599 sec latency 197237.557 ms 2024-04-26T18:38:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:38:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:38:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:38:35.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 600 sec 2024-04-26T18:38:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:36 smithi090 ceph-mon[29180]: pgmap v916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:36 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 197.637954 secs 2024-04-26T18:38:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:36 smithi119 ceph-mon[32752]: pgmap v916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:36 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 197.637954 secs 2024-04-26T18:38:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:36 smithi167 ceph-mon[32751]: pgmap v916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:36 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 197.637954 secs 2024-04-26T18:38:36.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 601 sec 2024-04-26T18:38:37.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 602 sec 2024-04-26T18:38:38.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:38 smithi167 ceph-mon[32751]: pgmap v917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:38 smithi090 ceph-mon[29180]: pgmap v917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:38 smithi119 ceph-mon[32752]: pgmap v917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:38.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 603 sec 2024-04-26T18:38:39.932 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 604 sec 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: pgmap v918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: pgmap v919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: pgmap v920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: pgmap v918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: pgmap v919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-26T18:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: pgmap v920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: pgmap v918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: pgmap v919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-26T18:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: pgmap v920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:38:40.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 605 sec 2024-04-26T18:38:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:41 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 202.638091 secs 2024-04-26T18:38:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:41 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 202.638091 secs 2024-04-26T18:38:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:41 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 202.638091 secs 2024-04-26T18:38:41.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 606 sec 2024-04-26T18:38:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:42 smithi167 ceph-mon[32751]: pgmap v921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:42 smithi090 ceph-mon[29180]: pgmap v921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:42 smithi119 ceph-mon[32752]: pgmap v921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:42.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 607 sec 2024-04-26T18:38:43.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 608 sec 2024-04-26T18:38:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:44 smithi090 ceph-mon[29180]: pgmap v922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T18:38:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:44 smithi119 ceph-mon[32752]: pgmap v922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T18:38:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:44 smithi167 ceph-mon[32751]: pgmap v922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T18:38:44.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 609 sec 2024-04-26T18:38:45.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 610 sec 2024-04-26T18:38:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:46 smithi090 ceph-mon[29180]: pgmap v923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:46 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 207.638185 secs 2024-04-26T18:38:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:46 smithi119 ceph-mon[32752]: pgmap v923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:46 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 207.638185 secs 2024-04-26T18:38:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:46 smithi167 ceph-mon[32751]: pgmap v923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:46 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 207.638185 secs 2024-04-26T18:38:46.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 611 sec 2024-04-26T18:38:47.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 612 sec 2024-04-26T18:38:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:48 smithi090 ceph-mon[29180]: pgmap v924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T18:38:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:48 smithi119 ceph-mon[32752]: pgmap v924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T18:38:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:48 smithi167 ceph-mon[32751]: pgmap v924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T18:38:48.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 613 sec 2024-04-26T18:38:49.933 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 614 sec 2024-04-26T18:38:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:50 smithi090 ceph-mon[29180]: pgmap v925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:38:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:50 smithi119 ceph-mon[32752]: pgmap v925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:38:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:50 smithi167 ceph-mon[32751]: pgmap v925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:38:50.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 615 sec 2024-04-26T18:38:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:51 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 212.638326 secs 2024-04-26T18:38:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:51 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 212.638326 secs 2024-04-26T18:38:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:51 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 212.638326 secs 2024-04-26T18:38:51.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 616 sec 2024-04-26T18:38:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:52 smithi119 ceph-mon[32752]: pgmap v926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:52 smithi167 ceph-mon[32751]: pgmap v926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:52 smithi090 ceph-mon[29180]: pgmap v926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:52.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 617 sec 2024-04-26T18:38:53.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 618 sec 2024-04-26T18:38:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:54 smithi119 ceph-mon[32752]: pgmap v927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:54 smithi167 ceph-mon[32751]: pgmap v927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:54 smithi090 ceph-mon[29180]: pgmap v927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:54.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 619 sec 2024-04-26T18:38:55.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 620 sec 2024-04-26T18:38:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:56 smithi119 ceph-mon[32752]: pgmap v928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:56 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 217.638442 secs 2024-04-26T18:38:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:56 smithi167 ceph-mon[32751]: pgmap v928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:56 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 217.638442 secs 2024-04-26T18:38:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:56 smithi090 ceph-mon[29180]: pgmap v928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:38:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:56 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 217.638442 secs 2024-04-26T18:38:56.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 621 sec 2024-04-26T18:38:57.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 622 sec 2024-04-26T18:38:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:38:58 smithi119 ceph-mon[32752]: pgmap v929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:38:58 smithi167 ceph-mon[32751]: pgmap v929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:38:58 smithi090 ceph-mon[29180]: pgmap v929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:38:58.934 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 623 sec 2024-04-26T18:38:59.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 624 sec 2024-04-26T18:39:00.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:00 smithi090 ceph-mon[29180]: pgmap v930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:00 smithi119 ceph-mon[32752]: pgmap v930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:00 smithi167 ceph-mon[32751]: pgmap v930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:00.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 625 sec 2024-04-26T18:39:01.443 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:01 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 222.638522 secs 2024-04-26T18:39:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:01 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 222.638522 secs 2024-04-26T18:39:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:01 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 222.638522 secs 2024-04-26T18:39:01.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 626 sec 2024-04-26T18:39:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:02 smithi119 ceph-mon[32752]: pgmap v931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:02 smithi167 ceph-mon[32751]: pgmap v931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:02 smithi090 ceph-mon[29180]: pgmap v931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:02.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 627 sec 2024-04-26T18:39:03.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 628 sec 2024-04-26T18:39:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:04 smithi119 ceph-mon[32752]: pgmap v932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:04 smithi167 ceph-mon[32751]: pgmap v932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:04 smithi090 ceph-mon[29180]: pgmap v932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:04.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 629 sec 2024-04-26T18:39:05.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 630 sec 2024-04-26T18:39:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:06 smithi119 ceph-mon[32752]: pgmap v933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:06 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 227.638582 secs 2024-04-26T18:39:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:06 smithi167 ceph-mon[32751]: pgmap v933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:06 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 227.638582 secs 2024-04-26T18:39:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:06 smithi090 ceph-mon[29180]: pgmap v933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:06 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 227.638582 secs 2024-04-26T18:39:06.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 631 sec 2024-04-26T18:39:07.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 632 sec 2024-04-26T18:39:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:08 smithi119 ceph-mon[32752]: pgmap v934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:08 smithi167 ceph-mon[32751]: pgmap v934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:08 smithi090 ceph-mon[29180]: pgmap v934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:08.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 633 sec 2024-04-26T18:39:09.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 634 sec 2024-04-26T18:39:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:10 smithi119 ceph-mon[32752]: pgmap v935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:10 smithi167 ceph-mon[32751]: pgmap v935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:10 smithi090 ceph-mon[29180]: pgmap v935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:10.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 635 sec 2024-04-26T18:39:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:11 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 232.638699 secs 2024-04-26T18:39:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:11 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 232.638699 secs 2024-04-26T18:39:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:11 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 232.638699 secs 2024-04-26T18:39:11.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 636 sec 2024-04-26T18:39:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:12 smithi119 ceph-mon[32752]: pgmap v936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:12 smithi167 ceph-mon[32751]: pgmap v936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:12 smithi090 ceph-mon[29180]: pgmap v936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:12.935 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 637 sec 2024-04-26T18:39:13.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 638 sec 2024-04-26T18:39:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:14 smithi119 ceph-mon[32752]: pgmap v937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:14 smithi167 ceph-mon[32751]: pgmap v937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:14 smithi090 ceph-mon[29180]: pgmap v937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:14.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 639 sec 2024-04-26T18:39:15.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 640 sec 2024-04-26T18:39:16.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:16 smithi090 ceph-mon[29180]: pgmap v938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:16.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:16 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 237.638777 secs 2024-04-26T18:39:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:16 smithi119 ceph-mon[32752]: pgmap v938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:16 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 237.638777 secs 2024-04-26T18:39:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:16 smithi167 ceph-mon[32751]: pgmap v938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:16 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 237.638777 secs 2024-04-26T18:39:16.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 641 sec 2024-04-26T18:39:17.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 642 sec 2024-04-26T18:39:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:18 smithi119 ceph-mon[32752]: pgmap v939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:18 smithi167 ceph-mon[32751]: pgmap v939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:18 smithi090 ceph-mon[29180]: pgmap v939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:18.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 643 sec 2024-04-26T18:39:19.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 644 sec 2024-04-26T18:39:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:20 smithi119 ceph-mon[32752]: pgmap v940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:20 smithi167 ceph-mon[32751]: pgmap v940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:20 smithi090 ceph-mon[29180]: pgmap v940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:20.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 645 sec 2024-04-26T18:39:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:21 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 242.638900 secs 2024-04-26T18:39:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:21 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 242.638900 secs 2024-04-26T18:39:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:21 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 242.638900 secs 2024-04-26T18:39:21.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 646 sec 2024-04-26T18:39:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:22 smithi119 ceph-mon[32752]: pgmap v941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:22 smithi167 ceph-mon[32751]: pgmap v941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:22 smithi090 ceph-mon[29180]: pgmap v941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:22.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 647 sec 2024-04-26T18:39:23.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 648 sec 2024-04-26T18:39:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:24 smithi119 ceph-mon[32752]: pgmap v942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:24 smithi167 ceph-mon[32751]: pgmap v942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:24 smithi090 ceph-mon[29180]: pgmap v942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:24.936 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 649 sec 2024-04-26T18:39:25.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 650 sec 2024-04-26T18:39:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:26 smithi119 ceph-mon[32752]: pgmap v943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:26 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 247.639036 secs 2024-04-26T18:39:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:26 smithi167 ceph-mon[32751]: pgmap v943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:26 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 247.639036 secs 2024-04-26T18:39:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:26 smithi090 ceph-mon[29180]: pgmap v943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:26 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 247.639036 secs 2024-04-26T18:39:26.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 651 sec 2024-04-26T18:39:27.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 652 sec 2024-04-26T18:39:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:28 smithi119 ceph-mon[32752]: pgmap v944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:28 smithi167 ceph-mon[32751]: pgmap v944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:28 smithi090 ceph-mon[29180]: pgmap v944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:28.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 653 sec 2024-04-26T18:39:29.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 654 sec 2024-04-26T18:39:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:30 smithi119 ceph-mon[32752]: pgmap v945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:30 smithi167 ceph-mon[32751]: pgmap v945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:30 smithi090 ceph-mon[29180]: pgmap v945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:30.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 655 sec 2024-04-26T18:39:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 252.639156 secs 2024-04-26T18:39:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 252.639156 secs 2024-04-26T18:39:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 252.639156 secs 2024-04-26T18:39:31.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 656 sec 2024-04-26T18:39:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:32 smithi119 ceph-mon[32752]: pgmap v946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:32 smithi167 ceph-mon[32751]: pgmap v946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:32 smithi090 ceph-mon[29180]: pgmap v946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:32.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 657 sec 2024-04-26T18:39:33.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 658 sec 2024-04-26T18:39:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:34 smithi119 ceph-mon[32752]: pgmap v947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:34 smithi167 ceph-mon[32751]: pgmap v947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:34 smithi090 ceph-mon[29180]: pgmap v947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:34.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 659 sec 2024-04-26T18:39:35.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 660 sec 2024-04-26T18:39:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:36 smithi119 ceph-mon[32752]: pgmap v948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:36 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 257.639271 secs 2024-04-26T18:39:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:36 smithi167 ceph-mon[32751]: pgmap v948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:36 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 257.639271 secs 2024-04-26T18:39:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:36 smithi090 ceph-mon[29180]: pgmap v948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:36 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 257.639271 secs 2024-04-26T18:39:36.937 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 661 sec 2024-04-26T18:39:37.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 662 sec 2024-04-26T18:39:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:38 smithi119 ceph-mon[32752]: pgmap v949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:38 smithi167 ceph-mon[32751]: pgmap v949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:38 smithi090 ceph-mon[29180]: pgmap v949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:38.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 663 sec 2024-04-26T18:39:39.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 664 sec 2024-04-26T18:39:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:40 smithi119 ceph-mon[32752]: pgmap v950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:39:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:39:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:39:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:39:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:40 smithi167 ceph-mon[32751]: pgmap v950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:39:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:39:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:39:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:39:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:40 smithi090 ceph-mon[29180]: pgmap v950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:39:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:39:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:39:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:39:40.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 665 sec 2024-04-26T18:39:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:41 smithi119 ceph-mon[32752]: pgmap v951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T18:39:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:41 smithi119 ceph-mon[32752]: pgmap v952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:41 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 262.639403 secs 2024-04-26T18:39:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:41 smithi167 ceph-mon[32751]: pgmap v951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T18:39:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:41 smithi167 ceph-mon[32751]: pgmap v952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:41 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 262.639403 secs 2024-04-26T18:39:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:41 smithi090 ceph-mon[29180]: pgmap v951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T18:39:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:41 smithi090 ceph-mon[29180]: pgmap v952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:41 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 262.639403 secs 2024-04-26T18:39:41.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 666 sec 2024-04-26T18:39:42.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 667 sec 2024-04-26T18:39:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:43 smithi119 ceph-mon[32752]: pgmap v953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:43 smithi167 ceph-mon[32751]: pgmap v953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:43 smithi090 ceph-mon[29180]: pgmap v953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:43.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 668 sec 2024-04-26T18:39:44.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 669 sec 2024-04-26T18:39:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:45 smithi119 ceph-mon[32752]: pgmap v954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:45 smithi167 ceph-mon[32751]: pgmap v954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:45 smithi090 ceph-mon[29180]: pgmap v954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:45.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 670 sec 2024-04-26T18:39:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:46 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 267.639539 secs 2024-04-26T18:39:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:46 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 267.639539 secs 2024-04-26T18:39:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:46 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 267.639539 secs 2024-04-26T18:39:46.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 671 sec 2024-04-26T18:39:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:47 smithi119 ceph-mon[32752]: pgmap v955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:47 smithi167 ceph-mon[32751]: pgmap v955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:47 smithi090 ceph-mon[29180]: pgmap v955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:39:47.938 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 672 sec 2024-04-26T18:39:48.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 673 sec 2024-04-26T18:39:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:49 smithi119 ceph-mon[32752]: pgmap v956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T18:39:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:49 smithi167 ceph-mon[32751]: pgmap v956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T18:39:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:49 smithi090 ceph-mon[29180]: pgmap v956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T18:39:49.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 674 sec 2024-04-26T18:39:50.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 675 sec 2024-04-26T18:39:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:51 smithi119 ceph-mon[32752]: pgmap v957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:39:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:51 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 272.639688 secs 2024-04-26T18:39:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:51 smithi167 ceph-mon[32751]: pgmap v957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:39:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:51 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 272.639688 secs 2024-04-26T18:39:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:51 smithi090 ceph-mon[29180]: pgmap v957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:39:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:51 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 272.639688 secs 2024-04-26T18:39:51.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 676 sec 2024-04-26T18:39:52.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 677 sec 2024-04-26T18:39:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:53 smithi119 ceph-mon[32752]: pgmap v958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:53 smithi167 ceph-mon[32751]: pgmap v958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:53 smithi090 ceph-mon[29180]: pgmap v958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:53.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 678 sec 2024-04-26T18:39:54.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 679 sec 2024-04-26T18:39:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:55 smithi119 ceph-mon[32752]: pgmap v959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:55 smithi167 ceph-mon[32751]: pgmap v959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:55 smithi090 ceph-mon[29180]: pgmap v959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:39:55.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 680 sec 2024-04-26T18:39:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:56 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 277.639806 secs 2024-04-26T18:39:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:56 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 277.639806 secs 2024-04-26T18:39:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:56 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 277.639806 secs 2024-04-26T18:39:56.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 681 sec 2024-04-26T18:39:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:57 smithi119 ceph-mon[32752]: pgmap v960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:57 smithi167 ceph-mon[32751]: pgmap v960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:57 smithi090 ceph-mon[29180]: pgmap v960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:57.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 682 sec 2024-04-26T18:39:58.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 683 sec 2024-04-26T18:39:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:39:59 smithi119 ceph-mon[32752]: pgmap v961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:39:59 smithi167 ceph-mon[32751]: pgmap v961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:39:59 smithi090 ceph-mon[29180]: pgmap v961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:39:59.939 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 684 sec 2024-04-26T18:40:00.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: Health detail: HEALTH_WARN 8 failed cephadm daemon(s); 1 MDSs report slow requests 2024-04-26T18:40:00.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: [WRN] CEPHADM_FAILED_DAEMON: 8 failed cephadm daemon(s) 2024-04-26T18:40:00.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.j on smithi090 is in unknown state 2024-04-26T18:40:00.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.d on smithi090 is in unknown state 2024-04-26T18:40:00.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.k on smithi119 is in error state 2024-04-26T18:40:00.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.e on smithi119 is in error state 2024-04-26T18:40:00.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.h on smithi119 is in error state 2024-04-26T18:40:00.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.l on smithi167 is in error state 2024-04-26T18:40:00.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.c on smithi167 is in error state 2024-04-26T18:40:00.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: daemon mds.i on smithi167 is in error state 2024-04-26T18:40:00.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-04-26T18:40:00.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:00 smithi090 ceph-mon[29180]: mds.g(mds.1): 9 slow requests are blocked > 180 secs 2024-04-26T18:40:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: Health detail: HEALTH_WARN 8 failed cephadm daemon(s); 1 MDSs report slow requests 2024-04-26T18:40:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: [WRN] CEPHADM_FAILED_DAEMON: 8 failed cephadm daemon(s) 2024-04-26T18:40:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.j on smithi090 is in unknown state 2024-04-26T18:40:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.d on smithi090 is in unknown state 2024-04-26T18:40:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.k on smithi119 is in error state 2024-04-26T18:40:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.e on smithi119 is in error state 2024-04-26T18:40:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.h on smithi119 is in error state 2024-04-26T18:40:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.l on smithi167 is in error state 2024-04-26T18:40:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.c on smithi167 is in error state 2024-04-26T18:40:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: daemon mds.i on smithi167 is in error state 2024-04-26T18:40:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-04-26T18:40:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:00 smithi119 ceph-mon[32752]: mds.g(mds.1): 9 slow requests are blocked > 180 secs 2024-04-26T18:40:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: Health detail: HEALTH_WARN 8 failed cephadm daemon(s); 1 MDSs report slow requests 2024-04-26T18:40:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: [WRN] CEPHADM_FAILED_DAEMON: 8 failed cephadm daemon(s) 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.j on smithi090 is in unknown state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.d on smithi090 is in unknown state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.k on smithi119 is in error state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.e on smithi119 is in error state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.h on smithi119 is in error state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.l on smithi167 is in error state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.c on smithi167 is in error state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: daemon mds.i on smithi167 is in error state 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2024-04-26T18:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:00 smithi167 ceph-mon[32751]: mds.g(mds.1): 9 slow requests are blocked > 180 secs 2024-04-26T18:40:00.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 685 sec 2024-04-26T18:40:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:01 smithi119 ceph-mon[32752]: pgmap v962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:01 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 282.639923 secs 2024-04-26T18:40:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:01 smithi167 ceph-mon[32751]: pgmap v962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:01 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 282.639923 secs 2024-04-26T18:40:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:01 smithi090 ceph-mon[29180]: pgmap v962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:01 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 282.639923 secs 2024-04-26T18:40:01.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 686 sec 2024-04-26T18:40:02.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 687 sec 2024-04-26T18:40:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:03 smithi119 ceph-mon[32752]: pgmap v963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:03.557 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:03 smithi090 ceph-mon[29180]: pgmap v963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:03 smithi167 ceph-mon[32751]: pgmap v963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:03.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 688 sec 2024-04-26T18:40:04.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 689 sec 2024-04-26T18:40:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:05 smithi119 ceph-mon[32752]: pgmap v964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:05 smithi167 ceph-mon[32751]: pgmap v964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:05 smithi090 ceph-mon[29180]: pgmap v964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:05.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 690 sec 2024-04-26T18:40:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:06 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 287.640040 secs 2024-04-26T18:40:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:06 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 287.640040 secs 2024-04-26T18:40:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:06 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 287.640040 secs 2024-04-26T18:40:06.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 691 sec 2024-04-26T18:40:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:07 smithi119 ceph-mon[32752]: pgmap v965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:07 smithi167 ceph-mon[32751]: pgmap v965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:07 smithi090 ceph-mon[29180]: pgmap v965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:07.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 692 sec 2024-04-26T18:40:08.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 693 sec 2024-04-26T18:40:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:09 smithi119 ceph-mon[32752]: pgmap v966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:09 smithi167 ceph-mon[32751]: pgmap v966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:09 smithi090 ceph-mon[29180]: pgmap v966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:09.940 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 694 sec 2024-04-26T18:40:10.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 695 sec 2024-04-26T18:40:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:11 smithi119 ceph-mon[32752]: pgmap v967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:11 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 292.640158 secs 2024-04-26T18:40:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:11 smithi167 ceph-mon[32751]: pgmap v967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:11 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 292.640158 secs 2024-04-26T18:40:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:11 smithi090 ceph-mon[29180]: pgmap v967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:11 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 292.640158 secs 2024-04-26T18:40:11.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 696 sec 2024-04-26T18:40:12.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 697 sec 2024-04-26T18:40:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:13 smithi119 ceph-mon[32752]: pgmap v968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:13 smithi167 ceph-mon[32751]: pgmap v968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:13 smithi090 ceph-mon[29180]: pgmap v968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:13.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 698 sec 2024-04-26T18:40:14.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 699 sec 2024-04-26T18:40:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:15 smithi119 ceph-mon[32752]: pgmap v969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:15 smithi167 ceph-mon[32751]: pgmap v969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:15 smithi090 ceph-mon[29180]: pgmap v969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:15.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 700 sec 2024-04-26T18:40:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:16 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 297.640273 secs 2024-04-26T18:40:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:16 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 297.640273 secs 2024-04-26T18:40:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:16 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 297.640273 secs 2024-04-26T18:40:16.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 701 sec 2024-04-26T18:40:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:17 smithi167 ceph-mon[32751]: pgmap v970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:17 smithi090 ceph-mon[29180]: pgmap v970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:17 smithi119 ceph-mon[32752]: pgmap v970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:17.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 702 sec 2024-04-26T18:40:18.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 703 sec 2024-04-26T18:40:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:19 smithi167 ceph-mon[32751]: pgmap v971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:19 smithi090 ceph-mon[29180]: pgmap v971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:19 smithi119 ceph-mon[32752]: pgmap v971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:19.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 704 sec 2024-04-26T18:40:20.941 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 705 sec 2024-04-26T18:40:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:21 smithi167 ceph-mon[32751]: pgmap v972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:21 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 302.640419 secs 2024-04-26T18:40:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:21 smithi090 ceph-mon[29180]: pgmap v972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:21 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 302.640419 secs 2024-04-26T18:40:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:21 smithi119 ceph-mon[32752]: pgmap v972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:21 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 302.640419 secs 2024-04-26T18:40:21.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 706 sec 2024-04-26T18:40:22.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 707 sec 2024-04-26T18:40:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:23 smithi167 ceph-mon[32751]: pgmap v973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:23 smithi090 ceph-mon[29180]: pgmap v973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:23 smithi119 ceph-mon[32752]: pgmap v973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:23.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 708 sec 2024-04-26T18:40:24.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 709 sec 2024-04-26T18:40:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:25 smithi090 ceph-mon[29180]: pgmap v974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:25 smithi119 ceph-mon[32752]: pgmap v974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:25 smithi167 ceph-mon[32751]: pgmap v974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:25.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 710 sec 2024-04-26T18:40:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:26 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 307.640535 secs 2024-04-26T18:40:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:26 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 307.640535 secs 2024-04-26T18:40:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:26 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 307.640535 secs 2024-04-26T18:40:26.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 711 sec 2024-04-26T18:40:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:27 smithi090 ceph-mon[29180]: pgmap v975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:27 smithi119 ceph-mon[32752]: pgmap v975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:27 smithi167 ceph-mon[32751]: pgmap v975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:27.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 712 sec 2024-04-26T18:40:28.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 713 sec 2024-04-26T18:40:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:29 smithi090 ceph-mon[29180]: pgmap v976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:29 smithi119 ceph-mon[32752]: pgmap v976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:29 smithi167 ceph-mon[32751]: pgmap v976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:29.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 714 sec 2024-04-26T18:40:30.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 715 sec 2024-04-26T18:40:31.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:31 smithi167 ceph-mon[32751]: pgmap v977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:31.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 312.640654 secs 2024-04-26T18:40:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:31 smithi090 ceph-mon[29180]: pgmap v977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 312.640654 secs 2024-04-26T18:40:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:31 smithi119 ceph-mon[32752]: pgmap v977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 312.640654 secs 2024-04-26T18:40:31.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 716 sec 2024-04-26T18:40:32.942 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 717 sec 2024-04-26T18:40:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:33 smithi090 ceph-mon[29180]: pgmap v978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:33 smithi119 ceph-mon[32752]: pgmap v978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:33 smithi167 ceph-mon[32751]: pgmap v978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:33.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 718 sec 2024-04-26T18:40:34.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 719 sec 2024-04-26T18:40:35.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:35 smithi119 ceph-mon[32752]: pgmap v979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:35.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:35 smithi119 ceph-mon[32752]: 6 slow requests, 5 included below; oldest blocked for > 316.979165 secs 2024-04-26T18:40:35.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:35 smithi119 ceph-mon[32752]: slow request 181.242517 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:40:35.609 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:35 smithi119 ceph-mon[32752]: slow request 316.979164 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:40:35.609 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:35 smithi119 ceph-mon[32752]: slow request 316.976269 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:40:35.609 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:35 smithi119 ceph-mon[32752]: slow request 316.970659 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:40:35.609 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:35 smithi119 ceph-mon[32752]: slow request 315.003933 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:40:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:35 smithi090 ceph-mon[29180]: pgmap v979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:35 smithi090 ceph-mon[29180]: 6 slow requests, 5 included below; oldest blocked for > 316.979165 secs 2024-04-26T18:40:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:35 smithi090 ceph-mon[29180]: slow request 181.242517 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:40:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:35 smithi090 ceph-mon[29180]: slow request 316.979164 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:40:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:35 smithi090 ceph-mon[29180]: slow request 316.976269 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:40:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:35 smithi090 ceph-mon[29180]: slow request 316.970659 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:40:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:35 smithi090 ceph-mon[29180]: slow request 315.003933 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:40:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:35 smithi167 ceph-mon[32751]: pgmap v979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:35 smithi167 ceph-mon[32751]: 6 slow requests, 5 included below; oldest blocked for > 316.979165 secs 2024-04-26T18:40:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:35 smithi167 ceph-mon[32751]: slow request 181.242517 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:40:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:35 smithi167 ceph-mon[32751]: slow request 316.979164 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:40:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:35 smithi167 ceph-mon[32751]: slow request 316.976269 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:40:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:35 smithi167 ceph-mon[32751]: slow request 316.970659 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:40:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:35 smithi167 ceph-mon[32751]: slow request 315.003933 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:40:35.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 720 sec 2024-04-26T18:40:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:36 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 317.640720 secs 2024-04-26T18:40:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:36 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 317.640720 secs 2024-04-26T18:40:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:36 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 317.640720 secs 2024-04-26T18:40:36.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 721 sec 2024-04-26T18:40:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:37 smithi090 ceph-mon[29180]: pgmap v980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:37 smithi119 ceph-mon[32752]: pgmap v980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:37 smithi167 ceph-mon[32751]: pgmap v980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:37.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 722 sec 2024-04-26T18:40:38.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 723 sec 2024-04-26T18:40:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:38 smithi119 ceph-mon[32752]: pgmap v981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:38 smithi119 ceph-mon[32752]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-04-26T18:40:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:40:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:40:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:38 smithi167 ceph-mon[32751]: pgmap v981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:38 smithi167 ceph-mon[32751]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-04-26T18:40:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:40:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:40:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:38 smithi090 ceph-mon[29180]: pgmap v981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:38 smithi090 ceph-mon[29180]: Health check update: 2 MDSs report slow requests (MDS_SLOW_REQUEST) 2024-04-26T18:40:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:40:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:40:39.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 724 sec 2024-04-26T18:40:40.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 725 sec 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: 6 slow requests, 1 included below; oldest blocked for > 321.979248 secs 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: slow request 278.974640 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire snap lock 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: pgmap v982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 322.640780 secs 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:40:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: 6 slow requests, 1 included below; oldest blocked for > 321.979248 secs 2024-04-26T18:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: slow request 278.974640 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire snap lock 2024-04-26T18:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: pgmap v982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 322.640780 secs 2024-04-26T18:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: 6 slow requests, 1 included below; oldest blocked for > 321.979248 secs 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: slow request 278.974640 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire snap lock 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: pgmap v982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 322.640780 secs 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:40:41.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 726 sec 2024-04-26T18:40:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:41 smithi119 ceph-mon[32752]: pgmap v983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:40:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:41 smithi119 ceph-mon[32752]: pgmap v984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:41 smithi167 ceph-mon[32751]: pgmap v983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:40:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:41 smithi167 ceph-mon[32751]: pgmap v984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:41 smithi090 ceph-mon[29180]: pgmap v983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:40:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:41 smithi090 ceph-mon[29180]: pgmap v984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:42.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 727 sec 2024-04-26T18:40:43.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 728 sec 2024-04-26T18:40:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:43 smithi119 ceph-mon[32752]: pgmap v985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:40:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:43 smithi167 ceph-mon[32751]: pgmap v985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:40:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:43 smithi090 ceph-mon[29180]: pgmap v985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:40:44.943 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 729 sec 2024-04-26T18:40:45.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 730 sec 2024-04-26T18:40:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:45 smithi119 ceph-mon[32752]: pgmap v986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:45 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 326.979307 secs 2024-04-26T18:40:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 327.640849 secs 2024-04-26T18:40:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:45 smithi167 ceph-mon[32751]: pgmap v986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:45 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 326.979307 secs 2024-04-26T18:40:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 327.640849 secs 2024-04-26T18:40:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:45 smithi090 ceph-mon[29180]: pgmap v986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:45 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 326.979307 secs 2024-04-26T18:40:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 327.640849 secs 2024-04-26T18:40:46.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 731 sec 2024-04-26T18:40:47.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 732 sec 2024-04-26T18:40:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:47 smithi119 ceph-mon[32752]: pgmap v987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:47 smithi167 ceph-mon[32751]: pgmap v987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:47 smithi090 ceph-mon[29180]: pgmap v987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:48.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 733 sec 2024-04-26T18:40:49.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 734 sec 2024-04-26T18:40:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:49 smithi119 ceph-mon[32752]: pgmap v988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:49 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 331.979366 secs 2024-04-26T18:40:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:49 smithi167 ceph-mon[32751]: pgmap v988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:49 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 331.979366 secs 2024-04-26T18:40:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:49 smithi090 ceph-mon[29180]: pgmap v988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:40:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:49 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 331.979366 secs 2024-04-26T18:40:50.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 735 sec 2024-04-26T18:40:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:51 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 332.640966 secs 2024-04-26T18:40:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:51 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 332.640966 secs 2024-04-26T18:40:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:51 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 332.640966 secs 2024-04-26T18:40:51.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 736 sec 2024-04-26T18:40:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:52 smithi119 ceph-mon[32752]: pgmap v989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:40:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:52 smithi167 ceph-mon[32751]: pgmap v989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:40:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:52 smithi090 ceph-mon[29180]: pgmap v989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:40:52.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 737 sec 2024-04-26T18:40:53.944 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 738 sec 2024-04-26T18:40:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:54 smithi119 ceph-mon[32752]: pgmap v990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:54 smithi167 ceph-mon[32751]: pgmap v990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:54 smithi090 ceph-mon[29180]: pgmap v990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:54.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 739 sec 2024-04-26T18:40:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:55 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 336.979452 secs 2024-04-26T18:40:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:55 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 336.979452 secs 2024-04-26T18:40:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:55 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 336.979452 secs 2024-04-26T18:40:55.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 740 sec 2024-04-26T18:40:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:56 smithi119 ceph-mon[32752]: pgmap v991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:56 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 337.641073 secs 2024-04-26T18:40:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:56 smithi167 ceph-mon[32751]: pgmap v991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:56 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 337.641073 secs 2024-04-26T18:40:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:56 smithi090 ceph-mon[29180]: pgmap v991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:40:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:56 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 337.641073 secs 2024-04-26T18:40:56.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 741 sec 2024-04-26T18:40:57.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 742 sec 2024-04-26T18:40:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:40:58 smithi119 ceph-mon[32752]: pgmap v992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:40:58 smithi167 ceph-mon[32751]: pgmap v992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:40:58 smithi090 ceph-mon[29180]: pgmap v992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:40:58.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 743 sec 2024-04-26T18:40:59.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 744 sec 2024-04-26T18:41:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:00 smithi119 ceph-mon[32752]: pgmap v993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:00 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 341.979534 secs 2024-04-26T18:41:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:00 smithi167 ceph-mon[32751]: pgmap v993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:00 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 341.979534 secs 2024-04-26T18:41:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:00 smithi090 ceph-mon[29180]: pgmap v993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:00 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 341.979534 secs 2024-04-26T18:41:00.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 745 sec 2024-04-26T18:41:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:01 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 342.641190 secs 2024-04-26T18:41:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:01 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 342.641190 secs 2024-04-26T18:41:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:01 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 342.641190 secs 2024-04-26T18:41:01.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 746 sec 2024-04-26T18:41:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:02 smithi119 ceph-mon[32752]: pgmap v994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:02 smithi167 ceph-mon[32751]: pgmap v994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:02 smithi090 ceph-mon[29180]: pgmap v994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:02.945 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 747 sec 2024-04-26T18:41:03.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 748 sec 2024-04-26T18:41:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:04 smithi119 ceph-mon[32752]: pgmap v995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:04 smithi167 ceph-mon[32751]: pgmap v995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:04 smithi090 ceph-mon[29180]: pgmap v995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:04.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 749 sec 2024-04-26T18:41:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:05 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 346.979617 secs 2024-04-26T18:41:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:05 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 346.979617 secs 2024-04-26T18:41:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:05 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 346.979617 secs 2024-04-26T18:41:05.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 750 sec 2024-04-26T18:41:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:06 smithi119 ceph-mon[32752]: pgmap v996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:06 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 347.641336 secs 2024-04-26T18:41:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:06 smithi167 ceph-mon[32751]: pgmap v996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:06 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 347.641336 secs 2024-04-26T18:41:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:06 smithi090 ceph-mon[29180]: pgmap v996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:06 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 347.641336 secs 2024-04-26T18:41:06.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 751 sec 2024-04-26T18:41:07.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 752 sec 2024-04-26T18:41:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:08 smithi119 ceph-mon[32752]: pgmap v997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:08 smithi167 ceph-mon[32751]: pgmap v997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:08 smithi090 ceph-mon[29180]: pgmap v997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:08.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 753 sec 2024-04-26T18:41:09.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 754 sec 2024-04-26T18:41:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:10 smithi119 ceph-mon[32752]: pgmap v998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:10 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 351.979694 secs 2024-04-26T18:41:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:10 smithi167 ceph-mon[32751]: pgmap v998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:10 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 351.979694 secs 2024-04-26T18:41:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:10 smithi090 ceph-mon[29180]: pgmap v998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:10 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 351.979694 secs 2024-04-26T18:41:10.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 755 sec 2024-04-26T18:41:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:11 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 352.641473 secs 2024-04-26T18:41:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:11 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 352.641473 secs 2024-04-26T18:41:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:11 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 352.641473 secs 2024-04-26T18:41:11.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 756 sec 2024-04-26T18:41:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:12 smithi167 ceph-mon[32751]: pgmap v999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:12 smithi090 ceph-mon[29180]: pgmap v999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:12 smithi119 ceph-mon[32752]: pgmap v999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:12.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 757 sec 2024-04-26T18:41:13.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 758 sec 2024-04-26T18:41:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:14 smithi167 ceph-mon[32751]: pgmap v1000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:14 smithi090 ceph-mon[29180]: pgmap v1000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:14 smithi119 ceph-mon[32752]: pgmap v1000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:14.946 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 759 sec 2024-04-26T18:41:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 356.979771 secs 2024-04-26T18:41:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 356.979771 secs 2024-04-26T18:41:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 356.979771 secs 2024-04-26T18:41:15.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 760 sec 2024-04-26T18:41:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:16 smithi090 ceph-mon[29180]: pgmap v1001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:16 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 357.641582 secs 2024-04-26T18:41:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:16 smithi119 ceph-mon[32752]: pgmap v1001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:16 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 357.641582 secs 2024-04-26T18:41:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:16 smithi167 ceph-mon[32751]: pgmap v1001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:41:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:16 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 357.641582 secs 2024-04-26T18:41:16.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 761 sec 2024-04-26T18:41:17.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 762 sec 2024-04-26T18:41:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:18 smithi090 ceph-mon[29180]: pgmap v1002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:18 smithi119 ceph-mon[32752]: pgmap v1002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:18 smithi167 ceph-mon[32751]: pgmap v1002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:18.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 763 sec 2024-04-26T18:41:19.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 764 sec 2024-04-26T18:41:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:20 smithi090 ceph-mon[29180]: pgmap v1003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:20 smithi090 ceph-mon[29180]: 6 slow requests, 4 included below; oldest blocked for > 361.979852 secs 2024-04-26T18:41:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:20 smithi090 ceph-mon[29180]: slow request 361.979852 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:41:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:20 smithi090 ceph-mon[29180]: slow request 361.976957 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:20 smithi090 ceph-mon[29180]: slow request 361.971346 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:20 smithi090 ceph-mon[29180]: slow request 360.004621 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:41:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:20 smithi119 ceph-mon[32752]: pgmap v1003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:20 smithi119 ceph-mon[32752]: 6 slow requests, 4 included below; oldest blocked for > 361.979852 secs 2024-04-26T18:41:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:20 smithi119 ceph-mon[32752]: slow request 361.979852 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:41:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:20 smithi119 ceph-mon[32752]: slow request 361.976957 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:20 smithi119 ceph-mon[32752]: slow request 361.971346 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:20 smithi119 ceph-mon[32752]: slow request 360.004621 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:41:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:20 smithi167 ceph-mon[32751]: pgmap v1003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:20 smithi167 ceph-mon[32751]: 6 slow requests, 4 included below; oldest blocked for > 361.979852 secs 2024-04-26T18:41:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:20 smithi167 ceph-mon[32751]: slow request 361.979852 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:41:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:20 smithi167 ceph-mon[32751]: slow request 361.976957 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:20 smithi167 ceph-mon[32751]: slow request 361.971346 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:20 smithi167 ceph-mon[32751]: slow request 360.004621 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:41:20.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 765 sec 2024-04-26T18:41:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:21 smithi090 ceph-mon[29180]: 9 slow requests, 5 included below; oldest blocked for > 362.641646 secs 2024-04-26T18:41:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:21 smithi090 ceph-mon[29180]: slow request 362.641646 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:21 smithi090 ceph-mon[29180]: slow request 362.640085 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:21 smithi090 ceph-mon[29180]: slow request 362.638616 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:21 smithi090 ceph-mon[29180]: slow request 362.638145 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:21 smithi090 ceph-mon[29180]: slow request 362.625880 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:21 smithi119 ceph-mon[32752]: 9 slow requests, 5 included below; oldest blocked for > 362.641646 secs 2024-04-26T18:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:21 smithi119 ceph-mon[32752]: slow request 362.641646 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:21 smithi119 ceph-mon[32752]: slow request 362.640085 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:21 smithi119 ceph-mon[32752]: slow request 362.638616 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:21 smithi119 ceph-mon[32752]: slow request 362.638145 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:21 smithi119 ceph-mon[32752]: slow request 362.625880 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:21 smithi167 ceph-mon[32751]: 9 slow requests, 5 included below; oldest blocked for > 362.641646 secs 2024-04-26T18:41:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:21 smithi167 ceph-mon[32751]: slow request 362.641646 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:21 smithi167 ceph-mon[32751]: slow request 362.640085 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:21 smithi167 ceph-mon[32751]: slow request 362.638616 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:21 smithi167 ceph-mon[32751]: slow request 362.638145 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:21 smithi167 ceph-mon[32751]: slow request 362.625880 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:41:21.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 766 sec 2024-04-26T18:41:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:22 smithi090 ceph-mon[29180]: pgmap v1004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:22 smithi119 ceph-mon[32752]: pgmap v1004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:22 smithi167 ceph-mon[32751]: pgmap v1004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:22.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 767 sec 2024-04-26T18:41:23.947 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 768 sec 2024-04-26T18:41:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:24 smithi090 ceph-mon[29180]: pgmap v1005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:24 smithi119 ceph-mon[32752]: pgmap v1005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:24 smithi167 ceph-mon[32751]: pgmap v1005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:24.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 769 sec 2024-04-26T18:41:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:25 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 366.979935 secs 2024-04-26T18:41:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:25 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 366.979935 secs 2024-04-26T18:41:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:25 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 366.979935 secs 2024-04-26T18:41:25.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 770 sec 2024-04-26T18:41:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:26 smithi090 ceph-mon[29180]: pgmap v1006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:26 smithi090 ceph-mon[29180]: 9 slow requests, 4 included below; oldest blocked for > 367.641754 secs 2024-04-26T18:41:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:26 smithi090 ceph-mon[29180]: slow request 364.057117 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:41:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:26 smithi090 ceph-mon[29180]: slow request 364.056781 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:41:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:26 smithi090 ceph-mon[29180]: slow request 363.947946 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:41:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:26 smithi090 ceph-mon[29180]: slow request 363.943658 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:26 smithi119 ceph-mon[32752]: pgmap v1006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:26 smithi119 ceph-mon[32752]: 9 slow requests, 4 included below; oldest blocked for > 367.641754 secs 2024-04-26T18:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:26 smithi119 ceph-mon[32752]: slow request 364.057117 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:26 smithi119 ceph-mon[32752]: slow request 364.056781 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:26 smithi119 ceph-mon[32752]: slow request 363.947946 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:26 smithi119 ceph-mon[32752]: slow request 363.943658 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:41:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:26 smithi167 ceph-mon[32751]: pgmap v1006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:26 smithi167 ceph-mon[32751]: 9 slow requests, 4 included below; oldest blocked for > 367.641754 secs 2024-04-26T18:41:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:26 smithi167 ceph-mon[32751]: slow request 364.057117 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:41:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:26 smithi167 ceph-mon[32751]: slow request 364.056781 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:41:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:26 smithi167 ceph-mon[32751]: slow request 363.947946 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:41:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:26 smithi167 ceph-mon[32751]: slow request 363.943658 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:41:26.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 771 sec 2024-04-26T18:41:27.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 772 sec 2024-04-26T18:41:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:28 smithi090 ceph-mon[29180]: pgmap v1007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:41:28.509+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8005850 con 0x7f2cd4104500 2024-04-26T18:41:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:41:28.509+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8005850 con 0x7f2cd4104500 2024-04-26T18:41:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:41:28.528+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac005850 con 0x56302893f0e0 2024-04-26T18:41:28.529 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:41:28.529+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac005850 con 0x56302893f0e0 2024-04-26T18:41:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:28 smithi119 ceph-mon[32752]: pgmap v1007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:28 smithi167 ceph-mon[32751]: pgmap v1007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:28.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 773 sec 2024-04-26T18:41:29.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 774 sec 2024-04-26T18:41:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:30 smithi090 ceph-mon[29180]: pgmap v1008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:30 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 371.980012 secs 2024-04-26T18:41:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:30 smithi119 ceph-mon[32752]: pgmap v1008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:30 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 371.980012 secs 2024-04-26T18:41:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:30 smithi167 ceph-mon[32751]: pgmap v1008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:30 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 371.980012 secs 2024-04-26T18:41:30.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 775 sec 2024-04-26T18:41:31.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 372.641820 secs 2024-04-26T18:41:31.277 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:41:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:41:30.875+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:41:31.278 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:41:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:41:30.875+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:41:31.278 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:41:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:41:30.875+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:41:31.278 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:41:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:41:30.875+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:41:31.278 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:41:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:41:30.875+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:41:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 372.641820 secs 2024-04-26T18:41:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 372.641820 secs 2024-04-26T18:41:31.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 776 sec 2024-04-26T18:41:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:32 smithi090 ceph-mon[29180]: pgmap v1009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:32 smithi119 ceph-mon[32752]: pgmap v1009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:32 smithi167 ceph-mon[32751]: pgmap v1009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:32.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 777 sec 2024-04-26T18:41:33.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 778 sec 2024-04-26T18:41:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:34 smithi090 ceph-mon[29180]: pgmap v1010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:34 smithi119 ceph-mon[32752]: pgmap v1010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:34 smithi167 ceph-mon[32751]: pgmap v1010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:34.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 779 sec 2024-04-26T18:41:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:35 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 376.980073 secs 2024-04-26T18:41:35.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:35 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 376.980073 secs 2024-04-26T18:41:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:35 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 376.980073 secs 2024-04-26T18:41:35.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 780 sec 2024-04-26T18:41:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:36 smithi090 ceph-mon[29180]: pgmap v1011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:36 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 377.642181 secs 2024-04-26T18:41:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:36 smithi119 ceph-mon[32752]: pgmap v1011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:36 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 377.642181 secs 2024-04-26T18:41:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:36 smithi167 ceph-mon[32751]: pgmap v1011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:36 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 377.642181 secs 2024-04-26T18:41:36.948 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 781 sec 2024-04-26T18:41:37.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 782 sec 2024-04-26T18:41:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:38 smithi090 ceph-mon[29180]: pgmap v1012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:38 smithi119 ceph-mon[32752]: pgmap v1012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:38 smithi167 ceph-mon[32751]: pgmap v1012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:38.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 783 sec 2024-04-26T18:41:39.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 784 sec 2024-04-26T18:41:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:40 smithi090 ceph-mon[29180]: pgmap v1013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:40 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 381.980157 secs 2024-04-26T18:41:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:40 smithi119 ceph-mon[32752]: pgmap v1013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:40 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 381.980157 secs 2024-04-26T18:41:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:40 smithi167 ceph-mon[32751]: pgmap v1013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:40 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 381.980157 secs 2024-04-26T18:41:40.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 785 sec 2024-04-26T18:41:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:41 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 382.642316 secs 2024-04-26T18:41:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:41:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:41 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 382.642316 secs 2024-04-26T18:41:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:41 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 382.642316 secs 2024-04-26T18:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:41:41.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 786 sec 2024-04-26T18:41:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:42 smithi119 ceph-mon[32752]: pgmap v1014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:41:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:41:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:41:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:42 smithi167 ceph-mon[32751]: pgmap v1014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:41:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:41:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:41:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:42 smithi090 ceph-mon[29180]: pgmap v1014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T18:41:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:41:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:41:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:41:42.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 787 sec 2024-04-26T18:41:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:43 smithi119 ceph-mon[32752]: pgmap v1015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T18:41:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:43 smithi167 ceph-mon[32751]: pgmap v1015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T18:41:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:43 smithi090 ceph-mon[29180]: pgmap v1015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T18:41:43.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 788 sec 2024-04-26T18:41:44.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 789 sec 2024-04-26T18:41:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:45 smithi119 ceph-mon[32752]: pgmap v1016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 190 B/s wr, 15 op/s 2024-04-26T18:41:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:45 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 386.980240 secs 2024-04-26T18:41:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:45 smithi167 ceph-mon[32751]: pgmap v1016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 190 B/s wr, 15 op/s 2024-04-26T18:41:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:45 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 386.980240 secs 2024-04-26T18:41:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:45 smithi090 ceph-mon[29180]: pgmap v1016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 190 B/s wr, 15 op/s 2024-04-26T18:41:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:45 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 386.980240 secs 2024-04-26T18:41:45.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 790 sec 2024-04-26T18:41:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:46 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 387.642426 secs 2024-04-26T18:41:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:46 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 387.642426 secs 2024-04-26T18:41:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:46 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 387.642426 secs 2024-04-26T18:41:46.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 791 sec 2024-04-26T18:41:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:47 smithi119 ceph-mon[32752]: pgmap v1017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:47 smithi167 ceph-mon[32751]: pgmap v1017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:47 smithi090 ceph-mon[29180]: pgmap v1017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:47.949 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 792 sec 2024-04-26T18:41:48.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 793 sec 2024-04-26T18:41:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:49 smithi119 ceph-mon[32752]: pgmap v1018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:49 smithi167 ceph-mon[32751]: pgmap v1018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:49 smithi090 ceph-mon[29180]: pgmap v1018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:49.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 794 sec 2024-04-26T18:41:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:50 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 391.980319 secs 2024-04-26T18:41:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:50 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 391.980319 secs 2024-04-26T18:41:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:50 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 391.980319 secs 2024-04-26T18:41:50.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 795 sec 2024-04-26T18:41:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:51 smithi119 ceph-mon[32752]: pgmap v1019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:51 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 392.642541 secs 2024-04-26T18:41:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:51 smithi167 ceph-mon[32751]: pgmap v1019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:51 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 392.642541 secs 2024-04-26T18:41:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:51 smithi090 ceph-mon[29180]: pgmap v1019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:51 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 392.642541 secs 2024-04-26T18:41:51.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 796 sec 2024-04-26T18:41:52.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 797 sec 2024-04-26T18:41:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:53 smithi119 ceph-mon[32752]: pgmap v1020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:53 smithi167 ceph-mon[32751]: pgmap v1020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:53 smithi090 ceph-mon[29180]: pgmap v1020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:41:53.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 798 sec 2024-04-26T18:41:54.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 799 sec 2024-04-26T18:41:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:55 smithi119 ceph-mon[32752]: pgmap v1021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:55 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 396.980404 secs 2024-04-26T18:41:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:55 smithi167 ceph-mon[32751]: pgmap v1021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:55 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 396.980404 secs 2024-04-26T18:41:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:55 smithi090 ceph-mon[29180]: pgmap v1021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:41:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:55 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 396.980404 secs 2024-04-26T18:41:55.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 800 sec 2024-04-26T18:41:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:56 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 397.642636 secs 2024-04-26T18:41:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:56 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 397.642636 secs 2024-04-26T18:41:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:56 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 397.642636 secs 2024-04-26T18:41:56.950 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 801 sec 2024-04-26T18:41:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:57 smithi119 ceph-mon[32752]: pgmap v1022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:57 smithi167 ceph-mon[32751]: pgmap v1022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:57 smithi090 ceph-mon[29180]: pgmap v1022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:57.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 802 sec 2024-04-26T18:41:58.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 803 sec 2024-04-26T18:41:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:41:59 smithi119 ceph-mon[32752]: pgmap v1023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:41:59 smithi167 ceph-mon[32751]: pgmap v1023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:41:59 smithi090 ceph-mon[29180]: pgmap v1023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:41:59.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 804 sec 2024-04-26T18:42:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:00 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 401.980485 secs 2024-04-26T18:42:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:00 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 401.980485 secs 2024-04-26T18:42:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:00 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 401.980485 secs 2024-04-26T18:42:00.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 805 sec 2024-04-26T18:42:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:01 smithi167 ceph-mon[32751]: pgmap v1024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:01 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 402.642772 secs 2024-04-26T18:42:01.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:01 smithi090 ceph-mon[29180]: pgmap v1024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:01.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:01 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 402.642772 secs 2024-04-26T18:42:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:01 smithi119 ceph-mon[32752]: pgmap v1024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:01 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 402.642772 secs 2024-04-26T18:42:01.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 806 sec 2024-04-26T18:42:02.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 807 sec 2024-04-26T18:42:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:03 smithi119 ceph-mon[32752]: pgmap v1025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:42:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:03 smithi167 ceph-mon[32751]: pgmap v1025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:42:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:03 smithi090 ceph-mon[29180]: pgmap v1025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:42:03.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 808 sec 2024-04-26T18:42:04.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 809 sec 2024-04-26T18:42:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:05 smithi119 ceph-mon[32752]: pgmap v1026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:05 smithi119 ceph-mon[32752]: 6 slow requests, 1 included below; oldest blocked for > 406.980570 secs 2024-04-26T18:42:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:05 smithi119 ceph-mon[32752]: slow request 363.975962 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:42:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:05 smithi167 ceph-mon[32751]: pgmap v1026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:05 smithi167 ceph-mon[32751]: 6 slow requests, 1 included below; oldest blocked for > 406.980570 secs 2024-04-26T18:42:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:05 smithi167 ceph-mon[32751]: slow request 363.975962 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:42:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:05 smithi090 ceph-mon[29180]: pgmap v1026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:05 smithi090 ceph-mon[29180]: 6 slow requests, 1 included below; oldest blocked for > 406.980570 secs 2024-04-26T18:42:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:05 smithi090 ceph-mon[29180]: slow request 363.975962 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:42:05.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 810 sec 2024-04-26T18:42:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:06 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 407.642890 secs 2024-04-26T18:42:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:06 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 407.642890 secs 2024-04-26T18:42:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:06 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 407.642890 secs 2024-04-26T18:42:06.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 811 sec 2024-04-26T18:42:07.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:07 smithi090 ceph-mon[29180]: pgmap v1027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:07 smithi119 ceph-mon[32752]: pgmap v1027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:07 smithi167 ceph-mon[32751]: pgmap v1027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:07.951 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 812 sec 2024-04-26T18:42:08.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 813 sec 2024-04-26T18:42:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:09 smithi119 ceph-mon[32752]: pgmap v1028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:09 smithi167 ceph-mon[32751]: pgmap v1028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:09 smithi090 ceph-mon[29180]: pgmap v1028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:09.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 814 sec 2024-04-26T18:42:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:10 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 411.980647 secs 2024-04-26T18:42:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:10 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 411.980647 secs 2024-04-26T18:42:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:10 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 411.980647 secs 2024-04-26T18:42:10.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 815 sec 2024-04-26T18:42:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:11 smithi119 ceph-mon[32752]: pgmap v1029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:11 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 412.643035 secs 2024-04-26T18:42:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:11 smithi167 ceph-mon[32751]: pgmap v1029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:11 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 412.643035 secs 2024-04-26T18:42:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:11 smithi090 ceph-mon[29180]: pgmap v1029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:11 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 412.643035 secs 2024-04-26T18:42:11.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 816 sec 2024-04-26T18:42:12.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 817 sec 2024-04-26T18:42:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:13 smithi119 ceph-mon[32752]: pgmap v1030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:13 smithi167 ceph-mon[32751]: pgmap v1030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:13 smithi090 ceph-mon[29180]: pgmap v1030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:42:13.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 818 sec 2024-04-26T18:42:14.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 819 sec 2024-04-26T18:42:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:15 smithi119 ceph-mon[32752]: pgmap v1031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 416.980723 secs 2024-04-26T18:42:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:15 smithi167 ceph-mon[32751]: pgmap v1031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 416.980723 secs 2024-04-26T18:42:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:15 smithi090 ceph-mon[29180]: pgmap v1031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:42:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 416.980723 secs 2024-04-26T18:42:15.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 820 sec 2024-04-26T18:42:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:16 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 417.643135 secs 2024-04-26T18:42:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:16 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 417.643135 secs 2024-04-26T18:42:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:16 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 417.643135 secs 2024-04-26T18:42:16.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 821 sec 2024-04-26T18:42:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:17 smithi119 ceph-mon[32752]: pgmap v1032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:17 smithi167 ceph-mon[32751]: pgmap v1032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:17 smithi090 ceph-mon[29180]: pgmap v1032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:17.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 822 sec 2024-04-26T18:42:18.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 823 sec 2024-04-26T18:42:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:19 smithi119 ceph-mon[32752]: pgmap v1033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:19 smithi167 ceph-mon[32751]: pgmap v1033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:19 smithi090 ceph-mon[29180]: pgmap v1033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:19.952 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 824 sec 2024-04-26T18:42:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:20 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 421.980802 secs 2024-04-26T18:42:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:20 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 421.980802 secs 2024-04-26T18:42:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:20 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 421.980802 secs 2024-04-26T18:42:20.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 825 sec 2024-04-26T18:42:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:21 smithi119 ceph-mon[32752]: pgmap v1034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:21 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 422.643259 secs 2024-04-26T18:42:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:21 smithi167 ceph-mon[32751]: pgmap v1034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:21 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 422.643259 secs 2024-04-26T18:42:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:21 smithi090 ceph-mon[29180]: pgmap v1034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:21 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 422.643259 secs 2024-04-26T18:42:21.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 826 sec 2024-04-26T18:42:22.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 827 sec 2024-04-26T18:42:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:23 smithi119 ceph-mon[32752]: pgmap v1035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:23 smithi167 ceph-mon[32751]: pgmap v1035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:23 smithi090 ceph-mon[29180]: pgmap v1035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:23.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 828 sec 2024-04-26T18:42:24.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 829 sec 2024-04-26T18:42:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:25 smithi119 ceph-mon[32752]: pgmap v1036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:25 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 426.980884 secs 2024-04-26T18:42:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:25 smithi167 ceph-mon[32751]: pgmap v1036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:25 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 426.980884 secs 2024-04-26T18:42:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:25 smithi090 ceph-mon[29180]: pgmap v1036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:25 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 426.980884 secs 2024-04-26T18:42:25.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 830 sec 2024-04-26T18:42:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:26 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 427.643348 secs 2024-04-26T18:42:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:26 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 427.643348 secs 2024-04-26T18:42:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:26 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 427.643348 secs 2024-04-26T18:42:26.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 831 sec 2024-04-26T18:42:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:27 smithi119 ceph-mon[32752]: pgmap v1037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:27 smithi167 ceph-mon[32751]: pgmap v1037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:27 smithi090 ceph-mon[29180]: pgmap v1037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:27.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 832 sec 2024-04-26T18:42:28.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 833 sec 2024-04-26T18:42:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:29 smithi119 ceph-mon[32752]: pgmap v1038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:29 smithi167 ceph-mon[32751]: pgmap v1038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:29 smithi090 ceph-mon[29180]: pgmap v1038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:29.953 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 834 sec 2024-04-26T18:42:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:30 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 431.980967 secs 2024-04-26T18:42:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:30 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 431.980967 secs 2024-04-26T18:42:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:30 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 431.980967 secs 2024-04-26T18:42:30.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 835 sec 2024-04-26T18:42:31.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:31 smithi090 ceph-mon[29180]: pgmap v1039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:31.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 432.643396 secs 2024-04-26T18:42:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:31 smithi119 ceph-mon[32752]: pgmap v1039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 432.643396 secs 2024-04-26T18:42:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:31 smithi167 ceph-mon[32751]: pgmap v1039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 432.643396 secs 2024-04-26T18:42:31.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 836 sec 2024-04-26T18:42:32.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 837 sec 2024-04-26T18:42:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:33 smithi119 ceph-mon[32752]: pgmap v1040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:33 smithi167 ceph-mon[32751]: pgmap v1040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:33 smithi090 ceph-mon[29180]: pgmap v1040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:33.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 838 sec 2024-04-26T18:42:34.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 839 sec 2024-04-26T18:42:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:35 smithi119 ceph-mon[32752]: pgmap v1041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:35 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 436.981057 secs 2024-04-26T18:42:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:35 smithi167 ceph-mon[32751]: pgmap v1041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:35 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 436.981057 secs 2024-04-26T18:42:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:35 smithi090 ceph-mon[29180]: pgmap v1041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:35 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 436.981057 secs 2024-04-26T18:42:35.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 840 sec 2024-04-26T18:42:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:36 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 437.643535 secs 2024-04-26T18:42:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:36 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 437.643535 secs 2024-04-26T18:42:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:36 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 437.643535 secs 2024-04-26T18:42:36.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 841 sec 2024-04-26T18:42:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:37 smithi119 ceph-mon[32752]: pgmap v1042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:37 smithi167 ceph-mon[32751]: pgmap v1042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:37 smithi090 ceph-mon[29180]: pgmap v1042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:37.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 842 sec 2024-04-26T18:42:38.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 843 sec 2024-04-26T18:42:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:39 smithi119 ceph-mon[32752]: pgmap v1043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:39 smithi167 ceph-mon[32751]: pgmap v1043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:39 smithi090 ceph-mon[29180]: pgmap v1043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:39.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 844 sec 2024-04-26T18:42:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:40 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 441.981136 secs 2024-04-26T18:42:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:40 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 441.981136 secs 2024-04-26T18:42:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:40 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 441.981136 secs 2024-04-26T18:42:40.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 845 sec 2024-04-26T18:42:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:41 smithi119 ceph-mon[32752]: pgmap v1044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:41 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 442.643649 secs 2024-04-26T18:42:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:41 smithi167 ceph-mon[32751]: pgmap v1044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:41 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 442.643649 secs 2024-04-26T18:42:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:41 smithi090 ceph-mon[29180]: pgmap v1044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:41 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 442.643649 secs 2024-04-26T18:42:41.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 846 sec 2024-04-26T18:42:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:42:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:42:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:42:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:42.954 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 847 sec 2024-04-26T18:42:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:43 smithi119 ceph-mon[32752]: pgmap v1045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:43 smithi167 ceph-mon[32751]: pgmap v1045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:43 smithi090 ceph-mon[29180]: pgmap v1045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:43.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 848 sec 2024-04-26T18:42:44.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 849 sec 2024-04-26T18:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:44 smithi119 ceph-mon[32752]: pgmap v1046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:44 smithi167 ceph-mon[32751]: pgmap v1046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:44 smithi090 ceph-mon[29180]: pgmap v1046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:42:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:45.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 850 sec 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 446.981204 secs 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 447.643760 secs 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:42:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:42:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 446.981204 secs 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 447.643760 secs 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:42:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:42:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 446.981204 secs 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 447.643760 secs 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:42:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:42:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:42:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:42:46.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 851 sec 2024-04-26T18:42:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:46 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:42:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:46 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:42:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:46 smithi090 ceph-mon[29180]: pgmap v1047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-26T18:42:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:46 smithi090 ceph-mon[29180]: pgmap v1048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:46 smithi090 ceph-mon[29180]: Health check update: 9 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:46 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:46 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:46 smithi119 ceph-mon[32752]: pgmap v1047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-26T18:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:46 smithi119 ceph-mon[32752]: pgmap v1048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:46 smithi119 ceph-mon[32752]: Health check update: 9 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:42:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:46 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:42:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:46 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:42:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:46 smithi167 ceph-mon[32751]: pgmap v1047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-26T18:42:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:46 smithi167 ceph-mon[32751]: pgmap v1048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:46 smithi167 ceph-mon[32751]: Health check update: 9 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-26T18:42:47.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 852 sec 2024-04-26T18:42:48.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 853 sec 2024-04-26T18:42:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:48 smithi090 ceph-mon[29180]: pgmap v1049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:48 smithi119 ceph-mon[32752]: pgmap v1049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:48 smithi167 ceph-mon[32751]: pgmap v1049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:49.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 854 sec 2024-04-26T18:42:50.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 855 sec 2024-04-26T18:42:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:50 smithi090 ceph-mon[29180]: pgmap v1050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:50 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 451.981283 secs 2024-04-26T18:42:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 452.643874 secs 2024-04-26T18:42:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:50 smithi119 ceph-mon[32752]: pgmap v1050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:50 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 451.981283 secs 2024-04-26T18:42:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 452.643874 secs 2024-04-26T18:42:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:50 smithi167 ceph-mon[32751]: pgmap v1050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:50 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 451.981283 secs 2024-04-26T18:42:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 452.643874 secs 2024-04-26T18:42:51.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 856 sec 2024-04-26T18:42:52.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 857 sec 2024-04-26T18:42:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:52 smithi090 ceph-mon[29180]: pgmap v1051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:52 smithi119 ceph-mon[32752]: pgmap v1051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:52 smithi167 ceph-mon[32751]: pgmap v1051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:53.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 858 sec 2024-04-26T18:42:54.955 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 859 sec 2024-04-26T18:42:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:54 smithi090 ceph-mon[29180]: pgmap v1052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:54 smithi119 ceph-mon[32752]: pgmap v1052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:54 smithi167 ceph-mon[32751]: pgmap v1052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T18:42:55.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 860 sec 2024-04-26T18:42:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:55 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 456.981367 secs 2024-04-26T18:42:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 457.643967 secs 2024-04-26T18:42:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:55 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 456.981367 secs 2024-04-26T18:42:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 457.643967 secs 2024-04-26T18:42:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:55 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 456.981367 secs 2024-04-26T18:42:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 457.643967 secs 2024-04-26T18:42:56.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 861 sec 2024-04-26T18:42:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:56 smithi090 ceph-mon[29180]: pgmap v1053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:42:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:56 smithi119 ceph-mon[32752]: pgmap v1053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:42:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:56 smithi167 ceph-mon[32751]: pgmap v1053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:42:57.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 862 sec 2024-04-26T18:42:58.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 863 sec 2024-04-26T18:42:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:42:58 smithi090 ceph-mon[29180]: pgmap v1054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:42:58 smithi119 ceph-mon[32752]: pgmap v1054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:42:58 smithi167 ceph-mon[32751]: pgmap v1054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:42:59.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 864 sec 2024-04-26T18:43:00.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 865 sec 2024-04-26T18:43:01.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:00 smithi167 ceph-mon[32751]: pgmap v1055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:01.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:00 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 461.981449 secs 2024-04-26T18:43:01.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 462.644064 secs 2024-04-26T18:43:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:00 smithi090 ceph-mon[29180]: pgmap v1055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:00 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 461.981449 secs 2024-04-26T18:43:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 462.644064 secs 2024-04-26T18:43:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:00 smithi119 ceph-mon[32752]: pgmap v1055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:00 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 461.981449 secs 2024-04-26T18:43:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 462.644064 secs 2024-04-26T18:43:01.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 866 sec 2024-04-26T18:43:02.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 867 sec 2024-04-26T18:43:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:02 smithi090 ceph-mon[29180]: pgmap v1056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:02 smithi119 ceph-mon[32752]: pgmap v1056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:02 smithi167 ceph-mon[32751]: pgmap v1056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:03.956 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 868 sec 2024-04-26T18:43:04.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 869 sec 2024-04-26T18:43:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:04 smithi090 ceph-mon[29180]: pgmap v1057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:04 smithi119 ceph-mon[32752]: pgmap v1057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:04 smithi167 ceph-mon[32751]: pgmap v1057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:05.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 870 sec 2024-04-26T18:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:05 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 466.981532 secs 2024-04-26T18:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 467.644171 secs 2024-04-26T18:43:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:05 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 466.981532 secs 2024-04-26T18:43:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 467.644171 secs 2024-04-26T18:43:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:05 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 466.981532 secs 2024-04-26T18:43:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 467.644171 secs 2024-04-26T18:43:06.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 871 sec 2024-04-26T18:43:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:06 smithi119 ceph-mon[32752]: pgmap v1058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:06 smithi167 ceph-mon[32751]: pgmap v1058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:06 smithi090 ceph-mon[29180]: pgmap v1058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:07.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 872 sec 2024-04-26T18:43:08.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 873 sec 2024-04-26T18:43:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:08 smithi119 ceph-mon[32752]: pgmap v1059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:08 smithi167 ceph-mon[32751]: pgmap v1059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:08 smithi090 ceph-mon[29180]: pgmap v1059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:09.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 874 sec 2024-04-26T18:43:10.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 875 sec 2024-04-26T18:43:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:10 smithi119 ceph-mon[32752]: pgmap v1060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:10 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 471.981619 secs 2024-04-26T18:43:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 472.644285 secs 2024-04-26T18:43:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:10 smithi167 ceph-mon[32751]: pgmap v1060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:10 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 471.981619 secs 2024-04-26T18:43:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 472.644285 secs 2024-04-26T18:43:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:10 smithi090 ceph-mon[29180]: pgmap v1060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:10 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 471.981619 secs 2024-04-26T18:43:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 472.644285 secs 2024-04-26T18:43:11.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 876 sec 2024-04-26T18:43:12.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 877 sec 2024-04-26T18:43:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:12 smithi119 ceph-mon[32752]: pgmap v1061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:12 smithi167 ceph-mon[32751]: pgmap v1061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:12 smithi090 ceph-mon[29180]: pgmap v1061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:13.957 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 878 sec 2024-04-26T18:43:14.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 879 sec 2024-04-26T18:43:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:14 smithi119 ceph-mon[32752]: pgmap v1062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:14 smithi167 ceph-mon[32751]: pgmap v1062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:14 smithi090 ceph-mon[29180]: pgmap v1062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:15.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 880 sec 2024-04-26T18:43:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 476.981700 secs 2024-04-26T18:43:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 477.644404 secs 2024-04-26T18:43:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 476.981700 secs 2024-04-26T18:43:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 477.644404 secs 2024-04-26T18:43:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 476.981700 secs 2024-04-26T18:43:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 477.644404 secs 2024-04-26T18:43:16.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 881 sec 2024-04-26T18:43:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:16 smithi119 ceph-mon[32752]: pgmap v1063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:16 smithi167 ceph-mon[32751]: pgmap v1063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:16 smithi090 ceph-mon[29180]: pgmap v1063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:17.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 882 sec 2024-04-26T18:43:18.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 883 sec 2024-04-26T18:43:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:18 smithi119 ceph-mon[32752]: pgmap v1064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:18 smithi167 ceph-mon[32751]: pgmap v1064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:18 smithi090 ceph-mon[29180]: pgmap v1064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:19.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 884 sec 2024-04-26T18:43:20.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 885 sec 2024-04-26T18:43:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:20 smithi119 ceph-mon[32752]: pgmap v1065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:20 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 481.981780 secs 2024-04-26T18:43:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 482.644521 secs 2024-04-26T18:43:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:20 smithi167 ceph-mon[32751]: pgmap v1065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:20 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 481.981780 secs 2024-04-26T18:43:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 482.644521 secs 2024-04-26T18:43:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:20 smithi090 ceph-mon[29180]: pgmap v1065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:20 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 481.981780 secs 2024-04-26T18:43:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 482.644521 secs 2024-04-26T18:43:21.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 886 sec 2024-04-26T18:43:22.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 887 sec 2024-04-26T18:43:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:22 smithi119 ceph-mon[32752]: pgmap v1066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:22 smithi167 ceph-mon[32751]: pgmap v1066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:22 smithi090 ceph-mon[29180]: pgmap v1066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:23.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 888 sec 2024-04-26T18:43:24.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 889 sec 2024-04-26T18:43:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:24 smithi090 ceph-mon[29180]: pgmap v1067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:24 smithi119 ceph-mon[32752]: pgmap v1067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:24 smithi167 ceph-mon[32751]: pgmap v1067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:25.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 890 sec 2024-04-26T18:43:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:25 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 486.981863 secs 2024-04-26T18:43:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 487.644628 secs 2024-04-26T18:43:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:25 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 486.981863 secs 2024-04-26T18:43:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 487.644628 secs 2024-04-26T18:43:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:25 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 486.981863 secs 2024-04-26T18:43:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 487.644628 secs 2024-04-26T18:43:26.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 891 sec 2024-04-26T18:43:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:26 smithi119 ceph-mon[32752]: pgmap v1068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:26 smithi167 ceph-mon[32751]: pgmap v1068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:26 smithi090 ceph-mon[29180]: pgmap v1068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:27.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 892 sec 2024-04-26T18:43:28.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 893 sec 2024-04-26T18:43:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:28 smithi119 ceph-mon[32752]: pgmap v1069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:28 smithi167 ceph-mon[32751]: pgmap v1069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:28 smithi090 ceph-mon[29180]: pgmap v1069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:29.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 894 sec 2024-04-26T18:43:30.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 895 sec 2024-04-26T18:43:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:30 smithi119 ceph-mon[32752]: pgmap v1070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:30 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 491.981942 secs 2024-04-26T18:43:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 492.644746 secs 2024-04-26T18:43:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:30 smithi167 ceph-mon[32751]: pgmap v1070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:30 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 491.981942 secs 2024-04-26T18:43:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 492.644746 secs 2024-04-26T18:43:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:30 smithi090 ceph-mon[29180]: pgmap v1070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:30 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 491.981942 secs 2024-04-26T18:43:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 492.644746 secs 2024-04-26T18:43:31.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 896 sec 2024-04-26T18:43:32.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 897 sec 2024-04-26T18:43:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:32 smithi119 ceph-mon[32752]: pgmap v1071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:32 smithi167 ceph-mon[32751]: pgmap v1071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:32 smithi090 ceph-mon[29180]: pgmap v1071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:33.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 898 sec 2024-04-26T18:43:34.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 899 sec 2024-04-26T18:43:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:34 smithi119 ceph-mon[32752]: pgmap v1072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:34 smithi167 ceph-mon[32751]: pgmap v1072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:34 smithi090 ceph-mon[29180]: pgmap v1072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:35.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 900 sec 2024-04-26T18:43:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:35 smithi119 ceph-mon[32752]: 6 slow requests, 1 included below; oldest blocked for > 496.982032 secs 2024-04-26T18:43:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:35 smithi119 ceph-mon[32752]: slow request 361.245385 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:43:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 497.644872 secs 2024-04-26T18:43:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:35 smithi167 ceph-mon[32751]: 6 slow requests, 1 included below; oldest blocked for > 496.982032 secs 2024-04-26T18:43:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:35 smithi167 ceph-mon[32751]: slow request 361.245385 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:43:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 497.644872 secs 2024-04-26T18:43:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:35 smithi090 ceph-mon[29180]: 6 slow requests, 1 included below; oldest blocked for > 496.982032 secs 2024-04-26T18:43:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:35 smithi090 ceph-mon[29180]: slow request 361.245385 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:43:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 497.644872 secs 2024-04-26T18:43:36.959 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 901 sec 2024-04-26T18:43:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:36 smithi119 ceph-mon[32752]: pgmap v1073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:36 smithi167 ceph-mon[32751]: pgmap v1073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:36 smithi090 ceph-mon[29180]: pgmap v1073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:37.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 902 sec 2024-04-26T18:43:38.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 903 sec 2024-04-26T18:43:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:38 smithi119 ceph-mon[32752]: pgmap v1074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:38 smithi167 ceph-mon[32751]: pgmap v1074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:38 smithi090 ceph-mon[29180]: pgmap v1074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:39.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 904 sec 2024-04-26T18:43:40.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 905 sec 2024-04-26T18:43:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:40 smithi119 ceph-mon[32752]: pgmap v1075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:40 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 501.982107 secs 2024-04-26T18:43:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 502.644981 secs 2024-04-26T18:43:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:40 smithi167 ceph-mon[32751]: pgmap v1075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:40 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 501.982107 secs 2024-04-26T18:43:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 502.644981 secs 2024-04-26T18:43:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:40 smithi090 ceph-mon[29180]: pgmap v1075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:40 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 501.982107 secs 2024-04-26T18:43:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 502.644981 secs 2024-04-26T18:43:41.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 906 sec 2024-04-26T18:43:42.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 907 sec 2024-04-26T18:43:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:42 smithi119 ceph-mon[32752]: pgmap v1076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:42 smithi167 ceph-mon[32751]: pgmap v1076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:42 smithi090 ceph-mon[29180]: pgmap v1076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:43:43.528+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8005b90 con 0x7f2cd4104500 2024-04-26T18:43:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:43:43.528+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8005b90 con 0x7f2cd4104500 2024-04-26T18:43:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:43:43.545+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac005b50 con 0x56302893f0e0 2024-04-26T18:43:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:43:43.545+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 8 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac005b50 con 0x56302893f0e0 2024-04-26T18:43:43.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 908 sec 2024-04-26T18:43:44.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 909 sec 2024-04-26T18:43:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:44 smithi119 ceph-mon[32752]: pgmap v1077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:44 smithi167 ceph-mon[32751]: pgmap v1077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:44 smithi090 ceph-mon[29180]: pgmap v1077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:43:45.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 910 sec 2024-04-26T18:43:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:45 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 506.982192 secs 2024-04-26T18:43:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 507.645093 secs 2024-04-26T18:43:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:43:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:45 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 506.982192 secs 2024-04-26T18:43:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 507.645093 secs 2024-04-26T18:43:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:43:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:45 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 506.982192 secs 2024-04-26T18:43:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 507.645093 secs 2024-04-26T18:43:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:43:46.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 911 sec 2024-04-26T18:43:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:47 smithi090 ceph-mon[29180]: pgmap v1078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:43:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:43:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:47 smithi119 ceph-mon[32752]: pgmap v1078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:43:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:43:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:47 smithi167 ceph-mon[32751]: pgmap v1078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:43:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:43:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:43:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:43:47.960 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 912 sec 2024-04-26T18:43:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:48 smithi090 ceph-mon[29180]: pgmap v1079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T18:43:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:48 smithi090 ceph-mon[29180]: pgmap v1080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:48 smithi119 ceph-mon[32752]: pgmap v1079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T18:43:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:48 smithi119 ceph-mon[32752]: pgmap v1080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:48 smithi167 ceph-mon[32751]: pgmap v1079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T18:43:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:48 smithi167 ceph-mon[32751]: pgmap v1080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:48.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 913 sec 2024-04-26T18:43:49.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 914 sec 2024-04-26T18:43:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:50 smithi090 ceph-mon[29180]: pgmap v1081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:50 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 511.982276 secs 2024-04-26T18:43:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:50 smithi119 ceph-mon[32752]: pgmap v1081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:50 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 511.982276 secs 2024-04-26T18:43:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:50 smithi167 ceph-mon[32751]: pgmap v1081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:50 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 511.982276 secs 2024-04-26T18:43:50.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 915 sec 2024-04-26T18:43:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:51 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 512.645165 secs 2024-04-26T18:43:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:51 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 512.645165 secs 2024-04-26T18:43:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:51 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 512.645165 secs 2024-04-26T18:43:51.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 916 sec 2024-04-26T18:43:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:52 smithi090 ceph-mon[29180]: pgmap v1082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:52 smithi119 ceph-mon[32752]: pgmap v1082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:52 smithi167 ceph-mon[32751]: pgmap v1082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:52.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 917 sec 2024-04-26T18:43:53.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 918 sec 2024-04-26T18:43:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:54 smithi090 ceph-mon[29180]: pgmap v1083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T18:43:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:54 smithi119 ceph-mon[32752]: pgmap v1083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T18:43:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:54 smithi167 ceph-mon[32751]: pgmap v1083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T18:43:54.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 919 sec 2024-04-26T18:43:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:55 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 516.982363 secs 2024-04-26T18:43:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:55 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 516.982363 secs 2024-04-26T18:43:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:55 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 516.982363 secs 2024-04-26T18:43:55.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 920 sec 2024-04-26T18:43:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:56 smithi090 ceph-mon[29180]: pgmap v1084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:56 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 517.645285 secs 2024-04-26T18:43:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:56 smithi119 ceph-mon[32752]: pgmap v1084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:56 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 517.645285 secs 2024-04-26T18:43:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:56 smithi167 ceph-mon[32751]: pgmap v1084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:43:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:56 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 517.645285 secs 2024-04-26T18:43:56.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 921 sec 2024-04-26T18:43:57.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 922 sec 2024-04-26T18:43:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:43:58 smithi119 ceph-mon[32752]: pgmap v1085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:43:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:43:58 smithi167 ceph-mon[32751]: pgmap v1085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:43:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:43:58 smithi090 ceph-mon[29180]: pgmap v1085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:43:58.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 923 sec 2024-04-26T18:43:59.961 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 924 sec 2024-04-26T18:44:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:00 smithi119 ceph-mon[32752]: pgmap v1086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:00 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 521.982447 secs 2024-04-26T18:44:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:00 smithi167 ceph-mon[32751]: pgmap v1086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:00 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 521.982447 secs 2024-04-26T18:44:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:00 smithi090 ceph-mon[29180]: pgmap v1086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:00 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 521.982447 secs 2024-04-26T18:44:00.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 925 sec 2024-04-26T18:44:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:01 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 522.645419 secs 2024-04-26T18:44:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:01 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 522.645419 secs 2024-04-26T18:44:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:01 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 522.645419 secs 2024-04-26T18:44:01.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 926 sec 2024-04-26T18:44:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:02 smithi119 ceph-mon[32752]: pgmap v1087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:02 smithi167 ceph-mon[32751]: pgmap v1087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:02 smithi090 ceph-mon[29180]: pgmap v1087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:02.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 927 sec 2024-04-26T18:44:03.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 928 sec 2024-04-26T18:44:04.428 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:04 smithi090 ceph-mon[29180]: pgmap v1088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:04 smithi119 ceph-mon[32752]: pgmap v1088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:04 smithi167 ceph-mon[32751]: pgmap v1088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:04.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 929 sec 2024-04-26T18:44:05.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:05 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 526.982531 secs 2024-04-26T18:44:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:05 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 526.982531 secs 2024-04-26T18:44:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:05 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 526.982531 secs 2024-04-26T18:44:05.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 930 sec 2024-04-26T18:44:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:06 smithi119 ceph-mon[32752]: pgmap v1089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:06 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 527.645550 secs 2024-04-26T18:44:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:06 smithi167 ceph-mon[32751]: pgmap v1089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:06 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 527.645550 secs 2024-04-26T18:44:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:06 smithi090 ceph-mon[29180]: pgmap v1089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:06 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 527.645550 secs 2024-04-26T18:44:06.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 931 sec 2024-04-26T18:44:07.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 932 sec 2024-04-26T18:44:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:08 smithi119 ceph-mon[32752]: pgmap v1090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:08 smithi167 ceph-mon[32751]: pgmap v1090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:08 smithi090 ceph-mon[29180]: pgmap v1090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:08.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 933 sec 2024-04-26T18:44:09.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 934 sec 2024-04-26T18:44:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:10 smithi119 ceph-mon[32752]: pgmap v1091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:10 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 531.982634 secs 2024-04-26T18:44:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:10 smithi167 ceph-mon[32751]: pgmap v1091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:10 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 531.982634 secs 2024-04-26T18:44:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:10 smithi090 ceph-mon[29180]: pgmap v1091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:10 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 531.982634 secs 2024-04-26T18:44:10.962 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 935 sec 2024-04-26T18:44:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:11 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 532.645691 secs 2024-04-26T18:44:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:11 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 532.645691 secs 2024-04-26T18:44:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:11 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 532.645691 secs 2024-04-26T18:44:11.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 936 sec 2024-04-26T18:44:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:12 smithi119 ceph-mon[32752]: pgmap v1092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:12 smithi167 ceph-mon[32751]: pgmap v1092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:12 smithi090 ceph-mon[29180]: pgmap v1092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:12.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 937 sec 2024-04-26T18:44:13.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 938 sec 2024-04-26T18:44:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:14 smithi119 ceph-mon[32752]: pgmap v1093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:14 smithi167 ceph-mon[32751]: pgmap v1093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:14 smithi090 ceph-mon[29180]: pgmap v1093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:14.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 939 sec 2024-04-26T18:44:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 536.982721 secs 2024-04-26T18:44:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 536.982721 secs 2024-04-26T18:44:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 536.982721 secs 2024-04-26T18:44:15.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 940 sec 2024-04-26T18:44:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:16 smithi119 ceph-mon[32752]: pgmap v1094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:16 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 537.645774 secs 2024-04-26T18:44:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:16 smithi167 ceph-mon[32751]: pgmap v1094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:16 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 537.645774 secs 2024-04-26T18:44:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:16 smithi090 ceph-mon[29180]: pgmap v1094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:16 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 537.645774 secs 2024-04-26T18:44:16.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 941 sec 2024-04-26T18:44:17.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 942 sec 2024-04-26T18:44:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:18 smithi119 ceph-mon[32752]: pgmap v1095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:18 smithi167 ceph-mon[32751]: pgmap v1095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:18 smithi090 ceph-mon[29180]: pgmap v1095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:18.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 943 sec 2024-04-26T18:44:19.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 944 sec 2024-04-26T18:44:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:20 smithi119 ceph-mon[32752]: pgmap v1096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:20 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 541.982803 secs 2024-04-26T18:44:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:20 smithi167 ceph-mon[32751]: pgmap v1096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:20 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 541.982803 secs 2024-04-26T18:44:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:20 smithi090 ceph-mon[29180]: pgmap v1096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:20 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 541.982803 secs 2024-04-26T18:44:20.963 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 945 sec 2024-04-26T18:44:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:21 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 542.645894 secs 2024-04-26T18:44:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:21 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 542.645894 secs 2024-04-26T18:44:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:21 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 542.645894 secs 2024-04-26T18:44:21.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 946 sec 2024-04-26T18:44:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:22 smithi119 ceph-mon[32752]: pgmap v1097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:22 smithi167 ceph-mon[32751]: pgmap v1097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:22 smithi090 ceph-mon[29180]: pgmap v1097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:22.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 947 sec 2024-04-26T18:44:23.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 948 sec 2024-04-26T18:44:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:24 smithi119 ceph-mon[32752]: pgmap v1098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:24 smithi167 ceph-mon[32751]: pgmap v1098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:24 smithi090 ceph-mon[29180]: pgmap v1098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:24.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 949 sec 2024-04-26T18:44:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:25 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 546.982890 secs 2024-04-26T18:44:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:25 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 546.982890 secs 2024-04-26T18:44:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:25 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 546.982890 secs 2024-04-26T18:44:25.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 950 sec 2024-04-26T18:44:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:26 smithi119 ceph-mon[32752]: pgmap v1099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:26 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 547.646012 secs 2024-04-26T18:44:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:26 smithi167 ceph-mon[32751]: pgmap v1099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:26 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 547.646012 secs 2024-04-26T18:44:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:26 smithi090 ceph-mon[29180]: pgmap v1099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:26 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 547.646012 secs 2024-04-26T18:44:26.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 951 sec 2024-04-26T18:44:27.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 952 sec 2024-04-26T18:44:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:44:28.533+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8005ec0 con 0x7f2cd4104500 2024-04-26T18:44:28.534 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:44:28.534+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8005ec0 con 0x7f2cd4104500 2024-04-26T18:44:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:44:28.551+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac005e30 con 0x56302893f0e0 2024-04-26T18:44:28.551 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:44:28.551+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 9 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac005e30 con 0x56302893f0e0 2024-04-26T18:44:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:28 smithi119 ceph-mon[32752]: pgmap v1100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:28 smithi167 ceph-mon[32751]: pgmap v1100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:28 smithi090 ceph-mon[29180]: pgmap v1100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:28.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 953 sec 2024-04-26T18:44:29.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 954 sec 2024-04-26T18:44:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:30 smithi119 ceph-mon[32752]: pgmap v1101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:30 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 551.982973 secs 2024-04-26T18:44:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:30 smithi167 ceph-mon[32751]: pgmap v1101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:30 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 551.982973 secs 2024-04-26T18:44:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:30 smithi090 ceph-mon[29180]: pgmap v1101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:30 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 551.982973 secs 2024-04-26T18:44:30.964 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 955 sec 2024-04-26T18:44:31.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 552.646129 secs 2024-04-26T18:44:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 552.646129 secs 2024-04-26T18:44:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 552.646129 secs 2024-04-26T18:44:31.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 956 sec 2024-04-26T18:44:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:32 smithi119 ceph-mon[32752]: pgmap v1102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:32 smithi167 ceph-mon[32751]: pgmap v1102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:32 smithi090 ceph-mon[29180]: pgmap v1102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:32.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 957 sec 2024-04-26T18:44:33.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 958 sec 2024-04-26T18:44:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:34 smithi119 ceph-mon[32752]: pgmap v1103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:34 smithi167 ceph-mon[32751]: pgmap v1103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:34 smithi090 ceph-mon[29180]: pgmap v1103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:34.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 959 sec 2024-04-26T18:44:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:35 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 556.983037 secs 2024-04-26T18:44:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:35 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 556.983037 secs 2024-04-26T18:44:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:35 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 556.983037 secs 2024-04-26T18:44:35.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 960 sec 2024-04-26T18:44:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:36 smithi119 ceph-mon[32752]: pgmap v1104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:36 smithi119 ceph-mon[32752]: 10 slow requests, 1 included below; oldest blocked for > 557.646245 secs 2024-04-26T18:44:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:36 smithi119 ceph-mon[32752]: slow request 184.515258 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire snap lock 2024-04-26T18:44:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:36 smithi167 ceph-mon[32751]: pgmap v1104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:36 smithi167 ceph-mon[32751]: 10 slow requests, 1 included below; oldest blocked for > 557.646245 secs 2024-04-26T18:44:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:36 smithi167 ceph-mon[32751]: slow request 184.515258 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire snap lock 2024-04-26T18:44:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:36 smithi090 ceph-mon[29180]: pgmap v1104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:36 smithi090 ceph-mon[29180]: 10 slow requests, 1 included below; oldest blocked for > 557.646245 secs 2024-04-26T18:44:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:36 smithi090 ceph-mon[29180]: slow request 184.515258 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently failed to acquire snap lock 2024-04-26T18:44:36.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 961 sec 2024-04-26T18:44:37.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 962 sec 2024-04-26T18:44:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:38 smithi119 ceph-mon[32752]: pgmap v1105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:38 smithi167 ceph-mon[32751]: pgmap v1105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:38 smithi090 ceph-mon[29180]: pgmap v1105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:38.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 963 sec 2024-04-26T18:44:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:44:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:44:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:44:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:44:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:44:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:44:39.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 964 sec 2024-04-26T18:44:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:40 smithi119 ceph-mon[32752]: pgmap v1106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:40 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 561.983092 secs 2024-04-26T18:44:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:40 smithi167 ceph-mon[32751]: pgmap v1106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:40 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 561.983092 secs 2024-04-26T18:44:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:40 smithi090 ceph-mon[29180]: pgmap v1106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:40 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 561.983092 secs 2024-04-26T18:44:40.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 965 sec 2024-04-26T18:44:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 562.646317 secs 2024-04-26T18:44:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 562.646317 secs 2024-04-26T18:44:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 562.646317 secs 2024-04-26T18:44:41.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 966 sec 2024-04-26T18:44:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:42 smithi119 ceph-mon[32752]: pgmap v1107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:42 smithi167 ceph-mon[32751]: pgmap v1107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:42 smithi090 ceph-mon[29180]: pgmap v1107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:42.965 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 967 sec 2024-04-26T18:44:43.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 968 sec 2024-04-26T18:44:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:44 smithi119 ceph-mon[32752]: pgmap v1108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:44 smithi167 ceph-mon[32751]: pgmap v1108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:44 smithi090 ceph-mon[29180]: pgmap v1108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:44.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 969 sec 2024-04-26T18:44:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:45 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 566.983172 secs 2024-04-26T18:44:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:45 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 566.983172 secs 2024-04-26T18:44:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:45 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 566.983172 secs 2024-04-26T18:44:45.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 970 sec 2024-04-26T18:44:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:46 smithi119 ceph-mon[32752]: pgmap v1109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 567.646461 secs 2024-04-26T18:44:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:46 smithi167 ceph-mon[32751]: pgmap v1109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 567.646461 secs 2024-04-26T18:44:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:46 smithi090 ceph-mon[29180]: pgmap v1109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:44:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 567.646461 secs 2024-04-26T18:44:46.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 971 sec 2024-04-26T18:44:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:44:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:44:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:44:47.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 972 sec 2024-04-26T18:44:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:48 smithi119 ceph-mon[32752]: pgmap v1110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:44:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:44:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:44:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:48 smithi167 ceph-mon[32751]: pgmap v1110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:44:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:44:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:44:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:48 smithi090 ceph-mon[29180]: pgmap v1110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:44:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:44:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:44:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:44:48.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 973 sec 2024-04-26T18:44:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:49 smithi119 ceph-mon[32752]: pgmap v1111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T18:44:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:49 smithi119 ceph-mon[32752]: pgmap v1112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T18:44:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:49 smithi167 ceph-mon[32751]: pgmap v1111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T18:44:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:49 smithi167 ceph-mon[32751]: pgmap v1112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T18:44:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:49 smithi090 ceph-mon[29180]: pgmap v1111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T18:44:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:49 smithi090 ceph-mon[29180]: pgmap v1112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T18:44:49.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 974 sec 2024-04-26T18:44:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:50 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 571.983253 secs 2024-04-26T18:44:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:50 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 571.983253 secs 2024-04-26T18:44:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:50 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 571.983253 secs 2024-04-26T18:44:50.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 975 sec 2024-04-26T18:44:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:51 smithi119 ceph-mon[32752]: pgmap v1113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 572.646576 secs 2024-04-26T18:44:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:51 smithi167 ceph-mon[32751]: pgmap v1113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 572.646576 secs 2024-04-26T18:44:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:51 smithi090 ceph-mon[29180]: pgmap v1113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 572.646576 secs 2024-04-26T18:44:51.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 976 sec 2024-04-26T18:44:52.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 977 sec 2024-04-26T18:44:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:53 smithi119 ceph-mon[32752]: pgmap v1114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:53 smithi167 ceph-mon[32751]: pgmap v1114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:53 smithi090 ceph-mon[29180]: pgmap v1114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:53.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 978 sec 2024-04-26T18:44:54.966 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 979 sec 2024-04-26T18:44:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:55 smithi119 ceph-mon[32752]: pgmap v1115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:55 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 576.983338 secs 2024-04-26T18:44:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:55 smithi167 ceph-mon[32751]: pgmap v1115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:55 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 576.983338 secs 2024-04-26T18:44:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:55 smithi090 ceph-mon[29180]: pgmap v1115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:55 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 576.983338 secs 2024-04-26T18:44:55.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 980 sec 2024-04-26T18:44:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 577.646663 secs 2024-04-26T18:44:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 577.646663 secs 2024-04-26T18:44:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 577.646663 secs 2024-04-26T18:44:56.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 981 sec 2024-04-26T18:44:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:57 smithi119 ceph-mon[32752]: pgmap v1116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:57 smithi167 ceph-mon[32751]: pgmap v1116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:57 smithi090 ceph-mon[29180]: pgmap v1116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:44:57.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 982 sec 2024-04-26T18:44:58.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 983 sec 2024-04-26T18:44:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:44:59 smithi119 ceph-mon[32752]: pgmap v1117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:44:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:44:59 smithi167 ceph-mon[32751]: pgmap v1117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:44:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:44:59 smithi090 ceph-mon[29180]: pgmap v1117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:44:59.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 984 sec 2024-04-26T18:45:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:00 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 581.983421 secs 2024-04-26T18:45:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:00 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 581.983421 secs 2024-04-26T18:45:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:00 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 581.983421 secs 2024-04-26T18:45:00.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 985 sec 2024-04-26T18:45:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:01 smithi119 ceph-mon[32752]: pgmap v1118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 582.646741 secs 2024-04-26T18:45:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:01 smithi167 ceph-mon[32751]: pgmap v1118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 582.646741 secs 2024-04-26T18:45:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:01 smithi090 ceph-mon[29180]: pgmap v1118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 582.646741 secs 2024-04-26T18:45:01.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 986 sec 2024-04-26T18:45:02.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 987 sec 2024-04-26T18:45:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:03 smithi119 ceph-mon[32752]: pgmap v1119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:03 smithi167 ceph-mon[32751]: pgmap v1119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:03 smithi090 ceph-mon[29180]: pgmap v1119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:03.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 988 sec 2024-04-26T18:45:04.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 989 sec 2024-04-26T18:45:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:05 smithi119 ceph-mon[32752]: pgmap v1120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:05 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 586.983505 secs 2024-04-26T18:45:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:05 smithi167 ceph-mon[32751]: pgmap v1120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:05 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 586.983505 secs 2024-04-26T18:45:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:05 smithi090 ceph-mon[29180]: pgmap v1120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:05 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 586.983505 secs 2024-04-26T18:45:05.967 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 990 sec 2024-04-26T18:45:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 587.646850 secs 2024-04-26T18:45:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 587.646850 secs 2024-04-26T18:45:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 587.646850 secs 2024-04-26T18:45:06.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 991 sec 2024-04-26T18:45:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:07 smithi167 ceph-mon[32751]: pgmap v1121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:07 smithi090 ceph-mon[29180]: pgmap v1121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:07 smithi119 ceph-mon[32752]: pgmap v1121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:07.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 992 sec 2024-04-26T18:45:08.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 993 sec 2024-04-26T18:45:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:09 smithi167 ceph-mon[32751]: pgmap v1122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:09 smithi090 ceph-mon[29180]: pgmap v1122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:09 smithi119 ceph-mon[32752]: pgmap v1122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:09.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 994 sec 2024-04-26T18:45:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:10 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 591.983581 secs 2024-04-26T18:45:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:10 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 591.983581 secs 2024-04-26T18:45:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:10 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 591.983581 secs 2024-04-26T18:45:10.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 995 sec 2024-04-26T18:45:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:11 smithi167 ceph-mon[32751]: pgmap v1123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 592.646969 secs 2024-04-26T18:45:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:11 smithi090 ceph-mon[29180]: pgmap v1123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 592.646969 secs 2024-04-26T18:45:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:11 smithi119 ceph-mon[32752]: pgmap v1123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 592.646969 secs 2024-04-26T18:45:11.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 996 sec 2024-04-26T18:45:12.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 997 sec 2024-04-26T18:45:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:13 smithi090 ceph-mon[29180]: pgmap v1124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:13 smithi119 ceph-mon[32752]: pgmap v1124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:13 smithi167 ceph-mon[32751]: pgmap v1124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:13.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 998 sec 2024-04-26T18:45:14.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 999 sec 2024-04-26T18:45:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:15 smithi090 ceph-mon[29180]: pgmap v1125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 596.983664 secs 2024-04-26T18:45:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:15 smithi119 ceph-mon[32752]: pgmap v1125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 596.983664 secs 2024-04-26T18:45:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:15 smithi167 ceph-mon[32751]: pgmap v1125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 596.983664 secs 2024-04-26T18:45:15.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1000 sec 2024-04-26T18:45:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 597.647104 secs 2024-04-26T18:45:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 597.647104 secs 2024-04-26T18:45:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 597.647104 secs 2024-04-26T18:45:16.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1001 sec 2024-04-26T18:45:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:17 smithi090 ceph-mon[29180]: pgmap v1126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:17 smithi119 ceph-mon[32752]: pgmap v1126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:17 smithi167 ceph-mon[32751]: pgmap v1126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:17.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1002 sec 2024-04-26T18:45:18.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1003 sec 2024-04-26T18:45:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:19 smithi090 ceph-mon[29180]: pgmap v1127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:19 smithi119 ceph-mon[32752]: pgmap v1127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:19 smithi167 ceph-mon[32751]: pgmap v1127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:19.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1004 sec 2024-04-26T18:45:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:20 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 601.983747 secs 2024-04-26T18:45:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:20 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 601.983747 secs 2024-04-26T18:45:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:20 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 601.983747 secs 2024-04-26T18:45:20.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1005 sec 2024-04-26T18:45:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:21 smithi090 ceph-mon[29180]: pgmap v1128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 602.647177 secs 2024-04-26T18:45:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:21 smithi119 ceph-mon[32752]: pgmap v1128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 602.647177 secs 2024-04-26T18:45:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:21 smithi167 ceph-mon[32751]: pgmap v1128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 602.647177 secs 2024-04-26T18:45:21.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1006 sec 2024-04-26T18:45:22.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1007 sec 2024-04-26T18:45:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:23 smithi090 ceph-mon[29180]: pgmap v1129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:23 smithi119 ceph-mon[32752]: pgmap v1129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:23 smithi167 ceph-mon[32751]: pgmap v1129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:23.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1008 sec 2024-04-26T18:45:24.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1009 sec 2024-04-26T18:45:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:25 smithi090 ceph-mon[29180]: pgmap v1130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:25 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 606.983849 secs 2024-04-26T18:45:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:25 smithi119 ceph-mon[32752]: pgmap v1130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:25 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 606.983849 secs 2024-04-26T18:45:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:25 smithi167 ceph-mon[32751]: pgmap v1130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:25 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 606.983849 secs 2024-04-26T18:45:25.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1010 sec 2024-04-26T18:45:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 607.647262 secs 2024-04-26T18:45:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 607.647262 secs 2024-04-26T18:45:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 607.647262 secs 2024-04-26T18:45:26.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1011 sec 2024-04-26T18:45:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:27 smithi090 ceph-mon[29180]: pgmap v1131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:27 smithi119 ceph-mon[32752]: pgmap v1131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:27 smithi167 ceph-mon[32751]: pgmap v1131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:27.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1012 sec 2024-04-26T18:45:28.969 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1013 sec 2024-04-26T18:45:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:29 smithi090 ceph-mon[29180]: pgmap v1132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:29 smithi119 ceph-mon[32752]: pgmap v1132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:29 smithi167 ceph-mon[32751]: pgmap v1132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:29.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1014 sec 2024-04-26T18:45:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:30 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 611.983934 secs 2024-04-26T18:45:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:30 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 611.983934 secs 2024-04-26T18:45:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:30 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 611.983934 secs 2024-04-26T18:45:30.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1015 sec 2024-04-26T18:45:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:31 smithi090 ceph-mon[29180]: pgmap v1133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 612.647351 secs 2024-04-26T18:45:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:31 smithi167 ceph-mon[32751]: pgmap v1133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 612.647351 secs 2024-04-26T18:45:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:31 smithi119 ceph-mon[32752]: pgmap v1133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 612.647351 secs 2024-04-26T18:45:31.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1016 sec 2024-04-26T18:45:32.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1017 sec 2024-04-26T18:45:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:33 smithi090 ceph-mon[29180]: pgmap v1134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:33 smithi119 ceph-mon[32752]: pgmap v1134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:33 smithi167 ceph-mon[32751]: pgmap v1134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:33.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1018 sec 2024-04-26T18:45:34.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1019 sec 2024-04-26T18:45:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:35 smithi090 ceph-mon[29180]: pgmap v1135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:35 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 616.983994 secs 2024-04-26T18:45:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:35 smithi119 ceph-mon[32752]: pgmap v1135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:35 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 616.983994 secs 2024-04-26T18:45:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:35 smithi167 ceph-mon[32751]: pgmap v1135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:35 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 616.983994 secs 2024-04-26T18:45:35.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1020 sec 2024-04-26T18:45:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 617.647450 secs 2024-04-26T18:45:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 617.647450 secs 2024-04-26T18:45:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 617.647450 secs 2024-04-26T18:45:36.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1021 sec 2024-04-26T18:45:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:37 smithi119 ceph-mon[32752]: pgmap v1136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:37 smithi167 ceph-mon[32751]: pgmap v1136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:37 smithi090 ceph-mon[29180]: pgmap v1136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:37.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1022 sec 2024-04-26T18:45:38.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1023 sec 2024-04-26T18:45:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:39 smithi119 ceph-mon[32752]: pgmap v1137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:39 smithi167 ceph-mon[32751]: pgmap v1137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:39 smithi090 ceph-mon[29180]: pgmap v1137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:39.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1024 sec 2024-04-26T18:45:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:40 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 621.984093 secs 2024-04-26T18:45:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:40 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 621.984093 secs 2024-04-26T18:45:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:40 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 621.984093 secs 2024-04-26T18:45:40.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1025 sec 2024-04-26T18:45:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:41 smithi119 ceph-mon[32752]: pgmap v1138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 622.647530 secs 2024-04-26T18:45:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:41 smithi167 ceph-mon[32751]: pgmap v1138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 622.647530 secs 2024-04-26T18:45:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:41 smithi090 ceph-mon[29180]: pgmap v1138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 622.647530 secs 2024-04-26T18:45:41.970 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1026 sec 2024-04-26T18:45:42.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1027 sec 2024-04-26T18:45:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:43 smithi119 ceph-mon[32752]: pgmap v1139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:43 smithi167 ceph-mon[32751]: pgmap v1139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:43 smithi090 ceph-mon[29180]: pgmap v1139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:43.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1028 sec 2024-04-26T18:45:44.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1029 sec 2024-04-26T18:45:45.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:45 smithi090 ceph-mon[29180]: pgmap v1140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:45.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:45 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 626.984179 secs 2024-04-26T18:45:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:45 smithi119 ceph-mon[32752]: pgmap v1140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:45 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 626.984179 secs 2024-04-26T18:45:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:45 smithi167 ceph-mon[32751]: pgmap v1140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:45:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:45 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 626.984179 secs 2024-04-26T18:45:45.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1030 sec 2024-04-26T18:45:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 627.647592 secs 2024-04-26T18:45:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 627.647592 secs 2024-04-26T18:45:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 627.647592 secs 2024-04-26T18:45:46.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1031 sec 2024-04-26T18:45:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:47 smithi119 ceph-mon[32752]: pgmap v1141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:47 smithi167 ceph-mon[32751]: pgmap v1141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:47 smithi090 ceph-mon[29180]: pgmap v1141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:47.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1032 sec 2024-04-26T18:45:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:45:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:45:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:45:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:45:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:45:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:45:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:45:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:45:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:45:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:45:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:45:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:45:48.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1033 sec 2024-04-26T18:45:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:49 smithi119 ceph-mon[32752]: pgmap v1142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:49 smithi119 ceph-mon[32752]: pgmap v1143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T18:45:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:49 smithi119 ceph-mon[32752]: pgmap v1144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:49 smithi167 ceph-mon[32751]: pgmap v1142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:49 smithi167 ceph-mon[32751]: pgmap v1143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T18:45:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:49 smithi167 ceph-mon[32751]: pgmap v1144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:49 smithi090 ceph-mon[29180]: pgmap v1142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:45:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:49 smithi090 ceph-mon[29180]: pgmap v1143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T18:45:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:49 smithi090 ceph-mon[29180]: pgmap v1144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:49.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1034 sec 2024-04-26T18:45:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:50 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 631.984233 secs 2024-04-26T18:45:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:50 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 631.984233 secs 2024-04-26T18:45:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:50 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 631.984233 secs 2024-04-26T18:45:50.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1035 sec 2024-04-26T18:45:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:51 smithi119 ceph-mon[32752]: pgmap v1145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 632.647671 secs 2024-04-26T18:45:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:51 smithi167 ceph-mon[32751]: pgmap v1145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 632.647671 secs 2024-04-26T18:45:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:51 smithi090 ceph-mon[29180]: pgmap v1145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 632.647671 secs 2024-04-26T18:45:51.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1036 sec 2024-04-26T18:45:52.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1037 sec 2024-04-26T18:45:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:53 smithi119 ceph-mon[32752]: pgmap v1146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:45:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:53 smithi167 ceph-mon[32751]: pgmap v1146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:45:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:53 smithi090 ceph-mon[29180]: pgmap v1146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:45:53.971 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1038 sec 2024-04-26T18:45:54.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1039 sec 2024-04-26T18:45:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:55 smithi119 ceph-mon[32752]: pgmap v1147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:55 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 636.984319 secs 2024-04-26T18:45:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:55 smithi167 ceph-mon[32751]: pgmap v1147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:55 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 636.984319 secs 2024-04-26T18:45:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:55 smithi090 ceph-mon[29180]: pgmap v1147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:55 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 636.984319 secs 2024-04-26T18:45:55.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1040 sec 2024-04-26T18:45:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 637.647805 secs 2024-04-26T18:45:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 637.647805 secs 2024-04-26T18:45:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 637.647805 secs 2024-04-26T18:45:56.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1041 sec 2024-04-26T18:45:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:57 smithi119 ceph-mon[32752]: pgmap v1148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:57 smithi167 ceph-mon[32751]: pgmap v1148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:57 smithi090 ceph-mon[29180]: pgmap v1148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:45:57.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1042 sec 2024-04-26T18:45:58.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1043 sec 2024-04-26T18:45:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:45:59 smithi119 ceph-mon[32752]: pgmap v1149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:45:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:45:59 smithi167 ceph-mon[32751]: pgmap v1149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:45:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:45:59 smithi090 ceph-mon[29180]: pgmap v1149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:45:59.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1044 sec 2024-04-26T18:46:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:00 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 641.984402 secs 2024-04-26T18:46:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:00 smithi119 ceph-mon[32752]: pgmap v1150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:00 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 641.984402 secs 2024-04-26T18:46:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:00 smithi167 ceph-mon[32751]: pgmap v1150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:00 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 641.984402 secs 2024-04-26T18:46:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:00 smithi090 ceph-mon[29180]: pgmap v1150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:00.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1045 sec 2024-04-26T18:46:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 642.647930 secs 2024-04-26T18:46:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 642.647930 secs 2024-04-26T18:46:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 642.647930 secs 2024-04-26T18:46:01.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1046 sec 2024-04-26T18:46:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:02 smithi119 ceph-mon[32752]: pgmap v1151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:02 smithi167 ceph-mon[32751]: pgmap v1151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:02 smithi090 ceph-mon[29180]: pgmap v1151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:02.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1047 sec 2024-04-26T18:46:03.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1048 sec 2024-04-26T18:46:04.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1049 sec 2024-04-26T18:46:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:05 smithi090 ceph-mon[29180]: pgmap v1152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:05 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 646.984487 secs 2024-04-26T18:46:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:05 smithi119 ceph-mon[32752]: pgmap v1152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:05 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 646.984487 secs 2024-04-26T18:46:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:05 smithi167 ceph-mon[32751]: pgmap v1152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:05 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 646.984487 secs 2024-04-26T18:46:05.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1050 sec 2024-04-26T18:46:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 647.648065 secs 2024-04-26T18:46:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 647.648065 secs 2024-04-26T18:46:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 647.648065 secs 2024-04-26T18:46:06.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1051 sec 2024-04-26T18:46:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:07 smithi119 ceph-mon[32752]: pgmap v1153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:07 smithi167 ceph-mon[32751]: pgmap v1153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:07 smithi090 ceph-mon[29180]: pgmap v1153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:07.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1052 sec 2024-04-26T18:46:08.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1053 sec 2024-04-26T18:46:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:09 smithi119 ceph-mon[32752]: pgmap v1154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:09 smithi167 ceph-mon[32751]: pgmap v1154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:09 smithi090 ceph-mon[29180]: pgmap v1154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:09.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1054 sec 2024-04-26T18:46:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:10 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 651.984568 secs 2024-04-26T18:46:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:10 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 651.984568 secs 2024-04-26T18:46:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:10 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 651.984568 secs 2024-04-26T18:46:10.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1055 sec 2024-04-26T18:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:11 smithi119 ceph-mon[32752]: pgmap v1155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 652.648198 secs 2024-04-26T18:46:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:11 smithi167 ceph-mon[32751]: pgmap v1155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 652.648198 secs 2024-04-26T18:46:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:11 smithi090 ceph-mon[29180]: pgmap v1155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 652.648198 secs 2024-04-26T18:46:11.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1056 sec 2024-04-26T18:46:12.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1057 sec 2024-04-26T18:46:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:13 smithi119 ceph-mon[32752]: pgmap v1156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:13 smithi167 ceph-mon[32751]: pgmap v1156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:13 smithi090 ceph-mon[29180]: pgmap v1156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:13.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1058 sec 2024-04-26T18:46:14.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1059 sec 2024-04-26T18:46:15.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:15 smithi090 ceph-mon[29180]: pgmap v1157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:15.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 656.984653 secs 2024-04-26T18:46:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:15 smithi119 ceph-mon[32752]: pgmap v1157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 656.984653 secs 2024-04-26T18:46:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:15 smithi167 ceph-mon[32751]: pgmap v1157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 656.984653 secs 2024-04-26T18:46:15.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1060 sec 2024-04-26T18:46:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 657.648266 secs 2024-04-26T18:46:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 657.648266 secs 2024-04-26T18:46:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 657.648266 secs 2024-04-26T18:46:16.973 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1061 sec 2024-04-26T18:46:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:17 smithi119 ceph-mon[32752]: pgmap v1158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:17 smithi167 ceph-mon[32751]: pgmap v1158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:17 smithi090 ceph-mon[29180]: pgmap v1158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:17.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1062 sec 2024-04-26T18:46:18.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1063 sec 2024-04-26T18:46:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:19 smithi119 ceph-mon[32752]: pgmap v1159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:19 smithi167 ceph-mon[32751]: pgmap v1159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:19 smithi090 ceph-mon[29180]: pgmap v1159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:19.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1064 sec 2024-04-26T18:46:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:20 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 661.984736 secs 2024-04-26T18:46:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:20 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 661.984736 secs 2024-04-26T18:46:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:20 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 661.984736 secs 2024-04-26T18:46:20.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1065 sec 2024-04-26T18:46:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:21 smithi119 ceph-mon[32752]: pgmap v1160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 662.648427 secs 2024-04-26T18:46:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:21 smithi167 ceph-mon[32751]: pgmap v1160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 662.648427 secs 2024-04-26T18:46:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:21 smithi090 ceph-mon[29180]: pgmap v1160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 662.648427 secs 2024-04-26T18:46:21.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1066 sec 2024-04-26T18:46:22.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1067 sec 2024-04-26T18:46:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:23 smithi119 ceph-mon[32752]: pgmap v1161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:23 smithi167 ceph-mon[32751]: pgmap v1161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:23 smithi090 ceph-mon[29180]: pgmap v1161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:23.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1068 sec 2024-04-26T18:46:24.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1069 sec 2024-04-26T18:46:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:25 smithi119 ceph-mon[32752]: pgmap v1162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:25 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 666.984825 secs 2024-04-26T18:46:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:25 smithi167 ceph-mon[32751]: pgmap v1162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:25 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 666.984825 secs 2024-04-26T18:46:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:25 smithi090 ceph-mon[29180]: pgmap v1162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:25 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 666.984825 secs 2024-04-26T18:46:25.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1070 sec 2024-04-26T18:46:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 667.648465 secs 2024-04-26T18:46:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 667.648465 secs 2024-04-26T18:46:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 667.648465 secs 2024-04-26T18:46:26.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1071 sec 2024-04-26T18:46:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:27 smithi119 ceph-mon[32752]: pgmap v1163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:27 smithi167 ceph-mon[32751]: pgmap v1163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:27 smithi090 ceph-mon[29180]: pgmap v1163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:27.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1072 sec 2024-04-26T18:46:28.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1073 sec 2024-04-26T18:46:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:29 smithi119 ceph-mon[32752]: pgmap v1164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:29 smithi167 ceph-mon[32751]: pgmap v1164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:29 smithi090 ceph-mon[29180]: pgmap v1164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:29.974 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1074 sec 2024-04-26T18:46:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:30 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 671.984903 secs 2024-04-26T18:46:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:30 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 671.984903 secs 2024-04-26T18:46:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:30 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 671.984903 secs 2024-04-26T18:46:30.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1075 sec 2024-04-26T18:46:31.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:46:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:46:30.929+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:46:31.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:46:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:46:30.929+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:46:31.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:46:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:46:30.929+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:46:31.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:46:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:46:30.929+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:46:31.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:46:30 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:46:30.929+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:46:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:31 smithi167 ceph-mon[32751]: pgmap v1165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 672.648596 secs 2024-04-26T18:46:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:31 smithi119 ceph-mon[32752]: pgmap v1165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 672.648596 secs 2024-04-26T18:46:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:31 smithi090 ceph-mon[29180]: pgmap v1165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 672.648596 secs 2024-04-26T18:46:31.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1076 sec 2024-04-26T18:46:32.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1077 sec 2024-04-26T18:46:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:33 smithi119 ceph-mon[32752]: pgmap v1166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:33 smithi167 ceph-mon[32751]: pgmap v1166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:33 smithi090 ceph-mon[29180]: pgmap v1166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:33.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1078 sec 2024-04-26T18:46:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:34 smithi119 ceph-mon[32752]: pgmap v1167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:34 smithi167 ceph-mon[32751]: pgmap v1167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:34 smithi090 ceph-mon[29180]: pgmap v1167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:34.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1079 sec 2024-04-26T18:46:35.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:35 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 676.984963 secs 2024-04-26T18:46:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:35 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 676.984963 secs 2024-04-26T18:46:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:35 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 676.984963 secs 2024-04-26T18:46:35.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1080 sec 2024-04-26T18:46:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 677.648662 secs 2024-04-26T18:46:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:36 smithi119 ceph-mon[32752]: pgmap v1168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 677.648662 secs 2024-04-26T18:46:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:36 smithi167 ceph-mon[32751]: pgmap v1168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 677.648662 secs 2024-04-26T18:46:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:36 smithi090 ceph-mon[29180]: pgmap v1168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:36.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1081 sec 2024-04-26T18:46:37.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1082 sec 2024-04-26T18:46:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:46:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:46:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:38 smithi167 ceph-mon[32751]: pgmap v1169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:46:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:46:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:38 smithi090 ceph-mon[29180]: pgmap v1169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:38.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1083 sec 2024-04-26T18:46:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:46:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:46:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:38 smithi119 ceph-mon[32752]: pgmap v1169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:39.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1084 sec 2024-04-26T18:46:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:40 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 681.985049 secs 2024-04-26T18:46:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:40 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 681.985049 secs 2024-04-26T18:46:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:40 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 681.985049 secs 2024-04-26T18:46:40.975 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1085 sec 2024-04-26T18:46:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:41 smithi090 ceph-mon[29180]: pgmap v1170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 682.648779 secs 2024-04-26T18:46:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:41 smithi119 ceph-mon[32752]: pgmap v1170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 682.648779 secs 2024-04-26T18:46:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:41 smithi167 ceph-mon[32751]: pgmap v1170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 682.648779 secs 2024-04-26T18:46:41.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1086 sec 2024-04-26T18:46:42.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1087 sec 2024-04-26T18:46:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:43 smithi119 ceph-mon[32752]: pgmap v1171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:43 smithi167 ceph-mon[32751]: pgmap v1171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:43 smithi090 ceph-mon[29180]: pgmap v1171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:43.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1088 sec 2024-04-26T18:46:44.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1089 sec 2024-04-26T18:46:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:45 smithi119 ceph-mon[32752]: pgmap v1172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:45 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 686.985135 secs 2024-04-26T18:46:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:45 smithi167 ceph-mon[32751]: pgmap v1172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:45 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 686.985135 secs 2024-04-26T18:46:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:45 smithi090 ceph-mon[29180]: pgmap v1172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T18:46:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:45 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 686.985135 secs 2024-04-26T18:46:45.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1090 sec 2024-04-26T18:46:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 687.648921 secs 2024-04-26T18:46:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 687.648921 secs 2024-04-26T18:46:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 687.648921 secs 2024-04-26T18:46:46.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1091 sec 2024-04-26T18:46:47.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:47 smithi090 ceph-mon[29180]: pgmap v1173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:47 smithi119 ceph-mon[32752]: pgmap v1173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:47 smithi167 ceph-mon[32751]: pgmap v1173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:46:47.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1092 sec 2024-04-26T18:46:48.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1093 sec 2024-04-26T18:46:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:49 smithi119 ceph-mon[32752]: pgmap v1174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:46:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:46:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:46:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:46:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:49 smithi167 ceph-mon[32751]: pgmap v1174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:46:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:46:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:46:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:46:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:49 smithi090 ceph-mon[29180]: pgmap v1174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:46:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:46:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:46:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:46:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:46:49.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1094 sec 2024-04-26T18:46:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:50 smithi119 ceph-mon[32752]: pgmap v1175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T18:46:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:50 smithi119 ceph-mon[32752]: pgmap v1176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:50 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 691.985209 secs 2024-04-26T18:46:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:50 smithi167 ceph-mon[32751]: pgmap v1175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T18:46:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:50 smithi167 ceph-mon[32751]: pgmap v1176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:50 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 691.985209 secs 2024-04-26T18:46:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:50 smithi090 ceph-mon[29180]: pgmap v1175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T18:46:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:50 smithi090 ceph-mon[29180]: pgmap v1176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:50 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 691.985209 secs 2024-04-26T18:46:50.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1095 sec 2024-04-26T18:46:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 692.648997 secs 2024-04-26T18:46:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 692.648997 secs 2024-04-26T18:46:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 692.648997 secs 2024-04-26T18:46:51.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1096 sec 2024-04-26T18:46:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:52 smithi119 ceph-mon[32752]: pgmap v1177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:52 smithi167 ceph-mon[32751]: pgmap v1177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:52 smithi090 ceph-mon[29180]: pgmap v1177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:52.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1097 sec 2024-04-26T18:46:53.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1098 sec 2024-04-26T18:46:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:54 smithi119 ceph-mon[32752]: pgmap v1178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T18:46:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:54 smithi167 ceph-mon[32751]: pgmap v1178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T18:46:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:54 smithi090 ceph-mon[29180]: pgmap v1178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T18:46:54.976 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1099 sec 2024-04-26T18:46:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:55 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 696.985290 secs 2024-04-26T18:46:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:55 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 696.985290 secs 2024-04-26T18:46:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:55 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 696.985290 secs 2024-04-26T18:46:55.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1100 sec 2024-04-26T18:46:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:56 smithi119 ceph-mon[32752]: pgmap v1179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 697.649107 secs 2024-04-26T18:46:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:56 smithi167 ceph-mon[32751]: pgmap v1179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 697.649107 secs 2024-04-26T18:46:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:56 smithi090 ceph-mon[29180]: pgmap v1179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:46:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 697.649107 secs 2024-04-26T18:46:56.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1101 sec 2024-04-26T18:46:57.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1102 sec 2024-04-26T18:46:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:46:58 smithi119 ceph-mon[32752]: pgmap v1180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T18:46:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:46:58 smithi167 ceph-mon[32751]: pgmap v1180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T18:46:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:46:58 smithi090 ceph-mon[29180]: pgmap v1180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T18:46:58.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1103 sec 2024-04-26T18:46:59.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1104 sec 2024-04-26T18:47:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:00 smithi119 ceph-mon[32752]: pgmap v1181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:47:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:00 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 701.985368 secs 2024-04-26T18:47:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:00 smithi167 ceph-mon[32751]: pgmap v1181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:47:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:00 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 701.985368 secs 2024-04-26T18:47:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:00 smithi090 ceph-mon[29180]: pgmap v1181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:47:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:00 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 701.985368 secs 2024-04-26T18:47:00.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1105 sec 2024-04-26T18:47:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 702.649207 secs 2024-04-26T18:47:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 702.649207 secs 2024-04-26T18:47:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 702.649207 secs 2024-04-26T18:47:01.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1106 sec 2024-04-26T18:47:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:02 smithi119 ceph-mon[32752]: pgmap v1182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:02 smithi167 ceph-mon[32751]: pgmap v1182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:02 smithi090 ceph-mon[29180]: pgmap v1182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:02.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1107 sec 2024-04-26T18:47:03.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1108 sec 2024-04-26T18:47:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:03 smithi090 ceph-mon[29180]: pgmap v1183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:03 smithi119 ceph-mon[32752]: pgmap v1183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:03 smithi167 ceph-mon[32751]: pgmap v1183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:04.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1109 sec 2024-04-26T18:47:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:05 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 706.985442 secs 2024-04-26T18:47:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:05 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 706.985442 secs 2024-04-26T18:47:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:05 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 706.985442 secs 2024-04-26T18:47:05.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1110 sec 2024-04-26T18:47:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:06 smithi090 ceph-mon[29180]: pgmap v1184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 707.649325 secs 2024-04-26T18:47:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:06 smithi119 ceph-mon[32752]: pgmap v1184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 707.649325 secs 2024-04-26T18:47:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:06 smithi167 ceph-mon[32751]: pgmap v1184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 707.649325 secs 2024-04-26T18:47:06.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1111 sec 2024-04-26T18:47:07.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1112 sec 2024-04-26T18:47:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:08 smithi119 ceph-mon[32752]: pgmap v1185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:08 smithi167 ceph-mon[32751]: pgmap v1185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:08 smithi090 ceph-mon[29180]: pgmap v1185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:08.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1113 sec 2024-04-26T18:47:09.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1114 sec 2024-04-26T18:47:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:10 smithi119 ceph-mon[32752]: pgmap v1186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:10 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 711.985524 secs 2024-04-26T18:47:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:10 smithi167 ceph-mon[32751]: pgmap v1186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:10 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 711.985524 secs 2024-04-26T18:47:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:10 smithi090 ceph-mon[29180]: pgmap v1186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:10 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 711.985524 secs 2024-04-26T18:47:10.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1115 sec 2024-04-26T18:47:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 712.649410 secs 2024-04-26T18:47:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 712.649410 secs 2024-04-26T18:47:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 712.649410 secs 2024-04-26T18:47:11.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1116 sec 2024-04-26T18:47:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:12 smithi119 ceph-mon[32752]: pgmap v1187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:12 smithi167 ceph-mon[32751]: pgmap v1187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:12 smithi090 ceph-mon[29180]: pgmap v1187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:12.977 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1117 sec 2024-04-26T18:47:13.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1118 sec 2024-04-26T18:47:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:14 smithi119 ceph-mon[32752]: pgmap v1188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:14 smithi167 ceph-mon[32751]: pgmap v1188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:14 smithi090 ceph-mon[29180]: pgmap v1188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:14.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1119 sec 2024-04-26T18:47:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:15 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 716.985607 secs 2024-04-26T18:47:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:15 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 716.985607 secs 2024-04-26T18:47:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:15 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 716.985607 secs 2024-04-26T18:47:15.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1120 sec 2024-04-26T18:47:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:16 smithi119 ceph-mon[32752]: pgmap v1189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 717.649549 secs 2024-04-26T18:47:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:16 smithi167 ceph-mon[32751]: pgmap v1189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 717.649549 secs 2024-04-26T18:47:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:16 smithi090 ceph-mon[29180]: pgmap v1189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 717.649549 secs 2024-04-26T18:47:16.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1121 sec 2024-04-26T18:47:17.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1122 sec 2024-04-26T18:47:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:18 smithi119 ceph-mon[32752]: pgmap v1190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:18 smithi167 ceph-mon[32751]: pgmap v1190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:18 smithi090 ceph-mon[29180]: pgmap v1190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:18.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1123 sec 2024-04-26T18:47:19.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1124 sec 2024-04-26T18:47:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:20 smithi119 ceph-mon[32752]: pgmap v1191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:20 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 721.985687 secs 2024-04-26T18:47:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:20 smithi167 ceph-mon[32751]: pgmap v1191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:20 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 721.985687 secs 2024-04-26T18:47:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:20 smithi090 ceph-mon[29180]: pgmap v1191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:20 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 721.985687 secs 2024-04-26T18:47:20.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1125 sec 2024-04-26T18:47:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:21 smithi119 ceph-mon[32752]: 10 slow requests, 5 included below; oldest blocked for > 722.649667 secs 2024-04-26T18:47:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:21 smithi119 ceph-mon[32752]: slow request 722.649666 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:21 smithi119 ceph-mon[32752]: slow request 722.648105 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:21 smithi119 ceph-mon[32752]: slow request 722.646636 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:21 smithi119 ceph-mon[32752]: slow request 722.646166 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:21 smithi119 ceph-mon[32752]: slow request 722.633901 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:21 smithi167 ceph-mon[32751]: 10 slow requests, 5 included below; oldest blocked for > 722.649667 secs 2024-04-26T18:47:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:21 smithi167 ceph-mon[32751]: slow request 722.649666 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:21 smithi167 ceph-mon[32751]: slow request 722.648105 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:21 smithi167 ceph-mon[32751]: slow request 722.646636 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:21 smithi167 ceph-mon[32751]: slow request 722.646166 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:21 smithi167 ceph-mon[32751]: slow request 722.633901 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:21 smithi090 ceph-mon[29180]: 10 slow requests, 5 included below; oldest blocked for > 722.649667 secs 2024-04-26T18:47:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:21 smithi090 ceph-mon[29180]: slow request 722.649666 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:21 smithi090 ceph-mon[29180]: slow request 722.648105 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:21 smithi090 ceph-mon[29180]: slow request 722.646636 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:21 smithi090 ceph-mon[29180]: slow request 722.646166 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:21 smithi090 ceph-mon[29180]: slow request 722.633901 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:47:21.978 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1126 sec 2024-04-26T18:47:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:22 smithi119 ceph-mon[32752]: pgmap v1192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:22 smithi167 ceph-mon[32751]: pgmap v1192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:22 smithi090 ceph-mon[29180]: pgmap v1192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:22.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1127 sec 2024-04-26T18:47:23.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1128 sec 2024-04-26T18:47:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:24 smithi119 ceph-mon[32752]: pgmap v1193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:24 smithi167 ceph-mon[32751]: pgmap v1193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:24 smithi090 ceph-mon[29180]: pgmap v1193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:24.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1129 sec 2024-04-26T18:47:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:25 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 726.985768 secs 2024-04-26T18:47:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:25 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 726.985768 secs 2024-04-26T18:47:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:25 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 726.985768 secs 2024-04-26T18:47:25.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1130 sec 2024-04-26T18:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:26 smithi119 ceph-mon[32752]: pgmap v1194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:26 smithi119 ceph-mon[32752]: 10 slow requests, 4 included below; oldest blocked for > 727.649797 secs 2024-04-26T18:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:26 smithi119 ceph-mon[32752]: slow request 724.065160 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:26 smithi119 ceph-mon[32752]: slow request 724.064824 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:26 smithi119 ceph-mon[32752]: slow request 723.955989 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:26 smithi119 ceph-mon[32752]: slow request 723.951701 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:47:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:26 smithi167 ceph-mon[32751]: pgmap v1194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:26 smithi167 ceph-mon[32751]: 10 slow requests, 4 included below; oldest blocked for > 727.649797 secs 2024-04-26T18:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:26 smithi167 ceph-mon[32751]: slow request 724.065160 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:26 smithi167 ceph-mon[32751]: slow request 724.064824 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:26 smithi167 ceph-mon[32751]: slow request 723.955989 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:26 smithi167 ceph-mon[32751]: slow request 723.951701 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:26 smithi090 ceph-mon[29180]: pgmap v1194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:26 smithi090 ceph-mon[29180]: 10 slow requests, 4 included below; oldest blocked for > 727.649797 secs 2024-04-26T18:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:26 smithi090 ceph-mon[29180]: slow request 724.065160 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:26 smithi090 ceph-mon[29180]: slow request 724.064824 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:26 smithi090 ceph-mon[29180]: slow request 723.955989 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:26 smithi090 ceph-mon[29180]: slow request 723.951701 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:47:26.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1131 sec 2024-04-26T18:47:27.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1132 sec 2024-04-26T18:47:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:28 smithi119 ceph-mon[32752]: pgmap v1195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:28.557 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:47:28.557+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006120 con 0x7f2cd4104500 2024-04-26T18:47:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:47:28.558+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2cbc084070 con 0x7f2cd4104500 2024-04-26T18:47:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:28 smithi167 ceph-mon[32751]: pgmap v1195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:47:28.573+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006000 con 0x56302893f0e0 2024-04-26T18:47:28.573 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:47:28.573+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 10 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006000 con 0x56302893f0e0 2024-04-26T18:47:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:28 smithi090 ceph-mon[29180]: pgmap v1195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:28.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1133 sec 2024-04-26T18:47:29.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1134 sec 2024-04-26T18:47:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:30 smithi119 ceph-mon[32752]: pgmap v1196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:30 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 731.985846 secs 2024-04-26T18:47:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:30 smithi167 ceph-mon[32751]: pgmap v1196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:30 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 731.985846 secs 2024-04-26T18:47:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:30 smithi090 ceph-mon[29180]: pgmap v1196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:30 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 731.985846 secs 2024-04-26T18:47:30.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1135 sec 2024-04-26T18:47:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 732.649906 secs 2024-04-26T18:47:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 732.649906 secs 2024-04-26T18:47:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 732.649906 secs 2024-04-26T18:47:31.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1136 sec 2024-04-26T18:47:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:32 smithi119 ceph-mon[32752]: pgmap v1197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:32 smithi167 ceph-mon[32751]: pgmap v1197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:32 smithi090 ceph-mon[29180]: pgmap v1197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:32.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1137 sec 2024-04-26T18:47:33.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1138 sec 2024-04-26T18:47:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:34 smithi119 ceph-mon[32752]: pgmap v1198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:34 smithi167 ceph-mon[32751]: pgmap v1198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:34 smithi090 ceph-mon[29180]: pgmap v1198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:34.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1139 sec 2024-04-26T18:47:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:35 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 736.985931 secs 2024-04-26T18:47:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:35 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 736.985931 secs 2024-04-26T18:47:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:35 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 736.985931 secs 2024-04-26T18:47:35.979 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1140 sec 2024-04-26T18:47:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:36 smithi119 ceph-mon[32752]: pgmap v1199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:36 smithi119 ceph-mon[32752]: 10 slow requests, 1 included below; oldest blocked for > 737.650016 secs 2024-04-26T18:47:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:36 smithi119 ceph-mon[32752]: slow request 364.519029 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:47:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:36 smithi167 ceph-mon[32751]: pgmap v1199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:36 smithi167 ceph-mon[32751]: 10 slow requests, 1 included below; oldest blocked for > 737.650016 secs 2024-04-26T18:47:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:36 smithi167 ceph-mon[32751]: slow request 364.519029 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:47:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:36 smithi090 ceph-mon[29180]: pgmap v1199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:36 smithi090 ceph-mon[29180]: 10 slow requests, 1 included below; oldest blocked for > 737.650016 secs 2024-04-26T18:47:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:36 smithi090 ceph-mon[29180]: slow request 364.519029 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:47:36.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1141 sec 2024-04-26T18:47:37.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1142 sec 2024-04-26T18:47:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:38 smithi119 ceph-mon[32752]: pgmap v1200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:38 smithi167 ceph-mon[32751]: pgmap v1200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:38 smithi090 ceph-mon[29180]: pgmap v1200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:38.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1143 sec 2024-04-26T18:47:39.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1144 sec 2024-04-26T18:47:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:40 smithi119 ceph-mon[32752]: pgmap v1201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:40 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 741.985989 secs 2024-04-26T18:47:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:40 smithi167 ceph-mon[32751]: pgmap v1201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:40 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 741.985989 secs 2024-04-26T18:47:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:40 smithi090 ceph-mon[29180]: pgmap v1201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:40 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 741.985989 secs 2024-04-26T18:47:40.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1145 sec 2024-04-26T18:47:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 742.650153 secs 2024-04-26T18:47:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 742.650153 secs 2024-04-26T18:47:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 742.650153 secs 2024-04-26T18:47:41.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1146 sec 2024-04-26T18:47:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:42 smithi119 ceph-mon[32752]: pgmap v1202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:42 smithi167 ceph-mon[32751]: pgmap v1202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:42 smithi090 ceph-mon[29180]: pgmap v1202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:42.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1147 sec 2024-04-26T18:47:43.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1148 sec 2024-04-26T18:47:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:44 smithi119 ceph-mon[32752]: pgmap v1203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:44 smithi167 ceph-mon[32751]: pgmap v1203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:44 smithi090 ceph-mon[29180]: pgmap v1203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:44.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1149 sec 2024-04-26T18:47:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:45 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 746.986091 secs 2024-04-26T18:47:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:45 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 746.986091 secs 2024-04-26T18:47:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:45 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 746.986091 secs 2024-04-26T18:47:45.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1150 sec 2024-04-26T18:47:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:46 smithi119 ceph-mon[32752]: pgmap v1204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 747.650267 secs 2024-04-26T18:47:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:46 smithi167 ceph-mon[32751]: pgmap v1204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 747.650267 secs 2024-04-26T18:47:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:46 smithi090 ceph-mon[29180]: pgmap v1204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 747.650267 secs 2024-04-26T18:47:46.980 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1151 sec 2024-04-26T18:47:47.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1152 sec 2024-04-26T18:47:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:48 smithi119 ceph-mon[32752]: pgmap v1205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:48 smithi167 ceph-mon[32751]: pgmap v1205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:48 smithi090 ceph-mon[29180]: pgmap v1205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:47:48.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1153 sec 2024-04-26T18:47:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:47:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:47:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:47:49.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1154 sec 2024-04-26T18:47:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:50 smithi119 ceph-mon[32752]: pgmap v1206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:50 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 751.986158 secs 2024-04-26T18:47:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:47:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:47:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:47:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:50 smithi167 ceph-mon[32751]: pgmap v1206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:50 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 751.986158 secs 2024-04-26T18:47:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:47:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:47:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:47:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:50 smithi090 ceph-mon[29180]: pgmap v1206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:47:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:50 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 751.986158 secs 2024-04-26T18:47:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:47:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:47:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:47:50.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1155 sec 2024-04-26T18:47:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:51 smithi119 ceph-mon[32752]: pgmap v1207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:47:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:51 smithi119 ceph-mon[32752]: pgmap v1208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 752.650358 secs 2024-04-26T18:47:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:51 smithi167 ceph-mon[32751]: pgmap v1207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:47:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:51 smithi167 ceph-mon[32751]: pgmap v1208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 752.650358 secs 2024-04-26T18:47:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:51 smithi090 ceph-mon[29180]: pgmap v1207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:47:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:51 smithi090 ceph-mon[29180]: pgmap v1208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 752.650358 secs 2024-04-26T18:47:51.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1156 sec 2024-04-26T18:47:52.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1157 sec 2024-04-26T18:47:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:53 smithi119 ceph-mon[32752]: pgmap v1209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:53 smithi167 ceph-mon[32751]: pgmap v1209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:53 smithi090 ceph-mon[29180]: pgmap v1209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:53.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1158 sec 2024-04-26T18:47:54.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1159 sec 2024-04-26T18:47:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:55 smithi167 ceph-mon[32751]: pgmap v1210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:55 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 756.986243 secs 2024-04-26T18:47:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:55 smithi090 ceph-mon[29180]: pgmap v1210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:55 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 756.986243 secs 2024-04-26T18:47:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:55 smithi119 ceph-mon[32752]: pgmap v1210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:55 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 756.986243 secs 2024-04-26T18:47:55.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1160 sec 2024-04-26T18:47:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 757.650511 secs 2024-04-26T18:47:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 757.650511 secs 2024-04-26T18:47:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 757.650511 secs 2024-04-26T18:47:56.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1161 sec 2024-04-26T18:47:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:57 smithi167 ceph-mon[32751]: pgmap v1211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:57 smithi090 ceph-mon[29180]: pgmap v1211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:57 smithi119 ceph-mon[32752]: pgmap v1211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:47:57.981 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1162 sec 2024-04-26T18:47:58.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1163 sec 2024-04-26T18:47:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:47:59 smithi167 ceph-mon[32751]: pgmap v1212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:47:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:47:59 smithi090 ceph-mon[29180]: pgmap v1212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:47:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:47:59 smithi119 ceph-mon[32752]: pgmap v1212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:47:59.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1164 sec 2024-04-26T18:48:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:00 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 761.986325 secs 2024-04-26T18:48:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:00 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 761.986325 secs 2024-04-26T18:48:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:00 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 761.986325 secs 2024-04-26T18:48:00.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1165 sec 2024-04-26T18:48:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:01 smithi090 ceph-mon[29180]: pgmap v1213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:48:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 762.650582 secs 2024-04-26T18:48:01.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:01 smithi167 ceph-mon[32751]: pgmap v1213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:48:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 762.650582 secs 2024-04-26T18:48:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:01 smithi119 ceph-mon[32752]: pgmap v1213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:48:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 762.650582 secs 2024-04-26T18:48:01.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1166 sec 2024-04-26T18:48:02.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1167 sec 2024-04-26T18:48:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:03 smithi090 ceph-mon[29180]: pgmap v1214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:03 smithi119 ceph-mon[32752]: pgmap v1214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:03 smithi167 ceph-mon[32751]: pgmap v1214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:03.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1168 sec 2024-04-26T18:48:04.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1169 sec 2024-04-26T18:48:05.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:05 smithi119 ceph-mon[32752]: pgmap v1215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:05.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:05 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 766.986387 secs 2024-04-26T18:48:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:05 smithi090 ceph-mon[29180]: pgmap v1215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:05 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 766.986387 secs 2024-04-26T18:48:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:05 smithi167 ceph-mon[32751]: pgmap v1215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:05 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 766.986387 secs 2024-04-26T18:48:05.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1170 sec 2024-04-26T18:48:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 767.650700 secs 2024-04-26T18:48:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 767.650700 secs 2024-04-26T18:48:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 767.650700 secs 2024-04-26T18:48:06.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1171 sec 2024-04-26T18:48:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:07 smithi090 ceph-mon[29180]: pgmap v1216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:07 smithi119 ceph-mon[32752]: pgmap v1216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:07 smithi167 ceph-mon[32751]: pgmap v1216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:07.982 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1172 sec 2024-04-26T18:48:08.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1173 sec 2024-04-26T18:48:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:09 smithi090 ceph-mon[29180]: pgmap v1217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:09 smithi119 ceph-mon[32752]: pgmap v1217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:09 smithi167 ceph-mon[32751]: pgmap v1217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:09.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1174 sec 2024-04-26T18:48:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:10 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 771.986443 secs 2024-04-26T18:48:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:10 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 771.986443 secs 2024-04-26T18:48:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:10 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 771.986443 secs 2024-04-26T18:48:10.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1175 sec 2024-04-26T18:48:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:11 smithi090 ceph-mon[29180]: pgmap v1218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 772.650818 secs 2024-04-26T18:48:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:11 smithi119 ceph-mon[32752]: pgmap v1218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 772.650818 secs 2024-04-26T18:48:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:11 smithi167 ceph-mon[32751]: pgmap v1218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 772.650818 secs 2024-04-26T18:48:11.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1176 sec 2024-04-26T18:48:12.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1177 sec 2024-04-26T18:48:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:13 smithi090 ceph-mon[29180]: pgmap v1219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:13 smithi119 ceph-mon[32752]: pgmap v1219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:13 smithi167 ceph-mon[32751]: pgmap v1219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:13.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1178 sec 2024-04-26T18:48:14.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1179 sec 2024-04-26T18:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:15 smithi090 ceph-mon[29180]: pgmap v1220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:15 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 776.986529 secs 2024-04-26T18:48:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:15 smithi119 ceph-mon[32752]: pgmap v1220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:15 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 776.986529 secs 2024-04-26T18:48:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:15 smithi167 ceph-mon[32751]: pgmap v1220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:15 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 776.986529 secs 2024-04-26T18:48:15.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1180 sec 2024-04-26T18:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 777.650892 secs 2024-04-26T18:48:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 777.650892 secs 2024-04-26T18:48:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 777.650892 secs 2024-04-26T18:48:16.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1181 sec 2024-04-26T18:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:17 smithi090 ceph-mon[29180]: pgmap v1221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:17 smithi119 ceph-mon[32752]: pgmap v1221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:17 smithi167 ceph-mon[32751]: pgmap v1221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:17.983 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1182 sec 2024-04-26T18:48:18.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1183 sec 2024-04-26T18:48:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:19 smithi090 ceph-mon[29180]: pgmap v1222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:19 smithi119 ceph-mon[32752]: pgmap v1222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:19 smithi167 ceph-mon[32751]: pgmap v1222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:19.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1184 sec 2024-04-26T18:48:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:20 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 781.986615 secs 2024-04-26T18:48:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:20 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 781.986615 secs 2024-04-26T18:48:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:20 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 781.986615 secs 2024-04-26T18:48:20.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1185 sec 2024-04-26T18:48:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:21 smithi090 ceph-mon[29180]: pgmap v1223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 782.651030 secs 2024-04-26T18:48:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:21 smithi119 ceph-mon[32752]: pgmap v1223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 782.651030 secs 2024-04-26T18:48:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:21 smithi167 ceph-mon[32751]: pgmap v1223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 782.651030 secs 2024-04-26T18:48:21.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1186 sec 2024-04-26T18:48:22.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1187 sec 2024-04-26T18:48:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:23 smithi090 ceph-mon[29180]: pgmap v1224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:23 smithi119 ceph-mon[32752]: pgmap v1224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:23 smithi167 ceph-mon[32751]: pgmap v1224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:23.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1188 sec 2024-04-26T18:48:24.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1189 sec 2024-04-26T18:48:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:25 smithi119 ceph-mon[32752]: pgmap v1225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:25 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 786.986701 secs 2024-04-26T18:48:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:25 smithi167 ceph-mon[32751]: pgmap v1225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:25 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 786.986701 secs 2024-04-26T18:48:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:25 smithi090 ceph-mon[29180]: pgmap v1225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:25 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 786.986701 secs 2024-04-26T18:48:25.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1190 sec 2024-04-26T18:48:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 787.651199 secs 2024-04-26T18:48:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 787.651199 secs 2024-04-26T18:48:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 787.651199 secs 2024-04-26T18:48:26.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1191 sec 2024-04-26T18:48:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:27 smithi119 ceph-mon[32752]: pgmap v1226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:27 smithi167 ceph-mon[32751]: pgmap v1226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:27 smithi090 ceph-mon[29180]: pgmap v1226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:27.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1192 sec 2024-04-26T18:48:28.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1193 sec 2024-04-26T18:48:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:29 smithi119 ceph-mon[32752]: pgmap v1227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:29 smithi167 ceph-mon[32751]: pgmap v1227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:29 smithi090 ceph-mon[29180]: pgmap v1227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:29.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1194 sec 2024-04-26T18:48:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:30 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 791.986783 secs 2024-04-26T18:48:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:30 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 791.986783 secs 2024-04-26T18:48:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:30 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 791.986783 secs 2024-04-26T18:48:30.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1195 sec 2024-04-26T18:48:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:31 smithi167 ceph-mon[32751]: pgmap v1228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 792.651339 secs 2024-04-26T18:48:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:31 smithi119 ceph-mon[32752]: pgmap v1228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 792.651339 secs 2024-04-26T18:48:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:31 smithi090 ceph-mon[29180]: pgmap v1228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 792.651339 secs 2024-04-26T18:48:31.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1196 sec 2024-04-26T18:48:32.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1197 sec 2024-04-26T18:48:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:33 smithi119 ceph-mon[32752]: pgmap v1229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:33 smithi167 ceph-mon[32751]: pgmap v1229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:33 smithi090 ceph-mon[29180]: pgmap v1229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:33.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1198 sec 2024-04-26T18:48:34.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1199 sec 2024-04-26T18:48:35.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:35 smithi090 ceph-mon[29180]: pgmap v1230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:35.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:35 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 796.986867 secs 2024-04-26T18:48:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:35 smithi119 ceph-mon[32752]: pgmap v1230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:35 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 796.986867 secs 2024-04-26T18:48:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:35 smithi167 ceph-mon[32751]: pgmap v1230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:35 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 796.986867 secs 2024-04-26T18:48:35.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1200 sec 2024-04-26T18:48:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 797.651395 secs 2024-04-26T18:48:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 797.651395 secs 2024-04-26T18:48:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 797.651395 secs 2024-04-26T18:48:36.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1201 sec 2024-04-26T18:48:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:37 smithi119 ceph-mon[32752]: pgmap v1231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:37 smithi167 ceph-mon[32751]: pgmap v1231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:37 smithi090 ceph-mon[29180]: pgmap v1231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:37.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1202 sec 2024-04-26T18:48:38.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1203 sec 2024-04-26T18:48:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:38 smithi119 ceph-mon[32752]: pgmap v1232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:38 smithi167 ceph-mon[32751]: pgmap v1232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:38 smithi090 ceph-mon[29180]: pgmap v1232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:39.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1204 sec 2024-04-26T18:48:40.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1205 sec 2024-04-26T18:48:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:40 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 801.986927 secs 2024-04-26T18:48:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:40 smithi119 ceph-mon[32752]: pgmap v1233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 802.651517 secs 2024-04-26T18:48:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:40 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 801.986927 secs 2024-04-26T18:48:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:40 smithi167 ceph-mon[32751]: pgmap v1233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 802.651517 secs 2024-04-26T18:48:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:40 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 801.986927 secs 2024-04-26T18:48:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:40 smithi090 ceph-mon[29180]: pgmap v1233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 802.651517 secs 2024-04-26T18:48:41.985 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1206 sec 2024-04-26T18:48:42.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1207 sec 2024-04-26T18:48:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:42 smithi119 ceph-mon[32752]: pgmap v1234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:42 smithi167 ceph-mon[32751]: pgmap v1234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:42 smithi090 ceph-mon[29180]: pgmap v1234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:43.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1208 sec 2024-04-26T18:48:44.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1209 sec 2024-04-26T18:48:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:44 smithi119 ceph-mon[32752]: pgmap v1235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:44 smithi167 ceph-mon[32751]: pgmap v1235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:44 smithi090 ceph-mon[29180]: pgmap v1235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:45.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1210 sec 2024-04-26T18:48:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:45 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 806.986989 secs 2024-04-26T18:48:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 807.651632 secs 2024-04-26T18:48:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:45 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 806.986989 secs 2024-04-26T18:48:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 807.651632 secs 2024-04-26T18:48:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:45 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 806.986989 secs 2024-04-26T18:48:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 807.651632 secs 2024-04-26T18:48:46.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1211 sec 2024-04-26T18:48:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:46 smithi119 ceph-mon[32752]: pgmap v1236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:46 smithi167 ceph-mon[32751]: pgmap v1236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:46 smithi090 ceph-mon[29180]: pgmap v1236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:47.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1212 sec 2024-04-26T18:48:48.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1213 sec 2024-04-26T18:48:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:48 smithi119 ceph-mon[32752]: pgmap v1237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:48 smithi167 ceph-mon[32751]: pgmap v1237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:48 smithi090 ceph-mon[29180]: pgmap v1237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:48:49.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1214 sec 2024-04-26T18:48:50.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1215 sec 2024-04-26T18:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:51 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 811.987071 secs 2024-04-26T18:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:51 smithi119 ceph-mon[32752]: pgmap v1238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 812.651694 secs 2024-04-26T18:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:48:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:51 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 811.987071 secs 2024-04-26T18:48:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:51 smithi167 ceph-mon[32751]: pgmap v1238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:48:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 812.651694 secs 2024-04-26T18:48:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:48:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:48:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:48:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:51 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 811.987071 secs 2024-04-26T18:48:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:51 smithi090 ceph-mon[29180]: pgmap v1238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:48:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:48:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 812.651694 secs 2024-04-26T18:48:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:48:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:48:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:48:51.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1216 sec 2024-04-26T18:48:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:52 smithi119 ceph-mon[32752]: pgmap v1239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:48:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:52 smithi119 ceph-mon[32752]: pgmap v1240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:52 smithi167 ceph-mon[32751]: pgmap v1239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:48:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:52 smithi167 ceph-mon[32751]: pgmap v1240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:52 smithi090 ceph-mon[29180]: pgmap v1239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:48:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:52 smithi090 ceph-mon[29180]: pgmap v1240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:52.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1217 sec 2024-04-26T18:48:53.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1218 sec 2024-04-26T18:48:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:54 smithi119 ceph-mon[32752]: pgmap v1241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:48:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:54 smithi167 ceph-mon[32751]: pgmap v1241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:48:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:54 smithi090 ceph-mon[29180]: pgmap v1241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:48:54.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1219 sec 2024-04-26T18:48:55.986 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1220 sec 2024-04-26T18:48:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:56 smithi119 ceph-mon[32752]: pgmap v1242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:56 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 816.987154 secs 2024-04-26T18:48:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 817.651818 secs 2024-04-26T18:48:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:56 smithi167 ceph-mon[32751]: pgmap v1242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:56 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 816.987154 secs 2024-04-26T18:48:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 817.651818 secs 2024-04-26T18:48:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:56 smithi090 ceph-mon[29180]: pgmap v1242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:56 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 816.987154 secs 2024-04-26T18:48:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 817.651818 secs 2024-04-26T18:48:56.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1221 sec 2024-04-26T18:48:57.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1222 sec 2024-04-26T18:48:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:48:58 smithi119 ceph-mon[32752]: pgmap v1243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:48:58 smithi167 ceph-mon[32751]: pgmap v1243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:48:58 smithi090 ceph-mon[29180]: pgmap v1243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:48:58.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1223 sec 2024-04-26T18:48:59.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1224 sec 2024-04-26T18:49:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:00 smithi119 ceph-mon[32752]: pgmap v1244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:00 smithi167 ceph-mon[32751]: pgmap v1244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:00 smithi090 ceph-mon[29180]: pgmap v1244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:00.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1225 sec 2024-04-26T18:49:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:01 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 821.987235 secs 2024-04-26T18:49:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 822.651955 secs 2024-04-26T18:49:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:01 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 821.987235 secs 2024-04-26T18:49:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 822.651955 secs 2024-04-26T18:49:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:01 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 821.987235 secs 2024-04-26T18:49:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 822.651955 secs 2024-04-26T18:49:01.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1226 sec 2024-04-26T18:49:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:02 smithi119 ceph-mon[32752]: pgmap v1245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:49:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:02 smithi167 ceph-mon[32751]: pgmap v1245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:49:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:02 smithi090 ceph-mon[29180]: pgmap v1245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:49:02.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1227 sec 2024-04-26T18:49:03.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1228 sec 2024-04-26T18:49:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:04 smithi119 ceph-mon[32752]: pgmap v1246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:04 smithi167 ceph-mon[32751]: pgmap v1246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:04 smithi090 ceph-mon[29180]: pgmap v1246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:04.987 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1229 sec 2024-04-26T18:49:05.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1230 sec 2024-04-26T18:49:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:06 smithi119 ceph-mon[32752]: pgmap v1247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:06 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 826.987320 secs 2024-04-26T18:49:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 827.652068 secs 2024-04-26T18:49:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:06 smithi167 ceph-mon[32751]: pgmap v1247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:06 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 826.987320 secs 2024-04-26T18:49:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 827.652068 secs 2024-04-26T18:49:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:06 smithi090 ceph-mon[29180]: pgmap v1247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:06 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 826.987320 secs 2024-04-26T18:49:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 827.652068 secs 2024-04-26T18:49:06.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1231 sec 2024-04-26T18:49:07.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1232 sec 2024-04-26T18:49:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:08 smithi090 ceph-mon[29180]: pgmap v1248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:08 smithi119 ceph-mon[32752]: pgmap v1248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:08 smithi167 ceph-mon[32751]: pgmap v1248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:08.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1233 sec 2024-04-26T18:49:09.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1234 sec 2024-04-26T18:49:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:10 smithi119 ceph-mon[32752]: pgmap v1249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:10 smithi167 ceph-mon[32751]: pgmap v1249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:10 smithi090 ceph-mon[29180]: pgmap v1249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:10.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1235 sec 2024-04-26T18:49:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:11 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 831.987403 secs 2024-04-26T18:49:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 832.652159 secs 2024-04-26T18:49:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:11 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 831.987403 secs 2024-04-26T18:49:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 832.652159 secs 2024-04-26T18:49:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:11 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 831.987403 secs 2024-04-26T18:49:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 832.652159 secs 2024-04-26T18:49:11.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1236 sec 2024-04-26T18:49:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:12 smithi119 ceph-mon[32752]: pgmap v1250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:12 smithi167 ceph-mon[32751]: pgmap v1250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:12 smithi090 ceph-mon[29180]: pgmap v1250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:12.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1237 sec 2024-04-26T18:49:13.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1238 sec 2024-04-26T18:49:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:14 smithi119 ceph-mon[32752]: pgmap v1251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:14 smithi167 ceph-mon[32751]: pgmap v1251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:14 smithi090 ceph-mon[29180]: pgmap v1251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:14.988 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1239 sec 2024-04-26T18:49:15.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1240 sec 2024-04-26T18:49:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:16 smithi119 ceph-mon[32752]: pgmap v1252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:16 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 836.987493 secs 2024-04-26T18:49:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 837.652268 secs 2024-04-26T18:49:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:16 smithi167 ceph-mon[32751]: pgmap v1252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:16 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 836.987493 secs 2024-04-26T18:49:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 837.652268 secs 2024-04-26T18:49:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:16 smithi090 ceph-mon[29180]: pgmap v1252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:16 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 836.987493 secs 2024-04-26T18:49:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 837.652268 secs 2024-04-26T18:49:16.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1241 sec 2024-04-26T18:49:17.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1242 sec 2024-04-26T18:49:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:18 smithi119 ceph-mon[32752]: pgmap v1253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:18 smithi167 ceph-mon[32751]: pgmap v1253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:18 smithi090 ceph-mon[29180]: pgmap v1253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:18.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1243 sec 2024-04-26T18:49:19.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1244 sec 2024-04-26T18:49:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:20 smithi167 ceph-mon[32751]: pgmap v1254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:20 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 841.987576 secs 2024-04-26T18:49:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:20 smithi090 ceph-mon[29180]: pgmap v1254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:20 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 841.987576 secs 2024-04-26T18:49:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:20 smithi119 ceph-mon[32752]: pgmap v1254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:20 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 841.987576 secs 2024-04-26T18:49:20.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1245 sec 2024-04-26T18:49:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 842.652406 secs 2024-04-26T18:49:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 842.652406 secs 2024-04-26T18:49:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 842.652406 secs 2024-04-26T18:49:21.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1246 sec 2024-04-26T18:49:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:22 smithi167 ceph-mon[32751]: pgmap v1255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:22 smithi090 ceph-mon[29180]: pgmap v1255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:22 smithi119 ceph-mon[32752]: pgmap v1255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:22.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1247 sec 2024-04-26T18:49:23.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1248 sec 2024-04-26T18:49:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:24 smithi167 ceph-mon[32751]: pgmap v1256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:24 smithi090 ceph-mon[29180]: pgmap v1256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:24 smithi119 ceph-mon[32752]: pgmap v1256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:24.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1249 sec 2024-04-26T18:49:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:25 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 846.987663 secs 2024-04-26T18:49:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:25 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 846.987663 secs 2024-04-26T18:49:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:25 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 846.987663 secs 2024-04-26T18:49:25.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1250 sec 2024-04-26T18:49:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:26 smithi090 ceph-mon[29180]: pgmap v1257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 847.652543 secs 2024-04-26T18:49:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:26 smithi119 ceph-mon[32752]: pgmap v1257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 847.652543 secs 2024-04-26T18:49:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:26 smithi167 ceph-mon[32751]: pgmap v1257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 847.652543 secs 2024-04-26T18:49:26.989 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1251 sec 2024-04-26T18:49:27.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1252 sec 2024-04-26T18:49:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:28 smithi090 ceph-mon[29180]: pgmap v1258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:28 smithi119 ceph-mon[32752]: pgmap v1258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:28 smithi167 ceph-mon[32751]: pgmap v1258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:28.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1253 sec 2024-04-26T18:49:29.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1254 sec 2024-04-26T18:49:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:30 smithi090 ceph-mon[29180]: pgmap v1259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:30 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 851.987722 secs 2024-04-26T18:49:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:30 smithi119 ceph-mon[32752]: pgmap v1259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:30 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 851.987722 secs 2024-04-26T18:49:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:30 smithi167 ceph-mon[32751]: pgmap v1259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:30 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 851.987722 secs 2024-04-26T18:49:30.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1255 sec 2024-04-26T18:49:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 852.652661 secs 2024-04-26T18:49:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 852.652661 secs 2024-04-26T18:49:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 852.652661 secs 2024-04-26T18:49:31.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1256 sec 2024-04-26T18:49:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:32 smithi090 ceph-mon[29180]: pgmap v1260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:32 smithi119 ceph-mon[32752]: pgmap v1260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:32 smithi167 ceph-mon[32751]: pgmap v1260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:32.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1257 sec 2024-04-26T18:49:33.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1258 sec 2024-04-26T18:49:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:34 smithi090 ceph-mon[29180]: pgmap v1261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:34 smithi119 ceph-mon[32752]: pgmap v1261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:34 smithi167 ceph-mon[32751]: pgmap v1261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:34.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1259 sec 2024-04-26T18:49:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:35 smithi090 ceph-mon[29180]: 7 slow requests, 5 included below; oldest blocked for > 856.987807 secs 2024-04-26T18:49:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:35 smithi090 ceph-mon[29180]: slow request 721.251159 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:49:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:35 smithi090 ceph-mon[29180]: slow request 183.805192 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:49:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:35 smithi090 ceph-mon[29180]: slow request 856.987806 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:49:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:35 smithi090 ceph-mon[29180]: slow request 856.984912 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:49:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:35 smithi090 ceph-mon[29180]: slow request 856.979301 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:49:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:35 smithi119 ceph-mon[32752]: 7 slow requests, 5 included below; oldest blocked for > 856.987807 secs 2024-04-26T18:49:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:35 smithi119 ceph-mon[32752]: slow request 721.251159 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:49:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:35 smithi119 ceph-mon[32752]: slow request 183.805192 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:49:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:35 smithi119 ceph-mon[32752]: slow request 856.987806 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:49:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:35 smithi119 ceph-mon[32752]: slow request 856.984912 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:49:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:35 smithi119 ceph-mon[32752]: slow request 856.979301 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:49:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:35 smithi167 ceph-mon[32751]: 7 slow requests, 5 included below; oldest blocked for > 856.987807 secs 2024-04-26T18:49:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:35 smithi167 ceph-mon[32751]: slow request 721.251159 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T18:49:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:35 smithi167 ceph-mon[32751]: slow request 183.805192 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:49:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:35 smithi167 ceph-mon[32751]: slow request 856.987806 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:49:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:35 smithi167 ceph-mon[32751]: slow request 856.984912 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:49:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:35 smithi167 ceph-mon[32751]: slow request 856.979301 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:49:35.990 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1260 sec 2024-04-26T18:49:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:36 smithi090 ceph-mon[29180]: pgmap v1262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 857.652722 secs 2024-04-26T18:49:36.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:36 smithi119 ceph-mon[32752]: pgmap v1262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:36.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 857.652722 secs 2024-04-26T18:49:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:36 smithi167 ceph-mon[32751]: pgmap v1262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 857.652722 secs 2024-04-26T18:49:36.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1261 sec 2024-04-26T18:49:37.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1262 sec 2024-04-26T18:49:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:38 smithi090 ceph-mon[29180]: pgmap v1263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:38 smithi119 ceph-mon[32752]: pgmap v1263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:38 smithi167 ceph-mon[32751]: pgmap v1263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:38.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1263 sec 2024-04-26T18:49:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:49:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:49:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:49:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:49:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T18:49:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:49:39.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1264 sec 2024-04-26T18:49:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:40 smithi090 ceph-mon[29180]: pgmap v1264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:40 smithi090 ceph-mon[29180]: 7 slow requests, 2 included below; oldest blocked for > 861.987863 secs 2024-04-26T18:49:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:40 smithi090 ceph-mon[29180]: slow request 860.012632 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:49:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:40 smithi090 ceph-mon[29180]: slow request 818.983255 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:49:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:40 smithi119 ceph-mon[32752]: pgmap v1264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:40 smithi119 ceph-mon[32752]: 7 slow requests, 2 included below; oldest blocked for > 861.987863 secs 2024-04-26T18:49:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:40 smithi119 ceph-mon[32752]: slow request 860.012632 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:49:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:40 smithi119 ceph-mon[32752]: slow request 818.983255 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:49:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:40 smithi167 ceph-mon[32751]: pgmap v1264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:40 smithi167 ceph-mon[32751]: 7 slow requests, 2 included below; oldest blocked for > 861.987863 secs 2024-04-26T18:49:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:40 smithi167 ceph-mon[32751]: slow request 860.012632 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:49:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:40 smithi167 ceph-mon[32751]: slow request 818.983255 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:49:40.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1265 sec 2024-04-26T18:49:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 862.652859 secs 2024-04-26T18:49:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 862.652859 secs 2024-04-26T18:49:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 862.652859 secs 2024-04-26T18:49:41.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1266 sec 2024-04-26T18:49:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:42 smithi090 ceph-mon[29180]: pgmap v1265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:42 smithi119 ceph-mon[32752]: pgmap v1265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:42 smithi167 ceph-mon[32751]: pgmap v1265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:42.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1267 sec 2024-04-26T18:49:43.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1268 sec 2024-04-26T18:49:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:44 smithi090 ceph-mon[29180]: pgmap v1266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:44 smithi119 ceph-mon[32752]: pgmap v1266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:44 smithi167 ceph-mon[32751]: pgmap v1266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:44.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1269 sec 2024-04-26T18:49:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 866.987926 secs 2024-04-26T18:49:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 866.987926 secs 2024-04-26T18:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 866.987926 secs 2024-04-26T18:49:45.991 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1270 sec 2024-04-26T18:49:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:46 smithi119 ceph-mon[32752]: pgmap v1267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 867.652964 secs 2024-04-26T18:49:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:46 smithi167 ceph-mon[32751]: pgmap v1267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 867.652964 secs 2024-04-26T18:49:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:46 smithi090 ceph-mon[29180]: pgmap v1267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 867.652964 secs 2024-04-26T18:49:46.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1271 sec 2024-04-26T18:49:47.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1272 sec 2024-04-26T18:49:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:48 smithi119 ceph-mon[32752]: pgmap v1268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:48 smithi167 ceph-mon[32751]: pgmap v1268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:48 smithi090 ceph-mon[29180]: pgmap v1268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:48.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1273 sec 2024-04-26T18:49:49.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1274 sec 2024-04-26T18:49:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:50 smithi119 ceph-mon[32752]: pgmap v1269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 871.988006 secs 2024-04-26T18:49:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:50 smithi167 ceph-mon[32751]: pgmap v1269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 871.988006 secs 2024-04-26T18:49:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:50 smithi090 ceph-mon[29180]: pgmap v1269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 871.988006 secs 2024-04-26T18:49:50.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1275 sec 2024-04-26T18:49:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 872.653087 secs 2024-04-26T18:49:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:49:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 872.653087 secs 2024-04-26T18:49:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:49:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 872.653087 secs 2024-04-26T18:49:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:49:51.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1276 sec 2024-04-26T18:49:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:52 smithi119 ceph-mon[32752]: pgmap v1270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:49:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:49:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:49:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:52 smithi167 ceph-mon[32751]: pgmap v1270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:49:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:49:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:49:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:52 smithi090 ceph-mon[29180]: pgmap v1270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:49:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:49:52.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1277 sec 2024-04-26T18:49:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:53 smithi119 ceph-mon[32752]: pgmap v1271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:49:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:53 smithi119 ceph-mon[32752]: pgmap v1272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:53 smithi167 ceph-mon[32751]: pgmap v1271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:49:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:53 smithi167 ceph-mon[32751]: pgmap v1272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:53 smithi090 ceph-mon[29180]: pgmap v1271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:49:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:53 smithi090 ceph-mon[29180]: pgmap v1272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:53.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1278 sec 2024-04-26T18:49:54.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1279 sec 2024-04-26T18:49:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:55 smithi119 ceph-mon[32752]: pgmap v1273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 876.988090 secs 2024-04-26T18:49:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:55 smithi167 ceph-mon[32751]: pgmap v1273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 876.988090 secs 2024-04-26T18:49:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:55 smithi090 ceph-mon[29180]: pgmap v1273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 876.988090 secs 2024-04-26T18:49:55.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1280 sec 2024-04-26T18:49:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 877.653184 secs 2024-04-26T18:49:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 877.653184 secs 2024-04-26T18:49:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 877.653184 secs 2024-04-26T18:49:56.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1281 sec 2024-04-26T18:49:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:57 smithi119 ceph-mon[32752]: pgmap v1274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:57 smithi167 ceph-mon[32751]: pgmap v1274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:57 smithi090 ceph-mon[29180]: pgmap v1274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:49:57.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1282 sec 2024-04-26T18:49:58.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1283 sec 2024-04-26T18:49:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:49:59 smithi119 ceph-mon[32752]: pgmap v1275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:49:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:49:59 smithi167 ceph-mon[32751]: pgmap v1275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:49:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:49:59 smithi090 ceph-mon[29180]: pgmap v1275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:49:59.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1284 sec 2024-04-26T18:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 881.988173 secs 2024-04-26T18:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: Health detail: HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T18:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: [WRN] CEPHADM_FAILED_DAEMON: 9 failed cephadm daemon(s) 2024-04-26T18:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.j on smithi090 is in error state 2024-04-26T18:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.d on smithi090 is in error state 2024-04-26T18:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.a on smithi090 is in error state 2024-04-26T18:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.k on smithi119 is in error state 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.e on smithi119 is in error state 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.h on smithi119 is in error state 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.l on smithi167 is in error state 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.c on smithi167 is in error state 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: daemon mds.i on smithi167 is in error state 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: mds.g(mds.1): 10 slow requests are blocked > 180 secs 2024-04-26T18:50:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:00 smithi119 ceph-mon[32752]: mds.b(mds.0): 7 slow requests are blocked > 180 secs 2024-04-26T18:50:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 881.988173 secs 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: Health detail: HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: [WRN] CEPHADM_FAILED_DAEMON: 9 failed cephadm daemon(s) 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.j on smithi090 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.d on smithi090 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.a on smithi090 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.k on smithi119 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.e on smithi119 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.h on smithi119 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.l on smithi167 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.c on smithi167 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: daemon mds.i on smithi167 is in error state 2024-04-26T18:50:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-04-26T18:50:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: mds.g(mds.1): 10 slow requests are blocked > 180 secs 2024-04-26T18:50:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:00 smithi167 ceph-mon[32751]: mds.b(mds.0): 7 slow requests are blocked > 180 secs 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 881.988173 secs 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: Health detail: HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: [WRN] CEPHADM_FAILED_DAEMON: 9 failed cephadm daemon(s) 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.j on smithi090 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.d on smithi090 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.a on smithi090 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.k on smithi119 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.e on smithi119 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.h on smithi119 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.l on smithi167 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.c on smithi167 is in error state 2024-04-26T18:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: daemon mds.i on smithi167 is in error state 2024-04-26T18:50:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: [WRN] MDS_SLOW_REQUEST: 2 MDSs report slow requests 2024-04-26T18:50:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: mds.g(mds.1): 10 slow requests are blocked > 180 secs 2024-04-26T18:50:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:00 smithi090 ceph-mon[29180]: mds.b(mds.0): 7 slow requests are blocked > 180 secs 2024-04-26T18:50:00.992 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1285 sec 2024-04-26T18:50:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:01 smithi167 ceph-mon[32751]: pgmap v1276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 882.653334 secs 2024-04-26T18:50:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:01 smithi119 ceph-mon[32752]: pgmap v1276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 882.653334 secs 2024-04-26T18:50:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:01 smithi090 ceph-mon[29180]: pgmap v1276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 882.653334 secs 2024-04-26T18:50:01.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1286 sec 2024-04-26T18:50:02.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1287 sec 2024-04-26T18:50:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:03 smithi119 ceph-mon[32752]: pgmap v1277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:50:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:03 smithi167 ceph-mon[32751]: pgmap v1277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:50:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:03 smithi090 ceph-mon[29180]: pgmap v1277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:50:03.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1288 sec 2024-04-26T18:50:04.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1289 sec 2024-04-26T18:50:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:05 smithi119 ceph-mon[32752]: pgmap v1278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 886.988234 secs 2024-04-26T18:50:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:05 smithi167 ceph-mon[32751]: pgmap v1278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 886.988234 secs 2024-04-26T18:50:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:05 smithi090 ceph-mon[29180]: pgmap v1278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 886.988234 secs 2024-04-26T18:50:05.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1290 sec 2024-04-26T18:50:06.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 887.653445 secs 2024-04-26T18:50:06.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 887.653445 secs 2024-04-26T18:50:06.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 887.653445 secs 2024-04-26T18:50:06.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1291 sec 2024-04-26T18:50:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:07 smithi119 ceph-mon[32752]: pgmap v1279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:07 smithi167 ceph-mon[32751]: pgmap v1279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:07 smithi090 ceph-mon[29180]: pgmap v1279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:07.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1292 sec 2024-04-26T18:50:08.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1293 sec 2024-04-26T18:50:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:09 smithi119 ceph-mon[32752]: pgmap v1280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:09.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:09 smithi167 ceph-mon[32751]: pgmap v1280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:09 smithi090 ceph-mon[29180]: pgmap v1280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:09.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1294 sec 2024-04-26T18:50:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 891.988316 secs 2024-04-26T18:50:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 891.988316 secs 2024-04-26T18:50:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 891.988316 secs 2024-04-26T18:50:10.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1295 sec 2024-04-26T18:50:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:11 smithi119 ceph-mon[32752]: pgmap v1281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:11.571 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 892.653543 secs 2024-04-26T18:50:11.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:11 smithi167 ceph-mon[32751]: pgmap v1281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:11.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 892.653543 secs 2024-04-26T18:50:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:11 smithi090 ceph-mon[29180]: pgmap v1281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:11.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 892.653543 secs 2024-04-26T18:50:11.993 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1296 sec 2024-04-26T18:50:12.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1297 sec 2024-04-26T18:50:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:13 smithi119 ceph-mon[32752]: pgmap v1282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:13.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:13 smithi167 ceph-mon[32751]: pgmap v1282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:13 smithi090 ceph-mon[29180]: pgmap v1282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:13.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1298 sec 2024-04-26T18:50:14.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1299 sec 2024-04-26T18:50:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:15 smithi119 ceph-mon[32752]: pgmap v1283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:15.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 896.988401 secs 2024-04-26T18:50:15.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:15 smithi167 ceph-mon[32751]: pgmap v1283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:15.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 896.988401 secs 2024-04-26T18:50:15.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:15 smithi090 ceph-mon[29180]: pgmap v1283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:15.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 896.988401 secs 2024-04-26T18:50:15.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1300 sec 2024-04-26T18:50:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 897.653634 secs 2024-04-26T18:50:16.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 897.653634 secs 2024-04-26T18:50:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 897.653634 secs 2024-04-26T18:50:16.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1301 sec 2024-04-26T18:50:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:17 smithi119 ceph-mon[32752]: pgmap v1284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:17 smithi167 ceph-mon[32751]: pgmap v1284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:17 smithi090 ceph-mon[29180]: pgmap v1284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:17.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1302 sec 2024-04-26T18:50:18.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1303 sec 2024-04-26T18:50:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:19 smithi119 ceph-mon[32752]: pgmap v1285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:19 smithi167 ceph-mon[32751]: pgmap v1285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:19 smithi090 ceph-mon[29180]: pgmap v1285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:19.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1304 sec 2024-04-26T18:50:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 901.988483 secs 2024-04-26T18:50:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 901.988483 secs 2024-04-26T18:50:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 901.988483 secs 2024-04-26T18:50:20.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1305 sec 2024-04-26T18:50:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:21 smithi119 ceph-mon[32752]: pgmap v1286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:21.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 902.653713 secs 2024-04-26T18:50:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:21 smithi167 ceph-mon[32751]: pgmap v1286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:21.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 902.653713 secs 2024-04-26T18:50:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:21 smithi090 ceph-mon[29180]: pgmap v1286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:21.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 902.653713 secs 2024-04-26T18:50:21.994 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1306 sec 2024-04-26T18:50:22.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1307 sec 2024-04-26T18:50:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:23 smithi119 ceph-mon[32752]: pgmap v1287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:23 smithi167 ceph-mon[32751]: pgmap v1287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:23 smithi090 ceph-mon[29180]: pgmap v1287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:23.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1308 sec 2024-04-26T18:50:24.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1309 sec 2024-04-26T18:50:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:25 smithi119 ceph-mon[32752]: pgmap v1288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:25.587 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 906.988566 secs 2024-04-26T18:50:25.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:25 smithi167 ceph-mon[32751]: pgmap v1288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:25.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 906.988566 secs 2024-04-26T18:50:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:25 smithi090 ceph-mon[29180]: pgmap v1288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:25.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 906.988566 secs 2024-04-26T18:50:25.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1310 sec 2024-04-26T18:50:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 907.653852 secs 2024-04-26T18:50:26.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 907.653852 secs 2024-04-26T18:50:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 907.653852 secs 2024-04-26T18:50:26.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1311 sec 2024-04-26T18:50:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:27 smithi119 ceph-mon[32752]: pgmap v1289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:27.612 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:27 smithi167 ceph-mon[32751]: pgmap v1289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:27.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:27 smithi090 ceph-mon[29180]: pgmap v1289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:27.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1312 sec 2024-04-26T18:50:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T18:50:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:50:28.489 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400afb0 tx=0x7f2cc400a2d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:50:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00a6280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400afb0 tx=0x7f2cc400a2d0 comp rx=0 tx=0).stop 2024-04-26T18:50:28.547 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00a6280 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T18:50:28.556 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00a6280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T18:50:28.562 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00a6280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:50:28.568 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 0x7f2cc00b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:50:28.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.467+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 0x7f2cc00b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:50:28.581 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 0x7f2cc00b31e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400fe40 tx=0x7f2cc4038040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:50:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.489+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-04-26T18:50:28.658 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.489+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T18:50:28.665 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.489+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:50:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.489+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T18:50:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.489+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=562 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:50:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=SESSION_RECONNECTING pgs=562 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_session_reset received session reset full=1 2024-04-26T18:50:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=SESSION_RECONNECTING pgs=562 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).reset_session 2024-04-26T18:50:28.726 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.490+0000 7f2cb9ffb640 0 client.17037 ms_handle_remote_reset on v2:172.21.15.119:6838/1002360318 2024-04-26T18:50:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.490+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T18:50:28.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T18:50:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.582+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006120 con 0x7f2cd4104500 2024-04-26T18:50:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.582+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006120 con 0x7f2cd4104500 2024-04-26T18:50:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.595+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006270 con 0x56302893f0e0 2024-04-26T18:50:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:50:28.595+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 11 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006270 con 0x56302893f0e0 2024-04-26T18:50:28.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1313 sec 2024-04-26T18:50:29.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:29 smithi090 ceph-mon[29180]: pgmap v1290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:29.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:29 smithi119 ceph-mon[32752]: pgmap v1290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:29.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:29 smithi167 ceph-mon[32751]: pgmap v1290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:29.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1314 sec 2024-04-26T18:50:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 911.988647 secs 2024-04-26T18:50:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 911.988647 secs 2024-04-26T18:50:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 911.988647 secs 2024-04-26T18:50:30.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1315 sec 2024-04-26T18:50:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:31 smithi119 ceph-mon[32752]: pgmap v1291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:31.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 912.653972 secs 2024-04-26T18:50:31.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:31 smithi090 ceph-mon[29180]: pgmap v1291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:31.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 912.653972 secs 2024-04-26T18:50:31.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:31 smithi167 ceph-mon[32751]: pgmap v1291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:31.664 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 912.653972 secs 2024-04-26T18:50:31.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1316 sec 2024-04-26T18:50:32.995 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1317 sec 2024-04-26T18:50:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:33 smithi119 ceph-mon[32752]: pgmap v1292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:33 smithi167 ceph-mon[32751]: pgmap v1292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:33 smithi090 ceph-mon[29180]: pgmap v1292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:33.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1318 sec 2024-04-26T18:50:34.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1319 sec 2024-04-26T18:50:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:35 smithi119 ceph-mon[32752]: pgmap v1293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:35.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 916.988733 secs 2024-04-26T18:50:35.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:35 smithi167 ceph-mon[32751]: pgmap v1293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:35.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 916.988733 secs 2024-04-26T18:50:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:35 smithi090 ceph-mon[29180]: pgmap v1293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:35.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 916.988733 secs 2024-04-26T18:50:35.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1320 sec 2024-04-26T18:50:36.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 917.654097 secs 2024-04-26T18:50:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 917.654097 secs 2024-04-26T18:50:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 917.654097 secs 2024-04-26T18:50:36.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1321 sec 2024-04-26T18:50:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:37 smithi119 ceph-mon[32752]: pgmap v1294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:37.612 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:37 smithi167 ceph-mon[32751]: pgmap v1294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:37 smithi090 ceph-mon[29180]: pgmap v1294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:37.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1322 sec 2024-04-26T18:50:38.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1323 sec 2024-04-26T18:50:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:39 smithi119 ceph-mon[32752]: pgmap v1295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:39 smithi167 ceph-mon[32751]: pgmap v1295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:39 smithi090 ceph-mon[29180]: pgmap v1295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:39.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1324 sec 2024-04-26T18:50:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 921.988813 secs 2024-04-26T18:50:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 921.988813 secs 2024-04-26T18:50:40.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 921.988813 secs 2024-04-26T18:50:40.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1325 sec 2024-04-26T18:50:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:41 smithi119 ceph-mon[32752]: pgmap v1296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:41.590 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 922.654238 secs 2024-04-26T18:50:41.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:41 smithi167 ceph-mon[32751]: pgmap v1296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:41.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 922.654238 secs 2024-04-26T18:50:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:41 smithi090 ceph-mon[29180]: pgmap v1296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:41.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 922.654238 secs 2024-04-26T18:50:41.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1326 sec 2024-04-26T18:50:42.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1327 sec 2024-04-26T18:50:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:43 smithi119 ceph-mon[32752]: pgmap v1297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:43 smithi167 ceph-mon[32751]: pgmap v1297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:43 smithi090 ceph-mon[29180]: pgmap v1297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:43.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1328 sec 2024-04-26T18:50:44.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1329 sec 2024-04-26T18:50:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:45 smithi119 ceph-mon[32752]: pgmap v1298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:45.569 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 926.988832 secs 2024-04-26T18:50:45.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:45 smithi167 ceph-mon[32751]: pgmap v1298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:45.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 926.988832 secs 2024-04-26T18:50:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:45 smithi090 ceph-mon[29180]: pgmap v1298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 926.988832 secs 2024-04-26T18:50:45.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1330 sec 2024-04-26T18:50:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 927.654348 secs 2024-04-26T18:50:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 927.654348 secs 2024-04-26T18:50:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 927.654348 secs 2024-04-26T18:50:46.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1331 sec 2024-04-26T18:50:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:47 smithi119 ceph-mon[32752]: pgmap v1299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:47.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:47 smithi167 ceph-mon[32751]: pgmap v1299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:47 smithi090 ceph-mon[29180]: pgmap v1299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:47.996 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1332 sec 2024-04-26T18:50:48.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1333 sec 2024-04-26T18:50:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:49 smithi119 ceph-mon[32752]: pgmap v1300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:49 smithi167 ceph-mon[32751]: pgmap v1300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:49 smithi090 ceph-mon[29180]: pgmap v1300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:49.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1334 sec 2024-04-26T18:50:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 931.988914 secs 2024-04-26T18:50:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 931.988914 secs 2024-04-26T18:50:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 931.988914 secs 2024-04-26T18:50:50.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1335 sec 2024-04-26T18:50:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:51 smithi119 ceph-mon[32752]: pgmap v1301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:51.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 932.654503 secs 2024-04-26T18:50:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:51 smithi167 ceph-mon[32751]: pgmap v1301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:51.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 932.654503 secs 2024-04-26T18:50:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:51 smithi090 ceph-mon[29180]: pgmap v1301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:50:51.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 932.654503 secs 2024-04-26T18:50:51.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1336 sec 2024-04-26T18:50:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:50:52.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:50:52.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:50:52.575 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:50:52.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:50:52.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:50:52.758 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:50:52.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:50:52.839 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:50:52.845 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:50:52.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:50:52.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:50:52.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1337 sec 2024-04-26T18:50:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:53 smithi119 ceph-mon[32752]: pgmap v1302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:53.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:53 smithi119 ceph-mon[32752]: pgmap v1303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:50:53.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:53 smithi119 ceph-mon[32752]: pgmap v1304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:50:53.618 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:53 smithi167 ceph-mon[32751]: pgmap v1302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:53.679 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:53 smithi167 ceph-mon[32751]: pgmap v1303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:50:53.757 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:53 smithi167 ceph-mon[32751]: pgmap v1304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:50:53.766 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:53 smithi090 ceph-mon[29180]: pgmap v1302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:50:53.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:53 smithi090 ceph-mon[29180]: pgmap v1303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:50:53.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:53 smithi090 ceph-mon[29180]: pgmap v1304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:50:53.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1338 sec 2024-04-26T18:50:54.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1339 sec 2024-04-26T18:50:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:55 smithi167 ceph-mon[32751]: pgmap v1305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:55.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 936.989000 secs 2024-04-26T18:50:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:55 smithi090 ceph-mon[29180]: pgmap v1305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:55.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 936.989000 secs 2024-04-26T18:50:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:55 smithi119 ceph-mon[32752]: pgmap v1305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:55.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 936.989000 secs 2024-04-26T18:50:55.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1340 sec 2024-04-26T18:50:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 937.654583 secs 2024-04-26T18:50:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 937.654583 secs 2024-04-26T18:50:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 937.654583 secs 2024-04-26T18:50:56.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1341 sec 2024-04-26T18:50:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:57 smithi167 ceph-mon[32751]: pgmap v1306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:57 smithi090 ceph-mon[29180]: pgmap v1306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:57 smithi119 ceph-mon[32752]: pgmap v1306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:57.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1342 sec 2024-04-26T18:50:58.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1343 sec 2024-04-26T18:50:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:50:59 smithi167 ceph-mon[32751]: pgmap v1307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:50:59 smithi090 ceph-mon[29180]: pgmap v1307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:50:59 smithi119 ceph-mon[32752]: pgmap v1307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:50:59.997 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1344 sec 2024-04-26T18:51:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 941.989085 secs 2024-04-26T18:51:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 941.989085 secs 2024-04-26T18:51:00.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 941.989085 secs 2024-04-26T18:51:00.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1345 sec 2024-04-26T18:51:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:01 smithi090 ceph-mon[29180]: pgmap v1308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:51:01.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 942.654690 secs 2024-04-26T18:51:01.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:01 smithi167 ceph-mon[32751]: pgmap v1308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:51:01.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 942.654690 secs 2024-04-26T18:51:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:01 smithi119 ceph-mon[32752]: pgmap v1308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:51:01.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 942.654690 secs 2024-04-26T18:51:01.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1346 sec 2024-04-26T18:51:02.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1347 sec 2024-04-26T18:51:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:03 smithi090 ceph-mon[29180]: pgmap v1309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:51:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:03 smithi119 ceph-mon[32752]: pgmap v1309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:51:03.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:03 smithi167 ceph-mon[32751]: pgmap v1309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:51:03.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1348 sec 2024-04-26T18:51:04.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1349 sec 2024-04-26T18:51:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:05 smithi090 ceph-mon[29180]: pgmap v1310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:05.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 946.989147 secs 2024-04-26T18:51:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:05 smithi119 ceph-mon[32752]: pgmap v1310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:05.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 946.989147 secs 2024-04-26T18:51:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:05 smithi167 ceph-mon[32751]: pgmap v1310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:05.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 946.989147 secs 2024-04-26T18:51:05.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1350 sec 2024-04-26T18:51:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 947.654803 secs 2024-04-26T18:51:06.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 947.654803 secs 2024-04-26T18:51:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 947.654803 secs 2024-04-26T18:51:06.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1351 sec 2024-04-26T18:51:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:07 smithi090 ceph-mon[29180]: pgmap v1311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:07 smithi119 ceph-mon[32752]: pgmap v1311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:07.920 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:07 smithi167 ceph-mon[32751]: pgmap v1311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:07.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1352 sec 2024-04-26T18:51:08.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1353 sec 2024-04-26T18:51:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:09 smithi090 ceph-mon[29180]: pgmap v1312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:09 smithi119 ceph-mon[32752]: pgmap v1312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:09 smithi167 ceph-mon[32751]: pgmap v1312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:09.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1354 sec 2024-04-26T18:51:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 951.989204 secs 2024-04-26T18:51:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 951.989204 secs 2024-04-26T18:51:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 951.989204 secs 2024-04-26T18:51:10.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1355 sec 2024-04-26T18:51:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:11 smithi090 ceph-mon[29180]: pgmap v1313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:11.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 952.654883 secs 2024-04-26T18:51:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:11 smithi119 ceph-mon[32752]: pgmap v1313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:11.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 952.654883 secs 2024-04-26T18:51:11.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:11 smithi167 ceph-mon[32751]: pgmap v1313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:11.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 952.654883 secs 2024-04-26T18:51:11.998 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1356 sec 2024-04-26T18:51:12.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1357 sec 2024-04-26T18:51:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:13 smithi090 ceph-mon[29180]: pgmap v1314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:13 smithi119 ceph-mon[32752]: pgmap v1314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:13 smithi167 ceph-mon[32751]: pgmap v1314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:13.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1358 sec 2024-04-26T18:51:14.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1359 sec 2024-04-26T18:51:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:15 smithi090 ceph-mon[29180]: pgmap v1315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:15.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 956.989290 secs 2024-04-26T18:51:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:15 smithi119 ceph-mon[32752]: pgmap v1315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 956.989290 secs 2024-04-26T18:51:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:15 smithi167 ceph-mon[32751]: pgmap v1315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:15.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 956.989290 secs 2024-04-26T18:51:15.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1360 sec 2024-04-26T18:51:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 957.654950 secs 2024-04-26T18:51:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 957.654950 secs 2024-04-26T18:51:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 957.654950 secs 2024-04-26T18:51:16.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1361 sec 2024-04-26T18:51:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:17 smithi090 ceph-mon[29180]: pgmap v1316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:17 smithi119 ceph-mon[32752]: pgmap v1316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:17 smithi167 ceph-mon[32751]: pgmap v1316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:17.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1362 sec 2024-04-26T18:51:18.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1363 sec 2024-04-26T18:51:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:19 smithi090 ceph-mon[29180]: pgmap v1317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:19 smithi119 ceph-mon[32752]: pgmap v1317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:19.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:19 smithi167 ceph-mon[32751]: pgmap v1317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:19.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1364 sec 2024-04-26T18:51:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 961.989373 secs 2024-04-26T18:51:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 961.989373 secs 2024-04-26T18:51:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 961.989373 secs 2024-04-26T18:51:20.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1365 sec 2024-04-26T18:51:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:21 smithi090 ceph-mon[29180]: pgmap v1318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:21.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 962.655061 secs 2024-04-26T18:51:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:21 smithi119 ceph-mon[32752]: pgmap v1318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:21.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 962.655061 secs 2024-04-26T18:51:21.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:21 smithi167 ceph-mon[32751]: pgmap v1318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:21.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 962.655061 secs 2024-04-26T18:51:21.999 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1366 sec 2024-04-26T18:51:23.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1367 sec 2024-04-26T18:51:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:23 smithi090 ceph-mon[29180]: pgmap v1319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:23 smithi119 ceph-mon[32752]: pgmap v1319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:23 smithi167 ceph-mon[32751]: pgmap v1319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:24.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1368 sec 2024-04-26T18:51:25.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1369 sec 2024-04-26T18:51:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:25 smithi090 ceph-mon[29180]: pgmap v1320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:25.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 966.989457 secs 2024-04-26T18:51:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:25 smithi119 ceph-mon[32752]: pgmap v1320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:25.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 966.989457 secs 2024-04-26T18:51:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:25 smithi167 ceph-mon[32751]: pgmap v1320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:25.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 966.989457 secs 2024-04-26T18:51:26.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1370 sec 2024-04-26T18:51:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 967.655152 secs 2024-04-26T18:51:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 967.655152 secs 2024-04-26T18:51:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 967.655152 secs 2024-04-26T18:51:27.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1371 sec 2024-04-26T18:51:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:27 smithi119 ceph-mon[32752]: pgmap v1321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:27 smithi167 ceph-mon[32751]: pgmap v1321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:27 smithi090 ceph-mon[29180]: pgmap v1321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:28.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1372 sec 2024-04-26T18:51:29.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1373 sec 2024-04-26T18:51:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:29 smithi119 ceph-mon[32752]: pgmap v1322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:29 smithi167 ceph-mon[32751]: pgmap v1322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:29 smithi090 ceph-mon[29180]: pgmap v1322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:30.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1374 sec 2024-04-26T18:51:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 971.989555 secs 2024-04-26T18:51:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 971.989555 secs 2024-04-26T18:51:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 971.989555 secs 2024-04-26T18:51:31.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1375 sec 2024-04-26T18:51:31.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:51:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:51:31.052+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:51:31.383 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:51:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:51:31.052+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:51:31.398 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:51:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:51:31.052+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:51:31.406 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:51:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:51:31.052+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:51:31.415 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:51:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:51:31.052+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:51:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:31 smithi167 ceph-mon[32751]: pgmap v1323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:31.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 972.655292 secs 2024-04-26T18:51:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:31 smithi119 ceph-mon[32752]: pgmap v1323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:31.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 972.655292 secs 2024-04-26T18:51:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:31 smithi090 ceph-mon[29180]: pgmap v1323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:31.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 972.655292 secs 2024-04-26T18:51:32.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1376 sec 2024-04-26T18:51:33.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1377 sec 2024-04-26T18:51:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:33 smithi119 ceph-mon[32752]: pgmap v1324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:33.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:33 smithi167 ceph-mon[32751]: pgmap v1324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:33 smithi090 ceph-mon[29180]: pgmap v1324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:34.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1378 sec 2024-04-26T18:51:35.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1379 sec 2024-04-26T18:51:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:35 smithi119 ceph-mon[32752]: pgmap v1325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 976.989642 secs 2024-04-26T18:51:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:35 smithi167 ceph-mon[32751]: pgmap v1325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 976.989642 secs 2024-04-26T18:51:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:35 smithi090 ceph-mon[29180]: pgmap v1325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 976.989642 secs 2024-04-26T18:51:36.000 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1380 sec 2024-04-26T18:51:36.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 977.655416 secs 2024-04-26T18:51:36.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 977.655416 secs 2024-04-26T18:51:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 977.655416 secs 2024-04-26T18:51:37.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1381 sec 2024-04-26T18:51:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:37 smithi119 ceph-mon[32752]: pgmap v1326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:37.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:37 smithi167 ceph-mon[32751]: pgmap v1326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:37 smithi090 ceph-mon[29180]: pgmap v1326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:38.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1382 sec 2024-04-26T18:51:39.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1383 sec 2024-04-26T18:51:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:39 smithi119 ceph-mon[32752]: pgmap v1327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:39.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:39 smithi167 ceph-mon[32751]: pgmap v1327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:39 smithi090 ceph-mon[29180]: pgmap v1327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:40.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1384 sec 2024-04-26T18:51:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 981.989724 secs 2024-04-26T18:51:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 981.989724 secs 2024-04-26T18:51:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 981.989724 secs 2024-04-26T18:51:41.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1385 sec 2024-04-26T18:51:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:41 smithi119 ceph-mon[32752]: pgmap v1328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:41.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 982.655550 secs 2024-04-26T18:51:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:41 smithi090 ceph-mon[29180]: pgmap v1328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:41.895 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 982.655550 secs 2024-04-26T18:51:41.904 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:41 smithi167 ceph-mon[32751]: pgmap v1328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:41.912 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 982.655550 secs 2024-04-26T18:51:42.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1386 sec 2024-04-26T18:51:43.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1387 sec 2024-04-26T18:51:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:43 smithi119 ceph-mon[32752]: pgmap v1329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:43 smithi167 ceph-mon[32751]: pgmap v1329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:43 smithi090 ceph-mon[29180]: pgmap v1329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T18:51:44.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1388 sec 2024-04-26T18:51:45.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1389 sec 2024-04-26T18:51:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:45 smithi119 ceph-mon[32752]: pgmap v1330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:45.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 986.989809 secs 2024-04-26T18:51:45.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:45 smithi167 ceph-mon[32751]: pgmap v1330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:45.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 986.989809 secs 2024-04-26T18:51:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:45 smithi090 ceph-mon[29180]: pgmap v1330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T18:51:45.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 986.989809 secs 2024-04-26T18:51:46.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1390 sec 2024-04-26T18:51:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 987.655639 secs 2024-04-26T18:51:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 987.655639 secs 2024-04-26T18:51:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 987.655639 secs 2024-04-26T18:51:47.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1391 sec 2024-04-26T18:51:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:47 smithi119 ceph-mon[32752]: pgmap v1331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:47.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:47 smithi167 ceph-mon[32751]: pgmap v1331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:47 smithi090 ceph-mon[29180]: pgmap v1331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:48.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1392 sec 2024-04-26T18:51:49.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1393 sec 2024-04-26T18:51:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:49 smithi119 ceph-mon[32752]: pgmap v1332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:49.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:49 smithi167 ceph-mon[32751]: pgmap v1332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:49 smithi090 ceph-mon[29180]: pgmap v1332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:51:50.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1394 sec 2024-04-26T18:51:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 991.989870 secs 2024-04-26T18:51:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 991.989870 secs 2024-04-26T18:51:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 991.989870 secs 2024-04-26T18:51:51.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1395 sec 2024-04-26T18:51:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:51 smithi119 ceph-mon[32752]: pgmap v1333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:51.824 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 992.655785 secs 2024-04-26T18:51:51.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:51 smithi167 ceph-mon[32751]: pgmap v1333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:51.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 992.655785 secs 2024-04-26T18:51:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:51 smithi090 ceph-mon[29180]: pgmap v1333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 992.655785 secs 2024-04-26T18:51:52.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1396 sec 2024-04-26T18:51:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:51:52.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:51:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:51:53.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1397 sec 2024-04-26T18:51:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:53 smithi119 ceph-mon[32752]: pgmap v1334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:53.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:51:53.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:51:53.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:51:53.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:53 smithi090 ceph-mon[29180]: pgmap v1334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:53.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:51:53.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:51:53.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:51:54.015 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:53 smithi167 ceph-mon[32751]: pgmap v1334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:51:54.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:51:54.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:51:54.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:51:54.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1398 sec 2024-04-26T18:51:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:54 smithi119 ceph-mon[32752]: pgmap v1335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:51:54.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:54 smithi119 ceph-mon[32752]: pgmap v1336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:51:54.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:54 smithi167 ceph-mon[32751]: pgmap v1335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:51:54.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:54 smithi167 ceph-mon[32751]: pgmap v1336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:51:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:54 smithi090 ceph-mon[29180]: pgmap v1335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:51:54.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:54 smithi090 ceph-mon[29180]: pgmap v1336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:51:55.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1399 sec 2024-04-26T18:51:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 996.989933 secs 2024-04-26T18:51:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 996.989933 secs 2024-04-26T18:51:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 996.989933 secs 2024-04-26T18:51:56.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1400 sec 2024-04-26T18:51:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:56 smithi119 ceph-mon[32752]: pgmap v1337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:51:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 997.655902 secs 2024-04-26T18:51:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:56 smithi167 ceph-mon[32751]: pgmap v1337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:51:56.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 997.655902 secs 2024-04-26T18:51:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:56 smithi090 ceph-mon[29180]: pgmap v1337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:51:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 997.655902 secs 2024-04-26T18:51:57.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1401 sec 2024-04-26T18:51:58.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1402 sec 2024-04-26T18:51:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:51:58.594+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T18:51:58.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:51:58.594+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T18:51:58.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:51:58.606+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T18:51:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:51:58.606+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 12 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T18:51:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:51:58 smithi119 ceph-mon[32752]: pgmap v1338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:51:58.822 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:51:58 smithi090 ceph-mon[29180]: pgmap v1338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:51:58.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:51:58 smithi167 ceph-mon[32751]: pgmap v1338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:51:59.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1403 sec 2024-04-26T18:52:00.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1404 sec 2024-04-26T18:52:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:00 smithi119 ceph-mon[32752]: pgmap v1339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:00.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1001.989992 secs 2024-04-26T18:52:00.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:00 smithi090 ceph-mon[29180]: pgmap v1339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:00.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1001.989992 secs 2024-04-26T18:52:00.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:00 smithi167 ceph-mon[32751]: pgmap v1339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:00.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1001.989992 secs 2024-04-26T18:52:01.002 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1405 sec 2024-04-26T18:52:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1002.655984 secs 2024-04-26T18:52:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1002.655984 secs 2024-04-26T18:52:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1002.655984 secs 2024-04-26T18:52:02.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1406 sec 2024-04-26T18:52:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:02 smithi119 ceph-mon[32752]: pgmap v1340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:02 smithi167 ceph-mon[32751]: pgmap v1340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:02 smithi090 ceph-mon[29180]: pgmap v1340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:03.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1407 sec 2024-04-26T18:52:04.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1408 sec 2024-04-26T18:52:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:03 smithi119 ceph-mon[32752]: pgmap v1341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:52:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:03 smithi167 ceph-mon[32751]: pgmap v1341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:52:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:03 smithi090 ceph-mon[29180]: pgmap v1341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:52:05.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1409 sec 2024-04-26T18:52:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1006.990101 secs 2024-04-26T18:52:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1006.990101 secs 2024-04-26T18:52:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1006.990101 secs 2024-04-26T18:52:06.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1410 sec 2024-04-26T18:52:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:06 smithi090 ceph-mon[29180]: pgmap v1342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:06.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1007.656095 secs 2024-04-26T18:52:06.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:06 smithi119 ceph-mon[32752]: pgmap v1342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:06.462 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1007.656095 secs 2024-04-26T18:52:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:06 smithi167 ceph-mon[32751]: pgmap v1342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:06.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1007.656095 secs 2024-04-26T18:52:07.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1411 sec 2024-04-26T18:52:08.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1412 sec 2024-04-26T18:52:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:08 smithi090 ceph-mon[29180]: pgmap v1343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:08 smithi119 ceph-mon[32752]: pgmap v1343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:08.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:08 smithi167 ceph-mon[32751]: pgmap v1343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:09.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1413 sec 2024-04-26T18:52:10.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1414 sec 2024-04-26T18:52:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:10 smithi090 ceph-mon[29180]: pgmap v1344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:10.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1011.990183 secs 2024-04-26T18:52:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:10 smithi119 ceph-mon[32752]: pgmap v1344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:10.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1011.990183 secs 2024-04-26T18:52:10.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:10 smithi167 ceph-mon[32751]: pgmap v1344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:10.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1011.990183 secs 2024-04-26T18:52:11.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1415 sec 2024-04-26T18:52:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1012.656247 secs 2024-04-26T18:52:11.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1012.656247 secs 2024-04-26T18:52:11.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1012.656247 secs 2024-04-26T18:52:12.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1416 sec 2024-04-26T18:52:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:12 smithi119 ceph-mon[32752]: pgmap v1345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:12.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:12 smithi167 ceph-mon[32751]: pgmap v1345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:12 smithi090 ceph-mon[29180]: pgmap v1345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:13.003 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1417 sec 2024-04-26T18:52:14.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1418 sec 2024-04-26T18:52:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:14 smithi119 ceph-mon[32752]: pgmap v1346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:14.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:14 smithi090 ceph-mon[29180]: pgmap v1346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:14.688 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:14 smithi167 ceph-mon[32751]: pgmap v1346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T18:52:15.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1419 sec 2024-04-26T18:52:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1016.990268 secs 2024-04-26T18:52:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1016.990268 secs 2024-04-26T18:52:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1016.990268 secs 2024-04-26T18:52:16.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1420 sec 2024-04-26T18:52:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:16 smithi119 ceph-mon[32752]: pgmap v1347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:16.613 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1017.656392 secs 2024-04-26T18:52:16.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:16 smithi090 ceph-mon[29180]: pgmap v1347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:16.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1017.656392 secs 2024-04-26T18:52:16.652 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:16 smithi167 ceph-mon[32751]: pgmap v1347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T18:52:16.659 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1017.656392 secs 2024-04-26T18:52:17.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1421 sec 2024-04-26T18:52:18.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1422 sec 2024-04-26T18:52:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:18 smithi119 ceph-mon[32752]: pgmap v1348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:18 smithi167 ceph-mon[32751]: pgmap v1348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:18 smithi090 ceph-mon[29180]: pgmap v1348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:19.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1423 sec 2024-04-26T18:52:20.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1424 sec 2024-04-26T18:52:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:20 smithi119 ceph-mon[32752]: pgmap v1349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:20.626 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1021.990325 secs 2024-04-26T18:52:20.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:20 smithi090 ceph-mon[29180]: pgmap v1349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:20.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1021.990325 secs 2024-04-26T18:52:20.703 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:20 smithi167 ceph-mon[32751]: pgmap v1349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:20.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1021.990325 secs 2024-04-26T18:52:21.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1425 sec 2024-04-26T18:52:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1022.656461 secs 2024-04-26T18:52:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1022.656461 secs 2024-04-26T18:52:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1022.656461 secs 2024-04-26T18:52:22.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1426 sec 2024-04-26T18:52:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:22 smithi119 ceph-mon[32752]: pgmap v1350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:22.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:22 smithi167 ceph-mon[32751]: pgmap v1350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:22 smithi090 ceph-mon[29180]: pgmap v1350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:23.004 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1427 sec 2024-04-26T18:52:24.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1428 sec 2024-04-26T18:52:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:24 smithi119 ceph-mon[32752]: pgmap v1351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:24.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:24 smithi167 ceph-mon[32751]: pgmap v1351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:24 smithi090 ceph-mon[29180]: pgmap v1351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:25.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1429 sec 2024-04-26T18:52:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1026.990406 secs 2024-04-26T18:52:25.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1026.990406 secs 2024-04-26T18:52:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1026.990406 secs 2024-04-26T18:52:26.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1430 sec 2024-04-26T18:52:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:26 smithi119 ceph-mon[32752]: pgmap v1352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:26.622 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1027.656597 secs 2024-04-26T18:52:26.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:26 smithi090 ceph-mon[29180]: pgmap v1352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:26.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1027.656597 secs 2024-04-26T18:52:26.645 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:26 smithi167 ceph-mon[32751]: pgmap v1352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:26.652 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1027.656597 secs 2024-04-26T18:52:27.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1431 sec 2024-04-26T18:52:28.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1432 sec 2024-04-26T18:52:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:28 smithi119 ceph-mon[32752]: pgmap v1353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:28.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:28 smithi167 ceph-mon[32751]: pgmap v1353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:28 smithi090 ceph-mon[29180]: pgmap v1353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:29.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1433 sec 2024-04-26T18:52:30.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1434 sec 2024-04-26T18:52:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:30 smithi119 ceph-mon[32752]: pgmap v1354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:30.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1031.990482 secs 2024-04-26T18:52:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:30 smithi167 ceph-mon[32751]: pgmap v1354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:30.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1031.990482 secs 2024-04-26T18:52:30.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:30 smithi090 ceph-mon[29180]: pgmap v1354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:30.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1031.990482 secs 2024-04-26T18:52:31.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1435 sec 2024-04-26T18:52:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1032.656668 secs 2024-04-26T18:52:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1032.656668 secs 2024-04-26T18:52:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1032.656668 secs 2024-04-26T18:52:32.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1436 sec 2024-04-26T18:52:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:32 smithi119 ceph-mon[32752]: pgmap v1355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:32.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:32 smithi167 ceph-mon[32751]: pgmap v1355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:32 smithi090 ceph-mon[29180]: pgmap v1355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:33.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1437 sec 2024-04-26T18:52:34.005 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1438 sec 2024-04-26T18:52:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:34 smithi119 ceph-mon[32752]: pgmap v1356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:34.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:34 smithi167 ceph-mon[32751]: pgmap v1356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:34 smithi090 ceph-mon[29180]: pgmap v1356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:35.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1439 sec 2024-04-26T18:52:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:35 smithi119 ceph-mon[32752]: 7 slow requests, 1 included below; oldest blocked for > 1036.990563 secs 2024-04-26T18:52:35.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:35 smithi119 ceph-mon[32752]: slow request 363.807948 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:52:35.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:35 smithi167 ceph-mon[32751]: 7 slow requests, 1 included below; oldest blocked for > 1036.990563 secs 2024-04-26T18:52:35.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:35 smithi167 ceph-mon[32751]: slow request 363.807948 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:52:35.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:35 smithi090 ceph-mon[29180]: 7 slow requests, 1 included below; oldest blocked for > 1036.990563 secs 2024-04-26T18:52:35.658 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:35 smithi090 ceph-mon[29180]: slow request 363.807948 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:52:36.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1440 sec 2024-04-26T18:52:36.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:36 smithi119 ceph-mon[32752]: pgmap v1357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:36.488 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1037.656736 secs 2024-04-26T18:52:36.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:36 smithi090 ceph-mon[29180]: pgmap v1357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:36.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1037.656736 secs 2024-04-26T18:52:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:36 smithi167 ceph-mon[32751]: pgmap v1357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:36.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1037.656736 secs 2024-04-26T18:52:37.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1441 sec 2024-04-26T18:52:38.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1442 sec 2024-04-26T18:52:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:38 smithi119 ceph-mon[32752]: pgmap v1358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:38.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:38 smithi167 ceph-mon[32751]: pgmap v1358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:38 smithi090 ceph-mon[29180]: pgmap v1358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:39.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1443 sec 2024-04-26T18:52:40.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1444 sec 2024-04-26T18:52:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:40 smithi119 ceph-mon[32752]: pgmap v1359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:40.660 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1041.990646 secs 2024-04-26T18:52:40.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:40 smithi090 ceph-mon[29180]: pgmap v1359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:40.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1041.990646 secs 2024-04-26T18:52:40.690 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:40 smithi167 ceph-mon[32751]: pgmap v1359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:40.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1041.990646 secs 2024-04-26T18:52:41.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1445 sec 2024-04-26T18:52:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1042.656818 secs 2024-04-26T18:52:41.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1042.656818 secs 2024-04-26T18:52:41.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1042.656818 secs 2024-04-26T18:52:42.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1446 sec 2024-04-26T18:52:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:42 smithi119 ceph-mon[32752]: pgmap v1360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:42.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:42 smithi167 ceph-mon[32751]: pgmap v1360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:42 smithi090 ceph-mon[29180]: pgmap v1360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:43.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1447 sec 2024-04-26T18:52:44.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1448 sec 2024-04-26T18:52:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:44 smithi119 ceph-mon[32752]: pgmap v1361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:44 smithi167 ceph-mon[32751]: pgmap v1361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:44 smithi090 ceph-mon[29180]: pgmap v1361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:45.006 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1449 sec 2024-04-26T18:52:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1046.990732 secs 2024-04-26T18:52:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1046.990732 secs 2024-04-26T18:52:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1046.990732 secs 2024-04-26T18:52:46.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1450 sec 2024-04-26T18:52:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:46 smithi119 ceph-mon[32752]: pgmap v1362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:46.664 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1047.656928 secs 2024-04-26T18:52:46.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:46 smithi090 ceph-mon[29180]: pgmap v1362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:46.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1047.656928 secs 2024-04-26T18:52:46.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:46 smithi167 ceph-mon[32751]: pgmap v1362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:46.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1047.656928 secs 2024-04-26T18:52:47.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1451 sec 2024-04-26T18:52:48.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1452 sec 2024-04-26T18:52:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:48 smithi119 ceph-mon[32752]: pgmap v1363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:48 smithi167 ceph-mon[32751]: pgmap v1363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:48 smithi090 ceph-mon[29180]: pgmap v1363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:49.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1453 sec 2024-04-26T18:52:50.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1454 sec 2024-04-26T18:52:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:50 smithi119 ceph-mon[32752]: pgmap v1364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:50.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1051.990810 secs 2024-04-26T18:52:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:50 smithi167 ceph-mon[32751]: pgmap v1364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:50.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1051.990810 secs 2024-04-26T18:52:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:50 smithi090 ceph-mon[29180]: pgmap v1364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:50.713 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1051.990810 secs 2024-04-26T18:52:51.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1455 sec 2024-04-26T18:52:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1052.657052 secs 2024-04-26T18:52:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1052.657052 secs 2024-04-26T18:52:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1052.657052 secs 2024-04-26T18:52:52.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1456 sec 2024-04-26T18:52:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:52 smithi119 ceph-mon[32752]: pgmap v1365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:52 smithi167 ceph-mon[32751]: pgmap v1365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:52 smithi090 ceph-mon[29180]: pgmap v1365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:53.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1457 sec 2024-04-26T18:52:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:52:53.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:52:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:52:54.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1458 sec 2024-04-26T18:52:54.512 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:54 smithi090 ceph-mon[29180]: pgmap v1366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:54.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:54 smithi167 ceph-mon[32751]: pgmap v1366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:54.638 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.646 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.670 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:54 smithi119 ceph-mon[32752]: pgmap v1366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:52:54.677 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.686 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.750 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:54.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:55.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1459 sec 2024-04-26T18:52:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1056.990870 secs 2024-04-26T18:52:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1056.990870 secs 2024-04-26T18:52:55.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1056.990870 secs 2024-04-26T18:52:56.007 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1460 sec 2024-04-26T18:52:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: pgmap v1367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:56.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1057.657140 secs 2024-04-26T18:52:56.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:56.579 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:56.588 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:56.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:56.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:56.614 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:56.623 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:56.632 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:56.683 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:56.746 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:56.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: pgmap v1367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:56.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1057.657140 secs 2024-04-26T18:52:56.920 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.059 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.068 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.077 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.086 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.095 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.132 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: pgmap v1367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:52:57.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1057.657140 secs 2024-04-26T18:52:57.154 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.240 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.407 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1461 sec 2024-04-26T18:52:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:52:57.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:52:57.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.578 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.586 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.601 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:52:57.650 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:52:57.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.736 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:52:57.884 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:52:57.892 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:52:57.900 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:52:57.907 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: pgmap v1368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 894 B/s rd, 1 op/s 2024-04-26T18:52:57.915 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: pgmap v1369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:57.922 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:57.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:52:57.938 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:52:57.947 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:57.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:52:58.043 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:52:58.088 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:52:58.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:52:58.198 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:52:58.206 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:52:58.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: pgmap v1368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 894 B/s rd, 1 op/s 2024-04-26T18:52:58.222 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: pgmap v1369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:58.230 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:58.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T18:52:58.244 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:52:58.253 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.406 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T18:52:58.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T18:52:58.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:52:58.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T18:52:58.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:52:58.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:52:58.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:52:58.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: pgmap v1368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 894 B/s rd, 1 op/s 2024-04-26T18:52:58.618 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: pgmap v1369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:58.627 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:52:58.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1462 sec 2024-04-26T18:52:59.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1463 sec 2024-04-26T18:52:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:52:58 smithi119 ceph-mon[32752]: pgmap v1370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:52:58 smithi167 ceph-mon[32751]: pgmap v1370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:52:59.433 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:52:58 smithi090 ceph-mon[29180]: pgmap v1370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:00.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1464 sec 2024-04-26T18:53:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1061.990953 secs 2024-04-26T18:53:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1061.990953 secs 2024-04-26T18:53:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1061.990953 secs 2024-04-26T18:53:01.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1465 sec 2024-04-26T18:53:01.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1062.657287 secs 2024-04-26T18:53:01.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:01 smithi090 ceph-mon[29180]: pgmap v1371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1062.657287 secs 2024-04-26T18:53:01.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:01 smithi167 ceph-mon[32751]: pgmap v1371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1062.657287 secs 2024-04-26T18:53:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:01 smithi119 ceph-mon[32752]: pgmap v1371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:02.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1466 sec 2024-04-26T18:53:03.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1467 sec 2024-04-26T18:53:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:03 smithi119 ceph-mon[32752]: pgmap v1372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:03 smithi167 ceph-mon[32751]: pgmap v1372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:03 smithi090 ceph-mon[29180]: pgmap v1372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:04.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1468 sec 2024-04-26T18:53:05.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1469 sec 2024-04-26T18:53:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:05 smithi167 ceph-mon[32751]: pgmap v1373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:05.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1066.991017 secs 2024-04-26T18:53:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:05 smithi090 ceph-mon[29180]: pgmap v1373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:05.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1066.991017 secs 2024-04-26T18:53:06.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1470 sec 2024-04-26T18:53:06.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:05 smithi119 ceph-mon[32752]: pgmap v1373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:53:06.101 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1066.991017 secs 2024-04-26T18:53:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1067.657405 secs 2024-04-26T18:53:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1067.657405 secs 2024-04-26T18:53:06.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1067.657405 secs 2024-04-26T18:53:07.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1471 sec 2024-04-26T18:53:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:07 smithi167 ceph-mon[32751]: pgmap v1374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:53:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:07 smithi090 ceph-mon[29180]: pgmap v1374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:53:08.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1472 sec 2024-04-26T18:53:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:07 smithi119 ceph-mon[32752]: pgmap v1374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:53:09.008 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1473 sec 2024-04-26T18:53:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:09 smithi167 ceph-mon[32751]: pgmap v1375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:09 smithi090 ceph-mon[29180]: pgmap v1375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:10.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1474 sec 2024-04-26T18:53:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:09 smithi119 ceph-mon[32752]: pgmap v1375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1071.991076 secs 2024-04-26T18:53:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1072.657556 secs 2024-04-26T18:53:11.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1475 sec 2024-04-26T18:53:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1071.991076 secs 2024-04-26T18:53:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1072.657556 secs 2024-04-26T18:53:11.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1071.991076 secs 2024-04-26T18:53:11.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1072.657556 secs 2024-04-26T18:53:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:11 smithi090 ceph-mon[29180]: pgmap v1376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:12.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1476 sec 2024-04-26T18:53:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:11 smithi119 ceph-mon[32752]: pgmap v1376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:11 smithi167 ceph-mon[32751]: pgmap v1376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:13.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1477 sec 2024-04-26T18:53:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:13 smithi090 ceph-mon[29180]: pgmap v1377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:14.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1478 sec 2024-04-26T18:53:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:13 smithi119 ceph-mon[32752]: pgmap v1377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:14.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:13 smithi167 ceph-mon[32751]: pgmap v1377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:15.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1479 sec 2024-04-26T18:53:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:15 smithi090 ceph-mon[29180]: pgmap v1378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:15.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1076.991139 secs 2024-04-26T18:53:15.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1077.657691 secs 2024-04-26T18:53:16.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1480 sec 2024-04-26T18:53:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:15 smithi119 ceph-mon[32752]: pgmap v1378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:16.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1076.991139 secs 2024-04-26T18:53:16.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1077.657691 secs 2024-04-26T18:53:16.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:15 smithi167 ceph-mon[32751]: pgmap v1378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:16.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1076.991139 secs 2024-04-26T18:53:16.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1077.657691 secs 2024-04-26T18:53:17.009 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1481 sec 2024-04-26T18:53:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:17 smithi090 ceph-mon[29180]: pgmap v1379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:18.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1482 sec 2024-04-26T18:53:18.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:17 smithi119 ceph-mon[32752]: pgmap v1379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:18.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:17 smithi167 ceph-mon[32751]: pgmap v1379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:19.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1483 sec 2024-04-26T18:53:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:19 smithi090 ceph-mon[29180]: pgmap v1380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:20.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1484 sec 2024-04-26T18:53:20.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:19 smithi119 ceph-mon[32752]: pgmap v1380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:19 smithi167 ceph-mon[32751]: pgmap v1380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1081.991196 secs 2024-04-26T18:53:20.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1082.657813 secs 2024-04-26T18:53:21.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1485 sec 2024-04-26T18:53:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1081.991196 secs 2024-04-26T18:53:21.075 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1082.657813 secs 2024-04-26T18:53:21.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1081.991196 secs 2024-04-26T18:53:21.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1082.657813 secs 2024-04-26T18:53:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:21 smithi090 ceph-mon[29180]: pgmap v1381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:22.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1486 sec 2024-04-26T18:53:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:21 smithi119 ceph-mon[32752]: pgmap v1381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:21 smithi167 ceph-mon[32751]: pgmap v1381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:23.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1487 sec 2024-04-26T18:53:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:23 smithi090 ceph-mon[29180]: pgmap v1382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:24.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1488 sec 2024-04-26T18:53:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:23 smithi119 ceph-mon[32752]: pgmap v1382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:23 smithi167 ceph-mon[32751]: pgmap v1382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:25.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1489 sec 2024-04-26T18:53:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:25 smithi090 ceph-mon[29180]: pgmap v1383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:25.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1086.991257 secs 2024-04-26T18:53:25.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1087.657926 secs 2024-04-26T18:53:26.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1490 sec 2024-04-26T18:53:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:25 smithi119 ceph-mon[32752]: pgmap v1383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:26.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1086.991257 secs 2024-04-26T18:53:26.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1087.657926 secs 2024-04-26T18:53:26.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:25 smithi167 ceph-mon[32751]: pgmap v1383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:26.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1086.991257 secs 2024-04-26T18:53:26.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1087.657926 secs 2024-04-26T18:53:27.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1491 sec 2024-04-26T18:53:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:27 smithi090 ceph-mon[29180]: pgmap v1384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:28.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1492 sec 2024-04-26T18:53:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:27 smithi119 ceph-mon[32752]: pgmap v1384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:27 smithi167 ceph-mon[32751]: pgmap v1384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:28.606 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:53:28.605+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T18:53:28.613 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:53:28.606+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 13 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T18:53:28.620 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:53:28.617+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac003690 con 0x56302893f0e0 2024-04-26T18:53:28.628 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:53:28.617+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 13 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac003690 con 0x56302893f0e0 2024-04-26T18:53:29.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1493 sec 2024-04-26T18:53:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:29 smithi090 ceph-mon[29180]: pgmap v1385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:30.010 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1494 sec 2024-04-26T18:53:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:29 smithi119 ceph-mon[32752]: pgmap v1385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:30.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:29 smithi167 ceph-mon[32751]: pgmap v1385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1091.991342 secs 2024-04-26T18:53:30.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1092.658066 secs 2024-04-26T18:53:31.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1495 sec 2024-04-26T18:53:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1091.991342 secs 2024-04-26T18:53:31.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1092.658066 secs 2024-04-26T18:53:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1091.991342 secs 2024-04-26T18:53:31.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1092.658066 secs 2024-04-26T18:53:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:31 smithi090 ceph-mon[29180]: pgmap v1386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:31.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:31 smithi167 ceph-mon[32751]: pgmap v1386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:32.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1496 sec 2024-04-26T18:53:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:31 smithi119 ceph-mon[32752]: pgmap v1386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:33.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1497 sec 2024-04-26T18:53:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:33 smithi090 ceph-mon[29180]: pgmap v1387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:34.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1498 sec 2024-04-26T18:53:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:33 smithi119 ceph-mon[32752]: pgmap v1387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:33 smithi167 ceph-mon[32751]: pgmap v1387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:35.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1499 sec 2024-04-26T18:53:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:35 smithi090 ceph-mon[29180]: pgmap v1388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:35.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1096.991430 secs 2024-04-26T18:53:35.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:35 smithi090 ceph-mon[29180]: 10 slow requests, 1 included below; oldest blocked for > 1097.658180 secs 2024-04-26T18:53:35.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:35 smithi090 ceph-mon[29180]: slow request 724.527193 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:53:36.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1500 sec 2024-04-26T18:53:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:35 smithi119 ceph-mon[32752]: pgmap v1388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:36.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1096.991430 secs 2024-04-26T18:53:36.078 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:35 smithi119 ceph-mon[32752]: 10 slow requests, 1 included below; oldest blocked for > 1097.658180 secs 2024-04-26T18:53:36.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:35 smithi119 ceph-mon[32752]: slow request 724.527193 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:53:36.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:35 smithi167 ceph-mon[32751]: pgmap v1388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:36.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1096.991430 secs 2024-04-26T18:53:36.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:35 smithi167 ceph-mon[32751]: 10 slow requests, 1 included below; oldest blocked for > 1097.658180 secs 2024-04-26T18:53:36.133 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:35 smithi167 ceph-mon[32751]: slow request 724.527193 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:53:37.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1501 sec 2024-04-26T18:53:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:37 smithi090 ceph-mon[29180]: pgmap v1389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:38.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1502 sec 2024-04-26T18:53:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:37 smithi119 ceph-mon[32752]: pgmap v1389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:38.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:37 smithi167 ceph-mon[32751]: pgmap v1389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:39.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1503 sec 2024-04-26T18:53:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:39 smithi090 ceph-mon[29180]: pgmap v1390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:40.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1504 sec 2024-04-26T18:53:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:39 smithi119 ceph-mon[32752]: pgmap v1390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:39 smithi167 ceph-mon[32751]: pgmap v1390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:41.011 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1505 sec 2024-04-26T18:53:41.067 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1101.991516 secs 2024-04-26T18:53:41.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1102.658295 secs 2024-04-26T18:53:41.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1101.991516 secs 2024-04-26T18:53:41.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1102.658295 secs 2024-04-26T18:53:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1101.991516 secs 2024-04-26T18:53:41.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1102.658295 secs 2024-04-26T18:53:42.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1506 sec 2024-04-26T18:53:42.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:41 smithi119 ceph-mon[32752]: pgmap v1391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:42.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:41 smithi167 ceph-mon[32751]: pgmap v1391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:41 smithi090 ceph-mon[29180]: pgmap v1391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:43.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1507 sec 2024-04-26T18:53:44.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1508 sec 2024-04-26T18:53:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:43 smithi119 ceph-mon[32752]: pgmap v1392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:43 smithi167 ceph-mon[32751]: pgmap v1392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:43 smithi090 ceph-mon[29180]: pgmap v1392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:45.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1509 sec 2024-04-26T18:53:46.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1510 sec 2024-04-26T18:53:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:45 smithi119 ceph-mon[32752]: pgmap v1393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:46.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1106.991601 secs 2024-04-26T18:53:46.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1107.658410 secs 2024-04-26T18:53:46.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:45 smithi167 ceph-mon[32751]: pgmap v1393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:46.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1106.991601 secs 2024-04-26T18:53:46.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1107.658410 secs 2024-04-26T18:53:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:45 smithi090 ceph-mon[29180]: pgmap v1393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:46.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1106.991601 secs 2024-04-26T18:53:46.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1107.658410 secs 2024-04-26T18:53:47.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1511 sec 2024-04-26T18:53:48.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1512 sec 2024-04-26T18:53:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:47 smithi119 ceph-mon[32752]: pgmap v1394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:48.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:47 smithi167 ceph-mon[32751]: pgmap v1394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:47 smithi090 ceph-mon[29180]: pgmap v1394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:49.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1513 sec 2024-04-26T18:53:50.012 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1514 sec 2024-04-26T18:53:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:49 smithi119 ceph-mon[32752]: pgmap v1395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:49 smithi167 ceph-mon[32751]: pgmap v1395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:49 smithi090 ceph-mon[29180]: pgmap v1395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:51.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1515 sec 2024-04-26T18:53:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1111.991688 secs 2024-04-26T18:53:51.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1112.658530 secs 2024-04-26T18:53:51.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1111.991688 secs 2024-04-26T18:53:51.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1112.658530 secs 2024-04-26T18:53:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1111.991688 secs 2024-04-26T18:53:51.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1112.658530 secs 2024-04-26T18:53:52.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1516 sec 2024-04-26T18:53:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:51 smithi119 ceph-mon[32752]: pgmap v1396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:51 smithi167 ceph-mon[32751]: pgmap v1396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:51 smithi090 ceph-mon[29180]: pgmap v1396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:53.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1517 sec 2024-04-26T18:53:54.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1518 sec 2024-04-26T18:53:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:53 smithi119 ceph-mon[32752]: pgmap v1397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:54.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:53 smithi090 ceph-mon[29180]: pgmap v1397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:54.148 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:53 smithi167 ceph-mon[32751]: pgmap v1397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:55.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1519 sec 2024-04-26T18:53:56.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1520 sec 2024-04-26T18:53:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:55 smithi119 ceph-mon[32752]: pgmap v1398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:56.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1116.991776 secs 2024-04-26T18:53:56.072 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1117.658666 secs 2024-04-26T18:53:56.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:55 smithi167 ceph-mon[32751]: pgmap v1398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:56.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1116.991776 secs 2024-04-26T18:53:56.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1117.658666 secs 2024-04-26T18:53:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:55 smithi090 ceph-mon[29180]: pgmap v1398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:53:56.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1116.991776 secs 2024-04-26T18:53:56.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1117.658666 secs 2024-04-26T18:53:57.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1521 sec 2024-04-26T18:53:57.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:53:57.222 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:53:57.229 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:53:58.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1522 sec 2024-04-26T18:53:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:58 smithi090 ceph-mon[29180]: pgmap v1399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:58.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.406 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:53:58.478 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:53:58.486 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:58 smithi119 ceph-mon[32752]: pgmap v1399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:58.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:53:58.582 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:53:58.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:58 smithi167 ceph-mon[32751]: pgmap v1399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:53:58.682 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:58.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:53:58.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:53:58.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:53:59.013 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1523 sec 2024-04-26T18:53:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:59 smithi119 ceph-mon[32752]: pgmap v1400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T18:53:59.577 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:53:59 smithi119 ceph-mon[32752]: pgmap v1401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T18:53:59.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:59 smithi167 ceph-mon[32751]: pgmap v1400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T18:53:59.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:53:59 smithi167 ceph-mon[32751]: pgmap v1401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T18:53:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:59 smithi090 ceph-mon[29180]: pgmap v1400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T18:53:59.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:53:59 smithi090 ceph-mon[29180]: pgmap v1401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T18:54:00.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1524 sec 2024-04-26T18:54:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1121.991828 secs 2024-04-26T18:54:00.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1121.991828 secs 2024-04-26T18:54:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1121.991828 secs 2024-04-26T18:54:01.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1525 sec 2024-04-26T18:54:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:01 smithi167 ceph-mon[32751]: pgmap v1402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:01.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1122.658782 secs 2024-04-26T18:54:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:01 smithi119 ceph-mon[32752]: pgmap v1402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1122.658782 secs 2024-04-26T18:54:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:01 smithi090 ceph-mon[29180]: pgmap v1402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:01.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1122.658782 secs 2024-04-26T18:54:02.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1526 sec 2024-04-26T18:54:03.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1527 sec 2024-04-26T18:54:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:03 smithi119 ceph-mon[32752]: pgmap v1403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:03 smithi167 ceph-mon[32751]: pgmap v1403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:03 smithi090 ceph-mon[29180]: pgmap v1403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:04.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1528 sec 2024-04-26T18:54:05.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1529 sec 2024-04-26T18:54:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:05 smithi119 ceph-mon[32752]: pgmap v1404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:05.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1126.991891 secs 2024-04-26T18:54:05.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:05 smithi167 ceph-mon[32751]: pgmap v1404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:05.611 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1126.991891 secs 2024-04-26T18:54:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:05 smithi090 ceph-mon[29180]: pgmap v1404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:05.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1126.991891 secs 2024-04-26T18:54:06.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1530 sec 2024-04-26T18:54:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1127.658879 secs 2024-04-26T18:54:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1127.658879 secs 2024-04-26T18:54:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1127.658879 secs 2024-04-26T18:54:07.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1531 sec 2024-04-26T18:54:07.459 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:07 smithi119 ceph-mon[32752]: pgmap v1405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:07.987 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:07 smithi167 ceph-mon[32751]: pgmap v1405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:08.057 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:07 smithi090 ceph-mon[29180]: pgmap v1405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:54:08.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1532 sec 2024-04-26T18:54:09.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1533 sec 2024-04-26T18:54:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:09 smithi119 ceph-mon[32752]: pgmap v1406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:54:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:09 smithi167 ceph-mon[32751]: pgmap v1406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:54:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:09 smithi090 ceph-mon[29180]: pgmap v1406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:54:10.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1534 sec 2024-04-26T18:54:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1131.991953 secs 2024-04-26T18:54:10.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1131.991953 secs 2024-04-26T18:54:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1131.991953 secs 2024-04-26T18:54:11.014 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1535 sec 2024-04-26T18:54:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:11 smithi119 ceph-mon[32752]: pgmap v1407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:11.581 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1132.658951 secs 2024-04-26T18:54:11.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:11 smithi167 ceph-mon[32751]: pgmap v1407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:11.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1132.658951 secs 2024-04-26T18:54:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:11 smithi090 ceph-mon[29180]: pgmap v1407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:11.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1132.658951 secs 2024-04-26T18:54:12.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1536 sec 2024-04-26T18:54:13.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1537 sec 2024-04-26T18:54:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:13 smithi119 ceph-mon[32752]: pgmap v1408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:13 smithi167 ceph-mon[32751]: pgmap v1408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:13 smithi090 ceph-mon[29180]: pgmap v1408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:14.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1538 sec 2024-04-26T18:54:15.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1539 sec 2024-04-26T18:54:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:15 smithi119 ceph-mon[32752]: pgmap v1409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:15.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1136.992015 secs 2024-04-26T18:54:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:15 smithi167 ceph-mon[32751]: pgmap v1409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:15.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1136.992015 secs 2024-04-26T18:54:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:15 smithi090 ceph-mon[29180]: pgmap v1409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:15.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1136.992015 secs 2024-04-26T18:54:16.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1540 sec 2024-04-26T18:54:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:16 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1137.659050 secs 2024-04-26T18:54:16.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:16 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1137.659050 secs 2024-04-26T18:54:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:16 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1137.659050 secs 2024-04-26T18:54:17.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1541 sec 2024-04-26T18:54:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:17 smithi119 ceph-mon[32752]: pgmap v1410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:17 smithi167 ceph-mon[32751]: pgmap v1410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:17 smithi090 ceph-mon[29180]: pgmap v1410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:18.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1542 sec 2024-04-26T18:54:19.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1543 sec 2024-04-26T18:54:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:19 smithi119 ceph-mon[32752]: pgmap v1411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:19 smithi167 ceph-mon[32751]: pgmap v1411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:19.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:19 smithi090 ceph-mon[29180]: pgmap v1411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:20.015 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1544 sec 2024-04-26T18:54:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1141.992076 secs 2024-04-26T18:54:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1141.992076 secs 2024-04-26T18:54:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1141.992076 secs 2024-04-26T18:54:21.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1545 sec 2024-04-26T18:54:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:21 smithi119 ceph-mon[32752]: pgmap v1412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:21.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1142.659191 secs 2024-04-26T18:54:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:21 smithi167 ceph-mon[32751]: pgmap v1412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:21.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1142.659191 secs 2024-04-26T18:54:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:21 smithi090 ceph-mon[29180]: pgmap v1412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:21.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1142.659191 secs 2024-04-26T18:54:22.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1546 sec 2024-04-26T18:54:23.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1547 sec 2024-04-26T18:54:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:23 smithi119 ceph-mon[32752]: pgmap v1413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:23 smithi167 ceph-mon[32751]: pgmap v1413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:23 smithi090 ceph-mon[29180]: pgmap v1413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:24.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1548 sec 2024-04-26T18:54:25.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1549 sec 2024-04-26T18:54:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:25 smithi119 ceph-mon[32752]: pgmap v1414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:25.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1146.992137 secs 2024-04-26T18:54:25.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:25 smithi167 ceph-mon[32751]: pgmap v1414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:25.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1146.992137 secs 2024-04-26T18:54:25.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:25 smithi090 ceph-mon[29180]: pgmap v1414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:25.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1146.992137 secs 2024-04-26T18:54:26.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1550 sec 2024-04-26T18:54:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:26 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1147.659308 secs 2024-04-26T18:54:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:26 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1147.659308 secs 2024-04-26T18:54:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:26 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1147.659308 secs 2024-04-26T18:54:27.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1551 sec 2024-04-26T18:54:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:27 smithi119 ceph-mon[32752]: pgmap v1415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:27 smithi167 ceph-mon[32751]: pgmap v1415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:27 smithi090 ceph-mon[29180]: pgmap v1415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:28.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1552 sec 2024-04-26T18:54:29.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1553 sec 2024-04-26T18:54:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:29 smithi119 ceph-mon[32752]: pgmap v1416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:29.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:29 smithi167 ceph-mon[32751]: pgmap v1416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:29 smithi090 ceph-mon[29180]: pgmap v1416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:30.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1554 sec 2024-04-26T18:54:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1151.992196 secs 2024-04-26T18:54:30.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1151.992196 secs 2024-04-26T18:54:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1151.992196 secs 2024-04-26T18:54:31.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1555 sec 2024-04-26T18:54:31.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:31 smithi167 ceph-mon[32751]: pgmap v1417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:31.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 1152.659424 secs 2024-04-26T18:54:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:31 smithi119 ceph-mon[32752]: pgmap v1417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:31.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 1152.659424 secs 2024-04-26T18:54:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:31 smithi090 ceph-mon[29180]: pgmap v1417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:31.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 1152.659424 secs 2024-04-26T18:54:32.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1556 sec 2024-04-26T18:54:33.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1557 sec 2024-04-26T18:54:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:33 smithi119 ceph-mon[32752]: pgmap v1418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:33.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:33 smithi167 ceph-mon[32751]: pgmap v1418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:33.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:33 smithi090 ceph-mon[29180]: pgmap v1418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:34.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1558 sec 2024-04-26T18:54:35.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1559 sec 2024-04-26T18:54:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:35 smithi119 ceph-mon[32752]: pgmap v1419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:35.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1156.992258 secs 2024-04-26T18:54:35.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:35 smithi167 ceph-mon[32751]: pgmap v1419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:35.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1156.992258 secs 2024-04-26T18:54:35.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:35 smithi090 ceph-mon[29180]: pgmap v1419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:35.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1156.992258 secs 2024-04-26T18:54:36.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1560 sec 2024-04-26T18:54:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:36 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 1157.659541 secs 2024-04-26T18:54:36.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:36 smithi119 ceph-mon[32752]: slow request 184.351306 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:54:36.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:36 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 1157.659541 secs 2024-04-26T18:54:36.613 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:36 smithi167 ceph-mon[32751]: slow request 184.351306 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:54:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:36 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 1157.659541 secs 2024-04-26T18:54:36.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:36 smithi090 ceph-mon[29180]: slow request 184.351306 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:54:37.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1561 sec 2024-04-26T18:54:37.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:37 smithi090 ceph-mon[29180]: pgmap v1420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:37.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:37 smithi119 ceph-mon[32752]: pgmap v1420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:37.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:37 smithi167 ceph-mon[32751]: pgmap v1420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:38.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1562 sec 2024-04-26T18:54:39.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1563 sec 2024-04-26T18:54:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:39 smithi119 ceph-mon[32752]: pgmap v1421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:39.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:54:39.583 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:54:39.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:39 smithi167 ceph-mon[32751]: pgmap v1421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:39.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:54:39.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:54:39.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:39 smithi090 ceph-mon[29180]: pgmap v1421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:39.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:54:39.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:54:40.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1564 sec 2024-04-26T18:54:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1161.992317 secs 2024-04-26T18:54:40.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1161.992317 secs 2024-04-26T18:54:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1161.992317 secs 2024-04-26T18:54:41.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1565 sec 2024-04-26T18:54:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:41 smithi119 ceph-mon[32752]: pgmap v1422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:41.598 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1162.659614 secs 2024-04-26T18:54:41.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:41 smithi167 ceph-mon[32751]: pgmap v1422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:41.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1162.659614 secs 2024-04-26T18:54:41.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:41 smithi090 ceph-mon[29180]: pgmap v1422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:41.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1162.659614 secs 2024-04-26T18:54:42.017 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1566 sec 2024-04-26T18:54:43.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1567 sec 2024-04-26T18:54:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:43 smithi119 ceph-mon[32752]: pgmap v1423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:43.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:43 smithi167 ceph-mon[32751]: pgmap v1423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:43 smithi090 ceph-mon[29180]: pgmap v1423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:44.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1568 sec 2024-04-26T18:54:45.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1569 sec 2024-04-26T18:54:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:45 smithi119 ceph-mon[32752]: pgmap v1424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:45.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1166.992401 secs 2024-04-26T18:54:45.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:45 smithi167 ceph-mon[32751]: pgmap v1424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:45.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1166.992401 secs 2024-04-26T18:54:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:45 smithi090 ceph-mon[29180]: pgmap v1424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1166.992401 secs 2024-04-26T18:54:46.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1570 sec 2024-04-26T18:54:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1167.659752 secs 2024-04-26T18:54:46.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1167.659752 secs 2024-04-26T18:54:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1167.659752 secs 2024-04-26T18:54:47.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1571 sec 2024-04-26T18:54:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:47 smithi119 ceph-mon[32752]: pgmap v1425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:47 smithi167 ceph-mon[32751]: pgmap v1425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:47 smithi090 ceph-mon[29180]: pgmap v1425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:48.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1572 sec 2024-04-26T18:54:49.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1573 sec 2024-04-26T18:54:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:49 smithi119 ceph-mon[32752]: pgmap v1426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:49.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:49 smithi167 ceph-mon[32751]: pgmap v1426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:49 smithi090 ceph-mon[29180]: pgmap v1426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T18:54:50.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1574 sec 2024-04-26T18:54:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1171.992505 secs 2024-04-26T18:54:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1171.992505 secs 2024-04-26T18:54:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1171.992505 secs 2024-04-26T18:54:51.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1575 sec 2024-04-26T18:54:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:51 smithi119 ceph-mon[32752]: pgmap v1427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:51.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1172.659869 secs 2024-04-26T18:54:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:51 smithi167 ceph-mon[32751]: pgmap v1427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:51.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1172.659869 secs 2024-04-26T18:54:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:51 smithi090 ceph-mon[29180]: pgmap v1427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:51.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1172.659869 secs 2024-04-26T18:54:52.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1576 sec 2024-04-26T18:54:53.018 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1577 sec 2024-04-26T18:54:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:53 smithi119 ceph-mon[32752]: pgmap v1428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:53 smithi167 ceph-mon[32751]: pgmap v1428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:53 smithi090 ceph-mon[29180]: pgmap v1428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:54.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1578 sec 2024-04-26T18:54:55.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1579 sec 2024-04-26T18:54:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:55 smithi119 ceph-mon[32752]: pgmap v1429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:55.582 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1176.992588 secs 2024-04-26T18:54:55.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:55 smithi090 ceph-mon[29180]: pgmap v1429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:55.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1176.992588 secs 2024-04-26T18:54:55.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:55 smithi167 ceph-mon[32751]: pgmap v1429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:54:55.678 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1176.992588 secs 2024-04-26T18:54:56.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1580 sec 2024-04-26T18:54:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1177.659988 secs 2024-04-26T18:54:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1177.659988 secs 2024-04-26T18:54:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1177.659988 secs 2024-04-26T18:54:57.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1581 sec 2024-04-26T18:54:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:57 smithi119 ceph-mon[32752]: pgmap v1430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:57 smithi167 ceph-mon[32751]: pgmap v1430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:57 smithi090 ceph-mon[29180]: pgmap v1430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:58.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1582 sec 2024-04-26T18:54:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:54:58.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:54:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:54:59.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1583 sec 2024-04-26T18:54:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:59 smithi119 ceph-mon[32752]: pgmap v1431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:59.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:54:59.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:54:59.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:54:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:54:59.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:59 smithi167 ceph-mon[32751]: pgmap v1431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:59.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:54:59.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:54:59.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:54:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:54:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:59 smithi090 ceph-mon[29180]: pgmap v1431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:54:59.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:54:59.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:54:59.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:54:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:55:00.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1584 sec 2024-04-26T18:55:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:00 smithi119 ceph-mon[32752]: pgmap v1432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T18:55:00.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:00 smithi119 ceph-mon[32752]: pgmap v1433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:00.598 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1181.992670 secs 2024-04-26T18:55:00.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:00 smithi167 ceph-mon[32751]: pgmap v1432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T18:55:00.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:00 smithi167 ceph-mon[32751]: pgmap v1433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:00.687 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1181.992670 secs 2024-04-26T18:55:00.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:00 smithi090 ceph-mon[29180]: pgmap v1432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T18:55:00.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:00 smithi090 ceph-mon[29180]: pgmap v1433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:00.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1181.992670 secs 2024-04-26T18:55:01.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1585 sec 2024-04-26T18:55:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1182.660114 secs 2024-04-26T18:55:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1182.660114 secs 2024-04-26T18:55:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1182.660114 secs 2024-04-26T18:55:02.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1586 sec 2024-04-26T18:55:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:02 smithi119 ceph-mon[32752]: pgmap v1434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:02 smithi167 ceph-mon[32751]: pgmap v1434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:02 smithi090 ceph-mon[29180]: pgmap v1434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:03.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1587 sec 2024-04-26T18:55:04.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1588 sec 2024-04-26T18:55:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:04 smithi119 ceph-mon[32752]: pgmap v1435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T18:55:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:04 smithi167 ceph-mon[32751]: pgmap v1435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T18:55:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:04 smithi090 ceph-mon[29180]: pgmap v1435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T18:55:05.019 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1589 sec 2024-04-26T18:55:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1186.992756 secs 2024-04-26T18:55:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1186.992756 secs 2024-04-26T18:55:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1186.992756 secs 2024-04-26T18:55:06.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1590 sec 2024-04-26T18:55:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:06 smithi119 ceph-mon[32752]: pgmap v1436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:06.577 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1187.660237 secs 2024-04-26T18:55:06.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:06 smithi167 ceph-mon[32751]: pgmap v1436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:06.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1187.660237 secs 2024-04-26T18:55:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:06 smithi090 ceph-mon[29180]: pgmap v1436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:06.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1187.660237 secs 2024-04-26T18:55:07.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1591 sec 2024-04-26T18:55:08.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1592 sec 2024-04-26T18:55:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:08 smithi119 ceph-mon[32752]: pgmap v1437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:08 smithi167 ceph-mon[32751]: pgmap v1437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:08 smithi090 ceph-mon[29180]: pgmap v1437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:55:09.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1593 sec 2024-04-26T18:55:10.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1594 sec 2024-04-26T18:55:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:10 smithi167 ceph-mon[32751]: pgmap v1438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:55:10.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1191.992837 secs 2024-04-26T18:55:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:10 smithi090 ceph-mon[29180]: pgmap v1438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:55:10.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1191.992837 secs 2024-04-26T18:55:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:10 smithi119 ceph-mon[32752]: pgmap v1438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:55:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1191.992837 secs 2024-04-26T18:55:11.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1595 sec 2024-04-26T18:55:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1192.660309 secs 2024-04-26T18:55:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1192.660309 secs 2024-04-26T18:55:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1192.660309 secs 2024-04-26T18:55:12.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1596 sec 2024-04-26T18:55:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:12 smithi167 ceph-mon[32751]: pgmap v1439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:12 smithi090 ceph-mon[29180]: pgmap v1439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:12 smithi119 ceph-mon[32752]: pgmap v1439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:13.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1597 sec 2024-04-26T18:55:14.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1598 sec 2024-04-26T18:55:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:14 smithi167 ceph-mon[32751]: pgmap v1440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:14 smithi090 ceph-mon[29180]: pgmap v1440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:14 smithi119 ceph-mon[32752]: pgmap v1440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:15.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1599 sec 2024-04-26T18:55:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1196.992897 secs 2024-04-26T18:55:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1196.992897 secs 2024-04-26T18:55:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1196.992897 secs 2024-04-26T18:55:16.020 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1600 sec 2024-04-26T18:55:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:16 smithi090 ceph-mon[29180]: pgmap v1441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:16.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1197.660433 secs 2024-04-26T18:55:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:16 smithi119 ceph-mon[32752]: pgmap v1441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:16.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1197.660433 secs 2024-04-26T18:55:16.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:16 smithi167 ceph-mon[32751]: pgmap v1441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:16.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1197.660433 secs 2024-04-26T18:55:17.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1601 sec 2024-04-26T18:55:18.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1602 sec 2024-04-26T18:55:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:18 smithi090 ceph-mon[29180]: pgmap v1442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:18 smithi119 ceph-mon[32752]: pgmap v1442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:18 smithi167 ceph-mon[32751]: pgmap v1442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:19.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1603 sec 2024-04-26T18:55:20.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1604 sec 2024-04-26T18:55:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:20 smithi090 ceph-mon[29180]: pgmap v1443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:20.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1201.992957 secs 2024-04-26T18:55:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:20 smithi119 ceph-mon[32752]: pgmap v1443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:20.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1201.992957 secs 2024-04-26T18:55:20.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:20 smithi167 ceph-mon[32751]: pgmap v1443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:20.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1201.992957 secs 2024-04-26T18:55:21.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1605 sec 2024-04-26T18:55:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1202.660563 secs 2024-04-26T18:55:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1202.660563 secs 2024-04-26T18:55:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1202.660563 secs 2024-04-26T18:55:22.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1606 sec 2024-04-26T18:55:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:22 smithi090 ceph-mon[29180]: pgmap v1444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:22 smithi119 ceph-mon[32752]: pgmap v1444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:22 smithi167 ceph-mon[32751]: pgmap v1444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:23.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1607 sec 2024-04-26T18:55:24.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1608 sec 2024-04-26T18:55:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:24 smithi090 ceph-mon[29180]: pgmap v1445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:24 smithi119 ceph-mon[32752]: pgmap v1445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:24 smithi167 ceph-mon[32751]: pgmap v1445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:25.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1609 sec 2024-04-26T18:55:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1206.993043 secs 2024-04-26T18:55:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1206.993043 secs 2024-04-26T18:55:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1206.993043 secs 2024-04-26T18:55:26.021 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1610 sec 2024-04-26T18:55:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:26 smithi090 ceph-mon[29180]: pgmap v1446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:26.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1207.660630 secs 2024-04-26T18:55:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:26 smithi119 ceph-mon[32752]: pgmap v1446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:26.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1207.660630 secs 2024-04-26T18:55:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:26 smithi167 ceph-mon[32751]: pgmap v1446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:26.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1207.660630 secs 2024-04-26T18:55:27.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1611 sec 2024-04-26T18:55:28.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1612 sec 2024-04-26T18:55:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:28 smithi090 ceph-mon[29180]: pgmap v1447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:28 smithi119 ceph-mon[32752]: pgmap v1447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:28 smithi167 ceph-mon[32751]: pgmap v1447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:29.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1613 sec 2024-04-26T18:55:30.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1614 sec 2024-04-26T18:55:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:30 smithi090 ceph-mon[29180]: pgmap v1448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:30.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1211.993126 secs 2024-04-26T18:55:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:30 smithi119 ceph-mon[32752]: pgmap v1448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:30.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1211.993126 secs 2024-04-26T18:55:30.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:30 smithi167 ceph-mon[32751]: pgmap v1448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:30.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1211.993126 secs 2024-04-26T18:55:31.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1615 sec 2024-04-26T18:55:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1212.660748 secs 2024-04-26T18:55:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1212.660748 secs 2024-04-26T18:55:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1212.660748 secs 2024-04-26T18:55:32.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1616 sec 2024-04-26T18:55:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:32 smithi090 ceph-mon[29180]: pgmap v1449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:32 smithi119 ceph-mon[32752]: pgmap v1449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:32 smithi167 ceph-mon[32751]: pgmap v1449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:33.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1617 sec 2024-04-26T18:55:34.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1618 sec 2024-04-26T18:55:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:34 smithi090 ceph-mon[29180]: pgmap v1450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:34 smithi119 ceph-mon[32752]: pgmap v1450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:34 smithi167 ceph-mon[32751]: pgmap v1450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:35.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1619 sec 2024-04-26T18:55:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1216.993185 secs 2024-04-26T18:55:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1216.993185 secs 2024-04-26T18:55:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1216.993185 secs 2024-04-26T18:55:36.022 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1620 sec 2024-04-26T18:55:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:36 smithi090 ceph-mon[29180]: pgmap v1451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:36.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1217.660813 secs 2024-04-26T18:55:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:36 smithi119 ceph-mon[32752]: pgmap v1451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:36.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1217.660813 secs 2024-04-26T18:55:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:36 smithi167 ceph-mon[32751]: pgmap v1451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:36.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1217.660813 secs 2024-04-26T18:55:37.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1621 sec 2024-04-26T18:55:38.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1622 sec 2024-04-26T18:55:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:38 smithi119 ceph-mon[32752]: pgmap v1452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:38 smithi167 ceph-mon[32751]: pgmap v1452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:38 smithi090 ceph-mon[29180]: pgmap v1452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:39.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1623 sec 2024-04-26T18:55:40.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1624 sec 2024-04-26T18:55:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:40 smithi119 ceph-mon[32752]: pgmap v1453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:40.819 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1221.993267 secs 2024-04-26T18:55:40.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:40 smithi167 ceph-mon[32751]: pgmap v1453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:40.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1221.993267 secs 2024-04-26T18:55:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:40 smithi090 ceph-mon[29180]: pgmap v1453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:40.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1221.993267 secs 2024-04-26T18:55:41.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1625 sec 2024-04-26T18:55:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1222.660877 secs 2024-04-26T18:55:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1222.660877 secs 2024-04-26T18:55:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1222.660877 secs 2024-04-26T18:55:42.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1626 sec 2024-04-26T18:55:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:42 smithi119 ceph-mon[32752]: pgmap v1454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:42 smithi167 ceph-mon[32751]: pgmap v1454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:42 smithi090 ceph-mon[29180]: pgmap v1454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:43.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1627 sec 2024-04-26T18:55:44.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1628 sec 2024-04-26T18:55:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:44 smithi119 ceph-mon[32752]: pgmap v1455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:44 smithi167 ceph-mon[32751]: pgmap v1455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:44 smithi090 ceph-mon[29180]: pgmap v1455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:45.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1629 sec 2024-04-26T18:55:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1226.993351 secs 2024-04-26T18:55:45.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1226.993351 secs 2024-04-26T18:55:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1226.993351 secs 2024-04-26T18:55:46.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1630 sec 2024-04-26T18:55:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:46 smithi119 ceph-mon[32752]: pgmap v1456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1227.660993 secs 2024-04-26T18:55:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:46 smithi167 ceph-mon[32751]: pgmap v1456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:46.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1227.660993 secs 2024-04-26T18:55:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:46 smithi090 ceph-mon[29180]: pgmap v1456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:46.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1227.660993 secs 2024-04-26T18:55:47.023 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1631 sec 2024-04-26T18:55:48.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1632 sec 2024-04-26T18:55:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:48 smithi119 ceph-mon[32752]: pgmap v1457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:48.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:48 smithi167 ceph-mon[32751]: pgmap v1457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:48 smithi090 ceph-mon[29180]: pgmap v1457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:49.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1633 sec 2024-04-26T18:55:50.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1634 sec 2024-04-26T18:55:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:50 smithi119 ceph-mon[32752]: pgmap v1458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:50.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1231.993431 secs 2024-04-26T18:55:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:50 smithi167 ceph-mon[32751]: pgmap v1458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:50.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1231.993431 secs 2024-04-26T18:55:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:50 smithi090 ceph-mon[29180]: pgmap v1458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:50.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1231.993431 secs 2024-04-26T18:55:51.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1635 sec 2024-04-26T18:55:51.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1232.661111 secs 2024-04-26T18:55:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1232.661111 secs 2024-04-26T18:55:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1232.661111 secs 2024-04-26T18:55:52.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1636 sec 2024-04-26T18:55:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:52 smithi119 ceph-mon[32752]: pgmap v1459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:52.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:52 smithi167 ceph-mon[32751]: pgmap v1459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:52 smithi090 ceph-mon[29180]: pgmap v1459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:53.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1637 sec 2024-04-26T18:55:54.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1638 sec 2024-04-26T18:55:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:54 smithi119 ceph-mon[32752]: pgmap v1460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:54 smithi167 ceph-mon[32751]: pgmap v1460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:54 smithi090 ceph-mon[29180]: pgmap v1460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:55.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1639 sec 2024-04-26T18:55:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1236.993518 secs 2024-04-26T18:55:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1236.993518 secs 2024-04-26T18:55:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1236.993518 secs 2024-04-26T18:55:56.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1640 sec 2024-04-26T18:55:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:56 smithi119 ceph-mon[32752]: pgmap v1461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:56.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1237.661225 secs 2024-04-26T18:55:56.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:56 smithi167 ceph-mon[32751]: pgmap v1461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:56.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1237.661225 secs 2024-04-26T18:55:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:56 smithi090 ceph-mon[29180]: pgmap v1461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:55:56.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1237.661225 secs 2024-04-26T18:55:57.024 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1641 sec 2024-04-26T18:55:58.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1642 sec 2024-04-26T18:55:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:58 smithi119 ceph-mon[32752]: pgmap v1462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:58.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:58 smithi167 ceph-mon[32751]: pgmap v1462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:58 smithi090 ceph-mon[29180]: pgmap v1462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:55:59.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1643 sec 2024-04-26T18:55:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:55:59.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:55:59.825 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:55:59.832 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:55:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:55:59.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:55:59.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:55:59.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:55:59.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:55:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:55:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:55:59.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:55:59.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:55:59.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:55:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:56:00.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1644 sec 2024-04-26T18:56:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:00 smithi119 ceph-mon[32752]: pgmap v1463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:00.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:00 smithi119 ceph-mon[32752]: pgmap v1464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:56:00.823 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:00 smithi119 ceph-mon[32752]: pgmap v1465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:00.834 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1241.993600 secs 2024-04-26T18:56:00.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:00 smithi167 ceph-mon[32751]: pgmap v1463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:00.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:00 smithi167 ceph-mon[32751]: pgmap v1464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:56:00.932 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:00 smithi167 ceph-mon[32751]: pgmap v1465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:00.941 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1241.993600 secs 2024-04-26T18:56:01.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:00 smithi090 ceph-mon[29180]: pgmap v1463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:01.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:00 smithi090 ceph-mon[29180]: pgmap v1464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:56:01.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:00 smithi090 ceph-mon[29180]: pgmap v1465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:01.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1241.993600 secs 2024-04-26T18:56:01.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1645 sec 2024-04-26T18:56:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1242.661346 secs 2024-04-26T18:56:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1242.661346 secs 2024-04-26T18:56:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1242.661346 secs 2024-04-26T18:56:02.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1646 sec 2024-04-26T18:56:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:02 smithi119 ceph-mon[32752]: pgmap v1466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:02 smithi167 ceph-mon[32751]: pgmap v1466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:02 smithi090 ceph-mon[29180]: pgmap v1466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:03.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1647 sec 2024-04-26T18:56:04.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1648 sec 2024-04-26T18:56:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:04 smithi119 ceph-mon[32752]: pgmap v1467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:04 smithi167 ceph-mon[32751]: pgmap v1467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:04 smithi090 ceph-mon[29180]: pgmap v1467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:05.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1649 sec 2024-04-26T18:56:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1246.993685 secs 2024-04-26T18:56:05.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1246.993685 secs 2024-04-26T18:56:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1246.993685 secs 2024-04-26T18:56:06.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1650 sec 2024-04-26T18:56:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:06 smithi119 ceph-mon[32752]: pgmap v1468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:06.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1247.661407 secs 2024-04-26T18:56:06.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:06 smithi167 ceph-mon[32751]: pgmap v1468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:06.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1247.661407 secs 2024-04-26T18:56:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:06 smithi090 ceph-mon[29180]: pgmap v1468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:56:06.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1247.661407 secs 2024-04-26T18:56:07.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1651 sec 2024-04-26T18:56:08.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1652 sec 2024-04-26T18:56:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:08 smithi119 ceph-mon[32752]: pgmap v1469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:56:08.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:08 smithi167 ceph-mon[32751]: pgmap v1469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:56:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:08 smithi090 ceph-mon[29180]: pgmap v1469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:56:09.025 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1653 sec 2024-04-26T18:56:10.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1654 sec 2024-04-26T18:56:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:10 smithi119 ceph-mon[32752]: pgmap v1470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:56:10.825 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1251.993766 secs 2024-04-26T18:56:10.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:10 smithi167 ceph-mon[32751]: pgmap v1470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:56:10.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1251.993766 secs 2024-04-26T18:56:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:10 smithi090 ceph-mon[29180]: pgmap v1470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:56:10.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1251.993766 secs 2024-04-26T18:56:11.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1655 sec 2024-04-26T18:56:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1252.661504 secs 2024-04-26T18:56:11.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1252.661504 secs 2024-04-26T18:56:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1252.661504 secs 2024-04-26T18:56:12.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1656 sec 2024-04-26T18:56:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:12 smithi119 ceph-mon[32752]: pgmap v1471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:12 smithi167 ceph-mon[32751]: pgmap v1471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:12 smithi090 ceph-mon[29180]: pgmap v1471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:13.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1657 sec 2024-04-26T18:56:14.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1658 sec 2024-04-26T18:56:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:14 smithi119 ceph-mon[32752]: pgmap v1472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:14.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:14 smithi167 ceph-mon[32751]: pgmap v1472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:14 smithi090 ceph-mon[29180]: pgmap v1472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:15.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1659 sec 2024-04-26T18:56:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1256.993822 secs 2024-04-26T18:56:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1256.993822 secs 2024-04-26T18:56:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1256.993822 secs 2024-04-26T18:56:16.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1660 sec 2024-04-26T18:56:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:16 smithi119 ceph-mon[32752]: pgmap v1473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:16.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1257.661620 secs 2024-04-26T18:56:16.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:16 smithi167 ceph-mon[32751]: pgmap v1473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:16.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1257.661620 secs 2024-04-26T18:56:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:16 smithi090 ceph-mon[29180]: pgmap v1473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:16.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1257.661620 secs 2024-04-26T18:56:17.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1661 sec 2024-04-26T18:56:18.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1662 sec 2024-04-26T18:56:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:18 smithi119 ceph-mon[32752]: pgmap v1474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:18.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:18 smithi167 ceph-mon[32751]: pgmap v1474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:18 smithi090 ceph-mon[29180]: pgmap v1474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:19.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1663 sec 2024-04-26T18:56:20.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1664 sec 2024-04-26T18:56:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:20 smithi119 ceph-mon[32752]: pgmap v1475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:20.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1261.993907 secs 2024-04-26T18:56:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:20 smithi167 ceph-mon[32751]: pgmap v1475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:20.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1261.993907 secs 2024-04-26T18:56:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:20 smithi090 ceph-mon[29180]: pgmap v1475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:20.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1261.993907 secs 2024-04-26T18:56:21.026 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1665 sec 2024-04-26T18:56:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1262.661737 secs 2024-04-26T18:56:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1262.661737 secs 2024-04-26T18:56:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1262.661737 secs 2024-04-26T18:56:22.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1666 sec 2024-04-26T18:56:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:22 smithi119 ceph-mon[32752]: pgmap v1476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:22.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:22 smithi167 ceph-mon[32751]: pgmap v1476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:22 smithi090 ceph-mon[29180]: pgmap v1476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:23.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1667 sec 2024-04-26T18:56:24.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1668 sec 2024-04-26T18:56:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:24 smithi119 ceph-mon[32752]: pgmap v1477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:24 smithi167 ceph-mon[32751]: pgmap v1477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:24 smithi090 ceph-mon[29180]: pgmap v1477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:25.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1669 sec 2024-04-26T18:56:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1266.993968 secs 2024-04-26T18:56:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1266.993968 secs 2024-04-26T18:56:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1266.993968 secs 2024-04-26T18:56:26.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1670 sec 2024-04-26T18:56:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:26 smithi119 ceph-mon[32752]: pgmap v1478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:26.839 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1267.661860 secs 2024-04-26T18:56:26.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:26 smithi167 ceph-mon[32751]: pgmap v1478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:26.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1267.661860 secs 2024-04-26T18:56:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:26 smithi090 ceph-mon[29180]: pgmap v1478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:26.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1267.661860 secs 2024-04-26T18:56:27.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1671 sec 2024-04-26T18:56:28.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1672 sec 2024-04-26T18:56:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:56:28.629+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T18:56:28.641 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:56:28.630+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 14 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T18:56:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:56:28.642+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac0066b0 con 0x56302893f0e0 2024-04-26T18:56:28.666 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:56:28.642+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 14 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac0066b0 con 0x56302893f0e0 2024-04-26T18:56:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:28 smithi119 ceph-mon[32752]: pgmap v1479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:28 smithi167 ceph-mon[32751]: pgmap v1479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:28 smithi090 ceph-mon[29180]: pgmap v1479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:29.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1673 sec 2024-04-26T18:56:30.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1674 sec 2024-04-26T18:56:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:30 smithi119 ceph-mon[32752]: pgmap v1480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:30.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1271.994026 secs 2024-04-26T18:56:30.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:30 smithi167 ceph-mon[32751]: pgmap v1480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:30.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1271.994026 secs 2024-04-26T18:56:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:30 smithi090 ceph-mon[29180]: pgmap v1480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:30.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1271.994026 secs 2024-04-26T18:56:31.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1675 sec 2024-04-26T18:56:31.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:56:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:56:31.106+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:56:31.387 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:56:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:56:31.106+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:56:31.396 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:56:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:56:31.106+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:56:31.405 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:56:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:56:31.106+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:56:31.415 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 18:56:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T18:56:31.107+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:56:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1272.661978 secs 2024-04-26T18:56:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1272.661978 secs 2024-04-26T18:56:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1272.661978 secs 2024-04-26T18:56:32.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1676 sec 2024-04-26T18:56:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:32 smithi119 ceph-mon[32752]: pgmap v1481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:32.841 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:32 smithi167 ceph-mon[32751]: pgmap v1481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:32.894 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:32 smithi090 ceph-mon[29180]: pgmap v1481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:33.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1677 sec 2024-04-26T18:56:34.027 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1678 sec 2024-04-26T18:56:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:34 smithi119 ceph-mon[32752]: pgmap v1482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:34 smithi167 ceph-mon[32751]: pgmap v1482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:34 smithi090 ceph-mon[29180]: pgmap v1482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:35.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1679 sec 2024-04-26T18:56:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1276.994086 secs 2024-04-26T18:56:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1276.994086 secs 2024-04-26T18:56:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1276.994086 secs 2024-04-26T18:56:36.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1680 sec 2024-04-26T18:56:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:36 smithi119 ceph-mon[32752]: pgmap v1483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:36.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1277.662096 secs 2024-04-26T18:56:36.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:36 smithi167 ceph-mon[32751]: pgmap v1483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:36.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1277.662096 secs 2024-04-26T18:56:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:36 smithi090 ceph-mon[29180]: pgmap v1483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:36.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1277.662096 secs 2024-04-26T18:56:37.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1681 sec 2024-04-26T18:56:38.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1682 sec 2024-04-26T18:56:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:38 smithi119 ceph-mon[32752]: pgmap v1484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:38.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:38 smithi167 ceph-mon[32751]: pgmap v1484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:38 smithi090 ceph-mon[29180]: pgmap v1484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:39.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1683 sec 2024-04-26T18:56:40.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1684 sec 2024-04-26T18:56:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:40 smithi119 ceph-mon[32752]: pgmap v1485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:40.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1281.994168 secs 2024-04-26T18:56:40.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:40 smithi167 ceph-mon[32751]: pgmap v1485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:40.841 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1281.994168 secs 2024-04-26T18:56:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:40 smithi090 ceph-mon[29180]: pgmap v1485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:40.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1281.994168 secs 2024-04-26T18:56:41.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1685 sec 2024-04-26T18:56:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1282.662229 secs 2024-04-26T18:56:41.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1282.662229 secs 2024-04-26T18:56:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1282.662229 secs 2024-04-26T18:56:42.028 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1686 sec 2024-04-26T18:56:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:42 smithi119 ceph-mon[32752]: pgmap v1486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:42 smithi167 ceph-mon[32751]: pgmap v1486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:42 smithi090 ceph-mon[29180]: pgmap v1486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T18:56:43.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1687 sec 2024-04-26T18:56:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:43 smithi119 ceph-mon[32752]: pgmap v1487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:43 smithi167 ceph-mon[32751]: pgmap v1487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:43 smithi090 ceph-mon[29180]: pgmap v1487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T18:56:44.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1688 sec 2024-04-26T18:56:45.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1689 sec 2024-04-26T18:56:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1286.994228 secs 2024-04-26T18:56:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1286.994228 secs 2024-04-26T18:56:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1286.994228 secs 2024-04-26T18:56:46.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1690 sec 2024-04-26T18:56:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:46 smithi119 ceph-mon[32752]: pgmap v1488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:46.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1287.662350 secs 2024-04-26T18:56:46.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:46 smithi167 ceph-mon[32751]: pgmap v1488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:46.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1287.662350 secs 2024-04-26T18:56:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:46 smithi090 ceph-mon[29180]: pgmap v1488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:46.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1287.662350 secs 2024-04-26T18:56:47.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1691 sec 2024-04-26T18:56:48.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1692 sec 2024-04-26T18:56:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:48 smithi119 ceph-mon[32752]: pgmap v1489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:48.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:48 smithi167 ceph-mon[32751]: pgmap v1489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:48 smithi090 ceph-mon[29180]: pgmap v1489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:49.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1693 sec 2024-04-26T18:56:50.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1694 sec 2024-04-26T18:56:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:50 smithi119 ceph-mon[32752]: pgmap v1490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:50.837 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1291.994314 secs 2024-04-26T18:56:50.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:50 smithi167 ceph-mon[32751]: pgmap v1490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:50.861 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1291.994314 secs 2024-04-26T18:56:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:50 smithi090 ceph-mon[29180]: pgmap v1490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:50.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1291.994314 secs 2024-04-26T18:56:51.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1695 sec 2024-04-26T18:56:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1292.662484 secs 2024-04-26T18:56:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1292.662484 secs 2024-04-26T18:56:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1292.662484 secs 2024-04-26T18:56:52.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1696 sec 2024-04-26T18:56:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:52 smithi119 ceph-mon[32752]: pgmap v1491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:52 smithi167 ceph-mon[32751]: pgmap v1491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:52 smithi090 ceph-mon[29180]: pgmap v1491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:53.029 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1697 sec 2024-04-26T18:56:54.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1698 sec 2024-04-26T18:56:54.757 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:54 smithi090 ceph-mon[29180]: pgmap v1492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:54 smithi119 ceph-mon[32752]: pgmap v1492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:54 smithi167 ceph-mon[32751]: pgmap v1492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:56:55.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1699 sec 2024-04-26T18:56:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1296.994396 secs 2024-04-26T18:56:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1296.994396 secs 2024-04-26T18:56:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1296.994396 secs 2024-04-26T18:56:56.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1700 sec 2024-04-26T18:56:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:56 smithi119 ceph-mon[32752]: pgmap v1493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:56.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1297.662598 secs 2024-04-26T18:56:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:56 smithi090 ceph-mon[29180]: pgmap v1493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:56.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1297.662598 secs 2024-04-26T18:56:56.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:56 smithi167 ceph-mon[32751]: pgmap v1493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:56.904 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1297.662598 secs 2024-04-26T18:56:57.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1701 sec 2024-04-26T18:56:58.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1702 sec 2024-04-26T18:56:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:56:58 smithi119 ceph-mon[32752]: pgmap v1494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:56:58 smithi167 ceph-mon[32751]: pgmap v1494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:56:58 smithi090 ceph-mon[29180]: pgmap v1494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:56:59.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1703 sec 2024-04-26T18:57:00.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1704 sec 2024-04-26T18:57:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:00 smithi119 ceph-mon[32752]: pgmap v1495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:00.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:57:00.822 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1301.994460 secs 2024-04-26T18:57:00.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:57:00.837 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:57:00.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:57:00.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:00 smithi167 ceph-mon[32751]: pgmap v1495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:00.880 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:57:00.888 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1301.994460 secs 2024-04-26T18:57:00.894 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:57:00.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:57:00.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:57:00.917 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:00 smithi090 ceph-mon[29180]: pgmap v1495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:00.926 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:57:00.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1301.994460 secs 2024-04-26T18:57:00.942 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:57:00.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:57:01.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:57:01.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1705 sec 2024-04-26T18:57:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:01 smithi119 ceph-mon[32752]: pgmap v1496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:57:01.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:01 smithi119 ceph-mon[32752]: pgmap v1497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:01.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1302.662717 secs 2024-04-26T18:57:01.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:01 smithi167 ceph-mon[32751]: pgmap v1496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:57:01.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:01 smithi167 ceph-mon[32751]: pgmap v1497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:01.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1302.662717 secs 2024-04-26T18:57:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:01 smithi090 ceph-mon[29180]: pgmap v1496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:57:01.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:01 smithi090 ceph-mon[29180]: pgmap v1497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:01.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1302.662717 secs 2024-04-26T18:57:02.030 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1706 sec 2024-04-26T18:57:03.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1707 sec 2024-04-26T18:57:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:03 smithi119 ceph-mon[32752]: pgmap v1498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:57:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:03 smithi167 ceph-mon[32751]: pgmap v1498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:57:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:03 smithi090 ceph-mon[29180]: pgmap v1498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:57:04.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1708 sec 2024-04-26T18:57:05.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1709 sec 2024-04-26T18:57:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:05 smithi119 ceph-mon[32752]: pgmap v1499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:05.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1306.994542 secs 2024-04-26T18:57:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:05 smithi167 ceph-mon[32751]: pgmap v1499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:05.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1306.994542 secs 2024-04-26T18:57:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:05 smithi090 ceph-mon[29180]: pgmap v1499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1306.994542 secs 2024-04-26T18:57:06.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1710 sec 2024-04-26T18:57:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1307.662832 secs 2024-04-26T18:57:06.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1307.662832 secs 2024-04-26T18:57:06.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1307.662832 secs 2024-04-26T18:57:07.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1711 sec 2024-04-26T18:57:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:07 smithi119 ceph-mon[32752]: pgmap v1500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:07.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:07 smithi167 ceph-mon[32751]: pgmap v1500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:07 smithi090 ceph-mon[29180]: pgmap v1500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:57:08.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1712 sec 2024-04-26T18:57:09.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1713 sec 2024-04-26T18:57:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:09 smithi119 ceph-mon[32752]: pgmap v1501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:57:09.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:09 smithi167 ceph-mon[32751]: pgmap v1501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:57:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:09 smithi090 ceph-mon[29180]: pgmap v1501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:57:10.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1714 sec 2024-04-26T18:57:10.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1311.994598 secs 2024-04-26T18:57:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1311.994598 secs 2024-04-26T18:57:10.833 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1311.994598 secs 2024-04-26T18:57:11.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1715 sec 2024-04-26T18:57:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:11 smithi119 ceph-mon[32752]: pgmap v1502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:57:11.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1312.662897 secs 2024-04-26T18:57:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:11 smithi167 ceph-mon[32751]: pgmap v1502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:57:11.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1312.662897 secs 2024-04-26T18:57:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:11 smithi090 ceph-mon[29180]: pgmap v1502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:57:11.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1312.662897 secs 2024-04-26T18:57:12.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1716 sec 2024-04-26T18:57:13.031 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1717 sec 2024-04-26T18:57:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:13 smithi119 ceph-mon[32752]: pgmap v1503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:13 smithi167 ceph-mon[32751]: pgmap v1503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:13 smithi090 ceph-mon[29180]: pgmap v1503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:14.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1718 sec 2024-04-26T18:57:15.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1719 sec 2024-04-26T18:57:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:15 smithi119 ceph-mon[32752]: pgmap v1504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:15.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1316.994686 secs 2024-04-26T18:57:15.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:15 smithi090 ceph-mon[29180]: pgmap v1504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:15.898 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1316.994686 secs 2024-04-26T18:57:15.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:15 smithi167 ceph-mon[32751]: pgmap v1504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:15.916 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1316.994686 secs 2024-04-26T18:57:16.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1720 sec 2024-04-26T18:57:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1317.663012 secs 2024-04-26T18:57:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1317.663012 secs 2024-04-26T18:57:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1317.663012 secs 2024-04-26T18:57:17.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1721 sec 2024-04-26T18:57:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:17 smithi119 ceph-mon[32752]: pgmap v1505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:17 smithi167 ceph-mon[32751]: pgmap v1505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:17 smithi090 ceph-mon[29180]: pgmap v1505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:18.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1722 sec 2024-04-26T18:57:19.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1723 sec 2024-04-26T18:57:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:19 smithi119 ceph-mon[32752]: pgmap v1506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:19.855 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:19 smithi167 ceph-mon[32751]: pgmap v1506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:19 smithi090 ceph-mon[29180]: pgmap v1506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:20.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1724 sec 2024-04-26T18:57:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1321.994769 secs 2024-04-26T18:57:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1321.994769 secs 2024-04-26T18:57:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1321.994769 secs 2024-04-26T18:57:21.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1725 sec 2024-04-26T18:57:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:21 smithi119 ceph-mon[32752]: pgmap v1507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:21.843 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1322.663130 secs 2024-04-26T18:57:21.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:21 smithi090 ceph-mon[29180]: pgmap v1507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:21.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1322.663130 secs 2024-04-26T18:57:22.009 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:21 smithi167 ceph-mon[32751]: pgmap v1507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:22.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1322.663130 secs 2024-04-26T18:57:22.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1726 sec 2024-04-26T18:57:23.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1727 sec 2024-04-26T18:57:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:23 smithi119 ceph-mon[32752]: pgmap v1508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:23 smithi167 ceph-mon[32751]: pgmap v1508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:23 smithi090 ceph-mon[29180]: pgmap v1508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:24.032 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1728 sec 2024-04-26T18:57:25.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1729 sec 2024-04-26T18:57:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:25 smithi119 ceph-mon[32752]: pgmap v1509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:25.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1326.994827 secs 2024-04-26T18:57:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:25 smithi167 ceph-mon[32751]: pgmap v1509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:25.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1326.994827 secs 2024-04-26T18:57:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:25 smithi090 ceph-mon[29180]: pgmap v1509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:25.914 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1326.994827 secs 2024-04-26T18:57:26.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1730 sec 2024-04-26T18:57:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1327.663239 secs 2024-04-26T18:57:26.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:26 smithi119 ceph-mon[32752]: pgmap v1510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1327.663239 secs 2024-04-26T18:57:26.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:26 smithi167 ceph-mon[32751]: pgmap v1510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1327.663239 secs 2024-04-26T18:57:26.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:26 smithi090 ceph-mon[29180]: pgmap v1510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:27.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1731 sec 2024-04-26T18:57:28.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1732 sec 2024-04-26T18:57:29.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1733 sec 2024-04-26T18:57:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:29 smithi119 ceph-mon[32752]: pgmap v1511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:29 smithi167 ceph-mon[32751]: pgmap v1511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:29 smithi090 ceph-mon[29180]: pgmap v1511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:30.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1734 sec 2024-04-26T18:57:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1331.994888 secs 2024-04-26T18:57:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1331.994888 secs 2024-04-26T18:57:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1331.994888 secs 2024-04-26T18:57:31.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1735 sec 2024-04-26T18:57:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:31 smithi119 ceph-mon[32752]: pgmap v1512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:31.575 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1332.663319 secs 2024-04-26T18:57:31.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:31 smithi167 ceph-mon[32751]: pgmap v1512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:31.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1332.663319 secs 2024-04-26T18:57:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:31 smithi090 ceph-mon[29180]: pgmap v1512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:31.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1332.663319 secs 2024-04-26T18:57:32.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1736 sec 2024-04-26T18:57:33.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1737 sec 2024-04-26T18:57:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:33 smithi119 ceph-mon[32752]: pgmap v1513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:33.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:33 smithi167 ceph-mon[32751]: pgmap v1513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:33 smithi090 ceph-mon[29180]: pgmap v1513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:34.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1738 sec 2024-04-26T18:57:35.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1739 sec 2024-04-26T18:57:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:35 smithi119 ceph-mon[32752]: pgmap v1514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:35.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1336.994950 secs 2024-04-26T18:57:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:35 smithi167 ceph-mon[32751]: pgmap v1514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:35.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1336.994950 secs 2024-04-26T18:57:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:35 smithi090 ceph-mon[29180]: pgmap v1514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:35.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1336.994950 secs 2024-04-26T18:57:36.033 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1740 sec 2024-04-26T18:57:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:36 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 1337.663434 secs 2024-04-26T18:57:36.637 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:36 smithi119 ceph-mon[32752]: slow request 364.355198 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:57:36.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:36 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 1337.663434 secs 2024-04-26T18:57:36.654 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:36 smithi090 ceph-mon[29180]: slow request 364.355198 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:57:36.661 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:36 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 1337.663434 secs 2024-04-26T18:57:36.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:36 smithi167 ceph-mon[32751]: slow request 364.355198 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:57:37.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1741 sec 2024-04-26T18:57:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:37 smithi119 ceph-mon[32752]: pgmap v1515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:37.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:37 smithi167 ceph-mon[32751]: pgmap v1515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:37 smithi090 ceph-mon[29180]: pgmap v1515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:38.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1742 sec 2024-04-26T18:57:39.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1743 sec 2024-04-26T18:57:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:39 smithi119 ceph-mon[32752]: pgmap v1516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:39.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:39 smithi167 ceph-mon[32751]: pgmap v1516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:39 smithi090 ceph-mon[29180]: pgmap v1516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:40.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1744 sec 2024-04-26T18:57:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1341.995060 secs 2024-04-26T18:57:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1341.995060 secs 2024-04-26T18:57:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1341.995060 secs 2024-04-26T18:57:41.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1745 sec 2024-04-26T18:57:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:41 smithi119 ceph-mon[32752]: pgmap v1517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:41.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1342.663559 secs 2024-04-26T18:57:41.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:41 smithi167 ceph-mon[32751]: pgmap v1517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:41.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1342.663559 secs 2024-04-26T18:57:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:41 smithi090 ceph-mon[29180]: pgmap v1517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:41.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1342.663559 secs 2024-04-26T18:57:42.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1746 sec 2024-04-26T18:57:43.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1747 sec 2024-04-26T18:57:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:43 smithi119 ceph-mon[32752]: pgmap v1518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:43 smithi167 ceph-mon[32751]: pgmap v1518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:43 smithi090 ceph-mon[29180]: pgmap v1518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:44.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1748 sec 2024-04-26T18:57:45.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1749 sec 2024-04-26T18:57:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:45 smithi119 ceph-mon[32752]: pgmap v1519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:45.646 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1346.995145 secs 2024-04-26T18:57:45.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:45 smithi090 ceph-mon[29180]: pgmap v1519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:45.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1346.995145 secs 2024-04-26T18:57:45.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:45 smithi167 ceph-mon[32751]: pgmap v1519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:45.680 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1346.995145 secs 2024-04-26T18:57:46.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1750 sec 2024-04-26T18:57:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1347.663676 secs 2024-04-26T18:57:46.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1347.663676 secs 2024-04-26T18:57:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1347.663676 secs 2024-04-26T18:57:47.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1751 sec 2024-04-26T18:57:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:47 smithi119 ceph-mon[32752]: pgmap v1520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:47.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:47 smithi167 ceph-mon[32751]: pgmap v1520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:47 smithi090 ceph-mon[29180]: pgmap v1520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:48.034 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1752 sec 2024-04-26T18:57:49.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1753 sec 2024-04-26T18:57:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:49 smithi119 ceph-mon[32752]: pgmap v1521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:49 smithi167 ceph-mon[32751]: pgmap v1521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:49 smithi090 ceph-mon[29180]: pgmap v1521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:50.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1754 sec 2024-04-26T18:57:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1351.995229 secs 2024-04-26T18:57:50.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1351.995229 secs 2024-04-26T18:57:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1351.995229 secs 2024-04-26T18:57:51.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1755 sec 2024-04-26T18:57:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:51 smithi119 ceph-mon[32752]: pgmap v1522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:51.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1352.663774 secs 2024-04-26T18:57:51.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:51 smithi167 ceph-mon[32751]: pgmap v1522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:51.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1352.663774 secs 2024-04-26T18:57:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:51 smithi090 ceph-mon[29180]: pgmap v1522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:51.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1352.663774 secs 2024-04-26T18:57:52.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1756 sec 2024-04-26T18:57:53.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1757 sec 2024-04-26T18:57:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:53 smithi119 ceph-mon[32752]: pgmap v1523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:53.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:53 smithi090 ceph-mon[29180]: pgmap v1523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:53.650 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:53 smithi167 ceph-mon[32751]: pgmap v1523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:54.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1758 sec 2024-04-26T18:57:55.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1759 sec 2024-04-26T18:57:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:55 smithi119 ceph-mon[32752]: pgmap v1524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:55.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1356.995313 secs 2024-04-26T18:57:55.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:55 smithi167 ceph-mon[32751]: pgmap v1524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:55.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1356.995313 secs 2024-04-26T18:57:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:55 smithi090 ceph-mon[29180]: pgmap v1524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:57:55.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1356.995313 secs 2024-04-26T18:57:56.035 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1760 sec 2024-04-26T18:57:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1357.663891 secs 2024-04-26T18:57:56.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1357.663891 secs 2024-04-26T18:57:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1357.663891 secs 2024-04-26T18:57:57.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1761 sec 2024-04-26T18:57:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:57 smithi119 ceph-mon[32752]: pgmap v1525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:57 smithi167 ceph-mon[32751]: pgmap v1525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:57 smithi090 ceph-mon[29180]: pgmap v1525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:58.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1762 sec 2024-04-26T18:57:59.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1763 sec 2024-04-26T18:57:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:57:59 smithi119 ceph-mon[32752]: pgmap v1526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:59.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:57:59 smithi090 ceph-mon[29180]: pgmap v1526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:57:59.641 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:57:59 smithi167 ceph-mon[32751]: pgmap v1526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:00.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1764 sec 2024-04-26T18:58:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1361.995394 secs 2024-04-26T18:58:00.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:58:00.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1361.995394 secs 2024-04-26T18:58:00.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:58:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1361.995394 secs 2024-04-26T18:58:00.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:58:01.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1765 sec 2024-04-26T18:58:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:01 smithi167 ceph-mon[32751]: pgmap v1527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:01.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1362.663954 secs 2024-04-26T18:58:01.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:58:01.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:58:01.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:58:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:01 smithi090 ceph-mon[29180]: pgmap v1527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:01.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1362.663954 secs 2024-04-26T18:58:01.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:58:01.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:58:01.766 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:58:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:01 smithi119 ceph-mon[32752]: pgmap v1527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:01.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1362.663954 secs 2024-04-26T18:58:01.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:58:01.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:58:01.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:58:02.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1766 sec 2024-04-26T18:58:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:02 smithi167 ceph-mon[32751]: pgmap v1528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:58:02.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:02 smithi167 ceph-mon[32751]: pgmap v1529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:02.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:02 smithi090 ceph-mon[29180]: pgmap v1528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:58:02.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:02 smithi090 ceph-mon[29180]: pgmap v1529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:02 smithi119 ceph-mon[32752]: pgmap v1528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T18:58:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:02 smithi119 ceph-mon[32752]: pgmap v1529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:03.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1767 sec 2024-04-26T18:58:04.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1768 sec 2024-04-26T18:58:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:04 smithi167 ceph-mon[32751]: pgmap v1530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:58:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:04 smithi090 ceph-mon[29180]: pgmap v1530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:58:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:04 smithi119 ceph-mon[32752]: pgmap v1530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T18:58:05.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1769 sec 2024-04-26T18:58:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1366.995479 secs 2024-04-26T18:58:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1366.995479 secs 2024-04-26T18:58:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1366.995479 secs 2024-04-26T18:58:06.036 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1770 sec 2024-04-26T18:58:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:06 smithi090 ceph-mon[29180]: pgmap v1531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:06.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1367.664047 secs 2024-04-26T18:58:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:06 smithi119 ceph-mon[32752]: pgmap v1531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:06.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1367.664047 secs 2024-04-26T18:58:06.883 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:06 smithi167 ceph-mon[32751]: pgmap v1531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:06.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1367.664047 secs 2024-04-26T18:58:07.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1771 sec 2024-04-26T18:58:08.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1772 sec 2024-04-26T18:58:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:08 smithi090 ceph-mon[29180]: pgmap v1532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:08 smithi119 ceph-mon[32752]: pgmap v1532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:08 smithi167 ceph-mon[32751]: pgmap v1532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:09.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1773 sec 2024-04-26T18:58:10.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1774 sec 2024-04-26T18:58:10.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:10 smithi119 ceph-mon[32752]: pgmap v1533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:10.613 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1371.995562 secs 2024-04-26T18:58:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:10 smithi090 ceph-mon[29180]: pgmap v1533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:10.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1371.995562 secs 2024-04-26T18:58:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:10 smithi167 ceph-mon[32751]: pgmap v1533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:58:10.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1371.995562 secs 2024-04-26T18:58:11.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1775 sec 2024-04-26T18:58:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1372.664180 secs 2024-04-26T18:58:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1372.664180 secs 2024-04-26T18:58:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1372.664180 secs 2024-04-26T18:58:12.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1776 sec 2024-04-26T18:58:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:12 smithi090 ceph-mon[29180]: pgmap v1534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:58:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:12 smithi119 ceph-mon[32752]: pgmap v1534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:58:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:12 smithi167 ceph-mon[32751]: pgmap v1534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:58:13.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1777 sec 2024-04-26T18:58:14.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1778 sec 2024-04-26T18:58:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:14 smithi090 ceph-mon[29180]: pgmap v1535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:14 smithi119 ceph-mon[32752]: pgmap v1535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:14.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:14 smithi167 ceph-mon[32751]: pgmap v1535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:15.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1779 sec 2024-04-26T18:58:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1376.995647 secs 2024-04-26T18:58:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1376.995647 secs 2024-04-26T18:58:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1376.995647 secs 2024-04-26T18:58:16.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1780 sec 2024-04-26T18:58:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:16 smithi090 ceph-mon[29180]: pgmap v1536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:16.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1377.664291 secs 2024-04-26T18:58:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:16 smithi119 ceph-mon[32752]: pgmap v1536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:16.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1377.664291 secs 2024-04-26T18:58:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:16 smithi167 ceph-mon[32751]: pgmap v1536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:16.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1377.664291 secs 2024-04-26T18:58:17.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1781 sec 2024-04-26T18:58:18.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1782 sec 2024-04-26T18:58:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:18 smithi090 ceph-mon[29180]: pgmap v1537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:18 smithi119 ceph-mon[32752]: pgmap v1537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:18 smithi167 ceph-mon[32751]: pgmap v1537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:19.037 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1783 sec 2024-04-26T18:58:20.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1784 sec 2024-04-26T18:58:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:20 smithi090 ceph-mon[29180]: pgmap v1538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:20.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1381.995731 secs 2024-04-26T18:58:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:20 smithi119 ceph-mon[32752]: pgmap v1538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:20.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1381.995731 secs 2024-04-26T18:58:20.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:20 smithi167 ceph-mon[32751]: pgmap v1538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:20.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1381.995731 secs 2024-04-26T18:58:21.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1785 sec 2024-04-26T18:58:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1382.664397 secs 2024-04-26T18:58:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1382.664397 secs 2024-04-26T18:58:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1382.664397 secs 2024-04-26T18:58:22.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1786 sec 2024-04-26T18:58:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:22 smithi090 ceph-mon[29180]: pgmap v1539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:22 smithi119 ceph-mon[32752]: pgmap v1539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:22 smithi167 ceph-mon[32751]: pgmap v1539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:23.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1787 sec 2024-04-26T18:58:24.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1788 sec 2024-04-26T18:58:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:24 smithi090 ceph-mon[29180]: pgmap v1540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:24 smithi119 ceph-mon[32752]: pgmap v1540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:24 smithi167 ceph-mon[32751]: pgmap v1540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:25.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1789 sec 2024-04-26T18:58:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1386.995815 secs 2024-04-26T18:58:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1386.995815 secs 2024-04-26T18:58:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1386.995815 secs 2024-04-26T18:58:26.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1790 sec 2024-04-26T18:58:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:26 smithi090 ceph-mon[29180]: pgmap v1541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:26.701 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1387.664536 secs 2024-04-26T18:58:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:26 smithi119 ceph-mon[32752]: pgmap v1541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:26.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1387.664536 secs 2024-04-26T18:58:26.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:26 smithi167 ceph-mon[32751]: pgmap v1541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:26.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1387.664536 secs 2024-04-26T18:58:27.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1791 sec 2024-04-26T18:58:28.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1792 sec 2024-04-26T18:58:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:28 smithi090 ceph-mon[29180]: pgmap v1542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:28 smithi119 ceph-mon[32752]: pgmap v1542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:28 smithi167 ceph-mon[32751]: pgmap v1542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:29.038 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1793 sec 2024-04-26T18:58:30.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1794 sec 2024-04-26T18:58:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:30 smithi119 ceph-mon[32752]: pgmap v1543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:30.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1391.995900 secs 2024-04-26T18:58:30.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:30 smithi167 ceph-mon[32751]: pgmap v1543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:30.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1391.995900 secs 2024-04-26T18:58:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:30 smithi090 ceph-mon[29180]: pgmap v1543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:30.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1391.995900 secs 2024-04-26T18:58:31.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1795 sec 2024-04-26T18:58:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1392.664659 secs 2024-04-26T18:58:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1392.664659 secs 2024-04-26T18:58:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1392.664659 secs 2024-04-26T18:58:32.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1796 sec 2024-04-26T18:58:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:32 smithi119 ceph-mon[32752]: pgmap v1544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:32 smithi167 ceph-mon[32751]: pgmap v1544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:32 smithi090 ceph-mon[29180]: pgmap v1544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:33.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1797 sec 2024-04-26T18:58:34.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1798 sec 2024-04-26T18:58:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:34 smithi119 ceph-mon[32752]: pgmap v1545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:34 smithi167 ceph-mon[32751]: pgmap v1545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:34 smithi090 ceph-mon[29180]: pgmap v1545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:35.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1799 sec 2024-04-26T18:58:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:35 smithi119 ceph-mon[32752]: 7 slow requests, 1 included below; oldest blocked for > 1396.995990 secs 2024-04-26T18:58:35.826 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:35 smithi119 ceph-mon[32752]: slow request 723.813375 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:58:35.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:35 smithi167 ceph-mon[32751]: 7 slow requests, 1 included below; oldest blocked for > 1396.995990 secs 2024-04-26T18:58:35.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:35 smithi167 ceph-mon[32751]: slow request 723.813375 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:58:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:35 smithi090 ceph-mon[29180]: 7 slow requests, 1 included below; oldest blocked for > 1396.995990 secs 2024-04-26T18:58:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:35 smithi090 ceph-mon[29180]: slow request 723.813375 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T18:58:36.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1800 sec 2024-04-26T18:58:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:36 smithi119 ceph-mon[32752]: pgmap v1546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1397.664775 secs 2024-04-26T18:58:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:36 smithi167 ceph-mon[32751]: pgmap v1546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:36.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1397.664775 secs 2024-04-26T18:58:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:36 smithi090 ceph-mon[29180]: pgmap v1546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:36.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1397.664775 secs 2024-04-26T18:58:37.039 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1801 sec 2024-04-26T18:58:38.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1802 sec 2024-04-26T18:58:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:38 smithi119 ceph-mon[32752]: pgmap v1547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:38 smithi167 ceph-mon[32751]: pgmap v1547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:38 smithi090 ceph-mon[29180]: pgmap v1547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:39.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1803 sec 2024-04-26T18:58:40.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1804 sec 2024-04-26T18:58:40.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:40 smithi090 ceph-mon[29180]: pgmap v1548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:40.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1401.996073 secs 2024-04-26T18:58:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:40 smithi119 ceph-mon[32752]: pgmap v1548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:40.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1401.996073 secs 2024-04-26T18:58:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:40 smithi167 ceph-mon[32751]: pgmap v1548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:40.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1401.996073 secs 2024-04-26T18:58:41.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1805 sec 2024-04-26T18:58:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1402.664915 secs 2024-04-26T18:58:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1402.664915 secs 2024-04-26T18:58:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1402.664915 secs 2024-04-26T18:58:42.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1806 sec 2024-04-26T18:58:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:42 smithi119 ceph-mon[32752]: pgmap v1549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:42 smithi167 ceph-mon[32751]: pgmap v1549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:42 smithi090 ceph-mon[29180]: pgmap v1549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:43.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1807 sec 2024-04-26T18:58:44.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1808 sec 2024-04-26T18:58:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:44 smithi119 ceph-mon[32752]: pgmap v1550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:44 smithi167 ceph-mon[32751]: pgmap v1550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:44 smithi090 ceph-mon[29180]: pgmap v1550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:45.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1809 sec 2024-04-26T18:58:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1406.996135 secs 2024-04-26T18:58:45.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1406.996135 secs 2024-04-26T18:58:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1406.996135 secs 2024-04-26T18:58:46.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1810 sec 2024-04-26T18:58:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:46 smithi119 ceph-mon[32752]: pgmap v1551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:46.834 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1407.665029 secs 2024-04-26T18:58:46.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:46 smithi167 ceph-mon[32751]: pgmap v1551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:46.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1407.665029 secs 2024-04-26T18:58:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:46 smithi090 ceph-mon[29180]: pgmap v1551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:46.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1407.665029 secs 2024-04-26T18:58:47.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1811 sec 2024-04-26T18:58:48.040 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1812 sec 2024-04-26T18:58:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:48 smithi119 ceph-mon[32752]: pgmap v1552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:48 smithi167 ceph-mon[32751]: pgmap v1552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:48 smithi090 ceph-mon[29180]: pgmap v1552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:49.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1813 sec 2024-04-26T18:58:50.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1814 sec 2024-04-26T18:58:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:50 smithi119 ceph-mon[32752]: pgmap v1553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:50.817 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1411.996217 secs 2024-04-26T18:58:50.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:50 smithi167 ceph-mon[32751]: pgmap v1553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:50.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1411.996217 secs 2024-04-26T18:58:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:50 smithi090 ceph-mon[29180]: pgmap v1553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:50.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1411.996217 secs 2024-04-26T18:58:51.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1815 sec 2024-04-26T18:58:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1412.665096 secs 2024-04-26T18:58:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1412.665096 secs 2024-04-26T18:58:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1412.665096 secs 2024-04-26T18:58:52.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1816 sec 2024-04-26T18:58:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:52 smithi119 ceph-mon[32752]: pgmap v1554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:52.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:52 smithi167 ceph-mon[32751]: pgmap v1554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:52 smithi090 ceph-mon[29180]: pgmap v1554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:53.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1817 sec 2024-04-26T18:58:54.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1818 sec 2024-04-26T18:58:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:54 smithi119 ceph-mon[32752]: pgmap v1555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:54 smithi167 ceph-mon[32751]: pgmap v1555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:54 smithi090 ceph-mon[29180]: pgmap v1555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:55.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1819 sec 2024-04-26T18:58:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1416.996281 secs 2024-04-26T18:58:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1416.996281 secs 2024-04-26T18:58:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1416.996281 secs 2024-04-26T18:58:56.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1820 sec 2024-04-26T18:58:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:56 smithi119 ceph-mon[32752]: pgmap v1556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:56.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1417.665166 secs 2024-04-26T18:58:56.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:56 smithi167 ceph-mon[32751]: pgmap v1556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:56.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1417.665166 secs 2024-04-26T18:58:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:56 smithi090 ceph-mon[29180]: pgmap v1556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:58:56.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1417.665166 secs 2024-04-26T18:58:57.041 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1821 sec 2024-04-26T18:58:58.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1822 sec 2024-04-26T18:58:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:58:58 smithi119 ceph-mon[32752]: pgmap v1557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:58:58 smithi167 ceph-mon[32751]: pgmap v1557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:58:58 smithi090 ceph-mon[29180]: pgmap v1557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:58:59.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1823 sec 2024-04-26T18:59:00.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1824 sec 2024-04-26T18:59:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:00 smithi119 ceph-mon[32752]: pgmap v1558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:00.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1421.996362 secs 2024-04-26T18:59:00.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:00 smithi167 ceph-mon[32751]: pgmap v1558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:00.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1421.996362 secs 2024-04-26T18:59:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:00 smithi090 ceph-mon[29180]: pgmap v1558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:00.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1421.996362 secs 2024-04-26T18:59:01.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1825 sec 2024-04-26T18:59:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1422.665287 secs 2024-04-26T18:59:01.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:59:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1422.665287 secs 2024-04-26T18:59:01.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:59:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1422.665287 secs 2024-04-26T18:59:01.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:59:02.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1826 sec 2024-04-26T18:59:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:02 smithi119 ceph-mon[32752]: pgmap v1559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:59:02.819 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:59:02.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:59:02.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:02 smithi090 ceph-mon[29180]: pgmap v1559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:02.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:59:02.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:59:02.974 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:59:02.982 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:02 smithi167 ceph-mon[32751]: pgmap v1559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:02.989 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:59:03.029 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:59:03.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T18:59:03.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1827 sec 2024-04-26T18:59:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:03 smithi119 ceph-mon[32752]: pgmap v1560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:59:03.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:03 smithi119 ceph-mon[32752]: pgmap v1561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:03 smithi167 ceph-mon[32751]: pgmap v1560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:59:03.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:03 smithi167 ceph-mon[32751]: pgmap v1561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:03 smithi090 ceph-mon[29180]: pgmap v1560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T18:59:03.923 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:03 smithi090 ceph-mon[29180]: pgmap v1561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:04.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1828 sec 2024-04-26T18:59:05.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1829 sec 2024-04-26T18:59:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:05 smithi119 ceph-mon[32752]: pgmap v1562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:05.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1426.996447 secs 2024-04-26T18:59:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:05 smithi167 ceph-mon[32751]: pgmap v1562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:05.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1426.996447 secs 2024-04-26T18:59:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:05 smithi090 ceph-mon[29180]: pgmap v1562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1426.996447 secs 2024-04-26T18:59:06.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1830 sec 2024-04-26T18:59:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1427.665413 secs 2024-04-26T18:59:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1427.665413 secs 2024-04-26T18:59:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1427.665413 secs 2024-04-26T18:59:07.042 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1831 sec 2024-04-26T18:59:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:07 smithi119 ceph-mon[32752]: pgmap v1563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:07.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:07 smithi167 ceph-mon[32751]: pgmap v1563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:07 smithi090 ceph-mon[29180]: pgmap v1563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:08.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1832 sec 2024-04-26T18:59:09.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1833 sec 2024-04-26T18:59:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:09 smithi119 ceph-mon[32752]: pgmap v1564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:59:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:09 smithi167 ceph-mon[32751]: pgmap v1564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:59:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:09 smithi090 ceph-mon[29180]: pgmap v1564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T18:59:10.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1834 sec 2024-04-26T18:59:10.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1431.996528 secs 2024-04-26T18:59:10.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1431.996528 secs 2024-04-26T18:59:10.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1431.996528 secs 2024-04-26T18:59:11.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1835 sec 2024-04-26T18:59:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:11 smithi119 ceph-mon[32752]: pgmap v1565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:11.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1432.665538 secs 2024-04-26T18:59:11.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:11 smithi167 ceph-mon[32751]: pgmap v1565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:11.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1432.665538 secs 2024-04-26T18:59:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:11 smithi090 ceph-mon[29180]: pgmap v1565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T18:59:11.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1432.665538 secs 2024-04-26T18:59:12.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1836 sec 2024-04-26T18:59:13.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1837 sec 2024-04-26T18:59:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:13 smithi119 ceph-mon[32752]: pgmap v1566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:59:13.890 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:13 smithi090 ceph-mon[29180]: pgmap v1566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:59:13.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:13 smithi167 ceph-mon[32751]: pgmap v1566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T18:59:14.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1838 sec 2024-04-26T18:59:15.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1839 sec 2024-04-26T18:59:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:15 smithi119 ceph-mon[32752]: pgmap v1567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:15.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1436.996617 secs 2024-04-26T18:59:15.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:15 smithi167 ceph-mon[32751]: pgmap v1567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:15.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1436.996617 secs 2024-04-26T18:59:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:15 smithi090 ceph-mon[29180]: pgmap v1567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1436.996617 secs 2024-04-26T18:59:16.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1840 sec 2024-04-26T18:59:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1437.665644 secs 2024-04-26T18:59:16.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1437.665644 secs 2024-04-26T18:59:16.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1437.665644 secs 2024-04-26T18:59:17.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1841 sec 2024-04-26T18:59:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:17 smithi119 ceph-mon[32752]: pgmap v1568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:17 smithi167 ceph-mon[32751]: pgmap v1568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:17 smithi090 ceph-mon[29180]: pgmap v1568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:18.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1842 sec 2024-04-26T18:59:19.043 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1843 sec 2024-04-26T18:59:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:19 smithi119 ceph-mon[32752]: pgmap v1569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:19.856 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:19 smithi167 ceph-mon[32751]: pgmap v1569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:19 smithi090 ceph-mon[29180]: pgmap v1569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:20.044 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1844 sec 2024-04-26T18:59:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:20 smithi119 ceph-mon[32752]: 7 slow requests, 4 included below; oldest blocked for > 1441.996699 secs 2024-04-26T18:59:20.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:20 smithi119 ceph-mon[32752]: slow request 1441.996699 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:59:20.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:20 smithi119 ceph-mon[32752]: slow request 1441.993804 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:20.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:20 smithi119 ceph-mon[32752]: slow request 1441.988194 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:20.835 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:20 smithi119 ceph-mon[32752]: slow request 1440.021468 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:59:20.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:20 smithi167 ceph-mon[32751]: 7 slow requests, 4 included below; oldest blocked for > 1441.996699 secs 2024-04-26T18:59:20.850 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:20 smithi167 ceph-mon[32751]: slow request 1441.996699 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:59:20.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:20 smithi167 ceph-mon[32751]: slow request 1441.993804 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:20.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:20 smithi167 ceph-mon[32751]: slow request 1441.988194 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:20.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:20 smithi167 ceph-mon[32751]: slow request 1440.021468 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:59:20.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:20 smithi090 ceph-mon[29180]: 7 slow requests, 4 included below; oldest blocked for > 1441.996699 secs 2024-04-26T18:59:20.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:20 smithi090 ceph-mon[29180]: slow request 1441.996699 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T18:59:20.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:20 smithi090 ceph-mon[29180]: slow request 1441.993804 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:20 smithi090 ceph-mon[29180]: slow request 1441.988194 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:20 smithi090 ceph-mon[29180]: slow request 1440.021468 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T18:59:21.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1845 sec 2024-04-26T18:59:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:21 smithi119 ceph-mon[32752]: pgmap v1570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:21.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:21 smithi119 ceph-mon[32752]: 11 slow requests, 5 included below; oldest blocked for > 1442.665742 secs 2024-04-26T18:59:21.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:21 smithi119 ceph-mon[32752]: slow request 1442.665741 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.830 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:21 smithi119 ceph-mon[32752]: slow request 1442.664180 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.839 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:21 smithi119 ceph-mon[32752]: slow request 1442.662711 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:21 smithi119 ceph-mon[32752]: slow request 1442.662241 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:21 smithi119 ceph-mon[32752]: slow request 1442.649976 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:21 smithi167 ceph-mon[32751]: pgmap v1570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:21.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:21 smithi167 ceph-mon[32751]: 11 slow requests, 5 included below; oldest blocked for > 1442.665742 secs 2024-04-26T18:59:21.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:21 smithi167 ceph-mon[32751]: slow request 1442.665741 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.895 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:21 smithi167 ceph-mon[32751]: slow request 1442.664180 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:21 smithi167 ceph-mon[32751]: slow request 1442.662711 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.913 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:21 smithi167 ceph-mon[32751]: slow request 1442.662241 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:21.923 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:21 smithi167 ceph-mon[32751]: slow request 1442.649976 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:22.001 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:21 smithi090 ceph-mon[29180]: pgmap v1570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:22.071 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:21 smithi090 ceph-mon[29180]: 11 slow requests, 5 included below; oldest blocked for > 1442.665742 secs 2024-04-26T18:59:22.081 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:21 smithi090 ceph-mon[29180]: slow request 1442.665741 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:22.090 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:21 smithi090 ceph-mon[29180]: slow request 1442.664180 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:21 smithi090 ceph-mon[29180]: slow request 1442.662711 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:22.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:21 smithi090 ceph-mon[29180]: slow request 1442.662241 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:22.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:21 smithi090 ceph-mon[29180]: slow request 1442.649976 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T18:59:22.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1846 sec 2024-04-26T18:59:23.044 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1847 sec 2024-04-26T18:59:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:23 smithi119 ceph-mon[32752]: pgmap v1571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:23 smithi167 ceph-mon[32751]: pgmap v1571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:23 smithi090 ceph-mon[29180]: pgmap v1571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:24.044 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1848 sec 2024-04-26T18:59:25.044 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1849 sec 2024-04-26T18:59:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:25 smithi119 ceph-mon[32752]: pgmap v1572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:25.832 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1446.996792 secs 2024-04-26T18:59:25.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:25 smithi090 ceph-mon[29180]: pgmap v1572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:25.917 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1446.996792 secs 2024-04-26T18:59:25.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:25 smithi167 ceph-mon[32751]: pgmap v1572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:25.935 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1446.996792 secs 2024-04-26T18:59:26.044 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1850 sec 2024-04-26T18:59:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:26 smithi119 ceph-mon[32752]: 11 slow requests, 4 included below; oldest blocked for > 1447.665850 secs 2024-04-26T18:59:26.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:26 smithi119 ceph-mon[32752]: slow request 1444.081213 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:59:26.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:26 smithi119 ceph-mon[32752]: slow request 1444.080877 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:59:26.824 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:26 smithi119 ceph-mon[32752]: slow request 1443.972042 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:59:26.831 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:26 smithi119 ceph-mon[32752]: slow request 1443.967754 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:59:26.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:26 smithi167 ceph-mon[32751]: 11 slow requests, 4 included below; oldest blocked for > 1447.665850 secs 2024-04-26T18:59:26.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:26 smithi167 ceph-mon[32751]: slow request 1444.081213 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:59:26.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:26 smithi167 ceph-mon[32751]: slow request 1444.080877 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:59:26.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:26 smithi167 ceph-mon[32751]: slow request 1443.972042 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:59:26.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:26 smithi167 ceph-mon[32751]: slow request 1443.967754 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:59:26.936 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:26 smithi090 ceph-mon[29180]: 11 slow requests, 4 included below; oldest blocked for > 1447.665850 secs 2024-04-26T18:59:27.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:26 smithi090 ceph-mon[29180]: slow request 1444.081213 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T18:59:27.085 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:26 smithi090 ceph-mon[29180]: slow request 1444.080877 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T18:59:27.095 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:26 smithi090 ceph-mon[29180]: slow request 1443.972042 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T18:59:27.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:26 smithi090 ceph-mon[29180]: slow request 1443.967754 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T18:59:27.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1851 sec 2024-04-26T18:59:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:27 smithi119 ceph-mon[32752]: pgmap v1573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:27 smithi167 ceph-mon[32751]: pgmap v1573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:27.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:27 smithi090 ceph-mon[29180]: pgmap v1573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:28.044 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1852 sec 2024-04-26T18:59:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:59:28.653+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T18:59:28.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:59:28.653+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 15 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T18:59:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:59:28.664+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac0066b0 con 0x56302893f0e0 2024-04-26T18:59:28.680 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T18:59:28.664+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 15 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac0066b0 con 0x56302893f0e0 2024-04-26T18:59:29.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1853 sec 2024-04-26T18:59:29.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:29 smithi119 ceph-mon[32752]: pgmap v1574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:29 smithi167 ceph-mon[32751]: pgmap v1574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:29.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:29 smithi090 ceph-mon[29180]: pgmap v1574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:30.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1854 sec 2024-04-26T18:59:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1451.996873 secs 2024-04-26T18:59:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1451.996873 secs 2024-04-26T18:59:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1451.996873 secs 2024-04-26T18:59:31.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1855 sec 2024-04-26T18:59:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:31 smithi119 ceph-mon[32752]: pgmap v1575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:31.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 1452.665987 secs 2024-04-26T18:59:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:31 smithi167 ceph-mon[32751]: pgmap v1575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:31.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 1452.665987 secs 2024-04-26T18:59:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:31 smithi090 ceph-mon[29180]: pgmap v1575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:31.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 1452.665987 secs 2024-04-26T18:59:32.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1856 sec 2024-04-26T18:59:33.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1857 sec 2024-04-26T18:59:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:33 smithi119 ceph-mon[32752]: pgmap v1576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:33 smithi167 ceph-mon[32751]: pgmap v1576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:33 smithi090 ceph-mon[29180]: pgmap v1576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:34.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1858 sec 2024-04-26T18:59:35.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1859 sec 2024-04-26T18:59:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:35 smithi119 ceph-mon[32752]: pgmap v1577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:35.860 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1456.996934 secs 2024-04-26T18:59:35.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:35 smithi167 ceph-mon[32751]: pgmap v1577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:35.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1456.996934 secs 2024-04-26T18:59:35.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:35 smithi090 ceph-mon[29180]: pgmap v1577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:35.901 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1456.996934 secs 2024-04-26T18:59:36.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1860 sec 2024-04-26T18:59:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:36 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 1457.666130 secs 2024-04-26T18:59:36.891 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:36 smithi119 ceph-mon[32752]: slow request 184.302994 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:59:36.922 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:36 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 1457.666130 secs 2024-04-26T18:59:36.932 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:36 smithi090 ceph-mon[29180]: slow request 184.302994 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:59:36.942 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:36 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 1457.666130 secs 2024-04-26T18:59:36.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:36 smithi167 ceph-mon[32751]: slow request 184.302994 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T18:59:37.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1861 sec 2024-04-26T18:59:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:37 smithi167 ceph-mon[32751]: pgmap v1578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:37 smithi090 ceph-mon[29180]: pgmap v1578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:38.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1862 sec 2024-04-26T18:59:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:37 smithi119 ceph-mon[32752]: pgmap v1578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:59:38.869 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:59:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:59:38.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:59:39.045 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1863 sec 2024-04-26T18:59:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T18:59:39.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T18:59:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:39 smithi167 ceph-mon[32751]: pgmap v1579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:39 smithi090 ceph-mon[29180]: pgmap v1579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:40.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1864 sec 2024-04-26T18:59:40.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:39 smithi119 ceph-mon[32752]: pgmap v1579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1461.996994 secs 2024-04-26T18:59:40.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1461.996994 secs 2024-04-26T18:59:41.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1865 sec 2024-04-26T18:59:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1461.996994 secs 2024-04-26T18:59:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:41 smithi090 ceph-mon[29180]: pgmap v1580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:41.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1462.666265 secs 2024-04-26T18:59:42.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1866 sec 2024-04-26T18:59:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:41 smithi119 ceph-mon[32752]: pgmap v1580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:42.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1462.666265 secs 2024-04-26T18:59:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:41 smithi167 ceph-mon[32751]: pgmap v1580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:42.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1462.666265 secs 2024-04-26T18:59:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:42 smithi090 ceph-mon[29180]: pgmap v1581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:43.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1867 sec 2024-04-26T18:59:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:42 smithi119 ceph-mon[32752]: pgmap v1581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:42 smithi167 ceph-mon[32751]: pgmap v1581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:44.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1868 sec 2024-04-26T18:59:45.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1869 sec 2024-04-26T18:59:45.067 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:44 smithi119 ceph-mon[32752]: pgmap v1582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:45.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:44 smithi167 ceph-mon[32751]: pgmap v1582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:44 smithi090 ceph-mon[29180]: pgmap v1582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:46.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1466.997054 secs 2024-04-26T18:59:46.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:45 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1467.666329 secs 2024-04-26T18:59:46.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1870 sec 2024-04-26T18:59:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1466.997054 secs 2024-04-26T18:59:46.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:45 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1467.666329 secs 2024-04-26T18:59:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1466.997054 secs 2024-04-26T18:59:46.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:45 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1467.666329 secs 2024-04-26T18:59:47.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1871 sec 2024-04-26T18:59:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:46 smithi167 ceph-mon[32751]: pgmap v1583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:46 smithi090 ceph-mon[29180]: pgmap v1583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:46 smithi119 ceph-mon[32752]: pgmap v1583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:48.046 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1872 sec 2024-04-26T18:59:49.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1873 sec 2024-04-26T18:59:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:48 smithi167 ceph-mon[32751]: pgmap v1584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:48 smithi090 ceph-mon[29180]: pgmap v1584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:48 smithi119 ceph-mon[32752]: pgmap v1584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:50.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1874 sec 2024-04-26T18:59:51.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1875 sec 2024-04-26T18:59:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:50 smithi167 ceph-mon[32751]: pgmap v1585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:51.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1471.997140 secs 2024-04-26T18:59:51.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:50 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1472.666450 secs 2024-04-26T18:59:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:50 smithi090 ceph-mon[29180]: pgmap v1585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:51.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1471.997140 secs 2024-04-26T18:59:51.141 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:50 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1472.666450 secs 2024-04-26T18:59:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:50 smithi119 ceph-mon[32752]: pgmap v1585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:51.364 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1471.997140 secs 2024-04-26T18:59:51.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:50 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1472.666450 secs 2024-04-26T18:59:52.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1876 sec 2024-04-26T18:59:53.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1877 sec 2024-04-26T18:59:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:52 smithi090 ceph-mon[29180]: pgmap v1586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:52 smithi119 ceph-mon[32752]: pgmap v1586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:52 smithi167 ceph-mon[32751]: pgmap v1586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:54.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1878 sec 2024-04-26T18:59:55.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1879 sec 2024-04-26T18:59:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:54 smithi090 ceph-mon[29180]: pgmap v1587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:54 smithi119 ceph-mon[32752]: pgmap v1587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:54 smithi167 ceph-mon[32751]: pgmap v1587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T18:59:56.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1880 sec 2024-04-26T18:59:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1476.997201 secs 2024-04-26T18:59:56.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1477.666566 secs 2024-04-26T18:59:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1476.997201 secs 2024-04-26T18:59:56.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1477.666566 secs 2024-04-26T18:59:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1476.997201 secs 2024-04-26T18:59:56.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1477.666566 secs 2024-04-26T18:59:57.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1881 sec 2024-04-26T18:59:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:56 smithi090 ceph-mon[29180]: pgmap v1588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:56 smithi119 ceph-mon[32752]: pgmap v1588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:56 smithi167 ceph-mon[32751]: pgmap v1588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:58.047 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1882 sec 2024-04-26T18:59:59.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1883 sec 2024-04-26T18:59:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 18:59:58 smithi090 ceph-mon[29180]: pgmap v1589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 18:59:58 smithi119 ceph-mon[32752]: pgmap v1589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T18:59:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 18:59:58 smithi167 ceph-mon[32751]: pgmap v1589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:00.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1884 sec 2024-04-26T19:00:01.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1885 sec 2024-04-26T19:00:01.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:00 smithi167 ceph-mon[32751]: pgmap v1590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:01.133 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1481.997287 secs 2024-04-26T19:00:01.143 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:00:01.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:00 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1482.666705 secs 2024-04-26T19:00:01.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:00 smithi090 ceph-mon[29180]: pgmap v1590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:01.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1481.997287 secs 2024-04-26T19:00:01.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:00:01.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:00 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1482.666705 secs 2024-04-26T19:00:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:00 smithi119 ceph-mon[32752]: pgmap v1590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:01.352 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1481.997287 secs 2024-04-26T19:00:01.359 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:00:01.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:00 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1482.666705 secs 2024-04-26T19:00:02.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1886 sec 2024-04-26T19:00:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:00:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:00:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:00:03.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1887 sec 2024-04-26T19:00:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:02 smithi119 ceph-mon[32752]: pgmap v1591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:00:03.319 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:00:03.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:02 smithi119 ceph-mon[32752]: pgmap v1592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:00:03.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:02 smithi119 ceph-mon[32752]: pgmap v1593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:00:03.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:00:03.411 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:02 smithi090 ceph-mon[29180]: pgmap v1591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:03.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:00:03.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:00:03.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:02 smithi090 ceph-mon[29180]: pgmap v1592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:00:03.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:02 smithi090 ceph-mon[29180]: pgmap v1593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:00:03.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:00:03.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:02 smithi167 ceph-mon[32751]: pgmap v1591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:03.516 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:00:03.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:00:03.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:02 smithi167 ceph-mon[32751]: pgmap v1592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:00:03.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:02 smithi167 ceph-mon[32751]: pgmap v1593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:00:03.675 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:00:04.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1888 sec 2024-04-26T19:00:05.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1889 sec 2024-04-26T19:00:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:05 smithi167 ceph-mon[32751]: pgmap v1594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:05 smithi167 ceph-mon[32751]: 7 slow requests, 1 included below; oldest blocked for > 1486.997377 secs 2024-04-26T19:00:05.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:05 smithi167 ceph-mon[32751]: slow request 1443.992769 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:00:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:05 smithi090 ceph-mon[29180]: pgmap v1594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:05 smithi090 ceph-mon[29180]: 7 slow requests, 1 included below; oldest blocked for > 1486.997377 secs 2024-04-26T19:00:05.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:05 smithi090 ceph-mon[29180]: slow request 1443.992769 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:00:06.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1890 sec 2024-04-26T19:00:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:05 smithi119 ceph-mon[32752]: pgmap v1594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:06.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:05 smithi119 ceph-mon[32752]: 7 slow requests, 1 included below; oldest blocked for > 1486.997377 secs 2024-04-26T19:00:06.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:05 smithi119 ceph-mon[32752]: slow request 1443.992769 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:00:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1487.666826 secs 2024-04-26T19:00:07.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1891 sec 2024-04-26T19:00:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1487.666826 secs 2024-04-26T19:00:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1487.666826 secs 2024-04-26T19:00:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:07 smithi090 ceph-mon[29180]: pgmap v1595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:08.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1892 sec 2024-04-26T19:00:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:07 smithi119 ceph-mon[32752]: pgmap v1595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:07 smithi167 ceph-mon[32751]: pgmap v1595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:08 smithi090 ceph-mon[29180]: pgmap v1596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:09.048 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1893 sec 2024-04-26T19:00:09.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:08 smithi119 ceph-mon[32752]: pgmap v1596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:08 smithi167 ceph-mon[32751]: pgmap v1596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:10.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1894 sec 2024-04-26T19:00:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1491.997458 secs 2024-04-26T19:00:10.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1491.997458 secs 2024-04-26T19:00:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1491.997458 secs 2024-04-26T19:00:11.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1895 sec 2024-04-26T19:00:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1492.666945 secs 2024-04-26T19:00:11.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:11 smithi119 ceph-mon[32752]: pgmap v1597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:11.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1492.666945 secs 2024-04-26T19:00:11.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:11 smithi167 ceph-mon[32751]: pgmap v1597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1492.666945 secs 2024-04-26T19:00:11.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:11 smithi090 ceph-mon[29180]: pgmap v1597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:00:12.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1896 sec 2024-04-26T19:00:13.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1897 sec 2024-04-26T19:00:13.659 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:00:13.660+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:00:13.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:00:13.660+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 16 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:00:13.682 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:00:13.670+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac0066b0 con 0x56302893f0e0 2024-04-26T19:00:13.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:00:13.670+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 16 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac0066b0 con 0x56302893f0e0 2024-04-26T19:00:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:13 smithi090 ceph-mon[29180]: pgmap v1598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:00:14.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1898 sec 2024-04-26T19:00:14.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:13 smithi119 ceph-mon[32752]: pgmap v1598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:00:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:13 smithi167 ceph-mon[32751]: pgmap v1598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:00:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:14 smithi090 ceph-mon[29180]: pgmap v1599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:15.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1899 sec 2024-04-26T19:00:15.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:14 smithi167 ceph-mon[32751]: pgmap v1599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:15.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:14 smithi119 ceph-mon[32752]: pgmap v1599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1496.997546 secs 2024-04-26T19:00:16.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1900 sec 2024-04-26T19:00:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1496.997546 secs 2024-04-26T19:00:16.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1496.997546 secs 2024-04-26T19:00:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1497.667066 secs 2024-04-26T19:00:16.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:16 smithi090 ceph-mon[29180]: pgmap v1600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:17.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1901 sec 2024-04-26T19:00:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1497.667066 secs 2024-04-26T19:00:17.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:16 smithi119 ceph-mon[32752]: pgmap v1600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:17.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1497.667066 secs 2024-04-26T19:00:17.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:16 smithi167 ceph-mon[32751]: pgmap v1600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:18.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1902 sec 2024-04-26T19:00:19.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1903 sec 2024-04-26T19:00:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:19 smithi119 ceph-mon[32752]: pgmap v1601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:19.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:19 smithi167 ceph-mon[32751]: pgmap v1601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:19 smithi090 ceph-mon[29180]: pgmap v1601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:20.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1904 sec 2024-04-26T19:00:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1501.997628 secs 2024-04-26T19:00:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1501.997628 secs 2024-04-26T19:00:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1501.997628 secs 2024-04-26T19:00:21.049 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1905 sec 2024-04-26T19:00:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1502.667184 secs 2024-04-26T19:00:21.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:21 smithi119 ceph-mon[32752]: pgmap v1602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:21.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1502.667184 secs 2024-04-26T19:00:21.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:21 smithi167 ceph-mon[32751]: pgmap v1602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1502.667184 secs 2024-04-26T19:00:21.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:21 smithi090 ceph-mon[29180]: pgmap v1602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:22.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1906 sec 2024-04-26T19:00:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:22 smithi090 ceph-mon[29180]: pgmap v1603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:23.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1907 sec 2024-04-26T19:00:23.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:22 smithi167 ceph-mon[32751]: pgmap v1603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:23.124 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:22 smithi119 ceph-mon[32752]: pgmap v1603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:24.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1908 sec 2024-04-26T19:00:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:24 smithi090 ceph-mon[29180]: pgmap v1604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:25.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1909 sec 2024-04-26T19:00:25.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:24 smithi119 ceph-mon[32752]: pgmap v1604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:24 smithi167 ceph-mon[32751]: pgmap v1604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1506.997715 secs 2024-04-26T19:00:25.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:25 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1507.667279 secs 2024-04-26T19:00:26.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1910 sec 2024-04-26T19:00:26.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1506.997715 secs 2024-04-26T19:00:26.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:25 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1507.667279 secs 2024-04-26T19:00:26.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1506.997715 secs 2024-04-26T19:00:26.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:25 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1507.667279 secs 2024-04-26T19:00:27.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1911 sec 2024-04-26T19:00:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:26 smithi167 ceph-mon[32751]: pgmap v1605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:27.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:26 smithi119 ceph-mon[32752]: pgmap v1605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:26 smithi090 ceph-mon[29180]: pgmap v1605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:28.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1912 sec 2024-04-26T19:00:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:28 smithi090 ceph-mon[29180]: pgmap v1606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:29.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1913 sec 2024-04-26T19:00:29.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:28 smithi167 ceph-mon[32751]: pgmap v1606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:29.081 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:28 smithi119 ceph-mon[32752]: pgmap v1606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:30.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1914 sec 2024-04-26T19:00:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1511.997798 secs 2024-04-26T19:00:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1511.997798 secs 2024-04-26T19:00:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1511.997798 secs 2024-04-26T19:00:31.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1915 sec 2024-04-26T19:00:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1512.667405 secs 2024-04-26T19:00:31.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:31 smithi167 ceph-mon[32751]: pgmap v1607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1512.667405 secs 2024-04-26T19:00:31.569 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:31 smithi119 ceph-mon[32752]: pgmap v1607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1512.667405 secs 2024-04-26T19:00:31.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:31 smithi090 ceph-mon[29180]: pgmap v1607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:32.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1916 sec 2024-04-26T19:00:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:32 smithi090 ceph-mon[29180]: pgmap v1608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:33.050 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1917 sec 2024-04-26T19:00:33.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:32 smithi119 ceph-mon[32752]: pgmap v1608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:33.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:32 smithi167 ceph-mon[32751]: pgmap v1608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:34.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1918 sec 2024-04-26T19:00:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:34 smithi090 ceph-mon[29180]: pgmap v1609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:35.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1919 sec 2024-04-26T19:00:35.141 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:34 smithi167 ceph-mon[32751]: pgmap v1609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:35.151 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:34 smithi119 ceph-mon[32752]: pgmap v1609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:36.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1920 sec 2024-04-26T19:00:36.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1516.997868 secs 2024-04-26T19:00:36.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:35 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1517.667519 secs 2024-04-26T19:00:36.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1516.997868 secs 2024-04-26T19:00:36.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:35 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1517.667519 secs 2024-04-26T19:00:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1516.997868 secs 2024-04-26T19:00:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:35 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1517.667519 secs 2024-04-26T19:00:37.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1921 sec 2024-04-26T19:00:37.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:36 smithi119 ceph-mon[32752]: pgmap v1610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:37.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:36 smithi167 ceph-mon[32751]: pgmap v1610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:36 smithi090 ceph-mon[29180]: pgmap v1610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:38.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1922 sec 2024-04-26T19:00:39.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1923 sec 2024-04-26T19:00:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:39 smithi119 ceph-mon[32752]: pgmap v1611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:39 smithi167 ceph-mon[32751]: pgmap v1611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:39 smithi090 ceph-mon[29180]: pgmap v1611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:40.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1924 sec 2024-04-26T19:00:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1521.997948 secs 2024-04-26T19:00:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1521.997948 secs 2024-04-26T19:00:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1521.997948 secs 2024-04-26T19:00:41.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1925 sec 2024-04-26T19:00:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1522.667612 secs 2024-04-26T19:00:41.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:41 smithi119 ceph-mon[32752]: pgmap v1612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1522.667612 secs 2024-04-26T19:00:41.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:41 smithi167 ceph-mon[32751]: pgmap v1612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1522.667612 secs 2024-04-26T19:00:41.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:41 smithi090 ceph-mon[29180]: pgmap v1612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:42.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1926 sec 2024-04-26T19:00:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:42 smithi090 ceph-mon[29180]: pgmap v1613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:43.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1927 sec 2024-04-26T19:00:43.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:42 smithi119 ceph-mon[32752]: pgmap v1613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:43.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:42 smithi167 ceph-mon[32751]: pgmap v1613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:44.051 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1928 sec 2024-04-26T19:00:45.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1929 sec 2024-04-26T19:00:45.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:44 smithi167 ceph-mon[32751]: pgmap v1614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:45.145 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:44 smithi119 ceph-mon[32752]: pgmap v1614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:45.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:44 smithi090 ceph-mon[29180]: pgmap v1614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:46.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1930 sec 2024-04-26T19:00:46.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1526.998036 secs 2024-04-26T19:00:46.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:45 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1527.667689 secs 2024-04-26T19:00:46.110 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1526.998036 secs 2024-04-26T19:00:46.120 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:45 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1527.667689 secs 2024-04-26T19:00:46.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1526.998036 secs 2024-04-26T19:00:46.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:45 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1527.667689 secs 2024-04-26T19:00:47.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1931 sec 2024-04-26T19:00:47.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:46 smithi167 ceph-mon[32751]: pgmap v1615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:47.125 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:46 smithi119 ceph-mon[32752]: pgmap v1615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:47.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:46 smithi090 ceph-mon[29180]: pgmap v1615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:48.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1932 sec 2024-04-26T19:00:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:48 smithi090 ceph-mon[29180]: pgmap v1616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:49.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1933 sec 2024-04-26T19:00:49.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:48 smithi119 ceph-mon[32752]: pgmap v1616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:49.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:48 smithi167 ceph-mon[32751]: pgmap v1616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:50.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1934 sec 2024-04-26T19:00:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1531.998118 secs 2024-04-26T19:00:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1531.998118 secs 2024-04-26T19:00:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1531.998118 secs 2024-04-26T19:00:51.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1935 sec 2024-04-26T19:00:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1532.667750 secs 2024-04-26T19:00:51.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:51 smithi119 ceph-mon[32752]: pgmap v1617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1532.667750 secs 2024-04-26T19:00:51.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:51 smithi167 ceph-mon[32751]: pgmap v1617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1532.667750 secs 2024-04-26T19:00:51.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:51 smithi090 ceph-mon[29180]: pgmap v1617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:52.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1936 sec 2024-04-26T19:00:53.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1937 sec 2024-04-26T19:00:53.140 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:52 smithi167 ceph-mon[32751]: pgmap v1618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:53.148 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:52 smithi119 ceph-mon[32752]: pgmap v1618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:53.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:52 smithi090 ceph-mon[29180]: pgmap v1618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:54.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1938 sec 2024-04-26T19:00:55.052 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1939 sec 2024-04-26T19:00:55.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:54 smithi119 ceph-mon[32752]: pgmap v1619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:54 smithi167 ceph-mon[32751]: pgmap v1619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:54 smithi090 ceph-mon[29180]: pgmap v1619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:56.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1940 sec 2024-04-26T19:00:56.139 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1536.998204 secs 2024-04-26T19:00:56.147 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1537.667813 secs 2024-04-26T19:00:56.156 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1536.998204 secs 2024-04-26T19:00:56.166 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1537.667813 secs 2024-04-26T19:00:56.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1536.998204 secs 2024-04-26T19:00:56.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1537.667813 secs 2024-04-26T19:00:57.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1941 sec 2024-04-26T19:00:57.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:56 smithi119 ceph-mon[32752]: pgmap v1620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:56 smithi167 ceph-mon[32751]: pgmap v1620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:56 smithi090 ceph-mon[29180]: pgmap v1620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:00:58.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1942 sec 2024-04-26T19:00:59.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1943 sec 2024-04-26T19:00:59.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:00:58 smithi167 ceph-mon[32751]: pgmap v1621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:59.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:00:58 smithi119 ceph-mon[32752]: pgmap v1621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:00:59.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:00:58 smithi090 ceph-mon[29180]: pgmap v1621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:00.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1944 sec 2024-04-26T19:01:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1541.998286 secs 2024-04-26T19:01:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1541.998286 secs 2024-04-26T19:01:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1541.998286 secs 2024-04-26T19:01:01.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1945 sec 2024-04-26T19:01:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1542.667931 secs 2024-04-26T19:01:01.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:01 smithi167 ceph-mon[32751]: pgmap v1622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1542.667931 secs 2024-04-26T19:01:01.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:01 smithi119 ceph-mon[32752]: pgmap v1622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1542.667931 secs 2024-04-26T19:01:01.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:01 smithi090 ceph-mon[29180]: pgmap v1622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:02.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1946 sec 2024-04-26T19:01:03.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1947 sec 2024-04-26T19:01:03.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:02 smithi167 ceph-mon[32751]: pgmap v1623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:03.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:01:03.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:02 smithi119 ceph-mon[32752]: pgmap v1623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:03.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:01:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:02 smithi090 ceph-mon[29180]: pgmap v1623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:03.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:01:04.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1948 sec 2024-04-26T19:01:04.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:01:04.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:01:04.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:03 smithi167 ceph-mon[32751]: pgmap v1624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:01:04.162 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:03 smithi167 ceph-mon[32751]: pgmap v1625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:04.170 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:01:04.225 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:01:04.237 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:01:04.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:03 smithi119 ceph-mon[32752]: pgmap v1624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:01:04.254 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:03 smithi119 ceph-mon[32752]: pgmap v1625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:04.261 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:01:04.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:01:04.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:01:04.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:03 smithi090 ceph-mon[29180]: pgmap v1624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:01:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:03 smithi090 ceph-mon[29180]: pgmap v1625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:04.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:01:05.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1949 sec 2024-04-26T19:01:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1546.998371 secs 2024-04-26T19:01:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1546.998371 secs 2024-04-26T19:01:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1546.998371 secs 2024-04-26T19:01:06.053 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1950 sec 2024-04-26T19:01:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:06 smithi119 ceph-mon[32752]: pgmap v1626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:06.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1547.668020 secs 2024-04-26T19:01:06.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:06 smithi167 ceph-mon[32751]: pgmap v1626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:06.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1547.668020 secs 2024-04-26T19:01:06.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:06 smithi090 ceph-mon[29180]: pgmap v1626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:06.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1547.668020 secs 2024-04-26T19:01:07.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1951 sec 2024-04-26T19:01:08.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1952 sec 2024-04-26T19:01:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:08 smithi090 ceph-mon[29180]: pgmap v1627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:01:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:08 smithi119 ceph-mon[32752]: pgmap v1627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:01:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:08 smithi167 ceph-mon[32751]: pgmap v1627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:01:09.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1953 sec 2024-04-26T19:01:10.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1954 sec 2024-04-26T19:01:10.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:10 smithi119 ceph-mon[32752]: pgmap v1628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:10.656 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1551.998452 secs 2024-04-26T19:01:10.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:10 smithi090 ceph-mon[29180]: pgmap v1628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:10.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1551.998452 secs 2024-04-26T19:01:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:10 smithi167 ceph-mon[32751]: pgmap v1628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1551.998452 secs 2024-04-26T19:01:11.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1955 sec 2024-04-26T19:01:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1552.668157 secs 2024-04-26T19:01:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1552.668157 secs 2024-04-26T19:01:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1552.668157 secs 2024-04-26T19:01:12.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1956 sec 2024-04-26T19:01:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:12 smithi090 ceph-mon[29180]: pgmap v1629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:12 smithi119 ceph-mon[32752]: pgmap v1629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:12 smithi167 ceph-mon[32751]: pgmap v1629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:01:13.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1957 sec 2024-04-26T19:01:14.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1958 sec 2024-04-26T19:01:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:14 smithi090 ceph-mon[29180]: pgmap v1630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:01:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:14 smithi119 ceph-mon[32752]: pgmap v1630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:01:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:14 smithi167 ceph-mon[32751]: pgmap v1630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:01:15.054 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1959 sec 2024-04-26T19:01:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1556.998513 secs 2024-04-26T19:01:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1556.998513 secs 2024-04-26T19:01:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1556.998513 secs 2024-04-26T19:01:16.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1960 sec 2024-04-26T19:01:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:16 smithi090 ceph-mon[29180]: pgmap v1631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:16.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1557.668227 secs 2024-04-26T19:01:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:16 smithi119 ceph-mon[32752]: pgmap v1631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:16.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1557.668227 secs 2024-04-26T19:01:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:16 smithi167 ceph-mon[32751]: pgmap v1631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:16.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1557.668227 secs 2024-04-26T19:01:17.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1961 sec 2024-04-26T19:01:18.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1962 sec 2024-04-26T19:01:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:18 smithi090 ceph-mon[29180]: pgmap v1632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:18 smithi119 ceph-mon[32752]: pgmap v1632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:18 smithi167 ceph-mon[32751]: pgmap v1632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:19.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1963 sec 2024-04-26T19:01:20.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1964 sec 2024-04-26T19:01:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:20 smithi090 ceph-mon[29180]: pgmap v1633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:20.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1561.998599 secs 2024-04-26T19:01:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:20 smithi119 ceph-mon[32752]: pgmap v1633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:20.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1561.998599 secs 2024-04-26T19:01:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:20 smithi167 ceph-mon[32751]: pgmap v1633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:20.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1561.998599 secs 2024-04-26T19:01:21.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1965 sec 2024-04-26T19:01:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1562.668341 secs 2024-04-26T19:01:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1562.668341 secs 2024-04-26T19:01:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1562.668341 secs 2024-04-26T19:01:22.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1966 sec 2024-04-26T19:01:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:22 smithi090 ceph-mon[29180]: pgmap v1634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:22 smithi119 ceph-mon[32752]: pgmap v1634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:22 smithi167 ceph-mon[32751]: pgmap v1634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:23.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1967 sec 2024-04-26T19:01:24.055 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1968 sec 2024-04-26T19:01:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:24 smithi119 ceph-mon[32752]: pgmap v1635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:24 smithi167 ceph-mon[32751]: pgmap v1635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:24 smithi090 ceph-mon[29180]: pgmap v1635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:25.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1969 sec 2024-04-26T19:01:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1566.998683 secs 2024-04-26T19:01:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1566.998683 secs 2024-04-26T19:01:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1566.998683 secs 2024-04-26T19:01:26.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1970 sec 2024-04-26T19:01:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:26 smithi119 ceph-mon[32752]: pgmap v1636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:26.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1567.668456 secs 2024-04-26T19:01:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:26 smithi167 ceph-mon[32751]: pgmap v1636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:26.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1567.668456 secs 2024-04-26T19:01:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:26 smithi090 ceph-mon[29180]: pgmap v1636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:26.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1567.668456 secs 2024-04-26T19:01:27.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1971 sec 2024-04-26T19:01:28.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1972 sec 2024-04-26T19:01:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:28 smithi119 ceph-mon[32752]: pgmap v1637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:28 smithi167 ceph-mon[32751]: pgmap v1637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:28 smithi090 ceph-mon[29180]: pgmap v1637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:29.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1973 sec 2024-04-26T19:01:30.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1974 sec 2024-04-26T19:01:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:30 smithi119 ceph-mon[32752]: pgmap v1638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:30.816 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1571.998740 secs 2024-04-26T19:01:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:30 smithi167 ceph-mon[32751]: pgmap v1638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:30.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1571.998740 secs 2024-04-26T19:01:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:30 smithi090 ceph-mon[29180]: pgmap v1638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:30.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1571.998740 secs 2024-04-26T19:01:31.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1975 sec 2024-04-26T19:01:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1572.668524 secs 2024-04-26T19:01:31.699 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:01:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:01:31.216+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:01:31.707 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:01:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:01:31.216+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:01:31.714 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:01:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:01:31.216+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:01:31.722 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:01:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:01:31.216+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:01:31.729 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:01:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:01:31.216+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:01:31.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1572.668524 secs 2024-04-26T19:01:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1572.668524 secs 2024-04-26T19:01:32.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1976 sec 2024-04-26T19:01:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:32 smithi119 ceph-mon[32752]: pgmap v1639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:32.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:32 smithi167 ceph-mon[32751]: pgmap v1639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:32 smithi090 ceph-mon[29180]: pgmap v1639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:33.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1977 sec 2024-04-26T19:01:34.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1978 sec 2024-04-26T19:01:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:34 smithi119 ceph-mon[32752]: pgmap v1640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:01:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:34 smithi167 ceph-mon[32751]: pgmap v1640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:01:34.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:34 smithi090 ceph-mon[29180]: pgmap v1640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:01:35.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1979 sec 2024-04-26T19:01:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:35 smithi119 ceph-mon[32752]: 7 slow requests, 1 included below; oldest blocked for > 1576.998830 secs 2024-04-26T19:01:35.819 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:35 smithi119 ceph-mon[32752]: slow request 1441.262182 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T19:01:35.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:35 smithi167 ceph-mon[32751]: 7 slow requests, 1 included below; oldest blocked for > 1576.998830 secs 2024-04-26T19:01:35.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:35 smithi167 ceph-mon[32751]: slow request 1441.262182 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T19:01:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:35 smithi090 ceph-mon[29180]: 7 slow requests, 1 included below; oldest blocked for > 1576.998830 secs 2024-04-26T19:01:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:35 smithi090 ceph-mon[29180]: slow request 1441.262182 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T19:01:36.056 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1980 sec 2024-04-26T19:01:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:36 smithi119 ceph-mon[32752]: pgmap v1641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:36.825 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1577.668653 secs 2024-04-26T19:01:36.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:36 smithi167 ceph-mon[32751]: pgmap v1641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:36.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1577.668653 secs 2024-04-26T19:01:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:36 smithi090 ceph-mon[29180]: pgmap v1641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:36.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1577.668653 secs 2024-04-26T19:01:37.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1981 sec 2024-04-26T19:01:38.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1982 sec 2024-04-26T19:01:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:38 smithi119 ceph-mon[32752]: pgmap v1642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:38.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:38 smithi167 ceph-mon[32751]: pgmap v1642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:38 smithi090 ceph-mon[29180]: pgmap v1642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:39.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1983 sec 2024-04-26T19:01:40.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1984 sec 2024-04-26T19:01:40.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:40 smithi090 ceph-mon[29180]: pgmap v1643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T19:01:40.728 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1581.998911 secs 2024-04-26T19:01:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:40 smithi119 ceph-mon[32752]: pgmap v1643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T19:01:40.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1581.998911 secs 2024-04-26T19:01:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:40 smithi167 ceph-mon[32751]: pgmap v1643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T19:01:40.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1581.998911 secs 2024-04-26T19:01:41.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1985 sec 2024-04-26T19:01:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1582.668704 secs 2024-04-26T19:01:41.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1582.668704 secs 2024-04-26T19:01:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1582.668704 secs 2024-04-26T19:01:42.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1986 sec 2024-04-26T19:01:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:42 smithi119 ceph-mon[32752]: pgmap v1644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:42.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:42 smithi167 ceph-mon[32751]: pgmap v1644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:42 smithi090 ceph-mon[29180]: pgmap v1644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T19:01:43.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1987 sec 2024-04-26T19:01:44.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1988 sec 2024-04-26T19:01:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:44 smithi119 ceph-mon[32752]: pgmap v1645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T19:01:44.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:44 smithi167 ceph-mon[32751]: pgmap v1645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T19:01:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:44 smithi090 ceph-mon[29180]: pgmap v1645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T19:01:45.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1989 sec 2024-04-26T19:01:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1586.998973 secs 2024-04-26T19:01:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1586.998973 secs 2024-04-26T19:01:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1586.998973 secs 2024-04-26T19:01:46.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1990 sec 2024-04-26T19:01:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:46 smithi119 ceph-mon[32752]: pgmap v1646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:01:46.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1587.668840 secs 2024-04-26T19:01:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:46 smithi167 ceph-mon[32751]: pgmap v1646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:01:46.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1587.668840 secs 2024-04-26T19:01:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:46 smithi090 ceph-mon[29180]: pgmap v1646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:01:46.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1587.668840 secs 2024-04-26T19:01:47.057 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1991 sec 2024-04-26T19:01:48.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1992 sec 2024-04-26T19:01:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:48 smithi119 ceph-mon[32752]: pgmap v1647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:48 smithi167 ceph-mon[32751]: pgmap v1647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:48 smithi090 ceph-mon[29180]: pgmap v1647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:49.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1993 sec 2024-04-26T19:01:50.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1994 sec 2024-04-26T19:01:50.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:50 smithi090 ceph-mon[29180]: pgmap v1648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:50.701 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1591.999060 secs 2024-04-26T19:01:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:50 smithi119 ceph-mon[32752]: pgmap v1648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:50.825 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1591.999060 secs 2024-04-26T19:01:50.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:50 smithi167 ceph-mon[32751]: pgmap v1648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:50.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1591.999060 secs 2024-04-26T19:01:51.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1995 sec 2024-04-26T19:01:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1592.668976 secs 2024-04-26T19:01:51.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1592.668976 secs 2024-04-26T19:01:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1592.668976 secs 2024-04-26T19:01:52.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1996 sec 2024-04-26T19:01:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:52 smithi119 ceph-mon[32752]: pgmap v1649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:52 smithi167 ceph-mon[32751]: pgmap v1649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:52 smithi090 ceph-mon[29180]: pgmap v1649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:53.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1997 sec 2024-04-26T19:01:54.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1998 sec 2024-04-26T19:01:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:54 smithi119 ceph-mon[32752]: pgmap v1650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:54 smithi167 ceph-mon[32751]: pgmap v1650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:54 smithi090 ceph-mon[29180]: pgmap v1650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:01:55.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 1999 sec 2024-04-26T19:01:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1596.999146 secs 2024-04-26T19:01:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1596.999146 secs 2024-04-26T19:01:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1596.999146 secs 2024-04-26T19:01:56.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2000 sec 2024-04-26T19:01:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:56 smithi119 ceph-mon[32752]: pgmap v1651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:56.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1597.669089 secs 2024-04-26T19:01:56.874 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:56 smithi167 ceph-mon[32751]: pgmap v1651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:56.882 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1597.669089 secs 2024-04-26T19:01:56.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:56 smithi090 ceph-mon[29180]: pgmap v1651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:56.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1597.669089 secs 2024-04-26T19:01:57.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2001 sec 2024-04-26T19:01:58.058 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2002 sec 2024-04-26T19:01:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:01:58 smithi119 ceph-mon[32752]: pgmap v1652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:58.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:01:58 smithi167 ceph-mon[32751]: pgmap v1652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:01:58 smithi090 ceph-mon[29180]: pgmap v1652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:01:59.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2003 sec 2024-04-26T19:02:00.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2004 sec 2024-04-26T19:02:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:00 smithi119 ceph-mon[32752]: pgmap v1653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:00.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1601.999229 secs 2024-04-26T19:02:00.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:00 smithi167 ceph-mon[32751]: pgmap v1653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:00.847 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1601.999229 secs 2024-04-26T19:02:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:00 smithi090 ceph-mon[29180]: pgmap v1653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1601.999229 secs 2024-04-26T19:02:01.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2005 sec 2024-04-26T19:02:01.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1602.669211 secs 2024-04-26T19:02:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1602.669211 secs 2024-04-26T19:02:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1602.669211 secs 2024-04-26T19:02:02.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2006 sec 2024-04-26T19:02:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:02 smithi119 ceph-mon[32752]: pgmap v1654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:02 smithi167 ceph-mon[32751]: pgmap v1654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:02 smithi090 ceph-mon[29180]: pgmap v1654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:03.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2007 sec 2024-04-26T19:02:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:02:03.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:02:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:02:04.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2008 sec 2024-04-26T19:02:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:04 smithi119 ceph-mon[32752]: pgmap v1655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:04.834 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:02:04.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:02:04.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:02:04.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:04 smithi167 ceph-mon[32751]: pgmap v1655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:04.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:02:04.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:02:04.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:02:04.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:04 smithi090 ceph-mon[29180]: pgmap v1655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:04.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:02:05.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:02:05.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:02:05.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2009 sec 2024-04-26T19:02:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:05 smithi119 ceph-mon[32752]: pgmap v1656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:05.826 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:05 smithi119 ceph-mon[32752]: pgmap v1657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:05.834 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1606.999313 secs 2024-04-26T19:02:05.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:05 smithi167 ceph-mon[32751]: pgmap v1656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:05.915 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:05 smithi167 ceph-mon[32751]: pgmap v1657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:06.001 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1606.999313 secs 2024-04-26T19:02:06.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:05 smithi090 ceph-mon[29180]: pgmap v1656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:06.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:05 smithi090 ceph-mon[29180]: pgmap v1657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:06.024 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1606.999313 secs 2024-04-26T19:02:06.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2010 sec 2024-04-26T19:02:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1607.669330 secs 2024-04-26T19:02:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1607.669330 secs 2024-04-26T19:02:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1607.669330 secs 2024-04-26T19:02:07.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2011 sec 2024-04-26T19:02:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:07 smithi119 ceph-mon[32752]: pgmap v1658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:07 smithi167 ceph-mon[32751]: pgmap v1658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:07 smithi090 ceph-mon[29180]: pgmap v1658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:08.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2012 sec 2024-04-26T19:02:09.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2013 sec 2024-04-26T19:02:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:09 smithi119 ceph-mon[32752]: pgmap v1659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:02:09.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:09 smithi167 ceph-mon[32751]: pgmap v1659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:02:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:09 smithi090 ceph-mon[29180]: pgmap v1659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:02:10.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2014 sec 2024-04-26T19:02:10.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1611.999393 secs 2024-04-26T19:02:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1611.999393 secs 2024-04-26T19:02:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1611.999393 secs 2024-04-26T19:02:11.059 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2015 sec 2024-04-26T19:02:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:11 smithi119 ceph-mon[32752]: pgmap v1660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:11.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1612.669453 secs 2024-04-26T19:02:11.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:11 smithi167 ceph-mon[32751]: pgmap v1660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:11.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1612.669453 secs 2024-04-26T19:02:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:11 smithi090 ceph-mon[29180]: pgmap v1660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:02:11.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1612.669453 secs 2024-04-26T19:02:12.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2016 sec 2024-04-26T19:02:13.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2017 sec 2024-04-26T19:02:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:13 smithi119 ceph-mon[32752]: pgmap v1661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 B/s rd, 1 op/s 2024-04-26T19:02:13.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:13 smithi167 ceph-mon[32751]: pgmap v1661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 B/s rd, 1 op/s 2024-04-26T19:02:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:13 smithi090 ceph-mon[29180]: pgmap v1661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 B/s rd, 1 op/s 2024-04-26T19:02:14.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2018 sec 2024-04-26T19:02:15.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2019 sec 2024-04-26T19:02:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:15 smithi119 ceph-mon[32752]: pgmap v1662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:02:15.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1616.999477 secs 2024-04-26T19:02:15.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:15 smithi167 ceph-mon[32751]: pgmap v1662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:02:15.873 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1616.999477 secs 2024-04-26T19:02:15.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:15 smithi090 ceph-mon[29180]: pgmap v1662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:02:15.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1616.999477 secs 2024-04-26T19:02:16.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2020 sec 2024-04-26T19:02:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1617.669515 secs 2024-04-26T19:02:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1617.669515 secs 2024-04-26T19:02:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1617.669515 secs 2024-04-26T19:02:17.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2021 sec 2024-04-26T19:02:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:17 smithi119 ceph-mon[32752]: pgmap v1663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:17 smithi167 ceph-mon[32751]: pgmap v1663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:17 smithi090 ceph-mon[29180]: pgmap v1663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:18.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2022 sec 2024-04-26T19:02:19.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2023 sec 2024-04-26T19:02:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:19 smithi119 ceph-mon[32752]: pgmap v1664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:19.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:19 smithi167 ceph-mon[32751]: pgmap v1664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:19 smithi090 ceph-mon[29180]: pgmap v1664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:20.060 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2024 sec 2024-04-26T19:02:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1621.999555 secs 2024-04-26T19:02:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1621.999555 secs 2024-04-26T19:02:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1621.999555 secs 2024-04-26T19:02:21.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2025 sec 2024-04-26T19:02:21.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:21 smithi090 ceph-mon[29180]: pgmap v1665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:21.839 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1622.669644 secs 2024-04-26T19:02:21.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:21 smithi167 ceph-mon[32751]: pgmap v1665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:21.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1622.669644 secs 2024-04-26T19:02:21.865 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:21 smithi119 ceph-mon[32752]: pgmap v1665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:21.873 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1622.669644 secs 2024-04-26T19:02:22.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2026 sec 2024-04-26T19:02:23.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2027 sec 2024-04-26T19:02:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:23 smithi119 ceph-mon[32752]: pgmap v1666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:23 smithi167 ceph-mon[32751]: pgmap v1666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:23 smithi090 ceph-mon[29180]: pgmap v1666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:24.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2028 sec 2024-04-26T19:02:25.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2029 sec 2024-04-26T19:02:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:25 smithi119 ceph-mon[32752]: pgmap v1667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:25.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1626.999642 secs 2024-04-26T19:02:25.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:25 smithi167 ceph-mon[32751]: pgmap v1667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:25.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1626.999642 secs 2024-04-26T19:02:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:25 smithi090 ceph-mon[29180]: pgmap v1667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:25.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1626.999642 secs 2024-04-26T19:02:26.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2030 sec 2024-04-26T19:02:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1627.669738 secs 2024-04-26T19:02:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1627.669738 secs 2024-04-26T19:02:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1627.669738 secs 2024-04-26T19:02:27.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2031 sec 2024-04-26T19:02:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:27 smithi119 ceph-mon[32752]: pgmap v1668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:27.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:27 smithi090 ceph-mon[29180]: pgmap v1668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:27.918 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:27 smithi167 ceph-mon[32751]: pgmap v1668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:28.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2032 sec 2024-04-26T19:02:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:02:28.678+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:02:28.692 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:02:28.678+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 17 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:02:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:02:28.686+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:02:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:02:28.687+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 17 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:02:29.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2033 sec 2024-04-26T19:02:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:29 smithi119 ceph-mon[32752]: pgmap v1669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:29 smithi167 ceph-mon[32751]: pgmap v1669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:29 smithi090 ceph-mon[29180]: pgmap v1669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:30.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2034 sec 2024-04-26T19:02:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1631.999726 secs 2024-04-26T19:02:30.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1631.999726 secs 2024-04-26T19:02:30.919 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1631.999726 secs 2024-04-26T19:02:31.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2035 sec 2024-04-26T19:02:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:31 smithi119 ceph-mon[32752]: pgmap v1670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:31.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1632.669853 secs 2024-04-26T19:02:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:31 smithi167 ceph-mon[32751]: pgmap v1670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:31.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1632.669853 secs 2024-04-26T19:02:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:31 smithi090 ceph-mon[29180]: pgmap v1670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:31.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1632.669853 secs 2024-04-26T19:02:32.061 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2036 sec 2024-04-26T19:02:33.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2037 sec 2024-04-26T19:02:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:33 smithi119 ceph-mon[32752]: pgmap v1671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:33 smithi167 ceph-mon[32751]: pgmap v1671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:33 smithi090 ceph-mon[29180]: pgmap v1671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:34.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2038 sec 2024-04-26T19:02:35.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2039 sec 2024-04-26T19:02:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:35 smithi119 ceph-mon[32752]: pgmap v1672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:35.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1636.999810 secs 2024-04-26T19:02:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:35 smithi167 ceph-mon[32751]: pgmap v1672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:35.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1636.999810 secs 2024-04-26T19:02:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:35 smithi090 ceph-mon[29180]: pgmap v1672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1636.999810 secs 2024-04-26T19:02:36.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2040 sec 2024-04-26T19:02:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:36 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 1637.669914 secs 2024-04-26T19:02:36.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:36 smithi167 ceph-mon[32751]: slow request 364.306778 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:02:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:36 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 1637.669914 secs 2024-04-26T19:02:36.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:36 smithi090 ceph-mon[29180]: slow request 364.306778 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:36 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 1637.669914 secs 2024-04-26T19:02:37.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:36 smithi119 ceph-mon[32752]: slow request 364.306778 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:02:37.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2041 sec 2024-04-26T19:02:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:37 smithi167 ceph-mon[32751]: pgmap v1673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:37 smithi090 ceph-mon[29180]: pgmap v1673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:37 smithi119 ceph-mon[32752]: pgmap v1673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:38.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2042 sec 2024-04-26T19:02:39.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2043 sec 2024-04-26T19:02:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:39 smithi167 ceph-mon[32751]: pgmap v1674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:39 smithi090 ceph-mon[29180]: pgmap v1674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:39 smithi119 ceph-mon[32752]: pgmap v1674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:40.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2044 sec 2024-04-26T19:02:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1641.999897 secs 2024-04-26T19:02:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1641.999897 secs 2024-04-26T19:02:40.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1641.999897 secs 2024-04-26T19:02:41.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2045 sec 2024-04-26T19:02:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:41 smithi167 ceph-mon[32751]: pgmap v1675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:41.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1642.670066 secs 2024-04-26T19:02:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:41 smithi090 ceph-mon[29180]: pgmap v1675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:41.922 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1642.670066 secs 2024-04-26T19:02:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:41 smithi119 ceph-mon[32752]: pgmap v1675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:42.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1642.670066 secs 2024-04-26T19:02:42.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2046 sec 2024-04-26T19:02:43.062 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2047 sec 2024-04-26T19:02:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:43 smithi090 ceph-mon[29180]: pgmap v1676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:43 smithi119 ceph-mon[32752]: pgmap v1676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:44.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2048 sec 2024-04-26T19:02:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:43 smithi167 ceph-mon[32751]: pgmap v1676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:45.063 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2049 sec 2024-04-26T19:02:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:45 smithi090 ceph-mon[29180]: pgmap v1677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:45.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1646.999956 secs 2024-04-26T19:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:45 smithi119 ceph-mon[32752]: pgmap v1677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1646.999956 secs 2024-04-26T19:02:46.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:45 smithi167 ceph-mon[32751]: pgmap v1677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:46.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1646.999956 secs 2024-04-26T19:02:46.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2050 sec 2024-04-26T19:02:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1647.670174 secs 2024-04-26T19:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1647.670174 secs 2024-04-26T19:02:47.063 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2051 sec 2024-04-26T19:02:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1647.670174 secs 2024-04-26T19:02:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:47 smithi090 ceph-mon[29180]: pgmap v1678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:47 smithi119 ceph-mon[32752]: pgmap v1678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:48.063 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2052 sec 2024-04-26T19:02:48.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:47 smithi167 ceph-mon[32751]: pgmap v1678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:49.063 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2053 sec 2024-04-26T19:02:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:49 smithi090 ceph-mon[29180]: pgmap v1679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:49 smithi119 ceph-mon[32752]: pgmap v1679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:50.063 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2054 sec 2024-04-26T19:02:50.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:49 smithi167 ceph-mon[32751]: pgmap v1679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1652.000017 secs 2024-04-26T19:02:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1652.000017 secs 2024-04-26T19:02:51.063 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2055 sec 2024-04-26T19:02:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1652.000017 secs 2024-04-26T19:02:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:51 smithi090 ceph-mon[29180]: pgmap v1680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:51.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1652.670291 secs 2024-04-26T19:02:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:51 smithi119 ceph-mon[32752]: pgmap v1680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:52.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1652.670291 secs 2024-04-26T19:02:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:51 smithi167 ceph-mon[32751]: pgmap v1680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:52.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1652.670291 secs 2024-04-26T19:02:52.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2056 sec 2024-04-26T19:02:53.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2057 sec 2024-04-26T19:02:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:53 smithi090 ceph-mon[29180]: pgmap v1681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:53 smithi119 ceph-mon[32752]: pgmap v1681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:54.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2058 sec 2024-04-26T19:02:54.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:53 smithi167 ceph-mon[32751]: pgmap v1681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:55.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2059 sec 2024-04-26T19:02:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:55 smithi090 ceph-mon[29180]: pgmap v1682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:55.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1657.000103 secs 2024-04-26T19:02:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:55 smithi119 ceph-mon[32752]: pgmap v1682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:56.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1657.000103 secs 2024-04-26T19:02:56.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:55 smithi167 ceph-mon[32751]: pgmap v1682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:02:56.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1657.000103 secs 2024-04-26T19:02:56.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2060 sec 2024-04-26T19:02:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1657.670434 secs 2024-04-26T19:02:56.903 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:56 smithi090 ceph-mon[29180]: pgmap v1683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1657.670434 secs 2024-04-26T19:02:57.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:56 smithi119 ceph-mon[32752]: pgmap v1683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1657.670434 secs 2024-04-26T19:02:57.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:56 smithi167 ceph-mon[32751]: pgmap v1683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:57.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2061 sec 2024-04-26T19:02:58.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2062 sec 2024-04-26T19:02:59.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2063 sec 2024-04-26T19:02:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:02:59 smithi119 ceph-mon[32752]: pgmap v1684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:02:59 smithi167 ceph-mon[32751]: pgmap v1684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:02:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:02:59 smithi090 ceph-mon[29180]: pgmap v1684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:00.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2064 sec 2024-04-26T19:03:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1662.000185 secs 2024-04-26T19:03:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1662.000185 secs 2024-04-26T19:03:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1662.000185 secs 2024-04-26T19:03:01.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2065 sec 2024-04-26T19:03:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:01 smithi167 ceph-mon[32751]: pgmap v1685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:01.515 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1662.670526 secs 2024-04-26T19:03:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:01 smithi119 ceph-mon[32752]: pgmap v1685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1662.670526 secs 2024-04-26T19:03:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:01 smithi090 ceph-mon[29180]: pgmap v1685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:01.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1662.670526 secs 2024-04-26T19:03:02.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2066 sec 2024-04-26T19:03:03.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2067 sec 2024-04-26T19:03:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:03 smithi119 ceph-mon[32752]: pgmap v1686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:03 smithi167 ceph-mon[32751]: pgmap v1686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:03 smithi090 ceph-mon[29180]: pgmap v1686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:04.064 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2068 sec 2024-04-26T19:03:05.065 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2069 sec 2024-04-26T19:03:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:05 smithi167 ceph-mon[32751]: pgmap v1687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:05.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:03:05.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1667.000250 secs 2024-04-26T19:03:05.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:05 smithi090 ceph-mon[29180]: pgmap v1687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:05.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:03:05.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.494 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.563 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1667.000250 secs 2024-04-26T19:03:05.629 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:05 smithi119 ceph-mon[32752]: pgmap v1687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:05.639 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:03:05.649 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.731 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.741 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:05.751 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1667.000250 secs 2024-04-26T19:03:06.065 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2070 sec 2024-04-26T19:03:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1667.670612 secs 2024-04-26T19:03:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1667.670612 secs 2024-04-26T19:03:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1667.670612 secs 2024-04-26T19:03:07.065 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2071 sec 2024-04-26T19:03:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: pgmap v1688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:07.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:07.722 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:07.730 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:07.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:07.890 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.898 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.907 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: pgmap v1688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:07.914 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.922 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.938 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:07.946 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:07.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:07.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:07.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: pgmap v1688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:08.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.078 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.123 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.216 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.225 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.233 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.241 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.249 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.256 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.264 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2072 sec 2024-04-26T19:03:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:03:08.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:03:08.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:03:08.762 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:03:08.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:08.916 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:03:08.924 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:03:08.932 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:03:08.940 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:03:08.948 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: pgmap v1689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T19:03:08.956 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: pgmap v1690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:08.964 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:08.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:03:08.980 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:03:09.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:03:09.139 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:03:09.148 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.187 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:03:09.196 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:03:09.206 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:03:09.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:03:09.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: pgmap v1689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T19:03:09.234 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: pgmap v1690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:09.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:09.326 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:03:09.336 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:03:09.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.406 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.489 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.498 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:03:09.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:03:09.517 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:03:09.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:03:09.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:03:09.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:03:09.583 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:03:09.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: pgmap v1689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T19:03:09.656 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: pgmap v1690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:09.665 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:03:09.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2073 sec 2024-04-26T19:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:09 smithi119 ceph-mon[32752]: pgmap v1691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:10.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:09 smithi167 ceph-mon[32751]: pgmap v1691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:10.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2074 sec 2024-04-26T19:03:10.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:09 smithi090 ceph-mon[29180]: pgmap v1691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:10.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1672.000312 secs 2024-04-26T19:03:10.967 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:10 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1672.670702 secs 2024-04-26T19:03:11.065 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2075 sec 2024-04-26T19:03:11.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1672.000312 secs 2024-04-26T19:03:11.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:10 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1672.670702 secs 2024-04-26T19:03:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1672.000312 secs 2024-04-26T19:03:11.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:10 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1672.670702 secs 2024-04-26T19:03:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:11 smithi119 ceph-mon[32752]: pgmap v1692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:12.065 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2076 sec 2024-04-26T19:03:12.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:11 smithi167 ceph-mon[32751]: pgmap v1692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:12.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:11 smithi090 ceph-mon[29180]: pgmap v1692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:13.065 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2077 sec 2024-04-26T19:03:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:13 smithi119 ceph-mon[32752]: pgmap v1693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:14.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:13 smithi167 ceph-mon[32751]: pgmap v1693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:14.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2078 sec 2024-04-26T19:03:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:13 smithi090 ceph-mon[29180]: pgmap v1693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:15.066 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2079 sec 2024-04-26T19:03:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1677.000396 secs 2024-04-26T19:03:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1677.000396 secs 2024-04-26T19:03:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1677.000396 secs 2024-04-26T19:03:16.066 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2080 sec 2024-04-26T19:03:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1677.670817 secs 2024-04-26T19:03:16.572 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:16 smithi119 ceph-mon[32752]: pgmap v1694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:16.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1677.670817 secs 2024-04-26T19:03:16.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:16 smithi167 ceph-mon[32751]: pgmap v1694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1677.670817 secs 2024-04-26T19:03:16.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:16 smithi090 ceph-mon[29180]: pgmap v1694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:03:17.066 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2081 sec 2024-04-26T19:03:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:17 smithi119 ceph-mon[32752]: pgmap v1695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:03:18.066 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2082 sec 2024-04-26T19:03:18.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:17 smithi167 ceph-mon[32751]: pgmap v1695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:03:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:17 smithi090 ceph-mon[29180]: pgmap v1695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:03:19.066 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2083 sec 2024-04-26T19:03:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:19 smithi119 ceph-mon[32752]: pgmap v1696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:20.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2084 sec 2024-04-26T19:03:20.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:19 smithi167 ceph-mon[32751]: pgmap v1696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:19 smithi090 ceph-mon[29180]: pgmap v1696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1682.000479 secs 2024-04-26T19:03:21.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:20 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1682.670930 secs 2024-04-26T19:03:21.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1682.000479 secs 2024-04-26T19:03:21.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:20 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1682.670930 secs 2024-04-26T19:03:21.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2085 sec 2024-04-26T19:03:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1682.000479 secs 2024-04-26T19:03:21.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:20 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1682.670930 secs 2024-04-26T19:03:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:21 smithi119 ceph-mon[32752]: pgmap v1697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:22.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2086 sec 2024-04-26T19:03:22.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:21 smithi167 ceph-mon[32751]: pgmap v1697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:21 smithi090 ceph-mon[29180]: pgmap v1697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:23.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2087 sec 2024-04-26T19:03:23.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:23 smithi090 ceph-mon[29180]: pgmap v1698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:23 smithi119 ceph-mon[32752]: pgmap v1698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:24.066 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2088 sec 2024-04-26T19:03:24.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:23 smithi167 ceph-mon[32751]: pgmap v1698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:25.066 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2089 sec 2024-04-26T19:03:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1687.000563 secs 2024-04-26T19:03:25.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1687.000563 secs 2024-04-26T19:03:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1687.000563 secs 2024-04-26T19:03:26.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2090 sec 2024-04-26T19:03:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1687.671062 secs 2024-04-26T19:03:26.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:26 smithi119 ceph-mon[32752]: pgmap v1699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:26.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1687.671062 secs 2024-04-26T19:03:26.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:26 smithi167 ceph-mon[32751]: pgmap v1699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1687.671062 secs 2024-04-26T19:03:26.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:26 smithi090 ceph-mon[29180]: pgmap v1699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:27.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2091 sec 2024-04-26T19:03:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:27 smithi119 ceph-mon[32752]: pgmap v1700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:28.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:27 smithi167 ceph-mon[32751]: pgmap v1700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:28.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2092 sec 2024-04-26T19:03:28.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:27 smithi090 ceph-mon[29180]: pgmap v1700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:29.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2093 sec 2024-04-26T19:03:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:29 smithi119 ceph-mon[32752]: pgmap v1701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:30.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2094 sec 2024-04-26T19:03:30.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:29 smithi167 ceph-mon[32751]: pgmap v1701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:29 smithi090 ceph-mon[29180]: pgmap v1701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1692.000646 secs 2024-04-26T19:03:31.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:30 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1692.671201 secs 2024-04-26T19:03:31.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1692.000646 secs 2024-04-26T19:03:31.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:30 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1692.671201 secs 2024-04-26T19:03:31.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2095 sec 2024-04-26T19:03:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1692.000646 secs 2024-04-26T19:03:31.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:30 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1692.671201 secs 2024-04-26T19:03:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:31 smithi167 ceph-mon[32751]: pgmap v1702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:31 smithi119 ceph-mon[32752]: pgmap v1702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:32.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2096 sec 2024-04-26T19:03:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:31 smithi090 ceph-mon[29180]: pgmap v1702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:33.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2097 sec 2024-04-26T19:03:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:33 smithi119 ceph-mon[32752]: pgmap v1703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:34.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2098 sec 2024-04-26T19:03:34.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:33 smithi167 ceph-mon[32751]: pgmap v1703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:33 smithi090 ceph-mon[29180]: pgmap v1703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:35.067 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2099 sec 2024-04-26T19:03:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1697.000733 secs 2024-04-26T19:03:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1697.000733 secs 2024-04-26T19:03:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1697.000733 secs 2024-04-26T19:03:36.068 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2100 sec 2024-04-26T19:03:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:36 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 1697.671301 secs 2024-04-26T19:03:36.626 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:36 smithi119 ceph-mon[32752]: slow request 724.363066 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:03:36.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:36 smithi119 ceph-mon[32752]: pgmap v1704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:36.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:36 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 1697.671301 secs 2024-04-26T19:03:36.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:36 smithi090 ceph-mon[29180]: slow request 724.363066 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:03:36.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:36 smithi090 ceph-mon[29180]: pgmap v1704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:36.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:36 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 1697.671301 secs 2024-04-26T19:03:36.684 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:36 smithi167 ceph-mon[32751]: slow request 724.363066 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:03:36.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:36 smithi167 ceph-mon[32751]: pgmap v1704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:37.068 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2101 sec 2024-04-26T19:03:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:37 smithi119 ceph-mon[32752]: pgmap v1705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:37 smithi167 ceph-mon[32751]: pgmap v1705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:38.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2102 sec 2024-04-26T19:03:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:37 smithi090 ceph-mon[29180]: pgmap v1705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:39.068 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2103 sec 2024-04-26T19:03:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:39 smithi119 ceph-mon[32752]: pgmap v1706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:40.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2104 sec 2024-04-26T19:03:40.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:39 smithi167 ceph-mon[32751]: pgmap v1706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:39 smithi090 ceph-mon[29180]: pgmap v1706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:40.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1702.000827 secs 2024-04-26T19:03:40.962 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:40 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1702.671425 secs 2024-04-26T19:03:41.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1702.000827 secs 2024-04-26T19:03:41.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:40 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1702.671425 secs 2024-04-26T19:03:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1702.000827 secs 2024-04-26T19:03:41.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:40 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1702.671425 secs 2024-04-26T19:03:41.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2105 sec 2024-04-26T19:03:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:41 smithi119 ceph-mon[32752]: pgmap v1707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:41 smithi167 ceph-mon[32751]: pgmap v1707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:42.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2106 sec 2024-04-26T19:03:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:41 smithi090 ceph-mon[29180]: pgmap v1707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:43.068 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2107 sec 2024-04-26T19:03:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:43 smithi119 ceph-mon[32752]: pgmap v1708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:43 smithi167 ceph-mon[32751]: pgmap v1708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:44.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2108 sec 2024-04-26T19:03:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:43 smithi090 ceph-mon[29180]: pgmap v1708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:45.068 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2109 sec 2024-04-26T19:03:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1707.000887 secs 2024-04-26T19:03:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1707.000887 secs 2024-04-26T19:03:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1707.000887 secs 2024-04-26T19:03:46.068 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2110 sec 2024-04-26T19:03:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1707.671559 secs 2024-04-26T19:03:46.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:46 smithi119 ceph-mon[32752]: pgmap v1709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:46.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1707.671559 secs 2024-04-26T19:03:46.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:46 smithi167 ceph-mon[32751]: pgmap v1709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1707.671559 secs 2024-04-26T19:03:46.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:46 smithi090 ceph-mon[29180]: pgmap v1709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:47.069 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2111 sec 2024-04-26T19:03:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:47 smithi119 ceph-mon[32752]: pgmap v1710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:47 smithi167 ceph-mon[32751]: pgmap v1710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:48.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2112 sec 2024-04-26T19:03:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:47 smithi090 ceph-mon[29180]: pgmap v1710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:49.069 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2113 sec 2024-04-26T19:03:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:49 smithi119 ceph-mon[32752]: pgmap v1711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:49 smithi167 ceph-mon[32751]: pgmap v1711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:50.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2114 sec 2024-04-26T19:03:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:49 smithi090 ceph-mon[29180]: pgmap v1711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1712.000954 secs 2024-04-26T19:03:51.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:50 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1712.671681 secs 2024-04-26T19:03:51.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2115 sec 2024-04-26T19:03:51.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1712.000954 secs 2024-04-26T19:03:51.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:50 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1712.671681 secs 2024-04-26T19:03:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1712.000954 secs 2024-04-26T19:03:51.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:50 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1712.671681 secs 2024-04-26T19:03:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:51 smithi119 ceph-mon[32752]: pgmap v1712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:51 smithi167 ceph-mon[32751]: pgmap v1712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:52.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2116 sec 2024-04-26T19:03:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:51 smithi090 ceph-mon[29180]: pgmap v1712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:53.069 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2117 sec 2024-04-26T19:03:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:53 smithi119 ceph-mon[32752]: pgmap v1713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:53 smithi167 ceph-mon[32751]: pgmap v1713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:54.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2118 sec 2024-04-26T19:03:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:53 smithi090 ceph-mon[29180]: pgmap v1713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:03:55.069 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2119 sec 2024-04-26T19:03:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1717.001040 secs 2024-04-26T19:03:55.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1717.001040 secs 2024-04-26T19:03:55.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1717.001040 secs 2024-04-26T19:03:56.069 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2120 sec 2024-04-26T19:03:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1717.671794 secs 2024-04-26T19:03:56.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:56 smithi119 ceph-mon[32752]: pgmap v1714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:56.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1717.671794 secs 2024-04-26T19:03:56.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:56 smithi167 ceph-mon[32751]: pgmap v1714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1717.671794 secs 2024-04-26T19:03:56.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:56 smithi090 ceph-mon[29180]: pgmap v1714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:57.070 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2121 sec 2024-04-26T19:03:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:57 smithi119 ceph-mon[32752]: pgmap v1715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:57 smithi167 ceph-mon[32751]: pgmap v1715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:58.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2122 sec 2024-04-26T19:03:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:57 smithi090 ceph-mon[29180]: pgmap v1715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:03:59.070 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2123 sec 2024-04-26T19:04:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:03:59 smithi119 ceph-mon[32752]: pgmap v1716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:03:59 smithi167 ceph-mon[32751]: pgmap v1716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:00.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2124 sec 2024-04-26T19:04:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:03:59 smithi090 ceph-mon[29180]: pgmap v1716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1722.001126 secs 2024-04-26T19:04:01.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:00 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1722.671944 secs 2024-04-26T19:04:01.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2125 sec 2024-04-26T19:04:01.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1722.001126 secs 2024-04-26T19:04:01.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:00 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1722.671944 secs 2024-04-26T19:04:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1722.001126 secs 2024-04-26T19:04:01.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:00 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1722.671944 secs 2024-04-26T19:04:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:01 smithi167 ceph-mon[32751]: pgmap v1717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:01 smithi119 ceph-mon[32752]: pgmap v1717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:02.070 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2126 sec 2024-04-26T19:04:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:01 smithi090 ceph-mon[29180]: pgmap v1717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:03.070 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2127 sec 2024-04-26T19:04:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:03 smithi119 ceph-mon[32752]: pgmap v1718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:03 smithi167 ceph-mon[32751]: pgmap v1718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:04.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2128 sec 2024-04-26T19:04:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:03 smithi090 ceph-mon[29180]: pgmap v1718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:05.070 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2129 sec 2024-04-26T19:04:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1727.001186 secs 2024-04-26T19:04:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1727.001186 secs 2024-04-26T19:04:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1727.001186 secs 2024-04-26T19:04:06.070 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2130 sec 2024-04-26T19:04:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1727.672063 secs 2024-04-26T19:04:06.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:06 smithi119 ceph-mon[32752]: pgmap v1719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1727.672063 secs 2024-04-26T19:04:06.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:06 smithi167 ceph-mon[32751]: pgmap v1719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1727.672063 secs 2024-04-26T19:04:06.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:06 smithi090 ceph-mon[29180]: pgmap v1719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:07.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2131 sec 2024-04-26T19:04:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:07 smithi119 ceph-mon[32752]: pgmap v1720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:07 smithi167 ceph-mon[32751]: pgmap v1720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:08.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2132 sec 2024-04-26T19:04:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:07 smithi090 ceph-mon[29180]: pgmap v1720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:04:09.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:09.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:09.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2133 sec 2024-04-26T19:04:09.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:04:09.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:09.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:09.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:04:09.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:09.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:04:10.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:04:10.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:09 smithi119 ceph-mon[32752]: pgmap v1721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:04:10.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:09 smithi119 ceph-mon[32752]: pgmap v1722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:10.123 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:10.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2134 sec 2024-04-26T19:04:10.139 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:04:10.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:04:10.154 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:09 smithi167 ceph-mon[32751]: pgmap v1721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:04:10.161 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:09 smithi167 ceph-mon[32751]: pgmap v1722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:10.169 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:10.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:04:10.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:04:10.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:09 smithi090 ceph-mon[29180]: pgmap v1721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:04:10.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:09 smithi090 ceph-mon[29180]: pgmap v1722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:10.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:04:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1732.001271 secs 2024-04-26T19:04:10.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:10 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1732.672179 secs 2024-04-26T19:04:10.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1732.001271 secs 2024-04-26T19:04:10.962 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:10 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1732.672179 secs 2024-04-26T19:04:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1732.001271 secs 2024-04-26T19:04:11.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:10 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1732.672179 secs 2024-04-26T19:04:11.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2135 sec 2024-04-26T19:04:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:11 smithi119 ceph-mon[32752]: pgmap v1723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:11 smithi167 ceph-mon[32751]: pgmap v1723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:12.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2136 sec 2024-04-26T19:04:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:11 smithi090 ceph-mon[29180]: pgmap v1723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:13.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2137 sec 2024-04-26T19:04:14.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2138 sec 2024-04-26T19:04:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:13 smithi119 ceph-mon[32752]: pgmap v1724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T19:04:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:13 smithi167 ceph-mon[32751]: pgmap v1724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T19:04:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:13 smithi090 ceph-mon[29180]: pgmap v1724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T19:04:15.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2139 sec 2024-04-26T19:04:16.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2140 sec 2024-04-26T19:04:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1737.001357 secs 2024-04-26T19:04:16.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:15 smithi119 ceph-mon[32752]: pgmap v1725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:16.319 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:15 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1737.672294 secs 2024-04-26T19:04:16.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1737.001357 secs 2024-04-26T19:04:16.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:15 smithi167 ceph-mon[32751]: pgmap v1725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:16.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:15 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1737.672294 secs 2024-04-26T19:04:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1737.001357 secs 2024-04-26T19:04:16.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:15 smithi090 ceph-mon[29180]: pgmap v1725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:16.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:15 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1737.672294 secs 2024-04-26T19:04:17.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2141 sec 2024-04-26T19:04:18.071 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2142 sec 2024-04-26T19:04:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:17 smithi119 ceph-mon[32752]: pgmap v1726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:17 smithi167 ceph-mon[32751]: pgmap v1726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:17 smithi090 ceph-mon[29180]: pgmap v1726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:04:19.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2143 sec 2024-04-26T19:04:20.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2144 sec 2024-04-26T19:04:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:19 smithi119 ceph-mon[32752]: pgmap v1727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:04:20.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:19 smithi167 ceph-mon[32751]: pgmap v1727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:04:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:19 smithi090 ceph-mon[29180]: pgmap v1727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:04:21.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2145 sec 2024-04-26T19:04:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1742.001439 secs 2024-04-26T19:04:21.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:20 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1742.672439 secs 2024-04-26T19:04:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1742.001439 secs 2024-04-26T19:04:21.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:20 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1742.672439 secs 2024-04-26T19:04:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1742.001439 secs 2024-04-26T19:04:21.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:20 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1742.672439 secs 2024-04-26T19:04:22.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2146 sec 2024-04-26T19:04:22.234 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:21 smithi119 ceph-mon[32752]: pgmap v1728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:21 smithi167 ceph-mon[32751]: pgmap v1728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:22.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:21 smithi090 ceph-mon[29180]: pgmap v1728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:23.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2147 sec 2024-04-26T19:04:24.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2148 sec 2024-04-26T19:04:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:23 smithi119 ceph-mon[32752]: pgmap v1729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:23 smithi167 ceph-mon[32751]: pgmap v1729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:23 smithi090 ceph-mon[29180]: pgmap v1729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:25.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2149 sec 2024-04-26T19:04:26.072 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2150 sec 2024-04-26T19:04:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1747.001522 secs 2024-04-26T19:04:26.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:25 smithi090 ceph-mon[29180]: pgmap v1730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:26.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:25 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1747.672537 secs 2024-04-26T19:04:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1747.001522 secs 2024-04-26T19:04:26.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:25 smithi119 ceph-mon[32752]: pgmap v1730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:26.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:25 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1747.672537 secs 2024-04-26T19:04:26.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1747.001522 secs 2024-04-26T19:04:26.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:25 smithi167 ceph-mon[32751]: pgmap v1730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:26.345 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:25 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1747.672537 secs 2024-04-26T19:04:27.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2151 sec 2024-04-26T19:04:28.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2152 sec 2024-04-26T19:04:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:27 smithi119 ceph-mon[32752]: pgmap v1731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:27 smithi167 ceph-mon[32751]: pgmap v1731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:28.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:27 smithi090 ceph-mon[29180]: pgmap v1731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:29.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2153 sec 2024-04-26T19:04:30.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2154 sec 2024-04-26T19:04:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:29 smithi119 ceph-mon[32752]: pgmap v1732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:29 smithi167 ceph-mon[32751]: pgmap v1732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:29 smithi090 ceph-mon[29180]: pgmap v1732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:31.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2155 sec 2024-04-26T19:04:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1752.001604 secs 2024-04-26T19:04:31.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:30 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 1752.672604 secs 2024-04-26T19:04:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1752.001604 secs 2024-04-26T19:04:31.353 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:30 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 1752.672604 secs 2024-04-26T19:04:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1752.001604 secs 2024-04-26T19:04:31.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:30 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 1752.672604 secs 2024-04-26T19:04:32.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2156 sec 2024-04-26T19:04:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:31 smithi119 ceph-mon[32752]: pgmap v1733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:32.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:31 smithi167 ceph-mon[32751]: pgmap v1733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:31 smithi090 ceph-mon[29180]: pgmap v1733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:33.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2157 sec 2024-04-26T19:04:34.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2158 sec 2024-04-26T19:04:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:33 smithi119 ceph-mon[32752]: pgmap v1734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:33 smithi167 ceph-mon[32751]: pgmap v1734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:33 smithi090 ceph-mon[29180]: pgmap v1734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:35.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2159 sec 2024-04-26T19:04:36.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2160 sec 2024-04-26T19:04:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1757.001667 secs 2024-04-26T19:04:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:35 smithi119 ceph-mon[32752]: pgmap v1735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:36.320 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:35 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 1757.672736 secs 2024-04-26T19:04:36.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:35 smithi119 ceph-mon[32752]: slow request 184.198086 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:04:36.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1757.001667 secs 2024-04-26T19:04:36.345 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:35 smithi167 ceph-mon[32751]: pgmap v1735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:36.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:35 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 1757.672736 secs 2024-04-26T19:04:36.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:35 smithi167 ceph-mon[32751]: slow request 184.198086 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:04:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1757.001667 secs 2024-04-26T19:04:36.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:35 smithi090 ceph-mon[29180]: pgmap v1735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:36.390 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:35 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 1757.672736 secs 2024-04-26T19:04:36.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:35 smithi090 ceph-mon[29180]: slow request 184.198086 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:04:37.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2161 sec 2024-04-26T19:04:38.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2162 sec 2024-04-26T19:04:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:37 smithi119 ceph-mon[32752]: pgmap v1736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:38.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:37 smithi167 ceph-mon[32751]: pgmap v1736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:37 smithi090 ceph-mon[29180]: pgmap v1736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:39.073 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2163 sec 2024-04-26T19:04:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T19:04:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:04:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T19:04:39.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:04:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T19:04:39.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:04:40.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2164 sec 2024-04-26T19:04:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:40 smithi090 ceph-mon[29180]: pgmap v1737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:40 smithi119 ceph-mon[32752]: pgmap v1737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:40.628 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:40 smithi167 ceph-mon[32751]: pgmap v1737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:41.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2165 sec 2024-04-26T19:04:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:41 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1762.001749 secs 2024-04-26T19:04:41.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1762.672888 secs 2024-04-26T19:04:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:41 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1762.001749 secs 2024-04-26T19:04:41.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1762.672888 secs 2024-04-26T19:04:41.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:41 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1762.001749 secs 2024-04-26T19:04:41.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1762.672888 secs 2024-04-26T19:04:42.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2166 sec 2024-04-26T19:04:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:42 smithi090 ceph-mon[29180]: pgmap v1738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:42 smithi119 ceph-mon[32752]: pgmap v1738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:42 smithi167 ceph-mon[32751]: pgmap v1738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:43.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2167 sec 2024-04-26T19:04:44.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2168 sec 2024-04-26T19:04:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:44 smithi090 ceph-mon[29180]: pgmap v1739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:44 smithi119 ceph-mon[32752]: pgmap v1739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:44.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:44 smithi167 ceph-mon[32751]: pgmap v1739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:45.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2169 sec 2024-04-26T19:04:46.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2170 sec 2024-04-26T19:04:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:46 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1767.001837 secs 2024-04-26T19:04:46.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:46 smithi090 ceph-mon[29180]: pgmap v1740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:46.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1767.672973 secs 2024-04-26T19:04:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:46 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1767.001837 secs 2024-04-26T19:04:46.673 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:46 smithi119 ceph-mon[32752]: pgmap v1740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:46.683 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1767.672973 secs 2024-04-26T19:04:46.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:46 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1767.001837 secs 2024-04-26T19:04:46.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:46 smithi167 ceph-mon[32751]: pgmap v1740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:46.710 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1767.672973 secs 2024-04-26T19:04:47.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2171 sec 2024-04-26T19:04:48.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2172 sec 2024-04-26T19:04:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:48 smithi090 ceph-mon[29180]: pgmap v1741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:48 smithi119 ceph-mon[32752]: pgmap v1741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:48 smithi167 ceph-mon[32751]: pgmap v1741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:49.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2173 sec 2024-04-26T19:04:50.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2174 sec 2024-04-26T19:04:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:50 smithi090 ceph-mon[29180]: pgmap v1742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:50 smithi119 ceph-mon[32752]: pgmap v1742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:50.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:50 smithi167 ceph-mon[32751]: pgmap v1742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:51.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2175 sec 2024-04-26T19:04:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:51 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1772.001897 secs 2024-04-26T19:04:51.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1772.673083 secs 2024-04-26T19:04:51.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:51 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1772.001897 secs 2024-04-26T19:04:51.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1772.673083 secs 2024-04-26T19:04:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:51 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1772.001897 secs 2024-04-26T19:04:51.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1772.673083 secs 2024-04-26T19:04:52.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2176 sec 2024-04-26T19:04:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:52 smithi119 ceph-mon[32752]: pgmap v1743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:52 smithi167 ceph-mon[32751]: pgmap v1743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:52 smithi090 ceph-mon[29180]: pgmap v1743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:53.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2177 sec 2024-04-26T19:04:54.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2178 sec 2024-04-26T19:04:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:54 smithi119 ceph-mon[32752]: pgmap v1744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:54 smithi167 ceph-mon[32751]: pgmap v1744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:54 smithi090 ceph-mon[29180]: pgmap v1744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:55.074 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2179 sec 2024-04-26T19:04:56.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2180 sec 2024-04-26T19:04:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:56 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1777.001988 secs 2024-04-26T19:04:56.573 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:56 smithi119 ceph-mon[32752]: pgmap v1745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:56.581 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1777.673194 secs 2024-04-26T19:04:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:56 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1777.001988 secs 2024-04-26T19:04:56.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:56 smithi167 ceph-mon[32751]: pgmap v1745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:56.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1777.673194 secs 2024-04-26T19:04:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:56 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1777.001988 secs 2024-04-26T19:04:56.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:56 smithi090 ceph-mon[29180]: pgmap v1745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:04:56.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1777.673194 secs 2024-04-26T19:04:57.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2181 sec 2024-04-26T19:04:58.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2182 sec 2024-04-26T19:04:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:04:58 smithi119 ceph-mon[32752]: pgmap v1746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:04:58 smithi167 ceph-mon[32751]: pgmap v1746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:04:58 smithi090 ceph-mon[29180]: pgmap v1746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:04:59.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2183 sec 2024-04-26T19:05:00.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2184 sec 2024-04-26T19:05:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:00 smithi119 ceph-mon[32752]: pgmap v1747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:00 smithi167 ceph-mon[32751]: pgmap v1747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:00 smithi090 ceph-mon[29180]: pgmap v1747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:01.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2185 sec 2024-04-26T19:05:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:01 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1782.002069 secs 2024-04-26T19:05:01.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1782.673328 secs 2024-04-26T19:05:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:01 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1782.002069 secs 2024-04-26T19:05:01.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1782.673328 secs 2024-04-26T19:05:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:01 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1782.002069 secs 2024-04-26T19:05:01.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1782.673328 secs 2024-04-26T19:05:02.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2186 sec 2024-04-26T19:05:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:02 smithi119 ceph-mon[32752]: pgmap v1748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:02.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:02 smithi167 ceph-mon[32751]: pgmap v1748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:02 smithi090 ceph-mon[29180]: pgmap v1748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:03.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2187 sec 2024-04-26T19:05:04.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2188 sec 2024-04-26T19:05:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:04 smithi119 ceph-mon[32752]: pgmap v1749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:04 smithi167 ceph-mon[32751]: pgmap v1749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:04 smithi090 ceph-mon[29180]: pgmap v1749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:05.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2189 sec 2024-04-26T19:05:06.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2190 sec 2024-04-26T19:05:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:06 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1787.002131 secs 2024-04-26T19:05:06.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:06 smithi119 ceph-mon[32752]: pgmap v1750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:06.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1787.673451 secs 2024-04-26T19:05:06.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:06 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1787.002131 secs 2024-04-26T19:05:06.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:06 smithi167 ceph-mon[32751]: pgmap v1750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:06.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1787.673451 secs 2024-04-26T19:05:06.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:06 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1787.002131 secs 2024-04-26T19:05:06.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:06 smithi090 ceph-mon[29180]: pgmap v1750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:06.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1787.673451 secs 2024-04-26T19:05:07.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2191 sec 2024-04-26T19:05:08.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2192 sec 2024-04-26T19:05:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:08 smithi119 ceph-mon[32752]: pgmap v1751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:08 smithi167 ceph-mon[32751]: pgmap v1751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:08 smithi090 ceph-mon[29180]: pgmap v1751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:09.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2193 sec 2024-04-26T19:05:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:05:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:05:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:05:10.075 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2194 sec 2024-04-26T19:05:10.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:10 smithi119 ceph-mon[32752]: pgmap v1752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:10.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:05:10.473 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:05:10.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:05:10.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1792.002195 secs 2024-04-26T19:05:10.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:10 smithi167 ceph-mon[32751]: pgmap v1752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:10.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:05:10.598 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:05:10.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:05:10.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1792.002195 secs 2024-04-26T19:05:10.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:10 smithi090 ceph-mon[29180]: pgmap v1752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:10.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:05:10.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:05:10.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:05:10.833 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1792.002195 secs 2024-04-26T19:05:11.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2195 sec 2024-04-26T19:05:11.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:11 smithi090 ceph-mon[29180]: pgmap v1753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:05:11.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:11 smithi090 ceph-mon[29180]: pgmap v1754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:11.542 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1792.673519 secs 2024-04-26T19:05:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:11 smithi119 ceph-mon[32752]: pgmap v1753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:05:11.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:11 smithi119 ceph-mon[32752]: pgmap v1754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:11.610 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1792.673519 secs 2024-04-26T19:05:11.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:11 smithi167 ceph-mon[32751]: pgmap v1753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:05:11.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:11 smithi167 ceph-mon[32751]: pgmap v1754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:11.634 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1792.673519 secs 2024-04-26T19:05:12.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2196 sec 2024-04-26T19:05:13.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2197 sec 2024-04-26T19:05:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:13 smithi119 ceph-mon[32752]: pgmap v1755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:13 smithi167 ceph-mon[32751]: pgmap v1755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:13 smithi090 ceph-mon[29180]: pgmap v1755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:14.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2198 sec 2024-04-26T19:05:15.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2199 sec 2024-04-26T19:05:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:15 smithi119 ceph-mon[32752]: pgmap v1756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:15.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1797.002279 secs 2024-04-26T19:05:15.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:15 smithi167 ceph-mon[32751]: pgmap v1756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:15.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1797.002279 secs 2024-04-26T19:05:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:15 smithi090 ceph-mon[29180]: pgmap v1756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:15.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1797.002279 secs 2024-04-26T19:05:16.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2200 sec 2024-04-26T19:05:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1797.673615 secs 2024-04-26T19:05:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1797.673615 secs 2024-04-26T19:05:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1797.673615 secs 2024-04-26T19:05:17.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2201 sec 2024-04-26T19:05:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:17 smithi119 ceph-mon[32752]: pgmap v1757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:17 smithi167 ceph-mon[32751]: pgmap v1757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:17 smithi090 ceph-mon[29180]: pgmap v1757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:05:18.076 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2202 sec 2024-04-26T19:05:19.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2203 sec 2024-04-26T19:05:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:19 smithi119 ceph-mon[32752]: pgmap v1758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:05:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:19 smithi167 ceph-mon[32751]: pgmap v1758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:05:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:19 smithi090 ceph-mon[29180]: pgmap v1758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:05:20.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2204 sec 2024-04-26T19:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1802.002363 secs 2024-04-26T19:05:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1802.002363 secs 2024-04-26T19:05:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1802.002363 secs 2024-04-26T19:05:21.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2205 sec 2024-04-26T19:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:21 smithi119 ceph-mon[32752]: pgmap v1759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:05:21.575 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1802.673754 secs 2024-04-26T19:05:21.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:21 smithi167 ceph-mon[32751]: pgmap v1759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:05:21.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1802.673754 secs 2024-04-26T19:05:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:21 smithi090 ceph-mon[29180]: pgmap v1759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:05:21.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1802.673754 secs 2024-04-26T19:05:22.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2206 sec 2024-04-26T19:05:23.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2207 sec 2024-04-26T19:05:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:23 smithi119 ceph-mon[32752]: pgmap v1760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:23 smithi167 ceph-mon[32751]: pgmap v1760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:23 smithi090 ceph-mon[29180]: pgmap v1760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:24.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2208 sec 2024-04-26T19:05:25.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2209 sec 2024-04-26T19:05:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:25 smithi119 ceph-mon[32752]: pgmap v1761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:25.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1807.002447 secs 2024-04-26T19:05:25.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:25 smithi167 ceph-mon[32751]: pgmap v1761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:25.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1807.002447 secs 2024-04-26T19:05:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:25 smithi090 ceph-mon[29180]: pgmap v1761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:25.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1807.002447 secs 2024-04-26T19:05:26.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2210 sec 2024-04-26T19:05:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1807.673875 secs 2024-04-26T19:05:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1807.673875 secs 2024-04-26T19:05:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1807.673875 secs 2024-04-26T19:05:27.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2211 sec 2024-04-26T19:05:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:27 smithi119 ceph-mon[32752]: pgmap v1762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:27.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:27 smithi167 ceph-mon[32751]: pgmap v1762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:27 smithi090 ceph-mon[29180]: pgmap v1762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:28.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2212 sec 2024-04-26T19:05:28.467 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 msgr2=0x7f2cc00b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T19:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 msgr2=0x7f2cc00b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T19:05:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 0x7f2cc00b31e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400fe40 tx=0x7f2cc4038040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:05:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 0x7f2cc00b31e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400fe40 tx=0x7f2cc4038040 comp rx=0 tx=0).stop 2024-04-26T19:05:28.528 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 msgr2=0x7f2cc00b31e0 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T19:05:28.538 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 msgr2=0x7f2cc00b31e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T19:05:28.548 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b0df0 0x7f2cc00b31e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:05:28.558 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:05:28.567 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.467+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:05:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b3720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5630289b92e0 tx=0x7f2cc400a2d0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:05:28.667 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.490+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000121) for more than 900000000 us, fault. 2024-04-26T19:05:28.679 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T19:05:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:05:28.804 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T19:05:28.878 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.490+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T19:05:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.703+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:05:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.703+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 18 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:05:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.714+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:05:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:05:28.714+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 18 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:05:29.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2213 sec 2024-04-26T19:05:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:29 smithi119 ceph-mon[32752]: pgmap v1763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:29 smithi167 ceph-mon[32751]: pgmap v1763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:29 smithi090 ceph-mon[29180]: pgmap v1763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:30.077 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2214 sec 2024-04-26T19:05:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1812.002529 secs 2024-04-26T19:05:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1812.002529 secs 2024-04-26T19:05:30.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1812.002529 secs 2024-04-26T19:05:31.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2215 sec 2024-04-26T19:05:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:31 smithi167 ceph-mon[32751]: pgmap v1764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:31.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1812.674013 secs 2024-04-26T19:05:31.609 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:31 smithi119 ceph-mon[32752]: pgmap v1764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:31.750 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1812.674013 secs 2024-04-26T19:05:31.761 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:31 smithi090 ceph-mon[29180]: pgmap v1764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:31.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1812.674013 secs 2024-04-26T19:05:32.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2216 sec 2024-04-26T19:05:33.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2217 sec 2024-04-26T19:05:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:33 smithi119 ceph-mon[32752]: pgmap v1765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:33 smithi167 ceph-mon[32751]: pgmap v1765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:33 smithi090 ceph-mon[29180]: pgmap v1765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:34.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2218 sec 2024-04-26T19:05:35.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2219 sec 2024-04-26T19:05:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:35 smithi119 ceph-mon[32752]: pgmap v1766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:35.637 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1817.002615 secs 2024-04-26T19:05:35.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:35 smithi090 ceph-mon[29180]: pgmap v1766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:35.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1817.002615 secs 2024-04-26T19:05:35.664 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:35 smithi167 ceph-mon[32751]: pgmap v1766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:35.672 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1817.002615 secs 2024-04-26T19:05:36.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2220 sec 2024-04-26T19:05:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 1817.674143 secs 2024-04-26T19:05:36.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:36 smithi119 ceph-mon[32752]: slow request 1444.543156 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:05:36.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 1817.674143 secs 2024-04-26T19:05:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:36 smithi167 ceph-mon[32751]: slow request 1444.543156 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:05:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 1817.674143 secs 2024-04-26T19:05:36.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:36 smithi090 ceph-mon[29180]: slow request 1444.543156 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:05:37.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2221 sec 2024-04-26T19:05:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:37 smithi119 ceph-mon[32752]: pgmap v1767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:37 smithi167 ceph-mon[32751]: pgmap v1767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:37 smithi090 ceph-mon[29180]: pgmap v1767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:38.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2222 sec 2024-04-26T19:05:39.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2223 sec 2024-04-26T19:05:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:39 smithi119 ceph-mon[32752]: pgmap v1768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:39 smithi167 ceph-mon[32751]: pgmap v1768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:39 smithi090 ceph-mon[29180]: pgmap v1768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:40.078 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2224 sec 2024-04-26T19:05:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1822.002698 secs 2024-04-26T19:05:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1822.002698 secs 2024-04-26T19:05:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1822.002698 secs 2024-04-26T19:05:41.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2225 sec 2024-04-26T19:05:41.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:41 smithi090 ceph-mon[29180]: pgmap v1769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:41.539 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1822.674268 secs 2024-04-26T19:05:41.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:41 smithi119 ceph-mon[32752]: pgmap v1769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:41.578 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1822.674268 secs 2024-04-26T19:05:41.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:41 smithi167 ceph-mon[32751]: pgmap v1769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:41.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1822.674268 secs 2024-04-26T19:05:42.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2226 sec 2024-04-26T19:05:43.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2227 sec 2024-04-26T19:05:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:43 smithi119 ceph-mon[32752]: pgmap v1770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:43 smithi167 ceph-mon[32751]: pgmap v1770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:43 smithi090 ceph-mon[29180]: pgmap v1770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:44.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2228 sec 2024-04-26T19:05:45.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2229 sec 2024-04-26T19:05:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:45 smithi119 ceph-mon[32752]: pgmap v1771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:45.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1827.002790 secs 2024-04-26T19:05:45.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:45 smithi167 ceph-mon[32751]: pgmap v1771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:45.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1827.002790 secs 2024-04-26T19:05:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:45 smithi090 ceph-mon[29180]: pgmap v1771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:45.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1827.002790 secs 2024-04-26T19:05:46.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2230 sec 2024-04-26T19:05:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1827.674399 secs 2024-04-26T19:05:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1827.674399 secs 2024-04-26T19:05:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1827.674399 secs 2024-04-26T19:05:47.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2231 sec 2024-04-26T19:05:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:47 smithi119 ceph-mon[32752]: pgmap v1772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:47.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:47 smithi167 ceph-mon[32751]: pgmap v1772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:47 smithi090 ceph-mon[29180]: pgmap v1772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:48.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2232 sec 2024-04-26T19:05:49.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2233 sec 2024-04-26T19:05:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:49 smithi119 ceph-mon[32752]: pgmap v1773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:49 smithi167 ceph-mon[32751]: pgmap v1773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:49 smithi090 ceph-mon[29180]: pgmap v1773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:50.079 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2234 sec 2024-04-26T19:05:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1832.002849 secs 2024-04-26T19:05:50.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1832.002849 secs 2024-04-26T19:05:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1832.002849 secs 2024-04-26T19:05:51.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2235 sec 2024-04-26T19:05:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:51 smithi119 ceph-mon[32752]: pgmap v1774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:51.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1832.674519 secs 2024-04-26T19:05:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:51 smithi167 ceph-mon[32751]: pgmap v1774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1832.674519 secs 2024-04-26T19:05:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:51 smithi090 ceph-mon[29180]: pgmap v1774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:51.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1832.674519 secs 2024-04-26T19:05:52.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2236 sec 2024-04-26T19:05:53.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2237 sec 2024-04-26T19:05:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:53 smithi119 ceph-mon[32752]: pgmap v1775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:53 smithi167 ceph-mon[32751]: pgmap v1775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:53 smithi090 ceph-mon[29180]: pgmap v1775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:54.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2238 sec 2024-04-26T19:05:55.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2239 sec 2024-04-26T19:05:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:55 smithi119 ceph-mon[32752]: pgmap v1776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:55.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1837.002936 secs 2024-04-26T19:05:55.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:55 smithi167 ceph-mon[32751]: pgmap v1776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:55.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1837.002936 secs 2024-04-26T19:05:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:55 smithi090 ceph-mon[29180]: pgmap v1776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:05:55.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1837.002936 secs 2024-04-26T19:05:56.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2240 sec 2024-04-26T19:05:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1837.674670 secs 2024-04-26T19:05:56.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1837.674670 secs 2024-04-26T19:05:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1837.674670 secs 2024-04-26T19:05:57.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2241 sec 2024-04-26T19:05:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:57 smithi119 ceph-mon[32752]: pgmap v1777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:57 smithi167 ceph-mon[32751]: pgmap v1777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:57 smithi090 ceph-mon[29180]: pgmap v1777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:58.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2242 sec 2024-04-26T19:05:59.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2243 sec 2024-04-26T19:05:59.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:05:59 smithi090 ceph-mon[29180]: pgmap v1778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:05:59 smithi119 ceph-mon[32752]: pgmap v1778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:05:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:05:59 smithi167 ceph-mon[32751]: pgmap v1778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:00.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2244 sec 2024-04-26T19:06:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1842.002997 secs 2024-04-26T19:06:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1842.002997 secs 2024-04-26T19:06:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1842.002997 secs 2024-04-26T19:06:01.080 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2245 sec 2024-04-26T19:06:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:01 smithi119 ceph-mon[32752]: pgmap v1779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:01.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1842.674789 secs 2024-04-26T19:06:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:01 smithi167 ceph-mon[32751]: pgmap v1779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:01.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1842.674789 secs 2024-04-26T19:06:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:01 smithi090 ceph-mon[29180]: pgmap v1779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:01.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1842.674789 secs 2024-04-26T19:06:02.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2246 sec 2024-04-26T19:06:03.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2247 sec 2024-04-26T19:06:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:03 smithi119 ceph-mon[32752]: pgmap v1780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:03 smithi167 ceph-mon[32751]: pgmap v1780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:03 smithi090 ceph-mon[29180]: pgmap v1780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:04.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2248 sec 2024-04-26T19:06:05.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2249 sec 2024-04-26T19:06:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:05 smithi119 ceph-mon[32752]: pgmap v1781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:05.575 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1847.003059 secs 2024-04-26T19:06:05.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:05 smithi167 ceph-mon[32751]: pgmap v1781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:05.598 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1847.003059 secs 2024-04-26T19:06:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:05 smithi090 ceph-mon[29180]: pgmap v1781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:05.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1847.003059 secs 2024-04-26T19:06:06.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2250 sec 2024-04-26T19:06:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1847.674929 secs 2024-04-26T19:06:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1847.674929 secs 2024-04-26T19:06:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1847.674929 secs 2024-04-26T19:06:07.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2251 sec 2024-04-26T19:06:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:07 smithi119 ceph-mon[32752]: pgmap v1782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:07 smithi167 ceph-mon[32751]: pgmap v1782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:07 smithi090 ceph-mon[29180]: pgmap v1782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:08.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2252 sec 2024-04-26T19:06:09.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2253 sec 2024-04-26T19:06:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:09 smithi119 ceph-mon[32752]: pgmap v1783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:09 smithi167 ceph-mon[32751]: pgmap v1783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:09 smithi090 ceph-mon[29180]: pgmap v1783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:10.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2254 sec 2024-04-26T19:06:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:06:10.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1852.003121 secs 2024-04-26T19:06:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:06:10.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1852.003121 secs 2024-04-26T19:06:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:06:10.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1852.003121 secs 2024-04-26T19:06:11.081 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2255 sec 2024-04-26T19:06:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:11 smithi167 ceph-mon[32751]: pgmap v1784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:11.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:06:11.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:06:11.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:11 smithi167 ceph-mon[32751]: pgmap v1785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:06:11.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:11 smithi167 ceph-mon[32751]: pgmap v1786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:11.638 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:06:11.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1852.674992 secs 2024-04-26T19:06:11.699 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:11 smithi090 ceph-mon[29180]: pgmap v1784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:11.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:06:11.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:06:11.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:11 smithi090 ceph-mon[29180]: pgmap v1785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:06:11.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:11 smithi090 ceph-mon[29180]: pgmap v1786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:11.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:06:11.796 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1852.674992 secs 2024-04-26T19:06:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:11 smithi119 ceph-mon[32752]: pgmap v1784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:11.867 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:06:11.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:06:11.974 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:11 smithi119 ceph-mon[32752]: pgmap v1785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:06:12.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:11 smithi119 ceph-mon[32752]: pgmap v1786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:12.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:06:12.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1852.674992 secs 2024-04-26T19:06:12.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2256 sec 2024-04-26T19:06:13.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2257 sec 2024-04-26T19:06:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:13 smithi119 ceph-mon[32752]: pgmap v1787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:06:13.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:13 smithi167 ceph-mon[32751]: pgmap v1787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:06:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:13 smithi090 ceph-mon[29180]: pgmap v1787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:06:14.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2258 sec 2024-04-26T19:06:15.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2259 sec 2024-04-26T19:06:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:15 smithi119 ceph-mon[32752]: pgmap v1788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:15.825 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1857.003206 secs 2024-04-26T19:06:15.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:15 smithi090 ceph-mon[29180]: pgmap v1788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:15.931 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1857.003206 secs 2024-04-26T19:06:15.942 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:15 smithi167 ceph-mon[32751]: pgmap v1788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:15.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1857.003206 secs 2024-04-26T19:06:16.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2260 sec 2024-04-26T19:06:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1857.675125 secs 2024-04-26T19:06:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1857.675125 secs 2024-04-26T19:06:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1857.675125 secs 2024-04-26T19:06:17.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2261 sec 2024-04-26T19:06:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:17 smithi119 ceph-mon[32752]: pgmap v1789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:17 smithi167 ceph-mon[32751]: pgmap v1789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:17 smithi090 ceph-mon[29180]: pgmap v1789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:18.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2262 sec 2024-04-26T19:06:19.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2263 sec 2024-04-26T19:06:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:19 smithi119 ceph-mon[32752]: pgmap v1790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:19 smithi167 ceph-mon[32751]: pgmap v1790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:19 smithi090 ceph-mon[29180]: pgmap v1790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:06:20.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2264 sec 2024-04-26T19:06:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1862.003267 secs 2024-04-26T19:06:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1862.003267 secs 2024-04-26T19:06:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1862.003267 secs 2024-04-26T19:06:21.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2265 sec 2024-04-26T19:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:21 smithi119 ceph-mon[32752]: pgmap v1791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:06:21.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1862.675178 secs 2024-04-26T19:06:21.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:21 smithi167 ceph-mon[32751]: pgmap v1791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:06:21.841 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1862.675178 secs 2024-04-26T19:06:21.901 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:21 smithi090 ceph-mon[29180]: pgmap v1791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:06:21.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1862.675178 secs 2024-04-26T19:06:22.082 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2266 sec 2024-04-26T19:06:23.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2267 sec 2024-04-26T19:06:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:23 smithi119 ceph-mon[32752]: pgmap v1792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:23 smithi167 ceph-mon[32751]: pgmap v1792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:23 smithi090 ceph-mon[29180]: pgmap v1792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:24.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2268 sec 2024-04-26T19:06:25.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2269 sec 2024-04-26T19:06:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:25 smithi119 ceph-mon[32752]: pgmap v1793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:25.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1867.003348 secs 2024-04-26T19:06:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:25 smithi167 ceph-mon[32751]: pgmap v1793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:25.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1867.003348 secs 2024-04-26T19:06:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:25 smithi090 ceph-mon[29180]: pgmap v1793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:25.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1867.003348 secs 2024-04-26T19:06:26.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2270 sec 2024-04-26T19:06:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1867.675291 secs 2024-04-26T19:06:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1867.675291 secs 2024-04-26T19:06:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1867.675291 secs 2024-04-26T19:06:27.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2271 sec 2024-04-26T19:06:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:27 smithi119 ceph-mon[32752]: pgmap v1794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:27.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:27 smithi167 ceph-mon[32751]: pgmap v1794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:27 smithi090 ceph-mon[29180]: pgmap v1794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:28.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2272 sec 2024-04-26T19:06:29.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2273 sec 2024-04-26T19:06:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:29 smithi119 ceph-mon[32752]: pgmap v1795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:29.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:29 smithi167 ceph-mon[32751]: pgmap v1795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:29 smithi090 ceph-mon[29180]: pgmap v1795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:30.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2274 sec 2024-04-26T19:06:30.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1872.003432 secs 2024-04-26T19:06:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1872.003432 secs 2024-04-26T19:06:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1872.003432 secs 2024-04-26T19:06:31.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2275 sec 2024-04-26T19:06:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:31 smithi090 ceph-mon[29180]: pgmap v1796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:31.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1872.675414 secs 2024-04-26T19:06:31.657 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:06:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:06:31.286+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:06:31.668 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:06:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:06:31.286+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:06:31.678 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:06:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:06:31.286+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:06:31.689 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:06:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:06:31.286+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:06:31.699 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:06:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:06:31.286+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:06:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:31 smithi119 ceph-mon[32752]: pgmap v1796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:31.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1872.675414 secs 2024-04-26T19:06:31.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:31 smithi167 ceph-mon[32751]: pgmap v1796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:31.955 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1872.675414 secs 2024-04-26T19:06:32.083 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2276 sec 2024-04-26T19:06:33.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2277 sec 2024-04-26T19:06:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:33 smithi119 ceph-mon[32752]: pgmap v1797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:33.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:33 smithi167 ceph-mon[32751]: pgmap v1797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:33 smithi090 ceph-mon[29180]: pgmap v1797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:34.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2278 sec 2024-04-26T19:06:35.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2279 sec 2024-04-26T19:06:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:35 smithi119 ceph-mon[32752]: pgmap v1798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:35.872 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1877.003508 secs 2024-04-26T19:06:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:35 smithi090 ceph-mon[29180]: pgmap v1798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:35.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1877.003508 secs 2024-04-26T19:06:35.904 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:35 smithi167 ceph-mon[32751]: pgmap v1798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:35.916 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1877.003508 secs 2024-04-26T19:06:36.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2280 sec 2024-04-26T19:06:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1877.675509 secs 2024-04-26T19:06:36.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1877.675509 secs 2024-04-26T19:06:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1877.675509 secs 2024-04-26T19:06:37.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2281 sec 2024-04-26T19:06:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:37 smithi119 ceph-mon[32752]: pgmap v1799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:37 smithi167 ceph-mon[32751]: pgmap v1799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:37 smithi090 ceph-mon[29180]: pgmap v1799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:38.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2282 sec 2024-04-26T19:06:39.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2283 sec 2024-04-26T19:06:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:39 smithi119 ceph-mon[32752]: pgmap v1800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:39 smithi167 ceph-mon[32751]: pgmap v1800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:39 smithi090 ceph-mon[29180]: pgmap v1800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:40.084 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2284 sec 2024-04-26T19:06:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1882.003563 secs 2024-04-26T19:06:40.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1882.003563 secs 2024-04-26T19:06:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1882.003563 secs 2024-04-26T19:06:41.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2285 sec 2024-04-26T19:06:41.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:41 smithi090 ceph-mon[29180]: pgmap v1801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T19:06:41.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1882.675591 secs 2024-04-26T19:06:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:41 smithi119 ceph-mon[32752]: pgmap v1801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T19:06:41.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1882.675591 secs 2024-04-26T19:06:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:41 smithi167 ceph-mon[32751]: pgmap v1801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T19:06:41.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1882.675591 secs 2024-04-26T19:06:42.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2286 sec 2024-04-26T19:06:43.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2287 sec 2024-04-26T19:06:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:43 smithi119 ceph-mon[32752]: pgmap v1802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T19:06:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:43 smithi167 ceph-mon[32751]: pgmap v1802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T19:06:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:43 smithi090 ceph-mon[29180]: pgmap v1802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T19:06:44.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2288 sec 2024-04-26T19:06:45.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2289 sec 2024-04-26T19:06:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:45 smithi119 ceph-mon[32752]: pgmap v1803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:45.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1887.003646 secs 2024-04-26T19:06:45.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:45 smithi167 ceph-mon[32751]: pgmap v1803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:45.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1887.003646 secs 2024-04-26T19:06:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:45 smithi090 ceph-mon[29180]: pgmap v1803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:06:45.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1887.003646 secs 2024-04-26T19:06:46.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2290 sec 2024-04-26T19:06:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1887.675705 secs 2024-04-26T19:06:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1887.675705 secs 2024-04-26T19:06:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1887.675705 secs 2024-04-26T19:06:47.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2291 sec 2024-04-26T19:06:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:47 smithi119 ceph-mon[32752]: pgmap v1804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:47 smithi167 ceph-mon[32751]: pgmap v1804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:47 smithi090 ceph-mon[29180]: pgmap v1804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:48.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2292 sec 2024-04-26T19:06:49.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2293 sec 2024-04-26T19:06:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:49 smithi119 ceph-mon[32752]: pgmap v1805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:49.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:49 smithi167 ceph-mon[32751]: pgmap v1805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:49 smithi090 ceph-mon[29180]: pgmap v1805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:50.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2294 sec 2024-04-26T19:06:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1892.003728 secs 2024-04-26T19:06:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1892.003728 secs 2024-04-26T19:06:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1892.003728 secs 2024-04-26T19:06:51.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2295 sec 2024-04-26T19:06:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1892.675816 secs 2024-04-26T19:06:51.826 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:51 smithi119 ceph-mon[32752]: pgmap v1806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:51.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1892.675816 secs 2024-04-26T19:06:51.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:51 smithi167 ceph-mon[32751]: pgmap v1806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1892.675816 secs 2024-04-26T19:06:51.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:51 smithi090 ceph-mon[29180]: pgmap v1806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:52.085 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2296 sec 2024-04-26T19:06:53.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2297 sec 2024-04-26T19:06:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:53 smithi119 ceph-mon[32752]: pgmap v1807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:53 smithi167 ceph-mon[32751]: pgmap v1807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:53 smithi090 ceph-mon[29180]: pgmap v1807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:54.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2298 sec 2024-04-26T19:06:55.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2299 sec 2024-04-26T19:06:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:55 smithi119 ceph-mon[32752]: pgmap v1808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:55.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1897.003814 secs 2024-04-26T19:06:55.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:55 smithi167 ceph-mon[32751]: pgmap v1808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:55.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1897.003814 secs 2024-04-26T19:06:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:55 smithi090 ceph-mon[29180]: pgmap v1808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:55.921 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1897.003814 secs 2024-04-26T19:06:56.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2300 sec 2024-04-26T19:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1897.675894 secs 2024-04-26T19:06:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1897.675894 secs 2024-04-26T19:06:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1897.675894 secs 2024-04-26T19:06:57.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2301 sec 2024-04-26T19:06:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:57 smithi119 ceph-mon[32752]: pgmap v1809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:57 smithi167 ceph-mon[32751]: pgmap v1809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:57 smithi090 ceph-mon[29180]: pgmap v1809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:06:58.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2302 sec 2024-04-26T19:06:59.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2303 sec 2024-04-26T19:06:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:06:59 smithi119 ceph-mon[32752]: pgmap v1810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:06:59 smithi167 ceph-mon[32751]: pgmap v1810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:06:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:06:59 smithi090 ceph-mon[29180]: pgmap v1810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:00.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2304 sec 2024-04-26T19:07:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1902.003898 secs 2024-04-26T19:07:00.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1902.003898 secs 2024-04-26T19:07:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1902.003898 secs 2024-04-26T19:07:01.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2305 sec 2024-04-26T19:07:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1902.676019 secs 2024-04-26T19:07:01.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:01 smithi119 ceph-mon[32752]: pgmap v1811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1902.676019 secs 2024-04-26T19:07:01.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:01 smithi167 ceph-mon[32751]: pgmap v1811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1902.676019 secs 2024-04-26T19:07:01.990 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:01 smithi090 ceph-mon[29180]: pgmap v1811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:02.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2306 sec 2024-04-26T19:07:03.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2307 sec 2024-04-26T19:07:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:03 smithi119 ceph-mon[32752]: pgmap v1812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:03 smithi167 ceph-mon[32751]: pgmap v1812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:03 smithi090 ceph-mon[29180]: pgmap v1812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:04.086 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2308 sec 2024-04-26T19:07:05.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2309 sec 2024-04-26T19:07:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:05 smithi119 ceph-mon[32752]: pgmap v1813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1907.003959 secs 2024-04-26T19:07:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:05 smithi167 ceph-mon[32751]: pgmap v1813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:05.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1907.003959 secs 2024-04-26T19:07:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:05 smithi090 ceph-mon[29180]: pgmap v1813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1907.003959 secs 2024-04-26T19:07:06.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2310 sec 2024-04-26T19:07:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1907.676150 secs 2024-04-26T19:07:06.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1907.676150 secs 2024-04-26T19:07:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1907.676150 secs 2024-04-26T19:07:07.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2311 sec 2024-04-26T19:07:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:07 smithi119 ceph-mon[32752]: pgmap v1814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:07 smithi167 ceph-mon[32751]: pgmap v1814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:07 smithi090 ceph-mon[29180]: pgmap v1814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:08.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2312 sec 2024-04-26T19:07:09.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2313 sec 2024-04-26T19:07:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:09 smithi119 ceph-mon[32752]: pgmap v1815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:09.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:09 smithi167 ceph-mon[32751]: pgmap v1815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:09 smithi090 ceph-mon[29180]: pgmap v1815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:10.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2314 sec 2024-04-26T19:07:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1912.004043 secs 2024-04-26T19:07:10.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:07:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1912.004043 secs 2024-04-26T19:07:10.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:07:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1912.004043 secs 2024-04-26T19:07:10.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:07:11.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2315 sec 2024-04-26T19:07:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1912.676268 secs 2024-04-26T19:07:11.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:11 smithi119 ceph-mon[32752]: pgmap v1816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:11.826 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:07:11.833 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:07:11.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:07:11.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1912.676268 secs 2024-04-26T19:07:11.856 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:11 smithi167 ceph-mon[32751]: pgmap v1816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:11.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:07:11.938 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:07:12.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:07:12.046 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1912.676268 secs 2024-04-26T19:07:12.054 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:11 smithi090 ceph-mon[29180]: pgmap v1816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:12.061 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:07:12.068 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:07:12.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:07:12.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2316 sec 2024-04-26T19:07:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:12 smithi119 ceph-mon[32752]: pgmap v1817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:07:12.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:12 smithi119 ceph-mon[32752]: pgmap v1818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:12.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:12 smithi090 ceph-mon[29180]: pgmap v1817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:07:12.925 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:12 smithi090 ceph-mon[29180]: pgmap v1818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:12.934 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:12 smithi167 ceph-mon[32751]: pgmap v1817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:07:12.949 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:12 smithi167 ceph-mon[32751]: pgmap v1818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:13.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2317 sec 2024-04-26T19:07:14.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2318 sec 2024-04-26T19:07:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:14 smithi119 ceph-mon[32752]: pgmap v1819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:07:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:14 smithi167 ceph-mon[32751]: pgmap v1819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:07:14.895 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:14 smithi090 ceph-mon[29180]: pgmap v1819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:07:15.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2319 sec 2024-04-26T19:07:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1917.004126 secs 2024-04-26T19:07:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1917.004126 secs 2024-04-26T19:07:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1917.004126 secs 2024-04-26T19:07:16.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2320 sec 2024-04-26T19:07:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:16 smithi167 ceph-mon[32751]: pgmap v1820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:16.841 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1917.676397 secs 2024-04-26T19:07:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:16 smithi090 ceph-mon[29180]: pgmap v1820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:16.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1917.676397 secs 2024-04-26T19:07:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:16 smithi119 ceph-mon[32752]: pgmap v1820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:17.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1917.676397 secs 2024-04-26T19:07:17.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2321 sec 2024-04-26T19:07:18.088 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2322 sec 2024-04-26T19:07:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:18 smithi167 ceph-mon[32751]: pgmap v1821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:07:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:18 smithi090 ceph-mon[29180]: pgmap v1821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:07:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:18 smithi119 ceph-mon[32752]: pgmap v1821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:07:19.087 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2323 sec 2024-04-26T19:07:20.088 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2324 sec 2024-04-26T19:07:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:20 smithi167 ceph-mon[32751]: pgmap v1822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:20.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1922.004188 secs 2024-04-26T19:07:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:20 smithi090 ceph-mon[29180]: pgmap v1822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:20.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1922.004188 secs 2024-04-26T19:07:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:20 smithi119 ceph-mon[32752]: pgmap v1822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:07:21.104 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1922.004188 secs 2024-04-26T19:07:21.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2325 sec 2024-04-26T19:07:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1922.676477 secs 2024-04-26T19:07:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1922.676477 secs 2024-04-26T19:07:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1922.676477 secs 2024-04-26T19:07:22.088 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2326 sec 2024-04-26T19:07:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:22 smithi090 ceph-mon[29180]: pgmap v1823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:07:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:22 smithi119 ceph-mon[32752]: pgmap v1823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:07:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:22 smithi167 ceph-mon[32751]: pgmap v1823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:07:23.088 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2327 sec 2024-04-26T19:07:24.088 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2328 sec 2024-04-26T19:07:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:24 smithi090 ceph-mon[29180]: pgmap v1824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:24 smithi119 ceph-mon[32752]: pgmap v1824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:24 smithi167 ceph-mon[32751]: pgmap v1824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:25.088 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2329 sec 2024-04-26T19:07:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1927.004273 secs 2024-04-26T19:07:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1927.004273 secs 2024-04-26T19:07:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1927.004273 secs 2024-04-26T19:07:26.088 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2330 sec 2024-04-26T19:07:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:26 smithi090 ceph-mon[29180]: pgmap v1825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:26.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1927.676592 secs 2024-04-26T19:07:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:26 smithi119 ceph-mon[32752]: pgmap v1825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:27.107 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1927.676592 secs 2024-04-26T19:07:27.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2331 sec 2024-04-26T19:07:27.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:26 smithi167 ceph-mon[32751]: pgmap v1825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:27.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1927.676592 secs 2024-04-26T19:07:28.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2332 sec 2024-04-26T19:07:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:28 smithi090 ceph-mon[29180]: pgmap v1826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:28 smithi119 ceph-mon[32752]: pgmap v1826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:28 smithi167 ceph-mon[32751]: pgmap v1826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:29.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2333 sec 2024-04-26T19:07:30.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2334 sec 2024-04-26T19:07:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:30 smithi090 ceph-mon[29180]: pgmap v1827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:30.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1932.004356 secs 2024-04-26T19:07:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:30 smithi119 ceph-mon[32752]: pgmap v1827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:31.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1932.004356 secs 2024-04-26T19:07:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:30 smithi167 ceph-mon[32751]: pgmap v1827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:31.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1932.004356 secs 2024-04-26T19:07:31.093 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2335 sec 2024-04-26T19:07:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1932.676709 secs 2024-04-26T19:07:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1932.676709 secs 2024-04-26T19:07:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1932.676709 secs 2024-04-26T19:07:32.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2336 sec 2024-04-26T19:07:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:32 smithi090 ceph-mon[29180]: pgmap v1828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:32 smithi119 ceph-mon[32752]: pgmap v1828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:33.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2337 sec 2024-04-26T19:07:33.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:32 smithi167 ceph-mon[32751]: pgmap v1828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:34.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2338 sec 2024-04-26T19:07:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:34 smithi090 ceph-mon[29180]: pgmap v1829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:34 smithi119 ceph-mon[32752]: pgmap v1829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:34 smithi167 ceph-mon[32751]: pgmap v1829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:35.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2339 sec 2024-04-26T19:07:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1937.004442 secs 2024-04-26T19:07:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1937.004442 secs 2024-04-26T19:07:36.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2340 sec 2024-04-26T19:07:36.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1937.004442 secs 2024-04-26T19:07:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:36 smithi090 ceph-mon[29180]: pgmap v1830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:36.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 1937.676822 secs 2024-04-26T19:07:36.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:36 smithi090 ceph-mon[29180]: slow request 364.202171 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:07:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:36 smithi119 ceph-mon[32752]: pgmap v1830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:37.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 1937.676822 secs 2024-04-26T19:07:37.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:36 smithi119 ceph-mon[32752]: slow request 364.202171 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:07:37.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:36 smithi167 ceph-mon[32751]: pgmap v1830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:37.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 1937.676822 secs 2024-04-26T19:07:37.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:36 smithi167 ceph-mon[32751]: slow request 364.202171 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:07:37.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2341 sec 2024-04-26T19:07:38.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2342 sec 2024-04-26T19:07:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:38 smithi090 ceph-mon[29180]: pgmap v1831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:38 smithi119 ceph-mon[32752]: pgmap v1831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:38 smithi167 ceph-mon[32751]: pgmap v1831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:07:39.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2343 sec 2024-04-26T19:07:40.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2344 sec 2024-04-26T19:07:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:40 smithi090 ceph-mon[29180]: pgmap v1832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:40.931 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1942.004524 secs 2024-04-26T19:07:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:40 smithi119 ceph-mon[32752]: pgmap v1832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:41.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1942.004524 secs 2024-04-26T19:07:41.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:40 smithi167 ceph-mon[32751]: pgmap v1832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1942.004524 secs 2024-04-26T19:07:41.089 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2345 sec 2024-04-26T19:07:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1942.676959 secs 2024-04-26T19:07:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1942.676959 secs 2024-04-26T19:07:42.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1942.676959 secs 2024-04-26T19:07:42.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2346 sec 2024-04-26T19:07:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:42 smithi090 ceph-mon[29180]: pgmap v1833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:42 smithi119 ceph-mon[32752]: pgmap v1833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:42 smithi167 ceph-mon[32751]: pgmap v1833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:43.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2347 sec 2024-04-26T19:07:44.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2348 sec 2024-04-26T19:07:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:44 smithi119 ceph-mon[32752]: pgmap v1834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:44 smithi167 ceph-mon[32751]: pgmap v1834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:45.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2349 sec 2024-04-26T19:07:45.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:44 smithi090 ceph-mon[29180]: pgmap v1834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1947.004609 secs 2024-04-26T19:07:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1947.004609 secs 2024-04-26T19:07:46.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2350 sec 2024-04-26T19:07:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1947.004609 secs 2024-04-26T19:07:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:46 smithi119 ceph-mon[32752]: pgmap v1835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:47.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1947.677090 secs 2024-04-26T19:07:47.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:46 smithi167 ceph-mon[32751]: pgmap v1835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:47.096 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1947.677090 secs 2024-04-26T19:07:47.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2351 sec 2024-04-26T19:07:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:46 smithi090 ceph-mon[29180]: pgmap v1835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:47.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1947.677090 secs 2024-04-26T19:07:48.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2352 sec 2024-04-26T19:07:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:48 smithi119 ceph-mon[32752]: pgmap v1836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:48 smithi167 ceph-mon[32751]: pgmap v1836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:49.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2353 sec 2024-04-26T19:07:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:48 smithi090 ceph-mon[29180]: pgmap v1836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:50.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2354 sec 2024-04-26T19:07:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:50 smithi119 ceph-mon[32752]: pgmap v1837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:51.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1952.004692 secs 2024-04-26T19:07:51.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2355 sec 2024-04-26T19:07:51.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:50 smithi167 ceph-mon[32751]: pgmap v1837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:51.179 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1952.004692 secs 2024-04-26T19:07:51.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:50 smithi090 ceph-mon[29180]: pgmap v1837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:51.194 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1952.004692 secs 2024-04-26T19:07:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1952.677187 secs 2024-04-26T19:07:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1952.677187 secs 2024-04-26T19:07:52.090 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2356 sec 2024-04-26T19:07:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1952.677187 secs 2024-04-26T19:07:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:52 smithi119 ceph-mon[32752]: pgmap v1838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:53.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2357 sec 2024-04-26T19:07:53.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:52 smithi167 ceph-mon[32751]: pgmap v1838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:53.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:52 smithi090 ceph-mon[29180]: pgmap v1838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:53 smithi119 ceph-mon[32752]: pgmap v1839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:54.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:53 smithi167 ceph-mon[32751]: pgmap v1839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:54.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2358 sec 2024-04-26T19:07:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:53 smithi090 ceph-mon[29180]: pgmap v1839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:55.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2359 sec 2024-04-26T19:07:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1957.004777 secs 2024-04-26T19:07:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1957.004777 secs 2024-04-26T19:07:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1957.004777 secs 2024-04-26T19:07:56.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2360 sec 2024-04-26T19:07:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:56 smithi119 ceph-mon[32752]: pgmap v1840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:56.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1957.677308 secs 2024-04-26T19:07:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:56 smithi167 ceph-mon[32751]: pgmap v1840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:56.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1957.677308 secs 2024-04-26T19:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:56 smithi090 ceph-mon[29180]: pgmap v1840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:07:56.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1957.677308 secs 2024-04-26T19:07:57.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2361 sec 2024-04-26T19:07:58.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2362 sec 2024-04-26T19:07:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:07:58 smithi119 ceph-mon[32752]: pgmap v1841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:07:58 smithi167 ceph-mon[32751]: pgmap v1841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:07:58 smithi090 ceph-mon[29180]: pgmap v1841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:07:59.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2363 sec 2024-04-26T19:08:00.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2364 sec 2024-04-26T19:08:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:00 smithi119 ceph-mon[32752]: pgmap v1842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:00.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1962.004860 secs 2024-04-26T19:08:00.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:00 smithi167 ceph-mon[32751]: pgmap v1842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:00.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1962.004860 secs 2024-04-26T19:08:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:00 smithi090 ceph-mon[29180]: pgmap v1842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:00.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1962.004860 secs 2024-04-26T19:08:01.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2365 sec 2024-04-26T19:08:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1962.677435 secs 2024-04-26T19:08:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1962.677435 secs 2024-04-26T19:08:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1962.677435 secs 2024-04-26T19:08:02.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2366 sec 2024-04-26T19:08:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:02 smithi119 ceph-mon[32752]: pgmap v1843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:02.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:02 smithi167 ceph-mon[32751]: pgmap v1843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:02 smithi090 ceph-mon[29180]: pgmap v1843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:03.091 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2367 sec 2024-04-26T19:08:04.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2368 sec 2024-04-26T19:08:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:04 smithi119 ceph-mon[32752]: pgmap v1844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:04 smithi167 ceph-mon[32751]: pgmap v1844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:04 smithi090 ceph-mon[29180]: pgmap v1844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:05.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2369 sec 2024-04-26T19:08:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1967.004922 secs 2024-04-26T19:08:05.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1967.004922 secs 2024-04-26T19:08:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1967.004922 secs 2024-04-26T19:08:06.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2370 sec 2024-04-26T19:08:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:06 smithi119 ceph-mon[32752]: pgmap v1845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:06.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1967.677550 secs 2024-04-26T19:08:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:06 smithi167 ceph-mon[32751]: pgmap v1845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:06.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1967.677550 secs 2024-04-26T19:08:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:06 smithi090 ceph-mon[29180]: pgmap v1845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:06.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1967.677550 secs 2024-04-26T19:08:07.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2371 sec 2024-04-26T19:08:08.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2372 sec 2024-04-26T19:08:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:08 smithi119 ceph-mon[32752]: pgmap v1846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:08.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:08 smithi167 ceph-mon[32751]: pgmap v1846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:08 smithi090 ceph-mon[29180]: pgmap v1846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:09.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2373 sec 2024-04-26T19:08:10.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2374 sec 2024-04-26T19:08:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:10 smithi119 ceph-mon[32752]: pgmap v1847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:10.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 1972.005007 secs 2024-04-26T19:08:10.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:10 smithi167 ceph-mon[32751]: pgmap v1847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:10.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 1972.005007 secs 2024-04-26T19:08:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:10 smithi090 ceph-mon[29180]: pgmap v1847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:10.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 1972.005007 secs 2024-04-26T19:08:11.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2375 sec 2024-04-26T19:08:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1972.677667 secs 2024-04-26T19:08:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1972.677667 secs 2024-04-26T19:08:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1972.677667 secs 2024-04-26T19:08:12.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2376 sec 2024-04-26T19:08:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:12 smithi119 ceph-mon[32752]: pgmap v1848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:12.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:08:12.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:12 smithi167 ceph-mon[32751]: pgmap v1848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:12.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:08:12.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:12 smithi090 ceph-mon[29180]: pgmap v1848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:12.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:08:13.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2377 sec 2024-04-26T19:08:14.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2378 sec 2024-04-26T19:08:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:14 smithi119 ceph-mon[32752]: pgmap v1849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:14.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:14 smithi167 ceph-mon[32751]: pgmap v1849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:14 smithi090 ceph-mon[29180]: pgmap v1849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:15.093 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2379 sec 2024-04-26T19:08:15.470 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:15 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 1977.005076 secs 2024-04-26T19:08:15.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:15 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 1977.005076 secs 2024-04-26T19:08:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:15 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 1977.005076 secs 2024-04-26T19:08:16.092 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2380 sec 2024-04-26T19:08:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: pgmap v1850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:16.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1977.677782 secs 2024-04-26T19:08:16.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:16.827 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:16.835 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:16.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:16.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:08:16.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:08:16.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: pgmap v1851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 984 B/s rd, 1 op/s 2024-04-26T19:08:16.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: pgmap v1852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:16.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: pgmap v1850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:17.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1977.677782 secs 2024-04-26T19:08:17.233 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:08:17.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:08:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: pgmap v1851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 984 B/s rd, 1 op/s 2024-04-26T19:08:17.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: pgmap v1852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:17.400 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.407 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: pgmap v1850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:17.415 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1977.677782 secs 2024-04-26T19:08:17.422 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.430 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.437 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:08:17.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:08:17.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: pgmap v1851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 984 B/s rd, 1 op/s 2024-04-26T19:08:17.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: pgmap v1852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:17.654 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:08:17.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2381 sec 2024-04-26T19:08:18.093 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2382 sec 2024-04-26T19:08:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:18 smithi090 ceph-mon[29180]: pgmap v1853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 609 B/s rd, 1 op/s 2024-04-26T19:08:18.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:18 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:08:18.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:18 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:08:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:18 smithi119 ceph-mon[32752]: pgmap v1853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 609 B/s rd, 1 op/s 2024-04-26T19:08:19.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:18 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:08:19.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:18 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:08:19.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:18 smithi167 ceph-mon[32751]: pgmap v1853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 609 B/s rd, 1 op/s 2024-04-26T19:08:19.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:18 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:08:19.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:18 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:08:19.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2383 sec 2024-04-26T19:08:20.093 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2384 sec 2024-04-26T19:08:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:20 smithi090 ceph-mon[29180]: pgmap v1854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:20.898 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:20 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 1982.005157 secs 2024-04-26T19:08:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:20 smithi119 ceph-mon[32752]: pgmap v1854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:21.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:20 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 1982.005157 secs 2024-04-26T19:08:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:20 smithi167 ceph-mon[32751]: pgmap v1854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:21.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:20 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 1982.005157 secs 2024-04-26T19:08:21.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2385 sec 2024-04-26T19:08:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1982.677874 secs 2024-04-26T19:08:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1982.677874 secs 2024-04-26T19:08:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1982.677874 secs 2024-04-26T19:08:22.093 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2386 sec 2024-04-26T19:08:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:22 smithi090 ceph-mon[29180]: pgmap v1855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:22 smithi119 ceph-mon[32752]: pgmap v1855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:23.125 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2387 sec 2024-04-26T19:08:23.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:22 smithi167 ceph-mon[32751]: pgmap v1855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:24.093 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2388 sec 2024-04-26T19:08:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:24 smithi119 ceph-mon[32752]: pgmap v1856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:24 smithi167 ceph-mon[32751]: pgmap v1856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:25.093 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2389 sec 2024-04-26T19:08:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:24 smithi090 ceph-mon[29180]: pgmap v1856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:25 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 1987.005240 secs 2024-04-26T19:08:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:25 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 1987.005240 secs 2024-04-26T19:08:26.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2390 sec 2024-04-26T19:08:26.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:25 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 1987.005240 secs 2024-04-26T19:08:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1987.677987 secs 2024-04-26T19:08:27.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:26 smithi119 ceph-mon[32752]: pgmap v1857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:08:27.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2391 sec 2024-04-26T19:08:27.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1987.677987 secs 2024-04-26T19:08:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:26 smithi167 ceph-mon[32751]: pgmap v1857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:08:27.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1987.677987 secs 2024-04-26T19:08:27.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:26 smithi090 ceph-mon[29180]: pgmap v1857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:08:28.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2392 sec 2024-04-26T19:08:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.726+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:08:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.727+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 19 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:08:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.727+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:08:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.727+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 20 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:08:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.734+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:08:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.735+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 19 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:08:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.735+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:08:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:08:28.735+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 20 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:08:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:28 smithi119 ceph-mon[32752]: pgmap v1858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:28 smithi167 ceph-mon[32751]: pgmap v1858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:29.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2393 sec 2024-04-26T19:08:29.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:28 smithi090 ceph-mon[29180]: pgmap v1858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:30.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2394 sec 2024-04-26T19:08:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:30 smithi119 ceph-mon[32752]: pgmap v1859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:31.067 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:30 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 1992.005318 secs 2024-04-26T19:08:31.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:30 smithi167 ceph-mon[32751]: pgmap v1859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:31.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:30 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 1992.005318 secs 2024-04-26T19:08:31.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2395 sec 2024-04-26T19:08:31.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:30 smithi090 ceph-mon[29180]: pgmap v1859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:31.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:30 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 1992.005318 secs 2024-04-26T19:08:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 1992.678123 secs 2024-04-26T19:08:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 1992.678123 secs 2024-04-26T19:08:32.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2396 sec 2024-04-26T19:08:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 1992.678123 secs 2024-04-26T19:08:32.985 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:32 smithi167 ceph-mon[32751]: pgmap v1860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:32 smithi119 ceph-mon[32752]: pgmap v1860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:33.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2397 sec 2024-04-26T19:08:33.213 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:32 smithi090 ceph-mon[29180]: pgmap v1860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:33 smithi119 ceph-mon[32752]: pgmap v1861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:34.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:33 smithi167 ceph-mon[32751]: pgmap v1861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:34.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2398 sec 2024-04-26T19:08:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:33 smithi090 ceph-mon[29180]: pgmap v1861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:35.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2399 sec 2024-04-26T19:08:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:35 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 1997.005405 secs 2024-04-26T19:08:35.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:35 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 1997.005405 secs 2024-04-26T19:08:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:35 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 1997.005405 secs 2024-04-26T19:08:36.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2400 sec 2024-04-26T19:08:36.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 1997.678234 secs 2024-04-26T19:08:36.411 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:36 smithi090 ceph-mon[29180]: slow request 724.315098 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:08:36.420 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:36 smithi090 ceph-mon[29180]: pgmap v1862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 1997.678234 secs 2024-04-26T19:08:36.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:36 smithi119 ceph-mon[32752]: slow request 724.315098 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:08:36.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:36 smithi119 ceph-mon[32752]: pgmap v1862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:36.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 1997.678234 secs 2024-04-26T19:08:36.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:36 smithi167 ceph-mon[32751]: slow request 724.315098 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:08:36.598 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:36 smithi167 ceph-mon[32751]: pgmap v1862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:37.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2401 sec 2024-04-26T19:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:37 smithi119 ceph-mon[32752]: pgmap v1863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:38.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2402 sec 2024-04-26T19:08:38.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:37 smithi167 ceph-mon[32751]: pgmap v1863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:38.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:37 smithi090 ceph-mon[29180]: pgmap v1863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:39.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2403 sec 2024-04-26T19:08:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:39 smithi119 ceph-mon[32752]: pgmap v1864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:39 smithi167 ceph-mon[32751]: pgmap v1864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:40.094 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2404 sec 2024-04-26T19:08:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:39 smithi090 ceph-mon[29180]: pgmap v1864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:40.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:40 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2002.005490 secs 2024-04-26T19:08:40.964 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2002.678354 secs 2024-04-26T19:08:41.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:40 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2002.005490 secs 2024-04-26T19:08:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2002.678354 secs 2024-04-26T19:08:41.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:40 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2002.005490 secs 2024-04-26T19:08:41.138 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2002.678354 secs 2024-04-26T19:08:41.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2405 sec 2024-04-26T19:08:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:41 smithi119 ceph-mon[32752]: pgmap v1865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:41 smithi167 ceph-mon[32751]: pgmap v1865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:42.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2406 sec 2024-04-26T19:08:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:41 smithi090 ceph-mon[29180]: pgmap v1865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:43.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2407 sec 2024-04-26T19:08:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:43 smithi119 ceph-mon[32752]: pgmap v1866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:44.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:43 smithi167 ceph-mon[32751]: pgmap v1866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:44.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2408 sec 2024-04-26T19:08:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:43 smithi090 ceph-mon[29180]: pgmap v1866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:45.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2409 sec 2024-04-26T19:08:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:45 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2007.005572 secs 2024-04-26T19:08:45.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:45 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2007.005572 secs 2024-04-26T19:08:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:45 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2007.005572 secs 2024-04-26T19:08:46.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2410 sec 2024-04-26T19:08:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2007.678463 secs 2024-04-26T19:08:46.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:46 smithi119 ceph-mon[32752]: pgmap v1867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2007.678463 secs 2024-04-26T19:08:46.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:46 smithi167 ceph-mon[32751]: pgmap v1867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2007.678463 secs 2024-04-26T19:08:46.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:46 smithi090 ceph-mon[29180]: pgmap v1867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:47.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2411 sec 2024-04-26T19:08:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:47 smithi119 ceph-mon[32752]: pgmap v1868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:48.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:47 smithi167 ceph-mon[32751]: pgmap v1868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:48.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2412 sec 2024-04-26T19:08:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:47 smithi090 ceph-mon[29180]: pgmap v1868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:49.095 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2413 sec 2024-04-26T19:08:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:49 smithi119 ceph-mon[32752]: pgmap v1869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:50.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2414 sec 2024-04-26T19:08:50.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:49 smithi167 ceph-mon[32751]: pgmap v1869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:49 smithi090 ceph-mon[29180]: pgmap v1869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:50 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2012.005657 secs 2024-04-26T19:08:51.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2012.678583 secs 2024-04-26T19:08:51.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:50 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2012.005657 secs 2024-04-26T19:08:51.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2012.678583 secs 2024-04-26T19:08:51.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2415 sec 2024-04-26T19:08:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:50 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2012.005657 secs 2024-04-26T19:08:51.265 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2012.678583 secs 2024-04-26T19:08:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:51 smithi090 ceph-mon[29180]: pgmap v1870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:51 smithi119 ceph-mon[32752]: pgmap v1870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:51 smithi167 ceph-mon[32751]: pgmap v1870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:52.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2416 sec 2024-04-26T19:08:53.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2417 sec 2024-04-26T19:08:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:53 smithi119 ceph-mon[32752]: pgmap v1871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:54.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:53 smithi167 ceph-mon[32751]: pgmap v1871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:54.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2418 sec 2024-04-26T19:08:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:53 smithi090 ceph-mon[29180]: pgmap v1871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:08:55.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2419 sec 2024-04-26T19:08:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:55 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2017.005739 secs 2024-04-26T19:08:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:55 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2017.005739 secs 2024-04-26T19:08:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:55 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2017.005739 secs 2024-04-26T19:08:56.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2420 sec 2024-04-26T19:08:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2017.678648 secs 2024-04-26T19:08:56.641 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:56 smithi119 ceph-mon[32752]: pgmap v1872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:56.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2017.678648 secs 2024-04-26T19:08:56.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:56 smithi090 ceph-mon[29180]: pgmap v1872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:56.670 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2017.678648 secs 2024-04-26T19:08:56.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:56 smithi167 ceph-mon[32751]: pgmap v1872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:57.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2421 sec 2024-04-26T19:08:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:57 smithi119 ceph-mon[32752]: pgmap v1873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:58.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:57 smithi167 ceph-mon[32751]: pgmap v1873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:58.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2422 sec 2024-04-26T19:08:58.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:57 smithi090 ceph-mon[29180]: pgmap v1873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:08:59.096 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2423 sec 2024-04-26T19:09:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:08:59 smithi119 ceph-mon[32752]: pgmap v1874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:00.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2424 sec 2024-04-26T19:09:00.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:08:59 smithi167 ceph-mon[32751]: pgmap v1874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:08:59 smithi090 ceph-mon[29180]: pgmap v1874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:00 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2022.005826 secs 2024-04-26T19:09:01.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2022.678785 secs 2024-04-26T19:09:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:00 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2022.005826 secs 2024-04-26T19:09:01.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2022.678785 secs 2024-04-26T19:09:01.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2425 sec 2024-04-26T19:09:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:00 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2022.005826 secs 2024-04-26T19:09:01.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2022.678785 secs 2024-04-26T19:09:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:01 smithi167 ceph-mon[32751]: pgmap v1875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:01 smithi119 ceph-mon[32752]: pgmap v1875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:02.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2426 sec 2024-04-26T19:09:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:01 smithi090 ceph-mon[29180]: pgmap v1875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:03.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2427 sec 2024-04-26T19:09:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:03 smithi119 ceph-mon[32752]: pgmap v1876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:04.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:03 smithi167 ceph-mon[32751]: pgmap v1876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:04.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2428 sec 2024-04-26T19:09:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:03 smithi090 ceph-mon[29180]: pgmap v1876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:05.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2429 sec 2024-04-26T19:09:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:05 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2027.005888 secs 2024-04-26T19:09:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:05 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2027.005888 secs 2024-04-26T19:09:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:05 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2027.005888 secs 2024-04-26T19:09:06.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2430 sec 2024-04-26T19:09:06.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2027.678927 secs 2024-04-26T19:09:06.535 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:06 smithi090 ceph-mon[29180]: pgmap v1877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2027.678927 secs 2024-04-26T19:09:06.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:06 smithi119 ceph-mon[32752]: pgmap v1877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:06.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2027.678927 secs 2024-04-26T19:09:06.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:06 smithi167 ceph-mon[32751]: pgmap v1877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:07.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2431 sec 2024-04-26T19:09:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:07 smithi119 ceph-mon[32752]: pgmap v1878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:07 smithi167 ceph-mon[32751]: pgmap v1878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:08.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2432 sec 2024-04-26T19:09:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:07 smithi090 ceph-mon[29180]: pgmap v1878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:09.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2433 sec 2024-04-26T19:09:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:09 smithi119 ceph-mon[32752]: pgmap v1879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:09 smithi167 ceph-mon[32751]: pgmap v1879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:10.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2434 sec 2024-04-26T19:09:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:09 smithi090 ceph-mon[29180]: pgmap v1879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:11.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:10 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2032.005952 secs 2024-04-26T19:09:11.037 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:10 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2032.679039 secs 2024-04-26T19:09:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:10 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2032.005952 secs 2024-04-26T19:09:11.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:10 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2032.679039 secs 2024-04-26T19:09:11.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:10 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2032.005952 secs 2024-04-26T19:09:11.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:10 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2032.679039 secs 2024-04-26T19:09:11.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2435 sec 2024-04-26T19:09:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:11 smithi119 ceph-mon[32752]: pgmap v1880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:11 smithi167 ceph-mon[32751]: pgmap v1880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:12.097 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2436 sec 2024-04-26T19:09:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:11 smithi090 ceph-mon[29180]: pgmap v1880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:13.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2437 sec 2024-04-26T19:09:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:13 smithi119 ceph-mon[32752]: pgmap v1881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:13 smithi167 ceph-mon[32751]: pgmap v1881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:14.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2438 sec 2024-04-26T19:09:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:13 smithi090 ceph-mon[29180]: pgmap v1881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:15.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2439 sec 2024-04-26T19:09:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:15 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2037.006036 secs 2024-04-26T19:09:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:15 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2037.006036 secs 2024-04-26T19:09:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:15 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2037.006036 secs 2024-04-26T19:09:16.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2440 sec 2024-04-26T19:09:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2037.679162 secs 2024-04-26T19:09:16.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:16 smithi119 ceph-mon[32752]: pgmap v1882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:16.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:09:16.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2037.679162 secs 2024-04-26T19:09:16.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:16 smithi167 ceph-mon[32751]: pgmap v1882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:16.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:09:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2037.679162 secs 2024-04-26T19:09:16.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:16 smithi090 ceph-mon[29180]: pgmap v1882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:16.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:09:17.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2441 sec 2024-04-26T19:09:18.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:17 smithi167 ceph-mon[32751]: pgmap v1883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:18.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:17 smithi119 ceph-mon[32752]: pgmap v1883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:18.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2442 sec 2024-04-26T19:09:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:17 smithi090 ceph-mon[29180]: pgmap v1883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:19.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2443 sec 2024-04-26T19:09:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:19 smithi167 ceph-mon[32751]: pgmap v1884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:19 smithi119 ceph-mon[32752]: pgmap v1884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:20.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2444 sec 2024-04-26T19:09:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:19 smithi090 ceph-mon[29180]: pgmap v1884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:21.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2445 sec 2024-04-26T19:09:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2042.006123 secs 2024-04-26T19:09:21.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.320 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.327 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:09:21.359 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:09:21.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: pgmap v1885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T19:09:21.377 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: pgmap v1886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:21.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:20 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2042.679260 secs 2024-04-26T19:09:21.405 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2042.006123 secs 2024-04-26T19:09:21.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:09:21.494 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:09:21.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: pgmap v1885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T19:09:21.510 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: pgmap v1886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:21.544 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:20 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2042.679260 secs 2024-04-26T19:09:21.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2042.006123 secs 2024-04-26T19:09:21.706 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:09:21.735 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:09:21.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: pgmap v1885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T19:09:21.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: pgmap v1886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:21.763 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:09:21.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:20 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2042.679260 secs 2024-04-26T19:09:22.098 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2446 sec 2024-04-26T19:09:23.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2447 sec 2024-04-26T19:09:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:23 smithi119 ceph-mon[32752]: pgmap v1887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-26T19:09:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:23 smithi167 ceph-mon[32751]: pgmap v1887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-26T19:09:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:23 smithi090 ceph-mon[29180]: pgmap v1887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-26T19:09:24.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2448 sec 2024-04-26T19:09:25.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2449 sec 2024-04-26T19:09:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:25 smithi119 ceph-mon[32752]: pgmap v1888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:25 smithi167 ceph-mon[32751]: pgmap v1888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:25 smithi090 ceph-mon[29180]: pgmap v1888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:26.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2450 sec 2024-04-26T19:09:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:26 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2047.006185 secs 2024-04-26T19:09:26.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2047.679407 secs 2024-04-26T19:09:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:26 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2047.006185 secs 2024-04-26T19:09:26.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2047.679407 secs 2024-04-26T19:09:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:26 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2047.006185 secs 2024-04-26T19:09:26.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2047.679407 secs 2024-04-26T19:09:27.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2451 sec 2024-04-26T19:09:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:27 smithi119 ceph-mon[32752]: pgmap v1889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:27.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:27 smithi167 ceph-mon[32751]: pgmap v1889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:27 smithi090 ceph-mon[29180]: pgmap v1889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:28.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2452 sec 2024-04-26T19:09:29.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2453 sec 2024-04-26T19:09:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:29 smithi119 ceph-mon[32752]: pgmap v1890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-26T19:09:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:29 smithi167 ceph-mon[32751]: pgmap v1890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-26T19:09:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:29 smithi090 ceph-mon[29180]: pgmap v1890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-26T19:09:30.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2454 sec 2024-04-26T19:09:31.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2455 sec 2024-04-26T19:09:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:31 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2052.006244 secs 2024-04-26T19:09:31.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:31 smithi119 ceph-mon[32752]: pgmap v1891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:09:31.352 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2052.679525 secs 2024-04-26T19:09:31.360 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:31 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2052.006244 secs 2024-04-26T19:09:31.368 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:31 smithi167 ceph-mon[32751]: pgmap v1891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:09:31.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2052.679525 secs 2024-04-26T19:09:31.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:31 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2052.006244 secs 2024-04-26T19:09:31.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:31 smithi090 ceph-mon[29180]: pgmap v1891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:09:31.400 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2052.679525 secs 2024-04-26T19:09:32.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2456 sec 2024-04-26T19:09:33.099 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2457 sec 2024-04-26T19:09:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:33 smithi119 ceph-mon[32752]: pgmap v1892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:33.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:33 smithi167 ceph-mon[32751]: pgmap v1892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:33 smithi090 ceph-mon[29180]: pgmap v1892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:34.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2458 sec 2024-04-26T19:09:35.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2459 sec 2024-04-26T19:09:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:35 smithi119 ceph-mon[32752]: pgmap v1893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:35 smithi167 ceph-mon[32751]: pgmap v1893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:35 smithi090 ceph-mon[29180]: pgmap v1893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:36.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2460 sec 2024-04-26T19:09:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:36 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2057.006305 secs 2024-04-26T19:09:36.402 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2057.679639 secs 2024-04-26T19:09:36.411 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:36 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2057.006305 secs 2024-04-26T19:09:36.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2057.679639 secs 2024-04-26T19:09:36.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:36 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2057.006305 secs 2024-04-26T19:09:36.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2057.679639 secs 2024-04-26T19:09:37.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2461 sec 2024-04-26T19:09:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:37 smithi119 ceph-mon[32752]: pgmap v1894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:37.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:37 smithi090 ceph-mon[29180]: pgmap v1894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:37.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:37 smithi167 ceph-mon[32751]: pgmap v1894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:38.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2462 sec 2024-04-26T19:09:39.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2463 sec 2024-04-26T19:09:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:39 smithi119 ceph-mon[32752]: pgmap v1895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:39.341 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:39 smithi090 ceph-mon[29180]: pgmap v1895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:39.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:39 smithi167 ceph-mon[32751]: pgmap v1895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:40.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2464 sec 2024-04-26T19:09:41.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2465 sec 2024-04-26T19:09:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:41 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2062.006386 secs 2024-04-26T19:09:41.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:41 smithi119 ceph-mon[32752]: pgmap v1896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:41.320 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2062.679728 secs 2024-04-26T19:09:41.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:41 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2062.006386 secs 2024-04-26T19:09:41.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:41 smithi167 ceph-mon[32751]: pgmap v1896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:41.368 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2062.679728 secs 2024-04-26T19:09:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:41 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2062.006386 secs 2024-04-26T19:09:41.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:41 smithi090 ceph-mon[29180]: pgmap v1896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:41.395 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2062.679728 secs 2024-04-26T19:09:42.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2466 sec 2024-04-26T19:09:43.100 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2467 sec 2024-04-26T19:09:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:43 smithi119 ceph-mon[32752]: pgmap v1897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:43.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:43 smithi090 ceph-mon[29180]: pgmap v1897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:43.436 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:43 smithi167 ceph-mon[32751]: pgmap v1897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:44.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2468 sec 2024-04-26T19:09:45.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2469 sec 2024-04-26T19:09:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:45 smithi119 ceph-mon[32752]: pgmap v1898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:45 smithi090 ceph-mon[29180]: pgmap v1898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:45.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:45 smithi167 ceph-mon[32751]: pgmap v1898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:46.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2470 sec 2024-04-26T19:09:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:46 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2067.006472 secs 2024-04-26T19:09:46.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2067.679807 secs 2024-04-26T19:09:46.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:46 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2067.006472 secs 2024-04-26T19:09:46.410 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2067.679807 secs 2024-04-26T19:09:46.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:46 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2067.006472 secs 2024-04-26T19:09:46.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2067.679807 secs 2024-04-26T19:09:47.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2471 sec 2024-04-26T19:09:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:47 smithi167 ceph-mon[32751]: pgmap v1899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:47 smithi090 ceph-mon[29180]: pgmap v1899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:47 smithi119 ceph-mon[32752]: pgmap v1899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:48.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2472 sec 2024-04-26T19:09:49.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2473 sec 2024-04-26T19:09:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:49 smithi090 ceph-mon[29180]: pgmap v1900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:49 smithi167 ceph-mon[32751]: pgmap v1900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:49 smithi119 ceph-mon[32752]: pgmap v1900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:50.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2474 sec 2024-04-26T19:09:51.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2475 sec 2024-04-26T19:09:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:51 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2072.006574 secs 2024-04-26T19:09:51.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:51 smithi167 ceph-mon[32751]: pgmap v1901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:51.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2072.679925 secs 2024-04-26T19:09:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:51 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2072.006574 secs 2024-04-26T19:09:51.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:51 smithi090 ceph-mon[29180]: pgmap v1901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:51.394 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2072.679925 secs 2024-04-26T19:09:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:51 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2072.006574 secs 2024-04-26T19:09:51.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:51 smithi119 ceph-mon[32752]: pgmap v1901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:51.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2072.679925 secs 2024-04-26T19:09:52.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2476 sec 2024-04-26T19:09:53.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2477 sec 2024-04-26T19:09:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:53 smithi167 ceph-mon[32751]: pgmap v1902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:53 smithi090 ceph-mon[29180]: pgmap v1902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:53 smithi119 ceph-mon[32752]: pgmap v1902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:54.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2478 sec 2024-04-26T19:09:55.101 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2479 sec 2024-04-26T19:09:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:55 smithi090 ceph-mon[29180]: pgmap v1903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:55 smithi119 ceph-mon[32752]: pgmap v1903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:55 smithi167 ceph-mon[32751]: pgmap v1903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:09:56.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2480 sec 2024-04-26T19:09:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:56 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2077.006661 secs 2024-04-26T19:09:56.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2077.680038 secs 2024-04-26T19:09:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:56 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2077.006661 secs 2024-04-26T19:09:56.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2077.680038 secs 2024-04-26T19:09:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:56 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2077.006661 secs 2024-04-26T19:09:56.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2077.680038 secs 2024-04-26T19:09:57.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2481 sec 2024-04-26T19:09:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:57 smithi090 ceph-mon[29180]: pgmap v1904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:57 smithi119 ceph-mon[32752]: pgmap v1904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:57 smithi167 ceph-mon[32751]: pgmap v1904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:58.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2482 sec 2024-04-26T19:09:59.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2483 sec 2024-04-26T19:09:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:09:59 smithi090 ceph-mon[29180]: pgmap v1905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:09:59 smithi119 ceph-mon[32752]: pgmap v1905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:09:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:09:59 smithi167 ceph-mon[32751]: pgmap v1905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:00.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2484 sec 2024-04-26T19:10:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:10:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:10:00.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:10:01.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2485 sec 2024-04-26T19:10:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:01 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2082.006745 secs 2024-04-26T19:10:01.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:01 smithi090 ceph-mon[29180]: pgmap v1906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:01.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2082.680154 secs 2024-04-26T19:10:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:01 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2082.006745 secs 2024-04-26T19:10:01.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:01 smithi167 ceph-mon[32751]: pgmap v1906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:01.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2082.680154 secs 2024-04-26T19:10:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:01 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2082.006745 secs 2024-04-26T19:10:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:01 smithi119 ceph-mon[32752]: pgmap v1906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:01.569 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2082.680154 secs 2024-04-26T19:10:02.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2486 sec 2024-04-26T19:10:03.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2487 sec 2024-04-26T19:10:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:03 smithi090 ceph-mon[29180]: pgmap v1907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:03 smithi119 ceph-mon[32752]: pgmap v1907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:03.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:03 smithi167 ceph-mon[32751]: pgmap v1907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:04.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2488 sec 2024-04-26T19:10:05.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2489 sec 2024-04-26T19:10:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:05 smithi090 ceph-mon[29180]: pgmap v1908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:05 smithi119 ceph-mon[32752]: pgmap v1908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:05 smithi167 ceph-mon[32751]: pgmap v1908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:06.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2490 sec 2024-04-26T19:10:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:06 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2087.006830 secs 2024-04-26T19:10:06.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2087.680284 secs 2024-04-26T19:10:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:06 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2087.006830 secs 2024-04-26T19:10:06.666 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2087.680284 secs 2024-04-26T19:10:06.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:06 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2087.006830 secs 2024-04-26T19:10:06.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2087.680284 secs 2024-04-26T19:10:07.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2491 sec 2024-04-26T19:10:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:07 smithi090 ceph-mon[29180]: pgmap v1909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:07 smithi119 ceph-mon[32752]: pgmap v1909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:07 smithi167 ceph-mon[32751]: pgmap v1909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:08.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2492 sec 2024-04-26T19:10:09.102 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2493 sec 2024-04-26T19:10:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:09 smithi090 ceph-mon[29180]: pgmap v1910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:09 smithi119 ceph-mon[32752]: pgmap v1910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:09.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:09 smithi167 ceph-mon[32751]: pgmap v1910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:10.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2494 sec 2024-04-26T19:10:11.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2495 sec 2024-04-26T19:10:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:11 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2092.006912 secs 2024-04-26T19:10:11.421 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:11 smithi090 ceph-mon[29180]: pgmap v1911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:11.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2092.680363 secs 2024-04-26T19:10:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:11 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2092.006912 secs 2024-04-26T19:10:11.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:11 smithi119 ceph-mon[32752]: pgmap v1911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:11.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2092.680363 secs 2024-04-26T19:10:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:11 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2092.006912 secs 2024-04-26T19:10:11.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:11 smithi167 ceph-mon[32751]: pgmap v1911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:11.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2092.680363 secs 2024-04-26T19:10:12.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2496 sec 2024-04-26T19:10:13.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2497 sec 2024-04-26T19:10:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:13 smithi090 ceph-mon[29180]: pgmap v1912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:13 smithi119 ceph-mon[32752]: pgmap v1912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:13 smithi167 ceph-mon[32751]: pgmap v1912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:14.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2498 sec 2024-04-26T19:10:15.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2499 sec 2024-04-26T19:10:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:15 smithi090 ceph-mon[29180]: pgmap v1913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:15 smithi119 ceph-mon[32752]: pgmap v1913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:15.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:15 smithi167 ceph-mon[32751]: pgmap v1913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:16.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2500 sec 2024-04-26T19:10:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:16 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2097.006997 secs 2024-04-26T19:10:16.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2097.680480 secs 2024-04-26T19:10:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:16 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2097.006997 secs 2024-04-26T19:10:16.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2097.680480 secs 2024-04-26T19:10:16.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:16 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2097.006997 secs 2024-04-26T19:10:16.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2097.680480 secs 2024-04-26T19:10:17.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2501 sec 2024-04-26T19:10:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:17 smithi090 ceph-mon[29180]: pgmap v1914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:17 smithi119 ceph-mon[32752]: pgmap v1914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:17 smithi167 ceph-mon[32751]: pgmap v1914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:18.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2502 sec 2024-04-26T19:10:19.103 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2503 sec 2024-04-26T19:10:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:19 smithi090 ceph-mon[29180]: pgmap v1915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:19 smithi119 ceph-mon[32752]: pgmap v1915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:19 smithi167 ceph-mon[32751]: pgmap v1915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:20.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2504 sec 2024-04-26T19:10:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:10:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:10:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:10:21.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2505 sec 2024-04-26T19:10:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:21 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2102.007097 secs 2024-04-26T19:10:21.626 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:21 smithi119 ceph-mon[32752]: pgmap v1916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:21.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2102.680540 secs 2024-04-26T19:10:21.645 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:10:21.654 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:10:21.663 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:10:21.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:21 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2102.007097 secs 2024-04-26T19:10:21.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:21 smithi090 ceph-mon[29180]: pgmap v1916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:21.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2102.680540 secs 2024-04-26T19:10:21.764 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:10:21.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:10:21.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:10:21.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:21 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2102.007097 secs 2024-04-26T19:10:21.874 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:21 smithi167 ceph-mon[32751]: pgmap v1916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:21.940 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2102.680540 secs 2024-04-26T19:10:21.948 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:10:21.956 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:10:22.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:10:22.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2506 sec 2024-04-26T19:10:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:22 smithi119 ceph-mon[32752]: pgmap v1917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T19:10:22.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:22 smithi119 ceph-mon[32752]: pgmap v1918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:22 smithi167 ceph-mon[32751]: pgmap v1917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T19:10:22.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:22 smithi167 ceph-mon[32751]: pgmap v1918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:22 smithi090 ceph-mon[29180]: pgmap v1917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T19:10:22.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:22 smithi090 ceph-mon[29180]: pgmap v1918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:23.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2507 sec 2024-04-26T19:10:24.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2508 sec 2024-04-26T19:10:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:24 smithi119 ceph-mon[32752]: pgmap v1919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T19:10:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:24 smithi167 ceph-mon[32751]: pgmap v1919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T19:10:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:24 smithi090 ceph-mon[29180]: pgmap v1919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T19:10:25.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2509 sec 2024-04-26T19:10:26.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2510 sec 2024-04-26T19:10:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:26 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2107.007167 secs 2024-04-26T19:10:26.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:26 smithi119 ceph-mon[32752]: pgmap v1920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:26.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2107.680632 secs 2024-04-26T19:10:26.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:26 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2107.007167 secs 2024-04-26T19:10:26.627 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:26 smithi167 ceph-mon[32751]: pgmap v1920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:26.634 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2107.680632 secs 2024-04-26T19:10:26.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:26 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2107.007167 secs 2024-04-26T19:10:26.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:26 smithi090 ceph-mon[29180]: pgmap v1920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:26.658 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2107.680632 secs 2024-04-26T19:10:27.104 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2511 sec 2024-04-26T19:10:28.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2512 sec 2024-04-26T19:10:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:28 smithi119 ceph-mon[32752]: pgmap v1921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:28 smithi167 ceph-mon[32751]: pgmap v1921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:28 smithi090 ceph-mon[29180]: pgmap v1921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:29.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2513 sec 2024-04-26T19:10:30.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2514 sec 2024-04-26T19:10:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:30 smithi119 ceph-mon[32752]: pgmap v1922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:30.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:30 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2112.007227 secs 2024-04-26T19:10:30.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:30 smithi167 ceph-mon[32751]: pgmap v1922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:30.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:30 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2112.007227 secs 2024-04-26T19:10:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:30 smithi090 ceph-mon[29180]: pgmap v1922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:10:30.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:30 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2112.007227 secs 2024-04-26T19:10:31.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2515 sec 2024-04-26T19:10:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2112.680749 secs 2024-04-26T19:10:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2112.680749 secs 2024-04-26T19:10:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2112.680749 secs 2024-04-26T19:10:32.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2516 sec 2024-04-26T19:10:32.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:32 smithi167 ceph-mon[32751]: pgmap v1923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:10:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:32 smithi119 ceph-mon[32752]: pgmap v1923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:10:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:32 smithi090 ceph-mon[29180]: pgmap v1923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:10:33.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2517 sec 2024-04-26T19:10:34.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2518 sec 2024-04-26T19:10:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:34 smithi119 ceph-mon[32752]: pgmap v1924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:34 smithi167 ceph-mon[32751]: pgmap v1924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:34 smithi090 ceph-mon[29180]: pgmap v1924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:35.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2519 sec 2024-04-26T19:10:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:35 smithi119 ceph-mon[32752]: 2 slow requests, 1 included below; oldest blocked for > 2117.007290 secs 2024-04-26T19:10:35.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:35 smithi119 ceph-mon[32752]: slow request 1443.824675 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:10:35.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:35 smithi167 ceph-mon[32751]: 2 slow requests, 1 included below; oldest blocked for > 2117.007290 secs 2024-04-26T19:10:35.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:35 smithi167 ceph-mon[32751]: slow request 1443.824675 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:10:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:35 smithi090 ceph-mon[29180]: 2 slow requests, 1 included below; oldest blocked for > 2117.007290 secs 2024-04-26T19:10:35.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:35 smithi090 ceph-mon[29180]: slow request 1443.824675 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:10:36.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2520 sec 2024-04-26T19:10:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:36 smithi119 ceph-mon[32752]: pgmap v1925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:36.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2117.680841 secs 2024-04-26T19:10:36.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:36 smithi167 ceph-mon[32751]: pgmap v1925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:36.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2117.680841 secs 2024-04-26T19:10:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:36 smithi090 ceph-mon[29180]: pgmap v1925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:36.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2117.680841 secs 2024-04-26T19:10:37.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2521 sec 2024-04-26T19:10:38.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2522 sec 2024-04-26T19:10:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:38 smithi119 ceph-mon[32752]: pgmap v1926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:38.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:38 smithi167 ceph-mon[32751]: pgmap v1926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:38 smithi090 ceph-mon[29180]: pgmap v1926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:39.105 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2523 sec 2024-04-26T19:10:40.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2524 sec 2024-04-26T19:10:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:40 smithi119 ceph-mon[32752]: pgmap v1927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:40.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:40 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2122.007374 secs 2024-04-26T19:10:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:40 smithi167 ceph-mon[32751]: pgmap v1927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:40.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:40 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2122.007374 secs 2024-04-26T19:10:40.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:40 smithi090 ceph-mon[29180]: pgmap v1927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:40.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:40 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2122.007374 secs 2024-04-26T19:10:41.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2525 sec 2024-04-26T19:10:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2122.680990 secs 2024-04-26T19:10:41.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2122.680990 secs 2024-04-26T19:10:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2122.680990 secs 2024-04-26T19:10:42.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2526 sec 2024-04-26T19:10:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:42 smithi119 ceph-mon[32752]: pgmap v1928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:42 smithi167 ceph-mon[32751]: pgmap v1928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:42 smithi090 ceph-mon[29180]: pgmap v1928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:43.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2527 sec 2024-04-26T19:10:44.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2528 sec 2024-04-26T19:10:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:44 smithi119 ceph-mon[32752]: pgmap v1929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:44.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:44 smithi167 ceph-mon[32751]: pgmap v1929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:44 smithi090 ceph-mon[29180]: pgmap v1929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:45.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2529 sec 2024-04-26T19:10:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:45 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2127.007459 secs 2024-04-26T19:10:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:45 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2127.007459 secs 2024-04-26T19:10:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:45 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2127.007459 secs 2024-04-26T19:10:46.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2530 sec 2024-04-26T19:10:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:46 smithi119 ceph-mon[32752]: pgmap v1930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:46.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2127.681098 secs 2024-04-26T19:10:46.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:46 smithi167 ceph-mon[32751]: pgmap v1930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:46.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2127.681098 secs 2024-04-26T19:10:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:46 smithi090 ceph-mon[29180]: pgmap v1930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:46.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2127.681098 secs 2024-04-26T19:10:47.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2531 sec 2024-04-26T19:10:48.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2532 sec 2024-04-26T19:10:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:48 smithi119 ceph-mon[32752]: pgmap v1931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:48 smithi167 ceph-mon[32751]: pgmap v1931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:48 smithi090 ceph-mon[29180]: pgmap v1931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:49.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2533 sec 2024-04-26T19:10:50.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2534 sec 2024-04-26T19:10:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:50 smithi119 ceph-mon[32752]: pgmap v1932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:50.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:50 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2132.007541 secs 2024-04-26T19:10:50.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:50 smithi167 ceph-mon[32751]: pgmap v1932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:50.613 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:50 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2132.007541 secs 2024-04-26T19:10:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:50 smithi090 ceph-mon[29180]: pgmap v1932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:50.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:50 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2132.007541 secs 2024-04-26T19:10:51.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2535 sec 2024-04-26T19:10:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2132.681215 secs 2024-04-26T19:10:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2132.681215 secs 2024-04-26T19:10:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2132.681215 secs 2024-04-26T19:10:52.106 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2536 sec 2024-04-26T19:10:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:52 smithi119 ceph-mon[32752]: pgmap v1933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:52 smithi167 ceph-mon[32751]: pgmap v1933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:52 smithi090 ceph-mon[29180]: pgmap v1933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:53.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2537 sec 2024-04-26T19:10:54.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2538 sec 2024-04-26T19:10:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:54 smithi119 ceph-mon[32752]: pgmap v1934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:54 smithi167 ceph-mon[32751]: pgmap v1934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:54 smithi090 ceph-mon[29180]: pgmap v1934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:55.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2539 sec 2024-04-26T19:10:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:55 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2137.007627 secs 2024-04-26T19:10:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:55 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2137.007627 secs 2024-04-26T19:10:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:55 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2137.007627 secs 2024-04-26T19:10:56.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2540 sec 2024-04-26T19:10:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:56 smithi119 ceph-mon[32752]: pgmap v1935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:56.614 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2137.681330 secs 2024-04-26T19:10:56.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:56 smithi167 ceph-mon[32751]: pgmap v1935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:56.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2137.681330 secs 2024-04-26T19:10:56.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:56 smithi090 ceph-mon[29180]: pgmap v1935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:10:56.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2137.681330 secs 2024-04-26T19:10:57.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2541 sec 2024-04-26T19:10:58.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2542 sec 2024-04-26T19:10:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:10:58 smithi119 ceph-mon[32752]: pgmap v1936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:10:58 smithi167 ceph-mon[32751]: pgmap v1936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:10:58 smithi090 ceph-mon[29180]: pgmap v1936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:10:59.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2543 sec 2024-04-26T19:11:00.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2544 sec 2024-04-26T19:11:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:00 smithi119 ceph-mon[32752]: pgmap v1937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:00.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:00 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2142.007711 secs 2024-04-26T19:11:00.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:00 smithi167 ceph-mon[32751]: pgmap v1937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:00.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:00 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2142.007711 secs 2024-04-26T19:11:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:00 smithi090 ceph-mon[29180]: pgmap v1937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:00.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:00 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2142.007711 secs 2024-04-26T19:11:01.107 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2545 sec 2024-04-26T19:11:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2142.681484 secs 2024-04-26T19:11:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2142.681484 secs 2024-04-26T19:11:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2142.681484 secs 2024-04-26T19:11:02.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2546 sec 2024-04-26T19:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:02 smithi119 ceph-mon[32752]: pgmap v1938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:02.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:02 smithi167 ceph-mon[32751]: pgmap v1938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:02 smithi090 ceph-mon[29180]: pgmap v1938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:03.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2547 sec 2024-04-26T19:11:04.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2548 sec 2024-04-26T19:11:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:04 smithi119 ceph-mon[32752]: pgmap v1939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:04.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:04 smithi167 ceph-mon[32751]: pgmap v1939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:04 smithi090 ceph-mon[29180]: pgmap v1939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:05.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2549 sec 2024-04-26T19:11:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:05 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2147.007818 secs 2024-04-26T19:11:05.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:05 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2147.007818 secs 2024-04-26T19:11:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:05 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2147.007818 secs 2024-04-26T19:11:06.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2550 sec 2024-04-26T19:11:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:06 smithi119 ceph-mon[32752]: pgmap v1940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:06.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2147.681562 secs 2024-04-26T19:11:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:06 smithi167 ceph-mon[32751]: pgmap v1940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:06.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2147.681562 secs 2024-04-26T19:11:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:06 smithi090 ceph-mon[29180]: pgmap v1940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:06.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2147.681562 secs 2024-04-26T19:11:07.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2551 sec 2024-04-26T19:11:08.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2552 sec 2024-04-26T19:11:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:08 smithi119 ceph-mon[32752]: pgmap v1941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:08.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:08 smithi167 ceph-mon[32751]: pgmap v1941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:08 smithi090 ceph-mon[29180]: pgmap v1941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:09.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2553 sec 2024-04-26T19:11:10.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2554 sec 2024-04-26T19:11:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:10 smithi119 ceph-mon[32752]: pgmap v1942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:10.628 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:10 smithi119 ceph-mon[32752]: 2 slow requests, 0 included below; oldest blocked for > 2152.007879 secs 2024-04-26T19:11:10.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:10 smithi090 ceph-mon[29180]: pgmap v1942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:10.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:10 smithi090 ceph-mon[29180]: 2 slow requests, 0 included below; oldest blocked for > 2152.007879 secs 2024-04-26T19:11:10.657 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:10 smithi167 ceph-mon[32751]: pgmap v1942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:10.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:10 smithi167 ceph-mon[32751]: 2 slow requests, 0 included below; oldest blocked for > 2152.007879 secs 2024-04-26T19:11:11.108 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2555 sec 2024-04-26T19:11:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2152.681625 secs 2024-04-26T19:11:11.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2152.681625 secs 2024-04-26T19:11:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2152.681625 secs 2024-04-26T19:11:12.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2556 sec 2024-04-26T19:11:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:12 smithi119 ceph-mon[32752]: pgmap v1943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:12 smithi167 ceph-mon[32751]: pgmap v1943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:12 smithi090 ceph-mon[29180]: pgmap v1943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:13.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2557 sec 2024-04-26T19:11:14.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2558 sec 2024-04-26T19:11:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:14 smithi167 ceph-mon[32751]: pgmap v1944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:14 smithi090 ceph-mon[29180]: pgmap v1944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:14 smithi119 ceph-mon[32752]: pgmap v1944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:15.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2559 sec 2024-04-26T19:11:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:15 smithi167 ceph-mon[32751]: 8 slow requests, 1 included below; oldest blocked for > 2157.007942 secs 2024-04-26T19:11:15.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:15 smithi167 ceph-mon[32751]: slow request 180.278077 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:11:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:15 smithi090 ceph-mon[29180]: 8 slow requests, 1 included below; oldest blocked for > 2157.007942 secs 2024-04-26T19:11:15.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:15 smithi090 ceph-mon[29180]: slow request 180.278077 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:11:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:15 smithi119 ceph-mon[32752]: 8 slow requests, 1 included below; oldest blocked for > 2157.007942 secs 2024-04-26T19:11:15.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:15 smithi119 ceph-mon[32752]: slow request 180.278077 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:11:16.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2560 sec 2024-04-26T19:11:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:16 smithi167 ceph-mon[32751]: pgmap v1945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:16.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2157.681745 secs 2024-04-26T19:11:16.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:16 smithi090 ceph-mon[29180]: pgmap v1945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:16.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2157.681745 secs 2024-04-26T19:11:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:16 smithi119 ceph-mon[32752]: pgmap v1945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:16.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2157.681745 secs 2024-04-26T19:11:17.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2561 sec 2024-04-26T19:11:18.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2562 sec 2024-04-26T19:11:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:18 smithi090 ceph-mon[29180]: pgmap v1946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:18 smithi119 ceph-mon[32752]: pgmap v1946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:18 smithi167 ceph-mon[32751]: pgmap v1946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:19.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2563 sec 2024-04-26T19:11:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:19 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:11:19.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:19 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:11:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:19 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:11:19.892 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:19 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:11:19.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:19 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:11:19.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:19 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:11:20.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2564 sec 2024-04-26T19:11:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:20 smithi090 ceph-mon[29180]: pgmap v1947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:20.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2162.008003 secs 2024-04-26T19:11:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:20 smithi119 ceph-mon[32752]: pgmap v1947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:20.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2162.008003 secs 2024-04-26T19:11:20.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:20 smithi167 ceph-mon[32751]: pgmap v1947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:20.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2162.008003 secs 2024-04-26T19:11:21.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2565 sec 2024-04-26T19:11:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2162.681865 secs 2024-04-26T19:11:21.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:11:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2162.681865 secs 2024-04-26T19:11:21.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:11:21.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2162.681865 secs 2024-04-26T19:11:21.908 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:11:22.109 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2566 sec 2024-04-26T19:11:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:22 smithi090 ceph-mon[29180]: pgmap v1948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:22.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:11:22.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:11:22.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:22 smithi090 ceph-mon[29180]: pgmap v1949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:11:22.806 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:22 smithi090 ceph-mon[29180]: pgmap v1950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:22.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:11:22.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:22 smithi167 ceph-mon[32751]: pgmap v1948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:22.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:11:22.882 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:11:22.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:22 smithi167 ceph-mon[32751]: pgmap v1949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:11:22.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:22 smithi167 ceph-mon[32751]: pgmap v1950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:22.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:11:22.913 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:22 smithi119 ceph-mon[32752]: pgmap v1948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:22.921 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:11:22.928 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:11:22.936 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:22 smithi119 ceph-mon[32752]: pgmap v1949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:11:23.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:22 smithi119 ceph-mon[32752]: pgmap v1950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:23.021 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:11:23.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2567 sec 2024-04-26T19:11:24.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2568 sec 2024-04-26T19:11:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:24 smithi119 ceph-mon[32752]: pgmap v1951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:25.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:24 smithi167 ceph-mon[32751]: pgmap v1951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:25.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2569 sec 2024-04-26T19:11:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:24 smithi090 ceph-mon[29180]: pgmap v1951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2167.008083 secs 2024-04-26T19:11:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2167.008083 secs 2024-04-26T19:11:26.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2570 sec 2024-04-26T19:11:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2167.008083 secs 2024-04-26T19:11:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2167.681944 secs 2024-04-26T19:11:27.120 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:26 smithi119 ceph-mon[32752]: pgmap v1952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:27.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2571 sec 2024-04-26T19:11:27.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2167.681944 secs 2024-04-26T19:11:27.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:26 smithi167 ceph-mon[32751]: pgmap v1952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:27.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2167.681944 secs 2024-04-26T19:11:27.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:26 smithi090 ceph-mon[29180]: pgmap v1952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:28.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2572 sec 2024-04-26T19:11:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:11:28.750+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:11:28.758 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:11:28.751+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 21 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:11:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:11:28.757+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:11:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:11:28.757+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 21 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:11:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:28 smithi119 ceph-mon[32752]: pgmap v1953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:11:29.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:28 smithi167 ceph-mon[32751]: pgmap v1953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:11:29.111 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2573 sec 2024-04-26T19:11:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:28 smithi090 ceph-mon[29180]: pgmap v1953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:11:30.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2574 sec 2024-04-26T19:11:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:30 smithi119 ceph-mon[32752]: pgmap v1954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:31.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2172.008149 secs 2024-04-26T19:11:31.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:30 smithi167 ceph-mon[32751]: pgmap v1954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:31.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2172.008149 secs 2024-04-26T19:11:31.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2575 sec 2024-04-26T19:11:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:30 smithi090 ceph-mon[29180]: pgmap v1954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:11:31.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2172.008149 secs 2024-04-26T19:11:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2172.682060 secs 2024-04-26T19:11:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2172.682060 secs 2024-04-26T19:11:32.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2576 sec 2024-04-26T19:11:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2172.682060 secs 2024-04-26T19:11:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:32 smithi119 ceph-mon[32752]: pgmap v1955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:11:33.111 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2577 sec 2024-04-26T19:11:33.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:32 smithi167 ceph-mon[32751]: pgmap v1955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:11:33.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:32 smithi090 ceph-mon[29180]: pgmap v1955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:11:34.111 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2578 sec 2024-04-26T19:11:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:34 smithi119 ceph-mon[32752]: pgmap v1956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:35.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:34 smithi167 ceph-mon[32751]: pgmap v1956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:35.110 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2579 sec 2024-04-26T19:11:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:34 smithi090 ceph-mon[29180]: pgmap v1956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2177.008212 secs 2024-04-26T19:11:36.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2580 sec 2024-04-26T19:11:36.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2177.008212 secs 2024-04-26T19:11:36.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2177.008212 secs 2024-04-26T19:11:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2177.682174 secs 2024-04-26T19:11:37.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:36 smithi119 ceph-mon[32752]: pgmap v1957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:37.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2581 sec 2024-04-26T19:11:37.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2177.682174 secs 2024-04-26T19:11:37.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:36 smithi167 ceph-mon[32751]: pgmap v1957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:37.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2177.682174 secs 2024-04-26T19:11:37.152 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:36 smithi090 ceph-mon[29180]: pgmap v1957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:38.111 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2582 sec 2024-04-26T19:11:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:38 smithi119 ceph-mon[32752]: pgmap v1958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:39.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2583 sec 2024-04-26T19:11:39.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:38 smithi167 ceph-mon[32751]: pgmap v1958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:39.255 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:38 smithi090 ceph-mon[29180]: pgmap v1958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:40.111 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2584 sec 2024-04-26T19:11:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:40 smithi119 ceph-mon[32752]: pgmap v1959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:41.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2182.008296 secs 2024-04-26T19:11:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:40 smithi167 ceph-mon[32751]: pgmap v1959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:41.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2182.008296 secs 2024-04-26T19:11:41.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2585 sec 2024-04-26T19:11:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:40 smithi090 ceph-mon[29180]: pgmap v1959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:41.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2182.008296 secs 2024-04-26T19:11:41.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2182.682309 secs 2024-04-26T19:11:42.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2182.682309 secs 2024-04-26T19:11:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2182.682309 secs 2024-04-26T19:11:42.111 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2586 sec 2024-04-26T19:11:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:42 smithi090 ceph-mon[29180]: pgmap v1960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:42 smithi119 ceph-mon[32752]: pgmap v1960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:42 smithi167 ceph-mon[32751]: pgmap v1960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:43.111 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2587 sec 2024-04-26T19:11:44.112 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2588 sec 2024-04-26T19:11:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:44 smithi119 ceph-mon[32752]: pgmap v1961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:44 smithi167 ceph-mon[32751]: pgmap v1961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:45.112 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2589 sec 2024-04-26T19:11:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:44 smithi090 ceph-mon[29180]: pgmap v1961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2187.008355 secs 2024-04-26T19:11:46.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2187.008355 secs 2024-04-26T19:11:46.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2590 sec 2024-04-26T19:11:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2187.008355 secs 2024-04-26T19:11:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2187.682447 secs 2024-04-26T19:11:47.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:46 smithi119 ceph-mon[32752]: pgmap v1962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2187.682447 secs 2024-04-26T19:11:47.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:46 smithi167 ceph-mon[32751]: pgmap v1962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:47.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2591 sec 2024-04-26T19:11:47.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2187.682447 secs 2024-04-26T19:11:47.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:46 smithi090 ceph-mon[29180]: pgmap v1962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:47 smithi119 ceph-mon[32752]: pgmap v1963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:47 smithi167 ceph-mon[32751]: pgmap v1963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:48.112 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2592 sec 2024-04-26T19:11:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:47 smithi090 ceph-mon[29180]: pgmap v1963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:49.112 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2593 sec 2024-04-26T19:11:50.112 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2594 sec 2024-04-26T19:11:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:50 smithi119 ceph-mon[32752]: pgmap v1964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:51.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2192.008439 secs 2024-04-26T19:11:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:50 smithi167 ceph-mon[32751]: pgmap v1964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:51.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2192.008439 secs 2024-04-26T19:11:51.112 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2595 sec 2024-04-26T19:11:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:50 smithi090 ceph-mon[29180]: pgmap v1964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:51.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2192.008439 secs 2024-04-26T19:11:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2192.682588 secs 2024-04-26T19:11:52.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2596 sec 2024-04-26T19:11:52.127 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2192.682588 secs 2024-04-26T19:11:52.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2192.682588 secs 2024-04-26T19:11:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:52 smithi119 ceph-mon[32752]: pgmap v1965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:52 smithi167 ceph-mon[32751]: pgmap v1965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:53.112 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2597 sec 2024-04-26T19:11:53.195 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:52 smithi090 ceph-mon[29180]: pgmap v1965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:54.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2598 sec 2024-04-26T19:11:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:54 smithi119 ceph-mon[32752]: pgmap v1966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:55.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:54 smithi167 ceph-mon[32751]: pgmap v1966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:55.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2599 sec 2024-04-26T19:11:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:54 smithi090 ceph-mon[29180]: pgmap v1966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:11:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2197.008524 secs 2024-04-26T19:11:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2197.008524 secs 2024-04-26T19:11:56.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2600 sec 2024-04-26T19:11:56.193 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2197.008524 secs 2024-04-26T19:11:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2197.682704 secs 2024-04-26T19:11:57.081 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:56 smithi119 ceph-mon[32752]: pgmap v1967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:57.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2197.682704 secs 2024-04-26T19:11:57.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:56 smithi167 ceph-mon[32751]: pgmap v1967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:57.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2601 sec 2024-04-26T19:11:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2197.682704 secs 2024-04-26T19:11:57.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:56 smithi090 ceph-mon[29180]: pgmap v1967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:58.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2602 sec 2024-04-26T19:11:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:11:58 smithi119 ceph-mon[32752]: pgmap v1968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:11:58 smithi167 ceph-mon[32751]: pgmap v1968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:11:59.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2603 sec 2024-04-26T19:11:59.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:11:58 smithi090 ceph-mon[29180]: pgmap v1968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:00.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2604 sec 2024-04-26T19:12:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:00 smithi119 ceph-mon[32752]: pgmap v1969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:01.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2202.008609 secs 2024-04-26T19:12:01.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:00 smithi167 ceph-mon[32751]: pgmap v1969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:01.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2202.008609 secs 2024-04-26T19:12:01.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2605 sec 2024-04-26T19:12:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:00 smithi090 ceph-mon[29180]: pgmap v1969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:01.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2202.008609 secs 2024-04-26T19:12:01.670 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:12:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:12:01.392+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:12:01.679 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:12:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:12:01.392+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:12:01.687 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:12:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:12:01.392+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:12:01.695 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:12:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:12:01.392+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:12:01.703 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:12:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:12:01.392+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:12:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2202.682820 secs 2024-04-26T19:12:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2202.682820 secs 2024-04-26T19:12:02.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2606 sec 2024-04-26T19:12:02.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2202.682820 secs 2024-04-26T19:12:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:02 smithi119 ceph-mon[32752]: pgmap v1970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:02 smithi167 ceph-mon[32751]: pgmap v1970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:03.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2607 sec 2024-04-26T19:12:03.200 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:02 smithi090 ceph-mon[29180]: pgmap v1970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:04.113 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2608 sec 2024-04-26T19:12:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:04 smithi119 ceph-mon[32752]: pgmap v1971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:04 smithi167 ceph-mon[32751]: pgmap v1971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:05.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2609 sec 2024-04-26T19:12:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:04 smithi090 ceph-mon[29180]: pgmap v1971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2207.008694 secs 2024-04-26T19:12:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2207.008694 secs 2024-04-26T19:12:06.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2610 sec 2024-04-26T19:12:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2207.008694 secs 2024-04-26T19:12:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2207.682890 secs 2024-04-26T19:12:07.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:06 smithi119 ceph-mon[32752]: pgmap v1972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:07.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2207.682890 secs 2024-04-26T19:12:07.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:06 smithi167 ceph-mon[32751]: pgmap v1972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:07.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2611 sec 2024-04-26T19:12:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2207.682890 secs 2024-04-26T19:12:07.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:06 smithi090 ceph-mon[29180]: pgmap v1972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:08.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2612 sec 2024-04-26T19:12:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:08 smithi119 ceph-mon[32752]: pgmap v1973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:09.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:08 smithi167 ceph-mon[32751]: pgmap v1973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:09.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2613 sec 2024-04-26T19:12:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:08 smithi090 ceph-mon[29180]: pgmap v1973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:09 smithi119 ceph-mon[32752]: pgmap v1974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:10.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:09 smithi167 ceph-mon[32751]: pgmap v1974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:10.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2614 sec 2024-04-26T19:12:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:09 smithi090 ceph-mon[29180]: pgmap v1974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2212.008782 secs 2024-04-26T19:12:11.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2615 sec 2024-04-26T19:12:11.142 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2212.008782 secs 2024-04-26T19:12:11.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2212.008782 secs 2024-04-26T19:12:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2212.682967 secs 2024-04-26T19:12:12.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:11 smithi119 ceph-mon[32752]: pgmap v1975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:12.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2212.682967 secs 2024-04-26T19:12:12.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:11 smithi090 ceph-mon[29180]: pgmap v1975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2212.682967 secs 2024-04-26T19:12:12.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:11 smithi167 ceph-mon[32751]: pgmap v1975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:12.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2616 sec 2024-04-26T19:12:13.114 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2617 sec 2024-04-26T19:12:14.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2618 sec 2024-04-26T19:12:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:14 smithi119 ceph-mon[32752]: pgmap v1976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:14 smithi167 ceph-mon[32751]: pgmap v1976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:15.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2619 sec 2024-04-26T19:12:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:14 smithi090 ceph-mon[29180]: pgmap v1976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:12:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2217.008826 secs 2024-04-26T19:12:16.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2217.008826 secs 2024-04-26T19:12:16.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2620 sec 2024-04-26T19:12:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2217.008826 secs 2024-04-26T19:12:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2217.683037 secs 2024-04-26T19:12:17.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:16 smithi119 ceph-mon[32752]: pgmap v1977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2217.683037 secs 2024-04-26T19:12:17.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:16 smithi167 ceph-mon[32751]: pgmap v1977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:17.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2621 sec 2024-04-26T19:12:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2217.683037 secs 2024-04-26T19:12:17.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:16 smithi090 ceph-mon[29180]: pgmap v1977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:18.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2622 sec 2024-04-26T19:12:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:18 smithi119 ceph-mon[32752]: pgmap v1978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:19.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:18 smithi167 ceph-mon[32751]: pgmap v1978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:19.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2623 sec 2024-04-26T19:12:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:18 smithi090 ceph-mon[29180]: pgmap v1978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:20.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2624 sec 2024-04-26T19:12:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:20 smithi119 ceph-mon[32752]: pgmap v1979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:21.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2222.008888 secs 2024-04-26T19:12:21.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:20 smithi167 ceph-mon[32751]: pgmap v1979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:21.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2222.008888 secs 2024-04-26T19:12:21.123 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2625 sec 2024-04-26T19:12:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:20 smithi090 ceph-mon[29180]: pgmap v1979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:21.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2222.008888 secs 2024-04-26T19:12:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2222.683107 secs 2024-04-26T19:12:22.096 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2222.683107 secs 2024-04-26T19:12:22.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2626 sec 2024-04-26T19:12:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2222.683107 secs 2024-04-26T19:12:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:22 smithi119 ceph-mon[32752]: pgmap v1980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:23.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:12:23.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:12:23.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:12:23.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:12:23.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:22 smithi167 ceph-mon[32751]: pgmap v1980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:23.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:12:23.179 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:12:23.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:12:23.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:12:23.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2627 sec 2024-04-26T19:12:23.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:22 smithi090 ceph-mon[29180]: pgmap v1980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:23.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:12:23.294 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:12:23.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:12:23.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:12:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:23 smithi119 ceph-mon[32752]: pgmap v1981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:12:24.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:23 smithi119 ceph-mon[32752]: pgmap v1982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:12:24.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:23 smithi167 ceph-mon[32751]: pgmap v1981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:12:24.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:23 smithi167 ceph-mon[32751]: pgmap v1982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:12:24.115 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2628 sec 2024-04-26T19:12:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:23 smithi090 ceph-mon[29180]: pgmap v1981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:12:24.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:23 smithi090 ceph-mon[29180]: pgmap v1982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:12:25.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2629 sec 2024-04-26T19:12:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:25 smithi119 ceph-mon[32752]: pgmap v1983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:26.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2227.008950 secs 2024-04-26T19:12:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:25 smithi167 ceph-mon[32751]: pgmap v1983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:26.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2227.008950 secs 2024-04-26T19:12:26.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2630 sec 2024-04-26T19:12:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:25 smithi090 ceph-mon[29180]: pgmap v1983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:26.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2227.008950 secs 2024-04-26T19:12:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2227.683164 secs 2024-04-26T19:12:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2227.683164 secs 2024-04-26T19:12:27.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2631 sec 2024-04-26T19:12:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2227.683164 secs 2024-04-26T19:12:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:27 smithi119 ceph-mon[32752]: pgmap v1984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:27 smithi167 ceph-mon[32751]: pgmap v1984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:28.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2632 sec 2024-04-26T19:12:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:27 smithi090 ceph-mon[29180]: pgmap v1984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:28 smithi119 ceph-mon[32752]: pgmap v1985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:28 smithi167 ceph-mon[32751]: pgmap v1985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:29.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2633 sec 2024-04-26T19:12:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:28 smithi090 ceph-mon[29180]: pgmap v1985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:30.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2634 sec 2024-04-26T19:12:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2232.009009 secs 2024-04-26T19:12:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2232.009009 secs 2024-04-26T19:12:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2232.009009 secs 2024-04-26T19:12:31.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2635 sec 2024-04-26T19:12:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:31 smithi119 ceph-mon[32752]: pgmap v1986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:31.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2232.683227 secs 2024-04-26T19:12:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:31 smithi167 ceph-mon[32751]: pgmap v1986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:31.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2232.683227 secs 2024-04-26T19:12:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:31 smithi090 ceph-mon[29180]: pgmap v1986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:12:31.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2232.683227 secs 2024-04-26T19:12:32.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2636 sec 2024-04-26T19:12:33.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2637 sec 2024-04-26T19:12:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:33 smithi119 ceph-mon[32752]: pgmap v1987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:12:33.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:33 smithi167 ceph-mon[32751]: pgmap v1987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:12:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:33 smithi090 ceph-mon[29180]: pgmap v1987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:12:34.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2638 sec 2024-04-26T19:12:35.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2639 sec 2024-04-26T19:12:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:35 smithi119 ceph-mon[32752]: pgmap v1988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:35.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2237.009115 secs 2024-04-26T19:12:35.868 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:35 smithi167 ceph-mon[32751]: pgmap v1988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:35.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2237.009115 secs 2024-04-26T19:12:35.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:35 smithi090 ceph-mon[29180]: pgmap v1988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:35.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2237.009115 secs 2024-04-26T19:12:36.116 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2640 sec 2024-04-26T19:12:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2237.683292 secs 2024-04-26T19:12:36.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2237.683292 secs 2024-04-26T19:12:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2237.683292 secs 2024-04-26T19:12:37.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2641 sec 2024-04-26T19:12:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:37 smithi119 ceph-mon[32752]: pgmap v1989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:37 smithi167 ceph-mon[32751]: pgmap v1989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:37 smithi090 ceph-mon[29180]: pgmap v1989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:38.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2642 sec 2024-04-26T19:12:39.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2643 sec 2024-04-26T19:12:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:39 smithi119 ceph-mon[32752]: pgmap v1990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:39 smithi167 ceph-mon[32751]: pgmap v1990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:39 smithi090 ceph-mon[29180]: pgmap v1990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:40.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2644 sec 2024-04-26T19:12:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2242.009173 secs 2024-04-26T19:12:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2242.009173 secs 2024-04-26T19:12:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2242.009173 secs 2024-04-26T19:12:41.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2645 sec 2024-04-26T19:12:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:41 smithi119 ceph-mon[32752]: pgmap v1991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:41.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2242.683405 secs 2024-04-26T19:12:41.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:41 smithi167 ceph-mon[32751]: pgmap v1991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:41.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2242.683405 secs 2024-04-26T19:12:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:41 smithi090 ceph-mon[29180]: pgmap v1991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:41.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2242.683405 secs 2024-04-26T19:12:42.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2646 sec 2024-04-26T19:12:43.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2647 sec 2024-04-26T19:12:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:43 smithi119 ceph-mon[32752]: pgmap v1992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:43 smithi167 ceph-mon[32751]: pgmap v1992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:43 smithi090 ceph-mon[29180]: pgmap v1992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:44.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2648 sec 2024-04-26T19:12:45.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2649 sec 2024-04-26T19:12:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:45 smithi119 ceph-mon[32752]: pgmap v1993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:45.886 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2247.009236 secs 2024-04-26T19:12:45.895 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:45 smithi090 ceph-mon[29180]: pgmap v1993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:45.904 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2247.009236 secs 2024-04-26T19:12:45.913 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:45 smithi167 ceph-mon[32751]: pgmap v1993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:45.922 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2247.009236 secs 2024-04-26T19:12:46.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2650 sec 2024-04-26T19:12:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2247.683428 secs 2024-04-26T19:12:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2247.683428 secs 2024-04-26T19:12:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2247.683428 secs 2024-04-26T19:12:47.117 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2651 sec 2024-04-26T19:12:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:47 smithi119 ceph-mon[32752]: pgmap v1994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:47.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:47 smithi167 ceph-mon[32751]: pgmap v1994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:47 smithi090 ceph-mon[29180]: pgmap v1994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:48.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2652 sec 2024-04-26T19:12:49.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2653 sec 2024-04-26T19:12:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:49 smithi119 ceph-mon[32752]: pgmap v1995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:49 smithi167 ceph-mon[32751]: pgmap v1995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:49 smithi090 ceph-mon[29180]: pgmap v1995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:50.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2654 sec 2024-04-26T19:12:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2252.009294 secs 2024-04-26T19:12:50.913 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2252.009294 secs 2024-04-26T19:12:50.921 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2252.009294 secs 2024-04-26T19:12:51.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2655 sec 2024-04-26T19:12:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:51 smithi119 ceph-mon[32752]: pgmap v1996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:51.920 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2252.683493 secs 2024-04-26T19:12:51.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:51 smithi090 ceph-mon[29180]: pgmap v1996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:51.937 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2252.683493 secs 2024-04-26T19:12:51.946 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:51 smithi167 ceph-mon[32751]: pgmap v1996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:51.955 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2252.683493 secs 2024-04-26T19:12:52.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2656 sec 2024-04-26T19:12:53.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2657 sec 2024-04-26T19:12:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:53 smithi119 ceph-mon[32752]: pgmap v1997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:53.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:53 smithi167 ceph-mon[32751]: pgmap v1997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:53 smithi090 ceph-mon[29180]: pgmap v1997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:54.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2658 sec 2024-04-26T19:12:55.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2659 sec 2024-04-26T19:12:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:55 smithi119 ceph-mon[32752]: pgmap v1998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:55.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2257.009357 secs 2024-04-26T19:12:55.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:55 smithi167 ceph-mon[32751]: pgmap v1998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:55.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2257.009357 secs 2024-04-26T19:12:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:55 smithi090 ceph-mon[29180]: pgmap v1998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:55.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2257.009357 secs 2024-04-26T19:12:56.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2660 sec 2024-04-26T19:12:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2257.683606 secs 2024-04-26T19:12:56.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2257.683606 secs 2024-04-26T19:12:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2257.683606 secs 2024-04-26T19:12:57.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2661 sec 2024-04-26T19:12:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:57 smithi119 ceph-mon[32752]: pgmap v1999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:57.914 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:57 smithi090 ceph-mon[29180]: pgmap v1999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:57.921 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:57 smithi167 ceph-mon[32751]: pgmap v1999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:12:58.118 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2662 sec 2024-04-26T19:12:59.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2663 sec 2024-04-26T19:12:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:12:59 smithi119 ceph-mon[32752]: pgmap v2000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:59.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:12:59 smithi167 ceph-mon[32751]: pgmap v2000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:12:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:12:59 smithi090 ceph-mon[29180]: pgmap v2000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:00.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2664 sec 2024-04-26T19:13:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2262.009439 secs 2024-04-26T19:13:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2262.009439 secs 2024-04-26T19:13:00.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2262.009439 secs 2024-04-26T19:13:01.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2665 sec 2024-04-26T19:13:01.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:01 smithi167 ceph-mon[32751]: pgmap v2001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:01.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2262.683688 secs 2024-04-26T19:13:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:01 smithi119 ceph-mon[32752]: pgmap v2001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:01.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2262.683688 secs 2024-04-26T19:13:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:01 smithi090 ceph-mon[29180]: pgmap v2001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:01.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2262.683688 secs 2024-04-26T19:13:02.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2666 sec 2024-04-26T19:13:03.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2667 sec 2024-04-26T19:13:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:03 smithi119 ceph-mon[32752]: pgmap v2002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:03 smithi167 ceph-mon[32751]: pgmap v2002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:03 smithi090 ceph-mon[29180]: pgmap v2002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:04.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2668 sec 2024-04-26T19:13:05.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2669 sec 2024-04-26T19:13:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:05 smithi119 ceph-mon[32752]: pgmap v2003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:05.883 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2267.009499 secs 2024-04-26T19:13:05.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:05 smithi090 ceph-mon[29180]: pgmap v2003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:05.914 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2267.009499 secs 2024-04-26T19:13:05.923 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:05 smithi167 ceph-mon[32751]: pgmap v2003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:05.931 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2267.009499 secs 2024-04-26T19:13:06.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2670 sec 2024-04-26T19:13:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2267.683758 secs 2024-04-26T19:13:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2267.683758 secs 2024-04-26T19:13:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2267.683758 secs 2024-04-26T19:13:07.119 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2671 sec 2024-04-26T19:13:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:07 smithi119 ceph-mon[32752]: pgmap v2004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:07 smithi167 ceph-mon[32751]: pgmap v2004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:07 smithi090 ceph-mon[29180]: pgmap v2004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:08.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2672 sec 2024-04-26T19:13:09.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2673 sec 2024-04-26T19:13:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:09 smithi119 ceph-mon[32752]: pgmap v2005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:09 smithi167 ceph-mon[32751]: pgmap v2005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:09 smithi090 ceph-mon[29180]: pgmap v2005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:10.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2674 sec 2024-04-26T19:13:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2272.009579 secs 2024-04-26T19:13:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2272.009579 secs 2024-04-26T19:13:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2272.009579 secs 2024-04-26T19:13:11.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2675 sec 2024-04-26T19:13:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:11 smithi119 ceph-mon[32752]: pgmap v2006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:11.816 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2272.683822 secs 2024-04-26T19:13:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:11 smithi167 ceph-mon[32751]: pgmap v2006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:11.950 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2272.683822 secs 2024-04-26T19:13:11.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:11 smithi090 ceph-mon[29180]: pgmap v2006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:11.969 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2272.683822 secs 2024-04-26T19:13:12.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2676 sec 2024-04-26T19:13:13.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2677 sec 2024-04-26T19:13:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:13 smithi119 ceph-mon[32752]: pgmap v2007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:13 smithi167 ceph-mon[32751]: pgmap v2007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:13 smithi090 ceph-mon[29180]: pgmap v2007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:13:14.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2678 sec 2024-04-26T19:13:15.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2679 sec 2024-04-26T19:13:15.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:15 smithi090 ceph-mon[29180]: pgmap v2008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:15.757 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2277.009663 secs 2024-04-26T19:13:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:15 smithi119 ceph-mon[32752]: pgmap v2008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2277.009663 secs 2024-04-26T19:13:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:15 smithi167 ceph-mon[32751]: pgmap v2008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:13:15.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2277.009663 secs 2024-04-26T19:13:16.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2680 sec 2024-04-26T19:13:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2277.683892 secs 2024-04-26T19:13:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2277.683892 secs 2024-04-26T19:13:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2277.683892 secs 2024-04-26T19:13:17.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2681 sec 2024-04-26T19:13:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:17 smithi119 ceph-mon[32752]: pgmap v2009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:17 smithi167 ceph-mon[32751]: pgmap v2009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:17 smithi090 ceph-mon[29180]: pgmap v2009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:18.120 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2682 sec 2024-04-26T19:13:19.121 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2683 sec 2024-04-26T19:13:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:19 smithi119 ceph-mon[32752]: pgmap v2010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:19.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:19 smithi167 ceph-mon[32751]: pgmap v2010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:19 smithi090 ceph-mon[29180]: pgmap v2010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:20.121 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2684 sec 2024-04-26T19:13:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2282.009720 secs 2024-04-26T19:13:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2282.009720 secs 2024-04-26T19:13:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2282.009720 secs 2024-04-26T19:13:21.121 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2685 sec 2024-04-26T19:13:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:21 smithi119 ceph-mon[32752]: pgmap v2011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:21.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2282.683970 secs 2024-04-26T19:13:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:21 smithi167 ceph-mon[32751]: pgmap v2011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:21.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2282.683970 secs 2024-04-26T19:13:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:21 smithi090 ceph-mon[29180]: pgmap v2011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:22.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2282.683970 secs 2024-04-26T19:13:22.121 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2686 sec 2024-04-26T19:13:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:13:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:13:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:13:23.121 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2687 sec 2024-04-26T19:13:23.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:23 smithi167 ceph-mon[32751]: pgmap v2012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:23.618 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.633 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:23 smithi090 ceph-mon[29180]: pgmap v2012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:23.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.825 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.838 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:23 smithi119 ceph-mon[32752]: pgmap v2012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:23.918 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.927 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:23.986 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:24.085 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:24.121 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2688 sec 2024-04-26T19:13:25.121 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2689 sec 2024-04-26T19:13:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:25 smithi119 ceph-mon[32752]: pgmap v2013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:25.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2287.009821 secs 2024-04-26T19:13:25.822 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:25.830 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:25.882 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:25.889 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:25.898 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:25 smithi090 ceph-mon[29180]: pgmap v2013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:25.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2287.009821 secs 2024-04-26T19:13:25.951 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:25.958 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:25.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:25 smithi167 ceph-mon[32751]: pgmap v2013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:26.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2287.009821 secs 2024-04-26T19:13:26.238 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.290 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2690 sec 2024-04-26T19:13:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2287.684056 secs 2024-04-26T19:13:26.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.943 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:26.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:26.957 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:26.968 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:26.977 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.149 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.179 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2287.684056 secs 2024-04-26T19:13:27.188 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.198 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.398 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.407 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2287.684056 secs 2024-04-26T19:13:27.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.635 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.645 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.654 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.663 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.672 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.680 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.714 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.723 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:27.793 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2691 sec 2024-04-26T19:13:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:13:27.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:13:27.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:13:27.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:13:27.914 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: pgmap v2014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:27.924 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.936 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:27.967 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.001 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.014 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:13:28.039 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:13:28.047 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:13:28.095 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:13:28.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: pgmap v2015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 998 B/s rd, 1 op/s 2024-04-26T19:13:28.198 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: pgmap v2016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 B/s rd, 1 op/s 2024-04-26T19:13:28.231 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:28.239 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:13:28.248 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:13:28.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:13:28.266 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:13:28.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: pgmap v2014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:28.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:28.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:28.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.372 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.380 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.392 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:13:28.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:13:28.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:13:28.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:13:28.534 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: pgmap v2015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 998 B/s rd, 1 op/s 2024-04-26T19:13:28.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: pgmap v2016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 B/s rd, 1 op/s 2024-04-26T19:13:28.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:28.602 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:13:28.611 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:13:28.618 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:13:28.626 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:13:28.681 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: pgmap v2014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:28.690 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:28.734 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:28.742 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.750 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.758 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.765 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:13:28.773 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:13:28.822 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:13:28.830 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:13:28.887 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:13:28.944 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: pgmap v2015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 998 B/s rd, 1 op/s 2024-04-26T19:13:28.952 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: pgmap v2016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 B/s rd, 1 op/s 2024-04-26T19:13:28.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:13:28.968 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2692 sec 2024-04-26T19:13:29.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2693 sec 2024-04-26T19:13:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:29 smithi119 ceph-mon[32752]: pgmap v2017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:30.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:29 smithi167 ceph-mon[32751]: pgmap v2017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:30.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2694 sec 2024-04-26T19:13:30.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:29 smithi090 ceph-mon[29180]: pgmap v2017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2292.009881 secs 2024-04-26T19:13:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2292.009881 secs 2024-04-26T19:13:31.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2695 sec 2024-04-26T19:13:31.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2292.009881 secs 2024-04-26T19:13:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2292.684189 secs 2024-04-26T19:13:31.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:31 smithi090 ceph-mon[29180]: pgmap v2018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2292.684189 secs 2024-04-26T19:13:31.986 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:31 smithi167 ceph-mon[32751]: pgmap v2018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:32.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2292.684189 secs 2024-04-26T19:13:32.129 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:31 smithi119 ceph-mon[32752]: pgmap v2018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:32.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2696 sec 2024-04-26T19:13:33.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2697 sec 2024-04-26T19:13:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:33 smithi119 ceph-mon[32752]: pgmap v2019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 B/s rd, 1 op/s 2024-04-26T19:13:34.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2698 sec 2024-04-26T19:13:34.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:33 smithi167 ceph-mon[32751]: pgmap v2019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 B/s rd, 1 op/s 2024-04-26T19:13:34.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:33 smithi090 ceph-mon[29180]: pgmap v2019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 B/s rd, 1 op/s 2024-04-26T19:13:35.123 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2699 sec 2024-04-26T19:13:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:35 smithi119 ceph-mon[32752]: pgmap v2020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:36.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2297.009942 secs 2024-04-26T19:13:36.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:35 smithi167 ceph-mon[32751]: pgmap v2020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:36.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2297.009942 secs 2024-04-26T19:13:36.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2700 sec 2024-04-26T19:13:36.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:35 smithi090 ceph-mon[29180]: pgmap v2020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:36.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2297.009942 secs 2024-04-26T19:13:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 2297.684306 secs 2024-04-26T19:13:37.082 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:36 smithi119 ceph-mon[32752]: slow request 724.209655 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:13:37.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 2297.684306 secs 2024-04-26T19:13:37.101 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:36 smithi167 ceph-mon[32751]: slow request 724.209655 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:13:37.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2701 sec 2024-04-26T19:13:37.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 2297.684306 secs 2024-04-26T19:13:37.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:36 smithi090 ceph-mon[29180]: slow request 724.209655 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:13:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:37 smithi119 ceph-mon[32752]: pgmap v2021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:13:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:37 smithi167 ceph-mon[32751]: pgmap v2021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:13:38.122 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2702 sec 2024-04-26T19:13:38.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:37 smithi090 ceph-mon[29180]: pgmap v2021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:13:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:38 smithi119 ceph-mon[32752]: pgmap v2022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:38 smithi167 ceph-mon[32751]: pgmap v2022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:39.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2703 sec 2024-04-26T19:13:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:38 smithi090 ceph-mon[29180]: pgmap v2022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:40.123 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2704 sec 2024-04-26T19:13:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2302.010002 secs 2024-04-26T19:13:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2302.010002 secs 2024-04-26T19:13:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2302.010002 secs 2024-04-26T19:13:41.123 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2705 sec 2024-04-26T19:13:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2302.684420 secs 2024-04-26T19:13:41.592 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:41 smithi119 ceph-mon[32752]: pgmap v2023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:41.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2302.684420 secs 2024-04-26T19:13:41.611 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:41 smithi167 ceph-mon[32751]: pgmap v2023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2302.684420 secs 2024-04-26T19:13:41.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:41 smithi090 ceph-mon[29180]: pgmap v2023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:42.125 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2706 sec 2024-04-26T19:13:43.123 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2707 sec 2024-04-26T19:13:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:43 smithi119 ceph-mon[32752]: pgmap v2024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:43 smithi167 ceph-mon[32751]: pgmap v2024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:44.123 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2708 sec 2024-04-26T19:13:44.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:43 smithi090 ceph-mon[29180]: pgmap v2024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:45.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2709 sec 2024-04-26T19:13:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:45 smithi119 ceph-mon[32752]: pgmap v2025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:46.100 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2307.010065 secs 2024-04-26T19:13:46.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:45 smithi167 ceph-mon[32751]: pgmap v2025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:46.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2307.010065 secs 2024-04-26T19:13:46.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2710 sec 2024-04-26T19:13:46.231 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:45 smithi090 ceph-mon[29180]: pgmap v2025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:46.241 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2307.010065 secs 2024-04-26T19:13:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2307.684530 secs 2024-04-26T19:13:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2307.684530 secs 2024-04-26T19:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2307.684530 secs 2024-04-26T19:13:47.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2711 sec 2024-04-26T19:13:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:47 smithi119 ceph-mon[32752]: pgmap v2026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:48.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:47 smithi167 ceph-mon[32751]: pgmap v2026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:48.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2712 sec 2024-04-26T19:13:48.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:47 smithi090 ceph-mon[29180]: pgmap v2026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:49.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2713 sec 2024-04-26T19:13:50.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:49 smithi119 ceph-mon[32752]: pgmap v2027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:49 smithi167 ceph-mon[32751]: pgmap v2027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:50.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2714 sec 2024-04-26T19:13:50.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:49 smithi090 ceph-mon[29180]: pgmap v2027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2312.010148 secs 2024-04-26T19:13:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2312.010148 secs 2024-04-26T19:13:51.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2715 sec 2024-04-26T19:13:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2312.010148 secs 2024-04-26T19:13:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2312.684651 secs 2024-04-26T19:13:52.109 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:51 smithi119 ceph-mon[32752]: pgmap v2028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:52.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2312.684651 secs 2024-04-26T19:13:52.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:51 smithi167 ceph-mon[32751]: pgmap v2028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:52.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2716 sec 2024-04-26T19:13:52.146 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2312.684651 secs 2024-04-26T19:13:52.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:51 smithi090 ceph-mon[29180]: pgmap v2028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:52 smithi119 ceph-mon[32752]: pgmap v2029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:52 smithi167 ceph-mon[32751]: pgmap v2029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:53.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2717 sec 2024-04-26T19:13:53.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:52 smithi090 ceph-mon[29180]: pgmap v2029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:54.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2718 sec 2024-04-26T19:13:55.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2719 sec 2024-04-26T19:13:56.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:55 smithi119 ceph-mon[32752]: pgmap v2030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:56.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2317.010210 secs 2024-04-26T19:13:56.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:55 smithi167 ceph-mon[32751]: pgmap v2030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:56.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2317.010210 secs 2024-04-26T19:13:56.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2720 sec 2024-04-26T19:13:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:55 smithi090 ceph-mon[29180]: pgmap v2030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:13:56.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2317.010210 secs 2024-04-26T19:13:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2317.684772 secs 2024-04-26T19:13:57.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2317.684772 secs 2024-04-26T19:13:57.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2721 sec 2024-04-26T19:13:57.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2317.684772 secs 2024-04-26T19:13:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:57 smithi119 ceph-mon[32752]: pgmap v2031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:58.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2722 sec 2024-04-26T19:13:58.141 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:57 smithi167 ceph-mon[32751]: pgmap v2031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:58.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:57 smithi090 ceph-mon[29180]: pgmap v2031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:13:59.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2723 sec 2024-04-26T19:14:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:13:59 smithi119 ceph-mon[32752]: pgmap v2032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:00.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:13:59 smithi167 ceph-mon[32751]: pgmap v2032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:00.125 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2724 sec 2024-04-26T19:14:00.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:13:59 smithi090 ceph-mon[29180]: pgmap v2032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2322.010293 secs 2024-04-26T19:14:01.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2322.010293 secs 2024-04-26T19:14:01.125 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2725 sec 2024-04-26T19:14:01.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2322.010293 secs 2024-04-26T19:14:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2322.684917 secs 2024-04-26T19:14:01.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:01 smithi167 ceph-mon[32751]: pgmap v2033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2322.684917 secs 2024-04-26T19:14:02.145 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:01 smithi119 ceph-mon[32752]: pgmap v2033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:02.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2726 sec 2024-04-26T19:14:02.187 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2322.684917 secs 2024-04-26T19:14:02.198 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:01 smithi090 ceph-mon[29180]: pgmap v2033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:02 smithi119 ceph-mon[32752]: pgmap v2034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:02 smithi167 ceph-mon[32751]: pgmap v2034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:03.125 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2727 sec 2024-04-26T19:14:03.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:02 smithi090 ceph-mon[29180]: pgmap v2034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:04.125 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2728 sec 2024-04-26T19:14:05.125 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2729 sec 2024-04-26T19:14:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:05 smithi119 ceph-mon[32752]: pgmap v2035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:06.072 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2327.010380 secs 2024-04-26T19:14:06.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:05 smithi167 ceph-mon[32751]: pgmap v2035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:06.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2327.010380 secs 2024-04-26T19:14:06.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2730 sec 2024-04-26T19:14:06.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:05 smithi090 ceph-mon[29180]: pgmap v2035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:06.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2327.010380 secs 2024-04-26T19:14:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2327.684984 secs 2024-04-26T19:14:07.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2327.684984 secs 2024-04-26T19:14:07.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2731 sec 2024-04-26T19:14:07.141 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2327.684984 secs 2024-04-26T19:14:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:07 smithi119 ceph-mon[32752]: pgmap v2036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:08.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:07 smithi167 ceph-mon[32751]: pgmap v2036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:08.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2732 sec 2024-04-26T19:14:08.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:07 smithi090 ceph-mon[29180]: pgmap v2036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:08 smithi119 ceph-mon[32752]: pgmap v2037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:09.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:08 smithi167 ceph-mon[32751]: pgmap v2037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:08 smithi090 ceph-mon[29180]: pgmap v2037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:09.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2733 sec 2024-04-26T19:14:10.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2734 sec 2024-04-26T19:14:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2332.010462 secs 2024-04-26T19:14:10.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2332.010462 secs 2024-04-26T19:14:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2332.010462 secs 2024-04-26T19:14:11.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2735 sec 2024-04-26T19:14:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2332.685094 secs 2024-04-26T19:14:11.604 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:11 smithi119 ceph-mon[32752]: pgmap v2038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:11.612 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2332.685094 secs 2024-04-26T19:14:11.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:11 smithi167 ceph-mon[32751]: pgmap v2038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2332.685094 secs 2024-04-26T19:14:11.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:11 smithi090 ceph-mon[29180]: pgmap v2038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:12.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2736 sec 2024-04-26T19:14:13.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2737 sec 2024-04-26T19:14:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:13 smithi119 ceph-mon[32752]: pgmap v2039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:14.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:13 smithi090 ceph-mon[29180]: pgmap v2039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:14.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:13 smithi167 ceph-mon[32751]: pgmap v2039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:14.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2738 sec 2024-04-26T19:14:15.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2739 sec 2024-04-26T19:14:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:15 smithi119 ceph-mon[32752]: pgmap v2040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:16.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:15 smithi119 ceph-mon[32752]: 8 slow requests, 1 included below; oldest blocked for > 2337.010546 secs 2024-04-26T19:14:16.104 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:15 smithi119 ceph-mon[32752]: slow request 360.280681 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:14:16.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:15 smithi167 ceph-mon[32751]: pgmap v2040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:16.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:15 smithi167 ceph-mon[32751]: 8 slow requests, 1 included below; oldest blocked for > 2337.010546 secs 2024-04-26T19:14:16.132 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:15 smithi167 ceph-mon[32751]: slow request 360.280681 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:14:16.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:15 smithi090 ceph-mon[29180]: pgmap v2040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:16.152 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:15 smithi090 ceph-mon[29180]: 8 slow requests, 1 included below; oldest blocked for > 2337.010546 secs 2024-04-26T19:14:16.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:15 smithi090 ceph-mon[29180]: slow request 360.280681 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:14:16.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2740 sec 2024-04-26T19:14:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2337.685227 secs 2024-04-26T19:14:17.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2337.685227 secs 2024-04-26T19:14:17.151 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2337.685227 secs 2024-04-26T19:14:17.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2741 sec 2024-04-26T19:14:18.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:17 smithi090 ceph-mon[29180]: pgmap v2041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:17 smithi119 ceph-mon[32752]: pgmap v2041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:17 smithi167 ceph-mon[32751]: pgmap v2041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:18.126 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2742 sec 2024-04-26T19:14:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:18 smithi119 ceph-mon[32752]: pgmap v2042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:18 smithi167 ceph-mon[32751]: pgmap v2042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:18 smithi090 ceph-mon[29180]: pgmap v2042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:19.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2743 sec 2024-04-26T19:14:20.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2744 sec 2024-04-26T19:14:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2342.010628 secs 2024-04-26T19:14:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2342.010628 secs 2024-04-26T19:14:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2342.010628 secs 2024-04-26T19:14:21.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2745 sec 2024-04-26T19:14:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2342.685344 secs 2024-04-26T19:14:21.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:21 smithi119 ceph-mon[32752]: pgmap v2043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:21.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2342.685344 secs 2024-04-26T19:14:21.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:21 smithi167 ceph-mon[32751]: pgmap v2043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2342.685344 secs 2024-04-26T19:14:21.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:21 smithi090 ceph-mon[29180]: pgmap v2043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:22.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2746 sec 2024-04-26T19:14:23.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2747 sec 2024-04-26T19:14:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:23 smithi119 ceph-mon[32752]: pgmap v2044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:24.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:23 smithi167 ceph-mon[32751]: pgmap v2044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:24.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:23 smithi090 ceph-mon[29180]: pgmap v2044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:24.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2748 sec 2024-04-26T19:14:25.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2749 sec 2024-04-26T19:14:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:25 smithi119 ceph-mon[32752]: pgmap v2045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:26.146 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2347.010713 secs 2024-04-26T19:14:26.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:25 smithi090 ceph-mon[29180]: pgmap v2045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:26.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2347.010713 secs 2024-04-26T19:14:26.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:25 smithi167 ceph-mon[32751]: pgmap v2045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:26.182 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2347.010713 secs 2024-04-26T19:14:26.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2750 sec 2024-04-26T19:14:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2347.685436 secs 2024-04-26T19:14:27.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2347.685436 secs 2024-04-26T19:14:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2347.685436 secs 2024-04-26T19:14:27.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2751 sec 2024-04-26T19:14:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:27 smithi119 ceph-mon[32752]: pgmap v2046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:28.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:14:28.102 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:28.117 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:28.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:27 smithi167 ceph-mon[32751]: pgmap v2046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:28.135 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:14:28.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:28.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:28.238 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:27 smithi090 ceph-mon[29180]: pgmap v2046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:28.248 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:14:28.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:28.267 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:28.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2752 sec 2024-04-26T19:14:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:14:28.774+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:14:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:14:28.775+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 22 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:14:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:14:28.778+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:14:28.803 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:14:28.778+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 22 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:14:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:14:29.067 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:14:29.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:28 smithi119 ceph-mon[32752]: pgmap v2047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T19:14:29.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:28 smithi119 ceph-mon[32752]: pgmap v2048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:14:29.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:29.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:14:29.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:14:29.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:28 smithi167 ceph-mon[32751]: pgmap v2047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T19:14:29.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:28 smithi167 ceph-mon[32751]: pgmap v2048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:14:29.239 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:29.247 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:14:29.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:14:29.265 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:28 smithi090 ceph-mon[29180]: pgmap v2047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T19:14:29.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:28 smithi090 ceph-mon[29180]: pgmap v2048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:14:29.332 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:14:29.340 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2753 sec 2024-04-26T19:14:30.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2754 sec 2024-04-26T19:14:31.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2755 sec 2024-04-26T19:14:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2352.010813 secs 2024-04-26T19:14:31.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:30 smithi119 ceph-mon[32752]: pgmap v2049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:31.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2352.685521 secs 2024-04-26T19:14:31.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2352.010813 secs 2024-04-26T19:14:31.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:30 smithi167 ceph-mon[32751]: pgmap v2049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:31.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2352.685521 secs 2024-04-26T19:14:31.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2352.010813 secs 2024-04-26T19:14:31.470 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:30 smithi090 ceph-mon[29180]: pgmap v2049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:31.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2352.685521 secs 2024-04-26T19:14:32.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2756 sec 2024-04-26T19:14:33.127 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2757 sec 2024-04-26T19:14:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:32 smithi119 ceph-mon[32752]: pgmap v2050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:32 smithi167 ceph-mon[32751]: pgmap v2050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:32 smithi090 ceph-mon[29180]: pgmap v2050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:34.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2758 sec 2024-04-26T19:14:35.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2759 sec 2024-04-26T19:14:35.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:34 smithi119 ceph-mon[32752]: pgmap v2051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:35.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:34 smithi090 ceph-mon[29180]: pgmap v2051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:35.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:34 smithi167 ceph-mon[32751]: pgmap v2051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:36.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2760 sec 2024-04-26T19:14:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:36 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2357.010898 secs 2024-04-26T19:14:36.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2357.685593 secs 2024-04-26T19:14:36.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:36 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2357.010898 secs 2024-04-26T19:14:36.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2357.685593 secs 2024-04-26T19:14:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:36 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2357.010898 secs 2024-04-26T19:14:36.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2357.685593 secs 2024-04-26T19:14:37.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2761 sec 2024-04-26T19:14:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:37 smithi119 ceph-mon[32752]: pgmap v2052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:37 smithi167 ceph-mon[32751]: pgmap v2052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:37 smithi090 ceph-mon[29180]: pgmap v2052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:14:38.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2762 sec 2024-04-26T19:14:39.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2763 sec 2024-04-26T19:14:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:39 smithi119 ceph-mon[32752]: pgmap v2053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:14:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:39 smithi167 ceph-mon[32751]: pgmap v2053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:14:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:39 smithi090 ceph-mon[29180]: pgmap v2053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:14:40.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2764 sec 2024-04-26T19:14:41.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2765 sec 2024-04-26T19:14:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:41 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2362.010958 secs 2024-04-26T19:14:41.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:41 smithi119 ceph-mon[32752]: pgmap v2054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:41.371 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2362.685660 secs 2024-04-26T19:14:41.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:41 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2362.010958 secs 2024-04-26T19:14:41.400 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:41 smithi090 ceph-mon[29180]: pgmap v2054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:41.412 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2362.685660 secs 2024-04-26T19:14:41.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:41 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2362.010958 secs 2024-04-26T19:14:41.437 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:41 smithi167 ceph-mon[32751]: pgmap v2054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:41.449 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2362.685660 secs 2024-04-26T19:14:42.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2766 sec 2024-04-26T19:14:43.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2767 sec 2024-04-26T19:14:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:43 smithi119 ceph-mon[32752]: pgmap v2055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:43 smithi167 ceph-mon[32751]: pgmap v2055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:43 smithi090 ceph-mon[29180]: pgmap v2055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:44.128 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2768 sec 2024-04-26T19:14:45.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2769 sec 2024-04-26T19:14:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:45 smithi119 ceph-mon[32752]: pgmap v2056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:45 smithi167 ceph-mon[32751]: pgmap v2056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:45 smithi090 ceph-mon[29180]: pgmap v2056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:46.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2770 sec 2024-04-26T19:14:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:46 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2367.011020 secs 2024-04-26T19:14:46.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2367.685798 secs 2024-04-26T19:14:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:46 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2367.011020 secs 2024-04-26T19:14:46.389 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2367.685798 secs 2024-04-26T19:14:46.398 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:46 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2367.011020 secs 2024-04-26T19:14:46.408 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2367.685798 secs 2024-04-26T19:14:47.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2771 sec 2024-04-26T19:14:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:47 smithi119 ceph-mon[32752]: pgmap v2057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:47 smithi167 ceph-mon[32751]: pgmap v2057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:47 smithi090 ceph-mon[29180]: pgmap v2057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:48.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2772 sec 2024-04-26T19:14:49.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2773 sec 2024-04-26T19:14:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:49 smithi119 ceph-mon[32752]: pgmap v2058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:49 smithi167 ceph-mon[32751]: pgmap v2058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:49 smithi090 ceph-mon[29180]: pgmap v2058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:50.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2774 sec 2024-04-26T19:14:51.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2775 sec 2024-04-26T19:14:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:51 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2372.011079 secs 2024-04-26T19:14:51.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:51 smithi119 ceph-mon[32752]: pgmap v2059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:51.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2372.685865 secs 2024-04-26T19:14:51.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:51 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2372.011079 secs 2024-04-26T19:14:51.392 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:51 smithi167 ceph-mon[32751]: pgmap v2059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:51.405 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2372.685865 secs 2024-04-26T19:14:51.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:51 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2372.011079 secs 2024-04-26T19:14:51.433 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:51 smithi090 ceph-mon[29180]: pgmap v2059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:51.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2372.685865 secs 2024-04-26T19:14:52.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2776 sec 2024-04-26T19:14:53.129 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2777 sec 2024-04-26T19:14:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:53 smithi119 ceph-mon[32752]: pgmap v2060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:53 smithi167 ceph-mon[32751]: pgmap v2060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:53 smithi090 ceph-mon[29180]: pgmap v2060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:54.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2778 sec 2024-04-26T19:14:55.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2779 sec 2024-04-26T19:14:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:55 smithi119 ceph-mon[32752]: pgmap v2061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:55 smithi167 ceph-mon[32751]: pgmap v2061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:55 smithi090 ceph-mon[29180]: pgmap v2061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:14:56.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2780 sec 2024-04-26T19:14:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:56 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2377.011163 secs 2024-04-26T19:14:56.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2377.685929 secs 2024-04-26T19:14:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:56 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2377.011163 secs 2024-04-26T19:14:56.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2377.685929 secs 2024-04-26T19:14:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:56 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2377.011163 secs 2024-04-26T19:14:56.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2377.685929 secs 2024-04-26T19:14:57.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2781 sec 2024-04-26T19:14:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:57 smithi119 ceph-mon[32752]: pgmap v2062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:57 smithi167 ceph-mon[32751]: pgmap v2062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:57 smithi090 ceph-mon[29180]: pgmap v2062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:58.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2782 sec 2024-04-26T19:14:59.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2783 sec 2024-04-26T19:14:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:14:59 smithi119 ceph-mon[32752]: pgmap v2063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:14:59 smithi167 ceph-mon[32751]: pgmap v2063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:14:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:14:59 smithi090 ceph-mon[29180]: pgmap v2063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:00.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2784 sec 2024-04-26T19:15:01.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2785 sec 2024-04-26T19:15:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:01 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2382.011222 secs 2024-04-26T19:15:01.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:01 smithi119 ceph-mon[32752]: pgmap v2064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:01.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2382.685998 secs 2024-04-26T19:15:01.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:01 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2382.011222 secs 2024-04-26T19:15:01.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:01 smithi167 ceph-mon[32751]: pgmap v2064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:01.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2382.685998 secs 2024-04-26T19:15:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:01 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2382.011222 secs 2024-04-26T19:15:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:01 smithi090 ceph-mon[29180]: pgmap v2064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:01.395 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2382.685998 secs 2024-04-26T19:15:02.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2786 sec 2024-04-26T19:15:03.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2787 sec 2024-04-26T19:15:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:03 smithi167 ceph-mon[32751]: pgmap v2065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:03 smithi090 ceph-mon[29180]: pgmap v2065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:03 smithi119 ceph-mon[32752]: pgmap v2065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:04.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2788 sec 2024-04-26T19:15:05.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2789 sec 2024-04-26T19:15:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:05 smithi167 ceph-mon[32751]: pgmap v2066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:05 smithi090 ceph-mon[29180]: pgmap v2066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:05 smithi119 ceph-mon[32752]: pgmap v2066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:06.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2790 sec 2024-04-26T19:15:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:06 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2387.011295 secs 2024-04-26T19:15:06.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2387.686136 secs 2024-04-26T19:15:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:06 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2387.011295 secs 2024-04-26T19:15:06.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2387.686136 secs 2024-04-26T19:15:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:06 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2387.011295 secs 2024-04-26T19:15:06.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2387.686136 secs 2024-04-26T19:15:07.130 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2791 sec 2024-04-26T19:15:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:07 smithi090 ceph-mon[29180]: pgmap v2067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:07 smithi119 ceph-mon[32752]: pgmap v2067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:07 smithi167 ceph-mon[32751]: pgmap v2067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:08.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2792 sec 2024-04-26T19:15:09.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2793 sec 2024-04-26T19:15:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:09 smithi090 ceph-mon[29180]: pgmap v2068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:09 smithi119 ceph-mon[32752]: pgmap v2068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:09 smithi167 ceph-mon[32751]: pgmap v2068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:10.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2794 sec 2024-04-26T19:15:11.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2795 sec 2024-04-26T19:15:11.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:11 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2392.011377 secs 2024-04-26T19:15:11.365 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:11 smithi119 ceph-mon[32752]: pgmap v2069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:11.376 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2392.686235 secs 2024-04-26T19:15:11.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:11 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2392.011377 secs 2024-04-26T19:15:11.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:11 smithi090 ceph-mon[29180]: pgmap v2069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:11.411 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2392.686235 secs 2024-04-26T19:15:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:11 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2392.011377 secs 2024-04-26T19:15:11.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:11 smithi167 ceph-mon[32751]: pgmap v2069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:11.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2392.686235 secs 2024-04-26T19:15:12.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2796 sec 2024-04-26T19:15:13.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2797 sec 2024-04-26T19:15:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:13 smithi090 ceph-mon[29180]: pgmap v2070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:13 smithi119 ceph-mon[32752]: pgmap v2070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:13 smithi167 ceph-mon[32751]: pgmap v2070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:14.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2798 sec 2024-04-26T19:15:15.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2799 sec 2024-04-26T19:15:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:15 smithi090 ceph-mon[29180]: pgmap v2071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:15 smithi119 ceph-mon[32752]: pgmap v2071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:15 smithi167 ceph-mon[32751]: pgmap v2071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:16.131 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2800 sec 2024-04-26T19:15:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:16 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2397.011461 secs 2024-04-26T19:15:16.452 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2397.686367 secs 2024-04-26T19:15:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:16 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2397.011461 secs 2024-04-26T19:15:16.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2397.686367 secs 2024-04-26T19:15:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:16 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2397.011461 secs 2024-04-26T19:15:16.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2397.686367 secs 2024-04-26T19:15:17.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2801 sec 2024-04-26T19:15:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:17 smithi090 ceph-mon[29180]: pgmap v2072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:17 smithi119 ceph-mon[32752]: pgmap v2072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:17 smithi167 ceph-mon[32751]: pgmap v2072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:18.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2802 sec 2024-04-26T19:15:19.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2803 sec 2024-04-26T19:15:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:19 smithi090 ceph-mon[29180]: pgmap v2073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:19 smithi119 ceph-mon[32752]: pgmap v2073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:19 smithi167 ceph-mon[32751]: pgmap v2073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:20.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2804 sec 2024-04-26T19:15:21.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2805 sec 2024-04-26T19:15:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:21 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2402.011543 secs 2024-04-26T19:15:21.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:21 smithi090 ceph-mon[29180]: pgmap v2074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:21.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2402.686506 secs 2024-04-26T19:15:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:21 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2402.011543 secs 2024-04-26T19:15:21.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:21 smithi119 ceph-mon[32752]: pgmap v2074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:21.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2402.686506 secs 2024-04-26T19:15:21.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:21 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2402.011543 secs 2024-04-26T19:15:21.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:21 smithi167 ceph-mon[32751]: pgmap v2074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:21.613 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2402.686506 secs 2024-04-26T19:15:22.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2806 sec 2024-04-26T19:15:23.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2807 sec 2024-04-26T19:15:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:23 smithi090 ceph-mon[29180]: pgmap v2075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:23 smithi119 ceph-mon[32752]: pgmap v2075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:23.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:23 smithi167 ceph-mon[32751]: pgmap v2075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:24.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2808 sec 2024-04-26T19:15:25.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2809 sec 2024-04-26T19:15:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:25 smithi090 ceph-mon[29180]: pgmap v2076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:25 smithi119 ceph-mon[32752]: pgmap v2076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:25 smithi167 ceph-mon[32751]: pgmap v2076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:26.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2810 sec 2024-04-26T19:15:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:26 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2407.011623 secs 2024-04-26T19:15:26.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2407.686571 secs 2024-04-26T19:15:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:26 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2407.011623 secs 2024-04-26T19:15:26.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2407.686571 secs 2024-04-26T19:15:26.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:26 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2407.011623 secs 2024-04-26T19:15:26.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2407.686571 secs 2024-04-26T19:15:27.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2811 sec 2024-04-26T19:15:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:27 smithi090 ceph-mon[29180]: pgmap v2077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:27 smithi119 ceph-mon[32752]: pgmap v2077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:27 smithi167 ceph-mon[32751]: pgmap v2077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:28.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2812 sec 2024-04-26T19:15:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:15:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:15:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:15:29.132 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2813 sec 2024-04-26T19:15:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:29 smithi090 ceph-mon[29180]: pgmap v2078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:29.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:15:29.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:15:29.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:15:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:29 smithi119 ceph-mon[32752]: pgmap v2078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:29.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:15:29.577 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:15:29.588 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:15:29.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:29 smithi167 ceph-mon[32751]: pgmap v2078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:29.648 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:15:29.657 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:15:29.666 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:15:30.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2814 sec 2024-04-26T19:15:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:30 smithi090 ceph-mon[29180]: pgmap v2079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:15:30.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:30 smithi090 ceph-mon[29180]: pgmap v2080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:30 smithi119 ceph-mon[32752]: pgmap v2079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:15:30.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:30 smithi119 ceph-mon[32752]: pgmap v2080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:30.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:30 smithi167 ceph-mon[32751]: pgmap v2079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:15:30.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:30 smithi167 ceph-mon[32751]: pgmap v2080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:31.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2815 sec 2024-04-26T19:15:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:31 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2412.011701 secs 2024-04-26T19:15:31.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2412.686639 secs 2024-04-26T19:15:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:31 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2412.011701 secs 2024-04-26T19:15:31.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2412.686639 secs 2024-04-26T19:15:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:31 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2412.011701 secs 2024-04-26T19:15:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2412.686639 secs 2024-04-26T19:15:32.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2816 sec 2024-04-26T19:15:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:32 smithi119 ceph-mon[32752]: pgmap v2081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:32 smithi167 ceph-mon[32751]: pgmap v2081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:32 smithi090 ceph-mon[29180]: pgmap v2081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:33.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2817 sec 2024-04-26T19:15:34.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2818 sec 2024-04-26T19:15:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:34 smithi119 ceph-mon[32752]: pgmap v2082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:15:34.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:34 smithi090 ceph-mon[29180]: pgmap v2082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:15:34.680 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:34 smithi167 ceph-mon[32751]: pgmap v2082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:15:35.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2819 sec 2024-04-26T19:15:36.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2820 sec 2024-04-26T19:15:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:36 smithi119 ceph-mon[32752]: pgmap v2083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:36.623 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:36 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2417.011790 secs 2024-04-26T19:15:36.632 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 2417.686714 secs 2024-04-26T19:15:36.644 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:36 smithi119 ceph-mon[32752]: slow request 1444.378479 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:15:36.658 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:36 smithi090 ceph-mon[29180]: pgmap v2083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:36.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:36 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2417.011790 secs 2024-04-26T19:15:36.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 2417.686714 secs 2024-04-26T19:15:36.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:36 smithi090 ceph-mon[29180]: slow request 1444.378479 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:15:36.705 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:36 smithi167 ceph-mon[32751]: pgmap v2083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:36.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:36 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2417.011790 secs 2024-04-26T19:15:36.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 2417.686714 secs 2024-04-26T19:15:36.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:36 smithi167 ceph-mon[32751]: slow request 1444.378479 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:15:37.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2821 sec 2024-04-26T19:15:38.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2822 sec 2024-04-26T19:15:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:38 smithi119 ceph-mon[32752]: pgmap v2084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:38 smithi167 ceph-mon[32751]: pgmap v2084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:38 smithi090 ceph-mon[29180]: pgmap v2084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:15:39.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2823 sec 2024-04-26T19:15:40.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2824 sec 2024-04-26T19:15:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:40 smithi119 ceph-mon[32752]: pgmap v2085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:15:40.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:40 smithi167 ceph-mon[32751]: pgmap v2085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:15:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:40 smithi090 ceph-mon[29180]: pgmap v2085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:15:41.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2825 sec 2024-04-26T19:15:41.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:41 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2422.011868 secs 2024-04-26T19:15:41.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2422.686782 secs 2024-04-26T19:15:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:41 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2422.011868 secs 2024-04-26T19:15:41.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2422.686782 secs 2024-04-26T19:15:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:41 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2422.011868 secs 2024-04-26T19:15:41.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2422.686782 secs 2024-04-26T19:15:42.133 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2826 sec 2024-04-26T19:15:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:42 smithi119 ceph-mon[32752]: pgmap v2086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:42.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:42 smithi090 ceph-mon[29180]: pgmap v2086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:42.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:42 smithi167 ceph-mon[32751]: pgmap v2086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:43.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2827 sec 2024-04-26T19:15:44.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2828 sec 2024-04-26T19:15:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:44 smithi119 ceph-mon[32752]: pgmap v2087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:44 smithi167 ceph-mon[32751]: pgmap v2087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:44 smithi090 ceph-mon[29180]: pgmap v2087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:45.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2829 sec 2024-04-26T19:15:46.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2830 sec 2024-04-26T19:15:46.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:46 smithi090 ceph-mon[29180]: pgmap v2088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:46.437 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:46 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2427.011956 secs 2024-04-26T19:15:46.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2427.686843 secs 2024-04-26T19:15:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:46 smithi119 ceph-mon[32752]: pgmap v2088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:46.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:46 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2427.011956 secs 2024-04-26T19:15:46.598 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2427.686843 secs 2024-04-26T19:15:46.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:46 smithi167 ceph-mon[32751]: pgmap v2088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:46.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:46 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2427.011956 secs 2024-04-26T19:15:46.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2427.686843 secs 2024-04-26T19:15:47.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2831 sec 2024-04-26T19:15:48.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2832 sec 2024-04-26T19:15:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:48 smithi119 ceph-mon[32752]: pgmap v2089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:48.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:48 smithi090 ceph-mon[29180]: pgmap v2089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:48.707 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:48 smithi167 ceph-mon[32751]: pgmap v2089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:49.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2833 sec 2024-04-26T19:15:50.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2834 sec 2024-04-26T19:15:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:50 smithi119 ceph-mon[32752]: pgmap v2090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:50 smithi167 ceph-mon[32751]: pgmap v2090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:50 smithi090 ceph-mon[29180]: pgmap v2090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:51.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2835 sec 2024-04-26T19:15:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:51 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2432.012038 secs 2024-04-26T19:15:51.584 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2432.686903 secs 2024-04-26T19:15:51.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:51 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2432.012038 secs 2024-04-26T19:15:51.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2432.686903 secs 2024-04-26T19:15:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:51 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2432.012038 secs 2024-04-26T19:15:51.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2432.686903 secs 2024-04-26T19:15:52.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2836 sec 2024-04-26T19:15:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:52 smithi119 ceph-mon[32752]: pgmap v2091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:52.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:52 smithi167 ceph-mon[32751]: pgmap v2091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:52 smithi090 ceph-mon[29180]: pgmap v2091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:53.134 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2837 sec 2024-04-26T19:15:54.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2838 sec 2024-04-26T19:15:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:54 smithi119 ceph-mon[32752]: pgmap v2092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:54.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:54 smithi167 ceph-mon[32751]: pgmap v2092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:54 smithi090 ceph-mon[29180]: pgmap v2092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:55.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2839 sec 2024-04-26T19:15:56.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2840 sec 2024-04-26T19:15:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:56 smithi119 ceph-mon[32752]: pgmap v2093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:56.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:56 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2437.012101 secs 2024-04-26T19:15:56.578 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2437.686960 secs 2024-04-26T19:15:56.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:56 smithi167 ceph-mon[32751]: pgmap v2093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:56.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:56 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2437.012101 secs 2024-04-26T19:15:56.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2437.686960 secs 2024-04-26T19:15:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:56 smithi090 ceph-mon[29180]: pgmap v2093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:15:56.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:56 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2437.012101 secs 2024-04-26T19:15:56.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2437.686960 secs 2024-04-26T19:15:57.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2841 sec 2024-04-26T19:15:58.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2842 sec 2024-04-26T19:15:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:15:58 smithi119 ceph-mon[32752]: pgmap v2094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:58.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:15:58 smithi167 ceph-mon[32751]: pgmap v2094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:58.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:15:58 smithi090 ceph-mon[29180]: pgmap v2094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:15:59.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2843 sec 2024-04-26T19:16:00.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2844 sec 2024-04-26T19:16:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:00 smithi119 ceph-mon[32752]: pgmap v2095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:00.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:00 smithi167 ceph-mon[32751]: pgmap v2095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:00 smithi090 ceph-mon[29180]: pgmap v2095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:01.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2845 sec 2024-04-26T19:16:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:01 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2442.012186 secs 2024-04-26T19:16:01.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2442.687028 secs 2024-04-26T19:16:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:01 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2442.012186 secs 2024-04-26T19:16:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2442.687028 secs 2024-04-26T19:16:01.669 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:01 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2442.012186 secs 2024-04-26T19:16:01.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2442.687028 secs 2024-04-26T19:16:02.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2846 sec 2024-04-26T19:16:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:02 smithi119 ceph-mon[32752]: pgmap v2096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:02 smithi167 ceph-mon[32751]: pgmap v2096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:02 smithi090 ceph-mon[29180]: pgmap v2096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:03.135 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2847 sec 2024-04-26T19:16:04.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2848 sec 2024-04-26T19:16:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:04 smithi119 ceph-mon[32752]: pgmap v2097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:04 smithi167 ceph-mon[32751]: pgmap v2097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:04 smithi090 ceph-mon[29180]: pgmap v2097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:05.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2849 sec 2024-04-26T19:16:06.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2850 sec 2024-04-26T19:16:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:06 smithi119 ceph-mon[32752]: pgmap v2098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:06.622 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:06 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2447.012271 secs 2024-04-26T19:16:06.632 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2447.687093 secs 2024-04-26T19:16:06.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:06 smithi090 ceph-mon[29180]: pgmap v2098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:06.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:06 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2447.012271 secs 2024-04-26T19:16:06.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2447.687093 secs 2024-04-26T19:16:06.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:06 smithi167 ceph-mon[32751]: pgmap v2098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:06.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:06 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2447.012271 secs 2024-04-26T19:16:06.682 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2447.687093 secs 2024-04-26T19:16:07.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2851 sec 2024-04-26T19:16:08.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2852 sec 2024-04-26T19:16:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:08 smithi119 ceph-mon[32752]: pgmap v2099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:08 smithi167 ceph-mon[32751]: pgmap v2099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:08 smithi090 ceph-mon[29180]: pgmap v2099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:09.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2853 sec 2024-04-26T19:16:10.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2854 sec 2024-04-26T19:16:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:10 smithi119 ceph-mon[32752]: pgmap v2100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:10 smithi167 ceph-mon[32751]: pgmap v2100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:10 smithi090 ceph-mon[29180]: pgmap v2100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:11.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2855 sec 2024-04-26T19:16:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:11 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2452.012353 secs 2024-04-26T19:16:11.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2452.687162 secs 2024-04-26T19:16:11.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:11 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2452.012353 secs 2024-04-26T19:16:11.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2452.687162 secs 2024-04-26T19:16:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:11 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2452.012353 secs 2024-04-26T19:16:11.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2452.687162 secs 2024-04-26T19:16:12.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2856 sec 2024-04-26T19:16:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:12 smithi119 ceph-mon[32752]: pgmap v2101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:12.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:12 smithi167 ceph-mon[32751]: pgmap v2101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:12 smithi090 ceph-mon[29180]: pgmap v2101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:13.136 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2857 sec 2024-04-26T19:16:14.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2858 sec 2024-04-26T19:16:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:14 smithi119 ceph-mon[32752]: pgmap v2102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:14 smithi167 ceph-mon[32751]: pgmap v2102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:14 smithi090 ceph-mon[29180]: pgmap v2102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:15.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2859 sec 2024-04-26T19:16:16.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2860 sec 2024-04-26T19:16:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:16 smithi119 ceph-mon[32752]: pgmap v2103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:16.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:16 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2457.012440 secs 2024-04-26T19:16:16.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2457.687206 secs 2024-04-26T19:16:16.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:16 smithi167 ceph-mon[32751]: pgmap v2103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:16.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:16 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2457.012440 secs 2024-04-26T19:16:16.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2457.687206 secs 2024-04-26T19:16:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:16 smithi090 ceph-mon[29180]: pgmap v2103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:16.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:16 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2457.012440 secs 2024-04-26T19:16:16.800 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2457.687206 secs 2024-04-26T19:16:17.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2861 sec 2024-04-26T19:16:18.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2862 sec 2024-04-26T19:16:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:18 smithi119 ceph-mon[32752]: pgmap v2104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:18.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:18 smithi167 ceph-mon[32751]: pgmap v2104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:18 smithi090 ceph-mon[29180]: pgmap v2104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:19.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2863 sec 2024-04-26T19:16:20.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2864 sec 2024-04-26T19:16:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:20 smithi119 ceph-mon[32752]: pgmap v2105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:20.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2462.012522 secs 2024-04-26T19:16:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:20 smithi167 ceph-mon[32751]: pgmap v2105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:20.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2462.012522 secs 2024-04-26T19:16:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:20 smithi090 ceph-mon[29180]: pgmap v2105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:20.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2462.012522 secs 2024-04-26T19:16:21.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2865 sec 2024-04-26T19:16:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2462.687330 secs 2024-04-26T19:16:21.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2462.687330 secs 2024-04-26T19:16:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2462.687330 secs 2024-04-26T19:16:22.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2866 sec 2024-04-26T19:16:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:22 smithi119 ceph-mon[32752]: pgmap v2106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:22 smithi167 ceph-mon[32751]: pgmap v2106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:22 smithi090 ceph-mon[29180]: pgmap v2106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:23.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2867 sec 2024-04-26T19:16:24.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2868 sec 2024-04-26T19:16:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:24 smithi119 ceph-mon[32752]: pgmap v2107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:24.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:24 smithi167 ceph-mon[32751]: pgmap v2107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:24.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:24 smithi090 ceph-mon[29180]: pgmap v2107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:25.137 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2869 sec 2024-04-26T19:16:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2467.012609 secs 2024-04-26T19:16:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2467.012609 secs 2024-04-26T19:16:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2467.012609 secs 2024-04-26T19:16:26.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2870 sec 2024-04-26T19:16:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:26 smithi119 ceph-mon[32752]: pgmap v2108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:26.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2467.687461 secs 2024-04-26T19:16:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:26 smithi167 ceph-mon[32751]: pgmap v2108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:26.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2467.687461 secs 2024-04-26T19:16:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:26 smithi090 ceph-mon[29180]: pgmap v2108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:26.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2467.687461 secs 2024-04-26T19:16:27.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2871 sec 2024-04-26T19:16:28.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2872 sec 2024-04-26T19:16:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:28 smithi119 ceph-mon[32752]: pgmap v2109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:28 smithi167 ceph-mon[32751]: pgmap v2109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:28 smithi090 ceph-mon[29180]: pgmap v2109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:29.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2873 sec 2024-04-26T19:16:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:16:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:16:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:16:30.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2874 sec 2024-04-26T19:16:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:30 smithi119 ceph-mon[32752]: pgmap v2110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:30.581 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:16:30.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:16:30.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:30 smithi119 ceph-mon[32752]: pgmap v2111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:16:30.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:30 smithi119 ceph-mon[32752]: pgmap v2112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:30.614 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:16:30.622 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2472.012694 secs 2024-04-26T19:16:30.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:30 smithi090 ceph-mon[29180]: pgmap v2110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:30.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:16:30.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:16:30.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:30 smithi090 ceph-mon[29180]: pgmap v2111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:16:30.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:30 smithi090 ceph-mon[29180]: pgmap v2112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:30.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:16:30.762 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2472.012694 secs 2024-04-26T19:16:30.770 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:30 smithi167 ceph-mon[32751]: pgmap v2110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:30.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:16:30.785 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:16:30.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:30 smithi167 ceph-mon[32751]: pgmap v2111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:16:30.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:30 smithi167 ceph-mon[32751]: pgmap v2112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:30.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:16:30.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2472.012694 secs 2024-04-26T19:16:31.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2875 sec 2024-04-26T19:16:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2472.687538 secs 2024-04-26T19:16:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2472.687538 secs 2024-04-26T19:16:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2472.687538 secs 2024-04-26T19:16:32.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2876 sec 2024-04-26T19:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:32 smithi119 ceph-mon[32752]: pgmap v2113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:32 smithi167 ceph-mon[32751]: pgmap v2113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:32 smithi090 ceph-mon[29180]: pgmap v2113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:33.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2877 sec 2024-04-26T19:16:34.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2878 sec 2024-04-26T19:16:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:34 smithi119 ceph-mon[32752]: pgmap v2114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:34.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:34 smithi167 ceph-mon[32751]: pgmap v2114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:34 smithi090 ceph-mon[29180]: pgmap v2114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:35.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2879 sec 2024-04-26T19:16:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2477.012757 secs 2024-04-26T19:16:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2477.012757 secs 2024-04-26T19:16:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2477.012757 secs 2024-04-26T19:16:36.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2880 sec 2024-04-26T19:16:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2477.687656 secs 2024-04-26T19:16:36.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:36 smithi167 ceph-mon[32751]: pgmap v2115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2477.687656 secs 2024-04-26T19:16:36.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:36 smithi090 ceph-mon[29180]: pgmap v2115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2477.687656 secs 2024-04-26T19:16:37.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:36 smithi119 ceph-mon[32752]: pgmap v2115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:16:37.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2881 sec 2024-04-26T19:16:38.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2882 sec 2024-04-26T19:16:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:38 smithi167 ceph-mon[32751]: pgmap v2116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:16:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:38 smithi090 ceph-mon[29180]: pgmap v2116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:16:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:38 smithi119 ceph-mon[32752]: pgmap v2116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:16:39.138 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2883 sec 2024-04-26T19:16:40.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2884 sec 2024-04-26T19:16:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:40 smithi167 ceph-mon[32751]: pgmap v2117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:16:40.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2482.012844 secs 2024-04-26T19:16:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:40 smithi090 ceph-mon[29180]: pgmap v2117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:16:40.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2482.012844 secs 2024-04-26T19:16:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:40 smithi119 ceph-mon[32752]: pgmap v2117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:16:41.103 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2482.012844 secs 2024-04-26T19:16:41.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2885 sec 2024-04-26T19:16:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2482.687718 secs 2024-04-26T19:16:41.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2482.687718 secs 2024-04-26T19:16:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2482.687718 secs 2024-04-26T19:16:42.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2886 sec 2024-04-26T19:16:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:42 smithi090 ceph-mon[29180]: pgmap v2118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:42 smithi119 ceph-mon[32752]: pgmap v2118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:42 smithi167 ceph-mon[32751]: pgmap v2118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:43.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2887 sec 2024-04-26T19:16:43.793 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:16:43.792+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:16:43.814 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:16:43.793+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 23 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca80063e0 con 0x7f2cd4104500 2024-04-26T19:16:43.825 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:16:43.799+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:16:43.835 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:16:43.800+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 23 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:16:44.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2888 sec 2024-04-26T19:16:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:44 smithi119 ceph-mon[32752]: pgmap v2119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:44 smithi167 ceph-mon[32751]: pgmap v2119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:44 smithi090 ceph-mon[29180]: pgmap v2119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:45.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2889 sec 2024-04-26T19:16:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2487.012908 secs 2024-04-26T19:16:45.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2487.012908 secs 2024-04-26T19:16:45.949 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2487.012908 secs 2024-04-26T19:16:46.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2890 sec 2024-04-26T19:16:46.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2487.687836 secs 2024-04-26T19:16:46.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:46 smithi090 ceph-mon[29180]: pgmap v2120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2487.687836 secs 2024-04-26T19:16:46.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:46 smithi167 ceph-mon[32751]: pgmap v2120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:46.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2487.687836 secs 2024-04-26T19:16:46.599 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:46 smithi119 ceph-mon[32752]: pgmap v2120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:47.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2891 sec 2024-04-26T19:16:48.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2892 sec 2024-04-26T19:16:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:48 smithi167 ceph-mon[32751]: pgmap v2121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:48 smithi090 ceph-mon[29180]: pgmap v2121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:48 smithi119 ceph-mon[32752]: pgmap v2121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:49.139 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2893 sec 2024-04-26T19:16:50.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2894 sec 2024-04-26T19:16:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:50 smithi167 ceph-mon[32751]: pgmap v2122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:50.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2492.012990 secs 2024-04-26T19:16:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:50 smithi090 ceph-mon[29180]: pgmap v2122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:50.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2492.012990 secs 2024-04-26T19:16:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:50 smithi119 ceph-mon[32752]: pgmap v2122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:51.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2492.012990 secs 2024-04-26T19:16:51.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2895 sec 2024-04-26T19:16:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2492.687991 secs 2024-04-26T19:16:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2492.687991 secs 2024-04-26T19:16:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2492.687991 secs 2024-04-26T19:16:52.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2896 sec 2024-04-26T19:16:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:52 smithi167 ceph-mon[32751]: pgmap v2123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:52 smithi090 ceph-mon[29180]: pgmap v2123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:52 smithi119 ceph-mon[32752]: pgmap v2123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:53.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2897 sec 2024-04-26T19:16:54.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2898 sec 2024-04-26T19:16:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:54 smithi090 ceph-mon[29180]: pgmap v2124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:54 smithi119 ceph-mon[32752]: pgmap v2124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:54 smithi167 ceph-mon[32751]: pgmap v2124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:16:55.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2899 sec 2024-04-26T19:16:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2497.013051 secs 2024-04-26T19:16:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2497.013051 secs 2024-04-26T19:16:56.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2497.013051 secs 2024-04-26T19:16:56.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2900 sec 2024-04-26T19:16:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2497.688103 secs 2024-04-26T19:16:56.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:56 smithi090 ceph-mon[29180]: pgmap v2125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2497.688103 secs 2024-04-26T19:16:57.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:56 smithi119 ceph-mon[32752]: pgmap v2125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2497.688103 secs 2024-04-26T19:16:57.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:56 smithi167 ceph-mon[32751]: pgmap v2125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:57.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2901 sec 2024-04-26T19:16:58.140 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2902 sec 2024-04-26T19:16:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:16:58 smithi090 ceph-mon[29180]: pgmap v2126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:16:58 smithi119 ceph-mon[32752]: pgmap v2126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:59.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:16:58 smithi167 ceph-mon[32751]: pgmap v2126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:16:59.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2903 sec 2024-04-26T19:17:00.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2904 sec 2024-04-26T19:17:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:00 smithi090 ceph-mon[29180]: pgmap v2127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:01.057 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2502.013110 secs 2024-04-26T19:17:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:00 smithi167 ceph-mon[32751]: pgmap v2127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:01.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2502.013110 secs 2024-04-26T19:17:01.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:00 smithi119 ceph-mon[32752]: pgmap v2127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:01.129 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2502.013110 secs 2024-04-26T19:17:01.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2905 sec 2024-04-26T19:17:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2502.688217 secs 2024-04-26T19:17:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2502.688217 secs 2024-04-26T19:17:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2502.688217 secs 2024-04-26T19:17:02.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2906 sec 2024-04-26T19:17:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:02 smithi090 ceph-mon[29180]: pgmap v2128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:02 smithi119 ceph-mon[32752]: pgmap v2128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:02 smithi167 ceph-mon[32751]: pgmap v2128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:03.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2907 sec 2024-04-26T19:17:04.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2908 sec 2024-04-26T19:17:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:04 smithi090 ceph-mon[29180]: pgmap v2129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:04 smithi119 ceph-mon[32752]: pgmap v2129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:05.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:04 smithi167 ceph-mon[32751]: pgmap v2129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:05.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2909 sec 2024-04-26T19:17:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2507.013196 secs 2024-04-26T19:17:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2507.013196 secs 2024-04-26T19:17:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2507.013196 secs 2024-04-26T19:17:06.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2910 sec 2024-04-26T19:17:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2507.688367 secs 2024-04-26T19:17:06.942 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:06 smithi090 ceph-mon[29180]: pgmap v2130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2507.688367 secs 2024-04-26T19:17:07.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:06 smithi119 ceph-mon[32752]: pgmap v2130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2507.688367 secs 2024-04-26T19:17:07.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:06 smithi167 ceph-mon[32751]: pgmap v2130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:07.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2911 sec 2024-04-26T19:17:08.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2912 sec 2024-04-26T19:17:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:08 smithi090 ceph-mon[29180]: pgmap v2131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:08 smithi119 ceph-mon[32752]: pgmap v2131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:08 smithi167 ceph-mon[32751]: pgmap v2131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:09.141 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2913 sec 2024-04-26T19:17:10.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2914 sec 2024-04-26T19:17:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:10 smithi090 ceph-mon[29180]: pgmap v2132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:10.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2512.013270 secs 2024-04-26T19:17:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:10 smithi119 ceph-mon[32752]: pgmap v2132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:11.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2512.013270 secs 2024-04-26T19:17:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:10 smithi167 ceph-mon[32751]: pgmap v2132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:11.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2512.013270 secs 2024-04-26T19:17:11.901 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2915 sec 2024-04-26T19:17:11.910 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2512.688470 secs 2024-04-26T19:17:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2512.688470 secs 2024-04-26T19:17:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2512.688470 secs 2024-04-26T19:17:12.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2916 sec 2024-04-26T19:17:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:12 smithi090 ceph-mon[29180]: pgmap v2133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:12.970 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:12 smithi119 ceph-mon[32752]: pgmap v2133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:12 smithi167 ceph-mon[32751]: pgmap v2133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:17:13.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2917 sec 2024-04-26T19:17:14.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2918 sec 2024-04-26T19:17:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:14 smithi119 ceph-mon[32752]: pgmap v2134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:14 smithi167 ceph-mon[32751]: pgmap v2134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:14 smithi090 ceph-mon[29180]: pgmap v2134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:17:15.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2919 sec 2024-04-26T19:17:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2517.013329 secs 2024-04-26T19:17:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2517.013329 secs 2024-04-26T19:17:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2517.013329 secs 2024-04-26T19:17:16.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2920 sec 2024-04-26T19:17:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2517.688546 secs 2024-04-26T19:17:16.542 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:16 smithi090 ceph-mon[29180]: pgmap v2135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:16.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2517.688546 secs 2024-04-26T19:17:16.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:16 smithi167 ceph-mon[32751]: pgmap v2135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:16.613 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2517.688546 secs 2024-04-26T19:17:16.621 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:16 smithi119 ceph-mon[32752]: pgmap v2135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:17.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2921 sec 2024-04-26T19:17:18.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2922 sec 2024-04-26T19:17:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:18 smithi167 ceph-mon[32751]: pgmap v2136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:18 smithi090 ceph-mon[29180]: pgmap v2136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:18 smithi119 ceph-mon[32752]: pgmap v2136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:19.142 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2923 sec 2024-04-26T19:17:20.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2924 sec 2024-04-26T19:17:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:20 smithi167 ceph-mon[32751]: pgmap v2137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:20.958 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2522.013411 secs 2024-04-26T19:17:20.973 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:20 smithi090 ceph-mon[29180]: pgmap v2137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:20.982 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2522.013411 secs 2024-04-26T19:17:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:20 smithi119 ceph-mon[32752]: pgmap v2137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:21.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2522.013411 secs 2024-04-26T19:17:21.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2925 sec 2024-04-26T19:17:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2522.688773 secs 2024-04-26T19:17:22.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2522.688773 secs 2024-04-26T19:17:22.201 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2522.688773 secs 2024-04-26T19:17:22.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2926 sec 2024-04-26T19:17:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:22 smithi090 ceph-mon[29180]: pgmap v2138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:22 smithi119 ceph-mon[32752]: pgmap v2138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:22 smithi167 ceph-mon[32751]: pgmap v2138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:23.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2927 sec 2024-04-26T19:17:24.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2928 sec 2024-04-26T19:17:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:24 smithi090 ceph-mon[29180]: pgmap v2139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:24 smithi119 ceph-mon[32752]: pgmap v2139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:24 smithi167 ceph-mon[32751]: pgmap v2139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:25.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2929 sec 2024-04-26T19:17:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2527.013498 secs 2024-04-26T19:17:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2527.013498 secs 2024-04-26T19:17:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2527.013498 secs 2024-04-26T19:17:26.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2930 sec 2024-04-26T19:17:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2527.688882 secs 2024-04-26T19:17:26.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:26 smithi090 ceph-mon[29180]: pgmap v2140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2527.688882 secs 2024-04-26T19:17:27.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:26 smithi119 ceph-mon[32752]: pgmap v2140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2527.688882 secs 2024-04-26T19:17:27.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:26 smithi167 ceph-mon[32751]: pgmap v2140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:27.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2931 sec 2024-04-26T19:17:28.143 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2932 sec 2024-04-26T19:17:28.799 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:17:28.800+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:17:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:17:28.800+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 24 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:17:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:17:28.806+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:17:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:17:28.806+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 24 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:17:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:28 smithi090 ceph-mon[29180]: pgmap v2141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:28 smithi119 ceph-mon[32752]: pgmap v2141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:28 smithi167 ceph-mon[32751]: pgmap v2141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:29.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2933 sec 2024-04-26T19:17:30.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2934 sec 2024-04-26T19:17:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:30 smithi090 ceph-mon[29180]: pgmap v2142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:30.910 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:17:30.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2532.013592 secs 2024-04-26T19:17:30.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:17:30.938 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:17:30.947 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:17:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:30 smithi119 ceph-mon[32752]: pgmap v2142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:31.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:17:31.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2532.013592 secs 2024-04-26T19:17:31.179 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:17:31.188 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:17:31.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:17:31.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2935 sec 2024-04-26T19:17:31.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:30 smithi167 ceph-mon[32751]: pgmap v2142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:31.224 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:17:31.233 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2532.013592 secs 2024-04-26T19:17:31.241 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:17:31.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:17:31.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:17:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-mon[29180]: pgmap v2143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:17:32.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-mon[29180]: pgmap v2144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:32.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2532.688968 secs 2024-04-26T19:17:32.522 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:17:31.441+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:17:32.530 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:17:31.441+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:17:32.537 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:17:31.441+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:17:32.612 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:17:31.441+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:17:32.703 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:17:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:17:31.441+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:17:32.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2936 sec 2024-04-26T19:17:32.720 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:31 smithi119 ceph-mon[32752]: pgmap v2143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:17:32.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:31 smithi119 ceph-mon[32752]: pgmap v2144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:32.736 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2532.688968 secs 2024-04-26T19:17:32.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:31 smithi167 ceph-mon[32751]: pgmap v2143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:17:32.752 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:31 smithi167 ceph-mon[32751]: pgmap v2144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:32.760 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2532.688968 secs 2024-04-26T19:17:33.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2937 sec 2024-04-26T19:17:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:33 smithi090 ceph-mon[29180]: pgmap v2145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:17:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:33 smithi119 ceph-mon[32752]: pgmap v2145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:17:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:33 smithi167 ceph-mon[32751]: pgmap v2145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:17:34.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2938 sec 2024-04-26T19:17:35.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2939 sec 2024-04-26T19:17:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:35 smithi090 ceph-mon[29180]: pgmap v2146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:36.049 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2537.013673 secs 2024-04-26T19:17:36.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:35 smithi167 ceph-mon[32751]: pgmap v2146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:36.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2537.013673 secs 2024-04-26T19:17:36.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:35 smithi119 ceph-mon[32752]: pgmap v2146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:36.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2537.013673 secs 2024-04-26T19:17:36.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2940 sec 2024-04-26T19:17:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2537.689076 secs 2024-04-26T19:17:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2537.689076 secs 2024-04-26T19:17:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2537.689076 secs 2024-04-26T19:17:37.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2941 sec 2024-04-26T19:17:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:37 smithi090 ceph-mon[29180]: pgmap v2147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:37 smithi119 ceph-mon[32752]: pgmap v2147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:37 smithi167 ceph-mon[32751]: pgmap v2147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:38.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2942 sec 2024-04-26T19:17:39.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2943 sec 2024-04-26T19:17:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:39 smithi119 ceph-mon[32752]: pgmap v2148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:39 smithi167 ceph-mon[32751]: pgmap v2148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:39 smithi090 ceph-mon[29180]: pgmap v2148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:17:40.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2944 sec 2024-04-26T19:17:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2542.013753 secs 2024-04-26T19:17:40.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2542.013753 secs 2024-04-26T19:17:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2542.013753 secs 2024-04-26T19:17:41.144 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2945 sec 2024-04-26T19:17:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:41 smithi119 ceph-mon[32752]: pgmap v2149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:17:41.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2542.689189 secs 2024-04-26T19:17:41.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:41 smithi167 ceph-mon[32751]: pgmap v2149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:17:41.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2542.689189 secs 2024-04-26T19:17:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:41 smithi090 ceph-mon[29180]: pgmap v2149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:17:41.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2542.689189 secs 2024-04-26T19:17:42.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2946 sec 2024-04-26T19:17:43.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2947 sec 2024-04-26T19:17:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:43 smithi119 ceph-mon[32752]: pgmap v2150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:43 smithi167 ceph-mon[32751]: pgmap v2150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:43 smithi090 ceph-mon[29180]: pgmap v2150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:44.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2948 sec 2024-04-26T19:17:45.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2949 sec 2024-04-26T19:17:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:45 smithi119 ceph-mon[32752]: pgmap v2151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:45.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2547.013836 secs 2024-04-26T19:17:45.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:45 smithi167 ceph-mon[32751]: pgmap v2151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:45.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2547.013836 secs 2024-04-26T19:17:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:45 smithi090 ceph-mon[29180]: pgmap v2151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:45.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2547.013836 secs 2024-04-26T19:17:46.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2950 sec 2024-04-26T19:17:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2547.689319 secs 2024-04-26T19:17:46.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2547.689319 secs 2024-04-26T19:17:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2547.689319 secs 2024-04-26T19:17:47.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2951 sec 2024-04-26T19:17:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:47 smithi119 ceph-mon[32752]: pgmap v2152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:47 smithi167 ceph-mon[32751]: pgmap v2152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:47 smithi090 ceph-mon[29180]: pgmap v2152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:48.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2952 sec 2024-04-26T19:17:49.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2953 sec 2024-04-26T19:17:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:49 smithi119 ceph-mon[32752]: pgmap v2153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:49 smithi167 ceph-mon[32751]: pgmap v2153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:49 smithi090 ceph-mon[29180]: pgmap v2153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:50.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2954 sec 2024-04-26T19:17:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2552.013923 secs 2024-04-26T19:17:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2552.013923 secs 2024-04-26T19:17:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2552.013923 secs 2024-04-26T19:17:51.145 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2955 sec 2024-04-26T19:17:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:51 smithi167 ceph-mon[32751]: pgmap v2154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2552.689389 secs 2024-04-26T19:17:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:51 smithi090 ceph-mon[29180]: pgmap v2154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:51.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2552.689389 secs 2024-04-26T19:17:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:51 smithi119 ceph-mon[32752]: pgmap v2154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:51.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2552.689389 secs 2024-04-26T19:17:52.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2956 sec 2024-04-26T19:17:53.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2957 sec 2024-04-26T19:17:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:53 smithi167 ceph-mon[32751]: pgmap v2155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:53 smithi090 ceph-mon[29180]: pgmap v2155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:53 smithi119 ceph-mon[32752]: pgmap v2155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:54.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2958 sec 2024-04-26T19:17:55.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2959 sec 2024-04-26T19:17:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:55 smithi167 ceph-mon[32751]: pgmap v2156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:55.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2557.014012 secs 2024-04-26T19:17:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:55 smithi090 ceph-mon[29180]: pgmap v2156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:55.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2557.014012 secs 2024-04-26T19:17:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:55 smithi119 ceph-mon[32752]: pgmap v2156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:55.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2557.014012 secs 2024-04-26T19:17:56.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2960 sec 2024-04-26T19:17:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2557.689503 secs 2024-04-26T19:17:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2557.689503 secs 2024-04-26T19:17:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2557.689503 secs 2024-04-26T19:17:57.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2961 sec 2024-04-26T19:17:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:57 smithi090 ceph-mon[29180]: pgmap v2157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:57 smithi119 ceph-mon[32752]: pgmap v2157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:57.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:57 smithi167 ceph-mon[32751]: pgmap v2157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:17:58.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2962 sec 2024-04-26T19:17:59.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2963 sec 2024-04-26T19:17:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:17:59 smithi090 ceph-mon[29180]: pgmap v2158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:17:59 smithi119 ceph-mon[32752]: pgmap v2158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:17:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:17:59 smithi167 ceph-mon[32751]: pgmap v2158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:00.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2964 sec 2024-04-26T19:18:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2562.014110 secs 2024-04-26T19:18:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2562.014110 secs 2024-04-26T19:18:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2562.014110 secs 2024-04-26T19:18:01.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2965 sec 2024-04-26T19:18:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:01 smithi090 ceph-mon[29180]: pgmap v2159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:01.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2562.689625 secs 2024-04-26T19:18:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:01 smithi167 ceph-mon[32751]: pgmap v2159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:01.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2562.689625 secs 2024-04-26T19:18:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:01 smithi119 ceph-mon[32752]: pgmap v2159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:01.831 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2562.689625 secs 2024-04-26T19:18:02.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2966 sec 2024-04-26T19:18:03.146 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2967 sec 2024-04-26T19:18:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:03 smithi090 ceph-mon[29180]: pgmap v2160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:03 smithi119 ceph-mon[32752]: pgmap v2160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:03.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:03 smithi167 ceph-mon[32751]: pgmap v2160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:04.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2968 sec 2024-04-26T19:18:05.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2969 sec 2024-04-26T19:18:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:05 smithi090 ceph-mon[29180]: pgmap v2161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:05.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2567.014193 secs 2024-04-26T19:18:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:05 smithi119 ceph-mon[32752]: pgmap v2161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:05.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2567.014193 secs 2024-04-26T19:18:05.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:05 smithi167 ceph-mon[32751]: pgmap v2161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:05.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2567.014193 secs 2024-04-26T19:18:06.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2970 sec 2024-04-26T19:18:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2567.689766 secs 2024-04-26T19:18:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2567.689766 secs 2024-04-26T19:18:06.859 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2567.689766 secs 2024-04-26T19:18:07.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2971 sec 2024-04-26T19:18:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:07 smithi090 ceph-mon[29180]: pgmap v2162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:07 smithi119 ceph-mon[32752]: pgmap v2162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:07 smithi167 ceph-mon[32751]: pgmap v2162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:08.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2972 sec 2024-04-26T19:18:09.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2973 sec 2024-04-26T19:18:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:09 smithi090 ceph-mon[29180]: pgmap v2163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:09 smithi119 ceph-mon[32752]: pgmap v2163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:09 smithi167 ceph-mon[32751]: pgmap v2163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:10.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2974 sec 2024-04-26T19:18:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2572.014274 secs 2024-04-26T19:18:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2572.014274 secs 2024-04-26T19:18:10.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2572.014274 secs 2024-04-26T19:18:11.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2975 sec 2024-04-26T19:18:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:11 smithi090 ceph-mon[29180]: pgmap v2164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:11.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2572.689888 secs 2024-04-26T19:18:11.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:11 smithi119 ceph-mon[32752]: pgmap v2164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:11.712 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2572.689888 secs 2024-04-26T19:18:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:11 smithi167 ceph-mon[32751]: pgmap v2164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:11.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2572.689888 secs 2024-04-26T19:18:12.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2976 sec 2024-04-26T19:18:13.147 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2977 sec 2024-04-26T19:18:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:13 smithi090 ceph-mon[29180]: pgmap v2165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:13 smithi119 ceph-mon[32752]: pgmap v2165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:13 smithi167 ceph-mon[32751]: pgmap v2165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:14.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2978 sec 2024-04-26T19:18:15.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2979 sec 2024-04-26T19:18:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:15 smithi090 ceph-mon[29180]: pgmap v2166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:15.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2577.014377 secs 2024-04-26T19:18:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:15 smithi119 ceph-mon[32752]: pgmap v2166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:15.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2577.014377 secs 2024-04-26T19:18:15.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:15 smithi167 ceph-mon[32751]: pgmap v2166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:15.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2577.014377 secs 2024-04-26T19:18:16.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2980 sec 2024-04-26T19:18:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2577.690014 secs 2024-04-26T19:18:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2577.690014 secs 2024-04-26T19:18:16.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2577.690014 secs 2024-04-26T19:18:17.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2981 sec 2024-04-26T19:18:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:17 smithi090 ceph-mon[29180]: pgmap v2167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:17 smithi119 ceph-mon[32752]: pgmap v2167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:17 smithi167 ceph-mon[32751]: pgmap v2167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:18.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2982 sec 2024-04-26T19:18:19.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2983 sec 2024-04-26T19:18:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:19 smithi090 ceph-mon[29180]: pgmap v2168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:19 smithi119 ceph-mon[32752]: pgmap v2168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:19.874 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:19 smithi167 ceph-mon[32751]: pgmap v2168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:20.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2984 sec 2024-04-26T19:18:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2582.014460 secs 2024-04-26T19:18:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2582.014460 secs 2024-04-26T19:18:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2582.014460 secs 2024-04-26T19:18:21.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2985 sec 2024-04-26T19:18:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:21 smithi119 ceph-mon[32752]: pgmap v2169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:21.826 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2582.690140 secs 2024-04-26T19:18:21.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:21 smithi167 ceph-mon[32751]: pgmap v2169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:21.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2582.690140 secs 2024-04-26T19:18:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:21 smithi090 ceph-mon[29180]: pgmap v2169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:21.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2582.690140 secs 2024-04-26T19:18:22.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2986 sec 2024-04-26T19:18:23.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2987 sec 2024-04-26T19:18:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:23 smithi119 ceph-mon[32752]: pgmap v2170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:23 smithi167 ceph-mon[32751]: pgmap v2170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:23 smithi090 ceph-mon[29180]: pgmap v2170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:24.148 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2988 sec 2024-04-26T19:18:25.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2989 sec 2024-04-26T19:18:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:25 smithi119 ceph-mon[32752]: pgmap v2171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:25.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2587.014543 secs 2024-04-26T19:18:25.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:25 smithi167 ceph-mon[32751]: pgmap v2171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:25.866 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2587.014543 secs 2024-04-26T19:18:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:25 smithi090 ceph-mon[29180]: pgmap v2171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:25.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2587.014543 secs 2024-04-26T19:18:26.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2990 sec 2024-04-26T19:18:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2587.690261 secs 2024-04-26T19:18:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2587.690261 secs 2024-04-26T19:18:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2587.690261 secs 2024-04-26T19:18:27.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2991 sec 2024-04-26T19:18:27.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:27 smithi090 ceph-mon[29180]: pgmap v2172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:27 smithi167 ceph-mon[32751]: pgmap v2172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:27.864 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:27 smithi119 ceph-mon[32752]: pgmap v2172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:28.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2992 sec 2024-04-26T19:18:29.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2993 sec 2024-04-26T19:18:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:29 smithi119 ceph-mon[32752]: pgmap v2173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:29 smithi167 ceph-mon[32751]: pgmap v2173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:29 smithi090 ceph-mon[29180]: pgmap v2173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:30.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2994 sec 2024-04-26T19:18:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2592.014627 secs 2024-04-26T19:18:30.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:18:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2592.014627 secs 2024-04-26T19:18:30.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:18:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2592.014627 secs 2024-04-26T19:18:30.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:18:31.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2995 sec 2024-04-26T19:18:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:31 smithi167 ceph-mon[32751]: pgmap v2174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:31.781 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2592.690336 secs 2024-04-26T19:18:31.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:18:31.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:18:31.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:18:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:31 smithi119 ceph-mon[32752]: pgmap v2174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:31.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2592.690336 secs 2024-04-26T19:18:31.850 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:18:31.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:18:31.866 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:18:31.939 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:31 smithi090 ceph-mon[29180]: pgmap v2174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:31.997 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2592.690336 secs 2024-04-26T19:18:32.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:18:32.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:18:32.022 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:18:32.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2996 sec 2024-04-26T19:18:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:32 smithi119 ceph-mon[32752]: pgmap v2175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:18:32.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:32 smithi119 ceph-mon[32752]: pgmap v2176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:18:32.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:32 smithi090 ceph-mon[29180]: pgmap v2175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:18:32.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:32 smithi090 ceph-mon[29180]: pgmap v2176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:18:32.910 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:32 smithi167 ceph-mon[32751]: pgmap v2175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:18:32.920 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:32 smithi167 ceph-mon[32751]: pgmap v2176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:18:33.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2997 sec 2024-04-26T19:18:34.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2998 sec 2024-04-26T19:18:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:34 smithi119 ceph-mon[32752]: pgmap v2177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:18:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:34 smithi167 ceph-mon[32751]: pgmap v2177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:18:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:34 smithi090 ceph-mon[29180]: pgmap v2177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:18:35.149 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 2999 sec 2024-04-26T19:18:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2597.014711 secs 2024-04-26T19:18:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2597.014711 secs 2024-04-26T19:18:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2597.014711 secs 2024-04-26T19:18:36.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3000 sec 2024-04-26T19:18:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:36 smithi119 ceph-mon[32752]: pgmap v2178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:18:36.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2597.690476 secs 2024-04-26T19:18:36.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:36 smithi167 ceph-mon[32751]: pgmap v2178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:18:36.866 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2597.690476 secs 2024-04-26T19:18:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:36 smithi090 ceph-mon[29180]: pgmap v2178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:18:36.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2597.690476 secs 2024-04-26T19:18:37.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3001 sec 2024-04-26T19:18:38.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3002 sec 2024-04-26T19:18:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:38 smithi119 ceph-mon[32752]: pgmap v2179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:18:38.861 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:38 smithi167 ceph-mon[32751]: pgmap v2179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:18:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:38 smithi090 ceph-mon[29180]: pgmap v2179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:18:39.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3003 sec 2024-04-26T19:18:40.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3004 sec 2024-04-26T19:18:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:40 smithi119 ceph-mon[32752]: pgmap v2180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:18:40.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2602.014814 secs 2024-04-26T19:18:40.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:40 smithi167 ceph-mon[32751]: pgmap v2180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:18:40.866 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2602.014814 secs 2024-04-26T19:18:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:40 smithi090 ceph-mon[29180]: pgmap v2180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:18:40.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2602.014814 secs 2024-04-26T19:18:41.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3005 sec 2024-04-26T19:18:41.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2602.690604 secs 2024-04-26T19:18:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2602.690604 secs 2024-04-26T19:18:41.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2602.690604 secs 2024-04-26T19:18:42.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3006 sec 2024-04-26T19:18:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:42 smithi119 ceph-mon[32752]: pgmap v2181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T19:18:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:42 smithi167 ceph-mon[32751]: pgmap v2181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T19:18:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:42 smithi090 ceph-mon[29180]: pgmap v2181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T19:18:43.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3007 sec 2024-04-26T19:18:44.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3008 sec 2024-04-26T19:18:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:44 smithi119 ceph-mon[32752]: pgmap v2182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:18:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:44 smithi167 ceph-mon[32751]: pgmap v2182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:18:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:44 smithi090 ceph-mon[29180]: pgmap v2182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:18:45.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3009 sec 2024-04-26T19:18:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2607.014874 secs 2024-04-26T19:18:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2607.014874 secs 2024-04-26T19:18:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2607.014874 secs 2024-04-26T19:18:46.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3010 sec 2024-04-26T19:18:46.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:46 smithi090 ceph-mon[29180]: pgmap v2183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:46.798 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2607.690695 secs 2024-04-26T19:18:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:46 smithi119 ceph-mon[32752]: pgmap v2183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:46.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2607.690695 secs 2024-04-26T19:18:46.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:46 smithi167 ceph-mon[32751]: pgmap v2183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:46.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2607.690695 secs 2024-04-26T19:18:47.150 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3011 sec 2024-04-26T19:18:48.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3012 sec 2024-04-26T19:18:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:48 smithi119 ceph-mon[32752]: pgmap v2184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:18:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:48 smithi167 ceph-mon[32751]: pgmap v2184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:18:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:48 smithi090 ceph-mon[29180]: pgmap v2184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:18:49.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3013 sec 2024-04-26T19:18:50.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3014 sec 2024-04-26T19:18:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:50 smithi119 ceph-mon[32752]: pgmap v2185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:50.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2612.014954 secs 2024-04-26T19:18:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:50 smithi167 ceph-mon[32751]: pgmap v2185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:50.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2612.014954 secs 2024-04-26T19:18:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:50 smithi090 ceph-mon[29180]: pgmap v2185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:50.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2612.014954 secs 2024-04-26T19:18:51.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3015 sec 2024-04-26T19:18:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2612.690812 secs 2024-04-26T19:18:51.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2612.690812 secs 2024-04-26T19:18:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2612.690812 secs 2024-04-26T19:18:52.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3016 sec 2024-04-26T19:18:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:52 smithi119 ceph-mon[32752]: pgmap v2186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:52 smithi167 ceph-mon[32751]: pgmap v2186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:52 smithi090 ceph-mon[29180]: pgmap v2186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:53.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3017 sec 2024-04-26T19:18:54.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3018 sec 2024-04-26T19:18:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:54 smithi119 ceph-mon[32752]: pgmap v2187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:54 smithi167 ceph-mon[32751]: pgmap v2187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:54 smithi090 ceph-mon[29180]: pgmap v2187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:55.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3019 sec 2024-04-26T19:18:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2617.015017 secs 2024-04-26T19:18:55.859 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2617.015017 secs 2024-04-26T19:18:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2617.015017 secs 2024-04-26T19:18:56.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3020 sec 2024-04-26T19:18:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:56 smithi119 ceph-mon[32752]: pgmap v2188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:56.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2617.690943 secs 2024-04-26T19:18:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:56 smithi167 ceph-mon[32751]: pgmap v2188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:56.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2617.690943 secs 2024-04-26T19:18:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:56 smithi090 ceph-mon[29180]: pgmap v2188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:18:56.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2617.690943 secs 2024-04-26T19:18:57.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3021 sec 2024-04-26T19:18:58.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3022 sec 2024-04-26T19:18:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:18:58 smithi119 ceph-mon[32752]: pgmap v2189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:18:58 smithi167 ceph-mon[32751]: pgmap v2189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:18:58 smithi090 ceph-mon[29180]: pgmap v2189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:18:59.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3023 sec 2024-04-26T19:19:00.151 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3024 sec 2024-04-26T19:19:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:00 smithi119 ceph-mon[32752]: pgmap v2190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:00.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2622.015095 secs 2024-04-26T19:19:00.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:00 smithi167 ceph-mon[32751]: pgmap v2190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:00.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2622.015095 secs 2024-04-26T19:19:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:00 smithi090 ceph-mon[29180]: pgmap v2190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:00.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2622.015095 secs 2024-04-26T19:19:01.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3025 sec 2024-04-26T19:19:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2622.691057 secs 2024-04-26T19:19:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2622.691057 secs 2024-04-26T19:19:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2622.691057 secs 2024-04-26T19:19:02.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3026 sec 2024-04-26T19:19:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:02 smithi119 ceph-mon[32752]: pgmap v2191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:02 smithi167 ceph-mon[32751]: pgmap v2191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:02 smithi090 ceph-mon[29180]: pgmap v2191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:03.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3027 sec 2024-04-26T19:19:04.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3028 sec 2024-04-26T19:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:04 smithi119 ceph-mon[32752]: pgmap v2192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:04 smithi167 ceph-mon[32751]: pgmap v2192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:04 smithi090 ceph-mon[29180]: pgmap v2192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:05.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3029 sec 2024-04-26T19:19:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2627.015158 secs 2024-04-26T19:19:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2627.015158 secs 2024-04-26T19:19:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2627.015158 secs 2024-04-26T19:19:06.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3030 sec 2024-04-26T19:19:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:06 smithi119 ceph-mon[32752]: pgmap v2193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:06.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2627.691167 secs 2024-04-26T19:19:06.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:06 smithi167 ceph-mon[32751]: pgmap v2193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:06.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2627.691167 secs 2024-04-26T19:19:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:06 smithi090 ceph-mon[29180]: pgmap v2193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:06.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2627.691167 secs 2024-04-26T19:19:07.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3031 sec 2024-04-26T19:19:08.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3032 sec 2024-04-26T19:19:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:08 smithi119 ceph-mon[32752]: pgmap v2194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:08 smithi167 ceph-mon[32751]: pgmap v2194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:08 smithi090 ceph-mon[29180]: pgmap v2194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:09.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3033 sec 2024-04-26T19:19:10.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3034 sec 2024-04-26T19:19:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:10 smithi119 ceph-mon[32752]: pgmap v2195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:10.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2632.015239 secs 2024-04-26T19:19:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:10 smithi167 ceph-mon[32751]: pgmap v2195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:10.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2632.015239 secs 2024-04-26T19:19:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:10 smithi090 ceph-mon[29180]: pgmap v2195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:10.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2632.015239 secs 2024-04-26T19:19:11.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3035 sec 2024-04-26T19:19:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2632.691283 secs 2024-04-26T19:19:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2632.691283 secs 2024-04-26T19:19:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2632.691283 secs 2024-04-26T19:19:12.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3036 sec 2024-04-26T19:19:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:12 smithi119 ceph-mon[32752]: pgmap v2196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:12 smithi167 ceph-mon[32751]: pgmap v2196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:12 smithi090 ceph-mon[29180]: pgmap v2196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:13.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3037 sec 2024-04-26T19:19:14.152 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3038 sec 2024-04-26T19:19:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:14 smithi119 ceph-mon[32752]: pgmap v2197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:14 smithi167 ceph-mon[32751]: pgmap v2197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:14 smithi090 ceph-mon[29180]: pgmap v2197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:15.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3039 sec 2024-04-26T19:19:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2637.015324 secs 2024-04-26T19:19:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2637.015324 secs 2024-04-26T19:19:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2637.015324 secs 2024-04-26T19:19:16.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3040 sec 2024-04-26T19:19:16.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:16 smithi090 ceph-mon[29180]: pgmap v2198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:16.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2637.691405 secs 2024-04-26T19:19:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:16 smithi119 ceph-mon[32752]: pgmap v2198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:16.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2637.691405 secs 2024-04-26T19:19:16.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:16 smithi167 ceph-mon[32751]: pgmap v2198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:16.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2637.691405 secs 2024-04-26T19:19:17.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3041 sec 2024-04-26T19:19:18.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3042 sec 2024-04-26T19:19:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:18 smithi119 ceph-mon[32752]: pgmap v2199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:18 smithi167 ceph-mon[32751]: pgmap v2199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:18 smithi090 ceph-mon[29180]: pgmap v2199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:19.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3043 sec 2024-04-26T19:19:20.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3044 sec 2024-04-26T19:19:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:20 smithi119 ceph-mon[32752]: pgmap v2200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:20.834 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2642.015382 secs 2024-04-26T19:19:20.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:20 smithi167 ceph-mon[32751]: pgmap v2200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:20.850 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2642.015382 secs 2024-04-26T19:19:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:20 smithi090 ceph-mon[29180]: pgmap v2200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:20.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2642.015382 secs 2024-04-26T19:19:21.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3045 sec 2024-04-26T19:19:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2642.691485 secs 2024-04-26T19:19:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2642.691485 secs 2024-04-26T19:19:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2642.691485 secs 2024-04-26T19:19:22.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3046 sec 2024-04-26T19:19:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:22 smithi119 ceph-mon[32752]: pgmap v2201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:22 smithi167 ceph-mon[32751]: pgmap v2201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:22 smithi090 ceph-mon[29180]: pgmap v2201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:23.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3047 sec 2024-04-26T19:19:24.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3048 sec 2024-04-26T19:19:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:24 smithi119 ceph-mon[32752]: pgmap v2202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:24 smithi167 ceph-mon[32751]: pgmap v2202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:24 smithi090 ceph-mon[29180]: pgmap v2202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:25.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3049 sec 2024-04-26T19:19:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2647.015465 secs 2024-04-26T19:19:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2647.015465 secs 2024-04-26T19:19:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2647.015465 secs 2024-04-26T19:19:26.153 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3050 sec 2024-04-26T19:19:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:26 smithi119 ceph-mon[32752]: pgmap v2203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:26.841 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2647.691626 secs 2024-04-26T19:19:26.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:26 smithi167 ceph-mon[32751]: pgmap v2203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:26.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2647.691626 secs 2024-04-26T19:19:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:26 smithi090 ceph-mon[29180]: pgmap v2203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:26.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2647.691626 secs 2024-04-26T19:19:27.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3051 sec 2024-04-26T19:19:28.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3052 sec 2024-04-26T19:19:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:28 smithi119 ceph-mon[32752]: pgmap v2204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:28.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:28 smithi167 ceph-mon[32751]: pgmap v2204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:28 smithi090 ceph-mon[29180]: pgmap v2204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:29.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3053 sec 2024-04-26T19:19:30.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3054 sec 2024-04-26T19:19:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:30 smithi119 ceph-mon[32752]: pgmap v2205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:30.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2652.015546 secs 2024-04-26T19:19:30.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:30 smithi167 ceph-mon[32751]: pgmap v2205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:30.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2652.015546 secs 2024-04-26T19:19:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:30 smithi090 ceph-mon[29180]: pgmap v2205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:30.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2652.015546 secs 2024-04-26T19:19:31.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3055 sec 2024-04-26T19:19:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2652.691779 secs 2024-04-26T19:19:31.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:19:31.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2652.691779 secs 2024-04-26T19:19:31.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:19:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2652.691779 secs 2024-04-26T19:19:31.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:19:32.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3056 sec 2024-04-26T19:19:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:32 smithi167 ceph-mon[32751]: pgmap v2206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:32.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:19:32.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:19:32.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:19:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:32 smithi090 ceph-mon[29180]: pgmap v2206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:32.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:19:32.894 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:19:32.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:19:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:32 smithi119 ceph-mon[32752]: pgmap v2206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:33.075 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:19:33.134 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:19:33.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:19:33.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3057 sec 2024-04-26T19:19:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:33 smithi167 ceph-mon[32751]: pgmap v2207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:19:33.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:33 smithi167 ceph-mon[32751]: pgmap v2208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:33 smithi090 ceph-mon[29180]: pgmap v2207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:19:33.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:33 smithi090 ceph-mon[29180]: pgmap v2208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:33 smithi119 ceph-mon[32752]: pgmap v2207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:19:34.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:33 smithi119 ceph-mon[32752]: pgmap v2208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:34.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3058 sec 2024-04-26T19:19:35.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3059 sec 2024-04-26T19:19:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:35 smithi167 ceph-mon[32751]: pgmap v2209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:35.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2657.015629 secs 2024-04-26T19:19:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:35 smithi090 ceph-mon[29180]: pgmap v2209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2657.015629 secs 2024-04-26T19:19:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:35 smithi119 ceph-mon[32752]: pgmap v2209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:36.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2657.015629 secs 2024-04-26T19:19:36.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3060 sec 2024-04-26T19:19:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2657.691882 secs 2024-04-26T19:19:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2657.691882 secs 2024-04-26T19:19:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2657.691882 secs 2024-04-26T19:19:37.154 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3061 sec 2024-04-26T19:19:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:37 smithi167 ceph-mon[32751]: pgmap v2210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:37 smithi090 ceph-mon[29180]: pgmap v2210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:37 smithi119 ceph-mon[32752]: pgmap v2210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:38.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3062 sec 2024-04-26T19:19:39.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3063 sec 2024-04-26T19:19:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:39 smithi167 ceph-mon[32751]: pgmap v2211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:19:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:39 smithi090 ceph-mon[29180]: pgmap v2211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:19:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:39 smithi119 ceph-mon[32752]: pgmap v2211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:19:40.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3064 sec 2024-04-26T19:19:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2662.015711 secs 2024-04-26T19:19:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2662.015711 secs 2024-04-26T19:19:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2662.015711 secs 2024-04-26T19:19:41.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3065 sec 2024-04-26T19:19:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:41 smithi090 ceph-mon[29180]: pgmap v2212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:41.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2662.692001 secs 2024-04-26T19:19:41.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:41 smithi119 ceph-mon[32752]: pgmap v2212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:41.964 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2662.692001 secs 2024-04-26T19:19:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:41 smithi167 ceph-mon[32751]: pgmap v2212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:19:42.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2662.692001 secs 2024-04-26T19:19:42.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3066 sec 2024-04-26T19:19:43.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3067 sec 2024-04-26T19:19:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:43 smithi090 ceph-mon[29180]: pgmap v2213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:19:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:43 smithi119 ceph-mon[32752]: pgmap v2213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:19:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:43 smithi167 ceph-mon[32751]: pgmap v2213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:19:44.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3068 sec 2024-04-26T19:19:45.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3069 sec 2024-04-26T19:19:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:45 smithi090 ceph-mon[29180]: pgmap v2214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:45.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2667.015815 secs 2024-04-26T19:19:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:45 smithi119 ceph-mon[32752]: pgmap v2214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:46.124 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2667.015815 secs 2024-04-26T19:19:46.132 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:45 smithi167 ceph-mon[32751]: pgmap v2214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:46.140 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2667.015815 secs 2024-04-26T19:19:46.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3070 sec 2024-04-26T19:19:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2667.692132 secs 2024-04-26T19:19:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2667.692132 secs 2024-04-26T19:19:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2667.692132 secs 2024-04-26T19:19:47.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3071 sec 2024-04-26T19:19:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:47 smithi090 ceph-mon[29180]: pgmap v2215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:47 smithi119 ceph-mon[32752]: pgmap v2215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:47 smithi167 ceph-mon[32751]: pgmap v2215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:48.155 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3072 sec 2024-04-26T19:19:49.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3073 sec 2024-04-26T19:19:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:49 smithi090 ceph-mon[29180]: pgmap v2216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:49 smithi119 ceph-mon[32752]: pgmap v2216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:49 smithi167 ceph-mon[32751]: pgmap v2216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:50.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3074 sec 2024-04-26T19:19:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2672.015875 secs 2024-04-26T19:19:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2672.015875 secs 2024-04-26T19:19:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2672.015875 secs 2024-04-26T19:19:51.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3075 sec 2024-04-26T19:19:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:51 smithi090 ceph-mon[29180]: pgmap v2217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:51.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2672.692251 secs 2024-04-26T19:19:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:51 smithi119 ceph-mon[32752]: pgmap v2217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:52.133 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2672.692251 secs 2024-04-26T19:19:52.150 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:51 smithi167 ceph-mon[32751]: pgmap v2217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:52.159 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2672.692251 secs 2024-04-26T19:19:52.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3076 sec 2024-04-26T19:19:53.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3077 sec 2024-04-26T19:19:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:53 smithi090 ceph-mon[29180]: pgmap v2218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:53 smithi119 ceph-mon[32752]: pgmap v2218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:53 smithi167 ceph-mon[32751]: pgmap v2218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:54.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3078 sec 2024-04-26T19:19:55.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3079 sec 2024-04-26T19:19:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:55 smithi090 ceph-mon[29180]: pgmap v2219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:55.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2677.015937 secs 2024-04-26T19:19:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:55 smithi119 ceph-mon[32752]: pgmap v2219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:56.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2677.015937 secs 2024-04-26T19:19:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:55 smithi167 ceph-mon[32751]: pgmap v2219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:19:56.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2677.015937 secs 2024-04-26T19:19:56.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3080 sec 2024-04-26T19:19:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2677.692415 secs 2024-04-26T19:19:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2677.692415 secs 2024-04-26T19:19:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2677.692415 secs 2024-04-26T19:19:57.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3081 sec 2024-04-26T19:19:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:57 smithi090 ceph-mon[29180]: pgmap v2220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:57 smithi119 ceph-mon[32752]: pgmap v2220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:57 smithi167 ceph-mon[32751]: pgmap v2220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:19:58.156 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3082 sec 2024-04-26T19:19:59.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3083 sec 2024-04-26T19:19:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:19:59 smithi090 ceph-mon[29180]: pgmap v2221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:19:59 smithi119 ceph-mon[32752]: pgmap v2221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:19:59 smithi167 ceph-mon[32751]: pgmap v2221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:00.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3084 sec 2024-04-26T19:20:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2682.015996 secs 2024-04-26T19:20:00.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:20:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2682.015996 secs 2024-04-26T19:20:01.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:20:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2682.015996 secs 2024-04-26T19:20:01.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:20:01.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3085 sec 2024-04-26T19:20:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:01 smithi090 ceph-mon[29180]: pgmap v2222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:01.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2682.692538 secs 2024-04-26T19:20:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:01 smithi167 ceph-mon[32751]: pgmap v2222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2682.692538 secs 2024-04-26T19:20:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:01 smithi119 ceph-mon[32752]: pgmap v2222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:02.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2682.692538 secs 2024-04-26T19:20:02.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3086 sec 2024-04-26T19:20:03.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3087 sec 2024-04-26T19:20:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:03 smithi119 ceph-mon[32752]: pgmap v2223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:03 smithi167 ceph-mon[32751]: pgmap v2223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:03 smithi090 ceph-mon[29180]: pgmap v2223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:04.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3088 sec 2024-04-26T19:20:05.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3089 sec 2024-04-26T19:20:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:05 smithi119 ceph-mon[32752]: pgmap v2224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:06.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2687.016081 secs 2024-04-26T19:20:06.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:05 smithi167 ceph-mon[32751]: pgmap v2224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:06.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2687.016081 secs 2024-04-26T19:20:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:05 smithi090 ceph-mon[29180]: pgmap v2224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:06.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2687.016081 secs 2024-04-26T19:20:06.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3090 sec 2024-04-26T19:20:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2687.692653 secs 2024-04-26T19:20:07.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2687.692653 secs 2024-04-26T19:20:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2687.692653 secs 2024-04-26T19:20:07.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3091 sec 2024-04-26T19:20:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:07 smithi119 ceph-mon[32752]: pgmap v2225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:07 smithi167 ceph-mon[32751]: pgmap v2225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:07 smithi090 ceph-mon[29180]: pgmap v2225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:08.157 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3092 sec 2024-04-26T19:20:09.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3093 sec 2024-04-26T19:20:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:09 smithi119 ceph-mon[32752]: pgmap v2226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:10.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:09 smithi167 ceph-mon[32751]: pgmap v2226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:09 smithi090 ceph-mon[29180]: pgmap v2226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:10.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3094 sec 2024-04-26T19:20:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2692.016157 secs 2024-04-26T19:20:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2692.016157 secs 2024-04-26T19:20:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2692.016157 secs 2024-04-26T19:20:11.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3095 sec 2024-04-26T19:20:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:11 smithi119 ceph-mon[32752]: pgmap v2227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:11.963 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2692.692773 secs 2024-04-26T19:20:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:11 smithi167 ceph-mon[32751]: pgmap v2227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:12.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2692.692773 secs 2024-04-26T19:20:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:11 smithi090 ceph-mon[29180]: pgmap v2227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:12.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2692.692773 secs 2024-04-26T19:20:12.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3096 sec 2024-04-26T19:20:13.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3097 sec 2024-04-26T19:20:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:13 smithi119 ceph-mon[32752]: pgmap v2228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:14.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:13 smithi167 ceph-mon[32751]: pgmap v2228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:13 smithi090 ceph-mon[29180]: pgmap v2228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:14.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3098 sec 2024-04-26T19:20:15.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3099 sec 2024-04-26T19:20:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:15 smithi119 ceph-mon[32752]: pgmap v2229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:16.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:15 smithi119 ceph-mon[32752]: 8 slow requests, 1 included below; oldest blocked for > 2697.016239 secs 2024-04-26T19:20:16.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:15 smithi119 ceph-mon[32752]: slow request 720.286375 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:20:16.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:15 smithi167 ceph-mon[32751]: pgmap v2229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:16.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:15 smithi167 ceph-mon[32751]: 8 slow requests, 1 included below; oldest blocked for > 2697.016239 secs 2024-04-26T19:20:16.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:15 smithi167 ceph-mon[32751]: slow request 720.286375 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:20:16.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:15 smithi090 ceph-mon[29180]: pgmap v2229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:16.249 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:15 smithi090 ceph-mon[29180]: 8 slow requests, 1 included below; oldest blocked for > 2697.016239 secs 2024-04-26T19:20:16.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:15 smithi090 ceph-mon[29180]: slow request 720.286375 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:20:16.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3100 sec 2024-04-26T19:20:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2697.692909 secs 2024-04-26T19:20:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2697.692909 secs 2024-04-26T19:20:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2697.692909 secs 2024-04-26T19:20:17.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3101 sec 2024-04-26T19:20:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:17 smithi119 ceph-mon[32752]: pgmap v2230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:17 smithi167 ceph-mon[32751]: pgmap v2230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:17 smithi090 ceph-mon[29180]: pgmap v2230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:18.158 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3102 sec 2024-04-26T19:20:19.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3103 sec 2024-04-26T19:20:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:19 smithi119 ceph-mon[32752]: pgmap v2231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:20.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:19 smithi167 ceph-mon[32751]: pgmap v2231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:19 smithi090 ceph-mon[29180]: pgmap v2231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:20.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3104 sec 2024-04-26T19:20:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2702.016324 secs 2024-04-26T19:20:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2702.016324 secs 2024-04-26T19:20:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2702.016324 secs 2024-04-26T19:20:21.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3105 sec 2024-04-26T19:20:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:21 smithi119 ceph-mon[32752]: pgmap v2232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:22.132 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2702.693042 secs 2024-04-26T19:20:22.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:21 smithi090 ceph-mon[29180]: pgmap v2232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:22.191 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2702.693042 secs 2024-04-26T19:20:22.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:21 smithi167 ceph-mon[32751]: pgmap v2232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:22.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2702.693042 secs 2024-04-26T19:20:22.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3106 sec 2024-04-26T19:20:23.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3107 sec 2024-04-26T19:20:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:23 smithi119 ceph-mon[32752]: pgmap v2233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:23 smithi167 ceph-mon[32751]: pgmap v2233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:23 smithi090 ceph-mon[29180]: pgmap v2233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:24.160 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3108 sec 2024-04-26T19:20:25.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3109 sec 2024-04-26T19:20:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:25 smithi119 ceph-mon[32752]: pgmap v2234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:26.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2707.016412 secs 2024-04-26T19:20:26.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:25 smithi167 ceph-mon[32751]: pgmap v2234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:26.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2707.016412 secs 2024-04-26T19:20:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:25 smithi090 ceph-mon[29180]: pgmap v2234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:26.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2707.016412 secs 2024-04-26T19:20:26.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3110 sec 2024-04-26T19:20:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2707.693164 secs 2024-04-26T19:20:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2707.693164 secs 2024-04-26T19:20:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2707.693164 secs 2024-04-26T19:20:27.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3111 sec 2024-04-26T19:20:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:27 smithi119 ceph-mon[32752]: pgmap v2235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:27 smithi167 ceph-mon[32751]: pgmap v2235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:27 smithi090 ceph-mon[29180]: pgmap v2235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:28.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3112 sec 2024-04-26T19:20:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T19:20:28.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T19:20:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b3720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5630289b92e0 tx=0x7f2cc400a2d0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:20:28.609 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b3720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5630289b92e0 tx=0x7f2cc400a2d0 comp rx=0 tx=0).stop 2024-04-26T19:20:28.616 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b3720 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T19:20:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b3720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T19:20:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b3720 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:20:28.638 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 0x7f2cc00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:20:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 0x7f2cc00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:20:28.653 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 0x7f2cc00b15b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010f60 tx=0x7f2cc4038040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:20:28.694 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.490+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000154) for more than 900000000 us, fault. 2024-04-26T19:20:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=1 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T19:20:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:20:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T19:20:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.490+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T19:20:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.823+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:20:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.823+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 25 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:20:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.827+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:20:28.909 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:20:28.827+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 25 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:20:29.159 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3113 sec 2024-04-26T19:20:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:29 smithi119 ceph-mon[32752]: pgmap v2236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:29 smithi167 ceph-mon[32751]: pgmap v2236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:29 smithi090 ceph-mon[29180]: pgmap v2236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:30.160 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3114 sec 2024-04-26T19:20:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2712.016496 secs 2024-04-26T19:20:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2712.016496 secs 2024-04-26T19:20:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2712.016496 secs 2024-04-26T19:20:31.160 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3115 sec 2024-04-26T19:20:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:31 smithi090 ceph-mon[29180]: pgmap v2237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:31.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2712.693277 secs 2024-04-26T19:20:31.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:31 smithi167 ceph-mon[32751]: pgmap v2237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:32.018 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2712.693277 secs 2024-04-26T19:20:32.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:31 smithi119 ceph-mon[32752]: pgmap v2237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:32.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2712.693277 secs 2024-04-26T19:20:32.160 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3116 sec 2024-04-26T19:20:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:20:33.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:20:33.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:20:33.080 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:20:33.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:20:33.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:20:33.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:20:33.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:20:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:20:33.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:20:33.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:20:33.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:20:33.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3117 sec 2024-04-26T19:20:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:33 smithi119 ceph-mon[32752]: pgmap v2238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:34.075 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:33 smithi119 ceph-mon[32752]: pgmap v2239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:20:34.083 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:33 smithi119 ceph-mon[32752]: pgmap v2240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:20:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:33 smithi090 ceph-mon[29180]: pgmap v2238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:34.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:33 smithi090 ceph-mon[29180]: pgmap v2239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:20:34.259 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:33 smithi090 ceph-mon[29180]: pgmap v2240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:20:34.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:33 smithi167 ceph-mon[32751]: pgmap v2238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:34.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:33 smithi167 ceph-mon[32751]: pgmap v2239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:20:34.282 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:33 smithi167 ceph-mon[32751]: pgmap v2240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:20:34.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3118 sec 2024-04-26T19:20:35.160 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3119 sec 2024-04-26T19:20:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:35 smithi119 ceph-mon[32752]: pgmap v2241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:36.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2717.016581 secs 2024-04-26T19:20:36.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:35 smithi167 ceph-mon[32751]: pgmap v2241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:36.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2717.016581 secs 2024-04-26T19:20:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:35 smithi090 ceph-mon[29180]: pgmap v2241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:36.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2717.016581 secs 2024-04-26T19:20:36.160 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3120 sec 2024-04-26T19:20:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 2717.693426 secs 2024-04-26T19:20:37.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:36 smithi119 ceph-mon[32752]: slow request 1444.330290 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:20:37.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 2717.693426 secs 2024-04-26T19:20:37.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:36 smithi167 ceph-mon[32751]: slow request 1444.330290 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:20:37.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 2717.693426 secs 2024-04-26T19:20:37.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:36 smithi090 ceph-mon[29180]: slow request 1444.330290 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:20:37.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3121 sec 2024-04-26T19:20:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:37 smithi119 ceph-mon[32752]: pgmap v2242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:37 smithi167 ceph-mon[32751]: pgmap v2242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:37 smithi090 ceph-mon[29180]: pgmap v2242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:38.160 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3122 sec 2024-04-26T19:20:39.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3123 sec 2024-04-26T19:20:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:39 smithi167 ceph-mon[32751]: pgmap v2243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:39 smithi090 ceph-mon[29180]: pgmap v2243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:39 smithi119 ceph-mon[32752]: pgmap v2243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:40.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3124 sec 2024-04-26T19:20:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2722.016665 secs 2024-04-26T19:20:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2722.016665 secs 2024-04-26T19:20:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2722.016665 secs 2024-04-26T19:20:41.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3125 sec 2024-04-26T19:20:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2722.693561 secs 2024-04-26T19:20:41.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:41 smithi090 ceph-mon[29180]: pgmap v2244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:41.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2722.693561 secs 2024-04-26T19:20:41.713 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:41 smithi119 ceph-mon[32752]: pgmap v2244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2722.693561 secs 2024-04-26T19:20:41.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:41 smithi167 ceph-mon[32751]: pgmap v2244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:20:42.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3126 sec 2024-04-26T19:20:43.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3127 sec 2024-04-26T19:20:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:43 smithi119 ceph-mon[32752]: pgmap v2245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:20:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:43 smithi167 ceph-mon[32751]: pgmap v2245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:20:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:43 smithi090 ceph-mon[29180]: pgmap v2245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:20:44.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3128 sec 2024-04-26T19:20:45.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3129 sec 2024-04-26T19:20:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:45 smithi119 ceph-mon[32752]: pgmap v2246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:46.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2727.016750 secs 2024-04-26T19:20:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:45 smithi167 ceph-mon[32751]: pgmap v2246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:46.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2727.016750 secs 2024-04-26T19:20:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:45 smithi090 ceph-mon[29180]: pgmap v2246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:46.193 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2727.016750 secs 2024-04-26T19:20:46.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3130 sec 2024-04-26T19:20:47.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2727.693668 secs 2024-04-26T19:20:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2727.693668 secs 2024-04-26T19:20:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2727.693668 secs 2024-04-26T19:20:47.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3131 sec 2024-04-26T19:20:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:47 smithi119 ceph-mon[32752]: pgmap v2247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:47 smithi167 ceph-mon[32751]: pgmap v2247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:47 smithi090 ceph-mon[29180]: pgmap v2247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:48.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3132 sec 2024-04-26T19:20:49.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3133 sec 2024-04-26T19:20:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:49 smithi119 ceph-mon[32752]: pgmap v2248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:49 smithi167 ceph-mon[32751]: pgmap v2248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:49 smithi090 ceph-mon[29180]: pgmap v2248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:50.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3134 sec 2024-04-26T19:20:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2732.016838 secs 2024-04-26T19:20:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2732.016838 secs 2024-04-26T19:20:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2732.016838 secs 2024-04-26T19:20:51.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3135 sec 2024-04-26T19:20:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2732.693766 secs 2024-04-26T19:20:52.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:51 smithi119 ceph-mon[32752]: pgmap v2249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:52.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2732.693766 secs 2024-04-26T19:20:52.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:51 smithi167 ceph-mon[32751]: pgmap v2249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2732.693766 secs 2024-04-26T19:20:52.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:51 smithi090 ceph-mon[29180]: pgmap v2249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:52.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3136 sec 2024-04-26T19:20:53.161 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3137 sec 2024-04-26T19:20:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:53 smithi119 ceph-mon[32752]: pgmap v2250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:53 smithi167 ceph-mon[32751]: pgmap v2250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:53 smithi090 ceph-mon[29180]: pgmap v2250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:54.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3138 sec 2024-04-26T19:20:55.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3139 sec 2024-04-26T19:20:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:55 smithi119 ceph-mon[32752]: pgmap v2251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:56.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2737.016924 secs 2024-04-26T19:20:56.096 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:55 smithi167 ceph-mon[32751]: pgmap v2251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:56.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2737.016924 secs 2024-04-26T19:20:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:55 smithi090 ceph-mon[29180]: pgmap v2251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:20:56.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2737.016924 secs 2024-04-26T19:20:56.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3140 sec 2024-04-26T19:20:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2737.693914 secs 2024-04-26T19:20:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2737.693914 secs 2024-04-26T19:20:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2737.693914 secs 2024-04-26T19:20:57.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3141 sec 2024-04-26T19:20:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:57 smithi119 ceph-mon[32752]: pgmap v2252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:58.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:57 smithi167 ceph-mon[32751]: pgmap v2252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:57 smithi090 ceph-mon[29180]: pgmap v2252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:20:58.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3142 sec 2024-04-26T19:20:59.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3143 sec 2024-04-26T19:21:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:20:59 smithi119 ceph-mon[32752]: pgmap v2253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:20:59 smithi167 ceph-mon[32751]: pgmap v2253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:20:59 smithi090 ceph-mon[29180]: pgmap v2253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:00.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3144 sec 2024-04-26T19:21:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2742.017011 secs 2024-04-26T19:21:01.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2742.017011 secs 2024-04-26T19:21:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2742.017011 secs 2024-04-26T19:21:01.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3145 sec 2024-04-26T19:21:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2742.694050 secs 2024-04-26T19:21:01.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:01 smithi167 ceph-mon[32751]: pgmap v2254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2742.694050 secs 2024-04-26T19:21:02.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:01 smithi119 ceph-mon[32752]: pgmap v2254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2742.694050 secs 2024-04-26T19:21:02.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:01 smithi090 ceph-mon[29180]: pgmap v2254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:02.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3146 sec 2024-04-26T19:21:03.162 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3147 sec 2024-04-26T19:21:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:03 smithi119 ceph-mon[32752]: pgmap v2255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:04.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:03 smithi167 ceph-mon[32751]: pgmap v2255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:03 smithi090 ceph-mon[29180]: pgmap v2255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:04.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3148 sec 2024-04-26T19:21:05.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3149 sec 2024-04-26T19:21:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:05 smithi119 ceph-mon[32752]: pgmap v2256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:06.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2747.017074 secs 2024-04-26T19:21:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:05 smithi167 ceph-mon[32751]: pgmap v2256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:06.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2747.017074 secs 2024-04-26T19:21:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:05 smithi090 ceph-mon[29180]: pgmap v2256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:06.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2747.017074 secs 2024-04-26T19:21:06.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3150 sec 2024-04-26T19:21:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2747.694164 secs 2024-04-26T19:21:07.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2747.694164 secs 2024-04-26T19:21:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2747.694164 secs 2024-04-26T19:21:07.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3151 sec 2024-04-26T19:21:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:07 smithi119 ceph-mon[32752]: pgmap v2257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:07 smithi167 ceph-mon[32751]: pgmap v2257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:07 smithi090 ceph-mon[29180]: pgmap v2257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:08.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3152 sec 2024-04-26T19:21:09.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3153 sec 2024-04-26T19:21:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:09 smithi119 ceph-mon[32752]: pgmap v2258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:09 smithi167 ceph-mon[32751]: pgmap v2258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:09 smithi090 ceph-mon[29180]: pgmap v2258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:10.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3154 sec 2024-04-26T19:21:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2752.017157 secs 2024-04-26T19:21:11.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2752.017157 secs 2024-04-26T19:21:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2752.017157 secs 2024-04-26T19:21:11.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3155 sec 2024-04-26T19:21:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2752.694282 secs 2024-04-26T19:21:12.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:11 smithi119 ceph-mon[32752]: pgmap v2259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:12.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2752.694282 secs 2024-04-26T19:21:12.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:11 smithi167 ceph-mon[32751]: pgmap v2259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2752.694282 secs 2024-04-26T19:21:12.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:11 smithi090 ceph-mon[29180]: pgmap v2259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:12.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3156 sec 2024-04-26T19:21:13.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3157 sec 2024-04-26T19:21:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:13 smithi119 ceph-mon[32752]: pgmap v2260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:14.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:13 smithi167 ceph-mon[32751]: pgmap v2260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:13 smithi090 ceph-mon[29180]: pgmap v2260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:14.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3158 sec 2024-04-26T19:21:15.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3159 sec 2024-04-26T19:21:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:15 smithi119 ceph-mon[32752]: pgmap v2261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:16.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2757.017219 secs 2024-04-26T19:21:16.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:15 smithi167 ceph-mon[32751]: pgmap v2261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:16.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2757.017219 secs 2024-04-26T19:21:16.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:15 smithi090 ceph-mon[29180]: pgmap v2261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:16.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2757.017219 secs 2024-04-26T19:21:16.163 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3160 sec 2024-04-26T19:21:17.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2757.694389 secs 2024-04-26T19:21:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2757.694389 secs 2024-04-26T19:21:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2757.694389 secs 2024-04-26T19:21:17.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3161 sec 2024-04-26T19:21:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:17 smithi119 ceph-mon[32752]: pgmap v2262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:17 smithi167 ceph-mon[32751]: pgmap v2262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:18.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:17 smithi090 ceph-mon[29180]: pgmap v2262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:18.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3162 sec 2024-04-26T19:21:19.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3163 sec 2024-04-26T19:21:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:19 smithi119 ceph-mon[32752]: pgmap v2263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:20.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:19 smithi167 ceph-mon[32751]: pgmap v2263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:19 smithi090 ceph-mon[29180]: pgmap v2263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:20.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3164 sec 2024-04-26T19:21:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2762.017280 secs 2024-04-26T19:21:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2762.017280 secs 2024-04-26T19:21:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2762.017280 secs 2024-04-26T19:21:21.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3165 sec 2024-04-26T19:21:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2762.694513 secs 2024-04-26T19:21:22.068 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:21 smithi119 ceph-mon[32752]: pgmap v2264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:22.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2762.694513 secs 2024-04-26T19:21:22.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:21 smithi167 ceph-mon[32751]: pgmap v2264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2762.694513 secs 2024-04-26T19:21:22.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:21 smithi090 ceph-mon[29180]: pgmap v2264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:22.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3166 sec 2024-04-26T19:21:23.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3167 sec 2024-04-26T19:21:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:23 smithi119 ceph-mon[32752]: pgmap v2265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:23 smithi167 ceph-mon[32751]: pgmap v2265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:23 smithi090 ceph-mon[29180]: pgmap v2265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:24.164 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3168 sec 2024-04-26T19:21:25.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3169 sec 2024-04-26T19:21:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:25 smithi119 ceph-mon[32752]: pgmap v2266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:26.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2767.017342 secs 2024-04-26T19:21:26.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:25 smithi167 ceph-mon[32751]: pgmap v2266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:26.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2767.017342 secs 2024-04-26T19:21:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:25 smithi090 ceph-mon[29180]: pgmap v2266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2767.017342 secs 2024-04-26T19:21:26.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3170 sec 2024-04-26T19:21:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2767.694621 secs 2024-04-26T19:21:27.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2767.694621 secs 2024-04-26T19:21:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2767.694621 secs 2024-04-26T19:21:27.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3171 sec 2024-04-26T19:21:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:27 smithi119 ceph-mon[32752]: pgmap v2267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:28.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:27 smithi167 ceph-mon[32751]: pgmap v2267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:27 smithi090 ceph-mon[29180]: pgmap v2267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:28.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3172 sec 2024-04-26T19:21:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:28 smithi119 ceph-mon[32752]: pgmap v2268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:29.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:28 smithi090 ceph-mon[29180]: pgmap v2268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:29.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:28 smithi167 ceph-mon[32751]: pgmap v2268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:29.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3173 sec 2024-04-26T19:21:30.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3174 sec 2024-04-26T19:21:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2772.017425 secs 2024-04-26T19:21:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2772.017425 secs 2024-04-26T19:21:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2772.017425 secs 2024-04-26T19:21:31.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3175 sec 2024-04-26T19:21:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2772.694738 secs 2024-04-26T19:21:31.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:31 smithi090 ceph-mon[29180]: pgmap v2269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2772.694738 secs 2024-04-26T19:21:31.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:31 smithi119 ceph-mon[32752]: pgmap v2269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2772.694738 secs 2024-04-26T19:21:31.856 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:31 smithi167 ceph-mon[32751]: pgmap v2269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:32.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3176 sec 2024-04-26T19:21:33.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3177 sec 2024-04-26T19:21:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:33 smithi119 ceph-mon[32752]: pgmap v2270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:34.067 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:21:34.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:21:34.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:21:34.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:21:34.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:33 smithi167 ceph-mon[32751]: pgmap v2270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:34.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:21:34.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:21:34.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:21:34.185 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:21:34.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:33 smithi090 ceph-mon[29180]: pgmap v2270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:34.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:21:34.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:21:34.299 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:21:34.307 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:21:34.314 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3178 sec 2024-04-26T19:21:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:34 smithi119 ceph-mon[32752]: pgmap v2271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:21:35.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:34 smithi119 ceph-mon[32752]: pgmap v2272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:35.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:34 smithi090 ceph-mon[29180]: pgmap v2271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:21:35.141 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:34 smithi090 ceph-mon[29180]: pgmap v2272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:35.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:34 smithi167 ceph-mon[32751]: pgmap v2271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:21:35.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:34 smithi167 ceph-mon[32751]: pgmap v2272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:35.165 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3179 sec 2024-04-26T19:21:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2777.017509 secs 2024-04-26T19:21:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2777.017509 secs 2024-04-26T19:21:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2777.017509 secs 2024-04-26T19:21:36.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3180 sec 2024-04-26T19:21:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:36 smithi119 ceph-mon[32752]: pgmap v2273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:37.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2777.694847 secs 2024-04-26T19:21:37.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:36 smithi167 ceph-mon[32751]: pgmap v2273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:37.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2777.694847 secs 2024-04-26T19:21:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:36 smithi090 ceph-mon[29180]: pgmap v2273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:37.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2777.694847 secs 2024-04-26T19:21:37.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3181 sec 2024-04-26T19:21:38.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3182 sec 2024-04-26T19:21:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:38 smithi119 ceph-mon[32752]: pgmap v2274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:21:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:38 smithi167 ceph-mon[32751]: pgmap v2274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:21:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:38 smithi090 ceph-mon[29180]: pgmap v2274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:21:39.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3183 sec 2024-04-26T19:21:40.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3184 sec 2024-04-26T19:21:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:40 smithi119 ceph-mon[32752]: pgmap v2275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:41.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2782.017592 secs 2024-04-26T19:21:41.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:40 smithi167 ceph-mon[32751]: pgmap v2275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:41.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2782.017592 secs 2024-04-26T19:21:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:40 smithi090 ceph-mon[29180]: pgmap v2275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:41.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2782.017592 secs 2024-04-26T19:21:41.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3185 sec 2024-04-26T19:21:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2782.694987 secs 2024-04-26T19:21:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2782.694987 secs 2024-04-26T19:21:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2782.694987 secs 2024-04-26T19:21:42.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3186 sec 2024-04-26T19:21:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:42 smithi119 ceph-mon[32752]: pgmap v2276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:42 smithi167 ceph-mon[32751]: pgmap v2276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:42 smithi090 ceph-mon[29180]: pgmap v2276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:21:43.166 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3187 sec 2024-04-26T19:21:44.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3188 sec 2024-04-26T19:21:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:44 smithi119 ceph-mon[32752]: pgmap v2277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:21:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:44 smithi167 ceph-mon[32751]: pgmap v2277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:21:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:44 smithi090 ceph-mon[29180]: pgmap v2277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:21:45.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3189 sec 2024-04-26T19:21:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2787.017678 secs 2024-04-26T19:21:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2787.017678 secs 2024-04-26T19:21:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2787.017678 secs 2024-04-26T19:21:46.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3190 sec 2024-04-26T19:21:47.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:46 smithi090 ceph-mon[29180]: pgmap v2278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:47.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2787.695117 secs 2024-04-26T19:21:47.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:46 smithi167 ceph-mon[32751]: pgmap v2278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:47.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2787.695117 secs 2024-04-26T19:21:47.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:46 smithi119 ceph-mon[32752]: pgmap v2278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:47.102 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2787.695117 secs 2024-04-26T19:21:47.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3191 sec 2024-04-26T19:21:48.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3192 sec 2024-04-26T19:21:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:48 smithi119 ceph-mon[32752]: pgmap v2279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:49.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:48 smithi167 ceph-mon[32751]: pgmap v2279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:48 smithi090 ceph-mon[29180]: pgmap v2279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:49.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3193 sec 2024-04-26T19:21:50.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3194 sec 2024-04-26T19:21:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:50 smithi119 ceph-mon[32752]: pgmap v2280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2792.017762 secs 2024-04-26T19:21:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:50 smithi167 ceph-mon[32751]: pgmap v2280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:51.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2792.017762 secs 2024-04-26T19:21:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:50 smithi090 ceph-mon[29180]: pgmap v2280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:51.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2792.017762 secs 2024-04-26T19:21:51.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3195 sec 2024-04-26T19:21:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2792.695269 secs 2024-04-26T19:21:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2792.695269 secs 2024-04-26T19:21:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2792.695269 secs 2024-04-26T19:21:52.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3196 sec 2024-04-26T19:21:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:52 smithi119 ceph-mon[32752]: pgmap v2281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:53.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:52 smithi167 ceph-mon[32751]: pgmap v2281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:52 smithi090 ceph-mon[29180]: pgmap v2281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:53.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3197 sec 2024-04-26T19:21:54.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3198 sec 2024-04-26T19:21:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:54 smithi119 ceph-mon[32752]: pgmap v2282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:55.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:54 smithi167 ceph-mon[32751]: pgmap v2282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:55.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:54 smithi090 ceph-mon[29180]: pgmap v2282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:21:55.167 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3199 sec 2024-04-26T19:21:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2797.017852 secs 2024-04-26T19:21:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2797.017852 secs 2024-04-26T19:21:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2797.017852 secs 2024-04-26T19:21:56.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3200 sec 2024-04-26T19:21:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:56 smithi119 ceph-mon[32752]: pgmap v2283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:57.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2797.695417 secs 2024-04-26T19:21:57.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:56 smithi167 ceph-mon[32751]: pgmap v2283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:57.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2797.695417 secs 2024-04-26T19:21:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:56 smithi090 ceph-mon[29180]: pgmap v2283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:57.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2797.695417 secs 2024-04-26T19:21:57.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3201 sec 2024-04-26T19:21:58.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3202 sec 2024-04-26T19:21:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:21:58 smithi119 ceph-mon[32752]: pgmap v2284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:59.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:21:58 smithi167 ceph-mon[32751]: pgmap v2284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:21:58 smithi090 ceph-mon[29180]: pgmap v2284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:21:59.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3203 sec 2024-04-26T19:22:00.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3204 sec 2024-04-26T19:22:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:00 smithi119 ceph-mon[32752]: pgmap v2285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:01.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2802.017913 secs 2024-04-26T19:22:01.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:00 smithi167 ceph-mon[32751]: pgmap v2285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:01.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2802.017913 secs 2024-04-26T19:22:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:00 smithi090 ceph-mon[29180]: pgmap v2285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:01.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2802.017913 secs 2024-04-26T19:22:01.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3205 sec 2024-04-26T19:22:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2802.695478 secs 2024-04-26T19:22:02.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2802.695478 secs 2024-04-26T19:22:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2802.695478 secs 2024-04-26T19:22:02.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3206 sec 2024-04-26T19:22:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:02 smithi119 ceph-mon[32752]: pgmap v2286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:02 smithi167 ceph-mon[32751]: pgmap v2286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:02 smithi090 ceph-mon[29180]: pgmap v2286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:03.168 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3207 sec 2024-04-26T19:22:04.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3208 sec 2024-04-26T19:22:04.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:04 smithi090 ceph-mon[29180]: pgmap v2287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:04 smithi119 ceph-mon[32752]: pgmap v2287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:05.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:04 smithi167 ceph-mon[32751]: pgmap v2287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:05.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3209 sec 2024-04-26T19:22:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2807.018000 secs 2024-04-26T19:22:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2807.018000 secs 2024-04-26T19:22:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2807.018000 secs 2024-04-26T19:22:06.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3210 sec 2024-04-26T19:22:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:06 smithi119 ceph-mon[32752]: pgmap v2288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:07.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2807.695596 secs 2024-04-26T19:22:07.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:06 smithi167 ceph-mon[32751]: pgmap v2288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:07.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2807.695596 secs 2024-04-26T19:22:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:06 smithi090 ceph-mon[29180]: pgmap v2288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:07.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2807.695596 secs 2024-04-26T19:22:07.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3211 sec 2024-04-26T19:22:08.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3212 sec 2024-04-26T19:22:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:08 smithi119 ceph-mon[32752]: pgmap v2289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:08 smithi167 ceph-mon[32751]: pgmap v2289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:08 smithi090 ceph-mon[29180]: pgmap v2289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:09.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3213 sec 2024-04-26T19:22:10.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3214 sec 2024-04-26T19:22:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:10 smithi119 ceph-mon[32752]: pgmap v2290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:11.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2812.018055 secs 2024-04-26T19:22:11.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:10 smithi090 ceph-mon[29180]: pgmap v2290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:11.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2812.018055 secs 2024-04-26T19:22:11.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:10 smithi167 ceph-mon[32751]: pgmap v2290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:11.195 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2812.018055 secs 2024-04-26T19:22:11.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3215 sec 2024-04-26T19:22:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2812.695714 secs 2024-04-26T19:22:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2812.695714 secs 2024-04-26T19:22:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2812.695714 secs 2024-04-26T19:22:12.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3216 sec 2024-04-26T19:22:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:12 smithi119 ceph-mon[32752]: pgmap v2291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:12 smithi167 ceph-mon[32751]: pgmap v2291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:12 smithi090 ceph-mon[29180]: pgmap v2291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:13.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3217 sec 2024-04-26T19:22:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:13 smithi119 ceph-mon[32752]: pgmap v2292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:13 smithi167 ceph-mon[32751]: pgmap v2292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:13 smithi090 ceph-mon[29180]: pgmap v2292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:14.169 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3218 sec 2024-04-26T19:22:15.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3219 sec 2024-04-26T19:22:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2817.018140 secs 2024-04-26T19:22:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2817.018140 secs 2024-04-26T19:22:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2817.018140 secs 2024-04-26T19:22:16.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3220 sec 2024-04-26T19:22:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:16 smithi090 ceph-mon[29180]: pgmap v2293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:16.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2817.695846 secs 2024-04-26T19:22:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:16 smithi119 ceph-mon[32752]: pgmap v2293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:16.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2817.695846 secs 2024-04-26T19:22:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:16 smithi167 ceph-mon[32751]: pgmap v2293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:16.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2817.695846 secs 2024-04-26T19:22:17.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3221 sec 2024-04-26T19:22:18.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3222 sec 2024-04-26T19:22:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:18 smithi119 ceph-mon[32752]: pgmap v2294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:18.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:18 smithi167 ceph-mon[32751]: pgmap v2294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:18 smithi090 ceph-mon[29180]: pgmap v2294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:19.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3223 sec 2024-04-26T19:22:20.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3224 sec 2024-04-26T19:22:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:20 smithi119 ceph-mon[32752]: pgmap v2295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:20.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2822.018222 secs 2024-04-26T19:22:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:20 smithi167 ceph-mon[32751]: pgmap v2295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:20.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2822.018222 secs 2024-04-26T19:22:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:20 smithi090 ceph-mon[29180]: pgmap v2295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:20.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2822.018222 secs 2024-04-26T19:22:21.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3225 sec 2024-04-26T19:22:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2822.695982 secs 2024-04-26T19:22:21.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2822.695982 secs 2024-04-26T19:22:21.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2822.695982 secs 2024-04-26T19:22:22.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3226 sec 2024-04-26T19:22:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:22 smithi119 ceph-mon[32752]: pgmap v2296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:22 smithi167 ceph-mon[32751]: pgmap v2296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:22 smithi090 ceph-mon[29180]: pgmap v2296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:23.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3227 sec 2024-04-26T19:22:24.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3228 sec 2024-04-26T19:22:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:24 smithi119 ceph-mon[32752]: pgmap v2297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:24.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:24 smithi167 ceph-mon[32751]: pgmap v2297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:24 smithi090 ceph-mon[29180]: pgmap v2297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:25.170 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3229 sec 2024-04-26T19:22:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2827.018284 secs 2024-04-26T19:22:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2827.018284 secs 2024-04-26T19:22:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2827.018284 secs 2024-04-26T19:22:26.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3230 sec 2024-04-26T19:22:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:26 smithi119 ceph-mon[32752]: pgmap v2298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:26.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2827.696097 secs 2024-04-26T19:22:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:26 smithi167 ceph-mon[32751]: pgmap v2298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:26.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2827.696097 secs 2024-04-26T19:22:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:26 smithi090 ceph-mon[29180]: pgmap v2298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:26.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2827.696097 secs 2024-04-26T19:22:27.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3231 sec 2024-04-26T19:22:28.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3232 sec 2024-04-26T19:22:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:28 smithi119 ceph-mon[32752]: pgmap v2299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:28 smithi167 ceph-mon[32751]: pgmap v2299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:28 smithi090 ceph-mon[29180]: pgmap v2299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:29.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3233 sec 2024-04-26T19:22:30.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3234 sec 2024-04-26T19:22:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:30 smithi119 ceph-mon[32752]: pgmap v2300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:30.835 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2832.018367 secs 2024-04-26T19:22:30.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:30 smithi167 ceph-mon[32751]: pgmap v2300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:30.855 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2832.018367 secs 2024-04-26T19:22:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:30 smithi090 ceph-mon[29180]: pgmap v2300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:30.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2832.018367 secs 2024-04-26T19:22:31.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3235 sec 2024-04-26T19:22:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2832.696214 secs 2024-04-26T19:22:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2832.696214 secs 2024-04-26T19:22:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2832.696214 secs 2024-04-26T19:22:32.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3236 sec 2024-04-26T19:22:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:32 smithi119 ceph-mon[32752]: pgmap v2301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:32 smithi167 ceph-mon[32751]: pgmap v2301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:32 smithi090 ceph-mon[29180]: pgmap v2301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:33.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3237 sec 2024-04-26T19:22:34.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3238 sec 2024-04-26T19:22:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:34 smithi119 ceph-mon[32752]: pgmap v2302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:34.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:22:34.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:22:34.828 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:22:34.835 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:22:34.843 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:34 smithi167 ceph-mon[32751]: pgmap v2302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:34.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:22:34.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:22:34.866 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:22:34.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:22:34.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:34 smithi090 ceph-mon[29180]: pgmap v2302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:34.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:22:34.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:22:34.942 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:22:34.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:22:35.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3239 sec 2024-04-26T19:22:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:35 smithi119 ceph-mon[32752]: pgmap v2303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:35.863 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2837.018452 secs 2024-04-26T19:22:35.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:35 smithi167 ceph-mon[32751]: pgmap v2303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:35.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2837.018452 secs 2024-04-26T19:22:35.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:35 smithi090 ceph-mon[29180]: pgmap v2303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:35.894 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2837.018452 secs 2024-04-26T19:22:36.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3240 sec 2024-04-26T19:22:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2837.696327 secs 2024-04-26T19:22:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2837.696327 secs 2024-04-26T19:22:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2837.696327 secs 2024-04-26T19:22:37.171 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3241 sec 2024-04-26T19:22:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:37 smithi119 ceph-mon[32752]: pgmap v2304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:37 smithi167 ceph-mon[32751]: pgmap v2304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:37 smithi090 ceph-mon[29180]: pgmap v2304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:38.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3242 sec 2024-04-26T19:22:39.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3243 sec 2024-04-26T19:22:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:39 smithi119 ceph-mon[32752]: pgmap v2305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:39 smithi167 ceph-mon[32751]: pgmap v2305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:39 smithi090 ceph-mon[29180]: pgmap v2305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:40.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3244 sec 2024-04-26T19:22:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2842.018531 secs 2024-04-26T19:22:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2842.018531 secs 2024-04-26T19:22:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2842.018531 secs 2024-04-26T19:22:41.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3245 sec 2024-04-26T19:22:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:41 smithi119 ceph-mon[32752]: pgmap v2306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:41.896 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2842.696410 secs 2024-04-26T19:22:41.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:41 smithi090 ceph-mon[29180]: pgmap v2306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:41.913 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2842.696410 secs 2024-04-26T19:22:41.921 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:41 smithi167 ceph-mon[32751]: pgmap v2306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:41.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2842.696410 secs 2024-04-26T19:22:42.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3246 sec 2024-04-26T19:22:43.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3247 sec 2024-04-26T19:22:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:43 smithi119 ceph-mon[32752]: pgmap v2307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:43 smithi167 ceph-mon[32751]: pgmap v2307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:43 smithi090 ceph-mon[29180]: pgmap v2307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:44.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3248 sec 2024-04-26T19:22:45.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3249 sec 2024-04-26T19:22:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:45 smithi119 ceph-mon[32752]: pgmap v2308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:45.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2847.018616 secs 2024-04-26T19:22:45.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:45 smithi167 ceph-mon[32751]: pgmap v2308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:45.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2847.018616 secs 2024-04-26T19:22:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:45 smithi090 ceph-mon[29180]: pgmap v2308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:22:45.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2847.018616 secs 2024-04-26T19:22:46.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3250 sec 2024-04-26T19:22:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2847.696519 secs 2024-04-26T19:22:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2847.696519 secs 2024-04-26T19:22:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2847.696519 secs 2024-04-26T19:22:47.172 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3251 sec 2024-04-26T19:22:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:47 smithi119 ceph-mon[32752]: pgmap v2309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:47.894 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:47 smithi090 ceph-mon[29180]: pgmap v2309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:47.903 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:47 smithi167 ceph-mon[32751]: pgmap v2309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:48.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3252 sec 2024-04-26T19:22:49.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3253 sec 2024-04-26T19:22:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:49 smithi119 ceph-mon[32752]: pgmap v2310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:49 smithi167 ceph-mon[32751]: pgmap v2310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:49 smithi090 ceph-mon[29180]: pgmap v2310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:50.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3254 sec 2024-04-26T19:22:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2852.018696 secs 2024-04-26T19:22:50.901 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2852.018696 secs 2024-04-26T19:22:50.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2852.018696 secs 2024-04-26T19:22:51.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3255 sec 2024-04-26T19:22:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:51 smithi119 ceph-mon[32752]: pgmap v2311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:51.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2852.696631 secs 2024-04-26T19:22:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:51 smithi167 ceph-mon[32751]: pgmap v2311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:51.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2852.696631 secs 2024-04-26T19:22:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:51 smithi090 ceph-mon[29180]: pgmap v2311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:51.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2852.696631 secs 2024-04-26T19:22:52.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3256 sec 2024-04-26T19:22:53.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3257 sec 2024-04-26T19:22:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:53 smithi119 ceph-mon[32752]: pgmap v2312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:53.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:53 smithi090 ceph-mon[29180]: pgmap v2312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:53.913 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:53 smithi167 ceph-mon[32751]: pgmap v2312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:54.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3258 sec 2024-04-26T19:22:55.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3259 sec 2024-04-26T19:22:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:55 smithi119 ceph-mon[32752]: pgmap v2313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:55.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2857.018780 secs 2024-04-26T19:22:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:55 smithi167 ceph-mon[32751]: pgmap v2313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:55.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2857.018780 secs 2024-04-26T19:22:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:55 smithi090 ceph-mon[29180]: pgmap v2313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:22:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2857.018780 secs 2024-04-26T19:22:56.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3260 sec 2024-04-26T19:22:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2857.696695 secs 2024-04-26T19:22:56.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2857.696695 secs 2024-04-26T19:22:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2857.696695 secs 2024-04-26T19:22:57.173 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3261 sec 2024-04-26T19:22:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:57 smithi167 ceph-mon[32751]: pgmap v2314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:57 smithi090 ceph-mon[29180]: pgmap v2314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:57 smithi119 ceph-mon[32752]: pgmap v2314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:58.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3262 sec 2024-04-26T19:22:59.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3263 sec 2024-04-26T19:22:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:22:59 smithi167 ceph-mon[32751]: pgmap v2315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:22:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:22:59 smithi090 ceph-mon[29180]: pgmap v2315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:22:59 smithi119 ceph-mon[32752]: pgmap v2315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:00.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3264 sec 2024-04-26T19:23:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2862.018863 secs 2024-04-26T19:23:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2862.018863 secs 2024-04-26T19:23:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2862.018863 secs 2024-04-26T19:23:01.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3265 sec 2024-04-26T19:23:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:01 smithi090 ceph-mon[29180]: pgmap v2316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:01.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2862.696781 secs 2024-04-26T19:23:01.895 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:23:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:23:01.549+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:23:01.901 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:23:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:23:01.549+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:23:01.909 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:23:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:23:01.549+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:23:01.916 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:23:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:23:01.549+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:23:01.924 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:23:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:23:01.549+0000 7f822733d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:23:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:01 smithi167 ceph-mon[32751]: pgmap v2316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2862.696781 secs 2024-04-26T19:23:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:01 smithi119 ceph-mon[32752]: pgmap v2316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:02.115 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2862.696781 secs 2024-04-26T19:23:02.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3266 sec 2024-04-26T19:23:03.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3267 sec 2024-04-26T19:23:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:03 smithi090 ceph-mon[29180]: pgmap v2317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:03 smithi119 ceph-mon[32752]: pgmap v2317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:03 smithi167 ceph-mon[32751]: pgmap v2317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:04.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3268 sec 2024-04-26T19:23:05.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3269 sec 2024-04-26T19:23:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:05 smithi090 ceph-mon[29180]: pgmap v2318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:05.927 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2867.018924 secs 2024-04-26T19:23:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:05 smithi119 ceph-mon[32752]: pgmap v2318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:06.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2867.018924 secs 2024-04-26T19:23:06.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:05 smithi167 ceph-mon[32751]: pgmap v2318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:06.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2867.018924 secs 2024-04-26T19:23:06.174 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3270 sec 2024-04-26T19:23:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2867.696925 secs 2024-04-26T19:23:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2867.696925 secs 2024-04-26T19:23:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2867.696925 secs 2024-04-26T19:23:07.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3271 sec 2024-04-26T19:23:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:07 smithi090 ceph-mon[29180]: pgmap v2319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:07 smithi119 ceph-mon[32752]: pgmap v2319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:07 smithi167 ceph-mon[32751]: pgmap v2319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:08.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3272 sec 2024-04-26T19:23:09.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3273 sec 2024-04-26T19:23:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:09 smithi090 ceph-mon[29180]: pgmap v2320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:09 smithi119 ceph-mon[32752]: pgmap v2320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:09 smithi167 ceph-mon[32751]: pgmap v2320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:10.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3274 sec 2024-04-26T19:23:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2872.018983 secs 2024-04-26T19:23:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2872.018983 secs 2024-04-26T19:23:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2872.018983 secs 2024-04-26T19:23:11.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3275 sec 2024-04-26T19:23:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:11 smithi090 ceph-mon[29180]: pgmap v2321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:11.927 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2872.697073 secs 2024-04-26T19:23:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:11 smithi119 ceph-mon[32752]: pgmap v2321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:11.963 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2872.697073 secs 2024-04-26T19:23:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:11 smithi167 ceph-mon[32751]: pgmap v2321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:12.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2872.697073 secs 2024-04-26T19:23:12.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3276 sec 2024-04-26T19:23:13.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3277 sec 2024-04-26T19:23:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:13 smithi090 ceph-mon[29180]: pgmap v2322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:13 smithi119 ceph-mon[32752]: pgmap v2322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:13 smithi167 ceph-mon[32751]: pgmap v2322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T19:23:14.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3278 sec 2024-04-26T19:23:15.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3279 sec 2024-04-26T19:23:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:15 smithi090 ceph-mon[29180]: pgmap v2323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:15.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2877.019043 secs 2024-04-26T19:23:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:15 smithi119 ceph-mon[32752]: pgmap v2323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:16.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2877.019043 secs 2024-04-26T19:23:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:15 smithi167 ceph-mon[32751]: pgmap v2323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T19:23:16.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2877.019043 secs 2024-04-26T19:23:16.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3280 sec 2024-04-26T19:23:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2877.697205 secs 2024-04-26T19:23:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2877.697205 secs 2024-04-26T19:23:17.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2877.697205 secs 2024-04-26T19:23:17.175 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3281 sec 2024-04-26T19:23:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:17 smithi090 ceph-mon[29180]: pgmap v2324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:17 smithi119 ceph-mon[32752]: pgmap v2324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:17 smithi167 ceph-mon[32751]: pgmap v2324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:18.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3282 sec 2024-04-26T19:23:19.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3283 sec 2024-04-26T19:23:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:19 smithi090 ceph-mon[29180]: pgmap v2325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:19 smithi119 ceph-mon[32752]: pgmap v2325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:19 smithi167 ceph-mon[32751]: pgmap v2325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:20.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3284 sec 2024-04-26T19:23:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:20 smithi090 ceph-mon[29180]: 8 slow requests, 4 included below; oldest blocked for > 2882.019099 secs 2024-04-26T19:23:20.970 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:20 smithi090 ceph-mon[29180]: slow request 2882.019099 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T19:23:20.977 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:20 smithi090 ceph-mon[29180]: slow request 2882.016204 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:20.987 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:20 smithi090 ceph-mon[29180]: slow request 2882.010594 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:20.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:20 smithi090 ceph-mon[29180]: slow request 2880.043868 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T19:23:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:20 smithi119 ceph-mon[32752]: 8 slow requests, 4 included below; oldest blocked for > 2882.019099 secs 2024-04-26T19:23:21.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:20 smithi119 ceph-mon[32752]: slow request 2882.019099 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T19:23:21.072 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:20 smithi119 ceph-mon[32752]: slow request 2882.016204 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:21.081 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:20 smithi119 ceph-mon[32752]: slow request 2882.010594 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:21.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:20 smithi119 ceph-mon[32752]: slow request 2880.043868 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T19:23:21.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:20 smithi167 ceph-mon[32751]: 8 slow requests, 4 included below; oldest blocked for > 2882.019099 secs 2024-04-26T19:23:21.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:20 smithi167 ceph-mon[32751]: slow request 2882.019099 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T19:23:21.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:20 smithi167 ceph-mon[32751]: slow request 2882.016204 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:21.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:20 smithi167 ceph-mon[32751]: slow request 2882.010594 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:21.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:20 smithi167 ceph-mon[32751]: slow request 2880.043868 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T19:23:21.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3285 sec 2024-04-26T19:23:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:21 smithi090 ceph-mon[29180]: pgmap v2326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:21.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:21 smithi090 ceph-mon[29180]: 13 slow requests, 5 included below; oldest blocked for > 2882.697343 secs 2024-04-26T19:23:21.980 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:21 smithi090 ceph-mon[29180]: slow request 2882.697343 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:21.990 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:21 smithi090 ceph-mon[29180]: slow request 2882.695782 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:21.999 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:21 smithi090 ceph-mon[29180]: slow request 2882.694313 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:21 smithi090 ceph-mon[29180]: slow request 2882.693843 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:21 smithi090 ceph-mon[29180]: slow request 2882.681578 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:21 smithi119 ceph-mon[32752]: pgmap v2326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:22.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:21 smithi119 ceph-mon[32752]: 13 slow requests, 5 included below; oldest blocked for > 2882.697343 secs 2024-04-26T19:23:22.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:21 smithi119 ceph-mon[32752]: slow request 2882.697343 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.081 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:21 smithi119 ceph-mon[32752]: slow request 2882.695782 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:21 smithi119 ceph-mon[32752]: slow request 2882.694313 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:21 smithi119 ceph-mon[32752]: slow request 2882.693843 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:21 smithi119 ceph-mon[32752]: slow request 2882.681578 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.118 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:21 smithi167 ceph-mon[32751]: pgmap v2326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:22.152 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:21 smithi167 ceph-mon[32751]: 13 slow requests, 5 included below; oldest blocked for > 2882.697343 secs 2024-04-26T19:23:22.161 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:21 smithi167 ceph-mon[32751]: slow request 2882.697343 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:21 smithi167 ceph-mon[32751]: slow request 2882.695782 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.181 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:21 smithi167 ceph-mon[32751]: slow request 2882.694313 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:21 smithi167 ceph-mon[32751]: slow request 2882.693843 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:21 smithi167 ceph-mon[32751]: slow request 2882.681578 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T19:23:22.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3286 sec 2024-04-26T19:23:23.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3287 sec 2024-04-26T19:23:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:23 smithi090 ceph-mon[29180]: pgmap v2327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:23 smithi119 ceph-mon[32752]: pgmap v2327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:23 smithi167 ceph-mon[32751]: pgmap v2327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:24.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3288 sec 2024-04-26T19:23:25.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3289 sec 2024-04-26T19:23:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:25 smithi090 ceph-mon[29180]: pgmap v2328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:26.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2887.019161 secs 2024-04-26T19:23:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:25 smithi119 ceph-mon[32752]: pgmap v2328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:26.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2887.019161 secs 2024-04-26T19:23:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:25 smithi167 ceph-mon[32751]: pgmap v2328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:26.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2887.019161 secs 2024-04-26T19:23:26.176 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3290 sec 2024-04-26T19:23:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:26 smithi119 ceph-mon[32752]: 13 slow requests, 4 included below; oldest blocked for > 2887.697461 secs 2024-04-26T19:23:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:26 smithi119 ceph-mon[32752]: slow request 2884.112824 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T19:23:27.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:26 smithi119 ceph-mon[32752]: slow request 2884.112488 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T19:23:27.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:26 smithi119 ceph-mon[32752]: slow request 2884.003653 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T19:23:27.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:26 smithi119 ceph-mon[32752]: slow request 2883.999365 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T19:23:27.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:26 smithi167 ceph-mon[32751]: 13 slow requests, 4 included below; oldest blocked for > 2887.697461 secs 2024-04-26T19:23:27.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:26 smithi167 ceph-mon[32751]: slow request 2884.112824 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T19:23:27.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:26 smithi167 ceph-mon[32751]: slow request 2884.112488 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T19:23:27.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:26 smithi167 ceph-mon[32751]: slow request 2884.003653 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T19:23:27.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:26 smithi167 ceph-mon[32751]: slow request 2883.999365 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T19:23:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:26 smithi090 ceph-mon[29180]: 13 slow requests, 4 included below; oldest blocked for > 2887.697461 secs 2024-04-26T19:23:27.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:26 smithi090 ceph-mon[29180]: slow request 2884.112824 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T19:23:27.298 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:26 smithi090 ceph-mon[29180]: slow request 2884.112488 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T19:23:27.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:26 smithi090 ceph-mon[29180]: slow request 2884.003653 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T19:23:27.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:26 smithi090 ceph-mon[29180]: slow request 2883.999365 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T19:23:27.323 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3291 sec 2024-04-26T19:23:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:27 smithi119 ceph-mon[32752]: pgmap v2329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:27 smithi167 ceph-mon[32751]: pgmap v2329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:27 smithi090 ceph-mon[29180]: pgmap v2329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:28.177 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3292 sec 2024-04-26T19:23:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:23:28.847+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:23:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:23:28.847+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 26 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:23:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:23:28.850+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:23:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:23:28.851+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 26 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:23:29.177 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3293 sec 2024-04-26T19:23:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:29 smithi119 ceph-mon[32752]: pgmap v2330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:29 smithi167 ceph-mon[32751]: pgmap v2330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:29 smithi090 ceph-mon[29180]: pgmap v2330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:30.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3294 sec 2024-04-26T19:23:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2892.019219 secs 2024-04-26T19:23:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2892.019219 secs 2024-04-26T19:23:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2892.019219 secs 2024-04-26T19:23:31.177 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3295 sec 2024-04-26T19:23:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:31 smithi167 ceph-mon[32751]: pgmap v2331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:31.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2892.697591 secs 2024-04-26T19:23:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:31 smithi119 ceph-mon[32752]: pgmap v2331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:32.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2892.697591 secs 2024-04-26T19:23:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:31 smithi090 ceph-mon[29180]: pgmap v2331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:32.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2892.697591 secs 2024-04-26T19:23:32.177 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3296 sec 2024-04-26T19:23:33.177 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3297 sec 2024-04-26T19:23:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:33 smithi119 ceph-mon[32752]: pgmap v2332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:34.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:33 smithi090 ceph-mon[29180]: pgmap v2332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:34.168 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:33 smithi167 ceph-mon[32751]: pgmap v2332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:34.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3298 sec 2024-04-26T19:23:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:23:35.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:23:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:23:35.177 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3299 sec 2024-04-26T19:23:35.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:35 smithi090 ceph-mon[29180]: pgmap v2333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2897.019288 secs 2024-04-26T19:23:35.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:35.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:35.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:35.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:35 smithi119 ceph-mon[32752]: pgmap v2333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:36.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2897.019288 secs 2024-04-26T19:23:36.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.188 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.196 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3300 sec 2024-04-26T19:23:36.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:35 smithi167 ceph-mon[32751]: pgmap v2333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:36.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2897.019288 secs 2024-04-26T19:23:36.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.234 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.242 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:36.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2897.697681 secs 2024-04-26T19:23:37.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:37.069 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:37.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2897.697681 secs 2024-04-26T19:23:37.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:37.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2897.697681 secs 2024-04-26T19:23:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:37.193 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:37.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3301 sec 2024-04-26T19:23:38.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: pgmap v2334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:38.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.044 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.052 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.060 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.068 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.181 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: pgmap v2334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:38.188 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.196 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.203 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.210 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.289 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: pgmap v2334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:38.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.386 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:38.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.408 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:38.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3302 sec 2024-04-26T19:23:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:23:39.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:23:39.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.080 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.154 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.163 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.173 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:23:39.182 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:23:39.193 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.202 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.211 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.219 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:23:39.333 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:23:39.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:23:39.352 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:23:39.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: pgmap v2335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-26T19:23:39.370 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: pgmap v2336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:39.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:39.389 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:23:39.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:23:39.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.511 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:23:39.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:23:39.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:23:39.664 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:23:39.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:23:39.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:23:39.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: pgmap v2335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-26T19:23:39.711 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: pgmap v2336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:39.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:39.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:23:39.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:23:39.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:23:39.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:23:39.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.938 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:39.990 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:23:40.001 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:23:40.010 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:23:40.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:23:40.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:23:40.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: pgmap v2335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-26T19:23:40.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: pgmap v2336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:40.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:23:40.124 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3303 sec 2024-04-26T19:23:40.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3304 sec 2024-04-26T19:23:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:40 smithi119 ceph-mon[32752]: pgmap v2337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:41.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2902.019343 secs 2024-04-26T19:23:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:40 smithi167 ceph-mon[32751]: pgmap v2337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:41.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2902.019343 secs 2024-04-26T19:23:41.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:40 smithi090 ceph-mon[29180]: pgmap v2337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:41.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2902.019343 secs 2024-04-26T19:23:41.177 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3305 sec 2024-04-26T19:23:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2902.697800 secs 2024-04-26T19:23:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2902.697800 secs 2024-04-26T19:23:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2902.697800 secs 2024-04-26T19:23:42.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3306 sec 2024-04-26T19:23:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:42 smithi119 ceph-mon[32752]: pgmap v2338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:42 smithi167 ceph-mon[32751]: pgmap v2338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:43.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:42 smithi090 ceph-mon[29180]: pgmap v2338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:43.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3307 sec 2024-04-26T19:23:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:43 smithi119 ceph-mon[32752]: pgmap v2339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:43 smithi167 ceph-mon[32751]: pgmap v2339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:43 smithi090 ceph-mon[29180]: pgmap v2339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:44.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3308 sec 2024-04-26T19:23:45.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3309 sec 2024-04-26T19:23:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2907.019427 secs 2024-04-26T19:23:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2907.019427 secs 2024-04-26T19:23:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2907.019427 secs 2024-04-26T19:23:46.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3310 sec 2024-04-26T19:23:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2907.697867 secs 2024-04-26T19:23:46.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:46 smithi090 ceph-mon[29180]: pgmap v2340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2907.697867 secs 2024-04-26T19:23:46.823 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:46 smithi119 ceph-mon[32752]: pgmap v2340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:46.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2907.697867 secs 2024-04-26T19:23:46.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:46 smithi167 ceph-mon[32751]: pgmap v2340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T19:23:47.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3311 sec 2024-04-26T19:23:48.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3312 sec 2024-04-26T19:23:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:48 smithi119 ceph-mon[32752]: pgmap v2341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:23:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:48 smithi167 ceph-mon[32751]: pgmap v2341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:23:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:48 smithi090 ceph-mon[29180]: pgmap v2341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:23:49.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3313 sec 2024-04-26T19:23:50.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3314 sec 2024-04-26T19:23:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:50 smithi119 ceph-mon[32752]: pgmap v2342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:51.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2912.019508 secs 2024-04-26T19:23:51.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:50 smithi167 ceph-mon[32751]: pgmap v2342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:51.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2912.019508 secs 2024-04-26T19:23:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:50 smithi090 ceph-mon[29180]: pgmap v2342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:51.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2912.019508 secs 2024-04-26T19:23:51.178 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3315 sec 2024-04-26T19:23:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2912.698003 secs 2024-04-26T19:23:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2912.698003 secs 2024-04-26T19:23:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2912.698003 secs 2024-04-26T19:23:52.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3316 sec 2024-04-26T19:23:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:52 smithi119 ceph-mon[32752]: pgmap v2343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:53.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:52 smithi090 ceph-mon[29180]: pgmap v2343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:53.160 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:52 smithi167 ceph-mon[32751]: pgmap v2343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:53.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3317 sec 2024-04-26T19:23:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:53 smithi167 ceph-mon[32751]: pgmap v2344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:53 smithi090 ceph-mon[29180]: pgmap v2344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:54.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3318 sec 2024-04-26T19:23:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:53 smithi119 ceph-mon[32752]: pgmap v2344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:23:55.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3319 sec 2024-04-26T19:23:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2917.019589 secs 2024-04-26T19:23:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2917.019589 secs 2024-04-26T19:23:55.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2917.019589 secs 2024-04-26T19:23:56.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3320 sec 2024-04-26T19:23:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2917.698084 secs 2024-04-26T19:23:56.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:56 smithi090 ceph-mon[29180]: pgmap v2345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2917.698084 secs 2024-04-26T19:23:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:56 smithi119 ceph-mon[32752]: pgmap v2345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:56.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2917.698084 secs 2024-04-26T19:23:56.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:56 smithi167 ceph-mon[32751]: pgmap v2345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:57.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3321 sec 2024-04-26T19:23:58.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3322 sec 2024-04-26T19:23:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:23:58 smithi119 ceph-mon[32752]: pgmap v2346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:59.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:23:58 smithi167 ceph-mon[32751]: pgmap v2346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:23:58 smithi090 ceph-mon[29180]: pgmap v2346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:23:59.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3323 sec 2024-04-26T19:24:00.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3324 sec 2024-04-26T19:24:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:00 smithi119 ceph-mon[32752]: pgmap v2347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:01.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2922.019671 secs 2024-04-26T19:24:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:00 smithi167 ceph-mon[32751]: pgmap v2347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:01.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2922.019671 secs 2024-04-26T19:24:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:00 smithi090 ceph-mon[29180]: pgmap v2347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:01.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2922.019671 secs 2024-04-26T19:24:01.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3325 sec 2024-04-26T19:24:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2922.698226 secs 2024-04-26T19:24:02.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2922.698226 secs 2024-04-26T19:24:02.179 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3326 sec 2024-04-26T19:24:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2922.698226 secs 2024-04-26T19:24:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:02 smithi167 ceph-mon[32751]: pgmap v2348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:02 smithi090 ceph-mon[29180]: pgmap v2348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:03.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3327 sec 2024-04-26T19:24:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:02 smithi119 ceph-mon[32752]: pgmap v2348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:03 smithi167 ceph-mon[32751]: pgmap v2349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:03 smithi090 ceph-mon[29180]: pgmap v2349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:04.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3328 sec 2024-04-26T19:24:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:03 smithi119 ceph-mon[32752]: pgmap v2349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:05.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3329 sec 2024-04-26T19:24:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:05 smithi090 ceph-mon[29180]: 8 slow requests, 1 included below; oldest blocked for > 2927.019754 secs 2024-04-26T19:24:05.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:05 smithi090 ceph-mon[29180]: slow request 2884.015146 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:24:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:05 smithi119 ceph-mon[32752]: 8 slow requests, 1 included below; oldest blocked for > 2927.019754 secs 2024-04-26T19:24:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:05 smithi119 ceph-mon[32752]: slow request 2884.015146 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:24:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:05 smithi167 ceph-mon[32751]: 8 slow requests, 1 included below; oldest blocked for > 2927.019754 secs 2024-04-26T19:24:05.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:05 smithi167 ceph-mon[32751]: slow request 2884.015146 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:24:06.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3330 sec 2024-04-26T19:24:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2927.698373 secs 2024-04-26T19:24:06.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:06 smithi090 ceph-mon[29180]: pgmap v2350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2927.698373 secs 2024-04-26T19:24:06.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:06 smithi119 ceph-mon[32752]: pgmap v2350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2927.698373 secs 2024-04-26T19:24:06.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:06 smithi167 ceph-mon[32751]: pgmap v2350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:07.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3331 sec 2024-04-26T19:24:08.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3332 sec 2024-04-26T19:24:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:08 smithi119 ceph-mon[32752]: pgmap v2351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:09.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:08 smithi167 ceph-mon[32751]: pgmap v2351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:08 smithi090 ceph-mon[29180]: pgmap v2351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:09.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3333 sec 2024-04-26T19:24:10.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3334 sec 2024-04-26T19:24:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:10 smithi119 ceph-mon[32752]: pgmap v2352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:11.121 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2932.019819 secs 2024-04-26T19:24:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:10 smithi090 ceph-mon[29180]: pgmap v2352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:11.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2932.019819 secs 2024-04-26T19:24:11.154 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:10 smithi167 ceph-mon[32751]: pgmap v2352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:11.161 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2932.019819 secs 2024-04-26T19:24:11.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3335 sec 2024-04-26T19:24:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2932.698509 secs 2024-04-26T19:24:12.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:11 smithi167 ceph-mon[32751]: pgmap v2353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2932.698509 secs 2024-04-26T19:24:12.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:11 smithi090 ceph-mon[29180]: pgmap v2353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:12.180 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3336 sec 2024-04-26T19:24:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2932.698509 secs 2024-04-26T19:24:12.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:11 smithi119 ceph-mon[32752]: pgmap v2353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:24:13.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3337 sec 2024-04-26T19:24:14.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3338 sec 2024-04-26T19:24:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:14 smithi119 ceph-mon[32752]: pgmap v2354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:15.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:14 smithi167 ceph-mon[32751]: pgmap v2354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:14 smithi090 ceph-mon[29180]: pgmap v2354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:24:15.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3339 sec 2024-04-26T19:24:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2937.019901 secs 2024-04-26T19:24:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2937.019901 secs 2024-04-26T19:24:16.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3340 sec 2024-04-26T19:24:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2937.019901 secs 2024-04-26T19:24:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2937.698597 secs 2024-04-26T19:24:17.159 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:16 smithi167 ceph-mon[32751]: pgmap v2355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:17.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2937.698597 secs 2024-04-26T19:24:17.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:16 smithi090 ceph-mon[29180]: pgmap v2355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:17.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3341 sec 2024-04-26T19:24:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2937.698597 secs 2024-04-26T19:24:17.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:16 smithi119 ceph-mon[32752]: pgmap v2355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:17 smithi167 ceph-mon[32751]: pgmap v2356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:17 smithi090 ceph-mon[29180]: pgmap v2356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:18.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3342 sec 2024-04-26T19:24:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:17 smithi119 ceph-mon[32752]: pgmap v2356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:19.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3343 sec 2024-04-26T19:24:20.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3344 sec 2024-04-26T19:24:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:20 smithi119 ceph-mon[32752]: pgmap v2357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:21.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2942.019958 secs 2024-04-26T19:24:21.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:20 smithi167 ceph-mon[32751]: pgmap v2357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:21.101 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2942.019958 secs 2024-04-26T19:24:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:20 smithi090 ceph-mon[29180]: pgmap v2357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:21.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2942.019958 secs 2024-04-26T19:24:21.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3345 sec 2024-04-26T19:24:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2942.698692 secs 2024-04-26T19:24:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2942.698692 secs 2024-04-26T19:24:22.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3346 sec 2024-04-26T19:24:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2942.698692 secs 2024-04-26T19:24:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:22 smithi167 ceph-mon[32751]: pgmap v2358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:23.152 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:22 smithi090 ceph-mon[29180]: pgmap v2358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:23.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3347 sec 2024-04-26T19:24:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:22 smithi119 ceph-mon[32752]: pgmap v2358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:23 smithi167 ceph-mon[32751]: pgmap v2359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:23 smithi090 ceph-mon[29180]: pgmap v2359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:24.181 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3348 sec 2024-04-26T19:24:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:23 smithi119 ceph-mon[32752]: pgmap v2359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:25.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3349 sec 2024-04-26T19:24:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2947.020019 secs 2024-04-26T19:24:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2947.020019 secs 2024-04-26T19:24:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2947.020019 secs 2024-04-26T19:24:26.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3350 sec 2024-04-26T19:24:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2947.698843 secs 2024-04-26T19:24:26.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:26 smithi090 ceph-mon[29180]: pgmap v2360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2947.698843 secs 2024-04-26T19:24:26.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:26 smithi119 ceph-mon[32752]: pgmap v2360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2947.698843 secs 2024-04-26T19:24:26.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:26 smithi167 ceph-mon[32751]: pgmap v2360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:27.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3351 sec 2024-04-26T19:24:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:27 smithi090 ceph-mon[29180]: pgmap v2361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:28.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3352 sec 2024-04-26T19:24:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:27 smithi119 ceph-mon[32752]: pgmap v2361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:27 smithi167 ceph-mon[32751]: pgmap v2361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:29.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3353 sec 2024-04-26T19:24:30.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:29 smithi090 ceph-mon[29180]: pgmap v2362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:30.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3354 sec 2024-04-26T19:24:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:29 smithi119 ceph-mon[32752]: pgmap v2362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:29 smithi167 ceph-mon[32751]: pgmap v2362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2952.020094 secs 2024-04-26T19:24:31.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2952.698917 secs 2024-04-26T19:24:31.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3355 sec 2024-04-26T19:24:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2952.020094 secs 2024-04-26T19:24:31.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2952.698917 secs 2024-04-26T19:24:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2952.020094 secs 2024-04-26T19:24:31.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2952.698917 secs 2024-04-26T19:24:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:31 smithi090 ceph-mon[29180]: pgmap v2363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:32.182 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3356 sec 2024-04-26T19:24:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:31 smithi119 ceph-mon[32752]: pgmap v2363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:31 smithi167 ceph-mon[32751]: pgmap v2363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:33.183 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3357 sec 2024-04-26T19:24:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:33 smithi090 ceph-mon[29180]: pgmap v2364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:34.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3358 sec 2024-04-26T19:24:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:33 smithi119 ceph-mon[32752]: pgmap v2364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:33 smithi167 ceph-mon[32751]: pgmap v2364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:35.183 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3359 sec 2024-04-26T19:24:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2957.020153 secs 2024-04-26T19:24:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2957.020153 secs 2024-04-26T19:24:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2957.020153 secs 2024-04-26T19:24:36.183 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3360 sec 2024-04-26T19:24:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2957.699033 secs 2024-04-26T19:24:36.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:36 smithi090 ceph-mon[29180]: pgmap v2365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2957.699033 secs 2024-04-26T19:24:36.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:36 smithi119 ceph-mon[32752]: pgmap v2365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:36.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2957.699033 secs 2024-04-26T19:24:36.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:36 smithi167 ceph-mon[32751]: pgmap v2365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:37.183 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3361 sec 2024-04-26T19:24:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:37 smithi090 ceph-mon[29180]: pgmap v2366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:38.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3362 sec 2024-04-26T19:24:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:37 smithi119 ceph-mon[32752]: pgmap v2366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:37 smithi167 ceph-mon[32751]: pgmap v2366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:24:39.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:39.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:39.183 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3363 sec 2024-04-26T19:24:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:24:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:39.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:39.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:24:39.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:39.400 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:24:40.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:24:40.141 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:39 smithi090 ceph-mon[29180]: pgmap v2367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:24:40.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:39 smithi090 ceph-mon[29180]: pgmap v2368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:40.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:40.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3364 sec 2024-04-26T19:24:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:24:40.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:24:40.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:39 smithi119 ceph-mon[32752]: pgmap v2367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:24:40.463 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:39 smithi119 ceph-mon[32752]: pgmap v2368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:40.472 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:40.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:24:40.488 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:24:40.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:39 smithi167 ceph-mon[32751]: pgmap v2367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:24:40.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:39 smithi167 ceph-mon[32751]: pgmap v2368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:40.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:24:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2962.020213 secs 2024-04-26T19:24:41.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2962.699179 secs 2024-04-26T19:24:41.183 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3365 sec 2024-04-26T19:24:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2962.020213 secs 2024-04-26T19:24:41.215 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2962.699179 secs 2024-04-26T19:24:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2962.020213 secs 2024-04-26T19:24:41.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2962.699179 secs 2024-04-26T19:24:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:41 smithi090 ceph-mon[29180]: pgmap v2369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:42.183 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3366 sec 2024-04-26T19:24:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:41 smithi119 ceph-mon[32752]: pgmap v2369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:41 smithi167 ceph-mon[32751]: pgmap v2369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:43.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3367 sec 2024-04-26T19:24:44.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3368 sec 2024-04-26T19:24:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:44 smithi090 ceph-mon[29180]: pgmap v2370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:24:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:44 smithi119 ceph-mon[32752]: pgmap v2370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:24:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:44 smithi167 ceph-mon[32751]: pgmap v2370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:24:45.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3369 sec 2024-04-26T19:24:46.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3370 sec 2024-04-26T19:24:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:46 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2967.020304 secs 2024-04-26T19:24:46.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:46 smithi090 ceph-mon[29180]: pgmap v2371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:46.400 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2967.699298 secs 2024-04-26T19:24:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:46 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2967.020304 secs 2024-04-26T19:24:46.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:46 smithi119 ceph-mon[32752]: pgmap v2371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:46.571 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2967.699298 secs 2024-04-26T19:24:46.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:46 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2967.020304 secs 2024-04-26T19:24:46.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:46 smithi167 ceph-mon[32751]: pgmap v2371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:46.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2967.699298 secs 2024-04-26T19:24:47.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3371 sec 2024-04-26T19:24:48.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3372 sec 2024-04-26T19:24:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:48 smithi090 ceph-mon[29180]: pgmap v2372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:48 smithi119 ceph-mon[32752]: pgmap v2372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:48 smithi167 ceph-mon[32751]: pgmap v2372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:24:49.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3373 sec 2024-04-26T19:24:50.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3374 sec 2024-04-26T19:24:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:50 smithi090 ceph-mon[29180]: pgmap v2373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:24:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:50 smithi119 ceph-mon[32752]: pgmap v2373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:24:50.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:50 smithi167 ceph-mon[32751]: pgmap v2373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:24:51.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3375 sec 2024-04-26T19:24:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:51 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2972.020386 secs 2024-04-26T19:24:51.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2972.699418 secs 2024-04-26T19:24:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:51 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2972.020386 secs 2024-04-26T19:24:51.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2972.699418 secs 2024-04-26T19:24:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:51 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2972.020386 secs 2024-04-26T19:24:51.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2972.699418 secs 2024-04-26T19:24:52.184 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3376 sec 2024-04-26T19:24:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:52 smithi090 ceph-mon[29180]: pgmap v2374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:52 smithi119 ceph-mon[32752]: pgmap v2374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:52.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:52 smithi167 ceph-mon[32751]: pgmap v2374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:53.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3377 sec 2024-04-26T19:24:54.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3378 sec 2024-04-26T19:24:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:54 smithi090 ceph-mon[29180]: pgmap v2375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:54 smithi119 ceph-mon[32752]: pgmap v2375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:54 smithi167 ceph-mon[32751]: pgmap v2375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:55.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3379 sec 2024-04-26T19:24:56.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3380 sec 2024-04-26T19:24:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:56 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2977.020471 secs 2024-04-26T19:24:56.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:56 smithi090 ceph-mon[29180]: pgmap v2376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:56.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2977.699535 secs 2024-04-26T19:24:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:56 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2977.020471 secs 2024-04-26T19:24:56.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:56 smithi119 ceph-mon[32752]: pgmap v2376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:56.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2977.699535 secs 2024-04-26T19:24:56.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:56 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2977.020471 secs 2024-04-26T19:24:56.618 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:56 smithi167 ceph-mon[32751]: pgmap v2376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:24:56.628 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2977.699535 secs 2024-04-26T19:24:57.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3381 sec 2024-04-26T19:24:58.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3382 sec 2024-04-26T19:24:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:24:58 smithi090 ceph-mon[29180]: pgmap v2377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:24:58 smithi119 ceph-mon[32752]: pgmap v2377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:58.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:24:58 smithi167 ceph-mon[32751]: pgmap v2377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:24:58.859 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:24:58.859+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:24:58.898 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:24:58.859+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 27 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca80063b0 con 0x7f2cd4104500 2024-04-26T19:24:58.910 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:24:58.861+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:24:58.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:24:58.861+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 27 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:24:59.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3383 sec 2024-04-26T19:25:00.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3384 sec 2024-04-26T19:25:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:00 smithi090 ceph-mon[29180]: pgmap v2378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:00.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2982.020555 secs 2024-04-26T19:25:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:00 smithi119 ceph-mon[32752]: pgmap v2378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:00.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2982.020555 secs 2024-04-26T19:25:00.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:00 smithi167 ceph-mon[32751]: pgmap v2378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:00.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2982.020555 secs 2024-04-26T19:25:01.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3385 sec 2024-04-26T19:25:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2982.699656 secs 2024-04-26T19:25:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2982.699656 secs 2024-04-26T19:25:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2982.699656 secs 2024-04-26T19:25:02.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3386 sec 2024-04-26T19:25:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:02 smithi119 ceph-mon[32752]: pgmap v2379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:02.920 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:02 smithi090 ceph-mon[29180]: pgmap v2379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:02.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:02 smithi167 ceph-mon[32751]: pgmap v2379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:03.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3387 sec 2024-04-26T19:25:04.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3388 sec 2024-04-26T19:25:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:04 smithi119 ceph-mon[32752]: pgmap v2380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:04.855 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:04 smithi167 ceph-mon[32751]: pgmap v2380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:04 smithi090 ceph-mon[29180]: pgmap v2380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:05.185 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3389 sec 2024-04-26T19:25:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2987.020640 secs 2024-04-26T19:25:05.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2987.020640 secs 2024-04-26T19:25:05.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2987.020640 secs 2024-04-26T19:25:06.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3390 sec 2024-04-26T19:25:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:06 smithi119 ceph-mon[32752]: pgmap v2381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:06.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2987.699727 secs 2024-04-26T19:25:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:06 smithi167 ceph-mon[32751]: pgmap v2381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:06.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2987.699727 secs 2024-04-26T19:25:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:06 smithi090 ceph-mon[29180]: pgmap v2381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:06.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2987.699727 secs 2024-04-26T19:25:07.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3391 sec 2024-04-26T19:25:08.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3392 sec 2024-04-26T19:25:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:08 smithi119 ceph-mon[32752]: pgmap v2382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:08.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:08 smithi090 ceph-mon[29180]: pgmap v2382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:08.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:08 smithi167 ceph-mon[32751]: pgmap v2382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:09.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3393 sec 2024-04-26T19:25:10.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3394 sec 2024-04-26T19:25:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:10 smithi119 ceph-mon[32752]: pgmap v2383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:10.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2992.020720 secs 2024-04-26T19:25:10.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:10 smithi167 ceph-mon[32751]: pgmap v2383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:10.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2992.020720 secs 2024-04-26T19:25:10.898 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:10 smithi090 ceph-mon[29180]: pgmap v2383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:10.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2992.020720 secs 2024-04-26T19:25:11.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3395 sec 2024-04-26T19:25:11.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2992.699791 secs 2024-04-26T19:25:11.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2992.699791 secs 2024-04-26T19:25:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2992.699791 secs 2024-04-26T19:25:12.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3396 sec 2024-04-26T19:25:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:12 smithi119 ceph-mon[32752]: pgmap v2384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:12 smithi167 ceph-mon[32751]: pgmap v2384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:12 smithi090 ceph-mon[29180]: pgmap v2384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:13.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3397 sec 2024-04-26T19:25:14.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3398 sec 2024-04-26T19:25:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:14 smithi119 ceph-mon[32752]: pgmap v2385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:14 smithi167 ceph-mon[32751]: pgmap v2385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:14 smithi090 ceph-mon[29180]: pgmap v2385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:15.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3399 sec 2024-04-26T19:25:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 2997.020803 secs 2024-04-26T19:25:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 2997.020803 secs 2024-04-26T19:25:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 2997.020803 secs 2024-04-26T19:25:16.186 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3400 sec 2024-04-26T19:25:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:16 smithi119 ceph-mon[32752]: pgmap v2386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:16.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 2997.699909 secs 2024-04-26T19:25:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:16 smithi167 ceph-mon[32751]: pgmap v2386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:16.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 2997.699909 secs 2024-04-26T19:25:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:16 smithi090 ceph-mon[29180]: pgmap v2386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:16.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 2997.699909 secs 2024-04-26T19:25:17.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3401 sec 2024-04-26T19:25:18.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3402 sec 2024-04-26T19:25:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:18 smithi119 ceph-mon[32752]: pgmap v2387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:18 smithi167 ceph-mon[32751]: pgmap v2387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:18 smithi090 ceph-mon[29180]: pgmap v2387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:19.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3403 sec 2024-04-26T19:25:20.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3404 sec 2024-04-26T19:25:20.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:20 smithi090 ceph-mon[29180]: pgmap v2388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:20.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3002.020864 secs 2024-04-26T19:25:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:20 smithi119 ceph-mon[32752]: pgmap v2388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:20.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3002.020864 secs 2024-04-26T19:25:20.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:20 smithi167 ceph-mon[32751]: pgmap v2388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:20.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3002.020864 secs 2024-04-26T19:25:21.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3405 sec 2024-04-26T19:25:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3002.700041 secs 2024-04-26T19:25:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3002.700041 secs 2024-04-26T19:25:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3002.700041 secs 2024-04-26T19:25:22.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3406 sec 2024-04-26T19:25:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:22 smithi119 ceph-mon[32752]: pgmap v2389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:22 smithi167 ceph-mon[32751]: pgmap v2389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:22 smithi090 ceph-mon[29180]: pgmap v2389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:23.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3407 sec 2024-04-26T19:25:24.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3408 sec 2024-04-26T19:25:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:24 smithi119 ceph-mon[32752]: pgmap v2390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:24 smithi167 ceph-mon[32751]: pgmap v2390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:24 smithi090 ceph-mon[29180]: pgmap v2390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:25.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3409 sec 2024-04-26T19:25:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3007.020923 secs 2024-04-26T19:25:25.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3007.020923 secs 2024-04-26T19:25:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3007.020923 secs 2024-04-26T19:25:26.187 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3410 sec 2024-04-26T19:25:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:26 smithi119 ceph-mon[32752]: pgmap v2391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:26.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3007.700143 secs 2024-04-26T19:25:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:26 smithi167 ceph-mon[32751]: pgmap v2391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:26.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3007.700143 secs 2024-04-26T19:25:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:26 smithi090 ceph-mon[29180]: pgmap v2391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:26.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3007.700143 secs 2024-04-26T19:25:27.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3411 sec 2024-04-26T19:25:28.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3412 sec 2024-04-26T19:25:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:28 smithi119 ceph-mon[32752]: pgmap v2392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:28 smithi167 ceph-mon[32751]: pgmap v2392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:28 smithi090 ceph-mon[29180]: pgmap v2392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:29.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3413 sec 2024-04-26T19:25:30.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3414 sec 2024-04-26T19:25:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:30 smithi119 ceph-mon[32752]: pgmap v2393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:30.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3012.021001 secs 2024-04-26T19:25:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:30 smithi167 ceph-mon[32751]: pgmap v2393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:30.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3012.021001 secs 2024-04-26T19:25:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:30 smithi090 ceph-mon[29180]: pgmap v2393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:30.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3012.021001 secs 2024-04-26T19:25:31.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3415 sec 2024-04-26T19:25:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3012.700216 secs 2024-04-26T19:25:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3012.700216 secs 2024-04-26T19:25:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3012.700216 secs 2024-04-26T19:25:32.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3416 sec 2024-04-26T19:25:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:32 smithi119 ceph-mon[32752]: pgmap v2394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:32 smithi167 ceph-mon[32751]: pgmap v2394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:32 smithi090 ceph-mon[29180]: pgmap v2394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:33.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3417 sec 2024-04-26T19:25:34.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3418 sec 2024-04-26T19:25:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:34 smithi119 ceph-mon[32752]: pgmap v2395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:34 smithi167 ceph-mon[32751]: pgmap v2395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:34 smithi090 ceph-mon[29180]: pgmap v2395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:35.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3419 sec 2024-04-26T19:25:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:35 smithi119 ceph-mon[32752]: 8 slow requests, 1 included below; oldest blocked for > 3017.021104 secs 2024-04-26T19:25:35.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:35 smithi119 ceph-mon[32752]: slow request 2881.284456 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T19:25:35.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:35 smithi167 ceph-mon[32751]: 8 slow requests, 1 included below; oldest blocked for > 3017.021104 secs 2024-04-26T19:25:35.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:35 smithi167 ceph-mon[32751]: slow request 2881.284456 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T19:25:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:35 smithi090 ceph-mon[29180]: 8 slow requests, 1 included below; oldest blocked for > 3017.021104 secs 2024-04-26T19:25:35.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:35 smithi090 ceph-mon[29180]: slow request 2881.284456 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T19:25:36.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3420 sec 2024-04-26T19:25:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:36 smithi119 ceph-mon[32752]: pgmap v2396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:36.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 3017.700276 secs 2024-04-26T19:25:36.822 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:36 smithi119 ceph-mon[32752]: slow request 1444.225625 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:25:36.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:36 smithi167 ceph-mon[32751]: pgmap v2396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:36.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 3017.700276 secs 2024-04-26T19:25:36.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:36 smithi167 ceph-mon[32751]: slow request 1444.225625 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:25:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:36 smithi090 ceph-mon[29180]: pgmap v2396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:36.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 3017.700276 secs 2024-04-26T19:25:36.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:36 smithi090 ceph-mon[29180]: slow request 1444.225625 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:25:37.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3421 sec 2024-04-26T19:25:38.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3422 sec 2024-04-26T19:25:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:38 smithi119 ceph-mon[32752]: pgmap v2397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:38 smithi167 ceph-mon[32751]: pgmap v2397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:38 smithi090 ceph-mon[29180]: pgmap v2397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:39.188 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3423 sec 2024-04-26T19:25:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:25:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:25:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:25:40.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3424 sec 2024-04-26T19:25:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:40 smithi119 ceph-mon[32752]: pgmap v2398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:40.816 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3022.021166 secs 2024-04-26T19:25:40.826 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:25:40.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:25:40.847 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:25:40.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:40 smithi167 ceph-mon[32751]: pgmap v2398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:40.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3022.021166 secs 2024-04-26T19:25:40.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:25:40.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:25:40.952 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:25:40.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:40 smithi090 ceph-mon[29180]: pgmap v2398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:40.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3022.021166 secs 2024-04-26T19:25:40.981 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:25:41.052 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:25:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:25:41.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3425 sec 2024-04-26T19:25:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:41 smithi119 ceph-mon[32752]: pgmap v2399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T19:25:41.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:41 smithi119 ceph-mon[32752]: pgmap v2400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:41.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3022.700411 secs 2024-04-26T19:25:41.847 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:41 smithi167 ceph-mon[32751]: pgmap v2399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T19:25:41.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:41 smithi167 ceph-mon[32751]: pgmap v2400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:41.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3022.700411 secs 2024-04-26T19:25:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:41 smithi090 ceph-mon[29180]: pgmap v2399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T19:25:41.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:41 smithi090 ceph-mon[29180]: pgmap v2400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:41.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3022.700411 secs 2024-04-26T19:25:42.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3426 sec 2024-04-26T19:25:43.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3427 sec 2024-04-26T19:25:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:43 smithi119 ceph-mon[32752]: pgmap v2401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:43 smithi167 ceph-mon[32751]: pgmap v2401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:43 smithi090 ceph-mon[29180]: pgmap v2401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:44.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3428 sec 2024-04-26T19:25:45.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3429 sec 2024-04-26T19:25:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:45 smithi119 ceph-mon[32752]: pgmap v2402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:45.816 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3027.021251 secs 2024-04-26T19:25:45.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:45 smithi167 ceph-mon[32751]: pgmap v2402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:45.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3027.021251 secs 2024-04-26T19:25:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:45 smithi090 ceph-mon[29180]: pgmap v2402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:45.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3027.021251 secs 2024-04-26T19:25:46.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3430 sec 2024-04-26T19:25:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3027.700554 secs 2024-04-26T19:25:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3027.700554 secs 2024-04-26T19:25:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3027.700554 secs 2024-04-26T19:25:47.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3431 sec 2024-04-26T19:25:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:47 smithi119 ceph-mon[32752]: pgmap v2403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:47 smithi167 ceph-mon[32751]: pgmap v2403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:47 smithi090 ceph-mon[29180]: pgmap v2403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:25:48.189 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3432 sec 2024-04-26T19:25:49.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3433 sec 2024-04-26T19:25:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:49 smithi119 ceph-mon[32752]: pgmap v2404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T19:25:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:49 smithi167 ceph-mon[32751]: pgmap v2404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T19:25:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:49 smithi090 ceph-mon[29180]: pgmap v2404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T19:25:50.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3434 sec 2024-04-26T19:25:50.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3032.021334 secs 2024-04-26T19:25:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3032.021334 secs 2024-04-26T19:25:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3032.021334 secs 2024-04-26T19:25:51.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3435 sec 2024-04-26T19:25:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:51 smithi119 ceph-mon[32752]: pgmap v2405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:25:51.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3032.700638 secs 2024-04-26T19:25:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:51 smithi167 ceph-mon[32751]: pgmap v2405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:25:51.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3032.700638 secs 2024-04-26T19:25:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:51 smithi090 ceph-mon[29180]: pgmap v2405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:25:51.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3032.700638 secs 2024-04-26T19:25:52.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3436 sec 2024-04-26T19:25:53.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3437 sec 2024-04-26T19:25:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:53 smithi119 ceph-mon[32752]: pgmap v2406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:53 smithi167 ceph-mon[32751]: pgmap v2406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:53 smithi090 ceph-mon[29180]: pgmap v2406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:54.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3438 sec 2024-04-26T19:25:55.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3439 sec 2024-04-26T19:25:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:55 smithi119 ceph-mon[32752]: pgmap v2407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:55.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3037.021417 secs 2024-04-26T19:25:55.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:55 smithi167 ceph-mon[32751]: pgmap v2407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:55.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3037.021417 secs 2024-04-26T19:25:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:55 smithi090 ceph-mon[29180]: pgmap v2407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:25:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3037.021417 secs 2024-04-26T19:25:56.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3440 sec 2024-04-26T19:25:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3037.700759 secs 2024-04-26T19:25:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3037.700759 secs 2024-04-26T19:25:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3037.700759 secs 2024-04-26T19:25:57.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3441 sec 2024-04-26T19:25:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:57 smithi119 ceph-mon[32752]: pgmap v2408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:57 smithi167 ceph-mon[32751]: pgmap v2408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:57 smithi090 ceph-mon[29180]: pgmap v2408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:58.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3442 sec 2024-04-26T19:25:59.190 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3443 sec 2024-04-26T19:25:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:25:59 smithi119 ceph-mon[32752]: pgmap v2409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:25:59 smithi167 ceph-mon[32751]: pgmap v2409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:25:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:25:59 smithi090 ceph-mon[29180]: pgmap v2409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:00.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3444 sec 2024-04-26T19:26:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3042.021502 secs 2024-04-26T19:26:00.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3042.021502 secs 2024-04-26T19:26:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3042.021502 secs 2024-04-26T19:26:01.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3445 sec 2024-04-26T19:26:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:01 smithi119 ceph-mon[32752]: pgmap v2410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:01.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3042.700841 secs 2024-04-26T19:26:01.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:01 smithi167 ceph-mon[32751]: pgmap v2410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:01.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3042.700841 secs 2024-04-26T19:26:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:01 smithi090 ceph-mon[29180]: pgmap v2410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:01.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3042.700841 secs 2024-04-26T19:26:02.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3446 sec 2024-04-26T19:26:03.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3447 sec 2024-04-26T19:26:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:03 smithi119 ceph-mon[32752]: pgmap v2411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:03.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:03 smithi090 ceph-mon[29180]: pgmap v2411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:03.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:03 smithi167 ceph-mon[32751]: pgmap v2411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:04.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3448 sec 2024-04-26T19:26:05.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3449 sec 2024-04-26T19:26:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:05 smithi119 ceph-mon[32752]: pgmap v2412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:05.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3047.021585 secs 2024-04-26T19:26:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:05 smithi167 ceph-mon[32751]: pgmap v2412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3047.021585 secs 2024-04-26T19:26:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:05 smithi090 ceph-mon[29180]: pgmap v2412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3047.021585 secs 2024-04-26T19:26:06.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3450 sec 2024-04-26T19:26:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3047.700957 secs 2024-04-26T19:26:06.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3047.700957 secs 2024-04-26T19:26:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3047.700957 secs 2024-04-26T19:26:07.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3451 sec 2024-04-26T19:26:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:07 smithi119 ceph-mon[32752]: pgmap v2413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:07 smithi167 ceph-mon[32751]: pgmap v2413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:07 smithi090 ceph-mon[29180]: pgmap v2413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:08.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3452 sec 2024-04-26T19:26:09.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3453 sec 2024-04-26T19:26:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:09 smithi119 ceph-mon[32752]: pgmap v2414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:09.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:09 smithi167 ceph-mon[32751]: pgmap v2414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:09 smithi090 ceph-mon[29180]: pgmap v2414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:10.191 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3454 sec 2024-04-26T19:26:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3052.021666 secs 2024-04-26T19:26:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3052.021666 secs 2024-04-26T19:26:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3052.021666 secs 2024-04-26T19:26:11.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3455 sec 2024-04-26T19:26:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:11 smithi119 ceph-mon[32752]: pgmap v2415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:11.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3052.701068 secs 2024-04-26T19:26:11.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:11 smithi167 ceph-mon[32751]: pgmap v2415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:11.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3052.701068 secs 2024-04-26T19:26:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:11 smithi090 ceph-mon[29180]: pgmap v2415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:11.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3052.701068 secs 2024-04-26T19:26:12.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3456 sec 2024-04-26T19:26:13.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3457 sec 2024-04-26T19:26:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:13 smithi119 ceph-mon[32752]: pgmap v2416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:13 smithi167 ceph-mon[32751]: pgmap v2416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:13 smithi090 ceph-mon[29180]: pgmap v2416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:14.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3458 sec 2024-04-26T19:26:15.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3459 sec 2024-04-26T19:26:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:15 smithi119 ceph-mon[32752]: pgmap v2417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:15.853 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3057.021748 secs 2024-04-26T19:26:15.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:15 smithi167 ceph-mon[32751]: pgmap v2417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:15.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3057.021748 secs 2024-04-26T19:26:15.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:15 smithi090 ceph-mon[29180]: pgmap v2417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:15.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3057.021748 secs 2024-04-26T19:26:16.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3460 sec 2024-04-26T19:26:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3057.701144 secs 2024-04-26T19:26:16.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3057.701144 secs 2024-04-26T19:26:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3057.701144 secs 2024-04-26T19:26:17.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3461 sec 2024-04-26T19:26:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:17 smithi119 ceph-mon[32752]: pgmap v2418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:17 smithi167 ceph-mon[32751]: pgmap v2418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:17 smithi090 ceph-mon[29180]: pgmap v2418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:18.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3462 sec 2024-04-26T19:26:19.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3463 sec 2024-04-26T19:26:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:19 smithi167 ceph-mon[32751]: pgmap v2419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:19 smithi090 ceph-mon[29180]: pgmap v2419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:19 smithi119 ceph-mon[32752]: pgmap v2419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:20.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3464 sec 2024-04-26T19:26:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3062.021845 secs 2024-04-26T19:26:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3062.021845 secs 2024-04-26T19:26:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3062.021845 secs 2024-04-26T19:26:21.192 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3465 sec 2024-04-26T19:26:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:21 smithi167 ceph-mon[32751]: pgmap v2420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:21.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3062.701277 secs 2024-04-26T19:26:21.921 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:21 smithi090 ceph-mon[29180]: pgmap v2420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:21.931 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3062.701277 secs 2024-04-26T19:26:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:21 smithi119 ceph-mon[32752]: pgmap v2420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:22.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3062.701277 secs 2024-04-26T19:26:22.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3466 sec 2024-04-26T19:26:23.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3467 sec 2024-04-26T19:26:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:23 smithi167 ceph-mon[32751]: pgmap v2421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:23 smithi090 ceph-mon[29180]: pgmap v2421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:23 smithi119 ceph-mon[32752]: pgmap v2421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:24.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3468 sec 2024-04-26T19:26:25.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3469 sec 2024-04-26T19:26:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:25 smithi090 ceph-mon[29180]: pgmap v2422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:25.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3067.021907 secs 2024-04-26T19:26:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:25 smithi119 ceph-mon[32752]: pgmap v2422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:26.113 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3067.021907 secs 2024-04-26T19:26:26.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:25 smithi167 ceph-mon[32751]: pgmap v2422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:26.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3067.021907 secs 2024-04-26T19:26:26.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3470 sec 2024-04-26T19:26:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3067.701414 secs 2024-04-26T19:26:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3067.701414 secs 2024-04-26T19:26:27.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3067.701414 secs 2024-04-26T19:26:27.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3471 sec 2024-04-26T19:26:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:27 smithi090 ceph-mon[29180]: pgmap v2423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:27 smithi119 ceph-mon[32752]: pgmap v2423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:27 smithi167 ceph-mon[32751]: pgmap v2423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:28.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3472 sec 2024-04-26T19:26:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:26:28.871+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:26:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:26:28.871+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 28 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:26:28.872+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:26:28.872+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 28 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:26:29.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3473 sec 2024-04-26T19:26:29.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:29 smithi090 ceph-mon[29180]: pgmap v2424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:29 smithi119 ceph-mon[32752]: pgmap v2424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:29 smithi167 ceph-mon[32751]: pgmap v2424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:30.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3474 sec 2024-04-26T19:26:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3072.021988 secs 2024-04-26T19:26:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3072.021988 secs 2024-04-26T19:26:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3072.021988 secs 2024-04-26T19:26:31.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3475 sec 2024-04-26T19:26:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:31 smithi090 ceph-mon[29180]: pgmap v2425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:31.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3072.701495 secs 2024-04-26T19:26:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:31 smithi167 ceph-mon[32751]: pgmap v2425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:31.979 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3072.701495 secs 2024-04-26T19:26:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:31 smithi119 ceph-mon[32752]: pgmap v2425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:32.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3072.701495 secs 2024-04-26T19:26:32.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3476 sec 2024-04-26T19:26:33.193 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3477 sec 2024-04-26T19:26:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:33 smithi090 ceph-mon[29180]: pgmap v2426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:33 smithi119 ceph-mon[32752]: pgmap v2426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:33 smithi167 ceph-mon[32751]: pgmap v2426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:34.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3478 sec 2024-04-26T19:26:35.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3479 sec 2024-04-26T19:26:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:35 smithi090 ceph-mon[29180]: pgmap v2427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:35.979 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3077.022094 secs 2024-04-26T19:26:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:35 smithi119 ceph-mon[32752]: pgmap v2427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:36.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3077.022094 secs 2024-04-26T19:26:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:35 smithi167 ceph-mon[32751]: pgmap v2427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:36.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3077.022094 secs 2024-04-26T19:26:36.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3480 sec 2024-04-26T19:26:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3077.701610 secs 2024-04-26T19:26:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3077.701610 secs 2024-04-26T19:26:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3077.701610 secs 2024-04-26T19:26:37.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3481 sec 2024-04-26T19:26:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:37 smithi090 ceph-mon[29180]: pgmap v2428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:37 smithi119 ceph-mon[32752]: pgmap v2428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:38.143 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:37 smithi167 ceph-mon[32751]: pgmap v2428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:38.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3482 sec 2024-04-26T19:26:39.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3483 sec 2024-04-26T19:26:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:39 smithi090 ceph-mon[29180]: pgmap v2429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:39 smithi119 ceph-mon[32752]: pgmap v2429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:39 smithi167 ceph-mon[32751]: pgmap v2429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:40.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3484 sec 2024-04-26T19:26:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3082.022178 secs 2024-04-26T19:26:40.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:26:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3082.022178 secs 2024-04-26T19:26:41.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:26:41.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3082.022178 secs 2024-04-26T19:26:41.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:26:41.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3485 sec 2024-04-26T19:26:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:41 smithi090 ceph-mon[29180]: pgmap v2430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:42.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3082.701718 secs 2024-04-26T19:26:42.042 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:26:42.050 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:26:42.057 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:41 smithi090 ceph-mon[29180]: pgmap v2431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:26:42.065 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:41 smithi090 ceph-mon[29180]: pgmap v2432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:42.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:26:42.154 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:41 smithi167 ceph-mon[32751]: pgmap v2430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:42.162 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3082.701718 secs 2024-04-26T19:26:42.170 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:26:42.178 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:26:42.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:41 smithi167 ceph-mon[32751]: pgmap v2431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:26:42.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:41 smithi167 ceph-mon[32751]: pgmap v2432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:42.254 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:26:42.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:41 smithi119 ceph-mon[32752]: pgmap v2430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:42.326 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3082.701718 secs 2024-04-26T19:26:42.333 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:26:42.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:26:42.400 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:41 smithi119 ceph-mon[32752]: pgmap v2431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:26:42.407 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:41 smithi119 ceph-mon[32752]: pgmap v2432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:42.415 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:26:42.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3486 sec 2024-04-26T19:26:43.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3487 sec 2024-04-26T19:26:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:43 smithi119 ceph-mon[32752]: pgmap v2433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:26:44.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:43 smithi167 ceph-mon[32751]: pgmap v2433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:26:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:43 smithi090 ceph-mon[29180]: pgmap v2433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:26:44.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3488 sec 2024-04-26T19:26:45.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3489 sec 2024-04-26T19:26:45.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:45 smithi090 ceph-mon[29180]: pgmap v2434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:45.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3087.022261 secs 2024-04-26T19:26:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:45 smithi119 ceph-mon[32752]: pgmap v2434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:46.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3087.022261 secs 2024-04-26T19:26:46.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:45 smithi167 ceph-mon[32751]: pgmap v2434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:46.173 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3087.022261 secs 2024-04-26T19:26:46.194 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3490 sec 2024-04-26T19:26:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3087.701833 secs 2024-04-26T19:26:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3087.701833 secs 2024-04-26T19:26:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3087.701833 secs 2024-04-26T19:26:47.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3491 sec 2024-04-26T19:26:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:47 smithi119 ceph-mon[32752]: pgmap v2435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:48.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:47 smithi167 ceph-mon[32751]: pgmap v2435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:47 smithi090 ceph-mon[29180]: pgmap v2435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:48.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3492 sec 2024-04-26T19:26:49.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3493 sec 2024-04-26T19:26:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:49 smithi119 ceph-mon[32752]: pgmap v2436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:49 smithi167 ceph-mon[32751]: pgmap v2436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:49 smithi090 ceph-mon[29180]: pgmap v2436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:26:50.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3494 sec 2024-04-26T19:26:51.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3092.022317 secs 2024-04-26T19:26:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3092.022317 secs 2024-04-26T19:26:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3092.022317 secs 2024-04-26T19:26:51.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3495 sec 2024-04-26T19:26:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3092.701904 secs 2024-04-26T19:26:52.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:51 smithi119 ceph-mon[32752]: pgmap v2437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:26:52.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3092.701904 secs 2024-04-26T19:26:52.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:51 smithi167 ceph-mon[32751]: pgmap v2437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:26:52.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3092.701904 secs 2024-04-26T19:26:52.231 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:51 smithi090 ceph-mon[29180]: pgmap v2437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:26:52.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3496 sec 2024-04-26T19:26:53.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3497 sec 2024-04-26T19:26:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:53 smithi119 ceph-mon[32752]: pgmap v2438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:54.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:53 smithi167 ceph-mon[32751]: pgmap v2438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:53 smithi090 ceph-mon[29180]: pgmap v2438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:54.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3498 sec 2024-04-26T19:26:55.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3499 sec 2024-04-26T19:26:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:55 smithi119 ceph-mon[32752]: pgmap v2439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:56.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3097.022404 secs 2024-04-26T19:26:56.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:55 smithi167 ceph-mon[32751]: pgmap v2439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:56.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3097.022404 secs 2024-04-26T19:26:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:55 smithi090 ceph-mon[29180]: pgmap v2439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:26:56.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3097.022404 secs 2024-04-26T19:26:56.195 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3500 sec 2024-04-26T19:26:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3097.702051 secs 2024-04-26T19:26:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3097.702051 secs 2024-04-26T19:26:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3097.702051 secs 2024-04-26T19:26:57.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3501 sec 2024-04-26T19:26:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:57 smithi119 ceph-mon[32752]: pgmap v2440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:57 smithi167 ceph-mon[32751]: pgmap v2440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:57 smithi090 ceph-mon[29180]: pgmap v2440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:26:58.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3502 sec 2024-04-26T19:26:59.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3503 sec 2024-04-26T19:27:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:26:59 smithi119 ceph-mon[32752]: pgmap v2441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:26:59 smithi167 ceph-mon[32751]: pgmap v2441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:26:59 smithi090 ceph-mon[29180]: pgmap v2441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:00.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3504 sec 2024-04-26T19:27:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3102.022486 secs 2024-04-26T19:27:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3102.022486 secs 2024-04-26T19:27:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3102.022486 secs 2024-04-26T19:27:01.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3505 sec 2024-04-26T19:27:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3102.702166 secs 2024-04-26T19:27:01.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:01 smithi090 ceph-mon[29180]: pgmap v2442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3102.702166 secs 2024-04-26T19:27:02.068 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:01 smithi119 ceph-mon[32752]: pgmap v2442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:02.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3102.702166 secs 2024-04-26T19:27:02.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:01 smithi167 ceph-mon[32751]: pgmap v2442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:02.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3506 sec 2024-04-26T19:27:03.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3507 sec 2024-04-26T19:27:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:03 smithi119 ceph-mon[32752]: pgmap v2443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:03 smithi167 ceph-mon[32751]: pgmap v2443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:03 smithi090 ceph-mon[29180]: pgmap v2443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:04.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3508 sec 2024-04-26T19:27:05.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3509 sec 2024-04-26T19:27:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:05 smithi119 ceph-mon[32752]: pgmap v2444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:06.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3107.022576 secs 2024-04-26T19:27:06.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:05 smithi167 ceph-mon[32751]: pgmap v2444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:06.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3107.022576 secs 2024-04-26T19:27:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:05 smithi090 ceph-mon[29180]: pgmap v2444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:06.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3107.022576 secs 2024-04-26T19:27:06.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3510 sec 2024-04-26T19:27:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3107.702274 secs 2024-04-26T19:27:07.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3107.702274 secs 2024-04-26T19:27:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3107.702274 secs 2024-04-26T19:27:07.196 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3511 sec 2024-04-26T19:27:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:07 smithi119 ceph-mon[32752]: pgmap v2445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:07 smithi167 ceph-mon[32751]: pgmap v2445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:07 smithi090 ceph-mon[29180]: pgmap v2445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:08.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3512 sec 2024-04-26T19:27:09.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3513 sec 2024-04-26T19:27:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:09 smithi119 ceph-mon[32752]: pgmap v2446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:09 smithi167 ceph-mon[32751]: pgmap v2446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:09 smithi090 ceph-mon[29180]: pgmap v2446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:10.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3514 sec 2024-04-26T19:27:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3112.022657 secs 2024-04-26T19:27:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3112.022657 secs 2024-04-26T19:27:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3112.022657 secs 2024-04-26T19:27:11.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3515 sec 2024-04-26T19:27:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3112.702413 secs 2024-04-26T19:27:12.087 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:11 smithi119 ceph-mon[32752]: pgmap v2447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:12.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3112.702413 secs 2024-04-26T19:27:12.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:11 smithi167 ceph-mon[32751]: pgmap v2447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3112.702413 secs 2024-04-26T19:27:12.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:11 smithi090 ceph-mon[29180]: pgmap v2447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:12.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3516 sec 2024-04-26T19:27:13.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3517 sec 2024-04-26T19:27:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:13 smithi119 ceph-mon[32752]: pgmap v2448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:14.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:13 smithi167 ceph-mon[32751]: pgmap v2448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:13 smithi090 ceph-mon[29180]: pgmap v2448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:14.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3518 sec 2024-04-26T19:27:15.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3519 sec 2024-04-26T19:27:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:15 smithi119 ceph-mon[32752]: pgmap v2449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:16.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3117.022743 secs 2024-04-26T19:27:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:15 smithi167 ceph-mon[32751]: pgmap v2449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:16.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3117.022743 secs 2024-04-26T19:27:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:15 smithi090 ceph-mon[29180]: pgmap v2449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:16.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3117.022743 secs 2024-04-26T19:27:16.197 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3520 sec 2024-04-26T19:27:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3117.702521 secs 2024-04-26T19:27:17.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3117.702521 secs 2024-04-26T19:27:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3117.702521 secs 2024-04-26T19:27:17.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3521 sec 2024-04-26T19:27:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:17 smithi119 ceph-mon[32752]: pgmap v2450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:17 smithi167 ceph-mon[32751]: pgmap v2450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:17 smithi090 ceph-mon[29180]: pgmap v2450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:18.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3522 sec 2024-04-26T19:27:19.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3523 sec 2024-04-26T19:27:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:19 smithi119 ceph-mon[32752]: pgmap v2451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:20.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:19 smithi167 ceph-mon[32751]: pgmap v2451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:19 smithi090 ceph-mon[29180]: pgmap v2451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:20.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3524 sec 2024-04-26T19:27:21.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3122.022827 secs 2024-04-26T19:27:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3122.022827 secs 2024-04-26T19:27:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3122.022827 secs 2024-04-26T19:27:21.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3525 sec 2024-04-26T19:27:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3122.702636 secs 2024-04-26T19:27:22.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:21 smithi119 ceph-mon[32752]: pgmap v2452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:22.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3122.702636 secs 2024-04-26T19:27:22.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:21 smithi090 ceph-mon[29180]: pgmap v2452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:22.161 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3122.702636 secs 2024-04-26T19:27:22.168 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:21 smithi167 ceph-mon[32751]: pgmap v2452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:22.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3526 sec 2024-04-26T19:27:23.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3527 sec 2024-04-26T19:27:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:23 smithi119 ceph-mon[32752]: pgmap v2453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:23 smithi167 ceph-mon[32751]: pgmap v2453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:23 smithi090 ceph-mon[29180]: pgmap v2453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:24.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3528 sec 2024-04-26T19:27:25.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3529 sec 2024-04-26T19:27:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:25 smithi119 ceph-mon[32752]: pgmap v2454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:26.133 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3127.022890 secs 2024-04-26T19:27:26.141 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:25 smithi090 ceph-mon[29180]: pgmap v2454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:26.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3127.022890 secs 2024-04-26T19:27:26.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:25 smithi167 ceph-mon[32751]: pgmap v2454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:26.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3127.022890 secs 2024-04-26T19:27:26.198 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3530 sec 2024-04-26T19:27:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3127.702785 secs 2024-04-26T19:27:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3127.702785 secs 2024-04-26T19:27:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3127.702785 secs 2024-04-26T19:27:27.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3531 sec 2024-04-26T19:27:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:27 smithi119 ceph-mon[32752]: pgmap v2455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:27 smithi167 ceph-mon[32751]: pgmap v2455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:27 smithi090 ceph-mon[29180]: pgmap v2455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:28.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3532 sec 2024-04-26T19:27:29.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3533 sec 2024-04-26T19:27:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:29 smithi119 ceph-mon[32752]: pgmap v2456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:29 smithi167 ceph-mon[32751]: pgmap v2456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:29 smithi090 ceph-mon[29180]: pgmap v2456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:30.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3534 sec 2024-04-26T19:27:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3132.022975 secs 2024-04-26T19:27:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3132.022975 secs 2024-04-26T19:27:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3132.022975 secs 2024-04-26T19:27:31.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3535 sec 2024-04-26T19:27:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3132.702924 secs 2024-04-26T19:27:32.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:31 smithi119 ceph-mon[32752]: pgmap v2457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:32.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3132.702924 secs 2024-04-26T19:27:32.127 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:31 smithi167 ceph-mon[32751]: pgmap v2457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:32.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3132.702924 secs 2024-04-26T19:27:32.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:31 smithi090 ceph-mon[29180]: pgmap v2457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:32.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3536 sec 2024-04-26T19:27:33.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3537 sec 2024-04-26T19:27:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:33 smithi119 ceph-mon[32752]: pgmap v2458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:33 smithi167 ceph-mon[32751]: pgmap v2458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:33 smithi090 ceph-mon[29180]: pgmap v2458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:34.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3538 sec 2024-04-26T19:27:35.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3539 sec 2024-04-26T19:27:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:35 smithi119 ceph-mon[32752]: pgmap v2459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:36.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3137.023061 secs 2024-04-26T19:27:36.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:35 smithi167 ceph-mon[32751]: pgmap v2459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:36.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3137.023061 secs 2024-04-26T19:27:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:35 smithi090 ceph-mon[29180]: pgmap v2459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:36.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3137.023061 secs 2024-04-26T19:27:36.199 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3540 sec 2024-04-26T19:27:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3137.703041 secs 2024-04-26T19:27:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3137.703041 secs 2024-04-26T19:27:37.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3541 sec 2024-04-26T19:27:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3137.703041 secs 2024-04-26T19:27:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:37 smithi167 ceph-mon[32751]: pgmap v2460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:37 smithi090 ceph-mon[29180]: pgmap v2460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:38.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3542 sec 2024-04-26T19:27:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:37 smithi119 ceph-mon[32752]: pgmap v2460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:39.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3543 sec 2024-04-26T19:27:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:39 smithi167 ceph-mon[32751]: pgmap v2461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:39 smithi090 ceph-mon[29180]: pgmap v2461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:40.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3544 sec 2024-04-26T19:27:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:39 smithi119 ceph-mon[32752]: pgmap v2461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3142.023121 secs 2024-04-26T19:27:41.152 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:27:41.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3142.023121 secs 2024-04-26T19:27:41.173 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:27:41.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3545 sec 2024-04-26T19:27:41.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3142.023121 secs 2024-04-26T19:27:41.219 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:27:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3142.703181 secs 2024-04-26T19:27:42.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:41 smithi090 ceph-mon[29180]: pgmap v2462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:42.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:27:42.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:27:42.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:27:42.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3546 sec 2024-04-26T19:27:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3142.703181 secs 2024-04-26T19:27:42.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:41 smithi119 ceph-mon[32752]: pgmap v2462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:42.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:27:42.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:27:42.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:27:42.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3142.703181 secs 2024-04-26T19:27:42.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:41 smithi167 ceph-mon[32751]: pgmap v2462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:42.488 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:27:42.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:27:42.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:27:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:42 smithi090 ceph-mon[29180]: pgmap v2463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:27:43.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:42 smithi090 ceph-mon[29180]: pgmap v2464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:43.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3547 sec 2024-04-26T19:27:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:42 smithi119 ceph-mon[32752]: pgmap v2463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:27:43.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:42 smithi119 ceph-mon[32752]: pgmap v2464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:42 smithi167 ceph-mon[32751]: pgmap v2463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:27:43.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:42 smithi167 ceph-mon[32751]: pgmap v2464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:44.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3548 sec 2024-04-26T19:27:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:44 smithi090 ceph-mon[29180]: pgmap v2465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:45.200 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3549 sec 2024-04-26T19:27:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:44 smithi119 ceph-mon[32752]: pgmap v2465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:44 smithi167 ceph-mon[32751]: pgmap v2465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3147.023203 secs 2024-04-26T19:27:46.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3550 sec 2024-04-26T19:27:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3147.023203 secs 2024-04-26T19:27:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3147.023203 secs 2024-04-26T19:27:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:46 smithi090 ceph-mon[29180]: pgmap v2466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:47.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3147.703287 secs 2024-04-26T19:27:47.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3551 sec 2024-04-26T19:27:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:46 smithi119 ceph-mon[32752]: pgmap v2466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:47.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3147.703287 secs 2024-04-26T19:27:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:46 smithi167 ceph-mon[32751]: pgmap v2466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:47.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3147.703287 secs 2024-04-26T19:27:48.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3552 sec 2024-04-26T19:27:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:48 smithi090 ceph-mon[29180]: pgmap v2467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:27:49.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3553 sec 2024-04-26T19:27:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:48 smithi119 ceph-mon[32752]: pgmap v2467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:27:49.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:48 smithi167 ceph-mon[32751]: pgmap v2467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:27:50.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3554 sec 2024-04-26T19:27:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:49 smithi119 ceph-mon[32752]: pgmap v2468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:49 smithi167 ceph-mon[32751]: pgmap v2468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:49 smithi090 ceph-mon[29180]: pgmap v2468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:27:51.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3555 sec 2024-04-26T19:27:51.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3152.023286 secs 2024-04-26T19:27:51.286 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3152.703416 secs 2024-04-26T19:27:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3152.023286 secs 2024-04-26T19:27:51.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3152.703416 secs 2024-04-26T19:27:51.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3152.023286 secs 2024-04-26T19:27:51.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3152.703416 secs 2024-04-26T19:27:52.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3556 sec 2024-04-26T19:27:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:51 smithi119 ceph-mon[32752]: pgmap v2469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:27:52.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:51 smithi167 ceph-mon[32751]: pgmap v2469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:27:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:51 smithi090 ceph-mon[29180]: pgmap v2469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:27:53.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3557 sec 2024-04-26T19:27:54.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3558 sec 2024-04-26T19:27:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:54 smithi119 ceph-mon[32752]: pgmap v2470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:54 smithi167 ceph-mon[32751]: pgmap v2470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:54 smithi090 ceph-mon[29180]: pgmap v2470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:27:55.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3559 sec 2024-04-26T19:27:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3157.023372 secs 2024-04-26T19:27:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3157.023372 secs 2024-04-26T19:27:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3157.023372 secs 2024-04-26T19:27:56.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3560 sec 2024-04-26T19:27:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:56 smithi119 ceph-mon[32752]: pgmap v2471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:56.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3157.703509 secs 2024-04-26T19:27:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:56 smithi167 ceph-mon[32751]: pgmap v2471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:56.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3157.703509 secs 2024-04-26T19:27:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:56 smithi090 ceph-mon[29180]: pgmap v2471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3157.703509 secs 2024-04-26T19:27:57.201 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3561 sec 2024-04-26T19:27:58.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3562 sec 2024-04-26T19:27:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:27:58 smithi119 ceph-mon[32752]: pgmap v2472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:27:58 smithi167 ceph-mon[32751]: pgmap v2472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:27:58 smithi090 ceph-mon[29180]: pgmap v2472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:27:59.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3563 sec 2024-04-26T19:28:00.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3564 sec 2024-04-26T19:28:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:00 smithi119 ceph-mon[32752]: pgmap v2473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:00.874 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3162.023455 secs 2024-04-26T19:28:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:00 smithi090 ceph-mon[29180]: pgmap v2473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:00.890 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3162.023455 secs 2024-04-26T19:28:00.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:00 smithi167 ceph-mon[32751]: pgmap v2473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:00.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3162.023455 secs 2024-04-26T19:28:01.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3565 sec 2024-04-26T19:28:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3162.703644 secs 2024-04-26T19:28:01.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3162.703644 secs 2024-04-26T19:28:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3162.703644 secs 2024-04-26T19:28:02.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3566 sec 2024-04-26T19:28:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:02 smithi119 ceph-mon[32752]: pgmap v2474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:02 smithi167 ceph-mon[32751]: pgmap v2474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:02 smithi090 ceph-mon[29180]: pgmap v2474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:03.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3567 sec 2024-04-26T19:28:04.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3568 sec 2024-04-26T19:28:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:04 smithi119 ceph-mon[32752]: pgmap v2475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:04.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:04 smithi167 ceph-mon[32751]: pgmap v2475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:04 smithi090 ceph-mon[29180]: pgmap v2475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:05.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3569 sec 2024-04-26T19:28:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3167.023540 secs 2024-04-26T19:28:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3167.023540 secs 2024-04-26T19:28:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3167.023540 secs 2024-04-26T19:28:06.202 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3570 sec 2024-04-26T19:28:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:06 smithi119 ceph-mon[32752]: pgmap v2476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:06.823 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3167.703754 secs 2024-04-26T19:28:06.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:06 smithi167 ceph-mon[32751]: pgmap v2476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:06.850 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3167.703754 secs 2024-04-26T19:28:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:06 smithi090 ceph-mon[29180]: pgmap v2476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3167.703754 secs 2024-04-26T19:28:07.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3571 sec 2024-04-26T19:28:08.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3572 sec 2024-04-26T19:28:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:08 smithi119 ceph-mon[32752]: pgmap v2477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:08 smithi167 ceph-mon[32751]: pgmap v2477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:08 smithi090 ceph-mon[29180]: pgmap v2477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:09.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3573 sec 2024-04-26T19:28:10.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3574 sec 2024-04-26T19:28:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:10 smithi119 ceph-mon[32752]: pgmap v2478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:10.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3172.023613 secs 2024-04-26T19:28:10.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:10 smithi167 ceph-mon[32751]: pgmap v2478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:10.861 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3172.023613 secs 2024-04-26T19:28:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:10 smithi090 ceph-mon[29180]: pgmap v2478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:10.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3172.023613 secs 2024-04-26T19:28:11.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3575 sec 2024-04-26T19:28:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3172.703865 secs 2024-04-26T19:28:11.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3172.703865 secs 2024-04-26T19:28:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3172.703865 secs 2024-04-26T19:28:12.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3576 sec 2024-04-26T19:28:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:12 smithi119 ceph-mon[32752]: pgmap v2479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:12.846 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:12 smithi167 ceph-mon[32751]: pgmap v2479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:12 smithi090 ceph-mon[29180]: pgmap v2479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:13.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3577 sec 2024-04-26T19:28:14.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3578 sec 2024-04-26T19:28:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:14 smithi119 ceph-mon[32752]: pgmap v2480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:14 smithi167 ceph-mon[32751]: pgmap v2480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:14 smithi090 ceph-mon[29180]: pgmap v2480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T19:28:15.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3579 sec 2024-04-26T19:28:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3177.023698 secs 2024-04-26T19:28:15.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3177.023698 secs 2024-04-26T19:28:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3177.023698 secs 2024-04-26T19:28:16.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3580 sec 2024-04-26T19:28:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:16 smithi119 ceph-mon[32752]: pgmap v2481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:16.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3177.703975 secs 2024-04-26T19:28:16.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:16 smithi167 ceph-mon[32751]: pgmap v2481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:16.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3177.703975 secs 2024-04-26T19:28:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:16 smithi090 ceph-mon[29180]: pgmap v2481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:16.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3177.703975 secs 2024-04-26T19:28:17.203 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3581 sec 2024-04-26T19:28:18.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3582 sec 2024-04-26T19:28:18.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:18 smithi090 ceph-mon[29180]: pgmap v2482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:18 smithi119 ceph-mon[32752]: pgmap v2482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:18 smithi167 ceph-mon[32751]: pgmap v2482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:19.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3583 sec 2024-04-26T19:28:20.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3584 sec 2024-04-26T19:28:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:20 smithi119 ceph-mon[32752]: pgmap v2483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:20.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3182.023778 secs 2024-04-26T19:28:20.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:20 smithi167 ceph-mon[32751]: pgmap v2483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:20.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3182.023778 secs 2024-04-26T19:28:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:20 smithi090 ceph-mon[29180]: pgmap v2483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3182.023778 secs 2024-04-26T19:28:21.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3585 sec 2024-04-26T19:28:21.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3182.704038 secs 2024-04-26T19:28:21.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3182.704038 secs 2024-04-26T19:28:21.839 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3182.704038 secs 2024-04-26T19:28:22.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3586 sec 2024-04-26T19:28:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:22 smithi119 ceph-mon[32752]: pgmap v2484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:22.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:22 smithi167 ceph-mon[32751]: pgmap v2484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:22 smithi090 ceph-mon[29180]: pgmap v2484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:23.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3587 sec 2024-04-26T19:28:24.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3588 sec 2024-04-26T19:28:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:24 smithi119 ceph-mon[32752]: pgmap v2485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:24.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:24 smithi167 ceph-mon[32751]: pgmap v2485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:24 smithi090 ceph-mon[29180]: pgmap v2485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:25.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3589 sec 2024-04-26T19:28:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3187.023830 secs 2024-04-26T19:28:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3187.023830 secs 2024-04-26T19:28:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3187.023830 secs 2024-04-26T19:28:26.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3590 sec 2024-04-26T19:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:26 smithi119 ceph-mon[32752]: pgmap v2486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:26.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3187.704123 secs 2024-04-26T19:28:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:26 smithi167 ceph-mon[32751]: pgmap v2486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:26.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3187.704123 secs 2024-04-26T19:28:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:26 smithi090 ceph-mon[29180]: pgmap v2486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:26.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3187.704123 secs 2024-04-26T19:28:27.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3591 sec 2024-04-26T19:28:28.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3592 sec 2024-04-26T19:28:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:28 smithi119 ceph-mon[32752]: pgmap v2487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:28.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:28 smithi167 ceph-mon[32751]: pgmap v2487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:28 smithi090 ceph-mon[29180]: pgmap v2487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:29.204 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3593 sec 2024-04-26T19:28:30.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3594 sec 2024-04-26T19:28:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:30 smithi119 ceph-mon[32752]: pgmap v2488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:30.820 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 3192.023889 secs 2024-04-26T19:28:30.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:30 smithi167 ceph-mon[32751]: pgmap v2488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:30.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 3192.023889 secs 2024-04-26T19:28:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:30 smithi090 ceph-mon[29180]: pgmap v2488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:30.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 3192.023889 secs 2024-04-26T19:28:31.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3595 sec 2024-04-26T19:28:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3192.704257 secs 2024-04-26T19:28:31.874 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3192.704257 secs 2024-04-26T19:28:31.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3192.704257 secs 2024-04-26T19:28:31.892 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:28:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:28:31.610+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:28:31.901 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:28:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:28:31.610+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:28:31.910 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:28:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:28:31.610+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:28:31.917 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:28:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:28:31.610+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:28:31.927 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:28:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:28:31.610+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:28:32.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3596 sec 2024-04-26T19:28:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:32 smithi119 ceph-mon[32752]: pgmap v2489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:32 smithi167 ceph-mon[32751]: pgmap v2489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:32.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:32 smithi090 ceph-mon[29180]: pgmap v2489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:33.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3597 sec 2024-04-26T19:28:34.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3598 sec 2024-04-26T19:28:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:34 smithi119 ceph-mon[32752]: pgmap v2490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:28:34.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:34 smithi167 ceph-mon[32751]: pgmap v2490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:28:34.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:34 smithi090 ceph-mon[29180]: pgmap v2490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:28:35.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3599 sec 2024-04-26T19:28:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:35 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3197.023972 secs 2024-04-26T19:28:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:35 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3197.023972 secs 2024-04-26T19:28:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:35 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3197.023972 secs 2024-04-26T19:28:36.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3600 sec 2024-04-26T19:28:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:36 smithi167 ceph-mon[32751]: pgmap v2491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:36.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3197.704316 secs 2024-04-26T19:28:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:36 smithi090 ceph-mon[29180]: pgmap v2491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:36.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3197.704316 secs 2024-04-26T19:28:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:36 smithi119 ceph-mon[32752]: pgmap v2491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:37.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3197.704316 secs 2024-04-26T19:28:37.205 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3601 sec 2024-04-26T19:28:38.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3602 sec 2024-04-26T19:28:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:38 smithi090 ceph-mon[29180]: pgmap v2492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:38.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:28:38.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:28:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:38 smithi119 ceph-mon[32752]: pgmap v2492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:39.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:28:39.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:28:39.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:38 smithi167 ceph-mon[32751]: pgmap v2492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:39.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:28:39.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:28:39.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3603 sec 2024-04-26T19:28:40.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3604 sec 2024-04-26T19:28:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:40 smithi090 ceph-mon[29180]: pgmap v2493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:28:40.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:40 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3202.024053 secs 2024-04-26T19:28:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:40 smithi119 ceph-mon[32752]: pgmap v2493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:28:41.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:40 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3202.024053 secs 2024-04-26T19:28:41.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:40 smithi167 ceph-mon[32751]: pgmap v2493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:28:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:40 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3202.024053 secs 2024-04-26T19:28:41.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3605 sec 2024-04-26T19:28:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3202.704407 secs 2024-04-26T19:28:41.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:28:41.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3202.704407 secs 2024-04-26T19:28:41.962 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:28:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3202.704407 secs 2024-04-26T19:28:42.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:28:42.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3606 sec 2024-04-26T19:28:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:42 smithi090 ceph-mon[29180]: pgmap v2494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:42.890 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:28:42.900 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:28:42.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:28:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:42 smithi119 ceph-mon[32752]: pgmap v2494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:43.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:28:43.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:28:43.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:28:43.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:42 smithi167 ceph-mon[32751]: pgmap v2494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:28:43.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:28:43.133 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:28:43.141 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:28:43.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3607 sec 2024-04-26T19:28:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:43 smithi090 ceph-mon[29180]: pgmap v2495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 190 B/s wr, 2 op/s 2024-04-26T19:28:43.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:43 smithi090 ceph-mon[29180]: pgmap v2496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:28:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:43 smithi119 ceph-mon[32752]: pgmap v2495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 190 B/s wr, 2 op/s 2024-04-26T19:28:44.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:43 smithi119 ceph-mon[32752]: pgmap v2496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:28:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:43 smithi167 ceph-mon[32751]: pgmap v2495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 190 B/s wr, 2 op/s 2024-04-26T19:28:44.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:43 smithi167 ceph-mon[32751]: pgmap v2496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:28:44.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3608 sec 2024-04-26T19:28:45.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3609 sec 2024-04-26T19:28:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:45 smithi090 ceph-mon[29180]: pgmap v2497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:45.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:45 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3207.024138 secs 2024-04-26T19:28:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:45 smithi119 ceph-mon[32752]: pgmap v2497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:46.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:45 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3207.024138 secs 2024-04-26T19:28:46.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:45 smithi167 ceph-mon[32751]: pgmap v2497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:46.140 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:45 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3207.024138 secs 2024-04-26T19:28:46.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3610 sec 2024-04-26T19:28:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3207.704526 secs 2024-04-26T19:28:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3207.704526 secs 2024-04-26T19:28:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3207.704526 secs 2024-04-26T19:28:47.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3611 sec 2024-04-26T19:28:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:47 smithi090 ceph-mon[29180]: pgmap v2498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:47 smithi119 ceph-mon[32752]: pgmap v2498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:47 smithi167 ceph-mon[32751]: pgmap v2498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:48.206 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3612 sec 2024-04-26T19:28:49.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3613 sec 2024-04-26T19:28:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:49 smithi090 ceph-mon[29180]: pgmap v2499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:28:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:49 smithi119 ceph-mon[32752]: pgmap v2499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:28:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:49 smithi167 ceph-mon[32751]: pgmap v2499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:28:50.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3614 sec 2024-04-26T19:28:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:50 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3212.024218 secs 2024-04-26T19:28:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:50 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3212.024218 secs 2024-04-26T19:28:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:50 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3212.024218 secs 2024-04-26T19:28:51.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3615 sec 2024-04-26T19:28:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:51 smithi090 ceph-mon[29180]: pgmap v2500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:51.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3212.704637 secs 2024-04-26T19:28:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:51 smithi119 ceph-mon[32752]: pgmap v2500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:52.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3212.704637 secs 2024-04-26T19:28:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:51 smithi167 ceph-mon[32751]: pgmap v2500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:28:52.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3212.704637 secs 2024-04-26T19:28:52.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3616 sec 2024-04-26T19:28:53.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3617 sec 2024-04-26T19:28:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:53 smithi090 ceph-mon[29180]: pgmap v2501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:28:54.139 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:53 smithi167 ceph-mon[32751]: pgmap v2501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:28:54.149 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:53 smithi119 ceph-mon[32752]: pgmap v2501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:28:54.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3618 sec 2024-04-26T19:28:55.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3619 sec 2024-04-26T19:28:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:55 smithi090 ceph-mon[29180]: pgmap v2502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:55.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:55 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3217.024280 secs 2024-04-26T19:28:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:55 smithi119 ceph-mon[32752]: pgmap v2502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:56.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:55 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3217.024280 secs 2024-04-26T19:28:56.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:55 smithi167 ceph-mon[32751]: pgmap v2502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:28:56.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:55 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3217.024280 secs 2024-04-26T19:28:56.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3620 sec 2024-04-26T19:28:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3217.704748 secs 2024-04-26T19:28:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3217.704748 secs 2024-04-26T19:28:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3217.704748 secs 2024-04-26T19:28:57.207 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3621 sec 2024-04-26T19:28:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:57 smithi090 ceph-mon[29180]: pgmap v2503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:57 smithi119 ceph-mon[32752]: pgmap v2503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:57 smithi167 ceph-mon[32751]: pgmap v2503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:28:58.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3622 sec 2024-04-26T19:28:59.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3623 sec 2024-04-26T19:28:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:28:59 smithi090 ceph-mon[29180]: pgmap v2504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:28:59 smithi119 ceph-mon[32752]: pgmap v2504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:28:59 smithi167 ceph-mon[32751]: pgmap v2504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:00.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3624 sec 2024-04-26T19:29:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:00 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3222.024355 secs 2024-04-26T19:29:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:00 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3222.024355 secs 2024-04-26T19:29:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:00 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3222.024355 secs 2024-04-26T19:29:01.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3625 sec 2024-04-26T19:29:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:01 smithi090 ceph-mon[29180]: pgmap v2505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:01.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3222.704884 secs 2024-04-26T19:29:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:01 smithi167 ceph-mon[32751]: pgmap v2505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:01.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3222.704884 secs 2024-04-26T19:29:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:01 smithi119 ceph-mon[32752]: pgmap v2505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:02.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3222.704884 secs 2024-04-26T19:29:02.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3626 sec 2024-04-26T19:29:03.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3627 sec 2024-04-26T19:29:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:03 smithi119 ceph-mon[32752]: pgmap v2506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:04.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:03 smithi167 ceph-mon[32751]: pgmap v2506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:03 smithi090 ceph-mon[29180]: pgmap v2506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:04.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3628 sec 2024-04-26T19:29:05.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3629 sec 2024-04-26T19:29:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:05 smithi090 ceph-mon[29180]: pgmap v2507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:05.944 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:05 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3227.024439 secs 2024-04-26T19:29:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:05 smithi119 ceph-mon[32752]: pgmap v2507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:06.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:05 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3227.024439 secs 2024-04-26T19:29:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:05 smithi167 ceph-mon[32751]: pgmap v2507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:06.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:05 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3227.024439 secs 2024-04-26T19:29:06.208 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3630 sec 2024-04-26T19:29:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3227.704955 secs 2024-04-26T19:29:07.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3227.704955 secs 2024-04-26T19:29:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3227.704955 secs 2024-04-26T19:29:07.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3631 sec 2024-04-26T19:29:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:07 smithi119 ceph-mon[32752]: pgmap v2508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:08.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:07 smithi167 ceph-mon[32751]: pgmap v2508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:07 smithi090 ceph-mon[29180]: pgmap v2508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:08.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3632 sec 2024-04-26T19:29:09.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3633 sec 2024-04-26T19:29:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:09 smithi119 ceph-mon[32752]: pgmap v2509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:10.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:09 smithi167 ceph-mon[32751]: pgmap v2509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:09 smithi090 ceph-mon[29180]: pgmap v2509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:10.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3634 sec 2024-04-26T19:29:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:10 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3232.024517 secs 2024-04-26T19:29:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:10 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3232.024517 secs 2024-04-26T19:29:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:10 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3232.024517 secs 2024-04-26T19:29:11.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3635 sec 2024-04-26T19:29:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:11 smithi119 ceph-mon[32752]: pgmap v2510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:12.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3232.705095 secs 2024-04-26T19:29:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:11 smithi167 ceph-mon[32751]: pgmap v2510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:12.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3232.705095 secs 2024-04-26T19:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:11 smithi090 ceph-mon[29180]: pgmap v2510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:12.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3232.705095 secs 2024-04-26T19:29:12.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3636 sec 2024-04-26T19:29:13.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3637 sec 2024-04-26T19:29:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:13 smithi119 ceph-mon[32752]: pgmap v2511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:14.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:13 smithi167 ceph-mon[32751]: pgmap v2511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:13 smithi090 ceph-mon[29180]: pgmap v2511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:14.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3638 sec 2024-04-26T19:29:15.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3639 sec 2024-04-26T19:29:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:15 smithi119 ceph-mon[32752]: pgmap v2512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:16.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:15 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3237.024600 secs 2024-04-26T19:29:16.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:15 smithi167 ceph-mon[32751]: pgmap v2512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:16.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:15 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3237.024600 secs 2024-04-26T19:29:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:15 smithi090 ceph-mon[29180]: pgmap v2512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:16.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:15 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3237.024600 secs 2024-04-26T19:29:16.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3640 sec 2024-04-26T19:29:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3237.705207 secs 2024-04-26T19:29:17.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3237.705207 secs 2024-04-26T19:29:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3237.705207 secs 2024-04-26T19:29:17.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3641 sec 2024-04-26T19:29:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:17 smithi119 ceph-mon[32752]: pgmap v2513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:18.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:17 smithi167 ceph-mon[32751]: pgmap v2513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:17 smithi090 ceph-mon[29180]: pgmap v2513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:18.209 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3642 sec 2024-04-26T19:29:19.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3643 sec 2024-04-26T19:29:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:19 smithi119 ceph-mon[32752]: pgmap v2514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:20.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:19 smithi167 ceph-mon[32751]: pgmap v2514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:20.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:19 smithi090 ceph-mon[29180]: pgmap v2514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:20.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3644 sec 2024-04-26T19:29:21.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:20 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3242.024680 secs 2024-04-26T19:29:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:20 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3242.024680 secs 2024-04-26T19:29:21.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:20 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3242.024680 secs 2024-04-26T19:29:21.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3645 sec 2024-04-26T19:29:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:21 smithi119 ceph-mon[32752]: pgmap v2515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:22.073 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3242.705298 secs 2024-04-26T19:29:22.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:21 smithi167 ceph-mon[32751]: pgmap v2515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:22.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3242.705298 secs 2024-04-26T19:29:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:21 smithi090 ceph-mon[29180]: pgmap v2515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:22.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3242.705298 secs 2024-04-26T19:29:22.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3646 sec 2024-04-26T19:29:23.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3647 sec 2024-04-26T19:29:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:23 smithi119 ceph-mon[32752]: pgmap v2516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:24.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:23 smithi167 ceph-mon[32751]: pgmap v2516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:23 smithi090 ceph-mon[29180]: pgmap v2516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:24.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3648 sec 2024-04-26T19:29:25.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3649 sec 2024-04-26T19:29:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:25 smithi119 ceph-mon[32752]: pgmap v2517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:26.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:25 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3247.024765 secs 2024-04-26T19:29:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:25 smithi167 ceph-mon[32751]: pgmap v2517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:26.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:25 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3247.024765 secs 2024-04-26T19:29:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:25 smithi090 ceph-mon[29180]: pgmap v2517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:25 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3247.024765 secs 2024-04-26T19:29:26.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3650 sec 2024-04-26T19:29:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3247.705436 secs 2024-04-26T19:29:27.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3247.705436 secs 2024-04-26T19:29:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3247.705436 secs 2024-04-26T19:29:27.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3651 sec 2024-04-26T19:29:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:27 smithi119 ceph-mon[32752]: pgmap v2518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:28.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:27 smithi167 ceph-mon[32751]: pgmap v2518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:27 smithi090 ceph-mon[29180]: pgmap v2518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:28.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3652 sec 2024-04-26T19:29:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:29:28.895+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:29:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:29:28.895+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:29:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:29:28.896+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 29 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:29:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:29:28.896+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 29 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006490 con 0x56302893f0e0 2024-04-26T19:29:29.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3653 sec 2024-04-26T19:29:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:29 smithi119 ceph-mon[32752]: pgmap v2519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:30.113 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:29 smithi167 ceph-mon[32751]: pgmap v2519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:29 smithi090 ceph-mon[29180]: pgmap v2519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:30.210 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3654 sec 2024-04-26T19:29:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:30 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3252.024821 secs 2024-04-26T19:29:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:30 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3252.024821 secs 2024-04-26T19:29:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:30 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3252.024821 secs 2024-04-26T19:29:31.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3655 sec 2024-04-26T19:29:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:31 smithi167 ceph-mon[32751]: pgmap v2520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:31.988 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3252.705518 secs 2024-04-26T19:29:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:31 smithi119 ceph-mon[32752]: pgmap v2520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:32.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3252.705518 secs 2024-04-26T19:29:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:31 smithi090 ceph-mon[29180]: pgmap v2520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:32.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3252.705518 secs 2024-04-26T19:29:32.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3656 sec 2024-04-26T19:29:33.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3657 sec 2024-04-26T19:29:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:33 smithi119 ceph-mon[32752]: pgmap v2521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:34.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:33 smithi167 ceph-mon[32751]: pgmap v2521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:33 smithi090 ceph-mon[29180]: pgmap v2521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:34.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3658 sec 2024-04-26T19:29:35.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3659 sec 2024-04-26T19:29:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:35 smithi119 ceph-mon[32752]: pgmap v2522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:36.124 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:35 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3257.024882 secs 2024-04-26T19:29:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:35 smithi090 ceph-mon[29180]: pgmap v2522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:36.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:35 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3257.024882 secs 2024-04-26T19:29:36.147 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:35 smithi167 ceph-mon[32751]: pgmap v2522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:36.194 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:35 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3257.024882 secs 2024-04-26T19:29:36.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3660 sec 2024-04-26T19:29:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 3257.705624 secs 2024-04-26T19:29:37.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:36 smithi119 ceph-mon[32752]: slow request 2884.574637 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:29:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 3257.705624 secs 2024-04-26T19:29:37.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:36 smithi167 ceph-mon[32751]: slow request 2884.574637 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:29:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 3257.705624 secs 2024-04-26T19:29:37.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:36 smithi090 ceph-mon[29180]: slow request 2884.574637 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:29:37.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3661 sec 2024-04-26T19:29:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:37 smithi119 ceph-mon[32752]: pgmap v2523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:37 smithi167 ceph-mon[32751]: pgmap v2523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:37 smithi090 ceph-mon[29180]: pgmap v2523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:38.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3662 sec 2024-04-26T19:29:39.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3663 sec 2024-04-26T19:29:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:39 smithi119 ceph-mon[32752]: pgmap v2524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:39 smithi167 ceph-mon[32751]: pgmap v2524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:39 smithi090 ceph-mon[29180]: pgmap v2524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:40.211 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3664 sec 2024-04-26T19:29:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:40 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3262.024941 secs 2024-04-26T19:29:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:40 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3262.024941 secs 2024-04-26T19:29:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:40 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3262.024941 secs 2024-04-26T19:29:41.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3665 sec 2024-04-26T19:29:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:41 smithi119 ceph-mon[32752]: pgmap v2525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:42.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3262.705707 secs 2024-04-26T19:29:42.139 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:41 smithi090 ceph-mon[29180]: pgmap v2525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:42.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3262.705707 secs 2024-04-26T19:29:42.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:41 smithi167 ceph-mon[32751]: pgmap v2525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:42.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3262.705707 secs 2024-04-26T19:29:42.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3666 sec 2024-04-26T19:29:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:29:43.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:29:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:29:43.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3667 sec 2024-04-26T19:29:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:43 smithi119 ceph-mon[32752]: pgmap v2526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:44.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:29:44.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:29:44.087 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:29:44.101 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:43 smithi167 ceph-mon[32751]: pgmap v2526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:44.180 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:29:44.187 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:29:44.266 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:29:44.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:43 smithi090 ceph-mon[29180]: pgmap v2526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:44.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:29:44.290 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:29:44.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:29:44.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3668 sec 2024-04-26T19:29:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:44 smithi119 ceph-mon[32752]: pgmap v2527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:29:45.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:44 smithi119 ceph-mon[32752]: pgmap v2528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:29:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:44 smithi167 ceph-mon[32751]: pgmap v2527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:29:45.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:44 smithi167 ceph-mon[32751]: pgmap v2528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:29:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:44 smithi090 ceph-mon[29180]: pgmap v2527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:29:45.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:44 smithi090 ceph-mon[29180]: pgmap v2528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:29:45.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3669 sec 2024-04-26T19:29:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:45 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3267.025004 secs 2024-04-26T19:29:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:45 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3267.025004 secs 2024-04-26T19:29:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:45 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3267.025004 secs 2024-04-26T19:29:46.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3670 sec 2024-04-26T19:29:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:46 smithi119 ceph-mon[32752]: pgmap v2529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:47.110 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3267.705826 secs 2024-04-26T19:29:47.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:46 smithi167 ceph-mon[32751]: pgmap v2529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:47.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3267.705826 secs 2024-04-26T19:29:47.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:46 smithi090 ceph-mon[29180]: pgmap v2529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:47.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3267.705826 secs 2024-04-26T19:29:47.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3671 sec 2024-04-26T19:29:48.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3672 sec 2024-04-26T19:29:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:48 smithi119 ceph-mon[32752]: pgmap v2530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:48 smithi167 ceph-mon[32751]: pgmap v2530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:48 smithi090 ceph-mon[29180]: pgmap v2530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:49.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3673 sec 2024-04-26T19:29:50.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3674 sec 2024-04-26T19:29:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:50 smithi119 ceph-mon[32752]: pgmap v2531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:51.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:50 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3272.025065 secs 2024-04-26T19:29:51.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:50 smithi167 ceph-mon[32751]: pgmap v2531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:51.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:50 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3272.025065 secs 2024-04-26T19:29:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:50 smithi090 ceph-mon[29180]: pgmap v2531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:51.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:50 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3272.025065 secs 2024-04-26T19:29:51.212 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3675 sec 2024-04-26T19:29:52.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3272.705946 secs 2024-04-26T19:29:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3272.705946 secs 2024-04-26T19:29:52.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3272.705946 secs 2024-04-26T19:29:52.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3676 sec 2024-04-26T19:29:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:52 smithi090 ceph-mon[29180]: pgmap v2532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:52 smithi119 ceph-mon[32752]: pgmap v2532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:52 smithi167 ceph-mon[32751]: pgmap v2532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:29:53.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3677 sec 2024-04-26T19:29:54.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3678 sec 2024-04-26T19:29:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:54 smithi119 ceph-mon[32752]: pgmap v2533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:29:55.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:54 smithi167 ceph-mon[32751]: pgmap v2533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:29:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:54 smithi090 ceph-mon[29180]: pgmap v2533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:29:55.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3679 sec 2024-04-26T19:29:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:55 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3277.025128 secs 2024-04-26T19:29:56.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:55 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3277.025128 secs 2024-04-26T19:29:56.162 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:55 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3277.025128 secs 2024-04-26T19:29:56.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3680 sec 2024-04-26T19:29:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:56 smithi119 ceph-mon[32752]: pgmap v2534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:57.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3277.706059 secs 2024-04-26T19:29:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:56 smithi167 ceph-mon[32751]: pgmap v2534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:57.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3277.706059 secs 2024-04-26T19:29:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:56 smithi090 ceph-mon[29180]: pgmap v2534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:29:57.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3277.706059 secs 2024-04-26T19:29:57.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3681 sec 2024-04-26T19:29:58.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3682 sec 2024-04-26T19:29:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:29:58 smithi119 ceph-mon[32752]: pgmap v2535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:59.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:29:58 smithi167 ceph-mon[32751]: pgmap v2535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:29:58 smithi090 ceph-mon[29180]: pgmap v2535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:29:59.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3683 sec 2024-04-26T19:30:00.213 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3684 sec 2024-04-26T19:30:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:00 smithi119 ceph-mon[32752]: pgmap v2536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:01.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:00 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3282.025189 secs 2024-04-26T19:30:01.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:30:01.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:00 smithi167 ceph-mon[32751]: pgmap v2536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:01.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:00 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3282.025189 secs 2024-04-26T19:30:01.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:30:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:00 smithi090 ceph-mon[29180]: pgmap v2536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:01.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:00 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3282.025189 secs 2024-04-26T19:30:01.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:30:01.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3685 sec 2024-04-26T19:30:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3282.706181 secs 2024-04-26T19:30:02.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3282.706181 secs 2024-04-26T19:30:02.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3282.706181 secs 2024-04-26T19:30:02.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3686 sec 2024-04-26T19:30:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:02 smithi119 ceph-mon[32752]: pgmap v2537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:02 smithi167 ceph-mon[32751]: pgmap v2537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:02 smithi090 ceph-mon[29180]: pgmap v2537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:03.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3687 sec 2024-04-26T19:30:04.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3688 sec 2024-04-26T19:30:05.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:04 smithi119 ceph-mon[32752]: pgmap v2538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:05.263 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:04 smithi090 ceph-mon[29180]: pgmap v2538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:05.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:04 smithi167 ceph-mon[32751]: pgmap v2538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:05.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3689 sec 2024-04-26T19:30:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:05 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3287.025250 secs 2024-04-26T19:30:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:05 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3287.025250 secs 2024-04-26T19:30:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:05 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3287.025250 secs 2024-04-26T19:30:06.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3690 sec 2024-04-26T19:30:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:06 smithi119 ceph-mon[32752]: pgmap v2539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:07.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3287.706308 secs 2024-04-26T19:30:07.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:06 smithi167 ceph-mon[32751]: pgmap v2539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:07.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3287.706308 secs 2024-04-26T19:30:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:06 smithi090 ceph-mon[29180]: pgmap v2539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:07.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3287.706308 secs 2024-04-26T19:30:07.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3691 sec 2024-04-26T19:30:08.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3692 sec 2024-04-26T19:30:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:08 smithi119 ceph-mon[32752]: pgmap v2540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:08 smithi167 ceph-mon[32751]: pgmap v2540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:08 smithi090 ceph-mon[29180]: pgmap v2540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:09.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3693 sec 2024-04-26T19:30:10.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3694 sec 2024-04-26T19:30:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:10 smithi119 ceph-mon[32752]: pgmap v2541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:11.114 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:10 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3292.025332 secs 2024-04-26T19:30:11.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:10 smithi167 ceph-mon[32751]: pgmap v2541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:11.131 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:10 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3292.025332 secs 2024-04-26T19:30:11.146 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:10 smithi090 ceph-mon[29180]: pgmap v2541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:11.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:10 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3292.025332 secs 2024-04-26T19:30:11.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3695 sec 2024-04-26T19:30:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3292.706448 secs 2024-04-26T19:30:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3292.706448 secs 2024-04-26T19:30:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3292.706448 secs 2024-04-26T19:30:12.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3696 sec 2024-04-26T19:30:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:12 smithi119 ceph-mon[32752]: pgmap v2542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:12 smithi167 ceph-mon[32751]: pgmap v2542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:12 smithi090 ceph-mon[29180]: pgmap v2542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:13.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3697 sec 2024-04-26T19:30:14.214 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3698 sec 2024-04-26T19:30:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:14 smithi119 ceph-mon[32752]: pgmap v2543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:14 smithi167 ceph-mon[32751]: pgmap v2543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:14 smithi090 ceph-mon[29180]: pgmap v2543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:15.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3699 sec 2024-04-26T19:30:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:15 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3297.025416 secs 2024-04-26T19:30:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:15 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3297.025416 secs 2024-04-26T19:30:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:15 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3297.025416 secs 2024-04-26T19:30:16.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3700 sec 2024-04-26T19:30:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:16 smithi167 ceph-mon[32751]: pgmap v2544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:17.126 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3297.706507 secs 2024-04-26T19:30:17.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:16 smithi090 ceph-mon[29180]: pgmap v2544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:17.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3297.706507 secs 2024-04-26T19:30:17.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3701 sec 2024-04-26T19:30:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:16 smithi119 ceph-mon[32752]: pgmap v2544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3297.706507 secs 2024-04-26T19:30:18.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3702 sec 2024-04-26T19:30:19.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:18 smithi167 ceph-mon[32751]: pgmap v2545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:18 smithi090 ceph-mon[29180]: pgmap v2545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:19.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3703 sec 2024-04-26T19:30:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:18 smithi119 ceph-mon[32752]: pgmap v2545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:20.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3704 sec 2024-04-26T19:30:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:20 smithi167 ceph-mon[32751]: pgmap v2546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:21.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:20 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3302.025496 secs 2024-04-26T19:30:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:20 smithi090 ceph-mon[29180]: pgmap v2546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:21.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:20 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3302.025496 secs 2024-04-26T19:30:21.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3705 sec 2024-04-26T19:30:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:20 smithi119 ceph-mon[32752]: pgmap v2546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:21.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:20 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3302.025496 secs 2024-04-26T19:30:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3302.706619 secs 2024-04-26T19:30:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3302.706619 secs 2024-04-26T19:30:22.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3706 sec 2024-04-26T19:30:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3302.706619 secs 2024-04-26T19:30:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:22 smithi090 ceph-mon[29180]: pgmap v2547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:23.215 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3707 sec 2024-04-26T19:30:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:22 smithi119 ceph-mon[32752]: pgmap v2547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:22 smithi167 ceph-mon[32751]: pgmap v2547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:24.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3708 sec 2024-04-26T19:30:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:24 smithi090 ceph-mon[29180]: pgmap v2548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:25.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3709 sec 2024-04-26T19:30:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:24 smithi119 ceph-mon[32752]: pgmap v2548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:24 smithi167 ceph-mon[32751]: pgmap v2548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:25 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3307.025581 secs 2024-04-26T19:30:26.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3710 sec 2024-04-26T19:30:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:25 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3307.025581 secs 2024-04-26T19:30:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:25 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3307.025581 secs 2024-04-26T19:30:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:26 smithi090 ceph-mon[29180]: pgmap v2549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:27.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3307.706689 secs 2024-04-26T19:30:27.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3711 sec 2024-04-26T19:30:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:26 smithi119 ceph-mon[32752]: pgmap v2549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:27.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3307.706689 secs 2024-04-26T19:30:27.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:26 smithi167 ceph-mon[32751]: pgmap v2549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:27.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3307.706689 secs 2024-04-26T19:30:28.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3712 sec 2024-04-26T19:30:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:28 smithi090 ceph-mon[29180]: pgmap v2550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:29.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3713 sec 2024-04-26T19:30:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:28 smithi119 ceph-mon[32752]: pgmap v2550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:28 smithi167 ceph-mon[32751]: pgmap v2550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:30.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3714 sec 2024-04-26T19:30:31.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:30 smithi167 ceph-mon[32751]: pgmap v2551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:31.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:30 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3312.025662 secs 2024-04-26T19:30:31.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:30 smithi090 ceph-mon[29180]: pgmap v2551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:31.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:30 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3312.025662 secs 2024-04-26T19:30:31.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3715 sec 2024-04-26T19:30:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:30 smithi119 ceph-mon[32752]: pgmap v2551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:31.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:30 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3312.025662 secs 2024-04-26T19:30:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3312.706767 secs 2024-04-26T19:30:32.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3716 sec 2024-04-26T19:30:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3312.706767 secs 2024-04-26T19:30:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3312.706767 secs 2024-04-26T19:30:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:32 smithi090 ceph-mon[29180]: pgmap v2552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:33.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3717 sec 2024-04-26T19:30:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:32 smithi119 ceph-mon[32752]: pgmap v2552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:32 smithi167 ceph-mon[32751]: pgmap v2552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:34.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3718 sec 2024-04-26T19:30:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:34 smithi090 ceph-mon[29180]: pgmap v2553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:35.216 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3719 sec 2024-04-26T19:30:35.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:34 smithi119 ceph-mon[32752]: pgmap v2553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:34 smithi167 ceph-mon[32751]: pgmap v2553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:35 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3317.025746 secs 2024-04-26T19:30:36.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3720 sec 2024-04-26T19:30:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:35 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3317.025746 secs 2024-04-26T19:30:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:35 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3317.025746 secs 2024-04-26T19:30:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:36 smithi090 ceph-mon[29180]: pgmap v2554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:37.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3317.706826 secs 2024-04-26T19:30:37.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3721 sec 2024-04-26T19:30:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:36 smithi119 ceph-mon[32752]: pgmap v2554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:37.318 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3317.706826 secs 2024-04-26T19:30:37.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:36 smithi167 ceph-mon[32751]: pgmap v2554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:37.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3317.706826 secs 2024-04-26T19:30:38.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3722 sec 2024-04-26T19:30:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:38 smithi090 ceph-mon[29180]: pgmap v2555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:39.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3723 sec 2024-04-26T19:30:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:38 smithi119 ceph-mon[32752]: pgmap v2555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:38 smithi167 ceph-mon[32751]: pgmap v2555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:40.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3724 sec 2024-04-26T19:30:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:39 smithi119 ceph-mon[32752]: pgmap v2556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:39 smithi167 ceph-mon[32751]: pgmap v2556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:39 smithi090 ceph-mon[29180]: pgmap v2556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:30:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:40 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3322.025819 secs 2024-04-26T19:30:41.221 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3322.706969 secs 2024-04-26T19:30:41.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3725 sec 2024-04-26T19:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:40 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3322.025819 secs 2024-04-26T19:30:41.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3322.706969 secs 2024-04-26T19:30:41.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:40 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3322.025819 secs 2024-04-26T19:30:41.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3322.706969 secs 2024-04-26T19:30:42.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3726 sec 2024-04-26T19:30:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:41 smithi119 ceph-mon[32752]: pgmap v2557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:41 smithi167 ceph-mon[32751]: pgmap v2557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:41 smithi090 ceph-mon[29180]: pgmap v2557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:43.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3727 sec 2024-04-26T19:30:44.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3728 sec 2024-04-26T19:30:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:44 smithi119 ceph-mon[32752]: pgmap v2558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:44.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:30:44.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:30:44.331 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:30:44.339 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:44 smithi119 ceph-mon[32752]: pgmap v2559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:30:44.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:44 smithi119 ceph-mon[32752]: pgmap v2560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:44.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:30:44.367 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:44 smithi167 ceph-mon[32751]: pgmap v2558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:44.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:30:44.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:30:44.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:30:44.404 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:44 smithi167 ceph-mon[32751]: pgmap v2559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:30:44.489 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:44 smithi167 ceph-mon[32751]: pgmap v2560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:44.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:30:44.507 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:44 smithi090 ceph-mon[29180]: pgmap v2558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:44.517 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:30:44.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:30:44.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:30:44.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:44 smithi090 ceph-mon[29180]: pgmap v2559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:30:44.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:44 smithi090 ceph-mon[29180]: pgmap v2560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:44.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:30:45.217 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3729 sec 2024-04-26T19:30:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:45 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3327.025906 secs 2024-04-26T19:30:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:45 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3327.025906 secs 2024-04-26T19:30:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:45 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3327.025906 secs 2024-04-26T19:30:46.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3730 sec 2024-04-26T19:30:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3327.707085 secs 2024-04-26T19:30:46.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:46 smithi119 ceph-mon[32752]: pgmap v2561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3327.707085 secs 2024-04-26T19:30:46.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:46 smithi167 ceph-mon[32751]: pgmap v2561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:46.941 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3327.707085 secs 2024-04-26T19:30:46.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:46 smithi090 ceph-mon[29180]: pgmap v2561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:47.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3731 sec 2024-04-26T19:30:48.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3732 sec 2024-04-26T19:30:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:48 smithi119 ceph-mon[32752]: pgmap v2562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:30:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:48 smithi167 ceph-mon[32751]: pgmap v2562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:30:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:48 smithi090 ceph-mon[29180]: pgmap v2562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:30:49.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3733 sec 2024-04-26T19:30:50.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3734 sec 2024-04-26T19:30:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:50 smithi119 ceph-mon[32752]: pgmap v2563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:51.078 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:50 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3332.025965 secs 2024-04-26T19:30:51.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:50 smithi167 ceph-mon[32751]: pgmap v2563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:51.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:50 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3332.025965 secs 2024-04-26T19:30:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:50 smithi090 ceph-mon[29180]: pgmap v2563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:51.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:50 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3332.025965 secs 2024-04-26T19:30:51.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3735 sec 2024-04-26T19:30:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3332.707210 secs 2024-04-26T19:30:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3332.707210 secs 2024-04-26T19:30:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3332.707210 secs 2024-04-26T19:30:52.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3736 sec 2024-04-26T19:30:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:52 smithi119 ceph-mon[32752]: pgmap v2564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:52 smithi167 ceph-mon[32751]: pgmap v2564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:52 smithi090 ceph-mon[29180]: pgmap v2564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:30:53.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3737 sec 2024-04-26T19:30:54.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3738 sec 2024-04-26T19:30:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:54 smithi090 ceph-mon[29180]: pgmap v2565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:30:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:54 smithi119 ceph-mon[32752]: pgmap v2565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:30:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:54 smithi167 ceph-mon[32751]: pgmap v2565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:30:55.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3739 sec 2024-04-26T19:30:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:55 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3337.026026 secs 2024-04-26T19:30:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:55 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3337.026026 secs 2024-04-26T19:30:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:55 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3337.026026 secs 2024-04-26T19:30:56.218 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3740 sec 2024-04-26T19:30:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3337.707322 secs 2024-04-26T19:30:57.110 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:56 smithi119 ceph-mon[32752]: pgmap v2566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:57.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3337.707322 secs 2024-04-26T19:30:57.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:56 smithi167 ceph-mon[32751]: pgmap v2566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:57.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3337.707322 secs 2024-04-26T19:30:57.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:56 smithi090 ceph-mon[29180]: pgmap v2566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:57.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3741 sec 2024-04-26T19:30:58.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3742 sec 2024-04-26T19:30:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:30:58 smithi167 ceph-mon[32751]: pgmap v2567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:30:58 smithi090 ceph-mon[29180]: pgmap v2567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:30:59.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3743 sec 2024-04-26T19:30:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:30:58 smithi119 ceph-mon[32752]: pgmap v2567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:00.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3744 sec 2024-04-26T19:31:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:00 smithi167 ceph-mon[32751]: pgmap v2568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:01.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:00 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3342.026084 secs 2024-04-26T19:31:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:00 smithi090 ceph-mon[29180]: pgmap v2568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:01.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:00 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3342.026084 secs 2024-04-26T19:31:01.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3745 sec 2024-04-26T19:31:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:00 smithi119 ceph-mon[32752]: pgmap v2568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:00 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3342.026084 secs 2024-04-26T19:31:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3342.707394 secs 2024-04-26T19:31:02.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3746 sec 2024-04-26T19:31:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3342.707394 secs 2024-04-26T19:31:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3342.707394 secs 2024-04-26T19:31:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:02 smithi090 ceph-mon[29180]: pgmap v2569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:03.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3747 sec 2024-04-26T19:31:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:02 smithi119 ceph-mon[32752]: pgmap v2569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:02 smithi167 ceph-mon[32751]: pgmap v2569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:04.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3748 sec 2024-04-26T19:31:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:04 smithi090 ceph-mon[29180]: pgmap v2570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:05.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3749 sec 2024-04-26T19:31:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:04 smithi119 ceph-mon[32752]: pgmap v2570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:04 smithi167 ceph-mon[32751]: pgmap v2570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:05 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3347.026146 secs 2024-04-26T19:31:06.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3750 sec 2024-04-26T19:31:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:05 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3347.026146 secs 2024-04-26T19:31:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:05 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3347.026146 secs 2024-04-26T19:31:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3347.707508 secs 2024-04-26T19:31:07.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:06 smithi090 ceph-mon[29180]: pgmap v2571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:07.219 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3751 sec 2024-04-26T19:31:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3347.707508 secs 2024-04-26T19:31:07.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:06 smithi119 ceph-mon[32752]: pgmap v2571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:07.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3347.707508 secs 2024-04-26T19:31:07.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:06 smithi167 ceph-mon[32751]: pgmap v2571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:08.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3752 sec 2024-04-26T19:31:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:08 smithi090 ceph-mon[29180]: pgmap v2572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:09.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3753 sec 2024-04-26T19:31:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:08 smithi119 ceph-mon[32752]: pgmap v2572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:08 smithi167 ceph-mon[32751]: pgmap v2572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:10.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3754 sec 2024-04-26T19:31:11.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:10 smithi119 ceph-mon[32752]: pgmap v2573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:11.113 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:10 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3352.026230 secs 2024-04-26T19:31:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:10 smithi090 ceph-mon[29180]: pgmap v2573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:11.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:10 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3352.026230 secs 2024-04-26T19:31:11.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3755 sec 2024-04-26T19:31:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:10 smithi167 ceph-mon[32751]: pgmap v2573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:11.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:10 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3352.026230 secs 2024-04-26T19:31:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3352.707645 secs 2024-04-26T19:31:12.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3756 sec 2024-04-26T19:31:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3352.707645 secs 2024-04-26T19:31:12.360 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3352.707645 secs 2024-04-26T19:31:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:12 smithi090 ceph-mon[29180]: pgmap v2574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:13.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3757 sec 2024-04-26T19:31:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:12 smithi119 ceph-mon[32752]: pgmap v2574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:12 smithi167 ceph-mon[32751]: pgmap v2574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:14.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3758 sec 2024-04-26T19:31:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:14 smithi090 ceph-mon[29180]: pgmap v2575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:15.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3759 sec 2024-04-26T19:31:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:14 smithi119 ceph-mon[32752]: pgmap v2575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:14 smithi167 ceph-mon[32751]: pgmap v2575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:15 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3357.026314 secs 2024-04-26T19:31:16.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3760 sec 2024-04-26T19:31:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:15 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3357.026314 secs 2024-04-26T19:31:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:15 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3357.026314 secs 2024-04-26T19:31:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3357.707721 secs 2024-04-26T19:31:17.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:16 smithi090 ceph-mon[29180]: pgmap v2576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:17.220 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3761 sec 2024-04-26T19:31:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3357.707721 secs 2024-04-26T19:31:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:16 smithi119 ceph-mon[32752]: pgmap v2576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3357.707721 secs 2024-04-26T19:31:17.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:16 smithi167 ceph-mon[32751]: pgmap v2576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:18.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3762 sec 2024-04-26T19:31:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:18 smithi090 ceph-mon[29180]: pgmap v2577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:19.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3763 sec 2024-04-26T19:31:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:18 smithi119 ceph-mon[32752]: pgmap v2577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:18 smithi167 ceph-mon[32751]: pgmap v2577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:20.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3764 sec 2024-04-26T19:31:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:20 smithi090 ceph-mon[29180]: pgmap v2578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:21.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:20 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3362.026396 secs 2024-04-26T19:31:21.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3765 sec 2024-04-26T19:31:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:20 smithi119 ceph-mon[32752]: pgmap v2578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:21.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:20 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3362.026396 secs 2024-04-26T19:31:21.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:20 smithi167 ceph-mon[32751]: pgmap v2578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:21.365 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:20 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3362.026396 secs 2024-04-26T19:31:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3362.707856 secs 2024-04-26T19:31:22.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3766 sec 2024-04-26T19:31:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3362.707856 secs 2024-04-26T19:31:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3362.707856 secs 2024-04-26T19:31:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:22 smithi090 ceph-mon[29180]: pgmap v2579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:23.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3767 sec 2024-04-26T19:31:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:22 smithi119 ceph-mon[32752]: pgmap v2579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:22 smithi167 ceph-mon[32751]: pgmap v2579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:24.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3768 sec 2024-04-26T19:31:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:23 smithi119 ceph-mon[32752]: pgmap v2580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:24.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:23 smithi167 ceph-mon[32751]: pgmap v2580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:24.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:23 smithi090 ceph-mon[29180]: pgmap v2580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:25.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3769 sec 2024-04-26T19:31:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:25 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3367.026480 secs 2024-04-26T19:31:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:25 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3367.026480 secs 2024-04-26T19:31:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:25 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3367.026480 secs 2024-04-26T19:31:26.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3770 sec 2024-04-26T19:31:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3367.707988 secs 2024-04-26T19:31:26.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:26 smithi119 ceph-mon[32752]: pgmap v2581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:26.856 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3367.707988 secs 2024-04-26T19:31:26.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:26 smithi167 ceph-mon[32751]: pgmap v2581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3367.707988 secs 2024-04-26T19:31:26.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:26 smithi090 ceph-mon[29180]: pgmap v2581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:27.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3771 sec 2024-04-26T19:31:28.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3772 sec 2024-04-26T19:31:29.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:28 smithi119 ceph-mon[32752]: pgmap v2582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:29.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:28 smithi167 ceph-mon[32751]: pgmap v2582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:29.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:28 smithi090 ceph-mon[29180]: pgmap v2582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:29.221 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3773 sec 2024-04-26T19:31:30.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3774 sec 2024-04-26T19:31:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:30 smithi119 ceph-mon[32752]: pgmap v2583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:31.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:30 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 3372.026561 secs 2024-04-26T19:31:31.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:30 smithi167 ceph-mon[32751]: pgmap v2583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:31.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:30 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 3372.026561 secs 2024-04-26T19:31:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:30 smithi090 ceph-mon[29180]: pgmap v2583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:31.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:30 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 3372.026561 secs 2024-04-26T19:31:31.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3775 sec 2024-04-26T19:31:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3372.708138 secs 2024-04-26T19:31:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3372.708138 secs 2024-04-26T19:31:32.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3776 sec 2024-04-26T19:31:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3372.708138 secs 2024-04-26T19:31:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:32 smithi167 ceph-mon[32751]: pgmap v2584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:32 smithi090 ceph-mon[29180]: pgmap v2584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:33.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3777 sec 2024-04-26T19:31:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:32 smithi119 ceph-mon[32752]: pgmap v2584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:31:34.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3778 sec 2024-04-26T19:31:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:34 smithi167 ceph-mon[32751]: pgmap v2585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:34 smithi090 ceph-mon[29180]: pgmap v2585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:35.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3779 sec 2024-04-26T19:31:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:34 smithi119 ceph-mon[32752]: pgmap v2585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:35 smithi167 ceph-mon[32751]: 9 slow requests, 1 included below; oldest blocked for > 3377.026646 secs 2024-04-26T19:31:36.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:35 smithi167 ceph-mon[32751]: slow request 183.163337 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:31:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:35 smithi090 ceph-mon[29180]: 9 slow requests, 1 included below; oldest blocked for > 3377.026646 secs 2024-04-26T19:31:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:35 smithi090 ceph-mon[29180]: slow request 183.163337 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:31:36.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3780 sec 2024-04-26T19:31:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:35 smithi119 ceph-mon[32752]: 9 slow requests, 1 included below; oldest blocked for > 3377.026646 secs 2024-04-26T19:31:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:35 smithi119 ceph-mon[32752]: slow request 183.163337 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:31:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3377.708252 secs 2024-04-26T19:31:37.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:36 smithi167 ceph-mon[32751]: pgmap v2586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3377.708252 secs 2024-04-26T19:31:37.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:36 smithi090 ceph-mon[29180]: pgmap v2586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:37.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3781 sec 2024-04-26T19:31:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3377.708252 secs 2024-04-26T19:31:37.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:36 smithi119 ceph-mon[32752]: pgmap v2586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:38.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3782 sec 2024-04-26T19:31:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:38 smithi090 ceph-mon[29180]: pgmap v2587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:39.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:38 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:31:39.141 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:38 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:31:39.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3783 sec 2024-04-26T19:31:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:38 smithi119 ceph-mon[32752]: pgmap v2587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:39.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:38 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:31:39.322 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:38 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:31:39.384 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:38 smithi167 ceph-mon[32751]: pgmap v2587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:39.392 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:38 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:31:39.401 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:38 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:31:40.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3784 sec 2024-04-26T19:31:41.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:40 smithi119 ceph-mon[32752]: pgmap v2588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:41.115 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3382.026727 secs 2024-04-26T19:31:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:40 smithi090 ceph-mon[29180]: pgmap v2588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:41.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3382.026727 secs 2024-04-26T19:31:41.222 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3785 sec 2024-04-26T19:31:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:40 smithi167 ceph-mon[32751]: pgmap v2588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:41.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3382.026727 secs 2024-04-26T19:31:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3382.708364 secs 2024-04-26T19:31:42.223 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3786 sec 2024-04-26T19:31:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3382.708364 secs 2024-04-26T19:31:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3382.708364 secs 2024-04-26T19:31:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:42 smithi090 ceph-mon[29180]: pgmap v2589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:43.223 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3787 sec 2024-04-26T19:31:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:42 smithi119 ceph-mon[32752]: pgmap v2589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:42 smithi167 ceph-mon[32751]: pgmap v2589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:44.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3788 sec 2024-04-26T19:31:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:43 smithi119 ceph-mon[32752]: pgmap v2590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:44.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:31:44.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:43 smithi167 ceph-mon[32751]: pgmap v2590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:44.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:31:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:43 smithi090 ceph-mon[29180]: pgmap v2590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:31:44.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:31:45.223 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3789 sec 2024-04-26T19:31:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:31:45.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:31:45.320 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:44 smithi119 ceph-mon[32752]: pgmap v2591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:31:45.330 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:44 smithi119 ceph-mon[32752]: pgmap v2592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:45.339 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:31:45.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:31:45.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:31:45.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:44 smithi167 ceph-mon[32751]: pgmap v2591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:31:45.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:44 smithi167 ceph-mon[32751]: pgmap v2592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:45.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:31:45.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:31:45.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:31:45.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:44 smithi090 ceph-mon[29180]: pgmap v2591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:31:45.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:44 smithi090 ceph-mon[29180]: pgmap v2592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:45.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:31:46.223 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3790 sec 2024-04-26T19:31:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3387.026814 secs 2024-04-26T19:31:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3387.026814 secs 2024-04-26T19:31:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3387.026814 secs 2024-04-26T19:31:47.223 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3791 sec 2024-04-26T19:31:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3387.708461 secs 2024-04-26T19:31:47.318 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:46 smithi119 ceph-mon[32752]: pgmap v2593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:47.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3387.708461 secs 2024-04-26T19:31:47.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:46 smithi167 ceph-mon[32751]: pgmap v2593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:47.512 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3387.708461 secs 2024-04-26T19:31:47.522 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:46 smithi090 ceph-mon[29180]: pgmap v2593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:48.223 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3792 sec 2024-04-26T19:31:49.223 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3793 sec 2024-04-26T19:31:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:49 smithi167 ceph-mon[32751]: pgmap v2594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:49 smithi090 ceph-mon[29180]: pgmap v2594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:49 smithi119 ceph-mon[32752]: pgmap v2594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:50.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3794 sec 2024-04-26T19:31:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3392.026873 secs 2024-04-26T19:31:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3392.026873 secs 2024-04-26T19:31:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3392.026873 secs 2024-04-26T19:31:51.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3795 sec 2024-04-26T19:31:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3392.708596 secs 2024-04-26T19:31:51.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:51 smithi090 ceph-mon[29180]: pgmap v2595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3392.708596 secs 2024-04-26T19:31:52.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:51 smithi119 ceph-mon[32752]: pgmap v2595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3392.708596 secs 2024-04-26T19:31:52.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:51 smithi167 ceph-mon[32751]: pgmap v2595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:31:52.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3796 sec 2024-04-26T19:31:53.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3797 sec 2024-04-26T19:31:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:53 smithi090 ceph-mon[29180]: pgmap v2596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:31:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:53 smithi119 ceph-mon[32752]: pgmap v2596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:31:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:53 smithi167 ceph-mon[32751]: pgmap v2596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:31:54.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3798 sec 2024-04-26T19:31:55.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3799 sec 2024-04-26T19:31:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:55 smithi090 ceph-mon[29180]: pgmap v2597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:31:55.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3397.026931 secs 2024-04-26T19:31:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:55 smithi119 ceph-mon[32752]: pgmap v2597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:31:56.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3397.026931 secs 2024-04-26T19:31:56.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:55 smithi167 ceph-mon[32751]: pgmap v2597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:31:56.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3397.026931 secs 2024-04-26T19:31:56.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3800 sec 2024-04-26T19:31:56.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3397.708728 secs 2024-04-26T19:31:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3397.708728 secs 2024-04-26T19:31:57.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3397.708728 secs 2024-04-26T19:31:57.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3801 sec 2024-04-26T19:31:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:57 smithi090 ceph-mon[29180]: pgmap v2598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:57 smithi119 ceph-mon[32752]: pgmap v2598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:57 smithi167 ceph-mon[32751]: pgmap v2598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:31:58.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3802 sec 2024-04-26T19:31:59.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3803 sec 2024-04-26T19:31:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:31:59 smithi090 ceph-mon[29180]: pgmap v2599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:31:59 smithi119 ceph-mon[32752]: pgmap v2599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:00.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:31:59 smithi167 ceph-mon[32751]: pgmap v2599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:00.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3804 sec 2024-04-26T19:32:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3402.027011 secs 2024-04-26T19:32:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3402.027011 secs 2024-04-26T19:32:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3402.027011 secs 2024-04-26T19:32:01.224 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3805 sec 2024-04-26T19:32:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3402.708865 secs 2024-04-26T19:32:01.969 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:01 smithi090 ceph-mon[29180]: pgmap v2600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:01.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3402.708865 secs 2024-04-26T19:32:01.987 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:01 smithi167 ceph-mon[32751]: pgmap v2600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3402.708865 secs 2024-04-26T19:32:02.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:01 smithi119 ceph-mon[32752]: pgmap v2600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:02.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3806 sec 2024-04-26T19:32:03.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3807 sec 2024-04-26T19:32:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:03 smithi090 ceph-mon[29180]: pgmap v2601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:03 smithi119 ceph-mon[32752]: pgmap v2601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:04.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:03 smithi167 ceph-mon[32751]: pgmap v2601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:04.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3808 sec 2024-04-26T19:32:05.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3809 sec 2024-04-26T19:32:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:05 smithi090 ceph-mon[29180]: pgmap v2602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:05.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3407.027069 secs 2024-04-26T19:32:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:05 smithi119 ceph-mon[32752]: pgmap v2602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:06.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3407.027069 secs 2024-04-26T19:32:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:05 smithi167 ceph-mon[32751]: pgmap v2602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:06.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3407.027069 secs 2024-04-26T19:32:06.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3810 sec 2024-04-26T19:32:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3407.708969 secs 2024-04-26T19:32:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3407.708969 secs 2024-04-26T19:32:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3407.708969 secs 2024-04-26T19:32:07.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3811 sec 2024-04-26T19:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:07 smithi090 ceph-mon[29180]: pgmap v2603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:07 smithi119 ceph-mon[32752]: pgmap v2603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:08.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:07 smithi167 ceph-mon[32751]: pgmap v2603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:08.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3812 sec 2024-04-26T19:32:09.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3813 sec 2024-04-26T19:32:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:09 smithi090 ceph-mon[29180]: pgmap v2604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:09 smithi119 ceph-mon[32752]: pgmap v2604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:09 smithi167 ceph-mon[32751]: pgmap v2604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:10.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3814 sec 2024-04-26T19:32:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3412.027150 secs 2024-04-26T19:32:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3412.027150 secs 2024-04-26T19:32:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3412.027150 secs 2024-04-26T19:32:11.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3815 sec 2024-04-26T19:32:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3412.709085 secs 2024-04-26T19:32:11.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:11 smithi090 ceph-mon[29180]: pgmap v2605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3412.709085 secs 2024-04-26T19:32:11.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:11 smithi119 ceph-mon[32752]: pgmap v2605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3412.709085 secs 2024-04-26T19:32:12.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:11 smithi167 ceph-mon[32751]: pgmap v2605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:12.225 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3816 sec 2024-04-26T19:32:13.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3817 sec 2024-04-26T19:32:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:13 smithi119 ceph-mon[32752]: pgmap v2606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:13 smithi167 ceph-mon[32751]: pgmap v2606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:13 smithi090 ceph-mon[29180]: pgmap v2606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:14.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3818 sec 2024-04-26T19:32:15.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3819 sec 2024-04-26T19:32:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:15 smithi119 ceph-mon[32752]: pgmap v2607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:16.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:15 smithi119 ceph-mon[32752]: 9 slow requests, 1 included below; oldest blocked for > 3417.027234 secs 2024-04-26T19:32:16.072 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:15 smithi119 ceph-mon[32752]: slow request 1440.297370 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:32:16.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:15 smithi167 ceph-mon[32751]: pgmap v2607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:16.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:15 smithi167 ceph-mon[32751]: 9 slow requests, 1 included below; oldest blocked for > 3417.027234 secs 2024-04-26T19:32:16.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:15 smithi167 ceph-mon[32751]: slow request 1440.297370 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:32:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:15 smithi090 ceph-mon[29180]: pgmap v2607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:16.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:15 smithi090 ceph-mon[29180]: 9 slow requests, 1 included below; oldest blocked for > 3417.027234 secs 2024-04-26T19:32:16.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:15 smithi090 ceph-mon[29180]: slow request 1440.297370 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:32:16.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3820 sec 2024-04-26T19:32:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3417.709146 secs 2024-04-26T19:32:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3417.709146 secs 2024-04-26T19:32:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3417.709146 secs 2024-04-26T19:32:17.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3821 sec 2024-04-26T19:32:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:17 smithi119 ceph-mon[32752]: pgmap v2608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:17 smithi167 ceph-mon[32751]: pgmap v2608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:17 smithi090 ceph-mon[29180]: pgmap v2608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:18.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3822 sec 2024-04-26T19:32:19.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3823 sec 2024-04-26T19:32:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:19 smithi119 ceph-mon[32752]: pgmap v2609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:20.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:19 smithi167 ceph-mon[32751]: pgmap v2609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:19 smithi090 ceph-mon[29180]: pgmap v2609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:20.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3824 sec 2024-04-26T19:32:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3422.027286 secs 2024-04-26T19:32:21.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3422.027286 secs 2024-04-26T19:32:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3422.027286 secs 2024-04-26T19:32:21.226 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3825 sec 2024-04-26T19:32:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3422.709266 secs 2024-04-26T19:32:22.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:21 smithi119 ceph-mon[32752]: pgmap v2610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:22.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3422.709266 secs 2024-04-26T19:32:22.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:21 smithi167 ceph-mon[32751]: pgmap v2610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3422.709266 secs 2024-04-26T19:32:22.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:21 smithi090 ceph-mon[29180]: pgmap v2610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:22.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3826 sec 2024-04-26T19:32:23.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3827 sec 2024-04-26T19:32:23.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:23 smithi090 ceph-mon[29180]: pgmap v2611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:23 smithi119 ceph-mon[32752]: pgmap v2611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:23 smithi167 ceph-mon[32751]: pgmap v2611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:24.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3828 sec 2024-04-26T19:32:25.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3829 sec 2024-04-26T19:32:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:25 smithi119 ceph-mon[32752]: pgmap v2612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:26.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3427.027366 secs 2024-04-26T19:32:26.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:25 smithi167 ceph-mon[32751]: pgmap v2612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:26.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3427.027366 secs 2024-04-26T19:32:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:25 smithi090 ceph-mon[29180]: pgmap v2612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3427.027366 secs 2024-04-26T19:32:26.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3830 sec 2024-04-26T19:32:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3427.709403 secs 2024-04-26T19:32:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3427.709403 secs 2024-04-26T19:32:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3427.709403 secs 2024-04-26T19:32:27.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3831 sec 2024-04-26T19:32:28.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:27 smithi090 ceph-mon[29180]: pgmap v2613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:28.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:27 smithi167 ceph-mon[32751]: pgmap v2613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:28.145 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:27 smithi119 ceph-mon[32752]: pgmap v2613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:28.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3832 sec 2024-04-26T19:32:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:32:28.921+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:32:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:32:28.921+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:32:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:32:28.921+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 30 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:32:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:32:28.921+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 30 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:32:29.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3833 sec 2024-04-26T19:32:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:29 smithi119 ceph-mon[32752]: pgmap v2614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:29 smithi167 ceph-mon[32751]: pgmap v2614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:29 smithi090 ceph-mon[29180]: pgmap v2614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:30.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3834 sec 2024-04-26T19:32:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3432.027449 secs 2024-04-26T19:32:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3432.027449 secs 2024-04-26T19:32:31.136 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3432.027449 secs 2024-04-26T19:32:31.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3835 sec 2024-04-26T19:32:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3432.709511 secs 2024-04-26T19:32:31.985 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:31 smithi167 ceph-mon[32751]: pgmap v2615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3432.709511 secs 2024-04-26T19:32:32.109 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:31 smithi119 ceph-mon[32752]: pgmap v2615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3432.709511 secs 2024-04-26T19:32:32.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:31 smithi090 ceph-mon[29180]: pgmap v2615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:32.227 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3836 sec 2024-04-26T19:32:33.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3837 sec 2024-04-26T19:32:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:33 smithi119 ceph-mon[32752]: pgmap v2616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:34.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:33 smithi167 ceph-mon[32751]: pgmap v2616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:34.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:33 smithi090 ceph-mon[29180]: pgmap v2616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:34.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3838 sec 2024-04-26T19:32:35.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3839 sec 2024-04-26T19:32:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:35 smithi119 ceph-mon[32752]: pgmap v2617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:36.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3437.027533 secs 2024-04-26T19:32:36.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:35 smithi167 ceph-mon[32751]: pgmap v2617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:36.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3437.027533 secs 2024-04-26T19:32:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:35 smithi090 ceph-mon[29180]: pgmap v2617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3437.027533 secs 2024-04-26T19:32:36.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3840 sec 2024-04-26T19:32:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3437.709628 secs 2024-04-26T19:32:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3437.709628 secs 2024-04-26T19:32:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3437.709628 secs 2024-04-26T19:32:37.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3841 sec 2024-04-26T19:32:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:37 smithi119 ceph-mon[32752]: pgmap v2618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:38.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:37 smithi167 ceph-mon[32751]: pgmap v2618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:37 smithi090 ceph-mon[29180]: pgmap v2618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:38.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3842 sec 2024-04-26T19:32:39.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3843 sec 2024-04-26T19:32:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:39 smithi119 ceph-mon[32752]: pgmap v2619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:39.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:39 smithi090 ceph-mon[29180]: pgmap v2619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:39.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:39 smithi167 ceph-mon[32751]: pgmap v2619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:40.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3844 sec 2024-04-26T19:32:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3442.027616 secs 2024-04-26T19:32:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3442.027616 secs 2024-04-26T19:32:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3442.027616 secs 2024-04-26T19:32:41.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3845 sec 2024-04-26T19:32:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3442.709711 secs 2024-04-26T19:32:41.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:41 smithi119 ceph-mon[32752]: pgmap v2620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3442.709711 secs 2024-04-26T19:32:41.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:41 smithi167 ceph-mon[32751]: pgmap v2620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3442.709711 secs 2024-04-26T19:32:41.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:41 smithi090 ceph-mon[29180]: pgmap v2620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:42.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3846 sec 2024-04-26T19:32:43.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3847 sec 2024-04-26T19:32:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:43 smithi090 ceph-mon[29180]: pgmap v2621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:43 smithi119 ceph-mon[32752]: pgmap v2621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:44.138 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:43 smithi167 ceph-mon[32751]: pgmap v2621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:44.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3848 sec 2024-04-26T19:32:45.228 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3849 sec 2024-04-26T19:32:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:45 smithi090 ceph-mon[29180]: pgmap v2622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:45.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:32:45.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3447.027711 secs 2024-04-26T19:32:45.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:32:45.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:32:45.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:32:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:45 smithi119 ceph-mon[32752]: pgmap v2622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:46.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:32:46.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3447.027711 secs 2024-04-26T19:32:46.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:32:46.117 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:32:46.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:32:46.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:45 smithi167 ceph-mon[32751]: pgmap v2622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:32:46.200 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:32:46.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3447.027711 secs 2024-04-26T19:32:46.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:32:46.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:32:46.274 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:32:46.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3850 sec 2024-04-26T19:32:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:46 smithi090 ceph-mon[29180]: pgmap v2623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:32:46.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:46 smithi090 ceph-mon[29180]: pgmap v2624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:46.894 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3447.709798 secs 2024-04-26T19:32:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:46 smithi119 ceph-mon[32752]: pgmap v2623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:32:47.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:46 smithi119 ceph-mon[32752]: pgmap v2624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:47.113 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3447.709798 secs 2024-04-26T19:32:47.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:46 smithi167 ceph-mon[32751]: pgmap v2623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:32:47.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:46 smithi167 ceph-mon[32751]: pgmap v2624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:47.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3447.709798 secs 2024-04-26T19:32:47.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3851 sec 2024-04-26T19:32:48.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3852 sec 2024-04-26T19:32:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:48 smithi090 ceph-mon[29180]: pgmap v2625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:32:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:48 smithi119 ceph-mon[32752]: pgmap v2625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:32:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:48 smithi167 ceph-mon[32751]: pgmap v2625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:32:49.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3853 sec 2024-04-26T19:32:50.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3854 sec 2024-04-26T19:32:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:50 smithi090 ceph-mon[29180]: pgmap v2626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:50.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3452.027813 secs 2024-04-26T19:32:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:50 smithi119 ceph-mon[32752]: pgmap v2626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:51.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3452.027813 secs 2024-04-26T19:32:51.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:50 smithi167 ceph-mon[32751]: pgmap v2626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:51.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3452.027813 secs 2024-04-26T19:32:51.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3855 sec 2024-04-26T19:32:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3452.709918 secs 2024-04-26T19:32:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3452.709918 secs 2024-04-26T19:32:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3452.709918 secs 2024-04-26T19:32:52.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3856 sec 2024-04-26T19:32:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:52 smithi090 ceph-mon[29180]: pgmap v2627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:52 smithi119 ceph-mon[32752]: pgmap v2627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:52 smithi167 ceph-mon[32751]: pgmap v2627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:53.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3857 sec 2024-04-26T19:32:54.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3858 sec 2024-04-26T19:32:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:54 smithi090 ceph-mon[29180]: pgmap v2628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:54 smithi119 ceph-mon[32752]: pgmap v2628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:55.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:54 smithi167 ceph-mon[32751]: pgmap v2628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:32:55.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3859 sec 2024-04-26T19:32:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3457.027896 secs 2024-04-26T19:32:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3457.027896 secs 2024-04-26T19:32:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3457.027896 secs 2024-04-26T19:32:56.229 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3860 sec 2024-04-26T19:32:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:56 smithi090 ceph-mon[29180]: pgmap v2629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:32:56.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3457.710034 secs 2024-04-26T19:32:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:56 smithi119 ceph-mon[32752]: pgmap v2629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:32:57.134 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3457.710034 secs 2024-04-26T19:32:57.143 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:56 smithi167 ceph-mon[32751]: pgmap v2629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:32:57.151 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3457.710034 secs 2024-04-26T19:32:57.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3861 sec 2024-04-26T19:32:58.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3862 sec 2024-04-26T19:32:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:32:58 smithi090 ceph-mon[29180]: pgmap v2630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:32:58 smithi119 ceph-mon[32752]: pgmap v2630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:32:58 smithi167 ceph-mon[32751]: pgmap v2630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:32:59.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3863 sec 2024-04-26T19:33:00.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3864 sec 2024-04-26T19:33:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:00 smithi090 ceph-mon[29180]: pgmap v2631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:00.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3462.027976 secs 2024-04-26T19:33:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:00 smithi119 ceph-mon[32752]: pgmap v2631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:01.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3462.027976 secs 2024-04-26T19:33:01.144 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:00 smithi167 ceph-mon[32751]: pgmap v2631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:01.170 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3462.027976 secs 2024-04-26T19:33:01.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3865 sec 2024-04-26T19:33:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3462.710169 secs 2024-04-26T19:33:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3462.710169 secs 2024-04-26T19:33:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3462.710169 secs 2024-04-26T19:33:02.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3866 sec 2024-04-26T19:33:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:02 smithi119 ceph-mon[32752]: pgmap v2632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:02 smithi167 ceph-mon[32751]: pgmap v2632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:02 smithi090 ceph-mon[29180]: pgmap v2632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:03.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3867 sec 2024-04-26T19:33:04.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3868 sec 2024-04-26T19:33:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:04 smithi119 ceph-mon[32752]: pgmap v2633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:04 smithi167 ceph-mon[32751]: pgmap v2633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:04 smithi090 ceph-mon[29180]: pgmap v2633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:05.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3869 sec 2024-04-26T19:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3467.028040 secs 2024-04-26T19:33:06.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3467.028040 secs 2024-04-26T19:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3467.028040 secs 2024-04-26T19:33:06.230 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3870 sec 2024-04-26T19:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:06 smithi119 ceph-mon[32752]: pgmap v2634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:07.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3467.710280 secs 2024-04-26T19:33:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:06 smithi167 ceph-mon[32751]: pgmap v2634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:07.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3467.710280 secs 2024-04-26T19:33:07.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:06 smithi090 ceph-mon[29180]: pgmap v2634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:07.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3467.710280 secs 2024-04-26T19:33:07.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3871 sec 2024-04-26T19:33:08.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3872 sec 2024-04-26T19:33:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:08 smithi119 ceph-mon[32752]: pgmap v2635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:08 smithi167 ceph-mon[32751]: pgmap v2635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:08 smithi090 ceph-mon[29180]: pgmap v2635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:09.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3873 sec 2024-04-26T19:33:10.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3874 sec 2024-04-26T19:33:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:10 smithi119 ceph-mon[32752]: pgmap v2636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:11.068 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3472.028122 secs 2024-04-26T19:33:11.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:10 smithi167 ceph-mon[32751]: pgmap v2636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:11.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3472.028122 secs 2024-04-26T19:33:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:10 smithi090 ceph-mon[29180]: pgmap v2636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:11.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3472.028122 secs 2024-04-26T19:33:11.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3875 sec 2024-04-26T19:33:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3472.710403 secs 2024-04-26T19:33:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3472.710403 secs 2024-04-26T19:33:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3472.710403 secs 2024-04-26T19:33:12.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3876 sec 2024-04-26T19:33:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:12 smithi119 ceph-mon[32752]: pgmap v2637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:13.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:12 smithi167 ceph-mon[32751]: pgmap v2637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:12 smithi090 ceph-mon[29180]: pgmap v2637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:13.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3877 sec 2024-04-26T19:33:13.927 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:33:13.927+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:33:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:33:13.927+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:33:13.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:33:13.927+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 31 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:33:14.004 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:33:13.927+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 31 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:33:14.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3878 sec 2024-04-26T19:33:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:14 smithi119 ceph-mon[32752]: pgmap v2638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:14 smithi167 ceph-mon[32751]: pgmap v2638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:14 smithi090 ceph-mon[29180]: pgmap v2638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:15.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3879 sec 2024-04-26T19:33:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3477.028183 secs 2024-04-26T19:33:16.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3477.028183 secs 2024-04-26T19:33:16.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3477.028183 secs 2024-04-26T19:33:16.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3880 sec 2024-04-26T19:33:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:16 smithi119 ceph-mon[32752]: pgmap v2639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:17.075 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3477.710452 secs 2024-04-26T19:33:17.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:16 smithi167 ceph-mon[32751]: pgmap v2639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:17.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3477.710452 secs 2024-04-26T19:33:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:16 smithi090 ceph-mon[29180]: pgmap v2639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:17.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3477.710452 secs 2024-04-26T19:33:17.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3881 sec 2024-04-26T19:33:18.231 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3882 sec 2024-04-26T19:33:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:18 smithi119 ceph-mon[32752]: pgmap v2640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:18 smithi167 ceph-mon[32751]: pgmap v2640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:18 smithi090 ceph-mon[29180]: pgmap v2640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:19.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3883 sec 2024-04-26T19:33:20.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3884 sec 2024-04-26T19:33:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:20 smithi119 ceph-mon[32752]: pgmap v2641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:21.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3482.028266 secs 2024-04-26T19:33:21.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:20 smithi167 ceph-mon[32751]: pgmap v2641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:21.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3482.028266 secs 2024-04-26T19:33:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:20 smithi090 ceph-mon[29180]: pgmap v2641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:21.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3482.028266 secs 2024-04-26T19:33:21.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3885 sec 2024-04-26T19:33:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3482.710593 secs 2024-04-26T19:33:22.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3482.710593 secs 2024-04-26T19:33:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3482.710593 secs 2024-04-26T19:33:22.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3886 sec 2024-04-26T19:33:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:22 smithi119 ceph-mon[32752]: pgmap v2642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:22 smithi167 ceph-mon[32751]: pgmap v2642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:22 smithi090 ceph-mon[29180]: pgmap v2642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:23.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3887 sec 2024-04-26T19:33:24.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3888 sec 2024-04-26T19:33:24.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:24 smithi090 ceph-mon[29180]: pgmap v2643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:24 smithi119 ceph-mon[32752]: pgmap v2643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:25.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:24 smithi167 ceph-mon[32751]: pgmap v2643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:25.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3889 sec 2024-04-26T19:33:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3487.028328 secs 2024-04-26T19:33:26.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3487.028328 secs 2024-04-26T19:33:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3487.028328 secs 2024-04-26T19:33:26.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3890 sec 2024-04-26T19:33:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:26 smithi119 ceph-mon[32752]: pgmap v2644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:27.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3487.710660 secs 2024-04-26T19:33:27.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:26 smithi167 ceph-mon[32751]: pgmap v2644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:27.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3487.710660 secs 2024-04-26T19:33:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:26 smithi090 ceph-mon[29180]: pgmap v2644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:27.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3487.710660 secs 2024-04-26T19:33:27.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3891 sec 2024-04-26T19:33:28.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3892 sec 2024-04-26T19:33:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:28 smithi119 ceph-mon[32752]: pgmap v2645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:28 smithi167 ceph-mon[32751]: pgmap v2645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:28 smithi090 ceph-mon[29180]: pgmap v2645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:29.232 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3893 sec 2024-04-26T19:33:30.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3894 sec 2024-04-26T19:33:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:30 smithi119 ceph-mon[32752]: pgmap v2646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:31.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3492.028387 secs 2024-04-26T19:33:31.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:30 smithi167 ceph-mon[32751]: pgmap v2646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:31.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3492.028387 secs 2024-04-26T19:33:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:30 smithi090 ceph-mon[29180]: pgmap v2646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:33:31.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3492.028387 secs 2024-04-26T19:33:31.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3895 sec 2024-04-26T19:33:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3492.710739 secs 2024-04-26T19:33:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3492.710739 secs 2024-04-26T19:33:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3492.710739 secs 2024-04-26T19:33:32.135 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:33:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:33:31.733+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:33:32.144 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:33:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:33:31.733+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:33:32.151 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:33:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:33:31.733+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:33:32.159 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:33:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:33:31.733+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:33:32.168 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:33:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:33:31.733+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:33:32.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3896 sec 2024-04-26T19:33:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:32 smithi119 ceph-mon[32752]: pgmap v2647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:32 smithi167 ceph-mon[32751]: pgmap v2647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:32 smithi090 ceph-mon[29180]: pgmap v2647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:33.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3897 sec 2024-04-26T19:33:34.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3898 sec 2024-04-26T19:33:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:34 smithi119 ceph-mon[32752]: pgmap v2648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:34 smithi167 ceph-mon[32751]: pgmap v2648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:34 smithi090 ceph-mon[29180]: pgmap v2648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:35.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3899 sec 2024-04-26T19:33:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3497.028477 secs 2024-04-26T19:33:36.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3497.028477 secs 2024-04-26T19:33:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3497.028477 secs 2024-04-26T19:33:36.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3900 sec 2024-04-26T19:33:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:36 smithi119 ceph-mon[32752]: pgmap v2649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:37.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3497.710815 secs 2024-04-26T19:33:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:36 smithi167 ceph-mon[32751]: pgmap v2649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:37.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3497.710815 secs 2024-04-26T19:33:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:36 smithi090 ceph-mon[29180]: pgmap v2649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:37.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3497.710815 secs 2024-04-26T19:33:37.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3901 sec 2024-04-26T19:33:38.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3902 sec 2024-04-26T19:33:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:38 smithi119 ceph-mon[32752]: pgmap v2650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:38 smithi167 ceph-mon[32751]: pgmap v2650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:38 smithi090 ceph-mon[29180]: pgmap v2650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:39.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3903 sec 2024-04-26T19:33:40.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3904 sec 2024-04-26T19:33:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:40 smithi119 ceph-mon[32752]: pgmap v2651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-26T19:33:41.067 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3502.028559 secs 2024-04-26T19:33:41.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:40 smithi167 ceph-mon[32751]: pgmap v2651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-26T19:33:41.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3502.028559 secs 2024-04-26T19:33:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:40 smithi090 ceph-mon[29180]: pgmap v2651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-26T19:33:41.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3502.028559 secs 2024-04-26T19:33:41.233 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3905 sec 2024-04-26T19:33:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3502.710875 secs 2024-04-26T19:33:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3502.710875 secs 2024-04-26T19:33:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3502.710875 secs 2024-04-26T19:33:42.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3906 sec 2024-04-26T19:33:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:42 smithi119 ceph-mon[32752]: pgmap v2652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:42 smithi167 ceph-mon[32751]: pgmap v2652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:42 smithi090 ceph-mon[29180]: pgmap v2652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T19:33:43.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3907 sec 2024-04-26T19:33:44.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3908 sec 2024-04-26T19:33:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:44 smithi119 ceph-mon[32752]: pgmap v2653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-26T19:33:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:44 smithi167 ceph-mon[32751]: pgmap v2653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-26T19:33:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:44 smithi090 ceph-mon[29180]: pgmap v2653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 255 B/s wr, 14 op/s 2024-04-26T19:33:45.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3909 sec 2024-04-26T19:33:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3507.028642 secs 2024-04-26T19:33:45.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:33:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3507.028642 secs 2024-04-26T19:33:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:33:46.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3507.028642 secs 2024-04-26T19:33:46.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:33:46.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3910 sec 2024-04-26T19:33:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:46 smithi119 ceph-mon[32752]: pgmap v2654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-26T19:33:47.083 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3507.710936 secs 2024-04-26T19:33:47.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.101 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.121 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:46 smithi090 ceph-mon[29180]: pgmap v2654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-26T19:33:47.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3507.710936 secs 2024-04-26T19:33:47.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.197 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.217 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:46 smithi167 ceph-mon[32751]: pgmap v2654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-26T19:33:47.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3507.710936 secs 2024-04-26T19:33:47.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.267 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:47.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3911 sec 2024-04-26T19:33:48.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3912 sec 2024-04-26T19:33:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: pgmap v2655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:48.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.822 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.832 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:48.860 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:48.876 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.883 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:48.892 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:48.900 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.908 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: pgmap v2655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:48.925 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.933 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:48.996 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.076 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.085 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.094 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: pgmap v2655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:33:49.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.187 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.197 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.208 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.239 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.379 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:49.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3913 sec 2024-04-26T19:33:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:33:49.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:33:49.822 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.831 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.880 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.891 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.901 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.911 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.921 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.930 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:49.940 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:33:49.950 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:33:49.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:33:49.970 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:33:50.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:33:50.129 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:33:50.137 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: pgmap v2656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:33:50.147 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: pgmap v2657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:50.158 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:50.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:33:50.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:33:50.219 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.232 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.243 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.255 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.268 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.302 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:33:50.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:33:50.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:33:50.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:33:50.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:33:50.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:33:50.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: pgmap v2656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:33:50.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: pgmap v2657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:50.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:50.531 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:33:50.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:33:50.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:33:50.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:33:50.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:33:50.757 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:33:50.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:33:50.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:33:50.788 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:33:50.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: pgmap v2656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:33:50.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: pgmap v2657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:50.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:33:50.861 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3512.028722 secs 2024-04-26T19:33:50.871 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3914 sec 2024-04-26T19:33:50.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3512.028722 secs 2024-04-26T19:33:50.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3512.028722 secs 2024-04-26T19:33:51.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3915 sec 2024-04-26T19:33:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3512.711002 secs 2024-04-26T19:33:51.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:51 smithi119 ceph-mon[32752]: pgmap v2658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:51.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3512.711002 secs 2024-04-26T19:33:51.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:51 smithi167 ceph-mon[32751]: pgmap v2658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:51.976 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3512.711002 secs 2024-04-26T19:33:51.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:51 smithi090 ceph-mon[29180]: pgmap v2658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:52.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3916 sec 2024-04-26T19:33:53.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3917 sec 2024-04-26T19:33:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:53 smithi119 ceph-mon[32752]: pgmap v2659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 B/s rd, 1 op/s 2024-04-26T19:33:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:53 smithi167 ceph-mon[32751]: pgmap v2659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 B/s rd, 1 op/s 2024-04-26T19:33:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:53 smithi090 ceph-mon[29180]: pgmap v2659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 B/s rd, 1 op/s 2024-04-26T19:33:54.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3918 sec 2024-04-26T19:33:55.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3919 sec 2024-04-26T19:33:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:55 smithi119 ceph-mon[32752]: pgmap v2660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:56.075 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3517.028805 secs 2024-04-26T19:33:56.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:55 smithi167 ceph-mon[32751]: pgmap v2660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:56.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3517.028805 secs 2024-04-26T19:33:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:55 smithi090 ceph-mon[29180]: pgmap v2660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:56.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3517.028805 secs 2024-04-26T19:33:56.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3920 sec 2024-04-26T19:33:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3517.711115 secs 2024-04-26T19:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3517.711115 secs 2024-04-26T19:33:57.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3921 sec 2024-04-26T19:33:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3517.711115 secs 2024-04-26T19:33:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:57 smithi167 ceph-mon[32751]: pgmap v2661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:57 smithi090 ceph-mon[29180]: pgmap v2661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:58.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3922 sec 2024-04-26T19:33:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:57 smithi119 ceph-mon[32752]: pgmap v2661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:33:59.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3923 sec 2024-04-26T19:34:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:33:59 smithi167 ceph-mon[32751]: pgmap v2662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:34:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:33:59 smithi090 ceph-mon[29180]: pgmap v2662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:34:00.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3924 sec 2024-04-26T19:34:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:33:59 smithi119 ceph-mon[32752]: pgmap v2662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:34:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3522.028886 secs 2024-04-26T19:34:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3522.028886 secs 2024-04-26T19:34:01.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3925 sec 2024-04-26T19:34:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3522.028886 secs 2024-04-26T19:34:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3522.711232 secs 2024-04-26T19:34:02.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:01 smithi167 ceph-mon[32751]: pgmap v2663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3522.711232 secs 2024-04-26T19:34:02.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:01 smithi090 ceph-mon[29180]: pgmap v2663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:02.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3926 sec 2024-04-26T19:34:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3522.711232 secs 2024-04-26T19:34:02.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:01 smithi119 ceph-mon[32752]: pgmap v2663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:03.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3927 sec 2024-04-26T19:34:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:03 smithi090 ceph-mon[29180]: pgmap v2664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:04.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3928 sec 2024-04-26T19:34:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:03 smithi119 ceph-mon[32752]: pgmap v2664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:04.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:03 smithi167 ceph-mon[32751]: pgmap v2664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:05.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3929 sec 2024-04-26T19:34:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:05 smithi090 ceph-mon[29180]: pgmap v2665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:06.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3527.028972 secs 2024-04-26T19:34:06.235 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3930 sec 2024-04-26T19:34:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:05 smithi119 ceph-mon[32752]: pgmap v2665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:06.368 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3527.028972 secs 2024-04-26T19:34:06.377 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:05 smithi167 ceph-mon[32751]: pgmap v2665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:06.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3527.028972 secs 2024-04-26T19:34:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3527.711344 secs 2024-04-26T19:34:07.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3931 sec 2024-04-26T19:34:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3527.711344 secs 2024-04-26T19:34:07.381 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3527.711344 secs 2024-04-26T19:34:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:07 smithi090 ceph-mon[29180]: pgmap v2666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:08.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3932 sec 2024-04-26T19:34:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:07 smithi119 ceph-mon[32752]: pgmap v2666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:07 smithi167 ceph-mon[32751]: pgmap v2666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:09.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3933 sec 2024-04-26T19:34:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:09 smithi090 ceph-mon[29180]: pgmap v2667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:10.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3934 sec 2024-04-26T19:34:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:09 smithi119 ceph-mon[32752]: pgmap v2667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:10.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:09 smithi167 ceph-mon[32751]: pgmap v2667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3532.029075 secs 2024-04-26T19:34:11.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3532.029075 secs 2024-04-26T19:34:11.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3935 sec 2024-04-26T19:34:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3532.029075 secs 2024-04-26T19:34:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3532.711459 secs 2024-04-26T19:34:12.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:11 smithi090 ceph-mon[29180]: pgmap v2668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:12.220 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3532.711459 secs 2024-04-26T19:34:12.230 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:11 smithi119 ceph-mon[32752]: pgmap v2668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:12.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3936 sec 2024-04-26T19:34:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3532.711459 secs 2024-04-26T19:34:12.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:11 smithi167 ceph-mon[32751]: pgmap v2668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:13.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3937 sec 2024-04-26T19:34:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:13 smithi090 ceph-mon[29180]: pgmap v2669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:14.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3938 sec 2024-04-26T19:34:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:13 smithi119 ceph-mon[32752]: pgmap v2669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:13 smithi167 ceph-mon[32751]: pgmap v2669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:34:15.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3939 sec 2024-04-26T19:34:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:15 smithi090 ceph-mon[29180]: pgmap v2670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:16.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3537.029143 secs 2024-04-26T19:34:16.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3940 sec 2024-04-26T19:34:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:15 smithi119 ceph-mon[32752]: pgmap v2670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:16.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3537.029143 secs 2024-04-26T19:34:16.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:15 smithi167 ceph-mon[32751]: pgmap v2670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:34:16.378 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3537.029143 secs 2024-04-26T19:34:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3537.711572 secs 2024-04-26T19:34:17.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3941 sec 2024-04-26T19:34:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3537.711572 secs 2024-04-26T19:34:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3537.711572 secs 2024-04-26T19:34:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:17 smithi090 ceph-mon[29180]: pgmap v2671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:18.236 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3942 sec 2024-04-26T19:34:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:17 smithi119 ceph-mon[32752]: pgmap v2671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:18.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:17 smithi167 ceph-mon[32751]: pgmap v2671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:19.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3943 sec 2024-04-26T19:34:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:19 smithi090 ceph-mon[29180]: pgmap v2672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:20.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3944 sec 2024-04-26T19:34:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:19 smithi119 ceph-mon[32752]: pgmap v2672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:19 smithi167 ceph-mon[32751]: pgmap v2672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3542.029224 secs 2024-04-26T19:34:21.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3945 sec 2024-04-26T19:34:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3542.029224 secs 2024-04-26T19:34:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3542.029224 secs 2024-04-26T19:34:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3542.711698 secs 2024-04-26T19:34:22.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:21 smithi090 ceph-mon[29180]: pgmap v2673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:22.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3946 sec 2024-04-26T19:34:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3542.711698 secs 2024-04-26T19:34:22.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:21 smithi119 ceph-mon[32752]: pgmap v2673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3542.711698 secs 2024-04-26T19:34:22.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:21 smithi167 ceph-mon[32751]: pgmap v2673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:23.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3947 sec 2024-04-26T19:34:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:23 smithi090 ceph-mon[29180]: pgmap v2674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:24.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3948 sec 2024-04-26T19:34:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:23 smithi119 ceph-mon[32752]: pgmap v2674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:23 smithi167 ceph-mon[32751]: pgmap v2674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:25.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3949 sec 2024-04-26T19:34:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:25 smithi090 ceph-mon[29180]: pgmap v2675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:26.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3547.029310 secs 2024-04-26T19:34:26.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3950 sec 2024-04-26T19:34:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:25 smithi119 ceph-mon[32752]: pgmap v2675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:26.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3547.029310 secs 2024-04-26T19:34:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:25 smithi167 ceph-mon[32751]: pgmap v2675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:26.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3547.029310 secs 2024-04-26T19:34:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3547.711819 secs 2024-04-26T19:34:27.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3951 sec 2024-04-26T19:34:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3547.711819 secs 2024-04-26T19:34:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3547.711819 secs 2024-04-26T19:34:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:27 smithi090 ceph-mon[29180]: pgmap v2676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:28.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3952 sec 2024-04-26T19:34:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:27 smithi119 ceph-mon[32752]: pgmap v2676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:27 smithi167 ceph-mon[32751]: pgmap v2676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:29.237 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3953 sec 2024-04-26T19:34:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:29 smithi090 ceph-mon[29180]: pgmap v2677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:30.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3954 sec 2024-04-26T19:34:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:29 smithi119 ceph-mon[32752]: pgmap v2677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:29 smithi167 ceph-mon[32751]: pgmap v2677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3552.029394 secs 2024-04-26T19:34:31.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3955 sec 2024-04-26T19:34:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3552.029394 secs 2024-04-26T19:34:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3552.029394 secs 2024-04-26T19:34:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3552.711921 secs 2024-04-26T19:34:32.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:31 smithi090 ceph-mon[29180]: pgmap v2678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:32.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3956 sec 2024-04-26T19:34:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3552.711921 secs 2024-04-26T19:34:32.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:31 smithi119 ceph-mon[32752]: pgmap v2678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3552.711921 secs 2024-04-26T19:34:32.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:31 smithi167 ceph-mon[32751]: pgmap v2678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:33.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3957 sec 2024-04-26T19:34:34.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3958 sec 2024-04-26T19:34:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:33 smithi119 ceph-mon[32752]: pgmap v2679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:34.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:33 smithi167 ceph-mon[32751]: pgmap v2679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:33 smithi090 ceph-mon[29180]: pgmap v2679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:35.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3959 sec 2024-04-26T19:34:36.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3960 sec 2024-04-26T19:34:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:35 smithi119 ceph-mon[32752]: pgmap v2680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:35 smithi119 ceph-mon[32752]: 9 slow requests, 2 included below; oldest blocked for > 3557.029481 secs 2024-04-26T19:34:36.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:35 smithi119 ceph-mon[32752]: slow request 2883.846866 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:34:36.334 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:35 smithi119 ceph-mon[32752]: slow request 363.166172 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:34:36.345 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:35 smithi167 ceph-mon[32751]: pgmap v2680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:36.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:35 smithi167 ceph-mon[32751]: 9 slow requests, 2 included below; oldest blocked for > 3557.029481 secs 2024-04-26T19:34:36.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:35 smithi167 ceph-mon[32751]: slow request 2883.846866 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:34:36.395 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:35 smithi167 ceph-mon[32751]: slow request 363.166172 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:34:36.406 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:35 smithi090 ceph-mon[29180]: pgmap v2680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:36.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:35 smithi090 ceph-mon[29180]: 9 slow requests, 2 included below; oldest blocked for > 3557.029481 secs 2024-04-26T19:34:36.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:35 smithi090 ceph-mon[29180]: slow request 2883.846866 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:34:36.437 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:35 smithi090 ceph-mon[29180]: slow request 363.166172 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:34:37.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3961 sec 2024-04-26T19:34:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3557.712032 secs 2024-04-26T19:34:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3557.712032 secs 2024-04-26T19:34:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3557.712032 secs 2024-04-26T19:34:38.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3962 sec 2024-04-26T19:34:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:37 smithi119 ceph-mon[32752]: pgmap v2681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:37 smithi167 ceph-mon[32751]: pgmap v2681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:37 smithi090 ceph-mon[29180]: pgmap v2681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:39.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3963 sec 2024-04-26T19:34:40.238 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3964 sec 2024-04-26T19:34:40.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:39 smithi119 ceph-mon[32752]: pgmap v2682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:40.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:39 smithi167 ceph-mon[32751]: pgmap v2682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:39 smithi090 ceph-mon[29180]: pgmap v2682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3562.029561 secs 2024-04-26T19:34:41.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3965 sec 2024-04-26T19:34:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3562.029561 secs 2024-04-26T19:34:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3562.029561 secs 2024-04-26T19:34:42.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3966 sec 2024-04-26T19:34:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3562.712150 secs 2024-04-26T19:34:42.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:41 smithi119 ceph-mon[32752]: pgmap v2683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:42.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3562.712150 secs 2024-04-26T19:34:42.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:41 smithi167 ceph-mon[32751]: pgmap v2683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3562.712150 secs 2024-04-26T19:34:42.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:41 smithi090 ceph-mon[29180]: pgmap v2683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:43.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3967 sec 2024-04-26T19:34:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:42 smithi119 ceph-mon[32752]: pgmap v2684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:43.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:42 smithi167 ceph-mon[32751]: pgmap v2684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:42 smithi090 ceph-mon[29180]: pgmap v2684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:44.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3968 sec 2024-04-26T19:34:45.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3969 sec 2024-04-26T19:34:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:45 smithi167 ceph-mon[32751]: pgmap v2685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:46.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3567.029645 secs 2024-04-26T19:34:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:45 smithi090 ceph-mon[29180]: pgmap v2685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:46.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3567.029645 secs 2024-04-26T19:34:46.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3970 sec 2024-04-26T19:34:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:45 smithi119 ceph-mon[32752]: pgmap v2685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:46.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3567.029645 secs 2024-04-26T19:34:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3567.712265 secs 2024-04-26T19:34:47.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3971 sec 2024-04-26T19:34:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3567.712265 secs 2024-04-26T19:34:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3567.712265 secs 2024-04-26T19:34:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:47 smithi090 ceph-mon[29180]: pgmap v2686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:48.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3972 sec 2024-04-26T19:34:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:47 smithi119 ceph-mon[32752]: pgmap v2686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:47 smithi167 ceph-mon[32751]: pgmap v2686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:34:49.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3973 sec 2024-04-26T19:34:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:49 smithi090 ceph-mon[29180]: pgmap v2687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:50.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:34:50.187 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:50.197 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:50.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3974 sec 2024-04-26T19:34:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:49 smithi119 ceph-mon[32752]: pgmap v2687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:50.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:34:50.322 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:50.333 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:50.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:49 smithi167 ceph-mon[32751]: pgmap v2687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:34:50.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:34:50.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:50.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3572.029727 secs 2024-04-26T19:34:51.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:34:51.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:34:51.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:51.239 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3975 sec 2024-04-26T19:34:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3572.029727 secs 2024-04-26T19:34:51.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:34:51.322 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:34:51.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:51.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3572.029727 secs 2024-04-26T19:34:51.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:34:51.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:34:51.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:34:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:51 smithi090 ceph-mon[29180]: pgmap v2688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T19:34:52.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:51 smithi090 ceph-mon[29180]: pgmap v2689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:52.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3572.712355 secs 2024-04-26T19:34:52.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3976 sec 2024-04-26T19:34:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:51 smithi119 ceph-mon[32752]: pgmap v2688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T19:34:52.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:51 smithi119 ceph-mon[32752]: pgmap v2689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:52.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3572.712355 secs 2024-04-26T19:34:52.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:51 smithi167 ceph-mon[32751]: pgmap v2688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T19:34:52.392 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:51 smithi167 ceph-mon[32751]: pgmap v2689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:52.457 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3572.712355 secs 2024-04-26T19:34:53.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3977 sec 2024-04-26T19:34:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:53 smithi090 ceph-mon[29180]: pgmap v2690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:54.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3978 sec 2024-04-26T19:34:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:53 smithi119 ceph-mon[32752]: pgmap v2690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:53 smithi167 ceph-mon[32751]: pgmap v2690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:55.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3979 sec 2024-04-26T19:34:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:55 smithi090 ceph-mon[29180]: pgmap v2691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:56.152 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3577.029832 secs 2024-04-26T19:34:56.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3980 sec 2024-04-26T19:34:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:55 smithi119 ceph-mon[32752]: pgmap v2691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:56.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3577.029832 secs 2024-04-26T19:34:56.365 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:55 smithi167 ceph-mon[32751]: pgmap v2691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:56.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3577.029832 secs 2024-04-26T19:34:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3577.712492 secs 2024-04-26T19:34:57.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3981 sec 2024-04-26T19:34:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3577.712492 secs 2024-04-26T19:34:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3577.712492 secs 2024-04-26T19:34:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:57 smithi090 ceph-mon[29180]: pgmap v2692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:58.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3982 sec 2024-04-26T19:34:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:57 smithi119 ceph-mon[32752]: pgmap v2692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:57 smithi167 ceph-mon[32751]: pgmap v2692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:34:59.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3983 sec 2024-04-26T19:35:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:34:59 smithi090 ceph-mon[29180]: pgmap v2693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:35:00.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3984 sec 2024-04-26T19:35:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:34:59 smithi119 ceph-mon[32752]: pgmap v2693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:35:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:34:59 smithi167 ceph-mon[32751]: pgmap v2693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T19:35:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3582.029916 secs 2024-04-26T19:35:01.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3582.029916 secs 2024-04-26T19:35:01.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3985 sec 2024-04-26T19:35:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3582.029916 secs 2024-04-26T19:35:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:01 smithi090 ceph-mon[29180]: pgmap v2694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:35:02.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3582.712609 secs 2024-04-26T19:35:02.240 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3986 sec 2024-04-26T19:35:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:01 smithi119 ceph-mon[32752]: pgmap v2694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:35:02.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3582.712609 secs 2024-04-26T19:35:02.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:01 smithi167 ceph-mon[32751]: pgmap v2694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:35:02.406 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3582.712609 secs 2024-04-26T19:35:03.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3987 sec 2024-04-26T19:35:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:03 smithi090 ceph-mon[29180]: pgmap v2695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:04.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3988 sec 2024-04-26T19:35:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:03 smithi119 ceph-mon[32752]: pgmap v2695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:03 smithi167 ceph-mon[32751]: pgmap v2695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:05.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3989 sec 2024-04-26T19:35:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:05 smithi090 ceph-mon[29180]: pgmap v2696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:06.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3587.030001 secs 2024-04-26T19:35:06.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3990 sec 2024-04-26T19:35:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:05 smithi119 ceph-mon[32752]: pgmap v2696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:06.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3587.030001 secs 2024-04-26T19:35:06.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:05 smithi167 ceph-mon[32751]: pgmap v2696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:06.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3587.030001 secs 2024-04-26T19:35:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3587.712703 secs 2024-04-26T19:35:07.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3991 sec 2024-04-26T19:35:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3587.712703 secs 2024-04-26T19:35:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3587.712703 secs 2024-04-26T19:35:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:07 smithi090 ceph-mon[29180]: pgmap v2697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:08.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3992 sec 2024-04-26T19:35:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:07 smithi119 ceph-mon[32752]: pgmap v2697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:08.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:07 smithi167 ceph-mon[32751]: pgmap v2697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:09.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3993 sec 2024-04-26T19:35:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:09 smithi090 ceph-mon[29180]: pgmap v2698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:10.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3994 sec 2024-04-26T19:35:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:09 smithi119 ceph-mon[32752]: pgmap v2698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:09 smithi167 ceph-mon[32751]: pgmap v2698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:11.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3592.030081 secs 2024-04-26T19:35:11.241 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3995 sec 2024-04-26T19:35:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3592.030081 secs 2024-04-26T19:35:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3592.030081 secs 2024-04-26T19:35:12.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3996 sec 2024-04-26T19:35:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:11 smithi119 ceph-mon[32752]: pgmap v2699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:12.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3592.712841 secs 2024-04-26T19:35:12.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:11 smithi167 ceph-mon[32751]: pgmap v2699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:12.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3592.712841 secs 2024-04-26T19:35:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:11 smithi090 ceph-mon[29180]: pgmap v2699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3592.712841 secs 2024-04-26T19:35:13.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3997 sec 2024-04-26T19:35:14.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3998 sec 2024-04-26T19:35:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:13 smithi119 ceph-mon[32752]: pgmap v2700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:13 smithi167 ceph-mon[32751]: pgmap v2700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:13 smithi090 ceph-mon[29180]: pgmap v2700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:15.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 3999 sec 2024-04-26T19:35:16.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4000 sec 2024-04-26T19:35:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:15 smithi119 ceph-mon[32752]: pgmap v2701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:16.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3597.030165 secs 2024-04-26T19:35:16.368 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:15 smithi167 ceph-mon[32751]: pgmap v2701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:16.377 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3597.030165 secs 2024-04-26T19:35:16.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:15 smithi090 ceph-mon[29180]: pgmap v2701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:16.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3597.030165 secs 2024-04-26T19:35:17.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4001 sec 2024-04-26T19:35:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3597.712953 secs 2024-04-26T19:35:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3597.712953 secs 2024-04-26T19:35:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3597.712953 secs 2024-04-26T19:35:18.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4002 sec 2024-04-26T19:35:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:17 smithi119 ceph-mon[32752]: pgmap v2702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:17 smithi167 ceph-mon[32751]: pgmap v2702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:17 smithi090 ceph-mon[29180]: pgmap v2702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:19.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4003 sec 2024-04-26T19:35:20.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4004 sec 2024-04-26T19:35:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:19 smithi119 ceph-mon[32752]: pgmap v2703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:19 smithi167 ceph-mon[32751]: pgmap v2703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:19 smithi090 ceph-mon[29180]: pgmap v2703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:21.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4005 sec 2024-04-26T19:35:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3602.030248 secs 2024-04-26T19:35:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3602.030248 secs 2024-04-26T19:35:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3602.030248 secs 2024-04-26T19:35:22.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4006 sec 2024-04-26T19:35:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:21 smithi119 ceph-mon[32752]: pgmap v2704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:22.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3602.713092 secs 2024-04-26T19:35:22.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:21 smithi167 ceph-mon[32751]: pgmap v2704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:22.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3602.713092 secs 2024-04-26T19:35:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:21 smithi090 ceph-mon[29180]: pgmap v2704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:22.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3602.713092 secs 2024-04-26T19:35:23.242 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4007 sec 2024-04-26T19:35:24.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4008 sec 2024-04-26T19:35:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:23 smithi119 ceph-mon[32752]: pgmap v2705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:23 smithi167 ceph-mon[32751]: pgmap v2705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:23 smithi090 ceph-mon[29180]: pgmap v2705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:25.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4009 sec 2024-04-26T19:35:26.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:25 smithi090 ceph-mon[29180]: pgmap v2706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:26.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3607.030330 secs 2024-04-26T19:35:26.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4010 sec 2024-04-26T19:35:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:25 smithi119 ceph-mon[32752]: pgmap v2706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:26.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3607.030330 secs 2024-04-26T19:35:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:25 smithi167 ceph-mon[32751]: pgmap v2706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:26.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3607.030330 secs 2024-04-26T19:35:27.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4011 sec 2024-04-26T19:35:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3607.713223 secs 2024-04-26T19:35:27.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:26 smithi119 ceph-mon[32752]: pgmap v2707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3607.713223 secs 2024-04-26T19:35:27.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:26 smithi167 ceph-mon[32751]: pgmap v2707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3607.713223 secs 2024-04-26T19:35:27.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:26 smithi090 ceph-mon[29180]: pgmap v2707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:28.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4012 sec 2024-04-26T19:35:28.469 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 msgr2=0x7f2cc00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T19:35:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 msgr2=0x7f2cc00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T19:35:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 0x7f2cc00b15b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010f60 tx=0x7f2cc4038040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:35:28.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 0x7f2cc00b15b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010f60 tx=0x7f2cc4038040 comp rx=0 tx=0).stop 2024-04-26T19:35:28.533 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 msgr2=0x7f2cc00b15b0 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T19:35:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 msgr2=0x7f2cc00b15b0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T19:35:28.634 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b11d0 0x7f2cc00b15b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:35:28.642 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:35:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.468+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:35:28.656 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.469+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b2280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401bc50 tx=0x7f2cbc0318e0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:35:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.490+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000100) for more than 900000000 us, fault. 2024-04-26T19:35:28.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=2 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T19:35:28.735 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:35:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T19:35:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.490+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T19:35:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.945+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:35:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.945+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:35:28.962 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.945+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 32 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:35:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:35:28.945+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 32 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:35:29.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4013 sec 2024-04-26T19:35:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:29 smithi090 ceph-mon[29180]: pgmap v2708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:29 smithi119 ceph-mon[32752]: pgmap v2708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:29.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:29 smithi167 ceph-mon[32751]: pgmap v2708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:30.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4014 sec 2024-04-26T19:35:31.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4015 sec 2024-04-26T19:35:31.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3612.030410 secs 2024-04-26T19:35:31.378 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:31 smithi167 ceph-mon[32751]: pgmap v2709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:31.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3612.713285 secs 2024-04-26T19:35:31.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3612.030410 secs 2024-04-26T19:35:31.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:31 smithi090 ceph-mon[29180]: pgmap v2709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:31.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3612.713285 secs 2024-04-26T19:35:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3612.030410 secs 2024-04-26T19:35:31.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:31 smithi119 ceph-mon[32752]: pgmap v2709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:31.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3612.713285 secs 2024-04-26T19:35:32.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4016 sec 2024-04-26T19:35:33.243 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4017 sec 2024-04-26T19:35:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:33 smithi090 ceph-mon[29180]: pgmap v2710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:33 smithi119 ceph-mon[32752]: pgmap v2710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:33.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:33 smithi167 ceph-mon[32751]: pgmap v2710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:34.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4018 sec 2024-04-26T19:35:35.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4019 sec 2024-04-26T19:35:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:35 smithi090 ceph-mon[29180]: pgmap v2711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:35 smithi119 ceph-mon[32752]: pgmap v2711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:35 smithi167 ceph-mon[32751]: pgmap v2711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:36.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4020 sec 2024-04-26T19:35:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:36 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3617.030495 secs 2024-04-26T19:35:36.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3617.713355 secs 2024-04-26T19:35:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:36 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3617.030495 secs 2024-04-26T19:35:36.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3617.713355 secs 2024-04-26T19:35:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:36 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3617.030495 secs 2024-04-26T19:35:36.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3617.713355 secs 2024-04-26T19:35:37.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4021 sec 2024-04-26T19:35:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:37 smithi119 ceph-mon[32752]: pgmap v2712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:37.625 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:37 smithi167 ceph-mon[32751]: pgmap v2712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:37.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:37 smithi090 ceph-mon[29180]: pgmap v2712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:38.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4022 sec 2024-04-26T19:35:39.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4023 sec 2024-04-26T19:35:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:39 smithi119 ceph-mon[32752]: pgmap v2713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:39 smithi167 ceph-mon[32751]: pgmap v2713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:39 smithi090 ceph-mon[29180]: pgmap v2713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:40.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4024 sec 2024-04-26T19:35:41.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4025 sec 2024-04-26T19:35:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:41 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3622.030577 secs 2024-04-26T19:35:41.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:41 smithi119 ceph-mon[32752]: pgmap v2714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:41.574 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3622.713447 secs 2024-04-26T19:35:41.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:41 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3622.030577 secs 2024-04-26T19:35:41.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:41 smithi167 ceph-mon[32751]: pgmap v2714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:41.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3622.713447 secs 2024-04-26T19:35:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:41 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3622.030577 secs 2024-04-26T19:35:41.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:41 smithi090 ceph-mon[29180]: pgmap v2714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:41.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3622.713447 secs 2024-04-26T19:35:42.244 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4026 sec 2024-04-26T19:35:43.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4027 sec 2024-04-26T19:35:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:43 smithi119 ceph-mon[32752]: pgmap v2715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:43.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:43 smithi090 ceph-mon[29180]: pgmap v2715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:43.661 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:43 smithi167 ceph-mon[32751]: pgmap v2715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:44.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4028 sec 2024-04-26T19:35:45.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4029 sec 2024-04-26T19:35:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:45 smithi119 ceph-mon[32752]: pgmap v2716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:45 smithi167 ceph-mon[32751]: pgmap v2716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:45 smithi090 ceph-mon[29180]: pgmap v2716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:46.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4030 sec 2024-04-26T19:35:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:46 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3627.030660 secs 2024-04-26T19:35:46.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3627.713535 secs 2024-04-26T19:35:46.662 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:46 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3627.030660 secs 2024-04-26T19:35:46.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3627.713535 secs 2024-04-26T19:35:46.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:46 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3627.030660 secs 2024-04-26T19:35:46.738 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3627.713535 secs 2024-04-26T19:35:47.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4031 sec 2024-04-26T19:35:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:47 smithi119 ceph-mon[32752]: pgmap v2717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:47 smithi167 ceph-mon[32751]: pgmap v2717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:47 smithi090 ceph-mon[29180]: pgmap v2717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:48.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4032 sec 2024-04-26T19:35:49.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4033 sec 2024-04-26T19:35:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:49 smithi119 ceph-mon[32752]: pgmap v2718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:49.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:49 smithi167 ceph-mon[32751]: pgmap v2718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:49 smithi090 ceph-mon[29180]: pgmap v2718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:35:50.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4034 sec 2024-04-26T19:35:51.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4035 sec 2024-04-26T19:35:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3632.030743 secs 2024-04-26T19:35:51.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: pgmap v2719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:51.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:35:51.577 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3632.713601 secs 2024-04-26T19:35:51.585 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:35:51.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:35:51.600 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: pgmap v2720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:35:51.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: pgmap v2721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:51.615 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:35:51.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3632.030743 secs 2024-04-26T19:35:51.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: pgmap v2719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:51.638 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:35:51.645 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3632.713601 secs 2024-04-26T19:35:51.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:35:51.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:35:51.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: pgmap v2720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:35:51.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: pgmap v2721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:51.882 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:35:51.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3632.030743 secs 2024-04-26T19:35:51.898 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: pgmap v2719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:35:51.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:35:51.913 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3632.713601 secs 2024-04-26T19:35:51.920 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:35:51.927 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:35:51.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: pgmap v2720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:35:51.942 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: pgmap v2721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:51.949 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:35:52.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4036 sec 2024-04-26T19:35:53.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4037 sec 2024-04-26T19:35:54.245 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4038 sec 2024-04-26T19:35:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:53 smithi119 ceph-mon[32752]: pgmap v2722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:35:54.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:53 smithi167 ceph-mon[32751]: pgmap v2722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:35:54.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:53 smithi090 ceph-mon[29180]: pgmap v2722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:35:55.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4039 sec 2024-04-26T19:35:56.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3637.030827 secs 2024-04-26T19:35:56.187 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:55 smithi090 ceph-mon[29180]: pgmap v2723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:56.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4040 sec 2024-04-26T19:35:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3637.030827 secs 2024-04-26T19:35:56.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:55 smithi119 ceph-mon[32752]: pgmap v2723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3637.030827 secs 2024-04-26T19:35:56.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:55 smithi167 ceph-mon[32751]: pgmap v2723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:57.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4041 sec 2024-04-26T19:35:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3637.713711 secs 2024-04-26T19:35:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3637.713711 secs 2024-04-26T19:35:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3637.713711 secs 2024-04-26T19:35:58.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4042 sec 2024-04-26T19:35:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:57 smithi119 ceph-mon[32752]: pgmap v2724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:57 smithi167 ceph-mon[32751]: pgmap v2724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:57 smithi090 ceph-mon[29180]: pgmap v2724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:35:59.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4043 sec 2024-04-26T19:36:00.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4044 sec 2024-04-26T19:36:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:35:59 smithi119 ceph-mon[32752]: pgmap v2725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:35:59 smithi167 ceph-mon[32751]: pgmap v2725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:35:59 smithi090 ceph-mon[29180]: pgmap v2725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:01.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3642.030885 secs 2024-04-26T19:36:01.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4045 sec 2024-04-26T19:36:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3642.030885 secs 2024-04-26T19:36:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3642.030885 secs 2024-04-26T19:36:02.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4046 sec 2024-04-26T19:36:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3642.713829 secs 2024-04-26T19:36:02.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:01 smithi119 ceph-mon[32752]: pgmap v2726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:36:02.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3642.713829 secs 2024-04-26T19:36:02.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:01 smithi167 ceph-mon[32751]: pgmap v2726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:36:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3642.713829 secs 2024-04-26T19:36:02.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:01 smithi090 ceph-mon[29180]: pgmap v2726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:36:03.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4047 sec 2024-04-26T19:36:04.246 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4048 sec 2024-04-26T19:36:04.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:03 smithi167 ceph-mon[32751]: pgmap v2727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:04.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:03 smithi119 ceph-mon[32752]: pgmap v2727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:03 smithi090 ceph-mon[29180]: pgmap v2727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:05.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4049 sec 2024-04-26T19:36:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3647.030975 secs 2024-04-26T19:36:06.149 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:05 smithi090 ceph-mon[29180]: pgmap v2728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:06.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4050 sec 2024-04-26T19:36:06.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3647.030975 secs 2024-04-26T19:36:06.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:05 smithi119 ceph-mon[32752]: pgmap v2728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:06.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3647.030975 secs 2024-04-26T19:36:06.400 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:05 smithi167 ceph-mon[32751]: pgmap v2728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:07.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4051 sec 2024-04-26T19:36:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3647.713941 secs 2024-04-26T19:36:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3647.713941 secs 2024-04-26T19:36:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3647.713941 secs 2024-04-26T19:36:08.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4052 sec 2024-04-26T19:36:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:07 smithi119 ceph-mon[32752]: pgmap v2729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:07 smithi167 ceph-mon[32751]: pgmap v2729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:07 smithi090 ceph-mon[29180]: pgmap v2729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:09.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4053 sec 2024-04-26T19:36:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:08 smithi119 ceph-mon[32752]: pgmap v2730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:08 smithi167 ceph-mon[32751]: pgmap v2730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:08 smithi090 ceph-mon[29180]: pgmap v2730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:10.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4054 sec 2024-04-26T19:36:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3652.031032 secs 2024-04-26T19:36:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3652.031032 secs 2024-04-26T19:36:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3652.031032 secs 2024-04-26T19:36:11.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4055 sec 2024-04-26T19:36:11.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3652.714022 secs 2024-04-26T19:36:11.713 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:11 smithi119 ceph-mon[32752]: pgmap v2731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3652.714022 secs 2024-04-26T19:36:11.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:11 smithi167 ceph-mon[32751]: pgmap v2731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3652.714022 secs 2024-04-26T19:36:11.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:11 smithi090 ceph-mon[29180]: pgmap v2731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:12.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4056 sec 2024-04-26T19:36:13.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4057 sec 2024-04-26T19:36:14.247 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4058 sec 2024-04-26T19:36:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:13 smithi119 ceph-mon[32752]: pgmap v2732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:13 smithi167 ceph-mon[32751]: pgmap v2732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:13 smithi090 ceph-mon[29180]: pgmap v2732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:15.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4059 sec 2024-04-26T19:36:16.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4060 sec 2024-04-26T19:36:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3657.031117 secs 2024-04-26T19:36:16.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:15 smithi167 ceph-mon[32751]: pgmap v2733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:16.335 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3657.031117 secs 2024-04-26T19:36:16.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:15 smithi119 ceph-mon[32752]: pgmap v2733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3657.031117 secs 2024-04-26T19:36:16.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:15 smithi090 ceph-mon[29180]: pgmap v2733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:17.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4061 sec 2024-04-26T19:36:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3657.714170 secs 2024-04-26T19:36:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3657.714170 secs 2024-04-26T19:36:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3657.714170 secs 2024-04-26T19:36:18.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4062 sec 2024-04-26T19:36:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:17 smithi119 ceph-mon[32752]: pgmap v2734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:17 smithi167 ceph-mon[32751]: pgmap v2734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:17 smithi090 ceph-mon[29180]: pgmap v2734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:19.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4063 sec 2024-04-26T19:36:20.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4064 sec 2024-04-26T19:36:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:19 smithi119 ceph-mon[32752]: pgmap v2735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:19 smithi167 ceph-mon[32751]: pgmap v2735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:19 smithi090 ceph-mon[29180]: pgmap v2735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:21.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4065 sec 2024-04-26T19:36:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3662.031175 secs 2024-04-26T19:36:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3662.031175 secs 2024-04-26T19:36:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3662.031175 secs 2024-04-26T19:36:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3662.714232 secs 2024-04-26T19:36:22.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:21 smithi090 ceph-mon[29180]: pgmap v2736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:22.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4066 sec 2024-04-26T19:36:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3662.714232 secs 2024-04-26T19:36:22.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:21 smithi119 ceph-mon[32752]: pgmap v2736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3662.714232 secs 2024-04-26T19:36:22.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:21 smithi167 ceph-mon[32751]: pgmap v2736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:23.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4067 sec 2024-04-26T19:36:24.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4068 sec 2024-04-26T19:36:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:23 smithi119 ceph-mon[32752]: pgmap v2737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:23 smithi167 ceph-mon[32751]: pgmap v2737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:23 smithi090 ceph-mon[29180]: pgmap v2737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:25.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4069 sec 2024-04-26T19:36:26.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3667.031262 secs 2024-04-26T19:36:26.188 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:25 smithi090 ceph-mon[29180]: pgmap v2738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:26.248 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4070 sec 2024-04-26T19:36:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3667.031262 secs 2024-04-26T19:36:26.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:25 smithi119 ceph-mon[32752]: pgmap v2738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:26.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3667.031262 secs 2024-04-26T19:36:26.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:25 smithi167 ceph-mon[32751]: pgmap v2738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:27.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4071 sec 2024-04-26T19:36:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3667.714366 secs 2024-04-26T19:36:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3667.714366 secs 2024-04-26T19:36:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3667.714366 secs 2024-04-26T19:36:28.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4072 sec 2024-04-26T19:36:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:27 smithi119 ceph-mon[32752]: pgmap v2739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:27 smithi167 ceph-mon[32751]: pgmap v2739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:27 smithi090 ceph-mon[29180]: pgmap v2739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:29.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4073 sec 2024-04-26T19:36:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:28 smithi119 ceph-mon[32752]: pgmap v2740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:28 smithi167 ceph-mon[32751]: pgmap v2740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:28 smithi090 ceph-mon[29180]: pgmap v2740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:30.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4074 sec 2024-04-26T19:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3672.031320 secs 2024-04-26T19:36:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3672.031320 secs 2024-04-26T19:36:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3672.031320 secs 2024-04-26T19:36:31.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4075 sec 2024-04-26T19:36:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3672.714457 secs 2024-04-26T19:36:31.777 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:31 smithi167 ceph-mon[32751]: pgmap v2741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3672.714457 secs 2024-04-26T19:36:31.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:31 smithi119 ceph-mon[32752]: pgmap v2741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3672.714457 secs 2024-04-26T19:36:31.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:31 smithi090 ceph-mon[29180]: pgmap v2741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:32.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4076 sec 2024-04-26T19:36:33.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4077 sec 2024-04-26T19:36:34.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4078 sec 2024-04-26T19:36:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:33 smithi119 ceph-mon[32752]: pgmap v2742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:33 smithi167 ceph-mon[32751]: pgmap v2742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:33 smithi090 ceph-mon[29180]: pgmap v2742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:35.249 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4079 sec 2024-04-26T19:36:36.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4080 sec 2024-04-26T19:36:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3677.031400 secs 2024-04-26T19:36:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:35 smithi119 ceph-mon[32752]: pgmap v2743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3677.031400 secs 2024-04-26T19:36:36.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:35 smithi167 ceph-mon[32751]: pgmap v2743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3677.031400 secs 2024-04-26T19:36:36.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:35 smithi090 ceph-mon[29180]: pgmap v2743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:37.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4081 sec 2024-04-26T19:36:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3677.714569 secs 2024-04-26T19:36:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3677.714569 secs 2024-04-26T19:36:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3677.714569 secs 2024-04-26T19:36:38.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4082 sec 2024-04-26T19:36:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:37 smithi119 ceph-mon[32752]: pgmap v2744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:37 smithi167 ceph-mon[32751]: pgmap v2744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:37 smithi090 ceph-mon[29180]: pgmap v2744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:39.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4083 sec 2024-04-26T19:36:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:39 smithi119 ceph-mon[32752]: pgmap v2745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:39 smithi167 ceph-mon[32751]: pgmap v2745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:39 smithi090 ceph-mon[29180]: pgmap v2745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:40.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4084 sec 2024-04-26T19:36:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3682.031478 secs 2024-04-26T19:36:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3682.031478 secs 2024-04-26T19:36:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3682.031478 secs 2024-04-26T19:36:41.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4085 sec 2024-04-26T19:36:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3682.714662 secs 2024-04-26T19:36:41.831 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:41 smithi119 ceph-mon[32752]: pgmap v2746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:41.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3682.714662 secs 2024-04-26T19:36:41.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:41 smithi167 ceph-mon[32751]: pgmap v2746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3682.714662 secs 2024-04-26T19:36:41.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:41 smithi090 ceph-mon[29180]: pgmap v2746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:42.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4086 sec 2024-04-26T19:36:43.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4087 sec 2024-04-26T19:36:44.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4088 sec 2024-04-26T19:36:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:43 smithi119 ceph-mon[32752]: pgmap v2747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:43 smithi167 ceph-mon[32751]: pgmap v2747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:43 smithi090 ceph-mon[29180]: pgmap v2747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:45.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4089 sec 2024-04-26T19:36:46.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4090 sec 2024-04-26T19:36:46.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3687.031538 secs 2024-04-26T19:36:46.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:45 smithi119 ceph-mon[32752]: pgmap v2748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:46.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3687.031538 secs 2024-04-26T19:36:46.352 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:45 smithi167 ceph-mon[32751]: pgmap v2748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3687.031538 secs 2024-04-26T19:36:46.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:45 smithi090 ceph-mon[29180]: pgmap v2748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:47.250 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4091 sec 2024-04-26T19:36:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3687.714736 secs 2024-04-26T19:36:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3687.714736 secs 2024-04-26T19:36:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3687.714736 secs 2024-04-26T19:36:48.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4092 sec 2024-04-26T19:36:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:47 smithi119 ceph-mon[32752]: pgmap v2749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:47 smithi167 ceph-mon[32751]: pgmap v2749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:47 smithi090 ceph-mon[29180]: pgmap v2749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:49.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4093 sec 2024-04-26T19:36:50.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4094 sec 2024-04-26T19:36:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:49 smithi119 ceph-mon[32752]: pgmap v2750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:49 smithi167 ceph-mon[32751]: pgmap v2750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:49 smithi090 ceph-mon[29180]: pgmap v2750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:36:51.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4095 sec 2024-04-26T19:36:51.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3692.031618 secs 2024-04-26T19:36:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3692.031618 secs 2024-04-26T19:36:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3692.031618 secs 2024-04-26T19:36:52.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4096 sec 2024-04-26T19:36:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3692.714801 secs 2024-04-26T19:36:52.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:51 smithi119 ceph-mon[32752]: pgmap v2751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:52.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:36:52.330 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:36:52.338 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:36:52.347 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:36:52.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3692.714801 secs 2024-04-26T19:36:52.366 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:51 smithi167 ceph-mon[32751]: pgmap v2751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:52.375 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:36:52.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:36:52.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:36:52.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:36:52.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3692.714801 secs 2024-04-26T19:36:52.541 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:51 smithi090 ceph-mon[29180]: pgmap v2751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:36:52.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:36:52.558 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:36:52.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:36:52.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:36:53.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4097 sec 2024-04-26T19:36:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:52 smithi119 ceph-mon[32752]: pgmap v2752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:36:53.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:52 smithi119 ceph-mon[32752]: pgmap v2753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:53.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:52 smithi167 ceph-mon[32751]: pgmap v2752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:36:53.438 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:52 smithi167 ceph-mon[32751]: pgmap v2753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:53.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:52 smithi090 ceph-mon[29180]: pgmap v2752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:36:53.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:52 smithi090 ceph-mon[29180]: pgmap v2753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:54.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4098 sec 2024-04-26T19:36:55.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4099 sec 2024-04-26T19:36:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:54 smithi119 ceph-mon[32752]: pgmap v2754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:54 smithi167 ceph-mon[32751]: pgmap v2754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:54 smithi090 ceph-mon[29180]: pgmap v2754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:56.251 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4100 sec 2024-04-26T19:36:56.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3697.031701 secs 2024-04-26T19:36:56.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3697.031701 secs 2024-04-26T19:36:56.395 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3697.031701 secs 2024-04-26T19:36:57.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4101 sec 2024-04-26T19:36:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3697.714915 secs 2024-04-26T19:36:57.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:56 smithi119 ceph-mon[32752]: pgmap v2755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3697.714915 secs 2024-04-26T19:36:57.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:56 smithi167 ceph-mon[32751]: pgmap v2755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3697.714915 secs 2024-04-26T19:36:57.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:56 smithi090 ceph-mon[29180]: pgmap v2755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:36:58.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4102 sec 2024-04-26T19:36:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:36:57 smithi119 ceph-mon[32752]: pgmap v2756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:36:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:36:57 smithi167 ceph-mon[32751]: pgmap v2756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:36:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:36:57 smithi090 ceph-mon[29180]: pgmap v2756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:36:59.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4103 sec 2024-04-26T19:37:00.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4104 sec 2024-04-26T19:37:01.021 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:00 smithi167 ceph-mon[32751]: pgmap v2757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:01.029 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3702.031778 secs 2024-04-26T19:37:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:00 smithi119 ceph-mon[32752]: pgmap v2757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:01.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3702.031778 secs 2024-04-26T19:37:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:00 smithi090 ceph-mon[29180]: pgmap v2757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:01.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3702.031778 secs 2024-04-26T19:37:01.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4105 sec 2024-04-26T19:37:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3702.715024 secs 2024-04-26T19:37:02.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3702.715024 secs 2024-04-26T19:37:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3702.715024 secs 2024-04-26T19:37:02.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4106 sec 2024-04-26T19:37:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:02 smithi119 ceph-mon[32752]: pgmap v2758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:37:03.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:02 smithi167 ceph-mon[32751]: pgmap v2758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:37:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:02 smithi090 ceph-mon[29180]: pgmap v2758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:37:03.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4107 sec 2024-04-26T19:37:04.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4108 sec 2024-04-26T19:37:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:04 smithi119 ceph-mon[32752]: pgmap v2759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:04 smithi167 ceph-mon[32751]: pgmap v2759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:05.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:04 smithi090 ceph-mon[29180]: pgmap v2759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:05.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4109 sec 2024-04-26T19:37:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3707.031864 secs 2024-04-26T19:37:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3707.031864 secs 2024-04-26T19:37:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3707.031864 secs 2024-04-26T19:37:06.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4110 sec 2024-04-26T19:37:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3707.715139 secs 2024-04-26T19:37:07.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:06 smithi119 ceph-mon[32752]: pgmap v2760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:07.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3707.715139 secs 2024-04-26T19:37:07.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:06 smithi167 ceph-mon[32751]: pgmap v2760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:07.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3707.715139 secs 2024-04-26T19:37:07.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:06 smithi090 ceph-mon[29180]: pgmap v2760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:07.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4111 sec 2024-04-26T19:37:08.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4112 sec 2024-04-26T19:37:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:08 smithi119 ceph-mon[32752]: pgmap v2761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:08 smithi167 ceph-mon[32751]: pgmap v2761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:08 smithi090 ceph-mon[29180]: pgmap v2761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:09.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4113 sec 2024-04-26T19:37:10.252 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4114 sec 2024-04-26T19:37:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:10 smithi119 ceph-mon[32752]: pgmap v2762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:11.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3712.031946 secs 2024-04-26T19:37:11.191 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:10 smithi090 ceph-mon[29180]: pgmap v2762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:11.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3712.031946 secs 2024-04-26T19:37:11.226 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:10 smithi167 ceph-mon[32751]: pgmap v2762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:11.235 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3712.031946 secs 2024-04-26T19:37:11.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4115 sec 2024-04-26T19:37:11.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3712.715260 secs 2024-04-26T19:37:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3712.715260 secs 2024-04-26T19:37:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3712.715260 secs 2024-04-26T19:37:12.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4116 sec 2024-04-26T19:37:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:12 smithi119 ceph-mon[32752]: pgmap v2763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:12 smithi167 ceph-mon[32751]: pgmap v2763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:12 smithi090 ceph-mon[29180]: pgmap v2763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:13.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4117 sec 2024-04-26T19:37:14.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4118 sec 2024-04-26T19:37:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:14 smithi119 ceph-mon[32752]: pgmap v2764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:14 smithi167 ceph-mon[32751]: pgmap v2764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:14 smithi090 ceph-mon[29180]: pgmap v2764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:15.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4119 sec 2024-04-26T19:37:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3717.032009 secs 2024-04-26T19:37:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3717.032009 secs 2024-04-26T19:37:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3717.032009 secs 2024-04-26T19:37:16.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4120 sec 2024-04-26T19:37:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3717.715372 secs 2024-04-26T19:37:17.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:16 smithi119 ceph-mon[32752]: pgmap v2765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:17.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3717.715372 secs 2024-04-26T19:37:17.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:16 smithi167 ceph-mon[32751]: pgmap v2765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3717.715372 secs 2024-04-26T19:37:17.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:16 smithi090 ceph-mon[29180]: pgmap v2765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:17.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4121 sec 2024-04-26T19:37:18.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4122 sec 2024-04-26T19:37:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:18 smithi119 ceph-mon[32752]: pgmap v2766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:18 smithi167 ceph-mon[32751]: pgmap v2766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:18 smithi090 ceph-mon[29180]: pgmap v2766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:19.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4123 sec 2024-04-26T19:37:20.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4124 sec 2024-04-26T19:37:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:20 smithi119 ceph-mon[32752]: pgmap v2767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:21.067 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3722.032092 secs 2024-04-26T19:37:21.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:20 smithi167 ceph-mon[32751]: pgmap v2767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:21.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3722.032092 secs 2024-04-26T19:37:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:20 smithi090 ceph-mon[29180]: pgmap v2767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:21.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3722.032092 secs 2024-04-26T19:37:21.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4125 sec 2024-04-26T19:37:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3722.715488 secs 2024-04-26T19:37:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3722.715488 secs 2024-04-26T19:37:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3722.715488 secs 2024-04-26T19:37:22.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4126 sec 2024-04-26T19:37:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:22 smithi119 ceph-mon[32752]: pgmap v2768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:23.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:22 smithi167 ceph-mon[32751]: pgmap v2768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:22 smithi090 ceph-mon[29180]: pgmap v2768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:23.253 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4127 sec 2024-04-26T19:37:24.254 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4128 sec 2024-04-26T19:37:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:24 smithi119 ceph-mon[32752]: pgmap v2769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:24 smithi167 ceph-mon[32751]: pgmap v2769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:24 smithi090 ceph-mon[29180]: pgmap v2769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:25.254 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4129 sec 2024-04-26T19:37:26.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3727.032177 secs 2024-04-26T19:37:26.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3727.032177 secs 2024-04-26T19:37:26.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3727.032177 secs 2024-04-26T19:37:26.254 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4130 sec 2024-04-26T19:37:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3727.715604 secs 2024-04-26T19:37:27.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:26 smithi119 ceph-mon[32752]: pgmap v2770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:27.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3727.715604 secs 2024-04-26T19:37:27.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:26 smithi167 ceph-mon[32751]: pgmap v2770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3727.715604 secs 2024-04-26T19:37:27.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:26 smithi090 ceph-mon[29180]: pgmap v2770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:27.254 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4131 sec 2024-04-26T19:37:28.254 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4132 sec 2024-04-26T19:37:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:28 smithi119 ceph-mon[32752]: pgmap v2771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:28 smithi167 ceph-mon[32751]: pgmap v2771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:28 smithi090 ceph-mon[29180]: pgmap v2771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:29.254 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4133 sec 2024-04-26T19:37:30.254 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4134 sec 2024-04-26T19:37:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:30 smithi119 ceph-mon[32752]: pgmap v2772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:31.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3732.032235 secs 2024-04-26T19:37:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:30 smithi167 ceph-mon[32751]: pgmap v2772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:31.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3732.032235 secs 2024-04-26T19:37:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:30 smithi090 ceph-mon[29180]: pgmap v2772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:31.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3732.032235 secs 2024-04-26T19:37:31.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4135 sec 2024-04-26T19:37:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3732.715713 secs 2024-04-26T19:37:32.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3732.715713 secs 2024-04-26T19:37:32.190 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3732.715713 secs 2024-04-26T19:37:32.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4136 sec 2024-04-26T19:37:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:32 smithi119 ceph-mon[32752]: pgmap v2773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:33.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:32 smithi167 ceph-mon[32751]: pgmap v2773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:32 smithi090 ceph-mon[29180]: pgmap v2773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:33.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4137 sec 2024-04-26T19:37:34.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4138 sec 2024-04-26T19:37:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:34 smithi119 ceph-mon[32752]: pgmap v2774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:35.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:34 smithi167 ceph-mon[32751]: pgmap v2774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:34 smithi090 ceph-mon[29180]: pgmap v2774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:35.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4139 sec 2024-04-26T19:37:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3737.032296 secs 2024-04-26T19:37:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3737.032296 secs 2024-04-26T19:37:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3737.032296 secs 2024-04-26T19:37:36.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4140 sec 2024-04-26T19:37:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3737.715816 secs 2024-04-26T19:37:37.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:36 smithi119 ceph-mon[32752]: pgmap v2775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3737.715816 secs 2024-04-26T19:37:37.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:36 smithi167 ceph-mon[32751]: pgmap v2775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3737.715816 secs 2024-04-26T19:37:37.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:36 smithi090 ceph-mon[29180]: pgmap v2775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:37.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4141 sec 2024-04-26T19:37:38.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4142 sec 2024-04-26T19:37:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:38 smithi119 ceph-mon[32752]: pgmap v2776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:39.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:38 smithi167 ceph-mon[32751]: pgmap v2776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:38 smithi090 ceph-mon[29180]: pgmap v2776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:39.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4143 sec 2024-04-26T19:37:40.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4144 sec 2024-04-26T19:37:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:40 smithi119 ceph-mon[32752]: pgmap v2777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:41.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3742.032376 secs 2024-04-26T19:37:41.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:40 smithi167 ceph-mon[32751]: pgmap v2777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:41.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3742.032376 secs 2024-04-26T19:37:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:40 smithi090 ceph-mon[29180]: pgmap v2777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:41.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3742.032376 secs 2024-04-26T19:37:41.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4145 sec 2024-04-26T19:37:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3742.715887 secs 2024-04-26T19:37:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3742.715887 secs 2024-04-26T19:37:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3742.715887 secs 2024-04-26T19:37:42.255 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4146 sec 2024-04-26T19:37:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:42 smithi119 ceph-mon[32752]: pgmap v2778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:42 smithi167 ceph-mon[32751]: pgmap v2778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:42 smithi090 ceph-mon[29180]: pgmap v2778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:43.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4147 sec 2024-04-26T19:37:44.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4148 sec 2024-04-26T19:37:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:44 smithi119 ceph-mon[32752]: pgmap v2779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:45.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:44 smithi167 ceph-mon[32751]: pgmap v2779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:44 smithi090 ceph-mon[29180]: pgmap v2779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:45.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4149 sec 2024-04-26T19:37:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3747.032470 secs 2024-04-26T19:37:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3747.032470 secs 2024-04-26T19:37:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3747.032470 secs 2024-04-26T19:37:46.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4150 sec 2024-04-26T19:37:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3747.716022 secs 2024-04-26T19:37:47.083 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:46 smithi119 ceph-mon[32752]: pgmap v2780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:47.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3747.716022 secs 2024-04-26T19:37:47.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:46 smithi167 ceph-mon[32751]: pgmap v2780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3747.716022 secs 2024-04-26T19:37:47.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:46 smithi090 ceph-mon[29180]: pgmap v2780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:47.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4151 sec 2024-04-26T19:37:48.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4152 sec 2024-04-26T19:37:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:48 smithi119 ceph-mon[32752]: pgmap v2781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:48 smithi167 ceph-mon[32751]: pgmap v2781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:48 smithi090 ceph-mon[29180]: pgmap v2781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:49.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4153 sec 2024-04-26T19:37:50.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4154 sec 2024-04-26T19:37:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:50 smithi119 ceph-mon[32752]: pgmap v2782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:51.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3752.032535 secs 2024-04-26T19:37:51.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:50 smithi167 ceph-mon[32751]: pgmap v2782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:51.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3752.032535 secs 2024-04-26T19:37:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:50 smithi090 ceph-mon[29180]: pgmap v2782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:37:51.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3752.032535 secs 2024-04-26T19:37:51.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4155 sec 2024-04-26T19:37:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3752.716138 secs 2024-04-26T19:37:52.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:37:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3752.716138 secs 2024-04-26T19:37:52.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:37:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3752.716138 secs 2024-04-26T19:37:52.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:37:52.256 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4156 sec 2024-04-26T19:37:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:52 smithi119 ceph-mon[32752]: pgmap v2783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:53.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:37:53.072 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:37:53.237 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:37:53.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:52 smithi090 ceph-mon[29180]: pgmap v2783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:53.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:37:53.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:37:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:37:53.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:52 smithi167 ceph-mon[32751]: pgmap v2783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:37:53.398 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:37:53.409 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:37:53.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:37:53.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4157 sec 2024-04-26T19:37:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:53 smithi119 ceph-mon[32752]: pgmap v2784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:37:54.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:53 smithi119 ceph-mon[32752]: pgmap v2785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:37:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:53 smithi167 ceph-mon[32751]: pgmap v2784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:37:54.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:53 smithi167 ceph-mon[32751]: pgmap v2785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:37:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:53 smithi090 ceph-mon[29180]: pgmap v2784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:37:54.216 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:53 smithi090 ceph-mon[29180]: pgmap v2785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:37:54.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4158 sec 2024-04-26T19:37:55.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4159 sec 2024-04-26T19:37:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:55 smithi119 ceph-mon[32752]: pgmap v2786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:56.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3757.032593 secs 2024-04-26T19:37:56.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:55 smithi167 ceph-mon[32751]: pgmap v2786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:56.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3757.032593 secs 2024-04-26T19:37:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:55 smithi090 ceph-mon[29180]: pgmap v2786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:56.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3757.032593 secs 2024-04-26T19:37:56.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4160 sec 2024-04-26T19:37:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3757.716277 secs 2024-04-26T19:37:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3757.716277 secs 2024-04-26T19:37:57.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4161 sec 2024-04-26T19:37:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3757.716277 secs 2024-04-26T19:37:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:57 smithi167 ceph-mon[32751]: pgmap v2787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:57 smithi090 ceph-mon[29180]: pgmap v2787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:58.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4162 sec 2024-04-26T19:37:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:57 smithi119 ceph-mon[32752]: pgmap v2787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:37:59.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4163 sec 2024-04-26T19:38:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:37:59 smithi167 ceph-mon[32751]: pgmap v2788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:38:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:37:59 smithi090 ceph-mon[29180]: pgmap v2788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:38:00.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4164 sec 2024-04-26T19:38:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:37:59 smithi119 ceph-mon[32752]: pgmap v2788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:38:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3762.032677 secs 2024-04-26T19:38:01.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3762.032677 secs 2024-04-26T19:38:01.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4165 sec 2024-04-26T19:38:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3762.032677 secs 2024-04-26T19:38:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:01 smithi167 ceph-mon[32751]: pgmap v2789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:38:02.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3762.716418 secs 2024-04-26T19:38:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:01 smithi090 ceph-mon[29180]: pgmap v2789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:38:02.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3762.716418 secs 2024-04-26T19:38:02.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4166 sec 2024-04-26T19:38:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:01 smithi119 ceph-mon[32752]: pgmap v2789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:38:02.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3762.716418 secs 2024-04-26T19:38:03.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4167 sec 2024-04-26T19:38:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:03 smithi090 ceph-mon[29180]: pgmap v2790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:38:04.257 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4168 sec 2024-04-26T19:38:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:03 smithi119 ceph-mon[32752]: pgmap v2790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:38:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:03 smithi167 ceph-mon[32751]: pgmap v2790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:38:05.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4169 sec 2024-04-26T19:38:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:05 smithi090 ceph-mon[29180]: pgmap v2791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:06.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3767.032762 secs 2024-04-26T19:38:06.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4170 sec 2024-04-26T19:38:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:05 smithi119 ceph-mon[32752]: pgmap v2791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:06.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3767.032762 secs 2024-04-26T19:38:06.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:05 smithi167 ceph-mon[32751]: pgmap v2791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:06.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3767.032762 secs 2024-04-26T19:38:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3767.716530 secs 2024-04-26T19:38:07.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4171 sec 2024-04-26T19:38:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3767.716530 secs 2024-04-26T19:38:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3767.716530 secs 2024-04-26T19:38:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:07 smithi090 ceph-mon[29180]: pgmap v2792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:08.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4172 sec 2024-04-26T19:38:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:07 smithi119 ceph-mon[32752]: pgmap v2792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:07 smithi167 ceph-mon[32751]: pgmap v2792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:09.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4173 sec 2024-04-26T19:38:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:09 smithi090 ceph-mon[29180]: pgmap v2793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:10.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4174 sec 2024-04-26T19:38:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:09 smithi119 ceph-mon[32752]: pgmap v2793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:09 smithi167 ceph-mon[32751]: pgmap v2793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3772.032843 secs 2024-04-26T19:38:11.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3772.032843 secs 2024-04-26T19:38:11.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4175 sec 2024-04-26T19:38:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3772.032843 secs 2024-04-26T19:38:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:11 smithi090 ceph-mon[29180]: pgmap v2794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:12.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3772.716642 secs 2024-04-26T19:38:12.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4176 sec 2024-04-26T19:38:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:11 smithi119 ceph-mon[32752]: pgmap v2794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:12.331 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3772.716642 secs 2024-04-26T19:38:12.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:11 smithi167 ceph-mon[32751]: pgmap v2794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:12.351 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3772.716642 secs 2024-04-26T19:38:13.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4177 sec 2024-04-26T19:38:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:13 smithi090 ceph-mon[29180]: pgmap v2795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:14.258 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4178 sec 2024-04-26T19:38:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:13 smithi119 ceph-mon[32752]: pgmap v2795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:13 smithi167 ceph-mon[32751]: pgmap v2795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:15.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4179 sec 2024-04-26T19:38:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:15 smithi090 ceph-mon[29180]: pgmap v2796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:16.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3777.032902 secs 2024-04-26T19:38:16.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4180 sec 2024-04-26T19:38:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:15 smithi119 ceph-mon[32752]: pgmap v2796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:16.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3777.032902 secs 2024-04-26T19:38:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:15 smithi167 ceph-mon[32751]: pgmap v2796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:16.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3777.032902 secs 2024-04-26T19:38:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3777.716750 secs 2024-04-26T19:38:17.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4181 sec 2024-04-26T19:38:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3777.716750 secs 2024-04-26T19:38:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3777.716750 secs 2024-04-26T19:38:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:17 smithi090 ceph-mon[29180]: pgmap v2797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:18.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4182 sec 2024-04-26T19:38:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:17 smithi119 ceph-mon[32752]: pgmap v2797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:18.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:17 smithi167 ceph-mon[32751]: pgmap v2797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:19.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4183 sec 2024-04-26T19:38:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:19 smithi090 ceph-mon[29180]: pgmap v2798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:20.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4184 sec 2024-04-26T19:38:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:19 smithi119 ceph-mon[32752]: pgmap v2798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:19 smithi167 ceph-mon[32751]: pgmap v2798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3782.032962 secs 2024-04-26T19:38:21.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4185 sec 2024-04-26T19:38:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3782.032962 secs 2024-04-26T19:38:21.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3782.032962 secs 2024-04-26T19:38:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:21 smithi090 ceph-mon[29180]: pgmap v2799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:22.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3782.716860 secs 2024-04-26T19:38:22.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4186 sec 2024-04-26T19:38:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:21 smithi119 ceph-mon[32752]: pgmap v2799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:22.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3782.716860 secs 2024-04-26T19:38:22.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:21 smithi167 ceph-mon[32751]: pgmap v2799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:22.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3782.716860 secs 2024-04-26T19:38:23.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4187 sec 2024-04-26T19:38:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:23 smithi090 ceph-mon[29180]: pgmap v2800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:24.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4188 sec 2024-04-26T19:38:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:23 smithi119 ceph-mon[32752]: pgmap v2800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:23 smithi167 ceph-mon[32751]: pgmap v2800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:25.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4189 sec 2024-04-26T19:38:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:25 smithi090 ceph-mon[29180]: pgmap v2801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:26.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3787.033048 secs 2024-04-26T19:38:26.259 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4190 sec 2024-04-26T19:38:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:25 smithi119 ceph-mon[32752]: pgmap v2801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:26.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3787.033048 secs 2024-04-26T19:38:26.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:25 smithi167 ceph-mon[32751]: pgmap v2801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:26.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3787.033048 secs 2024-04-26T19:38:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3787.716953 secs 2024-04-26T19:38:27.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4191 sec 2024-04-26T19:38:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3787.716953 secs 2024-04-26T19:38:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3787.716953 secs 2024-04-26T19:38:28.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4192 sec 2024-04-26T19:38:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:27 smithi119 ceph-mon[32752]: pgmap v2802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:27 smithi167 ceph-mon[32751]: pgmap v2802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:27 smithi090 ceph-mon[29180]: pgmap v2802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:38:28.970+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:38:28.978 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:38:28.970+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:38:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:38:28.970+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 33 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:38:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:38:28.970+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 33 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:38:29.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4193 sec 2024-04-26T19:38:30.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4194 sec 2024-04-26T19:38:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:29 smithi119 ceph-mon[32752]: pgmap v2803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:29 smithi167 ceph-mon[32751]: pgmap v2803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:29 smithi090 ceph-mon[29180]: pgmap v2803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3792.033109 secs 2024-04-26T19:38:31.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4195 sec 2024-04-26T19:38:31.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3792.033109 secs 2024-04-26T19:38:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3792.033109 secs 2024-04-26T19:38:32.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4196 sec 2024-04-26T19:38:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:31 smithi119 ceph-mon[32752]: pgmap v2804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:32.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3792.717072 secs 2024-04-26T19:38:32.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:31 smithi167 ceph-mon[32751]: pgmap v2804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:32.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3792.717072 secs 2024-04-26T19:38:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:31 smithi090 ceph-mon[29180]: pgmap v2804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:32.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3792.717072 secs 2024-04-26T19:38:33.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4197 sec 2024-04-26T19:38:34.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4198 sec 2024-04-26T19:38:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:33 smithi119 ceph-mon[32752]: pgmap v2805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:33 smithi167 ceph-mon[32751]: pgmap v2805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:33 smithi090 ceph-mon[29180]: pgmap v2805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:35.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4199 sec 2024-04-26T19:38:36.260 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4200 sec 2024-04-26T19:38:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:35 smithi119 ceph-mon[32752]: pgmap v2806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3797.033185 secs 2024-04-26T19:38:36.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:35 smithi167 ceph-mon[32751]: pgmap v2806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:36.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3797.033185 secs 2024-04-26T19:38:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:35 smithi090 ceph-mon[29180]: pgmap v2806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:36.411 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3797.033185 secs 2024-04-26T19:38:37.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4201 sec 2024-04-26T19:38:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3797.717225 secs 2024-04-26T19:38:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3797.717225 secs 2024-04-26T19:38:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3797.717225 secs 2024-04-26T19:38:38.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4202 sec 2024-04-26T19:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:37 smithi119 ceph-mon[32752]: pgmap v2807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:37 smithi167 ceph-mon[32751]: pgmap v2807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:37 smithi090 ceph-mon[29180]: pgmap v2807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:39.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4203 sec 2024-04-26T19:38:40.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4204 sec 2024-04-26T19:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:39 smithi119 ceph-mon[32752]: pgmap v2808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:39 smithi167 ceph-mon[32751]: pgmap v2808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:39 smithi090 ceph-mon[29180]: pgmap v2808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3802.033263 secs 2024-04-26T19:38:41.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4205 sec 2024-04-26T19:38:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3802.033263 secs 2024-04-26T19:38:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3802.033263 secs 2024-04-26T19:38:42.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4206 sec 2024-04-26T19:38:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:41 smithi119 ceph-mon[32752]: pgmap v2809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:42.319 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3802.717362 secs 2024-04-26T19:38:42.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:41 smithi167 ceph-mon[32751]: pgmap v2809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3802.717362 secs 2024-04-26T19:38:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:41 smithi090 ceph-mon[29180]: pgmap v2809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:42.422 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3802.717362 secs 2024-04-26T19:38:43.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4207 sec 2024-04-26T19:38:44.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4208 sec 2024-04-26T19:38:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:43 smithi119 ceph-mon[32752]: pgmap v2810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:43 smithi167 ceph-mon[32751]: pgmap v2810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:43 smithi090 ceph-mon[29180]: pgmap v2810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:38:45.261 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4209 sec 2024-04-26T19:38:46.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4210 sec 2024-04-26T19:38:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:45 smithi119 ceph-mon[32752]: pgmap v2811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:46.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3807.033339 secs 2024-04-26T19:38:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:45 smithi167 ceph-mon[32751]: pgmap v2811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:46.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3807.033339 secs 2024-04-26T19:38:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:45 smithi090 ceph-mon[29180]: pgmap v2811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:38:46.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3807.033339 secs 2024-04-26T19:38:47.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4211 sec 2024-04-26T19:38:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3807.717470 secs 2024-04-26T19:38:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3807.717470 secs 2024-04-26T19:38:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3807.717470 secs 2024-04-26T19:38:48.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4212 sec 2024-04-26T19:38:48.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:47 smithi167 ceph-mon[32751]: pgmap v2812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:48.336 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:47 smithi119 ceph-mon[32752]: pgmap v2812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:47 smithi090 ceph-mon[29180]: pgmap v2812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:49.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4213 sec 2024-04-26T19:38:50.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4214 sec 2024-04-26T19:38:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:49 smithi119 ceph-mon[32752]: pgmap v2813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:50.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:49 smithi167 ceph-mon[32751]: pgmap v2813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:49 smithi090 ceph-mon[29180]: pgmap v2813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:51.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4215 sec 2024-04-26T19:38:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3812.033411 secs 2024-04-26T19:38:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3812.033411 secs 2024-04-26T19:38:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3812.033411 secs 2024-04-26T19:38:52.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4216 sec 2024-04-26T19:38:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:51 smithi119 ceph-mon[32752]: pgmap v2814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:52.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3812.717578 secs 2024-04-26T19:38:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:51 smithi167 ceph-mon[32751]: pgmap v2814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3812.717578 secs 2024-04-26T19:38:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:51 smithi090 ceph-mon[29180]: pgmap v2814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:52.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3812.717578 secs 2024-04-26T19:38:53.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4217 sec 2024-04-26T19:38:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:38:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:38:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:38:54.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:53 smithi090 ceph-mon[29180]: pgmap v2815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:53 smithi119 ceph-mon[32752]: pgmap v2815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:54.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4218 sec 2024-04-26T19:38:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:53 smithi167 ceph-mon[32751]: pgmap v2815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:55.262 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4219 sec 2024-04-26T19:38:56.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4220 sec 2024-04-26T19:38:56.271 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:55 smithi119 ceph-mon[32752]: pgmap v2816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:56.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 3817.033472 secs 2024-04-26T19:38:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:55 smithi167 ceph-mon[32751]: pgmap v2816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:56.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 3817.033472 secs 2024-04-26T19:38:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:55 smithi090 ceph-mon[29180]: pgmap v2816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:38:56.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 3817.033472 secs 2024-04-26T19:38:57.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4221 sec 2024-04-26T19:38:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3817.717688 secs 2024-04-26T19:38:57.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:56 smithi119 ceph-mon[32752]: pgmap v2817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:57.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3817.717688 secs 2024-04-26T19:38:57.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:56 smithi167 ceph-mon[32751]: pgmap v2817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3817.717688 secs 2024-04-26T19:38:57.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:56 smithi090 ceph-mon[29180]: pgmap v2817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:38:58.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4222 sec 2024-04-26T19:38:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.320 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:38:58.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:38:58.352 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: pgmap v2818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 952 B/s rd, 1 op/s 2024-04-26T19:38:58.359 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: pgmap v2819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s rd, 1 op/s 2024-04-26T19:38:58.367 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.375 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:38:58.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:38:58.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: pgmap v2818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 952 B/s rd, 1 op/s 2024-04-26T19:38:58.518 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: pgmap v2819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s rd, 1 op/s 2024-04-26T19:38:58.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.575 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:58.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:38:58.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:38:58.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: pgmap v2818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 952 B/s rd, 1 op/s 2024-04-26T19:38:58.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: pgmap v2819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s rd, 1 op/s 2024-04-26T19:38:58.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:38:59.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4223 sec 2024-04-26T19:39:00.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4224 sec 2024-04-26T19:39:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:00 smithi167 ceph-mon[32751]: pgmap v2820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:39:00.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:00 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3822.033552 secs 2024-04-26T19:39:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:00 smithi090 ceph-mon[29180]: pgmap v2820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:39:00.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:00 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3822.033552 secs 2024-04-26T19:39:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:00 smithi119 ceph-mon[32752]: pgmap v2820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:39:01.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:00 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3822.033552 secs 2024-04-26T19:39:01.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4225 sec 2024-04-26T19:39:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3822.717813 secs 2024-04-26T19:39:01.884 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:39:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:39:01.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:39:01.893 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:39:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:39:01.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:39:01.902 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:39:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:39:01.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:39:01.911 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:39:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:39:01.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:39:01.920 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:39:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:39:01.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:39:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3822.717813 secs 2024-04-26T19:39:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3822.717813 secs 2024-04-26T19:39:02.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4226 sec 2024-04-26T19:39:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:02 smithi090 ceph-mon[29180]: pgmap v2821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:39:02.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:02 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:39:02.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:02 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:39:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:02 smithi119 ceph-mon[32752]: pgmap v2821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:39:03.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:02 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:39:03.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:02 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:39:03.129 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:02 smithi167 ceph-mon[32751]: pgmap v2821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:39:03.139 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:02 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:39:03.148 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:02 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:39:03.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4227 sec 2024-04-26T19:39:04.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4228 sec 2024-04-26T19:39:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:04 smithi090 ceph-mon[29180]: pgmap v2822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s rd, 1 op/s 2024-04-26T19:39:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:04 smithi119 ceph-mon[32752]: pgmap v2822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s rd, 1 op/s 2024-04-26T19:39:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:04 smithi167 ceph-mon[32751]: pgmap v2822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 585 B/s rd, 1 op/s 2024-04-26T19:39:05.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4229 sec 2024-04-26T19:39:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:05 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3827.033637 secs 2024-04-26T19:39:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:05 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3827.033637 secs 2024-04-26T19:39:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:05 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3827.033637 secs 2024-04-26T19:39:06.263 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4230 sec 2024-04-26T19:39:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:06 smithi090 ceph-mon[29180]: pgmap v2823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 234 B/s wr, 2 op/s 2024-04-26T19:39:06.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3827.717871 secs 2024-04-26T19:39:07.006 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:06 smithi119 ceph-mon[32752]: pgmap v2823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 234 B/s wr, 2 op/s 2024-04-26T19:39:07.015 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3827.717871 secs 2024-04-26T19:39:07.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:06 smithi167 ceph-mon[32751]: pgmap v2823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 234 B/s wr, 2 op/s 2024-04-26T19:39:07.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3827.717871 secs 2024-04-26T19:39:07.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4231 sec 2024-04-26T19:39:08.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4232 sec 2024-04-26T19:39:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:08 smithi090 ceph-mon[29180]: pgmap v2824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T19:39:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:08 smithi119 ceph-mon[32752]: pgmap v2824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T19:39:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:08 smithi167 ceph-mon[32751]: pgmap v2824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T19:39:09.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4233 sec 2024-04-26T19:39:10.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4234 sec 2024-04-26T19:39:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:10 smithi090 ceph-mon[29180]: pgmap v2825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:39:10.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:10 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3832.033707 secs 2024-04-26T19:39:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:10 smithi119 ceph-mon[32752]: pgmap v2825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:39:11.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:10 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3832.033707 secs 2024-04-26T19:39:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:10 smithi167 ceph-mon[32751]: pgmap v2825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:39:11.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:10 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3832.033707 secs 2024-04-26T19:39:11.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4235 sec 2024-04-26T19:39:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3832.717971 secs 2024-04-26T19:39:11.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3832.717971 secs 2024-04-26T19:39:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3832.717971 secs 2024-04-26T19:39:12.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4236 sec 2024-04-26T19:39:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:12 smithi090 ceph-mon[29180]: pgmap v2826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:39:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:12 smithi119 ceph-mon[32752]: pgmap v2826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:39:13.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:12 smithi167 ceph-mon[32751]: pgmap v2826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:39:13.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4237 sec 2024-04-26T19:39:14.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4238 sec 2024-04-26T19:39:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:14 smithi090 ceph-mon[29180]: pgmap v2827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:39:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:14 smithi119 ceph-mon[32752]: pgmap v2827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:39:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:14 smithi167 ceph-mon[32751]: pgmap v2827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:39:15.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4239 sec 2024-04-26T19:39:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:15 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3837.033781 secs 2024-04-26T19:39:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:15 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3837.033781 secs 2024-04-26T19:39:16.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:15 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3837.033781 secs 2024-04-26T19:39:16.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4240 sec 2024-04-26T19:39:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:16 smithi090 ceph-mon[29180]: pgmap v2828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:39:16.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3837.718041 secs 2024-04-26T19:39:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:16 smithi119 ceph-mon[32752]: pgmap v2828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:39:17.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3837.718041 secs 2024-04-26T19:39:17.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:16 smithi167 ceph-mon[32751]: pgmap v2828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:39:17.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3837.718041 secs 2024-04-26T19:39:17.264 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4241 sec 2024-04-26T19:39:18.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4242 sec 2024-04-26T19:39:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:18 smithi090 ceph-mon[29180]: pgmap v2829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:18 smithi119 ceph-mon[32752]: pgmap v2829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:18 smithi167 ceph-mon[32751]: pgmap v2829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:19.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4243 sec 2024-04-26T19:39:20.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4244 sec 2024-04-26T19:39:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:20 smithi090 ceph-mon[29180]: pgmap v2830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:20.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:20 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3842.033862 secs 2024-04-26T19:39:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:20 smithi119 ceph-mon[32752]: pgmap v2830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:21.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:20 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3842.033862 secs 2024-04-26T19:39:21.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:20 smithi167 ceph-mon[32751]: pgmap v2830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:21.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:20 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3842.033862 secs 2024-04-26T19:39:21.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4245 sec 2024-04-26T19:39:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3842.718157 secs 2024-04-26T19:39:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3842.718157 secs 2024-04-26T19:39:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3842.718157 secs 2024-04-26T19:39:22.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4246 sec 2024-04-26T19:39:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:22 smithi090 ceph-mon[29180]: pgmap v2831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:22 smithi119 ceph-mon[32752]: pgmap v2831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:22 smithi167 ceph-mon[32751]: pgmap v2831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:23.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4247 sec 2024-04-26T19:39:24.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4248 sec 2024-04-26T19:39:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:24 smithi090 ceph-mon[29180]: pgmap v2832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:24 smithi119 ceph-mon[32752]: pgmap v2832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:24 smithi167 ceph-mon[32751]: pgmap v2832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:25.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4249 sec 2024-04-26T19:39:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:25 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3847.033949 secs 2024-04-26T19:39:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:25 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3847.033949 secs 2024-04-26T19:39:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:25 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3847.033949 secs 2024-04-26T19:39:26.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4250 sec 2024-04-26T19:39:27.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:26 smithi090 ceph-mon[29180]: pgmap v2833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:27.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3847.718266 secs 2024-04-26T19:39:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:26 smithi119 ceph-mon[32752]: pgmap v2833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:27.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3847.718266 secs 2024-04-26T19:39:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:26 smithi167 ceph-mon[32751]: pgmap v2833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:27.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3847.718266 secs 2024-04-26T19:39:27.265 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4251 sec 2024-04-26T19:39:28.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4252 sec 2024-04-26T19:39:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:28 smithi090 ceph-mon[29180]: pgmap v2834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:28 smithi119 ceph-mon[32752]: pgmap v2834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:28 smithi167 ceph-mon[32751]: pgmap v2834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:29.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4253 sec 2024-04-26T19:39:30.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4254 sec 2024-04-26T19:39:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:30 smithi119 ceph-mon[32752]: pgmap v2835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:31.083 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:30 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3852.034029 secs 2024-04-26T19:39:31.091 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:30 smithi167 ceph-mon[32751]: pgmap v2835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:31.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:30 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3852.034029 secs 2024-04-26T19:39:31.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:30 smithi090 ceph-mon[29180]: pgmap v2835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:31.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:30 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3852.034029 secs 2024-04-26T19:39:31.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4255 sec 2024-04-26T19:39:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3852.718390 secs 2024-04-26T19:39:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3852.718390 secs 2024-04-26T19:39:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3852.718390 secs 2024-04-26T19:39:32.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4256 sec 2024-04-26T19:39:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:32 smithi119 ceph-mon[32752]: pgmap v2836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:32 smithi167 ceph-mon[32751]: pgmap v2836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:32 smithi090 ceph-mon[29180]: pgmap v2836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:33.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4257 sec 2024-04-26T19:39:34.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4258 sec 2024-04-26T19:39:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:34 smithi119 ceph-mon[32752]: pgmap v2837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:35.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:34 smithi167 ceph-mon[32751]: pgmap v2837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:34 smithi090 ceph-mon[29180]: pgmap v2837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:35.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4259 sec 2024-04-26T19:39:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:35 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3857.034110 secs 2024-04-26T19:39:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:35 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3857.034110 secs 2024-04-26T19:39:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:35 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3857.034110 secs 2024-04-26T19:39:36.266 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4260 sec 2024-04-26T19:39:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:36 smithi119 ceph-mon[32752]: pgmap v2838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:37.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 3857.718450 secs 2024-04-26T19:39:37.101 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:36 smithi119 ceph-mon[32752]: slow request 2884.410215 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:39:37.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:36 smithi167 ceph-mon[32751]: pgmap v2838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:37.143 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 3857.718450 secs 2024-04-26T19:39:37.152 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:36 smithi167 ceph-mon[32751]: slow request 2884.410215 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:39:37.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:36 smithi090 ceph-mon[29180]: pgmap v2838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:37.171 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 3857.718450 secs 2024-04-26T19:39:37.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:36 smithi090 ceph-mon[29180]: slow request 2884.410215 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:39:37.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4261 sec 2024-04-26T19:39:38.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4262 sec 2024-04-26T19:39:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:38 smithi119 ceph-mon[32752]: pgmap v2839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:38 smithi167 ceph-mon[32751]: pgmap v2839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:38 smithi090 ceph-mon[29180]: pgmap v2839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:39.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4263 sec 2024-04-26T19:39:40.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4264 sec 2024-04-26T19:39:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:40 smithi119 ceph-mon[32752]: pgmap v2840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:41.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:40 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3862.034193 secs 2024-04-26T19:39:41.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:40 smithi167 ceph-mon[32751]: pgmap v2840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:41.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:40 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3862.034193 secs 2024-04-26T19:39:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:40 smithi090 ceph-mon[29180]: pgmap v2840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:41.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:40 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3862.034193 secs 2024-04-26T19:39:41.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4265 sec 2024-04-26T19:39:41.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3862.718568 secs 2024-04-26T19:39:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3862.718568 secs 2024-04-26T19:39:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3862.718568 secs 2024-04-26T19:39:42.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4266 sec 2024-04-26T19:39:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:42 smithi119 ceph-mon[32752]: pgmap v2841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:42 smithi167 ceph-mon[32751]: pgmap v2841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:42 smithi090 ceph-mon[29180]: pgmap v2841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:43.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4267 sec 2024-04-26T19:39:44.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4268 sec 2024-04-26T19:39:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:44 smithi119 ceph-mon[32752]: pgmap v2842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:44 smithi167 ceph-mon[32751]: pgmap v2842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:44 smithi090 ceph-mon[29180]: pgmap v2842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:45.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4269 sec 2024-04-26T19:39:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:45 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3867.034275 secs 2024-04-26T19:39:46.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:45 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3867.034275 secs 2024-04-26T19:39:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:45 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3867.034275 secs 2024-04-26T19:39:46.267 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4270 sec 2024-04-26T19:39:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:46 smithi119 ceph-mon[32752]: pgmap v2843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:47.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3867.718682 secs 2024-04-26T19:39:47.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:46 smithi167 ceph-mon[32751]: pgmap v2843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:47.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3867.718682 secs 2024-04-26T19:39:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:46 smithi090 ceph-mon[29180]: pgmap v2843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:47.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3867.718682 secs 2024-04-26T19:39:47.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4271 sec 2024-04-26T19:39:48.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4272 sec 2024-04-26T19:39:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:48 smithi119 ceph-mon[32752]: pgmap v2844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:49.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:48 smithi167 ceph-mon[32751]: pgmap v2844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:48 smithi090 ceph-mon[29180]: pgmap v2844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:49.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4273 sec 2024-04-26T19:39:50.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4274 sec 2024-04-26T19:39:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:50 smithi119 ceph-mon[32752]: pgmap v2845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:51.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:50 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3872.034356 secs 2024-04-26T19:39:51.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:50 smithi167 ceph-mon[32751]: pgmap v2845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:51.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:50 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3872.034356 secs 2024-04-26T19:39:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:50 smithi090 ceph-mon[29180]: pgmap v2845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:51.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:50 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3872.034356 secs 2024-04-26T19:39:51.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4275 sec 2024-04-26T19:39:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3872.718753 secs 2024-04-26T19:39:52.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3872.718753 secs 2024-04-26T19:39:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3872.718753 secs 2024-04-26T19:39:52.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4276 sec 2024-04-26T19:39:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:52 smithi119 ceph-mon[32752]: pgmap v2846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:53.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:52 smithi167 ceph-mon[32751]: pgmap v2846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:52 smithi090 ceph-mon[29180]: pgmap v2846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:53.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4277 sec 2024-04-26T19:39:54.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4278 sec 2024-04-26T19:39:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:54 smithi119 ceph-mon[32752]: pgmap v2847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:55.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:54 smithi167 ceph-mon[32751]: pgmap v2847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:54 smithi090 ceph-mon[29180]: pgmap v2847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:55.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4279 sec 2024-04-26T19:39:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:55 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3877.034439 secs 2024-04-26T19:39:56.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:55 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3877.034439 secs 2024-04-26T19:39:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:55 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3877.034439 secs 2024-04-26T19:39:56.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4280 sec 2024-04-26T19:39:57.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:56 smithi090 ceph-mon[29180]: pgmap v2848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:57.037 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3877.718887 secs 2024-04-26T19:39:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:56 smithi119 ceph-mon[32752]: pgmap v2848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:57.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3877.718887 secs 2024-04-26T19:39:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:56 smithi167 ceph-mon[32751]: pgmap v2848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:39:57.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3877.718887 secs 2024-04-26T19:39:57.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4281 sec 2024-04-26T19:39:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:39:58.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:39:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:39:58.268 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4282 sec 2024-04-26T19:39:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:39:58 smithi167 ceph-mon[32751]: pgmap v2849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:39:58 smithi119 ceph-mon[32752]: pgmap v2849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:39:58 smithi090 ceph-mon[29180]: pgmap v2849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:39:59.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4283 sec 2024-04-26T19:40:00.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4284 sec 2024-04-26T19:40:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:00 smithi119 ceph-mon[32752]: pgmap v2850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:01.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:00 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3882.034516 secs 2024-04-26T19:40:01.109 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:40:01.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:00 smithi167 ceph-mon[32751]: pgmap v2850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:01.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:00 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3882.034516 secs 2024-04-26T19:40:01.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:40:01.146 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:00 smithi090 ceph-mon[29180]: pgmap v2850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:01.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:00 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3882.034516 secs 2024-04-26T19:40:01.165 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:40:01.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4285 sec 2024-04-26T19:40:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3882.719002 secs 2024-04-26T19:40:02.070 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:40:02.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:40:02.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3882.719002 secs 2024-04-26T19:40:02.182 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.245 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.258 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:40:02.268 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:40:02.277 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.287 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3882.719002 secs 2024-04-26T19:40:02.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:40:02.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:40:02.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:40:02.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4286 sec 2024-04-26T19:40:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:02 smithi119 ceph-mon[32752]: pgmap v2851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:03.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:02 smithi119 ceph-mon[32752]: pgmap v2852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 986 B/s rd, 1 op/s 2024-04-26T19:40:03.071 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:02 smithi119 ceph-mon[32752]: pgmap v2853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:03.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:02 smithi167 ceph-mon[32751]: pgmap v2851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:03.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:02 smithi167 ceph-mon[32751]: pgmap v2852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 986 B/s rd, 1 op/s 2024-04-26T19:40:03.151 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:02 smithi167 ceph-mon[32751]: pgmap v2853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:03.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:02 smithi090 ceph-mon[29180]: pgmap v2851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:03.238 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:02 smithi090 ceph-mon[29180]: pgmap v2852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 986 B/s rd, 1 op/s 2024-04-26T19:40:03.247 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:02 smithi090 ceph-mon[29180]: pgmap v2853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:03.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4287 sec 2024-04-26T19:40:04.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4288 sec 2024-04-26T19:40:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:04 smithi119 ceph-mon[32752]: pgmap v2854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:04 smithi167 ceph-mon[32751]: pgmap v2854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:04 smithi090 ceph-mon[29180]: pgmap v2854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:05.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4289 sec 2024-04-26T19:40:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:05 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3887.034553 secs 2024-04-26T19:40:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:05 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3887.034553 secs 2024-04-26T19:40:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:05 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3887.034553 secs 2024-04-26T19:40:06.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4290 sec 2024-04-26T19:40:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3887.719134 secs 2024-04-26T19:40:07.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:06 smithi119 ceph-mon[32752]: pgmap v2855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:07.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3887.719134 secs 2024-04-26T19:40:07.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:06 smithi167 ceph-mon[32751]: pgmap v2855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3887.719134 secs 2024-04-26T19:40:07.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:06 smithi090 ceph-mon[29180]: pgmap v2855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:07.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4291 sec 2024-04-26T19:40:08.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4292 sec 2024-04-26T19:40:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:08 smithi119 ceph-mon[32752]: pgmap v2856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s rd, 1 op/s 2024-04-26T19:40:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:08 smithi167 ceph-mon[32751]: pgmap v2856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s rd, 1 op/s 2024-04-26T19:40:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:08 smithi090 ceph-mon[29180]: pgmap v2856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 610 B/s rd, 1 op/s 2024-04-26T19:40:09.269 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4293 sec 2024-04-26T19:40:10.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4294 sec 2024-04-26T19:40:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:10 smithi119 ceph-mon[32752]: pgmap v2857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:11.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:10 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3892.034632 secs 2024-04-26T19:40:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:10 smithi167 ceph-mon[32751]: pgmap v2857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:11.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:10 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3892.034632 secs 2024-04-26T19:40:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:10 smithi090 ceph-mon[29180]: pgmap v2857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:11.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:10 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3892.034632 secs 2024-04-26T19:40:11.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4295 sec 2024-04-26T19:40:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3892.719264 secs 2024-04-26T19:40:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3892.719264 secs 2024-04-26T19:40:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3892.719264 secs 2024-04-26T19:40:12.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4296 sec 2024-04-26T19:40:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:12 smithi119 ceph-mon[32752]: pgmap v2858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:40:13.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:12 smithi167 ceph-mon[32751]: pgmap v2858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:40:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:12 smithi090 ceph-mon[29180]: pgmap v2858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:40:13.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4297 sec 2024-04-26T19:40:14.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4298 sec 2024-04-26T19:40:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:14 smithi119 ceph-mon[32752]: pgmap v2859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:15.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:14 smithi167 ceph-mon[32751]: pgmap v2859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:14 smithi090 ceph-mon[29180]: pgmap v2859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:15.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4299 sec 2024-04-26T19:40:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:15 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3897.034716 secs 2024-04-26T19:40:16.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:15 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3897.034716 secs 2024-04-26T19:40:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:15 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3897.034716 secs 2024-04-26T19:40:16.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4300 sec 2024-04-26T19:40:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3897.719316 secs 2024-04-26T19:40:17.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:16 smithi119 ceph-mon[32752]: pgmap v2860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:17.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3897.719316 secs 2024-04-26T19:40:17.084 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:16 smithi167 ceph-mon[32751]: pgmap v2860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3897.719316 secs 2024-04-26T19:40:17.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:16 smithi090 ceph-mon[29180]: pgmap v2860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:17.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4301 sec 2024-04-26T19:40:18.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4302 sec 2024-04-26T19:40:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:18 smithi119 ceph-mon[32752]: pgmap v2861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:19.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:18 smithi167 ceph-mon[32751]: pgmap v2861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:19.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:18 smithi090 ceph-mon[29180]: pgmap v2861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:19.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4303 sec 2024-04-26T19:40:20.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4304 sec 2024-04-26T19:40:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:20 smithi119 ceph-mon[32752]: pgmap v2862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:21.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:20 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3902.034797 secs 2024-04-26T19:40:21.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:20 smithi090 ceph-mon[29180]: pgmap v2862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:21.162 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:20 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3902.034797 secs 2024-04-26T19:40:21.170 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:20 smithi167 ceph-mon[32751]: pgmap v2862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:21.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:20 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3902.034797 secs 2024-04-26T19:40:21.270 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4305 sec 2024-04-26T19:40:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3902.719426 secs 2024-04-26T19:40:22.113 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3902.719426 secs 2024-04-26T19:40:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3902.719426 secs 2024-04-26T19:40:22.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4306 sec 2024-04-26T19:40:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:22 smithi119 ceph-mon[32752]: pgmap v2863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:23.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:22 smithi167 ceph-mon[32751]: pgmap v2863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:23.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:22 smithi090 ceph-mon[29180]: pgmap v2863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:23.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4307 sec 2024-04-26T19:40:24.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4308 sec 2024-04-26T19:40:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:24 smithi119 ceph-mon[32752]: pgmap v2864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:25.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:24 smithi167 ceph-mon[32751]: pgmap v2864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:24 smithi090 ceph-mon[29180]: pgmap v2864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:25.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4309 sec 2024-04-26T19:40:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:25 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3907.034861 secs 2024-04-26T19:40:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:25 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3907.034861 secs 2024-04-26T19:40:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:25 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3907.034861 secs 2024-04-26T19:40:26.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4310 sec 2024-04-26T19:40:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3907.719546 secs 2024-04-26T19:40:27.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:26 smithi119 ceph-mon[32752]: pgmap v2865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3907.719546 secs 2024-04-26T19:40:27.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:26 smithi167 ceph-mon[32751]: pgmap v2865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3907.719546 secs 2024-04-26T19:40:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:26 smithi090 ceph-mon[29180]: pgmap v2865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:27.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4311 sec 2024-04-26T19:40:28.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4312 sec 2024-04-26T19:40:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:28 smithi119 ceph-mon[32752]: pgmap v2866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:28 smithi167 ceph-mon[32751]: pgmap v2866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:28 smithi090 ceph-mon[29180]: pgmap v2866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:29.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4313 sec 2024-04-26T19:40:30.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4314 sec 2024-04-26T19:40:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:30 smithi090 ceph-mon[29180]: pgmap v2867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:30.955 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:30 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3912.034946 secs 2024-04-26T19:40:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:30 smithi119 ceph-mon[32752]: pgmap v2867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:31.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:30 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3912.034946 secs 2024-04-26T19:40:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:30 smithi167 ceph-mon[32751]: pgmap v2867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:31.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:30 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3912.034946 secs 2024-04-26T19:40:31.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4315 sec 2024-04-26T19:40:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3912.719662 secs 2024-04-26T19:40:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3912.719662 secs 2024-04-26T19:40:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3912.719662 secs 2024-04-26T19:40:32.271 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4316 sec 2024-04-26T19:40:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:32 smithi167 ceph-mon[32751]: pgmap v2868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:32 smithi090 ceph-mon[29180]: pgmap v2868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:33.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4317 sec 2024-04-26T19:40:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:32 smithi119 ceph-mon[32752]: pgmap v2868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:34.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4318 sec 2024-04-26T19:40:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:34 smithi167 ceph-mon[32751]: pgmap v2869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:34 smithi090 ceph-mon[29180]: pgmap v2869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:35.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4319 sec 2024-04-26T19:40:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:34 smithi119 ceph-mon[32752]: pgmap v2869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:35 smithi167 ceph-mon[32751]: 4 slow requests, 1 included below; oldest blocked for > 3917.035006 secs 2024-04-26T19:40:36.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:35 smithi167 ceph-mon[32751]: slow request 723.171697 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:40:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:35 smithi090 ceph-mon[29180]: 4 slow requests, 1 included below; oldest blocked for > 3917.035006 secs 2024-04-26T19:40:36.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:35 smithi090 ceph-mon[29180]: slow request 723.171697 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:40:36.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4320 sec 2024-04-26T19:40:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:35 smithi119 ceph-mon[32752]: 4 slow requests, 1 included below; oldest blocked for > 3917.035006 secs 2024-04-26T19:40:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:35 smithi119 ceph-mon[32752]: slow request 723.171697 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:40:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3917.719780 secs 2024-04-26T19:40:37.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:36 smithi167 ceph-mon[32751]: pgmap v2870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3917.719780 secs 2024-04-26T19:40:37.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:36 smithi090 ceph-mon[29180]: pgmap v2870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:37.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4321 sec 2024-04-26T19:40:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3917.719780 secs 2024-04-26T19:40:37.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:36 smithi119 ceph-mon[32752]: pgmap v2870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:38.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4322 sec 2024-04-26T19:40:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:38 smithi090 ceph-mon[29180]: pgmap v2871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:39.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4323 sec 2024-04-26T19:40:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:38 smithi119 ceph-mon[32752]: pgmap v2871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:38 smithi167 ceph-mon[32751]: pgmap v2871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:40.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4324 sec 2024-04-26T19:40:41.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:40 smithi119 ceph-mon[32752]: pgmap v2872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:41.113 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:40 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3922.035065 secs 2024-04-26T19:40:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:40 smithi090 ceph-mon[29180]: pgmap v2872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:41.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:40 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3922.035065 secs 2024-04-26T19:40:41.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4325 sec 2024-04-26T19:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:40 smithi167 ceph-mon[32751]: pgmap v2872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:41.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:40 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3922.035065 secs 2024-04-26T19:40:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3922.719914 secs 2024-04-26T19:40:42.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4326 sec 2024-04-26T19:40:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3922.719914 secs 2024-04-26T19:40:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3922.719914 secs 2024-04-26T19:40:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:42 smithi090 ceph-mon[29180]: pgmap v2873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:43.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4327 sec 2024-04-26T19:40:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:42 smithi119 ceph-mon[32752]: pgmap v2873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:42 smithi167 ceph-mon[32751]: pgmap v2873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:44.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4328 sec 2024-04-26T19:40:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:44 smithi090 ceph-mon[29180]: pgmap v2874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:45.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4329 sec 2024-04-26T19:40:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:44 smithi119 ceph-mon[32752]: pgmap v2874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:44 smithi167 ceph-mon[32751]: pgmap v2874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:46.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:45 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3927.035149 secs 2024-04-26T19:40:46.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4330 sec 2024-04-26T19:40:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:45 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3927.035149 secs 2024-04-26T19:40:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:45 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3927.035149 secs 2024-04-26T19:40:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3927.719957 secs 2024-04-26T19:40:47.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:46 smithi090 ceph-mon[29180]: pgmap v2875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:47.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4331 sec 2024-04-26T19:40:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3927.719957 secs 2024-04-26T19:40:47.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:46 smithi119 ceph-mon[32752]: pgmap v2875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:47.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3927.719957 secs 2024-04-26T19:40:47.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:46 smithi167 ceph-mon[32751]: pgmap v2875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:48.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4332 sec 2024-04-26T19:40:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:48 smithi090 ceph-mon[29180]: pgmap v2876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:49.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4333 sec 2024-04-26T19:40:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:48 smithi119 ceph-mon[32752]: pgmap v2876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:48 smithi167 ceph-mon[32751]: pgmap v2876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:50.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4334 sec 2024-04-26T19:40:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:50 smithi090 ceph-mon[29180]: pgmap v2877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:51.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:50 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3932.035226 secs 2024-04-26T19:40:51.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4335 sec 2024-04-26T19:40:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:50 smithi119 ceph-mon[32752]: pgmap v2877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:51.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:50 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3932.035226 secs 2024-04-26T19:40:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:50 smithi167 ceph-mon[32751]: pgmap v2877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:51.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:50 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3932.035226 secs 2024-04-26T19:40:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3932.720090 secs 2024-04-26T19:40:52.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4336 sec 2024-04-26T19:40:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3932.720090 secs 2024-04-26T19:40:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3932.720090 secs 2024-04-26T19:40:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:52 smithi090 ceph-mon[29180]: pgmap v2878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:53.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4337 sec 2024-04-26T19:40:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:52 smithi119 ceph-mon[32752]: pgmap v2878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:52 smithi167 ceph-mon[32751]: pgmap v2878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:54.273 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4338 sec 2024-04-26T19:40:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:54 smithi090 ceph-mon[29180]: pgmap v2879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:55.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4339 sec 2024-04-26T19:40:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:54 smithi119 ceph-mon[32752]: pgmap v2879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:54 smithi167 ceph-mon[32751]: pgmap v2879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:40:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:55 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3937.035306 secs 2024-04-26T19:40:56.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4340 sec 2024-04-26T19:40:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:55 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3937.035306 secs 2024-04-26T19:40:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:55 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3937.035306 secs 2024-04-26T19:40:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3937.720203 secs 2024-04-26T19:40:57.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:56 smithi090 ceph-mon[29180]: pgmap v2880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:57.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4341 sec 2024-04-26T19:40:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3937.720203 secs 2024-04-26T19:40:57.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:56 smithi119 ceph-mon[32752]: pgmap v2880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:57.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3937.720203 secs 2024-04-26T19:40:57.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:56 smithi167 ceph-mon[32751]: pgmap v2880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:58.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4342 sec 2024-04-26T19:40:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:40:58 smithi090 ceph-mon[29180]: pgmap v2881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:59.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4343 sec 2024-04-26T19:40:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:40:58 smithi119 ceph-mon[32752]: pgmap v2881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:40:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:40:58 smithi167 ceph-mon[32751]: pgmap v2881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:00.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4344 sec 2024-04-26T19:41:01.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:00 smithi167 ceph-mon[32751]: pgmap v2882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:01.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:00 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3942.035342 secs 2024-04-26T19:41:01.146 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:00 smithi090 ceph-mon[29180]: pgmap v2882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:01.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:00 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3942.035342 secs 2024-04-26T19:41:01.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4345 sec 2024-04-26T19:41:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:00 smithi119 ceph-mon[32752]: pgmap v2882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:01.404 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:00 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3942.035342 secs 2024-04-26T19:41:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3942.720300 secs 2024-04-26T19:41:02.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:41:02.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4346 sec 2024-04-26T19:41:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3942.720300 secs 2024-04-26T19:41:02.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:41:02.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3942.720300 secs 2024-04-26T19:41:02.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:41:03.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4347 sec 2024-04-26T19:41:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:02 smithi119 ceph-mon[32752]: pgmap v2883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:41:03.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:41:03.448 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:41:03.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:02 smithi090 ceph-mon[29180]: pgmap v2883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:03.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:41:03.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:41:03.480 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:41:03.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:02 smithi167 ceph-mon[32751]: pgmap v2883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:03.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:41:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:41:03.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:41:04.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4348 sec 2024-04-26T19:41:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:03 smithi119 ceph-mon[32752]: pgmap v2884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 946 B/s rd, 1 op/s 2024-04-26T19:41:04.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:03 smithi119 ceph-mon[32752]: pgmap v2885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T19:41:04.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:03 smithi167 ceph-mon[32751]: pgmap v2884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 946 B/s rd, 1 op/s 2024-04-26T19:41:04.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:03 smithi167 ceph-mon[32751]: pgmap v2885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T19:41:04.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:03 smithi090 ceph-mon[29180]: pgmap v2884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 946 B/s rd, 1 op/s 2024-04-26T19:41:04.394 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:03 smithi090 ceph-mon[29180]: pgmap v2885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T19:41:05.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4349 sec 2024-04-26T19:41:06.274 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4350 sec 2024-04-26T19:41:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:05 smithi119 ceph-mon[32752]: pgmap v2886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:06.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:05 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3947.035424 secs 2024-04-26T19:41:06.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:05 smithi167 ceph-mon[32751]: pgmap v2886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:06.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:05 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3947.035424 secs 2024-04-26T19:41:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:05 smithi090 ceph-mon[29180]: pgmap v2886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:06.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:05 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3947.035424 secs 2024-04-26T19:41:07.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4351 sec 2024-04-26T19:41:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3947.720385 secs 2024-04-26T19:41:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3947.720385 secs 2024-04-26T19:41:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3947.720385 secs 2024-04-26T19:41:08.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4352 sec 2024-04-26T19:41:08.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:07 smithi167 ceph-mon[32751]: pgmap v2887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:08.353 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:07 smithi119 ceph-mon[32752]: pgmap v2887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:07 smithi090 ceph-mon[29180]: pgmap v2887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:09.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4353 sec 2024-04-26T19:41:10.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4354 sec 2024-04-26T19:41:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:09 smithi119 ceph-mon[32752]: pgmap v2888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:09 smithi167 ceph-mon[32751]: pgmap v2888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:09 smithi090 ceph-mon[29180]: pgmap v2888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:11.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:10 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3952.035518 secs 2024-04-26T19:41:11.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4355 sec 2024-04-26T19:41:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:10 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3952.035518 secs 2024-04-26T19:41:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:10 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3952.035518 secs 2024-04-26T19:41:12.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4356 sec 2024-04-26T19:41:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:11 smithi119 ceph-mon[32752]: pgmap v2889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:12.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3952.720418 secs 2024-04-26T19:41:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:11 smithi167 ceph-mon[32751]: pgmap v2889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:12.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3952.720418 secs 2024-04-26T19:41:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:11 smithi090 ceph-mon[29180]: pgmap v2889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:41:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3952.720418 secs 2024-04-26T19:41:13.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4357 sec 2024-04-26T19:41:14.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4358 sec 2024-04-26T19:41:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:13 smithi119 ceph-mon[32752]: pgmap v2890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:41:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:13 smithi167 ceph-mon[32751]: pgmap v2890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:41:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:13 smithi090 ceph-mon[29180]: pgmap v2890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:41:15.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4359 sec 2024-04-26T19:41:16.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4360 sec 2024-04-26T19:41:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:15 smithi119 ceph-mon[32752]: pgmap v2891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:16.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:15 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3957.035603 secs 2024-04-26T19:41:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:15 smithi167 ceph-mon[32751]: pgmap v2891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:16.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:15 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3957.035603 secs 2024-04-26T19:41:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:15 smithi090 ceph-mon[29180]: pgmap v2891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:16.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:15 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3957.035603 secs 2024-04-26T19:41:17.275 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4361 sec 2024-04-26T19:41:17.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3957.720510 secs 2024-04-26T19:41:17.350 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3957.720510 secs 2024-04-26T19:41:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3957.720510 secs 2024-04-26T19:41:18.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4362 sec 2024-04-26T19:41:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:17 smithi119 ceph-mon[32752]: pgmap v2892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:17 smithi167 ceph-mon[32751]: pgmap v2892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:17 smithi090 ceph-mon[29180]: pgmap v2892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:19.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4363 sec 2024-04-26T19:41:20.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4364 sec 2024-04-26T19:41:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:19 smithi119 ceph-mon[32752]: pgmap v2893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:19 smithi167 ceph-mon[32751]: pgmap v2893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:19 smithi090 ceph-mon[29180]: pgmap v2893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:21.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4365 sec 2024-04-26T19:41:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:20 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3962.035687 secs 2024-04-26T19:41:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:20 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3962.035687 secs 2024-04-26T19:41:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:20 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3962.035687 secs 2024-04-26T19:41:22.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4366 sec 2024-04-26T19:41:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:21 smithi119 ceph-mon[32752]: pgmap v2894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:22.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3962.720626 secs 2024-04-26T19:41:22.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:21 smithi167 ceph-mon[32751]: pgmap v2894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:22.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3962.720626 secs 2024-04-26T19:41:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:21 smithi090 ceph-mon[29180]: pgmap v2894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:22.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3962.720626 secs 2024-04-26T19:41:23.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4367 sec 2024-04-26T19:41:24.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4368 sec 2024-04-26T19:41:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:23 smithi119 ceph-mon[32752]: pgmap v2895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:23 smithi167 ceph-mon[32751]: pgmap v2895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:23 smithi090 ceph-mon[29180]: pgmap v2895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:25.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4369 sec 2024-04-26T19:41:26.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4370 sec 2024-04-26T19:41:26.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:25 smithi167 ceph-mon[32751]: pgmap v2896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:26.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:25 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3967.035773 secs 2024-04-26T19:41:26.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:25 smithi119 ceph-mon[32752]: pgmap v2896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:26.359 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:25 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3967.035773 secs 2024-04-26T19:41:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:25 smithi090 ceph-mon[29180]: pgmap v2896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:26.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:25 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3967.035773 secs 2024-04-26T19:41:27.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4371 sec 2024-04-26T19:41:27.285 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3967.720743 secs 2024-04-26T19:41:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3967.720743 secs 2024-04-26T19:41:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3967.720743 secs 2024-04-26T19:41:28.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4372 sec 2024-04-26T19:41:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:27 smithi119 ceph-mon[32752]: pgmap v2897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:27 smithi167 ceph-mon[32751]: pgmap v2897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:27 smithi090 ceph-mon[29180]: pgmap v2897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:41:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:41:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 34 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T19:41:29.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:41:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 34 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T19:41:29.133 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:41:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 35 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:41:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:41:28.994+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 35 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:41:29.276 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4373 sec 2024-04-26T19:41:30.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4374 sec 2024-04-26T19:41:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:29 smithi119 ceph-mon[32752]: pgmap v2898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:29 smithi167 ceph-mon[32751]: pgmap v2898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:29 smithi090 ceph-mon[29180]: pgmap v2898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:30 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3972.035799 secs 2024-04-26T19:41:31.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4375 sec 2024-04-26T19:41:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:30 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3972.035799 secs 2024-04-26T19:41:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:30 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3972.035799 secs 2024-04-26T19:41:32.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4376 sec 2024-04-26T19:41:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:31 smithi119 ceph-mon[32752]: pgmap v2899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:32.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3972.720854 secs 2024-04-26T19:41:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:31 smithi167 ceph-mon[32751]: pgmap v2899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:32.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3972.720854 secs 2024-04-26T19:41:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:31 smithi090 ceph-mon[29180]: pgmap v2899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:32.405 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3972.720854 secs 2024-04-26T19:41:33.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4377 sec 2024-04-26T19:41:34.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4378 sec 2024-04-26T19:41:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:33 smithi119 ceph-mon[32752]: pgmap v2900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:33 smithi167 ceph-mon[32751]: pgmap v2900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:33 smithi090 ceph-mon[29180]: pgmap v2900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:35.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4379 sec 2024-04-26T19:41:36.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4380 sec 2024-04-26T19:41:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:35 smithi119 ceph-mon[32752]: pgmap v2901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:35 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3977.035889 secs 2024-04-26T19:41:36.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:35 smithi167 ceph-mon[32751]: pgmap v2901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:36.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:35 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3977.035889 secs 2024-04-26T19:41:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:35 smithi090 ceph-mon[29180]: pgmap v2901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:36.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:35 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3977.035889 secs 2024-04-26T19:41:37.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4381 sec 2024-04-26T19:41:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3977.720958 secs 2024-04-26T19:41:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3977.720958 secs 2024-04-26T19:41:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3977.720958 secs 2024-04-26T19:41:38.277 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4382 sec 2024-04-26T19:41:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:37 smithi119 ceph-mon[32752]: pgmap v2902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:37 smithi167 ceph-mon[32751]: pgmap v2902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:37 smithi090 ceph-mon[29180]: pgmap v2902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:39.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4383 sec 2024-04-26T19:41:40.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4384 sec 2024-04-26T19:41:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:39 smithi119 ceph-mon[32752]: pgmap v2903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:39 smithi167 ceph-mon[32751]: pgmap v2903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:39 smithi090 ceph-mon[29180]: pgmap v2903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:41.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4385 sec 2024-04-26T19:41:41.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:40 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3982.035971 secs 2024-04-26T19:41:41.344 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:40 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3982.035971 secs 2024-04-26T19:41:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:40 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3982.035971 secs 2024-04-26T19:41:42.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4386 sec 2024-04-26T19:41:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:41 smithi119 ceph-mon[32752]: pgmap v2904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:42.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3982.721077 secs 2024-04-26T19:41:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:41 smithi167 ceph-mon[32751]: pgmap v2904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:42.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3982.721077 secs 2024-04-26T19:41:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:41 smithi090 ceph-mon[29180]: pgmap v2904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:42.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3982.721077 secs 2024-04-26T19:41:43.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4387 sec 2024-04-26T19:41:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:42 smithi119 ceph-mon[32752]: pgmap v2905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:42 smithi167 ceph-mon[32751]: pgmap v2905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:42 smithi090 ceph-mon[29180]: pgmap v2905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:44.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4388 sec 2024-04-26T19:41:45.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4389 sec 2024-04-26T19:41:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:45 smithi119 ceph-mon[32752]: pgmap v2906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:45.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:45 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3987.036061 secs 2024-04-26T19:41:45.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:45 smithi167 ceph-mon[32751]: pgmap v2906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:45.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:45 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3987.036061 secs 2024-04-26T19:41:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:45 smithi090 ceph-mon[29180]: pgmap v2906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:45.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:45 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3987.036061 secs 2024-04-26T19:41:46.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4390 sec 2024-04-26T19:41:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3987.721194 secs 2024-04-26T19:41:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3987.721194 secs 2024-04-26T19:41:46.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3987.721194 secs 2024-04-26T19:41:47.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4391 sec 2024-04-26T19:41:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:47 smithi119 ceph-mon[32752]: pgmap v2907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:47 smithi167 ceph-mon[32751]: pgmap v2907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:47 smithi090 ceph-mon[29180]: pgmap v2907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:48.278 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4392 sec 2024-04-26T19:41:49.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4393 sec 2024-04-26T19:41:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:49 smithi119 ceph-mon[32752]: pgmap v2908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:49 smithi167 ceph-mon[32751]: pgmap v2908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:49 smithi090 ceph-mon[29180]: pgmap v2908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:50.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4394 sec 2024-04-26T19:41:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:50 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3992.036143 secs 2024-04-26T19:41:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:50 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3992.036143 secs 2024-04-26T19:41:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:50 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3992.036143 secs 2024-04-26T19:41:51.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4395 sec 2024-04-26T19:41:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:51 smithi119 ceph-mon[32752]: pgmap v2909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:51.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3992.721256 secs 2024-04-26T19:41:51.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:51 smithi167 ceph-mon[32751]: pgmap v2909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:51.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3992.721256 secs 2024-04-26T19:41:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:51 smithi090 ceph-mon[29180]: pgmap v2909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:51.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3992.721256 secs 2024-04-26T19:41:52.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4396 sec 2024-04-26T19:41:53.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4397 sec 2024-04-26T19:41:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:53 smithi119 ceph-mon[32752]: pgmap v2910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:53 smithi167 ceph-mon[32751]: pgmap v2910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:53 smithi090 ceph-mon[29180]: pgmap v2910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:54.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4398 sec 2024-04-26T19:41:55.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4399 sec 2024-04-26T19:41:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:55 smithi119 ceph-mon[32752]: pgmap v2911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:55.817 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:55 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 3997.036202 secs 2024-04-26T19:41:55.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:55 smithi167 ceph-mon[32751]: pgmap v2911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:55.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:55 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 3997.036202 secs 2024-04-26T19:41:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:55 smithi090 ceph-mon[29180]: pgmap v2911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:55 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 3997.036202 secs 2024-04-26T19:41:56.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4400 sec 2024-04-26T19:41:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 3997.721365 secs 2024-04-26T19:41:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 3997.721365 secs 2024-04-26T19:41:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 3997.721365 secs 2024-04-26T19:41:57.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4401 sec 2024-04-26T19:41:57.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:57 smithi090 ceph-mon[29180]: pgmap v2912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:57 smithi119 ceph-mon[32752]: pgmap v2912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:57.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:57 smithi167 ceph-mon[32751]: pgmap v2912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:41:58.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4402 sec 2024-04-26T19:41:59.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4403 sec 2024-04-26T19:41:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:41:59 smithi119 ceph-mon[32752]: pgmap v2913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:41:59 smithi167 ceph-mon[32751]: pgmap v2913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:41:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:41:59 smithi090 ceph-mon[29180]: pgmap v2913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:00.279 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4404 sec 2024-04-26T19:42:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:00 smithi119 ceph-mon[32752]: 10 slow requests, 1 included below; oldest blocked for > 4002.036284 secs 2024-04-26T19:42:00.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:00 smithi119 ceph-mon[32752]: slow request 184.081916 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:42:00.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:00 smithi167 ceph-mon[32751]: 10 slow requests, 1 included below; oldest blocked for > 4002.036284 secs 2024-04-26T19:42:00.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:00 smithi167 ceph-mon[32751]: slow request 184.081916 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:42:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:00 smithi090 ceph-mon[29180]: 10 slow requests, 1 included below; oldest blocked for > 4002.036284 secs 2024-04-26T19:42:00.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:00 smithi090 ceph-mon[29180]: slow request 184.081916 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:42:01.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4405 sec 2024-04-26T19:42:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:01 smithi119 ceph-mon[32752]: pgmap v2914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:01.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4002.721472 secs 2024-04-26T19:42:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:01 smithi167 ceph-mon[32751]: pgmap v2914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:01.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4002.721472 secs 2024-04-26T19:42:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:01 smithi090 ceph-mon[29180]: pgmap v2914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:01.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4002.721472 secs 2024-04-26T19:42:02.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4406 sec 2024-04-26T19:42:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:42:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:42:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:42:03.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4407 sec 2024-04-26T19:42:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:03 smithi119 ceph-mon[32752]: pgmap v2915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:03.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:42:03.819 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:03 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:42:03.830 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:42:03.838 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:42:03.845 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:42:03.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:03 smithi167 ceph-mon[32751]: pgmap v2915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:03.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:42:03.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:03 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:42:03.985 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:42:03.995 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:42:04.002 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:42:04.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:03 smithi090 ceph-mon[29180]: pgmap v2915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:04.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:42:04.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:03 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:42:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:42:04.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:42:04.229 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:42:04.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4408 sec 2024-04-26T19:42:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:04 smithi119 ceph-mon[32752]: pgmap v2916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:42:04.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:04 smithi119 ceph-mon[32752]: pgmap v2917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:42:04.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:04 smithi167 ceph-mon[32751]: pgmap v2916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:42:04.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:04 smithi167 ceph-mon[32751]: pgmap v2917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:42:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:04 smithi090 ceph-mon[29180]: pgmap v2916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:42:04.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:04 smithi090 ceph-mon[29180]: pgmap v2917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:42:05.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4409 sec 2024-04-26T19:42:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4007.036365 secs 2024-04-26T19:42:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4007.036365 secs 2024-04-26T19:42:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4007.036365 secs 2024-04-26T19:42:06.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4410 sec 2024-04-26T19:42:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:06 smithi119 ceph-mon[32752]: pgmap v2918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:06.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4007.721562 secs 2024-04-26T19:42:06.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:06 smithi167 ceph-mon[32751]: pgmap v2918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:06.832 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4007.721562 secs 2024-04-26T19:42:06.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:06 smithi090 ceph-mon[29180]: pgmap v2918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:06.906 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4007.721562 secs 2024-04-26T19:42:07.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4411 sec 2024-04-26T19:42:08.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4412 sec 2024-04-26T19:42:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:08 smithi119 ceph-mon[32752]: pgmap v2919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:42:08.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:08 smithi090 ceph-mon[29180]: pgmap v2919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:42:08.896 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:08 smithi167 ceph-mon[32751]: pgmap v2919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:42:09.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4413 sec 2024-04-26T19:42:10.280 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4414 sec 2024-04-26T19:42:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:10 smithi119 ceph-mon[32752]: pgmap v2920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4012.036445 secs 2024-04-26T19:42:10.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:10 smithi167 ceph-mon[32751]: pgmap v2920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:10.841 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4012.036445 secs 2024-04-26T19:42:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:10 smithi090 ceph-mon[29180]: pgmap v2920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:10.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4012.036445 secs 2024-04-26T19:42:11.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4415 sec 2024-04-26T19:42:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4012.721686 secs 2024-04-26T19:42:11.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4012.721686 secs 2024-04-26T19:42:11.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4012.721686 secs 2024-04-26T19:42:12.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4416 sec 2024-04-26T19:42:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:12 smithi119 ceph-mon[32752]: pgmap v2921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:12 smithi167 ceph-mon[32751]: pgmap v2921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:12 smithi090 ceph-mon[29180]: pgmap v2921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:42:13.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4417 sec 2024-04-26T19:42:14.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4418 sec 2024-04-26T19:42:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:14 smithi119 ceph-mon[32752]: pgmap v2922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:42:14.859 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:14 smithi167 ceph-mon[32751]: pgmap v2922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:42:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:14 smithi090 ceph-mon[29180]: pgmap v2922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:42:15.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4419 sec 2024-04-26T19:42:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4017.036527 secs 2024-04-26T19:42:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4017.036527 secs 2024-04-26T19:42:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4017.036527 secs 2024-04-26T19:42:16.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4420 sec 2024-04-26T19:42:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:16 smithi119 ceph-mon[32752]: pgmap v2923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:16.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4017.721799 secs 2024-04-26T19:42:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:16 smithi167 ceph-mon[32751]: pgmap v2923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:16.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4017.721799 secs 2024-04-26T19:42:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:16 smithi090 ceph-mon[29180]: pgmap v2923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:16.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4017.721799 secs 2024-04-26T19:42:17.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4421 sec 2024-04-26T19:42:18.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4422 sec 2024-04-26T19:42:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:18 smithi119 ceph-mon[32752]: pgmap v2924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:18 smithi167 ceph-mon[32751]: pgmap v2924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:18 smithi090 ceph-mon[29180]: pgmap v2924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:19.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4423 sec 2024-04-26T19:42:20.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4424 sec 2024-04-26T19:42:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:20 smithi119 ceph-mon[32752]: pgmap v2925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:20.869 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4022.036609 secs 2024-04-26T19:42:20.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:20 smithi090 ceph-mon[29180]: pgmap v2925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:20.889 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4022.036609 secs 2024-04-26T19:42:20.899 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:20 smithi167 ceph-mon[32751]: pgmap v2925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:20.908 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4022.036609 secs 2024-04-26T19:42:21.281 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4425 sec 2024-04-26T19:42:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4022.721879 secs 2024-04-26T19:42:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4022.721879 secs 2024-04-26T19:42:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4022.721879 secs 2024-04-26T19:42:22.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4426 sec 2024-04-26T19:42:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:22 smithi167 ceph-mon[32751]: pgmap v2926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:22 smithi090 ceph-mon[29180]: pgmap v2926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:22 smithi119 ceph-mon[32752]: pgmap v2926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:23.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4427 sec 2024-04-26T19:42:24.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4428 sec 2024-04-26T19:42:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:24 smithi167 ceph-mon[32751]: pgmap v2927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:24 smithi090 ceph-mon[29180]: pgmap v2927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:24 smithi119 ceph-mon[32752]: pgmap v2927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:25.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4429 sec 2024-04-26T19:42:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4027.036695 secs 2024-04-26T19:42:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4027.036695 secs 2024-04-26T19:42:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4027.036695 secs 2024-04-26T19:42:26.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4430 sec 2024-04-26T19:42:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:26 smithi167 ceph-mon[32751]: pgmap v2928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:26.925 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4027.722014 secs 2024-04-26T19:42:26.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:26 smithi090 ceph-mon[29180]: pgmap v2928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:26.943 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4027.722014 secs 2024-04-26T19:42:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:26 smithi119 ceph-mon[32752]: pgmap v2928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:27.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4027.722014 secs 2024-04-26T19:42:27.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4431 sec 2024-04-26T19:42:28.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4432 sec 2024-04-26T19:42:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:28 smithi090 ceph-mon[29180]: pgmap v2929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:28 smithi119 ceph-mon[32752]: pgmap v2929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:28 smithi167 ceph-mon[32751]: pgmap v2929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:29.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4433 sec 2024-04-26T19:42:30.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4434 sec 2024-04-26T19:42:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:30 smithi090 ceph-mon[29180]: pgmap v2930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:30.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4032.036777 secs 2024-04-26T19:42:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:30 smithi119 ceph-mon[32752]: pgmap v2930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:31.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4032.036777 secs 2024-04-26T19:42:31.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:30 smithi167 ceph-mon[32751]: pgmap v2930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:31.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4032.036777 secs 2024-04-26T19:42:31.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4435 sec 2024-04-26T19:42:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4032.722095 secs 2024-04-26T19:42:31.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4032.722095 secs 2024-04-26T19:42:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4032.722095 secs 2024-04-26T19:42:32.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4436 sec 2024-04-26T19:42:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:32 smithi090 ceph-mon[29180]: pgmap v2931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:32 smithi119 ceph-mon[32752]: pgmap v2931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:32 smithi167 ceph-mon[32751]: pgmap v2931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:33.282 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4437 sec 2024-04-26T19:42:34.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4438 sec 2024-04-26T19:42:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:34 smithi090 ceph-mon[29180]: pgmap v2932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:34 smithi119 ceph-mon[32752]: pgmap v2932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:34 smithi167 ceph-mon[32751]: pgmap v2932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:35.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4439 sec 2024-04-26T19:42:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4037.036841 secs 2024-04-26T19:42:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4037.036841 secs 2024-04-26T19:42:36.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4037.036841 secs 2024-04-26T19:42:36.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4440 sec 2024-04-26T19:42:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:36 smithi090 ceph-mon[29180]: pgmap v2933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:36.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4037.722204 secs 2024-04-26T19:42:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:36 smithi119 ceph-mon[32752]: pgmap v2933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:37.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4037.722204 secs 2024-04-26T19:42:37.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:36 smithi167 ceph-mon[32751]: pgmap v2933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:37.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4037.722204 secs 2024-04-26T19:42:37.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4441 sec 2024-04-26T19:42:38.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4442 sec 2024-04-26T19:42:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:38 smithi090 ceph-mon[29180]: pgmap v2934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:38 smithi119 ceph-mon[32752]: pgmap v2934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:38 smithi167 ceph-mon[32751]: pgmap v2934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:42:39.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4443 sec 2024-04-26T19:42:40.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4444 sec 2024-04-26T19:42:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:40 smithi090 ceph-mon[29180]: pgmap v2935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:40.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4042.036901 secs 2024-04-26T19:42:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:40 smithi119 ceph-mon[32752]: pgmap v2935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:41.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4042.036901 secs 2024-04-26T19:42:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:40 smithi167 ceph-mon[32751]: pgmap v2935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:41.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4042.036901 secs 2024-04-26T19:42:41.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4445 sec 2024-04-26T19:42:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4042.722322 secs 2024-04-26T19:42:41.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4042.722322 secs 2024-04-26T19:42:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4042.722322 secs 2024-04-26T19:42:42.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4446 sec 2024-04-26T19:42:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:42 smithi090 ceph-mon[29180]: pgmap v2936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:42 smithi119 ceph-mon[32752]: pgmap v2936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:42 smithi167 ceph-mon[32751]: pgmap v2936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:43.283 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4447 sec 2024-04-26T19:42:44.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4448 sec 2024-04-26T19:42:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:44 smithi090 ceph-mon[29180]: pgmap v2937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:44 smithi119 ceph-mon[32752]: pgmap v2937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:45.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:44 smithi167 ceph-mon[32751]: pgmap v2937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:45.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4449 sec 2024-04-26T19:42:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4047.036990 secs 2024-04-26T19:42:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4047.036990 secs 2024-04-26T19:42:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4047.036990 secs 2024-04-26T19:42:46.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4450 sec 2024-04-26T19:42:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:46 smithi090 ceph-mon[29180]: pgmap v2938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:46.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4047.722442 secs 2024-04-26T19:42:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:46 smithi119 ceph-mon[32752]: pgmap v2938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:47.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4047.722442 secs 2024-04-26T19:42:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:46 smithi167 ceph-mon[32751]: pgmap v2938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:47.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4047.722442 secs 2024-04-26T19:42:47.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4451 sec 2024-04-26T19:42:48.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4452 sec 2024-04-26T19:42:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:48 smithi090 ceph-mon[29180]: pgmap v2939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:48 smithi119 ceph-mon[32752]: pgmap v2939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:48 smithi167 ceph-mon[32751]: pgmap v2939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:49.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4453 sec 2024-04-26T19:42:50.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4454 sec 2024-04-26T19:42:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:50 smithi090 ceph-mon[29180]: pgmap v2940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:50.962 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4052.037049 secs 2024-04-26T19:42:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:50 smithi119 ceph-mon[32752]: pgmap v2940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:51.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4052.037049 secs 2024-04-26T19:42:51.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:50 smithi167 ceph-mon[32751]: pgmap v2940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:51.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4052.037049 secs 2024-04-26T19:42:51.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4455 sec 2024-04-26T19:42:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4052.722560 secs 2024-04-26T19:42:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4052.722560 secs 2024-04-26T19:42:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4052.722560 secs 2024-04-26T19:42:52.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4456 sec 2024-04-26T19:42:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:52 smithi090 ceph-mon[29180]: pgmap v2941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:52 smithi119 ceph-mon[32752]: pgmap v2941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:52 smithi167 ceph-mon[32751]: pgmap v2941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:53.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4457 sec 2024-04-26T19:42:54.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4458 sec 2024-04-26T19:42:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:54 smithi119 ceph-mon[32752]: pgmap v2942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:54 smithi167 ceph-mon[32751]: pgmap v2942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:54 smithi090 ceph-mon[29180]: pgmap v2942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:55.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4459 sec 2024-04-26T19:42:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4057.037110 secs 2024-04-26T19:42:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4057.037110 secs 2024-04-26T19:42:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4057.037110 secs 2024-04-26T19:42:56.284 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4460 sec 2024-04-26T19:42:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:56 smithi119 ceph-mon[32752]: pgmap v2943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:57.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4057.722651 secs 2024-04-26T19:42:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:56 smithi167 ceph-mon[32751]: pgmap v2943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:57.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4057.722651 secs 2024-04-26T19:42:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:56 smithi090 ceph-mon[29180]: pgmap v2943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:42:57.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4057.722651 secs 2024-04-26T19:42:57.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4461 sec 2024-04-26T19:42:58.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4462 sec 2024-04-26T19:42:58.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:42:58 smithi090 ceph-mon[29180]: pgmap v2944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:42:58 smithi119 ceph-mon[32752]: pgmap v2944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:42:58 smithi167 ceph-mon[32751]: pgmap v2944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:42:59.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4463 sec 2024-04-26T19:43:00.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4464 sec 2024-04-26T19:43:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:00 smithi119 ceph-mon[32752]: pgmap v2945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:01.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:00 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4062.037190 secs 2024-04-26T19:43:01.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:00 smithi167 ceph-mon[32751]: pgmap v2945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:01.096 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:00 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4062.037190 secs 2024-04-26T19:43:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:00 smithi090 ceph-mon[29180]: pgmap v2945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:01.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:00 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4062.037190 secs 2024-04-26T19:43:01.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4465 sec 2024-04-26T19:43:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4062.722786 secs 2024-04-26T19:43:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4062.722786 secs 2024-04-26T19:43:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4062.722786 secs 2024-04-26T19:43:02.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4466 sec 2024-04-26T19:43:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:02 smithi119 ceph-mon[32752]: pgmap v2946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:03.081 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:02 smithi167 ceph-mon[32751]: pgmap v2946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:02 smithi090 ceph-mon[29180]: pgmap v2946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:03.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4467 sec 2024-04-26T19:43:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:43:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:43:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:43:04.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4468 sec 2024-04-26T19:43:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:04 smithi119 ceph-mon[32752]: pgmap v2947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:05.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:43:05.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:43:05.082 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:04 smithi119 ceph-mon[32752]: pgmap v2948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:43:05.091 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:04 smithi119 ceph-mon[32752]: pgmap v2949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:05.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:43:05.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:04 smithi167 ceph-mon[32751]: pgmap v2947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:05.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:43:05.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:43:05.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:04 smithi167 ceph-mon[32751]: pgmap v2948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:43:05.206 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:04 smithi167 ceph-mon[32751]: pgmap v2949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:43:05.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:04 smithi090 ceph-mon[29180]: pgmap v2947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:05.262 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:43:05.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:43:05.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:04 smithi090 ceph-mon[29180]: pgmap v2948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:43:05.394 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:04 smithi090 ceph-mon[29180]: pgmap v2949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:05.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:43:05.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4469 sec 2024-04-26T19:43:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4067.037274 secs 2024-04-26T19:43:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4067.037274 secs 2024-04-26T19:43:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4067.037274 secs 2024-04-26T19:43:06.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4470 sec 2024-04-26T19:43:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4067.722905 secs 2024-04-26T19:43:07.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:06 smithi119 ceph-mon[32752]: pgmap v2950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4067.722905 secs 2024-04-26T19:43:07.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:06 smithi167 ceph-mon[32751]: pgmap v2950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:07.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4067.722905 secs 2024-04-26T19:43:07.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:06 smithi090 ceph-mon[29180]: pgmap v2950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:07.285 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4471 sec 2024-04-26T19:43:08.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4472 sec 2024-04-26T19:43:08.318 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:08 smithi119 ceph-mon[32752]: pgmap v2951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:43:08.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:08 smithi167 ceph-mon[32751]: pgmap v2951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:43:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:08 smithi090 ceph-mon[29180]: pgmap v2951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:43:09.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4473 sec 2024-04-26T19:43:10.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4474 sec 2024-04-26T19:43:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:10 smithi119 ceph-mon[32752]: pgmap v2952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:10 smithi167 ceph-mon[32751]: pgmap v2952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:10.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:10 smithi090 ceph-mon[29180]: pgmap v2952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:11.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4475 sec 2024-04-26T19:43:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4072.037356 secs 2024-04-26T19:43:11.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4072.723022 secs 2024-04-26T19:43:11.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4072.037356 secs 2024-04-26T19:43:11.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4072.723022 secs 2024-04-26T19:43:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4072.037356 secs 2024-04-26T19:43:11.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4072.723022 secs 2024-04-26T19:43:12.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4476 sec 2024-04-26T19:43:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:12 smithi119 ceph-mon[32752]: pgmap v2953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:12 smithi167 ceph-mon[32751]: pgmap v2953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:12 smithi090 ceph-mon[29180]: pgmap v2953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:43:13.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4477 sec 2024-04-26T19:43:14.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4478 sec 2024-04-26T19:43:14.398 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:14 smithi167 ceph-mon[32751]: pgmap v2954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:43:14.434 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:14 smithi119 ceph-mon[32752]: pgmap v2954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:43:14.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:14 smithi090 ceph-mon[29180]: pgmap v2954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:43:15.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4479 sec 2024-04-26T19:43:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4077.037433 secs 2024-04-26T19:43:15.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4077.037433 secs 2024-04-26T19:43:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4077.037433 secs 2024-04-26T19:43:16.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4480 sec 2024-04-26T19:43:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4077.723145 secs 2024-04-26T19:43:16.898 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:16 smithi119 ceph-mon[32752]: pgmap v2955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:16.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4077.723145 secs 2024-04-26T19:43:16.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:16 smithi090 ceph-mon[29180]: pgmap v2955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:16.923 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4077.723145 secs 2024-04-26T19:43:16.998 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:16 smithi167 ceph-mon[32751]: pgmap v2955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:17.286 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4481 sec 2024-04-26T19:43:18.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4482 sec 2024-04-26T19:43:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:18 smithi119 ceph-mon[32752]: pgmap v2956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:18 smithi167 ceph-mon[32751]: pgmap v2956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:18 smithi090 ceph-mon[29180]: pgmap v2956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:19.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4483 sec 2024-04-26T19:43:20.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4484 sec 2024-04-26T19:43:20.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:20 smithi167 ceph-mon[32751]: pgmap v2957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:20.362 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:20 smithi119 ceph-mon[32752]: pgmap v2957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:20 smithi090 ceph-mon[29180]: pgmap v2957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:21.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4485 sec 2024-04-26T19:43:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:21 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4082.037517 secs 2024-04-26T19:43:21.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4082.723256 secs 2024-04-26T19:43:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:21 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4082.037517 secs 2024-04-26T19:43:21.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4082.723256 secs 2024-04-26T19:43:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:21 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4082.037517 secs 2024-04-26T19:43:21.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4082.723256 secs 2024-04-26T19:43:22.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4486 sec 2024-04-26T19:43:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:22 smithi119 ceph-mon[32752]: pgmap v2958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:22 smithi167 ceph-mon[32751]: pgmap v2958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:22 smithi090 ceph-mon[29180]: pgmap v2958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:23.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4487 sec 2024-04-26T19:43:24.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4488 sec 2024-04-26T19:43:24.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:24 smithi167 ceph-mon[32751]: pgmap v2959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:24.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:24 smithi119 ceph-mon[32752]: pgmap v2959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:24 smithi090 ceph-mon[29180]: pgmap v2959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:25.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4489 sec 2024-04-26T19:43:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4087.037601 secs 2024-04-26T19:43:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4087.037601 secs 2024-04-26T19:43:25.945 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4087.037601 secs 2024-04-26T19:43:26.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4490 sec 2024-04-26T19:43:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4087.723373 secs 2024-04-26T19:43:26.833 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:26 smithi119 ceph-mon[32752]: pgmap v2960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:26.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4087.723373 secs 2024-04-26T19:43:26.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:26 smithi167 ceph-mon[32751]: pgmap v2960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4087.723373 secs 2024-04-26T19:43:26.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:26 smithi090 ceph-mon[29180]: pgmap v2960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:27.287 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4491 sec 2024-04-26T19:43:28.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4492 sec 2024-04-26T19:43:28.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:28 smithi119 ceph-mon[32752]: pgmap v2961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:28.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:28 smithi167 ceph-mon[32751]: pgmap v2961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:28 smithi090 ceph-mon[29180]: pgmap v2961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:29.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4493 sec 2024-04-26T19:43:30.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4494 sec 2024-04-26T19:43:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:30 smithi119 ceph-mon[32752]: pgmap v2962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:30 smithi167 ceph-mon[32751]: pgmap v2962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:30 smithi090 ceph-mon[29180]: pgmap v2962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:31.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4495 sec 2024-04-26T19:43:31.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:31 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4092.037679 secs 2024-04-26T19:43:31.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4092.723484 secs 2024-04-26T19:43:31.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:31 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4092.037679 secs 2024-04-26T19:43:31.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4092.723484 secs 2024-04-26T19:43:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:31 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4092.037679 secs 2024-04-26T19:43:31.424 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4092.723484 secs 2024-04-26T19:43:32.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4496 sec 2024-04-26T19:43:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:32 smithi119 ceph-mon[32752]: pgmap v2963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:32 smithi167 ceph-mon[32751]: pgmap v2963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:32 smithi090 ceph-mon[29180]: pgmap v2963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:33.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4497 sec 2024-04-26T19:43:34.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4498 sec 2024-04-26T19:43:34.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:34 smithi167 ceph-mon[32751]: pgmap v2964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:34.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:34 smithi119 ceph-mon[32752]: pgmap v2964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:34 smithi090 ceph-mon[29180]: pgmap v2964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:35.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4499 sec 2024-04-26T19:43:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4097.037763 secs 2024-04-26T19:43:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4097.037763 secs 2024-04-26T19:43:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4097.037763 secs 2024-04-26T19:43:36.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4500 sec 2024-04-26T19:43:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4097.723592 secs 2024-04-26T19:43:36.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:36 smithi119 ceph-mon[32752]: pgmap v2965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4097.723592 secs 2024-04-26T19:43:36.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:36 smithi167 ceph-mon[32751]: pgmap v2965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4097.723592 secs 2024-04-26T19:43:36.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:36 smithi090 ceph-mon[29180]: pgmap v2965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:37.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4501 sec 2024-04-26T19:43:38.288 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4502 sec 2024-04-26T19:43:38.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:38 smithi167 ceph-mon[32751]: pgmap v2966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:38.354 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:38 smithi119 ceph-mon[32752]: pgmap v2966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:38 smithi090 ceph-mon[29180]: pgmap v2966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:39.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4503 sec 2024-04-26T19:43:40.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4504 sec 2024-04-26T19:43:40.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:40 smithi167 ceph-mon[32751]: pgmap v2967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:40.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:40 smithi119 ceph-mon[32752]: pgmap v2967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:40 smithi090 ceph-mon[29180]: pgmap v2967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:41.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4505 sec 2024-04-26T19:43:41.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4102.037867 secs 2024-04-26T19:43:41.319 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4102.723715 secs 2024-04-26T19:43:41.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4102.037867 secs 2024-04-26T19:43:41.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4102.723715 secs 2024-04-26T19:43:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4102.037867 secs 2024-04-26T19:43:41.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4102.723715 secs 2024-04-26T19:43:42.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4506 sec 2024-04-26T19:43:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:42 smithi119 ceph-mon[32752]: pgmap v2968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:42 smithi167 ceph-mon[32751]: pgmap v2968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:42 smithi090 ceph-mon[29180]: pgmap v2968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:43.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4507 sec 2024-04-26T19:43:44.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4508 sec 2024-04-26T19:43:44.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:44 smithi119 ceph-mon[32752]: pgmap v2969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:44 smithi167 ceph-mon[32751]: pgmap v2969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:44 smithi090 ceph-mon[29180]: pgmap v2969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:45.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4509 sec 2024-04-26T19:43:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4107.037930 secs 2024-04-26T19:43:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4107.037930 secs 2024-04-26T19:43:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4107.037930 secs 2024-04-26T19:43:46.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4510 sec 2024-04-26T19:43:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4107.723832 secs 2024-04-26T19:43:46.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:46 smithi119 ceph-mon[32752]: pgmap v2970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4107.723832 secs 2024-04-26T19:43:46.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:46 smithi167 ceph-mon[32751]: pgmap v2970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4107.723832 secs 2024-04-26T19:43:46.961 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:46 smithi090 ceph-mon[29180]: pgmap v2970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:47.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4511 sec 2024-04-26T19:43:48.289 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4512 sec 2024-04-26T19:43:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:48 smithi119 ceph-mon[32752]: pgmap v2971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:48 smithi167 ceph-mon[32751]: pgmap v2971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:48 smithi090 ceph-mon[29180]: pgmap v2971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:49.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4513 sec 2024-04-26T19:43:50.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4514 sec 2024-04-26T19:43:50.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:50 smithi167 ceph-mon[32751]: pgmap v2972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:50.339 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:50 smithi119 ceph-mon[32752]: pgmap v2972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:50 smithi090 ceph-mon[29180]: pgmap v2972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:51.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4515 sec 2024-04-26T19:43:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4112.038016 secs 2024-04-26T19:43:51.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4112.723955 secs 2024-04-26T19:43:51.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4112.038016 secs 2024-04-26T19:43:51.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4112.723955 secs 2024-04-26T19:43:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4112.038016 secs 2024-04-26T19:43:51.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4112.723955 secs 2024-04-26T19:43:52.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4516 sec 2024-04-26T19:43:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:52 smithi119 ceph-mon[32752]: pgmap v2973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:52 smithi167 ceph-mon[32751]: pgmap v2973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:52.389 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:52 smithi090 ceph-mon[29180]: pgmap v2973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:53.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4517 sec 2024-04-26T19:43:54.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4518 sec 2024-04-26T19:43:54.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:54 smithi119 ceph-mon[32752]: pgmap v2974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:54.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:54 smithi167 ceph-mon[32751]: pgmap v2974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:54 smithi090 ceph-mon[29180]: pgmap v2974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:43:55.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4519 sec 2024-04-26T19:43:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4117.038102 secs 2024-04-26T19:43:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4117.038102 secs 2024-04-26T19:43:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4117.038102 secs 2024-04-26T19:43:56.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4520 sec 2024-04-26T19:43:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4117.724046 secs 2024-04-26T19:43:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:56 smithi119 ceph-mon[32752]: pgmap v2975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4117.724046 secs 2024-04-26T19:43:56.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:56 smithi167 ceph-mon[32751]: pgmap v2975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4117.724046 secs 2024-04-26T19:43:56.905 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:56 smithi090 ceph-mon[29180]: pgmap v2975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:57.290 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4521 sec 2024-04-26T19:43:58.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4522 sec 2024-04-26T19:43:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:43:58 smithi119 ceph-mon[32752]: pgmap v2976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:43:58 smithi167 ceph-mon[32751]: pgmap v2976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:43:58 smithi090 ceph-mon[29180]: pgmap v2976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:43:59.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4523 sec 2024-04-26T19:44:00.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4524 sec 2024-04-26T19:44:00.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:00 smithi167 ceph-mon[32751]: pgmap v2977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:00.415 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:00 smithi119 ceph-mon[32752]: pgmap v2977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:00.424 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:00 smithi090 ceph-mon[29180]: pgmap v2977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:01.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4525 sec 2024-04-26T19:44:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4122.038180 secs 2024-04-26T19:44:01.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4122.724181 secs 2024-04-26T19:44:01.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4122.038180 secs 2024-04-26T19:44:01.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4122.724181 secs 2024-04-26T19:44:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4122.038180 secs 2024-04-26T19:44:01.413 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4122.724181 secs 2024-04-26T19:44:02.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4526 sec 2024-04-26T19:44:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:02 smithi119 ceph-mon[32752]: pgmap v2978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:02 smithi167 ceph-mon[32751]: pgmap v2978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:02 smithi090 ceph-mon[29180]: pgmap v2978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:03.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4527 sec 2024-04-26T19:44:04.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4528 sec 2024-04-26T19:44:04.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:04 smithi167 ceph-mon[32751]: pgmap v2979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:44:04.335 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:04 smithi119 ceph-mon[32752]: pgmap v2979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:44:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:04 smithi090 ceph-mon[29180]: pgmap v2979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:44:05.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4529 sec 2024-04-26T19:44:05.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:44:05.319 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:44:05.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.383 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.391 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.400 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:44:05.409 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.460 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:05.535 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:06.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4530 sec 2024-04-26T19:44:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4127.038253 secs 2024-04-26T19:44:06.420 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4127.724270 secs 2024-04-26T19:44:06.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:06 smithi119 ceph-mon[32752]: pgmap v2980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:44:06.506 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4127.038253 secs 2024-04-26T19:44:06.515 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4127.724270 secs 2024-04-26T19:44:06.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:06 smithi090 ceph-mon[29180]: pgmap v2980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:44:06.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4127.038253 secs 2024-04-26T19:44:06.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4127.724270 secs 2024-04-26T19:44:06.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:06 smithi167 ceph-mon[32751]: pgmap v2980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:44:07.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4531 sec 2024-04-26T19:44:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.587 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.635 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.737 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.887 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.894 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.902 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.909 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.916 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:07.923 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:08.291 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4532 sec 2024-04-26T19:44:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:08.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:08.640 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.648 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.655 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.682 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.751 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:44:08.759 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:44:08.767 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:08.774 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:08.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:44:08.817 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:44:08.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: pgmap v2981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:44:08.870 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:08.901 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:08.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.950 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:08.987 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:44:09.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:44:09.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.057 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.064 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.071 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.078 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.086 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.093 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:44:09.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:44:09.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.184 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.197 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:44:09.325 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:44:09.335 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: pgmap v2981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:44:09.343 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.395 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:44:09.438 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:44:09.447 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.539 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.641 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.651 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:44:09.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:44:09.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.680 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.689 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.698 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.707 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.724 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:44:09.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:44:09.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: pgmap v2981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:44:09.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:09.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.922 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.983 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:44:09.990 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:44:09.999 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:44:10.007 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:44:10.016 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4533 sec 2024-04-26T19:44:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:44:10.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:09 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:44:10.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:09 smithi090 ceph-mon[29180]: pgmap v2982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 200 B/s wr, 2 op/s 2024-04-26T19:44:10.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:09 smithi090 ceph-mon[29180]: pgmap v2983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 749 B/s rd, 249 B/s wr, 1 op/s 2024-04-26T19:44:10.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4534 sec 2024-04-26T19:44:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:44:10.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:09 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:44:10.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:09 smithi119 ceph-mon[32752]: pgmap v2982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 200 B/s wr, 2 op/s 2024-04-26T19:44:10.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:09 smithi119 ceph-mon[32752]: pgmap v2983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 749 B/s rd, 249 B/s wr, 1 op/s 2024-04-26T19:44:10.408 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:44:10.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:09 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:44:10.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:09 smithi167 ceph-mon[32751]: pgmap v2982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 200 B/s wr, 2 op/s 2024-04-26T19:44:10.515 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:09 smithi167 ceph-mon[32751]: pgmap v2983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 749 B/s rd, 249 B/s wr, 1 op/s 2024-04-26T19:44:11.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4132.038328 secs 2024-04-26T19:44:11.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4535 sec 2024-04-26T19:44:11.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4132.038328 secs 2024-04-26T19:44:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4132.038328 secs 2024-04-26T19:44:12.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4536 sec 2024-04-26T19:44:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:11 smithi119 ceph-mon[32752]: pgmap v2984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 249 B/s wr, 2 op/s 2024-04-26T19:44:12.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4132.724394 secs 2024-04-26T19:44:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:11 smithi167 ceph-mon[32751]: pgmap v2984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 249 B/s wr, 2 op/s 2024-04-26T19:44:12.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4132.724394 secs 2024-04-26T19:44:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:11 smithi090 ceph-mon[29180]: pgmap v2984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 249 B/s wr, 2 op/s 2024-04-26T19:44:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4132.724394 secs 2024-04-26T19:44:13.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4537 sec 2024-04-26T19:44:14.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4538 sec 2024-04-26T19:44:14.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:13 smithi119 ceph-mon[32752]: pgmap v2985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 624 B/s rd, 1 op/s 2024-04-26T19:44:14.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:13 smithi167 ceph-mon[32751]: pgmap v2985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 624 B/s rd, 1 op/s 2024-04-26T19:44:14.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:13 smithi090 ceph-mon[29180]: pgmap v2985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 624 B/s rd, 1 op/s 2024-04-26T19:44:15.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4539 sec 2024-04-26T19:44:16.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4540 sec 2024-04-26T19:44:16.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:15 smithi119 ceph-mon[32752]: pgmap v2986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:16.322 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4137.038412 secs 2024-04-26T19:44:16.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:15 smithi167 ceph-mon[32751]: pgmap v2986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:16.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4137.038412 secs 2024-04-26T19:44:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:15 smithi090 ceph-mon[29180]: pgmap v2986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:16.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4137.038412 secs 2024-04-26T19:44:17.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4541 sec 2024-04-26T19:44:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4137.724453 secs 2024-04-26T19:44:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4137.724453 secs 2024-04-26T19:44:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4137.724453 secs 2024-04-26T19:44:18.292 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4542 sec 2024-04-26T19:44:18.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:17 smithi119 ceph-mon[32752]: pgmap v2987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:18.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:17 smithi167 ceph-mon[32751]: pgmap v2987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:17 smithi090 ceph-mon[29180]: pgmap v2987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:19.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4543 sec 2024-04-26T19:44:20.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4544 sec 2024-04-26T19:44:20.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:19 smithi119 ceph-mon[32752]: pgmap v2988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:44:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:19 smithi167 ceph-mon[32751]: pgmap v2988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:44:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:19 smithi090 ceph-mon[29180]: pgmap v2988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:44:21.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4545 sec 2024-04-26T19:44:21.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4142.038494 secs 2024-04-26T19:44:21.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4142.038494 secs 2024-04-26T19:44:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4142.038494 secs 2024-04-26T19:44:22.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4546 sec 2024-04-26T19:44:22.346 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:21 smithi167 ceph-mon[32751]: pgmap v2989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:22.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4142.724569 secs 2024-04-26T19:44:22.362 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:21 smithi119 ceph-mon[32752]: pgmap v2989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:22.369 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4142.724569 secs 2024-04-26T19:44:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:21 smithi090 ceph-mon[29180]: pgmap v2989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:22.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4142.724569 secs 2024-04-26T19:44:23.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4547 sec 2024-04-26T19:44:24.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4548 sec 2024-04-26T19:44:24.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:23 smithi119 ceph-mon[32752]: pgmap v2990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:24.345 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:23 smithi167 ceph-mon[32751]: pgmap v2990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:23 smithi090 ceph-mon[29180]: pgmap v2990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:25.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4549 sec 2024-04-26T19:44:26.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4550 sec 2024-04-26T19:44:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:25 smithi119 ceph-mon[32752]: pgmap v2991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:26.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4147.038581 secs 2024-04-26T19:44:26.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:25 smithi167 ceph-mon[32751]: pgmap v2991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:26.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4147.038581 secs 2024-04-26T19:44:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:25 smithi090 ceph-mon[29180]: pgmap v2991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:26.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4147.038581 secs 2024-04-26T19:44:27.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4551 sec 2024-04-26T19:44:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4147.724683 secs 2024-04-26T19:44:27.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4147.724683 secs 2024-04-26T19:44:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4147.724683 secs 2024-04-26T19:44:28.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:27 smithi090 ceph-mon[29180]: pgmap v2992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:28.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:27 smithi167 ceph-mon[32751]: pgmap v2992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:28.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:27 smithi119 ceph-mon[32752]: pgmap v2992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:28.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4552 sec 2024-04-26T19:44:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:44:29.015+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:44:29.115 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:44:29.015+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:44:29.124 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:44:29.015+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 36 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:44:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:44:29.015+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 36 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:44:29.293 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4553 sec 2024-04-26T19:44:30.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4554 sec 2024-04-26T19:44:30.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:29 smithi119 ceph-mon[32752]: pgmap v2993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:30.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:29 smithi167 ceph-mon[32751]: pgmap v2993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:29 smithi090 ceph-mon[29180]: pgmap v2993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:31.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4152.038664 secs 2024-04-26T19:44:31.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4555 sec 2024-04-26T19:44:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4152.038664 secs 2024-04-26T19:44:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4152.038664 secs 2024-04-26T19:44:32.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4556 sec 2024-04-26T19:44:32.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:31 smithi167 ceph-mon[32751]: pgmap v2994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:32.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4152.724822 secs 2024-04-26T19:44:32.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:31 smithi119 ceph-mon[32752]: pgmap v2994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:32.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4152.724822 secs 2024-04-26T19:44:32.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:31 smithi090 ceph-mon[29180]: pgmap v2994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:32.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4152.724822 secs 2024-04-26T19:44:32.400 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:44:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:44:31.917+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:44:32.408 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:44:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:44:31.917+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:44:32.416 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:44:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:44:31.917+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:44:32.453 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:44:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:44:31.917+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:44:32.459 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:44:31 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:44:31.917+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:44:33.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4557 sec 2024-04-26T19:44:34.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4558 sec 2024-04-26T19:44:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:33 smithi119 ceph-mon[32752]: pgmap v2995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:33 smithi167 ceph-mon[32751]: pgmap v2995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:33 smithi090 ceph-mon[29180]: pgmap v2995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:35.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4559 sec 2024-04-26T19:44:36.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4560 sec 2024-04-26T19:44:36.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:35 smithi119 ceph-mon[32752]: pgmap v2996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:36.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4157.038749 secs 2024-04-26T19:44:36.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:35 smithi167 ceph-mon[32751]: pgmap v2996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:36.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4157.038749 secs 2024-04-26T19:44:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:35 smithi090 ceph-mon[29180]: pgmap v2996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:36.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4157.038749 secs 2024-04-26T19:44:37.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4561 sec 2024-04-26T19:44:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 4157.724941 secs 2024-04-26T19:44:37.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:36 smithi119 ceph-mon[32752]: slow request 2884.361804 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:44:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 4157.724941 secs 2024-04-26T19:44:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:36 smithi167 ceph-mon[32751]: slow request 2884.361804 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:44:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 4157.724941 secs 2024-04-26T19:44:37.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:36 smithi090 ceph-mon[29180]: slow request 2884.361804 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:44:38.294 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4562 sec 2024-04-26T19:44:38.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:37 smithi167 ceph-mon[32751]: pgmap v2997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:38.425 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:37 smithi119 ceph-mon[32752]: pgmap v2997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:38.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:37 smithi090 ceph-mon[29180]: pgmap v2997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:39.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4563 sec 2024-04-26T19:44:40.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4564 sec 2024-04-26T19:44:40.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:39 smithi167 ceph-mon[32751]: pgmap v2998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:40.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:39 smithi119 ceph-mon[32752]: pgmap v2998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:39 smithi090 ceph-mon[29180]: pgmap v2998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:41.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4162.038821 secs 2024-04-26T19:44:41.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4565 sec 2024-04-26T19:44:41.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4162.038821 secs 2024-04-26T19:44:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4162.038821 secs 2024-04-26T19:44:42.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4566 sec 2024-04-26T19:44:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:41 smithi119 ceph-mon[32752]: pgmap v2999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:42.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4162.725084 secs 2024-04-26T19:44:42.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:41 smithi167 ceph-mon[32751]: pgmap v2999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:42.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4162.725084 secs 2024-04-26T19:44:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:41 smithi090 ceph-mon[29180]: pgmap v2999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4162.725084 secs 2024-04-26T19:44:43.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4567 sec 2024-04-26T19:44:44.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4568 sec 2024-04-26T19:44:44.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:43 smithi167 ceph-mon[32751]: pgmap v3000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:44.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:43 smithi119 ceph-mon[32752]: pgmap v3000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:44.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:43 smithi090 ceph-mon[29180]: pgmap v3000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T19:44:45.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4569 sec 2024-04-26T19:44:46.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4570 sec 2024-04-26T19:44:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:45 smithi119 ceph-mon[32752]: pgmap v3001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:46.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4167.038911 secs 2024-04-26T19:44:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:45 smithi167 ceph-mon[32751]: pgmap v3001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:46.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4167.038911 secs 2024-04-26T19:44:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:45 smithi090 ceph-mon[29180]: pgmap v3001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T19:44:46.406 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4167.038911 secs 2024-04-26T19:44:47.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4571 sec 2024-04-26T19:44:47.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4167.725198 secs 2024-04-26T19:44:47.365 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4167.725198 secs 2024-04-26T19:44:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4167.725198 secs 2024-04-26T19:44:48.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4572 sec 2024-04-26T19:44:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:47 smithi167 ceph-mon[32751]: pgmap v3002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:48.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:47 smithi119 ceph-mon[32752]: pgmap v3002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:47 smithi090 ceph-mon[29180]: pgmap v3002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:49.295 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4573 sec 2024-04-26T19:44:50.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4574 sec 2024-04-26T19:44:50.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:49 smithi167 ceph-mon[32751]: pgmap v3003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:50.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:49 smithi119 ceph-mon[32752]: pgmap v3003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:50.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:49 smithi090 ceph-mon[29180]: pgmap v3003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:51.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4575 sec 2024-04-26T19:44:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4172.038995 secs 2024-04-26T19:44:51.330 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4172.038995 secs 2024-04-26T19:44:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4172.038995 secs 2024-04-26T19:44:52.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4576 sec 2024-04-26T19:44:52.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:51 smithi119 ceph-mon[32752]: pgmap v3004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:52.322 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4172.725316 secs 2024-04-26T19:44:52.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:51 smithi167 ceph-mon[32751]: pgmap v3004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:52.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4172.725316 secs 2024-04-26T19:44:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:51 smithi090 ceph-mon[29180]: pgmap v3004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:52.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4172.725316 secs 2024-04-26T19:44:53.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4577 sec 2024-04-26T19:44:54.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4578 sec 2024-04-26T19:44:54.318 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:53 smithi119 ceph-mon[32752]: pgmap v3005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:54.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:53 smithi167 ceph-mon[32751]: pgmap v3005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:53 smithi090 ceph-mon[29180]: pgmap v3005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:55.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4579 sec 2024-04-26T19:44:56.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4580 sec 2024-04-26T19:44:56.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:55 smithi119 ceph-mon[32752]: pgmap v3006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:56.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4177.039054 secs 2024-04-26T19:44:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:55 smithi090 ceph-mon[29180]: pgmap v3006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:56.390 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4177.039054 secs 2024-04-26T19:44:56.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:55 smithi167 ceph-mon[32751]: pgmap v3006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:44:56.414 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4177.039054 secs 2024-04-26T19:44:57.296 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4581 sec 2024-04-26T19:44:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4177.725459 secs 2024-04-26T19:44:57.333 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4177.725459 secs 2024-04-26T19:44:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4177.725459 secs 2024-04-26T19:44:58.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:57 smithi090 ceph-mon[29180]: pgmap v3007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:58.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4582 sec 2024-04-26T19:44:58.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:57 smithi119 ceph-mon[32752]: pgmap v3007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:57 smithi167 ceph-mon[32751]: pgmap v3007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:44:59.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4583 sec 2024-04-26T19:45:00.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4584 sec 2024-04-26T19:45:00.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:44:59 smithi119 ceph-mon[32752]: pgmap v3008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:00.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:44:59 smithi167 ceph-mon[32751]: pgmap v3008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:44:59 smithi090 ceph-mon[29180]: pgmap v3008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:01.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4585 sec 2024-04-26T19:45:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:01 smithi119 ceph-mon[32752]: 10 slow requests, 1 included below; oldest blocked for > 4182.039111 secs 2024-04-26T19:45:01.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:01 smithi119 ceph-mon[32752]: slow request 364.084743 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:45:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:01 smithi167 ceph-mon[32751]: 10 slow requests, 1 included below; oldest blocked for > 4182.039111 secs 2024-04-26T19:45:01.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:01 smithi167 ceph-mon[32751]: slow request 364.084743 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:45:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:01 smithi090 ceph-mon[29180]: 10 slow requests, 1 included below; oldest blocked for > 4182.039111 secs 2024-04-26T19:45:01.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:01 smithi090 ceph-mon[29180]: slow request 364.084743 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:45:02.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4586 sec 2024-04-26T19:45:02.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:02 smithi119 ceph-mon[32752]: pgmap v3009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:02.331 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4182.725561 secs 2024-04-26T19:45:02.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:02 smithi090 ceph-mon[29180]: pgmap v3009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:02.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4182.725561 secs 2024-04-26T19:45:02.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:02 smithi167 ceph-mon[32751]: pgmap v3009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:02.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4182.725561 secs 2024-04-26T19:45:03.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4587 sec 2024-04-26T19:45:03.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:03 smithi119 ceph-mon[32752]: pgmap v3010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:03 smithi167 ceph-mon[32751]: pgmap v3010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:03 smithi090 ceph-mon[29180]: pgmap v3010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:04.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4588 sec 2024-04-26T19:45:05.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4589 sec 2024-04-26T19:45:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:05 smithi119 ceph-mon[32752]: pgmap v3011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:05.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:05 smithi167 ceph-mon[32751]: pgmap v3011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:05 smithi090 ceph-mon[29180]: pgmap v3011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:06.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4590 sec 2024-04-26T19:45:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:06 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4187.039174 secs 2024-04-26T19:45:06.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4187.725669 secs 2024-04-26T19:45:06.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:06 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4187.039174 secs 2024-04-26T19:45:06.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4187.725669 secs 2024-04-26T19:45:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:06 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4187.039174 secs 2024-04-26T19:45:06.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4187.725669 secs 2024-04-26T19:45:07.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4591 sec 2024-04-26T19:45:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:07 smithi119 ceph-mon[32752]: pgmap v3012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:45:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:07 smithi167 ceph-mon[32751]: pgmap v3012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:45:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:07 smithi090 ceph-mon[29180]: pgmap v3012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:45:08.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4592 sec 2024-04-26T19:45:09.297 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4593 sec 2024-04-26T19:45:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:09 smithi119 ceph-mon[32752]: pgmap v3013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:45:09.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:45:09.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:09.576 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:09.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:09 smithi167 ceph-mon[32751]: pgmap v3013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:45:09.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:45:09.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:09.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:09 smithi090 ceph-mon[29180]: pgmap v3013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:45:09.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:45:09.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:09.652 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:10.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4594 sec 2024-04-26T19:45:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:45:10.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:45:10.569 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:10 smithi119 ceph-mon[32752]: pgmap v3014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:10.716 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:10 smithi119 ceph-mon[32752]: pgmap v3015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:10.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:10.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:45:10.799 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:45:10.809 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:10 smithi090 ceph-mon[29180]: pgmap v3014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:10.820 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:10 smithi090 ceph-mon[29180]: pgmap v3015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:10.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:10.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:45:10.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:45:10.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:10 smithi167 ceph-mon[32751]: pgmap v3014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:10.868 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:10 smithi167 ceph-mon[32751]: pgmap v3015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:10.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:45:11.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4595 sec 2024-04-26T19:45:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:11 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4192.039250 secs 2024-04-26T19:45:11.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4192.725798 secs 2024-04-26T19:45:11.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:11 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4192.039250 secs 2024-04-26T19:45:11.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4192.725798 secs 2024-04-26T19:45:11.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:11 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4192.039250 secs 2024-04-26T19:45:11.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4192.725798 secs 2024-04-26T19:45:12.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4596 sec 2024-04-26T19:45:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:12 smithi119 ceph-mon[32752]: pgmap v3016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:12 smithi167 ceph-mon[32751]: pgmap v3016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:12 smithi090 ceph-mon[29180]: pgmap v3016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:45:13.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4597 sec 2024-04-26T19:45:14.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4598 sec 2024-04-26T19:45:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:14 smithi119 ceph-mon[32752]: pgmap v3017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:14.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:14 smithi090 ceph-mon[29180]: pgmap v3017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:14.914 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:14 smithi167 ceph-mon[32751]: pgmap v3017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:15.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4599 sec 2024-04-26T19:45:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4197.039328 secs 2024-04-26T19:45:15.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4197.039328 secs 2024-04-26T19:45:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4197.039328 secs 2024-04-26T19:45:16.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4600 sec 2024-04-26T19:45:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:16 smithi119 ceph-mon[32752]: pgmap v3018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:16.887 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4197.725867 secs 2024-04-26T19:45:16.950 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:16 smithi090 ceph-mon[29180]: pgmap v3018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:17.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4197.725867 secs 2024-04-26T19:45:17.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:16 smithi167 ceph-mon[32751]: pgmap v3018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4197.725867 secs 2024-04-26T19:45:17.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4601 sec 2024-04-26T19:45:18.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4602 sec 2024-04-26T19:45:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:18 smithi119 ceph-mon[32752]: pgmap v3019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:18.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:18 smithi167 ceph-mon[32751]: pgmap v3019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:18.898 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:18 smithi090 ceph-mon[29180]: pgmap v3019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:45:19.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4603 sec 2024-04-26T19:45:20.298 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4604 sec 2024-04-26T19:45:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:20 smithi119 ceph-mon[32752]: pgmap v3020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:45:20.848 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4202.039409 secs 2024-04-26T19:45:20.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:20 smithi090 ceph-mon[29180]: pgmap v3020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:45:20.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4202.039409 secs 2024-04-26T19:45:21.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:20 smithi167 ceph-mon[32751]: pgmap v3020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:45:21.176 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4202.039409 secs 2024-04-26T19:45:21.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4605 sec 2024-04-26T19:45:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4202.726014 secs 2024-04-26T19:45:21.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4202.726014 secs 2024-04-26T19:45:21.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4202.726014 secs 2024-04-26T19:45:22.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4606 sec 2024-04-26T19:45:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:22 smithi119 ceph-mon[32752]: pgmap v3021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:22.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:22 smithi167 ceph-mon[32751]: pgmap v3021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:22.897 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:22 smithi090 ceph-mon[29180]: pgmap v3021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:23.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4607 sec 2024-04-26T19:45:24.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4608 sec 2024-04-26T19:45:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:24 smithi119 ceph-mon[32752]: pgmap v3022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:24.869 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:24 smithi167 ceph-mon[32751]: pgmap v3022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:24.932 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:24 smithi090 ceph-mon[29180]: pgmap v3022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:25.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4609 sec 2024-04-26T19:45:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4207.039491 secs 2024-04-26T19:45:25.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4207.039491 secs 2024-04-26T19:45:25.917 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4207.039491 secs 2024-04-26T19:45:26.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4610 sec 2024-04-26T19:45:26.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:26 smithi090 ceph-mon[29180]: pgmap v3023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:26.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4207.726150 secs 2024-04-26T19:45:26.914 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:26 smithi167 ceph-mon[32751]: pgmap v3023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:26.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4207.726150 secs 2024-04-26T19:45:27.012 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:26 smithi119 ceph-mon[32752]: pgmap v3023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:27.078 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4207.726150 secs 2024-04-26T19:45:27.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4611 sec 2024-04-26T19:45:28.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4612 sec 2024-04-26T19:45:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:28 smithi119 ceph-mon[32752]: pgmap v3024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:28.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:28 smithi167 ceph-mon[32751]: pgmap v3024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:28.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:28 smithi090 ceph-mon[29180]: pgmap v3024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:29.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4613 sec 2024-04-26T19:45:30.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4614 sec 2024-04-26T19:45:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:30 smithi167 ceph-mon[32751]: pgmap v3025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:30.913 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4212.039572 secs 2024-04-26T19:45:30.947 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:30 smithi090 ceph-mon[29180]: pgmap v3025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:30.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4212.039572 secs 2024-04-26T19:45:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:30 smithi119 ceph-mon[32752]: pgmap v3025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:31.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4212.039572 secs 2024-04-26T19:45:31.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4615 sec 2024-04-26T19:45:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4212.726328 secs 2024-04-26T19:45:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4212.726328 secs 2024-04-26T19:45:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4212.726328 secs 2024-04-26T19:45:32.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4616 sec 2024-04-26T19:45:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:32 smithi167 ceph-mon[32751]: pgmap v3026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:32.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:32 smithi090 ceph-mon[29180]: pgmap v3026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:32 smithi119 ceph-mon[32752]: pgmap v3026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:33.299 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4617 sec 2024-04-26T19:45:34.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4618 sec 2024-04-26T19:45:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:34 smithi090 ceph-mon[29180]: pgmap v3027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:34 smithi119 ceph-mon[32752]: pgmap v3027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:35.101 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:34 smithi167 ceph-mon[32751]: pgmap v3027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:35.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4619 sec 2024-04-26T19:45:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4217.039658 secs 2024-04-26T19:45:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4217.039658 secs 2024-04-26T19:45:36.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4217.039658 secs 2024-04-26T19:45:36.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4620 sec 2024-04-26T19:45:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:36 smithi090 ceph-mon[29180]: pgmap v3028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:36.952 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4217.726465 secs 2024-04-26T19:45:37.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:36 smithi167 ceph-mon[32751]: pgmap v3028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:37.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4217.726465 secs 2024-04-26T19:45:37.169 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:36 smithi119 ceph-mon[32752]: pgmap v3028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:37.235 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4217.726465 secs 2024-04-26T19:45:37.342 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4621 sec 2024-04-26T19:45:38.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4622 sec 2024-04-26T19:45:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:38 smithi090 ceph-mon[29180]: pgmap v3029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:38 smithi119 ceph-mon[32752]: pgmap v3029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:39.113 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:38 smithi167 ceph-mon[32751]: pgmap v3029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:39.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4623 sec 2024-04-26T19:45:40.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4624 sec 2024-04-26T19:45:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:40 smithi090 ceph-mon[29180]: pgmap v3030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:40.921 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4222.039736 secs 2024-04-26T19:45:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:40 smithi119 ceph-mon[32752]: pgmap v3030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:41.084 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4222.039736 secs 2024-04-26T19:45:41.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:40 smithi167 ceph-mon[32751]: pgmap v3030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:41.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4222.039736 secs 2024-04-26T19:45:41.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4625 sec 2024-04-26T19:45:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4222.726576 secs 2024-04-26T19:45:41.966 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4222.726576 secs 2024-04-26T19:45:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4222.726576 secs 2024-04-26T19:45:42.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4626 sec 2024-04-26T19:45:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:42 smithi090 ceph-mon[29180]: pgmap v3031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:42 smithi119 ceph-mon[32752]: pgmap v3031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:43.133 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:42 smithi167 ceph-mon[32751]: pgmap v3031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:43.300 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4627 sec 2024-04-26T19:45:44.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4628 sec 2024-04-26T19:45:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:44 smithi090 ceph-mon[29180]: pgmap v3032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:44 smithi119 ceph-mon[32752]: pgmap v3032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:45.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:44 smithi167 ceph-mon[32751]: pgmap v3032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:45.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4629 sec 2024-04-26T19:45:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4227.039815 secs 2024-04-26T19:45:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4227.039815 secs 2024-04-26T19:45:46.101 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4227.039815 secs 2024-04-26T19:45:46.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4630 sec 2024-04-26T19:45:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:46 smithi090 ceph-mon[29180]: pgmap v3033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:46.915 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4227.726684 secs 2024-04-26T19:45:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:46 smithi119 ceph-mon[32752]: pgmap v3033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:47.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4227.726684 secs 2024-04-26T19:45:47.137 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:46 smithi167 ceph-mon[32751]: pgmap v3033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:47.262 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4227.726684 secs 2024-04-26T19:45:47.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4631 sec 2024-04-26T19:45:48.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4632 sec 2024-04-26T19:45:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:48 smithi090 ceph-mon[29180]: pgmap v3034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:48 smithi119 ceph-mon[32752]: pgmap v3034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:49.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:48 smithi167 ceph-mon[32751]: pgmap v3034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:49.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4633 sec 2024-04-26T19:45:50.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4634 sec 2024-04-26T19:45:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:50 smithi090 ceph-mon[29180]: pgmap v3035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:50.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4232.039875 secs 2024-04-26T19:45:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:50 smithi119 ceph-mon[32752]: pgmap v3035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:51.115 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4232.039875 secs 2024-04-26T19:45:51.208 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:50 smithi167 ceph-mon[32751]: pgmap v3035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:51.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4232.039875 secs 2024-04-26T19:45:51.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4635 sec 2024-04-26T19:45:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4232.726756 secs 2024-04-26T19:45:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4232.726756 secs 2024-04-26T19:45:52.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4232.726756 secs 2024-04-26T19:45:52.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4636 sec 2024-04-26T19:45:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:52 smithi090 ceph-mon[29180]: pgmap v3036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:52 smithi119 ceph-mon[32752]: pgmap v3036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:53.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:52 smithi167 ceph-mon[32751]: pgmap v3036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:53.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4637 sec 2024-04-26T19:45:54.301 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4638 sec 2024-04-26T19:45:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:54 smithi090 ceph-mon[29180]: pgmap v3037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:54 smithi119 ceph-mon[32752]: pgmap v3037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:54 smithi167 ceph-mon[32751]: pgmap v3037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:45:55.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4639 sec 2024-04-26T19:45:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4237.039958 secs 2024-04-26T19:45:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4237.039958 secs 2024-04-26T19:45:56.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4237.039958 secs 2024-04-26T19:45:56.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4640 sec 2024-04-26T19:45:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:56 smithi090 ceph-mon[29180]: pgmap v3038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:57.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4237.726816 secs 2024-04-26T19:45:57.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:56 smithi167 ceph-mon[32751]: pgmap v3038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:57.236 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4237.726816 secs 2024-04-26T19:45:57.249 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:56 smithi119 ceph-mon[32752]: pgmap v3038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:57.258 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4237.726816 secs 2024-04-26T19:45:57.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4641 sec 2024-04-26T19:45:58.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4642 sec 2024-04-26T19:45:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:45:58 smithi119 ceph-mon[32752]: pgmap v3039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:59.082 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:45:58 smithi167 ceph-mon[32751]: pgmap v3039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:45:58 smithi090 ceph-mon[29180]: pgmap v3039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:45:59.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4643 sec 2024-04-26T19:46:00.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4644 sec 2024-04-26T19:46:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:00 smithi119 ceph-mon[32752]: pgmap v3040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:01.109 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:00 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4242.040042 secs 2024-04-26T19:46:01.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:00 smithi090 ceph-mon[29180]: pgmap v3040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:01.161 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:00 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4242.040042 secs 2024-04-26T19:46:01.169 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:00 smithi167 ceph-mon[32751]: pgmap v3040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:01.177 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:00 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4242.040042 secs 2024-04-26T19:46:01.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4645 sec 2024-04-26T19:46:01.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4242.726950 secs 2024-04-26T19:46:02.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4242.726950 secs 2024-04-26T19:46:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4242.726950 secs 2024-04-26T19:46:02.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4646 sec 2024-04-26T19:46:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:02 smithi119 ceph-mon[32752]: pgmap v3041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:03.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:02 smithi167 ceph-mon[32751]: pgmap v3041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:02 smithi090 ceph-mon[29180]: pgmap v3041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:03.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4647 sec 2024-04-26T19:46:04.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4648 sec 2024-04-26T19:46:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:04 smithi119 ceph-mon[32752]: pgmap v3042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:05.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:04 smithi167 ceph-mon[32751]: pgmap v3042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:05.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:04 smithi090 ceph-mon[29180]: pgmap v3042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:05.310 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4649 sec 2024-04-26T19:46:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4247.040104 secs 2024-04-26T19:46:06.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4247.040104 secs 2024-04-26T19:46:06.204 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4247.040104 secs 2024-04-26T19:46:06.302 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4650 sec 2024-04-26T19:46:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4247.727110 secs 2024-04-26T19:46:07.128 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:06 smithi119 ceph-mon[32752]: pgmap v3043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:07.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4247.727110 secs 2024-04-26T19:46:07.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:06 smithi090 ceph-mon[29180]: pgmap v3043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:07.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4247.727110 secs 2024-04-26T19:46:07.165 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:06 smithi167 ceph-mon[32751]: pgmap v3043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:07.303 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4651 sec 2024-04-26T19:46:08.303 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4652 sec 2024-04-26T19:46:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:08 smithi119 ceph-mon[32752]: pgmap v3044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:09.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:08 smithi090 ceph-mon[29180]: pgmap v3044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:09.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:08 smithi167 ceph-mon[32751]: pgmap v3044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:09.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4653 sec 2024-04-26T19:46:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:46:10.149 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:46:10.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:46:10.303 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4654 sec 2024-04-26T19:46:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:10 smithi119 ceph-mon[32752]: pgmap v3045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:11.101 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4252.040174 secs 2024-04-26T19:46:11.147 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:46:11.211 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:46:11.242 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:46:11.309 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:10 smithi090 ceph-mon[29180]: pgmap v3045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4252.040174 secs 2024-04-26T19:46:11.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:46:11.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:46:11.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:46:11.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:10 smithi167 ceph-mon[32751]: pgmap v3045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:11.685 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4252.040174 secs 2024-04-26T19:46:11.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:46:11.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:46:11.895 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:46:11.972 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4655 sec 2024-04-26T19:46:12.074 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:11 smithi119 ceph-mon[32752]: pgmap v3046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:46:12.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:11 smithi119 ceph-mon[32752]: pgmap v3047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:12.171 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4252.727216 secs 2024-04-26T19:46:12.230 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:11 smithi167 ceph-mon[32751]: pgmap v3046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:46:12.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:11 smithi167 ceph-mon[32751]: pgmap v3047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:12.383 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4252.727216 secs 2024-04-26T19:46:12.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:11 smithi090 ceph-mon[29180]: pgmap v3046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:46:12.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:11 smithi090 ceph-mon[29180]: pgmap v3047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:12.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4252.727216 secs 2024-04-26T19:46:12.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4656 sec 2024-04-26T19:46:13.303 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4657 sec 2024-04-26T19:46:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:13 smithi090 ceph-mon[29180]: pgmap v3048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:46:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:13 smithi119 ceph-mon[32752]: pgmap v3048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:46:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:13 smithi167 ceph-mon[32751]: pgmap v3048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:46:14.303 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4658 sec 2024-04-26T19:46:15.303 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4659 sec 2024-04-26T19:46:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:15 smithi119 ceph-mon[32752]: pgmap v3049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:16.104 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4257.040255 secs 2024-04-26T19:46:16.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:15 smithi090 ceph-mon[29180]: pgmap v3049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:16.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4257.040255 secs 2024-04-26T19:46:16.198 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:15 smithi167 ceph-mon[32751]: pgmap v3049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:16.206 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4257.040255 secs 2024-04-26T19:46:16.303 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4660 sec 2024-04-26T19:46:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4257.727339 secs 2024-04-26T19:46:17.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4257.727339 secs 2024-04-26T19:46:17.197 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4257.727339 secs 2024-04-26T19:46:17.304 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4661 sec 2024-04-26T19:46:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:17 smithi119 ceph-mon[32752]: pgmap v3050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:18.095 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:17 smithi167 ceph-mon[32751]: pgmap v3050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:17 smithi090 ceph-mon[29180]: pgmap v3050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:46:18.304 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4662 sec 2024-04-26T19:46:19.304 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4663 sec 2024-04-26T19:46:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:19 smithi119 ceph-mon[32752]: pgmap v3051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T19:46:20.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:19 smithi167 ceph-mon[32751]: pgmap v3051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T19:46:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:19 smithi090 ceph-mon[29180]: pgmap v3051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T19:46:20.304 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4664 sec 2024-04-26T19:46:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4262.040336 secs 2024-04-26T19:46:21.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4262.040336 secs 2024-04-26T19:46:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4262.040336 secs 2024-04-26T19:46:21.304 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4665 sec 2024-04-26T19:46:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:21 smithi119 ceph-mon[32752]: pgmap v3052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:46:22.135 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4262.727483 secs 2024-04-26T19:46:22.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:21 smithi090 ceph-mon[29180]: pgmap v3052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:46:22.242 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4262.727483 secs 2024-04-26T19:46:22.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:21 smithi167 ceph-mon[32751]: pgmap v3052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:46:22.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4262.727483 secs 2024-04-26T19:46:22.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4666 sec 2024-04-26T19:46:23.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4667 sec 2024-04-26T19:46:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:23 smithi119 ceph-mon[32752]: pgmap v3053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:24.097 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:23 smithi167 ceph-mon[32751]: pgmap v3053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:24.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:23 smithi090 ceph-mon[29180]: pgmap v3053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:24.304 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4668 sec 2024-04-26T19:46:25.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4669 sec 2024-04-26T19:46:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:25 smithi119 ceph-mon[32752]: pgmap v3054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:26.113 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4267.040420 secs 2024-04-26T19:46:26.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:25 smithi090 ceph-mon[29180]: pgmap v3054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:26.191 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4267.040420 secs 2024-04-26T19:46:26.224 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:25 smithi167 ceph-mon[32751]: pgmap v3054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:26.270 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4267.040420 secs 2024-04-26T19:46:26.317 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4670 sec 2024-04-26T19:46:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4267.727563 secs 2024-04-26T19:46:27.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4267.727563 secs 2024-04-26T19:46:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4267.727563 secs 2024-04-26T19:46:27.304 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4671 sec 2024-04-26T19:46:28.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:27 smithi090 ceph-mon[29180]: pgmap v3055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:27 smithi119 ceph-mon[32752]: pgmap v3055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:27 smithi167 ceph-mon[32751]: pgmap v3055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:28.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4672 sec 2024-04-26T19:46:29.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4673 sec 2024-04-26T19:46:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:29 smithi119 ceph-mon[32752]: pgmap v3056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:30.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:29 smithi167 ceph-mon[32751]: pgmap v3056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:30.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:29 smithi090 ceph-mon[29180]: pgmap v3056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:30.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4674 sec 2024-04-26T19:46:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4272.040495 secs 2024-04-26T19:46:31.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4272.040495 secs 2024-04-26T19:46:31.240 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4272.040495 secs 2024-04-26T19:46:31.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4675 sec 2024-04-26T19:46:31.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:31 smithi167 ceph-mon[32751]: pgmap v3057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:32.008 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4272.727690 secs 2024-04-26T19:46:32.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:31 smithi119 ceph-mon[32752]: pgmap v3057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:32.103 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4272.727690 secs 2024-04-26T19:46:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:31 smithi090 ceph-mon[29180]: pgmap v3057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:32.250 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4272.727690 secs 2024-04-26T19:46:32.320 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4676 sec 2024-04-26T19:46:33.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4677 sec 2024-04-26T19:46:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:33 smithi119 ceph-mon[32752]: pgmap v3058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:34.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:33 smithi090 ceph-mon[29180]: pgmap v3058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:34.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:33 smithi167 ceph-mon[32751]: pgmap v3058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:34.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4678 sec 2024-04-26T19:46:35.305 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4679 sec 2024-04-26T19:46:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:35 smithi119 ceph-mon[32752]: pgmap v3059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:36.101 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4277.040576 secs 2024-04-26T19:46:36.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:35 smithi090 ceph-mon[29180]: pgmap v3059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:36.293 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4277.040576 secs 2024-04-26T19:46:36.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:35 smithi167 ceph-mon[32751]: pgmap v3059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:36.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4277.040576 secs 2024-04-26T19:46:36.322 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4680 sec 2024-04-26T19:46:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4277.727814 secs 2024-04-26T19:46:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4277.727814 secs 2024-04-26T19:46:37.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4277.727814 secs 2024-04-26T19:46:37.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4681 sec 2024-04-26T19:46:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:37 smithi119 ceph-mon[32752]: pgmap v3060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:38.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:37 smithi167 ceph-mon[32751]: pgmap v3060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:38.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:37 smithi090 ceph-mon[29180]: pgmap v3060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:38.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4682 sec 2024-04-26T19:46:39.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4683 sec 2024-04-26T19:46:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:39 smithi119 ceph-mon[32752]: pgmap v3061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:40.152 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:39 smithi090 ceph-mon[29180]: pgmap v3061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:40.190 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:39 smithi167 ceph-mon[32751]: pgmap v3061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:40.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4684 sec 2024-04-26T19:46:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4282.040657 secs 2024-04-26T19:46:41.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4282.040657 secs 2024-04-26T19:46:41.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4282.040657 secs 2024-04-26T19:46:41.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4685 sec 2024-04-26T19:46:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:41 smithi119 ceph-mon[32752]: pgmap v3062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:42.127 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4282.727948 secs 2024-04-26T19:46:42.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:41 smithi090 ceph-mon[29180]: pgmap v3062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:42.146 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4282.727948 secs 2024-04-26T19:46:42.154 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:41 smithi167 ceph-mon[32751]: pgmap v3062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:42.162 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4282.727948 secs 2024-04-26T19:46:42.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4686 sec 2024-04-26T19:46:43.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4687 sec 2024-04-26T19:46:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:43 smithi119 ceph-mon[32752]: pgmap v3063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:44.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:43 smithi167 ceph-mon[32751]: pgmap v3063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:44.146 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:43 smithi090 ceph-mon[29180]: pgmap v3063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:44.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4688 sec 2024-04-26T19:46:45.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4689 sec 2024-04-26T19:46:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:45 smithi119 ceph-mon[32752]: pgmap v3064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4287.040749 secs 2024-04-26T19:46:46.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:45 smithi167 ceph-mon[32751]: pgmap v3064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:46.133 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4287.040749 secs 2024-04-26T19:46:46.187 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:45 smithi090 ceph-mon[29180]: pgmap v3064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:46.231 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4287.040749 secs 2024-04-26T19:46:46.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4690 sec 2024-04-26T19:46:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4287.728068 secs 2024-04-26T19:46:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4287.728068 secs 2024-04-26T19:46:47.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4287.728068 secs 2024-04-26T19:46:47.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4691 sec 2024-04-26T19:46:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:47 smithi119 ceph-mon[32752]: pgmap v3065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:48.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:47 smithi167 ceph-mon[32751]: pgmap v3065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:48.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:47 smithi090 ceph-mon[29180]: pgmap v3065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:48.306 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4692 sec 2024-04-26T19:46:49.310 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4693 sec 2024-04-26T19:46:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:49 smithi119 ceph-mon[32752]: pgmap v3066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:50.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:49 smithi167 ceph-mon[32751]: pgmap v3066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:50.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:49 smithi090 ceph-mon[29180]: pgmap v3066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:50.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4694 sec 2024-04-26T19:46:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4292.040808 secs 2024-04-26T19:46:51.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4292.040808 secs 2024-04-26T19:46:51.632 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4292.040808 secs 2024-04-26T19:46:51.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4695 sec 2024-04-26T19:46:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:51 smithi119 ceph-mon[32752]: pgmap v3067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:52.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4292.728185 secs 2024-04-26T19:46:52.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:51 smithi090 ceph-mon[29180]: pgmap v3067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:52.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4292.728185 secs 2024-04-26T19:46:52.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:51 smithi167 ceph-mon[32751]: pgmap v3067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:46:52.281 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4292.728185 secs 2024-04-26T19:46:52.340 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4696 sec 2024-04-26T19:46:53.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4697 sec 2024-04-26T19:46:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:53 smithi119 ceph-mon[32752]: pgmap v3068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:54.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:53 smithi167 ceph-mon[32751]: pgmap v3068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:53 smithi090 ceph-mon[29180]: pgmap v3068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:54.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4698 sec 2024-04-26T19:46:55.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4699 sec 2024-04-26T19:46:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:55 smithi119 ceph-mon[32752]: pgmap v3069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:56.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4297.040896 secs 2024-04-26T19:46:56.085 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:55 smithi167 ceph-mon[32751]: pgmap v3069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:56.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4297.040896 secs 2024-04-26T19:46:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:55 smithi090 ceph-mon[29180]: pgmap v3069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:46:56.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4297.040896 secs 2024-04-26T19:46:56.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4700 sec 2024-04-26T19:46:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4297.728252 secs 2024-04-26T19:46:57.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4297.728252 secs 2024-04-26T19:46:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4297.728252 secs 2024-04-26T19:46:57.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4701 sec 2024-04-26T19:46:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:57 smithi119 ceph-mon[32752]: pgmap v3070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:58.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:57 smithi090 ceph-mon[29180]: pgmap v3070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:58.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:57 smithi167 ceph-mon[32751]: pgmap v3070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:46:58.307 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4702 sec 2024-04-26T19:46:59.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4703 sec 2024-04-26T19:47:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:46:59 smithi119 ceph-mon[32752]: pgmap v3071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:00.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:46:59 smithi167 ceph-mon[32751]: pgmap v3071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:00.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:46:59 smithi090 ceph-mon[29180]: pgmap v3071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:00.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4704 sec 2024-04-26T19:47:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:00 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4302.040982 secs 2024-04-26T19:47:01.113 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:00 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4302.040982 secs 2024-04-26T19:47:01.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:00 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4302.040982 secs 2024-04-26T19:47:01.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4705 sec 2024-04-26T19:47:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:01 smithi119 ceph-mon[32752]: pgmap v3072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:47:02.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4302.728334 secs 2024-04-26T19:47:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:01 smithi167 ceph-mon[32751]: pgmap v3072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:47:02.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4302.728334 secs 2024-04-26T19:47:02.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:01 smithi090 ceph-mon[29180]: pgmap v3072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:47:02.211 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4302.728334 secs 2024-04-26T19:47:02.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4706 sec 2024-04-26T19:47:03.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4707 sec 2024-04-26T19:47:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:03 smithi119 ceph-mon[32752]: pgmap v3073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:04.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:03 smithi090 ceph-mon[29180]: pgmap v3073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:04.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:03 smithi167 ceph-mon[32751]: pgmap v3073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:04.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4708 sec 2024-04-26T19:47:05.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4709 sec 2024-04-26T19:47:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:05 smithi119 ceph-mon[32752]: pgmap v3074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:06.110 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4307.041047 secs 2024-04-26T19:47:06.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:05 smithi090 ceph-mon[29180]: pgmap v3074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:06.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4307.041047 secs 2024-04-26T19:47:06.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:05 smithi167 ceph-mon[32751]: pgmap v3074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:06.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4307.041047 secs 2024-04-26T19:47:06.337 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4710 sec 2024-04-26T19:47:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4307.728473 secs 2024-04-26T19:47:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4307.728473 secs 2024-04-26T19:47:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4307.728473 secs 2024-04-26T19:47:07.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4711 sec 2024-04-26T19:47:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:07 smithi119 ceph-mon[32752]: pgmap v3075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:08.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:07 smithi167 ceph-mon[32751]: pgmap v3075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:08.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:07 smithi090 ceph-mon[29180]: pgmap v3075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:08.308 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4712 sec 2024-04-26T19:47:09.309 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4713 sec 2024-04-26T19:47:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:09 smithi119 ceph-mon[32752]: pgmap v3076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:10.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:09 smithi167 ceph-mon[32751]: pgmap v3076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:10.209 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:09 smithi090 ceph-mon[29180]: pgmap v3076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:10.309 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4714 sec 2024-04-26T19:47:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4312.041104 secs 2024-04-26T19:47:11.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:47:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4312.041104 secs 2024-04-26T19:47:11.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:47:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4312.041104 secs 2024-04-26T19:47:11.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:47:11.321 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4715 sec 2024-04-26T19:47:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:11 smithi167 ceph-mon[32751]: pgmap v3077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:47:12.100 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4312.728564 secs 2024-04-26T19:47:12.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:47:12.241 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:47:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:47:12.396 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:11 smithi119 ceph-mon[32752]: pgmap v3077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:47:12.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4312.728564 secs 2024-04-26T19:47:12.487 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:47:12.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:47:12.504 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:47:12.512 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:11 smithi090 ceph-mon[29180]: pgmap v3077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:47:12.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4312.728564 secs 2024-04-26T19:47:12.570 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:47:12.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:47:12.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:47:12.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4716 sec 2024-04-26T19:47:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:12 smithi167 ceph-mon[32751]: pgmap v3078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:47:13.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:12 smithi167 ceph-mon[32751]: pgmap v3079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:13.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:12 smithi090 ceph-mon[29180]: pgmap v3078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:47:13.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:12 smithi090 ceph-mon[29180]: pgmap v3079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:12 smithi119 ceph-mon[32752]: pgmap v3078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:47:13.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:12 smithi119 ceph-mon[32752]: pgmap v3079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:13.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4717 sec 2024-04-26T19:47:14.309 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4718 sec 2024-04-26T19:47:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:14 smithi167 ceph-mon[32751]: pgmap v3080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:47:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:14 smithi090 ceph-mon[29180]: pgmap v3080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:47:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:14 smithi119 ceph-mon[32752]: pgmap v3080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:47:15.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4719 sec 2024-04-26T19:47:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4317.041191 secs 2024-04-26T19:47:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4317.041191 secs 2024-04-26T19:47:16.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4317.041191 secs 2024-04-26T19:47:16.334 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4720 sec 2024-04-26T19:47:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:16 smithi090 ceph-mon[29180]: pgmap v3081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:17.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4317.728628 secs 2024-04-26T19:47:17.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:16 smithi167 ceph-mon[32751]: pgmap v3081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:17.486 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4317.728628 secs 2024-04-26T19:47:17.569 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:16 smithi119 ceph-mon[32752]: pgmap v3081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:17.623 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4317.728628 secs 2024-04-26T19:47:17.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4721 sec 2024-04-26T19:47:18.309 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4722 sec 2024-04-26T19:47:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:18 smithi090 ceph-mon[29180]: pgmap v3082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:47:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:18 smithi119 ceph-mon[32752]: pgmap v3082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:47:19.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:18 smithi167 ceph-mon[32751]: pgmap v3082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:47:19.430 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4723 sec 2024-04-26T19:47:20.310 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4724 sec 2024-04-26T19:47:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:20 smithi090 ceph-mon[29180]: pgmap v3083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:21.212 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4322.041269 secs 2024-04-26T19:47:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:20 smithi119 ceph-mon[32752]: pgmap v3083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:21.364 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4322.041269 secs 2024-04-26T19:47:21.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:20 smithi167 ceph-mon[32751]: pgmap v3083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:47:21.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4322.041269 secs 2024-04-26T19:47:21.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4725 sec 2024-04-26T19:47:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4322.728711 secs 2024-04-26T19:47:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4322.728711 secs 2024-04-26T19:47:22.313 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4726 sec 2024-04-26T19:47:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4322.728711 secs 2024-04-26T19:47:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:22 smithi090 ceph-mon[29180]: pgmap v3084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:47:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:22 smithi119 ceph-mon[32752]: pgmap v3084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:47:23.367 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:22 smithi167 ceph-mon[32751]: pgmap v3084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:47:23.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4727 sec 2024-04-26T19:47:24.310 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4728 sec 2024-04-26T19:47:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:24 smithi090 ceph-mon[29180]: pgmap v3085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:24 smithi119 ceph-mon[32752]: pgmap v3085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:25.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:24 smithi167 ceph-mon[32751]: pgmap v3085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:25.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4729 sec 2024-04-26T19:47:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4327.041354 secs 2024-04-26T19:47:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4327.041354 secs 2024-04-26T19:47:26.312 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4730 sec 2024-04-26T19:47:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4327.041354 secs 2024-04-26T19:47:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:26 smithi090 ceph-mon[29180]: pgmap v3086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:27.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4327.728822 secs 2024-04-26T19:47:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:26 smithi119 ceph-mon[32752]: pgmap v3086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:27.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4327.728822 secs 2024-04-26T19:47:27.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:26 smithi167 ceph-mon[32751]: pgmap v3086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:27.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4327.728822 secs 2024-04-26T19:47:27.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4731 sec 2024-04-26T19:47:28.310 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4732 sec 2024-04-26T19:47:29.040 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:47:29.040+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:47:29.067 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:47:29.040+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:47:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:47:29.040+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 37 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:47:29.164 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:47:29.040+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 37 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:47:29.208 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:28 smithi090 ceph-mon[29180]: pgmap v3087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:28 smithi119 ceph-mon[32752]: pgmap v3087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:29.312 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4733 sec 2024-04-26T19:47:29.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:28 smithi167 ceph-mon[32751]: pgmap v3087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:30.310 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4734 sec 2024-04-26T19:47:31.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:30 smithi167 ceph-mon[32751]: pgmap v3088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:31.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4332.041429 secs 2024-04-26T19:47:31.221 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:30 smithi090 ceph-mon[29180]: pgmap v3088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:31.266 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4332.041429 secs 2024-04-26T19:47:31.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:30 smithi119 ceph-mon[32752]: pgmap v3088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:31.424 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4332.041429 secs 2024-04-26T19:47:31.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4735 sec 2024-04-26T19:47:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4332.728927 secs 2024-04-26T19:47:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4332.728927 secs 2024-04-26T19:47:32.314 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4736 sec 2024-04-26T19:47:32.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4332.728927 secs 2024-04-26T19:47:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:32 smithi090 ceph-mon[29180]: pgmap v3089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:32 smithi119 ceph-mon[32752]: pgmap v3089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:33.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:32 smithi167 ceph-mon[32751]: pgmap v3089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:33.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4737 sec 2024-04-26T19:47:34.311 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4738 sec 2024-04-26T19:47:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:34 smithi090 ceph-mon[29180]: pgmap v3090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:34 smithi119 ceph-mon[32752]: pgmap v3090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:34 smithi167 ceph-mon[32751]: pgmap v3090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:35.330 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4739 sec 2024-04-26T19:47:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4337.041514 secs 2024-04-26T19:47:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4337.041514 secs 2024-04-26T19:47:36.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4337.041514 secs 2024-04-26T19:47:36.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4740 sec 2024-04-26T19:47:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:36 smithi090 ceph-mon[29180]: pgmap v3091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:37.192 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4337.728996 secs 2024-04-26T19:47:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:36 smithi119 ceph-mon[32752]: pgmap v3091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:37.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4337.728996 secs 2024-04-26T19:47:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:36 smithi167 ceph-mon[32751]: pgmap v3091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:37.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4337.728996 secs 2024-04-26T19:47:37.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4741 sec 2024-04-26T19:47:38.311 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4742 sec 2024-04-26T19:47:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:38 smithi090 ceph-mon[29180]: pgmap v3092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:38 smithi119 ceph-mon[32752]: pgmap v3092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:39.377 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:38 smithi167 ceph-mon[32751]: pgmap v3092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:39.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4743 sec 2024-04-26T19:47:40.311 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4744 sec 2024-04-26T19:47:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:40 smithi119 ceph-mon[32752]: pgmap v3093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:41.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4342.041596 secs 2024-04-26T19:47:41.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:40 smithi167 ceph-mon[32751]: pgmap v3093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:41.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4342.041596 secs 2024-04-26T19:47:41.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4745 sec 2024-04-26T19:47:41.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:40 smithi090 ceph-mon[29180]: pgmap v3093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:41.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4342.041596 secs 2024-04-26T19:47:42.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4342.729112 secs 2024-04-26T19:47:42.311 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4746 sec 2024-04-26T19:47:42.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4342.729112 secs 2024-04-26T19:47:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4342.729112 secs 2024-04-26T19:47:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:42 smithi119 ceph-mon[32752]: pgmap v3094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:43.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:42 smithi167 ceph-mon[32751]: pgmap v3094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:43.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4747 sec 2024-04-26T19:47:43.517 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:42 smithi090 ceph-mon[29180]: pgmap v3094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:44.311 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4748 sec 2024-04-26T19:47:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:44 smithi119 ceph-mon[32752]: pgmap v3095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:45.358 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:44 smithi167 ceph-mon[32751]: pgmap v3095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:45.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4749 sec 2024-04-26T19:47:45.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:44 smithi090 ceph-mon[29180]: pgmap v3095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4347.041682 secs 2024-04-26T19:47:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4347.041682 secs 2024-04-26T19:47:46.314 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4750 sec 2024-04-26T19:47:46.404 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4347.041682 secs 2024-04-26T19:47:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:46 smithi119 ceph-mon[32752]: pgmap v3096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:47.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4347.729224 secs 2024-04-26T19:47:47.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:46 smithi167 ceph-mon[32751]: pgmap v3096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:47.405 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4347.729224 secs 2024-04-26T19:47:47.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4751 sec 2024-04-26T19:47:47.442 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:46 smithi090 ceph-mon[29180]: pgmap v3096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:47.510 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4347.729224 secs 2024-04-26T19:47:48.312 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4752 sec 2024-04-26T19:47:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:48 smithi119 ceph-mon[32752]: pgmap v3097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:49.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:48 smithi167 ceph-mon[32751]: pgmap v3097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:49.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4753 sec 2024-04-26T19:47:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:48 smithi090 ceph-mon[29180]: pgmap v3097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:50.312 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4754 sec 2024-04-26T19:47:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:50 smithi119 ceph-mon[32752]: pgmap v3098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:51.339 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4352.041763 secs 2024-04-26T19:47:51.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:50 smithi167 ceph-mon[32751]: pgmap v3098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:51.373 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4352.041763 secs 2024-04-26T19:47:51.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4755 sec 2024-04-26T19:47:51.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:50 smithi090 ceph-mon[29180]: pgmap v3098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:51.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4352.041763 secs 2024-04-26T19:47:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4352.729338 secs 2024-04-26T19:47:52.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4352.729338 secs 2024-04-26T19:47:52.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4756 sec 2024-04-26T19:47:52.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4352.729338 secs 2024-04-26T19:47:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:52 smithi119 ceph-mon[32752]: pgmap v3099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:53.375 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:52 smithi167 ceph-mon[32751]: pgmap v3099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:53.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4757 sec 2024-04-26T19:47:53.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:52 smithi090 ceph-mon[29180]: pgmap v3099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:54.312 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4758 sec 2024-04-26T19:47:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:54 smithi119 ceph-mon[32752]: pgmap v3100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:55.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:54 smithi167 ceph-mon[32751]: pgmap v3100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:55.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4759 sec 2024-04-26T19:47:55.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:54 smithi090 ceph-mon[29180]: pgmap v3100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4357.041852 secs 2024-04-26T19:47:56.413 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4357.041852 secs 2024-04-26T19:47:56.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4760 sec 2024-04-26T19:47:56.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4357.041852 secs 2024-04-26T19:47:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:56 smithi119 ceph-mon[32752]: pgmap v3101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:57.352 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4357.729445 secs 2024-04-26T19:47:57.392 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:56 smithi167 ceph-mon[32751]: pgmap v3101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:57.401 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4357.729445 secs 2024-04-26T19:47:57.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4761 sec 2024-04-26T19:47:57.417 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:56 smithi090 ceph-mon[29180]: pgmap v3101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:47:57.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4357.729445 secs 2024-04-26T19:47:58.313 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4762 sec 2024-04-26T19:47:59.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:47:58 smithi090 ceph-mon[29180]: pgmap v3102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:47:58 smithi119 ceph-mon[32752]: pgmap v3102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:59.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:47:58 smithi167 ceph-mon[32751]: pgmap v3102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:47:59.412 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4763 sec 2024-04-26T19:48:00.313 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4764 sec 2024-04-26T19:48:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:00 smithi119 ceph-mon[32752]: pgmap v3103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:01.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:00 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4362.041934 secs 2024-04-26T19:48:01.449 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:00 smithi167 ceph-mon[32751]: pgmap v3103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:01.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:00 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4362.041934 secs 2024-04-26T19:48:01.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4765 sec 2024-04-26T19:48:01.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:00 smithi090 ceph-mon[29180]: pgmap v3103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:01.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:00 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4362.041934 secs 2024-04-26T19:48:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4362.729561 secs 2024-04-26T19:48:02.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4362.729561 secs 2024-04-26T19:48:02.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4766 sec 2024-04-26T19:48:02.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4362.729561 secs 2024-04-26T19:48:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:02 smithi119 ceph-mon[32752]: pgmap v3104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:03.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:02 smithi167 ceph-mon[32751]: pgmap v3104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:03.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4767 sec 2024-04-26T19:48:03.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:02 smithi090 ceph-mon[29180]: pgmap v3104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:04.313 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4768 sec 2024-04-26T19:48:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:04 smithi119 ceph-mon[32752]: pgmap v3105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:05.368 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:04 smithi167 ceph-mon[32751]: pgmap v3105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:05.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4769 sec 2024-04-26T19:48:05.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:04 smithi090 ceph-mon[29180]: pgmap v3105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4367.042016 secs 2024-04-26T19:48:06.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4367.042016 secs 2024-04-26T19:48:06.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4770 sec 2024-04-26T19:48:06.441 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4367.042016 secs 2024-04-26T19:48:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:06 smithi119 ceph-mon[32752]: pgmap v3106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:07.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4367.729629 secs 2024-04-26T19:48:07.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:06 smithi167 ceph-mon[32751]: pgmap v3106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:07.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4367.729629 secs 2024-04-26T19:48:07.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4771 sec 2024-04-26T19:48:07.492 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:06 smithi090 ceph-mon[29180]: pgmap v3106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:07.585 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4367.729629 secs 2024-04-26T19:48:08.314 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4772 sec 2024-04-26T19:48:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:08 smithi119 ceph-mon[32752]: pgmap v3107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:09.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:08 smithi167 ceph-mon[32751]: pgmap v3107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:09.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4773 sec 2024-04-26T19:48:09.471 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:08 smithi090 ceph-mon[29180]: pgmap v3107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:10.314 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4774 sec 2024-04-26T19:48:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:10 smithi119 ceph-mon[32752]: pgmap v3108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:11.353 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4372.042097 secs 2024-04-26T19:48:11.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:10 smithi167 ceph-mon[32751]: pgmap v3108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:11.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4372.042097 secs 2024-04-26T19:48:11.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4775 sec 2024-04-26T19:48:11.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:10 smithi090 ceph-mon[29180]: pgmap v3108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:11.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4372.042097 secs 2024-04-26T19:48:12.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4372.729753 secs 2024-04-26T19:48:12.234 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:48:12.273 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:48:12.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:48:12.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:48:12.314 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4776 sec 2024-04-26T19:48:12.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4372.729753 secs 2024-04-26T19:48:12.401 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:48:12.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:48:12.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:48:12.636 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:48:12.760 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4372.729753 secs 2024-04-26T19:48:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:48:12.899 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:48:12.935 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:48:12.965 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:48:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:12 smithi119 ceph-mon[32752]: pgmap v3109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:13.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:12 smithi119 ceph-mon[32752]: pgmap v3110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:48:13.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:12 smithi119 ceph-mon[32752]: pgmap v3111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:13.406 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:12 smithi167 ceph-mon[32751]: pgmap v3109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:13.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:12 smithi167 ceph-mon[32751]: pgmap v3110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:48:13.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:12 smithi167 ceph-mon[32751]: pgmap v3111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:13.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4777 sec 2024-04-26T19:48:13.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:12 smithi090 ceph-mon[29180]: pgmap v3109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:13.582 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:12 smithi090 ceph-mon[29180]: pgmap v3110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:48:13.591 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:12 smithi090 ceph-mon[29180]: pgmap v3111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:14.314 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4778 sec 2024-04-26T19:48:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:14 smithi119 ceph-mon[32752]: pgmap v3112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:15.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:14 smithi167 ceph-mon[32751]: pgmap v3112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:15.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4779 sec 2024-04-26T19:48:15.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:14 smithi090 ceph-mon[29180]: pgmap v3112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4377.042182 secs 2024-04-26T19:48:16.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4377.042182 secs 2024-04-26T19:48:16.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4780 sec 2024-04-26T19:48:16.424 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4377.042182 secs 2024-04-26T19:48:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4377.729872 secs 2024-04-26T19:48:17.327 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:16 smithi119 ceph-mon[32752]: pgmap v3113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:17.384 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4377.729872 secs 2024-04-26T19:48:17.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:16 smithi167 ceph-mon[32751]: pgmap v3113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:17.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4781 sec 2024-04-26T19:48:17.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4377.729872 secs 2024-04-26T19:48:17.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:16 smithi090 ceph-mon[29180]: pgmap v3113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:18.315 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4782 sec 2024-04-26T19:48:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:18 smithi119 ceph-mon[32752]: pgmap v3114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:48:19.345 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:18 smithi167 ceph-mon[32751]: pgmap v3114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:48:19.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4783 sec 2024-04-26T19:48:19.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:18 smithi090 ceph-mon[29180]: pgmap v3114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:48:20.315 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4784 sec 2024-04-26T19:48:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4382.042262 secs 2024-04-26T19:48:21.393 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:20 smithi119 ceph-mon[32752]: pgmap v3115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:21.441 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4382.042262 secs 2024-04-26T19:48:21.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:20 smithi167 ceph-mon[32751]: pgmap v3115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:21.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4785 sec 2024-04-26T19:48:21.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4382.042262 secs 2024-04-26T19:48:21.702 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:20 smithi090 ceph-mon[29180]: pgmap v3115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:48:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4382.729982 secs 2024-04-26T19:48:22.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4382.729982 secs 2024-04-26T19:48:22.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4786 sec 2024-04-26T19:48:22.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4382.729982 secs 2024-04-26T19:48:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:22 smithi119 ceph-mon[32752]: pgmap v3116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:48:23.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:22 smithi167 ceph-mon[32751]: pgmap v3116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:48:23.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4787 sec 2024-04-26T19:48:23.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:22 smithi090 ceph-mon[29180]: pgmap v3116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:48:24.315 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4788 sec 2024-04-26T19:48:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:24 smithi119 ceph-mon[32752]: pgmap v3117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:25.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:24 smithi167 ceph-mon[32751]: pgmap v3117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:25.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4789 sec 2024-04-26T19:48:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:24 smithi090 ceph-mon[29180]: pgmap v3117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4387.042344 secs 2024-04-26T19:48:26.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4387.042344 secs 2024-04-26T19:48:26.333 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4790 sec 2024-04-26T19:48:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4387.042344 secs 2024-04-26T19:48:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4387.730096 secs 2024-04-26T19:48:27.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:26 smithi119 ceph-mon[32752]: pgmap v3118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:27.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4387.730096 secs 2024-04-26T19:48:27.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:26 smithi167 ceph-mon[32751]: pgmap v3118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:27.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4791 sec 2024-04-26T19:48:27.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4387.730096 secs 2024-04-26T19:48:27.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:26 smithi090 ceph-mon[29180]: pgmap v3118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:28.316 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4792 sec 2024-04-26T19:48:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:28 smithi119 ceph-mon[32752]: pgmap v3119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:29.352 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:28 smithi167 ceph-mon[32751]: pgmap v3119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:29.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4793 sec 2024-04-26T19:48:29.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:28 smithi090 ceph-mon[29180]: pgmap v3119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:30.316 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4794 sec 2024-04-26T19:48:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4392.042398 secs 2024-04-26T19:48:31.364 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:30 smithi119 ceph-mon[32752]: pgmap v3120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:31.380 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4392.042398 secs 2024-04-26T19:48:31.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:30 smithi167 ceph-mon[32751]: pgmap v3120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:31.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4795 sec 2024-04-26T19:48:31.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4392.042398 secs 2024-04-26T19:48:31.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:30 smithi090 ceph-mon[29180]: pgmap v3120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4392.730206 secs 2024-04-26T19:48:32.397 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4392.730206 secs 2024-04-26T19:48:32.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4796 sec 2024-04-26T19:48:32.553 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4392.730206 secs 2024-04-26T19:48:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:33 smithi090 ceph-mon[29180]: pgmap v3121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:33 smithi119 ceph-mon[32752]: pgmap v3121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:33.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:33 smithi167 ceph-mon[32751]: pgmap v3121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:33.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4797 sec 2024-04-26T19:48:34.316 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4798 sec 2024-04-26T19:48:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:35 smithi119 ceph-mon[32752]: pgmap v3122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:35.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:35 smithi167 ceph-mon[32751]: pgmap v3122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:35.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4799 sec 2024-04-26T19:48:35.477 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:35 smithi090 ceph-mon[29180]: pgmap v3122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:36 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4397.042483 secs 2024-04-26T19:48:36.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:36 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4397.042483 secs 2024-04-26T19:48:36.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4800 sec 2024-04-26T19:48:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:36 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4397.042483 secs 2024-04-26T19:48:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4397.730307 secs 2024-04-26T19:48:37.411 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:37 smithi119 ceph-mon[32752]: pgmap v3123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:37.442 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4397.730307 secs 2024-04-26T19:48:37.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:37 smithi167 ceph-mon[32751]: pgmap v3123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:37.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4801 sec 2024-04-26T19:48:37.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4397.730307 secs 2024-04-26T19:48:37.536 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:37 smithi090 ceph-mon[29180]: pgmap v3123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:38.316 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4802 sec 2024-04-26T19:48:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:39 smithi119 ceph-mon[32752]: pgmap v3124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:39.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:39 smithi167 ceph-mon[32751]: pgmap v3124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:39.332 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4803 sec 2024-04-26T19:48:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:39 smithi090 ceph-mon[29180]: pgmap v3124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:40.317 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4804 sec 2024-04-26T19:48:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4402.042563 secs 2024-04-26T19:48:41.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:41 smithi119 ceph-mon[32752]: pgmap v3125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:41.424 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4402.042563 secs 2024-04-26T19:48:41.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:41 smithi167 ceph-mon[32751]: pgmap v3125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:41.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4805 sec 2024-04-26T19:48:41.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4402.042563 secs 2024-04-26T19:48:41.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:41 smithi090 ceph-mon[29180]: pgmap v3125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4402.730451 secs 2024-04-26T19:48:42.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4402.730451 secs 2024-04-26T19:48:42.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4806 sec 2024-04-26T19:48:42.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4402.730451 secs 2024-04-26T19:48:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:43 smithi119 ceph-mon[32752]: pgmap v3126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:43.351 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:43 smithi167 ceph-mon[32751]: pgmap v3126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:43.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4807 sec 2024-04-26T19:48:43.447 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:43 smithi090 ceph-mon[29180]: pgmap v3126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:44.317 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4808 sec 2024-04-26T19:48:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:45 smithi119 ceph-mon[32752]: pgmap v3127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:45.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:45 smithi167 ceph-mon[32751]: pgmap v3127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:45.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4809 sec 2024-04-26T19:48:45.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:45 smithi090 ceph-mon[29180]: pgmap v3127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4407.042647 secs 2024-04-26T19:48:46.440 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4407.042647 secs 2024-04-26T19:48:46.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4810 sec 2024-04-26T19:48:46.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4407.042647 secs 2024-04-26T19:48:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4407.730547 secs 2024-04-26T19:48:47.380 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:47 smithi119 ceph-mon[32752]: pgmap v3128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:47.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4407.730547 secs 2024-04-26T19:48:47.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:47 smithi167 ceph-mon[32751]: pgmap v3128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:47.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4811 sec 2024-04-26T19:48:47.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4407.730547 secs 2024-04-26T19:48:47.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:47 smithi090 ceph-mon[29180]: pgmap v3128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:48.317 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4812 sec 2024-04-26T19:48:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:49 smithi119 ceph-mon[32752]: pgmap v3129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:49.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:49 smithi167 ceph-mon[32751]: pgmap v3129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:49.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4813 sec 2024-04-26T19:48:49.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:49 smithi090 ceph-mon[29180]: pgmap v3129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:50.318 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4814 sec 2024-04-26T19:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4412.042734 secs 2024-04-26T19:48:51.385 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:51 smithi119 ceph-mon[32752]: pgmap v3130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:51.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4412.042734 secs 2024-04-26T19:48:51.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:51 smithi167 ceph-mon[32751]: pgmap v3130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:51.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4815 sec 2024-04-26T19:48:51.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4412.042734 secs 2024-04-26T19:48:51.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:51 smithi090 ceph-mon[29180]: pgmap v3130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4412.730666 secs 2024-04-26T19:48:52.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:52 smithi119 ceph-mon[32752]: pgmap v3131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:52.418 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4412.730666 secs 2024-04-26T19:48:52.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:52 smithi167 ceph-mon[32751]: pgmap v3131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:52.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4816 sec 2024-04-26T19:48:52.638 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4412.730666 secs 2024-04-26T19:48:52.698 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:52 smithi090 ceph-mon[29180]: pgmap v3131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:53.318 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4817 sec 2024-04-26T19:48:54.318 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4818 sec 2024-04-26T19:48:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:54 smithi090 ceph-mon[29180]: pgmap v3132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:54 smithi119 ceph-mon[32752]: pgmap v3132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:55.394 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4819 sec 2024-04-26T19:48:55.454 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:54 smithi167 ceph-mon[32751]: pgmap v3132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:48:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4417.042826 secs 2024-04-26T19:48:56.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4820 sec 2024-04-26T19:48:56.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4417.042826 secs 2024-04-26T19:48:56.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4417.042826 secs 2024-04-26T19:48:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4417.730773 secs 2024-04-26T19:48:57.362 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:56 smithi119 ceph-mon[32752]: pgmap v3133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:57.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4821 sec 2024-04-26T19:48:57.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4417.730773 secs 2024-04-26T19:48:57.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:56 smithi167 ceph-mon[32751]: pgmap v3133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:57.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4417.730773 secs 2024-04-26T19:48:57.756 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:56 smithi090 ceph-mon[29180]: pgmap v3133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:58.318 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4822 sec 2024-04-26T19:48:59.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:48:58 smithi090 ceph-mon[29180]: pgmap v3134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:48:58 smithi119 ceph-mon[32752]: pgmap v3134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:48:59.426 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4823 sec 2024-04-26T19:48:59.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:48:58 smithi167 ceph-mon[32751]: pgmap v3134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:00.318 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4824 sec 2024-04-26T19:49:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:00 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4422.042906 secs 2024-04-26T19:49:01.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:00 smithi119 ceph-mon[32752]: pgmap v3135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:01.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4825 sec 2024-04-26T19:49:01.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:00 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4422.042906 secs 2024-04-26T19:49:01.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:00 smithi167 ceph-mon[32751]: pgmap v3135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:01.494 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:00 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4422.042906 secs 2024-04-26T19:49:01.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:00 smithi090 ceph-mon[29180]: pgmap v3135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4422.730885 secs 2024-04-26T19:49:02.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4826 sec 2024-04-26T19:49:02.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4422.730885 secs 2024-04-26T19:49:02.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4422.730885 secs 2024-04-26T19:49:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:02 smithi119 ceph-mon[32752]: pgmap v3136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:03.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4827 sec 2024-04-26T19:49:03.422 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:02 smithi167 ceph-mon[32751]: pgmap v3136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:03.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:02 smithi090 ceph-mon[29180]: pgmap v3136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:04.319 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4828 sec 2024-04-26T19:49:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:04 smithi119 ceph-mon[32752]: pgmap v3137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:05.319 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4829 sec 2024-04-26T19:49:05.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:04 smithi167 ceph-mon[32751]: pgmap v3137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:04 smithi090 ceph-mon[29180]: pgmap v3137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4427.042992 secs 2024-04-26T19:49:06.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4830 sec 2024-04-26T19:49:06.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4427.042992 secs 2024-04-26T19:49:06.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4427.042992 secs 2024-04-26T19:49:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4427.730953 secs 2024-04-26T19:49:07.350 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:06 smithi119 ceph-mon[32752]: pgmap v3138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:07.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4831 sec 2024-04-26T19:49:07.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4427.730953 secs 2024-04-26T19:49:07.377 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:06 smithi167 ceph-mon[32751]: pgmap v3138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:07.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4427.730953 secs 2024-04-26T19:49:07.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:06 smithi090 ceph-mon[29180]: pgmap v3138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:08.319 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4832 sec 2024-04-26T19:49:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:08 smithi119 ceph-mon[32752]: pgmap v3139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:09.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4833 sec 2024-04-26T19:49:09.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:08 smithi167 ceph-mon[32751]: pgmap v3139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:09.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:08 smithi090 ceph-mon[29180]: pgmap v3139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:10.320 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4834 sec 2024-04-26T19:49:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4432.043072 secs 2024-04-26T19:49:11.491 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:10 smithi119 ceph-mon[32752]: pgmap v3140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:11.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4835 sec 2024-04-26T19:49:11.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4432.043072 secs 2024-04-26T19:49:11.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:10 smithi167 ceph-mon[32751]: pgmap v3140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:11.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4432.043072 secs 2024-04-26T19:49:11.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:10 smithi090 ceph-mon[29180]: pgmap v3140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:12.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4432.731073 secs 2024-04-26T19:49:12.249 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:49:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4432.731073 secs 2024-04-26T19:49:12.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:49:12.337 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4836 sec 2024-04-26T19:49:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4432.731073 secs 2024-04-26T19:49:12.437 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:49:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:12 smithi119 ceph-mon[32752]: pgmap v3141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:13.369 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:49:13.429 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:49:13.486 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:49:13.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4837 sec 2024-04-26T19:49:13.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:12 smithi167 ceph-mon[32751]: pgmap v3141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:13.736 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:49:13.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:49:13.913 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:49:13.923 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:12 smithi090 ceph-mon[29180]: pgmap v3141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:13.932 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:49:13.994 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:49:14.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:49:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:13 smithi119 ceph-mon[32752]: pgmap v3142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:49:14.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:13 smithi119 ceph-mon[32752]: pgmap v3143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:49:14.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4838 sec 2024-04-26T19:49:14.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:13 smithi167 ceph-mon[32751]: pgmap v3142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:49:14.612 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:13 smithi167 ceph-mon[32751]: pgmap v3143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:49:14.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:13 smithi090 ceph-mon[29180]: pgmap v3142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:49:14.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:13 smithi090 ceph-mon[29180]: pgmap v3143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:49:15.320 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4839 sec 2024-04-26T19:49:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:15 smithi119 ceph-mon[32752]: pgmap v3144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:16.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4437.043157 secs 2024-04-26T19:49:16.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4840 sec 2024-04-26T19:49:16.400 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:15 smithi167 ceph-mon[32751]: pgmap v3144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:16.408 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4437.043157 secs 2024-04-26T19:49:16.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:15 smithi090 ceph-mon[29180]: pgmap v3144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:16.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4437.043157 secs 2024-04-26T19:49:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4437.731206 secs 2024-04-26T19:49:17.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4841 sec 2024-04-26T19:49:17.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4437.731206 secs 2024-04-26T19:49:17.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4437.731206 secs 2024-04-26T19:49:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:17 smithi119 ceph-mon[32752]: pgmap v3145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:18.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4842 sec 2024-04-26T19:49:18.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:17 smithi167 ceph-mon[32751]: pgmap v3145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:18.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:17 smithi090 ceph-mon[29180]: pgmap v3145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:19.320 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4843 sec 2024-04-26T19:49:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:19 smithi119 ceph-mon[32752]: pgmap v3146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:20.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4844 sec 2024-04-26T19:49:20.383 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:19 smithi167 ceph-mon[32751]: pgmap v3146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:20.412 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:19 smithi090 ceph-mon[29180]: pgmap v3146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4442.043237 secs 2024-04-26T19:49:21.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4845 sec 2024-04-26T19:49:21.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4442.043237 secs 2024-04-26T19:49:21.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4442.043237 secs 2024-04-26T19:49:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4442.731318 secs 2024-04-26T19:49:22.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:21 smithi119 ceph-mon[32752]: pgmap v3147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:22.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4846 sec 2024-04-26T19:49:22.454 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4442.731318 secs 2024-04-26T19:49:22.489 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:21 smithi167 ceph-mon[32751]: pgmap v3147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:22.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4442.731318 secs 2024-04-26T19:49:22.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:21 smithi090 ceph-mon[29180]: pgmap v3147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:49:23.320 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4847 sec 2024-04-26T19:49:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:23 smithi119 ceph-mon[32752]: pgmap v3148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:49:24.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4848 sec 2024-04-26T19:49:24.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:23 smithi167 ceph-mon[32751]: pgmap v3148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:49:24.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:23 smithi090 ceph-mon[29180]: pgmap v3148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:49:25.321 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4849 sec 2024-04-26T19:49:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:25 smithi119 ceph-mon[32752]: pgmap v3149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:26.338 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4447.043324 secs 2024-04-26T19:49:26.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4850 sec 2024-04-26T19:49:26.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:25 smithi167 ceph-mon[32751]: pgmap v3149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:26.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4447.043324 secs 2024-04-26T19:49:26.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:25 smithi090 ceph-mon[29180]: pgmap v3149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:26.590 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4447.043324 secs 2024-04-26T19:49:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4447.731444 secs 2024-04-26T19:49:27.331 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4851 sec 2024-04-26T19:49:27.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4447.731444 secs 2024-04-26T19:49:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4447.731444 secs 2024-04-26T19:49:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:27 smithi119 ceph-mon[32752]: pgmap v3150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:28.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4852 sec 2024-04-26T19:49:28.405 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:27 smithi167 ceph-mon[32751]: pgmap v3150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:28.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:27 smithi090 ceph-mon[29180]: pgmap v3150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:29.321 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4853 sec 2024-04-26T19:49:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:29 smithi119 ceph-mon[32752]: pgmap v3151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:29 smithi167 ceph-mon[32751]: pgmap v3151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:30.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4854 sec 2024-04-26T19:49:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:29 smithi090 ceph-mon[29180]: pgmap v3151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4452.043403 secs 2024-04-26T19:49:31.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4855 sec 2024-04-26T19:49:31.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4452.043403 secs 2024-04-26T19:49:31.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4452.043403 secs 2024-04-26T19:49:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4452.731557 secs 2024-04-26T19:49:32.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:31 smithi119 ceph-mon[32752]: pgmap v3152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:32.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4856 sec 2024-04-26T19:49:32.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4452.731557 secs 2024-04-26T19:49:32.451 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:31 smithi167 ceph-mon[32751]: pgmap v3152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:32.505 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4452.731557 secs 2024-04-26T19:49:32.593 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:31 smithi090 ceph-mon[29180]: pgmap v3152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:33.321 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4857 sec 2024-04-26T19:49:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:33 smithi119 ceph-mon[32752]: pgmap v3153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:34.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4858 sec 2024-04-26T19:49:34.438 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:33 smithi167 ceph-mon[32751]: pgmap v3153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:34.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:33 smithi090 ceph-mon[29180]: pgmap v3153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:35.321 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4859 sec 2024-04-26T19:49:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:35 smithi119 ceph-mon[32752]: pgmap v3154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:36.379 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4457.043489 secs 2024-04-26T19:49:36.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4860 sec 2024-04-26T19:49:36.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:35 smithi167 ceph-mon[32751]: pgmap v3154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:36.739 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4457.043489 secs 2024-04-26T19:49:36.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:35 smithi090 ceph-mon[29180]: pgmap v3154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:36.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4457.043489 secs 2024-04-26T19:49:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:36 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 4457.731665 secs 2024-04-26T19:49:37.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:36 smithi119 ceph-mon[32752]: slow request 2884.257014 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:49:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:36 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 4457.731665 secs 2024-04-26T19:49:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:36 smithi167 ceph-mon[32751]: slow request 2884.257014 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:49:37.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4861 sec 2024-04-26T19:49:37.478 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:36 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 4457.731665 secs 2024-04-26T19:49:37.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:36 smithi090 ceph-mon[29180]: slow request 2884.257014 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:49:38.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:37 smithi119 ceph-mon[32752]: pgmap v3155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:38.331 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4862 sec 2024-04-26T19:49:38.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:37 smithi167 ceph-mon[32751]: pgmap v3155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:37 smithi090 ceph-mon[29180]: pgmap v3155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:39.322 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4863 sec 2024-04-26T19:49:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:39 smithi119 ceph-mon[32752]: pgmap v3156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:40.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4864 sec 2024-04-26T19:49:40.441 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:39 smithi167 ceph-mon[32751]: pgmap v3156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:40.484 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:39 smithi090 ceph-mon[29180]: pgmap v3156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:41 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4462.043572 secs 2024-04-26T19:49:41.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4865 sec 2024-04-26T19:49:41.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:41 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4462.043572 secs 2024-04-26T19:49:41.571 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:41 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4462.043572 secs 2024-04-26T19:49:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4462.731781 secs 2024-04-26T19:49:42.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:42 smithi119 ceph-mon[32752]: pgmap v3157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:42.344 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4866 sec 2024-04-26T19:49:42.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4462.731781 secs 2024-04-26T19:49:42.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:42 smithi167 ceph-mon[32751]: pgmap v3157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:42.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4462.731781 secs 2024-04-26T19:49:42.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:42 smithi090 ceph-mon[29180]: pgmap v3157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:43.322 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4867 sec 2024-04-26T19:49:44.059 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:49:44.058+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:49:44.117 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:49:44.058+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:49:44.157 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:49:44.058+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 38 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:49:44.204 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:49:44.059+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 38 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca80065b0 con 0x7f2cd4104500 2024-04-26T19:49:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:44 smithi119 ceph-mon[32752]: pgmap v3158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:44.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4868 sec 2024-04-26T19:49:44.398 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:44 smithi167 ceph-mon[32751]: pgmap v3158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:44.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:44 smithi090 ceph-mon[29180]: pgmap v3158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:45.322 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4869 sec 2024-04-26T19:49:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:46 smithi119 ceph-mon[32752]: pgmap v3159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:46.378 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:46 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4467.043657 secs 2024-04-26T19:49:46.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4870 sec 2024-04-26T19:49:46.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:46 smithi167 ceph-mon[32751]: pgmap v3159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:46.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:46 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4467.043657 secs 2024-04-26T19:49:46.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:46 smithi090 ceph-mon[29180]: pgmap v3159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:46.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:46 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4467.043657 secs 2024-04-26T19:49:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4467.731915 secs 2024-04-26T19:49:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4467.731915 secs 2024-04-26T19:49:47.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4871 sec 2024-04-26T19:49:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4467.731915 secs 2024-04-26T19:49:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:48 smithi119 ceph-mon[32752]: pgmap v3160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:48.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4872 sec 2024-04-26T19:49:48.365 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:48 smithi167 ceph-mon[32751]: pgmap v3160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:48 smithi090 ceph-mon[29180]: pgmap v3160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:49.323 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4873 sec 2024-04-26T19:49:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:50 smithi119 ceph-mon[32752]: pgmap v3161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:50.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4874 sec 2024-04-26T19:49:50.442 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:50 smithi167 ceph-mon[32751]: pgmap v3161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:50.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:50 smithi090 ceph-mon[29180]: pgmap v3161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:51 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4472.043739 secs 2024-04-26T19:49:51.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4875 sec 2024-04-26T19:49:51.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:51 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4472.043739 secs 2024-04-26T19:49:51.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:51 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4472.043739 secs 2024-04-26T19:49:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4472.732053 secs 2024-04-26T19:49:52.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:52 smithi119 ceph-mon[32752]: pgmap v3162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:52.334 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4876 sec 2024-04-26T19:49:52.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4472.732053 secs 2024-04-26T19:49:52.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:52 smithi167 ceph-mon[32751]: pgmap v3162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:52.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4472.732053 secs 2024-04-26T19:49:52.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:52 smithi090 ceph-mon[29180]: pgmap v3162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:53.323 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4877 sec 2024-04-26T19:49:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:54 smithi119 ceph-mon[32752]: pgmap v3163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:54.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4878 sec 2024-04-26T19:49:54.412 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:54 smithi167 ceph-mon[32751]: pgmap v3163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:54.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:54 smithi090 ceph-mon[29180]: pgmap v3163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:55.323 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4879 sec 2024-04-26T19:49:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:56 smithi119 ceph-mon[32752]: pgmap v3164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:56.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:56 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4477.043830 secs 2024-04-26T19:49:56.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4880 sec 2024-04-26T19:49:56.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:56 smithi167 ceph-mon[32751]: pgmap v3164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:56.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:56 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4477.043830 secs 2024-04-26T19:49:56.519 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:56 smithi090 ceph-mon[29180]: pgmap v3164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:49:56.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:56 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4477.043830 secs 2024-04-26T19:49:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4477.732169 secs 2024-04-26T19:49:57.333 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4881 sec 2024-04-26T19:49:57.413 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4477.732169 secs 2024-04-26T19:49:57.512 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4477.732169 secs 2024-04-26T19:49:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:49:58 smithi119 ceph-mon[32752]: pgmap v3165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:58.347 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4882 sec 2024-04-26T19:49:58.391 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:49:58 smithi167 ceph-mon[32751]: pgmap v3165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:58.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:49:58 smithi090 ceph-mon[29180]: pgmap v3165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:49:59.324 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4883 sec 2024-04-26T19:50:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:00 smithi119 ceph-mon[32752]: pgmap v3166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:00.322 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:50:00.333 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4884 sec 2024-04-26T19:50:00.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:00 smithi167 ceph-mon[32751]: pgmap v3166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:00.518 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:50:00.560 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:00 smithi090 ceph-mon[29180]: pgmap v3166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:00.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T19:50:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:01 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 4482.043913 secs 2024-04-26T19:50:01.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4885 sec 2024-04-26T19:50:01.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:01 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 4482.043913 secs 2024-04-26T19:50:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:01 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 4482.043913 secs 2024-04-26T19:50:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4482.732231 secs 2024-04-26T19:50:02.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:02 smithi119 ceph-mon[32752]: pgmap v3167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:02.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4886 sec 2024-04-26T19:50:02.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4482.732231 secs 2024-04-26T19:50:02.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:02 smithi167 ceph-mon[32751]: pgmap v3167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:02.515 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4482.732231 secs 2024-04-26T19:50:02.560 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:02 smithi090 ceph-mon[29180]: pgmap v3167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:02.688 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:50:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:50:01.965+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:50:02.778 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:50:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:50:01.965+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:50:02.860 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:50:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:50:01.965+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:50:02.901 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:50:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:50:01.965+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:50:02.923 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:50:01 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:50:01.965+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:50:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:03 smithi167 ceph-mon[32751]: pgmap v3168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:03.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4887 sec 2024-04-26T19:50:03.512 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:03 smithi090 ceph-mon[29180]: pgmap v3168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:03.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:03 smithi119 ceph-mon[32752]: pgmap v3168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:04.324 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4888 sec 2024-04-26T19:50:05.324 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4889 sec 2024-04-26T19:50:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:05 smithi119 ceph-mon[32752]: pgmap v3169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:50:06.097 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:05 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4487.043975 secs 2024-04-26T19:50:06.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:05 smithi090 ceph-mon[29180]: pgmap v3169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:50:06.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:05 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4487.043975 secs 2024-04-26T19:50:06.251 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:05 smithi167 ceph-mon[32751]: pgmap v3169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:50:06.360 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:05 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4487.043975 secs 2024-04-26T19:50:06.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4890 sec 2024-04-26T19:50:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4487.732352 secs 2024-04-26T19:50:06.922 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:50:07.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:06 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:50:07.229 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4487.732352 secs 2024-04-26T19:50:07.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:50:07.360 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:06 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:50:07.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4487.732352 secs 2024-04-26T19:50:07.409 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:50:07.417 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:06 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:50:07.425 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4891 sec 2024-04-26T19:50:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:07 smithi119 ceph-mon[32752]: pgmap v3170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:07 smithi090 ceph-mon[29180]: pgmap v3170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:08.154 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:07 smithi167 ceph-mon[32751]: pgmap v3170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:08.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4892 sec 2024-04-26T19:50:09.324 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4893 sec 2024-04-26T19:50:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:09 smithi119 ceph-mon[32752]: pgmap v3171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:50:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:09 smithi167 ceph-mon[32751]: pgmap v3171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:50:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:09 smithi090 ceph-mon[29180]: pgmap v3171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T19:50:10.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4894 sec 2024-04-26T19:50:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:10 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4492.044056 secs 2024-04-26T19:50:11.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:10 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4492.044056 secs 2024-04-26T19:50:11.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:10 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4492.044056 secs 2024-04-26T19:50:11.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4895 sec 2024-04-26T19:50:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4492.732446 secs 2024-04-26T19:50:12.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:11 smithi119 ceph-mon[32752]: pgmap v3172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:12.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4492.732446 secs 2024-04-26T19:50:12.194 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:11 smithi167 ceph-mon[32751]: pgmap v3172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:12.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4492.732446 secs 2024-04-26T19:50:12.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:11 smithi090 ceph-mon[29180]: pgmap v3172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:12.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4896 sec 2024-04-26T19:50:13.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4897 sec 2024-04-26T19:50:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:13 smithi119 ceph-mon[32752]: pgmap v3173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:14.102 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:50:14.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:50:14.276 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:50:14.359 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:50:14.432 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:13 smithi090 ceph-mon[29180]: pgmap v3173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:14.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:50:14.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:50:14.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:50:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:50:14.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:13 smithi167 ceph-mon[32751]: pgmap v3173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T19:50:14.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:50:14.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:50:14.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:50:14.937 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:50:14.984 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4898 sec 2024-04-26T19:50:15.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:14 smithi119 ceph-mon[32752]: pgmap v3174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 190 B/s wr, 3 op/s 2024-04-26T19:50:15.149 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:14 smithi119 ceph-mon[32752]: pgmap v3175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:15.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:14 smithi090 ceph-mon[29180]: pgmap v3174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 190 B/s wr, 3 op/s 2024-04-26T19:50:15.326 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:14 smithi090 ceph-mon[29180]: pgmap v3175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:15.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:14 smithi167 ceph-mon[32751]: pgmap v3174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 190 B/s wr, 3 op/s 2024-04-26T19:50:15.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:14 smithi167 ceph-mon[32751]: pgmap v3175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:15.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4899 sec 2024-04-26T19:50:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:15 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4497.044136 secs 2024-04-26T19:50:16.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:15 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4497.044136 secs 2024-04-26T19:50:16.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:15 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4497.044136 secs 2024-04-26T19:50:16.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4900 sec 2024-04-26T19:50:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:16 smithi119 ceph-mon[32752]: pgmap v3176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:17.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4497.732551 secs 2024-04-26T19:50:17.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:16 smithi090 ceph-mon[29180]: pgmap v3176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:17.220 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4497.732551 secs 2024-04-26T19:50:17.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:16 smithi167 ceph-mon[32751]: pgmap v3176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:17.239 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4497.732551 secs 2024-04-26T19:50:17.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4901 sec 2024-04-26T19:50:18.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4902 sec 2024-04-26T19:50:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:18 smithi119 ceph-mon[32752]: pgmap v3177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:50:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:18 smithi167 ceph-mon[32751]: pgmap v3177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:50:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:18 smithi090 ceph-mon[29180]: pgmap v3177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:50:19.325 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4903 sec 2024-04-26T19:50:20.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4904 sec 2024-04-26T19:50:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:20 smithi119 ceph-mon[32752]: pgmap v3178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:21.147 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:20 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4502.044217 secs 2024-04-26T19:50:21.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:20 smithi090 ceph-mon[29180]: pgmap v3178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:21.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:20 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4502.044217 secs 2024-04-26T19:50:21.237 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:20 smithi167 ceph-mon[32751]: pgmap v3178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:21.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:20 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4502.044217 secs 2024-04-26T19:50:21.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4905 sec 2024-04-26T19:50:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4502.732617 secs 2024-04-26T19:50:22.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4502.732617 secs 2024-04-26T19:50:22.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4502.732617 secs 2024-04-26T19:50:22.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4906 sec 2024-04-26T19:50:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:22 smithi119 ceph-mon[32752]: pgmap v3179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:23.101 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:22 smithi167 ceph-mon[32751]: pgmap v3179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:23.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:22 smithi090 ceph-mon[29180]: pgmap v3179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:50:23.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4907 sec 2024-04-26T19:50:24.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4908 sec 2024-04-26T19:50:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:24 smithi119 ceph-mon[32752]: pgmap v3180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:50:25.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:24 smithi167 ceph-mon[32751]: pgmap v3180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:50:25.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:24 smithi090 ceph-mon[29180]: pgmap v3180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:50:25.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4909 sec 2024-04-26T19:50:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:25 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4507.044303 secs 2024-04-26T19:50:26.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:25 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4507.044303 secs 2024-04-26T19:50:26.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:25 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4507.044303 secs 2024-04-26T19:50:26.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4910 sec 2024-04-26T19:50:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:26 smithi119 ceph-mon[32752]: pgmap v3181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:27.100 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4507.732731 secs 2024-04-26T19:50:27.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:26 smithi167 ceph-mon[32751]: pgmap v3181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:27.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4507.732731 secs 2024-04-26T19:50:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:26 smithi090 ceph-mon[29180]: pgmap v3181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:27.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4507.732731 secs 2024-04-26T19:50:27.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4911 sec 2024-04-26T19:50:28.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4912 sec 2024-04-26T19:50:28.470 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T19:50:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T19:50:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b2280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401bc50 tx=0x7f2cbc0318e0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:50:28.618 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b2280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401bc50 tx=0x7f2cbc0318e0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b2280 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T19:50:28.802 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 msgr2=0x7f2cc00b2280 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T19:50:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00a3dc0 0x7f2cc00b2280 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.975 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.051 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.469+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b47d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010ce0 tx=0x7f2cc4038040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:29.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:28 smithi167 ceph-mon[32751]: pgmap v3182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:29.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:28 smithi119 ceph-mon[32752]: pgmap v3182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:29.337 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.490+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000061) for more than 900000000 us, fault. 2024-04-26T19:50:29.393 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=3 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T19:50:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.626 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.490+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T19:50:29.702 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:28.490+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T19:50:29.779 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:29.064+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:50:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:29.064+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:50:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:29.064+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 39 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:50:29.982 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:50:29.064+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 39 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:50:29.993 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:28 smithi090 ceph-mon[29180]: pgmap v3182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:30.001 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4913 sec 2024-04-26T19:50:30.326 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4914 sec 2024-04-26T19:50:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:30 smithi119 ceph-mon[32752]: pgmap v3183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:31.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:30 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4512.044385 secs 2024-04-26T19:50:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:30 smithi090 ceph-mon[29180]: pgmap v3183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:31.244 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:30 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4512.044385 secs 2024-04-26T19:50:31.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:30 smithi167 ceph-mon[32751]: pgmap v3183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:31.348 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:30 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4512.044385 secs 2024-04-26T19:50:31.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4915 sec 2024-04-26T19:50:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4512.732803 secs 2024-04-26T19:50:32.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4512.732803 secs 2024-04-26T19:50:32.203 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4512.732803 secs 2024-04-26T19:50:32.327 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4916 sec 2024-04-26T19:50:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:32 smithi119 ceph-mon[32752]: pgmap v3184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:33.088 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:32 smithi167 ceph-mon[32751]: pgmap v3184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:33.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:32 smithi090 ceph-mon[29180]: pgmap v3184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:33.327 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4917 sec 2024-04-26T19:50:34.327 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4918 sec 2024-04-26T19:50:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:34 smithi119 ceph-mon[32752]: pgmap v3185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:34 smithi167 ceph-mon[32751]: pgmap v3185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:34 smithi090 ceph-mon[29180]: pgmap v3185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:35.327 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4919 sec 2024-04-26T19:50:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:35 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4517.044467 secs 2024-04-26T19:50:36.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:35 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4517.044467 secs 2024-04-26T19:50:36.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:35 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4517.044467 secs 2024-04-26T19:50:36.327 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4920 sec 2024-04-26T19:50:37.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:36 smithi090 ceph-mon[29180]: pgmap v3186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:37.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4517.732935 secs 2024-04-26T19:50:37.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:36 smithi119 ceph-mon[32752]: pgmap v3186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:37.153 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4517.732935 secs 2024-04-26T19:50:37.203 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:36 smithi167 ceph-mon[32751]: pgmap v3186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:37.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4517.732935 secs 2024-04-26T19:50:37.327 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4921 sec 2024-04-26T19:50:38.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4922 sec 2024-04-26T19:50:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:38 smithi119 ceph-mon[32752]: pgmap v3187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:39.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:38 smithi090 ceph-mon[29180]: pgmap v3187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:39.186 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:38 smithi167 ceph-mon[32751]: pgmap v3187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:39.327 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4923 sec 2024-04-26T19:50:40.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4924 sec 2024-04-26T19:50:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:40 smithi119 ceph-mon[32752]: pgmap v3188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:41.187 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:40 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4522.044547 secs 2024-04-26T19:50:41.260 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:40 smithi090 ceph-mon[29180]: pgmap v3188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:41.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:40 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4522.044547 secs 2024-04-26T19:50:41.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:40 smithi167 ceph-mon[32751]: pgmap v3188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:41.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:40 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4522.044547 secs 2024-04-26T19:50:41.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4925 sec 2024-04-26T19:50:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4522.733049 secs 2024-04-26T19:50:42.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4522.733049 secs 2024-04-26T19:50:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4522.733049 secs 2024-04-26T19:50:42.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4926 sec 2024-04-26T19:50:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:42 smithi119 ceph-mon[32752]: pgmap v3189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:42 smithi090 ceph-mon[29180]: pgmap v3189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:43.184 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:42 smithi167 ceph-mon[32751]: pgmap v3189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:43.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4927 sec 2024-04-26T19:50:44.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4928 sec 2024-04-26T19:50:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:44 smithi119 ceph-mon[32752]: pgmap v3190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:45.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:44 smithi090 ceph-mon[29180]: pgmap v3190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:45.272 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:44 smithi167 ceph-mon[32751]: pgmap v3190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:45.335 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4929 sec 2024-04-26T19:50:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:45 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4527.044632 secs 2024-04-26T19:50:46.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:45 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4527.044632 secs 2024-04-26T19:50:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:45 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4527.044632 secs 2024-04-26T19:50:46.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4930 sec 2024-04-26T19:50:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:46 smithi119 ceph-mon[32752]: pgmap v3191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:47.114 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4527.733146 secs 2024-04-26T19:50:47.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:46 smithi090 ceph-mon[29180]: pgmap v3191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:47.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4527.733146 secs 2024-04-26T19:50:47.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:46 smithi167 ceph-mon[32751]: pgmap v3191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:47.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4527.733146 secs 2024-04-26T19:50:47.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4931 sec 2024-04-26T19:50:48.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4932 sec 2024-04-26T19:50:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:48 smithi119 ceph-mon[32752]: pgmap v3192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:49.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:48 smithi167 ceph-mon[32751]: pgmap v3192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:49.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:48 smithi090 ceph-mon[29180]: pgmap v3192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:49.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4933 sec 2024-04-26T19:50:50.328 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4934 sec 2024-04-26T19:50:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:50 smithi119 ceph-mon[32752]: pgmap v3193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:51.120 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:50 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4532.044715 secs 2024-04-26T19:50:51.159 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:50 smithi090 ceph-mon[29180]: pgmap v3193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:51.232 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:50 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4532.044715 secs 2024-04-26T19:50:51.290 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:50 smithi167 ceph-mon[32751]: pgmap v3193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:51.372 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:50 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4532.044715 secs 2024-04-26T19:50:51.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4935 sec 2024-04-26T19:50:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4532.733299 secs 2024-04-26T19:50:52.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4532.733299 secs 2024-04-26T19:50:52.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4532.733299 secs 2024-04-26T19:50:52.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4936 sec 2024-04-26T19:50:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:52 smithi119 ceph-mon[32752]: pgmap v3194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:53.107 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:52 smithi167 ceph-mon[32751]: pgmap v3194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:53.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:52 smithi090 ceph-mon[29180]: pgmap v3194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:53.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4937 sec 2024-04-26T19:50:54.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4938 sec 2024-04-26T19:50:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:54 smithi119 ceph-mon[32752]: pgmap v3195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:55.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:54 smithi090 ceph-mon[29180]: pgmap v3195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:55.170 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:54 smithi167 ceph-mon[32751]: pgmap v3195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:50:55.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4939 sec 2024-04-26T19:50:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:55 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4537.044818 secs 2024-04-26T19:50:56.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:55 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4537.044818 secs 2024-04-26T19:50:56.228 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:55 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4537.044818 secs 2024-04-26T19:50:56.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4940 sec 2024-04-26T19:50:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:56 smithi119 ceph-mon[32752]: pgmap v3196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:57.134 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4537.733437 secs 2024-04-26T19:50:57.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:56 smithi090 ceph-mon[29180]: pgmap v3196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:57.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4537.733437 secs 2024-04-26T19:50:57.159 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:56 smithi167 ceph-mon[32751]: pgmap v3196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:57.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4537.733437 secs 2024-04-26T19:50:57.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4941 sec 2024-04-26T19:50:58.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4942 sec 2024-04-26T19:50:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:50:58 smithi119 ceph-mon[32752]: pgmap v3197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:59.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:50:58 smithi167 ceph-mon[32751]: pgmap v3197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:59.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:50:58 smithi090 ceph-mon[29180]: pgmap v3197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:50:59.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4943 sec 2024-04-26T19:51:00.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4944 sec 2024-04-26T19:51:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:00 smithi119 ceph-mon[32752]: pgmap v3198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:01.117 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:00 smithi119 ceph-mon[32752]: 5 slow requests, 1 included below; oldest blocked for > 4542.044905 secs 2024-04-26T19:51:01.125 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:00 smithi119 ceph-mon[32752]: slow request 724.090537 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:51:01.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:00 smithi090 ceph-mon[29180]: pgmap v3198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:01.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:00 smithi090 ceph-mon[29180]: 5 slow requests, 1 included below; oldest blocked for > 4542.044905 secs 2024-04-26T19:51:01.149 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:00 smithi090 ceph-mon[29180]: slow request 724.090537 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:51:01.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:00 smithi167 ceph-mon[32751]: pgmap v3198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:01.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:00 smithi167 ceph-mon[32751]: 5 slow requests, 1 included below; oldest blocked for > 4542.044905 secs 2024-04-26T19:51:01.174 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:00 smithi167 ceph-mon[32751]: slow request 724.090537 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:51:01.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4945 sec 2024-04-26T19:51:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4542.733509 secs 2024-04-26T19:51:02.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4542.733509 secs 2024-04-26T19:51:02.233 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4542.733509 secs 2024-04-26T19:51:02.329 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4946 sec 2024-04-26T19:51:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:02 smithi119 ceph-mon[32752]: pgmap v3199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:03.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:02 smithi090 ceph-mon[29180]: pgmap v3199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:03.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:02 smithi167 ceph-mon[32751]: pgmap v3199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:03.330 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4947 sec 2024-04-26T19:51:04.330 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4948 sec 2024-04-26T19:51:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:04 smithi119 ceph-mon[32752]: pgmap v3200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:05.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:04 smithi090 ceph-mon[29180]: pgmap v3200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:05.222 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:04 smithi167 ceph-mon[32751]: pgmap v3200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:05.330 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4949 sec 2024-04-26T19:51:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:05 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4547.044983 secs 2024-04-26T19:51:06.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:05 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4547.044983 secs 2024-04-26T19:51:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:05 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4547.044983 secs 2024-04-26T19:51:06.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4950 sec 2024-04-26T19:51:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:06 smithi167 ceph-mon[32751]: pgmap v3201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:07.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4547.733625 secs 2024-04-26T19:51:07.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:06 smithi090 ceph-mon[29180]: pgmap v3201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:07.265 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4547.733625 secs 2024-04-26T19:51:07.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:06 smithi119 ceph-mon[32752]: pgmap v3201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:07.438 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4547.733625 secs 2024-04-26T19:51:07.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4951 sec 2024-04-26T19:51:08.330 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4952 sec 2024-04-26T19:51:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:08 smithi167 ceph-mon[32751]: pgmap v3202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:09.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:08 smithi090 ceph-mon[29180]: pgmap v3202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:08 smithi119 ceph-mon[32752]: pgmap v3202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:09.336 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4953 sec 2024-04-26T19:51:10.330 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4954 sec 2024-04-26T19:51:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:10 smithi167 ceph-mon[32751]: pgmap v3203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:11.089 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:10 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4552.045065 secs 2024-04-26T19:51:11.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:10 smithi090 ceph-mon[29180]: pgmap v3203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:11.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:10 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4552.045065 secs 2024-04-26T19:51:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:10 smithi119 ceph-mon[32752]: pgmap v3203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:11.353 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:10 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4552.045065 secs 2024-04-26T19:51:11.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4955 sec 2024-04-26T19:51:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4552.733722 secs 2024-04-26T19:51:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4552.733722 secs 2024-04-26T19:51:12.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4956 sec 2024-04-26T19:51:12.405 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4552.733722 secs 2024-04-26T19:51:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:12 smithi167 ceph-mon[32751]: pgmap v3204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:12 smithi090 ceph-mon[29180]: pgmap v3204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:12 smithi119 ceph-mon[32752]: pgmap v3204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:13.331 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4957 sec 2024-04-26T19:51:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:51:14.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:51:14.346 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4958 sec 2024-04-26T19:51:14.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:51:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:14 smithi090 ceph-mon[29180]: pgmap v3205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:15.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:51:15.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:51:15.297 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:51:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:14 smithi119 ceph-mon[32752]: pgmap v3205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:15.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:51:15.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:51:15.427 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:51:15.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4959 sec 2024-04-26T19:51:15.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:14 smithi167 ceph-mon[32751]: pgmap v3205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:15.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:51:15.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:51:15.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:51:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:15 smithi090 ceph-mon[29180]: pgmap v3206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:51:16.289 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:15 smithi090 ceph-mon[29180]: pgmap v3207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:15 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4557.045128 secs 2024-04-26T19:51:16.442 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:15 smithi167 ceph-mon[32751]: pgmap v3206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:51:16.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:15 smithi167 ceph-mon[32751]: pgmap v3207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:16.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:15 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4557.045128 secs 2024-04-26T19:51:16.618 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:15 smithi119 ceph-mon[32752]: pgmap v3206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:51:16.669 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:15 smithi119 ceph-mon[32752]: pgmap v3207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:16.728 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:15 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4557.045128 secs 2024-04-26T19:51:16.869 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4960 sec 2024-04-26T19:51:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4557.733793 secs 2024-04-26T19:51:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4557.733793 secs 2024-04-26T19:51:17.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4557.733793 secs 2024-04-26T19:51:17.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4961 sec 2024-04-26T19:51:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:17 smithi090 ceph-mon[29180]: pgmap v3208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:17 smithi119 ceph-mon[32752]: pgmap v3208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:17 smithi167 ceph-mon[32751]: pgmap v3208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:18.331 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4962 sec 2024-04-26T19:51:19.331 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4963 sec 2024-04-26T19:51:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:19 smithi090 ceph-mon[29180]: pgmap v3209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:51:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:19 smithi119 ceph-mon[32752]: pgmap v3209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:51:20.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4964 sec 2024-04-26T19:51:20.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:19 smithi167 ceph-mon[32751]: pgmap v3209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:51:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:20 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4562.045187 secs 2024-04-26T19:51:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:20 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4562.045187 secs 2024-04-26T19:51:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:20 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4562.045187 secs 2024-04-26T19:51:21.331 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4965 sec 2024-04-26T19:51:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:21 smithi090 ceph-mon[29180]: pgmap v3210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:22.194 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4562.733939 secs 2024-04-26T19:51:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:21 smithi119 ceph-mon[32752]: pgmap v3210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:22.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4562.733939 secs 2024-04-26T19:51:22.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4966 sec 2024-04-26T19:51:22.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:21 smithi167 ceph-mon[32751]: pgmap v3210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:51:22.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4562.733939 secs 2024-04-26T19:51:23.332 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4967 sec 2024-04-26T19:51:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:23 smithi090 ceph-mon[29180]: pgmap v3211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:51:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:23 smithi119 ceph-mon[32752]: pgmap v3211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:51:24.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:23 smithi167 ceph-mon[32751]: pgmap v3211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:51:24.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4968 sec 2024-04-26T19:51:25.332 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4969 sec 2024-04-26T19:51:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:25 smithi090 ceph-mon[29180]: pgmap v3212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:51:26.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:25 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4567.045248 secs 2024-04-26T19:51:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:25 smithi119 ceph-mon[32752]: pgmap v3212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:51:26.369 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:25 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4567.045248 secs 2024-04-26T19:51:26.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4970 sec 2024-04-26T19:51:26.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:25 smithi167 ceph-mon[32751]: pgmap v3212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:51:26.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:25 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4567.045248 secs 2024-04-26T19:51:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4567.734045 secs 2024-04-26T19:51:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4567.734045 secs 2024-04-26T19:51:27.342 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4971 sec 2024-04-26T19:51:27.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4567.734045 secs 2024-04-26T19:51:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:27 smithi090 ceph-mon[29180]: pgmap v3213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:27 smithi119 ceph-mon[32752]: pgmap v3213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:28.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4972 sec 2024-04-26T19:51:28.453 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:27 smithi167 ceph-mon[32751]: pgmap v3213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:29.332 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4973 sec 2024-04-26T19:51:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:29 smithi090 ceph-mon[29180]: pgmap v3214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:29 smithi119 ceph-mon[32752]: pgmap v3214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:30.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:29 smithi167 ceph-mon[32751]: pgmap v3214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:30.340 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4974 sec 2024-04-26T19:51:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:30 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4572.045331 secs 2024-04-26T19:51:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:30 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4572.045331 secs 2024-04-26T19:51:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:30 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4572.045331 secs 2024-04-26T19:51:31.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4975 sec 2024-04-26T19:51:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:31 smithi090 ceph-mon[29180]: pgmap v3215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:32.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4572.734158 secs 2024-04-26T19:51:32.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:31 smithi167 ceph-mon[32751]: pgmap v3215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:32.385 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4572.734158 secs 2024-04-26T19:51:32.461 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:31 smithi119 ceph-mon[32752]: pgmap v3215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:32.641 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4572.734158 secs 2024-04-26T19:51:32.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4976 sec 2024-04-26T19:51:33.332 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4977 sec 2024-04-26T19:51:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:33 smithi119 ceph-mon[32752]: pgmap v3216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:34.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4978 sec 2024-04-26T19:51:34.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:33 smithi167 ceph-mon[32751]: pgmap v3216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:34.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:33 smithi090 ceph-mon[29180]: pgmap v3216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:35.333 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4979 sec 2024-04-26T19:51:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:35 smithi119 ceph-mon[32752]: pgmap v3217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:36.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:35 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4577.045412 secs 2024-04-26T19:51:36.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4980 sec 2024-04-26T19:51:36.457 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:35 smithi167 ceph-mon[32751]: pgmap v3217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:36.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:35 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4577.045412 secs 2024-04-26T19:51:36.594 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:35 smithi090 ceph-mon[29180]: pgmap v3217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:36.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:35 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4577.045412 secs 2024-04-26T19:51:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4577.734271 secs 2024-04-26T19:51:37.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4981 sec 2024-04-26T19:51:37.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4577.734271 secs 2024-04-26T19:51:37.513 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4577.734271 secs 2024-04-26T19:51:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:37 smithi119 ceph-mon[32752]: pgmap v3218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:38.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:37 smithi167 ceph-mon[32751]: pgmap v3218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:38.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4982 sec 2024-04-26T19:51:38.443 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:37 smithi090 ceph-mon[29180]: pgmap v3218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:39.333 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4983 sec 2024-04-26T19:51:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:39 smithi090 ceph-mon[29180]: pgmap v3219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:39 smithi119 ceph-mon[32752]: pgmap v3219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:40.333 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4984 sec 2024-04-26T19:51:40.358 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:39 smithi167 ceph-mon[32751]: pgmap v3219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:40 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4582.045494 secs 2024-04-26T19:51:41.336 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4985 sec 2024-04-26T19:51:41.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:40 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4582.045494 secs 2024-04-26T19:51:41.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:40 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4582.045494 secs 2024-04-26T19:51:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:41 smithi119 ceph-mon[32752]: pgmap v3220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:42.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4582.734397 secs 2024-04-26T19:51:42.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4986 sec 2024-04-26T19:51:42.440 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:41 smithi167 ceph-mon[32751]: pgmap v3220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:42.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4582.734397 secs 2024-04-26T19:51:42.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:41 smithi090 ceph-mon[29180]: pgmap v3220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:42.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4582.734397 secs 2024-04-26T19:51:43.334 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4987 sec 2024-04-26T19:51:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:43 smithi119 ceph-mon[32752]: pgmap v3221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:44.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4988 sec 2024-04-26T19:51:44.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:43 smithi167 ceph-mon[32751]: pgmap v3221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:44.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:43 smithi090 ceph-mon[29180]: pgmap v3221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:45.334 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4989 sec 2024-04-26T19:51:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:45 smithi119 ceph-mon[32752]: pgmap v3222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:46.364 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:45 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4587.045580 secs 2024-04-26T19:51:46.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4990 sec 2024-04-26T19:51:46.452 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:45 smithi167 ceph-mon[32751]: pgmap v3222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:46.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:45 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4587.045580 secs 2024-04-26T19:51:46.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:45 smithi090 ceph-mon[29180]: pgmap v3222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:46.482 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:45 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4587.045580 secs 2024-04-26T19:51:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4587.734467 secs 2024-04-26T19:51:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4587.734467 secs 2024-04-26T19:51:47.334 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4991 sec 2024-04-26T19:51:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4587.734467 secs 2024-04-26T19:51:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:47 smithi119 ceph-mon[32752]: pgmap v3223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:48.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4992 sec 2024-04-26T19:51:48.684 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:47 smithi167 ceph-mon[32751]: pgmap v3223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:48.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:47 smithi090 ceph-mon[29180]: pgmap v3223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:49.334 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4993 sec 2024-04-26T19:51:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:49 smithi119 ceph-mon[32752]: pgmap v3224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:50.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4994 sec 2024-04-26T19:51:50.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:49 smithi167 ceph-mon[32751]: pgmap v3224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:50.478 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:49 smithi090 ceph-mon[29180]: pgmap v3224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:50 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4592.045663 secs 2024-04-26T19:51:51.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4995 sec 2024-04-26T19:51:51.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:50 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4592.045663 secs 2024-04-26T19:51:51.434 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:50 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4592.045663 secs 2024-04-26T19:51:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:51 smithi119 ceph-mon[32752]: pgmap v3225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:52.353 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4592.734582 secs 2024-04-26T19:51:52.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4996 sec 2024-04-26T19:51:52.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:51 smithi167 ceph-mon[32751]: pgmap v3225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:52.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4592.734582 secs 2024-04-26T19:51:52.510 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:51 smithi090 ceph-mon[29180]: pgmap v3225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:52.520 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4592.734582 secs 2024-04-26T19:51:53.334 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4997 sec 2024-04-26T19:51:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:53 smithi119 ceph-mon[32752]: pgmap v3226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:54.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4998 sec 2024-04-26T19:51:54.390 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:53 smithi167 ceph-mon[32751]: pgmap v3226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:54.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:53 smithi090 ceph-mon[29180]: pgmap v3226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:55.335 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 4999 sec 2024-04-26T19:51:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:55 smithi119 ceph-mon[32752]: pgmap v3227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:56.337 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:55 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4597.045745 secs 2024-04-26T19:51:56.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5000 sec 2024-04-26T19:51:56.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:55 smithi167 ceph-mon[32751]: pgmap v3227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:56.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:55 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4597.045745 secs 2024-04-26T19:51:56.485 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:55 smithi090 ceph-mon[29180]: pgmap v3227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:51:56.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:55 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4597.045745 secs 2024-04-26T19:51:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4597.734705 secs 2024-04-26T19:51:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4597.734705 secs 2024-04-26T19:51:57.337 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5001 sec 2024-04-26T19:51:57.405 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4597.734705 secs 2024-04-26T19:51:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:57 smithi119 ceph-mon[32752]: pgmap v3228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:58.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5002 sec 2024-04-26T19:51:58.406 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:57 smithi167 ceph-mon[32751]: pgmap v3228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:58.438 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:57 smithi090 ceph-mon[29180]: pgmap v3228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:51:59.335 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5003 sec 2024-04-26T19:52:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:51:59 smithi119 ceph-mon[32752]: pgmap v3229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:00.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5004 sec 2024-04-26T19:52:00.397 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:51:59 smithi167 ceph-mon[32751]: pgmap v3229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:00.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:51:59 smithi090 ceph-mon[29180]: pgmap v3229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:01.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:00 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4602.045804 secs 2024-04-26T19:52:01.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:00 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4602.045804 secs 2024-04-26T19:52:01.395 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:00 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4602.045804 secs 2024-04-26T19:52:01.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5005 sec 2024-04-26T19:52:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:01 smithi119 ceph-mon[32752]: pgmap v3230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:02.381 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4602.734784 secs 2024-04-26T19:52:02.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5006 sec 2024-04-26T19:52:02.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:01 smithi167 ceph-mon[32751]: pgmap v3230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:02.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4602.734784 secs 2024-04-26T19:52:02.545 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:01 smithi090 ceph-mon[29180]: pgmap v3230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:02.596 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4602.734784 secs 2024-04-26T19:52:03.335 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5007 sec 2024-04-26T19:52:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:03 smithi119 ceph-mon[32752]: pgmap v3231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:04.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5008 sec 2024-04-26T19:52:04.442 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:03 smithi167 ceph-mon[32751]: pgmap v3231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:04.507 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:03 smithi090 ceph-mon[29180]: pgmap v3231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:05.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5009 sec 2024-04-26T19:52:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:05 smithi119 ceph-mon[32752]: pgmap v3232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:06.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:05 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4607.045890 secs 2024-04-26T19:52:06.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5010 sec 2024-04-26T19:52:06.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:05 smithi167 ceph-mon[32751]: pgmap v3232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:06.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:05 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4607.045890 secs 2024-04-26T19:52:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:05 smithi090 ceph-mon[29180]: pgmap v3232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:06.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:05 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4607.045890 secs 2024-04-26T19:52:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4607.734920 secs 2024-04-26T19:52:07.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5011 sec 2024-04-26T19:52:07.440 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4607.734920 secs 2024-04-26T19:52:07.499 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4607.734920 secs 2024-04-26T19:52:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:07 smithi119 ceph-mon[32752]: pgmap v3233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:08.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5012 sec 2024-04-26T19:52:08.375 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:07 smithi167 ceph-mon[32751]: pgmap v3233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:08.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:07 smithi090 ceph-mon[29180]: pgmap v3233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:09.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5013 sec 2024-04-26T19:52:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:09 smithi119 ceph-mon[32752]: pgmap v3234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:10.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5014 sec 2024-04-26T19:52:10.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:09 smithi167 ceph-mon[32751]: pgmap v3234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:10.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:09 smithi090 ceph-mon[29180]: pgmap v3234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:10 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4612.045975 secs 2024-04-26T19:52:11.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5015 sec 2024-04-26T19:52:11.390 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:10 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4612.045975 secs 2024-04-26T19:52:11.439 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:10 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4612.045975 secs 2024-04-26T19:52:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:11 smithi119 ceph-mon[32752]: pgmap v3235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:12.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4612.735018 secs 2024-04-26T19:52:12.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5016 sec 2024-04-26T19:52:12.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:11 smithi167 ceph-mon[32751]: pgmap v3235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:12.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4612.735018 secs 2024-04-26T19:52:12.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:11 smithi090 ceph-mon[29180]: pgmap v3235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:12.532 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4612.735018 secs 2024-04-26T19:52:13.336 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5017 sec 2024-04-26T19:52:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:13 smithi119 ceph-mon[32752]: pgmap v3236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:14.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5018 sec 2024-04-26T19:52:14.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:13 smithi167 ceph-mon[32751]: pgmap v3236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:14.459 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:13 smithi090 ceph-mon[29180]: pgmap v3236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:52:15.371 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:52:15.429 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:52:15.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:52:15.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5019 sec 2024-04-26T19:52:15.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:52:15.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:52:15.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:52:15.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:52:15.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:52:15.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:52:15.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:52:15.995 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:52:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:15 smithi119 ceph-mon[32752]: pgmap v3237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:16.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:15 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4617.046084 secs 2024-04-26T19:52:16.475 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:15 smithi119 ceph-mon[32752]: pgmap v3238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:52:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:15 smithi119 ceph-mon[32752]: pgmap v3239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:16.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5020 sec 2024-04-26T19:52:16.598 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:15 smithi167 ceph-mon[32751]: pgmap v3237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:16.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:15 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4617.046084 secs 2024-04-26T19:52:16.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:15 smithi167 ceph-mon[32751]: pgmap v3238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:52:16.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:15 smithi167 ceph-mon[32751]: pgmap v3239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:16.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:15 smithi090 ceph-mon[29180]: pgmap v3237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:15 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4617.046084 secs 2024-04-26T19:52:16.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:15 smithi090 ceph-mon[29180]: pgmap v3238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:52:16.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:15 smithi090 ceph-mon[29180]: pgmap v3239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4617.735133 secs 2024-04-26T19:52:17.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5021 sec 2024-04-26T19:52:17.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4617.735133 secs 2024-04-26T19:52:17.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4617.735133 secs 2024-04-26T19:52:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:17 smithi119 ceph-mon[32752]: pgmap v3240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:18.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5022 sec 2024-04-26T19:52:18.440 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:17 smithi167 ceph-mon[32751]: pgmap v3240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:18.537 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:17 smithi090 ceph-mon[29180]: pgmap v3240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:19.337 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5023 sec 2024-04-26T19:52:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:19 smithi119 ceph-mon[32752]: pgmap v3241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:19 smithi167 ceph-mon[32751]: pgmap v3241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:20.337 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5024 sec 2024-04-26T19:52:20.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:19 smithi090 ceph-mon[29180]: pgmap v3241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:20 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4622.046163 secs 2024-04-26T19:52:21.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5025 sec 2024-04-26T19:52:21.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:20 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4622.046163 secs 2024-04-26T19:52:21.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:20 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4622.046163 secs 2024-04-26T19:52:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4622.735229 secs 2024-04-26T19:52:22.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:22 smithi119 ceph-mon[32752]: pgmap v3242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:22.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5026 sec 2024-04-26T19:52:22.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4622.735229 secs 2024-04-26T19:52:22.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:22 smithi167 ceph-mon[32751]: pgmap v3242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:22.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4622.735229 secs 2024-04-26T19:52:22.658 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:22 smithi090 ceph-mon[29180]: pgmap v3242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:52:23.337 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5027 sec 2024-04-26T19:52:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:24 smithi119 ceph-mon[32752]: pgmap v3243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:52:24.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5028 sec 2024-04-26T19:52:24.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:24 smithi167 ceph-mon[32751]: pgmap v3243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:52:24.498 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:24 smithi090 ceph-mon[29180]: pgmap v3243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:52:25.337 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5029 sec 2024-04-26T19:52:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:26 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4627.046248 secs 2024-04-26T19:52:26.145 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:26 smithi090 ceph-mon[29180]: pgmap v3244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:52:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:26 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4627.046248 secs 2024-04-26T19:52:26.422 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:26 smithi119 ceph-mon[32752]: pgmap v3244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:52:26.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5030 sec 2024-04-26T19:52:26.515 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:26 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4627.046248 secs 2024-04-26T19:52:26.524 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:26 smithi167 ceph-mon[32751]: pgmap v3244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:52:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4627.735417 secs 2024-04-26T19:52:27.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5031 sec 2024-04-26T19:52:27.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4627.735417 secs 2024-04-26T19:52:27.517 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4627.735417 secs 2024-04-26T19:52:28.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:28 smithi119 ceph-mon[32752]: pgmap v3245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:28.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5032 sec 2024-04-26T19:52:28.435 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:28 smithi167 ceph-mon[32751]: pgmap v3245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:28.491 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:28 smithi090 ceph-mon[29180]: pgmap v3245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:29.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5033 sec 2024-04-26T19:52:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:30 smithi119 ceph-mon[32752]: pgmap v3246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:30.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5034 sec 2024-04-26T19:52:30.425 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:30 smithi167 ceph-mon[32751]: pgmap v3246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:30.468 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:30 smithi090 ceph-mon[29180]: pgmap v3246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:31.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:31 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4632.046346 secs 2024-04-26T19:52:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:31 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4632.046346 secs 2024-04-26T19:52:31.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5035 sec 2024-04-26T19:52:31.405 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:31 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4632.046346 secs 2024-04-26T19:52:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4632.735483 secs 2024-04-26T19:52:32.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:32 smithi119 ceph-mon[32752]: pgmap v3247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4632.735483 secs 2024-04-26T19:52:32.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:32 smithi167 ceph-mon[32751]: pgmap v3247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:32.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5036 sec 2024-04-26T19:52:32.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4632.735483 secs 2024-04-26T19:52:32.498 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:32 smithi090 ceph-mon[29180]: pgmap v3247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:33.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5037 sec 2024-04-26T19:52:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:34 smithi119 ceph-mon[32752]: pgmap v3248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:34.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:34 smithi167 ceph-mon[32751]: pgmap v3248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:34.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5038 sec 2024-04-26T19:52:34.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:34 smithi090 ceph-mon[29180]: pgmap v3248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:35.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5039 sec 2024-04-26T19:52:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:36 smithi119 ceph-mon[32752]: 5 slow requests, 1 included below; oldest blocked for > 4637.046431 secs 2024-04-26T19:52:36.415 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:36 smithi119 ceph-mon[32752]: slow request 1443.183122 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:52:36.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:36 smithi119 ceph-mon[32752]: pgmap v3249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:36.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5040 sec 2024-04-26T19:52:36.624 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:36 smithi167 ceph-mon[32751]: 5 slow requests, 1 included below; oldest blocked for > 4637.046431 secs 2024-04-26T19:52:36.640 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:36 smithi167 ceph-mon[32751]: slow request 1443.183122 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:52:36.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:36 smithi167 ceph-mon[32751]: pgmap v3249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:36.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:36 smithi090 ceph-mon[29180]: 5 slow requests, 1 included below; oldest blocked for > 4637.046431 secs 2024-04-26T19:52:36.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:36 smithi090 ceph-mon[29180]: slow request 1443.183122 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:52:36.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:36 smithi090 ceph-mon[29180]: pgmap v3249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4637.735610 secs 2024-04-26T19:52:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4637.735610 secs 2024-04-26T19:52:37.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5041 sec 2024-04-26T19:52:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4637.735610 secs 2024-04-26T19:52:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:38 smithi119 ceph-mon[32752]: pgmap v3250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:38 smithi167 ceph-mon[32751]: pgmap v3250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:38.338 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5042 sec 2024-04-26T19:52:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:38 smithi090 ceph-mon[29180]: pgmap v3250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:39.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5043 sec 2024-04-26T19:52:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:39 smithi119 ceph-mon[32752]: pgmap v3251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:40.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:39 smithi090 ceph-mon[29180]: pgmap v3251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:40.161 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:39 smithi167 ceph-mon[32751]: pgmap v3251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:40.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5044 sec 2024-04-26T19:52:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:40 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4642.046514 secs 2024-04-26T19:52:41.140 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:40 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4642.046514 secs 2024-04-26T19:52:41.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:40 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4642.046514 secs 2024-04-26T19:52:41.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5045 sec 2024-04-26T19:52:42.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4642.735737 secs 2024-04-26T19:52:42.091 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:41 smithi090 ceph-mon[29180]: pgmap v3252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:42.133 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4642.735737 secs 2024-04-26T19:52:42.275 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:41 smithi167 ceph-mon[32751]: pgmap v3252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:42.334 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4642.735737 secs 2024-04-26T19:52:42.410 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:41 smithi119 ceph-mon[32752]: pgmap v3252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:42.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5046 sec 2024-04-26T19:52:43.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5047 sec 2024-04-26T19:52:44.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:43 smithi119 ceph-mon[32752]: pgmap v3253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:44.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5048 sec 2024-04-26T19:52:44.426 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:43 smithi167 ceph-mon[32751]: pgmap v3253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:44.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:43 smithi090 ceph-mon[29180]: pgmap v3253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:45.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5049 sec 2024-04-26T19:52:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:46 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4647.046593 secs 2024-04-26T19:52:46.375 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:46 smithi119 ceph-mon[32752]: pgmap v3254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:46.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5050 sec 2024-04-26T19:52:46.489 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:46 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4647.046593 secs 2024-04-26T19:52:46.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:46 smithi167 ceph-mon[32751]: pgmap v3254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:46.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:46 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4647.046593 secs 2024-04-26T19:52:46.652 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:46 smithi090 ceph-mon[29180]: pgmap v3254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4647.735861 secs 2024-04-26T19:52:47.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5051 sec 2024-04-26T19:52:47.437 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4647.735861 secs 2024-04-26T19:52:47.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4647.735861 secs 2024-04-26T19:52:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:48 smithi119 ceph-mon[32752]: pgmap v3255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:48.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5052 sec 2024-04-26T19:52:48.523 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:48 smithi167 ceph-mon[32751]: pgmap v3255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:48.533 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:48 smithi090 ceph-mon[29180]: pgmap v3255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:49.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5053 sec 2024-04-26T19:52:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:49 smithi119 ceph-mon[32752]: pgmap v3256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:50.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:49 smithi167 ceph-mon[32751]: pgmap v3256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:50.192 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:49 smithi090 ceph-mon[29180]: pgmap v3256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:50.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5054 sec 2024-04-26T19:52:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:50 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4652.046675 secs 2024-04-26T19:52:51.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:50 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4652.046675 secs 2024-04-26T19:52:51.166 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:50 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4652.046675 secs 2024-04-26T19:52:51.339 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5055 sec 2024-04-26T19:52:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4652.735980 secs 2024-04-26T19:52:52.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:51 smithi119 ceph-mon[32752]: pgmap v3257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:52.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4652.735980 secs 2024-04-26T19:52:52.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:51 smithi090 ceph-mon[29180]: pgmap v3257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4652.735980 secs 2024-04-26T19:52:52.258 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:51 smithi167 ceph-mon[32751]: pgmap v3257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:52.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5056 sec 2024-04-26T19:52:53.340 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5057 sec 2024-04-26T19:52:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:54 smithi119 ceph-mon[32752]: pgmap v3258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:54.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5058 sec 2024-04-26T19:52:54.416 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:54 smithi167 ceph-mon[32751]: pgmap v3258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:54.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:54 smithi090 ceph-mon[29180]: pgmap v3258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:55.340 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5059 sec 2024-04-26T19:52:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:56 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4657.046752 secs 2024-04-26T19:52:56.324 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:56 smithi119 ceph-mon[32752]: pgmap v3259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:56.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5060 sec 2024-04-26T19:52:56.389 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:56 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4657.046752 secs 2024-04-26T19:52:56.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:56 smithi167 ceph-mon[32751]: pgmap v3259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:56.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:56 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4657.046752 secs 2024-04-26T19:52:56.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:56 smithi090 ceph-mon[29180]: pgmap v3259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:52:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4657.736077 secs 2024-04-26T19:52:57.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5061 sec 2024-04-26T19:52:57.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4657.736077 secs 2024-04-26T19:52:57.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4657.736077 secs 2024-04-26T19:52:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:52:58 smithi119 ceph-mon[32752]: pgmap v3260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:58.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5062 sec 2024-04-26T19:52:58.383 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:52:58 smithi167 ceph-mon[32751]: pgmap v3260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:58.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:52:58 smithi090 ceph-mon[29180]: pgmap v3260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:52:59.340 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5063 sec 2024-04-26T19:53:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:00 smithi119 ceph-mon[32752]: pgmap v3261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:00.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5064 sec 2024-04-26T19:53:00.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:00 smithi167 ceph-mon[32751]: pgmap v3261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:00 smithi090 ceph-mon[29180]: pgmap v3261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:01.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:01 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 4662.046828 secs 2024-04-26T19:53:01.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:01 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 4662.046828 secs 2024-04-26T19:53:01.397 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:01 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 4662.046828 secs 2024-04-26T19:53:01.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5065 sec 2024-04-26T19:53:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4662.736215 secs 2024-04-26T19:53:02.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:02 smithi119 ceph-mon[32752]: pgmap v3262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:02.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4662.736215 secs 2024-04-26T19:53:02.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:02 smithi167 ceph-mon[32751]: pgmap v3262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:02.344 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5066 sec 2024-04-26T19:53:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4662.736215 secs 2024-04-26T19:53:02.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:02 smithi090 ceph-mon[29180]: pgmap v3262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:03.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5067 sec 2024-04-26T19:53:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:04 smithi119 ceph-mon[32752]: pgmap v3263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:04.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5068 sec 2024-04-26T19:53:04.400 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:04 smithi167 ceph-mon[32751]: pgmap v3263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:04.496 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:04 smithi090 ceph-mon[29180]: pgmap v3263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:05.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5069 sec 2024-04-26T19:53:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:06 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 4667.046918 secs 2024-04-26T19:53:06.380 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:06 smithi119 ceph-mon[32752]: slow request 182.828360 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:53:06.483 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:06 smithi119 ceph-mon[32752]: pgmap v3264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:06.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5070 sec 2024-04-26T19:53:06.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:06 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 4667.046918 secs 2024-04-26T19:53:06.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:06 smithi167 ceph-mon[32751]: slow request 182.828360 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:53:06.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:06 smithi167 ceph-mon[32751]: pgmap v3264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:06.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:06 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 4667.046918 secs 2024-04-26T19:53:06.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:06 smithi090 ceph-mon[29180]: slow request 182.828360 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T19:53:06.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:06 smithi090 ceph-mon[29180]: pgmap v3264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4667.736336 secs 2024-04-26T19:53:07.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:53:07.441 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:53:07.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5071 sec 2024-04-26T19:53:07.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4667.736336 secs 2024-04-26T19:53:07.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:53:07.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:53:07.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4667.736336 secs 2024-04-26T19:53:07.728 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T19:53:07.787 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T19:53:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:08 smithi119 ceph-mon[32752]: pgmap v3265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:08.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5072 sec 2024-04-26T19:53:08.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:08 smithi167 ceph-mon[32751]: pgmap v3265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:08 smithi090 ceph-mon[29180]: pgmap v3265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:09.342 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5073 sec 2024-04-26T19:53:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:10 smithi119 ceph-mon[32752]: pgmap v3266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:10.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5074 sec 2024-04-26T19:53:10.398 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:10 smithi167 ceph-mon[32751]: pgmap v3266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:10.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:10 smithi090 ceph-mon[29180]: pgmap v3266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4672.046996 secs 2024-04-26T19:53:11.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5075 sec 2024-04-26T19:53:11.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4672.046996 secs 2024-04-26T19:53:11.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4672.046996 secs 2024-04-26T19:53:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4672.736458 secs 2024-04-26T19:53:12.333 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:12 smithi119 ceph-mon[32752]: pgmap v3267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:12.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5076 sec 2024-04-26T19:53:12.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4672.736458 secs 2024-04-26T19:53:12.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:12 smithi167 ceph-mon[32751]: pgmap v3267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:12.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4672.736458 secs 2024-04-26T19:53:12.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:12 smithi090 ceph-mon[29180]: pgmap v3267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:13.342 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5077 sec 2024-04-26T19:53:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:14 smithi167 ceph-mon[32751]: pgmap v3268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:14.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5078 sec 2024-04-26T19:53:14.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:14 smithi090 ceph-mon[29180]: pgmap v3268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:14 smithi119 ceph-mon[32752]: pgmap v3268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:15.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:53:15.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5079 sec 2024-04-26T19:53:15.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:53:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:53:16.342 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5080 sec 2024-04-26T19:53:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4677.047099 secs 2024-04-26T19:53:16.411 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:16 smithi090 ceph-mon[29180]: pgmap v3269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:16.465 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:53:16.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:53:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:53:16.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4677.047099 secs 2024-04-26T19:53:16.680 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:16 smithi167 ceph-mon[32751]: pgmap v3269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:16.688 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:53:16.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:53:16.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:53:16.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4677.047099 secs 2024-04-26T19:53:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:16 smithi119 ceph-mon[32752]: pgmap v3269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:17.075 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:53:17.114 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:53:17.150 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:53:17.342 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5081 sec 2024-04-26T19:53:17.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4677.736522 secs 2024-04-26T19:53:17.428 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:17 smithi090 ceph-mon[29180]: pgmap v3270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:53:17.443 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:17 smithi090 ceph-mon[29180]: pgmap v3271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4677.736522 secs 2024-04-26T19:53:17.583 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:17 smithi119 ceph-mon[32752]: pgmap v3270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:53:17.612 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:17 smithi119 ceph-mon[32752]: pgmap v3271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:17.673 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4677.736522 secs 2024-04-26T19:53:17.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:17 smithi167 ceph-mon[32751]: pgmap v3270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T19:53:17.766 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:17 smithi167 ceph-mon[32751]: pgmap v3271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:18.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5082 sec 2024-04-26T19:53:19.342 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5083 sec 2024-04-26T19:53:19.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:19 smithi090 ceph-mon[29180]: pgmap v3272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:53:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:19 smithi119 ceph-mon[32752]: pgmap v3272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:53:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:19 smithi167 ceph-mon[32751]: pgmap v3272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:53:20.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5084 sec 2024-04-26T19:53:20.390 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:20 smithi090 ceph-mon[29180]: pgmap v3273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:20 smithi119 ceph-mon[32752]: pgmap v3273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:20.611 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:20 smithi167 ceph-mon[32751]: pgmap v3273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:21.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5085 sec 2024-04-26T19:53:21.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4682.047180 secs 2024-04-26T19:53:21.447 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4682.736602 secs 2024-04-26T19:53:21.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4682.047180 secs 2024-04-26T19:53:21.660 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4682.736602 secs 2024-04-26T19:53:21.686 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4682.047180 secs 2024-04-26T19:53:21.694 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4682.736602 secs 2024-04-26T19:53:22.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5086 sec 2024-04-26T19:53:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:22 smithi090 ceph-mon[29180]: pgmap v3274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:22 smithi119 ceph-mon[32752]: pgmap v3274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:22.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:22 smithi167 ceph-mon[32751]: pgmap v3274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:23.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5087 sec 2024-04-26T19:53:24.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5088 sec 2024-04-26T19:53:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:24 smithi119 ceph-mon[32752]: pgmap v3275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:25.115 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:24 smithi167 ceph-mon[32751]: pgmap v3275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:25.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:24 smithi090 ceph-mon[29180]: pgmap v3275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:53:25.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5089 sec 2024-04-26T19:53:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4687.047273 secs 2024-04-26T19:53:26.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4687.047273 secs 2024-04-26T19:53:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4687.047273 secs 2024-04-26T19:53:26.346 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5090 sec 2024-04-26T19:53:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4687.736699 secs 2024-04-26T19:53:27.148 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:26 smithi119 ceph-mon[32752]: pgmap v3276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:53:27.197 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4687.736699 secs 2024-04-26T19:53:27.207 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:26 smithi090 ceph-mon[29180]: pgmap v3276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:53:27.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4687.736699 secs 2024-04-26T19:53:27.405 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:26 smithi167 ceph-mon[32751]: pgmap v3276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:53:27.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5091 sec 2024-04-26T19:53:28.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5092 sec 2024-04-26T19:53:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:28 smithi119 ceph-mon[32752]: pgmap v3277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:29.113 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:28 smithi167 ceph-mon[32751]: pgmap v3277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:29.162 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:53:29.084+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:53:29.214 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:53:29.084+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:53:29.230 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:53:29.085+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 40 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:53:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:53:29.085+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 40 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:53:29.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:28 smithi090 ceph-mon[29180]: pgmap v3277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:29.343 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5093 sec 2024-04-26T19:53:30.344 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5094 sec 2024-04-26T19:53:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:30 smithi119 ceph-mon[32752]: pgmap v3278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:31.135 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4692.047332 secs 2024-04-26T19:53:31.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:30 smithi090 ceph-mon[29180]: pgmap v3278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:31.223 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4692.047332 secs 2024-04-26T19:53:31.285 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:30 smithi167 ceph-mon[32751]: pgmap v3278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:31.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4692.047332 secs 2024-04-26T19:53:31.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5095 sec 2024-04-26T19:53:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4692.736787 secs 2024-04-26T19:53:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4692.736787 secs 2024-04-26T19:53:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4692.736787 secs 2024-04-26T19:53:32.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5096 sec 2024-04-26T19:53:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:32 smithi167 ceph-mon[32751]: pgmap v3279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:33.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:32 smithi090 ceph-mon[29180]: pgmap v3279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:32 smithi119 ceph-mon[32752]: pgmap v3279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:33.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5097 sec 2024-04-26T19:53:34.346 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5098 sec 2024-04-26T19:53:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:34 smithi167 ceph-mon[32751]: pgmap v3280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:34 smithi090 ceph-mon[29180]: pgmap v3280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:34 smithi119 ceph-mon[32752]: pgmap v3280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:35.344 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5099 sec 2024-04-26T19:53:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4697.047414 secs 2024-04-26T19:53:36.225 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4697.047414 secs 2024-04-26T19:53:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4697.047414 secs 2024-04-26T19:53:36.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5100 sec 2024-04-26T19:53:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4697.736909 secs 2024-04-26T19:53:37.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:36 smithi167 ceph-mon[32751]: pgmap v3281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4697.736909 secs 2024-04-26T19:53:37.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:36 smithi090 ceph-mon[29180]: pgmap v3281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4697.736909 secs 2024-04-26T19:53:37.363 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:36 smithi119 ceph-mon[32752]: pgmap v3281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:37.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5101 sec 2024-04-26T19:53:38.344 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5102 sec 2024-04-26T19:53:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:38 smithi090 ceph-mon[29180]: pgmap v3282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:38 smithi119 ceph-mon[32752]: pgmap v3282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:39.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5103 sec 2024-04-26T19:53:39.437 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:38 smithi167 ceph-mon[32751]: pgmap v3282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:40.345 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5104 sec 2024-04-26T19:53:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:40 smithi090 ceph-mon[29180]: pgmap v3283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:41.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4702.047496 secs 2024-04-26T19:53:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:40 smithi119 ceph-mon[32752]: pgmap v3283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:41.351 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4702.047496 secs 2024-04-26T19:53:41.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5105 sec 2024-04-26T19:53:41.433 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:40 smithi167 ceph-mon[32751]: pgmap v3283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:41.441 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4702.047496 secs 2024-04-26T19:53:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4702.737044 secs 2024-04-26T19:53:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4702.737044 secs 2024-04-26T19:53:42.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5106 sec 2024-04-26T19:53:42.516 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4702.737044 secs 2024-04-26T19:53:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:42 smithi090 ceph-mon[29180]: pgmap v3284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:42 smithi119 ceph-mon[32752]: pgmap v3284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:42 smithi167 ceph-mon[32751]: pgmap v3284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:43.348 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5107 sec 2024-04-26T19:53:44.345 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5108 sec 2024-04-26T19:53:45.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:44 smithi119 ceph-mon[32752]: pgmap v3285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:45.242 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:44 smithi090 ceph-mon[29180]: pgmap v3285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:44 smithi167 ceph-mon[32751]: pgmap v3285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:45.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5109 sec 2024-04-26T19:53:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4707.047587 secs 2024-04-26T19:53:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4707.047587 secs 2024-04-26T19:53:46.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5110 sec 2024-04-26T19:53:46.396 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4707.047587 secs 2024-04-26T19:53:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4707.737468 secs 2024-04-26T19:53:47.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:46 smithi090 ceph-mon[29180]: pgmap v3286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4707.737468 secs 2024-04-26T19:53:47.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:46 smithi119 ceph-mon[32752]: pgmap v3286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:47.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5111 sec 2024-04-26T19:53:47.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4707.737468 secs 2024-04-26T19:53:47.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:46 smithi167 ceph-mon[32751]: pgmap v3286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:48.345 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5112 sec 2024-04-26T19:53:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:48 smithi090 ceph-mon[29180]: pgmap v3287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:48 smithi119 ceph-mon[32752]: pgmap v3287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:49.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:48 smithi167 ceph-mon[32751]: pgmap v3287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:49.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5113 sec 2024-04-26T19:53:50.345 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5114 sec 2024-04-26T19:53:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:50 smithi090 ceph-mon[29180]: pgmap v3288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:51.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4712.047644 secs 2024-04-26T19:53:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:50 smithi119 ceph-mon[32752]: pgmap v3288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:51.411 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4712.047644 secs 2024-04-26T19:53:51.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5115 sec 2024-04-26T19:53:51.661 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:50 smithi167 ceph-mon[32751]: pgmap v3288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:51.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4712.047644 secs 2024-04-26T19:53:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4712.737581 secs 2024-04-26T19:53:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4712.737581 secs 2024-04-26T19:53:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4712.737581 secs 2024-04-26T19:53:52.345 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5116 sec 2024-04-26T19:53:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:52 smithi090 ceph-mon[29180]: pgmap v3289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:52 smithi119 ceph-mon[32752]: pgmap v3289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:52 smithi167 ceph-mon[32751]: pgmap v3289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:53.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5117 sec 2024-04-26T19:53:54.346 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5118 sec 2024-04-26T19:53:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:54 smithi090 ceph-mon[29180]: pgmap v3290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:54 smithi119 ceph-mon[32752]: pgmap v3290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:55.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5119 sec 2024-04-26T19:53:55.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:54 smithi167 ceph-mon[32751]: pgmap v3290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:53:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4717.047729 secs 2024-04-26T19:53:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4717.047729 secs 2024-04-26T19:53:56.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5120 sec 2024-04-26T19:53:56.454 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4717.047729 secs 2024-04-26T19:53:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4717.737692 secs 2024-04-26T19:53:57.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:56 smithi090 ceph-mon[29180]: pgmap v3291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4717.737692 secs 2024-04-26T19:53:57.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:56 smithi119 ceph-mon[32752]: pgmap v3291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:57.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5121 sec 2024-04-26T19:53:57.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4717.737692 secs 2024-04-26T19:53:57.526 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:56 smithi167 ceph-mon[32751]: pgmap v3291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:58.346 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5122 sec 2024-04-26T19:53:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:53:58 smithi119 ceph-mon[32752]: pgmap v3292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:59.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5123 sec 2024-04-26T19:53:59.415 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:53:58 smithi167 ceph-mon[32751]: pgmap v3292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:53:59.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:53:58 smithi090 ceph-mon[29180]: pgmap v3292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:00.346 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5124 sec 2024-04-26T19:54:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:00 smithi119 ceph-mon[32752]: pgmap v3293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:01.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4722.047830 secs 2024-04-26T19:54:01.412 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5125 sec 2024-04-26T19:54:01.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:00 smithi167 ceph-mon[32751]: pgmap v3293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:01.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4722.047830 secs 2024-04-26T19:54:01.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:00 smithi090 ceph-mon[29180]: pgmap v3293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:01.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4722.047830 secs 2024-04-26T19:54:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4722.737803 secs 2024-04-26T19:54:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4722.737803 secs 2024-04-26T19:54:02.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4722.737803 secs 2024-04-26T19:54:02.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5126 sec 2024-04-26T19:54:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:02 smithi119 ceph-mon[32752]: pgmap v3294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:03.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5127 sec 2024-04-26T19:54:03.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:02 smithi167 ceph-mon[32751]: pgmap v3294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:03.692 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:02 smithi090 ceph-mon[29180]: pgmap v3294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:04.347 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5128 sec 2024-04-26T19:54:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:04 smithi119 ceph-mon[32752]: pgmap v3295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:05.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5129 sec 2024-04-26T19:54:05.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:04 smithi167 ceph-mon[32751]: pgmap v3295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:05.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:04 smithi090 ceph-mon[29180]: pgmap v3295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4727.047920 secs 2024-04-26T19:54:06.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4727.047920 secs 2024-04-26T19:54:06.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5130 sec 2024-04-26T19:54:06.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4727.047920 secs 2024-04-26T19:54:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4727.737861 secs 2024-04-26T19:54:07.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:06 smithi119 ceph-mon[32752]: pgmap v3296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:07.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4727.737861 secs 2024-04-26T19:54:07.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:06 smithi167 ceph-mon[32751]: pgmap v3296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:07.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5131 sec 2024-04-26T19:54:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4727.737861 secs 2024-04-26T19:54:07.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:06 smithi090 ceph-mon[29180]: pgmap v3296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:08.347 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5132 sec 2024-04-26T19:54:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:08 smithi119 ceph-mon[32752]: pgmap v3297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:09.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:08 smithi167 ceph-mon[32751]: pgmap v3297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:09.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5133 sec 2024-04-26T19:54:09.477 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:08 smithi090 ceph-mon[29180]: pgmap v3297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:10.347 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5134 sec 2024-04-26T19:54:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:10 smithi119 ceph-mon[32752]: pgmap v3298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:11.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4732.048008 secs 2024-04-26T19:54:11.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:10 smithi167 ceph-mon[32751]: pgmap v3298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:11.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4732.048008 secs 2024-04-26T19:54:11.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5135 sec 2024-04-26T19:54:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:10 smithi090 ceph-mon[29180]: pgmap v3298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:11.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4732.048008 secs 2024-04-26T19:54:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4732.737969 secs 2024-04-26T19:54:12.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5136 sec 2024-04-26T19:54:12.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4732.737969 secs 2024-04-26T19:54:12.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4732.737969 secs 2024-04-26T19:54:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:12 smithi119 ceph-mon[32752]: pgmap v3299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:12 smithi167 ceph-mon[32751]: pgmap v3299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:13.347 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5137 sec 2024-04-26T19:54:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:12 smithi090 ceph-mon[29180]: pgmap v3299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:14.348 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5138 sec 2024-04-26T19:54:15.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:14 smithi090 ceph-mon[29180]: pgmap v3300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:14 smithi119 ceph-mon[32752]: pgmap v3300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:15.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:14 smithi167 ceph-mon[32751]: pgmap v3300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:15.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5139 sec 2024-04-26T19:54:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4737.048085 secs 2024-04-26T19:54:16.333 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:54:16.346 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4737.048085 secs 2024-04-26T19:54:16.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:54:16.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5140 sec 2024-04-26T19:54:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4737.048085 secs 2024-04-26T19:54:16.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:54:17.245 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4737.738031 secs 2024-04-26T19:54:17.315 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:16 smithi090 ceph-mon[29180]: pgmap v3301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:17.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:17.495 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:17.540 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:17.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:17.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4737.738031 secs 2024-04-26T19:54:17.634 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:16 smithi167 ceph-mon[32751]: pgmap v3301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:17.688 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:17.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:17.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:17.891 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:18.157 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4737.738031 secs 2024-04-26T19:54:18.227 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:16 smithi119 ceph-mon[32752]: pgmap v3301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:18.236 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:18.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:18.254 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:18.263 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:18.272 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5141 sec 2024-04-26T19:54:18.348 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5142 sec 2024-04-26T19:54:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: pgmap v3302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:19.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.389 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.398 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.479 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.509 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.518 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.587 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.665 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.673 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.682 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5143 sec 2024-04-26T19:54:19.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: pgmap v3302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:19.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.751 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.759 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:19.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.883 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:19.959 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: pgmap v3302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:20.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:20.167 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:20.238 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:20.246 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:20.254 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.263 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:20.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:20.288 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:54:20.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:54:20.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.346 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.382 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.530 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.760 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.882 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:54:20.894 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:54:20.906 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:20.920 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:54:20.929 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:54:20.939 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.952 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.964 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.977 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:20.989 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.002 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.122 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:54:21.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:54:21.273 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:21.341 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5144 sec 2024-04-26T19:54:21.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T19:54:21.446 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T19:54:21.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.476 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.544 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T19:54:21.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:54:21.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:54:21.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:54:21.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5145 sec 2024-04-26T19:54:21.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:21 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:54:21.739 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:21 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:54:21.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:21 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:54:21.761 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:21 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:54:21.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:21 smithi090 ceph-mon[29180]: pgmap v3303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:54:21.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:21 smithi090 ceph-mon[29180]: pgmap v3304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:21.922 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4742.048144 secs 2024-04-26T19:54:21.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:21 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:54:22.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:21 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:54:22.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:21 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:54:22.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:21 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:54:22.134 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:21 smithi167 ceph-mon[32751]: pgmap v3303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:54:22.146 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:21 smithi167 ceph-mon[32751]: pgmap v3304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:22.158 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4742.048144 secs 2024-04-26T19:54:22.182 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:21 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T19:54:22.244 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:21 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T19:54:22.255 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:21 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T19:54:22.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:21 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T19:54:22.370 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:21 smithi119 ceph-mon[32752]: pgmap v3303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T19:54:22.381 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:21 smithi119 ceph-mon[32752]: pgmap v3304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:22.391 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4742.048144 secs 2024-04-26T19:54:22.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5146 sec 2024-04-26T19:54:22.408 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4742.738111 secs 2024-04-26T19:54:22.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4742.738111 secs 2024-04-26T19:54:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4742.738111 secs 2024-04-26T19:54:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:23 smithi167 ceph-mon[32751]: pgmap v3305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:23.348 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5147 sec 2024-04-26T19:54:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:23 smithi090 ceph-mon[29180]: pgmap v3305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:23 smithi119 ceph-mon[32752]: pgmap v3305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:24.348 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5148 sec 2024-04-26T19:54:25.348 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5149 sec 2024-04-26T19:54:25.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:25 smithi090 ceph-mon[29180]: pgmap v3306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-26T19:54:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:25 smithi119 ceph-mon[32752]: pgmap v3306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-26T19:54:25.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:25 smithi167 ceph-mon[32751]: pgmap v3306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-26T19:54:26.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5150 sec 2024-04-26T19:54:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4747.048204 secs 2024-04-26T19:54:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4747.048204 secs 2024-04-26T19:54:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4747.048204 secs 2024-04-26T19:54:27.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5151 sec 2024-04-26T19:54:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:27 smithi090 ceph-mon[29180]: pgmap v3307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:27.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4747.738230 secs 2024-04-26T19:54:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:27 smithi119 ceph-mon[32752]: pgmap v3307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:27.641 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4747.738230 secs 2024-04-26T19:54:27.698 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:27 smithi167 ceph-mon[32751]: pgmap v3307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:27.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4747.738230 secs 2024-04-26T19:54:28.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5152 sec 2024-04-26T19:54:29.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5153 sec 2024-04-26T19:54:29.420 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:29 smithi090 ceph-mon[29180]: pgmap v3308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:29 smithi119 ceph-mon[32752]: pgmap v3308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:29 smithi167 ceph-mon[32751]: pgmap v3308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:54:30.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5154 sec 2024-04-26T19:54:31.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5155 sec 2024-04-26T19:54:31.384 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:31 smithi167 ceph-mon[32751]: pgmap v3309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:54:31.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4752.048261 secs 2024-04-26T19:54:31.405 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:31 smithi090 ceph-mon[29180]: pgmap v3309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:54:31.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4752.048261 secs 2024-04-26T19:54:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:31 smithi119 ceph-mon[32752]: pgmap v3309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:54:31.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4752.048261 secs 2024-04-26T19:54:32.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5156 sec 2024-04-26T19:54:32.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4752.738387 secs 2024-04-26T19:54:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4752.738387 secs 2024-04-26T19:54:32.647 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4752.738387 secs 2024-04-26T19:54:33.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5157 sec 2024-04-26T19:54:33.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:33 smithi090 ceph-mon[29180]: pgmap v3310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:33 smithi119 ceph-mon[32752]: pgmap v3310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:33 smithi167 ceph-mon[32751]: pgmap v3310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:34.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5158 sec 2024-04-26T19:54:34.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:34 smithi090 ceph-mon[29180]: pgmap v3311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:34 smithi119 ceph-mon[32752]: pgmap v3311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:34.686 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:34 smithi167 ceph-mon[32751]: pgmap v3311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:35.349 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5159 sec 2024-04-26T19:54:36.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5160 sec 2024-04-26T19:54:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4757.048325 secs 2024-04-26T19:54:36.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:36 smithi119 ceph-mon[32752]: pgmap v3312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:36.739 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4757.738424 secs 2024-04-26T19:54:36.798 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4757.048325 secs 2024-04-26T19:54:36.844 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:36 smithi090 ceph-mon[29180]: pgmap v3312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:36.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4757.738424 secs 2024-04-26T19:54:36.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4757.048325 secs 2024-04-26T19:54:36.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:36 smithi167 ceph-mon[32751]: pgmap v3312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:36.921 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4757.738424 secs 2024-04-26T19:54:37.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5161 sec 2024-04-26T19:54:38.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5162 sec 2024-04-26T19:54:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:38 smithi119 ceph-mon[32752]: pgmap v3313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:38.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:38 smithi167 ceph-mon[32751]: pgmap v3313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:38.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:38 smithi090 ceph-mon[29180]: pgmap v3313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:39.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5163 sec 2024-04-26T19:54:40.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5164 sec 2024-04-26T19:54:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:40 smithi119 ceph-mon[32752]: pgmap v3314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:40.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:40 smithi090 ceph-mon[29180]: pgmap v3314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:40.733 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:40 smithi167 ceph-mon[32751]: pgmap v3314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:41.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5165 sec 2024-04-26T19:54:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4762.048381 secs 2024-04-26T19:54:41.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4762.738508 secs 2024-04-26T19:54:41.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4762.048381 secs 2024-04-26T19:54:41.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4762.738508 secs 2024-04-26T19:54:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4762.048381 secs 2024-04-26T19:54:41.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4762.738508 secs 2024-04-26T19:54:42.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5166 sec 2024-04-26T19:54:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:42 smithi119 ceph-mon[32752]: pgmap v3315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:42.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:42 smithi167 ceph-mon[32751]: pgmap v3315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:42.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:42 smithi090 ceph-mon[29180]: pgmap v3315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:43.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5167 sec 2024-04-26T19:54:44.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5168 sec 2024-04-26T19:54:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:44 smithi119 ceph-mon[32752]: pgmap v3316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:44 smithi167 ceph-mon[32751]: pgmap v3316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:44 smithi090 ceph-mon[29180]: pgmap v3316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:45.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5169 sec 2024-04-26T19:54:46.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5170 sec 2024-04-26T19:54:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4767.048442 secs 2024-04-26T19:54:46.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:46 smithi119 ceph-mon[32752]: pgmap v3317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:46.572 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4767.738565 secs 2024-04-26T19:54:46.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4767.048442 secs 2024-04-26T19:54:46.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:46 smithi167 ceph-mon[32751]: pgmap v3317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:46.598 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4767.738565 secs 2024-04-26T19:54:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4767.048442 secs 2024-04-26T19:54:46.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:46 smithi090 ceph-mon[29180]: pgmap v3317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:46.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4767.738565 secs 2024-04-26T19:54:47.350 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5171 sec 2024-04-26T19:54:48.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5172 sec 2024-04-26T19:54:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:48 smithi119 ceph-mon[32752]: pgmap v3318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:48 smithi167 ceph-mon[32751]: pgmap v3318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:48 smithi090 ceph-mon[29180]: pgmap v3318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:49.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5173 sec 2024-04-26T19:54:50.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5174 sec 2024-04-26T19:54:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:50 smithi119 ceph-mon[32752]: pgmap v3319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:50 smithi167 ceph-mon[32751]: pgmap v3319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:50 smithi090 ceph-mon[29180]: pgmap v3319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:51.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5175 sec 2024-04-26T19:54:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4772.048494 secs 2024-04-26T19:54:51.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4772.738684 secs 2024-04-26T19:54:51.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4772.048494 secs 2024-04-26T19:54:51.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4772.738684 secs 2024-04-26T19:54:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4772.048494 secs 2024-04-26T19:54:51.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4772.738684 secs 2024-04-26T19:54:52.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5176 sec 2024-04-26T19:54:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:52 smithi119 ceph-mon[32752]: pgmap v3320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:52 smithi167 ceph-mon[32751]: pgmap v3320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:52 smithi090 ceph-mon[29180]: pgmap v3320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:53.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5177 sec 2024-04-26T19:54:54.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5178 sec 2024-04-26T19:54:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:54 smithi167 ceph-mon[32751]: pgmap v3321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:54 smithi090 ceph-mon[29180]: pgmap v3321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:54 smithi119 ceph-mon[32752]: pgmap v3321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:55.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5179 sec 2024-04-26T19:54:56.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5180 sec 2024-04-26T19:54:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4777.048556 secs 2024-04-26T19:54:56.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:56 smithi167 ceph-mon[32751]: pgmap v3322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4777.738797 secs 2024-04-26T19:54:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4777.048556 secs 2024-04-26T19:54:56.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:56 smithi090 ceph-mon[29180]: pgmap v3322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:56.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4777.738797 secs 2024-04-26T19:54:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4777.048556 secs 2024-04-26T19:54:56.815 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:56 smithi119 ceph-mon[32752]: pgmap v3322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:54:56.823 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4777.738797 secs 2024-04-26T19:54:57.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5181 sec 2024-04-26T19:54:58.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5182 sec 2024-04-26T19:54:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:54:58 smithi167 ceph-mon[32751]: pgmap v3323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:54:58 smithi090 ceph-mon[29180]: pgmap v3323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:54:58 smithi119 ceph-mon[32752]: pgmap v3323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:54:59.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5183 sec 2024-04-26T19:55:00.351 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5184 sec 2024-04-26T19:55:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:00 smithi167 ceph-mon[32751]: pgmap v3324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:00 smithi090 ceph-mon[29180]: pgmap v3324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:00 smithi119 ceph-mon[32752]: pgmap v3324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:01.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5185 sec 2024-04-26T19:55:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4782.048613 secs 2024-04-26T19:55:01.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4782.738920 secs 2024-04-26T19:55:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4782.048613 secs 2024-04-26T19:55:01.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4782.738920 secs 2024-04-26T19:55:01.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4782.048613 secs 2024-04-26T19:55:01.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4782.738920 secs 2024-04-26T19:55:02.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5186 sec 2024-04-26T19:55:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:02 smithi090 ceph-mon[29180]: pgmap v3325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:02.412 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:55:02.088+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:55:02.425 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:55:02.088+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:55:02.437 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:55:02.088+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:55:02.449 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:55:02.088+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:55:02.461 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 19:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T19:55:02.088+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:55:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:02 smithi119 ceph-mon[32752]: pgmap v3325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:02 smithi167 ceph-mon[32751]: pgmap v3325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:03.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5187 sec 2024-04-26T19:55:04.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5188 sec 2024-04-26T19:55:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:04 smithi090 ceph-mon[29180]: pgmap v3326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:04 smithi119 ceph-mon[32752]: pgmap v3326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:04 smithi167 ceph-mon[32751]: pgmap v3326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:05.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5189 sec 2024-04-26T19:55:06.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5190 sec 2024-04-26T19:55:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4787.048674 secs 2024-04-26T19:55:06.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:06 smithi090 ceph-mon[29180]: pgmap v3327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:06.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4787.739034 secs 2024-04-26T19:55:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4787.048674 secs 2024-04-26T19:55:06.896 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:06 smithi119 ceph-mon[32752]: pgmap v3327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:06.908 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4787.739034 secs 2024-04-26T19:55:06.919 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4787.048674 secs 2024-04-26T19:55:06.927 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:06 smithi167 ceph-mon[32751]: pgmap v3327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:06.938 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4787.739034 secs 2024-04-26T19:55:07.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5191 sec 2024-04-26T19:55:08.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5192 sec 2024-04-26T19:55:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:08 smithi090 ceph-mon[29180]: pgmap v3328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:08 smithi119 ceph-mon[32752]: pgmap v3328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:08.856 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:08 smithi167 ceph-mon[32751]: pgmap v3328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:09.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5193 sec 2024-04-26T19:55:10.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5194 sec 2024-04-26T19:55:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:10 smithi090 ceph-mon[29180]: pgmap v3329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:10 smithi119 ceph-mon[32752]: pgmap v3329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:10 smithi167 ceph-mon[32751]: pgmap v3329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:11.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5195 sec 2024-04-26T19:55:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4792.048731 secs 2024-04-26T19:55:11.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4792.739173 secs 2024-04-26T19:55:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4792.048731 secs 2024-04-26T19:55:11.844 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4792.739173 secs 2024-04-26T19:55:11.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4792.048731 secs 2024-04-26T19:55:11.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4792.739173 secs 2024-04-26T19:55:12.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5196 sec 2024-04-26T19:55:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:12 smithi090 ceph-mon[29180]: pgmap v3330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:12 smithi119 ceph-mon[32752]: pgmap v3330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:12.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:12 smithi167 ceph-mon[32751]: pgmap v3330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:13.352 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5197 sec 2024-04-26T19:55:14.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5198 sec 2024-04-26T19:55:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:14 smithi090 ceph-mon[29180]: pgmap v3331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:14 smithi119 ceph-mon[32752]: pgmap v3331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:14.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:14 smithi167 ceph-mon[32751]: pgmap v3331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:15.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5199 sec 2024-04-26T19:55:16.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5200 sec 2024-04-26T19:55:16.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4797.048796 secs 2024-04-26T19:55:16.699 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:16 smithi119 ceph-mon[32752]: pgmap v3332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:16.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4797.739307 secs 2024-04-26T19:55:16.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4797.048796 secs 2024-04-26T19:55:16.729 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:16 smithi090 ceph-mon[29180]: pgmap v3332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:16.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4797.739307 secs 2024-04-26T19:55:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4797.048796 secs 2024-04-26T19:55:16.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:16 smithi167 ceph-mon[32751]: pgmap v3332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T19:55:16.841 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4797.739307 secs 2024-04-26T19:55:17.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5201 sec 2024-04-26T19:55:18.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5202 sec 2024-04-26T19:55:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:18 smithi090 ceph-mon[29180]: pgmap v3333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:55:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:18 smithi119 ceph-mon[32752]: pgmap v3333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:55:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:18 smithi167 ceph-mon[32751]: pgmap v3333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T19:55:19.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5203 sec 2024-04-26T19:55:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:55:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:55:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:55:20.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5204 sec 2024-04-26T19:55:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:20 smithi090 ceph-mon[29180]: pgmap v3334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:21.149 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4802.048863 secs 2024-04-26T19:55:21.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:55:21.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:55:21.199 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:20 smithi119 ceph-mon[32752]: pgmap v3334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:21.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4802.048863 secs 2024-04-26T19:55:21.322 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.342 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:55:21.372 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:55:21.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5205 sec 2024-04-26T19:55:21.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:20 smithi167 ceph-mon[32751]: pgmap v3334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:21.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4802.048863 secs 2024-04-26T19:55:21.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.652 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:21.659 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:55:21.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:55:21.679 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:55:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4802.739420 secs 2024-04-26T19:55:22.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:21 smithi090 ceph-mon[29180]: pgmap v3335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 817 B/s rd, 1 op/s 2024-04-26T19:55:22.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:21 smithi090 ceph-mon[29180]: pgmap v3336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s rd, 2 op/s 2024-04-26T19:55:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4802.739420 secs 2024-04-26T19:55:22.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:21 smithi119 ceph-mon[32752]: pgmap v3335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 817 B/s rd, 1 op/s 2024-04-26T19:55:22.319 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:21 smithi119 ceph-mon[32752]: pgmap v3336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s rd, 2 op/s 2024-04-26T19:55:22.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4802.739420 secs 2024-04-26T19:55:22.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:21 smithi167 ceph-mon[32751]: pgmap v3335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 817 B/s rd, 1 op/s 2024-04-26T19:55:22.365 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:21 smithi167 ceph-mon[32751]: pgmap v3336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s rd, 2 op/s 2024-04-26T19:55:22.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5206 sec 2024-04-26T19:55:23.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5207 sec 2024-04-26T19:55:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:23 smithi119 ceph-mon[32752]: pgmap v3337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s rd, 2 op/s 2024-04-26T19:55:24.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:23 smithi167 ceph-mon[32751]: pgmap v3337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s rd, 2 op/s 2024-04-26T19:55:24.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5208 sec 2024-04-26T19:55:24.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:23 smithi090 ceph-mon[29180]: pgmap v3337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 993 B/s rd, 2 op/s 2024-04-26T19:55:25.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5209 sec 2024-04-26T19:55:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:25 smithi119 ceph-mon[32752]: pgmap v3338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:26.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4807.048923 secs 2024-04-26T19:55:26.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:25 smithi167 ceph-mon[32751]: pgmap v3338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:26.348 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4807.048923 secs 2024-04-26T19:55:26.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5210 sec 2024-04-26T19:55:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:25 smithi090 ceph-mon[29180]: pgmap v3338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:26.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4807.048923 secs 2024-04-26T19:55:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4807.739535 secs 2024-04-26T19:55:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4807.739535 secs 2024-04-26T19:55:27.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5211 sec 2024-04-26T19:55:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4807.739535 secs 2024-04-26T19:55:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:27 smithi119 ceph-mon[32752]: pgmap v3339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:28.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:27 smithi167 ceph-mon[32751]: pgmap v3339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:28.353 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5212 sec 2024-04-26T19:55:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:27 smithi090 ceph-mon[29180]: pgmap v3339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:29.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5213 sec 2024-04-26T19:55:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:29 smithi119 ceph-mon[32752]: pgmap v3340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:29 smithi167 ceph-mon[32751]: pgmap v3340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:30.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5214 sec 2024-04-26T19:55:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:29 smithi090 ceph-mon[29180]: pgmap v3340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:55:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4812.048981 secs 2024-04-26T19:55:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4812.048981 secs 2024-04-26T19:55:31.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5215 sec 2024-04-26T19:55:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4812.048981 secs 2024-04-26T19:55:32.233 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4812.739653 secs 2024-04-26T19:55:32.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:31 smithi167 ceph-mon[32751]: pgmap v3341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:55:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4812.739653 secs 2024-04-26T19:55:32.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:31 smithi090 ceph-mon[29180]: pgmap v3341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:55:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4812.739653 secs 2024-04-26T19:55:32.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:31 smithi119 ceph-mon[32752]: pgmap v3341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:55:32.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5216 sec 2024-04-26T19:55:33.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5217 sec 2024-04-26T19:55:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:33 smithi090 ceph-mon[29180]: pgmap v3342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:33 smithi119 ceph-mon[32752]: pgmap v3342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:34.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5218 sec 2024-04-26T19:55:34.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:33 smithi167 ceph-mon[32751]: pgmap v3342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:35.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5219 sec 2024-04-26T19:55:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:35 smithi119 ceph-mon[32752]: pgmap v3343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:36.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4817.049042 secs 2024-04-26T19:55:36.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:35 smithi167 ceph-mon[32751]: pgmap v3343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:36.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4817.049042 secs 2024-04-26T19:55:36.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5220 sec 2024-04-26T19:55:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:35 smithi090 ceph-mon[29180]: pgmap v3343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:36.473 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4817.049042 secs 2024-04-26T19:55:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4817.739786 secs 2024-04-26T19:55:37.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4817.739786 secs 2024-04-26T19:55:37.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5221 sec 2024-04-26T19:55:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4817.739786 secs 2024-04-26T19:55:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:37 smithi119 ceph-mon[32752]: pgmap v3344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:38.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:37 smithi167 ceph-mon[32751]: pgmap v3344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:38.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5222 sec 2024-04-26T19:55:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:37 smithi090 ceph-mon[29180]: pgmap v3344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:39.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5223 sec 2024-04-26T19:55:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:39 smithi119 ceph-mon[32752]: pgmap v3345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:39 smithi167 ceph-mon[32751]: pgmap v3345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:40.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5224 sec 2024-04-26T19:55:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:39 smithi090 ceph-mon[29180]: pgmap v3345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4822.049098 secs 2024-04-26T19:55:41.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4822.049098 secs 2024-04-26T19:55:41.354 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5225 sec 2024-04-26T19:55:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4822.049098 secs 2024-04-26T19:55:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4822.739866 secs 2024-04-26T19:55:42.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:41 smithi119 ceph-mon[32752]: pgmap v3346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:42.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4822.739866 secs 2024-04-26T19:55:42.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:41 smithi167 ceph-mon[32751]: pgmap v3346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:42.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5226 sec 2024-04-26T19:55:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4822.739866 secs 2024-04-26T19:55:42.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:41 smithi090 ceph-mon[29180]: pgmap v3346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:43.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5227 sec 2024-04-26T19:55:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:43 smithi119 ceph-mon[32752]: pgmap v3347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:44.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5228 sec 2024-04-26T19:55:44.388 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:43 smithi167 ceph-mon[32751]: pgmap v3347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:44.417 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:43 smithi090 ceph-mon[29180]: pgmap v3347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:45.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5229 sec 2024-04-26T19:55:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:45 smithi119 ceph-mon[32752]: pgmap v3348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:46.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4827.049159 secs 2024-04-26T19:55:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:45 smithi167 ceph-mon[32751]: pgmap v3348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:46.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4827.049159 secs 2024-04-26T19:55:46.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5230 sec 2024-04-26T19:55:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:45 smithi090 ceph-mon[29180]: pgmap v3348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:46.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4827.049159 secs 2024-04-26T19:55:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4827.739996 secs 2024-04-26T19:55:47.344 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4827.739996 secs 2024-04-26T19:55:47.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5231 sec 2024-04-26T19:55:47.426 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4827.739996 secs 2024-04-26T19:55:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:47 smithi119 ceph-mon[32752]: pgmap v3349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:48.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:47 smithi167 ceph-mon[32751]: pgmap v3349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:48.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5232 sec 2024-04-26T19:55:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:47 smithi090 ceph-mon[29180]: pgmap v3349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:49.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5233 sec 2024-04-26T19:55:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:49 smithi119 ceph-mon[32752]: pgmap v3350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:49 smithi167 ceph-mon[32751]: pgmap v3350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:50.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5234 sec 2024-04-26T19:55:50.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:49 smithi090 ceph-mon[29180]: pgmap v3350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4832.049216 secs 2024-04-26T19:55:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4832.049216 secs 2024-04-26T19:55:51.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5235 sec 2024-04-26T19:55:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4832.049216 secs 2024-04-26T19:55:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4832.740112 secs 2024-04-26T19:55:52.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:51 smithi119 ceph-mon[32752]: pgmap v3351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4832.740112 secs 2024-04-26T19:55:52.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:51 smithi167 ceph-mon[32751]: pgmap v3351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:52.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5236 sec 2024-04-26T19:55:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4832.740112 secs 2024-04-26T19:55:52.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:51 smithi090 ceph-mon[29180]: pgmap v3351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:53.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5237 sec 2024-04-26T19:55:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:53 smithi119 ceph-mon[32752]: pgmap v3352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:53 smithi167 ceph-mon[32751]: pgmap v3352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:54.355 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5238 sec 2024-04-26T19:55:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:53 smithi090 ceph-mon[29180]: pgmap v3352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:55.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5239 sec 2024-04-26T19:55:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:55 smithi119 ceph-mon[32752]: pgmap v3353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:56.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4837.049276 secs 2024-04-26T19:55:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:55 smithi167 ceph-mon[32751]: pgmap v3353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:56.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4837.049276 secs 2024-04-26T19:55:56.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5240 sec 2024-04-26T19:55:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:55 smithi090 ceph-mon[29180]: pgmap v3353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:55:56.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4837.049276 secs 2024-04-26T19:55:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4837.740231 secs 2024-04-26T19:55:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4837.740231 secs 2024-04-26T19:55:57.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5241 sec 2024-04-26T19:55:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4837.740231 secs 2024-04-26T19:55:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:57 smithi119 ceph-mon[32752]: pgmap v3354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:57 smithi167 ceph-mon[32751]: pgmap v3354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:58.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5242 sec 2024-04-26T19:55:58.477 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:57 smithi090 ceph-mon[29180]: pgmap v3354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:55:59.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5243 sec 2024-04-26T19:56:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:55:59 smithi119 ceph-mon[32752]: pgmap v3355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:55:59 smithi167 ceph-mon[32751]: pgmap v3355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:00.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5244 sec 2024-04-26T19:56:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:55:59 smithi090 ceph-mon[29180]: pgmap v3355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:01.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4842.049332 secs 2024-04-26T19:56:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4842.049332 secs 2024-04-26T19:56:01.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5245 sec 2024-04-26T19:56:01.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4842.049332 secs 2024-04-26T19:56:02.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4842.740348 secs 2024-04-26T19:56:02.292 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:01 smithi090 ceph-mon[29180]: pgmap v3356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4842.740348 secs 2024-04-26T19:56:02.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:01 smithi119 ceph-mon[32752]: pgmap v3356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:02.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4842.740348 secs 2024-04-26T19:56:02.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:01 smithi167 ceph-mon[32751]: pgmap v3356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:02.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5246 sec 2024-04-26T19:56:03.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5247 sec 2024-04-26T19:56:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:03 smithi119 ceph-mon[32752]: pgmap v3357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:03 smithi167 ceph-mon[32751]: pgmap v3357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:04.356 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5248 sec 2024-04-26T19:56:04.452 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:03 smithi090 ceph-mon[29180]: pgmap v3357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:05.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5249 sec 2024-04-26T19:56:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:05 smithi119 ceph-mon[32752]: pgmap v3358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:06.330 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:05 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 4847.049393 secs 2024-04-26T19:56:06.339 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:05 smithi119 ceph-mon[32752]: slow request 362.830834 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:56:06.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:05 smithi167 ceph-mon[32751]: pgmap v3358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:06.360 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:05 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 4847.049393 secs 2024-04-26T19:56:06.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:05 smithi167 ceph-mon[32751]: slow request 362.830834 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:56:06.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5250 sec 2024-04-26T19:56:06.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:05 smithi090 ceph-mon[29180]: pgmap v3358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:06.402 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:05 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 4847.049393 secs 2024-04-26T19:56:06.409 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:05 smithi090 ceph-mon[29180]: slow request 362.830834 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T19:56:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4847.740459 secs 2024-04-26T19:56:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4847.740459 secs 2024-04-26T19:56:07.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5251 sec 2024-04-26T19:56:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4847.740459 secs 2024-04-26T19:56:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:07 smithi119 ceph-mon[32752]: pgmap v3359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:08.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:07 smithi167 ceph-mon[32751]: pgmap v3359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:08.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5252 sec 2024-04-26T19:56:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:07 smithi090 ceph-mon[29180]: pgmap v3359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:09.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5253 sec 2024-04-26T19:56:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:09 smithi119 ceph-mon[32752]: pgmap v3360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:09 smithi167 ceph-mon[32751]: pgmap v3360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:10.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5254 sec 2024-04-26T19:56:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:09 smithi090 ceph-mon[29180]: pgmap v3360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4852.049448 secs 2024-04-26T19:56:11.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4852.049448 secs 2024-04-26T19:56:11.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5255 sec 2024-04-26T19:56:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4852.049448 secs 2024-04-26T19:56:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4852.740625 secs 2024-04-26T19:56:12.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:11 smithi119 ceph-mon[32752]: pgmap v3361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:12.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4852.740625 secs 2024-04-26T19:56:12.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:11 smithi167 ceph-mon[32751]: pgmap v3361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:12.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5256 sec 2024-04-26T19:56:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4852.740625 secs 2024-04-26T19:56:12.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:11 smithi090 ceph-mon[29180]: pgmap v3361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:13.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5257 sec 2024-04-26T19:56:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:13 smithi119 ceph-mon[32752]: pgmap v3362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:14.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:13 smithi167 ceph-mon[32751]: pgmap v3362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:14.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5258 sec 2024-04-26T19:56:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:13 smithi090 ceph-mon[29180]: pgmap v3362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T19:56:15.357 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5259 sec 2024-04-26T19:56:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:15 smithi119 ceph-mon[32752]: pgmap v3363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:16.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:15 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 4857.049504 secs 2024-04-26T19:56:16.326 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:15 smithi119 ceph-mon[32752]: slow request 2880.319639 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:56:16.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:15 smithi167 ceph-mon[32751]: pgmap v3363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:16.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:15 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 4857.049504 secs 2024-04-26T19:56:16.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:15 smithi167 ceph-mon[32751]: slow request 2880.319639 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:56:16.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5260 sec 2024-04-26T19:56:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:15 smithi090 ceph-mon[29180]: pgmap v3363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T19:56:16.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:15 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 4857.049504 secs 2024-04-26T19:56:16.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:15 smithi090 ceph-mon[29180]: slow request 2880.319639 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T19:56:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4857.740720 secs 2024-04-26T19:56:17.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5261 sec 2024-04-26T19:56:17.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4857.740720 secs 2024-04-26T19:56:17.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4857.740720 secs 2024-04-26T19:56:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:18 smithi119 ceph-mon[32752]: pgmap v3364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:18 smithi167 ceph-mon[32751]: pgmap v3364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:18.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5262 sec 2024-04-26T19:56:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:18 smithi090 ceph-mon[29180]: pgmap v3364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:19.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5263 sec 2024-04-26T19:56:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:20 smithi090 ceph-mon[29180]: pgmap v3365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:20 smithi119 ceph-mon[32752]: pgmap v3365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:20.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:20 smithi167 ceph-mon[32751]: pgmap v3365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:20.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5264 sec 2024-04-26T19:56:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4862.049560 secs 2024-04-26T19:56:21.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:56:21.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4862.049560 secs 2024-04-26T19:56:21.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:56:21.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5265 sec 2024-04-26T19:56:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4862.049560 secs 2024-04-26T19:56:21.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:56:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4862.740782 secs 2024-04-26T19:56:22.318 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:22 smithi119 ceph-mon[32752]: pgmap v3366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:22.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:56:22.338 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:56:22.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:56:22.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5266 sec 2024-04-26T19:56:22.368 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4862.740782 secs 2024-04-26T19:56:22.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:22 smithi167 ceph-mon[32751]: pgmap v3366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:22.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:56:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:56:22.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:56:22.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4862.740782 secs 2024-04-26T19:56:22.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:22 smithi090 ceph-mon[29180]: pgmap v3366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:22.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:56:22.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:56:22.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:56:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:23 smithi119 ceph-mon[32752]: pgmap v3367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T19:56:23.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:23 smithi119 ceph-mon[32752]: pgmap v3368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:23.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:23 smithi167 ceph-mon[32751]: pgmap v3367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T19:56:23.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:23 smithi167 ceph-mon[32751]: pgmap v3368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:23.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5267 sec 2024-04-26T19:56:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:23 smithi090 ceph-mon[29180]: pgmap v3367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T19:56:23.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:23 smithi090 ceph-mon[29180]: pgmap v3368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:24.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5268 sec 2024-04-26T19:56:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:25 smithi119 ceph-mon[32752]: pgmap v3369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T19:56:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:25 smithi167 ceph-mon[32751]: pgmap v3369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T19:56:25.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5269 sec 2024-04-26T19:56:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:25 smithi090 ceph-mon[29180]: pgmap v3369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T19:56:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4867.049622 secs 2024-04-26T19:56:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4867.049622 secs 2024-04-26T19:56:26.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5270 sec 2024-04-26T19:56:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4867.049622 secs 2024-04-26T19:56:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:27 smithi119 ceph-mon[32752]: pgmap v3370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:27.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4867.740864 secs 2024-04-26T19:56:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:27 smithi167 ceph-mon[32751]: pgmap v3370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:27.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4867.740864 secs 2024-04-26T19:56:27.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5271 sec 2024-04-26T19:56:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:27 smithi090 ceph-mon[29180]: pgmap v3370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:27.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4867.740864 secs 2024-04-26T19:56:28.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5272 sec 2024-04-26T19:56:29.109 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:56:29.108+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:56:29.119 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:56:29.108+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:56:29.132 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:56:29.108+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 41 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:56:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:56:29.108+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 41 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:56:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:29 smithi119 ceph-mon[32752]: pgmap v3371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T19:56:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:29 smithi167 ceph-mon[32751]: pgmap v3371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T19:56:29.358 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5273 sec 2024-04-26T19:56:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:29 smithi090 ceph-mon[29180]: pgmap v3371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T19:56:30.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5274 sec 2024-04-26T19:56:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:31 smithi119 ceph-mon[32752]: pgmap v3372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:31.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4872.049677 secs 2024-04-26T19:56:31.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:31 smithi167 ceph-mon[32751]: pgmap v3372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:31.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4872.049677 secs 2024-04-26T19:56:31.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5275 sec 2024-04-26T19:56:31.405 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:31 smithi090 ceph-mon[29180]: pgmap v3372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:56:31.413 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4872.049677 secs 2024-04-26T19:56:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4872.740934 secs 2024-04-26T19:56:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4872.740934 secs 2024-04-26T19:56:32.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5276 sec 2024-04-26T19:56:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4872.740934 secs 2024-04-26T19:56:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:33 smithi119 ceph-mon[32752]: pgmap v3373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:56:33.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:33 smithi167 ceph-mon[32751]: pgmap v3373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:56:33.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5277 sec 2024-04-26T19:56:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:33 smithi090 ceph-mon[29180]: pgmap v3373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:56:34.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5278 sec 2024-04-26T19:56:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:35 smithi119 ceph-mon[32752]: pgmap v3374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:35 smithi167 ceph-mon[32751]: pgmap v3374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:35.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5279 sec 2024-04-26T19:56:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:35 smithi090 ceph-mon[29180]: pgmap v3374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4877.049737 secs 2024-04-26T19:56:36.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4877.049737 secs 2024-04-26T19:56:36.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5280 sec 2024-04-26T19:56:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4877.049737 secs 2024-04-26T19:56:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:37 smithi119 ceph-mon[32752]: pgmap v3375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:56:37.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4877.741060 secs 2024-04-26T19:56:37.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:37 smithi167 ceph-mon[32751]: pgmap v3375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:56:37.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4877.741060 secs 2024-04-26T19:56:37.359 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5281 sec 2024-04-26T19:56:37.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:37 smithi090 ceph-mon[29180]: pgmap v3375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:56:37.428 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4877.741060 secs 2024-04-26T19:56:38.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5282 sec 2024-04-26T19:56:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:39 smithi119 ceph-mon[32752]: pgmap v3376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:39.336 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:39 smithi167 ceph-mon[32751]: pgmap v3376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:39.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5283 sec 2024-04-26T19:56:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:39 smithi090 ceph-mon[29180]: pgmap v3376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:40.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5284 sec 2024-04-26T19:56:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:41 smithi167 ceph-mon[32751]: pgmap v3377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:41.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4882.049793 secs 2024-04-26T19:56:41.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5285 sec 2024-04-26T19:56:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:41 smithi090 ceph-mon[29180]: pgmap v3377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:41.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4882.049793 secs 2024-04-26T19:56:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:41 smithi119 ceph-mon[32752]: pgmap v3377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:41.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4882.049793 secs 2024-04-26T19:56:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4882.741162 secs 2024-04-26T19:56:42.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5286 sec 2024-04-26T19:56:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4882.741162 secs 2024-04-26T19:56:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4882.741162 secs 2024-04-26T19:56:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:43 smithi167 ceph-mon[32751]: pgmap v3378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:43.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5287 sec 2024-04-26T19:56:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:43 smithi090 ceph-mon[29180]: pgmap v3378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:43 smithi119 ceph-mon[32752]: pgmap v3378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:44.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5288 sec 2024-04-26T19:56:45.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5289 sec 2024-04-26T19:56:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:45 smithi090 ceph-mon[29180]: pgmap v3379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:45 smithi119 ceph-mon[32752]: pgmap v3379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:45 smithi167 ceph-mon[32751]: pgmap v3379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:46.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5290 sec 2024-04-26T19:56:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4887.049849 secs 2024-04-26T19:56:46.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4887.049849 secs 2024-04-26T19:56:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4887.049849 secs 2024-04-26T19:56:47.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5291 sec 2024-04-26T19:56:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:47 smithi090 ceph-mon[29180]: pgmap v3380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:56:47.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4887.741286 secs 2024-04-26T19:56:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:47 smithi119 ceph-mon[32752]: pgmap v3380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:56:47.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4887.741286 secs 2024-04-26T19:56:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:47 smithi167 ceph-mon[32751]: pgmap v3380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T19:56:47.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4887.741286 secs 2024-04-26T19:56:48.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5292 sec 2024-04-26T19:56:49.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5293 sec 2024-04-26T19:56:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:49 smithi090 ceph-mon[29180]: pgmap v3381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:49 smithi119 ceph-mon[32752]: pgmap v3381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:49 smithi167 ceph-mon[32751]: pgmap v3381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:50.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5294 sec 2024-04-26T19:56:51.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5295 sec 2024-04-26T19:56:51.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:51 smithi090 ceph-mon[29180]: pgmap v3382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:51.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4892.049904 secs 2024-04-26T19:56:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:51 smithi119 ceph-mon[32752]: pgmap v3382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:51.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4892.049904 secs 2024-04-26T19:56:51.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:51 smithi167 ceph-mon[32751]: pgmap v3382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:51.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4892.049904 secs 2024-04-26T19:56:52.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5296 sec 2024-04-26T19:56:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4892.741413 secs 2024-04-26T19:56:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4892.741413 secs 2024-04-26T19:56:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4892.741413 secs 2024-04-26T19:56:53.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5297 sec 2024-04-26T19:56:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:53 smithi090 ceph-mon[29180]: pgmap v3383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:53 smithi119 ceph-mon[32752]: pgmap v3383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:53 smithi167 ceph-mon[32751]: pgmap v3383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:54.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5298 sec 2024-04-26T19:56:55.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5299 sec 2024-04-26T19:56:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:55 smithi090 ceph-mon[29180]: pgmap v3384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:55 smithi119 ceph-mon[32752]: pgmap v3384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:55 smithi167 ceph-mon[32751]: pgmap v3384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:56:56.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5300 sec 2024-04-26T19:56:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4897.049964 secs 2024-04-26T19:56:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4897.049964 secs 2024-04-26T19:56:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4897.049964 secs 2024-04-26T19:56:57.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5301 sec 2024-04-26T19:56:57.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:57 smithi090 ceph-mon[29180]: pgmap v3385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:57.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4897.741472 secs 2024-04-26T19:56:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:57 smithi119 ceph-mon[32752]: pgmap v3385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:57.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4897.741472 secs 2024-04-26T19:56:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:57 smithi167 ceph-mon[32751]: pgmap v3385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:57.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4897.741472 secs 2024-04-26T19:56:58.361 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5302 sec 2024-04-26T19:56:59.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5303 sec 2024-04-26T19:56:59.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:56:59 smithi090 ceph-mon[29180]: pgmap v3386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:56:59 smithi119 ceph-mon[32752]: pgmap v3386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:56:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:56:59 smithi167 ceph-mon[32751]: pgmap v3386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:00.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5304 sec 2024-04-26T19:57:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:00 smithi119 ceph-mon[32752]: pgmap v3387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:00 smithi167 ceph-mon[32751]: pgmap v3387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:00 smithi090 ceph-mon[29180]: pgmap v3387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:01.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5305 sec 2024-04-26T19:57:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4902.050013 secs 2024-04-26T19:57:01.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4902.741568 secs 2024-04-26T19:57:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4902.050013 secs 2024-04-26T19:57:01.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4902.741568 secs 2024-04-26T19:57:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4902.050013 secs 2024-04-26T19:57:01.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4902.741568 secs 2024-04-26T19:57:02.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5306 sec 2024-04-26T19:57:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:02 smithi090 ceph-mon[29180]: pgmap v3388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:02 smithi119 ceph-mon[32752]: pgmap v3388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:02 smithi167 ceph-mon[32751]: pgmap v3388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:03.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5307 sec 2024-04-26T19:57:04.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5308 sec 2024-04-26T19:57:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:04 smithi119 ceph-mon[32752]: pgmap v3389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:04 smithi167 ceph-mon[32751]: pgmap v3389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:04 smithi090 ceph-mon[29180]: pgmap v3389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:05.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5309 sec 2024-04-26T19:57:06.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5310 sec 2024-04-26T19:57:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4907.050075 secs 2024-04-26T19:57:06.816 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:06 smithi119 ceph-mon[32752]: pgmap v3390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:06.825 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4907.741645 secs 2024-04-26T19:57:06.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4907.050075 secs 2024-04-26T19:57:06.840 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:06 smithi167 ceph-mon[32751]: pgmap v3390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:06.848 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4907.741645 secs 2024-04-26T19:57:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4907.050075 secs 2024-04-26T19:57:06.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:06 smithi090 ceph-mon[29180]: pgmap v3390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:06.891 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4907.741645 secs 2024-04-26T19:57:07.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5311 sec 2024-04-26T19:57:08.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5312 sec 2024-04-26T19:57:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:08 smithi119 ceph-mon[32752]: pgmap v3391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:08 smithi167 ceph-mon[32751]: pgmap v3391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:08 smithi090 ceph-mon[29180]: pgmap v3391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:09.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5313 sec 2024-04-26T19:57:10.362 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5314 sec 2024-04-26T19:57:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:10 smithi119 ceph-mon[32752]: pgmap v3392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:10 smithi167 ceph-mon[32751]: pgmap v3392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:10 smithi090 ceph-mon[29180]: pgmap v3392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:11.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5315 sec 2024-04-26T19:57:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4912.050133 secs 2024-04-26T19:57:11.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4912.741737 secs 2024-04-26T19:57:11.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4912.050133 secs 2024-04-26T19:57:11.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4912.741737 secs 2024-04-26T19:57:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4912.050133 secs 2024-04-26T19:57:11.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4912.741737 secs 2024-04-26T19:57:12.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5316 sec 2024-04-26T19:57:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:12 smithi119 ceph-mon[32752]: pgmap v3393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:12 smithi167 ceph-mon[32751]: pgmap v3393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:12 smithi090 ceph-mon[29180]: pgmap v3393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:13.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5317 sec 2024-04-26T19:57:14.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5318 sec 2024-04-26T19:57:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:14 smithi119 ceph-mon[32752]: pgmap v3394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:14 smithi167 ceph-mon[32751]: pgmap v3394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:14 smithi090 ceph-mon[29180]: pgmap v3394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:15.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5319 sec 2024-04-26T19:57:16.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5320 sec 2024-04-26T19:57:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4917.050186 secs 2024-04-26T19:57:16.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:16 smithi119 ceph-mon[32752]: pgmap v3395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:16.824 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4917.741814 secs 2024-04-26T19:57:16.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4917.050186 secs 2024-04-26T19:57:16.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:16 smithi167 ceph-mon[32751]: pgmap v3395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:16.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4917.741814 secs 2024-04-26T19:57:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4917.050186 secs 2024-04-26T19:57:16.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:16 smithi090 ceph-mon[29180]: pgmap v3395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:16.896 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4917.741814 secs 2024-04-26T19:57:17.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5321 sec 2024-04-26T19:57:18.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5322 sec 2024-04-26T19:57:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:18 smithi119 ceph-mon[32752]: pgmap v3396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:18 smithi167 ceph-mon[32751]: pgmap v3396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:18 smithi090 ceph-mon[29180]: pgmap v3396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:19.363 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5323 sec 2024-04-26T19:57:20.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5324 sec 2024-04-26T19:57:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:20 smithi119 ceph-mon[32752]: pgmap v3397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:20 smithi167 ceph-mon[32751]: pgmap v3397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:20 smithi090 ceph-mon[29180]: pgmap v3397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:21.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5325 sec 2024-04-26T19:57:21.765 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4922.050242 secs 2024-04-26T19:57:21.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4922.741905 secs 2024-04-26T19:57:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4922.050242 secs 2024-04-26T19:57:21.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4922.741905 secs 2024-04-26T19:57:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4922.050242 secs 2024-04-26T19:57:21.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4922.741905 secs 2024-04-26T19:57:22.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5326 sec 2024-04-26T19:57:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:22 smithi119 ceph-mon[32752]: pgmap v3398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:22.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:57:22.826 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:57:22.835 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:57:22.846 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:22 smithi119 ceph-mon[32752]: pgmap v3399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:57:22.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:22 smithi119 ceph-mon[32752]: pgmap v3400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:57:22.947 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:57:22.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:22 smithi090 ceph-mon[29180]: pgmap v3398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:22.971 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:57:22.983 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:57:23.077 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:57:23.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:22 smithi090 ceph-mon[29180]: pgmap v3399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:57:23.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:22 smithi090 ceph-mon[29180]: pgmap v3400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:57:23.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:57:23.145 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:22 smithi167 ceph-mon[32751]: pgmap v3398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:23.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:57:23.174 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:57:23.185 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:57:23.194 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:22 smithi167 ceph-mon[32751]: pgmap v3399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:57:23.205 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:22 smithi167 ceph-mon[32751]: pgmap v3400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:57:23.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:57:23.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5327 sec 2024-04-26T19:57:24.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5328 sec 2024-04-26T19:57:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:24 smithi119 ceph-mon[32752]: pgmap v3401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:24 smithi167 ceph-mon[32751]: pgmap v3401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:24 smithi090 ceph-mon[29180]: pgmap v3401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:25.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5329 sec 2024-04-26T19:57:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4927.050300 secs 2024-04-26T19:57:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4927.050300 secs 2024-04-26T19:57:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4927.050300 secs 2024-04-26T19:57:26.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5330 sec 2024-04-26T19:57:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4927.742003 secs 2024-04-26T19:57:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:26 smithi119 ceph-mon[32752]: pgmap v3402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4927.742003 secs 2024-04-26T19:57:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:26 smithi167 ceph-mon[32751]: pgmap v3402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4927.742003 secs 2024-04-26T19:57:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:26 smithi090 ceph-mon[29180]: pgmap v3402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:27.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5331 sec 2024-04-26T19:57:28.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5332 sec 2024-04-26T19:57:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:28 smithi119 ceph-mon[32752]: pgmap v3403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:57:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:28 smithi167 ceph-mon[32751]: pgmap v3403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:57:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:28 smithi090 ceph-mon[29180]: pgmap v3403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:57:29.364 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5333 sec 2024-04-26T19:57:30.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5334 sec 2024-04-26T19:57:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:30 smithi119 ceph-mon[32752]: pgmap v3404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:30 smithi167 ceph-mon[32751]: pgmap v3404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:30 smithi090 ceph-mon[29180]: pgmap v3404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:57:31.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5335 sec 2024-04-26T19:57:31.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4932.050351 secs 2024-04-26T19:57:31.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4932.742120 secs 2024-04-26T19:57:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4932.050351 secs 2024-04-26T19:57:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4932.742120 secs 2024-04-26T19:57:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4932.050351 secs 2024-04-26T19:57:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4932.742120 secs 2024-04-26T19:57:32.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5336 sec 2024-04-26T19:57:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:32 smithi090 ceph-mon[29180]: pgmap v3405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:57:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:32 smithi119 ceph-mon[32752]: pgmap v3405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:57:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:32 smithi167 ceph-mon[32751]: pgmap v3405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:57:33.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5337 sec 2024-04-26T19:57:34.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5338 sec 2024-04-26T19:57:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:34 smithi119 ceph-mon[32752]: pgmap v3406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:34 smithi167 ceph-mon[32751]: pgmap v3406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:34 smithi090 ceph-mon[29180]: pgmap v3406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:35.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5339 sec 2024-04-26T19:57:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4937.050410 secs 2024-04-26T19:57:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4937.050410 secs 2024-04-26T19:57:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4937.050410 secs 2024-04-26T19:57:36.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5340 sec 2024-04-26T19:57:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4937.742255 secs 2024-04-26T19:57:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:36 smithi119 ceph-mon[32752]: pgmap v3407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4937.742255 secs 2024-04-26T19:57:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:36 smithi167 ceph-mon[32751]: pgmap v3407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4937.742255 secs 2024-04-26T19:57:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:36 smithi090 ceph-mon[29180]: pgmap v3407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:37.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5341 sec 2024-04-26T19:57:38.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5342 sec 2024-04-26T19:57:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:38 smithi119 ceph-mon[32752]: pgmap v3408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:38 smithi167 ceph-mon[32751]: pgmap v3408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:38 smithi090 ceph-mon[29180]: pgmap v3408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:39.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5343 sec 2024-04-26T19:57:40.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5344 sec 2024-04-26T19:57:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:40 smithi119 ceph-mon[32752]: pgmap v3409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:40 smithi167 ceph-mon[32751]: pgmap v3409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:40 smithi090 ceph-mon[29180]: pgmap v3409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:41.365 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5345 sec 2024-04-26T19:57:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4942.050466 secs 2024-04-26T19:57:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4942.742322 secs 2024-04-26T19:57:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4942.050466 secs 2024-04-26T19:57:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4942.742322 secs 2024-04-26T19:57:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4942.050466 secs 2024-04-26T19:57:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4942.742322 secs 2024-04-26T19:57:42.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5346 sec 2024-04-26T19:57:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:42 smithi119 ceph-mon[32752]: pgmap v3410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:42 smithi167 ceph-mon[32751]: pgmap v3410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:42 smithi090 ceph-mon[29180]: pgmap v3410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:43.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5347 sec 2024-04-26T19:57:44.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5348 sec 2024-04-26T19:57:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:44 smithi119 ceph-mon[32752]: pgmap v3411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:44 smithi167 ceph-mon[32751]: pgmap v3411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:44 smithi090 ceph-mon[29180]: pgmap v3411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:45.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5349 sec 2024-04-26T19:57:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4947.050542 secs 2024-04-26T19:57:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4947.050542 secs 2024-04-26T19:57:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4947.050542 secs 2024-04-26T19:57:46.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5350 sec 2024-04-26T19:57:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4947.742456 secs 2024-04-26T19:57:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:46 smithi119 ceph-mon[32752]: pgmap v3412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4947.742456 secs 2024-04-26T19:57:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:46 smithi167 ceph-mon[32751]: pgmap v3412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4947.742456 secs 2024-04-26T19:57:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:46 smithi090 ceph-mon[29180]: pgmap v3412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:47.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5351 sec 2024-04-26T19:57:48.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5352 sec 2024-04-26T19:57:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:48 smithi119 ceph-mon[32752]: pgmap v3413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:48 smithi167 ceph-mon[32751]: pgmap v3413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:48 smithi090 ceph-mon[29180]: pgmap v3413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:49.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5353 sec 2024-04-26T19:57:50.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5354 sec 2024-04-26T19:57:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:50 smithi119 ceph-mon[32752]: pgmap v3414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:50 smithi167 ceph-mon[32751]: pgmap v3414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:50 smithi090 ceph-mon[29180]: pgmap v3414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:51.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5355 sec 2024-04-26T19:57:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4952.050621 secs 2024-04-26T19:57:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4952.742580 secs 2024-04-26T19:57:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4952.050621 secs 2024-04-26T19:57:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4952.742580 secs 2024-04-26T19:57:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4952.050621 secs 2024-04-26T19:57:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4952.742580 secs 2024-04-26T19:57:52.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5356 sec 2024-04-26T19:57:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:52 smithi119 ceph-mon[32752]: pgmap v3415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:52 smithi167 ceph-mon[32751]: pgmap v3415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:52 smithi090 ceph-mon[29180]: pgmap v3415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:53.366 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5357 sec 2024-04-26T19:57:54.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5358 sec 2024-04-26T19:57:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:54 smithi090 ceph-mon[29180]: pgmap v3416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:54 smithi119 ceph-mon[32752]: pgmap v3416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:54 smithi167 ceph-mon[32751]: pgmap v3416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:57:55.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5359 sec 2024-04-26T19:57:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4957.050705 secs 2024-04-26T19:57:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4957.050705 secs 2024-04-26T19:57:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4957.050705 secs 2024-04-26T19:57:56.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5360 sec 2024-04-26T19:57:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4957.742721 secs 2024-04-26T19:57:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:56 smithi090 ceph-mon[29180]: pgmap v3417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4957.742721 secs 2024-04-26T19:57:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:56 smithi119 ceph-mon[32752]: pgmap v3417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4957.742721 secs 2024-04-26T19:57:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:56 smithi167 ceph-mon[32751]: pgmap v3417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:57.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5361 sec 2024-04-26T19:57:58.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5362 sec 2024-04-26T19:57:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:57:58 smithi119 ceph-mon[32752]: pgmap v3418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:57:58 smithi167 ceph-mon[32751]: pgmap v3418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:57:58 smithi090 ceph-mon[29180]: pgmap v3418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:57:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:57:59.119+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:57:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:57:59.119+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:57:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:57:59.119+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 42 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:57:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:57:59.119+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 42 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006680 con 0x56302893f0e0 2024-04-26T19:57:59.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5363 sec 2024-04-26T19:58:00.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5364 sec 2024-04-26T19:58:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:00 smithi119 ceph-mon[32752]: pgmap v3419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:00 smithi167 ceph-mon[32751]: pgmap v3419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:00 smithi090 ceph-mon[29180]: pgmap v3419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:01.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5365 sec 2024-04-26T19:58:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4962.050792 secs 2024-04-26T19:58:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4962.742840 secs 2024-04-26T19:58:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4962.050792 secs 2024-04-26T19:58:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4962.742840 secs 2024-04-26T19:58:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4962.050792 secs 2024-04-26T19:58:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4962.742840 secs 2024-04-26T19:58:02.367 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5366 sec 2024-04-26T19:58:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:02 smithi090 ceph-mon[29180]: pgmap v3420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:02 smithi119 ceph-mon[32752]: pgmap v3420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:02 smithi167 ceph-mon[32751]: pgmap v3420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:03.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5367 sec 2024-04-26T19:58:04.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5368 sec 2024-04-26T19:58:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:04 smithi119 ceph-mon[32752]: pgmap v3421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:04 smithi167 ceph-mon[32751]: pgmap v3421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:04 smithi090 ceph-mon[29180]: pgmap v3421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:05.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5369 sec 2024-04-26T19:58:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4967.050827 secs 2024-04-26T19:58:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4967.050827 secs 2024-04-26T19:58:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4967.050827 secs 2024-04-26T19:58:06.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5370 sec 2024-04-26T19:58:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4967.742946 secs 2024-04-26T19:58:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:06 smithi167 ceph-mon[32751]: pgmap v3422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4967.742946 secs 2024-04-26T19:58:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:06 smithi090 ceph-mon[29180]: pgmap v3422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4967.742946 secs 2024-04-26T19:58:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:06 smithi119 ceph-mon[32752]: pgmap v3422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:07.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5371 sec 2024-04-26T19:58:08.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5372 sec 2024-04-26T19:58:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:08 smithi119 ceph-mon[32752]: pgmap v3423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:08 smithi167 ceph-mon[32751]: pgmap v3423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:08 smithi090 ceph-mon[29180]: pgmap v3423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:09.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5373 sec 2024-04-26T19:58:10.368 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5374 sec 2024-04-26T19:58:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:10 smithi119 ceph-mon[32752]: pgmap v3424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:10 smithi167 ceph-mon[32751]: pgmap v3424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:10 smithi090 ceph-mon[29180]: pgmap v3424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:11.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5375 sec 2024-04-26T19:58:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4972.050886 secs 2024-04-26T19:58:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4972.743081 secs 2024-04-26T19:58:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4972.050886 secs 2024-04-26T19:58:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4972.743081 secs 2024-04-26T19:58:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4972.050886 secs 2024-04-26T19:58:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4972.743081 secs 2024-04-26T19:58:12.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5376 sec 2024-04-26T19:58:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:12 smithi119 ceph-mon[32752]: pgmap v3425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:12 smithi167 ceph-mon[32751]: pgmap v3425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:12 smithi090 ceph-mon[29180]: pgmap v3425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:13.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5377 sec 2024-04-26T19:58:14.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5378 sec 2024-04-26T19:58:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:14 smithi119 ceph-mon[32752]: pgmap v3426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:14 smithi167 ceph-mon[32751]: pgmap v3426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:14 smithi090 ceph-mon[29180]: pgmap v3426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:15.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5379 sec 2024-04-26T19:58:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4977.050946 secs 2024-04-26T19:58:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4977.050946 secs 2024-04-26T19:58:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4977.050946 secs 2024-04-26T19:58:16.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5380 sec 2024-04-26T19:58:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4977.743184 secs 2024-04-26T19:58:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:16 smithi167 ceph-mon[32751]: pgmap v3427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4977.743184 secs 2024-04-26T19:58:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:16 smithi090 ceph-mon[29180]: pgmap v3427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4977.743184 secs 2024-04-26T19:58:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:16 smithi119 ceph-mon[32752]: pgmap v3427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:17.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5381 sec 2024-04-26T19:58:18.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5382 sec 2024-04-26T19:58:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:18 smithi119 ceph-mon[32752]: pgmap v3428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:18 smithi167 ceph-mon[32751]: pgmap v3428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:18 smithi090 ceph-mon[29180]: pgmap v3428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:19.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5383 sec 2024-04-26T19:58:20.369 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5384 sec 2024-04-26T19:58:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:20 smithi119 ceph-mon[32752]: pgmap v3429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:20 smithi167 ceph-mon[32751]: pgmap v3429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:20 smithi090 ceph-mon[29180]: pgmap v3429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:21.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5385 sec 2024-04-26T19:58:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4982.051024 secs 2024-04-26T19:58:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4982.743246 secs 2024-04-26T19:58:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4982.051024 secs 2024-04-26T19:58:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4982.743246 secs 2024-04-26T19:58:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4982.051024 secs 2024-04-26T19:58:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4982.743246 secs 2024-04-26T19:58:22.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5386 sec 2024-04-26T19:58:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:22 smithi119 ceph-mon[32752]: pgmap v3430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:22 smithi167 ceph-mon[32751]: pgmap v3430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:22 smithi090 ceph-mon[29180]: pgmap v3430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:23.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5387 sec 2024-04-26T19:58:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:58:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:58:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:23 smithi119 ceph-mon[32752]: pgmap v3431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:58:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:23 smithi119 ceph-mon[32752]: pgmap v3432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:58:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:58:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:58:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:58:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:23 smithi167 ceph-mon[32751]: pgmap v3431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:58:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:23 smithi167 ceph-mon[32751]: pgmap v3432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:58:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:58:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:58:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:58:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:58:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:23 smithi090 ceph-mon[29180]: pgmap v3431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T19:58:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:23 smithi090 ceph-mon[29180]: pgmap v3432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T19:58:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:58:24.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5388 sec 2024-04-26T19:58:25.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5389 sec 2024-04-26T19:58:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4987.051087 secs 2024-04-26T19:58:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:25 smithi119 ceph-mon[32752]: pgmap v3433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4987.051087 secs 2024-04-26T19:58:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:25 smithi167 ceph-mon[32751]: pgmap v3433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:26.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5390 sec 2024-04-26T19:58:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4987.051087 secs 2024-04-26T19:58:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:25 smithi090 ceph-mon[29180]: pgmap v3433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4987.743361 secs 2024-04-26T19:58:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4987.743361 secs 2024-04-26T19:58:27.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5391 sec 2024-04-26T19:58:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4987.743361 secs 2024-04-26T19:58:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:27 smithi119 ceph-mon[32752]: pgmap v3434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:27 smithi167 ceph-mon[32751]: pgmap v3434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:28.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5392 sec 2024-04-26T19:58:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:27 smithi090 ceph-mon[29180]: pgmap v3434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:29.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5393 sec 2024-04-26T19:58:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:29 smithi119 ceph-mon[32752]: pgmap v3435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:29 smithi167 ceph-mon[32751]: pgmap v3435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:30.370 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5394 sec 2024-04-26T19:58:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:29 smithi090 ceph-mon[29180]: pgmap v3435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4992.051147 secs 2024-04-26T19:58:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4992.051147 secs 2024-04-26T19:58:31.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5395 sec 2024-04-26T19:58:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4992.051147 secs 2024-04-26T19:58:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4992.743484 secs 2024-04-26T19:58:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:31 smithi090 ceph-mon[29180]: pgmap v3436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4992.743484 secs 2024-04-26T19:58:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:31 smithi119 ceph-mon[32752]: pgmap v3436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4992.743484 secs 2024-04-26T19:58:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:31 smithi167 ceph-mon[32751]: pgmap v3436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:58:32.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5396 sec 2024-04-26T19:58:33.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5397 sec 2024-04-26T19:58:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:33 smithi119 ceph-mon[32752]: pgmap v3437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:58:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:33 smithi167 ceph-mon[32751]: pgmap v3437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:58:34.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5398 sec 2024-04-26T19:58:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:33 smithi090 ceph-mon[29180]: pgmap v3437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:58:35.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5399 sec 2024-04-26T19:58:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 4997.051231 secs 2024-04-26T19:58:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:35 smithi119 ceph-mon[32752]: pgmap v3438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 4997.051231 secs 2024-04-26T19:58:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:35 smithi167 ceph-mon[32751]: pgmap v3438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:36.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5400 sec 2024-04-26T19:58:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 4997.051231 secs 2024-04-26T19:58:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:35 smithi090 ceph-mon[29180]: pgmap v3438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 4997.743556 secs 2024-04-26T19:58:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 4997.743556 secs 2024-04-26T19:58:37.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5401 sec 2024-04-26T19:58:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 4997.743556 secs 2024-04-26T19:58:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:37 smithi119 ceph-mon[32752]: pgmap v3439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:37 smithi167 ceph-mon[32751]: pgmap v3439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:38.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5402 sec 2024-04-26T19:58:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:37 smithi090 ceph-mon[29180]: pgmap v3439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:39.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5403 sec 2024-04-26T19:58:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:39 smithi119 ceph-mon[32752]: pgmap v3440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:39 smithi167 ceph-mon[32751]: pgmap v3440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:40.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5404 sec 2024-04-26T19:58:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:39 smithi090 ceph-mon[29180]: pgmap v3440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5002.051311 secs 2024-04-26T19:58:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5002.051311 secs 2024-04-26T19:58:41.371 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5405 sec 2024-04-26T19:58:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5002.051311 secs 2024-04-26T19:58:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5002.743679 secs 2024-04-26T19:58:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:41 smithi119 ceph-mon[32752]: pgmap v3441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5002.743679 secs 2024-04-26T19:58:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:41 smithi167 ceph-mon[32751]: pgmap v3441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:42.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5406 sec 2024-04-26T19:58:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5002.743679 secs 2024-04-26T19:58:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:41 smithi090 ceph-mon[29180]: pgmap v3441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:43.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5407 sec 2024-04-26T19:58:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:43 smithi119 ceph-mon[32752]: pgmap v3442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:43 smithi167 ceph-mon[32751]: pgmap v3442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:44.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5408 sec 2024-04-26T19:58:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:43 smithi090 ceph-mon[29180]: pgmap v3442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:45.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5409 sec 2024-04-26T19:58:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5007.051396 secs 2024-04-26T19:58:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:45 smithi119 ceph-mon[32752]: pgmap v3443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5007.051396 secs 2024-04-26T19:58:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:45 smithi167 ceph-mon[32751]: pgmap v3443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:46.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5410 sec 2024-04-26T19:58:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5007.051396 secs 2024-04-26T19:58:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:45 smithi090 ceph-mon[29180]: pgmap v3443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5007.743800 secs 2024-04-26T19:58:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5007.743800 secs 2024-04-26T19:58:47.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5411 sec 2024-04-26T19:58:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5007.743800 secs 2024-04-26T19:58:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:47 smithi119 ceph-mon[32752]: pgmap v3444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:47 smithi167 ceph-mon[32751]: pgmap v3444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:48.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5412 sec 2024-04-26T19:58:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:47 smithi090 ceph-mon[29180]: pgmap v3444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:49.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5413 sec 2024-04-26T19:58:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:49 smithi119 ceph-mon[32752]: pgmap v3445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:49 smithi167 ceph-mon[32751]: pgmap v3445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:50.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5414 sec 2024-04-26T19:58:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:49 smithi090 ceph-mon[29180]: pgmap v3445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5012.051473 secs 2024-04-26T19:58:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5012.051473 secs 2024-04-26T19:58:51.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5415 sec 2024-04-26T19:58:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5012.051473 secs 2024-04-26T19:58:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5012.743942 secs 2024-04-26T19:58:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:51 smithi119 ceph-mon[32752]: pgmap v3446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5012.743942 secs 2024-04-26T19:58:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:51 smithi167 ceph-mon[32751]: pgmap v3446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:52.372 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5416 sec 2024-04-26T19:58:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5012.743942 secs 2024-04-26T19:58:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:51 smithi090 ceph-mon[29180]: pgmap v3446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:53.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5417 sec 2024-04-26T19:58:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:54 smithi119 ceph-mon[32752]: pgmap v3447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:54 smithi167 ceph-mon[32751]: pgmap v3447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:54.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5418 sec 2024-04-26T19:58:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:54 smithi090 ceph-mon[29180]: pgmap v3447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:55.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5419 sec 2024-04-26T19:58:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5017.051558 secs 2024-04-26T19:58:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:56 smithi119 ceph-mon[32752]: pgmap v3448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5017.051558 secs 2024-04-26T19:58:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:56 smithi167 ceph-mon[32751]: pgmap v3448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:56.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5420 sec 2024-04-26T19:58:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5017.051558 secs 2024-04-26T19:58:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:56 smithi090 ceph-mon[29180]: pgmap v3448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:58:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5017.744054 secs 2024-04-26T19:58:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5017.744054 secs 2024-04-26T19:58:57.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5421 sec 2024-04-26T19:58:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5017.744054 secs 2024-04-26T19:58:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:58:58 smithi119 ceph-mon[32752]: pgmap v3449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:58:58 smithi167 ceph-mon[32751]: pgmap v3449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:58.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5422 sec 2024-04-26T19:58:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:58:58 smithi090 ceph-mon[29180]: pgmap v3449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:58:59.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5423 sec 2024-04-26T19:59:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:00 smithi119 ceph-mon[32752]: pgmap v3450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:00 smithi167 ceph-mon[32751]: pgmap v3450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:00.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5424 sec 2024-04-26T19:59:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:00 smithi090 ceph-mon[29180]: pgmap v3450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5022.051639 secs 2024-04-26T19:59:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5022.051639 secs 2024-04-26T19:59:01.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5425 sec 2024-04-26T19:59:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5022.051639 secs 2024-04-26T19:59:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5022.744168 secs 2024-04-26T19:59:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:02 smithi090 ceph-mon[29180]: pgmap v3451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5022.744168 secs 2024-04-26T19:59:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:02 smithi119 ceph-mon[32752]: pgmap v3451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5022.744168 secs 2024-04-26T19:59:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:02 smithi167 ceph-mon[32751]: pgmap v3451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:02.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5426 sec 2024-04-26T19:59:03.373 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5427 sec 2024-04-26T19:59:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:04 smithi119 ceph-mon[32752]: pgmap v3452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:04 smithi167 ceph-mon[32751]: pgmap v3452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:04.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5428 sec 2024-04-26T19:59:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:04 smithi090 ceph-mon[29180]: pgmap v3452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:05.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5429 sec 2024-04-26T19:59:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5027.051722 secs 2024-04-26T19:59:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:06 smithi119 ceph-mon[32752]: pgmap v3453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5027.051722 secs 2024-04-26T19:59:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:06 smithi167 ceph-mon[32751]: pgmap v3453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:06.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5430 sec 2024-04-26T19:59:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5027.051722 secs 2024-04-26T19:59:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:06 smithi090 ceph-mon[29180]: pgmap v3453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5027.744285 secs 2024-04-26T19:59:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5027.744285 secs 2024-04-26T19:59:07.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5431 sec 2024-04-26T19:59:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5027.744285 secs 2024-04-26T19:59:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:08 smithi119 ceph-mon[32752]: pgmap v3454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:08 smithi167 ceph-mon[32751]: pgmap v3454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:08.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5432 sec 2024-04-26T19:59:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:08 smithi090 ceph-mon[29180]: pgmap v3454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:09.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5433 sec 2024-04-26T19:59:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:10 smithi119 ceph-mon[32752]: pgmap v3455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:10 smithi167 ceph-mon[32751]: pgmap v3455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:10.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5434 sec 2024-04-26T19:59:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:10 smithi090 ceph-mon[29180]: pgmap v3455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5032.051803 secs 2024-04-26T19:59:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5032.051803 secs 2024-04-26T19:59:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5032.051803 secs 2024-04-26T19:59:11.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5435 sec 2024-04-26T19:59:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5032.744408 secs 2024-04-26T19:59:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:12 smithi119 ceph-mon[32752]: pgmap v3456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5032.744408 secs 2024-04-26T19:59:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:12 smithi167 ceph-mon[32751]: pgmap v3456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:12.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5436 sec 2024-04-26T19:59:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5032.744408 secs 2024-04-26T19:59:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:12 smithi090 ceph-mon[29180]: pgmap v3456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:13.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5437 sec 2024-04-26T19:59:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:14 smithi119 ceph-mon[32752]: pgmap v3457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:14 smithi167 ceph-mon[32751]: pgmap v3457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:14.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5438 sec 2024-04-26T19:59:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:14 smithi090 ceph-mon[29180]: pgmap v3457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:15.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5439 sec 2024-04-26T19:59:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5037.051829 secs 2024-04-26T19:59:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:16 smithi119 ceph-mon[32752]: pgmap v3458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5037.051829 secs 2024-04-26T19:59:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:16 smithi167 ceph-mon[32751]: pgmap v3458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:16.374 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5440 sec 2024-04-26T19:59:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5037.051829 secs 2024-04-26T19:59:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:16 smithi090 ceph-mon[29180]: pgmap v3458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5037.744515 secs 2024-04-26T19:59:17.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5441 sec 2024-04-26T19:59:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5037.744515 secs 2024-04-26T19:59:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5037.744515 secs 2024-04-26T19:59:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:18 smithi167 ceph-mon[32751]: pgmap v3459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:18.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5442 sec 2024-04-26T19:59:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:18 smithi090 ceph-mon[29180]: pgmap v3459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:18 smithi119 ceph-mon[32752]: pgmap v3459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:19.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5443 sec 2024-04-26T19:59:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:20 smithi167 ceph-mon[32751]: pgmap v3460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:20.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5444 sec 2024-04-26T19:59:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:20 smithi090 ceph-mon[29180]: pgmap v3460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:20 smithi119 ceph-mon[32752]: pgmap v3460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5042.051887 secs 2024-04-26T19:59:21.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5445 sec 2024-04-26T19:59:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5042.051887 secs 2024-04-26T19:59:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5042.051887 secs 2024-04-26T19:59:22.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5446 sec 2024-04-26T19:59:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5042.744652 secs 2024-04-26T19:59:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:22 smithi090 ceph-mon[29180]: pgmap v3461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5042.744652 secs 2024-04-26T19:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:22 smithi119 ceph-mon[32752]: pgmap v3461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5042.744652 secs 2024-04-26T19:59:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:22 smithi167 ceph-mon[32751]: pgmap v3461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:23.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5447 sec 2024-04-26T19:59:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T19:59:24.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5448 sec 2024-04-26T19:59:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:24 smithi090 ceph-mon[29180]: pgmap v3462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:59:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:59:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:59:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:24 smithi119 ceph-mon[32752]: pgmap v3462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:59:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:59:24.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:59:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:24 smithi167 ceph-mon[32751]: pgmap v3462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T19:59:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T19:59:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T19:59:25.375 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5449 sec 2024-04-26T19:59:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:25 smithi090 ceph-mon[29180]: pgmap v3463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:59:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:25 smithi090 ceph-mon[29180]: pgmap v3464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:25 smithi119 ceph-mon[32752]: pgmap v3463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:59:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:25 smithi119 ceph-mon[32752]: pgmap v3464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:25 smithi167 ceph-mon[32751]: pgmap v3463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T19:59:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:25 smithi167 ceph-mon[32751]: pgmap v3464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:26.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5450 sec 2024-04-26T19:59:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5047.051972 secs 2024-04-26T19:59:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5047.051972 secs 2024-04-26T19:59:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5047.051972 secs 2024-04-26T19:59:27.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5451 sec 2024-04-26T19:59:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5047.744728 secs 2024-04-26T19:59:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:27 smithi090 ceph-mon[29180]: pgmap v3465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5047.744728 secs 2024-04-26T19:59:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:27 smithi119 ceph-mon[32752]: pgmap v3465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5047.744728 secs 2024-04-26T19:59:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:27 smithi167 ceph-mon[32751]: pgmap v3465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:28.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5452 sec 2024-04-26T19:59:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:59:29.131+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:59:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:59:29.131+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T19:59:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:59:29.131+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 43 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T19:59:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T19:59:29.131+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 43 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T19:59:29.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5453 sec 2024-04-26T19:59:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:29 smithi090 ceph-mon[29180]: pgmap v3466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:59:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:29 smithi119 ceph-mon[32752]: pgmap v3466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:59:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:29 smithi167 ceph-mon[32751]: pgmap v3466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T19:59:30.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5454 sec 2024-04-26T19:59:31.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:31 smithi167 ceph-mon[32751]: pgmap v3467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:31.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5052.052048 secs 2024-04-26T19:59:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:31 smithi090 ceph-mon[29180]: pgmap v3467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5052.052048 secs 2024-04-26T19:59:31.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5455 sec 2024-04-26T19:59:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:31 smithi119 ceph-mon[32752]: pgmap v3467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5052.052048 secs 2024-04-26T19:59:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5052.744816 secs 2024-04-26T19:59:32.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5456 sec 2024-04-26T19:59:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5052.744816 secs 2024-04-26T19:59:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5052.744816 secs 2024-04-26T19:59:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:33 smithi090 ceph-mon[29180]: pgmap v3468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:33.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5457 sec 2024-04-26T19:59:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:33 smithi119 ceph-mon[32752]: pgmap v3468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:33 smithi167 ceph-mon[32751]: pgmap v3468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T19:59:34.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5458 sec 2024-04-26T19:59:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:35 smithi090 ceph-mon[29180]: pgmap v3469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:59:35.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5459 sec 2024-04-26T19:59:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:35 smithi119 ceph-mon[32752]: pgmap v3469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:59:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:35 smithi167 ceph-mon[32751]: pgmap v3469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T19:59:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5057.052127 secs 2024-04-26T19:59:36.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5460 sec 2024-04-26T19:59:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5057.052127 secs 2024-04-26T19:59:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5057.052127 secs 2024-04-26T19:59:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5057.744891 secs 2024-04-26T19:59:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:37 smithi090 ceph-mon[29180]: pgmap v3470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:37.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5461 sec 2024-04-26T19:59:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5057.744891 secs 2024-04-26T19:59:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:37 smithi119 ceph-mon[32752]: pgmap v3470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5057.744891 secs 2024-04-26T19:59:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:37 smithi167 ceph-mon[32751]: pgmap v3470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:38.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5462 sec 2024-04-26T19:59:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:39 smithi090 ceph-mon[29180]: pgmap v3471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:39.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5463 sec 2024-04-26T19:59:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:39 smithi119 ceph-mon[32752]: pgmap v3471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:39 smithi167 ceph-mon[32751]: pgmap v3471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:40.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5464 sec 2024-04-26T19:59:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:41 smithi090 ceph-mon[29180]: pgmap v3472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5062.052210 secs 2024-04-26T19:59:41.376 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5465 sec 2024-04-26T19:59:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:41 smithi119 ceph-mon[32752]: pgmap v3472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5062.052210 secs 2024-04-26T19:59:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:41 smithi167 ceph-mon[32751]: pgmap v3472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5062.052210 secs 2024-04-26T19:59:42.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5466 sec 2024-04-26T19:59:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5062.745032 secs 2024-04-26T19:59:42.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5062.745032 secs 2024-04-26T19:59:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5062.745032 secs 2024-04-26T19:59:43.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5467 sec 2024-04-26T19:59:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:43 smithi119 ceph-mon[32752]: pgmap v3473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:43 smithi167 ceph-mon[32751]: pgmap v3473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:43 smithi090 ceph-mon[29180]: pgmap v3473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:44.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5468 sec 2024-04-26T19:59:45.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5469 sec 2024-04-26T19:59:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:45 smithi119 ceph-mon[32752]: pgmap v3474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:45.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:45 smithi167 ceph-mon[32751]: pgmap v3474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:45 smithi090 ceph-mon[29180]: pgmap v3474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:46.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5470 sec 2024-04-26T19:59:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5067.052293 secs 2024-04-26T19:59:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5067.745115 secs 2024-04-26T19:59:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:46 smithi119 ceph-mon[32752]: pgmap v3475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5067.052293 secs 2024-04-26T19:59:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5067.745115 secs 2024-04-26T19:59:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:46 smithi167 ceph-mon[32751]: pgmap v3475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5067.052293 secs 2024-04-26T19:59:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5067.745115 secs 2024-04-26T19:59:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:46 smithi090 ceph-mon[29180]: pgmap v3475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:47.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5471 sec 2024-04-26T19:59:48.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5472 sec 2024-04-26T19:59:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:48 smithi119 ceph-mon[32752]: pgmap v3476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:48 smithi167 ceph-mon[32751]: pgmap v3476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:48 smithi090 ceph-mon[29180]: pgmap v3476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:49.377 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5473 sec 2024-04-26T19:59:50.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5474 sec 2024-04-26T19:59:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:50 smithi119 ceph-mon[32752]: pgmap v3477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5072.052374 secs 2024-04-26T19:59:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:50 smithi167 ceph-mon[32751]: pgmap v3477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5072.052374 secs 2024-04-26T19:59:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:50 smithi090 ceph-mon[29180]: pgmap v3477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5072.052374 secs 2024-04-26T19:59:51.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5475 sec 2024-04-26T19:59:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5072.745231 secs 2024-04-26T19:59:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5072.745231 secs 2024-04-26T19:59:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5072.745231 secs 2024-04-26T19:59:52.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5476 sec 2024-04-26T19:59:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:52 smithi119 ceph-mon[32752]: pgmap v3478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:52 smithi167 ceph-mon[32751]: pgmap v3478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:52 smithi090 ceph-mon[29180]: pgmap v3478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:53.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5477 sec 2024-04-26T19:59:54.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5478 sec 2024-04-26T19:59:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:54 smithi119 ceph-mon[32752]: pgmap v3479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:54 smithi167 ceph-mon[32751]: pgmap v3479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:54 smithi090 ceph-mon[29180]: pgmap v3479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T19:59:55.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5479 sec 2024-04-26T19:59:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5077.052460 secs 2024-04-26T19:59:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5077.052460 secs 2024-04-26T19:59:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5077.052460 secs 2024-04-26T19:59:56.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5480 sec 2024-04-26T19:59:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5077.745285 secs 2024-04-26T19:59:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:56 smithi119 ceph-mon[32752]: pgmap v3480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5077.745285 secs 2024-04-26T19:59:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:56 smithi167 ceph-mon[32751]: pgmap v3480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5077.745285 secs 2024-04-26T19:59:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:56 smithi090 ceph-mon[29180]: pgmap v3480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:57.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5481 sec 2024-04-26T19:59:58.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5482 sec 2024-04-26T19:59:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 19:59:58 smithi119 ceph-mon[32752]: pgmap v3481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 19:59:58 smithi167 ceph-mon[32751]: pgmap v3481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 19:59:58 smithi090 ceph-mon[29180]: pgmap v3481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T19:59:59.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5483 sec 2024-04-26T20:00:00.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5484 sec 2024-04-26T20:00:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:00 smithi119 ceph-mon[32752]: pgmap v3482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5082.052550 secs 2024-04-26T20:00:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:00:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:00 smithi167 ceph-mon[32751]: pgmap v3482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5082.052550 secs 2024-04-26T20:00:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:00:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:00 smithi090 ceph-mon[29180]: pgmap v3482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5082.052550 secs 2024-04-26T20:00:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:00:01.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5485 sec 2024-04-26T20:00:02.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5082.745408 secs 2024-04-26T20:00:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5082.745408 secs 2024-04-26T20:00:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5082.745408 secs 2024-04-26T20:00:02.378 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5486 sec 2024-04-26T20:00:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:02 smithi119 ceph-mon[32752]: pgmap v3483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:02 smithi167 ceph-mon[32751]: pgmap v3483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:02 smithi090 ceph-mon[29180]: pgmap v3483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:03.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5487 sec 2024-04-26T20:00:04.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5488 sec 2024-04-26T20:00:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:04 smithi119 ceph-mon[32752]: pgmap v3484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:04 smithi167 ceph-mon[32751]: pgmap v3484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:04 smithi090 ceph-mon[29180]: pgmap v3484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:05.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5489 sec 2024-04-26T20:00:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5087.052633 secs 2024-04-26T20:00:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5087.052633 secs 2024-04-26T20:00:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5087.052633 secs 2024-04-26T20:00:06.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5490 sec 2024-04-26T20:00:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5087.745543 secs 2024-04-26T20:00:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:06 smithi119 ceph-mon[32752]: pgmap v3485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5087.745543 secs 2024-04-26T20:00:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:06 smithi167 ceph-mon[32751]: pgmap v3485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5087.745543 secs 2024-04-26T20:00:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:06 smithi090 ceph-mon[29180]: pgmap v3485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:07.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5491 sec 2024-04-26T20:00:08.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5492 sec 2024-04-26T20:00:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:08 smithi119 ceph-mon[32752]: pgmap v3486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:08 smithi167 ceph-mon[32751]: pgmap v3486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:08 smithi090 ceph-mon[29180]: pgmap v3486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:09.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5493 sec 2024-04-26T20:00:10.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5494 sec 2024-04-26T20:00:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:10 smithi119 ceph-mon[32752]: pgmap v3487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5092.052713 secs 2024-04-26T20:00:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:10 smithi167 ceph-mon[32751]: pgmap v3487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5092.052713 secs 2024-04-26T20:00:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:10 smithi090 ceph-mon[29180]: pgmap v3487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5092.052713 secs 2024-04-26T20:00:11.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5495 sec 2024-04-26T20:00:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5092.745667 secs 2024-04-26T20:00:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5092.745667 secs 2024-04-26T20:00:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5092.745667 secs 2024-04-26T20:00:12.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5496 sec 2024-04-26T20:00:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:12 smithi119 ceph-mon[32752]: pgmap v3488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:12 smithi167 ceph-mon[32751]: pgmap v3488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:12 smithi090 ceph-mon[29180]: pgmap v3488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:13.379 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5497 sec 2024-04-26T20:00:14.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5498 sec 2024-04-26T20:00:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:14 smithi119 ceph-mon[32752]: pgmap v3489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:14 smithi167 ceph-mon[32751]: pgmap v3489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:14 smithi090 ceph-mon[29180]: pgmap v3489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:15.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5499 sec 2024-04-26T20:00:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5097.052817 secs 2024-04-26T20:00:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5097.052817 secs 2024-04-26T20:00:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5097.052817 secs 2024-04-26T20:00:16.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5500 sec 2024-04-26T20:00:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5097.745744 secs 2024-04-26T20:00:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:16 smithi119 ceph-mon[32752]: pgmap v3490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5097.745744 secs 2024-04-26T20:00:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:16 smithi167 ceph-mon[32751]: pgmap v3490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5097.745744 secs 2024-04-26T20:00:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:16 smithi090 ceph-mon[29180]: pgmap v3490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:17.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5501 sec 2024-04-26T20:00:18.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5502 sec 2024-04-26T20:00:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:18 smithi119 ceph-mon[32752]: pgmap v3491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:18 smithi167 ceph-mon[32751]: pgmap v3491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:18 smithi090 ceph-mon[29180]: pgmap v3491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:19.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5503 sec 2024-04-26T20:00:20.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5504 sec 2024-04-26T20:00:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:20 smithi119 ceph-mon[32752]: pgmap v3492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5102.052876 secs 2024-04-26T20:00:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:20 smithi167 ceph-mon[32751]: pgmap v3492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5102.052876 secs 2024-04-26T20:00:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:20 smithi090 ceph-mon[29180]: pgmap v3492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5102.052876 secs 2024-04-26T20:00:21.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5505 sec 2024-04-26T20:00:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5102.745841 secs 2024-04-26T20:00:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5102.745841 secs 2024-04-26T20:00:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5102.745841 secs 2024-04-26T20:00:22.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5506 sec 2024-04-26T20:00:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:22 smithi167 ceph-mon[32751]: pgmap v3493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:22 smithi090 ceph-mon[29180]: pgmap v3493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:22 smithi119 ceph-mon[32752]: pgmap v3493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:23.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5507 sec 2024-04-26T20:00:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:00:24.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5508 sec 2024-04-26T20:00:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:24 smithi167 ceph-mon[32751]: pgmap v3494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:00:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:24 smithi090 ceph-mon[29180]: pgmap v3494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:00:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:24 smithi119 ceph-mon[32752]: pgmap v3494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:00:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:00:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:00:25.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5509 sec 2024-04-26T20:00:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:25 smithi090 ceph-mon[29180]: pgmap v3495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:00:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:25 smithi090 ceph-mon[29180]: pgmap v3496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5107.052958 secs 2024-04-26T20:00:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:25 smithi119 ceph-mon[32752]: pgmap v3495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:00:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:25 smithi119 ceph-mon[32752]: pgmap v3496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5107.052958 secs 2024-04-26T20:00:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:25 smithi167 ceph-mon[32751]: pgmap v3495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:00:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:25 smithi167 ceph-mon[32751]: pgmap v3496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5107.052958 secs 2024-04-26T20:00:26.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5510 sec 2024-04-26T20:00:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5107.745958 secs 2024-04-26T20:00:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5107.745958 secs 2024-04-26T20:00:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5107.745958 secs 2024-04-26T20:00:27.380 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5511 sec 2024-04-26T20:00:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:27 smithi090 ceph-mon[29180]: pgmap v3497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:27 smithi119 ceph-mon[32752]: pgmap v3497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:27 smithi167 ceph-mon[32751]: pgmap v3497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:28.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5512 sec 2024-04-26T20:00:29.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5513 sec 2024-04-26T20:00:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:29 smithi090 ceph-mon[29180]: pgmap v3498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:29 smithi119 ceph-mon[32752]: pgmap v3498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:29 smithi167 ceph-mon[32751]: pgmap v3498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:30.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5514 sec 2024-04-26T20:00:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5112.053042 secs 2024-04-26T20:00:31.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5112.053042 secs 2024-04-26T20:00:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5112.053042 secs 2024-04-26T20:00:31.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5515 sec 2024-04-26T20:00:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:31 smithi090 ceph-mon[29180]: pgmap v3499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5112.746073 secs 2024-04-26T20:00:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:31 smithi119 ceph-mon[32752]: pgmap v3499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5112.746073 secs 2024-04-26T20:00:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:31 smithi167 ceph-mon[32751]: pgmap v3499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:00:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5112.746073 secs 2024-04-26T20:00:32.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5516 sec 2024-04-26T20:00:32.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:00:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:00:32.137+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:00:32.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:00:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:00:32.137+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:00:32.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:00:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:00:32.137+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:00:32.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:00:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:00:32.137+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:00:32.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:00:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:00:32.137+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:00:33.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5517 sec 2024-04-26T20:00:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:33 smithi090 ceph-mon[29180]: pgmap v3500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:00:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:33 smithi119 ceph-mon[32752]: pgmap v3500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:00:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:33 smithi167 ceph-mon[32751]: pgmap v3500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:00:34.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5518 sec 2024-04-26T20:00:35.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5519 sec 2024-04-26T20:00:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:35 smithi090 ceph-mon[29180]: pgmap v3501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:00:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5117.053104 secs 2024-04-26T20:00:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:35 smithi119 ceph-mon[32752]: pgmap v3501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:00:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5117.053104 secs 2024-04-26T20:00:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:35 smithi167 ceph-mon[32751]: pgmap v3501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:00:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5117.053104 secs 2024-04-26T20:00:36.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5520 sec 2024-04-26T20:00:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5117.746213 secs 2024-04-26T20:00:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5117.746213 secs 2024-04-26T20:00:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5117.746213 secs 2024-04-26T20:00:37.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5521 sec 2024-04-26T20:00:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:37 smithi090 ceph-mon[29180]: pgmap v3502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:37 smithi119 ceph-mon[32752]: pgmap v3502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:37 smithi167 ceph-mon[32751]: pgmap v3502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:38.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5522 sec 2024-04-26T20:00:39.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5523 sec 2024-04-26T20:00:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:39 smithi090 ceph-mon[29180]: pgmap v3503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:39 smithi119 ceph-mon[32752]: pgmap v3503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:39 smithi167 ceph-mon[32751]: pgmap v3503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:40.381 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5524 sec 2024-04-26T20:00:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5122.053185 secs 2024-04-26T20:00:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5122.053185 secs 2024-04-26T20:00:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5122.053185 secs 2024-04-26T20:00:41.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5525 sec 2024-04-26T20:00:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:41 smithi090 ceph-mon[29180]: pgmap v3504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5122.746346 secs 2024-04-26T20:00:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:41 smithi119 ceph-mon[32752]: pgmap v3504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5122.746346 secs 2024-04-26T20:00:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:41 smithi167 ceph-mon[32751]: pgmap v3504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5122.746346 secs 2024-04-26T20:00:42.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5526 sec 2024-04-26T20:00:43.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5527 sec 2024-04-26T20:00:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:43 smithi090 ceph-mon[29180]: pgmap v3505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:43 smithi119 ceph-mon[32752]: pgmap v3505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:43 smithi167 ceph-mon[32751]: pgmap v3505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:44.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5529 sec 2024-04-26T20:00:45.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5530 sec 2024-04-26T20:00:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:45 smithi090 ceph-mon[29180]: pgmap v3506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5127.053247 secs 2024-04-26T20:00:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:45 smithi119 ceph-mon[32752]: pgmap v3506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5127.053247 secs 2024-04-26T20:00:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:45 smithi167 ceph-mon[32751]: pgmap v3506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5127.053247 secs 2024-04-26T20:00:46.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5531 sec 2024-04-26T20:00:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5127.746478 secs 2024-04-26T20:00:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5127.746478 secs 2024-04-26T20:00:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5127.746478 secs 2024-04-26T20:00:47.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5532 sec 2024-04-26T20:00:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:47 smithi119 ceph-mon[32752]: pgmap v3507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:47 smithi167 ceph-mon[32751]: pgmap v3507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:47 smithi090 ceph-mon[29180]: pgmap v3507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:48.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5533 sec 2024-04-26T20:00:49.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5534 sec 2024-04-26T20:00:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:49 smithi119 ceph-mon[32752]: pgmap v3508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:49 smithi167 ceph-mon[32751]: pgmap v3508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:49 smithi090 ceph-mon[29180]: pgmap v3508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:50.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5535 sec 2024-04-26T20:00:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5132.053330 secs 2024-04-26T20:00:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5132.053330 secs 2024-04-26T20:00:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5132.053330 secs 2024-04-26T20:00:51.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5536 sec 2024-04-26T20:00:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:51 smithi119 ceph-mon[32752]: pgmap v3509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5132.746549 secs 2024-04-26T20:00:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:51 smithi167 ceph-mon[32751]: pgmap v3509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5132.746549 secs 2024-04-26T20:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:51 smithi090 ceph-mon[29180]: pgmap v3509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5132.746549 secs 2024-04-26T20:00:52.382 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5537 sec 2024-04-26T20:00:53.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5538 sec 2024-04-26T20:00:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:53 smithi119 ceph-mon[32752]: pgmap v3510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:53 smithi167 ceph-mon[32751]: pgmap v3510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:53 smithi090 ceph-mon[29180]: pgmap v3510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:54.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5539 sec 2024-04-26T20:00:55.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5540 sec 2024-04-26T20:00:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:55 smithi119 ceph-mon[32752]: pgmap v3511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5137.053416 secs 2024-04-26T20:00:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:55 smithi167 ceph-mon[32751]: pgmap v3511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5137.053416 secs 2024-04-26T20:00:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:55 smithi090 ceph-mon[29180]: pgmap v3511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:00:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5137.053416 secs 2024-04-26T20:00:56.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5541 sec 2024-04-26T20:00:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5137.746676 secs 2024-04-26T20:00:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5137.746676 secs 2024-04-26T20:00:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5137.746676 secs 2024-04-26T20:00:57.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5542 sec 2024-04-26T20:00:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:57 smithi119 ceph-mon[32752]: pgmap v3512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:57 smithi167 ceph-mon[32751]: pgmap v3512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:57 smithi090 ceph-mon[29180]: pgmap v3512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:00:58.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5543 sec 2024-04-26T20:00:59.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5544 sec 2024-04-26T20:01:00.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:00:59 smithi090 ceph-mon[29180]: pgmap v3513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:00:59 smithi119 ceph-mon[32752]: pgmap v3513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:00:59 smithi167 ceph-mon[32751]: pgmap v3513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:00.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5545 sec 2024-04-26T20:01:01.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5142.053498 secs 2024-04-26T20:01:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5142.053498 secs 2024-04-26T20:01:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5142.053498 secs 2024-04-26T20:01:01.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5546 sec 2024-04-26T20:01:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:01 smithi119 ceph-mon[32752]: pgmap v3514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5142.746798 secs 2024-04-26T20:01:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:01 smithi167 ceph-mon[32751]: pgmap v3514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5142.746798 secs 2024-04-26T20:01:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:01 smithi090 ceph-mon[29180]: pgmap v3514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5142.746798 secs 2024-04-26T20:01:02.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5547 sec 2024-04-26T20:01:03.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5548 sec 2024-04-26T20:01:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:03 smithi119 ceph-mon[32752]: pgmap v3515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:03 smithi167 ceph-mon[32751]: pgmap v3515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:03 smithi090 ceph-mon[29180]: pgmap v3515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:04.383 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5549 sec 2024-04-26T20:01:05.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5550 sec 2024-04-26T20:01:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:05 smithi119 ceph-mon[32752]: pgmap v3516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5147.053558 secs 2024-04-26T20:01:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:05 smithi167 ceph-mon[32751]: pgmap v3516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5147.053558 secs 2024-04-26T20:01:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:05 smithi090 ceph-mon[29180]: pgmap v3516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5147.053558 secs 2024-04-26T20:01:06.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5551 sec 2024-04-26T20:01:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5147.746876 secs 2024-04-26T20:01:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5147.746876 secs 2024-04-26T20:01:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5147.746876 secs 2024-04-26T20:01:07.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5552 sec 2024-04-26T20:01:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:07 smithi119 ceph-mon[32752]: pgmap v3517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:07 smithi167 ceph-mon[32751]: pgmap v3517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:07 smithi090 ceph-mon[29180]: pgmap v3517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:08.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5553 sec 2024-04-26T20:01:09.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5554 sec 2024-04-26T20:01:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:09 smithi119 ceph-mon[32752]: pgmap v3518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:09 smithi167 ceph-mon[32751]: pgmap v3518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:09 smithi090 ceph-mon[29180]: pgmap v3518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:10.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5555 sec 2024-04-26T20:01:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5152.053638 secs 2024-04-26T20:01:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5152.053638 secs 2024-04-26T20:01:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5152.053638 secs 2024-04-26T20:01:11.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5556 sec 2024-04-26T20:01:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:11 smithi119 ceph-mon[32752]: pgmap v3519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5152.746970 secs 2024-04-26T20:01:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:11 smithi167 ceph-mon[32751]: pgmap v3519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5152.746970 secs 2024-04-26T20:01:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:11 smithi090 ceph-mon[29180]: pgmap v3519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5152.746970 secs 2024-04-26T20:01:12.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5557 sec 2024-04-26T20:01:13.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5558 sec 2024-04-26T20:01:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:13 smithi119 ceph-mon[32752]: pgmap v3520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:13 smithi167 ceph-mon[32751]: pgmap v3520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:13 smithi090 ceph-mon[29180]: pgmap v3520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:14.384 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5559 sec 2024-04-26T20:01:15.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5560 sec 2024-04-26T20:01:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:15 smithi090 ceph-mon[29180]: pgmap v3521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5157.053725 secs 2024-04-26T20:01:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:15 smithi119 ceph-mon[32752]: pgmap v3521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5157.053725 secs 2024-04-26T20:01:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:15 smithi167 ceph-mon[32751]: pgmap v3521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5157.053725 secs 2024-04-26T20:01:16.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5561 sec 2024-04-26T20:01:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5157.747106 secs 2024-04-26T20:01:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5157.747106 secs 2024-04-26T20:01:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5157.747106 secs 2024-04-26T20:01:17.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5562 sec 2024-04-26T20:01:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:17 smithi119 ceph-mon[32752]: pgmap v3522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:17 smithi167 ceph-mon[32751]: pgmap v3522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:17 smithi090 ceph-mon[29180]: pgmap v3522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:18.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5563 sec 2024-04-26T20:01:19.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5564 sec 2024-04-26T20:01:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:19 smithi119 ceph-mon[32752]: pgmap v3523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:19 smithi167 ceph-mon[32751]: pgmap v3523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:19 smithi090 ceph-mon[29180]: pgmap v3523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:20.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5565 sec 2024-04-26T20:01:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5162.053809 secs 2024-04-26T20:01:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5162.053809 secs 2024-04-26T20:01:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5162.053809 secs 2024-04-26T20:01:21.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5566 sec 2024-04-26T20:01:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:21 smithi119 ceph-mon[32752]: pgmap v3524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5162.747222 secs 2024-04-26T20:01:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:21 smithi167 ceph-mon[32751]: pgmap v3524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5162.747222 secs 2024-04-26T20:01:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:21 smithi090 ceph-mon[29180]: pgmap v3524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5162.747222 secs 2024-04-26T20:01:22.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5567 sec 2024-04-26T20:01:23.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5568 sec 2024-04-26T20:01:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:23 smithi119 ceph-mon[32752]: pgmap v3525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:23 smithi167 ceph-mon[32751]: pgmap v3525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:23 smithi090 ceph-mon[29180]: pgmap v3525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:24.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5569 sec 2024-04-26T20:01:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:01:25.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5570 sec 2024-04-26T20:01:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:25 smithi119 ceph-mon[32752]: pgmap v3526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5167.053827 secs 2024-04-26T20:01:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:01:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:25 smithi167 ceph-mon[32751]: pgmap v3526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5167.053827 secs 2024-04-26T20:01:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:25 smithi090 ceph-mon[29180]: pgmap v3526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5167.053827 secs 2024-04-26T20:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:01:26.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5571 sec 2024-04-26T20:01:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:26 smithi119 ceph-mon[32752]: pgmap v3527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:01:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:26 smithi119 ceph-mon[32752]: pgmap v3528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5167.747308 secs 2024-04-26T20:01:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:26 smithi167 ceph-mon[32751]: pgmap v3527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:01:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:26 smithi167 ceph-mon[32751]: pgmap v3528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5167.747308 secs 2024-04-26T20:01:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:26 smithi090 ceph-mon[29180]: pgmap v3527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:01:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:26 smithi090 ceph-mon[29180]: pgmap v3528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5167.747308 secs 2024-04-26T20:01:27.385 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5572 sec 2024-04-26T20:01:28.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5573 sec 2024-04-26T20:01:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:28 smithi119 ceph-mon[32752]: pgmap v3529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:01:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:28 smithi167 ceph-mon[32751]: pgmap v3529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:01:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:28 smithi090 ceph-mon[29180]: pgmap v3529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:01:29.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5574 sec 2024-04-26T20:01:30.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5575 sec 2024-04-26T20:01:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:30 smithi119 ceph-mon[32752]: pgmap v3530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5172.053908 secs 2024-04-26T20:01:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:30 smithi167 ceph-mon[32751]: pgmap v3530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5172.053908 secs 2024-04-26T20:01:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:30 smithi090 ceph-mon[29180]: pgmap v3530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5172.053908 secs 2024-04-26T20:01:31.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5576 sec 2024-04-26T20:01:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5172.747428 secs 2024-04-26T20:01:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5172.747428 secs 2024-04-26T20:01:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5172.747428 secs 2024-04-26T20:01:32.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5577 sec 2024-04-26T20:01:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:33 smithi119 ceph-mon[32752]: pgmap v3531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:33 smithi167 ceph-mon[32751]: pgmap v3531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:33 smithi090 ceph-mon[29180]: pgmap v3531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:01:33.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5578 sec 2024-04-26T20:01:34.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5579 sec 2024-04-26T20:01:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:35 smithi119 ceph-mon[32752]: pgmap v3532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:01:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:35 smithi167 ceph-mon[32751]: pgmap v3532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:01:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:35 smithi090 ceph-mon[29180]: pgmap v3532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:01:35.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5580 sec 2024-04-26T20:01:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5177.053992 secs 2024-04-26T20:01:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5177.053992 secs 2024-04-26T20:01:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5177.053992 secs 2024-04-26T20:01:36.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5581 sec 2024-04-26T20:01:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:37 smithi119 ceph-mon[32752]: pgmap v3533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:01:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5177.747571 secs 2024-04-26T20:01:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:37 smithi167 ceph-mon[32751]: pgmap v3533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:01:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5177.747571 secs 2024-04-26T20:01:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:37 smithi090 ceph-mon[29180]: pgmap v3533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:01:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5177.747571 secs 2024-04-26T20:01:37.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5582 sec 2024-04-26T20:01:38.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5583 sec 2024-04-26T20:01:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:39 smithi119 ceph-mon[32752]: pgmap v3534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:39 smithi167 ceph-mon[32751]: pgmap v3534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:39 smithi090 ceph-mon[29180]: pgmap v3534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:39.386 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5584 sec 2024-04-26T20:01:40.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5585 sec 2024-04-26T20:01:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:41 smithi119 ceph-mon[32752]: pgmap v3535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5182.054052 secs 2024-04-26T20:01:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:41 smithi167 ceph-mon[32751]: pgmap v3535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5182.054052 secs 2024-04-26T20:01:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:41 smithi090 ceph-mon[29180]: pgmap v3535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5182.054052 secs 2024-04-26T20:01:41.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5586 sec 2024-04-26T20:01:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5182.747692 secs 2024-04-26T20:01:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5182.747692 secs 2024-04-26T20:01:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5182.747692 secs 2024-04-26T20:01:42.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5587 sec 2024-04-26T20:01:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:43 smithi119 ceph-mon[32752]: pgmap v3536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:43 smithi167 ceph-mon[32751]: pgmap v3536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:43 smithi090 ceph-mon[29180]: pgmap v3536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:43.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5588 sec 2024-04-26T20:01:44.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5589 sec 2024-04-26T20:01:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:45 smithi119 ceph-mon[32752]: pgmap v3537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:01:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:45 smithi167 ceph-mon[32751]: pgmap v3537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:01:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:45 smithi090 ceph-mon[29180]: pgmap v3537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:01:45.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5590 sec 2024-04-26T20:01:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5187.054113 secs 2024-04-26T20:01:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5187.054113 secs 2024-04-26T20:01:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5187.054113 secs 2024-04-26T20:01:46.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5591 sec 2024-04-26T20:01:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:47 smithi119 ceph-mon[32752]: pgmap v3538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5187.747805 secs 2024-04-26T20:01:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:47 smithi167 ceph-mon[32751]: pgmap v3538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5187.747805 secs 2024-04-26T20:01:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:47 smithi090 ceph-mon[29180]: pgmap v3538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5187.747805 secs 2024-04-26T20:01:47.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5592 sec 2024-04-26T20:01:48.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5593 sec 2024-04-26T20:01:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:49 smithi119 ceph-mon[32752]: pgmap v3539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:49 smithi167 ceph-mon[32751]: pgmap v3539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:49 smithi090 ceph-mon[29180]: pgmap v3539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:49.387 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5594 sec 2024-04-26T20:01:50.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5595 sec 2024-04-26T20:01:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:51 smithi119 ceph-mon[32752]: pgmap v3540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5192.054194 secs 2024-04-26T20:01:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:51 smithi167 ceph-mon[32751]: pgmap v3540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5192.054194 secs 2024-04-26T20:01:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:51 smithi090 ceph-mon[29180]: pgmap v3540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5192.054194 secs 2024-04-26T20:01:51.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5596 sec 2024-04-26T20:01:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5192.747920 secs 2024-04-26T20:01:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5192.747920 secs 2024-04-26T20:01:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5192.747920 secs 2024-04-26T20:01:52.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5597 sec 2024-04-26T20:01:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:53 smithi119 ceph-mon[32752]: pgmap v3541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:53 smithi167 ceph-mon[32751]: pgmap v3541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:53 smithi090 ceph-mon[29180]: pgmap v3541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:53.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5598 sec 2024-04-26T20:01:54.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5599 sec 2024-04-26T20:01:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:55 smithi167 ceph-mon[32751]: pgmap v3542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:01:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:55 smithi090 ceph-mon[29180]: pgmap v3542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:01:55.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5600 sec 2024-04-26T20:01:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:55 smithi119 ceph-mon[32752]: pgmap v3542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:01:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5197.054280 secs 2024-04-26T20:01:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5197.054280 secs 2024-04-26T20:01:56.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5601 sec 2024-04-26T20:01:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5197.054280 secs 2024-04-26T20:01:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:57 smithi167 ceph-mon[32751]: pgmap v3543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5197.748023 secs 2024-04-26T20:01:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:57 smithi090 ceph-mon[29180]: pgmap v3543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5197.748023 secs 2024-04-26T20:01:57.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5602 sec 2024-04-26T20:01:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:57 smithi119 ceph-mon[32752]: pgmap v3543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:01:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5197.748023 secs 2024-04-26T20:01:58.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5603 sec 2024-04-26T20:01:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:01:59 smithi167 ceph-mon[32751]: pgmap v3544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:01:59 smithi090 ceph-mon[29180]: pgmap v3544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:01:59.388 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5604 sec 2024-04-26T20:01:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:01:59 smithi119 ceph-mon[32752]: pgmap v3544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:00.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5605 sec 2024-04-26T20:02:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:01 smithi090 ceph-mon[29180]: pgmap v3545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5202.054339 secs 2024-04-26T20:02:01.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5606 sec 2024-04-26T20:02:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:01 smithi119 ceph-mon[32752]: pgmap v3545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5202.054339 secs 2024-04-26T20:02:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:01 smithi167 ceph-mon[32751]: pgmap v3545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5202.054339 secs 2024-04-26T20:02:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5202.748156 secs 2024-04-26T20:02:02.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5607 sec 2024-04-26T20:02:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5202.748156 secs 2024-04-26T20:02:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5202.748156 secs 2024-04-26T20:02:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:03 smithi090 ceph-mon[29180]: pgmap v3546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:03.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5608 sec 2024-04-26T20:02:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:03 smithi119 ceph-mon[32752]: pgmap v3546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:03 smithi167 ceph-mon[32751]: pgmap v3546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:04.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5609 sec 2024-04-26T20:02:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:05 smithi090 ceph-mon[29180]: pgmap v3547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:05.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5610 sec 2024-04-26T20:02:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:05 smithi119 ceph-mon[32752]: pgmap v3547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:05 smithi167 ceph-mon[32751]: pgmap v3547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:06 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 5207.054445 secs 2024-04-26T20:02:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:06 smithi090 ceph-mon[29180]: slow request 722.835887 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:02:06.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5611 sec 2024-04-26T20:02:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:06 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 5207.054445 secs 2024-04-26T20:02:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:06 smithi119 ceph-mon[32752]: slow request 722.835887 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:02:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:06 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 5207.054445 secs 2024-04-26T20:02:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:06 smithi167 ceph-mon[32751]: slow request 722.835887 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:02:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:07 smithi090 ceph-mon[29180]: pgmap v3548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5207.748290 secs 2024-04-26T20:02:07.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5612 sec 2024-04-26T20:02:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:07 smithi119 ceph-mon[32752]: pgmap v3548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5207.748290 secs 2024-04-26T20:02:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:07 smithi167 ceph-mon[32751]: pgmap v3548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5207.748290 secs 2024-04-26T20:02:08.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5613 sec 2024-04-26T20:02:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:09 smithi090 ceph-mon[29180]: pgmap v3549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:09.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5614 sec 2024-04-26T20:02:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:09 smithi119 ceph-mon[32752]: pgmap v3549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:09 smithi167 ceph-mon[32751]: pgmap v3549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:10.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5615 sec 2024-04-26T20:02:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:11 smithi090 ceph-mon[29180]: pgmap v3550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5212.054526 secs 2024-04-26T20:02:11.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5616 sec 2024-04-26T20:02:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:11 smithi119 ceph-mon[32752]: pgmap v3550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5212.054526 secs 2024-04-26T20:02:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:11 smithi167 ceph-mon[32751]: pgmap v3550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5212.054526 secs 2024-04-26T20:02:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5212.748422 secs 2024-04-26T20:02:12.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5617 sec 2024-04-26T20:02:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5212.748422 secs 2024-04-26T20:02:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5212.748422 secs 2024-04-26T20:02:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:13 smithi090 ceph-mon[29180]: pgmap v3551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:13.389 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5618 sec 2024-04-26T20:02:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:13 smithi119 ceph-mon[32752]: pgmap v3551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:13 smithi167 ceph-mon[32751]: pgmap v3551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:14.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5619 sec 2024-04-26T20:02:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:15 smithi090 ceph-mon[29180]: pgmap v3552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:15.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5620 sec 2024-04-26T20:02:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:15 smithi119 ceph-mon[32752]: pgmap v3552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:15 smithi167 ceph-mon[32751]: pgmap v3552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5217.054610 secs 2024-04-26T20:02:16.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5621 sec 2024-04-26T20:02:16.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5217.054610 secs 2024-04-26T20:02:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5217.054610 secs 2024-04-26T20:02:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:17 smithi090 ceph-mon[29180]: pgmap v3553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:17.409 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5217.748473 secs 2024-04-26T20:02:17.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5622 sec 2024-04-26T20:02:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:17 smithi119 ceph-mon[32752]: pgmap v3553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:17.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5217.748473 secs 2024-04-26T20:02:17.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:17 smithi167 ceph-mon[32751]: pgmap v3553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:17.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5217.748473 secs 2024-04-26T20:02:18.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5623 sec 2024-04-26T20:02:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:19 smithi090 ceph-mon[29180]: pgmap v3554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:19.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5624 sec 2024-04-26T20:02:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:19 smithi119 ceph-mon[32752]: pgmap v3554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:19.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:19 smithi167 ceph-mon[32751]: pgmap v3554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:20.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5625 sec 2024-04-26T20:02:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:21 smithi090 ceph-mon[29180]: pgmap v3555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:21.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5222.054690 secs 2024-04-26T20:02:21.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5626 sec 2024-04-26T20:02:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:21 smithi119 ceph-mon[32752]: pgmap v3555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:21.644 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5222.054690 secs 2024-04-26T20:02:21.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:21 smithi167 ceph-mon[32751]: pgmap v3555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:21.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5222.054690 secs 2024-04-26T20:02:22.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5627 sec 2024-04-26T20:02:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5222.748587 secs 2024-04-26T20:02:22.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5222.748587 secs 2024-04-26T20:02:22.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5222.748587 secs 2024-04-26T20:02:23.390 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5628 sec 2024-04-26T20:02:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:23 smithi119 ceph-mon[32752]: pgmap v3556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:23.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:23 smithi167 ceph-mon[32751]: pgmap v3556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:23 smithi090 ceph-mon[29180]: pgmap v3556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:24.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5629 sec 2024-04-26T20:02:25.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5630 sec 2024-04-26T20:02:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:25 smithi119 ceph-mon[32752]: pgmap v3557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:25 smithi167 ceph-mon[32751]: pgmap v3557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:25 smithi090 ceph-mon[29180]: pgmap v3557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:02:26.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5631 sec 2024-04-26T20:02:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5227.054774 secs 2024-04-26T20:02:26.579 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:26.587 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:26.594 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:26.602 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:02:26.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5227.054774 secs 2024-04-26T20:02:26.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:26.626 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:26.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:26.644 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:02:26.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5227.054774 secs 2024-04-26T20:02:26.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:02:26.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:02:26.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:02:26.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:02:27.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5632 sec 2024-04-26T20:02:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:27 smithi119 ceph-mon[32752]: pgmap v3558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:27.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5227.748650 secs 2024-04-26T20:02:27.657 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:27 smithi119 ceph-mon[32752]: pgmap v3559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:02:27.669 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:27 smithi119 ceph-mon[32752]: pgmap v3560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:27.682 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:27 smithi090 ceph-mon[29180]: pgmap v3558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:27.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5227.748650 secs 2024-04-26T20:02:27.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:27 smithi090 ceph-mon[29180]: pgmap v3559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:02:27.718 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:27 smithi090 ceph-mon[29180]: pgmap v3560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:27.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:27 smithi167 ceph-mon[32751]: pgmap v3558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:27.741 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5227.748650 secs 2024-04-26T20:02:27.752 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:27 smithi167 ceph-mon[32751]: pgmap v3559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:02:27.764 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:27 smithi167 ceph-mon[32751]: pgmap v3560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:28.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5633 sec 2024-04-26T20:02:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:02:29.149+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:02:29.160 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:02:29.149+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 44 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:02:29.171 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:02:29.150+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:02:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:02:29.150+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 44 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:02:29.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5634 sec 2024-04-26T20:02:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:29 smithi119 ceph-mon[32752]: pgmap v3561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:02:29.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:29 smithi167 ceph-mon[32751]: pgmap v3561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:02:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:29 smithi090 ceph-mon[29180]: pgmap v3561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:02:30.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5635 sec 2024-04-26T20:02:31.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5636 sec 2024-04-26T20:02:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5232.054819 secs 2024-04-26T20:02:31.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:31 smithi119 ceph-mon[32752]: pgmap v3562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5232.054819 secs 2024-04-26T20:02:31.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:31 smithi167 ceph-mon[32751]: pgmap v3562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5232.054819 secs 2024-04-26T20:02:31.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:31 smithi090 ceph-mon[29180]: pgmap v3562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:32.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5637 sec 2024-04-26T20:02:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5232.748794 secs 2024-04-26T20:02:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5232.748794 secs 2024-04-26T20:02:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5232.748794 secs 2024-04-26T20:02:33.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5638 sec 2024-04-26T20:02:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:33 smithi119 ceph-mon[32752]: pgmap v3563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T20:02:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:33 smithi167 ceph-mon[32751]: pgmap v3563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T20:02:33.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:33 smithi090 ceph-mon[29180]: pgmap v3563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T20:02:34.391 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5639 sec 2024-04-26T20:02:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:34 smithi119 ceph-mon[32752]: pgmap v3564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:34.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:34 smithi167 ceph-mon[32751]: pgmap v3564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:34 smithi090 ceph-mon[29180]: pgmap v3564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:02:35.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5640 sec 2024-04-26T20:02:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5237.054907 secs 2024-04-26T20:02:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5237.054907 secs 2024-04-26T20:02:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5237.054907 secs 2024-04-26T20:02:36.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5641 sec 2024-04-26T20:02:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5237.748909 secs 2024-04-26T20:02:36.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:36 smithi090 ceph-mon[29180]: pgmap v3565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5237.748909 secs 2024-04-26T20:02:37.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:36 smithi119 ceph-mon[32752]: pgmap v3565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:02:37.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5237.748909 secs 2024-04-26T20:02:37.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:36 smithi167 ceph-mon[32751]: pgmap v3565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:02:37.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5642 sec 2024-04-26T20:02:38.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5643 sec 2024-04-26T20:02:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:39 smithi119 ceph-mon[32752]: pgmap v3566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:39 smithi167 ceph-mon[32751]: pgmap v3566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:39 smithi090 ceph-mon[29180]: pgmap v3566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:39.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5644 sec 2024-04-26T20:02:40.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5645 sec 2024-04-26T20:02:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5242.054991 secs 2024-04-26T20:02:41.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:41 smithi167 ceph-mon[32751]: pgmap v3567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5242.054991 secs 2024-04-26T20:02:41.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:41 smithi090 ceph-mon[29180]: pgmap v3567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:41.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5646 sec 2024-04-26T20:02:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5242.054991 secs 2024-04-26T20:02:41.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:41 smithi119 ceph-mon[32752]: pgmap v3567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5242.749028 secs 2024-04-26T20:02:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5242.749028 secs 2024-04-26T20:02:42.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5647 sec 2024-04-26T20:02:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5242.749028 secs 2024-04-26T20:02:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:43 smithi167 ceph-mon[32751]: pgmap v3568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:43 smithi090 ceph-mon[29180]: pgmap v3568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:43.392 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5648 sec 2024-04-26T20:02:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:43 smithi119 ceph-mon[32752]: pgmap v3568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:44.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5649 sec 2024-04-26T20:02:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:45 smithi167 ceph-mon[32751]: pgmap v3569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:45 smithi090 ceph-mon[29180]: pgmap v3569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:45.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5650 sec 2024-04-26T20:02:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:45 smithi119 ceph-mon[32752]: pgmap v3569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5247.055076 secs 2024-04-26T20:02:46.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5651 sec 2024-04-26T20:02:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5247.055076 secs 2024-04-26T20:02:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5247.055076 secs 2024-04-26T20:02:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5247.749144 secs 2024-04-26T20:02:47.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:47 smithi090 ceph-mon[29180]: pgmap v3570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:47.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5652 sec 2024-04-26T20:02:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5247.749144 secs 2024-04-26T20:02:47.582 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:47 smithi119 ceph-mon[32752]: pgmap v3570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:47.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5247.749144 secs 2024-04-26T20:02:47.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:47 smithi167 ceph-mon[32751]: pgmap v3570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:48.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5653 sec 2024-04-26T20:02:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:49 smithi090 ceph-mon[29180]: pgmap v3571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:49.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5654 sec 2024-04-26T20:02:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:49 smithi119 ceph-mon[32752]: pgmap v3571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:49 smithi167 ceph-mon[32751]: pgmap v3571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:50.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5655 sec 2024-04-26T20:02:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5252.055157 secs 2024-04-26T20:02:51.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:51 smithi090 ceph-mon[29180]: pgmap v3572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:51.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5656 sec 2024-04-26T20:02:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5252.055157 secs 2024-04-26T20:02:51.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:51 smithi119 ceph-mon[32752]: pgmap v3572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:51.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5252.055157 secs 2024-04-26T20:02:51.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:51 smithi167 ceph-mon[32751]: pgmap v3572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5252.749254 secs 2024-04-26T20:02:52.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5657 sec 2024-04-26T20:02:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5252.749254 secs 2024-04-26T20:02:52.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5252.749254 secs 2024-04-26T20:02:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:53 smithi090 ceph-mon[29180]: pgmap v3573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:53.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5658 sec 2024-04-26T20:02:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:53 smithi119 ceph-mon[32752]: pgmap v3573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:53 smithi167 ceph-mon[32751]: pgmap v3573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:54.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5659 sec 2024-04-26T20:02:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:55 smithi090 ceph-mon[29180]: pgmap v3574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:55.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5660 sec 2024-04-26T20:02:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:55 smithi119 ceph-mon[32752]: pgmap v3574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:55 smithi167 ceph-mon[32751]: pgmap v3574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:02:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5257.055243 secs 2024-04-26T20:02:56.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5661 sec 2024-04-26T20:02:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5257.055243 secs 2024-04-26T20:02:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5257.055243 secs 2024-04-26T20:02:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5257.749393 secs 2024-04-26T20:02:57.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:57 smithi090 ceph-mon[29180]: pgmap v3575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:57.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5662 sec 2024-04-26T20:02:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5257.749393 secs 2024-04-26T20:02:57.581 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:57 smithi119 ceph-mon[32752]: pgmap v3575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:57.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5257.749393 secs 2024-04-26T20:02:57.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:57 smithi167 ceph-mon[32751]: pgmap v3575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:58.394 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5663 sec 2024-04-26T20:02:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:02:59 smithi090 ceph-mon[29180]: pgmap v3576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:59.393 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5664 sec 2024-04-26T20:02:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:02:59 smithi119 ceph-mon[32752]: pgmap v3576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:02:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:02:59 smithi167 ceph-mon[32751]: pgmap v3576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:00.394 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5665 sec 2024-04-26T20:03:01.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:01 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 5262.055329 secs 2024-04-26T20:03:01.416 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:01 smithi167 ceph-mon[32751]: slow request 1444.100960 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:03:01.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:01 smithi167 ceph-mon[32751]: pgmap v3577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:01.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:01 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 5262.055329 secs 2024-04-26T20:03:01.443 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:01 smithi090 ceph-mon[29180]: slow request 1444.100960 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:03:01.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:01 smithi090 ceph-mon[29180]: pgmap v3577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:01.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5666 sec 2024-04-26T20:03:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:01 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 5262.055329 secs 2024-04-26T20:03:01.627 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:01 smithi119 ceph-mon[32752]: slow request 1444.100960 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:03:01.636 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:01 smithi119 ceph-mon[32752]: pgmap v3577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5262.749484 secs 2024-04-26T20:03:02.394 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5667 sec 2024-04-26T20:03:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5262.749484 secs 2024-04-26T20:03:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5262.749484 secs 2024-04-26T20:03:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:03 smithi090 ceph-mon[29180]: pgmap v3578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:03.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5668 sec 2024-04-26T20:03:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:03 smithi119 ceph-mon[32752]: pgmap v3578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:03 smithi167 ceph-mon[32751]: pgmap v3578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:04.394 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5669 sec 2024-04-26T20:03:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:05 smithi090 ceph-mon[29180]: pgmap v3579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:05.394 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5670 sec 2024-04-26T20:03:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:05 smithi119 ceph-mon[32752]: pgmap v3579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:05 smithi167 ceph-mon[32751]: pgmap v3579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5267.055410 secs 2024-04-26T20:03:06.394 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5671 sec 2024-04-26T20:03:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5267.055410 secs 2024-04-26T20:03:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5267.055410 secs 2024-04-26T20:03:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5267.749601 secs 2024-04-26T20:03:07.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:07 smithi090 ceph-mon[29180]: pgmap v3580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:07.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5672 sec 2024-04-26T20:03:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5267.749601 secs 2024-04-26T20:03:07.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:07 smithi119 ceph-mon[32752]: pgmap v3580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:07.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5267.749601 secs 2024-04-26T20:03:07.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:07 smithi167 ceph-mon[32751]: pgmap v3580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:08.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5673 sec 2024-04-26T20:03:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:09 smithi090 ceph-mon[29180]: pgmap v3581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:09.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5674 sec 2024-04-26T20:03:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:09 smithi119 ceph-mon[32752]: pgmap v3581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:09.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:09 smithi167 ceph-mon[32751]: pgmap v3581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:10.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5675 sec 2024-04-26T20:03:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5272.055489 secs 2024-04-26T20:03:11.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:11 smithi090 ceph-mon[29180]: pgmap v3582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:11.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5676 sec 2024-04-26T20:03:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5272.055489 secs 2024-04-26T20:03:11.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:11 smithi119 ceph-mon[32752]: pgmap v3582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5272.055489 secs 2024-04-26T20:03:11.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:11 smithi167 ceph-mon[32751]: pgmap v3582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:12.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5677 sec 2024-04-26T20:03:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5272.749713 secs 2024-04-26T20:03:12.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5272.749713 secs 2024-04-26T20:03:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5272.749713 secs 2024-04-26T20:03:13.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5678 sec 2024-04-26T20:03:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:13 smithi119 ceph-mon[32752]: pgmap v3583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:13.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:13 smithi167 ceph-mon[32751]: pgmap v3583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:13 smithi090 ceph-mon[29180]: pgmap v3583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:14.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5679 sec 2024-04-26T20:03:15.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5680 sec 2024-04-26T20:03:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:15 smithi119 ceph-mon[32752]: pgmap v3584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:15.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:15 smithi167 ceph-mon[32751]: pgmap v3584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:15 smithi090 ceph-mon[29180]: pgmap v3584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:16.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5681 sec 2024-04-26T20:03:16.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5277.055575 secs 2024-04-26T20:03:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5277.055575 secs 2024-04-26T20:03:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5277.055575 secs 2024-04-26T20:03:17.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5682 sec 2024-04-26T20:03:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5277.749831 secs 2024-04-26T20:03:17.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:17 smithi119 ceph-mon[32752]: pgmap v3585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:17.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5277.749831 secs 2024-04-26T20:03:17.628 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:17 smithi167 ceph-mon[32751]: pgmap v3585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:17.641 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5277.749831 secs 2024-04-26T20:03:17.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:17 smithi090 ceph-mon[29180]: pgmap v3585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:18.395 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5683 sec 2024-04-26T20:03:19.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5684 sec 2024-04-26T20:03:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:19 smithi119 ceph-mon[32752]: pgmap v3586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:19.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:19 smithi167 ceph-mon[32751]: pgmap v3586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:19 smithi090 ceph-mon[29180]: pgmap v3586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:20.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5685 sec 2024-04-26T20:03:21.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5686 sec 2024-04-26T20:03:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5282.055655 secs 2024-04-26T20:03:21.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:21 smithi119 ceph-mon[32752]: pgmap v3587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:21.662 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5282.055655 secs 2024-04-26T20:03:21.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:21 smithi090 ceph-mon[29180]: pgmap v3587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:21.681 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5282.055655 secs 2024-04-26T20:03:21.690 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:21 smithi167 ceph-mon[32751]: pgmap v3587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:22.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5687 sec 2024-04-26T20:03:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5282.749941 secs 2024-04-26T20:03:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5282.749941 secs 2024-04-26T20:03:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5282.749941 secs 2024-04-26T20:03:23.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5688 sec 2024-04-26T20:03:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:23 smithi119 ceph-mon[32752]: pgmap v3588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:23 smithi167 ceph-mon[32751]: pgmap v3588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:23 smithi090 ceph-mon[29180]: pgmap v3588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:24.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5689 sec 2024-04-26T20:03:25.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5690 sec 2024-04-26T20:03:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:25 smithi119 ceph-mon[32752]: pgmap v3589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:25.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:25 smithi167 ceph-mon[32751]: pgmap v3589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:25 smithi090 ceph-mon[29180]: pgmap v3589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:26.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5691 sec 2024-04-26T20:03:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5287.055740 secs 2024-04-26T20:03:26.582 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5287.750058 secs 2024-04-26T20:03:26.667 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:26 smithi119 ceph-mon[32752]: pgmap v3590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:26.685 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:26.697 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5287.055740 secs 2024-04-26T20:03:26.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5287.750058 secs 2024-04-26T20:03:26.714 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:26 smithi090 ceph-mon[29180]: pgmap v3590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:26.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:26.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5287.055740 secs 2024-04-26T20:03:26.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5287.750058 secs 2024-04-26T20:03:26.753 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:26 smithi167 ceph-mon[32751]: pgmap v3590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:26.765 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:03:27.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5692 sec 2024-04-26T20:03:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:27.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:27.575 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:27 smithi119 ceph-mon[32752]: pgmap v3591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:03:27.635 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:27 smithi119 ceph-mon[32752]: pgmap v3592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:27.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:03:27.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:27.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:27.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:27 smithi090 ceph-mon[29180]: pgmap v3591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:03:27.797 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:27 smithi090 ceph-mon[29180]: pgmap v3592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:27.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:03:27.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:03:27.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:03:27.835 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:27 smithi167 ceph-mon[32751]: pgmap v3591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:03:27.844 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:27 smithi167 ceph-mon[32751]: pgmap v3592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:27.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:03:28.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5693 sec 2024-04-26T20:03:29.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5694 sec 2024-04-26T20:03:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:29 smithi090 ceph-mon[29180]: pgmap v3593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:29 smithi119 ceph-mon[32752]: pgmap v3593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:29 smithi167 ceph-mon[32751]: pgmap v3593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:30.396 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5695 sec 2024-04-26T20:03:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5292.055824 secs 2024-04-26T20:03:31.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5292.055824 secs 2024-04-26T20:03:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5292.055824 secs 2024-04-26T20:03:31.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5696 sec 2024-04-26T20:03:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5292.750181 secs 2024-04-26T20:03:32.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:31 smithi090 ceph-mon[29180]: pgmap v3594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5292.750181 secs 2024-04-26T20:03:32.316 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:31 smithi119 ceph-mon[32752]: pgmap v3594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:32.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5292.750181 secs 2024-04-26T20:03:32.345 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:31 smithi167 ceph-mon[32751]: pgmap v3594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:32.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5697 sec 2024-04-26T20:03:33.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5698 sec 2024-04-26T20:03:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:33 smithi090 ceph-mon[29180]: pgmap v3595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T20:03:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:33 smithi119 ceph-mon[32752]: pgmap v3595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T20:03:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:33 smithi167 ceph-mon[32751]: pgmap v3595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T20:03:34.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5699 sec 2024-04-26T20:03:35.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5700 sec 2024-04-26T20:03:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:35 smithi090 ceph-mon[29180]: pgmap v3596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:36.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5297.055912 secs 2024-04-26T20:03:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:35 smithi119 ceph-mon[32752]: pgmap v3596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:36.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5297.055912 secs 2024-04-26T20:03:36.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:35 smithi167 ceph-mon[32751]: pgmap v3596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:03:36.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5297.055912 secs 2024-04-26T20:03:36.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5701 sec 2024-04-26T20:03:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5297.750315 secs 2024-04-26T20:03:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5297.750315 secs 2024-04-26T20:03:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5297.750315 secs 2024-04-26T20:03:37.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5702 sec 2024-04-26T20:03:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:37 smithi119 ceph-mon[32752]: pgmap v3597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:03:38.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:37 smithi167 ceph-mon[32751]: pgmap v3597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:03:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:37 smithi090 ceph-mon[29180]: pgmap v3597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:03:38.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5703 sec 2024-04-26T20:03:39.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5704 sec 2024-04-26T20:03:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:39 smithi119 ceph-mon[32752]: pgmap v3598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:40.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:39 smithi167 ceph-mon[32751]: pgmap v3598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:39 smithi090 ceph-mon[29180]: pgmap v3598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:40.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5705 sec 2024-04-26T20:03:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5302.055997 secs 2024-04-26T20:03:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5302.055997 secs 2024-04-26T20:03:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5302.055997 secs 2024-04-26T20:03:41.397 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5706 sec 2024-04-26T20:03:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5302.750383 secs 2024-04-26T20:03:42.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:41 smithi119 ceph-mon[32752]: pgmap v3599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:42.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5302.750383 secs 2024-04-26T20:03:42.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:41 smithi167 ceph-mon[32751]: pgmap v3599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5302.750383 secs 2024-04-26T20:03:42.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:41 smithi090 ceph-mon[29180]: pgmap v3599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:42.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5707 sec 2024-04-26T20:03:43.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5708 sec 2024-04-26T20:03:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:43 smithi119 ceph-mon[32752]: pgmap v3600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:43 smithi167 ceph-mon[32751]: pgmap v3600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:43 smithi090 ceph-mon[29180]: pgmap v3600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:44.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5709 sec 2024-04-26T20:03:45.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5710 sec 2024-04-26T20:03:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:45 smithi119 ceph-mon[32752]: pgmap v3601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:46.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5307.056060 secs 2024-04-26T20:03:46.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:45 smithi167 ceph-mon[32751]: pgmap v3601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:46.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5307.056060 secs 2024-04-26T20:03:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:45 smithi090 ceph-mon[29180]: pgmap v3601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:46.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5307.056060 secs 2024-04-26T20:03:46.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5711 sec 2024-04-26T20:03:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5307.750490 secs 2024-04-26T20:03:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5307.750490 secs 2024-04-26T20:03:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5307.750490 secs 2024-04-26T20:03:47.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5712 sec 2024-04-26T20:03:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:47 smithi119 ceph-mon[32752]: pgmap v3602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:48.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:47 smithi167 ceph-mon[32751]: pgmap v3602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:48.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:47 smithi090 ceph-mon[29180]: pgmap v3602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:48.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5713 sec 2024-04-26T20:03:49.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5714 sec 2024-04-26T20:03:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:49 smithi119 ceph-mon[32752]: pgmap v3603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:49 smithi167 ceph-mon[32751]: pgmap v3603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:49 smithi090 ceph-mon[29180]: pgmap v3603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:50.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5715 sec 2024-04-26T20:03:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5312.056119 secs 2024-04-26T20:03:51.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5312.056119 secs 2024-04-26T20:03:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5312.056119 secs 2024-04-26T20:03:51.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5716 sec 2024-04-26T20:03:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5312.750610 secs 2024-04-26T20:03:52.318 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:51 smithi119 ceph-mon[32752]: pgmap v3604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:52.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5312.750610 secs 2024-04-26T20:03:52.334 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:51 smithi167 ceph-mon[32751]: pgmap v3604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5312.750610 secs 2024-04-26T20:03:52.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:51 smithi090 ceph-mon[29180]: pgmap v3604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:52.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5717 sec 2024-04-26T20:03:53.398 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5718 sec 2024-04-26T20:03:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:53 smithi119 ceph-mon[32752]: pgmap v3605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:54.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:53 smithi167 ceph-mon[32751]: pgmap v3605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:53 smithi090 ceph-mon[29180]: pgmap v3605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:54.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5719 sec 2024-04-26T20:03:55.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5720 sec 2024-04-26T20:03:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5317.056205 secs 2024-04-26T20:03:56.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:55 smithi119 ceph-mon[32752]: pgmap v3606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:56.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5317.056205 secs 2024-04-26T20:03:56.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:55 smithi167 ceph-mon[32751]: pgmap v3606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5317.056205 secs 2024-04-26T20:03:56.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:55 smithi090 ceph-mon[29180]: pgmap v3606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:03:56.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5721 sec 2024-04-26T20:03:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5317.750746 secs 2024-04-26T20:03:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5317.750746 secs 2024-04-26T20:03:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5317.750746 secs 2024-04-26T20:03:57.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5722 sec 2024-04-26T20:03:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:57 smithi119 ceph-mon[32752]: pgmap v3607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:58.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:57 smithi167 ceph-mon[32751]: pgmap v3607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:57 smithi090 ceph-mon[29180]: pgmap v3607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:03:58.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5723 sec 2024-04-26T20:03:59.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5724 sec 2024-04-26T20:04:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:03:59 smithi119 ceph-mon[32752]: pgmap v3608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:03:59 smithi167 ceph-mon[32751]: pgmap v3608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:03:59 smithi090 ceph-mon[29180]: pgmap v3608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:00.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5725 sec 2024-04-26T20:04:01.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5322.056263 secs 2024-04-26T20:04:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5322.056263 secs 2024-04-26T20:04:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5322.056263 secs 2024-04-26T20:04:01.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5726 sec 2024-04-26T20:04:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5322.750810 secs 2024-04-26T20:04:02.323 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:01 smithi090 ceph-mon[29180]: pgmap v3609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:02.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5322.750810 secs 2024-04-26T20:04:02.341 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:01 smithi167 ceph-mon[32751]: pgmap v3609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:02.352 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5322.750810 secs 2024-04-26T20:04:02.364 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:01 smithi119 ceph-mon[32752]: pgmap v3609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:02.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5727 sec 2024-04-26T20:04:03.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5728 sec 2024-04-26T20:04:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:03 smithi119 ceph-mon[32752]: pgmap v3610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:03 smithi167 ceph-mon[32751]: pgmap v3610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:03 smithi090 ceph-mon[29180]: pgmap v3610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:04.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5729 sec 2024-04-26T20:04:05.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5730 sec 2024-04-26T20:04:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5327.056346 secs 2024-04-26T20:04:06.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:05 smithi119 ceph-mon[32752]: pgmap v3611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5327.056346 secs 2024-04-26T20:04:06.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:05 smithi167 ceph-mon[32751]: pgmap v3611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5327.056346 secs 2024-04-26T20:04:06.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:05 smithi090 ceph-mon[29180]: pgmap v3611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:06.399 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5731 sec 2024-04-26T20:04:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5327.750923 secs 2024-04-26T20:04:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5327.750923 secs 2024-04-26T20:04:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5327.750923 secs 2024-04-26T20:04:07.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5732 sec 2024-04-26T20:04:07.965 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:07 smithi119 ceph-mon[32752]: pgmap v3612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:07 smithi167 ceph-mon[32751]: pgmap v3612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:07 smithi090 ceph-mon[29180]: pgmap v3612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:08.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5733 sec 2024-04-26T20:04:09.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5734 sec 2024-04-26T20:04:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:09 smithi119 ceph-mon[32752]: pgmap v3613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:09 smithi167 ceph-mon[32751]: pgmap v3613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:09 smithi090 ceph-mon[29180]: pgmap v3613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:10.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5735 sec 2024-04-26T20:04:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5332.056428 secs 2024-04-26T20:04:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5332.056428 secs 2024-04-26T20:04:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5332.056428 secs 2024-04-26T20:04:11.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5736 sec 2024-04-26T20:04:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5332.751042 secs 2024-04-26T20:04:12.315 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:11 smithi119 ceph-mon[32752]: pgmap v3614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:12.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5332.751042 secs 2024-04-26T20:04:12.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:11 smithi167 ceph-mon[32751]: pgmap v3614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5332.751042 secs 2024-04-26T20:04:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:11 smithi090 ceph-mon[29180]: pgmap v3614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:12.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5737 sec 2024-04-26T20:04:13.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5738 sec 2024-04-26T20:04:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:13 smithi119 ceph-mon[32752]: pgmap v3615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:13 smithi167 ceph-mon[32751]: pgmap v3615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:13 smithi090 ceph-mon[29180]: pgmap v3615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:14.400 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5739 sec 2024-04-26T20:04:15.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5740 sec 2024-04-26T20:04:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5337.056531 secs 2024-04-26T20:04:16.318 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:15 smithi119 ceph-mon[32752]: pgmap v3616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:16.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5337.056531 secs 2024-04-26T20:04:16.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:15 smithi167 ceph-mon[32751]: pgmap v3616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5337.056531 secs 2024-04-26T20:04:16.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:15 smithi090 ceph-mon[29180]: pgmap v3616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:16.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5741 sec 2024-04-26T20:04:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5337.751158 secs 2024-04-26T20:04:17.409 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5337.751158 secs 2024-04-26T20:04:17.417 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5337.751158 secs 2024-04-26T20:04:17.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5742 sec 2024-04-26T20:04:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:17 smithi119 ceph-mon[32752]: pgmap v3617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:17 smithi167 ceph-mon[32751]: pgmap v3617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:17 smithi090 ceph-mon[29180]: pgmap v3617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:18.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5743 sec 2024-04-26T20:04:19.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5744 sec 2024-04-26T20:04:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:19 smithi119 ceph-mon[32752]: pgmap v3618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:19 smithi167 ceph-mon[32751]: pgmap v3618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:19 smithi090 ceph-mon[29180]: pgmap v3618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:20.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5745 sec 2024-04-26T20:04:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5342.056613 secs 2024-04-26T20:04:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5342.056613 secs 2024-04-26T20:04:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5342.056613 secs 2024-04-26T20:04:21.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5746 sec 2024-04-26T20:04:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5342.751296 secs 2024-04-26T20:04:22.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:21 smithi119 ceph-mon[32752]: pgmap v3619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:22.339 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5342.751296 secs 2024-04-26T20:04:22.351 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:21 smithi167 ceph-mon[32751]: pgmap v3619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:22.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5342.751296 secs 2024-04-26T20:04:22.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:21 smithi090 ceph-mon[29180]: pgmap v3619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:22.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5747 sec 2024-04-26T20:04:23.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5748 sec 2024-04-26T20:04:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:23 smithi119 ceph-mon[32752]: pgmap v3620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:23 smithi167 ceph-mon[32751]: pgmap v3620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:23 smithi090 ceph-mon[29180]: pgmap v3620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:24.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5749 sec 2024-04-26T20:04:25.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5750 sec 2024-04-26T20:04:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5347.056698 secs 2024-04-26T20:04:26.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:26 smithi119 ceph-mon[32752]: pgmap v3621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:26.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5347.056698 secs 2024-04-26T20:04:26.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:26 smithi167 ceph-mon[32751]: pgmap v3621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5347.056698 secs 2024-04-26T20:04:26.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:26 smithi090 ceph-mon[29180]: pgmap v3621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:26.401 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5751 sec 2024-04-26T20:04:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5347.751369 secs 2024-04-26T20:04:27.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5347.751369 secs 2024-04-26T20:04:27.442 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:27.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5347.751369 secs 2024-04-26T20:04:27.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:04:27.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5752 sec 2024-04-26T20:04:28.259 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:28 smithi119 ceph-mon[32752]: pgmap v3622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:28.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.317 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.432 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:28 smithi090 ceph-mon[29180]: pgmap v3622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:28.456 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.468 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.479 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.488 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:28 smithi167 ceph-mon[32751]: pgmap v3622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:28.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.525 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.537 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.611 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:28.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5753 sec 2024-04-26T20:04:29.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5754 sec 2024-04-26T20:04:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: pgmap v3623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:30.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.380 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.387 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.394 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.401 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.408 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.414 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.422 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.430 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: pgmap v3623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:30.438 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.532 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.540 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.555 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: pgmap v3623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:30.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.735 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.750 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.757 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.764 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:30.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.786 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:30.793 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5755 sec 2024-04-26T20:04:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:04:31.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:04:31.332 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5352.056764 secs 2024-04-26T20:04:31.341 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.388 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.507 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.516 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.623 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:31.634 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:31.669 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:31.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:04:31.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:04:31.702 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5352.056764 secs 2024-04-26T20:04:31.710 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.773 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.791 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:31.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:31.902 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:31.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:31.922 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:04:31.930 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:04:31.939 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5352.056764 secs 2024-04-26T20:04:31.947 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.001 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.012 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.023 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.090 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.098 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.194 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:04:32.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:04:32.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:04:32.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:04:32.234 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5756 sec 2024-04-26T20:04:32.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:04:32.342 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:04:32.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:04:32.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:04:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:32 smithi090 ceph-mon[29180]: pgmap v3624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-26T20:04:32.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:32 smithi090 ceph-mon[29180]: pgmap v3625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:32.400 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5352.751445 secs 2024-04-26T20:04:32.411 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:04:32.421 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:04:32.490 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:04:32.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:04:32.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:32 smithi167 ceph-mon[32751]: pgmap v3624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-26T20:04:32.754 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:32 smithi167 ceph-mon[32751]: pgmap v3625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:32.763 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5352.751445 secs 2024-04-26T20:04:32.772 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:04:32.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:04:32.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:04:32.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:04:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:32 smithi119 ceph-mon[32752]: pgmap v3624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-26T20:04:32.818 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:32 smithi119 ceph-mon[32752]: pgmap v3625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:32.825 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5352.751445 secs 2024-04-26T20:04:32.832 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5757 sec 2024-04-26T20:04:33.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5758 sec 2024-04-26T20:04:34.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:34 smithi119 ceph-mon[32752]: pgmap v3626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:34.352 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:34 smithi167 ceph-mon[32751]: pgmap v3626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:34 smithi090 ceph-mon[29180]: pgmap v3626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:34.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5759 sec 2024-04-26T20:04:35.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5760 sec 2024-04-26T20:04:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:36 smithi119 ceph-mon[32752]: pgmap v3627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:36.364 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5357.056849 secs 2024-04-26T20:04:36.374 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:36 smithi167 ceph-mon[32751]: pgmap v3627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:36.384 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5357.056849 secs 2024-04-26T20:04:36.442 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:36 smithi090 ceph-mon[29180]: pgmap v3627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:36.451 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5357.056849 secs 2024-04-26T20:04:36.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5761 sec 2024-04-26T20:04:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5357.751538 secs 2024-04-26T20:04:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5357.751538 secs 2024-04-26T20:04:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5357.751538 secs 2024-04-26T20:04:37.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5762 sec 2024-04-26T20:04:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:38 smithi119 ceph-mon[32752]: pgmap v3628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:38.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:38 smithi167 ceph-mon[32751]: pgmap v3628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:38 smithi090 ceph-mon[29180]: pgmap v3628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:38.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5763 sec 2024-04-26T20:04:39.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5764 sec 2024-04-26T20:04:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:40 smithi167 ceph-mon[32751]: pgmap v3629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:40 smithi090 ceph-mon[29180]: pgmap v3629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:40.402 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5765 sec 2024-04-26T20:04:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:40 smithi119 ceph-mon[32752]: pgmap v3629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:04:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5362.056909 secs 2024-04-26T20:04:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5362.056909 secs 2024-04-26T20:04:41.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5766 sec 2024-04-26T20:04:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5362.056909 secs 2024-04-26T20:04:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:42 smithi167 ceph-mon[32751]: pgmap v3630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:04:42.335 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5362.751652 secs 2024-04-26T20:04:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:42 smithi090 ceph-mon[29180]: pgmap v3630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:04:42.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5362.751652 secs 2024-04-26T20:04:42.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5767 sec 2024-04-26T20:04:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:42 smithi119 ceph-mon[32752]: pgmap v3630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:04:42.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5362.751652 secs 2024-04-26T20:04:43.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5768 sec 2024-04-26T20:04:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:44 smithi167 ceph-mon[32751]: pgmap v3631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:44 smithi090 ceph-mon[29180]: pgmap v3631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:44.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5769 sec 2024-04-26T20:04:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:44 smithi119 ceph-mon[32752]: pgmap v3631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:45.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5770 sec 2024-04-26T20:04:46.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:46 smithi119 ceph-mon[32752]: pgmap v3632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:46.390 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5367.056993 secs 2024-04-26T20:04:46.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:46 smithi090 ceph-mon[29180]: pgmap v3632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:46.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5367.056993 secs 2024-04-26T20:04:46.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5771 sec 2024-04-26T20:04:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:46 smithi167 ceph-mon[32751]: pgmap v3632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:46.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5367.056993 secs 2024-04-26T20:04:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5367.751730 secs 2024-04-26T20:04:47.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5772 sec 2024-04-26T20:04:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5367.751730 secs 2024-04-26T20:04:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5367.751730 secs 2024-04-26T20:04:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:48 smithi090 ceph-mon[29180]: pgmap v3633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:48.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5773 sec 2024-04-26T20:04:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:48 smithi119 ceph-mon[32752]: pgmap v3633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:48 smithi167 ceph-mon[32751]: pgmap v3633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:49.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5774 sec 2024-04-26T20:04:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:50 smithi090 ceph-mon[29180]: pgmap v3634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:50.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5775 sec 2024-04-26T20:04:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:50 smithi119 ceph-mon[32752]: pgmap v3634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:50 smithi167 ceph-mon[32751]: pgmap v3634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5372.057051 secs 2024-04-26T20:04:51.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5776 sec 2024-04-26T20:04:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5372.057051 secs 2024-04-26T20:04:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5372.057051 secs 2024-04-26T20:04:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:52 smithi090 ceph-mon[29180]: pgmap v3635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:52.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5372.751805 secs 2024-04-26T20:04:52.403 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5777 sec 2024-04-26T20:04:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:52 smithi119 ceph-mon[32752]: pgmap v3635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:52.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5372.751805 secs 2024-04-26T20:04:52.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:52 smithi167 ceph-mon[32751]: pgmap v3635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:52.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5372.751805 secs 2024-04-26T20:04:53.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5778 sec 2024-04-26T20:04:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:54 smithi090 ceph-mon[29180]: pgmap v3636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:54.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5779 sec 2024-04-26T20:04:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:54 smithi119 ceph-mon[32752]: pgmap v3636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:54 smithi167 ceph-mon[32751]: pgmap v3636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:55.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5780 sec 2024-04-26T20:04:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:56 smithi090 ceph-mon[29180]: pgmap v3637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:56.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5377.057134 secs 2024-04-26T20:04:56.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5781 sec 2024-04-26T20:04:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:56 smithi119 ceph-mon[32752]: pgmap v3637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:56.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5377.057134 secs 2024-04-26T20:04:56.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:56 smithi167 ceph-mon[32751]: pgmap v3637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:04:56.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5377.057134 secs 2024-04-26T20:04:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5377.751925 secs 2024-04-26T20:04:57.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5782 sec 2024-04-26T20:04:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5377.751925 secs 2024-04-26T20:04:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5377.751925 secs 2024-04-26T20:04:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:04:58 smithi090 ceph-mon[29180]: pgmap v3638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:58.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5783 sec 2024-04-26T20:04:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:04:58 smithi119 ceph-mon[32752]: pgmap v3638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:04:58 smithi167 ceph-mon[32751]: pgmap v3638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:04:59.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5784 sec 2024-04-26T20:05:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:00 smithi090 ceph-mon[29180]: pgmap v3639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:00.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5785 sec 2024-04-26T20:05:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:00 smithi119 ceph-mon[32752]: pgmap v3639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:00 smithi167 ceph-mon[32751]: pgmap v3639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5382.057216 secs 2024-04-26T20:05:01.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5786 sec 2024-04-26T20:05:01.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5382.057216 secs 2024-04-26T20:05:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5382.057216 secs 2024-04-26T20:05:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:02 smithi090 ceph-mon[29180]: pgmap v3640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:02.421 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5382.751998 secs 2024-04-26T20:05:02.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5787 sec 2024-04-26T20:05:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:02 smithi119 ceph-mon[32752]: pgmap v3640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:02.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5382.751998 secs 2024-04-26T20:05:02.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:02 smithi167 ceph-mon[32751]: pgmap v3640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:02.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5382.751998 secs 2024-04-26T20:05:03.404 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5788 sec 2024-04-26T20:05:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:04 smithi090 ceph-mon[29180]: pgmap v3641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:04.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5789 sec 2024-04-26T20:05:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:04 smithi119 ceph-mon[32752]: pgmap v3641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:04.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:04 smithi167 ceph-mon[32751]: pgmap v3641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:05.405 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5790 sec 2024-04-26T20:05:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:06 smithi090 ceph-mon[29180]: pgmap v3642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:06.441 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5387.057276 secs 2024-04-26T20:05:06.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5791 sec 2024-04-26T20:05:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:06 smithi119 ceph-mon[32752]: pgmap v3642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:06.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5387.057276 secs 2024-04-26T20:05:06.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:06 smithi167 ceph-mon[32751]: pgmap v3642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:06.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5387.057276 secs 2024-04-26T20:05:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5387.752119 secs 2024-04-26T20:05:07.405 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5792 sec 2024-04-26T20:05:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5387.752119 secs 2024-04-26T20:05:07.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5387.752119 secs 2024-04-26T20:05:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:08 smithi090 ceph-mon[29180]: pgmap v3643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:08.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5793 sec 2024-04-26T20:05:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:08 smithi119 ceph-mon[32752]: pgmap v3643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:08.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:08 smithi167 ceph-mon[32751]: pgmap v3643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:09.405 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5794 sec 2024-04-26T20:05:10.405 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5795 sec 2024-04-26T20:05:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:10 smithi119 ceph-mon[32752]: pgmap v3644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:10 smithi167 ceph-mon[32751]: pgmap v3644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:10 smithi090 ceph-mon[29180]: pgmap v3644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:11.405 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5796 sec 2024-04-26T20:05:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5392.057356 secs 2024-04-26T20:05:11.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5392.057356 secs 2024-04-26T20:05:11.661 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5392.057356 secs 2024-04-26T20:05:12.405 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5797 sec 2024-04-26T20:05:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:12 smithi119 ceph-mon[32752]: pgmap v3645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:12.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5392.752247 secs 2024-04-26T20:05:12.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:12 smithi167 ceph-mon[32751]: pgmap v3645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:12.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5392.752247 secs 2024-04-26T20:05:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:12 smithi090 ceph-mon[29180]: pgmap v3645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:12.746 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5392.752247 secs 2024-04-26T20:05:13.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5798 sec 2024-04-26T20:05:14.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5799 sec 2024-04-26T20:05:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:14 smithi119 ceph-mon[32752]: pgmap v3646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:14.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:14 smithi167 ceph-mon[32751]: pgmap v3646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:14.676 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:14 smithi090 ceph-mon[29180]: pgmap v3646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:15.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5800 sec 2024-04-26T20:05:16.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5801 sec 2024-04-26T20:05:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:16 smithi119 ceph-mon[32752]: pgmap v3647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:16.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5397.057442 secs 2024-04-26T20:05:16.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:16 smithi167 ceph-mon[32751]: pgmap v3647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:16.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5397.057442 secs 2024-04-26T20:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:16 smithi090 ceph-mon[29180]: pgmap v3647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:16.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5397.057442 secs 2024-04-26T20:05:17.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5802 sec 2024-04-26T20:05:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5397.752399 secs 2024-04-26T20:05:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5397.752399 secs 2024-04-26T20:05:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5397.752399 secs 2024-04-26T20:05:18.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5803 sec 2024-04-26T20:05:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:18 smithi119 ceph-mon[32752]: pgmap v3648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:18.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:18 smithi167 ceph-mon[32751]: pgmap v3648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:18.706 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:18 smithi090 ceph-mon[29180]: pgmap v3648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:19.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5804 sec 2024-04-26T20:05:20.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5805 sec 2024-04-26T20:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:20 smithi119 ceph-mon[32752]: pgmap v3649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:20.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:20 smithi167 ceph-mon[32751]: pgmap v3649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:20.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:20 smithi090 ceph-mon[29180]: pgmap v3649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:21.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5806 sec 2024-04-26T20:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5402.057498 secs 2024-04-26T20:05:21.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5402.057498 secs 2024-04-26T20:05:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5402.057498 secs 2024-04-26T20:05:22.406 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5807 sec 2024-04-26T20:05:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:22 smithi119 ceph-mon[32752]: pgmap v3650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:22.584 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5402.752481 secs 2024-04-26T20:05:22.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:22 smithi090 ceph-mon[29180]: pgmap v3650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:22.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5402.752481 secs 2024-04-26T20:05:22.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:22 smithi167 ceph-mon[32751]: pgmap v3650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:22.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5402.752481 secs 2024-04-26T20:05:23.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5808 sec 2024-04-26T20:05:24.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5809 sec 2024-04-26T20:05:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:24 smithi119 ceph-mon[32752]: pgmap v3651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:24 smithi167 ceph-mon[32751]: pgmap v3651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:24 smithi090 ceph-mon[29180]: pgmap v3651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:25.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5810 sec 2024-04-26T20:05:26.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5811 sec 2024-04-26T20:05:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:26 smithi119 ceph-mon[32752]: pgmap v3652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:26.591 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5407.057581 secs 2024-04-26T20:05:26.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:26 smithi090 ceph-mon[29180]: pgmap v3652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:26.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5407.057581 secs 2024-04-26T20:05:26.782 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:26 smithi167 ceph-mon[32751]: pgmap v3652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:26.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5407.057581 secs 2024-04-26T20:05:27.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5812 sec 2024-04-26T20:05:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5407.752611 secs 2024-04-26T20:05:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5407.752611 secs 2024-04-26T20:05:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5407.752611 secs 2024-04-26T20:05:28.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5813 sec 2024-04-26T20:05:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T20:05:28.535 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T20:05:28.602 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b47d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010ce0 tx=0x7f2cc4038040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:05:28.657 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b47d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4010ce0 tx=0x7f2cc4038040 comp rx=0 tx=0).stop 2024-04-26T20:05:28.688 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b47d0 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T20:05:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b47d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T20:05:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b47d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:05:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2280 0x7f2cc00b4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:05:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.470+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2280 0x7f2cc00b4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:05:29.076 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.471+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2280 0x7f2cc00b4d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401c010 tx=0x7f2cc4029040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:05:29.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:28 smithi167 ceph-mon[32751]: pgmap v3653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:29.183 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:28 smithi119 ceph-mon[32752]: pgmap v3653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:29.231 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.491+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000978) for more than 900000000 us, fault. 2024-04-26T20:05:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.491+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=4 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T20:05:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.491+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:05:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.491+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T20:05:29.404 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:28.491+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T20:05:29.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:28 smithi090 ceph-mon[29180]: pgmap v3653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:29.478 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:29.174+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:05:29.487 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:29.174+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:05:29.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:29.174+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 45 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:05:29.507 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:05:29.174+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 45 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:05:29.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5814 sec 2024-04-26T20:05:30.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5815 sec 2024-04-26T20:05:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:30 smithi119 ceph-mon[32752]: pgmap v3654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:30.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:30 smithi090 ceph-mon[29180]: pgmap v3654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:30.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:30 smithi167 ceph-mon[32751]: pgmap v3654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:31.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5816 sec 2024-04-26T20:05:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5412.057651 secs 2024-04-26T20:05:31.517 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:31.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:31.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:31.611 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5412.057651 secs 2024-04-26T20:05:31.623 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:31.713 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:31.756 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:31.812 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5412.057651 secs 2024-04-26T20:05:31.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:05:31.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:32.407 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5817 sec 2024-04-26T20:05:32.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:32 smithi090 ceph-mon[29180]: pgmap v3655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:32.576 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5412.752692 secs 2024-04-26T20:05:32.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:32.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:32.724 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:32 smithi090 ceph-mon[29180]: pgmap v3656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T20:05:32.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:32 smithi090 ceph-mon[29180]: pgmap v3657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:32.740 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:32.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:32 smithi167 ceph-mon[32751]: pgmap v3655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:32.851 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5412.752692 secs 2024-04-26T20:05:32.898 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:32.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:33.019 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:32 smithi167 ceph-mon[32751]: pgmap v3656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T20:05:33.026 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:32 smithi167 ceph-mon[32751]: pgmap v3657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:33.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:33.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:32 smithi119 ceph-mon[32752]: pgmap v3655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:33.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5412.752692 secs 2024-04-26T20:05:33.165 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:05:33.198 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:05:33.235 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:32 smithi119 ceph-mon[32752]: pgmap v3656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T20:05:33.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:32 smithi119 ceph-mon[32752]: pgmap v3657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:33.384 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:05:33.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5818 sec 2024-04-26T20:05:34.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5819 sec 2024-04-26T20:05:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:34 smithi119 ceph-mon[32752]: pgmap v3658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:35.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:34 smithi167 ceph-mon[32751]: pgmap v3658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:35.201 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:34 smithi090 ceph-mon[29180]: pgmap v3658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:35.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5820 sec 2024-04-26T20:05:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5417.057716 secs 2024-04-26T20:05:36.183 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5417.057716 secs 2024-04-26T20:05:36.232 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5417.057716 secs 2024-04-26T20:05:36.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5821 sec 2024-04-26T20:05:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5417.752808 secs 2024-04-26T20:05:37.104 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:36 smithi119 ceph-mon[32752]: pgmap v3659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:37.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5417.752808 secs 2024-04-26T20:05:37.199 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:36 smithi090 ceph-mon[29180]: pgmap v3659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:37.253 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5417.752808 secs 2024-04-26T20:05:37.280 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:36 smithi167 ceph-mon[32751]: pgmap v3659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:37.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5822 sec 2024-04-26T20:05:38.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5823 sec 2024-04-26T20:05:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:38 smithi119 ceph-mon[32752]: pgmap v3660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T20:05:39.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:38 smithi167 ceph-mon[32751]: pgmap v3660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T20:05:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:38 smithi090 ceph-mon[29180]: pgmap v3660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T20:05:39.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5824 sec 2024-04-26T20:05:40.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5825 sec 2024-04-26T20:05:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:40 smithi090 ceph-mon[29180]: pgmap v3661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:40.909 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5422.057813 secs 2024-04-26T20:05:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:40 smithi119 ceph-mon[32752]: pgmap v3661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:41.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5422.057813 secs 2024-04-26T20:05:41.132 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:40 smithi167 ceph-mon[32751]: pgmap v3661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:05:41.159 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5422.057813 secs 2024-04-26T20:05:41.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5826 sec 2024-04-26T20:05:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5422.752925 secs 2024-04-26T20:05:42.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5422.752925 secs 2024-04-26T20:05:42.180 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5422.752925 secs 2024-04-26T20:05:42.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5827 sec 2024-04-26T20:05:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:42 smithi119 ceph-mon[32752]: pgmap v3662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:05:43.106 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:42 smithi167 ceph-mon[32751]: pgmap v3662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:05:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:42 smithi090 ceph-mon[29180]: pgmap v3662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:05:43.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5828 sec 2024-04-26T20:05:44.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5829 sec 2024-04-26T20:05:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:44 smithi119 ceph-mon[32752]: pgmap v3663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:44 smithi167 ceph-mon[32751]: pgmap v3663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:44 smithi090 ceph-mon[29180]: pgmap v3663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:45.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5830 sec 2024-04-26T20:05:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5427.057901 secs 2024-04-26T20:05:46.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5427.057901 secs 2024-04-26T20:05:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5427.057901 secs 2024-04-26T20:05:46.408 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5831 sec 2024-04-26T20:05:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5427.753042 secs 2024-04-26T20:05:47.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:46 smithi119 ceph-mon[32752]: pgmap v3664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:47.141 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5427.753042 secs 2024-04-26T20:05:47.177 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:46 smithi090 ceph-mon[29180]: pgmap v3664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:47.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5427.753042 secs 2024-04-26T20:05:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:46 smithi167 ceph-mon[32751]: pgmap v3664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:47.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5832 sec 2024-04-26T20:05:48.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5833 sec 2024-04-26T20:05:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:48 smithi119 ceph-mon[32752]: pgmap v3665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:48 smithi167 ceph-mon[32751]: pgmap v3665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:48 smithi090 ceph-mon[29180]: pgmap v3665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:49.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5834 sec 2024-04-26T20:05:50.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5835 sec 2024-04-26T20:05:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:50 smithi119 ceph-mon[32752]: pgmap v3666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:51.082 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5432.057958 secs 2024-04-26T20:05:51.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:50 smithi167 ceph-mon[32751]: pgmap v3666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:51.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5432.057958 secs 2024-04-26T20:05:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:50 smithi090 ceph-mon[29180]: pgmap v3666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5432.057958 secs 2024-04-26T20:05:51.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5836 sec 2024-04-26T20:05:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5432.753133 secs 2024-04-26T20:05:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5432.753133 secs 2024-04-26T20:05:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5432.753133 secs 2024-04-26T20:05:52.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5837 sec 2024-04-26T20:05:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:52 smithi119 ceph-mon[32752]: pgmap v3667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:52 smithi167 ceph-mon[32751]: pgmap v3667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:52 smithi090 ceph-mon[29180]: pgmap v3667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:53.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5838 sec 2024-04-26T20:05:54.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5839 sec 2024-04-26T20:05:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:54 smithi119 ceph-mon[32752]: pgmap v3668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:54 smithi167 ceph-mon[32751]: pgmap v3668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:54 smithi090 ceph-mon[29180]: pgmap v3668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:05:55.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5840 sec 2024-04-26T20:05:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5437.058038 secs 2024-04-26T20:05:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5437.058038 secs 2024-04-26T20:05:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5437.058038 secs 2024-04-26T20:05:56.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5841 sec 2024-04-26T20:05:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5437.753245 secs 2024-04-26T20:05:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:56 smithi119 ceph-mon[32752]: pgmap v3669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5437.753245 secs 2024-04-26T20:05:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:56 smithi167 ceph-mon[32751]: pgmap v3669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5437.753245 secs 2024-04-26T20:05:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:56 smithi090 ceph-mon[29180]: pgmap v3669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:57.409 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5842 sec 2024-04-26T20:05:58.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5843 sec 2024-04-26T20:05:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:05:58 smithi119 ceph-mon[32752]: pgmap v3670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:05:58 smithi167 ceph-mon[32751]: pgmap v3670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:05:58 smithi090 ceph-mon[29180]: pgmap v3670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:05:59.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5844 sec 2024-04-26T20:06:00.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5845 sec 2024-04-26T20:06:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:00 smithi119 ceph-mon[32752]: pgmap v3671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5442.058117 secs 2024-04-26T20:06:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:00 smithi167 ceph-mon[32751]: pgmap v3671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5442.058117 secs 2024-04-26T20:06:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:00 smithi090 ceph-mon[29180]: pgmap v3671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5442.058117 secs 2024-04-26T20:06:01.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5846 sec 2024-04-26T20:06:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5442.753337 secs 2024-04-26T20:06:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5442.753337 secs 2024-04-26T20:06:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5442.753337 secs 2024-04-26T20:06:02.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:06:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:06:02.244+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:06:02.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:06:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:06:02.244+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:06:02.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:06:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:06:02.244+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:06:02.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:06:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:06:02.244+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:06:02.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:06:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:06:02.244+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:06:02.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5847 sec 2024-04-26T20:06:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:02 smithi119 ceph-mon[32752]: pgmap v3672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:02 smithi167 ceph-mon[32751]: pgmap v3672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:02 smithi090 ceph-mon[29180]: pgmap v3672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:03.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5848 sec 2024-04-26T20:06:04.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5849 sec 2024-04-26T20:06:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:04 smithi119 ceph-mon[32752]: pgmap v3673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:04 smithi167 ceph-mon[32751]: pgmap v3673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:04 smithi090 ceph-mon[29180]: pgmap v3673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:05.410 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5850 sec 2024-04-26T20:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5447.058221 secs 2024-04-26T20:06:06.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5447.058221 secs 2024-04-26T20:06:06.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5447.058221 secs 2024-04-26T20:06:06.411 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5851 sec 2024-04-26T20:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5447.753449 secs 2024-04-26T20:06:07.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:06 smithi119 ceph-mon[32752]: pgmap v3674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:07.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5447.753449 secs 2024-04-26T20:06:07.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:06 smithi090 ceph-mon[29180]: pgmap v3674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:07.253 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5447.753449 secs 2024-04-26T20:06:07.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:06 smithi167 ceph-mon[32751]: pgmap v3674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:07.411 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5852 sec 2024-04-26T20:06:08.411 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5853 sec 2024-04-26T20:06:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:08 smithi119 ceph-mon[32752]: pgmap v3675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:09.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:08 smithi090 ceph-mon[29180]: pgmap v3675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:09.167 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:08 smithi167 ceph-mon[32751]: pgmap v3675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:09.411 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5854 sec 2024-04-26T20:06:10.411 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5855 sec 2024-04-26T20:06:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:10 smithi119 ceph-mon[32752]: pgmap v3676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:11.348 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5452.058301 secs 2024-04-26T20:06:11.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:10 smithi090 ceph-mon[29180]: pgmap v3676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:11.394 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5452.058301 secs 2024-04-26T20:06:11.405 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:10 smithi167 ceph-mon[32751]: pgmap v3676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:11.414 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5452.058301 secs 2024-04-26T20:06:11.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5856 sec 2024-04-26T20:06:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5452.753570 secs 2024-04-26T20:06:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5452.753570 secs 2024-04-26T20:06:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5452.753570 secs 2024-04-26T20:06:12.411 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5857 sec 2024-04-26T20:06:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:12 smithi119 ceph-mon[32752]: pgmap v3677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:13.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:12 smithi167 ceph-mon[32751]: pgmap v3677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:13.394 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:12 smithi090 ceph-mon[29180]: pgmap v3677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T20:06:13.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5858 sec 2024-04-26T20:06:14.182 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:06:14.181+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:06:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:06:14.181+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:06:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:06:14.181+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 46 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:06:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:06:14.181+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 46 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:06:14.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5859 sec 2024-04-26T20:06:15.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:14 smithi119 ceph-mon[32752]: pgmap v3678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:15.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:14 smithi167 ceph-mon[32751]: pgmap v3678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:15.435 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:14 smithi090 ceph-mon[29180]: pgmap v3678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T20:06:15.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5860 sec 2024-04-26T20:06:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5457.058380 secs 2024-04-26T20:06:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5457.058380 secs 2024-04-26T20:06:16.389 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5457.058380 secs 2024-04-26T20:06:16.411 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5861 sec 2024-04-26T20:06:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5457.753689 secs 2024-04-26T20:06:17.443 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:16 smithi119 ceph-mon[32752]: pgmap v3679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:17.472 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5457.753689 secs 2024-04-26T20:06:17.495 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:16 smithi090 ceph-mon[29180]: pgmap v3679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:17.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5457.753689 secs 2024-04-26T20:06:17.618 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:16 smithi167 ceph-mon[32751]: pgmap v3679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:17.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5862 sec 2024-04-26T20:06:18.412 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5863 sec 2024-04-26T20:06:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:18 smithi119 ceph-mon[32752]: pgmap v3680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:19.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:18 smithi090 ceph-mon[29180]: pgmap v3680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:19.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:18 smithi167 ceph-mon[32751]: pgmap v3680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:19.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5864 sec 2024-04-26T20:06:20.412 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5865 sec 2024-04-26T20:06:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:20 smithi119 ceph-mon[32752]: pgmap v3681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:21.325 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5462.058461 secs 2024-04-26T20:06:21.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:20 smithi167 ceph-mon[32751]: pgmap v3681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:21.410 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5462.058461 secs 2024-04-26T20:06:21.554 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:20 smithi090 ceph-mon[29180]: pgmap v3681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5462.058461 secs 2024-04-26T20:06:21.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5866 sec 2024-04-26T20:06:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5462.753799 secs 2024-04-26T20:06:22.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5462.753799 secs 2024-04-26T20:06:22.404 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5462.753799 secs 2024-04-26T20:06:22.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5867 sec 2024-04-26T20:06:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:22 smithi119 ceph-mon[32752]: pgmap v3682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:23.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:22 smithi090 ceph-mon[29180]: pgmap v3682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:23.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:22 smithi167 ceph-mon[32751]: pgmap v3682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:23.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5868 sec 2024-04-26T20:06:24.412 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5869 sec 2024-04-26T20:06:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:24 smithi119 ceph-mon[32752]: pgmap v3683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:25.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:24 smithi167 ceph-mon[32751]: pgmap v3683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:24 smithi090 ceph-mon[29180]: pgmap v3683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:25.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5870 sec 2024-04-26T20:06:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5467.058546 secs 2024-04-26T20:06:26.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5467.058546 secs 2024-04-26T20:06:26.418 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5467.058546 secs 2024-04-26T20:06:26.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5871 sec 2024-04-26T20:06:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5467.753936 secs 2024-04-26T20:06:27.359 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:26 smithi119 ceph-mon[32752]: pgmap v3684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:27.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5467.753936 secs 2024-04-26T20:06:27.378 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:26 smithi167 ceph-mon[32751]: pgmap v3684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:27.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5467.753936 secs 2024-04-26T20:06:27.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:26 smithi090 ceph-mon[29180]: pgmap v3684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:27.412 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5872 sec 2024-04-26T20:06:28.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5873 sec 2024-04-26T20:06:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:28 smithi119 ceph-mon[32752]: pgmap v3685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:29.388 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:28 smithi090 ceph-mon[29180]: pgmap v3685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:29.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:28 smithi167 ceph-mon[32751]: pgmap v3685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:29.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5874 sec 2024-04-26T20:06:30.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5875 sec 2024-04-26T20:06:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:30 smithi119 ceph-mon[32752]: pgmap v3686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:31.370 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5472.058628 secs 2024-04-26T20:06:31.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:30 smithi090 ceph-mon[29180]: pgmap v3686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:31.449 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5472.058628 secs 2024-04-26T20:06:31.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:30 smithi167 ceph-mon[32751]: pgmap v3686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:31.533 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5472.058628 secs 2024-04-26T20:06:31.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5876 sec 2024-04-26T20:06:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5472.754046 secs 2024-04-26T20:06:32.303 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:32.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5472.754046 secs 2024-04-26T20:06:32.321 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:32.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5472.754046 secs 2024-04-26T20:06:32.337 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:06:32.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5877 sec 2024-04-26T20:06:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:33 smithi119 ceph-mon[32752]: pgmap v3687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:33.391 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.436 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:33.476 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:06:33.541 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:33 smithi090 ceph-mon[29180]: pgmap v3687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:33.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.559 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:33.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:06:33.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:33 smithi167 ceph-mon[32751]: pgmap v3687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:33.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:06:33.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:06:33.676 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:06:33.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5878 sec 2024-04-26T20:06:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:34 smithi119 ceph-mon[32752]: pgmap v3688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:06:34.346 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:34 smithi119 ceph-mon[32752]: pgmap v3689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:06:34.360 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:34 smithi167 ceph-mon[32751]: pgmap v3688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:06:34.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:34 smithi167 ceph-mon[32751]: pgmap v3689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:06:34.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:34 smithi090 ceph-mon[29180]: pgmap v3688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:06:34.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:34 smithi090 ceph-mon[29180]: pgmap v3689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:06:34.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5879 sec 2024-04-26T20:06:35.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5880 sec 2024-04-26T20:06:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:36 smithi119 ceph-mon[32752]: pgmap v3690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:36.350 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5477.058713 secs 2024-04-26T20:06:36.403 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:36 smithi090 ceph-mon[29180]: pgmap v3690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:36.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5477.058713 secs 2024-04-26T20:06:36.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:36 smithi167 ceph-mon[32751]: pgmap v3690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:36.440 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5477.058713 secs 2024-04-26T20:06:36.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5881 sec 2024-04-26T20:06:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5477.754157 secs 2024-04-26T20:06:37.358 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5477.754157 secs 2024-04-26T20:06:37.390 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5477.754157 secs 2024-04-26T20:06:37.426 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5882 sec 2024-04-26T20:06:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:38 smithi119 ceph-mon[32752]: pgmap v3691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:38.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:38 smithi167 ceph-mon[32751]: pgmap v3691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:38.494 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:38 smithi090 ceph-mon[29180]: pgmap v3691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:38.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5883 sec 2024-04-26T20:06:39.414 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5884 sec 2024-04-26T20:06:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:40 smithi119 ceph-mon[32752]: pgmap v3692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:40 smithi167 ceph-mon[32751]: pgmap v3692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:40.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:40 smithi090 ceph-mon[29180]: pgmap v3692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:40.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5885 sec 2024-04-26T20:06:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5482.058796 secs 2024-04-26T20:06:41.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5482.058796 secs 2024-04-26T20:06:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5482.058796 secs 2024-04-26T20:06:41.413 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5886 sec 2024-04-26T20:06:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:42 smithi119 ceph-mon[32752]: pgmap v3693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:42.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5482.754310 secs 2024-04-26T20:06:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:42 smithi090 ceph-mon[29180]: pgmap v3693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:42.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5482.754310 secs 2024-04-26T20:06:42.394 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:42 smithi167 ceph-mon[32751]: pgmap v3693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:06:42.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5482.754310 secs 2024-04-26T20:06:42.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5887 sec 2024-04-26T20:06:43.414 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5888 sec 2024-04-26T20:06:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:44 smithi119 ceph-mon[32752]: pgmap v3694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:06:44.363 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:44 smithi167 ceph-mon[32751]: pgmap v3694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:06:44.425 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:44 smithi090 ceph-mon[29180]: pgmap v3694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:06:44.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5889 sec 2024-04-26T20:06:45.414 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5890 sec 2024-04-26T20:06:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:46 smithi119 ceph-mon[32752]: pgmap v3695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:46.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5487.058880 secs 2024-04-26T20:06:46.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:46 smithi167 ceph-mon[32751]: pgmap v3695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:46.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5487.058880 secs 2024-04-26T20:06:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:46 smithi090 ceph-mon[29180]: pgmap v3695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:46.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5487.058880 secs 2024-04-26T20:06:46.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5891 sec 2024-04-26T20:06:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5487.754374 secs 2024-04-26T20:06:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5487.754374 secs 2024-04-26T20:06:47.449 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5487.754374 secs 2024-04-26T20:06:47.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5892 sec 2024-04-26T20:06:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:48 smithi119 ceph-mon[32752]: pgmap v3696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:48.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:48 smithi090 ceph-mon[29180]: pgmap v3696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:48.431 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:48 smithi167 ceph-mon[32751]: pgmap v3696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:48.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5893 sec 2024-04-26T20:06:49.414 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5894 sec 2024-04-26T20:06:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:50 smithi119 ceph-mon[32752]: pgmap v3697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:50.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:50 smithi167 ceph-mon[32751]: pgmap v3697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:50.515 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:50 smithi090 ceph-mon[29180]: pgmap v3697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:50.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5895 sec 2024-04-26T20:06:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5492.058942 secs 2024-04-26T20:06:51.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5492.058942 secs 2024-04-26T20:06:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5492.058942 secs 2024-04-26T20:06:51.414 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5896 sec 2024-04-26T20:06:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:52 smithi119 ceph-mon[32752]: pgmap v3698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:52.349 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5492.754415 secs 2024-04-26T20:06:52.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:52 smithi167 ceph-mon[32751]: pgmap v3698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:52.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5492.754415 secs 2024-04-26T20:06:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:52 smithi090 ceph-mon[29180]: pgmap v3698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:52.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5492.754415 secs 2024-04-26T20:06:52.414 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5897 sec 2024-04-26T20:06:53.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5898 sec 2024-04-26T20:06:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:54 smithi119 ceph-mon[32752]: pgmap v3699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:54.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:54 smithi167 ceph-mon[32751]: pgmap v3699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:54 smithi090 ceph-mon[29180]: pgmap v3699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:54.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5899 sec 2024-04-26T20:06:55.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5900 sec 2024-04-26T20:06:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:56 smithi119 ceph-mon[32752]: pgmap v3700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:56.335 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5497.059025 secs 2024-04-26T20:06:56.397 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:56 smithi090 ceph-mon[29180]: pgmap v3700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:56.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5497.059025 secs 2024-04-26T20:06:56.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:56 smithi167 ceph-mon[32751]: pgmap v3700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:06:56.527 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5497.059025 secs 2024-04-26T20:06:56.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5901 sec 2024-04-26T20:06:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5497.754541 secs 2024-04-26T20:06:57.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5497.754541 secs 2024-04-26T20:06:57.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5902 sec 2024-04-26T20:06:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5497.754541 secs 2024-04-26T20:06:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:06:58 smithi167 ceph-mon[32751]: pgmap v3701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:06:58 smithi090 ceph-mon[29180]: pgmap v3701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:58.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5903 sec 2024-04-26T20:06:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:06:58 smithi119 ceph-mon[32752]: pgmap v3701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:06:59.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5904 sec 2024-04-26T20:07:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:00 smithi167 ceph-mon[32751]: pgmap v3702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:00 smithi090 ceph-mon[29180]: pgmap v3702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:00.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5905 sec 2024-04-26T20:07:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:00 smithi119 ceph-mon[32752]: pgmap v3702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5502.059082 secs 2024-04-26T20:07:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5502.059082 secs 2024-04-26T20:07:01.415 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5906 sec 2024-04-26T20:07:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5502.059082 secs 2024-04-26T20:07:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:02 smithi167 ceph-mon[32751]: pgmap v3703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:02.346 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5502.754653 secs 2024-04-26T20:07:02.396 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:02 smithi090 ceph-mon[29180]: pgmap v3703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:02.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5502.754653 secs 2024-04-26T20:07:02.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5907 sec 2024-04-26T20:07:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:02 smithi119 ceph-mon[32752]: pgmap v3703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:02.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5502.754653 secs 2024-04-26T20:07:03.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5908 sec 2024-04-26T20:07:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:04 smithi090 ceph-mon[29180]: pgmap v3704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:04.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5909 sec 2024-04-26T20:07:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:04 smithi119 ceph-mon[32752]: pgmap v3704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:04.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:04 smithi167 ceph-mon[32751]: pgmap v3704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:05.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5910 sec 2024-04-26T20:07:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:06 smithi090 ceph-mon[29180]: pgmap v3705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:06.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5507.059165 secs 2024-04-26T20:07:06.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5911 sec 2024-04-26T20:07:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:06 smithi119 ceph-mon[32752]: pgmap v3705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:06.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5507.059165 secs 2024-04-26T20:07:06.684 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:06 smithi167 ceph-mon[32751]: pgmap v3705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:06.708 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5507.059165 secs 2024-04-26T20:07:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5507.754768 secs 2024-04-26T20:07:07.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5912 sec 2024-04-26T20:07:07.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5507.754768 secs 2024-04-26T20:07:07.729 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5507.754768 secs 2024-04-26T20:07:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:08 smithi090 ceph-mon[29180]: pgmap v3706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:08.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5913 sec 2024-04-26T20:07:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:08 smithi119 ceph-mon[32752]: pgmap v3706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:08 smithi167 ceph-mon[32751]: pgmap v3706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:09.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5914 sec 2024-04-26T20:07:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:10 smithi090 ceph-mon[29180]: pgmap v3707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:10.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5915 sec 2024-04-26T20:07:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:10 smithi119 ceph-mon[32752]: pgmap v3707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:10 smithi167 ceph-mon[32751]: pgmap v3707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5512.059247 secs 2024-04-26T20:07:11.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5916 sec 2024-04-26T20:07:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5512.059247 secs 2024-04-26T20:07:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5512.059247 secs 2024-04-26T20:07:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:12 smithi090 ceph-mon[29180]: pgmap v3708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5512.754918 secs 2024-04-26T20:07:12.416 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5917 sec 2024-04-26T20:07:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:12 smithi119 ceph-mon[32752]: pgmap v3708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5512.754918 secs 2024-04-26T20:07:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:12 smithi167 ceph-mon[32751]: pgmap v3708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5512.754918 secs 2024-04-26T20:07:13.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5918 sec 2024-04-26T20:07:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:14 smithi090 ceph-mon[29180]: pgmap v3709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:14.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5919 sec 2024-04-26T20:07:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:14 smithi119 ceph-mon[32752]: pgmap v3709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:14 smithi167 ceph-mon[32751]: pgmap v3709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:07:15.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5920 sec 2024-04-26T20:07:16.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:16 smithi119 ceph-mon[32752]: pgmap v3710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:16.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5517.059330 secs 2024-04-26T20:07:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:16 smithi090 ceph-mon[29180]: pgmap v3710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5517.059330 secs 2024-04-26T20:07:16.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5921 sec 2024-04-26T20:07:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:16 smithi167 ceph-mon[32751]: pgmap v3710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:07:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5517.059330 secs 2024-04-26T20:07:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5517.755047 secs 2024-04-26T20:07:17.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5922 sec 2024-04-26T20:07:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5517.755047 secs 2024-04-26T20:07:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5517.755047 secs 2024-04-26T20:07:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:18 smithi090 ceph-mon[29180]: pgmap v3711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:18.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5923 sec 2024-04-26T20:07:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:18 smithi119 ceph-mon[32752]: pgmap v3711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:18 smithi167 ceph-mon[32751]: pgmap v3711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:19.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5924 sec 2024-04-26T20:07:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:20 smithi090 ceph-mon[29180]: pgmap v3712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:20.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5925 sec 2024-04-26T20:07:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:20 smithi119 ceph-mon[32752]: pgmap v3712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:20 smithi167 ceph-mon[32751]: pgmap v3712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5522.059412 secs 2024-04-26T20:07:21.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5926 sec 2024-04-26T20:07:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5522.059412 secs 2024-04-26T20:07:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5522.059412 secs 2024-04-26T20:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:22 smithi090 ceph-mon[29180]: pgmap v3713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5522.755167 secs 2024-04-26T20:07:22.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5927 sec 2024-04-26T20:07:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:22 smithi119 ceph-mon[32752]: pgmap v3713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5522.755167 secs 2024-04-26T20:07:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:22 smithi167 ceph-mon[32751]: pgmap v3713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5522.755167 secs 2024-04-26T20:07:23.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5928 sec 2024-04-26T20:07:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:24 smithi090 ceph-mon[29180]: pgmap v3714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:24.417 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5929 sec 2024-04-26T20:07:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:24 smithi119 ceph-mon[32752]: pgmap v3714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:24 smithi167 ceph-mon[32751]: pgmap v3714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:25.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5930 sec 2024-04-26T20:07:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:26 smithi090 ceph-mon[29180]: pgmap v3715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5527.059496 secs 2024-04-26T20:07:26.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5931 sec 2024-04-26T20:07:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:26 smithi119 ceph-mon[32752]: pgmap v3715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5527.059496 secs 2024-04-26T20:07:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:26 smithi167 ceph-mon[32751]: pgmap v3715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5527.059496 secs 2024-04-26T20:07:27.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5932 sec 2024-04-26T20:07:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5527.755241 secs 2024-04-26T20:07:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5527.755241 secs 2024-04-26T20:07:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5527.755241 secs 2024-04-26T20:07:28.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5933 sec 2024-04-26T20:07:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:28 smithi119 ceph-mon[32752]: pgmap v3716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:28 smithi167 ceph-mon[32751]: pgmap v3716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:28 smithi090 ceph-mon[29180]: pgmap v3716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:29.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5934 sec 2024-04-26T20:07:30.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5935 sec 2024-04-26T20:07:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:30 smithi119 ceph-mon[32752]: pgmap v3717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:30 smithi167 ceph-mon[32751]: pgmap v3717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:30 smithi090 ceph-mon[29180]: pgmap v3717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:31.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5936 sec 2024-04-26T20:07:31.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5532.059576 secs 2024-04-26T20:07:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5532.059576 secs 2024-04-26T20:07:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5532.059576 secs 2024-04-26T20:07:32.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5937 sec 2024-04-26T20:07:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:32 smithi090 ceph-mon[29180]: pgmap v3718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5532.755366 secs 2024-04-26T20:07:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:32 smithi119 ceph-mon[32752]: pgmap v3718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5532.755366 secs 2024-04-26T20:07:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:32 smithi167 ceph-mon[32751]: pgmap v3718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5532.755366 secs 2024-04-26T20:07:33.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5938 sec 2024-04-26T20:07:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:07:34.418 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5939 sec 2024-04-26T20:07:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:34 smithi119 ceph-mon[32752]: pgmap v3719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:34 smithi119 ceph-mon[32752]: pgmap v3720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:07:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:34 smithi119 ceph-mon[32752]: pgmap v3721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:07:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:07:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:34 smithi167 ceph-mon[32751]: pgmap v3719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:34 smithi167 ceph-mon[32751]: pgmap v3720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:07:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:34 smithi167 ceph-mon[32751]: pgmap v3721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:07:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:07:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:34 smithi090 ceph-mon[29180]: pgmap v3719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:07:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:07:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:34 smithi090 ceph-mon[29180]: pgmap v3720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:07:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:34 smithi090 ceph-mon[29180]: pgmap v3721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:07:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:07:35.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5940 sec 2024-04-26T20:07:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:35 smithi119 ceph-mon[32752]: pgmap v3722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:35 smithi167 ceph-mon[32751]: pgmap v3722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:35 smithi090 ceph-mon[29180]: pgmap v3722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:36.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5941 sec 2024-04-26T20:07:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5537.059660 secs 2024-04-26T20:07:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5537.755523 secs 2024-04-26T20:07:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5537.059660 secs 2024-04-26T20:07:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5537.755523 secs 2024-04-26T20:07:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5537.059660 secs 2024-04-26T20:07:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5537.755523 secs 2024-04-26T20:07:37.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5942 sec 2024-04-26T20:07:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:37 smithi119 ceph-mon[32752]: pgmap v3723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:07:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:37 smithi167 ceph-mon[32751]: pgmap v3723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:07:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:37 smithi090 ceph-mon[29180]: pgmap v3723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:07:38.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5943 sec 2024-04-26T20:07:39.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5944 sec 2024-04-26T20:07:40.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5945 sec 2024-04-26T20:07:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:40 smithi119 ceph-mon[32752]: pgmap v3724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:40 smithi167 ceph-mon[32751]: pgmap v3724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:40 smithi090 ceph-mon[29180]: pgmap v3724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:41.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5946 sec 2024-04-26T20:07:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5542.059741 secs 2024-04-26T20:07:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5542.755615 secs 2024-04-26T20:07:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5542.059741 secs 2024-04-26T20:07:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5542.755615 secs 2024-04-26T20:07:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5542.059741 secs 2024-04-26T20:07:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5542.755615 secs 2024-04-26T20:07:42.419 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5947 sec 2024-04-26T20:07:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:42 smithi119 ceph-mon[32752]: pgmap v3725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:42 smithi167 ceph-mon[32751]: pgmap v3725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:42 smithi090 ceph-mon[29180]: pgmap v3725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:07:43.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5948 sec 2024-04-26T20:07:44.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5949 sec 2024-04-26T20:07:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:44 smithi119 ceph-mon[32752]: pgmap v3726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:07:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:44 smithi167 ceph-mon[32751]: pgmap v3726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:07:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:44 smithi090 ceph-mon[29180]: pgmap v3726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:07:45.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5950 sec 2024-04-26T20:07:46.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5951 sec 2024-04-26T20:07:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5547.059824 secs 2024-04-26T20:07:46.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:46 smithi119 ceph-mon[32752]: pgmap v3727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:46.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5547.755713 secs 2024-04-26T20:07:46.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5547.059824 secs 2024-04-26T20:07:46.613 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:46 smithi167 ceph-mon[32751]: pgmap v3727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:46.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5547.755713 secs 2024-04-26T20:07:46.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5547.059824 secs 2024-04-26T20:07:46.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:46 smithi090 ceph-mon[29180]: pgmap v3727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:46.804 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5547.755713 secs 2024-04-26T20:07:47.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5952 sec 2024-04-26T20:07:48.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5953 sec 2024-04-26T20:07:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:48 smithi119 ceph-mon[32752]: pgmap v3728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:48.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:48 smithi167 ceph-mon[32751]: pgmap v3728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:48 smithi090 ceph-mon[29180]: pgmap v3728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:49.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5954 sec 2024-04-26T20:07:50.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5955 sec 2024-04-26T20:07:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:50 smithi119 ceph-mon[32752]: pgmap v3729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:50.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:50 smithi167 ceph-mon[32751]: pgmap v3729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:50.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:50 smithi090 ceph-mon[29180]: pgmap v3729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:51.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5956 sec 2024-04-26T20:07:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5552.059881 secs 2024-04-26T20:07:51.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5552.755822 secs 2024-04-26T20:07:51.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5552.059881 secs 2024-04-26T20:07:51.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5552.755822 secs 2024-04-26T20:07:51.675 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5552.059881 secs 2024-04-26T20:07:51.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5552.755822 secs 2024-04-26T20:07:52.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5957 sec 2024-04-26T20:07:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:52 smithi119 ceph-mon[32752]: pgmap v3730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:52.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:52 smithi167 ceph-mon[32751]: pgmap v3730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:52.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:52 smithi090 ceph-mon[29180]: pgmap v3730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:53.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5958 sec 2024-04-26T20:07:54.420 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5959 sec 2024-04-26T20:07:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:54 smithi119 ceph-mon[32752]: pgmap v3731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:54.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:54 smithi167 ceph-mon[32751]: pgmap v3731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:54.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:54 smithi090 ceph-mon[29180]: pgmap v3731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:07:55.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5960 sec 2024-04-26T20:07:56.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5961 sec 2024-04-26T20:07:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5557.059965 secs 2024-04-26T20:07:56.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:56 smithi119 ceph-mon[32752]: pgmap v3732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:56.571 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5557.755965 secs 2024-04-26T20:07:56.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5557.059965 secs 2024-04-26T20:07:56.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:56 smithi167 ceph-mon[32751]: pgmap v3732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:56.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5557.755965 secs 2024-04-26T20:07:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5557.059965 secs 2024-04-26T20:07:56.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:56 smithi090 ceph-mon[29180]: pgmap v3732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:07:56.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5557.755965 secs 2024-04-26T20:07:57.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5962 sec 2024-04-26T20:07:58.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5963 sec 2024-04-26T20:07:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:07:58 smithi119 ceph-mon[32752]: pgmap v3733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:07:58.613 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:07:58 smithi167 ceph-mon[32751]: pgmap v3733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:07:58.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:07:58 smithi090 ceph-mon[29180]: pgmap v3733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:07:59.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5964 sec 2024-04-26T20:08:00.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5965 sec 2024-04-26T20:08:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:00 smithi119 ceph-mon[32752]: pgmap v3734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:00.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:00 smithi167 ceph-mon[32751]: pgmap v3734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:00.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:00 smithi090 ceph-mon[29180]: pgmap v3734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:01.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5966 sec 2024-04-26T20:08:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5562.060023 secs 2024-04-26T20:08:01.627 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5562.756079 secs 2024-04-26T20:08:01.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5562.060023 secs 2024-04-26T20:08:01.726 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5562.756079 secs 2024-04-26T20:08:01.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5562.060023 secs 2024-04-26T20:08:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5562.756079 secs 2024-04-26T20:08:02.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5967 sec 2024-04-26T20:08:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:02 smithi119 ceph-mon[32752]: pgmap v3735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:08:02.602 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:02 smithi167 ceph-mon[32751]: pgmap v3735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:08:02.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:02 smithi090 ceph-mon[29180]: pgmap v3735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:08:03.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5968 sec 2024-04-26T20:08:04.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5969 sec 2024-04-26T20:08:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:04 smithi119 ceph-mon[32752]: pgmap v3736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:08:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:04 smithi090 ceph-mon[29180]: pgmap v3736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:08:04.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:04 smithi167 ceph-mon[32751]: pgmap v3736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:08:05.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5970 sec 2024-04-26T20:08:06.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5971 sec 2024-04-26T20:08:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5567.060086 secs 2024-04-26T20:08:06.632 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:06 smithi119 ceph-mon[32752]: pgmap v3737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:06.680 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5567.756223 secs 2024-04-26T20:08:06.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5567.060086 secs 2024-04-26T20:08:06.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:06 smithi090 ceph-mon[29180]: pgmap v3737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:06.761 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5567.756223 secs 2024-04-26T20:08:06.772 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5567.060086 secs 2024-04-26T20:08:06.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:06 smithi167 ceph-mon[32751]: pgmap v3737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:06.894 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5567.756223 secs 2024-04-26T20:08:07.421 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5972 sec 2024-04-26T20:08:08.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5973 sec 2024-04-26T20:08:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:08 smithi167 ceph-mon[32751]: pgmap v3738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:08.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:08 smithi090 ceph-mon[29180]: pgmap v3738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:08 smithi119 ceph-mon[32752]: pgmap v3738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:09.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5974 sec 2024-04-26T20:08:10.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5975 sec 2024-04-26T20:08:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:10 smithi167 ceph-mon[32751]: pgmap v3739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:10.639 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:10 smithi090 ceph-mon[29180]: pgmap v3739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:10 smithi119 ceph-mon[32752]: pgmap v3739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:11.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5976 sec 2024-04-26T20:08:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5572.060164 secs 2024-04-26T20:08:11.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5572.756371 secs 2024-04-26T20:08:11.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5572.060164 secs 2024-04-26T20:08:11.705 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5572.756371 secs 2024-04-26T20:08:11.884 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5572.060164 secs 2024-04-26T20:08:11.933 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5572.756371 secs 2024-04-26T20:08:12.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5977 sec 2024-04-26T20:08:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:12 smithi167 ceph-mon[32751]: pgmap v3740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:12.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:12 smithi090 ceph-mon[29180]: pgmap v3740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:12 smithi119 ceph-mon[32752]: pgmap v3740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:13.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5978 sec 2024-04-26T20:08:14.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5979 sec 2024-04-26T20:08:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:14 smithi090 ceph-mon[29180]: pgmap v3741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:14 smithi119 ceph-mon[32752]: pgmap v3741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:14.859 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:14 smithi167 ceph-mon[32751]: pgmap v3741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:15.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5980 sec 2024-04-26T20:08:16.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5981 sec 2024-04-26T20:08:16.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5577.060250 secs 2024-04-26T20:08:16.612 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:16 smithi119 ceph-mon[32752]: pgmap v3742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:16.696 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5577.756490 secs 2024-04-26T20:08:16.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5577.060250 secs 2024-04-26T20:08:16.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:16 smithi090 ceph-mon[29180]: pgmap v3742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:16.719 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5577.756490 secs 2024-04-26T20:08:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5577.060250 secs 2024-04-26T20:08:16.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:16 smithi167 ceph-mon[32751]: pgmap v3742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:16.929 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5577.756490 secs 2024-04-26T20:08:17.422 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5982 sec 2024-04-26T20:08:18.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5983 sec 2024-04-26T20:08:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:18 smithi090 ceph-mon[29180]: pgmap v3743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:18 smithi119 ceph-mon[32752]: pgmap v3743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:18.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:18 smithi167 ceph-mon[32751]: pgmap v3743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:19.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5984 sec 2024-04-26T20:08:20.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5985 sec 2024-04-26T20:08:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:20 smithi090 ceph-mon[29180]: pgmap v3744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:20 smithi119 ceph-mon[32752]: pgmap v3744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:20.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:20 smithi167 ceph-mon[32751]: pgmap v3744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:21.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5986 sec 2024-04-26T20:08:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5582.060332 secs 2024-04-26T20:08:21.673 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5582.756601 secs 2024-04-26T20:08:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5582.060332 secs 2024-04-26T20:08:21.833 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5582.756601 secs 2024-04-26T20:08:21.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5582.060332 secs 2024-04-26T20:08:21.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5582.756601 secs 2024-04-26T20:08:22.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5987 sec 2024-04-26T20:08:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:22 smithi090 ceph-mon[29180]: pgmap v3745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:22 smithi119 ceph-mon[32752]: pgmap v3745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:22.904 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:22 smithi167 ceph-mon[32751]: pgmap v3745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:23.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5988 sec 2024-04-26T20:08:24.426 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5989 sec 2024-04-26T20:08:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:24 smithi090 ceph-mon[29180]: pgmap v3746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:24 smithi119 ceph-mon[32752]: pgmap v3746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:24.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:24 smithi167 ceph-mon[32751]: pgmap v3746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:25.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5990 sec 2024-04-26T20:08:26.423 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5991 sec 2024-04-26T20:08:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5587.060417 secs 2024-04-26T20:08:26.662 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:26 smithi090 ceph-mon[29180]: pgmap v3747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:26.765 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5587.756686 secs 2024-04-26T20:08:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5587.060417 secs 2024-04-26T20:08:26.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:26 smithi119 ceph-mon[32752]: pgmap v3747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:26.822 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5587.756686 secs 2024-04-26T20:08:26.831 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5587.060417 secs 2024-04-26T20:08:26.839 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:26 smithi167 ceph-mon[32751]: pgmap v3747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:26.847 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5587.756686 secs 2024-04-26T20:08:27.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5992 sec 2024-04-26T20:08:28.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5993 sec 2024-04-26T20:08:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:28 smithi090 ceph-mon[29180]: pgmap v3748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:28 smithi119 ceph-mon[32752]: pgmap v3748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:28.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:28 smithi167 ceph-mon[32751]: pgmap v3748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:29.201 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:08:29.200+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:08:29.296 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:08:29.200+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:08:29.306 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:08:29.201+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 47 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:08:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:08:29.201+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 47 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:08:29.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5994 sec 2024-04-26T20:08:30.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5995 sec 2024-04-26T20:08:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:30 smithi090 ceph-mon[29180]: pgmap v3749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:30 smithi167 ceph-mon[32751]: pgmap v3749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:30.854 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:30 smithi119 ceph-mon[32752]: pgmap v3749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:31.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5996 sec 2024-04-26T20:08:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5592.060496 secs 2024-04-26T20:08:31.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5592.756806 secs 2024-04-26T20:08:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5592.060496 secs 2024-04-26T20:08:31.771 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5592.756806 secs 2024-04-26T20:08:31.842 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5592.060496 secs 2024-04-26T20:08:31.893 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5592.756806 secs 2024-04-26T20:08:32.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5997 sec 2024-04-26T20:08:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:32 smithi090 ceph-mon[29180]: pgmap v3750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:32 smithi119 ceph-mon[32752]: pgmap v3750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:32.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:32 smithi167 ceph-mon[32751]: pgmap v3750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:33.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5998 sec 2024-04-26T20:08:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:08:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:08:33.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:08:34.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 5999 sec 2024-04-26T20:08:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:34 smithi090 ceph-mon[29180]: pgmap v3751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:34.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:08:34.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:08:34.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:34 smithi090 ceph-mon[29180]: pgmap v3752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:08:34.655 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:34 smithi090 ceph-mon[29180]: pgmap v3753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:34.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:08:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:34 smithi119 ceph-mon[32752]: pgmap v3751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:34.836 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:08:34.918 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:08:34.981 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:34 smithi119 ceph-mon[32752]: pgmap v3752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:08:34.995 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:34 smithi119 ceph-mon[32752]: pgmap v3753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:35.005 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:08:35.022 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:34 smithi167 ceph-mon[32751]: pgmap v3751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:35.032 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:08:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:08:35.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:34 smithi167 ceph-mon[32751]: pgmap v3752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:08:35.174 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:34 smithi167 ceph-mon[32751]: pgmap v3753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:35.289 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:08:35.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6000 sec 2024-04-26T20:08:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5597.060581 secs 2024-04-26T20:08:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5597.060581 secs 2024-04-26T20:08:36.083 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5597.060581 secs 2024-04-26T20:08:36.425 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6001 sec 2024-04-26T20:08:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5597.756923 secs 2024-04-26T20:08:36.912 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:36 smithi090 ceph-mon[29180]: pgmap v3754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5597.756923 secs 2024-04-26T20:08:37.081 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:36 smithi119 ceph-mon[32752]: pgmap v3754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:37.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5597.756923 secs 2024-04-26T20:08:37.116 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:36 smithi167 ceph-mon[32751]: pgmap v3754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:37.424 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6002 sec 2024-04-26T20:08:38.425 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6003 sec 2024-04-26T20:08:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:39 smithi119 ceph-mon[32752]: pgmap v3755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:08:39.375 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:39 smithi167 ceph-mon[32751]: pgmap v3755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:08:39.438 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:39 smithi090 ceph-mon[29180]: pgmap v3755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:08:39.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6004 sec 2024-04-26T20:08:40.425 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6005 sec 2024-04-26T20:08:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:41 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5602.060663 secs 2024-04-26T20:08:41.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:41 smithi119 ceph-mon[32752]: pgmap v3756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:41.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:41 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5602.060663 secs 2024-04-26T20:08:41.440 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:41 smithi090 ceph-mon[29180]: pgmap v3756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:41.528 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:41 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5602.060663 secs 2024-04-26T20:08:41.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:41 smithi167 ceph-mon[32751]: pgmap v3756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:41.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6006 sec 2024-04-26T20:08:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5602.757011 secs 2024-04-26T20:08:42.471 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5602.757011 secs 2024-04-26T20:08:42.532 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5602.757011 secs 2024-04-26T20:08:42.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6007 sec 2024-04-26T20:08:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:43 smithi119 ceph-mon[32752]: pgmap v3757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:43 smithi167 ceph-mon[32751]: pgmap v3757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:43 smithi090 ceph-mon[29180]: pgmap v3757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:08:43.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6008 sec 2024-04-26T20:08:44.425 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6009 sec 2024-04-26T20:08:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:45 smithi119 ceph-mon[32752]: pgmap v3758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:08:45.372 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:45 smithi167 ceph-mon[32751]: pgmap v3758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:08:45.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:45 smithi090 ceph-mon[29180]: pgmap v3758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:08:45.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6010 sec 2024-04-26T20:08:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:46 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5607.060747 secs 2024-04-26T20:08:46.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:46 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5607.060747 secs 2024-04-26T20:08:46.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:46 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5607.060747 secs 2024-04-26T20:08:46.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6011 sec 2024-04-26T20:08:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5607.757122 secs 2024-04-26T20:08:47.262 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:47 smithi090 ceph-mon[29180]: pgmap v3759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:47.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5607.757122 secs 2024-04-26T20:08:47.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:47 smithi167 ceph-mon[32751]: pgmap v3759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:47.418 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5607.757122 secs 2024-04-26T20:08:47.488 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:47 smithi119 ceph-mon[32752]: pgmap v3759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:47.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6012 sec 2024-04-26T20:08:48.425 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6013 sec 2024-04-26T20:08:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:49 smithi119 ceph-mon[32752]: pgmap v3760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:49.359 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:49 smithi167 ceph-mon[32751]: pgmap v3760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:49.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:49 smithi090 ceph-mon[29180]: pgmap v3760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:49.426 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6014 sec 2024-04-26T20:08:50.426 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6015 sec 2024-04-26T20:08:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:51 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5612.060818 secs 2024-04-26T20:08:51.350 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:51 smithi119 ceph-mon[32752]: pgmap v3761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:51.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:51 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5612.060818 secs 2024-04-26T20:08:51.464 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:51 smithi090 ceph-mon[29180]: pgmap v3761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:51.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:51 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5612.060818 secs 2024-04-26T20:08:51.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:51 smithi167 ceph-mon[32751]: pgmap v3761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:51.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6016 sec 2024-04-26T20:08:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5612.757231 secs 2024-04-26T20:08:52.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5612.757231 secs 2024-04-26T20:08:52.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6017 sec 2024-04-26T20:08:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5612.757231 secs 2024-04-26T20:08:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:53 smithi167 ceph-mon[32751]: pgmap v3762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:53.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:53 smithi090 ceph-mon[29180]: pgmap v3762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:53.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6018 sec 2024-04-26T20:08:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:53 smithi119 ceph-mon[32752]: pgmap v3762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:54.426 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6019 sec 2024-04-26T20:08:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:55 smithi167 ceph-mon[32751]: pgmap v3763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:55.392 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:55 smithi090 ceph-mon[29180]: pgmap v3763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:55.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6020 sec 2024-04-26T20:08:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:55 smithi119 ceph-mon[32752]: pgmap v3763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:08:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:56 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5617.060879 secs 2024-04-26T20:08:56.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6021 sec 2024-04-26T20:08:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:56 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5617.060879 secs 2024-04-26T20:08:56.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:56 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5617.060879 secs 2024-04-26T20:08:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5617.757350 secs 2024-04-26T20:08:57.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:57 smithi090 ceph-mon[29180]: pgmap v3764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:57.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5617.757350 secs 2024-04-26T20:08:57.702 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:57 smithi167 ceph-mon[32751]: pgmap v3764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:57.763 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5617.757350 secs 2024-04-26T20:08:57.823 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:57 smithi119 ceph-mon[32752]: pgmap v3764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:57.869 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6022 sec 2024-04-26T20:08:58.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6023 sec 2024-04-26T20:08:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:08:59 smithi090 ceph-mon[29180]: pgmap v3765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:59.426 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6024 sec 2024-04-26T20:08:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:08:59 smithi119 ceph-mon[32752]: pgmap v3765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:08:59.635 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:08:59 smithi167 ceph-mon[32751]: pgmap v3765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:00.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6025 sec 2024-04-26T20:09:01.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:01 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5622.060938 secs 2024-04-26T20:09:01.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:01 smithi167 ceph-mon[32751]: pgmap v3766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:01.534 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:01 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5622.060938 secs 2024-04-26T20:09:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:01 smithi090 ceph-mon[29180]: pgmap v3766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:01.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6026 sec 2024-04-26T20:09:01.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:01 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5622.060938 secs 2024-04-26T20:09:01.639 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:01 smithi119 ceph-mon[32752]: pgmap v3766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5622.757479 secs 2024-04-26T20:09:02.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6027 sec 2024-04-26T20:09:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5622.757479 secs 2024-04-26T20:09:02.580 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5622.757479 secs 2024-04-26T20:09:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:03 smithi090 ceph-mon[29180]: pgmap v3767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:03.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6028 sec 2024-04-26T20:09:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:03 smithi119 ceph-mon[32752]: pgmap v3767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:03.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:03 smithi167 ceph-mon[32751]: pgmap v3767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:04.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6029 sec 2024-04-26T20:09:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:05 smithi090 ceph-mon[29180]: pgmap v3768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:05.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6030 sec 2024-04-26T20:09:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:05 smithi119 ceph-mon[32752]: pgmap v3768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:05.605 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:05 smithi167 ceph-mon[32751]: pgmap v3768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:06 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5627.061021 secs 2024-04-26T20:09:06.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6031 sec 2024-04-26T20:09:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:06 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5627.061021 secs 2024-04-26T20:09:06.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:06 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5627.061021 secs 2024-04-26T20:09:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5627.757582 secs 2024-04-26T20:09:07.436 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:07 smithi090 ceph-mon[29180]: pgmap v3769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:07.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6032 sec 2024-04-26T20:09:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5627.757582 secs 2024-04-26T20:09:07.595 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:07 smithi119 ceph-mon[32752]: pgmap v3769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:07.662 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5627.757582 secs 2024-04-26T20:09:07.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:07 smithi167 ceph-mon[32751]: pgmap v3769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:08.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6033 sec 2024-04-26T20:09:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:09 smithi090 ceph-mon[29180]: pgmap v3770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:09.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6034 sec 2024-04-26T20:09:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:09 smithi119 ceph-mon[32752]: pgmap v3770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:09.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:09 smithi167 ceph-mon[32751]: pgmap v3770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:10.427 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6035 sec 2024-04-26T20:09:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:11 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5632.061096 secs 2024-04-26T20:09:11.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:11 smithi090 ceph-mon[29180]: pgmap v3771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:11.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:11 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5632.061096 secs 2024-04-26T20:09:11.632 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:11 smithi119 ceph-mon[32752]: pgmap v3771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:11.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6036 sec 2024-04-26T20:09:11.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:11 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5632.061096 secs 2024-04-26T20:09:11.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:11 smithi167 ceph-mon[32751]: pgmap v3771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5632.757667 secs 2024-04-26T20:09:12.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6037 sec 2024-04-26T20:09:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5632.757667 secs 2024-04-26T20:09:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5632.757667 secs 2024-04-26T20:09:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:13 smithi090 ceph-mon[29180]: pgmap v3772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:13.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6038 sec 2024-04-26T20:09:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:13 smithi119 ceph-mon[32752]: pgmap v3772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:13 smithi167 ceph-mon[32751]: pgmap v3772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:14.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6039 sec 2024-04-26T20:09:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:15 smithi090 ceph-mon[29180]: pgmap v3773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:15.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6040 sec 2024-04-26T20:09:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:15 smithi119 ceph-mon[32752]: pgmap v3773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:15.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:15 smithi167 ceph-mon[32751]: pgmap v3773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:16 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5637.061180 secs 2024-04-26T20:09:16.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6041 sec 2024-04-26T20:09:16.458 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:16 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5637.061180 secs 2024-04-26T20:09:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:16 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5637.061180 secs 2024-04-26T20:09:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5637.757734 secs 2024-04-26T20:09:17.450 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:17 smithi090 ceph-mon[29180]: pgmap v3774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:17.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6042 sec 2024-04-26T20:09:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5637.757734 secs 2024-04-26T20:09:17.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:17 smithi119 ceph-mon[32752]: pgmap v3774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:17.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5637.757734 secs 2024-04-26T20:09:17.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:17 smithi167 ceph-mon[32751]: pgmap v3774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:18.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6043 sec 2024-04-26T20:09:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:19 smithi090 ceph-mon[29180]: pgmap v3775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:19.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6044 sec 2024-04-26T20:09:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:19 smithi119 ceph-mon[32752]: pgmap v3775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:19.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:19 smithi167 ceph-mon[32751]: pgmap v3775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:20.428 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6045 sec 2024-04-26T20:09:21.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6046 sec 2024-04-26T20:09:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5642.061258 secs 2024-04-26T20:09:21.631 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:21 smithi119 ceph-mon[32752]: pgmap v3776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:21.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5642.061258 secs 2024-04-26T20:09:21.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:21 smithi090 ceph-mon[29180]: pgmap v3776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:21.752 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5642.061258 secs 2024-04-26T20:09:21.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:21 smithi167 ceph-mon[32751]: pgmap v3776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:22.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6047 sec 2024-04-26T20:09:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5642.757844 secs 2024-04-26T20:09:22.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5642.757844 secs 2024-04-26T20:09:22.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5642.757844 secs 2024-04-26T20:09:23.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6048 sec 2024-04-26T20:09:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:23 smithi119 ceph-mon[32752]: pgmap v3777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:23 smithi167 ceph-mon[32751]: pgmap v3777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:23 smithi090 ceph-mon[29180]: pgmap v3777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:24.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6049 sec 2024-04-26T20:09:25.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6050 sec 2024-04-26T20:09:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:25 smithi119 ceph-mon[32752]: pgmap v3778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:25.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:25 smithi090 ceph-mon[29180]: pgmap v3778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:25.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:25 smithi167 ceph-mon[32751]: pgmap v3778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:26.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6051 sec 2024-04-26T20:09:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:26 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5647.061321 secs 2024-04-26T20:09:26.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:26 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5647.061321 secs 2024-04-26T20:09:26.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:26 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5647.061321 secs 2024-04-26T20:09:27.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6052 sec 2024-04-26T20:09:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5647.757906 secs 2024-04-26T20:09:27.602 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:27 smithi119 ceph-mon[32752]: pgmap v3779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:27.650 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5647.757906 secs 2024-04-26T20:09:27.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:27 smithi090 ceph-mon[29180]: pgmap v3779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:27.668 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5647.757906 secs 2024-04-26T20:09:27.710 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:27 smithi167 ceph-mon[32751]: pgmap v3779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:28.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6053 sec 2024-04-26T20:09:29.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6054 sec 2024-04-26T20:09:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:29 smithi119 ceph-mon[32752]: pgmap v3780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:29.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:29 smithi167 ceph-mon[32751]: pgmap v3780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:29.662 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:29 smithi090 ceph-mon[29180]: pgmap v3780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:30.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6055 sec 2024-04-26T20:09:31.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6056 sec 2024-04-26T20:09:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:31 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5652.061402 secs 2024-04-26T20:09:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:31 smithi119 ceph-mon[32752]: pgmap v3781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:31 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5652.061402 secs 2024-04-26T20:09:31.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:31 smithi167 ceph-mon[32751]: pgmap v3781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:31 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5652.061402 secs 2024-04-26T20:09:31.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:31 smithi090 ceph-mon[29180]: pgmap v3781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:32.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6057 sec 2024-04-26T20:09:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5652.758029 secs 2024-04-26T20:09:32.597 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5652.758029 secs 2024-04-26T20:09:32.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5652.758029 secs 2024-04-26T20:09:33.414 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:33 smithi090 ceph-mon[29180]: pgmap v3782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:33.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6058 sec 2024-04-26T20:09:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:33 smithi119 ceph-mon[32752]: pgmap v3782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:33 smithi167 ceph-mon[32751]: pgmap v3782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:34.429 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6059 sec 2024-04-26T20:09:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:09:34.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:09:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:09:35.430 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6060 sec 2024-04-26T20:09:35.519 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:35 smithi119 ceph-mon[32752]: pgmap v3783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:35.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:35 smithi090 ceph-mon[29180]: pgmap v3783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:35.641 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:35 smithi167 ceph-mon[32751]: pgmap v3783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:36.430 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6061 sec 2024-04-26T20:09:36.630 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:36 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 5657.061467 secs 2024-04-26T20:09:36.715 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:36 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 5657.061467 secs 2024-04-26T20:09:36.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:36 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 5657.061467 secs 2024-04-26T20:09:37.430 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6062 sec 2024-04-26T20:09:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5657.758122 secs 2024-04-26T20:09:37.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:37 smithi119 ceph-mon[32752]: pgmap v3784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:37.649 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5657.758122 secs 2024-04-26T20:09:37.659 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:37 smithi090 ceph-mon[29180]: pgmap v3784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:37.671 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5657.758122 secs 2024-04-26T20:09:37.683 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:37 smithi167 ceph-mon[32751]: pgmap v3784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:38.430 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6063 sec 2024-04-26T20:09:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: pgmap v3785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:39.374 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.383 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.465 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.521 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:09:39.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:09:39.616 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: pgmap v3785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:39.762 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.842 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.854 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:09:39.867 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:09:39.921 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:39.958 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6064 sec 2024-04-26T20:09:40.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: pgmap v3785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:40.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:40.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:40.226 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:40.240 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:40.314 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:09:40.387 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:09:40.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:09:40.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6065 sec 2024-04-26T20:09:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:40 smithi119 ceph-mon[32752]: pgmap v3786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:09:40.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:40 smithi119 ceph-mon[32752]: pgmap v3787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:40.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:40 smithi167 ceph-mon[32751]: pgmap v3786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:09:40.588 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:40 smithi167 ceph-mon[32751]: pgmap v3787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:40 smithi090 ceph-mon[29180]: pgmap v3786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:09:40.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:40 smithi090 ceph-mon[29180]: pgmap v3787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:41.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6066 sec 2024-04-26T20:09:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:41 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5662.061549 secs 2024-04-26T20:09:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:41 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5662.061549 secs 2024-04-26T20:09:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:41 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5662.061549 secs 2024-04-26T20:09:42.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6067 sec 2024-04-26T20:09:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5662.758234 secs 2024-04-26T20:09:42.602 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:42 smithi119 ceph-mon[32752]: pgmap v3788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:42.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5662.758234 secs 2024-04-26T20:09:42.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:42 smithi167 ceph-mon[32751]: pgmap v3788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5662.758234 secs 2024-04-26T20:09:42.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:42 smithi090 ceph-mon[29180]: pgmap v3788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:43.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6068 sec 2024-04-26T20:09:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:09:43.633 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:09:43.644 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:09:43.652 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:09:43.681 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:09:43.690 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:09:44.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6069 sec 2024-04-26T20:09:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:44 smithi119 ceph-mon[32752]: pgmap v3789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T20:09:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:44 smithi167 ceph-mon[32751]: pgmap v3789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T20:09:44.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:44 smithi090 ceph-mon[29180]: pgmap v3789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T20:09:45.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6070 sec 2024-04-26T20:09:46.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6071 sec 2024-04-26T20:09:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:46 smithi119 ceph-mon[32752]: pgmap v3790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:46.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:46 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5667.061632 secs 2024-04-26T20:09:46.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:46 smithi090 ceph-mon[29180]: pgmap v3790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:46.716 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:46 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5667.061632 secs 2024-04-26T20:09:46.724 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:46 smithi167 ceph-mon[32751]: pgmap v3790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:46.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:46 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5667.061632 secs 2024-04-26T20:09:47.431 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6072 sec 2024-04-26T20:09:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5667.758307 secs 2024-04-26T20:09:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5667.758307 secs 2024-04-26T20:09:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5667.758307 secs 2024-04-26T20:09:48.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6073 sec 2024-04-26T20:09:48.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:48 smithi167 ceph-mon[32751]: pgmap v3791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:48.603 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:48 smithi119 ceph-mon[32752]: pgmap v3791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:48 smithi090 ceph-mon[29180]: pgmap v3791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:49 smithi119 ceph-mon[32752]: pgmap v3792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:09:49.409 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:49 smithi090 ceph-mon[29180]: pgmap v3792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:09:49.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6074 sec 2024-04-26T20:09:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:49 smithi167 ceph-mon[32751]: pgmap v3792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:09:50.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6075 sec 2024-04-26T20:09:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:50 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5672.061716 secs 2024-04-26T20:09:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:50 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5672.061716 secs 2024-04-26T20:09:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:50 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5672.061716 secs 2024-04-26T20:09:51.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6076 sec 2024-04-26T20:09:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5672.758418 secs 2024-04-26T20:09:52.183 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:51 smithi119 ceph-mon[32752]: pgmap v3793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:52.243 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5672.758418 secs 2024-04-26T20:09:52.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:51 smithi090 ceph-mon[29180]: pgmap v3793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:52.348 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5672.758418 secs 2024-04-26T20:09:52.400 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:51 smithi167 ceph-mon[32751]: pgmap v3793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:52.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6077 sec 2024-04-26T20:09:53.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6078 sec 2024-04-26T20:09:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:53 smithi119 ceph-mon[32752]: pgmap v3794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:53 smithi167 ceph-mon[32751]: pgmap v3794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:53 smithi090 ceph-mon[29180]: pgmap v3794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:54.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6079 sec 2024-04-26T20:09:55.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6080 sec 2024-04-26T20:09:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:55 smithi119 ceph-mon[32752]: pgmap v3795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:56.141 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:55 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5677.061799 secs 2024-04-26T20:09:56.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:55 smithi090 ceph-mon[29180]: pgmap v3795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:56.218 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:55 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5677.061799 secs 2024-04-26T20:09:56.263 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:55 smithi167 ceph-mon[32751]: pgmap v3795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:09:56.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:55 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5677.061799 secs 2024-04-26T20:09:56.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6081 sec 2024-04-26T20:09:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5677.758567 secs 2024-04-26T20:09:57.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5677.758567 secs 2024-04-26T20:09:57.180 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5677.758567 secs 2024-04-26T20:09:57.432 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6082 sec 2024-04-26T20:09:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:57 smithi119 ceph-mon[32752]: pgmap v3796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:58.096 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:57 smithi167 ceph-mon[32751]: pgmap v3796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:58.143 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:57 smithi090 ceph-mon[29180]: pgmap v3796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:09:58.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6083 sec 2024-04-26T20:09:59.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6084 sec 2024-04-26T20:10:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:09:59 smithi119 ceph-mon[32752]: pgmap v3797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:00.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:09:59 smithi167 ceph-mon[32751]: pgmap v3797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:00.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:09:59 smithi090 ceph-mon[29180]: pgmap v3797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:00.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6085 sec 2024-04-26T20:10:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:00 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5682.061886 secs 2024-04-26T20:10:01.078 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:10:01.153 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:00 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5682.061886 secs 2024-04-26T20:10:01.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:10:01.185 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:00 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5682.061886 secs 2024-04-26T20:10:01.197 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:10:01.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6086 sec 2024-04-26T20:10:02.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5682.758629 secs 2024-04-26T20:10:02.060 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:01 smithi090 ceph-mon[29180]: pgmap v3798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:02.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5682.758629 secs 2024-04-26T20:10:02.154 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:01 smithi167 ceph-mon[32751]: pgmap v3798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:02.221 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5682.758629 secs 2024-04-26T20:10:02.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:01 smithi119 ceph-mon[32752]: pgmap v3798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:02.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6087 sec 2024-04-26T20:10:03.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6088 sec 2024-04-26T20:10:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:03 smithi119 ceph-mon[32752]: pgmap v3799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:04.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:03 smithi167 ceph-mon[32751]: pgmap v3799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:03 smithi090 ceph-mon[29180]: pgmap v3799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:04.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6089 sec 2024-04-26T20:10:05.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6090 sec 2024-04-26T20:10:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:05 smithi119 ceph-mon[32752]: pgmap v3800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:06.139 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:05 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5687.061946 secs 2024-04-26T20:10:06.215 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:05 smithi090 ceph-mon[29180]: pgmap v3800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:06.243 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:05 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5687.061946 secs 2024-04-26T20:10:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:05 smithi167 ceph-mon[32751]: pgmap v3800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:06.356 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:05 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5687.061946 secs 2024-04-26T20:10:06.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6091 sec 2024-04-26T20:10:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5687.758743 secs 2024-04-26T20:10:07.093 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5687.758743 secs 2024-04-26T20:10:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5687.758743 secs 2024-04-26T20:10:07.433 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6092 sec 2024-04-26T20:10:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:07 smithi119 ceph-mon[32752]: pgmap v3801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:07 smithi090 ceph-mon[29180]: pgmap v3801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:08.196 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:07 smithi167 ceph-mon[32751]: pgmap v3801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:08.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6093 sec 2024-04-26T20:10:09.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6094 sec 2024-04-26T20:10:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:09 smithi119 ceph-mon[32752]: pgmap v3802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:10.111 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:09 smithi167 ceph-mon[32751]: pgmap v3802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:10.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:09 smithi090 ceph-mon[29180]: pgmap v3802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:10.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6095 sec 2024-04-26T20:10:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:10 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5692.062028 secs 2024-04-26T20:10:11.094 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:10 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5692.062028 secs 2024-04-26T20:10:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:10 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5692.062028 secs 2024-04-26T20:10:11.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6096 sec 2024-04-26T20:10:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5692.758886 secs 2024-04-26T20:10:12.126 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:11 smithi119 ceph-mon[32752]: pgmap v3803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:12.227 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5692.758886 secs 2024-04-26T20:10:12.291 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:11 smithi090 ceph-mon[29180]: pgmap v3803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:12.332 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5692.758886 secs 2024-04-26T20:10:12.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:11 smithi167 ceph-mon[32751]: pgmap v3803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:12.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6097 sec 2024-04-26T20:10:13.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6098 sec 2024-04-26T20:10:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:13 smithi119 ceph-mon[32752]: pgmap v3804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:14.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:13 smithi090 ceph-mon[29180]: pgmap v3804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:14.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:13 smithi167 ceph-mon[32751]: pgmap v3804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:14.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6099 sec 2024-04-26T20:10:15.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6100 sec 2024-04-26T20:10:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:15 smithi119 ceph-mon[32752]: pgmap v3805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:16.083 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5697.062112 secs 2024-04-26T20:10:16.114 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:15 smithi167 ceph-mon[32751]: pgmap v3805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:16.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5697.062112 secs 2024-04-26T20:10:16.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:15 smithi090 ceph-mon[29180]: pgmap v3805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:16.196 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5697.062112 secs 2024-04-26T20:10:16.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6101 sec 2024-04-26T20:10:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5697.759011 secs 2024-04-26T20:10:17.158 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5697.759011 secs 2024-04-26T20:10:17.170 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5697.759011 secs 2024-04-26T20:10:17.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6102 sec 2024-04-26T20:10:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:17 smithi119 ceph-mon[32752]: pgmap v3806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:18.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:17 smithi167 ceph-mon[32751]: pgmap v3806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:17 smithi090 ceph-mon[29180]: pgmap v3806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:18.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6103 sec 2024-04-26T20:10:19.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6104 sec 2024-04-26T20:10:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:19 smithi119 ceph-mon[32752]: pgmap v3807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:20.092 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:19 smithi167 ceph-mon[32751]: pgmap v3807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:19 smithi090 ceph-mon[29180]: pgmap v3807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:20.434 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6105 sec 2024-04-26T20:10:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:20 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5702.062193 secs 2024-04-26T20:10:21.142 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:20 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5702.062193 secs 2024-04-26T20:10:21.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:20 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5702.062193 secs 2024-04-26T20:10:21.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6106 sec 2024-04-26T20:10:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5702.759086 secs 2024-04-26T20:10:22.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:21 smithi119 ceph-mon[32752]: pgmap v3808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5702.759086 secs 2024-04-26T20:10:22.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:21 smithi167 ceph-mon[32751]: pgmap v3808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5702.759086 secs 2024-04-26T20:10:22.175 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:21 smithi090 ceph-mon[29180]: pgmap v3808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:22.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6107 sec 2024-04-26T20:10:23.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6108 sec 2024-04-26T20:10:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:23 smithi119 ceph-mon[32752]: pgmap v3809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:23 smithi090 ceph-mon[29180]: pgmap v3809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:24.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:23 smithi167 ceph-mon[32751]: pgmap v3809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:24.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6109 sec 2024-04-26T20:10:25.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6110 sec 2024-04-26T20:10:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:25 smithi119 ceph-mon[32752]: pgmap v3810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:26.125 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:25 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5707.062276 secs 2024-04-26T20:10:26.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:25 smithi090 ceph-mon[29180]: pgmap v3810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:26.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:25 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5707.062276 secs 2024-04-26T20:10:26.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:25 smithi167 ceph-mon[32751]: pgmap v3810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:26.179 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:25 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5707.062276 secs 2024-04-26T20:10:26.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6111 sec 2024-04-26T20:10:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5707.759222 secs 2024-04-26T20:10:27.112 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5707.759222 secs 2024-04-26T20:10:27.168 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5707.759222 secs 2024-04-26T20:10:27.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6112 sec 2024-04-26T20:10:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:27 smithi119 ceph-mon[32752]: pgmap v3811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:28.110 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:27 smithi167 ceph-mon[32751]: pgmap v3811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:28.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:27 smithi090 ceph-mon[29180]: pgmap v3811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:28.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6113 sec 2024-04-26T20:10:29.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6114 sec 2024-04-26T20:10:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:29 smithi119 ceph-mon[32752]: pgmap v3812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:29 smithi167 ceph-mon[32751]: pgmap v3812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:29 smithi090 ceph-mon[29180]: pgmap v3812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:30.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6115 sec 2024-04-26T20:10:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:30 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5712.062359 secs 2024-04-26T20:10:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:30 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5712.062359 secs 2024-04-26T20:10:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:30 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5712.062359 secs 2024-04-26T20:10:31.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6116 sec 2024-04-26T20:10:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5712.759362 secs 2024-04-26T20:10:32.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:31 smithi119 ceph-mon[32752]: pgmap v3813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:32.144 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5712.759362 secs 2024-04-26T20:10:32.174 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:31 smithi090 ceph-mon[29180]: pgmap v3813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:32.205 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5712.759362 secs 2024-04-26T20:10:32.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:31 smithi167 ceph-mon[32751]: pgmap v3813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:32.435 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6117 sec 2024-04-26T20:10:33.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6118 sec 2024-04-26T20:10:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:33 smithi119 ceph-mon[32752]: pgmap v3814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:34.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:33 smithi090 ceph-mon[29180]: pgmap v3814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:34.166 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:33 smithi167 ceph-mon[32751]: pgmap v3814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:34.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6119 sec 2024-04-26T20:10:35.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6120 sec 2024-04-26T20:10:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:35 smithi119 ceph-mon[32752]: pgmap v3815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:36.169 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:35 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5717.062444 secs 2024-04-26T20:10:36.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:35 smithi090 ceph-mon[29180]: pgmap v3815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:36.194 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:35 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5717.062444 secs 2024-04-26T20:10:36.207 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:35 smithi167 ceph-mon[32751]: pgmap v3815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:36.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:35 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5717.062444 secs 2024-04-26T20:10:36.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6121 sec 2024-04-26T20:10:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5717.759476 secs 2024-04-26T20:10:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5717.759476 secs 2024-04-26T20:10:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5717.759476 secs 2024-04-26T20:10:37.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6122 sec 2024-04-26T20:10:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:37 smithi167 ceph-mon[32751]: pgmap v3816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:37 smithi090 ceph-mon[29180]: pgmap v3816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:37 smithi119 ceph-mon[32752]: pgmap v3816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:38.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6123 sec 2024-04-26T20:10:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:10:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:10:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:10:39.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6124 sec 2024-04-26T20:10:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:39 smithi167 ceph-mon[32751]: pgmap v3817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:40.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:39 smithi090 ceph-mon[29180]: pgmap v3817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:39 smithi119 ceph-mon[32752]: pgmap v3817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:40.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6125 sec 2024-04-26T20:10:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:40 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5722.062546 secs 2024-04-26T20:10:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:40 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5722.062546 secs 2024-04-26T20:10:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:40 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5722.062546 secs 2024-04-26T20:10:41.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6126 sec 2024-04-26T20:10:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5722.759542 secs 2024-04-26T20:10:42.192 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:41 smithi090 ceph-mon[29180]: pgmap v3818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:42.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5722.759542 secs 2024-04-26T20:10:42.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:41 smithi167 ceph-mon[32751]: pgmap v3818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5722.759542 secs 2024-04-26T20:10:42.369 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:41 smithi119 ceph-mon[32752]: pgmap v3818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:42.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6127 sec 2024-04-26T20:10:43.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6128 sec 2024-04-26T20:10:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: pgmap v3819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:44.412 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:44.458 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:44.520 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:10:44.548 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:10:44.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: pgmap v3820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-26T20:10:44.564 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: pgmap v3821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-26T20:10:44.572 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:44.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: pgmap v3819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:44.586 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:44.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:44.736 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:10:44.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:10:44.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: pgmap v3820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-26T20:10:44.960 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: pgmap v3821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-26T20:10:45.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:45.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: pgmap v3819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:45.096 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:45.172 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:45.223 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:10:45.263 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:10:45.326 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: pgmap v3820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-26T20:10:45.343 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: pgmap v3821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-26T20:10:45.352 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:10:45.360 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6129 sec 2024-04-26T20:10:45.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6130 sec 2024-04-26T20:10:46.436 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6131 sec 2024-04-26T20:10:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:46 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5727.062605 secs 2024-04-26T20:10:46.581 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:46 smithi119 ceph-mon[32752]: pgmap v3822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:46.610 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:46 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5727.062605 secs 2024-04-26T20:10:46.618 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:46 smithi167 ceph-mon[32751]: pgmap v3822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:46 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5727.062605 secs 2024-04-26T20:10:46.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:46 smithi090 ceph-mon[29180]: pgmap v3822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:47.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6132 sec 2024-04-26T20:10:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5727.759628 secs 2024-04-26T20:10:47.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5727.759628 secs 2024-04-26T20:10:47.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5727.759628 secs 2024-04-26T20:10:48.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6133 sec 2024-04-26T20:10:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:48 smithi119 ceph-mon[32752]: pgmap v3823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-26T20:10:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:48 smithi167 ceph-mon[32751]: pgmap v3823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-26T20:10:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:48 smithi090 ceph-mon[29180]: pgmap v3823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-26T20:10:49.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6134 sec 2024-04-26T20:10:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:50 smithi119 ceph-mon[32752]: pgmap v3824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:50.347 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:50 smithi167 ceph-mon[32751]: pgmap v3824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:50.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:50 smithi090 ceph-mon[29180]: pgmap v3824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:50.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6135 sec 2024-04-26T20:10:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:51 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5732.062685 secs 2024-04-26T20:10:51.405 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:51 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5732.062685 secs 2024-04-26T20:10:51.453 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:51 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5732.062685 secs 2024-04-26T20:10:51.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6136 sec 2024-04-26T20:10:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5732.759687 secs 2024-04-26T20:10:52.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:52 smithi119 ceph-mon[32752]: pgmap v3825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5732.759687 secs 2024-04-26T20:10:52.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:52 smithi167 ceph-mon[32751]: pgmap v3825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5732.759687 secs 2024-04-26T20:10:52.400 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:52 smithi090 ceph-mon[29180]: pgmap v3825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:52.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6137 sec 2024-04-26T20:10:53.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6138 sec 2024-04-26T20:10:54.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6139 sec 2024-04-26T20:10:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:54 smithi119 ceph-mon[32752]: pgmap v3826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:10:54.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:54 smithi167 ceph-mon[32751]: pgmap v3826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:10:54.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:54 smithi090 ceph-mon[29180]: pgmap v3826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:10:55.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6140 sec 2024-04-26T20:10:56.437 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6141 sec 2024-04-26T20:10:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:56 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5737.062767 secs 2024-04-26T20:10:56.699 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:56 smithi119 ceph-mon[32752]: pgmap v3827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:56.723 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:56 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5737.062767 secs 2024-04-26T20:10:56.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:56 smithi090 ceph-mon[29180]: pgmap v3827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:56.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:56 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5737.062767 secs 2024-04-26T20:10:56.895 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:56 smithi167 ceph-mon[32751]: pgmap v3827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:10:57.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6142 sec 2024-04-26T20:10:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5737.759810 secs 2024-04-26T20:10:57.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5737.759810 secs 2024-04-26T20:10:57.671 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5737.759810 secs 2024-04-26T20:10:58.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6143 sec 2024-04-26T20:10:58.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:10:58 smithi167 ceph-mon[32751]: pgmap v3828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:58.656 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:10:58 smithi119 ceph-mon[32752]: pgmap v3828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:58.709 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:10:58 smithi090 ceph-mon[29180]: pgmap v3828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:10:59.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6144 sec 2024-04-26T20:11:00.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6145 sec 2024-04-26T20:11:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:00 smithi119 ceph-mon[32752]: pgmap v3829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:00.603 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:00 smithi167 ceph-mon[32751]: pgmap v3829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:00.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:00 smithi090 ceph-mon[29180]: pgmap v3829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:01.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6146 sec 2024-04-26T20:11:01.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:01 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5742.062845 secs 2024-04-26T20:11:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:01 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5742.062845 secs 2024-04-26T20:11:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:01 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5742.062845 secs 2024-04-26T20:11:02.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6147 sec 2024-04-26T20:11:02.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5742.759931 secs 2024-04-26T20:11:02.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:02 smithi167 ceph-mon[32751]: pgmap v3830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:02.639 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5742.759931 secs 2024-04-26T20:11:02.727 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:02 smithi119 ceph-mon[32752]: pgmap v3830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:02.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5742.759931 secs 2024-04-26T20:11:02.756 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:02 smithi090 ceph-mon[29180]: pgmap v3830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:03.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6148 sec 2024-04-26T20:11:04.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6149 sec 2024-04-26T20:11:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:04 smithi119 ceph-mon[32752]: pgmap v3831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:04.612 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:04 smithi167 ceph-mon[32751]: pgmap v3831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:04.662 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:04 smithi090 ceph-mon[29180]: pgmap v3831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:05.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6150 sec 2024-04-26T20:11:06.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6151 sec 2024-04-26T20:11:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:06 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5747.062929 secs 2024-04-26T20:11:06.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:06 smithi119 ceph-mon[32752]: pgmap v3832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:06.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:06 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5747.062929 secs 2024-04-26T20:11:06.583 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:06 smithi167 ceph-mon[32751]: pgmap v3832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:06 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5747.062929 secs 2024-04-26T20:11:06.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:06 smithi090 ceph-mon[29180]: pgmap v3832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:07.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6152 sec 2024-04-26T20:11:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5747.760063 secs 2024-04-26T20:11:07.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5747.760063 secs 2024-04-26T20:11:07.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5747.760063 secs 2024-04-26T20:11:08.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6153 sec 2024-04-26T20:11:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:08 smithi119 ceph-mon[32752]: pgmap v3833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:08.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:08 smithi167 ceph-mon[32751]: pgmap v3833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:08 smithi090 ceph-mon[29180]: pgmap v3833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:09.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6154 sec 2024-04-26T20:11:10.438 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6155 sec 2024-04-26T20:11:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:10 smithi119 ceph-mon[32752]: pgmap v3834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:10.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:10 smithi167 ceph-mon[32751]: pgmap v3834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:10.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:10 smithi090 ceph-mon[29180]: pgmap v3834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:11.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6156 sec 2024-04-26T20:11:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:11 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5752.063009 secs 2024-04-26T20:11:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:11 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5752.063009 secs 2024-04-26T20:11:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:11 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5752.063009 secs 2024-04-26T20:11:12.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6157 sec 2024-04-26T20:11:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5752.760189 secs 2024-04-26T20:11:12.619 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:12 smithi119 ceph-mon[32752]: pgmap v3835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:12.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5752.760189 secs 2024-04-26T20:11:12.712 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:12 smithi090 ceph-mon[29180]: pgmap v3835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:12.767 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5752.760189 secs 2024-04-26T20:11:12.773 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:12 smithi167 ceph-mon[32751]: pgmap v3835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:13.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6158 sec 2024-04-26T20:11:14.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6159 sec 2024-04-26T20:11:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:14 smithi119 ceph-mon[32752]: pgmap v3836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:14.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:14 smithi167 ceph-mon[32751]: pgmap v3836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:14 smithi090 ceph-mon[29180]: pgmap v3836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:15.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6160 sec 2024-04-26T20:11:16.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6161 sec 2024-04-26T20:11:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:16 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5757.063092 secs 2024-04-26T20:11:16.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:16 smithi119 ceph-mon[32752]: pgmap v3837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:16.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:16 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5757.063092 secs 2024-04-26T20:11:16.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:16 smithi090 ceph-mon[29180]: pgmap v3837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:16.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:16 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5757.063092 secs 2024-04-26T20:11:16.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:16 smithi167 ceph-mon[32751]: pgmap v3837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:17.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6162 sec 2024-04-26T20:11:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5757.760282 secs 2024-04-26T20:11:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5757.760282 secs 2024-04-26T20:11:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5757.760282 secs 2024-04-26T20:11:18.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6163 sec 2024-04-26T20:11:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:18 smithi119 ceph-mon[32752]: pgmap v3838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:18 smithi167 ceph-mon[32751]: pgmap v3838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:18 smithi090 ceph-mon[29180]: pgmap v3838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:19.439 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6164 sec 2024-04-26T20:11:20.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6165 sec 2024-04-26T20:11:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:20 smithi119 ceph-mon[32752]: pgmap v3839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:20.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:20 smithi167 ceph-mon[32751]: pgmap v3839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:20.656 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:20 smithi090 ceph-mon[29180]: pgmap v3839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:21.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6166 sec 2024-04-26T20:11:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:21 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5762.063205 secs 2024-04-26T20:11:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:21 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5762.063205 secs 2024-04-26T20:11:21.637 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:21 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5762.063205 secs 2024-04-26T20:11:22.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6167 sec 2024-04-26T20:11:22.470 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:22 smithi090 ceph-mon[29180]: 13 slow requests, 5 included below; oldest blocked for > 5762.760414 secs 2024-04-26T20:11:22.475 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:22 smithi090 ceph-mon[29180]: slow request 5762.760414 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:22 smithi090 ceph-mon[29180]: slow request 5762.758853 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:22 smithi090 ceph-mon[29180]: slow request 5762.757384 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:22 smithi090 ceph-mon[29180]: slow request 5762.756914 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.715 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:22 smithi090 ceph-mon[29180]: slow request 5762.744649 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:22 smithi090 ceph-mon[29180]: pgmap v3840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:22.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:22 smithi167 ceph-mon[32751]: 13 slow requests, 5 included below; oldest blocked for > 5762.760414 secs 2024-04-26T20:11:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:22 smithi167 ceph-mon[32751]: slow request 5762.760414 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:22 smithi167 ceph-mon[32751]: slow request 5762.758853 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.834 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:22 smithi167 ceph-mon[32751]: slow request 5762.757384 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.841 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:22 smithi167 ceph-mon[32751]: slow request 5762.756914 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.900 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:22 smithi167 ceph-mon[32751]: slow request 5762.744649 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:22.947 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:22 smithi167 ceph-mon[32751]: pgmap v3840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:23.000 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:22 smithi119 ceph-mon[32752]: 13 slow requests, 5 included below; oldest blocked for > 5762.760414 secs 2024-04-26T20:11:23.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:22 smithi119 ceph-mon[32752]: slow request 5762.760414 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:23.077 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:22 smithi119 ceph-mon[32752]: slow request 5762.758853 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:23.150 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:22 smithi119 ceph-mon[32752]: slow request 5762.757384 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:23.197 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:22 smithi119 ceph-mon[32752]: slow request 5762.756914 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:23.257 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:22 smithi119 ceph-mon[32752]: slow request 5762.744649 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:11:23.327 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:22 smithi119 ceph-mon[32752]: pgmap v3840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:23.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6168 sec 2024-04-26T20:11:24.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6169 sec 2024-04-26T20:11:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:24 smithi119 ceph-mon[32752]: pgmap v3841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:24.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:24 smithi167 ceph-mon[32751]: pgmap v3841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:24.642 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:24 smithi090 ceph-mon[29180]: pgmap v3841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:25.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6170 sec 2024-04-26T20:11:26.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6171 sec 2024-04-26T20:11:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:26 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5767.063264 secs 2024-04-26T20:11:26.615 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:26 smithi119 ceph-mon[32752]: pgmap v3842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:26.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:26 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5767.063264 secs 2024-04-26T20:11:26.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:26 smithi090 ceph-mon[29180]: pgmap v3842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:26.698 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:26 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5767.063264 secs 2024-04-26T20:11:26.706 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:26 smithi167 ceph-mon[32751]: pgmap v3842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:27.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6172 sec 2024-04-26T20:11:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:27 smithi119 ceph-mon[32752]: 13 slow requests, 4 included below; oldest blocked for > 5767.760541 secs 2024-04-26T20:11:27.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:27 smithi119 ceph-mon[32752]: slow request 5764.175905 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T20:11:27.604 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:27 smithi119 ceph-mon[32752]: slow request 5764.175568 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T20:11:27.653 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:27 smithi119 ceph-mon[32752]: slow request 5764.066734 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T20:11:27.692 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:27 smithi119 ceph-mon[32752]: slow request 5764.062445 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T20:11:27.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:27 smithi090 ceph-mon[29180]: 13 slow requests, 4 included below; oldest blocked for > 5767.760541 secs 2024-04-26T20:11:27.837 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:27 smithi090 ceph-mon[29180]: slow request 5764.175905 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T20:11:27.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:27 smithi090 ceph-mon[29180]: slow request 5764.175568 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T20:11:27.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:27 smithi090 ceph-mon[29180]: slow request 5764.066734 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T20:11:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:27 smithi090 ceph-mon[29180]: slow request 5764.062445 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T20:11:27.939 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:27 smithi167 ceph-mon[32751]: 13 slow requests, 4 included below; oldest blocked for > 5767.760541 secs 2024-04-26T20:11:27.985 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:27 smithi167 ceph-mon[32751]: slow request 5764.175905 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T20:11:28.038 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:27 smithi167 ceph-mon[32751]: slow request 5764.175568 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T20:11:28.161 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:27 smithi167 ceph-mon[32751]: slow request 5764.066734 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T20:11:28.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:27 smithi167 ceph-mon[32751]: slow request 5764.062445 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T20:11:28.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6173 sec 2024-04-26T20:11:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:28 smithi119 ceph-mon[32752]: pgmap v3843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:28.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:28 smithi090 ceph-mon[29180]: pgmap v3843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:28.643 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:28 smithi167 ceph-mon[32751]: pgmap v3843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:29.226 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:11:29.225+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:11:29.248 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:11:29.225+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:11:29.284 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:11:29.225+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 48 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:11:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:11:29.225+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 48 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:11:29.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6174 sec 2024-04-26T20:11:30.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6175 sec 2024-04-26T20:11:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:30 smithi119 ceph-mon[32752]: pgmap v3844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:30.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:30 smithi167 ceph-mon[32751]: pgmap v3844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:30 smithi090 ceph-mon[29180]: pgmap v3844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:11:31.440 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6176 sec 2024-04-26T20:11:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:31 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5772.063347 secs 2024-04-26T20:11:31.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:31 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5772.063347 secs 2024-04-26T20:11:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:31 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5772.063347 secs 2024-04-26T20:11:32.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6177 sec 2024-04-26T20:11:32.538 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5772.760677 secs 2024-04-26T20:11:32.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:32 smithi090 ceph-mon[29180]: pgmap v3845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:32.560 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:11:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:11:32.294+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:11:32.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:11:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:11:32.294+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:11:32.669 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:11:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:11:32.294+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:11:32.694 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:11:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:11:32.294+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:11:32.762 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:11:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:11:32.294+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:11:32.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5772.760677 secs 2024-04-26T20:11:32.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:32 smithi167 ceph-mon[32751]: pgmap v3845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:32.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5772.760677 secs 2024-04-26T20:11:32.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:32 smithi119 ceph-mon[32752]: pgmap v3845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:33.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6178 sec 2024-04-26T20:11:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:33 smithi119 ceph-mon[32752]: pgmap v3846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:33.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:33 smithi167 ceph-mon[32751]: pgmap v3846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:33.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:33 smithi090 ceph-mon[29180]: pgmap v3846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:34.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6179 sec 2024-04-26T20:11:35.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6180 sec 2024-04-26T20:11:36.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6181 sec 2024-04-26T20:11:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:36 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5777.063430 secs 2024-04-26T20:11:36.598 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:36 smithi119 ceph-mon[32752]: pgmap v3847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:11:36.643 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:36 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5777.063430 secs 2024-04-26T20:11:36.702 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:36 smithi090 ceph-mon[29180]: pgmap v3847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:11:36.879 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:36 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5777.063430 secs 2024-04-26T20:11:36.919 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:36 smithi167 ceph-mon[32751]: pgmap v3847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:11:37.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6182 sec 2024-04-26T20:11:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5777.760803 secs 2024-04-26T20:11:37.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5777.760803 secs 2024-04-26T20:11:37.637 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5777.760803 secs 2024-04-26T20:11:38.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6183 sec 2024-04-26T20:11:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:38 smithi119 ceph-mon[32752]: pgmap v3848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:38.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:38 smithi167 ceph-mon[32751]: pgmap v3848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:38 smithi090 ceph-mon[29180]: pgmap v3848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:39.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6184 sec 2024-04-26T20:11:40.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6185 sec 2024-04-26T20:11:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:40 smithi119 ceph-mon[32752]: pgmap v3849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:11:40.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:40 smithi167 ceph-mon[32751]: pgmap v3849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:11:40.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:40 smithi090 ceph-mon[29180]: pgmap v3849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:11:41.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6186 sec 2024-04-26T20:11:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:41 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5782.063518 secs 2024-04-26T20:11:41.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:41 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5782.063518 secs 2024-04-26T20:11:41.661 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:41 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5782.063518 secs 2024-04-26T20:11:42.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6187 sec 2024-04-26T20:11:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5782.760914 secs 2024-04-26T20:11:42.616 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:42 smithi119 ceph-mon[32752]: pgmap v3850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:42.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5782.760914 secs 2024-04-26T20:11:42.750 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:42 smithi090 ceph-mon[29180]: pgmap v3850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:42.786 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5782.760914 secs 2024-04-26T20:11:43.101 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:42 smithi167 ceph-mon[32751]: pgmap v3850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:43.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6188 sec 2024-04-26T20:11:44.441 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6189 sec 2024-04-26T20:11:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:44 smithi119 ceph-mon[32752]: pgmap v3851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:44.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:11:44.650 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:11:44.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:11:44.741 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:11:44.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:44 smithi090 ceph-mon[29180]: pgmap v3851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:44.828 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:11:44.839 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:11:44.845 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:11:44.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:11:44.916 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:44 smithi167 ceph-mon[32751]: pgmap v3851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:11:44.926 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:11:44.985 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:11:45.043 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:11:45.130 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:11:45.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6190 sec 2024-04-26T20:11:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:45 smithi119 ceph-mon[32752]: pgmap v3852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 189 B/s wr, 2 op/s 2024-04-26T20:11:45.635 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:45 smithi119 ceph-mon[32752]: pgmap v3853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:45.694 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:45 smithi090 ceph-mon[29180]: pgmap v3852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 189 B/s wr, 2 op/s 2024-04-26T20:11:45.721 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:45 smithi090 ceph-mon[29180]: pgmap v3853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:45.733 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:45 smithi167 ceph-mon[32751]: pgmap v3852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 189 B/s wr, 2 op/s 2024-04-26T20:11:45.747 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:45 smithi167 ceph-mon[32751]: pgmap v3853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:46.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6191 sec 2024-04-26T20:11:46.489 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:46 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5787.063603 secs 2024-04-26T20:11:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:46 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5787.063603 secs 2024-04-26T20:11:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:46 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5787.063603 secs 2024-04-26T20:11:47.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6192 sec 2024-04-26T20:11:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5787.761027 secs 2024-04-26T20:11:47.587 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:47 smithi119 ceph-mon[32752]: pgmap v3854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:47.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5787.761027 secs 2024-04-26T20:11:47.727 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:47 smithi090 ceph-mon[29180]: pgmap v3854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:47.781 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5787.761027 secs 2024-04-26T20:11:47.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:47 smithi167 ceph-mon[32751]: pgmap v3854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:48.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6193 sec 2024-04-26T20:11:49.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6194 sec 2024-04-26T20:11:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:49 smithi119 ceph-mon[32752]: pgmap v3855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:11:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:49 smithi167 ceph-mon[32751]: pgmap v3855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:11:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:49 smithi090 ceph-mon[29180]: pgmap v3855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:11:50.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6195 sec 2024-04-26T20:11:51.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6196 sec 2024-04-26T20:11:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:51 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5792.063685 secs 2024-04-26T20:11:51.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:51 smithi119 ceph-mon[32752]: pgmap v3856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:51 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5792.063685 secs 2024-04-26T20:11:51.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:51 smithi167 ceph-mon[32751]: pgmap v3856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:51 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5792.063685 secs 2024-04-26T20:11:51.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:51 smithi090 ceph-mon[29180]: pgmap v3856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:52.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6197 sec 2024-04-26T20:11:52.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5792.761148 secs 2024-04-26T20:11:52.617 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5792.761148 secs 2024-04-26T20:11:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5792.761148 secs 2024-04-26T20:11:53.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6198 sec 2024-04-26T20:11:53.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:53 smithi167 ceph-mon[32751]: pgmap v3857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:53.668 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:53 smithi119 ceph-mon[32752]: pgmap v3857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:53.764 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:53 smithi090 ceph-mon[29180]: pgmap v3857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:11:54.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6199 sec 2024-04-26T20:11:55.442 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6200 sec 2024-04-26T20:11:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:55 smithi119 ceph-mon[32752]: pgmap v3858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:11:55.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:55 smithi090 ceph-mon[29180]: pgmap v3858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:11:55.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:55 smithi167 ceph-mon[32751]: pgmap v3858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:11:56.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6201 sec 2024-04-26T20:11:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:56 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5797.063770 secs 2024-04-26T20:11:56.589 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:56 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5797.063770 secs 2024-04-26T20:11:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:56 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5797.063770 secs 2024-04-26T20:11:57.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6202 sec 2024-04-26T20:11:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5797.761274 secs 2024-04-26T20:11:57.597 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:57 smithi119 ceph-mon[32752]: pgmap v3859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:57.606 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5797.761274 secs 2024-04-26T20:11:57.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:57 smithi167 ceph-mon[32751]: pgmap v3859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5797.761274 secs 2024-04-26T20:11:57.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:57 smithi090 ceph-mon[29180]: pgmap v3859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:58.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6203 sec 2024-04-26T20:11:59.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6204 sec 2024-04-26T20:11:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:11:59 smithi119 ceph-mon[32752]: pgmap v3860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:59.604 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:11:59 smithi167 ceph-mon[32751]: pgmap v3860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:11:59.667 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:11:59 smithi090 ceph-mon[29180]: pgmap v3860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:00.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6205 sec 2024-04-26T20:12:01.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6206 sec 2024-04-26T20:12:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:01 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5802.063858 secs 2024-04-26T20:12:01.569 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:01 smithi119 ceph-mon[32752]: pgmap v3861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:01.584 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:01 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5802.063858 secs 2024-04-26T20:12:01.697 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:01 smithi167 ceph-mon[32751]: pgmap v3861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:01.768 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:01 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5802.063858 secs 2024-04-26T20:12:01.801 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:01 smithi090 ceph-mon[29180]: pgmap v3861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:02.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6207 sec 2024-04-26T20:12:02.533 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5802.761415 secs 2024-04-26T20:12:02.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5802.761415 secs 2024-04-26T20:12:02.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5802.761415 secs 2024-04-26T20:12:03.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6208 sec 2024-04-26T20:12:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:03 smithi119 ceph-mon[32752]: pgmap v3862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:03.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:03 smithi167 ceph-mon[32751]: pgmap v3862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:03.653 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:03 smithi090 ceph-mon[29180]: pgmap v3862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:04.443 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6209 sec 2024-04-26T20:12:05.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6210 sec 2024-04-26T20:12:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:05 smithi119 ceph-mon[32752]: pgmap v3863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:05.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:05 smithi167 ceph-mon[32751]: pgmap v3863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:05 smithi090 ceph-mon[29180]: pgmap v3863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:06.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6211 sec 2024-04-26T20:12:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:06 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5807.063921 secs 2024-04-26T20:12:06.623 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:06 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5807.063921 secs 2024-04-26T20:12:06.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:06 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5807.063921 secs 2024-04-26T20:12:07.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6212 sec 2024-04-26T20:12:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5807.761488 secs 2024-04-26T20:12:07.570 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:07 smithi119 ceph-mon[32752]: pgmap v3864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:07.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5807.761488 secs 2024-04-26T20:12:07.590 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:07 smithi167 ceph-mon[32751]: pgmap v3864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5807.761488 secs 2024-04-26T20:12:07.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:07 smithi090 ceph-mon[29180]: pgmap v3864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:08.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6213 sec 2024-04-26T20:12:09.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6214 sec 2024-04-26T20:12:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:09 smithi119 ceph-mon[32752]: pgmap v3865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:09.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:09 smithi090 ceph-mon[29180]: pgmap v3865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:09.591 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:09 smithi167 ceph-mon[32751]: pgmap v3865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:10.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6215 sec 2024-04-26T20:12:11.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6216 sec 2024-04-26T20:12:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:11 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5812.063979 secs 2024-04-26T20:12:11.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:11 smithi119 ceph-mon[32752]: pgmap v3866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:11.617 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:11 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5812.063979 secs 2024-04-26T20:12:11.665 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:11 smithi167 ceph-mon[32751]: pgmap v3866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:11.756 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:11 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5812.063979 secs 2024-04-26T20:12:11.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:11 smithi090 ceph-mon[29180]: pgmap v3866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:12.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6217 sec 2024-04-26T20:12:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5812.761587 secs 2024-04-26T20:12:12.628 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:12 smithi119 ceph-mon[32752]: pgmap v3867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:12.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5812.761587 secs 2024-04-26T20:12:12.717 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:12 smithi090 ceph-mon[29180]: pgmap v3867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:12.778 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5812.761587 secs 2024-04-26T20:12:12.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:12 smithi167 ceph-mon[32751]: pgmap v3867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:13.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6218 sec 2024-04-26T20:12:14.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6219 sec 2024-04-26T20:12:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:14 smithi119 ceph-mon[32752]: pgmap v3868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:15.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:14 smithi167 ceph-mon[32751]: pgmap v3868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:14 smithi090 ceph-mon[29180]: pgmap v3868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:15.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6220 sec 2024-04-26T20:12:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:15 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5817.064064 secs 2024-04-26T20:12:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:15 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5817.064064 secs 2024-04-26T20:12:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:15 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5817.064064 secs 2024-04-26T20:12:16.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6221 sec 2024-04-26T20:12:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5817.761688 secs 2024-04-26T20:12:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:16 smithi119 ceph-mon[32752]: pgmap v3869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5817.761688 secs 2024-04-26T20:12:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:16 smithi167 ceph-mon[32751]: pgmap v3869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5817.761688 secs 2024-04-26T20:12:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:16 smithi090 ceph-mon[29180]: pgmap v3869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:17.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6222 sec 2024-04-26T20:12:18.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6223 sec 2024-04-26T20:12:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:18 smithi119 ceph-mon[32752]: pgmap v3870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:18 smithi167 ceph-mon[32751]: pgmap v3870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:18 smithi090 ceph-mon[29180]: pgmap v3870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:19.444 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6224 sec 2024-04-26T20:12:20.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6225 sec 2024-04-26T20:12:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:20 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5822.064148 secs 2024-04-26T20:12:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:20 smithi119 ceph-mon[32752]: pgmap v3871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:20 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5822.064148 secs 2024-04-26T20:12:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:20 smithi167 ceph-mon[32751]: pgmap v3871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:20 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5822.064148 secs 2024-04-26T20:12:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:20 smithi090 ceph-mon[29180]: pgmap v3871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:21.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6226 sec 2024-04-26T20:12:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5822.761793 secs 2024-04-26T20:12:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5822.761793 secs 2024-04-26T20:12:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5822.761793 secs 2024-04-26T20:12:22.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6227 sec 2024-04-26T20:12:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:23 smithi119 ceph-mon[32752]: pgmap v3872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:23 smithi167 ceph-mon[32751]: pgmap v3872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:23 smithi090 ceph-mon[29180]: pgmap v3872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:23.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6228 sec 2024-04-26T20:12:24.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6229 sec 2024-04-26T20:12:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:25 smithi119 ceph-mon[32752]: pgmap v3873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:25 smithi167 ceph-mon[32751]: pgmap v3873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:25 smithi090 ceph-mon[29180]: pgmap v3873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:25.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6230 sec 2024-04-26T20:12:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:26 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5827.064209 secs 2024-04-26T20:12:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:26 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5827.064209 secs 2024-04-26T20:12:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:26 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5827.064209 secs 2024-04-26T20:12:26.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6231 sec 2024-04-26T20:12:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5827.761858 secs 2024-04-26T20:12:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:27 smithi119 ceph-mon[32752]: pgmap v3874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5827.761858 secs 2024-04-26T20:12:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:27 smithi167 ceph-mon[32751]: pgmap v3874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5827.761858 secs 2024-04-26T20:12:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:27 smithi090 ceph-mon[29180]: pgmap v3874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:27.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6232 sec 2024-04-26T20:12:28.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6233 sec 2024-04-26T20:12:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:29 smithi119 ceph-mon[32752]: pgmap v3875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:29 smithi167 ceph-mon[32751]: pgmap v3875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:29 smithi090 ceph-mon[29180]: pgmap v3875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:29.445 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6234 sec 2024-04-26T20:12:30.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6235 sec 2024-04-26T20:12:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:31 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5832.064268 secs 2024-04-26T20:12:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:31 smithi119 ceph-mon[32752]: pgmap v3876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:31 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5832.064268 secs 2024-04-26T20:12:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:31 smithi167 ceph-mon[32751]: pgmap v3876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:31 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5832.064268 secs 2024-04-26T20:12:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:31 smithi090 ceph-mon[29180]: pgmap v3876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:31.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6236 sec 2024-04-26T20:12:32.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5832.761981 secs 2024-04-26T20:12:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5832.761981 secs 2024-04-26T20:12:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5832.761981 secs 2024-04-26T20:12:32.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6237 sec 2024-04-26T20:12:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:33 smithi119 ceph-mon[32752]: pgmap v3877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:33 smithi167 ceph-mon[32751]: pgmap v3877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:33 smithi090 ceph-mon[29180]: pgmap v3877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:33.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6238 sec 2024-04-26T20:12:34.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6239 sec 2024-04-26T20:12:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:35 smithi119 ceph-mon[32752]: pgmap v3878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:35 smithi167 ceph-mon[32751]: pgmap v3878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:35 smithi090 ceph-mon[29180]: pgmap v3878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:35.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6240 sec 2024-04-26T20:12:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:36 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 5837.064328 secs 2024-04-26T20:12:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:36 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 5837.064328 secs 2024-04-26T20:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:36 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 5837.064328 secs 2024-04-26T20:12:36.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6241 sec 2024-04-26T20:12:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5837.762099 secs 2024-04-26T20:12:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:37 smithi119 ceph-mon[32752]: pgmap v3879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5837.762099 secs 2024-04-26T20:12:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:37 smithi167 ceph-mon[32751]: pgmap v3879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5837.762099 secs 2024-04-26T20:12:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:37 smithi090 ceph-mon[29180]: pgmap v3879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:37.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6242 sec 2024-04-26T20:12:38.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6243 sec 2024-04-26T20:12:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:39 smithi119 ceph-mon[32752]: pgmap v3880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:39 smithi167 ceph-mon[32751]: pgmap v3880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:39 smithi090 ceph-mon[29180]: pgmap v3880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:39.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6244 sec 2024-04-26T20:12:40.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6245 sec 2024-04-26T20:12:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:41 smithi119 ceph-mon[32752]: 12 slow requests, 5 included below; oldest blocked for > 5842.064410 secs 2024-04-26T20:12:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:41 smithi119 ceph-mon[32752]: slow request 182.429118 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:12:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:41 smithi119 ceph-mon[32752]: slow request 5842.064410 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T20:12:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:41 smithi119 ceph-mon[32752]: slow request 5842.061515 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:12:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:41 smithi119 ceph-mon[32752]: slow request 5842.055905 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:12:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:41 smithi119 ceph-mon[32752]: slow request 5840.089179 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T20:12:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:41 smithi119 ceph-mon[32752]: pgmap v3881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:41 smithi167 ceph-mon[32751]: 12 slow requests, 5 included below; oldest blocked for > 5842.064410 secs 2024-04-26T20:12:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:41 smithi167 ceph-mon[32751]: slow request 182.429118 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:12:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:41 smithi167 ceph-mon[32751]: slow request 5842.064410 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T20:12:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:41 smithi167 ceph-mon[32751]: slow request 5842.061515 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:12:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:41 smithi167 ceph-mon[32751]: slow request 5842.055905 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:12:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:41 smithi167 ceph-mon[32751]: slow request 5840.089179 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T20:12:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:41 smithi167 ceph-mon[32751]: pgmap v3881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:41 smithi090 ceph-mon[29180]: 12 slow requests, 5 included below; oldest blocked for > 5842.064410 secs 2024-04-26T20:12:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:41 smithi090 ceph-mon[29180]: slow request 182.429118 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:12:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:41 smithi090 ceph-mon[29180]: slow request 5842.064410 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T20:12:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:41 smithi090 ceph-mon[29180]: slow request 5842.061515 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:12:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:41 smithi090 ceph-mon[29180]: slow request 5842.055905 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T20:12:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:41 smithi090 ceph-mon[29180]: slow request 5840.089179 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T20:12:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:41 smithi090 ceph-mon[29180]: pgmap v3881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:41.446 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6246 sec 2024-04-26T20:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5842.762224 secs 2024-04-26T20:12:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5842.762224 secs 2024-04-26T20:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5842.762224 secs 2024-04-26T20:12:42.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6247 sec 2024-04-26T20:12:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:43 smithi119 ceph-mon[32752]: pgmap v3882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:12:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:12:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:43 smithi167 ceph-mon[32751]: pgmap v3882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:12:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:12:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:43 smithi090 ceph-mon[29180]: pgmap v3882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:12:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:12:43.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6248 sec 2024-04-26T20:12:44.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6249 sec 2024-04-26T20:12:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:45 smithi167 ceph-mon[32751]: pgmap v3883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:12:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:12:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:12:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:12:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:45 smithi090 ceph-mon[29180]: pgmap v3883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:12:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:12:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:12:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:12:45.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6250 sec 2024-04-26T20:12:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:45 smithi119 ceph-mon[32752]: pgmap v3883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:12:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:12:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:12:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:12:45.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:12:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:46 smithi167 ceph-mon[32751]: pgmap v3884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:12:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:46 smithi167 ceph-mon[32751]: pgmap v3885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:46 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 5847.064499 secs 2024-04-26T20:12:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:46 smithi167 ceph-mon[32751]: slow request 5804.059891 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:12:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:46 smithi090 ceph-mon[29180]: pgmap v3884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:12:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:46 smithi090 ceph-mon[29180]: pgmap v3885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:46 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 5847.064499 secs 2024-04-26T20:12:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:46 smithi090 ceph-mon[29180]: slow request 5804.059891 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:12:46.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6251 sec 2024-04-26T20:12:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:46 smithi119 ceph-mon[32752]: pgmap v3884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:12:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:46 smithi119 ceph-mon[32752]: pgmap v3885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:46.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:46 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 5847.064499 secs 2024-04-26T20:12:46.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:46 smithi119 ceph-mon[32752]: slow request 5804.059891 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:12:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5847.762374 secs 2024-04-26T20:12:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5847.762374 secs 2024-04-26T20:12:47.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6252 sec 2024-04-26T20:12:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5847.762374 secs 2024-04-26T20:12:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:48 smithi167 ceph-mon[32751]: pgmap v3886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:48 smithi090 ceph-mon[29180]: pgmap v3886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:48.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6253 sec 2024-04-26T20:12:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:48 smithi119 ceph-mon[32752]: pgmap v3886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:49.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6254 sec 2024-04-26T20:12:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:50 smithi090 ceph-mon[29180]: pgmap v3887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:50.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6255 sec 2024-04-26T20:12:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:50 smithi119 ceph-mon[32752]: pgmap v3887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:50 smithi167 ceph-mon[32751]: pgmap v3887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5852.064582 secs 2024-04-26T20:12:51.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6256 sec 2024-04-26T20:12:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5852.064582 secs 2024-04-26T20:12:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5852.064582 secs 2024-04-26T20:12:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5852.762521 secs 2024-04-26T20:12:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:52 smithi090 ceph-mon[29180]: pgmap v3888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:52.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6257 sec 2024-04-26T20:12:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5852.762521 secs 2024-04-26T20:12:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:52 smithi119 ceph-mon[32752]: pgmap v3888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5852.762521 secs 2024-04-26T20:12:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:52 smithi167 ceph-mon[32751]: pgmap v3888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:12:53.447 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6258 sec 2024-04-26T20:12:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:54 smithi090 ceph-mon[29180]: pgmap v3889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:12:54.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6259 sec 2024-04-26T20:12:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:54 smithi119 ceph-mon[32752]: pgmap v3889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:12:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:54 smithi167 ceph-mon[32751]: pgmap v3889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:12:55.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6260 sec 2024-04-26T20:12:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:56 smithi090 ceph-mon[29180]: pgmap v3890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:12:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5857.064661 secs 2024-04-26T20:12:56.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6261 sec 2024-04-26T20:12:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:56 smithi119 ceph-mon[32752]: pgmap v3890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:12:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5857.064661 secs 2024-04-26T20:12:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:56 smithi167 ceph-mon[32751]: pgmap v3890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:12:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5857.064661 secs 2024-04-26T20:12:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5857.762582 secs 2024-04-26T20:12:57.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6262 sec 2024-04-26T20:12:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5857.762582 secs 2024-04-26T20:12:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5857.762582 secs 2024-04-26T20:12:58.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:12:58 smithi090 ceph-mon[29180]: pgmap v3891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:58.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6263 sec 2024-04-26T20:12:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:12:58 smithi119 ceph-mon[32752]: pgmap v3891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:12:58 smithi167 ceph-mon[32751]: pgmap v3891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:12:59.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6264 sec 2024-04-26T20:13:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:00 smithi090 ceph-mon[29180]: pgmap v3892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:00.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6265 sec 2024-04-26T20:13:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:00 smithi119 ceph-mon[32752]: pgmap v3892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:00 smithi167 ceph-mon[32751]: pgmap v3892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5862.064742 secs 2024-04-26T20:13:01.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6266 sec 2024-04-26T20:13:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5862.064742 secs 2024-04-26T20:13:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5862.064742 secs 2024-04-26T20:13:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5862.762696 secs 2024-04-26T20:13:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:02 smithi090 ceph-mon[29180]: pgmap v3893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:02.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6267 sec 2024-04-26T20:13:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5862.762696 secs 2024-04-26T20:13:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:02 smithi119 ceph-mon[32752]: pgmap v3893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5862.762696 secs 2024-04-26T20:13:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:02 smithi167 ceph-mon[32751]: pgmap v3893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:03.448 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6268 sec 2024-04-26T20:13:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:04 smithi090 ceph-mon[29180]: pgmap v3894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:04.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6269 sec 2024-04-26T20:13:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:04 smithi119 ceph-mon[32752]: pgmap v3894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:04 smithi167 ceph-mon[32751]: pgmap v3894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:05.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6270 sec 2024-04-26T20:13:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:06 smithi090 ceph-mon[29180]: pgmap v3895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5867.064836 secs 2024-04-26T20:13:06.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6271 sec 2024-04-26T20:13:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:06 smithi119 ceph-mon[32752]: pgmap v3895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5867.064836 secs 2024-04-26T20:13:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:06 smithi167 ceph-mon[32751]: pgmap v3895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5867.064836 secs 2024-04-26T20:13:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5867.762817 secs 2024-04-26T20:13:07.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6272 sec 2024-04-26T20:13:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5867.762817 secs 2024-04-26T20:13:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5867.762817 secs 2024-04-26T20:13:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:08 smithi090 ceph-mon[29180]: pgmap v3896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:08.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6273 sec 2024-04-26T20:13:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:08 smithi119 ceph-mon[32752]: pgmap v3896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:08 smithi167 ceph-mon[32751]: pgmap v3896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:09.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6274 sec 2024-04-26T20:13:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:10 smithi090 ceph-mon[29180]: pgmap v3897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:10.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6275 sec 2024-04-26T20:13:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:10 smithi119 ceph-mon[32752]: pgmap v3897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:10 smithi167 ceph-mon[32751]: pgmap v3897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5872.064894 secs 2024-04-26T20:13:11.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6276 sec 2024-04-26T20:13:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5872.064894 secs 2024-04-26T20:13:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5872.064894 secs 2024-04-26T20:13:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5872.762920 secs 2024-04-26T20:13:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:12 smithi090 ceph-mon[29180]: pgmap v3898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:12.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6277 sec 2024-04-26T20:13:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5872.762920 secs 2024-04-26T20:13:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:12 smithi119 ceph-mon[32752]: pgmap v3898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5872.762920 secs 2024-04-26T20:13:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:12 smithi167 ceph-mon[32751]: pgmap v3898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:13.449 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6278 sec 2024-04-26T20:13:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:14 smithi090 ceph-mon[29180]: pgmap v3899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:14.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6279 sec 2024-04-26T20:13:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:14 smithi119 ceph-mon[32752]: pgmap v3899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:14 smithi167 ceph-mon[32751]: pgmap v3899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:15.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6280 sec 2024-04-26T20:13:16.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6281 sec 2024-04-26T20:13:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:16 smithi119 ceph-mon[32752]: pgmap v3900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5877.064979 secs 2024-04-26T20:13:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:16 smithi167 ceph-mon[32751]: pgmap v3900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5877.064979 secs 2024-04-26T20:13:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:16 smithi090 ceph-mon[29180]: pgmap v3900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5877.064979 secs 2024-04-26T20:13:17.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6282 sec 2024-04-26T20:13:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5877.763034 secs 2024-04-26T20:13:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5877.763034 secs 2024-04-26T20:13:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5877.763034 secs 2024-04-26T20:13:18.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6283 sec 2024-04-26T20:13:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:18 smithi119 ceph-mon[32752]: pgmap v3901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:18 smithi167 ceph-mon[32751]: pgmap v3901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:18 smithi090 ceph-mon[29180]: pgmap v3901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:19.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6284 sec 2024-04-26T20:13:20.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6285 sec 2024-04-26T20:13:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:20 smithi119 ceph-mon[32752]: pgmap v3902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:20.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:20 smithi167 ceph-mon[32751]: pgmap v3902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:20 smithi090 ceph-mon[29180]: pgmap v3902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:21.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6286 sec 2024-04-26T20:13:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5882.065038 secs 2024-04-26T20:13:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5882.065038 secs 2024-04-26T20:13:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5882.065038 secs 2024-04-26T20:13:22.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6287 sec 2024-04-26T20:13:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5882.763173 secs 2024-04-26T20:13:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:22 smithi119 ceph-mon[32752]: pgmap v3903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5882.763173 secs 2024-04-26T20:13:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:22 smithi167 ceph-mon[32751]: pgmap v3903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5882.763173 secs 2024-04-26T20:13:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:22 smithi090 ceph-mon[29180]: pgmap v3903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:23.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6288 sec 2024-04-26T20:13:24.450 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6289 sec 2024-04-26T20:13:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:24 smithi119 ceph-mon[32752]: pgmap v3904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:24 smithi167 ceph-mon[32751]: pgmap v3904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:24 smithi090 ceph-mon[29180]: pgmap v3904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:25.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6290 sec 2024-04-26T20:13:26.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6291 sec 2024-04-26T20:13:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:26 smithi119 ceph-mon[32752]: pgmap v3905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5887.065099 secs 2024-04-26T20:13:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:26 smithi167 ceph-mon[32751]: pgmap v3905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5887.065099 secs 2024-04-26T20:13:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:26 smithi090 ceph-mon[29180]: pgmap v3905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5887.065099 secs 2024-04-26T20:13:27.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6292 sec 2024-04-26T20:13:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5887.763296 secs 2024-04-26T20:13:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5887.763296 secs 2024-04-26T20:13:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5887.763296 secs 2024-04-26T20:13:28.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6293 sec 2024-04-26T20:13:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:28 smithi119 ceph-mon[32752]: pgmap v3906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:28 smithi167 ceph-mon[32751]: pgmap v3906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:28 smithi090 ceph-mon[29180]: pgmap v3906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:29.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6294 sec 2024-04-26T20:13:30.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6295 sec 2024-04-26T20:13:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:30 smithi119 ceph-mon[32752]: pgmap v3907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:30 smithi167 ceph-mon[32751]: pgmap v3907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:30 smithi090 ceph-mon[29180]: pgmap v3907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:31.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6296 sec 2024-04-26T20:13:31.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5892.065181 secs 2024-04-26T20:13:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5892.065181 secs 2024-04-26T20:13:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5892.065181 secs 2024-04-26T20:13:32.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6297 sec 2024-04-26T20:13:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5892.763401 secs 2024-04-26T20:13:32.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:32 smithi090 ceph-mon[29180]: pgmap v3908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5892.763401 secs 2024-04-26T20:13:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:32 smithi119 ceph-mon[32752]: pgmap v3908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5892.763401 secs 2024-04-26T20:13:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:32 smithi167 ceph-mon[32751]: pgmap v3908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:33.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6298 sec 2024-04-26T20:13:34.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6299 sec 2024-04-26T20:13:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:34 smithi119 ceph-mon[32752]: pgmap v3909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:34 smithi167 ceph-mon[32751]: pgmap v3909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:34 smithi090 ceph-mon[29180]: pgmap v3909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:35.451 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6300 sec 2024-04-26T20:13:36.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6301 sec 2024-04-26T20:13:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:36 smithi119 ceph-mon[32752]: pgmap v3910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:36 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 5897.065261 secs 2024-04-26T20:13:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:36 smithi119 ceph-mon[32752]: slow request 5761.328613 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T20:13:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:36 smithi167 ceph-mon[32751]: pgmap v3910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:36 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 5897.065261 secs 2024-04-26T20:13:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:36 smithi167 ceph-mon[32751]: slow request 5761.328613 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T20:13:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:36 smithi090 ceph-mon[29180]: pgmap v3910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:36 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 5897.065261 secs 2024-04-26T20:13:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:36 smithi090 ceph-mon[29180]: slow request 5761.328613 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T20:13:37.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6302 sec 2024-04-26T20:13:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5897.763522 secs 2024-04-26T20:13:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5897.763522 secs 2024-04-26T20:13:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5897.763522 secs 2024-04-26T20:13:38.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6303 sec 2024-04-26T20:13:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:38 smithi119 ceph-mon[32752]: pgmap v3911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:38 smithi167 ceph-mon[32751]: pgmap v3911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:38 smithi090 ceph-mon[29180]: pgmap v3911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:39.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6304 sec 2024-04-26T20:13:40.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6305 sec 2024-04-26T20:13:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:40 smithi119 ceph-mon[32752]: pgmap v3912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:40 smithi167 ceph-mon[32751]: pgmap v3912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:40 smithi090 ceph-mon[29180]: pgmap v3912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:41.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6306 sec 2024-04-26T20:13:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5902.065344 secs 2024-04-26T20:13:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5902.065344 secs 2024-04-26T20:13:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5902.065344 secs 2024-04-26T20:13:42.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6307 sec 2024-04-26T20:13:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5902.763587 secs 2024-04-26T20:13:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:42 smithi119 ceph-mon[32752]: pgmap v3913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5902.763587 secs 2024-04-26T20:13:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:42 smithi167 ceph-mon[32751]: pgmap v3913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5902.763587 secs 2024-04-26T20:13:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:42 smithi090 ceph-mon[29180]: pgmap v3913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:43.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6308 sec 2024-04-26T20:13:44.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6309 sec 2024-04-26T20:13:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:44 smithi119 ceph-mon[32752]: pgmap v3914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:44 smithi167 ceph-mon[32751]: pgmap v3914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:44 smithi090 ceph-mon[29180]: pgmap v3914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:45.452 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6310 sec 2024-04-26T20:13:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:13:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:13:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:13:46.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6311 sec 2024-04-26T20:13:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:46 smithi119 ceph-mon[32752]: pgmap v3915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5907.065405 secs 2024-04-26T20:13:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:13:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:13:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:13:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:46 smithi167 ceph-mon[32751]: pgmap v3915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5907.065405 secs 2024-04-26T20:13:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:13:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:13:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:13:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:46 smithi090 ceph-mon[29180]: pgmap v3915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:13:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5907.065405 secs 2024-04-26T20:13:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:13:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:13:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:13:47.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6312 sec 2024-04-26T20:13:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5907.763673 secs 2024-04-26T20:13:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:47 smithi119 ceph-mon[32752]: pgmap v3916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:13:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:47 smithi119 ceph-mon[32752]: pgmap v3917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5907.763673 secs 2024-04-26T20:13:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:47 smithi167 ceph-mon[32751]: pgmap v3916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:13:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:47 smithi167 ceph-mon[32751]: pgmap v3917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5907.763673 secs 2024-04-26T20:13:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:47 smithi090 ceph-mon[29180]: pgmap v3916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:13:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:47 smithi090 ceph-mon[29180]: pgmap v3917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:48.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6313 sec 2024-04-26T20:13:49.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6314 sec 2024-04-26T20:13:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:49 smithi119 ceph-mon[32752]: pgmap v3918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:13:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:49 smithi167 ceph-mon[32751]: pgmap v3918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:13:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:49 smithi090 ceph-mon[29180]: pgmap v3918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:13:50.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6315 sec 2024-04-26T20:13:51.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6316 sec 2024-04-26T20:13:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:51 smithi119 ceph-mon[32752]: pgmap v3919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5912.065461 secs 2024-04-26T20:13:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:51 smithi167 ceph-mon[32751]: pgmap v3919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5912.065461 secs 2024-04-26T20:13:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:51 smithi090 ceph-mon[29180]: pgmap v3919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5912.065461 secs 2024-04-26T20:13:52.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6317 sec 2024-04-26T20:13:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5912.763807 secs 2024-04-26T20:13:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5912.763807 secs 2024-04-26T20:13:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5912.763807 secs 2024-04-26T20:13:53.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6318 sec 2024-04-26T20:13:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:53 smithi119 ceph-mon[32752]: pgmap v3920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:53 smithi167 ceph-mon[32751]: pgmap v3920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:53 smithi090 ceph-mon[29180]: pgmap v3920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:54.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6319 sec 2024-04-26T20:13:55.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6320 sec 2024-04-26T20:13:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:55 smithi119 ceph-mon[32752]: pgmap v3921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:55 smithi167 ceph-mon[32751]: pgmap v3921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:55 smithi090 ceph-mon[29180]: pgmap v3921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:13:56.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6321 sec 2024-04-26T20:13:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5917.065521 secs 2024-04-26T20:13:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5917.065521 secs 2024-04-26T20:13:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5917.065521 secs 2024-04-26T20:13:57.453 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6322 sec 2024-04-26T20:13:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5917.763944 secs 2024-04-26T20:13:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:57 smithi119 ceph-mon[32752]: pgmap v3922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:13:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5917.763944 secs 2024-04-26T20:13:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:57 smithi167 ceph-mon[32751]: pgmap v3922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:13:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5917.763944 secs 2024-04-26T20:13:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:57 smithi090 ceph-mon[29180]: pgmap v3922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:13:58.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6323 sec 2024-04-26T20:13:59.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6324 sec 2024-04-26T20:13:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:13:59 smithi119 ceph-mon[32752]: pgmap v3923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:13:59 smithi167 ceph-mon[32751]: pgmap v3923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:13:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:13:59 smithi090 ceph-mon[29180]: pgmap v3923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:00.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6325 sec 2024-04-26T20:14:01.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6326 sec 2024-04-26T20:14:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:01 smithi119 ceph-mon[32752]: pgmap v3924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5922.065578 secs 2024-04-26T20:14:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:01 smithi167 ceph-mon[32751]: pgmap v3924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5922.065578 secs 2024-04-26T20:14:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:01 smithi090 ceph-mon[29180]: pgmap v3924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5922.065578 secs 2024-04-26T20:14:02.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6327 sec 2024-04-26T20:14:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5922.764068 secs 2024-04-26T20:14:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5922.764068 secs 2024-04-26T20:14:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5922.764068 secs 2024-04-26T20:14:03.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6328 sec 2024-04-26T20:14:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:03 smithi119 ceph-mon[32752]: pgmap v3925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:03 smithi167 ceph-mon[32751]: pgmap v3925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:03 smithi090 ceph-mon[29180]: pgmap v3925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:04.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6329 sec 2024-04-26T20:14:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:04 smithi119 ceph-mon[32752]: pgmap v3926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:04 smithi167 ceph-mon[32751]: pgmap v3926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:04 smithi090 ceph-mon[29180]: pgmap v3926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:05.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6330 sec 2024-04-26T20:14:06.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6331 sec 2024-04-26T20:14:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:06 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 5927.065662 secs 2024-04-26T20:14:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:06 smithi090 ceph-mon[29180]: slow request 1442.847104 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:14:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5927.764195 secs 2024-04-26T20:14:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:06 smithi090 ceph-mon[29180]: pgmap v3927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:06 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 5927.065662 secs 2024-04-26T20:14:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:06 smithi119 ceph-mon[32752]: slow request 1442.847104 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:14:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5927.764195 secs 2024-04-26T20:14:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:06 smithi119 ceph-mon[32752]: pgmap v3927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:06 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 5927.065662 secs 2024-04-26T20:14:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:06 smithi167 ceph-mon[32751]: slow request 1442.847104 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:14:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5927.764195 secs 2024-04-26T20:14:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:06 smithi167 ceph-mon[32751]: pgmap v3927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:07.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6332 sec 2024-04-26T20:14:08.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6333 sec 2024-04-26T20:14:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:08 smithi090 ceph-mon[29180]: pgmap v3928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:08 smithi119 ceph-mon[32752]: pgmap v3928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:08 smithi167 ceph-mon[32751]: pgmap v3928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:09.454 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6334 sec 2024-04-26T20:14:10.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6335 sec 2024-04-26T20:14:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:10 smithi090 ceph-mon[29180]: pgmap v3929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:10 smithi119 ceph-mon[32752]: pgmap v3929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:10 smithi167 ceph-mon[32751]: pgmap v3929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:11.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6336 sec 2024-04-26T20:14:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5932.065747 secs 2024-04-26T20:14:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5932.764323 secs 2024-04-26T20:14:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5932.065747 secs 2024-04-26T20:14:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5932.764323 secs 2024-04-26T20:14:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5932.065747 secs 2024-04-26T20:14:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5932.764323 secs 2024-04-26T20:14:12.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6337 sec 2024-04-26T20:14:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:12 smithi090 ceph-mon[29180]: pgmap v3930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:12 smithi119 ceph-mon[32752]: pgmap v3930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:12 smithi167 ceph-mon[32751]: pgmap v3930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:13.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6338 sec 2024-04-26T20:14:14.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6339 sec 2024-04-26T20:14:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:14 smithi090 ceph-mon[29180]: pgmap v3931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:14 smithi119 ceph-mon[32752]: pgmap v3931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:14 smithi167 ceph-mon[32751]: pgmap v3931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:15.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6340 sec 2024-04-26T20:14:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:15 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5937.065825 secs 2024-04-26T20:14:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:15 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5937.065825 secs 2024-04-26T20:14:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:15 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5937.065825 secs 2024-04-26T20:14:16.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6341 sec 2024-04-26T20:14:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5937.764447 secs 2024-04-26T20:14:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:16 smithi090 ceph-mon[29180]: pgmap v3932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:16.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5937.764447 secs 2024-04-26T20:14:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:16 smithi119 ceph-mon[32752]: pgmap v3932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5937.764447 secs 2024-04-26T20:14:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:16 smithi167 ceph-mon[32751]: pgmap v3932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:17.455 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6342 sec 2024-04-26T20:14:18.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6343 sec 2024-04-26T20:14:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:18 smithi090 ceph-mon[29180]: pgmap v3933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:18 smithi119 ceph-mon[32752]: pgmap v3933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:18 smithi167 ceph-mon[32751]: pgmap v3933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:19.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6344 sec 2024-04-26T20:14:20.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6345 sec 2024-04-26T20:14:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:21 smithi119 ceph-mon[32752]: pgmap v3934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5942.065884 secs 2024-04-26T20:14:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:21 smithi167 ceph-mon[32751]: pgmap v3934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5942.065884 secs 2024-04-26T20:14:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:21 smithi090 ceph-mon[29180]: pgmap v3934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5942.065884 secs 2024-04-26T20:14:21.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6346 sec 2024-04-26T20:14:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5942.764567 secs 2024-04-26T20:14:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5942.764567 secs 2024-04-26T20:14:22.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6347 sec 2024-04-26T20:14:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5942.764567 secs 2024-04-26T20:14:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:23 smithi167 ceph-mon[32751]: pgmap v3935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:23 smithi090 ceph-mon[29180]: pgmap v3935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:23.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6348 sec 2024-04-26T20:14:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:23 smithi119 ceph-mon[32752]: pgmap v3935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:24.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6349 sec 2024-04-26T20:14:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:25 smithi167 ceph-mon[32751]: pgmap v3936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:25 smithi090 ceph-mon[29180]: pgmap v3936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:25.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6350 sec 2024-04-26T20:14:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:25 smithi119 ceph-mon[32752]: pgmap v3936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5947.065945 secs 2024-04-26T20:14:26.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6351 sec 2024-04-26T20:14:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5947.065945 secs 2024-04-26T20:14:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5947.065945 secs 2024-04-26T20:14:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5947.764714 secs 2024-04-26T20:14:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:27 smithi090 ceph-mon[29180]: pgmap v3937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:27.456 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6352 sec 2024-04-26T20:14:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5947.764714 secs 2024-04-26T20:14:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:27 smithi119 ceph-mon[32752]: pgmap v3937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5947.764714 secs 2024-04-26T20:14:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:27 smithi167 ceph-mon[32751]: pgmap v3937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:28.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6353 sec 2024-04-26T20:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 49 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 49 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:14:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T20:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 50 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:14:29.253 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:14:29.251+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 50 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T20:14:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:29 smithi090 ceph-mon[29180]: pgmap v3938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:29.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6354 sec 2024-04-26T20:14:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:29 smithi119 ceph-mon[32752]: pgmap v3938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:29 smithi167 ceph-mon[32751]: pgmap v3938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:30.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6355 sec 2024-04-26T20:14:31.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:31 smithi167 ceph-mon[32751]: pgmap v3939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:31.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5952.066004 secs 2024-04-26T20:14:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:31 smithi090 ceph-mon[29180]: pgmap v3939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5952.066004 secs 2024-04-26T20:14:31.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6356 sec 2024-04-26T20:14:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:31 smithi119 ceph-mon[32752]: pgmap v3939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5952.066004 secs 2024-04-26T20:14:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5952.764787 secs 2024-04-26T20:14:32.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6357 sec 2024-04-26T20:14:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5952.764787 secs 2024-04-26T20:14:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5952.764787 secs 2024-04-26T20:14:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:33 smithi090 ceph-mon[29180]: pgmap v3940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:33.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6358 sec 2024-04-26T20:14:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:33 smithi119 ceph-mon[32752]: pgmap v3940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:33 smithi167 ceph-mon[32751]: pgmap v3940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:34.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6359 sec 2024-04-26T20:14:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:35 smithi090 ceph-mon[29180]: pgmap v3941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:35.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6360 sec 2024-04-26T20:14:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:35 smithi119 ceph-mon[32752]: pgmap v3941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:35 smithi167 ceph-mon[32751]: pgmap v3941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5957.066090 secs 2024-04-26T20:14:36.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6361 sec 2024-04-26T20:14:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5957.066090 secs 2024-04-26T20:14:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5957.066090 secs 2024-04-26T20:14:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5957.764908 secs 2024-04-26T20:14:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:37 smithi090 ceph-mon[29180]: pgmap v3942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:37.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6362 sec 2024-04-26T20:14:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5957.764908 secs 2024-04-26T20:14:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:37 smithi119 ceph-mon[32752]: pgmap v3942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5957.764908 secs 2024-04-26T20:14:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:37 smithi167 ceph-mon[32751]: pgmap v3942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:38.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6363 sec 2024-04-26T20:14:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:39 smithi090 ceph-mon[29180]: pgmap v3943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:39.457 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6364 sec 2024-04-26T20:14:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:39 smithi119 ceph-mon[32752]: pgmap v3943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:39 smithi167 ceph-mon[32751]: pgmap v3943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:40.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6365 sec 2024-04-26T20:14:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:41 smithi090 ceph-mon[29180]: pgmap v3944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5962.066173 secs 2024-04-26T20:14:41.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6366 sec 2024-04-26T20:14:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:41 smithi119 ceph-mon[32752]: pgmap v3944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5962.066173 secs 2024-04-26T20:14:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:41 smithi167 ceph-mon[32751]: pgmap v3944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5962.066173 secs 2024-04-26T20:14:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5962.765003 secs 2024-04-26T20:14:42.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6367 sec 2024-04-26T20:14:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5962.765003 secs 2024-04-26T20:14:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5962.765003 secs 2024-04-26T20:14:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:43 smithi090 ceph-mon[29180]: pgmap v3945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:43.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6368 sec 2024-04-26T20:14:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:43 smithi119 ceph-mon[32752]: pgmap v3945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:43 smithi167 ceph-mon[32751]: pgmap v3945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:44.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6369 sec 2024-04-26T20:14:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:45 smithi090 ceph-mon[29180]: pgmap v3946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:45.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6370 sec 2024-04-26T20:14:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:45 smithi119 ceph-mon[32752]: pgmap v3946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:45 smithi167 ceph-mon[32751]: pgmap v3946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:14:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5967.066258 secs 2024-04-26T20:14:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:14:46.448 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5967.066258 secs 2024-04-26T20:14:46.449 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:14:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5967.066258 secs 2024-04-26T20:14:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:14:46.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6371 sec 2024-04-26T20:14:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5967.765127 secs 2024-04-26T20:14:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:47 smithi119 ceph-mon[32752]: pgmap v3947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5967.765127 secs 2024-04-26T20:14:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:47 smithi090 ceph-mon[29180]: pgmap v3947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6372 sec 2024-04-26T20:14:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5967.765127 secs 2024-04-26T20:14:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:47 smithi167 ceph-mon[32751]: pgmap v3947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:48.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6373 sec 2024-04-26T20:14:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:48 smithi090 ceph-mon[29180]: pgmap v3948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:48 smithi119 ceph-mon[32752]: pgmap v3948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:48 smithi167 ceph-mon[32751]: pgmap v3948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:49.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6374 sec 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:14:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:14:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:14:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:14:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:14:50.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:14:50.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:14:50.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:14:50.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6375 sec 2024-04-26T20:14:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:50 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:14:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:50 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:14:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:50 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:14:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:50 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:14:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:50 smithi119 ceph-mon[32752]: pgmap v3949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:14:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:50 smithi119 ceph-mon[32752]: pgmap v3950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:50 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5972.066347 secs 2024-04-26T20:14:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:50 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:14:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:50 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:14:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:50 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:14:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:50 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:14:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:50 smithi167 ceph-mon[32751]: pgmap v3949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:14:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:50 smithi167 ceph-mon[32751]: pgmap v3950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:50 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5972.066347 secs 2024-04-26T20:14:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:50 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:14:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:50 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:14:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:50 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:14:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:50 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:14:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:50 smithi090 ceph-mon[29180]: pgmap v3949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:14:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:50 smithi090 ceph-mon[29180]: pgmap v3950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:50 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5972.066347 secs 2024-04-26T20:14:51.458 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6376 sec 2024-04-26T20:14:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5972.765249 secs 2024-04-26T20:14:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5972.765249 secs 2024-04-26T20:14:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5972.765249 secs 2024-04-26T20:14:52.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6377 sec 2024-04-26T20:14:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:52 smithi119 ceph-mon[32752]: pgmap v3951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:52 smithi167 ceph-mon[32751]: pgmap v3951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:52 smithi090 ceph-mon[29180]: pgmap v3951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:53.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6378 sec 2024-04-26T20:14:54.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6379 sec 2024-04-26T20:14:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:54 smithi119 ceph-mon[32752]: pgmap v3952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:54 smithi167 ceph-mon[32751]: pgmap v3952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:54 smithi090 ceph-mon[29180]: pgmap v3952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:55.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6380 sec 2024-04-26T20:14:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5977.066407 secs 2024-04-26T20:14:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5977.066407 secs 2024-04-26T20:14:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5977.066407 secs 2024-04-26T20:14:56.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6381 sec 2024-04-26T20:14:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:56 smithi119 ceph-mon[32752]: pgmap v3953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5977.765371 secs 2024-04-26T20:14:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:56 smithi167 ceph-mon[32751]: pgmap v3953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5977.765371 secs 2024-04-26T20:14:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:56 smithi090 ceph-mon[29180]: pgmap v3953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5977.765371 secs 2024-04-26T20:14:57.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6382 sec 2024-04-26T20:14:58.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6383 sec 2024-04-26T20:14:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:14:58 smithi119 ceph-mon[32752]: pgmap v3954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:14:58 smithi167 ceph-mon[32751]: pgmap v3954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:14:58 smithi090 ceph-mon[29180]: pgmap v3954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:14:59.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6384 sec 2024-04-26T20:15:00.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6385 sec 2024-04-26T20:15:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:00 smithi119 ceph-mon[32752]: pgmap v3955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:15:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:00 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5982.066490 secs 2024-04-26T20:15:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:00 smithi167 ceph-mon[32751]: pgmap v3955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:15:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:00 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5982.066490 secs 2024-04-26T20:15:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:00 smithi090 ceph-mon[29180]: pgmap v3955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:15:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:00 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5982.066490 secs 2024-04-26T20:15:01.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6386 sec 2024-04-26T20:15:02.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5982.765492 secs 2024-04-26T20:15:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5982.765492 secs 2024-04-26T20:15:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5982.765492 secs 2024-04-26T20:15:02.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6387 sec 2024-04-26T20:15:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:02 smithi119 ceph-mon[32752]: pgmap v3956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:02 smithi167 ceph-mon[32751]: pgmap v3956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:02 smithi090 ceph-mon[29180]: pgmap v3956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:03.459 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6388 sec 2024-04-26T20:15:04.460 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6389 sec 2024-04-26T20:15:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:04 smithi119 ceph-mon[32752]: pgmap v3957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:04 smithi167 ceph-mon[32751]: pgmap v3957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:04 smithi090 ceph-mon[29180]: pgmap v3957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:05.460 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6390 sec 2024-04-26T20:15:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:05 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5987.066563 secs 2024-04-26T20:15:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:05 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5987.066563 secs 2024-04-26T20:15:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:05 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5987.066563 secs 2024-04-26T20:15:06.460 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6391 sec 2024-04-26T20:15:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:06 smithi119 ceph-mon[32752]: pgmap v3958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5987.765615 secs 2024-04-26T20:15:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:06 smithi167 ceph-mon[32751]: pgmap v3958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5987.765615 secs 2024-04-26T20:15:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:06 smithi090 ceph-mon[29180]: pgmap v3958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5987.765615 secs 2024-04-26T20:15:07.460 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6392 sec 2024-04-26T20:15:08.460 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6393 sec 2024-04-26T20:15:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:08 smithi119 ceph-mon[32752]: pgmap v3959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:08 smithi167 ceph-mon[32751]: pgmap v3959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:08 smithi090 ceph-mon[29180]: pgmap v3959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:09.460 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6394 sec 2024-04-26T20:15:10.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6395 sec 2024-04-26T20:15:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:10 smithi119 ceph-mon[32752]: pgmap v3960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:10 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5992.066643 secs 2024-04-26T20:15:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:10 smithi167 ceph-mon[32751]: pgmap v3960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:10 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5992.066643 secs 2024-04-26T20:15:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:10 smithi090 ceph-mon[29180]: pgmap v3960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:10 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5992.066643 secs 2024-04-26T20:15:11.460 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6396 sec 2024-04-26T20:15:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5992.765767 secs 2024-04-26T20:15:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5992.765767 secs 2024-04-26T20:15:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5992.765767 secs 2024-04-26T20:15:12.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6397 sec 2024-04-26T20:15:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:12 smithi119 ceph-mon[32752]: pgmap v3961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:12 smithi167 ceph-mon[32751]: pgmap v3961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:12 smithi090 ceph-mon[29180]: pgmap v3961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:13.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6398 sec 2024-04-26T20:15:14.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6399 sec 2024-04-26T20:15:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:14 smithi119 ceph-mon[32752]: pgmap v3962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:14 smithi167 ceph-mon[32751]: pgmap v3962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:14 smithi090 ceph-mon[29180]: pgmap v3962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:15.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6400 sec 2024-04-26T20:15:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:15 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 5997.066724 secs 2024-04-26T20:15:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:15 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 5997.066724 secs 2024-04-26T20:15:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:15 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 5997.066724 secs 2024-04-26T20:15:16.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6401 sec 2024-04-26T20:15:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:16 smithi119 ceph-mon[32752]: pgmap v3963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 5997.765847 secs 2024-04-26T20:15:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:16 smithi167 ceph-mon[32751]: pgmap v3963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 5997.765847 secs 2024-04-26T20:15:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:16 smithi090 ceph-mon[29180]: pgmap v3963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 5997.765847 secs 2024-04-26T20:15:17.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6402 sec 2024-04-26T20:15:18.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6403 sec 2024-04-26T20:15:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:18 smithi119 ceph-mon[32752]: pgmap v3964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:18 smithi167 ceph-mon[32751]: pgmap v3964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:18 smithi090 ceph-mon[29180]: pgmap v3964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:19.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6404 sec 2024-04-26T20:15:20.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6405 sec 2024-04-26T20:15:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:20 smithi119 ceph-mon[32752]: pgmap v3965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:20 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6002.066805 secs 2024-04-26T20:15:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:20 smithi167 ceph-mon[32751]: pgmap v3965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:20 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6002.066805 secs 2024-04-26T20:15:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:20 smithi090 ceph-mon[29180]: pgmap v3965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:20 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6002.066805 secs 2024-04-26T20:15:21.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6406 sec 2024-04-26T20:15:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6002.765974 secs 2024-04-26T20:15:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6002.765974 secs 2024-04-26T20:15:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6002.765974 secs 2024-04-26T20:15:22.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6407 sec 2024-04-26T20:15:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:22 smithi119 ceph-mon[32752]: pgmap v3966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:22 smithi167 ceph-mon[32751]: pgmap v3966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:22 smithi090 ceph-mon[29180]: pgmap v3966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:23.461 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6408 sec 2024-04-26T20:15:24.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6409 sec 2024-04-26T20:15:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:24 smithi119 ceph-mon[32752]: pgmap v3967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:24 smithi167 ceph-mon[32751]: pgmap v3967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:24 smithi090 ceph-mon[29180]: pgmap v3967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:25.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6410 sec 2024-04-26T20:15:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:25 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6007.066892 secs 2024-04-26T20:15:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:25 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6007.066892 secs 2024-04-26T20:15:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:25 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6007.066892 secs 2024-04-26T20:15:26.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6411 sec 2024-04-26T20:15:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:26 smithi119 ceph-mon[32752]: pgmap v3968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6007.766074 secs 2024-04-26T20:15:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:26 smithi167 ceph-mon[32751]: pgmap v3968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6007.766074 secs 2024-04-26T20:15:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:26 smithi090 ceph-mon[29180]: pgmap v3968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6007.766074 secs 2024-04-26T20:15:27.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6412 sec 2024-04-26T20:15:28.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6413 sec 2024-04-26T20:15:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:29 smithi119 ceph-mon[32752]: pgmap v3969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:29 smithi167 ceph-mon[32751]: pgmap v3969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:29 smithi090 ceph-mon[29180]: pgmap v3969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:29.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6414 sec 2024-04-26T20:15:30.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6415 sec 2024-04-26T20:15:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:31 smithi119 ceph-mon[32752]: pgmap v3970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6012.066974 secs 2024-04-26T20:15:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:31 smithi167 ceph-mon[32751]: pgmap v3970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6012.066974 secs 2024-04-26T20:15:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:31 smithi090 ceph-mon[29180]: pgmap v3970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6012.066974 secs 2024-04-26T20:15:31.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6416 sec 2024-04-26T20:15:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6012.766191 secs 2024-04-26T20:15:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6012.766191 secs 2024-04-26T20:15:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6012.766191 secs 2024-04-26T20:15:32.462 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6417 sec 2024-04-26T20:15:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:33 smithi119 ceph-mon[32752]: pgmap v3971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:33 smithi167 ceph-mon[32751]: pgmap v3971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:33 smithi090 ceph-mon[29180]: pgmap v3971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:33.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6418 sec 2024-04-26T20:15:34.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6419 sec 2024-04-26T20:15:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:35 smithi119 ceph-mon[32752]: pgmap v3972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:35 smithi167 ceph-mon[32751]: pgmap v3972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:35 smithi090 ceph-mon[29180]: pgmap v3972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:35.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6420 sec 2024-04-26T20:15:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6017.067061 secs 2024-04-26T20:15:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6017.067061 secs 2024-04-26T20:15:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6017.067061 secs 2024-04-26T20:15:36.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6421 sec 2024-04-26T20:15:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:37 smithi119 ceph-mon[32752]: pgmap v3973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6017.766331 secs 2024-04-26T20:15:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:37 smithi167 ceph-mon[32751]: pgmap v3973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6017.766331 secs 2024-04-26T20:15:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:37 smithi090 ceph-mon[29180]: pgmap v3973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6017.766331 secs 2024-04-26T20:15:37.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6422 sec 2024-04-26T20:15:38.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6423 sec 2024-04-26T20:15:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:39 smithi119 ceph-mon[32752]: pgmap v3974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:39 smithi167 ceph-mon[32751]: pgmap v3974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:39 smithi090 ceph-mon[29180]: pgmap v3974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:39.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6424 sec 2024-04-26T20:15:40.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6425 sec 2024-04-26T20:15:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:41 smithi119 ceph-mon[32752]: pgmap v3975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:41 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 6022.067141 secs 2024-04-26T20:15:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:41 smithi119 ceph-mon[32752]: slow request 362.431850 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:15:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:41 smithi167 ceph-mon[32751]: pgmap v3975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:41 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 6022.067141 secs 2024-04-26T20:15:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:41 smithi167 ceph-mon[32751]: slow request 362.431850 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:15:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:41 smithi090 ceph-mon[29180]: pgmap v3975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:41 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 6022.067141 secs 2024-04-26T20:15:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:41 smithi090 ceph-mon[29180]: slow request 362.431850 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:15:41.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6426 sec 2024-04-26T20:15:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6022.766426 secs 2024-04-26T20:15:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6022.766426 secs 2024-04-26T20:15:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6022.766426 secs 2024-04-26T20:15:42.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6427 sec 2024-04-26T20:15:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:43 smithi119 ceph-mon[32752]: pgmap v3976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:43 smithi167 ceph-mon[32751]: pgmap v3976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:43 smithi090 ceph-mon[29180]: pgmap v3976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:43.463 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6428 sec 2024-04-26T20:15:44.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6429 sec 2024-04-26T20:15:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:45 smithi119 ceph-mon[32752]: pgmap v3977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:45 smithi167 ceph-mon[32751]: pgmap v3977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:45 smithi090 ceph-mon[29180]: pgmap v3977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:45.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6430 sec 2024-04-26T20:15:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6027.067230 secs 2024-04-26T20:15:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6027.067230 secs 2024-04-26T20:15:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6027.067230 secs 2024-04-26T20:15:46.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6431 sec 2024-04-26T20:15:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6027.766547 secs 2024-04-26T20:15:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:47 smithi119 ceph-mon[32752]: pgmap v3978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6027.766547 secs 2024-04-26T20:15:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:47 smithi167 ceph-mon[32751]: pgmap v3978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6027.766547 secs 2024-04-26T20:15:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:47 smithi090 ceph-mon[29180]: pgmap v3978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:47.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6432 sec 2024-04-26T20:15:48.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6433 sec 2024-04-26T20:15:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:49 smithi119 ceph-mon[32752]: pgmap v3979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:49 smithi167 ceph-mon[32751]: pgmap v3979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:49 smithi090 ceph-mon[29180]: pgmap v3979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:15:49.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6434 sec 2024-04-26T20:15:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:15:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:15:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:15:50.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6435 sec 2024-04-26T20:15:51.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6436 sec 2024-04-26T20:15:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:51 smithi119 ceph-mon[32752]: pgmap v3980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6032.067317 secs 2024-04-26T20:15:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:15:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:15:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:51 smithi167 ceph-mon[32751]: pgmap v3980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6032.067317 secs 2024-04-26T20:15:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:15:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:15:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:51 smithi090 ceph-mon[29180]: pgmap v3980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:15:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6032.067317 secs 2024-04-26T20:15:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:15:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:15:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:15:52.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6437 sec 2024-04-26T20:15:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6032.766630 secs 2024-04-26T20:15:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:52 smithi119 ceph-mon[32752]: pgmap v3981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T20:15:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:52 smithi119 ceph-mon[32752]: pgmap v3982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6032.766630 secs 2024-04-26T20:15:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:52 smithi167 ceph-mon[32751]: pgmap v3981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T20:15:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:52 smithi167 ceph-mon[32751]: pgmap v3982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6032.766630 secs 2024-04-26T20:15:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:52 smithi090 ceph-mon[29180]: pgmap v3981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T20:15:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:52 smithi090 ceph-mon[29180]: pgmap v3982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:53.464 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6438 sec 2024-04-26T20:15:54.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6439 sec 2024-04-26T20:15:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:54 smithi119 ceph-mon[32752]: pgmap v3983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:15:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:54 smithi167 ceph-mon[32751]: pgmap v3983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:15:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:54 smithi090 ceph-mon[29180]: pgmap v3983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:15:55.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6440 sec 2024-04-26T20:15:56.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6441 sec 2024-04-26T20:15:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:56 smithi119 ceph-mon[32752]: pgmap v3984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6037.067397 secs 2024-04-26T20:15:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:56 smithi167 ceph-mon[32751]: pgmap v3984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6037.067397 secs 2024-04-26T20:15:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:56 smithi090 ceph-mon[29180]: pgmap v3984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6037.067397 secs 2024-04-26T20:15:57.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6442 sec 2024-04-26T20:15:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6037.766707 secs 2024-04-26T20:15:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6037.766707 secs 2024-04-26T20:15:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6037.766707 secs 2024-04-26T20:15:58.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6443 sec 2024-04-26T20:15:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:15:58 smithi119 ceph-mon[32752]: pgmap v3985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:15:58 smithi167 ceph-mon[32751]: pgmap v3985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:15:58 smithi090 ceph-mon[29180]: pgmap v3985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:15:59.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6444 sec 2024-04-26T20:16:00.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6445 sec 2024-04-26T20:16:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:00 smithi119 ceph-mon[32752]: pgmap v3986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:00 smithi167 ceph-mon[32751]: pgmap v3986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:00 smithi090 ceph-mon[29180]: pgmap v3986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:01.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6446 sec 2024-04-26T20:16:01.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6042.067456 secs 2024-04-26T20:16:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6042.067456 secs 2024-04-26T20:16:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6042.067456 secs 2024-04-26T20:16:02.465 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6447 sec 2024-04-26T20:16:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6042.766806 secs 2024-04-26T20:16:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:02 smithi090 ceph-mon[29180]: pgmap v3987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:16:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6042.766806 secs 2024-04-26T20:16:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:02 smithi119 ceph-mon[32752]: pgmap v3987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:16:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6042.766806 secs 2024-04-26T20:16:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:02 smithi167 ceph-mon[32751]: pgmap v3987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:16:03.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6448 sec 2024-04-26T20:16:04.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6449 sec 2024-04-26T20:16:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:04 smithi119 ceph-mon[32752]: pgmap v3988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:04 smithi167 ceph-mon[32751]: pgmap v3988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:04 smithi090 ceph-mon[29180]: pgmap v3988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:05.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6450 sec 2024-04-26T20:16:06.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6451 sec 2024-04-26T20:16:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:06 smithi119 ceph-mon[32752]: pgmap v3989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6047.067537 secs 2024-04-26T20:16:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:06 smithi167 ceph-mon[32751]: pgmap v3989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6047.067537 secs 2024-04-26T20:16:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:06 smithi090 ceph-mon[29180]: pgmap v3989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6047.067537 secs 2024-04-26T20:16:07.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6452 sec 2024-04-26T20:16:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6047.766944 secs 2024-04-26T20:16:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6047.766944 secs 2024-04-26T20:16:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6047.766944 secs 2024-04-26T20:16:08.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6453 sec 2024-04-26T20:16:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:08 smithi119 ceph-mon[32752]: pgmap v3990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:08 smithi167 ceph-mon[32751]: pgmap v3990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:08 smithi090 ceph-mon[29180]: pgmap v3990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:09.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6454 sec 2024-04-26T20:16:10.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6455 sec 2024-04-26T20:16:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:10 smithi119 ceph-mon[32752]: pgmap v3991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:10 smithi167 ceph-mon[32751]: pgmap v3991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:10 smithi090 ceph-mon[29180]: pgmap v3991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:11.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6456 sec 2024-04-26T20:16:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6052.067619 secs 2024-04-26T20:16:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6052.067619 secs 2024-04-26T20:16:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6052.067619 secs 2024-04-26T20:16:12.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6457 sec 2024-04-26T20:16:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6052.767064 secs 2024-04-26T20:16:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:12 smithi119 ceph-mon[32752]: pgmap v3992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6052.767064 secs 2024-04-26T20:16:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:12 smithi167 ceph-mon[32751]: pgmap v3992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6052.767064 secs 2024-04-26T20:16:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:12 smithi090 ceph-mon[29180]: pgmap v3992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:13.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6458 sec 2024-04-26T20:16:14.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6459 sec 2024-04-26T20:16:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:14 smithi119 ceph-mon[32752]: pgmap v3993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:14 smithi167 ceph-mon[32751]: pgmap v3993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:14 smithi090 ceph-mon[29180]: pgmap v3993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:15.466 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6460 sec 2024-04-26T20:16:16.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6461 sec 2024-04-26T20:16:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:16 smithi119 ceph-mon[32752]: pgmap v3994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6057.067701 secs 2024-04-26T20:16:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:16 smithi167 ceph-mon[32751]: pgmap v3994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6057.067701 secs 2024-04-26T20:16:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:16 smithi090 ceph-mon[29180]: pgmap v3994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6057.067701 secs 2024-04-26T20:16:17.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6462 sec 2024-04-26T20:16:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6057.767204 secs 2024-04-26T20:16:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6057.767204 secs 2024-04-26T20:16:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6057.767204 secs 2024-04-26T20:16:18.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6463 sec 2024-04-26T20:16:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:18 smithi119 ceph-mon[32752]: pgmap v3995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:18 smithi167 ceph-mon[32751]: pgmap v3995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:18 smithi090 ceph-mon[29180]: pgmap v3995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:19.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6464 sec 2024-04-26T20:16:20.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6465 sec 2024-04-26T20:16:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:20 smithi119 ceph-mon[32752]: pgmap v3996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:20 smithi167 ceph-mon[32751]: pgmap v3996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:20 smithi090 ceph-mon[29180]: pgmap v3996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:21.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6466 sec 2024-04-26T20:16:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6062.067781 secs 2024-04-26T20:16:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6062.067781 secs 2024-04-26T20:16:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6062.067781 secs 2024-04-26T20:16:22.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6467 sec 2024-04-26T20:16:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6062.767326 secs 2024-04-26T20:16:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:22 smithi119 ceph-mon[32752]: pgmap v3997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6062.767326 secs 2024-04-26T20:16:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:22 smithi167 ceph-mon[32751]: pgmap v3997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6062.767326 secs 2024-04-26T20:16:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:22 smithi090 ceph-mon[29180]: pgmap v3997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:23.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6468 sec 2024-04-26T20:16:24.467 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6469 sec 2024-04-26T20:16:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:24 smithi119 ceph-mon[32752]: pgmap v3998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:24 smithi167 ceph-mon[32751]: pgmap v3998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:24 smithi090 ceph-mon[29180]: pgmap v3998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:25.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6470 sec 2024-04-26T20:16:26.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6471 sec 2024-04-26T20:16:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:26 smithi119 ceph-mon[32752]: pgmap v3999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6067.067829 secs 2024-04-26T20:16:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:26 smithi167 ceph-mon[32751]: pgmap v3999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6067.067829 secs 2024-04-26T20:16:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:26 smithi090 ceph-mon[29180]: pgmap v3999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6067.067829 secs 2024-04-26T20:16:27.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6472 sec 2024-04-26T20:16:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6067.767462 secs 2024-04-26T20:16:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6067.767462 secs 2024-04-26T20:16:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6067.767462 secs 2024-04-26T20:16:28.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6473 sec 2024-04-26T20:16:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:28 smithi119 ceph-mon[32752]: pgmap v4000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:28 smithi167 ceph-mon[32751]: pgmap v4000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:28 smithi090 ceph-mon[29180]: pgmap v4000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:29.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6474 sec 2024-04-26T20:16:30.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6475 sec 2024-04-26T20:16:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:30 smithi119 ceph-mon[32752]: pgmap v4001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:30 smithi167 ceph-mon[32751]: pgmap v4001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:30 smithi090 ceph-mon[29180]: pgmap v4001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:31.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6476 sec 2024-04-26T20:16:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6072.067912 secs 2024-04-26T20:16:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6072.067912 secs 2024-04-26T20:16:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6072.067912 secs 2024-04-26T20:16:32.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6477 sec 2024-04-26T20:16:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6072.767586 secs 2024-04-26T20:16:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:32 smithi090 ceph-mon[29180]: pgmap v4002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6072.767586 secs 2024-04-26T20:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:32 smithi119 ceph-mon[32752]: pgmap v4002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6072.767586 secs 2024-04-26T20:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:32 smithi167 ceph-mon[32751]: pgmap v4002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:33.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6478 sec 2024-04-26T20:16:34.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6479 sec 2024-04-26T20:16:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:34 smithi119 ceph-mon[32752]: pgmap v4003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:34 smithi167 ceph-mon[32751]: pgmap v4003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:34 smithi090 ceph-mon[29180]: pgmap v4003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:35.468 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6480 sec 2024-04-26T20:16:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:35 smithi119 ceph-mon[32752]: pgmap v4004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:35 smithi167 ceph-mon[32751]: pgmap v4004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:35 smithi090 ceph-mon[29180]: pgmap v4004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:36.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6481 sec 2024-04-26T20:16:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:36 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 6077.067997 secs 2024-04-26T20:16:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:36 smithi119 ceph-mon[32752]: slow request 2883.204688 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:16:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6077.767673 secs 2024-04-26T20:16:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:36 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 6077.067997 secs 2024-04-26T20:16:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:36 smithi167 ceph-mon[32751]: slow request 2883.204688 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:16:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6077.767673 secs 2024-04-26T20:16:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:36 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 6077.067997 secs 2024-04-26T20:16:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:36 smithi090 ceph-mon[29180]: slow request 2883.204688 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:16:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6077.767673 secs 2024-04-26T20:16:37.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6482 sec 2024-04-26T20:16:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:37 smithi119 ceph-mon[32752]: pgmap v4005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:37 smithi167 ceph-mon[32751]: pgmap v4005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:37 smithi090 ceph-mon[29180]: pgmap v4005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:38.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6483 sec 2024-04-26T20:16:39.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6484 sec 2024-04-26T20:16:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:40 smithi090 ceph-mon[29180]: pgmap v4006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:40.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6485 sec 2024-04-26T20:16:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:40 smithi119 ceph-mon[32752]: pgmap v4006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:40 smithi167 ceph-mon[32751]: pgmap v4006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6082.068100 secs 2024-04-26T20:16:41.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6486 sec 2024-04-26T20:16:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6082.068100 secs 2024-04-26T20:16:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6082.068100 secs 2024-04-26T20:16:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6082.767768 secs 2024-04-26T20:16:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:42 smithi090 ceph-mon[29180]: pgmap v4007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:42.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6487 sec 2024-04-26T20:16:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6082.767768 secs 2024-04-26T20:16:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:42 smithi119 ceph-mon[32752]: pgmap v4007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6082.767768 secs 2024-04-26T20:16:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:42 smithi167 ceph-mon[32751]: pgmap v4007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:43.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6488 sec 2024-04-26T20:16:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:44 smithi090 ceph-mon[29180]: pgmap v4008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:44.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6489 sec 2024-04-26T20:16:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:44 smithi119 ceph-mon[32752]: pgmap v4008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:44 smithi167 ceph-mon[32751]: pgmap v4008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:16:45.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6490 sec 2024-04-26T20:16:46.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:46 smithi119 ceph-mon[32752]: pgmap v4009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:46.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6087.068161 secs 2024-04-26T20:16:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:46 smithi090 ceph-mon[29180]: pgmap v4009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6087.068161 secs 2024-04-26T20:16:46.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6491 sec 2024-04-26T20:16:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:46 smithi167 ceph-mon[32751]: pgmap v4009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:16:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6087.068161 secs 2024-04-26T20:16:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6087.767892 secs 2024-04-26T20:16:47.469 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6492 sec 2024-04-26T20:16:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6087.767892 secs 2024-04-26T20:16:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6087.767892 secs 2024-04-26T20:16:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:48 smithi090 ceph-mon[29180]: pgmap v4010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:48.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6493 sec 2024-04-26T20:16:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:48 smithi119 ceph-mon[32752]: pgmap v4010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:48 smithi167 ceph-mon[32751]: pgmap v4010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:49.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6494 sec 2024-04-26T20:16:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:50 smithi090 ceph-mon[29180]: pgmap v4011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:50.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6495 sec 2024-04-26T20:16:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:50 smithi119 ceph-mon[32752]: pgmap v4011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:50 smithi167 ceph-mon[32751]: pgmap v4011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:16:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6092.068244 secs 2024-04-26T20:16:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:16:51.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6496 sec 2024-04-26T20:16:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6092.068244 secs 2024-04-26T20:16:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:16:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6092.068244 secs 2024-04-26T20:16:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:16:52.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6497 sec 2024-04-26T20:16:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6092.768016 secs 2024-04-26T20:16:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:52 smithi119 ceph-mon[32752]: pgmap v4012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:16:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:16:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:16:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6092.768016 secs 2024-04-26T20:16:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:52 smithi167 ceph-mon[32751]: pgmap v4012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:16:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:16:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:16:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6092.768016 secs 2024-04-26T20:16:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:52 smithi090 ceph-mon[29180]: pgmap v4012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:16:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:16:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:16:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:16:53.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6498 sec 2024-04-26T20:16:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:53 smithi119 ceph-mon[32752]: pgmap v4013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:16:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:53 smithi119 ceph-mon[32752]: pgmap v4014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:53 smithi167 ceph-mon[32751]: pgmap v4013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:16:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:53 smithi167 ceph-mon[32751]: pgmap v4014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:53 smithi090 ceph-mon[29180]: pgmap v4013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:16:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:53 smithi090 ceph-mon[29180]: pgmap v4014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:54.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6499 sec 2024-04-26T20:16:55.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6500 sec 2024-04-26T20:16:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:55 smithi119 ceph-mon[32752]: pgmap v4015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:55 smithi167 ceph-mon[32751]: pgmap v4015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:55 smithi090 ceph-mon[29180]: pgmap v4015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:56.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6501 sec 2024-04-26T20:16:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6097.068328 secs 2024-04-26T20:16:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6097.068328 secs 2024-04-26T20:16:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6097.068328 secs 2024-04-26T20:16:57.470 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6502 sec 2024-04-26T20:16:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6097.768161 secs 2024-04-26T20:16:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:57 smithi119 ceph-mon[32752]: pgmap v4016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6097.768161 secs 2024-04-26T20:16:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:57 smithi167 ceph-mon[32751]: pgmap v4016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6097.768161 secs 2024-04-26T20:16:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:57 smithi090 ceph-mon[29180]: pgmap v4016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:16:58.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6503 sec 2024-04-26T20:16:59.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6504 sec 2024-04-26T20:16:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:16:59 smithi119 ceph-mon[32752]: pgmap v4017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:16:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:16:59 smithi167 ceph-mon[32751]: pgmap v4017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:16:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:16:59 smithi090 ceph-mon[29180]: pgmap v4017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:17:00.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6505 sec 2024-04-26T20:17:01.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6506 sec 2024-04-26T20:17:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:01 smithi119 ceph-mon[32752]: pgmap v4018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6102.068411 secs 2024-04-26T20:17:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:01 smithi167 ceph-mon[32751]: pgmap v4018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6102.068411 secs 2024-04-26T20:17:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:01 smithi090 ceph-mon[29180]: pgmap v4018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6102.068411 secs 2024-04-26T20:17:02.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6102.768264 secs 2024-04-26T20:17:02.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6507 sec 2024-04-26T20:17:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6102.768264 secs 2024-04-26T20:17:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6102.768264 secs 2024-04-26T20:17:02.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:17:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:17:02.405+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:17:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:17:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:17:02.405+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:17:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:17:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:17:02.405+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:17:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:17:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:17:02.405+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:17:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:17:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:17:02.405+0000 7f82397a1700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:17:03.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6508 sec 2024-04-26T20:17:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:03 smithi119 ceph-mon[32752]: pgmap v4019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:17:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:03 smithi167 ceph-mon[32751]: pgmap v4019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:17:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:03 smithi090 ceph-mon[29180]: pgmap v4019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:17:04.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6509 sec 2024-04-26T20:17:05.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6510 sec 2024-04-26T20:17:05.510 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:05 smithi090 ceph-mon[29180]: pgmap v4020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:05 smithi119 ceph-mon[32752]: pgmap v4020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:05 smithi167 ceph-mon[32751]: pgmap v4020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:06.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6511 sec 2024-04-26T20:17:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6107.068494 secs 2024-04-26T20:17:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6107.068494 secs 2024-04-26T20:17:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6107.068494 secs 2024-04-26T20:17:07.471 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6512 sec 2024-04-26T20:17:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6107.768425 secs 2024-04-26T20:17:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:07 smithi119 ceph-mon[32752]: pgmap v4021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6107.768425 secs 2024-04-26T20:17:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:07 smithi167 ceph-mon[32751]: pgmap v4021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6107.768425 secs 2024-04-26T20:17:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:07 smithi090 ceph-mon[29180]: pgmap v4021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:08.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6513 sec 2024-04-26T20:17:09.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6514 sec 2024-04-26T20:17:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:09 smithi119 ceph-mon[32752]: pgmap v4022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:09 smithi167 ceph-mon[32751]: pgmap v4022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:09 smithi090 ceph-mon[29180]: pgmap v4022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:10.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6515 sec 2024-04-26T20:17:11.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6516 sec 2024-04-26T20:17:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:11 smithi119 ceph-mon[32752]: pgmap v4023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6112.068578 secs 2024-04-26T20:17:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:11 smithi167 ceph-mon[32751]: pgmap v4023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6112.068578 secs 2024-04-26T20:17:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:11 smithi090 ceph-mon[29180]: pgmap v4023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6112.068578 secs 2024-04-26T20:17:12.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6517 sec 2024-04-26T20:17:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6112.768549 secs 2024-04-26T20:17:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6112.768549 secs 2024-04-26T20:17:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6112.768549 secs 2024-04-26T20:17:13.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6518 sec 2024-04-26T20:17:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:13 smithi119 ceph-mon[32752]: pgmap v4024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:13 smithi167 ceph-mon[32751]: pgmap v4024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:13 smithi090 ceph-mon[29180]: pgmap v4024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T20:17:14.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6519 sec 2024-04-26T20:17:15.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6520 sec 2024-04-26T20:17:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:15 smithi119 ceph-mon[32752]: pgmap v4025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:15 smithi167 ceph-mon[32751]: pgmap v4025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:15 smithi090 ceph-mon[29180]: pgmap v4025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T20:17:16.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6117.068662 secs 2024-04-26T20:17:16.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6521 sec 2024-04-26T20:17:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6117.068662 secs 2024-04-26T20:17:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6117.068662 secs 2024-04-26T20:17:17.472 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6522 sec 2024-04-26T20:17:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6117.768672 secs 2024-04-26T20:17:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:17 smithi119 ceph-mon[32752]: pgmap v4026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6117.768672 secs 2024-04-26T20:17:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:17 smithi167 ceph-mon[32751]: pgmap v4026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6117.768672 secs 2024-04-26T20:17:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:17 smithi090 ceph-mon[29180]: pgmap v4026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:18.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6523 sec 2024-04-26T20:17:19.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6524 sec 2024-04-26T20:17:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:19 smithi119 ceph-mon[32752]: pgmap v4027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:19 smithi167 ceph-mon[32751]: pgmap v4027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:19 smithi090 ceph-mon[29180]: pgmap v4027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:20.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6525 sec 2024-04-26T20:17:21.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6526 sec 2024-04-26T20:17:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:21 smithi119 ceph-mon[32752]: pgmap v4028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6122.068744 secs 2024-04-26T20:17:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:21 smithi167 ceph-mon[32751]: pgmap v4028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6122.068744 secs 2024-04-26T20:17:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:21 smithi090 ceph-mon[29180]: pgmap v4028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6122.068744 secs 2024-04-26T20:17:22.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6527 sec 2024-04-26T20:17:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6122.768793 secs 2024-04-26T20:17:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6122.768793 secs 2024-04-26T20:17:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6122.768793 secs 2024-04-26T20:17:23.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6528 sec 2024-04-26T20:17:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:23 smithi119 ceph-mon[32752]: pgmap v4029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:23 smithi167 ceph-mon[32751]: pgmap v4029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:23 smithi090 ceph-mon[29180]: pgmap v4029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:24.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6529 sec 2024-04-26T20:17:25.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6530 sec 2024-04-26T20:17:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:25 smithi119 ceph-mon[32752]: pgmap v4030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:25 smithi167 ceph-mon[32751]: pgmap v4030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:25 smithi090 ceph-mon[29180]: pgmap v4030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:26.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6531 sec 2024-04-26T20:17:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6127.068824 secs 2024-04-26T20:17:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6127.068824 secs 2024-04-26T20:17:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6127.068824 secs 2024-04-26T20:17:27.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6532 sec 2024-04-26T20:17:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6127.768900 secs 2024-04-26T20:17:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:27 smithi119 ceph-mon[32752]: pgmap v4031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6127.768900 secs 2024-04-26T20:17:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:27 smithi167 ceph-mon[32751]: pgmap v4031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6127.768900 secs 2024-04-26T20:17:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:27 smithi090 ceph-mon[29180]: pgmap v4031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:28.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6533 sec 2024-04-26T20:17:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:17:29.275+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T20:17:29.276 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:17:29.275+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:17:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:17:29.276+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 51 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:17:29.277 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:17:29.276+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 51 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T20:17:29.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6534 sec 2024-04-26T20:17:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:29 smithi119 ceph-mon[32752]: pgmap v4032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:29 smithi167 ceph-mon[32751]: pgmap v4032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:29 smithi090 ceph-mon[29180]: pgmap v4032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:30.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6535 sec 2024-04-26T20:17:31.473 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6536 sec 2024-04-26T20:17:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:31 smithi119 ceph-mon[32752]: pgmap v4033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6132.068844 secs 2024-04-26T20:17:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:31 smithi167 ceph-mon[32751]: pgmap v4033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6132.068844 secs 2024-04-26T20:17:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:31 smithi090 ceph-mon[29180]: pgmap v4033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6132.068844 secs 2024-04-26T20:17:32.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6537 sec 2024-04-26T20:17:32.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6132.769023 secs 2024-04-26T20:17:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6132.769023 secs 2024-04-26T20:17:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6132.769023 secs 2024-04-26T20:17:33.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6538 sec 2024-04-26T20:17:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:33 smithi119 ceph-mon[32752]: pgmap v4034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:33 smithi167 ceph-mon[32751]: pgmap v4034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:33 smithi090 ceph-mon[29180]: pgmap v4034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:34.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6539 sec 2024-04-26T20:17:35.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6540 sec 2024-04-26T20:17:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:35 smithi119 ceph-mon[32752]: pgmap v4035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:35 smithi167 ceph-mon[32751]: pgmap v4035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:35 smithi090 ceph-mon[29180]: pgmap v4035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:36.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6541 sec 2024-04-26T20:17:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6137.068906 secs 2024-04-26T20:17:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6137.068906 secs 2024-04-26T20:17:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6137.068906 secs 2024-04-26T20:17:37.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6542 sec 2024-04-26T20:17:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:37 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 6137.769144 secs 2024-04-26T20:17:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:37 smithi119 ceph-mon[32752]: slow request 5764.638156 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:17:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:37 smithi119 ceph-mon[32752]: pgmap v4036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:37 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 6137.769144 secs 2024-04-26T20:17:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:37 smithi167 ceph-mon[32751]: slow request 5764.638156 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:17:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:37 smithi167 ceph-mon[32751]: pgmap v4036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:37 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 6137.769144 secs 2024-04-26T20:17:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:37 smithi090 ceph-mon[29180]: slow request 5764.638156 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:17:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:37 smithi090 ceph-mon[29180]: pgmap v4036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:38.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6543 sec 2024-04-26T20:17:39.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6544 sec 2024-04-26T20:17:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:39 smithi119 ceph-mon[32752]: pgmap v4037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:39 smithi167 ceph-mon[32751]: pgmap v4037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:39 smithi090 ceph-mon[29180]: pgmap v4037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:40.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6545 sec 2024-04-26T20:17:41.474 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6546 sec 2024-04-26T20:17:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:41 smithi119 ceph-mon[32752]: pgmap v4038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6142.068988 secs 2024-04-26T20:17:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:41 smithi167 ceph-mon[32751]: pgmap v4038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6142.068988 secs 2024-04-26T20:17:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:41 smithi090 ceph-mon[29180]: pgmap v4038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6142.068988 secs 2024-04-26T20:17:42.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6547 sec 2024-04-26T20:17:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6142.769244 secs 2024-04-26T20:17:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6142.769244 secs 2024-04-26T20:17:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6142.769244 secs 2024-04-26T20:17:43.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6548 sec 2024-04-26T20:17:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:43 smithi119 ceph-mon[32752]: pgmap v4039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:43 smithi167 ceph-mon[32751]: pgmap v4039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:43 smithi090 ceph-mon[29180]: pgmap v4039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:44.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6549 sec 2024-04-26T20:17:45.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6550 sec 2024-04-26T20:17:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:45 smithi119 ceph-mon[32752]: pgmap v4040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:45 smithi167 ceph-mon[32751]: pgmap v4040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:45 smithi090 ceph-mon[29180]: pgmap v4040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:46.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6551 sec 2024-04-26T20:17:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6147.069052 secs 2024-04-26T20:17:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6147.069052 secs 2024-04-26T20:17:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6147.069052 secs 2024-04-26T20:17:47.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6552 sec 2024-04-26T20:17:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6147.769372 secs 2024-04-26T20:17:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:47 smithi119 ceph-mon[32752]: pgmap v4041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6147.769372 secs 2024-04-26T20:17:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:47 smithi167 ceph-mon[32751]: pgmap v4041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6147.769372 secs 2024-04-26T20:17:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:47 smithi090 ceph-mon[29180]: pgmap v4041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:48.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6553 sec 2024-04-26T20:17:49.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6554 sec 2024-04-26T20:17:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:49 smithi119 ceph-mon[32752]: pgmap v4042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:49 smithi167 ceph-mon[32751]: pgmap v4042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:49 smithi090 ceph-mon[29180]: pgmap v4042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:50.475 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6555 sec 2024-04-26T20:17:51.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6556 sec 2024-04-26T20:17:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:51 smithi119 ceph-mon[32752]: pgmap v4043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6152.069110 secs 2024-04-26T20:17:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:51 smithi167 ceph-mon[32751]: pgmap v4043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6152.069110 secs 2024-04-26T20:17:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:51 smithi090 ceph-mon[29180]: pgmap v4043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:17:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6152.069110 secs 2024-04-26T20:17:52.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6557 sec 2024-04-26T20:17:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6152.769482 secs 2024-04-26T20:17:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:17:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6152.769482 secs 2024-04-26T20:17:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:17:52.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6152.769482 secs 2024-04-26T20:17:52.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:17:53.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6558 sec 2024-04-26T20:17:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:53 smithi119 ceph-mon[32752]: pgmap v4044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:17:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:17:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:17:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:53 smithi167 ceph-mon[32751]: pgmap v4044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:17:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:17:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:17:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:53 smithi090 ceph-mon[29180]: pgmap v4044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:17:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:17:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:17:53.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:17:54.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6559 sec 2024-04-26T20:17:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:54 smithi119 ceph-mon[32752]: pgmap v4045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:17:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:54 smithi119 ceph-mon[32752]: pgmap v4046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:17:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:54 smithi167 ceph-mon[32751]: pgmap v4045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:17:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:54 smithi167 ceph-mon[32751]: pgmap v4046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:17:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:54 smithi090 ceph-mon[29180]: pgmap v4045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T20:17:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:54 smithi090 ceph-mon[29180]: pgmap v4046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:17:55.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6560 sec 2024-04-26T20:17:56.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6561 sec 2024-04-26T20:17:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:56 smithi119 ceph-mon[32752]: pgmap v4047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6157.069194 secs 2024-04-26T20:17:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:56 smithi167 ceph-mon[32751]: pgmap v4047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6157.069194 secs 2024-04-26T20:17:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:56 smithi090 ceph-mon[29180]: pgmap v4047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6157.069194 secs 2024-04-26T20:17:57.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6562 sec 2024-04-26T20:17:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6157.769583 secs 2024-04-26T20:17:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6157.769583 secs 2024-04-26T20:17:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6157.769583 secs 2024-04-26T20:17:58.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6563 sec 2024-04-26T20:17:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:58 smithi119 ceph-mon[32752]: pgmap v4048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:58 smithi167 ceph-mon[32751]: pgmap v4048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:58 smithi090 ceph-mon[29180]: pgmap v4048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:59.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6564 sec 2024-04-26T20:17:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:17:59 smithi119 ceph-mon[32752]: pgmap v4049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:17:59 smithi167 ceph-mon[32751]: pgmap v4049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:17:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:17:59 smithi090 ceph-mon[29180]: pgmap v4049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:18:00.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6565 sec 2024-04-26T20:18:01.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6566 sec 2024-04-26T20:18:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6162.069253 secs 2024-04-26T20:18:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:01 smithi119 ceph-mon[32752]: pgmap v4050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:18:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6162.769719 secs 2024-04-26T20:18:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6162.069253 secs 2024-04-26T20:18:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:01 smithi167 ceph-mon[32751]: pgmap v4050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:18:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6162.769719 secs 2024-04-26T20:18:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6162.069253 secs 2024-04-26T20:18:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:01 smithi090 ceph-mon[29180]: pgmap v4050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:18:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6162.769719 secs 2024-04-26T20:18:02.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6567 sec 2024-04-26T20:18:03.476 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6568 sec 2024-04-26T20:18:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:03 smithi119 ceph-mon[32752]: pgmap v4051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:18:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:03 smithi167 ceph-mon[32751]: pgmap v4051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:18:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:03 smithi090 ceph-mon[29180]: pgmap v4051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:18:04.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6569 sec 2024-04-26T20:18:05.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6570 sec 2024-04-26T20:18:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:05 smithi119 ceph-mon[32752]: pgmap v4052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:05 smithi167 ceph-mon[32751]: pgmap v4052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:05 smithi090 ceph-mon[29180]: pgmap v4052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:06.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6571 sec 2024-04-26T20:18:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6167.069365 secs 2024-04-26T20:18:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6167.769840 secs 2024-04-26T20:18:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6167.069365 secs 2024-04-26T20:18:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6167.769840 secs 2024-04-26T20:18:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6167.069365 secs 2024-04-26T20:18:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6167.769840 secs 2024-04-26T20:18:07.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6572 sec 2024-04-26T20:18:07.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:07 smithi090 ceph-mon[29180]: pgmap v4053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:07 smithi119 ceph-mon[32752]: pgmap v4053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:07 smithi167 ceph-mon[32751]: pgmap v4053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:08.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6573 sec 2024-04-26T20:18:09.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6574 sec 2024-04-26T20:18:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:09 smithi119 ceph-mon[32752]: pgmap v4054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:09 smithi167 ceph-mon[32751]: pgmap v4054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:09 smithi090 ceph-mon[29180]: pgmap v4054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:10.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6575 sec 2024-04-26T20:18:11.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6576 sec 2024-04-26T20:18:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6172.069446 secs 2024-04-26T20:18:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:11 smithi119 ceph-mon[32752]: pgmap v4055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6172.769959 secs 2024-04-26T20:18:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6172.069446 secs 2024-04-26T20:18:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:11 smithi167 ceph-mon[32751]: pgmap v4055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6172.769959 secs 2024-04-26T20:18:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6172.069446 secs 2024-04-26T20:18:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:11 smithi090 ceph-mon[29180]: pgmap v4055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6172.769959 secs 2024-04-26T20:18:12.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6577 sec 2024-04-26T20:18:13.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6578 sec 2024-04-26T20:18:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:13 smithi119 ceph-mon[32752]: pgmap v4056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:13 smithi167 ceph-mon[32751]: pgmap v4056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:13 smithi090 ceph-mon[29180]: pgmap v4056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:18:14.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6579 sec 2024-04-26T20:18:15.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6580 sec 2024-04-26T20:18:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:15 smithi119 ceph-mon[32752]: pgmap v4057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:15 smithi167 ceph-mon[32751]: pgmap v4057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:15 smithi090 ceph-mon[29180]: pgmap v4057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:18:16.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6581 sec 2024-04-26T20:18:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6177.069518 secs 2024-04-26T20:18:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6177.770035 secs 2024-04-26T20:18:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6177.069518 secs 2024-04-26T20:18:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6177.770035 secs 2024-04-26T20:18:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6177.069518 secs 2024-04-26T20:18:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6177.770035 secs 2024-04-26T20:18:17.477 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6582 sec 2024-04-26T20:18:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:17 smithi119 ceph-mon[32752]: pgmap v4058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:17 smithi167 ceph-mon[32751]: pgmap v4058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:17 smithi090 ceph-mon[29180]: pgmap v4058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:18.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6583 sec 2024-04-26T20:18:19.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6584 sec 2024-04-26T20:18:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:19 smithi119 ceph-mon[32752]: pgmap v4059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:19 smithi167 ceph-mon[32751]: pgmap v4059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:19 smithi090 ceph-mon[29180]: pgmap v4059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:20.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6585 sec 2024-04-26T20:18:21.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6586 sec 2024-04-26T20:18:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6182.069601 secs 2024-04-26T20:18:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:21 smithi119 ceph-mon[32752]: pgmap v4060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6182.770154 secs 2024-04-26T20:18:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6182.069601 secs 2024-04-26T20:18:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:21 smithi167 ceph-mon[32751]: pgmap v4060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6182.770154 secs 2024-04-26T20:18:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6182.069601 secs 2024-04-26T20:18:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:21 smithi090 ceph-mon[29180]: pgmap v4060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6182.770154 secs 2024-04-26T20:18:22.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6587 sec 2024-04-26T20:18:23.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6588 sec 2024-04-26T20:18:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:23 smithi119 ceph-mon[32752]: pgmap v4061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:23 smithi167 ceph-mon[32751]: pgmap v4061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:23 smithi090 ceph-mon[29180]: pgmap v4061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:24.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6589 sec 2024-04-26T20:18:25.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6590 sec 2024-04-26T20:18:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:25 smithi119 ceph-mon[32752]: pgmap v4062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:25 smithi167 ceph-mon[32751]: pgmap v4062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:25 smithi090 ceph-mon[29180]: pgmap v4062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:26.478 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6591 sec 2024-04-26T20:18:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6187.069687 secs 2024-04-26T20:18:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6187.770265 secs 2024-04-26T20:18:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6187.069687 secs 2024-04-26T20:18:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6187.770265 secs 2024-04-26T20:18:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6187.069687 secs 2024-04-26T20:18:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6187.770265 secs 2024-04-26T20:18:27.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6592 sec 2024-04-26T20:18:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:27 smithi119 ceph-mon[32752]: pgmap v4063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:27 smithi167 ceph-mon[32751]: pgmap v4063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:27 smithi090 ceph-mon[29180]: pgmap v4063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:28.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6593 sec 2024-04-26T20:18:29.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6594 sec 2024-04-26T20:18:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:29 smithi119 ceph-mon[32752]: pgmap v4064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:29 smithi167 ceph-mon[32751]: pgmap v4064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:29 smithi090 ceph-mon[29180]: pgmap v4064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:30.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6595 sec 2024-04-26T20:18:31.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6596 sec 2024-04-26T20:18:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6192.069769 secs 2024-04-26T20:18:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:31 smithi119 ceph-mon[32752]: pgmap v4065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6192.770365 secs 2024-04-26T20:18:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6192.069769 secs 2024-04-26T20:18:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:31 smithi167 ceph-mon[32751]: pgmap v4065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6192.770365 secs 2024-04-26T20:18:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6192.069769 secs 2024-04-26T20:18:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:31 smithi090 ceph-mon[29180]: pgmap v4065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6192.770365 secs 2024-04-26T20:18:32.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6597 sec 2024-04-26T20:18:33.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6598 sec 2024-04-26T20:18:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:33 smithi119 ceph-mon[32752]: pgmap v4066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:33 smithi167 ceph-mon[32751]: pgmap v4066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:33 smithi090 ceph-mon[29180]: pgmap v4066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:34.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6599 sec 2024-04-26T20:18:35.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6600 sec 2024-04-26T20:18:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:35 smithi119 ceph-mon[32752]: pgmap v4067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:35 smithi167 ceph-mon[32751]: pgmap v4067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:35 smithi090 ceph-mon[29180]: pgmap v4067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:36.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6601 sec 2024-04-26T20:18:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6197.069801 secs 2024-04-26T20:18:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6197.770435 secs 2024-04-26T20:18:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6197.069801 secs 2024-04-26T20:18:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6197.770435 secs 2024-04-26T20:18:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6197.069801 secs 2024-04-26T20:18:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6197.770435 secs 2024-04-26T20:18:37.479 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6602 sec 2024-04-26T20:18:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:37 smithi119 ceph-mon[32752]: pgmap v4068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:37 smithi167 ceph-mon[32751]: pgmap v4068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:37 smithi090 ceph-mon[29180]: pgmap v4068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:38.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6603 sec 2024-04-26T20:18:39.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6604 sec 2024-04-26T20:18:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:39 smithi119 ceph-mon[32752]: pgmap v4069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:39 smithi167 ceph-mon[32751]: pgmap v4069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:39 smithi090 ceph-mon[29180]: pgmap v4069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:40.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6605 sec 2024-04-26T20:18:41.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6606 sec 2024-04-26T20:18:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6202.069826 secs 2024-04-26T20:18:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:41 smithi119 ceph-mon[32752]: pgmap v4070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6202.770543 secs 2024-04-26T20:18:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6202.069826 secs 2024-04-26T20:18:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:41 smithi167 ceph-mon[32751]: pgmap v4070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6202.770543 secs 2024-04-26T20:18:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6202.069826 secs 2024-04-26T20:18:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:41 smithi090 ceph-mon[29180]: pgmap v4070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6202.770543 secs 2024-04-26T20:18:42.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6607 sec 2024-04-26T20:18:43.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6608 sec 2024-04-26T20:18:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:43 smithi119 ceph-mon[32752]: pgmap v4071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:43 smithi167 ceph-mon[32751]: pgmap v4071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:43 smithi090 ceph-mon[29180]: pgmap v4071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:44.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6609 sec 2024-04-26T20:18:45.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6610 sec 2024-04-26T20:18:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:45 smithi119 ceph-mon[32752]: pgmap v4072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:45 smithi167 ceph-mon[32751]: pgmap v4072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:45 smithi090 ceph-mon[29180]: pgmap v4072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:46.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6611 sec 2024-04-26T20:18:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6207.069908 secs 2024-04-26T20:18:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6207.770616 secs 2024-04-26T20:18:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6207.069908 secs 2024-04-26T20:18:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6207.770616 secs 2024-04-26T20:18:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6207.069908 secs 2024-04-26T20:18:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6207.770616 secs 2024-04-26T20:18:47.480 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6612 sec 2024-04-26T20:18:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:47 smithi119 ceph-mon[32752]: pgmap v4073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:47 smithi167 ceph-mon[32751]: pgmap v4073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:47 smithi090 ceph-mon[29180]: pgmap v4073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:48.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6613 sec 2024-04-26T20:18:49.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6614 sec 2024-04-26T20:18:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:49 smithi119 ceph-mon[32752]: pgmap v4074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:49 smithi167 ceph-mon[32751]: pgmap v4074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:49 smithi090 ceph-mon[29180]: pgmap v4074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:18:50.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6615 sec 2024-04-26T20:18:51.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6616 sec 2024-04-26T20:18:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6212.069967 secs 2024-04-26T20:18:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:51 smithi119 ceph-mon[32752]: pgmap v4075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6212.770752 secs 2024-04-26T20:18:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6212.069967 secs 2024-04-26T20:18:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:51 smithi167 ceph-mon[32751]: pgmap v4075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6212.770752 secs 2024-04-26T20:18:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6212.069967 secs 2024-04-26T20:18:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:51 smithi090 ceph-mon[29180]: pgmap v4075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6212.770752 secs 2024-04-26T20:18:52.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6617 sec 2024-04-26T20:18:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:18:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:18:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:18:53.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6618 sec 2024-04-26T20:18:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:53 smithi167 ceph-mon[32751]: pgmap v4076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:18:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:18:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:53 smithi167 ceph-mon[32751]: pgmap v4077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:18:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:53 smithi167 ceph-mon[32751]: pgmap v4078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:18:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:18:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:53 smithi090 ceph-mon[29180]: pgmap v4076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:18:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:18:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:53 smithi090 ceph-mon[29180]: pgmap v4077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:18:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:53 smithi090 ceph-mon[29180]: pgmap v4078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:18:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:18:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:53 smithi119 ceph-mon[32752]: pgmap v4076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:18:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:18:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:18:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:53 smithi119 ceph-mon[32752]: pgmap v4077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:18:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:53 smithi119 ceph-mon[32752]: pgmap v4078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:18:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:18:54.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6619 sec 2024-04-26T20:18:55.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6620 sec 2024-04-26T20:18:56.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6621 sec 2024-04-26T20:18:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6217.070059 secs 2024-04-26T20:18:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:56 smithi119 ceph-mon[32752]: pgmap v4079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:18:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6217.070059 secs 2024-04-26T20:18:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:56 smithi167 ceph-mon[32751]: pgmap v4079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:18:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6217.070059 secs 2024-04-26T20:18:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:56 smithi090 ceph-mon[29180]: pgmap v4079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:18:57.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6622 sec 2024-04-26T20:18:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6217.770873 secs 2024-04-26T20:18:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6217.770873 secs 2024-04-26T20:18:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6217.770873 secs 2024-04-26T20:18:58.481 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6623 sec 2024-04-26T20:18:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:18:58 smithi119 ceph-mon[32752]: pgmap v4080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:18:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:18:58 smithi167 ceph-mon[32751]: pgmap v4080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:18:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:18:58 smithi090 ceph-mon[29180]: pgmap v4080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:18:59.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6624 sec 2024-04-26T20:19:00.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6625 sec 2024-04-26T20:19:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:00 smithi119 ceph-mon[32752]: pgmap v4081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:19:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:00 smithi167 ceph-mon[32751]: pgmap v4081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:19:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:00 smithi090 ceph-mon[29180]: pgmap v4081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:19:01.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6626 sec 2024-04-26T20:19:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6222.070114 secs 2024-04-26T20:19:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6222.070114 secs 2024-04-26T20:19:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6222.070114 secs 2024-04-26T20:19:02.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6627 sec 2024-04-26T20:19:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6222.770970 secs 2024-04-26T20:19:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:02 smithi119 ceph-mon[32752]: pgmap v4082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:19:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6222.770970 secs 2024-04-26T20:19:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:02 smithi167 ceph-mon[32751]: pgmap v4082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:19:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6222.770970 secs 2024-04-26T20:19:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:02 smithi090 ceph-mon[29180]: pgmap v4082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:19:03.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6628 sec 2024-04-26T20:19:04.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6629 sec 2024-04-26T20:19:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:04 smithi119 ceph-mon[32752]: pgmap v4083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T20:19:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:04 smithi167 ceph-mon[32751]: pgmap v4083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T20:19:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:04 smithi090 ceph-mon[29180]: pgmap v4083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T20:19:05.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6630 sec 2024-04-26T20:19:06.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6631 sec 2024-04-26T20:19:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6227.070199 secs 2024-04-26T20:19:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:06 smithi119 ceph-mon[32752]: pgmap v4084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6227.070199 secs 2024-04-26T20:19:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:06 smithi167 ceph-mon[32751]: pgmap v4084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6227.070199 secs 2024-04-26T20:19:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:06 smithi090 ceph-mon[29180]: pgmap v4084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:07.482 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6632 sec 2024-04-26T20:19:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6227.771067 secs 2024-04-26T20:19:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6227.771067 secs 2024-04-26T20:19:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6227.771067 secs 2024-04-26T20:19:08.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6633 sec 2024-04-26T20:19:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:08 smithi119 ceph-mon[32752]: pgmap v4085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:19:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:08 smithi167 ceph-mon[32751]: pgmap v4085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:19:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:08 smithi090 ceph-mon[29180]: pgmap v4085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:19:09.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6634 sec 2024-04-26T20:19:10.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6635 sec 2024-04-26T20:19:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:10 smithi119 ceph-mon[32752]: pgmap v4086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:10 smithi167 ceph-mon[32751]: pgmap v4086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:10 smithi090 ceph-mon[29180]: pgmap v4086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:11.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6636 sec 2024-04-26T20:19:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6232.070253 secs 2024-04-26T20:19:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6232.070253 secs 2024-04-26T20:19:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6232.070253 secs 2024-04-26T20:19:12.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6637 sec 2024-04-26T20:19:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6232.771185 secs 2024-04-26T20:19:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:12 smithi119 ceph-mon[32752]: pgmap v4087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6232.771185 secs 2024-04-26T20:19:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:12 smithi167 ceph-mon[32751]: pgmap v4087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6232.771185 secs 2024-04-26T20:19:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:12 smithi090 ceph-mon[29180]: pgmap v4087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:13.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6638 sec 2024-04-26T20:19:14.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6639 sec 2024-04-26T20:19:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:14 smithi119 ceph-mon[32752]: pgmap v4088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:14 smithi167 ceph-mon[32751]: pgmap v4088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:14 smithi090 ceph-mon[29180]: pgmap v4088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:15.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6640 sec 2024-04-26T20:19:16.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6641 sec 2024-04-26T20:19:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6237.070335 secs 2024-04-26T20:19:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:16 smithi119 ceph-mon[32752]: pgmap v4089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6237.070335 secs 2024-04-26T20:19:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:16 smithi167 ceph-mon[32751]: pgmap v4089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6237.070335 secs 2024-04-26T20:19:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:16 smithi090 ceph-mon[29180]: pgmap v4089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:17.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6642 sec 2024-04-26T20:19:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6237.771303 secs 2024-04-26T20:19:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6237.771303 secs 2024-04-26T20:19:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6237.771303 secs 2024-04-26T20:19:18.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6643 sec 2024-04-26T20:19:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:18 smithi119 ceph-mon[32752]: pgmap v4090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:18 smithi167 ceph-mon[32751]: pgmap v4090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:18 smithi090 ceph-mon[29180]: pgmap v4090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:19.483 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6644 sec 2024-04-26T20:19:20.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6645 sec 2024-04-26T20:19:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:20 smithi119 ceph-mon[32752]: pgmap v4091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:20 smithi167 ceph-mon[32751]: pgmap v4091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:20 smithi090 ceph-mon[29180]: pgmap v4091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:21.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6646 sec 2024-04-26T20:19:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6242.070419 secs 2024-04-26T20:19:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6242.070419 secs 2024-04-26T20:19:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6242.070419 secs 2024-04-26T20:19:22.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6647 sec 2024-04-26T20:19:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6242.771441 secs 2024-04-26T20:19:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:22 smithi119 ceph-mon[32752]: pgmap v4092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6242.771441 secs 2024-04-26T20:19:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:22 smithi167 ceph-mon[32751]: pgmap v4092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6242.771441 secs 2024-04-26T20:19:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:22 smithi090 ceph-mon[29180]: pgmap v4092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:23.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6648 sec 2024-04-26T20:19:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:23 smithi119 ceph-mon[32752]: pgmap v4093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:23 smithi167 ceph-mon[32751]: pgmap v4093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:23 smithi090 ceph-mon[29180]: pgmap v4093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:24.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6649 sec 2024-04-26T20:19:25.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6650 sec 2024-04-26T20:19:26.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6651 sec 2024-04-26T20:19:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6247.070506 secs 2024-04-26T20:19:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:26 smithi119 ceph-mon[32752]: pgmap v4094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6247.070506 secs 2024-04-26T20:19:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:26 smithi167 ceph-mon[32751]: pgmap v4094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6247.070506 secs 2024-04-26T20:19:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:26 smithi090 ceph-mon[29180]: pgmap v4094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:27.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6652 sec 2024-04-26T20:19:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6247.771550 secs 2024-04-26T20:19:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6247.771550 secs 2024-04-26T20:19:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6247.771550 secs 2024-04-26T20:19:28.484 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6653 sec 2024-04-26T20:19:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:28 smithi119 ceph-mon[32752]: pgmap v4095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:28 smithi167 ceph-mon[32751]: pgmap v4095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:28 smithi090 ceph-mon[29180]: pgmap v4095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:29.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6654 sec 2024-04-26T20:19:30.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6655 sec 2024-04-26T20:19:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:30 smithi119 ceph-mon[32752]: pgmap v4096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:30 smithi167 ceph-mon[32751]: pgmap v4096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:30 smithi090 ceph-mon[29180]: pgmap v4096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:31.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6656 sec 2024-04-26T20:19:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6252.070588 secs 2024-04-26T20:19:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6252.070588 secs 2024-04-26T20:19:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6252.070588 secs 2024-04-26T20:19:32.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6657 sec 2024-04-26T20:19:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6252.771645 secs 2024-04-26T20:19:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:32 smithi119 ceph-mon[32752]: pgmap v4097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6252.771645 secs 2024-04-26T20:19:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:32 smithi167 ceph-mon[32751]: pgmap v4097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6252.771645 secs 2024-04-26T20:19:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:32 smithi090 ceph-mon[29180]: pgmap v4097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:33.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6658 sec 2024-04-26T20:19:34.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6659 sec 2024-04-26T20:19:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:34 smithi119 ceph-mon[32752]: pgmap v4098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:34 smithi167 ceph-mon[32751]: pgmap v4098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:34 smithi090 ceph-mon[29180]: pgmap v4098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:35.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6660 sec 2024-04-26T20:19:36.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6661 sec 2024-04-26T20:19:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6257.070668 secs 2024-04-26T20:19:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:36 smithi119 ceph-mon[32752]: pgmap v4099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6257.070668 secs 2024-04-26T20:19:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:36 smithi167 ceph-mon[32751]: pgmap v4099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6257.070668 secs 2024-04-26T20:19:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:36 smithi090 ceph-mon[29180]: pgmap v4099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:37.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6662 sec 2024-04-26T20:19:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6257.771763 secs 2024-04-26T20:19:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6257.771763 secs 2024-04-26T20:19:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6257.771763 secs 2024-04-26T20:19:38.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6663 sec 2024-04-26T20:19:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:38 smithi119 ceph-mon[32752]: pgmap v4100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:38 smithi167 ceph-mon[32751]: pgmap v4100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:38 smithi090 ceph-mon[29180]: pgmap v4100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:39.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6664 sec 2024-04-26T20:19:40.485 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6665 sec 2024-04-26T20:19:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:40 smithi119 ceph-mon[32752]: pgmap v4101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:40 smithi167 ceph-mon[32751]: pgmap v4101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:40 smithi090 ceph-mon[29180]: pgmap v4101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:41.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6666 sec 2024-04-26T20:19:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6262.070752 secs 2024-04-26T20:19:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6262.070752 secs 2024-04-26T20:19:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6262.070752 secs 2024-04-26T20:19:42.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6667 sec 2024-04-26T20:19:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6262.771898 secs 2024-04-26T20:19:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:42 smithi119 ceph-mon[32752]: pgmap v4102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6262.771898 secs 2024-04-26T20:19:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:42 smithi167 ceph-mon[32751]: pgmap v4102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6262.771898 secs 2024-04-26T20:19:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:42 smithi090 ceph-mon[29180]: pgmap v4102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:43.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6668 sec 2024-04-26T20:19:44.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6669 sec 2024-04-26T20:19:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:44 smithi119 ceph-mon[32752]: pgmap v4103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:44 smithi167 ceph-mon[32751]: pgmap v4103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:44 smithi090 ceph-mon[29180]: pgmap v4103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:45.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6670 sec 2024-04-26T20:19:46.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6671 sec 2024-04-26T20:19:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6267.070848 secs 2024-04-26T20:19:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:46 smithi119 ceph-mon[32752]: pgmap v4104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6267.070848 secs 2024-04-26T20:19:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:46 smithi167 ceph-mon[32751]: pgmap v4104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6267.070848 secs 2024-04-26T20:19:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:46 smithi090 ceph-mon[29180]: pgmap v4104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:47.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6672 sec 2024-04-26T20:19:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6267.772035 secs 2024-04-26T20:19:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6267.772035 secs 2024-04-26T20:19:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6267.772035 secs 2024-04-26T20:19:48.486 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6673 sec 2024-04-26T20:19:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:48 smithi119 ceph-mon[32752]: pgmap v4105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:48 smithi167 ceph-mon[32751]: pgmap v4105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:48 smithi090 ceph-mon[29180]: pgmap v4105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:49.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6674 sec 2024-04-26T20:19:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:49 smithi090 ceph-mon[29180]: pgmap v4106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:49 smithi119 ceph-mon[32752]: pgmap v4106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:49 smithi167 ceph-mon[32751]: pgmap v4106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:19:50.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6675 sec 2024-04-26T20:19:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:50 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6272.070932 secs 2024-04-26T20:19:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:50 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6272.070932 secs 2024-04-26T20:19:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:50 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6272.070932 secs 2024-04-26T20:19:51.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6676 sec 2024-04-26T20:19:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6272.772150 secs 2024-04-26T20:19:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:51 smithi119 ceph-mon[32752]: pgmap v4107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6272.772150 secs 2024-04-26T20:19:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:51 smithi167 ceph-mon[32751]: pgmap v4107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6272.772150 secs 2024-04-26T20:19:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:51 smithi090 ceph-mon[29180]: pgmap v4107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:52.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6677 sec 2024-04-26T20:19:53.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6678 sec 2024-04-26T20:19:54.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6679 sec 2024-04-26T20:19:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:54 smithi119 ceph-mon[32752]: pgmap v4108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:19:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:19:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:19:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:19:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:54 smithi167 ceph-mon[32751]: pgmap v4108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:19:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:19:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:19:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:19:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:54 smithi090 ceph-mon[29180]: pgmap v4108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:19:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:19:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:19:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:19:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:19:55.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6680 sec 2024-04-26T20:19:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:55 smithi119 ceph-mon[32752]: pgmap v4109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:19:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:55 smithi119 ceph-mon[32752]: pgmap v4110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:19:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:55 smithi167 ceph-mon[32751]: pgmap v4109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:19:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:55 smithi167 ceph-mon[32751]: pgmap v4110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:19:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:55 smithi090 ceph-mon[29180]: pgmap v4109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:19:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:55 smithi090 ceph-mon[29180]: pgmap v4110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:19:56.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6681 sec 2024-04-26T20:19:56.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6277.070993 secs 2024-04-26T20:19:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6277.070993 secs 2024-04-26T20:19:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6277.070993 secs 2024-04-26T20:19:57.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6682 sec 2024-04-26T20:19:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6277.772265 secs 2024-04-26T20:19:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:57 smithi119 ceph-mon[32752]: pgmap v4111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:19:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6277.772265 secs 2024-04-26T20:19:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:57 smithi167 ceph-mon[32751]: pgmap v4111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:19:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6277.772265 secs 2024-04-26T20:19:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:57 smithi090 ceph-mon[29180]: pgmap v4111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:19:58.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6683 sec 2024-04-26T20:19:59.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6684 sec 2024-04-26T20:19:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:19:59 smithi119 ceph-mon[32752]: pgmap v4112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:19:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:19:59 smithi167 ceph-mon[32751]: pgmap v4112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:19:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:19:59 smithi090 ceph-mon[29180]: pgmap v4112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:20:00.487 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6685 sec 2024-04-26T20:20:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:20:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:20:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:20:01.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6686 sec 2024-04-26T20:20:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6282.071098 secs 2024-04-26T20:20:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:01 smithi119 ceph-mon[32752]: pgmap v4113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6282.071098 secs 2024-04-26T20:20:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:01 smithi167 ceph-mon[32751]: pgmap v4113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6282.071098 secs 2024-04-26T20:20:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:01 smithi090 ceph-mon[29180]: pgmap v4113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:02.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6687 sec 2024-04-26T20:20:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6282.772393 secs 2024-04-26T20:20:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6282.772393 secs 2024-04-26T20:20:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6282.772393 secs 2024-04-26T20:20:03.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6688 sec 2024-04-26T20:20:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:03 smithi119 ceph-mon[32752]: pgmap v4114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:03 smithi167 ceph-mon[32751]: pgmap v4114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:03 smithi090 ceph-mon[29180]: pgmap v4114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:04.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6689 sec 2024-04-26T20:20:05.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6690 sec 2024-04-26T20:20:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:05 smithi119 ceph-mon[32752]: pgmap v4115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:20:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:05 smithi167 ceph-mon[32751]: pgmap v4115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:20:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:05 smithi090 ceph-mon[29180]: pgmap v4115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:20:06.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6691 sec 2024-04-26T20:20:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6287.071182 secs 2024-04-26T20:20:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6287.071182 secs 2024-04-26T20:20:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6287.071182 secs 2024-04-26T20:20:07.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6692 sec 2024-04-26T20:20:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6287.772511 secs 2024-04-26T20:20:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:07 smithi119 ceph-mon[32752]: pgmap v4116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6287.772511 secs 2024-04-26T20:20:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:07 smithi167 ceph-mon[32751]: pgmap v4116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6287.772511 secs 2024-04-26T20:20:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:07 smithi090 ceph-mon[29180]: pgmap v4116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:08.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6693 sec 2024-04-26T20:20:09.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6694 sec 2024-04-26T20:20:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:09 smithi119 ceph-mon[32752]: pgmap v4117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:09 smithi167 ceph-mon[32751]: pgmap v4117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:09 smithi090 ceph-mon[29180]: pgmap v4117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:10.488 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6695 sec 2024-04-26T20:20:11.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6696 sec 2024-04-26T20:20:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6292.071256 secs 2024-04-26T20:20:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:11 smithi119 ceph-mon[32752]: pgmap v4118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6292.071256 secs 2024-04-26T20:20:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:11 smithi167 ceph-mon[32751]: pgmap v4118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6292.071256 secs 2024-04-26T20:20:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:11 smithi090 ceph-mon[29180]: pgmap v4118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:12.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6697 sec 2024-04-26T20:20:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6292.772602 secs 2024-04-26T20:20:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:12 smithi119 ceph-mon[32752]: pgmap v4119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6292.772602 secs 2024-04-26T20:20:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:12 smithi167 ceph-mon[32751]: pgmap v4119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6292.772602 secs 2024-04-26T20:20:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:12 smithi090 ceph-mon[29180]: pgmap v4119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:13.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6698 sec 2024-04-26T20:20:14.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6699 sec 2024-04-26T20:20:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:15 smithi119 ceph-mon[32752]: pgmap v4120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:15 smithi167 ceph-mon[32751]: pgmap v4120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:15 smithi090 ceph-mon[29180]: pgmap v4120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:15.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6700 sec 2024-04-26T20:20:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6297.071342 secs 2024-04-26T20:20:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6297.071342 secs 2024-04-26T20:20:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6297.071342 secs 2024-04-26T20:20:16.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6701 sec 2024-04-26T20:20:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6297.772676 secs 2024-04-26T20:20:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:17 smithi119 ceph-mon[32752]: pgmap v4121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6297.772676 secs 2024-04-26T20:20:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:17 smithi167 ceph-mon[32751]: pgmap v4121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6297.772676 secs 2024-04-26T20:20:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:17 smithi090 ceph-mon[29180]: pgmap v4121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:17.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6702 sec 2024-04-26T20:20:18.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6703 sec 2024-04-26T20:20:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:19 smithi119 ceph-mon[32752]: pgmap v4122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:19 smithi167 ceph-mon[32751]: pgmap v4122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:19 smithi090 ceph-mon[29180]: pgmap v4122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:19.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6704 sec 2024-04-26T20:20:20.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6705 sec 2024-04-26T20:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6302.071425 secs 2024-04-26T20:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:21 smithi119 ceph-mon[32752]: pgmap v4123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6302.071425 secs 2024-04-26T20:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:21 smithi167 ceph-mon[32751]: pgmap v4123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6302.071425 secs 2024-04-26T20:20:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:21 smithi090 ceph-mon[29180]: pgmap v4123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:21.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6706 sec 2024-04-26T20:20:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6302.772791 secs 2024-04-26T20:20:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6302.772791 secs 2024-04-26T20:20:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6302.772791 secs 2024-04-26T20:20:22.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6707 sec 2024-04-26T20:20:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:23 smithi119 ceph-mon[32752]: pgmap v4124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:23 smithi167 ceph-mon[32751]: pgmap v4124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:23 smithi090 ceph-mon[29180]: pgmap v4124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:23.489 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6708 sec 2024-04-26T20:20:24.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6709 sec 2024-04-26T20:20:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:25 smithi119 ceph-mon[32752]: pgmap v4125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:25 smithi167 ceph-mon[32751]: pgmap v4125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:25 smithi090 ceph-mon[29180]: pgmap v4125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:25.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6710 sec 2024-04-26T20:20:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6307.071503 secs 2024-04-26T20:20:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6307.071503 secs 2024-04-26T20:20:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6307.071503 secs 2024-04-26T20:20:26.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6711 sec 2024-04-26T20:20:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6307.772909 secs 2024-04-26T20:20:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:27 smithi119 ceph-mon[32752]: pgmap v4126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6307.772909 secs 2024-04-26T20:20:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:27 smithi167 ceph-mon[32751]: pgmap v4126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6307.772909 secs 2024-04-26T20:20:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:27 smithi090 ceph-mon[29180]: pgmap v4126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:27.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6712 sec 2024-04-26T20:20:28.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6713 sec 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 12 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=5 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2280 msgr2=0x7f2cc00b4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).tick idle (900022000) for more than 900000000 us, fault. 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2280 0x7f2cc00b4d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401c010 tx=0x7f2cc4029040 comp rx=0 tx=0).stop 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2280 msgr2=0x7f2cc00b4d10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2280 0x7f2cc00b4d10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:20:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.492+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.493+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b5250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401cf40 tx=0x7f2cc4046040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.493+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T20:20:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:28.493+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T20:20:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:29.299+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:20:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:29.299+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T20:20:29.300 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:29.299+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 52 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:20:29.301 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:20:29.299+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 52 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006390 con 0x7f2cd4104500 2024-04-26T20:20:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:29 smithi119 ceph-mon[32752]: pgmap v4127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:29 smithi167 ceph-mon[32751]: pgmap v4127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:29.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:29 smithi090 ceph-mon[29180]: pgmap v4127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:29.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6714 sec 2024-04-26T20:20:30.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6715 sec 2024-04-26T20:20:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6312.071586 secs 2024-04-26T20:20:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:31 smithi119 ceph-mon[32752]: pgmap v4128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6312.071586 secs 2024-04-26T20:20:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:31 smithi167 ceph-mon[32751]: pgmap v4128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6312.071586 secs 2024-04-26T20:20:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:31 smithi090 ceph-mon[29180]: pgmap v4128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:31.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6716 sec 2024-04-26T20:20:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6312.773027 secs 2024-04-26T20:20:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6312.773027 secs 2024-04-26T20:20:32.490 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6717 sec 2024-04-26T20:20:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6312.773027 secs 2024-04-26T20:20:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:33 smithi167 ceph-mon[32751]: pgmap v4129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:33 smithi090 ceph-mon[29180]: pgmap v4129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:33.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6718 sec 2024-04-26T20:20:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:33 smithi119 ceph-mon[32752]: pgmap v4129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:34.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6719 sec 2024-04-26T20:20:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:35 smithi167 ceph-mon[32751]: pgmap v4130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:35 smithi090 ceph-mon[29180]: pgmap v4130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:35.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6720 sec 2024-04-26T20:20:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:35 smithi119 ceph-mon[32752]: pgmap v4130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6317.071668 secs 2024-04-26T20:20:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6317.071668 secs 2024-04-26T20:20:36.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6721 sec 2024-04-26T20:20:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6317.071668 secs 2024-04-26T20:20:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6317.773146 secs 2024-04-26T20:20:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:37 smithi167 ceph-mon[32751]: pgmap v4131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6317.773146 secs 2024-04-26T20:20:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:37 smithi090 ceph-mon[29180]: pgmap v4131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:37.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6722 sec 2024-04-26T20:20:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6317.773146 secs 2024-04-26T20:20:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:37 smithi119 ceph-mon[32752]: pgmap v4131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:38.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6723 sec 2024-04-26T20:20:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:39 smithi090 ceph-mon[29180]: pgmap v4132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:39.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6724 sec 2024-04-26T20:20:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:39 smithi119 ceph-mon[32752]: pgmap v4132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:39 smithi167 ceph-mon[32751]: pgmap v4132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:40.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6725 sec 2024-04-26T20:20:41.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6726 sec 2024-04-26T20:20:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6322.071756 secs 2024-04-26T20:20:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:41 smithi119 ceph-mon[32752]: pgmap v4133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6322.071756 secs 2024-04-26T20:20:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:41 smithi167 ceph-mon[32751]: pgmap v4133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6322.071756 secs 2024-04-26T20:20:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:41 smithi090 ceph-mon[29180]: pgmap v4133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:42.491 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6727 sec 2024-04-26T20:20:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6322.773209 secs 2024-04-26T20:20:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6322.773209 secs 2024-04-26T20:20:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6322.773209 secs 2024-04-26T20:20:43.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6728 sec 2024-04-26T20:20:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:43 smithi119 ceph-mon[32752]: pgmap v4134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:43 smithi167 ceph-mon[32751]: pgmap v4134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:43.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:43 smithi090 ceph-mon[29180]: pgmap v4134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:44.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6729 sec 2024-04-26T20:20:45.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6730 sec 2024-04-26T20:20:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:45 smithi119 ceph-mon[32752]: pgmap v4135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:45 smithi167 ceph-mon[32751]: pgmap v4135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:45 smithi090 ceph-mon[29180]: pgmap v4135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6327.071842 secs 2024-04-26T20:20:46.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6731 sec 2024-04-26T20:20:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6327.071842 secs 2024-04-26T20:20:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6327.071842 secs 2024-04-26T20:20:47.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6732 sec 2024-04-26T20:20:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6327.773345 secs 2024-04-26T20:20:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:47 smithi119 ceph-mon[32752]: pgmap v4136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6327.773345 secs 2024-04-26T20:20:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:47 smithi167 ceph-mon[32751]: pgmap v4136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6327.773345 secs 2024-04-26T20:20:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:47 smithi090 ceph-mon[29180]: pgmap v4136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:48.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6733 sec 2024-04-26T20:20:49.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6734 sec 2024-04-26T20:20:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:49 smithi119 ceph-mon[32752]: pgmap v4137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:49 smithi167 ceph-mon[32751]: pgmap v4137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:49 smithi090 ceph-mon[29180]: pgmap v4137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:50.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6735 sec 2024-04-26T20:20:51.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6736 sec 2024-04-26T20:20:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6332.071929 secs 2024-04-26T20:20:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:51 smithi119 ceph-mon[32752]: pgmap v4138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6332.071929 secs 2024-04-26T20:20:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:51 smithi167 ceph-mon[32751]: pgmap v4138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6332.071929 secs 2024-04-26T20:20:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:51 smithi090 ceph-mon[29180]: pgmap v4138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:52.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6737 sec 2024-04-26T20:20:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6332.773405 secs 2024-04-26T20:20:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6332.773405 secs 2024-04-26T20:20:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6332.773405 secs 2024-04-26T20:20:53.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6738 sec 2024-04-26T20:20:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:53 smithi119 ceph-mon[32752]: pgmap v4139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:53 smithi167 ceph-mon[32751]: pgmap v4139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:53 smithi090 ceph-mon[29180]: pgmap v4139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:20:54.492 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6739 sec 2024-04-26T20:20:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:20:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:20:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:20:55.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6740 sec 2024-04-26T20:20:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:55 smithi119 ceph-mon[32752]: pgmap v4140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:20:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:20:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:20:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:55 smithi167 ceph-mon[32751]: pgmap v4140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:20:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:20:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:20:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:55 smithi090 ceph-mon[29180]: pgmap v4140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:20:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:20:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:20:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:20:56.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6741 sec 2024-04-26T20:20:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:56 smithi119 ceph-mon[32752]: pgmap v4141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:20:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:56 smithi119 ceph-mon[32752]: pgmap v4142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6337.071993 secs 2024-04-26T20:20:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:56 smithi167 ceph-mon[32751]: pgmap v4141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:20:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:56 smithi167 ceph-mon[32751]: pgmap v4142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6337.071993 secs 2024-04-26T20:20:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:56 smithi090 ceph-mon[29180]: pgmap v4141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:20:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:56 smithi090 ceph-mon[29180]: pgmap v4142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6337.071993 secs 2024-04-26T20:20:57.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6742 sec 2024-04-26T20:20:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6337.773540 secs 2024-04-26T20:20:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6337.773540 secs 2024-04-26T20:20:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6337.773540 secs 2024-04-26T20:20:58.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:20:58 smithi090 ceph-mon[29180]: pgmap v4143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:58.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6743 sec 2024-04-26T20:20:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:20:58 smithi119 ceph-mon[32752]: pgmap v4143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:20:58 smithi167 ceph-mon[32751]: pgmap v4143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:20:59.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6744 sec 2024-04-26T20:21:00.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6745 sec 2024-04-26T20:21:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:00 smithi119 ceph-mon[32752]: pgmap v4144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:00 smithi167 ceph-mon[32751]: pgmap v4144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:00 smithi090 ceph-mon[29180]: pgmap v4144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6342.072096 secs 2024-04-26T20:21:01.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6746 sec 2024-04-26T20:21:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6342.072096 secs 2024-04-26T20:21:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6342.072096 secs 2024-04-26T20:21:02.493 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6747 sec 2024-04-26T20:21:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6342.773618 secs 2024-04-26T20:21:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:02 smithi119 ceph-mon[32752]: pgmap v4145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6342.773618 secs 2024-04-26T20:21:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:02 smithi167 ceph-mon[32751]: pgmap v4145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6342.773618 secs 2024-04-26T20:21:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:02 smithi090 ceph-mon[29180]: pgmap v4145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:03.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6748 sec 2024-04-26T20:21:04.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6749 sec 2024-04-26T20:21:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:04 smithi119 ceph-mon[32752]: pgmap v4146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:21:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:04 smithi167 ceph-mon[32751]: pgmap v4146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:21:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:04 smithi090 ceph-mon[29180]: pgmap v4146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:21:05.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6750 sec 2024-04-26T20:21:06.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6751 sec 2024-04-26T20:21:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:06 smithi119 ceph-mon[32752]: pgmap v4147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:21:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6347.072179 secs 2024-04-26T20:21:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:06 smithi167 ceph-mon[32751]: pgmap v4147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:21:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6347.072179 secs 2024-04-26T20:21:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:06 smithi090 ceph-mon[29180]: pgmap v4147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:21:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6347.072179 secs 2024-04-26T20:21:07.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6752 sec 2024-04-26T20:21:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6347.773739 secs 2024-04-26T20:21:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6347.773739 secs 2024-04-26T20:21:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6347.773739 secs 2024-04-26T20:21:08.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6753 sec 2024-04-26T20:21:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:08 smithi119 ceph-mon[32752]: pgmap v4148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:08 smithi167 ceph-mon[32751]: pgmap v4148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:08 smithi090 ceph-mon[29180]: pgmap v4148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:09.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6754 sec 2024-04-26T20:21:10.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6755 sec 2024-04-26T20:21:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:10 smithi119 ceph-mon[32752]: pgmap v4149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:10 smithi167 ceph-mon[32751]: pgmap v4149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:10 smithi090 ceph-mon[29180]: pgmap v4149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:11.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6756 sec 2024-04-26T20:21:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6352.072263 secs 2024-04-26T20:21:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6352.072263 secs 2024-04-26T20:21:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6352.072263 secs 2024-04-26T20:21:12.494 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6757 sec 2024-04-26T20:21:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6352.773829 secs 2024-04-26T20:21:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:12 smithi119 ceph-mon[32752]: pgmap v4150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6352.773829 secs 2024-04-26T20:21:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:12 smithi167 ceph-mon[32751]: pgmap v4150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6352.773829 secs 2024-04-26T20:21:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:12 smithi090 ceph-mon[29180]: pgmap v4150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:13.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6758 sec 2024-04-26T20:21:14.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6759 sec 2024-04-26T20:21:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:14 smithi119 ceph-mon[32752]: pgmap v4151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:14 smithi167 ceph-mon[32751]: pgmap v4151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:14 smithi090 ceph-mon[29180]: pgmap v4151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:15.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6760 sec 2024-04-26T20:21:16.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6761 sec 2024-04-26T20:21:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:16 smithi119 ceph-mon[32752]: pgmap v4152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6357.072341 secs 2024-04-26T20:21:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:16 smithi167 ceph-mon[32751]: pgmap v4152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6357.072341 secs 2024-04-26T20:21:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:16 smithi090 ceph-mon[29180]: pgmap v4152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6357.072341 secs 2024-04-26T20:21:17.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6762 sec 2024-04-26T20:21:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6357.773946 secs 2024-04-26T20:21:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6357.773946 secs 2024-04-26T20:21:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6357.773946 secs 2024-04-26T20:21:18.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6763 sec 2024-04-26T20:21:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:18 smithi119 ceph-mon[32752]: pgmap v4153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:18 smithi167 ceph-mon[32751]: pgmap v4153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:18 smithi090 ceph-mon[29180]: pgmap v4153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:19.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6764 sec 2024-04-26T20:21:20.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6765 sec 2024-04-26T20:21:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:20 smithi119 ceph-mon[32752]: pgmap v4154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:20 smithi167 ceph-mon[32751]: pgmap v4154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:20 smithi090 ceph-mon[29180]: pgmap v4154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:21.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6766 sec 2024-04-26T20:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6362.072427 secs 2024-04-26T20:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6362.072427 secs 2024-04-26T20:21:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6362.072427 secs 2024-04-26T20:21:22.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6767 sec 2024-04-26T20:21:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6362.774063 secs 2024-04-26T20:21:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:22 smithi119 ceph-mon[32752]: pgmap v4155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6362.774063 secs 2024-04-26T20:21:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:22 smithi167 ceph-mon[32751]: pgmap v4155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6362.774063 secs 2024-04-26T20:21:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:22 smithi090 ceph-mon[29180]: pgmap v4155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:23.495 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6768 sec 2024-04-26T20:21:24.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6769 sec 2024-04-26T20:21:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:24 smithi119 ceph-mon[32752]: pgmap v4156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:24 smithi167 ceph-mon[32751]: pgmap v4156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:24 smithi090 ceph-mon[29180]: pgmap v4156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:25.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6770 sec 2024-04-26T20:21:26.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6771 sec 2024-04-26T20:21:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:26 smithi119 ceph-mon[32752]: pgmap v4157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6367.072486 secs 2024-04-26T20:21:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:26 smithi167 ceph-mon[32751]: pgmap v4157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6367.072486 secs 2024-04-26T20:21:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:26 smithi090 ceph-mon[29180]: pgmap v4157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6367.072486 secs 2024-04-26T20:21:27.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6772 sec 2024-04-26T20:21:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6367.774180 secs 2024-04-26T20:21:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6367.774180 secs 2024-04-26T20:21:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6367.774180 secs 2024-04-26T20:21:28.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6773 sec 2024-04-26T20:21:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:28 smithi119 ceph-mon[32752]: pgmap v4158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:28 smithi167 ceph-mon[32751]: pgmap v4158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:28 smithi090 ceph-mon[29180]: pgmap v4158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:29.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6774 sec 2024-04-26T20:21:30.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6775 sec 2024-04-26T20:21:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:30 smithi119 ceph-mon[32752]: pgmap v4159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:30 smithi167 ceph-mon[32751]: pgmap v4159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:30 smithi090 ceph-mon[29180]: pgmap v4159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:31.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6776 sec 2024-04-26T20:21:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6372.072570 secs 2024-04-26T20:21:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6372.072570 secs 2024-04-26T20:21:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6372.072570 secs 2024-04-26T20:21:32.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6777 sec 2024-04-26T20:21:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6372.774315 secs 2024-04-26T20:21:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:32 smithi119 ceph-mon[32752]: pgmap v4160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6372.774315 secs 2024-04-26T20:21:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:32 smithi167 ceph-mon[32751]: pgmap v4160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6372.774315 secs 2024-04-26T20:21:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:32 smithi090 ceph-mon[29180]: pgmap v4160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:33.496 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6778 sec 2024-04-26T20:21:34.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6779 sec 2024-04-26T20:21:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:34 smithi119 ceph-mon[32752]: pgmap v4161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:34 smithi167 ceph-mon[32751]: pgmap v4161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:34 smithi090 ceph-mon[29180]: pgmap v4161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:35.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6780 sec 2024-04-26T20:21:36.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6781 sec 2024-04-26T20:21:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:36 smithi119 ceph-mon[32752]: pgmap v4162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6377.072631 secs 2024-04-26T20:21:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:36 smithi167 ceph-mon[32751]: pgmap v4162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6377.072631 secs 2024-04-26T20:21:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:36 smithi090 ceph-mon[29180]: pgmap v4162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6377.072631 secs 2024-04-26T20:21:37.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6782 sec 2024-04-26T20:21:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6377.774386 secs 2024-04-26T20:21:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6377.774386 secs 2024-04-26T20:21:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6377.774386 secs 2024-04-26T20:21:38.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6783 sec 2024-04-26T20:21:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:38 smithi119 ceph-mon[32752]: pgmap v4163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:38 smithi167 ceph-mon[32751]: pgmap v4163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:38 smithi090 ceph-mon[29180]: pgmap v4163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:39.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6784 sec 2024-04-26T20:21:40.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6785 sec 2024-04-26T20:21:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:40 smithi119 ceph-mon[32752]: pgmap v4164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:40 smithi167 ceph-mon[32751]: pgmap v4164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:40 smithi090 ceph-mon[29180]: pgmap v4164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:41.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6786 sec 2024-04-26T20:21:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:41 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 6382.072718 secs 2024-04-26T20:21:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:41 smithi119 ceph-mon[32752]: slow request 722.437426 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:21:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:41 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 6382.072718 secs 2024-04-26T20:21:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:41 smithi167 ceph-mon[32751]: slow request 722.437426 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:21:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:41 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 6382.072718 secs 2024-04-26T20:21:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:41 smithi090 ceph-mon[29180]: slow request 722.437426 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:21:42.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6787 sec 2024-04-26T20:21:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6382.774503 secs 2024-04-26T20:21:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:42 smithi119 ceph-mon[32752]: pgmap v4165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6382.774503 secs 2024-04-26T20:21:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:42 smithi167 ceph-mon[32751]: pgmap v4165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6382.774503 secs 2024-04-26T20:21:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:42 smithi090 ceph-mon[29180]: pgmap v4165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:43.497 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6788 sec 2024-04-26T20:21:44.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6789 sec 2024-04-26T20:21:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:44 smithi119 ceph-mon[32752]: pgmap v4166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:44 smithi167 ceph-mon[32751]: pgmap v4166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:44 smithi090 ceph-mon[29180]: pgmap v4166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:45.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6790 sec 2024-04-26T20:21:46.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6791 sec 2024-04-26T20:21:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:46 smithi119 ceph-mon[32752]: pgmap v4167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6387.072802 secs 2024-04-26T20:21:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:46 smithi167 ceph-mon[32751]: pgmap v4167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6387.072802 secs 2024-04-26T20:21:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:46 smithi090 ceph-mon[29180]: pgmap v4167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6387.072802 secs 2024-04-26T20:21:47.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6792 sec 2024-04-26T20:21:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6387.774569 secs 2024-04-26T20:21:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6387.774569 secs 2024-04-26T20:21:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6387.774569 secs 2024-04-26T20:21:48.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6793 sec 2024-04-26T20:21:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:48 smithi167 ceph-mon[32751]: pgmap v4168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:48 smithi090 ceph-mon[29180]: pgmap v4168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:48 smithi119 ceph-mon[32752]: pgmap v4168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:49.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6794 sec 2024-04-26T20:21:50.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6795 sec 2024-04-26T20:21:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:50 smithi167 ceph-mon[32751]: pgmap v4169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:50 smithi090 ceph-mon[29180]: pgmap v4169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:50 smithi119 ceph-mon[32752]: pgmap v4169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:51.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6796 sec 2024-04-26T20:21:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6392.072882 secs 2024-04-26T20:21:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6392.774664 secs 2024-04-26T20:21:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:51 smithi090 ceph-mon[29180]: pgmap v4170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6392.072882 secs 2024-04-26T20:21:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6392.774664 secs 2024-04-26T20:21:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:51 smithi119 ceph-mon[32752]: pgmap v4170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6392.072882 secs 2024-04-26T20:21:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6392.774664 secs 2024-04-26T20:21:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:51 smithi167 ceph-mon[32751]: pgmap v4170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:52.498 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6797 sec 2024-04-26T20:21:53.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6798 sec 2024-04-26T20:21:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:53 smithi119 ceph-mon[32752]: pgmap v4171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:53 smithi167 ceph-mon[32751]: pgmap v4171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:53 smithi090 ceph-mon[29180]: pgmap v4171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:21:54.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6799 sec 2024-04-26T20:21:55.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6800 sec 2024-04-26T20:21:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:55 smithi167 ceph-mon[32751]: pgmap v4172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6397.072943 secs 2024-04-26T20:21:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:21:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:21:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:21:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:21:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:55 smithi090 ceph-mon[29180]: pgmap v4172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6397.072943 secs 2024-04-26T20:21:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:21:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:21:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:21:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:21:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:55 smithi119 ceph-mon[32752]: pgmap v4172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:21:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6397.072943 secs 2024-04-26T20:21:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:21:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:21:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:21:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:21:56.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6801 sec 2024-04-26T20:21:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:56 smithi167 ceph-mon[32751]: pgmap v4173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:21:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:56 smithi167 ceph-mon[32751]: pgmap v4174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6397.774731 secs 2024-04-26T20:21:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:56 smithi090 ceph-mon[29180]: pgmap v4173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:21:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:56 smithi090 ceph-mon[29180]: pgmap v4174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6397.774731 secs 2024-04-26T20:21:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:56 smithi119 ceph-mon[32752]: pgmap v4173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:21:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:56 smithi119 ceph-mon[32752]: pgmap v4174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:21:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6397.774731 secs 2024-04-26T20:21:57.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6802 sec 2024-04-26T20:21:58.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6803 sec 2024-04-26T20:21:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:21:58 smithi167 ceph-mon[32751]: pgmap v4175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:21:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:21:58 smithi090 ceph-mon[29180]: pgmap v4175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:21:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:21:58 smithi119 ceph-mon[32752]: pgmap v4175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:21:59.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6804 sec 2024-04-26T20:22:00.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6805 sec 2024-04-26T20:22:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:00 smithi167 ceph-mon[32751]: pgmap v4176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:00 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6402.073034 secs 2024-04-26T20:22:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:00 smithi090 ceph-mon[29180]: pgmap v4176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:00 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6402.073034 secs 2024-04-26T20:22:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:00 smithi119 ceph-mon[32752]: pgmap v4176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:00 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6402.073034 secs 2024-04-26T20:22:01.499 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6806 sec 2024-04-26T20:22:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6402.774825 secs 2024-04-26T20:22:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6402.774825 secs 2024-04-26T20:22:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6402.774825 secs 2024-04-26T20:22:02.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6807 sec 2024-04-26T20:22:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:02 smithi090 ceph-mon[29180]: pgmap v4177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:02 smithi119 ceph-mon[32752]: pgmap v4177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:02 smithi167 ceph-mon[32751]: pgmap v4177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:03.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6808 sec 2024-04-26T20:22:04.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6809 sec 2024-04-26T20:22:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:04 smithi090 ceph-mon[29180]: pgmap v4178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:22:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:04 smithi119 ceph-mon[32752]: pgmap v4178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:22:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:04 smithi167 ceph-mon[32751]: pgmap v4178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:22:05.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6810 sec 2024-04-26T20:22:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:05 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6407.073089 secs 2024-04-26T20:22:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:05 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6407.073089 secs 2024-04-26T20:22:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:05 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6407.073089 secs 2024-04-26T20:22:06.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6811 sec 2024-04-26T20:22:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:06 smithi090 ceph-mon[29180]: pgmap v4179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-26T20:22:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6407.774944 secs 2024-04-26T20:22:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:06 smithi119 ceph-mon[32752]: pgmap v4179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-26T20:22:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6407.774944 secs 2024-04-26T20:22:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:06 smithi167 ceph-mon[32751]: pgmap v4179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-26T20:22:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6407.774944 secs 2024-04-26T20:22:07.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6812 sec 2024-04-26T20:22:08.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6813 sec 2024-04-26T20:22:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:08 smithi090 ceph-mon[29180]: pgmap v4180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:08 smithi119 ceph-mon[32752]: pgmap v4180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:08 smithi167 ceph-mon[32751]: pgmap v4180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:09.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6814 sec 2024-04-26T20:22:10.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6815 sec 2024-04-26T20:22:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:10 smithi090 ceph-mon[29180]: pgmap v4181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:10 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6412.073169 secs 2024-04-26T20:22:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:10 smithi119 ceph-mon[32752]: pgmap v4181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:10 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6412.073169 secs 2024-04-26T20:22:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:10 smithi167 ceph-mon[32751]: pgmap v4181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:10 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6412.073169 secs 2024-04-26T20:22:11.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6816 sec 2024-04-26T20:22:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6412.775065 secs 2024-04-26T20:22:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6412.775065 secs 2024-04-26T20:22:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6412.775065 secs 2024-04-26T20:22:12.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6817 sec 2024-04-26T20:22:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:12 smithi090 ceph-mon[29180]: pgmap v4182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:12 smithi119 ceph-mon[32752]: pgmap v4182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:12 smithi167 ceph-mon[32751]: pgmap v4182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:13.500 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6818 sec 2024-04-26T20:22:14.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6819 sec 2024-04-26T20:22:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:14 smithi090 ceph-mon[29180]: pgmap v4183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:14 smithi119 ceph-mon[32752]: pgmap v4183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:14 smithi167 ceph-mon[32751]: pgmap v4183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:22:15.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6820 sec 2024-04-26T20:22:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:15 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6417.073227 secs 2024-04-26T20:22:16.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:15 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6417.073227 secs 2024-04-26T20:22:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:15 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6417.073227 secs 2024-04-26T20:22:16.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6821 sec 2024-04-26T20:22:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:16 smithi090 ceph-mon[29180]: pgmap v4184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6417.775186 secs 2024-04-26T20:22:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:16 smithi119 ceph-mon[32752]: pgmap v4184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6417.775186 secs 2024-04-26T20:22:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:16 smithi167 ceph-mon[32751]: pgmap v4184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6417.775186 secs 2024-04-26T20:22:17.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6822 sec 2024-04-26T20:22:18.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6823 sec 2024-04-26T20:22:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:18 smithi090 ceph-mon[29180]: pgmap v4185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:18 smithi119 ceph-mon[32752]: pgmap v4185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:19.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:18 smithi167 ceph-mon[32751]: pgmap v4185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:19.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6824 sec 2024-04-26T20:22:20.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6825 sec 2024-04-26T20:22:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:20 smithi090 ceph-mon[29180]: pgmap v4186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:20 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6422.073309 secs 2024-04-26T20:22:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:20 smithi119 ceph-mon[32752]: pgmap v4186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:20 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6422.073309 secs 2024-04-26T20:22:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:20 smithi167 ceph-mon[32751]: pgmap v4186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:20 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6422.073309 secs 2024-04-26T20:22:21.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6826 sec 2024-04-26T20:22:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6422.775266 secs 2024-04-26T20:22:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6422.775266 secs 2024-04-26T20:22:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6422.775266 secs 2024-04-26T20:22:22.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6827 sec 2024-04-26T20:22:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:22 smithi090 ceph-mon[29180]: pgmap v4187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:22 smithi119 ceph-mon[32752]: pgmap v4187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:22 smithi167 ceph-mon[32751]: pgmap v4187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:23.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6828 sec 2024-04-26T20:22:24.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6829 sec 2024-04-26T20:22:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:24 smithi090 ceph-mon[29180]: pgmap v4188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:24 smithi119 ceph-mon[32752]: pgmap v4188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:24 smithi167 ceph-mon[32751]: pgmap v4188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:25.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6830 sec 2024-04-26T20:22:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:25 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6427.073390 secs 2024-04-26T20:22:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:25 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6427.073390 secs 2024-04-26T20:22:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:25 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6427.073390 secs 2024-04-26T20:22:26.501 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6831 sec 2024-04-26T20:22:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:26 smithi119 ceph-mon[32752]: pgmap v4189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6427.775338 secs 2024-04-26T20:22:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:26 smithi167 ceph-mon[32751]: pgmap v4189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6427.775338 secs 2024-04-26T20:22:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:26 smithi090 ceph-mon[29180]: pgmap v4189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6427.775338 secs 2024-04-26T20:22:27.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6832 sec 2024-04-26T20:22:28.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6833 sec 2024-04-26T20:22:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:28 smithi119 ceph-mon[32752]: pgmap v4190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:28 smithi167 ceph-mon[32751]: pgmap v4190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:28 smithi090 ceph-mon[29180]: pgmap v4190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:29.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6834 sec 2024-04-26T20:22:30.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6835 sec 2024-04-26T20:22:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:30 smithi119 ceph-mon[32752]: pgmap v4191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:30 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6432.073448 secs 2024-04-26T20:22:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:30 smithi167 ceph-mon[32751]: pgmap v4191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:30 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6432.073448 secs 2024-04-26T20:22:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:30 smithi090 ceph-mon[29180]: pgmap v4191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:30 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6432.073448 secs 2024-04-26T20:22:31.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6836 sec 2024-04-26T20:22:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6432.775457 secs 2024-04-26T20:22:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6432.775457 secs 2024-04-26T20:22:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6432.775457 secs 2024-04-26T20:22:32.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6837 sec 2024-04-26T20:22:32.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:22:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:22:32.471+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:32.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:22:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:22:32.471+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:32.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:22:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:22:32.471+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:32.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:22:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:22:32.471+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:32.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:22:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:22:32.472+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:22:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:32 smithi119 ceph-mon[32752]: pgmap v4192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:32 smithi167 ceph-mon[32751]: pgmap v4192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:32 smithi090 ceph-mon[29180]: pgmap v4192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:33.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6838 sec 2024-04-26T20:22:34.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6839 sec 2024-04-26T20:22:35.502 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6840 sec 2024-04-26T20:22:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:35 smithi119 ceph-mon[32752]: pgmap v4193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:35 smithi167 ceph-mon[32751]: pgmap v4193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:35 smithi090 ceph-mon[29180]: pgmap v4193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:36.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6841 sec 2024-04-26T20:22:36.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:36 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 6437.073526 secs 2024-04-26T20:22:36.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:36 smithi090 ceph-mon[29180]: slow request 5763.890911 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:22:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:36 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 6437.073526 secs 2024-04-26T20:22:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:36 smithi119 ceph-mon[32752]: slow request 5763.890911 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:22:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:36 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 6437.073526 secs 2024-04-26T20:22:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:36 smithi167 ceph-mon[32751]: slow request 5763.890911 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:22:37.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6842 sec 2024-04-26T20:22:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:37 smithi119 ceph-mon[32752]: pgmap v4194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6437.775532 secs 2024-04-26T20:22:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:37 smithi167 ceph-mon[32751]: pgmap v4194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6437.775532 secs 2024-04-26T20:22:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:37 smithi090 ceph-mon[29180]: pgmap v4194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6437.775532 secs 2024-04-26T20:22:38.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6843 sec 2024-04-26T20:22:39.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6844 sec 2024-04-26T20:22:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:39 smithi119 ceph-mon[32752]: pgmap v4195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:39 smithi167 ceph-mon[32751]: pgmap v4195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:39 smithi090 ceph-mon[29180]: pgmap v4195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:40.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6845 sec 2024-04-26T20:22:41.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6846 sec 2024-04-26T20:22:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:41 smithi119 ceph-mon[32752]: pgmap v4196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6442.073608 secs 2024-04-26T20:22:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:41 smithi167 ceph-mon[32751]: pgmap v4196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6442.073608 secs 2024-04-26T20:22:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:41 smithi090 ceph-mon[29180]: pgmap v4196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6442.073608 secs 2024-04-26T20:22:42.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6847 sec 2024-04-26T20:22:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6442.775608 secs 2024-04-26T20:22:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6442.775608 secs 2024-04-26T20:22:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6442.775608 secs 2024-04-26T20:22:43.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6848 sec 2024-04-26T20:22:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:43 smithi119 ceph-mon[32752]: pgmap v4197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:43 smithi167 ceph-mon[32751]: pgmap v4197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:43 smithi090 ceph-mon[29180]: pgmap v4197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:22:44.318+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:22:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:22:44.318+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006350 con 0x7f2cd4104500 2024-04-26T20:22:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:22:44.318+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 53 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:22:44.319 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:22:44.318+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 53 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8006350 con 0x7f2cd4104500 2024-04-26T20:22:44.503 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6849 sec 2024-04-26T20:22:45.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6850 sec 2024-04-26T20:22:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:45 smithi119 ceph-mon[32752]: pgmap v4198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:45 smithi167 ceph-mon[32751]: pgmap v4198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:45 smithi090 ceph-mon[29180]: pgmap v4198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6447.073650 secs 2024-04-26T20:22:46.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6851 sec 2024-04-26T20:22:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6447.073650 secs 2024-04-26T20:22:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6447.073650 secs 2024-04-26T20:22:47.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6852 sec 2024-04-26T20:22:47.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:47 smithi090 ceph-mon[29180]: pgmap v4199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:47.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6447.775698 secs 2024-04-26T20:22:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:47 smithi119 ceph-mon[32752]: pgmap v4199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6447.775698 secs 2024-04-26T20:22:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:47 smithi167 ceph-mon[32751]: pgmap v4199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6447.775698 secs 2024-04-26T20:22:48.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6853 sec 2024-04-26T20:22:49.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6854 sec 2024-04-26T20:22:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:49 smithi119 ceph-mon[32752]: pgmap v4200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:49 smithi167 ceph-mon[32751]: pgmap v4200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:49 smithi090 ceph-mon[29180]: pgmap v4200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:50.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6855 sec 2024-04-26T20:22:51.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6856 sec 2024-04-26T20:22:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:51 smithi119 ceph-mon[32752]: pgmap v4201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6452.073735 secs 2024-04-26T20:22:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:51 smithi167 ceph-mon[32751]: pgmap v4201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6452.073735 secs 2024-04-26T20:22:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:51 smithi090 ceph-mon[29180]: pgmap v4201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6452.073735 secs 2024-04-26T20:22:52.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6857 sec 2024-04-26T20:22:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6452.775816 secs 2024-04-26T20:22:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6452.775816 secs 2024-04-26T20:22:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6452.775816 secs 2024-04-26T20:22:53.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6858 sec 2024-04-26T20:22:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:53 smithi119 ceph-mon[32752]: pgmap v4202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:53 smithi167 ceph-mon[32751]: pgmap v4202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:53 smithi090 ceph-mon[29180]: pgmap v4202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:54.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6859 sec 2024-04-26T20:22:55.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6860 sec 2024-04-26T20:22:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:55 smithi119 ceph-mon[32752]: pgmap v4203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:55 smithi167 ceph-mon[32751]: pgmap v4203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:55 smithi090 ceph-mon[29180]: pgmap v4203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:22:56.504 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6861 sec 2024-04-26T20:22:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6457.073834 secs 2024-04-26T20:22:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6457.073834 secs 2024-04-26T20:22:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6457.073834 secs 2024-04-26T20:22:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:22:57.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6862 sec 2024-04-26T20:22:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6457.775958 secs 2024-04-26T20:22:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:57 smithi119 ceph-mon[32752]: pgmap v4204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:57 smithi119 ceph-mon[32752]: pgmap v4205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:22:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:57 smithi119 ceph-mon[32752]: pgmap v4206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:22:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6457.775958 secs 2024-04-26T20:22:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:57 smithi167 ceph-mon[32751]: pgmap v4204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:57 smithi167 ceph-mon[32751]: pgmap v4205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:22:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:57 smithi167 ceph-mon[32751]: pgmap v4206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:22:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6457.775958 secs 2024-04-26T20:22:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:57 smithi090 ceph-mon[29180]: pgmap v4204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:22:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:22:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:22:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:57 smithi090 ceph-mon[29180]: pgmap v4205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:22:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:57 smithi090 ceph-mon[29180]: pgmap v4206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:22:58.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6863 sec 2024-04-26T20:22:58.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:22:58 smithi090 ceph-mon[29180]: pgmap v4207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:22:58 smithi119 ceph-mon[32752]: pgmap v4207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:22:58 smithi167 ceph-mon[32751]: pgmap v4207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:22:59.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6864 sec 2024-04-26T20:23:00.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6865 sec 2024-04-26T20:23:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:00 smithi090 ceph-mon[29180]: pgmap v4208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:23:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:00 smithi119 ceph-mon[32752]: pgmap v4208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:23:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:00 smithi167 ceph-mon[32751]: pgmap v4208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:23:01.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6866 sec 2024-04-26T20:23:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6462.073921 secs 2024-04-26T20:23:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6462.776084 secs 2024-04-26T20:23:01.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6462.073921 secs 2024-04-26T20:23:01.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6462.776084 secs 2024-04-26T20:23:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6462.073921 secs 2024-04-26T20:23:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6462.776084 secs 2024-04-26T20:23:02.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6867 sec 2024-04-26T20:23:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:02 smithi119 ceph-mon[32752]: pgmap v4209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:23:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:02 smithi167 ceph-mon[32751]: pgmap v4209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:23:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:02 smithi090 ceph-mon[29180]: pgmap v4209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:23:03.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6868 sec 2024-04-26T20:23:04.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6869 sec 2024-04-26T20:23:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:04 smithi090 ceph-mon[29180]: pgmap v4210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:23:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:04 smithi119 ceph-mon[32752]: pgmap v4210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:23:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:04 smithi167 ceph-mon[32751]: pgmap v4210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:23:05.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6870 sec 2024-04-26T20:23:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:05 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6467.074004 secs 2024-04-26T20:23:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:05 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6467.074004 secs 2024-04-26T20:23:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:05 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6467.074004 secs 2024-04-26T20:23:06.505 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6871 sec 2024-04-26T20:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6467.776203 secs 2024-04-26T20:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:06 smithi119 ceph-mon[32752]: pgmap v4211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6467.776203 secs 2024-04-26T20:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:06 smithi167 ceph-mon[32751]: pgmap v4211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:23:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6467.776203 secs 2024-04-26T20:23:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:06 smithi090 ceph-mon[29180]: pgmap v4211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:23:07.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6872 sec 2024-04-26T20:23:08.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6873 sec 2024-04-26T20:23:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:08 smithi090 ceph-mon[29180]: pgmap v4212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:08 smithi119 ceph-mon[32752]: pgmap v4212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:08 smithi167 ceph-mon[32751]: pgmap v4212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:09.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6874 sec 2024-04-26T20:23:10.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6875 sec 2024-04-26T20:23:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:10 smithi090 ceph-mon[29180]: pgmap v4213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:10 smithi119 ceph-mon[32752]: pgmap v4213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:10 smithi167 ceph-mon[32751]: pgmap v4213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:11.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6876 sec 2024-04-26T20:23:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6472.074106 secs 2024-04-26T20:23:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6472.776270 secs 2024-04-26T20:23:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6472.074106 secs 2024-04-26T20:23:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6472.776270 secs 2024-04-26T20:23:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6472.074106 secs 2024-04-26T20:23:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6472.776270 secs 2024-04-26T20:23:12.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6877 sec 2024-04-26T20:23:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:12 smithi119 ceph-mon[32752]: pgmap v4214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:12 smithi167 ceph-mon[32751]: pgmap v4214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:12 smithi090 ceph-mon[29180]: pgmap v4214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:13.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6878 sec 2024-04-26T20:23:14.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6879 sec 2024-04-26T20:23:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:14 smithi090 ceph-mon[29180]: pgmap v4215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:14 smithi119 ceph-mon[32752]: pgmap v4215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:14 smithi167 ceph-mon[32751]: pgmap v4215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:15.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6880 sec 2024-04-26T20:23:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:15 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6477.074187 secs 2024-04-26T20:23:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:15 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6477.074187 secs 2024-04-26T20:23:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:15 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6477.074187 secs 2024-04-26T20:23:16.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6881 sec 2024-04-26T20:23:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6477.776406 secs 2024-04-26T20:23:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:16 smithi119 ceph-mon[32752]: pgmap v4216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6477.776406 secs 2024-04-26T20:23:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:16 smithi167 ceph-mon[32751]: pgmap v4216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6477.776406 secs 2024-04-26T20:23:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:16 smithi090 ceph-mon[29180]: pgmap v4216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:17.506 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6882 sec 2024-04-26T20:23:18.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6883 sec 2024-04-26T20:23:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:18 smithi090 ceph-mon[29180]: pgmap v4217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:18 smithi119 ceph-mon[32752]: pgmap v4217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:18 smithi167 ceph-mon[32751]: pgmap v4217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:19.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6884 sec 2024-04-26T20:23:20.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6885 sec 2024-04-26T20:23:20.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:20 smithi119 ceph-mon[32752]: pgmap v4218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:20 smithi167 ceph-mon[32751]: pgmap v4218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:20 smithi090 ceph-mon[29180]: pgmap v4218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:21.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6886 sec 2024-04-26T20:23:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6482.074273 secs 2024-04-26T20:23:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6482.776526 secs 2024-04-26T20:23:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6482.074273 secs 2024-04-26T20:23:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6482.776526 secs 2024-04-26T20:23:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6482.074273 secs 2024-04-26T20:23:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6482.776526 secs 2024-04-26T20:23:22.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6887 sec 2024-04-26T20:23:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:22 smithi119 ceph-mon[32752]: pgmap v4219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:22 smithi167 ceph-mon[32751]: pgmap v4219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:22 smithi090 ceph-mon[29180]: pgmap v4219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:23.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6888 sec 2024-04-26T20:23:24.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6889 sec 2024-04-26T20:23:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:24 smithi119 ceph-mon[32752]: pgmap v4220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:24 smithi167 ceph-mon[32751]: pgmap v4220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:24 smithi090 ceph-mon[29180]: pgmap v4220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:25.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6890 sec 2024-04-26T20:23:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:25 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6487.074357 secs 2024-04-26T20:23:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:25 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6487.074357 secs 2024-04-26T20:23:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:25 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6487.074357 secs 2024-04-26T20:23:26.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6891 sec 2024-04-26T20:23:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6487.776613 secs 2024-04-26T20:23:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:26 smithi119 ceph-mon[32752]: pgmap v4221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6487.776613 secs 2024-04-26T20:23:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:26 smithi167 ceph-mon[32751]: pgmap v4221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6487.776613 secs 2024-04-26T20:23:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:26 smithi090 ceph-mon[29180]: pgmap v4221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:27.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6892 sec 2024-04-26T20:23:28.507 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6893 sec 2024-04-26T20:23:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:28 smithi119 ceph-mon[32752]: pgmap v4222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:28 smithi167 ceph-mon[32751]: pgmap v4222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:28 smithi090 ceph-mon[29180]: pgmap v4222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:23:29.324+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:23:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:23:29.324+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:23:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:23:29.324+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 54 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:23:29.325 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:23:29.325+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 54 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:23:29.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6894 sec 2024-04-26T20:23:30.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6895 sec 2024-04-26T20:23:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:30 smithi119 ceph-mon[32752]: pgmap v4223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:30 smithi167 ceph-mon[32751]: pgmap v4223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:30 smithi090 ceph-mon[29180]: pgmap v4223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:31.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6896 sec 2024-04-26T20:23:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6492.074440 secs 2024-04-26T20:23:31.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6492.776742 secs 2024-04-26T20:23:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6492.074440 secs 2024-04-26T20:23:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6492.776742 secs 2024-04-26T20:23:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6492.074440 secs 2024-04-26T20:23:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6492.776742 secs 2024-04-26T20:23:32.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6897 sec 2024-04-26T20:23:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:32 smithi119 ceph-mon[32752]: pgmap v4224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:32 smithi167 ceph-mon[32751]: pgmap v4224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:32 smithi090 ceph-mon[29180]: pgmap v4224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:33.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6898 sec 2024-04-26T20:23:34.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6899 sec 2024-04-26T20:23:34.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:34 smithi090 ceph-mon[29180]: pgmap v4225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:34 smithi119 ceph-mon[32752]: pgmap v4225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:34 smithi167 ceph-mon[32751]: pgmap v4225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:35.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6900 sec 2024-04-26T20:23:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:35 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6497.074523 secs 2024-04-26T20:23:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:35 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6497.074523 secs 2024-04-26T20:23:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:35 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6497.074523 secs 2024-04-26T20:23:36.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6901 sec 2024-04-26T20:23:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6497.776856 secs 2024-04-26T20:23:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:36 smithi119 ceph-mon[32752]: pgmap v4226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6497.776856 secs 2024-04-26T20:23:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:36 smithi167 ceph-mon[32751]: pgmap v4226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6497.776856 secs 2024-04-26T20:23:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:36 smithi090 ceph-mon[29180]: pgmap v4226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:37.508 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6902 sec 2024-04-26T20:23:38.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6903 sec 2024-04-26T20:23:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:38 smithi119 ceph-mon[32752]: pgmap v4227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:38 smithi167 ceph-mon[32751]: pgmap v4227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:38 smithi090 ceph-mon[29180]: pgmap v4227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:39.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6904 sec 2024-04-26T20:23:40.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6905 sec 2024-04-26T20:23:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:40 smithi119 ceph-mon[32752]: pgmap v4228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:40 smithi167 ceph-mon[32751]: pgmap v4228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:40 smithi090 ceph-mon[29180]: pgmap v4228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:41.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6906 sec 2024-04-26T20:23:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6502.074609 secs 2024-04-26T20:23:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6502.776954 secs 2024-04-26T20:23:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6502.074609 secs 2024-04-26T20:23:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6502.776954 secs 2024-04-26T20:23:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6502.074609 secs 2024-04-26T20:23:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6502.776954 secs 2024-04-26T20:23:42.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6907 sec 2024-04-26T20:23:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:42 smithi119 ceph-mon[32752]: pgmap v4229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:42 smithi167 ceph-mon[32751]: pgmap v4229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:42 smithi090 ceph-mon[29180]: pgmap v4229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:43.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6908 sec 2024-04-26T20:23:44.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6909 sec 2024-04-26T20:23:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:44 smithi119 ceph-mon[32752]: pgmap v4230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:44 smithi167 ceph-mon[32751]: pgmap v4230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:44 smithi090 ceph-mon[29180]: pgmap v4230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:45.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6910 sec 2024-04-26T20:23:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:45 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6507.074694 secs 2024-04-26T20:23:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:45 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6507.074694 secs 2024-04-26T20:23:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:45 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6507.074694 secs 2024-04-26T20:23:46.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6911 sec 2024-04-26T20:23:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6507.777023 secs 2024-04-26T20:23:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:46 smithi119 ceph-mon[32752]: pgmap v4231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6507.777023 secs 2024-04-26T20:23:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:46 smithi167 ceph-mon[32751]: pgmap v4231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6507.777023 secs 2024-04-26T20:23:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:46 smithi090 ceph-mon[29180]: pgmap v4231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:47.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6912 sec 2024-04-26T20:23:48.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6913 sec 2024-04-26T20:23:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:48 smithi119 ceph-mon[32752]: pgmap v4232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:48 smithi167 ceph-mon[32751]: pgmap v4232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:48 smithi090 ceph-mon[29180]: pgmap v4232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:49.509 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6914 sec 2024-04-26T20:23:50.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6915 sec 2024-04-26T20:23:50.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:50 smithi090 ceph-mon[29180]: pgmap v4233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:50 smithi119 ceph-mon[32752]: pgmap v4233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:50 smithi167 ceph-mon[32751]: pgmap v4233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:51.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6916 sec 2024-04-26T20:23:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6512.074770 secs 2024-04-26T20:23:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6512.777128 secs 2024-04-26T20:23:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6512.074770 secs 2024-04-26T20:23:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6512.777128 secs 2024-04-26T20:23:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6512.074770 secs 2024-04-26T20:23:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6512.777128 secs 2024-04-26T20:23:52.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6917 sec 2024-04-26T20:23:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:52 smithi119 ceph-mon[32752]: pgmap v4234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:52 smithi167 ceph-mon[32751]: pgmap v4234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:52 smithi090 ceph-mon[29180]: pgmap v4234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:53.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6918 sec 2024-04-26T20:23:54.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6919 sec 2024-04-26T20:23:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:54 smithi119 ceph-mon[32752]: pgmap v4235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:54 smithi167 ceph-mon[32751]: pgmap v4235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:54 smithi090 ceph-mon[29180]: pgmap v4235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:23:55.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6920 sec 2024-04-26T20:23:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6517.074834 secs 2024-04-26T20:23:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6517.074834 secs 2024-04-26T20:23:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6517.074834 secs 2024-04-26T20:23:56.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6921 sec 2024-04-26T20:23:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6517.777249 secs 2024-04-26T20:23:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:56 smithi119 ceph-mon[32752]: pgmap v4236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:23:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6517.777249 secs 2024-04-26T20:23:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:56 smithi167 ceph-mon[32751]: pgmap v4236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:23:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6517.777249 secs 2024-04-26T20:23:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:56 smithi090 ceph-mon[29180]: pgmap v4236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:23:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:23:57.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6922 sec 2024-04-26T20:23:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:23:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:23:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:57 smithi119 ceph-mon[32752]: pgmap v4237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:23:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:57 smithi119 ceph-mon[32752]: pgmap v4238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:23:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:23:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:23:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:23:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:23:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:57 smithi167 ceph-mon[32751]: pgmap v4237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:23:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:57 smithi167 ceph-mon[32751]: pgmap v4238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:23:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:23:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:23:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:23:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:23:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:57 smithi090 ceph-mon[29180]: pgmap v4237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:23:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:57 smithi090 ceph-mon[29180]: pgmap v4238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:23:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:23:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:23:58.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6923 sec 2024-04-26T20:23:59.510 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6924 sec 2024-04-26T20:24:00.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6925 sec 2024-04-26T20:24:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:00 smithi119 ceph-mon[32752]: pgmap v4239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:00 smithi167 ceph-mon[32751]: pgmap v4239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:00 smithi090 ceph-mon[29180]: pgmap v4239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:01.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6522.074896 secs 2024-04-26T20:24:01.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6926 sec 2024-04-26T20:24:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6522.074896 secs 2024-04-26T20:24:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6522.074896 secs 2024-04-26T20:24:02.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6927 sec 2024-04-26T20:24:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6522.777419 secs 2024-04-26T20:24:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:02 smithi119 ceph-mon[32752]: pgmap v4240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6522.777419 secs 2024-04-26T20:24:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:02 smithi167 ceph-mon[32751]: pgmap v4240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6522.777419 secs 2024-04-26T20:24:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:02 smithi090 ceph-mon[29180]: pgmap v4240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:03.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6928 sec 2024-04-26T20:24:04.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6929 sec 2024-04-26T20:24:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:04 smithi119 ceph-mon[32752]: pgmap v4241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:24:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:04 smithi167 ceph-mon[32751]: pgmap v4241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:24:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:04 smithi090 ceph-mon[29180]: pgmap v4241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:24:05.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6930 sec 2024-04-26T20:24:06.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6931 sec 2024-04-26T20:24:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6527.074973 secs 2024-04-26T20:24:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:06 smithi119 ceph-mon[32752]: pgmap v4242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6527.074973 secs 2024-04-26T20:24:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:06 smithi167 ceph-mon[32751]: pgmap v4242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6527.074973 secs 2024-04-26T20:24:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:06 smithi090 ceph-mon[29180]: pgmap v4242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:24:07.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6932 sec 2024-04-26T20:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6527.777482 secs 2024-04-26T20:24:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6527.777482 secs 2024-04-26T20:24:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6527.777482 secs 2024-04-26T20:24:08.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6933 sec 2024-04-26T20:24:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:08 smithi119 ceph-mon[32752]: pgmap v4243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:24:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:08 smithi167 ceph-mon[32751]: pgmap v4243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:24:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:08 smithi090 ceph-mon[29180]: pgmap v4243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:24:09.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6934 sec 2024-04-26T20:24:10.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6935 sec 2024-04-26T20:24:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:10 smithi119 ceph-mon[32752]: pgmap v4244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:10 smithi167 ceph-mon[32751]: pgmap v4244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:10 smithi090 ceph-mon[29180]: pgmap v4244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:11.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6936 sec 2024-04-26T20:24:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6532.075056 secs 2024-04-26T20:24:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6532.075056 secs 2024-04-26T20:24:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6532.075056 secs 2024-04-26T20:24:12.511 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6937 sec 2024-04-26T20:24:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6532.777551 secs 2024-04-26T20:24:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:12 smithi119 ceph-mon[32752]: pgmap v4245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6532.777551 secs 2024-04-26T20:24:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:12 smithi167 ceph-mon[32751]: pgmap v4245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6532.777551 secs 2024-04-26T20:24:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:12 smithi090 ceph-mon[29180]: pgmap v4245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:13.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6938 sec 2024-04-26T20:24:14.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6939 sec 2024-04-26T20:24:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:14 smithi119 ceph-mon[32752]: pgmap v4246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:14 smithi167 ceph-mon[32751]: pgmap v4246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:14 smithi090 ceph-mon[29180]: pgmap v4246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:15.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6940 sec 2024-04-26T20:24:16.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6941 sec 2024-04-26T20:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6537.075115 secs 2024-04-26T20:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:16 smithi119 ceph-mon[32752]: pgmap v4247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6537.075115 secs 2024-04-26T20:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:16 smithi167 ceph-mon[32751]: pgmap v4247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6537.075115 secs 2024-04-26T20:24:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:16 smithi090 ceph-mon[29180]: pgmap v4247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:17.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6942 sec 2024-04-26T20:24:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6537.777668 secs 2024-04-26T20:24:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6537.777668 secs 2024-04-26T20:24:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6537.777668 secs 2024-04-26T20:24:18.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6943 sec 2024-04-26T20:24:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:18 smithi119 ceph-mon[32752]: pgmap v4248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:18 smithi167 ceph-mon[32751]: pgmap v4248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:18 smithi090 ceph-mon[29180]: pgmap v4248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:19.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6944 sec 2024-04-26T20:24:20.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6945 sec 2024-04-26T20:24:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:20 smithi119 ceph-mon[32752]: pgmap v4249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:20 smithi167 ceph-mon[32751]: pgmap v4249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:20 smithi090 ceph-mon[29180]: pgmap v4249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:21.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6946 sec 2024-04-26T20:24:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6542.075198 secs 2024-04-26T20:24:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6542.075198 secs 2024-04-26T20:24:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6542.075198 secs 2024-04-26T20:24:22.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6947 sec 2024-04-26T20:24:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6542.777779 secs 2024-04-26T20:24:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:22 smithi119 ceph-mon[32752]: pgmap v4250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6542.777779 secs 2024-04-26T20:24:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:22 smithi167 ceph-mon[32751]: pgmap v4250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6542.777779 secs 2024-04-26T20:24:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:22 smithi090 ceph-mon[29180]: pgmap v4250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:23.512 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6948 sec 2024-04-26T20:24:24.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6949 sec 2024-04-26T20:24:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:24 smithi119 ceph-mon[32752]: pgmap v4251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:24 smithi167 ceph-mon[32751]: pgmap v4251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:24 smithi090 ceph-mon[29180]: pgmap v4251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:25.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6950 sec 2024-04-26T20:24:26.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6951 sec 2024-04-26T20:24:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6547.075283 secs 2024-04-26T20:24:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:26 smithi119 ceph-mon[32752]: pgmap v4252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6547.075283 secs 2024-04-26T20:24:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:26 smithi167 ceph-mon[32751]: pgmap v4252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6547.075283 secs 2024-04-26T20:24:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:26 smithi090 ceph-mon[29180]: pgmap v4252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:27.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6952 sec 2024-04-26T20:24:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6547.777851 secs 2024-04-26T20:24:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6547.777851 secs 2024-04-26T20:24:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6547.777851 secs 2024-04-26T20:24:28.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6953 sec 2024-04-26T20:24:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:28 smithi119 ceph-mon[32752]: pgmap v4253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:28 smithi167 ceph-mon[32751]: pgmap v4253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:28 smithi090 ceph-mon[29180]: pgmap v4253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:29.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6954 sec 2024-04-26T20:24:30.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6955 sec 2024-04-26T20:24:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:30 smithi119 ceph-mon[32752]: pgmap v4254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:30 smithi167 ceph-mon[32751]: pgmap v4254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:30 smithi090 ceph-mon[29180]: pgmap v4254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6552.075367 secs 2024-04-26T20:24:31.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6956 sec 2024-04-26T20:24:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6552.075367 secs 2024-04-26T20:24:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6552.075367 secs 2024-04-26T20:24:32.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6957 sec 2024-04-26T20:24:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6552.777991 secs 2024-04-26T20:24:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:32 smithi119 ceph-mon[32752]: pgmap v4255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6552.777991 secs 2024-04-26T20:24:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:32 smithi167 ceph-mon[32751]: pgmap v4255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6552.777991 secs 2024-04-26T20:24:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:32 smithi090 ceph-mon[29180]: pgmap v4255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:33.513 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6958 sec 2024-04-26T20:24:34.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6959 sec 2024-04-26T20:24:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:34 smithi119 ceph-mon[32752]: pgmap v4256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:34 smithi167 ceph-mon[32751]: pgmap v4256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:34 smithi090 ceph-mon[29180]: pgmap v4256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:35.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6960 sec 2024-04-26T20:24:36.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6961 sec 2024-04-26T20:24:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6557.075457 secs 2024-04-26T20:24:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:36 smithi119 ceph-mon[32752]: pgmap v4257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6557.075457 secs 2024-04-26T20:24:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:36 smithi167 ceph-mon[32751]: pgmap v4257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6557.075457 secs 2024-04-26T20:24:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:36 smithi090 ceph-mon[29180]: pgmap v4257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:37.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6962 sec 2024-04-26T20:24:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6557.778129 secs 2024-04-26T20:24:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6557.778129 secs 2024-04-26T20:24:37.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6557.778129 secs 2024-04-26T20:24:38.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6963 sec 2024-04-26T20:24:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:38 smithi119 ceph-mon[32752]: pgmap v4258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:38 smithi167 ceph-mon[32751]: pgmap v4258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:38 smithi090 ceph-mon[29180]: pgmap v4258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:39.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6964 sec 2024-04-26T20:24:40.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6965 sec 2024-04-26T20:24:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:40 smithi119 ceph-mon[32752]: pgmap v4259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:40 smithi167 ceph-mon[32751]: pgmap v4259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:40 smithi090 ceph-mon[29180]: pgmap v4259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:41.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6966 sec 2024-04-26T20:24:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6562.075542 secs 2024-04-26T20:24:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6562.075542 secs 2024-04-26T20:24:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6562.075542 secs 2024-04-26T20:24:42.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6967 sec 2024-04-26T20:24:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6562.778201 secs 2024-04-26T20:24:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:42 smithi119 ceph-mon[32752]: pgmap v4260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6562.778201 secs 2024-04-26T20:24:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:42 smithi167 ceph-mon[32751]: pgmap v4260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6562.778201 secs 2024-04-26T20:24:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:42 smithi090 ceph-mon[29180]: pgmap v4260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:43.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6968 sec 2024-04-26T20:24:44.514 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6969 sec 2024-04-26T20:24:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:44 smithi119 ceph-mon[32752]: pgmap v4261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:44 smithi167 ceph-mon[32751]: pgmap v4261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:44 smithi090 ceph-mon[29180]: pgmap v4261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:45.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6970 sec 2024-04-26T20:24:46.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6971 sec 2024-04-26T20:24:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6567.075623 secs 2024-04-26T20:24:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:46 smithi119 ceph-mon[32752]: pgmap v4262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6567.075623 secs 2024-04-26T20:24:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:46 smithi167 ceph-mon[32751]: pgmap v4262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6567.075623 secs 2024-04-26T20:24:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:46 smithi090 ceph-mon[29180]: pgmap v4262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:47.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6972 sec 2024-04-26T20:24:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6567.778306 secs 2024-04-26T20:24:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6567.778306 secs 2024-04-26T20:24:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6567.778306 secs 2024-04-26T20:24:48.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6973 sec 2024-04-26T20:24:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:48 smithi119 ceph-mon[32752]: pgmap v4263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:48 smithi167 ceph-mon[32751]: pgmap v4263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:48 smithi090 ceph-mon[29180]: pgmap v4263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:49.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6974 sec 2024-04-26T20:24:50.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6975 sec 2024-04-26T20:24:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:50 smithi119 ceph-mon[32752]: pgmap v4264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:50 smithi167 ceph-mon[32751]: pgmap v4264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:50 smithi090 ceph-mon[29180]: pgmap v4264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:51.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6976 sec 2024-04-26T20:24:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6572.075705 secs 2024-04-26T20:24:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6572.075705 secs 2024-04-26T20:24:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6572.075705 secs 2024-04-26T20:24:52.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6977 sec 2024-04-26T20:24:52.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6572.778424 secs 2024-04-26T20:24:52.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:52 smithi090 ceph-mon[29180]: pgmap v4265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6572.778424 secs 2024-04-26T20:24:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:52 smithi119 ceph-mon[32752]: pgmap v4265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6572.778424 secs 2024-04-26T20:24:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:52 smithi167 ceph-mon[32751]: pgmap v4265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:53.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6978 sec 2024-04-26T20:24:54.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6979 sec 2024-04-26T20:24:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:54 smithi119 ceph-mon[32752]: pgmap v4266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:54 smithi167 ceph-mon[32751]: pgmap v4266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:54 smithi090 ceph-mon[29180]: pgmap v4266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:55.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6980 sec 2024-04-26T20:24:56.515 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6981 sec 2024-04-26T20:24:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6577.075823 secs 2024-04-26T20:24:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:56 smithi119 ceph-mon[32752]: pgmap v4267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6577.075823 secs 2024-04-26T20:24:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:56 smithi167 ceph-mon[32751]: pgmap v4267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6577.075823 secs 2024-04-26T20:24:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:56 smithi090 ceph-mon[29180]: pgmap v4267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:24:57.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6982 sec 2024-04-26T20:24:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6577.778472 secs 2024-04-26T20:24:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:24:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6577.778472 secs 2024-04-26T20:24:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:24:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6577.778472 secs 2024-04-26T20:24:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:24:58.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6983 sec 2024-04-26T20:24:58.517 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:58 smithi119 ceph-mon[32752]: pgmap v4268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:58.517 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.517 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.518 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.518 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:24:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:58 smithi090 ceph-mon[29180]: pgmap v4268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:58.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:24:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:58 smithi167 ceph-mon[32751]: pgmap v4268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:24:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:24:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:24:59.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6984 sec 2024-04-26T20:25:00.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6985 sec 2024-04-26T20:25:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: pgmap v4269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: pgmap v4269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: pgmap v4269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6986 sec 2024-04-26T20:25:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6582.075892 secs 2024-04-26T20:25:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:25:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:25:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:25:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6582.075892 secs 2024-04-26T20:25:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:25:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:25:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:25:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6582.075892 secs 2024-04-26T20:25:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:25:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:25:02.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6987 sec 2024-04-26T20:25:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6582.778539 secs 2024-04-26T20:25:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:25:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:25:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:25:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:25:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:02 smithi119 ceph-mon[32752]: pgmap v4270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 886 B/s rd, 1 op/s 2024-04-26T20:25:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:02 smithi119 ceph-mon[32752]: pgmap v4271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6582.778539 secs 2024-04-26T20:25:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:25:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:25:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:25:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:25:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:02 smithi167 ceph-mon[32751]: pgmap v4270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 886 B/s rd, 1 op/s 2024-04-26T20:25:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:02 smithi167 ceph-mon[32751]: pgmap v4271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6582.778539 secs 2024-04-26T20:25:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:25:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:25:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:25:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:25:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:02 smithi090 ceph-mon[29180]: pgmap v4270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 886 B/s rd, 1 op/s 2024-04-26T20:25:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:02 smithi090 ceph-mon[29180]: pgmap v4271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:03.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6988 sec 2024-04-26T20:25:04.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6989 sec 2024-04-26T20:25:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:04 smithi119 ceph-mon[32752]: pgmap v4272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:04 smithi167 ceph-mon[32751]: pgmap v4272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:04 smithi090 ceph-mon[29180]: pgmap v4272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:05.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6990 sec 2024-04-26T20:25:06.516 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6991 sec 2024-04-26T20:25:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:06 smithi119 ceph-mon[32752]: pgmap v4273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6587.075947 secs 2024-04-26T20:25:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:06 smithi167 ceph-mon[32751]: pgmap v4273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6587.075947 secs 2024-04-26T20:25:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:06 smithi090 ceph-mon[29180]: pgmap v4273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6587.075947 secs 2024-04-26T20:25:07.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6992 sec 2024-04-26T20:25:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6587.778674 secs 2024-04-26T20:25:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6587.778674 secs 2024-04-26T20:25:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6587.778674 secs 2024-04-26T20:25:08.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6993 sec 2024-04-26T20:25:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:08 smithi119 ceph-mon[32752]: pgmap v4274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:08 smithi167 ceph-mon[32751]: pgmap v4274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:08 smithi090 ceph-mon[29180]: pgmap v4274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:09.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6994 sec 2024-04-26T20:25:10.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6995 sec 2024-04-26T20:25:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:10 smithi119 ceph-mon[32752]: pgmap v4275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:10 smithi167 ceph-mon[32751]: pgmap v4275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:10 smithi090 ceph-mon[29180]: pgmap v4275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T20:25:11.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6996 sec 2024-04-26T20:25:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6592.076053 secs 2024-04-26T20:25:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6592.076053 secs 2024-04-26T20:25:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6592.076053 secs 2024-04-26T20:25:12.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6997 sec 2024-04-26T20:25:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6592.778746 secs 2024-04-26T20:25:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:12 smithi119 ceph-mon[32752]: pgmap v4276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:25:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6592.778746 secs 2024-04-26T20:25:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:12 smithi167 ceph-mon[32751]: pgmap v4276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:25:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6592.778746 secs 2024-04-26T20:25:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:12 smithi090 ceph-mon[29180]: pgmap v4276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:25:13.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6998 sec 2024-04-26T20:25:14.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 6999 sec 2024-04-26T20:25:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:14 smithi119 ceph-mon[32752]: pgmap v4277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:14 smithi167 ceph-mon[32751]: pgmap v4277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:14 smithi090 ceph-mon[29180]: pgmap v4277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:15.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7000 sec 2024-04-26T20:25:16.517 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7001 sec 2024-04-26T20:25:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:16 smithi119 ceph-mon[32752]: pgmap v4278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6597.076137 secs 2024-04-26T20:25:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:16 smithi167 ceph-mon[32751]: pgmap v4278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6597.076137 secs 2024-04-26T20:25:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:16 smithi090 ceph-mon[29180]: pgmap v4278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6597.076137 secs 2024-04-26T20:25:17.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7002 sec 2024-04-26T20:25:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6597.778810 secs 2024-04-26T20:25:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6597.778810 secs 2024-04-26T20:25:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6597.778810 secs 2024-04-26T20:25:18.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7003 sec 2024-04-26T20:25:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:18 smithi167 ceph-mon[32751]: pgmap v4279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:18 smithi090 ceph-mon[29180]: pgmap v4279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:18 smithi119 ceph-mon[32752]: pgmap v4279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:19.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7004 sec 2024-04-26T20:25:20.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7005 sec 2024-04-26T20:25:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:20 smithi167 ceph-mon[32751]: pgmap v4280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:20 smithi090 ceph-mon[29180]: pgmap v4280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:20 smithi119 ceph-mon[32752]: pgmap v4280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:21.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7006 sec 2024-04-26T20:25:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6602.076218 secs 2024-04-26T20:25:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6602.076218 secs 2024-04-26T20:25:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6602.076218 secs 2024-04-26T20:25:22.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7007 sec 2024-04-26T20:25:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6602.778949 secs 2024-04-26T20:25:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:22 smithi167 ceph-mon[32751]: pgmap v4281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6602.778949 secs 2024-04-26T20:25:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:22 smithi090 ceph-mon[29180]: pgmap v4281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6602.778949 secs 2024-04-26T20:25:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:22 smithi119 ceph-mon[32752]: pgmap v4281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:23.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7008 sec 2024-04-26T20:25:24.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7009 sec 2024-04-26T20:25:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:24 smithi090 ceph-mon[29180]: pgmap v4282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:24 smithi119 ceph-mon[32752]: pgmap v4282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:24 smithi167 ceph-mon[32751]: pgmap v4282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:25.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7010 sec 2024-04-26T20:25:26.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7011 sec 2024-04-26T20:25:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:26 smithi090 ceph-mon[29180]: pgmap v4283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6607.076300 secs 2024-04-26T20:25:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:26 smithi119 ceph-mon[32752]: pgmap v4283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6607.076300 secs 2024-04-26T20:25:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:26 smithi167 ceph-mon[32751]: pgmap v4283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6607.076300 secs 2024-04-26T20:25:27.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7012 sec 2024-04-26T20:25:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6607.779064 secs 2024-04-26T20:25:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6607.779064 secs 2024-04-26T20:25:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6607.779064 secs 2024-04-26T20:25:28.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7013 sec 2024-04-26T20:25:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:28 smithi090 ceph-mon[29180]: pgmap v4284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:28 smithi119 ceph-mon[32752]: pgmap v4284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:28 smithi167 ceph-mon[32751]: pgmap v4284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:29.518 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7014 sec 2024-04-26T20:25:30.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7015 sec 2024-04-26T20:25:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:30 smithi090 ceph-mon[29180]: pgmap v4285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:30 smithi119 ceph-mon[32752]: pgmap v4285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:30 smithi167 ceph-mon[32751]: pgmap v4285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:31.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7016 sec 2024-04-26T20:25:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6612.076384 secs 2024-04-26T20:25:31.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6612.076384 secs 2024-04-26T20:25:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6612.076384 secs 2024-04-26T20:25:32.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7017 sec 2024-04-26T20:25:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6612.779179 secs 2024-04-26T20:25:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:32 smithi090 ceph-mon[29180]: pgmap v4286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6612.779179 secs 2024-04-26T20:25:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:32 smithi119 ceph-mon[32752]: pgmap v4286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6612.779179 secs 2024-04-26T20:25:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:32 smithi167 ceph-mon[32751]: pgmap v4286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:33.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7018 sec 2024-04-26T20:25:34.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7019 sec 2024-04-26T20:25:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:34 smithi090 ceph-mon[29180]: pgmap v4287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:34 smithi119 ceph-mon[32752]: pgmap v4287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:34 smithi167 ceph-mon[32751]: pgmap v4287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:35.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7020 sec 2024-04-26T20:25:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:35 smithi090 ceph-mon[29180]: pgmap v4288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:35 smithi119 ceph-mon[32752]: pgmap v4288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:35 smithi167 ceph-mon[32751]: pgmap v4288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:36.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7021 sec 2024-04-26T20:25:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6617.076466 secs 2024-04-26T20:25:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6617.779291 secs 2024-04-26T20:25:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6617.076466 secs 2024-04-26T20:25:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6617.779291 secs 2024-04-26T20:25:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6617.076466 secs 2024-04-26T20:25:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6617.779291 secs 2024-04-26T20:25:37.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7022 sec 2024-04-26T20:25:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:37 smithi090 ceph-mon[29180]: pgmap v4289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:37 smithi119 ceph-mon[32752]: pgmap v4289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:37 smithi167 ceph-mon[32751]: pgmap v4289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:38.519 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7023 sec 2024-04-26T20:25:39.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7024 sec 2024-04-26T20:25:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:39 smithi090 ceph-mon[29180]: pgmap v4290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:39 smithi119 ceph-mon[32752]: pgmap v4290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:39 smithi167 ceph-mon[32751]: pgmap v4290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:40.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7025 sec 2024-04-26T20:25:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:40 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6622.076545 secs 2024-04-26T20:25:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:40 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6622.076545 secs 2024-04-26T20:25:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:40 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6622.076545 secs 2024-04-26T20:25:41.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7026 sec 2024-04-26T20:25:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6622.779374 secs 2024-04-26T20:25:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:41 smithi119 ceph-mon[32752]: pgmap v4291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6622.779374 secs 2024-04-26T20:25:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:41 smithi167 ceph-mon[32751]: pgmap v4291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6622.779374 secs 2024-04-26T20:25:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:41 smithi090 ceph-mon[29180]: pgmap v4291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:42.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7027 sec 2024-04-26T20:25:43.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7028 sec 2024-04-26T20:25:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:43 smithi119 ceph-mon[32752]: pgmap v4292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:43 smithi167 ceph-mon[32751]: pgmap v4292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:43 smithi090 ceph-mon[29180]: pgmap v4292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:44.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7029 sec 2024-04-26T20:25:45.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7030 sec 2024-04-26T20:25:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:45 smithi119 ceph-mon[32752]: pgmap v4293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:45 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6627.076627 secs 2024-04-26T20:25:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:45 smithi167 ceph-mon[32751]: pgmap v4293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:45 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6627.076627 secs 2024-04-26T20:25:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:45 smithi090 ceph-mon[29180]: pgmap v4293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:45 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6627.076627 secs 2024-04-26T20:25:46.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7031 sec 2024-04-26T20:25:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6627.779513 secs 2024-04-26T20:25:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6627.779513 secs 2024-04-26T20:25:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6627.779513 secs 2024-04-26T20:25:47.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7032 sec 2024-04-26T20:25:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:47 smithi119 ceph-mon[32752]: pgmap v4294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:47 smithi167 ceph-mon[32751]: pgmap v4294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:47 smithi090 ceph-mon[29180]: pgmap v4294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:48.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7033 sec 2024-04-26T20:25:49.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7034 sec 2024-04-26T20:25:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:49 smithi119 ceph-mon[32752]: pgmap v4295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:49 smithi167 ceph-mon[32751]: pgmap v4295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:49 smithi090 ceph-mon[29180]: pgmap v4295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:50.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7035 sec 2024-04-26T20:25:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:50 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6632.076712 secs 2024-04-26T20:25:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:50 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6632.076712 secs 2024-04-26T20:25:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:50 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6632.076712 secs 2024-04-26T20:25:51.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7036 sec 2024-04-26T20:25:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6632.779643 secs 2024-04-26T20:25:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:51 smithi119 ceph-mon[32752]: pgmap v4296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6632.779643 secs 2024-04-26T20:25:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:51 smithi167 ceph-mon[32751]: pgmap v4296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6632.779643 secs 2024-04-26T20:25:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:51 smithi090 ceph-mon[29180]: pgmap v4296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:52.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7037 sec 2024-04-26T20:25:53.520 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7038 sec 2024-04-26T20:25:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:53 smithi119 ceph-mon[32752]: pgmap v4297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:53 smithi167 ceph-mon[32751]: pgmap v4297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:53 smithi090 ceph-mon[29180]: pgmap v4297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:54.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7039 sec 2024-04-26T20:25:55.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7040 sec 2024-04-26T20:25:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:55 smithi119 ceph-mon[32752]: pgmap v4298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6637.076817 secs 2024-04-26T20:25:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:55 smithi167 ceph-mon[32751]: pgmap v4298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6637.076817 secs 2024-04-26T20:25:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:55 smithi090 ceph-mon[29180]: pgmap v4298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:25:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6637.076817 secs 2024-04-26T20:25:56.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7041 sec 2024-04-26T20:25:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6637.779710 secs 2024-04-26T20:25:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6637.779710 secs 2024-04-26T20:25:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6637.779710 secs 2024-04-26T20:25:57.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7042 sec 2024-04-26T20:25:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:57 smithi119 ceph-mon[32752]: pgmap v4299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:57 smithi167 ceph-mon[32751]: pgmap v4299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:57 smithi090 ceph-mon[29180]: pgmap v4299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:25:58.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7043 sec 2024-04-26T20:25:59.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7044 sec 2024-04-26T20:26:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:25:59 smithi119 ceph-mon[32752]: pgmap v4300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:25:59 smithi167 ceph-mon[32751]: pgmap v4300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:25:59 smithi090 ceph-mon[29180]: pgmap v4300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:00.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7045 sec 2024-04-26T20:26:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:00 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6642.076899 secs 2024-04-26T20:26:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:00 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6642.076899 secs 2024-04-26T20:26:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:00 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6642.076899 secs 2024-04-26T20:26:01.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7046 sec 2024-04-26T20:26:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6642.779830 secs 2024-04-26T20:26:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:01 smithi119 ceph-mon[32752]: pgmap v4301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:26:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6642.779830 secs 2024-04-26T20:26:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:01 smithi167 ceph-mon[32751]: pgmap v4301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:26:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6642.779830 secs 2024-04-26T20:26:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:01 smithi090 ceph-mon[29180]: pgmap v4301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:26:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:02.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7047 sec 2024-04-26T20:26:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:26:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:26:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:02 smithi119 ceph-mon[32752]: pgmap v4302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T20:26:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:02 smithi119 ceph-mon[32752]: pgmap v4303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:26:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:26:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:02 smithi167 ceph-mon[32751]: pgmap v4302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T20:26:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:02 smithi167 ceph-mon[32751]: pgmap v4303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:26:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:26:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:02 smithi090 ceph-mon[29180]: pgmap v4302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T20:26:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:02 smithi090 ceph-mon[29180]: pgmap v4303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:26:03.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7048 sec 2024-04-26T20:26:04.521 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7049 sec 2024-04-26T20:26:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:04 smithi119 ceph-mon[32752]: pgmap v4304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:04 smithi167 ceph-mon[32751]: pgmap v4304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:04 smithi090 ceph-mon[29180]: pgmap v4304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:05.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7050 sec 2024-04-26T20:26:06.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6647.076961 secs 2024-04-26T20:26:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6647.076961 secs 2024-04-26T20:26:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6647.076961 secs 2024-04-26T20:26:06.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7051 sec 2024-04-26T20:26:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6647.779943 secs 2024-04-26T20:26:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:07 smithi119 ceph-mon[32752]: pgmap v4305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6647.779943 secs 2024-04-26T20:26:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:07 smithi167 ceph-mon[32751]: pgmap v4305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6647.779943 secs 2024-04-26T20:26:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:07 smithi090 ceph-mon[29180]: pgmap v4305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:07.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7052 sec 2024-04-26T20:26:08.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7053 sec 2024-04-26T20:26:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:09 smithi119 ceph-mon[32752]: pgmap v4306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:26:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:09 smithi167 ceph-mon[32751]: pgmap v4306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:26:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:09 smithi090 ceph-mon[29180]: pgmap v4306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:26:09.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7054 sec 2024-04-26T20:26:10.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7055 sec 2024-04-26T20:26:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6652.077053 secs 2024-04-26T20:26:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:11 smithi119 ceph-mon[32752]: pgmap v4307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6652.077053 secs 2024-04-26T20:26:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:11 smithi167 ceph-mon[32751]: pgmap v4307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6652.077053 secs 2024-04-26T20:26:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:11 smithi090 ceph-mon[29180]: pgmap v4307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:26:11.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7056 sec 2024-04-26T20:26:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6652.780057 secs 2024-04-26T20:26:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6652.780057 secs 2024-04-26T20:26:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6652.780057 secs 2024-04-26T20:26:12.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7057 sec 2024-04-26T20:26:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:13 smithi119 ceph-mon[32752]: pgmap v4308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:26:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:13 smithi167 ceph-mon[32751]: pgmap v4308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:26:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:13 smithi090 ceph-mon[29180]: pgmap v4308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:26:13.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7058 sec 2024-04-26T20:26:14.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7059 sec 2024-04-26T20:26:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:15 smithi119 ceph-mon[32752]: pgmap v4309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:15 smithi167 ceph-mon[32751]: pgmap v4309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:15 smithi090 ceph-mon[29180]: pgmap v4309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:15.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7060 sec 2024-04-26T20:26:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6657.077116 secs 2024-04-26T20:26:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6657.077116 secs 2024-04-26T20:26:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6657.077116 secs 2024-04-26T20:26:16.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7061 sec 2024-04-26T20:26:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6657.780195 secs 2024-04-26T20:26:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:17 smithi119 ceph-mon[32752]: pgmap v4310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6657.780195 secs 2024-04-26T20:26:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:17 smithi167 ceph-mon[32751]: pgmap v4310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6657.780195 secs 2024-04-26T20:26:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:17 smithi090 ceph-mon[29180]: pgmap v4310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:17.522 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7062 sec 2024-04-26T20:26:18.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7063 sec 2024-04-26T20:26:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:19 smithi119 ceph-mon[32752]: pgmap v4311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:19 smithi167 ceph-mon[32751]: pgmap v4311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:19 smithi090 ceph-mon[29180]: pgmap v4311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:19.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7064 sec 2024-04-26T20:26:20.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7065 sec 2024-04-26T20:26:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6662.077184 secs 2024-04-26T20:26:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:21 smithi119 ceph-mon[32752]: pgmap v4312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6662.077184 secs 2024-04-26T20:26:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:21 smithi167 ceph-mon[32751]: pgmap v4312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6662.077184 secs 2024-04-26T20:26:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:21 smithi090 ceph-mon[29180]: pgmap v4312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:21.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7066 sec 2024-04-26T20:26:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6662.780311 secs 2024-04-26T20:26:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6662.780311 secs 2024-04-26T20:26:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6662.780311 secs 2024-04-26T20:26:22.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7067 sec 2024-04-26T20:26:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:23 smithi119 ceph-mon[32752]: pgmap v4313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:23 smithi167 ceph-mon[32751]: pgmap v4313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:23 smithi090 ceph-mon[29180]: pgmap v4313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:23.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7068 sec 2024-04-26T20:26:24.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7069 sec 2024-04-26T20:26:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:25 smithi119 ceph-mon[32752]: pgmap v4314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:25 smithi167 ceph-mon[32751]: pgmap v4314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:25 smithi090 ceph-mon[29180]: pgmap v4314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:25.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7070 sec 2024-04-26T20:26:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6667.077264 secs 2024-04-26T20:26:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6667.077264 secs 2024-04-26T20:26:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6667.077264 secs 2024-04-26T20:26:26.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7071 sec 2024-04-26T20:26:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6667.780451 secs 2024-04-26T20:26:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:27 smithi119 ceph-mon[32752]: pgmap v4315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6667.780451 secs 2024-04-26T20:26:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:27 smithi167 ceph-mon[32751]: pgmap v4315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6667.780451 secs 2024-04-26T20:26:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:27 smithi090 ceph-mon[29180]: pgmap v4315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:27.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7072 sec 2024-04-26T20:26:28.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7073 sec 2024-04-26T20:26:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:29 smithi167 ceph-mon[32751]: pgmap v4316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:29.350 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:26:29.349+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:26:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:26:29.349+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:26:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:26:29.350+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 55 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:26:29.351 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:26:29.350+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 55 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:26:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:29 smithi090 ceph-mon[29180]: pgmap v4316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:29.523 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7074 sec 2024-04-26T20:26:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:29 smithi119 ceph-mon[32752]: pgmap v4316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:30.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7075 sec 2024-04-26T20:26:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6672.077337 secs 2024-04-26T20:26:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:31 smithi167 ceph-mon[32751]: pgmap v4317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6672.077337 secs 2024-04-26T20:26:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:31 smithi090 ceph-mon[29180]: pgmap v4317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:31.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7076 sec 2024-04-26T20:26:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6672.077337 secs 2024-04-26T20:26:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:31 smithi119 ceph-mon[32752]: pgmap v4317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6672.780564 secs 2024-04-26T20:26:32.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7077 sec 2024-04-26T20:26:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6672.780564 secs 2024-04-26T20:26:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6672.780564 secs 2024-04-26T20:26:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:33 smithi090 ceph-mon[29180]: pgmap v4318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:33.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7078 sec 2024-04-26T20:26:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:33 smithi119 ceph-mon[32752]: pgmap v4318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:33.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:33 smithi167 ceph-mon[32751]: pgmap v4318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:34.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7079 sec 2024-04-26T20:26:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:35 smithi090 ceph-mon[29180]: pgmap v4319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:35.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7080 sec 2024-04-26T20:26:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:35 smithi119 ceph-mon[32752]: pgmap v4319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:35 smithi167 ceph-mon[32751]: pgmap v4319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6677.077417 secs 2024-04-26T20:26:36.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7081 sec 2024-04-26T20:26:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6677.077417 secs 2024-04-26T20:26:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6677.077417 secs 2024-04-26T20:26:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6677.780680 secs 2024-04-26T20:26:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:37 smithi090 ceph-mon[29180]: pgmap v4320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:37.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7082 sec 2024-04-26T20:26:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6677.780680 secs 2024-04-26T20:26:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:37 smithi119 ceph-mon[32752]: pgmap v4320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6677.780680 secs 2024-04-26T20:26:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:37 smithi167 ceph-mon[32751]: pgmap v4320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:38.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7083 sec 2024-04-26T20:26:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:39 smithi090 ceph-mon[29180]: pgmap v4321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:39.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7084 sec 2024-04-26T20:26:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:39 smithi119 ceph-mon[32752]: pgmap v4321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:39 smithi167 ceph-mon[32751]: pgmap v4321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:40.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7085 sec 2024-04-26T20:26:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6682.077499 secs 2024-04-26T20:26:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:41 smithi090 ceph-mon[29180]: pgmap v4322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:41.524 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7086 sec 2024-04-26T20:26:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6682.077499 secs 2024-04-26T20:26:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:41 smithi119 ceph-mon[32752]: pgmap v4322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6682.077499 secs 2024-04-26T20:26:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:41 smithi167 ceph-mon[32751]: pgmap v4322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6682.780817 secs 2024-04-26T20:26:42.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7087 sec 2024-04-26T20:26:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6682.780817 secs 2024-04-26T20:26:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6682.780817 secs 2024-04-26T20:26:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:43 smithi090 ceph-mon[29180]: pgmap v4323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:43.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7088 sec 2024-04-26T20:26:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:43 smithi119 ceph-mon[32752]: pgmap v4323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:43 smithi167 ceph-mon[32751]: pgmap v4323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:44.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7089 sec 2024-04-26T20:26:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:45 smithi090 ceph-mon[29180]: pgmap v4324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:45.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7090 sec 2024-04-26T20:26:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:45 smithi119 ceph-mon[32752]: pgmap v4324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:45 smithi167 ceph-mon[32751]: pgmap v4324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6687.077575 secs 2024-04-26T20:26:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6687.077575 secs 2024-04-26T20:26:46.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7091 sec 2024-04-26T20:26:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6687.077575 secs 2024-04-26T20:26:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6687.780939 secs 2024-04-26T20:26:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:47 smithi090 ceph-mon[29180]: pgmap v4325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:47.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7092 sec 2024-04-26T20:26:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6687.780939 secs 2024-04-26T20:26:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:47 smithi119 ceph-mon[32752]: pgmap v4325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6687.780939 secs 2024-04-26T20:26:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:47 smithi167 ceph-mon[32751]: pgmap v4325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:48.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7093 sec 2024-04-26T20:26:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:49 smithi090 ceph-mon[29180]: pgmap v4326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:49.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7094 sec 2024-04-26T20:26:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:49 smithi119 ceph-mon[32752]: pgmap v4326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:49 smithi167 ceph-mon[32751]: pgmap v4326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:50.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7095 sec 2024-04-26T20:26:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6692.077657 secs 2024-04-26T20:26:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:51 smithi090 ceph-mon[29180]: pgmap v4327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:51.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7096 sec 2024-04-26T20:26:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6692.077657 secs 2024-04-26T20:26:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:51 smithi119 ceph-mon[32752]: pgmap v4327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6692.077657 secs 2024-04-26T20:26:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:51 smithi167 ceph-mon[32751]: pgmap v4327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6692.781089 secs 2024-04-26T20:26:52.525 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7097 sec 2024-04-26T20:26:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6692.781089 secs 2024-04-26T20:26:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6692.781089 secs 2024-04-26T20:26:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:53 smithi090 ceph-mon[29180]: pgmap v4328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:53.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7098 sec 2024-04-26T20:26:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:53 smithi119 ceph-mon[32752]: pgmap v4328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:53 smithi167 ceph-mon[32751]: pgmap v4328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:54.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7099 sec 2024-04-26T20:26:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:55 smithi090 ceph-mon[29180]: pgmap v4329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:55.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7100 sec 2024-04-26T20:26:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:55 smithi119 ceph-mon[32752]: pgmap v4329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:55 smithi167 ceph-mon[32751]: pgmap v4329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:26:56.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7101 sec 2024-04-26T20:26:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6697.077717 secs 2024-04-26T20:26:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6697.077717 secs 2024-04-26T20:26:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6697.077717 secs 2024-04-26T20:26:57.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7102 sec 2024-04-26T20:26:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6697.781245 secs 2024-04-26T20:26:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:57 smithi119 ceph-mon[32752]: pgmap v4330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6697.781245 secs 2024-04-26T20:26:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:57 smithi167 ceph-mon[32751]: pgmap v4330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6697.781245 secs 2024-04-26T20:26:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:57 smithi090 ceph-mon[29180]: pgmap v4330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:58.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7103 sec 2024-04-26T20:26:59.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7104 sec 2024-04-26T20:26:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:26:59 smithi119 ceph-mon[32752]: pgmap v4331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:26:59 smithi167 ceph-mon[32751]: pgmap v4331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:26:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:26:59 smithi090 ceph-mon[29180]: pgmap v4331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:00.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7105 sec 2024-04-26T20:27:01.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7106 sec 2024-04-26T20:27:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:01 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 6702.077818 secs 2024-04-26T20:27:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:01 smithi119 ceph-mon[32752]: slow request 2884.123449 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:27:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:01 smithi119 ceph-mon[32752]: pgmap v4332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:01 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 6702.077818 secs 2024-04-26T20:27:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:01 smithi167 ceph-mon[32751]: slow request 2884.123449 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:27:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:01 smithi167 ceph-mon[32751]: pgmap v4332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:01 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 6702.077818 secs 2024-04-26T20:27:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:01 smithi090 ceph-mon[29180]: slow request 2884.123449 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:27:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:01 smithi090 ceph-mon[29180]: pgmap v4332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:02.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6702.781354 secs 2024-04-26T20:27:02.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:27:02.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7107 sec 2024-04-26T20:27:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6702.781354 secs 2024-04-26T20:27:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:27:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6702.781354 secs 2024-04-26T20:27:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:27:03.526 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7108 sec 2024-04-26T20:27:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:03 smithi119 ceph-mon[32752]: pgmap v4333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:27:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:27:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:27:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:03 smithi167 ceph-mon[32751]: pgmap v4333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:27:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:27:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:27:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:03 smithi090 ceph-mon[29180]: pgmap v4333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:27:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:27:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:27:04.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7109 sec 2024-04-26T20:27:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:04 smithi119 ceph-mon[32752]: pgmap v4334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-26T20:27:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:04 smithi119 ceph-mon[32752]: pgmap v4335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:27:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:04 smithi167 ceph-mon[32751]: pgmap v4334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-26T20:27:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:04 smithi167 ceph-mon[32751]: pgmap v4335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:27:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:04 smithi090 ceph-mon[29180]: pgmap v4334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-26T20:27:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:04 smithi090 ceph-mon[29180]: pgmap v4335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T20:27:05.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7110 sec 2024-04-26T20:27:06.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:06 smithi090 ceph-mon[29180]: pgmap v4336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:06.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6707.077906 secs 2024-04-26T20:27:06.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7111 sec 2024-04-26T20:27:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:06 smithi119 ceph-mon[32752]: pgmap v4336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6707.077906 secs 2024-04-26T20:27:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:06 smithi167 ceph-mon[32751]: pgmap v4336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6707.077906 secs 2024-04-26T20:27:07.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7112 sec 2024-04-26T20:27:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6707.781472 secs 2024-04-26T20:27:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6707.781472 secs 2024-04-26T20:27:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6707.781472 secs 2024-04-26T20:27:08.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7113 sec 2024-04-26T20:27:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:08 smithi119 ceph-mon[32752]: pgmap v4337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:08 smithi167 ceph-mon[32751]: pgmap v4337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:08 smithi090 ceph-mon[29180]: pgmap v4337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:09.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7114 sec 2024-04-26T20:27:10.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7115 sec 2024-04-26T20:27:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:10 smithi119 ceph-mon[32752]: pgmap v4338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:10 smithi167 ceph-mon[32751]: pgmap v4338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:10 smithi090 ceph-mon[29180]: pgmap v4338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:11.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7116 sec 2024-04-26T20:27:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6712.077990 secs 2024-04-26T20:27:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6712.077990 secs 2024-04-26T20:27:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6712.077990 secs 2024-04-26T20:27:12.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7117 sec 2024-04-26T20:27:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6712.781594 secs 2024-04-26T20:27:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:12 smithi119 ceph-mon[32752]: pgmap v4339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6712.781594 secs 2024-04-26T20:27:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:12 smithi167 ceph-mon[32751]: pgmap v4339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6712.781594 secs 2024-04-26T20:27:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:12 smithi090 ceph-mon[29180]: pgmap v4339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:27:13.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7118 sec 2024-04-26T20:27:14.527 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7119 sec 2024-04-26T20:27:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:14 smithi119 ceph-mon[32752]: pgmap v4340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:27:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:14 smithi167 ceph-mon[32751]: pgmap v4340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:27:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:14 smithi090 ceph-mon[29180]: pgmap v4340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:27:15.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7120 sec 2024-04-26T20:27:16.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7121 sec 2024-04-26T20:27:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:16 smithi119 ceph-mon[32752]: pgmap v4341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6717.078095 secs 2024-04-26T20:27:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:16 smithi167 ceph-mon[32751]: pgmap v4341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6717.078095 secs 2024-04-26T20:27:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:16 smithi090 ceph-mon[29180]: pgmap v4341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6717.078095 secs 2024-04-26T20:27:17.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7122 sec 2024-04-26T20:27:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6717.781671 secs 2024-04-26T20:27:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6717.781671 secs 2024-04-26T20:27:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6717.781671 secs 2024-04-26T20:27:18.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7123 sec 2024-04-26T20:27:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:18 smithi119 ceph-mon[32752]: pgmap v4342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:18 smithi167 ceph-mon[32751]: pgmap v4342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:18 smithi090 ceph-mon[29180]: pgmap v4342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:19.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7124 sec 2024-04-26T20:27:20.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7125 sec 2024-04-26T20:27:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:20 smithi119 ceph-mon[32752]: pgmap v4343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:20 smithi167 ceph-mon[32751]: pgmap v4343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:20 smithi090 ceph-mon[29180]: pgmap v4343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:21.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7126 sec 2024-04-26T20:27:21.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6722.078180 secs 2024-04-26T20:27:21.596 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6722.078180 secs 2024-04-26T20:27:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6722.078180 secs 2024-04-26T20:27:22.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7127 sec 2024-04-26T20:27:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6722.781823 secs 2024-04-26T20:27:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:22 smithi119 ceph-mon[32752]: pgmap v4344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6722.781823 secs 2024-04-26T20:27:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:22 smithi167 ceph-mon[32751]: pgmap v4344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6722.781823 secs 2024-04-26T20:27:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:22 smithi090 ceph-mon[29180]: pgmap v4344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:23.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7128 sec 2024-04-26T20:27:24.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7129 sec 2024-04-26T20:27:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:24 smithi119 ceph-mon[32752]: pgmap v4345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:24 smithi167 ceph-mon[32751]: pgmap v4345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:24 smithi090 ceph-mon[29180]: pgmap v4345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:25.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7130 sec 2024-04-26T20:27:26.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7131 sec 2024-04-26T20:27:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:26 smithi119 ceph-mon[32752]: pgmap v4346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6727.078259 secs 2024-04-26T20:27:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:26 smithi167 ceph-mon[32751]: pgmap v4346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6727.078259 secs 2024-04-26T20:27:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:26 smithi090 ceph-mon[29180]: pgmap v4346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6727.078259 secs 2024-04-26T20:27:27.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7132 sec 2024-04-26T20:27:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6727.781896 secs 2024-04-26T20:27:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6727.781896 secs 2024-04-26T20:27:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6727.781896 secs 2024-04-26T20:27:28.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7133 sec 2024-04-26T20:27:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:28 smithi119 ceph-mon[32752]: pgmap v4347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:28 smithi167 ceph-mon[32751]: pgmap v4347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:28.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:28 smithi090 ceph-mon[29180]: pgmap v4347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:29.528 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7134 sec 2024-04-26T20:27:30.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7135 sec 2024-04-26T20:27:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:30 smithi119 ceph-mon[32752]: pgmap v4348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:30 smithi167 ceph-mon[32751]: pgmap v4348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:30 smithi090 ceph-mon[29180]: pgmap v4348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:31.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6732.078319 secs 2024-04-26T20:27:31.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7136 sec 2024-04-26T20:27:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6732.078319 secs 2024-04-26T20:27:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6732.078319 secs 2024-04-26T20:27:32.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7137 sec 2024-04-26T20:27:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6732.781969 secs 2024-04-26T20:27:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:32 smithi119 ceph-mon[32752]: pgmap v4349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6732.781969 secs 2024-04-26T20:27:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:32 smithi167 ceph-mon[32751]: pgmap v4349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6732.781969 secs 2024-04-26T20:27:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:32 smithi090 ceph-mon[29180]: pgmap v4349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:33.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7138 sec 2024-04-26T20:27:34.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7139 sec 2024-04-26T20:27:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:34 smithi119 ceph-mon[32752]: pgmap v4350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:34 smithi167 ceph-mon[32751]: pgmap v4350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:34 smithi090 ceph-mon[29180]: pgmap v4350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:35.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7140 sec 2024-04-26T20:27:36.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7141 sec 2024-04-26T20:27:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:36 smithi119 ceph-mon[32752]: pgmap v4351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6737.078408 secs 2024-04-26T20:27:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:36 smithi167 ceph-mon[32751]: pgmap v4351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6737.078408 secs 2024-04-26T20:27:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:36 smithi090 ceph-mon[29180]: pgmap v4351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6737.078408 secs 2024-04-26T20:27:37.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7142 sec 2024-04-26T20:27:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:37 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 6737.782098 secs 2024-04-26T20:27:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:37 smithi119 ceph-mon[32752]: slow request 5764.473863 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:27:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:37 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 6737.782098 secs 2024-04-26T20:27:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:37 smithi167 ceph-mon[32751]: slow request 5764.473863 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:27:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:37 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 6737.782098 secs 2024-04-26T20:27:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:37 smithi090 ceph-mon[29180]: slow request 5764.473863 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:27:38.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7143 sec 2024-04-26T20:27:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:38 smithi119 ceph-mon[32752]: pgmap v4352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:38 smithi167 ceph-mon[32751]: pgmap v4352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:38 smithi090 ceph-mon[29180]: pgmap v4352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:39.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7144 sec 2024-04-26T20:27:40.529 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7145 sec 2024-04-26T20:27:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:40 smithi119 ceph-mon[32752]: pgmap v4353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:40 smithi167 ceph-mon[32751]: pgmap v4353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:40 smithi090 ceph-mon[29180]: pgmap v4353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:41.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7146 sec 2024-04-26T20:27:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6742.078490 secs 2024-04-26T20:27:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6742.078490 secs 2024-04-26T20:27:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6742.078490 secs 2024-04-26T20:27:42.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7147 sec 2024-04-26T20:27:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6742.782233 secs 2024-04-26T20:27:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:42 smithi119 ceph-mon[32752]: pgmap v4354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6742.782233 secs 2024-04-26T20:27:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:42 smithi167 ceph-mon[32751]: pgmap v4354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6742.782233 secs 2024-04-26T20:27:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:42 smithi090 ceph-mon[29180]: pgmap v4354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:43.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7148 sec 2024-04-26T20:27:44.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7149 sec 2024-04-26T20:27:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:44 smithi119 ceph-mon[32752]: pgmap v4355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:44 smithi167 ceph-mon[32751]: pgmap v4355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:44 smithi090 ceph-mon[29180]: pgmap v4355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:45.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7150 sec 2024-04-26T20:27:46.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7151 sec 2024-04-26T20:27:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:46 smithi119 ceph-mon[32752]: pgmap v4356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6747.078573 secs 2024-04-26T20:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:46 smithi167 ceph-mon[32751]: pgmap v4356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6747.078573 secs 2024-04-26T20:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:46 smithi090 ceph-mon[29180]: pgmap v4356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6747.078573 secs 2024-04-26T20:27:47.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7152 sec 2024-04-26T20:27:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6747.782350 secs 2024-04-26T20:27:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6747.782350 secs 2024-04-26T20:27:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6747.782350 secs 2024-04-26T20:27:48.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7153 sec 2024-04-26T20:27:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:48 smithi119 ceph-mon[32752]: pgmap v4357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:48 smithi167 ceph-mon[32751]: pgmap v4357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:48 smithi090 ceph-mon[29180]: pgmap v4357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:49.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7154 sec 2024-04-26T20:27:50.530 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7155 sec 2024-04-26T20:27:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:50 smithi119 ceph-mon[32752]: pgmap v4358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:50 smithi167 ceph-mon[32751]: pgmap v4358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:50 smithi090 ceph-mon[29180]: pgmap v4358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:51.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7156 sec 2024-04-26T20:27:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6752.078659 secs 2024-04-26T20:27:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6752.078659 secs 2024-04-26T20:27:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6752.078659 secs 2024-04-26T20:27:52.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7157 sec 2024-04-26T20:27:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6752.782406 secs 2024-04-26T20:27:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:52 smithi119 ceph-mon[32752]: pgmap v4359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6752.782406 secs 2024-04-26T20:27:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:52 smithi167 ceph-mon[32751]: pgmap v4359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6752.782406 secs 2024-04-26T20:27:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:52 smithi090 ceph-mon[29180]: pgmap v4359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:53.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7158 sec 2024-04-26T20:27:54.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7159 sec 2024-04-26T20:27:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:54 smithi119 ceph-mon[32752]: pgmap v4360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:54 smithi167 ceph-mon[32751]: pgmap v4360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:54 smithi090 ceph-mon[29180]: pgmap v4360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:55.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7160 sec 2024-04-26T20:27:56.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7161 sec 2024-04-26T20:27:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:56 smithi119 ceph-mon[32752]: pgmap v4361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6757.078720 secs 2024-04-26T20:27:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:56 smithi167 ceph-mon[32751]: pgmap v4361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6757.078720 secs 2024-04-26T20:27:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:56 smithi090 ceph-mon[29180]: pgmap v4361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:27:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6757.078720 secs 2024-04-26T20:27:57.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7162 sec 2024-04-26T20:27:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6757.782528 secs 2024-04-26T20:27:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6757.782528 secs 2024-04-26T20:27:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6757.782528 secs 2024-04-26T20:27:58.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7163 sec 2024-04-26T20:27:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:27:58 smithi119 ceph-mon[32752]: pgmap v4362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:27:58 smithi167 ceph-mon[32751]: pgmap v4362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:27:58 smithi090 ceph-mon[29180]: pgmap v4362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:27:59.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7164 sec 2024-04-26T20:28:00.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7165 sec 2024-04-26T20:28:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:00 smithi119 ceph-mon[32752]: pgmap v4363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:00 smithi167 ceph-mon[32751]: pgmap v4363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:00 smithi090 ceph-mon[29180]: pgmap v4363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:01.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7166 sec 2024-04-26T20:28:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6762.078826 secs 2024-04-26T20:28:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6762.078826 secs 2024-04-26T20:28:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6762.078826 secs 2024-04-26T20:28:02.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7167 sec 2024-04-26T20:28:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6762.782595 secs 2024-04-26T20:28:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:02 smithi119 ceph-mon[32752]: pgmap v4364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6762.782595 secs 2024-04-26T20:28:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:02 smithi167 ceph-mon[32751]: pgmap v4364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:02.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6762.782595 secs 2024-04-26T20:28:02.581 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:02 smithi090 ceph-mon[29180]: pgmap v4364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:02.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:28:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:28:02.580+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:28:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:28:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:28:02.580+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:28:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:28:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:28:02.580+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:28:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:28:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:28:02.580+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:28:02.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:28:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:28:02.580+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:28:03.531 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7168 sec 2024-04-26T20:28:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:28:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:28:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:28:04.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7169 sec 2024-04-26T20:28:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:04 smithi119 ceph-mon[32752]: pgmap v4365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:28:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:28:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:28:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:04 smithi167 ceph-mon[32751]: pgmap v4365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:28:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:28:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:28:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:04 smithi090 ceph-mon[29180]: pgmap v4365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:28:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:28:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:28:05.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7170 sec 2024-04-26T20:28:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:05 smithi119 ceph-mon[32752]: pgmap v4366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 380 B/s wr, 15 op/s 2024-04-26T20:28:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:05 smithi119 ceph-mon[32752]: pgmap v4367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:05 smithi167 ceph-mon[32751]: pgmap v4366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 380 B/s wr, 15 op/s 2024-04-26T20:28:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:05 smithi167 ceph-mon[32751]: pgmap v4367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:05 smithi090 ceph-mon[29180]: pgmap v4366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 380 B/s wr, 15 op/s 2024-04-26T20:28:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:05 smithi090 ceph-mon[29180]: pgmap v4367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:06.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7171 sec 2024-04-26T20:28:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6767.078913 secs 2024-04-26T20:28:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6767.078913 secs 2024-04-26T20:28:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6767.078913 secs 2024-04-26T20:28:07.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7172 sec 2024-04-26T20:28:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6767.782674 secs 2024-04-26T20:28:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:07 smithi167 ceph-mon[32751]: pgmap v4368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6767.782674 secs 2024-04-26T20:28:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:07 smithi090 ceph-mon[29180]: pgmap v4368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6767.782674 secs 2024-04-26T20:28:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:07 smithi119 ceph-mon[32752]: pgmap v4368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:08.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7173 sec 2024-04-26T20:28:09.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7174 sec 2024-04-26T20:28:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:09 smithi167 ceph-mon[32751]: pgmap v4369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 467 B/s wr, 17 op/s 2024-04-26T20:28:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:09 smithi090 ceph-mon[29180]: pgmap v4369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 467 B/s wr, 17 op/s 2024-04-26T20:28:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:09 smithi119 ceph-mon[32752]: pgmap v4369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 467 B/s wr, 17 op/s 2024-04-26T20:28:10.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7175 sec 2024-04-26T20:28:11.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7176 sec 2024-04-26T20:28:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:11 smithi167 ceph-mon[32751]: pgmap v4370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6772.079002 secs 2024-04-26T20:28:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:11 smithi090 ceph-mon[29180]: pgmap v4370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6772.079002 secs 2024-04-26T20:28:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:11 smithi119 ceph-mon[32752]: pgmap v4370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6772.079002 secs 2024-04-26T20:28:12.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7177 sec 2024-04-26T20:28:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6772.782741 secs 2024-04-26T20:28:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6772.782741 secs 2024-04-26T20:28:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6772.782741 secs 2024-04-26T20:28:13.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7178 sec 2024-04-26T20:28:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:13 smithi090 ceph-mon[29180]: pgmap v4371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:13 smithi119 ceph-mon[32752]: pgmap v4371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:13 smithi167 ceph-mon[32751]: pgmap v4371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T20:28:14.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7179 sec 2024-04-26T20:28:15.532 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7180 sec 2024-04-26T20:28:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:15 smithi090 ceph-mon[29180]: pgmap v4372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:28:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:15 smithi119 ceph-mon[32752]: pgmap v4372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:28:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:15 smithi167 ceph-mon[32751]: pgmap v4372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:28:16.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7181 sec 2024-04-26T20:28:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6777.079082 secs 2024-04-26T20:28:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6777.079082 secs 2024-04-26T20:28:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6777.079082 secs 2024-04-26T20:28:17.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7182 sec 2024-04-26T20:28:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6777.782835 secs 2024-04-26T20:28:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:17 smithi090 ceph-mon[29180]: pgmap v4373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6777.782835 secs 2024-04-26T20:28:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:17 smithi119 ceph-mon[32752]: pgmap v4373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6777.782835 secs 2024-04-26T20:28:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:17 smithi167 ceph-mon[32751]: pgmap v4373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:18.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7183 sec 2024-04-26T20:28:19.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7184 sec 2024-04-26T20:28:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:19 smithi090 ceph-mon[29180]: pgmap v4374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:19 smithi119 ceph-mon[32752]: pgmap v4374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:19 smithi167 ceph-mon[32751]: pgmap v4374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:20.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7185 sec 2024-04-26T20:28:21.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7186 sec 2024-04-26T20:28:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:21 smithi090 ceph-mon[29180]: pgmap v4375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6782.079166 secs 2024-04-26T20:28:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:21 smithi119 ceph-mon[32752]: pgmap v4375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6782.079166 secs 2024-04-26T20:28:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:21 smithi167 ceph-mon[32751]: pgmap v4375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6782.079166 secs 2024-04-26T20:28:22.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7187 sec 2024-04-26T20:28:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6782.782916 secs 2024-04-26T20:28:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6782.782916 secs 2024-04-26T20:28:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6782.782916 secs 2024-04-26T20:28:23.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7188 sec 2024-04-26T20:28:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:23 smithi090 ceph-mon[29180]: pgmap v4376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:23 smithi119 ceph-mon[32752]: pgmap v4376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:23 smithi167 ceph-mon[32751]: pgmap v4376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:24.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7189 sec 2024-04-26T20:28:25.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7190 sec 2024-04-26T20:28:25.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:25 smithi090 ceph-mon[29180]: pgmap v4377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:25 smithi119 ceph-mon[32752]: pgmap v4377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:25 smithi167 ceph-mon[32751]: pgmap v4377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:26.533 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7191 sec 2024-04-26T20:28:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6787.079247 secs 2024-04-26T20:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6787.079247 secs 2024-04-26T20:28:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6787.079247 secs 2024-04-26T20:28:27.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7192 sec 2024-04-26T20:28:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6787.783088 secs 2024-04-26T20:28:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:27 smithi090 ceph-mon[29180]: pgmap v4378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6787.783088 secs 2024-04-26T20:28:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:27 smithi119 ceph-mon[32752]: pgmap v4378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6787.783088 secs 2024-04-26T20:28:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:27 smithi167 ceph-mon[32751]: pgmap v4378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:28.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7193 sec 2024-04-26T20:28:29.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7194 sec 2024-04-26T20:28:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:29 smithi090 ceph-mon[29180]: pgmap v4379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:29 smithi119 ceph-mon[32752]: pgmap v4379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:29 smithi167 ceph-mon[32751]: pgmap v4379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:30.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7195 sec 2024-04-26T20:28:31.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7196 sec 2024-04-26T20:28:31.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:31 smithi167 ceph-mon[32751]: pgmap v4380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:31.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6792.079330 secs 2024-04-26T20:28:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:31 smithi090 ceph-mon[29180]: pgmap v4380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6792.079330 secs 2024-04-26T20:28:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:31 smithi119 ceph-mon[32752]: pgmap v4380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6792.079330 secs 2024-04-26T20:28:32.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7197 sec 2024-04-26T20:28:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6792.783179 secs 2024-04-26T20:28:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6792.783179 secs 2024-04-26T20:28:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6792.783179 secs 2024-04-26T20:28:33.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7198 sec 2024-04-26T20:28:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:33 smithi119 ceph-mon[32752]: pgmap v4381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:33 smithi167 ceph-mon[32751]: pgmap v4381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:33 smithi090 ceph-mon[29180]: pgmap v4381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:34.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7199 sec 2024-04-26T20:28:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:34 smithi119 ceph-mon[32752]: pgmap v4382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:34 smithi167 ceph-mon[32751]: pgmap v4382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:34 smithi090 ceph-mon[29180]: pgmap v4382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:35.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7200 sec 2024-04-26T20:28:36.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7201 sec 2024-04-26T20:28:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6797.079412 secs 2024-04-26T20:28:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6797.783298 secs 2024-04-26T20:28:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:36 smithi090 ceph-mon[29180]: pgmap v4383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6797.079412 secs 2024-04-26T20:28:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6797.783298 secs 2024-04-26T20:28:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:36 smithi119 ceph-mon[32752]: pgmap v4383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6797.079412 secs 2024-04-26T20:28:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6797.783298 secs 2024-04-26T20:28:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:36 smithi167 ceph-mon[32751]: pgmap v4383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:37.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7202 sec 2024-04-26T20:28:38.534 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7203 sec 2024-04-26T20:28:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:38 smithi090 ceph-mon[29180]: pgmap v4384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:38 smithi119 ceph-mon[32752]: pgmap v4384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:38 smithi167 ceph-mon[32751]: pgmap v4384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:39.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7204 sec 2024-04-26T20:28:40.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7205 sec 2024-04-26T20:28:41.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7206 sec 2024-04-26T20:28:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:41 smithi119 ceph-mon[32752]: pgmap v4385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6802.079494 secs 2024-04-26T20:28:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:41 smithi167 ceph-mon[32751]: pgmap v4385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6802.079494 secs 2024-04-26T20:28:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:41 smithi090 ceph-mon[29180]: pgmap v4385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6802.079494 secs 2024-04-26T20:28:42.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7207 sec 2024-04-26T20:28:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6802.783360 secs 2024-04-26T20:28:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6802.783360 secs 2024-04-26T20:28:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6802.783360 secs 2024-04-26T20:28:43.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7208 sec 2024-04-26T20:28:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:43 smithi119 ceph-mon[32752]: pgmap v4386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:43 smithi167 ceph-mon[32751]: pgmap v4386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:43 smithi090 ceph-mon[29180]: pgmap v4386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:44.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7209 sec 2024-04-26T20:28:45.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7210 sec 2024-04-26T20:28:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:45 smithi119 ceph-mon[32752]: pgmap v4387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:45 smithi167 ceph-mon[32751]: pgmap v4387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:45 smithi090 ceph-mon[29180]: pgmap v4387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:46.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7211 sec 2024-04-26T20:28:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6807.079569 secs 2024-04-26T20:28:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6807.079569 secs 2024-04-26T20:28:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6807.079569 secs 2024-04-26T20:28:47.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7212 sec 2024-04-26T20:28:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6807.783478 secs 2024-04-26T20:28:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:47 smithi119 ceph-mon[32752]: pgmap v4388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6807.783478 secs 2024-04-26T20:28:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:47 smithi167 ceph-mon[32751]: pgmap v4388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6807.783478 secs 2024-04-26T20:28:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:47 smithi090 ceph-mon[29180]: pgmap v4388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:48.535 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7213 sec 2024-04-26T20:28:49.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7214 sec 2024-04-26T20:28:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:49 smithi119 ceph-mon[32752]: pgmap v4389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:49 smithi167 ceph-mon[32751]: pgmap v4389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:49 smithi090 ceph-mon[29180]: pgmap v4389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:50.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7215 sec 2024-04-26T20:28:51.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7216 sec 2024-04-26T20:28:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:51 smithi119 ceph-mon[32752]: pgmap v4390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6812.079652 secs 2024-04-26T20:28:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:51 smithi167 ceph-mon[32751]: pgmap v4390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6812.079652 secs 2024-04-26T20:28:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:51 smithi090 ceph-mon[29180]: pgmap v4390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6812.079652 secs 2024-04-26T20:28:52.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7217 sec 2024-04-26T20:28:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6812.783557 secs 2024-04-26T20:28:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6812.783557 secs 2024-04-26T20:28:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6812.783557 secs 2024-04-26T20:28:53.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7218 sec 2024-04-26T20:28:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:53 smithi119 ceph-mon[32752]: pgmap v4391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:53 smithi167 ceph-mon[32751]: pgmap v4391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:53 smithi090 ceph-mon[29180]: pgmap v4391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:54.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7219 sec 2024-04-26T20:28:55.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7220 sec 2024-04-26T20:28:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:55 smithi119 ceph-mon[32752]: pgmap v4392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:55 smithi167 ceph-mon[32751]: pgmap v4392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:55 smithi090 ceph-mon[29180]: pgmap v4392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:28:56.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7221 sec 2024-04-26T20:28:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6817.079737 secs 2024-04-26T20:28:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6817.079737 secs 2024-04-26T20:28:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6817.079737 secs 2024-04-26T20:28:57.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7222 sec 2024-04-26T20:28:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6817.783673 secs 2024-04-26T20:28:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:57 smithi119 ceph-mon[32752]: pgmap v4393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6817.783673 secs 2024-04-26T20:28:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:57 smithi167 ceph-mon[32751]: pgmap v4393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6817.783673 secs 2024-04-26T20:28:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:57 smithi090 ceph-mon[29180]: pgmap v4393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:58.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7223 sec 2024-04-26T20:28:59.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7224 sec 2024-04-26T20:28:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:28:59 smithi119 ceph-mon[32752]: pgmap v4394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:28:59 smithi167 ceph-mon[32751]: pgmap v4394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:28:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:28:59 smithi090 ceph-mon[29180]: pgmap v4394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:00.536 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7225 sec 2024-04-26T20:29:01.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7226 sec 2024-04-26T20:29:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:01 smithi119 ceph-mon[32752]: pgmap v4395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6822.079821 secs 2024-04-26T20:29:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:01 smithi167 ceph-mon[32751]: pgmap v4395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6822.079821 secs 2024-04-26T20:29:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:01 smithi090 ceph-mon[29180]: pgmap v4395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6822.079821 secs 2024-04-26T20:29:02.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7227 sec 2024-04-26T20:29:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6822.783794 secs 2024-04-26T20:29:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6822.783794 secs 2024-04-26T20:29:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6822.783794 secs 2024-04-26T20:29:03.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7228 sec 2024-04-26T20:29:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:03 smithi119 ceph-mon[32752]: pgmap v4396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:03 smithi167 ceph-mon[32751]: pgmap v4396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:03 smithi090 ceph-mon[29180]: pgmap v4396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:04.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7229 sec 2024-04-26T20:29:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:29:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:29:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:29:05.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7230 sec 2024-04-26T20:29:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:05 smithi167 ceph-mon[32751]: pgmap v4397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:29:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:29:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:05 smithi167 ceph-mon[32751]: pgmap v4398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:05 smithi167 ceph-mon[32751]: pgmap v4399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:29:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:05 smithi090 ceph-mon[29180]: pgmap v4397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:29:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:29:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:05 smithi090 ceph-mon[29180]: pgmap v4398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:05 smithi090 ceph-mon[29180]: pgmap v4399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:29:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:05 smithi119 ceph-mon[32752]: pgmap v4397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:29:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:29:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:05 smithi119 ceph-mon[32752]: pgmap v4398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:05 smithi119 ceph-mon[32752]: pgmap v4399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:29:06.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7231 sec 2024-04-26T20:29:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6827.079909 secs 2024-04-26T20:29:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6827.079909 secs 2024-04-26T20:29:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6827.079909 secs 2024-04-26T20:29:07.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7232 sec 2024-04-26T20:29:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:07 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6827.783872 secs 2024-04-26T20:29:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:07 smithi167 ceph-mon[32751]: pgmap v4400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:07 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6827.783872 secs 2024-04-26T20:29:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:07 smithi090 ceph-mon[29180]: pgmap v4400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:07 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6827.783872 secs 2024-04-26T20:29:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:07 smithi119 ceph-mon[32752]: pgmap v4400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:08.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7233 sec 2024-04-26T20:29:09.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7234 sec 2024-04-26T20:29:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:09 smithi090 ceph-mon[29180]: pgmap v4401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:09 smithi119 ceph-mon[32752]: pgmap v4401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:09 smithi167 ceph-mon[32751]: pgmap v4401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:10.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7235 sec 2024-04-26T20:29:11.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7236 sec 2024-04-26T20:29:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6832.079972 secs 2024-04-26T20:29:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:11 smithi090 ceph-mon[29180]: pgmap v4402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6832.079972 secs 2024-04-26T20:29:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:11 smithi119 ceph-mon[32752]: pgmap v4402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6832.079972 secs 2024-04-26T20:29:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:11 smithi167 ceph-mon[32751]: pgmap v4402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:29:12.537 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7237 sec 2024-04-26T20:29:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6832.783953 secs 2024-04-26T20:29:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6832.783953 secs 2024-04-26T20:29:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6832.783953 secs 2024-04-26T20:29:13.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7238 sec 2024-04-26T20:29:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:13 smithi119 ceph-mon[32752]: pgmap v4403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:29:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:13 smithi167 ceph-mon[32751]: pgmap v4403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:29:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:13 smithi090 ceph-mon[29180]: pgmap v4403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:29:14.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7239 sec 2024-04-26T20:29:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:14 smithi119 ceph-mon[32752]: pgmap v4404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:29:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:14 smithi167 ceph-mon[32751]: pgmap v4404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:29:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:14 smithi090 ceph-mon[29180]: pgmap v4404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:29:15.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7240 sec 2024-04-26T20:29:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:15 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6837.080061 secs 2024-04-26T20:29:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:15 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6837.080061 secs 2024-04-26T20:29:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:15 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6837.080061 secs 2024-04-26T20:29:16.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7241 sec 2024-04-26T20:29:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6837.784076 secs 2024-04-26T20:29:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:16 smithi119 ceph-mon[32752]: pgmap v4405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6837.784076 secs 2024-04-26T20:29:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:16 smithi167 ceph-mon[32751]: pgmap v4405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6837.784076 secs 2024-04-26T20:29:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:16 smithi090 ceph-mon[29180]: pgmap v4405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:17.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7242 sec 2024-04-26T20:29:18.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7243 sec 2024-04-26T20:29:19.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7244 sec 2024-04-26T20:29:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:19 smithi090 ceph-mon[29180]: pgmap v4406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:19 smithi119 ceph-mon[32752]: pgmap v4406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:19 smithi167 ceph-mon[32751]: pgmap v4406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:20.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7245 sec 2024-04-26T20:29:21.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7246 sec 2024-04-26T20:29:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6842.080146 secs 2024-04-26T20:29:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:21 smithi090 ceph-mon[29180]: pgmap v4407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6842.080146 secs 2024-04-26T20:29:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:21 smithi119 ceph-mon[32752]: pgmap v4407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6842.080146 secs 2024-04-26T20:29:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:21 smithi167 ceph-mon[32751]: pgmap v4407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:22.538 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7247 sec 2024-04-26T20:29:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6842.784205 secs 2024-04-26T20:29:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:22 smithi119 ceph-mon[32752]: pgmap v4408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6842.784205 secs 2024-04-26T20:29:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:22 smithi167 ceph-mon[32751]: pgmap v4408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6842.784205 secs 2024-04-26T20:29:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:22 smithi090 ceph-mon[29180]: pgmap v4408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:23.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7248 sec 2024-04-26T20:29:24.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7249 sec 2024-04-26T20:29:25.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7250 sec 2024-04-26T20:29:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:25 smithi090 ceph-mon[29180]: pgmap v4409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:25 smithi119 ceph-mon[32752]: pgmap v4409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:25 smithi167 ceph-mon[32751]: pgmap v4409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:26.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7251 sec 2024-04-26T20:29:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6847.080229 secs 2024-04-26T20:29:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6847.080229 secs 2024-04-26T20:29:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6847.080229 secs 2024-04-26T20:29:27.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7252 sec 2024-04-26T20:29:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6847.784311 secs 2024-04-26T20:29:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:27 smithi119 ceph-mon[32752]: pgmap v4410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6847.784311 secs 2024-04-26T20:29:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:27 smithi167 ceph-mon[32751]: pgmap v4410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6847.784311 secs 2024-04-26T20:29:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:27 smithi090 ceph-mon[29180]: pgmap v4410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:28.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7253 sec 2024-04-26T20:29:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:28 smithi119 ceph-mon[32752]: pgmap v4411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:28 smithi167 ceph-mon[32751]: pgmap v4411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:28 smithi090 ceph-mon[29180]: pgmap v4411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:29.375 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:29:29.374+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:29:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:29:29.374+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:29:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:29:29.375+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 56 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:29:29.376 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:29:29.375+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 56 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:29:29.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7254 sec 2024-04-26T20:29:30.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7255 sec 2024-04-26T20:29:31.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7256 sec 2024-04-26T20:29:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6852.080314 secs 2024-04-26T20:29:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:31 smithi119 ceph-mon[32752]: pgmap v4412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6852.080314 secs 2024-04-26T20:29:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:31 smithi167 ceph-mon[32751]: pgmap v4412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6852.080314 secs 2024-04-26T20:29:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:31 smithi090 ceph-mon[29180]: pgmap v4412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:32.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7257 sec 2024-04-26T20:29:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6852.784414 secs 2024-04-26T20:29:32.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:32 smithi090 ceph-mon[29180]: pgmap v4413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6852.784414 secs 2024-04-26T20:29:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:32 smithi119 ceph-mon[32752]: pgmap v4413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6852.784414 secs 2024-04-26T20:29:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:32 smithi167 ceph-mon[32751]: pgmap v4413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:33.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7258 sec 2024-04-26T20:29:34.539 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7259 sec 2024-04-26T20:29:35.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7260 sec 2024-04-26T20:29:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:35 smithi119 ceph-mon[32752]: pgmap v4414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:35 smithi167 ceph-mon[32751]: pgmap v4414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:35 smithi090 ceph-mon[29180]: pgmap v4414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:36.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7261 sec 2024-04-26T20:29:36.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6857.080393 secs 2024-04-26T20:29:36.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6857.784492 secs 2024-04-26T20:29:36.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:36 smithi090 ceph-mon[29180]: pgmap v4415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6857.080393 secs 2024-04-26T20:29:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6857.784492 secs 2024-04-26T20:29:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:36 smithi119 ceph-mon[32752]: pgmap v4415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6857.080393 secs 2024-04-26T20:29:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6857.784492 secs 2024-04-26T20:29:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:36 smithi167 ceph-mon[32751]: pgmap v4415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:37.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7262 sec 2024-04-26T20:29:38.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7263 sec 2024-04-26T20:29:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:38 smithi119 ceph-mon[32752]: pgmap v4416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:38 smithi167 ceph-mon[32751]: pgmap v4416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:38 smithi090 ceph-mon[29180]: pgmap v4416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:39.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7264 sec 2024-04-26T20:29:40.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7265 sec 2024-04-26T20:29:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:40 smithi119 ceph-mon[32752]: pgmap v4417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:40 smithi167 ceph-mon[32751]: pgmap v4417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:40 smithi090 ceph-mon[29180]: pgmap v4417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:41.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7266 sec 2024-04-26T20:29:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6862.080472 secs 2024-04-26T20:29:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6862.784570 secs 2024-04-26T20:29:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6862.080472 secs 2024-04-26T20:29:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6862.784570 secs 2024-04-26T20:29:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6862.080472 secs 2024-04-26T20:29:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6862.784570 secs 2024-04-26T20:29:42.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7267 sec 2024-04-26T20:29:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:42 smithi119 ceph-mon[32752]: pgmap v4418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:42 smithi167 ceph-mon[32751]: pgmap v4418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:42 smithi090 ceph-mon[29180]: pgmap v4418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:43.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7268 sec 2024-04-26T20:29:44.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7269 sec 2024-04-26T20:29:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:44 smithi119 ceph-mon[32752]: pgmap v4419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:44 smithi167 ceph-mon[32751]: pgmap v4419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:44 smithi090 ceph-mon[29180]: pgmap v4419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:45.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7270 sec 2024-04-26T20:29:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:45 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6867.080557 secs 2024-04-26T20:29:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:45 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6867.080557 secs 2024-04-26T20:29:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:45 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6867.080557 secs 2024-04-26T20:29:46.540 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7271 sec 2024-04-26T20:29:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6867.784634 secs 2024-04-26T20:29:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:46 smithi119 ceph-mon[32752]: pgmap v4420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6867.784634 secs 2024-04-26T20:29:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:46 smithi167 ceph-mon[32751]: pgmap v4420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6867.784634 secs 2024-04-26T20:29:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:46 smithi090 ceph-mon[29180]: pgmap v4420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:47.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7272 sec 2024-04-26T20:29:48.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7273 sec 2024-04-26T20:29:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:48 smithi119 ceph-mon[32752]: pgmap v4421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:48 smithi167 ceph-mon[32751]: pgmap v4421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:48 smithi090 ceph-mon[29180]: pgmap v4421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:49.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7274 sec 2024-04-26T20:29:50.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7275 sec 2024-04-26T20:29:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:50 smithi119 ceph-mon[32752]: pgmap v4422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:50 smithi167 ceph-mon[32751]: pgmap v4422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:50 smithi090 ceph-mon[29180]: pgmap v4422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:51.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7276 sec 2024-04-26T20:29:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6872.080641 secs 2024-04-26T20:29:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6872.784752 secs 2024-04-26T20:29:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6872.080641 secs 2024-04-26T20:29:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6872.784752 secs 2024-04-26T20:29:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6872.080641 secs 2024-04-26T20:29:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6872.784752 secs 2024-04-26T20:29:52.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7277 sec 2024-04-26T20:29:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:52 smithi119 ceph-mon[32752]: pgmap v4423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:52 smithi167 ceph-mon[32751]: pgmap v4423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:52 smithi090 ceph-mon[29180]: pgmap v4423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:53.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7278 sec 2024-04-26T20:29:54.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7279 sec 2024-04-26T20:29:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:54 smithi119 ceph-mon[32752]: pgmap v4424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:54 smithi167 ceph-mon[32751]: pgmap v4424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:54 smithi090 ceph-mon[29180]: pgmap v4424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:55.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7280 sec 2024-04-26T20:29:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6877.080727 secs 2024-04-26T20:29:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6877.080727 secs 2024-04-26T20:29:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6877.080727 secs 2024-04-26T20:29:56.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7281 sec 2024-04-26T20:29:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6877.784816 secs 2024-04-26T20:29:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:56 smithi119 ceph-mon[32752]: pgmap v4425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6877.784816 secs 2024-04-26T20:29:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:56 smithi167 ceph-mon[32751]: pgmap v4425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6877.784816 secs 2024-04-26T20:29:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:56 smithi090 ceph-mon[29180]: pgmap v4425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:29:57.541 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7282 sec 2024-04-26T20:29:58.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7283 sec 2024-04-26T20:29:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:29:58 smithi119 ceph-mon[32752]: pgmap v4426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:29:58 smithi167 ceph-mon[32751]: pgmap v4426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:29:58 smithi090 ceph-mon[29180]: pgmap v4426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:29:59.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7284 sec 2024-04-26T20:30:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:30:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:30:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:30:00.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7285 sec 2024-04-26T20:30:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6882.080811 secs 2024-04-26T20:30:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:01 smithi167 ceph-mon[32751]: pgmap v4427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6882.080811 secs 2024-04-26T20:30:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:01 smithi090 ceph-mon[29180]: pgmap v4427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:01.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7286 sec 2024-04-26T20:30:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6882.080811 secs 2024-04-26T20:30:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:01 smithi119 ceph-mon[32752]: pgmap v4427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6882.784895 secs 2024-04-26T20:30:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6882.784895 secs 2024-04-26T20:30:02.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7287 sec 2024-04-26T20:30:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6882.784895 secs 2024-04-26T20:30:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:03 smithi167 ceph-mon[32751]: pgmap v4428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:03 smithi090 ceph-mon[29180]: pgmap v4428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:03.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7288 sec 2024-04-26T20:30:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:03 smithi119 ceph-mon[32752]: pgmap v4428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:04.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7289 sec 2024-04-26T20:30:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:04 smithi119 ceph-mon[32752]: pgmap v4429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:30:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:04 smithi167 ceph-mon[32751]: pgmap v4429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:30:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:04 smithi090 ceph-mon[29180]: pgmap v4429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:30:05.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7290 sec 2024-04-26T20:30:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:30:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:30:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:05 smithi119 ceph-mon[32752]: pgmap v4430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:30:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:05 smithi119 ceph-mon[32752]: pgmap v4431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:05 smithi167 ceph-mon[32751]: pgmap v4430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:05 smithi167 ceph-mon[32751]: pgmap v4431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:30:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:30:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:30:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:05 smithi090 ceph-mon[29180]: pgmap v4430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:30:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:05 smithi090 ceph-mon[29180]: pgmap v4431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:30:06.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7291 sec 2024-04-26T20:30:06.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6887.080876 secs 2024-04-26T20:30:06.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6887.784972 secs 2024-04-26T20:30:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6887.080876 secs 2024-04-26T20:30:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6887.784972 secs 2024-04-26T20:30:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6887.080876 secs 2024-04-26T20:30:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6887.784972 secs 2024-04-26T20:30:07.542 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7292 sec 2024-04-26T20:30:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:07 smithi119 ceph-mon[32752]: pgmap v4432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:30:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:07 smithi167 ceph-mon[32751]: pgmap v4432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:30:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:07 smithi090 ceph-mon[29180]: pgmap v4432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:30:08.543 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7293 sec 2024-04-26T20:30:09.543 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7294 sec 2024-04-26T20:30:10.543 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7295 sec 2024-04-26T20:30:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:10 smithi119 ceph-mon[32752]: pgmap v4433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:10 smithi167 ceph-mon[32751]: pgmap v4433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:10 smithi090 ceph-mon[29180]: pgmap v4433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:11.543 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7296 sec 2024-04-26T20:30:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6892.080936 secs 2024-04-26T20:30:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6892.080936 secs 2024-04-26T20:30:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6892.080936 secs 2024-04-26T20:30:12.543 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7297 sec 2024-04-26T20:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:12 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6892.785088 secs 2024-04-26T20:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:12 smithi119 ceph-mon[32752]: pgmap v4434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:12 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6892.785088 secs 2024-04-26T20:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:12 smithi167 ceph-mon[32751]: pgmap v4434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:12 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6892.785088 secs 2024-04-26T20:30:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:12 smithi090 ceph-mon[29180]: pgmap v4434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:30:13.543 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7298 sec 2024-04-26T20:30:14.543 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7299 sec 2024-04-26T20:30:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:14 smithi119 ceph-mon[32752]: pgmap v4435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:30:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:14 smithi167 ceph-mon[32751]: pgmap v4435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:30:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:14 smithi090 ceph-mon[29180]: pgmap v4435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:30:15.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7300 sec 2024-04-26T20:30:16.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7301 sec 2024-04-26T20:30:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6897.080999 secs 2024-04-26T20:30:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:16 smithi119 ceph-mon[32752]: pgmap v4436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:30:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6897.080999 secs 2024-04-26T20:30:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:16 smithi167 ceph-mon[32751]: pgmap v4436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:30:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6897.080999 secs 2024-04-26T20:30:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:16 smithi090 ceph-mon[29180]: pgmap v4436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:30:17.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7302 sec 2024-04-26T20:30:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:17 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6897.785203 secs 2024-04-26T20:30:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:17 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6897.785203 secs 2024-04-26T20:30:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:17 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6897.785203 secs 2024-04-26T20:30:18.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7303 sec 2024-04-26T20:30:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:18 smithi119 ceph-mon[32752]: pgmap v4437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:18 smithi167 ceph-mon[32751]: pgmap v4437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:18 smithi090 ceph-mon[29180]: pgmap v4437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:19.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7304 sec 2024-04-26T20:30:20.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7305 sec 2024-04-26T20:30:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:20 smithi119 ceph-mon[32752]: pgmap v4438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:20 smithi167 ceph-mon[32751]: pgmap v4438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:20 smithi090 ceph-mon[29180]: pgmap v4438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:21.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7306 sec 2024-04-26T20:30:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6902.081099 secs 2024-04-26T20:30:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6902.081099 secs 2024-04-26T20:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6902.081099 secs 2024-04-26T20:30:22.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7307 sec 2024-04-26T20:30:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:22 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6902.785318 secs 2024-04-26T20:30:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:22 smithi119 ceph-mon[32752]: pgmap v4439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:22 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6902.785318 secs 2024-04-26T20:30:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:22 smithi167 ceph-mon[32751]: pgmap v4439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:22 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6902.785318 secs 2024-04-26T20:30:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:22 smithi090 ceph-mon[29180]: pgmap v4439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:23.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7308 sec 2024-04-26T20:30:24.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7309 sec 2024-04-26T20:30:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:24 smithi119 ceph-mon[32752]: pgmap v4440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:24 smithi167 ceph-mon[32751]: pgmap v4440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:24 smithi090 ceph-mon[29180]: pgmap v4440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:25.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7310 sec 2024-04-26T20:30:26.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7311 sec 2024-04-26T20:30:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6907.081184 secs 2024-04-26T20:30:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:26 smithi119 ceph-mon[32752]: pgmap v4441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6907.081184 secs 2024-04-26T20:30:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:26 smithi167 ceph-mon[32751]: pgmap v4441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6907.081184 secs 2024-04-26T20:30:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:26 smithi090 ceph-mon[29180]: pgmap v4441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:27.544 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7312 sec 2024-04-26T20:30:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:27 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6907.785430 secs 2024-04-26T20:30:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:27 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6907.785430 secs 2024-04-26T20:30:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:27 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6907.785430 secs 2024-04-26T20:30:28.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7313 sec 2024-04-26T20:30:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:28 smithi119 ceph-mon[32752]: pgmap v4442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:28 smithi167 ceph-mon[32751]: pgmap v4442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:28 smithi090 ceph-mon[29180]: pgmap v4442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:29.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7314 sec 2024-04-26T20:30:30.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7315 sec 2024-04-26T20:30:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:30 smithi119 ceph-mon[32752]: pgmap v4443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:30 smithi167 ceph-mon[32751]: pgmap v4443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:30 smithi090 ceph-mon[29180]: pgmap v4443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:31.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6912.081264 secs 2024-04-26T20:30:31.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7316 sec 2024-04-26T20:30:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6912.081264 secs 2024-04-26T20:30:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6912.081264 secs 2024-04-26T20:30:32.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7317 sec 2024-04-26T20:30:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:32 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6912.785495 secs 2024-04-26T20:30:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:32 smithi119 ceph-mon[32752]: pgmap v4444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:32 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6912.785495 secs 2024-04-26T20:30:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:32 smithi167 ceph-mon[32751]: pgmap v4444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:32 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6912.785495 secs 2024-04-26T20:30:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:32 smithi090 ceph-mon[29180]: pgmap v4444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:33.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7318 sec 2024-04-26T20:30:34.470 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:34 smithi090 ceph-mon[29180]: pgmap v4445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:34.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7319 sec 2024-04-26T20:30:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:34 smithi119 ceph-mon[32752]: pgmap v4445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:34 smithi167 ceph-mon[32751]: pgmap v4445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:35.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7320 sec 2024-04-26T20:30:36.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7321 sec 2024-04-26T20:30:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6917.081345 secs 2024-04-26T20:30:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:36 smithi119 ceph-mon[32752]: pgmap v4446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6917.081345 secs 2024-04-26T20:30:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:36 smithi167 ceph-mon[32751]: pgmap v4446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6917.081345 secs 2024-04-26T20:30:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:36 smithi090 ceph-mon[29180]: pgmap v4446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:37.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7322 sec 2024-04-26T20:30:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:37 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6917.785569 secs 2024-04-26T20:30:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:37 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6917.785569 secs 2024-04-26T20:30:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:37 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6917.785569 secs 2024-04-26T20:30:38.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7323 sec 2024-04-26T20:30:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:38 smithi119 ceph-mon[32752]: pgmap v4447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:38 smithi167 ceph-mon[32751]: pgmap v4447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:38 smithi090 ceph-mon[29180]: pgmap v4447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:39.545 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7324 sec 2024-04-26T20:30:40.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7325 sec 2024-04-26T20:30:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:40 smithi119 ceph-mon[32752]: pgmap v4448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:40 smithi167 ceph-mon[32751]: pgmap v4448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:40 smithi090 ceph-mon[29180]: pgmap v4448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:41.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7326 sec 2024-04-26T20:30:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6922.081428 secs 2024-04-26T20:30:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6922.081428 secs 2024-04-26T20:30:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6922.081428 secs 2024-04-26T20:30:42.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7327 sec 2024-04-26T20:30:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:42 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6922.785679 secs 2024-04-26T20:30:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:42 smithi119 ceph-mon[32752]: pgmap v4449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:42 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6922.785679 secs 2024-04-26T20:30:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:42 smithi167 ceph-mon[32751]: pgmap v4449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:42 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6922.785679 secs 2024-04-26T20:30:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:42 smithi090 ceph-mon[29180]: pgmap v4449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:43.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7328 sec 2024-04-26T20:30:44.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7329 sec 2024-04-26T20:30:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:44 smithi119 ceph-mon[32752]: pgmap v4450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:44 smithi167 ceph-mon[32751]: pgmap v4450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:44 smithi090 ceph-mon[29180]: pgmap v4450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:45.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7330 sec 2024-04-26T20:30:46.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7331 sec 2024-04-26T20:30:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6927.081509 secs 2024-04-26T20:30:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:46 smithi119 ceph-mon[32752]: pgmap v4451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6927.081509 secs 2024-04-26T20:30:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:46 smithi167 ceph-mon[32751]: pgmap v4451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6927.081509 secs 2024-04-26T20:30:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:46 smithi090 ceph-mon[29180]: pgmap v4451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:47.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7332 sec 2024-04-26T20:30:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6927.785776 secs 2024-04-26T20:30:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6927.785776 secs 2024-04-26T20:30:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6927.785776 secs 2024-04-26T20:30:48.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7333 sec 2024-04-26T20:30:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:48 smithi119 ceph-mon[32752]: pgmap v4452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:48 smithi167 ceph-mon[32751]: pgmap v4452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:48 smithi090 ceph-mon[29180]: pgmap v4452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:49.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7334 sec 2024-04-26T20:30:50.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7335 sec 2024-04-26T20:30:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:50 smithi119 ceph-mon[32752]: pgmap v4453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:50 smithi167 ceph-mon[32751]: pgmap v4453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:50 smithi090 ceph-mon[29180]: pgmap v4453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:51.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7336 sec 2024-04-26T20:30:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6932.081594 secs 2024-04-26T20:30:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6932.081594 secs 2024-04-26T20:30:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6932.081594 secs 2024-04-26T20:30:52.546 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7337 sec 2024-04-26T20:30:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:52 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6932.785874 secs 2024-04-26T20:30:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:52 smithi119 ceph-mon[32752]: pgmap v4454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:52 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6932.785874 secs 2024-04-26T20:30:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:52 smithi167 ceph-mon[32751]: pgmap v4454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:52 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6932.785874 secs 2024-04-26T20:30:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:52 smithi090 ceph-mon[29180]: pgmap v4454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:53.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7338 sec 2024-04-26T20:30:54.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7339 sec 2024-04-26T20:30:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:54 smithi119 ceph-mon[32752]: pgmap v4455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:54 smithi167 ceph-mon[32751]: pgmap v4455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:54 smithi090 ceph-mon[29180]: pgmap v4455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:55.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7340 sec 2024-04-26T20:30:56.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7341 sec 2024-04-26T20:30:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6937.081676 secs 2024-04-26T20:30:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:56 smithi119 ceph-mon[32752]: pgmap v4456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6937.081676 secs 2024-04-26T20:30:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:56 smithi167 ceph-mon[32751]: pgmap v4456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6937.081676 secs 2024-04-26T20:30:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:56 smithi090 ceph-mon[29180]: pgmap v4456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:30:57.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7342 sec 2024-04-26T20:30:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:57 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6937.785993 secs 2024-04-26T20:30:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:57 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6937.785993 secs 2024-04-26T20:30:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:57 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6937.785993 secs 2024-04-26T20:30:58.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7343 sec 2024-04-26T20:30:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:30:58 smithi119 ceph-mon[32752]: pgmap v4457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:30:58 smithi167 ceph-mon[32751]: pgmap v4457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:30:58 smithi090 ceph-mon[29180]: pgmap v4457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:30:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:30:59.387+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:30:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:30:59.387+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:30:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:30:59.387+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 57 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:30:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:30:59.387+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 57 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:30:59.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7344 sec 2024-04-26T20:31:00.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7345 sec 2024-04-26T20:31:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:00 smithi119 ceph-mon[32752]: pgmap v4458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:00 smithi167 ceph-mon[32751]: pgmap v4458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:00 smithi090 ceph-mon[29180]: pgmap v4458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:01.547 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7346 sec 2024-04-26T20:31:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6942.081759 secs 2024-04-26T20:31:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6942.081759 secs 2024-04-26T20:31:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6942.081759 secs 2024-04-26T20:31:02.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7347 sec 2024-04-26T20:31:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:02 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 6942.786142 secs 2024-04-26T20:31:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:02 smithi167 ceph-mon[32751]: pgmap v4459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:02 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 6942.786142 secs 2024-04-26T20:31:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:02 smithi090 ceph-mon[29180]: pgmap v4459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:02 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 6942.786142 secs 2024-04-26T20:31:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:02 smithi119 ceph-mon[32752]: pgmap v4459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:03.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7348 sec 2024-04-26T20:31:04.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7349 sec 2024-04-26T20:31:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:04 smithi167 ceph-mon[32751]: pgmap v4460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:04 smithi090 ceph-mon[29180]: pgmap v4460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:04 smithi119 ceph-mon[32752]: pgmap v4460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:05.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7350 sec 2024-04-26T20:31:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:31:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:31:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:31:06.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7351 sec 2024-04-26T20:31:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6947.081842 secs 2024-04-26T20:31:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:06 smithi090 ceph-mon[29180]: pgmap v4461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:31:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:31:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:31:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6947.081842 secs 2024-04-26T20:31:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:06 smithi119 ceph-mon[32752]: pgmap v4461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:31:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:31:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:31:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6947.081842 secs 2024-04-26T20:31:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:06 smithi167 ceph-mon[32751]: pgmap v4461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:31:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:31:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:31:07.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7352 sec 2024-04-26T20:31:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:07 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 6947.786205 secs 2024-04-26T20:31:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:07 smithi090 ceph-mon[29180]: slow request 182.947393 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T20:31:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:07 smithi090 ceph-mon[29180]: pgmap v4462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:31:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:07 smithi090 ceph-mon[29180]: pgmap v4463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T20:31:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:31:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:07 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 6947.786205 secs 2024-04-26T20:31:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:07 smithi119 ceph-mon[32752]: slow request 182.947393 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T20:31:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:07 smithi119 ceph-mon[32752]: pgmap v4462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:31:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:07 smithi119 ceph-mon[32752]: pgmap v4463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T20:31:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:31:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:07 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 6947.786205 secs 2024-04-26T20:31:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:07 smithi167 ceph-mon[32751]: slow request 182.947393 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T20:31:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:07 smithi167 ceph-mon[32751]: pgmap v4462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:31:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:07 smithi167 ceph-mon[32751]: pgmap v4463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T20:31:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:31:08.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7353 sec 2024-04-26T20:31:09.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7354 sec 2024-04-26T20:31:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:09 smithi090 ceph-mon[29180]: pgmap v4464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:31:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:09 smithi119 ceph-mon[32752]: pgmap v4464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:31:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:09 smithi167 ceph-mon[32751]: pgmap v4464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:31:10.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7355 sec 2024-04-26T20:31:11.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7356 sec 2024-04-26T20:31:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6952.081924 secs 2024-04-26T20:31:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:11 smithi090 ceph-mon[29180]: pgmap v4465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6952.081924 secs 2024-04-26T20:31:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:11 smithi119 ceph-mon[32752]: pgmap v4465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6952.081924 secs 2024-04-26T20:31:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:11 smithi167 ceph-mon[32751]: pgmap v4465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:12.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7357 sec 2024-04-26T20:31:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6952.786346 secs 2024-04-26T20:31:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6952.786346 secs 2024-04-26T20:31:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6952.786346 secs 2024-04-26T20:31:13.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7358 sec 2024-04-26T20:31:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:13 smithi090 ceph-mon[29180]: pgmap v4466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:13 smithi119 ceph-mon[32752]: pgmap v4466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:13 smithi167 ceph-mon[32751]: pgmap v4466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:14.548 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7359 sec 2024-04-26T20:31:15.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7360 sec 2024-04-26T20:31:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:15 smithi090 ceph-mon[29180]: pgmap v4467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:15 smithi119 ceph-mon[32752]: pgmap v4467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:15 smithi167 ceph-mon[32751]: pgmap v4467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:31:16.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7361 sec 2024-04-26T20:31:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6957.082008 secs 2024-04-26T20:31:16.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6957.082008 secs 2024-04-26T20:31:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6957.082008 secs 2024-04-26T20:31:17.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7362 sec 2024-04-26T20:31:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6957.786461 secs 2024-04-26T20:31:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:17 smithi090 ceph-mon[29180]: pgmap v4468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:31:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6957.786461 secs 2024-04-26T20:31:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:17 smithi119 ceph-mon[32752]: pgmap v4468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:31:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6957.786461 secs 2024-04-26T20:31:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:17 smithi167 ceph-mon[32751]: pgmap v4468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:31:18.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7363 sec 2024-04-26T20:31:19.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7364 sec 2024-04-26T20:31:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:19 smithi090 ceph-mon[29180]: pgmap v4469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:19 smithi119 ceph-mon[32752]: pgmap v4469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:19 smithi167 ceph-mon[32751]: pgmap v4469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:20.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7365 sec 2024-04-26T20:31:21.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7366 sec 2024-04-26T20:31:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6962.082091 secs 2024-04-26T20:31:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:21 smithi090 ceph-mon[29180]: pgmap v4470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6962.082091 secs 2024-04-26T20:31:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:21 smithi119 ceph-mon[32752]: pgmap v4470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6962.082091 secs 2024-04-26T20:31:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:21 smithi167 ceph-mon[32751]: pgmap v4470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:22.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7367 sec 2024-04-26T20:31:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6962.786580 secs 2024-04-26T20:31:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6962.786580 secs 2024-04-26T20:31:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6962.786580 secs 2024-04-26T20:31:23.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7368 sec 2024-04-26T20:31:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:23 smithi090 ceph-mon[29180]: pgmap v4471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:23 smithi119 ceph-mon[32752]: pgmap v4471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:23 smithi167 ceph-mon[32751]: pgmap v4471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:24.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7369 sec 2024-04-26T20:31:25.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7370 sec 2024-04-26T20:31:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:25 smithi090 ceph-mon[29180]: pgmap v4472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:25 smithi119 ceph-mon[32752]: pgmap v4472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:25 smithi167 ceph-mon[32751]: pgmap v4472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:26.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7371 sec 2024-04-26T20:31:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6967.082174 secs 2024-04-26T20:31:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6967.082174 secs 2024-04-26T20:31:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6967.082174 secs 2024-04-26T20:31:27.549 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7372 sec 2024-04-26T20:31:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6967.786699 secs 2024-04-26T20:31:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:27 smithi090 ceph-mon[29180]: pgmap v4473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6967.786699 secs 2024-04-26T20:31:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:27 smithi119 ceph-mon[32752]: pgmap v4473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6967.786699 secs 2024-04-26T20:31:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:27 smithi167 ceph-mon[32751]: pgmap v4473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:28.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7373 sec 2024-04-26T20:31:29.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7374 sec 2024-04-26T20:31:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:29 smithi090 ceph-mon[29180]: pgmap v4474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:29 smithi119 ceph-mon[32752]: pgmap v4474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:29 smithi167 ceph-mon[32751]: pgmap v4474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:30.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7375 sec 2024-04-26T20:31:31.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7376 sec 2024-04-26T20:31:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6972.082260 secs 2024-04-26T20:31:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:31 smithi119 ceph-mon[32752]: pgmap v4475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6972.082260 secs 2024-04-26T20:31:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:31 smithi167 ceph-mon[32751]: pgmap v4475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6972.082260 secs 2024-04-26T20:31:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:31 smithi090 ceph-mon[29180]: pgmap v4475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:32.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7377 sec 2024-04-26T20:31:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6972.786813 secs 2024-04-26T20:31:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6972.786813 secs 2024-04-26T20:31:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6972.786813 secs 2024-04-26T20:31:33.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7378 sec 2024-04-26T20:31:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:33 smithi119 ceph-mon[32752]: pgmap v4476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:33 smithi167 ceph-mon[32751]: pgmap v4476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:33 smithi090 ceph-mon[29180]: pgmap v4476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:34.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7379 sec 2024-04-26T20:31:35.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7380 sec 2024-04-26T20:31:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:35 smithi119 ceph-mon[32752]: pgmap v4477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:35 smithi167 ceph-mon[32751]: pgmap v4477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:35 smithi090 ceph-mon[29180]: pgmap v4477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:36.550 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7381 sec 2024-04-26T20:31:36.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6977.082345 secs 2024-04-26T20:31:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6977.082345 secs 2024-04-26T20:31:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6977.082345 secs 2024-04-26T20:31:37.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7382 sec 2024-04-26T20:31:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6977.786936 secs 2024-04-26T20:31:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:37 smithi119 ceph-mon[32752]: pgmap v4478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6977.786936 secs 2024-04-26T20:31:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:37 smithi167 ceph-mon[32751]: pgmap v4478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6977.786936 secs 2024-04-26T20:31:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:37 smithi090 ceph-mon[29180]: pgmap v4478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:38.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7383 sec 2024-04-26T20:31:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:38 smithi119 ceph-mon[32752]: pgmap v4479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:38 smithi167 ceph-mon[32751]: pgmap v4479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:38 smithi090 ceph-mon[29180]: pgmap v4479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:39.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7384 sec 2024-04-26T20:31:40.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7385 sec 2024-04-26T20:31:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:40 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6982.082428 secs 2024-04-26T20:31:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:40 smithi119 ceph-mon[32752]: pgmap v4480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:40 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6982.082428 secs 2024-04-26T20:31:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:40 smithi167 ceph-mon[32751]: pgmap v4480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:40 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6982.082428 secs 2024-04-26T20:31:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:40 smithi090 ceph-mon[29180]: pgmap v4480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:41.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7386 sec 2024-04-26T20:31:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6982.787055 secs 2024-04-26T20:31:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6982.787055 secs 2024-04-26T20:31:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6982.787055 secs 2024-04-26T20:31:42.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7387 sec 2024-04-26T20:31:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:42 smithi119 ceph-mon[32752]: pgmap v4481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:42 smithi167 ceph-mon[32751]: pgmap v4481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:42 smithi090 ceph-mon[29180]: pgmap v4481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:43.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7388 sec 2024-04-26T20:31:44.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7389 sec 2024-04-26T20:31:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:44 smithi119 ceph-mon[32752]: pgmap v4482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:44 smithi167 ceph-mon[32751]: pgmap v4482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:44 smithi090 ceph-mon[29180]: pgmap v4482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:45.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7390 sec 2024-04-26T20:31:46.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:45 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6987.082514 secs 2024-04-26T20:31:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:45 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6987.082514 secs 2024-04-26T20:31:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:45 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6987.082514 secs 2024-04-26T20:31:46.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7391 sec 2024-04-26T20:31:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6987.787166 secs 2024-04-26T20:31:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:46 smithi119 ceph-mon[32752]: pgmap v4483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6987.787166 secs 2024-04-26T20:31:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:46 smithi167 ceph-mon[32751]: pgmap v4483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6987.787166 secs 2024-04-26T20:31:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:46 smithi090 ceph-mon[29180]: pgmap v4483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:47.551 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7392 sec 2024-04-26T20:31:48.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7393 sec 2024-04-26T20:31:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:48 smithi119 ceph-mon[32752]: pgmap v4484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:48 smithi167 ceph-mon[32751]: pgmap v4484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:48 smithi090 ceph-mon[29180]: pgmap v4484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:49.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7394 sec 2024-04-26T20:31:50.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7395 sec 2024-04-26T20:31:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:50 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6992.082598 secs 2024-04-26T20:31:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:50 smithi119 ceph-mon[32752]: pgmap v4485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:50 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6992.082598 secs 2024-04-26T20:31:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:50 smithi167 ceph-mon[32751]: pgmap v4485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:50 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6992.082598 secs 2024-04-26T20:31:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:50 smithi090 ceph-mon[29180]: pgmap v4485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:51.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7396 sec 2024-04-26T20:31:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6992.787279 secs 2024-04-26T20:31:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6992.787279 secs 2024-04-26T20:31:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6992.787279 secs 2024-04-26T20:31:52.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7397 sec 2024-04-26T20:31:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:52 smithi119 ceph-mon[32752]: pgmap v4486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:52 smithi167 ceph-mon[32751]: pgmap v4486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:52 smithi090 ceph-mon[29180]: pgmap v4486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:31:53.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7398 sec 2024-04-26T20:31:54.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7399 sec 2024-04-26T20:31:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:54 smithi119 ceph-mon[32752]: pgmap v4487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:54 smithi167 ceph-mon[32751]: pgmap v4487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:54 smithi090 ceph-mon[29180]: pgmap v4487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:31:55.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7400 sec 2024-04-26T20:31:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 6997.082682 secs 2024-04-26T20:31:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 6997.082682 secs 2024-04-26T20:31:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 6997.082682 secs 2024-04-26T20:31:56.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7401 sec 2024-04-26T20:31:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 6997.787419 secs 2024-04-26T20:31:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:56 smithi119 ceph-mon[32752]: pgmap v4488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 6997.787419 secs 2024-04-26T20:31:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:56 smithi167 ceph-mon[32751]: pgmap v4488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 6997.787419 secs 2024-04-26T20:31:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:56 smithi090 ceph-mon[29180]: pgmap v4488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:57.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7402 sec 2024-04-26T20:31:58.552 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7403 sec 2024-04-26T20:31:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:31:58 smithi119 ceph-mon[32752]: pgmap v4489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:31:58 smithi167 ceph-mon[32751]: pgmap v4489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:31:58 smithi090 ceph-mon[29180]: pgmap v4489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:31:59.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7404 sec 2024-04-26T20:32:00.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7405 sec 2024-04-26T20:32:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:00 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7002.082767 secs 2024-04-26T20:32:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:00 smithi119 ceph-mon[32752]: pgmap v4490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:00 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7002.082767 secs 2024-04-26T20:32:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:00 smithi167 ceph-mon[32751]: pgmap v4490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:00 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7002.082767 secs 2024-04-26T20:32:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:00 smithi090 ceph-mon[29180]: pgmap v4490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:01.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7406 sec 2024-04-26T20:32:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7002.787530 secs 2024-04-26T20:32:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7002.787530 secs 2024-04-26T20:32:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7002.787530 secs 2024-04-26T20:32:02.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7407 sec 2024-04-26T20:32:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:02 smithi119 ceph-mon[32752]: pgmap v4491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:02 smithi167 ceph-mon[32751]: pgmap v4491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:02 smithi090 ceph-mon[29180]: pgmap v4491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:03.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7408 sec 2024-04-26T20:32:04.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7409 sec 2024-04-26T20:32:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:04 smithi119 ceph-mon[32752]: pgmap v4492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:04 smithi167 ceph-mon[32751]: pgmap v4492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:04 smithi090 ceph-mon[29180]: pgmap v4492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:05.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7410 sec 2024-04-26T20:32:06.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7007.082801 secs 2024-04-26T20:32:06.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:32:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7007.082801 secs 2024-04-26T20:32:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:32:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7007.082801 secs 2024-04-26T20:32:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:32:06.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7411 sec 2024-04-26T20:32:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7007.787644 secs 2024-04-26T20:32:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:07 smithi119 ceph-mon[32752]: pgmap v4493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:32:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:32:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:32:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7007.787644 secs 2024-04-26T20:32:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:07 smithi167 ceph-mon[32751]: pgmap v4493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:32:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:32:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:32:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7007.787644 secs 2024-04-26T20:32:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:07 smithi090 ceph-mon[29180]: pgmap v4493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:32:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:32:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:32:07.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7412 sec 2024-04-26T20:32:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:08 smithi119 ceph-mon[32752]: pgmap v4494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:32:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:08 smithi119 ceph-mon[32752]: pgmap v4495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:08 smithi167 ceph-mon[32751]: pgmap v4494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:32:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:08 smithi167 ceph-mon[32751]: pgmap v4495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:08 smithi090 ceph-mon[29180]: pgmap v4494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:32:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:08 smithi090 ceph-mon[29180]: pgmap v4495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:08.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7413 sec 2024-04-26T20:32:09.553 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7414 sec 2024-04-26T20:32:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:10 smithi119 ceph-mon[32752]: pgmap v4496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:10 smithi167 ceph-mon[32751]: pgmap v4496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:10 smithi090 ceph-mon[29180]: pgmap v4496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:10.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7415 sec 2024-04-26T20:32:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7012.082886 secs 2024-04-26T20:32:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7012.082886 secs 2024-04-26T20:32:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7012.082886 secs 2024-04-26T20:32:11.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7416 sec 2024-04-26T20:32:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7012.787762 secs 2024-04-26T20:32:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:12 smithi119 ceph-mon[32752]: pgmap v4497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7012.787762 secs 2024-04-26T20:32:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:12 smithi167 ceph-mon[32751]: pgmap v4497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7012.787762 secs 2024-04-26T20:32:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:12 smithi090 ceph-mon[29180]: pgmap v4497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:12.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7417 sec 2024-04-26T20:32:13.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7418 sec 2024-04-26T20:32:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:14 smithi119 ceph-mon[32752]: pgmap v4498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:32:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:14 smithi167 ceph-mon[32751]: pgmap v4498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:32:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:14 smithi090 ceph-mon[29180]: pgmap v4498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:32:14.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7419 sec 2024-04-26T20:32:15.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7420 sec 2024-04-26T20:32:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:16 smithi119 ceph-mon[32752]: pgmap v4499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7017.082948 secs 2024-04-26T20:32:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:16 smithi167 ceph-mon[32751]: pgmap v4499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7017.082948 secs 2024-04-26T20:32:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:16 smithi090 ceph-mon[29180]: pgmap v4499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:32:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7017.082948 secs 2024-04-26T20:32:16.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7421 sec 2024-04-26T20:32:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7017.787894 secs 2024-04-26T20:32:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7017.787894 secs 2024-04-26T20:32:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7017.787894 secs 2024-04-26T20:32:17.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7422 sec 2024-04-26T20:32:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:18 smithi119 ceph-mon[32752]: pgmap v4500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:32:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:18 smithi167 ceph-mon[32751]: pgmap v4500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:32:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:18 smithi090 ceph-mon[29180]: pgmap v4500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:32:18.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7423 sec 2024-04-26T20:32:19.554 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7424 sec 2024-04-26T20:32:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:20 smithi119 ceph-mon[32752]: pgmap v4501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:20 smithi167 ceph-mon[32751]: pgmap v4501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:20 smithi090 ceph-mon[29180]: pgmap v4501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:20.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7425 sec 2024-04-26T20:32:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7022.083032 secs 2024-04-26T20:32:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7022.083032 secs 2024-04-26T20:32:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7022.083032 secs 2024-04-26T20:32:21.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7426 sec 2024-04-26T20:32:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7022.788014 secs 2024-04-26T20:32:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:22 smithi119 ceph-mon[32752]: pgmap v4502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7022.788014 secs 2024-04-26T20:32:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:22 smithi167 ceph-mon[32751]: pgmap v4502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7022.788014 secs 2024-04-26T20:32:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:22 smithi090 ceph-mon[29180]: pgmap v4502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:22.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7427 sec 2024-04-26T20:32:23.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7428 sec 2024-04-26T20:32:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:24 smithi119 ceph-mon[32752]: pgmap v4503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:24 smithi167 ceph-mon[32751]: pgmap v4503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:24 smithi090 ceph-mon[29180]: pgmap v4503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:24.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7429 sec 2024-04-26T20:32:25.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7430 sec 2024-04-26T20:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:26 smithi167 ceph-mon[32751]: pgmap v4504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7027.083093 secs 2024-04-26T20:32:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:26 smithi090 ceph-mon[29180]: pgmap v4504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7027.083093 secs 2024-04-26T20:32:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:26 smithi119 ceph-mon[32752]: pgmap v4504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7027.083093 secs 2024-04-26T20:32:26.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7431 sec 2024-04-26T20:32:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7027.788130 secs 2024-04-26T20:32:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7027.788130 secs 2024-04-26T20:32:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7027.788130 secs 2024-04-26T20:32:27.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7432 sec 2024-04-26T20:32:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:28 smithi167 ceph-mon[32751]: pgmap v4505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:28 smithi090 ceph-mon[29180]: pgmap v4505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:28 smithi119 ceph-mon[32752]: pgmap v4505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:28.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7433 sec 2024-04-26T20:32:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:32:29.399+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:32:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:32:29.399+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:32:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:32:29.400+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 58 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:32:29.401 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:32:29.400+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 58 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:32:29.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7434 sec 2024-04-26T20:32:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:30 smithi090 ceph-mon[29180]: pgmap v4506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:30 smithi119 ceph-mon[32752]: pgmap v4506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:30.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7435 sec 2024-04-26T20:32:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:30 smithi167 ceph-mon[32751]: pgmap v4506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7032.083177 secs 2024-04-26T20:32:31.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7032.083177 secs 2024-04-26T20:32:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7032.083177 secs 2024-04-26T20:32:31.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7436 sec 2024-04-26T20:32:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7032.788269 secs 2024-04-26T20:32:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:32 smithi090 ceph-mon[29180]: pgmap v4507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7032.788269 secs 2024-04-26T20:32:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:32 smithi119 ceph-mon[32752]: pgmap v4507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:32.555 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7437 sec 2024-04-26T20:32:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7032.788269 secs 2024-04-26T20:32:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:32 smithi167 ceph-mon[32751]: pgmap v4507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:33.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7438 sec 2024-04-26T20:32:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:34 smithi090 ceph-mon[29180]: pgmap v4508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:34 smithi119 ceph-mon[32752]: pgmap v4508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:34.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7439 sec 2024-04-26T20:32:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:34 smithi167 ceph-mon[32751]: pgmap v4508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:35.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7440 sec 2024-04-26T20:32:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:36 smithi090 ceph-mon[29180]: pgmap v4509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7037.083235 secs 2024-04-26T20:32:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:36 smithi119 ceph-mon[32752]: pgmap v4509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7037.083235 secs 2024-04-26T20:32:36.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7441 sec 2024-04-26T20:32:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:36 smithi167 ceph-mon[32751]: pgmap v4509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7037.083235 secs 2024-04-26T20:32:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:37 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 7037.788417 secs 2024-04-26T20:32:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:37 smithi090 ceph-mon[29180]: slow request 5764.425280 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:32:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:37 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 7037.788417 secs 2024-04-26T20:32:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:37 smithi119 ceph-mon[32752]: slow request 5764.425280 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:32:37.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7442 sec 2024-04-26T20:32:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:37 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 7037.788417 secs 2024-04-26T20:32:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:37 smithi167 ceph-mon[32751]: slow request 5764.425280 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:32:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:38 smithi090 ceph-mon[29180]: pgmap v4510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:38 smithi119 ceph-mon[32752]: pgmap v4510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:38.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7443 sec 2024-04-26T20:32:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:38 smithi167 ceph-mon[32751]: pgmap v4510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:39.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7444 sec 2024-04-26T20:32:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:40 smithi119 ceph-mon[32752]: pgmap v4511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:40.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7445 sec 2024-04-26T20:32:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:40 smithi167 ceph-mon[32751]: pgmap v4511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:40 smithi090 ceph-mon[29180]: pgmap v4511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:41.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7446 sec 2024-04-26T20:32:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7042.083320 secs 2024-04-26T20:32:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7042.083320 secs 2024-04-26T20:32:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7042.083320 secs 2024-04-26T20:32:42.556 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7447 sec 2024-04-26T20:32:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7042.788518 secs 2024-04-26T20:32:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:42 smithi167 ceph-mon[32751]: pgmap v4512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7042.788518 secs 2024-04-26T20:32:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:42 smithi090 ceph-mon[29180]: pgmap v4512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7042.788518 secs 2024-04-26T20:32:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:42 smithi119 ceph-mon[32752]: pgmap v4512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:43.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7448 sec 2024-04-26T20:32:44.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7449 sec 2024-04-26T20:32:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:44 smithi167 ceph-mon[32751]: pgmap v4513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:44 smithi090 ceph-mon[29180]: pgmap v4513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:44 smithi119 ceph-mon[32752]: pgmap v4513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:45.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7450 sec 2024-04-26T20:32:46.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7451 sec 2024-04-26T20:32:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:46 smithi167 ceph-mon[32751]: pgmap v4514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7047.083380 secs 2024-04-26T20:32:46.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:46 smithi119 ceph-mon[32752]: pgmap v4514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:46.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7047.083380 secs 2024-04-26T20:32:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:46 smithi090 ceph-mon[29180]: pgmap v4514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7047.083380 secs 2024-04-26T20:32:47.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7452 sec 2024-04-26T20:32:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7047.788600 secs 2024-04-26T20:32:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7047.788600 secs 2024-04-26T20:32:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7047.788600 secs 2024-04-26T20:32:48.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7453 sec 2024-04-26T20:32:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:48 smithi090 ceph-mon[29180]: pgmap v4515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:48 smithi119 ceph-mon[32752]: pgmap v4515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:48 smithi167 ceph-mon[32751]: pgmap v4515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:49.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7454 sec 2024-04-26T20:32:50.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7455 sec 2024-04-26T20:32:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:50 smithi090 ceph-mon[29180]: pgmap v4516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:50 smithi119 ceph-mon[32752]: pgmap v4516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:50 smithi167 ceph-mon[32751]: pgmap v4516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:51.557 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7456 sec 2024-04-26T20:32:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7052.083463 secs 2024-04-26T20:32:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7052.083463 secs 2024-04-26T20:32:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7052.083463 secs 2024-04-26T20:32:52.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7457 sec 2024-04-26T20:32:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7052.788701 secs 2024-04-26T20:32:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:52 smithi090 ceph-mon[29180]: pgmap v4517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7052.788701 secs 2024-04-26T20:32:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:52 smithi119 ceph-mon[32752]: pgmap v4517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7052.788701 secs 2024-04-26T20:32:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:52 smithi167 ceph-mon[32751]: pgmap v4517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:53.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7458 sec 2024-04-26T20:32:54.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7459 sec 2024-04-26T20:32:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:54 smithi090 ceph-mon[29180]: pgmap v4518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:54 smithi119 ceph-mon[32752]: pgmap v4518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:54 smithi167 ceph-mon[32751]: pgmap v4518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:55.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7460 sec 2024-04-26T20:32:56.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7461 sec 2024-04-26T20:32:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:56 smithi090 ceph-mon[29180]: pgmap v4519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7057.083547 secs 2024-04-26T20:32:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:56 smithi119 ceph-mon[32752]: pgmap v4519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7057.083547 secs 2024-04-26T20:32:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:56 smithi167 ceph-mon[32751]: pgmap v4519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:32:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7057.083547 secs 2024-04-26T20:32:57.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7462 sec 2024-04-26T20:32:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7057.788790 secs 2024-04-26T20:32:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7057.788790 secs 2024-04-26T20:32:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7057.788790 secs 2024-04-26T20:32:58.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7463 sec 2024-04-26T20:32:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:32:58 smithi090 ceph-mon[29180]: pgmap v4520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:32:58 smithi119 ceph-mon[32752]: pgmap v4520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:32:58 smithi167 ceph-mon[32751]: pgmap v4520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:32:59.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7464 sec 2024-04-26T20:33:00.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7465 sec 2024-04-26T20:33:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:00 smithi090 ceph-mon[29180]: pgmap v4521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:00 smithi119 ceph-mon[32752]: pgmap v4521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:00 smithi167 ceph-mon[32751]: pgmap v4521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:01.558 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7466 sec 2024-04-26T20:33:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7062.083603 secs 2024-04-26T20:33:01.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7062.083603 secs 2024-04-26T20:33:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7062.083603 secs 2024-04-26T20:33:02.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7467 sec 2024-04-26T20:33:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7062.788862 secs 2024-04-26T20:33:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:02 smithi090 ceph-mon[29180]: pgmap v4522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7062.788862 secs 2024-04-26T20:33:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:02 smithi119 ceph-mon[32752]: pgmap v4522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7062.788862 secs 2024-04-26T20:33:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:02 smithi167 ceph-mon[32751]: pgmap v4522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:03.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:33:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:33:02.647+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:33:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:33:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:33:02.647+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:33:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:33:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:33:02.647+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:33:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:33:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:33:02.647+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:33:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:33:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:33:02.647+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:33:03.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7468 sec 2024-04-26T20:33:04.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7469 sec 2024-04-26T20:33:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:04 smithi090 ceph-mon[29180]: pgmap v4523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:04 smithi119 ceph-mon[32752]: pgmap v4523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:04 smithi167 ceph-mon[32751]: pgmap v4523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:05.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7470 sec 2024-04-26T20:33:06.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7471 sec 2024-04-26T20:33:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:06 smithi090 ceph-mon[29180]: pgmap v4524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:33:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7067.083683 secs 2024-04-26T20:33:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:06 smithi119 ceph-mon[32752]: pgmap v4524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:33:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7067.083683 secs 2024-04-26T20:33:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:06 smithi167 ceph-mon[32751]: pgmap v4524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:33:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7067.083683 secs 2024-04-26T20:33:07.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7472 sec 2024-04-26T20:33:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7067.788997 secs 2024-04-26T20:33:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:33:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7067.788997 secs 2024-04-26T20:33:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:33:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7067.788997 secs 2024-04-26T20:33:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:33:08.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7473 sec 2024-04-26T20:33:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:08 smithi090 ceph-mon[29180]: pgmap v4525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:33:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:33:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:33:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:33:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:08 smithi119 ceph-mon[32752]: pgmap v4525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:33:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:33:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:33:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:33:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:08 smithi167 ceph-mon[32751]: pgmap v4525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:33:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:33:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:33:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:33:09.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7474 sec 2024-04-26T20:33:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:09 smithi090 ceph-mon[29180]: pgmap v4526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 380 B/s wr, 2 op/s 2024-04-26T20:33:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:09 smithi090 ceph-mon[29180]: pgmap v4527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 467 B/s wr, 1 op/s 2024-04-26T20:33:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:09 smithi119 ceph-mon[32752]: pgmap v4526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 380 B/s wr, 2 op/s 2024-04-26T20:33:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:09 smithi119 ceph-mon[32752]: pgmap v4527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 467 B/s wr, 1 op/s 2024-04-26T20:33:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:09 smithi167 ceph-mon[32751]: pgmap v4526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 380 B/s wr, 2 op/s 2024-04-26T20:33:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:09 smithi167 ceph-mon[32751]: pgmap v4527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 467 B/s wr, 1 op/s 2024-04-26T20:33:10.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7475 sec 2024-04-26T20:33:11.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7476 sec 2024-04-26T20:33:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:11 smithi090 ceph-mon[29180]: pgmap v4528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:33:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7072.083781 secs 2024-04-26T20:33:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:11 smithi119 ceph-mon[32752]: pgmap v4528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:33:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7072.083781 secs 2024-04-26T20:33:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:11 smithi167 ceph-mon[32751]: pgmap v4528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:33:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7072.083781 secs 2024-04-26T20:33:12.559 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7477 sec 2024-04-26T20:33:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7072.789104 secs 2024-04-26T20:33:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7072.789104 secs 2024-04-26T20:33:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7072.789104 secs 2024-04-26T20:33:13.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7478 sec 2024-04-26T20:33:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:13 smithi119 ceph-mon[32752]: pgmap v4529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:33:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:13 smithi167 ceph-mon[32751]: pgmap v4529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:33:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:13 smithi090 ceph-mon[29180]: pgmap v4529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T20:33:14.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7479 sec 2024-04-26T20:33:15.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7480 sec 2024-04-26T20:33:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:15 smithi119 ceph-mon[32752]: pgmap v4530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:15 smithi167 ceph-mon[32751]: pgmap v4530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:15 smithi090 ceph-mon[29180]: pgmap v4530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:16.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7481 sec 2024-04-26T20:33:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7077.083865 secs 2024-04-26T20:33:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7077.083865 secs 2024-04-26T20:33:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7077.083865 secs 2024-04-26T20:33:17.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7482 sec 2024-04-26T20:33:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:17 smithi119 ceph-mon[32752]: pgmap v4531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7077.789223 secs 2024-04-26T20:33:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:17 smithi167 ceph-mon[32751]: pgmap v4531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7077.789223 secs 2024-04-26T20:33:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:17 smithi090 ceph-mon[29180]: pgmap v4531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7077.789223 secs 2024-04-26T20:33:18.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7483 sec 2024-04-26T20:33:19.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7484 sec 2024-04-26T20:33:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:19 smithi119 ceph-mon[32752]: pgmap v4532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:33:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:19 smithi167 ceph-mon[32751]: pgmap v4532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:33:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:19 smithi090 ceph-mon[29180]: pgmap v4532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:33:20.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7485 sec 2024-04-26T20:33:21.560 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7486 sec 2024-04-26T20:33:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:21 smithi119 ceph-mon[32752]: pgmap v4533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7082.083925 secs 2024-04-26T20:33:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:21 smithi167 ceph-mon[32751]: pgmap v4533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7082.083925 secs 2024-04-26T20:33:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:21 smithi090 ceph-mon[29180]: pgmap v4533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:33:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7082.083925 secs 2024-04-26T20:33:22.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7487 sec 2024-04-26T20:33:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7082.789296 secs 2024-04-26T20:33:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7082.789296 secs 2024-04-26T20:33:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7082.789296 secs 2024-04-26T20:33:23.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7488 sec 2024-04-26T20:33:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:23 smithi119 ceph-mon[32752]: pgmap v4534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:23 smithi167 ceph-mon[32751]: pgmap v4534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:23 smithi090 ceph-mon[29180]: pgmap v4534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:24.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7489 sec 2024-04-26T20:33:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:24 smithi119 ceph-mon[32752]: pgmap v4535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:24 smithi167 ceph-mon[32751]: pgmap v4535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:24 smithi090 ceph-mon[29180]: pgmap v4535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:25.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7490 sec 2024-04-26T20:33:26.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7491 sec 2024-04-26T20:33:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7087.084015 secs 2024-04-26T20:33:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:26 smithi119 ceph-mon[32752]: pgmap v4536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7087.789414 secs 2024-04-26T20:33:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7087.084015 secs 2024-04-26T20:33:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:26 smithi167 ceph-mon[32751]: pgmap v4536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7087.789414 secs 2024-04-26T20:33:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7087.084015 secs 2024-04-26T20:33:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:26 smithi090 ceph-mon[29180]: pgmap v4536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7087.789414 secs 2024-04-26T20:33:27.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7492 sec 2024-04-26T20:33:28.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7493 sec 2024-04-26T20:33:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:28 smithi119 ceph-mon[32752]: pgmap v4537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:28 smithi167 ceph-mon[32751]: pgmap v4537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:28 smithi090 ceph-mon[29180]: pgmap v4537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:29.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7494 sec 2024-04-26T20:33:30.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7495 sec 2024-04-26T20:33:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:30 smithi119 ceph-mon[32752]: pgmap v4538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:30 smithi167 ceph-mon[32751]: pgmap v4538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:30 smithi090 ceph-mon[29180]: pgmap v4538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:31.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7496 sec 2024-04-26T20:33:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7092.084067 secs 2024-04-26T20:33:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7092.789534 secs 2024-04-26T20:33:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7092.084067 secs 2024-04-26T20:33:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7092.789534 secs 2024-04-26T20:33:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7092.084067 secs 2024-04-26T20:33:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7092.789534 secs 2024-04-26T20:33:32.561 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7497 sec 2024-04-26T20:33:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:32 smithi119 ceph-mon[32752]: pgmap v4539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:32 smithi167 ceph-mon[32751]: pgmap v4539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:32 smithi090 ceph-mon[29180]: pgmap v4539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:33.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7498 sec 2024-04-26T20:33:34.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7499 sec 2024-04-26T20:33:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:34 smithi119 ceph-mon[32752]: pgmap v4540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:34 smithi167 ceph-mon[32751]: pgmap v4540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:34 smithi090 ceph-mon[29180]: pgmap v4540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:35.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7500 sec 2024-04-26T20:33:36.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7501 sec 2024-04-26T20:33:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7097.084149 secs 2024-04-26T20:33:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:36 smithi119 ceph-mon[32752]: pgmap v4541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7097.789656 secs 2024-04-26T20:33:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7097.084149 secs 2024-04-26T20:33:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:36 smithi167 ceph-mon[32751]: pgmap v4541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7097.789656 secs 2024-04-26T20:33:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7097.084149 secs 2024-04-26T20:33:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:36 smithi090 ceph-mon[29180]: pgmap v4541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7097.789656 secs 2024-04-26T20:33:37.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7502 sec 2024-04-26T20:33:38.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7503 sec 2024-04-26T20:33:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:38 smithi119 ceph-mon[32752]: pgmap v4542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:38 smithi167 ceph-mon[32751]: pgmap v4542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:38 smithi090 ceph-mon[29180]: pgmap v4542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:39.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7504 sec 2024-04-26T20:33:40.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7505 sec 2024-04-26T20:33:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:40 smithi119 ceph-mon[32752]: pgmap v4543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:40 smithi167 ceph-mon[32751]: pgmap v4543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:40 smithi090 ceph-mon[29180]: pgmap v4543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:41.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7506 sec 2024-04-26T20:33:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:41 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 7102.084232 secs 2024-04-26T20:33:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:41 smithi119 ceph-mon[32752]: slow request 1442.448940 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:33:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7102.789792 secs 2024-04-26T20:33:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:41 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 7102.084232 secs 2024-04-26T20:33:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:41 smithi167 ceph-mon[32751]: slow request 1442.448940 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:33:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7102.789792 secs 2024-04-26T20:33:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:41 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 7102.084232 secs 2024-04-26T20:33:41.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:41 smithi090 ceph-mon[29180]: slow request 1442.448940 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:33:41.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7102.789792 secs 2024-04-26T20:33:42.562 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7507 sec 2024-04-26T20:33:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:42 smithi119 ceph-mon[32752]: pgmap v4544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:42 smithi167 ceph-mon[32751]: pgmap v4544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:42 smithi090 ceph-mon[29180]: pgmap v4544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:43.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7508 sec 2024-04-26T20:33:44.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7509 sec 2024-04-26T20:33:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:44 smithi119 ceph-mon[32752]: pgmap v4545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:44 smithi167 ceph-mon[32751]: pgmap v4545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:44 smithi090 ceph-mon[29180]: pgmap v4545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:45.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7510 sec 2024-04-26T20:33:46.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7511 sec 2024-04-26T20:33:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7107.084316 secs 2024-04-26T20:33:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:46 smithi119 ceph-mon[32752]: pgmap v4546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7107.789873 secs 2024-04-26T20:33:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7107.084316 secs 2024-04-26T20:33:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:46 smithi167 ceph-mon[32751]: pgmap v4546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7107.789873 secs 2024-04-26T20:33:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7107.084316 secs 2024-04-26T20:33:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:46 smithi090 ceph-mon[29180]: pgmap v4546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7107.789873 secs 2024-04-26T20:33:47.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7512 sec 2024-04-26T20:33:48.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7513 sec 2024-04-26T20:33:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:48 smithi119 ceph-mon[32752]: pgmap v4547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:48 smithi167 ceph-mon[32751]: pgmap v4547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:48 smithi090 ceph-mon[29180]: pgmap v4547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:49.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7514 sec 2024-04-26T20:33:50.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7515 sec 2024-04-26T20:33:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:50 smithi119 ceph-mon[32752]: pgmap v4548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:50 smithi167 ceph-mon[32751]: pgmap v4548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:50 smithi090 ceph-mon[29180]: pgmap v4548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:51.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7516 sec 2024-04-26T20:33:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7112.084375 secs 2024-04-26T20:33:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7112.789993 secs 2024-04-26T20:33:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7112.084375 secs 2024-04-26T20:33:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7112.789993 secs 2024-04-26T20:33:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7112.084375 secs 2024-04-26T20:33:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7112.789993 secs 2024-04-26T20:33:52.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7517 sec 2024-04-26T20:33:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:52 smithi119 ceph-mon[32752]: pgmap v4549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:52 smithi167 ceph-mon[32751]: pgmap v4549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:52 smithi090 ceph-mon[29180]: pgmap v4549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:53.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7518 sec 2024-04-26T20:33:54.563 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7519 sec 2024-04-26T20:33:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:54 smithi119 ceph-mon[32752]: pgmap v4550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:54 smithi167 ceph-mon[32751]: pgmap v4550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:54 smithi090 ceph-mon[29180]: pgmap v4550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:33:55.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7520 sec 2024-04-26T20:33:56.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7521 sec 2024-04-26T20:33:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7117.084459 secs 2024-04-26T20:33:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:56 smithi119 ceph-mon[32752]: pgmap v4551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7117.790088 secs 2024-04-26T20:33:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7117.084459 secs 2024-04-26T20:33:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:56 smithi167 ceph-mon[32751]: pgmap v4551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7117.790088 secs 2024-04-26T20:33:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7117.084459 secs 2024-04-26T20:33:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:56 smithi090 ceph-mon[29180]: pgmap v4551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7117.790088 secs 2024-04-26T20:33:57.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7522 sec 2024-04-26T20:33:58.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7523 sec 2024-04-26T20:33:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:33:58 smithi119 ceph-mon[32752]: pgmap v4552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:33:58 smithi167 ceph-mon[32751]: pgmap v4552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:33:58 smithi090 ceph-mon[29180]: pgmap v4552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:33:59.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7524 sec 2024-04-26T20:34:00.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7525 sec 2024-04-26T20:34:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:00 smithi119 ceph-mon[32752]: pgmap v4553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:00 smithi167 ceph-mon[32751]: pgmap v4553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:00 smithi090 ceph-mon[29180]: pgmap v4553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:01.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7526 sec 2024-04-26T20:34:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7122.084542 secs 2024-04-26T20:34:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7122.790184 secs 2024-04-26T20:34:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7122.084542 secs 2024-04-26T20:34:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7122.790184 secs 2024-04-26T20:34:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7122.084542 secs 2024-04-26T20:34:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7122.790184 secs 2024-04-26T20:34:02.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7527 sec 2024-04-26T20:34:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:02 smithi119 ceph-mon[32752]: pgmap v4554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:02 smithi167 ceph-mon[32751]: pgmap v4554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:02 smithi090 ceph-mon[29180]: pgmap v4554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:03.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7528 sec 2024-04-26T20:34:04.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7529 sec 2024-04-26T20:34:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:04 smithi167 ceph-mon[32751]: pgmap v4555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:04 smithi090 ceph-mon[29180]: pgmap v4555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:04 smithi119 ceph-mon[32752]: pgmap v4555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:05.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7530 sec 2024-04-26T20:34:06.564 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7531 sec 2024-04-26T20:34:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7127.084650 secs 2024-04-26T20:34:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:06 smithi167 ceph-mon[32751]: pgmap v4556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:06 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 7127.790305 secs 2024-04-26T20:34:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:06 smithi167 ceph-mon[32751]: slow request 362.951493 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:34:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7127.084650 secs 2024-04-26T20:34:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:06 smithi090 ceph-mon[29180]: pgmap v4556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:06 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 7127.790305 secs 2024-04-26T20:34:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:06 smithi090 ceph-mon[29180]: slow request 362.951493 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:34:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7127.084650 secs 2024-04-26T20:34:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:06 smithi119 ceph-mon[32752]: pgmap v4556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:06 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 7127.790305 secs 2024-04-26T20:34:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:06 smithi119 ceph-mon[32752]: slow request 362.951493 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:34:07.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7532 sec 2024-04-26T20:34:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:34:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:34:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:34:08.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7533 sec 2024-04-26T20:34:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:08 smithi090 ceph-mon[29180]: pgmap v4557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:34:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:34:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:08 smithi090 ceph-mon[29180]: pgmap v4558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:34:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:08 smithi090 ceph-mon[29180]: pgmap v4559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:34:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:34:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:08 smithi119 ceph-mon[32752]: pgmap v4557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:34:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:34:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:08 smithi119 ceph-mon[32752]: pgmap v4558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:34:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:08 smithi119 ceph-mon[32752]: pgmap v4559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:34:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:34:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:08 smithi167 ceph-mon[32751]: pgmap v4557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:34:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:34:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:08 smithi167 ceph-mon[32751]: pgmap v4558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:34:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:08 smithi167 ceph-mon[32751]: pgmap v4559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:34:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:34:09.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7534 sec 2024-04-26T20:34:10.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7535 sec 2024-04-26T20:34:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7132.084708 secs 2024-04-26T20:34:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:11 smithi119 ceph-mon[32752]: pgmap v4560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:11.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7536 sec 2024-04-26T20:34:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7132.084708 secs 2024-04-26T20:34:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:11 smithi167 ceph-mon[32751]: pgmap v4560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7132.084708 secs 2024-04-26T20:34:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:11 smithi090 ceph-mon[29180]: pgmap v4560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7132.790430 secs 2024-04-26T20:34:12.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7537 sec 2024-04-26T20:34:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7132.790430 secs 2024-04-26T20:34:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7132.790430 secs 2024-04-26T20:34:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:13 smithi119 ceph-mon[32752]: pgmap v4561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:34:13.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7538 sec 2024-04-26T20:34:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:13 smithi167 ceph-mon[32751]: pgmap v4561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:34:13.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:13 smithi090 ceph-mon[29180]: pgmap v4561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:34:14.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7539 sec 2024-04-26T20:34:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:15 smithi119 ceph-mon[32752]: pgmap v4562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:15.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7540 sec 2024-04-26T20:34:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:15 smithi167 ceph-mon[32751]: pgmap v4562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:15 smithi090 ceph-mon[29180]: pgmap v4562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7137.084815 secs 2024-04-26T20:34:16.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7541 sec 2024-04-26T20:34:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7137.084815 secs 2024-04-26T20:34:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7137.084815 secs 2024-04-26T20:34:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7137.790488 secs 2024-04-26T20:34:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:17 smithi119 ceph-mon[32752]: pgmap v4563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:17.565 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7542 sec 2024-04-26T20:34:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7137.790488 secs 2024-04-26T20:34:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:17 smithi167 ceph-mon[32751]: pgmap v4563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7137.790488 secs 2024-04-26T20:34:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:17 smithi090 ceph-mon[29180]: pgmap v4563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:34:18.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7543 sec 2024-04-26T20:34:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:19 smithi119 ceph-mon[32752]: pgmap v4564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:34:19.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7544 sec 2024-04-26T20:34:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:19 smithi167 ceph-mon[32751]: pgmap v4564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:34:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:19 smithi090 ceph-mon[29180]: pgmap v4564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:34:20.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7545 sec 2024-04-26T20:34:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7142.084906 secs 2024-04-26T20:34:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:21 smithi119 ceph-mon[32752]: pgmap v4565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:21.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7546 sec 2024-04-26T20:34:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7142.084906 secs 2024-04-26T20:34:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:21 smithi167 ceph-mon[32751]: pgmap v4565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7142.084906 secs 2024-04-26T20:34:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:21 smithi090 ceph-mon[29180]: pgmap v4565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7142.790583 secs 2024-04-26T20:34:22.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7547 sec 2024-04-26T20:34:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7142.790583 secs 2024-04-26T20:34:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7142.790583 secs 2024-04-26T20:34:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:23 smithi119 ceph-mon[32752]: pgmap v4566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:23.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7548 sec 2024-04-26T20:34:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:23 smithi167 ceph-mon[32751]: pgmap v4566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:23 smithi090 ceph-mon[29180]: pgmap v4566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:24.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7549 sec 2024-04-26T20:34:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:25 smithi119 ceph-mon[32752]: pgmap v4567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:25.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7550 sec 2024-04-26T20:34:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:25 smithi167 ceph-mon[32751]: pgmap v4567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:25 smithi090 ceph-mon[29180]: pgmap v4567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7147.084991 secs 2024-04-26T20:34:26.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7551 sec 2024-04-26T20:34:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7147.084991 secs 2024-04-26T20:34:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7147.084991 secs 2024-04-26T20:34:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7147.790697 secs 2024-04-26T20:34:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:27 smithi119 ceph-mon[32752]: pgmap v4568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:27.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7552 sec 2024-04-26T20:34:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7147.790697 secs 2024-04-26T20:34:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:27 smithi167 ceph-mon[32751]: pgmap v4568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7147.790697 secs 2024-04-26T20:34:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:27 smithi090 ceph-mon[29180]: pgmap v4568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:28.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7553 sec 2024-04-26T20:34:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:29 smithi119 ceph-mon[32752]: pgmap v4569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:29.566 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7554 sec 2024-04-26T20:34:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:29 smithi167 ceph-mon[32751]: pgmap v4569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:29 smithi090 ceph-mon[29180]: pgmap v4569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:30.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7555 sec 2024-04-26T20:34:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7152.085076 secs 2024-04-26T20:34:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:31 smithi119 ceph-mon[32752]: pgmap v4570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:31.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7556 sec 2024-04-26T20:34:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7152.085076 secs 2024-04-26T20:34:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:31 smithi167 ceph-mon[32751]: pgmap v4570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7152.085076 secs 2024-04-26T20:34:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:31 smithi090 ceph-mon[29180]: pgmap v4570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:32.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7557 sec 2024-04-26T20:34:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7152.790841 secs 2024-04-26T20:34:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7152.790841 secs 2024-04-26T20:34:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7152.790841 secs 2024-04-26T20:34:33.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7558 sec 2024-04-26T20:34:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:33 smithi167 ceph-mon[32751]: pgmap v4571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:33 smithi090 ceph-mon[29180]: pgmap v4571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:33 smithi119 ceph-mon[32752]: pgmap v4571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:34.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7559 sec 2024-04-26T20:34:35.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7560 sec 2024-04-26T20:34:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:35 smithi167 ceph-mon[32751]: pgmap v4572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:35 smithi090 ceph-mon[29180]: pgmap v4572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:35 smithi119 ceph-mon[32752]: pgmap v4572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:36.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7561 sec 2024-04-26T20:34:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7157.085157 secs 2024-04-26T20:34:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7157.085157 secs 2024-04-26T20:34:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7157.085157 secs 2024-04-26T20:34:37.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7562 sec 2024-04-26T20:34:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7157.790911 secs 2024-04-26T20:34:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:37 smithi090 ceph-mon[29180]: pgmap v4573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7157.790911 secs 2024-04-26T20:34:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:37 smithi119 ceph-mon[32752]: pgmap v4573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7157.790911 secs 2024-04-26T20:34:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:37 smithi167 ceph-mon[32751]: pgmap v4573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:38.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7563 sec 2024-04-26T20:34:39.567 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7564 sec 2024-04-26T20:34:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:39 smithi090 ceph-mon[29180]: pgmap v4574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:39 smithi119 ceph-mon[32752]: pgmap v4574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:39 smithi167 ceph-mon[32751]: pgmap v4574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:40.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7565 sec 2024-04-26T20:34:41.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7566 sec 2024-04-26T20:34:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7162.085243 secs 2024-04-26T20:34:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:41 smithi090 ceph-mon[29180]: pgmap v4575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7162.085243 secs 2024-04-26T20:34:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:41 smithi119 ceph-mon[32752]: pgmap v4575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7162.085243 secs 2024-04-26T20:34:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:41 smithi167 ceph-mon[32751]: pgmap v4575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:42.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7567 sec 2024-04-26T20:34:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7162.790975 secs 2024-04-26T20:34:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7162.790975 secs 2024-04-26T20:34:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7162.790975 secs 2024-04-26T20:34:43.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7568 sec 2024-04-26T20:34:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:43 smithi090 ceph-mon[29180]: pgmap v4576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:43 smithi119 ceph-mon[32752]: pgmap v4576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:43 smithi167 ceph-mon[32751]: pgmap v4576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:44.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7569 sec 2024-04-26T20:34:45.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7570 sec 2024-04-26T20:34:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:45 smithi090 ceph-mon[29180]: pgmap v4577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:45 smithi119 ceph-mon[32752]: pgmap v4577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:45 smithi167 ceph-mon[32751]: pgmap v4577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:46.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7571 sec 2024-04-26T20:34:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7167.085328 secs 2024-04-26T20:34:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7167.085328 secs 2024-04-26T20:34:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7167.085328 secs 2024-04-26T20:34:47.568 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7572 sec 2024-04-26T20:34:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7167.791070 secs 2024-04-26T20:34:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:47 smithi090 ceph-mon[29180]: pgmap v4578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7167.791070 secs 2024-04-26T20:34:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:47 smithi119 ceph-mon[32752]: pgmap v4578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7167.791070 secs 2024-04-26T20:34:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:47 smithi167 ceph-mon[32751]: pgmap v4578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:48.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7573 sec 2024-04-26T20:34:49.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7574 sec 2024-04-26T20:34:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:49 smithi090 ceph-mon[29180]: pgmap v4579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:49 smithi119 ceph-mon[32752]: pgmap v4579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:49 smithi167 ceph-mon[32751]: pgmap v4579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:50.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7575 sec 2024-04-26T20:34:51.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7576 sec 2024-04-26T20:34:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7172.085411 secs 2024-04-26T20:34:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:51 smithi090 ceph-mon[29180]: pgmap v4580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7172.085411 secs 2024-04-26T20:34:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:51 smithi119 ceph-mon[32752]: pgmap v4580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7172.085411 secs 2024-04-26T20:34:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:51 smithi167 ceph-mon[32751]: pgmap v4580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:52.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7577 sec 2024-04-26T20:34:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7172.791201 secs 2024-04-26T20:34:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7172.791201 secs 2024-04-26T20:34:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7172.791201 secs 2024-04-26T20:34:53.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7578 sec 2024-04-26T20:34:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:53 smithi090 ceph-mon[29180]: pgmap v4581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:53 smithi119 ceph-mon[32752]: pgmap v4581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:53 smithi167 ceph-mon[32751]: pgmap v4581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:54.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7579 sec 2024-04-26T20:34:55.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7580 sec 2024-04-26T20:34:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:55 smithi090 ceph-mon[29180]: pgmap v4582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:55 smithi119 ceph-mon[32752]: pgmap v4582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:55 smithi167 ceph-mon[32751]: pgmap v4582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:34:56.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7581 sec 2024-04-26T20:34:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7177.085494 secs 2024-04-26T20:34:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7177.085494 secs 2024-04-26T20:34:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7177.085494 secs 2024-04-26T20:34:57.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7582 sec 2024-04-26T20:34:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7177.791330 secs 2024-04-26T20:34:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:57 smithi090 ceph-mon[29180]: pgmap v4583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7177.791330 secs 2024-04-26T20:34:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:57 smithi119 ceph-mon[32752]: pgmap v4583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7177.791330 secs 2024-04-26T20:34:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:57 smithi167 ceph-mon[32751]: pgmap v4583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:58.569 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7583 sec 2024-04-26T20:34:59.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7584 sec 2024-04-26T20:34:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:34:59 smithi090 ceph-mon[29180]: pgmap v4584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:34:59 smithi119 ceph-mon[32752]: pgmap v4584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:34:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:34:59 smithi167 ceph-mon[32751]: pgmap v4584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:00.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7585 sec 2024-04-26T20:35:01.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7586 sec 2024-04-26T20:35:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7182.085578 secs 2024-04-26T20:35:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:01 smithi119 ceph-mon[32752]: pgmap v4585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7182.085578 secs 2024-04-26T20:35:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:01 smithi167 ceph-mon[32751]: pgmap v4585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7182.085578 secs 2024-04-26T20:35:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:01 smithi090 ceph-mon[29180]: pgmap v4585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:02.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7587 sec 2024-04-26T20:35:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7182.791480 secs 2024-04-26T20:35:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7182.791480 secs 2024-04-26T20:35:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7182.791480 secs 2024-04-26T20:35:03.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7588 sec 2024-04-26T20:35:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:03 smithi119 ceph-mon[32752]: pgmap v4586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:03 smithi167 ceph-mon[32751]: pgmap v4586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:03 smithi090 ceph-mon[29180]: pgmap v4586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:04.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7589 sec 2024-04-26T20:35:05.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7590 sec 2024-04-26T20:35:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:05 smithi119 ceph-mon[32752]: pgmap v4587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:05 smithi167 ceph-mon[32751]: pgmap v4587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:05 smithi090 ceph-mon[29180]: pgmap v4587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:06.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7591 sec 2024-04-26T20:35:06.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7187.085684 secs 2024-04-26T20:35:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7187.085684 secs 2024-04-26T20:35:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7187.085684 secs 2024-04-26T20:35:07.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7592 sec 2024-04-26T20:35:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7187.791630 secs 2024-04-26T20:35:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:07 smithi119 ceph-mon[32752]: pgmap v4588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7187.791630 secs 2024-04-26T20:35:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:07 smithi167 ceph-mon[32751]: pgmap v4588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7187.791630 secs 2024-04-26T20:35:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:07 smithi090 ceph-mon[29180]: pgmap v4588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:08.570 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7593 sec 2024-04-26T20:35:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:35:09.571 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7594 sec 2024-04-26T20:35:09.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:09 smithi090 ceph-mon[29180]: pgmap v4589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:09.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.755 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:09 smithi119 ceph-mon[32752]: pgmap v4589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:09.755 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.755 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.755 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.755 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:09 smithi167 ceph-mon[32751]: pgmap v4589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:10.571 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7595 sec 2024-04-26T20:35:11.571 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7596 sec 2024-04-26T20:35:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7192.085770 secs 2024-04-26T20:35:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: pgmap v4590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7192.085770 secs 2024-04-26T20:35:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: pgmap v4590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7192.085770 secs 2024-04-26T20:35:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: pgmap v4590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:12.571 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7597 sec 2024-04-26T20:35:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7192.791710 secs 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7192.791710 secs 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7192.791710 secs 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:35:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:35:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:35:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:35:13.571 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7598 sec 2024-04-26T20:35:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:13 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:35:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:13 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:35:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:13 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:35:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:13 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:35:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:13 smithi119 ceph-mon[32752]: pgmap v4591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-26T20:35:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:13 smithi119 ceph-mon[32752]: pgmap v4592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:13 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:35:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:13 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:35:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:13 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:35:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:13 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:35:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:13 smithi167 ceph-mon[32751]: pgmap v4591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-26T20:35:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:13 smithi167 ceph-mon[32751]: pgmap v4592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:13 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:35:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:13 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:35:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:13 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:35:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:13 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:35:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:13 smithi090 ceph-mon[29180]: pgmap v4591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-26T20:35:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:13 smithi090 ceph-mon[29180]: pgmap v4592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:14.571 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7599 sec 2024-04-26T20:35:15.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7600 sec 2024-04-26T20:35:15.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:15 smithi090 ceph-mon[29180]: pgmap v4593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:15 smithi119 ceph-mon[32752]: pgmap v4593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:15 smithi167 ceph-mon[32751]: pgmap v4593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:16.571 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7601 sec 2024-04-26T20:35:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7197.085856 secs 2024-04-26T20:35:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7197.085856 secs 2024-04-26T20:35:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7197.085856 secs 2024-04-26T20:35:17.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7602 sec 2024-04-26T20:35:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7197.791820 secs 2024-04-26T20:35:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:17 smithi119 ceph-mon[32752]: pgmap v4594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7197.791820 secs 2024-04-26T20:35:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:17 smithi167 ceph-mon[32751]: pgmap v4594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7197.791820 secs 2024-04-26T20:35:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:17 smithi090 ceph-mon[29180]: pgmap v4594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:18.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7603 sec 2024-04-26T20:35:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:18 smithi119 ceph-mon[32752]: pgmap v4595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:18 smithi167 ceph-mon[32751]: pgmap v4595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:18 smithi090 ceph-mon[29180]: pgmap v4595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:19.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7604 sec 2024-04-26T20:35:20.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7605 sec 2024-04-26T20:35:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:20 smithi119 ceph-mon[32752]: pgmap v4596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:20 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7202.085921 secs 2024-04-26T20:35:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:20 smithi167 ceph-mon[32751]: pgmap v4596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:20 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7202.085921 secs 2024-04-26T20:35:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:20 smithi090 ceph-mon[29180]: pgmap v4596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:35:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:20 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7202.085921 secs 2024-04-26T20:35:21.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7606 sec 2024-04-26T20:35:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7202.791938 secs 2024-04-26T20:35:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7202.791938 secs 2024-04-26T20:35:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7202.791938 secs 2024-04-26T20:35:22.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7607 sec 2024-04-26T20:35:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:22 smithi119 ceph-mon[32752]: pgmap v4597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:35:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:22 smithi167 ceph-mon[32751]: pgmap v4597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:35:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:22 smithi090 ceph-mon[29180]: pgmap v4597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:35:23.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7608 sec 2024-04-26T20:35:24.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7609 sec 2024-04-26T20:35:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:24 smithi119 ceph-mon[32752]: pgmap v4598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:24 smithi167 ceph-mon[32751]: pgmap v4598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:24 smithi090 ceph-mon[29180]: pgmap v4598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:25.572 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7610 sec 2024-04-26T20:35:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:25 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7207.086026 secs 2024-04-26T20:35:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:25 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7207.086026 secs 2024-04-26T20:35:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:25 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7207.086026 secs 2024-04-26T20:35:26.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7611 sec 2024-04-26T20:35:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7207.792016 secs 2024-04-26T20:35:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:26 smithi119 ceph-mon[32752]: pgmap v4599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7207.792016 secs 2024-04-26T20:35:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:26 smithi167 ceph-mon[32751]: pgmap v4599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7207.792016 secs 2024-04-26T20:35:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:26 smithi090 ceph-mon[29180]: pgmap v4599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:27.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7612 sec 2024-04-26T20:35:28.494 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b5250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401cf40 tx=0x7f2cc4046040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b5250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401cf40 tx=0x7f2cc4046040 comp rx=0 tx=0).stop 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b5250 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000149) for more than 900000000 us, fault. 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 msgr2=0x7f2cc00b5250 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b15b0 0x7f2cc00b5250 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=6 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 0x7f2cc00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:35:28.495 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.493+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 0x7f2cc00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:35:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.494+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T20:35:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.494+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T20:35:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:28.494+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 0x7f2cc00b4920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400fd90 tx=0x7f2cc4029040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:35:28.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7613 sec 2024-04-26T20:35:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:28 smithi119 ceph-mon[32752]: pgmap v4600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:28 smithi167 ceph-mon[32751]: pgmap v4600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:28 smithi090 ceph-mon[29180]: pgmap v4600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:29.423+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:35:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:29.423+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:35:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:29.424+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 59 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:35:29.425 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:35:29.424+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 59 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:35:29.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7614 sec 2024-04-26T20:35:30.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7615 sec 2024-04-26T20:35:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:30 smithi119 ceph-mon[32752]: pgmap v4601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:30 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7212.086111 secs 2024-04-26T20:35:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:30 smithi167 ceph-mon[32751]: pgmap v4601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:30 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7212.086111 secs 2024-04-26T20:35:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:30 smithi090 ceph-mon[29180]: pgmap v4601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:30 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7212.086111 secs 2024-04-26T20:35:31.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7616 sec 2024-04-26T20:35:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7212.792154 secs 2024-04-26T20:35:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7212.792154 secs 2024-04-26T20:35:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7212.792154 secs 2024-04-26T20:35:32.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7617 sec 2024-04-26T20:35:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:32 smithi119 ceph-mon[32752]: pgmap v4602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:32 smithi167 ceph-mon[32751]: pgmap v4602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:32 smithi090 ceph-mon[29180]: pgmap v4602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:33.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7618 sec 2024-04-26T20:35:34.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7619 sec 2024-04-26T20:35:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:34 smithi119 ceph-mon[32752]: pgmap v4603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:34 smithi167 ceph-mon[32751]: pgmap v4603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:34 smithi090 ceph-mon[29180]: pgmap v4603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:35.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7620 sec 2024-04-26T20:35:36.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:35 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7217.086199 secs 2024-04-26T20:35:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:35 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7217.086199 secs 2024-04-26T20:35:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:35 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7217.086199 secs 2024-04-26T20:35:36.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7621 sec 2024-04-26T20:35:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7217.792276 secs 2024-04-26T20:35:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:36 smithi119 ceph-mon[32752]: pgmap v4604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7217.792276 secs 2024-04-26T20:35:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:36 smithi167 ceph-mon[32751]: pgmap v4604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7217.792276 secs 2024-04-26T20:35:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:36 smithi090 ceph-mon[29180]: pgmap v4604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:37.573 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7622 sec 2024-04-26T20:35:38.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7623 sec 2024-04-26T20:35:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:38 smithi119 ceph-mon[32752]: pgmap v4605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:38 smithi167 ceph-mon[32751]: pgmap v4605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:38 smithi090 ceph-mon[29180]: pgmap v4605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:39.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7624 sec 2024-04-26T20:35:40.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7625 sec 2024-04-26T20:35:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:40 smithi119 ceph-mon[32752]: pgmap v4606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:40 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7222.086260 secs 2024-04-26T20:35:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:40 smithi167 ceph-mon[32751]: pgmap v4606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:40 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7222.086260 secs 2024-04-26T20:35:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:40 smithi090 ceph-mon[29180]: pgmap v4606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:40 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7222.086260 secs 2024-04-26T20:35:41.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7626 sec 2024-04-26T20:35:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7222.792403 secs 2024-04-26T20:35:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7222.792403 secs 2024-04-26T20:35:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7222.792403 secs 2024-04-26T20:35:42.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7627 sec 2024-04-26T20:35:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:42 smithi167 ceph-mon[32751]: pgmap v4607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:42 smithi090 ceph-mon[29180]: pgmap v4607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:42 smithi119 ceph-mon[32752]: pgmap v4607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:43.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7628 sec 2024-04-26T20:35:44.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7629 sec 2024-04-26T20:35:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:44 smithi167 ceph-mon[32751]: pgmap v4608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:44 smithi090 ceph-mon[29180]: pgmap v4608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:44 smithi119 ceph-mon[32752]: pgmap v4608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:45.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7630 sec 2024-04-26T20:35:46.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:45 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7227.086344 secs 2024-04-26T20:35:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:45 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7227.086344 secs 2024-04-26T20:35:46.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:45 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7227.086344 secs 2024-04-26T20:35:46.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7631 sec 2024-04-26T20:35:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7227.792511 secs 2024-04-26T20:35:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:46 smithi167 ceph-mon[32751]: pgmap v4609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7227.792511 secs 2024-04-26T20:35:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:46 smithi090 ceph-mon[29180]: pgmap v4609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7227.792511 secs 2024-04-26T20:35:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:46 smithi119 ceph-mon[32752]: pgmap v4609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:47.574 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7632 sec 2024-04-26T20:35:48.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7633 sec 2024-04-26T20:35:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:48 smithi090 ceph-mon[29180]: pgmap v4610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:48 smithi119 ceph-mon[32752]: pgmap v4610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:48 smithi167 ceph-mon[32751]: pgmap v4610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:49.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7634 sec 2024-04-26T20:35:50.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7635 sec 2024-04-26T20:35:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:50 smithi090 ceph-mon[29180]: pgmap v4611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:50 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7232.086404 secs 2024-04-26T20:35:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:50 smithi119 ceph-mon[32752]: pgmap v4611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:50 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7232.086404 secs 2024-04-26T20:35:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:50 smithi167 ceph-mon[32751]: pgmap v4611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:50 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7232.086404 secs 2024-04-26T20:35:51.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7636 sec 2024-04-26T20:35:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7232.792632 secs 2024-04-26T20:35:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7232.792632 secs 2024-04-26T20:35:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7232.792632 secs 2024-04-26T20:35:52.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7637 sec 2024-04-26T20:35:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:52 smithi090 ceph-mon[29180]: pgmap v4612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:52 smithi119 ceph-mon[32752]: pgmap v4612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:52 smithi167 ceph-mon[32751]: pgmap v4612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:53.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7638 sec 2024-04-26T20:35:54.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7639 sec 2024-04-26T20:35:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:54 smithi090 ceph-mon[29180]: pgmap v4613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:54 smithi119 ceph-mon[32752]: pgmap v4613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:54 smithi167 ceph-mon[32751]: pgmap v4613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:35:55.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7640 sec 2024-04-26T20:35:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:55 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7237.086477 secs 2024-04-26T20:35:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:55 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7237.086477 secs 2024-04-26T20:35:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:55 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7237.086477 secs 2024-04-26T20:35:56.575 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7641 sec 2024-04-26T20:35:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7237.792695 secs 2024-04-26T20:35:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:56 smithi090 ceph-mon[29180]: pgmap v4614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7237.792695 secs 2024-04-26T20:35:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:56 smithi119 ceph-mon[32752]: pgmap v4614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7237.792695 secs 2024-04-26T20:35:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:56 smithi167 ceph-mon[32751]: pgmap v4614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:57.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7642 sec 2024-04-26T20:35:58.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7643 sec 2024-04-26T20:35:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:35:58 smithi090 ceph-mon[29180]: pgmap v4615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:35:58 smithi119 ceph-mon[32752]: pgmap v4615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:35:58 smithi167 ceph-mon[32751]: pgmap v4615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:35:59.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7644 sec 2024-04-26T20:36:00.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7645 sec 2024-04-26T20:36:01.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:00 smithi167 ceph-mon[32751]: pgmap v4616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:01.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:00 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7242.086563 secs 2024-04-26T20:36:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:00 smithi090 ceph-mon[29180]: pgmap v4616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:00 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7242.086563 secs 2024-04-26T20:36:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:00 smithi119 ceph-mon[32752]: pgmap v4616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:00 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7242.086563 secs 2024-04-26T20:36:01.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7646 sec 2024-04-26T20:36:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7242.792782 secs 2024-04-26T20:36:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7242.792782 secs 2024-04-26T20:36:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7242.792782 secs 2024-04-26T20:36:02.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7647 sec 2024-04-26T20:36:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:02 smithi090 ceph-mon[29180]: pgmap v4617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:02 smithi119 ceph-mon[32752]: pgmap v4617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:02 smithi167 ceph-mon[32751]: pgmap v4617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:03.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7648 sec 2024-04-26T20:36:04.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7649 sec 2024-04-26T20:36:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:04 smithi090 ceph-mon[29180]: pgmap v4618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:04 smithi119 ceph-mon[32752]: pgmap v4618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:04 smithi167 ceph-mon[32751]: pgmap v4618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:05.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7650 sec 2024-04-26T20:36:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:05 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7247.086642 secs 2024-04-26T20:36:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:05 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7247.086642 secs 2024-04-26T20:36:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:05 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7247.086642 secs 2024-04-26T20:36:06.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7651 sec 2024-04-26T20:36:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7247.792862 secs 2024-04-26T20:36:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:06 smithi090 ceph-mon[29180]: pgmap v4619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7247.792862 secs 2024-04-26T20:36:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:06 smithi119 ceph-mon[32752]: pgmap v4619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7247.792862 secs 2024-04-26T20:36:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:06 smithi167 ceph-mon[32751]: pgmap v4619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:07.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7652 sec 2024-04-26T20:36:08.576 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7653 sec 2024-04-26T20:36:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:08 smithi090 ceph-mon[29180]: pgmap v4620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:08 smithi119 ceph-mon[32752]: pgmap v4620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:08 smithi167 ceph-mon[32751]: pgmap v4620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:09.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7654 sec 2024-04-26T20:36:10.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7655 sec 2024-04-26T20:36:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:10 smithi090 ceph-mon[29180]: pgmap v4621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:10 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7252.086701 secs 2024-04-26T20:36:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:10 smithi119 ceph-mon[32752]: pgmap v4621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:10 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7252.086701 secs 2024-04-26T20:36:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:10 smithi167 ceph-mon[32751]: pgmap v4621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:10 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7252.086701 secs 2024-04-26T20:36:11.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7656 sec 2024-04-26T20:36:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7252.793013 secs 2024-04-26T20:36:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:36:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7252.793013 secs 2024-04-26T20:36:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:36:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7252.793013 secs 2024-04-26T20:36:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:36:12.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7657 sec 2024-04-26T20:36:13.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7658 sec 2024-04-26T20:36:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:13 smithi090 ceph-mon[29180]: pgmap v4622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:36:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:36:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:13 smithi119 ceph-mon[32752]: pgmap v4622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:36:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:36:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:13 smithi167 ceph-mon[32751]: pgmap v4622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:36:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:36:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:36:14.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7659 sec 2024-04-26T20:36:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:14 smithi090 ceph-mon[29180]: pgmap v4623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T20:36:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:14 smithi090 ceph-mon[29180]: pgmap v4624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:36:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:14 smithi119 ceph-mon[32752]: pgmap v4623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T20:36:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:14 smithi119 ceph-mon[32752]: pgmap v4624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:36:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:14 smithi167 ceph-mon[32751]: pgmap v4623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T20:36:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:14 smithi167 ceph-mon[32751]: pgmap v4624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T20:36:15.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7660 sec 2024-04-26T20:36:16.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7661 sec 2024-04-26T20:36:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7257.086793 secs 2024-04-26T20:36:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:16 smithi090 ceph-mon[29180]: pgmap v4625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7257.086793 secs 2024-04-26T20:36:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:16 smithi119 ceph-mon[32752]: pgmap v4625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7257.086793 secs 2024-04-26T20:36:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:16 smithi167 ceph-mon[32751]: pgmap v4625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:17.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7662 sec 2024-04-26T20:36:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7257.793136 secs 2024-04-26T20:36:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7257.793136 secs 2024-04-26T20:36:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7257.793136 secs 2024-04-26T20:36:18.577 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7663 sec 2024-04-26T20:36:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:18 smithi090 ceph-mon[29180]: pgmap v4626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:18 smithi119 ceph-mon[32752]: pgmap v4626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:18 smithi167 ceph-mon[32751]: pgmap v4626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:19.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7664 sec 2024-04-26T20:36:20.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7665 sec 2024-04-26T20:36:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:20 smithi090 ceph-mon[29180]: pgmap v4627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:20 smithi119 ceph-mon[32752]: pgmap v4627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:20 smithi167 ceph-mon[32751]: pgmap v4627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:21.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7666 sec 2024-04-26T20:36:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7262.086868 secs 2024-04-26T20:36:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7262.086868 secs 2024-04-26T20:36:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7262.086868 secs 2024-04-26T20:36:22.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7667 sec 2024-04-26T20:36:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7262.793252 secs 2024-04-26T20:36:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:22 smithi119 ceph-mon[32752]: pgmap v4628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7262.793252 secs 2024-04-26T20:36:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:22 smithi167 ceph-mon[32751]: pgmap v4628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7262.793252 secs 2024-04-26T20:36:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:22 smithi090 ceph-mon[29180]: pgmap v4628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:36:23.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7668 sec 2024-04-26T20:36:24.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7669 sec 2024-04-26T20:36:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:24 smithi119 ceph-mon[32752]: pgmap v4629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:36:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:24 smithi167 ceph-mon[32751]: pgmap v4629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:36:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:24 smithi090 ceph-mon[29180]: pgmap v4629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:36:25.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7670 sec 2024-04-26T20:36:26.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7671 sec 2024-04-26T20:36:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7267.086958 secs 2024-04-26T20:36:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:26 smithi119 ceph-mon[32752]: pgmap v4630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7267.086958 secs 2024-04-26T20:36:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:26 smithi167 ceph-mon[32751]: pgmap v4630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7267.086958 secs 2024-04-26T20:36:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:26 smithi090 ceph-mon[29180]: pgmap v4630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:27.578 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7672 sec 2024-04-26T20:36:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7267.793333 secs 2024-04-26T20:36:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7267.793333 secs 2024-04-26T20:36:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7267.793333 secs 2024-04-26T20:36:28.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7673 sec 2024-04-26T20:36:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:28 smithi119 ceph-mon[32752]: pgmap v4631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:28 smithi167 ceph-mon[32751]: pgmap v4631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:28 smithi090 ceph-mon[29180]: pgmap v4631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:29.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7674 sec 2024-04-26T20:36:30.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7675 sec 2024-04-26T20:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:30 smithi119 ceph-mon[32752]: pgmap v4632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:30 smithi167 ceph-mon[32751]: pgmap v4632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:30 smithi090 ceph-mon[29180]: pgmap v4632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:31.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7676 sec 2024-04-26T20:36:31.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7272.087017 secs 2024-04-26T20:36:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7272.087017 secs 2024-04-26T20:36:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7272.087017 secs 2024-04-26T20:36:32.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7677 sec 2024-04-26T20:36:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7272.793454 secs 2024-04-26T20:36:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:32 smithi119 ceph-mon[32752]: pgmap v4633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7272.793454 secs 2024-04-26T20:36:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:32 smithi167 ceph-mon[32751]: pgmap v4633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:32.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7272.793454 secs 2024-04-26T20:36:32.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:32 smithi090 ceph-mon[29180]: pgmap v4633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:33.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7678 sec 2024-04-26T20:36:34.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7679 sec 2024-04-26T20:36:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:34 smithi119 ceph-mon[32752]: pgmap v4634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:34 smithi167 ceph-mon[32751]: pgmap v4634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:34 smithi090 ceph-mon[29180]: pgmap v4634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:35.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7680 sec 2024-04-26T20:36:36.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7681 sec 2024-04-26T20:36:36.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7277.087098 secs 2024-04-26T20:36:36.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:36 smithi090 ceph-mon[29180]: pgmap v4635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7277.087098 secs 2024-04-26T20:36:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:36 smithi119 ceph-mon[32752]: pgmap v4635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7277.087098 secs 2024-04-26T20:36:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:36 smithi167 ceph-mon[32751]: pgmap v4635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:37.579 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7682 sec 2024-04-26T20:36:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7277.793551 secs 2024-04-26T20:36:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7277.793551 secs 2024-04-26T20:36:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7277.793551 secs 2024-04-26T20:36:38.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7683 sec 2024-04-26T20:36:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:38 smithi119 ceph-mon[32752]: pgmap v4636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:38 smithi167 ceph-mon[32751]: pgmap v4636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:38 smithi090 ceph-mon[29180]: pgmap v4636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:39.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7684 sec 2024-04-26T20:36:40.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7685 sec 2024-04-26T20:36:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:40 smithi119 ceph-mon[32752]: pgmap v4637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:40 smithi167 ceph-mon[32751]: pgmap v4637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:40 smithi090 ceph-mon[29180]: pgmap v4637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:41.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7686 sec 2024-04-26T20:36:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7282.087184 secs 2024-04-26T20:36:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7282.087184 secs 2024-04-26T20:36:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7282.087184 secs 2024-04-26T20:36:42.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7687 sec 2024-04-26T20:36:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7282.793677 secs 2024-04-26T20:36:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:42 smithi119 ceph-mon[32752]: pgmap v4638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7282.793677 secs 2024-04-26T20:36:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:42 smithi167 ceph-mon[32751]: pgmap v4638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7282.793677 secs 2024-04-26T20:36:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:42 smithi090 ceph-mon[29180]: pgmap v4638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:43.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7688 sec 2024-04-26T20:36:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:43 smithi119 ceph-mon[32752]: pgmap v4639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:43 smithi167 ceph-mon[32751]: pgmap v4639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:43 smithi090 ceph-mon[29180]: pgmap v4639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:44.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7689 sec 2024-04-26T20:36:45.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7690 sec 2024-04-26T20:36:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7287.087267 secs 2024-04-26T20:36:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:46 smithi119 ceph-mon[32752]: pgmap v4640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7287.087267 secs 2024-04-26T20:36:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:46 smithi167 ceph-mon[32751]: pgmap v4640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7287.087267 secs 2024-04-26T20:36:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:46 smithi090 ceph-mon[29180]: pgmap v4640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:46.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7691 sec 2024-04-26T20:36:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7287.793779 secs 2024-04-26T20:36:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7287.793779 secs 2024-04-26T20:36:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7287.793779 secs 2024-04-26T20:36:47.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7692 sec 2024-04-26T20:36:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:48 smithi167 ceph-mon[32751]: pgmap v4641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:48 smithi090 ceph-mon[29180]: pgmap v4641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:48 smithi119 ceph-mon[32752]: pgmap v4641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:48.580 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7693 sec 2024-04-26T20:36:49.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7694 sec 2024-04-26T20:36:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:50 smithi090 ceph-mon[29180]: pgmap v4642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:50 smithi119 ceph-mon[32752]: pgmap v4642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:50 smithi167 ceph-mon[32751]: pgmap v4642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:50.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7695 sec 2024-04-26T20:36:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7292.087351 secs 2024-04-26T20:36:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7292.087351 secs 2024-04-26T20:36:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7292.087351 secs 2024-04-26T20:36:51.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7696 sec 2024-04-26T20:36:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7292.793913 secs 2024-04-26T20:36:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:52 smithi090 ceph-mon[29180]: pgmap v4643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7292.793913 secs 2024-04-26T20:36:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:52 smithi119 ceph-mon[32752]: pgmap v4643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7292.793913 secs 2024-04-26T20:36:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:52 smithi167 ceph-mon[32751]: pgmap v4643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:52.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7697 sec 2024-04-26T20:36:53.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7698 sec 2024-04-26T20:36:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:54 smithi090 ceph-mon[29180]: pgmap v4644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:54 smithi119 ceph-mon[32752]: pgmap v4644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:54 smithi167 ceph-mon[32751]: pgmap v4644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:54.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7699 sec 2024-04-26T20:36:55.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7700 sec 2024-04-26T20:36:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7297.087437 secs 2024-04-26T20:36:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:56 smithi090 ceph-mon[29180]: pgmap v4645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7297.087437 secs 2024-04-26T20:36:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:56 smithi119 ceph-mon[32752]: pgmap v4645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7297.087437 secs 2024-04-26T20:36:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:56 smithi167 ceph-mon[32751]: pgmap v4645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:36:56.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7701 sec 2024-04-26T20:36:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7297.794016 secs 2024-04-26T20:36:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7297.794016 secs 2024-04-26T20:36:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7297.794016 secs 2024-04-26T20:36:57.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7702 sec 2024-04-26T20:36:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:36:58 smithi090 ceph-mon[29180]: pgmap v4646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:36:58 smithi119 ceph-mon[32752]: pgmap v4646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:36:58 smithi167 ceph-mon[32751]: pgmap v4646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:36:58.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7703 sec 2024-04-26T20:36:59.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7704 sec 2024-04-26T20:37:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:00 smithi090 ceph-mon[29180]: pgmap v4647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:00 smithi119 ceph-mon[32752]: pgmap v4647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:00 smithi167 ceph-mon[32751]: pgmap v4647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:00.581 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7705 sec 2024-04-26T20:37:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7302.087520 secs 2024-04-26T20:37:01.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7302.087520 secs 2024-04-26T20:37:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7302.087520 secs 2024-04-26T20:37:01.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7706 sec 2024-04-26T20:37:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7302.794130 secs 2024-04-26T20:37:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:02 smithi090 ceph-mon[29180]: pgmap v4648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7302.794130 secs 2024-04-26T20:37:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:02 smithi119 ceph-mon[32752]: pgmap v4648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7302.794130 secs 2024-04-26T20:37:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:02 smithi167 ceph-mon[32751]: pgmap v4648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:02.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7707 sec 2024-04-26T20:37:03.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7708 sec 2024-04-26T20:37:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:04 smithi090 ceph-mon[29180]: pgmap v4649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:04 smithi119 ceph-mon[32752]: pgmap v4649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:04 smithi167 ceph-mon[32751]: pgmap v4649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:04.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7709 sec 2024-04-26T20:37:05.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7710 sec 2024-04-26T20:37:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7307.087601 secs 2024-04-26T20:37:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:06 smithi090 ceph-mon[29180]: pgmap v4650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7307.087601 secs 2024-04-26T20:37:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:06 smithi119 ceph-mon[32752]: pgmap v4650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7307.087601 secs 2024-04-26T20:37:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:06 smithi167 ceph-mon[32751]: pgmap v4650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:06.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7711 sec 2024-04-26T20:37:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7307.794195 secs 2024-04-26T20:37:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7307.794195 secs 2024-04-26T20:37:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7307.794195 secs 2024-04-26T20:37:07.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7712 sec 2024-04-26T20:37:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:08 smithi090 ceph-mon[29180]: pgmap v4651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:08 smithi119 ceph-mon[32752]: pgmap v4651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:08 smithi167 ceph-mon[32751]: pgmap v4651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:08.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7713 sec 2024-04-26T20:37:09.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7714 sec 2024-04-26T20:37:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:10 smithi090 ceph-mon[29180]: pgmap v4652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:10 smithi119 ceph-mon[32752]: pgmap v4652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:10 smithi167 ceph-mon[32751]: pgmap v4652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:10.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7715 sec 2024-04-26T20:37:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7312.087683 secs 2024-04-26T20:37:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7312.087683 secs 2024-04-26T20:37:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7312.087683 secs 2024-04-26T20:37:11.582 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7716 sec 2024-04-26T20:37:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7312.794294 secs 2024-04-26T20:37:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:12 smithi090 ceph-mon[29180]: pgmap v4653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7312.794294 secs 2024-04-26T20:37:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:12 smithi119 ceph-mon[32752]: pgmap v4653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7312.794294 secs 2024-04-26T20:37:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:12 smithi167 ceph-mon[32751]: pgmap v4653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:12.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7717 sec 2024-04-26T20:37:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:37:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:37:13.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7718 sec 2024-04-26T20:37:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:37:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:14 smithi119 ceph-mon[32752]: pgmap v4654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:37:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:37:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:37:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:14 smithi167 ceph-mon[32751]: pgmap v4654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:37:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:37:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:37:14.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7719 sec 2024-04-26T20:37:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:14 smithi090 ceph-mon[29180]: pgmap v4654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:37:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:37:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:37:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:15 smithi119 ceph-mon[32752]: pgmap v4655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:37:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:15 smithi119 ceph-mon[32752]: pgmap v4656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:15 smithi167 ceph-mon[32751]: pgmap v4655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:37:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:15 smithi167 ceph-mon[32751]: pgmap v4656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:15.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7720 sec 2024-04-26T20:37:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:15 smithi090 ceph-mon[29180]: pgmap v4655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:37:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:15 smithi090 ceph-mon[29180]: pgmap v4656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:16.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7317.087769 secs 2024-04-26T20:37:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7317.087769 secs 2024-04-26T20:37:16.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7721 sec 2024-04-26T20:37:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7317.087769 secs 2024-04-26T20:37:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7317.794415 secs 2024-04-26T20:37:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:17 smithi119 ceph-mon[32752]: pgmap v4657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7317.794415 secs 2024-04-26T20:37:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:17 smithi167 ceph-mon[32751]: pgmap v4657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:17.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7722 sec 2024-04-26T20:37:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7317.794415 secs 2024-04-26T20:37:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:17 smithi090 ceph-mon[29180]: pgmap v4657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:18.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7723 sec 2024-04-26T20:37:19.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:19 smithi090 ceph-mon[29180]: pgmap v4658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:37:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:19 smithi119 ceph-mon[32752]: pgmap v4658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:37:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:19 smithi167 ceph-mon[32751]: pgmap v4658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:37:19.583 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7724 sec 2024-04-26T20:37:20.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7725 sec 2024-04-26T20:37:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:21 smithi119 ceph-mon[32752]: pgmap v4659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7322.087853 secs 2024-04-26T20:37:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:21 smithi167 ceph-mon[32751]: pgmap v4659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7322.087853 secs 2024-04-26T20:37:21.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7726 sec 2024-04-26T20:37:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:21 smithi090 ceph-mon[29180]: pgmap v4659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7322.087853 secs 2024-04-26T20:37:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7322.794563 secs 2024-04-26T20:37:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7322.794563 secs 2024-04-26T20:37:22.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7727 sec 2024-04-26T20:37:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7322.794563 secs 2024-04-26T20:37:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:23 smithi119 ceph-mon[32752]: pgmap v4660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:23 smithi167 ceph-mon[32751]: pgmap v4660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:23.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7728 sec 2024-04-26T20:37:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:23 smithi090 ceph-mon[29180]: pgmap v4660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:37:24.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7729 sec 2024-04-26T20:37:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:25 smithi119 ceph-mon[32752]: pgmap v4661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:37:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:25 smithi167 ceph-mon[32751]: pgmap v4661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:37:25.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7730 sec 2024-04-26T20:37:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:25 smithi090 ceph-mon[29180]: pgmap v4661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:37:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7327.087938 secs 2024-04-26T20:37:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7327.087938 secs 2024-04-26T20:37:26.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7731 sec 2024-04-26T20:37:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7327.087938 secs 2024-04-26T20:37:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7327.794641 secs 2024-04-26T20:37:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:27 smithi119 ceph-mon[32752]: pgmap v4662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7327.794641 secs 2024-04-26T20:37:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:27 smithi167 ceph-mon[32751]: pgmap v4662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:27.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7732 sec 2024-04-26T20:37:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7327.794641 secs 2024-04-26T20:37:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:27 smithi090 ceph-mon[29180]: pgmap v4662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:28.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7733 sec 2024-04-26T20:37:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:29 smithi119 ceph-mon[32752]: pgmap v4663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:29 smithi167 ceph-mon[32751]: pgmap v4663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:29.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7734 sec 2024-04-26T20:37:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:29 smithi090 ceph-mon[29180]: pgmap v4663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:30.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7735 sec 2024-04-26T20:37:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:31 smithi119 ceph-mon[32752]: pgmap v4664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7332.088027 secs 2024-04-26T20:37:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:31 smithi167 ceph-mon[32751]: pgmap v4664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7332.088027 secs 2024-04-26T20:37:31.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7736 sec 2024-04-26T20:37:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:31 smithi090 ceph-mon[29180]: pgmap v4664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7332.088027 secs 2024-04-26T20:37:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7332.794775 secs 2024-04-26T20:37:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7332.794775 secs 2024-04-26T20:37:32.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7737 sec 2024-04-26T20:37:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7332.794775 secs 2024-04-26T20:37:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:33 smithi119 ceph-mon[32752]: pgmap v4665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:33 smithi167 ceph-mon[32751]: pgmap v4665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:33.584 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7738 sec 2024-04-26T20:37:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:33 smithi090 ceph-mon[29180]: pgmap v4665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:34.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7739 sec 2024-04-26T20:37:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:35 smithi119 ceph-mon[32752]: pgmap v4666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:35 smithi167 ceph-mon[32751]: pgmap v4666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:35.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7740 sec 2024-04-26T20:37:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:35 smithi090 ceph-mon[29180]: pgmap v4666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:36.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7337.088090 secs 2024-04-26T20:37:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7337.088090 secs 2024-04-26T20:37:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7337.088090 secs 2024-04-26T20:37:36.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7741 sec 2024-04-26T20:37:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:37 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 7337.794914 secs 2024-04-26T20:37:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:37 smithi119 ceph-mon[32752]: slow request 5764.320263 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:37:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:37 smithi119 ceph-mon[32752]: pgmap v4667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:37 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 7337.794914 secs 2024-04-26T20:37:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:37 smithi167 ceph-mon[32751]: slow request 5764.320263 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:37:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:37 smithi167 ceph-mon[32751]: pgmap v4667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:37.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7742 sec 2024-04-26T20:37:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:37 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 7337.794914 secs 2024-04-26T20:37:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:37 smithi090 ceph-mon[29180]: slow request 5764.320263 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:37:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:37 smithi090 ceph-mon[29180]: pgmap v4667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:38.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7743 sec 2024-04-26T20:37:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:39 smithi119 ceph-mon[32752]: pgmap v4668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:39 smithi167 ceph-mon[32751]: pgmap v4668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:39.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7744 sec 2024-04-26T20:37:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:39 smithi090 ceph-mon[29180]: pgmap v4668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:40.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7745 sec 2024-04-26T20:37:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:41 smithi119 ceph-mon[32752]: pgmap v4669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7342.088176 secs 2024-04-26T20:37:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:41 smithi167 ceph-mon[32751]: pgmap v4669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7342.088176 secs 2024-04-26T20:37:41.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7746 sec 2024-04-26T20:37:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:41 smithi090 ceph-mon[29180]: pgmap v4669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7342.088176 secs 2024-04-26T20:37:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7342.795024 secs 2024-04-26T20:37:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7342.795024 secs 2024-04-26T20:37:42.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7747 sec 2024-04-26T20:37:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7342.795024 secs 2024-04-26T20:37:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:43 smithi119 ceph-mon[32752]: pgmap v4670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:43 smithi167 ceph-mon[32751]: pgmap v4670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:43.585 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7748 sec 2024-04-26T20:37:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:43 smithi090 ceph-mon[29180]: pgmap v4670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:44.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7749 sec 2024-04-26T20:37:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:45 smithi119 ceph-mon[32752]: pgmap v4671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:45 smithi167 ceph-mon[32751]: pgmap v4671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:45.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7750 sec 2024-04-26T20:37:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:45 smithi090 ceph-mon[29180]: pgmap v4671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7347.088258 secs 2024-04-26T20:37:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7347.088258 secs 2024-04-26T20:37:46.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7751 sec 2024-04-26T20:37:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7347.088258 secs 2024-04-26T20:37:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7347.795154 secs 2024-04-26T20:37:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:47 smithi119 ceph-mon[32752]: pgmap v4672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7347.795154 secs 2024-04-26T20:37:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:47 smithi167 ceph-mon[32751]: pgmap v4672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:47.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7752 sec 2024-04-26T20:37:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7347.795154 secs 2024-04-26T20:37:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:47 smithi090 ceph-mon[29180]: pgmap v4672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:48.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7753 sec 2024-04-26T20:37:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:49 smithi119 ceph-mon[32752]: pgmap v4673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:49 smithi167 ceph-mon[32751]: pgmap v4673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:49.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7754 sec 2024-04-26T20:37:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:49 smithi090 ceph-mon[29180]: pgmap v4673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:50.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7755 sec 2024-04-26T20:37:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:51 smithi119 ceph-mon[32752]: pgmap v4674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7352.088344 secs 2024-04-26T20:37:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:51 smithi167 ceph-mon[32751]: pgmap v4674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7352.088344 secs 2024-04-26T20:37:51.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7756 sec 2024-04-26T20:37:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:51 smithi090 ceph-mon[29180]: pgmap v4674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7352.088344 secs 2024-04-26T20:37:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7352.795297 secs 2024-04-26T20:37:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7352.795297 secs 2024-04-26T20:37:52.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7757 sec 2024-04-26T20:37:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7352.795297 secs 2024-04-26T20:37:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:53 smithi119 ceph-mon[32752]: pgmap v4675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:53 smithi167 ceph-mon[32751]: pgmap v4675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:53.586 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7758 sec 2024-04-26T20:37:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:53 smithi090 ceph-mon[29180]: pgmap v4675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:54.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7759 sec 2024-04-26T20:37:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:55 smithi119 ceph-mon[32752]: pgmap v4676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:55 smithi167 ceph-mon[32751]: pgmap v4676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:55.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7760 sec 2024-04-26T20:37:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:55 smithi090 ceph-mon[29180]: pgmap v4676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:37:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7357.088431 secs 2024-04-26T20:37:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7357.088431 secs 2024-04-26T20:37:56.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7761 sec 2024-04-26T20:37:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7357.088431 secs 2024-04-26T20:37:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7357.795414 secs 2024-04-26T20:37:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:57 smithi119 ceph-mon[32752]: pgmap v4677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7357.795414 secs 2024-04-26T20:37:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:57 smithi167 ceph-mon[32751]: pgmap v4677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:57.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7762 sec 2024-04-26T20:37:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7357.795414 secs 2024-04-26T20:37:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:57 smithi090 ceph-mon[29180]: pgmap v4677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:58.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7763 sec 2024-04-26T20:37:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:37:59 smithi119 ceph-mon[32752]: pgmap v4678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:37:59 smithi167 ceph-mon[32751]: pgmap v4678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:37:59.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7764 sec 2024-04-26T20:37:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:37:59 smithi090 ceph-mon[29180]: pgmap v4678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:00.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7765 sec 2024-04-26T20:38:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:01 smithi119 ceph-mon[32752]: pgmap v4679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7362.088517 secs 2024-04-26T20:38:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:01 smithi167 ceph-mon[32751]: pgmap v4679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7362.088517 secs 2024-04-26T20:38:01.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7766 sec 2024-04-26T20:38:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:01 smithi090 ceph-mon[29180]: pgmap v4679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7362.088517 secs 2024-04-26T20:38:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7362.795531 secs 2024-04-26T20:38:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7362.795531 secs 2024-04-26T20:38:02.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7767 sec 2024-04-26T20:38:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7362.795531 secs 2024-04-26T20:38:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:03 smithi119 ceph-mon[32752]: pgmap v4680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:03 smithi167 ceph-mon[32751]: pgmap v4680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:03.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7768 sec 2024-04-26T20:38:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:03 smithi090 ceph-mon[29180]: pgmap v4680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:04.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7769 sec 2024-04-26T20:38:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:05 smithi119 ceph-mon[32752]: pgmap v4681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:05 smithi167 ceph-mon[32751]: pgmap v4681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:05.587 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7770 sec 2024-04-26T20:38:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:05 smithi090 ceph-mon[29180]: pgmap v4681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:06.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:06 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 7367.088603 secs 2024-04-26T20:38:06.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:06 smithi090 ceph-mon[29180]: slow request 2882.870045 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:38:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:06 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 7367.088603 secs 2024-04-26T20:38:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:06 smithi119 ceph-mon[32752]: slow request 2882.870045 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:38:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:06 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 7367.088603 secs 2024-04-26T20:38:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:06 smithi167 ceph-mon[32751]: slow request 2882.870045 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:38:06.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7771 sec 2024-04-26T20:38:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7367.795595 secs 2024-04-26T20:38:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:07 smithi119 ceph-mon[32752]: pgmap v4682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7367.795595 secs 2024-04-26T20:38:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:07 smithi167 ceph-mon[32751]: pgmap v4682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:07.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7772 sec 2024-04-26T20:38:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7367.795595 secs 2024-04-26T20:38:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:07 smithi090 ceph-mon[29180]: pgmap v4682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:08.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7773 sec 2024-04-26T20:38:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:09 smithi119 ceph-mon[32752]: pgmap v4683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:09 smithi167 ceph-mon[32751]: pgmap v4683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:09.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7774 sec 2024-04-26T20:38:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:09 smithi090 ceph-mon[29180]: pgmap v4683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:10.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7775 sec 2024-04-26T20:38:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:11 smithi119 ceph-mon[32752]: pgmap v4684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7372.088691 secs 2024-04-26T20:38:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:11 smithi167 ceph-mon[32751]: pgmap v4684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7372.088691 secs 2024-04-26T20:38:11.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7776 sec 2024-04-26T20:38:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:11 smithi090 ceph-mon[29180]: pgmap v4684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7372.088691 secs 2024-04-26T20:38:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7372.795714 secs 2024-04-26T20:38:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7372.795714 secs 2024-04-26T20:38:12.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7777 sec 2024-04-26T20:38:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7372.795714 secs 2024-04-26T20:38:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:13 smithi119 ceph-mon[32752]: pgmap v4685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:13 smithi167 ceph-mon[32751]: pgmap v4685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:13.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7778 sec 2024-04-26T20:38:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:13 smithi090 ceph-mon[29180]: pgmap v4685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:38:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:38:14.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7779 sec 2024-04-26T20:38:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:38:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:15 smithi119 ceph-mon[32752]: pgmap v4686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:38:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:38:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:38:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:15 smithi167 ceph-mon[32751]: pgmap v4686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:38:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:38:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:38:15.588 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7780 sec 2024-04-26T20:38:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:15 smithi090 ceph-mon[29180]: pgmap v4686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:38:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:38:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:38:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:16 smithi119 ceph-mon[32752]: pgmap v4687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:38:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:16 smithi119 ceph-mon[32752]: pgmap v4688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7377.088778 secs 2024-04-26T20:38:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:16 smithi167 ceph-mon[32751]: pgmap v4687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:38:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:16 smithi167 ceph-mon[32751]: pgmap v4688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7377.088778 secs 2024-04-26T20:38:16.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7781 sec 2024-04-26T20:38:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:16 smithi090 ceph-mon[29180]: pgmap v4687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:38:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:16 smithi090 ceph-mon[29180]: pgmap v4688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7377.088778 secs 2024-04-26T20:38:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7377.795787 secs 2024-04-26T20:38:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7377.795787 secs 2024-04-26T20:38:17.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7782 sec 2024-04-26T20:38:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7377.795787 secs 2024-04-26T20:38:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:18 smithi119 ceph-mon[32752]: pgmap v4689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:18 smithi167 ceph-mon[32751]: pgmap v4689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:18.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7783 sec 2024-04-26T20:38:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:18 smithi090 ceph-mon[29180]: pgmap v4689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:19.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7784 sec 2024-04-26T20:38:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:20 smithi119 ceph-mon[32752]: pgmap v4690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:20 smithi167 ceph-mon[32751]: pgmap v4690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:20.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7785 sec 2024-04-26T20:38:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:20 smithi090 ceph-mon[29180]: pgmap v4690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7382.088863 secs 2024-04-26T20:38:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7382.088863 secs 2024-04-26T20:38:21.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7786 sec 2024-04-26T20:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7382.088863 secs 2024-04-26T20:38:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7382.795898 secs 2024-04-26T20:38:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:22 smithi119 ceph-mon[32752]: pgmap v4691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7382.795898 secs 2024-04-26T20:38:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:22 smithi167 ceph-mon[32751]: pgmap v4691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:22.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7787 sec 2024-04-26T20:38:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7382.795898 secs 2024-04-26T20:38:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:22 smithi090 ceph-mon[29180]: pgmap v4691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:38:23.589 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7788 sec 2024-04-26T20:38:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:24 smithi119 ceph-mon[32752]: pgmap v4692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:38:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:24 smithi167 ceph-mon[32751]: pgmap v4692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:38:24.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7789 sec 2024-04-26T20:38:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:24 smithi090 ceph-mon[29180]: pgmap v4692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:38:25.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7790 sec 2024-04-26T20:38:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:26 smithi167 ceph-mon[32751]: pgmap v4693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:38:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7387.088951 secs 2024-04-26T20:38:26.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7791 sec 2024-04-26T20:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:26 smithi090 ceph-mon[29180]: pgmap v4693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:38:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7387.088951 secs 2024-04-26T20:38:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:26 smithi119 ceph-mon[32752]: pgmap v4693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:38:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7387.088951 secs 2024-04-26T20:38:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7387.795992 secs 2024-04-26T20:38:27.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7792 sec 2024-04-26T20:38:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7387.795992 secs 2024-04-26T20:38:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7387.795992 secs 2024-04-26T20:38:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:28 smithi167 ceph-mon[32751]: pgmap v4694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:28.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7793 sec 2024-04-26T20:38:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:28 smithi090 ceph-mon[29180]: pgmap v4694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:28 smithi119 ceph-mon[32752]: pgmap v4694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:38:29.448+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:38:29.449 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:38:29.448+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:38:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:38:29.449+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 60 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:38:29.450 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:38:29.449+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 60 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:38:29.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7794 sec 2024-04-26T20:38:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:30 smithi167 ceph-mon[32751]: pgmap v4695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:30.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7795 sec 2024-04-26T20:38:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:30 smithi090 ceph-mon[29180]: pgmap v4695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:30 smithi119 ceph-mon[32752]: pgmap v4695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:31.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7796 sec 2024-04-26T20:38:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:31 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7392.089013 secs 2024-04-26T20:38:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:31 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7392.089013 secs 2024-04-26T20:38:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:31 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7392.089013 secs 2024-04-26T20:38:32.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7797 sec 2024-04-26T20:38:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7392.796125 secs 2024-04-26T20:38:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:32 smithi090 ceph-mon[29180]: pgmap v4696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7392.796125 secs 2024-04-26T20:38:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:32 smithi119 ceph-mon[32752]: pgmap v4696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7392.796125 secs 2024-04-26T20:38:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:32 smithi167 ceph-mon[32751]: pgmap v4696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:38:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:38:32.754+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:38:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:38:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:38:32.754+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:38:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:38:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:38:32.754+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:38:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:38:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:38:32.754+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:38:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:38:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:38:32.754+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:38:33.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7798 sec 2024-04-26T20:38:34.590 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7799 sec 2024-04-26T20:38:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:34 smithi090 ceph-mon[29180]: pgmap v4697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:34 smithi119 ceph-mon[32752]: pgmap v4697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:34 smithi167 ceph-mon[32751]: pgmap v4697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:35.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7800 sec 2024-04-26T20:38:36.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7801 sec 2024-04-26T20:38:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:36 smithi090 ceph-mon[29180]: pgmap v4698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7397.089098 secs 2024-04-26T20:38:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:36 smithi119 ceph-mon[32752]: pgmap v4698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7397.089098 secs 2024-04-26T20:38:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:36 smithi167 ceph-mon[32751]: pgmap v4698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7397.089098 secs 2024-04-26T20:38:37.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7802 sec 2024-04-26T20:38:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7397.796248 secs 2024-04-26T20:38:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7397.796248 secs 2024-04-26T20:38:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7397.796248 secs 2024-04-26T20:38:38.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7803 sec 2024-04-26T20:38:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:38 smithi090 ceph-mon[29180]: pgmap v4699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:38 smithi119 ceph-mon[32752]: pgmap v4699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:38 smithi167 ceph-mon[32751]: pgmap v4699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:39.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7804 sec 2024-04-26T20:38:40.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7805 sec 2024-04-26T20:38:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:40 smithi090 ceph-mon[29180]: pgmap v4700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:40 smithi119 ceph-mon[32752]: pgmap v4700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:40 smithi167 ceph-mon[32751]: pgmap v4700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:41.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7806 sec 2024-04-26T20:38:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7402.089186 secs 2024-04-26T20:38:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7402.089186 secs 2024-04-26T20:38:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7402.089186 secs 2024-04-26T20:38:42.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7807 sec 2024-04-26T20:38:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7402.796365 secs 2024-04-26T20:38:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:42 smithi090 ceph-mon[29180]: pgmap v4701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7402.796365 secs 2024-04-26T20:38:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:42 smithi119 ceph-mon[32752]: pgmap v4701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7402.796365 secs 2024-04-26T20:38:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:42 smithi167 ceph-mon[32751]: pgmap v4701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:43.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7808 sec 2024-04-26T20:38:44.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7809 sec 2024-04-26T20:38:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:44 smithi090 ceph-mon[29180]: pgmap v4702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:44 smithi119 ceph-mon[32752]: pgmap v4702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:44 smithi167 ceph-mon[32751]: pgmap v4702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:45.591 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7810 sec 2024-04-26T20:38:46.592 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7811 sec 2024-04-26T20:38:46.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:46 smithi119 ceph-mon[32752]: pgmap v4703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:46.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7407.089269 secs 2024-04-26T20:38:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:46 smithi090 ceph-mon[29180]: pgmap v4703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7407.089269 secs 2024-04-26T20:38:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:46 smithi167 ceph-mon[32751]: pgmap v4703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:38:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7407.089269 secs 2024-04-26T20:38:47.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7812 sec 2024-04-26T20:38:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7407.796481 secs 2024-04-26T20:38:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7407.796481 secs 2024-04-26T20:38:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7407.796481 secs 2024-04-26T20:38:48.592 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7813 sec 2024-04-26T20:38:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:48 smithi090 ceph-mon[29180]: pgmap v4704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:48 smithi119 ceph-mon[32752]: pgmap v4704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:48 smithi167 ceph-mon[32751]: pgmap v4704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:49.592 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7814 sec 2024-04-26T20:38:50.592 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7815 sec 2024-04-26T20:38:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:50 smithi090 ceph-mon[29180]: pgmap v4705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:50 smithi119 ceph-mon[32752]: pgmap v4705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:50 smithi167 ceph-mon[32751]: pgmap v4705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:51.592 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7816 sec 2024-04-26T20:38:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7412.089351 secs 2024-04-26T20:38:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7412.089351 secs 2024-04-26T20:38:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7412.089351 secs 2024-04-26T20:38:52.592 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7817 sec 2024-04-26T20:38:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7412.796611 secs 2024-04-26T20:38:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:52 smithi090 ceph-mon[29180]: pgmap v4706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7412.796611 secs 2024-04-26T20:38:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:52 smithi119 ceph-mon[32752]: pgmap v4706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7412.796611 secs 2024-04-26T20:38:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:52 smithi167 ceph-mon[32751]: pgmap v4706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:53.592 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7818 sec 2024-04-26T20:38:54.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7819 sec 2024-04-26T20:38:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:54 smithi090 ceph-mon[29180]: pgmap v4707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:54 smithi119 ceph-mon[32752]: pgmap v4707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:54 smithi167 ceph-mon[32751]: pgmap v4707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:55.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7820 sec 2024-04-26T20:38:56.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7821 sec 2024-04-26T20:38:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:56 smithi090 ceph-mon[29180]: pgmap v4708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7417.089434 secs 2024-04-26T20:38:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:56 smithi119 ceph-mon[32752]: pgmap v4708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7417.089434 secs 2024-04-26T20:38:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:56 smithi167 ceph-mon[32751]: pgmap v4708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:38:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7417.089434 secs 2024-04-26T20:38:57.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7822 sec 2024-04-26T20:38:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7417.796694 secs 2024-04-26T20:38:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7417.796694 secs 2024-04-26T20:38:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7417.796694 secs 2024-04-26T20:38:58.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7823 sec 2024-04-26T20:38:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:38:58 smithi119 ceph-mon[32752]: pgmap v4709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:38:58 smithi167 ceph-mon[32751]: pgmap v4709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:38:58 smithi090 ceph-mon[29180]: pgmap v4709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:38:59.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7824 sec 2024-04-26T20:39:00.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7825 sec 2024-04-26T20:39:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:00 smithi119 ceph-mon[32752]: pgmap v4710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:00 smithi167 ceph-mon[32751]: pgmap v4710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:00 smithi090 ceph-mon[29180]: pgmap v4710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:01.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7826 sec 2024-04-26T20:39:01.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7422.089518 secs 2024-04-26T20:39:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7422.089518 secs 2024-04-26T20:39:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7422.089518 secs 2024-04-26T20:39:02.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7827 sec 2024-04-26T20:39:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7422.796818 secs 2024-04-26T20:39:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:02 smithi119 ceph-mon[32752]: pgmap v4711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7422.796818 secs 2024-04-26T20:39:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:02 smithi167 ceph-mon[32751]: pgmap v4711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7422.796818 secs 2024-04-26T20:39:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:02 smithi090 ceph-mon[29180]: pgmap v4711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:03.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7828 sec 2024-04-26T20:39:04.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7829 sec 2024-04-26T20:39:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:04 smithi119 ceph-mon[32752]: pgmap v4712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:04 smithi167 ceph-mon[32751]: pgmap v4712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:04 smithi090 ceph-mon[29180]: pgmap v4712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:05.593 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7830 sec 2024-04-26T20:39:06.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7831 sec 2024-04-26T20:39:06.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:06 smithi090 ceph-mon[29180]: pgmap v4713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:06.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7427.089603 secs 2024-04-26T20:39:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:06 smithi119 ceph-mon[32752]: pgmap v4713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7427.089603 secs 2024-04-26T20:39:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:06 smithi167 ceph-mon[32751]: pgmap v4713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7427.089603 secs 2024-04-26T20:39:07.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7832 sec 2024-04-26T20:39:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7427.796896 secs 2024-04-26T20:39:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7427.796896 secs 2024-04-26T20:39:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7427.796896 secs 2024-04-26T20:39:08.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7833 sec 2024-04-26T20:39:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:08 smithi119 ceph-mon[32752]: pgmap v4714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:08 smithi167 ceph-mon[32751]: pgmap v4714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:08 smithi090 ceph-mon[29180]: pgmap v4714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:09.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7834 sec 2024-04-26T20:39:10.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7835 sec 2024-04-26T20:39:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:10 smithi119 ceph-mon[32752]: pgmap v4715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:10 smithi167 ceph-mon[32751]: pgmap v4715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:10 smithi090 ceph-mon[29180]: pgmap v4715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:11.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7836 sec 2024-04-26T20:39:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:11 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7432.089686 secs 2024-04-26T20:39:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:11 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7432.089686 secs 2024-04-26T20:39:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:11 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7432.089686 secs 2024-04-26T20:39:12.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7837 sec 2024-04-26T20:39:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7432.797019 secs 2024-04-26T20:39:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:12 smithi119 ceph-mon[32752]: pgmap v4716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7432.797019 secs 2024-04-26T20:39:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:12 smithi167 ceph-mon[32751]: pgmap v4716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7432.797019 secs 2024-04-26T20:39:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:12 smithi090 ceph-mon[29180]: pgmap v4716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:13.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7838 sec 2024-04-26T20:39:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:39:14.455+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:39:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:39:14.455+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:39:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:39:14.455+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 61 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:39:14.457 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:39:14.455+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 61 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:39:14.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7839 sec 2024-04-26T20:39:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:14 smithi119 ceph-mon[32752]: pgmap v4717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:14 smithi167 ceph-mon[32751]: pgmap v4717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:14 smithi090 ceph-mon[29180]: pgmap v4717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:39:15.594 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7840 sec 2024-04-26T20:39:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:39:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:39:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:39:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:39:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:39:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:39:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:39:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:39:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:39:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:39:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:39:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:39:16.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7841 sec 2024-04-26T20:39:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:16 smithi119 ceph-mon[32752]: pgmap v4718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:16.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7437.089779 secs 2024-04-26T20:39:16.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:16 smithi119 ceph-mon[32752]: pgmap v4719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:39:16.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:16 smithi119 ceph-mon[32752]: pgmap v4720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:16 smithi167 ceph-mon[32751]: pgmap v4718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7437.089779 secs 2024-04-26T20:39:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:16 smithi167 ceph-mon[32751]: pgmap v4719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:39:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:16 smithi167 ceph-mon[32751]: pgmap v4720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:16 smithi090 ceph-mon[29180]: pgmap v4718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:39:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7437.089779 secs 2024-04-26T20:39:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:16 smithi090 ceph-mon[29180]: pgmap v4719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:39:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:16 smithi090 ceph-mon[29180]: pgmap v4720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:17.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7842 sec 2024-04-26T20:39:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7437.797078 secs 2024-04-26T20:39:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7437.797078 secs 2024-04-26T20:39:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7437.797078 secs 2024-04-26T20:39:18.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7843 sec 2024-04-26T20:39:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:18 smithi119 ceph-mon[32752]: pgmap v4721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:39:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:18 smithi167 ceph-mon[32751]: pgmap v4721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:39:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:18 smithi090 ceph-mon[29180]: pgmap v4721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:39:19.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7844 sec 2024-04-26T20:39:20.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7845 sec 2024-04-26T20:39:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:20 smithi119 ceph-mon[32752]: pgmap v4722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:20 smithi167 ceph-mon[32751]: pgmap v4722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:20 smithi090 ceph-mon[29180]: pgmap v4722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:21.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7846 sec 2024-04-26T20:39:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:21 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7442.089869 secs 2024-04-26T20:39:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:21 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7442.089869 secs 2024-04-26T20:39:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:21 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7442.089869 secs 2024-04-26T20:39:22.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7847 sec 2024-04-26T20:39:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7442.797160 secs 2024-04-26T20:39:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:22 smithi119 ceph-mon[32752]: pgmap v4723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7442.797160 secs 2024-04-26T20:39:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:22 smithi167 ceph-mon[32751]: pgmap v4723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7442.797160 secs 2024-04-26T20:39:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:22 smithi090 ceph-mon[29180]: pgmap v4723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:23.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7848 sec 2024-04-26T20:39:24.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7849 sec 2024-04-26T20:39:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:24 smithi119 ceph-mon[32752]: pgmap v4724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:24 smithi167 ceph-mon[32751]: pgmap v4724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:24 smithi090 ceph-mon[29180]: pgmap v4724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:39:25.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7850 sec 2024-04-26T20:39:26.595 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7851 sec 2024-04-26T20:39:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:26 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7447.089928 secs 2024-04-26T20:39:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:26 smithi119 ceph-mon[32752]: pgmap v4725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:39:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:26 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7447.089928 secs 2024-04-26T20:39:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:26 smithi167 ceph-mon[32751]: pgmap v4725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:39:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:26 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7447.089928 secs 2024-04-26T20:39:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:26 smithi090 ceph-mon[29180]: pgmap v4725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:39:27.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7852 sec 2024-04-26T20:39:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7447.797295 secs 2024-04-26T20:39:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7447.797295 secs 2024-04-26T20:39:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7447.797295 secs 2024-04-26T20:39:28.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7853 sec 2024-04-26T20:39:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:28 smithi119 ceph-mon[32752]: pgmap v4726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:28 smithi167 ceph-mon[32751]: pgmap v4726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:28 smithi090 ceph-mon[29180]: pgmap v4726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:29.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7854 sec 2024-04-26T20:39:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:29 smithi119 ceph-mon[32752]: pgmap v4727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:29 smithi167 ceph-mon[32751]: pgmap v4727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:29 smithi090 ceph-mon[29180]: pgmap v4727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:30.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7855 sec 2024-04-26T20:39:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:30 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7452.090015 secs 2024-04-26T20:39:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:30 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7452.090015 secs 2024-04-26T20:39:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:30 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7452.090015 secs 2024-04-26T20:39:31.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7856 sec 2024-04-26T20:39:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7452.797411 secs 2024-04-26T20:39:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:31 smithi167 ceph-mon[32751]: pgmap v4728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7452.797411 secs 2024-04-26T20:39:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:31 smithi090 ceph-mon[29180]: pgmap v4728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7452.797411 secs 2024-04-26T20:39:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:31 smithi119 ceph-mon[32752]: pgmap v4728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:32.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7857 sec 2024-04-26T20:39:33.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7858 sec 2024-04-26T20:39:34.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7859 sec 2024-04-26T20:39:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:34 smithi119 ceph-mon[32752]: pgmap v4729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:34 smithi167 ceph-mon[32751]: pgmap v4729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:34 smithi090 ceph-mon[29180]: pgmap v4729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:35.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7860 sec 2024-04-26T20:39:36.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7861 sec 2024-04-26T20:39:36.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:36 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7457.090100 secs 2024-04-26T20:39:36.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:36 smithi090 ceph-mon[29180]: pgmap v4730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:36 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7457.090100 secs 2024-04-26T20:39:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:36 smithi119 ceph-mon[32752]: pgmap v4730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:36 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7457.090100 secs 2024-04-26T20:39:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:36 smithi167 ceph-mon[32751]: pgmap v4730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:37.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7862 sec 2024-04-26T20:39:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7457.797512 secs 2024-04-26T20:39:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7457.797512 secs 2024-04-26T20:39:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7457.797512 secs 2024-04-26T20:39:38.596 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7863 sec 2024-04-26T20:39:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:38 smithi119 ceph-mon[32752]: pgmap v4731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:38 smithi167 ceph-mon[32751]: pgmap v4731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:38 smithi090 ceph-mon[29180]: pgmap v4731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:39.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7864 sec 2024-04-26T20:39:40.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7865 sec 2024-04-26T20:39:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:40 smithi119 ceph-mon[32752]: pgmap v4732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:40 smithi167 ceph-mon[32751]: pgmap v4732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:40 smithi090 ceph-mon[29180]: pgmap v4732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:41.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7866 sec 2024-04-26T20:39:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:41 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7462.090162 secs 2024-04-26T20:39:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:41 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7462.090162 secs 2024-04-26T20:39:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:41 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7462.090162 secs 2024-04-26T20:39:42.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7867 sec 2024-04-26T20:39:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7462.797632 secs 2024-04-26T20:39:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:42 smithi119 ceph-mon[32752]: pgmap v4733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7462.797632 secs 2024-04-26T20:39:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:42 smithi167 ceph-mon[32751]: pgmap v4733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7462.797632 secs 2024-04-26T20:39:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:42 smithi090 ceph-mon[29180]: pgmap v4733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:43.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7868 sec 2024-04-26T20:39:44.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7869 sec 2024-04-26T20:39:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:44 smithi119 ceph-mon[32752]: pgmap v4734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:44 smithi167 ceph-mon[32751]: pgmap v4734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:44 smithi090 ceph-mon[29180]: pgmap v4734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:45.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7870 sec 2024-04-26T20:39:46.597 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7871 sec 2024-04-26T20:39:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:46 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7467.090247 secs 2024-04-26T20:39:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:46 smithi119 ceph-mon[32752]: pgmap v4735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:46 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7467.090247 secs 2024-04-26T20:39:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:46 smithi167 ceph-mon[32751]: pgmap v4735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:46 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7467.090247 secs 2024-04-26T20:39:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:46 smithi090 ceph-mon[29180]: pgmap v4735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:47.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7872 sec 2024-04-26T20:39:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7467.797751 secs 2024-04-26T20:39:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7467.797751 secs 2024-04-26T20:39:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7467.797751 secs 2024-04-26T20:39:48.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7873 sec 2024-04-26T20:39:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:48 smithi119 ceph-mon[32752]: pgmap v4736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:48 smithi167 ceph-mon[32751]: pgmap v4736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:48 smithi090 ceph-mon[29180]: pgmap v4736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:49.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7874 sec 2024-04-26T20:39:50.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7875 sec 2024-04-26T20:39:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:50 smithi119 ceph-mon[32752]: pgmap v4737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:50 smithi167 ceph-mon[32751]: pgmap v4737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:50 smithi090 ceph-mon[29180]: pgmap v4737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:51.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7876 sec 2024-04-26T20:39:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:51 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7472.090340 secs 2024-04-26T20:39:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:51 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7472.090340 secs 2024-04-26T20:39:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:51 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7472.090340 secs 2024-04-26T20:39:52.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7877 sec 2024-04-26T20:39:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7472.797869 secs 2024-04-26T20:39:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:52 smithi119 ceph-mon[32752]: pgmap v4738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7472.797869 secs 2024-04-26T20:39:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:52 smithi167 ceph-mon[32751]: pgmap v4738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7472.797869 secs 2024-04-26T20:39:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:52 smithi090 ceph-mon[29180]: pgmap v4738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:53.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7878 sec 2024-04-26T20:39:54.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7879 sec 2024-04-26T20:39:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:54 smithi119 ceph-mon[32752]: pgmap v4739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:54 smithi167 ceph-mon[32751]: pgmap v4739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:54 smithi090 ceph-mon[29180]: pgmap v4739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:39:55.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7880 sec 2024-04-26T20:39:56.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7881 sec 2024-04-26T20:39:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:56 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7477.090400 secs 2024-04-26T20:39:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:56 smithi119 ceph-mon[32752]: pgmap v4740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:56 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7477.090400 secs 2024-04-26T20:39:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:56 smithi167 ceph-mon[32751]: pgmap v4740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:56 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7477.090400 secs 2024-04-26T20:39:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:56 smithi090 ceph-mon[29180]: pgmap v4740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:57.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7882 sec 2024-04-26T20:39:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7477.797942 secs 2024-04-26T20:39:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7477.797942 secs 2024-04-26T20:39:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7477.797942 secs 2024-04-26T20:39:58.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7883 sec 2024-04-26T20:39:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:39:58 smithi119 ceph-mon[32752]: pgmap v4741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:39:58 smithi167 ceph-mon[32751]: pgmap v4741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:39:58 smithi090 ceph-mon[29180]: pgmap v4741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:39:59.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7884 sec 2024-04-26T20:40:00.598 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7885 sec 2024-04-26T20:40:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:00 smithi119 ceph-mon[32752]: pgmap v4742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:40:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:00 smithi167 ceph-mon[32751]: pgmap v4742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:40:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:00 smithi090 ceph-mon[29180]: pgmap v4742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:40:01.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7886 sec 2024-04-26T20:40:01.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:01 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7482.090487 secs 2024-04-26T20:40:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:01 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7482.090487 secs 2024-04-26T20:40:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:01 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7482.090487 secs 2024-04-26T20:40:02.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7887 sec 2024-04-26T20:40:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7482.798062 secs 2024-04-26T20:40:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:02 smithi119 ceph-mon[32752]: pgmap v4743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7482.798062 secs 2024-04-26T20:40:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:02 smithi167 ceph-mon[32751]: pgmap v4743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7482.798062 secs 2024-04-26T20:40:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:02 smithi090 ceph-mon[29180]: pgmap v4743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:03.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7888 sec 2024-04-26T20:40:04.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7889 sec 2024-04-26T20:40:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:04 smithi119 ceph-mon[32752]: pgmap v4744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:04 smithi167 ceph-mon[32751]: pgmap v4744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:04 smithi090 ceph-mon[29180]: pgmap v4744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:05.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7890 sec 2024-04-26T20:40:06.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7891 sec 2024-04-26T20:40:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:06 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7487.090570 secs 2024-04-26T20:40:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:06 smithi119 ceph-mon[32752]: pgmap v4745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:06 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7487.090570 secs 2024-04-26T20:40:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:06 smithi167 ceph-mon[32751]: pgmap v4745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:06 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7487.090570 secs 2024-04-26T20:40:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:06 smithi090 ceph-mon[29180]: pgmap v4745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:07.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7892 sec 2024-04-26T20:40:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:07 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 7487.798171 secs 2024-04-26T20:40:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:07 smithi119 ceph-mon[32752]: slow request 722.959359 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:40:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:07 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 7487.798171 secs 2024-04-26T20:40:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:07 smithi167 ceph-mon[32751]: slow request 722.959359 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:40:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:07 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 7487.798171 secs 2024-04-26T20:40:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:07 smithi090 ceph-mon[29180]: slow request 722.959359 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:40:08.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7893 sec 2024-04-26T20:40:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:08 smithi119 ceph-mon[32752]: pgmap v4746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:08 smithi167 ceph-mon[32751]: pgmap v4746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:08 smithi090 ceph-mon[29180]: pgmap v4746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:09.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7894 sec 2024-04-26T20:40:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:09 smithi119 ceph-mon[32752]: pgmap v4747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:09 smithi167 ceph-mon[32751]: pgmap v4747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:09 smithi090 ceph-mon[29180]: pgmap v4747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:10.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7895 sec 2024-04-26T20:40:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:10 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7492.090654 secs 2024-04-26T20:40:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:10 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7492.090654 secs 2024-04-26T20:40:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:10 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7492.090654 secs 2024-04-26T20:40:11.599 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7896 sec 2024-04-26T20:40:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7492.798287 secs 2024-04-26T20:40:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:11 smithi090 ceph-mon[29180]: pgmap v4748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7492.798287 secs 2024-04-26T20:40:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:11 smithi119 ceph-mon[32752]: pgmap v4748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7492.798287 secs 2024-04-26T20:40:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:11 smithi167 ceph-mon[32751]: pgmap v4748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:12.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7897 sec 2024-04-26T20:40:13.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7898 sec 2024-04-26T20:40:14.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7899 sec 2024-04-26T20:40:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:14 smithi119 ceph-mon[32752]: pgmap v4749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:14 smithi167 ceph-mon[32751]: pgmap v4749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:14 smithi090 ceph-mon[29180]: pgmap v4749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:15.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7900 sec 2024-04-26T20:40:16.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7901 sec 2024-04-26T20:40:16.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:16 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 7497.090738 secs 2024-04-26T20:40:16.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:16 smithi090 ceph-mon[29180]: pgmap v4750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:16.704 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:40:16.724 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:16 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 7497.090738 secs 2024-04-26T20:40:16.724 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:16 smithi119 ceph-mon[32752]: pgmap v4750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:16.724 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:40:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:16 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 7497.090738 secs 2024-04-26T20:40:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:16 smithi167 ceph-mon[32751]: pgmap v4750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:40:17.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7902 sec 2024-04-26T20:40:17.707 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7497.798352 secs 2024-04-26T20:40:17.762 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7497.798352 secs 2024-04-26T20:40:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7497.798352 secs 2024-04-26T20:40:18.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7903 sec 2024-04-26T20:40:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:18 smithi090 ceph-mon[29180]: pgmap v4751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:18 smithi119 ceph-mon[32752]: pgmap v4751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:18 smithi167 ceph-mon[32751]: pgmap v4751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:19.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7904 sec 2024-04-26T20:40:20.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7905 sec 2024-04-26T20:40:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: pgmap v4752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:40:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:40:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: pgmap v4752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:40:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:40:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: pgmap v4752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:40:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:40:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:40:21.600 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7906 sec 2024-04-26T20:40:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:21 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7502.090825 secs 2024-04-26T20:40:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:21 smithi119 ceph-mon[32752]: pgmap v4753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T20:40:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:21 smithi119 ceph-mon[32752]: pgmap v4754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:21 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7502.090825 secs 2024-04-26T20:40:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:21 smithi167 ceph-mon[32751]: pgmap v4753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T20:40:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:21 smithi167 ceph-mon[32751]: pgmap v4754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:21 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7502.090825 secs 2024-04-26T20:40:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:21 smithi090 ceph-mon[29180]: pgmap v4753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T20:40:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:21 smithi090 ceph-mon[29180]: pgmap v4754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:22.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7907 sec 2024-04-26T20:40:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7502.798438 secs 2024-04-26T20:40:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7502.798438 secs 2024-04-26T20:40:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7502.798438 secs 2024-04-26T20:40:23.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7908 sec 2024-04-26T20:40:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:23 smithi119 ceph-mon[32752]: pgmap v4755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T20:40:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:40:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:23 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:40:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:23 smithi167 ceph-mon[32751]: pgmap v4755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T20:40:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:40:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:23 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:40:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:23 smithi090 ceph-mon[29180]: pgmap v4755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T20:40:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:40:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:23 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:40:24.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7909 sec 2024-04-26T20:40:25.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7910 sec 2024-04-26T20:40:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:25 smithi119 ceph-mon[32752]: pgmap v4756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:25 smithi167 ceph-mon[32751]: pgmap v4756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:25.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:25 smithi090 ceph-mon[29180]: pgmap v4756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:26.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7911 sec 2024-04-26T20:40:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:26 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7507.090910 secs 2024-04-26T20:40:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:26 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7507.090910 secs 2024-04-26T20:40:26.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:26 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7507.090910 secs 2024-04-26T20:40:27.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7912 sec 2024-04-26T20:40:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7507.798557 secs 2024-04-26T20:40:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:27 smithi119 ceph-mon[32752]: pgmap v4757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7507.798557 secs 2024-04-26T20:40:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:27 smithi167 ceph-mon[32751]: pgmap v4757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7507.798557 secs 2024-04-26T20:40:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:27 smithi090 ceph-mon[29180]: pgmap v4757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:28.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7913 sec 2024-04-26T20:40:29.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7914 sec 2024-04-26T20:40:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:29 smithi119 ceph-mon[32752]: pgmap v4758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T20:40:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:29 smithi167 ceph-mon[32751]: pgmap v4758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T20:40:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:29 smithi090 ceph-mon[29180]: pgmap v4758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T20:40:30.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7915 sec 2024-04-26T20:40:31.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7916 sec 2024-04-26T20:40:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:31 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7512.090971 secs 2024-04-26T20:40:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:31 smithi119 ceph-mon[32752]: pgmap v4759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:40:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:31 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7512.090971 secs 2024-04-26T20:40:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:31 smithi167 ceph-mon[32751]: pgmap v4759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:40:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:31 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7512.090971 secs 2024-04-26T20:40:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:31 smithi090 ceph-mon[29180]: pgmap v4759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:40:32.601 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7917 sec 2024-04-26T20:40:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7512.798632 secs 2024-04-26T20:40:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7512.798632 secs 2024-04-26T20:40:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7512.798632 secs 2024-04-26T20:40:33.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7918 sec 2024-04-26T20:40:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:33 smithi119 ceph-mon[32752]: pgmap v4760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:33 smithi167 ceph-mon[32751]: pgmap v4760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:33 smithi090 ceph-mon[29180]: pgmap v4760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:34.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7919 sec 2024-04-26T20:40:35.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7920 sec 2024-04-26T20:40:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:35 smithi119 ceph-mon[32752]: pgmap v4761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:35 smithi167 ceph-mon[32751]: pgmap v4761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:35 smithi090 ceph-mon[29180]: pgmap v4761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:36.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7921 sec 2024-04-26T20:40:36.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:36 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7517.091055 secs 2024-04-26T20:40:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:36 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7517.091055 secs 2024-04-26T20:40:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:36 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7517.091055 secs 2024-04-26T20:40:37.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7922 sec 2024-04-26T20:40:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7517.798742 secs 2024-04-26T20:40:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:37 smithi119 ceph-mon[32752]: pgmap v4762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7517.798742 secs 2024-04-26T20:40:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:37 smithi167 ceph-mon[32751]: pgmap v4762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7517.798742 secs 2024-04-26T20:40:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:37 smithi090 ceph-mon[29180]: pgmap v4762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:38.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7923 sec 2024-04-26T20:40:39.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7924 sec 2024-04-26T20:40:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:39 smithi119 ceph-mon[32752]: pgmap v4763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:39 smithi167 ceph-mon[32751]: pgmap v4763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:39 smithi090 ceph-mon[29180]: pgmap v4763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:40.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7925 sec 2024-04-26T20:40:41.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7926 sec 2024-04-26T20:40:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:41 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7522.091118 secs 2024-04-26T20:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:41 smithi119 ceph-mon[32752]: pgmap v4764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:41 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7522.091118 secs 2024-04-26T20:40:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:41 smithi167 ceph-mon[32751]: pgmap v4764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:41 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7522.091118 secs 2024-04-26T20:40:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:41 smithi090 ceph-mon[29180]: pgmap v4764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:42.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7927 sec 2024-04-26T20:40:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7522.798816 secs 2024-04-26T20:40:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7522.798816 secs 2024-04-26T20:40:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7522.798816 secs 2024-04-26T20:40:43.602 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7928 sec 2024-04-26T20:40:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:43 smithi119 ceph-mon[32752]: pgmap v4765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:43 smithi167 ceph-mon[32751]: pgmap v4765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:43 smithi090 ceph-mon[29180]: pgmap v4765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:44.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7929 sec 2024-04-26T20:40:45.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7930 sec 2024-04-26T20:40:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:45 smithi119 ceph-mon[32752]: pgmap v4766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:45 smithi167 ceph-mon[32751]: pgmap v4766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:45 smithi090 ceph-mon[29180]: pgmap v4766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:46.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7931 sec 2024-04-26T20:40:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:46 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7527.091203 secs 2024-04-26T20:40:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:46 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7527.091203 secs 2024-04-26T20:40:46.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:46 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7527.091203 secs 2024-04-26T20:40:47.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7932 sec 2024-04-26T20:40:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7527.798880 secs 2024-04-26T20:40:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:47 smithi119 ceph-mon[32752]: pgmap v4767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7527.798880 secs 2024-04-26T20:40:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:47 smithi167 ceph-mon[32751]: pgmap v4767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7527.798880 secs 2024-04-26T20:40:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:47 smithi090 ceph-mon[29180]: pgmap v4767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:48.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7933 sec 2024-04-26T20:40:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:48 smithi119 ceph-mon[32752]: pgmap v4768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:48 smithi167 ceph-mon[32751]: pgmap v4768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:48 smithi090 ceph-mon[29180]: pgmap v4768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:49.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7934 sec 2024-04-26T20:40:50.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7935 sec 2024-04-26T20:40:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7532.091286 secs 2024-04-26T20:40:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:50 smithi119 ceph-mon[32752]: pgmap v4769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7532.091286 secs 2024-04-26T20:40:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:50 smithi167 ceph-mon[32751]: pgmap v4769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7532.091286 secs 2024-04-26T20:40:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:50 smithi090 ceph-mon[29180]: pgmap v4769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:51.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7936 sec 2024-04-26T20:40:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7532.798959 secs 2024-04-26T20:40:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7532.798959 secs 2024-04-26T20:40:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7532.798959 secs 2024-04-26T20:40:52.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7937 sec 2024-04-26T20:40:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:52 smithi119 ceph-mon[32752]: pgmap v4770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:52 smithi167 ceph-mon[32751]: pgmap v4770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:52 smithi090 ceph-mon[29180]: pgmap v4770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:53.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7938 sec 2024-04-26T20:40:54.603 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7939 sec 2024-04-26T20:40:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:54 smithi119 ceph-mon[32752]: pgmap v4771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:54 smithi167 ceph-mon[32751]: pgmap v4771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:54 smithi090 ceph-mon[29180]: pgmap v4771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:40:55.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7940 sec 2024-04-26T20:40:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7537.091368 secs 2024-04-26T20:40:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7537.091368 secs 2024-04-26T20:40:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7537.091368 secs 2024-04-26T20:40:56.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7941 sec 2024-04-26T20:40:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7537.799054 secs 2024-04-26T20:40:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:56 smithi119 ceph-mon[32752]: pgmap v4772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7537.799054 secs 2024-04-26T20:40:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:56 smithi167 ceph-mon[32751]: pgmap v4772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7537.799054 secs 2024-04-26T20:40:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:56 smithi090 ceph-mon[29180]: pgmap v4772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:57.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7942 sec 2024-04-26T20:40:58.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7943 sec 2024-04-26T20:40:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:40:58 smithi119 ceph-mon[32752]: pgmap v4773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:40:58 smithi167 ceph-mon[32751]: pgmap v4773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:40:58 smithi090 ceph-mon[29180]: pgmap v4773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:40:59.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7944 sec 2024-04-26T20:41:00.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7945 sec 2024-04-26T20:41:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7542.091452 secs 2024-04-26T20:41:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:00 smithi119 ceph-mon[32752]: pgmap v4774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7542.091452 secs 2024-04-26T20:41:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:00 smithi167 ceph-mon[32751]: pgmap v4774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7542.091452 secs 2024-04-26T20:41:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:00 smithi090 ceph-mon[29180]: pgmap v4774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:01.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7946 sec 2024-04-26T20:41:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7542.799171 secs 2024-04-26T20:41:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7542.799171 secs 2024-04-26T20:41:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7542.799171 secs 2024-04-26T20:41:02.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7947 sec 2024-04-26T20:41:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:02 smithi119 ceph-mon[32752]: pgmap v4775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:02 smithi167 ceph-mon[32751]: pgmap v4775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:02 smithi090 ceph-mon[29180]: pgmap v4775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:03.604 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7948 sec 2024-04-26T20:41:04.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7949 sec 2024-04-26T20:41:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:04 smithi119 ceph-mon[32752]: pgmap v4776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:04 smithi167 ceph-mon[32751]: pgmap v4776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:04 smithi090 ceph-mon[29180]: pgmap v4776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:05.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7950 sec 2024-04-26T20:41:06.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7547.091540 secs 2024-04-26T20:41:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7547.091540 secs 2024-04-26T20:41:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7547.091540 secs 2024-04-26T20:41:06.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7951 sec 2024-04-26T20:41:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7547.799289 secs 2024-04-26T20:41:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:06 smithi119 ceph-mon[32752]: pgmap v4777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7547.799289 secs 2024-04-26T20:41:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:06 smithi167 ceph-mon[32751]: pgmap v4777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7547.799289 secs 2024-04-26T20:41:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:06 smithi090 ceph-mon[29180]: pgmap v4777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:07.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7952 sec 2024-04-26T20:41:08.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7953 sec 2024-04-26T20:41:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:08 smithi119 ceph-mon[32752]: pgmap v4778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:08 smithi167 ceph-mon[32751]: pgmap v4778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:08 smithi090 ceph-mon[29180]: pgmap v4778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:09.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7954 sec 2024-04-26T20:41:10.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7955 sec 2024-04-26T20:41:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7552.091624 secs 2024-04-26T20:41:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:10 smithi119 ceph-mon[32752]: pgmap v4779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7552.091624 secs 2024-04-26T20:41:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:10 smithi167 ceph-mon[32751]: pgmap v4779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7552.091624 secs 2024-04-26T20:41:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:10 smithi090 ceph-mon[29180]: pgmap v4779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:11.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7956 sec 2024-04-26T20:41:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7552.799425 secs 2024-04-26T20:41:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7552.799425 secs 2024-04-26T20:41:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7552.799425 secs 2024-04-26T20:41:12.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7957 sec 2024-04-26T20:41:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:13 smithi119 ceph-mon[32752]: pgmap v4780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:13 smithi167 ceph-mon[32751]: pgmap v4780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:13 smithi090 ceph-mon[29180]: pgmap v4780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:13.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7958 sec 2024-04-26T20:41:14.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7959 sec 2024-04-26T20:41:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:15 smithi119 ceph-mon[32752]: pgmap v4781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:15 smithi167 ceph-mon[32751]: pgmap v4781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:15 smithi090 ceph-mon[29180]: pgmap v4781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:15.605 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7960 sec 2024-04-26T20:41:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:16 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7557.091701 secs 2024-04-26T20:41:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:16 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7557.091701 secs 2024-04-26T20:41:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:16 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7557.091701 secs 2024-04-26T20:41:16.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7961 sec 2024-04-26T20:41:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7557.799542 secs 2024-04-26T20:41:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:17 smithi119 ceph-mon[32752]: pgmap v4782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7557.799542 secs 2024-04-26T20:41:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:17 smithi167 ceph-mon[32751]: pgmap v4782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7557.799542 secs 2024-04-26T20:41:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:17 smithi090 ceph-mon[29180]: pgmap v4782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:17.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7962 sec 2024-04-26T20:41:18.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7963 sec 2024-04-26T20:41:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:19 smithi119 ceph-mon[32752]: pgmap v4783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:19 smithi167 ceph-mon[32751]: pgmap v4783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:19 smithi090 ceph-mon[29180]: pgmap v4783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:19.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7964 sec 2024-04-26T20:41:20.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7965 sec 2024-04-26T20:41:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:21 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7562.091810 secs 2024-04-26T20:41:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:21 smithi119 ceph-mon[32752]: pgmap v4784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:41:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:21 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7562.091810 secs 2024-04-26T20:41:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:21 smithi167 ceph-mon[32751]: pgmap v4784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:41:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:21 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7562.091810 secs 2024-04-26T20:41:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:21 smithi090 ceph-mon[29180]: pgmap v4784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:41:21.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7966 sec 2024-04-26T20:41:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7562.799607 secs 2024-04-26T20:41:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7562.799607 secs 2024-04-26T20:41:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7562.799607 secs 2024-04-26T20:41:22.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7967 sec 2024-04-26T20:41:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:23 smithi119 ceph-mon[32752]: pgmap v4785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:23 smithi167 ceph-mon[32751]: pgmap v4785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:23 smithi090 ceph-mon[29180]: pgmap v4785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:23.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7968 sec 2024-04-26T20:41:24.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7969 sec 2024-04-26T20:41:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: pgmap v4786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:41:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:41:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: pgmap v4787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-26T20:41:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: pgmap v4788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7970 sec 2024-04-26T20:41:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: pgmap v4786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:41:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:41:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: pgmap v4787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-26T20:41:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: pgmap v4788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: pgmap v4786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:41:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:41:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: pgmap v4787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-26T20:41:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: pgmap v4788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:41:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:26 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7567.091894 secs 2024-04-26T20:41:26.606 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7971 sec 2024-04-26T20:41:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:26 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7567.091894 secs 2024-04-26T20:41:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:26 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7567.091894 secs 2024-04-26T20:41:27.607 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7972 sec 2024-04-26T20:41:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7567.799731 secs 2024-04-26T20:41:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:27 smithi090 ceph-mon[29180]: pgmap v4789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7567.799731 secs 2024-04-26T20:41:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:27 smithi119 ceph-mon[32752]: pgmap v4789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7567.799731 secs 2024-04-26T20:41:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:27 smithi167 ceph-mon[32751]: pgmap v4789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:28.607 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7973 sec 2024-04-26T20:41:29.474 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:41:29.474+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:41:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:41:29.474+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:41:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:41:29.474+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 62 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:41:29.475 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:41:29.474+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 62 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:41:29.607 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7974 sec 2024-04-26T20:41:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:29 smithi090 ceph-mon[29180]: pgmap v4790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T20:41:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:29 smithi119 ceph-mon[32752]: pgmap v4790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T20:41:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:29 smithi167 ceph-mon[32751]: pgmap v4790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T20:41:30.607 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7975 sec 2024-04-26T20:41:31.607 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7976 sec 2024-04-26T20:41:31.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:31 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7572.091975 secs 2024-04-26T20:41:31.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:31 smithi167 ceph-mon[32751]: pgmap v4791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:31 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7572.091975 secs 2024-04-26T20:41:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:31 smithi090 ceph-mon[29180]: pgmap v4791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:31 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7572.091975 secs 2024-04-26T20:41:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:31 smithi119 ceph-mon[32752]: pgmap v4791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:32.607 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7977 sec 2024-04-26T20:41:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7572.799848 secs 2024-04-26T20:41:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7572.799848 secs 2024-04-26T20:41:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7572.799848 secs 2024-04-26T20:41:33.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7978 sec 2024-04-26T20:41:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:33 smithi090 ceph-mon[29180]: pgmap v4792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T20:41:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:33 smithi119 ceph-mon[32752]: pgmap v4792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T20:41:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:33 smithi167 ceph-mon[32751]: pgmap v4792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T20:41:34.607 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7979 sec 2024-04-26T20:41:35.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7980 sec 2024-04-26T20:41:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:35 smithi090 ceph-mon[29180]: pgmap v4793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:41:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:35 smithi119 ceph-mon[32752]: pgmap v4793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:41:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:35 smithi167 ceph-mon[32751]: pgmap v4793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:41:36.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7981 sec 2024-04-26T20:41:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:36 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7577.092054 secs 2024-04-26T20:41:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:36 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7577.092054 secs 2024-04-26T20:41:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:36 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7577.092054 secs 2024-04-26T20:41:37.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7982 sec 2024-04-26T20:41:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7577.799920 secs 2024-04-26T20:41:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:37 smithi090 ceph-mon[29180]: pgmap v4794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7577.799920 secs 2024-04-26T20:41:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:37 smithi119 ceph-mon[32752]: pgmap v4794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7577.799920 secs 2024-04-26T20:41:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:37 smithi167 ceph-mon[32751]: pgmap v4794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:38.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7983 sec 2024-04-26T20:41:39.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7984 sec 2024-04-26T20:41:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:39 smithi090 ceph-mon[29180]: pgmap v4795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:39 smithi119 ceph-mon[32752]: pgmap v4795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:39 smithi167 ceph-mon[32751]: pgmap v4795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:40.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7985 sec 2024-04-26T20:41:41.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7986 sec 2024-04-26T20:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:41 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7582.092114 secs 2024-04-26T20:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:41 smithi090 ceph-mon[29180]: pgmap v4796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:41 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7582.092114 secs 2024-04-26T20:41:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:41 smithi119 ceph-mon[32752]: pgmap v4796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:41 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7582.092114 secs 2024-04-26T20:41:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:41 smithi167 ceph-mon[32751]: pgmap v4796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:42.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7987 sec 2024-04-26T20:41:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7582.800053 secs 2024-04-26T20:41:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7582.800053 secs 2024-04-26T20:41:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7582.800053 secs 2024-04-26T20:41:43.608 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7988 sec 2024-04-26T20:41:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:43 smithi090 ceph-mon[29180]: pgmap v4797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:43 smithi119 ceph-mon[32752]: pgmap v4797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:43 smithi167 ceph-mon[32751]: pgmap v4797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:44.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7989 sec 2024-04-26T20:41:45.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7990 sec 2024-04-26T20:41:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:45 smithi090 ceph-mon[29180]: pgmap v4798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:45 smithi119 ceph-mon[32752]: pgmap v4798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:45 smithi167 ceph-mon[32751]: pgmap v4798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:46.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7991 sec 2024-04-26T20:41:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:46 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7587.092196 secs 2024-04-26T20:41:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:46 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7587.092196 secs 2024-04-26T20:41:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:46 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7587.092196 secs 2024-04-26T20:41:47.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7992 sec 2024-04-26T20:41:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7587.800169 secs 2024-04-26T20:41:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:47 smithi090 ceph-mon[29180]: pgmap v4799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7587.800169 secs 2024-04-26T20:41:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:47 smithi119 ceph-mon[32752]: pgmap v4799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7587.800169 secs 2024-04-26T20:41:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:47 smithi167 ceph-mon[32751]: pgmap v4799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:48.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7993 sec 2024-04-26T20:41:49.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7994 sec 2024-04-26T20:41:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:49 smithi090 ceph-mon[29180]: pgmap v4800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:49 smithi119 ceph-mon[32752]: pgmap v4800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:49 smithi167 ceph-mon[32751]: pgmap v4800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:50.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7995 sec 2024-04-26T20:41:51.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7996 sec 2024-04-26T20:41:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:51 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7592.092275 secs 2024-04-26T20:41:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:51 smithi090 ceph-mon[29180]: pgmap v4801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:41:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:51 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7592.092275 secs 2024-04-26T20:41:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:51 smithi119 ceph-mon[32752]: pgmap v4801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:41:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:51 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7592.092275 secs 2024-04-26T20:41:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:51 smithi167 ceph-mon[32751]: pgmap v4801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T20:41:52.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7997 sec 2024-04-26T20:41:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7592.800307 secs 2024-04-26T20:41:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7592.800307 secs 2024-04-26T20:41:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7592.800307 secs 2024-04-26T20:41:53.609 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7998 sec 2024-04-26T20:41:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:53 smithi119 ceph-mon[32752]: pgmap v4802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:53 smithi167 ceph-mon[32751]: pgmap v4802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:53 smithi090 ceph-mon[29180]: pgmap v4802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:54.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 7999 sec 2024-04-26T20:41:55.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8000 sec 2024-04-26T20:41:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:55 smithi119 ceph-mon[32752]: pgmap v4803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:55 smithi167 ceph-mon[32751]: pgmap v4803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:55 smithi090 ceph-mon[29180]: pgmap v4803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:56.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8001 sec 2024-04-26T20:41:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:56 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7597.092353 secs 2024-04-26T20:41:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:56 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7597.092353 secs 2024-04-26T20:41:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:56 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7597.092353 secs 2024-04-26T20:41:57.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8002 sec 2024-04-26T20:41:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7597.800425 secs 2024-04-26T20:41:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:57 smithi119 ceph-mon[32752]: pgmap v4804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7597.800425 secs 2024-04-26T20:41:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:57 smithi167 ceph-mon[32751]: pgmap v4804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7597.800425 secs 2024-04-26T20:41:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:57 smithi090 ceph-mon[29180]: pgmap v4804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:41:58.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8003 sec 2024-04-26T20:41:59.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8004 sec 2024-04-26T20:41:59.683 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:41:59 smithi090 ceph-mon[29180]: pgmap v4805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:41:59 smithi119 ceph-mon[32752]: pgmap v4805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:41:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:41:59 smithi167 ceph-mon[32751]: pgmap v4805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:00.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8005 sec 2024-04-26T20:42:01.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8006 sec 2024-04-26T20:42:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:01 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7602.092437 secs 2024-04-26T20:42:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:01 smithi119 ceph-mon[32752]: pgmap v4806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:01 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7602.092437 secs 2024-04-26T20:42:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:01 smithi167 ceph-mon[32751]: pgmap v4806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:01 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7602.092437 secs 2024-04-26T20:42:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:01 smithi090 ceph-mon[29180]: pgmap v4806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:02.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8007 sec 2024-04-26T20:42:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7602.800507 secs 2024-04-26T20:42:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7602.800507 secs 2024-04-26T20:42:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7602.800507 secs 2024-04-26T20:42:03.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8008 sec 2024-04-26T20:42:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:03 smithi119 ceph-mon[32752]: pgmap v4807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:03 smithi167 ceph-mon[32751]: pgmap v4807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:03 smithi090 ceph-mon[29180]: pgmap v4807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:04.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8009 sec 2024-04-26T20:42:05.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8010 sec 2024-04-26T20:42:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:05 smithi119 ceph-mon[32752]: pgmap v4808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:05 smithi167 ceph-mon[32751]: pgmap v4808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:05 smithi090 ceph-mon[29180]: pgmap v4808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:06.610 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8011 sec 2024-04-26T20:42:06.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:06 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7607.092520 secs 2024-04-26T20:42:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:06 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7607.092520 secs 2024-04-26T20:42:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:06 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7607.092520 secs 2024-04-26T20:42:07.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8012 sec 2024-04-26T20:42:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7607.800621 secs 2024-04-26T20:42:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:07 smithi119 ceph-mon[32752]: pgmap v4809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7607.800621 secs 2024-04-26T20:42:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:07 smithi167 ceph-mon[32751]: pgmap v4809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7607.800621 secs 2024-04-26T20:42:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:07 smithi090 ceph-mon[29180]: pgmap v4809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:08.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8013 sec 2024-04-26T20:42:09.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8014 sec 2024-04-26T20:42:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:09 smithi119 ceph-mon[32752]: pgmap v4810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:09 smithi167 ceph-mon[32751]: pgmap v4810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:09 smithi090 ceph-mon[29180]: pgmap v4810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:10.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8015 sec 2024-04-26T20:42:11.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8016 sec 2024-04-26T20:42:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:11 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7612.092602 secs 2024-04-26T20:42:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:11 smithi119 ceph-mon[32752]: pgmap v4811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:11 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7612.092602 secs 2024-04-26T20:42:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:11 smithi167 ceph-mon[32751]: pgmap v4811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:11 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7612.092602 secs 2024-04-26T20:42:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:11 smithi090 ceph-mon[29180]: pgmap v4811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:12.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8017 sec 2024-04-26T20:42:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7612.800737 secs 2024-04-26T20:42:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7612.800737 secs 2024-04-26T20:42:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7612.800737 secs 2024-04-26T20:42:13.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8018 sec 2024-04-26T20:42:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:13 smithi119 ceph-mon[32752]: pgmap v4812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:13 smithi167 ceph-mon[32751]: pgmap v4812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:13 smithi090 ceph-mon[29180]: pgmap v4812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:14.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8019 sec 2024-04-26T20:42:15.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8020 sec 2024-04-26T20:42:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:15 smithi119 ceph-mon[32752]: pgmap v4813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:15 smithi167 ceph-mon[32751]: pgmap v4813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:15 smithi090 ceph-mon[29180]: pgmap v4813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:16.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8021 sec 2024-04-26T20:42:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:16 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7617.092686 secs 2024-04-26T20:42:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:16 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7617.092686 secs 2024-04-26T20:42:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:16 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7617.092686 secs 2024-04-26T20:42:17.611 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8022 sec 2024-04-26T20:42:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7617.800835 secs 2024-04-26T20:42:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:17 smithi119 ceph-mon[32752]: pgmap v4814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7617.800835 secs 2024-04-26T20:42:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:17 smithi167 ceph-mon[32751]: pgmap v4814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7617.800835 secs 2024-04-26T20:42:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:17 smithi090 ceph-mon[29180]: pgmap v4814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:18.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8023 sec 2024-04-26T20:42:19.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8024 sec 2024-04-26T20:42:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:19 smithi119 ceph-mon[32752]: pgmap v4815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:19 smithi167 ceph-mon[32751]: pgmap v4815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:19 smithi090 ceph-mon[29180]: pgmap v4815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:20.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8025 sec 2024-04-26T20:42:21.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8026 sec 2024-04-26T20:42:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:21 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7622.092771 secs 2024-04-26T20:42:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:21 smithi119 ceph-mon[32752]: pgmap v4816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:21 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7622.092771 secs 2024-04-26T20:42:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:21 smithi167 ceph-mon[32751]: pgmap v4816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:21 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7622.092771 secs 2024-04-26T20:42:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:21 smithi090 ceph-mon[29180]: pgmap v4816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:22.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8027 sec 2024-04-26T20:42:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7622.800970 secs 2024-04-26T20:42:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7622.800970 secs 2024-04-26T20:42:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7622.800970 secs 2024-04-26T20:42:23.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8028 sec 2024-04-26T20:42:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:23 smithi119 ceph-mon[32752]: pgmap v4817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:23 smithi167 ceph-mon[32751]: pgmap v4817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:23 smithi090 ceph-mon[29180]: pgmap v4817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:24.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8029 sec 2024-04-26T20:42:25.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8030 sec 2024-04-26T20:42:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:25 smithi119 ceph-mon[32752]: pgmap v4818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:42:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:42:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:42:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:42:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:25 smithi167 ceph-mon[32751]: pgmap v4818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:42:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:42:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:42:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:42:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:25 smithi090 ceph-mon[29180]: pgmap v4818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:42:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:42:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:42:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:42:26.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8031 sec 2024-04-26T20:42:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:26 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7627.092859 secs 2024-04-26T20:42:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:26 smithi119 ceph-mon[32752]: pgmap v4819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T20:42:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:26 smithi119 ceph-mon[32752]: pgmap v4820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:26 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7627.092859 secs 2024-04-26T20:42:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:26 smithi167 ceph-mon[32751]: pgmap v4819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T20:42:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:26 smithi167 ceph-mon[32751]: pgmap v4820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:26 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7627.092859 secs 2024-04-26T20:42:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:26 smithi090 ceph-mon[29180]: pgmap v4819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T20:42:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:26 smithi090 ceph-mon[29180]: pgmap v4820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:27.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8032 sec 2024-04-26T20:42:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7627.801084 secs 2024-04-26T20:42:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7627.801084 secs 2024-04-26T20:42:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7627.801084 secs 2024-04-26T20:42:28.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8033 sec 2024-04-26T20:42:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:28 smithi119 ceph-mon[32752]: pgmap v4821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:42:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:28 smithi167 ceph-mon[32751]: pgmap v4821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:42:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:28 smithi090 ceph-mon[29180]: pgmap v4821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:42:29.612 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8034 sec 2024-04-26T20:42:30.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8035 sec 2024-04-26T20:42:30.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:30 smithi090 ceph-mon[29180]: pgmap v4822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:30 smithi119 ceph-mon[32752]: pgmap v4822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:30 smithi167 ceph-mon[32751]: pgmap v4822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:31.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8036 sec 2024-04-26T20:42:31.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:31 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7632.092941 secs 2024-04-26T20:42:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:31 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7632.092941 secs 2024-04-26T20:42:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:31 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7632.092941 secs 2024-04-26T20:42:32.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8037 sec 2024-04-26T20:42:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7632.801224 secs 2024-04-26T20:42:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:32 smithi119 ceph-mon[32752]: pgmap v4823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7632.801224 secs 2024-04-26T20:42:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:32 smithi167 ceph-mon[32751]: pgmap v4823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7632.801224 secs 2024-04-26T20:42:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:32 smithi090 ceph-mon[29180]: pgmap v4823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:42:33.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8038 sec 2024-04-26T20:42:34.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8039 sec 2024-04-26T20:42:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:34 smithi119 ceph-mon[32752]: pgmap v4824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:42:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:34 smithi167 ceph-mon[32751]: pgmap v4824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:42:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:34 smithi090 ceph-mon[29180]: pgmap v4824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T20:42:35.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8040 sec 2024-04-26T20:42:36.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8041 sec 2024-04-26T20:42:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:36 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7637.093025 secs 2024-04-26T20:42:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:36 smithi119 ceph-mon[32752]: pgmap v4825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:42:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:36 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7637.093025 secs 2024-04-26T20:42:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:36 smithi167 ceph-mon[32751]: pgmap v4825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:42:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:36 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7637.093025 secs 2024-04-26T20:42:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:36 smithi090 ceph-mon[29180]: pgmap v4825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:42:37.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8042 sec 2024-04-26T20:42:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7637.801291 secs 2024-04-26T20:42:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7637.801291 secs 2024-04-26T20:42:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7637.801291 secs 2024-04-26T20:42:38.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8043 sec 2024-04-26T20:42:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:38 smithi119 ceph-mon[32752]: pgmap v4826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:38 smithi167 ceph-mon[32751]: pgmap v4826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:38 smithi090 ceph-mon[29180]: pgmap v4826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:39.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8044 sec 2024-04-26T20:42:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:39 smithi119 ceph-mon[32752]: pgmap v4827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:39 smithi167 ceph-mon[32751]: pgmap v4827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:39 smithi090 ceph-mon[29180]: pgmap v4827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:40.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8045 sec 2024-04-26T20:42:41.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7642.093090 secs 2024-04-26T20:42:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7642.093090 secs 2024-04-26T20:42:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7642.093090 secs 2024-04-26T20:42:41.613 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8046 sec 2024-04-26T20:42:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7642.801412 secs 2024-04-26T20:42:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:41 smithi090 ceph-mon[29180]: pgmap v4828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7642.801412 secs 2024-04-26T20:42:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:41 smithi119 ceph-mon[32752]: pgmap v4828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7642.801412 secs 2024-04-26T20:42:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:41 smithi167 ceph-mon[32751]: pgmap v4828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:42.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8047 sec 2024-04-26T20:42:43.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8048 sec 2024-04-26T20:42:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:44 smithi119 ceph-mon[32752]: pgmap v4829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:44 smithi167 ceph-mon[32751]: pgmap v4829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:44.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8049 sec 2024-04-26T20:42:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:44 smithi090 ceph-mon[29180]: pgmap v4829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:45.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8050 sec 2024-04-26T20:42:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:46 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7647.093176 secs 2024-04-26T20:42:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:46 smithi119 ceph-mon[32752]: pgmap v4830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:46 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7647.093176 secs 2024-04-26T20:42:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:46 smithi167 ceph-mon[32751]: pgmap v4830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:46.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8051 sec 2024-04-26T20:42:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:46 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7647.093176 secs 2024-04-26T20:42:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:46 smithi090 ceph-mon[29180]: pgmap v4830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7647.801505 secs 2024-04-26T20:42:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7647.801505 secs 2024-04-26T20:42:47.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8052 sec 2024-04-26T20:42:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7647.801505 secs 2024-04-26T20:42:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:48 smithi119 ceph-mon[32752]: pgmap v4831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:48 smithi167 ceph-mon[32751]: pgmap v4831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:48.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8053 sec 2024-04-26T20:42:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:48 smithi090 ceph-mon[29180]: pgmap v4831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:42:49.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8054 sec 2024-04-26T20:42:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:50 smithi119 ceph-mon[32752]: pgmap v4832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:50 smithi167 ceph-mon[32751]: pgmap v4832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:50.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8055 sec 2024-04-26T20:42:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:50 smithi090 ceph-mon[29180]: pgmap v4832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:51 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7652.093261 secs 2024-04-26T20:42:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:51 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7652.093261 secs 2024-04-26T20:42:51.614 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8056 sec 2024-04-26T20:42:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:51 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7652.093261 secs 2024-04-26T20:42:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7652.801656 secs 2024-04-26T20:42:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:52 smithi119 ceph-mon[32752]: pgmap v4833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7652.801656 secs 2024-04-26T20:42:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:52 smithi167 ceph-mon[32751]: pgmap v4833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:52.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8057 sec 2024-04-26T20:42:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7652.801656 secs 2024-04-26T20:42:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:52 smithi090 ceph-mon[29180]: pgmap v4833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:53.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8058 sec 2024-04-26T20:42:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:54 smithi119 ceph-mon[32752]: pgmap v4834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:54 smithi167 ceph-mon[32751]: pgmap v4834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:54.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8059 sec 2024-04-26T20:42:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:54 smithi090 ceph-mon[29180]: pgmap v4834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:55.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8060 sec 2024-04-26T20:42:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:56 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7657.093343 secs 2024-04-26T20:42:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:56 smithi119 ceph-mon[32752]: pgmap v4835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:56 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7657.093343 secs 2024-04-26T20:42:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:56 smithi167 ceph-mon[32751]: pgmap v4835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:56.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8061 sec 2024-04-26T20:42:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:56 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7657.093343 secs 2024-04-26T20:42:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:56 smithi090 ceph-mon[29180]: pgmap v4835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:42:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7657.801746 secs 2024-04-26T20:42:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7657.801746 secs 2024-04-26T20:42:57.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8062 sec 2024-04-26T20:42:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7657.801746 secs 2024-04-26T20:42:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:42:58 smithi119 ceph-mon[32752]: pgmap v4836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:42:58 smithi167 ceph-mon[32751]: pgmap v4836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:58.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8063 sec 2024-04-26T20:42:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:42:58 smithi090 ceph-mon[29180]: pgmap v4836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:42:59.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8064 sec 2024-04-26T20:43:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:00 smithi119 ceph-mon[32752]: pgmap v4837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:00 smithi167 ceph-mon[32751]: pgmap v4837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:00.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8065 sec 2024-04-26T20:43:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:00 smithi090 ceph-mon[29180]: pgmap v4837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:01 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7662.093428 secs 2024-04-26T20:43:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:01 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7662.093428 secs 2024-04-26T20:43:01.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8066 sec 2024-04-26T20:43:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:01 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7662.093428 secs 2024-04-26T20:43:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7662.801826 secs 2024-04-26T20:43:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:02 smithi090 ceph-mon[29180]: pgmap v4838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7662.801826 secs 2024-04-26T20:43:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:02 smithi119 ceph-mon[32752]: pgmap v4838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7662.801826 secs 2024-04-26T20:43:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:02 smithi167 ceph-mon[32751]: pgmap v4838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:02.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8067 sec 2024-04-26T20:43:03.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8068 sec 2024-04-26T20:43:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:04 smithi119 ceph-mon[32752]: pgmap v4839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:04 smithi167 ceph-mon[32751]: pgmap v4839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:04.615 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8069 sec 2024-04-26T20:43:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:04 smithi090 ceph-mon[29180]: pgmap v4839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:05.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8070 sec 2024-04-26T20:43:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:06 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7667.093510 secs 2024-04-26T20:43:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:06 smithi119 ceph-mon[32752]: pgmap v4840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:06 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7667.093510 secs 2024-04-26T20:43:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:06 smithi167 ceph-mon[32751]: pgmap v4840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:06.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8071 sec 2024-04-26T20:43:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:06 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7667.093510 secs 2024-04-26T20:43:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:06 smithi090 ceph-mon[29180]: pgmap v4840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7667.801944 secs 2024-04-26T20:43:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7667.801944 secs 2024-04-26T20:43:07.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8072 sec 2024-04-26T20:43:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7667.801944 secs 2024-04-26T20:43:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:08 smithi119 ceph-mon[32752]: pgmap v4841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:08 smithi167 ceph-mon[32751]: pgmap v4841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:08.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8073 sec 2024-04-26T20:43:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:08 smithi090 ceph-mon[29180]: pgmap v4841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:09.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8074 sec 2024-04-26T20:43:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:10 smithi119 ceph-mon[32752]: pgmap v4842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:10 smithi167 ceph-mon[32751]: pgmap v4842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:10.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8075 sec 2024-04-26T20:43:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:10 smithi090 ceph-mon[29180]: pgmap v4842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:11 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7672.093595 secs 2024-04-26T20:43:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:11 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7672.093595 secs 2024-04-26T20:43:11.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8076 sec 2024-04-26T20:43:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:11 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7672.093595 secs 2024-04-26T20:43:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7672.802062 secs 2024-04-26T20:43:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:12 smithi119 ceph-mon[32752]: pgmap v4843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7672.802062 secs 2024-04-26T20:43:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:12 smithi167 ceph-mon[32751]: pgmap v4843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:12.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8077 sec 2024-04-26T20:43:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7672.802062 secs 2024-04-26T20:43:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:12 smithi090 ceph-mon[29180]: pgmap v4843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:13.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8078 sec 2024-04-26T20:43:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:14 smithi119 ceph-mon[32752]: pgmap v4844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:14 smithi167 ceph-mon[32751]: pgmap v4844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:14.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8079 sec 2024-04-26T20:43:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:14 smithi090 ceph-mon[29180]: pgmap v4844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:15.616 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8080 sec 2024-04-26T20:43:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:16 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 7677.093700 secs 2024-04-26T20:43:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:16 smithi167 ceph-mon[32751]: pgmap v4845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:16.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8081 sec 2024-04-26T20:43:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:16 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 7677.093700 secs 2024-04-26T20:43:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:16 smithi090 ceph-mon[29180]: pgmap v4845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:16 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 7677.093700 secs 2024-04-26T20:43:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:16 smithi119 ceph-mon[32752]: pgmap v4845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7677.802182 secs 2024-04-26T20:43:17.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8082 sec 2024-04-26T20:43:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7677.802182 secs 2024-04-26T20:43:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7677.802182 secs 2024-04-26T20:43:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:18 smithi167 ceph-mon[32751]: pgmap v4846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:18.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8083 sec 2024-04-26T20:43:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:18 smithi090 ceph-mon[29180]: pgmap v4846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:18.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:18 smithi119 ceph-mon[32752]: pgmap v4846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:19.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8084 sec 2024-04-26T20:43:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:20 smithi167 ceph-mon[32751]: pgmap v4847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:20.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8085 sec 2024-04-26T20:43:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:20 smithi090 ceph-mon[29180]: pgmap v4847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:20 smithi119 ceph-mon[32752]: pgmap v4847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:21.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8086 sec 2024-04-26T20:43:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:21 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 7682.093792 secs 2024-04-26T20:43:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:21 smithi090 ceph-mon[29180]: slow request 181.153087 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:43:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:21 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 7682.093792 secs 2024-04-26T20:43:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:21 smithi119 ceph-mon[32752]: slow request 181.153087 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:43:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:21 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 7682.093792 secs 2024-04-26T20:43:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:21 smithi167 ceph-mon[32751]: slow request 181.153087 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:43:22.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8087 sec 2024-04-26T20:43:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7682.802300 secs 2024-04-26T20:43:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:22 smithi090 ceph-mon[29180]: pgmap v4848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7682.802300 secs 2024-04-26T20:43:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:22 smithi119 ceph-mon[32752]: pgmap v4848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7682.802300 secs 2024-04-26T20:43:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:22 smithi167 ceph-mon[32751]: pgmap v4848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:23.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8088 sec 2024-04-26T20:43:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:43:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:23 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:43:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:43:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:23 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:43:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:43:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:23 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:43:24.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8089 sec 2024-04-26T20:43:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:24 smithi090 ceph-mon[29180]: pgmap v4849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:24 smithi119 ceph-mon[32752]: pgmap v4849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:24 smithi167 ceph-mon[32751]: pgmap v4849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:25.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8090 sec 2024-04-26T20:43:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:43:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:43:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:43:26.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8091 sec 2024-04-26T20:43:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7687.093848 secs 2024-04-26T20:43:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:26 smithi090 ceph-mon[29180]: pgmap v4850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:43:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:43:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:43:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7687.093848 secs 2024-04-26T20:43:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:26 smithi119 ceph-mon[32752]: pgmap v4850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:43:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:43:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:43:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7687.093848 secs 2024-04-26T20:43:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:26 smithi167 ceph-mon[32751]: pgmap v4850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:43:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:43:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:43:27.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8092 sec 2024-04-26T20:43:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7687.802410 secs 2024-04-26T20:43:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:27 smithi090 ceph-mon[29180]: pgmap v4851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:43:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:27 smithi090 ceph-mon[29180]: pgmap v4852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7687.802410 secs 2024-04-26T20:43:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:27 smithi119 ceph-mon[32752]: pgmap v4851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:43:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:27 smithi119 ceph-mon[32752]: pgmap v4852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7687.802410 secs 2024-04-26T20:43:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:27 smithi167 ceph-mon[32751]: pgmap v4851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:43:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:27 smithi167 ceph-mon[32751]: pgmap v4852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:28.617 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8093 sec 2024-04-26T20:43:29.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8094 sec 2024-04-26T20:43:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:29 smithi090 ceph-mon[29180]: pgmap v4853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:43:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:29 smithi119 ceph-mon[32752]: pgmap v4853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:43:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:29 smithi167 ceph-mon[32751]: pgmap v4853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:43:30.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8095 sec 2024-04-26T20:43:31.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8096 sec 2024-04-26T20:43:31.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7692.093937 secs 2024-04-26T20:43:31.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:31 smithi167 ceph-mon[32751]: pgmap v4854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7692.093937 secs 2024-04-26T20:43:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:31 smithi090 ceph-mon[29180]: pgmap v4854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7692.093937 secs 2024-04-26T20:43:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:31 smithi119 ceph-mon[32752]: pgmap v4854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:32.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8097 sec 2024-04-26T20:43:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7692.802528 secs 2024-04-26T20:43:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7692.802528 secs 2024-04-26T20:43:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7692.802528 secs 2024-04-26T20:43:33.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8098 sec 2024-04-26T20:43:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:33 smithi090 ceph-mon[29180]: pgmap v4855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:33 smithi119 ceph-mon[32752]: pgmap v4855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:33 smithi167 ceph-mon[32751]: pgmap v4855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:43:34.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8099 sec 2024-04-26T20:43:35.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8100 sec 2024-04-26T20:43:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:35 smithi090 ceph-mon[29180]: pgmap v4856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T20:43:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:35 smithi119 ceph-mon[32752]: pgmap v4856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T20:43:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:35 smithi167 ceph-mon[32751]: pgmap v4856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T20:43:36.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8101 sec 2024-04-26T20:43:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7697.094026 secs 2024-04-26T20:43:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7697.094026 secs 2024-04-26T20:43:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7697.094026 secs 2024-04-26T20:43:37.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8102 sec 2024-04-26T20:43:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7697.802646 secs 2024-04-26T20:43:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:37 smithi119 ceph-mon[32752]: pgmap v4857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T20:43:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7697.802646 secs 2024-04-26T20:43:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:37 smithi167 ceph-mon[32751]: pgmap v4857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T20:43:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7697.802646 secs 2024-04-26T20:43:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:37 smithi090 ceph-mon[29180]: pgmap v4857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T20:43:38.618 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8103 sec 2024-04-26T20:43:39.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8104 sec 2024-04-26T20:43:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:39 smithi119 ceph-mon[32752]: pgmap v4858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:43:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:39 smithi167 ceph-mon[32751]: pgmap v4858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:43:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:39 smithi090 ceph-mon[29180]: pgmap v4858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:43:40.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8105 sec 2024-04-26T20:43:41.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8106 sec 2024-04-26T20:43:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7702.094112 secs 2024-04-26T20:43:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:41 smithi119 ceph-mon[32752]: pgmap v4859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:43:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7702.094112 secs 2024-04-26T20:43:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:41 smithi167 ceph-mon[32751]: pgmap v4859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:43:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7702.094112 secs 2024-04-26T20:43:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:41 smithi090 ceph-mon[29180]: pgmap v4859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:43:42.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8107 sec 2024-04-26T20:43:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7702.802720 secs 2024-04-26T20:43:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7702.802720 secs 2024-04-26T20:43:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7702.802720 secs 2024-04-26T20:43:43.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8108 sec 2024-04-26T20:43:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:43 smithi119 ceph-mon[32752]: pgmap v4860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:43:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:43 smithi167 ceph-mon[32751]: pgmap v4860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:43:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:43 smithi090 ceph-mon[29180]: pgmap v4860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:43:44.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8109 sec 2024-04-26T20:43:45.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8110 sec 2024-04-26T20:43:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:45 smithi119 ceph-mon[32752]: pgmap v4861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:43:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:45 smithi167 ceph-mon[32751]: pgmap v4861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:43:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:45 smithi090 ceph-mon[29180]: pgmap v4861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:43:46.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8111 sec 2024-04-26T20:43:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7707.094173 secs 2024-04-26T20:43:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7707.094173 secs 2024-04-26T20:43:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7707.094173 secs 2024-04-26T20:43:47.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8112 sec 2024-04-26T20:43:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7707.802801 secs 2024-04-26T20:43:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:47 smithi119 ceph-mon[32752]: pgmap v4862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7707.802801 secs 2024-04-26T20:43:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:47 smithi167 ceph-mon[32751]: pgmap v4862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7707.802801 secs 2024-04-26T20:43:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:47 smithi090 ceph-mon[29180]: pgmap v4862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:48.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8113 sec 2024-04-26T20:43:49.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8114 sec 2024-04-26T20:43:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:49 smithi119 ceph-mon[32752]: pgmap v4863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:49 smithi167 ceph-mon[32751]: pgmap v4863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:49 smithi090 ceph-mon[29180]: pgmap v4863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:50.619 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8115 sec 2024-04-26T20:43:51.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8116 sec 2024-04-26T20:43:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7712.094275 secs 2024-04-26T20:43:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:51 smithi119 ceph-mon[32752]: pgmap v4864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7712.094275 secs 2024-04-26T20:43:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:51 smithi167 ceph-mon[32751]: pgmap v4864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7712.094275 secs 2024-04-26T20:43:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:51 smithi090 ceph-mon[29180]: pgmap v4864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:52.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8117 sec 2024-04-26T20:43:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7712.802892 secs 2024-04-26T20:43:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7712.802892 secs 2024-04-26T20:43:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7712.802892 secs 2024-04-26T20:43:53.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8118 sec 2024-04-26T20:43:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:53 smithi119 ceph-mon[32752]: pgmap v4865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:53 smithi167 ceph-mon[32751]: pgmap v4865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:53 smithi090 ceph-mon[29180]: pgmap v4865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:54.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8119 sec 2024-04-26T20:43:55.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8120 sec 2024-04-26T20:43:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:55 smithi119 ceph-mon[32752]: pgmap v4866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:55 smithi167 ceph-mon[32751]: pgmap v4866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:55 smithi090 ceph-mon[29180]: pgmap v4866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:43:56.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8121 sec 2024-04-26T20:43:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7717.094378 secs 2024-04-26T20:43:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7717.094378 secs 2024-04-26T20:43:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7717.094378 secs 2024-04-26T20:43:57.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8122 sec 2024-04-26T20:43:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7717.803012 secs 2024-04-26T20:43:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:57 smithi119 ceph-mon[32752]: pgmap v4867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7717.803012 secs 2024-04-26T20:43:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:57 smithi167 ceph-mon[32751]: pgmap v4867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7717.803012 secs 2024-04-26T20:43:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:57 smithi090 ceph-mon[29180]: pgmap v4867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:58.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8123 sec 2024-04-26T20:43:59.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8124 sec 2024-04-26T20:43:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:43:59 smithi119 ceph-mon[32752]: pgmap v4868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:43:59 smithi167 ceph-mon[32751]: pgmap v4868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:43:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:43:59 smithi090 ceph-mon[29180]: pgmap v4868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:00.620 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8125 sec 2024-04-26T20:44:01.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8126 sec 2024-04-26T20:44:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7722.094442 secs 2024-04-26T20:44:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:01 smithi119 ceph-mon[32752]: pgmap v4869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7722.094442 secs 2024-04-26T20:44:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:01 smithi167 ceph-mon[32751]: pgmap v4869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7722.094442 secs 2024-04-26T20:44:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:01 smithi090 ceph-mon[29180]: pgmap v4869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:02.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8127 sec 2024-04-26T20:44:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7722.803136 secs 2024-04-26T20:44:02.807 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7722.803136 secs 2024-04-26T20:44:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7722.803136 secs 2024-04-26T20:44:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:44:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:44:02.804+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:44:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:44:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:44:02.804+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:44:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:44:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:44:02.804+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:44:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:44:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:44:02.804+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:44:03.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:44:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:44:02.804+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:44:03.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8128 sec 2024-04-26T20:44:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:03 smithi119 ceph-mon[32752]: pgmap v4870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:03 smithi167 ceph-mon[32751]: pgmap v4870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:03 smithi090 ceph-mon[29180]: pgmap v4870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:04.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8129 sec 2024-04-26T20:44:05.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8130 sec 2024-04-26T20:44:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:05 smithi119 ceph-mon[32752]: pgmap v4871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:05 smithi167 ceph-mon[32751]: pgmap v4871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:05 smithi090 ceph-mon[29180]: pgmap v4871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:06.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8131 sec 2024-04-26T20:44:06.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7727.094519 secs 2024-04-26T20:44:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7727.094519 secs 2024-04-26T20:44:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7727.094519 secs 2024-04-26T20:44:07.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8132 sec 2024-04-26T20:44:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7727.803257 secs 2024-04-26T20:44:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:07 smithi119 ceph-mon[32752]: pgmap v4872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:44:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7727.803257 secs 2024-04-26T20:44:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:07 smithi167 ceph-mon[32751]: pgmap v4872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:44:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7727.803257 secs 2024-04-26T20:44:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:07 smithi090 ceph-mon[29180]: pgmap v4872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:44:08.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8133 sec 2024-04-26T20:44:09.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8134 sec 2024-04-26T20:44:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:09 smithi119 ceph-mon[32752]: pgmap v4873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:44:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:09 smithi167 ceph-mon[32751]: pgmap v4873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:44:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:09 smithi090 ceph-mon[29180]: pgmap v4873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:44:10.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8135 sec 2024-04-26T20:44:11.621 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8136 sec 2024-04-26T20:44:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7732.094578 secs 2024-04-26T20:44:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:11 smithi119 ceph-mon[32752]: pgmap v4874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7732.094578 secs 2024-04-26T20:44:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:11 smithi167 ceph-mon[32751]: pgmap v4874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7732.094578 secs 2024-04-26T20:44:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:11 smithi090 ceph-mon[29180]: pgmap v4874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:12.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8137 sec 2024-04-26T20:44:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7732.803411 secs 2024-04-26T20:44:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7732.803411 secs 2024-04-26T20:44:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7732.803411 secs 2024-04-26T20:44:13.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8138 sec 2024-04-26T20:44:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:13 smithi119 ceph-mon[32752]: pgmap v4875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:13 smithi167 ceph-mon[32751]: pgmap v4875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:13 smithi090 ceph-mon[29180]: pgmap v4875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:14.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8139 sec 2024-04-26T20:44:15.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8140 sec 2024-04-26T20:44:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:15 smithi119 ceph-mon[32752]: pgmap v4876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:15 smithi167 ceph-mon[32751]: pgmap v4876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:15 smithi090 ceph-mon[29180]: pgmap v4876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:44:16.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8141 sec 2024-04-26T20:44:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:16 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 7737.094663 secs 2024-04-26T20:44:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:16 smithi119 ceph-mon[32752]: slow request 5760.364799 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:44:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:16 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 7737.094663 secs 2024-04-26T20:44:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:16 smithi167 ceph-mon[32751]: slow request 5760.364799 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:44:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:16 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 7737.094663 secs 2024-04-26T20:44:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:16 smithi090 ceph-mon[29180]: slow request 5760.364799 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:44:17.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8142 sec 2024-04-26T20:44:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7737.803502 secs 2024-04-26T20:44:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:17 smithi119 ceph-mon[32752]: pgmap v4877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:44:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7737.803502 secs 2024-04-26T20:44:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:17 smithi167 ceph-mon[32751]: pgmap v4877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:44:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7737.803502 secs 2024-04-26T20:44:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:17 smithi090 ceph-mon[29180]: pgmap v4877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:44:18.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8143 sec 2024-04-26T20:44:19.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8144 sec 2024-04-26T20:44:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:19 smithi119 ceph-mon[32752]: pgmap v4878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:44:19.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:19 smithi090 ceph-mon[29180]: pgmap v4878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:44:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:19 smithi167 ceph-mon[32751]: pgmap v4878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:44:20.622 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8145 sec 2024-04-26T20:44:21.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8146 sec 2024-04-26T20:44:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7742.094748 secs 2024-04-26T20:44:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:21 smithi119 ceph-mon[32752]: pgmap v4879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:44:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7742.094748 secs 2024-04-26T20:44:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:21 smithi167 ceph-mon[32751]: pgmap v4879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:44:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7742.094748 secs 2024-04-26T20:44:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:21 smithi090 ceph-mon[29180]: pgmap v4879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:44:22.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8147 sec 2024-04-26T20:44:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7742.803619 secs 2024-04-26T20:44:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7742.803619 secs 2024-04-26T20:44:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7742.803619 secs 2024-04-26T20:44:23.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8148 sec 2024-04-26T20:44:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:23 smithi119 ceph-mon[32752]: pgmap v4880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:23 smithi167 ceph-mon[32751]: pgmap v4880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:23 smithi090 ceph-mon[29180]: pgmap v4880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:24.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8149 sec 2024-04-26T20:44:25.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8150 sec 2024-04-26T20:44:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:25 smithi119 ceph-mon[32752]: pgmap v4881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:25 smithi167 ceph-mon[32751]: pgmap v4881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:25 smithi090 ceph-mon[29180]: pgmap v4881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:26.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8151 sec 2024-04-26T20:44:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7747.094848 secs 2024-04-26T20:44:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:44:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7747.094848 secs 2024-04-26T20:44:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:44:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7747.094848 secs 2024-04-26T20:44:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:44:27.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8152 sec 2024-04-26T20:44:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7747.803762 secs 2024-04-26T20:44:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:27 smithi119 ceph-mon[32752]: pgmap v4882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:44:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:44:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:27 smithi119 ceph-mon[32752]: pgmap v4883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:44:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:27 smithi119 ceph-mon[32752]: pgmap v4884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7747.803762 secs 2024-04-26T20:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:27 smithi167 ceph-mon[32751]: pgmap v4882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:27 smithi167 ceph-mon[32751]: pgmap v4883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:27 smithi167 ceph-mon[32751]: pgmap v4884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:44:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7747.803762 secs 2024-04-26T20:44:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:27 smithi090 ceph-mon[29180]: pgmap v4882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:44:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:44:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:27 smithi090 ceph-mon[29180]: pgmap v4883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:44:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:27 smithi090 ceph-mon[29180]: pgmap v4884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:44:28.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8153 sec 2024-04-26T20:44:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:44:29.498+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:44:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:44:29.499+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:44:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:44:29.499+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 63 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:44:29.500 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:44:29.499+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 63 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:44:29.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8154 sec 2024-04-26T20:44:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:29 smithi119 ceph-mon[32752]: pgmap v4885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:29 smithi167 ceph-mon[32751]: pgmap v4885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:29 smithi090 ceph-mon[29180]: pgmap v4885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:30.623 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8155 sec 2024-04-26T20:44:31.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8156 sec 2024-04-26T20:44:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7752.094937 secs 2024-04-26T20:44:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7752.803838 secs 2024-04-26T20:44:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:31 smithi119 ceph-mon[32752]: pgmap v4886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7752.094937 secs 2024-04-26T20:44:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7752.803838 secs 2024-04-26T20:44:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:31 smithi167 ceph-mon[32751]: pgmap v4886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7752.094937 secs 2024-04-26T20:44:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7752.803838 secs 2024-04-26T20:44:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:31 smithi090 ceph-mon[29180]: pgmap v4886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:32.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8157 sec 2024-04-26T20:44:33.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8158 sec 2024-04-26T20:44:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:33 smithi119 ceph-mon[32752]: pgmap v4887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:44:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:33 smithi167 ceph-mon[32751]: pgmap v4887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:44:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:33 smithi090 ceph-mon[29180]: pgmap v4887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:44:34.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8159 sec 2024-04-26T20:44:35.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8160 sec 2024-04-26T20:44:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:35 smithi119 ceph-mon[32752]: pgmap v4888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:35 smithi167 ceph-mon[32751]: pgmap v4888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:35 smithi090 ceph-mon[29180]: pgmap v4888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:44:36.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8161 sec 2024-04-26T20:44:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7757.095020 secs 2024-04-26T20:44:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7757.803960 secs 2024-04-26T20:44:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7757.095020 secs 2024-04-26T20:44:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7757.803960 secs 2024-04-26T20:44:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7757.095020 secs 2024-04-26T20:44:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7757.803960 secs 2024-04-26T20:44:37.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8162 sec 2024-04-26T20:44:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:37 smithi119 ceph-mon[32752]: pgmap v4889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:44:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:37 smithi167 ceph-mon[32751]: pgmap v4889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:44:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:37 smithi090 ceph-mon[29180]: pgmap v4889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:44:38.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8163 sec 2024-04-26T20:44:39.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8164 sec 2024-04-26T20:44:40.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8165 sec 2024-04-26T20:44:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:40 smithi119 ceph-mon[32752]: pgmap v4890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:40 smithi167 ceph-mon[32751]: pgmap v4890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:40 smithi090 ceph-mon[29180]: pgmap v4890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:41.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8166 sec 2024-04-26T20:44:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7762.095106 secs 2024-04-26T20:44:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7762.095106 secs 2024-04-26T20:44:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7762.095106 secs 2024-04-26T20:44:42.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8167 sec 2024-04-26T20:44:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7762.804094 secs 2024-04-26T20:44:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:42 smithi119 ceph-mon[32752]: pgmap v4891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7762.804094 secs 2024-04-26T20:44:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:42 smithi167 ceph-mon[32751]: pgmap v4891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7762.804094 secs 2024-04-26T20:44:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:42 smithi090 ceph-mon[29180]: pgmap v4891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:43.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8168 sec 2024-04-26T20:44:44.624 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8169 sec 2024-04-26T20:44:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:44 smithi119 ceph-mon[32752]: pgmap v4892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:44 smithi167 ceph-mon[32751]: pgmap v4892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:44 smithi090 ceph-mon[29180]: pgmap v4892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:45.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8170 sec 2024-04-26T20:44:46.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8171 sec 2024-04-26T20:44:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:46 smithi119 ceph-mon[32752]: pgmap v4893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7767.095184 secs 2024-04-26T20:44:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:46 smithi167 ceph-mon[32751]: pgmap v4893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7767.095184 secs 2024-04-26T20:44:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:46 smithi090 ceph-mon[29180]: pgmap v4893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7767.095184 secs 2024-04-26T20:44:47.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8172 sec 2024-04-26T20:44:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7767.804236 secs 2024-04-26T20:44:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7767.804236 secs 2024-04-26T20:44:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7767.804236 secs 2024-04-26T20:44:48.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8173 sec 2024-04-26T20:44:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:48 smithi119 ceph-mon[32752]: pgmap v4894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:48 smithi167 ceph-mon[32751]: pgmap v4894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:48 smithi090 ceph-mon[29180]: pgmap v4894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:49.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8174 sec 2024-04-26T20:44:50.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8175 sec 2024-04-26T20:44:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:50 smithi119 ceph-mon[32752]: pgmap v4895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:50 smithi167 ceph-mon[32751]: pgmap v4895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:50 smithi090 ceph-mon[29180]: pgmap v4895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:51.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8176 sec 2024-04-26T20:44:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7772.095269 secs 2024-04-26T20:44:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7772.095269 secs 2024-04-26T20:44:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7772.095269 secs 2024-04-26T20:44:52.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8177 sec 2024-04-26T20:44:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7772.804343 secs 2024-04-26T20:44:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:52 smithi119 ceph-mon[32752]: pgmap v4896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7772.804343 secs 2024-04-26T20:44:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:52 smithi167 ceph-mon[32751]: pgmap v4896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7772.804343 secs 2024-04-26T20:44:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:52 smithi090 ceph-mon[29180]: pgmap v4896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:53.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8178 sec 2024-04-26T20:44:54.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8179 sec 2024-04-26T20:44:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:54 smithi119 ceph-mon[32752]: pgmap v4897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:54 smithi167 ceph-mon[32751]: pgmap v4897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:54 smithi090 ceph-mon[29180]: pgmap v4897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:55.625 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8180 sec 2024-04-26T20:44:56.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8181 sec 2024-04-26T20:44:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:56 smithi119 ceph-mon[32752]: pgmap v4898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7777.095328 secs 2024-04-26T20:44:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:56 smithi167 ceph-mon[32751]: pgmap v4898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7777.095328 secs 2024-04-26T20:44:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:56 smithi090 ceph-mon[29180]: pgmap v4898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7777.095328 secs 2024-04-26T20:44:57.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8182 sec 2024-04-26T20:44:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7777.804474 secs 2024-04-26T20:44:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7777.804474 secs 2024-04-26T20:44:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7777.804474 secs 2024-04-26T20:44:58.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8183 sec 2024-04-26T20:44:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:58 smithi119 ceph-mon[32752]: pgmap v4899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:58 smithi167 ceph-mon[32751]: pgmap v4899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:58 smithi090 ceph-mon[29180]: pgmap v4899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:44:59.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8184 sec 2024-04-26T20:44:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:44:59 smithi119 ceph-mon[32752]: pgmap v4900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:44:59 smithi167 ceph-mon[32751]: pgmap v4900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:44:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:44:59 smithi090 ceph-mon[29180]: pgmap v4900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:00.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8185 sec 2024-04-26T20:45:01.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8186 sec 2024-04-26T20:45:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7782.095414 secs 2024-04-26T20:45:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7782.804539 secs 2024-04-26T20:45:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:01 smithi119 ceph-mon[32752]: pgmap v4901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7782.095414 secs 2024-04-26T20:45:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7782.804539 secs 2024-04-26T20:45:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:01 smithi167 ceph-mon[32751]: pgmap v4901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7782.095414 secs 2024-04-26T20:45:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7782.804539 secs 2024-04-26T20:45:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:01 smithi090 ceph-mon[29180]: pgmap v4901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:02.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8187 sec 2024-04-26T20:45:03.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8188 sec 2024-04-26T20:45:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:03 smithi119 ceph-mon[32752]: pgmap v4902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:03 smithi167 ceph-mon[32751]: pgmap v4902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:03 smithi090 ceph-mon[29180]: pgmap v4902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:04.626 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8189 sec 2024-04-26T20:45:05.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8190 sec 2024-04-26T20:45:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:05 smithi119 ceph-mon[32752]: pgmap v4903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:05 smithi167 ceph-mon[32751]: pgmap v4903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:05 smithi090 ceph-mon[29180]: pgmap v4903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:06.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8191 sec 2024-04-26T20:45:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7787.095517 secs 2024-04-26T20:45:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7787.804619 secs 2024-04-26T20:45:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7787.095517 secs 2024-04-26T20:45:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7787.804619 secs 2024-04-26T20:45:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7787.095517 secs 2024-04-26T20:45:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7787.804619 secs 2024-04-26T20:45:07.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8192 sec 2024-04-26T20:45:08.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:07 smithi119 ceph-mon[32752]: pgmap v4904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:07 smithi167 ceph-mon[32751]: pgmap v4904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:07 smithi090 ceph-mon[29180]: pgmap v4904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:08.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8193 sec 2024-04-26T20:45:09.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8194 sec 2024-04-26T20:45:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:09 smithi119 ceph-mon[32752]: pgmap v4905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:09 smithi167 ceph-mon[32751]: pgmap v4905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:09 smithi090 ceph-mon[29180]: pgmap v4905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:10.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8195 sec 2024-04-26T20:45:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:10 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7792.095605 secs 2024-04-26T20:45:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:10 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7792.095605 secs 2024-04-26T20:45:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:10 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7792.095605 secs 2024-04-26T20:45:11.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8196 sec 2024-04-26T20:45:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7792.804732 secs 2024-04-26T20:45:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:11 smithi119 ceph-mon[32752]: pgmap v4906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7792.804732 secs 2024-04-26T20:45:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:11 smithi167 ceph-mon[32751]: pgmap v4906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7792.804732 secs 2024-04-26T20:45:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:11 smithi090 ceph-mon[29180]: pgmap v4906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:12.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8197 sec 2024-04-26T20:45:13.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8198 sec 2024-04-26T20:45:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:13 smithi167 ceph-mon[32751]: pgmap v4907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:13 smithi090 ceph-mon[29180]: pgmap v4907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:13 smithi119 ceph-mon[32752]: pgmap v4907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:14.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8199 sec 2024-04-26T20:45:15.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8200 sec 2024-04-26T20:45:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:15 smithi167 ceph-mon[32751]: pgmap v4908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:15 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7797.095691 secs 2024-04-26T20:45:16.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:15 smithi119 ceph-mon[32752]: pgmap v4908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:16.107 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:15 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7797.095691 secs 2024-04-26T20:45:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:15 smithi090 ceph-mon[29180]: pgmap v4908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:15 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7797.095691 secs 2024-04-26T20:45:16.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8201 sec 2024-04-26T20:45:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7797.804857 secs 2024-04-26T20:45:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7797.804857 secs 2024-04-26T20:45:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7797.804857 secs 2024-04-26T20:45:17.627 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8202 sec 2024-04-26T20:45:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:17 smithi167 ceph-mon[32751]: pgmap v4909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:17 smithi090 ceph-mon[29180]: pgmap v4909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:17 smithi119 ceph-mon[32752]: pgmap v4909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:18.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8203 sec 2024-04-26T20:45:19.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8204 sec 2024-04-26T20:45:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:19 smithi090 ceph-mon[29180]: pgmap v4910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:19 smithi119 ceph-mon[32752]: pgmap v4910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:19 smithi167 ceph-mon[32751]: pgmap v4910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:20.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8205 sec 2024-04-26T20:45:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:20 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7802.095774 secs 2024-04-26T20:45:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:20 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7802.095774 secs 2024-04-26T20:45:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:20 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7802.095774 secs 2024-04-26T20:45:21.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8206 sec 2024-04-26T20:45:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7802.804933 secs 2024-04-26T20:45:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:21 smithi090 ceph-mon[29180]: pgmap v4911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7802.804933 secs 2024-04-26T20:45:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:21 smithi119 ceph-mon[32752]: pgmap v4911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7802.804933 secs 2024-04-26T20:45:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:21 smithi167 ceph-mon[32751]: pgmap v4911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:22.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8207 sec 2024-04-26T20:45:23.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8208 sec 2024-04-26T20:45:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:23 smithi090 ceph-mon[29180]: pgmap v4912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:23 smithi119 ceph-mon[32752]: pgmap v4912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:23 smithi167 ceph-mon[32751]: pgmap v4912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:24.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8209 sec 2024-04-26T20:45:25.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8210 sec 2024-04-26T20:45:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:25 smithi090 ceph-mon[29180]: pgmap v4913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:25 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7807.095859 secs 2024-04-26T20:45:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:25 smithi119 ceph-mon[32752]: pgmap v4913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:25 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7807.095859 secs 2024-04-26T20:45:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:25 smithi167 ceph-mon[32751]: pgmap v4913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:25 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7807.095859 secs 2024-04-26T20:45:26.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8211 sec 2024-04-26T20:45:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7807.805073 secs 2024-04-26T20:45:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:45:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7807.805073 secs 2024-04-26T20:45:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:45:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7807.805073 secs 2024-04-26T20:45:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:45:27.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8212 sec 2024-04-26T20:45:28.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:28 smithi119 ceph-mon[32752]: pgmap v4914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:28.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.392 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:28 smithi090 ceph-mon[29180]: pgmap v4914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:28.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8213 sec 2024-04-26T20:45:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:28 smithi167 ceph-mon[32751]: pgmap v4914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:29.628 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8214 sec 2024-04-26T20:45:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:30 smithi119 ceph-mon[32752]: pgmap v4915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:30 smithi167 ceph-mon[32751]: pgmap v4915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:30 smithi090 ceph-mon[29180]: pgmap v4915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:30.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8215 sec 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7812.095948 secs 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:45:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7812.095948 secs 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:45:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7812.095948 secs 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:45:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:45:31.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8216 sec 2024-04-26T20:45:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7812.805159 secs 2024-04-26T20:45:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:45:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:45:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: pgmap v4916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:45:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:45:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: pgmap v4917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:45:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:32 smithi090 ceph-mon[29180]: pgmap v4918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:32.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7812.805159 secs 2024-04-26T20:45:32.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:45:32.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:45:32.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: pgmap v4916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:32.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:45:32.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:45:32.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: pgmap v4917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:45:32.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:32 smithi167 ceph-mon[32751]: pgmap v4918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7812.805159 secs 2024-04-26T20:45:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:45:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:45:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: pgmap v4916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:45:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:45:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: pgmap v4917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:45:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:32 smithi119 ceph-mon[32752]: pgmap v4918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:32.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8217 sec 2024-04-26T20:45:33.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8218 sec 2024-04-26T20:45:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:34 smithi090 ceph-mon[29180]: pgmap v4919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T20:45:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:34 smithi119 ceph-mon[32752]: pgmap v4919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T20:45:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:34 smithi167 ceph-mon[32751]: pgmap v4919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T20:45:34.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8219 sec 2024-04-26T20:45:35.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8220 sec 2024-04-26T20:45:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:36 smithi090 ceph-mon[29180]: pgmap v4920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7817.096007 secs 2024-04-26T20:45:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:36 smithi119 ceph-mon[32752]: pgmap v4920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7817.096007 secs 2024-04-26T20:45:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:36 smithi167 ceph-mon[32751]: pgmap v4920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7817.096007 secs 2024-04-26T20:45:36.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8221 sec 2024-04-26T20:45:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7817.805275 secs 2024-04-26T20:45:37.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7817.805275 secs 2024-04-26T20:45:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7817.805275 secs 2024-04-26T20:45:37.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8222 sec 2024-04-26T20:45:38.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:38 smithi090 ceph-mon[29180]: pgmap v4921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T20:45:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:38 smithi119 ceph-mon[32752]: pgmap v4921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T20:45:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:38 smithi167 ceph-mon[32751]: pgmap v4921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-26T20:45:38.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8223 sec 2024-04-26T20:45:39.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8224 sec 2024-04-26T20:45:40.629 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8225 sec 2024-04-26T20:45:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:40 smithi119 ceph-mon[32752]: pgmap v4922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:40 smithi167 ceph-mon[32751]: pgmap v4922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:40 smithi090 ceph-mon[29180]: pgmap v4922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:41.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8226 sec 2024-04-26T20:45:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7822.096112 secs 2024-04-26T20:45:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7822.096112 secs 2024-04-26T20:45:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7822.096112 secs 2024-04-26T20:45:42.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8227 sec 2024-04-26T20:45:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7822.805404 secs 2024-04-26T20:45:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:42 smithi119 ceph-mon[32752]: pgmap v4923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:45:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7822.805404 secs 2024-04-26T20:45:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:42 smithi167 ceph-mon[32751]: pgmap v4923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:45:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7822.805404 secs 2024-04-26T20:45:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:42 smithi090 ceph-mon[29180]: pgmap v4923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:45:43.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8228 sec 2024-04-26T20:45:44.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8229 sec 2024-04-26T20:45:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:44 smithi119 ceph-mon[32752]: pgmap v4924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:44 smithi167 ceph-mon[32751]: pgmap v4924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:44 smithi090 ceph-mon[29180]: pgmap v4924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:45.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8230 sec 2024-04-26T20:45:46.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8231 sec 2024-04-26T20:45:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:46 smithi119 ceph-mon[32752]: pgmap v4925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7827.096198 secs 2024-04-26T20:45:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:46 smithi167 ceph-mon[32751]: pgmap v4925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7827.096198 secs 2024-04-26T20:45:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:46 smithi090 ceph-mon[29180]: pgmap v4925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7827.096198 secs 2024-04-26T20:45:47.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8232 sec 2024-04-26T20:45:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7827.805522 secs 2024-04-26T20:45:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7827.805522 secs 2024-04-26T20:45:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7827.805522 secs 2024-04-26T20:45:48.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8233 sec 2024-04-26T20:45:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:48 smithi119 ceph-mon[32752]: pgmap v4926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:48 smithi167 ceph-mon[32751]: pgmap v4926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:48 smithi090 ceph-mon[29180]: pgmap v4926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:49.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8234 sec 2024-04-26T20:45:50.630 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8235 sec 2024-04-26T20:45:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:50 smithi119 ceph-mon[32752]: pgmap v4927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:50 smithi167 ceph-mon[32751]: pgmap v4927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:50 smithi090 ceph-mon[29180]: pgmap v4927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:51.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8236 sec 2024-04-26T20:45:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7832.096284 secs 2024-04-26T20:45:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7832.096284 secs 2024-04-26T20:45:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7832.096284 secs 2024-04-26T20:45:52.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8237 sec 2024-04-26T20:45:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7832.805649 secs 2024-04-26T20:45:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:52 smithi119 ceph-mon[32752]: pgmap v4928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7832.805649 secs 2024-04-26T20:45:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:52 smithi167 ceph-mon[32751]: pgmap v4928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7832.805649 secs 2024-04-26T20:45:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:52 smithi090 ceph-mon[29180]: pgmap v4928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:53.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8238 sec 2024-04-26T20:45:54.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8239 sec 2024-04-26T20:45:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:54 smithi119 ceph-mon[32752]: pgmap v4929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:54 smithi167 ceph-mon[32751]: pgmap v4929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:54 smithi090 ceph-mon[29180]: pgmap v4929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:55.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8240 sec 2024-04-26T20:45:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:55 smithi167 ceph-mon[32751]: pgmap v4930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:55 smithi090 ceph-mon[29180]: pgmap v4930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:55 smithi119 ceph-mon[32752]: pgmap v4930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:45:56.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8241 sec 2024-04-26T20:45:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7837.096368 secs 2024-04-26T20:45:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7837.805754 secs 2024-04-26T20:45:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7837.096368 secs 2024-04-26T20:45:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7837.805754 secs 2024-04-26T20:45:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7837.096368 secs 2024-04-26T20:45:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7837.805754 secs 2024-04-26T20:45:57.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8242 sec 2024-04-26T20:45:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:57 smithi090 ceph-mon[29180]: pgmap v4931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:57.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:57 smithi119 ceph-mon[32752]: pgmap v4931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:57 smithi167 ceph-mon[32751]: pgmap v4931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:45:58.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8243 sec 2024-04-26T20:45:59.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8244 sec 2024-04-26T20:46:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:45:59 smithi119 ceph-mon[32752]: pgmap v4932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:45:59 smithi167 ceph-mon[32751]: pgmap v4932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:45:59 smithi090 ceph-mon[29180]: pgmap v4932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:00.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8245 sec 2024-04-26T20:46:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7842.096452 secs 2024-04-26T20:46:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7842.096452 secs 2024-04-26T20:46:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7842.096452 secs 2024-04-26T20:46:01.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8246 sec 2024-04-26T20:46:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7842.805885 secs 2024-04-26T20:46:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:01 smithi167 ceph-mon[32751]: pgmap v4933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7842.805885 secs 2024-04-26T20:46:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:01 smithi090 ceph-mon[29180]: pgmap v4933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7842.805885 secs 2024-04-26T20:46:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:01 smithi119 ceph-mon[32752]: pgmap v4933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:02.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8247 sec 2024-04-26T20:46:03.631 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8248 sec 2024-04-26T20:46:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:03 smithi167 ceph-mon[32751]: pgmap v4934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:03 smithi090 ceph-mon[29180]: pgmap v4934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:03 smithi119 ceph-mon[32752]: pgmap v4934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:04.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8249 sec 2024-04-26T20:46:05.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8250 sec 2024-04-26T20:46:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:05 smithi167 ceph-mon[32751]: pgmap v4935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:05 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7847.096508 secs 2024-04-26T20:46:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:05 smithi090 ceph-mon[29180]: pgmap v4935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:05 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7847.096508 secs 2024-04-26T20:46:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:05 smithi119 ceph-mon[32752]: pgmap v4935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:05 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7847.096508 secs 2024-04-26T20:46:06.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8251 sec 2024-04-26T20:46:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7847.806011 secs 2024-04-26T20:46:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7847.806011 secs 2024-04-26T20:46:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7847.806011 secs 2024-04-26T20:46:07.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8252 sec 2024-04-26T20:46:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:07 smithi090 ceph-mon[29180]: pgmap v4936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:08.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:07 smithi119 ceph-mon[32752]: pgmap v4936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:07 smithi167 ceph-mon[32751]: pgmap v4936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:08.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8253 sec 2024-04-26T20:46:09.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8254 sec 2024-04-26T20:46:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:09 smithi090 ceph-mon[29180]: pgmap v4937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:09 smithi119 ceph-mon[32752]: pgmap v4937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:09 smithi167 ceph-mon[32751]: pgmap v4937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:10.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8255 sec 2024-04-26T20:46:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:10 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7852.096592 secs 2024-04-26T20:46:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:10 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7852.096592 secs 2024-04-26T20:46:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:10 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7852.096592 secs 2024-04-26T20:46:11.632 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8256 sec 2024-04-26T20:46:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7852.806073 secs 2024-04-26T20:46:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:11 smithi090 ceph-mon[29180]: pgmap v4938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7852.806073 secs 2024-04-26T20:46:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:11 smithi119 ceph-mon[32752]: pgmap v4938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7852.806073 secs 2024-04-26T20:46:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:11 smithi167 ceph-mon[32751]: pgmap v4938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:12.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8257 sec 2024-04-26T20:46:13.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8258 sec 2024-04-26T20:46:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:13 smithi090 ceph-mon[29180]: pgmap v4939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:13 smithi119 ceph-mon[32752]: pgmap v4939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:13 smithi167 ceph-mon[32751]: pgmap v4939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:14.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8259 sec 2024-04-26T20:46:15.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8260 sec 2024-04-26T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:15 smithi119 ceph-mon[32752]: pgmap v4940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:16.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:15 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7857.096680 secs 2024-04-26T20:46:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:15 smithi090 ceph-mon[29180]: pgmap v4940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:15 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7857.096680 secs 2024-04-26T20:46:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:15 smithi167 ceph-mon[32751]: pgmap v4940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:15 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7857.096680 secs 2024-04-26T20:46:16.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8261 sec 2024-04-26T20:46:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7857.806208 secs 2024-04-26T20:46:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7857.806208 secs 2024-04-26T20:46:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7857.806208 secs 2024-04-26T20:46:17.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8262 sec 2024-04-26T20:46:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:17 smithi090 ceph-mon[29180]: pgmap v4941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:17 smithi119 ceph-mon[32752]: pgmap v4941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:17 smithi167 ceph-mon[32751]: pgmap v4941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:18.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8263 sec 2024-04-26T20:46:19.633 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8264 sec 2024-04-26T20:46:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:19 smithi090 ceph-mon[29180]: pgmap v4942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:19 smithi119 ceph-mon[32752]: pgmap v4942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:19 smithi167 ceph-mon[32751]: pgmap v4942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:20.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8265 sec 2024-04-26T20:46:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:20 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 7862.096763 secs 2024-04-26T20:46:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:20 smithi090 ceph-mon[29180]: slow request 361.156058 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:46:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:20 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 7862.096763 secs 2024-04-26T20:46:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:20 smithi119 ceph-mon[32752]: slow request 361.156058 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:46:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:20 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 7862.096763 secs 2024-04-26T20:46:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:20 smithi167 ceph-mon[32751]: slow request 361.156058 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:46:21.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8266 sec 2024-04-26T20:46:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7862.806352 secs 2024-04-26T20:46:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:21 smithi090 ceph-mon[29180]: pgmap v4943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7862.806352 secs 2024-04-26T20:46:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:21 smithi119 ceph-mon[32752]: pgmap v4943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7862.806352 secs 2024-04-26T20:46:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:21 smithi167 ceph-mon[32751]: pgmap v4943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:22.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8267 sec 2024-04-26T20:46:23.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8268 sec 2024-04-26T20:46:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:23 smithi090 ceph-mon[29180]: pgmap v4944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:23 smithi119 ceph-mon[32752]: pgmap v4944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:23 smithi167 ceph-mon[32751]: pgmap v4944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:24.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8269 sec 2024-04-26T20:46:25.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8270 sec 2024-04-26T20:46:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:25 smithi090 ceph-mon[29180]: pgmap v4945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:25 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7867.096844 secs 2024-04-26T20:46:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:25 smithi119 ceph-mon[32752]: pgmap v4945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:25 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7867.096844 secs 2024-04-26T20:46:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:25 smithi167 ceph-mon[32751]: pgmap v4945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:25 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7867.096844 secs 2024-04-26T20:46:26.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8271 sec 2024-04-26T20:46:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7867.806453 secs 2024-04-26T20:46:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7867.806453 secs 2024-04-26T20:46:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7867.806453 secs 2024-04-26T20:46:27.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8272 sec 2024-04-26T20:46:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:27 smithi090 ceph-mon[29180]: pgmap v4946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:27 smithi119 ceph-mon[32752]: pgmap v4946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:27 smithi167 ceph-mon[32751]: pgmap v4946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:28.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8273 sec 2024-04-26T20:46:29.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8274 sec 2024-04-26T20:46:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:29 smithi090 ceph-mon[29180]: pgmap v4947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:29 smithi119 ceph-mon[32752]: pgmap v4947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:29 smithi167 ceph-mon[32751]: pgmap v4947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:30.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8275 sec 2024-04-26T20:46:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7872.096934 secs 2024-04-26T20:46:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:46:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7872.096934 secs 2024-04-26T20:46:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:46:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7872.096934 secs 2024-04-26T20:46:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:46:31.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8276 sec 2024-04-26T20:46:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7872.806579 secs 2024-04-26T20:46:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:32 smithi090 ceph-mon[29180]: pgmap v4948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:46:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:46:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.634 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8277 sec 2024-04-26T20:46:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7872.806579 secs 2024-04-26T20:46:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:32 smithi119 ceph-mon[32752]: pgmap v4948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:46:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:46:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7872.806579 secs 2024-04-26T20:46:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:32 smithi167 ceph-mon[32751]: pgmap v4948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:46:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:46:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:46:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:33 smithi090 ceph-mon[29180]: pgmap v4949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-26T20:46:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:33 smithi090 ceph-mon[29180]: pgmap v4950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:33.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8278 sec 2024-04-26T20:46:33.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:33 smithi167 ceph-mon[32751]: pgmap v4949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-26T20:46:33.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:33 smithi167 ceph-mon[32751]: pgmap v4950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:33 smithi119 ceph-mon[32752]: pgmap v4949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-26T20:46:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:33 smithi119 ceph-mon[32752]: pgmap v4950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:34.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8279 sec 2024-04-26T20:46:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:35 smithi090 ceph-mon[29180]: pgmap v4951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:35.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8280 sec 2024-04-26T20:46:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:35 smithi119 ceph-mon[32752]: pgmap v4951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:35 smithi167 ceph-mon[32751]: pgmap v4951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:36.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8281 sec 2024-04-26T20:46:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7877.097022 secs 2024-04-26T20:46:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7877.097022 secs 2024-04-26T20:46:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7877.097022 secs 2024-04-26T20:46:37.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8282 sec 2024-04-26T20:46:37.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7877.806646 secs 2024-04-26T20:46:37.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:37 smithi090 ceph-mon[29180]: pgmap v4952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7877.806646 secs 2024-04-26T20:46:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:37 smithi119 ceph-mon[32752]: pgmap v4952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7877.806646 secs 2024-04-26T20:46:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:37 smithi167 ceph-mon[32751]: pgmap v4952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:38.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8283 sec 2024-04-26T20:46:39.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8284 sec 2024-04-26T20:46:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:39 smithi119 ceph-mon[32752]: pgmap v4953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 554 B/s rd, 1 op/s 2024-04-26T20:46:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:39 smithi167 ceph-mon[32751]: pgmap v4953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 554 B/s rd, 1 op/s 2024-04-26T20:46:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:39 smithi090 ceph-mon[29180]: pgmap v4953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 554 B/s rd, 1 op/s 2024-04-26T20:46:40.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8285 sec 2024-04-26T20:46:41.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8286 sec 2024-04-26T20:46:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7882.097106 secs 2024-04-26T20:46:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:41 smithi119 ceph-mon[32752]: pgmap v4954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7882.097106 secs 2024-04-26T20:46:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:41 smithi167 ceph-mon[32751]: pgmap v4954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7882.097106 secs 2024-04-26T20:46:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:41 smithi090 ceph-mon[29180]: pgmap v4954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:46:42.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8287 sec 2024-04-26T20:46:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7882.806724 secs 2024-04-26T20:46:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7882.806724 secs 2024-04-26T20:46:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7882.806724 secs 2024-04-26T20:46:43.635 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8288 sec 2024-04-26T20:46:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:43 smithi119 ceph-mon[32752]: pgmap v4955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:46:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:43 smithi167 ceph-mon[32751]: pgmap v4955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:46:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:43 smithi090 ceph-mon[29180]: pgmap v4955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:46:44.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8289 sec 2024-04-26T20:46:45.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8290 sec 2024-04-26T20:46:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:45 smithi119 ceph-mon[32752]: pgmap v4956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:45 smithi167 ceph-mon[32751]: pgmap v4956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:45 smithi090 ceph-mon[29180]: pgmap v4956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:46.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8291 sec 2024-04-26T20:46:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7887.097207 secs 2024-04-26T20:46:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7887.097207 secs 2024-04-26T20:46:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7887.097207 secs 2024-04-26T20:46:47.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8292 sec 2024-04-26T20:46:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7887.806848 secs 2024-04-26T20:46:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:47 smithi119 ceph-mon[32752]: pgmap v4957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7887.806848 secs 2024-04-26T20:46:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:47 smithi167 ceph-mon[32751]: pgmap v4957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7887.806848 secs 2024-04-26T20:46:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:47 smithi090 ceph-mon[29180]: pgmap v4957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:48.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8293 sec 2024-04-26T20:46:49.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8294 sec 2024-04-26T20:46:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:49 smithi119 ceph-mon[32752]: pgmap v4958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:49 smithi167 ceph-mon[32751]: pgmap v4958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:49 smithi090 ceph-mon[29180]: pgmap v4958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:50.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8295 sec 2024-04-26T20:46:51.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8296 sec 2024-04-26T20:46:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7892.097266 secs 2024-04-26T20:46:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:51 smithi119 ceph-mon[32752]: pgmap v4959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7892.097266 secs 2024-04-26T20:46:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:51 smithi167 ceph-mon[32751]: pgmap v4959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7892.097266 secs 2024-04-26T20:46:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:51 smithi090 ceph-mon[29180]: pgmap v4959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:52.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8297 sec 2024-04-26T20:46:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7892.806951 secs 2024-04-26T20:46:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7892.806951 secs 2024-04-26T20:46:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7892.806951 secs 2024-04-26T20:46:53.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8298 sec 2024-04-26T20:46:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:53 smithi119 ceph-mon[32752]: pgmap v4960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:53 smithi167 ceph-mon[32751]: pgmap v4960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:53 smithi090 ceph-mon[29180]: pgmap v4960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:54.636 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8299 sec 2024-04-26T20:46:55.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8300 sec 2024-04-26T20:46:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:55 smithi119 ceph-mon[32752]: pgmap v4961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:55 smithi167 ceph-mon[32751]: pgmap v4961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:55 smithi090 ceph-mon[29180]: pgmap v4961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:46:56.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8301 sec 2024-04-26T20:46:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7897.097350 secs 2024-04-26T20:46:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7897.097350 secs 2024-04-26T20:46:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7897.097350 secs 2024-04-26T20:46:57.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8302 sec 2024-04-26T20:46:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7897.807032 secs 2024-04-26T20:46:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:57 smithi119 ceph-mon[32752]: pgmap v4962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7897.807032 secs 2024-04-26T20:46:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:57 smithi167 ceph-mon[32751]: pgmap v4962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7897.807032 secs 2024-04-26T20:46:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:57 smithi090 ceph-mon[29180]: pgmap v4962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:58.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8303 sec 2024-04-26T20:46:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:46:58 smithi119 ceph-mon[32752]: pgmap v4963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:46:58 smithi167 ceph-mon[32751]: pgmap v4963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:46:58 smithi090 ceph-mon[29180]: pgmap v4963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:46:59.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8304 sec 2024-04-26T20:47:00.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8305 sec 2024-04-26T20:47:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7902.097429 secs 2024-04-26T20:47:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:01 smithi167 ceph-mon[32751]: pgmap v4964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7902.097429 secs 2024-04-26T20:47:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:01 smithi090 ceph-mon[29180]: pgmap v4964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7902.097429 secs 2024-04-26T20:47:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:01 smithi119 ceph-mon[32752]: pgmap v4964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:01.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8306 sec 2024-04-26T20:47:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7902.807091 secs 2024-04-26T20:47:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7902.807091 secs 2024-04-26T20:47:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7902.807091 secs 2024-04-26T20:47:02.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8307 sec 2024-04-26T20:47:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:03 smithi167 ceph-mon[32751]: pgmap v4965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:03 smithi090 ceph-mon[29180]: pgmap v4965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:03 smithi119 ceph-mon[32752]: pgmap v4965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:03.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8308 sec 2024-04-26T20:47:04.637 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8309 sec 2024-04-26T20:47:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:05 smithi090 ceph-mon[29180]: pgmap v4966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:05 smithi119 ceph-mon[32752]: pgmap v4966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:05 smithi167 ceph-mon[32751]: pgmap v4966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:05.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8310 sec 2024-04-26T20:47:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7907.097512 secs 2024-04-26T20:47:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7907.097512 secs 2024-04-26T20:47:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7907.097512 secs 2024-04-26T20:47:06.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8311 sec 2024-04-26T20:47:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7907.807222 secs 2024-04-26T20:47:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:07 smithi090 ceph-mon[29180]: pgmap v4967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7907.807222 secs 2024-04-26T20:47:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:07 smithi119 ceph-mon[32752]: pgmap v4967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7907.807222 secs 2024-04-26T20:47:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:07 smithi167 ceph-mon[32751]: pgmap v4967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:07.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8312 sec 2024-04-26T20:47:08.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8313 sec 2024-04-26T20:47:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:09 smithi090 ceph-mon[29180]: pgmap v4968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:09 smithi119 ceph-mon[32752]: pgmap v4968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:09 smithi167 ceph-mon[32751]: pgmap v4968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:09.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8314 sec 2024-04-26T20:47:10.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8315 sec 2024-04-26T20:47:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7912.097593 secs 2024-04-26T20:47:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:11 smithi090 ceph-mon[29180]: pgmap v4969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7912.097593 secs 2024-04-26T20:47:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:11 smithi119 ceph-mon[32752]: pgmap v4969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7912.097593 secs 2024-04-26T20:47:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:11 smithi167 ceph-mon[32751]: pgmap v4969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:11.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8316 sec 2024-04-26T20:47:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7912.807325 secs 2024-04-26T20:47:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7912.807325 secs 2024-04-26T20:47:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7912.807325 secs 2024-04-26T20:47:12.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8317 sec 2024-04-26T20:47:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:13 smithi090 ceph-mon[29180]: pgmap v4970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:13 smithi119 ceph-mon[32752]: pgmap v4970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:13 smithi167 ceph-mon[32751]: pgmap v4970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:13.638 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8318 sec 2024-04-26T20:47:14.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8319 sec 2024-04-26T20:47:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:15 smithi090 ceph-mon[29180]: pgmap v4971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:15 smithi119 ceph-mon[32752]: pgmap v4971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:15 smithi167 ceph-mon[32751]: pgmap v4971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:15.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8320 sec 2024-04-26T20:47:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7917.097680 secs 2024-04-26T20:47:16.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7917.097680 secs 2024-04-26T20:47:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7917.097680 secs 2024-04-26T20:47:16.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8321 sec 2024-04-26T20:47:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7917.807459 secs 2024-04-26T20:47:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:17 smithi090 ceph-mon[29180]: pgmap v4972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7917.807459 secs 2024-04-26T20:47:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:17 smithi119 ceph-mon[32752]: pgmap v4972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7917.807459 secs 2024-04-26T20:47:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:17 smithi167 ceph-mon[32751]: pgmap v4972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:17.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8322 sec 2024-04-26T20:47:18.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8323 sec 2024-04-26T20:47:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:19 smithi090 ceph-mon[29180]: pgmap v4973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:19 smithi119 ceph-mon[32752]: pgmap v4973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:19 smithi167 ceph-mon[32751]: pgmap v4973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:19.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8324 sec 2024-04-26T20:47:20.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8325 sec 2024-04-26T20:47:21.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8326 sec 2024-04-26T20:47:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7922.097766 secs 2024-04-26T20:47:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:21 smithi119 ceph-mon[32752]: pgmap v4974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7922.097766 secs 2024-04-26T20:47:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:21 smithi167 ceph-mon[32751]: pgmap v4974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7922.097766 secs 2024-04-26T20:47:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:21 smithi090 ceph-mon[29180]: pgmap v4974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:22.639 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8327 sec 2024-04-26T20:47:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7922.807594 secs 2024-04-26T20:47:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7922.807594 secs 2024-04-26T20:47:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7922.807594 secs 2024-04-26T20:47:23.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8328 sec 2024-04-26T20:47:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:23 smithi119 ceph-mon[32752]: pgmap v4975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:23 smithi167 ceph-mon[32751]: pgmap v4975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:23 smithi090 ceph-mon[29180]: pgmap v4975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:24.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8329 sec 2024-04-26T20:47:25.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8330 sec 2024-04-26T20:47:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:25 smithi119 ceph-mon[32752]: pgmap v4976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:25 smithi167 ceph-mon[32751]: pgmap v4976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:25 smithi090 ceph-mon[29180]: pgmap v4976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:26.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8331 sec 2024-04-26T20:47:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7927.097848 secs 2024-04-26T20:47:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7927.097848 secs 2024-04-26T20:47:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7927.097848 secs 2024-04-26T20:47:27.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8332 sec 2024-04-26T20:47:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7927.807675 secs 2024-04-26T20:47:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:27 smithi119 ceph-mon[32752]: pgmap v4977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7927.807675 secs 2024-04-26T20:47:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:27 smithi167 ceph-mon[32751]: pgmap v4977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7927.807675 secs 2024-04-26T20:47:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:27 smithi090 ceph-mon[29180]: pgmap v4977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:28.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8333 sec 2024-04-26T20:47:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:28 smithi090 ceph-mon[29180]: pgmap v4978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:28 smithi119 ceph-mon[32752]: pgmap v4978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:28 smithi167 ceph-mon[32751]: pgmap v4978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.523+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:47:29.525 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.523+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:47:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.523+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 64 ==== auth_reply(proto 2 0 (0) Success) ==== 612+0+0 (secure 0 0 0) 0x7f2cac006460 con 0x56302893f0e0 2024-04-26T20:47:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.523+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:47:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.523+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 64 ==== auth_reply(proto 2 0 (0) Success) ==== 783+0+0 (secure 0 0 0) 0x7f2ca80062f0 con 0x7f2cd4104500 2024-04-26T20:47:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.523+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:47:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.523+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 65 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:47:29.526 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:47:29.524+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 65 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:47:29.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8334 sec 2024-04-26T20:47:30.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8335 sec 2024-04-26T20:47:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7932.097937 secs 2024-04-26T20:47:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:31 smithi167 ceph-mon[32751]: pgmap v4979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7932.097937 secs 2024-04-26T20:47:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:31 smithi090 ceph-mon[29180]: pgmap v4979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7932.097937 secs 2024-04-26T20:47:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:31 smithi119 ceph-mon[32752]: pgmap v4979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:31.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8336 sec 2024-04-26T20:47:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7932.807751 secs 2024-04-26T20:47:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7932.807751 secs 2024-04-26T20:47:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7932.807751 secs 2024-04-26T20:47:32.640 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8337 sec 2024-04-26T20:47:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:33 smithi090 ceph-mon[29180]: pgmap v4980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:47:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:47:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:47:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:47:33.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:33 smithi167 ceph-mon[32751]: pgmap v4980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:33.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:47:33.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:47:33.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:47:33.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:47:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:33 smithi119 ceph-mon[32752]: pgmap v4980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:47:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:47:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:47:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:47:33.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8338 sec 2024-04-26T20:47:34.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:34 smithi090 ceph-mon[29180]: pgmap v4981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:47:34.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:34 smithi090 ceph-mon[29180]: pgmap v4982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:47:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:34 smithi119 ceph-mon[32752]: pgmap v4981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:47:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:34 smithi119 ceph-mon[32752]: pgmap v4982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:47:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:34 smithi167 ceph-mon[32751]: pgmap v4981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:47:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:34 smithi167 ceph-mon[32751]: pgmap v4982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:47:34.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8339 sec 2024-04-26T20:47:35.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8340 sec 2024-04-26T20:47:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:36 smithi090 ceph-mon[29180]: pgmap v4983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7937.098022 secs 2024-04-26T20:47:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:36 smithi119 ceph-mon[32752]: pgmap v4983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7937.098022 secs 2024-04-26T20:47:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:36 smithi167 ceph-mon[32751]: pgmap v4983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7937.098022 secs 2024-04-26T20:47:36.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8341 sec 2024-04-26T20:47:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7937.807875 secs 2024-04-26T20:47:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7937.807875 secs 2024-04-26T20:47:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7937.807875 secs 2024-04-26T20:47:37.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8342 sec 2024-04-26T20:47:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:38 smithi090 ceph-mon[29180]: pgmap v4984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:38 smithi119 ceph-mon[32752]: pgmap v4984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:38 smithi167 ceph-mon[32751]: pgmap v4984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:38.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8343 sec 2024-04-26T20:47:39.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8344 sec 2024-04-26T20:47:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:40 smithi090 ceph-mon[29180]: pgmap v4985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:40 smithi119 ceph-mon[32752]: pgmap v4985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:40 smithi167 ceph-mon[32751]: pgmap v4985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:40.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8345 sec 2024-04-26T20:47:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7942.098107 secs 2024-04-26T20:47:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7942.098107 secs 2024-04-26T20:47:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7942.098107 secs 2024-04-26T20:47:41.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8346 sec 2024-04-26T20:47:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7942.808002 secs 2024-04-26T20:47:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:42 smithi090 ceph-mon[29180]: pgmap v4986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7942.808002 secs 2024-04-26T20:47:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:42 smithi119 ceph-mon[32752]: pgmap v4986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7942.808002 secs 2024-04-26T20:47:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:42 smithi167 ceph-mon[32751]: pgmap v4986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:47:42.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8347 sec 2024-04-26T20:47:43.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8348 sec 2024-04-26T20:47:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:44 smithi090 ceph-mon[29180]: pgmap v4987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:47:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:44 smithi119 ceph-mon[32752]: pgmap v4987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:47:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:44 smithi167 ceph-mon[32751]: pgmap v4987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:47:44.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8349 sec 2024-04-26T20:47:45.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8350 sec 2024-04-26T20:47:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:46 smithi090 ceph-mon[29180]: pgmap v4988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7947.098194 secs 2024-04-26T20:47:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:46 smithi119 ceph-mon[32752]: pgmap v4988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7947.098194 secs 2024-04-26T20:47:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:46 smithi167 ceph-mon[32751]: pgmap v4988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7947.098194 secs 2024-04-26T20:47:46.641 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8351 sec 2024-04-26T20:47:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7947.808093 secs 2024-04-26T20:47:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7947.808093 secs 2024-04-26T20:47:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7947.808093 secs 2024-04-26T20:47:47.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8352 sec 2024-04-26T20:47:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:48 smithi090 ceph-mon[29180]: pgmap v4989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:48 smithi119 ceph-mon[32752]: pgmap v4989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:48 smithi167 ceph-mon[32751]: pgmap v4989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:48.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8353 sec 2024-04-26T20:47:49.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8354 sec 2024-04-26T20:47:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:50 smithi090 ceph-mon[29180]: pgmap v4990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:50 smithi119 ceph-mon[32752]: pgmap v4990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:50 smithi167 ceph-mon[32751]: pgmap v4990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:50.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8355 sec 2024-04-26T20:47:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7952.098250 secs 2024-04-26T20:47:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7952.098250 secs 2024-04-26T20:47:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7952.098250 secs 2024-04-26T20:47:51.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8356 sec 2024-04-26T20:47:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7952.808220 secs 2024-04-26T20:47:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:52 smithi119 ceph-mon[32752]: pgmap v4991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7952.808220 secs 2024-04-26T20:47:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:52 smithi167 ceph-mon[32751]: pgmap v4991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7952.808220 secs 2024-04-26T20:47:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:52 smithi090 ceph-mon[29180]: pgmap v4991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:52.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8357 sec 2024-04-26T20:47:53.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8358 sec 2024-04-26T20:47:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:54 smithi119 ceph-mon[32752]: pgmap v4992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:54 smithi167 ceph-mon[32751]: pgmap v4992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:54 smithi090 ceph-mon[29180]: pgmap v4992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:54.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8359 sec 2024-04-26T20:47:55.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8360 sec 2024-04-26T20:47:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:56 smithi119 ceph-mon[32752]: pgmap v4993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7957.098333 secs 2024-04-26T20:47:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:56 smithi167 ceph-mon[32751]: pgmap v4993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7957.098333 secs 2024-04-26T20:47:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:56 smithi090 ceph-mon[29180]: pgmap v4993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:47:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7957.098333 secs 2024-04-26T20:47:56.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8361 sec 2024-04-26T20:47:57.415 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7957.808296 secs 2024-04-26T20:47:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7957.808296 secs 2024-04-26T20:47:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7957.808296 secs 2024-04-26T20:47:57.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8362 sec 2024-04-26T20:47:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:47:58 smithi119 ceph-mon[32752]: pgmap v4994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:47:58 smithi167 ceph-mon[32751]: pgmap v4994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:47:58 smithi090 ceph-mon[29180]: pgmap v4994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:47:58.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8363 sec 2024-04-26T20:47:59.642 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8364 sec 2024-04-26T20:48:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:00 smithi119 ceph-mon[32752]: pgmap v4995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:00 smithi167 ceph-mon[32751]: pgmap v4995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:00 smithi090 ceph-mon[29180]: pgmap v4995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:00.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8365 sec 2024-04-26T20:48:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7962.098421 secs 2024-04-26T20:48:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7962.098421 secs 2024-04-26T20:48:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7962.098421 secs 2024-04-26T20:48:01.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8366 sec 2024-04-26T20:48:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7962.808422 secs 2024-04-26T20:48:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:02 smithi119 ceph-mon[32752]: pgmap v4996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7962.808422 secs 2024-04-26T20:48:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:02 smithi167 ceph-mon[32751]: pgmap v4996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7962.808422 secs 2024-04-26T20:48:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:02 smithi090 ceph-mon[29180]: pgmap v4996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:02.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8367 sec 2024-04-26T20:48:03.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8368 sec 2024-04-26T20:48:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:04 smithi119 ceph-mon[32752]: pgmap v4997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:04 smithi167 ceph-mon[32751]: pgmap v4997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:04 smithi090 ceph-mon[29180]: pgmap v4997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:04.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8369 sec 2024-04-26T20:48:05.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8370 sec 2024-04-26T20:48:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:06 smithi119 ceph-mon[32752]: pgmap v4998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7967.098505 secs 2024-04-26T20:48:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:06 smithi167 ceph-mon[32751]: pgmap v4998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7967.098505 secs 2024-04-26T20:48:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:06 smithi090 ceph-mon[29180]: pgmap v4998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7967.098505 secs 2024-04-26T20:48:06.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8371 sec 2024-04-26T20:48:07.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7967.808546 secs 2024-04-26T20:48:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7967.808546 secs 2024-04-26T20:48:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7967.808546 secs 2024-04-26T20:48:07.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8372 sec 2024-04-26T20:48:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:08 smithi119 ceph-mon[32752]: pgmap v4999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:08 smithi167 ceph-mon[32751]: pgmap v4999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:08 smithi090 ceph-mon[29180]: pgmap v4999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:08.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8373 sec 2024-04-26T20:48:09.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8374 sec 2024-04-26T20:48:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:10 smithi119 ceph-mon[32752]: pgmap v5000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:10 smithi167 ceph-mon[32751]: pgmap v5000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:10 smithi090 ceph-mon[29180]: pgmap v5000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:10.643 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8375 sec 2024-04-26T20:48:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7972.098589 secs 2024-04-26T20:48:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7972.098589 secs 2024-04-26T20:48:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7972.098589 secs 2024-04-26T20:48:11.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8376 sec 2024-04-26T20:48:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7972.808643 secs 2024-04-26T20:48:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:12 smithi119 ceph-mon[32752]: pgmap v5001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7972.808643 secs 2024-04-26T20:48:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:12 smithi167 ceph-mon[32751]: pgmap v5001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7972.808643 secs 2024-04-26T20:48:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:12 smithi090 ceph-mon[29180]: pgmap v5001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:12.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8377 sec 2024-04-26T20:48:13.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8378 sec 2024-04-26T20:48:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:14 smithi119 ceph-mon[32752]: pgmap v5002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:14 smithi167 ceph-mon[32751]: pgmap v5002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:14 smithi090 ceph-mon[29180]: pgmap v5002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:14.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8379 sec 2024-04-26T20:48:15.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8380 sec 2024-04-26T20:48:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:16 smithi119 ceph-mon[32752]: pgmap v5003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7977.098666 secs 2024-04-26T20:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:16 smithi167 ceph-mon[32751]: pgmap v5003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7977.098666 secs 2024-04-26T20:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:16 smithi090 ceph-mon[29180]: pgmap v5003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7977.098666 secs 2024-04-26T20:48:16.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8381 sec 2024-04-26T20:48:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7977.808795 secs 2024-04-26T20:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7977.808795 secs 2024-04-26T20:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7977.808795 secs 2024-04-26T20:48:17.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8382 sec 2024-04-26T20:48:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:18 smithi119 ceph-mon[32752]: pgmap v5004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:18 smithi167 ceph-mon[32751]: pgmap v5004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:18 smithi090 ceph-mon[29180]: pgmap v5004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:18.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8383 sec 2024-04-26T20:48:19.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8384 sec 2024-04-26T20:48:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:20 smithi119 ceph-mon[32752]: pgmap v5005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:20 smithi167 ceph-mon[32751]: pgmap v5005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:20 smithi090 ceph-mon[29180]: pgmap v5005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:20.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8385 sec 2024-04-26T20:48:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7982.098750 secs 2024-04-26T20:48:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7982.098750 secs 2024-04-26T20:48:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7982.098750 secs 2024-04-26T20:48:21.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8386 sec 2024-04-26T20:48:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7982.808912 secs 2024-04-26T20:48:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:22 smithi119 ceph-mon[32752]: pgmap v5006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7982.808912 secs 2024-04-26T20:48:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:22 smithi167 ceph-mon[32751]: pgmap v5006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7982.808912 secs 2024-04-26T20:48:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:22 smithi090 ceph-mon[29180]: pgmap v5006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:22.644 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8387 sec 2024-04-26T20:48:23.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8388 sec 2024-04-26T20:48:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:24 smithi119 ceph-mon[32752]: pgmap v5007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:24 smithi167 ceph-mon[32751]: pgmap v5007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:24 smithi090 ceph-mon[29180]: pgmap v5007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:24.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8389 sec 2024-04-26T20:48:25.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8390 sec 2024-04-26T20:48:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:26 smithi119 ceph-mon[32752]: pgmap v5008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7987.098845 secs 2024-04-26T20:48:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:26 smithi167 ceph-mon[32751]: pgmap v5008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7987.098845 secs 2024-04-26T20:48:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:26 smithi090 ceph-mon[29180]: pgmap v5008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7987.098845 secs 2024-04-26T20:48:26.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8391 sec 2024-04-26T20:48:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7987.809025 secs 2024-04-26T20:48:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7987.809025 secs 2024-04-26T20:48:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7987.809025 secs 2024-04-26T20:48:27.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8392 sec 2024-04-26T20:48:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:28 smithi119 ceph-mon[32752]: pgmap v5009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:28 smithi167 ceph-mon[32751]: pgmap v5009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:28 smithi090 ceph-mon[29180]: pgmap v5009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:28.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8393 sec 2024-04-26T20:48:29.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8394 sec 2024-04-26T20:48:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:30 smithi119 ceph-mon[32752]: pgmap v5010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:30 smithi167 ceph-mon[32751]: pgmap v5010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:30 smithi090 ceph-mon[29180]: pgmap v5010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:30.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8395 sec 2024-04-26T20:48:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7992.098924 secs 2024-04-26T20:48:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7992.098924 secs 2024-04-26T20:48:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7992.098924 secs 2024-04-26T20:48:31.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8396 sec 2024-04-26T20:48:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7992.809164 secs 2024-04-26T20:48:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:32 smithi119 ceph-mon[32752]: pgmap v5011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7992.809164 secs 2024-04-26T20:48:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:32 smithi167 ceph-mon[32751]: pgmap v5011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7992.809164 secs 2024-04-26T20:48:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:32 smithi090 ceph-mon[29180]: pgmap v5011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:32.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8397 sec 2024-04-26T20:48:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:48:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:48:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:48:33.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8398 sec 2024-04-26T20:48:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:34 smithi119 ceph-mon[32752]: pgmap v5012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:48:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:48:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:48:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:34 smithi167 ceph-mon[32751]: pgmap v5012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:48:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:48:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:48:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:34 smithi090 ceph-mon[29180]: pgmap v5012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:48:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:48:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:48:34.645 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8399 sec 2024-04-26T20:48:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:35 smithi119 ceph-mon[32752]: pgmap v5013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:48:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:35 smithi119 ceph-mon[32752]: pgmap v5014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:35 smithi167 ceph-mon[32751]: pgmap v5013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:48:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:35 smithi167 ceph-mon[32751]: pgmap v5014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:35 smithi090 ceph-mon[29180]: pgmap v5013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:48:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:35 smithi090 ceph-mon[29180]: pgmap v5014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:35.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8400 sec 2024-04-26T20:48:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 7997.099000 secs 2024-04-26T20:48:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 7997.099000 secs 2024-04-26T20:48:36.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 7997.099000 secs 2024-04-26T20:48:36.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8401 sec 2024-04-26T20:48:37.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:37 smithi090 ceph-mon[29180]: pgmap v5015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:37.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 7997.809304 secs 2024-04-26T20:48:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:37 smithi119 ceph-mon[32752]: pgmap v5015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 7997.809304 secs 2024-04-26T20:48:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:37 smithi167 ceph-mon[32751]: pgmap v5015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 7997.809304 secs 2024-04-26T20:48:37.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8402 sec 2024-04-26T20:48:38.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8403 sec 2024-04-26T20:48:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:39 smithi119 ceph-mon[32752]: pgmap v5016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:48:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:39 smithi167 ceph-mon[32751]: pgmap v5016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:48:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:39 smithi090 ceph-mon[29180]: pgmap v5016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:48:39.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8404 sec 2024-04-26T20:48:40.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8405 sec 2024-04-26T20:48:41.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8406 sec 2024-04-26T20:48:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:41 smithi119 ceph-mon[32752]: pgmap v5017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8002.099105 secs 2024-04-26T20:48:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:41 smithi167 ceph-mon[32751]: pgmap v5017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8002.099105 secs 2024-04-26T20:48:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:41 smithi090 ceph-mon[29180]: pgmap v5017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8002.099105 secs 2024-04-26T20:48:42.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8407 sec 2024-04-26T20:48:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8002.809451 secs 2024-04-26T20:48:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8002.809451 secs 2024-04-26T20:48:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8002.809451 secs 2024-04-26T20:48:43.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8408 sec 2024-04-26T20:48:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:43 smithi119 ceph-mon[32752]: pgmap v5018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:43 smithi167 ceph-mon[32751]: pgmap v5018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:43 smithi090 ceph-mon[29180]: pgmap v5018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:48:44.646 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8409 sec 2024-04-26T20:48:45.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8410 sec 2024-04-26T20:48:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:45 smithi119 ceph-mon[32752]: pgmap v5019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:48:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:45 smithi167 ceph-mon[32751]: pgmap v5019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:48:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:45 smithi090 ceph-mon[29180]: pgmap v5019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:48:46.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8411 sec 2024-04-26T20:48:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8007.099187 secs 2024-04-26T20:48:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8007.099187 secs 2024-04-26T20:48:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8007.099187 secs 2024-04-26T20:48:47.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8412 sec 2024-04-26T20:48:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:47 smithi119 ceph-mon[32752]: pgmap v5020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8007.809519 secs 2024-04-26T20:48:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:47 smithi167 ceph-mon[32751]: pgmap v5020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8007.809519 secs 2024-04-26T20:48:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:47 smithi090 ceph-mon[29180]: pgmap v5020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8007.809519 secs 2024-04-26T20:48:48.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8413 sec 2024-04-26T20:48:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:48 smithi119 ceph-mon[32752]: pgmap v5021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:48 smithi167 ceph-mon[32751]: pgmap v5021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:48 smithi090 ceph-mon[29180]: pgmap v5021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:49.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8414 sec 2024-04-26T20:48:50.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8415 sec 2024-04-26T20:48:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:50 smithi090 ceph-mon[29180]: pgmap v5022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:50 smithi119 ceph-mon[32752]: pgmap v5022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:50 smithi167 ceph-mon[32751]: pgmap v5022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:51.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8416 sec 2024-04-26T20:48:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8012.099272 secs 2024-04-26T20:48:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8012.809597 secs 2024-04-26T20:48:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8012.099272 secs 2024-04-26T20:48:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8012.809597 secs 2024-04-26T20:48:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8012.099272 secs 2024-04-26T20:48:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8012.809597 secs 2024-04-26T20:48:52.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8417 sec 2024-04-26T20:48:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:52 smithi090 ceph-mon[29180]: pgmap v5023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:52 smithi119 ceph-mon[32752]: pgmap v5023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:52 smithi167 ceph-mon[32751]: pgmap v5023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:53.647 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8418 sec 2024-04-26T20:48:54.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8419 sec 2024-04-26T20:48:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:54 smithi090 ceph-mon[29180]: pgmap v5024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:54 smithi119 ceph-mon[32752]: pgmap v5024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:54 smithi167 ceph-mon[32751]: pgmap v5024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:48:55.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8420 sec 2024-04-26T20:48:56.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8421 sec 2024-04-26T20:48:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8017.099359 secs 2024-04-26T20:48:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:56 smithi090 ceph-mon[29180]: pgmap v5025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8017.809658 secs 2024-04-26T20:48:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8017.099359 secs 2024-04-26T20:48:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:56 smithi119 ceph-mon[32752]: pgmap v5025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8017.809658 secs 2024-04-26T20:48:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8017.099359 secs 2024-04-26T20:48:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:56 smithi167 ceph-mon[32751]: pgmap v5025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8017.809658 secs 2024-04-26T20:48:57.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8422 sec 2024-04-26T20:48:58.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8423 sec 2024-04-26T20:48:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:48:58 smithi119 ceph-mon[32752]: pgmap v5026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:48:58 smithi167 ceph-mon[32751]: pgmap v5026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:48:58 smithi090 ceph-mon[29180]: pgmap v5026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:48:59.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8424 sec 2024-04-26T20:49:00.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8425 sec 2024-04-26T20:49:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:00 smithi119 ceph-mon[32752]: pgmap v5027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:00 smithi167 ceph-mon[32751]: pgmap v5027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:00 smithi090 ceph-mon[29180]: pgmap v5027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:01.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8426 sec 2024-04-26T20:49:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8022.099432 secs 2024-04-26T20:49:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8022.809780 secs 2024-04-26T20:49:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8022.099432 secs 2024-04-26T20:49:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8022.809780 secs 2024-04-26T20:49:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8022.099432 secs 2024-04-26T20:49:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8022.809780 secs 2024-04-26T20:49:02.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8427 sec 2024-04-26T20:49:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:02 smithi119 ceph-mon[32752]: pgmap v5028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:02 smithi167 ceph-mon[32751]: pgmap v5028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:02 smithi090 ceph-mon[29180]: pgmap v5028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:03.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8428 sec 2024-04-26T20:49:04.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8429 sec 2024-04-26T20:49:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:04 smithi119 ceph-mon[32752]: pgmap v5029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:04 smithi167 ceph-mon[32751]: pgmap v5029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:04 smithi090 ceph-mon[29180]: pgmap v5029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:05.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8430 sec 2024-04-26T20:49:06.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8431 sec 2024-04-26T20:49:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8027.099516 secs 2024-04-26T20:49:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8027.809895 secs 2024-04-26T20:49:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:06 smithi119 ceph-mon[32752]: pgmap v5030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:49:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8027.099516 secs 2024-04-26T20:49:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8027.809895 secs 2024-04-26T20:49:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:06 smithi167 ceph-mon[32751]: pgmap v5030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:49:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8027.099516 secs 2024-04-26T20:49:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8027.809895 secs 2024-04-26T20:49:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:06 smithi090 ceph-mon[29180]: pgmap v5030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:49:07.648 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8432 sec 2024-04-26T20:49:08.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8433 sec 2024-04-26T20:49:08.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:08 smithi090 ceph-mon[29180]: pgmap v5031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:49:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:08 smithi119 ceph-mon[32752]: pgmap v5031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:49:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:08 smithi167 ceph-mon[32751]: pgmap v5031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T20:49:09.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8434 sec 2024-04-26T20:49:10.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8435 sec 2024-04-26T20:49:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:10 smithi119 ceph-mon[32752]: pgmap v5032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:10 smithi167 ceph-mon[32751]: pgmap v5032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:10 smithi090 ceph-mon[29180]: pgmap v5032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:11.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8436 sec 2024-04-26T20:49:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8032.099604 secs 2024-04-26T20:49:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8032.810030 secs 2024-04-26T20:49:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8032.099604 secs 2024-04-26T20:49:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8032.810030 secs 2024-04-26T20:49:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8032.099604 secs 2024-04-26T20:49:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8032.810030 secs 2024-04-26T20:49:12.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8437 sec 2024-04-26T20:49:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:12 smithi119 ceph-mon[32752]: pgmap v5033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:12 smithi167 ceph-mon[32751]: pgmap v5033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:12 smithi090 ceph-mon[29180]: pgmap v5033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:13.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8438 sec 2024-04-26T20:49:14.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8439 sec 2024-04-26T20:49:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:14 smithi119 ceph-mon[32752]: pgmap v5034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:14 smithi167 ceph-mon[32751]: pgmap v5034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:14 smithi090 ceph-mon[29180]: pgmap v5034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:49:15.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8440 sec 2024-04-26T20:49:16.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8441 sec 2024-04-26T20:49:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8037.099684 secs 2024-04-26T20:49:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8037.810158 secs 2024-04-26T20:49:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:16 smithi119 ceph-mon[32752]: pgmap v5035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:49:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8037.099684 secs 2024-04-26T20:49:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8037.810158 secs 2024-04-26T20:49:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:16 smithi167 ceph-mon[32751]: pgmap v5035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:49:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8037.099684 secs 2024-04-26T20:49:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8037.810158 secs 2024-04-26T20:49:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:16 smithi090 ceph-mon[29180]: pgmap v5035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:49:17.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8442 sec 2024-04-26T20:49:18.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8443 sec 2024-04-26T20:49:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:18 smithi119 ceph-mon[32752]: pgmap v5036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:49:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:18 smithi167 ceph-mon[32751]: pgmap v5036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:49:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:18 smithi090 ceph-mon[29180]: pgmap v5036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T20:49:19.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8444 sec 2024-04-26T20:49:20.649 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8445 sec 2024-04-26T20:49:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:20 smithi119 ceph-mon[32752]: pgmap v5037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:49:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:20 smithi167 ceph-mon[32751]: pgmap v5037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:49:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:20 smithi090 ceph-mon[29180]: pgmap v5037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T20:49:21.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8446 sec 2024-04-26T20:49:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8042.099764 secs 2024-04-26T20:49:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8042.810292 secs 2024-04-26T20:49:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8042.099764 secs 2024-04-26T20:49:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8042.810292 secs 2024-04-26T20:49:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8042.099764 secs 2024-04-26T20:49:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8042.810292 secs 2024-04-26T20:49:22.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8447 sec 2024-04-26T20:49:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:22 smithi119 ceph-mon[32752]: pgmap v5038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:22 smithi167 ceph-mon[32751]: pgmap v5038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:22 smithi090 ceph-mon[29180]: pgmap v5038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:23.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8448 sec 2024-04-26T20:49:24.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8449 sec 2024-04-26T20:49:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:24 smithi119 ceph-mon[32752]: pgmap v5039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:24 smithi167 ceph-mon[32751]: pgmap v5039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:24 smithi090 ceph-mon[29180]: pgmap v5039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:25.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8450 sec 2024-04-26T20:49:26.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8451 sec 2024-04-26T20:49:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8047.099825 secs 2024-04-26T20:49:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8047.810398 secs 2024-04-26T20:49:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:26 smithi119 ceph-mon[32752]: pgmap v5040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8047.099825 secs 2024-04-26T20:49:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8047.810398 secs 2024-04-26T20:49:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:26 smithi167 ceph-mon[32751]: pgmap v5040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8047.099825 secs 2024-04-26T20:49:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8047.810398 secs 2024-04-26T20:49:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:26 smithi090 ceph-mon[29180]: pgmap v5040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:27.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8452 sec 2024-04-26T20:49:28.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8453 sec 2024-04-26T20:49:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:28 smithi119 ceph-mon[32752]: pgmap v5041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:29.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:28 smithi167 ceph-mon[32751]: pgmap v5041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:28 smithi090 ceph-mon[29180]: pgmap v5041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:29.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8454 sec 2024-04-26T20:49:30.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8455 sec 2024-04-26T20:49:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:30 smithi119 ceph-mon[32752]: pgmap v5042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:30 smithi167 ceph-mon[32751]: pgmap v5042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:30 smithi090 ceph-mon[29180]: pgmap v5042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:31.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8456 sec 2024-04-26T20:49:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8052.099908 secs 2024-04-26T20:49:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8052.810496 secs 2024-04-26T20:49:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8052.099908 secs 2024-04-26T20:49:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8052.810496 secs 2024-04-26T20:49:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8052.099908 secs 2024-04-26T20:49:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8052.810496 secs 2024-04-26T20:49:32.650 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8457 sec 2024-04-26T20:49:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:32 smithi119 ceph-mon[32752]: pgmap v5043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:32 smithi167 ceph-mon[32751]: pgmap v5043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:32 smithi090 ceph-mon[29180]: pgmap v5043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:49:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:49:32.930+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:49:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:49:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:49:32.930+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:49:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:49:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:49:32.930+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:49:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:49:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:49:32.930+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:49:33.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:49:32 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:49:32.930+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:49:33.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8458 sec 2024-04-26T20:49:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:49:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:49:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:49:34.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8459 sec 2024-04-26T20:49:34.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:34 smithi167 ceph-mon[32751]: pgmap v5044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:49:34.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:49:34.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:49:34.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:34 smithi167 ceph-mon[32751]: pgmap v5045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T20:49:34.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:34 smithi167 ceph-mon[32751]: pgmap v5046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:34.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:49:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:34 smithi119 ceph-mon[32752]: pgmap v5044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:49:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:49:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:49:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:34 smithi119 ceph-mon[32752]: pgmap v5045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T20:49:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:34 smithi119 ceph-mon[32752]: pgmap v5046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:49:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:34 smithi090 ceph-mon[29180]: pgmap v5044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T20:49:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:49:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:49:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:34 smithi090 ceph-mon[29180]: pgmap v5045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T20:49:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:34 smithi090 ceph-mon[29180]: pgmap v5046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:49:35.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8460 sec 2024-04-26T20:49:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:35 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8057.099996 secs 2024-04-26T20:49:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:35 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8057.099996 secs 2024-04-26T20:49:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:35 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8057.099996 secs 2024-04-26T20:49:36.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8461 sec 2024-04-26T20:49:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8057.810633 secs 2024-04-26T20:49:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:36 smithi119 ceph-mon[32752]: pgmap v5047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8057.810633 secs 2024-04-26T20:49:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:36 smithi167 ceph-mon[32751]: pgmap v5047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8057.810633 secs 2024-04-26T20:49:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:36 smithi090 ceph-mon[29180]: pgmap v5047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:37.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8462 sec 2024-04-26T20:49:38.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8463 sec 2024-04-26T20:49:39.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8464 sec 2024-04-26T20:49:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:39 smithi119 ceph-mon[32752]: pgmap v5048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:39 smithi167 ceph-mon[32751]: pgmap v5048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:39 smithi090 ceph-mon[29180]: pgmap v5048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:40.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8465 sec 2024-04-26T20:49:41.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8466 sec 2024-04-26T20:49:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8062.100104 secs 2024-04-26T20:49:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:41 smithi119 ceph-mon[32752]: pgmap v5049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8062.100104 secs 2024-04-26T20:49:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:41 smithi167 ceph-mon[32751]: pgmap v5049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8062.100104 secs 2024-04-26T20:49:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:41 smithi090 ceph-mon[29180]: pgmap v5049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T20:49:42.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8467 sec 2024-04-26T20:49:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8062.810769 secs 2024-04-26T20:49:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8062.810769 secs 2024-04-26T20:49:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8062.810769 secs 2024-04-26T20:49:43.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8468 sec 2024-04-26T20:49:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:43 smithi119 ceph-mon[32752]: pgmap v5050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:49:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:43 smithi167 ceph-mon[32751]: pgmap v5050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:49:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:43 smithi090 ceph-mon[29180]: pgmap v5050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:49:44.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8469 sec 2024-04-26T20:49:45.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8470 sec 2024-04-26T20:49:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:45 smithi119 ceph-mon[32752]: pgmap v5051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:49:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:45 smithi167 ceph-mon[32751]: pgmap v5051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:49:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:45 smithi090 ceph-mon[29180]: pgmap v5051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:49:46.651 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8471 sec 2024-04-26T20:49:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8067.100188 secs 2024-04-26T20:49:46.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8067.100188 secs 2024-04-26T20:49:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8067.100188 secs 2024-04-26T20:49:47.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8472 sec 2024-04-26T20:49:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8067.810907 secs 2024-04-26T20:49:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:47 smithi119 ceph-mon[32752]: pgmap v5052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8067.810907 secs 2024-04-26T20:49:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:47 smithi167 ceph-mon[32751]: pgmap v5052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8067.810907 secs 2024-04-26T20:49:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:47 smithi090 ceph-mon[29180]: pgmap v5052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:48.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8473 sec 2024-04-26T20:49:49.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8474 sec 2024-04-26T20:49:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:49 smithi119 ceph-mon[32752]: pgmap v5053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:49 smithi167 ceph-mon[32751]: pgmap v5053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:49 smithi090 ceph-mon[29180]: pgmap v5053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:50.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8475 sec 2024-04-26T20:49:51.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8476 sec 2024-04-26T20:49:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8072.100274 secs 2024-04-26T20:49:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:51 smithi119 ceph-mon[32752]: pgmap v5054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8072.100274 secs 2024-04-26T20:49:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:51 smithi167 ceph-mon[32751]: pgmap v5054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8072.100274 secs 2024-04-26T20:49:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:51 smithi090 ceph-mon[29180]: pgmap v5054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:52.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8477 sec 2024-04-26T20:49:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8072.811017 secs 2024-04-26T20:49:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8072.811017 secs 2024-04-26T20:49:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8072.811017 secs 2024-04-26T20:49:53.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8478 sec 2024-04-26T20:49:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:53 smithi119 ceph-mon[32752]: pgmap v5055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:53 smithi167 ceph-mon[32751]: pgmap v5055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:53 smithi090 ceph-mon[29180]: pgmap v5055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:54.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8479 sec 2024-04-26T20:49:55.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8480 sec 2024-04-26T20:49:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:55 smithi119 ceph-mon[32752]: pgmap v5056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:55 smithi167 ceph-mon[32751]: pgmap v5056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:55 smithi090 ceph-mon[29180]: pgmap v5056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:56.652 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8481 sec 2024-04-26T20:49:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8077.100355 secs 2024-04-26T20:49:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8077.100355 secs 2024-04-26T20:49:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8077.100355 secs 2024-04-26T20:49:57.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8482 sec 2024-04-26T20:49:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8077.811123 secs 2024-04-26T20:49:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:57 smithi119 ceph-mon[32752]: pgmap v5057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8077.811123 secs 2024-04-26T20:49:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:57 smithi167 ceph-mon[32751]: pgmap v5057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8077.811123 secs 2024-04-26T20:49:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:57 smithi090 ceph-mon[29180]: pgmap v5057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:49:58.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8483 sec 2024-04-26T20:49:59.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8484 sec 2024-04-26T20:49:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:49:59 smithi119 ceph-mon[32752]: pgmap v5058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:49:59 smithi167 ceph-mon[32751]: pgmap v5058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:49:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:49:59 smithi090 ceph-mon[29180]: pgmap v5058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:00.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8485 sec 2024-04-26T20:50:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:50:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:50:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T20:50:01.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8486 sec 2024-04-26T20:50:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8082.100440 secs 2024-04-26T20:50:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:01 smithi119 ceph-mon[32752]: pgmap v5059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8082.100440 secs 2024-04-26T20:50:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:01 smithi167 ceph-mon[32751]: pgmap v5059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:01.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8082.100440 secs 2024-04-26T20:50:01.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:01 smithi090 ceph-mon[29180]: pgmap v5059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:02.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8487 sec 2024-04-26T20:50:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8082.811258 secs 2024-04-26T20:50:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8082.811258 secs 2024-04-26T20:50:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8082.811258 secs 2024-04-26T20:50:03.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8488 sec 2024-04-26T20:50:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:03 smithi119 ceph-mon[32752]: pgmap v5060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:03 smithi167 ceph-mon[32751]: pgmap v5060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:03 smithi090 ceph-mon[29180]: pgmap v5060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:04.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8489 sec 2024-04-26T20:50:05.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8490 sec 2024-04-26T20:50:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:05 smithi119 ceph-mon[32752]: pgmap v5061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:05 smithi167 ceph-mon[32751]: pgmap v5061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:05 smithi090 ceph-mon[29180]: pgmap v5061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:06.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8491 sec 2024-04-26T20:50:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8087.100519 secs 2024-04-26T20:50:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8087.100519 secs 2024-04-26T20:50:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8087.100519 secs 2024-04-26T20:50:07.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8492 sec 2024-04-26T20:50:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8087.811401 secs 2024-04-26T20:50:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:07 smithi119 ceph-mon[32752]: pgmap v5062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8087.811401 secs 2024-04-26T20:50:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:07 smithi167 ceph-mon[32751]: pgmap v5062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8087.811401 secs 2024-04-26T20:50:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:07 smithi090 ceph-mon[29180]: pgmap v5062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:08.653 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8493 sec 2024-04-26T20:50:09.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8494 sec 2024-04-26T20:50:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:09 smithi119 ceph-mon[32752]: pgmap v5063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:09 smithi167 ceph-mon[32751]: pgmap v5063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:09 smithi090 ceph-mon[29180]: pgmap v5063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:10.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8495 sec 2024-04-26T20:50:11.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8496 sec 2024-04-26T20:50:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8092.100601 secs 2024-04-26T20:50:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:11 smithi119 ceph-mon[32752]: pgmap v5064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8092.100601 secs 2024-04-26T20:50:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:11 smithi167 ceph-mon[32751]: pgmap v5064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8092.100601 secs 2024-04-26T20:50:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:11 smithi090 ceph-mon[29180]: pgmap v5064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:12.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8497 sec 2024-04-26T20:50:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8092.811495 secs 2024-04-26T20:50:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8092.811495 secs 2024-04-26T20:50:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8092.811495 secs 2024-04-26T20:50:13.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8498 sec 2024-04-26T20:50:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:13 smithi119 ceph-mon[32752]: pgmap v5065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:13 smithi167 ceph-mon[32751]: pgmap v5065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:13 smithi090 ceph-mon[29180]: pgmap v5065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T20:50:14.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8499 sec 2024-04-26T20:50:15.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8500 sec 2024-04-26T20:50:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:15 smithi119 ceph-mon[32752]: pgmap v5066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:15 smithi167 ceph-mon[32751]: pgmap v5066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:15 smithi090 ceph-mon[29180]: pgmap v5066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T20:50:16.654 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8501 sec 2024-04-26T20:50:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8097.100685 secs 2024-04-26T20:50:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8097.100685 secs 2024-04-26T20:50:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8097.100685 secs 2024-04-26T20:50:17.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8502 sec 2024-04-26T20:50:17.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8097.811598 secs 2024-04-26T20:50:17.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:17 smithi090 ceph-mon[29180]: pgmap v5067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8097.811598 secs 2024-04-26T20:50:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:17 smithi119 ceph-mon[32752]: pgmap v5067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8097.811598 secs 2024-04-26T20:50:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:17 smithi167 ceph-mon[32751]: pgmap v5067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:18.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8503 sec 2024-04-26T20:50:19.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8504 sec 2024-04-26T20:50:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:19 smithi119 ceph-mon[32752]: pgmap v5068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:19 smithi167 ceph-mon[32751]: pgmap v5068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:19 smithi090 ceph-mon[29180]: pgmap v5068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:20.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8505 sec 2024-04-26T20:50:21.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8506 sec 2024-04-26T20:50:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8102.100763 secs 2024-04-26T20:50:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:21 smithi119 ceph-mon[32752]: pgmap v5069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8102.100763 secs 2024-04-26T20:50:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:21 smithi167 ceph-mon[32751]: pgmap v5069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8102.100763 secs 2024-04-26T20:50:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:21 smithi090 ceph-mon[29180]: pgmap v5069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:22.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8507 sec 2024-04-26T20:50:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8102.811728 secs 2024-04-26T20:50:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8102.811728 secs 2024-04-26T20:50:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8102.811728 secs 2024-04-26T20:50:23.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8508 sec 2024-04-26T20:50:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:23 smithi119 ceph-mon[32752]: pgmap v5070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:23 smithi167 ceph-mon[32751]: pgmap v5070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:23 smithi090 ceph-mon[29180]: pgmap v5070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:24.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8509 sec 2024-04-26T20:50:25.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8510 sec 2024-04-26T20:50:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:25 smithi119 ceph-mon[32752]: pgmap v5071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:25 smithi167 ceph-mon[32751]: pgmap v5071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:25 smithi090 ceph-mon[29180]: pgmap v5071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:26.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8511 sec 2024-04-26T20:50:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8107.100819 secs 2024-04-26T20:50:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8107.100819 secs 2024-04-26T20:50:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8107.100819 secs 2024-04-26T20:50:27.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8512 sec 2024-04-26T20:50:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8107.811854 secs 2024-04-26T20:50:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:27 smithi119 ceph-mon[32752]: pgmap v5072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8107.811854 secs 2024-04-26T20:50:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:27 smithi167 ceph-mon[32751]: pgmap v5072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8107.811854 secs 2024-04-26T20:50:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:27 smithi090 ceph-mon[29180]: pgmap v5072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 msgr2=0x7f2cc00b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T20:50:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 msgr2=0x7f2cc00b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T20:50:28.496 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 0x7f2cc00b4920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400fd90 tx=0x7f2cc4029040 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 0x7f2cc00b4920 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400fd90 tx=0x7f2cc4029040 comp rx=0 tx=0).stop 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000155) for more than 900000000 us, fault. 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 msgr2=0x7f2cc00b4920 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 0x7f2cc00b4920 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 0x7f2cc00b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=7 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b1a00 msgr2=0x7f2cc00b4920 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 0x7f2cc00b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.494+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:50:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.495+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 0x7f2cc00b2f00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40193f0 tx=0x7f2cc4004730 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:50:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.495+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T20:50:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:28.495+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T20:50:28.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8513 sec 2024-04-26T20:50:29.549 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:29.548+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:50:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:29.548+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:50:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:29.548+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 66 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006580 con 0x7f2cd4104500 2024-04-26T20:50:29.550 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:50:29.549+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 66 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:50:29.655 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8514 sec 2024-04-26T20:50:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:29 smithi119 ceph-mon[32752]: pgmap v5073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:29 smithi167 ceph-mon[32751]: pgmap v5073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:29 smithi090 ceph-mon[29180]: pgmap v5073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:30.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8515 sec 2024-04-26T20:50:31.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8516 sec 2024-04-26T20:50:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8112.100877 secs 2024-04-26T20:50:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:31 smithi119 ceph-mon[32752]: pgmap v5074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8112.100877 secs 2024-04-26T20:50:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:31 smithi167 ceph-mon[32751]: pgmap v5074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8112.100877 secs 2024-04-26T20:50:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:31 smithi090 ceph-mon[29180]: pgmap v5074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:32.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8517 sec 2024-04-26T20:50:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8112.811981 secs 2024-04-26T20:50:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8112.811981 secs 2024-04-26T20:50:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8112.811981 secs 2024-04-26T20:50:33.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8518 sec 2024-04-26T20:50:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:33 smithi119 ceph-mon[32752]: pgmap v5075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:33 smithi167 ceph-mon[32751]: pgmap v5075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:33 smithi090 ceph-mon[29180]: pgmap v5075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:34.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8519 sec 2024-04-26T20:50:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:50:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:50:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:50:35.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8520 sec 2024-04-26T20:50:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:35 smithi119 ceph-mon[32752]: pgmap v5076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:50:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:50:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:50:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:35 smithi167 ceph-mon[32751]: pgmap v5076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:50:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:50:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:50:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:35 smithi090 ceph-mon[29180]: pgmap v5076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:50:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:50:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:50:36.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8521 sec 2024-04-26T20:50:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8117.100945 secs 2024-04-26T20:50:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:36 smithi119 ceph-mon[32752]: pgmap v5077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:50:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:36 smithi119 ceph-mon[32752]: pgmap v5078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8117.100945 secs 2024-04-26T20:50:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:36 smithi167 ceph-mon[32751]: pgmap v5077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:50:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:36 smithi167 ceph-mon[32751]: pgmap v5078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8117.100945 secs 2024-04-26T20:50:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:36 smithi090 ceph-mon[29180]: pgmap v5077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:50:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:36 smithi090 ceph-mon[29180]: pgmap v5078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:37.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8522 sec 2024-04-26T20:50:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8117.812073 secs 2024-04-26T20:50:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8117.812073 secs 2024-04-26T20:50:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8117.812073 secs 2024-04-26T20:50:38.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8523 sec 2024-04-26T20:50:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:38 smithi119 ceph-mon[32752]: pgmap v5079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:50:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:38 smithi167 ceph-mon[32751]: pgmap v5079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:50:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:38 smithi090 ceph-mon[29180]: pgmap v5079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:50:39.656 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8524 sec 2024-04-26T20:50:40.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8525 sec 2024-04-26T20:50:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:40 smithi119 ceph-mon[32752]: pgmap v5080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:40 smithi167 ceph-mon[32751]: pgmap v5080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:40 smithi090 ceph-mon[29180]: pgmap v5080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:41.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8526 sec 2024-04-26T20:50:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8122.101030 secs 2024-04-26T20:50:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8122.101030 secs 2024-04-26T20:50:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8122.101030 secs 2024-04-26T20:50:42.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8527 sec 2024-04-26T20:50:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8122.812184 secs 2024-04-26T20:50:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:42 smithi119 ceph-mon[32752]: pgmap v5081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8122.812184 secs 2024-04-26T20:50:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:42 smithi167 ceph-mon[32751]: pgmap v5081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8122.812184 secs 2024-04-26T20:50:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:42 smithi090 ceph-mon[29180]: pgmap v5081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:50:43.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8528 sec 2024-04-26T20:50:44.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8529 sec 2024-04-26T20:50:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:44 smithi119 ceph-mon[32752]: pgmap v5082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:50:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:44 smithi167 ceph-mon[32751]: pgmap v5082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:50:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:44 smithi090 ceph-mon[29180]: pgmap v5082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:50:45.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8530 sec 2024-04-26T20:50:46.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8531 sec 2024-04-26T20:50:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8127.101113 secs 2024-04-26T20:50:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:46 smithi119 ceph-mon[32752]: pgmap v5083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:50:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8127.101113 secs 2024-04-26T20:50:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:46 smithi167 ceph-mon[32751]: pgmap v5083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:50:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8127.101113 secs 2024-04-26T20:50:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:46 smithi090 ceph-mon[29180]: pgmap v5083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:50:47.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8532 sec 2024-04-26T20:50:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8127.812256 secs 2024-04-26T20:50:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8127.812256 secs 2024-04-26T20:50:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8127.812256 secs 2024-04-26T20:50:48.657 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8533 sec 2024-04-26T20:50:48.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:48 smithi090 ceph-mon[29180]: pgmap v5084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:48 smithi167 ceph-mon[32751]: pgmap v5084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:48 smithi119 ceph-mon[32752]: pgmap v5084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:49.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8534 sec 2024-04-26T20:50:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:49 smithi167 ceph-mon[32751]: pgmap v5085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:49 smithi090 ceph-mon[29180]: pgmap v5085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:49 smithi119 ceph-mon[32752]: pgmap v5085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:50.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8535 sec 2024-04-26T20:50:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8132.101199 secs 2024-04-26T20:50:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8132.101199 secs 2024-04-26T20:50:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8132.101199 secs 2024-04-26T20:50:51.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8536 sec 2024-04-26T20:50:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8132.812353 secs 2024-04-26T20:50:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:51 smithi119 ceph-mon[32752]: pgmap v5086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8132.812353 secs 2024-04-26T20:50:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:51 smithi167 ceph-mon[32751]: pgmap v5086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8132.812353 secs 2024-04-26T20:50:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:51 smithi090 ceph-mon[29180]: pgmap v5086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:52.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8537 sec 2024-04-26T20:50:53.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8538 sec 2024-04-26T20:50:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:54 smithi119 ceph-mon[32752]: pgmap v5087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:54 smithi167 ceph-mon[32751]: pgmap v5087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:54 smithi090 ceph-mon[29180]: pgmap v5087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:54.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8539 sec 2024-04-26T20:50:55.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8540 sec 2024-04-26T20:50:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8137.101256 secs 2024-04-26T20:50:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:56 smithi119 ceph-mon[32752]: pgmap v5088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8137.101256 secs 2024-04-26T20:50:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:56 smithi167 ceph-mon[32751]: pgmap v5088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8137.101256 secs 2024-04-26T20:50:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:56 smithi090 ceph-mon[29180]: pgmap v5088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:50:56.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8541 sec 2024-04-26T20:50:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8137.812470 secs 2024-04-26T20:50:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8137.812470 secs 2024-04-26T20:50:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8137.812470 secs 2024-04-26T20:50:57.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8542 sec 2024-04-26T20:50:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:50:58 smithi119 ceph-mon[32752]: pgmap v5089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:50:58 smithi167 ceph-mon[32751]: pgmap v5089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:50:58 smithi090 ceph-mon[29180]: pgmap v5089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:50:58.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8543 sec 2024-04-26T20:50:59.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8544 sec 2024-04-26T20:51:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:00 smithi119 ceph-mon[32752]: pgmap v5090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:00 smithi167 ceph-mon[32751]: pgmap v5090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:00 smithi090 ceph-mon[29180]: pgmap v5090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:00.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8545 sec 2024-04-26T20:51:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8142.101341 secs 2024-04-26T20:51:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8142.101341 secs 2024-04-26T20:51:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8142.101341 secs 2024-04-26T20:51:01.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8546 sec 2024-04-26T20:51:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8142.812540 secs 2024-04-26T20:51:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:02 smithi119 ceph-mon[32752]: pgmap v5091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8142.812540 secs 2024-04-26T20:51:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:02 smithi167 ceph-mon[32751]: pgmap v5091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8142.812540 secs 2024-04-26T20:51:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:02 smithi090 ceph-mon[29180]: pgmap v5091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:02.658 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8547 sec 2024-04-26T20:51:03.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8548 sec 2024-04-26T20:51:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:04 smithi119 ceph-mon[32752]: pgmap v5092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:04 smithi167 ceph-mon[32751]: pgmap v5092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:04 smithi090 ceph-mon[29180]: pgmap v5092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:04.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8549 sec 2024-04-26T20:51:05.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8550 sec 2024-04-26T20:51:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8147.101425 secs 2024-04-26T20:51:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:06 smithi119 ceph-mon[32752]: pgmap v5093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8147.101425 secs 2024-04-26T20:51:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:06 smithi167 ceph-mon[32751]: pgmap v5093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8147.101425 secs 2024-04-26T20:51:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:06 smithi090 ceph-mon[29180]: pgmap v5093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:06.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8551 sec 2024-04-26T20:51:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8147.812656 secs 2024-04-26T20:51:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8147.812656 secs 2024-04-26T20:51:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8147.812656 secs 2024-04-26T20:51:07.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8552 sec 2024-04-26T20:51:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:08 smithi119 ceph-mon[32752]: pgmap v5094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:08 smithi167 ceph-mon[32751]: pgmap v5094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:08 smithi090 ceph-mon[29180]: pgmap v5094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:08.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8553 sec 2024-04-26T20:51:09.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8554 sec 2024-04-26T20:51:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:10 smithi119 ceph-mon[32752]: pgmap v5095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:10 smithi167 ceph-mon[32751]: pgmap v5095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:10 smithi090 ceph-mon[29180]: pgmap v5095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:10.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8555 sec 2024-04-26T20:51:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8152.101503 secs 2024-04-26T20:51:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8152.101503 secs 2024-04-26T20:51:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8152.101503 secs 2024-04-26T20:51:11.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8556 sec 2024-04-26T20:51:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8152.812773 secs 2024-04-26T20:51:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:12 smithi119 ceph-mon[32752]: pgmap v5096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8152.812773 secs 2024-04-26T20:51:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:12 smithi167 ceph-mon[32751]: pgmap v5096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8152.812773 secs 2024-04-26T20:51:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:12 smithi090 ceph-mon[29180]: pgmap v5096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:12.659 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8557 sec 2024-04-26T20:51:13.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8558 sec 2024-04-26T20:51:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:14 smithi119 ceph-mon[32752]: pgmap v5097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:14 smithi167 ceph-mon[32751]: pgmap v5097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:14 smithi090 ceph-mon[29180]: pgmap v5097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:14.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8559 sec 2024-04-26T20:51:15.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8560 sec 2024-04-26T20:51:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8157.101587 secs 2024-04-26T20:51:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:16 smithi119 ceph-mon[32752]: pgmap v5098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8157.101587 secs 2024-04-26T20:51:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:16 smithi167 ceph-mon[32751]: pgmap v5098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8157.101587 secs 2024-04-26T20:51:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:16 smithi090 ceph-mon[29180]: pgmap v5098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:16.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8561 sec 2024-04-26T20:51:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8157.812911 secs 2024-04-26T20:51:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8157.812911 secs 2024-04-26T20:51:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8157.812911 secs 2024-04-26T20:51:17.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8562 sec 2024-04-26T20:51:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:18 smithi119 ceph-mon[32752]: pgmap v5099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:18 smithi167 ceph-mon[32751]: pgmap v5099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:18 smithi090 ceph-mon[29180]: pgmap v5099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:18.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8563 sec 2024-04-26T20:51:19.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8564 sec 2024-04-26T20:51:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:20 smithi119 ceph-mon[32752]: pgmap v5100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:20 smithi167 ceph-mon[32751]: pgmap v5100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:20 smithi090 ceph-mon[29180]: pgmap v5100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:20.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8565 sec 2024-04-26T20:51:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8162.101687 secs 2024-04-26T20:51:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8162.101687 secs 2024-04-26T20:51:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8162.101687 secs 2024-04-26T20:51:21.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8566 sec 2024-04-26T20:51:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8162.813020 secs 2024-04-26T20:51:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:22 smithi119 ceph-mon[32752]: pgmap v5101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8162.813020 secs 2024-04-26T20:51:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:22 smithi167 ceph-mon[32751]: pgmap v5101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8162.813020 secs 2024-04-26T20:51:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:22 smithi090 ceph-mon[29180]: pgmap v5101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:22.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8567 sec 2024-04-26T20:51:23.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8568 sec 2024-04-26T20:51:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:24 smithi119 ceph-mon[32752]: pgmap v5102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:24 smithi167 ceph-mon[32751]: pgmap v5102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:24 smithi090 ceph-mon[29180]: pgmap v5102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:24.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8569 sec 2024-04-26T20:51:25.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8570 sec 2024-04-26T20:51:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8167.101772 secs 2024-04-26T20:51:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:26 smithi119 ceph-mon[32752]: pgmap v5103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8167.101772 secs 2024-04-26T20:51:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:26 smithi167 ceph-mon[32751]: pgmap v5103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8167.101772 secs 2024-04-26T20:51:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:26 smithi090 ceph-mon[29180]: pgmap v5103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:26.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8571 sec 2024-04-26T20:51:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8167.813161 secs 2024-04-26T20:51:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8167.813161 secs 2024-04-26T20:51:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8167.813161 secs 2024-04-26T20:51:27.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8572 sec 2024-04-26T20:51:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:28 smithi119 ceph-mon[32752]: pgmap v5104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:28 smithi167 ceph-mon[32751]: pgmap v5104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:28 smithi090 ceph-mon[29180]: pgmap v5104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:28.660 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8573 sec 2024-04-26T20:51:29.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8574 sec 2024-04-26T20:51:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:30 smithi119 ceph-mon[32752]: pgmap v5105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:30 smithi167 ceph-mon[32751]: pgmap v5105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:30 smithi090 ceph-mon[29180]: pgmap v5105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:30.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8575 sec 2024-04-26T20:51:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8172.101851 secs 2024-04-26T20:51:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8172.101851 secs 2024-04-26T20:51:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8172.101851 secs 2024-04-26T20:51:31.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8576 sec 2024-04-26T20:51:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8172.813268 secs 2024-04-26T20:51:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:32 smithi119 ceph-mon[32752]: pgmap v5106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8172.813268 secs 2024-04-26T20:51:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:32 smithi167 ceph-mon[32751]: pgmap v5106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8172.813268 secs 2024-04-26T20:51:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:32 smithi090 ceph-mon[29180]: pgmap v5106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:32.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8577 sec 2024-04-26T20:51:33.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8578 sec 2024-04-26T20:51:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:34 smithi119 ceph-mon[32752]: pgmap v5107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:34 smithi167 ceph-mon[32751]: pgmap v5107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:34 smithi090 ceph-mon[29180]: pgmap v5107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:34.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8579 sec 2024-04-26T20:51:35.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:51:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:51:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:51:35.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8580 sec 2024-04-26T20:51:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8177.101933 secs 2024-04-26T20:51:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:36 smithi119 ceph-mon[32752]: pgmap v5108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:51:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:51:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:51:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8177.101933 secs 2024-04-26T20:51:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:36 smithi167 ceph-mon[32751]: pgmap v5108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:51:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:51:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:51:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8177.101933 secs 2024-04-26T20:51:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:36 smithi090 ceph-mon[29180]: pgmap v5108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:51:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:51:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:51:36.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8581 sec 2024-04-26T20:51:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8177.813327 secs 2024-04-26T20:51:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:37 smithi119 ceph-mon[32752]: pgmap v5109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:51:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:37 smithi119 ceph-mon[32752]: pgmap v5110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8177.813327 secs 2024-04-26T20:51:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:37 smithi167 ceph-mon[32751]: pgmap v5109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:51:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:37 smithi167 ceph-mon[32751]: pgmap v5110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8177.813327 secs 2024-04-26T20:51:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:37 smithi090 ceph-mon[29180]: pgmap v5109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:51:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:37 smithi090 ceph-mon[29180]: pgmap v5110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:37.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8582 sec 2024-04-26T20:51:38.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8583 sec 2024-04-26T20:51:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:39 smithi119 ceph-mon[32752]: pgmap v5111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:51:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:39 smithi167 ceph-mon[32751]: pgmap v5111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:51:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:39 smithi090 ceph-mon[29180]: pgmap v5111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:51:39.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8584 sec 2024-04-26T20:51:40.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8585 sec 2024-04-26T20:51:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8182.102015 secs 2024-04-26T20:51:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:41 smithi119 ceph-mon[32752]: pgmap v5112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8182.102015 secs 2024-04-26T20:51:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:41 smithi167 ceph-mon[32751]: pgmap v5112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8182.102015 secs 2024-04-26T20:51:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:41 smithi090 ceph-mon[29180]: pgmap v5112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:41.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8586 sec 2024-04-26T20:51:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8182.813419 secs 2024-04-26T20:51:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8182.813419 secs 2024-04-26T20:51:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8182.813419 secs 2024-04-26T20:51:42.661 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8587 sec 2024-04-26T20:51:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:43 smithi119 ceph-mon[32752]: pgmap v5113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:43 smithi167 ceph-mon[32751]: pgmap v5113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:43 smithi090 ceph-mon[29180]: pgmap v5113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:43.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8588 sec 2024-04-26T20:51:44.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8589 sec 2024-04-26T20:51:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:45 smithi119 ceph-mon[32752]: pgmap v5114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:45 smithi167 ceph-mon[32751]: pgmap v5114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:45 smithi090 ceph-mon[29180]: pgmap v5114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:51:45.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8590 sec 2024-04-26T20:51:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8187.102104 secs 2024-04-26T20:51:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8187.102104 secs 2024-04-26T20:51:46.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8591 sec 2024-04-26T20:51:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8187.102104 secs 2024-04-26T20:51:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8187.813539 secs 2024-04-26T20:51:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:47 smithi167 ceph-mon[32751]: pgmap v5115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:51:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8187.813539 secs 2024-04-26T20:51:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:47 smithi090 ceph-mon[29180]: pgmap v5115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:51:47.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8592 sec 2024-04-26T20:51:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8187.813539 secs 2024-04-26T20:51:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:47 smithi119 ceph-mon[32752]: pgmap v5115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:51:48.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8593 sec 2024-04-26T20:51:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:49 smithi167 ceph-mon[32751]: pgmap v5116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:49 smithi090 ceph-mon[29180]: pgmap v5116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:49.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8594 sec 2024-04-26T20:51:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:49 smithi119 ceph-mon[32752]: pgmap v5116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:50.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8595 sec 2024-04-26T20:51:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8192.102167 secs 2024-04-26T20:51:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:51 smithi167 ceph-mon[32751]: pgmap v5117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8192.102167 secs 2024-04-26T20:51:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:51 smithi090 ceph-mon[29180]: pgmap v5117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:51.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8596 sec 2024-04-26T20:51:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8192.102167 secs 2024-04-26T20:51:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:51 smithi119 ceph-mon[32752]: pgmap v5117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8192.813606 secs 2024-04-26T20:51:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8192.813606 secs 2024-04-26T20:51:52.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8597 sec 2024-04-26T20:51:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8192.813606 secs 2024-04-26T20:51:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:53 smithi090 ceph-mon[29180]: pgmap v5118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:53.662 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8598 sec 2024-04-26T20:51:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:53 smithi119 ceph-mon[32752]: pgmap v5118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:53 smithi167 ceph-mon[32751]: pgmap v5118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:54.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8599 sec 2024-04-26T20:51:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:55 smithi090 ceph-mon[29180]: pgmap v5119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:55.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8600 sec 2024-04-26T20:51:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:55 smithi119 ceph-mon[32752]: pgmap v5119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:55 smithi167 ceph-mon[32751]: pgmap v5119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:51:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8197.102226 secs 2024-04-26T20:51:56.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8601 sec 2024-04-26T20:51:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8197.102226 secs 2024-04-26T20:51:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8197.102226 secs 2024-04-26T20:51:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8197.813678 secs 2024-04-26T20:51:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:57 smithi090 ceph-mon[29180]: pgmap v5120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:57.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8602 sec 2024-04-26T20:51:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8197.813678 secs 2024-04-26T20:51:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:57 smithi119 ceph-mon[32752]: pgmap v5120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8197.813678 secs 2024-04-26T20:51:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:57 smithi167 ceph-mon[32751]: pgmap v5120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:58.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8603 sec 2024-04-26T20:51:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:51:59 smithi090 ceph-mon[29180]: pgmap v5121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:59.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8604 sec 2024-04-26T20:51:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:51:59 smithi119 ceph-mon[32752]: pgmap v5121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:51:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:51:59 smithi167 ceph-mon[32751]: pgmap v5121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:00.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8605 sec 2024-04-26T20:52:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8202.102307 secs 2024-04-26T20:52:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:01 smithi090 ceph-mon[29180]: pgmap v5122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:01.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8606 sec 2024-04-26T20:52:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8202.102307 secs 2024-04-26T20:52:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:01 smithi119 ceph-mon[32752]: pgmap v5122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8202.102307 secs 2024-04-26T20:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:01 smithi167 ceph-mon[32751]: pgmap v5122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8202.813759 secs 2024-04-26T20:52:02.663 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8607 sec 2024-04-26T20:52:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8202.813759 secs 2024-04-26T20:52:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8202.813759 secs 2024-04-26T20:52:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:03 smithi090 ceph-mon[29180]: pgmap v5123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:03.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8608 sec 2024-04-26T20:52:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:03 smithi119 ceph-mon[32752]: pgmap v5123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:03 smithi167 ceph-mon[32751]: pgmap v5123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:04.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8609 sec 2024-04-26T20:52:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:05 smithi090 ceph-mon[29180]: pgmap v5124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:05.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8610 sec 2024-04-26T20:52:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:05 smithi119 ceph-mon[32752]: pgmap v5124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:05 smithi167 ceph-mon[32751]: pgmap v5124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8207.102386 secs 2024-04-26T20:52:06.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8611 sec 2024-04-26T20:52:06.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8207.102386 secs 2024-04-26T20:52:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8207.102386 secs 2024-04-26T20:52:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:07 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 8207.813821 secs 2024-04-26T20:52:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:07 smithi090 ceph-mon[29180]: slow request 1442.975009 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:52:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:07 smithi090 ceph-mon[29180]: pgmap v5125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:07.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8612 sec 2024-04-26T20:52:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:07 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 8207.813821 secs 2024-04-26T20:52:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:07 smithi119 ceph-mon[32752]: slow request 1442.975009 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:52:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:07 smithi119 ceph-mon[32752]: pgmap v5125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:07 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 8207.813821 secs 2024-04-26T20:52:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:07 smithi167 ceph-mon[32751]: slow request 1442.975009 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T20:52:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:07 smithi167 ceph-mon[32751]: pgmap v5125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:08.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8613 sec 2024-04-26T20:52:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:09 smithi090 ceph-mon[29180]: pgmap v5126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:09.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8614 sec 2024-04-26T20:52:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:09 smithi119 ceph-mon[32752]: pgmap v5126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:09 smithi167 ceph-mon[32751]: pgmap v5126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:10.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8615 sec 2024-04-26T20:52:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8212.102468 secs 2024-04-26T20:52:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:11 smithi090 ceph-mon[29180]: pgmap v5127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:11.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8616 sec 2024-04-26T20:52:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8212.102468 secs 2024-04-26T20:52:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:11 smithi119 ceph-mon[32752]: pgmap v5127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8212.102468 secs 2024-04-26T20:52:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:11 smithi167 ceph-mon[32751]: pgmap v5127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8212.813901 secs 2024-04-26T20:52:12.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8617 sec 2024-04-26T20:52:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8212.813901 secs 2024-04-26T20:52:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8212.813901 secs 2024-04-26T20:52:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:13 smithi090 ceph-mon[29180]: pgmap v5128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:13.664 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8618 sec 2024-04-26T20:52:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:13 smithi119 ceph-mon[32752]: pgmap v5128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:13 smithi167 ceph-mon[32751]: pgmap v5128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:14.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8619 sec 2024-04-26T20:52:15.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8620 sec 2024-04-26T20:52:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:15 smithi119 ceph-mon[32752]: pgmap v5129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:15 smithi167 ceph-mon[32751]: pgmap v5129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:15 smithi090 ceph-mon[29180]: pgmap v5129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:16.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8621 sec 2024-04-26T20:52:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8217.102551 secs 2024-04-26T20:52:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8217.102551 secs 2024-04-26T20:52:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8217.102551 secs 2024-04-26T20:52:17.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8622 sec 2024-04-26T20:52:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8217.813962 secs 2024-04-26T20:52:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:17 smithi119 ceph-mon[32752]: pgmap v5130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8217.813962 secs 2024-04-26T20:52:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:17 smithi167 ceph-mon[32751]: pgmap v5130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8217.813962 secs 2024-04-26T20:52:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:17 smithi090 ceph-mon[29180]: pgmap v5130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:18.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8623 sec 2024-04-26T20:52:19.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8624 sec 2024-04-26T20:52:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:19 smithi119 ceph-mon[32752]: pgmap v5131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:19 smithi167 ceph-mon[32751]: pgmap v5131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:19 smithi090 ceph-mon[29180]: pgmap v5131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:20.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8625 sec 2024-04-26T20:52:21.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8626 sec 2024-04-26T20:52:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:21 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 8222.102634 secs 2024-04-26T20:52:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:21 smithi119 ceph-mon[32752]: slow request 721.161929 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:52:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:21 smithi119 ceph-mon[32752]: pgmap v5132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8222.814039 secs 2024-04-26T20:52:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:21 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 8222.102634 secs 2024-04-26T20:52:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:21 smithi167 ceph-mon[32751]: slow request 721.161929 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:52:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:21 smithi167 ceph-mon[32751]: pgmap v5132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8222.814039 secs 2024-04-26T20:52:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:21 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 8222.102634 secs 2024-04-26T20:52:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:21 smithi090 ceph-mon[29180]: slow request 721.161929 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:52:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:21 smithi090 ceph-mon[29180]: pgmap v5132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8222.814039 secs 2024-04-26T20:52:22.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8627 sec 2024-04-26T20:52:23.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8628 sec 2024-04-26T20:52:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:23 smithi119 ceph-mon[32752]: pgmap v5133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:23 smithi167 ceph-mon[32751]: pgmap v5133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:23 smithi090 ceph-mon[29180]: pgmap v5133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:24.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8629 sec 2024-04-26T20:52:25.665 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8630 sec 2024-04-26T20:52:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:25 smithi119 ceph-mon[32752]: pgmap v5134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:25 smithi167 ceph-mon[32751]: pgmap v5134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:25 smithi090 ceph-mon[29180]: pgmap v5134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:26.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8631 sec 2024-04-26T20:52:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8227.102715 secs 2024-04-26T20:52:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8227.814150 secs 2024-04-26T20:52:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8227.102715 secs 2024-04-26T20:52:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8227.814150 secs 2024-04-26T20:52:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8227.102715 secs 2024-04-26T20:52:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8227.814150 secs 2024-04-26T20:52:27.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8632 sec 2024-04-26T20:52:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:27 smithi119 ceph-mon[32752]: pgmap v5135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:27 smithi167 ceph-mon[32751]: pgmap v5135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:27 smithi090 ceph-mon[29180]: pgmap v5135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:28.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8633 sec 2024-04-26T20:52:29.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8634 sec 2024-04-26T20:52:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:29 smithi119 ceph-mon[32752]: pgmap v5136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:29 smithi167 ceph-mon[32751]: pgmap v5136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:29 smithi090 ceph-mon[29180]: pgmap v5136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:30.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8635 sec 2024-04-26T20:52:31.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8636 sec 2024-04-26T20:52:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8232.102800 secs 2024-04-26T20:52:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:31 smithi119 ceph-mon[32752]: pgmap v5137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8232.814267 secs 2024-04-26T20:52:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8232.102800 secs 2024-04-26T20:52:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:31 smithi167 ceph-mon[32751]: pgmap v5137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8232.814267 secs 2024-04-26T20:52:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8232.102800 secs 2024-04-26T20:52:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:31 smithi090 ceph-mon[29180]: pgmap v5137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8232.814267 secs 2024-04-26T20:52:32.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8637 sec 2024-04-26T20:52:33.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8638 sec 2024-04-26T20:52:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:33 smithi119 ceph-mon[32752]: pgmap v5138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:33 smithi167 ceph-mon[32751]: pgmap v5138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:33 smithi090 ceph-mon[29180]: pgmap v5138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:34.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8639 sec 2024-04-26T20:52:35.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8640 sec 2024-04-26T20:52:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:35 smithi119 ceph-mon[32752]: pgmap v5139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:35 smithi167 ceph-mon[32751]: pgmap v5139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:35 smithi090 ceph-mon[29180]: pgmap v5139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:36.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8641 sec 2024-04-26T20:52:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8237.102889 secs 2024-04-26T20:52:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8237.814411 secs 2024-04-26T20:52:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:52:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8237.102889 secs 2024-04-26T20:52:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8237.814411 secs 2024-04-26T20:52:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:52:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8237.102889 secs 2024-04-26T20:52:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8237.814411 secs 2024-04-26T20:52:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:52:37.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8642 sec 2024-04-26T20:52:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:37 smithi119 ceph-mon[32752]: pgmap v5140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:52:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:52:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:52:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:37 smithi167 ceph-mon[32751]: pgmap v5140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:52:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:52:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:52:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:37 smithi090 ceph-mon[29180]: pgmap v5140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:52:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:52:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:52:38.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8643 sec 2024-04-26T20:52:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:38 smithi119 ceph-mon[32752]: pgmap v5141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:52:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:38 smithi119 ceph-mon[32752]: pgmap v5142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:38 smithi167 ceph-mon[32751]: pgmap v5141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:52:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:38 smithi167 ceph-mon[32751]: pgmap v5142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:38 smithi090 ceph-mon[29180]: pgmap v5141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:52:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:38 smithi090 ceph-mon[29180]: pgmap v5142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:39.666 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8644 sec 2024-04-26T20:52:40.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8645 sec 2024-04-26T20:52:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:40 smithi090 ceph-mon[29180]: pgmap v5143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:40 smithi119 ceph-mon[32752]: pgmap v5143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:40 smithi167 ceph-mon[32751]: pgmap v5143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:41.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8646 sec 2024-04-26T20:52:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8242.102951 secs 2024-04-26T20:52:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8242.814477 secs 2024-04-26T20:52:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8242.102951 secs 2024-04-26T20:52:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8242.814477 secs 2024-04-26T20:52:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8242.102951 secs 2024-04-26T20:52:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8242.814477 secs 2024-04-26T20:52:42.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8647 sec 2024-04-26T20:52:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:42 smithi090 ceph-mon[29180]: pgmap v5144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:42 smithi119 ceph-mon[32752]: pgmap v5144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:42 smithi167 ceph-mon[32751]: pgmap v5144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:43.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8648 sec 2024-04-26T20:52:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:43 smithi090 ceph-mon[29180]: pgmap v5145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:52:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:43 smithi119 ceph-mon[32752]: pgmap v5145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:52:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:43 smithi167 ceph-mon[32751]: pgmap v5145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:52:44.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8649 sec 2024-04-26T20:52:45.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8650 sec 2024-04-26T20:52:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:45 smithi090 ceph-mon[29180]: pgmap v5146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8247.814547 secs 2024-04-26T20:52:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:45 smithi119 ceph-mon[32752]: pgmap v5146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8247.814547 secs 2024-04-26T20:52:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:45 smithi167 ceph-mon[32751]: pgmap v5146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:52:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8247.814547 secs 2024-04-26T20:52:46.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8651 sec 2024-04-26T20:52:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8247.103053 secs 2024-04-26T20:52:46.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8247.103053 secs 2024-04-26T20:52:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8247.103053 secs 2024-04-26T20:52:47.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8652 sec 2024-04-26T20:52:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:47 smithi090 ceph-mon[29180]: pgmap v5147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:52:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:47 smithi119 ceph-mon[32752]: pgmap v5147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:52:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:47 smithi167 ceph-mon[32751]: pgmap v5147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:52:48.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8653 sec 2024-04-26T20:52:49.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8654 sec 2024-04-26T20:52:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:49 smithi119 ceph-mon[32752]: pgmap v5148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:49 smithi167 ceph-mon[32751]: pgmap v5148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:49 smithi090 ceph-mon[29180]: pgmap v5148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:50.667 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8655 sec 2024-04-26T20:52:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8252.814619 secs 2024-04-26T20:52:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8252.814619 secs 2024-04-26T20:52:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8252.814619 secs 2024-04-26T20:52:51.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8656 sec 2024-04-26T20:52:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8252.103139 secs 2024-04-26T20:52:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:51 smithi119 ceph-mon[32752]: pgmap v5149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8252.103139 secs 2024-04-26T20:52:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:51 smithi167 ceph-mon[32751]: pgmap v5149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8252.103139 secs 2024-04-26T20:52:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:51 smithi090 ceph-mon[29180]: pgmap v5149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:52.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8657 sec 2024-04-26T20:52:53.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8658 sec 2024-04-26T20:52:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:53 smithi119 ceph-mon[32752]: pgmap v5150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:53 smithi167 ceph-mon[32751]: pgmap v5150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:53 smithi090 ceph-mon[29180]: pgmap v5150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:54.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8659 sec 2024-04-26T20:52:55.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8660 sec 2024-04-26T20:52:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:55 smithi119 ceph-mon[32752]: pgmap v5151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8257.814722 secs 2024-04-26T20:52:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:55 smithi167 ceph-mon[32751]: pgmap v5151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8257.814722 secs 2024-04-26T20:52:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:55 smithi090 ceph-mon[29180]: pgmap v5151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:52:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8257.814722 secs 2024-04-26T20:52:56.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8661 sec 2024-04-26T20:52:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8257.103222 secs 2024-04-26T20:52:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8257.103222 secs 2024-04-26T20:52:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8257.103222 secs 2024-04-26T20:52:57.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8662 sec 2024-04-26T20:52:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:57 smithi119 ceph-mon[32752]: pgmap v5152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:57 smithi167 ceph-mon[32751]: pgmap v5152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:57 smithi090 ceph-mon[29180]: pgmap v5152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:52:58.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8663 sec 2024-04-26T20:52:59.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8664 sec 2024-04-26T20:53:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:52:59 smithi119 ceph-mon[32752]: pgmap v5153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:52:59 smithi167 ceph-mon[32751]: pgmap v5153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:52:59 smithi090 ceph-mon[29180]: pgmap v5153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:00.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8665 sec 2024-04-26T20:53:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8262.814812 secs 2024-04-26T20:53:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8262.814812 secs 2024-04-26T20:53:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8262.814812 secs 2024-04-26T20:53:01.668 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8666 sec 2024-04-26T20:53:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8262.103307 secs 2024-04-26T20:53:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:01 smithi119 ceph-mon[32752]: pgmap v5154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8262.103307 secs 2024-04-26T20:53:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:01 smithi167 ceph-mon[32751]: pgmap v5154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8262.103307 secs 2024-04-26T20:53:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:01 smithi090 ceph-mon[29180]: pgmap v5154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:02.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8667 sec 2024-04-26T20:53:03.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8668 sec 2024-04-26T20:53:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:03 smithi119 ceph-mon[32752]: pgmap v5155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:03 smithi167 ceph-mon[32751]: pgmap v5155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:03 smithi090 ceph-mon[29180]: pgmap v5155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:04.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8669 sec 2024-04-26T20:53:05.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8670 sec 2024-04-26T20:53:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:05 smithi119 ceph-mon[32752]: pgmap v5156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8267.814904 secs 2024-04-26T20:53:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:05 smithi167 ceph-mon[32751]: pgmap v5156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8267.814904 secs 2024-04-26T20:53:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:05 smithi090 ceph-mon[29180]: pgmap v5156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8267.814904 secs 2024-04-26T20:53:06.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8671 sec 2024-04-26T20:53:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8267.103391 secs 2024-04-26T20:53:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8267.103391 secs 2024-04-26T20:53:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8267.103391 secs 2024-04-26T20:53:07.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8672 sec 2024-04-26T20:53:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:07 smithi119 ceph-mon[32752]: pgmap v5157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:07 smithi167 ceph-mon[32751]: pgmap v5157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:07 smithi090 ceph-mon[29180]: pgmap v5157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:08.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8673 sec 2024-04-26T20:53:09.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8674 sec 2024-04-26T20:53:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:09 smithi119 ceph-mon[32752]: pgmap v5158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:09 smithi167 ceph-mon[32751]: pgmap v5158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:09 smithi090 ceph-mon[29180]: pgmap v5158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:10.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8675 sec 2024-04-26T20:53:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8272.815028 secs 2024-04-26T20:53:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8272.815028 secs 2024-04-26T20:53:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8272.815028 secs 2024-04-26T20:53:11.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8676 sec 2024-04-26T20:53:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8272.103498 secs 2024-04-26T20:53:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:11 smithi119 ceph-mon[32752]: pgmap v5159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8272.103498 secs 2024-04-26T20:53:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:11 smithi167 ceph-mon[32751]: pgmap v5159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8272.103498 secs 2024-04-26T20:53:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:11 smithi090 ceph-mon[29180]: pgmap v5159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:12.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8677 sec 2024-04-26T20:53:13.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8678 sec 2024-04-26T20:53:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:13 smithi119 ceph-mon[32752]: pgmap v5160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:13 smithi167 ceph-mon[32751]: pgmap v5160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:13 smithi090 ceph-mon[29180]: pgmap v5160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:14.669 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8679 sec 2024-04-26T20:53:15.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8680 sec 2024-04-26T20:53:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:15 smithi119 ceph-mon[32752]: pgmap v5161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8277.815088 secs 2024-04-26T20:53:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:15 smithi167 ceph-mon[32751]: pgmap v5161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8277.815088 secs 2024-04-26T20:53:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:15 smithi090 ceph-mon[29180]: pgmap v5161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8277.815088 secs 2024-04-26T20:53:16.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8681 sec 2024-04-26T20:53:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8277.103558 secs 2024-04-26T20:53:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8277.103558 secs 2024-04-26T20:53:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8277.103558 secs 2024-04-26T20:53:17.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8682 sec 2024-04-26T20:53:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:17 smithi119 ceph-mon[32752]: pgmap v5162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:17 smithi167 ceph-mon[32751]: pgmap v5162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:17 smithi090 ceph-mon[29180]: pgmap v5162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:18.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8683 sec 2024-04-26T20:53:19.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8684 sec 2024-04-26T20:53:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:19 smithi167 ceph-mon[32751]: pgmap v5163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:19 smithi090 ceph-mon[29180]: pgmap v5163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:19 smithi119 ceph-mon[32752]: pgmap v5163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:20.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8685 sec 2024-04-26T20:53:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8282.815207 secs 2024-04-26T20:53:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8282.815207 secs 2024-04-26T20:53:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8282.815207 secs 2024-04-26T20:53:21.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8686 sec 2024-04-26T20:53:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8282.103643 secs 2024-04-26T20:53:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:21 smithi167 ceph-mon[32751]: pgmap v5164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8282.103643 secs 2024-04-26T20:53:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:21 smithi090 ceph-mon[29180]: pgmap v5164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8282.103643 secs 2024-04-26T20:53:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:21 smithi119 ceph-mon[32752]: pgmap v5164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:22.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8687 sec 2024-04-26T20:53:23.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8688 sec 2024-04-26T20:53:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:23 smithi167 ceph-mon[32751]: pgmap v5165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:23 smithi090 ceph-mon[29180]: pgmap v5165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:23 smithi119 ceph-mon[32752]: pgmap v5165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:24.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8689 sec 2024-04-26T20:53:25.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8690 sec 2024-04-26T20:53:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:25 smithi090 ceph-mon[29180]: pgmap v5166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8287.815316 secs 2024-04-26T20:53:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:25 smithi119 ceph-mon[32752]: pgmap v5166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8287.815316 secs 2024-04-26T20:53:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:25 smithi167 ceph-mon[32751]: pgmap v5166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8287.815316 secs 2024-04-26T20:53:26.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8691 sec 2024-04-26T20:53:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8287.103728 secs 2024-04-26T20:53:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8287.103728 secs 2024-04-26T20:53:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8287.103728 secs 2024-04-26T20:53:27.670 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8692 sec 2024-04-26T20:53:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:27 smithi090 ceph-mon[29180]: pgmap v5167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:27 smithi119 ceph-mon[32752]: pgmap v5167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:27 smithi167 ceph-mon[32751]: pgmap v5167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:28.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8693 sec 2024-04-26T20:53:29.574 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:53:29.573+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006530 con 0x7f2cd4104500 2024-04-26T20:53:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:53:29.573+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:53:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:53:29.574+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 67 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:53:29.575 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:53:29.574+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 67 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006530 con 0x7f2cd4104500 2024-04-26T20:53:29.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8694 sec 2024-04-26T20:53:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:29 smithi090 ceph-mon[29180]: pgmap v5168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:29 smithi119 ceph-mon[32752]: pgmap v5168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:29 smithi167 ceph-mon[32751]: pgmap v5168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:30.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8695 sec 2024-04-26T20:53:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8292.815429 secs 2024-04-26T20:53:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8292.815429 secs 2024-04-26T20:53:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8292.815429 secs 2024-04-26T20:53:31.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8696 sec 2024-04-26T20:53:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8292.103835 secs 2024-04-26T20:53:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:31 smithi090 ceph-mon[29180]: pgmap v5169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8292.103835 secs 2024-04-26T20:53:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:31 smithi119 ceph-mon[32752]: pgmap v5169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8292.103835 secs 2024-04-26T20:53:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:31 smithi167 ceph-mon[32751]: pgmap v5169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:32.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8697 sec 2024-04-26T20:53:33.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8698 sec 2024-04-26T20:53:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:33 smithi090 ceph-mon[29180]: pgmap v5170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:33 smithi119 ceph-mon[32752]: pgmap v5170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:33 smithi167 ceph-mon[32751]: pgmap v5170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:34.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8699 sec 2024-04-26T20:53:35.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8700 sec 2024-04-26T20:53:36.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:35 smithi167 ceph-mon[32751]: pgmap v5171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:36.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8297.815540 secs 2024-04-26T20:53:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:35 smithi090 ceph-mon[29180]: pgmap v5171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8297.815540 secs 2024-04-26T20:53:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:35 smithi119 ceph-mon[32752]: pgmap v5171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8297.815540 secs 2024-04-26T20:53:36.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8701 sec 2024-04-26T20:53:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8297.103918 secs 2024-04-26T20:53:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:53:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8297.103918 secs 2024-04-26T20:53:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:53:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8297.103918 secs 2024-04-26T20:53:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:53:37.671 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8702 sec 2024-04-26T20:53:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:37 smithi090 ceph-mon[29180]: pgmap v5172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:53:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:53:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:37 smithi090 ceph-mon[29180]: pgmap v5173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:53:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:37 smithi090 ceph-mon[29180]: pgmap v5174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:53:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:53:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:37 smithi119 ceph-mon[32752]: pgmap v5172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:53:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:53:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:37 smithi119 ceph-mon[32752]: pgmap v5173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:53:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:37 smithi119 ceph-mon[32752]: pgmap v5174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:53:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:53:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:37 smithi167 ceph-mon[32751]: pgmap v5172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:53:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:53:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:37 smithi167 ceph-mon[32751]: pgmap v5173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T20:53:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:37 smithi167 ceph-mon[32751]: pgmap v5174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:53:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:53:38.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8703 sec 2024-04-26T20:53:39.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8704 sec 2024-04-26T20:53:40.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8705 sec 2024-04-26T20:53:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:40 smithi119 ceph-mon[32752]: pgmap v5175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:40 smithi167 ceph-mon[32751]: pgmap v5175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:40 smithi090 ceph-mon[29180]: pgmap v5175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:41.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8706 sec 2024-04-26T20:53:41.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8302.103978 secs 2024-04-26T20:53:41.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8302.815658 secs 2024-04-26T20:53:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8302.103978 secs 2024-04-26T20:53:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8302.815658 secs 2024-04-26T20:53:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8302.103978 secs 2024-04-26T20:53:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8302.815658 secs 2024-04-26T20:53:42.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8707 sec 2024-04-26T20:53:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:42 smithi119 ceph-mon[32752]: pgmap v5176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:42 smithi167 ceph-mon[32751]: pgmap v5176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:42 smithi090 ceph-mon[29180]: pgmap v5176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:43.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8708 sec 2024-04-26T20:53:44.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8709 sec 2024-04-26T20:53:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:44 smithi119 ceph-mon[32752]: pgmap v5177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:44 smithi167 ceph-mon[32751]: pgmap v5177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:44 smithi090 ceph-mon[29180]: pgmap v5177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:45.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8710 sec 2024-04-26T20:53:46.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8711 sec 2024-04-26T20:53:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8307.104057 secs 2024-04-26T20:53:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:46 smithi119 ceph-mon[32752]: pgmap v5178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8307.815735 secs 2024-04-26T20:53:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8307.104057 secs 2024-04-26T20:53:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:46 smithi167 ceph-mon[32751]: pgmap v5178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8307.815735 secs 2024-04-26T20:53:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8307.104057 secs 2024-04-26T20:53:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:46 smithi090 ceph-mon[29180]: pgmap v5178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:53:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8307.815735 secs 2024-04-26T20:53:47.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8712 sec 2024-04-26T20:53:48.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8713 sec 2024-04-26T20:53:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:48 smithi119 ceph-mon[32752]: pgmap v5179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:53:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:48 smithi167 ceph-mon[32751]: pgmap v5179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:53:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:48 smithi090 ceph-mon[29180]: pgmap v5179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:53:49.672 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8714 sec 2024-04-26T20:53:50.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8715 sec 2024-04-26T20:53:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:50 smithi119 ceph-mon[32752]: pgmap v5180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:50 smithi167 ceph-mon[32751]: pgmap v5180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:50 smithi090 ceph-mon[29180]: pgmap v5180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:51.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8716 sec 2024-04-26T20:53:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8312.104115 secs 2024-04-26T20:53:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8312.815850 secs 2024-04-26T20:53:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8312.104115 secs 2024-04-26T20:53:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8312.815850 secs 2024-04-26T20:53:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8312.104115 secs 2024-04-26T20:53:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8312.815850 secs 2024-04-26T20:53:52.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8717 sec 2024-04-26T20:53:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:52 smithi119 ceph-mon[32752]: pgmap v5181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:52 smithi167 ceph-mon[32751]: pgmap v5181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:52 smithi090 ceph-mon[29180]: pgmap v5181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:53.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8718 sec 2024-04-26T20:53:54.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8719 sec 2024-04-26T20:53:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:54 smithi119 ceph-mon[32752]: pgmap v5182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:54 smithi167 ceph-mon[32751]: pgmap v5182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:54 smithi090 ceph-mon[29180]: pgmap v5182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:53:55.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8720 sec 2024-04-26T20:53:56.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8721 sec 2024-04-26T20:53:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8317.104197 secs 2024-04-26T20:53:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:56 smithi119 ceph-mon[32752]: pgmap v5183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8317.815923 secs 2024-04-26T20:53:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8317.104197 secs 2024-04-26T20:53:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:56 smithi167 ceph-mon[32751]: pgmap v5183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8317.815923 secs 2024-04-26T20:53:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8317.104197 secs 2024-04-26T20:53:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:56 smithi090 ceph-mon[29180]: pgmap v5183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8317.815923 secs 2024-04-26T20:53:57.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8722 sec 2024-04-26T20:53:58.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8723 sec 2024-04-26T20:53:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:53:58 smithi119 ceph-mon[32752]: pgmap v5184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:53:58 smithi167 ceph-mon[32751]: pgmap v5184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:53:58 smithi090 ceph-mon[29180]: pgmap v5184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:53:59.673 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8724 sec 2024-04-26T20:54:00.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8725 sec 2024-04-26T20:54:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:00 smithi119 ceph-mon[32752]: pgmap v5185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:00 smithi167 ceph-mon[32751]: pgmap v5185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:00 smithi090 ceph-mon[29180]: pgmap v5185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:01.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8726 sec 2024-04-26T20:54:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8322.104283 secs 2024-04-26T20:54:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8322.816060 secs 2024-04-26T20:54:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8322.104283 secs 2024-04-26T20:54:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8322.816060 secs 2024-04-26T20:54:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8322.104283 secs 2024-04-26T20:54:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8322.816060 secs 2024-04-26T20:54:02.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8727 sec 2024-04-26T20:54:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:02 smithi119 ceph-mon[32752]: pgmap v5186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:02 smithi167 ceph-mon[32751]: pgmap v5186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:02 smithi090 ceph-mon[29180]: pgmap v5186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:03.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8728 sec 2024-04-26T20:54:04.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8729 sec 2024-04-26T20:54:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:04 smithi119 ceph-mon[32752]: pgmap v5187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:04 smithi167 ceph-mon[32751]: pgmap v5187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:04 smithi090 ceph-mon[29180]: pgmap v5187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:05.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8730 sec 2024-04-26T20:54:06.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8731 sec 2024-04-26T20:54:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8327.104369 secs 2024-04-26T20:54:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:06 smithi119 ceph-mon[32752]: pgmap v5188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8327.816177 secs 2024-04-26T20:54:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8327.104369 secs 2024-04-26T20:54:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:06 smithi167 ceph-mon[32751]: pgmap v5188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8327.816177 secs 2024-04-26T20:54:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8327.104369 secs 2024-04-26T20:54:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:06 smithi090 ceph-mon[29180]: pgmap v5188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8327.816177 secs 2024-04-26T20:54:07.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8732 sec 2024-04-26T20:54:08.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8733 sec 2024-04-26T20:54:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:08 smithi167 ceph-mon[32751]: pgmap v5189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:08 smithi090 ceph-mon[29180]: pgmap v5189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:08 smithi119 ceph-mon[32752]: pgmap v5189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:09.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8734 sec 2024-04-26T20:54:10.674 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8735 sec 2024-04-26T20:54:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:10 smithi167 ceph-mon[32751]: pgmap v5190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:10 smithi090 ceph-mon[29180]: pgmap v5190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:10 smithi119 ceph-mon[32752]: pgmap v5190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:11.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8736 sec 2024-04-26T20:54:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8332.104454 secs 2024-04-26T20:54:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8332.816313 secs 2024-04-26T20:54:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8332.104454 secs 2024-04-26T20:54:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8332.816313 secs 2024-04-26T20:54:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8332.104454 secs 2024-04-26T20:54:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8332.816313 secs 2024-04-26T20:54:12.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8737 sec 2024-04-26T20:54:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:12 smithi167 ceph-mon[32751]: pgmap v5191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:12 smithi090 ceph-mon[29180]: pgmap v5191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:12 smithi119 ceph-mon[32752]: pgmap v5191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:13.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8738 sec 2024-04-26T20:54:14.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8739 sec 2024-04-26T20:54:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:14 smithi090 ceph-mon[29180]: pgmap v5192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:14 smithi119 ceph-mon[32752]: pgmap v5192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:14 smithi167 ceph-mon[32751]: pgmap v5192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:15.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8740 sec 2024-04-26T20:54:16.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8741 sec 2024-04-26T20:54:16.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8337.104538 secs 2024-04-26T20:54:16.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:16 smithi119 ceph-mon[32752]: pgmap v5193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:16.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8337.816405 secs 2024-04-26T20:54:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8337.104538 secs 2024-04-26T20:54:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:16 smithi090 ceph-mon[29180]: pgmap v5193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8337.816405 secs 2024-04-26T20:54:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8337.104538 secs 2024-04-26T20:54:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:16 smithi167 ceph-mon[32751]: pgmap v5193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8337.816405 secs 2024-04-26T20:54:17.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8742 sec 2024-04-26T20:54:18.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8743 sec 2024-04-26T20:54:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:18 smithi090 ceph-mon[29180]: pgmap v5194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:18 smithi119 ceph-mon[32752]: pgmap v5194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:18 smithi167 ceph-mon[32751]: pgmap v5194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:19.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8744 sec 2024-04-26T20:54:20.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8745 sec 2024-04-26T20:54:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:20 smithi090 ceph-mon[29180]: pgmap v5195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:20 smithi119 ceph-mon[32752]: pgmap v5195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:20 smithi167 ceph-mon[32751]: pgmap v5195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:21.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8746 sec 2024-04-26T20:54:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8342.104621 secs 2024-04-26T20:54:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8342.816558 secs 2024-04-26T20:54:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8342.104621 secs 2024-04-26T20:54:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8342.816558 secs 2024-04-26T20:54:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8342.104621 secs 2024-04-26T20:54:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8342.816558 secs 2024-04-26T20:54:22.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8747 sec 2024-04-26T20:54:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:22 smithi090 ceph-mon[29180]: pgmap v5196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:22 smithi119 ceph-mon[32752]: pgmap v5196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:22 smithi167 ceph-mon[32751]: pgmap v5196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:23.675 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8748 sec 2024-04-26T20:54:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:23 smithi090 ceph-mon[29180]: pgmap v5197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:23 smithi119 ceph-mon[32752]: pgmap v5197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:23 smithi167 ceph-mon[32751]: pgmap v5197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:24.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8749 sec 2024-04-26T20:54:25.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8750 sec 2024-04-26T20:54:26.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8751 sec 2024-04-26T20:54:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8347.104701 secs 2024-04-26T20:54:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:26 smithi119 ceph-mon[32752]: pgmap v5198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8347.816679 secs 2024-04-26T20:54:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8347.104701 secs 2024-04-26T20:54:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:26 smithi167 ceph-mon[32751]: pgmap v5198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8347.816679 secs 2024-04-26T20:54:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8347.104701 secs 2024-04-26T20:54:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:26 smithi090 ceph-mon[29180]: pgmap v5198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8347.816679 secs 2024-04-26T20:54:27.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8752 sec 2024-04-26T20:54:28.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8753 sec 2024-04-26T20:54:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:28 smithi119 ceph-mon[32752]: pgmap v5199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:28 smithi167 ceph-mon[32751]: pgmap v5199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:28 smithi090 ceph-mon[29180]: pgmap v5199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:29.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8754 sec 2024-04-26T20:54:30.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8755 sec 2024-04-26T20:54:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:30 smithi119 ceph-mon[32752]: pgmap v5200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:30 smithi167 ceph-mon[32751]: pgmap v5200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:30 smithi090 ceph-mon[29180]: pgmap v5200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:31.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8756 sec 2024-04-26T20:54:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8352.104781 secs 2024-04-26T20:54:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8352.816816 secs 2024-04-26T20:54:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8352.104781 secs 2024-04-26T20:54:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8352.816816 secs 2024-04-26T20:54:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8352.104781 secs 2024-04-26T20:54:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8352.816816 secs 2024-04-26T20:54:32.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8757 sec 2024-04-26T20:54:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:32 smithi119 ceph-mon[32752]: pgmap v5201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:32 smithi167 ceph-mon[32751]: pgmap v5201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:32 smithi090 ceph-mon[29180]: pgmap v5201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:33.676 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8758 sec 2024-04-26T20:54:34.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8759 sec 2024-04-26T20:54:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:34 smithi119 ceph-mon[32752]: pgmap v5202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:54:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:34 smithi167 ceph-mon[32751]: pgmap v5202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:54:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:34 smithi090 ceph-mon[29180]: pgmap v5202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T20:54:35.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8760 sec 2024-04-26T20:54:36.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8761 sec 2024-04-26T20:54:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8357.104866 secs 2024-04-26T20:54:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:36 smithi119 ceph-mon[32752]: pgmap v5203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:54:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8357.816936 secs 2024-04-26T20:54:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8357.104866 secs 2024-04-26T20:54:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:36 smithi167 ceph-mon[32751]: pgmap v5203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:54:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8357.816936 secs 2024-04-26T20:54:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8357.104866 secs 2024-04-26T20:54:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:36 smithi090 ceph-mon[29180]: pgmap v5203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:54:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8357.816936 secs 2024-04-26T20:54:37.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8762 sec 2024-04-26T20:54:38.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8763 sec 2024-04-26T20:54:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:38 smithi119 ceph-mon[32752]: pgmap v5204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:54:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:54:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:54:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:54:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:54:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:38 smithi167 ceph-mon[32751]: pgmap v5204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:54:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:54:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:54:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:54:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:54:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:38 smithi090 ceph-mon[29180]: pgmap v5204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T20:54:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:54:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:54:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:54:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:54:39.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8764 sec 2024-04-26T20:54:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:39 smithi119 ceph-mon[32752]: pgmap v5205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 190 B/s wr, 2 op/s 2024-04-26T20:54:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:39 smithi119 ceph-mon[32752]: pgmap v5206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-26T20:54:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:39 smithi167 ceph-mon[32751]: pgmap v5205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 190 B/s wr, 2 op/s 2024-04-26T20:54:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:39 smithi167 ceph-mon[32751]: pgmap v5206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-26T20:54:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:39 smithi090 ceph-mon[29180]: pgmap v5205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 190 B/s wr, 2 op/s 2024-04-26T20:54:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:39 smithi090 ceph-mon[29180]: pgmap v5206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-26T20:54:40.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8765 sec 2024-04-26T20:54:41.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8766 sec 2024-04-26T20:54:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8362.104925 secs 2024-04-26T20:54:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:41 smithi119 ceph-mon[32752]: pgmap v5207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T20:54:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8362.817054 secs 2024-04-26T20:54:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8362.104925 secs 2024-04-26T20:54:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:41 smithi167 ceph-mon[32751]: pgmap v5207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T20:54:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8362.817054 secs 2024-04-26T20:54:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8362.104925 secs 2024-04-26T20:54:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:41 smithi090 ceph-mon[29180]: pgmap v5207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T20:54:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8362.817054 secs 2024-04-26T20:54:42.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8767 sec 2024-04-26T20:54:43.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8768 sec 2024-04-26T20:54:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:43 smithi119 ceph-mon[32752]: pgmap v5208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:54:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:43 smithi167 ceph-mon[32751]: pgmap v5208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:54:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:43 smithi090 ceph-mon[29180]: pgmap v5208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:54:44.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8769 sec 2024-04-26T20:54:45.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8770 sec 2024-04-26T20:54:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:45 smithi119 ceph-mon[32752]: pgmap v5209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:54:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:45 smithi167 ceph-mon[32751]: pgmap v5209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:54:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:45 smithi090 ceph-mon[29180]: pgmap v5209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:54:46.677 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8771 sec 2024-04-26T20:54:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8367.104987 secs 2024-04-26T20:54:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8367.817179 secs 2024-04-26T20:54:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8367.104987 secs 2024-04-26T20:54:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8367.817179 secs 2024-04-26T20:54:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8367.104987 secs 2024-04-26T20:54:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8367.817179 secs 2024-04-26T20:54:47.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8772 sec 2024-04-26T20:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:47 smithi167 ceph-mon[32751]: pgmap v5210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:54:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:47 smithi090 ceph-mon[29180]: pgmap v5210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:54:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:47 smithi119 ceph-mon[32752]: pgmap v5210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:54:48.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8773 sec 2024-04-26T20:54:49.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8774 sec 2024-04-26T20:54:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:49 smithi167 ceph-mon[32751]: pgmap v5211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:54:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:49 smithi090 ceph-mon[29180]: pgmap v5211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:54:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:49 smithi119 ceph-mon[32752]: pgmap v5211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:54:50.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8775 sec 2024-04-26T20:54:51.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8776 sec 2024-04-26T20:54:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8372.105067 secs 2024-04-26T20:54:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:51 smithi167 ceph-mon[32751]: pgmap v5212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8372.817246 secs 2024-04-26T20:54:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8372.105067 secs 2024-04-26T20:54:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:51 smithi090 ceph-mon[29180]: pgmap v5212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8372.817246 secs 2024-04-26T20:54:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8372.105067 secs 2024-04-26T20:54:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:51 smithi119 ceph-mon[32752]: pgmap v5212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8372.817246 secs 2024-04-26T20:54:52.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8777 sec 2024-04-26T20:54:53.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8778 sec 2024-04-26T20:54:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:53 smithi090 ceph-mon[29180]: pgmap v5213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:53 smithi119 ceph-mon[32752]: pgmap v5213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:53 smithi167 ceph-mon[32751]: pgmap v5213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:54.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8779 sec 2024-04-26T20:54:55.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8780 sec 2024-04-26T20:54:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:55 smithi090 ceph-mon[29180]: pgmap v5214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:55 smithi119 ceph-mon[32752]: pgmap v5214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:55 smithi167 ceph-mon[32751]: pgmap v5214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:54:56.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8781 sec 2024-04-26T20:54:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8377.105151 secs 2024-04-26T20:54:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8377.817366 secs 2024-04-26T20:54:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8377.105151 secs 2024-04-26T20:54:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8377.817366 secs 2024-04-26T20:54:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8377.105151 secs 2024-04-26T20:54:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8377.817366 secs 2024-04-26T20:54:57.678 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8782 sec 2024-04-26T20:54:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:57 smithi090 ceph-mon[29180]: pgmap v5215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:57 smithi119 ceph-mon[32752]: pgmap v5215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:57 smithi167 ceph-mon[32751]: pgmap v5215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:54:58.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8783 sec 2024-04-26T20:54:59.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8784 sec 2024-04-26T20:54:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:54:59 smithi090 ceph-mon[29180]: pgmap v5216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:54:59 smithi119 ceph-mon[32752]: pgmap v5216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:54:59 smithi167 ceph-mon[32751]: pgmap v5216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:00.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8785 sec 2024-04-26T20:55:01.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8786 sec 2024-04-26T20:55:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 8382.105235 secs 2024-04-26T20:55:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:01 smithi090 ceph-mon[29180]: pgmap v5217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8382.817484 secs 2024-04-26T20:55:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 8382.105235 secs 2024-04-26T20:55:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:01 smithi119 ceph-mon[32752]: pgmap v5217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8382.817484 secs 2024-04-26T20:55:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 8382.105235 secs 2024-04-26T20:55:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:01 smithi167 ceph-mon[32751]: pgmap v5217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8382.817484 secs 2024-04-26T20:55:02.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8787 sec 2024-04-26T20:55:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:55:02.981+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:55:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:55:02.981+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:55:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:55:02.981+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:55:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:55:02.981+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:55:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 20:55:02 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T20:55:02.981+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T20:55:03.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8788 sec 2024-04-26T20:55:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:03 smithi090 ceph-mon[29180]: pgmap v5218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:03 smithi119 ceph-mon[32752]: pgmap v5218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:03 smithi167 ceph-mon[32751]: pgmap v5218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:04.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8789 sec 2024-04-26T20:55:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:04 smithi090 ceph-mon[29180]: pgmap v5219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:04 smithi119 ceph-mon[32752]: pgmap v5219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:04 smithi167 ceph-mon[32751]: pgmap v5219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:05.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8790 sec 2024-04-26T20:55:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8387.817601 secs 2024-04-26T20:55:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8387.817601 secs 2024-04-26T20:55:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8387.817601 secs 2024-04-26T20:55:06.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8791 sec 2024-04-26T20:55:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:06 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8387.105320 secs 2024-04-26T20:55:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:06 smithi090 ceph-mon[29180]: pgmap v5220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:06.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:06 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8387.105320 secs 2024-04-26T20:55:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:06 smithi167 ceph-mon[32751]: pgmap v5220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:06 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8387.105320 secs 2024-04-26T20:55:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:06 smithi119 ceph-mon[32752]: pgmap v5220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:07.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8792 sec 2024-04-26T20:55:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:55:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:55:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:55:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:55:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:55:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:55:08.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8793 sec 2024-04-26T20:55:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:08 smithi090 ceph-mon[29180]: pgmap v5221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:08 smithi119 ceph-mon[32752]: pgmap v5221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:08 smithi167 ceph-mon[32751]: pgmap v5221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:09.679 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8794 sec 2024-04-26T20:55:10.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8795 sec 2024-04-26T20:55:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:11 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8392.105406 secs 2024-04-26T20:55:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:11 smithi119 ceph-mon[32752]: pgmap v5222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8392.817667 secs 2024-04-26T20:55:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:11 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8392.105406 secs 2024-04-26T20:55:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:11 smithi167 ceph-mon[32751]: pgmap v5222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8392.817667 secs 2024-04-26T20:55:11.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:11 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8392.105406 secs 2024-04-26T20:55:11.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:11 smithi090 ceph-mon[29180]: pgmap v5222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:11.580 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8392.817667 secs 2024-04-26T20:55:11.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8796 sec 2024-04-26T20:55:12.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8797 sec 2024-04-26T20:55:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:13 smithi119 ceph-mon[32752]: pgmap v5223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:13 smithi167 ceph-mon[32751]: pgmap v5223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:13 smithi090 ceph-mon[29180]: pgmap v5223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:13.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8798 sec 2024-04-26T20:55:14.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8799 sec 2024-04-26T20:55:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:15 smithi119 ceph-mon[32752]: pgmap v5224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:15 smithi167 ceph-mon[32751]: pgmap v5224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:15 smithi090 ceph-mon[29180]: pgmap v5224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T20:55:15.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8800 sec 2024-04-26T20:55:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:16 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8397.105490 secs 2024-04-26T20:55:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8397.817785 secs 2024-04-26T20:55:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:16 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8397.105490 secs 2024-04-26T20:55:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8397.817785 secs 2024-04-26T20:55:16.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8801 sec 2024-04-26T20:55:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:16 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8397.105490 secs 2024-04-26T20:55:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8397.817785 secs 2024-04-26T20:55:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:17 smithi167 ceph-mon[32751]: pgmap v5225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:17 smithi090 ceph-mon[29180]: pgmap v5225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:17.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8802 sec 2024-04-26T20:55:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:17 smithi119 ceph-mon[32752]: pgmap v5225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:18.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8803 sec 2024-04-26T20:55:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:19 smithi167 ceph-mon[32751]: pgmap v5226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:19 smithi090 ceph-mon[29180]: pgmap v5226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:19.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8804 sec 2024-04-26T20:55:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:19 smithi119 ceph-mon[32752]: pgmap v5226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:20.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8805 sec 2024-04-26T20:55:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:21 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8402.105578 secs 2024-04-26T20:55:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:21 smithi167 ceph-mon[32751]: pgmap v5227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8402.817884 secs 2024-04-26T20:55:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:21 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8402.105578 secs 2024-04-26T20:55:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:21 smithi090 ceph-mon[29180]: pgmap v5227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8402.817884 secs 2024-04-26T20:55:21.680 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8806 sec 2024-04-26T20:55:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:21 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8402.105578 secs 2024-04-26T20:55:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:21 smithi119 ceph-mon[32752]: pgmap v5227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8402.817884 secs 2024-04-26T20:55:22.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8807 sec 2024-04-26T20:55:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:23 smithi167 ceph-mon[32751]: pgmap v5228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:23 smithi090 ceph-mon[29180]: pgmap v5228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:23.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8808 sec 2024-04-26T20:55:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:23 smithi119 ceph-mon[32752]: pgmap v5228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:24.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8809 sec 2024-04-26T20:55:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:25 smithi090 ceph-mon[29180]: pgmap v5229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8407.105663 secs 2024-04-26T20:55:25.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8810 sec 2024-04-26T20:55:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:25 smithi119 ceph-mon[32752]: pgmap v5229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8407.105663 secs 2024-04-26T20:55:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:25 smithi167 ceph-mon[32751]: pgmap v5229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8407.105663 secs 2024-04-26T20:55:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8407.818024 secs 2024-04-26T20:55:26.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8811 sec 2024-04-26T20:55:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8407.818024 secs 2024-04-26T20:55:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8407.818024 secs 2024-04-26T20:55:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:27 smithi090 ceph-mon[29180]: pgmap v5230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:27.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8812 sec 2024-04-26T20:55:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:27 smithi119 ceph-mon[32752]: pgmap v5230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:27 smithi167 ceph-mon[32751]: pgmap v5230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:28.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8813 sec 2024-04-26T20:55:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:29 smithi090 ceph-mon[29180]: pgmap v5231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:29.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8814 sec 2024-04-26T20:55:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:29 smithi119 ceph-mon[32752]: pgmap v5231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:29 smithi167 ceph-mon[32751]: pgmap v5231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:30.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8815 sec 2024-04-26T20:55:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8412.105749 secs 2024-04-26T20:55:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:30 smithi090 ceph-mon[29180]: pgmap v5232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8412.818157 secs 2024-04-26T20:55:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8412.105749 secs 2024-04-26T20:55:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:30 smithi119 ceph-mon[32752]: pgmap v5232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8412.818157 secs 2024-04-26T20:55:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8412.105749 secs 2024-04-26T20:55:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:30 smithi167 ceph-mon[32751]: pgmap v5232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8412.818157 secs 2024-04-26T20:55:31.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8816 sec 2024-04-26T20:55:32.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8817 sec 2024-04-26T20:55:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:33 smithi119 ceph-mon[32752]: pgmap v5233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:33 smithi167 ceph-mon[32751]: pgmap v5233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:33 smithi090 ceph-mon[29180]: pgmap v5233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:33.681 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8818 sec 2024-04-26T20:55:34.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8819 sec 2024-04-26T20:55:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:35 smithi167 ceph-mon[32751]: pgmap v5234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8417.105829 secs 2024-04-26T20:55:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:35 smithi090 ceph-mon[29180]: pgmap v5234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8417.105829 secs 2024-04-26T20:55:35.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8820 sec 2024-04-26T20:55:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:35 smithi119 ceph-mon[32752]: pgmap v5234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8417.105829 secs 2024-04-26T20:55:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8417.818272 secs 2024-04-26T20:55:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8417.818272 secs 2024-04-26T20:55:36.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8821 sec 2024-04-26T20:55:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8417.818272 secs 2024-04-26T20:55:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:37 smithi167 ceph-mon[32751]: pgmap v5235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:37 smithi090 ceph-mon[29180]: pgmap v5235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:37.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8822 sec 2024-04-26T20:55:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:37 smithi119 ceph-mon[32752]: pgmap v5235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:55:38.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8823 sec 2024-04-26T20:55:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:39 smithi167 ceph-mon[32751]: pgmap v5236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:55:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:39 smithi090 ceph-mon[29180]: pgmap v5236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:39.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:55:39.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.676 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:39 smithi119 ceph-mon[32752]: pgmap v5236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:39.676 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:55:39.676 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.677 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.677 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.677 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:39.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8824 sec 2024-04-26T20:55:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8422.105914 secs 2024-04-26T20:55:40.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8825 sec 2024-04-26T20:55:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8422.105914 secs 2024-04-26T20:55:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8422.105914 secs 2024-04-26T20:55:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: pgmap v5237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8422.818336 secs 2024-04-26T20:55:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8826 sec 2024-04-26T20:55:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: pgmap v5237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8422.818336 secs 2024-04-26T20:55:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: pgmap v5237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8422.818336 secs 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:42.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8827 sec 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:55:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:43 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:55:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:43 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:55:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:43 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:55:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:43 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:55:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:43 smithi090 ceph-mon[29180]: pgmap v5238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T20:55:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:43 smithi090 ceph-mon[29180]: pgmap v5239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:43.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8828 sec 2024-04-26T20:55:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:43 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:55:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:43 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:55:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:43 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:55:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:43 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:55:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:43 smithi119 ceph-mon[32752]: pgmap v5238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T20:55:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:43 smithi119 ceph-mon[32752]: pgmap v5239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:43 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T20:55:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:43 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T20:55:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:43 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T20:55:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:43 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T20:55:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:43 smithi167 ceph-mon[32751]: pgmap v5238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T20:55:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:43 smithi167 ceph-mon[32751]: pgmap v5239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:44.592 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:55:44.591+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006530 con 0x7f2cd4104500 2024-04-26T20:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:55:44.591+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:55:44.591+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 68 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8006530 con 0x7f2cd4104500 2024-04-26T20:55:44.593 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:55:44.591+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 68 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:55:44.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8829 sec 2024-04-26T20:55:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:45 smithi090 ceph-mon[29180]: pgmap v5240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8427.105974 secs 2024-04-26T20:55:45.682 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8830 sec 2024-04-26T20:55:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:45 smithi119 ceph-mon[32752]: pgmap v5240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8427.105974 secs 2024-04-26T20:55:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:45 smithi167 ceph-mon[32751]: pgmap v5240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8427.105974 secs 2024-04-26T20:55:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8427.818486 secs 2024-04-26T20:55:46.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8831 sec 2024-04-26T20:55:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8427.818486 secs 2024-04-26T20:55:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8427.818486 secs 2024-04-26T20:55:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:47 smithi090 ceph-mon[29180]: pgmap v5241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:47.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8832 sec 2024-04-26T20:55:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:47 smithi119 ceph-mon[32752]: pgmap v5241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:47 smithi167 ceph-mon[32751]: pgmap v5241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:55:48.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8833 sec 2024-04-26T20:55:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:49 smithi090 ceph-mon[29180]: pgmap v5242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-26T20:55:49.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8834 sec 2024-04-26T20:55:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:49 smithi119 ceph-mon[32752]: pgmap v5242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-26T20:55:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:49 smithi167 ceph-mon[32751]: pgmap v5242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-26T20:55:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8432.106036 secs 2024-04-26T20:55:50.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8835 sec 2024-04-26T20:55:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8432.106036 secs 2024-04-26T20:55:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8432.106036 secs 2024-04-26T20:55:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:51 smithi090 ceph-mon[29180]: pgmap v5243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 B/s rd, 1 op/s 2024-04-26T20:55:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8432.818569 secs 2024-04-26T20:55:51.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8836 sec 2024-04-26T20:55:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:51 smithi119 ceph-mon[32752]: pgmap v5243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 B/s rd, 1 op/s 2024-04-26T20:55:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8432.818569 secs 2024-04-26T20:55:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:51 smithi167 ceph-mon[32751]: pgmap v5243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 972 B/s rd, 1 op/s 2024-04-26T20:55:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8432.818569 secs 2024-04-26T20:55:52.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8837 sec 2024-04-26T20:55:53.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:53 smithi090 ceph-mon[29180]: pgmap v5244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T20:55:53.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8838 sec 2024-04-26T20:55:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:53 smithi119 ceph-mon[32752]: pgmap v5244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T20:55:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:53 smithi167 ceph-mon[32751]: pgmap v5244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T20:55:54.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8839 sec 2024-04-26T20:55:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:55 smithi090 ceph-mon[29180]: pgmap v5245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8437.106097 secs 2024-04-26T20:55:55.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8840 sec 2024-04-26T20:55:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:55 smithi119 ceph-mon[32752]: pgmap v5245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8437.106097 secs 2024-04-26T20:55:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:55 smithi167 ceph-mon[32751]: pgmap v5245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:55:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8437.106097 secs 2024-04-26T20:55:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8437.818696 secs 2024-04-26T20:55:56.683 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8841 sec 2024-04-26T20:55:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8437.818696 secs 2024-04-26T20:55:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8437.818696 secs 2024-04-26T20:55:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:57 smithi090 ceph-mon[29180]: pgmap v5246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:55:57.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8842 sec 2024-04-26T20:55:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:57 smithi119 ceph-mon[32752]: pgmap v5246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:55:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:57 smithi167 ceph-mon[32751]: pgmap v5246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:55:58.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8843 sec 2024-04-26T20:55:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:55:59 smithi090 ceph-mon[29180]: pgmap v5247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:55:59.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8844 sec 2024-04-26T20:55:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:55:59 smithi119 ceph-mon[32752]: pgmap v5247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:55:59.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:55:59 smithi167 ceph-mon[32751]: pgmap v5247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T20:56:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8442.106158 secs 2024-04-26T20:56:00.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8845 sec 2024-04-26T20:56:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8442.106158 secs 2024-04-26T20:56:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8442.106158 secs 2024-04-26T20:56:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:01 smithi090 ceph-mon[29180]: pgmap v5248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8442.818844 secs 2024-04-26T20:56:01.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8846 sec 2024-04-26T20:56:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:01 smithi119 ceph-mon[32752]: pgmap v5248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8442.818844 secs 2024-04-26T20:56:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:01 smithi167 ceph-mon[32751]: pgmap v5248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8442.818844 secs 2024-04-26T20:56:02.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8847 sec 2024-04-26T20:56:03.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8848 sec 2024-04-26T20:56:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:03 smithi119 ceph-mon[32752]: pgmap v5249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:03 smithi167 ceph-mon[32751]: pgmap v5249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:03 smithi090 ceph-mon[29180]: pgmap v5249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:04.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8849 sec 2024-04-26T20:56:05.684 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8850 sec 2024-04-26T20:56:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:05 smithi119 ceph-mon[32752]: pgmap v5250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8447.106216 secs 2024-04-26T20:56:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:05 smithi167 ceph-mon[32751]: pgmap v5250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8447.106216 secs 2024-04-26T20:56:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:05 smithi090 ceph-mon[29180]: pgmap v5250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8447.106216 secs 2024-04-26T20:56:06.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8851 sec 2024-04-26T20:56:06.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8447.818987 secs 2024-04-26T20:56:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8447.818987 secs 2024-04-26T20:56:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8447.818987 secs 2024-04-26T20:56:07.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8852 sec 2024-04-26T20:56:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:07 smithi119 ceph-mon[32752]: pgmap v5251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:07 smithi167 ceph-mon[32751]: pgmap v5251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:07 smithi090 ceph-mon[29180]: pgmap v5251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:08.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8853 sec 2024-04-26T20:56:09.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8854 sec 2024-04-26T20:56:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:09 smithi119 ceph-mon[32752]: pgmap v5252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:09 smithi167 ceph-mon[32751]: pgmap v5252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:09 smithi090 ceph-mon[29180]: pgmap v5252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:10.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8855 sec 2024-04-26T20:56:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8452.106273 secs 2024-04-26T20:56:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8452.106273 secs 2024-04-26T20:56:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8452.106273 secs 2024-04-26T20:56:11.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8856 sec 2024-04-26T20:56:11.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:11 smithi090 ceph-mon[29180]: pgmap v5253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:11.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8452.819053 secs 2024-04-26T20:56:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:11 smithi119 ceph-mon[32752]: pgmap v5253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8452.819053 secs 2024-04-26T20:56:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:11 smithi167 ceph-mon[32751]: pgmap v5253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8452.819053 secs 2024-04-26T20:56:12.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8857 sec 2024-04-26T20:56:13.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8858 sec 2024-04-26T20:56:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:13 smithi119 ceph-mon[32752]: pgmap v5254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:13 smithi167 ceph-mon[32751]: pgmap v5254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:13 smithi090 ceph-mon[29180]: pgmap v5254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:14.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8859 sec 2024-04-26T20:56:15.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8860 sec 2024-04-26T20:56:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:15 smithi119 ceph-mon[32752]: pgmap v5255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8457.106331 secs 2024-04-26T20:56:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:15 smithi167 ceph-mon[32751]: pgmap v5255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8457.106331 secs 2024-04-26T20:56:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:15 smithi090 ceph-mon[29180]: pgmap v5255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8457.106331 secs 2024-04-26T20:56:16.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8861 sec 2024-04-26T20:56:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8457.819196 secs 2024-04-26T20:56:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8457.819196 secs 2024-04-26T20:56:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8457.819196 secs 2024-04-26T20:56:17.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8862 sec 2024-04-26T20:56:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:17 smithi119 ceph-mon[32752]: pgmap v5256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:17 smithi167 ceph-mon[32751]: pgmap v5256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:17 smithi090 ceph-mon[29180]: pgmap v5256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:18.685 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8863 sec 2024-04-26T20:56:19.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8864 sec 2024-04-26T20:56:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:19 smithi119 ceph-mon[32752]: pgmap v5257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:19 smithi167 ceph-mon[32751]: pgmap v5257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:19 smithi090 ceph-mon[29180]: pgmap v5257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:20.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8865 sec 2024-04-26T20:56:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8462.106390 secs 2024-04-26T20:56:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8462.106390 secs 2024-04-26T20:56:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8462.106390 secs 2024-04-26T20:56:21.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8866 sec 2024-04-26T20:56:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:21 smithi119 ceph-mon[32752]: pgmap v5258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8462.819324 secs 2024-04-26T20:56:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:21 smithi167 ceph-mon[32751]: pgmap v5258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8462.819324 secs 2024-04-26T20:56:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:21 smithi090 ceph-mon[29180]: pgmap v5258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8462.819324 secs 2024-04-26T20:56:22.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8867 sec 2024-04-26T20:56:23.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8868 sec 2024-04-26T20:56:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:23 smithi119 ceph-mon[32752]: pgmap v5259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:23 smithi167 ceph-mon[32751]: pgmap v5259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:23 smithi090 ceph-mon[29180]: pgmap v5259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:24.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8869 sec 2024-04-26T20:56:25.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8870 sec 2024-04-26T20:56:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:25 smithi119 ceph-mon[32752]: pgmap v5260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8467.106450 secs 2024-04-26T20:56:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:25 smithi167 ceph-mon[32751]: pgmap v5260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8467.106450 secs 2024-04-26T20:56:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:25 smithi090 ceph-mon[29180]: pgmap v5260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8467.106450 secs 2024-04-26T20:56:26.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8871 sec 2024-04-26T20:56:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8467.819447 secs 2024-04-26T20:56:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8467.819447 secs 2024-04-26T20:56:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8467.819447 secs 2024-04-26T20:56:27.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8872 sec 2024-04-26T20:56:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:27 smithi119 ceph-mon[32752]: pgmap v5261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:27 smithi167 ceph-mon[32751]: pgmap v5261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:27 smithi090 ceph-mon[29180]: pgmap v5261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:28.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8873 sec 2024-04-26T20:56:29.598 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:56:29.598+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T20:56:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:56:29.598+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:56:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:56:29.598+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 69 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T20:56:29.599 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:56:29.598+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 69 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:56:29.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8874 sec 2024-04-26T20:56:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:29 smithi119 ceph-mon[32752]: pgmap v5262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:29 smithi167 ceph-mon[32751]: pgmap v5262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:29 smithi090 ceph-mon[29180]: pgmap v5262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:30.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8875 sec 2024-04-26T20:56:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8472.106502 secs 2024-04-26T20:56:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8472.106502 secs 2024-04-26T20:56:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8472.106502 secs 2024-04-26T20:56:31.686 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8876 sec 2024-04-26T20:56:31.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:31 smithi090 ceph-mon[29180]: pgmap v5263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:31.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8472.819564 secs 2024-04-26T20:56:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:31 smithi119 ceph-mon[32752]: pgmap v5263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8472.819564 secs 2024-04-26T20:56:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:31 smithi167 ceph-mon[32751]: pgmap v5263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8472.819564 secs 2024-04-26T20:56:32.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8877 sec 2024-04-26T20:56:33.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8878 sec 2024-04-26T20:56:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:33 smithi119 ceph-mon[32752]: pgmap v5264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:33 smithi167 ceph-mon[32751]: pgmap v5264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:33 smithi090 ceph-mon[29180]: pgmap v5264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:34.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8879 sec 2024-04-26T20:56:35.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8880 sec 2024-04-26T20:56:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:35 smithi119 ceph-mon[32752]: pgmap v5265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8477.106564 secs 2024-04-26T20:56:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:35 smithi167 ceph-mon[32751]: pgmap v5265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8477.106564 secs 2024-04-26T20:56:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:35 smithi090 ceph-mon[29180]: pgmap v5265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8477.106564 secs 2024-04-26T20:56:36.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8881 sec 2024-04-26T20:56:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8477.819638 secs 2024-04-26T20:56:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8477.819638 secs 2024-04-26T20:56:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8477.819638 secs 2024-04-26T20:56:37.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8882 sec 2024-04-26T20:56:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:37 smithi119 ceph-mon[32752]: pgmap v5266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:37 smithi167 ceph-mon[32751]: pgmap v5266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:37 smithi090 ceph-mon[29180]: pgmap v5266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:38.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8883 sec 2024-04-26T20:56:39.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8884 sec 2024-04-26T20:56:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:39 smithi119 ceph-mon[32752]: pgmap v5267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:39 smithi167 ceph-mon[32751]: pgmap v5267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:39 smithi090 ceph-mon[29180]: pgmap v5267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:40.687 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8885 sec 2024-04-26T20:56:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:40 smithi119 ceph-mon[32752]: pgmap v5268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:40 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8482.106626 secs 2024-04-26T20:56:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8482.819761 secs 2024-04-26T20:56:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:40 smithi167 ceph-mon[32751]: pgmap v5268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:40 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8482.106626 secs 2024-04-26T20:56:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8482.819761 secs 2024-04-26T20:56:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:40 smithi090 ceph-mon[29180]: pgmap v5268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:40 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8482.106626 secs 2024-04-26T20:56:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8482.819761 secs 2024-04-26T20:56:41.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8886 sec 2024-04-26T20:56:42.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8887 sec 2024-04-26T20:56:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:42 smithi119 ceph-mon[32752]: pgmap v5269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:56:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:42 smithi167 ceph-mon[32751]: pgmap v5269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:56:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:42 smithi090 ceph-mon[29180]: pgmap v5269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:56:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:43.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8888 sec 2024-04-26T20:56:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:56:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:56:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:43 smithi119 ceph-mon[32752]: pgmap v5270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T20:56:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:43 smithi119 ceph-mon[32752]: pgmap v5271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T20:56:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:56:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:56:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:43 smithi167 ceph-mon[32751]: pgmap v5270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T20:56:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:43 smithi167 ceph-mon[32751]: pgmap v5271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T20:56:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:44.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:56:44.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:56:44.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:43 smithi090 ceph-mon[29180]: pgmap v5270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T20:56:44.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:43 smithi090 ceph-mon[29180]: pgmap v5271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T20:56:44.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:56:44.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8889 sec 2024-04-26T20:56:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:44 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8487.106685 secs 2024-04-26T20:56:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:44 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8487.106685 secs 2024-04-26T20:56:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:44 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8487.106685 secs 2024-04-26T20:56:45.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8890 sec 2024-04-26T20:56:46.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:45 smithi119 ceph-mon[32752]: pgmap v5272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:46.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8487.819840 secs 2024-04-26T20:56:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:45 smithi167 ceph-mon[32751]: pgmap v5272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8487.819840 secs 2024-04-26T20:56:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:45 smithi090 ceph-mon[29180]: pgmap v5272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8487.819840 secs 2024-04-26T20:56:46.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8891 sec 2024-04-26T20:56:47.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8892 sec 2024-04-26T20:56:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:48 smithi119 ceph-mon[32752]: pgmap v5273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:48 smithi167 ceph-mon[32751]: pgmap v5273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:48 smithi090 ceph-mon[29180]: pgmap v5273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:48.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8893 sec 2024-04-26T20:56:49.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8894 sec 2024-04-26T20:56:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:50 smithi119 ceph-mon[32752]: pgmap v5274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8492.106744 secs 2024-04-26T20:56:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:50 smithi167 ceph-mon[32751]: pgmap v5274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8492.106744 secs 2024-04-26T20:56:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:50 smithi090 ceph-mon[29180]: pgmap v5274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8492.106744 secs 2024-04-26T20:56:50.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8895 sec 2024-04-26T20:56:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8492.819948 secs 2024-04-26T20:56:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8492.819948 secs 2024-04-26T20:56:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8492.819948 secs 2024-04-26T20:56:51.688 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8896 sec 2024-04-26T20:56:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:52 smithi167 ceph-mon[32751]: pgmap v5275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:52 smithi090 ceph-mon[29180]: pgmap v5275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:52 smithi119 ceph-mon[32752]: pgmap v5275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:56:52.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8897 sec 2024-04-26T20:56:53.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8898 sec 2024-04-26T20:56:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:54 smithi167 ceph-mon[32751]: pgmap v5276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:56:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:54 smithi090 ceph-mon[29180]: pgmap v5276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:56:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:54 smithi119 ceph-mon[32752]: pgmap v5276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:56:54.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8899 sec 2024-04-26T20:56:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8497.106795 secs 2024-04-26T20:56:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8497.106795 secs 2024-04-26T20:56:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8497.106795 secs 2024-04-26T20:56:55.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8900 sec 2024-04-26T20:56:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:56 smithi090 ceph-mon[29180]: pgmap v5277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8497.820086 secs 2024-04-26T20:56:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:56 smithi119 ceph-mon[32752]: pgmap v5277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8497.820086 secs 2024-04-26T20:56:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:56 smithi167 ceph-mon[32751]: pgmap v5277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:56:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8497.820086 secs 2024-04-26T20:56:56.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8901 sec 2024-04-26T20:56:57.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8902 sec 2024-04-26T20:56:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:56:58 smithi090 ceph-mon[29180]: pgmap v5278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:56:58 smithi119 ceph-mon[32752]: pgmap v5278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:56:58 smithi167 ceph-mon[32751]: pgmap v5278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:56:58.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8903 sec 2024-04-26T20:56:59.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8904 sec 2024-04-26T20:57:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:00 smithi090 ceph-mon[29180]: pgmap v5279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8502.106855 secs 2024-04-26T20:57:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:00 smithi119 ceph-mon[32752]: pgmap v5279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8502.106855 secs 2024-04-26T20:57:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:00 smithi167 ceph-mon[32751]: pgmap v5279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8502.106855 secs 2024-04-26T20:57:00.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8905 sec 2024-04-26T20:57:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8502.820215 secs 2024-04-26T20:57:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8502.820215 secs 2024-04-26T20:57:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8502.820215 secs 2024-04-26T20:57:01.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8906 sec 2024-04-26T20:57:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:02 smithi090 ceph-mon[29180]: pgmap v5280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:02 smithi119 ceph-mon[32752]: pgmap v5280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:02 smithi167 ceph-mon[32751]: pgmap v5280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:02.689 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8907 sec 2024-04-26T20:57:03.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8908 sec 2024-04-26T20:57:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:04 smithi090 ceph-mon[29180]: pgmap v5281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:04 smithi119 ceph-mon[32752]: pgmap v5281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:04 smithi167 ceph-mon[32751]: pgmap v5281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:04.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8909 sec 2024-04-26T20:57:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:05 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8507.106915 secs 2024-04-26T20:57:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:05 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8507.106915 secs 2024-04-26T20:57:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:05 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8507.106915 secs 2024-04-26T20:57:05.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8910 sec 2024-04-26T20:57:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:06 smithi090 ceph-mon[29180]: pgmap v5282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8507.820348 secs 2024-04-26T20:57:06.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:06 smithi167 ceph-mon[32751]: pgmap v5282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:06.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8507.820348 secs 2024-04-26T20:57:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:06 smithi119 ceph-mon[32752]: pgmap v5282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8507.820348 secs 2024-04-26T20:57:06.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8911 sec 2024-04-26T20:57:07.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8912 sec 2024-04-26T20:57:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:08 smithi090 ceph-mon[29180]: pgmap v5283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:08 smithi119 ceph-mon[32752]: pgmap v5283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:08 smithi167 ceph-mon[32751]: pgmap v5283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:08.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8913 sec 2024-04-26T20:57:09.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8914 sec 2024-04-26T20:57:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:10 smithi090 ceph-mon[29180]: pgmap v5284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:10.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:10 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8512.106975 secs 2024-04-26T20:57:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:10 smithi119 ceph-mon[32752]: pgmap v5284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:10 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8512.106975 secs 2024-04-26T20:57:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:10 smithi167 ceph-mon[32751]: pgmap v5284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:10 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8512.106975 secs 2024-04-26T20:57:10.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8915 sec 2024-04-26T20:57:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8512.820428 secs 2024-04-26T20:57:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8512.820428 secs 2024-04-26T20:57:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8512.820428 secs 2024-04-26T20:57:11.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8916 sec 2024-04-26T20:57:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:12 smithi090 ceph-mon[29180]: pgmap v5285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:12 smithi119 ceph-mon[32752]: pgmap v5285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:12 smithi167 ceph-mon[32751]: pgmap v5285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:12.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8917 sec 2024-04-26T20:57:13.690 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8918 sec 2024-04-26T20:57:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:14 smithi090 ceph-mon[29180]: pgmap v5286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:14 smithi119 ceph-mon[32752]: pgmap v5286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:14 smithi167 ceph-mon[32751]: pgmap v5286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:14.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8919 sec 2024-04-26T20:57:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:15 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8517.107022 secs 2024-04-26T20:57:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:15 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8517.107022 secs 2024-04-26T20:57:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:15 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8517.107022 secs 2024-04-26T20:57:15.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8920 sec 2024-04-26T20:57:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:16 smithi090 ceph-mon[29180]: pgmap v5287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8517.820544 secs 2024-04-26T20:57:16.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:16 smithi119 ceph-mon[32752]: pgmap v5287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:16.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8517.820544 secs 2024-04-26T20:57:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:16 smithi167 ceph-mon[32751]: pgmap v5287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8517.820544 secs 2024-04-26T20:57:16.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8921 sec 2024-04-26T20:57:17.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8922 sec 2024-04-26T20:57:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:18 smithi090 ceph-mon[29180]: pgmap v5288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:18 smithi119 ceph-mon[32752]: pgmap v5288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:18 smithi167 ceph-mon[32751]: pgmap v5288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:18.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8923 sec 2024-04-26T20:57:19.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8924 sec 2024-04-26T20:57:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:20 smithi119 ceph-mon[32752]: pgmap v5289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:20 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8522.107081 secs 2024-04-26T20:57:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:20 smithi167 ceph-mon[32751]: pgmap v5289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:20 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8522.107081 secs 2024-04-26T20:57:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:20 smithi090 ceph-mon[29180]: pgmap v5289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:20 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8522.107081 secs 2024-04-26T20:57:20.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8925 sec 2024-04-26T20:57:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8522.820609 secs 2024-04-26T20:57:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8522.820609 secs 2024-04-26T20:57:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8522.820609 secs 2024-04-26T20:57:21.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8926 sec 2024-04-26T20:57:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:22 smithi119 ceph-mon[32752]: pgmap v5290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:22 smithi167 ceph-mon[32751]: pgmap v5290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:22 smithi090 ceph-mon[29180]: pgmap v5290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:22.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8927 sec 2024-04-26T20:57:23.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8928 sec 2024-04-26T20:57:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:24 smithi119 ceph-mon[32752]: pgmap v5291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:24 smithi167 ceph-mon[32751]: pgmap v5291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:24 smithi090 ceph-mon[29180]: pgmap v5291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:24.691 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8929 sec 2024-04-26T20:57:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:25 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8527.107140 secs 2024-04-26T20:57:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:25 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8527.107140 secs 2024-04-26T20:57:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:25 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8527.107140 secs 2024-04-26T20:57:25.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8930 sec 2024-04-26T20:57:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:26 smithi119 ceph-mon[32752]: pgmap v5292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8527.820727 secs 2024-04-26T20:57:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:26 smithi167 ceph-mon[32751]: pgmap v5292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8527.820727 secs 2024-04-26T20:57:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:26 smithi090 ceph-mon[29180]: pgmap v5292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8527.820727 secs 2024-04-26T20:57:26.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8931 sec 2024-04-26T20:57:27.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8932 sec 2024-04-26T20:57:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:28 smithi119 ceph-mon[32752]: pgmap v5293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:28 smithi167 ceph-mon[32751]: pgmap v5293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:28 smithi090 ceph-mon[29180]: pgmap v5293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:28.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8933 sec 2024-04-26T20:57:29.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8934 sec 2024-04-26T20:57:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:30 smithi119 ceph-mon[32752]: pgmap v5294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:30 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8532.107203 secs 2024-04-26T20:57:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:30 smithi167 ceph-mon[32751]: pgmap v5294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:30 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8532.107203 secs 2024-04-26T20:57:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:30 smithi090 ceph-mon[29180]: pgmap v5294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:30 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8532.107203 secs 2024-04-26T20:57:30.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8935 sec 2024-04-26T20:57:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8532.820846 secs 2024-04-26T20:57:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8532.820846 secs 2024-04-26T20:57:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8532.820846 secs 2024-04-26T20:57:31.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8936 sec 2024-04-26T20:57:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:32 smithi119 ceph-mon[32752]: pgmap v5295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:32 smithi167 ceph-mon[32751]: pgmap v5295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:32 smithi090 ceph-mon[29180]: pgmap v5295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:32.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8937 sec 2024-04-26T20:57:33.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8938 sec 2024-04-26T20:57:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:34 smithi119 ceph-mon[32752]: pgmap v5296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:34 smithi167 ceph-mon[32751]: pgmap v5296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:34 smithi090 ceph-mon[29180]: pgmap v5296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:34.692 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8939 sec 2024-04-26T20:57:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:35 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8537.107260 secs 2024-04-26T20:57:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:35 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8537.107260 secs 2024-04-26T20:57:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:35 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8537.107260 secs 2024-04-26T20:57:35.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8940 sec 2024-04-26T20:57:36.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:36 smithi167 ceph-mon[32751]: pgmap v5297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:36.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8537.820983 secs 2024-04-26T20:57:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:36 smithi119 ceph-mon[32752]: pgmap v5297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8537.820983 secs 2024-04-26T20:57:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:36 smithi090 ceph-mon[29180]: pgmap v5297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8537.820983 secs 2024-04-26T20:57:36.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8941 sec 2024-04-26T20:57:37.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8942 sec 2024-04-26T20:57:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:38 smithi119 ceph-mon[32752]: pgmap v5298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:38 smithi167 ceph-mon[32751]: pgmap v5298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:38 smithi090 ceph-mon[29180]: pgmap v5298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:38.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8943 sec 2024-04-26T20:57:39.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8944 sec 2024-04-26T20:57:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:40 smithi119 ceph-mon[32752]: pgmap v5299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:40 smithi119 ceph-mon[32752]: 8 slow requests, 1 included below; oldest blocked for > 8542.107320 secs 2024-04-26T20:57:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:40 smithi119 ceph-mon[32752]: slow request 2882.472028 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:57:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:40 smithi167 ceph-mon[32751]: pgmap v5299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:40 smithi167 ceph-mon[32751]: 8 slow requests, 1 included below; oldest blocked for > 8542.107320 secs 2024-04-26T20:57:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:40 smithi167 ceph-mon[32751]: slow request 2882.472028 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:57:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:40 smithi090 ceph-mon[29180]: pgmap v5299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:57:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:40 smithi090 ceph-mon[29180]: 8 slow requests, 1 included below; oldest blocked for > 8542.107320 secs 2024-04-26T20:57:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:40 smithi090 ceph-mon[29180]: slow request 2882.472028 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T20:57:40.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8945 sec 2024-04-26T20:57:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8542.821100 secs 2024-04-26T20:57:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8542.821100 secs 2024-04-26T20:57:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8542.821100 secs 2024-04-26T20:57:41.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8946 sec 2024-04-26T20:57:42.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:42 smithi090 ceph-mon[29180]: pgmap v5300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:42 smithi119 ceph-mon[32752]: pgmap v5300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:42 smithi167 ceph-mon[32751]: pgmap v5300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:42.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8947 sec 2024-04-26T20:57:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:57:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:57:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:57:43.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8948 sec 2024-04-26T20:57:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:44 smithi119 ceph-mon[32752]: pgmap v5301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:57:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:57:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:57:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:44 smithi167 ceph-mon[32751]: pgmap v5301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:57:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:57:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:57:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:44 smithi090 ceph-mon[29180]: pgmap v5301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:44.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:57:44.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:57:44.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:57:44.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8949 sec 2024-04-26T20:57:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:45 smithi119 ceph-mon[32752]: pgmap v5302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:57:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:45 smithi119 ceph-mon[32752]: pgmap v5303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:45.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:45 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8547.107381 secs 2024-04-26T20:57:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:45 smithi167 ceph-mon[32751]: pgmap v5302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:57:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:45 smithi167 ceph-mon[32751]: pgmap v5303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:45 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8547.107381 secs 2024-04-26T20:57:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:45 smithi090 ceph-mon[29180]: pgmap v5302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T20:57:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:45 smithi090 ceph-mon[29180]: pgmap v5303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:45 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8547.107381 secs 2024-04-26T20:57:45.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8950 sec 2024-04-26T20:57:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8547.821252 secs 2024-04-26T20:57:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8547.821252 secs 2024-04-26T20:57:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8547.821252 secs 2024-04-26T20:57:46.693 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8951 sec 2024-04-26T20:57:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:47 smithi119 ceph-mon[32752]: pgmap v5304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:47 smithi167 ceph-mon[32751]: pgmap v5304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:47 smithi090 ceph-mon[29180]: pgmap v5304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:47.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8952 sec 2024-04-26T20:57:48.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8953 sec 2024-04-26T20:57:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:49 smithi119 ceph-mon[32752]: pgmap v5305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:57:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:49 smithi167 ceph-mon[32751]: pgmap v5305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:57:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:49 smithi090 ceph-mon[29180]: pgmap v5305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T20:57:49.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8954 sec 2024-04-26T20:57:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:50 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8552.107441 secs 2024-04-26T20:57:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:50 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8552.107441 secs 2024-04-26T20:57:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:50 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8552.107441 secs 2024-04-26T20:57:50.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8955 sec 2024-04-26T20:57:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:51 smithi119 ceph-mon[32752]: pgmap v5306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8552.821389 secs 2024-04-26T20:57:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:51 smithi167 ceph-mon[32751]: pgmap v5306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8552.821389 secs 2024-04-26T20:57:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:51 smithi090 ceph-mon[29180]: pgmap v5306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8552.821389 secs 2024-04-26T20:57:51.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8956 sec 2024-04-26T20:57:52.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8957 sec 2024-04-26T20:57:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:53 smithi119 ceph-mon[32752]: pgmap v5307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:53 smithi167 ceph-mon[32751]: pgmap v5307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:53.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:53 smithi090 ceph-mon[29180]: pgmap v5307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:57:53.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8958 sec 2024-04-26T20:57:54.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8959 sec 2024-04-26T20:57:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:55 smithi119 ceph-mon[32752]: pgmap v5308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:57:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:55 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8557.107502 secs 2024-04-26T20:57:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:55 smithi167 ceph-mon[32751]: pgmap v5308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:57:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:55 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8557.107502 secs 2024-04-26T20:57:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:55 smithi090 ceph-mon[29180]: pgmap v5308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:57:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:55 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8557.107502 secs 2024-04-26T20:57:55.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8960 sec 2024-04-26T20:57:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8557.821512 secs 2024-04-26T20:57:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8557.821512 secs 2024-04-26T20:57:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8557.821512 secs 2024-04-26T20:57:56.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8961 sec 2024-04-26T20:57:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:57 smithi119 ceph-mon[32752]: pgmap v5309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:57 smithi167 ceph-mon[32751]: pgmap v5309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:57 smithi090 ceph-mon[29180]: pgmap v5309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:57.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8962 sec 2024-04-26T20:57:58.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8963 sec 2024-04-26T20:57:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:57:59 smithi119 ceph-mon[32752]: pgmap v5310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:57:59 smithi167 ceph-mon[32751]: pgmap v5310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:57:59 smithi090 ceph-mon[29180]: pgmap v5310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:57:59.694 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8964 sec 2024-04-26T20:58:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:00 smithi119 ceph-mon[32752]: 8 slow requests, 0 included below; oldest blocked for > 8562.107563 secs 2024-04-26T20:58:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:00 smithi167 ceph-mon[32751]: 8 slow requests, 0 included below; oldest blocked for > 8562.107563 secs 2024-04-26T20:58:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:00 smithi090 ceph-mon[29180]: 8 slow requests, 0 included below; oldest blocked for > 8562.107563 secs 2024-04-26T20:58:00.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8965 sec 2024-04-26T20:58:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:01 smithi119 ceph-mon[32752]: pgmap v5311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8562.821650 secs 2024-04-26T20:58:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:01 smithi167 ceph-mon[32751]: pgmap v5311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8562.821650 secs 2024-04-26T20:58:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:01 smithi090 ceph-mon[29180]: pgmap v5311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8562.821650 secs 2024-04-26T20:58:01.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8966 sec 2024-04-26T20:58:02.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8967 sec 2024-04-26T20:58:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:03 smithi119 ceph-mon[32752]: pgmap v5312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:03 smithi167 ceph-mon[32751]: pgmap v5312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:03 smithi090 ceph-mon[29180]: pgmap v5312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:03.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8968 sec 2024-04-26T20:58:04.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8969 sec 2024-04-26T20:58:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:05 smithi119 ceph-mon[32752]: pgmap v5313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:05 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 8567.107621 secs 2024-04-26T20:58:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:05 smithi119 ceph-mon[32752]: slow request 181.869565 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T20:58:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:05 smithi167 ceph-mon[32751]: pgmap v5313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:05 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 8567.107621 secs 2024-04-26T20:58:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:05 smithi167 ceph-mon[32751]: slow request 181.869565 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T20:58:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:05 smithi090 ceph-mon[29180]: pgmap v5313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:05 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 8567.107621 secs 2024-04-26T20:58:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:05 smithi090 ceph-mon[29180]: slow request 181.869565 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T20:58:05.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8970 sec 2024-04-26T20:58:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8567.821773 secs 2024-04-26T20:58:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8567.821773 secs 2024-04-26T20:58:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8567.821773 secs 2024-04-26T20:58:06.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8971 sec 2024-04-26T20:58:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:07 smithi119 ceph-mon[32752]: pgmap v5314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:58:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:07 smithi167 ceph-mon[32751]: pgmap v5314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:58:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:07 smithi090 ceph-mon[29180]: pgmap v5314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T20:58:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T20:58:07.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8972 sec 2024-04-26T20:58:08.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8973 sec 2024-04-26T20:58:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:09 smithi119 ceph-mon[32752]: pgmap v5315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:09 smithi167 ceph-mon[32751]: pgmap v5315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:09 smithi090 ceph-mon[29180]: pgmap v5315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:09.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8974 sec 2024-04-26T20:58:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8572.107680 secs 2024-04-26T20:58:10.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8572.107680 secs 2024-04-26T20:58:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8572.107680 secs 2024-04-26T20:58:10.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8975 sec 2024-04-26T20:58:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:11 smithi119 ceph-mon[32752]: pgmap v5316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8572.821897 secs 2024-04-26T20:58:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:11 smithi167 ceph-mon[32751]: pgmap v5316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8572.821897 secs 2024-04-26T20:58:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:11 smithi090 ceph-mon[29180]: pgmap v5316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8572.821897 secs 2024-04-26T20:58:11.695 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8976 sec 2024-04-26T20:58:12.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8977 sec 2024-04-26T20:58:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:13 smithi119 ceph-mon[32752]: pgmap v5317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:13 smithi167 ceph-mon[32751]: pgmap v5317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:13 smithi090 ceph-mon[29180]: pgmap v5317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:13.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8978 sec 2024-04-26T20:58:14.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8979 sec 2024-04-26T20:58:15.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:15 smithi119 ceph-mon[32752]: pgmap v5318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:15.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8577.107740 secs 2024-04-26T20:58:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:15 smithi167 ceph-mon[32751]: pgmap v5318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8577.107740 secs 2024-04-26T20:58:15.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:15 smithi090 ceph-mon[29180]: pgmap v5318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:15.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8577.107740 secs 2024-04-26T20:58:15.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8980 sec 2024-04-26T20:58:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8577.821969 secs 2024-04-26T20:58:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8577.821969 secs 2024-04-26T20:58:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8577.821969 secs 2024-04-26T20:58:16.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8981 sec 2024-04-26T20:58:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:17 smithi119 ceph-mon[32752]: pgmap v5319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:17 smithi167 ceph-mon[32751]: pgmap v5319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:17 smithi090 ceph-mon[29180]: pgmap v5319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:17.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8982 sec 2024-04-26T20:58:18.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8983 sec 2024-04-26T20:58:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:19 smithi119 ceph-mon[32752]: pgmap v5320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:19 smithi167 ceph-mon[32751]: pgmap v5320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:19 smithi090 ceph-mon[29180]: pgmap v5320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:19.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8984 sec 2024-04-26T20:58:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8582.107796 secs 2024-04-26T20:58:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8582.107796 secs 2024-04-26T20:58:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8582.107796 secs 2024-04-26T20:58:20.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8985 sec 2024-04-26T20:58:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:21 smithi119 ceph-mon[32752]: pgmap v5321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8582.822098 secs 2024-04-26T20:58:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:21 smithi167 ceph-mon[32751]: pgmap v5321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8582.822098 secs 2024-04-26T20:58:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:21 smithi090 ceph-mon[29180]: pgmap v5321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8582.822098 secs 2024-04-26T20:58:21.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8986 sec 2024-04-26T20:58:22.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8987 sec 2024-04-26T20:58:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:23 smithi119 ceph-mon[32752]: pgmap v5322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:23 smithi167 ceph-mon[32751]: pgmap v5322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:23 smithi090 ceph-mon[29180]: pgmap v5322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:23.696 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8988 sec 2024-04-26T20:58:24.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8989 sec 2024-04-26T20:58:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:25 smithi167 ceph-mon[32751]: pgmap v5323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8587.107856 secs 2024-04-26T20:58:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:25 smithi090 ceph-mon[29180]: pgmap v5323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8587.107856 secs 2024-04-26T20:58:25.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8990 sec 2024-04-26T20:58:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:25 smithi119 ceph-mon[32752]: pgmap v5323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8587.107856 secs 2024-04-26T20:58:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8587.822219 secs 2024-04-26T20:58:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8587.822219 secs 2024-04-26T20:58:26.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8991 sec 2024-04-26T20:58:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8587.822219 secs 2024-04-26T20:58:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:27 smithi167 ceph-mon[32751]: pgmap v5324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:27 smithi090 ceph-mon[29180]: pgmap v5324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:27.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8992 sec 2024-04-26T20:58:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:27 smithi119 ceph-mon[32752]: pgmap v5324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:28.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8993 sec 2024-04-26T20:58:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:29 smithi167 ceph-mon[32751]: pgmap v5325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:29 smithi090 ceph-mon[29180]: pgmap v5325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:29.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8994 sec 2024-04-26T20:58:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:29 smithi119 ceph-mon[32752]: pgmap v5325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8592.107918 secs 2024-04-26T20:58:30.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8995 sec 2024-04-26T20:58:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8592.107918 secs 2024-04-26T20:58:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8592.107918 secs 2024-04-26T20:58:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:31 smithi090 ceph-mon[29180]: pgmap v5326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8592.822388 secs 2024-04-26T20:58:31.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8996 sec 2024-04-26T20:58:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:31 smithi119 ceph-mon[32752]: pgmap v5326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8592.822388 secs 2024-04-26T20:58:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:31 smithi167 ceph-mon[32751]: pgmap v5326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8592.822388 secs 2024-04-26T20:58:32.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8997 sec 2024-04-26T20:58:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:33 smithi090 ceph-mon[29180]: pgmap v5327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:33.697 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8998 sec 2024-04-26T20:58:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:33 smithi119 ceph-mon[32752]: pgmap v5327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:33 smithi167 ceph-mon[32751]: pgmap v5327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:34.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 8999 sec 2024-04-26T20:58:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:35 smithi090 ceph-mon[29180]: pgmap v5328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8597.107975 secs 2024-04-26T20:58:35.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9000 sec 2024-04-26T20:58:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:35 smithi119 ceph-mon[32752]: pgmap v5328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8597.107975 secs 2024-04-26T20:58:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:35 smithi167 ceph-mon[32751]: pgmap v5328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8597.107975 secs 2024-04-26T20:58:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8597.822515 secs 2024-04-26T20:58:36.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9001 sec 2024-04-26T20:58:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8597.822515 secs 2024-04-26T20:58:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8597.822515 secs 2024-04-26T20:58:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:37 smithi090 ceph-mon[29180]: pgmap v5329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:37.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9002 sec 2024-04-26T20:58:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:37 smithi119 ceph-mon[32752]: pgmap v5329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:37 smithi167 ceph-mon[32751]: pgmap v5329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:38.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9003 sec 2024-04-26T20:58:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:39 smithi090 ceph-mon[29180]: pgmap v5330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:39.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9004 sec 2024-04-26T20:58:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:39 smithi119 ceph-mon[32752]: pgmap v5330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:39 smithi167 ceph-mon[32751]: pgmap v5330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8602.108035 secs 2024-04-26T20:58:40.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9005 sec 2024-04-26T20:58:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8602.108035 secs 2024-04-26T20:58:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8602.108035 secs 2024-04-26T20:58:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:41 smithi090 ceph-mon[29180]: pgmap v5331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8602.822599 secs 2024-04-26T20:58:41.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9006 sec 2024-04-26T20:58:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:41 smithi119 ceph-mon[32752]: pgmap v5331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8602.822599 secs 2024-04-26T20:58:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:41 smithi167 ceph-mon[32751]: pgmap v5331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8602.822599 secs 2024-04-26T20:58:42.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9007 sec 2024-04-26T20:58:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:43 smithi090 ceph-mon[29180]: pgmap v5332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:43.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9008 sec 2024-04-26T20:58:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:43 smithi119 ceph-mon[32752]: pgmap v5332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:43 smithi167 ceph-mon[32751]: pgmap v5332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:58:44.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9009 sec 2024-04-26T20:58:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:58:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:58:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:45 smithi090 ceph-mon[29180]: pgmap v5333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:58:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:58:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:45 smithi090 ceph-mon[29180]: pgmap v5334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:58:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:45 smithi090 ceph-mon[29180]: pgmap v5335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:58:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8607.108091 secs 2024-04-26T20:58:45.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9010 sec 2024-04-26T20:58:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:45 smithi119 ceph-mon[32752]: pgmap v5333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:58:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:58:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:45 smithi119 ceph-mon[32752]: pgmap v5334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:58:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:45 smithi119 ceph-mon[32752]: pgmap v5335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:58:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8607.108091 secs 2024-04-26T20:58:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:45 smithi167 ceph-mon[32751]: pgmap v5333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:58:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:58:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:45 smithi167 ceph-mon[32751]: pgmap v5334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:58:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:45 smithi167 ceph-mon[32751]: pgmap v5335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:58:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8607.108091 secs 2024-04-26T20:58:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8607.822686 secs 2024-04-26T20:58:46.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9011 sec 2024-04-26T20:58:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8607.822686 secs 2024-04-26T20:58:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8607.822686 secs 2024-04-26T20:58:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:47 smithi090 ceph-mon[29180]: pgmap v5336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:47.698 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9012 sec 2024-04-26T20:58:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:47 smithi119 ceph-mon[32752]: pgmap v5336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:47 smithi167 ceph-mon[32751]: pgmap v5336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:48.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9013 sec 2024-04-26T20:58:49.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9014 sec 2024-04-26T20:58:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:49 smithi090 ceph-mon[29180]: pgmap v5337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:49 smithi119 ceph-mon[32752]: pgmap v5337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:49 smithi167 ceph-mon[32751]: pgmap v5337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:50.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9015 sec 2024-04-26T20:58:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8612.108151 secs 2024-04-26T20:58:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8612.108151 secs 2024-04-26T20:58:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8612.108151 secs 2024-04-26T20:58:51.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9016 sec 2024-04-26T20:58:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:51 smithi090 ceph-mon[29180]: pgmap v5338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8612.822771 secs 2024-04-26T20:58:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:51 smithi119 ceph-mon[32752]: pgmap v5338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8612.822771 secs 2024-04-26T20:58:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:51 smithi167 ceph-mon[32751]: pgmap v5338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:58:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8612.822771 secs 2024-04-26T20:58:52.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9017 sec 2024-04-26T20:58:53.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9018 sec 2024-04-26T20:58:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:53 smithi090 ceph-mon[29180]: pgmap v5339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:58:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:53 smithi119 ceph-mon[32752]: pgmap v5339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:58:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:53 smithi167 ceph-mon[32751]: pgmap v5339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:58:54.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9019 sec 2024-04-26T20:58:55.699 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9020 sec 2024-04-26T20:58:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:55 smithi090 ceph-mon[29180]: pgmap v5340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:58:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8617.108208 secs 2024-04-26T20:58:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:55 smithi119 ceph-mon[32752]: pgmap v5340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:58:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8617.108208 secs 2024-04-26T20:58:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:55 smithi167 ceph-mon[32751]: pgmap v5340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:58:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8617.108208 secs 2024-04-26T20:58:56.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9021 sec 2024-04-26T20:58:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8617.822831 secs 2024-04-26T20:58:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8617.822831 secs 2024-04-26T20:58:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8617.822831 secs 2024-04-26T20:58:57.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9022 sec 2024-04-26T20:58:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:57 smithi090 ceph-mon[29180]: pgmap v5341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:57 smithi119 ceph-mon[32752]: pgmap v5341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:57 smithi167 ceph-mon[32751]: pgmap v5341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:58:58.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9023 sec 2024-04-26T20:58:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:58 smithi119 ceph-mon[32752]: pgmap v5342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:58 smithi167 ceph-mon[32751]: pgmap v5342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:58 smithi090 ceph-mon[29180]: pgmap v5342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:58:59.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9024 sec 2024-04-26T20:59:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:58:59 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8622.108266 secs 2024-04-26T20:59:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:58:59 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8622.108266 secs 2024-04-26T20:59:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:58:59 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8622.108266 secs 2024-04-26T20:59:00.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9025 sec 2024-04-26T20:59:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:00 smithi119 ceph-mon[32752]: pgmap v5343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8622.822910 secs 2024-04-26T20:59:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:00 smithi167 ceph-mon[32751]: pgmap v5343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8622.822910 secs 2024-04-26T20:59:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:00 smithi090 ceph-mon[29180]: pgmap v5343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8622.822910 secs 2024-04-26T20:59:01.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9026 sec 2024-04-26T20:59:02.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9027 sec 2024-04-26T20:59:03.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9028 sec 2024-04-26T20:59:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:03 smithi090 ceph-mon[29180]: pgmap v5344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:03 smithi119 ceph-mon[32752]: pgmap v5344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:03 smithi167 ceph-mon[32751]: pgmap v5344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:04.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9029 sec 2024-04-26T20:59:05.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9030 sec 2024-04-26T20:59:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:05 smithi090 ceph-mon[29180]: pgmap v5345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8627.108325 secs 2024-04-26T20:59:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:05 smithi119 ceph-mon[32752]: pgmap v5345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8627.108325 secs 2024-04-26T20:59:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:05 smithi167 ceph-mon[32751]: pgmap v5345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8627.108325 secs 2024-04-26T20:59:06.700 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9031 sec 2024-04-26T20:59:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8627.822981 secs 2024-04-26T20:59:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8627.822981 secs 2024-04-26T20:59:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8627.822981 secs 2024-04-26T20:59:07.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9032 sec 2024-04-26T20:59:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:07 smithi090 ceph-mon[29180]: pgmap v5346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:07 smithi119 ceph-mon[32752]: pgmap v5346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:07 smithi167 ceph-mon[32751]: pgmap v5346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:08.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9033 sec 2024-04-26T20:59:09.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9034 sec 2024-04-26T20:59:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:09 smithi090 ceph-mon[29180]: pgmap v5347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:09 smithi119 ceph-mon[32752]: pgmap v5347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:09 smithi167 ceph-mon[32751]: pgmap v5347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:10.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9035 sec 2024-04-26T20:59:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8632.108387 secs 2024-04-26T20:59:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8632.108387 secs 2024-04-26T20:59:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8632.108387 secs 2024-04-26T20:59:11.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9036 sec 2024-04-26T20:59:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:11 smithi119 ceph-mon[32752]: pgmap v5348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8632.823097 secs 2024-04-26T20:59:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:11 smithi167 ceph-mon[32751]: pgmap v5348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8632.823097 secs 2024-04-26T20:59:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:11 smithi090 ceph-mon[29180]: pgmap v5348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8632.823097 secs 2024-04-26T20:59:12.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9037 sec 2024-04-26T20:59:13.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:12 smithi090 ceph-mon[29180]: pgmap v5349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:12 smithi119 ceph-mon[32752]: pgmap v5349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:12 smithi167 ceph-mon[32751]: pgmap v5349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:13.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9038 sec 2024-04-26T20:59:14.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9039 sec 2024-04-26T20:59:15.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9040 sec 2024-04-26T20:59:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:15 smithi090 ceph-mon[29180]: pgmap v5350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8637.108449 secs 2024-04-26T20:59:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:15 smithi119 ceph-mon[32752]: pgmap v5350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8637.108449 secs 2024-04-26T20:59:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:15 smithi167 ceph-mon[32751]: pgmap v5350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8637.108449 secs 2024-04-26T20:59:16.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9041 sec 2024-04-26T20:59:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8637.823232 secs 2024-04-26T20:59:16.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8637.823232 secs 2024-04-26T20:59:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8637.823232 secs 2024-04-26T20:59:17.701 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9042 sec 2024-04-26T20:59:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:17 smithi090 ceph-mon[29180]: pgmap v5351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:17 smithi119 ceph-mon[32752]: pgmap v5351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:17 smithi167 ceph-mon[32751]: pgmap v5351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:18.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9043 sec 2024-04-26T20:59:19.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9044 sec 2024-04-26T20:59:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:19 smithi090 ceph-mon[29180]: pgmap v5352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:19 smithi119 ceph-mon[32752]: pgmap v5352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:19 smithi167 ceph-mon[32751]: pgmap v5352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:20.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9045 sec 2024-04-26T20:59:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8642.108512 secs 2024-04-26T20:59:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8642.108512 secs 2024-04-26T20:59:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8642.108512 secs 2024-04-26T20:59:21.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9046 sec 2024-04-26T20:59:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:21 smithi090 ceph-mon[29180]: pgmap v5353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8642.823348 secs 2024-04-26T20:59:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:21 smithi119 ceph-mon[32752]: pgmap v5353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8642.823348 secs 2024-04-26T20:59:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:21 smithi167 ceph-mon[32751]: pgmap v5353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8642.823348 secs 2024-04-26T20:59:22.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9047 sec 2024-04-26T20:59:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:22 smithi090 ceph-mon[29180]: pgmap v5354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:22 smithi119 ceph-mon[32752]: pgmap v5354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:22 smithi167 ceph-mon[32751]: pgmap v5354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:23.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9048 sec 2024-04-26T20:59:24.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9049 sec 2024-04-26T20:59:25.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9050 sec 2024-04-26T20:59:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:25 smithi090 ceph-mon[29180]: pgmap v5355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8647.108573 secs 2024-04-26T20:59:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:25 smithi119 ceph-mon[32752]: pgmap v5355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8647.108573 secs 2024-04-26T20:59:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:25 smithi167 ceph-mon[32751]: pgmap v5355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8647.108573 secs 2024-04-26T20:59:26.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9051 sec 2024-04-26T20:59:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8647.823462 secs 2024-04-26T20:59:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8647.823462 secs 2024-04-26T20:59:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8647.823462 secs 2024-04-26T20:59:27.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9052 sec 2024-04-26T20:59:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:27 smithi090 ceph-mon[29180]: pgmap v5356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:27 smithi119 ceph-mon[32752]: pgmap v5356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:27 smithi167 ceph-mon[32751]: pgmap v5356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:28.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9053 sec 2024-04-26T20:59:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:59:29.622+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:59:29.623 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:59:29.622+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T20:59:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:59:29.628+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 70 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T20:59:29.629 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T20:59:29.628+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 70 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T20:59:29.702 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9054 sec 2024-04-26T20:59:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:29 smithi090 ceph-mon[29180]: pgmap v5357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:29 smithi119 ceph-mon[32752]: pgmap v5357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:29 smithi167 ceph-mon[32751]: pgmap v5357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:30.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9055 sec 2024-04-26T20:59:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8652.108634 secs 2024-04-26T20:59:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8652.108634 secs 2024-04-26T20:59:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8652.108634 secs 2024-04-26T20:59:31.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9056 sec 2024-04-26T20:59:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:31 smithi119 ceph-mon[32752]: pgmap v5358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8652.823585 secs 2024-04-26T20:59:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:31 smithi167 ceph-mon[32751]: pgmap v5358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8652.823585 secs 2024-04-26T20:59:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:31 smithi090 ceph-mon[29180]: pgmap v5358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8652.823585 secs 2024-04-26T20:59:32.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9057 sec 2024-04-26T20:59:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:32 smithi119 ceph-mon[32752]: pgmap v5359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:32 smithi167 ceph-mon[32751]: pgmap v5359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:32 smithi090 ceph-mon[29180]: pgmap v5359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:33.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9058 sec 2024-04-26T20:59:34.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9059 sec 2024-04-26T20:59:35.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9060 sec 2024-04-26T20:59:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:35 smithi090 ceph-mon[29180]: pgmap v5360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8657.108694 secs 2024-04-26T20:59:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:35 smithi119 ceph-mon[32752]: pgmap v5360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8657.108694 secs 2024-04-26T20:59:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:35 smithi167 ceph-mon[32751]: pgmap v5360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8657.108694 secs 2024-04-26T20:59:36.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9061 sec 2024-04-26T20:59:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8657.823724 secs 2024-04-26T20:59:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8657.823724 secs 2024-04-26T20:59:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8657.823724 secs 2024-04-26T20:59:37.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9062 sec 2024-04-26T20:59:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:37 smithi090 ceph-mon[29180]: pgmap v5361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:37 smithi119 ceph-mon[32752]: pgmap v5361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:37 smithi167 ceph-mon[32751]: pgmap v5361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:38.703 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9063 sec 2024-04-26T20:59:39.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9064 sec 2024-04-26T20:59:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:39 smithi090 ceph-mon[29180]: pgmap v5362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:39 smithi119 ceph-mon[32752]: pgmap v5362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:39 smithi167 ceph-mon[32751]: pgmap v5362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:40.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9065 sec 2024-04-26T20:59:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8662.108755 secs 2024-04-26T20:59:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:40 smithi119 ceph-mon[32752]: pgmap v5363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8662.823833 secs 2024-04-26T20:59:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8662.108755 secs 2024-04-26T20:59:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:40 smithi167 ceph-mon[32751]: pgmap v5363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8662.823833 secs 2024-04-26T20:59:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8662.108755 secs 2024-04-26T20:59:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:40 smithi090 ceph-mon[29180]: pgmap v5363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8662.823833 secs 2024-04-26T20:59:41.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9066 sec 2024-04-26T20:59:42.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9067 sec 2024-04-26T20:59:43.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9068 sec 2024-04-26T20:59:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:43 smithi090 ceph-mon[29180]: pgmap v5364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:43 smithi119 ceph-mon[32752]: pgmap v5364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:43 smithi167 ceph-mon[32751]: pgmap v5364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:44.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9069 sec 2024-04-26T20:59:45.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9070 sec 2024-04-26T20:59:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:45 smithi090 ceph-mon[29180]: pgmap v5365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8667.108830 secs 2024-04-26T20:59:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:59:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:59:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:59:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:45 smithi119 ceph-mon[32752]: pgmap v5365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8667.108830 secs 2024-04-26T20:59:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:59:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:59:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:59:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:45 smithi167 ceph-mon[32751]: pgmap v5365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T20:59:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T20:59:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8667.108830 secs 2024-04-26T20:59:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T20:59:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T20:59:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T20:59:46.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9071 sec 2024-04-26T20:59:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:46 smithi090 ceph-mon[29180]: pgmap v5366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:59:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:46 smithi090 ceph-mon[29180]: pgmap v5367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8667.823950 secs 2024-04-26T20:59:46.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:46 smithi119 ceph-mon[32752]: pgmap v5366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:59:46.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:46 smithi119 ceph-mon[32752]: pgmap v5367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:46.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8667.823950 secs 2024-04-26T20:59:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:46 smithi167 ceph-mon[32751]: pgmap v5366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T20:59:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:46 smithi167 ceph-mon[32751]: pgmap v5367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8667.823950 secs 2024-04-26T20:59:47.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9072 sec 2024-04-26T20:59:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:47 smithi119 ceph-mon[32752]: pgmap v5368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:59:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:47 smithi167 ceph-mon[32751]: pgmap v5368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:59:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:47 smithi090 ceph-mon[29180]: pgmap v5368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T20:59:48.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9073 sec 2024-04-26T20:59:49.704 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9074 sec 2024-04-26T20:59:50.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9075 sec 2024-04-26T20:59:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:50 smithi119 ceph-mon[32752]: pgmap v5369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8672.108915 secs 2024-04-26T20:59:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:50 smithi167 ceph-mon[32751]: pgmap v5369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8672.108915 secs 2024-04-26T20:59:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:50 smithi090 ceph-mon[29180]: pgmap v5369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8672.108915 secs 2024-04-26T20:59:51.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9076 sec 2024-04-26T20:59:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8672.824102 secs 2024-04-26T20:59:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8672.824102 secs 2024-04-26T20:59:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8672.824102 secs 2024-04-26T20:59:52.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9077 sec 2024-04-26T20:59:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:52 smithi119 ceph-mon[32752]: pgmap v5370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:52 smithi167 ceph-mon[32751]: pgmap v5370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:52 smithi090 ceph-mon[29180]: pgmap v5370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:53.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9078 sec 2024-04-26T20:59:54.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9079 sec 2024-04-26T20:59:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:54 smithi119 ceph-mon[32752]: pgmap v5371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:54 smithi167 ceph-mon[32751]: pgmap v5371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:54 smithi090 ceph-mon[29180]: pgmap v5371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T20:59:55.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9080 sec 2024-04-26T20:59:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8677.109002 secs 2024-04-26T20:59:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8677.109002 secs 2024-04-26T20:59:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8677.109002 secs 2024-04-26T20:59:56.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9081 sec 2024-04-26T20:59:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:56 smithi119 ceph-mon[32752]: pgmap v5372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:59:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8677.824195 secs 2024-04-26T20:59:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:56 smithi167 ceph-mon[32751]: pgmap v5372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:59:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8677.824195 secs 2024-04-26T20:59:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:56 smithi090 ceph-mon[29180]: pgmap v5372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T20:59:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8677.824195 secs 2024-04-26T20:59:57.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9082 sec 2024-04-26T20:59:58.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9083 sec 2024-04-26T20:59:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 20:59:58 smithi119 ceph-mon[32752]: pgmap v5373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 20:59:58 smithi167 ceph-mon[32751]: pgmap v5373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 20:59:58 smithi090 ceph-mon[29180]: pgmap v5373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T20:59:59.705 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9084 sec 2024-04-26T21:00:00.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9085 sec 2024-04-26T21:00:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:00 smithi119 ceph-mon[32752]: pgmap v5374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8682.109087 secs 2024-04-26T21:00:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:00:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:00 smithi167 ceph-mon[32751]: pgmap v5374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8682.109087 secs 2024-04-26T21:00:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:00:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:00 smithi090 ceph-mon[29180]: pgmap v5374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8682.109087 secs 2024-04-26T21:00:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:00:01.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9086 sec 2024-04-26T21:00:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8682.824294 secs 2024-04-26T21:00:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8682.824294 secs 2024-04-26T21:00:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8682.824294 secs 2024-04-26T21:00:02.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9087 sec 2024-04-26T21:00:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:02 smithi119 ceph-mon[32752]: pgmap v5375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:02 smithi167 ceph-mon[32751]: pgmap v5375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:02 smithi090 ceph-mon[29180]: pgmap v5375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:00:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:00:03.107+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:00:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:00:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:00:03.107+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:00:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:00:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:00:03.107+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:00:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:00:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:00:03.107+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:00:03.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:00:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:00:03.107+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:00:03.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9088 sec 2024-04-26T21:00:04.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9089 sec 2024-04-26T21:00:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:04 smithi119 ceph-mon[32752]: pgmap v5376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:04 smithi167 ceph-mon[32751]: pgmap v5376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:04 smithi090 ceph-mon[29180]: pgmap v5376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:05.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9090 sec 2024-04-26T21:00:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8687.109147 secs 2024-04-26T21:00:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8687.109147 secs 2024-04-26T21:00:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8687.109147 secs 2024-04-26T21:00:06.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9091 sec 2024-04-26T21:00:06.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:06 smithi167 ceph-mon[32751]: pgmap v5377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:06.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8687.824416 secs 2024-04-26T21:00:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:06 smithi119 ceph-mon[32752]: pgmap v5377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8687.824416 secs 2024-04-26T21:00:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:06 smithi090 ceph-mon[29180]: pgmap v5377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8687.824416 secs 2024-04-26T21:00:07.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9092 sec 2024-04-26T21:00:08.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9093 sec 2024-04-26T21:00:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:08 smithi119 ceph-mon[32752]: pgmap v5378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:08 smithi167 ceph-mon[32751]: pgmap v5378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:08 smithi090 ceph-mon[29180]: pgmap v5378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:09.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9094 sec 2024-04-26T21:00:10.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9095 sec 2024-04-26T21:00:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:10 smithi119 ceph-mon[32752]: pgmap v5379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8692.109226 secs 2024-04-26T21:00:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:10 smithi167 ceph-mon[32751]: pgmap v5379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8692.109226 secs 2024-04-26T21:00:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:10 smithi090 ceph-mon[29180]: pgmap v5379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8692.109226 secs 2024-04-26T21:00:11.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9096 sec 2024-04-26T21:00:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8692.824527 secs 2024-04-26T21:00:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8692.824527 secs 2024-04-26T21:00:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8692.824527 secs 2024-04-26T21:00:12.706 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9097 sec 2024-04-26T21:00:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:12 smithi119 ceph-mon[32752]: pgmap v5380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:12 smithi167 ceph-mon[32751]: pgmap v5380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:12 smithi090 ceph-mon[29180]: pgmap v5380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T21:00:13.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9098 sec 2024-04-26T21:00:14.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9099 sec 2024-04-26T21:00:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:14 smithi119 ceph-mon[32752]: pgmap v5381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:14 smithi167 ceph-mon[32751]: pgmap v5381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:14 smithi090 ceph-mon[29180]: pgmap v5381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T21:00:15.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9100 sec 2024-04-26T21:00:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8697.109310 secs 2024-04-26T21:00:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8697.109310 secs 2024-04-26T21:00:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8697.109310 secs 2024-04-26T21:00:16.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9101 sec 2024-04-26T21:00:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:16 smithi119 ceph-mon[32752]: pgmap v5382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:00:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8697.824636 secs 2024-04-26T21:00:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:16 smithi167 ceph-mon[32751]: pgmap v5382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:00:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8697.824636 secs 2024-04-26T21:00:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:16 smithi090 ceph-mon[29180]: pgmap v5382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:00:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8697.824636 secs 2024-04-26T21:00:17.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9102 sec 2024-04-26T21:00:18.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9103 sec 2024-04-26T21:00:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:18 smithi119 ceph-mon[32752]: pgmap v5383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:00:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:18 smithi167 ceph-mon[32751]: pgmap v5383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:00:18.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:18 smithi090 ceph-mon[29180]: pgmap v5383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:00:19.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9104 sec 2024-04-26T21:00:20.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9105 sec 2024-04-26T21:00:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:20 smithi119 ceph-mon[32752]: pgmap v5384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:00:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8702.109391 secs 2024-04-26T21:00:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:20 smithi167 ceph-mon[32751]: pgmap v5384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:00:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8702.109391 secs 2024-04-26T21:00:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:20 smithi090 ceph-mon[29180]: pgmap v5384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:00:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8702.109391 secs 2024-04-26T21:00:21.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9106 sec 2024-04-26T21:00:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8702.824764 secs 2024-04-26T21:00:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8702.824764 secs 2024-04-26T21:00:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8702.824764 secs 2024-04-26T21:00:22.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9107 sec 2024-04-26T21:00:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:22 smithi119 ceph-mon[32752]: pgmap v5385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:22 smithi167 ceph-mon[32751]: pgmap v5385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:22 smithi090 ceph-mon[29180]: pgmap v5385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:23.707 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9108 sec 2024-04-26T21:00:24.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9109 sec 2024-04-26T21:00:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:24 smithi119 ceph-mon[32752]: pgmap v5386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:24 smithi167 ceph-mon[32751]: pgmap v5386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:24 smithi090 ceph-mon[29180]: pgmap v5386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:25.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9110 sec 2024-04-26T21:00:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8707.109477 secs 2024-04-26T21:00:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8707.109477 secs 2024-04-26T21:00:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8707.109477 secs 2024-04-26T21:00:26.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9111 sec 2024-04-26T21:00:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:26 smithi119 ceph-mon[32752]: pgmap v5387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8707.824895 secs 2024-04-26T21:00:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:26 smithi167 ceph-mon[32751]: pgmap v5387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8707.824895 secs 2024-04-26T21:00:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:26 smithi090 ceph-mon[29180]: pgmap v5387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8707.824895 secs 2024-04-26T21:00:27.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9112 sec 2024-04-26T21:00:28.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9113 sec 2024-04-26T21:00:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:28 smithi119 ceph-mon[32752]: pgmap v5388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:28 smithi167 ceph-mon[32751]: pgmap v5388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:28 smithi090 ceph-mon[29180]: pgmap v5388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:29.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9114 sec 2024-04-26T21:00:30.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9115 sec 2024-04-26T21:00:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:30 smithi119 ceph-mon[32752]: pgmap v5389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8712.109560 secs 2024-04-26T21:00:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:30 smithi167 ceph-mon[32751]: pgmap v5389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8712.109560 secs 2024-04-26T21:00:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:30 smithi090 ceph-mon[29180]: pgmap v5389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8712.109560 secs 2024-04-26T21:00:31.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9116 sec 2024-04-26T21:00:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8712.825021 secs 2024-04-26T21:00:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8712.825021 secs 2024-04-26T21:00:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8712.825021 secs 2024-04-26T21:00:32.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9117 sec 2024-04-26T21:00:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:32 smithi119 ceph-mon[32752]: pgmap v5390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:32 smithi167 ceph-mon[32751]: pgmap v5390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:32 smithi090 ceph-mon[29180]: pgmap v5390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:33.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9118 sec 2024-04-26T21:00:34.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9119 sec 2024-04-26T21:00:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:34 smithi119 ceph-mon[32752]: pgmap v5391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:34 smithi167 ceph-mon[32751]: pgmap v5391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:34 smithi090 ceph-mon[29180]: pgmap v5391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:35.708 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9120 sec 2024-04-26T21:00:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8717.109644 secs 2024-04-26T21:00:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8717.109644 secs 2024-04-26T21:00:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8717.109644 secs 2024-04-26T21:00:36.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9121 sec 2024-04-26T21:00:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:36 smithi119 ceph-mon[32752]: pgmap v5392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8717.825147 secs 2024-04-26T21:00:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:36 smithi167 ceph-mon[32751]: pgmap v5392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8717.825147 secs 2024-04-26T21:00:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:36 smithi090 ceph-mon[29180]: pgmap v5392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8717.825147 secs 2024-04-26T21:00:37.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9122 sec 2024-04-26T21:00:38.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9123 sec 2024-04-26T21:00:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:38 smithi119 ceph-mon[32752]: pgmap v5393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:38 smithi167 ceph-mon[32751]: pgmap v5393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:38 smithi090 ceph-mon[29180]: pgmap v5393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:39.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9124 sec 2024-04-26T21:00:40.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9125 sec 2024-04-26T21:00:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:40 smithi119 ceph-mon[32752]: pgmap v5394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8722.109727 secs 2024-04-26T21:00:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:40 smithi167 ceph-mon[32751]: pgmap v5394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8722.109727 secs 2024-04-26T21:00:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:40 smithi090 ceph-mon[29180]: pgmap v5394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8722.109727 secs 2024-04-26T21:00:41.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9126 sec 2024-04-26T21:00:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8722.825270 secs 2024-04-26T21:00:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8722.825270 secs 2024-04-26T21:00:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8722.825270 secs 2024-04-26T21:00:42.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9127 sec 2024-04-26T21:00:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:42 smithi119 ceph-mon[32752]: pgmap v5395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:42 smithi167 ceph-mon[32751]: pgmap v5395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:42 smithi090 ceph-mon[29180]: pgmap v5395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:43.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9128 sec 2024-04-26T21:00:44.709 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9129 sec 2024-04-26T21:00:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:44 smithi119 ceph-mon[32752]: pgmap v5396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:44 smithi167 ceph-mon[32751]: pgmap v5396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:44 smithi090 ceph-mon[29180]: pgmap v5396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:00:45.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9130 sec 2024-04-26T21:00:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8727.109828 secs 2024-04-26T21:00:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8727.109828 secs 2024-04-26T21:00:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8727.109828 secs 2024-04-26T21:00:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:00:46.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9131 sec 2024-04-26T21:00:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:46 smithi119 ceph-mon[32752]: pgmap v5397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8727.825388 secs 2024-04-26T21:00:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:00:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:00:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:00:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:46 smithi167 ceph-mon[32751]: pgmap v5397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8727.825388 secs 2024-04-26T21:00:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:00:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:00:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:00:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:46 smithi090 ceph-mon[29180]: pgmap v5397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:00:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8727.825388 secs 2024-04-26T21:00:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:00:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:00:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:00:47.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9132 sec 2024-04-26T21:00:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:47 smithi119 ceph-mon[32752]: pgmap v5398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:00:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:47 smithi119 ceph-mon[32752]: pgmap v5399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:47 smithi167 ceph-mon[32751]: pgmap v5398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:00:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:47 smithi167 ceph-mon[32751]: pgmap v5399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:47 smithi090 ceph-mon[29180]: pgmap v5398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:00:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:47 smithi090 ceph-mon[29180]: pgmap v5399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:48.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9133 sec 2024-04-26T21:00:49.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9134 sec 2024-04-26T21:00:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:49 smithi119 ceph-mon[32752]: pgmap v5400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:00:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:49 smithi167 ceph-mon[32751]: pgmap v5400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:00:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:49 smithi090 ceph-mon[29180]: pgmap v5400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:00:50.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9135 sec 2024-04-26T21:00:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8732.109882 secs 2024-04-26T21:00:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8732.109882 secs 2024-04-26T21:00:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8732.109882 secs 2024-04-26T21:00:51.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9136 sec 2024-04-26T21:00:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:51 smithi119 ceph-mon[32752]: pgmap v5401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8732.825460 secs 2024-04-26T21:00:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:51 smithi167 ceph-mon[32751]: pgmap v5401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8732.825460 secs 2024-04-26T21:00:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:51 smithi090 ceph-mon[29180]: pgmap v5401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8732.825460 secs 2024-04-26T21:00:52.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9137 sec 2024-04-26T21:00:53.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9138 sec 2024-04-26T21:00:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:53 smithi167 ceph-mon[32751]: pgmap v5402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:00:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:53 smithi090 ceph-mon[29180]: pgmap v5402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:00:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:53 smithi119 ceph-mon[32752]: pgmap v5402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:00:54.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9139 sec 2024-04-26T21:00:55.710 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9140 sec 2024-04-26T21:00:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:55 smithi167 ceph-mon[32751]: pgmap v5403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8737.109939 secs 2024-04-26T21:00:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:55 smithi090 ceph-mon[29180]: pgmap v5403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8737.109939 secs 2024-04-26T21:00:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:55 smithi119 ceph-mon[32752]: pgmap v5403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:00:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8737.109939 secs 2024-04-26T21:00:56.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9141 sec 2024-04-26T21:00:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8737.825590 secs 2024-04-26T21:00:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8737.825590 secs 2024-04-26T21:00:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8737.825590 secs 2024-04-26T21:00:57.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9142 sec 2024-04-26T21:00:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:57 smithi167 ceph-mon[32751]: pgmap v5404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:00:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:57 smithi090 ceph-mon[29180]: pgmap v5404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:00:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:57 smithi119 ceph-mon[32752]: pgmap v5404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:00:58.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9143 sec 2024-04-26T21:00:59.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9144 sec 2024-04-26T21:00:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:00:59 smithi090 ceph-mon[29180]: pgmap v5405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:00:59 smithi119 ceph-mon[32752]: pgmap v5405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:00:59 smithi167 ceph-mon[32751]: pgmap v5405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:00.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9145 sec 2024-04-26T21:01:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8742.110003 secs 2024-04-26T21:01:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8742.110003 secs 2024-04-26T21:01:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8742.110003 secs 2024-04-26T21:01:01.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9146 sec 2024-04-26T21:01:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:01 smithi167 ceph-mon[32751]: pgmap v5406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8742.825656 secs 2024-04-26T21:01:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:01 smithi090 ceph-mon[29180]: pgmap v5406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8742.825656 secs 2024-04-26T21:01:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:01 smithi119 ceph-mon[32752]: pgmap v5406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8742.825656 secs 2024-04-26T21:01:02.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9147 sec 2024-04-26T21:01:03.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9148 sec 2024-04-26T21:01:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:03 smithi090 ceph-mon[29180]: pgmap v5407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:03 smithi119 ceph-mon[32752]: pgmap v5407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:03 smithi167 ceph-mon[32751]: pgmap v5407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:04.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9149 sec 2024-04-26T21:01:05.711 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9150 sec 2024-04-26T21:01:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:05 smithi090 ceph-mon[29180]: pgmap v5408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:05 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 8747.110059 secs 2024-04-26T21:01:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:05 smithi090 ceph-mon[29180]: slow request 361.872003 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:01:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:05 smithi119 ceph-mon[32752]: pgmap v5408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:05 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 8747.110059 secs 2024-04-26T21:01:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:05 smithi119 ceph-mon[32752]: slow request 361.872003 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:01:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:05 smithi167 ceph-mon[32751]: pgmap v5408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:05 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 8747.110059 secs 2024-04-26T21:01:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:05 smithi167 ceph-mon[32751]: slow request 361.872003 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:01:06.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9151 sec 2024-04-26T21:01:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8747.825785 secs 2024-04-26T21:01:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8747.825785 secs 2024-04-26T21:01:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8747.825785 secs 2024-04-26T21:01:07.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9152 sec 2024-04-26T21:01:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:07 smithi090 ceph-mon[29180]: pgmap v5409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:07.989 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:07 smithi167 ceph-mon[32751]: pgmap v5409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:07 smithi119 ceph-mon[32752]: pgmap v5409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:08.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9153 sec 2024-04-26T21:01:09.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9154 sec 2024-04-26T21:01:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:09 smithi090 ceph-mon[29180]: pgmap v5410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:09 smithi119 ceph-mon[32752]: pgmap v5410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:09 smithi167 ceph-mon[32751]: pgmap v5410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:10.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9155 sec 2024-04-26T21:01:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8752.110117 secs 2024-04-26T21:01:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8752.110117 secs 2024-04-26T21:01:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8752.110117 secs 2024-04-26T21:01:11.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9156 sec 2024-04-26T21:01:11.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:11 smithi090 ceph-mon[29180]: pgmap v5411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:11.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8752.825869 secs 2024-04-26T21:01:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:11 smithi119 ceph-mon[32752]: pgmap v5411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8752.825869 secs 2024-04-26T21:01:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:11 smithi167 ceph-mon[32751]: pgmap v5411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8752.825869 secs 2024-04-26T21:01:12.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9157 sec 2024-04-26T21:01:13.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9158 sec 2024-04-26T21:01:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:13 smithi090 ceph-mon[29180]: pgmap v5412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:13 smithi119 ceph-mon[32752]: pgmap v5412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:13 smithi167 ceph-mon[32751]: pgmap v5412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:01:14.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9159 sec 2024-04-26T21:01:15.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9160 sec 2024-04-26T21:01:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:15 smithi090 ceph-mon[29180]: pgmap v5413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8757.110163 secs 2024-04-26T21:01:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:15 smithi119 ceph-mon[32752]: pgmap v5413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8757.110163 secs 2024-04-26T21:01:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:15 smithi167 ceph-mon[32751]: pgmap v5413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:01:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8757.110163 secs 2024-04-26T21:01:16.712 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9161 sec 2024-04-26T21:01:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8757.825997 secs 2024-04-26T21:01:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8757.825997 secs 2024-04-26T21:01:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8757.825997 secs 2024-04-26T21:01:17.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9162 sec 2024-04-26T21:01:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:17 smithi090 ceph-mon[29180]: pgmap v5414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:17 smithi119 ceph-mon[32752]: pgmap v5414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:17 smithi167 ceph-mon[32751]: pgmap v5414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:18.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9163 sec 2024-04-26T21:01:19.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9164 sec 2024-04-26T21:01:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:19 smithi090 ceph-mon[29180]: pgmap v5415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:19 smithi119 ceph-mon[32752]: pgmap v5415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:19 smithi167 ceph-mon[32751]: pgmap v5415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:20.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9165 sec 2024-04-26T21:01:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8762.110222 secs 2024-04-26T21:01:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8762.110222 secs 2024-04-26T21:01:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8762.110222 secs 2024-04-26T21:01:21.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9166 sec 2024-04-26T21:01:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:21 smithi090 ceph-mon[29180]: pgmap v5416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8762.826147 secs 2024-04-26T21:01:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:21 smithi119 ceph-mon[32752]: pgmap v5416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8762.826147 secs 2024-04-26T21:01:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:21 smithi167 ceph-mon[32751]: pgmap v5416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8762.826147 secs 2024-04-26T21:01:22.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9167 sec 2024-04-26T21:01:23.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9168 sec 2024-04-26T21:01:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:23 smithi119 ceph-mon[32752]: pgmap v5417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:23 smithi167 ceph-mon[32751]: pgmap v5417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:23 smithi090 ceph-mon[29180]: pgmap v5417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:24.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9169 sec 2024-04-26T21:01:25.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9170 sec 2024-04-26T21:01:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:25 smithi119 ceph-mon[32752]: pgmap v5418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8767.110283 secs 2024-04-26T21:01:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:25 smithi167 ceph-mon[32751]: pgmap v5418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8767.110283 secs 2024-04-26T21:01:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:25 smithi090 ceph-mon[29180]: pgmap v5418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8767.110283 secs 2024-04-26T21:01:26.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9171 sec 2024-04-26T21:01:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8767.826205 secs 2024-04-26T21:01:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8767.826205 secs 2024-04-26T21:01:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8767.826205 secs 2024-04-26T21:01:27.713 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9172 sec 2024-04-26T21:01:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:27 smithi119 ceph-mon[32752]: pgmap v5419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:27 smithi167 ceph-mon[32751]: pgmap v5419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:27 smithi090 ceph-mon[29180]: pgmap v5419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:28.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9173 sec 2024-04-26T21:01:29.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9174 sec 2024-04-26T21:01:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:29 smithi119 ceph-mon[32752]: pgmap v5420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:29 smithi167 ceph-mon[32751]: pgmap v5420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:29 smithi090 ceph-mon[29180]: pgmap v5420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:30.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9175 sec 2024-04-26T21:01:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8772.110345 secs 2024-04-26T21:01:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:30 smithi119 ceph-mon[32752]: pgmap v5421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8772.826271 secs 2024-04-26T21:01:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8772.110345 secs 2024-04-26T21:01:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:30 smithi167 ceph-mon[32751]: pgmap v5421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8772.826271 secs 2024-04-26T21:01:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8772.110345 secs 2024-04-26T21:01:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:30 smithi090 ceph-mon[29180]: pgmap v5421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8772.826271 secs 2024-04-26T21:01:31.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9176 sec 2024-04-26T21:01:32.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9177 sec 2024-04-26T21:01:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:33 smithi119 ceph-mon[32752]: pgmap v5422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:33 smithi167 ceph-mon[32751]: pgmap v5422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:33 smithi090 ceph-mon[29180]: pgmap v5422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:33.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9178 sec 2024-04-26T21:01:34.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9179 sec 2024-04-26T21:01:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:35 smithi119 ceph-mon[32752]: pgmap v5423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:35.567 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8777.110404 secs 2024-04-26T21:01:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:35 smithi167 ceph-mon[32751]: pgmap v5423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8777.110404 secs 2024-04-26T21:01:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:35 smithi090 ceph-mon[29180]: pgmap v5423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8777.110404 secs 2024-04-26T21:01:35.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9180 sec 2024-04-26T21:01:36.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8777.826425 secs 2024-04-26T21:01:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8777.826425 secs 2024-04-26T21:01:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8777.826425 secs 2024-04-26T21:01:36.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9181 sec 2024-04-26T21:01:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:37 smithi119 ceph-mon[32752]: pgmap v5424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:37 smithi167 ceph-mon[32751]: pgmap v5424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:37 smithi090 ceph-mon[29180]: pgmap v5424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:37.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9182 sec 2024-04-26T21:01:38.714 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9183 sec 2024-04-26T21:01:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:39 smithi119 ceph-mon[32752]: pgmap v5425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:39 smithi167 ceph-mon[32751]: pgmap v5425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:39 smithi090 ceph-mon[29180]: pgmap v5425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:39.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9184 sec 2024-04-26T21:01:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8782.110436 secs 2024-04-26T21:01:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8782.110436 secs 2024-04-26T21:01:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8782.110436 secs 2024-04-26T21:01:40.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9185 sec 2024-04-26T21:01:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:41 smithi119 ceph-mon[32752]: pgmap v5426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8782.826524 secs 2024-04-26T21:01:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:41 smithi167 ceph-mon[32751]: pgmap v5426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8782.826524 secs 2024-04-26T21:01:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:41 smithi090 ceph-mon[29180]: pgmap v5426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8782.826524 secs 2024-04-26T21:01:41.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9186 sec 2024-04-26T21:01:42.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9187 sec 2024-04-26T21:01:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:43 smithi119 ceph-mon[32752]: pgmap v5427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:43 smithi167 ceph-mon[32751]: pgmap v5427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:43 smithi090 ceph-mon[29180]: pgmap v5427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:43.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9188 sec 2024-04-26T21:01:44.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9189 sec 2024-04-26T21:01:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:45 smithi119 ceph-mon[32752]: pgmap v5428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8787.110496 secs 2024-04-26T21:01:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:45 smithi167 ceph-mon[32751]: pgmap v5428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8787.110496 secs 2024-04-26T21:01:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:45 smithi090 ceph-mon[29180]: pgmap v5428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:01:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8787.110496 secs 2024-04-26T21:01:45.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9190 sec 2024-04-26T21:01:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8787.826604 secs 2024-04-26T21:01:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8787.826604 secs 2024-04-26T21:01:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8787.826604 secs 2024-04-26T21:01:46.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9191 sec 2024-04-26T21:01:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:47 smithi119 ceph-mon[32752]: pgmap v5429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:01:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:47 smithi167 ceph-mon[32751]: pgmap v5429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:01:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:47 smithi090 ceph-mon[29180]: pgmap v5429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:01:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:01:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:01:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:01:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:01:47.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9192 sec 2024-04-26T21:01:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:48 smithi119 ceph-mon[32752]: pgmap v5430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:48 smithi167 ceph-mon[32751]: pgmap v5430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:48 smithi090 ceph-mon[29180]: pgmap v5430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:48.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9193 sec 2024-04-26T21:01:49.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9194 sec 2024-04-26T21:01:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:50 smithi119 ceph-mon[32752]: pgmap v5431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:01:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8792.110555 secs 2024-04-26T21:01:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:50 smithi167 ceph-mon[32751]: pgmap v5431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:01:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8792.110555 secs 2024-04-26T21:01:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:50 smithi090 ceph-mon[29180]: pgmap v5431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:01:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8792.110555 secs 2024-04-26T21:01:50.715 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9195 sec 2024-04-26T21:01:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8792.826727 secs 2024-04-26T21:01:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8792.826727 secs 2024-04-26T21:01:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8792.826727 secs 2024-04-26T21:01:51.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9196 sec 2024-04-26T21:01:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:52 smithi119 ceph-mon[32752]: pgmap v5432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:52 smithi167 ceph-mon[32751]: pgmap v5432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:52 smithi090 ceph-mon[29180]: pgmap v5432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:52.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9197 sec 2024-04-26T21:01:53.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9198 sec 2024-04-26T21:01:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:54 smithi119 ceph-mon[32752]: pgmap v5433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:54 smithi167 ceph-mon[32751]: pgmap v5433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:54 smithi090 ceph-mon[29180]: pgmap v5433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:54.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9199 sec 2024-04-26T21:01:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8797.110614 secs 2024-04-26T21:01:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8797.110614 secs 2024-04-26T21:01:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8797.110614 secs 2024-04-26T21:01:55.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9200 sec 2024-04-26T21:01:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:56 smithi119 ceph-mon[32752]: pgmap v5434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8797.826824 secs 2024-04-26T21:01:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:56 smithi167 ceph-mon[32751]: pgmap v5434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8797.826824 secs 2024-04-26T21:01:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:56 smithi090 ceph-mon[29180]: pgmap v5434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8797.826824 secs 2024-04-26T21:01:56.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9201 sec 2024-04-26T21:01:57.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9202 sec 2024-04-26T21:01:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:01:58 smithi119 ceph-mon[32752]: pgmap v5435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:01:58 smithi167 ceph-mon[32751]: pgmap v5435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:01:58 smithi090 ceph-mon[29180]: pgmap v5435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:01:58.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9203 sec 2024-04-26T21:01:59.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9204 sec 2024-04-26T21:02:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:00 smithi119 ceph-mon[32752]: pgmap v5436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8802.110668 secs 2024-04-26T21:02:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:00 smithi167 ceph-mon[32751]: pgmap v5436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8802.110668 secs 2024-04-26T21:02:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:00 smithi090 ceph-mon[29180]: pgmap v5436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8802.110668 secs 2024-04-26T21:02:00.716 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9205 sec 2024-04-26T21:02:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8802.826938 secs 2024-04-26T21:02:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8802.826938 secs 2024-04-26T21:02:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8802.826938 secs 2024-04-26T21:02:01.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9206 sec 2024-04-26T21:02:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:02 smithi119 ceph-mon[32752]: pgmap v5437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:02 smithi167 ceph-mon[32751]: pgmap v5437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:02 smithi090 ceph-mon[29180]: pgmap v5437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:02.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9207 sec 2024-04-26T21:02:03.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9208 sec 2024-04-26T21:02:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:04 smithi119 ceph-mon[32752]: pgmap v5438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:04 smithi167 ceph-mon[32751]: pgmap v5438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:04 smithi090 ceph-mon[29180]: pgmap v5438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:04.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9209 sec 2024-04-26T21:02:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8807.110726 secs 2024-04-26T21:02:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8807.110726 secs 2024-04-26T21:02:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8807.110726 secs 2024-04-26T21:02:05.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9210 sec 2024-04-26T21:02:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:06 smithi119 ceph-mon[32752]: pgmap v5439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8807.827071 secs 2024-04-26T21:02:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:06 smithi167 ceph-mon[32751]: pgmap v5439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8807.827071 secs 2024-04-26T21:02:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:06 smithi090 ceph-mon[29180]: pgmap v5439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8807.827071 secs 2024-04-26T21:02:06.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9211 sec 2024-04-26T21:02:07.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9212 sec 2024-04-26T21:02:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:08 smithi119 ceph-mon[32752]: pgmap v5440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:08 smithi167 ceph-mon[32751]: pgmap v5440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:08 smithi090 ceph-mon[29180]: pgmap v5440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:08.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9213 sec 2024-04-26T21:02:09.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9214 sec 2024-04-26T21:02:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:10 smithi119 ceph-mon[32752]: pgmap v5441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8812.110775 secs 2024-04-26T21:02:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:10 smithi167 ceph-mon[32751]: pgmap v5441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8812.110775 secs 2024-04-26T21:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:10 smithi090 ceph-mon[29180]: pgmap v5441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8812.110775 secs 2024-04-26T21:02:10.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9215 sec 2024-04-26T21:02:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8812.827173 secs 2024-04-26T21:02:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8812.827173 secs 2024-04-26T21:02:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8812.827173 secs 2024-04-26T21:02:11.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9216 sec 2024-04-26T21:02:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:12 smithi119 ceph-mon[32752]: pgmap v5442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:12 smithi167 ceph-mon[32751]: pgmap v5442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:12 smithi090 ceph-mon[29180]: pgmap v5442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:12.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9217 sec 2024-04-26T21:02:13.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9218 sec 2024-04-26T21:02:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:14 smithi119 ceph-mon[32752]: pgmap v5443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:14 smithi167 ceph-mon[32751]: pgmap v5443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:14 smithi090 ceph-mon[29180]: pgmap v5443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:14.717 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9219 sec 2024-04-26T21:02:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8817.110835 secs 2024-04-26T21:02:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8817.110835 secs 2024-04-26T21:02:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8817.110835 secs 2024-04-26T21:02:15.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9220 sec 2024-04-26T21:02:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:16 smithi119 ceph-mon[32752]: pgmap v5444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8817.827280 secs 2024-04-26T21:02:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:16 smithi167 ceph-mon[32751]: pgmap v5444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8817.827280 secs 2024-04-26T21:02:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:16 smithi090 ceph-mon[29180]: pgmap v5444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8817.827280 secs 2024-04-26T21:02:16.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9221 sec 2024-04-26T21:02:17.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9222 sec 2024-04-26T21:02:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:18 smithi119 ceph-mon[32752]: pgmap v5445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:18 smithi167 ceph-mon[32751]: pgmap v5445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:18 smithi090 ceph-mon[29180]: pgmap v5445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:18.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9223 sec 2024-04-26T21:02:19.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9224 sec 2024-04-26T21:02:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:20 smithi119 ceph-mon[32752]: pgmap v5446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8822.110894 secs 2024-04-26T21:02:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:20 smithi167 ceph-mon[32751]: pgmap v5446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8822.110894 secs 2024-04-26T21:02:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:20 smithi090 ceph-mon[29180]: pgmap v5446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8822.110894 secs 2024-04-26T21:02:20.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9225 sec 2024-04-26T21:02:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8822.827412 secs 2024-04-26T21:02:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8822.827412 secs 2024-04-26T21:02:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8822.827412 secs 2024-04-26T21:02:21.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9226 sec 2024-04-26T21:02:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:22 smithi119 ceph-mon[32752]: pgmap v5447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:22 smithi167 ceph-mon[32751]: pgmap v5447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:22 smithi090 ceph-mon[29180]: pgmap v5447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:22.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9227 sec 2024-04-26T21:02:23.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9228 sec 2024-04-26T21:02:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:24 smithi119 ceph-mon[32752]: pgmap v5448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:24 smithi167 ceph-mon[32751]: pgmap v5448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:24 smithi090 ceph-mon[29180]: pgmap v5448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:24.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9229 sec 2024-04-26T21:02:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8827.110957 secs 2024-04-26T21:02:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8827.110957 secs 2024-04-26T21:02:25.718 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9230 sec 2024-04-26T21:02:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8827.110957 secs 2024-04-26T21:02:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:26 smithi167 ceph-mon[32751]: pgmap v5449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8827.827528 secs 2024-04-26T21:02:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:26 smithi090 ceph-mon[29180]: pgmap v5449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8827.827528 secs 2024-04-26T21:02:26.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9231 sec 2024-04-26T21:02:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:26 smithi119 ceph-mon[32752]: pgmap v5449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8827.827528 secs 2024-04-26T21:02:27.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9232 sec 2024-04-26T21:02:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:28 smithi167 ceph-mon[32751]: pgmap v5450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:28 smithi090 ceph-mon[29180]: pgmap v5450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:28.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9233 sec 2024-04-26T21:02:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:28 smithi119 ceph-mon[32752]: pgmap v5450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:02:29.646+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:02:29.647 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:02:29.646+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:02:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:02:29.646+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 71 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:02:29.648 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:02:29.646+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 71 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:02:29.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9234 sec 2024-04-26T21:02:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:30 smithi167 ceph-mon[32751]: pgmap v5451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8832.111018 secs 2024-04-26T21:02:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:30 smithi090 ceph-mon[29180]: pgmap v5451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8832.111018 secs 2024-04-26T21:02:30.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9235 sec 2024-04-26T21:02:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:30 smithi119 ceph-mon[32752]: pgmap v5451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8832.111018 secs 2024-04-26T21:02:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8832.827594 secs 2024-04-26T21:02:31.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9236 sec 2024-04-26T21:02:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8832.827594 secs 2024-04-26T21:02:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8832.827594 secs 2024-04-26T21:02:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:32 smithi090 ceph-mon[29180]: pgmap v5452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:32.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9237 sec 2024-04-26T21:02:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:32 smithi119 ceph-mon[32752]: pgmap v5452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:32 smithi167 ceph-mon[32751]: pgmap v5452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:33.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9238 sec 2024-04-26T21:02:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:34 smithi090 ceph-mon[29180]: pgmap v5453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:34.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9239 sec 2024-04-26T21:02:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:34 smithi119 ceph-mon[32752]: pgmap v5453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:34 smithi167 ceph-mon[32751]: pgmap v5453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8837.111080 secs 2024-04-26T21:02:35.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9240 sec 2024-04-26T21:02:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8837.111080 secs 2024-04-26T21:02:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8837.111080 secs 2024-04-26T21:02:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:36 smithi090 ceph-mon[29180]: pgmap v5454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8837.827740 secs 2024-04-26T21:02:36.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9241 sec 2024-04-26T21:02:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:36 smithi167 ceph-mon[32751]: pgmap v5454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8837.827740 secs 2024-04-26T21:02:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:36 smithi119 ceph-mon[32752]: pgmap v5454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8837.827740 secs 2024-04-26T21:02:37.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9242 sec 2024-04-26T21:02:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:38 smithi090 ceph-mon[29180]: pgmap v5455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:38.719 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9243 sec 2024-04-26T21:02:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:38 smithi119 ceph-mon[32752]: pgmap v5455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:38 smithi167 ceph-mon[32751]: pgmap v5455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:39.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9244 sec 2024-04-26T21:02:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:40 smithi090 ceph-mon[29180]: pgmap v5456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8842.111142 secs 2024-04-26T21:02:40.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9245 sec 2024-04-26T21:02:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:40 smithi119 ceph-mon[32752]: pgmap v5456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8842.111142 secs 2024-04-26T21:02:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:40 smithi167 ceph-mon[32751]: pgmap v5456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8842.111142 secs 2024-04-26T21:02:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8842.827871 secs 2024-04-26T21:02:41.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9246 sec 2024-04-26T21:02:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8842.827871 secs 2024-04-26T21:02:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8842.827871 secs 2024-04-26T21:02:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:42 smithi090 ceph-mon[29180]: pgmap v5457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:42.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9247 sec 2024-04-26T21:02:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:42 smithi119 ceph-mon[32752]: pgmap v5457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:42 smithi167 ceph-mon[32751]: pgmap v5457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:43.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9248 sec 2024-04-26T21:02:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:44 smithi090 ceph-mon[29180]: pgmap v5458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:44.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9249 sec 2024-04-26T21:02:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:44 smithi119 ceph-mon[32752]: pgmap v5458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:44 smithi167 ceph-mon[32751]: pgmap v5458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8847.111202 secs 2024-04-26T21:02:45.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9250 sec 2024-04-26T21:02:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8847.111202 secs 2024-04-26T21:02:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8847.111202 secs 2024-04-26T21:02:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:46 smithi090 ceph-mon[29180]: pgmap v5459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8847.827942 secs 2024-04-26T21:02:46.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:46 smithi119 ceph-mon[32752]: pgmap v5459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:46.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8847.827942 secs 2024-04-26T21:02:46.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9251 sec 2024-04-26T21:02:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:46 smithi167 ceph-mon[32751]: pgmap v5459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:02:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8847.827942 secs 2024-04-26T21:02:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:47.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9252 sec 2024-04-26T21:02:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:02:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:48 smithi090 ceph-mon[29180]: pgmap v5460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:02:48.720 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9253 sec 2024-04-26T21:02:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:48 smithi119 ceph-mon[32752]: pgmap v5460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:02:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:48 smithi167 ceph-mon[32751]: pgmap v5460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:02:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:02:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:02:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:02:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:49 smithi090 ceph-mon[29180]: pgmap v5461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:02:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:49 smithi090 ceph-mon[29180]: pgmap v5462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:02:49.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9254 sec 2024-04-26T21:02:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:49 smithi119 ceph-mon[32752]: pgmap v5461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:02:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:49 smithi119 ceph-mon[32752]: pgmap v5462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:02:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:49 smithi167 ceph-mon[32751]: pgmap v5461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:02:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:49 smithi167 ceph-mon[32751]: pgmap v5462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:02:50.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9255 sec 2024-04-26T21:02:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8852.111262 secs 2024-04-26T21:02:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8852.111262 secs 2024-04-26T21:02:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8852.111262 secs 2024-04-26T21:02:51.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9256 sec 2024-04-26T21:02:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:51 smithi119 ceph-mon[32752]: pgmap v5463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8852.828058 secs 2024-04-26T21:02:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:51 smithi167 ceph-mon[32751]: pgmap v5463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8852.828058 secs 2024-04-26T21:02:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:51 smithi090 ceph-mon[29180]: pgmap v5463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8852.828058 secs 2024-04-26T21:02:52.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9257 sec 2024-04-26T21:02:53.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9258 sec 2024-04-26T21:02:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:53 smithi119 ceph-mon[32752]: pgmap v5464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:53 smithi167 ceph-mon[32751]: pgmap v5464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:53 smithi090 ceph-mon[29180]: pgmap v5464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:54.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9259 sec 2024-04-26T21:02:55.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9260 sec 2024-04-26T21:02:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:55 smithi119 ceph-mon[32752]: pgmap v5465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8857.111320 secs 2024-04-26T21:02:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:55 smithi167 ceph-mon[32751]: pgmap v5465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8857.111320 secs 2024-04-26T21:02:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:55 smithi090 ceph-mon[29180]: pgmap v5465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8857.111320 secs 2024-04-26T21:02:56.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9261 sec 2024-04-26T21:02:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8857.828155 secs 2024-04-26T21:02:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8857.828155 secs 2024-04-26T21:02:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8857.828155 secs 2024-04-26T21:02:57.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9262 sec 2024-04-26T21:02:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:57 smithi119 ceph-mon[32752]: pgmap v5466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:57 smithi167 ceph-mon[32751]: pgmap v5466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:57 smithi090 ceph-mon[29180]: pgmap v5466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:02:58.721 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9263 sec 2024-04-26T21:02:59.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9264 sec 2024-04-26T21:02:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:02:59 smithi119 ceph-mon[32752]: pgmap v5467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:02:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:02:59 smithi167 ceph-mon[32751]: pgmap v5467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:02:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:02:59 smithi090 ceph-mon[29180]: pgmap v5467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:03:00.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9265 sec 2024-04-26T21:03:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8862.111379 secs 2024-04-26T21:03:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8862.111379 secs 2024-04-26T21:03:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8862.111379 secs 2024-04-26T21:03:01.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9266 sec 2024-04-26T21:03:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:01 smithi119 ceph-mon[32752]: pgmap v5468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8862.828269 secs 2024-04-26T21:03:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:01 smithi167 ceph-mon[32751]: pgmap v5468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8862.828269 secs 2024-04-26T21:03:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:01 smithi090 ceph-mon[29180]: pgmap v5468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8862.828269 secs 2024-04-26T21:03:02.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9267 sec 2024-04-26T21:03:03.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9268 sec 2024-04-26T21:03:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:03 smithi119 ceph-mon[32752]: pgmap v5469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:03 smithi167 ceph-mon[32751]: pgmap v5469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:03 smithi090 ceph-mon[29180]: pgmap v5469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:04.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9269 sec 2024-04-26T21:03:05.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9270 sec 2024-04-26T21:03:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:05 smithi119 ceph-mon[32752]: pgmap v5470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8867.111440 secs 2024-04-26T21:03:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:05 smithi167 ceph-mon[32751]: pgmap v5470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8867.111440 secs 2024-04-26T21:03:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:05 smithi090 ceph-mon[29180]: pgmap v5470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8867.111440 secs 2024-04-26T21:03:06.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:06 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 8867.828406 secs 2024-04-26T21:03:06.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:06 smithi167 ceph-mon[32751]: slow request 182.465173 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:03:06.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9271 sec 2024-04-26T21:03:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:06 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 8867.828406 secs 2024-04-26T21:03:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:06 smithi119 ceph-mon[32752]: slow request 182.465173 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:03:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:06 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 8867.828406 secs 2024-04-26T21:03:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:06 smithi090 ceph-mon[29180]: slow request 182.465173 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:03:07.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9272 sec 2024-04-26T21:03:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:07 smithi119 ceph-mon[32752]: pgmap v5471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T21:03:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:03:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:07 smithi167 ceph-mon[32751]: pgmap v5471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T21:03:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:03:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:07 smithi090 ceph-mon[29180]: pgmap v5471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T21:03:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:03:08.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9273 sec 2024-04-26T21:03:09.722 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9274 sec 2024-04-26T21:03:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:09 smithi119 ceph-mon[32752]: pgmap v5472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:09 smithi167 ceph-mon[32751]: pgmap v5472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:09 smithi090 ceph-mon[29180]: pgmap v5472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:10.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9275 sec 2024-04-26T21:03:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8872.111499 secs 2024-04-26T21:03:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8872.111499 secs 2024-04-26T21:03:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8872.111499 secs 2024-04-26T21:03:11.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9276 sec 2024-04-26T21:03:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:11 smithi119 ceph-mon[32752]: pgmap v5473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8872.828448 secs 2024-04-26T21:03:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:11 smithi167 ceph-mon[32751]: pgmap v5473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8872.828448 secs 2024-04-26T21:03:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:11 smithi090 ceph-mon[29180]: pgmap v5473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8872.828448 secs 2024-04-26T21:03:12.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9277 sec 2024-04-26T21:03:13.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9278 sec 2024-04-26T21:03:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:13 smithi119 ceph-mon[32752]: pgmap v5474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:13 smithi167 ceph-mon[32751]: pgmap v5474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:13 smithi090 ceph-mon[29180]: pgmap v5474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:14.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9279 sec 2024-04-26T21:03:15.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9280 sec 2024-04-26T21:03:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:15 smithi119 ceph-mon[32752]: pgmap v5475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8877.111559 secs 2024-04-26T21:03:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:15 smithi167 ceph-mon[32751]: pgmap v5475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8877.111559 secs 2024-04-26T21:03:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:15 smithi090 ceph-mon[29180]: pgmap v5475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8877.111559 secs 2024-04-26T21:03:16.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9281 sec 2024-04-26T21:03:16.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8877.828563 secs 2024-04-26T21:03:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8877.828563 secs 2024-04-26T21:03:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8877.828563 secs 2024-04-26T21:03:17.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9282 sec 2024-04-26T21:03:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:17 smithi119 ceph-mon[32752]: pgmap v5476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:17 smithi167 ceph-mon[32751]: pgmap v5476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:17 smithi090 ceph-mon[29180]: pgmap v5476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:18.723 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9283 sec 2024-04-26T21:03:19.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9284 sec 2024-04-26T21:03:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:19 smithi119 ceph-mon[32752]: pgmap v5477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:19 smithi167 ceph-mon[32751]: pgmap v5477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:19 smithi090 ceph-mon[29180]: pgmap v5477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:20.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9285 sec 2024-04-26T21:03:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8882.111621 secs 2024-04-26T21:03:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8882.111621 secs 2024-04-26T21:03:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8882.111621 secs 2024-04-26T21:03:21.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9286 sec 2024-04-26T21:03:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:21 smithi119 ceph-mon[32752]: pgmap v5478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8882.828700 secs 2024-04-26T21:03:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:21 smithi167 ceph-mon[32751]: pgmap v5478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8882.828700 secs 2024-04-26T21:03:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:21 smithi090 ceph-mon[29180]: pgmap v5478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8882.828700 secs 2024-04-26T21:03:22.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9287 sec 2024-04-26T21:03:23.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9288 sec 2024-04-26T21:03:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:23 smithi119 ceph-mon[32752]: pgmap v5479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:23 smithi167 ceph-mon[32751]: pgmap v5479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:23 smithi090 ceph-mon[29180]: pgmap v5479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:24.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9289 sec 2024-04-26T21:03:25.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9290 sec 2024-04-26T21:03:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:25 smithi119 ceph-mon[32752]: pgmap v5480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8887.111678 secs 2024-04-26T21:03:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:25 smithi167 ceph-mon[32751]: pgmap v5480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8887.111678 secs 2024-04-26T21:03:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:25 smithi090 ceph-mon[29180]: pgmap v5480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8887.111678 secs 2024-04-26T21:03:26.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9291 sec 2024-04-26T21:03:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8887.828805 secs 2024-04-26T21:03:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8887.828805 secs 2024-04-26T21:03:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8887.828805 secs 2024-04-26T21:03:27.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9292 sec 2024-04-26T21:03:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:27 smithi119 ceph-mon[32752]: pgmap v5481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:27 smithi167 ceph-mon[32751]: pgmap v5481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:27 smithi090 ceph-mon[29180]: pgmap v5481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:28.724 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9293 sec 2024-04-26T21:03:29.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9294 sec 2024-04-26T21:03:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:29 smithi119 ceph-mon[32752]: pgmap v5482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:29 smithi167 ceph-mon[32751]: pgmap v5482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:29 smithi090 ceph-mon[29180]: pgmap v5482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:30.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9295 sec 2024-04-26T21:03:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8892.111737 secs 2024-04-26T21:03:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8892.111737 secs 2024-04-26T21:03:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8892.111737 secs 2024-04-26T21:03:31.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9296 sec 2024-04-26T21:03:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:31 smithi119 ceph-mon[32752]: pgmap v5483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8892.828943 secs 2024-04-26T21:03:31.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:31 smithi090 ceph-mon[29180]: pgmap v5483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:31.814 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8892.828943 secs 2024-04-26T21:03:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:31 smithi167 ceph-mon[32751]: pgmap v5483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8892.828943 secs 2024-04-26T21:03:32.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9297 sec 2024-04-26T21:03:33.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9298 sec 2024-04-26T21:03:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:33 smithi119 ceph-mon[32752]: pgmap v5484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:33 smithi167 ceph-mon[32751]: pgmap v5484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:33 smithi090 ceph-mon[29180]: pgmap v5484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:34.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9299 sec 2024-04-26T21:03:35.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9300 sec 2024-04-26T21:03:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:35 smithi119 ceph-mon[32752]: pgmap v5485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8897.111821 secs 2024-04-26T21:03:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:35 smithi167 ceph-mon[32751]: pgmap v5485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8897.111821 secs 2024-04-26T21:03:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:35 smithi090 ceph-mon[29180]: pgmap v5485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8897.111821 secs 2024-04-26T21:03:36.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9301 sec 2024-04-26T21:03:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8897.829022 secs 2024-04-26T21:03:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8897.829022 secs 2024-04-26T21:03:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8897.829022 secs 2024-04-26T21:03:37.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9302 sec 2024-04-26T21:03:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:37 smithi119 ceph-mon[32752]: pgmap v5486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:37 smithi167 ceph-mon[32751]: pgmap v5486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:37 smithi090 ceph-mon[29180]: pgmap v5486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:38.725 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9303 sec 2024-04-26T21:03:39.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9304 sec 2024-04-26T21:03:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:39 smithi119 ceph-mon[32752]: pgmap v5487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:39 smithi167 ceph-mon[32751]: pgmap v5487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:39 smithi090 ceph-mon[29180]: pgmap v5487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:40.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9305 sec 2024-04-26T21:03:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8902.111910 secs 2024-04-26T21:03:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8902.111910 secs 2024-04-26T21:03:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8902.111910 secs 2024-04-26T21:03:41.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9306 sec 2024-04-26T21:03:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:41 smithi119 ceph-mon[32752]: pgmap v5488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8902.829144 secs 2024-04-26T21:03:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:41 smithi167 ceph-mon[32751]: pgmap v5488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8902.829144 secs 2024-04-26T21:03:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:41 smithi090 ceph-mon[29180]: pgmap v5488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8902.829144 secs 2024-04-26T21:03:42.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9307 sec 2024-04-26T21:03:43.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9308 sec 2024-04-26T21:03:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:43 smithi119 ceph-mon[32752]: pgmap v5489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:43 smithi167 ceph-mon[32751]: pgmap v5489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:43 smithi090 ceph-mon[29180]: pgmap v5489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:44.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9309 sec 2024-04-26T21:03:45.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9310 sec 2024-04-26T21:03:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:45 smithi119 ceph-mon[32752]: pgmap v5490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8907.111970 secs 2024-04-26T21:03:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:45 smithi167 ceph-mon[32751]: pgmap v5490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8907.111970 secs 2024-04-26T21:03:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:45 smithi090 ceph-mon[29180]: pgmap v5490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:03:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8907.111970 secs 2024-04-26T21:03:46.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9311 sec 2024-04-26T21:03:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8907.829286 secs 2024-04-26T21:03:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8907.829286 secs 2024-04-26T21:03:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8907.829286 secs 2024-04-26T21:03:47.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9312 sec 2024-04-26T21:03:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:47 smithi119 ceph-mon[32752]: pgmap v5491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:47 smithi167 ceph-mon[32751]: pgmap v5491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:47 smithi090 ceph-mon[29180]: pgmap v5491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:48.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9313 sec 2024-04-26T21:03:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:03:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:03:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:03:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:03:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:03:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:03:49.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9314 sec 2024-04-26T21:03:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:49 smithi119 ceph-mon[32752]: pgmap v5492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:49 smithi119 ceph-mon[32752]: pgmap v5493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:03:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:49 smithi119 ceph-mon[32752]: pgmap v5494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:49 smithi167 ceph-mon[32751]: pgmap v5492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:49 smithi167 ceph-mon[32751]: pgmap v5493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:03:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:49 smithi167 ceph-mon[32751]: pgmap v5494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:49 smithi090 ceph-mon[29180]: pgmap v5492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:03:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:49 smithi090 ceph-mon[29180]: pgmap v5493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:03:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:49 smithi090 ceph-mon[29180]: pgmap v5494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:50.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9315 sec 2024-04-26T21:03:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8912.112057 secs 2024-04-26T21:03:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8912.112057 secs 2024-04-26T21:03:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8912.112057 secs 2024-04-26T21:03:51.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9316 sec 2024-04-26T21:03:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:51 smithi119 ceph-mon[32752]: pgmap v5495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8912.829430 secs 2024-04-26T21:03:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:51 smithi167 ceph-mon[32751]: pgmap v5495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8912.829430 secs 2024-04-26T21:03:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:51 smithi090 ceph-mon[29180]: pgmap v5495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8912.829430 secs 2024-04-26T21:03:52.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9317 sec 2024-04-26T21:03:53.726 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9318 sec 2024-04-26T21:03:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:53 smithi119 ceph-mon[32752]: pgmap v5496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:03:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:53 smithi167 ceph-mon[32751]: pgmap v5496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:03:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:53 smithi090 ceph-mon[29180]: pgmap v5496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:03:54.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9319 sec 2024-04-26T21:03:55.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9320 sec 2024-04-26T21:03:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:55 smithi119 ceph-mon[32752]: pgmap v5497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8917.112141 secs 2024-04-26T21:03:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:55 smithi167 ceph-mon[32751]: pgmap v5497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8917.112141 secs 2024-04-26T21:03:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:55 smithi090 ceph-mon[29180]: pgmap v5497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8917.112141 secs 2024-04-26T21:03:56.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9321 sec 2024-04-26T21:03:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8917.829551 secs 2024-04-26T21:03:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8917.829551 secs 2024-04-26T21:03:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8917.829551 secs 2024-04-26T21:03:57.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9322 sec 2024-04-26T21:03:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:57 smithi167 ceph-mon[32751]: pgmap v5498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:57 smithi090 ceph-mon[29180]: pgmap v5498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:57 smithi119 ceph-mon[32752]: pgmap v5498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:03:58.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9323 sec 2024-04-26T21:03:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:03:59.659+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:03:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:03:59.659+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:03:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:03:59.659+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 72 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:03:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:03:59.659+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 72 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:03:59.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9324 sec 2024-04-26T21:03:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:03:59 smithi167 ceph-mon[32751]: pgmap v5499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:03:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:03:59 smithi090 ceph-mon[29180]: pgmap v5499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:04:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:03:59 smithi119 ceph-mon[32752]: pgmap v5499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:04:00.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9325 sec 2024-04-26T21:04:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8922.112229 secs 2024-04-26T21:04:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8922.112229 secs 2024-04-26T21:04:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8922.112229 secs 2024-04-26T21:04:01.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9326 sec 2024-04-26T21:04:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:01 smithi090 ceph-mon[29180]: pgmap v5500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8922.829647 secs 2024-04-26T21:04:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:01 smithi119 ceph-mon[32752]: pgmap v5500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8922.829647 secs 2024-04-26T21:04:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:01 smithi167 ceph-mon[32751]: pgmap v5500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8922.829647 secs 2024-04-26T21:04:02.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9327 sec 2024-04-26T21:04:03.727 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9328 sec 2024-04-26T21:04:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:03 smithi090 ceph-mon[29180]: pgmap v5501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:03 smithi119 ceph-mon[32752]: pgmap v5501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:03 smithi167 ceph-mon[32751]: pgmap v5501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:04.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9329 sec 2024-04-26T21:04:05.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9330 sec 2024-04-26T21:04:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:05 smithi090 ceph-mon[29180]: pgmap v5502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8927.112318 secs 2024-04-26T21:04:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:05 smithi119 ceph-mon[32752]: pgmap v5502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8927.112318 secs 2024-04-26T21:04:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:05 smithi167 ceph-mon[32751]: pgmap v5502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8927.112318 secs 2024-04-26T21:04:06.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9331 sec 2024-04-26T21:04:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8927.829746 secs 2024-04-26T21:04:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8927.829746 secs 2024-04-26T21:04:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8927.829746 secs 2024-04-26T21:04:07.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9332 sec 2024-04-26T21:04:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:07 smithi090 ceph-mon[29180]: pgmap v5503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:07 smithi119 ceph-mon[32752]: pgmap v5503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:07 smithi167 ceph-mon[32751]: pgmap v5503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:08.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9333 sec 2024-04-26T21:04:09.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9334 sec 2024-04-26T21:04:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:09 smithi090 ceph-mon[29180]: pgmap v5504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:09 smithi119 ceph-mon[32752]: pgmap v5504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:09 smithi167 ceph-mon[32751]: pgmap v5504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:10.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9335 sec 2024-04-26T21:04:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8932.112403 secs 2024-04-26T21:04:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8932.112403 secs 2024-04-26T21:04:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8932.112403 secs 2024-04-26T21:04:11.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9336 sec 2024-04-26T21:04:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:11 smithi090 ceph-mon[29180]: pgmap v5505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8932.829859 secs 2024-04-26T21:04:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:11 smithi119 ceph-mon[32752]: pgmap v5505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8932.829859 secs 2024-04-26T21:04:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:11 smithi167 ceph-mon[32751]: pgmap v5505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8932.829859 secs 2024-04-26T21:04:12.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9337 sec 2024-04-26T21:04:13.728 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9338 sec 2024-04-26T21:04:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:13 smithi090 ceph-mon[29180]: pgmap v5506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:13 smithi119 ceph-mon[32752]: pgmap v5506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:13 smithi167 ceph-mon[32751]: pgmap v5506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:14.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9339 sec 2024-04-26T21:04:15.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9340 sec 2024-04-26T21:04:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:15 smithi090 ceph-mon[29180]: pgmap v5507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8937.112463 secs 2024-04-26T21:04:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:15 smithi119 ceph-mon[32752]: pgmap v5507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8937.112463 secs 2024-04-26T21:04:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:15 smithi167 ceph-mon[32751]: pgmap v5507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8937.112463 secs 2024-04-26T21:04:16.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9341 sec 2024-04-26T21:04:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8937.829985 secs 2024-04-26T21:04:16.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8937.829985 secs 2024-04-26T21:04:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8937.829985 secs 2024-04-26T21:04:17.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9342 sec 2024-04-26T21:04:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:17 smithi090 ceph-mon[29180]: pgmap v5508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:17 smithi119 ceph-mon[32752]: pgmap v5508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:17 smithi167 ceph-mon[32751]: pgmap v5508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:18.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9343 sec 2024-04-26T21:04:19.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9344 sec 2024-04-26T21:04:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:19 smithi090 ceph-mon[29180]: pgmap v5509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:19 smithi119 ceph-mon[32752]: pgmap v5509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:19 smithi167 ceph-mon[32751]: pgmap v5509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:20.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9345 sec 2024-04-26T21:04:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:20 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 8942.112552 secs 2024-04-26T21:04:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:20 smithi090 ceph-mon[29180]: slow request 1441.171848 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:04:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:20 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 8942.112552 secs 2024-04-26T21:04:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:20 smithi119 ceph-mon[32752]: slow request 1441.171848 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:04:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:20 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 8942.112552 secs 2024-04-26T21:04:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:20 smithi167 ceph-mon[32751]: slow request 1441.171848 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:04:21.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9346 sec 2024-04-26T21:04:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:21 smithi090 ceph-mon[29180]: pgmap v5510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8942.830107 secs 2024-04-26T21:04:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:21 smithi119 ceph-mon[32752]: pgmap v5510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8942.830107 secs 2024-04-26T21:04:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:21 smithi167 ceph-mon[32751]: pgmap v5510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8942.830107 secs 2024-04-26T21:04:22.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9347 sec 2024-04-26T21:04:23.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9348 sec 2024-04-26T21:04:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:23 smithi090 ceph-mon[29180]: pgmap v5511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:23 smithi119 ceph-mon[32752]: pgmap v5511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:23 smithi167 ceph-mon[32751]: pgmap v5511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:24.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9349 sec 2024-04-26T21:04:25.729 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9350 sec 2024-04-26T21:04:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:25 smithi119 ceph-mon[32752]: pgmap v5512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8947.112635 secs 2024-04-26T21:04:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:25 smithi167 ceph-mon[32751]: pgmap v5512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8947.112635 secs 2024-04-26T21:04:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:25 smithi090 ceph-mon[29180]: pgmap v5512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8947.112635 secs 2024-04-26T21:04:26.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9351 sec 2024-04-26T21:04:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8947.830249 secs 2024-04-26T21:04:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8947.830249 secs 2024-04-26T21:04:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8947.830249 secs 2024-04-26T21:04:27.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9352 sec 2024-04-26T21:04:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:27 smithi119 ceph-mon[32752]: pgmap v5513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:27 smithi167 ceph-mon[32751]: pgmap v5513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:27 smithi090 ceph-mon[29180]: pgmap v5513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:28.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9353 sec 2024-04-26T21:04:29.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9354 sec 2024-04-26T21:04:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:29 smithi119 ceph-mon[32752]: pgmap v5514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:29 smithi167 ceph-mon[32751]: pgmap v5514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:29 smithi090 ceph-mon[29180]: pgmap v5514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:30.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9355 sec 2024-04-26T21:04:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8952.112720 secs 2024-04-26T21:04:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8952.112720 secs 2024-04-26T21:04:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8952.112720 secs 2024-04-26T21:04:31.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9356 sec 2024-04-26T21:04:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:31 smithi119 ceph-mon[32752]: pgmap v5515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8952.830405 secs 2024-04-26T21:04:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:31 smithi167 ceph-mon[32751]: pgmap v5515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8952.830405 secs 2024-04-26T21:04:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:31 smithi090 ceph-mon[29180]: pgmap v5515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8952.830405 secs 2024-04-26T21:04:32.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9357 sec 2024-04-26T21:04:33.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9358 sec 2024-04-26T21:04:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:33 smithi119 ceph-mon[32752]: pgmap v5516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:33 smithi167 ceph-mon[32751]: pgmap v5516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:33 smithi090 ceph-mon[29180]: pgmap v5516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:34.730 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9359 sec 2024-04-26T21:04:35.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9360 sec 2024-04-26T21:04:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:35 smithi119 ceph-mon[32752]: pgmap v5517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:35 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 8957.112819 secs 2024-04-26T21:04:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:35 smithi119 ceph-mon[32752]: slow request 5763.249510 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:04:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:35 smithi167 ceph-mon[32751]: pgmap v5517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:35 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 8957.112819 secs 2024-04-26T21:04:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:35 smithi167 ceph-mon[32751]: slow request 5763.249510 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:04:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:35 smithi090 ceph-mon[29180]: pgmap v5517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:35 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 8957.112819 secs 2024-04-26T21:04:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:35 smithi090 ceph-mon[29180]: slow request 5763.249510 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:04:36.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9361 sec 2024-04-26T21:04:36.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8957.830515 secs 2024-04-26T21:04:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8957.830515 secs 2024-04-26T21:04:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8957.830515 secs 2024-04-26T21:04:37.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9362 sec 2024-04-26T21:04:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:37 smithi119 ceph-mon[32752]: pgmap v5518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:37 smithi167 ceph-mon[32751]: pgmap v5518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:37 smithi090 ceph-mon[29180]: pgmap v5518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:38.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9363 sec 2024-04-26T21:04:39.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9364 sec 2024-04-26T21:04:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:39 smithi119 ceph-mon[32752]: pgmap v5519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:39 smithi167 ceph-mon[32751]: pgmap v5519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:39 smithi090 ceph-mon[29180]: pgmap v5519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:40.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9365 sec 2024-04-26T21:04:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8962.112904 secs 2024-04-26T21:04:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:40 smithi119 ceph-mon[32752]: pgmap v5520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8962.830580 secs 2024-04-26T21:04:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8962.112904 secs 2024-04-26T21:04:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:40 smithi167 ceph-mon[32751]: pgmap v5520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8962.830580 secs 2024-04-26T21:04:41.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8962.112904 secs 2024-04-26T21:04:41.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:40 smithi090 ceph-mon[29180]: pgmap v5520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:41.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8962.830580 secs 2024-04-26T21:04:41.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9366 sec 2024-04-26T21:04:42.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9367 sec 2024-04-26T21:04:43.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9368 sec 2024-04-26T21:04:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:43 smithi119 ceph-mon[32752]: pgmap v5521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:43 smithi167 ceph-mon[32751]: pgmap v5521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:43 smithi090 ceph-mon[29180]: pgmap v5521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:44.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9369 sec 2024-04-26T21:04:45.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9370 sec 2024-04-26T21:04:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:45 smithi119 ceph-mon[32752]: pgmap v5522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8967.112989 secs 2024-04-26T21:04:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:45 smithi167 ceph-mon[32751]: pgmap v5522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8967.112989 secs 2024-04-26T21:04:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:45 smithi090 ceph-mon[29180]: pgmap v5522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8967.112989 secs 2024-04-26T21:04:46.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9371 sec 2024-04-26T21:04:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8967.830683 secs 2024-04-26T21:04:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8967.830683 secs 2024-04-26T21:04:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8967.830683 secs 2024-04-26T21:04:47.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9372 sec 2024-04-26T21:04:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:47 smithi119 ceph-mon[32752]: pgmap v5523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:47 smithi167 ceph-mon[32751]: pgmap v5523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:47 smithi090 ceph-mon[29180]: pgmap v5523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:04:48.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9373 sec 2024-04-26T21:04:49.731 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9374 sec 2024-04-26T21:04:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:49 smithi119 ceph-mon[32752]: pgmap v5524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:04:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:49 smithi167 ceph-mon[32751]: pgmap v5524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:04:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:49 smithi090 ceph-mon[29180]: pgmap v5524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:04:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:04:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:04:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:04:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:04:50.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9375 sec 2024-04-26T21:04:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:50 smithi119 ceph-mon[32752]: pgmap v5525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:04:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:50 smithi119 ceph-mon[32752]: pgmap v5526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8972.113094 secs 2024-04-26T21:04:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:50 smithi167 ceph-mon[32751]: pgmap v5525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:04:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:50 smithi167 ceph-mon[32751]: pgmap v5526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8972.113094 secs 2024-04-26T21:04:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:50 smithi090 ceph-mon[29180]: pgmap v5525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:04:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:50 smithi090 ceph-mon[29180]: pgmap v5526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8972.113094 secs 2024-04-26T21:04:51.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9376 sec 2024-04-26T21:04:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8972.830803 secs 2024-04-26T21:04:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8972.830803 secs 2024-04-26T21:04:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8972.830803 secs 2024-04-26T21:04:52.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9377 sec 2024-04-26T21:04:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:52 smithi119 ceph-mon[32752]: pgmap v5527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:52 smithi167 ceph-mon[32751]: pgmap v5527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:52 smithi090 ceph-mon[29180]: pgmap v5527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:53.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9378 sec 2024-04-26T21:04:54.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9379 sec 2024-04-26T21:04:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:54 smithi119 ceph-mon[32752]: pgmap v5528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:04:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:54 smithi167 ceph-mon[32751]: pgmap v5528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:04:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:54 smithi090 ceph-mon[29180]: pgmap v5528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:04:55.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9380 sec 2024-04-26T21:04:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8977.113180 secs 2024-04-26T21:04:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:55 smithi119 ceph-mon[32752]: pgmap v5529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8977.830923 secs 2024-04-26T21:04:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8977.113180 secs 2024-04-26T21:04:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:55 smithi167 ceph-mon[32751]: pgmap v5529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8977.830923 secs 2024-04-26T21:04:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8977.113180 secs 2024-04-26T21:04:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:55 smithi090 ceph-mon[29180]: pgmap v5529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:04:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8977.830923 secs 2024-04-26T21:04:56.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9381 sec 2024-04-26T21:04:57.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9382 sec 2024-04-26T21:04:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:04:58 smithi119 ceph-mon[32752]: pgmap v5530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:04:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:04:58 smithi167 ceph-mon[32751]: pgmap v5530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:04:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:04:58 smithi090 ceph-mon[29180]: pgmap v5530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:04:58.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9383 sec 2024-04-26T21:04:59.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9384 sec 2024-04-26T21:05:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:00 smithi119 ceph-mon[32752]: pgmap v5531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:05:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8982.113262 secs 2024-04-26T21:05:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:00 smithi167 ceph-mon[32751]: pgmap v5531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:05:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8982.113262 secs 2024-04-26T21:05:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:00 smithi090 ceph-mon[29180]: pgmap v5531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:05:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8982.113262 secs 2024-04-26T21:05:00.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9385 sec 2024-04-26T21:05:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8982.831041 secs 2024-04-26T21:05:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8982.831041 secs 2024-04-26T21:05:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8982.831041 secs 2024-04-26T21:05:01.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9386 sec 2024-04-26T21:05:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:02 smithi119 ceph-mon[32752]: pgmap v5532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:02 smithi167 ceph-mon[32751]: pgmap v5532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:02 smithi090 ceph-mon[29180]: pgmap v5532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:02.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9387 sec 2024-04-26T21:05:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:05:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:05:03.161+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:05:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:05:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:05:03.161+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:05:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:05:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:05:03.161+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:05:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:05:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:05:03.161+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:05:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:05:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:05:03.161+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:05:03.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9388 sec 2024-04-26T21:05:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:04 smithi119 ceph-mon[32752]: pgmap v5533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:05:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:04 smithi167 ceph-mon[32751]: pgmap v5533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:05:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:04 smithi090 ceph-mon[29180]: pgmap v5533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:05:04.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9389 sec 2024-04-26T21:05:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8987.113344 secs 2024-04-26T21:05:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8987.113344 secs 2024-04-26T21:05:05.732 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9390 sec 2024-04-26T21:05:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8987.113344 secs 2024-04-26T21:05:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:06 smithi167 ceph-mon[32751]: pgmap v5534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8987.831159 secs 2024-04-26T21:05:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:06 smithi090 ceph-mon[29180]: pgmap v5534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8987.831159 secs 2024-04-26T21:05:06.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9391 sec 2024-04-26T21:05:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:06 smithi119 ceph-mon[32752]: pgmap v5534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8987.831159 secs 2024-04-26T21:05:07.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9392 sec 2024-04-26T21:05:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:08 smithi167 ceph-mon[32751]: pgmap v5535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T21:05:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:08 smithi090 ceph-mon[29180]: pgmap v5535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T21:05:08.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9393 sec 2024-04-26T21:05:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:08 smithi119 ceph-mon[32752]: pgmap v5535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T21:05:09.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9394 sec 2024-04-26T21:05:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:10 smithi090 ceph-mon[29180]: pgmap v5536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8992.113429 secs 2024-04-26T21:05:10.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9395 sec 2024-04-26T21:05:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:10 smithi119 ceph-mon[32752]: pgmap v5536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8992.113429 secs 2024-04-26T21:05:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:10 smithi167 ceph-mon[32751]: pgmap v5536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8992.113429 secs 2024-04-26T21:05:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8992.831298 secs 2024-04-26T21:05:11.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9396 sec 2024-04-26T21:05:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8992.831298 secs 2024-04-26T21:05:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8992.831298 secs 2024-04-26T21:05:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:12 smithi090 ceph-mon[29180]: pgmap v5537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:12.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9397 sec 2024-04-26T21:05:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:12 smithi119 ceph-mon[32752]: pgmap v5537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:12 smithi167 ceph-mon[32751]: pgmap v5537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:13.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9398 sec 2024-04-26T21:05:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:14 smithi090 ceph-mon[29180]: pgmap v5538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:14.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9399 sec 2024-04-26T21:05:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:14 smithi119 ceph-mon[32752]: pgmap v5538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:14 smithi167 ceph-mon[32751]: pgmap v5538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:05:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 8997.113513 secs 2024-04-26T21:05:15.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9400 sec 2024-04-26T21:05:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 8997.113513 secs 2024-04-26T21:05:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 8997.113513 secs 2024-04-26T21:05:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:16 smithi090 ceph-mon[29180]: pgmap v5539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:05:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 8997.831369 secs 2024-04-26T21:05:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:16 smithi119 ceph-mon[32752]: pgmap v5539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:05:16.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 8997.831369 secs 2024-04-26T21:05:16.733 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9401 sec 2024-04-26T21:05:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:16 smithi167 ceph-mon[32751]: pgmap v5539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T21:05:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 8997.831369 secs 2024-04-26T21:05:17.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9402 sec 2024-04-26T21:05:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:18 smithi090 ceph-mon[29180]: pgmap v5540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:05:18.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9403 sec 2024-04-26T21:05:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:18 smithi119 ceph-mon[32752]: pgmap v5540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:05:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:18 smithi167 ceph-mon[32751]: pgmap v5540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:05:19.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9404 sec 2024-04-26T21:05:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:20 smithi090 ceph-mon[29180]: pgmap v5541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:05:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9002.113597 secs 2024-04-26T21:05:20.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9405 sec 2024-04-26T21:05:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:20 smithi119 ceph-mon[32752]: pgmap v5541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:05:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9002.113597 secs 2024-04-26T21:05:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:20 smithi167 ceph-mon[32751]: pgmap v5541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:05:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9002.113597 secs 2024-04-26T21:05:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9002.831487 secs 2024-04-26T21:05:21.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9406 sec 2024-04-26T21:05:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9002.831487 secs 2024-04-26T21:05:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9002.831487 secs 2024-04-26T21:05:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:22 smithi090 ceph-mon[29180]: pgmap v5542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:22.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9407 sec 2024-04-26T21:05:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:22 smithi119 ceph-mon[32752]: pgmap v5542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:22 smithi167 ceph-mon[32751]: pgmap v5542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:23.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9408 sec 2024-04-26T21:05:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:24 smithi090 ceph-mon[29180]: pgmap v5543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:24.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9409 sec 2024-04-26T21:05:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:24 smithi119 ceph-mon[32752]: pgmap v5543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:24 smithi167 ceph-mon[32751]: pgmap v5543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9007.113680 secs 2024-04-26T21:05:25.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9410 sec 2024-04-26T21:05:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9007.113680 secs 2024-04-26T21:05:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9007.113680 secs 2024-04-26T21:05:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:26 smithi090 ceph-mon[29180]: pgmap v5544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9007.831626 secs 2024-04-26T21:05:26.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9411 sec 2024-04-26T21:05:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:26 smithi119 ceph-mon[32752]: pgmap v5544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9007.831626 secs 2024-04-26T21:05:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:26 smithi167 ceph-mon[32751]: pgmap v5544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9007.831626 secs 2024-04-26T21:05:27.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9412 sec 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 msgr2=0x7f2cc00b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 msgr2=0x7f2cc00b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 0x7f2cc00b2f00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40193f0 tx=0x7f2cc4004730 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 0x7f2cc00b2f00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40193f0 tx=0x7f2cc4004730 comp rx=0 tx=0).stop 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 msgr2=0x7f2cc00b2f00 unknown :-1 s=STATE_CLOSED l=1).tick see no progress in more than 10000000 us during connecting to v2:172.21.15.90:6800/2946218715, fault. 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 -- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 msgr2=0x5630289b8a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=0).tick idle (900000173) for more than 900000000 us, fault. 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 msgr2=0x7f2cc00b2f00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-04-26T21:05:28.497 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b2b20 0x7f2cc00b2f00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=READY pgs=627 cs=8 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault initiating reconnect 2024-04-26T21:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2ccb7fe640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b3440 0x7f2cc00b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 unknown :-1 s=BANNER_CONNECTING pgs=627 cs=9 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.495+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b3440 0x7f2cc00b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.496+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3143758137 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f2cc00b3440 0x7f2cc00b3820 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401bc50 tx=0x7f2cc408a040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.496+0000 7f2cd259c640 1 --2- 172.21.15.90:0/3906338670 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x5630289b6650 0x5630289b8a70 crc :-1 s=READY pgs=627 cs=9 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e40f233fab90ce7 server_cookie=ba6670014ceff153 in_seq=0 out_seq=0 2024-04-26T21:05:28.498 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:28.496+0000 7f2cb9ffb640 10 client.17037 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T21:05:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:28 smithi090 ceph-mon[29180]: pgmap v5545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:28.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9413 sec 2024-04-26T21:05:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:28 smithi119 ceph-mon[32752]: pgmap v5545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:28 smithi167 ceph-mon[32751]: pgmap v5545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:29.672 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:29.671+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:05:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:29.671+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:05:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:29.672+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 73 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:05:29.673 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:05:29.672+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 73 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:05:29.734 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9414 sec 2024-04-26T21:05:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:30 smithi090 ceph-mon[29180]: pgmap v5546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9012.113765 secs 2024-04-26T21:05:30.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9415 sec 2024-04-26T21:05:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:30 smithi119 ceph-mon[32752]: pgmap v5546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9012.113765 secs 2024-04-26T21:05:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:30 smithi167 ceph-mon[32751]: pgmap v5546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9012.113765 secs 2024-04-26T21:05:31.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9416 sec 2024-04-26T21:05:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9012.831744 secs 2024-04-26T21:05:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9012.831744 secs 2024-04-26T21:05:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9012.831744 secs 2024-04-26T21:05:32.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9417 sec 2024-04-26T21:05:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:32 smithi119 ceph-mon[32752]: pgmap v5547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:32 smithi167 ceph-mon[32751]: pgmap v5547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:32 smithi090 ceph-mon[29180]: pgmap v5547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:33.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9418 sec 2024-04-26T21:05:34.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9419 sec 2024-04-26T21:05:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:34 smithi119 ceph-mon[32752]: pgmap v5548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:34 smithi167 ceph-mon[32751]: pgmap v5548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:34 smithi090 ceph-mon[29180]: pgmap v5548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:35.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9420 sec 2024-04-26T21:05:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9017.113852 secs 2024-04-26T21:05:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9017.113852 secs 2024-04-26T21:05:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9017.113852 secs 2024-04-26T21:05:36.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:36 smithi167 ceph-mon[32751]: pgmap v5549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:36.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9017.831862 secs 2024-04-26T21:05:36.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9421 sec 2024-04-26T21:05:36.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:36 smithi090 ceph-mon[29180]: pgmap v5549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:36.742 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9017.831862 secs 2024-04-26T21:05:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:36 smithi119 ceph-mon[32752]: pgmap v5549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9017.831862 secs 2024-04-26T21:05:37.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9422 sec 2024-04-26T21:05:38.735 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9423 sec 2024-04-26T21:05:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:38 smithi119 ceph-mon[32752]: pgmap v5550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:38 smithi167 ceph-mon[32751]: pgmap v5550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:38 smithi090 ceph-mon[29180]: pgmap v5550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:39.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9424 sec 2024-04-26T21:05:40.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9425 sec 2024-04-26T21:05:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:40 smithi119 ceph-mon[32752]: pgmap v5551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9022.113942 secs 2024-04-26T21:05:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:40 smithi167 ceph-mon[32751]: pgmap v5551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9022.113942 secs 2024-04-26T21:05:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:40 smithi090 ceph-mon[29180]: pgmap v5551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9022.113942 secs 2024-04-26T21:05:41.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9426 sec 2024-04-26T21:05:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9022.831951 secs 2024-04-26T21:05:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9022.831951 secs 2024-04-26T21:05:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9022.831951 secs 2024-04-26T21:05:42.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9427 sec 2024-04-26T21:05:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:42 smithi119 ceph-mon[32752]: pgmap v5552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:42 smithi167 ceph-mon[32751]: pgmap v5552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:42 smithi090 ceph-mon[29180]: pgmap v5552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:43.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9428 sec 2024-04-26T21:05:44.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9429 sec 2024-04-26T21:05:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:44 smithi119 ceph-mon[32752]: pgmap v5553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:44 smithi167 ceph-mon[32751]: pgmap v5553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:44 smithi090 ceph-mon[29180]: pgmap v5553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:05:45.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9430 sec 2024-04-26T21:05:45.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9027.114024 secs 2024-04-26T21:05:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9027.114024 secs 2024-04-26T21:05:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9027.114024 secs 2024-04-26T21:05:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:46 smithi119 ceph-mon[32752]: pgmap v5554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:46.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9027.832066 secs 2024-04-26T21:05:46.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9431 sec 2024-04-26T21:05:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:46 smithi167 ceph-mon[32751]: pgmap v5554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9027.832066 secs 2024-04-26T21:05:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:46 smithi090 ceph-mon[29180]: pgmap v5554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9027.832066 secs 2024-04-26T21:05:47.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9432 sec 2024-04-26T21:05:48.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9433 sec 2024-04-26T21:05:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:48 smithi119 ceph-mon[32752]: pgmap v5555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:48 smithi167 ceph-mon[32751]: pgmap v5555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:48 smithi090 ceph-mon[29180]: pgmap v5555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:49.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9434 sec 2024-04-26T21:05:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:05:50.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9435 sec 2024-04-26T21:05:50.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:50 smithi119 ceph-mon[32752]: pgmap v5556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:50.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9032.114086 secs 2024-04-26T21:05:50.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:50 smithi090 ceph-mon[29180]: pgmap v5556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:50.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9032.114086 secs 2024-04-26T21:05:50.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:50 smithi167 ceph-mon[32751]: pgmap v5556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:05:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9032.114086 secs 2024-04-26T21:05:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:51.736 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9436 sec 2024-04-26T21:05:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9032.832187 secs 2024-04-26T21:05:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9032.832187 secs 2024-04-26T21:05:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9032.832187 secs 2024-04-26T21:05:52.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9437 sec 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: pgmap v5557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: pgmap v5557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: pgmap v5557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:05:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:53.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9438 sec 2024-04-26T21:05:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:05:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:05:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:05:54.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9439 sec 2024-04-26T21:05:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:54 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:05:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:54 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:05:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:54 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:05:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:54 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:05:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:54 smithi090 ceph-mon[29180]: pgmap v5558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-26T21:05:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:54 smithi090 ceph-mon[29180]: pgmap v5559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-26T21:05:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:54 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:05:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:54 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:05:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:54 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:05:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:54 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:05:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:54 smithi119 ceph-mon[32752]: pgmap v5558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-26T21:05:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:54 smithi119 ceph-mon[32752]: pgmap v5559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-26T21:05:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:54 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:05:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:54 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:05:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:54 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:05:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:54 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:05:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:54 smithi167 ceph-mon[32751]: pgmap v5558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-26T21:05:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:54 smithi167 ceph-mon[32751]: pgmap v5559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-26T21:05:55.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9440 sec 2024-04-26T21:05:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9037.114144 secs 2024-04-26T21:05:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9037.114144 secs 2024-04-26T21:05:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9037.114144 secs 2024-04-26T21:05:56.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9441 sec 2024-04-26T21:05:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:56 smithi119 ceph-mon[32752]: pgmap v5560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:05:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9037.832311 secs 2024-04-26T21:05:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:56 smithi167 ceph-mon[32751]: pgmap v5560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:05:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9037.832311 secs 2024-04-26T21:05:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:56 smithi090 ceph-mon[29180]: pgmap v5560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:05:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9037.832311 secs 2024-04-26T21:05:57.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9442 sec 2024-04-26T21:05:58.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9443 sec 2024-04-26T21:05:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:05:58 smithi119 ceph-mon[32752]: pgmap v5561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:05:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:05:58 smithi167 ceph-mon[32751]: pgmap v5561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:05:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:05:58 smithi090 ceph-mon[29180]: pgmap v5561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:05:59.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9444 sec 2024-04-26T21:06:00.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9445 sec 2024-04-26T21:06:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:00 smithi119 ceph-mon[32752]: pgmap v5562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:06:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9042.114227 secs 2024-04-26T21:06:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:00 smithi167 ceph-mon[32751]: pgmap v5562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:06:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9042.114227 secs 2024-04-26T21:06:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:00 smithi090 ceph-mon[29180]: pgmap v5562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:06:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9042.114227 secs 2024-04-26T21:06:01.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9446 sec 2024-04-26T21:06:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9042.832389 secs 2024-04-26T21:06:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9042.832389 secs 2024-04-26T21:06:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9042.832389 secs 2024-04-26T21:06:02.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9447 sec 2024-04-26T21:06:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:02 smithi119 ceph-mon[32752]: pgmap v5563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:06:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:02 smithi167 ceph-mon[32751]: pgmap v5563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:06:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:02 smithi090 ceph-mon[29180]: pgmap v5563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:06:03.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9448 sec 2024-04-26T21:06:04.737 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9449 sec 2024-04-26T21:06:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:04 smithi119 ceph-mon[32752]: pgmap v5564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:06:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:04 smithi167 ceph-mon[32751]: pgmap v5564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:06:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:04 smithi090 ceph-mon[29180]: pgmap v5564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:06:05.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9450 sec 2024-04-26T21:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9047.114310 secs 2024-04-26T21:06:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9047.114310 secs 2024-04-26T21:06:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9047.114310 secs 2024-04-26T21:06:06.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9451 sec 2024-04-26T21:06:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:06 smithi167 ceph-mon[32751]: pgmap v5565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:06 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 9047.832538 secs 2024-04-26T21:06:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:06 smithi167 ceph-mon[32751]: slow request 362.469305 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:06 smithi119 ceph-mon[32752]: pgmap v5565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:06 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 9047.832538 secs 2024-04-26T21:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:06 smithi119 ceph-mon[32752]: slow request 362.469305 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:06:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:06 smithi090 ceph-mon[29180]: pgmap v5565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:06 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 9047.832538 secs 2024-04-26T21:06:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:06 smithi090 ceph-mon[29180]: slow request 362.469305 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:06:07.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9452 sec 2024-04-26T21:06:08.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9453 sec 2024-04-26T21:06:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:08 smithi119 ceph-mon[32752]: pgmap v5566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:08 smithi167 ceph-mon[32751]: pgmap v5566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:08 smithi090 ceph-mon[29180]: pgmap v5566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:09.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9454 sec 2024-04-26T21:06:10.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9455 sec 2024-04-26T21:06:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:10 smithi119 ceph-mon[32752]: pgmap v5567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9052.114369 secs 2024-04-26T21:06:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:10 smithi167 ceph-mon[32751]: pgmap v5567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9052.114369 secs 2024-04-26T21:06:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:10 smithi090 ceph-mon[29180]: pgmap v5567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9052.114369 secs 2024-04-26T21:06:11.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9456 sec 2024-04-26T21:06:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9052.832650 secs 2024-04-26T21:06:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9052.832650 secs 2024-04-26T21:06:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9052.832650 secs 2024-04-26T21:06:12.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9457 sec 2024-04-26T21:06:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:12 smithi119 ceph-mon[32752]: pgmap v5568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:12 smithi167 ceph-mon[32751]: pgmap v5568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:12 smithi090 ceph-mon[29180]: pgmap v5568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:13.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9458 sec 2024-04-26T21:06:14.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9459 sec 2024-04-26T21:06:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:14 smithi119 ceph-mon[32752]: pgmap v5569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:14 smithi167 ceph-mon[32751]: pgmap v5569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:14 smithi090 ceph-mon[29180]: pgmap v5569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:15.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9460 sec 2024-04-26T21:06:16.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9057.114427 secs 2024-04-26T21:06:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9057.114427 secs 2024-04-26T21:06:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9057.114427 secs 2024-04-26T21:06:16.738 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9461 sec 2024-04-26T21:06:16.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:16 smithi119 ceph-mon[32752]: pgmap v5570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9057.832776 secs 2024-04-26T21:06:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:16 smithi167 ceph-mon[32751]: pgmap v5570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9057.832776 secs 2024-04-26T21:06:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:16 smithi090 ceph-mon[29180]: pgmap v5570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9057.832776 secs 2024-04-26T21:06:17.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9462 sec 2024-04-26T21:06:18.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9463 sec 2024-04-26T21:06:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:18 smithi119 ceph-mon[32752]: pgmap v5571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:18 smithi167 ceph-mon[32751]: pgmap v5571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:18 smithi090 ceph-mon[29180]: pgmap v5571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:19.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9464 sec 2024-04-26T21:06:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:19 smithi119 ceph-mon[32752]: pgmap v5572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:19 smithi167 ceph-mon[32751]: pgmap v5572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:19 smithi090 ceph-mon[29180]: pgmap v5572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:20.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9465 sec 2024-04-26T21:06:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9062.114528 secs 2024-04-26T21:06:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9062.832855 secs 2024-04-26T21:06:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9062.114528 secs 2024-04-26T21:06:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9062.832855 secs 2024-04-26T21:06:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9062.114528 secs 2024-04-26T21:06:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9062.832855 secs 2024-04-26T21:06:21.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9466 sec 2024-04-26T21:06:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:21 smithi119 ceph-mon[32752]: pgmap v5573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:21 smithi167 ceph-mon[32751]: pgmap v5573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:21 smithi090 ceph-mon[29180]: pgmap v5573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:22.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9467 sec 2024-04-26T21:06:23.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9468 sec 2024-04-26T21:06:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:23 smithi119 ceph-mon[32752]: pgmap v5574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:23 smithi167 ceph-mon[32751]: pgmap v5574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:23 smithi090 ceph-mon[29180]: pgmap v5574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:24.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9469 sec 2024-04-26T21:06:25.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9470 sec 2024-04-26T21:06:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:25 smithi119 ceph-mon[32752]: pgmap v5575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9067.114609 secs 2024-04-26T21:06:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:25 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9067.832954 secs 2024-04-26T21:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:25 smithi167 ceph-mon[32751]: pgmap v5575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9067.114609 secs 2024-04-26T21:06:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:25 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9067.832954 secs 2024-04-26T21:06:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:25 smithi090 ceph-mon[29180]: pgmap v5575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9067.114609 secs 2024-04-26T21:06:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:25 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9067.832954 secs 2024-04-26T21:06:26.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9471 sec 2024-04-26T21:06:27.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9472 sec 2024-04-26T21:06:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:27 smithi119 ceph-mon[32752]: pgmap v5576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:28.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:27 smithi167 ceph-mon[32751]: pgmap v5576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:27 smithi090 ceph-mon[29180]: pgmap v5576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:28.739 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9473 sec 2024-04-26T21:06:29.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9474 sec 2024-04-26T21:06:30.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:29 smithi119 ceph-mon[32752]: pgmap v5577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:29 smithi167 ceph-mon[32751]: pgmap v5577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:29 smithi090 ceph-mon[29180]: pgmap v5577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:30.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9475 sec 2024-04-26T21:06:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9072.114691 secs 2024-04-26T21:06:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9072.833088 secs 2024-04-26T21:06:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9072.114691 secs 2024-04-26T21:06:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9072.833088 secs 2024-04-26T21:06:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9072.114691 secs 2024-04-26T21:06:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9072.833088 secs 2024-04-26T21:06:31.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9476 sec 2024-04-26T21:06:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:31 smithi119 ceph-mon[32752]: pgmap v5578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:31 smithi167 ceph-mon[32751]: pgmap v5578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:31 smithi090 ceph-mon[29180]: pgmap v5578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:32.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9477 sec 2024-04-26T21:06:33.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9478 sec 2024-04-26T21:06:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:33 smithi119 ceph-mon[32752]: pgmap v5579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:33 smithi167 ceph-mon[32751]: pgmap v5579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:34.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:33 smithi090 ceph-mon[29180]: pgmap v5579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:34.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9479 sec 2024-04-26T21:06:35.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9480 sec 2024-04-26T21:06:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:35 smithi119 ceph-mon[32752]: pgmap v5580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9077.114775 secs 2024-04-26T21:06:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:35 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9077.833203 secs 2024-04-26T21:06:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:35 smithi167 ceph-mon[32751]: pgmap v5580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9077.114775 secs 2024-04-26T21:06:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:35 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9077.833203 secs 2024-04-26T21:06:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:35 smithi090 ceph-mon[29180]: pgmap v5580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9077.114775 secs 2024-04-26T21:06:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:35 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9077.833203 secs 2024-04-26T21:06:36.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9481 sec 2024-04-26T21:06:37.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9482 sec 2024-04-26T21:06:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:37 smithi119 ceph-mon[32752]: pgmap v5581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:37 smithi167 ceph-mon[32751]: pgmap v5581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:37 smithi090 ceph-mon[29180]: pgmap v5581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:38.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9483 sec 2024-04-26T21:06:39.740 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9484 sec 2024-04-26T21:06:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:39 smithi119 ceph-mon[32752]: pgmap v5582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:39 smithi167 ceph-mon[32751]: pgmap v5582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:39 smithi090 ceph-mon[29180]: pgmap v5582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:40.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9485 sec 2024-04-26T21:06:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9082.114838 secs 2024-04-26T21:06:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9082.833319 secs 2024-04-26T21:06:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9082.114838 secs 2024-04-26T21:06:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9082.833319 secs 2024-04-26T21:06:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9082.114838 secs 2024-04-26T21:06:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9082.833319 secs 2024-04-26T21:06:41.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9486 sec 2024-04-26T21:06:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:41 smithi119 ceph-mon[32752]: pgmap v5583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:41 smithi167 ceph-mon[32751]: pgmap v5583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:41 smithi090 ceph-mon[29180]: pgmap v5583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:42.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9487 sec 2024-04-26T21:06:43.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9488 sec 2024-04-26T21:06:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:43 smithi119 ceph-mon[32752]: pgmap v5584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:43 smithi167 ceph-mon[32751]: pgmap v5584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:43 smithi090 ceph-mon[29180]: pgmap v5584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:44.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9489 sec 2024-04-26T21:06:45.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9490 sec 2024-04-26T21:06:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:45 smithi119 ceph-mon[32752]: pgmap v5585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9087.114900 secs 2024-04-26T21:06:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:45 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9087.833430 secs 2024-04-26T21:06:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:45 smithi167 ceph-mon[32751]: pgmap v5585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9087.114900 secs 2024-04-26T21:06:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:45 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9087.833430 secs 2024-04-26T21:06:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:45 smithi090 ceph-mon[29180]: pgmap v5585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9087.114900 secs 2024-04-26T21:06:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:45 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9087.833430 secs 2024-04-26T21:06:46.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9491 sec 2024-04-26T21:06:47.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9492 sec 2024-04-26T21:06:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:47 smithi119 ceph-mon[32752]: pgmap v5586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:47 smithi167 ceph-mon[32751]: pgmap v5586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:47 smithi090 ceph-mon[29180]: pgmap v5586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:48.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9493 sec 2024-04-26T21:06:49.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9494 sec 2024-04-26T21:06:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:49 smithi119 ceph-mon[32752]: pgmap v5587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:49 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9092.114960 secs 2024-04-26T21:06:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:49 smithi167 ceph-mon[32751]: pgmap v5587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:49 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9092.114960 secs 2024-04-26T21:06:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:49 smithi090 ceph-mon[29180]: pgmap v5587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:06:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:49 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9092.114960 secs 2024-04-26T21:06:50.741 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9495 sec 2024-04-26T21:06:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:50 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9092.833520 secs 2024-04-26T21:06:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:50 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9092.833520 secs 2024-04-26T21:06:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:50 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9092.833520 secs 2024-04-26T21:06:51.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9496 sec 2024-04-26T21:06:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:51 smithi119 ceph-mon[32752]: pgmap v5588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:51 smithi167 ceph-mon[32751]: pgmap v5588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:51 smithi090 ceph-mon[29180]: pgmap v5588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:52.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9497 sec 2024-04-26T21:06:53.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9498 sec 2024-04-26T21:06:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:53 smithi119 ceph-mon[32752]: pgmap v5589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:53 smithi167 ceph-mon[32751]: pgmap v5589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:53 smithi090 ceph-mon[29180]: pgmap v5589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:06:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:06:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:54.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9499 sec 2024-04-26T21:06:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:54 smithi119 ceph-mon[32752]: pgmap v5590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:06:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:54 smithi119 ceph-mon[32752]: pgmap v5591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:06:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:54 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9097.115021 secs 2024-04-26T21:06:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:54 smithi167 ceph-mon[32751]: pgmap v5590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:06:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:54 smithi167 ceph-mon[32751]: pgmap v5591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:06:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:54 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9097.115021 secs 2024-04-26T21:06:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:06:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:06:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:54 smithi090 ceph-mon[29180]: pgmap v5590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:06:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:54 smithi090 ceph-mon[29180]: pgmap v5591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:06:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:06:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:54 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9097.115021 secs 2024-04-26T21:06:55.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9500 sec 2024-04-26T21:06:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9097.833598 secs 2024-04-26T21:06:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9097.833598 secs 2024-04-26T21:06:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9097.833598 secs 2024-04-26T21:06:56.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9501 sec 2024-04-26T21:06:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:56 smithi119 ceph-mon[32752]: pgmap v5592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:06:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:56 smithi167 ceph-mon[32751]: pgmap v5592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:06:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:56 smithi090 ceph-mon[29180]: pgmap v5592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:06:57.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9502 sec 2024-04-26T21:06:58.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9503 sec 2024-04-26T21:06:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:06:59 smithi119 ceph-mon[32752]: pgmap v5593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T21:06:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:06:59 smithi167 ceph-mon[32751]: pgmap v5593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T21:06:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:06:59 smithi090 ceph-mon[29180]: pgmap v5593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T21:06:59.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9504 sec 2024-04-26T21:07:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9102.115082 secs 2024-04-26T21:07:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9102.115082 secs 2024-04-26T21:07:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9102.115082 secs 2024-04-26T21:07:00.742 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9505 sec 2024-04-26T21:07:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:01 smithi119 ceph-mon[32752]: pgmap v5594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9102.833707 secs 2024-04-26T21:07:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:01 smithi167 ceph-mon[32751]: pgmap v5594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9102.833707 secs 2024-04-26T21:07:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:01 smithi090 ceph-mon[29180]: pgmap v5594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9102.833707 secs 2024-04-26T21:07:01.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9506 sec 2024-04-26T21:07:02.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9507 sec 2024-04-26T21:07:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:03 smithi119 ceph-mon[32752]: pgmap v5595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:03 smithi167 ceph-mon[32751]: pgmap v5595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:03 smithi090 ceph-mon[29180]: pgmap v5595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:03.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9508 sec 2024-04-26T21:07:04.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9509 sec 2024-04-26T21:07:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:05 smithi119 ceph-mon[32752]: pgmap v5596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:07:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:05 smithi119 ceph-mon[32752]: 14 slow requests, 1 included below; oldest blocked for > 9107.115140 secs 2024-04-26T21:07:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:05 smithi119 ceph-mon[32752]: slow request 721.877083 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:07:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:05 smithi167 ceph-mon[32751]: pgmap v5596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:07:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:05 smithi167 ceph-mon[32751]: 14 slow requests, 1 included below; oldest blocked for > 9107.115140 secs 2024-04-26T21:07:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:05 smithi167 ceph-mon[32751]: slow request 721.877083 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:07:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:05 smithi090 ceph-mon[29180]: pgmap v5596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:07:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:05 smithi090 ceph-mon[29180]: 14 slow requests, 1 included below; oldest blocked for > 9107.115140 secs 2024-04-26T21:07:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:05 smithi090 ceph-mon[29180]: slow request 721.877083 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:07:05.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9510 sec 2024-04-26T21:07:06.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9107.833787 secs 2024-04-26T21:07:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9107.833787 secs 2024-04-26T21:07:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9107.833787 secs 2024-04-26T21:07:06.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9511 sec 2024-04-26T21:07:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:07 smithi119 ceph-mon[32752]: pgmap v5597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:07 smithi167 ceph-mon[32751]: pgmap v5597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:07 smithi090 ceph-mon[29180]: pgmap v5597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:07.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9512 sec 2024-04-26T21:07:08.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9513 sec 2024-04-26T21:07:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:09 smithi119 ceph-mon[32752]: pgmap v5598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:09 smithi167 ceph-mon[32751]: pgmap v5598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:09 smithi090 ceph-mon[29180]: pgmap v5598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:09.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9514 sec 2024-04-26T21:07:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9112.115195 secs 2024-04-26T21:07:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9112.115195 secs 2024-04-26T21:07:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9112.115195 secs 2024-04-26T21:07:10.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9515 sec 2024-04-26T21:07:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:11 smithi119 ceph-mon[32752]: pgmap v5599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9112.833908 secs 2024-04-26T21:07:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:11 smithi167 ceph-mon[32751]: pgmap v5599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9112.833908 secs 2024-04-26T21:07:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:11 smithi090 ceph-mon[29180]: pgmap v5599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9112.833908 secs 2024-04-26T21:07:11.743 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9516 sec 2024-04-26T21:07:12.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9517 sec 2024-04-26T21:07:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:13 smithi119 ceph-mon[32752]: pgmap v5600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:13 smithi167 ceph-mon[32751]: pgmap v5600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:13 smithi090 ceph-mon[29180]: pgmap v5600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:13.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9518 sec 2024-04-26T21:07:14.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9519 sec 2024-04-26T21:07:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:15 smithi119 ceph-mon[32752]: pgmap v5601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9117.115255 secs 2024-04-26T21:07:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:15 smithi167 ceph-mon[32751]: pgmap v5601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9117.115255 secs 2024-04-26T21:07:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:15 smithi090 ceph-mon[29180]: pgmap v5601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9117.115255 secs 2024-04-26T21:07:15.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9520 sec 2024-04-26T21:07:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9117.833985 secs 2024-04-26T21:07:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9117.833985 secs 2024-04-26T21:07:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9117.833985 secs 2024-04-26T21:07:16.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9521 sec 2024-04-26T21:07:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:17 smithi119 ceph-mon[32752]: pgmap v5602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:17 smithi167 ceph-mon[32751]: pgmap v5602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:17 smithi090 ceph-mon[29180]: pgmap v5602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:17.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9522 sec 2024-04-26T21:07:18.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9523 sec 2024-04-26T21:07:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:19 smithi119 ceph-mon[32752]: pgmap v5603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:19.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:19 smithi167 ceph-mon[32751]: pgmap v5603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:19.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:19 smithi090 ceph-mon[29180]: pgmap v5603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:19.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9524 sec 2024-04-26T21:07:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9122.115316 secs 2024-04-26T21:07:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9122.115316 secs 2024-04-26T21:07:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9122.115316 secs 2024-04-26T21:07:20.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9525 sec 2024-04-26T21:07:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:21 smithi119 ceph-mon[32752]: pgmap v5604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9122.834063 secs 2024-04-26T21:07:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:21 smithi167 ceph-mon[32751]: pgmap v5604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9122.834063 secs 2024-04-26T21:07:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:21 smithi090 ceph-mon[29180]: pgmap v5604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9122.834063 secs 2024-04-26T21:07:21.744 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9526 sec 2024-04-26T21:07:22.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9527 sec 2024-04-26T21:07:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:23 smithi119 ceph-mon[32752]: pgmap v5605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:23 smithi167 ceph-mon[32751]: pgmap v5605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:23 smithi090 ceph-mon[29180]: pgmap v5605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:23.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9528 sec 2024-04-26T21:07:24.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9529 sec 2024-04-26T21:07:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:25 smithi119 ceph-mon[32752]: pgmap v5606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9127.115374 secs 2024-04-26T21:07:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:25 smithi167 ceph-mon[32751]: pgmap v5606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9127.115374 secs 2024-04-26T21:07:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:25 smithi090 ceph-mon[29180]: pgmap v5606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9127.115374 secs 2024-04-26T21:07:25.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9530 sec 2024-04-26T21:07:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9127.834170 secs 2024-04-26T21:07:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9127.834170 secs 2024-04-26T21:07:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9127.834170 secs 2024-04-26T21:07:26.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9531 sec 2024-04-26T21:07:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:27 smithi119 ceph-mon[32752]: pgmap v5607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:27 smithi167 ceph-mon[32751]: pgmap v5607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:27 smithi090 ceph-mon[29180]: pgmap v5607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:27.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9532 sec 2024-04-26T21:07:28.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9533 sec 2024-04-26T21:07:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:29 smithi119 ceph-mon[32752]: pgmap v5608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:29 smithi167 ceph-mon[32751]: pgmap v5608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:29 smithi090 ceph-mon[29180]: pgmap v5608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:29.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9534 sec 2024-04-26T21:07:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9132.115428 secs 2024-04-26T21:07:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9132.115428 secs 2024-04-26T21:07:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9132.115428 secs 2024-04-26T21:07:30.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9535 sec 2024-04-26T21:07:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:31 smithi119 ceph-mon[32752]: pgmap v5609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9132.834303 secs 2024-04-26T21:07:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:31 smithi167 ceph-mon[32751]: pgmap v5609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9132.834303 secs 2024-04-26T21:07:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:31 smithi090 ceph-mon[29180]: pgmap v5609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9132.834303 secs 2024-04-26T21:07:31.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9536 sec 2024-04-26T21:07:32.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9537 sec 2024-04-26T21:07:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:33 smithi119 ceph-mon[32752]: pgmap v5610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:33 smithi167 ceph-mon[32751]: pgmap v5610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:33 smithi090 ceph-mon[29180]: pgmap v5610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:33.745 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9538 sec 2024-04-26T21:07:34.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9539 sec 2024-04-26T21:07:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:35 smithi119 ceph-mon[32752]: pgmap v5611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9137.115488 secs 2024-04-26T21:07:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:35 smithi167 ceph-mon[32751]: pgmap v5611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9137.115488 secs 2024-04-26T21:07:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:35 smithi090 ceph-mon[29180]: pgmap v5611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9137.115488 secs 2024-04-26T21:07:35.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9540 sec 2024-04-26T21:07:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9137.834414 secs 2024-04-26T21:07:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9137.834414 secs 2024-04-26T21:07:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9137.834414 secs 2024-04-26T21:07:36.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9541 sec 2024-04-26T21:07:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:37 smithi119 ceph-mon[32752]: pgmap v5612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:37 smithi167 ceph-mon[32751]: pgmap v5612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:37 smithi090 ceph-mon[29180]: pgmap v5612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:37.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9542 sec 2024-04-26T21:07:38.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9543 sec 2024-04-26T21:07:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:39 smithi119 ceph-mon[32752]: pgmap v5613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:39 smithi167 ceph-mon[32751]: pgmap v5613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:39 smithi090 ceph-mon[29180]: pgmap v5613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:39.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9544 sec 2024-04-26T21:07:40.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9545 sec 2024-04-26T21:07:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9142.115542 secs 2024-04-26T21:07:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:40 smithi119 ceph-mon[32752]: pgmap v5614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9142.834529 secs 2024-04-26T21:07:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9142.115542 secs 2024-04-26T21:07:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:40 smithi167 ceph-mon[32751]: pgmap v5614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9142.834529 secs 2024-04-26T21:07:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9142.115542 secs 2024-04-26T21:07:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:40 smithi090 ceph-mon[29180]: pgmap v5614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9142.834529 secs 2024-04-26T21:07:41.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9546 sec 2024-04-26T21:07:42.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9547 sec 2024-04-26T21:07:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:43 smithi119 ceph-mon[32752]: pgmap v5615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:43 smithi167 ceph-mon[32751]: pgmap v5615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:43 smithi090 ceph-mon[29180]: pgmap v5615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:43.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9548 sec 2024-04-26T21:07:44.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9549 sec 2024-04-26T21:07:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:45 smithi119 ceph-mon[32752]: pgmap v5616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9147.115601 secs 2024-04-26T21:07:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:45 smithi167 ceph-mon[32751]: pgmap v5616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9147.115601 secs 2024-04-26T21:07:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:45 smithi090 ceph-mon[29180]: pgmap v5616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9147.115601 secs 2024-04-26T21:07:45.746 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9550 sec 2024-04-26T21:07:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9147.834600 secs 2024-04-26T21:07:46.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9147.834600 secs 2024-04-26T21:07:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9147.834600 secs 2024-04-26T21:07:46.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9551 sec 2024-04-26T21:07:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:47 smithi119 ceph-mon[32752]: pgmap v5617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:47 smithi167 ceph-mon[32751]: pgmap v5617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:47 smithi090 ceph-mon[29180]: pgmap v5617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:47.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9552 sec 2024-04-26T21:07:48.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9553 sec 2024-04-26T21:07:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:49 smithi119 ceph-mon[32752]: pgmap v5618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:49 smithi167 ceph-mon[32751]: pgmap v5618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:49 smithi090 ceph-mon[29180]: pgmap v5618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:49.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9554 sec 2024-04-26T21:07:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9152.115662 secs 2024-04-26T21:07:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9152.115662 secs 2024-04-26T21:07:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9152.115662 secs 2024-04-26T21:07:50.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9555 sec 2024-04-26T21:07:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:51 smithi119 ceph-mon[32752]: pgmap v5619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9152.834731 secs 2024-04-26T21:07:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:51 smithi167 ceph-mon[32751]: pgmap v5619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9152.834731 secs 2024-04-26T21:07:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:51 smithi090 ceph-mon[29180]: pgmap v5619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9152.834731 secs 2024-04-26T21:07:51.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9556 sec 2024-04-26T21:07:52.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9557 sec 2024-04-26T21:07:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:53 smithi119 ceph-mon[32752]: pgmap v5620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:53 smithi167 ceph-mon[32751]: pgmap v5620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:53 smithi090 ceph-mon[29180]: pgmap v5620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:07:53.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9558 sec 2024-04-26T21:07:54.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9559 sec 2024-04-26T21:07:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:55 smithi119 ceph-mon[32752]: pgmap v5621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9157.115725 secs 2024-04-26T21:07:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:07:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:55 smithi167 ceph-mon[32751]: pgmap v5621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9157.115725 secs 2024-04-26T21:07:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:55.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:07:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:55 smithi090 ceph-mon[29180]: pgmap v5621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:07:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:07:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9157.115725 secs 2024-04-26T21:07:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:07:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:07:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:07:55.747 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9560 sec 2024-04-26T21:07:56.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:56 smithi090 ceph-mon[29180]: pgmap v5622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:07:56.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:56 smithi090 ceph-mon[29180]: pgmap v5623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:56.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9157.834864 secs 2024-04-26T21:07:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:56 smithi119 ceph-mon[32752]: pgmap v5622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:07:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:56 smithi119 ceph-mon[32752]: pgmap v5623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9157.834864 secs 2024-04-26T21:07:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:56 smithi167 ceph-mon[32751]: pgmap v5622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:07:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:56 smithi167 ceph-mon[32751]: pgmap v5623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9157.834864 secs 2024-04-26T21:07:56.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9561 sec 2024-04-26T21:07:57.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9562 sec 2024-04-26T21:07:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:07:58 smithi119 ceph-mon[32752]: pgmap v5624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:07:58 smithi167 ceph-mon[32751]: pgmap v5624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:07:58 smithi090 ceph-mon[29180]: pgmap v5624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:07:58.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9563 sec 2024-04-26T21:07:59.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9564 sec 2024-04-26T21:08:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:00 smithi119 ceph-mon[32752]: pgmap v5625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9162.115782 secs 2024-04-26T21:08:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:00 smithi167 ceph-mon[32751]: pgmap v5625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9162.115782 secs 2024-04-26T21:08:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:00 smithi090 ceph-mon[29180]: pgmap v5625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9162.115782 secs 2024-04-26T21:08:00.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9565 sec 2024-04-26T21:08:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9162.834940 secs 2024-04-26T21:08:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9162.834940 secs 2024-04-26T21:08:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9162.834940 secs 2024-04-26T21:08:01.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9566 sec 2024-04-26T21:08:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:02 smithi119 ceph-mon[32752]: pgmap v5626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:02 smithi167 ceph-mon[32751]: pgmap v5626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:02 smithi090 ceph-mon[29180]: pgmap v5626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:02.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9567 sec 2024-04-26T21:08:03.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9568 sec 2024-04-26T21:08:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:04 smithi119 ceph-mon[32752]: pgmap v5627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:08:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:04 smithi167 ceph-mon[32751]: pgmap v5627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:08:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:04 smithi090 ceph-mon[29180]: pgmap v5627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:08:04.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9569 sec 2024-04-26T21:08:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9167.115843 secs 2024-04-26T21:08:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9167.115843 secs 2024-04-26T21:08:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9167.115843 secs 2024-04-26T21:08:05.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9570 sec 2024-04-26T21:08:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:06 smithi119 ceph-mon[32752]: pgmap v5628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:08:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9167.835020 secs 2024-04-26T21:08:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:06 smithi167 ceph-mon[32751]: pgmap v5628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:08:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9167.835020 secs 2024-04-26T21:08:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:06 smithi090 ceph-mon[29180]: pgmap v5628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:08:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9167.835020 secs 2024-04-26T21:08:06.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9571 sec 2024-04-26T21:08:07.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9572 sec 2024-04-26T21:08:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:08 smithi119 ceph-mon[32752]: pgmap v5629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:08 smithi167 ceph-mon[32751]: pgmap v5629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:08 smithi090 ceph-mon[29180]: pgmap v5629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:08.748 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9573 sec 2024-04-26T21:08:09.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9574 sec 2024-04-26T21:08:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:10 smithi119 ceph-mon[32752]: pgmap v5630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9172.115877 secs 2024-04-26T21:08:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:10 smithi167 ceph-mon[32751]: pgmap v5630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9172.115877 secs 2024-04-26T21:08:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:10 smithi090 ceph-mon[29180]: pgmap v5630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9172.115877 secs 2024-04-26T21:08:10.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9575 sec 2024-04-26T21:08:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9172.835152 secs 2024-04-26T21:08:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9172.835152 secs 2024-04-26T21:08:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9172.835152 secs 2024-04-26T21:08:11.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9576 sec 2024-04-26T21:08:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:12 smithi119 ceph-mon[32752]: pgmap v5631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:12 smithi167 ceph-mon[32751]: pgmap v5631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:12 smithi090 ceph-mon[29180]: pgmap v5631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:12.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9577 sec 2024-04-26T21:08:13.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9578 sec 2024-04-26T21:08:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:14 smithi119 ceph-mon[32752]: pgmap v5632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:14 smithi167 ceph-mon[32751]: pgmap v5632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:14 smithi090 ceph-mon[29180]: pgmap v5632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:14.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9579 sec 2024-04-26T21:08:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9177.115937 secs 2024-04-26T21:08:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9177.115937 secs 2024-04-26T21:08:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9177.115937 secs 2024-04-26T21:08:15.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9580 sec 2024-04-26T21:08:16.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:16 smithi090 ceph-mon[29180]: pgmap v5633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:16.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9177.835296 secs 2024-04-26T21:08:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:16 smithi119 ceph-mon[32752]: pgmap v5633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9177.835296 secs 2024-04-26T21:08:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:16 smithi167 ceph-mon[32751]: pgmap v5633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9177.835296 secs 2024-04-26T21:08:16.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9581 sec 2024-04-26T21:08:17.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9582 sec 2024-04-26T21:08:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:18 smithi119 ceph-mon[32752]: pgmap v5634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:18 smithi167 ceph-mon[32751]: pgmap v5634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:18 smithi090 ceph-mon[29180]: pgmap v5634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:18.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9583 sec 2024-04-26T21:08:19.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9584 sec 2024-04-26T21:08:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:20 smithi119 ceph-mon[32752]: pgmap v5635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9182.115998 secs 2024-04-26T21:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:20 smithi167 ceph-mon[32751]: pgmap v5635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9182.115998 secs 2024-04-26T21:08:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:20 smithi090 ceph-mon[29180]: pgmap v5635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9182.115998 secs 2024-04-26T21:08:20.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9585 sec 2024-04-26T21:08:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9182.835419 secs 2024-04-26T21:08:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9182.835419 secs 2024-04-26T21:08:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9182.835419 secs 2024-04-26T21:08:21.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9586 sec 2024-04-26T21:08:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:22 smithi119 ceph-mon[32752]: pgmap v5636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:22 smithi167 ceph-mon[32751]: pgmap v5636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:22 smithi090 ceph-mon[29180]: pgmap v5636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:22.749 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9587 sec 2024-04-26T21:08:23.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9588 sec 2024-04-26T21:08:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:24 smithi119 ceph-mon[32752]: pgmap v5637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:24 smithi167 ceph-mon[32751]: pgmap v5637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:24 smithi090 ceph-mon[29180]: pgmap v5637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:24.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9589 sec 2024-04-26T21:08:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9187.116049 secs 2024-04-26T21:08:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9187.116049 secs 2024-04-26T21:08:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9187.116049 secs 2024-04-26T21:08:25.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9590 sec 2024-04-26T21:08:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:26 smithi119 ceph-mon[32752]: pgmap v5638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9187.835543 secs 2024-04-26T21:08:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:26 smithi167 ceph-mon[32751]: pgmap v5638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9187.835543 secs 2024-04-26T21:08:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:26 smithi090 ceph-mon[29180]: pgmap v5638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9187.835543 secs 2024-04-26T21:08:26.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9591 sec 2024-04-26T21:08:27.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9592 sec 2024-04-26T21:08:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:28 smithi119 ceph-mon[32752]: pgmap v5639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:28 smithi167 ceph-mon[32751]: pgmap v5639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:28 smithi090 ceph-mon[29180]: pgmap v5639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:28.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9593 sec 2024-04-26T21:08:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:08:29.696+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:08:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:08:29.696+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:08:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:08:29.696+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 74 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:08:29.697 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:08:29.696+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 74 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:08:29.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9594 sec 2024-04-26T21:08:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:30 smithi119 ceph-mon[32752]: pgmap v5640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9192.116110 secs 2024-04-26T21:08:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:30 smithi167 ceph-mon[32751]: pgmap v5640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9192.116110 secs 2024-04-26T21:08:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:30 smithi090 ceph-mon[29180]: pgmap v5640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9192.116110 secs 2024-04-26T21:08:30.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9595 sec 2024-04-26T21:08:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9192.835672 secs 2024-04-26T21:08:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9192.835672 secs 2024-04-26T21:08:31.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9596 sec 2024-04-26T21:08:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9192.835672 secs 2024-04-26T21:08:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:32 smithi167 ceph-mon[32751]: pgmap v5641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:32 smithi090 ceph-mon[29180]: pgmap v5641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:32.750 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9597 sec 2024-04-26T21:08:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:32 smithi119 ceph-mon[32752]: pgmap v5641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:33.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9598 sec 2024-04-26T21:08:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:34 smithi167 ceph-mon[32751]: pgmap v5642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:34 smithi090 ceph-mon[29180]: pgmap v5642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:34.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9599 sec 2024-04-26T21:08:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:34 smithi119 ceph-mon[32752]: pgmap v5642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9197.116171 secs 2024-04-26T21:08:35.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9600 sec 2024-04-26T21:08:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9197.116171 secs 2024-04-26T21:08:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9197.116171 secs 2024-04-26T21:08:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:36 smithi090 ceph-mon[29180]: pgmap v5643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9197.835770 secs 2024-04-26T21:08:36.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:36 smithi167 ceph-mon[32751]: pgmap v5643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9197.835770 secs 2024-04-26T21:08:36.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9601 sec 2024-04-26T21:08:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:36 smithi119 ceph-mon[32752]: pgmap v5643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9197.835770 secs 2024-04-26T21:08:37.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9602 sec 2024-04-26T21:08:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:38 smithi090 ceph-mon[29180]: pgmap v5644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:38.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9603 sec 2024-04-26T21:08:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:38 smithi119 ceph-mon[32752]: pgmap v5644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:38 smithi167 ceph-mon[32751]: pgmap v5644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:08:39.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9604 sec 2024-04-26T21:08:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:40 smithi090 ceph-mon[29180]: pgmap v5645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9202.116234 secs 2024-04-26T21:08:40.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9605 sec 2024-04-26T21:08:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:40 smithi119 ceph-mon[32752]: pgmap v5645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9202.116234 secs 2024-04-26T21:08:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:40 smithi167 ceph-mon[32751]: pgmap v5645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9202.116234 secs 2024-04-26T21:08:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9202.835848 secs 2024-04-26T21:08:41.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9606 sec 2024-04-26T21:08:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9202.835848 secs 2024-04-26T21:08:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9202.835848 secs 2024-04-26T21:08:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:42 smithi090 ceph-mon[29180]: pgmap v5646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:42.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9607 sec 2024-04-26T21:08:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:42 smithi119 ceph-mon[32752]: pgmap v5646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:42 smithi167 ceph-mon[32751]: pgmap v5646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:43.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9608 sec 2024-04-26T21:08:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:44 smithi090 ceph-mon[29180]: pgmap v5647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:44.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9609 sec 2024-04-26T21:08:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:44 smithi119 ceph-mon[32752]: pgmap v5647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:44 smithi167 ceph-mon[32751]: pgmap v5647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9207.116292 secs 2024-04-26T21:08:45.751 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9610 sec 2024-04-26T21:08:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9207.116292 secs 2024-04-26T21:08:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9207.116292 secs 2024-04-26T21:08:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:46 smithi090 ceph-mon[29180]: pgmap v5648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9207.835964 secs 2024-04-26T21:08:46.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:46 smithi119 ceph-mon[32752]: pgmap v5648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:46.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9207.835964 secs 2024-04-26T21:08:46.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9611 sec 2024-04-26T21:08:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:46 smithi167 ceph-mon[32751]: pgmap v5648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9207.835964 secs 2024-04-26T21:08:47.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9612 sec 2024-04-26T21:08:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:48 smithi090 ceph-mon[29180]: pgmap v5649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:48.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9613 sec 2024-04-26T21:08:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:48 smithi119 ceph-mon[32752]: pgmap v5649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:48 smithi167 ceph-mon[32751]: pgmap v5649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:49.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9614 sec 2024-04-26T21:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:50 smithi090 ceph-mon[29180]: pgmap v5650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9212.116351 secs 2024-04-26T21:08:50.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9615 sec 2024-04-26T21:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:50 smithi119 ceph-mon[32752]: pgmap v5650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9212.116351 secs 2024-04-26T21:08:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:50 smithi167 ceph-mon[32751]: pgmap v5650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9212.116351 secs 2024-04-26T21:08:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9212.836085 secs 2024-04-26T21:08:51.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9616 sec 2024-04-26T21:08:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9212.836085 secs 2024-04-26T21:08:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9212.836085 secs 2024-04-26T21:08:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:52 smithi090 ceph-mon[29180]: pgmap v5651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:52.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9617 sec 2024-04-26T21:08:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:52 smithi119 ceph-mon[32752]: pgmap v5651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:52 smithi167 ceph-mon[32751]: pgmap v5651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:53.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9618 sec 2024-04-26T21:08:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:54 smithi090 ceph-mon[29180]: pgmap v5652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:54.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9619 sec 2024-04-26T21:08:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:54 smithi119 ceph-mon[32752]: pgmap v5652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:54 smithi167 ceph-mon[32751]: pgmap v5652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:08:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9217.116409 secs 2024-04-26T21:08:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:55.752 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9620 sec 2024-04-26T21:08:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9217.116409 secs 2024-04-26T21:08:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9217.116409 secs 2024-04-26T21:08:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:08:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:56 smithi090 ceph-mon[29180]: pgmap v5653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9217.836178 secs 2024-04-26T21:08:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:56 smithi090 ceph-mon[29180]: pgmap v5654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:08:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:56 smithi090 ceph-mon[29180]: pgmap v5655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:08:56.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9621 sec 2024-04-26T21:08:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:56 smithi119 ceph-mon[32752]: pgmap v5653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9217.836178 secs 2024-04-26T21:08:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:56 smithi119 ceph-mon[32752]: pgmap v5654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:08:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:56 smithi119 ceph-mon[32752]: pgmap v5655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:08:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:56 smithi167 ceph-mon[32751]: pgmap v5653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:08:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9217.836178 secs 2024-04-26T21:08:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:08:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:08:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:56 smithi167 ceph-mon[32751]: pgmap v5654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:08:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:56 smithi167 ceph-mon[32751]: pgmap v5655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:08:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:08:57.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9622 sec 2024-04-26T21:08:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:57 smithi119 ceph-mon[32752]: pgmap v5656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:08:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:57 smithi167 ceph-mon[32751]: pgmap v5656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:08:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:57 smithi090 ceph-mon[29180]: pgmap v5656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:08:58.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9623 sec 2024-04-26T21:08:59.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9624 sec 2024-04-26T21:09:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:08:59 smithi119 ceph-mon[32752]: pgmap v5657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:08:59 smithi167 ceph-mon[32751]: pgmap v5657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:08:59 smithi090 ceph-mon[29180]: pgmap v5657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:00.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9625 sec 2024-04-26T21:09:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9222.116468 secs 2024-04-26T21:09:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:00 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9222.836253 secs 2024-04-26T21:09:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9222.116468 secs 2024-04-26T21:09:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:00 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9222.836253 secs 2024-04-26T21:09:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9222.116468 secs 2024-04-26T21:09:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:00 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9222.836253 secs 2024-04-26T21:09:01.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9626 sec 2024-04-26T21:09:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:01 smithi119 ceph-mon[32752]: pgmap v5658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:01 smithi167 ceph-mon[32751]: pgmap v5658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:01 smithi090 ceph-mon[29180]: pgmap v5658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:02.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9627 sec 2024-04-26T21:09:03.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9628 sec 2024-04-26T21:09:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:03 smithi119 ceph-mon[32752]: pgmap v5659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:03 smithi167 ceph-mon[32751]: pgmap v5659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:03 smithi090 ceph-mon[29180]: pgmap v5659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:04.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9629 sec 2024-04-26T21:09:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9227.116528 secs 2024-04-26T21:09:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9227.116528 secs 2024-04-26T21:09:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:04 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9227.116528 secs 2024-04-26T21:09:05.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9630 sec 2024-04-26T21:09:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9227.836399 secs 2024-04-26T21:09:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:06 smithi119 ceph-mon[32752]: pgmap v5660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:09:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9227.836399 secs 2024-04-26T21:09:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:06 smithi167 ceph-mon[32751]: pgmap v5660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:09:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9227.836399 secs 2024-04-26T21:09:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:06 smithi090 ceph-mon[29180]: pgmap v5660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:09:06.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9631 sec 2024-04-26T21:09:07.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9632 sec 2024-04-26T21:09:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:07 smithi119 ceph-mon[32752]: pgmap v5661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:07 smithi167 ceph-mon[32751]: pgmap v5661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:07 smithi090 ceph-mon[29180]: pgmap v5661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:08.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9633 sec 2024-04-26T21:09:09.753 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9634 sec 2024-04-26T21:09:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:09 smithi119 ceph-mon[32752]: pgmap v5662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:09 smithi167 ceph-mon[32751]: pgmap v5662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:09 smithi090 ceph-mon[29180]: pgmap v5662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:10.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9635 sec 2024-04-26T21:09:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9232.116588 secs 2024-04-26T21:09:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:10 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9232.836506 secs 2024-04-26T21:09:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9232.116588 secs 2024-04-26T21:09:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:10 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9232.836506 secs 2024-04-26T21:09:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9232.116588 secs 2024-04-26T21:09:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:10 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9232.836506 secs 2024-04-26T21:09:11.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9636 sec 2024-04-26T21:09:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:11 smithi119 ceph-mon[32752]: pgmap v5663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:11 smithi167 ceph-mon[32751]: pgmap v5663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:11 smithi090 ceph-mon[29180]: pgmap v5663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:12.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9637 sec 2024-04-26T21:09:13.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9638 sec 2024-04-26T21:09:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:13 smithi119 ceph-mon[32752]: pgmap v5664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:13 smithi167 ceph-mon[32751]: pgmap v5664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:13 smithi090 ceph-mon[29180]: pgmap v5664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:14.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9639 sec 2024-04-26T21:09:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9237.116647 secs 2024-04-26T21:09:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9237.116647 secs 2024-04-26T21:09:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9237.116647 secs 2024-04-26T21:09:15.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9640 sec 2024-04-26T21:09:16.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9237.836566 secs 2024-04-26T21:09:16.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:16 smithi090 ceph-mon[29180]: pgmap v5665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9237.836566 secs 2024-04-26T21:09:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:16 smithi119 ceph-mon[32752]: pgmap v5665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9237.836566 secs 2024-04-26T21:09:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:16 smithi167 ceph-mon[32751]: pgmap v5665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:16.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9641 sec 2024-04-26T21:09:17.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9642 sec 2024-04-26T21:09:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:17 smithi119 ceph-mon[32752]: pgmap v5666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:17 smithi167 ceph-mon[32751]: pgmap v5666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:17 smithi090 ceph-mon[29180]: pgmap v5666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:18.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9643 sec 2024-04-26T21:09:19.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9644 sec 2024-04-26T21:09:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:19 smithi119 ceph-mon[32752]: pgmap v5667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:19 smithi167 ceph-mon[32751]: pgmap v5667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:19 smithi090 ceph-mon[29180]: pgmap v5667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:20.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9645 sec 2024-04-26T21:09:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9242.116709 secs 2024-04-26T21:09:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9242.836696 secs 2024-04-26T21:09:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9242.116709 secs 2024-04-26T21:09:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9242.836696 secs 2024-04-26T21:09:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9242.116709 secs 2024-04-26T21:09:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9242.836696 secs 2024-04-26T21:09:21.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9646 sec 2024-04-26T21:09:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:21 smithi119 ceph-mon[32752]: pgmap v5668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:21 smithi167 ceph-mon[32751]: pgmap v5668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:21 smithi090 ceph-mon[29180]: pgmap v5668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:22.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9647 sec 2024-04-26T21:09:23.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9648 sec 2024-04-26T21:09:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:23 smithi119 ceph-mon[32752]: pgmap v5669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:23 smithi167 ceph-mon[32751]: pgmap v5669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:23 smithi090 ceph-mon[29180]: pgmap v5669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:24.754 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9649 sec 2024-04-26T21:09:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9247.116768 secs 2024-04-26T21:09:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9247.116768 secs 2024-04-26T21:09:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9247.116768 secs 2024-04-26T21:09:25.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9650 sec 2024-04-26T21:09:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9247.836822 secs 2024-04-26T21:09:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:26 smithi119 ceph-mon[32752]: pgmap v5670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9247.836822 secs 2024-04-26T21:09:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:26 smithi167 ceph-mon[32751]: pgmap v5670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9247.836822 secs 2024-04-26T21:09:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:26 smithi090 ceph-mon[29180]: pgmap v5670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:26.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9651 sec 2024-04-26T21:09:27.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9652 sec 2024-04-26T21:09:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:27 smithi119 ceph-mon[32752]: pgmap v5671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:27 smithi167 ceph-mon[32751]: pgmap v5671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:27 smithi090 ceph-mon[29180]: pgmap v5671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:28.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9653 sec 2024-04-26T21:09:29.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9654 sec 2024-04-26T21:09:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:29 smithi119 ceph-mon[32752]: pgmap v5672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:29 smithi167 ceph-mon[32751]: pgmap v5672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:29 smithi090 ceph-mon[29180]: pgmap v5672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:30.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9655 sec 2024-04-26T21:09:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9252.116829 secs 2024-04-26T21:09:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9252.836965 secs 2024-04-26T21:09:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9252.116829 secs 2024-04-26T21:09:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9252.836965 secs 2024-04-26T21:09:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9252.116829 secs 2024-04-26T21:09:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9252.836965 secs 2024-04-26T21:09:31.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9656 sec 2024-04-26T21:09:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:31 smithi119 ceph-mon[32752]: pgmap v5673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:31 smithi167 ceph-mon[32751]: pgmap v5673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:31 smithi090 ceph-mon[29180]: pgmap v5673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:32.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9657 sec 2024-04-26T21:09:33.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9658 sec 2024-04-26T21:09:34.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:33 smithi119 ceph-mon[32752]: pgmap v5674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:33 smithi167 ceph-mon[32751]: pgmap v5674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:33 smithi090 ceph-mon[29180]: pgmap v5674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:34.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9659 sec 2024-04-26T21:09:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9257.116885 secs 2024-04-26T21:09:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9257.116885 secs 2024-04-26T21:09:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9257.116885 secs 2024-04-26T21:09:35.755 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9660 sec 2024-04-26T21:09:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9257.837090 secs 2024-04-26T21:09:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:36 smithi119 ceph-mon[32752]: pgmap v5675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9257.837090 secs 2024-04-26T21:09:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:36 smithi167 ceph-mon[32751]: pgmap v5675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9257.837090 secs 2024-04-26T21:09:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:36 smithi090 ceph-mon[29180]: pgmap v5675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:36.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9661 sec 2024-04-26T21:09:37.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9662 sec 2024-04-26T21:09:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:37 smithi119 ceph-mon[32752]: pgmap v5676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:37 smithi167 ceph-mon[32751]: pgmap v5676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:37 smithi090 ceph-mon[29180]: pgmap v5676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:38.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9663 sec 2024-04-26T21:09:39.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9664 sec 2024-04-26T21:09:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:39 smithi119 ceph-mon[32752]: pgmap v5677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:39 smithi167 ceph-mon[32751]: pgmap v5677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:39 smithi090 ceph-mon[29180]: pgmap v5677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:40.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9665 sec 2024-04-26T21:09:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9262.116946 secs 2024-04-26T21:09:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9262.837183 secs 2024-04-26T21:09:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9262.116946 secs 2024-04-26T21:09:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9262.837183 secs 2024-04-26T21:09:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9262.116946 secs 2024-04-26T21:09:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9262.837183 secs 2024-04-26T21:09:41.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9666 sec 2024-04-26T21:09:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:41 smithi119 ceph-mon[32752]: pgmap v5678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:41 smithi167 ceph-mon[32751]: pgmap v5678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:41 smithi090 ceph-mon[29180]: pgmap v5678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:42.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9667 sec 2024-04-26T21:09:43.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9668 sec 2024-04-26T21:09:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:43 smithi119 ceph-mon[32752]: pgmap v5679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:43 smithi167 ceph-mon[32751]: pgmap v5679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:43 smithi090 ceph-mon[29180]: pgmap v5679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:44.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9669 sec 2024-04-26T21:09:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9267.117005 secs 2024-04-26T21:09:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9267.117005 secs 2024-04-26T21:09:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9267.117005 secs 2024-04-26T21:09:45.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9670 sec 2024-04-26T21:09:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9267.837327 secs 2024-04-26T21:09:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:46 smithi119 ceph-mon[32752]: pgmap v5680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9267.837327 secs 2024-04-26T21:09:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:46 smithi167 ceph-mon[32751]: pgmap v5680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9267.837327 secs 2024-04-26T21:09:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:46 smithi090 ceph-mon[29180]: pgmap v5680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:46.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9671 sec 2024-04-26T21:09:47.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9672 sec 2024-04-26T21:09:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:47 smithi119 ceph-mon[32752]: pgmap v5681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:47 smithi167 ceph-mon[32751]: pgmap v5681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:47 smithi090 ceph-mon[29180]: pgmap v5681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:48.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9673 sec 2024-04-26T21:09:49.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9674 sec 2024-04-26T21:09:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:49 smithi119 ceph-mon[32752]: pgmap v5682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:49 smithi167 ceph-mon[32751]: pgmap v5682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:49 smithi090 ceph-mon[29180]: pgmap v5682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:50.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9675 sec 2024-04-26T21:09:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9272.117063 secs 2024-04-26T21:09:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:50 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9272.837446 secs 2024-04-26T21:09:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9272.117063 secs 2024-04-26T21:09:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:50 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9272.837446 secs 2024-04-26T21:09:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9272.117063 secs 2024-04-26T21:09:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:50 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9272.837446 secs 2024-04-26T21:09:51.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9676 sec 2024-04-26T21:09:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:51 smithi119 ceph-mon[32752]: pgmap v5683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:51 smithi167 ceph-mon[32751]: pgmap v5683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:51 smithi090 ceph-mon[29180]: pgmap v5683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:52.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9677 sec 2024-04-26T21:09:53.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9678 sec 2024-04-26T21:09:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:53 smithi119 ceph-mon[32752]: pgmap v5684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:53 smithi167 ceph-mon[32751]: pgmap v5684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:53 smithi090 ceph-mon[29180]: pgmap v5684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:09:54.756 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9679 sec 2024-04-26T21:09:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9277.117118 secs 2024-04-26T21:09:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9277.117118 secs 2024-04-26T21:09:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9277.117118 secs 2024-04-26T21:09:55.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9680 sec 2024-04-26T21:09:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9277.837567 secs 2024-04-26T21:09:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:56 smithi119 ceph-mon[32752]: pgmap v5685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9277.837567 secs 2024-04-26T21:09:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:56 smithi167 ceph-mon[32751]: pgmap v5685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9277.837567 secs 2024-04-26T21:09:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:56 smithi090 ceph-mon[29180]: pgmap v5685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:09:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:09:56.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9681 sec 2024-04-26T21:09:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:57 smithi119 ceph-mon[32752]: pgmap v5686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:09:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:57 smithi119 ceph-mon[32752]: pgmap v5687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:09:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:57 smithi167 ceph-mon[32751]: pgmap v5686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:09:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:57 smithi167 ceph-mon[32751]: pgmap v5687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:09:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:09:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:09:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:57 smithi090 ceph-mon[29180]: pgmap v5686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:09:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:57 smithi090 ceph-mon[29180]: pgmap v5687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:09:57.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9682 sec 2024-04-26T21:09:58.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9683 sec 2024-04-26T21:09:59.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9684 sec 2024-04-26T21:09:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:09:59 smithi119 ceph-mon[32752]: pgmap v5688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:09:59 smithi167 ceph-mon[32751]: pgmap v5688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:09:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:09:59 smithi090 ceph-mon[29180]: pgmap v5688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:10:00.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9685 sec 2024-04-26T21:10:00.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9282.117179 secs 2024-04-26T21:10:00.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:10:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9282.117179 secs 2024-04-26T21:10:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:10:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9282.117179 secs 2024-04-26T21:10:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:10:01.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9686 sec 2024-04-26T21:10:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:01 smithi119 ceph-mon[32752]: pgmap v5689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:10:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9282.837650 secs 2024-04-26T21:10:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:01 smithi167 ceph-mon[32751]: pgmap v5689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:10:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9282.837650 secs 2024-04-26T21:10:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:01 smithi090 ceph-mon[29180]: pgmap v5689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:10:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9282.837650 secs 2024-04-26T21:10:02.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9687 sec 2024-04-26T21:10:03.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9688 sec 2024-04-26T21:10:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:03 smithi119 ceph-mon[32752]: pgmap v5690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:10:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:03 smithi167 ceph-mon[32751]: pgmap v5690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:10:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:03 smithi090 ceph-mon[29180]: pgmap v5690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:10:04.757 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9689 sec 2024-04-26T21:10:05.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9690 sec 2024-04-26T21:10:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:05 smithi119 ceph-mon[32752]: pgmap v5691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T21:10:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9287.117240 secs 2024-04-26T21:10:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:05 smithi167 ceph-mon[32751]: pgmap v5691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T21:10:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9287.117240 secs 2024-04-26T21:10:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:05 smithi090 ceph-mon[29180]: pgmap v5691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T21:10:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9287.117240 secs 2024-04-26T21:10:06.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9691 sec 2024-04-26T21:10:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9287.837732 secs 2024-04-26T21:10:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9287.837732 secs 2024-04-26T21:10:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9287.837732 secs 2024-04-26T21:10:07.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9692 sec 2024-04-26T21:10:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:07 smithi119 ceph-mon[32752]: pgmap v5692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T21:10:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:07 smithi167 ceph-mon[32751]: pgmap v5692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T21:10:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:07 smithi090 ceph-mon[29180]: pgmap v5692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T21:10:08.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9693 sec 2024-04-26T21:10:09.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9694 sec 2024-04-26T21:10:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:09 smithi119 ceph-mon[32752]: pgmap v5693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:10:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:09 smithi167 ceph-mon[32751]: pgmap v5693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:10:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:09 smithi090 ceph-mon[29180]: pgmap v5693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:10:10.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9695 sec 2024-04-26T21:10:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9292.117300 secs 2024-04-26T21:10:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9292.117300 secs 2024-04-26T21:10:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9292.117300 secs 2024-04-26T21:10:11.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9696 sec 2024-04-26T21:10:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:11 smithi119 ceph-mon[32752]: pgmap v5694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:10:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9292.837859 secs 2024-04-26T21:10:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:11 smithi167 ceph-mon[32751]: pgmap v5694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:10:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9292.837859 secs 2024-04-26T21:10:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:11 smithi090 ceph-mon[29180]: pgmap v5694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:10:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9292.837859 secs 2024-04-26T21:10:12.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9697 sec 2024-04-26T21:10:13.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9698 sec 2024-04-26T21:10:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:13 smithi167 ceph-mon[32751]: pgmap v5695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:10:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:13 smithi090 ceph-mon[29180]: pgmap v5695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:10:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:13 smithi119 ceph-mon[32752]: pgmap v5695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:10:14.758 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9699 sec 2024-04-26T21:10:15.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9700 sec 2024-04-26T21:10:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:15 smithi167 ceph-mon[32751]: pgmap v5696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:10:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9297.117360 secs 2024-04-26T21:10:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:15 smithi090 ceph-mon[29180]: pgmap v5696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:10:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9297.117360 secs 2024-04-26T21:10:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:15 smithi119 ceph-mon[32752]: pgmap v5696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:10:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9297.117360 secs 2024-04-26T21:10:16.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9701 sec 2024-04-26T21:10:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9297.837940 secs 2024-04-26T21:10:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9297.837940 secs 2024-04-26T21:10:16.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9297.837940 secs 2024-04-26T21:10:17.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9702 sec 2024-04-26T21:10:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:17 smithi167 ceph-mon[32751]: pgmap v5697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:10:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:17 smithi090 ceph-mon[29180]: pgmap v5697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:10:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:17 smithi119 ceph-mon[32752]: pgmap v5697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T21:10:18.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9703 sec 2024-04-26T21:10:19.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9704 sec 2024-04-26T21:10:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:19 smithi090 ceph-mon[29180]: pgmap v5698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:10:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:19 smithi119 ceph-mon[32752]: pgmap v5698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:10:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:19 smithi167 ceph-mon[32751]: pgmap v5698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:10:20.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9705 sec 2024-04-26T21:10:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9302.117418 secs 2024-04-26T21:10:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9302.117418 secs 2024-04-26T21:10:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9302.117418 secs 2024-04-26T21:10:21.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9706 sec 2024-04-26T21:10:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:21 smithi090 ceph-mon[29180]: pgmap v5699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9302.838004 secs 2024-04-26T21:10:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:21 smithi119 ceph-mon[32752]: pgmap v5699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9302.838004 secs 2024-04-26T21:10:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:21 smithi167 ceph-mon[32751]: pgmap v5699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9302.838004 secs 2024-04-26T21:10:22.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9707 sec 2024-04-26T21:10:23.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9708 sec 2024-04-26T21:10:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:23 smithi090 ceph-mon[29180]: pgmap v5700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:23 smithi119 ceph-mon[32752]: pgmap v5700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:23 smithi167 ceph-mon[32751]: pgmap v5700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:24.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9709 sec 2024-04-26T21:10:25.759 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9710 sec 2024-04-26T21:10:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:25 smithi090 ceph-mon[29180]: pgmap v5701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9307.117478 secs 2024-04-26T21:10:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:25 smithi119 ceph-mon[32752]: pgmap v5701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9307.117478 secs 2024-04-26T21:10:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:25 smithi167 ceph-mon[32751]: pgmap v5701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9307.117478 secs 2024-04-26T21:10:26.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9711 sec 2024-04-26T21:10:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9307.838099 secs 2024-04-26T21:10:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9307.838099 secs 2024-04-26T21:10:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9307.838099 secs 2024-04-26T21:10:27.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9712 sec 2024-04-26T21:10:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:27 smithi090 ceph-mon[29180]: pgmap v5702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:27 smithi119 ceph-mon[32752]: pgmap v5702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:27 smithi167 ceph-mon[32751]: pgmap v5702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:28.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9713 sec 2024-04-26T21:10:29.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9714 sec 2024-04-26T21:10:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:29 smithi090 ceph-mon[29180]: pgmap v5703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:29 smithi119 ceph-mon[32752]: pgmap v5703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:29 smithi167 ceph-mon[32751]: pgmap v5703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:30.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9715 sec 2024-04-26T21:10:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9312.117536 secs 2024-04-26T21:10:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9312.117536 secs 2024-04-26T21:10:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9312.117536 secs 2024-04-26T21:10:31.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9716 sec 2024-04-26T21:10:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:31 smithi090 ceph-mon[29180]: pgmap v5704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9312.838225 secs 2024-04-26T21:10:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:31 smithi119 ceph-mon[32752]: pgmap v5704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9312.838225 secs 2024-04-26T21:10:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:31 smithi167 ceph-mon[32751]: pgmap v5704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9312.838225 secs 2024-04-26T21:10:32.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9717 sec 2024-04-26T21:10:33.608 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:10:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:10:33.287+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:10:33.608 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:10:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:10:33.287+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:10:33.608 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:10:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:10:33.287+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:10:33.609 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:10:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:10:33.287+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:10:33.609 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:10:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:10:33.287+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:10:33.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9718 sec 2024-04-26T21:10:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:33 smithi090 ceph-mon[29180]: pgmap v5705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:33 smithi119 ceph-mon[32752]: pgmap v5705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:33 smithi167 ceph-mon[32751]: pgmap v5705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:34.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9719 sec 2024-04-26T21:10:35.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9720 sec 2024-04-26T21:10:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:35 smithi090 ceph-mon[29180]: pgmap v5706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:10:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9317.117596 secs 2024-04-26T21:10:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:35 smithi119 ceph-mon[32752]: pgmap v5706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:10:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9317.117596 secs 2024-04-26T21:10:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:35 smithi167 ceph-mon[32751]: pgmap v5706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 85 B/s wr, 11 op/s 2024-04-26T21:10:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9317.117596 secs 2024-04-26T21:10:36.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9721 sec 2024-04-26T21:10:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9317.838373 secs 2024-04-26T21:10:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9317.838373 secs 2024-04-26T21:10:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9317.838373 secs 2024-04-26T21:10:37.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9722 sec 2024-04-26T21:10:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:37 smithi090 ceph-mon[29180]: pgmap v5707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:10:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:37 smithi119 ceph-mon[32752]: pgmap v5707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:10:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:37 smithi167 ceph-mon[32751]: pgmap v5707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T21:10:38.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9723 sec 2024-04-26T21:10:39.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9724 sec 2024-04-26T21:10:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:39 smithi090 ceph-mon[29180]: pgmap v5708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:39 smithi119 ceph-mon[32752]: pgmap v5708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:39 smithi167 ceph-mon[32751]: pgmap v5708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:40.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9725 sec 2024-04-26T21:10:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9322.117656 secs 2024-04-26T21:10:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9322.117656 secs 2024-04-26T21:10:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9322.117656 secs 2024-04-26T21:10:41.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9726 sec 2024-04-26T21:10:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:41 smithi119 ceph-mon[32752]: pgmap v5709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9322.838490 secs 2024-04-26T21:10:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:41 smithi167 ceph-mon[32751]: pgmap v5709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9322.838490 secs 2024-04-26T21:10:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:41 smithi090 ceph-mon[29180]: pgmap v5709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9322.838490 secs 2024-04-26T21:10:42.760 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9727 sec 2024-04-26T21:10:43.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9728 sec 2024-04-26T21:10:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:43 smithi119 ceph-mon[32752]: pgmap v5710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:43 smithi167 ceph-mon[32751]: pgmap v5710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:43 smithi090 ceph-mon[29180]: pgmap v5710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:44.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9729 sec 2024-04-26T21:10:45.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9730 sec 2024-04-26T21:10:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:45 smithi119 ceph-mon[32752]: pgmap v5711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9327.117717 secs 2024-04-26T21:10:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:45 smithi167 ceph-mon[32751]: pgmap v5711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9327.117717 secs 2024-04-26T21:10:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:45 smithi090 ceph-mon[29180]: pgmap v5711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:10:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9327.117717 secs 2024-04-26T21:10:46.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9731 sec 2024-04-26T21:10:46.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9327.838567 secs 2024-04-26T21:10:47.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9327.838567 secs 2024-04-26T21:10:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9327.838567 secs 2024-04-26T21:10:47.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9732 sec 2024-04-26T21:10:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:47 smithi090 ceph-mon[29180]: pgmap v5712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T21:10:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:47 smithi119 ceph-mon[32752]: pgmap v5712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T21:10:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:47 smithi167 ceph-mon[32751]: pgmap v5712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T21:10:48.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9733 sec 2024-04-26T21:10:49.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9734 sec 2024-04-26T21:10:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:49 smithi119 ceph-mon[32752]: pgmap v5713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T21:10:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:49 smithi167 ceph-mon[32751]: pgmap v5713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T21:10:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:49 smithi090 ceph-mon[29180]: pgmap v5713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 85 B/s wr, 4 op/s 2024-04-26T21:10:50.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9735 sec 2024-04-26T21:10:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9332.117776 secs 2024-04-26T21:10:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9332.117776 secs 2024-04-26T21:10:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9332.117776 secs 2024-04-26T21:10:51.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9736 sec 2024-04-26T21:10:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:51 smithi119 ceph-mon[32752]: pgmap v5714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9332.838697 secs 2024-04-26T21:10:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:51 smithi167 ceph-mon[32751]: pgmap v5714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9332.838697 secs 2024-04-26T21:10:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:51 smithi090 ceph-mon[29180]: pgmap v5714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9332.838697 secs 2024-04-26T21:10:52.761 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9737 sec 2024-04-26T21:10:53.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9738 sec 2024-04-26T21:10:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:53 smithi119 ceph-mon[32752]: pgmap v5715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:53 smithi167 ceph-mon[32751]: pgmap v5715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:53 smithi090 ceph-mon[29180]: pgmap v5715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:54.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9739 sec 2024-04-26T21:10:55.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9740 sec 2024-04-26T21:10:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:55 smithi119 ceph-mon[32752]: pgmap v5716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 9337.117797 secs 2024-04-26T21:10:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:55 smithi167 ceph-mon[32751]: pgmap v5716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 9337.117797 secs 2024-04-26T21:10:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:55 smithi090 ceph-mon[29180]: pgmap v5716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:10:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 9337.117797 secs 2024-04-26T21:10:56.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9741 sec 2024-04-26T21:10:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9337.838759 secs 2024-04-26T21:10:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9337.838759 secs 2024-04-26T21:10:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9337.838759 secs 2024-04-26T21:10:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:10:57.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9742 sec 2024-04-26T21:10:57.988 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:57 smithi090 ceph-mon[29180]: pgmap v5717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:58.024 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:57 smithi119 ceph-mon[32752]: pgmap v5717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:57 smithi167 ceph-mon[32751]: pgmap v5717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:10:58.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9743 sec 2024-04-26T21:10:59.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9744 sec 2024-04-26T21:11:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:10:59 smithi119 ceph-mon[32752]: pgmap v5718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:10:59 smithi167 ceph-mon[32751]: pgmap v5718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:10:59 smithi090 ceph-mon[29180]: pgmap v5718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:00.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9745 sec 2024-04-26T21:11:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9342.117863 secs 2024-04-26T21:11:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:00 smithi090 ceph-mon[29180]: pgmap v5719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:00 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9342.838825 secs 2024-04-26T21:11:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9342.117863 secs 2024-04-26T21:11:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:00 smithi119 ceph-mon[32752]: pgmap v5719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:00 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9342.838825 secs 2024-04-26T21:11:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9342.117863 secs 2024-04-26T21:11:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:00 smithi167 ceph-mon[32751]: pgmap v5719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:00 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9342.838825 secs 2024-04-26T21:11:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:01.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9746 sec 2024-04-26T21:11:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:02 smithi090 ceph-mon[29180]: pgmap v5720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-26T21:11:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:02 smithi090 ceph-mon[29180]: pgmap v5721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:02 smithi119 ceph-mon[32752]: pgmap v5720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-26T21:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:02 smithi119 ceph-mon[32752]: pgmap v5721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:11:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:11:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:02 smithi167 ceph-mon[32751]: pgmap v5720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-26T21:11:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:02 smithi167 ceph-mon[32751]: pgmap v5721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:11:02.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9747 sec 2024-04-26T21:11:03.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:11:03.416 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:03 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:11:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:11:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:03 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:11:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:11:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:03 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:11:03.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9748 sec 2024-04-26T21:11:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:04 smithi119 ceph-mon[32752]: pgmap v5722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T21:11:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:04 smithi167 ceph-mon[32751]: pgmap v5722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T21:11:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:04 smithi090 ceph-mon[29180]: pgmap v5722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T21:11:04.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9749 sec 2024-04-26T21:11:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9347.117923 secs 2024-04-26T21:11:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9347.117923 secs 2024-04-26T21:11:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9347.117923 secs 2024-04-26T21:11:05.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9750 sec 2024-04-26T21:11:06.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:06 smithi167 ceph-mon[32751]: pgmap v5723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T21:11:06.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9347.838941 secs 2024-04-26T21:11:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:06 smithi119 ceph-mon[32752]: pgmap v5723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T21:11:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9347.838941 secs 2024-04-26T21:11:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:06 smithi090 ceph-mon[29180]: pgmap v5723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T21:11:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9347.838941 secs 2024-04-26T21:11:06.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9751 sec 2024-04-26T21:11:07.762 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9752 sec 2024-04-26T21:11:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:08 smithi119 ceph-mon[32752]: pgmap v5724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:11:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:08 smithi167 ceph-mon[32751]: pgmap v5724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:11:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:08 smithi090 ceph-mon[29180]: pgmap v5724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:11:08.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9753 sec 2024-04-26T21:11:09.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9754 sec 2024-04-26T21:11:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:10 smithi119 ceph-mon[32752]: pgmap v5725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T21:11:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9352.117982 secs 2024-04-26T21:11:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:10 smithi167 ceph-mon[32751]: pgmap v5725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T21:11:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9352.117982 secs 2024-04-26T21:11:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:10 smithi090 ceph-mon[29180]: pgmap v5725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T21:11:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9352.117982 secs 2024-04-26T21:11:10.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9755 sec 2024-04-26T21:11:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9352.839066 secs 2024-04-26T21:11:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9352.839066 secs 2024-04-26T21:11:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9352.839066 secs 2024-04-26T21:11:11.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9756 sec 2024-04-26T21:11:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:12 smithi119 ceph-mon[32752]: pgmap v5726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:11:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:12 smithi167 ceph-mon[32751]: pgmap v5726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:11:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:12 smithi090 ceph-mon[29180]: pgmap v5726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:11:12.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9757 sec 2024-04-26T21:11:13.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9758 sec 2024-04-26T21:11:14.419 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:14 smithi090 ceph-mon[29180]: pgmap v5727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:11:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:14 smithi119 ceph-mon[32752]: pgmap v5727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:11:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:14 smithi167 ceph-mon[32751]: pgmap v5727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:11:14.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9759 sec 2024-04-26T21:11:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9357.118067 secs 2024-04-26T21:11:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9357.118067 secs 2024-04-26T21:11:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9357.118067 secs 2024-04-26T21:11:15.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9760 sec 2024-04-26T21:11:16.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:16 smithi119 ceph-mon[32752]: pgmap v5728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:11:16.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9357.839212 secs 2024-04-26T21:11:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:16 smithi167 ceph-mon[32751]: pgmap v5728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:11:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9357.839212 secs 2024-04-26T21:11:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:16 smithi090 ceph-mon[29180]: pgmap v5728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:11:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9357.839212 secs 2024-04-26T21:11:16.763 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9761 sec 2024-04-26T21:11:17.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9762 sec 2024-04-26T21:11:18.429 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:18 smithi090 ceph-mon[29180]: pgmap v5729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:18 smithi119 ceph-mon[32752]: pgmap v5729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:18 smithi167 ceph-mon[32751]: pgmap v5729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:18.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9763 sec 2024-04-26T21:11:19.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9764 sec 2024-04-26T21:11:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:20 smithi119 ceph-mon[32752]: pgmap v5730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9362.118151 secs 2024-04-26T21:11:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:20 smithi167 ceph-mon[32751]: pgmap v5730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9362.118151 secs 2024-04-26T21:11:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:20 smithi090 ceph-mon[29180]: pgmap v5730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9362.118151 secs 2024-04-26T21:11:20.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9765 sec 2024-04-26T21:11:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9362.839289 secs 2024-04-26T21:11:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9362.839289 secs 2024-04-26T21:11:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9362.839289 secs 2024-04-26T21:11:21.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9766 sec 2024-04-26T21:11:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:22 smithi119 ceph-mon[32752]: pgmap v5731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:22 smithi167 ceph-mon[32751]: pgmap v5731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:22 smithi090 ceph-mon[29180]: pgmap v5731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:22.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9767 sec 2024-04-26T21:11:23.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9768 sec 2024-04-26T21:11:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:24 smithi119 ceph-mon[32752]: pgmap v5732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:24 smithi167 ceph-mon[32751]: pgmap v5732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:24 smithi090 ceph-mon[29180]: pgmap v5732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:24.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9769 sec 2024-04-26T21:11:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9367.118242 secs 2024-04-26T21:11:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9367.118242 secs 2024-04-26T21:11:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9367.118242 secs 2024-04-26T21:11:25.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9770 sec 2024-04-26T21:11:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:26 smithi119 ceph-mon[32752]: pgmap v5733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9367.839412 secs 2024-04-26T21:11:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:26 smithi167 ceph-mon[32751]: pgmap v5733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9367.839412 secs 2024-04-26T21:11:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:26 smithi090 ceph-mon[29180]: pgmap v5733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9367.839412 secs 2024-04-26T21:11:26.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9771 sec 2024-04-26T21:11:27.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9772 sec 2024-04-26T21:11:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:28 smithi119 ceph-mon[32752]: pgmap v5734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:28 smithi167 ceph-mon[32751]: pgmap v5734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:28 smithi090 ceph-mon[29180]: pgmap v5734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:28.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9773 sec 2024-04-26T21:11:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:11:29.720+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:11:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:11:29.720+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:11:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:11:29.720+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 75 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:11:29.721 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:11:29.720+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 75 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:11:29.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9774 sec 2024-04-26T21:11:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:30 smithi119 ceph-mon[32752]: pgmap v5735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9372.118328 secs 2024-04-26T21:11:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:30 smithi167 ceph-mon[32751]: pgmap v5735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9372.118328 secs 2024-04-26T21:11:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:30 smithi090 ceph-mon[29180]: pgmap v5735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9372.118328 secs 2024-04-26T21:11:30.764 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9775 sec 2024-04-26T21:11:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9372.839528 secs 2024-04-26T21:11:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9372.839528 secs 2024-04-26T21:11:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9372.839528 secs 2024-04-26T21:11:31.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9776 sec 2024-04-26T21:11:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:32 smithi119 ceph-mon[32752]: pgmap v5736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:32 smithi167 ceph-mon[32751]: pgmap v5736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:32 smithi090 ceph-mon[29180]: pgmap v5736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:32.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9777 sec 2024-04-26T21:11:33.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9778 sec 2024-04-26T21:11:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:34 smithi119 ceph-mon[32752]: pgmap v5737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:34 smithi167 ceph-mon[32751]: pgmap v5737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:34 smithi090 ceph-mon[29180]: pgmap v5737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:34.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9779 sec 2024-04-26T21:11:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9377.118414 secs 2024-04-26T21:11:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9377.118414 secs 2024-04-26T21:11:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9377.118414 secs 2024-04-26T21:11:35.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9780 sec 2024-04-26T21:11:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:36 smithi119 ceph-mon[32752]: pgmap v5738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9377.839621 secs 2024-04-26T21:11:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:36 smithi167 ceph-mon[32751]: pgmap v5738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9377.839621 secs 2024-04-26T21:11:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:36 smithi090 ceph-mon[29180]: pgmap v5738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9377.839621 secs 2024-04-26T21:11:36.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9781 sec 2024-04-26T21:11:37.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9782 sec 2024-04-26T21:11:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:38 smithi119 ceph-mon[32752]: pgmap v5739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:38 smithi167 ceph-mon[32751]: pgmap v5739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:38 smithi090 ceph-mon[29180]: pgmap v5739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:38.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9783 sec 2024-04-26T21:11:39.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9784 sec 2024-04-26T21:11:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:40 smithi119 ceph-mon[32752]: pgmap v5740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9382.118512 secs 2024-04-26T21:11:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:40 smithi167 ceph-mon[32751]: pgmap v5740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9382.118512 secs 2024-04-26T21:11:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:40 smithi090 ceph-mon[29180]: pgmap v5740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9382.118512 secs 2024-04-26T21:11:40.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9785 sec 2024-04-26T21:11:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9382.839756 secs 2024-04-26T21:11:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9382.839756 secs 2024-04-26T21:11:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9382.839756 secs 2024-04-26T21:11:41.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9786 sec 2024-04-26T21:11:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:42 smithi119 ceph-mon[32752]: pgmap v5741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:42 smithi167 ceph-mon[32751]: pgmap v5741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:42 smithi090 ceph-mon[29180]: pgmap v5741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:42.765 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9787 sec 2024-04-26T21:11:43.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9788 sec 2024-04-26T21:11:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:44 smithi119 ceph-mon[32752]: pgmap v5742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:44 smithi167 ceph-mon[32751]: pgmap v5742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:44 smithi090 ceph-mon[29180]: pgmap v5742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:44.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9789 sec 2024-04-26T21:11:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9387.118597 secs 2024-04-26T21:11:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9387.118597 secs 2024-04-26T21:11:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9387.118597 secs 2024-04-26T21:11:45.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9790 sec 2024-04-26T21:11:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:46 smithi119 ceph-mon[32752]: pgmap v5743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9387.839827 secs 2024-04-26T21:11:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:46 smithi167 ceph-mon[32751]: pgmap v5743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9387.839827 secs 2024-04-26T21:11:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:46 smithi090 ceph-mon[29180]: pgmap v5743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9387.839827 secs 2024-04-26T21:11:46.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9791 sec 2024-04-26T21:11:47.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9792 sec 2024-04-26T21:11:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:48 smithi119 ceph-mon[32752]: pgmap v5744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:48 smithi167 ceph-mon[32751]: pgmap v5744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:48 smithi090 ceph-mon[29180]: pgmap v5744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:48.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9793 sec 2024-04-26T21:11:49.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9794 sec 2024-04-26T21:11:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:50 smithi119 ceph-mon[32752]: pgmap v5745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9392.118682 secs 2024-04-26T21:11:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:50 smithi167 ceph-mon[32751]: pgmap v5745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9392.118682 secs 2024-04-26T21:11:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:50 smithi090 ceph-mon[29180]: pgmap v5745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9392.118682 secs 2024-04-26T21:11:50.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9795 sec 2024-04-26T21:11:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9392.839906 secs 2024-04-26T21:11:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9392.839906 secs 2024-04-26T21:11:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9392.839906 secs 2024-04-26T21:11:51.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9796 sec 2024-04-26T21:11:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:52 smithi119 ceph-mon[32752]: pgmap v5746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:52 smithi167 ceph-mon[32751]: pgmap v5746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:52 smithi090 ceph-mon[29180]: pgmap v5746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:52.766 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9797 sec 2024-04-26T21:11:53.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9798 sec 2024-04-26T21:11:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:54 smithi119 ceph-mon[32752]: pgmap v5747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:54 smithi167 ceph-mon[32751]: pgmap v5747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:54 smithi090 ceph-mon[29180]: pgmap v5747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:54.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9799 sec 2024-04-26T21:11:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9397.118767 secs 2024-04-26T21:11:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9397.118767 secs 2024-04-26T21:11:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9397.118767 secs 2024-04-26T21:11:55.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9800 sec 2024-04-26T21:11:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:56 smithi119 ceph-mon[32752]: pgmap v5748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9397.839997 secs 2024-04-26T21:11:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:56 smithi167 ceph-mon[32751]: pgmap v5748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9397.839997 secs 2024-04-26T21:11:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:56 smithi090 ceph-mon[29180]: pgmap v5748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:11:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9397.839997 secs 2024-04-26T21:11:56.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9801 sec 2024-04-26T21:11:57.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9802 sec 2024-04-26T21:11:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:11:58 smithi119 ceph-mon[32752]: pgmap v5749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:11:58 smithi167 ceph-mon[32751]: pgmap v5749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:11:58 smithi090 ceph-mon[29180]: pgmap v5749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:11:58.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9803 sec 2024-04-26T21:11:59.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9804 sec 2024-04-26T21:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:00 smithi119 ceph-mon[32752]: pgmap v5750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9402.118856 secs 2024-04-26T21:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:00 smithi167 ceph-mon[32751]: pgmap v5750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9402.118856 secs 2024-04-26T21:12:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:00 smithi090 ceph-mon[29180]: pgmap v5750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9402.118856 secs 2024-04-26T21:12:00.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9805 sec 2024-04-26T21:12:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9402.840113 secs 2024-04-26T21:12:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9402.840113 secs 2024-04-26T21:12:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9402.840113 secs 2024-04-26T21:12:01.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9806 sec 2024-04-26T21:12:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:02 smithi119 ceph-mon[32752]: pgmap v5751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:02 smithi167 ceph-mon[32751]: pgmap v5751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:02 smithi090 ceph-mon[29180]: pgmap v5751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:12:02.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9807 sec 2024-04-26T21:12:03.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9808 sec 2024-04-26T21:12:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:04 smithi119 ceph-mon[32752]: pgmap v5752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:04 smithi167 ceph-mon[32751]: pgmap v5752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:04 smithi090 ceph-mon[29180]: pgmap v5752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:04.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9809 sec 2024-04-26T21:12:05.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9407.118945 secs 2024-04-26T21:12:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9407.118945 secs 2024-04-26T21:12:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9407.118945 secs 2024-04-26T21:12:05.767 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9810 sec 2024-04-26T21:12:06.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9811 sec 2024-04-26T21:12:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: pgmap v5753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 9407.840228 secs 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: slow request 722.476995 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: pgmap v5754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 B/s rd, 1 op/s 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: pgmap v5755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: pgmap v5753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 9407.840228 secs 2024-04-26T21:12:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: slow request 722.476995 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:12:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: pgmap v5754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 B/s rd, 1 op/s 2024-04-26T21:12:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: pgmap v5755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: pgmap v5753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 9407.840228 secs 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: slow request 722.476995 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: pgmap v5754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 973 B/s rd, 1 op/s 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: pgmap v5755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:12:07.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9812 sec 2024-04-26T21:12:08.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9813 sec 2024-04-26T21:12:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:08 smithi119 ceph-mon[32752]: pgmap v5756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 B/s rd, 1 op/s 2024-04-26T21:12:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:08 smithi167 ceph-mon[32751]: pgmap v5756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 B/s rd, 1 op/s 2024-04-26T21:12:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:08 smithi090 ceph-mon[29180]: pgmap v5756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 601 B/s rd, 1 op/s 2024-04-26T21:12:09.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9814 sec 2024-04-26T21:12:10.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9815 sec 2024-04-26T21:12:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:10 smithi119 ceph-mon[32752]: pgmap v5757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9412.119004 secs 2024-04-26T21:12:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:10 smithi167 ceph-mon[32751]: pgmap v5757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9412.119004 secs 2024-04-26T21:12:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:10 smithi090 ceph-mon[29180]: pgmap v5757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9412.119004 secs 2024-04-26T21:12:11.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9816 sec 2024-04-26T21:12:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9412.840343 secs 2024-04-26T21:12:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9412.840343 secs 2024-04-26T21:12:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9412.840343 secs 2024-04-26T21:12:12.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9817 sec 2024-04-26T21:12:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:12 smithi119 ceph-mon[32752]: pgmap v5758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:12 smithi167 ceph-mon[32751]: pgmap v5758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:12 smithi090 ceph-mon[29180]: pgmap v5758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:13.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9818 sec 2024-04-26T21:12:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:12:14.725+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:12:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:12:14.725+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 132 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:12:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:12:14.726+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 76 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:12:14.727 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:12:14.726+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 76 ==== auth_reply(proto 2 0 (0) Success) ==== 233+0+0 (secure 0 0 0) 0x7f2ca8006510 con 0x7f2cd4104500 2024-04-26T21:12:14.768 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9819 sec 2024-04-26T21:12:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:14 smithi119 ceph-mon[32752]: pgmap v5759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:14 smithi167 ceph-mon[32751]: pgmap v5759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:14 smithi090 ceph-mon[29180]: pgmap v5759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:15.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9820 sec 2024-04-26T21:12:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9417.119064 secs 2024-04-26T21:12:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:15 smithi119 ceph-mon[32752]: pgmap v5760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:12:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9417.119064 secs 2024-04-26T21:12:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:15 smithi167 ceph-mon[32751]: pgmap v5760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:12:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9417.119064 secs 2024-04-26T21:12:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:15 smithi090 ceph-mon[29180]: pgmap v5760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:12:16.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9821 sec 2024-04-26T21:12:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9417.840429 secs 2024-04-26T21:12:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9417.840429 secs 2024-04-26T21:12:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9417.840429 secs 2024-04-26T21:12:17.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9822 sec 2024-04-26T21:12:18.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:17 smithi090 ceph-mon[29180]: pgmap v5761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:17 smithi119 ceph-mon[32752]: pgmap v5761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:17 smithi167 ceph-mon[32751]: pgmap v5761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:18.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9823 sec 2024-04-26T21:12:19.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9824 sec 2024-04-26T21:12:20.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9825 sec 2024-04-26T21:12:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:20 smithi119 ceph-mon[32752]: pgmap v5762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9422.119150 secs 2024-04-26T21:12:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:20 smithi167 ceph-mon[32751]: pgmap v5762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9422.119150 secs 2024-04-26T21:12:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:20 smithi090 ceph-mon[29180]: pgmap v5762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9422.119150 secs 2024-04-26T21:12:21.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9826 sec 2024-04-26T21:12:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9422.840494 secs 2024-04-26T21:12:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9422.840494 secs 2024-04-26T21:12:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9422.840494 secs 2024-04-26T21:12:22.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9827 sec 2024-04-26T21:12:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:22 smithi119 ceph-mon[32752]: pgmap v5763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:22 smithi167 ceph-mon[32751]: pgmap v5763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:22 smithi090 ceph-mon[29180]: pgmap v5763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:23.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9828 sec 2024-04-26T21:12:24.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9829 sec 2024-04-26T21:12:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:24 smithi119 ceph-mon[32752]: pgmap v5764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:24 smithi167 ceph-mon[32751]: pgmap v5764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:24 smithi090 ceph-mon[29180]: pgmap v5764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:25.769 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9830 sec 2024-04-26T21:12:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9427.119235 secs 2024-04-26T21:12:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:25 smithi119 ceph-mon[32752]: pgmap v5765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9427.119235 secs 2024-04-26T21:12:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:25 smithi167 ceph-mon[32751]: pgmap v5765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9427.119235 secs 2024-04-26T21:12:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:25 smithi090 ceph-mon[29180]: pgmap v5765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:26.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9831 sec 2024-04-26T21:12:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9427.840633 secs 2024-04-26T21:12:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9427.840633 secs 2024-04-26T21:12:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9427.840633 secs 2024-04-26T21:12:27.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9832 sec 2024-04-26T21:12:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:27 smithi119 ceph-mon[32752]: pgmap v5766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:27 smithi167 ceph-mon[32751]: pgmap v5766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:27 smithi090 ceph-mon[29180]: pgmap v5766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:28.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9833 sec 2024-04-26T21:12:29.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9834 sec 2024-04-26T21:12:30.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9835 sec 2024-04-26T21:12:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:30 smithi119 ceph-mon[32752]: pgmap v5767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9432.119315 secs 2024-04-26T21:12:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:30 smithi167 ceph-mon[32751]: pgmap v5767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9432.119315 secs 2024-04-26T21:12:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:30 smithi090 ceph-mon[29180]: pgmap v5767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9432.119315 secs 2024-04-26T21:12:31.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9836 sec 2024-04-26T21:12:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9432.840709 secs 2024-04-26T21:12:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9432.840709 secs 2024-04-26T21:12:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9432.840709 secs 2024-04-26T21:12:32.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9837 sec 2024-04-26T21:12:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:32 smithi119 ceph-mon[32752]: pgmap v5768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:32 smithi167 ceph-mon[32751]: pgmap v5768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:32 smithi090 ceph-mon[29180]: pgmap v5768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:33.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9838 sec 2024-04-26T21:12:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:33 smithi119 ceph-mon[32752]: pgmap v5769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:33 smithi167 ceph-mon[32751]: pgmap v5769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:33 smithi090 ceph-mon[29180]: pgmap v5769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:34.770 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9839 sec 2024-04-26T21:12:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9437.119398 secs 2024-04-26T21:12:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9437.119398 secs 2024-04-26T21:12:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9437.119398 secs 2024-04-26T21:12:35.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9840 sec 2024-04-26T21:12:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9437.840842 secs 2024-04-26T21:12:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:36 smithi119 ceph-mon[32752]: pgmap v5770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9437.840842 secs 2024-04-26T21:12:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:36 smithi167 ceph-mon[32751]: pgmap v5770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9437.840842 secs 2024-04-26T21:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:36 smithi090 ceph-mon[29180]: pgmap v5770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:36.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9841 sec 2024-04-26T21:12:37.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9842 sec 2024-04-26T21:12:38.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9843 sec 2024-04-26T21:12:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:38 smithi119 ceph-mon[32752]: pgmap v5771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:38 smithi167 ceph-mon[32751]: pgmap v5771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:38 smithi090 ceph-mon[29180]: pgmap v5771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:39.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9844 sec 2024-04-26T21:12:40.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9845 sec 2024-04-26T21:12:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:40 smithi119 ceph-mon[32752]: pgmap v5772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9442.119460 secs 2024-04-26T21:12:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:40 smithi167 ceph-mon[32751]: pgmap v5772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9442.119460 secs 2024-04-26T21:12:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:40 smithi090 ceph-mon[29180]: pgmap v5772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9442.119460 secs 2024-04-26T21:12:41.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9846 sec 2024-04-26T21:12:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9442.840961 secs 2024-04-26T21:12:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9442.840961 secs 2024-04-26T21:12:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9442.840961 secs 2024-04-26T21:12:42.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9847 sec 2024-04-26T21:12:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:42 smithi119 ceph-mon[32752]: pgmap v5773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:42 smithi167 ceph-mon[32751]: pgmap v5773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:42 smithi090 ceph-mon[29180]: pgmap v5773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:43.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9848 sec 2024-04-26T21:12:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:43 smithi119 ceph-mon[32752]: pgmap v5774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:43 smithi167 ceph-mon[32751]: pgmap v5774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:43 smithi090 ceph-mon[29180]: pgmap v5774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:44.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9849 sec 2024-04-26T21:12:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9447.119543 secs 2024-04-26T21:12:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9447.119543 secs 2024-04-26T21:12:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9447.119543 secs 2024-04-26T21:12:45.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9850 sec 2024-04-26T21:12:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9447.841137 secs 2024-04-26T21:12:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:46 smithi119 ceph-mon[32752]: pgmap v5775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9447.841137 secs 2024-04-26T21:12:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:46 smithi167 ceph-mon[32751]: pgmap v5775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9447.841137 secs 2024-04-26T21:12:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:46 smithi090 ceph-mon[29180]: pgmap v5775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:46.771 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9851 sec 2024-04-26T21:12:47.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9852 sec 2024-04-26T21:12:48.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9853 sec 2024-04-26T21:12:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:48 smithi119 ceph-mon[32752]: pgmap v5776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:48 smithi167 ceph-mon[32751]: pgmap v5776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:49.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:48 smithi090 ceph-mon[29180]: pgmap v5776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:49.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9854 sec 2024-04-26T21:12:50.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9855 sec 2024-04-26T21:12:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:50 smithi119 ceph-mon[32752]: pgmap v5777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9452.119629 secs 2024-04-26T21:12:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:50 smithi167 ceph-mon[32751]: pgmap v5777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9452.119629 secs 2024-04-26T21:12:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:50 smithi090 ceph-mon[29180]: pgmap v5777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9452.119629 secs 2024-04-26T21:12:51.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9856 sec 2024-04-26T21:12:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9452.841196 secs 2024-04-26T21:12:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9452.841196 secs 2024-04-26T21:12:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9452.841196 secs 2024-04-26T21:12:52.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9857 sec 2024-04-26T21:12:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:52 smithi119 ceph-mon[32752]: pgmap v5778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:52 smithi167 ceph-mon[32751]: pgmap v5778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:52 smithi090 ceph-mon[29180]: pgmap v5778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:53.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9858 sec 2024-04-26T21:12:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:53 smithi119 ceph-mon[32752]: pgmap v5779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:53 smithi167 ceph-mon[32751]: pgmap v5779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:53 smithi090 ceph-mon[29180]: pgmap v5779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:12:54.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9859 sec 2024-04-26T21:12:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9457.119714 secs 2024-04-26T21:12:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9457.119714 secs 2024-04-26T21:12:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9457.119714 secs 2024-04-26T21:12:55.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9860 sec 2024-04-26T21:12:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9457.841288 secs 2024-04-26T21:12:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:56 smithi119 ceph-mon[32752]: pgmap v5780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9457.841288 secs 2024-04-26T21:12:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:56 smithi167 ceph-mon[32751]: pgmap v5780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9457.841288 secs 2024-04-26T21:12:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:56 smithi090 ceph-mon[29180]: pgmap v5780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:56.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9861 sec 2024-04-26T21:12:57.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9862 sec 2024-04-26T21:12:58.772 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9863 sec 2024-04-26T21:12:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:58 smithi119 ceph-mon[32752]: pgmap v5781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:58 smithi167 ceph-mon[32751]: pgmap v5781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:58 smithi090 ceph-mon[29180]: pgmap v5781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:12:59.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9864 sec 2024-04-26T21:13:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:12:59 smithi119 ceph-mon[32752]: pgmap v5782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:00.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:12:59 smithi167 ceph-mon[32751]: pgmap v5782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:12:59 smithi090 ceph-mon[29180]: pgmap v5782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:00.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9865 sec 2024-04-26T21:13:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9462.119800 secs 2024-04-26T21:13:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:00 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9462.841420 secs 2024-04-26T21:13:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9462.119800 secs 2024-04-26T21:13:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:00 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9462.841420 secs 2024-04-26T21:13:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9462.119800 secs 2024-04-26T21:13:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:00 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9462.841420 secs 2024-04-26T21:13:01.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9866 sec 2024-04-26T21:13:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:01 smithi119 ceph-mon[32752]: pgmap v5783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:01 smithi167 ceph-mon[32751]: pgmap v5783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:01 smithi090 ceph-mon[29180]: pgmap v5783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:02.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9867 sec 2024-04-26T21:13:03.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9868 sec 2024-04-26T21:13:04.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9869 sec 2024-04-26T21:13:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:04 smithi119 ceph-mon[32752]: pgmap v5784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:04 smithi167 ceph-mon[32751]: pgmap v5784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:04 smithi090 ceph-mon[29180]: pgmap v5784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:05.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9870 sec 2024-04-26T21:13:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9467.119888 secs 2024-04-26T21:13:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9467.119888 secs 2024-04-26T21:13:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9467.119888 secs 2024-04-26T21:13:06.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9871 sec 2024-04-26T21:13:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9467.841571 secs 2024-04-26T21:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: pgmap v5785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: pgmap v5786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-26T21:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: pgmap v5787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:13:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9467.841571 secs 2024-04-26T21:13:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: pgmap v5785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: pgmap v5786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-26T21:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: pgmap v5787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:13:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9467.841571 secs 2024-04-26T21:13:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: pgmap v5785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:13:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:13:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:13:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: pgmap v5786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-26T21:13:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: pgmap v5787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:13:07.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9872 sec 2024-04-26T21:13:08.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9873 sec 2024-04-26T21:13:09.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9874 sec 2024-04-26T21:13:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:09 smithi090 ceph-mon[29180]: pgmap v5788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:09 smithi119 ceph-mon[32752]: pgmap v5788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:09 smithi167 ceph-mon[32751]: pgmap v5788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:10.773 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9875 sec 2024-04-26T21:13:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9472.119971 secs 2024-04-26T21:13:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9472.119971 secs 2024-04-26T21:13:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9472.119971 secs 2024-04-26T21:13:11.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9876 sec 2024-04-26T21:13:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:11 smithi090 ceph-mon[29180]: pgmap v5789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9472.841666 secs 2024-04-26T21:13:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:11 smithi119 ceph-mon[32752]: pgmap v5789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9472.841666 secs 2024-04-26T21:13:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:11 smithi167 ceph-mon[32751]: pgmap v5789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9472.841666 secs 2024-04-26T21:13:12.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9877 sec 2024-04-26T21:13:13.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9878 sec 2024-04-26T21:13:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:13 smithi090 ceph-mon[29180]: pgmap v5790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-26T21:13:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:13 smithi119 ceph-mon[32752]: pgmap v5790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-26T21:13:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:13 smithi167 ceph-mon[32751]: pgmap v5790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-26T21:13:14.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9879 sec 2024-04-26T21:13:15.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9880 sec 2024-04-26T21:13:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:15 smithi090 ceph-mon[29180]: pgmap v5791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9477.120031 secs 2024-04-26T21:13:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:15 smithi119 ceph-mon[32752]: pgmap v5791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9477.120031 secs 2024-04-26T21:13:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:15 smithi167 ceph-mon[32751]: pgmap v5791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:13:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9477.120031 secs 2024-04-26T21:13:16.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9881 sec 2024-04-26T21:13:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9477.841785 secs 2024-04-26T21:13:16.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9477.841785 secs 2024-04-26T21:13:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9477.841785 secs 2024-04-26T21:13:17.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9882 sec 2024-04-26T21:13:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:17 smithi090 ceph-mon[29180]: pgmap v5792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:13:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:17 smithi119 ceph-mon[32752]: pgmap v5792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:13:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:17 smithi167 ceph-mon[32751]: pgmap v5792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:13:18.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9883 sec 2024-04-26T21:13:19.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9884 sec 2024-04-26T21:13:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:19 smithi090 ceph-mon[29180]: pgmap v5793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:19 smithi119 ceph-mon[32752]: pgmap v5793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:19 smithi167 ceph-mon[32751]: pgmap v5793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:20.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9885 sec 2024-04-26T21:13:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9482.120115 secs 2024-04-26T21:13:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9482.120115 secs 2024-04-26T21:13:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9482.120115 secs 2024-04-26T21:13:21.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9886 sec 2024-04-26T21:13:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:21 smithi090 ceph-mon[29180]: pgmap v5794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9482.841910 secs 2024-04-26T21:13:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:21 smithi119 ceph-mon[32752]: pgmap v5794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9482.841910 secs 2024-04-26T21:13:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:21 smithi167 ceph-mon[32751]: pgmap v5794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9482.841910 secs 2024-04-26T21:13:22.774 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9887 sec 2024-04-26T21:13:23.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9888 sec 2024-04-26T21:13:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:23 smithi119 ceph-mon[32752]: pgmap v5795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:23 smithi167 ceph-mon[32751]: pgmap v5795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:23 smithi090 ceph-mon[29180]: pgmap v5795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:24.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9889 sec 2024-04-26T21:13:25.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9890 sec 2024-04-26T21:13:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:25 smithi119 ceph-mon[32752]: pgmap v5796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9487.120199 secs 2024-04-26T21:13:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:25 smithi167 ceph-mon[32751]: pgmap v5796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9487.120199 secs 2024-04-26T21:13:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:25 smithi090 ceph-mon[29180]: pgmap v5796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9487.120199 secs 2024-04-26T21:13:26.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9891 sec 2024-04-26T21:13:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9487.842025 secs 2024-04-26T21:13:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9487.842025 secs 2024-04-26T21:13:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9487.842025 secs 2024-04-26T21:13:27.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9892 sec 2024-04-26T21:13:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:27 smithi119 ceph-mon[32752]: pgmap v5797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:27 smithi167 ceph-mon[32751]: pgmap v5797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:27 smithi090 ceph-mon[29180]: pgmap v5797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:28.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9893 sec 2024-04-26T21:13:29.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9894 sec 2024-04-26T21:13:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:29 smithi119 ceph-mon[32752]: pgmap v5798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:29 smithi167 ceph-mon[32751]: pgmap v5798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:29 smithi090 ceph-mon[29180]: pgmap v5798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:30.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9895 sec 2024-04-26T21:13:31.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9492.120287 secs 2024-04-26T21:13:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9492.120287 secs 2024-04-26T21:13:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9492.120287 secs 2024-04-26T21:13:31.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9896 sec 2024-04-26T21:13:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:31 smithi119 ceph-mon[32752]: pgmap v5799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9492.842141 secs 2024-04-26T21:13:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:31 smithi167 ceph-mon[32751]: pgmap v5799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9492.842141 secs 2024-04-26T21:13:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:31 smithi090 ceph-mon[29180]: pgmap v5799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9492.842141 secs 2024-04-26T21:13:32.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9897 sec 2024-04-26T21:13:33.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9898 sec 2024-04-26T21:13:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:33 smithi119 ceph-mon[32752]: pgmap v5800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:33 smithi167 ceph-mon[32751]: pgmap v5800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:33 smithi090 ceph-mon[29180]: pgmap v5800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:34.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9899 sec 2024-04-26T21:13:35.775 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9900 sec 2024-04-26T21:13:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:35 smithi119 ceph-mon[32752]: pgmap v5801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9497.120370 secs 2024-04-26T21:13:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:35 smithi167 ceph-mon[32751]: pgmap v5801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9497.120370 secs 2024-04-26T21:13:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:35 smithi090 ceph-mon[29180]: pgmap v5801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9497.120370 secs 2024-04-26T21:13:36.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9901 sec 2024-04-26T21:13:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9497.842255 secs 2024-04-26T21:13:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9497.842255 secs 2024-04-26T21:13:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9497.842255 secs 2024-04-26T21:13:37.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9902 sec 2024-04-26T21:13:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:37 smithi119 ceph-mon[32752]: pgmap v5802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:37 smithi167 ceph-mon[32751]: pgmap v5802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:37 smithi090 ceph-mon[29180]: pgmap v5802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:38.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9903 sec 2024-04-26T21:13:39.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9904 sec 2024-04-26T21:13:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:39 smithi090 ceph-mon[29180]: pgmap v5803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:39 smithi119 ceph-mon[32752]: pgmap v5803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:39 smithi167 ceph-mon[32751]: pgmap v5803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:40.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9905 sec 2024-04-26T21:13:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9502.120454 secs 2024-04-26T21:13:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9502.120454 secs 2024-04-26T21:13:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9502.120454 secs 2024-04-26T21:13:41.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9906 sec 2024-04-26T21:13:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:41 smithi119 ceph-mon[32752]: pgmap v5804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9502.842369 secs 2024-04-26T21:13:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:41 smithi167 ceph-mon[32751]: pgmap v5804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9502.842369 secs 2024-04-26T21:13:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:41 smithi090 ceph-mon[29180]: pgmap v5804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9502.842369 secs 2024-04-26T21:13:42.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9907 sec 2024-04-26T21:13:43.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9908 sec 2024-04-26T21:13:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:43 smithi119 ceph-mon[32752]: pgmap v5805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:43 smithi167 ceph-mon[32751]: pgmap v5805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:43 smithi090 ceph-mon[29180]: pgmap v5805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:44.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9909 sec 2024-04-26T21:13:45.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9910 sec 2024-04-26T21:13:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:45 smithi119 ceph-mon[32752]: pgmap v5806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9507.120534 secs 2024-04-26T21:13:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:45 smithi167 ceph-mon[32751]: pgmap v5806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9507.120534 secs 2024-04-26T21:13:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:45 smithi090 ceph-mon[29180]: pgmap v5806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9507.120534 secs 2024-04-26T21:13:46.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9911 sec 2024-04-26T21:13:46.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9507.842498 secs 2024-04-26T21:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9507.842498 secs 2024-04-26T21:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9507.842498 secs 2024-04-26T21:13:47.776 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9912 sec 2024-04-26T21:13:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:47 smithi119 ceph-mon[32752]: pgmap v5807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:47 smithi167 ceph-mon[32751]: pgmap v5807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:47 smithi090 ceph-mon[29180]: pgmap v5807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:48.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9913 sec 2024-04-26T21:13:49.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9914 sec 2024-04-26T21:13:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:49 smithi119 ceph-mon[32752]: pgmap v5808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:49 smithi167 ceph-mon[32751]: pgmap v5808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:49 smithi090 ceph-mon[29180]: pgmap v5808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:50.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9915 sec 2024-04-26T21:13:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9512.120617 secs 2024-04-26T21:13:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9512.120617 secs 2024-04-26T21:13:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9512.120617 secs 2024-04-26T21:13:51.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9916 sec 2024-04-26T21:13:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:51 smithi119 ceph-mon[32752]: pgmap v5809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9512.842631 secs 2024-04-26T21:13:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:51 smithi167 ceph-mon[32751]: pgmap v5809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9512.842631 secs 2024-04-26T21:13:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:51 smithi090 ceph-mon[29180]: pgmap v5809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9512.842631 secs 2024-04-26T21:13:52.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9917 sec 2024-04-26T21:13:53.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9918 sec 2024-04-26T21:13:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:53 smithi119 ceph-mon[32752]: pgmap v5810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:53 smithi167 ceph-mon[32751]: pgmap v5810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:53 smithi090 ceph-mon[29180]: pgmap v5810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:54.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9919 sec 2024-04-26T21:13:55.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9920 sec 2024-04-26T21:13:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:55 smithi119 ceph-mon[32752]: pgmap v5811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 9517.120701 secs 2024-04-26T21:13:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:55 smithi167 ceph-mon[32751]: pgmap v5811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 9517.120701 secs 2024-04-26T21:13:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:55 smithi090 ceph-mon[29180]: pgmap v5811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:13:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 9517.120701 secs 2024-04-26T21:13:56.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9921 sec 2024-04-26T21:13:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9517.842769 secs 2024-04-26T21:13:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9517.842769 secs 2024-04-26T21:13:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9517.842769 secs 2024-04-26T21:13:57.777 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9922 sec 2024-04-26T21:13:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:57 smithi119 ceph-mon[32752]: pgmap v5812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:57 smithi167 ceph-mon[32751]: pgmap v5812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:57 smithi090 ceph-mon[29180]: pgmap v5812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:13:58.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9923 sec 2024-04-26T21:13:59.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9924 sec 2024-04-26T21:14:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:13:59 smithi119 ceph-mon[32752]: pgmap v5813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:13:59 smithi167 ceph-mon[32751]: pgmap v5813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:13:59 smithi090 ceph-mon[29180]: pgmap v5813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:00.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9925 sec 2024-04-26T21:14:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:00 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 9522.120797 secs 2024-04-26T21:14:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:00 smithi119 ceph-mon[32752]: slow request 180.118794 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:14:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:00 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 9522.120797 secs 2024-04-26T21:14:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:00 smithi167 ceph-mon[32751]: slow request 180.118794 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:14:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:00 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 9522.120797 secs 2024-04-26T21:14:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:00 smithi090 ceph-mon[29180]: slow request 180.118794 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:14:01.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9926 sec 2024-04-26T21:14:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:01 smithi119 ceph-mon[32752]: pgmap v5814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9522.842867 secs 2024-04-26T21:14:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:01 smithi167 ceph-mon[32751]: pgmap v5814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9522.842867 secs 2024-04-26T21:14:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:01 smithi090 ceph-mon[29180]: pgmap v5814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9522.842867 secs 2024-04-26T21:14:02.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9927 sec 2024-04-26T21:14:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:02 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:14:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:02 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:14:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:02 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:14:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:02 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:14:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:02 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:14:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:02 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:14:03.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9928 sec 2024-04-26T21:14:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:03 smithi119 ceph-mon[32752]: pgmap v5815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:03 smithi167 ceph-mon[32751]: pgmap v5815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:03 smithi090 ceph-mon[29180]: pgmap v5815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:04.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9929 sec 2024-04-26T21:14:05.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9930 sec 2024-04-26T21:14:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:05 smithi119 ceph-mon[32752]: pgmap v5816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:05 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9527.120887 secs 2024-04-26T21:14:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:05 smithi167 ceph-mon[32751]: pgmap v5816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:05 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9527.120887 secs 2024-04-26T21:14:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:05 smithi090 ceph-mon[29180]: pgmap v5816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:05 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9527.120887 secs 2024-04-26T21:14:06.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9931 sec 2024-04-26T21:14:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9527.842988 secs 2024-04-26T21:14:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9527.842988 secs 2024-04-26T21:14:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9527.842988 secs 2024-04-26T21:14:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:14:07.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9932 sec 2024-04-26T21:14:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:07 smithi119 ceph-mon[32752]: pgmap v5817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:14:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:07 smithi167 ceph-mon[32751]: pgmap v5817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:14:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:07 smithi090 ceph-mon[29180]: pgmap v5817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:14:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:14:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:14:08.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9933 sec 2024-04-26T21:14:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:08 smithi119 ceph-mon[32752]: pgmap v5818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:14:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:08 smithi119 ceph-mon[32752]: pgmap v5819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:14:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:08 smithi167 ceph-mon[32751]: pgmap v5818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:14:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:08 smithi167 ceph-mon[32751]: pgmap v5819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:14:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:08 smithi090 ceph-mon[29180]: pgmap v5818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:14:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:08 smithi090 ceph-mon[29180]: pgmap v5819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:14:09.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9934 sec 2024-04-26T21:14:10.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:09 smithi090 ceph-mon[29180]: pgmap v5820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:09 smithi119 ceph-mon[32752]: pgmap v5820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:09 smithi167 ceph-mon[32751]: pgmap v5820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:10.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9935 sec 2024-04-26T21:14:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:10 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9532.120949 secs 2024-04-26T21:14:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:10 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9532.843136 secs 2024-04-26T21:14:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:10 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9532.120949 secs 2024-04-26T21:14:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:10 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9532.843136 secs 2024-04-26T21:14:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:10 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9532.120949 secs 2024-04-26T21:14:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:10 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9532.843136 secs 2024-04-26T21:14:11.778 INFO:tasks.workunit.client.0.smithi090.stdout: 10 cleanup 9936 sec 2024-04-26T21:14:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:11 smithi119 ceph-mon[32752]: pgmap v5821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:11 smithi167 ceph-mon[32751]: pgmap v5821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:11 smithi090 ceph-mon[29180]: pgmap v5821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:12.552 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-26T21:14:12.553 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-26T21:14:12.553 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T21:14:12.835 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi090 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-26T21:14:13.081 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b33a051ba9a84675a0d4de0d86ca7a7d Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi090 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-04-26T21:14:13.084 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-26T21:14:13.095 INFO:tasks.quiescer:joining Quiescers 2024-04-26T21:14:13.095 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-26T21:14:13.095 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T21:14:13.335 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:14:14.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:14 smithi090 ceph-mon[29180]: pgmap v5822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:14 smithi119 ceph-mon[32752]: pgmap v5822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:14 smithi167 ceph-mon[32751]: pgmap v5822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:14.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.963+0000 7f5e0383c700 1 -- 172.21.15.90:0/318407142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 msgr2=0x7f5dfc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.963+0000 7f5e0383c700 1 --2- 172.21.15.90:0/318407142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc1038f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009990 tx=0x7f5dec009ca0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.963+0000 7f5e0383c700 1 -- 172.21.15.90:0/318407142 shutdown_connections 2024-04-26T21:14:14.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.963+0000 7f5e0383c700 1 --2- 172.21.15.90:0/318407142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc103e30 0x7f5dfc110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.963+0000 7f5e0383c700 1 --2- 172.21.15.90:0/318407142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dfc1094c0 0x7f5dfc1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.963+0000 7f5e0383c700 1 --2- 172.21.15.90:0/318407142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc1038f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.963+0000 7f5e0383c700 1 -- 172.21.15.90:0/318407142 >> 172.21.15.90:0/318407142 conn(0x7f5dfc07a580 msgr2=0x7f5dfc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.964+0000 7f5e0383c700 1 -- 172.21.15.90:0/318407142 shutdown_connections 2024-04-26T21:14:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.964+0000 7f5e0383c700 1 -- 172.21.15.90:0/318407142 wait complete. 2024-04-26T21:14:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.964+0000 7f5e0383c700 1 Processor -- start 2024-04-26T21:14:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.964+0000 7f5e0383c700 1 -- start start 2024-04-26T21:14:14.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e0383c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e0383c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dfc103e30 0x7f5dfc19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e0383c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc1094c0 0x7f5dfc199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e0383c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc1135c0 con 0x7f5dfc103470 2024-04-26T21:14:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e0383c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc113440 con 0x7f5dfc1094c0 2024-04-26T21:14:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e0383c700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5dfc113740 con 0x7f5dfc103e30 2024-04-26T21:14:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc19f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54788/0 (socket says 172.21.15.90:54788) 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 -- 172.21.15.90:0/782767192 learned_addr learned my addr 172.21.15.90:0/782767192 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e01dd9700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc1094c0 0x7f5dfc199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e00dd7700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dfc103e30 0x7f5dfc19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 -- 172.21.15.90:0/782767192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dfc103e30 msgr2=0x7f5dfc19fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dfc103e30 0x7f5dfc19fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 -- 172.21.15.90:0/782767192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc1094c0 msgr2=0x7f5dfc199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc1094c0 0x7f5dfc199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 -- 172.21.15.90:0/782767192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dfc19a0c0 con 0x7f5dfc103470 2024-04-26T21:14:14.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e00dd7700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dfc103e30 0x7f5dfc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e01dd9700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc1094c0 0x7f5dfc199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e015d8700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc19f7a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5df800c9b0 tx=0x7f5df800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5df27fc700 1 -- 172.21.15.90:0/782767192 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8007b00 con 0x7f5dfc103470 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dfc19a3b0 con 0x7f5dfc103470 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.965+0000 7f5df27fc700 1 -- 172.21.15.90:0/782767192 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5df8007ca0 con 0x7f5dfc103470 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.966+0000 7f5df27fc700 1 -- 172.21.15.90:0/782767192 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8018660 con 0x7f5dfc103470 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.966+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dfc1a6670 con 0x7f5dfc103470 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.966+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dfc06a110 con 0x7f5dfc103470 2024-04-26T21:14:14.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.968+0000 7f5df27fc700 1 -- 172.21.15.90:0/782767192 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f5df800f450 con 0x7f5dfc103470 2024-04-26T21:14:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.968+0000 7f5df27fc700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5de80a3db0 0x7f5de80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.968+0000 7f5e00dd7700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5de80a3db0 0x7f5de80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.968+0000 7f5df27fc700 1 -- 172.21.15.90:0/782767192 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f5df80d1cb0 con 0x7f5dfc103470 2024-04-26T21:14:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.968+0000 7f5e00dd7700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5de80a3db0 0x7f5de80a6270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00b2e0 tx=0x7f5dec005e80 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:14.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:14.972+0000 7f5df27fc700 1 -- 172.21.15.90:0/782767192 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5df8014030 con 0x7f5dfc103470 2024-04-26T21:14:15.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.213+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dfc19ae90 con 0x7f5dfc103470 2024-04-26T21:14:15.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.214+0000 7f5df27fc700 1 -- 172.21.15.90:0/782767192 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 370 v370) ==== 95+0+5300 (secure 0 0 0) 0x7f5df809d180 con 0x7f5dfc103470 2024-04-26T21:14:15.215 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T21:14:15.216 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":370,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":370,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:08:49.029287+0000","modified":"2024-04-26T21:14:01.684512+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":183,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24449,"mds_1":16965,"mds_2":17019},"failed":[],"damaged":[],"stopped":[],"info":{"gid_16965":{"gid":16965,"name":"g","rank":1,"incarnation":326,"state":"up:active","state_seq":2241,"addr":"172.21.15.90:6835/439129317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6834","nonce":439129317},{"type":"v1","addr":"172.21.15.90:6835","nonce":439129317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_17019":{"gid":17019,"name":"j","rank":2,"incarnation":333,"state":"up:active","state_seq":7,"addr":"172.21.15.90:6837/507519675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6836","nonce":507519675},{"type":"v1","addr":"172.21.15.90:6837","nonce":507519675}]},"join_fscid":-1,"laggy_since":"2024-04-26T18:35:37.704099+0000","export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6835/570376817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":570376817},{"type":"v1","addr":"172.21.15.167:6835","nonce":570376817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":2783,"addr":"172.21.15.119:6839/1002360318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6838","nonce":1002360318},{"type":"v1","addr":"172.21.15.119:6839","nonce":1002360318}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24449,"qdb_cluster":[24449,16965,17019]},"id":1}]} 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5de80a3db0 msgr2=0x7f5de80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5de80a3db0 0x7f5de80a6270 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5dec00b2e0 tx=0x7f5dec005e80 comp rx=0 tx=0).stop 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 msgr2=0x7f5dfc19f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc19f7a0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5df800c9b0 tx=0x7f5df800cd70 comp rx=0 tx=0).stop 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 shutdown_connections 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f5de80a3db0 0x7f5de80a6270 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5dfc103470 0x7f5dfc19f7a0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc1094c0 0x7f5dfc199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 --2- 172.21.15.90:0/782767192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5dfc103e30 0x7f5dfc19fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 >> 172.21.15.90:0/782767192 conn(0x7f5dfc07a580 msgr2=0x7f5dfc102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 shutdown_connections 2024-04-26T21:14:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:15.216+0000 7f5e0383c700 1 -- 172.21.15.90:0/782767192 wait complete. 2024-04-26T21:14:15.219 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 370 2024-04-26T21:14:15.542 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:15 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9537.121030 secs 2024-04-26T21:14:15.542 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:15 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/782767192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:14:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:15 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9537.121030 secs 2024-04-26T21:14:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:15 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/782767192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:14:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:15 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9537.121030 secs 2024-04-26T21:14:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:15 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/782767192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T21:14:15.864 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24449 quiesce db --cancel --all' 2024-04-26T21:14:15.865 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24449 quiesce db --cancel --all 2024-04-26T21:14:16.101 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:14:16.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:16 smithi090 ceph-mon[29180]: pgmap v5823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:16.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9537.843241 secs 2024-04-26T21:14:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:16 smithi119 ceph-mon[32752]: pgmap v5823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:16.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9537.843241 secs 2024-04-26T21:14:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:16 smithi167 ceph-mon[32751]: pgmap v5823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:14:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9537.843241 secs 2024-04-26T21:14:17.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 -- 172.21.15.90:0/3463468882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1164102cd0 msgr2=0x7f116410f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 --2- 172.21.15.90:0/3463468882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1164102cd0 0x7f116410f1c0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f115c00b0a0 tx=0x7f115c00b3b0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 -- 172.21.15.90:0/3463468882 shutdown_connections 2024-04-26T21:14:17.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 --2- 172.21.15.90:0/3463468882 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1164102cd0 0x7f116410f1c0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 --2- 172.21.15.90:0/3463468882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1164108380 0x7f1164108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 --2- 172.21.15.90:0/3463468882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1164102310 0x7f1164102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 -- 172.21.15.90:0/3463468882 >> 172.21.15.90:0/3463468882 conn(0x7f1164079290 msgr2=0x7f11640796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:17.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 -- 172.21.15.90:0/3463468882 shutdown_connections 2024-04-26T21:14:17.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 -- 172.21.15.90:0/3463468882 wait complete. 2024-04-26T21:14:17.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 Processor -- start 2024-04-26T21:14:17.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.705+0000 7f116988b700 1 -- start start 2024-04-26T21:14:17.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f116988b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1164102310 0x7f1164103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f116988b700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1164102cd0 0x7f1164104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f116988b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1164108380 0x7f1164104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f116988b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f11641124d0 con 0x7f1164108380 2024-04-26T21:14:17.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f116988b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f1164112350 con 0x7f1164102310 2024-04-26T21:14:17.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f116988b700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1164112650 con 0x7f1164102cd0 2024-04-26T21:14:17.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1164108380 0x7f1164104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1164108380 0x7f1164104a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54810/0 (socket says 172.21.15.90:54810) 2024-04-26T21:14:17.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 -- 172.21.15.90:0/347895969 learned_addr learned my addr 172.21.15.90:0/347895969 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:14:17.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f1162ffd700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1164102310 0x7f1164103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f115a7fc700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1164102cd0 0x7f1164104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 -- 172.21.15.90:0/347895969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1164102cd0 msgr2=0x7f1164104530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.713 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1164102cd0 0x7f1164104530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 -- 172.21.15.90:0/347895969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1164102310 msgr2=0x7f1164103ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1164102310 0x7f1164103ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f11637fe700 1 -- 172.21.15.90:0/347895969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11641a7f80 con 0x7f1164108380 2024-04-26T21:14:17.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.706+0000 7f1162ffd700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1164102310 0x7f1164103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:17.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.707+0000 7f11637fe700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1164108380 0x7f1164104a70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f115c0062a0 tx=0x7f115c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:17.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.707+0000 7f1168889700 1 -- 172.21.15.90:0/347895969 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c021070 con 0x7f1164108380 2024-04-26T21:14:17.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.707+0000 7f116988b700 1 -- 172.21.15.90:0/347895969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11641a8210 con 0x7f1164108380 2024-04-26T21:14:17.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.707+0000 7f1168889700 1 -- 172.21.15.90:0/347895969 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115c007e30 con 0x7f1164108380 2024-04-26T21:14:17.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.707+0000 7f1168889700 1 -- 172.21.15.90:0/347895969 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c0046d0 con 0x7f1164108380 2024-04-26T21:14:17.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.707+0000 7f116988b700 1 -- 172.21.15.90:0/347895969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11641a86c0 con 0x7f1164108380 2024-04-26T21:14:17.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.709+0000 7f1168889700 1 -- 172.21.15.90:0/347895969 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f115c007980 con 0x7f1164108380 2024-04-26T21:14:17.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.709+0000 7f1168889700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11500a3db0 0x7f11500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.710+0000 7f1162ffd700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11500a3db0 0x7f11500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.710+0000 7f1168889700 1 -- 172.21.15.90:0/347895969 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f115c00f020 con 0x7f1164108380 2024-04-26T21:14:17.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.710+0000 7f1162ffd700 1 --2- 172.21.15.90:0/347895969 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f11500a3db0 0x7f11500a6270 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1154009fd0 tx=0x7f1154009260 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:17.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.710+0000 7f117411db80 1 Processor -- start 2024-04-26T21:14:17.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.710+0000 7f117411db80 1 -- start start 2024-04-26T21:14:17.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f117411db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c8e10 0x560baf1c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f117411db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c9730 0x560baf1c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f117411db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c37e0 0x560baf1c3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f117411db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560baf1158a0 con 0x560baf1c9730 2024-04-26T21:14:17.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f117411db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560baf07b9a0 con 0x560baf1c37e0 2024-04-26T21:14:17.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f117411db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560baf111860 con 0x560baf1c8e10 2024-04-26T21:14:17.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c9730 0x560baf1c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c9730 0x560baf1c31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54816/0 (socket says 172.21.15.90:54816) 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 -- 172.21.15.90:0/3795004602 learned_addr learned my addr 172.21.15.90:0/3795004602 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f11637fe700 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c37e0 0x560baf1c3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f1162ffd700 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c8e10 0x560baf1c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 -- 172.21.15.90:0/3795004602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c8e10 msgr2=0x560baf1c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c8e10 0x560baf1c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 -- 172.21.15.90:0/3795004602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c37e0 msgr2=0x560baf1c3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c37e0 0x560baf1c3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 -- 172.21.15.90:0/3795004602 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560baf062940 con 0x560baf1c9730 2024-04-26T21:14:17.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f11637fe700 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c37e0 0x560baf1c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f115a7fc700 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c9730 0x560baf1c31c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f114c0046c0 tx=0x7f114c00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fb25e468fd9d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f1159ffb700 1 -- 172.21.15.90:0/3795004602 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c004a30 con 0x560baf1c9730 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f1159ffb700 1 -- 172.21.15.90:0/3795004602 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c015070 con 0x560baf1c9730 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.711+0000 7f1159ffb700 1 -- 172.21.15.90:0/3795004602 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c003bb0 con 0x560baf1c9730 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 -- 172.21.15.90:0/3795004602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c9730 msgr2=0x560baf1c31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c9730 0x560baf1c31c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f114c0046c0 tx=0x7f114c00b9e0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 -- 172.21.15.90:0/3795004602 shutdown_connections 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c37e0 0x560baf1c3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c8e10 0x560baf1c91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 --2- 172.21.15.90:0/3795004602 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c9730 0x560baf1c31c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 -- 172.21.15.90:0/3795004602 >> 172.21.15.90:0/3795004602 conn(0x560baf0f1630 msgr2=0x560baf1bdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 -- 172.21.15.90:0/3795004602 shutdown_connections 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.712+0000 7f117411db80 1 -- 172.21.15.90:0/3795004602 wait complete. 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 -- start start 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 Processor -- start 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c37e0 0x560baf1cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c8e10 0x560baf1cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c9730 0x560baf1ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x560baf1d01a0 con 0x560baf1c8e10 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x560baf0cfaf0 con 0x560baf1c9730 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560baf1d0320 con 0x560baf1c37e0 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f117411db80 10 client.0 _ll_register_callbacks cb 0x560baf0ad9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f115a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c8e10 0x560baf1cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f115a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c8e10 0x560baf1cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54832/0 (socket says 172.21.15.90:54832) 2024-04-26T21:14:17.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f115a7fc700 1 -- 172.21.15.90:0/1814881825 learned_addr learned my addr 172.21.15.90:0/1814881825 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f11637fe700 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c9730 0x560baf1ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f1162ffd700 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c37e0 0x560baf1cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.713+0000 7f115a7fc700 1 -- 172.21.15.90:0/1814881825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c37e0 msgr2=0x560baf1cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f115a7fc700 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560baf1c37e0 0x560baf1cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f115a7fc700 1 -- 172.21.15.90:0/1814881825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c9730 msgr2=0x560baf1ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f115a7fc700 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560baf1c9730 0x560baf1ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f115a7fc700 1 -- 172.21.15.90:0/1814881825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560baf0d34f0 con 0x560baf1c8e10 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f115a7fc700 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x560baf1c8e10 0x560baf1cc720 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f114c00b690 tx=0x7f114c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46234a45d5365721 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:14:17.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f1142ffd700 10 client.0 ms_handle_connect on v2:172.21.15.90:3300/0 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f1142ffd700 1 -- 172.21.15.90:0/1814881825 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114c028060 con 0x560baf1c8e10 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f1142ffd700 1 -- 172.21.15.90:0/1814881825 <== mon.0 v2:172.21.15.90:3300/0 2 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f114c005410 con 0x560baf1c8e10 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 1 -- 172.21.15.90:0/1814881825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f114c015070 con 0x560baf1c8e10 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f1142ffd700 1 -- 172.21.15.90:0/1814881825 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_get_version_reply(handle=1 version=370) ==== 24+0+0 (secure 0 0 0) 0x7f114c030050 con 0x560baf1c8e10 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 10 client.17235 fetch_fsmap learned FSMap version 370 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 1 -- 172.21.15.90:0/1814881825 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,fsmap=370,monmap=4+}) -- 0x560baf1da980 con 0x560baf1c8e10 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f1142ffd700 1 -- 172.21.15.90:0/1814881825 <== mon.0 v2:172.21.15.90:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f114c015070 con 0x560baf1c8e10 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f1142ffd700 1 -- 172.21.15.90:0/1814881825 <== mon.0 v2:172.21.15.90:3300/0 5 ==== fsmap(e 370) ==== 3522+0+0 (secure 0 0 0) 0x7f114c011600 con 0x560baf1c8e10 2024-04-26T21:14:17.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 10 client.17235 fetch_fsmap finished waiting for FSMap version 370 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 10 client.17235 resolve_mds: validated gid 24449 aka daemon mds.b 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 20 client.17235 populate_metadata read hostname 'smithi090' 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560baf1dd250 0x560baf1df670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 4 client.17235 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.714+0000 7f117411db80 1 -- 172.21.15.90:0/1814881825 --> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560baf123280 con 0x560baf1dd250 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.715+0000 7f1162ffd700 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560baf1dd250 0x560baf1df670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.715+0000 7f1142ffd700 10 client.17235 ms_handle_connect on v2:172.21.15.119:6838/1002360318 2024-04-26T21:14:17.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:14:17.715+0000 7f1162ffd700 1 --2- 172.21.15.90:0/1814881825 >> [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] conn(0x560baf1dd250 0x560baf1df670 crc :-1 s=READY pgs=1321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e35e5acf01c1e832 server_cookie=273dd0bd79affad4 in_seq=0 out_seq=0 2024-04-26T21:14:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:18 smithi119 ceph-mon[32752]: pgmap v5824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:14:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:18 smithi167 ceph-mon[32751]: pgmap v5824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:14:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:18 smithi090 ceph-mon[29180]: pgmap v5824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:14:20.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:20 smithi090 ceph-mon[29180]: pgmap v5825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:20.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9542.121092 secs 2024-04-26T21:14:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:20 smithi119 ceph-mon[32752]: pgmap v5825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9542.121092 secs 2024-04-26T21:14:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:20 smithi167 ceph-mon[32751]: pgmap v5825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9542.121092 secs 2024-04-26T21:14:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9542.843367 secs 2024-04-26T21:14:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9542.843367 secs 2024-04-26T21:14:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9542.843367 secs 2024-04-26T21:14:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:22 smithi119 ceph-mon[32752]: pgmap v5826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:22 smithi167 ceph-mon[32751]: pgmap v5826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:22 smithi090 ceph-mon[29180]: pgmap v5826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:24 smithi119 ceph-mon[32752]: pgmap v5827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:24 smithi167 ceph-mon[32751]: pgmap v5827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:24 smithi090 ceph-mon[29180]: pgmap v5827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:25 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9547.121173 secs 2024-04-26T21:14:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:25 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9547.121173 secs 2024-04-26T21:14:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:25 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9547.121173 secs 2024-04-26T21:14:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:26 smithi119 ceph-mon[32752]: pgmap v5828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9547.843486 secs 2024-04-26T21:14:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:26 smithi167 ceph-mon[32751]: pgmap v5828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9547.843486 secs 2024-04-26T21:14:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:26 smithi090 ceph-mon[29180]: pgmap v5828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9547.843486 secs 2024-04-26T21:14:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:28 smithi119 ceph-mon[32752]: pgmap v5829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:28 smithi167 ceph-mon[32751]: pgmap v5829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:28 smithi090 ceph-mon[29180]: pgmap v5829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T21:14:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:14:29.743+0000 7f2cca7fc640 1 -- 172.21.15.90:0/3143758137 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ca80064e0 con 0x7f2cd4104500 2024-04-26T21:14:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:14:29.743+0000 7f2cb8ff9640 1 -- 172.21.15.90:0/3906338670 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:14:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:14:29.744+0000 7f2ccb7fe640 1 -- 172.21.15.90:0/3143758137 <== mon.0 v2:172.21.15.90:3300/0 77 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f2ca80064e0 con 0x7f2cd4104500 2024-04-26T21:14:29.745 INFO:teuthology.task.background_exec.ubuntu@smithi090.front.sepia.ceph.com.smithi090.stderr:2024-04-26T21:14:29.744+0000 7f2cb9ffb640 1 -- 172.21.15.90:0/3906338670 <== mon.0 v2:172.21.15.90:3300/0 77 ==== auth_reply(proto 2 0 (0) Success) ==== 409+0+0 (secure 0 0 0) 0x7f2cac006650 con 0x56302893f0e0 2024-04-26T21:14:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:30 smithi119 ceph-mon[32752]: pgmap v5830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9552.121256 secs 2024-04-26T21:14:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:30 smithi167 ceph-mon[32751]: pgmap v5830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9552.121256 secs 2024-04-26T21:14:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:30 smithi090 ceph-mon[29180]: pgmap v5830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T21:14:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9552.121256 secs 2024-04-26T21:14:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9552.843588 secs 2024-04-26T21:14:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9552.843588 secs 2024-04-26T21:14:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9552.843588 secs 2024-04-26T21:14:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:32 smithi119 ceph-mon[32752]: pgmap v5831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:32 smithi167 ceph-mon[32751]: pgmap v5831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:32 smithi090 ceph-mon[29180]: pgmap v5831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:34 smithi119 ceph-mon[32752]: pgmap v5832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:34 smithi167 ceph-mon[32751]: pgmap v5832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:34 smithi090 ceph-mon[29180]: pgmap v5832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:35 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9557.121337 secs 2024-04-26T21:14:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:35 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9557.121337 secs 2024-04-26T21:14:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:35 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9557.121337 secs 2024-04-26T21:14:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:36 smithi167 ceph-mon[32751]: pgmap v5833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9557.843737 secs 2024-04-26T21:14:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:36 smithi119 ceph-mon[32752]: pgmap v5833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9557.843737 secs 2024-04-26T21:14:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:36 smithi090 ceph-mon[29180]: pgmap v5833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9557.843737 secs 2024-04-26T21:14:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:38 smithi119 ceph-mon[32752]: pgmap v5834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:38 smithi167 ceph-mon[32751]: pgmap v5834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:38 smithi090 ceph-mon[29180]: pgmap v5834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:40 smithi119 ceph-mon[32752]: pgmap v5835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9562.121399 secs 2024-04-26T21:14:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:40 smithi167 ceph-mon[32751]: pgmap v5835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9562.121399 secs 2024-04-26T21:14:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:40 smithi090 ceph-mon[29180]: pgmap v5835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9562.121399 secs 2024-04-26T21:14:41.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9562.843862 secs 2024-04-26T21:14:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9562.843862 secs 2024-04-26T21:14:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9562.843862 secs 2024-04-26T21:14:42.795 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:42 smithi090 ceph-mon[29180]: pgmap v5836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:42 smithi119 ceph-mon[32752]: pgmap v5836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:42 smithi167 ceph-mon[32751]: pgmap v5836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:44.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:44 smithi119 ceph-mon[32752]: pgmap v5837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:44 smithi167 ceph-mon[32751]: pgmap v5837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:44 smithi090 ceph-mon[29180]: pgmap v5837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:45 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9567.121483 secs 2024-04-26T21:14:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:45 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9567.121483 secs 2024-04-26T21:14:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:45 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9567.121483 secs 2024-04-26T21:14:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:46 smithi119 ceph-mon[32752]: pgmap v5838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9567.843936 secs 2024-04-26T21:14:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:46 smithi167 ceph-mon[32751]: pgmap v5838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9567.843936 secs 2024-04-26T21:14:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:46 smithi090 ceph-mon[29180]: pgmap v5838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9567.843936 secs 2024-04-26T21:14:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:48 smithi119 ceph-mon[32752]: pgmap v5839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:48 smithi167 ceph-mon[32751]: pgmap v5839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:48 smithi090 ceph-mon[29180]: pgmap v5839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:50 smithi119 ceph-mon[32752]: pgmap v5840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:50 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9572.121566 secs 2024-04-26T21:14:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:50 smithi167 ceph-mon[32751]: pgmap v5840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:50 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9572.121566 secs 2024-04-26T21:14:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:50 smithi090 ceph-mon[29180]: pgmap v5840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:50 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9572.121566 secs 2024-04-26T21:14:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9572.844059 secs 2024-04-26T21:14:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9572.844059 secs 2024-04-26T21:14:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9572.844059 secs 2024-04-26T21:14:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:52 smithi119 ceph-mon[32752]: pgmap v5841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:52 smithi167 ceph-mon[32751]: pgmap v5841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:52 smithi090 ceph-mon[29180]: pgmap v5841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:54 smithi119 ceph-mon[32752]: pgmap v5842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:54 smithi167 ceph-mon[32751]: pgmap v5842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:54 smithi090 ceph-mon[29180]: pgmap v5842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:14:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9577.121651 secs 2024-04-26T21:14:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9577.121651 secs 2024-04-26T21:14:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9577.121651 secs 2024-04-26T21:14:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:56 smithi119 ceph-mon[32752]: pgmap v5843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9577.844194 secs 2024-04-26T21:14:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:56 smithi167 ceph-mon[32751]: pgmap v5843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9577.844194 secs 2024-04-26T21:14:56.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:56 smithi090 ceph-mon[29180]: pgmap v5843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:56.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9577.844194 secs 2024-04-26T21:14:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:14:58 smithi119 ceph-mon[32752]: pgmap v5844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:14:58 smithi167 ceph-mon[32751]: pgmap v5844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:14:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:14:58 smithi090 ceph-mon[29180]: pgmap v5844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:00 smithi119 ceph-mon[32752]: pgmap v5845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:00 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 9582.121711 secs 2024-04-26T21:15:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:00 smithi119 ceph-mon[32752]: slow request 5764.167342 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:15:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:00 smithi167 ceph-mon[32751]: pgmap v5845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:00 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 9582.121711 secs 2024-04-26T21:15:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:00 smithi167 ceph-mon[32751]: slow request 5764.167342 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:15:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:00 smithi090 ceph-mon[29180]: pgmap v5845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:00 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 9582.121711 secs 2024-04-26T21:15:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:00 smithi090 ceph-mon[29180]: slow request 5764.167342 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:15:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9582.844336 secs 2024-04-26T21:15:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9582.844336 secs 2024-04-26T21:15:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9582.844336 secs 2024-04-26T21:15:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:02 smithi119 ceph-mon[32752]: pgmap v5846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:02 smithi167 ceph-mon[32751]: pgmap v5846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:02 smithi090 ceph-mon[29180]: pgmap v5846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:04 smithi119 ceph-mon[32752]: pgmap v5847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:04 smithi167 ceph-mon[32751]: pgmap v5847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:04 smithi090 ceph-mon[29180]: pgmap v5847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:05 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9587.121818 secs 2024-04-26T21:15:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:05 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9587.121818 secs 2024-04-26T21:15:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:05 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9587.121818 secs 2024-04-26T21:15:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:06 smithi119 ceph-mon[32752]: pgmap v5848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9587.844468 secs 2024-04-26T21:15:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:06 smithi167 ceph-mon[32751]: pgmap v5848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9587.844468 secs 2024-04-26T21:15:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:06 smithi090 ceph-mon[29180]: pgmap v5848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9587.844468 secs 2024-04-26T21:15:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:15:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:08 smithi119 ceph-mon[32752]: pgmap v5849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:15:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:08 smithi167 ceph-mon[32751]: pgmap v5849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:15:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:08 smithi090 ceph-mon[29180]: pgmap v5849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:15:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:15:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:15:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:09 smithi119 ceph-mon[32752]: pgmap v5850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:15:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:09 smithi119 ceph-mon[32752]: pgmap v5851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:15:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:09 smithi167 ceph-mon[32751]: pgmap v5850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:15:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:09 smithi167 ceph-mon[32751]: pgmap v5851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:15:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:09 smithi090 ceph-mon[29180]: pgmap v5850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:15:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:09 smithi090 ceph-mon[29180]: pgmap v5851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:15:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:10 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9592.121906 secs 2024-04-26T21:15:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:10 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9592.121906 secs 2024-04-26T21:15:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:10 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9592.121906 secs 2024-04-26T21:15:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:11 smithi119 ceph-mon[32752]: pgmap v5852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9592.844562 secs 2024-04-26T21:15:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:11 smithi167 ceph-mon[32751]: pgmap v5852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9592.844562 secs 2024-04-26T21:15:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:11 smithi090 ceph-mon[29180]: pgmap v5852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9592.844562 secs 2024-04-26T21:15:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:13 smithi119 ceph-mon[32752]: pgmap v5853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:15:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:13 smithi167 ceph-mon[32751]: pgmap v5853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:15:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:13 smithi090 ceph-mon[29180]: pgmap v5853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:15:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:15 smithi167 ceph-mon[32751]: pgmap v5854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:15 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9597.121968 secs 2024-04-26T21:15:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:15 smithi090 ceph-mon[29180]: pgmap v5854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:15 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9597.121968 secs 2024-04-26T21:15:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:15 smithi119 ceph-mon[32752]: pgmap v5854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:15 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9597.121968 secs 2024-04-26T21:15:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9597.844675 secs 2024-04-26T21:15:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9597.844675 secs 2024-04-26T21:15:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9597.844675 secs 2024-04-26T21:15:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:17 smithi167 ceph-mon[32751]: pgmap v5855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:17 smithi090 ceph-mon[29180]: pgmap v5855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:17 smithi119 ceph-mon[32752]: pgmap v5855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:15:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:19 smithi167 ceph-mon[32751]: pgmap v5856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:15:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:19 smithi090 ceph-mon[29180]: pgmap v5856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:15:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:19 smithi119 ceph-mon[32752]: pgmap v5856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:15:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9602.122031 secs 2024-04-26T21:15:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9602.122031 secs 2024-04-26T21:15:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9602.122031 secs 2024-04-26T21:15:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:21 smithi090 ceph-mon[29180]: pgmap v5857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9602.844797 secs 2024-04-26T21:15:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:21 smithi119 ceph-mon[32752]: pgmap v5857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9602.844797 secs 2024-04-26T21:15:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:21 smithi167 ceph-mon[32751]: pgmap v5857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9602.844797 secs 2024-04-26T21:15:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:23 smithi090 ceph-mon[29180]: pgmap v5858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:23 smithi119 ceph-mon[32752]: pgmap v5858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:23 smithi167 ceph-mon[32751]: pgmap v5858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:25 smithi090 ceph-mon[29180]: pgmap v5859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:25 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9607.122113 secs 2024-04-26T21:15:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:25 smithi119 ceph-mon[32752]: pgmap v5859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:25 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9607.122113 secs 2024-04-26T21:15:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:25 smithi167 ceph-mon[32751]: pgmap v5859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:25 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9607.122113 secs 2024-04-26T21:15:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9607.844945 secs 2024-04-26T21:15:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9607.844945 secs 2024-04-26T21:15:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9607.844945 secs 2024-04-26T21:15:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:27 smithi090 ceph-mon[29180]: pgmap v5860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:27 smithi119 ceph-mon[32752]: pgmap v5860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:27 smithi167 ceph-mon[32751]: pgmap v5860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:29 smithi090 ceph-mon[29180]: pgmap v5861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:29 smithi119 ceph-mon[32752]: pgmap v5861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:29 smithi167 ceph-mon[32751]: pgmap v5861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9612.122195 secs 2024-04-26T21:15:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9612.122195 secs 2024-04-26T21:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9612.122195 secs 2024-04-26T21:15:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:31 smithi090 ceph-mon[29180]: pgmap v5862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9612.845010 secs 2024-04-26T21:15:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:31 smithi119 ceph-mon[32752]: pgmap v5862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9612.845010 secs 2024-04-26T21:15:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:31 smithi167 ceph-mon[32751]: pgmap v5862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9612.845010 secs 2024-04-26T21:15:33.271 DEBUG:teuthology.orchestra.run:got remote process result: 124 2024-04-26T21:15:33.272 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/quiescer.py", line 419, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/quiescer.py", line 378, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error quiescing set 'f82e6e3': 9 (EBADF) 2024-04-26T21:15:33.277 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T21:15:33.289 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T21:15:33.304 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T21:15:33.314 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T21:15:33.314 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T21:15:33.314 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T21:15:33.315 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T21:15:33.375 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T21:15:33.441 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T21:15:33.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 -- 172.21.15.90:0/178914459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 msgr2=0x7f7710106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:33.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 --2- 172.21.15.90:0/178914459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f7710106be0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7700009900 tx=0x7f770002a410 comp rx=0 tx=0).stop 2024-04-26T21:15:33.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 -- 172.21.15.90:0/178914459 shutdown_connections 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 --2- 172.21.15.90:0/178914459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7710102980 0x7f771010e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 --2- 172.21.15.90:0/178914459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7710101fc0 0x7f7710102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 --2- 172.21.15.90:0/178914459 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f7710106be0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 -- 172.21.15.90:0/178914459 >> 172.21.15.90:0/178914459 conn(0x7f7710077d50 msgr2=0x7f77100ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 -- 172.21.15.90:0/178914459 shutdown_connections 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 -- 172.21.15.90:0/178914459 wait complete. 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 Processor -- start 2024-04-26T21:15:33.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.582+0000 7f7716493640 1 -- start start 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7716493640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7710101fc0 0x7f7710199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7716493640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7710102980 0x7f771019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7716493640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f77101a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7716493640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7710111af0 con 0x7f7710106800 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7716493640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7710111970 con 0x7f7710101fc0 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7716493640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7710111c70 con 0x7f7710102980 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f77101a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:33.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f77101a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55626/0 (socket says 172.21.15.90:55626) 2024-04-26T21:15:33.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 -- 172.21.15.90:0/4041546713 learned_addr learned my addr 172.21.15.90:0/4041546713 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:15:33.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f770ffff640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7710101fc0 0x7f7710199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:33.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f770f7fe640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7710102980 0x7f771019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:33.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 -- 172.21.15.90:0/4041546713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7710102980 msgr2=0x7f771019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:33.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7710102980 0x7f771019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:33.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 -- 172.21.15.90:0/4041546713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7710101fc0 msgr2=0x7f7710199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:15:33.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7710101fc0 0x7f7710199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 -- 172.21.15.90:0/4041546713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77101a54f0 con 0x7f7710106800 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f770f7fe640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7710102980 0x7f771019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f7714a30640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f77101a4fb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f770800eb10 tx=0x7f770800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708004b20 con 0x7f7710106800 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.583+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7708004cc0 con 0x7f7710106800 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.584+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77101a56c0 con 0x7f7710106800 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.584+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708010670 con 0x7f7710106800 2024-04-26T21:15:33.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.584+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101a5b90 con 0x7f7710106800 2024-04-26T21:15:33.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.584+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77100506c0 con 0x7f7710106800 2024-04-26T21:15:33.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.586+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f7708005000 con 0x7f7710106800 2024-04-26T21:15:33.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.586+0000 7f770d7fa640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76f80a3dc0 0x7f76f80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:15:33.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.586+0000 7f770ffff640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76f80a3dc0 0x7f76f80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:15:33.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.586+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f77080d22b0 con 0x7f7710106800 2024-04-26T21:15:33.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.587+0000 7f770ffff640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76f80a3dc0 0x7f76f80a6280 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7700009230 tx=0x7f77000058d0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:15:33.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.590+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7708053ff0 con 0x7f7710106800 2024-04-26T21:15:33.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:15:33.792+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7710062d70 con 0x7f76f80a3dc0 2024-04-26T21:15:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:33 smithi090 ceph-mon[29180]: pgmap v5863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:33 smithi119 ceph-mon[32752]: pgmap v5863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:33 smithi167 ceph-mon[32751]: pgmap v5863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:35 smithi090 ceph-mon[29180]: from='client.17241 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:15:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:35 smithi090 ceph-mon[29180]: pgmap v5864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:35 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9617.122279 secs 2024-04-26T21:15:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:35 smithi119 ceph-mon[32752]: from='client.17241 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:15:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:35 smithi119 ceph-mon[32752]: pgmap v5864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:35 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9617.122279 secs 2024-04-26T21:15:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:35 smithi167 ceph-mon[32751]: from='client.17241 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:15:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:35 smithi167 ceph-mon[32751]: pgmap v5864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:35 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9617.122279 secs 2024-04-26T21:15:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9617.845118 secs 2024-04-26T21:15:36.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9617.845118 secs 2024-04-26T21:15:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9617.845118 secs 2024-04-26T21:15:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:37 smithi090 ceph-mon[29180]: pgmap v5865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:37 smithi119 ceph-mon[32752]: pgmap v5865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:37 smithi167 ceph-mon[32751]: pgmap v5865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:39 smithi090 ceph-mon[29180]: pgmap v5866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:39 smithi119 ceph-mon[32752]: pgmap v5866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:39 smithi167 ceph-mon[32751]: pgmap v5866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9622.122342 secs 2024-04-26T21:15:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9622.122342 secs 2024-04-26T21:15:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9622.122342 secs 2024-04-26T21:15:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:41 smithi090 ceph-mon[29180]: pgmap v5867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9622.845279 secs 2024-04-26T21:15:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:41 smithi119 ceph-mon[32752]: pgmap v5867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9622.845279 secs 2024-04-26T21:15:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:41 smithi167 ceph-mon[32751]: pgmap v5867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9622.845279 secs 2024-04-26T21:15:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:43 smithi090 ceph-mon[29180]: pgmap v5868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:43 smithi119 ceph-mon[32752]: pgmap v5868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:43 smithi167 ceph-mon[32751]: pgmap v5868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:45 smithi119 ceph-mon[32752]: pgmap v5869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:45 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9627.122427 secs 2024-04-26T21:15:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:45 smithi167 ceph-mon[32751]: pgmap v5869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:45 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9627.122427 secs 2024-04-26T21:15:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:45 smithi090 ceph-mon[29180]: pgmap v5869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:45 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9627.122427 secs 2024-04-26T21:15:46.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9627.845413 secs 2024-04-26T21:15:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9627.845413 secs 2024-04-26T21:15:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9627.845413 secs 2024-04-26T21:15:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:47 smithi119 ceph-mon[32752]: pgmap v5870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:47 smithi167 ceph-mon[32751]: pgmap v5870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:47 smithi090 ceph-mon[29180]: pgmap v5870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:49 smithi119 ceph-mon[32752]: pgmap v5871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:49 smithi167 ceph-mon[32751]: pgmap v5871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:49 smithi090 ceph-mon[29180]: pgmap v5871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:15:51.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:50 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9632.122510 secs 2024-04-26T21:15:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:50 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9632.122510 secs 2024-04-26T21:15:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:50 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9632.122510 secs 2024-04-26T21:15:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:51 smithi119 ceph-mon[32752]: pgmap v5872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9632.845525 secs 2024-04-26T21:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:51 smithi167 ceph-mon[32751]: pgmap v5872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9632.845525 secs 2024-04-26T21:15:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:51 smithi090 ceph-mon[29180]: pgmap v5872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:15:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9632.845525 secs 2024-04-26T21:15:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:53 smithi119 ceph-mon[32752]: pgmap v5873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:53 smithi167 ceph-mon[32751]: pgmap v5873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:53 smithi090 ceph-mon[29180]: pgmap v5873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:55 smithi119 ceph-mon[32752]: pgmap v5874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9637.122592 secs 2024-04-26T21:15:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:55 smithi167 ceph-mon[32751]: pgmap v5874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9637.122592 secs 2024-04-26T21:15:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:55 smithi090 ceph-mon[29180]: pgmap v5874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:15:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9637.122592 secs 2024-04-26T21:15:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9637.845596 secs 2024-04-26T21:15:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9637.845596 secs 2024-04-26T21:15:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9637.845596 secs 2024-04-26T21:15:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:57 smithi119 ceph-mon[32752]: pgmap v5875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:57 smithi167 ceph-mon[32751]: pgmap v5875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:15:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:57 smithi090 ceph-mon[29180]: pgmap v5875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:15:59 smithi119 ceph-mon[32752]: pgmap v5876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:15:59 smithi167 ceph-mon[32751]: pgmap v5876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:15:59 smithi090 ceph-mon[29180]: pgmap v5876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:00 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9642.122653 secs 2024-04-26T21:16:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:00 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9642.122653 secs 2024-04-26T21:16:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:00 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9642.122653 secs 2024-04-26T21:16:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:01 smithi119 ceph-mon[32752]: pgmap v5877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9642.845712 secs 2024-04-26T21:16:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:01 smithi167 ceph-mon[32751]: pgmap v5877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9642.845712 secs 2024-04-26T21:16:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:01 smithi090 ceph-mon[29180]: pgmap v5877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9642.845712 secs 2024-04-26T21:16:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:16:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:16:03.338+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:16:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:16:03.338+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:16:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:16:03.338+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:16:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:16:03.338+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:03.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:16:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:16:03.338+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:16:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:03 smithi119 ceph-mon[32752]: pgmap v5878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:03 smithi167 ceph-mon[32751]: pgmap v5878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:03 smithi090 ceph-mon[29180]: pgmap v5878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:05 smithi119 ceph-mon[32752]: pgmap v5879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9647.122737 secs 2024-04-26T21:16:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:05 smithi167 ceph-mon[32751]: pgmap v5879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9647.122737 secs 2024-04-26T21:16:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:05 smithi090 ceph-mon[29180]: pgmap v5879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9647.122737 secs 2024-04-26T21:16:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:06 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 9647.845829 secs 2024-04-26T21:16:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:06 smithi167 ceph-mon[32751]: slow request 2883.007017 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:16:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:06 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 9647.845829 secs 2024-04-26T21:16:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:06 smithi119 ceph-mon[32752]: slow request 2883.007017 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:16:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:06 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 9647.845829 secs 2024-04-26T21:16:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:06 smithi090 ceph-mon[29180]: slow request 2883.007017 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:16:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:07 smithi119 ceph-mon[32752]: pgmap v5880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:16:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:16:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:07 smithi167 ceph-mon[32751]: pgmap v5880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:16:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:16:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:07 smithi090 ceph-mon[29180]: pgmap v5880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:16:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:16:08.992 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:16:09.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:09 smithi090 ceph-mon[29180]: pgmap v5881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:09.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:09.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:09.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:09.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:09 smithi119 ceph-mon[32752]: pgmap v5881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:09 smithi167 ceph-mon[32751]: pgmap v5881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9652.122825 secs 2024-04-26T21:16:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9652.122825 secs 2024-04-26T21:16:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9652.122825 secs 2024-04-26T21:16:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: pgmap v5882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:16:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9652.845895 secs 2024-04-26T21:16:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: pgmap v5882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:16:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9652.845895 secs 2024-04-26T21:16:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: pgmap v5882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:16:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9652.845895 secs 2024-04-26T21:16:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:16:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:16:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:16:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:16:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:16:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:16:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:16:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:16:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:13 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:16:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:13 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:16:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:13 smithi119 ceph-mon[32752]: pgmap v5883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:16:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:13 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:16:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:13 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:16:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:13 smithi119 ceph-mon[32752]: pgmap v5884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 199 B/s wr, 2 op/s 2024-04-26T21:16:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:13 smithi119 ceph-mon[32752]: pgmap v5885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s rd, 248 B/s wr, 1 op/s 2024-04-26T21:16:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:13 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:16:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:13 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:16:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:13 smithi167 ceph-mon[32751]: pgmap v5883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:16:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:13 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:16:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:13 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:16:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:13 smithi167 ceph-mon[32751]: pgmap v5884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 199 B/s wr, 2 op/s 2024-04-26T21:16:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:13 smithi167 ceph-mon[32751]: pgmap v5885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s rd, 248 B/s wr, 1 op/s 2024-04-26T21:16:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:13 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:16:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:13 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:16:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:13 smithi090 ceph-mon[29180]: pgmap v5883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:16:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:13 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:16:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:13 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:16:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:13 smithi090 ceph-mon[29180]: pgmap v5884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 199 B/s wr, 2 op/s 2024-04-26T21:16:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:13 smithi090 ceph-mon[29180]: pgmap v5885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 744 B/s rd, 248 B/s wr, 1 op/s 2024-04-26T21:16:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:15 smithi119 ceph-mon[32752]: pgmap v5886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-26T21:16:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9657.122903 secs 2024-04-26T21:16:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:15 smithi167 ceph-mon[32751]: pgmap v5886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-26T21:16:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9657.122903 secs 2024-04-26T21:16:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:15 smithi090 ceph-mon[29180]: pgmap v5886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-26T21:16:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9657.122903 secs 2024-04-26T21:16:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9657.845984 secs 2024-04-26T21:16:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9657.845984 secs 2024-04-26T21:16:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9657.845984 secs 2024-04-26T21:16:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:17 smithi119 ceph-mon[32752]: pgmap v5887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-26T21:16:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:17 smithi167 ceph-mon[32751]: pgmap v5887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-26T21:16:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:17 smithi090 ceph-mon[29180]: pgmap v5887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 248 B/s wr, 2 op/s 2024-04-26T21:16:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:19 smithi119 ceph-mon[32752]: pgmap v5888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:19 smithi167 ceph-mon[32751]: pgmap v5888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:19 smithi090 ceph-mon[29180]: pgmap v5888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:21.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9662.122985 secs 2024-04-26T21:16:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9662.122985 secs 2024-04-26T21:16:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9662.122985 secs 2024-04-26T21:16:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:21 smithi119 ceph-mon[32752]: pgmap v5889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9662.846100 secs 2024-04-26T21:16:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:21 smithi167 ceph-mon[32751]: pgmap v5889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9662.846100 secs 2024-04-26T21:16:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:21 smithi090 ceph-mon[29180]: pgmap v5889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9662.846100 secs 2024-04-26T21:16:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:22 smithi119 ceph-mon[32752]: pgmap v5890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:16:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:22 smithi167 ceph-mon[32751]: pgmap v5890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:16:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:22 smithi090 ceph-mon[29180]: pgmap v5890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:16:25.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:25 smithi090 ceph-mon[29180]: pgmap v5891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:25.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9667.123046 secs 2024-04-26T21:16:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:25 smithi119 ceph-mon[32752]: pgmap v5891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9667.123046 secs 2024-04-26T21:16:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:25 smithi167 ceph-mon[32751]: pgmap v5891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9667.123046 secs 2024-04-26T21:16:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9667.846163 secs 2024-04-26T21:16:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9667.846163 secs 2024-04-26T21:16:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9667.846163 secs 2024-04-26T21:16:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:27 smithi119 ceph-mon[32752]: pgmap v5892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:27 smithi167 ceph-mon[32751]: pgmap v5892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:27 smithi090 ceph-mon[29180]: pgmap v5892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:29.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:29 smithi119 ceph-mon[32752]: pgmap v5893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:29 smithi167 ceph-mon[32751]: pgmap v5893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:29 smithi090 ceph-mon[29180]: pgmap v5893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9672.123132 secs 2024-04-26T21:16:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9672.123132 secs 2024-04-26T21:16:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9672.123132 secs 2024-04-26T21:16:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:31 smithi119 ceph-mon[32752]: pgmap v5894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9672.846297 secs 2024-04-26T21:16:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:31 smithi167 ceph-mon[32751]: pgmap v5894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9672.846297 secs 2024-04-26T21:16:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:31 smithi090 ceph-mon[29180]: pgmap v5894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9672.846297 secs 2024-04-26T21:16:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:33 smithi119 ceph-mon[32752]: pgmap v5895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:33 smithi167 ceph-mon[32751]: pgmap v5895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:33 smithi090 ceph-mon[29180]: pgmap v5895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:35 smithi119 ceph-mon[32752]: pgmap v5896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9677.123218 secs 2024-04-26T21:16:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:35 smithi167 ceph-mon[32751]: pgmap v5896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9677.123218 secs 2024-04-26T21:16:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:35 smithi090 ceph-mon[29180]: pgmap v5896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9677.123218 secs 2024-04-26T21:16:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9677.846431 secs 2024-04-26T21:16:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9677.846431 secs 2024-04-26T21:16:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9677.846431 secs 2024-04-26T21:16:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:37 smithi119 ceph-mon[32752]: pgmap v5897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:37 smithi167 ceph-mon[32751]: pgmap v5897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:37 smithi090 ceph-mon[29180]: pgmap v5897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:39 smithi119 ceph-mon[32752]: pgmap v5898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:39 smithi167 ceph-mon[32751]: pgmap v5898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:39 smithi090 ceph-mon[29180]: pgmap v5898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9682.123313 secs 2024-04-26T21:16:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:40 smithi119 ceph-mon[32752]: pgmap v5899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9682.846511 secs 2024-04-26T21:16:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9682.123313 secs 2024-04-26T21:16:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:40 smithi167 ceph-mon[32751]: pgmap v5899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9682.846511 secs 2024-04-26T21:16:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9682.123313 secs 2024-04-26T21:16:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:40 smithi090 ceph-mon[29180]: pgmap v5899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9682.846511 secs 2024-04-26T21:16:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:43 smithi119 ceph-mon[32752]: pgmap v5900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:43 smithi167 ceph-mon[32751]: pgmap v5900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:43 smithi090 ceph-mon[29180]: pgmap v5900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:45.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:45 smithi090 ceph-mon[29180]: pgmap v5901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:45.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9687.123397 secs 2024-04-26T21:16:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:45 smithi119 ceph-mon[32752]: pgmap v5901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9687.123397 secs 2024-04-26T21:16:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:45 smithi167 ceph-mon[32751]: pgmap v5901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9687.123397 secs 2024-04-26T21:16:46.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9687.846629 secs 2024-04-26T21:16:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9687.846629 secs 2024-04-26T21:16:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9687.846629 secs 2024-04-26T21:16:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:47 smithi119 ceph-mon[32752]: pgmap v5902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:47 smithi167 ceph-mon[32751]: pgmap v5902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:47 smithi090 ceph-mon[29180]: pgmap v5902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:49 smithi119 ceph-mon[32752]: pgmap v5903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:49 smithi167 ceph-mon[32751]: pgmap v5903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:49 smithi090 ceph-mon[29180]: pgmap v5903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9692.123472 secs 2024-04-26T21:16:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9692.123472 secs 2024-04-26T21:16:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9692.123472 secs 2024-04-26T21:16:51.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:51 smithi090 ceph-mon[29180]: pgmap v5904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:51.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9692.846699 secs 2024-04-26T21:16:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:51 smithi119 ceph-mon[32752]: pgmap v5904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9692.846699 secs 2024-04-26T21:16:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:51 smithi167 ceph-mon[32751]: pgmap v5904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9692.846699 secs 2024-04-26T21:16:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:53 smithi119 ceph-mon[32752]: pgmap v5905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:53 smithi167 ceph-mon[32751]: pgmap v5905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:53 smithi090 ceph-mon[29180]: pgmap v5905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:55 smithi119 ceph-mon[32752]: pgmap v5906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9697.123559 secs 2024-04-26T21:16:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:55 smithi167 ceph-mon[32751]: pgmap v5906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9697.123559 secs 2024-04-26T21:16:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:55 smithi090 ceph-mon[29180]: pgmap v5906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9697.123559 secs 2024-04-26T21:16:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9697.846823 secs 2024-04-26T21:16:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9697.846823 secs 2024-04-26T21:16:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9697.846823 secs 2024-04-26T21:16:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:57 smithi119 ceph-mon[32752]: pgmap v5907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:57 smithi167 ceph-mon[32751]: pgmap v5907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:57 smithi090 ceph-mon[29180]: pgmap v5907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:16:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:16:59 smithi119 ceph-mon[32752]: pgmap v5908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:16:59 smithi167 ceph-mon[32751]: pgmap v5908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:16:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:16:59 smithi090 ceph-mon[29180]: pgmap v5908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:00.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:00 smithi090 ceph-mon[29180]: 10 slow requests, 1 included below; oldest blocked for > 9702.123633 secs 2024-04-26T21:17:00.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:00 smithi090 ceph-mon[29180]: slow request 360.121630 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:17:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:00 smithi119 ceph-mon[32752]: 10 slow requests, 1 included below; oldest blocked for > 9702.123633 secs 2024-04-26T21:17:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:00 smithi119 ceph-mon[32752]: slow request 360.121630 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:17:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:00 smithi167 ceph-mon[32751]: 10 slow requests, 1 included below; oldest blocked for > 9702.123633 secs 2024-04-26T21:17:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:00 smithi167 ceph-mon[32751]: slow request 360.121630 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:17:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:01 smithi119 ceph-mon[32752]: pgmap v5909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9702.846902 secs 2024-04-26T21:17:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:01 smithi167 ceph-mon[32751]: pgmap v5909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9702.846902 secs 2024-04-26T21:17:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:01 smithi090 ceph-mon[29180]: pgmap v5909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9702.846902 secs 2024-04-26T21:17:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:03 smithi119 ceph-mon[32752]: pgmap v5910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:03 smithi167 ceph-mon[32751]: pgmap v5910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:03 smithi090 ceph-mon[29180]: pgmap v5910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:05 smithi119 ceph-mon[32752]: pgmap v5911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9707.123701 secs 2024-04-26T21:17:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:05 smithi167 ceph-mon[32751]: pgmap v5911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9707.123701 secs 2024-04-26T21:17:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:05 smithi090 ceph-mon[29180]: pgmap v5911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9707.123701 secs 2024-04-26T21:17:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9707.847009 secs 2024-04-26T21:17:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9707.847009 secs 2024-04-26T21:17:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9707.847009 secs 2024-04-26T21:17:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:07 smithi119 ceph-mon[32752]: pgmap v5912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:07 smithi167 ceph-mon[32751]: pgmap v5912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:07 smithi090 ceph-mon[29180]: pgmap v5912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:09 smithi119 ceph-mon[32752]: pgmap v5913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:09 smithi167 ceph-mon[32751]: pgmap v5913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:09 smithi090 ceph-mon[29180]: pgmap v5913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9712.123782 secs 2024-04-26T21:17:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9712.123782 secs 2024-04-26T21:17:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9712.123782 secs 2024-04-26T21:17:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:11 smithi119 ceph-mon[32752]: pgmap v5914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9712.847069 secs 2024-04-26T21:17:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:11 smithi167 ceph-mon[32751]: pgmap v5914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9712.847069 secs 2024-04-26T21:17:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:11 smithi090 ceph-mon[29180]: pgmap v5914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9712.847069 secs 2024-04-26T21:17:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:13 smithi119 ceph-mon[32752]: pgmap v5915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:13 smithi167 ceph-mon[32751]: pgmap v5915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:13 smithi090 ceph-mon[29180]: pgmap v5915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:17:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:14 smithi119 ceph-mon[32752]: pgmap v5916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:17:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:14 smithi119 ceph-mon[32752]: pgmap v5917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:14 smithi167 ceph-mon[32751]: pgmap v5916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:17:14.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:14 smithi167 ceph-mon[32751]: pgmap v5917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:14.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:17:14.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:17:14.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:14 smithi090 ceph-mon[29180]: pgmap v5916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:17:14.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:14 smithi090 ceph-mon[29180]: pgmap v5917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:14.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:17:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9717.123866 secs 2024-04-26T21:17:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9717.123866 secs 2024-04-26T21:17:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9717.123866 secs 2024-04-26T21:17:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9717.847197 secs 2024-04-26T21:17:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:16 smithi119 ceph-mon[32752]: pgmap v5918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9717.847197 secs 2024-04-26T21:17:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:16 smithi167 ceph-mon[32751]: pgmap v5918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9717.847197 secs 2024-04-26T21:17:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:16 smithi090 ceph-mon[29180]: pgmap v5918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:18 smithi119 ceph-mon[32752]: pgmap v5919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T21:17:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:18 smithi167 ceph-mon[32751]: pgmap v5919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T21:17:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:18 smithi090 ceph-mon[29180]: pgmap v5919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T21:17:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:20 smithi119 ceph-mon[32752]: pgmap v5920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9722.123951 secs 2024-04-26T21:17:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:20 smithi167 ceph-mon[32751]: pgmap v5920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:21.099 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9722.123951 secs 2024-04-26T21:17:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:20 smithi090 ceph-mon[29180]: pgmap v5920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9722.123951 secs 2024-04-26T21:17:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9722.847315 secs 2024-04-26T21:17:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9722.847315 secs 2024-04-26T21:17:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9722.847315 secs 2024-04-26T21:17:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:22 smithi119 ceph-mon[32752]: pgmap v5921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:22 smithi167 ceph-mon[32751]: pgmap v5921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:22 smithi090 ceph-mon[29180]: pgmap v5921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:17:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:24 smithi119 ceph-mon[32752]: pgmap v5922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:17:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:24 smithi167 ceph-mon[32751]: pgmap v5922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:17:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:24 smithi090 ceph-mon[29180]: pgmap v5922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:17:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9727.124039 secs 2024-04-26T21:17:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9727.124039 secs 2024-04-26T21:17:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9727.124039 secs 2024-04-26T21:17:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9727.847437 secs 2024-04-26T21:17:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:26 smithi119 ceph-mon[32752]: pgmap v5923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9727.847437 secs 2024-04-26T21:17:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:26 smithi167 ceph-mon[32751]: pgmap v5923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9727.847437 secs 2024-04-26T21:17:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:26 smithi090 ceph-mon[29180]: pgmap v5923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:28 smithi119 ceph-mon[32752]: pgmap v5924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:28 smithi167 ceph-mon[32751]: pgmap v5924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:28 smithi090 ceph-mon[29180]: pgmap v5924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:30 smithi119 ceph-mon[32752]: pgmap v5925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9732.124128 secs 2024-04-26T21:17:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:30 smithi167 ceph-mon[32751]: pgmap v5925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9732.124128 secs 2024-04-26T21:17:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:30 smithi090 ceph-mon[29180]: pgmap v5925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9732.124128 secs 2024-04-26T21:17:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9732.847548 secs 2024-04-26T21:17:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9732.847548 secs 2024-04-26T21:17:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9732.847548 secs 2024-04-26T21:17:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:32 smithi119 ceph-mon[32752]: pgmap v5926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:32 smithi167 ceph-mon[32751]: pgmap v5926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:32 smithi090 ceph-mon[29180]: pgmap v5926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:33 smithi090 ceph-mon[29180]: pgmap v5927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:33 smithi119 ceph-mon[32752]: pgmap v5927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:33 smithi167 ceph-mon[32751]: pgmap v5927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9737.124212 secs 2024-04-26T21:17:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9737.124212 secs 2024-04-26T21:17:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9737.124212 secs 2024-04-26T21:17:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9737.847657 secs 2024-04-26T21:17:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:36 smithi167 ceph-mon[32751]: pgmap v5928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9737.847657 secs 2024-04-26T21:17:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:36 smithi090 ceph-mon[29180]: pgmap v5928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9737.847657 secs 2024-04-26T21:17:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:36 smithi119 ceph-mon[32752]: pgmap v5928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:38 smithi119 ceph-mon[32752]: pgmap v5929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:38 smithi167 ceph-mon[32751]: pgmap v5929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:38 smithi090 ceph-mon[29180]: pgmap v5929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:40 smithi119 ceph-mon[32752]: pgmap v5930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9742.124296 secs 2024-04-26T21:17:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:40 smithi167 ceph-mon[32751]: pgmap v5930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9742.124296 secs 2024-04-26T21:17:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:40 smithi090 ceph-mon[29180]: pgmap v5930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9742.124296 secs 2024-04-26T21:17:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9742.847773 secs 2024-04-26T21:17:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9742.847773 secs 2024-04-26T21:17:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9742.847773 secs 2024-04-26T21:17:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:42 smithi119 ceph-mon[32752]: pgmap v5931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:42 smithi167 ceph-mon[32751]: pgmap v5931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:42 smithi090 ceph-mon[29180]: pgmap v5931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:44 smithi119 ceph-mon[32752]: pgmap v5932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:44 smithi167 ceph-mon[32751]: pgmap v5932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:44 smithi090 ceph-mon[29180]: pgmap v5932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9747.124354 secs 2024-04-26T21:17:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9747.124354 secs 2024-04-26T21:17:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9747.124354 secs 2024-04-26T21:17:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9747.847889 secs 2024-04-26T21:17:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:46 smithi119 ceph-mon[32752]: pgmap v5933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9747.847889 secs 2024-04-26T21:17:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:46 smithi167 ceph-mon[32751]: pgmap v5933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9747.847889 secs 2024-04-26T21:17:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:46 smithi090 ceph-mon[29180]: pgmap v5933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:48 smithi119 ceph-mon[32752]: pgmap v5934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:48 smithi167 ceph-mon[32751]: pgmap v5934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:48 smithi090 ceph-mon[29180]: pgmap v5934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:50 smithi119 ceph-mon[32752]: pgmap v5935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9752.124442 secs 2024-04-26T21:17:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:50 smithi167 ceph-mon[32751]: pgmap v5935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9752.124442 secs 2024-04-26T21:17:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:50 smithi090 ceph-mon[29180]: pgmap v5935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9752.124442 secs 2024-04-26T21:17:52.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9752.848013 secs 2024-04-26T21:17:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9752.848013 secs 2024-04-26T21:17:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9752.848013 secs 2024-04-26T21:17:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:52 smithi119 ceph-mon[32752]: pgmap v5936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:52 smithi167 ceph-mon[32751]: pgmap v5936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:52 smithi090 ceph-mon[29180]: pgmap v5936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:53 smithi119 ceph-mon[32752]: pgmap v5937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:53 smithi167 ceph-mon[32751]: pgmap v5937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:53 smithi090 ceph-mon[29180]: pgmap v5937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:17:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9757.124546 secs 2024-04-26T21:17:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9757.124546 secs 2024-04-26T21:17:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9757.124546 secs 2024-04-26T21:17:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9757.848144 secs 2024-04-26T21:17:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:56 smithi167 ceph-mon[32751]: pgmap v5938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9757.848144 secs 2024-04-26T21:17:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:56 smithi090 ceph-mon[29180]: pgmap v5938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9757.848144 secs 2024-04-26T21:17:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:56 smithi119 ceph-mon[32752]: pgmap v5938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:17:58 smithi119 ceph-mon[32752]: pgmap v5939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:17:58 smithi167 ceph-mon[32751]: pgmap v5939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:17:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:17:58 smithi090 ceph-mon[29180]: pgmap v5939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:00 smithi119 ceph-mon[32752]: pgmap v5940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:00 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9762.124626 secs 2024-04-26T21:18:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:00 smithi167 ceph-mon[32751]: pgmap v5940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:00 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9762.124626 secs 2024-04-26T21:18:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:00 smithi090 ceph-mon[29180]: pgmap v5940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:00 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9762.124626 secs 2024-04-26T21:18:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9762.848267 secs 2024-04-26T21:18:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9762.848267 secs 2024-04-26T21:18:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9762.848267 secs 2024-04-26T21:18:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:02 smithi119 ceph-mon[32752]: pgmap v5941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:02 smithi167 ceph-mon[32751]: pgmap v5941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:02 smithi090 ceph-mon[29180]: pgmap v5941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:04 smithi119 ceph-mon[32752]: pgmap v5942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:04 smithi167 ceph-mon[32751]: pgmap v5942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:04 smithi090 ceph-mon[29180]: pgmap v5942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:05 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9767.124709 secs 2024-04-26T21:18:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:05 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9767.124709 secs 2024-04-26T21:18:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:05 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9767.124709 secs 2024-04-26T21:18:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9767.848393 secs 2024-04-26T21:18:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:06 smithi119 ceph-mon[32752]: pgmap v5943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9767.848393 secs 2024-04-26T21:18:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:06 smithi167 ceph-mon[32751]: pgmap v5943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9767.848393 secs 2024-04-26T21:18:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:06 smithi090 ceph-mon[29180]: pgmap v5943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:08 smithi119 ceph-mon[32752]: pgmap v5944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:08 smithi167 ceph-mon[32751]: pgmap v5944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:08 smithi090 ceph-mon[29180]: pgmap v5944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:10 smithi119 ceph-mon[32752]: pgmap v5945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:10 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9772.124816 secs 2024-04-26T21:18:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:10 smithi167 ceph-mon[32751]: pgmap v5945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:10 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9772.124816 secs 2024-04-26T21:18:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:10 smithi090 ceph-mon[29180]: pgmap v5945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:10 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9772.124816 secs 2024-04-26T21:18:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9772.848512 secs 2024-04-26T21:18:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9772.848512 secs 2024-04-26T21:18:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9772.848512 secs 2024-04-26T21:18:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:12 smithi119 ceph-mon[32752]: pgmap v5946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:12 smithi167 ceph-mon[32751]: pgmap v5946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:12 smithi090 ceph-mon[29180]: pgmap v5946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:13 smithi119 ceph-mon[32752]: pgmap v5947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:13 smithi167 ceph-mon[32751]: pgmap v5947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:13 smithi090 ceph-mon[29180]: pgmap v5947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:14 smithi119 ceph-mon[32752]: pgmap v5948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:18:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:14 smithi119 ceph-mon[32752]: pgmap v5949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:18:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:14 smithi167 ceph-mon[32751]: pgmap v5948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:18:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:14 smithi167 ceph-mon[32751]: pgmap v5949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:18:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:18:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:18:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:18:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:14 smithi090 ceph-mon[29180]: pgmap v5948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:18:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:14 smithi090 ceph-mon[29180]: pgmap v5949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:18:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:15 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9777.124902 secs 2024-04-26T21:18:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:15 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9777.848633 secs 2024-04-26T21:18:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:15 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9777.124902 secs 2024-04-26T21:18:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:15 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9777.848633 secs 2024-04-26T21:18:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:15 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9777.124902 secs 2024-04-26T21:18:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:15 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9777.848633 secs 2024-04-26T21:18:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:16 smithi119 ceph-mon[32752]: pgmap v5950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:16 smithi167 ceph-mon[32751]: pgmap v5950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:16 smithi090 ceph-mon[29180]: pgmap v5950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:19 smithi119 ceph-mon[32752]: pgmap v5951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:19 smithi167 ceph-mon[32751]: pgmap v5951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:19.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:19 smithi090 ceph-mon[29180]: pgmap v5951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:20 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9782.124991 secs 2024-04-26T21:18:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:20 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9782.124991 secs 2024-04-26T21:18:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:20 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9782.124991 secs 2024-04-26T21:18:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:21 smithi119 ceph-mon[32752]: pgmap v5952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9782.848726 secs 2024-04-26T21:18:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:21 smithi167 ceph-mon[32751]: pgmap v5952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9782.848726 secs 2024-04-26T21:18:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:21 smithi090 ceph-mon[29180]: pgmap v5952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:18:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9782.848726 secs 2024-04-26T21:18:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:23 smithi119 ceph-mon[32752]: pgmap v5953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:18:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:23 smithi167 ceph-mon[32751]: pgmap v5953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:18:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:23 smithi090 ceph-mon[29180]: pgmap v5953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:18:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:25 smithi119 ceph-mon[32752]: pgmap v5954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:18:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:25 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9787.125094 secs 2024-04-26T21:18:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:25 smithi167 ceph-mon[32751]: pgmap v5954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:18:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:25 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9787.125094 secs 2024-04-26T21:18:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:25 smithi090 ceph-mon[29180]: pgmap v5954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:18:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:25 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9787.125094 secs 2024-04-26T21:18:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9787.848823 secs 2024-04-26T21:18:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9787.848823 secs 2024-04-26T21:18:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9787.848823 secs 2024-04-26T21:18:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:27 smithi119 ceph-mon[32752]: pgmap v5955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:27 smithi167 ceph-mon[32751]: pgmap v5955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:27 smithi090 ceph-mon[29180]: pgmap v5955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:29 smithi119 ceph-mon[32752]: pgmap v5956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:29 smithi167 ceph-mon[32751]: pgmap v5956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:29 smithi090 ceph-mon[29180]: pgmap v5956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:30 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9792.125179 secs 2024-04-26T21:18:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:30 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9792.125179 secs 2024-04-26T21:18:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:30 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9792.125179 secs 2024-04-26T21:18:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:31 smithi119 ceph-mon[32752]: pgmap v5957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9792.848942 secs 2024-04-26T21:18:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:31 smithi167 ceph-mon[32751]: pgmap v5957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9792.848942 secs 2024-04-26T21:18:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:31 smithi090 ceph-mon[29180]: pgmap v5957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9792.848942 secs 2024-04-26T21:18:33.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:18:33.607+0000 7f76f7fff640 1 -- 172.21.15.90:0/4041546713 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- auth(proto 2 165 bytes epoch 0) -- 0x7f76f00034c0 con 0x7f7710106800 2024-04-26T21:18:33.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:18:33.608+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mon.0 v2:172.21.15.90:3300/0 7 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (secure 0 0 0) 0x7f76f00034c0 con 0x7f7710106800 2024-04-26T21:18:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:33 smithi119 ceph-mon[32752]: pgmap v5958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:33 smithi167 ceph-mon[32751]: pgmap v5958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:33 smithi090 ceph-mon[29180]: pgmap v5958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:35 smithi119 ceph-mon[32752]: pgmap v5959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:35 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9797.125266 secs 2024-04-26T21:18:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:35 smithi167 ceph-mon[32751]: pgmap v5959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:35 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9797.125266 secs 2024-04-26T21:18:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:35 smithi090 ceph-mon[29180]: pgmap v5959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:35 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9797.125266 secs 2024-04-26T21:18:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9797.849090 secs 2024-04-26T21:18:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9797.849090 secs 2024-04-26T21:18:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9797.849090 secs 2024-04-26T21:18:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:37 smithi167 ceph-mon[32751]: pgmap v5960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:37 smithi090 ceph-mon[29180]: pgmap v5960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:37 smithi119 ceph-mon[32752]: pgmap v5960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:39 smithi167 ceph-mon[32751]: pgmap v5961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:39 smithi090 ceph-mon[29180]: pgmap v5961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:39 smithi119 ceph-mon[32752]: pgmap v5961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:40 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9802.125327 secs 2024-04-26T21:18:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:40 smithi119 ceph-mon[32752]: pgmap v5962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9802.849211 secs 2024-04-26T21:18:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:40 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9802.125327 secs 2024-04-26T21:18:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:40 smithi167 ceph-mon[32751]: pgmap v5962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9802.849211 secs 2024-04-26T21:18:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:40 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9802.125327 secs 2024-04-26T21:18:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:40 smithi090 ceph-mon[29180]: pgmap v5962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9802.849211 secs 2024-04-26T21:18:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:43 smithi119 ceph-mon[32752]: pgmap v5963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:43 smithi167 ceph-mon[32751]: pgmap v5963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:43 smithi090 ceph-mon[29180]: pgmap v5963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:45 smithi119 ceph-mon[32752]: pgmap v5964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:45 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9807.125413 secs 2024-04-26T21:18:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:45 smithi167 ceph-mon[32751]: pgmap v5964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:45 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9807.125413 secs 2024-04-26T21:18:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:45 smithi090 ceph-mon[29180]: pgmap v5964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:45 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9807.125413 secs 2024-04-26T21:18:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9807.849333 secs 2024-04-26T21:18:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9807.849333 secs 2024-04-26T21:18:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9807.849333 secs 2024-04-26T21:18:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:47 smithi119 ceph-mon[32752]: pgmap v5965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:47 smithi167 ceph-mon[32751]: pgmap v5965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:47 smithi090 ceph-mon[29180]: pgmap v5965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:49 smithi119 ceph-mon[32752]: pgmap v5966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:49 smithi167 ceph-mon[32751]: pgmap v5966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:49.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:49 smithi090 ceph-mon[29180]: pgmap v5966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:50 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9812.125488 secs 2024-04-26T21:18:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:50 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9812.125488 secs 2024-04-26T21:18:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:50 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9812.125488 secs 2024-04-26T21:18:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:51 smithi119 ceph-mon[32752]: pgmap v5967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9812.849470 secs 2024-04-26T21:18:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:51 smithi167 ceph-mon[32751]: pgmap v5967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9812.849470 secs 2024-04-26T21:18:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:51 smithi090 ceph-mon[29180]: pgmap v5967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9812.849470 secs 2024-04-26T21:18:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:53 smithi119 ceph-mon[32752]: pgmap v5968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:53 smithi167 ceph-mon[32751]: pgmap v5968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:53 smithi090 ceph-mon[29180]: pgmap v5968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:55 smithi119 ceph-mon[32752]: pgmap v5969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:55 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9817.125569 secs 2024-04-26T21:18:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:55 smithi167 ceph-mon[32751]: pgmap v5969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:55 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9817.125569 secs 2024-04-26T21:18:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:55 smithi090 ceph-mon[29180]: pgmap v5969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:55 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9817.125569 secs 2024-04-26T21:18:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9817.849587 secs 2024-04-26T21:18:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9817.849587 secs 2024-04-26T21:18:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9817.849587 secs 2024-04-26T21:18:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:57 smithi167 ceph-mon[32751]: pgmap v5970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:57 smithi090 ceph-mon[29180]: pgmap v5970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:57 smithi119 ceph-mon[32752]: pgmap v5970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:18:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:18:59 smithi167 ceph-mon[32751]: pgmap v5971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:18:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:18:59 smithi090 ceph-mon[29180]: pgmap v5971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:18:59 smithi119 ceph-mon[32752]: pgmap v5971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:00 smithi167 ceph-mon[32751]: 10 slow requests, 0 included below; oldest blocked for > 9822.125628 secs 2024-04-26T21:19:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:00 smithi090 ceph-mon[29180]: 10 slow requests, 0 included below; oldest blocked for > 9822.125628 secs 2024-04-26T21:19:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:00 smithi119 ceph-mon[32752]: 10 slow requests, 0 included below; oldest blocked for > 9822.125628 secs 2024-04-26T21:19:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:01 smithi167 ceph-mon[32751]: pgmap v5972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9822.849679 secs 2024-04-26T21:19:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:01 smithi090 ceph-mon[29180]: pgmap v5972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9822.849679 secs 2024-04-26T21:19:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:01 smithi119 ceph-mon[32752]: pgmap v5972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9822.849679 secs 2024-04-26T21:19:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:03 smithi090 ceph-mon[29180]: pgmap v5973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:03 smithi119 ceph-mon[32752]: pgmap v5973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:03 smithi167 ceph-mon[32751]: pgmap v5973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:05 smithi090 ceph-mon[29180]: pgmap v5974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:05 smithi090 ceph-mon[29180]: 16 slow requests, 2 included below; oldest blocked for > 9827.125714 secs 2024-04-26T21:19:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:05 smithi090 ceph-mon[29180]: slow request 1441.887658 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:19:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:05 smithi090 ceph-mon[29180]: slow request 181.533958 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:19:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:05 smithi119 ceph-mon[32752]: pgmap v5974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:05 smithi119 ceph-mon[32752]: 16 slow requests, 2 included below; oldest blocked for > 9827.125714 secs 2024-04-26T21:19:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:05 smithi119 ceph-mon[32752]: slow request 1441.887658 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:19:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:05 smithi119 ceph-mon[32752]: slow request 181.533958 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:19:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:05 smithi167 ceph-mon[32751]: pgmap v5974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:05 smithi167 ceph-mon[32751]: 16 slow requests, 2 included below; oldest blocked for > 9827.125714 secs 2024-04-26T21:19:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:05 smithi167 ceph-mon[32751]: slow request 1441.887658 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:19:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:05 smithi167 ceph-mon[32751]: slow request 181.533958 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:19:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9827.849824 secs 2024-04-26T21:19:06.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9827.849824 secs 2024-04-26T21:19:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9827.849824 secs 2024-04-26T21:19:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:07 smithi090 ceph-mon[29180]: pgmap v5975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:19:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:19:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:07 smithi119 ceph-mon[32752]: pgmap v5975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:19:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:19:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:07 smithi167 ceph-mon[32751]: pgmap v5975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:19:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:19:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:09 smithi090 ceph-mon[29180]: pgmap v5976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:09 smithi119 ceph-mon[32752]: pgmap v5976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:09 smithi167 ceph-mon[32751]: pgmap v5976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:10 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9832.125800 secs 2024-04-26T21:19:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:10 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9832.125800 secs 2024-04-26T21:19:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:10 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9832.125800 secs 2024-04-26T21:19:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:11 smithi090 ceph-mon[29180]: pgmap v5977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9832.849945 secs 2024-04-26T21:19:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:11 smithi119 ceph-mon[32752]: pgmap v5977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9832.849945 secs 2024-04-26T21:19:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:11 smithi167 ceph-mon[32751]: pgmap v5977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9832.849945 secs 2024-04-26T21:19:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:13 smithi090 ceph-mon[29180]: pgmap v5978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:13 smithi119 ceph-mon[32752]: pgmap v5978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:13 smithi167 ceph-mon[32751]: pgmap v5978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:19:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:15 smithi090 ceph-mon[29180]: pgmap v5979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:15 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9837.125868 secs 2024-04-26T21:19:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:19:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:15 smithi119 ceph-mon[32752]: pgmap v5979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:15 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9837.125868 secs 2024-04-26T21:19:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:19:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:15 smithi167 ceph-mon[32751]: pgmap v5979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:15 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9837.125868 secs 2024-04-26T21:19:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:19:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:19:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:19:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:16 smithi090 ceph-mon[29180]: pgmap v5980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:19:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:16 smithi090 ceph-mon[29180]: pgmap v5981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9837.850008 secs 2024-04-26T21:19:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:16 smithi119 ceph-mon[32752]: pgmap v5980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:19:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:16 smithi119 ceph-mon[32752]: pgmap v5981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9837.850008 secs 2024-04-26T21:19:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:16 smithi167 ceph-mon[32751]: pgmap v5980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:19:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:16 smithi167 ceph-mon[32751]: pgmap v5981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9837.850008 secs 2024-04-26T21:19:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:18 smithi090 ceph-mon[29180]: pgmap v5982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:19:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:18 smithi119 ceph-mon[32752]: pgmap v5982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:19:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:18 smithi167 ceph-mon[32751]: pgmap v5982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:19:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:20 smithi090 ceph-mon[29180]: pgmap v5983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:20 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9842.125931 secs 2024-04-26T21:19:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:20 smithi119 ceph-mon[32752]: pgmap v5983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:20 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9842.125931 secs 2024-04-26T21:19:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:20 smithi167 ceph-mon[32751]: pgmap v5983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:20 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9842.125931 secs 2024-04-26T21:19:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9842.850124 secs 2024-04-26T21:19:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9842.850124 secs 2024-04-26T21:19:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9842.850124 secs 2024-04-26T21:19:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:22 smithi090 ceph-mon[29180]: pgmap v5984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:22 smithi119 ceph-mon[32752]: pgmap v5984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:22 smithi167 ceph-mon[32751]: pgmap v5984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:19:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:24 smithi090 ceph-mon[29180]: pgmap v5985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:19:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:24 smithi119 ceph-mon[32752]: pgmap v5985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:19:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:24 smithi167 ceph-mon[32751]: pgmap v5985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:19:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:25 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9847.125993 secs 2024-04-26T21:19:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:25 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9847.125993 secs 2024-04-26T21:19:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:25 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9847.125993 secs 2024-04-26T21:19:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:26 smithi119 ceph-mon[32752]: pgmap v5986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9847.850241 secs 2024-04-26T21:19:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:26 smithi167 ceph-mon[32751]: pgmap v5986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9847.850241 secs 2024-04-26T21:19:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:26 smithi090 ceph-mon[29180]: pgmap v5986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:19:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9847.850241 secs 2024-04-26T21:19:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:28 smithi119 ceph-mon[32752]: pgmap v5987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:28 smithi167 ceph-mon[32751]: pgmap v5987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:28 smithi090 ceph-mon[29180]: pgmap v5987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:30 smithi119 ceph-mon[32752]: pgmap v5988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:30 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9852.126077 secs 2024-04-26T21:19:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:30 smithi167 ceph-mon[32751]: pgmap v5988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:30 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9852.126077 secs 2024-04-26T21:19:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:30 smithi090 ceph-mon[29180]: pgmap v5988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:30 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9852.126077 secs 2024-04-26T21:19:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9852.850320 secs 2024-04-26T21:19:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9852.850320 secs 2024-04-26T21:19:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9852.850320 secs 2024-04-26T21:19:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:32 smithi119 ceph-mon[32752]: pgmap v5989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:32 smithi167 ceph-mon[32751]: pgmap v5989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:32 smithi090 ceph-mon[29180]: pgmap v5989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:34 smithi119 ceph-mon[32752]: pgmap v5990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-26T21:19:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:34 smithi167 ceph-mon[32751]: pgmap v5990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-26T21:19:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:34 smithi090 ceph-mon[29180]: pgmap v5990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-26T21:19:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:35 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9857.126163 secs 2024-04-26T21:19:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:35 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9857.126163 secs 2024-04-26T21:19:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:35 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9857.126163 secs 2024-04-26T21:19:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:36 smithi090 ceph-mon[29180]: pgmap v5991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9857.850441 secs 2024-04-26T21:19:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:36 smithi167 ceph-mon[32751]: pgmap v5991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9857.850441 secs 2024-04-26T21:19:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:36 smithi119 ceph-mon[32752]: pgmap v5991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9857.850441 secs 2024-04-26T21:19:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:38 smithi119 ceph-mon[32752]: pgmap v5992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:38 smithi167 ceph-mon[32751]: pgmap v5992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:38 smithi090 ceph-mon[29180]: pgmap v5992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:19:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:40 smithi119 ceph-mon[32752]: pgmap v5993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:40 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9862.126250 secs 2024-04-26T21:19:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:40 smithi167 ceph-mon[32751]: pgmap v5993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:40 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9862.126250 secs 2024-04-26T21:19:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:40 smithi090 ceph-mon[29180]: pgmap v5993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:40 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9862.126250 secs 2024-04-26T21:19:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9862.850579 secs 2024-04-26T21:19:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9862.850579 secs 2024-04-26T21:19:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9862.850579 secs 2024-04-26T21:19:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:42 smithi119 ceph-mon[32752]: pgmap v5994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:42 smithi167 ceph-mon[32751]: pgmap v5994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:42 smithi090 ceph-mon[29180]: pgmap v5994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:44 smithi119 ceph-mon[32752]: pgmap v5995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:44 smithi167 ceph-mon[32751]: pgmap v5995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:44 smithi090 ceph-mon[29180]: pgmap v5995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:45 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9867.126309 secs 2024-04-26T21:19:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:45 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9867.126309 secs 2024-04-26T21:19:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:45 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9867.126309 secs 2024-04-26T21:19:46.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:46 smithi119 ceph-mon[32752]: pgmap v5996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:19:46.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9867.850694 secs 2024-04-26T21:19:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:46 smithi167 ceph-mon[32751]: pgmap v5996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:19:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9867.850694 secs 2024-04-26T21:19:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:46 smithi090 ceph-mon[29180]: pgmap v5996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:19:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9867.850694 secs 2024-04-26T21:19:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:48 smithi119 ceph-mon[32752]: pgmap v5997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:48 smithi167 ceph-mon[32751]: pgmap v5997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:48 smithi090 ceph-mon[29180]: pgmap v5997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:50 smithi119 ceph-mon[32752]: pgmap v5998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:50 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9872.126388 secs 2024-04-26T21:19:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:50 smithi167 ceph-mon[32751]: pgmap v5998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:50 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9872.126388 secs 2024-04-26T21:19:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:50 smithi090 ceph-mon[29180]: pgmap v5998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:50 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9872.126388 secs 2024-04-26T21:19:52.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9872.850811 secs 2024-04-26T21:19:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9872.850811 secs 2024-04-26T21:19:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9872.850811 secs 2024-04-26T21:19:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:52 smithi119 ceph-mon[32752]: pgmap v5999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:52 smithi167 ceph-mon[32751]: pgmap v5999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:52 smithi090 ceph-mon[29180]: pgmap v5999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:54 smithi119 ceph-mon[32752]: pgmap v6000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:54 smithi167 ceph-mon[32751]: pgmap v6000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:54 smithi090 ceph-mon[29180]: pgmap v6000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:19:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:55 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9877.126464 secs 2024-04-26T21:19:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:55 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9877.126464 secs 2024-04-26T21:19:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:55 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9877.126464 secs 2024-04-26T21:19:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:56 smithi119 ceph-mon[32752]: pgmap v6001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:19:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9877.850925 secs 2024-04-26T21:19:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:56 smithi167 ceph-mon[32751]: pgmap v6001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:19:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9877.850925 secs 2024-04-26T21:19:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:56 smithi090 ceph-mon[29180]: pgmap v6001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:19:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9877.850925 secs 2024-04-26T21:19:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:19:58 smithi119 ceph-mon[32752]: pgmap v6002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:19:58 smithi167 ceph-mon[32751]: pgmap v6002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:19:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:19:58 smithi090 ceph-mon[29180]: pgmap v6002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:00 smithi119 ceph-mon[32752]: pgmap v6003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:00 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9882.126549 secs 2024-04-26T21:20:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:20:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:00 smithi167 ceph-mon[32751]: pgmap v6003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:00 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9882.126549 secs 2024-04-26T21:20:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:20:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:00 smithi090 ceph-mon[29180]: pgmap v6003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:00 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9882.126549 secs 2024-04-26T21:20:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:20:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9882.851062 secs 2024-04-26T21:20:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9882.851062 secs 2024-04-26T21:20:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9882.851062 secs 2024-04-26T21:20:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:02 smithi119 ceph-mon[32752]: pgmap v6004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:02 smithi167 ceph-mon[32751]: pgmap v6004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:02 smithi090 ceph-mon[29180]: pgmap v6004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:04 smithi119 ceph-mon[32752]: pgmap v6005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:04 smithi167 ceph-mon[32751]: pgmap v6005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:04 smithi090 ceph-mon[29180]: pgmap v6005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:05 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9887.126633 secs 2024-04-26T21:20:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:05 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9887.126633 secs 2024-04-26T21:20:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:05 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9887.126633 secs 2024-04-26T21:20:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:06 smithi119 ceph-mon[32752]: pgmap v6006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:20:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9887.851184 secs 2024-04-26T21:20:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:06 smithi167 ceph-mon[32751]: pgmap v6006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:20:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9887.851184 secs 2024-04-26T21:20:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:06 smithi090 ceph-mon[29180]: pgmap v6006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:20:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9887.851184 secs 2024-04-26T21:20:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:08 smithi119 ceph-mon[32752]: pgmap v6007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:08 smithi167 ceph-mon[32751]: pgmap v6007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:08 smithi090 ceph-mon[29180]: pgmap v6007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:10 smithi119 ceph-mon[32752]: pgmap v6008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:10 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9892.126695 secs 2024-04-26T21:20:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:10 smithi167 ceph-mon[32751]: pgmap v6008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:10 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9892.126695 secs 2024-04-26T21:20:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:10 smithi090 ceph-mon[29180]: pgmap v6008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:10 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9892.126695 secs 2024-04-26T21:20:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9892.851299 secs 2024-04-26T21:20:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9892.851299 secs 2024-04-26T21:20:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9892.851299 secs 2024-04-26T21:20:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:12 smithi119 ceph-mon[32752]: pgmap v6009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:12 smithi167 ceph-mon[32751]: pgmap v6009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:12 smithi090 ceph-mon[29180]: pgmap v6009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:14 smithi119 ceph-mon[32752]: pgmap v6010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:14 smithi167 ceph-mon[32751]: pgmap v6010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:14 smithi090 ceph-mon[29180]: pgmap v6010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:15 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9897.126779 secs 2024-04-26T21:20:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:15 smithi119 ceph-mon[32752]: pgmap v6011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:20:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:15 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9897.851409 secs 2024-04-26T21:20:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:15 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9897.126779 secs 2024-04-26T21:20:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:15 smithi167 ceph-mon[32751]: pgmap v6011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:20:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:15 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9897.851409 secs 2024-04-26T21:20:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:15 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9897.126779 secs 2024-04-26T21:20:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:15 smithi090 ceph-mon[29180]: pgmap v6011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:20:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:20:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:15 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9897.851409 secs 2024-04-26T21:20:17.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:17.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:17.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:16 smithi119 ceph-mon[32752]: pgmap v6012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:20:17.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:16 smithi119 ceph-mon[32752]: pgmap v6013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:17.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:20:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:16 smithi167 ceph-mon[32751]: pgmap v6012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:20:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:16 smithi167 ceph-mon[32751]: pgmap v6013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:20:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:20:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:20:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:16 smithi090 ceph-mon[29180]: pgmap v6012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:20:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:16 smithi090 ceph-mon[29180]: pgmap v6013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:20:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:19 smithi090 ceph-mon[29180]: pgmap v6014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:20:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:19 smithi119 ceph-mon[32752]: pgmap v6014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:20:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:19 smithi167 ceph-mon[32751]: pgmap v6014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:20:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:20 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9902.126823 secs 2024-04-26T21:20:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:20 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9902.126823 secs 2024-04-26T21:20:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:20 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9902.126823 secs 2024-04-26T21:20:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:21 smithi090 ceph-mon[29180]: pgmap v6015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9902.851535 secs 2024-04-26T21:20:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:21 smithi119 ceph-mon[32752]: pgmap v6015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9902.851535 secs 2024-04-26T21:20:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:21 smithi167 ceph-mon[32751]: pgmap v6015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9902.851535 secs 2024-04-26T21:20:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:23 smithi090 ceph-mon[29180]: pgmap v6016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:20:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:23 smithi119 ceph-mon[32752]: pgmap v6016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:20:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:23 smithi167 ceph-mon[32751]: pgmap v6016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:20:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:25 smithi090 ceph-mon[29180]: pgmap v6017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:25 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9907.126906 secs 2024-04-26T21:20:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:25 smithi119 ceph-mon[32752]: pgmap v6017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:25.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:25 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9907.126906 secs 2024-04-26T21:20:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:25 smithi167 ceph-mon[32751]: pgmap v6017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:20:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:25 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9907.126906 secs 2024-04-26T21:20:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9907.851628 secs 2024-04-26T21:20:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9907.851628 secs 2024-04-26T21:20:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9907.851628 secs 2024-04-26T21:20:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:27 smithi090 ceph-mon[29180]: pgmap v6018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:20:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:27 smithi119 ceph-mon[32752]: pgmap v6018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:20:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:27 smithi167 ceph-mon[32751]: pgmap v6018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:20:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:29 smithi090 ceph-mon[29180]: pgmap v6019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:29 smithi119 ceph-mon[32752]: pgmap v6019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:29 smithi167 ceph-mon[32751]: pgmap v6019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:30 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9912.126971 secs 2024-04-26T21:20:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:30 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9912.126971 secs 2024-04-26T21:20:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:30 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9912.126971 secs 2024-04-26T21:20:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:31 smithi090 ceph-mon[29180]: pgmap v6020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9912.851752 secs 2024-04-26T21:20:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:31 smithi119 ceph-mon[32752]: pgmap v6020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9912.851752 secs 2024-04-26T21:20:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:31 smithi167 ceph-mon[32751]: pgmap v6020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9912.851752 secs 2024-04-26T21:20:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:33 smithi090 ceph-mon[29180]: pgmap v6021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:33 smithi119 ceph-mon[32752]: pgmap v6021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:33 smithi167 ceph-mon[32751]: pgmap v6021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:35.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:35 smithi090 ceph-mon[29180]: pgmap v6022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:35.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:35 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9917.127032 secs 2024-04-26T21:20:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:35 smithi119 ceph-mon[32752]: pgmap v6022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:35 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9917.127032 secs 2024-04-26T21:20:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:35 smithi167 ceph-mon[32751]: pgmap v6022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:35 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9917.127032 secs 2024-04-26T21:20:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9917.851845 secs 2024-04-26T21:20:36.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9917.851845 secs 2024-04-26T21:20:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9917.851845 secs 2024-04-26T21:20:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:37 smithi090 ceph-mon[29180]: pgmap v6023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:37 smithi119 ceph-mon[32752]: pgmap v6023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:37 smithi167 ceph-mon[32751]: pgmap v6023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:39 smithi090 ceph-mon[29180]: pgmap v6024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:39 smithi119 ceph-mon[32752]: pgmap v6024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:39 smithi167 ceph-mon[32751]: pgmap v6024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:40 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9922.127093 secs 2024-04-26T21:20:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:40 smithi119 ceph-mon[32752]: pgmap v6025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9922.851958 secs 2024-04-26T21:20:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:40 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9922.127093 secs 2024-04-26T21:20:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:40 smithi167 ceph-mon[32751]: pgmap v6025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9922.851958 secs 2024-04-26T21:20:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:40 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9922.127093 secs 2024-04-26T21:20:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:40 smithi090 ceph-mon[29180]: pgmap v6025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9922.851958 secs 2024-04-26T21:20:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:43 smithi090 ceph-mon[29180]: pgmap v6026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:43 smithi119 ceph-mon[32752]: pgmap v6026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:43 smithi167 ceph-mon[32751]: pgmap v6026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:45 smithi090 ceph-mon[29180]: pgmap v6027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:45 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9927.127178 secs 2024-04-26T21:20:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:45 smithi119 ceph-mon[32752]: pgmap v6027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:45.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:45 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9927.127178 secs 2024-04-26T21:20:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:45 smithi167 ceph-mon[32751]: pgmap v6027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:45 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9927.127178 secs 2024-04-26T21:20:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9927.852069 secs 2024-04-26T21:20:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9927.852069 secs 2024-04-26T21:20:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9927.852069 secs 2024-04-26T21:20:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:47 smithi090 ceph-mon[29180]: pgmap v6028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:47.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:47 smithi119 ceph-mon[32752]: pgmap v6028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:47 smithi167 ceph-mon[32751]: pgmap v6028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:49 smithi090 ceph-mon[29180]: pgmap v6029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:49 smithi119 ceph-mon[32752]: pgmap v6029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:49 smithi167 ceph-mon[32751]: pgmap v6029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:50 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9932.127264 secs 2024-04-26T21:20:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:50 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9932.127264 secs 2024-04-26T21:20:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:50 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9932.127264 secs 2024-04-26T21:20:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:51 smithi090 ceph-mon[29180]: pgmap v6030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9932.852189 secs 2024-04-26T21:20:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:51 smithi119 ceph-mon[32752]: pgmap v6030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9932.852189 secs 2024-04-26T21:20:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:51 smithi167 ceph-mon[32751]: pgmap v6030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9932.852189 secs 2024-04-26T21:20:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:53 smithi090 ceph-mon[29180]: pgmap v6031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:53 smithi119 ceph-mon[32752]: pgmap v6031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:53 smithi167 ceph-mon[32751]: pgmap v6031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:55 smithi090 ceph-mon[29180]: pgmap v6032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:55 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9937.127349 secs 2024-04-26T21:20:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:55 smithi119 ceph-mon[32752]: pgmap v6032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:55 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9937.127349 secs 2024-04-26T21:20:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:55 smithi167 ceph-mon[32751]: pgmap v6032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:20:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:55 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9937.127349 secs 2024-04-26T21:20:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9937.852306 secs 2024-04-26T21:20:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9937.852306 secs 2024-04-26T21:20:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9937.852306 secs 2024-04-26T21:20:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:57 smithi090 ceph-mon[29180]: pgmap v6033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:57 smithi119 ceph-mon[32752]: pgmap v6033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:57 smithi167 ceph-mon[32751]: pgmap v6033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:20:59 smithi090 ceph-mon[29180]: pgmap v6034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:20:59 smithi119 ceph-mon[32752]: pgmap v6034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:20:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:20:59 smithi167 ceph-mon[32751]: pgmap v6034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:00 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 9942.127436 secs 2024-04-26T21:21:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:00 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 9942.127436 secs 2024-04-26T21:21:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:00 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 9942.127436 secs 2024-04-26T21:21:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:01 smithi090 ceph-mon[29180]: pgmap v6035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9942.852444 secs 2024-04-26T21:21:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:01 smithi119 ceph-mon[32752]: pgmap v6035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9942.852444 secs 2024-04-26T21:21:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:01 smithi167 ceph-mon[32751]: pgmap v6035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9942.852444 secs 2024-04-26T21:21:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:03 smithi090 ceph-mon[29180]: pgmap v6036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:03.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.411+0000 7f770d7fa640 1 -- 172.21.15.90:0/4041546713 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f7710062d70 con 0x7f76f80a3dc0 2024-04-26T21:21:03.412 INFO:teuthology.orchestra.run.smithi090.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T18:11:58", "first": "2024-04-26T18:12:00", "last": "2024-04-26T18:35:00", "last_pruned": "2024-04-26T18:34:05", "created_count": 24, "pruned_count": 17, "active": true} 2024-04-26T21:21:03.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76f80a3dc0 msgr2=0x7f76f80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76f80a3dc0 0x7f76f80a6280 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7700009230 tx=0x7f77000058d0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 msgr2=0x7f77101a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f77101a4fb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f770800eb10 tx=0x7f770800c2d0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 shutdown_connections 2024-04-26T21:21:03.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f76f80a3dc0 0x7f76f80a6280 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7710106800 0x7f77101a4fb0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.416 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7710102980 0x7f771019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 --2- 172.21.15.90:0/4041546713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7710101fc0 0x7f7710199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 >> 172.21.15.90:0/4041546713 conn(0x7f7710077d50 msgr2=0x7f77101045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:03.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 shutdown_connections 2024-04-26T21:21:03.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.414+0000 7f7716493640 1 -- 172.21.15.90:0/4041546713 wait complete. 2024-04-26T21:21:03.432 DEBUG:teuthology.orchestra.run.smithi090:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T21:21:03.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 -- 172.21.15.90:0/926868119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5054108b30 msgr2=0x7f5054108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 --2- 172.21.15.90:0/926868119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5054108b30 0x7f5054108f10 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5044009900 tx=0x7f504402a480 comp rx=0 tx=0).stop 2024-04-26T21:21:03.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 -- 172.21.15.90:0/926868119 shutdown_connections 2024-04-26T21:21:03.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 --2- 172.21.15.90:0/926868119 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505406bb20 0x7f505410f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 --2- 172.21.15.90:0/926868119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f505406b160 0x7f505406b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 --2- 172.21.15.90:0/926868119 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5054108b30 0x7f5054108f10 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 -- 172.21.15.90:0/926868119 >> 172.21.15.90:0/926868119 conn(0x7f5054077630 msgr2=0x7f5054077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:03.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 -- 172.21.15.90:0/926868119 shutdown_connections 2024-04-26T21:21:03.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.544+0000 7f505ad97640 1 -- 172.21.15.90:0/926868119 wait complete. 2024-04-26T21:21:03.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 Processor -- start 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 -- start start 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505406b160 0x7f5054198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505406bb20 0x7f5054199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5054108b30 0x7f505419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f5054112c50 con 0x7f505406b160 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f5054112ad0 con 0x7f5054108b30 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f505ad97640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5054112dd0 con 0x7f505406bb20 2024-04-26T21:21:03.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f5058b33640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505406b160 0x7f5054198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f5058b33640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505406b160 0x7f5054198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43584/0 (socket says 172.21.15.90:43584) 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f5058b33640 1 -- 172.21.15.90:0/4008128361 learned_addr learned my addr 172.21.15.90:0/4008128361 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f5059334640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5054108b30 0x7f505419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.545+0000 7f5053fff640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505406bb20 0x7f5054199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5058b33640 1 -- 172.21.15.90:0/4008128361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505406bb20 msgr2=0x7f5054199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5058b33640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505406bb20 0x7f5054199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5058b33640 1 -- 172.21.15.90:0/4008128361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5054108b30 msgr2=0x7f505419d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5058b33640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5054108b30 0x7f505419d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5058b33640 1 -- 172.21.15.90:0/4008128361 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f505419dfc0 con 0x7f505406b160 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5058b33640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505406b160 0x7f5054198fd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f504402aa70 tx=0x7f5044003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5051ffb640 1 -- 172.21.15.90:0/4008128361 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504403f070 con 0x7f505406b160 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f505419e250 con 0x7f505406b160 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5051ffb640 1 -- 172.21.15.90:0/4008128361 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f504403bb20 con 0x7f505406b160 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f5051ffb640 1 -- 172.21.15.90:0/4008128361 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044035030 con 0x7f505406b160 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.546+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50541a5b40 con 0x7f505406b160 2024-04-26T21:21:03.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.547+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5054104040 con 0x7f505406b160 2024-04-26T21:21:03.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.548+0000 7f5051ffb640 1 -- 172.21.15.90:0/4008128361 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f5044042070 con 0x7f505406b160 2024-04-26T21:21:03.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.549+0000 7f5051ffb640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f503c0a3dc0 0x7f503c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.549+0000 7f5053fff640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f503c0a3dc0 0x7f503c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.549+0000 7f5051ffb640 1 -- 172.21.15.90:0/4008128361 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f50440f25e0 con 0x7f505406b160 2024-04-26T21:21:03.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.549+0000 7f5053fff640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f503c0a3dc0 0x7f503c0a6280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f50480095f0 tx=0x7f5048005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:03.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.552+0000 7f5051ffb640 1 -- 172.21.15.90:0/4008128361 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f50440bda20 con 0x7f505406b160 2024-04-26T21:21:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:03 smithi119 ceph-mon[32752]: pgmap v6036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:03 smithi167 ceph-mon[32751]: pgmap v6036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:03.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.754+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 --> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f5054100d80 con 0x7f503c0a3dc0 2024-04-26T21:21:03.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.762+0000 7f5051ffb640 1 -- 172.21.15.90:0/4008128361 <== mgr.24752 v2:172.21.15.90:6800/2946218715 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f5054100d80 con 0x7f503c0a3dc0 2024-04-26T21:21:03.764 INFO:teuthology.orchestra.run.smithi090.stdout:/ 1m 6m3h 2024-04-26T21:21:03.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.764+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f503c0a3dc0 msgr2=0x7f503c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.764+0000 7f505ad97640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f503c0a3dc0 0x7f503c0a6280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f50480095f0 tx=0x7f5048005f90 comp rx=0 tx=0).stop 2024-04-26T21:21:03.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.764+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505406b160 msgr2=0x7f5054198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.764+0000 7f505ad97640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505406b160 0x7f5054198fd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f504402aa70 tx=0x7f5044003c10 comp rx=0 tx=0).stop 2024-04-26T21:21:03.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.764+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 shutdown_connections 2024-04-26T21:21:03.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.765+0000 7f505ad97640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f503c0a3dc0 0x7f503c0a6280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.765+0000 7f505ad97640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5054108b30 0x7f505419d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.765+0000 7f505ad97640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f505406bb20 0x7f5054199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.765+0000 7f505ad97640 1 --2- 172.21.15.90:0/4008128361 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f505406b160 0x7f5054198fd0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.765+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 >> 172.21.15.90:0/4008128361 conn(0x7f5054077630 msgr2=0x7f5054100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:03.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.765+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 shutdown_connections 2024-04-26T21:21:03.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.765+0000 7f505ad97640 1 -- 172.21.15.90:0/4008128361 wait complete. 2024-04-26T21:21:03.776 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:21:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:21:03.479+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:21:03.776 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:21:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:21:03.479+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:21:03.776 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:21:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:21:03.479+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:21:03.776 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:21:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:21:03.479+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:21:03.776 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:21:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:21:03.479+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:21:03.777 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T21:21:03.796 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:21:03.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 -- 172.21.15.90:0/1779205556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 msgr2=0x7f6224101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 --2- 172.21.15.90:0/1779205556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224101c80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6214009930 tx=0x7f621402a420 comp rx=0 tx=0).stop 2024-04-26T21:21:03.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 -- 172.21.15.90:0/1779205556 shutdown_connections 2024-04-26T21:21:03.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 --2- 172.21.15.90:0/1779205556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62241021c0 0x7f622410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 --2- 172.21.15.90:0/1779205556 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224101c80 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 --2- 172.21.15.90:0/1779205556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6224107810 0x7f6224107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 -- 172.21.15.90:0/1779205556 >> 172.21.15.90:0/1779205556 conn(0x7f6224077bb0 msgr2=0x7f6224077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 -- 172.21.15.90:0/1779205556 shutdown_connections 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 -- 172.21.15.90:0/1779205556 wait complete. 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.896+0000 7f622a197640 1 Processor -- start 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f622a197640 1 -- start start 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f622a197640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f622a197640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62241021c0 0x7f6224199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f622a197640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6224107810 0x7f622419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f622a197640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111b20 con 0x7f6224101800 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f622a197640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f62241119a0 con 0x7f62241021c0 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f622a197640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111ca0 con 0x7f6224107810 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43600/0 (socket says 172.21.15.90:43600) 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 -- 172.21.15.90:0/1892711757 learned_addr learned my addr 172.21.15.90:0/1892711757 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f6222ffd640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62241021c0 0x7f6224199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f6223fff640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6224107810 0x7f622419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 -- 172.21.15.90:0/1892711757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6224107810 msgr2=0x7f622419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6224107810 0x7f622419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 -- 172.21.15.90:0/1892711757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62241021c0 msgr2=0x7f6224199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62241021c0 0x7f6224199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 -- 172.21.15.90:0/1892711757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622419e120 con 0x7f6224101800 2024-04-26T21:21:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f6223fff640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6224107810 0x7f622419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:21:03.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f62237fe640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224199130 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f621800c970 tx=0x7f621800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:03.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.897+0000 7f6220ff9640 1 -- 172.21.15.90:0/1892711757 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6218007bb0 con 0x7f6224101800 2024-04-26T21:21:03.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.898+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622419e410 con 0x7f6224101800 2024-04-26T21:21:03.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.898+0000 7f6220ff9640 1 -- 172.21.15.90:0/1892711757 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6218007d50 con 0x7f6224101800 2024-04-26T21:21:03.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.898+0000 7f6220ff9640 1 -- 172.21.15.90:0/1892711757 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621800f450 con 0x7f6224101800 2024-04-26T21:21:03.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.898+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241a5ca0 con 0x7f6224101800 2024-04-26T21:21:03.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.898+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6224102f40 con 0x7f6224101800 2024-04-26T21:21:03.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.900+0000 7f6220ff9640 1 -- 172.21.15.90:0/1892711757 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f6218004f00 con 0x7f6224101800 2024-04-26T21:21:03.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.900+0000 7f6220ff9640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f620c0a3d20 0x7f620c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:03.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.900+0000 7f6222ffd640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f620c0a3d20 0x7f620c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:03.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.900+0000 7f6220ff9640 1 -- 172.21.15.90:0/1892711757 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f62180d1760 con 0x7f6224101800 2024-04-26T21:21:03.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.901+0000 7f6222ffd640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f620c0a3d20 0x7f620c0a61e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6214031440 tx=0x7f6214005a20 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:03.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:03.905+0000 7f6220ff9640 1 -- 172.21.15.90:0/1892711757 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6218014030 con 0x7f6224101800 2024-04-26T21:21:04.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.084+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6224069ba0 con 0x7f6224101800 2024-04-26T21:21:04.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.086+0000 7f6220ff9640 1 -- 172.21.15.90:0/1892711757 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) ==== 74+0+30278 (secure 0 0 0) 0x7f621809cba0 con 0x7f6224101800 2024-04-26T21:21:04.087 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T21:21:04.088 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":183,"fsid":"b8878a18-03f6-11ef-bc93-c7b262605968","created":"2024-04-26T18:00:36.466392+0000","modified":"2024-04-26T18:35:07.565211+0000","last_up_change":"2024-04-26T18:07:46.388718+0000","last_in_change":"2024-04-26T18:07:26.634485+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:03:57.856561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:08:42.405005+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:08:46.009193+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":41,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7d909b27-dec0-46c5-adc0-0eb84d381e2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6803","nonce":1246374052}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6805","nonce":1246374052}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6809","nonce":1246374052}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":1246374052},{"type":"v1","addr":"172.21.15.90:6807","nonce":1246374052}]},"public_addr":"172.21.15.90:6803/1246374052","cluster_addr":"172.21.15.90:6805/1246374052","heartbeat_back_addr":"172.21.15.90:6809/1246374052","heartbeat_front_addr":"172.21.15.90:6807/1246374052","state":["exists","up"]},{"osd":1,"uuid":"9122fc04-ae29-4ed2-a91f-4dcb62208684","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6801","nonce":447582206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6803","nonce":447582206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6807","nonce":447582206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":447582206},{"type":"v1","addr":"172.21.15.119:6805","nonce":447582206}]},"public_addr":"172.21.15.119:6801/447582206","cluster_addr":"172.21.15.119:6803/447582206","heartbeat_back_addr":"172.21.15.119:6807/447582206","heartbeat_front_addr":"172.21.15.119:6805/447582206","state":["exists","up"]},{"osd":2,"uuid":"d283b32e-2f13-4fbf-a75b-957df3b02b70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6801","nonce":1709871603}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6803","nonce":1709871603}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6807","nonce":1709871603}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":1709871603},{"type":"v1","addr":"172.21.15.167:6805","nonce":1709871603}]},"public_addr":"172.21.15.167:6801/1709871603","cluster_addr":"172.21.15.167:6803/1709871603","heartbeat_back_addr":"172.21.15.167:6807/1709871603","heartbeat_front_addr":"172.21.15.167:6805/1709871603","state":["exists","up"]},{"osd":3,"uuid":"8548cc2a-05b4-4e96-9e1c-22b26116170d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6811","nonce":2988562238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6813","nonce":2988562238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6817","nonce":2988562238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2988562238},{"type":"v1","addr":"172.21.15.90:6815","nonce":2988562238}]},"public_addr":"172.21.15.90:6811/2988562238","cluster_addr":"172.21.15.90:6813/2988562238","heartbeat_back_addr":"172.21.15.90:6817/2988562238","heartbeat_front_addr":"172.21.15.90:6815/2988562238","state":["exists","up"]},{"osd":4,"uuid":"11b6b750-1a3f-4a50-bacf-145c1586412b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6809","nonce":2105013830}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6811","nonce":2105013830}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6815","nonce":2105013830}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2105013830},{"type":"v1","addr":"172.21.15.119:6813","nonce":2105013830}]},"public_addr":"172.21.15.119:6809/2105013830","cluster_addr":"172.21.15.119:6811/2105013830","heartbeat_back_addr":"172.21.15.119:6815/2105013830","heartbeat_front_addr":"172.21.15.119:6813/2105013830","state":["exists","up"]},{"osd":5,"uuid":"592a3d64-facb-42f0-928d-ab7326e69642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6809","nonce":2447709216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6811","nonce":2447709216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6815","nonce":2447709216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":2447709216},{"type":"v1","addr":"172.21.15.167:6813","nonce":2447709216}]},"public_addr":"172.21.15.167:6809/2447709216","cluster_addr":"172.21.15.167:6811/2447709216","heartbeat_back_addr":"172.21.15.167:6815/2447709216","heartbeat_front_addr":"172.21.15.167:6813/2447709216","state":["exists","up"]},{"osd":6,"uuid":"24eb3641-a329-4a5f-b672-06b21c53894b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6819","nonce":2940618965}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6821","nonce":2940618965}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6825","nonce":2940618965}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":2940618965},{"type":"v1","addr":"172.21.15.90:6823","nonce":2940618965}]},"public_addr":"172.21.15.90:6819/2940618965","cluster_addr":"172.21.15.90:6821/2940618965","heartbeat_back_addr":"172.21.15.90:6825/2940618965","heartbeat_front_addr":"172.21.15.90:6823/2940618965","state":["exists","up"]},{"osd":7,"uuid":"6aea2198-ac95-4f94-9953-dd093977e34c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6817","nonce":146780393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6819","nonce":146780393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6823","nonce":146780393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":146780393},{"type":"v1","addr":"172.21.15.119:6821","nonce":146780393}]},"public_addr":"172.21.15.119:6817/146780393","cluster_addr":"172.21.15.119:6819/146780393","heartbeat_back_addr":"172.21.15.119:6823/146780393","heartbeat_front_addr":"172.21.15.119:6821/146780393","state":["exists","up"]},{"osd":8,"uuid":"0b9aca44-5296-4726-b01f-80153f4c806d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6817","nonce":1376531844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6819","nonce":1376531844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6823","nonce":1376531844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":1376531844},{"type":"v1","addr":"172.21.15.167:6821","nonce":1376531844}]},"public_addr":"172.21.15.167:6817/1376531844","cluster_addr":"172.21.15.167:6819/1376531844","heartbeat_back_addr":"172.21.15.167:6823/1376531844","heartbeat_front_addr":"172.21.15.167:6821/1376531844","state":["exists","up"]},{"osd":9,"uuid":"d262c94d-6bc8-4107-851a-36b1db393708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6827","nonce":1702789383}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6829","nonce":1702789383}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6832","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6833","nonce":1702789383}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6830","nonce":1702789383},{"type":"v1","addr":"172.21.15.90:6831","nonce":1702789383}]},"public_addr":"172.21.15.90:6827/1702789383","cluster_addr":"172.21.15.90:6829/1702789383","heartbeat_back_addr":"172.21.15.90:6833/1702789383","heartbeat_front_addr":"172.21.15.90:6831/1702789383","state":["exists","up"]},{"osd":10,"uuid":"8873f617-32db-4f50-b47e-b1d3f42eaf4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6825","nonce":249896871}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6827","nonce":249896871}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6831","nonce":249896871}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":249896871},{"type":"v1","addr":"172.21.15.119:6829","nonce":249896871}]},"public_addr":"172.21.15.119:6825/249896871","cluster_addr":"172.21.15.119:6827/249896871","heartbeat_back_addr":"172.21.15.119:6831/249896871","heartbeat_front_addr":"172.21.15.119:6829/249896871","state":["exists","up"]},{"osd":11,"uuid":"123cd211-4456-4b3d-8ad3-bd47cdda36a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":82,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6825","nonce":3743670962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6827","nonce":3743670962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6831","nonce":3743670962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3743670962},{"type":"v1","addr":"172.21.15.167:6829","nonce":3743670962}]},"public_addr":"172.21.15.167:6825/3743670962","cluster_addr":"172.21.15.167:6827/3743670962","heartbeat_back_addr":"172.21.15.167:6831/3743670962","heartbeat_front_addr":"172.21.15.167:6829/3743670962","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:05.892264+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:30.538745+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:03:55.650379+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:20.419281+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:04:45.463976+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:10.636654+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:05:36.214103+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:01.748820+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:27.026006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:06:53.166990+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:18.948480+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:07:45.232347+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6837/2196548810":"2024-04-27T18:34:42.471750+0000","172.21.15.90:6836/2196548810":"2024-04-27T18:34:42.471750+0000","172.21.15.90:6836/2850537869":"2024-04-27T18:33:56.077213+0000","172.21.15.90:6840/2568313147":"2024-04-27T18:33:33.657527+0000","172.21.15.90:6839/2261571250":"2024-04-27T18:34:42.464232+0000","172.21.15.90:6837/1881511598":"2024-04-27T18:33:23.673790+0000","172.21.15.90:6841/349095443":"2024-04-27T18:32:26.580352+0000","172.21.15.90:6840/349095443":"2024-04-27T18:32:26.580352+0000","172.21.15.90:6836/3651305274":"2024-04-27T18:32:26.161941+0000","172.21.15.90:6839/2673484677":"2024-04-27T18:32:04.394949+0000","172.21.15.90:6838/2673484677":"2024-04-27T18:32:04.394949+0000","172.21.15.90:6835/4072394290":"2024-04-27T18:31:52.651357+0000","172.21.15.90:6834/4072394290":"2024-04-27T18:31:52.651357+0000","172.21.15.90:6837/2470728764":"2024-04-27T18:31:29.658307+0000","172.21.15.90:6835/2911456063":"2024-04-27T18:31:08.868881+0000","172.21.15.90:6838/2261571250":"2024-04-27T18:34:42.464232+0000","172.21.15.119:6833/1677033493":"2024-04-27T18:30:17.628297+0000","172.21.15.119:6832/1677033493":"2024-04-27T18:30:17.628297+0000","172.21.15.167:6833/37569032":"2024-04-27T18:29:51.144130+0000","172.21.15.90:6841/3724478516":"2024-04-27T18:29:08.636186+0000","172.21.15.90:6840/3724478516":"2024-04-27T18:29:08.636186+0000","172.21.15.119:6833/117919007":"2024-04-27T18:29:02.817125+0000","172.21.15.167:6839/874199594":"2024-04-27T18:19:39.104871+0000","172.21.15.119:6836/3690677501":"2024-04-27T18:21:48.374193+0000","172.21.15.90:6841/2568313147":"2024-04-27T18:33:33.657527+0000","172.21.15.90:0/484221164":"2024-04-27T18:11:57.154935+0000","172.21.15.167:6839/4026544385":"2024-04-27T18:21:27.335081+0000","172.21.15.167:6833/4236765679":"2024-04-27T18:25:55.586295+0000","172.21.15.167:6839/2850927327":"2024-04-27T18:22:12.517491+0000","172.21.15.119:6832/137310630":"2024-04-27T18:24:38.623226+0000","172.21.15.119:6837/845543419":"2024-04-27T18:20:52.104457+0000","172.21.15.90:6834/2911456063":"2024-04-27T18:31:08.868881+0000","172.21.15.167:6838/4026544385":"2024-04-27T18:21:27.335081+0000","172.21.15.167:6832/4236765679":"2024-04-27T18:25:55.586295+0000","172.21.15.167:6838/1142719310":"2024-04-27T18:20:30.686908+0000","172.21.15.119:6832/4202449285":"2024-04-27T18:27:46.134304+0000","172.21.15.119:6837/3534880230":"2024-04-27T18:19:03.863263+0000","172.21.15.167:6836/3885172962":"2024-04-27T18:28:32.603623+0000","172.21.15.90:0/1492327080":"2024-04-27T18:01:31.766717+0000","172.21.15.119:6836/845543419":"2024-04-27T18:20:52.104457+0000","172.21.15.90:6840/720001018":"2024-04-27T18:30:57.638207+0000","172.21.15.90:6839/862440561":"2024-04-27T18:30:12.187956+0000","172.21.15.167:6838/4229007587":"2024-04-27T18:18:32.371481+0000","172.21.15.90:6841/720001018":"2024-04-27T18:30:57.638207+0000","172.21.15.119:6835/114254184":"2024-04-27T18:27:14.404770+0000","172.21.15.167:6838/874199594":"2024-04-27T18:19:39.104871+0000","172.21.15.119:6836/3534880230":"2024-04-27T18:19:03.863263+0000","172.21.15.119:6834/3561315595":"2024-04-27T18:28:09.257409+0000","172.21.15.167:6839/4229007587":"2024-04-27T18:18:32.371481+0000","172.21.15.119:6832/117919007":"2024-04-27T18:29:02.817125+0000","172.21.15.119:6834/2177592042":"2024-04-27T18:24:02.157556+0000","172.21.15.90:6836/1335338695":"2024-04-27T18:30:40.746838+0000","172.21.15.90:6800/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.119:6833/137310630":"2024-04-27T18:24:38.623226+0000","172.21.15.119:6836/1113931480":"2024-04-27T18:23:37.536968+0000","172.21.15.119:6834/114254184":"2024-04-27T18:27:14.404770+0000","172.21.15.167:6837/1187834156":"2024-04-27T18:24:38.071710+0000","172.21.15.90:6834/641065147":"2024-04-27T18:33:55.379573+0000","172.21.15.90:0/2846931038":"2024-04-27T18:01:31.766717+0000","172.21.15.90:6836/2470728764":"2024-04-27T18:31:29.658307+0000","172.21.15.90:6838/862440561":"2024-04-27T18:30:12.187956+0000","172.21.15.90:0/227996061":"2024-04-27T18:01:03.883791+0000","172.21.15.167:6837/3885172962":"2024-04-27T18:28:32.603623+0000","172.21.15.119:6836/4116054973":"2024-04-27T18:22:45.044763+0000","172.21.15.119:6833/4202449285":"2024-04-27T18:27:46.134304+0000","172.21.15.90:0/295280840":"2024-04-27T18:01:03.883791+0000","172.21.15.90:0/1501521801":"2024-04-27T18:11:57.154935+0000","172.21.15.167:6832/2550962231":"2024-04-27T18:26:53.240503+0000","172.21.15.90:6801/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.90:6837/2797511346":"2024-04-27T18:35:07.565181+0000","172.21.15.119:6837/4116054973":"2024-04-27T18:22:45.044763+0000","172.21.15.90:6836/1881511598":"2024-04-27T18:33:23.673790+0000","172.21.15.90:6800/3660618461":"2024-04-27T18:11:57.154935+0000","172.21.15.119:6837/3690677501":"2024-04-27T18:21:48.374193+0000","172.21.15.167:6832/37569032":"2024-04-27T18:29:51.144130+0000","172.21.15.90:6839/2946680060":"2024-04-27T18:29:29.402623+0000","172.21.15.119:6837/229859293":"2024-04-27T18:19:59.025433+0000","172.21.15.167:6837/4250199742":"2024-04-27T18:26:31.337524+0000","172.21.15.90:6840/3362405108":"2024-04-27T18:29:51.138558+0000","172.21.15.90:6838/2946680060":"2024-04-27T18:29:29.402623+0000","172.21.15.90:6801/1868991041":"2024-04-27T18:01:31.766717+0000","172.21.15.90:0/651646115":"2024-04-27T18:01:31.766717+0000","172.21.15.119:6834/2611460544":"2024-04-27T18:23:06.054300+0000","172.21.15.90:6835/641065147":"2024-04-27T18:33:55.379573+0000","172.21.15.167:6832/766230459":"2024-04-27T18:24:03.896947+0000","172.21.15.167:6836/200145753":"2024-04-27T18:23:41.447349+0000","172.21.15.119:6835/2611460544":"2024-04-27T18:23:06.054300+0000","172.21.15.167:6833/766230459":"2024-04-27T18:24:03.896947+0000","172.21.15.90:6838/1000850976":"2024-04-27T18:33:02.666963+0000","172.21.15.90:6841/3362405108":"2024-04-27T18:29:51.138558+0000","172.21.15.119:6837/1113931480":"2024-04-27T18:23:37.536968+0000","172.21.15.167:6837/200145753":"2024-04-27T18:23:41.447349+0000","172.21.15.119:6835/2177592042":"2024-04-27T18:24:02.157556+0000","172.21.15.167:6833/3757608812":"2024-04-27T18:24:59.368511+0000","172.21.15.119:6836/229859293":"2024-04-27T18:19:59.025433+0000","172.21.15.167:6832/3757608812":"2024-04-27T18:24:59.368511+0000","172.21.15.119:6832/1237239761":"2024-04-27T18:25:55.597886+0000","172.21.15.119:6835/958836808":"2024-04-27T18:25:23.814056+0000","172.21.15.90:6834/1044509479":"2024-04-27T18:32:57.665121+0000","172.21.15.167:6836/3309576493":"2024-04-27T18:25:34.885367+0000","172.21.15.167:6837/3309576493":"2024-04-27T18:25:34.885367+0000","172.21.15.167:6839/1142719310":"2024-04-27T18:20:30.686908+0000","172.21.15.90:0/794009814":"2024-04-27T18:01:03.883791+0000","172.21.15.119:6833/1237239761":"2024-04-27T18:25:55.597886+0000","172.21.15.90:6801/3660618461":"2024-04-27T18:11:57.154935+0000","172.21.15.119:6834/3224484301":"2024-04-27T18:26:30.320937+0000","172.21.15.119:6835/3224484301":"2024-04-27T18:26:30.320937+0000","172.21.15.90:6839/1000850976":"2024-04-27T18:33:02.666963+0000","172.21.15.167:6836/4250199742":"2024-04-27T18:26:31.337524+0000","172.21.15.119:6832/925527879":"2024-04-27T18:26:54.125475+0000","172.21.15.90:6835/1044509479":"2024-04-27T18:32:57.665121+0000","172.21.15.90:0/1315801302":"2024-04-27T18:11:57.154935+0000","172.21.15.167:6832/2849164668":"2024-04-27T18:28:07.616546+0000","172.21.15.90:6837/3651305274":"2024-04-27T18:32:26.161941+0000","172.21.15.90:6800/4035087696":"2024-04-27T18:01:03.883791+0000","172.21.15.119:6833/925527879":"2024-04-27T18:26:54.125475+0000","172.21.15.119:6834/958836808":"2024-04-27T18:25:23.814056+0000","172.21.15.167:6836/1881734029":"2024-04-27T18:27:37.159916+0000","172.21.15.90:6836/2797511346":"2024-04-27T18:35:07.565181+0000","172.21.15.90:0/2718220414":"2024-04-27T18:11:57.154935+0000","172.21.15.167:6833/2550962231":"2024-04-27T18:26:53.240503+0000","172.21.15.167:6837/1881734029":"2024-04-27T18:27:37.159916+0000","172.21.15.90:6837/2850537869":"2024-04-27T18:33:56.077213+0000","172.21.15.90:6837/1335338695":"2024-04-27T18:30:40.746838+0000","172.21.15.167:6838/2850927327":"2024-04-27T18:22:12.517491+0000","172.21.15.167:6833/2849164668":"2024-04-27T18:28:07.616546+0000","172.21.15.167:6836/1187834156":"2024-04-27T18:24:38.071710+0000","172.21.15.119:6835/3561315595":"2024-04-27T18:28:09.257409+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:21:04.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f620c0a3d20 msgr2=0x7f620c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f620c0a3d20 0x7f620c0a61e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6214031440 tx=0x7f6214005a20 comp rx=0 tx=0).stop 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 msgr2=0x7f6224199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224199130 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f621800c970 tx=0x7f621800ce30 comp rx=0 tx=0).stop 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 shutdown_connections 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f620c0a3d20 0x7f620c0a61e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6224107810 0x7f622419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62241021c0 0x7f6224199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 --2- 172.21.15.90:0/1892711757 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6224101800 0x7f6224199130 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 >> 172.21.15.90:0/1892711757 conn(0x7f6224077bb0 msgr2=0x7f622406b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:04.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 shutdown_connections 2024-04-26T21:21:04.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.088+0000 7f622a197640 1 -- 172.21.15.90:0/1892711757 wait complete. 2024-04-26T21:21:04.101 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T18:03:57.856561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T18:08:42.405005+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T18:08:46.009193+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '179', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 41, 'snap_epoch': 179, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:21:04.101 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:21:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 -- 172.21.15.90:0/2213397510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810108960 msgr2=0x7fc810108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 --2- 172.21.15.90:0/2213397510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810108960 0x7fc810108d40 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc800009930 tx=0x7fc80002a420 comp rx=0 tx=0).stop 2024-04-26T21:21:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 -- 172.21.15.90:0/2213397510 shutdown_connections 2024-04-26T21:21:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 --2- 172.21.15.90:0/2213397510 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc810103310 0x7fc81010f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 --2- 172.21.15.90:0/2213397510 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc810102950 0x7fc810102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.206 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 --2- 172.21.15.90:0/2213397510 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810108960 0x7fc810108d40 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 -- 172.21.15.90:0/2213397510 >> 172.21.15.90:0/2213397510 conn(0x7fc8100fe440 msgr2=0x7fc810100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 -- 172.21.15.90:0/2213397510 shutdown_connections 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.201+0000 7fc8149f7640 1 -- 172.21.15.90:0/2213397510 wait complete. 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 Processor -- start 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 -- start start 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810102950 0x7fc8101991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc810103310 0x7fc8101996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc810108960 0x7fc81019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7fc810112c20 con 0x7fc810102950 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7fc810112aa0 con 0x7fc810103310 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc8149f7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc810112da0 con 0x7fc810108960 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810102950 0x7fc8101991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810102950 0x7fc8101991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43604/0 (socket says 172.21.15.90:43604) 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 -- 172.21.15.90:0/1516479168 learned_addr learned my addr 172.21.15.90:0/1516479168 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80dd9b640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc810103310 0x7fc8101996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80ed9d640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc810108960 0x7fc81019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 -- 172.21.15.90:0/1516479168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc810108960 msgr2=0x7fc81019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc810108960 0x7fc81019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 -- 172.21.15.90:0/1516479168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc810103310 msgr2=0x7fc8101996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc810103310 0x7fc8101996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80e59c640 1 -- 172.21.15.90:0/1516479168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81019e190 con 0x7fc810102950 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.202+0000 7fc80ed9d640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc810108960 0x7fc81019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.203+0000 7fc80e59c640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810102950 0x7fc8101991a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc800005940 tx=0x7fc8000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:04.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.203+0000 7fc7ff7fe640 1 -- 172.21.15.90:0/1516479168 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80003f030 con 0x7fc810102950 2024-04-26T21:21:04.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.203+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81019e420 con 0x7fc810102950 2024-04-26T21:21:04.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.203+0000 7fc7ff7fe640 1 -- 172.21.15.90:0/1516479168 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc800003cb0 con 0x7fc810102950 2024-04-26T21:21:04.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.203+0000 7fc7ff7fe640 1 -- 172.21.15.90:0/1516479168 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800035070 con 0x7fc810102950 2024-04-26T21:21:04.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.203+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8101a5d10 con 0x7fc810102950 2024-04-26T21:21:04.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.204+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc810069ba0 con 0x7fc810102950 2024-04-26T21:21:04.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.205+0000 7fc7ff7fe640 1 -- 172.21.15.90:0/1516479168 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7fc800042030 con 0x7fc810102950 2024-04-26T21:21:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.206+0000 7fc7ff7fe640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc7f80a3d20 0x7fc7f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.206+0000 7fc80dd9b640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc7f80a3d20 0x7fc7f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.206+0000 7fc7ff7fe640 1 -- 172.21.15.90:0/1516479168 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7fc8000f24f0 con 0x7fc810102950 2024-04-26T21:21:04.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.206+0000 7fc80dd9b640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc7f80a3d20 0x7fc7f80a61e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc810100b20 tx=0x7fc804005f90 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:04.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.210+0000 7fc7ff7fe640 1 -- 172.21.15.90:0/1516479168 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc8000bd8d0 con 0x7fc810102950 2024-04-26T21:21:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:04 smithi090 ceph-mon[29180]: from='client.17259 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:04 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1892711757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:21:04.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.387+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc81019a4f0 con 0x7fc810102950 2024-04-26T21:21:04.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.388+0000 7fc7ff7fe640 1 -- 172.21.15.90:0/1516479168 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) ==== 93+0+10 (secure 0 0 0) 0x7fc8000bd1d0 con 0x7fc810102950 2024-04-26T21:21:04.390 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-26T21:21:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc7f80a3d20 msgr2=0x7fc7f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc7f80a3d20 0x7fc7f80a61e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc810100b20 tx=0x7fc804005f90 comp rx=0 tx=0).stop 2024-04-26T21:21:04.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810102950 msgr2=0x7fc8101991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810102950 0x7fc8101991a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc800005940 tx=0x7fc8000042f0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 shutdown_connections 2024-04-26T21:21:04.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7fc7f80a3d20 0x7fc7f80a61e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc810108960 0x7fc81019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc810103310 0x7fc8101996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 --2- 172.21.15.90:0/1516479168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc810102950 0x7fc8101991a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 >> 172.21.15.90:0/1516479168 conn(0x7fc8100fe440 msgr2=0x7fc8100fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:04.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 shutdown_connections 2024-04-26T21:21:04.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.390+0000 7fc8149f7640 1 -- 172.21.15.90:0/1516479168 wait complete. 2024-04-26T21:21:04.402 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:21:04.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.502+0000 7f7eafc93640 1 -- 172.21.15.90:0/1480886274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea81044d0 msgr2=0x7f7ea810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 --2- 172.21.15.90:0/1480886274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea81044d0 0x7f7ea810c9b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009900 tx=0x7f7e9802a510 comp rx=0 tx=0).stop 2024-04-26T21:21:04.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- 172.21.15.90:0/1480886274 shutdown_connections 2024-04-26T21:21:04.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 --2- 172.21.15.90:0/1480886274 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea810cef0 0x7f7ea810f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 --2- 172.21.15.90:0/1480886274 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea81044d0 0x7f7ea810c9b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 --2- 172.21.15.90:0/1480886274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ea8103bb0 0x7f7ea8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- 172.21.15.90:0/1480886274 >> 172.21.15.90:0/1480886274 conn(0x7f7ea8078930 msgr2=0x7f7ea8078d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- 172.21.15.90:0/1480886274 shutdown_connections 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- 172.21.15.90:0/1480886274 wait complete. 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 Processor -- start 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- start start 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea8103bb0 0x7f7ea8199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea81044d0 0x7f7ea8199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ea810cef0 0x7f7ea819daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8112be0 con 0x7f7ea8103bb0 2024-04-26T21:21:04.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8112a60 con 0x7f7ea810cef0 2024-04-26T21:21:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.503+0000 7f7eafc93640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8112d60 con 0x7f7ea81044d0 2024-04-26T21:21:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea8103bb0 0x7f7ea8199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea8103bb0 0x7f7ea8199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43626/0 (socket says 172.21.15.90:43626) 2024-04-26T21:21:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 -- 172.21.15.90:0/3482139897 learned_addr learned my addr 172.21.15.90:0/3482139897 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:21:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eae230640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ea810cef0 0x7f7ea819daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 -- 172.21.15.90:0/3482139897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea81044d0 msgr2=0x7f7ea8199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea81044d0 0x7f7ea8199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 -- 172.21.15.90:0/3482139897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ea810cef0 msgr2=0x7f7ea819daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ea810cef0 0x7f7ea819daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 -- 172.21.15.90:0/3482139897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea819e220 con 0x7f7ea8103bb0 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eada2f640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea8103bb0 0x7f7ea8199230 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400e9f0 tx=0x7f7ea400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7e9effd640 1 -- 172.21.15.90:0/3482139897 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400cd80 con 0x7f7ea8103bb0 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7e9effd640 1 -- 172.21.15.90:0/3482139897 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ea40108e0 con 0x7f7ea8103bb0 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7e9effd640 1 -- 172.21.15.90:0/3482139897 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4018980 con 0x7f7ea8103bb0 2024-04-26T21:21:04.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea819e510 con 0x7f7ea8103bb0 2024-04-26T21:21:04.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.504+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81a5da0 con 0x7f7ea8103bb0 2024-04-26T21:21:04.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.505+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea8069750 con 0x7f7ea8103bb0 2024-04-26T21:21:04.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.506+0000 7f7e9effd640 1 -- 172.21.15.90:0/3482139897 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f7ea4010430 con 0x7f7ea8103bb0 2024-04-26T21:21:04.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.507+0000 7f7e9effd640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7e940a3d20 0x7f7e940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.507+0000 7f7ead22e640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7e940a3d20 0x7f7e940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.507+0000 7f7e9effd640 1 -- 172.21.15.90:0/3482139897 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f7ea40d22a0 con 0x7f7ea8103bb0 2024-04-26T21:21:04.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.507+0000 7f7ead22e640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7e940a3d20 0x7f7e940a61e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7e980096c0 tx=0x7f7e98031400 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:04.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.511+0000 7f7e9effd640 1 -- 172.21.15.90:0/3482139897 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ea409d6e0 con 0x7f7ea8103bb0 2024-04-26T21:21:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:04 smithi119 ceph-mon[32752]: from='client.17259 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:04 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1892711757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:21:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:04 smithi167 ceph-mon[32751]: from='client.17259 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T21:21:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:04 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1892711757' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T21:21:04.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.687+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ea81080a0 con 0x7f7ea8103bb0 2024-04-26T21:21:04.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.687+0000 7f7e9effd640 1 -- 172.21.15.90:0/3482139897 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v183) ==== 104+0+11 (secure 0 0 0) 0x7f7ea409cfe0 con 0x7f7ea8103bb0 2024-04-26T21:21:04.692 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T21:21:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7e940a3d20 msgr2=0x7f7e940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7e940a3d20 0x7f7e940a61e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7e980096c0 tx=0x7f7e98031400 comp rx=0 tx=0).stop 2024-04-26T21:21:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea8103bb0 msgr2=0x7f7ea8199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea8103bb0 0x7f7ea8199230 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7ea400e9f0 tx=0x7f7ea400eeb0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 shutdown_connections 2024-04-26T21:21:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7e940a3d20 0x7f7e940a61e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ea810cef0 0x7f7ea819daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ea81044d0 0x7f7ea8199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 --2- 172.21.15.90:0/3482139897 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7ea8103bb0 0x7f7ea8199230 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 >> 172.21.15.90:0/3482139897 conn(0x7f7ea8078930 msgr2=0x7f7ea80ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:04.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 shutdown_connections 2024-04-26T21:21:04.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.689+0000 7f7eafc93640 1 -- 172.21.15.90:0/3482139897 wait complete. 2024-04-26T21:21:04.701 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:21:04.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 -- 172.21.15.90:0/3341121286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 msgr2=0x7efc54107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 --2- 172.21.15.90:0/3341121286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc54107bf0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7efc50009930 tx=0x7efc5002a420 comp rx=0 tx=0).stop 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 -- 172.21.15.90:0/3341121286 shutdown_connections 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 --2- 172.21.15.90:0/3341121286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc541021c0 0x7efc5410e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 --2- 172.21.15.90:0/3341121286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc54101800 0x7efc54101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 --2- 172.21.15.90:0/3341121286 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc54107bf0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 -- 172.21.15.90:0/3341121286 >> 172.21.15.90:0/3341121286 conn(0x7efc54078930 msgr2=0x7efc54078d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 -- 172.21.15.90:0/3341121286 shutdown_connections 2024-04-26T21:21:04.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 -- 172.21.15.90:0/3341121286 wait complete. 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 Processor -- start 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 -- start start 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc54101800 0x7efc54199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc541021c0 0x7efc5419a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc541a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.801+0000 7efc5b87b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7efc54111ad0 con 0x7efc54107810 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc5b87b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7efc54111950 con 0x7efc54101800 2024-04-26T21:21:04.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc5b87b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efc54111c50 con 0x7efc541021c0 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc541a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc541a5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43648/0 (socket says 172.21.15.90:43648) 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 -- 172.21.15.90:0/1146536793 learned_addr learned my addr 172.21.15.90:0/1146536793 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59617640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc54101800 0x7efc54199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 -- 172.21.15.90:0/1146536793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc541021c0 msgr2=0x7efc5419a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc58e16640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc541021c0 0x7efc5419a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc541021c0 0x7efc5419a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 -- 172.21.15.90:0/1146536793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc54101800 msgr2=0x7efc54199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc54101800 0x7efc54199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 -- 172.21.15.90:0/1146536793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc541a5590 con 0x7efc54107810 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59617640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc54101800 0x7efc54199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc59e18640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc541a5050 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00eb10 tx=0x7efc4c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc4a7fc640 1 -- 172.21.15.90:0/1146536793 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c004b20 con 0x7efc54107810 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc4a7fc640 1 -- 172.21.15.90:0/1146536793 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc4c004cc0 con 0x7efc54107810 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc541a5760 con 0x7efc54107810 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc4a7fc640 1 -- 172.21.15.90:0/1146536793 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c010670 con 0x7efc54107810 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.802+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc541a5bd0 con 0x7efc54107810 2024-04-26T21:21:04.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.803+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc54069750 con 0x7efc54107810 2024-04-26T21:21:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.804+0000 7efc4a7fc640 1 -- 172.21.15.90:0/1146536793 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7efc4c005020 con 0x7efc54107810 2024-04-26T21:21:04.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.805+0000 7efc4a7fc640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efc400a3d20 0x7efc400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.805+0000 7efc59617640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efc400a3d20 0x7efc400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.805+0000 7efc4a7fc640 1 -- 172.21.15.90:0/1146536793 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7efc4c0d2010 con 0x7efc54107810 2024-04-26T21:21:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.805+0000 7efc59617640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efc400a3d20 0x7efc400a61e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc50009900 tx=0x7efc50000bc0 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:04.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.809+0000 7efc4a7fc640 1 -- 172.21.15.90:0/1146536793 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc4c09d450 con 0x7efc54107810 2024-04-26T21:21:04.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.989+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efc54109d70 con 0x7efc54107810 2024-04-26T21:21:04.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.989+0000 7efc4a7fc640 1 -- 172.21.15.90:0/1146536793 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v183) ==== 100+0+11 (secure 0 0 0) 0x7efc4c09cd50 con 0x7efc54107810 2024-04-26T21:21:04.991 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 64 2024-04-26T21:21:04.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efc400a3d20 msgr2=0x7efc400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efc400a3d20 0x7efc400a61e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7efc50009900 tx=0x7efc50000bc0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 msgr2=0x7efc541a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:04.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc541a5050 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00eb10 tx=0x7efc4c00c2d0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 shutdown_connections 2024-04-26T21:21:04.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7efc400a3d20 0x7efc400a61e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efc54107810 0x7efc541a5050 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efc541021c0 0x7efc5419a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 --2- 172.21.15.90:0/1146536793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc54101800 0x7efc54199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 >> 172.21.15.90:0/1146536793 conn(0x7efc54078930 msgr2=0x7efc54104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 shutdown_connections 2024-04-26T21:21:04.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:04.991+0000 7efc5b87b640 1 -- 172.21.15.90:0/1146536793 wait complete. 2024-04-26T21:21:05.004 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T21:21:05.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 -- 172.21.15.90:0/3667100708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc1021c0 msgr2=0x7f7dfc10e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:05.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 --2- 172.21.15.90:0/3667100708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc1021c0 0x7f7dfc10e6d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009cd0 tx=0x7f7dec02a8e0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.101 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 -- 172.21.15.90:0/3667100708 shutdown_connections 2024-04-26T21:21:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 --2- 172.21.15.90:0/3667100708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc1021c0 0x7f7dfc10e6d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 --2- 172.21.15.90:0/3667100708 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dfc101800 0x7f7dfc101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 --2- 172.21.15.90:0/3667100708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc107810 0x7f7dfc107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 -- 172.21.15.90:0/3667100708 >> 172.21.15.90:0/3667100708 conn(0x7f7dfc077bb0 msgr2=0x7f7dfc077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 -- 172.21.15.90:0/3667100708 shutdown_connections 2024-04-26T21:21:05.102 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.100+0000 7f7e00924640 1 -- 172.21.15.90:0/3667100708 wait complete. 2024-04-26T21:21:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 Processor -- start 2024-04-26T21:21:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 -- start start 2024-04-26T21:21:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc101800 0x7f7dfc199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dfc1021c0 0x7f7dfc199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc107810 0x7f7dfc19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc111ad0 con 0x7f7dfc101800 2024-04-26T21:21:05.105 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc111950 con 0x7f7dfc107810 2024-04-26T21:21:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7e00924640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc111c50 con 0x7f7dfc1021c0 2024-04-26T21:21:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc101800 0x7f7dfc199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc101800 0x7f7dfc199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43668/0 (socket says 172.21.15.90:43668) 2024-04-26T21:21:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 -- 172.21.15.90:0/2029655272 learned_addr learned my addr 172.21.15.90:0/2029655272 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:21:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7df9d9b640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dfc1021c0 0x7f7dfc199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfad9d640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc107810 0x7f7dfc19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:05.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 -- 172.21.15.90:0/2029655272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dfc1021c0 msgr2=0x7f7dfc199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dfc1021c0 0x7f7dfc199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 -- 172.21.15.90:0/2029655272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc107810 msgr2=0x7f7dfc19d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc107810 0x7f7dfc19d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 -- 172.21.15.90:0/2029655272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc19e110 con 0x7f7dfc101800 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfad9d640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc107810 0x7f7dfc19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.101+0000 7f7dfa59c640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc101800 0x7f7dfc199120 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7df400ea60 tx=0x7f7df400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.102+0000 7f7df37fe640 1 -- 172.21.15.90:0/2029655272 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df400cde0 con 0x7f7dfc101800 2024-04-26T21:21:05.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.102+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc19e400 con 0x7f7dfc101800 2024-04-26T21:21:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.102+0000 7f7df37fe640 1 -- 172.21.15.90:0/2029655272 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7df40035f0 con 0x7f7dfc101800 2024-04-26T21:21:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.102+0000 7f7df37fe640 1 -- 172.21.15.90:0/2029655272 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4010690 con 0x7f7dfc101800 2024-04-26T21:21:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.102+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc1a5c90 con 0x7f7dfc101800 2024-04-26T21:21:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.103+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc069ba0 con 0x7f7dfc101800 2024-04-26T21:21:05.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.104+0000 7f7df37fe640 1 -- 172.21.15.90:0/2029655272 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f7df40037a0 con 0x7f7dfc101800 2024-04-26T21:21:05.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.104+0000 7f7df37fe640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7de80a3d20 0x7f7de80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:05.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.104+0000 7f7df9d9b640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7de80a3d20 0x7f7de80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:05.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.104+0000 7f7df37fe640 1 -- 172.21.15.90:0/2029655272 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f7df40d1cc0 con 0x7f7dfc101800 2024-04-26T21:21:05.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.105+0000 7f7df9d9b640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7de80a3d20 0x7f7de80a61e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc06c0b0 tx=0x7f7de400a250 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:05.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.109+0000 7f7df37fe640 1 -- 172.21.15.90:0/2029655272 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7df409d100 con 0x7f7dfc101800 2024-04-26T21:21:05.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.334+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7dfc19a470 con 0x7f7dfc101800 2024-04-26T21:21:05.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.342+0000 7f7df37fe640 1 -- 172.21.15.90:0/2029655272 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+746061 (secure 0 0 0) 0x7f7df409ca00 con 0x7f7dfc101800 2024-04-26T21:21:05.354 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T21:21:05.354 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T21:21:05.354 INFO:teuthology.orchestra.run.smithi090.stdout: "epoch": 24, 2024-04-26T21:21:05.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.355 INFO:teuthology.orchestra.run.smithi090.stdout: "active_gid": 24752, 2024-04-26T21:21:05.355 INFO:teuthology.orchestra.run.smithi090.stdout: "active_name": "x", 2024-04-26T21:21:05.355 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addrs": { 2024-04-26T21:21:05.355 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-26T21:21:05.355 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.355 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6800", 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2946218715 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v1", 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:6801", 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2946218715 2024-04-26T21:21:05.356 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: "active_addr": "172.21.15.90:6801/2946218715", 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: "active_change": "2024-04-26T18:11:57.155140+0000", 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: "available": true, 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: "standbys": [ 2024-04-26T21:21:05.357 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24706, 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "z", 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.358 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:05.359 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.360 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.361 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.362 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.362 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.362 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.362 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.362 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.362 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.362 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.363 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.363 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.363 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.363 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.363 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.363 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.363 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.364 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.364 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.364 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.364 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.364 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.364 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.365 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.365 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.365 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.365 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.365 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.365 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.366 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.366 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.366 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.366 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.366 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.368 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.372 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.373 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.373 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-04-26T21:21:05.373 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.373 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.373 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.373 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.373 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.374 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.375 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-04-26T21:21:05.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.375 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-04-26T21:21:05.375 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-04-26T21:21:05.376 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.377 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-04-26T21:21:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-04-26T21:21:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.378 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.379 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:21:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.380 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.381 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-04-26T21:21:05.382 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.383 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.384 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.385 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.386 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-04-26T21:21:05.387 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.388 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-26T21:21:05.389 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:21:05.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:05.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.391 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.391 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-04-26T21:21:05.391 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:05.391 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-04-26T21:21:05.391 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.391 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.392 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:21:05.393 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.394 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.395 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.396 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-04-26T21:21:05.397 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.400 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.401 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.402 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.402 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.402 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.402 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.402 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.403 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.405 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.406 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.406 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.406 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.406 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.406 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.406 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.406 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.407 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-04-26T21:21:05.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:05.409 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.409 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.409 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:05.409 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.409 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.409 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.409 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.410 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-04-26T21:21:05.411 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-04-26T21:21:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-04-26T21:21:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-04-26T21:21:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-04-26T21:21:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.412 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.413 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.414 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.415 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-04-26T21:21:05.416 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:21:05.417 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.418 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.419 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.420 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.421 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-04-26T21:21:05.422 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.423 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.424 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.425 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-04-26T21:21:05.425 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.425 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.425 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.425 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:21:05.425 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.426 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.427 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-04-26T21:21:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-04-26T21:21:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.428 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:05.429 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.429 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.429 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.429 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:21:05.429 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.429 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.429 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.430 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.430 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:21:05.430 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:21:05.430 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:05.430 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.430 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.430 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.433 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.434 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.434 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-04-26T21:21:05.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-04-26T21:21:05.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.435 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:21:05.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.436 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.436 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.436 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.436 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-04-26T21:21:05.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-04-26T21:21:05.436 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.436 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.437 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.437 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.437 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.437 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:21:05.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.438 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.438 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.438 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.438 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-04-26T21:21:05.438 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-04-26T21:21:05.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.439 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.439 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.439 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.439 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.439 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.439 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:21:05.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.440 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.440 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.440 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-04-26T21:21:05.440 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.441 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:05.442 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.442 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.442 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.442 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.442 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-04-26T21:21:05.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-04-26T21:21:05.443 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.443 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.443 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.443 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:21:05.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-04-26T21:21:05.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.445 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-04-26T21:21:05.446 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.446 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.446 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.446 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-04-26T21:21:05.446 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-04-26T21:21:05.446 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:05.447 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.448 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:21:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-04-26T21:21:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.449 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.450 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.452 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-04-26T21:21:05.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:21:05.452 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.452 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.452 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.452 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:21:05.452 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.453 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:21:05.453 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.453 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.453 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.454 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.454 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-04-26T21:21:05.454 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:21:05.454 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.454 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.454 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.454 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.456 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-04-26T21:21:05.456 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-04-26T21:21:05.456 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.456 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.456 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.456 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:21:05.456 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.457 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-04-26T21:21:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-04-26T21:21:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:21:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.458 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-04-26T21:21:05.459 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-04-26T21:21:05.460 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.460 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.460 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.460 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:21:05.460 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.460 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.460 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-04-26T21:21:05.461 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.462 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.462 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.462 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:21:05.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.462 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.462 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.465 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-04-26T21:21:05.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-04-26T21:21:05.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.466 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:21:05.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-04-26T21:21:05.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.468 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:21:05.468 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.468 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.468 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.468 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:21:05.468 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.469 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.469 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.469 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.469 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-04-26T21:21:05.469 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:21:05.469 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.469 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.470 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.470 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:21:05.470 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.470 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.470 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:21:05.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.471 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:05.473 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:21:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.474 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.476 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.476 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:21:05.476 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.476 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.476 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.476 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-04-26T21:21:05.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-04-26T21:21:05.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.477 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-04-26T21:21:05.477 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.477 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.478 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:21:05.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.478 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.478 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.478 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.478 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.479 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-04-26T21:21:05.480 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.481 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:21:05.482 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.482 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.482 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.482 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.482 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-04-26T21:21:05.482 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-04-26T21:21:05.482 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.483 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.483 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:05.483 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.483 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.483 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:21:05.484 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.484 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-04-26T21:21:05.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:21:05.484 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:21:05.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:21:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-04-26T21:21:05.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-04-26T21:21:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:21:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.488 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-04-26T21:21:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-04-26T21:21:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:21:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.490 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-04-26T21:21:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-04-26T21:21:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.491 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:21:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.492 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.494 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-04-26T21:21:05.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:21:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.498 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.498 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.500 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.500 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.500 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.500 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.501 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.501 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.501 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.501 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.501 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.501 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.501 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.502 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.502 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.503 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.503 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.503 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.503 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.503 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.504 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.504 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.504 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:21:05.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:21:05.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.505 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.505 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.505 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.505 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.506 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.506 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:21:05.506 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.506 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.506 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.506 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:21:05.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:21:05.507 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.507 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.507 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.507 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-04-26T21:21:05.507 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.508 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-04-26T21:21:05.509 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-04-26T21:21:05.509 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.509 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.509 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.509 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.509 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.510 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:21:05.510 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.510 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.510 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.513 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-04-26T21:21:05.514 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-04-26T21:21:05.514 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.514 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.514 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.514 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.515 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.515 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:21:05.515 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.515 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.515 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-04-26T21:21:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-26T21:21:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.518 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.519 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:21:05.521 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.522 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.522 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:21:05.522 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.522 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.522 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.522 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:21:05.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.523 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.523 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-04-26T21:21:05.523 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-04-26T21:21:05.523 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.524 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.524 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:21:05.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.525 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-04-26T21:21:05.525 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-04-26T21:21:05.525 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.526 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.526 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.526 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.527 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:21:05.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.527 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-04-26T21:21:05.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-04-26T21:21:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.528 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.529 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:21:05.529 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.529 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.529 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.529 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.529 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-04-26T21:21:05.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-04-26T21:21:05.530 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.530 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.531 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.531 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:21:05.531 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.531 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.532 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-04-26T21:21:05.532 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:21:05.532 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.532 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.532 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.532 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.532 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:21:05.533 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.533 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.533 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.535 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.535 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:21:05.535 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.535 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.535 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.535 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.536 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-04-26T21:21:05.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:21:05.538 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.539 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.539 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.539 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.539 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-04-26T21:21:05.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-04-26T21:21:05.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.543 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-04-26T21:21:05.543 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:21:05.543 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.543 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.543 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.543 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.544 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.544 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.544 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:21:05.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.545 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-04-26T21:21:05.545 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:21:05.545 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.545 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.545 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.546 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.546 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.546 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.546 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:21:05.546 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.546 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.547 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.547 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.547 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.547 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.547 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.547 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-04-26T21:21:05.547 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.548 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.549 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.550 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.550 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.550 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.551 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.551 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.552 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.552 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.552 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.553 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.553 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.554 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.554 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.554 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.554 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.554 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.554 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.554 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.555 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.555 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.555 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.555 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.555 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.555 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.555 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.556 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.556 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.556 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.556 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.556 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.556 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.557 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-04-26T21:21:05.558 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-04-26T21:21:05.558 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.558 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-04-26T21:21:05.558 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.559 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.559 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:21:05.559 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.559 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.559 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.559 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.560 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.561 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.562 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-04-26T21:21:05.562 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.562 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.562 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.562 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:21:05.564 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.565 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.565 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.565 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.565 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.565 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.565 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.566 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.566 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.566 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.566 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:21:05.566 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.567 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.568 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.568 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.568 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:21:05.568 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:21:05.568 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.569 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.570 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.570 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:21:05.570 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:21:05.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.571 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.571 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.571 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.571 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.571 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.572 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.573 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.573 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.573 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.574 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:21:05.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:21:05.574 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.575 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.575 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.576 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.576 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.576 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.576 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.576 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:21:05.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:21:05.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.577 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.577 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.577 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.577 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.577 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.577 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.578 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.578 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.578 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.578 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.578 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:21:05.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:21:05.579 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.579 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.579 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.579 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.579 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.579 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:21:05.580 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.581 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.581 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.581 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.581 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.581 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.581 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:21:05.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:21:05.583 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.583 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.583 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.583 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.583 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.583 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.584 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.584 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.584 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.584 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.584 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:21:05.584 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.585 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.586 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.586 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.586 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.586 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.586 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:21:05.586 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.587 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.588 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.589 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.589 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.589 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.589 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.590 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.590 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:21:05.590 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:21:05.590 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.590 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.592 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.592 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:21:05.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:21:05.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.593 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.594 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.594 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.594 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-04-26T21:21:05.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:21:05.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.594 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.594 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.595 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.596 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.596 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:21:05.596 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:21:05.596 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.596 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.596 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:05.597 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.597 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.597 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.597 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.597 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.598 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.598 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:21:05.598 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:21:05.598 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.598 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.598 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.599 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.599 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.599 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.599 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.600 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:21:05.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:21:05.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.601 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.602 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:21:05.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:21:05.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.602 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.602 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.602 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.603 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.604 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:21:05.605 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-04-26T21:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.606 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:21:05.607 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.608 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.609 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.610 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.610 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.610 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.610 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.610 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.610 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.610 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.612 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.612 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.612 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.612 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.614 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.614 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.614 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.615 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:21:05.615 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:21:05.615 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.615 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.615 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.615 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.615 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.616 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.618 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.618 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.618 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:21:05.620 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.621 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.622 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.622 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:21:05.622 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:21:05.622 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.622 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.623 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.624 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.624 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.624 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:21:05.624 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:21:05.624 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.624 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.624 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.625 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.625 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.625 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.625 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.625 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.625 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.626 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.626 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:21:05.626 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:21:05.626 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.627 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.628 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.629 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:21:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:21:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.630 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.631 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.632 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:21:05.633 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-04-26T21:21:05.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.634 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.634 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.634 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.635 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.636 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.636 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.636 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.637 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:05.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.638 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.638 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.638 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.639 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:21:05.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:21:05.639 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.639 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.639 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.640 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.641 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:21:05.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:21:05.641 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.641 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.641 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.641 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.642 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.642 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.642 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.642 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.643 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.643 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:21:05.643 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:21:05.643 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.643 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.643 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.644 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.644 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.644 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.644 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.644 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.644 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.645 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.645 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.645 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.645 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:21:05.645 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:21:05.645 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.645 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.646 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:05.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.646 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.646 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.646 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.647 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.647 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.647 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.647 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.647 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:21:05.647 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:21:05.648 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.648 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.648 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.648 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.648 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.648 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.649 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.649 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.649 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.649 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:21:05.649 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:21:05.650 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.650 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.650 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.650 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:05.650 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.650 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.651 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.651 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.651 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.651 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.651 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-04-26T21:21:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-04-26T21:21:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.652 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.653 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.653 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.653 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.654 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-04-26T21:21:05.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-04-26T21:21:05.654 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.654 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.654 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.655 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.655 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.656 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:21:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-04-26T21:21:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-04-26T21:21:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-04-26T21:21:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.659 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-26T21:21:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-26T21:21:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.661 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.662 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.662 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.662 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.662 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.662 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.663 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.664 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.664 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.664 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.664 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.664 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.664 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.664 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.665 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.665 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.666 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.666 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.667 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.667 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.667 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.667 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.667 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.668 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.669 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.669 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.669 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.671 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-04-26T21:21:05.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-04-26T21:21:05.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.672 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.672 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:21:05.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:21:05.674 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.674 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.674 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.674 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.674 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.674 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.674 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.675 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.675 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.675 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.675 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.675 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.675 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:05.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:05.676 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.676 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.676 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.676 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-26T21:21:05.676 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.676 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.676 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.677 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.677 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.677 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.677 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.677 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:05.677 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:05.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.678 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-04-26T21:21:05.678 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.678 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.678 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-26T21:21:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-26T21:21:05.680 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.680 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-04-26T21:21:05.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.682 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.682 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.682 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-04-26T21:21:05.682 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.682 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.683 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.683 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-26T21:21:05.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-26T21:21:05.683 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-04-26T21:21:05.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.684 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.685 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-04-26T21:21:05.685 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.686 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-26T21:21:05.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-26T21:21:05.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-26T21:21:05.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-26T21:21:05.687 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-26T21:21:05.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.688 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-04-26T21:21:05.688 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-04-26T21:21:05.688 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.689 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.690 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.690 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.690 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-26T21:21:05.690 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.690 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.690 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.691 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.693 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.693 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.693 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.693 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.693 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.694 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.694 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.694 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.694 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.694 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.694 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.694 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.695 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.696 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.696 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.696 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.696 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.696 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.696 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.697 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.697 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.697 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.697 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.697 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.698 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.698 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.698 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.698 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.698 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.699 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.699 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.699 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.699 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.699 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.699 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.700 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.700 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.700 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.700 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.700 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.700 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.700 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.701 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.701 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.701 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.701 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.701 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.701 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.702 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.702 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.702 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.702 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-04-26T21:21:05.702 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-04-26T21:21:05.702 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.703 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-04-26T21:21:05.703 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.703 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.703 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.703 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:21:05.704 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.704 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.704 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.704 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-04-26T21:21:05.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.705 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:21:05.706 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.706 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.706 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-04-26T21:21:05.706 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-04-26T21:21:05.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-04-26T21:21:05.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.707 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.707 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:21:05.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.708 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.708 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-04-26T21:21:05.708 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-04-26T21:21:05.709 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.709 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-26T21:21:05.709 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.709 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.709 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.710 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:21:05.710 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.710 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.710 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.710 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.710 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-04-26T21:21:05.710 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-04-26T21:21:05.711 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.711 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.711 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.711 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.711 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.711 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.711 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:21:05.712 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.712 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:05.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.714 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:21:05.714 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.714 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.714 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.714 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-04-26T21:21:05.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-04-26T21:21:05.715 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.715 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.717 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.718 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-04-26T21:21:05.718 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.718 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.718 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.718 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:21:05.718 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.718 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.719 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.719 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.719 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.719 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.719 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-04-26T21:21:05.720 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.720 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.720 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.720 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.720 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.721 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.721 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.721 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.721 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.721 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.721 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.721 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.722 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.722 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.722 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.722 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.722 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.722 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.722 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.723 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.723 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.723 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.723 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.723 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.723 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.723 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.724 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.724 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.724 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.724 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.724 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.724 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.724 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.725 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.725 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.725 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.725 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.725 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.725 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.725 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.726 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.726 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.726 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.726 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.726 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.726 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.726 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.727 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.728 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.729 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.729 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.729 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.729 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-04-26T21:21:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-04-26T21:21:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.730 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-26T21:21:05.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.731 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.731 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.732 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-04-26T21:21:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.735 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.735 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:05.735 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.735 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.735 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.735 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.736 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.736 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.736 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-04-26T21:21:05.737 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-04-26T21:21:05.737 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:21:05.737 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.737 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-04-26T21:21:05.737 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-04-26T21:21:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-04-26T21:21:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.739 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:21:05.739 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.739 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.739 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.739 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-04-26T21:21:05.740 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-04-26T21:21:05.740 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.740 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.740 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-04-26T21:21:05.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.741 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.741 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:21:05.741 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.741 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.741 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.741 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-04-26T21:21:05.742 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-04-26T21:21:05.742 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.742 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.742 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.742 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.742 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.743 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.743 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.743 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:21:05.743 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.743 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.744 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:05.744 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:05.744 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.744 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.744 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-04-26T21:21:05.744 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-04-26T21:21:05.745 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.745 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:21:05.745 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.745 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.745 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.745 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.746 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.746 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.746 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.746 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.746 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.746 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.747 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.747 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.747 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.747 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.747 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.747 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.748 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.748 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.748 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.748 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.748 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.749 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.749 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.749 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.750 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.750 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.750 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.750 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.750 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.751 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.751 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.751 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.751 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.751 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.751 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.752 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.752 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.752 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.752 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.752 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.752 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.752 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.753 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.753 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.753 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.753 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.753 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.753 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.753 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.754 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.754 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.754 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.754 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.754 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.754 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.755 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.755 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.755 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.755 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.755 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.755 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.756 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.756 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.756 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.756 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-04-26T21:21:05.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-04-26T21:21:05.756 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.756 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.757 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.757 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.757 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.757 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.757 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:21:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-04-26T21:21:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-04-26T21:21:05.758 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.759 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.759 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.759 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-04-26T21:21:05.759 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.759 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.759 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.759 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-04-26T21:21:05.760 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.760 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.760 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.760 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.760 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-26T21:21:05.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-26T21:21:05.761 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.761 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.761 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.761 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-04-26T21:21:05.761 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.761 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.762 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.762 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:21:05.762 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.762 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.762 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:05.763 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-04-26T21:21:05.764 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.764 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:21:05.764 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.764 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.764 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.764 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.764 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-04-26T21:21:05.765 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-04-26T21:21:05.765 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.765 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.765 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.765 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.765 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.766 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.766 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.766 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:21:05.766 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.766 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.766 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.767 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-04-26T21:21:05.767 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-04-26T21:21:05.767 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.767 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.767 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.767 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-04-26T21:21:05.767 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.768 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.768 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.768 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:21:05.768 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.768 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.768 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.768 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.769 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.769 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.769 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.769 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-04-26T21:21:05.769 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.769 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.769 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.770 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.770 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.770 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.770 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.770 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.770 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.770 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.771 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.771 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.771 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.771 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.771 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.771 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.772 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.772 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.772 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.772 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.772 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.773 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.773 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.773 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.773 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.773 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.774 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.774 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.774 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.774 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.774 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.774 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.774 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.775 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.775 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.775 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.775 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.775 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.775 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.775 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.776 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.776 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.776 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.776 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.776 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.776 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.776 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.777 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.777 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.777 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.777 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.777 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.777 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.778 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.778 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.778 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.778 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.779 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.780 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.780 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.780 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.780 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.780 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.780 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-04-26T21:21:05.780 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.781 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.782 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.783 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.783 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.783 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.783 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.783 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.783 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.783 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.784 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.785 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.786 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.786 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.786 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.786 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.786 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.786 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.787 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.787 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.787 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.787 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.787 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.787 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.787 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.788 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.788 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.788 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.788 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.788 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.788 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.788 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.789 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.790 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.790 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.790 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.790 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.790 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.790 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.790 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:21:05.791 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:21:05.792 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.793 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.794 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.794 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-04-26T21:21:05.794 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.794 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.794 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:21:05.794 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.795 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.795 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.795 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.795 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.795 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.795 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.795 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.796 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.796 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.796 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.796 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.796 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.796 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.797 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.798 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.799 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.800 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.801 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.802 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.803 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.803 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.803 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.803 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.803 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.803 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.803 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.804 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.804 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.804 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.804 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.804 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.804 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.804 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-04-26T21:21:05.805 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.805 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.805 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.805 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-04-26T21:21:05.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-04-26T21:21:05.805 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.806 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.806 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.806 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-04-26T21:21:05.806 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.806 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.806 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.806 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:21:05.807 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:21:05.807 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.807 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.807 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.807 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.807 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.808 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.809 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.810 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.811 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.811 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.811 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.811 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.811 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.811 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.811 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.812 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.813 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.813 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.813 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.813 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.813 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.813 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.813 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.814 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.814 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.814 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.814 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.815 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.815 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.815 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.815 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.816 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.816 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.816 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.816 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.816 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.817 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.818 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.818 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.818 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.818 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.818 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:21:05.818 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:21:05.818 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.819 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-04-26T21:21:05.819 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-04-26T21:21:05.819 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.819 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-04-26T21:21:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:21:05.820 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-04-26T21:21:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-04-26T21:21:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.821 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.822 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.822 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-04-26T21:21:05.822 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.822 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.822 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.822 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:21:05.823 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.823 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.823 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.823 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.823 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-04-26T21:21:05.823 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-04-26T21:21:05.823 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.824 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.824 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.824 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.824 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.824 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.824 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.824 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:21:05.825 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.825 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.825 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.825 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.825 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-04-26T21:21:05.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-04-26T21:21:05.825 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.826 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.826 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.826 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-04-26T21:21:05.826 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.826 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.826 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:21:05.827 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:21:05.827 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.827 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.827 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.827 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.827 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-04-26T21:21:05.828 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.828 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.828 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.828 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.828 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.829 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.829 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.829 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.829 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.829 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.829 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.829 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.830 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.831 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.832 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.833 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.834 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.836 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.836 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.836 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.836 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.836 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.836 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.836 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.837 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.838 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.839 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.841 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.842 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.842 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.842 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.842 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.842 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.842 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.843 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.844 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.845 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.845 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.845 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.845 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.845 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.845 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.846 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.846 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.846 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.846 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.846 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.846 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.846 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.847 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.847 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.847 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.847 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.847 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.847 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.848 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.849 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.850 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.850 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.850 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.850 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.850 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.850 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.850 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.851 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.852 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.853 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.853 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.853 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.853 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.853 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.853 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.854 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.854 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.854 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.854 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.854 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.854 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.854 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.856 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.857 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.857 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.857 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.858 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.859 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.860 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.861 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.862 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.863 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.864 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.864 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.864 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.864 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.865 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.865 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.865 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.865 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.866 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.866 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.866 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.866 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.866 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.866 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.867 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.867 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.867 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.867 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.867 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.867 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.868 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.868 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.868 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.868 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.868 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.869 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.870 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.870 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.870 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-04-26T21:21:05.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-26T21:21:05.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.870 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.870 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.871 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.871 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.871 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.871 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.871 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-26T21:21:05.871 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-04-26T21:21:05.871 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.872 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.873 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.873 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-04-26T21:21:05.873 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.873 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.873 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.873 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.873 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.874 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.874 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.874 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.874 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.874 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.874 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.874 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.875 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.876 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.876 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.876 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.876 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.876 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.877 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.877 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.877 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.877 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.877 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.877 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.878 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.879 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.879 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.879 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.879 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.879 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.879 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.879 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.880 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.880 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.880 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.880 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.880 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.880 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.880 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.881 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.882 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.882 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.882 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.882 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.882 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.882 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.882 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.883 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.884 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.885 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.885 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.885 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.885 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.885 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.885 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.885 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.886 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.886 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.886 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.886 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.886 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.886 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.886 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.887 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.888 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.889 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.889 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.889 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.889 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.889 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.889 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.890 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.891 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.892 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.892 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.892 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.892 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.892 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.892 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.892 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.893 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.895 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.896 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.896 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.896 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.896 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.896 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.896 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.897 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.898 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.899 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.900 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.900 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.900 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.900 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.900 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.900 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.900 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.901 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.901 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.901 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.901 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.901 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.901 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.902 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.903 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.904 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.905 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.906 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-04-26T21:21:05.906 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-04-26T21:21:05.906 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:05.906 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.906 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.906 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-26T21:21:05.906 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-04-26T21:21:05.907 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.907 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.907 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-04-26T21:21:05.907 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:21:05.907 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.907 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.907 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.908 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.908 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.908 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-04-26T21:21:05.908 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.908 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.908 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.909 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-04-26T21:21:05.909 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:21:05.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.909 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.909 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.909 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.910 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.910 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.910 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.910 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:21:05.910 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.910 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.910 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.911 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-26T21:21:05.911 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-26T21:21:05.911 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.911 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.911 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.912 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.912 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.912 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.912 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.912 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.912 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.913 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.914 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.914 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.914 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.914 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.914 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.914 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.914 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.915 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.916 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.916 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.916 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.916 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.916 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.916 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.916 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.917 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.917 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.917 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.917 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.917 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.917 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.917 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.918 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.918 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.918 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.918 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.918 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.918 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.919 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.919 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.919 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.919 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.921 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.921 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.921 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.922 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.922 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.922 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.922 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.922 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.922 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.923 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.923 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.923 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.923 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.924 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.924 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.924 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.924 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.924 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.924 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.925 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.925 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.925 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.925 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.925 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.926 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.926 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.926 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-04-26T21:21:05.926 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-04-26T21:21:05.926 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.926 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.927 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.928 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.928 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:05.928 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:05.928 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:05.928 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.928 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.929 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:05.929 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.929 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.929 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.929 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:21:05.929 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.929 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.930 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.930 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.930 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.930 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.930 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.930 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-04-26T21:21:05.930 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.931 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.931 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.931 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-04-26T21:21:05.931 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-04-26T21:21:05.931 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.931 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.931 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.932 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.933 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.933 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-04-26T21:21:05.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-04-26T21:21:05.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.933 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.933 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.934 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.934 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:21:05.934 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:21:05.934 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.934 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.935 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.935 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.935 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.935 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.935 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.935 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.936 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.936 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.936 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.936 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.936 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.936 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.936 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.937 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.937 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.937 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.937 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.937 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.937 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.937 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.938 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.938 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.938 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.938 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.938 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.938 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.938 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.939 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.939 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.939 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.939 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.939 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.939 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.939 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.940 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.940 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.940 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.943 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.943 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.943 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.943 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.943 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.943 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.944 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.945 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.946 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.947 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.948 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.949 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.949 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:21:05.949 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:21:05.949 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.949 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.949 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.949 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.950 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.951 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:05.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:05.953 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.953 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.953 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.953 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-26T21:21:05.953 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.953 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.954 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.954 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:21:05.954 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.954 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.954 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.954 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.954 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.955 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-04-26T21:21:05.956 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.957 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.957 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.957 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-04-26T21:21:05.957 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.957 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.957 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.957 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.958 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.959 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.960 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.960 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-26T21:21:05.960 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-26T21:21:05.960 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.961 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-26T21:21:05.962 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.963 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.963 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.963 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.963 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.963 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.964 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.965 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.966 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.966 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.966 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.966 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.966 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.966 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.966 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.967 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.968 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.968 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.968 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.968 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.968 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.968 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.968 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.969 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.970 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.971 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.972 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.973 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.974 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.975 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.975 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.975 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.975 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.975 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.975 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-04-26T21:21:05.975 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-04-26T21:21:05.976 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.976 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.976 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.976 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.976 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.976 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.976 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.977 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-04-26T21:21:05.978 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.978 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.978 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.978 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-04-26T21:21:05.978 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-04-26T21:21:05.978 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.979 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.980 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.981 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.981 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.981 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.981 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.981 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.981 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.981 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.982 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.982 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.982 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.982 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.982 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.982 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.983 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.984 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.984 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.984 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.984 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.984 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.985 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.985 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.985 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.985 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.985 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.985 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.986 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:05.987 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.988 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.989 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.989 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.989 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.989 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.989 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.989 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.989 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.990 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:05.991 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.992 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.993 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.994 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:05.997 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:05.998 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:05.999 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.000 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.001 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.002 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.003 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.004 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.005 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.006 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.006 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.006 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.006 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.006 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.006 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.006 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.007 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.008 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.009 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.010 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.011 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.012 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.013 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.014 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.015 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.016 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.017 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.018 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.019 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.020 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.021 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.022 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.023 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.024 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.025 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.026 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.026 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.026 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.026 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.026 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-04-26T21:21:06.026 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-04-26T21:21:06.026 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.027 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.028 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-04-26T21:21:06.029 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.030 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.031 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.032 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.033 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.034 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.035 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-04-26T21:21:06.036 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.037 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.038 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.039 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.040 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-04-26T21:21:06.041 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.042 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.043 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.044 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-04-26T21:21:06.045 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:21:06.046 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.047 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.048 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.049 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.050 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.051 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.052 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.053 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.053 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.053 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.053 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.053 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.053 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.053 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.054 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.055 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.056 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.057 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.058 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.059 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.060 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.061 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.062 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.063 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.064 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.064 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.064 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.064 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.064 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.064 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.064 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.065 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.066 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.067 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.067 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.067 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.067 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.067 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.067 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.067 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.068 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.069 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.070 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.070 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.070 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.070 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.070 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.070 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.070 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.071 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.074 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.075 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.075 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-04-26T21:21:06.075 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.075 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.075 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.075 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-04-26T21:21:06.075 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.076 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.077 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.078 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.078 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-04-26T21:21:06.078 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.078 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.078 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.078 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.079 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.079 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.079 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.079 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.079 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.079 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.079 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.080 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.081 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.082 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.082 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.082 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.082 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.082 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.082 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.083 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.084 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.085 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.087 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.087 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.087 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.088 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.088 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.088 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.089 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.090 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.090 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.091 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.091 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.092 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-04-26T21:21:06.092 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.092 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.092 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.093 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-04-26T21:21:06.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-04-26T21:21:06.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.093 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.094 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.094 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.094 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.095 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.095 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:21:06.095 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.096 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.096 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-04-26T21:21:06.097 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-04-26T21:21:06.097 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.097 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.098 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.098 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.098 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.098 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.099 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:21:06.099 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.099 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.099 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.099 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.099 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-04-26T21:21:06.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-04-26T21:21:06.100 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.100 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.100 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.101 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.101 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.102 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:21:06.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.102 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.103 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.103 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.103 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-04-26T21:21:06.103 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-04-26T21:21:06.104 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.104 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.104 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.104 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.105 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.105 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.105 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.105 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:21:06.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.106 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.106 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-04-26T21:21:06.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-04-26T21:21:06.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.107 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.107 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.108 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.108 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.108 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:21:06.108 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.110 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-04-26T21:21:06.110 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-04-26T21:21:06.110 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.110 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.111 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.111 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.112 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.113 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.113 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.113 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-04-26T21:21:06.113 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-04-26T21:21:06.114 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.114 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.114 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.114 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.115 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.115 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.115 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.115 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.115 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.116 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-04-26T21:21:06.116 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-04-26T21:21:06.116 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.116 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.116 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:21:06.117 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.117 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.117 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.117 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.118 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-26T21:21:06.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-26T21:21:06.118 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.118 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.118 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.119 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.121 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-04-26T21:21:06.121 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-04-26T21:21:06.121 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.121 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.122 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.122 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.122 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.123 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-04-26T21:21:06.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-04-26T21:21:06.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.123 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.123 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-04-26T21:21:06.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.124 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.125 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.125 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-04-26T21:21:06.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-04-26T21:21:06.125 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.126 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.126 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.126 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.126 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.126 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-04-26T21:21:06.127 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.128 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.130 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.130 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.131 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.131 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.131 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.131 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.131 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.132 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.132 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.132 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.133 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.133 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.133 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.133 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.134 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.134 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.134 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.135 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.135 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.135 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.135 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.137 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.137 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.137 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.138 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.138 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.138 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.138 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.139 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.139 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.140 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.140 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.140 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.140 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.141 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.141 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.141 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.141 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.142 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.142 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.142 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.142 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.143 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.143 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.144 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.144 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.144 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.144 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.145 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.145 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.145 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.146 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.146 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.147 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.147 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.147 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.147 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.148 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.148 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.148 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.148 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.149 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.149 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.149 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.150 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-04-26T21:21:06.151 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-04-26T21:21:06.151 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.151 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.151 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.152 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.152 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.153 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.153 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.153 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.153 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.154 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-04-26T21:21:06.154 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-04-26T21:21:06.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.155 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.156 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.156 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.156 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.156 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.157 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.157 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.157 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.157 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.158 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-04-26T21:21:06.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-04-26T21:21:06.158 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.158 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:21:06.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-04-26T21:21:06.160 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.161 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.162 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.162 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.162 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.162 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.162 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.162 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.163 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.164 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.165 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.166 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.166 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.166 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.166 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.167 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.168 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.168 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.168 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.168 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.168 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.168 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.168 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.170 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.170 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.170 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.171 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.171 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.171 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-26T21:21:06.171 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.171 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.172 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.172 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.172 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.172 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.172 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.173 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.174 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.174 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.174 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.174 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.174 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.174 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.174 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.175 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.176 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.177 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.178 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.178 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.178 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.178 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.178 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.179 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.179 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.179 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.179 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.179 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.180 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.180 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.181 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.181 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.181 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.181 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.182 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.182 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.182 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.183 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.183 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.183 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.183 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.183 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.184 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-04-26T21:21:06.184 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-04-26T21:21:06.184 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.184 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.184 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.185 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-04-26T21:21:06.185 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.186 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:21:06.186 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.186 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.186 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.187 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-04-26T21:21:06.187 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-04-26T21:21:06.187 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.187 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.187 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.188 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.189 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-04-26T21:21:06.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:21:06.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.189 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.189 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.189 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.190 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.191 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.192 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.192 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:21:06.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.192 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.192 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-04-26T21:21:06.193 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-04-26T21:21:06.194 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:06.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.194 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.194 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-04-26T21:21:06.194 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.194 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.194 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-04-26T21:21:06.195 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.196 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.197 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.198 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.199 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.200 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.201 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.202 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.203 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.204 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.207 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.207 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.207 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.207 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.207 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-04-26T21:21:06.207 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-04-26T21:21:06.207 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.212 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: "gid": 24749, 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "y", 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr_features": 4540701547738038271, 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-04-26T21:21:06.213 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.217 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.217 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.217 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.217 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.217 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.217 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.217 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.218 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.219 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.220 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.221 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-04-26T21:21:06.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:21:06.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-04-26T21:21:06.229 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.230 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.230 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.230 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.230 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.230 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.230 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.230 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.231 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.232 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.233 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-04-26T21:21:06.233 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-04-26T21:21:06.233 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.233 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.233 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.234 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.234 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.234 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.234 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.234 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:21:06.234 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.234 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.235 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-04-26T21:21:06.239 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:21:06.240 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.241 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-04-26T21:21:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:21:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:21:06.242 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:06.243 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.244 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-04-26T21:21:06.245 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-04-26T21:21:06.246 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:21:06.247 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.248 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.249 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.250 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-04-26T21:21:06.251 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:21:06.252 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.253 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.254 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.255 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.256 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.257 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.258 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.259 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.260 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.262 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-26T21:21:06.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-04-26T21:21:06.264 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-04-26T21:21:06.265 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-04-26T21:21:06.265 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-04-26T21:21:06.265 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-04-26T21:21:06.265 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-04-26T21:21:06.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.265 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.266 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.267 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.268 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-04-26T21:21:06.269 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:21:06.270 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.271 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:21:06.272 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.272 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.272 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.273 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.274 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-04-26T21:21:06.275 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.276 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.280 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.281 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:21:06.282 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:21:06.283 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.284 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-04-26T21:21:06.285 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-04-26T21:21:06.286 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:21:06.287 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.290 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-04-26T21:21:06.291 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:06.292 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.293 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.294 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.295 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.295 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-04-26T21:21:06.295 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:21:06.295 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.295 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:21:06.296 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.296 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.296 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.296 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-04-26T21:21:06.296 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.296 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.296 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:21:06.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.298 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:21:06.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:21:06.301 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.302 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.303 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:21:06.304 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.305 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.306 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.307 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.308 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.309 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.311 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-04-26T21:21:06.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.313 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.315 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:21:06.317 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.317 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.317 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.317 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:21:06.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:21:06.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.319 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.320 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.321 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:21:06.321 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.321 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.321 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.321 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-04-26T21:21:06.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.322 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.327 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-04-26T21:21:06.327 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-04-26T21:21:06.327 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-04-26T21:21:06.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.328 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:21:06.329 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.330 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.331 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:06.332 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.333 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:21:06.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.335 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-04-26T21:21:06.335 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-04-26T21:21:06.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.463 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.463 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.463 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:06.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:21:06.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.464 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.464 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.465 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.466 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-04-26T21:21:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-04-26T21:21:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.467 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.469 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.471 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.473 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.473 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.473 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.473 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.473 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.473 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.473 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.474 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.476 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.478 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.479 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.480 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.480 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.480 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.480 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.480 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:21:06.480 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.480 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-04-26T21:21:06.481 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.482 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.482 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.482 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:21:06.482 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.482 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.482 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.482 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.483 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-04-26T21:21:06.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.486 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.487 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-04-26T21:21:06.488 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.489 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.490 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.490 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.490 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-04-26T21:21:06.490 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-04-26T21:21:06.490 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.490 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.490 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.491 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.492 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-04-26T21:21:06.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-04-26T21:21:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.494 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:21:06.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.497 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.499 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-04-26T21:21:06.500 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:21:06.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.501 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.501 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.501 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-04-26T21:21:06.502 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:21:06.503 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.506 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-04-26T21:21:06.507 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.508 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.509 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.510 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.511 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.512 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.514 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.515 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.516 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.519 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.520 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.521 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.522 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.524 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-04-26T21:21:06.525 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-04-26T21:21:06.528 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.528 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.528 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.528 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:21:06.528 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:21:06.528 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.528 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.529 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.531 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.531 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.531 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.531 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.531 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.531 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.532 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.533 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.533 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.533 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:21:06.535 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.538 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.539 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.540 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.541 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:21:06.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.543 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.545 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.546 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.547 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.548 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.552 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.553 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:21:06.554 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.555 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.556 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.556 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.556 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.556 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-04-26T21:21:06.556 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:21:06.556 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.556 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.558 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.558 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:21:06.558 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:21:06.558 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.558 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:06.559 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.559 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.559 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.559 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.559 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.559 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.560 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.561 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.562 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:21:06.563 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.564 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.566 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.566 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.566 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.566 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.566 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.566 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.566 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.567 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:21:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:21:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-04-26T21:21:06.569 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.570 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.571 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:21:06.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.573 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.574 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.575 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.576 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.577 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:21:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:21:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.578 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:21:06.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.580 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.581 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.581 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.581 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:21:06.581 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:21:06.581 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.582 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.583 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.584 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.585 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.586 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.587 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:21:06.588 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.589 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.590 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.591 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-04-26T21:21:06.592 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.593 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.594 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:21:06.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.596 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.597 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.598 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:06.598 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.598 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.598 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.598 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.598 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.598 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.599 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.600 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:21:06.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.603 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.604 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.605 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.606 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.607 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.608 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-04-26T21:21:06.609 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.610 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.614 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.615 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-26T21:21:06.616 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.617 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.618 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.619 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.620 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.621 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.622 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.623 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.626 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-04-26T21:21:06.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.630 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.630 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.630 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.630 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.630 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:06.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-26T21:21:06.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.638 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.639 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-04-26T21:21:06.639 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.639 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-26T21:21:06.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-04-26T21:21:06.641 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.642 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-26T21:21:06.643 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.644 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.645 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.646 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.647 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.648 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.650 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.651 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.652 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.653 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-04-26T21:21:06.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.655 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.656 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.657 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-04-26T21:21:06.657 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.657 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.657 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:21:06.657 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.658 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.659 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-26T21:21:06.660 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.661 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.662 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:06.663 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:21:06.664 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.666 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.667 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.668 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.673 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.674 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-04-26T21:21:06.676 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.677 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.678 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.679 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.680 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-04-26T21:21:06.682 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.683 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.685 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.686 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:06.687 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.688 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.689 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.690 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.690 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.690 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.690 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.690 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.690 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.690 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.691 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.692 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.693 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.694 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.695 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.695 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.695 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.695 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.695 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.695 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.695 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.696 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-04-26T21:21:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-04-26T21:21:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.699 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-04-26T21:21:06.700 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-26T21:21:06.701 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.702 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.703 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.704 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.705 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.705 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.705 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-04-26T21:21:06.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-04-26T21:21:06.705 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.705 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.705 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.706 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-04-26T21:21:06.707 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.708 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.709 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.710 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.711 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.712 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.713 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.714 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.715 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.715 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.715 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.715 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.715 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.716 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.719 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.720 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.721 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.722 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.722 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.722 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.722 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.722 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.722 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.722 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.723 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.724 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.725 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.726 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.727 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.728 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.729 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-04-26T21:21:06.730 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.731 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.732 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.733 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.734 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.736 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.737 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.738 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.739 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.740 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.740 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.742 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.742 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.742 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.742 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.742 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.742 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.743 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:21:06.744 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.745 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.746 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.747 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.747 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.747 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.747 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.747 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.747 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.747 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.748 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.748 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.748 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.748 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.748 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.748 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.748 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.749 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.751 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.751 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.751 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.751 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.751 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.751 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.751 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.752 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.753 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.754 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.756 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-04-26T21:21:06.757 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.758 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.759 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.760 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-04-26T21:21:06.761 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.762 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.764 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.765 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.766 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.767 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.768 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.769 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.770 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.771 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.772 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.773 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.774 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.775 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.776 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.777 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.778 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.779 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.779 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.779 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.779 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.779 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.779 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.779 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.780 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.781 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.782 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.783 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.784 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.785 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.786 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.787 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.788 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.789 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.790 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.791 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.793 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.794 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.795 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.795 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.795 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.795 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.795 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.795 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.795 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.796 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.797 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.797 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.797 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.797 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.797 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.797 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.797 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.798 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.799 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.800 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.802 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.802 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.802 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.802 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.802 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.802 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.803 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.804 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.806 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.807 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.808 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.809 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.810 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.811 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.812 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.813 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.814 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.816 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.817 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.818 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.820 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.820 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.820 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.820 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.820 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.820 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.820 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.821 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.822 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.823 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.824 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.825 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.826 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.828 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.829 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.830 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.831 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.832 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.833 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.834 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-04-26T21:21:06.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-04-26T21:21:06.836 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.836 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.836 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.836 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-26T21:21:06.836 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-04-26T21:21:06.836 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.836 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.837 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.838 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.839 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.840 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.841 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.842 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.843 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.844 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.844 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.844 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.844 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.844 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.844 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.844 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.845 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.846 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.847 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.848 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.849 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-04-26T21:21:06.850 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.851 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:06.852 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.853 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.854 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.854 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.854 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-04-26T21:21:06.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.856 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.857 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:21:06.858 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.859 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.860 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.861 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.862 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.863 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.864 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.865 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.866 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.867 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-04-26T21:21:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-04-26T21:21:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.868 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.869 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.870 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-04-26T21:21:06.871 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.872 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.873 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-26T21:21:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:21:06.874 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.875 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.876 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-04-26T21:21:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-04-26T21:21:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-04-26T21:21:06.877 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.878 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.879 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.880 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-26T21:21:06.881 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.882 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.883 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.884 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.885 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.886 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.887 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.888 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.889 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.890 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.891 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-04-26T21:21:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:21:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.892 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.893 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.894 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.895 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.896 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.897 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-04-26T21:21:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-04-26T21:21:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-04-26T21:21:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.898 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.899 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.900 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.901 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.902 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.903 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.904 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.905 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.906 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.907 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.908 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.908 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.908 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.908 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.908 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.908 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.908 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.909 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.910 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.910 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.910 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.910 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.910 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:06.910 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:06.910 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.911 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.912 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.913 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.913 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.913 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.913 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.913 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.913 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.913 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.914 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.915 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.916 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.917 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.918 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.919 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.920 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.921 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.921 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.921 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.921 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.921 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.921 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.921 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.922 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.923 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.924 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.925 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.926 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.927 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.928 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.929 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.930 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.931 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.932 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.933 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.934 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.935 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.936 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.936 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.936 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.936 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.936 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.936 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.936 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.937 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.938 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.939 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.939 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.939 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.939 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.939 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.939 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.941 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.941 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.942 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.943 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.944 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.945 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.946 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.946 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.946 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.946 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.946 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.946 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.946 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.947 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.948 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.949 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-04-26T21:21:06.950 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.951 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.952 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.953 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.954 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.954 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-04-26T21:21:06.954 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-04-26T21:21:06.954 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:06.954 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.954 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.954 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.955 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.956 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-04-26T21:21:06.957 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.958 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.959 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.960 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:06.961 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-04-26T21:21:06.962 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.963 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.964 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.965 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.966 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.966 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-04-26T21:21:06.966 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-04-26T21:21:06.966 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.966 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.966 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:06.966 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.967 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.968 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.969 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-04-26T21:21:06.970 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:21:06.971 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.972 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.973 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.974 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.974 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.974 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.974 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.974 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.974 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.974 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.975 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.976 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.977 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.978 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.979 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.980 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.981 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.982 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.982 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.982 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.982 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.982 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.982 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.982 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.983 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.984 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.985 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.986 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.987 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.988 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.989 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.990 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.990 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.990 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.990 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.990 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.990 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:06.990 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.991 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.992 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.992 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:06.992 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.992 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.992 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.992 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.992 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.993 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.994 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.994 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.994 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:06.994 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:06.994 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.994 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.994 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.995 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.996 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:06.997 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:06.998 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:06.999 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-04-26T21:21:07.000 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.001 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.002 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.003 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.004 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.004 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.004 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.004 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.004 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.004 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.004 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.005 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.006 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.007 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.007 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.007 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.007 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.007 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.007 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.007 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.008 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.009 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.010 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.011 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.012 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.013 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-04-26T21:21:07.014 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:21:07.015 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.016 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.017 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.018 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.019 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.020 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-04-26T21:21:07.021 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.022 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.023 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.024 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.025 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.026 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.027 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.028 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.029 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.030 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.030 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.030 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.030 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.030 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.030 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.030 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.031 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.032 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-04-26T21:21:07.033 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-04-26T21:21:07.034 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.034 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.034 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.034 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.034 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.034 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.034 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-04-26T21:21:07.035 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.036 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.037 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.038 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.038 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.038 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.038 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.038 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.038 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.038 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.039 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.040 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.041 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.042 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.043 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.044 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.045 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.045 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.045 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.045 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.045 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.045 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.045 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.046 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.048 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-04-26T21:21:07.049 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.050 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.051 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.052 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.053 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.054 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.055 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.056 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.057 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.058 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.059 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.060 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.060 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.060 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.060 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.060 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.060 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.060 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.061 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.062 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.063 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.064 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.065 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.066 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.066 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.066 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.066 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.066 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.066 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.066 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.067 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.068 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.069 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.070 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.071 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-04-26T21:21:07.072 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-04-26T21:21:07.073 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.074 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.075 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.076 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.077 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-04-26T21:21:07.078 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.079 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.080 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.081 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.081 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.081 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.081 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.081 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.081 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.081 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.082 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.083 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.084 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.085 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.086 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.087 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.088 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.088 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.088 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.088 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.088 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.088 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.088 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.089 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.090 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.091 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.092 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.093 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-04-26T21:21:07.094 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.095 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.096 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:21:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.097 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.098 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: "modules": [ 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard", 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: "iostat", 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: "nfs", 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: "restful", 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: "snap_schedule" 2024-04-26T21:21:07.099 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: "available_modules": [ 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "alerts", 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.100 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.101 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T21:21:07.102 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.102 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.102 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.102 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.102 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.102 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.102 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.103 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.104 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.105 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.106 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.107 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.108 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.109 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.110 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_destination": { 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_destination", 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.111 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email address to send alerts to", 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.112 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_from_name": { 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_from_name", 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "Ceph", 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.113 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Email From: name", 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_host": { 2024-04-26T21:21:07.114 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_host", 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.115 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP server", 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_password": { 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_password", 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.116 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.117 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.117 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.117 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.117 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.117 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Password to authenticate with", 2024-04-26T21:21:07.117 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.117 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_port": { 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_port", 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.118 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "465", 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP port", 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.119 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_sender": { 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_sender", 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.120 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SMTP envelope sender", 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_ssl": { 2024-04-26T21:21:07.121 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_ssl", 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.122 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: "smtp_user": { 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "smtp_user", 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.123 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "User to authenticate as", 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.124 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "balancer", 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.125 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.126 INFO:teuthology.orchestra.run.smithi090.stdout: "active": { 2024-04-26T21:21:07.126 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "active", 2024-04-26T21:21:07.126 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.126 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.126 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.126 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.126 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.127 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_time": { 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_time", 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0000", 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.128 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "begin_weekday": { 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "begin_weekday", 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:07.129 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:21:07.130 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_max_iterations": { 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_max_iterations", 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.131 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "25", 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "250", 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.132 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_metrics": { 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_metrics", 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.133 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: "crush_compat_step": { 2024-04-26T21:21:07.134 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crush_compat_step", 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.5", 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0.001", 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "0.999", 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.135 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "aggressiveness of optimization", 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: "end_time": { 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_time", 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.136 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2359", 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T21:21:07.137 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: "end_weekday": { 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "end_weekday", 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.138 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "0", 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "6", 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.139 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.140 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.141 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.142 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.143 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.143 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.143 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.143 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.143 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.143 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.145 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.145 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.145 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.145 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.146 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.146 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.146 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.146 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.146 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.146 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.146 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.147 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.148 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.149 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: "min_score": { 2024-04-26T21:21:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_score", 2024-04-26T21:21:07.150 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.151 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.152 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "upmap", 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "crush-compat", 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "none", 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "read", 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap", 2024-04-26T21:21:07.153 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap-read" 2024-04-26T21:21:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Balancer mode", 2024-04-26T21:21:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.154 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_ids": { 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_ids", 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.155 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:07.156 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.157 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.157 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.157 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:07.157 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.157 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.157 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.157 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_deviation": { 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_deviation", 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.158 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T21:21:07.159 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: "upmap_max_optimizations": { 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "upmap_max_optimizations", 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.160 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.161 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm", 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.162 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_down_multiplier": { 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_down_multiplier", 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.163 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_refresh_rate": { 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_refresh_rate", 2024-04-26T21:21:07.164 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "20", 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T21:21:07.165 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: "agent_starting_port": { 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "agent_starting_port", 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.166 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4721", 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.167 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_ptrace": { 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_ptrace", 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.168 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_interval": { 2024-04-26T21:21:07.169 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_interval", 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.170 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: "autotune_memory_target_ratio": { 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:07.171 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0.7", 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.172 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm_log_destination": { 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cephadm_log_destination", 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.173 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: "file", 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: "file,syslog", 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: "syslog" 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.174 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: "cgroups_split": { 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cgroups_split", 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.175 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.176 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: "config_checks_enabled": { 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_checks_enabled", 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.177 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.178 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "config_dashboard": { 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_dashboard", 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.179 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_alertmanager": { 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_alertmanager", 2024-04-26T21:21:07.180 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:07.181 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_base": { 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_base", 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.182 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Container image name, without the tag", 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.183 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_elasticsearch": { 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_elasticsearch", 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T21:21:07.184 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "elasticsearch container image", 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.185 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_grafana": { 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_grafana", 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.186 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_haproxy": { 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_haproxy", 2024-04-26T21:21:07.187 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "HAproxy container image", 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.188 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_agent": { 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_agent", 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.189 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger agent container image", 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.190 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_collector": { 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_collector", 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.191 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger collector container image", 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_jaeger_query": { 2024-04-26T21:21:07.192 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_jaeger_query", 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.193 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Jaeger query container image", 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_keepalived": { 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_keepalived", 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.194 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Keepalived container image", 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.195 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_loki": { 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_loki", 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T21:21:07.196 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.197 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.197 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.197 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Loki container image", 2024-04-26T21:21:07.197 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.197 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.197 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.197 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_node_exporter": { 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_node_exporter", 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.198 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_nvmeof": { 2024-04-26T21:21:07.199 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_nvmeof", 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.200 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Nvme-of container image", 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_prometheus": { 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_prometheus", 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.201 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Prometheus container image", 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.202 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_promtail": { 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_promtail", 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.203 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Promtail container image", 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_samba": { 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_samba", 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.205 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Samba/SMB container image", 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: "container_image_snmp_gateway": { 2024-04-26T21:21:07.206 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_image_snmp_gateway", 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.207 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "SNMP Gateway container image", 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: "container_init": { 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "container_init", 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.208 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.209 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.210 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.210 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.210 INFO:teuthology.orchestra.run.smithi090.stdout: "daemon_cache_timeout": { 2024-04-26T21:21:07.210 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "daemon_cache_timeout", 2024-04-26T21:21:07.210 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.210 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.210 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.211 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "default_cephadm_command_timeout": { 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "900", 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.212 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: "default_registry": { 2024-04-26T21:21:07.213 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "default_registry", 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "docker.io", 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.214 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: "device_cache_timeout": { 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_cache_timeout", 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.215 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1800", 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache device inventory", 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.216 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "device_enhanced_scan": { 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_enhanced_scan", 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.217 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.218 INFO:teuthology.orchestra.run.smithi090.stdout: "facts_cache_timeout": { 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "facts_cache_timeout", 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.219 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "seconds to cache host facts data", 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: "grafana_dashboards_path": { 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "grafana_dashboards_path", 2024-04-26T21:21:07.220 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T21:21:07.221 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: "host_check_interval": { 2024-04-26T21:21:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "host_check_interval", 2024-04-26T21:21:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.222 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to perform a host check", 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.223 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: "hw_monitoring": { 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hw_monitoring", 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.224 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.225 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "inventory_list_all": { 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "inventory_list_all", 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.226 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.227 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.228 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.229 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.230 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.230 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.230 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.230 INFO:teuthology.orchestra.run.smithi090.stdout: "log_refresh_metadata": { 2024-04-26T21:21:07.230 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_refresh_metadata", 2024-04-26T21:21:07.230 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.230 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.231 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.232 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.233 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.234 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.235 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.236 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.236 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.236 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.236 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.236 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.236 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.236 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.237 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.238 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.239 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "*", 2024-04-26T21:21:07.240 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "max_count_per_host": { 2024-04-26T21:21:07.241 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_count_per_host", 2024-04-26T21:21:07.242 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.242 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.242 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.242 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.242 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.242 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.242 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of daemons per service per host", 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "max_osd_draining_count": { 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_osd_draining_count", 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.243 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.244 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: "migration_current": { 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "migration_current", 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.245 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "internal - do not modify", 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.246 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "mode": { 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mode", 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "root", 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.247 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm-package", 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: "root" 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.248 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.249 INFO:teuthology.orchestra.run.smithi090.stdout: "oob_default_addr": { 2024-04-26T21:21:07.249 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "oob_default_addr", 2024-04-26T21:21:07.249 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.249 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.249 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.249 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "169.254.1.1", 2024-04-26T21:21:07.249 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.250 INFO:teuthology.orchestra.run.smithi090.stdout: "prometheus_alerts_path": { 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus_alerts_path", 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.251 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_insecure": { 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_insecure", 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.252 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.253 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_password": { 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_password", 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.254 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.255 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_url": { 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_url", 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "registry_username": { 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "registry_username", 2024-04-26T21:21:07.257 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.258 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.258 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.258 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.258 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.258 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.258 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.258 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: "secure_monitoring_stack": { 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "secure_monitoring_stack", 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.259 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.260 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: "service_discovery_port": { 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "service_discovery_port", 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8765", 2024-04-26T21:21:07.261 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.262 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.262 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.262 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "cephadm service discovery port", 2024-04-26T21:21:07.262 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.262 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.262 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.262 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "ssh_config_file": { 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssh_config_file", 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.263 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: "use_agent": { 2024-04-26T21:21:07.264 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_agent", 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T21:21:07.265 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.266 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.266 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.266 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.266 INFO:teuthology.orchestra.run.smithi090.stdout: "use_repo_digest": { 2024-04-26T21:21:07.266 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "use_repo_digest", 2024-04-26T21:21:07.266 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.266 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.267 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_failed_host_check": { 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_failed_host_check", 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.268 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.269 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_daemons": { 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_daemons", 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.270 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_on_stray_hosts": { 2024-04-26T21:21:07.271 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_on_stray_hosts", 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.272 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.273 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crash", 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.274 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.275 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.276 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.277 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.278 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.279 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.280 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: "retain_interval": { 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retain_interval", 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "31536000", 2024-04-26T21:21:07.283 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.284 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_recent_interval": { 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_recent_interval", 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1209600", 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.285 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.286 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T21:21:07.286 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.286 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.286 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.286 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.286 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.286 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dashboard", 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T21:21:07.287 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.288 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.289 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.290 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.291 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_ENABLED": { 2024-04-26T21:21:07.292 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.293 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.294 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.294 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.294 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.294 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.294 INFO:teuthology.orchestra.run.smithi090.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T21:21:07.294 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T21:21:07.294 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.295 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.296 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.297 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.298 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.299 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.299 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.299 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.299 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.299 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.299 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.299 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.300 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T21:21:07.301 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.302 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.303 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.304 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.305 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.306 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T21:21:07.307 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T21:21:07.307 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.307 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.307 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.307 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.307 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.307 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T21:21:07.308 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.309 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.310 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.311 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.312 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:07.313 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.313 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.313 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.313 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.313 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.313 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.313 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.314 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.315 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_URL": { 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_URL", 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.316 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T21:21:07.317 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.318 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.318 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.318 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:07.318 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.318 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.318 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.318 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.319 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.320 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.321 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.322 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.322 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.322 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.322 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.322 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.322 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.322 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.323 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.324 INFO:teuthology.orchestra.run.smithi090.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.325 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T21:21:07.326 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "[]", 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.327 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.328 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "{}", 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.329 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.330 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.331 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.331 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.331 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.331 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.331 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.331 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.331 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.332 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T21:21:07.333 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T21:21:07.334 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.334 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.334 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.334 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.334 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.334 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.334 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T21:21:07.335 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.336 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.336 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.336 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.336 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.336 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.336 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.336 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.337 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.338 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.339 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.340 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.341 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T21:21:07.342 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.343 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.344 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.345 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.345 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.345 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.345 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.345 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.345 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.345 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.346 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.347 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.348 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T21:21:07.349 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.350 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.350 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.350 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.350 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.350 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.350 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.350 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.351 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8", 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.352 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "45", 2024-04-26T21:21:07.353 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.354 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.355 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T21:21:07.356 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "admin", 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.357 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.358 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.359 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.360 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T21:21:07.361 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.362 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.363 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.364 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.365 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T21:21:07.366 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.367 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: "cross_origin_url": { 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cross_origin_url", 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.368 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.369 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: "crt_file": { 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "crt_file", 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.370 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.371 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "debug": { 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "debug", 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.372 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Enable/disable debug options", 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: "jwt_token_ttl": { 2024-04-26T21:21:07.373 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "jwt_token_ttl", 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "28800", 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.374 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.375 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.376 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.377 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.378 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.379 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.379 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.379 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.379 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.379 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.379 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.379 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.380 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.381 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.382 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.383 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.384 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.385 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.386 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.386 INFO:teuthology.orchestra.run.smithi090.stdout: "motd": { 2024-04-26T21:21:07.386 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "motd", 2024-04-26T21:21:07.386 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.386 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.386 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.386 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "The message of the day", 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.387 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect_resolve_ip_addr": { 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.388 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.389 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.389 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.389 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.389 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.389 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.389 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:07.389 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.390 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.391 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8080", 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.392 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.393 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.394 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl_server_port": { 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl_server_port", 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8443", 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.395 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-26T21:21:07.396 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "redirect", 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.397 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.398 INFO:teuthology.orchestra.run.smithi090.stdout: "redirect" 2024-04-26T21:21:07.398 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.398 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.398 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.398 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.398 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.398 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-26T21:21:07.399 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-26T21:21:07.400 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.400 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.400 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.400 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.400 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.400 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.400 INFO:teuthology.orchestra.run.smithi090.stdout: "url_prefix": { 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url_prefix", 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.401 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.402 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_monitoring": { 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_monitoring", 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.403 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "monitor device health metrics", 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.404 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.405 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.406 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.407 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.408 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.409 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.410 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.411 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.412 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.413 INFO:teuthology.orchestra.run.smithi090.stdout: "mark_out_threshold": { 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mark_out_threshold", 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "2419200", 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.414 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: "pool_name": { 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pool_name", 2024-04-26T21:21:07.415 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "device_health_metrics", 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T21:21:07.416 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: "retention_period": { 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "retention_period", 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.417 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.418 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15552000", 2024-04-26T21:21:07.418 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.418 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.418 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.418 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long to retain device health metrics", 2024-04-26T21:21:07.418 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.419 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.419 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.419 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.419 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_frequency": { 2024-04-26T21:21:07.419 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_frequency", 2024-04-26T21:21:07.419 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.419 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.420 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: "self_heal": { 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "self_heal", 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.421 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.422 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.423 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "warn_threshold": { 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "warn_threshold", 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.424 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7257600", 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.425 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "diskprediction_local", 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.426 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.427 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.428 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.429 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.429 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.429 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.429 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.429 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.429 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.429 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.430 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.431 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.432 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.433 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.434 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.434 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.434 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.434 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.434 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.434 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.435 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "predict_interval": { 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predict_interval", 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "86400", 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.436 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: "predictor_model": { 2024-04-26T21:21:07.437 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "predictor_model", 2024-04-26T21:21:07.438 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.438 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.438 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.438 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "prophetstor", 2024-04-26T21:21:07.438 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.438 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.438 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:07.439 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "600", 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.440 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "influx", 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": false, 2024-04-26T21:21:07.441 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "influxdb python module not found", 2024-04-26T21:21:07.442 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.442 INFO:teuthology.orchestra.run.smithi090.stdout: "batch_size": { 2024-04-26T21:21:07.442 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "batch_size", 2024-04-26T21:21:07.442 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.442 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.442 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.442 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5000", 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.443 INFO:teuthology.orchestra.run.smithi090.stdout: "database": { 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "database", 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "ceph", 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.444 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "hostname": { 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "hostname", 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.445 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server hostname", 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.446 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "30", 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "5", 2024-04-26T21:21:07.447 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.448 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.449 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.450 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.451 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.452 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.453 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.454 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.455 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "password": { 2024-04-26T21:21:07.456 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "password", 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.457 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "password of InfluxDB server user", 2024-04-26T21:21:07.458 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.458 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.458 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.458 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.458 INFO:teuthology.orchestra.run.smithi090.stdout: "port": { 2024-04-26T21:21:07.458 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "port", 2024-04-26T21:21:07.458 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "8086", 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "InfluxDB server port", 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.459 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "ssl": { 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ssl", 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "false", 2024-04-26T21:21:07.460 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.461 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.461 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.461 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:21:07.461 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.461 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.461 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.461 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "threads": { 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threads", 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:07.462 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "32", 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "username": { 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "username", 2024-04-26T21:21:07.463 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "username of InfluxDB server user", 2024-04-26T21:21:07.464 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: "verify_ssl": { 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "verify_ssl", 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.465 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "true", 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.466 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "insights", 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.467 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.468 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.469 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.470 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.471 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.472 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.473 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.474 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.475 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.476 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "iostat", 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.477 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.478 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.478 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.478 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.478 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.478 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.478 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.478 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.479 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.480 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.481 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.482 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.483 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.484 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.485 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.486 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.486 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.486 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.486 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "k8sevents", 2024-04-26T21:21:07.486 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.486 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.486 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_event_retention_days": { 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "ceph_event_retention_days", 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "7", 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.487 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: "config_check_secs": { 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "config_check_secs", 2024-04-26T21:21:07.488 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "10", 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T21:21:07.489 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.490 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.490 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.490 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.490 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.490 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.490 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.490 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.491 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.492 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.493 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.494 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.495 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.496 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.497 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.498 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "localpool", 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "failure_domain": { 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "failure_domain", 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.499 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "host", 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "failure domain for any created local pool", 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.500 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.501 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.502 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.503 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.503 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.503 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.503 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.503 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.503 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.503 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.504 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.505 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.505 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.505 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.505 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.505 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.505 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.505 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.506 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.507 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.508 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.509 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min_size": { 2024-04-26T21:21:07.510 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "min_size", 2024-04-26T21:21:07.510 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.510 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.510 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.510 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.510 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default min_size for any created local pool", 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.511 INFO:teuthology.orchestra.run.smithi090.stdout: "num_rep": { 2024-04-26T21:21:07.512 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "num_rep", 2024-04-26T21:21:07.512 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.512 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.512 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.512 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3", 2024-04-26T21:21:07.512 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.512 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default replica count for any created local pool", 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_num": { 2024-04-26T21:21:07.513 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_num", 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "128", 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.514 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: "prefix": { 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prefix", 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.515 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "name prefix for any created local pool", 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.516 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: "subtree": { 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "subtree", 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.517 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "rack", 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.518 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mds_autoscaler", 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.519 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.520 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.521 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.522 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.523 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.524 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.525 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.526 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.526 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.526 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.526 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.526 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.526 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.526 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.527 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.528 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirroring", 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.529 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.530 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.531 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.532 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.532 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.532 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.532 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.532 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.532 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.532 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.533 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.534 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.535 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.535 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.535 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.535 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.535 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.535 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.535 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.536 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.537 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "nfs", 2024-04-26T21:21:07.538 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.539 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.539 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.539 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.539 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.539 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.539 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.539 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.540 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.542 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.542 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.542 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.542 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.542 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.542 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.542 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.543 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.544 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.545 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.546 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.547 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.548 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "fail_fs": { 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "fail_fs", 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.549 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.550 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.551 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.552 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.553 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.554 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.554 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.554 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.554 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.554 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.557 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.557 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.557 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.557 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.558 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.558 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.558 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.558 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.558 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.558 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.558 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.559 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.560 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.561 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.562 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.562 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator": { 2024-04-26T21:21:07.562 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "orchestrator", 2024-04-26T21:21:07.562 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.562 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.562 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.562 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: "cephadm", 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: "rook", 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: "test_orchestrator" 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.563 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Orchestrator backend", 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.564 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_perf_query", 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.565 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.566 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.567 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.567 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.567 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.567 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.567 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.567 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.567 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.568 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.569 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.570 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.571 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.572 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.573 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "osd_support", 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.574 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.575 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.575 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.575 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.575 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.575 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.575 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.575 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.576 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.577 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.577 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.577 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.577 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.577 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.577 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.577 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.578 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.579 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.580 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.581 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.582 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.582 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.582 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.582 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.582 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.582 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.582 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.583 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "pg_autoscaler", 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.584 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.585 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.586 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.587 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.587 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.587 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.587 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.587 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.587 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.587 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.588 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.589 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.590 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.591 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.592 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.593 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.594 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: "threshold": { 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "threshold", 2024-04-26T21:21:07.595 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:07.596 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.596 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.596 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "3.0", 2024-04-26T21:21:07.596 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1.0", 2024-04-26T21:21:07.596 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.596 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.596 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "scaling threshold", 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.597 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "progress", 2024-04-26T21:21:07.598 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.598 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.598 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.598 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_pg_recovery_event": { 2024-04-26T21:21:07.598 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_pg_recovery_event", 2024-04-26T21:21:07.598 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.598 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.599 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.600 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.601 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.602 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.603 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.604 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.605 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.606 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.607 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.607 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.607 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.607 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.607 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.607 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.607 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.608 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.609 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: "max_completed_events": { 2024-04-26T21:21:07.610 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_completed_events", 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "50", 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.611 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "number of past completed events to remember", 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: "sleep_interval": { 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "sleep_interval", 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.612 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "5", 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "how long the module is going to sleep", 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.613 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "prometheus", 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.614 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "cache": { 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "cache", 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.615 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.616 INFO:teuthology.orchestra.run.smithi090.stdout: "exclude_perf_counters": { 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "exclude_perf_counters", 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.617 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.618 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.619 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.620 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.621 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.622 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.623 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.624 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.625 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.626 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.626 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.626 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.626 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.626 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.626 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.626 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools": { 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools", 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.627 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.628 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "300", 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.629 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: "scrape_interval": { 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "scrape_interval", 2024-04-26T21:21:07.630 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15.0", 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.631 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.632 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "::", 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.633 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "9283", 2024-04-26T21:21:07.634 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.635 INFO:teuthology.orchestra.run.smithi090.stdout: "stale_cache_strategy": { 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stale_cache_strategy", 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "log", 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.636 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_behaviour": { 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_behaviour", 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.637 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "default", 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: "default", 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: "error" 2024-04-26T21:21:07.638 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: "standby_error_status_code": { 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "standby_error_status_code", 2024-04-26T21:21:07.639 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "500", 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "400", 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "599", 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.640 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-26T21:21:07.641 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.642 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.642 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.642 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.642 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.642 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.642 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.642 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.643 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.644 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.645 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.646 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.647 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.648 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.649 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.650 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.650 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.650 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.650 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.650 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.650 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.650 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_snap_create": { 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_snap_create", 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.651 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10", 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.652 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "mirror_snapshot_schedule": { 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.653 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.654 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.654 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.654 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.654 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.654 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.654 INFO:teuthology.orchestra.run.smithi090.stdout: "trash_purge_schedule": { 2024-04-26T21:21:07.654 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "trash_purge_schedule", 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.655 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.656 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "restful", 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "enable_auth": { 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enable_auth", 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.657 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.658 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.659 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.659 INFO:teuthology.orchestra.run.smithi090.stdout: "key_file": { 2024-04-26T21:21:07.659 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "key_file", 2024-04-26T21:21:07.659 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.659 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.659 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.659 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.660 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.661 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.662 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.662 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.662 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.662 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.662 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.662 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.662 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.663 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.664 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.665 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.666 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.666 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.666 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.666 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.666 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.666 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.666 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.667 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.668 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.668 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.668 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.668 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.668 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.668 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.668 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.669 INFO:teuthology.orchestra.run.smithi090.stdout: "server_addr": { 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_addr", 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.670 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: "server_port": { 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "server_port", 2024-04-26T21:21:07.671 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.672 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rgw", 2024-04-26T21:21:07.673 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.674 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.675 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.676 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.676 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.676 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.676 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.676 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.676 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.676 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.677 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.678 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.679 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.680 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.681 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.682 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rook", 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.683 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.684 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.685 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.686 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.686 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.686 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.686 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.686 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.686 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.686 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.687 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.688 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.689 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.690 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.691 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: "storage_class": { 2024-04-26T21:21:07.692 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "storage_class", 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "local", 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.693 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.694 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "selftest", 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.695 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.696 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.697 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.698 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.699 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.700 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.701 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.702 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.703 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: "roption1": { 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption1", 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.704 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "roption2": { 2024-04-26T21:21:07.705 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "roption2", 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "xyz", 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.706 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption1": { 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption1", 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.707 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.708 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption2": { 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption2", 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.709 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.710 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.710 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.710 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.710 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.710 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.710 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.710 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption3": { 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption3", 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "float", 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.711 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption4": { 2024-04-26T21:21:07.712 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption4", 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.713 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption5": { 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption5", 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.714 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.715 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption6": { 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption6", 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.716 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.717 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "rwoption7": { 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rwoption7", 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "1", 2024-04-26T21:21:07.718 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "42", 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: "testkey": { 2024-04-26T21:21:07.719 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testkey", 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.720 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: "testlkey": { 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testlkey", 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.721 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.722 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.723 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.723 INFO:teuthology.orchestra.run.smithi090.stdout: "testnewline": { 2024-04-26T21:21:07.723 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "testnewline", 2024-04-26T21:21:07.723 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.723 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.723 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.723 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.724 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.725 INFO:teuthology.orchestra.run.smithi090.stdout: "allow_m_granularity": { 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "allow_m_granularity", 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.726 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.727 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T21:21:07.727 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.727 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.727 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.727 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.727 INFO:teuthology.orchestra.run.smithi090.stdout: "dump_on_update": { 2024-04-26T21:21:07.727 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "dump_on_update", 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.728 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "dump database to debug log on update", 2024-04-26T21:21:07.729 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.729 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.729 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.729 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.729 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.729 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.729 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.730 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.731 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.732 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.733 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.734 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.735 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.736 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.737 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.738 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.738 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.738 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.738 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.738 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.738 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.738 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "stats", 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.739 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.740 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.741 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.742 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.742 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.742 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.742 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.742 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.742 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.742 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.743 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.744 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.745 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.746 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.746 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.746 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.746 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.746 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.746 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.746 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.747 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "status", 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.749 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.750 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.751 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.752 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.753 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.754 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.755 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.756 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.756 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.757 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.757 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.758 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.758 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.758 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.758 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.758 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.758 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.758 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telegraf", 2024-04-26T21:21:07.759 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "address": { 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "address", 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.760 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.761 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "15", 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.762 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.763 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.764 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.765 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.766 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.766 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.766 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.766 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.766 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.766 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.766 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.767 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.767 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.767 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.767 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.767 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.767 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.767 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.768 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.769 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.770 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.770 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.770 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.770 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.770 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.770 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.770 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.772 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "telemetry", 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_basic": { 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_basic", 2024-04-26T21:21:07.773 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T21:21:07.774 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_crash": { 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_crash", 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.775 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.776 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_device": { 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_device", 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.777 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.778 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.778 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.778 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T21:21:07.778 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.778 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.778 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_ident": { 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_ident", 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.779 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.780 INFO:teuthology.orchestra.run.smithi090.stdout: "channel_perf": { 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "channel_perf", 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.781 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.782 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T21:21:07.782 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.782 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.782 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.782 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.782 INFO:teuthology.orchestra.run.smithi090.stdout: "contact": { 2024-04-26T21:21:07.782 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "contact", 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.783 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: "description": { 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "description", 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.784 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.785 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: "device_url": { 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "device_url", 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.786 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.787 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.788 INFO:teuthology.orchestra.run.smithi090.stdout: "enabled": { 2024-04-26T21:21:07.788 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "enabled", 2024-04-26T21:21:07.788 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.788 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.788 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.788 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.788 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.789 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "24", 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "8", 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.790 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: "last_opt_revision": { 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "last_opt_revision", 2024-04-26T21:21:07.791 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "1", 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.792 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard": { 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard", 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.793 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.794 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: "leaderboard_description": { 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "leaderboard_description", 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.795 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.796 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.796 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.796 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.796 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.796 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.796 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.796 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.797 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.798 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.799 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.800 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.801 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.802 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.803 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.804 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "organization": { 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "organization", 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.805 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.806 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "proxy": { 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "proxy", 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.807 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "url": { 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "url", 2024-04-26T21:21:07.808 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.809 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "test_orchestrator", 2024-04-26T21:21:07.810 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.811 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.812 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.813 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.814 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.815 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.816 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.817 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.818 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.819 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.820 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.821 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.822 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.822 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.822 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.822 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.822 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.822 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.822 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.823 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.824 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.824 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.824 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.824 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.824 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.824 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.824 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.825 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.826 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.827 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.828 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: "max_concurrent_clones": { 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "max_concurrent_clones", 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.829 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.830 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "4", 2024-04-26T21:21:07.830 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.830 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.830 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.830 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T21:21:07.830 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.830 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: "periodic_async_work": { 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "periodic_async_work", 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.831 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Periodically check for async work", 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.832 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_delay": { 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_delay", 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "0", 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.833 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.834 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.834 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T21:21:07.834 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.834 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.834 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.834 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.834 INFO:teuthology.orchestra.run.smithi090.stdout: "snapshot_clone_no_wait": { 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "True", 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.835 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.836 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix", 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "can_run": true, 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "error_string": "", 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "module_options": { 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "discovery_interval": { 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "discovery_interval", 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "uint", 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.837 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "100", 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.838 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.839 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.839 INFO:teuthology.orchestra.run.smithi090.stdout: "identifier": { 2024-04-26T21:21:07.839 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "identifier", 2024-04-26T21:21:07.839 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.839 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.839 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.839 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.840 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "interval": { 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "interval", 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "secs", 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "60", 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.841 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: "log_level": { 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_level", 2024-04-26T21:21:07.842 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.843 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.843 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.843 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.843 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.843 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.843 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.843 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.844 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster": { 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster", 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.845 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.846 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_cluster_level": { 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_cluster_level", 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "info", 2024-04-26T21:21:07.847 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [ 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "", 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "critical", 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "debug", 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "error", 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "info", 2024-04-26T21:21:07.848 INFO:teuthology.orchestra.run.smithi090.stdout: "warning" 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "log_to_file": { 2024-04-26T21:21:07.849 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "log_to_file", 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "bool", 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 1, 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "False", 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.850 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_host": { 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_host", 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.851 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "", 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.852 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_port": { 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_port", 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "int", 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.853 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "10051", 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.854 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "zabbix_sender": { 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "zabbix_sender", 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "str", 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "level": "advanced", 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "flags": 0, 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "min": "", 2024-04-26T21:21:07.855 INFO:teuthology.orchestra.run.smithi090.stdout: "max": "", 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: "enum_allowed": [], 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: "desc": "", 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: "long_desc": "", 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: "tags": [], 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: "see_also": [] 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.856 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: "services": { 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: "dashboard": "https://172.21.15.90:8443/" 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: "always_on_modules": { 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: "octopus": [ 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T21:21:07.857 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-26T21:21:07.858 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.859 INFO:teuthology.orchestra.run.smithi090.stdout: "pacific": [ 2024-04-26T21:21:07.859 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-26T21:21:07.859 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T21:21:07.859 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-26T21:21:07.859 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-26T21:21:07.859 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-26T21:21:07.859 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "quincy": [ 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T21:21:07.860 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-26T21:21:07.861 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "reef": [ 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-26T21:21:07.862 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: "squid": [ 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: "balancer", 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T21:21:07.863 INFO:teuthology.orchestra.run.smithi090.stdout: "devicehealth", 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: "orchestrator", 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: "pg_autoscaler", 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: "progress", 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: "rbd_support", 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: "status", 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: "telemetry", 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: "volumes" 2024-04-26T21:21:07.864 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: "last_failure_osd_epoch": 93, 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: "active_clients": [ 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "devicehealth", 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.865 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2162630739 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "libcephsqlite", 2024-04-26T21:21:07.866 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2988008865 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.867 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "rbd_support", 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 1387644418 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.868 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-26T21:21:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-26T21:21:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-26T21:21:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-26T21:21:07.869 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 515563834 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "snap_schedule", 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.870 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 2929304563 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: "name": "volumes", 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: "addrvec": [ 2024-04-26T21:21:07.871 INFO:teuthology.orchestra.run.smithi090.stdout: { 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout: "type": "v2", 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout: "addr": "172.21.15.90:0", 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout: "nonce": 935570053 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T21:21:07.872 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T21:21:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7de80a3d20 msgr2=0x7f7de80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7de80a3d20 0x7f7de80a61e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc06c0b0 tx=0x7f7de400a250 comp rx=0 tx=0).stop 2024-04-26T21:21:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc101800 msgr2=0x7f7dfc199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc101800 0x7f7dfc199120 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7df400ea60 tx=0x7f7df400ef20 comp rx=0 tx=0).stop 2024-04-26T21:21:07.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 shutdown_connections 2024-04-26T21:21:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f7de80a3d20 0x7f7de80a61e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dfc107810 0x7f7dfc19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7dfc1021c0 0x7f7dfc199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 --2- 172.21.15.90:0/2029655272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7dfc101800 0x7f7dfc199120 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 >> 172.21.15.90:0/2029655272 conn(0x7f7dfc077bb0 msgr2=0x7f7dfc105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 shutdown_connections 2024-04-26T21:21:07.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:05.348+0000 7f7e00924640 1 -- 172.21.15.90:0/2029655272 wait complete. 2024-04-26T21:21:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:05 smithi090 ceph-mon[29180]: pgmap v6037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:05 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1516479168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:05 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/3482139897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9947.127495 secs 2024-04-26T21:21:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:05 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1146536793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:05 smithi167 ceph-mon[32751]: pgmap v6037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:07.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:05 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1516479168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:05 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/3482139897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9947.127495 secs 2024-04-26T21:21:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:05 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1146536793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:06 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/2029655272' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:21:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9947.852557 secs 2024-04-26T21:21:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:07 smithi167 ceph-mon[32751]: pgmap v6038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:21:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:21:07.878 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:05 smithi119 ceph-mon[32752]: pgmap v6037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:05 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1516479168' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:05 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/3482139897' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9947.127495 secs 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:05 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1146536793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:06 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/2029655272' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9947.852557 secs 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:07 smithi119 ceph-mon[32752]: pgmap v6038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:21:07.879 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:21:07.887 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T21:21:07.887 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T21:21:07.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:06 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/2029655272' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T21:21:07.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9947.852557 secs 2024-04-26T21:21:07.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:07 smithi090 ceph-mon[29180]: pgmap v6038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:07.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:21:07.888 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:21:08.138 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:21:09.494 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:09 smithi090 ceph-mon[29180]: pgmap v6039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:09 smithi119 ceph-mon[32752]: pgmap v6039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:09 smithi167 ceph-mon[32751]: pgmap v6039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:09.830 INFO:teuthology.orchestra.run.smithi090.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-04-26T21:21:10.411 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9952.127580 secs 2024-04-26T21:21:10.437 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-26T21:21:10.438 ERROR:teuthology.run_tasks:Manager failed: check-counter Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/__init__.py", line 132, in __exit__ self.end() File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/check_counter.py", line 84, in end proc = manager.admin_socket(daemon_type, daemon_id, ["perf", "dump"]) File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/ceph_manager.py", line 1893, in admin_socket return shell( File "/home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks/ceph_manager.py", line 43, in shell return remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi090 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump' 2024-04-26T21:21:10.448 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T21:21:10.459 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T21:21:10.460 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T21:21:10.460 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:10.480 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T21:21:10.480 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:21:10.537 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid b8878a18-03f6-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-26T21:21:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9952.127580 secs 2024-04-26T21:21:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9952.127580 secs 2024-04-26T21:21:10.813 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/b8878a18-03f6-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T21:21:11.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:11 smithi090 ceph-mon[29180]: pgmap v6040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-26T21:21:11.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9952.852621 secs 2024-04-26T21:21:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:11 smithi119 ceph-mon[32752]: pgmap v6040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-26T21:21:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9952.852621 secs 2024-04-26T21:21:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:11 smithi167 ceph-mon[32751]: pgmap v6040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-26T21:21:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9952.852621 secs 2024-04-26T21:21:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 -- 172.21.15.90:0/3415014780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103470 msgr2=0x7f74f81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 --2- 172.21.15.90:0/3415014780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103470 0x7f74f81038f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f74e8009990 tx=0x7f74e8009ca0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 -- 172.21.15.90:0/3415014780 shutdown_connections 2024-04-26T21:21:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 --2- 172.21.15.90:0/3415014780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8103e30 0x7f74f8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 --2- 172.21.15.90:0/3415014780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f81094c0 0x7f74f81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 --2- 172.21.15.90:0/3415014780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103470 0x7f74f81038f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 -- 172.21.15.90:0/3415014780 >> 172.21.15.90:0/3415014780 conn(0x7f74f807a580 msgr2=0x7f74f807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 -- 172.21.15.90:0/3415014780 shutdown_connections 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.511+0000 7f74feba9700 1 -- 172.21.15.90:0/3415014780 wait complete. 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 Processor -- start 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 -- start start 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8103470 0x7f74f819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103e30 0x7f74f819fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f81094c0 0x7f74f8199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8113640 con 0x7f74f8103e30 2024-04-26T21:21:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 -- 0x7f74f81134c0 con 0x7f74f8103470 2024-04-26T21:21:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.512+0000 7f74feba9700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74f81137c0 con 0x7f74f81094c0 2024-04-26T21:21:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103e30 0x7f74f819fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74fc945700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8103470 0x7f74f819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103e30 0x7f74f819fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38198/0 (socket says 172.21.15.90:38198) 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 -- 172.21.15.90:0/1880947751 learned_addr learned my addr 172.21.15.90:0/1880947751 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74fd146700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f81094c0 0x7f74f8199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 -- 172.21.15.90:0/1880947751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f81094c0 msgr2=0x7f74f8199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f81094c0 0x7f74f8199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 -- 172.21.15.90:0/1880947751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8103470 msgr2=0x7f74f819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8103470 0x7f74f819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 -- 172.21.15.90:0/1880947751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f819a160 con 0x7f74f8103e30 2024-04-26T21:21:12.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74fc945700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8103470 0x7f74f819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:21:12.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f7fff700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103e30 0x7f74f819fd80 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f74e8000c00 tx=0x7f74e8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f5ffb700 1 -- 172.21.15.90:0/1880947751 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8027070 con 0x7f74f8103e30 2024-04-26T21:21:12.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f5ffb700 1 -- 172.21.15.90:0/1880947751 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74e8022470 con 0x7f74f8103e30 2024-04-26T21:21:12.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74f5ffb700 1 -- 172.21.15.90:0/1880947751 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8017650 con 0x7f74f8103e30 2024-04-26T21:21:12.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.513+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f819a3f0 con 0x7f74f8103e30 2024-04-26T21:21:12.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.514+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f81a6710 con 0x7f74f8103e30 2024-04-26T21:21:12.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.515+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74f806a110 con 0x7f74f8103e30 2024-04-26T21:21:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.516+0000 7f74f5ffb700 1 -- 172.21.15.90:0/1880947751 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) ==== 138777+0+0 (secure 0 0 0) 0x7f74e802a070 con 0x7f74f8103e30 2024-04-26T21:21:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.516+0000 7f74f5ffb700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f74e40a3e30 0x7f74e40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:21:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.516+0000 7f74fc945700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f74e40a3e30 0x7f74e40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:21:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.516+0000 7f74f5ffb700 1 -- 172.21.15.90:0/1880947751 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 13227+0+0 (secure 0 0 0) 0x7f74e80d27a0 con 0x7f74f8103e30 2024-04-26T21:21:12.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.517+0000 7f74fc945700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f74e40a3e30 0x7f74e40a62f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f74f819ab40 tx=0x7f74f000d040 comp rx=0 tx=0).ready entity=mgr.24752 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:21:12.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.521+0000 7f74f5ffb700 1 -- 172.21.15.90:0/1880947751 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74e8013070 con 0x7f74f8103e30 2024-04-26T21:21:12.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.734+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f74f819af60 con 0x7f74f8103e30 2024-04-26T21:21:12.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.734+0000 7f74f5ffb700 1 -- 172.21.15.90:0/1880947751 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 128 entries v183) ==== 82+0+7811 (secure 0 0 0) 0x7f74f819af60 con 0x7f74f8103e30 2024-04-26T21:21:12.736 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6837/2196548810 2024-04-27T18:34:42.471750+0000 2024-04-26T21:21:12.736 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6836/2196548810 2024-04-27T18:34:42.471750+0000 2024-04-26T21:21:12.736 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6836/2850537869 2024-04-27T18:33:56.077213+0000 2024-04-26T21:21:12.736 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6840/2568313147 2024-04-27T18:33:33.657527+0000 2024-04-26T21:21:12.736 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6839/2261571250 2024-04-27T18:34:42.464232+0000 2024-04-26T21:21:12.737 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6837/1881511598 2024-04-27T18:33:23.673790+0000 2024-04-26T21:21:12.737 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6841/349095443 2024-04-27T18:32:26.580352+0000 2024-04-26T21:21:12.737 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6840/349095443 2024-04-27T18:32:26.580352+0000 2024-04-26T21:21:12.737 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6836/3651305274 2024-04-27T18:32:26.161941+0000 2024-04-26T21:21:12.737 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6839/2673484677 2024-04-27T18:32:04.394949+0000 2024-04-26T21:21:12.737 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6838/2673484677 2024-04-27T18:32:04.394949+0000 2024-04-26T21:21:12.738 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6835/4072394290 2024-04-27T18:31:52.651357+0000 2024-04-26T21:21:12.738 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6834/4072394290 2024-04-27T18:31:52.651357+0000 2024-04-26T21:21:12.738 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6837/2470728764 2024-04-27T18:31:29.658307+0000 2024-04-26T21:21:12.738 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6835/2911456063 2024-04-27T18:31:08.868881+0000 2024-04-26T21:21:12.738 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6838/2261571250 2024-04-27T18:34:42.464232+0000 2024-04-26T21:21:12.738 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6833/1677033493 2024-04-27T18:30:17.628297+0000 2024-04-26T21:21:12.738 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6832/1677033493 2024-04-27T18:30:17.628297+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6833/37569032 2024-04-27T18:29:51.144130+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6841/3724478516 2024-04-27T18:29:08.636186+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6840/3724478516 2024-04-27T18:29:08.636186+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6833/117919007 2024-04-27T18:29:02.817125+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6839/874199594 2024-04-27T18:19:39.104871+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6836/3690677501 2024-04-27T18:21:48.374193+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6841/2568313147 2024-04-27T18:33:33.657527+0000 2024-04-26T21:21:12.739 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/484221164 2024-04-27T18:11:57.154935+0000 2024-04-26T21:21:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6839/4026544385 2024-04-27T18:21:27.335081+0000 2024-04-26T21:21:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6833/4236765679 2024-04-27T18:25:55.586295+0000 2024-04-26T21:21:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6839/2850927327 2024-04-27T18:22:12.517491+0000 2024-04-26T21:21:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6832/137310630 2024-04-27T18:24:38.623226+0000 2024-04-26T21:21:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6837/845543419 2024-04-27T18:20:52.104457+0000 2024-04-26T21:21:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6834/2911456063 2024-04-27T18:31:08.868881+0000 2024-04-26T21:21:12.740 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6838/4026544385 2024-04-27T18:21:27.335081+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6832/4236765679 2024-04-27T18:25:55.586295+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6838/1142719310 2024-04-27T18:20:30.686908+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6832/4202449285 2024-04-27T18:27:46.134304+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6837/3534880230 2024-04-27T18:19:03.863263+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6836/3885172962 2024-04-27T18:28:32.603623+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1492327080 2024-04-27T18:01:31.766717+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6836/845543419 2024-04-27T18:20:52.104457+0000 2024-04-26T21:21:12.741 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6840/720001018 2024-04-27T18:30:57.638207+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6839/862440561 2024-04-27T18:30:12.187956+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6838/4229007587 2024-04-27T18:18:32.371481+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6841/720001018 2024-04-27T18:30:57.638207+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6835/114254184 2024-04-27T18:27:14.404770+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6838/874199594 2024-04-27T18:19:39.104871+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6836/3534880230 2024-04-27T18:19:03.863263+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6834/3561315595 2024-04-27T18:28:09.257409+0000 2024-04-26T21:21:12.742 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6839/4229007587 2024-04-27T18:18:32.371481+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6832/117919007 2024-04-27T18:29:02.817125+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6834/2177592042 2024-04-27T18:24:02.157556+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6836/1335338695 2024-04-27T18:30:40.746838+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1868991041 2024-04-27T18:01:31.766717+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6833/137310630 2024-04-27T18:24:38.623226+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6836/1113931480 2024-04-27T18:23:37.536968+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6834/114254184 2024-04-27T18:27:14.404770+0000 2024-04-26T21:21:12.743 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6837/1187834156 2024-04-27T18:24:38.071710+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6834/641065147 2024-04-27T18:33:55.379573+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2846931038 2024-04-27T18:01:31.766717+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6836/2470728764 2024-04-27T18:31:29.658307+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6838/862440561 2024-04-27T18:30:12.187956+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/227996061 2024-04-27T18:01:03.883791+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6837/3885172962 2024-04-27T18:28:32.603623+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6836/4116054973 2024-04-27T18:22:45.044763+0000 2024-04-26T21:21:12.744 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6833/4202449285 2024-04-27T18:27:46.134304+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/295280840 2024-04-27T18:01:03.883791+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1501521801 2024-04-27T18:11:57.154935+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6832/2550962231 2024-04-27T18:26:53.240503+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/4035087696 2024-04-27T18:01:03.883791+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6837/2797511346 2024-04-27T18:35:07.565181+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6837/4116054973 2024-04-27T18:22:45.044763+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6836/1881511598 2024-04-27T18:33:23.673790+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3660618461 2024-04-27T18:11:57.154935+0000 2024-04-26T21:21:12.745 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6837/3690677501 2024-04-27T18:21:48.374193+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6832/37569032 2024-04-27T18:29:51.144130+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6839/2946680060 2024-04-27T18:29:29.402623+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6837/229859293 2024-04-27T18:19:59.025433+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6837/4250199742 2024-04-27T18:26:31.337524+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6840/3362405108 2024-04-27T18:29:51.138558+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6838/2946680060 2024-04-27T18:29:29.402623+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1868991041 2024-04-27T18:01:31.766717+0000 2024-04-26T21:21:12.746 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/651646115 2024-04-27T18:01:31.766717+0000 2024-04-26T21:21:12.747 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6834/2611460544 2024-04-27T18:23:06.054300+0000 2024-04-26T21:21:12.747 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6835/641065147 2024-04-27T18:33:55.379573+0000 2024-04-26T21:21:12.747 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6832/766230459 2024-04-27T18:24:03.896947+0000 2024-04-26T21:21:12.747 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6836/200145753 2024-04-27T18:23:41.447349+0000 2024-04-26T21:21:12.747 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6835/2611460544 2024-04-27T18:23:06.054300+0000 2024-04-26T21:21:12.747 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6833/766230459 2024-04-27T18:24:03.896947+0000 2024-04-26T21:21:12.747 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6838/1000850976 2024-04-27T18:33:02.666963+0000 2024-04-26T21:21:12.748 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6841/3362405108 2024-04-27T18:29:51.138558+0000 2024-04-26T21:21:12.748 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6837/1113931480 2024-04-27T18:23:37.536968+0000 2024-04-26T21:21:12.748 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6837/200145753 2024-04-27T18:23:41.447349+0000 2024-04-26T21:21:12.748 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6835/2177592042 2024-04-27T18:24:02.157556+0000 2024-04-26T21:21:12.748 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6833/3757608812 2024-04-27T18:24:59.368511+0000 2024-04-26T21:21:12.748 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6836/229859293 2024-04-27T18:19:59.025433+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6832/3757608812 2024-04-27T18:24:59.368511+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6832/1237239761 2024-04-27T18:25:55.597886+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6835/958836808 2024-04-27T18:25:23.814056+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6834/1044509479 2024-04-27T18:32:57.665121+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6836/3309576493 2024-04-27T18:25:34.885367+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6837/3309576493 2024-04-27T18:25:34.885367+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6839/1142719310 2024-04-27T18:20:30.686908+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/794009814 2024-04-27T18:01:03.883791+0000 2024-04-26T21:21:12.749 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6833/1237239761 2024-04-27T18:25:55.597886+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3660618461 2024-04-27T18:11:57.154935+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6834/3224484301 2024-04-27T18:26:30.320937+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6835/3224484301 2024-04-27T18:26:30.320937+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6839/1000850976 2024-04-27T18:33:02.666963+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6836/4250199742 2024-04-27T18:26:31.337524+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6832/925527879 2024-04-27T18:26:54.125475+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6835/1044509479 2024-04-27T18:32:57.665121+0000 2024-04-26T21:21:12.750 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1315801302 2024-04-27T18:11:57.154935+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6832/2849164668 2024-04-27T18:28:07.616546+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6837/3651305274 2024-04-27T18:32:26.161941+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/4035087696 2024-04-27T18:01:03.883791+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6833/925527879 2024-04-27T18:26:54.125475+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6834/958836808 2024-04-27T18:25:23.814056+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6836/1881734029 2024-04-27T18:27:37.159916+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6836/2797511346 2024-04-27T18:35:07.565181+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2718220414 2024-04-27T18:11:57.154935+0000 2024-04-26T21:21:12.751 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6833/2550962231 2024-04-27T18:26:53.240503+0000 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6837/1881734029 2024-04-27T18:27:37.159916+0000 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6837/2850537869 2024-04-27T18:33:56.077213+0000 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6837/1335338695 2024-04-27T18:30:40.746838+0000 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6838/2850927327 2024-04-27T18:22:12.517491+0000 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6833/2849164668 2024-04-27T18:28:07.616546+0000 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.167:6836/1187834156 2024-04-27T18:24:38.071710+0000 2024-04-26T21:21:12.752 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.119:6835/3561315595 2024-04-27T18:28:09.257409+0000 2024-04-26T21:21:12.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.736+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f74e40a3e30 msgr2=0x7f74e40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.736+0000 7f74feba9700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f74e40a3e30 0x7f74e40a62f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f74f819ab40 tx=0x7f74f000d040 comp rx=0 tx=0).stop 2024-04-26T21:21:12.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.736+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103e30 msgr2=0x7f74f819fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:21:12.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.736+0000 7f74feba9700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103e30 0x7f74f819fd80 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f74e8000c00 tx=0x7f74e8009580 comp rx=0 tx=0).stop 2024-04-26T21:21:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 shutdown_connections 2024-04-26T21:21:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:6800/2946218715,v1:172.21.15.90:6801/2946218715] conn(0x7f74e40a3e30 0x7f74e40a62f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74f81094c0 0x7f74f8199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74f8103470 0x7f74f819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 --2- 172.21.15.90:0/1880947751 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74f8103e30 0x7f74f819fd80 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:21:12.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 >> 172.21.15.90:0/1880947751 conn(0x7f74f807a580 msgr2=0x7f74f8102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:21:12.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 shutdown_connections 2024-04-26T21:21:12.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T21:21:12.737+0000 7f74feba9700 1 -- 172.21.15.90:0/1880947751 wait complete. 2024-04-26T21:21:12.755 INFO:teuthology.orchestra.run.smithi090.stderr:listed 128 entries 2024-04-26T21:21:13.466 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:13 smithi090 ceph-mon[29180]: pgmap v6041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-26T21:21:13.467 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:13 smithi090 ceph-mon[29180]: from='client.? 172.21.15.90:0/1880947751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:21:13.495 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T21:21:13.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:21:13.496 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T21:21:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:13 smithi119 ceph-mon[32752]: pgmap v6041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-26T21:21:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:13 smithi119 ceph-mon[32752]: from='client.? 172.21.15.90:0/1880947751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:21:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:13 smithi167 ceph-mon[32751]: pgmap v6041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-26T21:21:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:13 smithi167 ceph-mon[32751]: from='client.? 172.21.15.90:0/1880947751' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T21:21:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:15 smithi119 ceph-mon[32752]: pgmap v6042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-26T21:21:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9957.127664 secs 2024-04-26T21:21:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:15 smithi167 ceph-mon[32751]: pgmap v6042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-26T21:21:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9957.127664 secs 2024-04-26T21:21:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:15 smithi090 ceph-mon[29180]: pgmap v6042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 16 op/s 2024-04-26T21:21:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9957.127664 secs 2024-04-26T21:21:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9957.852754 secs 2024-04-26T21:21:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9957.852754 secs 2024-04-26T21:21:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9957.852754 secs 2024-04-26T21:21:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:21:17.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:17 smithi119 ceph-mon[32752]: pgmap v6043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-26T21:21:17.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:17.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:17.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:17 smithi119 ceph-mon[32752]: pgmap v6044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 285 B/s wr, 17 op/s 2024-04-26T21:21:17.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:17 smithi119 ceph-mon[32752]: pgmap v6045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 350 B/s wr, 21 op/s 2024-04-26T21:21:17.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:21:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:17 smithi167 ceph-mon[32751]: pgmap v6043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-26T21:21:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:17 smithi167 ceph-mon[32751]: pgmap v6044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 285 B/s wr, 17 op/s 2024-04-26T21:21:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:17 smithi167 ceph-mon[32751]: pgmap v6045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 350 B/s wr, 21 op/s 2024-04-26T21:21:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:21:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:17 smithi090 ceph-mon[29180]: pgmap v6043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 255 B/s wr, 15 op/s 2024-04-26T21:21:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:21:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:21:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:17 smithi090 ceph-mon[29180]: pgmap v6044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 285 B/s wr, 17 op/s 2024-04-26T21:21:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:17 smithi090 ceph-mon[29180]: pgmap v6045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 350 B/s wr, 21 op/s 2024-04-26T21:21:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:21:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:18 smithi090 ceph-mon[29180]: pgmap v6046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:18 smithi119 ceph-mon[32752]: pgmap v6046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:18 smithi167 ceph-mon[32751]: pgmap v6046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9962.127749 secs 2024-04-26T21:21:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9962.127749 secs 2024-04-26T21:21:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9962.127749 secs 2024-04-26T21:21:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9962.852876 secs 2024-04-26T21:21:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:21 smithi090 ceph-mon[29180]: pgmap v6047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9962.852876 secs 2024-04-26T21:21:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:21 smithi119 ceph-mon[32752]: pgmap v6047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9962.852876 secs 2024-04-26T21:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:21 smithi167 ceph-mon[32751]: pgmap v6047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:22 smithi090 ceph-mon[29180]: pgmap v6048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:21:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:22 smithi119 ceph-mon[32752]: pgmap v6048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:21:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:22 smithi167 ceph-mon[32751]: pgmap v6048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:21:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:24 smithi090 ceph-mon[29180]: pgmap v6049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:24 smithi119 ceph-mon[32752]: pgmap v6049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:24 smithi167 ceph-mon[32751]: pgmap v6049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:21:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9967.127834 secs 2024-04-26T21:21:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:25 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9967.853021 secs 2024-04-26T21:21:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9967.127834 secs 2024-04-26T21:21:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:25 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9967.853021 secs 2024-04-26T21:21:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9967.127834 secs 2024-04-26T21:21:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:25 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9967.853021 secs 2024-04-26T21:21:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:26 smithi090 ceph-mon[29180]: pgmap v6050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:21:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:26 smithi119 ceph-mon[32752]: pgmap v6050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:21:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:26 smithi167 ceph-mon[32751]: pgmap v6050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:21:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:28 smithi090 ceph-mon[29180]: pgmap v6051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:28 smithi119 ceph-mon[32752]: pgmap v6051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:28 smithi167 ceph-mon[32751]: pgmap v6051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9972.127895 secs 2024-04-26T21:21:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9972.127895 secs 2024-04-26T21:21:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9972.127895 secs 2024-04-26T21:21:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9972.853145 secs 2024-04-26T21:21:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:31 smithi090 ceph-mon[29180]: pgmap v6052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9972.853145 secs 2024-04-26T21:21:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:31 smithi119 ceph-mon[32752]: pgmap v6052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9972.853145 secs 2024-04-26T21:21:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:31 smithi167 ceph-mon[32751]: pgmap v6052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:32 smithi090 ceph-mon[29180]: pgmap v6053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:32 smithi119 ceph-mon[32752]: pgmap v6053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:32 smithi167 ceph-mon[32751]: pgmap v6053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:34 smithi090 ceph-mon[29180]: pgmap v6054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:34 smithi119 ceph-mon[32752]: pgmap v6054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:34 smithi167 ceph-mon[32751]: pgmap v6054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9977.127959 secs 2024-04-26T21:21:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:35 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9977.853289 secs 2024-04-26T21:21:36.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9977.127959 secs 2024-04-26T21:21:36.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:35 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9977.853289 secs 2024-04-26T21:21:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9977.127959 secs 2024-04-26T21:21:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:35 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9977.853289 secs 2024-04-26T21:21:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:36 smithi090 ceph-mon[29180]: pgmap v6055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:36 smithi119 ceph-mon[32752]: pgmap v6055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:36 smithi167 ceph-mon[32751]: pgmap v6055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:38 smithi090 ceph-mon[29180]: pgmap v6056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:39.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:38 smithi119 ceph-mon[32752]: pgmap v6056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:38 smithi167 ceph-mon[32751]: pgmap v6056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9982.128021 secs 2024-04-26T21:21:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9982.128021 secs 2024-04-26T21:21:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9982.128021 secs 2024-04-26T21:21:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9982.853428 secs 2024-04-26T21:21:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:41 smithi090 ceph-mon[29180]: pgmap v6057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9982.853428 secs 2024-04-26T21:21:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:41 smithi119 ceph-mon[32752]: pgmap v6057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9982.853428 secs 2024-04-26T21:21:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:41 smithi167 ceph-mon[32751]: pgmap v6057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:42 smithi090 ceph-mon[29180]: pgmap v6058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:42 smithi119 ceph-mon[32752]: pgmap v6058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:42 smithi167 ceph-mon[32751]: pgmap v6058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:44 smithi090 ceph-mon[29180]: pgmap v6059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:44 smithi119 ceph-mon[32752]: pgmap v6059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:44 smithi167 ceph-mon[32751]: pgmap v6059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9987.128086 secs 2024-04-26T21:21:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:45 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9987.853524 secs 2024-04-26T21:21:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9987.128086 secs 2024-04-26T21:21:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:45 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9987.853524 secs 2024-04-26T21:21:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9987.128086 secs 2024-04-26T21:21:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:45 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9987.853524 secs 2024-04-26T21:21:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:46 smithi090 ceph-mon[29180]: pgmap v6060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:47.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:46 smithi119 ceph-mon[32752]: pgmap v6060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:46 smithi167 ceph-mon[32751]: pgmap v6060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:49 smithi090 ceph-mon[29180]: pgmap v6061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:49 smithi119 ceph-mon[32752]: pgmap v6061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:49 smithi167 ceph-mon[32751]: pgmap v6061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9992.128149 secs 2024-04-26T21:21:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:50 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9992.853664 secs 2024-04-26T21:21:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9992.128149 secs 2024-04-26T21:21:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:50 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9992.853664 secs 2024-04-26T21:21:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9992.128149 secs 2024-04-26T21:21:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:50 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9992.853664 secs 2024-04-26T21:21:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:51 smithi090 ceph-mon[29180]: pgmap v6062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:51 smithi119 ceph-mon[32752]: pgmap v6062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:51 smithi167 ceph-mon[32751]: pgmap v6062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:53 smithi090 ceph-mon[29180]: pgmap v6063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:53 smithi119 ceph-mon[32752]: pgmap v6063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:53 smithi167 ceph-mon[32751]: pgmap v6063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:55 smithi090 ceph-mon[29180]: pgmap v6064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 9997.128235 secs 2024-04-26T21:21:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 9997.853804 secs 2024-04-26T21:21:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:55 smithi119 ceph-mon[32752]: pgmap v6064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 9997.128235 secs 2024-04-26T21:21:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 9997.853804 secs 2024-04-26T21:21:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:55 smithi167 ceph-mon[32751]: pgmap v6064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:21:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 9997.128235 secs 2024-04-26T21:21:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 9997.853804 secs 2024-04-26T21:21:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:57 smithi090 ceph-mon[29180]: pgmap v6065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:57 smithi119 ceph-mon[32752]: pgmap v6065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:21:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:57 smithi167 ceph-mon[32751]: pgmap v6065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:21:59 smithi090 ceph-mon[29180]: pgmap v6066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:21:59 smithi119 ceph-mon[32752]: pgmap v6066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:21:59 smithi167 ceph-mon[32751]: pgmap v6066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10002.128299 secs 2024-04-26T21:22:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:00 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10002.853912 secs 2024-04-26T21:22:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10002.128299 secs 2024-04-26T21:22:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:00 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10002.853912 secs 2024-04-26T21:22:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10002.128299 secs 2024-04-26T21:22:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:00 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10002.853912 secs 2024-04-26T21:22:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:01 smithi119 ceph-mon[32752]: pgmap v6067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:01 smithi167 ceph-mon[32751]: pgmap v6067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:01 smithi090 ceph-mon[29180]: pgmap v6067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:03 smithi119 ceph-mon[32752]: pgmap v6068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:03 smithi167 ceph-mon[32751]: pgmap v6068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:03 smithi090 ceph-mon[29180]: pgmap v6068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:05 smithi119 ceph-mon[32752]: pgmap v6069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:05 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 10007.128385 secs 2024-04-26T21:22:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:05 smithi119 ceph-mon[32752]: slow request 361.536629 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:22:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:05 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10007.854026 secs 2024-04-26T21:22:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:05 smithi167 ceph-mon[32751]: pgmap v6069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:05 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 10007.128385 secs 2024-04-26T21:22:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:05 smithi167 ceph-mon[32751]: slow request 361.536629 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:22:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:05 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10007.854026 secs 2024-04-26T21:22:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:05 smithi090 ceph-mon[29180]: pgmap v6069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:05 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 10007.128385 secs 2024-04-26T21:22:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:05 smithi090 ceph-mon[29180]: slow request 361.536629 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:22:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:05 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10007.854026 secs 2024-04-26T21:22:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:07 smithi119 ceph-mon[32752]: pgmap v6070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:07 smithi167 ceph-mon[32751]: pgmap v6070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:07 smithi090 ceph-mon[29180]: pgmap v6070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:10.205 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:09 smithi090 ceph-mon[29180]: pgmap v6071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:09 smithi119 ceph-mon[32752]: pgmap v6071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:09 smithi167 ceph-mon[32751]: pgmap v6071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10012.128467 secs 2024-04-26T21:22:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:10 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10012.854145 secs 2024-04-26T21:22:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10012.128467 secs 2024-04-26T21:22:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:10 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10012.854145 secs 2024-04-26T21:22:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10012.128467 secs 2024-04-26T21:22:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:10 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10012.854145 secs 2024-04-26T21:22:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:11 smithi119 ceph-mon[32752]: pgmap v6072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:11 smithi167 ceph-mon[32751]: pgmap v6072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:11 smithi090 ceph-mon[29180]: pgmap v6072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:13 smithi119 ceph-mon[32752]: pgmap v6073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:13 smithi167 ceph-mon[32751]: pgmap v6073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:13 smithi090 ceph-mon[29180]: pgmap v6073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:22:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:15 smithi119 ceph-mon[32752]: pgmap v6074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10017.128552 secs 2024-04-26T21:22:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:15 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10017.854272 secs 2024-04-26T21:22:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:15 smithi167 ceph-mon[32751]: pgmap v6074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10017.128552 secs 2024-04-26T21:22:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:15 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10017.854272 secs 2024-04-26T21:22:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:15 smithi090 ceph-mon[29180]: pgmap v6074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:22:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10017.128552 secs 2024-04-26T21:22:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:15 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10017.854272 secs 2024-04-26T21:22:17.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:22:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:17 smithi119 ceph-mon[32752]: pgmap v6075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:18.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:18.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:18.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:17 smithi119 ceph-mon[32752]: pgmap v6076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:22:18.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:17 smithi119 ceph-mon[32752]: pgmap v6077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:22:18.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:22:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:17 smithi167 ceph-mon[32751]: pgmap v6075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:18.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:18.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:18.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:17 smithi167 ceph-mon[32751]: pgmap v6076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:22:18.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:17 smithi167 ceph-mon[32751]: pgmap v6077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:22:18.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:22:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:17 smithi090 ceph-mon[29180]: pgmap v6075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:22:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:22:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:17 smithi090 ceph-mon[29180]: pgmap v6076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:22:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:17 smithi090 ceph-mon[29180]: pgmap v6077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:22:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:22:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:20 smithi119 ceph-mon[32752]: pgmap v6078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10022.128639 secs 2024-04-26T21:22:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:20 smithi167 ceph-mon[32751]: pgmap v6078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10022.128639 secs 2024-04-26T21:22:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:20 smithi090 ceph-mon[29180]: pgmap v6078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10022.128639 secs 2024-04-26T21:22:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10022.854398 secs 2024-04-26T21:22:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10022.854398 secs 2024-04-26T21:22:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10022.854398 secs 2024-04-26T21:22:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:22 smithi119 ceph-mon[32752]: pgmap v6079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:22 smithi167 ceph-mon[32751]: pgmap v6079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:22 smithi090 ceph-mon[29180]: pgmap v6079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:24 smithi119 ceph-mon[32752]: pgmap v6080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:24 smithi167 ceph-mon[32751]: pgmap v6080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:24 smithi090 ceph-mon[29180]: pgmap v6080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10027.128722 secs 2024-04-26T21:22:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10027.128722 secs 2024-04-26T21:22:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10027.128722 secs 2024-04-26T21:22:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:26 smithi090 ceph-mon[29180]: pgmap v6081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10027.854557 secs 2024-04-26T21:22:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:26 smithi119 ceph-mon[32752]: pgmap v6081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10027.854557 secs 2024-04-26T21:22:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:26 smithi167 ceph-mon[32751]: pgmap v6081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:22:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10027.854557 secs 2024-04-26T21:22:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:28 smithi090 ceph-mon[29180]: pgmap v6082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:22:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:28 smithi119 ceph-mon[32752]: pgmap v6082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:22:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:28 smithi167 ceph-mon[32751]: pgmap v6082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:22:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:30 smithi119 ceph-mon[32752]: pgmap v6083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10032.128827 secs 2024-04-26T21:22:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:30 smithi167 ceph-mon[32751]: pgmap v6083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10032.128827 secs 2024-04-26T21:22:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:30 smithi090 ceph-mon[29180]: pgmap v6083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10032.128827 secs 2024-04-26T21:22:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10032.854640 secs 2024-04-26T21:22:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10032.854640 secs 2024-04-26T21:22:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10032.854640 secs 2024-04-26T21:22:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:32 smithi119 ceph-mon[32752]: pgmap v6084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:32 smithi167 ceph-mon[32751]: pgmap v6084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:32 smithi090 ceph-mon[29180]: pgmap v6084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:34 smithi119 ceph-mon[32752]: pgmap v6085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:34 smithi167 ceph-mon[32751]: pgmap v6085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:34 smithi090 ceph-mon[29180]: pgmap v6085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10037.128911 secs 2024-04-26T21:22:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10037.128911 secs 2024-04-26T21:22:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10037.128911 secs 2024-04-26T21:22:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:36 smithi167 ceph-mon[32751]: pgmap v6086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10037.854734 secs 2024-04-26T21:22:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:36 smithi119 ceph-mon[32752]: pgmap v6086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10037.854734 secs 2024-04-26T21:22:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:36 smithi090 ceph-mon[29180]: pgmap v6086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10037.854734 secs 2024-04-26T21:22:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:38 smithi119 ceph-mon[32752]: pgmap v6087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:38 smithi167 ceph-mon[32751]: pgmap v6087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:38 smithi090 ceph-mon[29180]: pgmap v6087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:40 smithi119 ceph-mon[32752]: pgmap v6088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10042.129000 secs 2024-04-26T21:22:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:40 smithi167 ceph-mon[32751]: pgmap v6088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10042.129000 secs 2024-04-26T21:22:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:40 smithi090 ceph-mon[29180]: pgmap v6088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10042.129000 secs 2024-04-26T21:22:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10042.854858 secs 2024-04-26T21:22:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10042.854858 secs 2024-04-26T21:22:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10042.854858 secs 2024-04-26T21:22:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:42 smithi119 ceph-mon[32752]: pgmap v6089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:42 smithi167 ceph-mon[32751]: pgmap v6089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:42 smithi090 ceph-mon[29180]: pgmap v6089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:44 smithi119 ceph-mon[32752]: pgmap v6090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:44 smithi167 ceph-mon[32751]: pgmap v6090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:44 smithi090 ceph-mon[29180]: pgmap v6090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10047.129085 secs 2024-04-26T21:22:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10047.129085 secs 2024-04-26T21:22:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10047.129085 secs 2024-04-26T21:22:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:46 smithi119 ceph-mon[32752]: pgmap v6091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10047.855010 secs 2024-04-26T21:22:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:46 smithi167 ceph-mon[32751]: pgmap v6091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10047.855010 secs 2024-04-26T21:22:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:46 smithi090 ceph-mon[29180]: pgmap v6091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10047.855010 secs 2024-04-26T21:22:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:48 smithi119 ceph-mon[32752]: pgmap v6092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:48 smithi167 ceph-mon[32751]: pgmap v6092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:48 smithi090 ceph-mon[29180]: pgmap v6092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:50 smithi119 ceph-mon[32752]: pgmap v6093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10052.129172 secs 2024-04-26T21:22:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:50 smithi167 ceph-mon[32751]: pgmap v6093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10052.129172 secs 2024-04-26T21:22:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:50 smithi090 ceph-mon[29180]: pgmap v6093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10052.129172 secs 2024-04-26T21:22:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10052.855065 secs 2024-04-26T21:22:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10052.855065 secs 2024-04-26T21:22:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10052.855065 secs 2024-04-26T21:22:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:52 smithi119 ceph-mon[32752]: pgmap v6094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:52 smithi167 ceph-mon[32751]: pgmap v6094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:52 smithi090 ceph-mon[29180]: pgmap v6094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:54 smithi119 ceph-mon[32752]: pgmap v6095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:54 smithi167 ceph-mon[32751]: pgmap v6095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:54 smithi090 ceph-mon[29180]: pgmap v6095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:22:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10057.129261 secs 2024-04-26T21:22:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:55 smithi119 ceph-mon[32752]: pgmap v6096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10057.855199 secs 2024-04-26T21:22:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10057.129261 secs 2024-04-26T21:22:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:55 smithi167 ceph-mon[32751]: pgmap v6096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10057.855199 secs 2024-04-26T21:22:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10057.129261 secs 2024-04-26T21:22:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:55 smithi090 ceph-mon[29180]: pgmap v6096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10057.855199 secs 2024-04-26T21:22:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:22:58 smithi119 ceph-mon[32752]: pgmap v6097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:22:58 smithi167 ceph-mon[32751]: pgmap v6097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:22:59.079 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:22:58 smithi090 ceph-mon[29180]: pgmap v6097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:00 smithi119 ceph-mon[32752]: pgmap v6098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:00 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 10062.129343 secs 2024-04-26T21:23:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:00 smithi119 ceph-mon[32752]: slow request 720.127341 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:23:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:00 smithi167 ceph-mon[32751]: pgmap v6098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:00 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 10062.129343 secs 2024-04-26T21:23:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:00 smithi167 ceph-mon[32751]: slow request 720.127341 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:23:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:00 smithi090 ceph-mon[29180]: pgmap v6098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:00 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 10062.129343 secs 2024-04-26T21:23:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:00 smithi090 ceph-mon[29180]: slow request 720.127341 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:23:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10062.855336 secs 2024-04-26T21:23:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10062.855336 secs 2024-04-26T21:23:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10062.855336 secs 2024-04-26T21:23:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:02 smithi119 ceph-mon[32752]: pgmap v6099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:02 smithi167 ceph-mon[32751]: pgmap v6099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:02 smithi090 ceph-mon[29180]: pgmap v6099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:04 smithi119 ceph-mon[32752]: pgmap v6100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:04 smithi167 ceph-mon[32751]: pgmap v6100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:05.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:04 smithi090 ceph-mon[29180]: pgmap v6100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10067.129433 secs 2024-04-26T21:23:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10067.129433 secs 2024-04-26T21:23:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10067.129433 secs 2024-04-26T21:23:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:06 smithi167 ceph-mon[32751]: pgmap v6101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10067.855458 secs 2024-04-26T21:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:06 smithi119 ceph-mon[32752]: pgmap v6101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10067.855458 secs 2024-04-26T21:23:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:06 smithi090 ceph-mon[29180]: pgmap v6101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10067.855458 secs 2024-04-26T21:23:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:08 smithi119 ceph-mon[32752]: pgmap v6102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:08 smithi167 ceph-mon[32751]: pgmap v6102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:08 smithi090 ceph-mon[29180]: pgmap v6102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:10 smithi119 ceph-mon[32752]: pgmap v6103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10072.129535 secs 2024-04-26T21:23:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:10 smithi167 ceph-mon[32751]: pgmap v6103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10072.129535 secs 2024-04-26T21:23:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:10 smithi090 ceph-mon[29180]: pgmap v6103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10072.129535 secs 2024-04-26T21:23:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10072.855523 secs 2024-04-26T21:23:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10072.855523 secs 2024-04-26T21:23:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10072.855523 secs 2024-04-26T21:23:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:12 smithi119 ceph-mon[32752]: pgmap v6104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:12 smithi167 ceph-mon[32751]: pgmap v6104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:12 smithi090 ceph-mon[29180]: pgmap v6104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:14 smithi119 ceph-mon[32752]: pgmap v6105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:14 smithi167 ceph-mon[32751]: pgmap v6105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:14 smithi090 ceph-mon[29180]: pgmap v6105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10077.129618 secs 2024-04-26T21:23:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10077.129618 secs 2024-04-26T21:23:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10077.129618 secs 2024-04-26T21:23:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:16 smithi119 ceph-mon[32752]: pgmap v6106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10077.855683 secs 2024-04-26T21:23:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:16 smithi167 ceph-mon[32751]: pgmap v6106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10077.855683 secs 2024-04-26T21:23:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:16 smithi090 ceph-mon[29180]: pgmap v6106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10077.855683 secs 2024-04-26T21:23:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:23:18.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:18 smithi119 ceph-mon[32752]: pgmap v6107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:18.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:18.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:18.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:23:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:18 smithi167 ceph-mon[32751]: pgmap v6107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:23:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:18 smithi090 ceph-mon[29180]: pgmap v6107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:23:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:23:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:23:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:19 smithi119 ceph-mon[32752]: pgmap v6108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:23:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:19 smithi119 ceph-mon[32752]: pgmap v6109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:19 smithi167 ceph-mon[32751]: pgmap v6108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:23:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:19 smithi167 ceph-mon[32751]: pgmap v6109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:19 smithi090 ceph-mon[29180]: pgmap v6108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:23:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:19 smithi090 ceph-mon[29180]: pgmap v6109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10082.129703 secs 2024-04-26T21:23:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10082.129703 secs 2024-04-26T21:23:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10082.129703 secs 2024-04-26T21:23:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:21 smithi119 ceph-mon[32752]: pgmap v6110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10082.855812 secs 2024-04-26T21:23:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:21 smithi167 ceph-mon[32751]: pgmap v6110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10082.855812 secs 2024-04-26T21:23:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:21 smithi090 ceph-mon[29180]: pgmap v6110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10082.855812 secs 2024-04-26T21:23:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:23 smithi119 ceph-mon[32752]: pgmap v6111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:23:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:23 smithi167 ceph-mon[32751]: pgmap v6111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:23:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:23 smithi090 ceph-mon[29180]: pgmap v6111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:23:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:25 smithi119 ceph-mon[32752]: pgmap v6112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10087.129784 secs 2024-04-26T21:23:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:25 smithi167 ceph-mon[32751]: pgmap v6112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10087.129784 secs 2024-04-26T21:23:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:25 smithi090 ceph-mon[29180]: pgmap v6112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10087.129784 secs 2024-04-26T21:23:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10087.855890 secs 2024-04-26T21:23:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10087.855890 secs 2024-04-26T21:23:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10087.855890 secs 2024-04-26T21:23:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:27 smithi119 ceph-mon[32752]: pgmap v6113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:27 smithi167 ceph-mon[32751]: pgmap v6113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:27 smithi090 ceph-mon[29180]: pgmap v6113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:23:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:29 smithi119 ceph-mon[32752]: pgmap v6114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:23:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:29 smithi167 ceph-mon[32751]: pgmap v6114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:23:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:29 smithi090 ceph-mon[29180]: pgmap v6114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:23:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10092.129868 secs 2024-04-26T21:23:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10092.129868 secs 2024-04-26T21:23:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10092.129868 secs 2024-04-26T21:23:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:31 smithi119 ceph-mon[32752]: pgmap v6115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10092.856044 secs 2024-04-26T21:23:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:31 smithi167 ceph-mon[32751]: pgmap v6115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10092.856044 secs 2024-04-26T21:23:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:31 smithi090 ceph-mon[29180]: pgmap v6115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10092.856044 secs 2024-04-26T21:23:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:33 smithi119 ceph-mon[32752]: pgmap v6116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:33 smithi167 ceph-mon[32751]: pgmap v6116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:33 smithi090 ceph-mon[29180]: pgmap v6116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:35 smithi119 ceph-mon[32752]: pgmap v6117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10097.129952 secs 2024-04-26T21:23:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:35 smithi167 ceph-mon[32751]: pgmap v6117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10097.129952 secs 2024-04-26T21:23:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:35 smithi090 ceph-mon[29180]: pgmap v6117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10097.129952 secs 2024-04-26T21:23:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10097.856167 secs 2024-04-26T21:23:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10097.856167 secs 2024-04-26T21:23:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10097.856167 secs 2024-04-26T21:23:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:37 smithi119 ceph-mon[32752]: pgmap v6118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:37 smithi167 ceph-mon[32751]: pgmap v6118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:37 smithi090 ceph-mon[29180]: pgmap v6118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:39 smithi119 ceph-mon[32752]: pgmap v6119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:39 smithi167 ceph-mon[32751]: pgmap v6119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:39 smithi090 ceph-mon[29180]: pgmap v6119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10102.130039 secs 2024-04-26T21:23:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10102.130039 secs 2024-04-26T21:23:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10102.130039 secs 2024-04-26T21:23:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:41 smithi119 ceph-mon[32752]: pgmap v6120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10102.856299 secs 2024-04-26T21:23:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:41 smithi167 ceph-mon[32751]: pgmap v6120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10102.856299 secs 2024-04-26T21:23:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:41 smithi090 ceph-mon[29180]: pgmap v6120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10102.856299 secs 2024-04-26T21:23:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:43 smithi119 ceph-mon[32752]: pgmap v6121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:43 smithi167 ceph-mon[32751]: pgmap v6121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:43 smithi090 ceph-mon[29180]: pgmap v6121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:45 smithi119 ceph-mon[32752]: pgmap v6122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10107.130119 secs 2024-04-26T21:23:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:45 smithi167 ceph-mon[32751]: pgmap v6122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10107.130119 secs 2024-04-26T21:23:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:45 smithi090 ceph-mon[29180]: pgmap v6122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10107.130119 secs 2024-04-26T21:23:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10107.856428 secs 2024-04-26T21:23:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10107.856428 secs 2024-04-26T21:23:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10107.856428 secs 2024-04-26T21:23:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:47 smithi119 ceph-mon[32752]: pgmap v6123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:47 smithi167 ceph-mon[32751]: pgmap v6123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:47 smithi090 ceph-mon[29180]: pgmap v6123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:49 smithi119 ceph-mon[32752]: pgmap v6124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:49 smithi167 ceph-mon[32751]: pgmap v6124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:49 smithi090 ceph-mon[29180]: pgmap v6124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10112.130195 secs 2024-04-26T21:23:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10112.130195 secs 2024-04-26T21:23:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10112.130195 secs 2024-04-26T21:23:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:51 smithi119 ceph-mon[32752]: pgmap v6125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10112.856488 secs 2024-04-26T21:23:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:51 smithi167 ceph-mon[32751]: pgmap v6125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10112.856488 secs 2024-04-26T21:23:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:51 smithi090 ceph-mon[29180]: pgmap v6125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10112.856488 secs 2024-04-26T21:23:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:53 smithi119 ceph-mon[32752]: pgmap v6126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:53 smithi167 ceph-mon[32751]: pgmap v6126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:53 smithi090 ceph-mon[29180]: pgmap v6126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:55 smithi119 ceph-mon[32752]: pgmap v6127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10117.130281 secs 2024-04-26T21:23:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:55 smithi167 ceph-mon[32751]: pgmap v6127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10117.130281 secs 2024-04-26T21:23:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:55 smithi090 ceph-mon[29180]: pgmap v6127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:23:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10117.130281 secs 2024-04-26T21:23:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10117.856560 secs 2024-04-26T21:23:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10117.856560 secs 2024-04-26T21:23:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10117.856560 secs 2024-04-26T21:23:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:57 smithi119 ceph-mon[32752]: pgmap v6128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:57 smithi167 ceph-mon[32751]: pgmap v6128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:23:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:57 smithi090 ceph-mon[29180]: pgmap v6128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:23:59 smithi167 ceph-mon[32751]: pgmap v6129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:23:59 smithi090 ceph-mon[29180]: pgmap v6129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:23:59 smithi119 ceph-mon[32752]: pgmap v6129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 10122.130364 secs 2024-04-26T21:24:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 10122.130364 secs 2024-04-26T21:24:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 10122.130364 secs 2024-04-26T21:24:02.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:01 smithi090 ceph-mon[29180]: pgmap v6130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:02.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10122.856623 secs 2024-04-26T21:24:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:01 smithi167 ceph-mon[32751]: pgmap v6130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10122.856623 secs 2024-04-26T21:24:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:01 smithi119 ceph-mon[32752]: pgmap v6130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10122.856623 secs 2024-04-26T21:24:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:03 smithi090 ceph-mon[29180]: pgmap v6131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:03 smithi119 ceph-mon[32752]: pgmap v6131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:03 smithi167 ceph-mon[32751]: pgmap v6131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:06.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:05 smithi167 ceph-mon[32751]: pgmap v6132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:06.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:05 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 10127.130450 secs 2024-04-26T21:24:06.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:05 smithi167 ceph-mon[32751]: slow request 181.398867 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:24:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:05 smithi090 ceph-mon[29180]: pgmap v6132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:05 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 10127.130450 secs 2024-04-26T21:24:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:05 smithi090 ceph-mon[29180]: slow request 181.398867 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:24:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:05 smithi119 ceph-mon[32752]: pgmap v6132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:05 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 10127.130450 secs 2024-04-26T21:24:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:05 smithi119 ceph-mon[32752]: slow request 181.398867 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:24:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:06 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 10127.856741 secs 2024-04-26T21:24:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:06 smithi090 ceph-mon[29180]: slow request 1442.493508 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:24:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:06 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:24:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:06 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:24:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:06 smithi090 ceph-mon[29180]: pgmap v6133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:06 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 10127.856741 secs 2024-04-26T21:24:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:06 smithi119 ceph-mon[32752]: slow request 1442.493508 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:24:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:06 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:24:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:06 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:24:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:06 smithi119 ceph-mon[32752]: pgmap v6133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:06 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 10127.856741 secs 2024-04-26T21:24:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:06 smithi167 ceph-mon[32751]: slow request 1442.493508 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:24:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:06 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:24:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:06 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:24:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:06 smithi167 ceph-mon[32751]: pgmap v6133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:09 smithi119 ceph-mon[32752]: pgmap v6134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:09 smithi167 ceph-mon[32751]: pgmap v6134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:09 smithi090 ceph-mon[29180]: pgmap v6134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:10 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 10132.130536 secs 2024-04-26T21:24:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:10 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 10132.130536 secs 2024-04-26T21:24:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:10 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 10132.130536 secs 2024-04-26T21:24:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:11 smithi119 ceph-mon[32752]: pgmap v6135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10132.856832 secs 2024-04-26T21:24:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:11 smithi167 ceph-mon[32751]: pgmap v6135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10132.856832 secs 2024-04-26T21:24:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:11 smithi090 ceph-mon[29180]: pgmap v6135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10132.856832 secs 2024-04-26T21:24:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:13 smithi119 ceph-mon[32752]: pgmap v6136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:13 smithi167 ceph-mon[32751]: pgmap v6136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:13 smithi090 ceph-mon[29180]: pgmap v6136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:15 smithi119 ceph-mon[32752]: pgmap v6137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:15 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10137.130622 secs 2024-04-26T21:24:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:15 smithi119 ceph-mon[32752]: slow request 181.362357 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:24:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:15 smithi167 ceph-mon[32751]: pgmap v6137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:15 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10137.130622 secs 2024-04-26T21:24:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:15 smithi167 ceph-mon[32751]: slow request 181.362357 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:24:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:15 smithi090 ceph-mon[29180]: pgmap v6137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:15 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10137.130622 secs 2024-04-26T21:24:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:15 smithi090 ceph-mon[29180]: slow request 181.362357 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:24:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10137.856941 secs 2024-04-26T21:24:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10137.856941 secs 2024-04-26T21:24:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10137.856941 secs 2024-04-26T21:24:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:17 smithi119 ceph-mon[32752]: pgmap v6138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:17 smithi167 ceph-mon[32751]: pgmap v6138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:17 smithi090 ceph-mon[29180]: pgmap v6138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:19 smithi119 ceph-mon[32752]: pgmap v6139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:19 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:24:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:19 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:24:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:24:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:19 smithi167 ceph-mon[32751]: pgmap v6139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:19 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:24:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:19 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:24:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:24:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:19 smithi090 ceph-mon[29180]: pgmap v6139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:19 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:24:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:19 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:24:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:24:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:24:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:24:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:24:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:20 smithi119 ceph-mon[32752]: pgmap v6140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:24:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:20 smithi119 ceph-mon[32752]: pgmap v6141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10142.130707 secs 2024-04-26T21:24:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:20 smithi167 ceph-mon[32751]: pgmap v6140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:24:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:20 smithi167 ceph-mon[32751]: pgmap v6141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10142.130707 secs 2024-04-26T21:24:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:20 smithi090 ceph-mon[29180]: pgmap v6140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:24:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:20 smithi090 ceph-mon[29180]: pgmap v6141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10142.130707 secs 2024-04-26T21:24:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10142.857056 secs 2024-04-26T21:24:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10142.857056 secs 2024-04-26T21:24:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10142.857056 secs 2024-04-26T21:24:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:22 smithi119 ceph-mon[32752]: pgmap v6142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:22 smithi167 ceph-mon[32751]: pgmap v6142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:22 smithi090 ceph-mon[29180]: pgmap v6142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:24 smithi119 ceph-mon[32752]: pgmap v6143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:24:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:24 smithi167 ceph-mon[32751]: pgmap v6143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:24:24.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:24 smithi090 ceph-mon[29180]: pgmap v6143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:24:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10147.130796 secs 2024-04-26T21:24:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10147.130796 secs 2024-04-26T21:24:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10147.130796 secs 2024-04-26T21:24:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:26 smithi119 ceph-mon[32752]: pgmap v6144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10147.857134 secs 2024-04-26T21:24:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:26 smithi167 ceph-mon[32751]: pgmap v6144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10147.857134 secs 2024-04-26T21:24:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:26 smithi090 ceph-mon[29180]: pgmap v6144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:24:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10147.857134 secs 2024-04-26T21:24:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:28 smithi119 ceph-mon[32752]: pgmap v6145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:24:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:28 smithi167 ceph-mon[32751]: pgmap v6145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:24:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:28 smithi090 ceph-mon[29180]: pgmap v6145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:24:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:30 smithi119 ceph-mon[32752]: pgmap v6146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:24:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10152.130881 secs 2024-04-26T21:24:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:30 smithi167 ceph-mon[32751]: pgmap v6146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:24:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10152.130881 secs 2024-04-26T21:24:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:30 smithi090 ceph-mon[29180]: pgmap v6146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:24:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10152.130881 secs 2024-04-26T21:24:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10152.857288 secs 2024-04-26T21:24:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10152.857288 secs 2024-04-26T21:24:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10152.857288 secs 2024-04-26T21:24:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:32 smithi119 ceph-mon[32752]: pgmap v6147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:32 smithi167 ceph-mon[32751]: pgmap v6147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:32.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:32 smithi090 ceph-mon[29180]: pgmap v6147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:34 smithi119 ceph-mon[32752]: pgmap v6148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:34 smithi167 ceph-mon[32751]: pgmap v6148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:34.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:34 smithi090 ceph-mon[29180]: pgmap v6148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10157.130945 secs 2024-04-26T21:24:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10157.130945 secs 2024-04-26T21:24:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10157.130945 secs 2024-04-26T21:24:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:36 smithi119 ceph-mon[32752]: pgmap v6149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10157.857353 secs 2024-04-26T21:24:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:36 smithi167 ceph-mon[32751]: pgmap v6149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10157.857353 secs 2024-04-26T21:24:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:36 smithi090 ceph-mon[29180]: pgmap v6149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10157.857353 secs 2024-04-26T21:24:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:38 smithi119 ceph-mon[32752]: pgmap v6150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:38 smithi167 ceph-mon[32751]: pgmap v6150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:38 smithi090 ceph-mon[29180]: pgmap v6150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:40 smithi119 ceph-mon[32752]: pgmap v6151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10162.131033 secs 2024-04-26T21:24:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:40 smithi167 ceph-mon[32751]: pgmap v6151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10162.131033 secs 2024-04-26T21:24:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:40 smithi090 ceph-mon[29180]: pgmap v6151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10162.131033 secs 2024-04-26T21:24:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10162.857499 secs 2024-04-26T21:24:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10162.857499 secs 2024-04-26T21:24:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10162.857499 secs 2024-04-26T21:24:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:42 smithi119 ceph-mon[32752]: pgmap v6152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:42 smithi167 ceph-mon[32751]: pgmap v6152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:42 smithi090 ceph-mon[29180]: pgmap v6152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:44 smithi119 ceph-mon[32752]: pgmap v6153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:44 smithi167 ceph-mon[32751]: pgmap v6153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:44 smithi090 ceph-mon[29180]: pgmap v6153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10167.131095 secs 2024-04-26T21:24:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10167.131095 secs 2024-04-26T21:24:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10167.131095 secs 2024-04-26T21:24:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:46 smithi119 ceph-mon[32752]: pgmap v6154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10167.857619 secs 2024-04-26T21:24:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:46 smithi167 ceph-mon[32751]: pgmap v6154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10167.857619 secs 2024-04-26T21:24:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:46 smithi090 ceph-mon[29180]: pgmap v6154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10167.857619 secs 2024-04-26T21:24:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:48 smithi119 ceph-mon[32752]: pgmap v6155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:48 smithi167 ceph-mon[32751]: pgmap v6155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:48 smithi090 ceph-mon[29180]: pgmap v6155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:50 smithi119 ceph-mon[32752]: pgmap v6156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10172.131180 secs 2024-04-26T21:24:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:50 smithi167 ceph-mon[32751]: pgmap v6156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10172.131180 secs 2024-04-26T21:24:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:50 smithi090 ceph-mon[29180]: pgmap v6156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10172.131180 secs 2024-04-26T21:24:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10172.857739 secs 2024-04-26T21:24:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10172.857739 secs 2024-04-26T21:24:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10172.857739 secs 2024-04-26T21:24:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:52 smithi119 ceph-mon[32752]: pgmap v6157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:52 smithi167 ceph-mon[32751]: pgmap v6157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:52 smithi090 ceph-mon[29180]: pgmap v6157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:54 smithi119 ceph-mon[32752]: pgmap v6158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:54 smithi167 ceph-mon[32751]: pgmap v6158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:54 smithi090 ceph-mon[29180]: pgmap v6158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10177.131264 secs 2024-04-26T21:24:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10177.131264 secs 2024-04-26T21:24:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10177.131264 secs 2024-04-26T21:24:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:56 smithi119 ceph-mon[32752]: pgmap v6159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10177.857872 secs 2024-04-26T21:24:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:56 smithi167 ceph-mon[32751]: pgmap v6159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10177.857872 secs 2024-04-26T21:24:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:56 smithi090 ceph-mon[29180]: pgmap v6159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:24:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10177.857872 secs 2024-04-26T21:24:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:24:58 smithi119 ceph-mon[32752]: pgmap v6160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:24:58 smithi167 ceph-mon[32751]: pgmap v6160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:24:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:24:58 smithi090 ceph-mon[29180]: pgmap v6160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:00 smithi119 ceph-mon[32752]: pgmap v6161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10182.131350 secs 2024-04-26T21:25:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:00 smithi167 ceph-mon[32751]: pgmap v6161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10182.131350 secs 2024-04-26T21:25:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:00 smithi090 ceph-mon[29180]: pgmap v6161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10182.131350 secs 2024-04-26T21:25:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10182.858026 secs 2024-04-26T21:25:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10182.858026 secs 2024-04-26T21:25:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10182.858026 secs 2024-04-26T21:25:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:02 smithi167 ceph-mon[32751]: pgmap v6162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:02 smithi090 ceph-mon[29180]: pgmap v6162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:02 smithi119 ceph-mon[32752]: pgmap v6162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:04 smithi167 ceph-mon[32751]: pgmap v6163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:04 smithi090 ceph-mon[29180]: pgmap v6163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:04 smithi119 ceph-mon[32752]: pgmap v6163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10187.131436 secs 2024-04-26T21:25:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10187.131436 secs 2024-04-26T21:25:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10187.131436 secs 2024-04-26T21:25:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:06 smithi090 ceph-mon[29180]: pgmap v6164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10187.858161 secs 2024-04-26T21:25:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:06 smithi167 ceph-mon[32751]: pgmap v6164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10187.858161 secs 2024-04-26T21:25:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:06 smithi119 ceph-mon[32752]: pgmap v6164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:07.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10187.858161 secs 2024-04-26T21:25:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:08 smithi090 ceph-mon[29180]: pgmap v6165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:08 smithi119 ceph-mon[32752]: pgmap v6165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:08 smithi167 ceph-mon[32751]: pgmap v6165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:10 smithi090 ceph-mon[29180]: pgmap v6166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10192.131515 secs 2024-04-26T21:25:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:10 smithi119 ceph-mon[32752]: pgmap v6166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10192.131515 secs 2024-04-26T21:25:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:10 smithi167 ceph-mon[32751]: pgmap v6166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10192.131515 secs 2024-04-26T21:25:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10192.858239 secs 2024-04-26T21:25:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10192.858239 secs 2024-04-26T21:25:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10192.858239 secs 2024-04-26T21:25:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:12 smithi090 ceph-mon[29180]: pgmap v6167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:12 smithi119 ceph-mon[32752]: pgmap v6167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:12 smithi167 ceph-mon[32751]: pgmap v6167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:14 smithi090 ceph-mon[29180]: pgmap v6168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:14 smithi119 ceph-mon[32752]: pgmap v6168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:14 smithi167 ceph-mon[32751]: pgmap v6168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10197.131596 secs 2024-04-26T21:25:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10197.131596 secs 2024-04-26T21:25:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10197.131596 secs 2024-04-26T21:25:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:16 smithi090 ceph-mon[29180]: pgmap v6169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10197.858329 secs 2024-04-26T21:25:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:16 smithi119 ceph-mon[32752]: pgmap v6169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10197.858329 secs 2024-04-26T21:25:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:16 smithi167 ceph-mon[32751]: pgmap v6169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10197.858329 secs 2024-04-26T21:25:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:18 smithi090 ceph-mon[29180]: pgmap v6170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:18 smithi119 ceph-mon[32752]: pgmap v6170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:18 smithi167 ceph-mon[32751]: pgmap v6170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:25:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:20 smithi090 ceph-mon[29180]: pgmap v6171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10202.131688 secs 2024-04-26T21:25:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:25:20.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:20 smithi119 ceph-mon[32752]: pgmap v6171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:20.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10202.131688 secs 2024-04-26T21:25:20.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:20.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:20.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:25:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:20 smithi167 ceph-mon[32751]: pgmap v6171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10202.131688 secs 2024-04-26T21:25:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:25:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:25:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:25:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:21 smithi090 ceph-mon[29180]: pgmap v6172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:25:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:21 smithi090 ceph-mon[29180]: pgmap v6173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10202.858430 secs 2024-04-26T21:25:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:21 smithi119 ceph-mon[32752]: pgmap v6172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:25:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:21 smithi119 ceph-mon[32752]: pgmap v6173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10202.858430 secs 2024-04-26T21:25:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:21 smithi167 ceph-mon[32751]: pgmap v6172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:25:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:21 smithi167 ceph-mon[32751]: pgmap v6173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10202.858430 secs 2024-04-26T21:25:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:23 smithi090 ceph-mon[29180]: pgmap v6174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:23 smithi119 ceph-mon[32752]: pgmap v6174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:23 smithi167 ceph-mon[32751]: pgmap v6174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:25 smithi090 ceph-mon[29180]: pgmap v6175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10207.131773 secs 2024-04-26T21:25:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:25 smithi119 ceph-mon[32752]: pgmap v6175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10207.131773 secs 2024-04-26T21:25:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:25 smithi167 ceph-mon[32751]: pgmap v6175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10207.131773 secs 2024-04-26T21:25:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10207.858552 secs 2024-04-26T21:25:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10207.858552 secs 2024-04-26T21:25:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10207.858552 secs 2024-04-26T21:25:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:27 smithi090 ceph-mon[29180]: pgmap v6176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:27 smithi119 ceph-mon[32752]: pgmap v6176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:27 smithi167 ceph-mon[32751]: pgmap v6176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:25:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:29 smithi090 ceph-mon[29180]: pgmap v6177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:25:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:29 smithi119 ceph-mon[32752]: pgmap v6177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:25:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:29 smithi167 ceph-mon[32751]: pgmap v6177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:25:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10212.131823 secs 2024-04-26T21:25:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10212.131823 secs 2024-04-26T21:25:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10212.131823 secs 2024-04-26T21:25:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:31 smithi119 ceph-mon[32752]: pgmap v6178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:25:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10212.858617 secs 2024-04-26T21:25:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:31 smithi167 ceph-mon[32751]: pgmap v6178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:25:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10212.858617 secs 2024-04-26T21:25:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:31 smithi090 ceph-mon[29180]: pgmap v6178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:25:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10212.858617 secs 2024-04-26T21:25:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:33 smithi119 ceph-mon[32752]: pgmap v6179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:33 smithi167 ceph-mon[32751]: pgmap v6179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:33 smithi090 ceph-mon[29180]: pgmap v6179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:35 smithi119 ceph-mon[32752]: pgmap v6180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10217.131887 secs 2024-04-26T21:25:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:35 smithi167 ceph-mon[32751]: pgmap v6180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10217.131887 secs 2024-04-26T21:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:35 smithi090 ceph-mon[29180]: pgmap v6180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10217.131887 secs 2024-04-26T21:25:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10217.858731 secs 2024-04-26T21:25:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10217.858731 secs 2024-04-26T21:25:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10217.858731 secs 2024-04-26T21:25:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:37 smithi119 ceph-mon[32752]: pgmap v6181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:37 smithi167 ceph-mon[32751]: pgmap v6181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:37 smithi090 ceph-mon[29180]: pgmap v6181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:39 smithi119 ceph-mon[32752]: pgmap v6182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:39 smithi167 ceph-mon[32751]: pgmap v6182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:39 smithi090 ceph-mon[29180]: pgmap v6182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10222.131976 secs 2024-04-26T21:25:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10222.131976 secs 2024-04-26T21:25:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10222.131976 secs 2024-04-26T21:25:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:41 smithi119 ceph-mon[32752]: pgmap v6183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10222.858826 secs 2024-04-26T21:25:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:41 smithi167 ceph-mon[32751]: pgmap v6183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10222.858826 secs 2024-04-26T21:25:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:41 smithi090 ceph-mon[29180]: pgmap v6183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10222.858826 secs 2024-04-26T21:25:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:43 smithi119 ceph-mon[32752]: pgmap v6184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:43 smithi167 ceph-mon[32751]: pgmap v6184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:43 smithi090 ceph-mon[29180]: pgmap v6184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:45 smithi119 ceph-mon[32752]: pgmap v6185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10227.132035 secs 2024-04-26T21:25:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:45 smithi167 ceph-mon[32751]: pgmap v6185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10227.132035 secs 2024-04-26T21:25:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:45 smithi090 ceph-mon[29180]: pgmap v6185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10227.132035 secs 2024-04-26T21:25:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10227.858945 secs 2024-04-26T21:25:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10227.858945 secs 2024-04-26T21:25:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10227.858945 secs 2024-04-26T21:25:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:47 smithi119 ceph-mon[32752]: pgmap v6186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:47 smithi167 ceph-mon[32751]: pgmap v6186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:47 smithi090 ceph-mon[29180]: pgmap v6186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:49 smithi119 ceph-mon[32752]: pgmap v6187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:49 smithi167 ceph-mon[32751]: pgmap v6187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:49 smithi090 ceph-mon[29180]: pgmap v6187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:50.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10232.132100 secs 2024-04-26T21:25:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10232.132100 secs 2024-04-26T21:25:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10232.132100 secs 2024-04-26T21:25:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:51 smithi119 ceph-mon[32752]: pgmap v6188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10232.859066 secs 2024-04-26T21:25:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:51 smithi167 ceph-mon[32751]: pgmap v6188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10232.859066 secs 2024-04-26T21:25:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:51 smithi090 ceph-mon[29180]: pgmap v6188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10232.859066 secs 2024-04-26T21:25:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:53 smithi119 ceph-mon[32752]: pgmap v6189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:53 smithi167 ceph-mon[32751]: pgmap v6189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:53 smithi090 ceph-mon[29180]: pgmap v6189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:55 smithi119 ceph-mon[32752]: pgmap v6190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10237.132185 secs 2024-04-26T21:25:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:55 smithi167 ceph-mon[32751]: pgmap v6190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10237.132185 secs 2024-04-26T21:25:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:55 smithi090 ceph-mon[29180]: pgmap v6190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:25:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10237.132185 secs 2024-04-26T21:25:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10237.859161 secs 2024-04-26T21:25:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10237.859161 secs 2024-04-26T21:25:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10237.859161 secs 2024-04-26T21:25:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:57 smithi119 ceph-mon[32752]: pgmap v6191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:57 smithi167 ceph-mon[32751]: pgmap v6191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:25:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:57 smithi090 ceph-mon[29180]: pgmap v6191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:25:59 smithi119 ceph-mon[32752]: pgmap v6192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:25:59 smithi167 ceph-mon[32751]: pgmap v6192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:25:59 smithi090 ceph-mon[29180]: pgmap v6192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:01.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10242.132248 secs 2024-04-26T21:26:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10242.132248 secs 2024-04-26T21:26:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10242.132248 secs 2024-04-26T21:26:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:01 smithi119 ceph-mon[32752]: pgmap v6193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10242.859284 secs 2024-04-26T21:26:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:01 smithi167 ceph-mon[32751]: pgmap v6193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10242.859284 secs 2024-04-26T21:26:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:01 smithi090 ceph-mon[29180]: pgmap v6193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10242.859284 secs 2024-04-26T21:26:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:03 smithi090 ceph-mon[29180]: pgmap v6194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:03.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:26:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:26:03.547+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:26:03.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:26:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:26:03.547+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:26:03.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:26:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:26:03.547+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:26:03.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:26:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:26:03.547+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:26:03.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:26:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:26:03.547+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:26:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:03 smithi119 ceph-mon[32752]: pgmap v6194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:03 smithi167 ceph-mon[32751]: pgmap v6194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:05 smithi119 ceph-mon[32752]: pgmap v6195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:05 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10247.132335 secs 2024-04-26T21:26:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:05 smithi119 ceph-mon[32752]: slow request 5762.913777 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:26:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:05 smithi167 ceph-mon[32751]: pgmap v6195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:05 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10247.132335 secs 2024-04-26T21:26:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:05 smithi167 ceph-mon[32751]: slow request 5762.913777 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:26:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:05 smithi090 ceph-mon[29180]: pgmap v6195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:05 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10247.132335 secs 2024-04-26T21:26:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:05 smithi090 ceph-mon[29180]: slow request 5762.913777 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:26:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10247.859418 secs 2024-04-26T21:26:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10247.859418 secs 2024-04-26T21:26:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10247.859418 secs 2024-04-26T21:26:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:07 smithi119 ceph-mon[32752]: pgmap v6196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:07 smithi167 ceph-mon[32751]: pgmap v6196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:07 smithi090 ceph-mon[29180]: pgmap v6196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:09 smithi119 ceph-mon[32752]: pgmap v6197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:09 smithi167 ceph-mon[32751]: pgmap v6197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:09 smithi090 ceph-mon[29180]: pgmap v6197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10252.132421 secs 2024-04-26T21:26:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10252.132421 secs 2024-04-26T21:26:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10252.132421 secs 2024-04-26T21:26:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:11 smithi119 ceph-mon[32752]: pgmap v6198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10252.859536 secs 2024-04-26T21:26:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:11 smithi167 ceph-mon[32751]: pgmap v6198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10252.859536 secs 2024-04-26T21:26:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:11 smithi090 ceph-mon[29180]: pgmap v6198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10252.859536 secs 2024-04-26T21:26:13.499 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 745, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1374, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 747, in recv_stderr raise socket.timeout() socket.timeout 2024-04-26T21:26:13.509 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 697, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/buffered_pipe.py", line 154, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 908, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 109, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/file.py", line 275, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 1359, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/virtualenv/lib/python3.8/site-packages/paramiko/channel.py", line 699, in recv raise socket.timeout() socket.timeout 2024-04-26T21:26:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:13 smithi119 ceph-mon[32752]: pgmap v6199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:13 smithi167 ceph-mon[32751]: pgmap v6199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:13 smithi090 ceph-mon[29180]: pgmap v6199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:15 smithi119 ceph-mon[32752]: pgmap v6200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10257.132501 secs 2024-04-26T21:26:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:15 smithi167 ceph-mon[32751]: pgmap v6200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10257.132501 secs 2024-04-26T21:26:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:15 smithi090 ceph-mon[29180]: pgmap v6200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10257.132501 secs 2024-04-26T21:26:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10257.859656 secs 2024-04-26T21:26:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10257.859656 secs 2024-04-26T21:26:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10257.859656 secs 2024-04-26T21:26:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:17 smithi119 ceph-mon[32752]: pgmap v6201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:17 smithi167 ceph-mon[32751]: pgmap v6201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:17 smithi090 ceph-mon[29180]: pgmap v6201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:19 smithi119 ceph-mon[32752]: pgmap v6202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:19 smithi167 ceph-mon[32751]: pgmap v6202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:19 smithi090 ceph-mon[29180]: pgmap v6202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:26:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10262.132587 secs 2024-04-26T21:26:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10262.132587 secs 2024-04-26T21:26:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10262.132587 secs 2024-04-26T21:26:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:26:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:21 smithi119 ceph-mon[32752]: pgmap v6203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10262.859748 secs 2024-04-26T21:26:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:21 smithi167 ceph-mon[32751]: pgmap v6203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10262.859748 secs 2024-04-26T21:26:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:21 smithi090 ceph-mon[29180]: pgmap v6203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:26:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10262.859748 secs 2024-04-26T21:26:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: pgmap v6204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:23.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:23.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: pgmap v6204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: pgmap v6204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:26:23.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:26:23.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:26:23.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:26:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:26:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:26:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:26:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:26:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:26:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:26:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: pgmap v6205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:26:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:24 smithi090 ceph-mon[29180]: pgmap v6206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:26:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:26:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:26:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:26:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:26:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:26:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: pgmap v6205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:26:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:24 smithi119 ceph-mon[32752]: pgmap v6206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: pgmap v6205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:26:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:24 smithi167 ceph-mon[32751]: pgmap v6206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10267.132672 secs 2024-04-26T21:26:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10267.132672 secs 2024-04-26T21:26:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10267.132672 secs 2024-04-26T21:26:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:26 smithi090 ceph-mon[29180]: pgmap v6207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10267.859866 secs 2024-04-26T21:26:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:26 smithi119 ceph-mon[32752]: pgmap v6207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10267.859866 secs 2024-04-26T21:26:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:26 smithi167 ceph-mon[32751]: pgmap v6207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10267.859866 secs 2024-04-26T21:26:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:28 smithi090 ceph-mon[29180]: pgmap v6208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:28 smithi119 ceph-mon[32752]: pgmap v6208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:28 smithi167 ceph-mon[32751]: pgmap v6208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:29 smithi119 ceph-mon[32752]: pgmap v6209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:29 smithi167 ceph-mon[32751]: pgmap v6209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:29 smithi090 ceph-mon[29180]: pgmap v6209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:31.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10272.132753 secs 2024-04-26T21:26:31.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10272.859964 secs 2024-04-26T21:26:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10272.132753 secs 2024-04-26T21:26:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10272.859964 secs 2024-04-26T21:26:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10272.132753 secs 2024-04-26T21:26:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10272.859964 secs 2024-04-26T21:26:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:31 smithi119 ceph-mon[32752]: pgmap v6210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:31 smithi167 ceph-mon[32751]: pgmap v6210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:31 smithi090 ceph-mon[29180]: pgmap v6210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:26:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:34 smithi119 ceph-mon[32752]: pgmap v6211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:26:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:34 smithi167 ceph-mon[32751]: pgmap v6211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:26:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:34 smithi090 ceph-mon[29180]: pgmap v6211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:26:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10277.132839 secs 2024-04-26T21:26:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10277.132839 secs 2024-04-26T21:26:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10277.132839 secs 2024-04-26T21:26:36.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:36 smithi167 ceph-mon[32751]: pgmap v6212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:36.743 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10277.860087 secs 2024-04-26T21:26:36.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:36 smithi090 ceph-mon[29180]: pgmap v6212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:36.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10277.860087 secs 2024-04-26T21:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:36 smithi119 ceph-mon[32752]: pgmap v6212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10277.860087 secs 2024-04-26T21:26:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:38 smithi119 ceph-mon[32752]: pgmap v6213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:38 smithi167 ceph-mon[32751]: pgmap v6213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:38 smithi090 ceph-mon[29180]: pgmap v6213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:40 smithi119 ceph-mon[32752]: pgmap v6214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10282.132900 secs 2024-04-26T21:26:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:40 smithi167 ceph-mon[32751]: pgmap v6214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10282.132900 secs 2024-04-26T21:26:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:40 smithi090 ceph-mon[29180]: pgmap v6214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10282.132900 secs 2024-04-26T21:26:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10282.860209 secs 2024-04-26T21:26:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10282.860209 secs 2024-04-26T21:26:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10282.860209 secs 2024-04-26T21:26:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:42 smithi119 ceph-mon[32752]: pgmap v6215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:42 smithi167 ceph-mon[32751]: pgmap v6215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:42 smithi090 ceph-mon[29180]: pgmap v6215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:44.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:44 smithi119 ceph-mon[32752]: pgmap v6216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:44 smithi167 ceph-mon[32751]: pgmap v6216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:44 smithi090 ceph-mon[29180]: pgmap v6216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10287.132986 secs 2024-04-26T21:26:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10287.132986 secs 2024-04-26T21:26:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10287.132986 secs 2024-04-26T21:26:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:46 smithi119 ceph-mon[32752]: pgmap v6217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10287.860332 secs 2024-04-26T21:26:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:46 smithi167 ceph-mon[32751]: pgmap v6217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10287.860332 secs 2024-04-26T21:26:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:46 smithi090 ceph-mon[29180]: pgmap v6217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10287.860332 secs 2024-04-26T21:26:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:48 smithi119 ceph-mon[32752]: pgmap v6218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:48 smithi167 ceph-mon[32751]: pgmap v6218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:48 smithi090 ceph-mon[29180]: pgmap v6218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:50 smithi119 ceph-mon[32752]: pgmap v6219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10292.133042 secs 2024-04-26T21:26:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:50 smithi167 ceph-mon[32751]: pgmap v6219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10292.133042 secs 2024-04-26T21:26:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:50 smithi090 ceph-mon[29180]: pgmap v6219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10292.133042 secs 2024-04-26T21:26:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10292.860469 secs 2024-04-26T21:26:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10292.860469 secs 2024-04-26T21:26:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10292.860469 secs 2024-04-26T21:26:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:52 smithi119 ceph-mon[32752]: pgmap v6220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:52 smithi167 ceph-mon[32751]: pgmap v6220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:52 smithi090 ceph-mon[29180]: pgmap v6220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:54 smithi119 ceph-mon[32752]: pgmap v6221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:54 smithi167 ceph-mon[32751]: pgmap v6221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:54 smithi090 ceph-mon[29180]: pgmap v6221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:26:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10297.133118 secs 2024-04-26T21:26:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10297.133118 secs 2024-04-26T21:26:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10297.133118 secs 2024-04-26T21:26:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:56 smithi119 ceph-mon[32752]: pgmap v6222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10297.860586 secs 2024-04-26T21:26:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:56 smithi167 ceph-mon[32751]: pgmap v6222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10297.860586 secs 2024-04-26T21:26:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:56 smithi090 ceph-mon[29180]: pgmap v6222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10297.860586 secs 2024-04-26T21:26:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:26:58 smithi119 ceph-mon[32752]: pgmap v6223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:26:58 smithi167 ceph-mon[32751]: pgmap v6223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:26:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:26:58 smithi090 ceph-mon[29180]: pgmap v6223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:00 smithi119 ceph-mon[32752]: pgmap v6224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10302.133201 secs 2024-04-26T21:27:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:00 smithi167 ceph-mon[32751]: pgmap v6224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10302.133201 secs 2024-04-26T21:27:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:00 smithi090 ceph-mon[29180]: pgmap v6224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10302.133201 secs 2024-04-26T21:27:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10302.860712 secs 2024-04-26T21:27:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10302.860712 secs 2024-04-26T21:27:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10302.860712 secs 2024-04-26T21:27:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:02 smithi119 ceph-mon[32752]: pgmap v6225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:02 smithi167 ceph-mon[32751]: pgmap v6225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:02 smithi090 ceph-mon[29180]: pgmap v6225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:04 smithi119 ceph-mon[32752]: pgmap v6226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:04 smithi167 ceph-mon[32751]: pgmap v6226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:04 smithi090 ceph-mon[29180]: pgmap v6226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:05 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10307.133286 secs 2024-04-26T21:27:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:05 smithi119 ceph-mon[32752]: slow request 361.401703 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:27:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:05 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10307.133286 secs 2024-04-26T21:27:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:05 smithi167 ceph-mon[32751]: slow request 361.401703 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:27:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:05 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10307.133286 secs 2024-04-26T21:27:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:05 smithi090 ceph-mon[29180]: slow request 361.401703 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:27:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:06 smithi119 ceph-mon[32752]: pgmap v6227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10307.860793 secs 2024-04-26T21:27:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:06 smithi167 ceph-mon[32751]: pgmap v6227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10307.860793 secs 2024-04-26T21:27:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:06 smithi090 ceph-mon[29180]: pgmap v6227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10307.860793 secs 2024-04-26T21:27:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:08 smithi119 ceph-mon[32752]: pgmap v6228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:08 smithi167 ceph-mon[32751]: pgmap v6228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:08 smithi090 ceph-mon[29180]: pgmap v6228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:10 smithi119 ceph-mon[32752]: pgmap v6229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10312.133371 secs 2024-04-26T21:27:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:10 smithi167 ceph-mon[32751]: pgmap v6229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10312.133371 secs 2024-04-26T21:27:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:10 smithi090 ceph-mon[29180]: pgmap v6229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10312.133371 secs 2024-04-26T21:27:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10312.860882 secs 2024-04-26T21:27:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10312.860882 secs 2024-04-26T21:27:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10312.860882 secs 2024-04-26T21:27:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:12 smithi167 ceph-mon[32751]: pgmap v6230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:12 smithi090 ceph-mon[29180]: pgmap v6230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:12 smithi119 ceph-mon[32752]: pgmap v6230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:14 smithi167 ceph-mon[32751]: pgmap v6231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:14 smithi090 ceph-mon[29180]: pgmap v6231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:14 smithi119 ceph-mon[32752]: pgmap v6231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:15 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10317.133454 secs 2024-04-26T21:27:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:15 smithi167 ceph-mon[32751]: slow request 361.365189 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:27:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:15 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10317.133454 secs 2024-04-26T21:27:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:15 smithi090 ceph-mon[29180]: slow request 361.365189 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:27:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:15 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10317.133454 secs 2024-04-26T21:27:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:15 smithi119 ceph-mon[32752]: slow request 361.365189 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:27:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:16 smithi167 ceph-mon[32751]: pgmap v6232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10317.860959 secs 2024-04-26T21:27:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:16 smithi090 ceph-mon[29180]: pgmap v6232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10317.860959 secs 2024-04-26T21:27:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:16 smithi119 ceph-mon[32752]: pgmap v6232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10317.860959 secs 2024-04-26T21:27:18.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:18 smithi090 ceph-mon[29180]: pgmap v6233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:18 smithi119 ceph-mon[32752]: pgmap v6233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:18 smithi167 ceph-mon[32751]: pgmap v6233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:20.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:20 smithi119 ceph-mon[32752]: pgmap v6234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:20.856 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10322.133557 secs 2024-04-26T21:27:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:20 smithi090 ceph-mon[29180]: pgmap v6234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10322.133557 secs 2024-04-26T21:27:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:20 smithi167 ceph-mon[32751]: pgmap v6234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10322.133557 secs 2024-04-26T21:27:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10322.861094 secs 2024-04-26T21:27:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10322.861094 secs 2024-04-26T21:27:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10322.861094 secs 2024-04-26T21:27:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:22 smithi090 ceph-mon[29180]: pgmap v6235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:22 smithi119 ceph-mon[32752]: pgmap v6235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:22 smithi167 ceph-mon[32751]: pgmap v6235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:27:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:24 smithi090 ceph-mon[29180]: pgmap v6236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:24.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:24.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:24 smithi119 ceph-mon[32752]: pgmap v6236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:24 smithi167 ceph-mon[32751]: pgmap v6236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:25 smithi090 ceph-mon[29180]: pgmap v6237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T21:27:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:25 smithi090 ceph-mon[29180]: pgmap v6238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10327.133642 secs 2024-04-26T21:27:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:25 smithi119 ceph-mon[32752]: pgmap v6237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T21:27:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:25 smithi119 ceph-mon[32752]: pgmap v6238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10327.133642 secs 2024-04-26T21:27:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:27:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:27:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:25 smithi167 ceph-mon[32751]: pgmap v6237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T21:27:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:25 smithi167 ceph-mon[32751]: pgmap v6238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:27:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10327.133642 secs 2024-04-26T21:27:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10327.861249 secs 2024-04-26T21:27:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10327.861249 secs 2024-04-26T21:27:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10327.861249 secs 2024-04-26T21:27:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:27 smithi090 ceph-mon[29180]: pgmap v6239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:27 smithi119 ceph-mon[32752]: pgmap v6239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:27 smithi167 ceph-mon[32751]: pgmap v6239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:29 smithi119 ceph-mon[32752]: pgmap v6240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:29 smithi167 ceph-mon[32751]: pgmap v6240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:29 smithi090 ceph-mon[29180]: pgmap v6240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:31.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10332.133727 secs 2024-04-26T21:27:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10332.133727 secs 2024-04-26T21:27:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10332.133727 secs 2024-04-26T21:27:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10332.861404 secs 2024-04-26T21:27:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:31 smithi119 ceph-mon[32752]: pgmap v6241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10332.861404 secs 2024-04-26T21:27:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:31 smithi167 ceph-mon[32751]: pgmap v6241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10332.861404 secs 2024-04-26T21:27:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:31 smithi090 ceph-mon[29180]: pgmap v6241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:27:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:33 smithi119 ceph-mon[32752]: pgmap v6242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T21:27:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:33 smithi167 ceph-mon[32751]: pgmap v6242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T21:27:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:33 smithi090 ceph-mon[29180]: pgmap v6242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T21:27:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:35 smithi119 ceph-mon[32752]: pgmap v6243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:27:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10337.133815 secs 2024-04-26T21:27:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:35 smithi167 ceph-mon[32751]: pgmap v6243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:27:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10337.133815 secs 2024-04-26T21:27:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:35 smithi090 ceph-mon[29180]: pgmap v6243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:27:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10337.133815 secs 2024-04-26T21:27:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10337.861537 secs 2024-04-26T21:27:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10337.861537 secs 2024-04-26T21:27:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10337.861537 secs 2024-04-26T21:27:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:37 smithi119 ceph-mon[32752]: pgmap v6244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:37 smithi167 ceph-mon[32751]: pgmap v6244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:37 smithi090 ceph-mon[29180]: pgmap v6244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:39 smithi119 ceph-mon[32752]: pgmap v6245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:39 smithi167 ceph-mon[32751]: pgmap v6245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:39 smithi090 ceph-mon[29180]: pgmap v6245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10342.133899 secs 2024-04-26T21:27:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10342.133899 secs 2024-04-26T21:27:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10342.133899 secs 2024-04-26T21:27:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10342.861633 secs 2024-04-26T21:27:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:41 smithi119 ceph-mon[32752]: pgmap v6246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10342.861633 secs 2024-04-26T21:27:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:41 smithi167 ceph-mon[32751]: pgmap v6246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10342.861633 secs 2024-04-26T21:27:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:41 smithi090 ceph-mon[29180]: pgmap v6246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:43 smithi119 ceph-mon[32752]: pgmap v6247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:43 smithi167 ceph-mon[32751]: pgmap v6247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:43 smithi090 ceph-mon[29180]: pgmap v6247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:45 smithi119 ceph-mon[32752]: pgmap v6248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10347.133960 secs 2024-04-26T21:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:45 smithi167 ceph-mon[32751]: pgmap v6248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10347.133960 secs 2024-04-26T21:27:46.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:45 smithi090 ceph-mon[29180]: pgmap v6248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:46.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10347.133960 secs 2024-04-26T21:27:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10347.861742 secs 2024-04-26T21:27:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10347.861742 secs 2024-04-26T21:27:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10347.861742 secs 2024-04-26T21:27:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:47 smithi119 ceph-mon[32752]: pgmap v6249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:47 smithi167 ceph-mon[32751]: pgmap v6249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:47 smithi090 ceph-mon[29180]: pgmap v6249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:49 smithi119 ceph-mon[32752]: pgmap v6250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:49 smithi167 ceph-mon[32751]: pgmap v6250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:49 smithi090 ceph-mon[29180]: pgmap v6250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10352.134018 secs 2024-04-26T21:27:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10352.134018 secs 2024-04-26T21:27:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10352.134018 secs 2024-04-26T21:27:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10352.861857 secs 2024-04-26T21:27:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:51 smithi119 ceph-mon[32752]: pgmap v6251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10352.861857 secs 2024-04-26T21:27:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:51 smithi167 ceph-mon[32751]: pgmap v6251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10352.861857 secs 2024-04-26T21:27:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:51 smithi090 ceph-mon[29180]: pgmap v6251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:53 smithi119 ceph-mon[32752]: pgmap v6252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:53 smithi167 ceph-mon[32751]: pgmap v6252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:53 smithi090 ceph-mon[29180]: pgmap v6252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:55 smithi090 ceph-mon[29180]: pgmap v6253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10357.134101 secs 2024-04-26T21:27:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:55 smithi119 ceph-mon[32752]: pgmap v6253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10357.134101 secs 2024-04-26T21:27:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:55 smithi167 ceph-mon[32751]: pgmap v6253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:27:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10357.134101 secs 2024-04-26T21:27:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10357.861994 secs 2024-04-26T21:27:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10357.861994 secs 2024-04-26T21:27:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10357.861994 secs 2024-04-26T21:27:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:57 smithi119 ceph-mon[32752]: pgmap v6254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:57 smithi167 ceph-mon[32751]: pgmap v6254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:27:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:57 smithi090 ceph-mon[29180]: pgmap v6254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:27:59 smithi119 ceph-mon[32752]: pgmap v6255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:27:59 smithi167 ceph-mon[32751]: pgmap v6255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:27:59 smithi090 ceph-mon[29180]: pgmap v6255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10362.134185 secs 2024-04-26T21:28:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10362.134185 secs 2024-04-26T21:28:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10362.134185 secs 2024-04-26T21:28:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10362.862140 secs 2024-04-26T21:28:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:01 smithi119 ceph-mon[32752]: pgmap v6256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10362.862140 secs 2024-04-26T21:28:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:01 smithi167 ceph-mon[32751]: pgmap v6256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10362.862140 secs 2024-04-26T21:28:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:01 smithi090 ceph-mon[29180]: pgmap v6256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:03 smithi167 ceph-mon[32751]: pgmap v6257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:03 smithi090 ceph-mon[29180]: pgmap v6257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:03 smithi119 ceph-mon[32752]: pgmap v6257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:05 smithi167 ceph-mon[32751]: pgmap v6258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:05 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10367.134278 secs 2024-04-26T21:28:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:05 smithi167 ceph-mon[32751]: slow request 721.542522 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:28:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:05 smithi090 ceph-mon[29180]: pgmap v6258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:05 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10367.134278 secs 2024-04-26T21:28:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:05 smithi090 ceph-mon[29180]: slow request 721.542522 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:28:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:05 smithi119 ceph-mon[32752]: pgmap v6258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:05 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10367.134278 secs 2024-04-26T21:28:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:05 smithi119 ceph-mon[32752]: slow request 721.542522 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:28:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10367.862271 secs 2024-04-26T21:28:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10367.862271 secs 2024-04-26T21:28:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10367.862271 secs 2024-04-26T21:28:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:07 smithi167 ceph-mon[32751]: pgmap v6259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:07 smithi090 ceph-mon[29180]: pgmap v6259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:07 smithi119 ceph-mon[32752]: pgmap v6259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:09 smithi090 ceph-mon[29180]: pgmap v6260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:09 smithi119 ceph-mon[32752]: pgmap v6260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:09 smithi167 ceph-mon[32751]: pgmap v6260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10372.134363 secs 2024-04-26T21:28:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10372.134363 secs 2024-04-26T21:28:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10372.134363 secs 2024-04-26T21:28:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10372.862366 secs 2024-04-26T21:28:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:11 smithi090 ceph-mon[29180]: pgmap v6261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10372.862366 secs 2024-04-26T21:28:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:11 smithi119 ceph-mon[32752]: pgmap v6261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10372.862366 secs 2024-04-26T21:28:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:11 smithi167 ceph-mon[32751]: pgmap v6261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:13 smithi090 ceph-mon[29180]: pgmap v6262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:13 smithi119 ceph-mon[32752]: pgmap v6262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:13 smithi167 ceph-mon[32751]: pgmap v6262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:15 smithi090 ceph-mon[29180]: pgmap v6263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10377.134446 secs 2024-04-26T21:28:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:15 smithi119 ceph-mon[32752]: pgmap v6263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10377.134446 secs 2024-04-26T21:28:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:15 smithi167 ceph-mon[32751]: pgmap v6263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10377.134446 secs 2024-04-26T21:28:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10377.862490 secs 2024-04-26T21:28:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10377.862490 secs 2024-04-26T21:28:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10377.862490 secs 2024-04-26T21:28:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:17 smithi090 ceph-mon[29180]: pgmap v6264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:17 smithi119 ceph-mon[32752]: pgmap v6264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:17 smithi167 ceph-mon[32751]: pgmap v6264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:20.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:19 smithi119 ceph-mon[32752]: pgmap v6265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:19 smithi090 ceph-mon[29180]: pgmap v6265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:19 smithi167 ceph-mon[32751]: pgmap v6265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:20 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10382.134529 secs 2024-04-26T21:28:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:20 smithi090 ceph-mon[29180]: slow request 2881.193825 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:28:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:20 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10382.134529 secs 2024-04-26T21:28:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:20 smithi119 ceph-mon[32752]: slow request 2881.193825 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:28:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:20 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10382.134529 secs 2024-04-26T21:28:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:20 smithi167 ceph-mon[32751]: slow request 2881.193825 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:28:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10382.862580 secs 2024-04-26T21:28:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:21 smithi090 ceph-mon[29180]: pgmap v6266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10382.862580 secs 2024-04-26T21:28:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:21 smithi119 ceph-mon[32752]: pgmap v6266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10382.862580 secs 2024-04-26T21:28:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:21 smithi167 ceph-mon[32751]: pgmap v6266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:23 smithi090 ceph-mon[29180]: pgmap v6267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:23 smithi119 ceph-mon[32752]: pgmap v6267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:23 smithi167 ceph-mon[32751]: pgmap v6267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:28:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:25 smithi090 ceph-mon[29180]: pgmap v6268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10387.134601 secs 2024-04-26T21:28:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:28:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:25 smithi119 ceph-mon[32752]: pgmap v6268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10387.134601 secs 2024-04-26T21:28:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:28:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:25 smithi167 ceph-mon[32751]: pgmap v6268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10387.134601 secs 2024-04-26T21:28:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:28:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:28:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:28:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:26 smithi090 ceph-mon[29180]: pgmap v6269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:28:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:26 smithi090 ceph-mon[29180]: pgmap v6270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10387.862699 secs 2024-04-26T21:28:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:26 smithi119 ceph-mon[32752]: pgmap v6269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:28:27.477 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:26 smithi119 ceph-mon[32752]: pgmap v6270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:27.478 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10387.862699 secs 2024-04-26T21:28:27.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:26 smithi167 ceph-mon[32751]: pgmap v6269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:28:27.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:26 smithi167 ceph-mon[32751]: pgmap v6270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:27.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10387.862699 secs 2024-04-26T21:28:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:27 smithi090 ceph-mon[29180]: pgmap v6271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:28:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:27 smithi119 ceph-mon[32752]: pgmap v6271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:28:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:27 smithi167 ceph-mon[32751]: pgmap v6271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:28:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:30 smithi119 ceph-mon[32752]: pgmap v6272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10392.134683 secs 2024-04-26T21:28:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:30 smithi167 ceph-mon[32751]: pgmap v6272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10392.134683 secs 2024-04-26T21:28:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:30 smithi090 ceph-mon[29180]: pgmap v6272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10392.134683 secs 2024-04-26T21:28:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10392.862785 secs 2024-04-26T21:28:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10392.862785 secs 2024-04-26T21:28:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10392.862785 secs 2024-04-26T21:28:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:32 smithi119 ceph-mon[32752]: pgmap v6273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:32 smithi167 ceph-mon[32751]: pgmap v6273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:32 smithi090 ceph-mon[29180]: pgmap v6273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:34 smithi119 ceph-mon[32752]: pgmap v6274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:34 smithi167 ceph-mon[32751]: pgmap v6274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:34 smithi090 ceph-mon[29180]: pgmap v6274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:28:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10397.134769 secs 2024-04-26T21:28:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10397.134769 secs 2024-04-26T21:28:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10397.134769 secs 2024-04-26T21:28:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:36 smithi119 ceph-mon[32752]: pgmap v6275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:28:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10397.862930 secs 2024-04-26T21:28:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:36 smithi167 ceph-mon[32751]: pgmap v6275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:28:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10397.862930 secs 2024-04-26T21:28:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:36 smithi090 ceph-mon[29180]: pgmap v6275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:28:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10397.862930 secs 2024-04-26T21:28:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:38 smithi119 ceph-mon[32752]: pgmap v6276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:38 smithi167 ceph-mon[32751]: pgmap v6276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:38 smithi090 ceph-mon[29180]: pgmap v6276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:40 smithi167 ceph-mon[32751]: pgmap v6277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10402.134854 secs 2024-04-26T21:28:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:40 smithi090 ceph-mon[29180]: pgmap v6277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10402.134854 secs 2024-04-26T21:28:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:40 smithi119 ceph-mon[32752]: pgmap v6277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10402.134854 secs 2024-04-26T21:28:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10402.863075 secs 2024-04-26T21:28:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10402.863075 secs 2024-04-26T21:28:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10402.863075 secs 2024-04-26T21:28:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:42 smithi167 ceph-mon[32751]: pgmap v6278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:42 smithi090 ceph-mon[29180]: pgmap v6278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:42 smithi119 ceph-mon[32752]: pgmap v6278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:44 smithi090 ceph-mon[29180]: pgmap v6279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:44 smithi119 ceph-mon[32752]: pgmap v6279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:44 smithi167 ceph-mon[32751]: pgmap v6279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10407.134938 secs 2024-04-26T21:28:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10407.134938 secs 2024-04-26T21:28:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10407.134938 secs 2024-04-26T21:28:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:46 smithi090 ceph-mon[29180]: pgmap v6280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10407.863220 secs 2024-04-26T21:28:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:46 smithi119 ceph-mon[32752]: pgmap v6280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10407.863220 secs 2024-04-26T21:28:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:46 smithi167 ceph-mon[32751]: pgmap v6280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10407.863220 secs 2024-04-26T21:28:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:48 smithi090 ceph-mon[29180]: pgmap v6281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:48 smithi119 ceph-mon[32752]: pgmap v6281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:48 smithi167 ceph-mon[32751]: pgmap v6281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:50.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:50 smithi119 ceph-mon[32752]: pgmap v6282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:50.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10412.135027 secs 2024-04-26T21:28:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:50 smithi090 ceph-mon[29180]: pgmap v6282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10412.135027 secs 2024-04-26T21:28:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:50 smithi167 ceph-mon[32751]: pgmap v6282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10412.135027 secs 2024-04-26T21:28:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10412.863350 secs 2024-04-26T21:28:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10412.863350 secs 2024-04-26T21:28:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10412.863350 secs 2024-04-26T21:28:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:52 smithi090 ceph-mon[29180]: pgmap v6283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:52 smithi119 ceph-mon[32752]: pgmap v6283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:52 smithi167 ceph-mon[32751]: pgmap v6283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:54 smithi090 ceph-mon[29180]: pgmap v6284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:54 smithi119 ceph-mon[32752]: pgmap v6284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:54 smithi167 ceph-mon[32751]: pgmap v6284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:28:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10417.135089 secs 2024-04-26T21:28:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10417.135089 secs 2024-04-26T21:28:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10417.135089 secs 2024-04-26T21:28:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:56 smithi090 ceph-mon[29180]: pgmap v6285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10417.863480 secs 2024-04-26T21:28:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:56 smithi119 ceph-mon[32752]: pgmap v6285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10417.863480 secs 2024-04-26T21:28:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:56 smithi167 ceph-mon[32751]: pgmap v6285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10417.863480 secs 2024-04-26T21:28:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:28:58 smithi090 ceph-mon[29180]: pgmap v6286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:28:58 smithi119 ceph-mon[32752]: pgmap v6286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:28:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:28:58 smithi167 ceph-mon[32751]: pgmap v6286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:00 smithi090 ceph-mon[29180]: pgmap v6287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10422.135152 secs 2024-04-26T21:29:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:00 smithi119 ceph-mon[32752]: pgmap v6287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10422.135152 secs 2024-04-26T21:29:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:00 smithi167 ceph-mon[32751]: pgmap v6287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10422.135152 secs 2024-04-26T21:29:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10422.863649 secs 2024-04-26T21:29:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10422.863649 secs 2024-04-26T21:29:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10422.863649 secs 2024-04-26T21:29:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:02 smithi090 ceph-mon[29180]: pgmap v6288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:02 smithi119 ceph-mon[32752]: pgmap v6288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:02 smithi167 ceph-mon[32751]: pgmap v6288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:04 smithi090 ceph-mon[29180]: pgmap v6289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:04 smithi119 ceph-mon[32752]: pgmap v6289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:04 smithi167 ceph-mon[32751]: pgmap v6289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10427.135238 secs 2024-04-26T21:29:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10427.135238 secs 2024-04-26T21:29:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10427.135238 secs 2024-04-26T21:29:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:06 smithi090 ceph-mon[29180]: pgmap v6290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10427.863708 secs 2024-04-26T21:29:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:06 smithi167 ceph-mon[32751]: pgmap v6290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10427.863708 secs 2024-04-26T21:29:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:06 smithi119 ceph-mon[32752]: pgmap v6290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10427.863708 secs 2024-04-26T21:29:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:08 smithi090 ceph-mon[29180]: pgmap v6291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:08 smithi119 ceph-mon[32752]: pgmap v6291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:08 smithi167 ceph-mon[32751]: pgmap v6291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:10 smithi119 ceph-mon[32752]: pgmap v6292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10432.135299 secs 2024-04-26T21:29:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:10 smithi167 ceph-mon[32751]: pgmap v6292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10432.135299 secs 2024-04-26T21:29:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:10 smithi090 ceph-mon[29180]: pgmap v6292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10432.135299 secs 2024-04-26T21:29:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10432.863831 secs 2024-04-26T21:29:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10432.863831 secs 2024-04-26T21:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10432.863831 secs 2024-04-26T21:29:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:12 smithi090 ceph-mon[29180]: pgmap v6293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:12 smithi119 ceph-mon[32752]: pgmap v6293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:12 smithi167 ceph-mon[32751]: pgmap v6293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:14 smithi119 ceph-mon[32752]: pgmap v6294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:14 smithi167 ceph-mon[32751]: pgmap v6294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:14 smithi090 ceph-mon[29180]: pgmap v6294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10437.135385 secs 2024-04-26T21:29:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10437.135385 secs 2024-04-26T21:29:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10437.135385 secs 2024-04-26T21:29:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:16 smithi119 ceph-mon[32752]: pgmap v6295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10437.863948 secs 2024-04-26T21:29:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:16 smithi167 ceph-mon[32751]: pgmap v6295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10437.863948 secs 2024-04-26T21:29:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:16 smithi090 ceph-mon[29180]: pgmap v6295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10437.863948 secs 2024-04-26T21:29:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:18 smithi119 ceph-mon[32752]: pgmap v6296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:18 smithi167 ceph-mon[32751]: pgmap v6296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:18 smithi090 ceph-mon[29180]: pgmap v6296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:20 smithi119 ceph-mon[32752]: pgmap v6297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10442.135469 secs 2024-04-26T21:29:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:20 smithi167 ceph-mon[32751]: pgmap v6297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10442.135469 secs 2024-04-26T21:29:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:20 smithi090 ceph-mon[29180]: pgmap v6297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10442.135469 secs 2024-04-26T21:29:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10442.864053 secs 2024-04-26T21:29:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10442.864053 secs 2024-04-26T21:29:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10442.864053 secs 2024-04-26T21:29:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:22 smithi119 ceph-mon[32752]: pgmap v6298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:22 smithi167 ceph-mon[32751]: pgmap v6298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:22 smithi090 ceph-mon[29180]: pgmap v6298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:24 smithi119 ceph-mon[32752]: pgmap v6299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:24 smithi167 ceph-mon[32751]: pgmap v6299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:24 smithi090 ceph-mon[29180]: pgmap v6299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10447.135561 secs 2024-04-26T21:29:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10447.135561 secs 2024-04-26T21:29:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10447.135561 secs 2024-04-26T21:29:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:29:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:26 smithi119 ceph-mon[32752]: pgmap v6300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10447.864130 secs 2024-04-26T21:29:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:29:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:26 smithi167 ceph-mon[32751]: pgmap v6300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10447.864130 secs 2024-04-26T21:29:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:29:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:26 smithi090 ceph-mon[29180]: pgmap v6300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10447.864130 secs 2024-04-26T21:29:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:29:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:29:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:29:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:27 smithi119 ceph-mon[32752]: pgmap v6301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:29:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:27 smithi119 ceph-mon[32752]: pgmap v6302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:27 smithi167 ceph-mon[32751]: pgmap v6301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:29:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:27 smithi167 ceph-mon[32751]: pgmap v6302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:27 smithi090 ceph-mon[29180]: pgmap v6301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:29:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:27 smithi090 ceph-mon[29180]: pgmap v6302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:29 smithi119 ceph-mon[32752]: pgmap v6303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:29:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:29 smithi167 ceph-mon[32751]: pgmap v6303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:29:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:29 smithi090 ceph-mon[29180]: pgmap v6303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:29:31.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10452.135662 secs 2024-04-26T21:29:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10452.135662 secs 2024-04-26T21:29:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10452.135662 secs 2024-04-26T21:29:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:31 smithi119 ceph-mon[32752]: pgmap v6304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10452.864225 secs 2024-04-26T21:29:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:31 smithi167 ceph-mon[32751]: pgmap v6304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10452.864225 secs 2024-04-26T21:29:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:31 smithi090 ceph-mon[29180]: pgmap v6304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10452.864225 secs 2024-04-26T21:29:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:33 smithi119 ceph-mon[32752]: pgmap v6305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:29:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:33 smithi167 ceph-mon[32751]: pgmap v6305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:29:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:33 smithi090 ceph-mon[29180]: pgmap v6305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:29:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:35 smithi119 ceph-mon[32752]: pgmap v6306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10457.135722 secs 2024-04-26T21:29:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:35 smithi167 ceph-mon[32751]: pgmap v6306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10457.135722 secs 2024-04-26T21:29:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:35 smithi090 ceph-mon[29180]: pgmap v6306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:29:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10457.135722 secs 2024-04-26T21:29:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10457.864348 secs 2024-04-26T21:29:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10457.864348 secs 2024-04-26T21:29:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10457.864348 secs 2024-04-26T21:29:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:37 smithi119 ceph-mon[32752]: pgmap v6307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:29:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:37 smithi167 ceph-mon[32751]: pgmap v6307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:29:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:37 smithi090 ceph-mon[29180]: pgmap v6307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:29:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:39 smithi119 ceph-mon[32752]: pgmap v6308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:39 smithi167 ceph-mon[32751]: pgmap v6308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:39 smithi090 ceph-mon[29180]: pgmap v6308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10462.135808 secs 2024-04-26T21:29:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:41 smithi167 ceph-mon[32751]: pgmap v6309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10462.864454 secs 2024-04-26T21:29:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10462.135808 secs 2024-04-26T21:29:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:41 smithi090 ceph-mon[29180]: pgmap v6309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10462.864454 secs 2024-04-26T21:29:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10462.135808 secs 2024-04-26T21:29:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:41 smithi119 ceph-mon[32752]: pgmap v6309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10462.864454 secs 2024-04-26T21:29:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:43 smithi090 ceph-mon[29180]: pgmap v6310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:43 smithi119 ceph-mon[32752]: pgmap v6310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:43 smithi167 ceph-mon[32751]: pgmap v6310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:45 smithi090 ceph-mon[29180]: pgmap v6311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10467.135893 secs 2024-04-26T21:29:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:45 smithi119 ceph-mon[32752]: pgmap v6311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10467.135893 secs 2024-04-26T21:29:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:45 smithi167 ceph-mon[32751]: pgmap v6311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10467.135893 secs 2024-04-26T21:29:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10467.864574 secs 2024-04-26T21:29:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10467.864574 secs 2024-04-26T21:29:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10467.864574 secs 2024-04-26T21:29:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:47 smithi090 ceph-mon[29180]: pgmap v6312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:47 smithi119 ceph-mon[32752]: pgmap v6312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:47 smithi167 ceph-mon[32751]: pgmap v6312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:49 smithi090 ceph-mon[29180]: pgmap v6313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:49 smithi119 ceph-mon[32752]: pgmap v6313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:49 smithi167 ceph-mon[32751]: pgmap v6313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10472.136001 secs 2024-04-26T21:29:50.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10472.136001 secs 2024-04-26T21:29:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10472.136001 secs 2024-04-26T21:29:51.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:51 smithi090 ceph-mon[29180]: pgmap v6314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:51.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10472.864695 secs 2024-04-26T21:29:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:51 smithi119 ceph-mon[32752]: pgmap v6314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10472.864695 secs 2024-04-26T21:29:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:51 smithi167 ceph-mon[32751]: pgmap v6314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10472.864695 secs 2024-04-26T21:29:53.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:53 smithi090 ceph-mon[29180]: pgmap v6315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:53 smithi119 ceph-mon[32752]: pgmap v6315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:53 smithi167 ceph-mon[32751]: pgmap v6315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:55 smithi090 ceph-mon[29180]: pgmap v6316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10477.136061 secs 2024-04-26T21:29:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:55 smithi119 ceph-mon[32752]: pgmap v6316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10477.136061 secs 2024-04-26T21:29:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:55 smithi167 ceph-mon[32751]: pgmap v6316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:29:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10477.136061 secs 2024-04-26T21:29:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10477.864804 secs 2024-04-26T21:29:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10477.864804 secs 2024-04-26T21:29:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10477.864804 secs 2024-04-26T21:29:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:57 smithi090 ceph-mon[29180]: pgmap v6317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:57 smithi119 ceph-mon[32752]: pgmap v6317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:57 smithi167 ceph-mon[32751]: pgmap v6317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:29:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:29:59 smithi090 ceph-mon[29180]: pgmap v6318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:29:59 smithi119 ceph-mon[32752]: pgmap v6318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:29:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:29:59 smithi167 ceph-mon[32751]: pgmap v6318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:30:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10482.136146 secs 2024-04-26T21:30:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:30:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10482.136146 secs 2024-04-26T21:30:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:30:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10482.136146 secs 2024-04-26T21:30:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:30:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:01 smithi090 ceph-mon[29180]: pgmap v6319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10482.864925 secs 2024-04-26T21:30:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:01 smithi119 ceph-mon[32752]: pgmap v6319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10482.864925 secs 2024-04-26T21:30:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:01 smithi167 ceph-mon[32751]: pgmap v6319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10482.864925 secs 2024-04-26T21:30:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:03 smithi090 ceph-mon[29180]: pgmap v6320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:03 smithi119 ceph-mon[32752]: pgmap v6320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:03 smithi167 ceph-mon[32751]: pgmap v6320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:05 smithi090 ceph-mon[29180]: pgmap v6321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10487.136233 secs 2024-04-26T21:30:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:05 smithi119 ceph-mon[32752]: pgmap v6321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10487.136233 secs 2024-04-26T21:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:05 smithi167 ceph-mon[32751]: pgmap v6321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10487.136233 secs 2024-04-26T21:30:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10487.865043 secs 2024-04-26T21:30:06.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10487.865043 secs 2024-04-26T21:30:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10487.865043 secs 2024-04-26T21:30:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:07 smithi090 ceph-mon[29180]: pgmap v6322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:07 smithi119 ceph-mon[32752]: pgmap v6322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:07 smithi167 ceph-mon[32751]: pgmap v6322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:09 smithi119 ceph-mon[32752]: pgmap v6323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:09 smithi167 ceph-mon[32751]: pgmap v6323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:09 smithi090 ceph-mon[29180]: pgmap v6323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10492.136319 secs 2024-04-26T21:30:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10492.136319 secs 2024-04-26T21:30:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10492.136319 secs 2024-04-26T21:30:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:11 smithi119 ceph-mon[32752]: pgmap v6324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10492.865115 secs 2024-04-26T21:30:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:11 smithi167 ceph-mon[32751]: pgmap v6324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10492.865115 secs 2024-04-26T21:30:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:11 smithi090 ceph-mon[29180]: pgmap v6324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10492.865115 secs 2024-04-26T21:30:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:13 smithi119 ceph-mon[32752]: pgmap v6325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:13 smithi167 ceph-mon[32751]: pgmap v6325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:13 smithi090 ceph-mon[29180]: pgmap v6325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:15.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:15 smithi090 ceph-mon[29180]: pgmap v6326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:15.645 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10497.136405 secs 2024-04-26T21:30:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:15 smithi119 ceph-mon[32752]: pgmap v6326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10497.136405 secs 2024-04-26T21:30:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:15 smithi167 ceph-mon[32751]: pgmap v6326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10497.136405 secs 2024-04-26T21:30:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10497.865207 secs 2024-04-26T21:30:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10497.865207 secs 2024-04-26T21:30:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10497.865207 secs 2024-04-26T21:30:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:17 smithi119 ceph-mon[32752]: pgmap v6327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:17 smithi167 ceph-mon[32751]: pgmap v6327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:17 smithi090 ceph-mon[29180]: pgmap v6327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:19 smithi119 ceph-mon[32752]: pgmap v6328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:19 smithi167 ceph-mon[32751]: pgmap v6328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:19 smithi090 ceph-mon[29180]: pgmap v6328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:20.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10502.136489 secs 2024-04-26T21:30:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10502.136489 secs 2024-04-26T21:30:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10502.136489 secs 2024-04-26T21:30:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:21 smithi119 ceph-mon[32752]: pgmap v6329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10502.865327 secs 2024-04-26T21:30:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:21 smithi167 ceph-mon[32751]: pgmap v6329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10502.865327 secs 2024-04-26T21:30:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:21 smithi090 ceph-mon[29180]: pgmap v6329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10502.865327 secs 2024-04-26T21:30:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:23 smithi119 ceph-mon[32752]: pgmap v6330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:23 smithi167 ceph-mon[32751]: pgmap v6330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:23 smithi090 ceph-mon[29180]: pgmap v6330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:25 smithi119 ceph-mon[32752]: pgmap v6331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10507.136566 secs 2024-04-26T21:30:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:25 smithi167 ceph-mon[32751]: pgmap v6331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10507.136566 secs 2024-04-26T21:30:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:25 smithi090 ceph-mon[29180]: pgmap v6331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10507.136566 secs 2024-04-26T21:30:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10507.865447 secs 2024-04-26T21:30:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10507.865447 secs 2024-04-26T21:30:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10507.865447 secs 2024-04-26T21:30:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:30:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:27 smithi119 ceph-mon[32752]: pgmap v6332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:30:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:27 smithi167 ceph-mon[32751]: pgmap v6332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:30:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:27 smithi090 ceph-mon[29180]: pgmap v6332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:30:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:30:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:30:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:28 smithi119 ceph-mon[32752]: pgmap v6333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:30:28.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:28 smithi119 ceph-mon[32752]: pgmap v6334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:30:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:28 smithi167 ceph-mon[32751]: pgmap v6333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:30:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:28 smithi167 ceph-mon[32751]: pgmap v6334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:30:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:28 smithi090 ceph-mon[29180]: pgmap v6333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:30:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:28 smithi090 ceph-mon[29180]: pgmap v6334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:30:30.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:30 smithi090 ceph-mon[29180]: pgmap v6335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:30.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10512.136648 secs 2024-04-26T21:30:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:30 smithi119 ceph-mon[32752]: pgmap v6335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10512.136648 secs 2024-04-26T21:30:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:30 smithi167 ceph-mon[32751]: pgmap v6335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10512.136648 secs 2024-04-26T21:30:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10512.865574 secs 2024-04-26T21:30:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10512.865574 secs 2024-04-26T21:30:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10512.865574 secs 2024-04-26T21:30:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:32 smithi119 ceph-mon[32752]: pgmap v6336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:32 smithi167 ceph-mon[32751]: pgmap v6336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:32 smithi090 ceph-mon[29180]: pgmap v6336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:34 smithi119 ceph-mon[32752]: pgmap v6337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:30:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:34 smithi167 ceph-mon[32751]: pgmap v6337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:30:34.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:34 smithi090 ceph-mon[29180]: pgmap v6337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:30:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10517.136733 secs 2024-04-26T21:30:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10517.136733 secs 2024-04-26T21:30:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10517.136733 secs 2024-04-26T21:30:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:36 smithi167 ceph-mon[32751]: pgmap v6338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10517.865694 secs 2024-04-26T21:30:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:36 smithi119 ceph-mon[32752]: pgmap v6338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10517.865694 secs 2024-04-26T21:30:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:36 smithi090 ceph-mon[29180]: pgmap v6338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:30:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10517.865694 secs 2024-04-26T21:30:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:38 smithi119 ceph-mon[32752]: pgmap v6339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:30:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:38 smithi167 ceph-mon[32751]: pgmap v6339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:30:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:38 smithi090 ceph-mon[29180]: pgmap v6339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:30:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:40 smithi119 ceph-mon[32752]: pgmap v6340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10522.136820 secs 2024-04-26T21:30:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:40 smithi167 ceph-mon[32751]: pgmap v6340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10522.136820 secs 2024-04-26T21:30:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:40 smithi090 ceph-mon[29180]: pgmap v6340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10522.136820 secs 2024-04-26T21:30:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10522.865811 secs 2024-04-26T21:30:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10522.865811 secs 2024-04-26T21:30:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10522.865811 secs 2024-04-26T21:30:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:42 smithi119 ceph-mon[32752]: pgmap v6341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:42 smithi167 ceph-mon[32751]: pgmap v6341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:42 smithi090 ceph-mon[29180]: pgmap v6341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:44 smithi119 ceph-mon[32752]: pgmap v6342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:44 smithi167 ceph-mon[32751]: pgmap v6342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:44 smithi090 ceph-mon[29180]: pgmap v6342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10527.136906 secs 2024-04-26T21:30:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10527.136906 secs 2024-04-26T21:30:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10527.136906 secs 2024-04-26T21:30:46.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:46 smithi090 ceph-mon[29180]: pgmap v6343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:46.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10527.865928 secs 2024-04-26T21:30:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:46 smithi119 ceph-mon[32752]: pgmap v6343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10527.865928 secs 2024-04-26T21:30:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:46 smithi167 ceph-mon[32751]: pgmap v6343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10527.865928 secs 2024-04-26T21:30:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:48 smithi119 ceph-mon[32752]: pgmap v6344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:48 smithi167 ceph-mon[32751]: pgmap v6344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:48 smithi090 ceph-mon[29180]: pgmap v6344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:50 smithi119 ceph-mon[32752]: pgmap v6345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10532.136991 secs 2024-04-26T21:30:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:50 smithi167 ceph-mon[32751]: pgmap v6345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10532.136991 secs 2024-04-26T21:30:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:50 smithi090 ceph-mon[29180]: pgmap v6345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10532.136991 secs 2024-04-26T21:30:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10532.866056 secs 2024-04-26T21:30:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10532.866056 secs 2024-04-26T21:30:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10532.866056 secs 2024-04-26T21:30:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:52 smithi119 ceph-mon[32752]: pgmap v6346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:52 smithi167 ceph-mon[32751]: pgmap v6346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:52 smithi090 ceph-mon[29180]: pgmap v6346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:54 smithi119 ceph-mon[32752]: pgmap v6347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:54 smithi167 ceph-mon[32751]: pgmap v6347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:54 smithi090 ceph-mon[29180]: pgmap v6347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10537.137051 secs 2024-04-26T21:30:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10537.137051 secs 2024-04-26T21:30:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10537.137051 secs 2024-04-26T21:30:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:56 smithi119 ceph-mon[32752]: pgmap v6348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10537.866195 secs 2024-04-26T21:30:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:56 smithi167 ceph-mon[32751]: pgmap v6348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10537.866195 secs 2024-04-26T21:30:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:56 smithi090 ceph-mon[29180]: pgmap v6348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:30:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10537.866195 secs 2024-04-26T21:30:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:30:58 smithi119 ceph-mon[32752]: pgmap v6349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:30:58 smithi167 ceph-mon[32751]: pgmap v6349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:30:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:30:58 smithi090 ceph-mon[29180]: pgmap v6349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:00 smithi119 ceph-mon[32752]: pgmap v6350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10542.137138 secs 2024-04-26T21:31:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:00 smithi167 ceph-mon[32751]: pgmap v6350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10542.137138 secs 2024-04-26T21:31:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:00 smithi090 ceph-mon[29180]: pgmap v6350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10542.137138 secs 2024-04-26T21:31:01.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10542.866284 secs 2024-04-26T21:31:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10542.866284 secs 2024-04-26T21:31:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10542.866284 secs 2024-04-26T21:31:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:02 smithi119 ceph-mon[32752]: pgmap v6351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:02 smithi167 ceph-mon[32751]: pgmap v6351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:02 smithi090 ceph-mon[29180]: pgmap v6351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:04 smithi119 ceph-mon[32752]: pgmap v6352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:04 smithi167 ceph-mon[32751]: pgmap v6352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:04 smithi090 ceph-mon[29180]: pgmap v6352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10547.137222 secs 2024-04-26T21:31:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10547.137222 secs 2024-04-26T21:31:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10547.137222 secs 2024-04-26T21:31:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:06 smithi119 ceph-mon[32752]: pgmap v6353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10547.866418 secs 2024-04-26T21:31:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:06 smithi167 ceph-mon[32751]: pgmap v6353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10547.866418 secs 2024-04-26T21:31:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:06 smithi090 ceph-mon[29180]: pgmap v6353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10547.866418 secs 2024-04-26T21:31:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:08 smithi119 ceph-mon[32752]: pgmap v6354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:08 smithi167 ceph-mon[32751]: pgmap v6354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:08 smithi090 ceph-mon[29180]: pgmap v6354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:10 smithi119 ceph-mon[32752]: pgmap v6355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10552.137305 secs 2024-04-26T21:31:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:10 smithi167 ceph-mon[32751]: pgmap v6355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10552.137305 secs 2024-04-26T21:31:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:10 smithi090 ceph-mon[29180]: pgmap v6355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10552.137305 secs 2024-04-26T21:31:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10552.866540 secs 2024-04-26T21:31:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10552.866540 secs 2024-04-26T21:31:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10552.866540 secs 2024-04-26T21:31:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:12 smithi119 ceph-mon[32752]: pgmap v6356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:12 smithi167 ceph-mon[32751]: pgmap v6356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:12 smithi090 ceph-mon[29180]: pgmap v6356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:14 smithi119 ceph-mon[32752]: pgmap v6357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:14 smithi167 ceph-mon[32751]: pgmap v6357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:14 smithi090 ceph-mon[29180]: pgmap v6357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10557.137391 secs 2024-04-26T21:31:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10557.137391 secs 2024-04-26T21:31:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10557.137391 secs 2024-04-26T21:31:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:16 smithi119 ceph-mon[32752]: pgmap v6358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10557.866663 secs 2024-04-26T21:31:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:16 smithi167 ceph-mon[32751]: pgmap v6358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10557.866663 secs 2024-04-26T21:31:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:16 smithi090 ceph-mon[29180]: pgmap v6358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10557.866663 secs 2024-04-26T21:31:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:18 smithi119 ceph-mon[32752]: pgmap v6359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:18 smithi167 ceph-mon[32751]: pgmap v6359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:18 smithi090 ceph-mon[29180]: pgmap v6359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:20 smithi119 ceph-mon[32752]: pgmap v6360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10562.137453 secs 2024-04-26T21:31:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:20 smithi167 ceph-mon[32751]: pgmap v6360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10562.137453 secs 2024-04-26T21:31:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:20 smithi090 ceph-mon[29180]: pgmap v6360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10562.137453 secs 2024-04-26T21:31:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10562.866790 secs 2024-04-26T21:31:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10562.866790 secs 2024-04-26T21:31:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10562.866790 secs 2024-04-26T21:31:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:22 smithi119 ceph-mon[32752]: pgmap v6361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:22 smithi167 ceph-mon[32751]: pgmap v6361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:22 smithi090 ceph-mon[29180]: pgmap v6361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:24 smithi167 ceph-mon[32751]: pgmap v6362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:24 smithi090 ceph-mon[29180]: pgmap v6362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:24 smithi119 ceph-mon[32752]: pgmap v6362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10567.137565 secs 2024-04-26T21:31:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10567.137565 secs 2024-04-26T21:31:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10567.137565 secs 2024-04-26T21:31:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:26 smithi167 ceph-mon[32751]: pgmap v6363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10567.866908 secs 2024-04-26T21:31:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:26 smithi090 ceph-mon[29180]: pgmap v6363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10567.866908 secs 2024-04-26T21:31:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:26 smithi119 ceph-mon[32752]: pgmap v6363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10567.866908 secs 2024-04-26T21:31:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:31:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:31:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:31:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:28 smithi090 ceph-mon[29180]: pgmap v6364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:31:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:31:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:28 smithi090 ceph-mon[29180]: pgmap v6365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:31:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:28 smithi090 ceph-mon[29180]: pgmap v6366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:31:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:31:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:28 smithi119 ceph-mon[32752]: pgmap v6364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:31:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:31:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:28 smithi119 ceph-mon[32752]: pgmap v6365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:31:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:28 smithi119 ceph-mon[32752]: pgmap v6366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:31:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:31:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:28 smithi167 ceph-mon[32751]: pgmap v6364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:31:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:31:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:28 smithi167 ceph-mon[32751]: pgmap v6365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:31:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:28 smithi167 ceph-mon[32751]: pgmap v6366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:31:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:31:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:29 smithi119 ceph-mon[32752]: pgmap v6367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:29 smithi167 ceph-mon[32751]: pgmap v6367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:29 smithi090 ceph-mon[29180]: pgmap v6367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:31.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10572.137650 secs 2024-04-26T21:31:31.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10572.867026 secs 2024-04-26T21:31:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10572.137650 secs 2024-04-26T21:31:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10572.867026 secs 2024-04-26T21:31:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10572.137650 secs 2024-04-26T21:31:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10572.867026 secs 2024-04-26T21:31:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:31 smithi119 ceph-mon[32752]: pgmap v6368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:31 smithi167 ceph-mon[32751]: pgmap v6368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:31 smithi090 ceph-mon[29180]: pgmap v6368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:33.936 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:31:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:31:33.660+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:33.936 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:31:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:31:33.660+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:33.936 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:31:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:31:33.660+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:33.937 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:31:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:31:33.660+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:33.937 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:31:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:31:33.660+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:31:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:33 smithi119 ceph-mon[32752]: pgmap v6369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:33 smithi167 ceph-mon[32751]: pgmap v6369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:34.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:33 smithi090 ceph-mon[29180]: pgmap v6369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10577.137736 secs 2024-04-26T21:31:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10577.137736 secs 2024-04-26T21:31:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10577.137736 secs 2024-04-26T21:31:36.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10577.867095 secs 2024-04-26T21:31:36.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:36 smithi167 ceph-mon[32751]: pgmap v6370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10577.867095 secs 2024-04-26T21:31:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:36 smithi119 ceph-mon[32752]: pgmap v6370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10577.867095 secs 2024-04-26T21:31:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:36 smithi090 ceph-mon[29180]: pgmap v6370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:31:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:37 smithi119 ceph-mon[32752]: pgmap v6371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:31:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:37 smithi167 ceph-mon[32751]: pgmap v6371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:31:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:37 smithi090 ceph-mon[29180]: pgmap v6371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:31:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:39 smithi119 ceph-mon[32752]: pgmap v6372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:39 smithi167 ceph-mon[32751]: pgmap v6372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:39 smithi090 ceph-mon[29180]: pgmap v6372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10582.137821 secs 2024-04-26T21:31:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10582.867219 secs 2024-04-26T21:31:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10582.137821 secs 2024-04-26T21:31:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10582.867219 secs 2024-04-26T21:31:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10582.137821 secs 2024-04-26T21:31:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10582.867219 secs 2024-04-26T21:31:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:41 smithi119 ceph-mon[32752]: pgmap v6373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:41 smithi167 ceph-mon[32751]: pgmap v6373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:41 smithi090 ceph-mon[29180]: pgmap v6373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:43 smithi119 ceph-mon[32752]: pgmap v6374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:43 smithi167 ceph-mon[32751]: pgmap v6374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:43 smithi090 ceph-mon[29180]: pgmap v6374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10587.137906 secs 2024-04-26T21:31:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10587.137906 secs 2024-04-26T21:31:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10587.137906 secs 2024-04-26T21:31:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10587.867328 secs 2024-04-26T21:31:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:46 smithi119 ceph-mon[32752]: pgmap v6375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10587.867328 secs 2024-04-26T21:31:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:46 smithi167 ceph-mon[32751]: pgmap v6375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10587.867328 secs 2024-04-26T21:31:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:46 smithi090 ceph-mon[29180]: pgmap v6375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:47 smithi119 ceph-mon[32752]: pgmap v6376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:47 smithi167 ceph-mon[32751]: pgmap v6376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:47 smithi090 ceph-mon[29180]: pgmap v6376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:49 smithi119 ceph-mon[32752]: pgmap v6377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:49 smithi167 ceph-mon[32751]: pgmap v6377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:49 smithi090 ceph-mon[29180]: pgmap v6377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:31:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10592.137992 secs 2024-04-26T21:31:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:50 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10592.867465 secs 2024-04-26T21:31:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10592.137992 secs 2024-04-26T21:31:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:50 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10592.867465 secs 2024-04-26T21:31:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10592.137992 secs 2024-04-26T21:31:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:50 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10592.867465 secs 2024-04-26T21:31:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:51 smithi119 ceph-mon[32752]: pgmap v6378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:51 smithi167 ceph-mon[32751]: pgmap v6378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:51 smithi090 ceph-mon[29180]: pgmap v6378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:53 smithi119 ceph-mon[32752]: pgmap v6379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:53 smithi167 ceph-mon[32751]: pgmap v6379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:53 smithi090 ceph-mon[29180]: pgmap v6379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:31:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10597.138052 secs 2024-04-26T21:31:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10597.138052 secs 2024-04-26T21:31:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10597.138052 secs 2024-04-26T21:31:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10597.867529 secs 2024-04-26T21:31:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:56 smithi119 ceph-mon[32752]: pgmap v6380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10597.867529 secs 2024-04-26T21:31:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:56 smithi167 ceph-mon[32751]: pgmap v6380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10597.867529 secs 2024-04-26T21:31:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:56 smithi090 ceph-mon[29180]: pgmap v6380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:57 smithi119 ceph-mon[32752]: pgmap v6381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:57 smithi167 ceph-mon[32751]: pgmap v6381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:31:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:57 smithi090 ceph-mon[29180]: pgmap v6381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:31:59 smithi119 ceph-mon[32752]: pgmap v6382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:31:59 smithi167 ceph-mon[32751]: pgmap v6382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:00.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:31:59 smithi090 ceph-mon[29180]: pgmap v6382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:01.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10602.138166 secs 2024-04-26T21:32:01.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:00 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10602.867648 secs 2024-04-26T21:32:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10602.138166 secs 2024-04-26T21:32:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:00 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10602.867648 secs 2024-04-26T21:32:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10602.138166 secs 2024-04-26T21:32:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:00 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10602.867648 secs 2024-04-26T21:32:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:01 smithi119 ceph-mon[32752]: pgmap v6383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:01 smithi167 ceph-mon[32751]: pgmap v6383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:01 smithi090 ceph-mon[29180]: pgmap v6383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:03 smithi119 ceph-mon[32752]: pgmap v6384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:03 smithi167 ceph-mon[32751]: pgmap v6384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:03 smithi090 ceph-mon[29180]: pgmap v6384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10607.138230 secs 2024-04-26T21:32:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10607.138230 secs 2024-04-26T21:32:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10607.138230 secs 2024-04-26T21:32:06.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10607.867726 secs 2024-04-26T21:32:06.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:06 smithi167 ceph-mon[32751]: pgmap v6385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10607.867726 secs 2024-04-26T21:32:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:06 smithi119 ceph-mon[32752]: pgmap v6385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10607.867726 secs 2024-04-26T21:32:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:06 smithi090 ceph-mon[29180]: pgmap v6385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:07 smithi119 ceph-mon[32752]: pgmap v6386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:07 smithi167 ceph-mon[32751]: pgmap v6386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:07 smithi090 ceph-mon[29180]: pgmap v6386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:09 smithi119 ceph-mon[32752]: pgmap v6387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:09 smithi167 ceph-mon[32751]: pgmap v6387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:09 smithi090 ceph-mon[29180]: pgmap v6387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10612.138288 secs 2024-04-26T21:32:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:10 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10612.867790 secs 2024-04-26T21:32:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10612.138288 secs 2024-04-26T21:32:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:10 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10612.867790 secs 2024-04-26T21:32:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10612.138288 secs 2024-04-26T21:32:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:10 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10612.867790 secs 2024-04-26T21:32:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:11 smithi119 ceph-mon[32752]: pgmap v6388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:11 smithi167 ceph-mon[32751]: pgmap v6388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:11 smithi090 ceph-mon[29180]: pgmap v6388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:32:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:13 smithi119 ceph-mon[32752]: pgmap v6389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:13 smithi167 ceph-mon[32751]: pgmap v6389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:13 smithi090 ceph-mon[29180]: pgmap v6389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:32:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10617.138355 secs 2024-04-26T21:32:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10617.138355 secs 2024-04-26T21:32:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10617.138355 secs 2024-04-26T21:32:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10617.867925 secs 2024-04-26T21:32:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:16 smithi119 ceph-mon[32752]: pgmap v6390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:32:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10617.867925 secs 2024-04-26T21:32:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:16 smithi167 ceph-mon[32751]: pgmap v6390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:32:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10617.867925 secs 2024-04-26T21:32:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:16 smithi090 ceph-mon[29180]: pgmap v6390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:32:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:17 smithi119 ceph-mon[32752]: pgmap v6391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:32:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:17 smithi167 ceph-mon[32751]: pgmap v6391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:32:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:17 smithi090 ceph-mon[29180]: pgmap v6391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:32:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:19 smithi090 ceph-mon[29180]: pgmap v6392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:19 smithi119 ceph-mon[32752]: pgmap v6392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:19 smithi167 ceph-mon[32751]: pgmap v6392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10622.138416 secs 2024-04-26T21:32:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10622.868068 secs 2024-04-26T21:32:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10622.138416 secs 2024-04-26T21:32:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10622.868068 secs 2024-04-26T21:32:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10622.138416 secs 2024-04-26T21:32:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10622.868068 secs 2024-04-26T21:32:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:21 smithi119 ceph-mon[32752]: pgmap v6393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:21 smithi167 ceph-mon[32751]: pgmap v6393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:21 smithi090 ceph-mon[29180]: pgmap v6393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:23 smithi119 ceph-mon[32752]: pgmap v6394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:23 smithi167 ceph-mon[32751]: pgmap v6394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:24.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:23 smithi090 ceph-mon[29180]: pgmap v6394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10627.138501 secs 2024-04-26T21:32:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10627.138501 secs 2024-04-26T21:32:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10627.138501 secs 2024-04-26T21:32:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10627.868204 secs 2024-04-26T21:32:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:26 smithi119 ceph-mon[32752]: pgmap v6395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10627.868204 secs 2024-04-26T21:32:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:26 smithi167 ceph-mon[32751]: pgmap v6395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10627.868204 secs 2024-04-26T21:32:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:26 smithi090 ceph-mon[29180]: pgmap v6395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:27 smithi119 ceph-mon[32752]: pgmap v6396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:27 smithi167 ceph-mon[32751]: pgmap v6396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:27 smithi090 ceph-mon[29180]: pgmap v6396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:32:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:32:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:32:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:28 smithi119 ceph-mon[32752]: pgmap v6397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:32:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:28 smithi119 ceph-mon[32752]: pgmap v6398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:32:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:32:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:32:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:32:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:28 smithi167 ceph-mon[32751]: pgmap v6397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:32:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:28 smithi167 ceph-mon[32751]: pgmap v6398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:32:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:32:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:32:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:32:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:28 smithi090 ceph-mon[29180]: pgmap v6397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:32:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:28 smithi090 ceph-mon[29180]: pgmap v6398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:32:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10632.138587 secs 2024-04-26T21:32:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10632.138587 secs 2024-04-26T21:32:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10632.138587 secs 2024-04-26T21:32:31.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10632.868322 secs 2024-04-26T21:32:31.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:31 smithi090 ceph-mon[29180]: pgmap v6399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10632.868322 secs 2024-04-26T21:32:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:31 smithi119 ceph-mon[32752]: pgmap v6399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10632.868322 secs 2024-04-26T21:32:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:31 smithi167 ceph-mon[32751]: pgmap v6399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:33 smithi119 ceph-mon[32752]: pgmap v6400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:32:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:33 smithi167 ceph-mon[32751]: pgmap v6400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:32:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:33 smithi090 ceph-mon[29180]: pgmap v6400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:32:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:35 smithi119 ceph-mon[32752]: pgmap v6401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10637.138675 secs 2024-04-26T21:32:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:35 smithi167 ceph-mon[32751]: pgmap v6401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10637.138675 secs 2024-04-26T21:32:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:35 smithi090 ceph-mon[29180]: pgmap v6401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10637.138675 secs 2024-04-26T21:32:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10637.868442 secs 2024-04-26T21:32:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10637.868442 secs 2024-04-26T21:32:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10637.868442 secs 2024-04-26T21:32:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:37 smithi119 ceph-mon[32752]: pgmap v6402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:37 smithi167 ceph-mon[32751]: pgmap v6402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:37 smithi090 ceph-mon[29180]: pgmap v6402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:32:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:39 smithi119 ceph-mon[32752]: pgmap v6403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:32:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:39 smithi167 ceph-mon[32751]: pgmap v6403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:32:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:39 smithi090 ceph-mon[29180]: pgmap v6403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:32:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10642.138761 secs 2024-04-26T21:32:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10642.868561 secs 2024-04-26T21:32:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:41 smithi090 ceph-mon[29180]: pgmap v6404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10642.138761 secs 2024-04-26T21:32:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10642.868561 secs 2024-04-26T21:32:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:41 smithi119 ceph-mon[32752]: pgmap v6404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10642.138761 secs 2024-04-26T21:32:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10642.868561 secs 2024-04-26T21:32:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:41 smithi167 ceph-mon[32751]: pgmap v6404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:43 smithi119 ceph-mon[32752]: pgmap v6405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:43 smithi167 ceph-mon[32751]: pgmap v6405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:44.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:43 smithi090 ceph-mon[29180]: pgmap v6405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:45 smithi119 ceph-mon[32752]: pgmap v6406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10647.138825 secs 2024-04-26T21:32:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:45 smithi167 ceph-mon[32751]: pgmap v6406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10647.138825 secs 2024-04-26T21:32:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:45 smithi090 ceph-mon[29180]: pgmap v6406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10647.138825 secs 2024-04-26T21:32:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10647.868674 secs 2024-04-26T21:32:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10647.868674 secs 2024-04-26T21:32:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10647.868674 secs 2024-04-26T21:32:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:47 smithi119 ceph-mon[32752]: pgmap v6407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:47 smithi167 ceph-mon[32751]: pgmap v6407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:47 smithi090 ceph-mon[29180]: pgmap v6407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:49 smithi119 ceph-mon[32752]: pgmap v6408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:49 smithi167 ceph-mon[32751]: pgmap v6408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:49 smithi090 ceph-mon[29180]: pgmap v6408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10652.138908 secs 2024-04-26T21:32:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10652.138908 secs 2024-04-26T21:32:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10652.138908 secs 2024-04-26T21:32:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10652.868747 secs 2024-04-26T21:32:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:51 smithi090 ceph-mon[29180]: pgmap v6409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10652.868747 secs 2024-04-26T21:32:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:51 smithi119 ceph-mon[32752]: pgmap v6409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10652.868747 secs 2024-04-26T21:32:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:51 smithi167 ceph-mon[32751]: pgmap v6409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:53 smithi119 ceph-mon[32752]: pgmap v6410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:53 smithi167 ceph-mon[32751]: pgmap v6410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:53 smithi090 ceph-mon[29180]: pgmap v6410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:55 smithi119 ceph-mon[32752]: pgmap v6411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10657.139000 secs 2024-04-26T21:32:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:55 smithi167 ceph-mon[32751]: pgmap v6411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10657.139000 secs 2024-04-26T21:32:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:55 smithi090 ceph-mon[29180]: pgmap v6411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:32:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10657.139000 secs 2024-04-26T21:32:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10657.868818 secs 2024-04-26T21:32:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10657.868818 secs 2024-04-26T21:32:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10657.868818 secs 2024-04-26T21:32:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:57 smithi119 ceph-mon[32752]: pgmap v6412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:57 smithi167 ceph-mon[32751]: pgmap v6412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:32:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:57 smithi090 ceph-mon[29180]: pgmap v6412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:32:59 smithi119 ceph-mon[32752]: pgmap v6413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:32:59 smithi167 ceph-mon[32751]: pgmap v6413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:32:59 smithi090 ceph-mon[29180]: pgmap v6413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:01.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10662.139084 secs 2024-04-26T21:33:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10662.139084 secs 2024-04-26T21:33:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10662.139084 secs 2024-04-26T21:33:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10662.868939 secs 2024-04-26T21:33:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:01 smithi119 ceph-mon[32752]: pgmap v6414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10662.868939 secs 2024-04-26T21:33:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:01 smithi167 ceph-mon[32751]: pgmap v6414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10662.868939 secs 2024-04-26T21:33:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:01 smithi090 ceph-mon[29180]: pgmap v6414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:03 smithi119 ceph-mon[32752]: pgmap v6415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:03 smithi167 ceph-mon[32751]: pgmap v6415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:04.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:03 smithi090 ceph-mon[29180]: pgmap v6415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:05 smithi119 ceph-mon[32752]: pgmap v6416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:05 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10667.139165 secs 2024-04-26T21:33:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:05 smithi119 ceph-mon[32752]: slow request 721.407582 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:33:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:05 smithi167 ceph-mon[32751]: pgmap v6416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:05 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10667.139165 secs 2024-04-26T21:33:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:05 smithi167 ceph-mon[32751]: slow request 721.407582 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:33:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:05 smithi090 ceph-mon[29180]: pgmap v6416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:05 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10667.139165 secs 2024-04-26T21:33:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:05 smithi090 ceph-mon[29180]: slow request 721.407582 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:33:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10667.869045 secs 2024-04-26T21:33:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10667.869045 secs 2024-04-26T21:33:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10667.869045 secs 2024-04-26T21:33:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:07 smithi119 ceph-mon[32752]: pgmap v6417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:07 smithi167 ceph-mon[32751]: pgmap v6417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:07 smithi090 ceph-mon[29180]: pgmap v6417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:09 smithi119 ceph-mon[32752]: pgmap v6418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:09 smithi167 ceph-mon[32751]: pgmap v6418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:09 smithi090 ceph-mon[29180]: pgmap v6418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10672.139247 secs 2024-04-26T21:33:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10672.139247 secs 2024-04-26T21:33:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10672.139247 secs 2024-04-26T21:33:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10672.869177 secs 2024-04-26T21:33:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:11 smithi119 ceph-mon[32752]: pgmap v6419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10672.869177 secs 2024-04-26T21:33:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:11 smithi167 ceph-mon[32751]: pgmap v6419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10672.869177 secs 2024-04-26T21:33:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:11 smithi090 ceph-mon[29180]: pgmap v6419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:13 smithi119 ceph-mon[32752]: pgmap v6420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:13 smithi167 ceph-mon[32751]: pgmap v6420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:13 smithi090 ceph-mon[29180]: pgmap v6420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:15 smithi119 ceph-mon[32752]: pgmap v6421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:15 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10677.139331 secs 2024-04-26T21:33:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:15 smithi119 ceph-mon[32752]: slow request 721.371066 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:33:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:15 smithi167 ceph-mon[32751]: pgmap v6421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:15 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10677.139331 secs 2024-04-26T21:33:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:15 smithi167 ceph-mon[32751]: slow request 721.371066 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:33:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:15 smithi090 ceph-mon[29180]: pgmap v6421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:15 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10677.139331 secs 2024-04-26T21:33:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:15 smithi090 ceph-mon[29180]: slow request 721.371066 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:33:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10677.869311 secs 2024-04-26T21:33:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10677.869311 secs 2024-04-26T21:33:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10677.869311 secs 2024-04-26T21:33:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:17 smithi119 ceph-mon[32752]: pgmap v6422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:17 smithi167 ceph-mon[32751]: pgmap v6422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:17 smithi090 ceph-mon[29180]: pgmap v6422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:19 smithi119 ceph-mon[32752]: pgmap v6423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:19 smithi167 ceph-mon[32751]: pgmap v6423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:19 smithi090 ceph-mon[29180]: pgmap v6423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10682.139420 secs 2024-04-26T21:33:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10682.139420 secs 2024-04-26T21:33:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10682.139420 secs 2024-04-26T21:33:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10682.869415 secs 2024-04-26T21:33:22.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:21 smithi090 ceph-mon[29180]: pgmap v6424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10682.869415 secs 2024-04-26T21:33:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:21 smithi119 ceph-mon[32752]: pgmap v6424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10682.869415 secs 2024-04-26T21:33:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:21 smithi167 ceph-mon[32751]: pgmap v6424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:23 smithi119 ceph-mon[32752]: pgmap v6425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:23 smithi167 ceph-mon[32751]: pgmap v6425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:23 smithi090 ceph-mon[29180]: pgmap v6425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:25 smithi119 ceph-mon[32752]: pgmap v6426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10687.139504 secs 2024-04-26T21:33:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:25 smithi167 ceph-mon[32751]: pgmap v6426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10687.139504 secs 2024-04-26T21:33:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:25 smithi090 ceph-mon[29180]: pgmap v6426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10687.139504 secs 2024-04-26T21:33:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10687.869481 secs 2024-04-26T21:33:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10687.869481 secs 2024-04-26T21:33:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10687.869481 secs 2024-04-26T21:33:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:27 smithi119 ceph-mon[32752]: pgmap v6427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:27 smithi167 ceph-mon[32751]: pgmap v6427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:27 smithi090 ceph-mon[29180]: pgmap v6427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:28.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:33:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:33:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:33:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:29 smithi167 ceph-mon[32751]: pgmap v6428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:33:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:33:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:33:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:29 smithi090 ceph-mon[29180]: pgmap v6428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:33:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:33:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:33:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:29 smithi119 ceph-mon[32752]: pgmap v6428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:30.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:33:30.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:33:30.323 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:33:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:30 smithi167 ceph-mon[32751]: pgmap v6429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10692.139589 secs 2024-04-26T21:33:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:30 smithi090 ceph-mon[29180]: pgmap v6429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10692.139589 secs 2024-04-26T21:33:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:30 smithi119 ceph-mon[32752]: pgmap v6429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10692.139589 secs 2024-04-26T21:33:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10692.869588 secs 2024-04-26T21:33:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10692.869588 secs 2024-04-26T21:33:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10692.869588 secs 2024-04-26T21:33:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:32 smithi090 ceph-mon[29180]: pgmap v6430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:32 smithi119 ceph-mon[32752]: pgmap v6430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:32 smithi167 ceph-mon[32751]: pgmap v6430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:34 smithi090 ceph-mon[29180]: pgmap v6431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:33:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:34 smithi119 ceph-mon[32752]: pgmap v6431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:33:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:34 smithi167 ceph-mon[32751]: pgmap v6431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:33:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10697.139670 secs 2024-04-26T21:33:36.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10697.139670 secs 2024-04-26T21:33:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10697.139670 secs 2024-04-26T21:33:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:36 smithi090 ceph-mon[29180]: pgmap v6432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10697.869650 secs 2024-04-26T21:33:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:36 smithi119 ceph-mon[32752]: pgmap v6432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10697.869650 secs 2024-04-26T21:33:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:36 smithi167 ceph-mon[32751]: pgmap v6432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10697.869650 secs 2024-04-26T21:33:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:38 smithi090 ceph-mon[29180]: pgmap v6433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:38 smithi119 ceph-mon[32752]: pgmap v6433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:38 smithi167 ceph-mon[32751]: pgmap v6433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:40 smithi090 ceph-mon[29180]: pgmap v6434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10702.139747 secs 2024-04-26T21:33:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:40 smithi119 ceph-mon[32752]: pgmap v6434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10702.139747 secs 2024-04-26T21:33:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:40 smithi167 ceph-mon[32751]: pgmap v6434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:33:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10702.139747 secs 2024-04-26T21:33:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10702.869739 secs 2024-04-26T21:33:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10702.869739 secs 2024-04-26T21:33:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10702.869739 secs 2024-04-26T21:33:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:42 smithi090 ceph-mon[29180]: pgmap v6435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:42 smithi119 ceph-mon[32752]: pgmap v6435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:42 smithi167 ceph-mon[32751]: pgmap v6435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:44 smithi090 ceph-mon[29180]: pgmap v6436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:44 smithi119 ceph-mon[32752]: pgmap v6436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:44 smithi167 ceph-mon[32751]: pgmap v6436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10707.139850 secs 2024-04-26T21:33:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10707.139850 secs 2024-04-26T21:33:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10707.139850 secs 2024-04-26T21:33:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:46 smithi090 ceph-mon[29180]: pgmap v6437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10707.869852 secs 2024-04-26T21:33:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:46 smithi119 ceph-mon[32752]: pgmap v6437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10707.869852 secs 2024-04-26T21:33:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:46 smithi167 ceph-mon[32751]: pgmap v6437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10707.869852 secs 2024-04-26T21:33:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:48 smithi090 ceph-mon[29180]: pgmap v6438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:48 smithi119 ceph-mon[32752]: pgmap v6438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:48 smithi167 ceph-mon[32751]: pgmap v6438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:50 smithi090 ceph-mon[29180]: pgmap v6439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10712.139883 secs 2024-04-26T21:33:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:50 smithi119 ceph-mon[32752]: pgmap v6439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10712.139883 secs 2024-04-26T21:33:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:50 smithi167 ceph-mon[32751]: pgmap v6439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10712.139883 secs 2024-04-26T21:33:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10712.869968 secs 2024-04-26T21:33:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10712.869968 secs 2024-04-26T21:33:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10712.869968 secs 2024-04-26T21:33:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:52 smithi090 ceph-mon[29180]: pgmap v6440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:52 smithi119 ceph-mon[32752]: pgmap v6440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:52 smithi167 ceph-mon[32751]: pgmap v6440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:54 smithi090 ceph-mon[29180]: pgmap v6441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:54 smithi119 ceph-mon[32752]: pgmap v6441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:54 smithi167 ceph-mon[32751]: pgmap v6441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:33:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10717.139945 secs 2024-04-26T21:33:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10717.139945 secs 2024-04-26T21:33:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10717.139945 secs 2024-04-26T21:33:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:56 smithi090 ceph-mon[29180]: pgmap v6442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10717.870090 secs 2024-04-26T21:33:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:56 smithi119 ceph-mon[32752]: pgmap v6442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10717.870090 secs 2024-04-26T21:33:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:56 smithi167 ceph-mon[32751]: pgmap v6442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10717.870090 secs 2024-04-26T21:33:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:33:58 smithi119 ceph-mon[32752]: pgmap v6443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:33:58 smithi167 ceph-mon[32751]: pgmap v6443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:33:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:33:58 smithi090 ceph-mon[29180]: pgmap v6443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:01.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:00 smithi090 ceph-mon[29180]: pgmap v6444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:01.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10722.140008 secs 2024-04-26T21:34:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:00 smithi119 ceph-mon[32752]: pgmap v6444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10722.140008 secs 2024-04-26T21:34:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:00 smithi167 ceph-mon[32751]: pgmap v6444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10722.140008 secs 2024-04-26T21:34:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10722.870212 secs 2024-04-26T21:34:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10722.870212 secs 2024-04-26T21:34:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10722.870212 secs 2024-04-26T21:34:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:02 smithi119 ceph-mon[32752]: pgmap v6445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:02 smithi167 ceph-mon[32751]: pgmap v6445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:02 smithi090 ceph-mon[29180]: pgmap v6445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:04 smithi119 ceph-mon[32752]: pgmap v6446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:04 smithi167 ceph-mon[32751]: pgmap v6446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:04 smithi090 ceph-mon[29180]: pgmap v6446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:06.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10727.140085 secs 2024-04-26T21:34:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10727.140085 secs 2024-04-26T21:34:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10727.140085 secs 2024-04-26T21:34:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:06 smithi119 ceph-mon[32752]: pgmap v6447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10727.870302 secs 2024-04-26T21:34:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:06 smithi167 ceph-mon[32751]: pgmap v6447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10727.870302 secs 2024-04-26T21:34:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:06 smithi090 ceph-mon[29180]: pgmap v6447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10727.870302 secs 2024-04-26T21:34:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:08 smithi090 ceph-mon[29180]: pgmap v6448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:08 smithi119 ceph-mon[32752]: pgmap v6448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:08 smithi167 ceph-mon[32751]: pgmap v6448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:10 smithi119 ceph-mon[32752]: pgmap v6449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10732.140171 secs 2024-04-26T21:34:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:10 smithi167 ceph-mon[32751]: pgmap v6449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10732.140171 secs 2024-04-26T21:34:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:10 smithi090 ceph-mon[29180]: pgmap v6449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10732.140171 secs 2024-04-26T21:34:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10732.870430 secs 2024-04-26T21:34:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10732.870430 secs 2024-04-26T21:34:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10732.870430 secs 2024-04-26T21:34:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:12 smithi119 ceph-mon[32752]: pgmap v6450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:12 smithi167 ceph-mon[32751]: pgmap v6450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:12 smithi090 ceph-mon[29180]: pgmap v6450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:13 smithi119 ceph-mon[32752]: pgmap v6451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:13 smithi167 ceph-mon[32751]: pgmap v6451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:13 smithi090 ceph-mon[29180]: pgmap v6451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:15.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10737.140233 secs 2024-04-26T21:34:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10737.140233 secs 2024-04-26T21:34:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10737.140233 secs 2024-04-26T21:34:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:16 smithi119 ceph-mon[32752]: pgmap v6452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10737.870513 secs 2024-04-26T21:34:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:16 smithi167 ceph-mon[32751]: pgmap v6452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10737.870513 secs 2024-04-26T21:34:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:16 smithi090 ceph-mon[29180]: pgmap v6452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10737.870513 secs 2024-04-26T21:34:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:18 smithi119 ceph-mon[32752]: pgmap v6453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:18 smithi167 ceph-mon[32751]: pgmap v6453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:18 smithi090 ceph-mon[29180]: pgmap v6453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:20 smithi119 ceph-mon[32752]: pgmap v6454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10742.140316 secs 2024-04-26T21:34:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:20 smithi167 ceph-mon[32751]: pgmap v6454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10742.140316 secs 2024-04-26T21:34:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:20 smithi090 ceph-mon[29180]: pgmap v6454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10742.140316 secs 2024-04-26T21:34:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10742.870652 secs 2024-04-26T21:34:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10742.870652 secs 2024-04-26T21:34:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10742.870652 secs 2024-04-26T21:34:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:22 smithi119 ceph-mon[32752]: pgmap v6455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:22 smithi167 ceph-mon[32751]: pgmap v6455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:22 smithi090 ceph-mon[29180]: pgmap v6455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:24 smithi119 ceph-mon[32752]: pgmap v6456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:24 smithi167 ceph-mon[32751]: pgmap v6456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:24 smithi090 ceph-mon[29180]: pgmap v6456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10747.140377 secs 2024-04-26T21:34:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10747.140377 secs 2024-04-26T21:34:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10747.140377 secs 2024-04-26T21:34:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:26 smithi119 ceph-mon[32752]: pgmap v6457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10747.870733 secs 2024-04-26T21:34:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:26 smithi167 ceph-mon[32751]: pgmap v6457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10747.870733 secs 2024-04-26T21:34:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:26 smithi090 ceph-mon[29180]: pgmap v6457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10747.870733 secs 2024-04-26T21:34:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:28 smithi119 ceph-mon[32752]: pgmap v6458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:28 smithi167 ceph-mon[32751]: pgmap v6458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:28 smithi090 ceph-mon[29180]: pgmap v6458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:30 smithi119 ceph-mon[32752]: pgmap v6459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:34:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10752.140444 secs 2024-04-26T21:34:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:34:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:34:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:34:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:30 smithi167 ceph-mon[32751]: pgmap v6459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:34:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10752.140444 secs 2024-04-26T21:34:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:34:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:34:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:34:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:30 smithi090 ceph-mon[29180]: pgmap v6459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:34:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10752.140444 secs 2024-04-26T21:34:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:34:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:34:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:34:31.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:31 smithi090 ceph-mon[29180]: pgmap v6460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:34:31.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:31 smithi090 ceph-mon[29180]: pgmap v6461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:31.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10752.870828 secs 2024-04-26T21:34:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:31 smithi119 ceph-mon[32752]: pgmap v6460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:34:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:31 smithi119 ceph-mon[32752]: pgmap v6461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10752.870828 secs 2024-04-26T21:34:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:31 smithi167 ceph-mon[32751]: pgmap v6460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:34:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:31 smithi167 ceph-mon[32751]: pgmap v6461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10752.870828 secs 2024-04-26T21:34:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:33 smithi119 ceph-mon[32752]: pgmap v6462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:34:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:33 smithi167 ceph-mon[32751]: pgmap v6462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:34:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:33 smithi090 ceph-mon[29180]: pgmap v6462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:34:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:35 smithi119 ceph-mon[32752]: pgmap v6463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10757.140528 secs 2024-04-26T21:34:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:35 smithi167 ceph-mon[32751]: pgmap v6463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10757.140528 secs 2024-04-26T21:34:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:35 smithi090 ceph-mon[29180]: pgmap v6463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10757.140528 secs 2024-04-26T21:34:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10757.870923 secs 2024-04-26T21:34:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10757.870923 secs 2024-04-26T21:34:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10757.870923 secs 2024-04-26T21:34:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:37 smithi119 ceph-mon[32752]: pgmap v6464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:37 smithi167 ceph-mon[32751]: pgmap v6464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:37 smithi090 ceph-mon[29180]: pgmap v6464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:34:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:39 smithi119 ceph-mon[32752]: pgmap v6465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:34:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:39 smithi167 ceph-mon[32751]: pgmap v6465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:34:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:39 smithi090 ceph-mon[29180]: pgmap v6465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:34:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10762.140614 secs 2024-04-26T21:34:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10762.140614 secs 2024-04-26T21:34:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10762.140614 secs 2024-04-26T21:34:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:41 smithi119 ceph-mon[32752]: pgmap v6466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:34:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10762.871002 secs 2024-04-26T21:34:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:41 smithi167 ceph-mon[32751]: pgmap v6466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:34:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10762.871002 secs 2024-04-26T21:34:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:41 smithi090 ceph-mon[29180]: pgmap v6466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:34:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10762.871002 secs 2024-04-26T21:34:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:43 smithi119 ceph-mon[32752]: pgmap v6467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:43 smithi167 ceph-mon[32751]: pgmap v6467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:43 smithi090 ceph-mon[29180]: pgmap v6467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:45 smithi167 ceph-mon[32751]: pgmap v6468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10767.140676 secs 2024-04-26T21:34:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:45 smithi090 ceph-mon[29180]: pgmap v6468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10767.140676 secs 2024-04-26T21:34:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:45 smithi119 ceph-mon[32752]: pgmap v6468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10767.140676 secs 2024-04-26T21:34:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10767.871139 secs 2024-04-26T21:34:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10767.871139 secs 2024-04-26T21:34:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10767.871139 secs 2024-04-26T21:34:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:47 smithi167 ceph-mon[32751]: pgmap v6469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:47 smithi090 ceph-mon[29180]: pgmap v6469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:47 smithi119 ceph-mon[32752]: pgmap v6469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:49 smithi167 ceph-mon[32751]: pgmap v6470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:49 smithi090 ceph-mon[29180]: pgmap v6470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:49 smithi119 ceph-mon[32752]: pgmap v6470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10772.140760 secs 2024-04-26T21:34:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10772.140760 secs 2024-04-26T21:34:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10772.140760 secs 2024-04-26T21:34:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:51 smithi090 ceph-mon[29180]: pgmap v6471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10772.871259 secs 2024-04-26T21:34:51.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:51 smithi119 ceph-mon[32752]: pgmap v6471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:51.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10772.871259 secs 2024-04-26T21:34:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:51 smithi167 ceph-mon[32751]: pgmap v6471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10772.871259 secs 2024-04-26T21:34:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:53 smithi090 ceph-mon[29180]: pgmap v6472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:53 smithi119 ceph-mon[32752]: pgmap v6472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:53 smithi167 ceph-mon[32751]: pgmap v6472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:55 smithi090 ceph-mon[29180]: pgmap v6473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10777.140824 secs 2024-04-26T21:34:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:55 smithi119 ceph-mon[32752]: pgmap v6473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10777.140824 secs 2024-04-26T21:34:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:55 smithi167 ceph-mon[32751]: pgmap v6473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:34:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10777.140824 secs 2024-04-26T21:34:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10777.871372 secs 2024-04-26T21:34:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10777.871372 secs 2024-04-26T21:34:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10777.871372 secs 2024-04-26T21:34:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:57 smithi090 ceph-mon[29180]: pgmap v6474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:57 smithi119 ceph-mon[32752]: pgmap v6474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:57 smithi167 ceph-mon[32751]: pgmap v6474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:34:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:34:59 smithi090 ceph-mon[29180]: pgmap v6475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:34:59 smithi119 ceph-mon[32752]: pgmap v6475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:34:59 smithi167 ceph-mon[32751]: pgmap v6475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:00 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 10782.140907 secs 2024-04-26T21:35:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:00 smithi090 ceph-mon[29180]: slow request 1440.138904 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:35:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:00 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 10782.140907 secs 2024-04-26T21:35:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:00 smithi119 ceph-mon[32752]: slow request 1440.138904 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:35:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:00 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 10782.140907 secs 2024-04-26T21:35:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:00 smithi167 ceph-mon[32751]: slow request 1440.138904 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:35:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:01 smithi090 ceph-mon[29180]: pgmap v6476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10782.871528 secs 2024-04-26T21:35:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:01 smithi119 ceph-mon[32752]: pgmap v6476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10782.871528 secs 2024-04-26T21:35:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:01 smithi167 ceph-mon[32751]: pgmap v6476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10782.871528 secs 2024-04-26T21:35:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:03 smithi090 ceph-mon[29180]: pgmap v6477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:03 smithi119 ceph-mon[32752]: pgmap v6477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:03 smithi167 ceph-mon[32751]: pgmap v6477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:05 smithi090 ceph-mon[29180]: pgmap v6478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10787.140965 secs 2024-04-26T21:35:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:05 smithi119 ceph-mon[32752]: pgmap v6478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10787.140965 secs 2024-04-26T21:35:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:05 smithi167 ceph-mon[32751]: pgmap v6478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10787.140965 secs 2024-04-26T21:35:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10787.871597 secs 2024-04-26T21:35:06.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10787.871597 secs 2024-04-26T21:35:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10787.871597 secs 2024-04-26T21:35:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:07 smithi090 ceph-mon[29180]: pgmap v6479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:07 smithi119 ceph-mon[32752]: pgmap v6479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:07 smithi167 ceph-mon[32751]: pgmap v6479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:09 smithi090 ceph-mon[29180]: pgmap v6480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:09 smithi119 ceph-mon[32752]: pgmap v6480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:09 smithi167 ceph-mon[32751]: pgmap v6480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10792.141051 secs 2024-04-26T21:35:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10792.141051 secs 2024-04-26T21:35:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10792.141051 secs 2024-04-26T21:35:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:11 smithi090 ceph-mon[29180]: pgmap v6481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10792.871672 secs 2024-04-26T21:35:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:11 smithi119 ceph-mon[32752]: pgmap v6481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10792.871672 secs 2024-04-26T21:35:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:11 smithi167 ceph-mon[32751]: pgmap v6481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10792.871672 secs 2024-04-26T21:35:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:13 smithi119 ceph-mon[32752]: pgmap v6482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:13 smithi167 ceph-mon[32751]: pgmap v6482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:13 smithi090 ceph-mon[29180]: pgmap v6482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:15 smithi119 ceph-mon[32752]: pgmap v6483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10797.141110 secs 2024-04-26T21:35:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:15 smithi167 ceph-mon[32751]: pgmap v6483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10797.141110 secs 2024-04-26T21:35:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:15 smithi090 ceph-mon[29180]: pgmap v6483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10797.141110 secs 2024-04-26T21:35:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10797.871791 secs 2024-04-26T21:35:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10797.871791 secs 2024-04-26T21:35:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10797.871791 secs 2024-04-26T21:35:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:17 smithi119 ceph-mon[32752]: pgmap v6484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:17 smithi167 ceph-mon[32751]: pgmap v6484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:17 smithi090 ceph-mon[29180]: pgmap v6484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:19 smithi119 ceph-mon[32752]: pgmap v6485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:19 smithi167 ceph-mon[32751]: pgmap v6485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:19 smithi090 ceph-mon[29180]: pgmap v6485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10802.141193 secs 2024-04-26T21:35:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10802.141193 secs 2024-04-26T21:35:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10802.141193 secs 2024-04-26T21:35:21.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:21 smithi119 ceph-mon[32752]: pgmap v6486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:21.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10802.871852 secs 2024-04-26T21:35:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:21 smithi167 ceph-mon[32751]: pgmap v6486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10802.871852 secs 2024-04-26T21:35:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:21 smithi090 ceph-mon[29180]: pgmap v6486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10802.871852 secs 2024-04-26T21:35:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:23 smithi119 ceph-mon[32752]: pgmap v6487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:23 smithi167 ceph-mon[32751]: pgmap v6487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:23 smithi090 ceph-mon[29180]: pgmap v6487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:25 smithi119 ceph-mon[32752]: pgmap v6488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10807.141273 secs 2024-04-26T21:35:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:25 smithi167 ceph-mon[32751]: pgmap v6488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10807.141273 secs 2024-04-26T21:35:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:25 smithi090 ceph-mon[29180]: pgmap v6488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10807.141273 secs 2024-04-26T21:35:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10807.871971 secs 2024-04-26T21:35:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10807.871971 secs 2024-04-26T21:35:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10807.871971 secs 2024-04-26T21:35:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:27 smithi119 ceph-mon[32752]: pgmap v6489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:27 smithi167 ceph-mon[32751]: pgmap v6489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:27 smithi090 ceph-mon[29180]: pgmap v6489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:29 smithi119 ceph-mon[32752]: pgmap v6490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:29 smithi167 ceph-mon[32751]: pgmap v6490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:29 smithi090 ceph-mon[29180]: pgmap v6490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:31.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10812.141333 secs 2024-04-26T21:35:31.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:35:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10812.141333 secs 2024-04-26T21:35:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:35:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10812.141333 secs 2024-04-26T21:35:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:35:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:31 smithi119 ceph-mon[32752]: pgmap v6491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10812.872058 secs 2024-04-26T21:35:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:35:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:35:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:35:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:31 smithi167 ceph-mon[32751]: pgmap v6491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10812.872058 secs 2024-04-26T21:35:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:35:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:35:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:35:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:31 smithi090 ceph-mon[29180]: pgmap v6491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10812.872058 secs 2024-04-26T21:35:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:35:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:35:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:35:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:32 smithi119 ceph-mon[32752]: pgmap v6492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:35:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:32 smithi119 ceph-mon[32752]: pgmap v6493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:32 smithi167 ceph-mon[32751]: pgmap v6492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:35:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:32 smithi167 ceph-mon[32751]: pgmap v6493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:32 smithi090 ceph-mon[29180]: pgmap v6492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:35:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:32 smithi090 ceph-mon[29180]: pgmap v6493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:34 smithi119 ceph-mon[32752]: pgmap v6494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:35:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:34 smithi167 ceph-mon[32751]: pgmap v6494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:35:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:34 smithi090 ceph-mon[29180]: pgmap v6494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:35:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10817.141409 secs 2024-04-26T21:35:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10817.141409 secs 2024-04-26T21:35:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10817.141409 secs 2024-04-26T21:35:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:36 smithi167 ceph-mon[32751]: pgmap v6495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10817.872174 secs 2024-04-26T21:35:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:36 smithi119 ceph-mon[32752]: pgmap v6495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10817.872174 secs 2024-04-26T21:35:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:36 smithi090 ceph-mon[29180]: pgmap v6495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10817.872174 secs 2024-04-26T21:35:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:38 smithi119 ceph-mon[32752]: pgmap v6496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:38 smithi167 ceph-mon[32751]: pgmap v6496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:38 smithi090 ceph-mon[29180]: pgmap v6496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:40 smithi119 ceph-mon[32752]: pgmap v6497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10822.141495 secs 2024-04-26T21:35:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:40 smithi167 ceph-mon[32751]: pgmap v6497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10822.141495 secs 2024-04-26T21:35:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:40 smithi090 ceph-mon[29180]: pgmap v6497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:35:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10822.141495 secs 2024-04-26T21:35:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10822.872253 secs 2024-04-26T21:35:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10822.872253 secs 2024-04-26T21:35:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10822.872253 secs 2024-04-26T21:35:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:42 smithi090 ceph-mon[29180]: pgmap v6498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:35:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:42 smithi119 ceph-mon[32752]: pgmap v6498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:35:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:42 smithi167 ceph-mon[32751]: pgmap v6498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:35:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:44 smithi119 ceph-mon[32752]: pgmap v6499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:44 smithi167 ceph-mon[32751]: pgmap v6499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:44 smithi090 ceph-mon[29180]: pgmap v6499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10827.141575 secs 2024-04-26T21:35:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10827.141575 secs 2024-04-26T21:35:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10827.141575 secs 2024-04-26T21:35:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:46 smithi119 ceph-mon[32752]: pgmap v6500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10827.872371 secs 2024-04-26T21:35:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:46 smithi167 ceph-mon[32751]: pgmap v6500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10827.872371 secs 2024-04-26T21:35:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:46 smithi090 ceph-mon[29180]: pgmap v6500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10827.872371 secs 2024-04-26T21:35:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:48 smithi119 ceph-mon[32752]: pgmap v6501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:48 smithi167 ceph-mon[32751]: pgmap v6501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:48 smithi090 ceph-mon[29180]: pgmap v6501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:50 smithi119 ceph-mon[32752]: pgmap v6502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10832.141656 secs 2024-04-26T21:35:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:50 smithi167 ceph-mon[32751]: pgmap v6502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10832.141656 secs 2024-04-26T21:35:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:50 smithi090 ceph-mon[29180]: pgmap v6502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10832.141656 secs 2024-04-26T21:35:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10832.872484 secs 2024-04-26T21:35:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10832.872484 secs 2024-04-26T21:35:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10832.872484 secs 2024-04-26T21:35:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:52 smithi119 ceph-mon[32752]: pgmap v6503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:52 smithi167 ceph-mon[32751]: pgmap v6503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:52 smithi090 ceph-mon[29180]: pgmap v6503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:54 smithi119 ceph-mon[32752]: pgmap v6504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:54 smithi167 ceph-mon[32751]: pgmap v6504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:54 smithi090 ceph-mon[29180]: pgmap v6504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10837.141715 secs 2024-04-26T21:35:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10837.141715 secs 2024-04-26T21:35:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10837.141715 secs 2024-04-26T21:35:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:56 smithi119 ceph-mon[32752]: pgmap v6505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10837.872604 secs 2024-04-26T21:35:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:56 smithi167 ceph-mon[32751]: pgmap v6505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10837.872604 secs 2024-04-26T21:35:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:56 smithi090 ceph-mon[29180]: pgmap v6505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:35:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10837.872604 secs 2024-04-26T21:35:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:35:58 smithi119 ceph-mon[32752]: pgmap v6506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:35:58 smithi167 ceph-mon[32751]: pgmap v6506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:35:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:35:58 smithi090 ceph-mon[29180]: pgmap v6506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:00 smithi119 ceph-mon[32752]: pgmap v6507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10842.141841 secs 2024-04-26T21:36:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:00 smithi167 ceph-mon[32751]: pgmap v6507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10842.141841 secs 2024-04-26T21:36:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:00 smithi090 ceph-mon[29180]: pgmap v6507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10842.141841 secs 2024-04-26T21:36:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10842.872713 secs 2024-04-26T21:36:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10842.872713 secs 2024-04-26T21:36:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10842.872713 secs 2024-04-26T21:36:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:02 smithi119 ceph-mon[32752]: pgmap v6508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:02 smithi167 ceph-mon[32751]: pgmap v6508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:02 smithi090 ceph-mon[29180]: pgmap v6508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:04 smithi119 ceph-mon[32752]: pgmap v6509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:04 smithi167 ceph-mon[32751]: pgmap v6509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:04 smithi090 ceph-mon[29180]: pgmap v6509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10847.141930 secs 2024-04-26T21:36:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10847.141930 secs 2024-04-26T21:36:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10847.141930 secs 2024-04-26T21:36:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:06 smithi119 ceph-mon[32752]: pgmap v6510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10847.872786 secs 2024-04-26T21:36:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:06 smithi167 ceph-mon[32751]: pgmap v6510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10847.872786 secs 2024-04-26T21:36:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:06 smithi090 ceph-mon[29180]: pgmap v6510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10847.872786 secs 2024-04-26T21:36:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:08 smithi119 ceph-mon[32752]: pgmap v6511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:08 smithi167 ceph-mon[32751]: pgmap v6511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:08 smithi090 ceph-mon[29180]: pgmap v6511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:10 smithi119 ceph-mon[32752]: pgmap v6512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10852.142015 secs 2024-04-26T21:36:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:10 smithi167 ceph-mon[32751]: pgmap v6512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10852.142015 secs 2024-04-26T21:36:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:10 smithi090 ceph-mon[29180]: pgmap v6512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10852.142015 secs 2024-04-26T21:36:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10852.872863 secs 2024-04-26T21:36:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10852.872863 secs 2024-04-26T21:36:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10852.872863 secs 2024-04-26T21:36:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:12 smithi119 ceph-mon[32752]: pgmap v6513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:12 smithi167 ceph-mon[32751]: pgmap v6513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:12 smithi090 ceph-mon[29180]: pgmap v6513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:14 smithi119 ceph-mon[32752]: pgmap v6514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:14 smithi167 ceph-mon[32751]: pgmap v6514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:14 smithi090 ceph-mon[29180]: pgmap v6514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10857.142077 secs 2024-04-26T21:36:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10857.142077 secs 2024-04-26T21:36:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10857.142077 secs 2024-04-26T21:36:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:16 smithi119 ceph-mon[32752]: pgmap v6515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10857.872978 secs 2024-04-26T21:36:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:16 smithi167 ceph-mon[32751]: pgmap v6515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10857.872978 secs 2024-04-26T21:36:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:16 smithi090 ceph-mon[29180]: pgmap v6515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10857.872978 secs 2024-04-26T21:36:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:18 smithi119 ceph-mon[32752]: pgmap v6516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:18 smithi167 ceph-mon[32751]: pgmap v6516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:18 smithi090 ceph-mon[29180]: pgmap v6516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:20 smithi119 ceph-mon[32752]: pgmap v6517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10862.142164 secs 2024-04-26T21:36:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:20 smithi167 ceph-mon[32751]: pgmap v6517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10862.142164 secs 2024-04-26T21:36:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:20 smithi090 ceph-mon[29180]: pgmap v6517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10862.142164 secs 2024-04-26T21:36:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10862.873079 secs 2024-04-26T21:36:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10862.873079 secs 2024-04-26T21:36:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10862.873079 secs 2024-04-26T21:36:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:22 smithi119 ceph-mon[32752]: pgmap v6518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:22 smithi167 ceph-mon[32751]: pgmap v6518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:22 smithi090 ceph-mon[29180]: pgmap v6518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:24 smithi119 ceph-mon[32752]: pgmap v6519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:24 smithi167 ceph-mon[32751]: pgmap v6519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:24 smithi090 ceph-mon[29180]: pgmap v6519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10867.142248 secs 2024-04-26T21:36:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10867.142248 secs 2024-04-26T21:36:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10867.142248 secs 2024-04-26T21:36:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:26 smithi167 ceph-mon[32751]: pgmap v6520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10867.873141 secs 2024-04-26T21:36:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:26 smithi090 ceph-mon[29180]: pgmap v6520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10867.873141 secs 2024-04-26T21:36:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:26 smithi119 ceph-mon[32752]: pgmap v6520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10867.873141 secs 2024-04-26T21:36:28.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:28 smithi090 ceph-mon[29180]: pgmap v6521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:28 smithi167 ceph-mon[32751]: pgmap v6521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:28 smithi119 ceph-mon[32752]: pgmap v6521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:30 smithi167 ceph-mon[32751]: pgmap v6522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10872.142332 secs 2024-04-26T21:36:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:30 smithi090 ceph-mon[29180]: pgmap v6522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10872.142332 secs 2024-04-26T21:36:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:30 smithi119 ceph-mon[32752]: pgmap v6522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10872.142332 secs 2024-04-26T21:36:32.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10872.873261 secs 2024-04-26T21:36:32.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:36:32.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.087 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10872.873261 secs 2024-04-26T21:36:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:36:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10872.873261 secs 2024-04-26T21:36:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:36:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:32 smithi119 ceph-mon[32752]: pgmap v6523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:32 smithi090 ceph-mon[29180]: pgmap v6523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:32 smithi167 ceph-mon[32751]: pgmap v6523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: pgmap v6524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: pgmap v6524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: pgmap v6524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:34.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:34.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:36:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:36:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:36:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:36:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:36:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: pgmap v6525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:36:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: pgmap v6526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:36:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10877.142419 secs 2024-04-26T21:36:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:36:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:36:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:36:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:36:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:36:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:36:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: pgmap v6525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:36:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: pgmap v6526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:36:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10877.142419 secs 2024-04-26T21:36:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:36:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:36:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:36:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:36:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:36:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: pgmap v6525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:36:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: pgmap v6526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:36:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:36:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10877.142419 secs 2024-04-26T21:36:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10877.873396 secs 2024-04-26T21:36:36.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10877.873396 secs 2024-04-26T21:36:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10877.873396 secs 2024-04-26T21:36:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:37 smithi090 ceph-mon[29180]: pgmap v6527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:36:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:37 smithi119 ceph-mon[32752]: pgmap v6527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:36:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:37 smithi167 ceph-mon[32751]: pgmap v6527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T21:36:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:39 smithi090 ceph-mon[29180]: pgmap v6528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:39 smithi119 ceph-mon[32752]: pgmap v6528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:39 smithi167 ceph-mon[32751]: pgmap v6528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10882.142505 secs 2024-04-26T21:36:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10882.142505 secs 2024-04-26T21:36:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10882.142505 secs 2024-04-26T21:36:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:41 smithi090 ceph-mon[29180]: pgmap v6529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10882.873463 secs 2024-04-26T21:36:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:41 smithi119 ceph-mon[32752]: pgmap v6529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10882.873463 secs 2024-04-26T21:36:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:41 smithi167 ceph-mon[32751]: pgmap v6529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10882.873463 secs 2024-04-26T21:36:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:43 smithi090 ceph-mon[29180]: pgmap v6530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:43 smithi119 ceph-mon[32752]: pgmap v6530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:43 smithi167 ceph-mon[32751]: pgmap v6530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 214 B/s wr, 2 op/s 2024-04-26T21:36:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:45 smithi090 ceph-mon[29180]: pgmap v6531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T21:36:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10887.142592 secs 2024-04-26T21:36:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:45 smithi119 ceph-mon[32752]: pgmap v6531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T21:36:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10887.142592 secs 2024-04-26T21:36:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:45 smithi167 ceph-mon[32751]: pgmap v6531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T21:36:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10887.142592 secs 2024-04-26T21:36:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10887.873580 secs 2024-04-26T21:36:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10887.873580 secs 2024-04-26T21:36:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10887.873580 secs 2024-04-26T21:36:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:47 smithi090 ceph-mon[29180]: pgmap v6532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:36:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:47 smithi119 ceph-mon[32752]: pgmap v6532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:36:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:47 smithi167 ceph-mon[32751]: pgmap v6532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:36:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:49 smithi090 ceph-mon[29180]: pgmap v6533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:36:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:49 smithi119 ceph-mon[32752]: pgmap v6533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:36:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:49 smithi167 ceph-mon[32751]: pgmap v6533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:36:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10892.142679 secs 2024-04-26T21:36:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10892.142679 secs 2024-04-26T21:36:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10892.142679 secs 2024-04-26T21:36:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:51 smithi090 ceph-mon[29180]: pgmap v6534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10892.873655 secs 2024-04-26T21:36:51.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:51 smithi119 ceph-mon[32752]: pgmap v6534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:51.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10892.873655 secs 2024-04-26T21:36:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:51 smithi167 ceph-mon[32751]: pgmap v6534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10892.873655 secs 2024-04-26T21:36:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:53 smithi090 ceph-mon[29180]: pgmap v6535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:53 smithi119 ceph-mon[32752]: pgmap v6535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:53 smithi167 ceph-mon[32751]: pgmap v6535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:55 smithi090 ceph-mon[29180]: pgmap v6536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10897.142767 secs 2024-04-26T21:36:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:55 smithi119 ceph-mon[32752]: pgmap v6536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10897.142767 secs 2024-04-26T21:36:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:55 smithi167 ceph-mon[32751]: pgmap v6536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:36:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10897.142767 secs 2024-04-26T21:36:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10897.873724 secs 2024-04-26T21:36:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10897.873724 secs 2024-04-26T21:36:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10897.873724 secs 2024-04-26T21:36:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:57 smithi119 ceph-mon[32752]: pgmap v6537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:57 smithi167 ceph-mon[32751]: pgmap v6537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:57 smithi090 ceph-mon[29180]: pgmap v6537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:36:59.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:36:59 smithi090 ceph-mon[29180]: pgmap v6538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:36:59 smithi119 ceph-mon[32752]: pgmap v6538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:36:59 smithi167 ceph-mon[32751]: pgmap v6538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10902.142858 secs 2024-04-26T21:37:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10902.142858 secs 2024-04-26T21:37:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10902.142858 secs 2024-04-26T21:37:02.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:01 smithi090 ceph-mon[29180]: pgmap v6539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:02.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10902.873839 secs 2024-04-26T21:37:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:01 smithi119 ceph-mon[32752]: pgmap v6539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10902.873839 secs 2024-04-26T21:37:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:01 smithi167 ceph-mon[32751]: pgmap v6539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10902.873839 secs 2024-04-26T21:37:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:03 smithi119 ceph-mon[32752]: pgmap v6540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:03 smithi167 ceph-mon[32751]: pgmap v6540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:03 smithi090 ceph-mon[29180]: pgmap v6540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:04.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:37:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:37:03.711+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:37:04.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:37:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:37:03.711+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:37:04.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:37:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:37:03.711+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:37:04.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:37:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:37:03.711+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:37:04.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:37:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:37:03.711+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:37:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:05 smithi119 ceph-mon[32752]: pgmap v6541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10907.142920 secs 2024-04-26T21:37:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:05 smithi167 ceph-mon[32751]: pgmap v6541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10907.142920 secs 2024-04-26T21:37:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:05 smithi090 ceph-mon[29180]: pgmap v6541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10907.142920 secs 2024-04-26T21:37:06.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10907.873962 secs 2024-04-26T21:37:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10907.873962 secs 2024-04-26T21:37:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10907.873962 secs 2024-04-26T21:37:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:07 smithi119 ceph-mon[32752]: pgmap v6542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:07 smithi167 ceph-mon[32751]: pgmap v6542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:07 smithi090 ceph-mon[29180]: pgmap v6542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:09 smithi119 ceph-mon[32752]: pgmap v6543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:09 smithi167 ceph-mon[32751]: pgmap v6543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:09 smithi090 ceph-mon[29180]: pgmap v6543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10912.142982 secs 2024-04-26T21:37:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10912.142982 secs 2024-04-26T21:37:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10912.142982 secs 2024-04-26T21:37:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:11 smithi119 ceph-mon[32752]: pgmap v6544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10912.874078 secs 2024-04-26T21:37:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:11 smithi167 ceph-mon[32751]: pgmap v6544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10912.874078 secs 2024-04-26T21:37:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:11 smithi090 ceph-mon[29180]: pgmap v6544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10912.874078 secs 2024-04-26T21:37:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:13 smithi119 ceph-mon[32752]: pgmap v6545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:13 smithi167 ceph-mon[32751]: pgmap v6545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:13 smithi090 ceph-mon[29180]: pgmap v6545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:15 smithi119 ceph-mon[32752]: pgmap v6546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10917.143040 secs 2024-04-26T21:37:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:15 smithi167 ceph-mon[32751]: pgmap v6546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10917.143040 secs 2024-04-26T21:37:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:15 smithi090 ceph-mon[29180]: pgmap v6546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10917.143040 secs 2024-04-26T21:37:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10917.874193 secs 2024-04-26T21:37:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10917.874193 secs 2024-04-26T21:37:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10917.874193 secs 2024-04-26T21:37:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:17 smithi119 ceph-mon[32752]: pgmap v6547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:18.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:17 smithi167 ceph-mon[32751]: pgmap v6547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:17 smithi090 ceph-mon[29180]: pgmap v6547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:19 smithi119 ceph-mon[32752]: pgmap v6548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:19 smithi167 ceph-mon[32751]: pgmap v6548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:19 smithi090 ceph-mon[29180]: pgmap v6548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10922.143102 secs 2024-04-26T21:37:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10922.143102 secs 2024-04-26T21:37:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10922.143102 secs 2024-04-26T21:37:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:21 smithi119 ceph-mon[32752]: pgmap v6549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10922.874310 secs 2024-04-26T21:37:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:21 smithi167 ceph-mon[32751]: pgmap v6549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10922.874310 secs 2024-04-26T21:37:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:21 smithi090 ceph-mon[29180]: pgmap v6549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10922.874310 secs 2024-04-26T21:37:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:23 smithi119 ceph-mon[32752]: pgmap v6550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:23 smithi167 ceph-mon[32751]: pgmap v6550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:23 smithi090 ceph-mon[29180]: pgmap v6550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:25 smithi119 ceph-mon[32752]: pgmap v6551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10927.143188 secs 2024-04-26T21:37:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:25 smithi167 ceph-mon[32751]: pgmap v6551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10927.143188 secs 2024-04-26T21:37:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:25 smithi090 ceph-mon[29180]: pgmap v6551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10927.143188 secs 2024-04-26T21:37:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10927.874425 secs 2024-04-26T21:37:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10927.874425 secs 2024-04-26T21:37:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10927.874425 secs 2024-04-26T21:37:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:27 smithi119 ceph-mon[32752]: pgmap v6552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:27 smithi167 ceph-mon[32751]: pgmap v6552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:27 smithi090 ceph-mon[29180]: pgmap v6552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:29 smithi119 ceph-mon[32752]: pgmap v6553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:29 smithi167 ceph-mon[32751]: pgmap v6553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:29 smithi090 ceph-mon[29180]: pgmap v6553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10932.143262 secs 2024-04-26T21:37:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10932.143262 secs 2024-04-26T21:37:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10932.143262 secs 2024-04-26T21:37:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:31 smithi090 ceph-mon[29180]: pgmap v6554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10932.874546 secs 2024-04-26T21:37:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:31 smithi119 ceph-mon[32752]: pgmap v6554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10932.874546 secs 2024-04-26T21:37:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:31 smithi167 ceph-mon[32751]: pgmap v6554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10932.874546 secs 2024-04-26T21:37:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:33 smithi119 ceph-mon[32752]: pgmap v6555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:33 smithi167 ceph-mon[32751]: pgmap v6555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:33 smithi090 ceph-mon[29180]: pgmap v6555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:37:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:37:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:37:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:35 smithi119 ceph-mon[32752]: pgmap v6556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10937.143333 secs 2024-04-26T21:37:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:35 smithi167 ceph-mon[32751]: pgmap v6556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10937.143333 secs 2024-04-26T21:37:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:35 smithi090 ceph-mon[29180]: pgmap v6556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10937.143333 secs 2024-04-26T21:37:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10937.874636 secs 2024-04-26T21:37:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:37:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:37:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:36 smithi119 ceph-mon[32752]: pgmap v6557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T21:37:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:36 smithi119 ceph-mon[32752]: pgmap v6558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10937.874636 secs 2024-04-26T21:37:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:37:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:37:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:36 smithi167 ceph-mon[32751]: pgmap v6557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T21:37:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:36 smithi167 ceph-mon[32751]: pgmap v6558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10937.874636 secs 2024-04-26T21:37:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:37:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:37:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:36 smithi090 ceph-mon[29180]: pgmap v6557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-26T21:37:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:36 smithi090 ceph-mon[29180]: pgmap v6558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:37:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:38 smithi090 ceph-mon[29180]: pgmap v6559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T21:37:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:38 smithi119 ceph-mon[32752]: pgmap v6559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T21:37:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:38 smithi167 ceph-mon[32751]: pgmap v6559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T21:37:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:40 smithi090 ceph-mon[29180]: pgmap v6560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10942.143393 secs 2024-04-26T21:37:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:40 smithi119 ceph-mon[32752]: pgmap v6560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10942.143393 secs 2024-04-26T21:37:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:40 smithi167 ceph-mon[32751]: pgmap v6560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10942.143393 secs 2024-04-26T21:37:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10942.874751 secs 2024-04-26T21:37:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10942.874751 secs 2024-04-26T21:37:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10942.874751 secs 2024-04-26T21:37:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:42 smithi090 ceph-mon[29180]: pgmap v6561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:42 smithi119 ceph-mon[32752]: pgmap v6561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:42 smithi167 ceph-mon[32751]: pgmap v6561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:44 smithi090 ceph-mon[29180]: pgmap v6562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:44 smithi119 ceph-mon[32752]: pgmap v6562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:44 smithi167 ceph-mon[32751]: pgmap v6562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:37:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10947.143475 secs 2024-04-26T21:37:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10947.143475 secs 2024-04-26T21:37:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10947.143475 secs 2024-04-26T21:37:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10947.874866 secs 2024-04-26T21:37:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:46 smithi090 ceph-mon[29180]: pgmap v6563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:37:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10947.874866 secs 2024-04-26T21:37:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:46 smithi119 ceph-mon[32752]: pgmap v6563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:37:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10947.874866 secs 2024-04-26T21:37:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:46 smithi167 ceph-mon[32751]: pgmap v6563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:37:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:48 smithi090 ceph-mon[29180]: pgmap v6564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:48 smithi119 ceph-mon[32752]: pgmap v6564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:48 smithi167 ceph-mon[32751]: pgmap v6564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:50 smithi090 ceph-mon[29180]: pgmap v6565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10952.143558 secs 2024-04-26T21:37:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:50 smithi119 ceph-mon[32752]: pgmap v6565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10952.143558 secs 2024-04-26T21:37:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:50 smithi167 ceph-mon[32751]: pgmap v6565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10952.143558 secs 2024-04-26T21:37:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10952.874985 secs 2024-04-26T21:37:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10952.874985 secs 2024-04-26T21:37:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10952.874985 secs 2024-04-26T21:37:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:52 smithi119 ceph-mon[32752]: pgmap v6566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:52 smithi167 ceph-mon[32751]: pgmap v6566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:52 smithi090 ceph-mon[29180]: pgmap v6566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:54 smithi119 ceph-mon[32752]: pgmap v6567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:54 smithi167 ceph-mon[32751]: pgmap v6567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:54 smithi090 ceph-mon[29180]: pgmap v6567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:37:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10957.143643 secs 2024-04-26T21:37:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10957.143643 secs 2024-04-26T21:37:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10957.143643 secs 2024-04-26T21:37:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10957.875102 secs 2024-04-26T21:37:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:56 smithi119 ceph-mon[32752]: pgmap v6568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10957.875102 secs 2024-04-26T21:37:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:56 smithi167 ceph-mon[32751]: pgmap v6568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10957.875102 secs 2024-04-26T21:37:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:56 smithi090 ceph-mon[29180]: pgmap v6568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:37:58 smithi119 ceph-mon[32752]: pgmap v6569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:37:58 smithi167 ceph-mon[32751]: pgmap v6569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:37:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:37:58 smithi090 ceph-mon[29180]: pgmap v6569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:00 smithi119 ceph-mon[32752]: pgmap v6570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10962.143730 secs 2024-04-26T21:38:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:00 smithi167 ceph-mon[32751]: pgmap v6570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10962.143730 secs 2024-04-26T21:38:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:00 smithi090 ceph-mon[29180]: pgmap v6570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10962.143730 secs 2024-04-26T21:38:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10962.875238 secs 2024-04-26T21:38:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10962.875238 secs 2024-04-26T21:38:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10962.875238 secs 2024-04-26T21:38:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:02 smithi119 ceph-mon[32752]: pgmap v6571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:02 smithi167 ceph-mon[32751]: pgmap v6571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:02 smithi090 ceph-mon[29180]: pgmap v6571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:04 smithi119 ceph-mon[32752]: pgmap v6572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:04 smithi167 ceph-mon[32751]: pgmap v6572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:04 smithi090 ceph-mon[29180]: pgmap v6572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:06.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10967.143816 secs 2024-04-26T21:38:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10967.143816 secs 2024-04-26T21:38:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10967.143816 secs 2024-04-26T21:38:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10967.875347 secs 2024-04-26T21:38:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:06 smithi119 ceph-mon[32752]: pgmap v6573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10967.875347 secs 2024-04-26T21:38:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:06 smithi167 ceph-mon[32751]: pgmap v6573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10967.875347 secs 2024-04-26T21:38:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:06 smithi090 ceph-mon[29180]: pgmap v6573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:08 smithi119 ceph-mon[32752]: pgmap v6574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:08 smithi167 ceph-mon[32751]: pgmap v6574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:09.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:08 smithi090 ceph-mon[29180]: pgmap v6574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:10 smithi119 ceph-mon[32752]: pgmap v6575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10972.143875 secs 2024-04-26T21:38:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:10 smithi167 ceph-mon[32751]: pgmap v6575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10972.143875 secs 2024-04-26T21:38:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:10 smithi090 ceph-mon[29180]: pgmap v6575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10972.143875 secs 2024-04-26T21:38:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10972.875468 secs 2024-04-26T21:38:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10972.875468 secs 2024-04-26T21:38:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10972.875468 secs 2024-04-26T21:38:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:12 smithi119 ceph-mon[32752]: pgmap v6576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:12 smithi167 ceph-mon[32751]: pgmap v6576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:12 smithi090 ceph-mon[29180]: pgmap v6576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:14 smithi119 ceph-mon[32752]: pgmap v6577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:14 smithi167 ceph-mon[32751]: pgmap v6577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:14 smithi090 ceph-mon[29180]: pgmap v6577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10977.143958 secs 2024-04-26T21:38:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10977.143958 secs 2024-04-26T21:38:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10977.143958 secs 2024-04-26T21:38:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10977.875565 secs 2024-04-26T21:38:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:16 smithi119 ceph-mon[32752]: pgmap v6578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10977.875565 secs 2024-04-26T21:38:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:16 smithi167 ceph-mon[32751]: pgmap v6578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10977.875565 secs 2024-04-26T21:38:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:16 smithi090 ceph-mon[29180]: pgmap v6578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:17 smithi090 ceph-mon[29180]: pgmap v6579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:17 smithi119 ceph-mon[32752]: pgmap v6579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:17 smithi167 ceph-mon[32751]: pgmap v6579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:20 smithi090 ceph-mon[29180]: pgmap v6580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10982.144018 secs 2024-04-26T21:38:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:20 smithi119 ceph-mon[32752]: pgmap v6580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10982.144018 secs 2024-04-26T21:38:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:20 smithi167 ceph-mon[32751]: pgmap v6580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10982.144018 secs 2024-04-26T21:38:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10982.875684 secs 2024-04-26T21:38:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10982.875684 secs 2024-04-26T21:38:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10982.875684 secs 2024-04-26T21:38:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:22 smithi090 ceph-mon[29180]: pgmap v6581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:22 smithi119 ceph-mon[32752]: pgmap v6581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:22 smithi167 ceph-mon[32751]: pgmap v6581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:24 smithi090 ceph-mon[29180]: pgmap v6582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:24 smithi119 ceph-mon[32752]: pgmap v6582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:24 smithi167 ceph-mon[32751]: pgmap v6582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10987.144081 secs 2024-04-26T21:38:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10987.144081 secs 2024-04-26T21:38:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10987.144081 secs 2024-04-26T21:38:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10987.875797 secs 2024-04-26T21:38:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:26 smithi090 ceph-mon[29180]: pgmap v6583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10987.875797 secs 2024-04-26T21:38:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:26 smithi119 ceph-mon[32752]: pgmap v6583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10987.875797 secs 2024-04-26T21:38:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:26 smithi167 ceph-mon[32751]: pgmap v6583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:28 smithi119 ceph-mon[32752]: pgmap v6584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:28 smithi167 ceph-mon[32751]: pgmap v6584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:28 smithi090 ceph-mon[29180]: pgmap v6584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:30 smithi119 ceph-mon[32752]: pgmap v6585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10992.144187 secs 2024-04-26T21:38:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:30 smithi167 ceph-mon[32751]: pgmap v6585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10992.144187 secs 2024-04-26T21:38:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:30 smithi090 ceph-mon[29180]: pgmap v6585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10992.144187 secs 2024-04-26T21:38:32.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10992.875914 secs 2024-04-26T21:38:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10992.875914 secs 2024-04-26T21:38:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10992.875914 secs 2024-04-26T21:38:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:32 smithi119 ceph-mon[32752]: pgmap v6586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:32 smithi167 ceph-mon[32751]: pgmap v6586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:32 smithi090 ceph-mon[29180]: pgmap v6586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:34 smithi119 ceph-mon[32752]: pgmap v6587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:34 smithi167 ceph-mon[32751]: pgmap v6587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:34 smithi090 ceph-mon[29180]: pgmap v6587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:36.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 10997.144266 secs 2024-04-26T21:38:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 10997.144266 secs 2024-04-26T21:38:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 10997.144266 secs 2024-04-26T21:38:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 10997.876050 secs 2024-04-26T21:38:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:36 smithi119 ceph-mon[32752]: pgmap v6588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:38:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:38:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:38:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:38:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 10997.876050 secs 2024-04-26T21:38:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:36 smithi167 ceph-mon[32751]: pgmap v6588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:38:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:38:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:38:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:38:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 10997.876050 secs 2024-04-26T21:38:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:36 smithi090 ceph-mon[29180]: pgmap v6588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:38:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:38:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:38:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:37 smithi119 ceph-mon[32752]: pgmap v6589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:37 smithi119 ceph-mon[32752]: pgmap v6590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:37 smithi167 ceph-mon[32751]: pgmap v6589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:37 smithi167 ceph-mon[32751]: pgmap v6590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:37 smithi090 ceph-mon[29180]: pgmap v6589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:37 smithi090 ceph-mon[29180]: pgmap v6590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:39 smithi119 ceph-mon[32752]: pgmap v6591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:39 smithi167 ceph-mon[32751]: pgmap v6591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:39 smithi090 ceph-mon[29180]: pgmap v6591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11002.144352 secs 2024-04-26T21:38:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11002.144352 secs 2024-04-26T21:38:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11002.144352 secs 2024-04-26T21:38:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:41 smithi119 ceph-mon[32752]: pgmap v6592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11002.876165 secs 2024-04-26T21:38:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:41 smithi167 ceph-mon[32751]: pgmap v6592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11002.876165 secs 2024-04-26T21:38:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:41 smithi090 ceph-mon[29180]: pgmap v6592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11002.876165 secs 2024-04-26T21:38:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:43 smithi119 ceph-mon[32752]: pgmap v6593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:38:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:43 smithi167 ceph-mon[32751]: pgmap v6593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:38:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:43 smithi090 ceph-mon[29180]: pgmap v6593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:38:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:45 smithi119 ceph-mon[32752]: pgmap v6594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11007.144438 secs 2024-04-26T21:38:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:45 smithi167 ceph-mon[32751]: pgmap v6594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11007.144438 secs 2024-04-26T21:38:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:45 smithi090 ceph-mon[29180]: pgmap v6594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:38:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11007.144438 secs 2024-04-26T21:38:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11007.876329 secs 2024-04-26T21:38:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11007.876329 secs 2024-04-26T21:38:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11007.876329 secs 2024-04-26T21:38:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:47 smithi119 ceph-mon[32752]: pgmap v6595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:38:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:47 smithi167 ceph-mon[32751]: pgmap v6595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:38:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:47 smithi090 ceph-mon[29180]: pgmap v6595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:38:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:49 smithi119 ceph-mon[32752]: pgmap v6596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:49 smithi167 ceph-mon[32751]: pgmap v6596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:49 smithi090 ceph-mon[29180]: pgmap v6596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:51.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11012.144523 secs 2024-04-26T21:38:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11012.144523 secs 2024-04-26T21:38:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11012.144523 secs 2024-04-26T21:38:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:51 smithi119 ceph-mon[32752]: pgmap v6597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11012.876401 secs 2024-04-26T21:38:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:51 smithi167 ceph-mon[32751]: pgmap v6597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11012.876401 secs 2024-04-26T21:38:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:51 smithi090 ceph-mon[29180]: pgmap v6597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11012.876401 secs 2024-04-26T21:38:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:52 smithi119 ceph-mon[32752]: pgmap v6598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:52 smithi167 ceph-mon[32751]: pgmap v6598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:53.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:52 smithi090 ceph-mon[29180]: pgmap v6598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:55 smithi119 ceph-mon[32752]: pgmap v6599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11017.144584 secs 2024-04-26T21:38:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:55 smithi167 ceph-mon[32751]: pgmap v6599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11017.144584 secs 2024-04-26T21:38:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:55 smithi090 ceph-mon[29180]: pgmap v6599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:38:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11017.144584 secs 2024-04-26T21:38:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11017.876519 secs 2024-04-26T21:38:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11017.876519 secs 2024-04-26T21:38:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11017.876519 secs 2024-04-26T21:38:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:57 smithi119 ceph-mon[32752]: pgmap v6600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:57 smithi167 ceph-mon[32751]: pgmap v6600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:38:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:57 smithi090 ceph-mon[29180]: pgmap v6600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:38:59 smithi119 ceph-mon[32752]: pgmap v6601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:38:59 smithi167 ceph-mon[32751]: pgmap v6601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:38:59 smithi090 ceph-mon[29180]: pgmap v6601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11022.144668 secs 2024-04-26T21:39:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11022.144668 secs 2024-04-26T21:39:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11022.144668 secs 2024-04-26T21:39:02.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:01 smithi090 ceph-mon[29180]: pgmap v6602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:02.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11022.876670 secs 2024-04-26T21:39:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:01 smithi119 ceph-mon[32752]: pgmap v6602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11022.876670 secs 2024-04-26T21:39:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:01 smithi167 ceph-mon[32751]: pgmap v6602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11022.876670 secs 2024-04-26T21:39:04.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:03 smithi119 ceph-mon[32752]: pgmap v6603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:03 smithi167 ceph-mon[32751]: pgmap v6603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:03 smithi090 ceph-mon[29180]: pgmap v6603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:05 smithi119 ceph-mon[32752]: pgmap v6604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11027.144751 secs 2024-04-26T21:39:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:05 smithi167 ceph-mon[32751]: pgmap v6604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11027.144751 secs 2024-04-26T21:39:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:05 smithi090 ceph-mon[29180]: pgmap v6604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11027.144751 secs 2024-04-26T21:39:06.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11027.876779 secs 2024-04-26T21:39:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11027.876779 secs 2024-04-26T21:39:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11027.876779 secs 2024-04-26T21:39:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:07 smithi119 ceph-mon[32752]: pgmap v6605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:07 smithi167 ceph-mon[32751]: pgmap v6605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:07 smithi090 ceph-mon[29180]: pgmap v6605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:09 smithi119 ceph-mon[32752]: pgmap v6606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:09 smithi167 ceph-mon[32751]: pgmap v6606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:09 smithi090 ceph-mon[29180]: pgmap v6606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11032.144848 secs 2024-04-26T21:39:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11032.144848 secs 2024-04-26T21:39:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11032.144848 secs 2024-04-26T21:39:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:11 smithi119 ceph-mon[32752]: pgmap v6607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11032.876888 secs 2024-04-26T21:39:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:11 smithi167 ceph-mon[32751]: pgmap v6607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11032.876888 secs 2024-04-26T21:39:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:11 smithi090 ceph-mon[29180]: pgmap v6607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11032.876888 secs 2024-04-26T21:39:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:13 smithi119 ceph-mon[32752]: pgmap v6608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:13 smithi167 ceph-mon[32751]: pgmap v6608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:13 smithi090 ceph-mon[29180]: pgmap v6608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:15 smithi119 ceph-mon[32752]: pgmap v6609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11037.144931 secs 2024-04-26T21:39:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:15 smithi167 ceph-mon[32751]: pgmap v6609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11037.144931 secs 2024-04-26T21:39:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:15 smithi090 ceph-mon[29180]: pgmap v6609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11037.144931 secs 2024-04-26T21:39:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11037.877030 secs 2024-04-26T21:39:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11037.877030 secs 2024-04-26T21:39:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11037.877030 secs 2024-04-26T21:39:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:17 smithi119 ceph-mon[32752]: pgmap v6610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:17 smithi167 ceph-mon[32751]: pgmap v6610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:17 smithi090 ceph-mon[29180]: pgmap v6610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:19 smithi119 ceph-mon[32752]: pgmap v6611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:19 smithi167 ceph-mon[32751]: pgmap v6611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:19 smithi090 ceph-mon[29180]: pgmap v6611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11042.144994 secs 2024-04-26T21:39:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11042.144994 secs 2024-04-26T21:39:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11042.144994 secs 2024-04-26T21:39:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:21 smithi119 ceph-mon[32752]: pgmap v6612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11042.877145 secs 2024-04-26T21:39:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:21 smithi167 ceph-mon[32751]: pgmap v6612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11042.877145 secs 2024-04-26T21:39:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:21 smithi090 ceph-mon[29180]: pgmap v6612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11042.877145 secs 2024-04-26T21:39:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:23 smithi119 ceph-mon[32752]: pgmap v6613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:23 smithi167 ceph-mon[32751]: pgmap v6613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:23 smithi090 ceph-mon[29180]: pgmap v6613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:25 smithi119 ceph-mon[32752]: pgmap v6614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11047.145080 secs 2024-04-26T21:39:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:25 smithi167 ceph-mon[32751]: pgmap v6614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11047.145080 secs 2024-04-26T21:39:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:25 smithi090 ceph-mon[29180]: pgmap v6614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11047.145080 secs 2024-04-26T21:39:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11047.877225 secs 2024-04-26T21:39:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11047.877225 secs 2024-04-26T21:39:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11047.877225 secs 2024-04-26T21:39:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:27 smithi119 ceph-mon[32752]: pgmap v6615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:27 smithi167 ceph-mon[32751]: pgmap v6615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:27 smithi090 ceph-mon[29180]: pgmap v6615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:29 smithi119 ceph-mon[32752]: pgmap v6616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:29 smithi167 ceph-mon[32751]: pgmap v6616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:29 smithi090 ceph-mon[29180]: pgmap v6616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11052.145166 secs 2024-04-26T21:39:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11052.145166 secs 2024-04-26T21:39:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11052.145166 secs 2024-04-26T21:39:32.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:31 smithi090 ceph-mon[29180]: pgmap v6617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:32.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11052.877340 secs 2024-04-26T21:39:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:31 smithi119 ceph-mon[32752]: pgmap v6617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11052.877340 secs 2024-04-26T21:39:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:31 smithi167 ceph-mon[32751]: pgmap v6617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11052.877340 secs 2024-04-26T21:39:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:33 smithi119 ceph-mon[32752]: pgmap v6618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:33 smithi167 ceph-mon[32751]: pgmap v6618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:33 smithi090 ceph-mon[29180]: pgmap v6618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:35 smithi090 ceph-mon[29180]: pgmap v6619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11057.145251 secs 2024-04-26T21:39:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:35 smithi119 ceph-mon[32752]: pgmap v6619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11057.145251 secs 2024-04-26T21:39:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:35 smithi167 ceph-mon[32751]: pgmap v6619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11057.145251 secs 2024-04-26T21:39:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11057.877469 secs 2024-04-26T21:39:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:39:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11057.877469 secs 2024-04-26T21:39:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:39:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11057.877469 secs 2024-04-26T21:39:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:39:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:37 smithi119 ceph-mon[32752]: pgmap v6620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:39:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:39:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:39:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:37 smithi167 ceph-mon[32751]: pgmap v6620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:39:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:39:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:39:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:37 smithi090 ceph-mon[29180]: pgmap v6620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:39:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:39:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:39:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:38 smithi119 ceph-mon[32752]: pgmap v6621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:39:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:38 smithi119 ceph-mon[32752]: pgmap v6622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:39:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:38 smithi167 ceph-mon[32751]: pgmap v6621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:39:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:38 smithi167 ceph-mon[32751]: pgmap v6622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:39:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:38 smithi090 ceph-mon[29180]: pgmap v6621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:39:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:38 smithi090 ceph-mon[29180]: pgmap v6622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:39:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:40 smithi119 ceph-mon[32752]: pgmap v6623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11062.145336 secs 2024-04-26T21:39:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:40 smithi167 ceph-mon[32751]: pgmap v6623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11062.145336 secs 2024-04-26T21:39:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:40 smithi090 ceph-mon[29180]: pgmap v6623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11062.145336 secs 2024-04-26T21:39:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11062.877557 secs 2024-04-26T21:39:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11062.877557 secs 2024-04-26T21:39:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11062.877557 secs 2024-04-26T21:39:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:42 smithi119 ceph-mon[32752]: pgmap v6624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:42 smithi167 ceph-mon[32751]: pgmap v6624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:42 smithi090 ceph-mon[29180]: pgmap v6624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:44 smithi119 ceph-mon[32752]: pgmap v6625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:44 smithi167 ceph-mon[32751]: pgmap v6625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:44 smithi090 ceph-mon[29180]: pgmap v6625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11067.145420 secs 2024-04-26T21:39:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11067.145420 secs 2024-04-26T21:39:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11067.145420 secs 2024-04-26T21:39:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:46 smithi119 ceph-mon[32752]: pgmap v6626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11067.877686 secs 2024-04-26T21:39:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:46 smithi167 ceph-mon[32751]: pgmap v6626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11067.877686 secs 2024-04-26T21:39:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:46 smithi090 ceph-mon[29180]: pgmap v6626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:39:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11067.877686 secs 2024-04-26T21:39:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:48 smithi119 ceph-mon[32752]: pgmap v6627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:39:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:48 smithi167 ceph-mon[32751]: pgmap v6627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:39:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:48 smithi090 ceph-mon[29180]: pgmap v6627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:39:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:50 smithi119 ceph-mon[32752]: pgmap v6628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11072.145506 secs 2024-04-26T21:39:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:50 smithi167 ceph-mon[32751]: pgmap v6628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11072.145506 secs 2024-04-26T21:39:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:50 smithi090 ceph-mon[29180]: pgmap v6628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11072.145506 secs 2024-04-26T21:39:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11072.877762 secs 2024-04-26T21:39:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11072.877762 secs 2024-04-26T21:39:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11072.877762 secs 2024-04-26T21:39:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:52 smithi119 ceph-mon[32752]: pgmap v6629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:52 smithi167 ceph-mon[32751]: pgmap v6629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:52 smithi090 ceph-mon[29180]: pgmap v6629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:54 smithi167 ceph-mon[32751]: pgmap v6630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:54 smithi090 ceph-mon[29180]: pgmap v6630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:54 smithi119 ceph-mon[32752]: pgmap v6630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:39:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11077.145589 secs 2024-04-26T21:39:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11077.145589 secs 2024-04-26T21:39:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11077.145589 secs 2024-04-26T21:39:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:56 smithi167 ceph-mon[32751]: pgmap v6631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11077.877858 secs 2024-04-26T21:39:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:56 smithi090 ceph-mon[29180]: pgmap v6631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11077.877858 secs 2024-04-26T21:39:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:56 smithi119 ceph-mon[32752]: pgmap v6631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11077.877858 secs 2024-04-26T21:39:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:39:58 smithi167 ceph-mon[32751]: pgmap v6632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:39:58 smithi090 ceph-mon[29180]: pgmap v6632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:39:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:39:58 smithi119 ceph-mon[32752]: pgmap v6632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:00 smithi090 ceph-mon[29180]: pgmap v6633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11082.145675 secs 2024-04-26T21:40:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:40:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:00 smithi119 ceph-mon[32752]: pgmap v6633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11082.145675 secs 2024-04-26T21:40:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:40:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:00 smithi167 ceph-mon[32751]: pgmap v6633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11082.145675 secs 2024-04-26T21:40:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:40:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11082.877956 secs 2024-04-26T21:40:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11082.877956 secs 2024-04-26T21:40:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11082.877956 secs 2024-04-26T21:40:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:02 smithi090 ceph-mon[29180]: pgmap v6634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:02 smithi119 ceph-mon[32752]: pgmap v6634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:02 smithi167 ceph-mon[32751]: pgmap v6634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:04 smithi090 ceph-mon[29180]: pgmap v6635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:04 smithi119 ceph-mon[32752]: pgmap v6635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:04 smithi167 ceph-mon[32751]: pgmap v6635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:05 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 11087.145762 secs 2024-04-26T21:40:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:05 smithi090 ceph-mon[29180]: slow request 1441.554006 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:40:06.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:05 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 11087.145762 secs 2024-04-26T21:40:06.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:05 smithi167 ceph-mon[32751]: slow request 1441.554006 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:40:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:05 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 11087.145762 secs 2024-04-26T21:40:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:05 smithi119 ceph-mon[32752]: slow request 1441.554006 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:40:07.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:06 smithi090 ceph-mon[29180]: pgmap v6636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:07.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11087.878091 secs 2024-04-26T21:40:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:06 smithi119 ceph-mon[32752]: pgmap v6636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11087.878091 secs 2024-04-26T21:40:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:06 smithi167 ceph-mon[32751]: pgmap v6636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11087.878091 secs 2024-04-26T21:40:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:08 smithi090 ceph-mon[29180]: pgmap v6637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:08 smithi119 ceph-mon[32752]: pgmap v6637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:08 smithi167 ceph-mon[32751]: pgmap v6637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:10 smithi090 ceph-mon[29180]: pgmap v6638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11092.145829 secs 2024-04-26T21:40:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:10 smithi119 ceph-mon[32752]: pgmap v6638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11092.145829 secs 2024-04-26T21:40:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:10 smithi167 ceph-mon[32751]: pgmap v6638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11092.145829 secs 2024-04-26T21:40:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11092.878234 secs 2024-04-26T21:40:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11092.878234 secs 2024-04-26T21:40:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11092.878234 secs 2024-04-26T21:40:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:12 smithi090 ceph-mon[29180]: pgmap v6639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:13.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:12 smithi119 ceph-mon[32752]: pgmap v6639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:12 smithi167 ceph-mon[32751]: pgmap v6639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:14 smithi090 ceph-mon[29180]: pgmap v6640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:14 smithi119 ceph-mon[32752]: pgmap v6640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:14 smithi167 ceph-mon[32751]: pgmap v6640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11097.145892 secs 2024-04-26T21:40:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11097.145892 secs 2024-04-26T21:40:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11097.145892 secs 2024-04-26T21:40:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:16 smithi090 ceph-mon[29180]: pgmap v6641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11097.878395 secs 2024-04-26T21:40:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:16 smithi119 ceph-mon[32752]: pgmap v6641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11097.878395 secs 2024-04-26T21:40:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:16 smithi167 ceph-mon[32751]: pgmap v6641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11097.878395 secs 2024-04-26T21:40:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:18 smithi090 ceph-mon[29180]: pgmap v6642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:18 smithi119 ceph-mon[32752]: pgmap v6642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:18 smithi167 ceph-mon[32751]: pgmap v6642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:20 smithi090 ceph-mon[29180]: pgmap v6643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11102.145952 secs 2024-04-26T21:40:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:20 smithi119 ceph-mon[32752]: pgmap v6643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11102.145952 secs 2024-04-26T21:40:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:20 smithi167 ceph-mon[32751]: pgmap v6643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11102.145952 secs 2024-04-26T21:40:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11102.878486 secs 2024-04-26T21:40:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11102.878486 secs 2024-04-26T21:40:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11102.878486 secs 2024-04-26T21:40:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:22 smithi090 ceph-mon[29180]: pgmap v6644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:22 smithi119 ceph-mon[32752]: pgmap v6644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:22 smithi167 ceph-mon[32751]: pgmap v6644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:24 smithi119 ceph-mon[32752]: pgmap v6645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:24 smithi167 ceph-mon[32751]: pgmap v6645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:24 smithi090 ceph-mon[29180]: pgmap v6645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11107.146038 secs 2024-04-26T21:40:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11107.146038 secs 2024-04-26T21:40:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11107.146038 secs 2024-04-26T21:40:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:26 smithi119 ceph-mon[32752]: pgmap v6646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11107.878606 secs 2024-04-26T21:40:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:26 smithi167 ceph-mon[32751]: pgmap v6646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11107.878606 secs 2024-04-26T21:40:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:26 smithi090 ceph-mon[29180]: pgmap v6646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11107.878606 secs 2024-04-26T21:40:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:28 smithi119 ceph-mon[32752]: pgmap v6647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:28 smithi167 ceph-mon[32751]: pgmap v6647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:28 smithi090 ceph-mon[29180]: pgmap v6647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:30 smithi119 ceph-mon[32752]: pgmap v6648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11112.146120 secs 2024-04-26T21:40:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:30 smithi167 ceph-mon[32751]: pgmap v6648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11112.146120 secs 2024-04-26T21:40:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:30 smithi090 ceph-mon[29180]: pgmap v6648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11112.146120 secs 2024-04-26T21:40:32.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11112.878694 secs 2024-04-26T21:40:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11112.878694 secs 2024-04-26T21:40:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11112.878694 secs 2024-04-26T21:40:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:32 smithi119 ceph-mon[32752]: pgmap v6649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:32 smithi167 ceph-mon[32751]: pgmap v6649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:32 smithi090 ceph-mon[29180]: pgmap v6649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:34 smithi119 ceph-mon[32752]: pgmap v6650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:34 smithi167 ceph-mon[32751]: pgmap v6650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:34 smithi090 ceph-mon[29180]: pgmap v6650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:36.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11117.146189 secs 2024-04-26T21:40:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11117.146189 secs 2024-04-26T21:40:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11117.146189 secs 2024-04-26T21:40:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:36 smithi119 ceph-mon[32752]: pgmap v6651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11117.878837 secs 2024-04-26T21:40:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:36 smithi167 ceph-mon[32751]: pgmap v6651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11117.878837 secs 2024-04-26T21:40:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:36 smithi090 ceph-mon[29180]: pgmap v6651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11117.878837 secs 2024-04-26T21:40:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:40:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:40:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:40:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:38 smithi119 ceph-mon[32752]: pgmap v6652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:39.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:40:39.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:40:39.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:40:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:38 smithi167 ceph-mon[32751]: pgmap v6652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:40:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:40:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:40:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:38 smithi090 ceph-mon[29180]: pgmap v6652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:40:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:40:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:40:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:39 smithi119 ceph-mon[32752]: pgmap v6653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:40:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:39 smithi119 ceph-mon[32752]: pgmap v6654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:40:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:39 smithi167 ceph-mon[32751]: pgmap v6653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:40:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:39 smithi167 ceph-mon[32751]: pgmap v6654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:40:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:39 smithi090 ceph-mon[29180]: pgmap v6653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:40:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:39 smithi090 ceph-mon[29180]: pgmap v6654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11122.146276 secs 2024-04-26T21:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11122.146276 secs 2024-04-26T21:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11122.146276 secs 2024-04-26T21:40:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:41 smithi119 ceph-mon[32752]: pgmap v6655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11122.878902 secs 2024-04-26T21:40:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:41 smithi167 ceph-mon[32751]: pgmap v6655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11122.878902 secs 2024-04-26T21:40:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:41 smithi090 ceph-mon[29180]: pgmap v6655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11122.878902 secs 2024-04-26T21:40:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:43 smithi119 ceph-mon[32752]: pgmap v6656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:40:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:43 smithi167 ceph-mon[32751]: pgmap v6656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:40:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:43 smithi090 ceph-mon[29180]: pgmap v6656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:40:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:45 smithi119 ceph-mon[32752]: pgmap v6657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11127.146362 secs 2024-04-26T21:40:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:45 smithi167 ceph-mon[32751]: pgmap v6657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11127.146362 secs 2024-04-26T21:40:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:45 smithi090 ceph-mon[29180]: pgmap v6657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11127.146362 secs 2024-04-26T21:40:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11127.879014 secs 2024-04-26T21:40:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11127.879014 secs 2024-04-26T21:40:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11127.879014 secs 2024-04-26T21:40:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:47 smithi119 ceph-mon[32752]: pgmap v6658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:47 smithi167 ceph-mon[32751]: pgmap v6658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:47 smithi090 ceph-mon[29180]: pgmap v6658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:40:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:49 smithi119 ceph-mon[32752]: pgmap v6659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:40:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:49 smithi167 ceph-mon[32751]: pgmap v6659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:40:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:49 smithi090 ceph-mon[29180]: pgmap v6659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:40:51.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11132.146424 secs 2024-04-26T21:40:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11132.146424 secs 2024-04-26T21:40:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11132.146424 secs 2024-04-26T21:40:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:51 smithi119 ceph-mon[32752]: pgmap v6660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11132.879145 secs 2024-04-26T21:40:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:51 smithi167 ceph-mon[32751]: pgmap v6660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11132.879145 secs 2024-04-26T21:40:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:51 smithi090 ceph-mon[29180]: pgmap v6660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11132.879145 secs 2024-04-26T21:40:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:53 smithi090 ceph-mon[29180]: pgmap v6661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:53 smithi119 ceph-mon[32752]: pgmap v6661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:53 smithi167 ceph-mon[32751]: pgmap v6661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:55 smithi119 ceph-mon[32752]: pgmap v6662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:55 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11137.146506 secs 2024-04-26T21:40:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:55 smithi167 ceph-mon[32751]: pgmap v6662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:55 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11137.146506 secs 2024-04-26T21:40:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:55 smithi090 ceph-mon[29180]: pgmap v6662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:40:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:55 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11137.146506 secs 2024-04-26T21:40:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11137.879239 secs 2024-04-26T21:40:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:56 smithi119 ceph-mon[32752]: pgmap v6663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11137.879239 secs 2024-04-26T21:40:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:56 smithi167 ceph-mon[32751]: pgmap v6663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11137.879239 secs 2024-04-26T21:40:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:56 smithi090 ceph-mon[29180]: pgmap v6663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:40:59 smithi119 ceph-mon[32752]: pgmap v6664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:40:59 smithi167 ceph-mon[32751]: pgmap v6664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:40:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:40:59 smithi090 ceph-mon[29180]: pgmap v6664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:00 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11142.146591 secs 2024-04-26T21:41:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:00 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11142.146591 secs 2024-04-26T21:41:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:00 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11142.146591 secs 2024-04-26T21:41:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:01 smithi119 ceph-mon[32752]: pgmap v6665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11142.879312 secs 2024-04-26T21:41:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:01 smithi167 ceph-mon[32751]: pgmap v6665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11142.879312 secs 2024-04-26T21:41:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:01 smithi090 ceph-mon[29180]: pgmap v6665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11142.879312 secs 2024-04-26T21:41:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:03 smithi119 ceph-mon[32752]: pgmap v6666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:03 smithi167 ceph-mon[32751]: pgmap v6666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:03 smithi090 ceph-mon[29180]: pgmap v6666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:05 smithi119 ceph-mon[32752]: pgmap v6667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:05 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11147.146671 secs 2024-04-26T21:41:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:05 smithi167 ceph-mon[32751]: pgmap v6667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:05 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11147.146671 secs 2024-04-26T21:41:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:05 smithi090 ceph-mon[29180]: pgmap v6667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:05 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11147.146671 secs 2024-04-26T21:41:06.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11147.879464 secs 2024-04-26T21:41:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11147.879464 secs 2024-04-26T21:41:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11147.879464 secs 2024-04-26T21:41:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:07 smithi119 ceph-mon[32752]: pgmap v6668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:07 smithi167 ceph-mon[32751]: pgmap v6668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:07 smithi090 ceph-mon[29180]: pgmap v6668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:09 smithi119 ceph-mon[32752]: pgmap v6669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:09 smithi167 ceph-mon[32751]: pgmap v6669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:09 smithi090 ceph-mon[29180]: pgmap v6669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:10 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11152.146730 secs 2024-04-26T21:41:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:10 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11152.146730 secs 2024-04-26T21:41:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:10 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11152.146730 secs 2024-04-26T21:41:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:11 smithi119 ceph-mon[32752]: pgmap v6670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11152.879526 secs 2024-04-26T21:41:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:11 smithi167 ceph-mon[32751]: pgmap v6670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11152.879526 secs 2024-04-26T21:41:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:11 smithi090 ceph-mon[29180]: pgmap v6670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11152.879526 secs 2024-04-26T21:41:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:13 smithi119 ceph-mon[32752]: pgmap v6671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:13 smithi167 ceph-mon[32751]: pgmap v6671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:13 smithi090 ceph-mon[29180]: pgmap v6671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:15 smithi119 ceph-mon[32752]: pgmap v6672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:15 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11157.146791 secs 2024-04-26T21:41:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:15 smithi167 ceph-mon[32751]: pgmap v6672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:15 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11157.146791 secs 2024-04-26T21:41:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:15 smithi090 ceph-mon[29180]: pgmap v6672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:15 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11157.146791 secs 2024-04-26T21:41:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11157.879618 secs 2024-04-26T21:41:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11157.879618 secs 2024-04-26T21:41:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11157.879618 secs 2024-04-26T21:41:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:17 smithi119 ceph-mon[32752]: pgmap v6673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:17 smithi167 ceph-mon[32751]: pgmap v6673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:17 smithi090 ceph-mon[29180]: pgmap v6673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:19 smithi119 ceph-mon[32752]: pgmap v6674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:19 smithi167 ceph-mon[32751]: pgmap v6674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:19 smithi090 ceph-mon[29180]: pgmap v6674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:20 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11162.146850 secs 2024-04-26T21:41:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:20 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11162.146850 secs 2024-04-26T21:41:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:20 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11162.146850 secs 2024-04-26T21:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:21 smithi119 ceph-mon[32752]: pgmap v6675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11162.879728 secs 2024-04-26T21:41:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:21 smithi167 ceph-mon[32751]: pgmap v6675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11162.879728 secs 2024-04-26T21:41:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:21 smithi090 ceph-mon[29180]: pgmap v6675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11162.879728 secs 2024-04-26T21:41:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:23 smithi119 ceph-mon[32752]: pgmap v6676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:23 smithi167 ceph-mon[32751]: pgmap v6676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:23 smithi090 ceph-mon[29180]: pgmap v6676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:25 smithi119 ceph-mon[32752]: pgmap v6677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11167.146932 secs 2024-04-26T21:41:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:25 smithi167 ceph-mon[32751]: pgmap v6677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11167.146932 secs 2024-04-26T21:41:25.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:25 smithi090 ceph-mon[29180]: pgmap v6677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:25.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11167.146932 secs 2024-04-26T21:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11167.879836 secs 2024-04-26T21:41:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11167.879836 secs 2024-04-26T21:41:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11167.879836 secs 2024-04-26T21:41:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:27 smithi119 ceph-mon[32752]: pgmap v6678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:27 smithi167 ceph-mon[32751]: pgmap v6678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:27 smithi090 ceph-mon[29180]: pgmap v6678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:29 smithi119 ceph-mon[32752]: pgmap v6679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:29 smithi167 ceph-mon[32751]: pgmap v6679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:29 smithi090 ceph-mon[29180]: pgmap v6679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11172.146994 secs 2024-04-26T21:41:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11172.146994 secs 2024-04-26T21:41:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11172.146994 secs 2024-04-26T21:41:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:31 smithi119 ceph-mon[32752]: pgmap v6680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11172.879910 secs 2024-04-26T21:41:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:31 smithi167 ceph-mon[32751]: pgmap v6680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11172.879910 secs 2024-04-26T21:41:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:31 smithi090 ceph-mon[29180]: pgmap v6680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11172.879910 secs 2024-04-26T21:41:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:33 smithi119 ceph-mon[32752]: pgmap v6681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:33 smithi167 ceph-mon[32751]: pgmap v6681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:33 smithi090 ceph-mon[29180]: pgmap v6681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:35 smithi119 ceph-mon[32752]: pgmap v6682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11177.147079 secs 2024-04-26T21:41:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:35 smithi167 ceph-mon[32751]: pgmap v6682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11177.147079 secs 2024-04-26T21:41:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:35 smithi090 ceph-mon[29180]: pgmap v6682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11177.147079 secs 2024-04-26T21:41:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11177.880043 secs 2024-04-26T21:41:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11177.880043 secs 2024-04-26T21:41:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11177.880043 secs 2024-04-26T21:41:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:37 smithi119 ceph-mon[32752]: pgmap v6683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:37 smithi167 ceph-mon[32751]: pgmap v6683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:37 smithi090 ceph-mon[29180]: pgmap v6683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:41:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:41:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:41:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:39 smithi119 ceph-mon[32752]: pgmap v6684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:39.559 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:39 smithi090 ceph-mon[29180]: pgmap v6684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:39 smithi167 ceph-mon[32751]: pgmap v6684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 11182.147164 secs 2024-04-26T21:41:40.562 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 11182.147164 secs 2024-04-26T21:41:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 11182.147164 secs 2024-04-26T21:41:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:41 smithi119 ceph-mon[32752]: pgmap v6685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11182.880131 secs 2024-04-26T21:41:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:41 smithi167 ceph-mon[32751]: pgmap v6685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11182.880131 secs 2024-04-26T21:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:41 smithi090 ceph-mon[29180]: pgmap v6685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11182.880131 secs 2024-04-26T21:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: pgmap v6686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:41:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:41:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: pgmap v6687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T21:41:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: pgmap v6688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T21:41:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: pgmap v6686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:41:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:41:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: pgmap v6687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T21:41:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: pgmap v6688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T21:41:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: pgmap v6686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: pgmap v6687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: pgmap v6688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-26T21:41:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:41:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:45 smithi119 ceph-mon[32752]: pgmap v6689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:45 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11187.147222 secs 2024-04-26T21:41:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:45 smithi167 ceph-mon[32751]: pgmap v6689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:45 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11187.147222 secs 2024-04-26T21:41:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:45 smithi090 ceph-mon[29180]: pgmap v6689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:45 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11187.147222 secs 2024-04-26T21:41:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11187.880250 secs 2024-04-26T21:41:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:46 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:41:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:46 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:41:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11187.880250 secs 2024-04-26T21:41:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:46 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:41:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:46 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:41:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11187.880250 secs 2024-04-26T21:41:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:46 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:41:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:46 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:41:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:47 smithi119 ceph-mon[32752]: pgmap v6690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:47 smithi167 ceph-mon[32751]: pgmap v6690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:47 smithi090 ceph-mon[29180]: pgmap v6690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:49 smithi119 ceph-mon[32752]: pgmap v6691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:49 smithi167 ceph-mon[32751]: pgmap v6691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:49 smithi090 ceph-mon[29180]: pgmap v6691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11192.147282 secs 2024-04-26T21:41:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11192.147282 secs 2024-04-26T21:41:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11192.147282 secs 2024-04-26T21:41:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11192.880367 secs 2024-04-26T21:41:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:51 smithi119 ceph-mon[32752]: pgmap v6692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11192.880367 secs 2024-04-26T21:41:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:51 smithi167 ceph-mon[32751]: pgmap v6692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11192.880367 secs 2024-04-26T21:41:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:51 smithi090 ceph-mon[29180]: pgmap v6692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:53 smithi119 ceph-mon[32752]: pgmap v6693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:41:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:53 smithi167 ceph-mon[32751]: pgmap v6693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:41:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:53 smithi090 ceph-mon[29180]: pgmap v6693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:41:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:55 smithi119 ceph-mon[32752]: pgmap v6694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:55 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11197.147373 secs 2024-04-26T21:41:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:55 smithi167 ceph-mon[32751]: pgmap v6694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:55 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11197.147373 secs 2024-04-26T21:41:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:55 smithi090 ceph-mon[29180]: pgmap v6694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:41:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:55 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11197.147373 secs 2024-04-26T21:41:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11197.880459 secs 2024-04-26T21:41:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11197.880459 secs 2024-04-26T21:41:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11197.880459 secs 2024-04-26T21:41:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:57 smithi119 ceph-mon[32752]: pgmap v6695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:57 smithi167 ceph-mon[32751]: pgmap v6695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:41:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:57 smithi090 ceph-mon[29180]: pgmap v6695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:41:59 smithi119 ceph-mon[32752]: pgmap v6696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:41:59 smithi167 ceph-mon[32751]: pgmap v6696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:41:59 smithi090 ceph-mon[29180]: pgmap v6696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11202.147457 secs 2024-04-26T21:42:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11202.147457 secs 2024-04-26T21:42:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11202.147457 secs 2024-04-26T21:42:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11202.880576 secs 2024-04-26T21:42:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:01 smithi167 ceph-mon[32751]: pgmap v6697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11202.880576 secs 2024-04-26T21:42:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:01 smithi090 ceph-mon[29180]: pgmap v6697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11202.880576 secs 2024-04-26T21:42:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:01 smithi119 ceph-mon[32752]: pgmap v6697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:03 smithi167 ceph-mon[32751]: pgmap v6698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:03 smithi090 ceph-mon[29180]: pgmap v6698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:03 smithi119 ceph-mon[32752]: pgmap v6698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:05 smithi167 ceph-mon[32751]: pgmap v6699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:05 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11207.147540 secs 2024-04-26T21:42:06.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:05 smithi090 ceph-mon[29180]: pgmap v6699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:06.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:05 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11207.147540 secs 2024-04-26T21:42:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:05 smithi119 ceph-mon[32752]: pgmap v6699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:05 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11207.147540 secs 2024-04-26T21:42:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11207.880642 secs 2024-04-26T21:42:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11207.880642 secs 2024-04-26T21:42:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11207.880642 secs 2024-04-26T21:42:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:07 smithi090 ceph-mon[29180]: pgmap v6700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:07 smithi119 ceph-mon[32752]: pgmap v6700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:07 smithi167 ceph-mon[32751]: pgmap v6700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:09 smithi090 ceph-mon[29180]: pgmap v6701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:09 smithi119 ceph-mon[32752]: pgmap v6701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:09 smithi167 ceph-mon[32751]: pgmap v6701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:10 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11212.147601 secs 2024-04-26T21:42:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:10 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11212.147601 secs 2024-04-26T21:42:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:10 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11212.147601 secs 2024-04-26T21:42:12.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11212.880758 secs 2024-04-26T21:42:12.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:11 smithi090 ceph-mon[29180]: pgmap v6702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11212.880758 secs 2024-04-26T21:42:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:11 smithi119 ceph-mon[32752]: pgmap v6702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11212.880758 secs 2024-04-26T21:42:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:11 smithi167 ceph-mon[32751]: pgmap v6702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:13 smithi090 ceph-mon[29180]: pgmap v6703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:13 smithi119 ceph-mon[32752]: pgmap v6703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:13 smithi167 ceph-mon[32751]: pgmap v6703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:15 smithi090 ceph-mon[29180]: pgmap v6704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:15 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11217.147688 secs 2024-04-26T21:42:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:15 smithi119 ceph-mon[32752]: pgmap v6704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:15 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11217.147688 secs 2024-04-26T21:42:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:15 smithi167 ceph-mon[32751]: pgmap v6704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:15 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11217.147688 secs 2024-04-26T21:42:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11217.880826 secs 2024-04-26T21:42:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11217.880826 secs 2024-04-26T21:42:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11217.880826 secs 2024-04-26T21:42:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:17 smithi090 ceph-mon[29180]: pgmap v6705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:17 smithi119 ceph-mon[32752]: pgmap v6705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:17 smithi167 ceph-mon[32751]: pgmap v6705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:19 smithi090 ceph-mon[29180]: pgmap v6706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:19 smithi119 ceph-mon[32752]: pgmap v6706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:19 smithi167 ceph-mon[32751]: pgmap v6706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:20 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11222.147783 secs 2024-04-26T21:42:21.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:20 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11222.147783 secs 2024-04-26T21:42:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:20 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11222.147783 secs 2024-04-26T21:42:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11222.880961 secs 2024-04-26T21:42:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:21 smithi090 ceph-mon[29180]: pgmap v6707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11222.880961 secs 2024-04-26T21:42:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:21 smithi119 ceph-mon[32752]: pgmap v6707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11222.880961 secs 2024-04-26T21:42:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:21 smithi167 ceph-mon[32751]: pgmap v6707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:23 smithi090 ceph-mon[29180]: pgmap v6708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:23 smithi119 ceph-mon[32752]: pgmap v6708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:23 smithi167 ceph-mon[32751]: pgmap v6708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:25 smithi090 ceph-mon[29180]: pgmap v6709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:25 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11227.147815 secs 2024-04-26T21:42:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:25 smithi119 ceph-mon[32752]: pgmap v6709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:25 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11227.147815 secs 2024-04-26T21:42:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:25 smithi167 ceph-mon[32751]: pgmap v6709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:25 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11227.147815 secs 2024-04-26T21:42:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11227.881025 secs 2024-04-26T21:42:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11227.881025 secs 2024-04-26T21:42:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11227.881025 secs 2024-04-26T21:42:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:27 smithi090 ceph-mon[29180]: pgmap v6710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:27 smithi119 ceph-mon[32752]: pgmap v6710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:27 smithi167 ceph-mon[32751]: pgmap v6710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:29 smithi119 ceph-mon[32752]: pgmap v6711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:29 smithi167 ceph-mon[32751]: pgmap v6711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:29 smithi090 ceph-mon[29180]: pgmap v6711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11232.147900 secs 2024-04-26T21:42:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11232.147900 secs 2024-04-26T21:42:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11232.147900 secs 2024-04-26T21:42:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11232.881151 secs 2024-04-26T21:42:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:31 smithi119 ceph-mon[32752]: pgmap v6712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11232.881151 secs 2024-04-26T21:42:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:31 smithi167 ceph-mon[32751]: pgmap v6712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11232.881151 secs 2024-04-26T21:42:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:31 smithi090 ceph-mon[29180]: pgmap v6712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:33 smithi090 ceph-mon[29180]: pgmap v6713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:34.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:42:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:42:33.822+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:42:34.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:42:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:42:33.822+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:42:34.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:42:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:42:33.822+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:42:34.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:42:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:42:33.822+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:42:34.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:42:33 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:42:33.822+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:42:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:33 smithi119 ceph-mon[32752]: pgmap v6713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:33 smithi167 ceph-mon[32751]: pgmap v6713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:35 smithi119 ceph-mon[32752]: pgmap v6714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:35 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11237.147986 secs 2024-04-26T21:42:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:35 smithi167 ceph-mon[32751]: pgmap v6714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:35 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11237.147986 secs 2024-04-26T21:42:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:35 smithi090 ceph-mon[29180]: pgmap v6714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:35 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11237.147986 secs 2024-04-26T21:42:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11237.881255 secs 2024-04-26T21:42:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11237.881255 secs 2024-04-26T21:42:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11237.881255 secs 2024-04-26T21:42:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:37 smithi119 ceph-mon[32752]: pgmap v6715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:37 smithi167 ceph-mon[32751]: pgmap v6715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:37 smithi090 ceph-mon[29180]: pgmap v6715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:42:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:39 smithi119 ceph-mon[32752]: pgmap v6716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:39 smithi167 ceph-mon[32751]: pgmap v6716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:39 smithi090 ceph-mon[29180]: pgmap v6716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11242.148047 secs 2024-04-26T21:42:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11242.148047 secs 2024-04-26T21:42:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11242.148047 secs 2024-04-26T21:42:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11242.881395 secs 2024-04-26T21:42:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:41 smithi119 ceph-mon[32752]: pgmap v6717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11242.881395 secs 2024-04-26T21:42:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:41 smithi167 ceph-mon[32751]: pgmap v6717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11242.881395 secs 2024-04-26T21:42:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:41 smithi090 ceph-mon[29180]: pgmap v6717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:42:44.169 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:43 smithi167 ceph-mon[32751]: pgmap v6718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:43 smithi119 ceph-mon[32752]: pgmap v6718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:43 smithi090 ceph-mon[29180]: pgmap v6718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:46.268 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:45 smithi167 ceph-mon[32751]: pgmap v6719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:46.268 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:45 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11247.148106 secs 2024-04-26T21:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:45 smithi119 ceph-mon[32752]: pgmap v6719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:45 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11247.148106 secs 2024-04-26T21:42:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:45 smithi090 ceph-mon[29180]: pgmap v6719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:45 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11247.148106 secs 2024-04-26T21:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11247.881520 secs 2024-04-26T21:42:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11247.881520 secs 2024-04-26T21:42:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11247.881520 secs 2024-04-26T21:42:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:47 smithi119 ceph-mon[32752]: pgmap v6720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:42:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:47 smithi167 ceph-mon[32751]: pgmap v6720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:42:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:47 smithi090 ceph-mon[29180]: pgmap v6720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T21:42:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:42:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:42:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:42:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:48 smithi119 ceph-mon[32752]: pgmap v6721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 196 B/s wr, 15 op/s 2024-04-26T21:42:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:48 smithi119 ceph-mon[32752]: pgmap v6722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T21:42:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:48 smithi167 ceph-mon[32751]: pgmap v6721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 196 B/s wr, 15 op/s 2024-04-26T21:42:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:48 smithi167 ceph-mon[32751]: pgmap v6722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T21:42:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:48 smithi090 ceph-mon[29180]: pgmap v6721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 196 B/s wr, 15 op/s 2024-04-26T21:42:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:48 smithi090 ceph-mon[29180]: pgmap v6722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T21:42:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:50 smithi119 ceph-mon[32752]: pgmap v6723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11252.148168 secs 2024-04-26T21:42:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:50 smithi167 ceph-mon[32751]: pgmap v6723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11252.148168 secs 2024-04-26T21:42:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:50 smithi090 ceph-mon[29180]: pgmap v6723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11252.148168 secs 2024-04-26T21:42:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11252.881644 secs 2024-04-26T21:42:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11252.881644 secs 2024-04-26T21:42:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11252.881644 secs 2024-04-26T21:42:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:52 smithi119 ceph-mon[32752]: pgmap v6724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:52 smithi167 ceph-mon[32751]: pgmap v6724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:52 smithi090 ceph-mon[29180]: pgmap v6724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:54 smithi119 ceph-mon[32752]: pgmap v6725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T21:42:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:54 smithi167 ceph-mon[32751]: pgmap v6725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T21:42:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:54 smithi090 ceph-mon[29180]: pgmap v6725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-26T21:42:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:55 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11257.148251 secs 2024-04-26T21:42:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:55 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11257.148251 secs 2024-04-26T21:42:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:55 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11257.148251 secs 2024-04-26T21:42:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:56 smithi119 ceph-mon[32752]: pgmap v6726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11257.881716 secs 2024-04-26T21:42:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:56 smithi167 ceph-mon[32751]: pgmap v6726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11257.881716 secs 2024-04-26T21:42:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:56 smithi090 ceph-mon[29180]: pgmap v6726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:42:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11257.881716 secs 2024-04-26T21:42:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:42:58 smithi119 ceph-mon[32752]: pgmap v6727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:42:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:42:58 smithi167 ceph-mon[32751]: pgmap v6727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:42:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:42:58 smithi090 ceph-mon[29180]: pgmap v6727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:43:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:00 smithi119 ceph-mon[32752]: pgmap v6728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11262.148333 secs 2024-04-26T21:43:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:00 smithi167 ceph-mon[32751]: pgmap v6728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11262.148333 secs 2024-04-26T21:43:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:00 smithi090 ceph-mon[29180]: pgmap v6728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11262.148333 secs 2024-04-26T21:43:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11262.881845 secs 2024-04-26T21:43:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11262.881845 secs 2024-04-26T21:43:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11262.881845 secs 2024-04-26T21:43:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:02 smithi119 ceph-mon[32752]: pgmap v6729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:02 smithi167 ceph-mon[32751]: pgmap v6729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:02 smithi090 ceph-mon[29180]: pgmap v6729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:04 smithi119 ceph-mon[32752]: pgmap v6730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:04 smithi167 ceph-mon[32751]: pgmap v6730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:04 smithi090 ceph-mon[29180]: pgmap v6730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:05 smithi119 ceph-mon[32752]: 13 slow requests, 1 included below; oldest blocked for > 11267.148415 secs 2024-04-26T21:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:05 smithi119 ceph-mon[32752]: slow request 2881.910358 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:43:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:05 smithi167 ceph-mon[32751]: 13 slow requests, 1 included below; oldest blocked for > 11267.148415 secs 2024-04-26T21:43:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:05 smithi167 ceph-mon[32751]: slow request 2881.910358 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:43:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:05 smithi090 ceph-mon[29180]: 13 slow requests, 1 included below; oldest blocked for > 11267.148415 secs 2024-04-26T21:43:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:05 smithi090 ceph-mon[29180]: slow request 2881.910358 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:43:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:06 smithi119 ceph-mon[32752]: pgmap v6731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11267.881944 secs 2024-04-26T21:43:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:06 smithi167 ceph-mon[32751]: pgmap v6731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11267.881944 secs 2024-04-26T21:43:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:06 smithi090 ceph-mon[29180]: pgmap v6731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11267.881944 secs 2024-04-26T21:43:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:08 smithi119 ceph-mon[32752]: pgmap v6732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:08 smithi167 ceph-mon[32751]: pgmap v6732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:08 smithi090 ceph-mon[29180]: pgmap v6732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:10 smithi119 ceph-mon[32752]: pgmap v6733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:10 smithi167 ceph-mon[32751]: pgmap v6733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:10 smithi090 ceph-mon[29180]: pgmap v6733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11272.148499 secs 2024-04-26T21:43:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11272.882068 secs 2024-04-26T21:43:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11272.148499 secs 2024-04-26T21:43:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11272.882068 secs 2024-04-26T21:43:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11272.148499 secs 2024-04-26T21:43:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11272.882068 secs 2024-04-26T21:43:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:12 smithi119 ceph-mon[32752]: pgmap v6734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:12 smithi167 ceph-mon[32751]: pgmap v6734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:12 smithi090 ceph-mon[29180]: pgmap v6734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:14 smithi119 ceph-mon[32752]: pgmap v6735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:14 smithi167 ceph-mon[32751]: pgmap v6735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:14 smithi090 ceph-mon[29180]: pgmap v6735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:43:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:15 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11277.148584 secs 2024-04-26T21:43:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:15 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11277.148584 secs 2024-04-26T21:43:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:15 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11277.148584 secs 2024-04-26T21:43:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:16 smithi119 ceph-mon[32752]: pgmap v6736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11277.882191 secs 2024-04-26T21:43:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:16 smithi167 ceph-mon[32751]: pgmap v6736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11277.882191 secs 2024-04-26T21:43:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:16 smithi090 ceph-mon[29180]: pgmap v6736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:43:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11277.882191 secs 2024-04-26T21:43:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:18 smithi119 ceph-mon[32752]: pgmap v6737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:43:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:18 smithi167 ceph-mon[32751]: pgmap v6737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:43:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:18 smithi090 ceph-mon[29180]: pgmap v6737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:43:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:20 smithi119 ceph-mon[32752]: pgmap v6738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:20 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11282.148675 secs 2024-04-26T21:43:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:20 smithi167 ceph-mon[32751]: pgmap v6738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:20 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11282.148675 secs 2024-04-26T21:43:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:20 smithi090 ceph-mon[29180]: pgmap v6738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:20 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11282.148675 secs 2024-04-26T21:43:21.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11282.882314 secs 2024-04-26T21:43:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11282.882314 secs 2024-04-26T21:43:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11282.882314 secs 2024-04-26T21:43:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:22 smithi119 ceph-mon[32752]: pgmap v6739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:22 smithi167 ceph-mon[32751]: pgmap v6739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:22 smithi090 ceph-mon[29180]: pgmap v6739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:24 smithi119 ceph-mon[32752]: pgmap v6740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:24.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:24 smithi167 ceph-mon[32751]: pgmap v6740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:24 smithi090 ceph-mon[29180]: pgmap v6740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:25 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11287.148761 secs 2024-04-26T21:43:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:25 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11287.148761 secs 2024-04-26T21:43:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:25 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11287.148761 secs 2024-04-26T21:43:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:26 smithi119 ceph-mon[32752]: pgmap v6741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11287.882467 secs 2024-04-26T21:43:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:26 smithi167 ceph-mon[32751]: pgmap v6741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11287.882467 secs 2024-04-26T21:43:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:26 smithi090 ceph-mon[29180]: pgmap v6741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11287.882467 secs 2024-04-26T21:43:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:28 smithi119 ceph-mon[32752]: pgmap v6742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:28 smithi167 ceph-mon[32751]: pgmap v6742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:28 smithi090 ceph-mon[29180]: pgmap v6742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:30 smithi119 ceph-mon[32752]: pgmap v6743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11292.148849 secs 2024-04-26T21:43:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:30 smithi167 ceph-mon[32751]: pgmap v6743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11292.148849 secs 2024-04-26T21:43:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:30 smithi090 ceph-mon[29180]: pgmap v6743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11292.148849 secs 2024-04-26T21:43:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11292.882548 secs 2024-04-26T21:43:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11292.882548 secs 2024-04-26T21:43:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11292.882548 secs 2024-04-26T21:43:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:32 smithi119 ceph-mon[32752]: pgmap v6744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:32 smithi167 ceph-mon[32751]: pgmap v6744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:32 smithi090 ceph-mon[29180]: pgmap v6744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:34 smithi119 ceph-mon[32752]: pgmap v6745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:34 smithi167 ceph-mon[32751]: pgmap v6745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:34 smithi090 ceph-mon[29180]: pgmap v6745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:35.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:35 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11297.148911 secs 2024-04-26T21:43:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:35 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11297.148911 secs 2024-04-26T21:43:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:35 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11297.148911 secs 2024-04-26T21:43:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:36 smithi119 ceph-mon[32752]: pgmap v6746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11297.882620 secs 2024-04-26T21:43:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:36 smithi167 ceph-mon[32751]: pgmap v6746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11297.882620 secs 2024-04-26T21:43:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:36 smithi090 ceph-mon[29180]: pgmap v6746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11297.882620 secs 2024-04-26T21:43:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:38 smithi119 ceph-mon[32752]: pgmap v6747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:38 smithi167 ceph-mon[32751]: pgmap v6747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:38 smithi090 ceph-mon[29180]: pgmap v6747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:40 smithi119 ceph-mon[32752]: pgmap v6748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11302.148973 secs 2024-04-26T21:43:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:40 smithi167 ceph-mon[32751]: pgmap v6748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11302.148973 secs 2024-04-26T21:43:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:40 smithi090 ceph-mon[29180]: pgmap v6748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11302.148973 secs 2024-04-26T21:43:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11302.882698 secs 2024-04-26T21:43:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11302.882698 secs 2024-04-26T21:43:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11302.882698 secs 2024-04-26T21:43:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:42 smithi119 ceph-mon[32752]: pgmap v6749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:42 smithi167 ceph-mon[32751]: pgmap v6749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:42 smithi090 ceph-mon[29180]: pgmap v6749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:44 smithi119 ceph-mon[32752]: pgmap v6750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:44 smithi167 ceph-mon[32751]: pgmap v6750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:44 smithi090 ceph-mon[29180]: pgmap v6750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:45 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11307.149058 secs 2024-04-26T21:43:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:45 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11307.149058 secs 2024-04-26T21:43:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:45 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11307.149058 secs 2024-04-26T21:43:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:46 smithi119 ceph-mon[32752]: pgmap v6751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11307.882775 secs 2024-04-26T21:43:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:46 smithi167 ceph-mon[32751]: pgmap v6751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11307.882775 secs 2024-04-26T21:43:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:46 smithi090 ceph-mon[29180]: pgmap v6751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:43:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11307.882775 secs 2024-04-26T21:43:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:48 smithi119 ceph-mon[32752]: pgmap v6752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:43:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:48 smithi167 ceph-mon[32751]: pgmap v6752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:43:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:48 smithi090 ceph-mon[29180]: pgmap v6752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:43:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:43:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:43:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:43:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:43:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:49 smithi119 ceph-mon[32752]: pgmap v6753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-26T21:43:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:49 smithi119 ceph-mon[32752]: pgmap v6754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-26T21:43:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:49 smithi167 ceph-mon[32751]: pgmap v6753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-26T21:43:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:49 smithi167 ceph-mon[32751]: pgmap v6754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-26T21:43:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:49 smithi090 ceph-mon[29180]: pgmap v6753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-26T21:43:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:49 smithi090 ceph-mon[29180]: pgmap v6754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-26T21:43:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11312.149121 secs 2024-04-26T21:43:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11312.149121 secs 2024-04-26T21:43:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11312.149121 secs 2024-04-26T21:43:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:51 smithi119 ceph-mon[32752]: pgmap v6755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11312.882849 secs 2024-04-26T21:43:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:51 smithi167 ceph-mon[32751]: pgmap v6755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11312.882849 secs 2024-04-26T21:43:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:51 smithi090 ceph-mon[29180]: pgmap v6755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11312.882849 secs 2024-04-26T21:43:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:53 smithi119 ceph-mon[32752]: pgmap v6756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:53 smithi167 ceph-mon[32751]: pgmap v6756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:53 smithi090 ceph-mon[29180]: pgmap v6756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:55 smithi119 ceph-mon[32752]: pgmap v6757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:55 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11317.149183 secs 2024-04-26T21:43:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:55 smithi167 ceph-mon[32751]: pgmap v6757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:55 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11317.149183 secs 2024-04-26T21:43:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:55 smithi090 ceph-mon[29180]: pgmap v6757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:55 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11317.149183 secs 2024-04-26T21:43:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11317.882929 secs 2024-04-26T21:43:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11317.882929 secs 2024-04-26T21:43:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11317.882929 secs 2024-04-26T21:43:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:57 smithi119 ceph-mon[32752]: pgmap v6758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:57 smithi167 ceph-mon[32751]: pgmap v6758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:57 smithi090 ceph-mon[29180]: pgmap v6758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:43:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:43:59 smithi119 ceph-mon[32752]: pgmap v6759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:43:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:43:59 smithi167 ceph-mon[32751]: pgmap v6759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:43:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:43:59 smithi090 ceph-mon[29180]: pgmap v6759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:44:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11322.149244 secs 2024-04-26T21:44:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11322.149244 secs 2024-04-26T21:44:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11322.149244 secs 2024-04-26T21:44:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:01 smithi119 ceph-mon[32752]: pgmap v6760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11322.883009 secs 2024-04-26T21:44:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:01 smithi167 ceph-mon[32751]: pgmap v6760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11322.883009 secs 2024-04-26T21:44:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:01 smithi090 ceph-mon[29180]: pgmap v6760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11322.883009 secs 2024-04-26T21:44:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:03 smithi167 ceph-mon[32751]: pgmap v6761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:03 smithi090 ceph-mon[29180]: pgmap v6761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:03 smithi119 ceph-mon[32752]: pgmap v6761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:05 smithi167 ceph-mon[32751]: pgmap v6762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:05 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11327.149329 secs 2024-04-26T21:44:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:05 smithi090 ceph-mon[29180]: pgmap v6762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:05 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11327.149329 secs 2024-04-26T21:44:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:05 smithi119 ceph-mon[32752]: pgmap v6762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:05 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11327.149329 secs 2024-04-26T21:44:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11327.883142 secs 2024-04-26T21:44:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11327.883142 secs 2024-04-26T21:44:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11327.883142 secs 2024-04-26T21:44:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:07 smithi167 ceph-mon[32751]: pgmap v6763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:07 smithi090 ceph-mon[29180]: pgmap v6763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:07 smithi119 ceph-mon[32752]: pgmap v6763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:09 smithi167 ceph-mon[32751]: pgmap v6764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:09 smithi090 ceph-mon[29180]: pgmap v6764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:09 smithi119 ceph-mon[32752]: pgmap v6764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:10 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11332.149414 secs 2024-04-26T21:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:10 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11332.149414 secs 2024-04-26T21:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:10 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11332.149414 secs 2024-04-26T21:44:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:11 smithi090 ceph-mon[29180]: pgmap v6765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11332.883281 secs 2024-04-26T21:44:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:11 smithi119 ceph-mon[32752]: pgmap v6765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11332.883281 secs 2024-04-26T21:44:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:11 smithi167 ceph-mon[32751]: pgmap v6765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11332.883281 secs 2024-04-26T21:44:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:13 smithi090 ceph-mon[29180]: pgmap v6766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:13 smithi119 ceph-mon[32752]: pgmap v6766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:13 smithi167 ceph-mon[32751]: pgmap v6766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:15 smithi090 ceph-mon[29180]: pgmap v6767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:15 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11337.149497 secs 2024-04-26T21:44:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:15 smithi119 ceph-mon[32752]: pgmap v6767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:15 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11337.149497 secs 2024-04-26T21:44:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:15 smithi167 ceph-mon[32751]: pgmap v6767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:15 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11337.149497 secs 2024-04-26T21:44:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11337.883453 secs 2024-04-26T21:44:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11337.883453 secs 2024-04-26T21:44:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11337.883453 secs 2024-04-26T21:44:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:17 smithi090 ceph-mon[29180]: pgmap v6768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:17 smithi119 ceph-mon[32752]: pgmap v6768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:17 smithi167 ceph-mon[32751]: pgmap v6768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:19 smithi090 ceph-mon[29180]: pgmap v6769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:19 smithi119 ceph-mon[32752]: pgmap v6769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:19 smithi167 ceph-mon[32751]: pgmap v6769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:20 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11342.149581 secs 2024-04-26T21:44:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:20 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11342.149581 secs 2024-04-26T21:44:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:20 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11342.149581 secs 2024-04-26T21:44:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:21 smithi090 ceph-mon[29180]: pgmap v6770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11342.883535 secs 2024-04-26T21:44:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:21 smithi119 ceph-mon[32752]: pgmap v6770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11342.883535 secs 2024-04-26T21:44:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:21 smithi167 ceph-mon[32751]: pgmap v6770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11342.883535 secs 2024-04-26T21:44:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:23 smithi090 ceph-mon[29180]: pgmap v6771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:23 smithi119 ceph-mon[32752]: pgmap v6771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:23 smithi167 ceph-mon[32751]: pgmap v6771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:25 smithi090 ceph-mon[29180]: pgmap v6772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:25 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11347.149665 secs 2024-04-26T21:44:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:25 smithi119 ceph-mon[32752]: pgmap v6772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:25 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11347.149665 secs 2024-04-26T21:44:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:25 smithi167 ceph-mon[32751]: pgmap v6772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:25 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11347.149665 secs 2024-04-26T21:44:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11347.883616 secs 2024-04-26T21:44:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11347.883616 secs 2024-04-26T21:44:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11347.883616 secs 2024-04-26T21:44:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:27 smithi090 ceph-mon[29180]: pgmap v6773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:27 smithi119 ceph-mon[32752]: pgmap v6773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:27 smithi167 ceph-mon[32751]: pgmap v6773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:29 smithi090 ceph-mon[29180]: pgmap v6774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:29 smithi119 ceph-mon[32752]: pgmap v6774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:29.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:29 smithi167 ceph-mon[32751]: pgmap v6774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11352.149748 secs 2024-04-26T21:44:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11352.149748 secs 2024-04-26T21:44:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11352.149748 secs 2024-04-26T21:44:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:31 smithi090 ceph-mon[29180]: pgmap v6775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11352.883682 secs 2024-04-26T21:44:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:31 smithi119 ceph-mon[32752]: pgmap v6775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11352.883682 secs 2024-04-26T21:44:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:31 smithi167 ceph-mon[32751]: pgmap v6775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11352.883682 secs 2024-04-26T21:44:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:33 smithi090 ceph-mon[29180]: pgmap v6776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:33 smithi119 ceph-mon[32752]: pgmap v6776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:33 smithi167 ceph-mon[32751]: pgmap v6776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:35 smithi090 ceph-mon[29180]: pgmap v6777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:35 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11357.149827 secs 2024-04-26T21:44:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:35 smithi119 ceph-mon[32752]: pgmap v6777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:35 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11357.149827 secs 2024-04-26T21:44:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:35 smithi167 ceph-mon[32751]: pgmap v6777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:35 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11357.149827 secs 2024-04-26T21:44:36.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11357.883840 secs 2024-04-26T21:44:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11357.883840 secs 2024-04-26T21:44:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11357.883840 secs 2024-04-26T21:44:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:37 smithi119 ceph-mon[32752]: pgmap v6778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:37 smithi167 ceph-mon[32751]: pgmap v6778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:37 smithi090 ceph-mon[29180]: pgmap v6778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:39 smithi119 ceph-mon[32752]: pgmap v6779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:39 smithi167 ceph-mon[32751]: pgmap v6779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:39 smithi090 ceph-mon[29180]: pgmap v6779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 11362.149897 secs 2024-04-26T21:44:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 11362.149897 secs 2024-04-26T21:44:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 11362.149897 secs 2024-04-26T21:44:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:41 smithi119 ceph-mon[32752]: pgmap v6780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11362.883991 secs 2024-04-26T21:44:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:41 smithi167 ceph-mon[32751]: pgmap v6780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11362.883991 secs 2024-04-26T21:44:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:41 smithi090 ceph-mon[29180]: pgmap v6780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11362.883991 secs 2024-04-26T21:44:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:43 smithi119 ceph-mon[32752]: pgmap v6781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:43 smithi167 ceph-mon[32751]: pgmap v6781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:43 smithi090 ceph-mon[29180]: pgmap v6781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:45 smithi119 ceph-mon[32752]: pgmap v6782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:45 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11367.149986 secs 2024-04-26T21:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:45 smithi119 ceph-mon[32752]: slow request 183.448703 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:45 smithi167 ceph-mon[32751]: pgmap v6782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:45 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11367.149986 secs 2024-04-26T21:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:45 smithi167 ceph-mon[32751]: slow request 183.448703 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:45 smithi090 ceph-mon[29180]: pgmap v6782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:45 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11367.149986 secs 2024-04-26T21:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:45 smithi090 ceph-mon[29180]: slow request 183.448703 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:44:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11367.884108 secs 2024-04-26T21:44:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11367.884108 secs 2024-04-26T21:44:46.821 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11367.884108 secs 2024-04-26T21:44:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:47 smithi119 ceph-mon[32752]: pgmap v6783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:44:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:44:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:47 smithi167 ceph-mon[32751]: pgmap v6783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:44:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:44:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:47 smithi090 ceph-mon[29180]: pgmap v6783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T21:44:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:44:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:44:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:49 smithi119 ceph-mon[32752]: pgmap v6784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:49 smithi119 ceph-mon[32752]: pgmap v6785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:44:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:49 smithi119 ceph-mon[32752]: pgmap v6786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:44:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:49 smithi167 ceph-mon[32751]: pgmap v6784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:49 smithi167 ceph-mon[32751]: pgmap v6785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:44:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:49 smithi167 ceph-mon[32751]: pgmap v6786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:44:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:49 smithi090 ceph-mon[29180]: pgmap v6784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:44:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:44:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:44:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:49 smithi090 ceph-mon[29180]: pgmap v6785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:44:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:49 smithi090 ceph-mon[29180]: pgmap v6786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:44:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11372.150088 secs 2024-04-26T21:44:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11372.150088 secs 2024-04-26T21:44:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11372.150088 secs 2024-04-26T21:44:51.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11372.884242 secs 2024-04-26T21:44:51.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:51 smithi119 ceph-mon[32752]: pgmap v6787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11372.884242 secs 2024-04-26T21:44:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:51 smithi167 ceph-mon[32751]: pgmap v6787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11372.884242 secs 2024-04-26T21:44:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:51 smithi090 ceph-mon[29180]: pgmap v6787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:53 smithi090 ceph-mon[29180]: pgmap v6788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:44:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:53 smithi119 ceph-mon[32752]: pgmap v6788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:44:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:53 smithi167 ceph-mon[32751]: pgmap v6788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:44:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:55 smithi090 ceph-mon[29180]: pgmap v6789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11377.150175 secs 2024-04-26T21:44:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:55 smithi119 ceph-mon[32752]: pgmap v6789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11377.150175 secs 2024-04-26T21:44:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:55 smithi167 ceph-mon[32751]: pgmap v6789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11377.150175 secs 2024-04-26T21:44:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11377.884412 secs 2024-04-26T21:44:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11377.884412 secs 2024-04-26T21:44:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11377.884412 secs 2024-04-26T21:44:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:57 smithi090 ceph-mon[29180]: pgmap v6790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:57 smithi119 ceph-mon[32752]: pgmap v6790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:44:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:57 smithi167 ceph-mon[32751]: pgmap v6790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:44:59 smithi090 ceph-mon[29180]: pgmap v6791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:45:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:44:59 smithi119 ceph-mon[32752]: pgmap v6791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:45:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:44:59 smithi167 ceph-mon[32751]: pgmap v6791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:45:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11382.150262 secs 2024-04-26T21:45:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11382.150262 secs 2024-04-26T21:45:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11382.150262 secs 2024-04-26T21:45:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11382.884540 secs 2024-04-26T21:45:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:01 smithi090 ceph-mon[29180]: pgmap v6792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11382.884540 secs 2024-04-26T21:45:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:01 smithi119 ceph-mon[32752]: pgmap v6792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11382.884540 secs 2024-04-26T21:45:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:01 smithi167 ceph-mon[32751]: pgmap v6792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:03 smithi090 ceph-mon[29180]: pgmap v6793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:03 smithi119 ceph-mon[32752]: pgmap v6793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:03 smithi167 ceph-mon[32751]: pgmap v6793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:05 smithi167 ceph-mon[32751]: pgmap v6794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:05 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11387.150348 secs 2024-04-26T21:45:06.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:05 smithi167 ceph-mon[32751]: slow request 1441.418765 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:45:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:05 smithi090 ceph-mon[29180]: pgmap v6794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:05 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11387.150348 secs 2024-04-26T21:45:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:05 smithi090 ceph-mon[29180]: slow request 1441.418765 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:45:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:05 smithi119 ceph-mon[32752]: pgmap v6794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:05 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11387.150348 secs 2024-04-26T21:45:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:05 smithi119 ceph-mon[32752]: slow request 1441.418765 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:45:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11387.884668 secs 2024-04-26T21:45:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11387.884668 secs 2024-04-26T21:45:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11387.884668 secs 2024-04-26T21:45:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:07 smithi090 ceph-mon[29180]: pgmap v6795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:07 smithi119 ceph-mon[32752]: pgmap v6795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:07 smithi167 ceph-mon[32751]: pgmap v6795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:09 smithi090 ceph-mon[29180]: pgmap v6796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:09 smithi119 ceph-mon[32752]: pgmap v6796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:09 smithi167 ceph-mon[32751]: pgmap v6796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11392.150432 secs 2024-04-26T21:45:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11392.150432 secs 2024-04-26T21:45:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11392.150432 secs 2024-04-26T21:45:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11392.884798 secs 2024-04-26T21:45:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:11 smithi119 ceph-mon[32752]: pgmap v6797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11392.884798 secs 2024-04-26T21:45:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:11 smithi167 ceph-mon[32751]: pgmap v6797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11392.884798 secs 2024-04-26T21:45:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:11 smithi090 ceph-mon[29180]: pgmap v6797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:13 smithi119 ceph-mon[32752]: pgmap v6798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:13 smithi167 ceph-mon[32751]: pgmap v6798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:13 smithi090 ceph-mon[29180]: pgmap v6798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:15 smithi119 ceph-mon[32752]: pgmap v6799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:15 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11397.150514 secs 2024-04-26T21:45:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:15 smithi119 ceph-mon[32752]: slow request 1441.382249 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:45:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:15 smithi167 ceph-mon[32751]: pgmap v6799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:15 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11397.150514 secs 2024-04-26T21:45:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:15 smithi167 ceph-mon[32751]: slow request 1441.382249 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:45:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:15 smithi090 ceph-mon[29180]: pgmap v6799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:15 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11397.150514 secs 2024-04-26T21:45:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:15 smithi090 ceph-mon[29180]: slow request 1441.382249 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T21:45:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11397.884873 secs 2024-04-26T21:45:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11397.884873 secs 2024-04-26T21:45:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11397.884873 secs 2024-04-26T21:45:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:17 smithi119 ceph-mon[32752]: pgmap v6800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:17 smithi167 ceph-mon[32751]: pgmap v6800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:17 smithi090 ceph-mon[29180]: pgmap v6800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:19 smithi119 ceph-mon[32752]: pgmap v6801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:19 smithi167 ceph-mon[32751]: pgmap v6801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:19 smithi090 ceph-mon[29180]: pgmap v6801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:21.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11402.150603 secs 2024-04-26T21:45:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11402.150603 secs 2024-04-26T21:45:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11402.150603 secs 2024-04-26T21:45:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11402.884997 secs 2024-04-26T21:45:22.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:21 smithi119 ceph-mon[32752]: pgmap v6802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11402.884997 secs 2024-04-26T21:45:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:21 smithi167 ceph-mon[32751]: pgmap v6802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11402.884997 secs 2024-04-26T21:45:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:21 smithi090 ceph-mon[29180]: pgmap v6802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:23 smithi119 ceph-mon[32752]: pgmap v6803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:23 smithi167 ceph-mon[32751]: pgmap v6803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:23 smithi090 ceph-mon[29180]: pgmap v6803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:25 smithi119 ceph-mon[32752]: pgmap v6804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11407.150687 secs 2024-04-26T21:45:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:25 smithi167 ceph-mon[32751]: pgmap v6804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11407.150687 secs 2024-04-26T21:45:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:25 smithi090 ceph-mon[29180]: pgmap v6804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11407.150687 secs 2024-04-26T21:45:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11407.885064 secs 2024-04-26T21:45:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11407.885064 secs 2024-04-26T21:45:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11407.885064 secs 2024-04-26T21:45:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:27 smithi119 ceph-mon[32752]: pgmap v6805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:27 smithi167 ceph-mon[32751]: pgmap v6805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:27 smithi090 ceph-mon[29180]: pgmap v6805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:29 smithi119 ceph-mon[32752]: pgmap v6806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:29 smithi167 ceph-mon[32751]: pgmap v6806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:29 smithi090 ceph-mon[29180]: pgmap v6806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11412.150769 secs 2024-04-26T21:45:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11412.150769 secs 2024-04-26T21:45:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11412.150769 secs 2024-04-26T21:45:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11412.885182 secs 2024-04-26T21:45:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:31 smithi119 ceph-mon[32752]: pgmap v6807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11412.885182 secs 2024-04-26T21:45:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:31 smithi167 ceph-mon[32751]: pgmap v6807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11412.885182 secs 2024-04-26T21:45:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:31 smithi090 ceph-mon[29180]: pgmap v6807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:34.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:33 smithi119 ceph-mon[32752]: pgmap v6808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:33 smithi167 ceph-mon[32751]: pgmap v6808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:33 smithi090 ceph-mon[29180]: pgmap v6808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:35 smithi119 ceph-mon[32752]: pgmap v6809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11417.150859 secs 2024-04-26T21:45:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:35 smithi167 ceph-mon[32751]: pgmap v6809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11417.150859 secs 2024-04-26T21:45:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:35 smithi090 ceph-mon[29180]: pgmap v6809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11417.150859 secs 2024-04-26T21:45:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11417.885296 secs 2024-04-26T21:45:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11417.885296 secs 2024-04-26T21:45:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11417.885296 secs 2024-04-26T21:45:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:37 smithi119 ceph-mon[32752]: pgmap v6810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:37 smithi167 ceph-mon[32751]: pgmap v6810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:37 smithi090 ceph-mon[29180]: pgmap v6810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:39 smithi119 ceph-mon[32752]: pgmap v6811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:39 smithi167 ceph-mon[32751]: pgmap v6811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:39 smithi090 ceph-mon[29180]: pgmap v6811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:40 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11422.150944 secs 2024-04-26T21:45:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:40 smithi119 ceph-mon[32752]: slow request 5762.515652 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:45:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:40 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11422.150944 secs 2024-04-26T21:45:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:40 smithi167 ceph-mon[32751]: slow request 5762.515652 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:45:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:40 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11422.150944 secs 2024-04-26T21:45:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:40 smithi090 ceph-mon[29180]: slow request 5762.515652 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:45:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11422.885432 secs 2024-04-26T21:45:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:41 smithi119 ceph-mon[32752]: pgmap v6812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11422.885432 secs 2024-04-26T21:45:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:41 smithi167 ceph-mon[32751]: pgmap v6812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11422.885432 secs 2024-04-26T21:45:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:41 smithi090 ceph-mon[29180]: pgmap v6812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:43 smithi119 ceph-mon[32752]: pgmap v6813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:43 smithi167 ceph-mon[32751]: pgmap v6813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:43 smithi090 ceph-mon[29180]: pgmap v6813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:45 smithi119 ceph-mon[32752]: pgmap v6814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11427.151029 secs 2024-04-26T21:45:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:45 smithi167 ceph-mon[32751]: pgmap v6814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11427.151029 secs 2024-04-26T21:45:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:45 smithi090 ceph-mon[29180]: pgmap v6814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11427.151029 secs 2024-04-26T21:45:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11427.885547 secs 2024-04-26T21:45:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11427.885547 secs 2024-04-26T21:45:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11427.885547 secs 2024-04-26T21:45:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:47 smithi119 ceph-mon[32752]: pgmap v6815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:47 smithi167 ceph-mon[32751]: pgmap v6815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:47 smithi090 ceph-mon[29180]: pgmap v6815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:45:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:45:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:49 smithi119 ceph-mon[32752]: pgmap v6816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:45:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:49 smithi167 ceph-mon[32751]: pgmap v6816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:45:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:49 smithi090 ceph-mon[29180]: pgmap v6816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:45:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:45:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:45:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:45:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:50 smithi119 ceph-mon[32752]: pgmap v6817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:45:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:50 smithi119 ceph-mon[32752]: pgmap v6818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11432.151090 secs 2024-04-26T21:45:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:50 smithi167 ceph-mon[32751]: pgmap v6817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:45:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:50 smithi167 ceph-mon[32751]: pgmap v6818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11432.151090 secs 2024-04-26T21:45:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:50 smithi090 ceph-mon[29180]: pgmap v6817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:45:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:50 smithi090 ceph-mon[29180]: pgmap v6818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11432.151090 secs 2024-04-26T21:45:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11432.885681 secs 2024-04-26T21:45:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11432.885681 secs 2024-04-26T21:45:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11432.885681 secs 2024-04-26T21:45:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:52 smithi119 ceph-mon[32752]: pgmap v6819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:52 smithi167 ceph-mon[32751]: pgmap v6819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:52 smithi090 ceph-mon[29180]: pgmap v6819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:54 smithi119 ceph-mon[32752]: pgmap v6820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:54 smithi167 ceph-mon[32751]: pgmap v6820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:54 smithi090 ceph-mon[29180]: pgmap v6820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11437.151172 secs 2024-04-26T21:45:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11437.151172 secs 2024-04-26T21:45:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11437.151172 secs 2024-04-26T21:45:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:57 smithi119 ceph-mon[32752]: pgmap v6821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:57 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11437.885800 secs 2024-04-26T21:45:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:57 smithi167 ceph-mon[32751]: pgmap v6821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:57 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11437.885800 secs 2024-04-26T21:45:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:56 smithi090 ceph-mon[29180]: pgmap v6821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:45:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11437.885800 secs 2024-04-26T21:45:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:45:58 smithi119 ceph-mon[32752]: pgmap v6822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:45:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:45:58 smithi167 ceph-mon[32751]: pgmap v6822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:45:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:45:58 smithi090 ceph-mon[29180]: pgmap v6822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:46:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:00 smithi090 ceph-mon[29180]: pgmap v6823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:46:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11442.151232 secs 2024-04-26T21:46:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:00 smithi119 ceph-mon[32752]: pgmap v6823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:46:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11442.151232 secs 2024-04-26T21:46:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:00 smithi167 ceph-mon[32751]: pgmap v6823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:46:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11442.151232 secs 2024-04-26T21:46:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11442.885914 secs 2024-04-26T21:46:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11442.885914 secs 2024-04-26T21:46:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11442.885914 secs 2024-04-26T21:46:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:02 smithi090 ceph-mon[29180]: pgmap v6824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:02 smithi119 ceph-mon[32752]: pgmap v6824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:02 smithi167 ceph-mon[32751]: pgmap v6824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:04 smithi119 ceph-mon[32752]: pgmap v6825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:04 smithi167 ceph-mon[32751]: pgmap v6825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:04 smithi090 ceph-mon[29180]: pgmap v6825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:06.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11447.151319 secs 2024-04-26T21:46:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11447.151319 secs 2024-04-26T21:46:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11447.151319 secs 2024-04-26T21:46:06.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:06 smithi167 ceph-mon[32751]: pgmap v6826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:06.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11447.885980 secs 2024-04-26T21:46:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:06 smithi119 ceph-mon[32752]: pgmap v6826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11447.885980 secs 2024-04-26T21:46:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:06 smithi090 ceph-mon[29180]: pgmap v6826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11447.885980 secs 2024-04-26T21:46:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:08 smithi119 ceph-mon[32752]: pgmap v6827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:08 smithi167 ceph-mon[32751]: pgmap v6827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:08 smithi090 ceph-mon[29180]: pgmap v6827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:10 smithi119 ceph-mon[32752]: pgmap v6828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11452.151400 secs 2024-04-26T21:46:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:10 smithi167 ceph-mon[32751]: pgmap v6828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11452.151400 secs 2024-04-26T21:46:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:10 smithi090 ceph-mon[29180]: pgmap v6828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11452.151400 secs 2024-04-26T21:46:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11452.886114 secs 2024-04-26T21:46:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11452.886114 secs 2024-04-26T21:46:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11452.886114 secs 2024-04-26T21:46:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:12 smithi119 ceph-mon[32752]: pgmap v6829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:12 smithi167 ceph-mon[32751]: pgmap v6829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:12 smithi090 ceph-mon[29180]: pgmap v6829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:14 smithi119 ceph-mon[32752]: pgmap v6830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:14 smithi167 ceph-mon[32751]: pgmap v6830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:14 smithi090 ceph-mon[29180]: pgmap v6830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11457.151482 secs 2024-04-26T21:46:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11457.151482 secs 2024-04-26T21:46:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11457.151482 secs 2024-04-26T21:46:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:16 smithi119 ceph-mon[32752]: pgmap v6831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11457.886194 secs 2024-04-26T21:46:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:16 smithi167 ceph-mon[32751]: pgmap v6831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11457.886194 secs 2024-04-26T21:46:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:16 smithi090 ceph-mon[29180]: pgmap v6831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11457.886194 secs 2024-04-26T21:46:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:18 smithi119 ceph-mon[32752]: pgmap v6832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:18 smithi167 ceph-mon[32751]: pgmap v6832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:18 smithi090 ceph-mon[29180]: pgmap v6832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:20 smithi090 ceph-mon[29180]: pgmap v6833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11462.151566 secs 2024-04-26T21:46:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:20 smithi119 ceph-mon[32752]: pgmap v6833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11462.151566 secs 2024-04-26T21:46:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:20 smithi167 ceph-mon[32751]: pgmap v6833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11462.151566 secs 2024-04-26T21:46:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11462.886318 secs 2024-04-26T21:46:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11462.886318 secs 2024-04-26T21:46:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11462.886318 secs 2024-04-26T21:46:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:22 smithi119 ceph-mon[32752]: pgmap v6834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:22 smithi167 ceph-mon[32751]: pgmap v6834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:22 smithi090 ceph-mon[29180]: pgmap v6834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:24 smithi119 ceph-mon[32752]: pgmap v6835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:24 smithi167 ceph-mon[32751]: pgmap v6835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:24 smithi090 ceph-mon[29180]: pgmap v6835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11467.151651 secs 2024-04-26T21:46:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11467.151651 secs 2024-04-26T21:46:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11467.151651 secs 2024-04-26T21:46:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:26 smithi119 ceph-mon[32752]: pgmap v6836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11467.886469 secs 2024-04-26T21:46:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:26 smithi167 ceph-mon[32751]: pgmap v6836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11467.886469 secs 2024-04-26T21:46:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:26 smithi090 ceph-mon[29180]: pgmap v6836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11467.886469 secs 2024-04-26T21:46:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:28 smithi119 ceph-mon[32752]: pgmap v6837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:28 smithi167 ceph-mon[32751]: pgmap v6837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:28 smithi090 ceph-mon[29180]: pgmap v6837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:30 smithi119 ceph-mon[32752]: pgmap v6838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11472.151734 secs 2024-04-26T21:46:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:30 smithi167 ceph-mon[32751]: pgmap v6838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11472.151734 secs 2024-04-26T21:46:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:30 smithi090 ceph-mon[29180]: pgmap v6838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11472.151734 secs 2024-04-26T21:46:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11472.886532 secs 2024-04-26T21:46:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11472.886532 secs 2024-04-26T21:46:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11472.886532 secs 2024-04-26T21:46:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:32 smithi119 ceph-mon[32752]: pgmap v6839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:32 smithi167 ceph-mon[32751]: pgmap v6839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:32 smithi090 ceph-mon[29180]: pgmap v6839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:34 smithi119 ceph-mon[32752]: pgmap v6840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:34 smithi167 ceph-mon[32751]: pgmap v6840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:34 smithi090 ceph-mon[29180]: pgmap v6840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:36.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11477.151825 secs 2024-04-26T21:46:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11477.151825 secs 2024-04-26T21:46:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11477.151825 secs 2024-04-26T21:46:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:36 smithi167 ceph-mon[32751]: pgmap v6841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11477.886599 secs 2024-04-26T21:46:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:36 smithi119 ceph-mon[32752]: pgmap v6841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11477.886599 secs 2024-04-26T21:46:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:36 smithi090 ceph-mon[29180]: pgmap v6841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11477.886599 secs 2024-04-26T21:46:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:38 smithi119 ceph-mon[32752]: pgmap v6842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:38 smithi167 ceph-mon[32751]: pgmap v6842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:38 smithi090 ceph-mon[29180]: pgmap v6842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:40 smithi119 ceph-mon[32752]: pgmap v6843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11482.151906 secs 2024-04-26T21:46:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:40 smithi167 ceph-mon[32751]: pgmap v6843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11482.151906 secs 2024-04-26T21:46:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:40 smithi090 ceph-mon[29180]: pgmap v6843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11482.151906 secs 2024-04-26T21:46:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11482.886717 secs 2024-04-26T21:46:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11482.886717 secs 2024-04-26T21:46:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11482.886717 secs 2024-04-26T21:46:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:42 smithi119 ceph-mon[32752]: pgmap v6844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:42 smithi167 ceph-mon[32751]: pgmap v6844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:42 smithi090 ceph-mon[29180]: pgmap v6844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:44 smithi119 ceph-mon[32752]: pgmap v6845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:44 smithi167 ceph-mon[32751]: pgmap v6845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:44 smithi090 ceph-mon[29180]: pgmap v6845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11487.151967 secs 2024-04-26T21:46:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11487.151967 secs 2024-04-26T21:46:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11487.151967 secs 2024-04-26T21:46:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:46 smithi119 ceph-mon[32752]: pgmap v6846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11487.886836 secs 2024-04-26T21:46:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:46 smithi167 ceph-mon[32751]: pgmap v6846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11487.886836 secs 2024-04-26T21:46:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:46 smithi090 ceph-mon[29180]: pgmap v6846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11487.886836 secs 2024-04-26T21:46:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:48 smithi119 ceph-mon[32752]: pgmap v6847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:48 smithi167 ceph-mon[32751]: pgmap v6847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:48 smithi090 ceph-mon[29180]: pgmap v6847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:46:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:50 smithi119 ceph-mon[32752]: pgmap v6848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11492.152032 secs 2024-04-26T21:46:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:50 smithi167 ceph-mon[32751]: pgmap v6848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11492.152032 secs 2024-04-26T21:46:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:50 smithi090 ceph-mon[29180]: pgmap v6848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11492.152032 secs 2024-04-26T21:46:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11492.886902 secs 2024-04-26T21:46:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11492.886902 secs 2024-04-26T21:46:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11492.886902 secs 2024-04-26T21:46:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:52 smithi119 ceph-mon[32752]: pgmap v6849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:52 smithi167 ceph-mon[32751]: pgmap v6849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:52 smithi090 ceph-mon[29180]: pgmap v6849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:46:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:46:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:46:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:54 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:46:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:54 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:46:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:54 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:46:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:54 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:46:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:54 smithi119 ceph-mon[32752]: pgmap v6850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:54 smithi119 ceph-mon[32752]: pgmap v6851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:54 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:54 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:54 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:54 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:54 smithi167 ceph-mon[32751]: pgmap v6850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:54 smithi167 ceph-mon[32751]: pgmap v6851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:54 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:54 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:46:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:54 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:46:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:54 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:46:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:54 smithi090 ceph-mon[29180]: pgmap v6850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:46:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:54 smithi090 ceph-mon[29180]: pgmap v6851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11497.152114 secs 2024-04-26T21:46:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11497.152114 secs 2024-04-26T21:46:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11497.152114 secs 2024-04-26T21:46:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:56 smithi119 ceph-mon[32752]: pgmap v6852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11497.886994 secs 2024-04-26T21:46:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:56 smithi167 ceph-mon[32751]: pgmap v6852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11497.886994 secs 2024-04-26T21:46:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:56 smithi090 ceph-mon[29180]: pgmap v6852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11497.886994 secs 2024-04-26T21:46:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:46:58 smithi119 ceph-mon[32752]: pgmap v6853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:46:58 smithi167 ceph-mon[32751]: pgmap v6853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:46:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:46:58 smithi090 ceph-mon[29180]: pgmap v6853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:47:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:00 smithi119 ceph-mon[32752]: pgmap v6854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:47:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11502.152177 secs 2024-04-26T21:47:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:00 smithi167 ceph-mon[32751]: pgmap v6854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:47:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11502.152177 secs 2024-04-26T21:47:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:00 smithi090 ceph-mon[29180]: pgmap v6854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:47:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11502.152177 secs 2024-04-26T21:47:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11502.887113 secs 2024-04-26T21:47:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11502.887113 secs 2024-04-26T21:47:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11502.887113 secs 2024-04-26T21:47:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:02 smithi119 ceph-mon[32752]: pgmap v6855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:47:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:02 smithi167 ceph-mon[32751]: pgmap v6855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:47:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:02 smithi090 ceph-mon[29180]: pgmap v6855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1 KiB/s rd, 2 op/s 2024-04-26T21:47:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:04 smithi119 ceph-mon[32752]: pgmap v6856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:47:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:04 smithi167 ceph-mon[32751]: pgmap v6856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:47:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:04 smithi090 ceph-mon[29180]: pgmap v6856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:47:06.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11507.152257 secs 2024-04-26T21:47:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11507.152257 secs 2024-04-26T21:47:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11507.152257 secs 2024-04-26T21:47:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:06 smithi119 ceph-mon[32752]: pgmap v6857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11507.887262 secs 2024-04-26T21:47:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:06 smithi167 ceph-mon[32751]: pgmap v6857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11507.887262 secs 2024-04-26T21:47:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:06 smithi090 ceph-mon[29180]: pgmap v6857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11507.887262 secs 2024-04-26T21:47:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:08 smithi119 ceph-mon[32752]: pgmap v6858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:08 smithi167 ceph-mon[32751]: pgmap v6858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:08 smithi090 ceph-mon[29180]: pgmap v6858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:10 smithi119 ceph-mon[32752]: pgmap v6859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11512.152342 secs 2024-04-26T21:47:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:10 smithi167 ceph-mon[32751]: pgmap v6859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11512.152342 secs 2024-04-26T21:47:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:10 smithi090 ceph-mon[29180]: pgmap v6859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11512.152342 secs 2024-04-26T21:47:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11512.887414 secs 2024-04-26T21:47:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11512.887414 secs 2024-04-26T21:47:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11512.887414 secs 2024-04-26T21:47:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:12 smithi119 ceph-mon[32752]: pgmap v6860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:12 smithi167 ceph-mon[32751]: pgmap v6860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:12 smithi090 ceph-mon[29180]: pgmap v6860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:14 smithi119 ceph-mon[32752]: pgmap v6861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:14 smithi167 ceph-mon[32751]: pgmap v6861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:14 smithi090 ceph-mon[29180]: pgmap v6861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11517.152428 secs 2024-04-26T21:47:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11517.152428 secs 2024-04-26T21:47:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11517.152428 secs 2024-04-26T21:47:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:17 smithi119 ceph-mon[32752]: pgmap v6862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:17 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11517.887542 secs 2024-04-26T21:47:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:17 smithi167 ceph-mon[32751]: pgmap v6862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:17 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11517.887542 secs 2024-04-26T21:47:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:17 smithi090 ceph-mon[29180]: pgmap v6862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:17 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11517.887542 secs 2024-04-26T21:47:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:19 smithi119 ceph-mon[32752]: pgmap v6863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:19 smithi167 ceph-mon[32751]: pgmap v6863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:19 smithi090 ceph-mon[29180]: pgmap v6863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:21 smithi119 ceph-mon[32752]: pgmap v6864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:21 smithi119 ceph-mon[32752]: 19 slow requests, 4 included below; oldest blocked for > 11522.152513 secs 2024-04-26T21:47:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:21 smithi119 ceph-mon[32752]: slow request 11522.152513 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T21:47:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:21 smithi119 ceph-mon[32752]: slow request 11522.149618 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:21 smithi119 ceph-mon[32752]: slow request 11522.144008 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:21 smithi119 ceph-mon[32752]: slow request 11520.177282 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T21:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:21 smithi167 ceph-mon[32751]: pgmap v6864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:21 smithi167 ceph-mon[32751]: 19 slow requests, 4 included below; oldest blocked for > 11522.152513 secs 2024-04-26T21:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:21 smithi167 ceph-mon[32751]: slow request 11522.152513 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T21:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:21 smithi167 ceph-mon[32751]: slow request 11522.149618 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:21 smithi167 ceph-mon[32751]: slow request 11522.144008 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:21 smithi167 ceph-mon[32751]: slow request 11520.177282 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T21:47:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:21 smithi090 ceph-mon[29180]: pgmap v6864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:21 smithi090 ceph-mon[29180]: 19 slow requests, 4 included below; oldest blocked for > 11522.152513 secs 2024-04-26T21:47:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:21 smithi090 ceph-mon[29180]: slow request 11522.152513 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-26T21:47:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:21 smithi090 ceph-mon[29180]: slow request 11522.149618 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:21 smithi090 ceph-mon[29180]: slow request 11522.144008 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:21 smithi090 ceph-mon[29180]: slow request 11520.177282 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-26T21:47:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:22 smithi119 ceph-mon[32752]: 15 slow requests, 5 included below; oldest blocked for > 11522.887660 secs 2024-04-26T21:47:22.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:22 smithi119 ceph-mon[32752]: slow request 11522.887659 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:22 smithi119 ceph-mon[32752]: slow request 11522.886098 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:22 smithi119 ceph-mon[32752]: slow request 11522.884629 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.328 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:22 smithi119 ceph-mon[32752]: slow request 11522.884159 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:22 smithi119 ceph-mon[32752]: slow request 11522.871894 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:22 smithi167 ceph-mon[32751]: 15 slow requests, 5 included below; oldest blocked for > 11522.887660 secs 2024-04-26T21:47:22.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:22 smithi167 ceph-mon[32751]: slow request 11522.887659 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:22 smithi167 ceph-mon[32751]: slow request 11522.886098 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:22 smithi167 ceph-mon[32751]: slow request 11522.884629 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.330 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:22 smithi167 ceph-mon[32751]: slow request 11522.884159 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.331 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:22 smithi167 ceph-mon[32751]: slow request 11522.871894 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:22 smithi090 ceph-mon[29180]: 15 slow requests, 5 included below; oldest blocked for > 11522.887660 secs 2024-04-26T21:47:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:22 smithi090 ceph-mon[29180]: slow request 11522.887659 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:22 smithi090 ceph-mon[29180]: slow request 11522.886098 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:22 smithi090 ceph-mon[29180]: slow request 11522.884629 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:22 smithi090 ceph-mon[29180]: slow request 11522.884159 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:22 smithi090 ceph-mon[29180]: slow request 11522.871894 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-26T21:47:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:23 smithi119 ceph-mon[32752]: pgmap v6865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:23 smithi167 ceph-mon[32751]: pgmap v6865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:23 smithi090 ceph-mon[29180]: pgmap v6865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:24 smithi119 ceph-mon[32752]: pgmap v6866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:24 smithi167 ceph-mon[32751]: pgmap v6866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:24 smithi090 ceph-mon[29180]: pgmap v6866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11527.152597 secs 2024-04-26T21:47:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11527.152597 secs 2024-04-26T21:47:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11527.152597 secs 2024-04-26T21:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:26 smithi119 ceph-mon[32752]: pgmap v6867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:26 smithi119 ceph-mon[32752]: 15 slow requests, 4 included below; oldest blocked for > 11527.887789 secs 2024-04-26T21:47:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:26 smithi119 ceph-mon[32752]: slow request 11524.303152 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T21:47:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:26 smithi119 ceph-mon[32752]: slow request 11524.302816 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T21:47:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:26 smithi119 ceph-mon[32752]: slow request 11524.193981 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T21:47:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:26 smithi119 ceph-mon[32752]: slow request 11524.189693 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T21:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:26 smithi167 ceph-mon[32751]: pgmap v6867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:26 smithi167 ceph-mon[32751]: 15 slow requests, 4 included below; oldest blocked for > 11527.887789 secs 2024-04-26T21:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:26 smithi167 ceph-mon[32751]: slow request 11524.303152 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T21:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:26 smithi167 ceph-mon[32751]: slow request 11524.302816 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T21:47:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:26 smithi167 ceph-mon[32751]: slow request 11524.193981 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T21:47:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:26 smithi167 ceph-mon[32751]: slow request 11524.189693 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T21:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:26 smithi090 ceph-mon[29180]: pgmap v6867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:26 smithi090 ceph-mon[29180]: 15 slow requests, 4 included below; oldest blocked for > 11527.887789 secs 2024-04-26T21:47:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:26 smithi090 ceph-mon[29180]: slow request 11524.303152 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-26T21:47:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:26 smithi090 ceph-mon[29180]: slow request 11524.302816 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-26T21:47:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:26 smithi090 ceph-mon[29180]: slow request 11524.193981 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-26T21:47:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:26 smithi090 ceph-mon[29180]: slow request 11524.189693 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-26T21:47:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:28 smithi119 ceph-mon[32752]: pgmap v6868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:28 smithi167 ceph-mon[32751]: pgmap v6868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:28 smithi090 ceph-mon[29180]: pgmap v6868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:30 smithi119 ceph-mon[32752]: pgmap v6869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11532.152681 secs 2024-04-26T21:47:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:30 smithi167 ceph-mon[32751]: pgmap v6869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11532.152681 secs 2024-04-26T21:47:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:30 smithi090 ceph-mon[29180]: pgmap v6869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11532.152681 secs 2024-04-26T21:47:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11532.887856 secs 2024-04-26T21:47:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11532.887856 secs 2024-04-26T21:47:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11532.887856 secs 2024-04-26T21:47:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:32 smithi119 ceph-mon[32752]: pgmap v6870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:32 smithi167 ceph-mon[32751]: pgmap v6870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:32 smithi090 ceph-mon[29180]: pgmap v6870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:34 smithi119 ceph-mon[32752]: pgmap v6871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:34 smithi167 ceph-mon[32751]: pgmap v6871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:34 smithi090 ceph-mon[29180]: pgmap v6871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:36.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11537.152765 secs 2024-04-26T21:47:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11537.152765 secs 2024-04-26T21:47:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11537.152765 secs 2024-04-26T21:47:36.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:36 smithi167 ceph-mon[32751]: pgmap v6872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:36.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11537.887923 secs 2024-04-26T21:47:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:36 smithi119 ceph-mon[32752]: pgmap v6872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11537.887923 secs 2024-04-26T21:47:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:36 smithi090 ceph-mon[29180]: pgmap v6872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11537.887923 secs 2024-04-26T21:47:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:38 smithi119 ceph-mon[32752]: pgmap v6873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:38 smithi167 ceph-mon[32751]: pgmap v6873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:38 smithi090 ceph-mon[29180]: pgmap v6873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:40 smithi119 ceph-mon[32752]: pgmap v6874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11542.152825 secs 2024-04-26T21:47:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:40 smithi167 ceph-mon[32751]: pgmap v6874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11542.152825 secs 2024-04-26T21:47:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:40 smithi090 ceph-mon[29180]: pgmap v6874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11542.152825 secs 2024-04-26T21:47:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11542.888027 secs 2024-04-26T21:47:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11542.888027 secs 2024-04-26T21:47:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11542.888027 secs 2024-04-26T21:47:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:42 smithi119 ceph-mon[32752]: pgmap v6875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:42 smithi167 ceph-mon[32751]: pgmap v6875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:42 smithi090 ceph-mon[29180]: pgmap v6875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:44 smithi119 ceph-mon[32752]: pgmap v6876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:44 smithi167 ceph-mon[32751]: pgmap v6876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:44 smithi090 ceph-mon[29180]: pgmap v6876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:45 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11547.152908 secs 2024-04-26T21:47:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:45 smithi119 ceph-mon[32752]: slow request 363.451625 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:47:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:45 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11547.152908 secs 2024-04-26T21:47:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:45 smithi167 ceph-mon[32751]: slow request 363.451625 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:47:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:45 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11547.152908 secs 2024-04-26T21:47:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:45 smithi090 ceph-mon[29180]: slow request 363.451625 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:47:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:46 smithi119 ceph-mon[32752]: pgmap v6877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11547.888131 secs 2024-04-26T21:47:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:46 smithi167 ceph-mon[32751]: pgmap v6877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11547.888131 secs 2024-04-26T21:47:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:46 smithi090 ceph-mon[29180]: pgmap v6877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11547.888131 secs 2024-04-26T21:47:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:48 smithi119 ceph-mon[32752]: pgmap v6878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:48 smithi167 ceph-mon[32751]: pgmap v6878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:48 smithi090 ceph-mon[29180]: pgmap v6878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T21:47:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:50 smithi119 ceph-mon[32752]: pgmap v6879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11552.152970 secs 2024-04-26T21:47:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:50 smithi167 ceph-mon[32751]: pgmap v6879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11552.152970 secs 2024-04-26T21:47:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:50 smithi090 ceph-mon[29180]: pgmap v6879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T21:47:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11552.152970 secs 2024-04-26T21:47:51.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11552.888240 secs 2024-04-26T21:47:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11552.888240 secs 2024-04-26T21:47:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11552.888240 secs 2024-04-26T21:47:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:52 smithi119 ceph-mon[32752]: pgmap v6880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:52 smithi167 ceph-mon[32751]: pgmap v6880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:52 smithi090 ceph-mon[29180]: pgmap v6880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:47:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:54 smithi119 ceph-mon[32752]: pgmap v6881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:54 smithi167 ceph-mon[32751]: pgmap v6881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:54 smithi090 ceph-mon[29180]: pgmap v6881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:47:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:47:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:55 smithi119 ceph-mon[32752]: pgmap v6882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T21:47:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:55 smithi119 ceph-mon[32752]: pgmap v6883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:47:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11557.153058 secs 2024-04-26T21:47:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:55 smithi167 ceph-mon[32751]: pgmap v6882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T21:47:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:55 smithi167 ceph-mon[32751]: pgmap v6883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:47:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11557.153058 secs 2024-04-26T21:47:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:47:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:47:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:55 smithi090 ceph-mon[29180]: pgmap v6882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T21:47:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:55 smithi090 ceph-mon[29180]: pgmap v6883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:47:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:47:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11557.153058 secs 2024-04-26T21:47:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11557.888341 secs 2024-04-26T21:47:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11557.888341 secs 2024-04-26T21:47:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11557.888341 secs 2024-04-26T21:47:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:57 smithi119 ceph-mon[32752]: pgmap v6884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:47:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:57 smithi167 ceph-mon[32751]: pgmap v6884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:47:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:57 smithi090 ceph-mon[29180]: pgmap v6884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:47:59 smithi119 ceph-mon[32752]: pgmap v6885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:47:59 smithi167 ceph-mon[32751]: pgmap v6885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:47:59 smithi090 ceph-mon[29180]: pgmap v6885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11562.153144 secs 2024-04-26T21:48:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11562.153144 secs 2024-04-26T21:48:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11562.153144 secs 2024-04-26T21:48:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11562.888461 secs 2024-04-26T21:48:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:01 smithi119 ceph-mon[32752]: pgmap v6886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11562.888461 secs 2024-04-26T21:48:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:01 smithi167 ceph-mon[32751]: pgmap v6886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11562.888461 secs 2024-04-26T21:48:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:01 smithi090 ceph-mon[29180]: pgmap v6886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:03 smithi119 ceph-mon[32752]: pgmap v6887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T21:48:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:03 smithi167 ceph-mon[32751]: pgmap v6887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T21:48:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:03 smithi090 ceph-mon[29180]: pgmap v6887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T21:48:04.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:48:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:48:03.888+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:48:04.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:48:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:48:03.888+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:48:04.378 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:48:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:48:03.888+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:48:04.378 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:48:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:48:03.888+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:48:04.378 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:48:03 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:48:03.888+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:48:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:05 smithi119 ceph-mon[32752]: pgmap v6888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:48:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:05 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11567.153227 secs 2024-04-26T21:48:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:05 smithi119 ceph-mon[32752]: slow request 11524.148619 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:48:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:05 smithi167 ceph-mon[32751]: pgmap v6888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:48:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:05 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11567.153227 secs 2024-04-26T21:48:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:05 smithi167 ceph-mon[32751]: slow request 11524.148619 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:48:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:05 smithi090 ceph-mon[29180]: pgmap v6888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:48:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:05 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11567.153227 secs 2024-04-26T21:48:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:05 smithi090 ceph-mon[29180]: slow request 11524.148619 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:48:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:06 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 11567.888526 secs 2024-04-26T21:48:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:06 smithi119 ceph-mon[32752]: slow request 2882.525293 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:48:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:06 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 11567.888526 secs 2024-04-26T21:48:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:06 smithi167 ceph-mon[32751]: slow request 2882.525293 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:48:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:06 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 11567.888526 secs 2024-04-26T21:48:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:06 smithi090 ceph-mon[29180]: slow request 2882.525293 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:48:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:07 smithi119 ceph-mon[32752]: pgmap v6889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:07 smithi167 ceph-mon[32751]: pgmap v6889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:07 smithi090 ceph-mon[29180]: pgmap v6889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:09 smithi119 ceph-mon[32752]: pgmap v6890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:09 smithi167 ceph-mon[32751]: pgmap v6890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:09 smithi090 ceph-mon[29180]: pgmap v6890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11572.153311 secs 2024-04-26T21:48:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11572.153311 secs 2024-04-26T21:48:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11572.153311 secs 2024-04-26T21:48:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11572.888648 secs 2024-04-26T21:48:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:11 smithi119 ceph-mon[32752]: pgmap v6891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11572.888648 secs 2024-04-26T21:48:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:11 smithi167 ceph-mon[32751]: pgmap v6891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11572.888648 secs 2024-04-26T21:48:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:11 smithi090 ceph-mon[29180]: pgmap v6891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:13 smithi119 ceph-mon[32752]: pgmap v6892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:13 smithi167 ceph-mon[32751]: pgmap v6892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:13 smithi090 ceph-mon[29180]: pgmap v6892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:15 smithi119 ceph-mon[32752]: pgmap v6893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11577.153401 secs 2024-04-26T21:48:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:15 smithi167 ceph-mon[32751]: pgmap v6893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11577.153401 secs 2024-04-26T21:48:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:15 smithi090 ceph-mon[29180]: pgmap v6893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11577.153401 secs 2024-04-26T21:48:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11577.888767 secs 2024-04-26T21:48:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11577.888767 secs 2024-04-26T21:48:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11577.888767 secs 2024-04-26T21:48:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:17 smithi119 ceph-mon[32752]: pgmap v6894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:17 smithi167 ceph-mon[32751]: pgmap v6894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:17 smithi090 ceph-mon[29180]: pgmap v6894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:19 smithi119 ceph-mon[32752]: pgmap v6895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:19 smithi167 ceph-mon[32751]: pgmap v6895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:19 smithi090 ceph-mon[29180]: pgmap v6895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11582.153485 secs 2024-04-26T21:48:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11582.153485 secs 2024-04-26T21:48:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11582.153485 secs 2024-04-26T21:48:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11582.888852 secs 2024-04-26T21:48:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:21 smithi119 ceph-mon[32752]: pgmap v6896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11582.888852 secs 2024-04-26T21:48:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:21 smithi167 ceph-mon[32751]: pgmap v6896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11582.888852 secs 2024-04-26T21:48:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:21 smithi090 ceph-mon[29180]: pgmap v6896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:23 smithi119 ceph-mon[32752]: pgmap v6897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:23 smithi167 ceph-mon[32751]: pgmap v6897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:23 smithi090 ceph-mon[29180]: pgmap v6897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:25 smithi119 ceph-mon[32752]: pgmap v6898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11587.153570 secs 2024-04-26T21:48:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:25 smithi167 ceph-mon[32751]: pgmap v6898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11587.153570 secs 2024-04-26T21:48:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:25 smithi090 ceph-mon[29180]: pgmap v6898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11587.153570 secs 2024-04-26T21:48:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11587.888975 secs 2024-04-26T21:48:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11587.888975 secs 2024-04-26T21:48:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11587.888975 secs 2024-04-26T21:48:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:28 smithi119 ceph-mon[32752]: pgmap v6899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:28 smithi167 ceph-mon[32751]: pgmap v6899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:28 smithi090 ceph-mon[29180]: pgmap v6899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:30 smithi119 ceph-mon[32752]: pgmap v6900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:30 smithi167 ceph-mon[32751]: pgmap v6900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:30 smithi090 ceph-mon[29180]: pgmap v6900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11592.153652 secs 2024-04-26T21:48:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11592.153652 secs 2024-04-26T21:48:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11592.153652 secs 2024-04-26T21:48:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:32 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11592.889089 secs 2024-04-26T21:48:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:32 smithi119 ceph-mon[32752]: pgmap v6901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:32 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11592.889089 secs 2024-04-26T21:48:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:32 smithi167 ceph-mon[32751]: pgmap v6901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:32 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11592.889089 secs 2024-04-26T21:48:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:32 smithi090 ceph-mon[29180]: pgmap v6901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:34 smithi119 ceph-mon[32752]: pgmap v6902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:34 smithi167 ceph-mon[32751]: pgmap v6902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:34 smithi090 ceph-mon[29180]: pgmap v6902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:36 smithi119 ceph-mon[32752]: pgmap v6903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11597.153738 secs 2024-04-26T21:48:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:36 smithi167 ceph-mon[32751]: pgmap v6903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11597.153738 secs 2024-04-26T21:48:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:36 smithi090 ceph-mon[29180]: pgmap v6903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11597.153738 secs 2024-04-26T21:48:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:37 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11597.889224 secs 2024-04-26T21:48:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:37 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11597.889224 secs 2024-04-26T21:48:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:37 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11597.889224 secs 2024-04-26T21:48:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:38 smithi119 ceph-mon[32752]: pgmap v6904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:38 smithi167 ceph-mon[32751]: pgmap v6904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:38 smithi090 ceph-mon[29180]: pgmap v6904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:39 smithi119 ceph-mon[32752]: pgmap v6905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:39 smithi167 ceph-mon[32751]: pgmap v6905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:39 smithi090 ceph-mon[29180]: pgmap v6905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11602.153846 secs 2024-04-26T21:48:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11602.153846 secs 2024-04-26T21:48:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11602.153846 secs 2024-04-26T21:48:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11602.889338 secs 2024-04-26T21:48:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:41 smithi167 ceph-mon[32751]: pgmap v6906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11602.889338 secs 2024-04-26T21:48:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:41 smithi090 ceph-mon[29180]: pgmap v6906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11602.889338 secs 2024-04-26T21:48:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:41 smithi119 ceph-mon[32752]: pgmap v6906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:43 smithi119 ceph-mon[32752]: pgmap v6907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:43 smithi167 ceph-mon[32751]: pgmap v6907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:43 smithi090 ceph-mon[29180]: pgmap v6907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:45 smithi119 ceph-mon[32752]: pgmap v6908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11607.153934 secs 2024-04-26T21:48:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:45 smithi167 ceph-mon[32751]: pgmap v6908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11607.153934 secs 2024-04-26T21:48:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:45 smithi090 ceph-mon[29180]: pgmap v6908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11607.153934 secs 2024-04-26T21:48:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11607.889419 secs 2024-04-26T21:48:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11607.889419 secs 2024-04-26T21:48:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11607.889419 secs 2024-04-26T21:48:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:47 smithi119 ceph-mon[32752]: pgmap v6909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:47 smithi167 ceph-mon[32751]: pgmap v6909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:47 smithi090 ceph-mon[29180]: pgmap v6909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:49 smithi119 ceph-mon[32752]: pgmap v6910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:49 smithi167 ceph-mon[32751]: pgmap v6910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:49 smithi090 ceph-mon[29180]: pgmap v6910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11612.153993 secs 2024-04-26T21:48:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11612.153993 secs 2024-04-26T21:48:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11612.153993 secs 2024-04-26T21:48:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11612.889555 secs 2024-04-26T21:48:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:51 smithi119 ceph-mon[32752]: pgmap v6911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11612.889555 secs 2024-04-26T21:48:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:51 smithi167 ceph-mon[32751]: pgmap v6911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11612.889555 secs 2024-04-26T21:48:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:51 smithi090 ceph-mon[29180]: pgmap v6911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:53 smithi119 ceph-mon[32752]: pgmap v6912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:53 smithi167 ceph-mon[32751]: pgmap v6912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:53 smithi090 ceph-mon[29180]: pgmap v6912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:48:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:55 smithi119 ceph-mon[32752]: pgmap v6913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11617.154094 secs 2024-04-26T21:48:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:55 smithi167 ceph-mon[32751]: pgmap v6913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11617.154094 secs 2024-04-26T21:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:55 smithi090 ceph-mon[29180]: pgmap v6913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11617.154094 secs 2024-04-26T21:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:48:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:48:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:48:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11617.889621 secs 2024-04-26T21:48:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:56 smithi119 ceph-mon[32752]: pgmap v6914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:48:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:56 smithi119 ceph-mon[32752]: pgmap v6915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11617.889621 secs 2024-04-26T21:48:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:56 smithi167 ceph-mon[32751]: pgmap v6914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:48:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:56 smithi167 ceph-mon[32751]: pgmap v6915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11617.889621 secs 2024-04-26T21:48:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:56 smithi090 ceph-mon[29180]: pgmap v6914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:48:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:56 smithi090 ceph-mon[29180]: pgmap v6915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:48:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:48:58 smithi119 ceph-mon[32752]: pgmap v6916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:48:59.342 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:48:58 smithi167 ceph-mon[32751]: pgmap v6916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:48:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:48:58 smithi090 ceph-mon[29180]: pgmap v6916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:49:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:00 smithi119 ceph-mon[32752]: pgmap v6917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11622.154179 secs 2024-04-26T21:49:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:00 smithi167 ceph-mon[32751]: pgmap v6917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11622.154179 secs 2024-04-26T21:49:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:00 smithi090 ceph-mon[29180]: pgmap v6917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11622.154179 secs 2024-04-26T21:49:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11622.889734 secs 2024-04-26T21:49:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11622.889734 secs 2024-04-26T21:49:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11622.889734 secs 2024-04-26T21:49:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:02 smithi119 ceph-mon[32752]: pgmap v6918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:02 smithi167 ceph-mon[32751]: pgmap v6918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:02 smithi090 ceph-mon[29180]: pgmap v6918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:05 smithi119 ceph-mon[32752]: pgmap v6919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:05 smithi167 ceph-mon[32751]: pgmap v6919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:04 smithi090 ceph-mon[29180]: pgmap v6919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:06.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11627.154263 secs 2024-04-26T21:49:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11627.154263 secs 2024-04-26T21:49:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11627.154263 secs 2024-04-26T21:49:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:07 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11627.889870 secs 2024-04-26T21:49:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:07 smithi119 ceph-mon[32752]: pgmap v6920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:49:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:07 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11627.889870 secs 2024-04-26T21:49:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:07 smithi167 ceph-mon[32751]: pgmap v6920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:49:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:07 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11627.889870 secs 2024-04-26T21:49:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:07 smithi090 ceph-mon[29180]: pgmap v6920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:49:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:09 smithi119 ceph-mon[32752]: pgmap v6921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:09 smithi167 ceph-mon[32751]: pgmap v6921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:09 smithi090 ceph-mon[29180]: pgmap v6921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:11 smithi119 ceph-mon[32752]: pgmap v6922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11632.154345 secs 2024-04-26T21:49:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:11 smithi167 ceph-mon[32751]: pgmap v6922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11632.154345 secs 2024-04-26T21:49:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:11 smithi090 ceph-mon[29180]: pgmap v6922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11632.154345 secs 2024-04-26T21:49:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:12 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11632.889986 secs 2024-04-26T21:49:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:12 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11632.889986 secs 2024-04-26T21:49:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:12 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11632.889986 secs 2024-04-26T21:49:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:13 smithi119 ceph-mon[32752]: pgmap v6923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:13 smithi167 ceph-mon[32751]: pgmap v6923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:13 smithi090 ceph-mon[29180]: pgmap v6923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:15 smithi119 ceph-mon[32752]: pgmap v6924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:15 smithi167 ceph-mon[32751]: pgmap v6924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:15 smithi090 ceph-mon[29180]: pgmap v6924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11637.154428 secs 2024-04-26T21:49:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11637.154428 secs 2024-04-26T21:49:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11637.154428 secs 2024-04-26T21:49:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:17 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11637.890104 secs 2024-04-26T21:49:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:17 smithi119 ceph-mon[32752]: pgmap v6925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:17 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11637.890104 secs 2024-04-26T21:49:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:17 smithi167 ceph-mon[32751]: pgmap v6925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:17 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11637.890104 secs 2024-04-26T21:49:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:17 smithi090 ceph-mon[29180]: pgmap v6925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:18 smithi119 ceph-mon[32752]: pgmap v6926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:18 smithi167 ceph-mon[32751]: pgmap v6926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:18 smithi090 ceph-mon[29180]: pgmap v6926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:21.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:20 smithi119 ceph-mon[32752]: pgmap v6927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:21.007 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11642.154512 secs 2024-04-26T21:49:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:20 smithi167 ceph-mon[32751]: pgmap v6927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11642.154512 secs 2024-04-26T21:49:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:20 smithi090 ceph-mon[29180]: pgmap v6927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11642.154512 secs 2024-04-26T21:49:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11642.890209 secs 2024-04-26T21:49:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11642.890209 secs 2024-04-26T21:49:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11642.890209 secs 2024-04-26T21:49:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:22 smithi119 ceph-mon[32752]: pgmap v6928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:22 smithi167 ceph-mon[32751]: pgmap v6928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:22 smithi090 ceph-mon[29180]: pgmap v6928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:24 smithi119 ceph-mon[32752]: pgmap v6929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:24 smithi167 ceph-mon[32751]: pgmap v6929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:24 smithi090 ceph-mon[29180]: pgmap v6929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11647.154598 secs 2024-04-26T21:49:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11647.154598 secs 2024-04-26T21:49:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11647.154598 secs 2024-04-26T21:49:27.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11647.890326 secs 2024-04-26T21:49:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:26 smithi090 ceph-mon[29180]: pgmap v6930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11647.890326 secs 2024-04-26T21:49:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:26 smithi119 ceph-mon[32752]: pgmap v6930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11647.890326 secs 2024-04-26T21:49:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:26 smithi167 ceph-mon[32751]: pgmap v6930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:28 smithi119 ceph-mon[32752]: pgmap v6931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:28 smithi167 ceph-mon[32751]: pgmap v6931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:28 smithi090 ceph-mon[29180]: pgmap v6931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:30 smithi119 ceph-mon[32752]: pgmap v6932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11652.154682 secs 2024-04-26T21:49:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:30 smithi167 ceph-mon[32751]: pgmap v6932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11652.154682 secs 2024-04-26T21:49:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:30 smithi090 ceph-mon[29180]: pgmap v6932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11652.154682 secs 2024-04-26T21:49:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11652.890441 secs 2024-04-26T21:49:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11652.890441 secs 2024-04-26T21:49:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11652.890441 secs 2024-04-26T21:49:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:32 smithi119 ceph-mon[32752]: pgmap v6933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:32 smithi167 ceph-mon[32751]: pgmap v6933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:32 smithi090 ceph-mon[29180]: pgmap v6933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:34 smithi119 ceph-mon[32752]: pgmap v6934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:34 smithi167 ceph-mon[32751]: pgmap v6934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:34 smithi090 ceph-mon[29180]: pgmap v6934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:35 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11657.154768 secs 2024-04-26T21:49:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:35 smithi119 ceph-mon[32752]: slow request 11521.418120 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T21:49:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:35 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11657.154768 secs 2024-04-26T21:49:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:35 smithi167 ceph-mon[32751]: slow request 11521.418120 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T21:49:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:35 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11657.154768 secs 2024-04-26T21:49:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:35 smithi090 ceph-mon[29180]: slow request 11521.418120 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-26T21:49:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11657.890558 secs 2024-04-26T21:49:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:36 smithi119 ceph-mon[32752]: pgmap v6935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11657.890558 secs 2024-04-26T21:49:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:36 smithi167 ceph-mon[32751]: pgmap v6935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11657.890558 secs 2024-04-26T21:49:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:36 smithi090 ceph-mon[29180]: pgmap v6935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:38 smithi119 ceph-mon[32752]: pgmap v6936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:38 smithi167 ceph-mon[32751]: pgmap v6936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:38 smithi090 ceph-mon[29180]: pgmap v6936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:40 smithi119 ceph-mon[32752]: pgmap v6937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11662.154857 secs 2024-04-26T21:49:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:40 smithi167 ceph-mon[32751]: pgmap v6937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11662.154857 secs 2024-04-26T21:49:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:40 smithi090 ceph-mon[29180]: pgmap v6937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11662.154857 secs 2024-04-26T21:49:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11662.890678 secs 2024-04-26T21:49:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11662.890678 secs 2024-04-26T21:49:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11662.890678 secs 2024-04-26T21:49:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:42 smithi167 ceph-mon[32751]: pgmap v6938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:42 smithi090 ceph-mon[29180]: pgmap v6938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:42 smithi119 ceph-mon[32752]: pgmap v6938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:44 smithi167 ceph-mon[32751]: pgmap v6939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:44 smithi090 ceph-mon[29180]: pgmap v6939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:44 smithi119 ceph-mon[32752]: pgmap v6939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11667.154943 secs 2024-04-26T21:49:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11667.154943 secs 2024-04-26T21:49:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11667.154943 secs 2024-04-26T21:49:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11667.890797 secs 2024-04-26T21:49:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:46 smithi167 ceph-mon[32751]: pgmap v6940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11667.890797 secs 2024-04-26T21:49:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:46 smithi090 ceph-mon[29180]: pgmap v6940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11667.890797 secs 2024-04-26T21:49:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:46 smithi119 ceph-mon[32752]: pgmap v6940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:48 smithi090 ceph-mon[29180]: pgmap v6941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:48 smithi119 ceph-mon[32752]: pgmap v6941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:48 smithi167 ceph-mon[32751]: pgmap v6941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:51.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:50 smithi119 ceph-mon[32752]: pgmap v6942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:51.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11672.155028 secs 2024-04-26T21:49:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:50 smithi090 ceph-mon[29180]: pgmap v6942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11672.155028 secs 2024-04-26T21:49:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:50 smithi167 ceph-mon[32751]: pgmap v6942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11672.155028 secs 2024-04-26T21:49:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11672.890897 secs 2024-04-26T21:49:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11672.890897 secs 2024-04-26T21:49:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11672.890897 secs 2024-04-26T21:49:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:52 smithi090 ceph-mon[29180]: pgmap v6943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:52 smithi119 ceph-mon[32752]: pgmap v6943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:52 smithi167 ceph-mon[32751]: pgmap v6943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:54 smithi090 ceph-mon[29180]: pgmap v6944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:54 smithi119 ceph-mon[32752]: pgmap v6944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:54 smithi167 ceph-mon[32751]: pgmap v6944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:49:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11677.155114 secs 2024-04-26T21:49:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11677.155114 secs 2024-04-26T21:49:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11677.155114 secs 2024-04-26T21:49:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:49:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11677.890983 secs 2024-04-26T21:49:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:56 smithi090 ceph-mon[29180]: pgmap v6945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:49:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11677.890983 secs 2024-04-26T21:49:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:56 smithi119 ceph-mon[32752]: pgmap v6945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:49:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11677.890983 secs 2024-04-26T21:49:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:56 smithi167 ceph-mon[32751]: pgmap v6945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:49:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:49:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:49:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:49:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:57 smithi090 ceph-mon[29180]: pgmap v6946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:49:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:57 smithi090 ceph-mon[29180]: pgmap v6947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:57 smithi119 ceph-mon[32752]: pgmap v6946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:49:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:57 smithi119 ceph-mon[32752]: pgmap v6947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:49:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:57 smithi167 ceph-mon[32751]: pgmap v6946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T21:49:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:57 smithi167 ceph-mon[32751]: pgmap v6947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:49:59 smithi090 ceph-mon[29180]: pgmap v6948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:49:59 smithi119 ceph-mon[32752]: pgmap v6948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:49:59 smithi167 ceph-mon[32751]: pgmap v6948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11682.155199 secs 2024-04-26T21:50:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:50:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11682.155199 secs 2024-04-26T21:50:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:50:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11682.155199 secs 2024-04-26T21:50:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T21:50:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:01 smithi090 ceph-mon[29180]: pgmap v6949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11682.891120 secs 2024-04-26T21:50:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:01 smithi119 ceph-mon[32752]: pgmap v6949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11682.891120 secs 2024-04-26T21:50:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:01 smithi167 ceph-mon[32751]: pgmap v6949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11682.891120 secs 2024-04-26T21:50:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:03 smithi090 ceph-mon[29180]: pgmap v6950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:50:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:03 smithi119 ceph-mon[32752]: pgmap v6950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:50:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:03 smithi167 ceph-mon[32751]: pgmap v6950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T21:50:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:05 smithi167 ceph-mon[32751]: pgmap v6951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11687.155284 secs 2024-04-26T21:50:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:05 smithi090 ceph-mon[29180]: pgmap v6951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11687.155284 secs 2024-04-26T21:50:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:05 smithi119 ceph-mon[32752]: pgmap v6951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:50:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11687.155284 secs 2024-04-26T21:50:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11687.891193 secs 2024-04-26T21:50:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11687.891193 secs 2024-04-26T21:50:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11687.891193 secs 2024-04-26T21:50:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:07 smithi090 ceph-mon[29180]: pgmap v6952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:50:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:07 smithi119 ceph-mon[32752]: pgmap v6952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:50:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:07 smithi167 ceph-mon[32751]: pgmap v6952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:50:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:09 smithi090 ceph-mon[29180]: pgmap v6953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:09 smithi119 ceph-mon[32752]: pgmap v6953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:09 smithi167 ceph-mon[32751]: pgmap v6953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11692.155368 secs 2024-04-26T21:50:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11692.155368 secs 2024-04-26T21:50:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11692.155368 secs 2024-04-26T21:50:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:11 smithi119 ceph-mon[32752]: pgmap v6954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11692.891312 secs 2024-04-26T21:50:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:11 smithi167 ceph-mon[32751]: pgmap v6954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11692.891312 secs 2024-04-26T21:50:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:11 smithi090 ceph-mon[29180]: pgmap v6954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11692.891312 secs 2024-04-26T21:50:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:13 smithi119 ceph-mon[32752]: pgmap v6955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:13 smithi167 ceph-mon[32751]: pgmap v6955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:13 smithi090 ceph-mon[29180]: pgmap v6955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:15 smithi119 ceph-mon[32752]: pgmap v6956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11697.155453 secs 2024-04-26T21:50:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:15 smithi167 ceph-mon[32751]: pgmap v6956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11697.155453 secs 2024-04-26T21:50:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:15 smithi090 ceph-mon[29180]: pgmap v6956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11697.155453 secs 2024-04-26T21:50:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11697.891437 secs 2024-04-26T21:50:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11697.891437 secs 2024-04-26T21:50:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11697.891437 secs 2024-04-26T21:50:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:17 smithi119 ceph-mon[32752]: pgmap v6957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:17 smithi167 ceph-mon[32751]: pgmap v6957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:17 smithi090 ceph-mon[29180]: pgmap v6957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:19 smithi119 ceph-mon[32752]: pgmap v6958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:19 smithi167 ceph-mon[32751]: pgmap v6958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:19 smithi090 ceph-mon[29180]: pgmap v6958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:21.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11702.155541 secs 2024-04-26T21:50:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11702.155541 secs 2024-04-26T21:50:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11702.155541 secs 2024-04-26T21:50:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:21 smithi119 ceph-mon[32752]: pgmap v6959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:22.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11702.891555 secs 2024-04-26T21:50:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:21 smithi167 ceph-mon[32751]: pgmap v6959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11702.891555 secs 2024-04-26T21:50:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:21 smithi090 ceph-mon[29180]: pgmap v6959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11702.891555 secs 2024-04-26T21:50:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:23 smithi119 ceph-mon[32752]: pgmap v6960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:23 smithi167 ceph-mon[32751]: pgmap v6960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:23 smithi090 ceph-mon[29180]: pgmap v6960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:25 smithi119 ceph-mon[32752]: pgmap v6961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11707.155625 secs 2024-04-26T21:50:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:25 smithi167 ceph-mon[32751]: pgmap v6961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11707.155625 secs 2024-04-26T21:50:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:25 smithi090 ceph-mon[29180]: pgmap v6961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11707.155625 secs 2024-04-26T21:50:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11707.891625 secs 2024-04-26T21:50:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11707.891625 secs 2024-04-26T21:50:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11707.891625 secs 2024-04-26T21:50:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:27 smithi119 ceph-mon[32752]: pgmap v6962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:27 smithi167 ceph-mon[32751]: pgmap v6962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:27 smithi090 ceph-mon[29180]: pgmap v6962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:29 smithi090 ceph-mon[29180]: pgmap v6963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:29 smithi119 ceph-mon[32752]: pgmap v6963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:29 smithi167 ceph-mon[32751]: pgmap v6963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:31.146 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11712.155712 secs 2024-04-26T21:50:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11712.155712 secs 2024-04-26T21:50:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11712.155712 secs 2024-04-26T21:50:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:31 smithi119 ceph-mon[32752]: pgmap v6964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11712.891704 secs 2024-04-26T21:50:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:31 smithi167 ceph-mon[32751]: pgmap v6964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11712.891704 secs 2024-04-26T21:50:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:31 smithi090 ceph-mon[29180]: pgmap v6964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11712.891704 secs 2024-04-26T21:50:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:33 smithi119 ceph-mon[32752]: pgmap v6965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:33 smithi167 ceph-mon[32751]: pgmap v6965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:33 smithi090 ceph-mon[29180]: pgmap v6965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:35 smithi119 ceph-mon[32752]: pgmap v6966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11717.155815 secs 2024-04-26T21:50:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:35 smithi167 ceph-mon[32751]: pgmap v6966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11717.155815 secs 2024-04-26T21:50:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:35 smithi090 ceph-mon[29180]: pgmap v6966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11717.155815 secs 2024-04-26T21:50:37.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11717.891812 secs 2024-04-26T21:50:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11717.891812 secs 2024-04-26T21:50:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11717.891812 secs 2024-04-26T21:50:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:37 smithi119 ceph-mon[32752]: pgmap v6967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:37 smithi167 ceph-mon[32751]: pgmap v6967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:37 smithi090 ceph-mon[29180]: pgmap v6967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:39 smithi119 ceph-mon[32752]: pgmap v6968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:39 smithi167 ceph-mon[32751]: pgmap v6968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:39 smithi090 ceph-mon[29180]: pgmap v6968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11722.155878 secs 2024-04-26T21:50:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11722.155878 secs 2024-04-26T21:50:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11722.155878 secs 2024-04-26T21:50:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:41 smithi119 ceph-mon[32752]: pgmap v6969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11722.891930 secs 2024-04-26T21:50:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:41 smithi167 ceph-mon[32751]: pgmap v6969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11722.891930 secs 2024-04-26T21:50:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:41 smithi090 ceph-mon[29180]: pgmap v6969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11722.891930 secs 2024-04-26T21:50:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:43 smithi119 ceph-mon[32752]: pgmap v6970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:43 smithi167 ceph-mon[32751]: pgmap v6970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:43 smithi090 ceph-mon[29180]: pgmap v6970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:45 smithi090 ceph-mon[29180]: pgmap v6971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11727.155940 secs 2024-04-26T21:50:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:45 smithi119 ceph-mon[32752]: pgmap v6971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11727.155940 secs 2024-04-26T21:50:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:45 smithi167 ceph-mon[32751]: pgmap v6971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11727.155940 secs 2024-04-26T21:50:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11727.892051 secs 2024-04-26T21:50:47.833 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11727.892051 secs 2024-04-26T21:50:47.833 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11727.892051 secs 2024-04-26T21:50:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:47 smithi119 ceph-mon[32752]: pgmap v6972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:47 smithi167 ceph-mon[32751]: pgmap v6972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:47 smithi090 ceph-mon[29180]: pgmap v6972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:49 smithi119 ceph-mon[32752]: pgmap v6973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:49 smithi167 ceph-mon[32751]: pgmap v6973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:49 smithi090 ceph-mon[29180]: pgmap v6973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11732.156000 secs 2024-04-26T21:50:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11732.156000 secs 2024-04-26T21:50:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11732.156000 secs 2024-04-26T21:50:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:51 smithi119 ceph-mon[32752]: pgmap v6974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11732.892164 secs 2024-04-26T21:50:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:51 smithi167 ceph-mon[32751]: pgmap v6974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11732.892164 secs 2024-04-26T21:50:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:51 smithi090 ceph-mon[29180]: pgmap v6974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11732.892164 secs 2024-04-26T21:50:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:53 smithi119 ceph-mon[32752]: pgmap v6975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:53 smithi167 ceph-mon[32751]: pgmap v6975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:53 smithi090 ceph-mon[29180]: pgmap v6975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:55 smithi119 ceph-mon[32752]: pgmap v6976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11737.156058 secs 2024-04-26T21:50:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:55 smithi167 ceph-mon[32751]: pgmap v6976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11737.156058 secs 2024-04-26T21:50:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:55 smithi090 ceph-mon[29180]: pgmap v6976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:50:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11737.156058 secs 2024-04-26T21:50:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11737.892309 secs 2024-04-26T21:50:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11737.892309 secs 2024-04-26T21:50:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11737.892309 secs 2024-04-26T21:50:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:50:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:57 smithi119 ceph-mon[32752]: pgmap v6977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:50:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:57 smithi167 ceph-mon[32751]: pgmap v6977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:50:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:57 smithi090 ceph-mon[29180]: pgmap v6977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:50:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:50:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:50:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:50:59.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:58 smithi119 ceph-mon[32752]: pgmap v6978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:50:59.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:50:58 smithi119 ceph-mon[32752]: pgmap v6979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:50:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:58 smithi167 ceph-mon[32751]: pgmap v6978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:50:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:50:58 smithi167 ceph-mon[32751]: pgmap v6979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:50:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:58 smithi090 ceph-mon[29180]: pgmap v6978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:50:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:50:58 smithi090 ceph-mon[29180]: pgmap v6979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:51:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:01 smithi119 ceph-mon[32752]: pgmap v6980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11742.156161 secs 2024-04-26T21:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:01 smithi167 ceph-mon[32751]: pgmap v6980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11742.156161 secs 2024-04-26T21:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:01 smithi090 ceph-mon[29180]: pgmap v6980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11742.156161 secs 2024-04-26T21:51:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:02 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11742.892426 secs 2024-04-26T21:51:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:02 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11742.892426 secs 2024-04-26T21:51:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:02 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11742.892426 secs 2024-04-26T21:51:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:03 smithi119 ceph-mon[32752]: pgmap v6981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:03 smithi167 ceph-mon[32751]: pgmap v6981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:03 smithi090 ceph-mon[29180]: pgmap v6981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:05 smithi119 ceph-mon[32752]: pgmap v6982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:51:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:05 smithi167 ceph-mon[32751]: pgmap v6982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:51:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:05 smithi090 ceph-mon[29180]: pgmap v6982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:51:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11747.156221 secs 2024-04-26T21:51:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11747.156221 secs 2024-04-26T21:51:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11747.156221 secs 2024-04-26T21:51:07.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:07 smithi090 ceph-mon[29180]: pgmap v6983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:07.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:07 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11747.892490 secs 2024-04-26T21:51:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:07 smithi119 ceph-mon[32752]: pgmap v6983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:07 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11747.892490 secs 2024-04-26T21:51:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:07 smithi167 ceph-mon[32751]: pgmap v6983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:51:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:07 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11747.892490 secs 2024-04-26T21:51:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:09 smithi119 ceph-mon[32752]: pgmap v6984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:51:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:09 smithi167 ceph-mon[32751]: pgmap v6984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:51:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:09 smithi090 ceph-mon[29180]: pgmap v6984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:51:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:11 smithi119 ceph-mon[32752]: pgmap v6985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11752.156284 secs 2024-04-26T21:51:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:11 smithi167 ceph-mon[32751]: pgmap v6985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11752.156284 secs 2024-04-26T21:51:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:11 smithi090 ceph-mon[29180]: pgmap v6985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11752.156284 secs 2024-04-26T21:51:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:12 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11752.892589 secs 2024-04-26T21:51:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:12 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11752.892589 secs 2024-04-26T21:51:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:12 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11752.892589 secs 2024-04-26T21:51:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:13 smithi119 ceph-mon[32752]: pgmap v6986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:13 smithi167 ceph-mon[32751]: pgmap v6986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:13 smithi090 ceph-mon[29180]: pgmap v6986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:15 smithi119 ceph-mon[32752]: pgmap v6987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:15 smithi167 ceph-mon[32751]: pgmap v6987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:15 smithi090 ceph-mon[29180]: pgmap v6987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11757.156368 secs 2024-04-26T21:51:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11757.156368 secs 2024-04-26T21:51:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11757.156368 secs 2024-04-26T21:51:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:17 smithi119 ceph-mon[32752]: pgmap v6988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:17 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11757.892718 secs 2024-04-26T21:51:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:17 smithi167 ceph-mon[32751]: pgmap v6988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:17 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11757.892718 secs 2024-04-26T21:51:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:17 smithi090 ceph-mon[29180]: pgmap v6988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:17 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11757.892718 secs 2024-04-26T21:51:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:18 smithi119 ceph-mon[32752]: pgmap v6989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:18.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:18 smithi090 ceph-mon[29180]: pgmap v6989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:18.504 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:18 smithi167 ceph-mon[32751]: pgmap v6989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:20 smithi090 ceph-mon[29180]: pgmap v6990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11762.156453 secs 2024-04-26T21:51:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:20 smithi119 ceph-mon[32752]: pgmap v6990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11762.156453 secs 2024-04-26T21:51:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:20 smithi167 ceph-mon[32751]: pgmap v6990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11762.156453 secs 2024-04-26T21:51:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11762.892864 secs 2024-04-26T21:51:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11762.892864 secs 2024-04-26T21:51:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11762.892864 secs 2024-04-26T21:51:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:22 smithi090 ceph-mon[29180]: pgmap v6991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:22 smithi119 ceph-mon[32752]: pgmap v6991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:22 smithi167 ceph-mon[32751]: pgmap v6991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:24 smithi090 ceph-mon[29180]: pgmap v6992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:24 smithi119 ceph-mon[32752]: pgmap v6992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:24 smithi167 ceph-mon[32751]: pgmap v6992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11767.156538 secs 2024-04-26T21:51:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11767.156538 secs 2024-04-26T21:51:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11767.156538 secs 2024-04-26T21:51:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:26 smithi090 ceph-mon[29180]: pgmap v6993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11767.892988 secs 2024-04-26T21:51:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:26 smithi119 ceph-mon[32752]: pgmap v6993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11767.892988 secs 2024-04-26T21:51:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:26 smithi167 ceph-mon[32751]: pgmap v6993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11767.892988 secs 2024-04-26T21:51:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:28 smithi090 ceph-mon[29180]: pgmap v6994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:28 smithi119 ceph-mon[32752]: pgmap v6994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:28 smithi167 ceph-mon[32751]: pgmap v6994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:30 smithi090 ceph-mon[29180]: pgmap v6995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11772.156622 secs 2024-04-26T21:51:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:30 smithi119 ceph-mon[32752]: pgmap v6995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11772.156622 secs 2024-04-26T21:51:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:30 smithi167 ceph-mon[32751]: pgmap v6995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11772.156622 secs 2024-04-26T21:51:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11772.893086 secs 2024-04-26T21:51:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11772.893086 secs 2024-04-26T21:51:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11772.893086 secs 2024-04-26T21:51:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:32 smithi119 ceph-mon[32752]: pgmap v6996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:32 smithi167 ceph-mon[32751]: pgmap v6996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:32 smithi090 ceph-mon[29180]: pgmap v6996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:34 smithi119 ceph-mon[32752]: pgmap v6997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:34 smithi167 ceph-mon[32751]: pgmap v6997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:34 smithi090 ceph-mon[29180]: pgmap v6997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11777.156707 secs 2024-04-26T21:51:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11777.156707 secs 2024-04-26T21:51:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11777.156707 secs 2024-04-26T21:51:36.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:36 smithi167 ceph-mon[32751]: pgmap v6998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:36.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11777.893210 secs 2024-04-26T21:51:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:36 smithi119 ceph-mon[32752]: pgmap v6998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11777.893210 secs 2024-04-26T21:51:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:36 smithi090 ceph-mon[29180]: pgmap v6998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11777.893210 secs 2024-04-26T21:51:38.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:38 smithi090 ceph-mon[29180]: pgmap v6999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:38 smithi119 ceph-mon[32752]: pgmap v6999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:38 smithi167 ceph-mon[32751]: pgmap v6999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:40 smithi119 ceph-mon[32752]: pgmap v7000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11782.156813 secs 2024-04-26T21:51:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:40 smithi167 ceph-mon[32751]: pgmap v7000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11782.156813 secs 2024-04-26T21:51:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:40 smithi090 ceph-mon[29180]: pgmap v7000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11782.156813 secs 2024-04-26T21:51:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11782.893349 secs 2024-04-26T21:51:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11782.893349 secs 2024-04-26T21:51:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11782.893349 secs 2024-04-26T21:51:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:42 smithi119 ceph-mon[32752]: pgmap v7001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:42 smithi167 ceph-mon[32751]: pgmap v7001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:42 smithi090 ceph-mon[29180]: pgmap v7001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:44 smithi119 ceph-mon[32752]: pgmap v7002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:44 smithi167 ceph-mon[32751]: pgmap v7002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:44 smithi090 ceph-mon[29180]: pgmap v7002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11787.156897 secs 2024-04-26T21:51:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11787.156897 secs 2024-04-26T21:51:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11787.156897 secs 2024-04-26T21:51:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:46 smithi119 ceph-mon[32752]: pgmap v7003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11787.893469 secs 2024-04-26T21:51:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:46 smithi167 ceph-mon[32751]: pgmap v7003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11787.893469 secs 2024-04-26T21:51:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:46 smithi090 ceph-mon[29180]: pgmap v7003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11787.893469 secs 2024-04-26T21:51:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:48 smithi119 ceph-mon[32752]: pgmap v7004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:48 smithi167 ceph-mon[32751]: pgmap v7004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:48 smithi090 ceph-mon[29180]: pgmap v7004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:50 smithi119 ceph-mon[32752]: pgmap v7005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11792.156960 secs 2024-04-26T21:51:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:50 smithi167 ceph-mon[32751]: pgmap v7005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11792.156960 secs 2024-04-26T21:51:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:50 smithi090 ceph-mon[29180]: pgmap v7005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11792.156960 secs 2024-04-26T21:51:51.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11792.893616 secs 2024-04-26T21:51:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11792.893616 secs 2024-04-26T21:51:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11792.893616 secs 2024-04-26T21:51:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:52 smithi119 ceph-mon[32752]: pgmap v7006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:52 smithi167 ceph-mon[32751]: pgmap v7006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:52 smithi090 ceph-mon[29180]: pgmap v7006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:54 smithi119 ceph-mon[32752]: pgmap v7007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:54 smithi167 ceph-mon[32751]: pgmap v7007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:54 smithi090 ceph-mon[29180]: pgmap v7007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:51:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11797.157046 secs 2024-04-26T21:51:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11797.157046 secs 2024-04-26T21:51:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11797.157046 secs 2024-04-26T21:51:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:56 smithi119 ceph-mon[32752]: pgmap v7008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11797.893758 secs 2024-04-26T21:51:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:56 smithi167 ceph-mon[32751]: pgmap v7008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11797.893758 secs 2024-04-26T21:51:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:56 smithi090 ceph-mon[29180]: pgmap v7008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:51:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11797.893758 secs 2024-04-26T21:51:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:51:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:58 smithi119 ceph-mon[32752]: pgmap v7009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:51:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:58 smithi167 ceph-mon[32751]: pgmap v7009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:51:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:58 smithi090 ceph-mon[29180]: pgmap v7009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:51:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:51:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:51:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:51:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:59 smithi119 ceph-mon[32752]: pgmap v7010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:51:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:51:59 smithi119 ceph-mon[32752]: pgmap v7011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:51:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:59 smithi167 ceph-mon[32751]: pgmap v7010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:51:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:51:59 smithi167 ceph-mon[32751]: pgmap v7011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:51:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:59 smithi090 ceph-mon[29180]: pgmap v7010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:51:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:51:59 smithi090 ceph-mon[29180]: pgmap v7011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:52:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11802.157131 secs 2024-04-26T21:52:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11802.157131 secs 2024-04-26T21:52:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11802.157131 secs 2024-04-26T21:52:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:01 smithi119 ceph-mon[32752]: pgmap v7012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11802.893837 secs 2024-04-26T21:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:01 smithi167 ceph-mon[32751]: pgmap v7012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11802.893837 secs 2024-04-26T21:52:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:01 smithi090 ceph-mon[29180]: pgmap v7012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11802.893837 secs 2024-04-26T21:52:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:03 smithi119 ceph-mon[32752]: pgmap v7013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:52:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:03 smithi167 ceph-mon[32751]: pgmap v7013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:52:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:03 smithi090 ceph-mon[29180]: pgmap v7013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T21:52:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:05 smithi119 ceph-mon[32752]: pgmap v7014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11807.157192 secs 2024-04-26T21:52:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:05 smithi167 ceph-mon[32751]: pgmap v7014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11807.157192 secs 2024-04-26T21:52:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:05 smithi090 ceph-mon[29180]: pgmap v7014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11807.157192 secs 2024-04-26T21:52:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11807.893977 secs 2024-04-26T21:52:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11807.893977 secs 2024-04-26T21:52:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11807.893977 secs 2024-04-26T21:52:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:07 smithi119 ceph-mon[32752]: pgmap v7015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:07 smithi167 ceph-mon[32751]: pgmap v7015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:07 smithi090 ceph-mon[29180]: pgmap v7015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:09 smithi119 ceph-mon[32752]: pgmap v7016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:52:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:09 smithi167 ceph-mon[32751]: pgmap v7016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:52:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:09 smithi090 ceph-mon[29180]: pgmap v7016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:52:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11812.157276 secs 2024-04-26T21:52:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11812.157276 secs 2024-04-26T21:52:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11812.157276 secs 2024-04-26T21:52:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:11 smithi119 ceph-mon[32752]: pgmap v7017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11812.894116 secs 2024-04-26T21:52:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:11 smithi167 ceph-mon[32751]: pgmap v7017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11812.894116 secs 2024-04-26T21:52:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:11 smithi090 ceph-mon[29180]: pgmap v7017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11812.894116 secs 2024-04-26T21:52:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:13 smithi119 ceph-mon[32752]: pgmap v7018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:13 smithi167 ceph-mon[32751]: pgmap v7018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:13 smithi090 ceph-mon[29180]: pgmap v7018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:15 smithi119 ceph-mon[32752]: pgmap v7019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11817.157335 secs 2024-04-26T21:52:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:15 smithi167 ceph-mon[32751]: pgmap v7019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11817.157335 secs 2024-04-26T21:52:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:15 smithi090 ceph-mon[29180]: pgmap v7019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11817.157335 secs 2024-04-26T21:52:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11817.894250 secs 2024-04-26T21:52:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11817.894250 secs 2024-04-26T21:52:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11817.894250 secs 2024-04-26T21:52:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:17 smithi119 ceph-mon[32752]: pgmap v7020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:17 smithi167 ceph-mon[32751]: pgmap v7020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:17 smithi090 ceph-mon[29180]: pgmap v7020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:19 smithi119 ceph-mon[32752]: pgmap v7021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:19 smithi167 ceph-mon[32751]: pgmap v7021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:19 smithi090 ceph-mon[29180]: pgmap v7021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11822.157419 secs 2024-04-26T21:52:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11822.157419 secs 2024-04-26T21:52:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11822.157419 secs 2024-04-26T21:52:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:21 smithi119 ceph-mon[32752]: pgmap v7022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11822.894396 secs 2024-04-26T21:52:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:21 smithi167 ceph-mon[32751]: pgmap v7022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11822.894396 secs 2024-04-26T21:52:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:21 smithi090 ceph-mon[29180]: pgmap v7022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11822.894396 secs 2024-04-26T21:52:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:23 smithi119 ceph-mon[32752]: pgmap v7023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:23 smithi167 ceph-mon[32751]: pgmap v7023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:23 smithi090 ceph-mon[29180]: pgmap v7023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:25 smithi119 ceph-mon[32752]: pgmap v7024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11827.157503 secs 2024-04-26T21:52:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:25 smithi167 ceph-mon[32751]: pgmap v7024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11827.157503 secs 2024-04-26T21:52:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:25 smithi090 ceph-mon[29180]: pgmap v7024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11827.157503 secs 2024-04-26T21:52:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11827.894460 secs 2024-04-26T21:52:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11827.894460 secs 2024-04-26T21:52:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11827.894460 secs 2024-04-26T21:52:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:27 smithi119 ceph-mon[32752]: pgmap v7025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:27 smithi167 ceph-mon[32751]: pgmap v7025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:27 smithi090 ceph-mon[29180]: pgmap v7025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:29 smithi119 ceph-mon[32752]: pgmap v7026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:29 smithi167 ceph-mon[32751]: pgmap v7026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:29 smithi090 ceph-mon[29180]: pgmap v7026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11832.157587 secs 2024-04-26T21:52:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11832.157587 secs 2024-04-26T21:52:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11832.157587 secs 2024-04-26T21:52:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:31 smithi119 ceph-mon[32752]: pgmap v7027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11832.894575 secs 2024-04-26T21:52:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:31 smithi167 ceph-mon[32751]: pgmap v7027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11832.894575 secs 2024-04-26T21:52:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:31 smithi090 ceph-mon[29180]: pgmap v7027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11832.894575 secs 2024-04-26T21:52:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:33 smithi119 ceph-mon[32752]: pgmap v7028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:33 smithi167 ceph-mon[32751]: pgmap v7028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:33 smithi090 ceph-mon[29180]: pgmap v7028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:35 smithi119 ceph-mon[32752]: pgmap v7029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11837.157671 secs 2024-04-26T21:52:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:35 smithi167 ceph-mon[32751]: pgmap v7029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11837.157671 secs 2024-04-26T21:52:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:35 smithi090 ceph-mon[29180]: pgmap v7029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11837.157671 secs 2024-04-26T21:52:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11837.894655 secs 2024-04-26T21:52:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11837.894655 secs 2024-04-26T21:52:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11837.894655 secs 2024-04-26T21:52:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:37 smithi167 ceph-mon[32751]: pgmap v7030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:37 smithi090 ceph-mon[29180]: pgmap v7030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:37 smithi119 ceph-mon[32752]: pgmap v7030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:39 smithi167 ceph-mon[32751]: pgmap v7031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:39 smithi090 ceph-mon[29180]: pgmap v7031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:39 smithi119 ceph-mon[32752]: pgmap v7031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11842.157760 secs 2024-04-26T21:52:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11842.157760 secs 2024-04-26T21:52:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11842.157760 secs 2024-04-26T21:52:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:41 smithi167 ceph-mon[32751]: pgmap v7032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11842.894792 secs 2024-04-26T21:52:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:41 smithi090 ceph-mon[29180]: pgmap v7032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11842.894792 secs 2024-04-26T21:52:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:41 smithi119 ceph-mon[32752]: pgmap v7032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11842.894792 secs 2024-04-26T21:52:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:43 smithi090 ceph-mon[29180]: pgmap v7033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:43 smithi119 ceph-mon[32752]: pgmap v7033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:43 smithi167 ceph-mon[32751]: pgmap v7033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:45 smithi090 ceph-mon[29180]: pgmap v7034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11847.157849 secs 2024-04-26T21:52:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:45 smithi119 ceph-mon[32752]: pgmap v7034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11847.157849 secs 2024-04-26T21:52:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:45 smithi167 ceph-mon[32751]: pgmap v7034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11847.157849 secs 2024-04-26T21:52:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11847.894906 secs 2024-04-26T21:52:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11847.894906 secs 2024-04-26T21:52:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11847.894906 secs 2024-04-26T21:52:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:47 smithi090 ceph-mon[29180]: pgmap v7035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:47 smithi119 ceph-mon[32752]: pgmap v7035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:47 smithi167 ceph-mon[32751]: pgmap v7035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:49 smithi090 ceph-mon[29180]: pgmap v7036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:49 smithi119 ceph-mon[32752]: pgmap v7036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:49 smithi167 ceph-mon[32751]: pgmap v7036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11852.157939 secs 2024-04-26T21:52:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11852.157939 secs 2024-04-26T21:52:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11852.157939 secs 2024-04-26T21:52:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:51 smithi090 ceph-mon[29180]: pgmap v7037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11852.895046 secs 2024-04-26T21:52:51.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:51 smithi119 ceph-mon[32752]: pgmap v7037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:51.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11852.895046 secs 2024-04-26T21:52:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:51 smithi167 ceph-mon[32751]: pgmap v7037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11852.895046 secs 2024-04-26T21:52:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:53 smithi090 ceph-mon[29180]: pgmap v7038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:53 smithi119 ceph-mon[32752]: pgmap v7038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:53 smithi167 ceph-mon[32751]: pgmap v7038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:55 smithi090 ceph-mon[29180]: pgmap v7039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11857.158022 secs 2024-04-26T21:52:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:55 smithi119 ceph-mon[32752]: pgmap v7039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11857.158022 secs 2024-04-26T21:52:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:55 smithi167 ceph-mon[32751]: pgmap v7039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:52:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11857.158022 secs 2024-04-26T21:52:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11857.895158 secs 2024-04-26T21:52:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11857.895158 secs 2024-04-26T21:52:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11857.895158 secs 2024-04-26T21:52:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:57 smithi090 ceph-mon[29180]: pgmap v7040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:57 smithi119 ceph-mon[32752]: pgmap v7040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:57 smithi167 ceph-mon[32751]: pgmap v7040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:52:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:59 smithi090 ceph-mon[29180]: pgmap v7041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:52:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:52:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:52:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:59 smithi090 ceph-mon[29180]: pgmap v7042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:52:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:59 smithi090 ceph-mon[29180]: pgmap v7043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:52:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:52:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:53:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:59 smithi119 ceph-mon[32752]: pgmap v7041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:59 smithi119 ceph-mon[32752]: pgmap v7042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:53:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:59 smithi119 ceph-mon[32752]: pgmap v7043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:52:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:53:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:59 smithi167 ceph-mon[32751]: pgmap v7041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:53:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:53:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:59 smithi167 ceph-mon[32751]: pgmap v7042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T21:53:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:59 smithi167 ceph-mon[32751]: pgmap v7043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:52:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:53:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11862.158108 secs 2024-04-26T21:53:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11862.158108 secs 2024-04-26T21:53:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11862.158108 secs 2024-04-26T21:53:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11862.895222 secs 2024-04-26T21:53:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:01 smithi119 ceph-mon[32752]: pgmap v7044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11862.895222 secs 2024-04-26T21:53:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:01 smithi167 ceph-mon[32751]: pgmap v7044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11862.895222 secs 2024-04-26T21:53:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:01 smithi090 ceph-mon[29180]: pgmap v7044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:03 smithi119 ceph-mon[32752]: pgmap v7045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:53:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:03 smithi167 ceph-mon[32751]: pgmap v7045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:53:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:03 smithi090 ceph-mon[29180]: pgmap v7045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:53:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:05 smithi119 ceph-mon[32752]: pgmap v7046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11867.158166 secs 2024-04-26T21:53:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:05 smithi167 ceph-mon[32751]: pgmap v7046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11867.158166 secs 2024-04-26T21:53:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:05 smithi090 ceph-mon[29180]: pgmap v7046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11867.158166 secs 2024-04-26T21:53:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11867.895315 secs 2024-04-26T21:53:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11867.895315 secs 2024-04-26T21:53:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11867.895315 secs 2024-04-26T21:53:08.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:07 smithi119 ceph-mon[32752]: pgmap v7047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:07 smithi167 ceph-mon[32751]: pgmap v7047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:07 smithi090 ceph-mon[29180]: pgmap v7047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:53:10.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:09 smithi090 ceph-mon[29180]: pgmap v7048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:53:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:09 smithi119 ceph-mon[32752]: pgmap v7048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:53:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:09 smithi167 ceph-mon[32751]: pgmap v7048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:53:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11872.158251 secs 2024-04-26T21:53:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11872.158251 secs 2024-04-26T21:53:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11872.158251 secs 2024-04-26T21:53:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11872.895416 secs 2024-04-26T21:53:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:11 smithi119 ceph-mon[32752]: pgmap v7049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11872.895416 secs 2024-04-26T21:53:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:11 smithi167 ceph-mon[32751]: pgmap v7049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11872.895416 secs 2024-04-26T21:53:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:11 smithi090 ceph-mon[29180]: pgmap v7049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:13 smithi119 ceph-mon[32752]: pgmap v7050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:13 smithi167 ceph-mon[32751]: pgmap v7050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:13 smithi090 ceph-mon[29180]: pgmap v7050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:15 smithi119 ceph-mon[32752]: pgmap v7051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11877.158332 secs 2024-04-26T21:53:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:15 smithi167 ceph-mon[32751]: pgmap v7051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11877.158332 secs 2024-04-26T21:53:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:15 smithi090 ceph-mon[29180]: pgmap v7051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11877.158332 secs 2024-04-26T21:53:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11877.895525 secs 2024-04-26T21:53:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11877.895525 secs 2024-04-26T21:53:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11877.895525 secs 2024-04-26T21:53:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:17 smithi119 ceph-mon[32752]: pgmap v7052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:17 smithi167 ceph-mon[32751]: pgmap v7052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:17 smithi090 ceph-mon[29180]: pgmap v7052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:19 smithi119 ceph-mon[32752]: pgmap v7053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:19 smithi167 ceph-mon[32751]: pgmap v7053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:19 smithi090 ceph-mon[29180]: pgmap v7053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11882.158415 secs 2024-04-26T21:53:21.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11882.158415 secs 2024-04-26T21:53:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11882.158415 secs 2024-04-26T21:53:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11882.895635 secs 2024-04-26T21:53:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:21 smithi119 ceph-mon[32752]: pgmap v7054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11882.895635 secs 2024-04-26T21:53:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:21 smithi167 ceph-mon[32751]: pgmap v7054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11882.895635 secs 2024-04-26T21:53:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:21 smithi090 ceph-mon[29180]: pgmap v7054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:23 smithi119 ceph-mon[32752]: pgmap v7055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:23 smithi167 ceph-mon[32751]: pgmap v7055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:23 smithi090 ceph-mon[29180]: pgmap v7055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:25 smithi119 ceph-mon[32752]: pgmap v7056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11887.158497 secs 2024-04-26T21:53:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:25 smithi167 ceph-mon[32751]: pgmap v7056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11887.158497 secs 2024-04-26T21:53:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:25 smithi090 ceph-mon[29180]: pgmap v7056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11887.158497 secs 2024-04-26T21:53:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11887.895753 secs 2024-04-26T21:53:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11887.895753 secs 2024-04-26T21:53:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11887.895753 secs 2024-04-26T21:53:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:27 smithi119 ceph-mon[32752]: pgmap v7057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:27 smithi167 ceph-mon[32751]: pgmap v7057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:27 smithi090 ceph-mon[29180]: pgmap v7057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:29 smithi119 ceph-mon[32752]: pgmap v7058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:29 smithi167 ceph-mon[32751]: pgmap v7058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:29 smithi090 ceph-mon[29180]: pgmap v7058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11892.158582 secs 2024-04-26T21:53:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11892.158582 secs 2024-04-26T21:53:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11892.158582 secs 2024-04-26T21:53:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11892.895870 secs 2024-04-26T21:53:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:31 smithi119 ceph-mon[32752]: pgmap v7059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11892.895870 secs 2024-04-26T21:53:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:31 smithi167 ceph-mon[32751]: pgmap v7059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11892.895870 secs 2024-04-26T21:53:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:31 smithi090 ceph-mon[29180]: pgmap v7059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:33 smithi119 ceph-mon[32752]: pgmap v7060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:33 smithi167 ceph-mon[32751]: pgmap v7060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:33 smithi090 ceph-mon[29180]: pgmap v7060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:34.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:53:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:53:34.001+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:53:34.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:53:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:53:34.001+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:53:34.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:53:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:53:34.001+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:53:34.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:53:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:53:34.001+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:53:34.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:53:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:53:34.001+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:53:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:35 smithi119 ceph-mon[32752]: pgmap v7061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11897.158664 secs 2024-04-26T21:53:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:35 smithi167 ceph-mon[32751]: pgmap v7061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11897.158664 secs 2024-04-26T21:53:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:35 smithi090 ceph-mon[29180]: pgmap v7061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11897.158664 secs 2024-04-26T21:53:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:36 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 11897.895980 secs 2024-04-26T21:53:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:36 smithi119 ceph-mon[32752]: slow request 11524.764993 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:53:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:36 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 11897.895980 secs 2024-04-26T21:53:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:36 smithi167 ceph-mon[32751]: slow request 11524.764993 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:53:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:36 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 11897.895980 secs 2024-04-26T21:53:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:36 smithi090 ceph-mon[29180]: slow request 11524.764993 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:53:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:37 smithi119 ceph-mon[32752]: pgmap v7062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:37 smithi167 ceph-mon[32751]: pgmap v7062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:37 smithi090 ceph-mon[29180]: pgmap v7062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:40 smithi119 ceph-mon[32752]: pgmap v7063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:40 smithi167 ceph-mon[32751]: pgmap v7063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:40 smithi090 ceph-mon[29180]: pgmap v7063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11902.158747 secs 2024-04-26T21:53:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11902.158747 secs 2024-04-26T21:53:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11902.158747 secs 2024-04-26T21:53:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:42 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11902.896086 secs 2024-04-26T21:53:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:42 smithi119 ceph-mon[32752]: pgmap v7064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:42 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11902.896086 secs 2024-04-26T21:53:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:42 smithi167 ceph-mon[32751]: pgmap v7064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:42 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11902.896086 secs 2024-04-26T21:53:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:42 smithi090 ceph-mon[29180]: pgmap v7064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:44 smithi119 ceph-mon[32752]: pgmap v7065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:44 smithi167 ceph-mon[32751]: pgmap v7065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:44 smithi090 ceph-mon[29180]: pgmap v7065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:46 smithi119 ceph-mon[32752]: pgmap v7066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:46 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 11907.158849 secs 2024-04-26T21:53:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:46 smithi119 ceph-mon[32752]: slow request 723.457566 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:53:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:46 smithi167 ceph-mon[32751]: pgmap v7066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:46 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 11907.158849 secs 2024-04-26T21:53:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:46 smithi167 ceph-mon[32751]: slow request 723.457566 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:46 smithi090 ceph-mon[29180]: pgmap v7066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:46 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 11907.158849 secs 2024-04-26T21:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:46 smithi090 ceph-mon[29180]: slow request 723.457566 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:53:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:47 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11907.896180 secs 2024-04-26T21:53:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:47 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11907.896180 secs 2024-04-26T21:53:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:47 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11907.896180 secs 2024-04-26T21:53:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:48 smithi119 ceph-mon[32752]: pgmap v7067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:48 smithi167 ceph-mon[32751]: pgmap v7067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:48 smithi090 ceph-mon[29180]: pgmap v7067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T21:53:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:50 smithi119 ceph-mon[32752]: pgmap v7068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:50 smithi167 ceph-mon[32751]: pgmap v7068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:50 smithi090 ceph-mon[29180]: pgmap v7068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T21:53:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11912.158939 secs 2024-04-26T21:53:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11912.158939 secs 2024-04-26T21:53:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11912.158939 secs 2024-04-26T21:53:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:52 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11912.896313 secs 2024-04-26T21:53:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:52 smithi119 ceph-mon[32752]: pgmap v7069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:52 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11912.896313 secs 2024-04-26T21:53:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:52 smithi167 ceph-mon[32751]: pgmap v7069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:52 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11912.896313 secs 2024-04-26T21:53:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:52 smithi090 ceph-mon[29180]: pgmap v7069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:54 smithi119 ceph-mon[32752]: pgmap v7070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:54 smithi167 ceph-mon[32751]: pgmap v7070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:54 smithi090 ceph-mon[29180]: pgmap v7070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:56 smithi119 ceph-mon[32752]: pgmap v7071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11917.159029 secs 2024-04-26T21:53:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:56 smithi167 ceph-mon[32751]: pgmap v7071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11917.159029 secs 2024-04-26T21:53:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:56 smithi090 ceph-mon[29180]: pgmap v7071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:53:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11917.159029 secs 2024-04-26T21:53:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:57 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11917.896375 secs 2024-04-26T21:53:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:57 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11917.896375 secs 2024-04-26T21:53:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:57 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11917.896375 secs 2024-04-26T21:53:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:58 smithi119 ceph-mon[32752]: pgmap v7072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:58 smithi167 ceph-mon[32751]: pgmap v7072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:58 smithi090 ceph-mon[29180]: pgmap v7072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:53:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:53:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:53:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:53:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:53:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:54:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:00 smithi167 ceph-mon[32751]: pgmap v7073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:54:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:00 smithi090 ceph-mon[29180]: pgmap v7073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:00.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:54:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:00 smithi119 ceph-mon[32752]: pgmap v7073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:54:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:54:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:54:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:01 smithi167 ceph-mon[32751]: pgmap v7074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:54:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:01 smithi167 ceph-mon[32751]: pgmap v7075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:54:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11922.159114 secs 2024-04-26T21:54:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:01 smithi090 ceph-mon[29180]: pgmap v7074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:54:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:01 smithi090 ceph-mon[29180]: pgmap v7075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:54:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11922.159114 secs 2024-04-26T21:54:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:01 smithi119 ceph-mon[32752]: pgmap v7074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T21:54:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:01 smithi119 ceph-mon[32752]: pgmap v7075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:54:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11922.159114 secs 2024-04-26T21:54:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:02 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11922.896467 secs 2024-04-26T21:54:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:02 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11922.896467 secs 2024-04-26T21:54:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:02 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11922.896467 secs 2024-04-26T21:54:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:03 smithi167 ceph-mon[32751]: pgmap v7076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:54:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:03 smithi090 ceph-mon[29180]: pgmap v7076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:54:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:03 smithi119 ceph-mon[32752]: pgmap v7076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:54:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:04 smithi090 ceph-mon[29180]: pgmap v7077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:04 smithi119 ceph-mon[32752]: pgmap v7077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:04 smithi167 ceph-mon[32751]: pgmap v7077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11927.159196 secs 2024-04-26T21:54:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11927.159196 secs 2024-04-26T21:54:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11927.159196 secs 2024-04-26T21:54:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:06 smithi119 ceph-mon[32752]: pgmap v7078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11927.896586 secs 2024-04-26T21:54:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:06 smithi167 ceph-mon[32751]: pgmap v7078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11927.896586 secs 2024-04-26T21:54:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:06 smithi090 ceph-mon[29180]: pgmap v7078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11927.896586 secs 2024-04-26T21:54:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:08 smithi119 ceph-mon[32752]: pgmap v7079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:54:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:08 smithi167 ceph-mon[32751]: pgmap v7079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:54:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:08 smithi090 ceph-mon[29180]: pgmap v7079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:54:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:10 smithi119 ceph-mon[32752]: pgmap v7080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11932.159285 secs 2024-04-26T21:54:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:10 smithi167 ceph-mon[32751]: pgmap v7080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11932.159285 secs 2024-04-26T21:54:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:10 smithi090 ceph-mon[29180]: pgmap v7080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11932.159285 secs 2024-04-26T21:54:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11932.896701 secs 2024-04-26T21:54:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11932.896701 secs 2024-04-26T21:54:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11932.896701 secs 2024-04-26T21:54:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:12 smithi119 ceph-mon[32752]: pgmap v7081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:54:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:12 smithi167 ceph-mon[32751]: pgmap v7081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:54:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:12 smithi090 ceph-mon[29180]: pgmap v7081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T21:54:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:14 smithi119 ceph-mon[32752]: pgmap v7082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:14 smithi167 ceph-mon[32751]: pgmap v7082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:14 smithi090 ceph-mon[29180]: pgmap v7082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T21:54:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11937.159369 secs 2024-04-26T21:54:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11937.159369 secs 2024-04-26T21:54:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11937.159369 secs 2024-04-26T21:54:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:16 smithi119 ceph-mon[32752]: pgmap v7083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11937.896821 secs 2024-04-26T21:54:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:16 smithi167 ceph-mon[32751]: pgmap v7083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11937.896821 secs 2024-04-26T21:54:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:16 smithi090 ceph-mon[29180]: pgmap v7083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11937.896821 secs 2024-04-26T21:54:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:18 smithi119 ceph-mon[32752]: pgmap v7084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:18 smithi167 ceph-mon[32751]: pgmap v7084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:18 smithi090 ceph-mon[29180]: pgmap v7084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:20 smithi119 ceph-mon[32752]: pgmap v7085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11942.159454 secs 2024-04-26T21:54:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:20 smithi167 ceph-mon[32751]: pgmap v7085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11942.159454 secs 2024-04-26T21:54:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:20 smithi090 ceph-mon[29180]: pgmap v7085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11942.159454 secs 2024-04-26T21:54:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11942.896936 secs 2024-04-26T21:54:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11942.896936 secs 2024-04-26T21:54:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11942.896936 secs 2024-04-26T21:54:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:22 smithi119 ceph-mon[32752]: pgmap v7086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:22 smithi167 ceph-mon[32751]: pgmap v7086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:22 smithi090 ceph-mon[29180]: pgmap v7086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:24 smithi119 ceph-mon[32752]: pgmap v7087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:24 smithi167 ceph-mon[32751]: pgmap v7087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:24 smithi090 ceph-mon[29180]: pgmap v7087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11947.159537 secs 2024-04-26T21:54:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11947.159537 secs 2024-04-26T21:54:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11947.159537 secs 2024-04-26T21:54:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:26 smithi119 ceph-mon[32752]: pgmap v7088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11947.897074 secs 2024-04-26T21:54:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:26 smithi167 ceph-mon[32751]: pgmap v7088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11947.897074 secs 2024-04-26T21:54:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:26 smithi090 ceph-mon[29180]: pgmap v7088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11947.897074 secs 2024-04-26T21:54:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:28 smithi119 ceph-mon[32752]: pgmap v7089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:28 smithi167 ceph-mon[32751]: pgmap v7089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:28 smithi090 ceph-mon[29180]: pgmap v7089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:30 smithi119 ceph-mon[32752]: pgmap v7090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11952.159621 secs 2024-04-26T21:54:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:30 smithi167 ceph-mon[32751]: pgmap v7090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11952.159621 secs 2024-04-26T21:54:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:30 smithi090 ceph-mon[29180]: pgmap v7090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11952.159621 secs 2024-04-26T21:54:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11952.897217 secs 2024-04-26T21:54:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11952.897217 secs 2024-04-26T21:54:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11952.897217 secs 2024-04-26T21:54:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:32 smithi119 ceph-mon[32752]: pgmap v7091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:32 smithi167 ceph-mon[32751]: pgmap v7091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:32 smithi090 ceph-mon[29180]: pgmap v7091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:34 smithi119 ceph-mon[32752]: pgmap v7092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:34 smithi167 ceph-mon[32751]: pgmap v7092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:34 smithi090 ceph-mon[29180]: pgmap v7092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11957.159708 secs 2024-04-26T21:54:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11957.159708 secs 2024-04-26T21:54:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11957.159708 secs 2024-04-26T21:54:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11957.897328 secs 2024-04-26T21:54:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:36 smithi119 ceph-mon[32752]: pgmap v7093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11957.897328 secs 2024-04-26T21:54:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:36 smithi167 ceph-mon[32751]: pgmap v7093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11957.897328 secs 2024-04-26T21:54:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:36 smithi090 ceph-mon[29180]: pgmap v7093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:38 smithi119 ceph-mon[32752]: pgmap v7094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:38 smithi167 ceph-mon[32751]: pgmap v7094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:38 smithi090 ceph-mon[29180]: pgmap v7094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:40 smithi119 ceph-mon[32752]: pgmap v7095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11962.159818 secs 2024-04-26T21:54:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:40 smithi167 ceph-mon[32751]: pgmap v7095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11962.159818 secs 2024-04-26T21:54:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:40 smithi090 ceph-mon[29180]: pgmap v7095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11962.159818 secs 2024-04-26T21:54:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11962.897455 secs 2024-04-26T21:54:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11962.897455 secs 2024-04-26T21:54:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11962.897455 secs 2024-04-26T21:54:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:42 smithi119 ceph-mon[32752]: pgmap v7096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:42 smithi167 ceph-mon[32751]: pgmap v7096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:42 smithi090 ceph-mon[29180]: pgmap v7096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:44 smithi119 ceph-mon[32752]: pgmap v7097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:44 smithi167 ceph-mon[32751]: pgmap v7097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:44 smithi090 ceph-mon[29180]: pgmap v7097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11967.159903 secs 2024-04-26T21:54:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11967.159903 secs 2024-04-26T21:54:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11967.159903 secs 2024-04-26T21:54:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11967.897535 secs 2024-04-26T21:54:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:46 smithi119 ceph-mon[32752]: pgmap v7098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11967.897535 secs 2024-04-26T21:54:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:46 smithi167 ceph-mon[32751]: pgmap v7098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11967.897535 secs 2024-04-26T21:54:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:46 smithi090 ceph-mon[29180]: pgmap v7098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:48 smithi119 ceph-mon[32752]: pgmap v7099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:48 smithi167 ceph-mon[32751]: pgmap v7099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:48 smithi090 ceph-mon[29180]: pgmap v7099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:50 smithi119 ceph-mon[32752]: pgmap v7100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11972.159992 secs 2024-04-26T21:54:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:50 smithi167 ceph-mon[32751]: pgmap v7100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11972.159992 secs 2024-04-26T21:54:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:50 smithi090 ceph-mon[29180]: pgmap v7100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11972.159992 secs 2024-04-26T21:54:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11972.897611 secs 2024-04-26T21:54:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11972.897611 secs 2024-04-26T21:54:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11972.897611 secs 2024-04-26T21:54:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:52 smithi119 ceph-mon[32752]: pgmap v7101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:52 smithi167 ceph-mon[32751]: pgmap v7101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:52 smithi090 ceph-mon[29180]: pgmap v7101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:54 smithi090 ceph-mon[29180]: pgmap v7102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:54 smithi119 ceph-mon[32752]: pgmap v7102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:54 smithi167 ceph-mon[32751]: pgmap v7102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:54:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11977.160076 secs 2024-04-26T21:54:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11977.160076 secs 2024-04-26T21:54:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11977.160076 secs 2024-04-26T21:54:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11977.897679 secs 2024-04-26T21:54:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:56 smithi167 ceph-mon[32751]: pgmap v7103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11977.897679 secs 2024-04-26T21:54:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:56 smithi090 ceph-mon[29180]: pgmap v7103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11977.897679 secs 2024-04-26T21:54:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:56 smithi119 ceph-mon[32752]: pgmap v7103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:58 smithi167 ceph-mon[32751]: pgmap v7104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:58 smithi090 ceph-mon[29180]: pgmap v7104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:54:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:58 smithi119 ceph-mon[32752]: pgmap v7104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:54:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:54:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:54:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:55:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:00 smithi090 ceph-mon[29180]: pgmap v7105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11982.160145 secs 2024-04-26T21:55:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:55:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:00 smithi119 ceph-mon[32752]: pgmap v7105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11982.160145 secs 2024-04-26T21:55:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:55:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:00 smithi167 ceph-mon[32751]: pgmap v7105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11982.160145 secs 2024-04-26T21:55:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:55:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:55:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:55:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:01 smithi090 ceph-mon[29180]: pgmap v7106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:55:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:01 smithi090 ceph-mon[29180]: pgmap v7107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11982.897769 secs 2024-04-26T21:55:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:01 smithi119 ceph-mon[32752]: pgmap v7106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:55:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:01 smithi119 ceph-mon[32752]: pgmap v7107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11982.897769 secs 2024-04-26T21:55:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:01 smithi167 ceph-mon[32751]: pgmap v7106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:55:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:01 smithi167 ceph-mon[32751]: pgmap v7107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11982.897769 secs 2024-04-26T21:55:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:03 smithi090 ceph-mon[29180]: pgmap v7108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:55:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:03 smithi119 ceph-mon[32752]: pgmap v7108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:55:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:03 smithi167 ceph-mon[32751]: pgmap v7108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:55:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:05 smithi167 ceph-mon[32751]: pgmap v7109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:06.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11987.160227 secs 2024-04-26T21:55:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:05 smithi090 ceph-mon[29180]: pgmap v7109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11987.160227 secs 2024-04-26T21:55:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:05 smithi119 ceph-mon[32752]: pgmap v7109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11987.160227 secs 2024-04-26T21:55:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11987.897898 secs 2024-04-26T21:55:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11987.897898 secs 2024-04-26T21:55:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11987.897898 secs 2024-04-26T21:55:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:07 smithi090 ceph-mon[29180]: pgmap v7110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:07 smithi119 ceph-mon[32752]: pgmap v7110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:07 smithi167 ceph-mon[32751]: pgmap v7110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:09 smithi090 ceph-mon[29180]: pgmap v7111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:09 smithi119 ceph-mon[32752]: pgmap v7111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:09 smithi167 ceph-mon[32751]: pgmap v7111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:55:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11992.160310 secs 2024-04-26T21:55:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11992.160310 secs 2024-04-26T21:55:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11992.160310 secs 2024-04-26T21:55:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:11 smithi090 ceph-mon[29180]: pgmap v7112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:55:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11992.898036 secs 2024-04-26T21:55:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:11 smithi119 ceph-mon[32752]: pgmap v7112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:55:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11992.898036 secs 2024-04-26T21:55:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:11 smithi167 ceph-mon[32751]: pgmap v7112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:55:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11992.898036 secs 2024-04-26T21:55:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:13 smithi090 ceph-mon[29180]: pgmap v7113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:13 smithi119 ceph-mon[32752]: pgmap v7113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:13 smithi167 ceph-mon[32751]: pgmap v7113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:15 smithi090 ceph-mon[29180]: pgmap v7114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 11997.160395 secs 2024-04-26T21:55:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:15 smithi119 ceph-mon[32752]: pgmap v7114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 11997.160395 secs 2024-04-26T21:55:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:15 smithi167 ceph-mon[32751]: pgmap v7114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 11997.160395 secs 2024-04-26T21:55:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 11997.898159 secs 2024-04-26T21:55:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 11997.898159 secs 2024-04-26T21:55:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 11997.898159 secs 2024-04-26T21:55:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:17 smithi090 ceph-mon[29180]: pgmap v7115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:18.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:17 smithi119 ceph-mon[32752]: pgmap v7115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:17 smithi167 ceph-mon[32751]: pgmap v7115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:19 smithi090 ceph-mon[29180]: pgmap v7116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:19 smithi119 ceph-mon[32752]: pgmap v7116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:19 smithi167 ceph-mon[32751]: pgmap v7116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12002.160479 secs 2024-04-26T21:55:21.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12002.160479 secs 2024-04-26T21:55:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12002.160479 secs 2024-04-26T21:55:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:21 smithi090 ceph-mon[29180]: pgmap v7117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12002.898298 secs 2024-04-26T21:55:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:21 smithi119 ceph-mon[32752]: pgmap v7117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12002.898298 secs 2024-04-26T21:55:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:21 smithi167 ceph-mon[32751]: pgmap v7117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12002.898298 secs 2024-04-26T21:55:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:23 smithi090 ceph-mon[29180]: pgmap v7118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:23 smithi119 ceph-mon[32752]: pgmap v7118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:23 smithi167 ceph-mon[32751]: pgmap v7118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:25 smithi119 ceph-mon[32752]: pgmap v7119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12007.160562 secs 2024-04-26T21:55:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:25 smithi167 ceph-mon[32751]: pgmap v7119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12007.160562 secs 2024-04-26T21:55:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:25 smithi090 ceph-mon[29180]: pgmap v7119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12007.160562 secs 2024-04-26T21:55:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12007.898407 secs 2024-04-26T21:55:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12007.898407 secs 2024-04-26T21:55:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12007.898407 secs 2024-04-26T21:55:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:27 smithi119 ceph-mon[32752]: pgmap v7120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:27 smithi167 ceph-mon[32751]: pgmap v7120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:27 smithi090 ceph-mon[29180]: pgmap v7120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:29 smithi119 ceph-mon[32752]: pgmap v7121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:29 smithi167 ceph-mon[32751]: pgmap v7121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:29 smithi090 ceph-mon[29180]: pgmap v7121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12012.160644 secs 2024-04-26T21:55:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12012.160644 secs 2024-04-26T21:55:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12012.160644 secs 2024-04-26T21:55:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:31 smithi119 ceph-mon[32752]: pgmap v7122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12012.898517 secs 2024-04-26T21:55:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:31 smithi167 ceph-mon[32751]: pgmap v7122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12012.898517 secs 2024-04-26T21:55:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:31 smithi090 ceph-mon[29180]: pgmap v7122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12012.898517 secs 2024-04-26T21:55:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:33 smithi119 ceph-mon[32752]: pgmap v7123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:33 smithi167 ceph-mon[32751]: pgmap v7123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:33 smithi090 ceph-mon[29180]: pgmap v7123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:35 smithi119 ceph-mon[32752]: pgmap v7124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12017.160728 secs 2024-04-26T21:55:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:35 smithi167 ceph-mon[32751]: pgmap v7124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12017.160728 secs 2024-04-26T21:55:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:35 smithi090 ceph-mon[29180]: pgmap v7124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12017.160728 secs 2024-04-26T21:55:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12017.898627 secs 2024-04-26T21:55:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12017.898627 secs 2024-04-26T21:55:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12017.898627 secs 2024-04-26T21:55:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:37 smithi119 ceph-mon[32752]: pgmap v7125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:37 smithi167 ceph-mon[32751]: pgmap v7125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:37 smithi090 ceph-mon[29180]: pgmap v7125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:40.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:39 smithi090 ceph-mon[29180]: pgmap v7126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:39 smithi119 ceph-mon[32752]: pgmap v7126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:39 smithi167 ceph-mon[32751]: pgmap v7126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12022.160812 secs 2024-04-26T21:55:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12022.160812 secs 2024-04-26T21:55:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12022.160812 secs 2024-04-26T21:55:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:41 smithi119 ceph-mon[32752]: pgmap v7127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12022.898690 secs 2024-04-26T21:55:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:41 smithi167 ceph-mon[32751]: pgmap v7127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12022.898690 secs 2024-04-26T21:55:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:41 smithi090 ceph-mon[29180]: pgmap v7127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12022.898690 secs 2024-04-26T21:55:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:43 smithi119 ceph-mon[32752]: pgmap v7128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:43 smithi167 ceph-mon[32751]: pgmap v7128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:43 smithi090 ceph-mon[29180]: pgmap v7128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:45 smithi119 ceph-mon[32752]: pgmap v7129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12027.160872 secs 2024-04-26T21:55:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:45 smithi167 ceph-mon[32751]: pgmap v7129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12027.160872 secs 2024-04-26T21:55:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:45 smithi090 ceph-mon[29180]: pgmap v7129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12027.160872 secs 2024-04-26T21:55:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12027.898808 secs 2024-04-26T21:55:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12027.898808 secs 2024-04-26T21:55:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12027.898808 secs 2024-04-26T21:55:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:47 smithi119 ceph-mon[32752]: pgmap v7130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:47 smithi167 ceph-mon[32751]: pgmap v7130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:47 smithi090 ceph-mon[29180]: pgmap v7130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:55:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:49 smithi119 ceph-mon[32752]: pgmap v7131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:49 smithi167 ceph-mon[32751]: pgmap v7131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:49 smithi090 ceph-mon[29180]: pgmap v7131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:51.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12032.160933 secs 2024-04-26T21:55:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12032.160933 secs 2024-04-26T21:55:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12032.160933 secs 2024-04-26T21:55:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:51 smithi119 ceph-mon[32752]: pgmap v7132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12032.898906 secs 2024-04-26T21:55:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:51 smithi167 ceph-mon[32751]: pgmap v7132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12032.898906 secs 2024-04-26T21:55:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:51 smithi090 ceph-mon[29180]: pgmap v7132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12032.898906 secs 2024-04-26T21:55:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:53 smithi119 ceph-mon[32752]: pgmap v7133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:53 smithi167 ceph-mon[32751]: pgmap v7133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:53 smithi090 ceph-mon[29180]: pgmap v7133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:55 smithi119 ceph-mon[32752]: pgmap v7134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12037.161014 secs 2024-04-26T21:55:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:55 smithi167 ceph-mon[32751]: pgmap v7134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12037.161014 secs 2024-04-26T21:55:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:55 smithi090 ceph-mon[29180]: pgmap v7134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:55:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12037.161014 secs 2024-04-26T21:55:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12037.899056 secs 2024-04-26T21:55:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12037.899056 secs 2024-04-26T21:55:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12037.899056 secs 2024-04-26T21:55:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:57 smithi119 ceph-mon[32752]: pgmap v7135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:57 smithi167 ceph-mon[32751]: pgmap v7135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:55:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:57 smithi090 ceph-mon[29180]: pgmap v7135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T21:56:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:55:59 smithi119 ceph-mon[32752]: pgmap v7136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:55:59 smithi167 ceph-mon[32751]: pgmap v7136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:55:59 smithi090 ceph-mon[29180]: pgmap v7136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12042.161076 secs 2024-04-26T21:56:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12042.161076 secs 2024-04-26T21:56:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12042.161076 secs 2024-04-26T21:56:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:56:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:01 smithi119 ceph-mon[32752]: pgmap v7137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12042.899150 secs 2024-04-26T21:56:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:56:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:01 smithi167 ceph-mon[32751]: pgmap v7137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12042.899150 secs 2024-04-26T21:56:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:56:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:01 smithi090 ceph-mon[29180]: pgmap v7137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12042.899150 secs 2024-04-26T21:56:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:56:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:56:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:56:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:02 smithi119 ceph-mon[32752]: pgmap v7138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:56:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:02 smithi119 ceph-mon[32752]: pgmap v7139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:02 smithi167 ceph-mon[32751]: pgmap v7138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:56:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:02 smithi167 ceph-mon[32751]: pgmap v7139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:02 smithi090 ceph-mon[29180]: pgmap v7138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:56:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:02 smithi090 ceph-mon[29180]: pgmap v7139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:04 smithi119 ceph-mon[32752]: pgmap v7140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:56:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:04 smithi167 ceph-mon[32751]: pgmap v7140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:56:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:04 smithi090 ceph-mon[29180]: pgmap v7140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:56:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12047.161138 secs 2024-04-26T21:56:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12047.161138 secs 2024-04-26T21:56:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12047.161138 secs 2024-04-26T21:56:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:06 smithi119 ceph-mon[32752]: pgmap v7141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12047.899282 secs 2024-04-26T21:56:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:06 smithi167 ceph-mon[32751]: pgmap v7141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12047.899282 secs 2024-04-26T21:56:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:06 smithi090 ceph-mon[29180]: pgmap v7141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12047.899282 secs 2024-04-26T21:56:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:08 smithi119 ceph-mon[32752]: pgmap v7142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:56:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:08 smithi167 ceph-mon[32751]: pgmap v7142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:56:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:08 smithi090 ceph-mon[29180]: pgmap v7142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:56:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:11 smithi119 ceph-mon[32752]: pgmap v7143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12052.161199 secs 2024-04-26T21:56:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:11 smithi167 ceph-mon[32751]: pgmap v7143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12052.161199 secs 2024-04-26T21:56:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:11 smithi090 ceph-mon[29180]: pgmap v7143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:56:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12052.161199 secs 2024-04-26T21:56:12.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:12 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12052.899414 secs 2024-04-26T21:56:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:12 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12052.899414 secs 2024-04-26T21:56:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:12 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12052.899414 secs 2024-04-26T21:56:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:13 smithi119 ceph-mon[32752]: pgmap v7144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:56:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:13 smithi167 ceph-mon[32751]: pgmap v7144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:56:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:13 smithi090 ceph-mon[29180]: pgmap v7144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:56:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:15 smithi119 ceph-mon[32752]: pgmap v7145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:15 smithi167 ceph-mon[32751]: pgmap v7145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:15 smithi090 ceph-mon[29180]: pgmap v7145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12057.161255 secs 2024-04-26T21:56:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12057.161255 secs 2024-04-26T21:56:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12057.161255 secs 2024-04-26T21:56:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:17 smithi119 ceph-mon[32752]: pgmap v7146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:17 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12057.899542 secs 2024-04-26T21:56:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:17 smithi167 ceph-mon[32751]: pgmap v7146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:17 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12057.899542 secs 2024-04-26T21:56:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:17 smithi090 ceph-mon[29180]: pgmap v7146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:17 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12057.899542 secs 2024-04-26T21:56:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:19 smithi119 ceph-mon[32752]: pgmap v7147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:19 smithi167 ceph-mon[32751]: pgmap v7147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:19 smithi090 ceph-mon[29180]: pgmap v7147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:21 smithi119 ceph-mon[32752]: pgmap v7148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12062.161357 secs 2024-04-26T21:56:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:21 smithi167 ceph-mon[32751]: pgmap v7148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12062.161357 secs 2024-04-26T21:56:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:21 smithi090 ceph-mon[29180]: pgmap v7148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12062.161357 secs 2024-04-26T21:56:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:22 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12062.899654 secs 2024-04-26T21:56:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:22 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12062.899654 secs 2024-04-26T21:56:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:22 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12062.899654 secs 2024-04-26T21:56:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:23 smithi119 ceph-mon[32752]: pgmap v7149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:23 smithi167 ceph-mon[32751]: pgmap v7149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:23 smithi090 ceph-mon[29180]: pgmap v7149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:25 smithi119 ceph-mon[32752]: pgmap v7150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:25 smithi167 ceph-mon[32751]: pgmap v7150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:25 smithi090 ceph-mon[29180]: pgmap v7150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12067.161440 secs 2024-04-26T21:56:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12067.161440 secs 2024-04-26T21:56:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12067.161440 secs 2024-04-26T21:56:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:27 smithi119 ceph-mon[32752]: pgmap v7151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:27 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12067.899778 secs 2024-04-26T21:56:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:27 smithi167 ceph-mon[32751]: pgmap v7151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:27 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12067.899778 secs 2024-04-26T21:56:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:27 smithi090 ceph-mon[29180]: pgmap v7151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:27 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12067.899778 secs 2024-04-26T21:56:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:29 smithi119 ceph-mon[32752]: pgmap v7152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:29 smithi167 ceph-mon[32751]: pgmap v7152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:29 smithi090 ceph-mon[29180]: pgmap v7152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:31 smithi119 ceph-mon[32752]: pgmap v7153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12072.161523 secs 2024-04-26T21:56:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:31 smithi167 ceph-mon[32751]: pgmap v7153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12072.161523 secs 2024-04-26T21:56:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:31 smithi090 ceph-mon[29180]: pgmap v7153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12072.161523 secs 2024-04-26T21:56:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:32 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 12072.899842 secs 2024-04-26T21:56:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:32 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 12072.899842 secs 2024-04-26T21:56:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:32 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 12072.899842 secs 2024-04-26T21:56:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:33 smithi167 ceph-mon[32751]: pgmap v7154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:33 smithi090 ceph-mon[29180]: pgmap v7154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:33 smithi119 ceph-mon[32752]: pgmap v7154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:35 smithi167 ceph-mon[32751]: pgmap v7155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:35 smithi090 ceph-mon[29180]: pgmap v7155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:35 smithi119 ceph-mon[32752]: pgmap v7155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12077.161604 secs 2024-04-26T21:56:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12077.161604 secs 2024-04-26T21:56:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12077.161604 secs 2024-04-26T21:56:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:37 smithi090 ceph-mon[29180]: pgmap v7156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:37 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 12077.899935 secs 2024-04-26T21:56:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:37 smithi090 ceph-mon[29180]: slow request 181.640068 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:56:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:37 smithi119 ceph-mon[32752]: pgmap v7156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:37 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 12077.899935 secs 2024-04-26T21:56:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:37 smithi119 ceph-mon[32752]: slow request 181.640068 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:56:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:37 smithi167 ceph-mon[32751]: pgmap v7156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:37 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 12077.899935 secs 2024-04-26T21:56:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:37 smithi167 ceph-mon[32751]: slow request 181.640068 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T21:56:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:39 smithi090 ceph-mon[29180]: pgmap v7157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:39 smithi119 ceph-mon[32752]: pgmap v7157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:39 smithi167 ceph-mon[32751]: pgmap v7157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:40 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T21:56:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:40 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:56:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:40 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T21:56:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:40 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:56:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:40 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T21:56:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:40 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T21:56:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:41 smithi090 ceph-mon[29180]: pgmap v7158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12082.161690 secs 2024-04-26T21:56:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:41 smithi119 ceph-mon[32752]: pgmap v7158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12082.161690 secs 2024-04-26T21:56:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:41 smithi167 ceph-mon[32751]: pgmap v7158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12082.161690 secs 2024-04-26T21:56:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12082.900057 secs 2024-04-26T21:56:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12082.900057 secs 2024-04-26T21:56:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12082.900057 secs 2024-04-26T21:56:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:43 smithi090 ceph-mon[29180]: pgmap v7159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:43 smithi119 ceph-mon[32752]: pgmap v7159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:43 smithi167 ceph-mon[32751]: pgmap v7159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:44 smithi090 ceph-mon[29180]: pgmap v7160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:44 smithi119 ceph-mon[32752]: pgmap v7160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:44 smithi167 ceph-mon[32751]: pgmap v7160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12087.161775 secs 2024-04-26T21:56:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12087.161775 secs 2024-04-26T21:56:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12087.161775 secs 2024-04-26T21:56:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:46 smithi167 ceph-mon[32751]: pgmap v7161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12087.900191 secs 2024-04-26T21:56:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:46 smithi090 ceph-mon[29180]: pgmap v7161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12087.900191 secs 2024-04-26T21:56:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:46 smithi119 ceph-mon[32752]: pgmap v7161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12087.900191 secs 2024-04-26T21:56:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:48 smithi167 ceph-mon[32751]: pgmap v7162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:48 smithi090 ceph-mon[29180]: pgmap v7162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:48 smithi119 ceph-mon[32752]: pgmap v7162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:50 smithi167 ceph-mon[32751]: pgmap v7163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12092.161837 secs 2024-04-26T21:56:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:50 smithi090 ceph-mon[29180]: pgmap v7163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12092.161837 secs 2024-04-26T21:56:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:50 smithi119 ceph-mon[32752]: pgmap v7163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12092.161837 secs 2024-04-26T21:56:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12092.900328 secs 2024-04-26T21:56:51.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12092.900328 secs 2024-04-26T21:56:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12092.900328 secs 2024-04-26T21:56:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:52 smithi090 ceph-mon[29180]: pgmap v7164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:52 smithi119 ceph-mon[32752]: pgmap v7164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:52 smithi167 ceph-mon[32751]: pgmap v7164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:54 smithi090 ceph-mon[29180]: pgmap v7165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:54 smithi119 ceph-mon[32752]: pgmap v7165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:54 smithi167 ceph-mon[32751]: pgmap v7165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12097.161897 secs 2024-04-26T21:56:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12097.161897 secs 2024-04-26T21:56:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12097.161897 secs 2024-04-26T21:56:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:56 smithi090 ceph-mon[29180]: pgmap v7166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12097.900470 secs 2024-04-26T21:56:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:56 smithi119 ceph-mon[32752]: pgmap v7166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12097.900470 secs 2024-04-26T21:56:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:56 smithi167 ceph-mon[32751]: pgmap v7166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:56:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12097.900470 secs 2024-04-26T21:56:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:56:58 smithi090 ceph-mon[29180]: pgmap v7167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:56:58 smithi119 ceph-mon[32752]: pgmap v7167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:56:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:56:58 smithi167 ceph-mon[32751]: pgmap v7167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:00 smithi090 ceph-mon[29180]: pgmap v7168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12102.161960 secs 2024-04-26T21:57:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:00 smithi119 ceph-mon[32752]: pgmap v7168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12102.161960 secs 2024-04-26T21:57:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:00 smithi167 ceph-mon[32751]: pgmap v7168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12102.161960 secs 2024-04-26T21:57:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12102.900591 secs 2024-04-26T21:57:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12102.900591 secs 2024-04-26T21:57:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12102.900591 secs 2024-04-26T21:57:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:57:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:02 smithi090 ceph-mon[29180]: pgmap v7169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:02 smithi119 ceph-mon[32752]: pgmap v7169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:02 smithi167 ceph-mon[32751]: pgmap v7169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: pgmap v7170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:57:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:57:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:57:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: pgmap v7170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:57:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: pgmap v7170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T21:57:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T21:57:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T21:57:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:57:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:57:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: pgmap v7171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:57:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: pgmap v7172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12107.162044 secs 2024-04-26T21:57:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:57:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:57:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:57:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:57:05.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:05.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:05.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: pgmap v7171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:57:05.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: pgmap v7172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:05.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:05.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12107.162044 secs 2024-04-26T21:57:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T21:57:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T21:57:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T21:57:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T21:57:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T21:57:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:57:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:57:05.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: pgmap v7171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T21:57:05.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: pgmap v7172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:05.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:57:05.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12107.162044 secs 2024-04-26T21:57:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12107.900669 secs 2024-04-26T21:57:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12107.900669 secs 2024-04-26T21:57:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12107.900669 secs 2024-04-26T21:57:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:07 smithi119 ceph-mon[32752]: pgmap v7173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:07 smithi167 ceph-mon[32751]: pgmap v7173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:07 smithi090 ceph-mon[29180]: pgmap v7173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:09 smithi119 ceph-mon[32752]: pgmap v7174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:09 smithi167 ceph-mon[32751]: pgmap v7174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:09 smithi090 ceph-mon[29180]: pgmap v7174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:11.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12112.162129 secs 2024-04-26T21:57:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12112.162129 secs 2024-04-26T21:57:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12112.162129 secs 2024-04-26T21:57:12.046 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12112.900742 secs 2024-04-26T21:57:12.046 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:11 smithi090 ceph-mon[29180]: pgmap v7175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12112.900742 secs 2024-04-26T21:57:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:11 smithi119 ceph-mon[32752]: pgmap v7175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12112.900742 secs 2024-04-26T21:57:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:11 smithi167 ceph-mon[32751]: pgmap v7175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:57:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:13 smithi119 ceph-mon[32752]: pgmap v7176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:57:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:13 smithi167 ceph-mon[32751]: pgmap v7176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:57:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:13 smithi090 ceph-mon[29180]: pgmap v7176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T21:57:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:15 smithi119 ceph-mon[32752]: pgmap v7177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:57:16.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12117.162212 secs 2024-04-26T21:57:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:15 smithi167 ceph-mon[32751]: pgmap v7177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:57:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12117.162212 secs 2024-04-26T21:57:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:15 smithi090 ceph-mon[29180]: pgmap v7177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:57:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12117.162212 secs 2024-04-26T21:57:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12117.900859 secs 2024-04-26T21:57:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12117.900859 secs 2024-04-26T21:57:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12117.900859 secs 2024-04-26T21:57:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:17 smithi119 ceph-mon[32752]: pgmap v7178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:17 smithi167 ceph-mon[32751]: pgmap v7178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:17 smithi090 ceph-mon[29180]: pgmap v7178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:19 smithi119 ceph-mon[32752]: pgmap v7179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:19 smithi167 ceph-mon[32751]: pgmap v7179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:19 smithi090 ceph-mon[29180]: pgmap v7179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12122.162296 secs 2024-04-26T21:57:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12122.162296 secs 2024-04-26T21:57:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12122.162296 secs 2024-04-26T21:57:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12122.900992 secs 2024-04-26T21:57:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:21 smithi119 ceph-mon[32752]: pgmap v7180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12122.900992 secs 2024-04-26T21:57:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:21 smithi167 ceph-mon[32751]: pgmap v7180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12122.900992 secs 2024-04-26T21:57:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:21 smithi090 ceph-mon[29180]: pgmap v7180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:23 smithi119 ceph-mon[32752]: pgmap v7181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:23 smithi167 ceph-mon[32751]: pgmap v7181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:23 smithi090 ceph-mon[29180]: pgmap v7181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:25 smithi119 ceph-mon[32752]: pgmap v7182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12127.162381 secs 2024-04-26T21:57:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:25 smithi167 ceph-mon[32751]: pgmap v7182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12127.162381 secs 2024-04-26T21:57:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:25 smithi090 ceph-mon[29180]: pgmap v7182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12127.162381 secs 2024-04-26T21:57:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12127.901128 secs 2024-04-26T21:57:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12127.901128 secs 2024-04-26T21:57:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12127.901128 secs 2024-04-26T21:57:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:27 smithi119 ceph-mon[32752]: pgmap v7183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:27 smithi167 ceph-mon[32751]: pgmap v7183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:27 smithi090 ceph-mon[29180]: pgmap v7183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:29 smithi119 ceph-mon[32752]: pgmap v7184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:29 smithi167 ceph-mon[32751]: pgmap v7184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:29 smithi090 ceph-mon[29180]: pgmap v7184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12132.162466 secs 2024-04-26T21:57:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12132.162466 secs 2024-04-26T21:57:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12132.162466 secs 2024-04-26T21:57:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12132.901245 secs 2024-04-26T21:57:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:31 smithi119 ceph-mon[32752]: pgmap v7185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12132.901245 secs 2024-04-26T21:57:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:31 smithi167 ceph-mon[32751]: pgmap v7185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12132.901245 secs 2024-04-26T21:57:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:31 smithi090 ceph-mon[29180]: pgmap v7185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:33 smithi119 ceph-mon[32752]: pgmap v7186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:33 smithi167 ceph-mon[32751]: pgmap v7186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:33 smithi090 ceph-mon[29180]: pgmap v7186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:35 smithi119 ceph-mon[32752]: pgmap v7187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12137.162550 secs 2024-04-26T21:57:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:35 smithi167 ceph-mon[32751]: pgmap v7187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12137.162550 secs 2024-04-26T21:57:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:35 smithi090 ceph-mon[29180]: pgmap v7187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12137.162550 secs 2024-04-26T21:57:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12137.901393 secs 2024-04-26T21:57:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12137.901393 secs 2024-04-26T21:57:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12137.901393 secs 2024-04-26T21:57:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:37 smithi119 ceph-mon[32752]: pgmap v7188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:37 smithi167 ceph-mon[32751]: pgmap v7188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:37 smithi090 ceph-mon[29180]: pgmap v7188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:39 smithi167 ceph-mon[32751]: pgmap v7189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:39 smithi090 ceph-mon[29180]: pgmap v7189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:39 smithi119 ceph-mon[32752]: pgmap v7189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12142.162634 secs 2024-04-26T21:57:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12142.162634 secs 2024-04-26T21:57:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12142.162634 secs 2024-04-26T21:57:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12142.901519 secs 2024-04-26T21:57:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:41 smithi167 ceph-mon[32751]: pgmap v7190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12142.901519 secs 2024-04-26T21:57:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:41 smithi090 ceph-mon[29180]: pgmap v7190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12142.901519 secs 2024-04-26T21:57:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:41 smithi119 ceph-mon[32752]: pgmap v7190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:43 smithi090 ceph-mon[29180]: pgmap v7191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:43 smithi119 ceph-mon[32752]: pgmap v7191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:43 smithi167 ceph-mon[32751]: pgmap v7191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:45 smithi090 ceph-mon[29180]: pgmap v7192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12147.162695 secs 2024-04-26T21:57:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:45 smithi119 ceph-mon[32752]: pgmap v7192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12147.162695 secs 2024-04-26T21:57:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:45 smithi167 ceph-mon[32751]: pgmap v7192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12147.162695 secs 2024-04-26T21:57:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12147.901654 secs 2024-04-26T21:57:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12147.901654 secs 2024-04-26T21:57:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12147.901654 secs 2024-04-26T21:57:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:47 smithi090 ceph-mon[29180]: pgmap v7193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:47 smithi119 ceph-mon[32752]: pgmap v7193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:47 smithi167 ceph-mon[32751]: pgmap v7193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:49 smithi090 ceph-mon[29180]: pgmap v7194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:49 smithi119 ceph-mon[32752]: pgmap v7194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:49 smithi167 ceph-mon[32751]: pgmap v7194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12152.162779 secs 2024-04-26T21:57:51.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12152.162779 secs 2024-04-26T21:57:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12152.162779 secs 2024-04-26T21:57:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12152.901771 secs 2024-04-26T21:57:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:51 smithi090 ceph-mon[29180]: pgmap v7195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12152.901771 secs 2024-04-26T21:57:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:51 smithi119 ceph-mon[32752]: pgmap v7195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12152.901771 secs 2024-04-26T21:57:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:51 smithi167 ceph-mon[32751]: pgmap v7195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:53 smithi090 ceph-mon[29180]: pgmap v7196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:53 smithi119 ceph-mon[32752]: pgmap v7196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:53 smithi167 ceph-mon[32751]: pgmap v7196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:55 smithi090 ceph-mon[29180]: pgmap v7197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12157.162830 secs 2024-04-26T21:57:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:55 smithi119 ceph-mon[32752]: pgmap v7197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12157.162830 secs 2024-04-26T21:57:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:55 smithi167 ceph-mon[32751]: pgmap v7197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:57:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12157.162830 secs 2024-04-26T21:57:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12157.901880 secs 2024-04-26T21:57:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12157.901880 secs 2024-04-26T21:57:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12157.901880 secs 2024-04-26T21:57:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:57 smithi090 ceph-mon[29180]: pgmap v7198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:57 smithi119 ceph-mon[32752]: pgmap v7198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:57:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:57 smithi167 ceph-mon[32751]: pgmap v7198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:57:59 smithi090 ceph-mon[29180]: pgmap v7199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:57:59 smithi119 ceph-mon[32752]: pgmap v7199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:57:59 smithi167 ceph-mon[32751]: pgmap v7199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12162.162891 secs 2024-04-26T21:58:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12162.162891 secs 2024-04-26T21:58:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12162.162891 secs 2024-04-26T21:58:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12162.902001 secs 2024-04-26T21:58:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:01 smithi090 ceph-mon[29180]: pgmap v7200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12162.902001 secs 2024-04-26T21:58:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:01 smithi119 ceph-mon[32752]: pgmap v7200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12162.902001 secs 2024-04-26T21:58:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:01 smithi167 ceph-mon[32751]: pgmap v7200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:03 smithi090 ceph-mon[29180]: pgmap v7201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:03 smithi119 ceph-mon[32752]: pgmap v7201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:03 smithi167 ceph-mon[32751]: pgmap v7201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:58:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:05 smithi090 ceph-mon[29180]: pgmap v7202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12167.162955 secs 2024-04-26T21:58:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:06.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:05 smithi167 ceph-mon[32751]: pgmap v7202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:06.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12167.162955 secs 2024-04-26T21:58:06.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:06.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:05 smithi119 ceph-mon[32752]: pgmap v7202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12167.162955 secs 2024-04-26T21:58:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12167.902144 secs 2024-04-26T21:58:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:06 smithi119 ceph-mon[32752]: pgmap v7203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 904 B/s rd, 1 op/s 2024-04-26T21:58:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:06 smithi119 ceph-mon[32752]: pgmap v7204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12167.902144 secs 2024-04-26T21:58:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:06 smithi167 ceph-mon[32751]: pgmap v7203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 904 B/s rd, 1 op/s 2024-04-26T21:58:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:06 smithi167 ceph-mon[32751]: pgmap v7204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12167.902144 secs 2024-04-26T21:58:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:58:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:58:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:06 smithi090 ceph-mon[29180]: pgmap v7203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 904 B/s rd, 1 op/s 2024-04-26T21:58:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:06 smithi090 ceph-mon[29180]: pgmap v7204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:58:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:09 smithi090 ceph-mon[29180]: pgmap v7205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-26T21:58:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:09 smithi119 ceph-mon[32752]: pgmap v7205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-26T21:58:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:09 smithi167 ceph-mon[32751]: pgmap v7205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-26T21:58:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12172.163038 secs 2024-04-26T21:58:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:11 smithi090 ceph-mon[29180]: pgmap v7206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12172.163038 secs 2024-04-26T21:58:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:11 smithi119 ceph-mon[32752]: pgmap v7206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12172.163038 secs 2024-04-26T21:58:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:11 smithi167 ceph-mon[32751]: pgmap v7206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12172.902260 secs 2024-04-26T21:58:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12172.902260 secs 2024-04-26T21:58:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12172.902260 secs 2024-04-26T21:58:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:13 smithi090 ceph-mon[29180]: pgmap v7207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:13 smithi119 ceph-mon[32752]: pgmap v7207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:13 smithi167 ceph-mon[32751]: pgmap v7207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:15 smithi090 ceph-mon[29180]: pgmap v7208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:15 smithi119 ceph-mon[32752]: pgmap v7208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:15 smithi167 ceph-mon[32751]: pgmap v7208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:58:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12177.163068 secs 2024-04-26T21:58:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12177.163068 secs 2024-04-26T21:58:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12177.163068 secs 2024-04-26T21:58:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12177.902393 secs 2024-04-26T21:58:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:17 smithi167 ceph-mon[32751]: pgmap v7209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:58:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12177.902393 secs 2024-04-26T21:58:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:17 smithi090 ceph-mon[29180]: pgmap v7209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:58:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12177.902393 secs 2024-04-26T21:58:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:17 smithi119 ceph-mon[32752]: pgmap v7209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:58:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:19 smithi090 ceph-mon[29180]: pgmap v7210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:19 smithi119 ceph-mon[32752]: pgmap v7210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:19 smithi167 ceph-mon[32751]: pgmap v7210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:21.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12182.163152 secs 2024-04-26T21:58:21.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:21 smithi119 ceph-mon[32752]: pgmap v7211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12182.163152 secs 2024-04-26T21:58:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:21 smithi090 ceph-mon[29180]: pgmap v7211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12182.163152 secs 2024-04-26T21:58:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:21 smithi167 ceph-mon[32751]: pgmap v7211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12182.902462 secs 2024-04-26T21:58:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:22 smithi090 ceph-mon[29180]: pgmap v7212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12182.902462 secs 2024-04-26T21:58:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:22 smithi119 ceph-mon[32752]: pgmap v7212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12182.902462 secs 2024-04-26T21:58:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:22 smithi167 ceph-mon[32751]: pgmap v7212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:25 smithi090 ceph-mon[29180]: pgmap v7213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:25 smithi119 ceph-mon[32752]: pgmap v7213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:25 smithi167 ceph-mon[32751]: pgmap v7213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12187.163233 secs 2024-04-26T21:58:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12187.163233 secs 2024-04-26T21:58:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12187.163233 secs 2024-04-26T21:58:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12187.902579 secs 2024-04-26T21:58:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:27 smithi090 ceph-mon[29180]: pgmap v7214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12187.902579 secs 2024-04-26T21:58:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:27 smithi119 ceph-mon[32752]: pgmap v7214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12187.902579 secs 2024-04-26T21:58:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:27 smithi167 ceph-mon[32751]: pgmap v7214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:29 smithi090 ceph-mon[29180]: pgmap v7215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:29 smithi119 ceph-mon[32752]: pgmap v7215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:29 smithi167 ceph-mon[32751]: pgmap v7215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12192.163317 secs 2024-04-26T21:58:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:31 smithi090 ceph-mon[29180]: pgmap v7216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12192.163317 secs 2024-04-26T21:58:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:31 smithi119 ceph-mon[32752]: pgmap v7216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12192.163317 secs 2024-04-26T21:58:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:31 smithi167 ceph-mon[32751]: pgmap v7216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12192.902696 secs 2024-04-26T21:58:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12192.902696 secs 2024-04-26T21:58:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12192.902696 secs 2024-04-26T21:58:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:33 smithi090 ceph-mon[29180]: pgmap v7217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:33 smithi119 ceph-mon[32752]: pgmap v7217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:33 smithi167 ceph-mon[32751]: pgmap v7217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:34 smithi090 ceph-mon[29180]: pgmap v7218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:34.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:58:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:58:34.069+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:58:34.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:58:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:58:34.069+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:58:34.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:58:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:58:34.069+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:58:34.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:58:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:58:34.069+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:58:34.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 21:58:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T21:58:34.069+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T21:58:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:34 smithi119 ceph-mon[32752]: pgmap v7218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:34 smithi167 ceph-mon[32751]: pgmap v7218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:35 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 12197.163385 secs 2024-04-26T21:58:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:35 smithi167 ceph-mon[32751]: slow request 11523.980770 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:58:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:35 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 12197.163385 secs 2024-04-26T21:58:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:35 smithi090 ceph-mon[29180]: slow request 11523.980770 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:58:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:35 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 12197.163385 secs 2024-04-26T21:58:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:35 smithi119 ceph-mon[32752]: slow request 11523.980770 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:58:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12197.902756 secs 2024-04-26T21:58:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:36 smithi090 ceph-mon[29180]: pgmap v7219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12197.902756 secs 2024-04-26T21:58:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:36 smithi119 ceph-mon[32752]: pgmap v7219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12197.902756 secs 2024-04-26T21:58:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:36 smithi167 ceph-mon[32751]: pgmap v7219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:39 smithi090 ceph-mon[29180]: pgmap v7220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:39 smithi119 ceph-mon[32752]: pgmap v7220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:39 smithi167 ceph-mon[32751]: pgmap v7220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12202.163447 secs 2024-04-26T21:58:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:41 smithi090 ceph-mon[29180]: pgmap v7221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12202.163447 secs 2024-04-26T21:58:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:41 smithi119 ceph-mon[32752]: pgmap v7221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12202.163447 secs 2024-04-26T21:58:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:41 smithi167 ceph-mon[32751]: pgmap v7221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12202.902873 secs 2024-04-26T21:58:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12202.902873 secs 2024-04-26T21:58:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12202.902873 secs 2024-04-26T21:58:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:43 smithi090 ceph-mon[29180]: pgmap v7222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:43 smithi119 ceph-mon[32752]: pgmap v7222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:43 smithi167 ceph-mon[32751]: pgmap v7222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:44 smithi119 ceph-mon[32752]: pgmap v7223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:44 smithi167 ceph-mon[32751]: pgmap v7223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:44 smithi090 ceph-mon[29180]: pgmap v7223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12207.163531 secs 2024-04-26T21:58:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12207.163531 secs 2024-04-26T21:58:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12207.163531 secs 2024-04-26T21:58:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12207.902939 secs 2024-04-26T21:58:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:46 smithi090 ceph-mon[29180]: pgmap v7224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12207.902939 secs 2024-04-26T21:58:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:46 smithi119 ceph-mon[32752]: pgmap v7224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12207.902939 secs 2024-04-26T21:58:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:46 smithi167 ceph-mon[32751]: pgmap v7224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:49 smithi090 ceph-mon[29180]: pgmap v7225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:49 smithi119 ceph-mon[32752]: pgmap v7225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:49 smithi167 ceph-mon[32751]: pgmap v7225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:51.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12212.163638 secs 2024-04-26T21:58:51.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:51 smithi119 ceph-mon[32752]: pgmap v7226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12212.163638 secs 2024-04-26T21:58:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:51 smithi090 ceph-mon[29180]: pgmap v7226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12212.163638 secs 2024-04-26T21:58:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:51 smithi167 ceph-mon[32751]: pgmap v7226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T21:58:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12212.903074 secs 2024-04-26T21:58:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12212.903074 secs 2024-04-26T21:58:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12212.903074 secs 2024-04-26T21:58:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:53 smithi090 ceph-mon[29180]: pgmap v7227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:53 smithi119 ceph-mon[32752]: pgmap v7227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:53 smithi167 ceph-mon[32751]: pgmap v7227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:55 smithi090 ceph-mon[29180]: pgmap v7228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:55 smithi119 ceph-mon[32752]: pgmap v7228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:55 smithi167 ceph-mon[32751]: pgmap v7228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:58:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12217.163722 secs 2024-04-26T21:58:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12217.163722 secs 2024-04-26T21:58:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12217.163722 secs 2024-04-26T21:58:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12217.903209 secs 2024-04-26T21:58:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:57 smithi090 ceph-mon[29180]: pgmap v7229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12217.903209 secs 2024-04-26T21:58:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:57 smithi119 ceph-mon[32752]: pgmap v7229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12217.903209 secs 2024-04-26T21:58:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:57 smithi167 ceph-mon[32751]: pgmap v7229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:58:58 smithi090 ceph-mon[29180]: pgmap v7230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:58:58 smithi119 ceph-mon[32752]: pgmap v7230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:58:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:58:58 smithi167 ceph-mon[32751]: pgmap v7230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:01 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 12222.163807 secs 2024-04-26T21:59:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:01 smithi090 ceph-mon[29180]: slow request 2880.161804 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:59:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:01 smithi090 ceph-mon[29180]: pgmap v7231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:01 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 12222.163807 secs 2024-04-26T21:59:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:01 smithi119 ceph-mon[32752]: slow request 2880.161804 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:59:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:01 smithi119 ceph-mon[32752]: pgmap v7231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:01 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 12222.163807 secs 2024-04-26T21:59:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:01 smithi167 ceph-mon[32751]: slow request 2880.161804 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T21:59:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:01 smithi167 ceph-mon[32751]: pgmap v7231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12222.903328 secs 2024-04-26T21:59:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12222.903328 secs 2024-04-26T21:59:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12222.903328 secs 2024-04-26T21:59:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:03 smithi090 ceph-mon[29180]: pgmap v7232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:03 smithi119 ceph-mon[32752]: pgmap v7232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:03 smithi167 ceph-mon[32751]: pgmap v7232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:05 smithi090 ceph-mon[29180]: pgmap v7233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:05 smithi119 ceph-mon[32752]: pgmap v7233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:05 smithi167 ceph-mon[32751]: pgmap v7233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12227.163868 secs 2024-04-26T21:59:06.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12227.163868 secs 2024-04-26T21:59:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12227.163868 secs 2024-04-26T21:59:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12227.903443 secs 2024-04-26T21:59:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:07 smithi090 ceph-mon[29180]: pgmap v7234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:59:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12227.903443 secs 2024-04-26T21:59:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:07 smithi119 ceph-mon[32752]: pgmap v7234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:59:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12227.903443 secs 2024-04-26T21:59:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:07 smithi167 ceph-mon[32751]: pgmap v7234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T21:59:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T21:59:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T21:59:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T21:59:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:08 smithi090 ceph-mon[29180]: pgmap v7235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:59:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:08 smithi090 ceph-mon[29180]: pgmap v7236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:08 smithi119 ceph-mon[32752]: pgmap v7235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:59:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:08 smithi119 ceph-mon[32752]: pgmap v7236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:08 smithi167 ceph-mon[32751]: pgmap v7235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T21:59:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:08 smithi167 ceph-mon[32751]: pgmap v7236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:09 smithi090 ceph-mon[29180]: pgmap v7237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:09 smithi119 ceph-mon[32752]: pgmap v7237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:09 smithi167 ceph-mon[32751]: pgmap v7237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12232.163930 secs 2024-04-26T21:59:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12232.163930 secs 2024-04-26T21:59:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12232.163930 secs 2024-04-26T21:59:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12232.903559 secs 2024-04-26T21:59:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:11 smithi090 ceph-mon[29180]: pgmap v7238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12232.903559 secs 2024-04-26T21:59:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:11 smithi119 ceph-mon[32752]: pgmap v7238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12232.903559 secs 2024-04-26T21:59:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:11 smithi167 ceph-mon[32751]: pgmap v7238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:13 smithi090 ceph-mon[29180]: pgmap v7239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:59:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:13 smithi119 ceph-mon[32752]: pgmap v7239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:59:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:13 smithi167 ceph-mon[32751]: pgmap v7239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T21:59:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:15 smithi090 ceph-mon[29180]: pgmap v7240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12237.163991 secs 2024-04-26T21:59:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:15 smithi119 ceph-mon[32752]: pgmap v7240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12237.163991 secs 2024-04-26T21:59:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:15 smithi167 ceph-mon[32751]: pgmap v7240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T21:59:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12237.163991 secs 2024-04-26T21:59:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12237.903652 secs 2024-04-26T21:59:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12237.903652 secs 2024-04-26T21:59:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12237.903652 secs 2024-04-26T21:59:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:17 smithi090 ceph-mon[29180]: pgmap v7241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:59:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:17 smithi119 ceph-mon[32752]: pgmap v7241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:59:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:17 smithi167 ceph-mon[32751]: pgmap v7241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T21:59:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:19 smithi090 ceph-mon[29180]: pgmap v7242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:19 smithi119 ceph-mon[32752]: pgmap v7242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:19 smithi167 ceph-mon[32751]: pgmap v7242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12242.164097 secs 2024-04-26T21:59:21.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12242.164097 secs 2024-04-26T21:59:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12242.164097 secs 2024-04-26T21:59:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12242.903771 secs 2024-04-26T21:59:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:21 smithi119 ceph-mon[32752]: pgmap v7243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12242.903771 secs 2024-04-26T21:59:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:21 smithi167 ceph-mon[32751]: pgmap v7243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12242.903771 secs 2024-04-26T21:59:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:21 smithi090 ceph-mon[29180]: pgmap v7243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:23 smithi119 ceph-mon[32752]: pgmap v7244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:23 smithi167 ceph-mon[32751]: pgmap v7244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:23 smithi090 ceph-mon[29180]: pgmap v7244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:25 smithi119 ceph-mon[32752]: pgmap v7245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12247.164160 secs 2024-04-26T21:59:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:25 smithi167 ceph-mon[32751]: pgmap v7245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12247.164160 secs 2024-04-26T21:59:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:25 smithi090 ceph-mon[29180]: pgmap v7245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12247.164160 secs 2024-04-26T21:59:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12247.903869 secs 2024-04-26T21:59:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12247.903869 secs 2024-04-26T21:59:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12247.903869 secs 2024-04-26T21:59:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:27 smithi119 ceph-mon[32752]: pgmap v7246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:27 smithi167 ceph-mon[32751]: pgmap v7246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:27 smithi090 ceph-mon[29180]: pgmap v7246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:29 smithi119 ceph-mon[32752]: pgmap v7247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:29 smithi167 ceph-mon[32751]: pgmap v7247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:29 smithi090 ceph-mon[29180]: pgmap v7247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12252.164246 secs 2024-04-26T21:59:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12252.164246 secs 2024-04-26T21:59:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12252.164246 secs 2024-04-26T21:59:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12252.903936 secs 2024-04-26T21:59:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:31 smithi119 ceph-mon[32752]: pgmap v7248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12252.903936 secs 2024-04-26T21:59:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:31 smithi167 ceph-mon[32751]: pgmap v7248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12252.903936 secs 2024-04-26T21:59:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:31 smithi090 ceph-mon[29180]: pgmap v7248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:33 smithi119 ceph-mon[32752]: pgmap v7249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:33 smithi167 ceph-mon[32751]: pgmap v7249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:33 smithi090 ceph-mon[29180]: pgmap v7249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:35 smithi119 ceph-mon[32752]: pgmap v7250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12257.164331 secs 2024-04-26T21:59:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:35 smithi167 ceph-mon[32751]: pgmap v7250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12257.164331 secs 2024-04-26T21:59:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:35 smithi090 ceph-mon[29180]: pgmap v7250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12257.164331 secs 2024-04-26T21:59:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:36 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 12257.904015 secs 2024-04-26T21:59:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:36 smithi119 ceph-mon[32752]: slow request 361.644147 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:59:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:36 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 12257.904015 secs 2024-04-26T21:59:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:36 smithi167 ceph-mon[32751]: slow request 361.644147 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:59:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:36 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 12257.904015 secs 2024-04-26T21:59:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:36 smithi090 ceph-mon[29180]: slow request 361.644147 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T21:59:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:37 smithi119 ceph-mon[32752]: pgmap v7251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:37 smithi167 ceph-mon[32751]: pgmap v7251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:37 smithi090 ceph-mon[29180]: pgmap v7251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:39 smithi119 ceph-mon[32752]: pgmap v7252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:39 smithi167 ceph-mon[32751]: pgmap v7252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:39 smithi090 ceph-mon[29180]: pgmap v7252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:41.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12262.164415 secs 2024-04-26T21:59:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12262.164415 secs 2024-04-26T21:59:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12262.164415 secs 2024-04-26T21:59:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12262.904133 secs 2024-04-26T21:59:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:41 smithi119 ceph-mon[32752]: pgmap v7253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12262.904133 secs 2024-04-26T21:59:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:41 smithi167 ceph-mon[32751]: pgmap v7253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12262.904133 secs 2024-04-26T21:59:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:41 smithi090 ceph-mon[29180]: pgmap v7253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:43 smithi119 ceph-mon[32752]: pgmap v7254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:43 smithi167 ceph-mon[32751]: pgmap v7254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:43 smithi090 ceph-mon[29180]: pgmap v7254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:45 smithi119 ceph-mon[32752]: pgmap v7255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12267.164494 secs 2024-04-26T21:59:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:45 smithi167 ceph-mon[32751]: pgmap v7255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12267.164494 secs 2024-04-26T21:59:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:45 smithi090 ceph-mon[29180]: pgmap v7255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12267.164494 secs 2024-04-26T21:59:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12267.904249 secs 2024-04-26T21:59:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12267.904249 secs 2024-04-26T21:59:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12267.904249 secs 2024-04-26T21:59:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:47 smithi119 ceph-mon[32752]: pgmap v7256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:47 smithi167 ceph-mon[32751]: pgmap v7256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:47 smithi090 ceph-mon[29180]: pgmap v7256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:49 smithi119 ceph-mon[32752]: pgmap v7257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:49 smithi167 ceph-mon[32751]: pgmap v7257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:49 smithi090 ceph-mon[29180]: pgmap v7257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:51.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12272.164576 secs 2024-04-26T21:59:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12272.164576 secs 2024-04-26T21:59:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12272.164576 secs 2024-04-26T21:59:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12272.904369 secs 2024-04-26T21:59:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:51 smithi119 ceph-mon[32752]: pgmap v7258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12272.904369 secs 2024-04-26T21:59:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:51 smithi167 ceph-mon[32751]: pgmap v7258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12272.904369 secs 2024-04-26T21:59:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:51 smithi090 ceph-mon[29180]: pgmap v7258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:53 smithi119 ceph-mon[32752]: pgmap v7259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:53 smithi167 ceph-mon[32751]: pgmap v7259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:53 smithi090 ceph-mon[29180]: pgmap v7259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:55 smithi119 ceph-mon[32752]: pgmap v7260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12277.164660 secs 2024-04-26T21:59:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:55 smithi167 ceph-mon[32751]: pgmap v7260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12277.164660 secs 2024-04-26T21:59:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:55 smithi090 ceph-mon[29180]: pgmap v7260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T21:59:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12277.164660 secs 2024-04-26T21:59:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12277.904505 secs 2024-04-26T21:59:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12277.904505 secs 2024-04-26T21:59:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12277.904505 secs 2024-04-26T21:59:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:57 smithi119 ceph-mon[32752]: pgmap v7261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:57 smithi167 ceph-mon[32751]: pgmap v7261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T21:59:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:57 smithi090 ceph-mon[29180]: pgmap v7261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 21:59:59 smithi119 ceph-mon[32752]: pgmap v7262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 21:59:59 smithi167 ceph-mon[32751]: pgmap v7262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 21:59:59 smithi090 ceph-mon[29180]: pgmap v7262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12282.164744 secs 2024-04-26T22:00:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:00:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12282.164744 secs 2024-04-26T22:00:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:00:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12282.164744 secs 2024-04-26T22:00:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:00:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12282.904574 secs 2024-04-26T22:00:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:01 smithi119 ceph-mon[32752]: pgmap v7263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12282.904574 secs 2024-04-26T22:00:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:01 smithi167 ceph-mon[32751]: pgmap v7263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12282.904574 secs 2024-04-26T22:00:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:01 smithi090 ceph-mon[29180]: pgmap v7263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:03 smithi119 ceph-mon[32752]: pgmap v7264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:03 smithi167 ceph-mon[32751]: pgmap v7264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:03 smithi090 ceph-mon[29180]: pgmap v7264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:05 smithi090 ceph-mon[29180]: pgmap v7265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12287.164826 secs 2024-04-26T22:00:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:05 smithi119 ceph-mon[32752]: pgmap v7265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12287.164826 secs 2024-04-26T22:00:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:05 smithi167 ceph-mon[32751]: pgmap v7265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12287.164826 secs 2024-04-26T22:00:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12287.904643 secs 2024-04-26T22:00:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12287.904643 secs 2024-04-26T22:00:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12287.904643 secs 2024-04-26T22:00:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:00:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:07 smithi119 ceph-mon[32752]: pgmap v7266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:00:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:00:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:00:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:07 smithi167 ceph-mon[32751]: pgmap v7266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:00:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:00:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:00:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:07 smithi090 ceph-mon[29180]: pgmap v7266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:00:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:00:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:00:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:08 smithi119 ceph-mon[32752]: pgmap v7267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:00:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:08 smithi119 ceph-mon[32752]: pgmap v7268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:00:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:08 smithi167 ceph-mon[32751]: pgmap v7267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:00:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:08 smithi167 ceph-mon[32751]: pgmap v7268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:00:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:08 smithi090 ceph-mon[29180]: pgmap v7267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:00:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:08 smithi090 ceph-mon[29180]: pgmap v7268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:00:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:10 smithi119 ceph-mon[32752]: pgmap v7269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12292.164914 secs 2024-04-26T22:00:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:10 smithi167 ceph-mon[32751]: pgmap v7269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12292.164914 secs 2024-04-26T22:00:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:10 smithi090 ceph-mon[29180]: pgmap v7269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12292.164914 secs 2024-04-26T22:00:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12292.904779 secs 2024-04-26T22:00:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12292.904779 secs 2024-04-26T22:00:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12292.904779 secs 2024-04-26T22:00:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:13 smithi119 ceph-mon[32752]: pgmap v7270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:13 smithi167 ceph-mon[32751]: pgmap v7270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:13 smithi090 ceph-mon[29180]: pgmap v7270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:15 smithi119 ceph-mon[32752]: pgmap v7271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:15 smithi167 ceph-mon[32751]: pgmap v7271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:15 smithi090 ceph-mon[29180]: pgmap v7271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12297.164975 secs 2024-04-26T22:00:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12297.164975 secs 2024-04-26T22:00:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12297.164975 secs 2024-04-26T22:00:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:17 smithi119 ceph-mon[32752]: pgmap v7272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12297.904886 secs 2024-04-26T22:00:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:17 smithi167 ceph-mon[32751]: pgmap v7272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12297.904886 secs 2024-04-26T22:00:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:17 smithi090 ceph-mon[29180]: pgmap v7272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:00:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12297.904886 secs 2024-04-26T22:00:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:19 smithi119 ceph-mon[32752]: pgmap v7273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:00:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:19 smithi167 ceph-mon[32751]: pgmap v7273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:00:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:19 smithi090 ceph-mon[29180]: pgmap v7273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:00:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:21 smithi119 ceph-mon[32752]: pgmap v7274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12302.165081 secs 2024-04-26T22:00:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:21 smithi167 ceph-mon[32751]: pgmap v7274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12302.165081 secs 2024-04-26T22:00:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:21 smithi090 ceph-mon[29180]: pgmap v7274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12302.165081 secs 2024-04-26T22:00:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12302.905007 secs 2024-04-26T22:00:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12302.905007 secs 2024-04-26T22:00:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12302.905007 secs 2024-04-26T22:00:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:23 smithi119 ceph-mon[32752]: pgmap v7275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:23 smithi167 ceph-mon[32751]: pgmap v7275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:23 smithi090 ceph-mon[29180]: pgmap v7275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:25 smithi119 ceph-mon[32752]: pgmap v7276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:25 smithi167 ceph-mon[32751]: pgmap v7276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:25 smithi090 ceph-mon[29180]: pgmap v7276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12307.165165 secs 2024-04-26T22:00:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12307.165165 secs 2024-04-26T22:00:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12307.165165 secs 2024-04-26T22:00:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:27 smithi119 ceph-mon[32752]: pgmap v7277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12307.905126 secs 2024-04-26T22:00:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:27 smithi167 ceph-mon[32751]: pgmap v7277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12307.905126 secs 2024-04-26T22:00:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:27 smithi090 ceph-mon[29180]: pgmap v7277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12307.905126 secs 2024-04-26T22:00:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:29 smithi119 ceph-mon[32752]: pgmap v7278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:29 smithi167 ceph-mon[32751]: pgmap v7278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:29 smithi090 ceph-mon[29180]: pgmap v7278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:31 smithi119 ceph-mon[32752]: pgmap v7279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12312.165250 secs 2024-04-26T22:00:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:31 smithi167 ceph-mon[32751]: pgmap v7279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12312.165250 secs 2024-04-26T22:00:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:31 smithi090 ceph-mon[29180]: pgmap v7279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12312.165250 secs 2024-04-26T22:00:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12312.905242 secs 2024-04-26T22:00:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12312.905242 secs 2024-04-26T22:00:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12312.905242 secs 2024-04-26T22:00:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:33 smithi119 ceph-mon[32752]: pgmap v7280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:33 smithi167 ceph-mon[32751]: pgmap v7280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:33 smithi090 ceph-mon[29180]: pgmap v7280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:35 smithi167 ceph-mon[32751]: pgmap v7281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:35 smithi090 ceph-mon[29180]: pgmap v7281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:35 smithi119 ceph-mon[32752]: pgmap v7281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12317.165332 secs 2024-04-26T22:00:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12317.165332 secs 2024-04-26T22:00:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12317.165332 secs 2024-04-26T22:00:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:37 smithi167 ceph-mon[32751]: pgmap v7282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12317.905356 secs 2024-04-26T22:00:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:37 smithi090 ceph-mon[29180]: pgmap v7282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12317.905356 secs 2024-04-26T22:00:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:37 smithi119 ceph-mon[32752]: pgmap v7282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12317.905356 secs 2024-04-26T22:00:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:39 smithi167 ceph-mon[32751]: pgmap v7283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:39 smithi090 ceph-mon[29180]: pgmap v7283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:39 smithi119 ceph-mon[32752]: pgmap v7283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:41 smithi090 ceph-mon[29180]: pgmap v7284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12322.165392 secs 2024-04-26T22:00:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:41 smithi119 ceph-mon[32752]: pgmap v7284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12322.165392 secs 2024-04-26T22:00:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:41 smithi167 ceph-mon[32751]: pgmap v7284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12322.165392 secs 2024-04-26T22:00:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12322.905458 secs 2024-04-26T22:00:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12322.905458 secs 2024-04-26T22:00:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12322.905458 secs 2024-04-26T22:00:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:43 smithi090 ceph-mon[29180]: pgmap v7285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:43 smithi119 ceph-mon[32752]: pgmap v7285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:43 smithi167 ceph-mon[32751]: pgmap v7285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:45 smithi090 ceph-mon[29180]: pgmap v7286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:45 smithi119 ceph-mon[32752]: pgmap v7286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:45 smithi167 ceph-mon[32751]: pgmap v7286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12327.165478 secs 2024-04-26T22:00:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12327.165478 secs 2024-04-26T22:00:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12327.165478 secs 2024-04-26T22:00:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:47 smithi090 ceph-mon[29180]: pgmap v7287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12327.905581 secs 2024-04-26T22:00:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:47 smithi119 ceph-mon[32752]: pgmap v7287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12327.905581 secs 2024-04-26T22:00:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:47 smithi167 ceph-mon[32751]: pgmap v7287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12327.905581 secs 2024-04-26T22:00:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:49 smithi090 ceph-mon[29180]: pgmap v7288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:49 smithi119 ceph-mon[32752]: pgmap v7288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:49 smithi167 ceph-mon[32751]: pgmap v7288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:51.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:51 smithi119 ceph-mon[32752]: pgmap v7289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:51.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12332.165563 secs 2024-04-26T22:00:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:51 smithi090 ceph-mon[29180]: pgmap v7289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12332.165563 secs 2024-04-26T22:00:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:51 smithi167 ceph-mon[32751]: pgmap v7289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12332.165563 secs 2024-04-26T22:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12332.905701 secs 2024-04-26T22:00:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12332.905701 secs 2024-04-26T22:00:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12332.905701 secs 2024-04-26T22:00:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:53 smithi090 ceph-mon[29180]: pgmap v7290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:53 smithi119 ceph-mon[32752]: pgmap v7290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:53 smithi167 ceph-mon[32751]: pgmap v7290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:55 smithi090 ceph-mon[29180]: pgmap v7291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:55 smithi119 ceph-mon[32752]: pgmap v7291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:55 smithi167 ceph-mon[32751]: pgmap v7291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:00:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12337.165647 secs 2024-04-26T22:00:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12337.165647 secs 2024-04-26T22:00:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12337.165647 secs 2024-04-26T22:00:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:57 smithi090 ceph-mon[29180]: pgmap v7292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12337.905780 secs 2024-04-26T22:00:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:57 smithi119 ceph-mon[32752]: pgmap v7292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12337.905780 secs 2024-04-26T22:00:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:57 smithi167 ceph-mon[32751]: pgmap v7292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12337.905780 secs 2024-04-26T22:00:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:00:59 smithi090 ceph-mon[29180]: pgmap v7293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:00:59 smithi119 ceph-mon[32752]: pgmap v7293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:00:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:00:59 smithi167 ceph-mon[32751]: pgmap v7293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:01 smithi090 ceph-mon[29180]: pgmap v7294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12342.165735 secs 2024-04-26T22:01:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:01 smithi119 ceph-mon[32752]: pgmap v7294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12342.165735 secs 2024-04-26T22:01:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:01 smithi167 ceph-mon[32751]: pgmap v7294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12342.165735 secs 2024-04-26T22:01:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12342.905894 secs 2024-04-26T22:01:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12342.905894 secs 2024-04-26T22:01:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12342.905894 secs 2024-04-26T22:01:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:03 smithi090 ceph-mon[29180]: pgmap v7295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:03 smithi119 ceph-mon[32752]: pgmap v7295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:03 smithi167 ceph-mon[32751]: pgmap v7295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:04 smithi119 ceph-mon[32752]: pgmap v7296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:04 smithi167 ceph-mon[32751]: pgmap v7296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:04 smithi090 ceph-mon[29180]: pgmap v7296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12347.165823 secs 2024-04-26T22:01:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12347.165823 secs 2024-04-26T22:01:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12347.165823 secs 2024-04-26T22:01:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:06 smithi090 ceph-mon[29180]: pgmap v7297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12347.906017 secs 2024-04-26T22:01:06.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:06 smithi167 ceph-mon[32751]: pgmap v7297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:06.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12347.906017 secs 2024-04-26T22:01:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:06 smithi119 ceph-mon[32752]: pgmap v7297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12347.906017 secs 2024-04-26T22:01:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:08 smithi119 ceph-mon[32752]: pgmap v7298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:01:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:01:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:08 smithi167 ceph-mon[32751]: pgmap v7298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:01:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:01:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:08 smithi090 ceph-mon[29180]: pgmap v7298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:01:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:01:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:01:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:01:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:09 smithi119 ceph-mon[32752]: pgmap v7299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:01:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:09 smithi119 ceph-mon[32752]: pgmap v7300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:09 smithi167 ceph-mon[32751]: pgmap v7299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:01:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:09 smithi167 ceph-mon[32751]: pgmap v7300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:09 smithi090 ceph-mon[29180]: pgmap v7299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:01:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:09 smithi090 ceph-mon[29180]: pgmap v7300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:11.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12352.165883 secs 2024-04-26T22:01:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12352.165883 secs 2024-04-26T22:01:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12352.165883 secs 2024-04-26T22:01:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:11 smithi119 ceph-mon[32752]: pgmap v7301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12352.906150 secs 2024-04-26T22:01:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:11 smithi167 ceph-mon[32751]: pgmap v7301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12352.906150 secs 2024-04-26T22:01:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:11 smithi090 ceph-mon[29180]: pgmap v7301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12352.906150 secs 2024-04-26T22:01:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:13 smithi119 ceph-mon[32752]: pgmap v7302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:01:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:13 smithi167 ceph-mon[32751]: pgmap v7302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:01:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:13 smithi090 ceph-mon[29180]: pgmap v7302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:01:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:15 smithi119 ceph-mon[32752]: pgmap v7303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12357.165968 secs 2024-04-26T22:01:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:15 smithi167 ceph-mon[32751]: pgmap v7303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12357.165968 secs 2024-04-26T22:01:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:15 smithi090 ceph-mon[29180]: pgmap v7303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12357.165968 secs 2024-04-26T22:01:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12357.906273 secs 2024-04-26T22:01:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12357.906273 secs 2024-04-26T22:01:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12357.906273 secs 2024-04-26T22:01:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:17 smithi119 ceph-mon[32752]: pgmap v7304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:17 smithi167 ceph-mon[32751]: pgmap v7304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:17 smithi090 ceph-mon[29180]: pgmap v7304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:01:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:19 smithi119 ceph-mon[32752]: pgmap v7305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:01:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:19 smithi167 ceph-mon[32751]: pgmap v7305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:01:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:19 smithi090 ceph-mon[29180]: pgmap v7305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:01:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12362.166055 secs 2024-04-26T22:01:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12362.166055 secs 2024-04-26T22:01:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12362.166055 secs 2024-04-26T22:01:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:21 smithi119 ceph-mon[32752]: pgmap v7306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12362.906415 secs 2024-04-26T22:01:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:21 smithi167 ceph-mon[32751]: pgmap v7306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12362.906415 secs 2024-04-26T22:01:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:21 smithi090 ceph-mon[29180]: pgmap v7306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12362.906415 secs 2024-04-26T22:01:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:23 smithi119 ceph-mon[32752]: pgmap v7307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:23 smithi167 ceph-mon[32751]: pgmap v7307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:23 smithi090 ceph-mon[29180]: pgmap v7307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:25 smithi119 ceph-mon[32752]: pgmap v7308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12367.166159 secs 2024-04-26T22:01:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:25 smithi167 ceph-mon[32751]: pgmap v7308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12367.166159 secs 2024-04-26T22:01:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:25 smithi090 ceph-mon[29180]: pgmap v7308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12367.166159 secs 2024-04-26T22:01:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12367.906537 secs 2024-04-26T22:01:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12367.906537 secs 2024-04-26T22:01:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12367.906537 secs 2024-04-26T22:01:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:27 smithi119 ceph-mon[32752]: pgmap v7309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:27 smithi167 ceph-mon[32751]: pgmap v7309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:27 smithi090 ceph-mon[29180]: pgmap v7309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:29 smithi119 ceph-mon[32752]: pgmap v7310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:29 smithi167 ceph-mon[32751]: pgmap v7310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:29 smithi090 ceph-mon[29180]: pgmap v7310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12372.166244 secs 2024-04-26T22:01:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12372.166244 secs 2024-04-26T22:01:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12372.166244 secs 2024-04-26T22:01:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:31 smithi119 ceph-mon[32752]: pgmap v7311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12372.906657 secs 2024-04-26T22:01:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:31 smithi167 ceph-mon[32751]: pgmap v7311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12372.906657 secs 2024-04-26T22:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:31 smithi090 ceph-mon[29180]: pgmap v7311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12372.906657 secs 2024-04-26T22:01:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:33 smithi119 ceph-mon[32752]: pgmap v7312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:33 smithi167 ceph-mon[32751]: pgmap v7312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:33 smithi090 ceph-mon[29180]: pgmap v7312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:35 smithi119 ceph-mon[32752]: pgmap v7313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12377.166307 secs 2024-04-26T22:01:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:35 smithi167 ceph-mon[32751]: pgmap v7313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12377.166307 secs 2024-04-26T22:01:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:35 smithi090 ceph-mon[29180]: pgmap v7313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12377.166307 secs 2024-04-26T22:01:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12377.906737 secs 2024-04-26T22:01:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12377.906737 secs 2024-04-26T22:01:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12377.906737 secs 2024-04-26T22:01:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:37 smithi119 ceph-mon[32752]: pgmap v7314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:37 smithi167 ceph-mon[32751]: pgmap v7314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:37 smithi090 ceph-mon[29180]: pgmap v7314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:39 smithi090 ceph-mon[29180]: pgmap v7315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:39 smithi119 ceph-mon[32752]: pgmap v7315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:39 smithi167 ceph-mon[32751]: pgmap v7315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12382.166383 secs 2024-04-26T22:01:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12382.166383 secs 2024-04-26T22:01:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12382.166383 secs 2024-04-26T22:01:42.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:41 smithi090 ceph-mon[29180]: pgmap v7316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:42.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12382.906799 secs 2024-04-26T22:01:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:41 smithi119 ceph-mon[32752]: pgmap v7316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12382.906799 secs 2024-04-26T22:01:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:41 smithi167 ceph-mon[32751]: pgmap v7316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12382.906799 secs 2024-04-26T22:01:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:43 smithi119 ceph-mon[32752]: pgmap v7317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:43 smithi167 ceph-mon[32751]: pgmap v7317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:43 smithi090 ceph-mon[29180]: pgmap v7317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:45 smithi119 ceph-mon[32752]: pgmap v7318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12387.166461 secs 2024-04-26T22:01:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:45 smithi167 ceph-mon[32751]: pgmap v7318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12387.166461 secs 2024-04-26T22:01:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:45 smithi090 ceph-mon[29180]: pgmap v7318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12387.166461 secs 2024-04-26T22:01:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12387.906892 secs 2024-04-26T22:01:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12387.906892 secs 2024-04-26T22:01:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12387.906892 secs 2024-04-26T22:01:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:47 smithi119 ceph-mon[32752]: pgmap v7319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:47 smithi167 ceph-mon[32751]: pgmap v7319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:47 smithi090 ceph-mon[29180]: pgmap v7319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:49 smithi119 ceph-mon[32752]: pgmap v7320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:49 smithi167 ceph-mon[32751]: pgmap v7320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:49 smithi090 ceph-mon[29180]: pgmap v7320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12392.166545 secs 2024-04-26T22:01:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12392.166545 secs 2024-04-26T22:01:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12392.166545 secs 2024-04-26T22:01:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:51 smithi119 ceph-mon[32752]: pgmap v7321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12392.907047 secs 2024-04-26T22:01:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:51 smithi167 ceph-mon[32751]: pgmap v7321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12392.907047 secs 2024-04-26T22:01:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:51 smithi090 ceph-mon[29180]: pgmap v7321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12392.907047 secs 2024-04-26T22:01:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:53 smithi119 ceph-mon[32752]: pgmap v7322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:53 smithi167 ceph-mon[32751]: pgmap v7322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:53 smithi090 ceph-mon[29180]: pgmap v7322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:55 smithi119 ceph-mon[32752]: pgmap v7323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12397.166632 secs 2024-04-26T22:01:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:55 smithi167 ceph-mon[32751]: pgmap v7323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12397.166632 secs 2024-04-26T22:01:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:55 smithi090 ceph-mon[29180]: pgmap v7323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:01:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12397.166632 secs 2024-04-26T22:01:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12397.907193 secs 2024-04-26T22:01:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12397.907193 secs 2024-04-26T22:01:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12397.907193 secs 2024-04-26T22:01:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:57 smithi119 ceph-mon[32752]: pgmap v7324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:57 smithi167 ceph-mon[32751]: pgmap v7324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:01:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:57 smithi090 ceph-mon[29180]: pgmap v7324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:01:59 smithi119 ceph-mon[32752]: pgmap v7325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:01:59 smithi167 ceph-mon[32751]: pgmap v7325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:01:59 smithi090 ceph-mon[29180]: pgmap v7325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12402.166716 secs 2024-04-26T22:02:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12402.166716 secs 2024-04-26T22:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12402.166716 secs 2024-04-26T22:02:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:01 smithi119 ceph-mon[32752]: pgmap v7326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12402.907309 secs 2024-04-26T22:02:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:01 smithi167 ceph-mon[32751]: pgmap v7326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12402.907309 secs 2024-04-26T22:02:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:01 smithi090 ceph-mon[29180]: pgmap v7326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12402.907309 secs 2024-04-26T22:02:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:03 smithi119 ceph-mon[32752]: pgmap v7327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:03 smithi167 ceph-mon[32751]: pgmap v7327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:03 smithi090 ceph-mon[29180]: pgmap v7327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:05 smithi119 ceph-mon[32752]: pgmap v7328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12407.166780 secs 2024-04-26T22:02:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:05 smithi167 ceph-mon[32751]: pgmap v7328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12407.166780 secs 2024-04-26T22:02:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:05 smithi090 ceph-mon[29180]: pgmap v7328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12407.166780 secs 2024-04-26T22:02:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12407.907424 secs 2024-04-26T22:02:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12407.907424 secs 2024-04-26T22:02:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12407.907424 secs 2024-04-26T22:02:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:07 smithi167 ceph-mon[32751]: pgmap v7329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:07 smithi090 ceph-mon[29180]: pgmap v7329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:07 smithi119 ceph-mon[32752]: pgmap v7329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:02:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:09 smithi167 ceph-mon[32751]: pgmap v7330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:02:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:09 smithi090 ceph-mon[29180]: pgmap v7330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:02:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:09 smithi119 ceph-mon[32752]: pgmap v7330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:02:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:02:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:10 smithi167 ceph-mon[32751]: pgmap v7331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:10 smithi167 ceph-mon[32751]: pgmap v7332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12412.166866 secs 2024-04-26T22:02:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:10 smithi090 ceph-mon[29180]: pgmap v7331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:02:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:10 smithi090 ceph-mon[29180]: pgmap v7332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12412.166866 secs 2024-04-26T22:02:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:10 smithi119 ceph-mon[32752]: pgmap v7331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:02:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:10 smithi119 ceph-mon[32752]: pgmap v7332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12412.166866 secs 2024-04-26T22:02:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12412.907540 secs 2024-04-26T22:02:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12412.907540 secs 2024-04-26T22:02:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12412.907540 secs 2024-04-26T22:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:12 smithi090 ceph-mon[29180]: pgmap v7333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:12 smithi119 ceph-mon[32752]: pgmap v7333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:12 smithi167 ceph-mon[32751]: pgmap v7333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:14 smithi090 ceph-mon[29180]: pgmap v7334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:02:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:14 smithi119 ceph-mon[32752]: pgmap v7334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:02:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:14 smithi167 ceph-mon[32751]: pgmap v7334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:02:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12417.166928 secs 2024-04-26T22:02:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12417.166928 secs 2024-04-26T22:02:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12417.166928 secs 2024-04-26T22:02:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:16 smithi090 ceph-mon[29180]: pgmap v7335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12417.907619 secs 2024-04-26T22:02:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:16 smithi119 ceph-mon[32752]: pgmap v7335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12417.907619 secs 2024-04-26T22:02:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:16 smithi167 ceph-mon[32751]: pgmap v7335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:02:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12417.907619 secs 2024-04-26T22:02:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:18 smithi090 ceph-mon[29180]: pgmap v7336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:02:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:18 smithi119 ceph-mon[32752]: pgmap v7336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:02:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:18 smithi167 ceph-mon[32751]: pgmap v7336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:02:21.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:20 smithi119 ceph-mon[32752]: pgmap v7337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:02:21.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12422.167014 secs 2024-04-26T22:02:21.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:20 smithi090 ceph-mon[29180]: pgmap v7337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:02:21.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12422.167014 secs 2024-04-26T22:02:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:20 smithi167 ceph-mon[32751]: pgmap v7337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:02:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12422.167014 secs 2024-04-26T22:02:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12422.907736 secs 2024-04-26T22:02:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12422.907736 secs 2024-04-26T22:02:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12422.907736 secs 2024-04-26T22:02:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:22 smithi090 ceph-mon[29180]: pgmap v7338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:22 smithi119 ceph-mon[32752]: pgmap v7338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:22 smithi167 ceph-mon[32751]: pgmap v7338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:24 smithi090 ceph-mon[29180]: pgmap v7339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:24 smithi119 ceph-mon[32752]: pgmap v7339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:24 smithi167 ceph-mon[32751]: pgmap v7339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:26.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12427.167107 secs 2024-04-26T22:02:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12427.167107 secs 2024-04-26T22:02:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12427.167107 secs 2024-04-26T22:02:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:26 smithi090 ceph-mon[29180]: pgmap v7340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12427.907856 secs 2024-04-26T22:02:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:26 smithi119 ceph-mon[32752]: pgmap v7340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12427.907856 secs 2024-04-26T22:02:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:26 smithi167 ceph-mon[32751]: pgmap v7340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12427.907856 secs 2024-04-26T22:02:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:28 smithi090 ceph-mon[29180]: pgmap v7341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:28 smithi119 ceph-mon[32752]: pgmap v7341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:28 smithi167 ceph-mon[32751]: pgmap v7341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:30 smithi090 ceph-mon[29180]: pgmap v7342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12432.167167 secs 2024-04-26T22:02:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:30 smithi119 ceph-mon[32752]: pgmap v7342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12432.167167 secs 2024-04-26T22:02:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:30 smithi167 ceph-mon[32751]: pgmap v7342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12432.167167 secs 2024-04-26T22:02:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12432.907969 secs 2024-04-26T22:02:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12432.907969 secs 2024-04-26T22:02:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12432.907969 secs 2024-04-26T22:02:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:32 smithi090 ceph-mon[29180]: pgmap v7343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:32 smithi119 ceph-mon[32752]: pgmap v7343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:32 smithi167 ceph-mon[32751]: pgmap v7343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:34 smithi090 ceph-mon[29180]: pgmap v7344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:34 smithi119 ceph-mon[32752]: pgmap v7344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:34 smithi167 ceph-mon[32751]: pgmap v7344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:36.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12437.167251 secs 2024-04-26T22:02:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12437.167251 secs 2024-04-26T22:02:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12437.167251 secs 2024-04-26T22:02:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:36 smithi119 ceph-mon[32752]: pgmap v7345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12437.908083 secs 2024-04-26T22:02:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:36 smithi167 ceph-mon[32751]: pgmap v7345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12437.908083 secs 2024-04-26T22:02:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:36 smithi090 ceph-mon[29180]: pgmap v7345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12437.908083 secs 2024-04-26T22:02:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:38 smithi119 ceph-mon[32752]: pgmap v7346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:38 smithi167 ceph-mon[32751]: pgmap v7346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:38 smithi090 ceph-mon[29180]: pgmap v7346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:40 smithi119 ceph-mon[32752]: pgmap v7347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12442.167337 secs 2024-04-26T22:02:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:40 smithi167 ceph-mon[32751]: pgmap v7347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12442.167337 secs 2024-04-26T22:02:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:40 smithi090 ceph-mon[29180]: pgmap v7347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12442.167337 secs 2024-04-26T22:02:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12442.908222 secs 2024-04-26T22:02:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12442.908222 secs 2024-04-26T22:02:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12442.908222 secs 2024-04-26T22:02:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:42 smithi119 ceph-mon[32752]: pgmap v7348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:42 smithi167 ceph-mon[32751]: pgmap v7348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:42 smithi090 ceph-mon[29180]: pgmap v7348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:44 smithi119 ceph-mon[32752]: pgmap v7349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:44 smithi167 ceph-mon[32751]: pgmap v7349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:44 smithi090 ceph-mon[29180]: pgmap v7349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12447.167424 secs 2024-04-26T22:02:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12447.167424 secs 2024-04-26T22:02:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12447.167424 secs 2024-04-26T22:02:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:46 smithi119 ceph-mon[32752]: pgmap v7350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12447.908338 secs 2024-04-26T22:02:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:46 smithi167 ceph-mon[32751]: pgmap v7350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12447.908338 secs 2024-04-26T22:02:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:46 smithi090 ceph-mon[29180]: pgmap v7350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12447.908338 secs 2024-04-26T22:02:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:48 smithi119 ceph-mon[32752]: pgmap v7351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:48 smithi167 ceph-mon[32751]: pgmap v7351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:48 smithi090 ceph-mon[29180]: pgmap v7351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:50 smithi119 ceph-mon[32752]: pgmap v7352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12452.167507 secs 2024-04-26T22:02:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:50 smithi167 ceph-mon[32751]: pgmap v7352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12452.167507 secs 2024-04-26T22:02:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:50 smithi090 ceph-mon[29180]: pgmap v7352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12452.167507 secs 2024-04-26T22:02:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12452.908438 secs 2024-04-26T22:02:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12452.908438 secs 2024-04-26T22:02:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12452.908438 secs 2024-04-26T22:02:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:52 smithi119 ceph-mon[32752]: pgmap v7353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:52 smithi167 ceph-mon[32751]: pgmap v7353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:52 smithi090 ceph-mon[29180]: pgmap v7353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:54 smithi119 ceph-mon[32752]: pgmap v7354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:54 smithi167 ceph-mon[32751]: pgmap v7354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:54 smithi090 ceph-mon[29180]: pgmap v7354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12457.167590 secs 2024-04-26T22:02:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12457.167590 secs 2024-04-26T22:02:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12457.167590 secs 2024-04-26T22:02:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:56 smithi090 ceph-mon[29180]: pgmap v7355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12457.908550 secs 2024-04-26T22:02:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:56 smithi119 ceph-mon[32752]: pgmap v7355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12457.908550 secs 2024-04-26T22:02:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:56 smithi167 ceph-mon[32751]: pgmap v7355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:02:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12457.908550 secs 2024-04-26T22:02:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:02:58 smithi119 ceph-mon[32752]: pgmap v7356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:02:58 smithi167 ceph-mon[32751]: pgmap v7356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:02:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:02:58 smithi090 ceph-mon[29180]: pgmap v7356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:00 smithi119 ceph-mon[32752]: pgmap v7357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12462.167690 secs 2024-04-26T22:03:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:00 smithi167 ceph-mon[32751]: pgmap v7357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12462.167690 secs 2024-04-26T22:03:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:00 smithi090 ceph-mon[29180]: pgmap v7357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12462.167690 secs 2024-04-26T22:03:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12462.908663 secs 2024-04-26T22:03:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12462.908663 secs 2024-04-26T22:03:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12462.908663 secs 2024-04-26T22:03:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:02 smithi119 ceph-mon[32752]: pgmap v7358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:02 smithi167 ceph-mon[32751]: pgmap v7358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:02 smithi090 ceph-mon[29180]: pgmap v7358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:04 smithi119 ceph-mon[32752]: pgmap v7359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:04 smithi167 ceph-mon[32751]: pgmap v7359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:04 smithi090 ceph-mon[29180]: pgmap v7359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:06.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12467.167774 secs 2024-04-26T22:03:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12467.167774 secs 2024-04-26T22:03:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12467.167774 secs 2024-04-26T22:03:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:06 smithi119 ceph-mon[32752]: pgmap v7360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12467.908781 secs 2024-04-26T22:03:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:06 smithi167 ceph-mon[32751]: pgmap v7360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12467.908781 secs 2024-04-26T22:03:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:06 smithi090 ceph-mon[29180]: pgmap v7360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12467.908781 secs 2024-04-26T22:03:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:08 smithi119 ceph-mon[32752]: pgmap v7361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:08 smithi167 ceph-mon[32751]: pgmap v7361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:08 smithi090 ceph-mon[29180]: pgmap v7361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:03:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:03:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:03:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:03:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:03:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:03:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:10 smithi119 ceph-mon[32752]: pgmap v7362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12472.167863 secs 2024-04-26T22:03:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:10 smithi119 ceph-mon[32752]: pgmap v7363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:03:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:10 smithi119 ceph-mon[32752]: pgmap v7364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:10 smithi167 ceph-mon[32751]: pgmap v7362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12472.167863 secs 2024-04-26T22:03:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:10 smithi167 ceph-mon[32751]: pgmap v7363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:03:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:10 smithi167 ceph-mon[32751]: pgmap v7364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:10 smithi090 ceph-mon[29180]: pgmap v7362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12472.167863 secs 2024-04-26T22:03:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:10 smithi090 ceph-mon[29180]: pgmap v7363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:03:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:10 smithi090 ceph-mon[29180]: pgmap v7364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:12.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12472.908871 secs 2024-04-26T22:03:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12472.908871 secs 2024-04-26T22:03:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12472.908871 secs 2024-04-26T22:03:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:12 smithi090 ceph-mon[29180]: pgmap v7365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:12 smithi119 ceph-mon[32752]: pgmap v7365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:12 smithi167 ceph-mon[32751]: pgmap v7365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:14 smithi119 ceph-mon[32752]: pgmap v7366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:14 smithi167 ceph-mon[32751]: pgmap v7366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:14 smithi090 ceph-mon[29180]: pgmap v7366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12477.167954 secs 2024-04-26T22:03:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12477.167954 secs 2024-04-26T22:03:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12477.167954 secs 2024-04-26T22:03:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12477.908990 secs 2024-04-26T22:03:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:16 smithi119 ceph-mon[32752]: pgmap v7367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12477.908990 secs 2024-04-26T22:03:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:16 smithi167 ceph-mon[32751]: pgmap v7367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12477.908990 secs 2024-04-26T22:03:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:16 smithi090 ceph-mon[29180]: pgmap v7367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:03:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:18 smithi119 ceph-mon[32752]: pgmap v7368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:03:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:18 smithi167 ceph-mon[32751]: pgmap v7368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:03:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:18 smithi090 ceph-mon[29180]: pgmap v7368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:03:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12482.168044 secs 2024-04-26T22:03:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:20 smithi119 ceph-mon[32752]: pgmap v7369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:03:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12482.168044 secs 2024-04-26T22:03:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:20 smithi167 ceph-mon[32751]: pgmap v7369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:03:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12482.168044 secs 2024-04-26T22:03:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:20 smithi090 ceph-mon[29180]: pgmap v7369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:03:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12482.909099 secs 2024-04-26T22:03:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12482.909099 secs 2024-04-26T22:03:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12482.909099 secs 2024-04-26T22:03:23.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:23 smithi119 ceph-mon[32752]: pgmap v7370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:23 smithi167 ceph-mon[32751]: pgmap v7370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:23 smithi090 ceph-mon[29180]: pgmap v7370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:25 smithi119 ceph-mon[32752]: pgmap v7371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:25 smithi167 ceph-mon[32751]: pgmap v7371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:25 smithi090 ceph-mon[29180]: pgmap v7371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12487.168147 secs 2024-04-26T22:03:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12487.168147 secs 2024-04-26T22:03:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12487.168147 secs 2024-04-26T22:03:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12487.909208 secs 2024-04-26T22:03:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:27 smithi119 ceph-mon[32752]: pgmap v7372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12487.909208 secs 2024-04-26T22:03:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:27 smithi167 ceph-mon[32751]: pgmap v7372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12487.909208 secs 2024-04-26T22:03:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:27 smithi090 ceph-mon[29180]: pgmap v7372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:29 smithi119 ceph-mon[32752]: pgmap v7373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:29 smithi167 ceph-mon[32751]: pgmap v7373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:29 smithi090 ceph-mon[29180]: pgmap v7373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12492.168231 secs 2024-04-26T22:03:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:31 smithi119 ceph-mon[32752]: pgmap v7374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12492.168231 secs 2024-04-26T22:03:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:31 smithi167 ceph-mon[32751]: pgmap v7374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12492.168231 secs 2024-04-26T22:03:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:31 smithi090 ceph-mon[29180]: pgmap v7374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12492.909302 secs 2024-04-26T22:03:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12492.909302 secs 2024-04-26T22:03:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12492.909302 secs 2024-04-26T22:03:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:33 smithi119 ceph-mon[32752]: pgmap v7375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:33 smithi167 ceph-mon[32751]: pgmap v7375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:33 smithi090 ceph-mon[29180]: pgmap v7375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:35 smithi119 ceph-mon[32752]: pgmap v7376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:35 smithi167 ceph-mon[32751]: pgmap v7376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:35 smithi090 ceph-mon[29180]: pgmap v7376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12497.168315 secs 2024-04-26T22:03:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12497.168315 secs 2024-04-26T22:03:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12497.168315 secs 2024-04-26T22:03:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:37 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 12497.909422 secs 2024-04-26T22:03:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:37 smithi119 ceph-mon[32752]: slow request 11524.601187 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:03:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:37 smithi119 ceph-mon[32752]: pgmap v7377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:37 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 12497.909422 secs 2024-04-26T22:03:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:37 smithi167 ceph-mon[32751]: slow request 11524.601187 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:03:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:37 smithi167 ceph-mon[32751]: pgmap v7377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:37 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 12497.909422 secs 2024-04-26T22:03:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:37 smithi090 ceph-mon[29180]: slow request 11524.601187 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:03:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:37 smithi090 ceph-mon[29180]: pgmap v7377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:39 smithi119 ceph-mon[32752]: pgmap v7378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:39 smithi167 ceph-mon[32751]: pgmap v7378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:39 smithi090 ceph-mon[29180]: pgmap v7378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12502.168400 secs 2024-04-26T22:03:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:41 smithi119 ceph-mon[32752]: pgmap v7379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12502.168400 secs 2024-04-26T22:03:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:41 smithi167 ceph-mon[32751]: pgmap v7379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12502.168400 secs 2024-04-26T22:03:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:41 smithi090 ceph-mon[29180]: pgmap v7379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12502.909522 secs 2024-04-26T22:03:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12502.909522 secs 2024-04-26T22:03:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12502.909522 secs 2024-04-26T22:03:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:43 smithi167 ceph-mon[32751]: pgmap v7380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:43 smithi090 ceph-mon[29180]: pgmap v7380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:43 smithi119 ceph-mon[32752]: pgmap v7380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:45 smithi167 ceph-mon[32751]: pgmap v7381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:45 smithi090 ceph-mon[29180]: pgmap v7381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:45 smithi119 ceph-mon[32752]: pgmap v7381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12507.168482 secs 2024-04-26T22:03:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12507.168482 secs 2024-04-26T22:03:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12507.168482 secs 2024-04-26T22:03:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12507.909639 secs 2024-04-26T22:03:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:47 smithi167 ceph-mon[32751]: pgmap v7382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12507.909639 secs 2024-04-26T22:03:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:47 smithi090 ceph-mon[29180]: pgmap v7382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12507.909639 secs 2024-04-26T22:03:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:47 smithi119 ceph-mon[32752]: pgmap v7382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:49 smithi090 ceph-mon[29180]: pgmap v7383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:49 smithi119 ceph-mon[32752]: pgmap v7383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:49 smithi167 ceph-mon[32751]: pgmap v7383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:51.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12512.168566 secs 2024-04-26T22:03:51.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:51 smithi119 ceph-mon[32752]: pgmap v7384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12512.168566 secs 2024-04-26T22:03:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:51 smithi090 ceph-mon[29180]: pgmap v7384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12512.168566 secs 2024-04-26T22:03:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:51 smithi167 ceph-mon[32751]: pgmap v7384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12512.909718 secs 2024-04-26T22:03:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12512.909718 secs 2024-04-26T22:03:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12512.909718 secs 2024-04-26T22:03:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:53 smithi090 ceph-mon[29180]: pgmap v7385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:53 smithi119 ceph-mon[32752]: pgmap v7385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:53 smithi167 ceph-mon[32751]: pgmap v7385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:55 smithi090 ceph-mon[29180]: pgmap v7386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:55 smithi119 ceph-mon[32752]: pgmap v7386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:55 smithi167 ceph-mon[32751]: pgmap v7386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:03:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12517.168651 secs 2024-04-26T22:03:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12517.168651 secs 2024-04-26T22:03:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12517.168651 secs 2024-04-26T22:03:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12517.909778 secs 2024-04-26T22:03:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:57 smithi090 ceph-mon[29180]: pgmap v7387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12517.909778 secs 2024-04-26T22:03:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:57 smithi119 ceph-mon[32752]: pgmap v7387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12517.909778 secs 2024-04-26T22:03:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:57 smithi167 ceph-mon[32751]: pgmap v7387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:03:59 smithi090 ceph-mon[29180]: pgmap v7388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:03:59 smithi119 ceph-mon[32752]: pgmap v7388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:03:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:03:59 smithi167 ceph-mon[32751]: pgmap v7388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12522.168740 secs 2024-04-26T22:04:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:01 smithi090 ceph-mon[29180]: pgmap v7389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12522.168740 secs 2024-04-26T22:04:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:01 smithi119 ceph-mon[32752]: pgmap v7389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12522.168740 secs 2024-04-26T22:04:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:01 smithi167 ceph-mon[32751]: pgmap v7389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12522.909922 secs 2024-04-26T22:04:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12522.909922 secs 2024-04-26T22:04:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12522.909922 secs 2024-04-26T22:04:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:03 smithi090 ceph-mon[29180]: pgmap v7390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:03 smithi119 ceph-mon[32752]: pgmap v7390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:03 smithi167 ceph-mon[32751]: pgmap v7390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:04.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:04:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:04:04.181+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:04:04.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:04:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:04:04.181+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:04:04.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:04:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:04:04.181+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:04:04.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:04:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:04:04.181+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:04:04.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:04:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:04:04.181+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:04:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:05 smithi090 ceph-mon[29180]: pgmap v7391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:05 smithi119 ceph-mon[32752]: pgmap v7391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:05 smithi167 ceph-mon[32751]: pgmap v7391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:06 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 12527.168849 secs 2024-04-26T22:04:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:06 smithi090 ceph-mon[29180]: slow request 2881.577093 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:04:06.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:06 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 12527.168849 secs 2024-04-26T22:04:06.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:06 smithi167 ceph-mon[32751]: slow request 2881.577093 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:04:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:06 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 12527.168849 secs 2024-04-26T22:04:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:06 smithi119 ceph-mon[32752]: slow request 2881.577093 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:04:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:07 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 12527.910060 secs 2024-04-26T22:04:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:07 smithi090 ceph-mon[29180]: slow request 5763.071248 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:04:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:07 smithi090 ceph-mon[29180]: pgmap v7392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:07 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 12527.910060 secs 2024-04-26T22:04:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:07 smithi119 ceph-mon[32752]: slow request 5763.071248 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:04:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:07 smithi119 ceph-mon[32752]: pgmap v7392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:07 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 12527.910060 secs 2024-04-26T22:04:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:07 smithi167 ceph-mon[32751]: slow request 5763.071248 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:04:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:07 smithi167 ceph-mon[32751]: pgmap v7392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:09 smithi090 ceph-mon[29180]: pgmap v7393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:09 smithi119 ceph-mon[32752]: pgmap v7393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:09 smithi167 ceph-mon[32751]: pgmap v7393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:04:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12532.168934 secs 2024-04-26T22:04:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:11 smithi119 ceph-mon[32752]: pgmap v7394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T22:04:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:04:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12532.168934 secs 2024-04-26T22:04:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:11 smithi167 ceph-mon[32751]: pgmap v7394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T22:04:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:04:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12532.168934 secs 2024-04-26T22:04:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:11 smithi090 ceph-mon[29180]: pgmap v7394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-26T22:04:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:04:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:04:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:04:12.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12532.910121 secs 2024-04-26T22:04:12.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:12 smithi090 ceph-mon[29180]: pgmap v7395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T22:04:12.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:12 smithi090 ceph-mon[29180]: pgmap v7396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12532.910121 secs 2024-04-26T22:04:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:12 smithi119 ceph-mon[32752]: pgmap v7395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T22:04:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:12 smithi119 ceph-mon[32752]: pgmap v7396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12532.910121 secs 2024-04-26T22:04:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:12 smithi167 ceph-mon[32751]: pgmap v7395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-26T22:04:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:12 smithi167 ceph-mon[32751]: pgmap v7396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:14 smithi119 ceph-mon[32752]: pgmap v7397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T22:04:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:14 smithi167 ceph-mon[32751]: pgmap v7397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T22:04:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:14 smithi090 ceph-mon[29180]: pgmap v7397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T22:04:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:16 smithi119 ceph-mon[32752]: pgmap v7398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12537.169024 secs 2024-04-26T22:04:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:16 smithi167 ceph-mon[32751]: pgmap v7398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12537.169024 secs 2024-04-26T22:04:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:16 smithi090 ceph-mon[29180]: pgmap v7398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12537.169024 secs 2024-04-26T22:04:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12537.910240 secs 2024-04-26T22:04:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12537.910240 secs 2024-04-26T22:04:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12537.910240 secs 2024-04-26T22:04:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:18 smithi119 ceph-mon[32752]: pgmap v7399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:18 smithi167 ceph-mon[32751]: pgmap v7399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:18 smithi090 ceph-mon[29180]: pgmap v7399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-26T22:04:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:20 smithi119 ceph-mon[32752]: pgmap v7400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:04:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:20 smithi167 ceph-mon[32751]: pgmap v7400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:04:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:20 smithi090 ceph-mon[29180]: pgmap v7400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:04:21.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12542.169111 secs 2024-04-26T22:04:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12542.169111 secs 2024-04-26T22:04:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12542.169111 secs 2024-04-26T22:04:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12542.910312 secs 2024-04-26T22:04:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:22 smithi119 ceph-mon[32752]: pgmap v7401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:04:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12542.910312 secs 2024-04-26T22:04:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:22 smithi167 ceph-mon[32751]: pgmap v7401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:04:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12542.910312 secs 2024-04-26T22:04:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:22 smithi090 ceph-mon[29180]: pgmap v7401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:04:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:23 smithi119 ceph-mon[32752]: pgmap v7402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:23 smithi167 ceph-mon[32751]: pgmap v7402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:23 smithi090 ceph-mon[29180]: pgmap v7402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:25 smithi119 ceph-mon[32752]: pgmap v7403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12547.169195 secs 2024-04-26T22:04:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:25 smithi167 ceph-mon[32751]: pgmap v7403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12547.169195 secs 2024-04-26T22:04:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:25 smithi090 ceph-mon[29180]: pgmap v7403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12547.169195 secs 2024-04-26T22:04:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12547.910453 secs 2024-04-26T22:04:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12547.910453 secs 2024-04-26T22:04:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12547.910453 secs 2024-04-26T22:04:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:27 smithi119 ceph-mon[32752]: pgmap v7404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:27 smithi167 ceph-mon[32751]: pgmap v7404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:27 smithi090 ceph-mon[29180]: pgmap v7404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:29 smithi119 ceph-mon[32752]: pgmap v7405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:29 smithi167 ceph-mon[32751]: pgmap v7405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:29 smithi090 ceph-mon[29180]: pgmap v7405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12552.169281 secs 2024-04-26T22:04:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12552.169281 secs 2024-04-26T22:04:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12552.169281 secs 2024-04-26T22:04:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12552.910526 secs 2024-04-26T22:04:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:31 smithi119 ceph-mon[32752]: pgmap v7406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12552.910526 secs 2024-04-26T22:04:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:31 smithi167 ceph-mon[32751]: pgmap v7406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12552.910526 secs 2024-04-26T22:04:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:31 smithi090 ceph-mon[29180]: pgmap v7406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:33 smithi119 ceph-mon[32752]: pgmap v7407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:33 smithi167 ceph-mon[32751]: pgmap v7407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:33 smithi090 ceph-mon[29180]: pgmap v7407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:35 smithi119 ceph-mon[32752]: pgmap v7408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12557.169368 secs 2024-04-26T22:04:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:35 smithi167 ceph-mon[32751]: pgmap v7408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12557.169368 secs 2024-04-26T22:04:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:35 smithi090 ceph-mon[29180]: pgmap v7408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12557.169368 secs 2024-04-26T22:04:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12557.910587 secs 2024-04-26T22:04:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12557.910587 secs 2024-04-26T22:04:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12557.910587 secs 2024-04-26T22:04:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:37 smithi119 ceph-mon[32752]: pgmap v7409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:37 smithi167 ceph-mon[32751]: pgmap v7409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:37 smithi090 ceph-mon[29180]: pgmap v7409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:39 smithi119 ceph-mon[32752]: pgmap v7410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:39 smithi167 ceph-mon[32751]: pgmap v7410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:39 smithi090 ceph-mon[29180]: pgmap v7410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12562.169430 secs 2024-04-26T22:04:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12562.169430 secs 2024-04-26T22:04:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12562.169430 secs 2024-04-26T22:04:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12562.910656 secs 2024-04-26T22:04:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:41 smithi119 ceph-mon[32752]: pgmap v7411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12562.910656 secs 2024-04-26T22:04:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:41 smithi167 ceph-mon[32751]: pgmap v7411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12562.910656 secs 2024-04-26T22:04:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:41 smithi090 ceph-mon[29180]: pgmap v7411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:43 smithi119 ceph-mon[32752]: pgmap v7412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:43 smithi167 ceph-mon[32751]: pgmap v7412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:43 smithi090 ceph-mon[29180]: pgmap v7412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:45 smithi167 ceph-mon[32751]: pgmap v7413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12567.169515 secs 2024-04-26T22:04:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:45 smithi090 ceph-mon[29180]: pgmap v7413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12567.169515 secs 2024-04-26T22:04:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:45 smithi119 ceph-mon[32752]: pgmap v7413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12567.169515 secs 2024-04-26T22:04:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12567.910743 secs 2024-04-26T22:04:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12567.910743 secs 2024-04-26T22:04:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12567.910743 secs 2024-04-26T22:04:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:47 smithi167 ceph-mon[32751]: pgmap v7414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:47 smithi090 ceph-mon[29180]: pgmap v7414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:47 smithi119 ceph-mon[32752]: pgmap v7414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:49 smithi167 ceph-mon[32751]: pgmap v7415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:49 smithi090 ceph-mon[29180]: pgmap v7415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:49 smithi119 ceph-mon[32752]: pgmap v7415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12572.169599 secs 2024-04-26T22:04:51.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12572.169599 secs 2024-04-26T22:04:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12572.169599 secs 2024-04-26T22:04:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12572.910880 secs 2024-04-26T22:04:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:51 smithi090 ceph-mon[29180]: pgmap v7416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12572.910880 secs 2024-04-26T22:04:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:51 smithi119 ceph-mon[32752]: pgmap v7416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12572.910880 secs 2024-04-26T22:04:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:51 smithi167 ceph-mon[32751]: pgmap v7416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:53 smithi090 ceph-mon[29180]: pgmap v7417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:53 smithi119 ceph-mon[32752]: pgmap v7417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:53 smithi167 ceph-mon[32751]: pgmap v7417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:55 smithi090 ceph-mon[29180]: pgmap v7418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12577.169684 secs 2024-04-26T22:04:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:55 smithi119 ceph-mon[32752]: pgmap v7418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12577.169684 secs 2024-04-26T22:04:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:55 smithi167 ceph-mon[32751]: pgmap v7418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:04:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12577.169684 secs 2024-04-26T22:04:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12577.910956 secs 2024-04-26T22:04:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12577.910956 secs 2024-04-26T22:04:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12577.910956 secs 2024-04-26T22:04:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:57 smithi090 ceph-mon[29180]: pgmap v7419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:57 smithi119 ceph-mon[32752]: pgmap v7419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:04:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:57 smithi167 ceph-mon[32751]: pgmap v7419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:04:59 smithi090 ceph-mon[29180]: pgmap v7420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:04:59 smithi119 ceph-mon[32752]: pgmap v7420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:04:59 smithi167 ceph-mon[32751]: pgmap v7420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:01.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12582.169769 secs 2024-04-26T22:05:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12582.169769 secs 2024-04-26T22:05:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12582.169769 secs 2024-04-26T22:05:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12582.911049 secs 2024-04-26T22:05:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:01 smithi090 ceph-mon[29180]: pgmap v7421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12582.911049 secs 2024-04-26T22:05:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:01 smithi119 ceph-mon[32752]: pgmap v7421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12582.911049 secs 2024-04-26T22:05:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:01 smithi167 ceph-mon[32751]: pgmap v7421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:03 smithi090 ceph-mon[29180]: pgmap v7422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:03 smithi119 ceph-mon[32752]: pgmap v7422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:03 smithi167 ceph-mon[32751]: pgmap v7422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:05 smithi167 ceph-mon[32751]: pgmap v7423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12587.169854 secs 2024-04-26T22:05:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:05 smithi090 ceph-mon[29180]: pgmap v7423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12587.169854 secs 2024-04-26T22:05:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:05 smithi119 ceph-mon[32752]: pgmap v7423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12587.169854 secs 2024-04-26T22:05:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12587.911146 secs 2024-04-26T22:05:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12587.911146 secs 2024-04-26T22:05:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12587.911146 secs 2024-04-26T22:05:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:07 smithi090 ceph-mon[29180]: pgmap v7424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:05:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:07 smithi119 ceph-mon[32752]: pgmap v7424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:05:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:07 smithi167 ceph-mon[32751]: pgmap v7424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:05:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:09 smithi090 ceph-mon[29180]: pgmap v7425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:10.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:09 smithi119 ceph-mon[32752]: pgmap v7425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:09 smithi167 ceph-mon[32751]: pgmap v7425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12592.169944 secs 2024-04-26T22:05:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12592.169944 secs 2024-04-26T22:05:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12592.169944 secs 2024-04-26T22:05:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:05:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12592.911211 secs 2024-04-26T22:05:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:11 smithi090 ceph-mon[29180]: pgmap v7426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:05:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:05:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12592.911211 secs 2024-04-26T22:05:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:11 smithi119 ceph-mon[32752]: pgmap v7426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:05:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:05:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12592.911211 secs 2024-04-26T22:05:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:11 smithi167 ceph-mon[32751]: pgmap v7426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:05:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:05:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:05:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:05:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:12 smithi090 ceph-mon[29180]: pgmap v7427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:12 smithi090 ceph-mon[29180]: pgmap v7428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:12 smithi119 ceph-mon[32752]: pgmap v7427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:12 smithi119 ceph-mon[32752]: pgmap v7428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:12 smithi167 ceph-mon[32751]: pgmap v7427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:12 smithi167 ceph-mon[32751]: pgmap v7428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:05:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:14 smithi119 ceph-mon[32752]: pgmap v7429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:14 smithi167 ceph-mon[32751]: pgmap v7429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:14 smithi090 ceph-mon[29180]: pgmap v7429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12597.170056 secs 2024-04-26T22:05:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12597.170056 secs 2024-04-26T22:05:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12597.170056 secs 2024-04-26T22:05:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:16 smithi090 ceph-mon[29180]: pgmap v7430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12597.911308 secs 2024-04-26T22:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:16 smithi119 ceph-mon[32752]: pgmap v7430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12597.911308 secs 2024-04-26T22:05:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:16 smithi167 ceph-mon[32751]: pgmap v7430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12597.911308 secs 2024-04-26T22:05:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:18 smithi119 ceph-mon[32752]: pgmap v7431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:05:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:18 smithi167 ceph-mon[32751]: pgmap v7431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:05:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:18 smithi090 ceph-mon[29180]: pgmap v7431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:05:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:20 smithi119 ceph-mon[32752]: pgmap v7432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12602.170122 secs 2024-04-26T22:05:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:20 smithi167 ceph-mon[32751]: pgmap v7432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12602.170122 secs 2024-04-26T22:05:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:20 smithi090 ceph-mon[29180]: pgmap v7432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:05:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12602.170122 secs 2024-04-26T22:05:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12602.911402 secs 2024-04-26T22:05:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12602.911402 secs 2024-04-26T22:05:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12602.911402 secs 2024-04-26T22:05:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:22 smithi119 ceph-mon[32752]: pgmap v7433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:05:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:22 smithi167 ceph-mon[32751]: pgmap v7433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:05:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:22 smithi090 ceph-mon[29180]: pgmap v7433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:05:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:24 smithi119 ceph-mon[32752]: pgmap v7434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:24 smithi167 ceph-mon[32751]: pgmap v7434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:24 smithi090 ceph-mon[29180]: pgmap v7434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12607.170209 secs 2024-04-26T22:05:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12607.170209 secs 2024-04-26T22:05:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12607.170209 secs 2024-04-26T22:05:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:26 smithi119 ceph-mon[32752]: pgmap v7435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12607.911517 secs 2024-04-26T22:05:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:26 smithi167 ceph-mon[32751]: pgmap v7435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12607.911517 secs 2024-04-26T22:05:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:26 smithi090 ceph-mon[29180]: pgmap v7435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12607.911517 secs 2024-04-26T22:05:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:28 smithi119 ceph-mon[32752]: pgmap v7436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:28 smithi167 ceph-mon[32751]: pgmap v7436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:28 smithi090 ceph-mon[29180]: pgmap v7436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:30 smithi119 ceph-mon[32752]: pgmap v7437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12612.170293 secs 2024-04-26T22:05:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:30 smithi167 ceph-mon[32751]: pgmap v7437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12612.170293 secs 2024-04-26T22:05:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:30 smithi090 ceph-mon[29180]: pgmap v7437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12612.170293 secs 2024-04-26T22:05:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12612.911607 secs 2024-04-26T22:05:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12612.911607 secs 2024-04-26T22:05:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12612.911607 secs 2024-04-26T22:05:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:32 smithi119 ceph-mon[32752]: pgmap v7438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:32 smithi167 ceph-mon[32751]: pgmap v7438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:32 smithi090 ceph-mon[29180]: pgmap v7438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:34 smithi119 ceph-mon[32752]: pgmap v7439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:34 smithi167 ceph-mon[32751]: pgmap v7439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:34 smithi090 ceph-mon[29180]: pgmap v7439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:36.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12617.170377 secs 2024-04-26T22:05:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12617.170377 secs 2024-04-26T22:05:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12617.170377 secs 2024-04-26T22:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:36 smithi119 ceph-mon[32752]: pgmap v7440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:36 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 12617.911749 secs 2024-04-26T22:05:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:36 smithi119 ceph-mon[32752]: slow request 721.651881 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:05:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:36 smithi167 ceph-mon[32751]: pgmap v7440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:36 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 12617.911749 secs 2024-04-26T22:05:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:36 smithi167 ceph-mon[32751]: slow request 721.651881 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:05:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:36 smithi090 ceph-mon[29180]: pgmap v7440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:36 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 12617.911749 secs 2024-04-26T22:05:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:36 smithi090 ceph-mon[29180]: slow request 721.651881 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:05:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:38 smithi119 ceph-mon[32752]: pgmap v7441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:38 smithi167 ceph-mon[32751]: pgmap v7441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:38 smithi090 ceph-mon[29180]: pgmap v7441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:40 smithi119 ceph-mon[32752]: pgmap v7442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12622.170467 secs 2024-04-26T22:05:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:40 smithi167 ceph-mon[32751]: pgmap v7442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12622.170467 secs 2024-04-26T22:05:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:40 smithi090 ceph-mon[29180]: pgmap v7442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12622.170467 secs 2024-04-26T22:05:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12622.911895 secs 2024-04-26T22:05:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12622.911895 secs 2024-04-26T22:05:42.367 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12622.911895 secs 2024-04-26T22:05:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:42 smithi119 ceph-mon[32752]: pgmap v7443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:42 smithi167 ceph-mon[32751]: pgmap v7443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:42 smithi090 ceph-mon[29180]: pgmap v7443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:44 smithi119 ceph-mon[32752]: pgmap v7444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:44 smithi167 ceph-mon[32751]: pgmap v7444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:44 smithi090 ceph-mon[29180]: pgmap v7444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:45 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 12627.170552 secs 2024-04-26T22:05:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:45 smithi119 ceph-mon[32752]: slow request 1443.469269 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:05:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:45 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 12627.170552 secs 2024-04-26T22:05:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:45 smithi167 ceph-mon[32751]: slow request 1443.469269 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:05:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:45 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 12627.170552 secs 2024-04-26T22:05:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:45 smithi090 ceph-mon[29180]: slow request 1443.469269 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:05:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:46 smithi119 ceph-mon[32752]: pgmap v7445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12627.911999 secs 2024-04-26T22:05:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:46 smithi167 ceph-mon[32751]: pgmap v7445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12627.911999 secs 2024-04-26T22:05:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:46 smithi090 ceph-mon[29180]: pgmap v7445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12627.911999 secs 2024-04-26T22:05:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:48 smithi119 ceph-mon[32752]: pgmap v7446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:48 smithi167 ceph-mon[32751]: pgmap v7446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:48 smithi090 ceph-mon[29180]: pgmap v7446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:50 smithi119 ceph-mon[32752]: pgmap v7447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12632.170636 secs 2024-04-26T22:05:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:50 smithi167 ceph-mon[32751]: pgmap v7447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12632.170636 secs 2024-04-26T22:05:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:50 smithi090 ceph-mon[29180]: pgmap v7447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12632.170636 secs 2024-04-26T22:05:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12632.912061 secs 2024-04-26T22:05:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12632.912061 secs 2024-04-26T22:05:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12632.912061 secs 2024-04-26T22:05:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:52 smithi119 ceph-mon[32752]: pgmap v7448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:52 smithi167 ceph-mon[32751]: pgmap v7448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:52 smithi090 ceph-mon[29180]: pgmap v7448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:54 smithi119 ceph-mon[32752]: pgmap v7449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:54 smithi167 ceph-mon[32751]: pgmap v7449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:54 smithi090 ceph-mon[29180]: pgmap v7449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:05:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12637.170720 secs 2024-04-26T22:05:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12637.170720 secs 2024-04-26T22:05:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12637.170720 secs 2024-04-26T22:05:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:56 smithi119 ceph-mon[32752]: pgmap v7450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12637.912136 secs 2024-04-26T22:05:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:56 smithi167 ceph-mon[32751]: pgmap v7450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12637.912136 secs 2024-04-26T22:05:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:56 smithi090 ceph-mon[29180]: pgmap v7450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12637.912136 secs 2024-04-26T22:05:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:05:58 smithi119 ceph-mon[32752]: pgmap v7451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:05:58 smithi167 ceph-mon[32751]: pgmap v7451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:05:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:05:58 smithi090 ceph-mon[29180]: pgmap v7451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:00 smithi119 ceph-mon[32752]: pgmap v7452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12642.170826 secs 2024-04-26T22:06:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:00 smithi167 ceph-mon[32751]: pgmap v7452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12642.170826 secs 2024-04-26T22:06:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:00 smithi090 ceph-mon[29180]: pgmap v7452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12642.170826 secs 2024-04-26T22:06:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12642.912202 secs 2024-04-26T22:06:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12642.912202 secs 2024-04-26T22:06:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12642.912202 secs 2024-04-26T22:06:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:02 smithi119 ceph-mon[32752]: pgmap v7453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:02 smithi167 ceph-mon[32751]: pgmap v7453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:02 smithi090 ceph-mon[29180]: pgmap v7453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:05 smithi119 ceph-mon[32752]: pgmap v7454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:05 smithi167 ceph-mon[32751]: pgmap v7454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:05 smithi090 ceph-mon[29180]: pgmap v7454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12647.170916 secs 2024-04-26T22:06:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12647.170916 secs 2024-04-26T22:06:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12647.170916 secs 2024-04-26T22:06:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:07 smithi119 ceph-mon[32752]: pgmap v7455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12647.912311 secs 2024-04-26T22:06:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:07 smithi167 ceph-mon[32751]: pgmap v7455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12647.912311 secs 2024-04-26T22:06:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:07 smithi090 ceph-mon[29180]: pgmap v7455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12647.912311 secs 2024-04-26T22:06:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:09 smithi119 ceph-mon[32752]: pgmap v7456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:09 smithi167 ceph-mon[32751]: pgmap v7456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:09 smithi090 ceph-mon[29180]: pgmap v7456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:11 smithi119 ceph-mon[32752]: pgmap v7457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12652.170997 secs 2024-04-26T22:06:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:11 smithi167 ceph-mon[32751]: pgmap v7457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12652.170997 secs 2024-04-26T22:06:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:11 smithi090 ceph-mon[29180]: pgmap v7457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12652.170997 secs 2024-04-26T22:06:12.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12652.912464 secs 2024-04-26T22:06:12.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12652.912464 secs 2024-04-26T22:06:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12652.912464 secs 2024-04-26T22:06:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:06:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:13 smithi119 ceph-mon[32752]: pgmap v7458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:06:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:13 smithi167 ceph-mon[32751]: pgmap v7458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:06:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:13 smithi090 ceph-mon[29180]: pgmap v7458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:06:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:06:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:06:14.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:14 smithi119 ceph-mon[32752]: pgmap v7459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:14 smithi167 ceph-mon[32751]: pgmap v7459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:14 smithi090 ceph-mon[29180]: pgmap v7459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:16 smithi119 ceph-mon[32752]: pgmap v7460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12657.171077 secs 2024-04-26T22:06:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:16 smithi167 ceph-mon[32751]: pgmap v7460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12657.171077 secs 2024-04-26T22:06:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:16 smithi090 ceph-mon[29180]: pgmap v7460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12657.171077 secs 2024-04-26T22:06:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12657.912585 secs 2024-04-26T22:06:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12657.912585 secs 2024-04-26T22:06:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12657.912585 secs 2024-04-26T22:06:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:18 smithi119 ceph-mon[32752]: pgmap v7461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:18 smithi167 ceph-mon[32751]: pgmap v7461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:18 smithi090 ceph-mon[29180]: pgmap v7461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:20 smithi119 ceph-mon[32752]: pgmap v7462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 2 op/s 2024-04-26T22:06:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:20 smithi167 ceph-mon[32751]: pgmap v7462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 2 op/s 2024-04-26T22:06:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:20 smithi090 ceph-mon[29180]: pgmap v7462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 2 op/s 2024-04-26T22:06:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12662.171138 secs 2024-04-26T22:06:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12662.171138 secs 2024-04-26T22:06:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12662.171138 secs 2024-04-26T22:06:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:22 smithi119 ceph-mon[32752]: pgmap v7463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12662.912684 secs 2024-04-26T22:06:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:22 smithi167 ceph-mon[32751]: pgmap v7463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12662.912684 secs 2024-04-26T22:06:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:22 smithi090 ceph-mon[29180]: pgmap v7463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12662.912684 secs 2024-04-26T22:06:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:24 smithi119 ceph-mon[32752]: pgmap v7464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:24 smithi167 ceph-mon[32751]: pgmap v7464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:24 smithi090 ceph-mon[29180]: pgmap v7464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:06:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:26 smithi119 ceph-mon[32752]: pgmap v7465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12667.171222 secs 2024-04-26T22:06:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:26 smithi167 ceph-mon[32751]: pgmap v7465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12667.171222 secs 2024-04-26T22:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:26 smithi090 ceph-mon[29180]: pgmap v7465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12667.171222 secs 2024-04-26T22:06:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12667.912807 secs 2024-04-26T22:06:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12667.912807 secs 2024-04-26T22:06:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12667.912807 secs 2024-04-26T22:06:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:28 smithi167 ceph-mon[32751]: pgmap v7466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:28 smithi090 ceph-mon[29180]: pgmap v7466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:28 smithi119 ceph-mon[32752]: pgmap v7466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:30 smithi167 ceph-mon[32751]: pgmap v7467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:30 smithi090 ceph-mon[29180]: pgmap v7467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:30 smithi119 ceph-mon[32752]: pgmap v7467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12672.171306 secs 2024-04-26T22:06:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12672.171306 secs 2024-04-26T22:06:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12672.171306 secs 2024-04-26T22:06:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:32 smithi090 ceph-mon[29180]: pgmap v7468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12672.912928 secs 2024-04-26T22:06:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:32 smithi119 ceph-mon[32752]: pgmap v7468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12672.912928 secs 2024-04-26T22:06:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:32 smithi167 ceph-mon[32751]: pgmap v7468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12672.912928 secs 2024-04-26T22:06:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:34 smithi090 ceph-mon[29180]: pgmap v7469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:34 smithi119 ceph-mon[32752]: pgmap v7469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:34 smithi167 ceph-mon[32751]: pgmap v7469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:36.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:36 smithi167 ceph-mon[32751]: pgmap v7470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:36.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12677.171396 secs 2024-04-26T22:06:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:36 smithi090 ceph-mon[29180]: pgmap v7470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12677.171396 secs 2024-04-26T22:06:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:36 smithi119 ceph-mon[32752]: pgmap v7470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12677.171396 secs 2024-04-26T22:06:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12677.913052 secs 2024-04-26T22:06:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12677.913052 secs 2024-04-26T22:06:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12677.913052 secs 2024-04-26T22:06:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:38 smithi090 ceph-mon[29180]: pgmap v7471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:38 smithi119 ceph-mon[32752]: pgmap v7471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:38 smithi167 ceph-mon[32751]: pgmap v7471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:40 smithi090 ceph-mon[29180]: pgmap v7472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:40 smithi119 ceph-mon[32752]: pgmap v7472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:40 smithi167 ceph-mon[32751]: pgmap v7472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12682.171482 secs 2024-04-26T22:06:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12682.171482 secs 2024-04-26T22:06:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12682.171482 secs 2024-04-26T22:06:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:42 smithi090 ceph-mon[29180]: pgmap v7473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12682.913173 secs 2024-04-26T22:06:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:42 smithi119 ceph-mon[32752]: pgmap v7473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12682.913173 secs 2024-04-26T22:06:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:42 smithi167 ceph-mon[32751]: pgmap v7473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12682.913173 secs 2024-04-26T22:06:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:44 smithi090 ceph-mon[29180]: pgmap v7474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:44 smithi119 ceph-mon[32752]: pgmap v7474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:44 smithi167 ceph-mon[32751]: pgmap v7474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:46 smithi090 ceph-mon[29180]: pgmap v7475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12687.171565 secs 2024-04-26T22:06:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:46 smithi119 ceph-mon[32752]: pgmap v7475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12687.171565 secs 2024-04-26T22:06:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:46 smithi167 ceph-mon[32751]: pgmap v7475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12687.171565 secs 2024-04-26T22:06:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12687.913324 secs 2024-04-26T22:06:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12687.913324 secs 2024-04-26T22:06:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12687.913324 secs 2024-04-26T22:06:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:48 smithi090 ceph-mon[29180]: pgmap v7476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:48 smithi119 ceph-mon[32752]: pgmap v7476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:48 smithi167 ceph-mon[32751]: pgmap v7476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:50 smithi090 ceph-mon[29180]: pgmap v7477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:50 smithi119 ceph-mon[32752]: pgmap v7477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:50 smithi167 ceph-mon[32751]: pgmap v7477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:06:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12692.171651 secs 2024-04-26T22:06:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12692.171651 secs 2024-04-26T22:06:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12692.171651 secs 2024-04-26T22:06:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:52 smithi090 ceph-mon[29180]: pgmap v7478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12692.913452 secs 2024-04-26T22:06:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:52 smithi119 ceph-mon[32752]: pgmap v7478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12692.913452 secs 2024-04-26T22:06:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:52 smithi167 ceph-mon[32751]: pgmap v7478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12692.913452 secs 2024-04-26T22:06:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:54 smithi090 ceph-mon[29180]: pgmap v7479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:54 smithi119 ceph-mon[32752]: pgmap v7479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:54 smithi167 ceph-mon[32751]: pgmap v7479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:56 smithi090 ceph-mon[29180]: pgmap v7480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12697.171735 secs 2024-04-26T22:06:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:56 smithi119 ceph-mon[32752]: pgmap v7480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12697.171735 secs 2024-04-26T22:06:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:56 smithi167 ceph-mon[32751]: pgmap v7480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:06:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12697.171735 secs 2024-04-26T22:06:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12697.913595 secs 2024-04-26T22:06:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12697.913595 secs 2024-04-26T22:06:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12697.913595 secs 2024-04-26T22:06:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:06:58 smithi119 ceph-mon[32752]: pgmap v7481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:06:58 smithi167 ceph-mon[32751]: pgmap v7481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:06:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:06:58 smithi090 ceph-mon[29180]: pgmap v7481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:00 smithi119 ceph-mon[32752]: pgmap v7482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:07:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:00 smithi167 ceph-mon[32751]: pgmap v7482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:07:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:00 smithi090 ceph-mon[29180]: pgmap v7482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:07:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12702.171842 secs 2024-04-26T22:07:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12702.171842 secs 2024-04-26T22:07:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12702.171842 secs 2024-04-26T22:07:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:02 smithi119 ceph-mon[32752]: pgmap v7483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12702.913673 secs 2024-04-26T22:07:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:02 smithi167 ceph-mon[32751]: pgmap v7483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12702.913673 secs 2024-04-26T22:07:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:02 smithi090 ceph-mon[29180]: pgmap v7483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12702.913673 secs 2024-04-26T22:07:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:04 smithi119 ceph-mon[32752]: pgmap v7484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:04 smithi167 ceph-mon[32751]: pgmap v7484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:04 smithi090 ceph-mon[29180]: pgmap v7484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:06 smithi119 ceph-mon[32752]: pgmap v7485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12707.171927 secs 2024-04-26T22:07:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:06 smithi167 ceph-mon[32751]: pgmap v7485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12707.171927 secs 2024-04-26T22:07:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:06 smithi090 ceph-mon[29180]: pgmap v7485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12707.171927 secs 2024-04-26T22:07:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12707.913793 secs 2024-04-26T22:07:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12707.913793 secs 2024-04-26T22:07:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12707.913793 secs 2024-04-26T22:07:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:08 smithi119 ceph-mon[32752]: pgmap v7486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:08 smithi167 ceph-mon[32751]: pgmap v7486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:08 smithi090 ceph-mon[29180]: pgmap v7486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:10 smithi119 ceph-mon[32752]: pgmap v7487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:07:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:10 smithi167 ceph-mon[32751]: pgmap v7487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:07:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:10 smithi090 ceph-mon[29180]: pgmap v7487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:07:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12712.172011 secs 2024-04-26T22:07:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12712.172011 secs 2024-04-26T22:07:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12712.172011 secs 2024-04-26T22:07:12.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:12 smithi090 ceph-mon[29180]: pgmap v7488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:12.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12712.913921 secs 2024-04-26T22:07:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:12 smithi119 ceph-mon[32752]: pgmap v7488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12712.913921 secs 2024-04-26T22:07:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:12 smithi167 ceph-mon[32751]: pgmap v7488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12712.913921 secs 2024-04-26T22:07:13.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:13.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.456 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:07:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:14 smithi119 ceph-mon[32752]: pgmap v7489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:14 smithi167 ceph-mon[32751]: pgmap v7489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:14 smithi090 ceph-mon[29180]: pgmap v7489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: pgmap v7490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12717.172102 secs 2024-04-26T22:07:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: pgmap v7490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12717.172102 secs 2024-04-26T22:07:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: pgmap v7490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12717.172102 secs 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:07:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:07:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12717.914031 secs 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:07:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:07:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:07:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:07:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: pgmap v7491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 B/s rd, 1 op/s 2024-04-26T22:07:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: pgmap v7492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:07:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12717.914031 secs 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:07:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:07:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: pgmap v7491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 B/s rd, 1 op/s 2024-04-26T22:07:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: pgmap v7492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:07:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:07:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:07:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12717.914031 secs 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:07:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:07:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:07:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: pgmap v7491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 890 B/s rd, 1 op/s 2024-04-26T22:07:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: pgmap v7492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:07:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:18 smithi090 ceph-mon[29180]: pgmap v7493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T22:07:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:18 smithi119 ceph-mon[32752]: pgmap v7493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T22:07:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:18 smithi167 ceph-mon[32751]: pgmap v7493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T22:07:21.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:20 smithi119 ceph-mon[32752]: pgmap v7494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:21.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12722.172187 secs 2024-04-26T22:07:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:20 smithi090 ceph-mon[29180]: pgmap v7494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12722.172187 secs 2024-04-26T22:07:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:20 smithi167 ceph-mon[32751]: pgmap v7494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12722.172187 secs 2024-04-26T22:07:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12722.914125 secs 2024-04-26T22:07:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12722.914125 secs 2024-04-26T22:07:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12722.914125 secs 2024-04-26T22:07:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:22 smithi090 ceph-mon[29180]: pgmap v7495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:22 smithi119 ceph-mon[32752]: pgmap v7495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:22 smithi167 ceph-mon[32751]: pgmap v7495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:24 smithi090 ceph-mon[29180]: pgmap v7496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:24 smithi119 ceph-mon[32752]: pgmap v7496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:24 smithi167 ceph-mon[32751]: pgmap v7496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:07:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12727.172273 secs 2024-04-26T22:07:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12727.172273 secs 2024-04-26T22:07:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12727.172273 secs 2024-04-26T22:07:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12727.914239 secs 2024-04-26T22:07:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:26 smithi090 ceph-mon[29180]: pgmap v7497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:07:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12727.914239 secs 2024-04-26T22:07:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:26 smithi119 ceph-mon[32752]: pgmap v7497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:07:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12727.914239 secs 2024-04-26T22:07:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:26 smithi167 ceph-mon[32751]: pgmap v7497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:07:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:28 smithi090 ceph-mon[29180]: pgmap v7498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:28 smithi119 ceph-mon[32752]: pgmap v7498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:28 smithi167 ceph-mon[32751]: pgmap v7498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:30 smithi090 ceph-mon[29180]: pgmap v7499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12732.172355 secs 2024-04-26T22:07:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:30 smithi119 ceph-mon[32752]: pgmap v7499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12732.172355 secs 2024-04-26T22:07:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:30 smithi167 ceph-mon[32751]: pgmap v7499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12732.172355 secs 2024-04-26T22:07:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12732.914311 secs 2024-04-26T22:07:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12732.914311 secs 2024-04-26T22:07:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12732.914311 secs 2024-04-26T22:07:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:32 smithi090 ceph-mon[29180]: pgmap v7500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:32 smithi119 ceph-mon[32752]: pgmap v7500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:32 smithi167 ceph-mon[32751]: pgmap v7500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:34 smithi090 ceph-mon[29180]: pgmap v7501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:34 smithi119 ceph-mon[32752]: pgmap v7501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:34 smithi167 ceph-mon[32751]: pgmap v7501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12737.172442 secs 2024-04-26T22:07:36.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12737.172442 secs 2024-04-26T22:07:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12737.172442 secs 2024-04-26T22:07:37.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12737.914417 secs 2024-04-26T22:07:37.151 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:36 smithi090 ceph-mon[29180]: pgmap v7502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:37.234 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12737.914417 secs 2024-04-26T22:07:37.235 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:36 smithi167 ceph-mon[32751]: pgmap v7502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12737.914417 secs 2024-04-26T22:07:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:36 smithi119 ceph-mon[32752]: pgmap v7502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:38 smithi119 ceph-mon[32752]: pgmap v7503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:38 smithi167 ceph-mon[32751]: pgmap v7503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:38 smithi090 ceph-mon[29180]: pgmap v7503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:40 smithi119 ceph-mon[32752]: pgmap v7504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12742.172527 secs 2024-04-26T22:07:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:40 smithi167 ceph-mon[32751]: pgmap v7504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12742.172527 secs 2024-04-26T22:07:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:40 smithi090 ceph-mon[29180]: pgmap v7504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12742.172527 secs 2024-04-26T22:07:42.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12742.914479 secs 2024-04-26T22:07:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12742.914479 secs 2024-04-26T22:07:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12742.914479 secs 2024-04-26T22:07:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:42 smithi119 ceph-mon[32752]: pgmap v7505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:42 smithi167 ceph-mon[32751]: pgmap v7505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:42 smithi090 ceph-mon[29180]: pgmap v7505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:44 smithi119 ceph-mon[32752]: pgmap v7506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:44 smithi167 ceph-mon[32751]: pgmap v7506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:44 smithi090 ceph-mon[29180]: pgmap v7506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12747.172609 secs 2024-04-26T22:07:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12747.172609 secs 2024-04-26T22:07:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12747.172609 secs 2024-04-26T22:07:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12747.914543 secs 2024-04-26T22:07:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:46 smithi119 ceph-mon[32752]: pgmap v7507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12747.914543 secs 2024-04-26T22:07:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:46 smithi167 ceph-mon[32751]: pgmap v7507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12747.914543 secs 2024-04-26T22:07:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:46 smithi090 ceph-mon[29180]: pgmap v7507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:48 smithi119 ceph-mon[32752]: pgmap v7508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:48 smithi167 ceph-mon[32751]: pgmap v7508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:48 smithi090 ceph-mon[29180]: pgmap v7508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:50 smithi119 ceph-mon[32752]: pgmap v7509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12752.172695 secs 2024-04-26T22:07:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:50 smithi167 ceph-mon[32751]: pgmap v7509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12752.172695 secs 2024-04-26T22:07:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:50 smithi090 ceph-mon[29180]: pgmap v7509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12752.172695 secs 2024-04-26T22:07:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12752.914673 secs 2024-04-26T22:07:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12752.914673 secs 2024-04-26T22:07:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12752.914673 secs 2024-04-26T22:07:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:52 smithi119 ceph-mon[32752]: pgmap v7510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:52 smithi167 ceph-mon[32751]: pgmap v7510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:53.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:52 smithi090 ceph-mon[29180]: pgmap v7510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:54 smithi119 ceph-mon[32752]: pgmap v7511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:54 smithi167 ceph-mon[32751]: pgmap v7511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:54 smithi090 ceph-mon[29180]: pgmap v7511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:07:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12757.172777 secs 2024-04-26T22:07:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12757.172777 secs 2024-04-26T22:07:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12757.172777 secs 2024-04-26T22:07:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12757.914776 secs 2024-04-26T22:07:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:56 smithi119 ceph-mon[32752]: pgmap v7512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12757.914776 secs 2024-04-26T22:07:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:56 smithi167 ceph-mon[32751]: pgmap v7512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12757.914776 secs 2024-04-26T22:07:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:56 smithi090 ceph-mon[29180]: pgmap v7512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:07:58 smithi119 ceph-mon[32752]: pgmap v7513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:07:58 smithi167 ceph-mon[32751]: pgmap v7513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:07:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:07:58 smithi090 ceph-mon[29180]: pgmap v7513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:00 smithi119 ceph-mon[32752]: pgmap v7514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12762.172832 secs 2024-04-26T22:08:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:00 smithi167 ceph-mon[32751]: pgmap v7514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12762.172832 secs 2024-04-26T22:08:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:00 smithi090 ceph-mon[29180]: pgmap v7514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12762.172832 secs 2024-04-26T22:08:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12762.914893 secs 2024-04-26T22:08:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12762.914893 secs 2024-04-26T22:08:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12762.914893 secs 2024-04-26T22:08:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:02 smithi119 ceph-mon[32752]: pgmap v7515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:02 smithi167 ceph-mon[32751]: pgmap v7515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:02 smithi090 ceph-mon[29180]: pgmap v7515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:04 smithi119 ceph-mon[32752]: pgmap v7516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:04 smithi167 ceph-mon[32751]: pgmap v7516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:04 smithi090 ceph-mon[29180]: pgmap v7516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:06.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12767.172918 secs 2024-04-26T22:08:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12767.172918 secs 2024-04-26T22:08:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12767.172918 secs 2024-04-26T22:08:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12767.915013 secs 2024-04-26T22:08:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:06 smithi119 ceph-mon[32752]: pgmap v7517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12767.915013 secs 2024-04-26T22:08:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:06 smithi167 ceph-mon[32751]: pgmap v7517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12767.915013 secs 2024-04-26T22:08:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:06 smithi090 ceph-mon[29180]: pgmap v7517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:08 smithi119 ceph-mon[32752]: pgmap v7518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:08 smithi167 ceph-mon[32751]: pgmap v7518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:08 smithi090 ceph-mon[29180]: pgmap v7518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:10 smithi119 ceph-mon[32752]: pgmap v7519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12772.173002 secs 2024-04-26T22:08:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:10 smithi167 ceph-mon[32751]: pgmap v7519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12772.173002 secs 2024-04-26T22:08:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:10 smithi090 ceph-mon[29180]: pgmap v7519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12772.173002 secs 2024-04-26T22:08:12.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12772.915131 secs 2024-04-26T22:08:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12772.915131 secs 2024-04-26T22:08:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12772.915131 secs 2024-04-26T22:08:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:12 smithi119 ceph-mon[32752]: pgmap v7520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:12 smithi167 ceph-mon[32751]: pgmap v7520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:12 smithi090 ceph-mon[29180]: pgmap v7520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:14 smithi119 ceph-mon[32752]: pgmap v7521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:14 smithi167 ceph-mon[32751]: pgmap v7521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:14 smithi090 ceph-mon[29180]: pgmap v7521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12777.173086 secs 2024-04-26T22:08:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:08:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12777.173086 secs 2024-04-26T22:08:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:08:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12777.173086 secs 2024-04-26T22:08:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:08:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12777.915266 secs 2024-04-26T22:08:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:16 smithi119 ceph-mon[32752]: pgmap v7522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12777.915266 secs 2024-04-26T22:08:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:16 smithi167 ceph-mon[32751]: pgmap v7522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12777.915266 secs 2024-04-26T22:08:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:16 smithi090 ceph-mon[29180]: pgmap v7522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:17 smithi119 ceph-mon[32752]: pgmap v7523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T22:08:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:17 smithi119 ceph-mon[32752]: pgmap v7524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:17 smithi167 ceph-mon[32751]: pgmap v7523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T22:08:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:17 smithi167 ceph-mon[32751]: pgmap v7524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:08:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:08:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:17 smithi090 ceph-mon[29180]: pgmap v7523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T22:08:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:17 smithi090 ceph-mon[29180]: pgmap v7524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:08:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:20 smithi090 ceph-mon[29180]: pgmap v7525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:20 smithi119 ceph-mon[32752]: pgmap v7525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:20 smithi167 ceph-mon[32751]: pgmap v7525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:21.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12782.173170 secs 2024-04-26T22:08:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12782.173170 secs 2024-04-26T22:08:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12782.173170 secs 2024-04-26T22:08:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12782.915398 secs 2024-04-26T22:08:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:22 smithi119 ceph-mon[32752]: pgmap v7526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12782.915398 secs 2024-04-26T22:08:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:22 smithi167 ceph-mon[32751]: pgmap v7526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12782.915398 secs 2024-04-26T22:08:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:22 smithi090 ceph-mon[29180]: pgmap v7526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:08:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:24 smithi119 ceph-mon[32752]: pgmap v7527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:08:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:24 smithi167 ceph-mon[32751]: pgmap v7527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:08:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:24 smithi090 ceph-mon[29180]: pgmap v7527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:08:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12787.173258 secs 2024-04-26T22:08:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:26 smithi119 ceph-mon[32752]: pgmap v7528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 992 B/s rd, 2 op/s 2024-04-26T22:08:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12787.173258 secs 2024-04-26T22:08:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:26 smithi167 ceph-mon[32751]: pgmap v7528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 992 B/s rd, 2 op/s 2024-04-26T22:08:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12787.173258 secs 2024-04-26T22:08:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:26 smithi090 ceph-mon[29180]: pgmap v7528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 992 B/s rd, 2 op/s 2024-04-26T22:08:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12787.915516 secs 2024-04-26T22:08:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12787.915516 secs 2024-04-26T22:08:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12787.915516 secs 2024-04-26T22:08:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:28 smithi119 ceph-mon[32752]: pgmap v7529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T22:08:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:28 smithi167 ceph-mon[32751]: pgmap v7529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T22:08:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:28 smithi090 ceph-mon[29180]: pgmap v7529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T22:08:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:30 smithi119 ceph-mon[32752]: pgmap v7530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:30 smithi167 ceph-mon[32751]: pgmap v7530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:30 smithi090 ceph-mon[29180]: pgmap v7530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12792.173344 secs 2024-04-26T22:08:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12792.173344 secs 2024-04-26T22:08:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12792.173344 secs 2024-04-26T22:08:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12792.915643 secs 2024-04-26T22:08:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:32 smithi119 ceph-mon[32752]: pgmap v7531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:08:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12792.915643 secs 2024-04-26T22:08:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:32 smithi167 ceph-mon[32751]: pgmap v7531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:08:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12792.915643 secs 2024-04-26T22:08:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:32 smithi090 ceph-mon[29180]: pgmap v7531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:08:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:34 smithi119 ceph-mon[32752]: pgmap v7532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:08:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:34 smithi167 ceph-mon[32751]: pgmap v7532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:08:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:34 smithi090 ceph-mon[29180]: pgmap v7532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:08:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12797.173428 secs 2024-04-26T22:08:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:36 smithi119 ceph-mon[32752]: pgmap v7533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12797.173428 secs 2024-04-26T22:08:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:36 smithi167 ceph-mon[32751]: pgmap v7533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12797.173428 secs 2024-04-26T22:08:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:36 smithi090 ceph-mon[29180]: pgmap v7533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:37 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 12797.915750 secs 2024-04-26T22:08:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:37 smithi119 ceph-mon[32752]: slow request 11524.552614 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:08:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:37 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 12797.915750 secs 2024-04-26T22:08:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:37 smithi167 ceph-mon[32751]: slow request 11524.552614 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:08:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:37 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 12797.915750 secs 2024-04-26T22:08:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:37 smithi090 ceph-mon[29180]: slow request 11524.552614 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:08:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:38 smithi119 ceph-mon[32752]: pgmap v7534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:38 smithi167 ceph-mon[32751]: pgmap v7534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:38 smithi090 ceph-mon[29180]: pgmap v7534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:40 smithi119 ceph-mon[32752]: pgmap v7535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:40 smithi167 ceph-mon[32751]: pgmap v7535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:40 smithi090 ceph-mon[29180]: pgmap v7535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12802.173488 secs 2024-04-26T22:08:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12802.173488 secs 2024-04-26T22:08:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12802.173488 secs 2024-04-26T22:08:42.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12802.915883 secs 2024-04-26T22:08:42.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:42 smithi090 ceph-mon[29180]: pgmap v7536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12802.915883 secs 2024-04-26T22:08:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:42 smithi119 ceph-mon[32752]: pgmap v7536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12802.915883 secs 2024-04-26T22:08:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:42 smithi167 ceph-mon[32751]: pgmap v7536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:44 smithi119 ceph-mon[32752]: pgmap v7537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:44 smithi167 ceph-mon[32751]: pgmap v7537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:44 smithi090 ceph-mon[29180]: pgmap v7537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12807.173574 secs 2024-04-26T22:08:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:46 smithi119 ceph-mon[32752]: pgmap v7538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12807.173574 secs 2024-04-26T22:08:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:46 smithi167 ceph-mon[32751]: pgmap v7538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12807.173574 secs 2024-04-26T22:08:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:46 smithi090 ceph-mon[29180]: pgmap v7538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12807.915945 secs 2024-04-26T22:08:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:47 smithi119 ceph-mon[32752]: pgmap v7539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12807.915945 secs 2024-04-26T22:08:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:47 smithi167 ceph-mon[32751]: pgmap v7539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12807.915945 secs 2024-04-26T22:08:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:47 smithi090 ceph-mon[29180]: pgmap v7539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:50 smithi119 ceph-mon[32752]: pgmap v7540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:50.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:50 smithi167 ceph-mon[32751]: pgmap v7540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:50 smithi090 ceph-mon[29180]: pgmap v7540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12812.173671 secs 2024-04-26T22:08:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12812.173671 secs 2024-04-26T22:08:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12812.173671 secs 2024-04-26T22:08:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12812.916064 secs 2024-04-26T22:08:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:52 smithi119 ceph-mon[32752]: pgmap v7541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12812.916064 secs 2024-04-26T22:08:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:52 smithi167 ceph-mon[32751]: pgmap v7541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12812.916064 secs 2024-04-26T22:08:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:52 smithi090 ceph-mon[29180]: pgmap v7541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:54 smithi119 ceph-mon[32752]: pgmap v7542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:54 smithi167 ceph-mon[32751]: pgmap v7542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:54 smithi090 ceph-mon[29180]: pgmap v7542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12817.173757 secs 2024-04-26T22:08:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:56 smithi119 ceph-mon[32752]: pgmap v7543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12817.173757 secs 2024-04-26T22:08:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:56 smithi167 ceph-mon[32751]: pgmap v7543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12817.173757 secs 2024-04-26T22:08:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:56 smithi090 ceph-mon[29180]: pgmap v7543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:08:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12817.916197 secs 2024-04-26T22:08:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12817.916197 secs 2024-04-26T22:08:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12817.916197 secs 2024-04-26T22:08:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:08:58 smithi119 ceph-mon[32752]: pgmap v7544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:08:58 smithi167 ceph-mon[32751]: pgmap v7544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:08:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:08:58 smithi090 ceph-mon[29180]: pgmap v7544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:00 smithi119 ceph-mon[32752]: pgmap v7545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:00 smithi167 ceph-mon[32751]: pgmap v7545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:00 smithi090 ceph-mon[29180]: pgmap v7545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12822.173830 secs 2024-04-26T22:09:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12822.173830 secs 2024-04-26T22:09:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12822.173830 secs 2024-04-26T22:09:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12822.916313 secs 2024-04-26T22:09:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:02 smithi119 ceph-mon[32752]: pgmap v7546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12822.916313 secs 2024-04-26T22:09:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:02 smithi167 ceph-mon[32751]: pgmap v7546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12822.916313 secs 2024-04-26T22:09:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:02 smithi090 ceph-mon[29180]: pgmap v7546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:04 smithi119 ceph-mon[32752]: pgmap v7547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:04 smithi167 ceph-mon[32751]: pgmap v7547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:04 smithi090 ceph-mon[29180]: pgmap v7547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:06 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 12827.173917 secs 2024-04-26T22:09:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:06 smithi119 ceph-mon[32752]: slow request 2881.442334 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:09:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:06 smithi119 ceph-mon[32752]: pgmap v7548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:06 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 12827.173917 secs 2024-04-26T22:09:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:06 smithi167 ceph-mon[32751]: slow request 2881.442334 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:09:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:06 smithi167 ceph-mon[32751]: pgmap v7548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:06 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 12827.173917 secs 2024-04-26T22:09:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:06 smithi090 ceph-mon[29180]: slow request 2881.442334 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:09:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:06 smithi090 ceph-mon[29180]: pgmap v7548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12827.916429 secs 2024-04-26T22:09:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12827.916429 secs 2024-04-26T22:09:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12827.916429 secs 2024-04-26T22:09:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:08 smithi119 ceph-mon[32752]: pgmap v7549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:08 smithi167 ceph-mon[32751]: pgmap v7549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:08 smithi090 ceph-mon[29180]: pgmap v7549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:10 smithi119 ceph-mon[32752]: pgmap v7550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:10 smithi167 ceph-mon[32751]: pgmap v7550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:10 smithi090 ceph-mon[29180]: pgmap v7550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12832.174001 secs 2024-04-26T22:09:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12832.174001 secs 2024-04-26T22:09:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12832.174001 secs 2024-04-26T22:09:12.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12832.916543 secs 2024-04-26T22:09:12.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:12 smithi090 ceph-mon[29180]: pgmap v7551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12832.916543 secs 2024-04-26T22:09:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:12 smithi119 ceph-mon[32752]: pgmap v7551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12832.916543 secs 2024-04-26T22:09:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:12 smithi167 ceph-mon[32751]: pgmap v7551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:13 smithi119 ceph-mon[32752]: pgmap v7552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:13 smithi167 ceph-mon[32751]: pgmap v7552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:13 smithi090 ceph-mon[29180]: pgmap v7552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:16 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 12837.174083 secs 2024-04-26T22:09:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:16 smithi119 ceph-mon[32752]: slow request 2881.405818 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T22:09:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:16 smithi119 ceph-mon[32752]: pgmap v7553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:16 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 12837.174083 secs 2024-04-26T22:09:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:16 smithi167 ceph-mon[32751]: slow request 2881.405818 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T22:09:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:16 smithi167 ceph-mon[32751]: pgmap v7553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:16 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 12837.174083 secs 2024-04-26T22:09:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:16 smithi090 ceph-mon[29180]: slow request 2881.405818 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T22:09:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:16 smithi090 ceph-mon[29180]: pgmap v7553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12837.916678 secs 2024-04-26T22:09:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12837.916678 secs 2024-04-26T22:09:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12837.916678 secs 2024-04-26T22:09:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:18 smithi119 ceph-mon[32752]: pgmap v7554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:09:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:09:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:09:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:09:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:18 smithi167 ceph-mon[32751]: pgmap v7554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:09:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:09:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:09:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:09:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:18 smithi090 ceph-mon[29180]: pgmap v7554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:09:18.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:09:18.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:09:18.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:09:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:19 smithi119 ceph-mon[32752]: pgmap v7555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:09:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:19 smithi119 ceph-mon[32752]: pgmap v7556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:09:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:19 smithi167 ceph-mon[32751]: pgmap v7555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:09:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:19 smithi167 ceph-mon[32751]: pgmap v7556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:09:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:19 smithi090 ceph-mon[29180]: pgmap v7555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:09:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:19 smithi090 ceph-mon[29180]: pgmap v7556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:09:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:21 smithi119 ceph-mon[32752]: pgmap v7557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12842.174143 secs 2024-04-26T22:09:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:21 smithi167 ceph-mon[32751]: pgmap v7557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12842.174143 secs 2024-04-26T22:09:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:21 smithi090 ceph-mon[29180]: pgmap v7557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12842.174143 secs 2024-04-26T22:09:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12842.916789 secs 2024-04-26T22:09:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12842.916789 secs 2024-04-26T22:09:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12842.916789 secs 2024-04-26T22:09:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:23 smithi119 ceph-mon[32752]: pgmap v7558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:23 smithi167 ceph-mon[32751]: pgmap v7558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:23 smithi090 ceph-mon[29180]: pgmap v7558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:25 smithi119 ceph-mon[32752]: pgmap v7559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:25 smithi167 ceph-mon[32751]: pgmap v7559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:25 smithi090 ceph-mon[29180]: pgmap v7559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:26.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12847.174228 secs 2024-04-26T22:09:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12847.174228 secs 2024-04-26T22:09:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12847.174228 secs 2024-04-26T22:09:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12847.916938 secs 2024-04-26T22:09:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:27 smithi119 ceph-mon[32752]: pgmap v7560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12847.916938 secs 2024-04-26T22:09:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:27 smithi167 ceph-mon[32751]: pgmap v7560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12847.916938 secs 2024-04-26T22:09:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:27 smithi090 ceph-mon[29180]: pgmap v7560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:09:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:29 smithi119 ceph-mon[32752]: pgmap v7561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:09:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:29 smithi167 ceph-mon[32751]: pgmap v7561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:09:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:29 smithi090 ceph-mon[29180]: pgmap v7561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:09:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:31 smithi119 ceph-mon[32752]: pgmap v7562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12852.174330 secs 2024-04-26T22:09:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:31 smithi167 ceph-mon[32751]: pgmap v7562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12852.174330 secs 2024-04-26T22:09:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:31 smithi090 ceph-mon[29180]: pgmap v7562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12852.174330 secs 2024-04-26T22:09:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12852.917008 secs 2024-04-26T22:09:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12852.917008 secs 2024-04-26T22:09:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12852.917008 secs 2024-04-26T22:09:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:33 smithi119 ceph-mon[32752]: pgmap v7563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:33 smithi167 ceph-mon[32751]: pgmap v7563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:33 smithi090 ceph-mon[29180]: pgmap v7563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:34.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:09:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:09:34.232+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:09:34.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:09:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:09:34.232+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:09:34.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:09:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:09:34.232+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:09:34.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:09:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:09:34.232+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:09:34.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:09:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:09:34.232+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:09:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:35 smithi119 ceph-mon[32752]: pgmap v7564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:35 smithi167 ceph-mon[32751]: pgmap v7564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:35 smithi090 ceph-mon[29180]: pgmap v7564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:36.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12857.174418 secs 2024-04-26T22:09:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12857.174418 secs 2024-04-26T22:09:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12857.174418 secs 2024-04-26T22:09:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12857.917100 secs 2024-04-26T22:09:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:37 smithi119 ceph-mon[32752]: pgmap v7565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12857.917100 secs 2024-04-26T22:09:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:37 smithi167 ceph-mon[32751]: pgmap v7565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12857.917100 secs 2024-04-26T22:09:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:37 smithi090 ceph-mon[29180]: pgmap v7565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:38 smithi119 ceph-mon[32752]: pgmap v7566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:38 smithi167 ceph-mon[32751]: pgmap v7566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:38 smithi090 ceph-mon[29180]: pgmap v7566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:40 smithi119 ceph-mon[32752]: pgmap v7567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12862.174521 secs 2024-04-26T22:09:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:40 smithi167 ceph-mon[32751]: pgmap v7567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12862.174521 secs 2024-04-26T22:09:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:40 smithi090 ceph-mon[29180]: pgmap v7567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12862.174521 secs 2024-04-26T22:09:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12862.917217 secs 2024-04-26T22:09:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12862.917217 secs 2024-04-26T22:09:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12862.917217 secs 2024-04-26T22:09:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:42 smithi119 ceph-mon[32752]: pgmap v7568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:42 smithi167 ceph-mon[32751]: pgmap v7568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:42 smithi090 ceph-mon[29180]: pgmap v7568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:44 smithi119 ceph-mon[32752]: pgmap v7569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:44 smithi167 ceph-mon[32751]: pgmap v7569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:44 smithi090 ceph-mon[29180]: pgmap v7569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12867.174611 secs 2024-04-26T22:09:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12867.174611 secs 2024-04-26T22:09:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12867.174611 secs 2024-04-26T22:09:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12867.917352 secs 2024-04-26T22:09:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:46 smithi119 ceph-mon[32752]: pgmap v7570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12867.917352 secs 2024-04-26T22:09:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:46 smithi167 ceph-mon[32751]: pgmap v7570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12867.917352 secs 2024-04-26T22:09:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:46 smithi090 ceph-mon[29180]: pgmap v7570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:48 smithi119 ceph-mon[32752]: pgmap v7571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:48 smithi167 ceph-mon[32751]: pgmap v7571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:48 smithi090 ceph-mon[29180]: pgmap v7571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:09:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:50 smithi119 ceph-mon[32752]: pgmap v7572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12872.174671 secs 2024-04-26T22:09:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:50 smithi167 ceph-mon[32751]: pgmap v7572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12872.174671 secs 2024-04-26T22:09:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:50 smithi090 ceph-mon[29180]: pgmap v7572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:09:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12872.174671 secs 2024-04-26T22:09:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12872.917468 secs 2024-04-26T22:09:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12872.917468 secs 2024-04-26T22:09:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12872.917468 secs 2024-04-26T22:09:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:52 smithi119 ceph-mon[32752]: pgmap v7573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:52 smithi167 ceph-mon[32751]: pgmap v7573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:52 smithi090 ceph-mon[29180]: pgmap v7573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:54 smithi119 ceph-mon[32752]: pgmap v7574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:54 smithi167 ceph-mon[32751]: pgmap v7574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:54 smithi090 ceph-mon[29180]: pgmap v7574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:09:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12877.174756 secs 2024-04-26T22:09:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12877.174756 secs 2024-04-26T22:09:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12877.174756 secs 2024-04-26T22:09:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12877.917545 secs 2024-04-26T22:09:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:56 smithi119 ceph-mon[32752]: pgmap v7575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12877.917545 secs 2024-04-26T22:09:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:56 smithi167 ceph-mon[32751]: pgmap v7575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12877.917545 secs 2024-04-26T22:09:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:56 smithi090 ceph-mon[29180]: pgmap v7575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:09:58 smithi119 ceph-mon[32752]: pgmap v7576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:09:58 smithi167 ceph-mon[32751]: pgmap v7576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:09:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:09:58 smithi090 ceph-mon[29180]: pgmap v7576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:00 smithi119 ceph-mon[32752]: pgmap v7577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12882.174799 secs 2024-04-26T22:10:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:10:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:00 smithi167 ceph-mon[32751]: pgmap v7577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12882.174799 secs 2024-04-26T22:10:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:10:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:00 smithi090 ceph-mon[29180]: pgmap v7577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12882.174799 secs 2024-04-26T22:10:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:10:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12882.917653 secs 2024-04-26T22:10:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12882.917653 secs 2024-04-26T22:10:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12882.917653 secs 2024-04-26T22:10:03.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:02 smithi119 ceph-mon[32752]: pgmap v7578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:02 smithi167 ceph-mon[32751]: pgmap v7578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:02 smithi090 ceph-mon[29180]: pgmap v7578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:04 smithi119 ceph-mon[32752]: pgmap v7579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:04 smithi167 ceph-mon[32751]: pgmap v7579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:04 smithi090 ceph-mon[29180]: pgmap v7579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12887.174887 secs 2024-04-26T22:10:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12887.174887 secs 2024-04-26T22:10:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12887.174887 secs 2024-04-26T22:10:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12887.917716 secs 2024-04-26T22:10:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:06 smithi119 ceph-mon[32752]: pgmap v7580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12887.917716 secs 2024-04-26T22:10:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:06 smithi167 ceph-mon[32751]: pgmap v7580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12887.917716 secs 2024-04-26T22:10:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:06 smithi090 ceph-mon[29180]: pgmap v7580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:08 smithi119 ceph-mon[32752]: pgmap v7581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:08 smithi167 ceph-mon[32751]: pgmap v7581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:08 smithi090 ceph-mon[29180]: pgmap v7581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:10 smithi119 ceph-mon[32752]: pgmap v7582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12892.174971 secs 2024-04-26T22:10:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:10 smithi167 ceph-mon[32751]: pgmap v7582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12892.174971 secs 2024-04-26T22:10:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:10 smithi090 ceph-mon[29180]: pgmap v7582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12892.174971 secs 2024-04-26T22:10:12.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12892.917843 secs 2024-04-26T22:10:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12892.917843 secs 2024-04-26T22:10:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12892.917843 secs 2024-04-26T22:10:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:12 smithi119 ceph-mon[32752]: pgmap v7583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:12 smithi167 ceph-mon[32751]: pgmap v7583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:12 smithi090 ceph-mon[29180]: pgmap v7583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:14 smithi119 ceph-mon[32752]: pgmap v7584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:14 smithi167 ceph-mon[32751]: pgmap v7584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:14 smithi090 ceph-mon[29180]: pgmap v7584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12897.175056 secs 2024-04-26T22:10:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12897.175056 secs 2024-04-26T22:10:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12897.175056 secs 2024-04-26T22:10:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12897.917912 secs 2024-04-26T22:10:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:17 smithi119 ceph-mon[32752]: pgmap v7585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12897.917912 secs 2024-04-26T22:10:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:17 smithi167 ceph-mon[32751]: pgmap v7585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12897.917912 secs 2024-04-26T22:10:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:17 smithi090 ceph-mon[29180]: pgmap v7585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:10:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:10:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:10:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:19 smithi119 ceph-mon[32752]: pgmap v7586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:10:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:10:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:10:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:19 smithi167 ceph-mon[32751]: pgmap v7586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:10:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:10:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:10:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:19 smithi090 ceph-mon[29180]: pgmap v7586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:10:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:10:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:10:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:20 smithi119 ceph-mon[32752]: pgmap v7587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:10:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:20 smithi119 ceph-mon[32752]: pgmap v7588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:20 smithi167 ceph-mon[32751]: pgmap v7587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:10:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:20 smithi167 ceph-mon[32751]: pgmap v7588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:20 smithi090 ceph-mon[29180]: pgmap v7587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:10:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:20 smithi090 ceph-mon[29180]: pgmap v7588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12902.175138 secs 2024-04-26T22:10:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12902.175138 secs 2024-04-26T22:10:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12902.175138 secs 2024-04-26T22:10:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:22 smithi119 ceph-mon[32752]: pgmap v7589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12902.918061 secs 2024-04-26T22:10:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:22 smithi167 ceph-mon[32751]: pgmap v7589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12902.918061 secs 2024-04-26T22:10:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:22 smithi090 ceph-mon[29180]: pgmap v7589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12902.918061 secs 2024-04-26T22:10:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:24 smithi119 ceph-mon[32752]: pgmap v7590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:10:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:24 smithi167 ceph-mon[32751]: pgmap v7590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:10:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:24 smithi090 ceph-mon[29180]: pgmap v7590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:10:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:26 smithi119 ceph-mon[32752]: pgmap v7591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12907.175221 secs 2024-04-26T22:10:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:26 smithi167 ceph-mon[32751]: pgmap v7591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12907.175221 secs 2024-04-26T22:10:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:26 smithi090 ceph-mon[29180]: pgmap v7591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12907.175221 secs 2024-04-26T22:10:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12907.918205 secs 2024-04-26T22:10:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12907.918205 secs 2024-04-26T22:10:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12907.918205 secs 2024-04-26T22:10:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:28 smithi119 ceph-mon[32752]: pgmap v7592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:28 smithi167 ceph-mon[32751]: pgmap v7592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:28 smithi090 ceph-mon[29180]: pgmap v7592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:10:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:30 smithi119 ceph-mon[32752]: pgmap v7593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:10:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:30 smithi167 ceph-mon[32751]: pgmap v7593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:10:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:30 smithi090 ceph-mon[29180]: pgmap v7593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:10:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12912.175305 secs 2024-04-26T22:10:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12912.175305 secs 2024-04-26T22:10:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12912.175305 secs 2024-04-26T22:10:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:32 smithi119 ceph-mon[32752]: pgmap v7594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12912.918320 secs 2024-04-26T22:10:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:32 smithi167 ceph-mon[32751]: pgmap v7594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12912.918320 secs 2024-04-26T22:10:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:32 smithi090 ceph-mon[29180]: pgmap v7594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12912.918320 secs 2024-04-26T22:10:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:34 smithi119 ceph-mon[32752]: pgmap v7595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:34 smithi167 ceph-mon[32751]: pgmap v7595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:34 smithi090 ceph-mon[29180]: pgmap v7595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:36 smithi119 ceph-mon[32752]: pgmap v7596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12917.175390 secs 2024-04-26T22:10:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:36 smithi167 ceph-mon[32751]: pgmap v7596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12917.175390 secs 2024-04-26T22:10:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:36 smithi090 ceph-mon[29180]: pgmap v7596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12917.175390 secs 2024-04-26T22:10:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12917.918441 secs 2024-04-26T22:10:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12917.918441 secs 2024-04-26T22:10:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12917.918441 secs 2024-04-26T22:10:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:38 smithi167 ceph-mon[32751]: pgmap v7597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:38 smithi090 ceph-mon[29180]: pgmap v7597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:38 smithi119 ceph-mon[32752]: pgmap v7597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:40 smithi167 ceph-mon[32751]: pgmap v7598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:40 smithi090 ceph-mon[29180]: pgmap v7598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:40 smithi119 ceph-mon[32752]: pgmap v7598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12922.175450 secs 2024-04-26T22:10:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12922.175450 secs 2024-04-26T22:10:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12922.175450 secs 2024-04-26T22:10:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:42 smithi090 ceph-mon[29180]: pgmap v7599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12922.918521 secs 2024-04-26T22:10:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:42 smithi119 ceph-mon[32752]: pgmap v7599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12922.918521 secs 2024-04-26T22:10:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:42 smithi167 ceph-mon[32751]: pgmap v7599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12922.918521 secs 2024-04-26T22:10:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:44 smithi090 ceph-mon[29180]: pgmap v7600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:44 smithi119 ceph-mon[32752]: pgmap v7600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:44 smithi167 ceph-mon[32751]: pgmap v7600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:46 smithi090 ceph-mon[29180]: pgmap v7601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12927.175536 secs 2024-04-26T22:10:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:46 smithi119 ceph-mon[32752]: pgmap v7601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12927.175536 secs 2024-04-26T22:10:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:46 smithi167 ceph-mon[32751]: pgmap v7601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12927.175536 secs 2024-04-26T22:10:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12927.918587 secs 2024-04-26T22:10:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12927.918587 secs 2024-04-26T22:10:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12927.918587 secs 2024-04-26T22:10:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:48 smithi090 ceph-mon[29180]: pgmap v7602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:48 smithi119 ceph-mon[32752]: pgmap v7602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:48 smithi167 ceph-mon[32751]: pgmap v7602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:50 smithi090 ceph-mon[29180]: pgmap v7603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:50 smithi119 ceph-mon[32752]: pgmap v7603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:50 smithi167 ceph-mon[32751]: pgmap v7603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12932.175619 secs 2024-04-26T22:10:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12932.175619 secs 2024-04-26T22:10:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12932.175619 secs 2024-04-26T22:10:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:52 smithi090 ceph-mon[29180]: pgmap v7604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12932.918652 secs 2024-04-26T22:10:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:52 smithi119 ceph-mon[32752]: pgmap v7604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12932.918652 secs 2024-04-26T22:10:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:52 smithi167 ceph-mon[32751]: pgmap v7604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12932.918652 secs 2024-04-26T22:10:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:54 smithi090 ceph-mon[29180]: pgmap v7605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:54 smithi119 ceph-mon[32752]: pgmap v7605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:54 smithi167 ceph-mon[32751]: pgmap v7605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:56 smithi090 ceph-mon[29180]: pgmap v7606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12937.175704 secs 2024-04-26T22:10:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:56 smithi119 ceph-mon[32752]: pgmap v7606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12937.175704 secs 2024-04-26T22:10:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:56 smithi167 ceph-mon[32751]: pgmap v7606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:10:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12937.175704 secs 2024-04-26T22:10:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12937.918719 secs 2024-04-26T22:10:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12937.918719 secs 2024-04-26T22:10:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12937.918719 secs 2024-04-26T22:10:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:10:58 smithi090 ceph-mon[29180]: pgmap v7607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:10:58 smithi119 ceph-mon[32752]: pgmap v7607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:10:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:10:58 smithi167 ceph-mon[32751]: pgmap v7607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:00 smithi090 ceph-mon[29180]: pgmap v7608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:00 smithi119 ceph-mon[32752]: pgmap v7608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:00 smithi167 ceph-mon[32751]: pgmap v7608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12942.175795 secs 2024-04-26T22:11:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12942.175795 secs 2024-04-26T22:11:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12942.175795 secs 2024-04-26T22:11:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:02 smithi090 ceph-mon[29180]: pgmap v7609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12942.918799 secs 2024-04-26T22:11:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:02 smithi119 ceph-mon[32752]: pgmap v7609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12942.918799 secs 2024-04-26T22:11:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:02 smithi167 ceph-mon[32751]: pgmap v7609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12942.918799 secs 2024-04-26T22:11:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:04 smithi090 ceph-mon[29180]: pgmap v7610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:04 smithi119 ceph-mon[32752]: pgmap v7610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:04 smithi167 ceph-mon[32751]: pgmap v7610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:06 smithi119 ceph-mon[32752]: pgmap v7611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12947.175879 secs 2024-04-26T22:11:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:06 smithi167 ceph-mon[32751]: pgmap v7611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12947.175879 secs 2024-04-26T22:11:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:06 smithi090 ceph-mon[29180]: pgmap v7611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12947.175879 secs 2024-04-26T22:11:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12947.918877 secs 2024-04-26T22:11:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12947.918877 secs 2024-04-26T22:11:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12947.918877 secs 2024-04-26T22:11:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:08 smithi119 ceph-mon[32752]: pgmap v7612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:08 smithi167 ceph-mon[32751]: pgmap v7612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:08 smithi090 ceph-mon[29180]: pgmap v7612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:10 smithi119 ceph-mon[32752]: pgmap v7613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:10 smithi167 ceph-mon[32751]: pgmap v7613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:10 smithi090 ceph-mon[29180]: pgmap v7613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12952.175963 secs 2024-04-26T22:11:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12952.175963 secs 2024-04-26T22:11:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12952.175963 secs 2024-04-26T22:11:12.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:12 smithi090 ceph-mon[29180]: pgmap v7614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:12.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12952.918990 secs 2024-04-26T22:11:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:12 smithi119 ceph-mon[32752]: pgmap v7614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12952.918990 secs 2024-04-26T22:11:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:12 smithi167 ceph-mon[32751]: pgmap v7614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12952.918990 secs 2024-04-26T22:11:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:14 smithi119 ceph-mon[32752]: pgmap v7615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:14 smithi167 ceph-mon[32751]: pgmap v7615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:14 smithi090 ceph-mon[29180]: pgmap v7615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:16 smithi119 ceph-mon[32752]: pgmap v7616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12957.176048 secs 2024-04-26T22:11:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:16 smithi167 ceph-mon[32751]: pgmap v7616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12957.176048 secs 2024-04-26T22:11:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:16 smithi090 ceph-mon[29180]: pgmap v7616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12957.176048 secs 2024-04-26T22:11:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12957.919117 secs 2024-04-26T22:11:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12957.919117 secs 2024-04-26T22:11:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12957.919117 secs 2024-04-26T22:11:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:18 smithi119 ceph-mon[32752]: pgmap v7617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:18 smithi167 ceph-mon[32751]: pgmap v7617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:18 smithi090 ceph-mon[29180]: pgmap v7617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:11:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:11:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:11:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:20 smithi119 ceph-mon[32752]: pgmap v7618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:11:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:11:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:11:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:20 smithi167 ceph-mon[32751]: pgmap v7618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:11:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:11:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:11:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:20 smithi090 ceph-mon[29180]: pgmap v7618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:11:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:11:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:11:21.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:21 smithi119 ceph-mon[32752]: pgmap v7619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:11:21.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:21 smithi119 ceph-mon[32752]: pgmap v7620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:21.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12962.176181 secs 2024-04-26T22:11:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:21 smithi167 ceph-mon[32751]: pgmap v7619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:11:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:21 smithi167 ceph-mon[32751]: pgmap v7620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12962.176181 secs 2024-04-26T22:11:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:21 smithi090 ceph-mon[29180]: pgmap v7619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:11:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:21 smithi090 ceph-mon[29180]: pgmap v7620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12962.176181 secs 2024-04-26T22:11:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12962.919234 secs 2024-04-26T22:11:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12962.919234 secs 2024-04-26T22:11:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12962.919234 secs 2024-04-26T22:11:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:23 smithi119 ceph-mon[32752]: pgmap v7621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:23 smithi167 ceph-mon[32751]: pgmap v7621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:23 smithi090 ceph-mon[29180]: pgmap v7621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:25 smithi119 ceph-mon[32752]: pgmap v7622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:25 smithi167 ceph-mon[32751]: pgmap v7622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:25 smithi090 ceph-mon[29180]: pgmap v7622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12967.176245 secs 2024-04-26T22:11:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12967.176245 secs 2024-04-26T22:11:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12967.176245 secs 2024-04-26T22:11:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:27 smithi119 ceph-mon[32752]: pgmap v7623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12967.919348 secs 2024-04-26T22:11:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:27 smithi167 ceph-mon[32751]: pgmap v7623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12967.919348 secs 2024-04-26T22:11:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:27 smithi090 ceph-mon[29180]: pgmap v7623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:11:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12967.919348 secs 2024-04-26T22:11:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:29 smithi119 ceph-mon[32752]: pgmap v7624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:11:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:29 smithi167 ceph-mon[32751]: pgmap v7624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:11:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:29 smithi090 ceph-mon[29180]: pgmap v7624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:11:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:31 smithi119 ceph-mon[32752]: pgmap v7625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:11:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12972.176327 secs 2024-04-26T22:11:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:31 smithi167 ceph-mon[32751]: pgmap v7625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:11:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12972.176327 secs 2024-04-26T22:11:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:31 smithi090 ceph-mon[29180]: pgmap v7625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:11:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12972.176327 secs 2024-04-26T22:11:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12972.919459 secs 2024-04-26T22:11:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12972.919459 secs 2024-04-26T22:11:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12972.919459 secs 2024-04-26T22:11:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:33 smithi119 ceph-mon[32752]: pgmap v7626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:33 smithi167 ceph-mon[32751]: pgmap v7626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:33 smithi090 ceph-mon[29180]: pgmap v7626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:35 smithi119 ceph-mon[32752]: pgmap v7627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:35 smithi167 ceph-mon[32751]: pgmap v7627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:35 smithi090 ceph-mon[29180]: pgmap v7627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:36.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12977.176411 secs 2024-04-26T22:11:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12977.176411 secs 2024-04-26T22:11:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12977.176411 secs 2024-04-26T22:11:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:37 smithi119 ceph-mon[32752]: pgmap v7628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12977.919598 secs 2024-04-26T22:11:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:37 smithi167 ceph-mon[32751]: pgmap v7628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12977.919598 secs 2024-04-26T22:11:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:37 smithi090 ceph-mon[29180]: pgmap v7628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12977.919598 secs 2024-04-26T22:11:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:38 smithi119 ceph-mon[32752]: pgmap v7629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:38 smithi167 ceph-mon[32751]: pgmap v7629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:38 smithi090 ceph-mon[29180]: pgmap v7629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:40 smithi119 ceph-mon[32752]: pgmap v7630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12982.176498 secs 2024-04-26T22:11:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:40 smithi167 ceph-mon[32751]: pgmap v7630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12982.176498 secs 2024-04-26T22:11:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:40 smithi090 ceph-mon[29180]: pgmap v7630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12982.176498 secs 2024-04-26T22:11:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12982.919739 secs 2024-04-26T22:11:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12982.919739 secs 2024-04-26T22:11:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12982.919739 secs 2024-04-26T22:11:42.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:42 smithi090 ceph-mon[29180]: pgmap v7631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:42 smithi119 ceph-mon[32752]: pgmap v7631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:42 smithi167 ceph-mon[32751]: pgmap v7631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:44 smithi119 ceph-mon[32752]: pgmap v7632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:44 smithi167 ceph-mon[32751]: pgmap v7632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:44 smithi090 ceph-mon[29180]: pgmap v7632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:45.765 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12987.176583 secs 2024-04-26T22:11:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12987.176583 secs 2024-04-26T22:11:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12987.176583 secs 2024-04-26T22:11:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:46 smithi119 ceph-mon[32752]: pgmap v7633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12987.919854 secs 2024-04-26T22:11:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:46 smithi167 ceph-mon[32751]: pgmap v7633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12987.919854 secs 2024-04-26T22:11:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:46 smithi090 ceph-mon[29180]: pgmap v7633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12987.919854 secs 2024-04-26T22:11:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:48 smithi119 ceph-mon[32752]: pgmap v7634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:48 smithi167 ceph-mon[32751]: pgmap v7634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:48 smithi090 ceph-mon[29180]: pgmap v7634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:50 smithi119 ceph-mon[32752]: pgmap v7635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12992.176642 secs 2024-04-26T22:11:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:50 smithi167 ceph-mon[32751]: pgmap v7635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12992.176642 secs 2024-04-26T22:11:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:50 smithi090 ceph-mon[29180]: pgmap v7635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12992.176642 secs 2024-04-26T22:11:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12992.919971 secs 2024-04-26T22:11:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12992.919971 secs 2024-04-26T22:11:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12992.919971 secs 2024-04-26T22:11:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:52 smithi119 ceph-mon[32752]: pgmap v7636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:52 smithi167 ceph-mon[32751]: pgmap v7636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:52 smithi090 ceph-mon[29180]: pgmap v7636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:54 smithi119 ceph-mon[32752]: pgmap v7637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:54 smithi167 ceph-mon[32751]: pgmap v7637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:54 smithi090 ceph-mon[29180]: pgmap v7637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:11:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 12997.176729 secs 2024-04-26T22:11:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 12997.176729 secs 2024-04-26T22:11:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 12997.176729 secs 2024-04-26T22:11:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:56 smithi119 ceph-mon[32752]: pgmap v7638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 12997.920066 secs 2024-04-26T22:11:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:56 smithi167 ceph-mon[32751]: pgmap v7638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 12997.920066 secs 2024-04-26T22:11:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:56 smithi090 ceph-mon[29180]: pgmap v7638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 12997.920066 secs 2024-04-26T22:11:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:11:58 smithi119 ceph-mon[32752]: pgmap v7639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:11:58 smithi167 ceph-mon[32751]: pgmap v7639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:11:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:11:58 smithi090 ceph-mon[29180]: pgmap v7639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:00 smithi119 ceph-mon[32752]: pgmap v7640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 13002.176833 secs 2024-04-26T22:12:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:00 smithi167 ceph-mon[32751]: pgmap v7640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 13002.176833 secs 2024-04-26T22:12:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:00 smithi090 ceph-mon[29180]: pgmap v7640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 13002.176833 secs 2024-04-26T22:12:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13002.920166 secs 2024-04-26T22:12:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13002.920166 secs 2024-04-26T22:12:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13002.920166 secs 2024-04-26T22:12:02.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:02 smithi119 ceph-mon[32752]: pgmap v7641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:02 smithi167 ceph-mon[32751]: pgmap v7641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:02 smithi090 ceph-mon[29180]: pgmap v7641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:04 smithi119 ceph-mon[32752]: pgmap v7642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:04 smithi167 ceph-mon[32751]: pgmap v7642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:04 smithi090 ceph-mon[29180]: pgmap v7642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 13007.176898 secs 2024-04-26T22:12:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 13007.176898 secs 2024-04-26T22:12:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 13007.176898 secs 2024-04-26T22:12:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:06 smithi119 ceph-mon[32752]: pgmap v7643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13007.920279 secs 2024-04-26T22:12:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:06 smithi167 ceph-mon[32751]: pgmap v7643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13007.920279 secs 2024-04-26T22:12:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:06 smithi090 ceph-mon[29180]: pgmap v7643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13007.920279 secs 2024-04-26T22:12:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:08 smithi119 ceph-mon[32752]: pgmap v7644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:08 smithi167 ceph-mon[32751]: pgmap v7644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:08 smithi090 ceph-mon[29180]: pgmap v7644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:10 smithi167 ceph-mon[32751]: pgmap v7645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 13012.176986 secs 2024-04-26T22:12:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:10 smithi090 ceph-mon[29180]: pgmap v7645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 13012.176986 secs 2024-04-26T22:12:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:10 smithi119 ceph-mon[32752]: pgmap v7645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 13012.176986 secs 2024-04-26T22:12:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13012.920385 secs 2024-04-26T22:12:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13012.920385 secs 2024-04-26T22:12:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13012.920385 secs 2024-04-26T22:12:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:12 smithi167 ceph-mon[32751]: pgmap v7646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:12 smithi090 ceph-mon[29180]: pgmap v7646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:12 smithi119 ceph-mon[32752]: pgmap v7646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:14 smithi167 ceph-mon[32751]: pgmap v7647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:14 smithi090 ceph-mon[29180]: pgmap v7647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:14 smithi119 ceph-mon[32752]: pgmap v7647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 13017.177072 secs 2024-04-26T22:12:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 13017.177072 secs 2024-04-26T22:12:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 13017.177072 secs 2024-04-26T22:12:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:16 smithi090 ceph-mon[29180]: pgmap v7648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13017.920536 secs 2024-04-26T22:12:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:16 smithi119 ceph-mon[32752]: pgmap v7648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13017.920536 secs 2024-04-26T22:12:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:16 smithi167 ceph-mon[32751]: pgmap v7648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13017.920536 secs 2024-04-26T22:12:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:18 smithi090 ceph-mon[29180]: pgmap v7649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:18 smithi119 ceph-mon[32752]: pgmap v7649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:18 smithi167 ceph-mon[32751]: pgmap v7649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:12:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:12:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:12:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:20 smithi090 ceph-mon[29180]: pgmap v7650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 13022.177163 secs 2024-04-26T22:12:21.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:20 smithi119 ceph-mon[32752]: pgmap v7650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:21.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 13022.177163 secs 2024-04-26T22:12:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:20 smithi167 ceph-mon[32751]: pgmap v7650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 13022.177163 secs 2024-04-26T22:12:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13022.920593 secs 2024-04-26T22:12:22.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13022.920593 secs 2024-04-26T22:12:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13022.920593 secs 2024-04-26T22:12:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:22 smithi090 ceph-mon[29180]: pgmap v7651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:22 smithi119 ceph-mon[32752]: pgmap v7651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:22 smithi167 ceph-mon[32751]: pgmap v7651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: pgmap v7652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:12:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:12:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: pgmap v7653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-26T22:12:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: pgmap v7654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: pgmap v7652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:12:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: pgmap v7653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-26T22:12:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: pgmap v7654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: pgmap v7652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:12:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:12:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: pgmap v7653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-26T22:12:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: pgmap v7654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:25.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:12:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13027.177249 secs 2024-04-26T22:12:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13027.177249 secs 2024-04-26T22:12:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13027.177249 secs 2024-04-26T22:12:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13027.920737 secs 2024-04-26T22:12:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:27 smithi090 ceph-mon[29180]: pgmap v7655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:27 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:12:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:27 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:12:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13027.920737 secs 2024-04-26T22:12:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:27 smithi119 ceph-mon[32752]: pgmap v7655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:27 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:12:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:27 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:12:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13027.920737 secs 2024-04-26T22:12:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:27 smithi167 ceph-mon[32751]: pgmap v7655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:27 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:12:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:27 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:12:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:29 smithi090 ceph-mon[29180]: pgmap v7656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T22:12:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:29 smithi119 ceph-mon[32752]: pgmap v7656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T22:12:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:29 smithi167 ceph-mon[32751]: pgmap v7656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T22:12:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13032.177335 secs 2024-04-26T22:12:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:31 smithi090 ceph-mon[29180]: pgmap v7657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13032.177335 secs 2024-04-26T22:12:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:31 smithi119 ceph-mon[32752]: pgmap v7657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13032.177335 secs 2024-04-26T22:12:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:31 smithi167 ceph-mon[32751]: pgmap v7657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13032.920820 secs 2024-04-26T22:12:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13032.920820 secs 2024-04-26T22:12:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13032.920820 secs 2024-04-26T22:12:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:33 smithi090 ceph-mon[29180]: pgmap v7658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T22:12:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:33 smithi119 ceph-mon[32752]: pgmap v7658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T22:12:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:33 smithi167 ceph-mon[32751]: pgmap v7658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 592 B/s rd, 1 op/s 2024-04-26T22:12:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:35 smithi090 ceph-mon[29180]: pgmap v7659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:12:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:35 smithi119 ceph-mon[32752]: pgmap v7659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:12:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:35 smithi167 ceph-mon[32751]: pgmap v7659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13037.177424 secs 2024-04-26T22:12:36.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13037.177424 secs 2024-04-26T22:12:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13037.177424 secs 2024-04-26T22:12:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13037.920938 secs 2024-04-26T22:12:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:37 smithi090 ceph-mon[29180]: pgmap v7660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13037.920938 secs 2024-04-26T22:12:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:37 smithi119 ceph-mon[32752]: pgmap v7660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13037.920938 secs 2024-04-26T22:12:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:37 smithi167 ceph-mon[32751]: pgmap v7660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:39 smithi090 ceph-mon[29180]: pgmap v7661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:39 smithi119 ceph-mon[32752]: pgmap v7661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:39 smithi167 ceph-mon[32751]: pgmap v7661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13042.177511 secs 2024-04-26T22:12:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:41 smithi090 ceph-mon[29180]: pgmap v7662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13042.177511 secs 2024-04-26T22:12:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:41 smithi119 ceph-mon[32752]: pgmap v7662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13042.177511 secs 2024-04-26T22:12:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:41 smithi167 ceph-mon[32751]: pgmap v7662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13042.921058 secs 2024-04-26T22:12:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13042.921058 secs 2024-04-26T22:12:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13042.921058 secs 2024-04-26T22:12:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:43 smithi090 ceph-mon[29180]: pgmap v7663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:43 smithi119 ceph-mon[32752]: pgmap v7663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:43 smithi167 ceph-mon[32751]: pgmap v7663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:45 smithi090 ceph-mon[29180]: pgmap v7664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:45 smithi119 ceph-mon[32752]: pgmap v7664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:45 smithi167 ceph-mon[32751]: pgmap v7664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13047.177571 secs 2024-04-26T22:12:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13047.177571 secs 2024-04-26T22:12:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13047.177571 secs 2024-04-26T22:12:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13047.921175 secs 2024-04-26T22:12:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:47 smithi090 ceph-mon[29180]: pgmap v7665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13047.921175 secs 2024-04-26T22:12:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:47 smithi119 ceph-mon[32752]: pgmap v7665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13047.921175 secs 2024-04-26T22:12:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:47 smithi167 ceph-mon[32751]: pgmap v7665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:49 smithi119 ceph-mon[32752]: pgmap v7666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:49 smithi167 ceph-mon[32751]: pgmap v7666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:49 smithi090 ceph-mon[29180]: pgmap v7666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13052.177655 secs 2024-04-26T22:12:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:51 smithi119 ceph-mon[32752]: pgmap v7667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13052.177655 secs 2024-04-26T22:12:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:51 smithi167 ceph-mon[32751]: pgmap v7667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13052.177655 secs 2024-04-26T22:12:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:51 smithi090 ceph-mon[29180]: pgmap v7667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13052.921298 secs 2024-04-26T22:12:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13052.921298 secs 2024-04-26T22:12:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13052.921298 secs 2024-04-26T22:12:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:53 smithi119 ceph-mon[32752]: pgmap v7668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:53 smithi167 ceph-mon[32751]: pgmap v7668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:53 smithi090 ceph-mon[29180]: pgmap v7668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:55 smithi119 ceph-mon[32752]: pgmap v7669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:55 smithi167 ceph-mon[32751]: pgmap v7669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:55 smithi090 ceph-mon[29180]: pgmap v7669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:12:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13057.177716 secs 2024-04-26T22:12:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13057.177716 secs 2024-04-26T22:12:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13057.177716 secs 2024-04-26T22:12:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13057.921428 secs 2024-04-26T22:12:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:57 smithi119 ceph-mon[32752]: pgmap v7670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13057.921428 secs 2024-04-26T22:12:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:57 smithi167 ceph-mon[32751]: pgmap v7670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13057.921428 secs 2024-04-26T22:12:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:57 smithi090 ceph-mon[29180]: pgmap v7670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:12:59 smithi119 ceph-mon[32752]: pgmap v7671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:12:59 smithi167 ceph-mon[32751]: pgmap v7671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:12:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:12:59 smithi090 ceph-mon[29180]: pgmap v7671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13062.177822 secs 2024-04-26T22:13:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:01 smithi119 ceph-mon[32752]: pgmap v7672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13062.177822 secs 2024-04-26T22:13:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:01 smithi167 ceph-mon[32751]: pgmap v7672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13062.177822 secs 2024-04-26T22:13:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:01 smithi090 ceph-mon[29180]: pgmap v7672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13062.921553 secs 2024-04-26T22:13:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13062.921553 secs 2024-04-26T22:13:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13062.921553 secs 2024-04-26T22:13:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:03 smithi119 ceph-mon[32752]: pgmap v7673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:03 smithi167 ceph-mon[32751]: pgmap v7673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:03 smithi090 ceph-mon[29180]: pgmap v7673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:05 smithi119 ceph-mon[32752]: pgmap v7674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:05 smithi167 ceph-mon[32751]: pgmap v7674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:05 smithi090 ceph-mon[29180]: pgmap v7674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:06.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13067.177909 secs 2024-04-26T22:13:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13067.177909 secs 2024-04-26T22:13:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13067.177909 secs 2024-04-26T22:13:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13067.921637 secs 2024-04-26T22:13:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:07 smithi119 ceph-mon[32752]: pgmap v7675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13067.921637 secs 2024-04-26T22:13:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:07 smithi167 ceph-mon[32751]: pgmap v7675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13067.921637 secs 2024-04-26T22:13:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:07 smithi090 ceph-mon[29180]: pgmap v7675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:09 smithi119 ceph-mon[32752]: pgmap v7676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:09 smithi167 ceph-mon[32751]: pgmap v7676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:09 smithi090 ceph-mon[29180]: pgmap v7676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:11 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13072.177993 secs 2024-04-26T22:13:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:11 smithi119 ceph-mon[32752]: pgmap v7677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:11 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13072.177993 secs 2024-04-26T22:13:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:11 smithi167 ceph-mon[32751]: pgmap v7677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:11 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13072.177993 secs 2024-04-26T22:13:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:11 smithi090 ceph-mon[29180]: pgmap v7677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:12.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13072.921760 secs 2024-04-26T22:13:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13072.921760 secs 2024-04-26T22:13:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13072.921760 secs 2024-04-26T22:13:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:13 smithi119 ceph-mon[32752]: pgmap v7678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:13 smithi167 ceph-mon[32751]: pgmap v7678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:13 smithi090 ceph-mon[29180]: pgmap v7678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:15 smithi119 ceph-mon[32752]: pgmap v7679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:15 smithi167 ceph-mon[32751]: pgmap v7679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:15 smithi090 ceph-mon[29180]: pgmap v7679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:16 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13077.178100 secs 2024-04-26T22:13:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:16 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13077.178100 secs 2024-04-26T22:13:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:16 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13077.178100 secs 2024-04-26T22:13:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13077.921882 secs 2024-04-26T22:13:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:17 smithi119 ceph-mon[32752]: pgmap v7680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13077.921882 secs 2024-04-26T22:13:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:17 smithi167 ceph-mon[32751]: pgmap v7680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13077.921882 secs 2024-04-26T22:13:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:17 smithi090 ceph-mon[29180]: pgmap v7680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:19.461 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:19 smithi090 ceph-mon[29180]: pgmap v7681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:19 smithi119 ceph-mon[32752]: pgmap v7681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:19 smithi167 ceph-mon[32751]: pgmap v7681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:21 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13082.178184 secs 2024-04-26T22:13:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:21 smithi119 ceph-mon[32752]: pgmap v7682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:21 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13082.178184 secs 2024-04-26T22:13:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:21 smithi167 ceph-mon[32751]: pgmap v7682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:21 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13082.178184 secs 2024-04-26T22:13:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:21 smithi090 ceph-mon[29180]: pgmap v7682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13082.921974 secs 2024-04-26T22:13:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13082.921974 secs 2024-04-26T22:13:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13082.921974 secs 2024-04-26T22:13:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:23 smithi119 ceph-mon[32752]: pgmap v7683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:23 smithi167 ceph-mon[32751]: pgmap v7683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:23 smithi090 ceph-mon[29180]: pgmap v7683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:24 smithi119 ceph-mon[32752]: pgmap v7684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:13:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:24 smithi167 ceph-mon[32751]: pgmap v7684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:13:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:24 smithi090 ceph-mon[29180]: pgmap v7684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:13:25.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13087.178245 secs 2024-04-26T22:13:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13087.178245 secs 2024-04-26T22:13:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13087.178245 secs 2024-04-26T22:13:26.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13087.922066 secs 2024-04-26T22:13:26.738 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:26 smithi167 ceph-mon[32751]: pgmap v7685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13087.922066 secs 2024-04-26T22:13:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:26 smithi119 ceph-mon[32752]: pgmap v7685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13087.922066 secs 2024-04-26T22:13:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:26 smithi090 ceph-mon[29180]: pgmap v7685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:29 smithi167 ceph-mon[32751]: pgmap v7686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:13:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:13:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:29 smithi119 ceph-mon[32752]: pgmap v7686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:13:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:13:29.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:29 smithi090 ceph-mon[29180]: pgmap v7686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:13:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:13:29.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:13:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:30 smithi119 ceph-mon[32752]: pgmap v7687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T22:13:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:30 smithi119 ceph-mon[32752]: pgmap v7688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:30 smithi167 ceph-mon[32751]: pgmap v7687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T22:13:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:30 smithi167 ceph-mon[32751]: pgmap v7688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:30 smithi090 ceph-mon[29180]: pgmap v7687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T22:13:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:30 smithi090 ceph-mon[29180]: pgmap v7688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:31 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13092.178335 secs 2024-04-26T22:13:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:31 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13092.178335 secs 2024-04-26T22:13:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:31 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13092.178335 secs 2024-04-26T22:13:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:32 smithi119 ceph-mon[32752]: pgmap v7689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13092.922125 secs 2024-04-26T22:13:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:32 smithi167 ceph-mon[32751]: pgmap v7689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13092.922125 secs 2024-04-26T22:13:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:32 smithi090 ceph-mon[29180]: pgmap v7689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13092.922125 secs 2024-04-26T22:13:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:34 smithi119 ceph-mon[32752]: pgmap v7690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-26T22:13:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:34 smithi167 ceph-mon[32751]: pgmap v7690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-26T22:13:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:34 smithi090 ceph-mon[29180]: pgmap v7690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-26T22:13:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:36 smithi119 ceph-mon[32752]: pgmap v7691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:36 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13097.178421 secs 2024-04-26T22:13:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:36 smithi167 ceph-mon[32751]: pgmap v7691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:36 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13097.178421 secs 2024-04-26T22:13:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:36 smithi090 ceph-mon[29180]: pgmap v7691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:36 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13097.178421 secs 2024-04-26T22:13:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:37 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 13097.922249 secs 2024-04-26T22:13:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:37 smithi119 ceph-mon[32752]: slow request 11524.447597 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:13:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:37 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 13097.922249 secs 2024-04-26T22:13:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:37 smithi167 ceph-mon[32751]: slow request 11524.447597 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:13:37.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:37 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 13097.922249 secs 2024-04-26T22:13:37.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:37 smithi090 ceph-mon[29180]: slow request 11524.447597 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:13:38.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:38 smithi167 ceph-mon[32751]: pgmap v7692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:38 smithi119 ceph-mon[32752]: pgmap v7692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:38 smithi090 ceph-mon[29180]: pgmap v7692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:40 smithi119 ceph-mon[32752]: pgmap v7693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:13:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:40 smithi167 ceph-mon[32751]: pgmap v7693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:13:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:40 smithi090 ceph-mon[29180]: pgmap v7693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:13:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13102.178505 secs 2024-04-26T22:13:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13102.178505 secs 2024-04-26T22:13:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13102.178505 secs 2024-04-26T22:13:42.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:42 smithi090 ceph-mon[29180]: pgmap v7694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:42.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13102.922414 secs 2024-04-26T22:13:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:42 smithi119 ceph-mon[32752]: pgmap v7694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13102.922414 secs 2024-04-26T22:13:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:42 smithi167 ceph-mon[32751]: pgmap v7694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13102.922414 secs 2024-04-26T22:13:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:44 smithi119 ceph-mon[32752]: pgmap v7695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:44 smithi167 ceph-mon[32751]: pgmap v7695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:44 smithi090 ceph-mon[29180]: pgmap v7695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:46 smithi119 ceph-mon[32752]: pgmap v7696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13107.178592 secs 2024-04-26T22:13:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:46 smithi167 ceph-mon[32751]: pgmap v7696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13107.178592 secs 2024-04-26T22:13:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:46 smithi090 ceph-mon[29180]: pgmap v7696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13107.178592 secs 2024-04-26T22:13:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13107.922507 secs 2024-04-26T22:13:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13107.922507 secs 2024-04-26T22:13:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13107.922507 secs 2024-04-26T22:13:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:48 smithi119 ceph-mon[32752]: pgmap v7697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:48 smithi167 ceph-mon[32751]: pgmap v7697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:48 smithi090 ceph-mon[29180]: pgmap v7697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:50 smithi119 ceph-mon[32752]: pgmap v7698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:50 smithi167 ceph-mon[32751]: pgmap v7698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:50 smithi090 ceph-mon[29180]: pgmap v7698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13112.178654 secs 2024-04-26T22:13:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13112.178654 secs 2024-04-26T22:13:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13112.178654 secs 2024-04-26T22:13:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:52 smithi119 ceph-mon[32752]: pgmap v7699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13112.922638 secs 2024-04-26T22:13:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:52 smithi167 ceph-mon[32751]: pgmap v7699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13112.922638 secs 2024-04-26T22:13:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:52 smithi090 ceph-mon[29180]: pgmap v7699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13112.922638 secs 2024-04-26T22:13:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:54 smithi119 ceph-mon[32752]: pgmap v7700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:54 smithi167 ceph-mon[32751]: pgmap v7700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:54 smithi090 ceph-mon[29180]: pgmap v7700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:56 smithi119 ceph-mon[32752]: pgmap v7701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13117.178742 secs 2024-04-26T22:13:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:56 smithi167 ceph-mon[32751]: pgmap v7701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13117.178742 secs 2024-04-26T22:13:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:56 smithi090 ceph-mon[29180]: pgmap v7701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:13:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13117.178742 secs 2024-04-26T22:13:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13117.922796 secs 2024-04-26T22:13:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13117.922796 secs 2024-04-26T22:13:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13117.922796 secs 2024-04-26T22:13:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:13:58 smithi119 ceph-mon[32752]: pgmap v7702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:13:58 smithi167 ceph-mon[32751]: pgmap v7702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:13:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:13:58 smithi090 ceph-mon[29180]: pgmap v7702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:00 smithi119 ceph-mon[32752]: pgmap v7703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:00 smithi167 ceph-mon[32751]: pgmap v7703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:00 smithi090 ceph-mon[29180]: pgmap v7703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:01 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13122.178828 secs 2024-04-26T22:14:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:01 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13122.178828 secs 2024-04-26T22:14:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:01 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13122.178828 secs 2024-04-26T22:14:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:02 smithi119 ceph-mon[32752]: pgmap v7704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13122.922917 secs 2024-04-26T22:14:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:02 smithi167 ceph-mon[32751]: pgmap v7704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13122.922917 secs 2024-04-26T22:14:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:02 smithi090 ceph-mon[29180]: pgmap v7704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13122.922917 secs 2024-04-26T22:14:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:04 smithi119 ceph-mon[32752]: pgmap v7705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:04 smithi167 ceph-mon[32751]: pgmap v7705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:04 smithi090 ceph-mon[29180]: pgmap v7705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:06 smithi119 ceph-mon[32752]: pgmap v7706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:06 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13127.178890 secs 2024-04-26T22:14:06.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:06 smithi090 ceph-mon[29180]: pgmap v7706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:06.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:06 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13127.178890 secs 2024-04-26T22:14:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:06 smithi167 ceph-mon[32751]: pgmap v7706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:06 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13127.178890 secs 2024-04-26T22:14:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13127.923018 secs 2024-04-26T22:14:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13127.923018 secs 2024-04-26T22:14:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13127.923018 secs 2024-04-26T22:14:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:08 smithi119 ceph-mon[32752]: pgmap v7707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:08 smithi167 ceph-mon[32751]: pgmap v7707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:08 smithi090 ceph-mon[29180]: pgmap v7707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:09 smithi119 ceph-mon[32752]: pgmap v7708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:09 smithi167 ceph-mon[32751]: pgmap v7708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:09 smithi090 ceph-mon[29180]: pgmap v7708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13132.178976 secs 2024-04-26T22:14:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13132.178976 secs 2024-04-26T22:14:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13132.178976 secs 2024-04-26T22:14:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:11 smithi119 ceph-mon[32752]: pgmap v7709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13132.923119 secs 2024-04-26T22:14:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:11 smithi167 ceph-mon[32751]: pgmap v7709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13132.923119 secs 2024-04-26T22:14:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:11 smithi090 ceph-mon[29180]: pgmap v7709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13132.923119 secs 2024-04-26T22:14:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:13 smithi119 ceph-mon[32752]: pgmap v7710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:13 smithi167 ceph-mon[32751]: pgmap v7710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:13 smithi090 ceph-mon[29180]: pgmap v7710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:15 smithi119 ceph-mon[32752]: pgmap v7711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13137.179050 secs 2024-04-26T22:14:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:15 smithi167 ceph-mon[32751]: pgmap v7711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13137.179050 secs 2024-04-26T22:14:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:15 smithi090 ceph-mon[29180]: pgmap v7711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13137.179050 secs 2024-04-26T22:14:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13137.923241 secs 2024-04-26T22:14:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13137.923241 secs 2024-04-26T22:14:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13137.923241 secs 2024-04-26T22:14:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:17 smithi119 ceph-mon[32752]: pgmap v7712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:17 smithi167 ceph-mon[32751]: pgmap v7712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:17 smithi090 ceph-mon[29180]: pgmap v7712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:19 smithi119 ceph-mon[32752]: pgmap v7713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:19 smithi167 ceph-mon[32751]: pgmap v7713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:19 smithi090 ceph-mon[29180]: pgmap v7713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13142.179113 secs 2024-04-26T22:14:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13142.179113 secs 2024-04-26T22:14:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13142.179113 secs 2024-04-26T22:14:21.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:21 smithi090 ceph-mon[29180]: pgmap v7714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:21.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13142.923367 secs 2024-04-26T22:14:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:21 smithi119 ceph-mon[32752]: pgmap v7714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13142.923367 secs 2024-04-26T22:14:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:21 smithi167 ceph-mon[32751]: pgmap v7714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13142.923367 secs 2024-04-26T22:14:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:23 smithi119 ceph-mon[32752]: pgmap v7715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:23 smithi167 ceph-mon[32751]: pgmap v7715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:23 smithi090 ceph-mon[29180]: pgmap v7715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:25 smithi119 ceph-mon[32752]: pgmap v7716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:25 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13147.179166 secs 2024-04-26T22:14:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:25 smithi167 ceph-mon[32751]: pgmap v7716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:25 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13147.179166 secs 2024-04-26T22:14:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:25 smithi090 ceph-mon[29180]: pgmap v7716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:25 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13147.179166 secs 2024-04-26T22:14:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13147.923438 secs 2024-04-26T22:14:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13147.923438 secs 2024-04-26T22:14:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13147.923438 secs 2024-04-26T22:14:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:27 smithi119 ceph-mon[32752]: pgmap v7717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:27 smithi167 ceph-mon[32751]: pgmap v7717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:27 smithi090 ceph-mon[29180]: pgmap v7717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:29 smithi119 ceph-mon[32752]: pgmap v7718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:14:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:14:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:14:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:14:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:29 smithi167 ceph-mon[32751]: pgmap v7718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:14:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:14:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:14:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:14:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:29 smithi090 ceph-mon[29180]: pgmap v7718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:14:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:14:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:14:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:14:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:30 smithi119 ceph-mon[32752]: pgmap v7719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-26T22:14:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:30 smithi119 ceph-mon[32752]: pgmap v7720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:30 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13152.179252 secs 2024-04-26T22:14:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:30 smithi167 ceph-mon[32751]: pgmap v7719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-26T22:14:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:30 smithi167 ceph-mon[32751]: pgmap v7720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:30 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13152.179252 secs 2024-04-26T22:14:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:30 smithi090 ceph-mon[29180]: pgmap v7719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-26T22:14:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:30 smithi090 ceph-mon[29180]: pgmap v7720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:30 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13152.179252 secs 2024-04-26T22:14:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13152.923553 secs 2024-04-26T22:14:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13152.923553 secs 2024-04-26T22:14:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13152.923553 secs 2024-04-26T22:14:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:32 smithi119 ceph-mon[32752]: pgmap v7721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:32 smithi167 ceph-mon[32751]: pgmap v7721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:32 smithi090 ceph-mon[29180]: pgmap v7721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:34 smithi119 ceph-mon[32752]: pgmap v7722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T22:14:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:34 smithi167 ceph-mon[32751]: pgmap v7722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T22:14:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:34 smithi090 ceph-mon[29180]: pgmap v7722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-26T22:14:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:35 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13157.179338 secs 2024-04-26T22:14:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:35 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13157.179338 secs 2024-04-26T22:14:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:35 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13157.179338 secs 2024-04-26T22:14:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:36 smithi119 ceph-mon[32752]: pgmap v7723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13157.923622 secs 2024-04-26T22:14:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:36 smithi167 ceph-mon[32751]: pgmap v7723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13157.923622 secs 2024-04-26T22:14:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:36 smithi090 ceph-mon[29180]: pgmap v7723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:14:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13157.923622 secs 2024-04-26T22:14:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:38 smithi119 ceph-mon[32752]: pgmap v7724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T22:14:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:38 smithi167 ceph-mon[32751]: pgmap v7724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T22:14:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:38 smithi090 ceph-mon[29180]: pgmap v7724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T22:14:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:40 smithi119 ceph-mon[32752]: pgmap v7725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:14:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:40 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13162.179418 secs 2024-04-26T22:14:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:40 smithi167 ceph-mon[32751]: pgmap v7725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:14:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:40 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13162.179418 secs 2024-04-26T22:14:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:40 smithi090 ceph-mon[29180]: pgmap v7725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-26T22:14:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:40 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13162.179418 secs 2024-04-26T22:14:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13162.923704 secs 2024-04-26T22:14:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13162.923704 secs 2024-04-26T22:14:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13162.923704 secs 2024-04-26T22:14:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:42 smithi167 ceph-mon[32751]: pgmap v7726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:14:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:42 smithi090 ceph-mon[29180]: pgmap v7726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:14:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:42 smithi119 ceph-mon[32752]: pgmap v7726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:14:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:44 smithi167 ceph-mon[32751]: pgmap v7727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:44 smithi090 ceph-mon[29180]: pgmap v7727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:44 smithi119 ceph-mon[32752]: pgmap v7727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:45 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13167.179481 secs 2024-04-26T22:14:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:45 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13167.179481 secs 2024-04-26T22:14:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:45 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13167.179481 secs 2024-04-26T22:14:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:46 smithi167 ceph-mon[32751]: pgmap v7728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13167.923862 secs 2024-04-26T22:14:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:46 smithi090 ceph-mon[29180]: pgmap v7728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13167.923862 secs 2024-04-26T22:14:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:46 smithi119 ceph-mon[32752]: pgmap v7728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13167.923862 secs 2024-04-26T22:14:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:48 smithi090 ceph-mon[29180]: pgmap v7729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:14:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:48 smithi119 ceph-mon[32752]: pgmap v7729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:14:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:48 smithi167 ceph-mon[32751]: pgmap v7729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:14:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:50 smithi090 ceph-mon[29180]: pgmap v7730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:50 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13172.179567 secs 2024-04-26T22:14:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:50 smithi119 ceph-mon[32752]: pgmap v7730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:50 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13172.179567 secs 2024-04-26T22:14:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:50 smithi167 ceph-mon[32751]: pgmap v7730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:14:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:50 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13172.179567 secs 2024-04-26T22:14:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13172.923929 secs 2024-04-26T22:14:51.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13172.923929 secs 2024-04-26T22:14:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13172.923929 secs 2024-04-26T22:14:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:52 smithi090 ceph-mon[29180]: pgmap v7731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:52 smithi119 ceph-mon[32752]: pgmap v7731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:52 smithi167 ceph-mon[32751]: pgmap v7731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:54 smithi090 ceph-mon[29180]: pgmap v7732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:54 smithi119 ceph-mon[32752]: pgmap v7732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:54 smithi167 ceph-mon[32751]: pgmap v7732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:14:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:55 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13177.179648 secs 2024-04-26T22:14:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:55 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13177.179648 secs 2024-04-26T22:14:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:55 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13177.179648 secs 2024-04-26T22:14:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:56 smithi090 ceph-mon[29180]: pgmap v7733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:14:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13177.924064 secs 2024-04-26T22:14:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:56 smithi119 ceph-mon[32752]: pgmap v7733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:14:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13177.924064 secs 2024-04-26T22:14:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:56 smithi167 ceph-mon[32751]: pgmap v7733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:14:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13177.924064 secs 2024-04-26T22:14:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:14:58 smithi090 ceph-mon[29180]: pgmap v7734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:14:58 smithi119 ceph-mon[32752]: pgmap v7734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:14:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:14:58 smithi167 ceph-mon[32751]: pgmap v7734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:00 smithi090 ceph-mon[29180]: pgmap v7735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:00 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13182.179735 secs 2024-04-26T22:15:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:00 smithi119 ceph-mon[32752]: pgmap v7735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:00 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13182.179735 secs 2024-04-26T22:15:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:00 smithi167 ceph-mon[32751]: pgmap v7735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:00 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13182.179735 secs 2024-04-26T22:15:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13182.924204 secs 2024-04-26T22:15:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13182.924204 secs 2024-04-26T22:15:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13182.924204 secs 2024-04-26T22:15:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:02 smithi090 ceph-mon[29180]: pgmap v7736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:02 smithi119 ceph-mon[32752]: pgmap v7736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:02 smithi167 ceph-mon[32751]: pgmap v7736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:04.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:15:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:15:04.343+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:15:04.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:15:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:15:04.343+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:15:04.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:15:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:15:04.343+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:15:04.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:15:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:15:04.343+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:15:04.605 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:15:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:15:04.343+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:15:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:04 smithi090 ceph-mon[29180]: pgmap v7737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:04 smithi119 ceph-mon[32752]: pgmap v7737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:04 smithi167 ceph-mon[32751]: pgmap v7737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:05 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13187.179844 secs 2024-04-26T22:15:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:05 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13187.179844 secs 2024-04-26T22:15:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:05 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13187.179844 secs 2024-04-26T22:15:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:06 smithi090 ceph-mon[29180]: pgmap v7738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:15:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13187.924325 secs 2024-04-26T22:15:06.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:06 smithi167 ceph-mon[32751]: pgmap v7738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:15:06.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13187.924325 secs 2024-04-26T22:15:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:06 smithi119 ceph-mon[32752]: pgmap v7738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-26T22:15:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13187.924325 secs 2024-04-26T22:15:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:08 smithi090 ceph-mon[29180]: pgmap v7739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:08 smithi119 ceph-mon[32752]: pgmap v7739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:08 smithi167 ceph-mon[32751]: pgmap v7739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:10 smithi119 ceph-mon[32752]: pgmap v7740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:10 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13192.179929 secs 2024-04-26T22:15:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:10 smithi167 ceph-mon[32751]: pgmap v7740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:10 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13192.179929 secs 2024-04-26T22:15:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:10 smithi090 ceph-mon[29180]: pgmap v7740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:10 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13192.179929 secs 2024-04-26T22:15:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13192.924445 secs 2024-04-26T22:15:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13192.924445 secs 2024-04-26T22:15:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13192.924445 secs 2024-04-26T22:15:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:12 smithi119 ceph-mon[32752]: pgmap v7741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:12 smithi167 ceph-mon[32751]: pgmap v7741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:12 smithi090 ceph-mon[29180]: pgmap v7741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:14.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:14 smithi090 ceph-mon[29180]: pgmap v7742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:14 smithi119 ceph-mon[32752]: pgmap v7742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:14 smithi167 ceph-mon[32751]: pgmap v7742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:15 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13197.180017 secs 2024-04-26T22:15:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:15 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13197.180017 secs 2024-04-26T22:15:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:15 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13197.180017 secs 2024-04-26T22:15:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:16 smithi119 ceph-mon[32752]: pgmap v7743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13197.924565 secs 2024-04-26T22:15:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:16 smithi167 ceph-mon[32751]: pgmap v7743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13197.924565 secs 2024-04-26T22:15:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:16 smithi090 ceph-mon[29180]: pgmap v7743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13197.924565 secs 2024-04-26T22:15:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:18 smithi119 ceph-mon[32752]: pgmap v7744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:18 smithi167 ceph-mon[32751]: pgmap v7744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:18 smithi090 ceph-mon[29180]: pgmap v7744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:20 smithi119 ceph-mon[32752]: pgmap v7745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:20 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 13202.180105 secs 2024-04-26T22:15:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:20 smithi167 ceph-mon[32751]: pgmap v7745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:20 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 13202.180105 secs 2024-04-26T22:15:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:20 smithi090 ceph-mon[29180]: pgmap v7745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:15:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:20 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 13202.180105 secs 2024-04-26T22:15:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13202.924632 secs 2024-04-26T22:15:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13202.924632 secs 2024-04-26T22:15:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13202.924632 secs 2024-04-26T22:15:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:22 smithi119 ceph-mon[32752]: pgmap v7746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:22 smithi167 ceph-mon[32751]: pgmap v7746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:22 smithi090 ceph-mon[29180]: pgmap v7746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:24 smithi119 ceph-mon[32752]: pgmap v7747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:24 smithi167 ceph-mon[32751]: pgmap v7747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:24 smithi090 ceph-mon[29180]: pgmap v7747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:25 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 13207.180165 secs 2024-04-26T22:15:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:25 smithi119 ceph-mon[32752]: slow request 182.146271 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:15:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:25 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 13207.180165 secs 2024-04-26T22:15:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:25 smithi167 ceph-mon[32751]: slow request 182.146271 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:15:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:25 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 13207.180165 secs 2024-04-26T22:15:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:25 smithi090 ceph-mon[29180]: slow request 182.146271 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:15:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:26 smithi119 ceph-mon[32752]: pgmap v7748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13207.924726 secs 2024-04-26T22:15:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:26 smithi167 ceph-mon[32751]: pgmap v7748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13207.924726 secs 2024-04-26T22:15:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:26 smithi090 ceph-mon[29180]: pgmap v7748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13207.924726 secs 2024-04-26T22:15:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:27 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:15:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:27 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:15:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:27 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:15:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:27 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:15:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:27 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:15:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:27 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:15:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:28 smithi119 ceph-mon[32752]: pgmap v7749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:28 smithi167 ceph-mon[32751]: pgmap v7749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:28 smithi090 ceph-mon[29180]: pgmap v7749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:15:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:15:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:15:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:30 smithi119 ceph-mon[32752]: pgmap v7750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:30 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13212.180252 secs 2024-04-26T22:15:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:15:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:15:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:30 smithi119 ceph-mon[32752]: pgmap v7751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 B/s rd, 1 op/s 2024-04-26T22:15:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:30 smithi119 ceph-mon[32752]: pgmap v7752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:30 smithi167 ceph-mon[32751]: pgmap v7750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:30 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13212.180252 secs 2024-04-26T22:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:30 smithi167 ceph-mon[32751]: pgmap v7751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 B/s rd, 1 op/s 2024-04-26T22:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:30 smithi167 ceph-mon[32751]: pgmap v7752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:15:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:30 smithi090 ceph-mon[29180]: pgmap v7750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:30 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13212.180252 secs 2024-04-26T22:15:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:15:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:15:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:30 smithi090 ceph-mon[29180]: pgmap v7751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 B/s rd, 1 op/s 2024-04-26T22:15:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:30 smithi090 ceph-mon[29180]: pgmap v7752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:15:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13212.924863 secs 2024-04-26T22:15:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13212.924863 secs 2024-04-26T22:15:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13212.924863 secs 2024-04-26T22:15:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:32 smithi119 ceph-mon[32752]: pgmap v7753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-26T22:15:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:32 smithi167 ceph-mon[32751]: pgmap v7753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-26T22:15:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:32 smithi090 ceph-mon[29180]: pgmap v7753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-26T22:15:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:35 smithi119 ceph-mon[32752]: pgmap v7754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:35 smithi167 ceph-mon[32751]: pgmap v7754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:35 smithi090 ceph-mon[29180]: pgmap v7754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13217.180311 secs 2024-04-26T22:15:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13217.180311 secs 2024-04-26T22:15:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13217.180311 secs 2024-04-26T22:15:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13217.924972 secs 2024-04-26T22:15:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:37 smithi167 ceph-mon[32751]: pgmap v7755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13217.924972 secs 2024-04-26T22:15:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:37 smithi090 ceph-mon[29180]: pgmap v7755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13217.924972 secs 2024-04-26T22:15:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:37 smithi119 ceph-mon[32752]: pgmap v7755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:15:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:39 smithi167 ceph-mon[32751]: pgmap v7756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-26T22:15:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:39 smithi090 ceph-mon[29180]: pgmap v7756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-26T22:15:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:39 smithi119 ceph-mon[32752]: pgmap v7756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-26T22:15:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13222.180399 secs 2024-04-26T22:15:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:41 smithi090 ceph-mon[29180]: pgmap v7757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:15:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13222.180399 secs 2024-04-26T22:15:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:41 smithi119 ceph-mon[32752]: pgmap v7757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:15:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13222.180399 secs 2024-04-26T22:15:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:41 smithi167 ceph-mon[32751]: pgmap v7757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:15:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13222.925114 secs 2024-04-26T22:15:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13222.925114 secs 2024-04-26T22:15:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13222.925114 secs 2024-04-26T22:15:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:43 smithi090 ceph-mon[29180]: pgmap v7758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:43 smithi119 ceph-mon[32752]: pgmap v7758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:43 smithi167 ceph-mon[32751]: pgmap v7758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:45 smithi090 ceph-mon[29180]: pgmap v7759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:45 smithi119 ceph-mon[32752]: pgmap v7759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:45 smithi167 ceph-mon[32751]: pgmap v7759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13227.180481 secs 2024-04-26T22:15:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13227.180481 secs 2024-04-26T22:15:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13227.180481 secs 2024-04-26T22:15:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13227.925259 secs 2024-04-26T22:15:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:47 smithi090 ceph-mon[29180]: pgmap v7760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13227.925259 secs 2024-04-26T22:15:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:47 smithi119 ceph-mon[32752]: pgmap v7760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13227.925259 secs 2024-04-26T22:15:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:47 smithi167 ceph-mon[32751]: pgmap v7760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:49 smithi090 ceph-mon[29180]: pgmap v7761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:49 smithi119 ceph-mon[32752]: pgmap v7761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:49 smithi167 ceph-mon[32751]: pgmap v7761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:51.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13232.180566 secs 2024-04-26T22:15:51.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:51 smithi119 ceph-mon[32752]: pgmap v7762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13232.180566 secs 2024-04-26T22:15:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:51 smithi090 ceph-mon[29180]: pgmap v7762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13232.180566 secs 2024-04-26T22:15:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:51 smithi167 ceph-mon[32751]: pgmap v7762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13232.925409 secs 2024-04-26T22:15:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13232.925409 secs 2024-04-26T22:15:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13232.925409 secs 2024-04-26T22:15:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:53 smithi090 ceph-mon[29180]: pgmap v7763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:53 smithi119 ceph-mon[32752]: pgmap v7763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:53 smithi167 ceph-mon[32751]: pgmap v7763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:55 smithi090 ceph-mon[29180]: pgmap v7764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:55 smithi119 ceph-mon[32752]: pgmap v7764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:55 smithi167 ceph-mon[32751]: pgmap v7764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:15:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13237.180644 secs 2024-04-26T22:15:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13237.180644 secs 2024-04-26T22:15:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13237.180644 secs 2024-04-26T22:15:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13237.925455 secs 2024-04-26T22:15:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:57 smithi090 ceph-mon[29180]: pgmap v7765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13237.925455 secs 2024-04-26T22:15:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:57 smithi119 ceph-mon[32752]: pgmap v7765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13237.925455 secs 2024-04-26T22:15:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:57 smithi167 ceph-mon[32751]: pgmap v7765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:15:59 smithi090 ceph-mon[29180]: pgmap v7766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:15:59 smithi119 ceph-mon[32752]: pgmap v7766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:15:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:15:59 smithi167 ceph-mon[32751]: pgmap v7766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13242.180719 secs 2024-04-26T22:16:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:01 smithi090 ceph-mon[29180]: pgmap v7767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13242.180719 secs 2024-04-26T22:16:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:01 smithi119 ceph-mon[32752]: pgmap v7767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13242.180719 secs 2024-04-26T22:16:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:01 smithi167 ceph-mon[32751]: pgmap v7767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13242.925551 secs 2024-04-26T22:16:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13242.925551 secs 2024-04-26T22:16:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13242.925551 secs 2024-04-26T22:16:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:03 smithi090 ceph-mon[29180]: pgmap v7768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:03 smithi119 ceph-mon[32752]: pgmap v7768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:03 smithi167 ceph-mon[32751]: pgmap v7768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:05 smithi119 ceph-mon[32752]: pgmap v7769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:05 smithi167 ceph-mon[32751]: pgmap v7769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:05 smithi090 ceph-mon[29180]: pgmap v7769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:06.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13247.180801 secs 2024-04-26T22:16:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13247.180801 secs 2024-04-26T22:16:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13247.180801 secs 2024-04-26T22:16:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13247.925633 secs 2024-04-26T22:16:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:07 smithi119 ceph-mon[32752]: pgmap v7770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13247.925633 secs 2024-04-26T22:16:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:07 smithi167 ceph-mon[32751]: pgmap v7770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13247.925633 secs 2024-04-26T22:16:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:07 smithi090 ceph-mon[29180]: pgmap v7770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:09 smithi119 ceph-mon[32752]: pgmap v7771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:09 smithi167 ceph-mon[32751]: pgmap v7771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:09 smithi090 ceph-mon[29180]: pgmap v7771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13252.180860 secs 2024-04-26T22:16:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:11 smithi119 ceph-mon[32752]: pgmap v7772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13252.180860 secs 2024-04-26T22:16:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:11 smithi167 ceph-mon[32751]: pgmap v7772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13252.180860 secs 2024-04-26T22:16:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:11 smithi090 ceph-mon[29180]: pgmap v7772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13252.925752 secs 2024-04-26T22:16:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13252.925752 secs 2024-04-26T22:16:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13252.925752 secs 2024-04-26T22:16:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:13 smithi119 ceph-mon[32752]: pgmap v7773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:13 smithi167 ceph-mon[32751]: pgmap v7773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:13 smithi090 ceph-mon[29180]: pgmap v7773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:16:15.428 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:15 smithi090 ceph-mon[29180]: pgmap v7774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:15 smithi119 ceph-mon[32752]: pgmap v7774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:15 smithi167 ceph-mon[32751]: pgmap v7774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:16:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13257.180948 secs 2024-04-26T22:16:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13257.180948 secs 2024-04-26T22:16:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13257.180948 secs 2024-04-26T22:16:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13257.925864 secs 2024-04-26T22:16:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:17 smithi119 ceph-mon[32752]: pgmap v7775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13257.925864 secs 2024-04-26T22:16:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:17 smithi167 ceph-mon[32751]: pgmap v7775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13257.925864 secs 2024-04-26T22:16:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:17 smithi090 ceph-mon[29180]: pgmap v7775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:19 smithi119 ceph-mon[32752]: pgmap v7776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:19 smithi167 ceph-mon[32751]: pgmap v7776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:19 smithi090 ceph-mon[29180]: pgmap v7776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:21 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 13262.181009 secs 2024-04-26T22:16:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:21 smithi119 ceph-mon[32752]: slow request 5761.240305 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:16:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:21 smithi119 ceph-mon[32752]: pgmap v7777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:21 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 13262.181009 secs 2024-04-26T22:16:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:21 smithi167 ceph-mon[32751]: slow request 5761.240305 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:16:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:21 smithi167 ceph-mon[32751]: pgmap v7777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:21 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 13262.181009 secs 2024-04-26T22:16:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:21 smithi090 ceph-mon[29180]: slow request 5761.240305 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:16:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:21 smithi090 ceph-mon[29180]: pgmap v7777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13262.925980 secs 2024-04-26T22:16:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13262.925980 secs 2024-04-26T22:16:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13262.925980 secs 2024-04-26T22:16:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:23 smithi119 ceph-mon[32752]: pgmap v7778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:23 smithi167 ceph-mon[32751]: pgmap v7778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:23 smithi090 ceph-mon[29180]: pgmap v7778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:25 smithi119 ceph-mon[32752]: pgmap v7779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:25 smithi167 ceph-mon[32751]: pgmap v7779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:25 smithi090 ceph-mon[29180]: pgmap v7779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13267.181069 secs 2024-04-26T22:16:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13267.181069 secs 2024-04-26T22:16:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13267.181069 secs 2024-04-26T22:16:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13267.926107 secs 2024-04-26T22:16:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:27 smithi119 ceph-mon[32752]: pgmap v7780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13267.926107 secs 2024-04-26T22:16:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:27 smithi167 ceph-mon[32751]: pgmap v7780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13267.926107 secs 2024-04-26T22:16:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:27 smithi090 ceph-mon[29180]: pgmap v7780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:29 smithi119 ceph-mon[32752]: pgmap v7781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:29 smithi167 ceph-mon[32751]: pgmap v7781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:29 smithi090 ceph-mon[29180]: pgmap v7781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:16:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:16:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:16:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13272.181153 secs 2024-04-26T22:16:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:31 smithi119 ceph-mon[32752]: pgmap v7782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:16:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:16:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:16:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13272.181153 secs 2024-04-26T22:16:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:31 smithi167 ceph-mon[32751]: pgmap v7782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:16:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:16:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:16:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13272.181153 secs 2024-04-26T22:16:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:31 smithi090 ceph-mon[29180]: pgmap v7782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:16:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:16:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13272.926192 secs 2024-04-26T22:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:32 smithi119 ceph-mon[32752]: pgmap v7783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:16:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:32 smithi119 ceph-mon[32752]: pgmap v7784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13272.926192 secs 2024-04-26T22:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:32 smithi167 ceph-mon[32751]: pgmap v7783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:32 smithi167 ceph-mon[32751]: pgmap v7784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13272.926192 secs 2024-04-26T22:16:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:32 smithi090 ceph-mon[29180]: pgmap v7783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:16:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:32 smithi090 ceph-mon[29180]: pgmap v7784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:34 smithi119 ceph-mon[32752]: pgmap v7785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:16:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:34 smithi167 ceph-mon[32751]: pgmap v7785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:16:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:34 smithi090 ceph-mon[29180]: pgmap v7785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:16:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:36 smithi119 ceph-mon[32752]: pgmap v7786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13277.181238 secs 2024-04-26T22:16:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:36 smithi167 ceph-mon[32751]: pgmap v7786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13277.181238 secs 2024-04-26T22:16:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:36 smithi090 ceph-mon[29180]: pgmap v7786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13277.181238 secs 2024-04-26T22:16:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13277.926310 secs 2024-04-26T22:16:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13277.926310 secs 2024-04-26T22:16:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13277.926310 secs 2024-04-26T22:16:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:38 smithi119 ceph-mon[32752]: pgmap v7787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:38 smithi167 ceph-mon[32751]: pgmap v7787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:38 smithi090 ceph-mon[29180]: pgmap v7787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:40 smithi119 ceph-mon[32752]: pgmap v7788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:40 smithi167 ceph-mon[32751]: pgmap v7788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:40 smithi090 ceph-mon[29180]: pgmap v7788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:16:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13282.181324 secs 2024-04-26T22:16:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13282.181324 secs 2024-04-26T22:16:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13282.181324 secs 2024-04-26T22:16:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13282.926407 secs 2024-04-26T22:16:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:42 smithi119 ceph-mon[32752]: pgmap v7789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:16:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13282.926407 secs 2024-04-26T22:16:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:42 smithi167 ceph-mon[32751]: pgmap v7789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:16:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13282.926407 secs 2024-04-26T22:16:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:42 smithi090 ceph-mon[29180]: pgmap v7789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:16:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:44 smithi119 ceph-mon[32752]: pgmap v7790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:44 smithi167 ceph-mon[32751]: pgmap v7790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:44 smithi090 ceph-mon[29180]: pgmap v7790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:46 smithi119 ceph-mon[32752]: pgmap v7791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13287.181408 secs 2024-04-26T22:16:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:46 smithi167 ceph-mon[32751]: pgmap v7791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13287.181408 secs 2024-04-26T22:16:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:46 smithi090 ceph-mon[29180]: pgmap v7791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13287.181408 secs 2024-04-26T22:16:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13287.926450 secs 2024-04-26T22:16:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13287.926450 secs 2024-04-26T22:16:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13287.926450 secs 2024-04-26T22:16:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:48 smithi119 ceph-mon[32752]: pgmap v7792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:48 smithi167 ceph-mon[32751]: pgmap v7792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:48 smithi090 ceph-mon[29180]: pgmap v7792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:50 smithi119 ceph-mon[32752]: pgmap v7793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:50 smithi167 ceph-mon[32751]: pgmap v7793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:50 smithi090 ceph-mon[29180]: pgmap v7793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13292.181493 secs 2024-04-26T22:16:51.585 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13292.181493 secs 2024-04-26T22:16:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13292.181493 secs 2024-04-26T22:16:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13292.926589 secs 2024-04-26T22:16:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:52 smithi119 ceph-mon[32752]: pgmap v7794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13292.926589 secs 2024-04-26T22:16:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:52 smithi167 ceph-mon[32751]: pgmap v7794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13292.926589 secs 2024-04-26T22:16:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:52 smithi090 ceph-mon[29180]: pgmap v7794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:53 smithi167 ceph-mon[32751]: pgmap v7795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:53 smithi090 ceph-mon[29180]: pgmap v7795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:53 smithi119 ceph-mon[32752]: pgmap v7795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:55 smithi090 ceph-mon[29180]: pgmap v7796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:55 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13297.181576 secs 2024-04-26T22:16:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:55 smithi119 ceph-mon[32752]: pgmap v7796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:55 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13297.181576 secs 2024-04-26T22:16:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:55 smithi167 ceph-mon[32751]: pgmap v7796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:16:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:55 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13297.181576 secs 2024-04-26T22:16:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13297.926726 secs 2024-04-26T22:16:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13297.926726 secs 2024-04-26T22:16:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13297.926726 secs 2024-04-26T22:16:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:57 smithi090 ceph-mon[29180]: pgmap v7797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:57 smithi119 ceph-mon[32752]: pgmap v7797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:16:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:57 smithi167 ceph-mon[32751]: pgmap v7797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:16:59 smithi090 ceph-mon[29180]: pgmap v7798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:16:59 smithi119 ceph-mon[32752]: pgmap v7798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:16:59 smithi167 ceph-mon[32751]: pgmap v7798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:00 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13302.181637 secs 2024-04-26T22:17:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:00 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13302.181637 secs 2024-04-26T22:17:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:00 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13302.181637 secs 2024-04-26T22:17:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13302.926838 secs 2024-04-26T22:17:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:01 smithi090 ceph-mon[29180]: pgmap v7799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13302.926838 secs 2024-04-26T22:17:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:01 smithi119 ceph-mon[32752]: pgmap v7799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13302.926838 secs 2024-04-26T22:17:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:01 smithi167 ceph-mon[32751]: pgmap v7799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:03 smithi090 ceph-mon[29180]: pgmap v7800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:03 smithi119 ceph-mon[32752]: pgmap v7800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:03 smithi167 ceph-mon[32751]: pgmap v7800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:05 smithi167 ceph-mon[32751]: pgmap v7801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:06.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:05 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13307.181719 secs 2024-04-26T22:17:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:05 smithi090 ceph-mon[29180]: pgmap v7801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:05 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13307.181719 secs 2024-04-26T22:17:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:05 smithi119 ceph-mon[32752]: pgmap v7801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:05 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13307.181719 secs 2024-04-26T22:17:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13307.926937 secs 2024-04-26T22:17:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13307.926937 secs 2024-04-26T22:17:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13307.926937 secs 2024-04-26T22:17:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:07 smithi090 ceph-mon[29180]: pgmap v7802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:07 smithi119 ceph-mon[32752]: pgmap v7802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:07 smithi167 ceph-mon[32751]: pgmap v7802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:09 smithi090 ceph-mon[29180]: pgmap v7803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:09 smithi119 ceph-mon[32752]: pgmap v7803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:09 smithi167 ceph-mon[32751]: pgmap v7803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:10 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13312.181804 secs 2024-04-26T22:17:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:10 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13312.181804 secs 2024-04-26T22:17:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:10 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13312.181804 secs 2024-04-26T22:17:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13312.927038 secs 2024-04-26T22:17:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:11 smithi119 ceph-mon[32752]: pgmap v7804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13312.927038 secs 2024-04-26T22:17:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:11 smithi167 ceph-mon[32751]: pgmap v7804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13312.927038 secs 2024-04-26T22:17:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:11 smithi090 ceph-mon[29180]: pgmap v7804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:13 smithi119 ceph-mon[32752]: pgmap v7805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:13 smithi167 ceph-mon[32751]: pgmap v7805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:13 smithi090 ceph-mon[29180]: pgmap v7805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:16.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:15 smithi090 ceph-mon[29180]: pgmap v7806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:16.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:15 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13317.181894 secs 2024-04-26T22:17:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:15 smithi119 ceph-mon[32752]: pgmap v7806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:15 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13317.181894 secs 2024-04-26T22:17:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:15 smithi167 ceph-mon[32751]: pgmap v7806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:15 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13317.181894 secs 2024-04-26T22:17:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13317.927176 secs 2024-04-26T22:17:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13317.927176 secs 2024-04-26T22:17:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13317.927176 secs 2024-04-26T22:17:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:17 smithi119 ceph-mon[32752]: pgmap v7807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:17 smithi167 ceph-mon[32751]: pgmap v7807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:17 smithi090 ceph-mon[29180]: pgmap v7807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:19 smithi119 ceph-mon[32752]: pgmap v7808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:19 smithi167 ceph-mon[32751]: pgmap v7808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:19 smithi090 ceph-mon[29180]: pgmap v7808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:21.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:20 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13322.181958 secs 2024-04-26T22:17:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:20 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13322.181958 secs 2024-04-26T22:17:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:20 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13322.181958 secs 2024-04-26T22:17:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13322.927248 secs 2024-04-26T22:17:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:21 smithi119 ceph-mon[32752]: pgmap v7809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13322.927248 secs 2024-04-26T22:17:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:21 smithi167 ceph-mon[32751]: pgmap v7809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13322.927248 secs 2024-04-26T22:17:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:21 smithi090 ceph-mon[29180]: pgmap v7809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:23 smithi119 ceph-mon[32752]: pgmap v7810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:23 smithi167 ceph-mon[32751]: pgmap v7810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:23 smithi090 ceph-mon[29180]: pgmap v7810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:25 smithi119 ceph-mon[32752]: pgmap v7811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:25 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13327.182043 secs 2024-04-26T22:17:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:25 smithi167 ceph-mon[32751]: pgmap v7811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:25 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13327.182043 secs 2024-04-26T22:17:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:25 smithi090 ceph-mon[29180]: pgmap v7811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:25 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13327.182043 secs 2024-04-26T22:17:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13327.927373 secs 2024-04-26T22:17:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13327.927373 secs 2024-04-26T22:17:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13327.927373 secs 2024-04-26T22:17:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:27 smithi090 ceph-mon[29180]: pgmap v7812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:27 smithi119 ceph-mon[32752]: pgmap v7812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:27 smithi167 ceph-mon[32751]: pgmap v7812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:29 smithi119 ceph-mon[32752]: pgmap v7813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:29 smithi167 ceph-mon[32751]: pgmap v7813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:29 smithi090 ceph-mon[29180]: pgmap v7813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:30 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13332.182126 secs 2024-04-26T22:17:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:17:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:30 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13332.182126 secs 2024-04-26T22:17:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:17:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:30 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13332.182126 secs 2024-04-26T22:17:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:17:32.219 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13332.927454 secs 2024-04-26T22:17:32.219 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:31 smithi119 ceph-mon[32752]: pgmap v7814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:32.219 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.219 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.219 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.220 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13332.927454 secs 2024-04-26T22:17:32.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:31 smithi090 ceph-mon[29180]: pgmap v7814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:32.311 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.312 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13332.927454 secs 2024-04-26T22:17:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:31 smithi167 ceph-mon[32751]: pgmap v7814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:33 smithi119 ceph-mon[32752]: pgmap v7815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:33 smithi167 ceph-mon[32751]: pgmap v7815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:33 smithi090 ceph-mon[29180]: pgmap v7815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:34.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:17:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: pgmap v7816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: pgmap v7817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 1 op/s 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: pgmap v7818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:35 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13337.182212 secs 2024-04-26T22:17:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: pgmap v7816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: pgmap v7817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 1 op/s 2024-04-26T22:17:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: pgmap v7818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:35.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:35 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13337.182212 secs 2024-04-26T22:17:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:17:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:17:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: pgmap v7816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:17:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:17:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:17:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:17:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: pgmap v7817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 B/s rd, 1 op/s 2024-04-26T22:17:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: pgmap v7818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:17:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:35 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13337.182212 secs 2024-04-26T22:17:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:36 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 13337.927576 secs 2024-04-26T22:17:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:36 smithi119 ceph-mon[32752]: slow request 1441.667708 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:17:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:36 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 13337.927576 secs 2024-04-26T22:17:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:36 smithi167 ceph-mon[32751]: slow request 1441.667708 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:17:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:36 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 13337.927576 secs 2024-04-26T22:17:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:36 smithi090 ceph-mon[29180]: slow request 1441.667708 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:17:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:37 smithi119 ceph-mon[32752]: pgmap v7819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 B/s rd, 1 op/s 2024-04-26T22:17:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:37 smithi167 ceph-mon[32751]: pgmap v7819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 B/s rd, 1 op/s 2024-04-26T22:17:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:37 smithi090 ceph-mon[29180]: pgmap v7819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 B/s rd, 1 op/s 2024-04-26T22:17:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:39 smithi090 ceph-mon[29180]: pgmap v7820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:39 smithi119 ceph-mon[32752]: pgmap v7820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:39 smithi167 ceph-mon[32751]: pgmap v7820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:40 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13342.182296 secs 2024-04-26T22:17:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:40 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13342.182296 secs 2024-04-26T22:17:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:40 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13342.182296 secs 2024-04-26T22:17:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13342.927640 secs 2024-04-26T22:17:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:41 smithi090 ceph-mon[29180]: pgmap v7821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13342.927640 secs 2024-04-26T22:17:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:41 smithi119 ceph-mon[32752]: pgmap v7821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13342.927640 secs 2024-04-26T22:17:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:41 smithi167 ceph-mon[32751]: pgmap v7821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:43 smithi090 ceph-mon[29180]: pgmap v7822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 B/s rd, 1 op/s 2024-04-26T22:17:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:43 smithi119 ceph-mon[32752]: pgmap v7822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 B/s rd, 1 op/s 2024-04-26T22:17:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:43 smithi167 ceph-mon[32751]: pgmap v7822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 B/s rd, 1 op/s 2024-04-26T22:17:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:45 smithi090 ceph-mon[29180]: pgmap v7823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:17:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:45 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13347.182380 secs 2024-04-26T22:17:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:45 smithi119 ceph-mon[32752]: pgmap v7823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:17:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:45 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13347.182380 secs 2024-04-26T22:17:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:45 smithi167 ceph-mon[32751]: pgmap v7823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:17:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:45 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13347.182380 secs 2024-04-26T22:17:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13347.927765 secs 2024-04-26T22:17:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13347.927765 secs 2024-04-26T22:17:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13347.927765 secs 2024-04-26T22:17:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:47 smithi119 ceph-mon[32752]: pgmap v7824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:47 smithi167 ceph-mon[32751]: pgmap v7824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:47 smithi090 ceph-mon[29180]: pgmap v7824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:49 smithi119 ceph-mon[32752]: pgmap v7825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:49 smithi167 ceph-mon[32751]: pgmap v7825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:49 smithi090 ceph-mon[29180]: pgmap v7825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:51.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:50 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13352.182466 secs 2024-04-26T22:17:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:50 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13352.182466 secs 2024-04-26T22:17:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:50 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13352.182466 secs 2024-04-26T22:17:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13352.927888 secs 2024-04-26T22:17:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:51 smithi119 ceph-mon[32752]: pgmap v7826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13352.927888 secs 2024-04-26T22:17:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:51 smithi167 ceph-mon[32751]: pgmap v7826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13352.927888 secs 2024-04-26T22:17:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:51 smithi090 ceph-mon[29180]: pgmap v7826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:53 smithi119 ceph-mon[32752]: pgmap v7827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:53 smithi167 ceph-mon[32751]: pgmap v7827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:53 smithi090 ceph-mon[29180]: pgmap v7827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:55 smithi119 ceph-mon[32752]: pgmap v7828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:55 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13357.182572 secs 2024-04-26T22:17:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:55 smithi167 ceph-mon[32751]: pgmap v7828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:55 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13357.182572 secs 2024-04-26T22:17:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:55 smithi090 ceph-mon[29180]: pgmap v7828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:17:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:55 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13357.182572 secs 2024-04-26T22:17:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13357.927976 secs 2024-04-26T22:17:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13357.927976 secs 2024-04-26T22:17:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13357.927976 secs 2024-04-26T22:17:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:57 smithi119 ceph-mon[32752]: pgmap v7829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:57 smithi167 ceph-mon[32751]: pgmap v7829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:17:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:57 smithi090 ceph-mon[29180]: pgmap v7829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:17:59 smithi119 ceph-mon[32752]: pgmap v7830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:17:59 smithi167 ceph-mon[32751]: pgmap v7830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:17:59 smithi090 ceph-mon[29180]: pgmap v7830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:00 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13362.182657 secs 2024-04-26T22:18:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:00 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13362.182657 secs 2024-04-26T22:18:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:00 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13362.182657 secs 2024-04-26T22:18:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13362.928104 secs 2024-04-26T22:18:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:01 smithi119 ceph-mon[32752]: pgmap v7831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13362.928104 secs 2024-04-26T22:18:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:01 smithi167 ceph-mon[32751]: pgmap v7831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13362.928104 secs 2024-04-26T22:18:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:01 smithi090 ceph-mon[29180]: pgmap v7831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:03 smithi119 ceph-mon[32752]: pgmap v7832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:03 smithi167 ceph-mon[32751]: pgmap v7832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:03 smithi090 ceph-mon[29180]: pgmap v7832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:05 smithi119 ceph-mon[32752]: pgmap v7833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:05 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13367.182744 secs 2024-04-26T22:18:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:05 smithi167 ceph-mon[32751]: pgmap v7833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:05 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13367.182744 secs 2024-04-26T22:18:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:05 smithi090 ceph-mon[29180]: pgmap v7833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:05 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13367.182744 secs 2024-04-26T22:18:07.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13367.928231 secs 2024-04-26T22:18:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13367.928231 secs 2024-04-26T22:18:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13367.928231 secs 2024-04-26T22:18:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:07 smithi119 ceph-mon[32752]: pgmap v7834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:07 smithi167 ceph-mon[32751]: pgmap v7834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:07 smithi090 ceph-mon[29180]: pgmap v7834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:09 smithi119 ceph-mon[32752]: pgmap v7835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:09 smithi167 ceph-mon[32751]: pgmap v7835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:09 smithi090 ceph-mon[29180]: pgmap v7835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:10 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13372.182828 secs 2024-04-26T22:18:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:10 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13372.182828 secs 2024-04-26T22:18:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:10 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13372.182828 secs 2024-04-26T22:18:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13372.928420 secs 2024-04-26T22:18:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:11 smithi119 ceph-mon[32752]: pgmap v7836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13372.928420 secs 2024-04-26T22:18:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:11 smithi167 ceph-mon[32751]: pgmap v7836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13372.928420 secs 2024-04-26T22:18:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:11 smithi090 ceph-mon[29180]: pgmap v7836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:13 smithi119 ceph-mon[32752]: pgmap v7837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:13 smithi167 ceph-mon[32751]: pgmap v7837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:13 smithi090 ceph-mon[29180]: pgmap v7837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:15 smithi119 ceph-mon[32752]: pgmap v7838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:15 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13377.182909 secs 2024-04-26T22:18:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:15 smithi167 ceph-mon[32751]: pgmap v7838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:15 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13377.182909 secs 2024-04-26T22:18:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:15 smithi090 ceph-mon[29180]: pgmap v7838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:15 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13377.182909 secs 2024-04-26T22:18:17.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13377.928545 secs 2024-04-26T22:18:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13377.928545 secs 2024-04-26T22:18:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13377.928545 secs 2024-04-26T22:18:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:17 smithi119 ceph-mon[32752]: pgmap v7839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:17 smithi167 ceph-mon[32751]: pgmap v7839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:17 smithi090 ceph-mon[29180]: pgmap v7839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:19 smithi119 ceph-mon[32752]: pgmap v7840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:19 smithi167 ceph-mon[32751]: pgmap v7840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:19 smithi090 ceph-mon[29180]: pgmap v7840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:20 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13382.183000 secs 2024-04-26T22:18:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:20 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13382.183000 secs 2024-04-26T22:18:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:20 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13382.183000 secs 2024-04-26T22:18:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13382.928672 secs 2024-04-26T22:18:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:21 smithi119 ceph-mon[32752]: pgmap v7841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13382.928672 secs 2024-04-26T22:18:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:21 smithi167 ceph-mon[32751]: pgmap v7841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13382.928672 secs 2024-04-26T22:18:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:21 smithi090 ceph-mon[29180]: pgmap v7841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:23 smithi119 ceph-mon[32752]: pgmap v7842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:23 smithi167 ceph-mon[32751]: pgmap v7842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:23 smithi090 ceph-mon[29180]: pgmap v7842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:25 smithi119 ceph-mon[32752]: pgmap v7843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:25 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 13387.183085 secs 2024-04-26T22:18:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:25 smithi119 ceph-mon[32752]: slow request 362.149191 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:18:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:25 smithi167 ceph-mon[32751]: pgmap v7843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:25 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 13387.183085 secs 2024-04-26T22:18:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:25 smithi167 ceph-mon[32751]: slow request 362.149191 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:18:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:25 smithi090 ceph-mon[29180]: pgmap v7843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:25 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 13387.183085 secs 2024-04-26T22:18:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:25 smithi090 ceph-mon[29180]: slow request 362.149191 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:18:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13387.928736 secs 2024-04-26T22:18:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13387.928736 secs 2024-04-26T22:18:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13387.928736 secs 2024-04-26T22:18:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:27 smithi119 ceph-mon[32752]: pgmap v7844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:27 smithi167 ceph-mon[32751]: pgmap v7844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:27 smithi090 ceph-mon[29180]: pgmap v7844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:29 smithi119 ceph-mon[32752]: pgmap v7845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:29 smithi167 ceph-mon[32751]: pgmap v7845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:29 smithi090 ceph-mon[29180]: pgmap v7845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:30 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13392.183170 secs 2024-04-26T22:18:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:30 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13392.183170 secs 2024-04-26T22:18:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:30 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13392.183170 secs 2024-04-26T22:18:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13392.928858 secs 2024-04-26T22:18:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:31 smithi119 ceph-mon[32752]: pgmap v7846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13392.928858 secs 2024-04-26T22:18:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:31 smithi167 ceph-mon[32751]: pgmap v7846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13392.928858 secs 2024-04-26T22:18:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:31 smithi090 ceph-mon[29180]: pgmap v7846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:33 smithi119 ceph-mon[32752]: pgmap v7847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:33 smithi167 ceph-mon[32751]: pgmap v7847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:33 smithi090 ceph-mon[29180]: pgmap v7847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:18:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:18:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:18:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:36 smithi119 ceph-mon[32752]: pgmap v7848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13397.183234 secs 2024-04-26T22:18:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:36 smithi167 ceph-mon[32751]: pgmap v7848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13397.183234 secs 2024-04-26T22:18:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:36 smithi090 ceph-mon[29180]: pgmap v7848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13397.183234 secs 2024-04-26T22:18:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13397.928962 secs 2024-04-26T22:18:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:18:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:18:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:37 smithi119 ceph-mon[32752]: pgmap v7849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 909 B/s rd, 1 op/s 2024-04-26T22:18:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:37 smithi119 ceph-mon[32752]: pgmap v7850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13397.928962 secs 2024-04-26T22:18:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:18:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:18:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:37 smithi167 ceph-mon[32751]: pgmap v7849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 909 B/s rd, 1 op/s 2024-04-26T22:18:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:37 smithi167 ceph-mon[32751]: pgmap v7850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13397.928962 secs 2024-04-26T22:18:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:18:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:18:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:37 smithi090 ceph-mon[29180]: pgmap v7849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 909 B/s rd, 1 op/s 2024-04-26T22:18:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:37 smithi090 ceph-mon[29180]: pgmap v7850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:18:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:39 smithi090 ceph-mon[29180]: pgmap v7851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T22:18:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:39 smithi119 ceph-mon[32752]: pgmap v7851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T22:18:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:39 smithi167 ceph-mon[32751]: pgmap v7851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-26T22:18:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13402.183321 secs 2024-04-26T22:18:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:41 smithi119 ceph-mon[32752]: pgmap v7852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13402.183321 secs 2024-04-26T22:18:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:41 smithi167 ceph-mon[32751]: pgmap v7852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13402.183321 secs 2024-04-26T22:18:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:41 smithi090 ceph-mon[29180]: pgmap v7852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13402.929078 secs 2024-04-26T22:18:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13402.929078 secs 2024-04-26T22:18:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13402.929078 secs 2024-04-26T22:18:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:43 smithi119 ceph-mon[32752]: pgmap v7853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:43.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:43 smithi167 ceph-mon[32751]: pgmap v7853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:43 smithi090 ceph-mon[29180]: pgmap v7853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:45 smithi119 ceph-mon[32752]: pgmap v7854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:45 smithi167 ceph-mon[32751]: pgmap v7854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:45 smithi090 ceph-mon[29180]: pgmap v7854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:18:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13407.183404 secs 2024-04-26T22:18:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13407.183404 secs 2024-04-26T22:18:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13407.183404 secs 2024-04-26T22:18:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13407.929143 secs 2024-04-26T22:18:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:47 smithi119 ceph-mon[32752]: pgmap v7855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:18:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13407.929143 secs 2024-04-26T22:18:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:47 smithi167 ceph-mon[32751]: pgmap v7855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:18:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13407.929143 secs 2024-04-26T22:18:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:47 smithi090 ceph-mon[29180]: pgmap v7855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:18:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:49 smithi119 ceph-mon[32752]: pgmap v7856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:49 smithi167 ceph-mon[32751]: pgmap v7856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:49 smithi090 ceph-mon[29180]: pgmap v7856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13412.183488 secs 2024-04-26T22:18:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:51 smithi119 ceph-mon[32752]: pgmap v7857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13412.183488 secs 2024-04-26T22:18:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:51 smithi167 ceph-mon[32751]: pgmap v7857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13412.183488 secs 2024-04-26T22:18:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:51 smithi090 ceph-mon[29180]: pgmap v7857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13412.929255 secs 2024-04-26T22:18:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13412.929255 secs 2024-04-26T22:18:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13412.929255 secs 2024-04-26T22:18:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:53 smithi119 ceph-mon[32752]: pgmap v7858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:53.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:53 smithi167 ceph-mon[32751]: pgmap v7858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:53 smithi090 ceph-mon[29180]: pgmap v7858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:55 smithi119 ceph-mon[32752]: pgmap v7859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:55 smithi167 ceph-mon[32751]: pgmap v7859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:55 smithi090 ceph-mon[29180]: pgmap v7859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:18:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13417.183573 secs 2024-04-26T22:18:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13417.183573 secs 2024-04-26T22:18:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13417.183573 secs 2024-04-26T22:18:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13417.929372 secs 2024-04-26T22:18:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:57 smithi119 ceph-mon[32752]: pgmap v7860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13417.929372 secs 2024-04-26T22:18:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:57 smithi167 ceph-mon[32751]: pgmap v7860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13417.929372 secs 2024-04-26T22:18:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:57 smithi090 ceph-mon[29180]: pgmap v7860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:18:59 smithi119 ceph-mon[32752]: pgmap v7861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:18:59 smithi167 ceph-mon[32751]: pgmap v7861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:18:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:18:59 smithi090 ceph-mon[29180]: pgmap v7861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13422.183655 secs 2024-04-26T22:19:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:01 smithi119 ceph-mon[32752]: pgmap v7862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13422.183655 secs 2024-04-26T22:19:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:01 smithi167 ceph-mon[32751]: pgmap v7862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13422.183655 secs 2024-04-26T22:19:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:01 smithi090 ceph-mon[29180]: pgmap v7862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:02.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13422.929492 secs 2024-04-26T22:19:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13422.929492 secs 2024-04-26T22:19:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13422.929492 secs 2024-04-26T22:19:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:03 smithi119 ceph-mon[32752]: pgmap v7863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:03 smithi167 ceph-mon[32751]: pgmap v7863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:03 smithi090 ceph-mon[29180]: pgmap v7863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:05 smithi119 ceph-mon[32752]: pgmap v7864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:05 smithi167 ceph-mon[32751]: pgmap v7864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:05 smithi090 ceph-mon[29180]: pgmap v7864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13427.183739 secs 2024-04-26T22:19:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13427.183739 secs 2024-04-26T22:19:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13427.183739 secs 2024-04-26T22:19:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13427.929608 secs 2024-04-26T22:19:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:07 smithi167 ceph-mon[32751]: pgmap v7865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13427.929608 secs 2024-04-26T22:19:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:07 smithi119 ceph-mon[32752]: pgmap v7865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13427.929608 secs 2024-04-26T22:19:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:07 smithi090 ceph-mon[29180]: pgmap v7865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:09 smithi119 ceph-mon[32752]: pgmap v7866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:09 smithi167 ceph-mon[32751]: pgmap v7866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:09 smithi090 ceph-mon[29180]: pgmap v7866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13432.183849 secs 2024-04-26T22:19:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:11 smithi119 ceph-mon[32752]: pgmap v7867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13432.183849 secs 2024-04-26T22:19:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:11 smithi167 ceph-mon[32751]: pgmap v7867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13432.183849 secs 2024-04-26T22:19:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:11 smithi090 ceph-mon[29180]: pgmap v7867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13432.929753 secs 2024-04-26T22:19:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13432.929753 secs 2024-04-26T22:19:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13432.929753 secs 2024-04-26T22:19:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:13 smithi119 ceph-mon[32752]: pgmap v7868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:13 smithi167 ceph-mon[32751]: pgmap v7868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:13 smithi090 ceph-mon[29180]: pgmap v7868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:15 smithi119 ceph-mon[32752]: pgmap v7869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:15 smithi167 ceph-mon[32751]: pgmap v7869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:15 smithi090 ceph-mon[29180]: pgmap v7869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13437.183908 secs 2024-04-26T22:19:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13437.183908 secs 2024-04-26T22:19:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13437.183908 secs 2024-04-26T22:19:17.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13437.929880 secs 2024-04-26T22:19:17.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:17 smithi090 ceph-mon[29180]: pgmap v7870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13437.929880 secs 2024-04-26T22:19:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:17 smithi119 ceph-mon[32752]: pgmap v7870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13437.929880 secs 2024-04-26T22:19:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:17 smithi167 ceph-mon[32751]: pgmap v7870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:19 smithi119 ceph-mon[32752]: pgmap v7871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:19 smithi167 ceph-mon[32751]: pgmap v7871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:19 smithi090 ceph-mon[29180]: pgmap v7871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13442.183970 secs 2024-04-26T22:19:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:21 smithi119 ceph-mon[32752]: pgmap v7872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13442.183970 secs 2024-04-26T22:19:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:21 smithi167 ceph-mon[32751]: pgmap v7872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13442.183970 secs 2024-04-26T22:19:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:21 smithi090 ceph-mon[29180]: pgmap v7872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13442.930008 secs 2024-04-26T22:19:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13442.930008 secs 2024-04-26T22:19:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13442.930008 secs 2024-04-26T22:19:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:23 smithi119 ceph-mon[32752]: pgmap v7873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:23 smithi167 ceph-mon[32751]: pgmap v7873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:23 smithi090 ceph-mon[29180]: pgmap v7873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:25 smithi119 ceph-mon[32752]: pgmap v7874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:25 smithi167 ceph-mon[32751]: pgmap v7874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:25 smithi090 ceph-mon[29180]: pgmap v7874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13447.184054 secs 2024-04-26T22:19:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13447.184054 secs 2024-04-26T22:19:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13447.184054 secs 2024-04-26T22:19:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13447.930171 secs 2024-04-26T22:19:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:27 smithi119 ceph-mon[32752]: pgmap v7875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13447.930171 secs 2024-04-26T22:19:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:27 smithi167 ceph-mon[32751]: pgmap v7875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13447.930171 secs 2024-04-26T22:19:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:27 smithi090 ceph-mon[29180]: pgmap v7875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:29 smithi119 ceph-mon[32752]: pgmap v7876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:29 smithi167 ceph-mon[32751]: pgmap v7876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:29 smithi090 ceph-mon[29180]: pgmap v7876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13452.184142 secs 2024-04-26T22:19:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:31 smithi119 ceph-mon[32752]: pgmap v7877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13452.184142 secs 2024-04-26T22:19:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:31 smithi167 ceph-mon[32751]: pgmap v7877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13452.184142 secs 2024-04-26T22:19:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:31 smithi090 ceph-mon[29180]: pgmap v7877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13452.930310 secs 2024-04-26T22:19:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13452.930310 secs 2024-04-26T22:19:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13452.930310 secs 2024-04-26T22:19:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:33 smithi119 ceph-mon[32752]: pgmap v7878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:33 smithi167 ceph-mon[32751]: pgmap v7878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:33 smithi090 ceph-mon[29180]: pgmap v7878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:19:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:35 smithi119 ceph-mon[32752]: pgmap v7879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:35 smithi167 ceph-mon[32751]: pgmap v7879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:35 smithi090 ceph-mon[29180]: pgmap v7879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13457.184226 secs 2024-04-26T22:19:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:19:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13457.184226 secs 2024-04-26T22:19:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:19:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13457.184226 secs 2024-04-26T22:19:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:19:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13457.930457 secs 2024-04-26T22:19:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:37 smithi119 ceph-mon[32752]: pgmap v7880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:19:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:19:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:19:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13457.930457 secs 2024-04-26T22:19:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:37 smithi167 ceph-mon[32751]: pgmap v7880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:19:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:19:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:19:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13457.930457 secs 2024-04-26T22:19:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:37 smithi090 ceph-mon[29180]: pgmap v7880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:19:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:19:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:19:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:38 smithi119 ceph-mon[32752]: pgmap v7881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:19:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:38 smithi119 ceph-mon[32752]: pgmap v7882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:38 smithi167 ceph-mon[32751]: pgmap v7881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:19:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:38 smithi167 ceph-mon[32751]: pgmap v7882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:38 smithi090 ceph-mon[29180]: pgmap v7881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:19:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:38 smithi090 ceph-mon[29180]: pgmap v7882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:39 smithi119 ceph-mon[32752]: pgmap v7883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:39 smithi167 ceph-mon[32751]: pgmap v7883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:39 smithi090 ceph-mon[29180]: pgmap v7883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:40 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13462.184323 secs 2024-04-26T22:19:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:40 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13462.184323 secs 2024-04-26T22:19:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:40 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13462.184323 secs 2024-04-26T22:19:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13462.930519 secs 2024-04-26T22:19:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:41 smithi119 ceph-mon[32752]: pgmap v7884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13462.930519 secs 2024-04-26T22:19:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:41 smithi167 ceph-mon[32751]: pgmap v7884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13462.930519 secs 2024-04-26T22:19:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:41 smithi090 ceph-mon[29180]: pgmap v7884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:43 smithi119 ceph-mon[32752]: pgmap v7885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:19:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:43 smithi167 ceph-mon[32751]: pgmap v7885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:19:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:43 smithi090 ceph-mon[29180]: pgmap v7885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:19:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:45 smithi119 ceph-mon[32752]: pgmap v7886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:45 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13467.184408 secs 2024-04-26T22:19:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:45 smithi167 ceph-mon[32751]: pgmap v7886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:45 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13467.184408 secs 2024-04-26T22:19:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:45 smithi090 ceph-mon[29180]: pgmap v7886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:19:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:45 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13467.184408 secs 2024-04-26T22:19:47.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13467.930647 secs 2024-04-26T22:19:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13467.930647 secs 2024-04-26T22:19:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13467.930647 secs 2024-04-26T22:19:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:47 smithi119 ceph-mon[32752]: pgmap v7887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:19:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:47 smithi167 ceph-mon[32751]: pgmap v7887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:19:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:47 smithi090 ceph-mon[29180]: pgmap v7887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:19:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:49 smithi119 ceph-mon[32752]: pgmap v7888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:49 smithi167 ceph-mon[32751]: pgmap v7888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:49 smithi090 ceph-mon[29180]: pgmap v7888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:51.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:50 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13472.184495 secs 2024-04-26T22:19:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:50 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13472.184495 secs 2024-04-26T22:19:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:50 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13472.184495 secs 2024-04-26T22:19:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13472.930763 secs 2024-04-26T22:19:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:51 smithi119 ceph-mon[32752]: pgmap v7889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13472.930763 secs 2024-04-26T22:19:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:51 smithi167 ceph-mon[32751]: pgmap v7889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13472.930763 secs 2024-04-26T22:19:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:51 smithi090 ceph-mon[29180]: pgmap v7889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:53 smithi119 ceph-mon[32752]: pgmap v7890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:53 smithi167 ceph-mon[32751]: pgmap v7890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:53 smithi090 ceph-mon[29180]: pgmap v7890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:55 smithi119 ceph-mon[32752]: pgmap v7891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:55 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13477.184596 secs 2024-04-26T22:19:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:55 smithi167 ceph-mon[32751]: pgmap v7891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:55 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13477.184596 secs 2024-04-26T22:19:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:55 smithi090 ceph-mon[29180]: pgmap v7891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:19:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:55 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13477.184596 secs 2024-04-26T22:19:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13477.930884 secs 2024-04-26T22:19:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13477.930884 secs 2024-04-26T22:19:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13477.930884 secs 2024-04-26T22:19:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:57 smithi119 ceph-mon[32752]: pgmap v7892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:57 smithi167 ceph-mon[32751]: pgmap v7892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:19:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:57 smithi090 ceph-mon[29180]: pgmap v7892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:19:59 smithi119 ceph-mon[32752]: pgmap v7893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:19:59 smithi167 ceph-mon[32751]: pgmap v7893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:19:59 smithi090 ceph-mon[29180]: pgmap v7893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:00 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13482.184680 secs 2024-04-26T22:20:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:20:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:00 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13482.184680 secs 2024-04-26T22:20:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:20:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:00 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13482.184680 secs 2024-04-26T22:20:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:20:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13482.931004 secs 2024-04-26T22:20:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:01 smithi119 ceph-mon[32752]: pgmap v7894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13482.931004 secs 2024-04-26T22:20:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:01 smithi167 ceph-mon[32751]: pgmap v7894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13482.931004 secs 2024-04-26T22:20:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:01 smithi090 ceph-mon[29180]: pgmap v7894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:03 smithi119 ceph-mon[32752]: pgmap v7895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:03 smithi167 ceph-mon[32751]: pgmap v7895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:03 smithi090 ceph-mon[29180]: pgmap v7895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:05 smithi119 ceph-mon[32752]: pgmap v7896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:05 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13487.184762 secs 2024-04-26T22:20:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:05 smithi167 ceph-mon[32751]: pgmap v7896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:05 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13487.184762 secs 2024-04-26T22:20:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:05 smithi090 ceph-mon[29180]: pgmap v7896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:05 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13487.184762 secs 2024-04-26T22:20:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13487.931069 secs 2024-04-26T22:20:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13487.931069 secs 2024-04-26T22:20:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13487.931069 secs 2024-04-26T22:20:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:07 smithi119 ceph-mon[32752]: pgmap v7897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:07 smithi167 ceph-mon[32751]: pgmap v7897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:07 smithi090 ceph-mon[29180]: pgmap v7897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:09 smithi119 ceph-mon[32752]: pgmap v7898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:09 smithi167 ceph-mon[32751]: pgmap v7898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:09 smithi090 ceph-mon[29180]: pgmap v7898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:10 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13492.184846 secs 2024-04-26T22:20:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:10 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13492.184846 secs 2024-04-26T22:20:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:10 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13492.184846 secs 2024-04-26T22:20:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13492.931142 secs 2024-04-26T22:20:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:11 smithi119 ceph-mon[32752]: pgmap v7899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13492.931142 secs 2024-04-26T22:20:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:11 smithi167 ceph-mon[32751]: pgmap v7899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13492.931142 secs 2024-04-26T22:20:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:11 smithi090 ceph-mon[29180]: pgmap v7899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:13 smithi119 ceph-mon[32752]: pgmap v7900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:13 smithi167 ceph-mon[32751]: pgmap v7900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:13 smithi090 ceph-mon[29180]: pgmap v7900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:15 smithi119 ceph-mon[32752]: pgmap v7901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:15 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 13497.184930 secs 2024-04-26T22:20:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:15 smithi119 ceph-mon[32752]: slow request 11520.455065 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:20:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:15 smithi167 ceph-mon[32751]: pgmap v7901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:15 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 13497.184930 secs 2024-04-26T22:20:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:15 smithi167 ceph-mon[32751]: slow request 11520.455065 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:20:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:15 smithi090 ceph-mon[29180]: pgmap v7901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:15 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 13497.184930 secs 2024-04-26T22:20:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:15 smithi090 ceph-mon[29180]: slow request 11520.455065 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:20:17.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13497.931262 secs 2024-04-26T22:20:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13497.931262 secs 2024-04-26T22:20:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13497.931262 secs 2024-04-26T22:20:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:18 smithi119 ceph-mon[32752]: pgmap v7902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:18 smithi167 ceph-mon[32751]: pgmap v7902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:17 smithi090 ceph-mon[29180]: pgmap v7902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:20:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:20 smithi090 ceph-mon[29180]: pgmap v7903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:20 smithi119 ceph-mon[32752]: pgmap v7903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:20 smithi167 ceph-mon[32751]: pgmap v7903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13502.185015 secs 2024-04-26T22:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13502.185015 secs 2024-04-26T22:20:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13502.185015 secs 2024-04-26T22:20:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13502.931420 secs 2024-04-26T22:20:22.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:22 smithi119 ceph-mon[32752]: pgmap v7904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13502.931420 secs 2024-04-26T22:20:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:22 smithi167 ceph-mon[32751]: pgmap v7904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13502.931420 secs 2024-04-26T22:20:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:22 smithi090 ceph-mon[29180]: pgmap v7904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:24 smithi119 ceph-mon[32752]: pgmap v7905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:24 smithi167 ceph-mon[32751]: pgmap v7905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:24 smithi090 ceph-mon[29180]: pgmap v7905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:26 smithi119 ceph-mon[32752]: pgmap v7906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13507.185096 secs 2024-04-26T22:20:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:26 smithi167 ceph-mon[32751]: pgmap v7906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13507.185096 secs 2024-04-26T22:20:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:26 smithi090 ceph-mon[29180]: pgmap v7906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13507.185096 secs 2024-04-26T22:20:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13507.931559 secs 2024-04-26T22:20:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13507.931559 secs 2024-04-26T22:20:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13507.931559 secs 2024-04-26T22:20:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:28 smithi119 ceph-mon[32752]: pgmap v7907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:28 smithi167 ceph-mon[32751]: pgmap v7907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:28 smithi090 ceph-mon[29180]: pgmap v7907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:30 smithi119 ceph-mon[32752]: pgmap v7908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:30 smithi167 ceph-mon[32751]: pgmap v7908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:30 smithi090 ceph-mon[29180]: pgmap v7908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 13512.185181 secs 2024-04-26T22:20:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 13512.185181 secs 2024-04-26T22:20:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 13512.185181 secs 2024-04-26T22:20:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13512.931630 secs 2024-04-26T22:20:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:32 smithi119 ceph-mon[32752]: pgmap v7909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13512.931630 secs 2024-04-26T22:20:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:32 smithi167 ceph-mon[32751]: pgmap v7909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13512.931630 secs 2024-04-26T22:20:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:32 smithi090 ceph-mon[29180]: pgmap v7909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:34 smithi119 ceph-mon[32752]: pgmap v7910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:34 smithi167 ceph-mon[32751]: pgmap v7910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:34 smithi090 ceph-mon[29180]: pgmap v7910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:34.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:20:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:20:34.409+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:20:34.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:20:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:20:34.409+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:20:34.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:20:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:20:34.409+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:20:34.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:20:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:20:34.409+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:20:34.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:20:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:20:34.409+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:20:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:36 smithi119 ceph-mon[32752]: pgmap v7911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13517.185266 secs 2024-04-26T22:20:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:36 smithi167 ceph-mon[32751]: pgmap v7911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13517.185266 secs 2024-04-26T22:20:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:36 smithi090 ceph-mon[29180]: pgmap v7911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13517.185266 secs 2024-04-26T22:20:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13517.931756 secs 2024-04-26T22:20:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:20:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13517.931756 secs 2024-04-26T22:20:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:20:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13517.931756 secs 2024-04-26T22:20:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:20:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:38 smithi167 ceph-mon[32751]: pgmap v7912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:20:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:20:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:20:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:38 smithi090 ceph-mon[29180]: pgmap v7912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:20:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:20:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:20:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:38 smithi119 ceph-mon[32752]: pgmap v7912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:20:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:20:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:20:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:39 smithi167 ceph-mon[32751]: pgmap v7913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:20:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:39 smithi167 ceph-mon[32751]: pgmap v7914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:20:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:20:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:20:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:39 smithi090 ceph-mon[29180]: pgmap v7913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:20:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:39 smithi090 ceph-mon[29180]: pgmap v7914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:20:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:20:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:20:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:39 smithi119 ceph-mon[32752]: pgmap v7913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:20:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:39 smithi119 ceph-mon[32752]: pgmap v7914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:20:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:20:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:20:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:41 smithi167 ceph-mon[32751]: pgmap v7915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13522.185353 secs 2024-04-26T22:20:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:41 smithi090 ceph-mon[29180]: pgmap v7915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13522.185353 secs 2024-04-26T22:20:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:41 smithi119 ceph-mon[32752]: pgmap v7915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13522.185353 secs 2024-04-26T22:20:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13522.931819 secs 2024-04-26T22:20:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13522.931819 secs 2024-04-26T22:20:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13522.931819 secs 2024-04-26T22:20:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:43 smithi167 ceph-mon[32751]: pgmap v7916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:43 smithi090 ceph-mon[29180]: pgmap v7916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:43 smithi119 ceph-mon[32752]: pgmap v7916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:45 smithi090 ceph-mon[29180]: pgmap v7917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:45 smithi119 ceph-mon[32752]: pgmap v7917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:45 smithi167 ceph-mon[32751]: pgmap v7917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13527.185435 secs 2024-04-26T22:20:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13527.185435 secs 2024-04-26T22:20:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13527.185435 secs 2024-04-26T22:20:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:47 smithi090 ceph-mon[29180]: pgmap v7918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13527.931967 secs 2024-04-26T22:20:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:47 smithi119 ceph-mon[32752]: pgmap v7918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13527.931967 secs 2024-04-26T22:20:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:47 smithi167 ceph-mon[32751]: pgmap v7918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-26T22:20:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13527.931967 secs 2024-04-26T22:20:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:49 smithi090 ceph-mon[29180]: pgmap v7919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-26T22:20:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:49 smithi119 ceph-mon[32752]: pgmap v7919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-26T22:20:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:49 smithi167 ceph-mon[32751]: pgmap v7919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-26T22:20:51.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:51 smithi119 ceph-mon[32752]: pgmap v7920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:20:51.356 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13532.185516 secs 2024-04-26T22:20:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:51 smithi090 ceph-mon[29180]: pgmap v7920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:20:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13532.185516 secs 2024-04-26T22:20:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:51 smithi167 ceph-mon[32751]: pgmap v7920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:20:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13532.185516 secs 2024-04-26T22:20:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13532.932109 secs 2024-04-26T22:20:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13532.932109 secs 2024-04-26T22:20:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13532.932109 secs 2024-04-26T22:20:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:53 smithi090 ceph-mon[29180]: pgmap v7921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:53 smithi119 ceph-mon[32752]: pgmap v7921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:53 smithi167 ceph-mon[32751]: pgmap v7921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:55 smithi090 ceph-mon[29180]: pgmap v7922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:55 smithi119 ceph-mon[32752]: pgmap v7922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:55 smithi167 ceph-mon[32751]: pgmap v7922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:20:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13537.185601 secs 2024-04-26T22:20:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13537.185601 secs 2024-04-26T22:20:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13537.185601 secs 2024-04-26T22:20:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:57 smithi090 ceph-mon[29180]: pgmap v7923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13537.932261 secs 2024-04-26T22:20:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:57 smithi119 ceph-mon[32752]: pgmap v7923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13537.932261 secs 2024-04-26T22:20:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:57 smithi167 ceph-mon[32751]: pgmap v7923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13537.932261 secs 2024-04-26T22:20:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:20:59 smithi090 ceph-mon[29180]: pgmap v7924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:20:59 smithi119 ceph-mon[32752]: pgmap v7924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:20:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:20:59 smithi167 ceph-mon[32751]: pgmap v7924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:01 smithi090 ceph-mon[29180]: pgmap v7925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13542.185686 secs 2024-04-26T22:21:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:01 smithi119 ceph-mon[32752]: pgmap v7925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13542.185686 secs 2024-04-26T22:21:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:01 smithi167 ceph-mon[32751]: pgmap v7925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13542.185686 secs 2024-04-26T22:21:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13542.932403 secs 2024-04-26T22:21:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13542.932403 secs 2024-04-26T22:21:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13542.932403 secs 2024-04-26T22:21:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:03 smithi090 ceph-mon[29180]: pgmap v7926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:03 smithi119 ceph-mon[32752]: pgmap v7926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:03 smithi167 ceph-mon[32751]: pgmap v7926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:05 smithi090 ceph-mon[29180]: pgmap v7927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:05 smithi119 ceph-mon[32752]: pgmap v7927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:05 smithi167 ceph-mon[32751]: pgmap v7927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13547.185769 secs 2024-04-26T22:21:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13547.185769 secs 2024-04-26T22:21:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13547.185769 secs 2024-04-26T22:21:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:07 smithi090 ceph-mon[29180]: pgmap v7928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13547.932462 secs 2024-04-26T22:21:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:07 smithi167 ceph-mon[32751]: pgmap v7928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13547.932462 secs 2024-04-26T22:21:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:07 smithi119 ceph-mon[32752]: pgmap v7928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13547.932462 secs 2024-04-26T22:21:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:09 smithi119 ceph-mon[32752]: pgmap v7929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:09 smithi167 ceph-mon[32751]: pgmap v7929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:09 smithi090 ceph-mon[29180]: pgmap v7929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:11 smithi119 ceph-mon[32752]: pgmap v7930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13552.185855 secs 2024-04-26T22:21:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:11 smithi167 ceph-mon[32751]: pgmap v7930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13552.185855 secs 2024-04-26T22:21:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:11 smithi090 ceph-mon[29180]: pgmap v7930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13552.185855 secs 2024-04-26T22:21:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13552.932582 secs 2024-04-26T22:21:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13552.932582 secs 2024-04-26T22:21:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13552.932582 secs 2024-04-26T22:21:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:13 smithi119 ceph-mon[32752]: pgmap v7931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:13 smithi167 ceph-mon[32751]: pgmap v7931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:13 smithi090 ceph-mon[29180]: pgmap v7931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:15 smithi119 ceph-mon[32752]: pgmap v7932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:15 smithi167 ceph-mon[32751]: pgmap v7932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:15 smithi090 ceph-mon[29180]: pgmap v7932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13557.185915 secs 2024-04-26T22:21:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13557.185915 secs 2024-04-26T22:21:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13557.185915 secs 2024-04-26T22:21:17.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:17 smithi090 ceph-mon[29180]: pgmap v7933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:17.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13557.932643 secs 2024-04-26T22:21:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:17 smithi119 ceph-mon[32752]: pgmap v7933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13557.932643 secs 2024-04-26T22:21:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:17 smithi167 ceph-mon[32751]: pgmap v7933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13557.932643 secs 2024-04-26T22:21:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:19 smithi119 ceph-mon[32752]: pgmap v7934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:19 smithi167 ceph-mon[32751]: pgmap v7934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:19 smithi090 ceph-mon[29180]: pgmap v7934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:21 smithi119 ceph-mon[32752]: pgmap v7935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13562.185991 secs 2024-04-26T22:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:21 smithi167 ceph-mon[32751]: pgmap v7935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13562.185991 secs 2024-04-26T22:21:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:21 smithi090 ceph-mon[29180]: pgmap v7935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13562.185991 secs 2024-04-26T22:21:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13562.932773 secs 2024-04-26T22:21:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13562.932773 secs 2024-04-26T22:21:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13562.932773 secs 2024-04-26T22:21:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:23 smithi119 ceph-mon[32752]: pgmap v7936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:23 smithi167 ceph-mon[32751]: pgmap v7936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:23 smithi090 ceph-mon[29180]: pgmap v7936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:25 smithi119 ceph-mon[32752]: pgmap v7937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:25 smithi167 ceph-mon[32751]: pgmap v7937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:25 smithi090 ceph-mon[29180]: pgmap v7937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13567.186053 secs 2024-04-26T22:21:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13567.186053 secs 2024-04-26T22:21:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13567.186053 secs 2024-04-26T22:21:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:27 smithi119 ceph-mon[32752]: pgmap v7938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13567.932862 secs 2024-04-26T22:21:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:27 smithi167 ceph-mon[32751]: pgmap v7938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13567.932862 secs 2024-04-26T22:21:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:27 smithi090 ceph-mon[29180]: pgmap v7938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13567.932862 secs 2024-04-26T22:21:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:29 smithi119 ceph-mon[32752]: pgmap v7939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:29 smithi167 ceph-mon[32751]: pgmap v7939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:29 smithi090 ceph-mon[29180]: pgmap v7939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:31 smithi119 ceph-mon[32752]: pgmap v7940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13572.186110 secs 2024-04-26T22:21:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:31 smithi167 ceph-mon[32751]: pgmap v7940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13572.186110 secs 2024-04-26T22:21:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:31 smithi090 ceph-mon[29180]: pgmap v7940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13572.186110 secs 2024-04-26T22:21:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13572.932975 secs 2024-04-26T22:21:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13572.932975 secs 2024-04-26T22:21:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13572.932975 secs 2024-04-26T22:21:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:33 smithi119 ceph-mon[32752]: pgmap v7941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:33 smithi167 ceph-mon[32751]: pgmap v7941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:33 smithi090 ceph-mon[29180]: pgmap v7941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:35 smithi119 ceph-mon[32752]: pgmap v7942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:35 smithi167 ceph-mon[32751]: pgmap v7942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:35 smithi090 ceph-mon[29180]: pgmap v7942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:36 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13577.186172 secs 2024-04-26T22:21:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:36 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13577.186172 secs 2024-04-26T22:21:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:36 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13577.186172 secs 2024-04-26T22:21:37.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:37 smithi167 ceph-mon[32751]: pgmap v7943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:37.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13577.933098 secs 2024-04-26T22:21:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:37 smithi119 ceph-mon[32752]: pgmap v7943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13577.933098 secs 2024-04-26T22:21:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:37 smithi090 ceph-mon[29180]: pgmap v7943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13577.933098 secs 2024-04-26T22:21:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:21:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:21:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:21:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:39 smithi119 ceph-mon[32752]: pgmap v7944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:21:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:21:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:21:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:39 smithi167 ceph-mon[32751]: pgmap v7944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:21:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:21:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:21:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:39 smithi090 ceph-mon[29180]: pgmap v7944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:21:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:21:39.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:21:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:40 smithi119 ceph-mon[32752]: pgmap v7945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:21:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:40 smithi119 ceph-mon[32752]: pgmap v7946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:40 smithi167 ceph-mon[32751]: pgmap v7945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:21:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:40 smithi167 ceph-mon[32751]: pgmap v7946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:40 smithi090 ceph-mon[29180]: pgmap v7945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:21:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:40 smithi090 ceph-mon[29180]: pgmap v7946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13582.186260 secs 2024-04-26T22:21:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13582.186260 secs 2024-04-26T22:21:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13582.186260 secs 2024-04-26T22:21:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:42 smithi119 ceph-mon[32752]: pgmap v7947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13582.933230 secs 2024-04-26T22:21:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:42 smithi167 ceph-mon[32751]: pgmap v7947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13582.933230 secs 2024-04-26T22:21:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:42 smithi090 ceph-mon[29180]: pgmap v7947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13582.933230 secs 2024-04-26T22:21:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:44 smithi119 ceph-mon[32752]: pgmap v7948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:21:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:44 smithi167 ceph-mon[32751]: pgmap v7948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:21:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:44 smithi090 ceph-mon[29180]: pgmap v7948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:21:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:46 smithi119 ceph-mon[32752]: pgmap v7949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13587.186319 secs 2024-04-26T22:21:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:46 smithi167 ceph-mon[32751]: pgmap v7949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13587.186319 secs 2024-04-26T22:21:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:46 smithi090 ceph-mon[29180]: pgmap v7949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13587.186319 secs 2024-04-26T22:21:47.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13587.933345 secs 2024-04-26T22:21:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13587.933345 secs 2024-04-26T22:21:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13587.933345 secs 2024-04-26T22:21:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:48 smithi119 ceph-mon[32752]: pgmap v7950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:48 smithi167 ceph-mon[32751]: pgmap v7950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:48 smithi090 ceph-mon[29180]: pgmap v7950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:21:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:50 smithi119 ceph-mon[32752]: pgmap v7951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:21:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:50 smithi167 ceph-mon[32751]: pgmap v7951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:21:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:50 smithi090 ceph-mon[29180]: pgmap v7951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:21:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13592.186406 secs 2024-04-26T22:21:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13592.186406 secs 2024-04-26T22:21:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13592.186406 secs 2024-04-26T22:21:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:52 smithi119 ceph-mon[32752]: pgmap v7952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13592.933467 secs 2024-04-26T22:21:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:52 smithi167 ceph-mon[32751]: pgmap v7952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13592.933467 secs 2024-04-26T22:21:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:52 smithi090 ceph-mon[29180]: pgmap v7952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13592.933467 secs 2024-04-26T22:21:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:54 smithi119 ceph-mon[32752]: pgmap v7953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:54 smithi167 ceph-mon[32751]: pgmap v7953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:54 smithi090 ceph-mon[29180]: pgmap v7953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:56 smithi119 ceph-mon[32752]: pgmap v7954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13597.186490 secs 2024-04-26T22:21:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:56 smithi167 ceph-mon[32751]: pgmap v7954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13597.186490 secs 2024-04-26T22:21:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:56 smithi090 ceph-mon[29180]: pgmap v7954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:21:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13597.186490 secs 2024-04-26T22:21:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13597.933527 secs 2024-04-26T22:21:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13597.933527 secs 2024-04-26T22:21:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13597.933527 secs 2024-04-26T22:21:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:21:58 smithi119 ceph-mon[32752]: pgmap v7955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:21:58 smithi167 ceph-mon[32751]: pgmap v7955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:21:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:21:58 smithi090 ceph-mon[29180]: pgmap v7955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:00 smithi119 ceph-mon[32752]: pgmap v7956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:00 smithi167 ceph-mon[32751]: pgmap v7956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:00 smithi090 ceph-mon[29180]: pgmap v7956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13602.186576 secs 2024-04-26T22:22:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13602.186576 secs 2024-04-26T22:22:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13602.186576 secs 2024-04-26T22:22:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:02 smithi119 ceph-mon[32752]: pgmap v7957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13602.933595 secs 2024-04-26T22:22:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:02 smithi167 ceph-mon[32751]: pgmap v7957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13602.933595 secs 2024-04-26T22:22:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:02 smithi090 ceph-mon[29180]: pgmap v7957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13602.933595 secs 2024-04-26T22:22:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:04 smithi119 ceph-mon[32752]: pgmap v7958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:04 smithi167 ceph-mon[32751]: pgmap v7958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:04 smithi090 ceph-mon[29180]: pgmap v7958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:06 smithi119 ceph-mon[32752]: pgmap v7959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13607.186661 secs 2024-04-26T22:22:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:06 smithi167 ceph-mon[32751]: pgmap v7959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13607.186661 secs 2024-04-26T22:22:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:06 smithi090 ceph-mon[29180]: pgmap v7959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13607.186661 secs 2024-04-26T22:22:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13607.933746 secs 2024-04-26T22:22:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13607.933746 secs 2024-04-26T22:22:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13607.933746 secs 2024-04-26T22:22:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:08 smithi119 ceph-mon[32752]: pgmap v7960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:08 smithi167 ceph-mon[32751]: pgmap v7960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:08 smithi090 ceph-mon[29180]: pgmap v7960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:10 smithi119 ceph-mon[32752]: pgmap v7961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:10 smithi167 ceph-mon[32751]: pgmap v7961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:10 smithi090 ceph-mon[29180]: pgmap v7961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13612.186744 secs 2024-04-26T22:22:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13612.186744 secs 2024-04-26T22:22:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13612.186744 secs 2024-04-26T22:22:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:12 smithi119 ceph-mon[32752]: pgmap v7962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13612.933865 secs 2024-04-26T22:22:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:12 smithi167 ceph-mon[32751]: pgmap v7962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13612.933865 secs 2024-04-26T22:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:12 smithi090 ceph-mon[29180]: pgmap v7962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13612.933865 secs 2024-04-26T22:22:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:13 smithi167 ceph-mon[32751]: pgmap v7963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:13 smithi090 ceph-mon[29180]: pgmap v7963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:13 smithi119 ceph-mon[32752]: pgmap v7963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:15 smithi119 ceph-mon[32752]: pgmap v7964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:15 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13617.186847 secs 2024-04-26T22:22:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:15 smithi167 ceph-mon[32751]: pgmap v7964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:15 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13617.186847 secs 2024-04-26T22:22:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:15 smithi090 ceph-mon[29180]: pgmap v7964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:15 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13617.186847 secs 2024-04-26T22:22:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13617.933983 secs 2024-04-26T22:22:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13617.933983 secs 2024-04-26T22:22:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13617.933983 secs 2024-04-26T22:22:17.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:17 smithi090 ceph-mon[29180]: pgmap v7965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:17 smithi119 ceph-mon[32752]: pgmap v7965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:17 smithi167 ceph-mon[32751]: pgmap v7965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:19 smithi119 ceph-mon[32752]: pgmap v7966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:19 smithi167 ceph-mon[32751]: pgmap v7966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:19 smithi090 ceph-mon[29180]: pgmap v7966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13622.186932 secs 2024-04-26T22:22:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13622.186932 secs 2024-04-26T22:22:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13622.186932 secs 2024-04-26T22:22:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:21 smithi119 ceph-mon[32752]: pgmap v7967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13622.934142 secs 2024-04-26T22:22:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:21 smithi167 ceph-mon[32751]: pgmap v7967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13622.934142 secs 2024-04-26T22:22:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:21 smithi090 ceph-mon[29180]: pgmap v7967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13622.934142 secs 2024-04-26T22:22:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:23 smithi119 ceph-mon[32752]: pgmap v7968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:23 smithi167 ceph-mon[32751]: pgmap v7968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:23 smithi090 ceph-mon[29180]: pgmap v7968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:25 smithi119 ceph-mon[32752]: pgmap v7969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:25 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13627.187021 secs 2024-04-26T22:22:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:25 smithi167 ceph-mon[32751]: pgmap v7969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:25 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13627.187021 secs 2024-04-26T22:22:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:25 smithi090 ceph-mon[29180]: pgmap v7969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:25 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13627.187021 secs 2024-04-26T22:22:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13627.934245 secs 2024-04-26T22:22:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13627.934245 secs 2024-04-26T22:22:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13627.934245 secs 2024-04-26T22:22:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:27 smithi119 ceph-mon[32752]: pgmap v7970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:27 smithi167 ceph-mon[32751]: pgmap v7970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:27 smithi090 ceph-mon[29180]: pgmap v7970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:29 smithi119 ceph-mon[32752]: pgmap v7971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:29 smithi167 ceph-mon[32751]: pgmap v7971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:29 smithi090 ceph-mon[29180]: pgmap v7971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13632.187105 secs 2024-04-26T22:22:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13632.187105 secs 2024-04-26T22:22:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13632.187105 secs 2024-04-26T22:22:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:31 smithi119 ceph-mon[32752]: pgmap v7972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13632.934372 secs 2024-04-26T22:22:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:31 smithi167 ceph-mon[32751]: pgmap v7972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13632.934372 secs 2024-04-26T22:22:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:31 smithi090 ceph-mon[29180]: pgmap v7972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13632.934372 secs 2024-04-26T22:22:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:33 smithi119 ceph-mon[32752]: pgmap v7973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:33 smithi167 ceph-mon[32751]: pgmap v7973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:33 smithi090 ceph-mon[29180]: pgmap v7973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:35 smithi119 ceph-mon[32752]: pgmap v7974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:35 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13637.187185 secs 2024-04-26T22:22:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:35 smithi167 ceph-mon[32751]: pgmap v7974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:35 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13637.187185 secs 2024-04-26T22:22:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:35 smithi090 ceph-mon[29180]: pgmap v7974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:35 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13637.187185 secs 2024-04-26T22:22:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13637.934486 secs 2024-04-26T22:22:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13637.934486 secs 2024-04-26T22:22:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13637.934486 secs 2024-04-26T22:22:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:37 smithi119 ceph-mon[32752]: pgmap v7975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:37 smithi167 ceph-mon[32751]: pgmap v7975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:37 smithi090 ceph-mon[29180]: pgmap v7975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:22:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:22:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:22:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:39 smithi167 ceph-mon[32751]: pgmap v7976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:22:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:22:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:39 smithi167 ceph-mon[32751]: pgmap v7977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:22:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:39 smithi090 ceph-mon[29180]: pgmap v7976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:22:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:22:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:39 smithi090 ceph-mon[29180]: pgmap v7977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:22:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:39 smithi119 ceph-mon[32752]: pgmap v7976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:22:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:22:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:39 smithi119 ceph-mon[32752]: pgmap v7977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:22:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13642.187258 secs 2024-04-26T22:22:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13642.187258 secs 2024-04-26T22:22:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13642.187258 secs 2024-04-26T22:22:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13642.934562 secs 2024-04-26T22:22:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:41 smithi167 ceph-mon[32751]: pgmap v7978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13642.934562 secs 2024-04-26T22:22:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:41 smithi090 ceph-mon[29180]: pgmap v7978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13642.934562 secs 2024-04-26T22:22:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:41 smithi119 ceph-mon[32752]: pgmap v7978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:44 smithi119 ceph-mon[32752]: pgmap v7979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:44 smithi167 ceph-mon[32751]: pgmap v7979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:44 smithi090 ceph-mon[29180]: pgmap v7979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13647.187342 secs 2024-04-26T22:22:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:46 smithi119 ceph-mon[32752]: pgmap v7980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13647.187342 secs 2024-04-26T22:22:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:46 smithi167 ceph-mon[32751]: pgmap v7980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13647.187342 secs 2024-04-26T22:22:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:46 smithi090 ceph-mon[29180]: pgmap v7980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:47.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13647.934679 secs 2024-04-26T22:22:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13647.934679 secs 2024-04-26T22:22:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13647.934679 secs 2024-04-26T22:22:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:48 smithi119 ceph-mon[32752]: pgmap v7981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:48 smithi167 ceph-mon[32751]: pgmap v7981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:48 smithi090 ceph-mon[29180]: pgmap v7981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:50 smithi119 ceph-mon[32752]: pgmap v7982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:50 smithi167 ceph-mon[32751]: pgmap v7982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:50 smithi090 ceph-mon[29180]: pgmap v7982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:22:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13652.187427 secs 2024-04-26T22:22:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13652.187427 secs 2024-04-26T22:22:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13652.187427 secs 2024-04-26T22:22:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13652.934799 secs 2024-04-26T22:22:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:52 smithi119 ceph-mon[32752]: pgmap v7983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13652.934799 secs 2024-04-26T22:22:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:52 smithi167 ceph-mon[32751]: pgmap v7983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13652.934799 secs 2024-04-26T22:22:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:52 smithi090 ceph-mon[29180]: pgmap v7983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:54 smithi119 ceph-mon[32752]: pgmap v7984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:54 smithi167 ceph-mon[32751]: pgmap v7984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:54 smithi090 ceph-mon[29180]: pgmap v7984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13657.187512 secs 2024-04-26T22:22:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:56 smithi119 ceph-mon[32752]: pgmap v7985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13657.187512 secs 2024-04-26T22:22:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:56 smithi167 ceph-mon[32751]: pgmap v7985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13657.187512 secs 2024-04-26T22:22:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:56 smithi090 ceph-mon[29180]: pgmap v7985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:22:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13657.934881 secs 2024-04-26T22:22:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13657.934881 secs 2024-04-26T22:22:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13657.934881 secs 2024-04-26T22:22:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:22:58 smithi119 ceph-mon[32752]: pgmap v7986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:22:58 smithi167 ceph-mon[32751]: pgmap v7986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:22:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:22:58 smithi090 ceph-mon[29180]: pgmap v7986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:00 smithi119 ceph-mon[32752]: pgmap v7987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:00 smithi167 ceph-mon[32751]: pgmap v7987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:00 smithi090 ceph-mon[29180]: pgmap v7987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13662.187596 secs 2024-04-26T22:23:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13662.187596 secs 2024-04-26T22:23:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13662.187596 secs 2024-04-26T22:23:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13662.935004 secs 2024-04-26T22:23:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:02 smithi119 ceph-mon[32752]: pgmap v7988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13662.935004 secs 2024-04-26T22:23:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:02 smithi167 ceph-mon[32751]: pgmap v7988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13662.935004 secs 2024-04-26T22:23:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:02 smithi090 ceph-mon[29180]: pgmap v7988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:04 smithi119 ceph-mon[32752]: pgmap v7989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:04 smithi167 ceph-mon[32751]: pgmap v7989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:04 smithi090 ceph-mon[29180]: pgmap v7989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:05 smithi090 ceph-mon[29180]: pgmap v7990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:05 smithi119 ceph-mon[32752]: pgmap v7990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:05 smithi167 ceph-mon[32751]: pgmap v7990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:06 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13667.187682 secs 2024-04-26T22:23:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:06 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13667.187682 secs 2024-04-26T22:23:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:06 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13667.187682 secs 2024-04-26T22:23:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13667.935075 secs 2024-04-26T22:23:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:07 smithi090 ceph-mon[29180]: pgmap v7991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:07.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13667.935075 secs 2024-04-26T22:23:07.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:07 smithi167 ceph-mon[32751]: pgmap v7991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13667.935075 secs 2024-04-26T22:23:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:07 smithi119 ceph-mon[32752]: pgmap v7991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:10 smithi119 ceph-mon[32752]: pgmap v7992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:10 smithi167 ceph-mon[32751]: pgmap v7992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:10 smithi090 ceph-mon[29180]: pgmap v7992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:11 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13672.187766 secs 2024-04-26T22:23:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:11 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13672.187766 secs 2024-04-26T22:23:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:11 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13672.187766 secs 2024-04-26T22:23:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13672.935192 secs 2024-04-26T22:23:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:12 smithi119 ceph-mon[32752]: pgmap v7993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13672.935192 secs 2024-04-26T22:23:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:12 smithi167 ceph-mon[32751]: pgmap v7993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13672.935192 secs 2024-04-26T22:23:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:12 smithi090 ceph-mon[29180]: pgmap v7993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:14 smithi119 ceph-mon[32752]: pgmap v7994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:14 smithi167 ceph-mon[32751]: pgmap v7994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:14 smithi090 ceph-mon[29180]: pgmap v7994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13677.187828 secs 2024-04-26T22:23:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:16 smithi119 ceph-mon[32752]: pgmap v7995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13677.187828 secs 2024-04-26T22:23:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:16 smithi167 ceph-mon[32751]: pgmap v7995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13677.187828 secs 2024-04-26T22:23:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:16 smithi090 ceph-mon[29180]: pgmap v7995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13677.935331 secs 2024-04-26T22:23:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13677.935331 secs 2024-04-26T22:23:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13677.935331 secs 2024-04-26T22:23:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:18 smithi119 ceph-mon[32752]: pgmap v7996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:18 smithi167 ceph-mon[32751]: pgmap v7996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:18 smithi090 ceph-mon[29180]: pgmap v7996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:20 smithi119 ceph-mon[32752]: pgmap v7997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:20 smithi167 ceph-mon[32751]: pgmap v7997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:20 smithi090 ceph-mon[29180]: pgmap v7997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:21 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13682.187913 secs 2024-04-26T22:23:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:21 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13682.187913 secs 2024-04-26T22:23:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:21 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13682.187913 secs 2024-04-26T22:23:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13682.935462 secs 2024-04-26T22:23:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:22 smithi119 ceph-mon[32752]: pgmap v7998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13682.935462 secs 2024-04-26T22:23:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:22 smithi167 ceph-mon[32751]: pgmap v7998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13682.935462 secs 2024-04-26T22:23:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:22 smithi090 ceph-mon[29180]: pgmap v7998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:23 smithi167 ceph-mon[32751]: pgmap v7999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:23 smithi090 ceph-mon[29180]: pgmap v7999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:23 smithi119 ceph-mon[32752]: pgmap v7999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:26 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13687.188002 secs 2024-04-26T22:23:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:26 smithi119 ceph-mon[32752]: pgmap v8000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:26 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13687.188002 secs 2024-04-26T22:23:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:26 smithi167 ceph-mon[32751]: pgmap v8000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:26 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13687.188002 secs 2024-04-26T22:23:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:26 smithi090 ceph-mon[29180]: pgmap v8000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13687.935579 secs 2024-04-26T22:23:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13687.935579 secs 2024-04-26T22:23:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13687.935579 secs 2024-04-26T22:23:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:28 smithi119 ceph-mon[32752]: pgmap v8001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:28 smithi167 ceph-mon[32751]: pgmap v8001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:28 smithi090 ceph-mon[29180]: pgmap v8001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:30 smithi119 ceph-mon[32752]: pgmap v8002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:30 smithi167 ceph-mon[32751]: pgmap v8002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:30 smithi090 ceph-mon[29180]: pgmap v8002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 13692.188065 secs 2024-04-26T22:23:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 13692.188065 secs 2024-04-26T22:23:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 13692.188065 secs 2024-04-26T22:23:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13692.935707 secs 2024-04-26T22:23:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:32 smithi119 ceph-mon[32752]: pgmap v8003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13692.935707 secs 2024-04-26T22:23:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:32 smithi167 ceph-mon[32751]: pgmap v8003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13692.935707 secs 2024-04-26T22:23:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:32 smithi090 ceph-mon[29180]: pgmap v8003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:34 smithi119 ceph-mon[32752]: pgmap v8004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:34 smithi167 ceph-mon[32751]: pgmap v8004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:34 smithi090 ceph-mon[29180]: pgmap v8004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:36 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 13697.188148 secs 2024-04-26T22:23:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:36 smithi119 ceph-mon[32752]: slow request 180.522283 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:23:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:36 smithi119 ceph-mon[32752]: pgmap v8005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:36 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 13697.188148 secs 2024-04-26T22:23:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:36 smithi167 ceph-mon[32751]: slow request 180.522283 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:23:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:36 smithi167 ceph-mon[32751]: pgmap v8005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:36 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 13697.188148 secs 2024-04-26T22:23:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:36 smithi090 ceph-mon[29180]: slow request 180.522283 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:23:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:36 smithi090 ceph-mon[29180]: pgmap v8005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13697.935804 secs 2024-04-26T22:23:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13697.935804 secs 2024-04-26T22:23:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13697.935804 secs 2024-04-26T22:23:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:38 smithi119 ceph-mon[32752]: pgmap v8006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:38 smithi167 ceph-mon[32751]: pgmap v8006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:38 smithi090 ceph-mon[29180]: pgmap v8006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:23:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:23:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:23:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:23:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:23:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:23:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:40 smithi167 ceph-mon[32751]: pgmap v8007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:23:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:23:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:23:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:40 smithi167 ceph-mon[32751]: pgmap v8008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:23:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:40 smithi167 ceph-mon[32751]: pgmap v8009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:23:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:40 smithi090 ceph-mon[29180]: pgmap v8007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:23:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:23:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:23:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:40 smithi090 ceph-mon[29180]: pgmap v8008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:23:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:40 smithi090 ceph-mon[29180]: pgmap v8009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:23:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:40 smithi119 ceph-mon[32752]: pgmap v8007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:23:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:23:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:23:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:40 smithi119 ceph-mon[32752]: pgmap v8008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:23:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:40 smithi119 ceph-mon[32752]: pgmap v8009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:23:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13702.188210 secs 2024-04-26T22:23:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13702.188210 secs 2024-04-26T22:23:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13702.188210 secs 2024-04-26T22:23:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13702.935927 secs 2024-04-26T22:23:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:42 smithi090 ceph-mon[29180]: pgmap v8010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13702.935927 secs 2024-04-26T22:23:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:42 smithi119 ceph-mon[32752]: pgmap v8010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13702.935927 secs 2024-04-26T22:23:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:42 smithi167 ceph-mon[32751]: pgmap v8010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:45 smithi119 ceph-mon[32752]: pgmap v8011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:45 smithi167 ceph-mon[32751]: pgmap v8011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:45 smithi090 ceph-mon[29180]: pgmap v8011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13707.188294 secs 2024-04-26T22:23:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13707.188294 secs 2024-04-26T22:23:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13707.188294 secs 2024-04-26T22:23:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13707.936045 secs 2024-04-26T22:23:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:47 smithi119 ceph-mon[32752]: pgmap v8012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13707.936045 secs 2024-04-26T22:23:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:47 smithi167 ceph-mon[32751]: pgmap v8012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13707.936045 secs 2024-04-26T22:23:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:47 smithi090 ceph-mon[29180]: pgmap v8012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:23:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:49 smithi119 ceph-mon[32752]: pgmap v8013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:23:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:49 smithi167 ceph-mon[32751]: pgmap v8013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:23:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:49 smithi090 ceph-mon[29180]: pgmap v8013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:23:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13712.188380 secs 2024-04-26T22:23:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:51 smithi119 ceph-mon[32752]: pgmap v8014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:23:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13712.188380 secs 2024-04-26T22:23:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:51 smithi167 ceph-mon[32751]: pgmap v8014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:23:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13712.188380 secs 2024-04-26T22:23:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:51 smithi090 ceph-mon[29180]: pgmap v8014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:23:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13712.936168 secs 2024-04-26T22:23:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13712.936168 secs 2024-04-26T22:23:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13712.936168 secs 2024-04-26T22:23:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:53 smithi167 ceph-mon[32751]: pgmap v8015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:53 smithi090 ceph-mon[29180]: pgmap v8015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:53 smithi119 ceph-mon[32752]: pgmap v8015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:55 smithi167 ceph-mon[32751]: pgmap v8016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:55 smithi090 ceph-mon[29180]: pgmap v8016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:55 smithi119 ceph-mon[32752]: pgmap v8016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:23:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13717.188466 secs 2024-04-26T22:23:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13717.188466 secs 2024-04-26T22:23:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13717.188466 secs 2024-04-26T22:23:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13717.936286 secs 2024-04-26T22:23:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:57 smithi090 ceph-mon[29180]: pgmap v8017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13717.936286 secs 2024-04-26T22:23:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:57 smithi119 ceph-mon[32752]: pgmap v8017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13717.936286 secs 2024-04-26T22:23:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:57 smithi167 ceph-mon[32751]: pgmap v8017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:23:59 smithi090 ceph-mon[29180]: pgmap v8018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:23:59 smithi119 ceph-mon[32752]: pgmap v8018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:23:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:23:59 smithi167 ceph-mon[32751]: pgmap v8018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13722.188549 secs 2024-04-26T22:24:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:01 smithi090 ceph-mon[29180]: pgmap v8019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13722.188549 secs 2024-04-26T22:24:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:01 smithi119 ceph-mon[32752]: pgmap v8019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13722.188549 secs 2024-04-26T22:24:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:01 smithi167 ceph-mon[32751]: pgmap v8019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13722.936430 secs 2024-04-26T22:24:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13722.936430 secs 2024-04-26T22:24:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13722.936430 secs 2024-04-26T22:24:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:03 smithi090 ceph-mon[29180]: pgmap v8020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:03 smithi119 ceph-mon[32752]: pgmap v8020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:03 smithi167 ceph-mon[32751]: pgmap v8020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:05 smithi090 ceph-mon[29180]: pgmap v8021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:05 smithi119 ceph-mon[32752]: pgmap v8021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:05 smithi167 ceph-mon[32751]: pgmap v8021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13727.188610 secs 2024-04-26T22:24:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13727.188610 secs 2024-04-26T22:24:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13727.188610 secs 2024-04-26T22:24:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13727.936471 secs 2024-04-26T22:24:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:07 smithi090 ceph-mon[29180]: pgmap v8022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13727.936471 secs 2024-04-26T22:24:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:07 smithi167 ceph-mon[32751]: pgmap v8022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13727.936471 secs 2024-04-26T22:24:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:07 smithi119 ceph-mon[32752]: pgmap v8022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:09 smithi090 ceph-mon[29180]: pgmap v8023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:09 smithi119 ceph-mon[32752]: pgmap v8023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:09 smithi167 ceph-mon[32751]: pgmap v8023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13732.188693 secs 2024-04-26T22:24:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:11 smithi090 ceph-mon[29180]: pgmap v8024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13732.188693 secs 2024-04-26T22:24:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:11 smithi119 ceph-mon[32752]: pgmap v8024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13732.188693 secs 2024-04-26T22:24:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:11 smithi167 ceph-mon[32751]: pgmap v8024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13732.936591 secs 2024-04-26T22:24:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13732.936591 secs 2024-04-26T22:24:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13732.936591 secs 2024-04-26T22:24:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:13 smithi090 ceph-mon[29180]: pgmap v8025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:13 smithi119 ceph-mon[32752]: pgmap v8025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:13 smithi167 ceph-mon[32751]: pgmap v8025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:15 smithi090 ceph-mon[29180]: pgmap v8026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:15 smithi119 ceph-mon[32752]: pgmap v8026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:15 smithi167 ceph-mon[32751]: pgmap v8026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13737.188756 secs 2024-04-26T22:24:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13737.188756 secs 2024-04-26T22:24:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13737.188756 secs 2024-04-26T22:24:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13737.936719 secs 2024-04-26T22:24:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:17 smithi090 ceph-mon[29180]: pgmap v8027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13737.936719 secs 2024-04-26T22:24:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:17 smithi119 ceph-mon[32752]: pgmap v8027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13737.936719 secs 2024-04-26T22:24:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:17 smithi167 ceph-mon[32751]: pgmap v8027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:19 smithi090 ceph-mon[29180]: pgmap v8028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:19 smithi119 ceph-mon[32752]: pgmap v8028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:19 smithi167 ceph-mon[32751]: pgmap v8028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13742.188848 secs 2024-04-26T22:24:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:21 smithi090 ceph-mon[29180]: pgmap v8029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13742.188848 secs 2024-04-26T22:24:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:21 smithi119 ceph-mon[32752]: pgmap v8029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13742.188848 secs 2024-04-26T22:24:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:21 smithi167 ceph-mon[32751]: pgmap v8029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13742.936835 secs 2024-04-26T22:24:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13742.936835 secs 2024-04-26T22:24:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13742.936835 secs 2024-04-26T22:24:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:23 smithi119 ceph-mon[32752]: pgmap v8030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:23 smithi167 ceph-mon[32751]: pgmap v8030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:23 smithi090 ceph-mon[29180]: pgmap v8030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:25 smithi119 ceph-mon[32752]: pgmap v8031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:25 smithi167 ceph-mon[32751]: pgmap v8031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:25 smithi090 ceph-mon[29180]: pgmap v8031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:26 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 13747.188887 secs 2024-04-26T22:24:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:26 smithi119 ceph-mon[32752]: slow request 722.154993 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:24:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:26 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 13747.188887 secs 2024-04-26T22:24:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:26 smithi167 ceph-mon[32751]: slow request 722.154993 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:24:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:26 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 13747.188887 secs 2024-04-26T22:24:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:26 smithi090 ceph-mon[29180]: slow request 722.154993 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:24:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13747.936917 secs 2024-04-26T22:24:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:27 smithi119 ceph-mon[32752]: pgmap v8032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13747.936917 secs 2024-04-26T22:24:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:27 smithi167 ceph-mon[32751]: pgmap v8032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13747.936917 secs 2024-04-26T22:24:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:27 smithi090 ceph-mon[29180]: pgmap v8032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:29 smithi119 ceph-mon[32752]: pgmap v8033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:29 smithi167 ceph-mon[32751]: pgmap v8033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:29 smithi090 ceph-mon[29180]: pgmap v8033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13752.188976 secs 2024-04-26T22:24:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:31 smithi119 ceph-mon[32752]: pgmap v8034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13752.188976 secs 2024-04-26T22:24:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:31 smithi167 ceph-mon[32751]: pgmap v8034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13752.188976 secs 2024-04-26T22:24:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:31 smithi090 ceph-mon[29180]: pgmap v8034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13752.937032 secs 2024-04-26T22:24:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13752.937032 secs 2024-04-26T22:24:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13752.937032 secs 2024-04-26T22:24:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:33 smithi119 ceph-mon[32752]: pgmap v8035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:33 smithi167 ceph-mon[32751]: pgmap v8035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:33 smithi090 ceph-mon[29180]: pgmap v8035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:35 smithi119 ceph-mon[32752]: pgmap v8036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:35 smithi167 ceph-mon[32751]: pgmap v8036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:35 smithi090 ceph-mon[29180]: pgmap v8036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13757.189082 secs 2024-04-26T22:24:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13757.189082 secs 2024-04-26T22:24:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13757.189082 secs 2024-04-26T22:24:37.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13757.937148 secs 2024-04-26T22:24:37.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:37 smithi167 ceph-mon[32751]: pgmap v8037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13757.937148 secs 2024-04-26T22:24:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:37 smithi119 ceph-mon[32752]: pgmap v8037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13757.937148 secs 2024-04-26T22:24:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:37 smithi090 ceph-mon[29180]: pgmap v8037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:39 smithi119 ceph-mon[32752]: pgmap v8038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:39 smithi167 ceph-mon[32751]: pgmap v8038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:39 smithi090 ceph-mon[29180]: pgmap v8038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:24:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:24:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:24:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13762.189144 secs 2024-04-26T22:24:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:41 smithi119 ceph-mon[32752]: pgmap v8039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:24:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:24:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:24:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13762.189144 secs 2024-04-26T22:24:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:41 smithi167 ceph-mon[32751]: pgmap v8039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:24:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:24:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:24:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13762.189144 secs 2024-04-26T22:24:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:41 smithi090 ceph-mon[29180]: pgmap v8039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:24:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:24:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:24:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13762.937235 secs 2024-04-26T22:24:42.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:42 smithi119 ceph-mon[32752]: pgmap v8040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:24:42.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:42 smithi119 ceph-mon[32752]: pgmap v8041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13762.937235 secs 2024-04-26T22:24:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:42 smithi167 ceph-mon[32751]: pgmap v8040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:24:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:42 smithi167 ceph-mon[32751]: pgmap v8041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13762.937235 secs 2024-04-26T22:24:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:42 smithi090 ceph-mon[29180]: pgmap v8040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:24:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:42 smithi090 ceph-mon[29180]: pgmap v8041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:44.517 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:44 smithi090 ceph-mon[29180]: pgmap v8042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:24:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:44 smithi119 ceph-mon[32752]: pgmap v8042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:24:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:44 smithi167 ceph-mon[32751]: pgmap v8042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:24:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:46 smithi119 ceph-mon[32752]: pgmap v8043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13767.189230 secs 2024-04-26T22:24:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:46 smithi167 ceph-mon[32751]: pgmap v8043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13767.189230 secs 2024-04-26T22:24:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:46 smithi090 ceph-mon[29180]: pgmap v8043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13767.189230 secs 2024-04-26T22:24:47.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13767.937358 secs 2024-04-26T22:24:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13767.937358 secs 2024-04-26T22:24:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13767.937358 secs 2024-04-26T22:24:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:48 smithi119 ceph-mon[32752]: pgmap v8044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:48 smithi167 ceph-mon[32751]: pgmap v8044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:48 smithi090 ceph-mon[29180]: pgmap v8044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:50 smithi119 ceph-mon[32752]: pgmap v8045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:50 smithi167 ceph-mon[32751]: pgmap v8045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:50 smithi090 ceph-mon[29180]: pgmap v8045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:24:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13772.189312 secs 2024-04-26T22:24:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13772.189312 secs 2024-04-26T22:24:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13772.189312 secs 2024-04-26T22:24:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13772.937452 secs 2024-04-26T22:24:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:52 smithi119 ceph-mon[32752]: pgmap v8046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:24:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13772.937452 secs 2024-04-26T22:24:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:52 smithi167 ceph-mon[32751]: pgmap v8046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:24:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13772.937452 secs 2024-04-26T22:24:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:52 smithi090 ceph-mon[29180]: pgmap v8046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:24:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:54 smithi119 ceph-mon[32752]: pgmap v8047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:54 smithi167 ceph-mon[32751]: pgmap v8047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:54 smithi090 ceph-mon[29180]: pgmap v8047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:56 smithi119 ceph-mon[32752]: pgmap v8048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13777.189396 secs 2024-04-26T22:24:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:56 smithi167 ceph-mon[32751]: pgmap v8048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13777.189396 secs 2024-04-26T22:24:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:56 smithi090 ceph-mon[29180]: pgmap v8048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:24:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13777.189396 secs 2024-04-26T22:24:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13777.937569 secs 2024-04-26T22:24:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13777.937569 secs 2024-04-26T22:24:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13777.937569 secs 2024-04-26T22:24:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:24:58 smithi119 ceph-mon[32752]: pgmap v8049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:24:58 smithi167 ceph-mon[32751]: pgmap v8049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:24:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:24:58 smithi090 ceph-mon[29180]: pgmap v8049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:00 smithi119 ceph-mon[32752]: pgmap v8050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:00 smithi167 ceph-mon[32751]: pgmap v8050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:00 smithi090 ceph-mon[29180]: pgmap v8050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13782.189477 secs 2024-04-26T22:25:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13782.189477 secs 2024-04-26T22:25:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13782.189477 secs 2024-04-26T22:25:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13782.937693 secs 2024-04-26T22:25:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:02 smithi119 ceph-mon[32752]: pgmap v8051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13782.937693 secs 2024-04-26T22:25:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:02 smithi167 ceph-mon[32751]: pgmap v8051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13782.937693 secs 2024-04-26T22:25:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:02 smithi090 ceph-mon[29180]: pgmap v8051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:04 smithi119 ceph-mon[32752]: pgmap v8052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:04 smithi167 ceph-mon[32751]: pgmap v8052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:04 smithi090 ceph-mon[29180]: pgmap v8052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:06 smithi119 ceph-mon[32752]: pgmap v8053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13787.189560 secs 2024-04-26T22:25:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:06 smithi167 ceph-mon[32751]: pgmap v8053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13787.189560 secs 2024-04-26T22:25:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:06 smithi090 ceph-mon[29180]: pgmap v8053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13787.189560 secs 2024-04-26T22:25:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13787.937807 secs 2024-04-26T22:25:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13787.937807 secs 2024-04-26T22:25:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13787.937807 secs 2024-04-26T22:25:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:08 smithi119 ceph-mon[32752]: pgmap v8054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:08 smithi167 ceph-mon[32751]: pgmap v8054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:08 smithi090 ceph-mon[29180]: pgmap v8054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:10 smithi119 ceph-mon[32752]: pgmap v8055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:10 smithi167 ceph-mon[32751]: pgmap v8055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:10 smithi090 ceph-mon[29180]: pgmap v8055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13792.189646 secs 2024-04-26T22:25:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13792.189646 secs 2024-04-26T22:25:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13792.189646 secs 2024-04-26T22:25:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13792.937924 secs 2024-04-26T22:25:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:12 smithi119 ceph-mon[32752]: pgmap v8056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13792.937924 secs 2024-04-26T22:25:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:12 smithi167 ceph-mon[32751]: pgmap v8056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13792.937924 secs 2024-04-26T22:25:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:12 smithi090 ceph-mon[29180]: pgmap v8056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:14 smithi119 ceph-mon[32752]: pgmap v8057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:14 smithi167 ceph-mon[32751]: pgmap v8057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:14 smithi090 ceph-mon[29180]: pgmap v8057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:16 smithi119 ceph-mon[32752]: pgmap v8058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13797.189729 secs 2024-04-26T22:25:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:16 smithi167 ceph-mon[32751]: pgmap v8058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13797.189729 secs 2024-04-26T22:25:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:16 smithi090 ceph-mon[29180]: pgmap v8058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13797.189729 secs 2024-04-26T22:25:17.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13797.938022 secs 2024-04-26T22:25:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13797.938022 secs 2024-04-26T22:25:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13797.938022 secs 2024-04-26T22:25:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:18 smithi119 ceph-mon[32752]: pgmap v8059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:18 smithi167 ceph-mon[32751]: pgmap v8059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:18 smithi090 ceph-mon[29180]: pgmap v8059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:20 smithi119 ceph-mon[32752]: pgmap v8060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:20 smithi167 ceph-mon[32751]: pgmap v8060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:20 smithi090 ceph-mon[29180]: pgmap v8060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13802.189815 secs 2024-04-26T22:25:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13802.189815 secs 2024-04-26T22:25:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13802.189815 secs 2024-04-26T22:25:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13802.938140 secs 2024-04-26T22:25:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:22 smithi119 ceph-mon[32752]: pgmap v8061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13802.938140 secs 2024-04-26T22:25:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:22 smithi167 ceph-mon[32751]: pgmap v8061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13802.938140 secs 2024-04-26T22:25:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:22 smithi090 ceph-mon[29180]: pgmap v8061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:24 smithi119 ceph-mon[32752]: pgmap v8062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:24 smithi167 ceph-mon[32751]: pgmap v8062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:24 smithi090 ceph-mon[29180]: pgmap v8062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:26 smithi119 ceph-mon[32752]: pgmap v8063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13807.189900 secs 2024-04-26T22:25:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:26 smithi167 ceph-mon[32751]: pgmap v8063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13807.189900 secs 2024-04-26T22:25:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:26 smithi090 ceph-mon[29180]: pgmap v8063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13807.189900 secs 2024-04-26T22:25:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13807.938231 secs 2024-04-26T22:25:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13807.938231 secs 2024-04-26T22:25:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13807.938231 secs 2024-04-26T22:25:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:28 smithi167 ceph-mon[32751]: pgmap v8064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:28 smithi090 ceph-mon[29180]: pgmap v8064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:28 smithi119 ceph-mon[32752]: pgmap v8064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:30 smithi167 ceph-mon[32751]: pgmap v8065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:30 smithi090 ceph-mon[29180]: pgmap v8065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:30 smithi119 ceph-mon[32752]: pgmap v8065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:31.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13812.189991 secs 2024-04-26T22:25:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13812.189991 secs 2024-04-26T22:25:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13812.189991 secs 2024-04-26T22:25:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13812.938364 secs 2024-04-26T22:25:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:32 smithi090 ceph-mon[29180]: pgmap v8066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13812.938364 secs 2024-04-26T22:25:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:32 smithi119 ceph-mon[32752]: pgmap v8066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13812.938364 secs 2024-04-26T22:25:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:32 smithi167 ceph-mon[32751]: pgmap v8066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:33 smithi090 ceph-mon[29180]: pgmap v8067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:33 smithi119 ceph-mon[32752]: pgmap v8067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:33 smithi167 ceph-mon[32751]: pgmap v8067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:34.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:25:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:25:34.534+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:25:34.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:25:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:25:34.534+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:25:34.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:25:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:25:34.534+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:25:34.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:25:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:25:34.534+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:25:34.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:25:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:25:34.534+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:35 smithi090 ceph-mon[29180]: pgmap v8068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:35 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13817.190075 secs 2024-04-26T22:25:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:35 smithi119 ceph-mon[32752]: pgmap v8068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:35 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13817.190075 secs 2024-04-26T22:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:35 smithi167 ceph-mon[32751]: pgmap v8068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:35 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13817.190075 secs 2024-04-26T22:25:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13817.938474 secs 2024-04-26T22:25:37.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13817.938474 secs 2024-04-26T22:25:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13817.938474 secs 2024-04-26T22:25:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:37 smithi090 ceph-mon[29180]: pgmap v8069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:37 smithi119 ceph-mon[32752]: pgmap v8069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:37 smithi167 ceph-mon[32751]: pgmap v8069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:39 smithi090 ceph-mon[29180]: pgmap v8070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:25:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:39 smithi119 ceph-mon[32752]: pgmap v8070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:25:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:39 smithi167 ceph-mon[32751]: pgmap v8070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:25:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:40 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13822.190162 secs 2024-04-26T22:25:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:40 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13822.190162 secs 2024-04-26T22:25:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:40 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13822.190162 secs 2024-04-26T22:25:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13822.938585 secs 2024-04-26T22:25:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:41 smithi090 ceph-mon[29180]: pgmap v8071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:25:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:25:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:25:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:25:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:25:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13822.938585 secs 2024-04-26T22:25:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:41 smithi119 ceph-mon[32752]: pgmap v8071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:25:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:25:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:25:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:25:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:25:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13822.938585 secs 2024-04-26T22:25:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:41 smithi167 ceph-mon[32751]: pgmap v8071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:25:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:25:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:25:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:25:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:25:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:42 smithi090 ceph-mon[29180]: pgmap v8072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-26T22:25:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:42 smithi090 ceph-mon[29180]: pgmap v8073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:42 smithi119 ceph-mon[32752]: pgmap v8072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-26T22:25:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:42 smithi119 ceph-mon[32752]: pgmap v8073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:42 smithi167 ceph-mon[32751]: pgmap v8072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-26T22:25:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:42 smithi167 ceph-mon[32751]: pgmap v8073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:44 smithi090 ceph-mon[29180]: pgmap v8074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:44 smithi119 ceph-mon[32752]: pgmap v8074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:44 smithi167 ceph-mon[32751]: pgmap v8074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:46.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:45 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13827.190243 secs 2024-04-26T22:25:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:45 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13827.190243 secs 2024-04-26T22:25:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:45 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13827.190243 secs 2024-04-26T22:25:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:46 smithi090 ceph-mon[29180]: pgmap v8075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13827.938701 secs 2024-04-26T22:25:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:46 smithi119 ceph-mon[32752]: pgmap v8075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13827.938701 secs 2024-04-26T22:25:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:46 smithi167 ceph-mon[32751]: pgmap v8075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-26T22:25:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13827.938701 secs 2024-04-26T22:25:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:48 smithi090 ceph-mon[29180]: pgmap v8076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:25:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:48 smithi119 ceph-mon[32752]: pgmap v8076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:25:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:48 smithi167 ceph-mon[32751]: pgmap v8076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:25:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:50 smithi119 ceph-mon[32752]: pgmap v8077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:25:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:50 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13832.190302 secs 2024-04-26T22:25:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:50 smithi167 ceph-mon[32751]: pgmap v8077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:25:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:50 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13832.190302 secs 2024-04-26T22:25:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:50 smithi090 ceph-mon[29180]: pgmap v8077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:25:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:50 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13832.190302 secs 2024-04-26T22:25:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13832.938838 secs 2024-04-26T22:25:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13832.938838 secs 2024-04-26T22:25:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13832.938838 secs 2024-04-26T22:25:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:52 smithi119 ceph-mon[32752]: pgmap v8078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:25:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:52 smithi167 ceph-mon[32751]: pgmap v8078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:25:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:52 smithi090 ceph-mon[29180]: pgmap v8078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:25:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:54 smithi119 ceph-mon[32752]: pgmap v8079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:54 smithi167 ceph-mon[32751]: pgmap v8079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:55.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:54 smithi090 ceph-mon[29180]: pgmap v8079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:25:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:55 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13837.190384 secs 2024-04-26T22:25:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:55 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13837.190384 secs 2024-04-26T22:25:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:55 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13837.190384 secs 2024-04-26T22:25:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:56 smithi119 ceph-mon[32752]: pgmap v8080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13837.938962 secs 2024-04-26T22:25:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:56 smithi167 ceph-mon[32751]: pgmap v8080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13837.938962 secs 2024-04-26T22:25:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:56 smithi090 ceph-mon[29180]: pgmap v8080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13837.938962 secs 2024-04-26T22:25:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:25:58 smithi119 ceph-mon[32752]: pgmap v8081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:25:58 smithi167 ceph-mon[32751]: pgmap v8081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:25:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:25:58 smithi090 ceph-mon[29180]: pgmap v8081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:00 smithi119 ceph-mon[32752]: pgmap v8082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:00 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13842.190469 secs 2024-04-26T22:26:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:00 smithi167 ceph-mon[32751]: pgmap v8082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:00 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13842.190469 secs 2024-04-26T22:26:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:00 smithi090 ceph-mon[29180]: pgmap v8082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:00 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13842.190469 secs 2024-04-26T22:26:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13842.939087 secs 2024-04-26T22:26:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13842.939087 secs 2024-04-26T22:26:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13842.939087 secs 2024-04-26T22:26:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:02 smithi119 ceph-mon[32752]: pgmap v8083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:02 smithi167 ceph-mon[32751]: pgmap v8083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:02 smithi090 ceph-mon[29180]: pgmap v8083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:04 smithi119 ceph-mon[32752]: pgmap v8084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:04 smithi167 ceph-mon[32751]: pgmap v8084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:04 smithi090 ceph-mon[29180]: pgmap v8084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:05 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13847.190555 secs 2024-04-26T22:26:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:05 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13847.190555 secs 2024-04-26T22:26:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:05 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13847.190555 secs 2024-04-26T22:26:07.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:06 smithi167 ceph-mon[32751]: pgmap v8085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:07.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13847.939192 secs 2024-04-26T22:26:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:06 smithi119 ceph-mon[32752]: pgmap v8085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13847.939192 secs 2024-04-26T22:26:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:06 smithi090 ceph-mon[29180]: pgmap v8085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13847.939192 secs 2024-04-26T22:26:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:08 smithi119 ceph-mon[32752]: pgmap v8086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:08 smithi167 ceph-mon[32751]: pgmap v8086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:08 smithi090 ceph-mon[29180]: pgmap v8086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:10 smithi119 ceph-mon[32752]: pgmap v8087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:10 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13852.190641 secs 2024-04-26T22:26:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:10 smithi167 ceph-mon[32751]: pgmap v8087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:10 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13852.190641 secs 2024-04-26T22:26:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:10 smithi090 ceph-mon[29180]: pgmap v8087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:10 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13852.190641 secs 2024-04-26T22:26:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13852.939349 secs 2024-04-26T22:26:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13852.939349 secs 2024-04-26T22:26:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13852.939349 secs 2024-04-26T22:26:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:12 smithi119 ceph-mon[32752]: pgmap v8088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:12 smithi167 ceph-mon[32751]: pgmap v8088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:12 smithi090 ceph-mon[29180]: pgmap v8088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:26:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:14 smithi119 ceph-mon[32752]: pgmap v8089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:14 smithi167 ceph-mon[32751]: pgmap v8089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:14 smithi090 ceph-mon[29180]: pgmap v8089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:26:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:15 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13857.190706 secs 2024-04-26T22:26:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:15 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13857.190706 secs 2024-04-26T22:26:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:15 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13857.190706 secs 2024-04-26T22:26:17.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:16 smithi090 ceph-mon[29180]: pgmap v8090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:17.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13857.939471 secs 2024-04-26T22:26:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:16 smithi119 ceph-mon[32752]: pgmap v8090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13857.939471 secs 2024-04-26T22:26:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:16 smithi167 ceph-mon[32751]: pgmap v8090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13857.939471 secs 2024-04-26T22:26:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:18 smithi119 ceph-mon[32752]: pgmap v8091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:18 smithi167 ceph-mon[32751]: pgmap v8091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:18 smithi090 ceph-mon[29180]: pgmap v8091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:20 smithi119 ceph-mon[32752]: pgmap v8092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:20 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13862.190811 secs 2024-04-26T22:26:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:20 smithi167 ceph-mon[32751]: pgmap v8092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:20 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13862.190811 secs 2024-04-26T22:26:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:20 smithi090 ceph-mon[29180]: pgmap v8092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:20 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13862.190811 secs 2024-04-26T22:26:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13862.939568 secs 2024-04-26T22:26:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13862.939568 secs 2024-04-26T22:26:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13862.939568 secs 2024-04-26T22:26:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:22 smithi119 ceph-mon[32752]: pgmap v8093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:22 smithi167 ceph-mon[32751]: pgmap v8093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:22 smithi090 ceph-mon[29180]: pgmap v8093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:24 smithi119 ceph-mon[32752]: pgmap v8094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:24 smithi167 ceph-mon[32751]: pgmap v8094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:24 smithi090 ceph-mon[29180]: pgmap v8094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:25 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13867.190879 secs 2024-04-26T22:26:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:25 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13867.190879 secs 2024-04-26T22:26:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:25 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13867.190879 secs 2024-04-26T22:26:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:26 smithi119 ceph-mon[32752]: pgmap v8095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13867.939699 secs 2024-04-26T22:26:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:26 smithi167 ceph-mon[32751]: pgmap v8095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13867.939699 secs 2024-04-26T22:26:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:26 smithi090 ceph-mon[29180]: pgmap v8095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13867.939699 secs 2024-04-26T22:26:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:28 smithi119 ceph-mon[32752]: pgmap v8096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:28 smithi167 ceph-mon[32751]: pgmap v8096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:28 smithi090 ceph-mon[29180]: pgmap v8096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:30 smithi119 ceph-mon[32752]: pgmap v8097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:30 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13872.190969 secs 2024-04-26T22:26:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:30 smithi167 ceph-mon[32751]: pgmap v8097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:30 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13872.190969 secs 2024-04-26T22:26:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:30 smithi090 ceph-mon[29180]: pgmap v8097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:30 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13872.190969 secs 2024-04-26T22:26:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13872.939812 secs 2024-04-26T22:26:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13872.939812 secs 2024-04-26T22:26:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13872.939812 secs 2024-04-26T22:26:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:32 smithi119 ceph-mon[32752]: pgmap v8098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:32 smithi167 ceph-mon[32751]: pgmap v8098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:32 smithi090 ceph-mon[29180]: pgmap v8098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:34 smithi119 ceph-mon[32752]: pgmap v8099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:34 smithi167 ceph-mon[32751]: pgmap v8099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:34 smithi090 ceph-mon[29180]: pgmap v8099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:35 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 13877.191054 secs 2024-04-26T22:26:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:35 smithi119 ceph-mon[32752]: slow request 360.525188 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:26:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:35 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 13877.191054 secs 2024-04-26T22:26:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:35 smithi167 ceph-mon[32751]: slow request 360.525188 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:26:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:35 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 13877.191054 secs 2024-04-26T22:26:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:35 smithi090 ceph-mon[29180]: slow request 360.525188 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:26:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:36 smithi119 ceph-mon[32752]: pgmap v8100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13877.939892 secs 2024-04-26T22:26:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:36 smithi167 ceph-mon[32751]: pgmap v8100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13877.939892 secs 2024-04-26T22:26:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:36 smithi090 ceph-mon[29180]: pgmap v8100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13877.939892 secs 2024-04-26T22:26:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:39 smithi119 ceph-mon[32752]: pgmap v8101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:39 smithi167 ceph-mon[32751]: pgmap v8101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:39 smithi090 ceph-mon[29180]: pgmap v8101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:41 smithi119 ceph-mon[32752]: pgmap v8102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13882.191141 secs 2024-04-26T22:26:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:41 smithi167 ceph-mon[32751]: pgmap v8102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13882.191141 secs 2024-04-26T22:26:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:41 smithi090 ceph-mon[29180]: pgmap v8102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13882.191141 secs 2024-04-26T22:26:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13882.939955 secs 2024-04-26T22:26:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:26:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13882.939955 secs 2024-04-26T22:26:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:26:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13882.939955 secs 2024-04-26T22:26:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:26:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:43 smithi119 ceph-mon[32752]: pgmap v8103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:26:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:26:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:43 smithi119 ceph-mon[32752]: pgmap v8104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:26:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:43 smithi119 ceph-mon[32752]: pgmap v8105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:26:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:26:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:43 smithi167 ceph-mon[32751]: pgmap v8103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:26:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:26:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:43 smithi167 ceph-mon[32751]: pgmap v8104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:26:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:43 smithi167 ceph-mon[32751]: pgmap v8105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:26:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:26:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:43 smithi090 ceph-mon[29180]: pgmap v8103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:26:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:26:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:43 smithi090 ceph-mon[29180]: pgmap v8104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:26:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:43 smithi090 ceph-mon[29180]: pgmap v8105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:26:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:26:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:44 smithi119 ceph-mon[32752]: pgmap v8106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:44 smithi167 ceph-mon[32751]: pgmap v8106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:44 smithi090 ceph-mon[29180]: pgmap v8106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:45 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13887.191200 secs 2024-04-26T22:26:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:45 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13887.191200 secs 2024-04-26T22:26:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:45 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13887.191200 secs 2024-04-26T22:26:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13887.940022 secs 2024-04-26T22:26:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:46 smithi119 ceph-mon[32752]: pgmap v8107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13887.940022 secs 2024-04-26T22:26:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:46 smithi167 ceph-mon[32751]: pgmap v8107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13887.940022 secs 2024-04-26T22:26:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:46 smithi090 ceph-mon[29180]: pgmap v8107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:49 smithi119 ceph-mon[32752]: pgmap v8108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:49 smithi167 ceph-mon[32751]: pgmap v8108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:49 smithi090 ceph-mon[29180]: pgmap v8108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13892.191260 secs 2024-04-26T22:26:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:51 smithi119 ceph-mon[32752]: pgmap v8109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13892.191260 secs 2024-04-26T22:26:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:51 smithi167 ceph-mon[32751]: pgmap v8109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13892.191260 secs 2024-04-26T22:26:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:51 smithi090 ceph-mon[29180]: pgmap v8109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:26:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13892.940140 secs 2024-04-26T22:26:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13892.940140 secs 2024-04-26T22:26:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13892.940140 secs 2024-04-26T22:26:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:53 smithi119 ceph-mon[32752]: pgmap v8110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:26:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:53 smithi167 ceph-mon[32751]: pgmap v8110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:26:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:53 smithi090 ceph-mon[29180]: pgmap v8110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:26:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:55 smithi119 ceph-mon[32752]: pgmap v8111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:55 smithi167 ceph-mon[32751]: pgmap v8111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:55 smithi090 ceph-mon[29180]: pgmap v8111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13897.191346 secs 2024-04-26T22:26:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13897.940254 secs 2024-04-26T22:26:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13897.191346 secs 2024-04-26T22:26:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13897.940254 secs 2024-04-26T22:26:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13897.191346 secs 2024-04-26T22:26:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13897.940254 secs 2024-04-26T22:26:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:57 smithi119 ceph-mon[32752]: pgmap v8112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:57 smithi167 ceph-mon[32751]: pgmap v8112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:57 smithi090 ceph-mon[29180]: pgmap v8112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:26:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:26:59 smithi119 ceph-mon[32752]: pgmap v8113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:26:59 smithi167 ceph-mon[32751]: pgmap v8113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:26:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:26:59 smithi090 ceph-mon[29180]: pgmap v8113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13902.191428 secs 2024-04-26T22:27:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:01 smithi119 ceph-mon[32752]: pgmap v8114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13902.940392 secs 2024-04-26T22:27:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13902.191428 secs 2024-04-26T22:27:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:01 smithi167 ceph-mon[32751]: pgmap v8114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13902.940392 secs 2024-04-26T22:27:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13902.191428 secs 2024-04-26T22:27:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:01 smithi090 ceph-mon[29180]: pgmap v8114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13902.940392 secs 2024-04-26T22:27:03.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:03 smithi090 ceph-mon[29180]: pgmap v8115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:03 smithi119 ceph-mon[32752]: pgmap v8115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:03 smithi167 ceph-mon[32751]: pgmap v8115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:05 smithi119 ceph-mon[32752]: pgmap v8116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:05 smithi167 ceph-mon[32751]: pgmap v8116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:05 smithi090 ceph-mon[29180]: pgmap v8116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13907.191515 secs 2024-04-26T22:27:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13907.940484 secs 2024-04-26T22:27:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13907.191515 secs 2024-04-26T22:27:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13907.940484 secs 2024-04-26T22:27:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13907.191515 secs 2024-04-26T22:27:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13907.940484 secs 2024-04-26T22:27:07.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:07 smithi167 ceph-mon[32751]: pgmap v8117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:07 smithi119 ceph-mon[32752]: pgmap v8117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:07 smithi090 ceph-mon[29180]: pgmap v8117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:09 smithi119 ceph-mon[32752]: pgmap v8118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:09 smithi167 ceph-mon[32751]: pgmap v8118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:09 smithi090 ceph-mon[29180]: pgmap v8118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13912.191601 secs 2024-04-26T22:27:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:11 smithi119 ceph-mon[32752]: pgmap v8119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13912.940552 secs 2024-04-26T22:27:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13912.191601 secs 2024-04-26T22:27:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:11 smithi167 ceph-mon[32751]: pgmap v8119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13912.940552 secs 2024-04-26T22:27:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13912.191601 secs 2024-04-26T22:27:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:11 smithi090 ceph-mon[29180]: pgmap v8119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13912.940552 secs 2024-04-26T22:27:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:13 smithi119 ceph-mon[32752]: pgmap v8120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:13 smithi167 ceph-mon[32751]: pgmap v8120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:13 smithi090 ceph-mon[29180]: pgmap v8120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:15 smithi119 ceph-mon[32752]: pgmap v8121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:15 smithi167 ceph-mon[32751]: pgmap v8121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:15 smithi090 ceph-mon[29180]: pgmap v8121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13917.191684 secs 2024-04-26T22:27:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13917.940689 secs 2024-04-26T22:27:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13917.191684 secs 2024-04-26T22:27:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13917.940689 secs 2024-04-26T22:27:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13917.191684 secs 2024-04-26T22:27:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13917.940689 secs 2024-04-26T22:27:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:17 smithi119 ceph-mon[32752]: pgmap v8122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:17 smithi167 ceph-mon[32751]: pgmap v8122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:17 smithi090 ceph-mon[29180]: pgmap v8122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:19 smithi119 ceph-mon[32752]: pgmap v8123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:19 smithi167 ceph-mon[32751]: pgmap v8123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:19 smithi090 ceph-mon[29180]: pgmap v8123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:20 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13922.191768 secs 2024-04-26T22:27:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:20 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13922.191768 secs 2024-04-26T22:27:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:20 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13922.191768 secs 2024-04-26T22:27:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:21 smithi119 ceph-mon[32752]: pgmap v8124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13922.940818 secs 2024-04-26T22:27:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:21 smithi167 ceph-mon[32751]: pgmap v8124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13922.940818 secs 2024-04-26T22:27:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:21 smithi090 ceph-mon[29180]: pgmap v8124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13922.940818 secs 2024-04-26T22:27:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:23 smithi119 ceph-mon[32752]: pgmap v8125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:23 smithi167 ceph-mon[32751]: pgmap v8125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:23 smithi090 ceph-mon[29180]: pgmap v8125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:25 smithi119 ceph-mon[32752]: pgmap v8126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:25 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13927.191852 secs 2024-04-26T22:27:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:25 smithi167 ceph-mon[32751]: pgmap v8126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:25 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13927.191852 secs 2024-04-26T22:27:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:25 smithi090 ceph-mon[29180]: pgmap v8126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:25 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13927.191852 secs 2024-04-26T22:27:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13927.940879 secs 2024-04-26T22:27:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13927.940879 secs 2024-04-26T22:27:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13927.940879 secs 2024-04-26T22:27:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:27 smithi119 ceph-mon[32752]: pgmap v8127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:27 smithi167 ceph-mon[32751]: pgmap v8127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:27 smithi090 ceph-mon[29180]: pgmap v8127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:29 smithi119 ceph-mon[32752]: pgmap v8128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:29 smithi167 ceph-mon[32751]: pgmap v8128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:29 smithi090 ceph-mon[29180]: pgmap v8128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:30 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13932.191941 secs 2024-04-26T22:27:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:30 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13932.191941 secs 2024-04-26T22:27:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:30 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13932.191941 secs 2024-04-26T22:27:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:31 smithi119 ceph-mon[32752]: pgmap v8129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13932.941006 secs 2024-04-26T22:27:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:31 smithi167 ceph-mon[32751]: pgmap v8129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13932.941006 secs 2024-04-26T22:27:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:31 smithi090 ceph-mon[29180]: pgmap v8129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13932.941006 secs 2024-04-26T22:27:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:33 smithi119 ceph-mon[32752]: pgmap v8130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:33 smithi167 ceph-mon[32751]: pgmap v8130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:33 smithi090 ceph-mon[29180]: pgmap v8130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:35 smithi119 ceph-mon[32752]: pgmap v8131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:35 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13937.192026 secs 2024-04-26T22:27:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:35 smithi167 ceph-mon[32751]: pgmap v8131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:35 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13937.192026 secs 2024-04-26T22:27:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:35 smithi090 ceph-mon[29180]: pgmap v8131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:35 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13937.192026 secs 2024-04-26T22:27:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13937.941083 secs 2024-04-26T22:27:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13937.941083 secs 2024-04-26T22:27:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13937.941083 secs 2024-04-26T22:27:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:37 smithi119 ceph-mon[32752]: pgmap v8132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:37 smithi167 ceph-mon[32751]: pgmap v8132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:37 smithi090 ceph-mon[29180]: pgmap v8132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:39 smithi119 ceph-mon[32752]: pgmap v8133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:39 smithi167 ceph-mon[32751]: pgmap v8133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:39 smithi090 ceph-mon[29180]: pgmap v8133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:40 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13942.192110 secs 2024-04-26T22:27:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:40 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13942.192110 secs 2024-04-26T22:27:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:40 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13942.192110 secs 2024-04-26T22:27:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:41 smithi119 ceph-mon[32752]: pgmap v8134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13942.941208 secs 2024-04-26T22:27:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:41 smithi167 ceph-mon[32751]: pgmap v8134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13942.941208 secs 2024-04-26T22:27:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:41 smithi090 ceph-mon[29180]: pgmap v8134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13942.941208 secs 2024-04-26T22:27:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:27:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:27:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:27:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:43 smithi119 ceph-mon[32752]: pgmap v8135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:43 smithi090 ceph-mon[29180]: pgmap v8135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:43 smithi167 ceph-mon[32751]: pgmap v8135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: pgmap v8136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13947.192172 secs 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:27:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:45 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: pgmap v8136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13947.192172 secs 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:27:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:45 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:27:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: pgmap v8136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:27:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13947.192172 secs 2024-04-26T22:27:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:27:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:45 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:27:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13947.941285 secs 2024-04-26T22:27:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:27:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:27:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:27:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:27:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: pgmap v8137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 894 B/s rd, 1 op/s 2024-04-26T22:27:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: pgmap v8138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13947.941285 secs 2024-04-26T22:27:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:27:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:27:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:27:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:27:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: pgmap v8137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 894 B/s rd, 1 op/s 2024-04-26T22:27:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: pgmap v8138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13947.941285 secs 2024-04-26T22:27:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: pgmap v8137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 894 B/s rd, 1 op/s 2024-04-26T22:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: pgmap v8138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:27:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:48 smithi090 ceph-mon[29180]: pgmap v8139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s rd, 1 op/s 2024-04-26T22:27:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:48 smithi119 ceph-mon[32752]: pgmap v8139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s rd, 1 op/s 2024-04-26T22:27:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:48 smithi167 ceph-mon[32751]: pgmap v8139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s rd, 1 op/s 2024-04-26T22:27:51.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:50 smithi119 ceph-mon[32752]: pgmap v8140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:51.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:50 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13952.192233 secs 2024-04-26T22:27:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:50 smithi090 ceph-mon[29180]: pgmap v8140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:50 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13952.192233 secs 2024-04-26T22:27:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:50 smithi167 ceph-mon[32751]: pgmap v8140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:50 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13952.192233 secs 2024-04-26T22:27:52.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13952.941404 secs 2024-04-26T22:27:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13952.941404 secs 2024-04-26T22:27:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13952.941404 secs 2024-04-26T22:27:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:52 smithi090 ceph-mon[29180]: pgmap v8141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:52 smithi119 ceph-mon[32752]: pgmap v8141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:52 smithi167 ceph-mon[32751]: pgmap v8141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:54 smithi090 ceph-mon[29180]: pgmap v8142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:54 smithi119 ceph-mon[32752]: pgmap v8142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:54 smithi167 ceph-mon[32751]: pgmap v8142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:27:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:55 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13957.192319 secs 2024-04-26T22:27:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:55 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13957.192319 secs 2024-04-26T22:27:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:55 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13957.192319 secs 2024-04-26T22:27:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13957.941513 secs 2024-04-26T22:27:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:56 smithi119 ceph-mon[32752]: pgmap v8143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:27:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13957.941513 secs 2024-04-26T22:27:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:56 smithi167 ceph-mon[32751]: pgmap v8143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:27:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13957.941513 secs 2024-04-26T22:27:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:56 smithi090 ceph-mon[29180]: pgmap v8143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:27:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:27:58 smithi119 ceph-mon[32752]: pgmap v8144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:27:58 smithi167 ceph-mon[32751]: pgmap v8144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:27:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:27:58 smithi090 ceph-mon[29180]: pgmap v8144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:00 smithi119 ceph-mon[32752]: pgmap v8145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:00 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13962.192383 secs 2024-04-26T22:28:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:00 smithi167 ceph-mon[32751]: pgmap v8145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:00 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13962.192383 secs 2024-04-26T22:28:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:00 smithi090 ceph-mon[29180]: pgmap v8145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:00 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13962.192383 secs 2024-04-26T22:28:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13962.941586 secs 2024-04-26T22:28:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13962.941586 secs 2024-04-26T22:28:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13962.941586 secs 2024-04-26T22:28:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:02 smithi119 ceph-mon[32752]: pgmap v8146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:02 smithi167 ceph-mon[32751]: pgmap v8146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:02 smithi090 ceph-mon[29180]: pgmap v8146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:05.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:04 smithi119 ceph-mon[32752]: pgmap v8147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:04 smithi167 ceph-mon[32751]: pgmap v8147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:04 smithi090 ceph-mon[29180]: pgmap v8147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:05 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13967.192467 secs 2024-04-26T22:28:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:05 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13967.192467 secs 2024-04-26T22:28:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:05 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13967.192467 secs 2024-04-26T22:28:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13967.941703 secs 2024-04-26T22:28:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:06 smithi090 ceph-mon[29180]: pgmap v8148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:07.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13967.941703 secs 2024-04-26T22:28:07.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:06 smithi167 ceph-mon[32751]: pgmap v8148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13967.941703 secs 2024-04-26T22:28:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:06 smithi119 ceph-mon[32752]: pgmap v8148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:08 smithi119 ceph-mon[32752]: pgmap v8149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:08 smithi167 ceph-mon[32751]: pgmap v8149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:08 smithi090 ceph-mon[29180]: pgmap v8149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:10 smithi119 ceph-mon[32752]: pgmap v8150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:10 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13972.192550 secs 2024-04-26T22:28:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:10 smithi167 ceph-mon[32751]: pgmap v8150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:10 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13972.192550 secs 2024-04-26T22:28:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:10 smithi090 ceph-mon[29180]: pgmap v8150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:10 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13972.192550 secs 2024-04-26T22:28:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13972.941820 secs 2024-04-26T22:28:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13972.941820 secs 2024-04-26T22:28:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13972.941820 secs 2024-04-26T22:28:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:12 smithi119 ceph-mon[32752]: pgmap v8151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:12 smithi167 ceph-mon[32751]: pgmap v8151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:12 smithi090 ceph-mon[29180]: pgmap v8151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:14 smithi119 ceph-mon[32752]: pgmap v8152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:14 smithi167 ceph-mon[32751]: pgmap v8152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:14 smithi090 ceph-mon[29180]: pgmap v8152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:15 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13977.192636 secs 2024-04-26T22:28:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:15 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13977.192636 secs 2024-04-26T22:28:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:15 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13977.192636 secs 2024-04-26T22:28:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13977.941940 secs 2024-04-26T22:28:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:16 smithi119 ceph-mon[32752]: pgmap v8153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13977.941940 secs 2024-04-26T22:28:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:16 smithi167 ceph-mon[32751]: pgmap v8153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13977.941940 secs 2024-04-26T22:28:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:16 smithi090 ceph-mon[29180]: pgmap v8153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:18 smithi119 ceph-mon[32752]: pgmap v8154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:18 smithi167 ceph-mon[32751]: pgmap v8154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:18 smithi090 ceph-mon[29180]: pgmap v8154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:20 smithi119 ceph-mon[32752]: pgmap v8155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:20 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13982.192723 secs 2024-04-26T22:28:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:20 smithi167 ceph-mon[32751]: pgmap v8155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:20 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13982.192723 secs 2024-04-26T22:28:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:20 smithi090 ceph-mon[29180]: pgmap v8155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:20 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13982.192723 secs 2024-04-26T22:28:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13982.942059 secs 2024-04-26T22:28:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13982.942059 secs 2024-04-26T22:28:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13982.942059 secs 2024-04-26T22:28:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:22 smithi119 ceph-mon[32752]: pgmap v8156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:22 smithi167 ceph-mon[32751]: pgmap v8156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:22 smithi090 ceph-mon[29180]: pgmap v8156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:24 smithi119 ceph-mon[32752]: pgmap v8157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:24 smithi167 ceph-mon[32751]: pgmap v8157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:24 smithi090 ceph-mon[29180]: pgmap v8157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:25 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13987.192824 secs 2024-04-26T22:28:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:25 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13987.192824 secs 2024-04-26T22:28:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:25 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13987.192824 secs 2024-04-26T22:28:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13987.942194 secs 2024-04-26T22:28:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:26 smithi119 ceph-mon[32752]: pgmap v8158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13987.942194 secs 2024-04-26T22:28:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:26 smithi167 ceph-mon[32751]: pgmap v8158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13987.942194 secs 2024-04-26T22:28:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:26 smithi090 ceph-mon[29180]: pgmap v8158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:28 smithi119 ceph-mon[32752]: pgmap v8159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:28 smithi167 ceph-mon[32751]: pgmap v8159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:28 smithi090 ceph-mon[29180]: pgmap v8159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:30 smithi119 ceph-mon[32752]: pgmap v8160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:30 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13992.192885 secs 2024-04-26T22:28:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:30 smithi167 ceph-mon[32751]: pgmap v8160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:30 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13992.192885 secs 2024-04-26T22:28:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:30 smithi090 ceph-mon[29180]: pgmap v8160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:30 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13992.192885 secs 2024-04-26T22:28:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13992.942317 secs 2024-04-26T22:28:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13992.942317 secs 2024-04-26T22:28:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13992.942317 secs 2024-04-26T22:28:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:32 smithi119 ceph-mon[32752]: pgmap v8161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:32 smithi167 ceph-mon[32751]: pgmap v8161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:32 smithi090 ceph-mon[29180]: pgmap v8161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:34 smithi119 ceph-mon[32752]: pgmap v8162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:34 smithi167 ceph-mon[32751]: pgmap v8162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:34 smithi090 ceph-mon[29180]: pgmap v8162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:35 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 13997.192973 secs 2024-04-26T22:28:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:35 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 13997.192973 secs 2024-04-26T22:28:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:35 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 13997.192973 secs 2024-04-26T22:28:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 13997.942454 secs 2024-04-26T22:28:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:36 smithi119 ceph-mon[32752]: pgmap v8163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 13997.942454 secs 2024-04-26T22:28:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:36 smithi167 ceph-mon[32751]: pgmap v8163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 13997.942454 secs 2024-04-26T22:28:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:36 smithi090 ceph-mon[29180]: pgmap v8163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:38 smithi119 ceph-mon[32752]: pgmap v8164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:38 smithi167 ceph-mon[32751]: pgmap v8164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:38 smithi090 ceph-mon[29180]: pgmap v8164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:40 smithi119 ceph-mon[32752]: pgmap v8165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:40 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14002.193034 secs 2024-04-26T22:28:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:40 smithi167 ceph-mon[32751]: pgmap v8165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:40 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14002.193034 secs 2024-04-26T22:28:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:40 smithi090 ceph-mon[29180]: pgmap v8165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:40 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14002.193034 secs 2024-04-26T22:28:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14002.942582 secs 2024-04-26T22:28:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14002.942582 secs 2024-04-26T22:28:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14002.942582 secs 2024-04-26T22:28:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:42 smithi119 ceph-mon[32752]: pgmap v8166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:42 smithi167 ceph-mon[32751]: pgmap v8166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:42 smithi090 ceph-mon[29180]: pgmap v8166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:45 smithi119 ceph-mon[32752]: pgmap v8167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:45 smithi167 ceph-mon[32751]: pgmap v8167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:45 smithi090 ceph-mon[29180]: pgmap v8167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:28:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14007.193117 secs 2024-04-26T22:28:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:28:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14007.193117 secs 2024-04-26T22:28:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:28:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14007.193117 secs 2024-04-26T22:28:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:28:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14007.942710 secs 2024-04-26T22:28:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:47 smithi119 ceph-mon[32752]: pgmap v8168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14007.942710 secs 2024-04-26T22:28:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:47 smithi167 ceph-mon[32751]: pgmap v8168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14007.942710 secs 2024-04-26T22:28:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:47 smithi090 ceph-mon[29180]: pgmap v8168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:28:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:28:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:28:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:48 smithi119 ceph-mon[32752]: pgmap v8169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T22:28:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:48 smithi119 ceph-mon[32752]: pgmap v8170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:28:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:28:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:48 smithi167 ceph-mon[32751]: pgmap v8169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T22:28:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:48 smithi167 ceph-mon[32751]: pgmap v8170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:28:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:28:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:48 smithi090 ceph-mon[29180]: pgmap v8169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-26T22:28:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:48 smithi090 ceph-mon[29180]: pgmap v8170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:28:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:50 smithi119 ceph-mon[32752]: pgmap v8171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:50 smithi167 ceph-mon[32751]: pgmap v8171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:50 smithi090 ceph-mon[29180]: pgmap v8171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14012.193201 secs 2024-04-26T22:28:51.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14012.193201 secs 2024-04-26T22:28:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14012.193201 secs 2024-04-26T22:28:52.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14012.942789 secs 2024-04-26T22:28:52.549 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:52 smithi090 ceph-mon[29180]: pgmap v8172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14012.942789 secs 2024-04-26T22:28:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:52 smithi119 ceph-mon[32752]: pgmap v8172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14012.942789 secs 2024-04-26T22:28:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:52 smithi167 ceph-mon[32751]: pgmap v8172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:54 smithi090 ceph-mon[29180]: pgmap v8173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:28:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:54 smithi119 ceph-mon[32752]: pgmap v8173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:28:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:54 smithi167 ceph-mon[32751]: pgmap v8173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:28:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14017.193286 secs 2024-04-26T22:28:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:56 smithi119 ceph-mon[32752]: pgmap v8174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14017.193286 secs 2024-04-26T22:28:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:56 smithi167 ceph-mon[32751]: pgmap v8174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14017.193286 secs 2024-04-26T22:28:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:56 smithi090 ceph-mon[29180]: pgmap v8174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:28:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14017.942919 secs 2024-04-26T22:28:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14017.942919 secs 2024-04-26T22:28:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14017.942919 secs 2024-04-26T22:28:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:28:58 smithi119 ceph-mon[32752]: pgmap v8175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:28:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:28:58 smithi167 ceph-mon[32751]: pgmap v8175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:28:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:28:58 smithi090 ceph-mon[29180]: pgmap v8175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:29:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:00 smithi119 ceph-mon[32752]: pgmap v8176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:00 smithi167 ceph-mon[32751]: pgmap v8176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:00 smithi090 ceph-mon[29180]: pgmap v8176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14022.193373 secs 2024-04-26T22:29:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14022.193373 secs 2024-04-26T22:29:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14022.193373 secs 2024-04-26T22:29:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14022.943057 secs 2024-04-26T22:29:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:02 smithi119 ceph-mon[32752]: pgmap v8177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14022.943057 secs 2024-04-26T22:29:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:02 smithi167 ceph-mon[32751]: pgmap v8177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14022.943057 secs 2024-04-26T22:29:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:02 smithi090 ceph-mon[29180]: pgmap v8177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:04 smithi119 ceph-mon[32752]: pgmap v8178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:04 smithi167 ceph-mon[32751]: pgmap v8178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:04 smithi090 ceph-mon[29180]: pgmap v8178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14027.193459 secs 2024-04-26T22:29:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:06 smithi119 ceph-mon[32752]: pgmap v8179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14027.193459 secs 2024-04-26T22:29:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:06 smithi167 ceph-mon[32751]: pgmap v8179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14027.193459 secs 2024-04-26T22:29:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:06 smithi090 ceph-mon[29180]: pgmap v8179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14027.943197 secs 2024-04-26T22:29:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14027.943197 secs 2024-04-26T22:29:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14027.943197 secs 2024-04-26T22:29:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:08 smithi119 ceph-mon[32752]: pgmap v8180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:08 smithi167 ceph-mon[32751]: pgmap v8180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:08 smithi090 ceph-mon[29180]: pgmap v8180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:10 smithi119 ceph-mon[32752]: pgmap v8181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:10 smithi167 ceph-mon[32751]: pgmap v8181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:10 smithi090 ceph-mon[29180]: pgmap v8181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14032.193518 secs 2024-04-26T22:29:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14032.193518 secs 2024-04-26T22:29:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14032.193518 secs 2024-04-26T22:29:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14032.943334 secs 2024-04-26T22:29:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:12 smithi119 ceph-mon[32752]: pgmap v8182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14032.943334 secs 2024-04-26T22:29:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:12 smithi167 ceph-mon[32751]: pgmap v8182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14032.943334 secs 2024-04-26T22:29:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:12 smithi090 ceph-mon[29180]: pgmap v8182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:14 smithi119 ceph-mon[32752]: pgmap v8183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:14 smithi167 ceph-mon[32751]: pgmap v8183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:14 smithi090 ceph-mon[29180]: pgmap v8183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14037.193603 secs 2024-04-26T22:29:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:16 smithi119 ceph-mon[32752]: pgmap v8184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14037.193603 secs 2024-04-26T22:29:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:16 smithi167 ceph-mon[32751]: pgmap v8184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14037.193603 secs 2024-04-26T22:29:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:16 smithi090 ceph-mon[29180]: pgmap v8184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14037.943454 secs 2024-04-26T22:29:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14037.943454 secs 2024-04-26T22:29:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14037.943454 secs 2024-04-26T22:29:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:18 smithi119 ceph-mon[32752]: pgmap v8185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:18 smithi167 ceph-mon[32751]: pgmap v8185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:18 smithi090 ceph-mon[29180]: pgmap v8185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:20 smithi119 ceph-mon[32752]: pgmap v8186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:20 smithi167 ceph-mon[32751]: pgmap v8186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:20 smithi090 ceph-mon[29180]: pgmap v8186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:21.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14042.193708 secs 2024-04-26T22:29:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14042.193708 secs 2024-04-26T22:29:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14042.193708 secs 2024-04-26T22:29:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14042.943572 secs 2024-04-26T22:29:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:22 smithi119 ceph-mon[32752]: pgmap v8187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14042.943572 secs 2024-04-26T22:29:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:22 smithi167 ceph-mon[32751]: pgmap v8187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14042.943572 secs 2024-04-26T22:29:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:22 smithi090 ceph-mon[29180]: pgmap v8187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:24 smithi119 ceph-mon[32752]: pgmap v8188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:24 smithi167 ceph-mon[32751]: pgmap v8188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:24 smithi090 ceph-mon[29180]: pgmap v8188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14047.193814 secs 2024-04-26T22:29:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:26 smithi119 ceph-mon[32752]: pgmap v8189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14047.193814 secs 2024-04-26T22:29:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:26 smithi167 ceph-mon[32751]: pgmap v8189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14047.193814 secs 2024-04-26T22:29:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:26 smithi090 ceph-mon[29180]: pgmap v8189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14047.943731 secs 2024-04-26T22:29:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14047.943731 secs 2024-04-26T22:29:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14047.943731 secs 2024-04-26T22:29:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:28 smithi119 ceph-mon[32752]: pgmap v8190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:28 smithi167 ceph-mon[32751]: pgmap v8190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:28 smithi090 ceph-mon[29180]: pgmap v8190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:30 smithi119 ceph-mon[32752]: pgmap v8191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:30 smithi167 ceph-mon[32751]: pgmap v8191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:30 smithi090 ceph-mon[29180]: pgmap v8191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14052.193897 secs 2024-04-26T22:29:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14052.193897 secs 2024-04-26T22:29:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14052.193897 secs 2024-04-26T22:29:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14052.943803 secs 2024-04-26T22:29:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:32 smithi119 ceph-mon[32752]: pgmap v8192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14052.943803 secs 2024-04-26T22:29:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:32 smithi167 ceph-mon[32751]: pgmap v8192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14052.943803 secs 2024-04-26T22:29:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:32 smithi090 ceph-mon[29180]: pgmap v8192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:34 smithi119 ceph-mon[32752]: pgmap v8193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:34 smithi167 ceph-mon[32751]: pgmap v8193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:34 smithi090 ceph-mon[29180]: pgmap v8193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14057.193956 secs 2024-04-26T22:29:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:36 smithi119 ceph-mon[32752]: pgmap v8194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14057.193956 secs 2024-04-26T22:29:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:36 smithi167 ceph-mon[32751]: pgmap v8194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14057.193956 secs 2024-04-26T22:29:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:36 smithi090 ceph-mon[29180]: pgmap v8194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14057.943920 secs 2024-04-26T22:29:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14057.943920 secs 2024-04-26T22:29:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14057.943920 secs 2024-04-26T22:29:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:38 smithi119 ceph-mon[32752]: pgmap v8195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:38 smithi167 ceph-mon[32751]: pgmap v8195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:38 smithi090 ceph-mon[29180]: pgmap v8195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:40 smithi119 ceph-mon[32752]: pgmap v8196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:40 smithi167 ceph-mon[32751]: pgmap v8196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:40 smithi090 ceph-mon[29180]: pgmap v8196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14062.194046 secs 2024-04-26T22:29:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14062.194046 secs 2024-04-26T22:29:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14062.194046 secs 2024-04-26T22:29:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14062.944038 secs 2024-04-26T22:29:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:42 smithi119 ceph-mon[32752]: pgmap v8197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14062.944038 secs 2024-04-26T22:29:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:42 smithi167 ceph-mon[32751]: pgmap v8197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14062.944038 secs 2024-04-26T22:29:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:42 smithi090 ceph-mon[29180]: pgmap v8197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:44 smithi119 ceph-mon[32752]: pgmap v8198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:44 smithi167 ceph-mon[32751]: pgmap v8198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:44 smithi090 ceph-mon[29180]: pgmap v8198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:46 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 14067.194129 secs 2024-04-26T22:29:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:46 smithi119 ceph-mon[32752]: slow request 2883.492847 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:29:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:46 smithi119 ceph-mon[32752]: pgmap v8199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:46 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 14067.194129 secs 2024-04-26T22:29:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:46 smithi167 ceph-mon[32751]: slow request 2883.492847 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:29:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:46 smithi167 ceph-mon[32751]: pgmap v8199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:46 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 14067.194129 secs 2024-04-26T22:29:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:46 smithi090 ceph-mon[29180]: slow request 2883.492847 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:29:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:46 smithi090 ceph-mon[29180]: pgmap v8199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:29:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14067.944130 secs 2024-04-26T22:29:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:29:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14067.944130 secs 2024-04-26T22:29:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:29:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14067.944130 secs 2024-04-26T22:29:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:29:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:48 smithi167 ceph-mon[32751]: pgmap v8200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:29:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:29:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:29:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:48 smithi090 ceph-mon[29180]: pgmap v8200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:29:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:29:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:29:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:48 smithi119 ceph-mon[32752]: pgmap v8200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:29:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:29:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:29:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:29:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:49 smithi167 ceph-mon[32751]: pgmap v8201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:29:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:49 smithi167 ceph-mon[32751]: pgmap v8202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:29:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:49 smithi090 ceph-mon[29180]: pgmap v8201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:29:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:49 smithi090 ceph-mon[29180]: pgmap v8202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:29:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:49 smithi119 ceph-mon[32752]: pgmap v8201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:29:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:49 smithi119 ceph-mon[32752]: pgmap v8202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:51 smithi167 ceph-mon[32751]: pgmap v8203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14072.194189 secs 2024-04-26T22:29:51.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:51 smithi119 ceph-mon[32752]: pgmap v8203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:51.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14072.194189 secs 2024-04-26T22:29:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:51 smithi090 ceph-mon[29180]: pgmap v8203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14072.194189 secs 2024-04-26T22:29:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14072.944212 secs 2024-04-26T22:29:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14072.944212 secs 2024-04-26T22:29:52.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14072.944212 secs 2024-04-26T22:29:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:53 smithi090 ceph-mon[29180]: pgmap v8204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:53 smithi119 ceph-mon[32752]: pgmap v8204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:53 smithi167 ceph-mon[32751]: pgmap v8204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:55 smithi090 ceph-mon[29180]: pgmap v8205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:55 smithi119 ceph-mon[32752]: pgmap v8205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:55 smithi167 ceph-mon[32751]: pgmap v8205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14077.194271 secs 2024-04-26T22:29:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14077.194271 secs 2024-04-26T22:29:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14077.194271 secs 2024-04-26T22:29:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14077.944324 secs 2024-04-26T22:29:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:57 smithi090 ceph-mon[29180]: pgmap v8206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14077.944324 secs 2024-04-26T22:29:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:57 smithi119 ceph-mon[32752]: pgmap v8206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14077.944324 secs 2024-04-26T22:29:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:57 smithi167 ceph-mon[32751]: pgmap v8206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:29:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:29:59 smithi090 ceph-mon[29180]: pgmap v8207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:29:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:29:59 smithi119 ceph-mon[32752]: pgmap v8207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:29:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:29:59 smithi167 ceph-mon[32751]: pgmap v8207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:30:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:30:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:30:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:30:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:01 smithi090 ceph-mon[29180]: pgmap v8208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14082.194355 secs 2024-04-26T22:30:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:01 smithi119 ceph-mon[32752]: pgmap v8208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14082.194355 secs 2024-04-26T22:30:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:01 smithi167 ceph-mon[32751]: pgmap v8208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14082.194355 secs 2024-04-26T22:30:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14082.944466 secs 2024-04-26T22:30:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14082.944466 secs 2024-04-26T22:30:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14082.944466 secs 2024-04-26T22:30:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:03 smithi090 ceph-mon[29180]: pgmap v8209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:03 smithi119 ceph-mon[32752]: pgmap v8209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:03 smithi167 ceph-mon[32751]: pgmap v8209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:05 smithi090 ceph-mon[29180]: pgmap v8210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:05 smithi119 ceph-mon[32752]: pgmap v8210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:05 smithi167 ceph-mon[32751]: pgmap v8210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14087.194441 secs 2024-04-26T22:30:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14087.194441 secs 2024-04-26T22:30:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14087.194441 secs 2024-04-26T22:30:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14087.944596 secs 2024-04-26T22:30:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:07 smithi090 ceph-mon[29180]: pgmap v8211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:07.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14087.944596 secs 2024-04-26T22:30:07.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:07 smithi167 ceph-mon[32751]: pgmap v8211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14087.944596 secs 2024-04-26T22:30:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:07 smithi119 ceph-mon[32752]: pgmap v8211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:08 smithi119 ceph-mon[32752]: pgmap v8212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:08 smithi167 ceph-mon[32751]: pgmap v8212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:08 smithi090 ceph-mon[29180]: pgmap v8212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:10 smithi119 ceph-mon[32752]: pgmap v8213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:10 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14092.194523 secs 2024-04-26T22:30:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:10 smithi167 ceph-mon[32751]: pgmap v8213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:10 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14092.194523 secs 2024-04-26T22:30:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:10 smithi090 ceph-mon[29180]: pgmap v8213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:10 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14092.194523 secs 2024-04-26T22:30:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14092.944721 secs 2024-04-26T22:30:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14092.944721 secs 2024-04-26T22:30:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14092.944721 secs 2024-04-26T22:30:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:12 smithi119 ceph-mon[32752]: pgmap v8214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:12 smithi167 ceph-mon[32751]: pgmap v8214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:12 smithi090 ceph-mon[29180]: pgmap v8214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:14 smithi119 ceph-mon[32752]: pgmap v8215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:14 smithi167 ceph-mon[32751]: pgmap v8215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:14 smithi090 ceph-mon[29180]: pgmap v8215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:15 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14097.194609 secs 2024-04-26T22:30:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:15 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14097.194609 secs 2024-04-26T22:30:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:15 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14097.194609 secs 2024-04-26T22:30:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14097.944838 secs 2024-04-26T22:30:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:16 smithi119 ceph-mon[32752]: pgmap v8216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14097.944838 secs 2024-04-26T22:30:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:16 smithi167 ceph-mon[32751]: pgmap v8216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14097.944838 secs 2024-04-26T22:30:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:16 smithi090 ceph-mon[29180]: pgmap v8216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:18 smithi119 ceph-mon[32752]: pgmap v8217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:18 smithi167 ceph-mon[32751]: pgmap v8217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:18 smithi090 ceph-mon[29180]: pgmap v8217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:20 smithi119 ceph-mon[32752]: pgmap v8218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:20 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14102.194693 secs 2024-04-26T22:30:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:20 smithi167 ceph-mon[32751]: pgmap v8218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:20 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14102.194693 secs 2024-04-26T22:30:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:20 smithi090 ceph-mon[29180]: pgmap v8218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:20 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14102.194693 secs 2024-04-26T22:30:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14102.944937 secs 2024-04-26T22:30:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14102.944937 secs 2024-04-26T22:30:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14102.944937 secs 2024-04-26T22:30:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:22 smithi119 ceph-mon[32752]: pgmap v8219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:22 smithi167 ceph-mon[32751]: pgmap v8219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:22 smithi090 ceph-mon[29180]: pgmap v8219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:24 smithi119 ceph-mon[32752]: pgmap v8220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:24 smithi167 ceph-mon[32751]: pgmap v8220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:24 smithi090 ceph-mon[29180]: pgmap v8220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:25 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14107.194748 secs 2024-04-26T22:30:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:25 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14107.194748 secs 2024-04-26T22:30:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:25 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14107.194748 secs 2024-04-26T22:30:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14107.945030 secs 2024-04-26T22:30:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:26 smithi119 ceph-mon[32752]: pgmap v8221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14107.945030 secs 2024-04-26T22:30:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:26 smithi167 ceph-mon[32751]: pgmap v8221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14107.945030 secs 2024-04-26T22:30:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:26 smithi090 ceph-mon[29180]: pgmap v8221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:28 smithi119 ceph-mon[32752]: pgmap v8222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:28 smithi167 ceph-mon[32751]: pgmap v8222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:28 smithi090 ceph-mon[29180]: pgmap v8222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:30 smithi119 ceph-mon[32752]: pgmap v8223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:30 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14112.194800 secs 2024-04-26T22:30:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:30 smithi167 ceph-mon[32751]: pgmap v8223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:30 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14112.194800 secs 2024-04-26T22:30:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:30 smithi090 ceph-mon[29180]: pgmap v8223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:30 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14112.194800 secs 2024-04-26T22:30:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14112.945148 secs 2024-04-26T22:30:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14112.945148 secs 2024-04-26T22:30:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14112.945148 secs 2024-04-26T22:30:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:32 smithi119 ceph-mon[32752]: pgmap v8224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:32 smithi167 ceph-mon[32751]: pgmap v8224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:32 smithi090 ceph-mon[29180]: pgmap v8224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:35 smithi119 ceph-mon[32752]: pgmap v8225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:35 smithi167 ceph-mon[32751]: pgmap v8225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:34 smithi090 ceph-mon[29180]: pgmap v8225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14117.194860 secs 2024-04-26T22:30:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14117.194860 secs 2024-04-26T22:30:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14117.194860 secs 2024-04-26T22:30:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14117.945244 secs 2024-04-26T22:30:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:37 smithi119 ceph-mon[32752]: pgmap v8226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14117.945244 secs 2024-04-26T22:30:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:37 smithi167 ceph-mon[32751]: pgmap v8226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14117.945244 secs 2024-04-26T22:30:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:37 smithi090 ceph-mon[29180]: pgmap v8226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:39 smithi119 ceph-mon[32752]: pgmap v8227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:39 smithi167 ceph-mon[32751]: pgmap v8227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:39 smithi090 ceph-mon[29180]: pgmap v8227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:41 smithi119 ceph-mon[32752]: pgmap v8228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14122.194922 secs 2024-04-26T22:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:41 smithi167 ceph-mon[32751]: pgmap v8228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14122.194922 secs 2024-04-26T22:30:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:41 smithi090 ceph-mon[29180]: pgmap v8228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14122.194922 secs 2024-04-26T22:30:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14122.945403 secs 2024-04-26T22:30:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14122.945403 secs 2024-04-26T22:30:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14122.945403 secs 2024-04-26T22:30:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:43 smithi119 ceph-mon[32752]: pgmap v8229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:43 smithi167 ceph-mon[32751]: pgmap v8229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:43 smithi090 ceph-mon[29180]: pgmap v8229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:45 smithi119 ceph-mon[32752]: pgmap v8230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:45 smithi167 ceph-mon[32751]: pgmap v8230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:45 smithi090 ceph-mon[29180]: pgmap v8230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:30:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14127.195024 secs 2024-04-26T22:30:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14127.195024 secs 2024-04-26T22:30:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14127.195024 secs 2024-04-26T22:30:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14127.945448 secs 2024-04-26T22:30:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:47 smithi119 ceph-mon[32752]: pgmap v8231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14127.945448 secs 2024-04-26T22:30:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:47 smithi167 ceph-mon[32751]: pgmap v8231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14127.945448 secs 2024-04-26T22:30:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:47 smithi090 ceph-mon[29180]: pgmap v8231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:30:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:30:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:30:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:49 smithi119 ceph-mon[32752]: pgmap v8232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:30:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:30:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:30:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:49 smithi167 ceph-mon[32751]: pgmap v8232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:30:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:30:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:30:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:49 smithi090 ceph-mon[29180]: pgmap v8232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:30:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:30:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:30:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:30:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:50 smithi119 ceph-mon[32752]: pgmap v8233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:30:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:50 smithi119 ceph-mon[32752]: pgmap v8234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:50 smithi167 ceph-mon[32751]: pgmap v8233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:30:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:50 smithi167 ceph-mon[32751]: pgmap v8234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:50 smithi090 ceph-mon[29180]: pgmap v8233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:30:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:50 smithi090 ceph-mon[29180]: pgmap v8234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14132.195086 secs 2024-04-26T22:30:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14132.195086 secs 2024-04-26T22:30:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14132.195086 secs 2024-04-26T22:30:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:52 smithi119 ceph-mon[32752]: pgmap v8235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14132.945566 secs 2024-04-26T22:30:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:52 smithi167 ceph-mon[32751]: pgmap v8235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14132.945566 secs 2024-04-26T22:30:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:52 smithi090 ceph-mon[29180]: pgmap v8235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14132.945566 secs 2024-04-26T22:30:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:54 smithi119 ceph-mon[32752]: pgmap v8236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:30:54.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:54 smithi167 ceph-mon[32751]: pgmap v8236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:30:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:54 smithi090 ceph-mon[29180]: pgmap v8236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:30:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:56 smithi167 ceph-mon[32751]: pgmap v8237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14137.195169 secs 2024-04-26T22:30:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:56 smithi090 ceph-mon[29180]: pgmap v8237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14137.195169 secs 2024-04-26T22:30:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:56 smithi119 ceph-mon[32752]: pgmap v8237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14137.195169 secs 2024-04-26T22:30:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14137.945644 secs 2024-04-26T22:30:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14137.945644 secs 2024-04-26T22:30:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14137.945644 secs 2024-04-26T22:30:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:30:58 smithi167 ceph-mon[32751]: pgmap v8238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:30:58 smithi090 ceph-mon[29180]: pgmap v8238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:30:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:30:58 smithi119 ceph-mon[32752]: pgmap v8238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:00 smithi167 ceph-mon[32751]: pgmap v8239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:31:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:00 smithi090 ceph-mon[29180]: pgmap v8239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:31:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:00 smithi119 ceph-mon[32752]: pgmap v8239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:31:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14142.195232 secs 2024-04-26T22:31:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14142.195232 secs 2024-04-26T22:31:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14142.195232 secs 2024-04-26T22:31:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:02 smithi090 ceph-mon[29180]: pgmap v8240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14142.945773 secs 2024-04-26T22:31:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:02 smithi119 ceph-mon[32752]: pgmap v8240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14142.945773 secs 2024-04-26T22:31:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:02 smithi167 ceph-mon[32751]: pgmap v8240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14142.945773 secs 2024-04-26T22:31:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:04 smithi090 ceph-mon[29180]: pgmap v8241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:04 smithi119 ceph-mon[32752]: pgmap v8241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:04 smithi167 ceph-mon[32751]: pgmap v8241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:04.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:31:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:31:04.582+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:31:04.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:31:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:31:04.582+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:31:04.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:31:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:31:04.582+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:31:04.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:31:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:31:04.582+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:31:04.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:31:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:31:04.582+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:31:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:06 smithi090 ceph-mon[29180]: pgmap v8242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:06 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 14147.195317 secs 2024-04-26T22:31:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:06 smithi090 ceph-mon[29180]: slow request 5761.957261 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:31:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:06 smithi119 ceph-mon[32752]: pgmap v8242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:06 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 14147.195317 secs 2024-04-26T22:31:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:06 smithi119 ceph-mon[32752]: slow request 5761.957261 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:31:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:06 smithi167 ceph-mon[32751]: pgmap v8242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:06 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 14147.195317 secs 2024-04-26T22:31:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:06 smithi167 ceph-mon[32751]: slow request 5761.957261 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:31:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14147.945884 secs 2024-04-26T22:31:07.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14147.945884 secs 2024-04-26T22:31:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14147.945884 secs 2024-04-26T22:31:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:08 smithi090 ceph-mon[29180]: pgmap v8243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:08 smithi119 ceph-mon[32752]: pgmap v8243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:08 smithi167 ceph-mon[32751]: pgmap v8243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:10 smithi090 ceph-mon[29180]: pgmap v8244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:10 smithi119 ceph-mon[32752]: pgmap v8244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:10.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:10 smithi167 ceph-mon[32751]: pgmap v8244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14152.195377 secs 2024-04-26T22:31:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14152.195377 secs 2024-04-26T22:31:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14152.195377 secs 2024-04-26T22:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:12 smithi090 ceph-mon[29180]: pgmap v8245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14152.945993 secs 2024-04-26T22:31:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:12 smithi119 ceph-mon[32752]: pgmap v8245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14152.945993 secs 2024-04-26T22:31:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:12 smithi167 ceph-mon[32751]: pgmap v8245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14152.945993 secs 2024-04-26T22:31:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:14 smithi090 ceph-mon[29180]: pgmap v8246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:14 smithi119 ceph-mon[32752]: pgmap v8246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:14 smithi167 ceph-mon[32751]: pgmap v8246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:16 smithi090 ceph-mon[29180]: pgmap v8247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14157.195463 secs 2024-04-26T22:31:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:16 smithi119 ceph-mon[32752]: pgmap v8247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14157.195463 secs 2024-04-26T22:31:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:16 smithi167 ceph-mon[32751]: pgmap v8247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14157.195463 secs 2024-04-26T22:31:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14157.946141 secs 2024-04-26T22:31:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14157.946141 secs 2024-04-26T22:31:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14157.946141 secs 2024-04-26T22:31:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:18 smithi090 ceph-mon[29180]: pgmap v8248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:18 smithi119 ceph-mon[32752]: pgmap v8248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:18 smithi167 ceph-mon[32751]: pgmap v8248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:20 smithi090 ceph-mon[29180]: pgmap v8249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:20 smithi119 ceph-mon[32752]: pgmap v8249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:20 smithi167 ceph-mon[32751]: pgmap v8249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:31:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14162.195547 secs 2024-04-26T22:31:21.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14162.195547 secs 2024-04-26T22:31:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14162.195547 secs 2024-04-26T22:31:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:22 smithi090 ceph-mon[29180]: pgmap v8250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14162.946278 secs 2024-04-26T22:31:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:22 smithi119 ceph-mon[32752]: pgmap v8250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14162.946278 secs 2024-04-26T22:31:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:22 smithi167 ceph-mon[32751]: pgmap v8250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14162.946278 secs 2024-04-26T22:31:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:24 smithi090 ceph-mon[29180]: pgmap v8251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:24 smithi119 ceph-mon[32752]: pgmap v8251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:24 smithi167 ceph-mon[32751]: pgmap v8251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:26 smithi119 ceph-mon[32752]: pgmap v8252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14167.195631 secs 2024-04-26T22:31:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:26 smithi167 ceph-mon[32751]: pgmap v8252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14167.195631 secs 2024-04-26T22:31:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:26 smithi090 ceph-mon[29180]: pgmap v8252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14167.195631 secs 2024-04-26T22:31:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14167.946409 secs 2024-04-26T22:31:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14167.946409 secs 2024-04-26T22:31:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14167.946409 secs 2024-04-26T22:31:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:28 smithi119 ceph-mon[32752]: pgmap v8253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:28 smithi167 ceph-mon[32751]: pgmap v8253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:28 smithi090 ceph-mon[29180]: pgmap v8253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:31:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:30 smithi119 ceph-mon[32752]: pgmap v8254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:30 smithi167 ceph-mon[32751]: pgmap v8254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:30 smithi090 ceph-mon[29180]: pgmap v8254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14172.195713 secs 2024-04-26T22:31:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14172.195713 secs 2024-04-26T22:31:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14172.195713 secs 2024-04-26T22:31:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:32 smithi119 ceph-mon[32752]: pgmap v8255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14172.946519 secs 2024-04-26T22:31:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:32 smithi167 ceph-mon[32751]: pgmap v8255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14172.946519 secs 2024-04-26T22:31:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:32 smithi090 ceph-mon[29180]: pgmap v8255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14172.946519 secs 2024-04-26T22:31:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:34 smithi119 ceph-mon[32752]: pgmap v8256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:34 smithi167 ceph-mon[32751]: pgmap v8256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:34 smithi090 ceph-mon[29180]: pgmap v8256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:36 smithi119 ceph-mon[32752]: pgmap v8257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14177.195774 secs 2024-04-26T22:31:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:36 smithi167 ceph-mon[32751]: pgmap v8257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14177.195774 secs 2024-04-26T22:31:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:36 smithi090 ceph-mon[29180]: pgmap v8257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14177.195774 secs 2024-04-26T22:31:37.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14177.946617 secs 2024-04-26T22:31:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14177.946617 secs 2024-04-26T22:31:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14177.946617 secs 2024-04-26T22:31:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:38 smithi119 ceph-mon[32752]: pgmap v8258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:38 smithi167 ceph-mon[32751]: pgmap v8258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:38 smithi090 ceph-mon[29180]: pgmap v8258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:40 smithi119 ceph-mon[32752]: pgmap v8259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:40 smithi167 ceph-mon[32751]: pgmap v8259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:40 smithi090 ceph-mon[29180]: pgmap v8259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14182.195862 secs 2024-04-26T22:31:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14182.195862 secs 2024-04-26T22:31:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14182.195862 secs 2024-04-26T22:31:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:42 smithi119 ceph-mon[32752]: pgmap v8260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14182.946755 secs 2024-04-26T22:31:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:42 smithi167 ceph-mon[32751]: pgmap v8260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14182.946755 secs 2024-04-26T22:31:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:42 smithi090 ceph-mon[29180]: pgmap v8260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14182.946755 secs 2024-04-26T22:31:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:44 smithi119 ceph-mon[32752]: pgmap v8261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:44 smithi167 ceph-mon[32751]: pgmap v8261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:44 smithi090 ceph-mon[29180]: pgmap v8261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:46 smithi119 ceph-mon[32752]: pgmap v8262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14187.195953 secs 2024-04-26T22:31:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:46 smithi167 ceph-mon[32751]: pgmap v8262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14187.195953 secs 2024-04-26T22:31:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:46 smithi090 ceph-mon[29180]: pgmap v8262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14187.195953 secs 2024-04-26T22:31:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14187.946877 secs 2024-04-26T22:31:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14187.946877 secs 2024-04-26T22:31:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14187.946877 secs 2024-04-26T22:31:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:48 smithi119 ceph-mon[32752]: pgmap v8263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:48 smithi167 ceph-mon[32751]: pgmap v8263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:48 smithi090 ceph-mon[29180]: pgmap v8263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:31:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:31:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:31:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:31:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:50 smithi119 ceph-mon[32752]: pgmap v8264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:31:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:31:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:31:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:50 smithi167 ceph-mon[32751]: pgmap v8264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:50.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:31:50.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:31:50.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:31:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:50 smithi090 ceph-mon[29180]: pgmap v8264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:31:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:31:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:31:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:31:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:51 smithi119 ceph-mon[32752]: pgmap v8265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:31:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:51 smithi119 ceph-mon[32752]: pgmap v8266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:51.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14192.196013 secs 2024-04-26T22:31:51.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:51 smithi090 ceph-mon[29180]: pgmap v8265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:31:51.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:51 smithi090 ceph-mon[29180]: pgmap v8266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:51.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14192.196013 secs 2024-04-26T22:31:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:51 smithi167 ceph-mon[32751]: pgmap v8265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:31:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:51 smithi167 ceph-mon[32751]: pgmap v8266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14192.196013 secs 2024-04-26T22:31:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14192.946994 secs 2024-04-26T22:31:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14192.946994 secs 2024-04-26T22:31:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14192.946994 secs 2024-04-26T22:31:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:53 smithi119 ceph-mon[32752]: pgmap v8267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:53 smithi167 ceph-mon[32751]: pgmap v8267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:53 smithi090 ceph-mon[29180]: pgmap v8267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:55 smithi119 ceph-mon[32752]: pgmap v8268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:55 smithi167 ceph-mon[32751]: pgmap v8268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:55 smithi090 ceph-mon[29180]: pgmap v8268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14197.196116 secs 2024-04-26T22:31:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14197.196116 secs 2024-04-26T22:31:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14197.196116 secs 2024-04-26T22:31:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:57 smithi119 ceph-mon[32752]: pgmap v8269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14197.947101 secs 2024-04-26T22:31:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:57 smithi167 ceph-mon[32751]: pgmap v8269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14197.947101 secs 2024-04-26T22:31:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:57 smithi090 ceph-mon[29180]: pgmap v8269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:31:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14197.947101 secs 2024-04-26T22:31:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:31:59 smithi119 ceph-mon[32752]: pgmap v8270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:31:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:31:59 smithi167 ceph-mon[32751]: pgmap v8270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:31:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:31:59 smithi090 ceph-mon[29180]: pgmap v8270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:32:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:01 smithi119 ceph-mon[32752]: pgmap v8271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:32:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14202.196203 secs 2024-04-26T22:32:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:01 smithi167 ceph-mon[32751]: pgmap v8271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:32:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14202.196203 secs 2024-04-26T22:32:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:01 smithi090 ceph-mon[29180]: pgmap v8271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:32:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14202.196203 secs 2024-04-26T22:32:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14202.947204 secs 2024-04-26T22:32:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14202.947204 secs 2024-04-26T22:32:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14202.947204 secs 2024-04-26T22:32:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:03 smithi119 ceph-mon[32752]: pgmap v8272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:03 smithi167 ceph-mon[32751]: pgmap v8272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:03 smithi090 ceph-mon[29180]: pgmap v8272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:05 smithi119 ceph-mon[32752]: pgmap v8273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:05 smithi167 ceph-mon[32751]: pgmap v8273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:05 smithi090 ceph-mon[29180]: pgmap v8273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14207.196298 secs 2024-04-26T22:32:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14207.196298 secs 2024-04-26T22:32:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14207.196298 secs 2024-04-26T22:32:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:07 smithi119 ceph-mon[32752]: pgmap v8274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14207.947322 secs 2024-04-26T22:32:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:07 smithi167 ceph-mon[32751]: pgmap v8274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14207.947322 secs 2024-04-26T22:32:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:07 smithi090 ceph-mon[29180]: pgmap v8274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14207.947322 secs 2024-04-26T22:32:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:09 smithi119 ceph-mon[32752]: pgmap v8275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:09 smithi167 ceph-mon[32751]: pgmap v8275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:09 smithi090 ceph-mon[29180]: pgmap v8275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:11 smithi119 ceph-mon[32752]: pgmap v8276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14212.196381 secs 2024-04-26T22:32:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:11 smithi167 ceph-mon[32751]: pgmap v8276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14212.196381 secs 2024-04-26T22:32:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:11 smithi090 ceph-mon[29180]: pgmap v8276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14212.196381 secs 2024-04-26T22:32:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14212.947439 secs 2024-04-26T22:32:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14212.947439 secs 2024-04-26T22:32:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14212.947439 secs 2024-04-26T22:32:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:13 smithi119 ceph-mon[32752]: pgmap v8277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:13 smithi167 ceph-mon[32751]: pgmap v8277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:13 smithi090 ceph-mon[29180]: pgmap v8277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:15 smithi119 ceph-mon[32752]: pgmap v8278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:15 smithi167 ceph-mon[32751]: pgmap v8278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:15 smithi090 ceph-mon[29180]: pgmap v8278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14217.196464 secs 2024-04-26T22:32:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14217.196464 secs 2024-04-26T22:32:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14217.196464 secs 2024-04-26T22:32:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:17 smithi119 ceph-mon[32752]: pgmap v8279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14217.947478 secs 2024-04-26T22:32:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:17 smithi167 ceph-mon[32751]: pgmap v8279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14217.947478 secs 2024-04-26T22:32:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:17 smithi090 ceph-mon[29180]: pgmap v8279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14217.947478 secs 2024-04-26T22:32:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:19 smithi119 ceph-mon[32752]: pgmap v8280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:19 smithi167 ceph-mon[32751]: pgmap v8280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:19 smithi090 ceph-mon[29180]: pgmap v8280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:21 smithi119 ceph-mon[32752]: pgmap v8281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14222.196546 secs 2024-04-26T22:32:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:21 smithi167 ceph-mon[32751]: pgmap v8281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14222.196546 secs 2024-04-26T22:32:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:21 smithi090 ceph-mon[29180]: pgmap v8281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14222.196546 secs 2024-04-26T22:32:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14222.947580 secs 2024-04-26T22:32:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14222.947580 secs 2024-04-26T22:32:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14222.947580 secs 2024-04-26T22:32:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:23 smithi119 ceph-mon[32752]: pgmap v8282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:23 smithi167 ceph-mon[32751]: pgmap v8282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:23 smithi090 ceph-mon[29180]: pgmap v8282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:25 smithi119 ceph-mon[32752]: pgmap v8283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:25 smithi167 ceph-mon[32751]: pgmap v8283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:25 smithi090 ceph-mon[29180]: pgmap v8283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14227.196632 secs 2024-04-26T22:32:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14227.196632 secs 2024-04-26T22:32:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14227.196632 secs 2024-04-26T22:32:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:27 smithi119 ceph-mon[32752]: pgmap v8284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14227.947695 secs 2024-04-26T22:32:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:27 smithi167 ceph-mon[32751]: pgmap v8284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14227.947695 secs 2024-04-26T22:32:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:27 smithi090 ceph-mon[29180]: pgmap v8284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14227.947695 secs 2024-04-26T22:32:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:29 smithi119 ceph-mon[32752]: pgmap v8285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:29 smithi167 ceph-mon[32751]: pgmap v8285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:29 smithi090 ceph-mon[29180]: pgmap v8285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:31 smithi119 ceph-mon[32752]: pgmap v8286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14232.196692 secs 2024-04-26T22:32:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:31 smithi167 ceph-mon[32751]: pgmap v8286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14232.196692 secs 2024-04-26T22:32:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:31 smithi090 ceph-mon[29180]: pgmap v8286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14232.196692 secs 2024-04-26T22:32:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14232.947815 secs 2024-04-26T22:32:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14232.947815 secs 2024-04-26T22:32:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14232.947815 secs 2024-04-26T22:32:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:33 smithi167 ceph-mon[32751]: pgmap v8287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:33 smithi090 ceph-mon[29180]: pgmap v8287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:33 smithi119 ceph-mon[32752]: pgmap v8287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:35 smithi167 ceph-mon[32751]: pgmap v8288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:35 smithi090 ceph-mon[29180]: pgmap v8288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:35 smithi119 ceph-mon[32752]: pgmap v8288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:36 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 14237.196806 secs 2024-04-26T22:32:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:36 smithi167 ceph-mon[32751]: slow request 720.530940 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:32:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:36 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 14237.196806 secs 2024-04-26T22:32:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:36 smithi090 ceph-mon[29180]: slow request 720.530940 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:32:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:36 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 14237.196806 secs 2024-04-26T22:32:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:36 smithi119 ceph-mon[32752]: slow request 720.530940 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:32:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:37 smithi167 ceph-mon[32751]: pgmap v8289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14237.947892 secs 2024-04-26T22:32:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:37 smithi090 ceph-mon[29180]: pgmap v8289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14237.947892 secs 2024-04-26T22:32:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:37 smithi119 ceph-mon[32752]: pgmap v8289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14237.947892 secs 2024-04-26T22:32:39.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:39 smithi167 ceph-mon[32751]: pgmap v8290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:39 smithi090 ceph-mon[29180]: pgmap v8290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:39 smithi119 ceph-mon[32752]: pgmap v8290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:41 smithi090 ceph-mon[29180]: pgmap v8291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14242.196890 secs 2024-04-26T22:32:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:41 smithi119 ceph-mon[32752]: pgmap v8291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:41.821 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14242.196890 secs 2024-04-26T22:32:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:41 smithi167 ceph-mon[32751]: pgmap v8291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14242.196890 secs 2024-04-26T22:32:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14242.948014 secs 2024-04-26T22:32:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14242.948014 secs 2024-04-26T22:32:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14242.948014 secs 2024-04-26T22:32:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:43 smithi090 ceph-mon[29180]: pgmap v8292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:43 smithi119 ceph-mon[32752]: pgmap v8292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:43 smithi167 ceph-mon[32751]: pgmap v8292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:45 smithi090 ceph-mon[29180]: pgmap v8293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:45 smithi119 ceph-mon[32752]: pgmap v8293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:45 smithi167 ceph-mon[32751]: pgmap v8293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14247.196952 secs 2024-04-26T22:32:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14247.196952 secs 2024-04-26T22:32:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14247.196952 secs 2024-04-26T22:32:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:47 smithi090 ceph-mon[29180]: pgmap v8294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14247.948148 secs 2024-04-26T22:32:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:47 smithi119 ceph-mon[32752]: pgmap v8294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14247.948148 secs 2024-04-26T22:32:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:47 smithi167 ceph-mon[32751]: pgmap v8294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14247.948148 secs 2024-04-26T22:32:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:49 smithi090 ceph-mon[29180]: pgmap v8295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:49 smithi119 ceph-mon[32752]: pgmap v8295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:49 smithi167 ceph-mon[32751]: pgmap v8295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:32:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:32:50.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:32:50.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:32:50.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:32:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:32:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:32:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:32:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:32:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:32:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:32:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:32:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:32:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:51 smithi090 ceph-mon[29180]: pgmap v8296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14252.197023 secs 2024-04-26T22:32:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:51 smithi090 ceph-mon[29180]: pgmap v8297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:32:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:51 smithi090 ceph-mon[29180]: pgmap v8298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:51.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:51 smithi119 ceph-mon[32752]: pgmap v8296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:51.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14252.197023 secs 2024-04-26T22:32:51.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:51 smithi119 ceph-mon[32752]: pgmap v8297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:32:51.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:51 smithi119 ceph-mon[32752]: pgmap v8298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:51 smithi167 ceph-mon[32751]: pgmap v8296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:32:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14252.197023 secs 2024-04-26T22:32:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:51 smithi167 ceph-mon[32751]: pgmap v8297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:32:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:51 smithi167 ceph-mon[32751]: pgmap v8298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14252.948261 secs 2024-04-26T22:32:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14252.948261 secs 2024-04-26T22:32:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14252.948261 secs 2024-04-26T22:32:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:53 smithi090 ceph-mon[29180]: pgmap v8299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:32:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:53 smithi119 ceph-mon[32752]: pgmap v8299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:32:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:53 smithi167 ceph-mon[32751]: pgmap v8299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:32:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:55 smithi090 ceph-mon[29180]: pgmap v8300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:55 smithi119 ceph-mon[32752]: pgmap v8300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:55 smithi167 ceph-mon[32751]: pgmap v8300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14257.197106 secs 2024-04-26T22:32:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14257.197106 secs 2024-04-26T22:32:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14257.197106 secs 2024-04-26T22:32:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14257.948387 secs 2024-04-26T22:32:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:57 smithi090 ceph-mon[29180]: pgmap v8301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14257.948387 secs 2024-04-26T22:32:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:57 smithi119 ceph-mon[32752]: pgmap v8301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14257.948387 secs 2024-04-26T22:32:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:57 smithi167 ceph-mon[32751]: pgmap v8301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:32:59 smithi119 ceph-mon[32752]: pgmap v8302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:32:59 smithi167 ceph-mon[32751]: pgmap v8302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:32:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:32:59 smithi090 ceph-mon[29180]: pgmap v8302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:33:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14262.197190 secs 2024-04-26T22:33:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:01 smithi119 ceph-mon[32752]: pgmap v8303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:33:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14262.197190 secs 2024-04-26T22:33:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:01 smithi167 ceph-mon[32751]: pgmap v8303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:33:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14262.197190 secs 2024-04-26T22:33:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:01 smithi090 ceph-mon[29180]: pgmap v8303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:33:02.693 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14262.948479 secs 2024-04-26T22:33:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14262.948479 secs 2024-04-26T22:33:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14262.948479 secs 2024-04-26T22:33:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:03 smithi119 ceph-mon[32752]: pgmap v8304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:03 smithi167 ceph-mon[32751]: pgmap v8304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:03 smithi090 ceph-mon[29180]: pgmap v8304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:04 smithi119 ceph-mon[32752]: pgmap v8305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:04 smithi167 ceph-mon[32751]: pgmap v8305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:04 smithi090 ceph-mon[29180]: pgmap v8305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:05 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14267.197273 secs 2024-04-26T22:33:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:05 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14267.197273 secs 2024-04-26T22:33:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:05 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14267.197273 secs 2024-04-26T22:33:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14267.948591 secs 2024-04-26T22:33:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:06 smithi119 ceph-mon[32752]: pgmap v8306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14267.948591 secs 2024-04-26T22:33:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:06 smithi167 ceph-mon[32751]: pgmap v8306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14267.948591 secs 2024-04-26T22:33:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:06 smithi090 ceph-mon[29180]: pgmap v8306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:09 smithi119 ceph-mon[32752]: pgmap v8307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:09 smithi167 ceph-mon[32751]: pgmap v8307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:09 smithi090 ceph-mon[29180]: pgmap v8307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14272.197334 secs 2024-04-26T22:33:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:11 smithi119 ceph-mon[32752]: pgmap v8308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14272.197334 secs 2024-04-26T22:33:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:11 smithi167 ceph-mon[32751]: pgmap v8308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14272.197334 secs 2024-04-26T22:33:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:11 smithi090 ceph-mon[29180]: pgmap v8308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14272.948675 secs 2024-04-26T22:33:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14272.948675 secs 2024-04-26T22:33:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14272.948675 secs 2024-04-26T22:33:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:13 smithi119 ceph-mon[32752]: pgmap v8309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:13 smithi167 ceph-mon[32751]: pgmap v8309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:13 smithi090 ceph-mon[29180]: pgmap v8309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:15 smithi119 ceph-mon[32752]: pgmap v8310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:15 smithi167 ceph-mon[32751]: pgmap v8310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:15 smithi090 ceph-mon[29180]: pgmap v8310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14277.197418 secs 2024-04-26T22:33:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14277.197418 secs 2024-04-26T22:33:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14277.197418 secs 2024-04-26T22:33:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14277.948794 secs 2024-04-26T22:33:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:17 smithi167 ceph-mon[32751]: pgmap v8311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14277.948794 secs 2024-04-26T22:33:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:17 smithi090 ceph-mon[29180]: pgmap v8311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14277.948794 secs 2024-04-26T22:33:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:17 smithi119 ceph-mon[32752]: pgmap v8311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:19 smithi167 ceph-mon[32751]: pgmap v8312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:19 smithi090 ceph-mon[29180]: pgmap v8312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:19 smithi119 ceph-mon[32752]: pgmap v8312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:21.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14282.197502 secs 2024-04-26T22:33:21.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:21 smithi119 ceph-mon[32752]: pgmap v8313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14282.197502 secs 2024-04-26T22:33:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:21 smithi090 ceph-mon[29180]: pgmap v8313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14282.197502 secs 2024-04-26T22:33:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:21 smithi167 ceph-mon[32751]: pgmap v8313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14282.948922 secs 2024-04-26T22:33:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14282.948922 secs 2024-04-26T22:33:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14282.948922 secs 2024-04-26T22:33:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:23 smithi090 ceph-mon[29180]: pgmap v8314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:23 smithi119 ceph-mon[32752]: pgmap v8314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:23 smithi167 ceph-mon[32751]: pgmap v8314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:25 smithi090 ceph-mon[29180]: pgmap v8315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:25 smithi119 ceph-mon[32752]: pgmap v8315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:25 smithi167 ceph-mon[32751]: pgmap v8315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14287.197585 secs 2024-04-26T22:33:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14287.197585 secs 2024-04-26T22:33:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14287.197585 secs 2024-04-26T22:33:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14287.949046 secs 2024-04-26T22:33:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:27 smithi090 ceph-mon[29180]: pgmap v8316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14287.949046 secs 2024-04-26T22:33:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:27 smithi119 ceph-mon[32752]: pgmap v8316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14287.949046 secs 2024-04-26T22:33:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:27 smithi167 ceph-mon[32751]: pgmap v8316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:29 smithi090 ceph-mon[29180]: pgmap v8317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:29 smithi119 ceph-mon[32752]: pgmap v8317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:29 smithi167 ceph-mon[32751]: pgmap v8317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14292.197675 secs 2024-04-26T22:33:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:31 smithi090 ceph-mon[29180]: pgmap v8318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14292.197675 secs 2024-04-26T22:33:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:31 smithi119 ceph-mon[32752]: pgmap v8318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14292.197675 secs 2024-04-26T22:33:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:31 smithi167 ceph-mon[32751]: pgmap v8318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14292.949138 secs 2024-04-26T22:33:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14292.949138 secs 2024-04-26T22:33:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14292.949138 secs 2024-04-26T22:33:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:33 smithi090 ceph-mon[29180]: pgmap v8319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:33 smithi119 ceph-mon[32752]: pgmap v8319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:33 smithi167 ceph-mon[32751]: pgmap v8319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:35 smithi090 ceph-mon[29180]: pgmap v8320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:35 smithi119 ceph-mon[32752]: pgmap v8320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:35 smithi167 ceph-mon[32751]: pgmap v8320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14297.197758 secs 2024-04-26T22:33:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14297.197758 secs 2024-04-26T22:33:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14297.197758 secs 2024-04-26T22:33:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14297.949254 secs 2024-04-26T22:33:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:37 smithi090 ceph-mon[29180]: pgmap v8321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14297.949254 secs 2024-04-26T22:33:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:37 smithi119 ceph-mon[32752]: pgmap v8321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14297.949254 secs 2024-04-26T22:33:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:37 smithi167 ceph-mon[32751]: pgmap v8321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:39 smithi119 ceph-mon[32752]: pgmap v8322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:39 smithi167 ceph-mon[32751]: pgmap v8322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:39 smithi090 ceph-mon[29180]: pgmap v8322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14302.197848 secs 2024-04-26T22:33:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:41 smithi119 ceph-mon[32752]: pgmap v8323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14302.197848 secs 2024-04-26T22:33:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:41 smithi167 ceph-mon[32751]: pgmap v8323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14302.197848 secs 2024-04-26T22:33:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:41 smithi090 ceph-mon[29180]: pgmap v8323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14302.949374 secs 2024-04-26T22:33:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14302.949374 secs 2024-04-26T22:33:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14302.949374 secs 2024-04-26T22:33:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:43 smithi119 ceph-mon[32752]: pgmap v8324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:43 smithi167 ceph-mon[32751]: pgmap v8324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:43 smithi090 ceph-mon[29180]: pgmap v8324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:45 smithi119 ceph-mon[32752]: pgmap v8325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:45 smithi167 ceph-mon[32751]: pgmap v8325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:45 smithi090 ceph-mon[29180]: pgmap v8325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14307.197909 secs 2024-04-26T22:33:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:46 smithi119 ceph-mon[32752]: pgmap v8326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14307.197909 secs 2024-04-26T22:33:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:46 smithi167 ceph-mon[32751]: pgmap v8326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14307.197909 secs 2024-04-26T22:33:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:46 smithi090 ceph-mon[29180]: pgmap v8326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14307.949496 secs 2024-04-26T22:33:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14307.949496 secs 2024-04-26T22:33:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14307.949496 secs 2024-04-26T22:33:48.773 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:48 smithi090 ceph-mon[29180]: pgmap v8327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:48 smithi119 ceph-mon[32752]: pgmap v8327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:48 smithi167 ceph-mon[32751]: pgmap v8327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:33:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14312.197968 secs 2024-04-26T22:33:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:51 smithi119 ceph-mon[32752]: pgmap v8328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:33:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:33:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:33:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:33:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14312.197968 secs 2024-04-26T22:33:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:51 smithi167 ceph-mon[32751]: pgmap v8328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:33:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:33:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:33:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:33:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14312.197968 secs 2024-04-26T22:33:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:51 smithi090 ceph-mon[29180]: pgmap v8328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:33:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:33:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:33:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:33:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:33:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14312.949568 secs 2024-04-26T22:33:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:52 smithi167 ceph-mon[32751]: pgmap v8329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:33:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:52 smithi167 ceph-mon[32751]: pgmap v8330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:33:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14312.949568 secs 2024-04-26T22:33:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:52 smithi090 ceph-mon[29180]: pgmap v8329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:33:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:52 smithi090 ceph-mon[29180]: pgmap v8330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:33:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14312.949568 secs 2024-04-26T22:33:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:52 smithi119 ceph-mon[32752]: pgmap v8329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:33:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:52 smithi119 ceph-mon[32752]: pgmap v8330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:33:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:54 smithi167 ceph-mon[32751]: pgmap v8331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:33:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:54 smithi090 ceph-mon[29180]: pgmap v8331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:33:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:54 smithi119 ceph-mon[32752]: pgmap v8331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:33:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14317.198028 secs 2024-04-26T22:33:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:56 smithi167 ceph-mon[32751]: pgmap v8332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:33:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14317.198028 secs 2024-04-26T22:33:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:56 smithi090 ceph-mon[29180]: pgmap v8332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:33:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14317.198028 secs 2024-04-26T22:33:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:56 smithi119 ceph-mon[32752]: pgmap v8332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:33:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14317.949686 secs 2024-04-26T22:33:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14317.949686 secs 2024-04-26T22:33:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14317.949686 secs 2024-04-26T22:33:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:33:58 smithi167 ceph-mon[32751]: pgmap v8333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:33:58.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:33:58 smithi090 ceph-mon[29180]: pgmap v8333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:33:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:33:58 smithi119 ceph-mon[32752]: pgmap v8333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:34:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:00 smithi090 ceph-mon[29180]: pgmap v8334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:00 smithi119 ceph-mon[32752]: pgmap v8334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:00 smithi167 ceph-mon[32751]: pgmap v8334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14322.198111 secs 2024-04-26T22:34:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14322.198111 secs 2024-04-26T22:34:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14322.198111 secs 2024-04-26T22:34:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14322.949808 secs 2024-04-26T22:34:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:02 smithi090 ceph-mon[29180]: pgmap v8335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:34:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14322.949808 secs 2024-04-26T22:34:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:02 smithi119 ceph-mon[32752]: pgmap v8335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:34:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14322.949808 secs 2024-04-26T22:34:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:02 smithi167 ceph-mon[32751]: pgmap v8335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:34:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:04 smithi090 ceph-mon[29180]: pgmap v8336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:04 smithi119 ceph-mon[32752]: pgmap v8336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:04 smithi167 ceph-mon[32751]: pgmap v8336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14327.198195 secs 2024-04-26T22:34:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:06 smithi090 ceph-mon[29180]: pgmap v8337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14327.198195 secs 2024-04-26T22:34:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:06 smithi119 ceph-mon[32752]: pgmap v8337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14327.198195 secs 2024-04-26T22:34:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:06 smithi167 ceph-mon[32751]: pgmap v8337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:07 smithi090 ceph-mon[29180]: 17 slow requests, 1 included below; oldest blocked for > 14327.949943 secs 2024-04-26T22:34:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:07 smithi090 ceph-mon[29180]: slow request 181.107804 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:34:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:07 smithi119 ceph-mon[32752]: 17 slow requests, 1 included below; oldest blocked for > 14327.949943 secs 2024-04-26T22:34:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:07 smithi119 ceph-mon[32752]: slow request 181.107804 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:34:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:07 smithi167 ceph-mon[32751]: 17 slow requests, 1 included below; oldest blocked for > 14327.949943 secs 2024-04-26T22:34:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:07 smithi167 ceph-mon[32751]: slow request 181.107804 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:34:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:08 smithi090 ceph-mon[29180]: pgmap v8338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T22:34:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:08 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:34:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:08 smithi119 ceph-mon[32752]: pgmap v8338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T22:34:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:08 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:34:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:08 smithi167 ceph-mon[32751]: pgmap v8338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T22:34:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:08 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:34:10.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:10 smithi167 ceph-mon[32751]: pgmap v8339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:10 smithi090 ceph-mon[29180]: pgmap v8339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:10 smithi119 ceph-mon[32752]: pgmap v8339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14332.198256 secs 2024-04-26T22:34:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14332.198256 secs 2024-04-26T22:34:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14332.198256 secs 2024-04-26T22:34:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14332.950064 secs 2024-04-26T22:34:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:12 smithi090 ceph-mon[29180]: pgmap v8340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14332.950064 secs 2024-04-26T22:34:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:12 smithi119 ceph-mon[32752]: pgmap v8340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14332.950064 secs 2024-04-26T22:34:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:12 smithi167 ceph-mon[32751]: pgmap v8340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:14 smithi090 ceph-mon[29180]: pgmap v8341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:14 smithi119 ceph-mon[32752]: pgmap v8341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:14 smithi167 ceph-mon[32751]: pgmap v8341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14337.198366 secs 2024-04-26T22:34:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:16 smithi090 ceph-mon[29180]: pgmap v8342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14337.198366 secs 2024-04-26T22:34:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:16 smithi119 ceph-mon[32752]: pgmap v8342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14337.198366 secs 2024-04-26T22:34:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:16 smithi167 ceph-mon[32751]: pgmap v8342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14337.950200 secs 2024-04-26T22:34:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14337.950200 secs 2024-04-26T22:34:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14337.950200 secs 2024-04-26T22:34:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:18 smithi090 ceph-mon[29180]: pgmap v8343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:18 smithi119 ceph-mon[32752]: pgmap v8343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:18 smithi167 ceph-mon[32751]: pgmap v8343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:20 smithi090 ceph-mon[29180]: pgmap v8344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:20 smithi119 ceph-mon[32752]: pgmap v8344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:20 smithi167 ceph-mon[32751]: pgmap v8344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14342.198447 secs 2024-04-26T22:34:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14342.198447 secs 2024-04-26T22:34:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14342.198447 secs 2024-04-26T22:34:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14342.950335 secs 2024-04-26T22:34:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:22 smithi090 ceph-mon[29180]: pgmap v8345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14342.950335 secs 2024-04-26T22:34:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:22 smithi119 ceph-mon[32752]: pgmap v8345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14342.950335 secs 2024-04-26T22:34:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:22 smithi167 ceph-mon[32751]: pgmap v8345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:24 smithi119 ceph-mon[32752]: pgmap v8346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:24 smithi167 ceph-mon[32751]: pgmap v8346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:24 smithi090 ceph-mon[29180]: pgmap v8346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14347.198535 secs 2024-04-26T22:34:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:26 smithi119 ceph-mon[32752]: pgmap v8347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14347.198535 secs 2024-04-26T22:34:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:26 smithi167 ceph-mon[32751]: pgmap v8347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14347.198535 secs 2024-04-26T22:34:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:26 smithi090 ceph-mon[29180]: pgmap v8347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14347.950500 secs 2024-04-26T22:34:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14347.950500 secs 2024-04-26T22:34:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14347.950500 secs 2024-04-26T22:34:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:28 smithi119 ceph-mon[32752]: pgmap v8348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:28 smithi167 ceph-mon[32751]: pgmap v8348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:28 smithi090 ceph-mon[29180]: pgmap v8348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:30 smithi119 ceph-mon[32752]: pgmap v8349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:30 smithi167 ceph-mon[32751]: pgmap v8349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:30 smithi090 ceph-mon[29180]: pgmap v8349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14352.198594 secs 2024-04-26T22:34:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:31 smithi119 ceph-mon[32752]: pgmap v8350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14352.198594 secs 2024-04-26T22:34:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:31 smithi167 ceph-mon[32751]: pgmap v8350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14352.198594 secs 2024-04-26T22:34:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:31 smithi090 ceph-mon[29180]: pgmap v8350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14352.950569 secs 2024-04-26T22:34:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14352.950569 secs 2024-04-26T22:34:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14352.950569 secs 2024-04-26T22:34:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:33 smithi119 ceph-mon[32752]: pgmap v8351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:33 smithi167 ceph-mon[32751]: pgmap v8351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:33 smithi090 ceph-mon[29180]: pgmap v8351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14357.198678 secs 2024-04-26T22:34:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:36 smithi090 ceph-mon[29180]: pgmap v8352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14357.198678 secs 2024-04-26T22:34:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:36 smithi119 ceph-mon[32752]: pgmap v8352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14357.198678 secs 2024-04-26T22:34:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:36 smithi167 ceph-mon[32751]: pgmap v8352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14357.950662 secs 2024-04-26T22:34:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14357.950662 secs 2024-04-26T22:34:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14357.950662 secs 2024-04-26T22:34:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:38 smithi090 ceph-mon[29180]: pgmap v8353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:38 smithi119 ceph-mon[32752]: pgmap v8353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:38 smithi167 ceph-mon[32751]: pgmap v8353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:40.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:40 smithi167 ceph-mon[32751]: pgmap v8354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:40 smithi090 ceph-mon[29180]: pgmap v8354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:40 smithi119 ceph-mon[32752]: pgmap v8354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14362.198761 secs 2024-04-26T22:34:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14362.198761 secs 2024-04-26T22:34:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14362.198761 secs 2024-04-26T22:34:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14362.950785 secs 2024-04-26T22:34:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:42 smithi090 ceph-mon[29180]: pgmap v8355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14362.950785 secs 2024-04-26T22:34:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:42 smithi119 ceph-mon[32752]: pgmap v8355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14362.950785 secs 2024-04-26T22:34:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:42 smithi167 ceph-mon[32751]: pgmap v8355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:44 smithi090 ceph-mon[29180]: pgmap v8356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:44 smithi119 ceph-mon[32752]: pgmap v8356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:44 smithi167 ceph-mon[32751]: pgmap v8356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14367.198822 secs 2024-04-26T22:34:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:46 smithi090 ceph-mon[29180]: pgmap v8357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14367.198822 secs 2024-04-26T22:34:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:46 smithi119 ceph-mon[32752]: pgmap v8357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14367.198822 secs 2024-04-26T22:34:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:46 smithi167 ceph-mon[32751]: pgmap v8357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14367.950906 secs 2024-04-26T22:34:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14367.950906 secs 2024-04-26T22:34:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14367.950906 secs 2024-04-26T22:34:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:48 smithi090 ceph-mon[29180]: pgmap v8358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:48 smithi119 ceph-mon[32752]: pgmap v8358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:48 smithi167 ceph-mon[32751]: pgmap v8358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:50 smithi090 ceph-mon[29180]: pgmap v8359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:50 smithi119 ceph-mon[32752]: pgmap v8359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:50 smithi167 ceph-mon[32751]: pgmap v8359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:34:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14372.198879 secs 2024-04-26T22:34:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:34:51.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14372.198879 secs 2024-04-26T22:34:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:34:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14372.198879 secs 2024-04-26T22:34:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:34:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14372.951030 secs 2024-04-26T22:34:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:52 smithi119 ceph-mon[32752]: pgmap v8360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:34:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:34:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:34:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14372.951030 secs 2024-04-26T22:34:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:52 smithi167 ceph-mon[32751]: pgmap v8360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:34:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:34:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:34:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14372.951030 secs 2024-04-26T22:34:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:52 smithi090 ceph-mon[29180]: pgmap v8360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:34:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:34:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:34:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:34:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:53 smithi119 ceph-mon[32752]: pgmap v8361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:34:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:53 smithi119 ceph-mon[32752]: pgmap v8362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:53 smithi167 ceph-mon[32751]: pgmap v8361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:34:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:53 smithi167 ceph-mon[32751]: pgmap v8362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:53 smithi090 ceph-mon[29180]: pgmap v8361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:34:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:53 smithi090 ceph-mon[29180]: pgmap v8362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:55 smithi119 ceph-mon[32752]: pgmap v8363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:55 smithi167 ceph-mon[32751]: pgmap v8363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:55 smithi090 ceph-mon[29180]: pgmap v8363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14377.198962 secs 2024-04-26T22:34:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14377.198962 secs 2024-04-26T22:34:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14377.198962 secs 2024-04-26T22:34:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14377.951100 secs 2024-04-26T22:34:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:57 smithi119 ceph-mon[32752]: pgmap v8364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14377.951100 secs 2024-04-26T22:34:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:57 smithi167 ceph-mon[32751]: pgmap v8364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14377.951100 secs 2024-04-26T22:34:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:57 smithi090 ceph-mon[29180]: pgmap v8364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:34:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:34:59 smithi119 ceph-mon[32752]: pgmap v8365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:34:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:34:59 smithi167 ceph-mon[32751]: pgmap v8365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:34:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:34:59 smithi090 ceph-mon[29180]: pgmap v8365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:35:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:01 smithi119 ceph-mon[32752]: pgmap v8366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14382.199042 secs 2024-04-26T22:35:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:01 smithi167 ceph-mon[32751]: pgmap v8366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14382.199042 secs 2024-04-26T22:35:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:01 smithi090 ceph-mon[29180]: pgmap v8366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14382.199042 secs 2024-04-26T22:35:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14382.951245 secs 2024-04-26T22:35:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14382.951245 secs 2024-04-26T22:35:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14382.951245 secs 2024-04-26T22:35:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:03 smithi119 ceph-mon[32752]: pgmap v8367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:35:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:03 smithi167 ceph-mon[32751]: pgmap v8367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:35:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:03 smithi090 ceph-mon[29180]: pgmap v8367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:35:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:05 smithi119 ceph-mon[32752]: pgmap v8368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:05.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:05 smithi090 ceph-mon[29180]: pgmap v8368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:05.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:05 smithi167 ceph-mon[32751]: pgmap v8368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14387.199124 secs 2024-04-26T22:35:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14387.199124 secs 2024-04-26T22:35:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14387.199124 secs 2024-04-26T22:35:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14387.951406 secs 2024-04-26T22:35:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:07 smithi119 ceph-mon[32752]: pgmap v8369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14387.951406 secs 2024-04-26T22:35:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:07 smithi167 ceph-mon[32751]: pgmap v8369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:07.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14387.951406 secs 2024-04-26T22:35:07.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:07 smithi090 ceph-mon[29180]: pgmap v8369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:09 smithi119 ceph-mon[32752]: pgmap v8370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:09 smithi167 ceph-mon[32751]: pgmap v8370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:09 smithi090 ceph-mon[29180]: pgmap v8370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:11 smithi119 ceph-mon[32752]: pgmap v8371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14392.199208 secs 2024-04-26T22:35:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:11 smithi167 ceph-mon[32751]: pgmap v8371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14392.199208 secs 2024-04-26T22:35:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:11 smithi090 ceph-mon[29180]: pgmap v8371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14392.199208 secs 2024-04-26T22:35:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14392.951473 secs 2024-04-26T22:35:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14392.951473 secs 2024-04-26T22:35:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14392.951473 secs 2024-04-26T22:35:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:13 smithi119 ceph-mon[32752]: pgmap v8372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:13 smithi167 ceph-mon[32751]: pgmap v8372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:13 smithi090 ceph-mon[29180]: pgmap v8372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:15 smithi119 ceph-mon[32752]: pgmap v8373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:15 smithi167 ceph-mon[32751]: pgmap v8373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:15 smithi090 ceph-mon[29180]: pgmap v8373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14397.199289 secs 2024-04-26T22:35:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14397.199289 secs 2024-04-26T22:35:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14397.199289 secs 2024-04-26T22:35:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14397.951591 secs 2024-04-26T22:35:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:17 smithi119 ceph-mon[32752]: pgmap v8374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14397.951591 secs 2024-04-26T22:35:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:17 smithi167 ceph-mon[32751]: pgmap v8374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14397.951591 secs 2024-04-26T22:35:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:17 smithi090 ceph-mon[29180]: pgmap v8374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:19 smithi119 ceph-mon[32752]: pgmap v8375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:19 smithi167 ceph-mon[32751]: pgmap v8375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:19 smithi090 ceph-mon[29180]: pgmap v8375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:21 smithi119 ceph-mon[32752]: pgmap v8376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14402.199373 secs 2024-04-26T22:35:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:21 smithi167 ceph-mon[32751]: pgmap v8376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14402.199373 secs 2024-04-26T22:35:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:21 smithi090 ceph-mon[29180]: pgmap v8376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14402.199373 secs 2024-04-26T22:35:22.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14402.951702 secs 2024-04-26T22:35:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14402.951702 secs 2024-04-26T22:35:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14402.951702 secs 2024-04-26T22:35:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:23 smithi119 ceph-mon[32752]: pgmap v8377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:23 smithi167 ceph-mon[32751]: pgmap v8377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:23 smithi090 ceph-mon[29180]: pgmap v8377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:25 smithi119 ceph-mon[32752]: pgmap v8378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:25 smithi167 ceph-mon[32751]: pgmap v8378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:25 smithi090 ceph-mon[29180]: pgmap v8378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:26.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14407.199452 secs 2024-04-26T22:35:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14407.199452 secs 2024-04-26T22:35:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14407.199452 secs 2024-04-26T22:35:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14407.951812 secs 2024-04-26T22:35:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:27 smithi119 ceph-mon[32752]: pgmap v8379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14407.951812 secs 2024-04-26T22:35:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:27 smithi167 ceph-mon[32751]: pgmap v8379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14407.951812 secs 2024-04-26T22:35:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:27 smithi090 ceph-mon[29180]: pgmap v8379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:29 smithi119 ceph-mon[32752]: pgmap v8380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:29 smithi167 ceph-mon[32751]: pgmap v8380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:29 smithi090 ceph-mon[29180]: pgmap v8380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:31 smithi119 ceph-mon[32752]: pgmap v8381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14412.199534 secs 2024-04-26T22:35:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:31 smithi167 ceph-mon[32751]: pgmap v8381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14412.199534 secs 2024-04-26T22:35:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:31 smithi090 ceph-mon[29180]: pgmap v8381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14412.199534 secs 2024-04-26T22:35:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14412.951892 secs 2024-04-26T22:35:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14412.951892 secs 2024-04-26T22:35:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14412.951892 secs 2024-04-26T22:35:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:33 smithi119 ceph-mon[32752]: pgmap v8382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:33 smithi167 ceph-mon[32751]: pgmap v8382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:33 smithi090 ceph-mon[29180]: pgmap v8382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:35 smithi119 ceph-mon[32752]: pgmap v8383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:35 smithi167 ceph-mon[32751]: pgmap v8383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:35 smithi090 ceph-mon[29180]: pgmap v8383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14417.199609 secs 2024-04-26T22:35:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14417.199609 secs 2024-04-26T22:35:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14417.199609 secs 2024-04-26T22:35:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14417.952010 secs 2024-04-26T22:35:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:37 smithi119 ceph-mon[32752]: pgmap v8384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14417.952010 secs 2024-04-26T22:35:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:37 smithi167 ceph-mon[32751]: pgmap v8384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14417.952010 secs 2024-04-26T22:35:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:37 smithi090 ceph-mon[29180]: pgmap v8384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:39 smithi119 ceph-mon[32752]: pgmap v8385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:39 smithi167 ceph-mon[32751]: pgmap v8385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:39 smithi090 ceph-mon[29180]: pgmap v8385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:41 smithi119 ceph-mon[32752]: pgmap v8386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14422.199672 secs 2024-04-26T22:35:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:41 smithi167 ceph-mon[32751]: pgmap v8386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14422.199672 secs 2024-04-26T22:35:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:41 smithi090 ceph-mon[29180]: pgmap v8386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14422.199672 secs 2024-04-26T22:35:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14422.952150 secs 2024-04-26T22:35:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14422.952150 secs 2024-04-26T22:35:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14422.952150 secs 2024-04-26T22:35:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:43 smithi119 ceph-mon[32752]: pgmap v8387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:43 smithi167 ceph-mon[32751]: pgmap v8387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:43 smithi090 ceph-mon[29180]: pgmap v8387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:45 smithi119 ceph-mon[32752]: pgmap v8388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:45 smithi167 ceph-mon[32751]: pgmap v8388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:45 smithi090 ceph-mon[29180]: pgmap v8388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14427.199757 secs 2024-04-26T22:35:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14427.199757 secs 2024-04-26T22:35:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14427.199757 secs 2024-04-26T22:35:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14427.952266 secs 2024-04-26T22:35:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:47 smithi119 ceph-mon[32752]: pgmap v8389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14427.952266 secs 2024-04-26T22:35:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:47 smithi167 ceph-mon[32751]: pgmap v8389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14427.952266 secs 2024-04-26T22:35:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:47 smithi090 ceph-mon[29180]: pgmap v8389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:49 smithi119 ceph-mon[32752]: pgmap v8390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:49 smithi167 ceph-mon[32751]: pgmap v8390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:49 smithi090 ceph-mon[29180]: pgmap v8390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:51 smithi119 ceph-mon[32752]: pgmap v8391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14432.199840 secs 2024-04-26T22:35:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:51 smithi167 ceph-mon[32751]: pgmap v8391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14432.199840 secs 2024-04-26T22:35:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:51 smithi090 ceph-mon[29180]: pgmap v8391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:35:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14432.199840 secs 2024-04-26T22:35:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14432.952398 secs 2024-04-26T22:35:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:35:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14432.952398 secs 2024-04-26T22:35:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:35:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14432.952398 secs 2024-04-26T22:35:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:35:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:53 smithi119 ceph-mon[32752]: pgmap v8392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:35:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:35:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:35:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:53 smithi167 ceph-mon[32751]: pgmap v8392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:35:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:35:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:35:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:53 smithi090 ceph-mon[29180]: pgmap v8392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:35:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:35:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:35:53.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:35:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:54 smithi119 ceph-mon[32752]: pgmap v8393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:35:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:54 smithi119 ceph-mon[32752]: pgmap v8394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:35:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:54 smithi167 ceph-mon[32751]: pgmap v8393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:35:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:54 smithi167 ceph-mon[32751]: pgmap v8394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:35:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:54 smithi090 ceph-mon[29180]: pgmap v8393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:35:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:54 smithi090 ceph-mon[29180]: pgmap v8394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:35:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:56 smithi119 ceph-mon[32752]: pgmap v8395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14437.199921 secs 2024-04-26T22:35:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:56 smithi167 ceph-mon[32751]: pgmap v8395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14437.199921 secs 2024-04-26T22:35:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:56 smithi090 ceph-mon[29180]: pgmap v8395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14437.199921 secs 2024-04-26T22:35:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14437.952523 secs 2024-04-26T22:35:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14437.952523 secs 2024-04-26T22:35:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14437.952523 secs 2024-04-26T22:35:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:35:58 smithi167 ceph-mon[32751]: pgmap v8396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:35:58 smithi090 ceph-mon[29180]: pgmap v8396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:35:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:35:58 smithi119 ceph-mon[32752]: pgmap v8396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:00 smithi167 ceph-mon[32751]: pgmap v8397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:00 smithi090 ceph-mon[29180]: pgmap v8397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:00 smithi119 ceph-mon[32752]: pgmap v8397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14442.199998 secs 2024-04-26T22:36:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14442.199998 secs 2024-04-26T22:36:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14442.199998 secs 2024-04-26T22:36:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:02 smithi090 ceph-mon[29180]: pgmap v8398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14442.952640 secs 2024-04-26T22:36:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:02 smithi119 ceph-mon[32752]: pgmap v8398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14442.952640 secs 2024-04-26T22:36:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:02 smithi167 ceph-mon[32751]: pgmap v8398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14442.952640 secs 2024-04-26T22:36:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:04 smithi090 ceph-mon[29180]: pgmap v8399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:36:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:04 smithi119 ceph-mon[32752]: pgmap v8399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:36:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:04 smithi167 ceph-mon[32751]: pgmap v8399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:36:05.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:36:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:36:04.707+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:36:05.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:36:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:36:04.707+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:36:05.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:36:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:36:04.707+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:36:05.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:36:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:36:04.707+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:36:05.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:36:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:36:04.707+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:36:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:06 smithi090 ceph-mon[29180]: pgmap v8400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14447.200059 secs 2024-04-26T22:36:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:06 smithi119 ceph-mon[32752]: pgmap v8400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14447.200059 secs 2024-04-26T22:36:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:06 smithi167 ceph-mon[32751]: pgmap v8400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14447.200059 secs 2024-04-26T22:36:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:07 smithi090 ceph-mon[29180]: 17 slow requests, 1 included below; oldest blocked for > 14447.952758 secs 2024-04-26T22:36:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:07 smithi090 ceph-mon[29180]: slow request 5762.589525 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:36:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:07 smithi119 ceph-mon[32752]: 17 slow requests, 1 included below; oldest blocked for > 14447.952758 secs 2024-04-26T22:36:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:07 smithi119 ceph-mon[32752]: slow request 5762.589525 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:36:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:07 smithi167 ceph-mon[32751]: 17 slow requests, 1 included below; oldest blocked for > 14447.952758 secs 2024-04-26T22:36:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:07 smithi167 ceph-mon[32751]: slow request 5762.589525 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:36:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:08 smithi090 ceph-mon[29180]: pgmap v8401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:08 smithi119 ceph-mon[32752]: pgmap v8401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:08 smithi167 ceph-mon[32751]: pgmap v8401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:10 smithi090 ceph-mon[29180]: pgmap v8402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:10 smithi119 ceph-mon[32752]: pgmap v8402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:10 smithi167 ceph-mon[32751]: pgmap v8402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14452.200143 secs 2024-04-26T22:36:11.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14452.200143 secs 2024-04-26T22:36:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14452.200143 secs 2024-04-26T22:36:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:12 smithi090 ceph-mon[29180]: pgmap v8403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14452.952877 secs 2024-04-26T22:36:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:12 smithi119 ceph-mon[32752]: pgmap v8403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14452.952877 secs 2024-04-26T22:36:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:12 smithi167 ceph-mon[32751]: pgmap v8403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14452.952877 secs 2024-04-26T22:36:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:14 smithi090 ceph-mon[29180]: pgmap v8404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:14 smithi119 ceph-mon[32752]: pgmap v8404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:14 smithi167 ceph-mon[32751]: pgmap v8404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:16 smithi090 ceph-mon[29180]: pgmap v8405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14457.200225 secs 2024-04-26T22:36:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:16 smithi119 ceph-mon[32752]: pgmap v8405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14457.200225 secs 2024-04-26T22:36:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:16 smithi167 ceph-mon[32751]: pgmap v8405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14457.200225 secs 2024-04-26T22:36:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14457.953014 secs 2024-04-26T22:36:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14457.953014 secs 2024-04-26T22:36:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14457.953014 secs 2024-04-26T22:36:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:18 smithi090 ceph-mon[29180]: pgmap v8406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:18 smithi119 ceph-mon[32752]: pgmap v8406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:18 smithi167 ceph-mon[32751]: pgmap v8406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-26T22:36:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:20 smithi090 ceph-mon[29180]: pgmap v8407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:20 smithi119 ceph-mon[32752]: pgmap v8407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:20 smithi167 ceph-mon[32751]: pgmap v8407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T22:36:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14462.200311 secs 2024-04-26T22:36:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14462.200311 secs 2024-04-26T22:36:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14462.200311 secs 2024-04-26T22:36:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:22 smithi090 ceph-mon[29180]: pgmap v8408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14462.953136 secs 2024-04-26T22:36:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:22 smithi119 ceph-mon[32752]: pgmap v8408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14462.953136 secs 2024-04-26T22:36:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:22 smithi167 ceph-mon[32751]: pgmap v8408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14462.953136 secs 2024-04-26T22:36:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:24 smithi090 ceph-mon[29180]: pgmap v8409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:24 smithi119 ceph-mon[32752]: pgmap v8409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:24 smithi167 ceph-mon[32751]: pgmap v8409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:26 smithi119 ceph-mon[32752]: pgmap v8410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:26 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 14467.200396 secs 2024-04-26T22:36:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:26 smithi119 ceph-mon[32752]: slow request 1442.166502 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:36:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:26 smithi167 ceph-mon[32751]: pgmap v8410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:26 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 14467.200396 secs 2024-04-26T22:36:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:26 smithi167 ceph-mon[32751]: slow request 1442.166502 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:36:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:26 smithi090 ceph-mon[29180]: pgmap v8410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:26 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 14467.200396 secs 2024-04-26T22:36:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:26 smithi090 ceph-mon[29180]: slow request 1442.166502 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:36:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14467.953251 secs 2024-04-26T22:36:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14467.953251 secs 2024-04-26T22:36:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14467.953251 secs 2024-04-26T22:36:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:28 smithi119 ceph-mon[32752]: pgmap v8411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:28 smithi167 ceph-mon[32751]: pgmap v8411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:28 smithi090 ceph-mon[29180]: pgmap v8411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:30 smithi119 ceph-mon[32752]: pgmap v8412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:30 smithi167 ceph-mon[32751]: pgmap v8412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:30 smithi090 ceph-mon[29180]: pgmap v8412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14472.200477 secs 2024-04-26T22:36:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14472.200477 secs 2024-04-26T22:36:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14472.200477 secs 2024-04-26T22:36:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:32 smithi119 ceph-mon[32752]: pgmap v8413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14472.953370 secs 2024-04-26T22:36:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:32 smithi167 ceph-mon[32751]: pgmap v8413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14472.953370 secs 2024-04-26T22:36:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:32 smithi090 ceph-mon[29180]: pgmap v8413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14472.953370 secs 2024-04-26T22:36:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:34 smithi119 ceph-mon[32752]: pgmap v8414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:34 smithi167 ceph-mon[32751]: pgmap v8414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:34 smithi090 ceph-mon[29180]: pgmap v8414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:36 smithi119 ceph-mon[32752]: pgmap v8415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14477.200555 secs 2024-04-26T22:36:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:36 smithi167 ceph-mon[32751]: pgmap v8415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14477.200555 secs 2024-04-26T22:36:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:36 smithi090 ceph-mon[29180]: pgmap v8415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14477.200555 secs 2024-04-26T22:36:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14477.953484 secs 2024-04-26T22:36:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14477.953484 secs 2024-04-26T22:36:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14477.953484 secs 2024-04-26T22:36:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:38 smithi119 ceph-mon[32752]: pgmap v8416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:38 smithi167 ceph-mon[32751]: pgmap v8416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:38 smithi090 ceph-mon[29180]: pgmap v8416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:39.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:39 smithi090 ceph-mon[29180]: pgmap v8417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:39 smithi119 ceph-mon[32752]: pgmap v8417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:39 smithi167 ceph-mon[32751]: pgmap v8417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:40 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14482.200643 secs 2024-04-26T22:36:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:40 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14482.200643 secs 2024-04-26T22:36:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:40 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14482.200643 secs 2024-04-26T22:36:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:41 smithi119 ceph-mon[32752]: pgmap v8418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:41 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14482.953563 secs 2024-04-26T22:36:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:41 smithi167 ceph-mon[32751]: pgmap v8418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:41 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14482.953563 secs 2024-04-26T22:36:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:41 smithi090 ceph-mon[29180]: pgmap v8418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:41 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14482.953563 secs 2024-04-26T22:36:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:43 smithi119 ceph-mon[32752]: pgmap v8419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:43 smithi167 ceph-mon[32751]: pgmap v8419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:43 smithi090 ceph-mon[29180]: pgmap v8419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:45 smithi119 ceph-mon[32752]: pgmap v8420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:45 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14487.200726 secs 2024-04-26T22:36:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:45 smithi167 ceph-mon[32751]: pgmap v8420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:45 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14487.200726 secs 2024-04-26T22:36:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:45 smithi090 ceph-mon[29180]: pgmap v8420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:45 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14487.200726 secs 2024-04-26T22:36:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:46 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14487.953627 secs 2024-04-26T22:36:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:46 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14487.953627 secs 2024-04-26T22:36:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:46 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14487.953627 secs 2024-04-26T22:36:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:47 smithi119 ceph-mon[32752]: pgmap v8421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:47 smithi167 ceph-mon[32751]: pgmap v8421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:47 smithi090 ceph-mon[29180]: pgmap v8421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:49 smithi119 ceph-mon[32752]: pgmap v8422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:49 smithi167 ceph-mon[32751]: pgmap v8422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:49 smithi090 ceph-mon[29180]: pgmap v8422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:36:51.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:50 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14492.200810 secs 2024-04-26T22:36:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:50 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14492.200810 secs 2024-04-26T22:36:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:50 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14492.200810 secs 2024-04-26T22:36:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:51 smithi119 ceph-mon[32752]: pgmap v8423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:51 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14492.953744 secs 2024-04-26T22:36:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:51 smithi167 ceph-mon[32751]: pgmap v8423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:51 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14492.953744 secs 2024-04-26T22:36:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:51 smithi090 ceph-mon[29180]: pgmap v8423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:51 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14492.953744 secs 2024-04-26T22:36:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:36:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:36:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:36:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:54 smithi119 ceph-mon[32752]: pgmap v8424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:36:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:36:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:54 smithi119 ceph-mon[32752]: pgmap v8425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:36:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:54 smithi119 ceph-mon[32752]: pgmap v8426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:36:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:54 smithi167 ceph-mon[32751]: pgmap v8424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:36:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:36:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:54 smithi167 ceph-mon[32751]: pgmap v8425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:36:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:54 smithi167 ceph-mon[32751]: pgmap v8426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:36:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:53 smithi090 ceph-mon[29180]: pgmap v8424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:36:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:36:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:36:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:54 smithi090 ceph-mon[29180]: pgmap v8425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:36:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:54 smithi090 ceph-mon[29180]: pgmap v8426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:36:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14497.200871 secs 2024-04-26T22:36:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:56 smithi119 ceph-mon[32752]: pgmap v8427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14497.200871 secs 2024-04-26T22:36:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:56 smithi167 ceph-mon[32751]: pgmap v8427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14497.200871 secs 2024-04-26T22:36:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:56 smithi090 ceph-mon[29180]: pgmap v8427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14497.953836 secs 2024-04-26T22:36:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14497.953836 secs 2024-04-26T22:36:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14497.953836 secs 2024-04-26T22:36:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:58 smithi119 ceph-mon[32752]: pgmap v8428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:36:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:58 smithi167 ceph-mon[32751]: pgmap v8428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:36:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:58 smithi090 ceph-mon[29180]: pgmap v8428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:36:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:36:59 smithi119 ceph-mon[32752]: pgmap v8429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:59.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:36:59 smithi167 ceph-mon[32751]: pgmap v8429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:36:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:36:59 smithi090 ceph-mon[29180]: pgmap v8429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:37:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:00 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14502.200930 secs 2024-04-26T22:37:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:00 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14502.200930 secs 2024-04-26T22:37:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:00 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14502.200930 secs 2024-04-26T22:37:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:01 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14502.953956 secs 2024-04-26T22:37:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:01 smithi167 ceph-mon[32751]: pgmap v8430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:37:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:01 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14502.953956 secs 2024-04-26T22:37:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:01 smithi090 ceph-mon[29180]: pgmap v8430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:37:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:01 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14502.953956 secs 2024-04-26T22:37:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:01 smithi119 ceph-mon[32752]: pgmap v8430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:37:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:04 smithi119 ceph-mon[32752]: pgmap v8431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:04 smithi167 ceph-mon[32751]: pgmap v8431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:04 smithi090 ceph-mon[29180]: pgmap v8431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:05 smithi119 ceph-mon[32752]: pgmap v8432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:05 smithi167 ceph-mon[32751]: pgmap v8432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:05 smithi090 ceph-mon[29180]: pgmap v8432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14507.200992 secs 2024-04-26T22:37:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14507.200992 secs 2024-04-26T22:37:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14507.200992 secs 2024-04-26T22:37:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:07 smithi119 ceph-mon[32752]: 17 slow requests, 1 included below; oldest blocked for > 14507.954081 secs 2024-04-26T22:37:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:07 smithi119 ceph-mon[32752]: slow request 361.111941 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:37:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:07 smithi119 ceph-mon[32752]: pgmap v8433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:07 smithi167 ceph-mon[32751]: 17 slow requests, 1 included below; oldest blocked for > 14507.954081 secs 2024-04-26T22:37:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:07 smithi167 ceph-mon[32751]: slow request 361.111941 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:37:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:07 smithi167 ceph-mon[32751]: pgmap v8433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:07 smithi090 ceph-mon[29180]: 17 slow requests, 1 included below; oldest blocked for > 14507.954081 secs 2024-04-26T22:37:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:07 smithi090 ceph-mon[29180]: slow request 361.111941 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:37:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:07 smithi090 ceph-mon[29180]: pgmap v8433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:09 smithi119 ceph-mon[32752]: pgmap v8434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:09 smithi167 ceph-mon[32751]: pgmap v8434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:09 smithi090 ceph-mon[29180]: pgmap v8434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:10 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14512.201053 secs 2024-04-26T22:37:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:10 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14512.201053 secs 2024-04-26T22:37:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:10 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14512.201053 secs 2024-04-26T22:37:11.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:11 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14512.954205 secs 2024-04-26T22:37:11.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:11 smithi090 ceph-mon[29180]: pgmap v8435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:11 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14512.954205 secs 2024-04-26T22:37:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:11 smithi167 ceph-mon[32751]: pgmap v8435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:11 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14512.954205 secs 2024-04-26T22:37:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:11 smithi119 ceph-mon[32752]: pgmap v8435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:37:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:13 smithi119 ceph-mon[32752]: pgmap v8436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:13 smithi167 ceph-mon[32751]: pgmap v8436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:13 smithi090 ceph-mon[29180]: pgmap v8436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:37:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:15 smithi119 ceph-mon[32752]: pgmap v8437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:15 smithi167 ceph-mon[32751]: pgmap v8437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:15 smithi090 ceph-mon[29180]: pgmap v8437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14517.201138 secs 2024-04-26T22:37:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14517.201138 secs 2024-04-26T22:37:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14517.201138 secs 2024-04-26T22:37:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14517.954326 secs 2024-04-26T22:37:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:17 smithi119 ceph-mon[32752]: pgmap v8438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14517.954326 secs 2024-04-26T22:37:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:17 smithi167 ceph-mon[32751]: pgmap v8438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14517.954326 secs 2024-04-26T22:37:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:17 smithi090 ceph-mon[29180]: pgmap v8438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:19 smithi119 ceph-mon[32752]: pgmap v8439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:19 smithi167 ceph-mon[32751]: pgmap v8439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:19 smithi090 ceph-mon[29180]: pgmap v8439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:20 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14522.201220 secs 2024-04-26T22:37:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:20 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14522.201220 secs 2024-04-26T22:37:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:20 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14522.201220 secs 2024-04-26T22:37:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:21 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14522.954453 secs 2024-04-26T22:37:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:21 smithi090 ceph-mon[29180]: pgmap v8440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:21 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14522.954453 secs 2024-04-26T22:37:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:21 smithi119 ceph-mon[32752]: pgmap v8440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:21 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14522.954453 secs 2024-04-26T22:37:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:21 smithi167 ceph-mon[32751]: pgmap v8440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:23 smithi119 ceph-mon[32752]: pgmap v8441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:23 smithi167 ceph-mon[32751]: pgmap v8441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:23 smithi090 ceph-mon[29180]: pgmap v8441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:25 smithi119 ceph-mon[32752]: pgmap v8442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:25 smithi167 ceph-mon[32751]: pgmap v8442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:25 smithi090 ceph-mon[29180]: pgmap v8442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14527.201303 secs 2024-04-26T22:37:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14527.201303 secs 2024-04-26T22:37:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14527.201303 secs 2024-04-26T22:37:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14527.954572 secs 2024-04-26T22:37:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:27 smithi119 ceph-mon[32752]: pgmap v8443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14527.954572 secs 2024-04-26T22:37:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:27 smithi167 ceph-mon[32751]: pgmap v8443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14527.954572 secs 2024-04-26T22:37:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:27 smithi090 ceph-mon[29180]: pgmap v8443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:29 smithi119 ceph-mon[32752]: pgmap v8444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:29 smithi167 ceph-mon[32751]: pgmap v8444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:29 smithi090 ceph-mon[29180]: pgmap v8444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:30 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14532.201388 secs 2024-04-26T22:37:30.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:30 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14532.201388 secs 2024-04-26T22:37:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:30 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14532.201388 secs 2024-04-26T22:37:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:31 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14532.954651 secs 2024-04-26T22:37:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:31 smithi090 ceph-mon[29180]: pgmap v8445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:31 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14532.954651 secs 2024-04-26T22:37:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:31 smithi119 ceph-mon[32752]: pgmap v8445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:31 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14532.954651 secs 2024-04-26T22:37:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:31 smithi167 ceph-mon[32751]: pgmap v8445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:33 smithi119 ceph-mon[32752]: pgmap v8446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:33 smithi167 ceph-mon[32751]: pgmap v8446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:33 smithi090 ceph-mon[29180]: pgmap v8446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:35 smithi119 ceph-mon[32752]: pgmap v8447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:35 smithi167 ceph-mon[32751]: pgmap v8447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:35 smithi090 ceph-mon[29180]: pgmap v8447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14537.201469 secs 2024-04-26T22:37:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14537.201469 secs 2024-04-26T22:37:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14537.201469 secs 2024-04-26T22:37:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14537.954777 secs 2024-04-26T22:37:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:37 smithi119 ceph-mon[32752]: pgmap v8448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14537.954777 secs 2024-04-26T22:37:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:37 smithi167 ceph-mon[32751]: pgmap v8448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14537.954777 secs 2024-04-26T22:37:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:37 smithi090 ceph-mon[29180]: pgmap v8448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:39 smithi119 ceph-mon[32752]: pgmap v8449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:39 smithi167 ceph-mon[32751]: pgmap v8449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:39 smithi090 ceph-mon[29180]: pgmap v8449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:40 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14542.201529 secs 2024-04-26T22:37:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:40 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14542.201529 secs 2024-04-26T22:37:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:40 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14542.201529 secs 2024-04-26T22:37:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:41 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14542.954899 secs 2024-04-26T22:37:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:41 smithi090 ceph-mon[29180]: pgmap v8450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:41 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14542.954899 secs 2024-04-26T22:37:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:41 smithi119 ceph-mon[32752]: pgmap v8450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:41 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14542.954899 secs 2024-04-26T22:37:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:41 smithi167 ceph-mon[32751]: pgmap v8450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:43 smithi119 ceph-mon[32752]: pgmap v8451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:43 smithi167 ceph-mon[32751]: pgmap v8451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:43 smithi090 ceph-mon[29180]: pgmap v8451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:45 smithi119 ceph-mon[32752]: pgmap v8452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:45 smithi167 ceph-mon[32751]: pgmap v8452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:45 smithi090 ceph-mon[29180]: pgmap v8452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14547.201610 secs 2024-04-26T22:37:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14547.201610 secs 2024-04-26T22:37:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14547.201610 secs 2024-04-26T22:37:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14547.955029 secs 2024-04-26T22:37:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:47 smithi119 ceph-mon[32752]: pgmap v8453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14547.955029 secs 2024-04-26T22:37:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:47 smithi167 ceph-mon[32751]: pgmap v8453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14547.955029 secs 2024-04-26T22:37:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:47 smithi090 ceph-mon[29180]: pgmap v8453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:49 smithi119 ceph-mon[32752]: pgmap v8454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:49 smithi167 ceph-mon[32751]: pgmap v8454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:49 smithi090 ceph-mon[29180]: pgmap v8454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:50 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14552.201694 secs 2024-04-26T22:37:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:50 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14552.201694 secs 2024-04-26T22:37:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:50 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14552.201694 secs 2024-04-26T22:37:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:51 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14552.955110 secs 2024-04-26T22:37:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:51 smithi090 ceph-mon[29180]: pgmap v8455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:51.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:51 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14552.955110 secs 2024-04-26T22:37:51.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:51 smithi119 ceph-mon[32752]: pgmap v8455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:51 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14552.955110 secs 2024-04-26T22:37:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:51 smithi167 ceph-mon[32751]: pgmap v8455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:53 smithi119 ceph-mon[32752]: pgmap v8456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:37:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:53 smithi167 ceph-mon[32751]: pgmap v8456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:37:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:53 smithi090 ceph-mon[29180]: pgmap v8456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:37:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:37:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:55 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14557.201782 secs 2024-04-26T22:37:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:55 smithi119 ceph-mon[32752]: pgmap v8457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:55 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14557.201782 secs 2024-04-26T22:37:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:55 smithi167 ceph-mon[32751]: pgmap v8457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:55 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14557.201782 secs 2024-04-26T22:37:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:55 smithi090 ceph-mon[29180]: pgmap v8457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:37:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14557.955180 secs 2024-04-26T22:37:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:37:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:37:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14557.955180 secs 2024-04-26T22:37:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:37:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:37:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14557.955180 secs 2024-04-26T22:37:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:37:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:37:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:37:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:37:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:37:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:37:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:58 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:37:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:58 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:37:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:58 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:37:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:58 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:37:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:58 smithi119 ceph-mon[32752]: pgmap v8458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-26T22:37:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:37:58 smithi119 ceph-mon[32752]: pgmap v8459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:37:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:58 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:37:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:58 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:37:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:58 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:37:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:58 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:37:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:58 smithi167 ceph-mon[32751]: pgmap v8458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-26T22:37:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:37:58 smithi167 ceph-mon[32751]: pgmap v8459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:37:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:58 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:37:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:58 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:37:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:58 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:37:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:58 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:37:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:58 smithi090 ceph-mon[29180]: pgmap v8458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-26T22:37:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:37:58 smithi090 ceph-mon[29180]: pgmap v8459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:00 smithi167 ceph-mon[32751]: pgmap v8460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:00 smithi090 ceph-mon[29180]: pgmap v8460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:00 smithi119 ceph-mon[32752]: pgmap v8460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14562.201863 secs 2024-04-26T22:38:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14562.201863 secs 2024-04-26T22:38:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14562.201863 secs 2024-04-26T22:38:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14562.955293 secs 2024-04-26T22:38:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:02 smithi167 ceph-mon[32751]: pgmap v8461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14562.955293 secs 2024-04-26T22:38:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:02 smithi090 ceph-mon[29180]: pgmap v8461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14562.955293 secs 2024-04-26T22:38:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:02 smithi119 ceph-mon[32752]: pgmap v8461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:04 smithi167 ceph-mon[32751]: pgmap v8462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-26T22:38:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:04 smithi090 ceph-mon[29180]: pgmap v8462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-26T22:38:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:04 smithi119 ceph-mon[32752]: pgmap v8462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-26T22:38:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:06 smithi090 ceph-mon[29180]: pgmap v8463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14567.201947 secs 2024-04-26T22:38:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:06 smithi119 ceph-mon[32752]: pgmap v8463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14567.201947 secs 2024-04-26T22:38:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:06 smithi167 ceph-mon[32751]: pgmap v8463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:38:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14567.201947 secs 2024-04-26T22:38:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14567.955422 secs 2024-04-26T22:38:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14567.955422 secs 2024-04-26T22:38:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14567.955422 secs 2024-04-26T22:38:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:08 smithi090 ceph-mon[29180]: pgmap v8464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:38:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:08 smithi119 ceph-mon[32752]: pgmap v8464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:38:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:08 smithi167 ceph-mon[32751]: pgmap v8464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:38:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:10 smithi090 ceph-mon[29180]: pgmap v8465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:10 smithi119 ceph-mon[32752]: pgmap v8465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:10 smithi167 ceph-mon[32751]: pgmap v8465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14572.202030 secs 2024-04-26T22:38:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14572.202030 secs 2024-04-26T22:38:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14572.202030 secs 2024-04-26T22:38:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14572.955519 secs 2024-04-26T22:38:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:12 smithi090 ceph-mon[29180]: pgmap v8466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14572.955519 secs 2024-04-26T22:38:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:12 smithi119 ceph-mon[32752]: pgmap v8466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14572.955519 secs 2024-04-26T22:38:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:12 smithi167 ceph-mon[32751]: pgmap v8466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:14.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:14 smithi167 ceph-mon[32751]: pgmap v8467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:14 smithi090 ceph-mon[29180]: pgmap v8467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:14 smithi119 ceph-mon[32752]: pgmap v8467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:16 smithi090 ceph-mon[29180]: pgmap v8468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14577.202113 secs 2024-04-26T22:38:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:16 smithi119 ceph-mon[32752]: pgmap v8468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14577.202113 secs 2024-04-26T22:38:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:16 smithi167 ceph-mon[32751]: pgmap v8468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14577.202113 secs 2024-04-26T22:38:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14577.955641 secs 2024-04-26T22:38:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14577.955641 secs 2024-04-26T22:38:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14577.955641 secs 2024-04-26T22:38:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:18 smithi090 ceph-mon[29180]: pgmap v8469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:18 smithi119 ceph-mon[32752]: pgmap v8469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:18 smithi167 ceph-mon[32751]: pgmap v8469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:20 smithi090 ceph-mon[29180]: pgmap v8470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:20 smithi119 ceph-mon[32752]: pgmap v8470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:20 smithi167 ceph-mon[32751]: pgmap v8470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14582.202197 secs 2024-04-26T22:38:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14582.202197 secs 2024-04-26T22:38:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14582.202197 secs 2024-04-26T22:38:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14582.955769 secs 2024-04-26T22:38:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:22 smithi090 ceph-mon[29180]: pgmap v8471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14582.955769 secs 2024-04-26T22:38:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:22 smithi119 ceph-mon[32752]: pgmap v8471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:22.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14582.955769 secs 2024-04-26T22:38:22.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:22 smithi167 ceph-mon[32751]: pgmap v8471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:24 smithi090 ceph-mon[29180]: pgmap v8472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:24 smithi119 ceph-mon[32752]: pgmap v8472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:24 smithi167 ceph-mon[32751]: pgmap v8472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:26 smithi090 ceph-mon[29180]: pgmap v8473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14587.202281 secs 2024-04-26T22:38:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:26 smithi119 ceph-mon[32752]: pgmap v8473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14587.202281 secs 2024-04-26T22:38:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:26 smithi167 ceph-mon[32751]: pgmap v8473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14587.202281 secs 2024-04-26T22:38:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14587.955893 secs 2024-04-26T22:38:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14587.955893 secs 2024-04-26T22:38:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14587.955893 secs 2024-04-26T22:38:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:28 smithi119 ceph-mon[32752]: pgmap v8474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:28 smithi167 ceph-mon[32751]: pgmap v8474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:28 smithi090 ceph-mon[29180]: pgmap v8474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:30 smithi119 ceph-mon[32752]: pgmap v8475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:30 smithi167 ceph-mon[32751]: pgmap v8475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:30 smithi090 ceph-mon[29180]: pgmap v8475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14592.202364 secs 2024-04-26T22:38:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14592.202364 secs 2024-04-26T22:38:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14592.202364 secs 2024-04-26T22:38:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14592.956056 secs 2024-04-26T22:38:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:32 smithi090 ceph-mon[29180]: pgmap v8476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14592.956056 secs 2024-04-26T22:38:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:32 smithi119 ceph-mon[32752]: pgmap v8476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14592.956056 secs 2024-04-26T22:38:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:32 smithi167 ceph-mon[32751]: pgmap v8476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:34 smithi119 ceph-mon[32752]: pgmap v8477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:34 smithi167 ceph-mon[32751]: pgmap v8477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:34 smithi090 ceph-mon[29180]: pgmap v8477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:36 smithi119 ceph-mon[32752]: pgmap v8478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14597.202447 secs 2024-04-26T22:38:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:36 smithi167 ceph-mon[32751]: pgmap v8478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14597.202447 secs 2024-04-26T22:38:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:36 smithi090 ceph-mon[29180]: pgmap v8478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14597.202447 secs 2024-04-26T22:38:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14597.956195 secs 2024-04-26T22:38:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14597.956195 secs 2024-04-26T22:38:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14597.956195 secs 2024-04-26T22:38:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:38 smithi119 ceph-mon[32752]: pgmap v8479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:38 smithi167 ceph-mon[32751]: pgmap v8479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:38 smithi090 ceph-mon[29180]: pgmap v8479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:40 smithi119 ceph-mon[32752]: pgmap v8480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:40 smithi167 ceph-mon[32751]: pgmap v8480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:40 smithi090 ceph-mon[29180]: pgmap v8480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14602.202533 secs 2024-04-26T22:38:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14602.202533 secs 2024-04-26T22:38:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14602.202533 secs 2024-04-26T22:38:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14602.956333 secs 2024-04-26T22:38:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:42 smithi119 ceph-mon[32752]: pgmap v8481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14602.956333 secs 2024-04-26T22:38:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:42 smithi167 ceph-mon[32751]: pgmap v8481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14602.956333 secs 2024-04-26T22:38:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:42 smithi090 ceph-mon[29180]: pgmap v8481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:44 smithi119 ceph-mon[32752]: pgmap v8482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:44 smithi167 ceph-mon[32751]: pgmap v8482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:44 smithi090 ceph-mon[29180]: pgmap v8482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:46 smithi119 ceph-mon[32752]: pgmap v8483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14607.202618 secs 2024-04-26T22:38:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:46 smithi167 ceph-mon[32751]: pgmap v8483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14607.202618 secs 2024-04-26T22:38:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:46 smithi090 ceph-mon[29180]: pgmap v8483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14607.202618 secs 2024-04-26T22:38:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14607.956442 secs 2024-04-26T22:38:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14607.956442 secs 2024-04-26T22:38:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14607.956442 secs 2024-04-26T22:38:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:48 smithi119 ceph-mon[32752]: pgmap v8484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:48 smithi167 ceph-mon[32751]: pgmap v8484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:48 smithi090 ceph-mon[29180]: pgmap v8484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:50 smithi119 ceph-mon[32752]: pgmap v8485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:50 smithi167 ceph-mon[32751]: pgmap v8485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:50 smithi090 ceph-mon[29180]: pgmap v8485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:51.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14612.202703 secs 2024-04-26T22:38:51.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14612.202703 secs 2024-04-26T22:38:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14612.202703 secs 2024-04-26T22:38:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14612.956518 secs 2024-04-26T22:38:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:52 smithi119 ceph-mon[32752]: pgmap v8486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14612.956518 secs 2024-04-26T22:38:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:52 smithi167 ceph-mon[32751]: pgmap v8486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14612.956518 secs 2024-04-26T22:38:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:52 smithi090 ceph-mon[29180]: pgmap v8486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:53 smithi119 ceph-mon[32752]: pgmap v8487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:53 smithi167 ceph-mon[32751]: pgmap v8487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:53 smithi090 ceph-mon[29180]: pgmap v8487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:55 smithi119 ceph-mon[32752]: pgmap v8488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:55 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14617.202810 secs 2024-04-26T22:38:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:55 smithi167 ceph-mon[32751]: pgmap v8488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:55 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14617.202810 secs 2024-04-26T22:38:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:55 smithi090 ceph-mon[29180]: pgmap v8488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:38:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:55 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14617.202810 secs 2024-04-26T22:38:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:56 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14617.956636 secs 2024-04-26T22:38:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:56 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14617.956636 secs 2024-04-26T22:38:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:56 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14617.956636 secs 2024-04-26T22:38:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:57 smithi119 ceph-mon[32752]: pgmap v8489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:38:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:57 smithi167 ceph-mon[32751]: pgmap v8489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:38:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:57 smithi090 ceph-mon[29180]: pgmap v8489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:38:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:38:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:38:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:38:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:58 smithi119 ceph-mon[32752]: pgmap v8490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T22:38:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:58 smithi119 ceph-mon[32752]: pgmap v8491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T22:38:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:38:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:38:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:38:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:58 smithi167 ceph-mon[32751]: pgmap v8490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T22:38:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:58 smithi167 ceph-mon[32751]: pgmap v8491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T22:38:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:38:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:38:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:38:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:38:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:58 smithi090 ceph-mon[29180]: pgmap v8490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-26T22:38:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:58 smithi090 ceph-mon[29180]: pgmap v8491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-26T22:38:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:38:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:39:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14622.202877 secs 2024-04-26T22:39:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:01 smithi119 ceph-mon[32752]: pgmap v8492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14622.202877 secs 2024-04-26T22:39:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:01 smithi167 ceph-mon[32751]: pgmap v8492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:01.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14622.202877 secs 2024-04-26T22:39:01.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:01 smithi090 ceph-mon[29180]: pgmap v8492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14622.956768 secs 2024-04-26T22:39:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14622.956768 secs 2024-04-26T22:39:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14622.956768 secs 2024-04-26T22:39:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:03 smithi119 ceph-mon[32752]: pgmap v8493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:03 smithi167 ceph-mon[32751]: pgmap v8493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:03.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:03 smithi090 ceph-mon[29180]: pgmap v8493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:05 smithi119 ceph-mon[32752]: pgmap v8494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:05 smithi167 ceph-mon[32751]: pgmap v8494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:05 smithi090 ceph-mon[29180]: pgmap v8494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14627.202960 secs 2024-04-26T22:39:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14627.202960 secs 2024-04-26T22:39:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14627.202960 secs 2024-04-26T22:39:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14627.956886 secs 2024-04-26T22:39:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:07 smithi119 ceph-mon[32752]: pgmap v8495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14627.956886 secs 2024-04-26T22:39:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:07 smithi167 ceph-mon[32751]: pgmap v8495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14627.956886 secs 2024-04-26T22:39:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:07 smithi090 ceph-mon[29180]: pgmap v8495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:39:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:09 smithi119 ceph-mon[32752]: pgmap v8496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:39:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:09 smithi167 ceph-mon[32751]: pgmap v8496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:39:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:09 smithi090 ceph-mon[29180]: pgmap v8496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:39:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14632.203051 secs 2024-04-26T22:39:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:11 smithi119 ceph-mon[32752]: pgmap v8497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14632.203051 secs 2024-04-26T22:39:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:11 smithi167 ceph-mon[32751]: pgmap v8497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14632.203051 secs 2024-04-26T22:39:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:11 smithi090 ceph-mon[29180]: pgmap v8497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14632.957033 secs 2024-04-26T22:39:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14632.957033 secs 2024-04-26T22:39:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14632.957033 secs 2024-04-26T22:39:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:13 smithi119 ceph-mon[32752]: pgmap v8498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:13 smithi167 ceph-mon[32751]: pgmap v8498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:13 smithi090 ceph-mon[29180]: pgmap v8498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:15 smithi119 ceph-mon[32752]: pgmap v8499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:15 smithi167 ceph-mon[32751]: pgmap v8499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:15 smithi090 ceph-mon[29180]: pgmap v8499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:16.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14637.203134 secs 2024-04-26T22:39:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14637.203134 secs 2024-04-26T22:39:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14637.203134 secs 2024-04-26T22:39:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14637.957139 secs 2024-04-26T22:39:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:17 smithi119 ceph-mon[32752]: pgmap v8500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14637.957139 secs 2024-04-26T22:39:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:17 smithi167 ceph-mon[32751]: pgmap v8500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14637.957139 secs 2024-04-26T22:39:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:17 smithi090 ceph-mon[29180]: pgmap v8500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:19 smithi119 ceph-mon[32752]: pgmap v8501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:19 smithi167 ceph-mon[32751]: pgmap v8501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:19 smithi090 ceph-mon[29180]: pgmap v8501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14642.203222 secs 2024-04-26T22:39:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:21 smithi119 ceph-mon[32752]: pgmap v8502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14642.203222 secs 2024-04-26T22:39:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:21 smithi167 ceph-mon[32751]: pgmap v8502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14642.203222 secs 2024-04-26T22:39:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:21 smithi090 ceph-mon[29180]: pgmap v8502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14642.957236 secs 2024-04-26T22:39:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14642.957236 secs 2024-04-26T22:39:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14642.957236 secs 2024-04-26T22:39:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:23 smithi119 ceph-mon[32752]: pgmap v8503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:23 smithi167 ceph-mon[32751]: pgmap v8503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:23 smithi090 ceph-mon[29180]: pgmap v8503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:25 smithi119 ceph-mon[32752]: pgmap v8504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:25 smithi167 ceph-mon[32751]: pgmap v8504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:25 smithi090 ceph-mon[29180]: pgmap v8504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14647.203305 secs 2024-04-26T22:39:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14647.203305 secs 2024-04-26T22:39:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14647.203305 secs 2024-04-26T22:39:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14647.957352 secs 2024-04-26T22:39:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:27 smithi119 ceph-mon[32752]: pgmap v8505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14647.957352 secs 2024-04-26T22:39:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:27 smithi167 ceph-mon[32751]: pgmap v8505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14647.957352 secs 2024-04-26T22:39:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:27 smithi090 ceph-mon[29180]: pgmap v8505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:29 smithi119 ceph-mon[32752]: pgmap v8506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:29 smithi167 ceph-mon[32751]: pgmap v8506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:29 smithi090 ceph-mon[29180]: pgmap v8506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14652.203388 secs 2024-04-26T22:39:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:31 smithi119 ceph-mon[32752]: pgmap v8507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14652.203388 secs 2024-04-26T22:39:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:31 smithi167 ceph-mon[32751]: pgmap v8507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14652.203388 secs 2024-04-26T22:39:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:31 smithi090 ceph-mon[29180]: pgmap v8507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14652.957490 secs 2024-04-26T22:39:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14652.957490 secs 2024-04-26T22:39:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14652.957490 secs 2024-04-26T22:39:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:33 smithi119 ceph-mon[32752]: pgmap v8508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:33 smithi167 ceph-mon[32751]: pgmap v8508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:33 smithi090 ceph-mon[29180]: pgmap v8508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:35 smithi119 ceph-mon[32752]: pgmap v8509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:35 smithi167 ceph-mon[32751]: pgmap v8509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:35 smithi090 ceph-mon[29180]: pgmap v8509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14657.203472 secs 2024-04-26T22:39:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14657.203472 secs 2024-04-26T22:39:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14657.203472 secs 2024-04-26T22:39:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14657.957589 secs 2024-04-26T22:39:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:37 smithi119 ceph-mon[32752]: pgmap v8510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14657.957589 secs 2024-04-26T22:39:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:37 smithi167 ceph-mon[32751]: pgmap v8510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14657.957589 secs 2024-04-26T22:39:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:37 smithi090 ceph-mon[29180]: pgmap v8510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:39 smithi119 ceph-mon[32752]: pgmap v8511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:39 smithi167 ceph-mon[32751]: pgmap v8511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:39 smithi090 ceph-mon[29180]: pgmap v8511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14662.203558 secs 2024-04-26T22:39:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:41 smithi119 ceph-mon[32752]: pgmap v8512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14662.203558 secs 2024-04-26T22:39:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:41 smithi167 ceph-mon[32751]: pgmap v8512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14662.203558 secs 2024-04-26T22:39:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:41 smithi090 ceph-mon[29180]: pgmap v8512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14662.957710 secs 2024-04-26T22:39:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14662.957710 secs 2024-04-26T22:39:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14662.957710 secs 2024-04-26T22:39:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:43 smithi167 ceph-mon[32751]: pgmap v8513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:43 smithi090 ceph-mon[29180]: pgmap v8513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:43 smithi119 ceph-mon[32752]: pgmap v8513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:45 smithi167 ceph-mon[32751]: pgmap v8514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:45 smithi090 ceph-mon[29180]: pgmap v8514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:45 smithi119 ceph-mon[32752]: pgmap v8514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14667.203642 secs 2024-04-26T22:39:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14667.203642 secs 2024-04-26T22:39:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14667.203642 secs 2024-04-26T22:39:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14667.957824 secs 2024-04-26T22:39:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:47 smithi167 ceph-mon[32751]: pgmap v8515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14667.957824 secs 2024-04-26T22:39:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:47 smithi090 ceph-mon[29180]: pgmap v8515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14667.957824 secs 2024-04-26T22:39:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:47 smithi119 ceph-mon[32752]: pgmap v8515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:49 smithi090 ceph-mon[29180]: pgmap v8516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:49 smithi119 ceph-mon[32752]: pgmap v8516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:49 smithi167 ceph-mon[32751]: pgmap v8516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:51.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14672.203700 secs 2024-04-26T22:39:51.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:51 smithi119 ceph-mon[32752]: pgmap v8517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14672.203700 secs 2024-04-26T22:39:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:51 smithi090 ceph-mon[29180]: pgmap v8517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14672.203700 secs 2024-04-26T22:39:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:51 smithi167 ceph-mon[32751]: pgmap v8517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:52.625 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14672.957940 secs 2024-04-26T22:39:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14672.957940 secs 2024-04-26T22:39:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14672.957940 secs 2024-04-26T22:39:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:53 smithi090 ceph-mon[29180]: pgmap v8518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:53 smithi119 ceph-mon[32752]: pgmap v8518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:53 smithi167 ceph-mon[32751]: pgmap v8518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:55 smithi090 ceph-mon[29180]: pgmap v8519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:55 smithi119 ceph-mon[32752]: pgmap v8519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:55 smithi167 ceph-mon[32751]: pgmap v8519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:39:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14677.203784 secs 2024-04-26T22:39:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14677.203784 secs 2024-04-26T22:39:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14677.203784 secs 2024-04-26T22:39:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14677.958065 secs 2024-04-26T22:39:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:57 smithi090 ceph-mon[29180]: pgmap v8520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14677.958065 secs 2024-04-26T22:39:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:57 smithi119 ceph-mon[32752]: pgmap v8520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14677.958065 secs 2024-04-26T22:39:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:57 smithi167 ceph-mon[32751]: pgmap v8520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:39:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:39:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:39:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:59 smithi090 ceph-mon[29180]: pgmap v8521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:39:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:39:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:39:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:39:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:59 smithi119 ceph-mon[32752]: pgmap v8521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:39:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:39:59.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:39:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:39:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:59 smithi167 ceph-mon[32751]: pgmap v8521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:39:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:39:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:39:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:39:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:40:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:00 smithi090 ceph-mon[29180]: pgmap v8522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:40:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:00 smithi090 ceph-mon[29180]: pgmap v8523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:40:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:00 smithi119 ceph-mon[32752]: pgmap v8522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:40:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:00 smithi119 ceph-mon[32752]: pgmap v8523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:40:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:00 smithi167 ceph-mon[32751]: pgmap v8522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:40:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:00 smithi167 ceph-mon[32751]: pgmap v8523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:40:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14682.203873 secs 2024-04-26T22:40:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14682.203873 secs 2024-04-26T22:40:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14682.203873 secs 2024-04-26T22:40:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14682.958137 secs 2024-04-26T22:40:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:02 smithi090 ceph-mon[29180]: pgmap v8524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14682.958137 secs 2024-04-26T22:40:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:02 smithi119 ceph-mon[32752]: pgmap v8524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14682.958137 secs 2024-04-26T22:40:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:02 smithi167 ceph-mon[32751]: pgmap v8524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:04 smithi090 ceph-mon[29180]: pgmap v8525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:40:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:04 smithi119 ceph-mon[32752]: pgmap v8525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:40:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:04 smithi167 ceph-mon[32751]: pgmap v8525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:40:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:06 smithi119 ceph-mon[32752]: pgmap v8526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14687.203964 secs 2024-04-26T22:40:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:06 smithi167 ceph-mon[32751]: pgmap v8526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14687.203964 secs 2024-04-26T22:40:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:06 smithi090 ceph-mon[29180]: pgmap v8526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14687.203964 secs 2024-04-26T22:40:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14687.958259 secs 2024-04-26T22:40:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14687.958259 secs 2024-04-26T22:40:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14687.958259 secs 2024-04-26T22:40:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:08 smithi119 ceph-mon[32752]: pgmap v8527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:08 smithi167 ceph-mon[32751]: pgmap v8527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:08 smithi090 ceph-mon[29180]: pgmap v8527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:40:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:10 smithi119 ceph-mon[32752]: pgmap v8528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:40:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:10 smithi167 ceph-mon[32751]: pgmap v8528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:40:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:10 smithi090 ceph-mon[29180]: pgmap v8528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:40:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14692.204046 secs 2024-04-26T22:40:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14692.204046 secs 2024-04-26T22:40:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14692.204046 secs 2024-04-26T22:40:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14692.958370 secs 2024-04-26T22:40:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:12 smithi119 ceph-mon[32752]: pgmap v8529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14692.958370 secs 2024-04-26T22:40:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:12 smithi167 ceph-mon[32751]: pgmap v8529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14692.958370 secs 2024-04-26T22:40:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:12 smithi090 ceph-mon[29180]: pgmap v8529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:14 smithi119 ceph-mon[32752]: pgmap v8530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:14 smithi167 ceph-mon[32751]: pgmap v8530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:14 smithi090 ceph-mon[29180]: pgmap v8530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:16 smithi119 ceph-mon[32752]: pgmap v8531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14697.204132 secs 2024-04-26T22:40:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:16 smithi167 ceph-mon[32751]: pgmap v8531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14697.204132 secs 2024-04-26T22:40:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:16 smithi090 ceph-mon[29180]: pgmap v8531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14697.204132 secs 2024-04-26T22:40:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14697.958484 secs 2024-04-26T22:40:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14697.958484 secs 2024-04-26T22:40:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14697.958484 secs 2024-04-26T22:40:18.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:18 smithi090 ceph-mon[29180]: pgmap v8532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:18 smithi119 ceph-mon[32752]: pgmap v8532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:18 smithi167 ceph-mon[32751]: pgmap v8532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:20 smithi119 ceph-mon[32752]: pgmap v8533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:20 smithi167 ceph-mon[32751]: pgmap v8533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:20 smithi090 ceph-mon[29180]: pgmap v8533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14702.204193 secs 2024-04-26T22:40:21.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14702.204193 secs 2024-04-26T22:40:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14702.204193 secs 2024-04-26T22:40:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14702.958602 secs 2024-04-26T22:40:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:22 smithi119 ceph-mon[32752]: pgmap v8534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14702.958602 secs 2024-04-26T22:40:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:22 smithi167 ceph-mon[32751]: pgmap v8534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14702.958602 secs 2024-04-26T22:40:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:22 smithi090 ceph-mon[29180]: pgmap v8534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:24 smithi119 ceph-mon[32752]: pgmap v8535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:24 smithi167 ceph-mon[32751]: pgmap v8535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:24 smithi090 ceph-mon[29180]: pgmap v8535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:26 smithi119 ceph-mon[32752]: pgmap v8536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14707.204275 secs 2024-04-26T22:40:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:26 smithi167 ceph-mon[32751]: pgmap v8536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14707.204275 secs 2024-04-26T22:40:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:26 smithi090 ceph-mon[29180]: pgmap v8536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14707.204275 secs 2024-04-26T22:40:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14707.958720 secs 2024-04-26T22:40:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14707.958720 secs 2024-04-26T22:40:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14707.958720 secs 2024-04-26T22:40:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:28 smithi119 ceph-mon[32752]: pgmap v8537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:28 smithi167 ceph-mon[32751]: pgmap v8537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:28 smithi090 ceph-mon[29180]: pgmap v8537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:30 smithi119 ceph-mon[32752]: pgmap v8538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:30 smithi167 ceph-mon[32751]: pgmap v8538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:30 smithi090 ceph-mon[29180]: pgmap v8538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14712.204360 secs 2024-04-26T22:40:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14712.204360 secs 2024-04-26T22:40:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14712.204360 secs 2024-04-26T22:40:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14712.958826 secs 2024-04-26T22:40:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:32 smithi119 ceph-mon[32752]: pgmap v8539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14712.958826 secs 2024-04-26T22:40:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:32 smithi167 ceph-mon[32751]: pgmap v8539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14712.958826 secs 2024-04-26T22:40:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:32 smithi090 ceph-mon[29180]: pgmap v8539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:33.766 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:33 smithi090 ceph-mon[29180]: pgmap v8540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:33 smithi119 ceph-mon[32752]: pgmap v8540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:33 smithi167 ceph-mon[32751]: pgmap v8540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:36 smithi119 ceph-mon[32752]: pgmap v8541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:36 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 14717.204441 secs 2024-04-26T22:40:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:36 smithi119 ceph-mon[32752]: slow request 11523.341132 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:40:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:36 smithi167 ceph-mon[32751]: pgmap v8541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:36 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 14717.204441 secs 2024-04-26T22:40:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:36 smithi167 ceph-mon[32751]: slow request 11523.341132 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:40:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:36 smithi090 ceph-mon[29180]: pgmap v8541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:36 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 14717.204441 secs 2024-04-26T22:40:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:36 smithi090 ceph-mon[29180]: slow request 11523.341132 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:40:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14717.958942 secs 2024-04-26T22:40:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14717.958942 secs 2024-04-26T22:40:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14717.958942 secs 2024-04-26T22:40:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:38 smithi119 ceph-mon[32752]: pgmap v8542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:38 smithi167 ceph-mon[32751]: pgmap v8542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:38 smithi090 ceph-mon[29180]: pgmap v8542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:40 smithi119 ceph-mon[32752]: pgmap v8543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:40 smithi167 ceph-mon[32751]: pgmap v8543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:40 smithi090 ceph-mon[29180]: pgmap v8543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14722.204523 secs 2024-04-26T22:40:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14722.204523 secs 2024-04-26T22:40:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14722.204523 secs 2024-04-26T22:40:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14722.959060 secs 2024-04-26T22:40:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:42 smithi119 ceph-mon[32752]: pgmap v8544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14722.959060 secs 2024-04-26T22:40:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:42 smithi167 ceph-mon[32751]: pgmap v8544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14722.959060 secs 2024-04-26T22:40:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:42 smithi090 ceph-mon[29180]: pgmap v8544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:44 smithi119 ceph-mon[32752]: pgmap v8545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:44 smithi167 ceph-mon[32751]: pgmap v8545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:44 smithi090 ceph-mon[29180]: pgmap v8545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:46 smithi119 ceph-mon[32752]: pgmap v8546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14727.204608 secs 2024-04-26T22:40:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:46 smithi167 ceph-mon[32751]: pgmap v8546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14727.204608 secs 2024-04-26T22:40:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:46 smithi090 ceph-mon[29180]: pgmap v8546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14727.204608 secs 2024-04-26T22:40:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14727.959177 secs 2024-04-26T22:40:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14727.959177 secs 2024-04-26T22:40:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14727.959177 secs 2024-04-26T22:40:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:48 smithi119 ceph-mon[32752]: pgmap v8547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:48 smithi167 ceph-mon[32751]: pgmap v8547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:48 smithi090 ceph-mon[29180]: pgmap v8547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:50 smithi119 ceph-mon[32752]: pgmap v8548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:50 smithi167 ceph-mon[32751]: pgmap v8548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:50 smithi090 ceph-mon[29180]: pgmap v8548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14732.204691 secs 2024-04-26T22:40:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14732.204691 secs 2024-04-26T22:40:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14732.204691 secs 2024-04-26T22:40:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14732.959297 secs 2024-04-26T22:40:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:52 smithi119 ceph-mon[32752]: pgmap v8549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14732.959297 secs 2024-04-26T22:40:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:52 smithi167 ceph-mon[32751]: pgmap v8549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14732.959297 secs 2024-04-26T22:40:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:52 smithi090 ceph-mon[29180]: pgmap v8549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:54 smithi119 ceph-mon[32752]: pgmap v8550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:54 smithi167 ceph-mon[32751]: pgmap v8550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:54 smithi090 ceph-mon[29180]: pgmap v8550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:56 smithi119 ceph-mon[32752]: pgmap v8551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14737.204772 secs 2024-04-26T22:40:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:56 smithi167 ceph-mon[32751]: pgmap v8551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14737.204772 secs 2024-04-26T22:40:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:56 smithi090 ceph-mon[29180]: pgmap v8551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:40:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14737.204772 secs 2024-04-26T22:40:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14737.959409 secs 2024-04-26T22:40:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14737.959409 secs 2024-04-26T22:40:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14737.959409 secs 2024-04-26T22:40:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:58 smithi167 ceph-mon[32751]: pgmap v8552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:58 smithi090 ceph-mon[29180]: pgmap v8552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:58 smithi119 ceph-mon[32752]: pgmap v8552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:40:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:40:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:40:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:40:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:40:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:40:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:41:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:00 smithi090 ceph-mon[29180]: pgmap v8553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:41:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:41:00.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:41:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:00 smithi119 ceph-mon[32752]: pgmap v8553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:41:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:41:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:41:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:00 smithi167 ceph-mon[32751]: pgmap v8553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:41:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:41:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:41:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:01 smithi090 ceph-mon[29180]: pgmap v8554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:41:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14742.204861 secs 2024-04-26T22:41:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:01 smithi119 ceph-mon[32752]: pgmap v8554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:41:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14742.204861 secs 2024-04-26T22:41:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:01 smithi167 ceph-mon[32751]: pgmap v8554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:41:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14742.204861 secs 2024-04-26T22:41:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14742.959542 secs 2024-04-26T22:41:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14742.959542 secs 2024-04-26T22:41:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14742.959542 secs 2024-04-26T22:41:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:03 smithi090 ceph-mon[29180]: pgmap v8555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:03 smithi119 ceph-mon[32752]: pgmap v8555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:03 smithi167 ceph-mon[32751]: pgmap v8555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:05 smithi090 ceph-mon[29180]: pgmap v8556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:41:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:05 smithi119 ceph-mon[32752]: pgmap v8556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:41:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:05 smithi167 ceph-mon[32751]: pgmap v8556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:41:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14747.204942 secs 2024-04-26T22:41:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14747.204942 secs 2024-04-26T22:41:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14747.204942 secs 2024-04-26T22:41:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14747.959623 secs 2024-04-26T22:41:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:07 smithi090 ceph-mon[29180]: pgmap v8557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:41:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14747.959623 secs 2024-04-26T22:41:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:07 smithi119 ceph-mon[32752]: pgmap v8557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:41:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14747.959623 secs 2024-04-26T22:41:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:07 smithi167 ceph-mon[32751]: pgmap v8557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:41:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:09 smithi090 ceph-mon[29180]: pgmap v8558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:09 smithi119 ceph-mon[32752]: pgmap v8558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:09 smithi167 ceph-mon[32751]: pgmap v8558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:11 smithi090 ceph-mon[29180]: pgmap v8559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14752.205025 secs 2024-04-26T22:41:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:11 smithi119 ceph-mon[32752]: pgmap v8559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14752.205025 secs 2024-04-26T22:41:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:11 smithi167 ceph-mon[32751]: pgmap v8559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:41:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14752.205025 secs 2024-04-26T22:41:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14752.959739 secs 2024-04-26T22:41:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14752.959739 secs 2024-04-26T22:41:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14752.959739 secs 2024-04-26T22:41:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:13 smithi090 ceph-mon[29180]: pgmap v8560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:13 smithi119 ceph-mon[32752]: pgmap v8560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:13 smithi167 ceph-mon[32751]: pgmap v8560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:15 smithi090 ceph-mon[29180]: pgmap v8561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:15 smithi119 ceph-mon[32752]: pgmap v8561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:15 smithi167 ceph-mon[32751]: pgmap v8561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14757.205110 secs 2024-04-26T22:41:16.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14757.205110 secs 2024-04-26T22:41:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14757.205110 secs 2024-04-26T22:41:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14757.959812 secs 2024-04-26T22:41:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:17 smithi090 ceph-mon[29180]: pgmap v8562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14757.959812 secs 2024-04-26T22:41:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:17 smithi119 ceph-mon[32752]: pgmap v8562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14757.959812 secs 2024-04-26T22:41:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:17 smithi167 ceph-mon[32751]: pgmap v8562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:19 smithi090 ceph-mon[29180]: pgmap v8563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:19 smithi119 ceph-mon[32752]: pgmap v8563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:19 smithi167 ceph-mon[32751]: pgmap v8563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:21 smithi090 ceph-mon[29180]: pgmap v8564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14762.205195 secs 2024-04-26T22:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:21 smithi119 ceph-mon[32752]: pgmap v8564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14762.205195 secs 2024-04-26T22:41:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:21 smithi167 ceph-mon[32751]: pgmap v8564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14762.205195 secs 2024-04-26T22:41:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14762.959928 secs 2024-04-26T22:41:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14762.959928 secs 2024-04-26T22:41:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14762.959928 secs 2024-04-26T22:41:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:23 smithi119 ceph-mon[32752]: pgmap v8565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:23 smithi167 ceph-mon[32751]: pgmap v8565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:23 smithi090 ceph-mon[29180]: pgmap v8565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:25 smithi119 ceph-mon[32752]: pgmap v8566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:25 smithi167 ceph-mon[32751]: pgmap v8566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:25 smithi090 ceph-mon[29180]: pgmap v8566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14767.205281 secs 2024-04-26T22:41:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14767.205281 secs 2024-04-26T22:41:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14767.205281 secs 2024-04-26T22:41:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14767.960046 secs 2024-04-26T22:41:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:27 smithi119 ceph-mon[32752]: pgmap v8567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14767.960046 secs 2024-04-26T22:41:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:27 smithi167 ceph-mon[32751]: pgmap v8567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14767.960046 secs 2024-04-26T22:41:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:27 smithi090 ceph-mon[29180]: pgmap v8567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:29 smithi119 ceph-mon[32752]: pgmap v8568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:29 smithi167 ceph-mon[32751]: pgmap v8568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:29 smithi090 ceph-mon[29180]: pgmap v8568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:31 smithi119 ceph-mon[32752]: pgmap v8569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14772.205365 secs 2024-04-26T22:41:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:31 smithi167 ceph-mon[32751]: pgmap v8569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14772.205365 secs 2024-04-26T22:41:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:31 smithi090 ceph-mon[29180]: pgmap v8569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14772.205365 secs 2024-04-26T22:41:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14772.960166 secs 2024-04-26T22:41:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14772.960166 secs 2024-04-26T22:41:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14772.960166 secs 2024-04-26T22:41:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:33 smithi119 ceph-mon[32752]: pgmap v8570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:33 smithi167 ceph-mon[32751]: pgmap v8570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:33 smithi090 ceph-mon[29180]: pgmap v8570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:35.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:41:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:41:34.755+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:41:35.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:41:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:41:34.755+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:41:35.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:41:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:41:34.755+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:41:35.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:41:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:41:34.755+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:41:35.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:41:34 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:41:34.755+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:41:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:35 smithi119 ceph-mon[32752]: pgmap v8571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:35 smithi167 ceph-mon[32751]: pgmap v8571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:35 smithi090 ceph-mon[29180]: pgmap v8571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14777.205454 secs 2024-04-26T22:41:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14777.205454 secs 2024-04-26T22:41:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14777.205454 secs 2024-04-26T22:41:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:37 smithi119 ceph-mon[32752]: 17 slow requests, 1 included below; oldest blocked for > 14777.960225 secs 2024-04-26T22:41:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:37 smithi119 ceph-mon[32752]: slow request 2881.700357 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:41:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:37 smithi119 ceph-mon[32752]: pgmap v8572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:37 smithi167 ceph-mon[32751]: 17 slow requests, 1 included below; oldest blocked for > 14777.960225 secs 2024-04-26T22:41:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:37 smithi167 ceph-mon[32751]: slow request 2881.700357 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:41:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:37 smithi167 ceph-mon[32751]: pgmap v8572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:37 smithi090 ceph-mon[29180]: 17 slow requests, 1 included below; oldest blocked for > 14777.960225 secs 2024-04-26T22:41:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:37 smithi090 ceph-mon[29180]: slow request 2881.700357 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:41:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:37 smithi090 ceph-mon[29180]: pgmap v8572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:39 smithi119 ceph-mon[32752]: pgmap v8573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:39 smithi167 ceph-mon[32751]: pgmap v8573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:39 smithi090 ceph-mon[29180]: pgmap v8573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:41 smithi119 ceph-mon[32752]: pgmap v8574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14782.205539 secs 2024-04-26T22:41:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:41 smithi167 ceph-mon[32751]: pgmap v8574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14782.205539 secs 2024-04-26T22:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:41 smithi090 ceph-mon[29180]: pgmap v8574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14782.205539 secs 2024-04-26T22:41:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14782.960313 secs 2024-04-26T22:41:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14782.960313 secs 2024-04-26T22:41:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14782.960313 secs 2024-04-26T22:41:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:43 smithi119 ceph-mon[32752]: pgmap v8575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:43 smithi167 ceph-mon[32751]: pgmap v8575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:43 smithi090 ceph-mon[29180]: pgmap v8575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:45 smithi119 ceph-mon[32752]: pgmap v8576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:45 smithi167 ceph-mon[32751]: pgmap v8576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:45 smithi090 ceph-mon[29180]: pgmap v8576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:46.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14787.205621 secs 2024-04-26T22:41:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14787.205621 secs 2024-04-26T22:41:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14787.205621 secs 2024-04-26T22:41:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14787.960452 secs 2024-04-26T22:41:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:47 smithi119 ceph-mon[32752]: pgmap v8577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14787.960452 secs 2024-04-26T22:41:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:47 smithi167 ceph-mon[32751]: pgmap v8577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14787.960452 secs 2024-04-26T22:41:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:47 smithi090 ceph-mon[29180]: pgmap v8577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:49 smithi119 ceph-mon[32752]: pgmap v8578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:49 smithi167 ceph-mon[32751]: pgmap v8578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:49 smithi090 ceph-mon[29180]: pgmap v8578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:41:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:51 smithi119 ceph-mon[32752]: pgmap v8579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14792.205709 secs 2024-04-26T22:41:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:51 smithi167 ceph-mon[32751]: pgmap v8579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14792.205709 secs 2024-04-26T22:41:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:51 smithi090 ceph-mon[29180]: pgmap v8579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:41:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14792.205709 secs 2024-04-26T22:41:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14792.960566 secs 2024-04-26T22:41:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14792.960566 secs 2024-04-26T22:41:52.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14792.960566 secs 2024-04-26T22:41:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:53 smithi119 ceph-mon[32752]: pgmap v8580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:53 smithi167 ceph-mon[32751]: pgmap v8580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:53 smithi090 ceph-mon[29180]: pgmap v8580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:55 smithi119 ceph-mon[32752]: pgmap v8581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:55 smithi167 ceph-mon[32751]: pgmap v8581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:55 smithi090 ceph-mon[29180]: pgmap v8581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:41:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14797.205816 secs 2024-04-26T22:41:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14797.205816 secs 2024-04-26T22:41:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14797.205816 secs 2024-04-26T22:41:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14797.960691 secs 2024-04-26T22:41:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:57 smithi119 ceph-mon[32752]: pgmap v8582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14797.960691 secs 2024-04-26T22:41:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:57 smithi167 ceph-mon[32751]: pgmap v8582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14797.960691 secs 2024-04-26T22:41:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:57 smithi090 ceph-mon[29180]: pgmap v8582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:41:59 smithi119 ceph-mon[32752]: pgmap v8583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:41:59 smithi167 ceph-mon[32751]: pgmap v8583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:41:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:41:59 smithi090 ceph-mon[29180]: pgmap v8583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:42:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:42:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:42:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:01 smithi119 ceph-mon[32752]: pgmap v8584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14802.205884 secs 2024-04-26T22:42:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:42:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:42:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:42:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:01 smithi167 ceph-mon[32751]: pgmap v8584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14802.205884 secs 2024-04-26T22:42:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:42:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:42:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:42:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:01 smithi090 ceph-mon[29180]: pgmap v8584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14802.205884 secs 2024-04-26T22:42:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:42:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:42:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:42:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:02 smithi119 ceph-mon[32752]: pgmap v8585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:42:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:02 smithi119 ceph-mon[32752]: pgmap v8586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14802.960811 secs 2024-04-26T22:42:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:02 smithi167 ceph-mon[32751]: pgmap v8585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:42:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:02 smithi167 ceph-mon[32751]: pgmap v8586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14802.960811 secs 2024-04-26T22:42:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:02 smithi090 ceph-mon[29180]: pgmap v8585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:42:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:02 smithi090 ceph-mon[29180]: pgmap v8586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14802.960811 secs 2024-04-26T22:42:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:04 smithi119 ceph-mon[32752]: pgmap v8587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:42:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:04 smithi167 ceph-mon[32751]: pgmap v8587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:42:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:04 smithi090 ceph-mon[29180]: pgmap v8587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:42:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:06 smithi119 ceph-mon[32752]: pgmap v8588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14807.205946 secs 2024-04-26T22:42:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:06 smithi167 ceph-mon[32751]: pgmap v8588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14807.205946 secs 2024-04-26T22:42:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:06 smithi090 ceph-mon[29180]: pgmap v8588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14807.205946 secs 2024-04-26T22:42:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14807.960909 secs 2024-04-26T22:42:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14807.960909 secs 2024-04-26T22:42:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14807.960909 secs 2024-04-26T22:42:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:08 smithi119 ceph-mon[32752]: pgmap v8589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:08 smithi167 ceph-mon[32751]: pgmap v8589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:08 smithi090 ceph-mon[29180]: pgmap v8589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:10 smithi119 ceph-mon[32752]: pgmap v8590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:10 smithi167 ceph-mon[32751]: pgmap v8590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:10 smithi090 ceph-mon[29180]: pgmap v8590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:42:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14812.206032 secs 2024-04-26T22:42:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14812.206032 secs 2024-04-26T22:42:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14812.206032 secs 2024-04-26T22:42:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:12 smithi119 ceph-mon[32752]: pgmap v8591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:42:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14812.961018 secs 2024-04-26T22:42:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:12 smithi167 ceph-mon[32751]: pgmap v8591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:42:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14812.961018 secs 2024-04-26T22:42:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:12 smithi090 ceph-mon[29180]: pgmap v8591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:42:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14812.961018 secs 2024-04-26T22:42:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:14 smithi119 ceph-mon[32752]: pgmap v8592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:14 smithi167 ceph-mon[32751]: pgmap v8592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:14 smithi090 ceph-mon[29180]: pgmap v8592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:16 smithi119 ceph-mon[32752]: pgmap v8593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14817.206093 secs 2024-04-26T22:42:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:16 smithi167 ceph-mon[32751]: pgmap v8593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14817.206093 secs 2024-04-26T22:42:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:16 smithi090 ceph-mon[29180]: pgmap v8593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14817.206093 secs 2024-04-26T22:42:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14817.961161 secs 2024-04-26T22:42:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14817.961161 secs 2024-04-26T22:42:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14817.961161 secs 2024-04-26T22:42:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:18 smithi119 ceph-mon[32752]: pgmap v8594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:18 smithi167 ceph-mon[32751]: pgmap v8594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:18 smithi090 ceph-mon[29180]: pgmap v8594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:20 smithi119 ceph-mon[32752]: pgmap v8595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:20 smithi167 ceph-mon[32751]: pgmap v8595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:20 smithi090 ceph-mon[29180]: pgmap v8595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14822.206154 secs 2024-04-26T22:42:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14822.206154 secs 2024-04-26T22:42:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14822.206154 secs 2024-04-26T22:42:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:22 smithi119 ceph-mon[32752]: pgmap v8596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14822.961312 secs 2024-04-26T22:42:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:22 smithi167 ceph-mon[32751]: pgmap v8596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14822.961312 secs 2024-04-26T22:42:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:22 smithi090 ceph-mon[29180]: pgmap v8596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14822.961312 secs 2024-04-26T22:42:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:24 smithi167 ceph-mon[32751]: pgmap v8597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:24 smithi090 ceph-mon[29180]: pgmap v8597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:24 smithi119 ceph-mon[32752]: pgmap v8597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:26 smithi167 ceph-mon[32751]: pgmap v8598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14827.206213 secs 2024-04-26T22:42:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:26 smithi090 ceph-mon[29180]: pgmap v8598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14827.206213 secs 2024-04-26T22:42:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:26 smithi119 ceph-mon[32752]: pgmap v8598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14827.206213 secs 2024-04-26T22:42:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14827.961432 secs 2024-04-26T22:42:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14827.961432 secs 2024-04-26T22:42:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14827.961432 secs 2024-04-26T22:42:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:28 smithi167 ceph-mon[32751]: pgmap v8599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:28 smithi090 ceph-mon[29180]: pgmap v8599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:28 smithi119 ceph-mon[32752]: pgmap v8599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:30 smithi090 ceph-mon[29180]: pgmap v8600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:30 smithi119 ceph-mon[32752]: pgmap v8600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:30 smithi167 ceph-mon[32751]: pgmap v8600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14832.206299 secs 2024-04-26T22:42:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14832.206299 secs 2024-04-26T22:42:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14832.206299 secs 2024-04-26T22:42:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:32 smithi090 ceph-mon[29180]: pgmap v8601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14832.961591 secs 2024-04-26T22:42:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:32 smithi119 ceph-mon[32752]: pgmap v8601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14832.961591 secs 2024-04-26T22:42:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:32 smithi167 ceph-mon[32751]: pgmap v8601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14832.961591 secs 2024-04-26T22:42:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:34 smithi090 ceph-mon[29180]: pgmap v8602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:34 smithi119 ceph-mon[32752]: pgmap v8602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:34 smithi167 ceph-mon[32751]: pgmap v8602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:36 smithi090 ceph-mon[29180]: pgmap v8603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14837.206384 secs 2024-04-26T22:42:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:36 smithi119 ceph-mon[32752]: pgmap v8603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14837.206384 secs 2024-04-26T22:42:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:36 smithi167 ceph-mon[32751]: pgmap v8603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14837.206384 secs 2024-04-26T22:42:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14837.961663 secs 2024-04-26T22:42:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14837.961663 secs 2024-04-26T22:42:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14837.961663 secs 2024-04-26T22:42:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:38 smithi090 ceph-mon[29180]: pgmap v8604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:38 smithi119 ceph-mon[32752]: pgmap v8604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:38 smithi167 ceph-mon[32751]: pgmap v8604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:40 smithi090 ceph-mon[29180]: pgmap v8605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:40 smithi119 ceph-mon[32752]: pgmap v8605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:40 smithi167 ceph-mon[32751]: pgmap v8605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14842.206470 secs 2024-04-26T22:42:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14842.206470 secs 2024-04-26T22:42:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14842.206470 secs 2024-04-26T22:42:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:42 smithi090 ceph-mon[29180]: pgmap v8606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14842.961758 secs 2024-04-26T22:42:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:42 smithi119 ceph-mon[32752]: pgmap v8606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14842.961758 secs 2024-04-26T22:42:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:42 smithi167 ceph-mon[32751]: pgmap v8606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14842.961758 secs 2024-04-26T22:42:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:44 smithi090 ceph-mon[29180]: pgmap v8607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:44 smithi119 ceph-mon[32752]: pgmap v8607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:44 smithi167 ceph-mon[32751]: pgmap v8607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:46.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:46 smithi167 ceph-mon[32751]: pgmap v8608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:46.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14847.206552 secs 2024-04-26T22:42:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:46 smithi090 ceph-mon[29180]: pgmap v8608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14847.206552 secs 2024-04-26T22:42:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:46 smithi119 ceph-mon[32752]: pgmap v8608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14847.206552 secs 2024-04-26T22:42:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14847.961875 secs 2024-04-26T22:42:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14847.961875 secs 2024-04-26T22:42:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14847.961875 secs 2024-04-26T22:42:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:48 smithi090 ceph-mon[29180]: pgmap v8609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:48 smithi119 ceph-mon[32752]: pgmap v8609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:48 smithi167 ceph-mon[32751]: pgmap v8609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:50 smithi090 ceph-mon[29180]: pgmap v8610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:50 smithi119 ceph-mon[32752]: pgmap v8610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:50 smithi167 ceph-mon[32751]: pgmap v8610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14852.206634 secs 2024-04-26T22:42:51.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14852.206634 secs 2024-04-26T22:42:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14852.206634 secs 2024-04-26T22:42:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:52 smithi090 ceph-mon[29180]: pgmap v8611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14852.961975 secs 2024-04-26T22:42:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:52 smithi119 ceph-mon[32752]: pgmap v8611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14852.961975 secs 2024-04-26T22:42:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:52 smithi167 ceph-mon[32751]: pgmap v8611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14852.961975 secs 2024-04-26T22:42:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:54 smithi119 ceph-mon[32752]: pgmap v8612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:54 smithi167 ceph-mon[32751]: pgmap v8612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:54 smithi090 ceph-mon[29180]: pgmap v8612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:56 smithi119 ceph-mon[32752]: pgmap v8613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14857.206716 secs 2024-04-26T22:42:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:56 smithi167 ceph-mon[32751]: pgmap v8613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14857.206716 secs 2024-04-26T22:42:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:56 smithi090 ceph-mon[29180]: pgmap v8613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:42:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14857.206716 secs 2024-04-26T22:42:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14857.962107 secs 2024-04-26T22:42:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14857.962107 secs 2024-04-26T22:42:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14857.962107 secs 2024-04-26T22:42:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:42:58 smithi119 ceph-mon[32752]: pgmap v8614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:42:58 smithi167 ceph-mon[32751]: pgmap v8614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:42:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:42:58 smithi090 ceph-mon[29180]: pgmap v8614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:00 smithi119 ceph-mon[32752]: pgmap v8615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:00 smithi167 ceph-mon[32751]: pgmap v8615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:00 smithi090 ceph-mon[29180]: pgmap v8615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 14862.206827 secs 2024-04-26T22:43:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:43:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 14862.206827 secs 2024-04-26T22:43:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:43:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 14862.206827 secs 2024-04-26T22:43:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:43:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:02 smithi119 ceph-mon[32752]: pgmap v8616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14862.962197 secs 2024-04-26T22:43:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:02 smithi090 ceph-mon[29180]: pgmap v8616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14862.962197 secs 2024-04-26T22:43:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:02 smithi167 ceph-mon[32751]: pgmap v8616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14862.962197 secs 2024-04-26T22:43:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:04 smithi090 ceph-mon[29180]: pgmap v8617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:04.764 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:04 smithi119 ceph-mon[32752]: pgmap v8617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:04 smithi167 ceph-mon[32751]: pgmap v8617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: pgmap v8618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14867.206913 secs 2024-04-26T22:43:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:43:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:43:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: pgmap v8619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-26T22:43:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: pgmap v8620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: pgmap v8618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14867.206913 secs 2024-04-26T22:43:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:43:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:43:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: pgmap v8619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-26T22:43:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: pgmap v8620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: pgmap v8618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14867.206913 secs 2024-04-26T22:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:43:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:43:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: pgmap v8619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-26T22:43:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: pgmap v8620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:43:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:07 smithi119 ceph-mon[32752]: 17 slow requests, 1 included below; oldest blocked for > 14867.962282 secs 2024-04-26T22:43:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:07 smithi119 ceph-mon[32752]: slow request 721.120143 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:43:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:43:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:43:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:07 smithi167 ceph-mon[32751]: 17 slow requests, 1 included below; oldest blocked for > 14867.962282 secs 2024-04-26T22:43:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:07 smithi167 ceph-mon[32751]: slow request 721.120143 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:43:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:43:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:43:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:07 smithi090 ceph-mon[29180]: 17 slow requests, 1 included below; oldest blocked for > 14867.962282 secs 2024-04-26T22:43:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:07 smithi090 ceph-mon[29180]: slow request 721.120143 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:43:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:43:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:43:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:08 smithi119 ceph-mon[32752]: pgmap v8621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T22:43:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:08 smithi167 ceph-mon[32751]: pgmap v8621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T22:43:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:08 smithi090 ceph-mon[29180]: pgmap v8621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T22:43:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:10 smithi119 ceph-mon[32752]: pgmap v8622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:10 smithi167 ceph-mon[32751]: pgmap v8622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:10 smithi090 ceph-mon[29180]: pgmap v8622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14872.207003 secs 2024-04-26T22:43:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14872.207003 secs 2024-04-26T22:43:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14872.207003 secs 2024-04-26T22:43:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14872.962433 secs 2024-04-26T22:43:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:12 smithi119 ceph-mon[32752]: pgmap v8623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14872.962433 secs 2024-04-26T22:43:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:12 smithi167 ceph-mon[32751]: pgmap v8623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14872.962433 secs 2024-04-26T22:43:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:12 smithi090 ceph-mon[29180]: pgmap v8623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:14.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:14 smithi119 ceph-mon[32752]: pgmap v8624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T22:43:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:14 smithi167 ceph-mon[32751]: pgmap v8624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T22:43:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:14 smithi090 ceph-mon[29180]: pgmap v8624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T22:43:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14877.207065 secs 2024-04-26T22:43:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:16 smithi119 ceph-mon[32752]: pgmap v8625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:43:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14877.207065 secs 2024-04-26T22:43:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:16 smithi167 ceph-mon[32751]: pgmap v8625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:43:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14877.207065 secs 2024-04-26T22:43:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:16 smithi090 ceph-mon[29180]: pgmap v8625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:43:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14877.962524 secs 2024-04-26T22:43:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14877.962524 secs 2024-04-26T22:43:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14877.962524 secs 2024-04-26T22:43:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:18 smithi119 ceph-mon[32752]: pgmap v8626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:18 smithi167 ceph-mon[32751]: pgmap v8626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:18 smithi090 ceph-mon[29180]: pgmap v8626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:20 smithi119 ceph-mon[32752]: pgmap v8627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:20 smithi167 ceph-mon[32751]: pgmap v8627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:20 smithi090 ceph-mon[29180]: pgmap v8627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14882.207126 secs 2024-04-26T22:43:21.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14882.207126 secs 2024-04-26T22:43:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14882.207126 secs 2024-04-26T22:43:22.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14882.962603 secs 2024-04-26T22:43:22.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:22 smithi090 ceph-mon[29180]: pgmap v8628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14882.962603 secs 2024-04-26T22:43:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:22 smithi119 ceph-mon[32752]: pgmap v8628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14882.962603 secs 2024-04-26T22:43:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:22 smithi167 ceph-mon[32751]: pgmap v8628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:24.773 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:24 smithi090 ceph-mon[29180]: pgmap v8629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:24 smithi119 ceph-mon[32752]: pgmap v8629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:24 smithi167 ceph-mon[32751]: pgmap v8629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14887.207211 secs 2024-04-26T22:43:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:26 smithi119 ceph-mon[32752]: pgmap v8630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14887.207211 secs 2024-04-26T22:43:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:26 smithi167 ceph-mon[32751]: pgmap v8630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14887.207211 secs 2024-04-26T22:43:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:26 smithi090 ceph-mon[29180]: pgmap v8630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14887.962699 secs 2024-04-26T22:43:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14887.962699 secs 2024-04-26T22:43:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14887.962699 secs 2024-04-26T22:43:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:28 smithi119 ceph-mon[32752]: pgmap v8631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:28 smithi167 ceph-mon[32751]: pgmap v8631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:28 smithi090 ceph-mon[29180]: pgmap v8631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:30 smithi119 ceph-mon[32752]: pgmap v8632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:30 smithi167 ceph-mon[32751]: pgmap v8632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:30 smithi090 ceph-mon[29180]: pgmap v8632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14892.207287 secs 2024-04-26T22:43:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14892.207287 secs 2024-04-26T22:43:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14892.207287 secs 2024-04-26T22:43:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14892.962820 secs 2024-04-26T22:43:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:32 smithi119 ceph-mon[32752]: pgmap v8633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14892.962820 secs 2024-04-26T22:43:33.701 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:32 smithi167 ceph-mon[32751]: pgmap v8633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:33.702 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14892.962820 secs 2024-04-26T22:43:33.702 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:32 smithi090 ceph-mon[29180]: pgmap v8633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:34 smithi119 ceph-mon[32752]: pgmap v8634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:34 smithi167 ceph-mon[32751]: pgmap v8634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:34 smithi090 ceph-mon[29180]: pgmap v8634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:35 smithi119 ceph-mon[32752]: pgmap v8635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:35 smithi167 ceph-mon[32751]: pgmap v8635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:35 smithi090 ceph-mon[29180]: pgmap v8635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14897.207370 secs 2024-04-26T22:43:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14897.207370 secs 2024-04-26T22:43:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14897.207370 secs 2024-04-26T22:43:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14897.962893 secs 2024-04-26T22:43:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:37 smithi119 ceph-mon[32752]: pgmap v8636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14897.962893 secs 2024-04-26T22:43:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:37 smithi167 ceph-mon[32751]: pgmap v8636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14897.962893 secs 2024-04-26T22:43:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:37 smithi090 ceph-mon[29180]: pgmap v8636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:40 smithi090 ceph-mon[29180]: pgmap v8637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:40 smithi119 ceph-mon[32752]: pgmap v8637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:40 smithi167 ceph-mon[32751]: pgmap v8637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14902.207450 secs 2024-04-26T22:43:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14902.207450 secs 2024-04-26T22:43:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14902.207450 secs 2024-04-26T22:43:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14902.962951 secs 2024-04-26T22:43:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:42 smithi090 ceph-mon[29180]: pgmap v8638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14902.962951 secs 2024-04-26T22:43:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:42 smithi119 ceph-mon[32752]: pgmap v8638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14902.962951 secs 2024-04-26T22:43:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:42 smithi167 ceph-mon[32751]: pgmap v8638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:44 smithi090 ceph-mon[29180]: pgmap v8639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:44 smithi119 ceph-mon[32752]: pgmap v8639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:44 smithi167 ceph-mon[32751]: pgmap v8639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:46.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14907.207531 secs 2024-04-26T22:43:46.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:46 smithi167 ceph-mon[32751]: pgmap v8640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14907.207531 secs 2024-04-26T22:43:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:46 smithi090 ceph-mon[29180]: pgmap v8640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14907.207531 secs 2024-04-26T22:43:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:46 smithi119 ceph-mon[32752]: pgmap v8640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14907.963070 secs 2024-04-26T22:43:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14907.963070 secs 2024-04-26T22:43:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14907.963070 secs 2024-04-26T22:43:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:48 smithi090 ceph-mon[29180]: pgmap v8641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:48 smithi119 ceph-mon[32752]: pgmap v8641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:48 smithi167 ceph-mon[32751]: pgmap v8641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:50 smithi090 ceph-mon[29180]: pgmap v8642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:50 smithi119 ceph-mon[32752]: pgmap v8642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:50 smithi167 ceph-mon[32751]: pgmap v8642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14912.207602 secs 2024-04-26T22:43:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14912.207602 secs 2024-04-26T22:43:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14912.207602 secs 2024-04-26T22:43:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14912.963180 secs 2024-04-26T22:43:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:52 smithi090 ceph-mon[29180]: pgmap v8643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14912.963180 secs 2024-04-26T22:43:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:52 smithi119 ceph-mon[32752]: pgmap v8643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14912.963180 secs 2024-04-26T22:43:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:52 smithi167 ceph-mon[32751]: pgmap v8643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:54 smithi090 ceph-mon[29180]: pgmap v8644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:54 smithi119 ceph-mon[32752]: pgmap v8644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:54 smithi167 ceph-mon[32751]: pgmap v8644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14917.207688 secs 2024-04-26T22:43:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:56 smithi090 ceph-mon[29180]: pgmap v8645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14917.207688 secs 2024-04-26T22:43:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:56 smithi119 ceph-mon[32752]: pgmap v8645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14917.207688 secs 2024-04-26T22:43:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:56 smithi167 ceph-mon[32751]: pgmap v8645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:43:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14917.963303 secs 2024-04-26T22:43:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14917.963303 secs 2024-04-26T22:43:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14917.963303 secs 2024-04-26T22:43:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:43:58 smithi090 ceph-mon[29180]: pgmap v8646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:43:58 smithi119 ceph-mon[32752]: pgmap v8646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:43:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:43:58 smithi167 ceph-mon[32751]: pgmap v8646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:00 smithi090 ceph-mon[29180]: pgmap v8647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:00 smithi119 ceph-mon[32752]: pgmap v8647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:00 smithi167 ceph-mon[32751]: pgmap v8647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14922.207773 secs 2024-04-26T22:44:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14922.207773 secs 2024-04-26T22:44:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14922.207773 secs 2024-04-26T22:44:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14922.963422 secs 2024-04-26T22:44:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:02 smithi119 ceph-mon[32752]: pgmap v8648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14922.963422 secs 2024-04-26T22:44:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:02 smithi167 ceph-mon[32751]: pgmap v8648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14922.963422 secs 2024-04-26T22:44:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:02 smithi090 ceph-mon[29180]: pgmap v8648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:04.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:04 smithi119 ceph-mon[32752]: pgmap v8649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:04 smithi167 ceph-mon[32751]: pgmap v8649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:04 smithi090 ceph-mon[29180]: pgmap v8649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:06.427 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14927.207837 secs 2024-04-26T22:44:06.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:06 smithi167 ceph-mon[32751]: pgmap v8650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:06.428 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:44:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14927.207837 secs 2024-04-26T22:44:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:06 smithi119 ceph-mon[32752]: pgmap v8650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:44:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14927.207837 secs 2024-04-26T22:44:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:06 smithi090 ceph-mon[29180]: pgmap v8650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:44:07.434 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14927.963486 secs 2024-04-26T22:44:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14927.963486 secs 2024-04-26T22:44:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14927.963486 secs 2024-04-26T22:44:08.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:08 smithi167 ceph-mon[32751]: pgmap v8651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:08 smithi119 ceph-mon[32752]: pgmap v8651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:08 smithi090 ceph-mon[29180]: pgmap v8651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: pgmap v8652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:44:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:44:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: pgmap v8653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T22:44:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: pgmap v8654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: pgmap v8652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: pgmap v8653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T22:44:10.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: pgmap v8654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:10.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: pgmap v8652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: pgmap v8653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-26T22:44:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: pgmap v8654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:44:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14932.207923 secs 2024-04-26T22:44:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14932.207923 secs 2024-04-26T22:44:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14932.207923 secs 2024-04-26T22:44:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14932.963603 secs 2024-04-26T22:44:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:12 smithi090 ceph-mon[29180]: pgmap v8655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14932.963603 secs 2024-04-26T22:44:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:12 smithi119 ceph-mon[32752]: pgmap v8655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14932.963603 secs 2024-04-26T22:44:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:12 smithi167 ceph-mon[32751]: pgmap v8655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:14 smithi090 ceph-mon[29180]: pgmap v8656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:14 smithi119 ceph-mon[32752]: pgmap v8656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:14 smithi167 ceph-mon[32751]: pgmap v8656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:16.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14937.207985 secs 2024-04-26T22:44:16.621 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:16 smithi167 ceph-mon[32751]: pgmap v8657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14937.207985 secs 2024-04-26T22:44:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:16 smithi090 ceph-mon[29180]: pgmap v8657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14937.207985 secs 2024-04-26T22:44:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:16 smithi119 ceph-mon[32752]: pgmap v8657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:44:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14937.963721 secs 2024-04-26T22:44:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14937.963721 secs 2024-04-26T22:44:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14937.963721 secs 2024-04-26T22:44:18.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:18 smithi167 ceph-mon[32751]: pgmap v8658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-26T22:44:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:18 smithi119 ceph-mon[32752]: pgmap v8658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-26T22:44:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:18 smithi090 ceph-mon[29180]: pgmap v8658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-26T22:44:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:20 smithi119 ceph-mon[32752]: pgmap v8659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:44:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:20 smithi167 ceph-mon[32751]: pgmap v8659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:44:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:20 smithi090 ceph-mon[29180]: pgmap v8659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:44:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14942.208046 secs 2024-04-26T22:44:21.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14942.208046 secs 2024-04-26T22:44:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14942.208046 secs 2024-04-26T22:44:22.716 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14942.963841 secs 2024-04-26T22:44:22.717 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:22 smithi119 ceph-mon[32752]: pgmap v8660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:22.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14942.963841 secs 2024-04-26T22:44:22.794 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:22 smithi090 ceph-mon[29180]: pgmap v8660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14942.963841 secs 2024-04-26T22:44:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:22 smithi167 ceph-mon[32751]: pgmap v8660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:24 smithi119 ceph-mon[32752]: pgmap v8661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:24 smithi167 ceph-mon[32751]: pgmap v8661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:24 smithi090 ceph-mon[29180]: pgmap v8661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:26 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14947.208129 secs 2024-04-26T22:44:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:26 smithi119 ceph-mon[32752]: pgmap v8662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:26 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14947.208129 secs 2024-04-26T22:44:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:26 smithi167 ceph-mon[32751]: pgmap v8662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:26 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14947.208129 secs 2024-04-26T22:44:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:26 smithi090 ceph-mon[29180]: pgmap v8662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:27.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14947.963965 secs 2024-04-26T22:44:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14947.963965 secs 2024-04-26T22:44:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14947.963965 secs 2024-04-26T22:44:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:28 smithi119 ceph-mon[32752]: pgmap v8663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:28 smithi167 ceph-mon[32751]: pgmap v8663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:28 smithi090 ceph-mon[29180]: pgmap v8663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:30 smithi119 ceph-mon[32752]: pgmap v8664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:30 smithi167 ceph-mon[32751]: pgmap v8664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:30 smithi090 ceph-mon[29180]: pgmap v8664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14952.208206 secs 2024-04-26T22:44:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14952.208206 secs 2024-04-26T22:44:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14952.208206 secs 2024-04-26T22:44:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14952.964039 secs 2024-04-26T22:44:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:32 smithi119 ceph-mon[32752]: pgmap v8665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14952.964039 secs 2024-04-26T22:44:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:32 smithi167 ceph-mon[32751]: pgmap v8665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14952.964039 secs 2024-04-26T22:44:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:32 smithi090 ceph-mon[29180]: pgmap v8665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:34 smithi119 ceph-mon[32752]: pgmap v8666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:34.849 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:34 smithi167 ceph-mon[32751]: pgmap v8666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:34.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:34 smithi090 ceph-mon[29180]: pgmap v8666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:36 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 14957.208294 secs 2024-04-26T22:44:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:36 smithi119 ceph-mon[32752]: slow request 1440.542428 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:44:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:36 smithi119 ceph-mon[32752]: pgmap v8667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:36 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 14957.208294 secs 2024-04-26T22:44:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:36 smithi167 ceph-mon[32751]: slow request 1440.542428 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:44:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:36 smithi167 ceph-mon[32751]: pgmap v8667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:36 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 14957.208294 secs 2024-04-26T22:44:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:36 smithi090 ceph-mon[29180]: slow request 1440.542428 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:44:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:36 smithi090 ceph-mon[29180]: pgmap v8667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14957.964153 secs 2024-04-26T22:44:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14957.964153 secs 2024-04-26T22:44:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14957.964153 secs 2024-04-26T22:44:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:38 smithi119 ceph-mon[32752]: pgmap v8668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:38 smithi167 ceph-mon[32751]: pgmap v8668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:38 smithi090 ceph-mon[29180]: pgmap v8668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:40 smithi119 ceph-mon[32752]: pgmap v8669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:40 smithi167 ceph-mon[32751]: pgmap v8669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:40 smithi090 ceph-mon[29180]: pgmap v8669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14962.208378 secs 2024-04-26T22:44:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14962.208378 secs 2024-04-26T22:44:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14962.208378 secs 2024-04-26T22:44:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14962.964265 secs 2024-04-26T22:44:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:42 smithi119 ceph-mon[32752]: pgmap v8670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14962.964265 secs 2024-04-26T22:44:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:42 smithi167 ceph-mon[32751]: pgmap v8670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14962.964265 secs 2024-04-26T22:44:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:42 smithi090 ceph-mon[29180]: pgmap v8670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:44 smithi119 ceph-mon[32752]: pgmap v8671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:44 smithi167 ceph-mon[32751]: pgmap v8671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:44 smithi090 ceph-mon[29180]: pgmap v8671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14967.208463 secs 2024-04-26T22:44:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:46 smithi119 ceph-mon[32752]: pgmap v8672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14967.208463 secs 2024-04-26T22:44:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:46 smithi167 ceph-mon[32751]: pgmap v8672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14967.208463 secs 2024-04-26T22:44:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:46 smithi090 ceph-mon[29180]: pgmap v8672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14967.964361 secs 2024-04-26T22:44:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14967.964361 secs 2024-04-26T22:44:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14967.964361 secs 2024-04-26T22:44:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:48 smithi119 ceph-mon[32752]: pgmap v8673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:48 smithi167 ceph-mon[32751]: pgmap v8673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:48 smithi090 ceph-mon[29180]: pgmap v8673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:50 smithi119 ceph-mon[32752]: pgmap v8674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:50 smithi167 ceph-mon[32751]: pgmap v8674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:50 smithi090 ceph-mon[29180]: pgmap v8674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:51.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14972.208550 secs 2024-04-26T22:44:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14972.208550 secs 2024-04-26T22:44:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14972.208550 secs 2024-04-26T22:44:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14972.964441 secs 2024-04-26T22:44:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:52 smithi119 ceph-mon[32752]: pgmap v8675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14972.964441 secs 2024-04-26T22:44:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:52 smithi167 ceph-mon[32751]: pgmap v8675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14972.964441 secs 2024-04-26T22:44:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:52 smithi090 ceph-mon[29180]: pgmap v8675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:54 smithi119 ceph-mon[32752]: pgmap v8676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:54 smithi167 ceph-mon[32751]: pgmap v8676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:54 smithi090 ceph-mon[29180]: pgmap v8676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:44:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14977.208637 secs 2024-04-26T22:44:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:56 smithi119 ceph-mon[32752]: pgmap v8677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14977.208637 secs 2024-04-26T22:44:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:56 smithi167 ceph-mon[32751]: pgmap v8677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14977.208637 secs 2024-04-26T22:44:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:56 smithi090 ceph-mon[29180]: pgmap v8677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14977.964510 secs 2024-04-26T22:44:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14977.964510 secs 2024-04-26T22:44:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14977.964510 secs 2024-04-26T22:44:58.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:44:58 smithi090 ceph-mon[29180]: pgmap v8678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:44:58 smithi119 ceph-mon[32752]: pgmap v8678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:44:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:44:58 smithi167 ceph-mon[32751]: pgmap v8678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:00 smithi119 ceph-mon[32752]: pgmap v8679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:00 smithi167 ceph-mon[32751]: pgmap v8679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:00 smithi090 ceph-mon[29180]: pgmap v8679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14982.208720 secs 2024-04-26T22:45:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14982.208720 secs 2024-04-26T22:45:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14982.208720 secs 2024-04-26T22:45:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14982.964627 secs 2024-04-26T22:45:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:02 smithi119 ceph-mon[32752]: pgmap v8680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14982.964627 secs 2024-04-26T22:45:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:02 smithi167 ceph-mon[32751]: pgmap v8680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14982.964627 secs 2024-04-26T22:45:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:02 smithi090 ceph-mon[29180]: pgmap v8680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:03 smithi119 ceph-mon[32752]: pgmap v8681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:03 smithi167 ceph-mon[32751]: pgmap v8681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:03 smithi090 ceph-mon[29180]: pgmap v8681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14987.208827 secs 2024-04-26T22:45:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:06 smithi119 ceph-mon[32752]: pgmap v8682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14987.208827 secs 2024-04-26T22:45:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:06 smithi167 ceph-mon[32751]: pgmap v8682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14987.208827 secs 2024-04-26T22:45:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:06 smithi090 ceph-mon[29180]: pgmap v8682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14987.964739 secs 2024-04-26T22:45:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14987.964739 secs 2024-04-26T22:45:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14987.964739 secs 2024-04-26T22:45:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:08 smithi119 ceph-mon[32752]: pgmap v8683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:08 smithi167 ceph-mon[32751]: pgmap v8683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:08 smithi090 ceph-mon[29180]: pgmap v8683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:10 smithi119 ceph-mon[32752]: pgmap v8684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:45:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:45:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:45:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:45:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:10 smithi167 ceph-mon[32751]: pgmap v8684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:45:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:45:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:45:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:45:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:10 smithi090 ceph-mon[29180]: pgmap v8684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:45:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:45:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:45:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:45:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:11 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14992.208900 secs 2024-04-26T22:45:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:11 smithi119 ceph-mon[32752]: pgmap v8685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T22:45:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:11 smithi119 ceph-mon[32752]: pgmap v8686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:11 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14992.208900 secs 2024-04-26T22:45:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:11 smithi167 ceph-mon[32751]: pgmap v8685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T22:45:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:11 smithi167 ceph-mon[32751]: pgmap v8686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:11 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14992.208900 secs 2024-04-26T22:45:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:11 smithi090 ceph-mon[29180]: pgmap v8685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T22:45:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:11 smithi090 ceph-mon[29180]: pgmap v8686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14992.964850 secs 2024-04-26T22:45:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14992.964850 secs 2024-04-26T22:45:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14992.964850 secs 2024-04-26T22:45:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:13 smithi119 ceph-mon[32752]: pgmap v8687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T22:45:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:13 smithi167 ceph-mon[32751]: pgmap v8687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T22:45:13.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:13 smithi090 ceph-mon[29180]: pgmap v8687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T22:45:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:15 smithi119 ceph-mon[32752]: pgmap v8688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:15 smithi167 ceph-mon[32751]: pgmap v8688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:15 smithi090 ceph-mon[29180]: pgmap v8688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:16.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 14997.208981 secs 2024-04-26T22:45:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 14997.208981 secs 2024-04-26T22:45:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 14997.208981 secs 2024-04-26T22:45:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 14997.964910 secs 2024-04-26T22:45:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:17 smithi119 ceph-mon[32752]: pgmap v8689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 14997.964910 secs 2024-04-26T22:45:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:17 smithi167 ceph-mon[32751]: pgmap v8689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 14997.964910 secs 2024-04-26T22:45:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:17 smithi090 ceph-mon[29180]: pgmap v8689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:45:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:19 smithi119 ceph-mon[32752]: pgmap v8690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T22:45:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:19 smithi167 ceph-mon[32751]: pgmap v8690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T22:45:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:19 smithi090 ceph-mon[29180]: pgmap v8690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 580 B/s rd, 1 op/s 2024-04-26T22:45:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:21 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15002.209088 secs 2024-04-26T22:45:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:21 smithi119 ceph-mon[32752]: pgmap v8691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:45:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:21 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15002.209088 secs 2024-04-26T22:45:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:21 smithi167 ceph-mon[32751]: pgmap v8691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:45:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:21 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15002.209088 secs 2024-04-26T22:45:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:21 smithi090 ceph-mon[29180]: pgmap v8691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:45:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15002.965017 secs 2024-04-26T22:45:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15002.965017 secs 2024-04-26T22:45:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15002.965017 secs 2024-04-26T22:45:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:23 smithi119 ceph-mon[32752]: pgmap v8692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:23 smithi167 ceph-mon[32751]: pgmap v8692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:23 smithi090 ceph-mon[29180]: pgmap v8692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:24 smithi119 ceph-mon[32752]: pgmap v8693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:24 smithi167 ceph-mon[32751]: pgmap v8693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:24 smithi090 ceph-mon[29180]: pgmap v8693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:25 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15007.209173 secs 2024-04-26T22:45:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:25 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15007.209173 secs 2024-04-26T22:45:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:25 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15007.209173 secs 2024-04-26T22:45:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:26 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15007.965167 secs 2024-04-26T22:45:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:26 smithi090 ceph-mon[29180]: pgmap v8694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:26 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15007.965167 secs 2024-04-26T22:45:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:26 smithi119 ceph-mon[32752]: pgmap v8694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:26 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15007.965167 secs 2024-04-26T22:45:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:26 smithi167 ceph-mon[32751]: pgmap v8694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:29 smithi119 ceph-mon[32752]: pgmap v8695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:29 smithi167 ceph-mon[32751]: pgmap v8695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:29 smithi090 ceph-mon[29180]: pgmap v8695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:31 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15012.209234 secs 2024-04-26T22:45:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:31 smithi119 ceph-mon[32752]: pgmap v8696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:31 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15012.209234 secs 2024-04-26T22:45:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:31 smithi167 ceph-mon[32751]: pgmap v8696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:31 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15012.209234 secs 2024-04-26T22:45:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:31 smithi090 ceph-mon[29180]: pgmap v8696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15012.965306 secs 2024-04-26T22:45:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15012.965306 secs 2024-04-26T22:45:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15012.965306 secs 2024-04-26T22:45:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:33 smithi119 ceph-mon[32752]: pgmap v8697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:33 smithi167 ceph-mon[32751]: pgmap v8697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:33 smithi090 ceph-mon[29180]: pgmap v8697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:35 smithi119 ceph-mon[32752]: pgmap v8698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:35 smithi167 ceph-mon[32751]: pgmap v8698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:35 smithi090 ceph-mon[29180]: pgmap v8698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:36 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15017.209319 secs 2024-04-26T22:45:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:36 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15017.209319 secs 2024-04-26T22:45:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:36 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15017.209319 secs 2024-04-26T22:45:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15017.965437 secs 2024-04-26T22:45:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:37 smithi119 ceph-mon[32752]: pgmap v8699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15017.965437 secs 2024-04-26T22:45:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:37 smithi167 ceph-mon[32751]: pgmap v8699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15017.965437 secs 2024-04-26T22:45:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:37 smithi090 ceph-mon[29180]: pgmap v8699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:39 smithi119 ceph-mon[32752]: pgmap v8700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:39 smithi167 ceph-mon[32751]: pgmap v8700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:39 smithi090 ceph-mon[29180]: pgmap v8700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:41 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15022.209403 secs 2024-04-26T22:45:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:41 smithi119 ceph-mon[32752]: pgmap v8701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:41 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15022.209403 secs 2024-04-26T22:45:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:41 smithi167 ceph-mon[32751]: pgmap v8701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:41 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15022.209403 secs 2024-04-26T22:45:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:41 smithi090 ceph-mon[29180]: pgmap v8701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15022.965529 secs 2024-04-26T22:45:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15022.965529 secs 2024-04-26T22:45:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15022.965529 secs 2024-04-26T22:45:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:43 smithi119 ceph-mon[32752]: pgmap v8702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:43 smithi167 ceph-mon[32751]: pgmap v8702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:43 smithi090 ceph-mon[29180]: pgmap v8702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:45 smithi119 ceph-mon[32752]: pgmap v8703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:45 smithi167 ceph-mon[32751]: pgmap v8703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:45.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:45 smithi090 ceph-mon[29180]: pgmap v8703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:46 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15027.209487 secs 2024-04-26T22:45:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:46 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15027.209487 secs 2024-04-26T22:45:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:46 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15027.209487 secs 2024-04-26T22:45:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15027.965652 secs 2024-04-26T22:45:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:47 smithi167 ceph-mon[32751]: pgmap v8704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15027.965652 secs 2024-04-26T22:45:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:47 smithi090 ceph-mon[29180]: pgmap v8704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15027.965652 secs 2024-04-26T22:45:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:47 smithi119 ceph-mon[32752]: pgmap v8704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:49 smithi167 ceph-mon[32751]: pgmap v8705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:49 smithi090 ceph-mon[29180]: pgmap v8705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:49 smithi119 ceph-mon[32752]: pgmap v8705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:51 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15032.209570 secs 2024-04-26T22:45:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:51 smithi167 ceph-mon[32751]: pgmap v8706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:51.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:51 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15032.209570 secs 2024-04-26T22:45:51.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:51 smithi119 ceph-mon[32752]: pgmap v8706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:51 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15032.209570 secs 2024-04-26T22:45:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:51 smithi090 ceph-mon[29180]: pgmap v8706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15032.965715 secs 2024-04-26T22:45:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15032.965715 secs 2024-04-26T22:45:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15032.965715 secs 2024-04-26T22:45:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:53 smithi090 ceph-mon[29180]: pgmap v8707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:53 smithi119 ceph-mon[32752]: pgmap v8707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:53 smithi167 ceph-mon[32751]: pgmap v8707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:55.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:55 smithi090 ceph-mon[29180]: pgmap v8708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:55 smithi119 ceph-mon[32752]: pgmap v8708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:55 smithi167 ceph-mon[32751]: pgmap v8708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:45:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15037.209656 secs 2024-04-26T22:45:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15037.209656 secs 2024-04-26T22:45:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15037.209656 secs 2024-04-26T22:45:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15037.965802 secs 2024-04-26T22:45:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:57 smithi090 ceph-mon[29180]: pgmap v8709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15037.965802 secs 2024-04-26T22:45:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:57 smithi119 ceph-mon[32752]: pgmap v8709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15037.965802 secs 2024-04-26T22:45:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:57 smithi167 ceph-mon[32751]: pgmap v8709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:45:59 smithi090 ceph-mon[29180]: pgmap v8710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:45:59 smithi119 ceph-mon[32752]: pgmap v8710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:45:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:45:59 smithi167 ceph-mon[32751]: pgmap v8710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:01 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 15042.209744 secs 2024-04-26T22:46:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:01 smithi090 ceph-mon[29180]: pgmap v8711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:01 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 15042.209744 secs 2024-04-26T22:46:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:01 smithi119 ceph-mon[32752]: pgmap v8711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:01 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 15042.209744 secs 2024-04-26T22:46:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:01 smithi167 ceph-mon[32751]: pgmap v8711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15042.965890 secs 2024-04-26T22:46:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15042.965890 secs 2024-04-26T22:46:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15042.965890 secs 2024-04-26T22:46:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:03 smithi090 ceph-mon[29180]: pgmap v8712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:03 smithi119 ceph-mon[32752]: pgmap v8712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:03 smithi167 ceph-mon[32751]: pgmap v8712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:05 smithi090 ceph-mon[29180]: pgmap v8713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:05 smithi119 ceph-mon[32752]: pgmap v8713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:05 smithi167 ceph-mon[32751]: pgmap v8713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:06 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15047.209828 secs 2024-04-26T22:46:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:06 smithi090 ceph-mon[29180]: slow request 181.088451 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:46:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:06 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15047.209828 secs 2024-04-26T22:46:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:06 smithi119 ceph-mon[32752]: slow request 181.088451 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:46:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:06 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15047.209828 secs 2024-04-26T22:46:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:06 smithi167 ceph-mon[32751]: slow request 181.088451 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:46:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15047.966008 secs 2024-04-26T22:46:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:07 smithi090 ceph-mon[29180]: pgmap v8714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:46:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:46:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15047.966008 secs 2024-04-26T22:46:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:07 smithi119 ceph-mon[32752]: pgmap v8714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:46:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:46:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15047.966008 secs 2024-04-26T22:46:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:07 smithi167 ceph-mon[32751]: pgmap v8714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T22:46:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:46:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:09 smithi090 ceph-mon[29180]: pgmap v8715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:09 smithi119 ceph-mon[32752]: pgmap v8715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:09 smithi167 ceph-mon[32751]: pgmap v8715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:46:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15052.209890 secs 2024-04-26T22:46:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:11 smithi090 ceph-mon[29180]: pgmap v8716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:46:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:46:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15052.209890 secs 2024-04-26T22:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:11 smithi119 ceph-mon[32752]: pgmap v8716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:46:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15052.209890 secs 2024-04-26T22:46:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:11 smithi167 ceph-mon[32751]: pgmap v8716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:46:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:46:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15052.966072 secs 2024-04-26T22:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:12 smithi090 ceph-mon[29180]: pgmap v8717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:12 smithi090 ceph-mon[29180]: pgmap v8718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15052.966072 secs 2024-04-26T22:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:12 smithi119 ceph-mon[32752]: pgmap v8717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:12 smithi119 ceph-mon[32752]: pgmap v8718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15052.966072 secs 2024-04-26T22:46:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:12 smithi167 ceph-mon[32751]: pgmap v8717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:46:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:12 smithi167 ceph-mon[32751]: pgmap v8718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:14 smithi090 ceph-mon[29180]: pgmap v8719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:46:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:14 smithi119 ceph-mon[32752]: pgmap v8719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:46:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:14 smithi167 ceph-mon[32751]: pgmap v8719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:46:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15057.209973 secs 2024-04-26T22:46:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:16 smithi090 ceph-mon[29180]: pgmap v8720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15057.209973 secs 2024-04-26T22:46:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:16 smithi119 ceph-mon[32752]: pgmap v8720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15057.209973 secs 2024-04-26T22:46:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:16 smithi167 ceph-mon[32751]: pgmap v8720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15057.966192 secs 2024-04-26T22:46:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15057.966192 secs 2024-04-26T22:46:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15057.966192 secs 2024-04-26T22:46:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:18 smithi119 ceph-mon[32752]: pgmap v8721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:18 smithi167 ceph-mon[32751]: pgmap v8721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:18 smithi090 ceph-mon[29180]: pgmap v8721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:20 smithi119 ceph-mon[32752]: pgmap v8722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:20 smithi167 ceph-mon[32751]: pgmap v8722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:20 smithi090 ceph-mon[29180]: pgmap v8722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:46:21.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15062.210063 secs 2024-04-26T22:46:21.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15062.210063 secs 2024-04-26T22:46:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15062.210063 secs 2024-04-26T22:46:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15062.966307 secs 2024-04-26T22:46:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:22 smithi119 ceph-mon[32752]: pgmap v8723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:46:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15062.966307 secs 2024-04-26T22:46:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:22 smithi167 ceph-mon[32751]: pgmap v8723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:46:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15062.966307 secs 2024-04-26T22:46:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:22 smithi090 ceph-mon[29180]: pgmap v8723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:46:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:24 smithi119 ceph-mon[32752]: pgmap v8724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:24 smithi167 ceph-mon[32751]: pgmap v8724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:24 smithi090 ceph-mon[29180]: pgmap v8724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15067.210125 secs 2024-04-26T22:46:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:26 smithi119 ceph-mon[32752]: pgmap v8725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15067.210125 secs 2024-04-26T22:46:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:26 smithi167 ceph-mon[32751]: pgmap v8725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15067.210125 secs 2024-04-26T22:46:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:26 smithi090 ceph-mon[29180]: pgmap v8725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15067.966440 secs 2024-04-26T22:46:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15067.966440 secs 2024-04-26T22:46:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15067.966440 secs 2024-04-26T22:46:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:28 smithi119 ceph-mon[32752]: pgmap v8726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:28 smithi167 ceph-mon[32751]: pgmap v8726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:28 smithi090 ceph-mon[29180]: pgmap v8726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:30 smithi119 ceph-mon[32752]: pgmap v8727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:30 smithi167 ceph-mon[32751]: pgmap v8727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:30 smithi090 ceph-mon[29180]: pgmap v8727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:31.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15072.210210 secs 2024-04-26T22:46:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15072.210210 secs 2024-04-26T22:46:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15072.210210 secs 2024-04-26T22:46:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15072.966554 secs 2024-04-26T22:46:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:32 smithi119 ceph-mon[32752]: pgmap v8728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15072.966554 secs 2024-04-26T22:46:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:32 smithi167 ceph-mon[32751]: pgmap v8728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15072.966554 secs 2024-04-26T22:46:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:32 smithi090 ceph-mon[29180]: pgmap v8728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:34 smithi119 ceph-mon[32752]: pgmap v8729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:34 smithi167 ceph-mon[32751]: pgmap v8729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:34 smithi090 ceph-mon[29180]: pgmap v8729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15077.210294 secs 2024-04-26T22:46:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:36 smithi119 ceph-mon[32752]: pgmap v8730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15077.210294 secs 2024-04-26T22:46:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:36 smithi167 ceph-mon[32751]: pgmap v8730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15077.210294 secs 2024-04-26T22:46:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:36 smithi090 ceph-mon[29180]: pgmap v8730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15077.966636 secs 2024-04-26T22:46:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15077.966636 secs 2024-04-26T22:46:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15077.966636 secs 2024-04-26T22:46:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:38 smithi119 ceph-mon[32752]: pgmap v8731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:38 smithi167 ceph-mon[32751]: pgmap v8731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:38.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:38 smithi090 ceph-mon[29180]: pgmap v8731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:40 smithi119 ceph-mon[32752]: pgmap v8732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:40 smithi167 ceph-mon[32751]: pgmap v8732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:40 smithi090 ceph-mon[29180]: pgmap v8732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15082.210378 secs 2024-04-26T22:46:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15082.210378 secs 2024-04-26T22:46:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15082.210378 secs 2024-04-26T22:46:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15082.966758 secs 2024-04-26T22:46:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:42 smithi119 ceph-mon[32752]: pgmap v8733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15082.966758 secs 2024-04-26T22:46:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:42 smithi167 ceph-mon[32751]: pgmap v8733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15082.966758 secs 2024-04-26T22:46:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:42 smithi090 ceph-mon[29180]: pgmap v8733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:44 smithi119 ceph-mon[32752]: pgmap v8734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:44 smithi167 ceph-mon[32751]: pgmap v8734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:44 smithi090 ceph-mon[29180]: pgmap v8734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15087.210461 secs 2024-04-26T22:46:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:46 smithi119 ceph-mon[32752]: pgmap v8735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15087.210461 secs 2024-04-26T22:46:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:46 smithi167 ceph-mon[32751]: pgmap v8735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15087.210461 secs 2024-04-26T22:46:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:46 smithi090 ceph-mon[29180]: pgmap v8735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15087.966877 secs 2024-04-26T22:46:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15087.966877 secs 2024-04-26T22:46:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15087.966877 secs 2024-04-26T22:46:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:48 smithi119 ceph-mon[32752]: pgmap v8736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:48 smithi167 ceph-mon[32751]: pgmap v8736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:48 smithi090 ceph-mon[29180]: pgmap v8736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:46:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:50 smithi119 ceph-mon[32752]: pgmap v8737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:50 smithi167 ceph-mon[32751]: pgmap v8737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:50 smithi090 ceph-mon[29180]: pgmap v8737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:46:51.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15092.210546 secs 2024-04-26T22:46:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15092.210546 secs 2024-04-26T22:46:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15092.210546 secs 2024-04-26T22:46:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15092.966997 secs 2024-04-26T22:46:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:52 smithi119 ceph-mon[32752]: pgmap v8738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15092.966997 secs 2024-04-26T22:46:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:52 smithi167 ceph-mon[32751]: pgmap v8738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15092.966997 secs 2024-04-26T22:46:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:52 smithi090 ceph-mon[29180]: pgmap v8738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:54 smithi119 ceph-mon[32752]: pgmap v8739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:54 smithi167 ceph-mon[32751]: pgmap v8739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:54 smithi090 ceph-mon[29180]: pgmap v8739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15097.210628 secs 2024-04-26T22:46:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:56 smithi119 ceph-mon[32752]: pgmap v8740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15097.210628 secs 2024-04-26T22:46:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:56 smithi167 ceph-mon[32751]: pgmap v8740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15097.210628 secs 2024-04-26T22:46:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:56 smithi090 ceph-mon[29180]: pgmap v8740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:46:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15097.967129 secs 2024-04-26T22:46:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15097.967129 secs 2024-04-26T22:46:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15097.967129 secs 2024-04-26T22:46:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:46:58 smithi119 ceph-mon[32752]: pgmap v8741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:46:58 smithi167 ceph-mon[32751]: pgmap v8741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:46:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:46:58 smithi090 ceph-mon[29180]: pgmap v8741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:00 smithi119 ceph-mon[32752]: pgmap v8742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:00 smithi167 ceph-mon[32751]: pgmap v8742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:00 smithi090 ceph-mon[29180]: pgmap v8742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:01 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15102.210712 secs 2024-04-26T22:47:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:01 smithi119 ceph-mon[32752]: slow request 5760.208710 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:47:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:01 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15102.210712 secs 2024-04-26T22:47:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:01 smithi167 ceph-mon[32751]: slow request 5760.208710 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:47:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:01 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15102.210712 secs 2024-04-26T22:47:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:01 smithi090 ceph-mon[29180]: slow request 5760.208710 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:47:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15102.967248 secs 2024-04-26T22:47:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:02 smithi119 ceph-mon[32752]: pgmap v8743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15102.967248 secs 2024-04-26T22:47:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:02 smithi167 ceph-mon[32751]: pgmap v8743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15102.967248 secs 2024-04-26T22:47:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:02 smithi090 ceph-mon[29180]: pgmap v8743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:04 smithi119 ceph-mon[32752]: pgmap v8744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:04 smithi167 ceph-mon[32751]: pgmap v8744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:04 smithi090 ceph-mon[29180]: pgmap v8744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:47:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:47:04.876+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:47:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:47:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:47:04.876+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:47:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:47:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:47:04.876+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:47:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:47:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:47:04.876+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:47:05.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:47:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:47:04.876+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:47:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15107.210814 secs 2024-04-26T22:47:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:06 smithi119 ceph-mon[32752]: pgmap v8745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15107.210814 secs 2024-04-26T22:47:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:06 smithi167 ceph-mon[32751]: pgmap v8745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15107.210814 secs 2024-04-26T22:47:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:06 smithi090 ceph-mon[29180]: pgmap v8745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15107.967411 secs 2024-04-26T22:47:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15107.967411 secs 2024-04-26T22:47:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15107.967411 secs 2024-04-26T22:47:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:08 smithi119 ceph-mon[32752]: pgmap v8746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:08 smithi167 ceph-mon[32751]: pgmap v8746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:08 smithi090 ceph-mon[29180]: pgmap v8746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:09.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:09 smithi119 ceph-mon[32752]: pgmap v8747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:47:09.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:09 smithi167 ceph-mon[32751]: pgmap v8747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:47:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:09 smithi090 ceph-mon[29180]: pgmap v8747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:47:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:10 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15112.210903 secs 2024-04-26T22:47:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:10 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15112.210903 secs 2024-04-26T22:47:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:10 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15112.210903 secs 2024-04-26T22:47:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:11 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15112.967552 secs 2024-04-26T22:47:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:11 smithi119 ceph-mon[32752]: pgmap v8748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:47:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:11 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15112.967552 secs 2024-04-26T22:47:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:11 smithi167 ceph-mon[32751]: pgmap v8748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:47:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:11 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15112.967552 secs 2024-04-26T22:47:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:11 smithi090 ceph-mon[29180]: pgmap v8748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:47:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:47:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:47:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:12 smithi119 ceph-mon[32752]: pgmap v8749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T22:47:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:12 smithi119 ceph-mon[32752]: pgmap v8750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:47:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:47:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:12 smithi167 ceph-mon[32751]: pgmap v8749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T22:47:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:12 smithi167 ceph-mon[32751]: pgmap v8750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:47:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:47:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:47:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:12 smithi090 ceph-mon[29180]: pgmap v8749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 190 B/s wr, 14 op/s 2024-04-26T22:47:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:12 smithi090 ceph-mon[29180]: pgmap v8750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:47:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:14 smithi167 ceph-mon[32751]: pgmap v8751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:14 smithi090 ceph-mon[29180]: pgmap v8751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:14 smithi119 ceph-mon[32752]: pgmap v8751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:15 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15117.210990 secs 2024-04-26T22:47:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:15 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15117.210990 secs 2024-04-26T22:47:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:15 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15117.210990 secs 2024-04-26T22:47:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:16 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15117.967704 secs 2024-04-26T22:47:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:16 smithi090 ceph-mon[29180]: pgmap v8752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:17.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:16 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15117.967704 secs 2024-04-26T22:47:17.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:16 smithi167 ceph-mon[32751]: pgmap v8752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:16 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15117.967704 secs 2024-04-26T22:47:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:16 smithi119 ceph-mon[32752]: pgmap v8752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-26T22:47:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:18 smithi090 ceph-mon[29180]: pgmap v8753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:47:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:18 smithi119 ceph-mon[32752]: pgmap v8753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:47:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:18 smithi167 ceph-mon[32751]: pgmap v8753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:47:21.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:20 smithi119 ceph-mon[32752]: pgmap v8754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:47:21.106 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:20 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15122.211076 secs 2024-04-26T22:47:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:20 smithi090 ceph-mon[29180]: pgmap v8754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:47:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:20 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15122.211076 secs 2024-04-26T22:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:20 smithi167 ceph-mon[32751]: pgmap v8754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:47:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:20 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15122.211076 secs 2024-04-26T22:47:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:21 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15122.967840 secs 2024-04-26T22:47:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:21 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15122.967840 secs 2024-04-26T22:47:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:21 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15122.967840 secs 2024-04-26T22:47:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:22 smithi090 ceph-mon[29180]: pgmap v8755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:47:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:22 smithi119 ceph-mon[32752]: pgmap v8755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:47:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:22 smithi167 ceph-mon[32751]: pgmap v8755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:47:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:24 smithi090 ceph-mon[29180]: pgmap v8756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:24 smithi119 ceph-mon[32752]: pgmap v8756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:24 smithi167 ceph-mon[32751]: pgmap v8756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:25 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15127.211138 secs 2024-04-26T22:47:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:25 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15127.211138 secs 2024-04-26T22:47:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:25 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15127.211138 secs 2024-04-26T22:47:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:26 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15127.967969 secs 2024-04-26T22:47:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:26 smithi090 ceph-mon[29180]: pgmap v8757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:26 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15127.967969 secs 2024-04-26T22:47:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:26 smithi119 ceph-mon[32752]: pgmap v8757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:26 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15127.967969 secs 2024-04-26T22:47:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:26 smithi167 ceph-mon[32751]: pgmap v8757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:28 smithi090 ceph-mon[29180]: pgmap v8758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:28 smithi119 ceph-mon[32752]: pgmap v8758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:28 smithi167 ceph-mon[32751]: pgmap v8758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:30 smithi090 ceph-mon[29180]: pgmap v8759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:30 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15132.211221 secs 2024-04-26T22:47:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:30 smithi119 ceph-mon[32752]: pgmap v8759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:30 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15132.211221 secs 2024-04-26T22:47:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:30 smithi167 ceph-mon[32751]: pgmap v8759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:30 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15132.211221 secs 2024-04-26T22:47:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:31 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15132.968107 secs 2024-04-26T22:47:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:31 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15132.968107 secs 2024-04-26T22:47:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:31 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15132.968107 secs 2024-04-26T22:47:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:32 smithi090 ceph-mon[29180]: pgmap v8760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:32 smithi119 ceph-mon[32752]: pgmap v8760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:32 smithi167 ceph-mon[32751]: pgmap v8760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:34.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:34 smithi090 ceph-mon[29180]: pgmap v8761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:34 smithi119 ceph-mon[32752]: pgmap v8761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:34 smithi167 ceph-mon[32751]: pgmap v8761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:35 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15137.211305 secs 2024-04-26T22:47:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:35 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15137.211305 secs 2024-04-26T22:47:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:35 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15137.211305 secs 2024-04-26T22:47:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:36 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15137.968231 secs 2024-04-26T22:47:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:36 smithi090 ceph-mon[29180]: pgmap v8762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:36 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15137.968231 secs 2024-04-26T22:47:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:36 smithi119 ceph-mon[32752]: pgmap v8762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:36 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15137.968231 secs 2024-04-26T22:47:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:36 smithi167 ceph-mon[32751]: pgmap v8762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:38 smithi090 ceph-mon[29180]: pgmap v8763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:38 smithi119 ceph-mon[32752]: pgmap v8763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:38 smithi167 ceph-mon[32751]: pgmap v8763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:40 smithi119 ceph-mon[32752]: pgmap v8764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:40 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15142.211392 secs 2024-04-26T22:47:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:40 smithi167 ceph-mon[32751]: pgmap v8764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:40 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15142.211392 secs 2024-04-26T22:47:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:40 smithi090 ceph-mon[29180]: pgmap v8764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:40 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15142.211392 secs 2024-04-26T22:47:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:41 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15142.968342 secs 2024-04-26T22:47:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:41 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15142.968342 secs 2024-04-26T22:47:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:41 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15142.968342 secs 2024-04-26T22:47:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:42 smithi119 ceph-mon[32752]: pgmap v8765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:42 smithi167 ceph-mon[32751]: pgmap v8765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:42 smithi090 ceph-mon[29180]: pgmap v8765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:44 smithi119 ceph-mon[32752]: pgmap v8766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:44 smithi167 ceph-mon[32751]: pgmap v8766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:44 smithi090 ceph-mon[29180]: pgmap v8766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:45 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15147.211476 secs 2024-04-26T22:47:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:45 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15147.211476 secs 2024-04-26T22:47:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:45 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15147.211476 secs 2024-04-26T22:47:47.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:46 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15147.968486 secs 2024-04-26T22:47:47.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:46 smithi167 ceph-mon[32751]: pgmap v8767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:46 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15147.968486 secs 2024-04-26T22:47:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:46 smithi119 ceph-mon[32752]: pgmap v8767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:46 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15147.968486 secs 2024-04-26T22:47:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:46 smithi090 ceph-mon[29180]: pgmap v8767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:48 smithi119 ceph-mon[32752]: pgmap v8768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:48 smithi167 ceph-mon[32751]: pgmap v8768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:48 smithi090 ceph-mon[29180]: pgmap v8768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:50 smithi119 ceph-mon[32752]: pgmap v8769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:50 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15152.211564 secs 2024-04-26T22:47:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:50 smithi167 ceph-mon[32751]: pgmap v8769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:50 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15152.211564 secs 2024-04-26T22:47:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:50 smithi090 ceph-mon[29180]: pgmap v8769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:50 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15152.211564 secs 2024-04-26T22:47:52.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:51 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15152.968557 secs 2024-04-26T22:47:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:51 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15152.968557 secs 2024-04-26T22:47:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:51 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15152.968557 secs 2024-04-26T22:47:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:52 smithi119 ceph-mon[32752]: pgmap v8770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:52 smithi167 ceph-mon[32751]: pgmap v8770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:52 smithi090 ceph-mon[29180]: pgmap v8770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:54 smithi119 ceph-mon[32752]: pgmap v8771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:54 smithi167 ceph-mon[32751]: pgmap v8771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:54 smithi090 ceph-mon[29180]: pgmap v8771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:47:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:55 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15157.211648 secs 2024-04-26T22:47:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:55 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15157.211648 secs 2024-04-26T22:47:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:55 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15157.211648 secs 2024-04-26T22:47:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:56 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15157.968674 secs 2024-04-26T22:47:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:56 smithi119 ceph-mon[32752]: pgmap v8772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:56 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15157.968674 secs 2024-04-26T22:47:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:56 smithi167 ceph-mon[32751]: pgmap v8772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:56 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15157.968674 secs 2024-04-26T22:47:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:56 smithi090 ceph-mon[29180]: pgmap v8772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:47:58 smithi119 ceph-mon[32752]: pgmap v8773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:47:58 smithi167 ceph-mon[32751]: pgmap v8773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:47:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:47:58 smithi090 ceph-mon[29180]: pgmap v8773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:00 smithi119 ceph-mon[32752]: pgmap v8774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:00 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15162.211734 secs 2024-04-26T22:48:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:00 smithi167 ceph-mon[32751]: pgmap v8774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:00 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15162.211734 secs 2024-04-26T22:48:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:00 smithi090 ceph-mon[29180]: pgmap v8774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:00 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15162.211734 secs 2024-04-26T22:48:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:01 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15162.968794 secs 2024-04-26T22:48:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:01 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15162.968794 secs 2024-04-26T22:48:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:01 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15162.968794 secs 2024-04-26T22:48:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:02 smithi119 ceph-mon[32752]: pgmap v8775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:02 smithi167 ceph-mon[32751]: pgmap v8775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:02 smithi090 ceph-mon[29180]: pgmap v8775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:04 smithi119 ceph-mon[32752]: pgmap v8776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:04 smithi167 ceph-mon[32751]: pgmap v8776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:04 smithi090 ceph-mon[29180]: pgmap v8776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:05 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15167.211820 secs 2024-04-26T22:48:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:05 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15167.211820 secs 2024-04-26T22:48:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:05 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15167.211820 secs 2024-04-26T22:48:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:06 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15167.968912 secs 2024-04-26T22:48:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:06 smithi119 ceph-mon[32752]: pgmap v8777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:06 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15167.968912 secs 2024-04-26T22:48:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:06 smithi167 ceph-mon[32751]: pgmap v8777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:06 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15167.968912 secs 2024-04-26T22:48:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:06 smithi090 ceph-mon[29180]: pgmap v8777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:08 smithi119 ceph-mon[32752]: pgmap v8778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:08 smithi167 ceph-mon[32751]: pgmap v8778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:08 smithi090 ceph-mon[29180]: pgmap v8778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:10 smithi119 ceph-mon[32752]: pgmap v8779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:10 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15172.211904 secs 2024-04-26T22:48:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:10 smithi167 ceph-mon[32751]: pgmap v8779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:10 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15172.211904 secs 2024-04-26T22:48:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:10 smithi090 ceph-mon[29180]: pgmap v8779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:10 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15172.211904 secs 2024-04-26T22:48:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:11 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15172.969008 secs 2024-04-26T22:48:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:11 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15172.969008 secs 2024-04-26T22:48:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:11 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15172.969008 secs 2024-04-26T22:48:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:48:13.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:13 smithi119 ceph-mon[32752]: pgmap v8780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:13.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.482 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:13 smithi090 ceph-mon[29180]: pgmap v8780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:13 smithi167 ceph-mon[32751]: pgmap v8780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:48:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:15 smithi119 ceph-mon[32752]: pgmap v8781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:15 smithi167 ceph-mon[32751]: pgmap v8781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:15 smithi090 ceph-mon[29180]: pgmap v8781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15177.211969 secs 2024-04-26T22:48:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15177.211969 secs 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15177.211969 secs 2024-04-26T22:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:48:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:48:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:48:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:17 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:48:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:17 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:48:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15177.969088 secs 2024-04-26T22:48:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:17 smithi119 ceph-mon[32752]: pgmap v8782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:17 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:48:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:17 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:48:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:17 smithi119 ceph-mon[32752]: pgmap v8783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:17 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:17 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15177.969088 secs 2024-04-26T22:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:17 smithi167 ceph-mon[32751]: pgmap v8782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:17 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:17 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:48:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:17 smithi167 ceph-mon[32751]: pgmap v8783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:17 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:17 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15177.969088 secs 2024-04-26T22:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:17 smithi090 ceph-mon[29180]: pgmap v8782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:17 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:17 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:48:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:17 smithi090 ceph-mon[29180]: pgmap v8783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:19 smithi119 ceph-mon[32752]: pgmap v8784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:19 smithi167 ceph-mon[32751]: pgmap v8784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:19 smithi090 ceph-mon[29180]: pgmap v8784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:21 smithi119 ceph-mon[32752]: pgmap v8785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15182.212054 secs 2024-04-26T22:48:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:21 smithi167 ceph-mon[32751]: pgmap v8785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15182.212054 secs 2024-04-26T22:48:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:21 smithi090 ceph-mon[29180]: pgmap v8785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15182.212054 secs 2024-04-26T22:48:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15182.969167 secs 2024-04-26T22:48:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15182.969167 secs 2024-04-26T22:48:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15182.969167 secs 2024-04-26T22:48:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:23 smithi119 ceph-mon[32752]: pgmap v8786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:23 smithi167 ceph-mon[32751]: pgmap v8786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:23 smithi090 ceph-mon[29180]: pgmap v8786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:25 smithi119 ceph-mon[32752]: pgmap v8787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:25 smithi167 ceph-mon[32751]: pgmap v8787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:25 smithi090 ceph-mon[29180]: pgmap v8787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15187.212139 secs 2024-04-26T22:48:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15187.212139 secs 2024-04-26T22:48:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15187.212139 secs 2024-04-26T22:48:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15187.969283 secs 2024-04-26T22:48:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:27 smithi119 ceph-mon[32752]: pgmap v8788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15187.969283 secs 2024-04-26T22:48:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:27 smithi167 ceph-mon[32751]: pgmap v8788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15187.969283 secs 2024-04-26T22:48:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:27 smithi090 ceph-mon[29180]: pgmap v8788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-26T22:48:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:29 smithi119 ceph-mon[32752]: pgmap v8789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:29 smithi167 ceph-mon[32751]: pgmap v8789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:29 smithi090 ceph-mon[29180]: pgmap v8789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:31 smithi119 ceph-mon[32752]: pgmap v8790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15192.212200 secs 2024-04-26T22:48:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:31 smithi167 ceph-mon[32751]: pgmap v8790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15192.212200 secs 2024-04-26T22:48:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:31 smithi090 ceph-mon[29180]: pgmap v8790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15192.212200 secs 2024-04-26T22:48:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:32 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15192.969407 secs 2024-04-26T22:48:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:32 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15192.969407 secs 2024-04-26T22:48:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:32 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15192.969407 secs 2024-04-26T22:48:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:33 smithi119 ceph-mon[32752]: pgmap v8791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:33 smithi167 ceph-mon[32751]: pgmap v8791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:33 smithi090 ceph-mon[29180]: pgmap v8791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:35 smithi119 ceph-mon[32752]: pgmap v8792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:35 smithi167 ceph-mon[32751]: pgmap v8792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:35 smithi090 ceph-mon[29180]: pgmap v8792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15197.212285 secs 2024-04-26T22:48:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15197.212285 secs 2024-04-26T22:48:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15197.212285 secs 2024-04-26T22:48:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15197.969523 secs 2024-04-26T22:48:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:37 smithi119 ceph-mon[32752]: pgmap v8793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15197.969523 secs 2024-04-26T22:48:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:37 smithi167 ceph-mon[32751]: pgmap v8793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:37.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15197.969523 secs 2024-04-26T22:48:37.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:37 smithi090 ceph-mon[29180]: pgmap v8793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:39 smithi119 ceph-mon[32752]: pgmap v8794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:39 smithi167 ceph-mon[32751]: pgmap v8794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:39 smithi090 ceph-mon[29180]: pgmap v8794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:41 smithi119 ceph-mon[32752]: pgmap v8795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15202.212372 secs 2024-04-26T22:48:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:41 smithi167 ceph-mon[32751]: pgmap v8795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15202.212372 secs 2024-04-26T22:48:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:41 smithi090 ceph-mon[29180]: pgmap v8795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15202.212372 secs 2024-04-26T22:48:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15202.969674 secs 2024-04-26T22:48:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15202.969674 secs 2024-04-26T22:48:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15202.969674 secs 2024-04-26T22:48:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:43 smithi167 ceph-mon[32751]: pgmap v8796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:43 smithi090 ceph-mon[29180]: pgmap v8796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:43 smithi119 ceph-mon[32752]: pgmap v8796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:45 smithi167 ceph-mon[32751]: pgmap v8797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:45 smithi090 ceph-mon[29180]: pgmap v8797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:45 smithi119 ceph-mon[32752]: pgmap v8797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15207.212459 secs 2024-04-26T22:48:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15207.212459 secs 2024-04-26T22:48:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15207.212459 secs 2024-04-26T22:48:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15207.969736 secs 2024-04-26T22:48:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:47 smithi090 ceph-mon[29180]: pgmap v8798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:47.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15207.969736 secs 2024-04-26T22:48:47.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:47 smithi167 ceph-mon[32751]: pgmap v8798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15207.969736 secs 2024-04-26T22:48:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:47 smithi119 ceph-mon[32752]: pgmap v8798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:49 smithi090 ceph-mon[29180]: pgmap v8799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:49 smithi119 ceph-mon[32752]: pgmap v8799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:49 smithi167 ceph-mon[32751]: pgmap v8799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:51.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:51 smithi119 ceph-mon[32752]: pgmap v8800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:51.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15212.212541 secs 2024-04-26T22:48:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:51 smithi090 ceph-mon[29180]: pgmap v8800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15212.212541 secs 2024-04-26T22:48:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:51 smithi167 ceph-mon[32751]: pgmap v8800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15212.212541 secs 2024-04-26T22:48:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15212.969873 secs 2024-04-26T22:48:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15212.969873 secs 2024-04-26T22:48:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15212.969873 secs 2024-04-26T22:48:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:53 smithi090 ceph-mon[29180]: pgmap v8801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:53 smithi119 ceph-mon[32752]: pgmap v8801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:53 smithi167 ceph-mon[32751]: pgmap v8801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:55 smithi090 ceph-mon[29180]: pgmap v8802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:55 smithi119 ceph-mon[32752]: pgmap v8802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:55 smithi167 ceph-mon[32751]: pgmap v8802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:48:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15217.212618 secs 2024-04-26T22:48:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15217.212618 secs 2024-04-26T22:48:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15217.212618 secs 2024-04-26T22:48:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15217.969970 secs 2024-04-26T22:48:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:57 smithi090 ceph-mon[29180]: pgmap v8803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15217.969970 secs 2024-04-26T22:48:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:57 smithi119 ceph-mon[32752]: pgmap v8803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15217.969970 secs 2024-04-26T22:48:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:57 smithi167 ceph-mon[32751]: pgmap v8803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:48:59 smithi090 ceph-mon[29180]: pgmap v8804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:48:59 smithi119 ceph-mon[32752]: pgmap v8804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:48:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:48:59 smithi167 ceph-mon[32751]: pgmap v8804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:01 smithi090 ceph-mon[29180]: pgmap v8805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15222.212702 secs 2024-04-26T22:49:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:01 smithi119 ceph-mon[32752]: pgmap v8805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15222.212702 secs 2024-04-26T22:49:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:01 smithi167 ceph-mon[32751]: pgmap v8805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15222.212702 secs 2024-04-26T22:49:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15222.970089 secs 2024-04-26T22:49:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15222.970089 secs 2024-04-26T22:49:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15222.970089 secs 2024-04-26T22:49:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:03 smithi090 ceph-mon[29180]: pgmap v8806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:03 smithi119 ceph-mon[32752]: pgmap v8806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:03 smithi167 ceph-mon[32751]: pgmap v8806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:05 smithi090 ceph-mon[29180]: pgmap v8807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:05 smithi119 ceph-mon[32752]: pgmap v8807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:05 smithi167 ceph-mon[32751]: pgmap v8807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:06 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15227.212780 secs 2024-04-26T22:49:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:06 smithi090 ceph-mon[29180]: slow request 361.091403 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:49:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:06 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15227.212780 secs 2024-04-26T22:49:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:06 smithi119 ceph-mon[32752]: slow request 361.091403 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:49:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:06 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15227.212780 secs 2024-04-26T22:49:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:06 smithi167 ceph-mon[32751]: slow request 361.091403 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:49:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:07 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15227.970211 secs 2024-04-26T22:49:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:07 smithi090 ceph-mon[29180]: pgmap v8808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:07 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15227.970211 secs 2024-04-26T22:49:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:07 smithi119 ceph-mon[32752]: pgmap v8808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:07 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15227.970211 secs 2024-04-26T22:49:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:07 smithi167 ceph-mon[32751]: pgmap v8808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:09 smithi090 ceph-mon[29180]: pgmap v8809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:09 smithi119 ceph-mon[32752]: pgmap v8809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:09 smithi167 ceph-mon[32751]: pgmap v8809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:11 smithi090 ceph-mon[29180]: pgmap v8810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15232.212803 secs 2024-04-26T22:49:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:11 smithi119 ceph-mon[32752]: pgmap v8810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15232.212803 secs 2024-04-26T22:49:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:11 smithi167 ceph-mon[32751]: pgmap v8810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15232.212803 secs 2024-04-26T22:49:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:12 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15232.970304 secs 2024-04-26T22:49:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:12 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15232.970304 secs 2024-04-26T22:49:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:12 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15232.970304 secs 2024-04-26T22:49:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:13 smithi119 ceph-mon[32752]: pgmap v8811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:13 smithi167 ceph-mon[32751]: pgmap v8811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:13 smithi090 ceph-mon[29180]: pgmap v8811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:15 smithi119 ceph-mon[32752]: pgmap v8812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:15 smithi167 ceph-mon[32751]: pgmap v8812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:15 smithi090 ceph-mon[29180]: pgmap v8812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15237.212865 secs 2024-04-26T22:49:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15237.212865 secs 2024-04-26T22:49:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15237.212865 secs 2024-04-26T22:49:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:49:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:17 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15237.970421 secs 2024-04-26T22:49:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:17 smithi119 ceph-mon[32752]: pgmap v8813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:17 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15237.970421 secs 2024-04-26T22:49:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:17 smithi167 ceph-mon[32751]: pgmap v8813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:17 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15237.970421 secs 2024-04-26T22:49:17.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:17 smithi090 ceph-mon[29180]: pgmap v8813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:17.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.892 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:17.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:49:17.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:49:17.893 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:49:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:18 smithi119 ceph-mon[32752]: pgmap v8814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T22:49:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:18 smithi119 ceph-mon[32752]: pgmap v8815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:18 smithi167 ceph-mon[32751]: pgmap v8814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T22:49:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:18 smithi167 ceph-mon[32751]: pgmap v8815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:18 smithi090 ceph-mon[29180]: pgmap v8814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T22:49:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:18 smithi090 ceph-mon[29180]: pgmap v8815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:20 smithi119 ceph-mon[32752]: pgmap v8816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:20 smithi167 ceph-mon[32751]: pgmap v8816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:20 smithi090 ceph-mon[29180]: pgmap v8816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15242.212927 secs 2024-04-26T22:49:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15242.212927 secs 2024-04-26T22:49:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15242.212927 secs 2024-04-26T22:49:22.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:22 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15242.970490 secs 2024-04-26T22:49:22.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:22 smithi090 ceph-mon[29180]: pgmap v8817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:22 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15242.970490 secs 2024-04-26T22:49:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:22 smithi119 ceph-mon[32752]: pgmap v8817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:22 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15242.970490 secs 2024-04-26T22:49:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:22 smithi167 ceph-mon[32751]: pgmap v8817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:24 smithi119 ceph-mon[32752]: pgmap v8818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:49:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:24 smithi167 ceph-mon[32751]: pgmap v8818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:49:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:24 smithi090 ceph-mon[29180]: pgmap v8818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T22:49:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15247.212989 secs 2024-04-26T22:49:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:26 smithi119 ceph-mon[32752]: pgmap v8819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15247.212989 secs 2024-04-26T22:49:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:26 smithi167 ceph-mon[32751]: pgmap v8819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15247.212989 secs 2024-04-26T22:49:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:26 smithi090 ceph-mon[29180]: pgmap v8819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:49:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:27 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15247.970609 secs 2024-04-26T22:49:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:27 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15247.970609 secs 2024-04-26T22:49:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:27 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15247.970609 secs 2024-04-26T22:49:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:28 smithi119 ceph-mon[32752]: pgmap v8820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:49:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:28 smithi167 ceph-mon[32751]: pgmap v8820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:49:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:28 smithi090 ceph-mon[29180]: pgmap v8820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:49:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:29 smithi119 ceph-mon[32752]: pgmap v8821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:29 smithi167 ceph-mon[32751]: pgmap v8821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:29 smithi090 ceph-mon[29180]: pgmap v8821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:30 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15252.213046 secs 2024-04-26T22:49:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:30 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15252.213046 secs 2024-04-26T22:49:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:30 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15252.213046 secs 2024-04-26T22:49:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:31 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15252.970730 secs 2024-04-26T22:49:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:31 smithi119 ceph-mon[32752]: pgmap v8822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:31 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15252.970730 secs 2024-04-26T22:49:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:31 smithi167 ceph-mon[32751]: pgmap v8822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:31 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15252.970730 secs 2024-04-26T22:49:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:31 smithi090 ceph-mon[29180]: pgmap v8822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:34 smithi119 ceph-mon[32752]: pgmap v8823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:34 smithi167 ceph-mon[32751]: pgmap v8823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:34 smithi090 ceph-mon[29180]: pgmap v8823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15257.213131 secs 2024-04-26T22:49:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:36 smithi119 ceph-mon[32752]: pgmap v8824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15257.213131 secs 2024-04-26T22:49:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:36 smithi167 ceph-mon[32751]: pgmap v8824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15257.213131 secs 2024-04-26T22:49:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:36 smithi090 ceph-mon[29180]: pgmap v8824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:37 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15257.970879 secs 2024-04-26T22:49:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:37 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15257.970879 secs 2024-04-26T22:49:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:37 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15257.970879 secs 2024-04-26T22:49:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:38 smithi119 ceph-mon[32752]: pgmap v8825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:38 smithi167 ceph-mon[32751]: pgmap v8825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:38 smithi090 ceph-mon[29180]: pgmap v8825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:40 smithi119 ceph-mon[32752]: pgmap v8826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:40 smithi167 ceph-mon[32751]: pgmap v8826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:40 smithi090 ceph-mon[29180]: pgmap v8826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15262.213215 secs 2024-04-26T22:49:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15262.213215 secs 2024-04-26T22:49:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15262.213215 secs 2024-04-26T22:49:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:42 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15262.970971 secs 2024-04-26T22:49:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:42 smithi119 ceph-mon[32752]: pgmap v8827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:42 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15262.970971 secs 2024-04-26T22:49:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:42 smithi167 ceph-mon[32751]: pgmap v8827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:42 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15262.970971 secs 2024-04-26T22:49:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:42 smithi090 ceph-mon[29180]: pgmap v8827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:44 smithi119 ceph-mon[32752]: pgmap v8828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:44 smithi167 ceph-mon[32751]: pgmap v8828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:44 smithi090 ceph-mon[29180]: pgmap v8828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15267.213283 secs 2024-04-26T22:49:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:46 smithi119 ceph-mon[32752]: pgmap v8829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15267.213283 secs 2024-04-26T22:49:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:46 smithi167 ceph-mon[32751]: pgmap v8829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15267.213283 secs 2024-04-26T22:49:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:46 smithi090 ceph-mon[29180]: pgmap v8829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:47 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15267.971106 secs 2024-04-26T22:49:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:47 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15267.971106 secs 2024-04-26T22:49:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:47 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15267.971106 secs 2024-04-26T22:49:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:48 smithi119 ceph-mon[32752]: pgmap v8830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:48 smithi167 ceph-mon[32751]: pgmap v8830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:48 smithi090 ceph-mon[29180]: pgmap v8830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:50 smithi119 ceph-mon[32752]: pgmap v8831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:50 smithi167 ceph-mon[32751]: pgmap v8831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:50 smithi090 ceph-mon[29180]: pgmap v8831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15272.213388 secs 2024-04-26T22:49:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15272.213388 secs 2024-04-26T22:49:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15272.213388 secs 2024-04-26T22:49:52.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:52 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15272.971187 secs 2024-04-26T22:49:52.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:52 smithi090 ceph-mon[29180]: pgmap v8832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:52 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15272.971187 secs 2024-04-26T22:49:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:52 smithi119 ceph-mon[32752]: pgmap v8832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:52 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15272.971187 secs 2024-04-26T22:49:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:52 smithi167 ceph-mon[32751]: pgmap v8832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:54 smithi119 ceph-mon[32752]: pgmap v8833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:54 smithi167 ceph-mon[32751]: pgmap v8833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:54 smithi090 ceph-mon[29180]: pgmap v8833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15277.213447 secs 2024-04-26T22:49:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:56 smithi119 ceph-mon[32752]: pgmap v8834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15277.213447 secs 2024-04-26T22:49:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:56 smithi167 ceph-mon[32751]: pgmap v8834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15277.213447 secs 2024-04-26T22:49:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:56 smithi090 ceph-mon[29180]: pgmap v8834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:49:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:57 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15277.971296 secs 2024-04-26T22:49:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:57 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15277.971296 secs 2024-04-26T22:49:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:57 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15277.971296 secs 2024-04-26T22:49:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:49:58 smithi119 ceph-mon[32752]: pgmap v8835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:49:58 smithi167 ceph-mon[32751]: pgmap v8835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:49:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:49:58 smithi090 ceph-mon[29180]: pgmap v8835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:00 smithi119 ceph-mon[32752]: pgmap v8836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:50:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:00 smithi167 ceph-mon[32751]: pgmap v8836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:50:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:00 smithi090 ceph-mon[29180]: pgmap v8836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T22:50:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15282.213530 secs 2024-04-26T22:50:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15282.213530 secs 2024-04-26T22:50:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15282.213530 secs 2024-04-26T22:50:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:02 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 15282.971445 secs 2024-04-26T22:50:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:02 smithi119 ceph-mon[32752]: pgmap v8837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:02 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 15282.971445 secs 2024-04-26T22:50:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:02 smithi167 ceph-mon[32751]: pgmap v8837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:02 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 15282.971445 secs 2024-04-26T22:50:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:02 smithi090 ceph-mon[29180]: pgmap v8837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:04 smithi119 ceph-mon[32752]: pgmap v8838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:04 smithi167 ceph-mon[32751]: pgmap v8838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:04 smithi090 ceph-mon[29180]: pgmap v8838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15287.213615 secs 2024-04-26T22:50:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:06 smithi119 ceph-mon[32752]: pgmap v8839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15287.213615 secs 2024-04-26T22:50:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:06 smithi167 ceph-mon[32751]: pgmap v8839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15287.213615 secs 2024-04-26T22:50:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:06 smithi090 ceph-mon[29180]: pgmap v8839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:07 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 15287.971520 secs 2024-04-26T22:50:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:07 smithi119 ceph-mon[32752]: slow request 180.839908 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:50:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:07 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 15287.971520 secs 2024-04-26T22:50:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:07 smithi167 ceph-mon[32751]: slow request 180.839908 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:50:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:07 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 15287.971520 secs 2024-04-26T22:50:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:07 smithi090 ceph-mon[29180]: slow request 180.839908 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T22:50:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:08 smithi119 ceph-mon[32752]: pgmap v8840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:08 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T22:50:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:08 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:50:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:08 smithi167 ceph-mon[32751]: pgmap v8840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:08 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T22:50:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:08 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:50:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:08 smithi090 ceph-mon[29180]: pgmap v8840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:08 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T22:50:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:08 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T22:50:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:10 smithi119 ceph-mon[32752]: pgmap v8841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:10 smithi167 ceph-mon[32751]: pgmap v8841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:10 smithi090 ceph-mon[29180]: pgmap v8841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15292.213699 secs 2024-04-26T22:50:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15292.213699 secs 2024-04-26T22:50:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15292.213699 secs 2024-04-26T22:50:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15292.971612 secs 2024-04-26T22:50:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:12 smithi119 ceph-mon[32752]: pgmap v8842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15292.971612 secs 2024-04-26T22:50:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:12 smithi167 ceph-mon[32751]: pgmap v8842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15292.971612 secs 2024-04-26T22:50:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:12 smithi090 ceph-mon[29180]: pgmap v8842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:14 smithi119 ceph-mon[32752]: pgmap v8843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:14 smithi167 ceph-mon[32751]: pgmap v8843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:14 smithi090 ceph-mon[29180]: pgmap v8843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15297.213784 secs 2024-04-26T22:50:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:16 smithi119 ceph-mon[32752]: pgmap v8844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15297.213784 secs 2024-04-26T22:50:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:16 smithi167 ceph-mon[32751]: pgmap v8844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15297.213784 secs 2024-04-26T22:50:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:16 smithi090 ceph-mon[29180]: pgmap v8844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15297.971750 secs 2024-04-26T22:50:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15297.971750 secs 2024-04-26T22:50:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15297.971750 secs 2024-04-26T22:50:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:50:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:18 smithi119 ceph-mon[32752]: pgmap v8845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:50:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:18 smithi167 ceph-mon[32751]: pgmap v8845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:50:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:18 smithi090 ceph-mon[29180]: pgmap v8845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:50:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:50:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:50:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:19 smithi119 ceph-mon[32752]: pgmap v8846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:50:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:19 smithi119 ceph-mon[32752]: pgmap v8847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:50:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:19 smithi167 ceph-mon[32751]: pgmap v8846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:50:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:19 smithi167 ceph-mon[32751]: pgmap v8847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:50:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:19 smithi090 ceph-mon[29180]: pgmap v8846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:50:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:19 smithi090 ceph-mon[29180]: pgmap v8847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:50:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:21 smithi119 ceph-mon[32752]: pgmap v8848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15302.213847 secs 2024-04-26T22:50:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:21 smithi167 ceph-mon[32751]: pgmap v8848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15302.213847 secs 2024-04-26T22:50:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:21 smithi090 ceph-mon[29180]: pgmap v8848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15302.213847 secs 2024-04-26T22:50:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15302.971863 secs 2024-04-26T22:50:22.579 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15302.971863 secs 2024-04-26T22:50:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15302.971863 secs 2024-04-26T22:50:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:23 smithi167 ceph-mon[32751]: pgmap v8849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:23 smithi090 ceph-mon[29180]: pgmap v8849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:23 smithi119 ceph-mon[32752]: pgmap v8849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:25 smithi167 ceph-mon[32751]: pgmap v8850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:25 smithi090 ceph-mon[29180]: pgmap v8850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:25 smithi119 ceph-mon[32752]: pgmap v8850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15307.213928 secs 2024-04-26T22:50:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15307.213928 secs 2024-04-26T22:50:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15307.213928 secs 2024-04-26T22:50:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15307.971976 secs 2024-04-26T22:50:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:27 smithi167 ceph-mon[32751]: pgmap v8851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15307.971976 secs 2024-04-26T22:50:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:27 smithi090 ceph-mon[29180]: pgmap v8851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15307.971976 secs 2024-04-26T22:50:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:27 smithi119 ceph-mon[32752]: pgmap v8851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:50:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:29 smithi090 ceph-mon[29180]: pgmap v8852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:50:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:29 smithi119 ceph-mon[32752]: pgmap v8852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:50:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:29 smithi167 ceph-mon[32751]: pgmap v8852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:50:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:31 smithi090 ceph-mon[29180]: pgmap v8853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15312.214011 secs 2024-04-26T22:50:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:31 smithi119 ceph-mon[32752]: pgmap v8853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15312.214011 secs 2024-04-26T22:50:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:31 smithi167 ceph-mon[32751]: pgmap v8853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15312.214011 secs 2024-04-26T22:50:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15312.972093 secs 2024-04-26T22:50:32.638 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15312.972093 secs 2024-04-26T22:50:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15312.972093 secs 2024-04-26T22:50:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:33 smithi090 ceph-mon[29180]: pgmap v8854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:33 smithi119 ceph-mon[32752]: pgmap v8854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:33 smithi167 ceph-mon[32751]: pgmap v8854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:35 smithi090 ceph-mon[29180]: pgmap v8855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:35 smithi119 ceph-mon[32752]: pgmap v8855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:35 smithi167 ceph-mon[32751]: pgmap v8855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15317.214075 secs 2024-04-26T22:50:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15317.214075 secs 2024-04-26T22:50:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15317.214075 secs 2024-04-26T22:50:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15317.972209 secs 2024-04-26T22:50:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:37 smithi090 ceph-mon[29180]: pgmap v8856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15317.972209 secs 2024-04-26T22:50:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:37 smithi119 ceph-mon[32752]: pgmap v8856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15317.972209 secs 2024-04-26T22:50:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:37 smithi167 ceph-mon[32751]: pgmap v8856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:39 smithi090 ceph-mon[29180]: pgmap v8857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:39 smithi119 ceph-mon[32752]: pgmap v8857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:39 smithi167 ceph-mon[32751]: pgmap v8857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:41 smithi090 ceph-mon[29180]: pgmap v8858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15322.214160 secs 2024-04-26T22:50:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:41 smithi119 ceph-mon[32752]: pgmap v8858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15322.214160 secs 2024-04-26T22:50:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:41 smithi167 ceph-mon[32751]: pgmap v8858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15322.214160 secs 2024-04-26T22:50:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15322.972344 secs 2024-04-26T22:50:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15322.972344 secs 2024-04-26T22:50:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15322.972344 secs 2024-04-26T22:50:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:43 smithi090 ceph-mon[29180]: pgmap v8859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:43 smithi119 ceph-mon[32752]: pgmap v8859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:43 smithi167 ceph-mon[32751]: pgmap v8859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:45 smithi090 ceph-mon[29180]: pgmap v8860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:45 smithi119 ceph-mon[32752]: pgmap v8860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:45 smithi167 ceph-mon[32751]: pgmap v8860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15327.214244 secs 2024-04-26T22:50:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15327.214244 secs 2024-04-26T22:50:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15327.214244 secs 2024-04-26T22:50:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15327.972463 secs 2024-04-26T22:50:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:47 smithi090 ceph-mon[29180]: pgmap v8861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:47.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15327.972463 secs 2024-04-26T22:50:47.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:47 smithi167 ceph-mon[32751]: pgmap v8861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15327.972463 secs 2024-04-26T22:50:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:47 smithi119 ceph-mon[32752]: pgmap v8861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:49 smithi090 ceph-mon[29180]: pgmap v8862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:49 smithi119 ceph-mon[32752]: pgmap v8862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:49 smithi167 ceph-mon[32751]: pgmap v8862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:51 smithi090 ceph-mon[29180]: pgmap v8863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15332.214325 secs 2024-04-26T22:50:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:51 smithi119 ceph-mon[32752]: pgmap v8863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15332.214325 secs 2024-04-26T22:50:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:51 smithi167 ceph-mon[32751]: pgmap v8863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15332.214325 secs 2024-04-26T22:50:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15332.972578 secs 2024-04-26T22:50:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15332.972578 secs 2024-04-26T22:50:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15332.972578 secs 2024-04-26T22:50:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:53 smithi119 ceph-mon[32752]: pgmap v8864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:53 smithi167 ceph-mon[32751]: pgmap v8864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:53 smithi090 ceph-mon[29180]: pgmap v8864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:55 smithi119 ceph-mon[32752]: pgmap v8865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:55 smithi167 ceph-mon[32751]: pgmap v8865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:55 smithi090 ceph-mon[29180]: pgmap v8865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:50:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15337.214408 secs 2024-04-26T22:50:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15337.214408 secs 2024-04-26T22:50:56.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15337.214408 secs 2024-04-26T22:50:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15337.972694 secs 2024-04-26T22:50:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:57 smithi119 ceph-mon[32752]: pgmap v8866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15337.972694 secs 2024-04-26T22:50:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:57 smithi167 ceph-mon[32751]: pgmap v8866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15337.972694 secs 2024-04-26T22:50:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:57 smithi090 ceph-mon[29180]: pgmap v8866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:50:59 smithi119 ceph-mon[32752]: pgmap v8867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:50:59 smithi167 ceph-mon[32751]: pgmap v8867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:50:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:50:59 smithi090 ceph-mon[29180]: pgmap v8867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:01 smithi119 ceph-mon[32752]: pgmap v8868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:01 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15342.214492 secs 2024-04-26T22:51:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:01 smithi119 ceph-mon[32752]: slow request 11524.260123 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:51:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:01 smithi167 ceph-mon[32751]: pgmap v8868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:01 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15342.214492 secs 2024-04-26T22:51:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:01 smithi167 ceph-mon[32751]: slow request 11524.260123 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:51:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:01 smithi090 ceph-mon[29180]: pgmap v8868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:01 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15342.214492 secs 2024-04-26T22:51:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:01 smithi090 ceph-mon[29180]: slow request 11524.260123 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:51:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15342.972774 secs 2024-04-26T22:51:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15342.972774 secs 2024-04-26T22:51:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15342.972774 secs 2024-04-26T22:51:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:03 smithi119 ceph-mon[32752]: pgmap v8869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:03 smithi167 ceph-mon[32751]: pgmap v8869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:03 smithi090 ceph-mon[29180]: pgmap v8869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:05 smithi119 ceph-mon[32752]: pgmap v8870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:05 smithi167 ceph-mon[32751]: pgmap v8870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:05 smithi090 ceph-mon[29180]: pgmap v8870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15347.214575 secs 2024-04-26T22:51:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15347.214575 secs 2024-04-26T22:51:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15347.214575 secs 2024-04-26T22:51:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15347.972891 secs 2024-04-26T22:51:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:07 smithi119 ceph-mon[32752]: pgmap v8871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15347.972891 secs 2024-04-26T22:51:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:07 smithi167 ceph-mon[32751]: pgmap v8871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15347.972891 secs 2024-04-26T22:51:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:07 smithi090 ceph-mon[29180]: pgmap v8871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:09 smithi119 ceph-mon[32752]: pgmap v8872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:09 smithi167 ceph-mon[32751]: pgmap v8872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:09 smithi090 ceph-mon[29180]: pgmap v8872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:11 smithi119 ceph-mon[32752]: pgmap v8873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15352.214658 secs 2024-04-26T22:51:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:11 smithi167 ceph-mon[32751]: pgmap v8873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15352.214658 secs 2024-04-26T22:51:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:11 smithi090 ceph-mon[29180]: pgmap v8873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15352.214658 secs 2024-04-26T22:51:12.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15352.973029 secs 2024-04-26T22:51:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15352.973029 secs 2024-04-26T22:51:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15352.973029 secs 2024-04-26T22:51:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:13 smithi119 ceph-mon[32752]: pgmap v8874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:13 smithi167 ceph-mon[32751]: pgmap v8874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:13 smithi090 ceph-mon[29180]: pgmap v8874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:15 smithi119 ceph-mon[32752]: pgmap v8875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:15 smithi167 ceph-mon[32751]: pgmap v8875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:15 smithi090 ceph-mon[29180]: pgmap v8875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15357.214723 secs 2024-04-26T22:51:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15357.214723 secs 2024-04-26T22:51:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15357.214723 secs 2024-04-26T22:51:17.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15357.973146 secs 2024-04-26T22:51:17.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:17 smithi167 ceph-mon[32751]: pgmap v8876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15357.973146 secs 2024-04-26T22:51:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:17 smithi119 ceph-mon[32752]: pgmap v8876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15357.973146 secs 2024-04-26T22:51:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:17 smithi090 ceph-mon[29180]: pgmap v8876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:51:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:19 smithi119 ceph-mon[32752]: pgmap v8877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:51:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:19 smithi167 ceph-mon[32751]: pgmap v8877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:51:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:19 smithi090 ceph-mon[29180]: pgmap v8877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:51:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:51:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:51:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:20 smithi119 ceph-mon[32752]: pgmap v8878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:51:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:20 smithi119 ceph-mon[32752]: pgmap v8879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:20 smithi167 ceph-mon[32751]: pgmap v8878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:51:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:20 smithi167 ceph-mon[32751]: pgmap v8879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:20 smithi090 ceph-mon[29180]: pgmap v8878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:51:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:20 smithi090 ceph-mon[29180]: pgmap v8879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15362.214813 secs 2024-04-26T22:51:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15362.214813 secs 2024-04-26T22:51:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15362.214813 secs 2024-04-26T22:51:22.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:22 smithi090 ceph-mon[29180]: pgmap v8880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:22.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15362.973218 secs 2024-04-26T22:51:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:22 smithi119 ceph-mon[32752]: pgmap v8880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15362.973218 secs 2024-04-26T22:51:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:22 smithi167 ceph-mon[32751]: pgmap v8880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15362.973218 secs 2024-04-26T22:51:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:24 smithi119 ceph-mon[32752]: pgmap v8881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:51:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:24 smithi167 ceph-mon[32751]: pgmap v8881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:51:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:24 smithi090 ceph-mon[29180]: pgmap v8881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:51:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:26 smithi119 ceph-mon[32752]: pgmap v8882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15367.214896 secs 2024-04-26T22:51:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:26 smithi167 ceph-mon[32751]: pgmap v8882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15367.214896 secs 2024-04-26T22:51:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:26 smithi090 ceph-mon[29180]: pgmap v8882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15367.214896 secs 2024-04-26T22:51:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15367.973299 secs 2024-04-26T22:51:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15367.973299 secs 2024-04-26T22:51:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15367.973299 secs 2024-04-26T22:51:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:28 smithi119 ceph-mon[32752]: pgmap v8883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:28 smithi167 ceph-mon[32751]: pgmap v8883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:28 smithi090 ceph-mon[29180]: pgmap v8883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:51:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:30 smithi119 ceph-mon[32752]: pgmap v8884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:51:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:30 smithi167 ceph-mon[32751]: pgmap v8884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:51:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:30 smithi090 ceph-mon[29180]: pgmap v8884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:51:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15372.214979 secs 2024-04-26T22:51:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15372.214979 secs 2024-04-26T22:51:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15372.214979 secs 2024-04-26T22:51:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:32 smithi119 ceph-mon[32752]: pgmap v8885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15372.973397 secs 2024-04-26T22:51:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:32 smithi167 ceph-mon[32751]: pgmap v8885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15372.973397 secs 2024-04-26T22:51:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:32 smithi090 ceph-mon[29180]: pgmap v8885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15372.973397 secs 2024-04-26T22:51:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:34 smithi119 ceph-mon[32752]: pgmap v8886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:34 smithi167 ceph-mon[32751]: pgmap v8886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:34 smithi090 ceph-mon[29180]: pgmap v8886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:36 smithi119 ceph-mon[32752]: pgmap v8887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15377.215040 secs 2024-04-26T22:51:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:36 smithi167 ceph-mon[32751]: pgmap v8887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15377.215040 secs 2024-04-26T22:51:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:36 smithi090 ceph-mon[29180]: pgmap v8887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15377.215040 secs 2024-04-26T22:51:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15377.973510 secs 2024-04-26T22:51:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15377.973510 secs 2024-04-26T22:51:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15377.973510 secs 2024-04-26T22:51:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:38 smithi119 ceph-mon[32752]: pgmap v8888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:38 smithi167 ceph-mon[32751]: pgmap v8888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:38 smithi090 ceph-mon[29180]: pgmap v8888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:40 smithi119 ceph-mon[32752]: pgmap v8889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:40 smithi167 ceph-mon[32751]: pgmap v8889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:40 smithi090 ceph-mon[29180]: pgmap v8889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15382.215102 secs 2024-04-26T22:51:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15382.215102 secs 2024-04-26T22:51:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15382.215102 secs 2024-04-26T22:51:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:42 smithi119 ceph-mon[32752]: pgmap v8890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15382.973623 secs 2024-04-26T22:51:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:42 smithi167 ceph-mon[32751]: pgmap v8890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15382.973623 secs 2024-04-26T22:51:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:42 smithi090 ceph-mon[29180]: pgmap v8890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15382.973623 secs 2024-04-26T22:51:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:44 smithi119 ceph-mon[32752]: pgmap v8891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:44 smithi167 ceph-mon[32751]: pgmap v8891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:44 smithi090 ceph-mon[29180]: pgmap v8891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:46 smithi119 ceph-mon[32752]: pgmap v8892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15387.215187 secs 2024-04-26T22:51:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:46 smithi167 ceph-mon[32751]: pgmap v8892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15387.215187 secs 2024-04-26T22:51:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:46 smithi090 ceph-mon[29180]: pgmap v8892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15387.215187 secs 2024-04-26T22:51:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15387.973691 secs 2024-04-26T22:51:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15387.973691 secs 2024-04-26T22:51:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15387.973691 secs 2024-04-26T22:51:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:48 smithi119 ceph-mon[32752]: pgmap v8893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:48 smithi167 ceph-mon[32751]: pgmap v8893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:48 smithi090 ceph-mon[29180]: pgmap v8893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:50 smithi119 ceph-mon[32752]: pgmap v8894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:50 smithi167 ceph-mon[32751]: pgmap v8894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:50 smithi090 ceph-mon[29180]: pgmap v8894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15392.215272 secs 2024-04-26T22:51:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15392.215272 secs 2024-04-26T22:51:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15392.215272 secs 2024-04-26T22:51:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:52 smithi119 ceph-mon[32752]: pgmap v8895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15392.973790 secs 2024-04-26T22:51:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:52 smithi167 ceph-mon[32751]: pgmap v8895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15392.973790 secs 2024-04-26T22:51:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:52 smithi090 ceph-mon[29180]: pgmap v8895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15392.973790 secs 2024-04-26T22:51:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:53 smithi167 ceph-mon[32751]: pgmap v8896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:53 smithi090 ceph-mon[29180]: pgmap v8896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:53 smithi119 ceph-mon[32752]: pgmap v8896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:55 smithi119 ceph-mon[32752]: pgmap v8897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:55 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15397.215355 secs 2024-04-26T22:51:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:55 smithi167 ceph-mon[32751]: pgmap v8897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:55 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15397.215355 secs 2024-04-26T22:51:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:55 smithi090 ceph-mon[29180]: pgmap v8897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:51:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:55 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15397.215355 secs 2024-04-26T22:51:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15397.973905 secs 2024-04-26T22:51:57.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15397.973905 secs 2024-04-26T22:51:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15397.973905 secs 2024-04-26T22:51:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:57 smithi119 ceph-mon[32752]: pgmap v8898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:57 smithi167 ceph-mon[32751]: pgmap v8898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:51:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:57 smithi090 ceph-mon[29180]: pgmap v8898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:00.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:51:59 smithi119 ceph-mon[32752]: pgmap v8899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:51:59 smithi167 ceph-mon[32751]: pgmap v8899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:51:59 smithi090 ceph-mon[29180]: pgmap v8899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:00 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15402.215437 secs 2024-04-26T22:52:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:00 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15402.215437 secs 2024-04-26T22:52:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:00 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15402.215437 secs 2024-04-26T22:52:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:01 smithi119 ceph-mon[32752]: pgmap v8900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15402.974019 secs 2024-04-26T22:52:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:01 smithi167 ceph-mon[32751]: pgmap v8900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15402.974019 secs 2024-04-26T22:52:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:01 smithi090 ceph-mon[29180]: pgmap v8900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15402.974019 secs 2024-04-26T22:52:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:03 smithi119 ceph-mon[32752]: pgmap v8901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:03 smithi167 ceph-mon[32751]: pgmap v8901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:03 smithi090 ceph-mon[29180]: pgmap v8901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:05.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:52:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:52:04.929+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:52:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:52:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:52:04.929+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:52:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:52:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:52:04.929+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:52:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:52:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:52:04.929+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:52:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:52:04 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:52:04.929+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:52:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:05 smithi119 ceph-mon[32752]: pgmap v8902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:05 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15407.215521 secs 2024-04-26T22:52:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:05 smithi119 ceph-mon[32752]: slow request 5761.623764 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:52:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:05 smithi167 ceph-mon[32751]: pgmap v8902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:05 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15407.215521 secs 2024-04-26T22:52:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:05 smithi167 ceph-mon[32751]: slow request 5761.623764 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:52:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:05 smithi090 ceph-mon[29180]: pgmap v8902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:05 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15407.215521 secs 2024-04-26T22:52:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:05 smithi090 ceph-mon[29180]: slow request 5761.623764 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:52:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15407.974116 secs 2024-04-26T22:52:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15407.974116 secs 2024-04-26T22:52:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15407.974116 secs 2024-04-26T22:52:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:07 smithi119 ceph-mon[32752]: pgmap v8903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:07 smithi167 ceph-mon[32751]: pgmap v8903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:07 smithi090 ceph-mon[29180]: pgmap v8903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:09 smithi119 ceph-mon[32752]: pgmap v8904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:09 smithi167 ceph-mon[32751]: pgmap v8904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:09 smithi090 ceph-mon[29180]: pgmap v8904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:10 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15412.215602 secs 2024-04-26T22:52:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:10 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15412.215602 secs 2024-04-26T22:52:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:10 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15412.215602 secs 2024-04-26T22:52:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:11 smithi119 ceph-mon[32752]: pgmap v8905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15412.974233 secs 2024-04-26T22:52:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:11 smithi167 ceph-mon[32751]: pgmap v8905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15412.974233 secs 2024-04-26T22:52:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:11 smithi090 ceph-mon[29180]: pgmap v8905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15412.974233 secs 2024-04-26T22:52:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:13 smithi119 ceph-mon[32752]: pgmap v8906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:13 smithi167 ceph-mon[32751]: pgmap v8906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:13 smithi090 ceph-mon[29180]: pgmap v8906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:15 smithi119 ceph-mon[32752]: pgmap v8907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:15 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15417.215687 secs 2024-04-26T22:52:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:15 smithi167 ceph-mon[32751]: pgmap v8907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:15 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15417.215687 secs 2024-04-26T22:52:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:15 smithi090 ceph-mon[29180]: pgmap v8907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:15 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15417.215687 secs 2024-04-26T22:52:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15417.974357 secs 2024-04-26T22:52:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15417.974357 secs 2024-04-26T22:52:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15417.974357 secs 2024-04-26T22:52:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:17 smithi167 ceph-mon[32751]: pgmap v8908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:17 smithi090 ceph-mon[29180]: pgmap v8908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:17 smithi119 ceph-mon[32752]: pgmap v8908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:52:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:19 smithi090 ceph-mon[29180]: pgmap v8909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:19 smithi090 ceph-mon[29180]: pgmap v8910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:52:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:19 smithi090 ceph-mon[29180]: pgmap v8911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:52:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:19 smithi119 ceph-mon[32752]: pgmap v8909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:19 smithi119 ceph-mon[32752]: pgmap v8910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:52:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:19 smithi119 ceph-mon[32752]: pgmap v8911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:52:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:19 smithi167 ceph-mon[32751]: pgmap v8909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T22:52:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:52:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:52:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:19 smithi167 ceph-mon[32751]: pgmap v8910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:52:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:19 smithi167 ceph-mon[32751]: pgmap v8911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:52:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:20 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15422.215773 secs 2024-04-26T22:52:21.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:20 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15422.215773 secs 2024-04-26T22:52:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:20 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15422.215773 secs 2024-04-26T22:52:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15422.974472 secs 2024-04-26T22:52:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:21 smithi090 ceph-mon[29180]: pgmap v8912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15422.974472 secs 2024-04-26T22:52:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:21 smithi119 ceph-mon[32752]: pgmap v8912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15422.974472 secs 2024-04-26T22:52:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:21 smithi167 ceph-mon[32751]: pgmap v8912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:24 smithi119 ceph-mon[32752]: pgmap v8913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:24 smithi167 ceph-mon[32751]: pgmap v8913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:24 smithi090 ceph-mon[29180]: pgmap v8913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15427.215827 secs 2024-04-26T22:52:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:26 smithi119 ceph-mon[32752]: pgmap v8914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15427.215827 secs 2024-04-26T22:52:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:26 smithi167 ceph-mon[32751]: pgmap v8914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15427.215827 secs 2024-04-26T22:52:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:26 smithi090 ceph-mon[29180]: pgmap v8914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:52:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15427.974593 secs 2024-04-26T22:52:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15427.974593 secs 2024-04-26T22:52:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15427.974593 secs 2024-04-26T22:52:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:28 smithi119 ceph-mon[32752]: pgmap v8915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:52:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:28 smithi167 ceph-mon[32751]: pgmap v8915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:52:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:28 smithi090 ceph-mon[29180]: pgmap v8915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:52:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:30 smithi119 ceph-mon[32752]: pgmap v8916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:52:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:30 smithi167 ceph-mon[32751]: pgmap v8916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:52:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:30 smithi090 ceph-mon[29180]: pgmap v8916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:52:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15432.215912 secs 2024-04-26T22:52:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15432.215912 secs 2024-04-26T22:52:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15432.215912 secs 2024-04-26T22:52:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15432.974712 secs 2024-04-26T22:52:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:32 smithi119 ceph-mon[32752]: pgmap v8917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15432.974712 secs 2024-04-26T22:52:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:32 smithi167 ceph-mon[32751]: pgmap v8917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15432.974712 secs 2024-04-26T22:52:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:32 smithi090 ceph-mon[29180]: pgmap v8917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:34 smithi119 ceph-mon[32752]: pgmap v8918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:34 smithi167 ceph-mon[32751]: pgmap v8918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:34 smithi090 ceph-mon[29180]: pgmap v8918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:35 smithi167 ceph-mon[32751]: pgmap v8919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:35 smithi090 ceph-mon[29180]: pgmap v8919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:35 smithi119 ceph-mon[32752]: pgmap v8919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15437.215998 secs 2024-04-26T22:52:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15437.215998 secs 2024-04-26T22:52:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15437.215998 secs 2024-04-26T22:52:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15437.974836 secs 2024-04-26T22:52:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:37 smithi167 ceph-mon[32751]: pgmap v8920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15437.974836 secs 2024-04-26T22:52:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:37 smithi090 ceph-mon[29180]: pgmap v8920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15437.974836 secs 2024-04-26T22:52:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:37 smithi119 ceph-mon[32752]: pgmap v8920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:40 smithi119 ceph-mon[32752]: pgmap v8921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:40 smithi167 ceph-mon[32751]: pgmap v8921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:40 smithi090 ceph-mon[29180]: pgmap v8921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15442.216084 secs 2024-04-26T22:52:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15442.216084 secs 2024-04-26T22:52:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15442.216084 secs 2024-04-26T22:52:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15442.974915 secs 2024-04-26T22:52:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:42 smithi119 ceph-mon[32752]: pgmap v8922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15442.974915 secs 2024-04-26T22:52:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:42 smithi167 ceph-mon[32751]: pgmap v8922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15442.974915 secs 2024-04-26T22:52:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:42 smithi090 ceph-mon[29180]: pgmap v8922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:44 smithi119 ceph-mon[32752]: pgmap v8923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:44 smithi167 ceph-mon[32751]: pgmap v8923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:44 smithi090 ceph-mon[29180]: pgmap v8923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15447.216167 secs 2024-04-26T22:52:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:46 smithi119 ceph-mon[32752]: pgmap v8924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15447.216167 secs 2024-04-26T22:52:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:46 smithi167 ceph-mon[32751]: pgmap v8924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15447.216167 secs 2024-04-26T22:52:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:46 smithi090 ceph-mon[29180]: pgmap v8924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15447.974979 secs 2024-04-26T22:52:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15447.974979 secs 2024-04-26T22:52:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15447.974979 secs 2024-04-26T22:52:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:48 smithi119 ceph-mon[32752]: pgmap v8925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:48 smithi167 ceph-mon[32751]: pgmap v8925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:48 smithi090 ceph-mon[29180]: pgmap v8925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:50 smithi119 ceph-mon[32752]: pgmap v8926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:50 smithi167 ceph-mon[32751]: pgmap v8926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:50 smithi090 ceph-mon[29180]: pgmap v8926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15452.216229 secs 2024-04-26T22:52:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15452.216229 secs 2024-04-26T22:52:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15452.216229 secs 2024-04-26T22:52:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15452.975123 secs 2024-04-26T22:52:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:52 smithi167 ceph-mon[32751]: pgmap v8927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15452.975123 secs 2024-04-26T22:52:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:52 smithi090 ceph-mon[29180]: pgmap v8927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15452.975123 secs 2024-04-26T22:52:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:52 smithi119 ceph-mon[32752]: pgmap v8927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:53 smithi167 ceph-mon[32751]: pgmap v8928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:53 smithi090 ceph-mon[29180]: pgmap v8928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:53 smithi119 ceph-mon[32752]: pgmap v8928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:52:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15457.216313 secs 2024-04-26T22:52:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:56 smithi119 ceph-mon[32752]: pgmap v8929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15457.216313 secs 2024-04-26T22:52:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:56 smithi167 ceph-mon[32751]: pgmap v8929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15457.216313 secs 2024-04-26T22:52:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:56 smithi090 ceph-mon[29180]: pgmap v8929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15457.975247 secs 2024-04-26T22:52:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15457.975247 secs 2024-04-26T22:52:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15457.975247 secs 2024-04-26T22:52:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:52:58 smithi119 ceph-mon[32752]: pgmap v8930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:52:58 smithi167 ceph-mon[32751]: pgmap v8930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:52:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:52:58 smithi090 ceph-mon[29180]: pgmap v8930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:00 smithi119 ceph-mon[32752]: pgmap v8931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:00 smithi167 ceph-mon[32751]: pgmap v8931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:00 smithi090 ceph-mon[29180]: pgmap v8931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15462.216397 secs 2024-04-26T22:53:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15462.216397 secs 2024-04-26T22:53:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15462.216397 secs 2024-04-26T22:53:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15462.975320 secs 2024-04-26T22:53:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:02 smithi119 ceph-mon[32752]: pgmap v8932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15462.975320 secs 2024-04-26T22:53:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:02 smithi167 ceph-mon[32751]: pgmap v8932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15462.975320 secs 2024-04-26T22:53:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:02 smithi090 ceph-mon[29180]: pgmap v8932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:03 smithi167 ceph-mon[32751]: pgmap v8933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:03 smithi090 ceph-mon[29180]: pgmap v8933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:03 smithi119 ceph-mon[32752]: pgmap v8933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15467.216481 secs 2024-04-26T22:53:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:06 smithi119 ceph-mon[32752]: pgmap v8934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15467.216481 secs 2024-04-26T22:53:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:06 smithi167 ceph-mon[32751]: pgmap v8934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15467.216481 secs 2024-04-26T22:53:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:06 smithi090 ceph-mon[29180]: pgmap v8934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:07 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 15467.975446 secs 2024-04-26T22:53:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:07 smithi119 ceph-mon[32752]: slow request 360.843834 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:53:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:07 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 15467.975446 secs 2024-04-26T22:53:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:07 smithi167 ceph-mon[32751]: slow request 360.843834 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:53:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:07 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 15467.975446 secs 2024-04-26T22:53:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:07 smithi090 ceph-mon[29180]: slow request 360.843834 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:53:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:08 smithi119 ceph-mon[32752]: pgmap v8935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:08 smithi167 ceph-mon[32751]: pgmap v8935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:08 smithi090 ceph-mon[29180]: pgmap v8935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:10 smithi119 ceph-mon[32752]: pgmap v8936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:10 smithi167 ceph-mon[32751]: pgmap v8936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:10 smithi090 ceph-mon[29180]: pgmap v8936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15472.216569 secs 2024-04-26T22:53:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15472.216569 secs 2024-04-26T22:53:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15472.216569 secs 2024-04-26T22:53:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15472.975572 secs 2024-04-26T22:53:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:12 smithi167 ceph-mon[32751]: pgmap v8937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15472.975572 secs 2024-04-26T22:53:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:12 smithi090 ceph-mon[29180]: pgmap v8937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15472.975572 secs 2024-04-26T22:53:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:12 smithi119 ceph-mon[32752]: pgmap v8937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:13 smithi167 ceph-mon[32751]: pgmap v8938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:13 smithi090 ceph-mon[29180]: pgmap v8938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:13 smithi119 ceph-mon[32752]: pgmap v8938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15477.216630 secs 2024-04-26T22:53:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:16 smithi119 ceph-mon[32752]: pgmap v8939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15477.216630 secs 2024-04-26T22:53:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:16 smithi167 ceph-mon[32751]: pgmap v8939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15477.216630 secs 2024-04-26T22:53:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:16 smithi090 ceph-mon[29180]: pgmap v8939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15477.975653 secs 2024-04-26T22:53:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15477.975653 secs 2024-04-26T22:53:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15477.975653 secs 2024-04-26T22:53:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:18 smithi119 ceph-mon[32752]: pgmap v8940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:18 smithi167 ceph-mon[32751]: pgmap v8940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:18 smithi090 ceph-mon[29180]: pgmap v8940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:20 smithi167 ceph-mon[32751]: pgmap v8941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:53:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:20 smithi090 ceph-mon[29180]: pgmap v8941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:53:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:20 smithi119 ceph-mon[32752]: pgmap v8941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:53:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:53:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:53:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:53:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15482.216718 secs 2024-04-26T22:53:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:21 smithi167 ceph-mon[32751]: pgmap v8942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:53:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:21 smithi167 ceph-mon[32751]: pgmap v8943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15482.216718 secs 2024-04-26T22:53:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:21 smithi090 ceph-mon[29180]: pgmap v8942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:53:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:21 smithi090 ceph-mon[29180]: pgmap v8943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15482.216718 secs 2024-04-26T22:53:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:21 smithi119 ceph-mon[32752]: pgmap v8942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:53:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:21 smithi119 ceph-mon[32752]: pgmap v8943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15482.975776 secs 2024-04-26T22:53:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:22 smithi167 ceph-mon[32751]: pgmap v8944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:53:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15482.975776 secs 2024-04-26T22:53:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:22 smithi090 ceph-mon[29180]: pgmap v8944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:53:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15482.975776 secs 2024-04-26T22:53:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:22 smithi119 ceph-mon[32752]: pgmap v8944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:53:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:25 smithi167 ceph-mon[32751]: pgmap v8945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:25 smithi090 ceph-mon[29180]: pgmap v8945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:25 smithi119 ceph-mon[32752]: pgmap v8945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15487.216803 secs 2024-04-26T22:53:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15487.216803 secs 2024-04-26T22:53:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15487.216803 secs 2024-04-26T22:53:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15487.975916 secs 2024-04-26T22:53:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:27 smithi090 ceph-mon[29180]: pgmap v8946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15487.975916 secs 2024-04-26T22:53:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:27 smithi119 ceph-mon[32752]: pgmap v8946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15487.975916 secs 2024-04-26T22:53:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:27 smithi167 ceph-mon[32751]: pgmap v8946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:53:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:29 smithi090 ceph-mon[29180]: pgmap v8947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:53:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:29 smithi119 ceph-mon[32752]: pgmap v8947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:53:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:29 smithi167 ceph-mon[32751]: pgmap v8947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:53:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15492.216888 secs 2024-04-26T22:53:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:31 smithi090 ceph-mon[29180]: pgmap v8948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:53:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15492.216888 secs 2024-04-26T22:53:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:31 smithi119 ceph-mon[32752]: pgmap v8948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:53:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15492.216888 secs 2024-04-26T22:53:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:31 smithi167 ceph-mon[32751]: pgmap v8948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:53:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15492.976033 secs 2024-04-26T22:53:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15492.976033 secs 2024-04-26T22:53:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15492.976033 secs 2024-04-26T22:53:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:33 smithi090 ceph-mon[29180]: pgmap v8949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:33 smithi119 ceph-mon[32752]: pgmap v8949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:33 smithi167 ceph-mon[32751]: pgmap v8949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:35 smithi090 ceph-mon[29180]: pgmap v8950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:35 smithi119 ceph-mon[32752]: pgmap v8950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:35 smithi167 ceph-mon[32751]: pgmap v8950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15497.216950 secs 2024-04-26T22:53:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15497.216950 secs 2024-04-26T22:53:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15497.216950 secs 2024-04-26T22:53:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15497.976154 secs 2024-04-26T22:53:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:37 smithi090 ceph-mon[29180]: pgmap v8951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15497.976154 secs 2024-04-26T22:53:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:37 smithi119 ceph-mon[32752]: pgmap v8951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15497.976154 secs 2024-04-26T22:53:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:37 smithi167 ceph-mon[32751]: pgmap v8951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:39 smithi090 ceph-mon[29180]: pgmap v8952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:39 smithi119 ceph-mon[32752]: pgmap v8952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:39 smithi167 ceph-mon[32751]: pgmap v8952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15502.217013 secs 2024-04-26T22:53:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:41 smithi090 ceph-mon[29180]: pgmap v8953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15502.217013 secs 2024-04-26T22:53:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:41 smithi119 ceph-mon[32752]: pgmap v8953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15502.217013 secs 2024-04-26T22:53:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:41 smithi167 ceph-mon[32751]: pgmap v8953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15502.976216 secs 2024-04-26T22:53:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15502.976216 secs 2024-04-26T22:53:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15502.976216 secs 2024-04-26T22:53:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:43 smithi090 ceph-mon[29180]: pgmap v8954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:43 smithi119 ceph-mon[32752]: pgmap v8954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:43 smithi167 ceph-mon[32751]: pgmap v8954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:45 smithi090 ceph-mon[29180]: pgmap v8955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:45 smithi119 ceph-mon[32752]: pgmap v8955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:45 smithi167 ceph-mon[32751]: pgmap v8955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15507.217072 secs 2024-04-26T22:53:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15507.217072 secs 2024-04-26T22:53:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15507.217072 secs 2024-04-26T22:53:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15507.976336 secs 2024-04-26T22:53:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:47 smithi090 ceph-mon[29180]: pgmap v8956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:47.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15507.976336 secs 2024-04-26T22:53:47.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:47 smithi167 ceph-mon[32751]: pgmap v8956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15507.976336 secs 2024-04-26T22:53:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:47 smithi119 ceph-mon[32752]: pgmap v8956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:49 smithi090 ceph-mon[29180]: pgmap v8957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:49 smithi119 ceph-mon[32752]: pgmap v8957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:49 smithi167 ceph-mon[32751]: pgmap v8957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15512.217158 secs 2024-04-26T22:53:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:51 smithi090 ceph-mon[29180]: pgmap v8958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15512.217158 secs 2024-04-26T22:53:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:51 smithi119 ceph-mon[32752]: pgmap v8958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15512.217158 secs 2024-04-26T22:53:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:51 smithi167 ceph-mon[32751]: pgmap v8958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15512.976452 secs 2024-04-26T22:53:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15512.976452 secs 2024-04-26T22:53:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15512.976452 secs 2024-04-26T22:53:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:53 smithi119 ceph-mon[32752]: pgmap v8959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:53 smithi167 ceph-mon[32751]: pgmap v8959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:53 smithi090 ceph-mon[29180]: pgmap v8959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:55 smithi119 ceph-mon[32752]: pgmap v8960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:55 smithi167 ceph-mon[32751]: pgmap v8960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:55 smithi090 ceph-mon[29180]: pgmap v8960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:53:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15517.217246 secs 2024-04-26T22:53:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15517.217246 secs 2024-04-26T22:53:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15517.217246 secs 2024-04-26T22:53:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15517.976516 secs 2024-04-26T22:53:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:57 smithi119 ceph-mon[32752]: pgmap v8961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15517.976516 secs 2024-04-26T22:53:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:57 smithi167 ceph-mon[32751]: pgmap v8961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15517.976516 secs 2024-04-26T22:53:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:57 smithi090 ceph-mon[29180]: pgmap v8961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:53:59 smithi119 ceph-mon[32752]: pgmap v8962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:53:59 smithi167 ceph-mon[32751]: pgmap v8962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:53:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:53:59 smithi090 ceph-mon[29180]: pgmap v8962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15522.217331 secs 2024-04-26T22:54:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:01 smithi119 ceph-mon[32752]: pgmap v8963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15522.217331 secs 2024-04-26T22:54:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:01 smithi167 ceph-mon[32751]: pgmap v8963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15522.217331 secs 2024-04-26T22:54:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:01 smithi090 ceph-mon[29180]: pgmap v8963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15522.976594 secs 2024-04-26T22:54:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15522.976594 secs 2024-04-26T22:54:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15522.976594 secs 2024-04-26T22:54:03.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:03 smithi090 ceph-mon[29180]: pgmap v8964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:03 smithi119 ceph-mon[32752]: pgmap v8964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:03 smithi167 ceph-mon[32751]: pgmap v8964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:05 smithi119 ceph-mon[32752]: pgmap v8965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:05 smithi167 ceph-mon[32751]: pgmap v8965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:05 smithi090 ceph-mon[29180]: pgmap v8965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15527.217416 secs 2024-04-26T22:54:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15527.217416 secs 2024-04-26T22:54:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15527.217416 secs 2024-04-26T22:54:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15527.976736 secs 2024-04-26T22:54:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:07 smithi119 ceph-mon[32752]: pgmap v8966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15527.976736 secs 2024-04-26T22:54:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:07 smithi167 ceph-mon[32751]: pgmap v8966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15527.976736 secs 2024-04-26T22:54:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:07 smithi090 ceph-mon[29180]: pgmap v8966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:09 smithi119 ceph-mon[32752]: pgmap v8967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:09 smithi167 ceph-mon[32751]: pgmap v8967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:09 smithi090 ceph-mon[29180]: pgmap v8967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15532.217500 secs 2024-04-26T22:54:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:11 smithi119 ceph-mon[32752]: pgmap v8968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15532.217500 secs 2024-04-26T22:54:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:11 smithi167 ceph-mon[32751]: pgmap v8968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15532.217500 secs 2024-04-26T22:54:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:11 smithi090 ceph-mon[29180]: pgmap v8968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15532.976822 secs 2024-04-26T22:54:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15532.976822 secs 2024-04-26T22:54:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15532.976822 secs 2024-04-26T22:54:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:13 smithi119 ceph-mon[32752]: pgmap v8969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:13 smithi167 ceph-mon[32751]: pgmap v8969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:13 smithi090 ceph-mon[29180]: pgmap v8969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:15 smithi119 ceph-mon[32752]: pgmap v8970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:15 smithi167 ceph-mon[32751]: pgmap v8970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:15 smithi090 ceph-mon[29180]: pgmap v8970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15537.217582 secs 2024-04-26T22:54:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15537.217582 secs 2024-04-26T22:54:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15537.217582 secs 2024-04-26T22:54:17.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15537.976936 secs 2024-04-26T22:54:17.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:17 smithi167 ceph-mon[32751]: pgmap v8971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15537.976936 secs 2024-04-26T22:54:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:17 smithi119 ceph-mon[32752]: pgmap v8971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15537.976936 secs 2024-04-26T22:54:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:17 smithi090 ceph-mon[29180]: pgmap v8971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:19 smithi119 ceph-mon[32752]: pgmap v8972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:19 smithi167 ceph-mon[32751]: pgmap v8972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:19 smithi090 ceph-mon[29180]: pgmap v8972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:54:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15542.217666 secs 2024-04-26T22:54:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:21 smithi119 ceph-mon[32752]: pgmap v8973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:54:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15542.217666 secs 2024-04-26T22:54:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:21 smithi167 ceph-mon[32751]: pgmap v8973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:54:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15542.217666 secs 2024-04-26T22:54:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:21 smithi090 ceph-mon[29180]: pgmap v8973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:54:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:54:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:54:22.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15542.977024 secs 2024-04-26T22:54:22.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:22 smithi090 ceph-mon[29180]: pgmap v8974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:54:22.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:22 smithi090 ceph-mon[29180]: pgmap v8975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15542.977024 secs 2024-04-26T22:54:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:22 smithi119 ceph-mon[32752]: pgmap v8974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:54:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:22 smithi119 ceph-mon[32752]: pgmap v8975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15542.977024 secs 2024-04-26T22:54:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:22 smithi167 ceph-mon[32751]: pgmap v8974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:54:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:22 smithi167 ceph-mon[32751]: pgmap v8975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:24 smithi119 ceph-mon[32752]: pgmap v8976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:54:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:24 smithi167 ceph-mon[32751]: pgmap v8976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:54:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:24 smithi090 ceph-mon[29180]: pgmap v8976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:54:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15547.217749 secs 2024-04-26T22:54:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:26 smithi119 ceph-mon[32752]: pgmap v8977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15547.217749 secs 2024-04-26T22:54:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:26 smithi167 ceph-mon[32751]: pgmap v8977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15547.217749 secs 2024-04-26T22:54:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:26 smithi090 ceph-mon[29180]: pgmap v8977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15547.977163 secs 2024-04-26T22:54:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15547.977163 secs 2024-04-26T22:54:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15547.977163 secs 2024-04-26T22:54:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:28 smithi119 ceph-mon[32752]: pgmap v8978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:28 smithi167 ceph-mon[32751]: pgmap v8978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:28 smithi090 ceph-mon[29180]: pgmap v8978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:30 smithi119 ceph-mon[32752]: pgmap v8979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:30 smithi167 ceph-mon[32751]: pgmap v8979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:30 smithi090 ceph-mon[29180]: pgmap v8979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:54:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15552.217834 secs 2024-04-26T22:54:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15552.217834 secs 2024-04-26T22:54:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15552.217834 secs 2024-04-26T22:54:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15552.977301 secs 2024-04-26T22:54:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:32 smithi119 ceph-mon[32752]: pgmap v8980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:54:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15552.977301 secs 2024-04-26T22:54:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:32 smithi167 ceph-mon[32751]: pgmap v8980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:54:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15552.977301 secs 2024-04-26T22:54:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:32 smithi090 ceph-mon[29180]: pgmap v8980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:54:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:34 smithi119 ceph-mon[32752]: pgmap v8981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:34 smithi167 ceph-mon[32751]: pgmap v8981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:34 smithi090 ceph-mon[29180]: pgmap v8981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15557.217895 secs 2024-04-26T22:54:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:36 smithi119 ceph-mon[32752]: pgmap v8982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15557.217895 secs 2024-04-26T22:54:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:36 smithi167 ceph-mon[32751]: pgmap v8982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15557.217895 secs 2024-04-26T22:54:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:36 smithi090 ceph-mon[29180]: pgmap v8982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15557.977400 secs 2024-04-26T22:54:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15557.977400 secs 2024-04-26T22:54:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15557.977400 secs 2024-04-26T22:54:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:38 smithi119 ceph-mon[32752]: pgmap v8983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:38 smithi167 ceph-mon[32751]: pgmap v8983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:38 smithi090 ceph-mon[29180]: pgmap v8983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:40 smithi119 ceph-mon[32752]: pgmap v8984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:40 smithi167 ceph-mon[32751]: pgmap v8984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:40 smithi090 ceph-mon[29180]: pgmap v8984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15562.217956 secs 2024-04-26T22:54:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15562.217956 secs 2024-04-26T22:54:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15562.217956 secs 2024-04-26T22:54:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15562.977447 secs 2024-04-26T22:54:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:42 smithi119 ceph-mon[32752]: pgmap v8985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15562.977447 secs 2024-04-26T22:54:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:42 smithi167 ceph-mon[32751]: pgmap v8985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15562.977447 secs 2024-04-26T22:54:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:42 smithi090 ceph-mon[29180]: pgmap v8985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:44 smithi119 ceph-mon[32752]: pgmap v8986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:44 smithi167 ceph-mon[32751]: pgmap v8986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:44 smithi090 ceph-mon[29180]: pgmap v8986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15567.218019 secs 2024-04-26T22:54:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:46 smithi119 ceph-mon[32752]: pgmap v8987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15567.218019 secs 2024-04-26T22:54:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:46 smithi167 ceph-mon[32751]: pgmap v8987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15567.218019 secs 2024-04-26T22:54:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:46 smithi090 ceph-mon[29180]: pgmap v8987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15567.977558 secs 2024-04-26T22:54:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15567.977558 secs 2024-04-26T22:54:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15567.977558 secs 2024-04-26T22:54:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:48 smithi119 ceph-mon[32752]: pgmap v8988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:48 smithi167 ceph-mon[32751]: pgmap v8988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:48 smithi090 ceph-mon[29180]: pgmap v8988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:50 smithi119 ceph-mon[32752]: pgmap v8989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:50 smithi167 ceph-mon[32751]: pgmap v8989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:50 smithi090 ceph-mon[29180]: pgmap v8989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15572.218094 secs 2024-04-26T22:54:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15572.218094 secs 2024-04-26T22:54:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15572.218094 secs 2024-04-26T22:54:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15572.977682 secs 2024-04-26T22:54:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:52 smithi119 ceph-mon[32752]: pgmap v8990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15572.977682 secs 2024-04-26T22:54:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:52 smithi167 ceph-mon[32751]: pgmap v8990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15572.977682 secs 2024-04-26T22:54:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:52 smithi090 ceph-mon[29180]: pgmap v8990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:54 smithi119 ceph-mon[32752]: pgmap v8991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:54 smithi167 ceph-mon[32751]: pgmap v8991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:54 smithi090 ceph-mon[29180]: pgmap v8991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15577.218180 secs 2024-04-26T22:54:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:56 smithi119 ceph-mon[32752]: pgmap v8992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15577.218180 secs 2024-04-26T22:54:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:56 smithi167 ceph-mon[32751]: pgmap v8992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15577.218180 secs 2024-04-26T22:54:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:56 smithi090 ceph-mon[29180]: pgmap v8992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:54:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15577.977805 secs 2024-04-26T22:54:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15577.977805 secs 2024-04-26T22:54:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15577.977805 secs 2024-04-26T22:54:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:54:58 smithi167 ceph-mon[32751]: pgmap v8993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:54:58 smithi090 ceph-mon[29180]: pgmap v8993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:54:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:54:58 smithi119 ceph-mon[32752]: pgmap v8993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:00 smithi167 ceph-mon[32751]: pgmap v8994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:00 smithi090 ceph-mon[29180]: pgmap v8994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:00 smithi119 ceph-mon[32752]: pgmap v8994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15582.218241 secs 2024-04-26T22:55:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15582.218241 secs 2024-04-26T22:55:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15582.218241 secs 2024-04-26T22:55:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15582.977933 secs 2024-04-26T22:55:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:02 smithi090 ceph-mon[29180]: pgmap v8995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15582.977933 secs 2024-04-26T22:55:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:02 smithi119 ceph-mon[32752]: pgmap v8995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15582.977933 secs 2024-04-26T22:55:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:02 smithi167 ceph-mon[32751]: pgmap v8995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:03 smithi090 ceph-mon[29180]: pgmap v8996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:03 smithi119 ceph-mon[32752]: pgmap v8996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:03 smithi167 ceph-mon[32751]: pgmap v8996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:06 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15587.218326 secs 2024-04-26T22:55:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:06 smithi090 ceph-mon[29180]: slow request 721.096949 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:55:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:06 smithi090 ceph-mon[29180]: pgmap v8997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:06 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15587.218326 secs 2024-04-26T22:55:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:06 smithi119 ceph-mon[32752]: slow request 721.096949 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:55:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:06 smithi119 ceph-mon[32752]: pgmap v8997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:06 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15587.218326 secs 2024-04-26T22:55:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:06 smithi167 ceph-mon[32751]: slow request 721.096949 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T22:55:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:06 smithi167 ceph-mon[32751]: pgmap v8997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:07 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 15587.978075 secs 2024-04-26T22:55:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:07 smithi090 ceph-mon[29180]: slow request 1441.135936 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:55:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:07 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 15587.978075 secs 2024-04-26T22:55:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:07 smithi119 ceph-mon[32752]: slow request 1441.135936 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:55:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:07 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 15587.978075 secs 2024-04-26T22:55:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:07 smithi167 ceph-mon[32751]: slow request 1441.135936 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:55:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:08 smithi090 ceph-mon[29180]: pgmap v8998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:08 smithi119 ceph-mon[32752]: pgmap v8998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:08 smithi167 ceph-mon[32751]: pgmap v8998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:10 smithi090 ceph-mon[29180]: pgmap v8999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:10 smithi119 ceph-mon[32752]: pgmap v8999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:10 smithi167 ceph-mon[32751]: pgmap v8999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15592.218410 secs 2024-04-26T22:55:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15592.218410 secs 2024-04-26T22:55:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15592.218410 secs 2024-04-26T22:55:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15592.978157 secs 2024-04-26T22:55:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:12 smithi090 ceph-mon[29180]: pgmap v9000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15592.978157 secs 2024-04-26T22:55:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:12 smithi119 ceph-mon[32752]: pgmap v9000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15592.978157 secs 2024-04-26T22:55:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:12 smithi167 ceph-mon[32751]: pgmap v9000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:14 smithi090 ceph-mon[29180]: pgmap v9001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:14 smithi119 ceph-mon[32752]: pgmap v9001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:14 smithi167 ceph-mon[32751]: pgmap v9001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15597.218495 secs 2024-04-26T22:55:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:16 smithi119 ceph-mon[32752]: pgmap v9002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15597.218495 secs 2024-04-26T22:55:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:16 smithi167 ceph-mon[32751]: pgmap v9002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15597.218495 secs 2024-04-26T22:55:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:16 smithi090 ceph-mon[29180]: pgmap v9002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:17.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15597.978297 secs 2024-04-26T22:55:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15597.978297 secs 2024-04-26T22:55:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15597.978297 secs 2024-04-26T22:55:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:18 smithi119 ceph-mon[32752]: pgmap v9003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:18 smithi167 ceph-mon[32751]: pgmap v9003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:18 smithi090 ceph-mon[29180]: pgmap v9003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:20 smithi119 ceph-mon[32752]: pgmap v9004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:20 smithi167 ceph-mon[32751]: pgmap v9004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:20 smithi090 ceph-mon[29180]: pgmap v9004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:21.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15602.218581 secs 2024-04-26T22:55:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15602.218581 secs 2024-04-26T22:55:21.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15602.218581 secs 2024-04-26T22:55:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15602.978409 secs 2024-04-26T22:55:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:22 smithi119 ceph-mon[32752]: pgmap v9005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:55:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15602.978409 secs 2024-04-26T22:55:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:22 smithi167 ceph-mon[32751]: pgmap v9005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:55:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15602.978409 secs 2024-04-26T22:55:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:22 smithi090 ceph-mon[29180]: pgmap v9005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:55:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:55:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:55:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:55:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:23 smithi119 ceph-mon[32752]: pgmap v9006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:55:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:23 smithi119 ceph-mon[32752]: pgmap v9007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:23 smithi167 ceph-mon[32751]: pgmap v9006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:55:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:23 smithi167 ceph-mon[32751]: pgmap v9007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:23 smithi090 ceph-mon[29180]: pgmap v9006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T22:55:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:23 smithi090 ceph-mon[29180]: pgmap v9007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:25 smithi119 ceph-mon[32752]: pgmap v9008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:25 smithi167 ceph-mon[32751]: pgmap v9008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:25 smithi090 ceph-mon[29180]: pgmap v9008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15607.218664 secs 2024-04-26T22:55:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15607.218664 secs 2024-04-26T22:55:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15607.218664 secs 2024-04-26T22:55:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15607.978470 secs 2024-04-26T22:55:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:27 smithi119 ceph-mon[32752]: pgmap v9009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15607.978470 secs 2024-04-26T22:55:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:27 smithi167 ceph-mon[32751]: pgmap v9009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15607.978470 secs 2024-04-26T22:55:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:27 smithi090 ceph-mon[29180]: pgmap v9009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:29 smithi119 ceph-mon[32752]: pgmap v9010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:55:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:29 smithi167 ceph-mon[32751]: pgmap v9010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:55:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:29 smithi090 ceph-mon[29180]: pgmap v9010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:55:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:31 smithi119 ceph-mon[32752]: pgmap v9011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15612.218748 secs 2024-04-26T22:55:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:31 smithi167 ceph-mon[32751]: pgmap v9011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15612.218748 secs 2024-04-26T22:55:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:31 smithi090 ceph-mon[29180]: pgmap v9011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:55:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15612.218748 secs 2024-04-26T22:55:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15612.978535 secs 2024-04-26T22:55:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15612.978535 secs 2024-04-26T22:55:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15612.978535 secs 2024-04-26T22:55:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:33 smithi119 ceph-mon[32752]: pgmap v9012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:55:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:33 smithi167 ceph-mon[32751]: pgmap v9012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:55:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:33 smithi090 ceph-mon[29180]: pgmap v9012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:55:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:35 smithi119 ceph-mon[32752]: pgmap v9013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:35 smithi167 ceph-mon[32751]: pgmap v9013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:35 smithi090 ceph-mon[29180]: pgmap v9013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15617.218824 secs 2024-04-26T22:55:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15617.218824 secs 2024-04-26T22:55:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15617.218824 secs 2024-04-26T22:55:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15617.978637 secs 2024-04-26T22:55:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:37 smithi119 ceph-mon[32752]: pgmap v9014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15617.978637 secs 2024-04-26T22:55:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:37 smithi167 ceph-mon[32751]: pgmap v9014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15617.978637 secs 2024-04-26T22:55:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:37 smithi090 ceph-mon[29180]: pgmap v9014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:39 smithi119 ceph-mon[32752]: pgmap v9015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:39 smithi167 ceph-mon[32751]: pgmap v9015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:39 smithi090 ceph-mon[29180]: pgmap v9015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:41 smithi119 ceph-mon[32752]: pgmap v9016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15622.218886 secs 2024-04-26T22:55:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:41 smithi167 ceph-mon[32751]: pgmap v9016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15622.218886 secs 2024-04-26T22:55:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:41 smithi090 ceph-mon[29180]: pgmap v9016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15622.218886 secs 2024-04-26T22:55:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15622.978749 secs 2024-04-26T22:55:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15622.978749 secs 2024-04-26T22:55:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15622.978749 secs 2024-04-26T22:55:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:43 smithi119 ceph-mon[32752]: pgmap v9017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:43 smithi167 ceph-mon[32751]: pgmap v9017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:43 smithi090 ceph-mon[29180]: pgmap v9017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:45 smithi119 ceph-mon[32752]: pgmap v9018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:45 smithi167 ceph-mon[32751]: pgmap v9018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:45 smithi090 ceph-mon[29180]: pgmap v9018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15627.218969 secs 2024-04-26T22:55:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15627.218969 secs 2024-04-26T22:55:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15627.218969 secs 2024-04-26T22:55:47.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15627.978864 secs 2024-04-26T22:55:47.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:47 smithi167 ceph-mon[32751]: pgmap v9019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15627.978864 secs 2024-04-26T22:55:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:47 smithi119 ceph-mon[32752]: pgmap v9019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15627.978864 secs 2024-04-26T22:55:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:47 smithi090 ceph-mon[29180]: pgmap v9019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:49 smithi119 ceph-mon[32752]: pgmap v9020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:49 smithi167 ceph-mon[32751]: pgmap v9020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:49 smithi090 ceph-mon[29180]: pgmap v9020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:51 smithi119 ceph-mon[32752]: pgmap v9021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15632.219030 secs 2024-04-26T22:55:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:51 smithi167 ceph-mon[32751]: pgmap v9021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15632.219030 secs 2024-04-26T22:55:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:51 smithi090 ceph-mon[29180]: pgmap v9021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15632.219030 secs 2024-04-26T22:55:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15632.978976 secs 2024-04-26T22:55:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15632.978976 secs 2024-04-26T22:55:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15632.978976 secs 2024-04-26T22:55:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:53 smithi119 ceph-mon[32752]: pgmap v9022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:53 smithi167 ceph-mon[32751]: pgmap v9022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:53 smithi090 ceph-mon[29180]: pgmap v9022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:55 smithi119 ceph-mon[32752]: pgmap v9023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:55 smithi167 ceph-mon[32751]: pgmap v9023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:55 smithi090 ceph-mon[29180]: pgmap v9023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:55:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15637.219093 secs 2024-04-26T22:55:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15637.219093 secs 2024-04-26T22:55:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15637.219093 secs 2024-04-26T22:55:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15637.979115 secs 2024-04-26T22:55:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:57 smithi119 ceph-mon[32752]: pgmap v9024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15637.979115 secs 2024-04-26T22:55:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:57 smithi167 ceph-mon[32751]: pgmap v9024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15637.979115 secs 2024-04-26T22:55:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:57 smithi090 ceph-mon[29180]: pgmap v9024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:55:59 smithi119 ceph-mon[32752]: pgmap v9025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:55:59 smithi167 ceph-mon[32751]: pgmap v9025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:55:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:55:59 smithi090 ceph-mon[29180]: pgmap v9025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:01 smithi119 ceph-mon[32752]: pgmap v9026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15642.219177 secs 2024-04-26T22:56:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:01 smithi167 ceph-mon[32751]: pgmap v9026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15642.219177 secs 2024-04-26T22:56:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:01 smithi090 ceph-mon[29180]: pgmap v9026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15642.219177 secs 2024-04-26T22:56:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15642.979208 secs 2024-04-26T22:56:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15642.979208 secs 2024-04-26T22:56:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15642.979208 secs 2024-04-26T22:56:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:03 smithi119 ceph-mon[32752]: pgmap v9027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:03 smithi167 ceph-mon[32751]: pgmap v9027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:03 smithi090 ceph-mon[29180]: pgmap v9027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:05 smithi119 ceph-mon[32752]: pgmap v9028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:05.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:05 smithi167 ceph-mon[32751]: pgmap v9028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:05 smithi090 ceph-mon[29180]: pgmap v9028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15647.219263 secs 2024-04-26T22:56:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15647.219263 secs 2024-04-26T22:56:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15647.219263 secs 2024-04-26T22:56:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15647.979350 secs 2024-04-26T22:56:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:07 smithi119 ceph-mon[32752]: pgmap v9029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15647.979350 secs 2024-04-26T22:56:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:07 smithi167 ceph-mon[32751]: pgmap v9029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15647.979350 secs 2024-04-26T22:56:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:07 smithi090 ceph-mon[29180]: pgmap v9029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:09 smithi119 ceph-mon[32752]: pgmap v9030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:09 smithi167 ceph-mon[32751]: pgmap v9030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:09 smithi090 ceph-mon[29180]: pgmap v9030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:11 smithi119 ceph-mon[32752]: pgmap v9031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15652.219359 secs 2024-04-26T22:56:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:11 smithi167 ceph-mon[32751]: pgmap v9031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15652.219359 secs 2024-04-26T22:56:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:11 smithi090 ceph-mon[29180]: pgmap v9031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15652.219359 secs 2024-04-26T22:56:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15652.979475 secs 2024-04-26T22:56:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15652.979475 secs 2024-04-26T22:56:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15652.979475 secs 2024-04-26T22:56:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:13 smithi119 ceph-mon[32752]: pgmap v9032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:13 smithi167 ceph-mon[32751]: pgmap v9032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:13 smithi090 ceph-mon[29180]: pgmap v9032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:15 smithi119 ceph-mon[32752]: pgmap v9033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:15 smithi167 ceph-mon[32751]: pgmap v9033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:15 smithi090 ceph-mon[29180]: pgmap v9033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15657.219444 secs 2024-04-26T22:56:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15657.219444 secs 2024-04-26T22:56:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15657.219444 secs 2024-04-26T22:56:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15657.979611 secs 2024-04-26T22:56:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:17 smithi119 ceph-mon[32752]: pgmap v9034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15657.979611 secs 2024-04-26T22:56:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:17 smithi167 ceph-mon[32751]: pgmap v9034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15657.979611 secs 2024-04-26T22:56:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:17 smithi090 ceph-mon[29180]: pgmap v9034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:19.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:19 smithi119 ceph-mon[32752]: pgmap v9035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:19 smithi167 ceph-mon[32751]: pgmap v9035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:19 smithi090 ceph-mon[29180]: pgmap v9035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:21 smithi119 ceph-mon[32752]: pgmap v9036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15662.219530 secs 2024-04-26T22:56:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:21 smithi167 ceph-mon[32751]: pgmap v9036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15662.219530 secs 2024-04-26T22:56:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:21 smithi090 ceph-mon[29180]: pgmap v9036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15662.219530 secs 2024-04-26T22:56:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15662.979751 secs 2024-04-26T22:56:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:56:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15662.979751 secs 2024-04-26T22:56:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:56:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15662.979751 secs 2024-04-26T22:56:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:56:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:23 smithi119 ceph-mon[32752]: pgmap v9037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:56:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:56:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:56:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:23 smithi167 ceph-mon[32751]: pgmap v9037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:56:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:56:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:56:23.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:23 smithi090 ceph-mon[29180]: pgmap v9037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:23.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:56:23.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:56:23.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:56:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:24 smithi119 ceph-mon[32752]: pgmap v9038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:56:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:24 smithi119 ceph-mon[32752]: pgmap v9039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:56:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:24 smithi167 ceph-mon[32751]: pgmap v9038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:56:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:24 smithi167 ceph-mon[32751]: pgmap v9039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:56:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:24 smithi090 ceph-mon[29180]: pgmap v9038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T22:56:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:24 smithi090 ceph-mon[29180]: pgmap v9039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T22:56:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:26 smithi167 ceph-mon[32751]: pgmap v9040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15667.219607 secs 2024-04-26T22:56:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:26 smithi090 ceph-mon[29180]: pgmap v9040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15667.219607 secs 2024-04-26T22:56:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:26 smithi119 ceph-mon[32752]: pgmap v9040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15667.219607 secs 2024-04-26T22:56:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15667.979879 secs 2024-04-26T22:56:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15667.979879 secs 2024-04-26T22:56:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15667.979879 secs 2024-04-26T22:56:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:28 smithi167 ceph-mon[32751]: pgmap v9041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:28 smithi090 ceph-mon[29180]: pgmap v9041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:28 smithi119 ceph-mon[32752]: pgmap v9041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:30 smithi167 ceph-mon[32751]: pgmap v9042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:30 smithi090 ceph-mon[29180]: pgmap v9042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:30 smithi119 ceph-mon[32752]: pgmap v9042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15672.219691 secs 2024-04-26T22:56:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15672.219691 secs 2024-04-26T22:56:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15672.219691 secs 2024-04-26T22:56:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:32 smithi090 ceph-mon[29180]: pgmap v9043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15672.980005 secs 2024-04-26T22:56:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:32 smithi119 ceph-mon[32752]: pgmap v9043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15672.980005 secs 2024-04-26T22:56:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:32 smithi167 ceph-mon[32751]: pgmap v9043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:56:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15672.980005 secs 2024-04-26T22:56:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:34 smithi090 ceph-mon[29180]: pgmap v9044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:56:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:34 smithi119 ceph-mon[32752]: pgmap v9044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:56:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:34 smithi167 ceph-mon[32751]: pgmap v9044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:56:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:36 smithi090 ceph-mon[29180]: pgmap v9045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15677.219773 secs 2024-04-26T22:56:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:36 smithi119 ceph-mon[32752]: pgmap v9045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15677.219773 secs 2024-04-26T22:56:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:36 smithi167 ceph-mon[32751]: pgmap v9045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15677.219773 secs 2024-04-26T22:56:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15677.980078 secs 2024-04-26T22:56:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15677.980078 secs 2024-04-26T22:56:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15677.980078 secs 2024-04-26T22:56:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:38 smithi090 ceph-mon[29180]: pgmap v9046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:38 smithi119 ceph-mon[32752]: pgmap v9046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:38 smithi167 ceph-mon[32751]: pgmap v9046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:40 smithi090 ceph-mon[29180]: pgmap v9047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:40 smithi119 ceph-mon[32752]: pgmap v9047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:40 smithi167 ceph-mon[32751]: pgmap v9047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15682.219824 secs 2024-04-26T22:56:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15682.219824 secs 2024-04-26T22:56:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15682.219824 secs 2024-04-26T22:56:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:42 smithi090 ceph-mon[29180]: pgmap v9048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15682.980150 secs 2024-04-26T22:56:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:42 smithi119 ceph-mon[32752]: pgmap v9048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15682.980150 secs 2024-04-26T22:56:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:42 smithi167 ceph-mon[32751]: pgmap v9048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15682.980150 secs 2024-04-26T22:56:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:44 smithi090 ceph-mon[29180]: pgmap v9049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:44 smithi119 ceph-mon[32752]: pgmap v9049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:44 smithi167 ceph-mon[32751]: pgmap v9049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:46 smithi090 ceph-mon[29180]: pgmap v9050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15687.219887 secs 2024-04-26T22:56:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:46 smithi119 ceph-mon[32752]: pgmap v9050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15687.219887 secs 2024-04-26T22:56:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:46 smithi167 ceph-mon[32751]: pgmap v9050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15687.219887 secs 2024-04-26T22:56:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15687.980266 secs 2024-04-26T22:56:47.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15687.980266 secs 2024-04-26T22:56:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15687.980266 secs 2024-04-26T22:56:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:48 smithi090 ceph-mon[29180]: pgmap v9051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:48 smithi119 ceph-mon[32752]: pgmap v9051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:48 smithi167 ceph-mon[32751]: pgmap v9051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:50 smithi090 ceph-mon[29180]: pgmap v9052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:50 smithi119 ceph-mon[32752]: pgmap v9052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:50 smithi167 ceph-mon[32751]: pgmap v9052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15692.219948 secs 2024-04-26T22:56:51.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15692.219948 secs 2024-04-26T22:56:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15692.219948 secs 2024-04-26T22:56:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:52 smithi090 ceph-mon[29180]: pgmap v9053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15692.980391 secs 2024-04-26T22:56:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:52 smithi119 ceph-mon[32752]: pgmap v9053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15692.980391 secs 2024-04-26T22:56:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:52 smithi167 ceph-mon[32751]: pgmap v9053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15692.980391 secs 2024-04-26T22:56:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:54 smithi090 ceph-mon[29180]: pgmap v9054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:54 smithi119 ceph-mon[32752]: pgmap v9054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:54 smithi167 ceph-mon[32751]: pgmap v9054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:56:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:56 smithi090 ceph-mon[29180]: pgmap v9055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15697.220035 secs 2024-04-26T22:56:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:56 smithi119 ceph-mon[32752]: pgmap v9055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15697.220035 secs 2024-04-26T22:56:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:56 smithi167 ceph-mon[32751]: pgmap v9055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:56:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15697.220035 secs 2024-04-26T22:56:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15697.980489 secs 2024-04-26T22:56:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15697.980489 secs 2024-04-26T22:56:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15697.980489 secs 2024-04-26T22:56:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:56:58 smithi119 ceph-mon[32752]: pgmap v9056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:56:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:56:58 smithi167 ceph-mon[32751]: pgmap v9056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:56:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:56:58 smithi090 ceph-mon[29180]: pgmap v9056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:57:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:00 smithi119 ceph-mon[32752]: pgmap v9057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:00 smithi167 ceph-mon[32751]: pgmap v9057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:00 smithi090 ceph-mon[29180]: pgmap v9057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15702.220097 secs 2024-04-26T22:57:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15702.220097 secs 2024-04-26T22:57:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15702.220097 secs 2024-04-26T22:57:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:02 smithi119 ceph-mon[32752]: pgmap v9058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:57:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15702.980600 secs 2024-04-26T22:57:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:02 smithi167 ceph-mon[32751]: pgmap v9058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:57:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15702.980600 secs 2024-04-26T22:57:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:02 smithi090 ceph-mon[29180]: pgmap v9058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:57:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15702.980600 secs 2024-04-26T22:57:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:04 smithi119 ceph-mon[32752]: pgmap v9059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:57:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:04 smithi167 ceph-mon[32751]: pgmap v9059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:57:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:04 smithi090 ceph-mon[29180]: pgmap v9059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T22:57:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:06 smithi119 ceph-mon[32752]: pgmap v9060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:06 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15707.220181 secs 2024-04-26T22:57:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:06 smithi119 ceph-mon[32752]: slow request 5761.488598 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:57:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:06 smithi167 ceph-mon[32751]: pgmap v9060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:06 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15707.220181 secs 2024-04-26T22:57:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:06 smithi167 ceph-mon[32751]: slow request 5761.488598 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:57:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:06 smithi090 ceph-mon[29180]: pgmap v9060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:06 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15707.220181 secs 2024-04-26T22:57:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:06 smithi090 ceph-mon[29180]: slow request 5761.488598 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:57:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15707.980666 secs 2024-04-26T22:57:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15707.980666 secs 2024-04-26T22:57:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15707.980666 secs 2024-04-26T22:57:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:08 smithi119 ceph-mon[32752]: pgmap v9061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:08 smithi167 ceph-mon[32751]: pgmap v9061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:08 smithi090 ceph-mon[29180]: pgmap v9061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:10 smithi119 ceph-mon[32752]: pgmap v9062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:10 smithi167 ceph-mon[32751]: pgmap v9062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:10 smithi090 ceph-mon[29180]: pgmap v9062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15712.220265 secs 2024-04-26T22:57:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15712.220265 secs 2024-04-26T22:57:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15712.220265 secs 2024-04-26T22:57:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:12 smithi119 ceph-mon[32752]: pgmap v9063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15712.980773 secs 2024-04-26T22:57:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:12 smithi167 ceph-mon[32751]: pgmap v9063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15712.980773 secs 2024-04-26T22:57:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:12 smithi090 ceph-mon[29180]: pgmap v9063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15712.980773 secs 2024-04-26T22:57:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:14 smithi119 ceph-mon[32752]: pgmap v9064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:14 smithi167 ceph-mon[32751]: pgmap v9064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:14 smithi090 ceph-mon[29180]: pgmap v9064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:16 smithi119 ceph-mon[32752]: pgmap v9065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:16 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15717.220349 secs 2024-04-26T22:57:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:16 smithi119 ceph-mon[32752]: slow request 5761.452084 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T22:57:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:16 smithi167 ceph-mon[32751]: pgmap v9065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:16 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15717.220349 secs 2024-04-26T22:57:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:16 smithi167 ceph-mon[32751]: slow request 5761.452084 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T22:57:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:16 smithi090 ceph-mon[29180]: pgmap v9065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:16 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15717.220349 secs 2024-04-26T22:57:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:16 smithi090 ceph-mon[29180]: slow request 5761.452084 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-26T22:57:17.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15717.980885 secs 2024-04-26T22:57:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15717.980885 secs 2024-04-26T22:57:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15717.980885 secs 2024-04-26T22:57:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:18 smithi119 ceph-mon[32752]: pgmap v9066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:18 smithi167 ceph-mon[32751]: pgmap v9066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:18 smithi090 ceph-mon[29180]: pgmap v9066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T22:57:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:20 smithi119 ceph-mon[32752]: pgmap v9067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:20 smithi167 ceph-mon[32751]: pgmap v9067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:20 smithi090 ceph-mon[29180]: pgmap v9067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T22:57:21.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15722.220432 secs 2024-04-26T22:57:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15722.220432 secs 2024-04-26T22:57:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15722.220432 secs 2024-04-26T22:57:22.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:22 smithi090 ceph-mon[29180]: pgmap v9068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:22.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15722.981006 secs 2024-04-26T22:57:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:22 smithi119 ceph-mon[32752]: pgmap v9068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15722.981006 secs 2024-04-26T22:57:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:22 smithi167 ceph-mon[32751]: pgmap v9068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15722.981006 secs 2024-04-26T22:57:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:57:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:57:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:57:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:57:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:57:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:57:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:57:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:57:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:57:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:57:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:57:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:57:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:24 smithi119 ceph-mon[32752]: pgmap v9069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:24 smithi119 ceph-mon[32752]: pgmap v9070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:57:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:24 smithi119 ceph-mon[32752]: pgmap v9071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:24 smithi167 ceph-mon[32751]: pgmap v9069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:24 smithi167 ceph-mon[32751]: pgmap v9070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:57:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:24 smithi167 ceph-mon[32751]: pgmap v9071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:24 smithi090 ceph-mon[29180]: pgmap v9069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:24 smithi090 ceph-mon[29180]: pgmap v9070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T22:57:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:24 smithi090 ceph-mon[29180]: pgmap v9071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15727.220509 secs 2024-04-26T22:57:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:26 smithi119 ceph-mon[32752]: pgmap v9072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15727.220509 secs 2024-04-26T22:57:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:26 smithi167 ceph-mon[32751]: pgmap v9072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15727.220509 secs 2024-04-26T22:57:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:26 smithi090 ceph-mon[29180]: pgmap v9072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15727.981143 secs 2024-04-26T22:57:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15727.981143 secs 2024-04-26T22:57:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15727.981143 secs 2024-04-26T22:57:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:28 smithi119 ceph-mon[32752]: pgmap v9073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:57:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:28 smithi167 ceph-mon[32751]: pgmap v9073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:57:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:28 smithi090 ceph-mon[29180]: pgmap v9073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T22:57:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:30 smithi119 ceph-mon[32752]: pgmap v9074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:30 smithi167 ceph-mon[32751]: pgmap v9074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:30 smithi090 ceph-mon[29180]: pgmap v9074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15732.220568 secs 2024-04-26T22:57:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15732.220568 secs 2024-04-26T22:57:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15732.220568 secs 2024-04-26T22:57:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15732.981280 secs 2024-04-26T22:57:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:32 smithi119 ceph-mon[32752]: pgmap v9075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15732.981280 secs 2024-04-26T22:57:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:32 smithi167 ceph-mon[32751]: pgmap v9075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15732.981280 secs 2024-04-26T22:57:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:32 smithi090 ceph-mon[29180]: pgmap v9075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:57:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:34 smithi119 ceph-mon[32752]: pgmap v9076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:57:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:34 smithi167 ceph-mon[32751]: pgmap v9076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:57:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:34 smithi090 ceph-mon[29180]: pgmap v9076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:57:35.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:57:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:57:35.051+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:57:35.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:57:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:57:35.051+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:57:35.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:57:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:57:35.051+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:57:35.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:57:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:57:35.051+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:57:35.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 22:57:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T22:57:35.051+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T22:57:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15737.220648 secs 2024-04-26T22:57:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:36 smithi119 ceph-mon[32752]: pgmap v9077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15737.220648 secs 2024-04-26T22:57:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:36 smithi167 ceph-mon[32751]: pgmap v9077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15737.220648 secs 2024-04-26T22:57:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:36 smithi090 ceph-mon[29180]: pgmap v9077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15737.981341 secs 2024-04-26T22:57:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15737.981341 secs 2024-04-26T22:57:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15737.981341 secs 2024-04-26T22:57:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:38 smithi119 ceph-mon[32752]: pgmap v9078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:38 smithi167 ceph-mon[32751]: pgmap v9078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:38 smithi090 ceph-mon[29180]: pgmap v9078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:40 smithi119 ceph-mon[32752]: pgmap v9079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:40 smithi167 ceph-mon[32751]: pgmap v9079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:40 smithi090 ceph-mon[29180]: pgmap v9079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15742.220731 secs 2024-04-26T22:57:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15742.220731 secs 2024-04-26T22:57:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15742.220731 secs 2024-04-26T22:57:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15742.981475 secs 2024-04-26T22:57:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:42 smithi119 ceph-mon[32752]: pgmap v9080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15742.981475 secs 2024-04-26T22:57:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:42 smithi167 ceph-mon[32751]: pgmap v9080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15742.981475 secs 2024-04-26T22:57:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:42 smithi090 ceph-mon[29180]: pgmap v9080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:44 smithi119 ceph-mon[32752]: pgmap v9081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:44 smithi167 ceph-mon[32751]: pgmap v9081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:44 smithi090 ceph-mon[29180]: pgmap v9081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15747.220838 secs 2024-04-26T22:57:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:46 smithi119 ceph-mon[32752]: pgmap v9082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15747.220838 secs 2024-04-26T22:57:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:46 smithi167 ceph-mon[32751]: pgmap v9082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15747.220838 secs 2024-04-26T22:57:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:46 smithi090 ceph-mon[29180]: pgmap v9082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15747.981552 secs 2024-04-26T22:57:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15747.981552 secs 2024-04-26T22:57:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15747.981552 secs 2024-04-26T22:57:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:48 smithi119 ceph-mon[32752]: pgmap v9083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:48 smithi167 ceph-mon[32751]: pgmap v9083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:48 smithi090 ceph-mon[29180]: pgmap v9083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:50 smithi119 ceph-mon[32752]: pgmap v9084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:50 smithi167 ceph-mon[32751]: pgmap v9084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:50 smithi090 ceph-mon[29180]: pgmap v9084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T22:57:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15752.220928 secs 2024-04-26T22:57:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15752.220928 secs 2024-04-26T22:57:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15752.220928 secs 2024-04-26T22:57:52.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15752.981613 secs 2024-04-26T22:57:52.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:52 smithi090 ceph-mon[29180]: pgmap v9085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15752.981613 secs 2024-04-26T22:57:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:52 smithi119 ceph-mon[32752]: pgmap v9085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15752.981613 secs 2024-04-26T22:57:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:52 smithi167 ceph-mon[32751]: pgmap v9085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:54 smithi119 ceph-mon[32752]: pgmap v9086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:54 smithi167 ceph-mon[32751]: pgmap v9086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:54 smithi090 ceph-mon[29180]: pgmap v9086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:57:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15757.220991 secs 2024-04-26T22:57:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:56 smithi119 ceph-mon[32752]: pgmap v9087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15757.220991 secs 2024-04-26T22:57:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:56 smithi167 ceph-mon[32751]: pgmap v9087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15757.220991 secs 2024-04-26T22:57:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:56 smithi090 ceph-mon[29180]: pgmap v9087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15757.981676 secs 2024-04-26T22:57:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15757.981676 secs 2024-04-26T22:57:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15757.981676 secs 2024-04-26T22:57:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:57:58 smithi119 ceph-mon[32752]: pgmap v9088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:57:58 smithi167 ceph-mon[32751]: pgmap v9088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:57:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:57:58 smithi090 ceph-mon[29180]: pgmap v9088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:00 smithi119 ceph-mon[32752]: pgmap v9089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:00 smithi167 ceph-mon[32751]: pgmap v9089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:00 smithi090 ceph-mon[29180]: pgmap v9089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15762.221054 secs 2024-04-26T22:58:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15762.221054 secs 2024-04-26T22:58:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15762.221054 secs 2024-04-26T22:58:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15762.981803 secs 2024-04-26T22:58:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:02 smithi119 ceph-mon[32752]: pgmap v9090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15762.981803 secs 2024-04-26T22:58:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:02 smithi167 ceph-mon[32751]: pgmap v9090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15762.981803 secs 2024-04-26T22:58:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:02 smithi090 ceph-mon[29180]: pgmap v9090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:04 smithi119 ceph-mon[32752]: pgmap v9091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:04 smithi167 ceph-mon[32751]: pgmap v9091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:04 smithi090 ceph-mon[29180]: pgmap v9091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15767.221115 secs 2024-04-26T22:58:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:06 smithi119 ceph-mon[32752]: pgmap v9092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15767.221115 secs 2024-04-26T22:58:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:06 smithi167 ceph-mon[32751]: pgmap v9092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15767.221115 secs 2024-04-26T22:58:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:06 smithi090 ceph-mon[29180]: pgmap v9092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15767.981938 secs 2024-04-26T22:58:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15767.981938 secs 2024-04-26T22:58:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15767.981938 secs 2024-04-26T22:58:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:08 smithi167 ceph-mon[32751]: pgmap v9093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:08 smithi090 ceph-mon[29180]: pgmap v9093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:08 smithi119 ceph-mon[32752]: pgmap v9093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:10 smithi167 ceph-mon[32751]: pgmap v9094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:10 smithi090 ceph-mon[29180]: pgmap v9094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:10 smithi119 ceph-mon[32752]: pgmap v9094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15772.221200 secs 2024-04-26T22:58:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15772.221200 secs 2024-04-26T22:58:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15772.221200 secs 2024-04-26T22:58:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15772.982060 secs 2024-04-26T22:58:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:12 smithi090 ceph-mon[29180]: pgmap v9095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15772.982060 secs 2024-04-26T22:58:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:12 smithi119 ceph-mon[32752]: pgmap v9095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15772.982060 secs 2024-04-26T22:58:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:12 smithi167 ceph-mon[32751]: pgmap v9095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T22:58:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:14 smithi090 ceph-mon[29180]: pgmap v9096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:14 smithi119 ceph-mon[32752]: pgmap v9096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:14 smithi167 ceph-mon[32751]: pgmap v9096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T22:58:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15777.221282 secs 2024-04-26T22:58:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:16 smithi090 ceph-mon[29180]: pgmap v9097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15777.221282 secs 2024-04-26T22:58:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:16 smithi119 ceph-mon[32752]: pgmap v9097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15777.221282 secs 2024-04-26T22:58:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:16 smithi167 ceph-mon[32751]: pgmap v9097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15777.982181 secs 2024-04-26T22:58:17.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15777.982181 secs 2024-04-26T22:58:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15777.982181 secs 2024-04-26T22:58:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:18 smithi090 ceph-mon[29180]: pgmap v9098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:18 smithi119 ceph-mon[32752]: pgmap v9098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:18 smithi167 ceph-mon[32751]: pgmap v9098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:20 smithi090 ceph-mon[29180]: pgmap v9099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:20 smithi119 ceph-mon[32752]: pgmap v9099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:20 smithi167 ceph-mon[32751]: pgmap v9099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15782.221345 secs 2024-04-26T22:58:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15782.221345 secs 2024-04-26T22:58:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15782.221345 secs 2024-04-26T22:58:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15782.982329 secs 2024-04-26T22:58:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:22 smithi090 ceph-mon[29180]: pgmap v9100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15782.982329 secs 2024-04-26T22:58:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:22 smithi119 ceph-mon[32752]: pgmap v9100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15782.982329 secs 2024-04-26T22:58:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:22 smithi167 ceph-mon[32751]: pgmap v9100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:58:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:24 smithi167 ceph-mon[32751]: pgmap v9101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:24 smithi090 ceph-mon[29180]: pgmap v9101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.910 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:24 smithi119 ceph-mon[32752]: pgmap v9101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:24.911 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.911 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.911 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:24.911 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:25 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15787.221430 secs 2024-04-26T22:58:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:25 smithi119 ceph-mon[32752]: pgmap v9102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:25 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15787.221430 secs 2024-04-26T22:58:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:25 smithi167 ceph-mon[32751]: pgmap v9102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:25 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15787.221430 secs 2024-04-26T22:58:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:25 smithi090 ceph-mon[29180]: pgmap v9102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15787.982423 secs 2024-04-26T22:58:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:58:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:26 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:58:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15787.982423 secs 2024-04-26T22:58:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:58:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:26 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:58:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15787.982423 secs 2024-04-26T22:58:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T22:58:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:26 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T22:58:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:58:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: pgmap v9103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-26T22:58:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: pgmap v9104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:58:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:58:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:58:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:28.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: pgmap v9103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-26T22:58:28.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: pgmap v9104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:28.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T22:58:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:58:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: pgmap v9103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-26T22:58:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: pgmap v9104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:58:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:29 smithi119 ceph-mon[32752]: pgmap v9105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:29 smithi167 ceph-mon[32751]: pgmap v9105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:29 smithi090 ceph-mon[29180]: pgmap v9105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:30 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15792.221489 secs 2024-04-26T22:58:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:30 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15792.221489 secs 2024-04-26T22:58:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:30 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15792.221489 secs 2024-04-26T22:58:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15792.982560 secs 2024-04-26T22:58:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:31 smithi119 ceph-mon[32752]: pgmap v9106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15792.982560 secs 2024-04-26T22:58:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:31 smithi167 ceph-mon[32751]: pgmap v9106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15792.982560 secs 2024-04-26T22:58:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:31 smithi090 ceph-mon[29180]: pgmap v9106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:33 smithi119 ceph-mon[32752]: pgmap v9107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 537 B/s rd, 1 op/s 2024-04-26T22:58:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:33 smithi167 ceph-mon[32751]: pgmap v9107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 537 B/s rd, 1 op/s 2024-04-26T22:58:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:33 smithi090 ceph-mon[29180]: pgmap v9107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 537 B/s rd, 1 op/s 2024-04-26T22:58:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:35 smithi119 ceph-mon[32752]: pgmap v9108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:35 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15797.221576 secs 2024-04-26T22:58:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:35 smithi167 ceph-mon[32751]: pgmap v9108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:35 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15797.221576 secs 2024-04-26T22:58:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:35 smithi090 ceph-mon[29180]: pgmap v9108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T22:58:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:35 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15797.221576 secs 2024-04-26T22:58:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15797.982676 secs 2024-04-26T22:58:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15797.982676 secs 2024-04-26T22:58:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15797.982676 secs 2024-04-26T22:58:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:37 smithi119 ceph-mon[32752]: pgmap v9109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:58:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:37 smithi167 ceph-mon[32751]: pgmap v9109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:58:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:37 smithi090 ceph-mon[29180]: pgmap v9109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:58:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:40 smithi119 ceph-mon[32752]: pgmap v9110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:40 smithi167 ceph-mon[32751]: pgmap v9110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:40 smithi090 ceph-mon[29180]: pgmap v9110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15802.221659 secs 2024-04-26T22:58:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15802.221659 secs 2024-04-26T22:58:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15802.221659 secs 2024-04-26T22:58:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15802.982766 secs 2024-04-26T22:58:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:42 smithi119 ceph-mon[32752]: pgmap v9111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15802.982766 secs 2024-04-26T22:58:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:42 smithi167 ceph-mon[32751]: pgmap v9111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15802.982766 secs 2024-04-26T22:58:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:42 smithi090 ceph-mon[29180]: pgmap v9111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:44 smithi090 ceph-mon[29180]: pgmap v9112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:44 smithi119 ceph-mon[32752]: pgmap v9112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:44 smithi167 ceph-mon[32751]: pgmap v9112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:46 smithi119 ceph-mon[32752]: pgmap v9113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15807.221741 secs 2024-04-26T22:58:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:46 smithi167 ceph-mon[32751]: pgmap v9113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15807.221741 secs 2024-04-26T22:58:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:46 smithi090 ceph-mon[29180]: pgmap v9113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15807.221741 secs 2024-04-26T22:58:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15807.982846 secs 2024-04-26T22:58:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15807.982846 secs 2024-04-26T22:58:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15807.982846 secs 2024-04-26T22:58:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:48 smithi119 ceph-mon[32752]: pgmap v9114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:48 smithi167 ceph-mon[32751]: pgmap v9114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:48 smithi090 ceph-mon[29180]: pgmap v9114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:50 smithi119 ceph-mon[32752]: pgmap v9115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:50 smithi167 ceph-mon[32751]: pgmap v9115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:50 smithi090 ceph-mon[29180]: pgmap v9115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15812.221829 secs 2024-04-26T22:58:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15812.221829 secs 2024-04-26T22:58:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15812.221829 secs 2024-04-26T22:58:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15812.982970 secs 2024-04-26T22:58:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:52 smithi119 ceph-mon[32752]: pgmap v9116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15812.982970 secs 2024-04-26T22:58:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:52 smithi167 ceph-mon[32751]: pgmap v9116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15812.982970 secs 2024-04-26T22:58:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:52 smithi090 ceph-mon[29180]: pgmap v9116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:54 smithi119 ceph-mon[32752]: pgmap v9117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:54 smithi167 ceph-mon[32751]: pgmap v9117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:54 smithi090 ceph-mon[29180]: pgmap v9117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:56 smithi119 ceph-mon[32752]: pgmap v9118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15817.221891 secs 2024-04-26T22:58:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:56 smithi167 ceph-mon[32751]: pgmap v9118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15817.221891 secs 2024-04-26T22:58:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:56 smithi090 ceph-mon[29180]: pgmap v9118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:58:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15817.221891 secs 2024-04-26T22:58:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15817.983083 secs 2024-04-26T22:58:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15817.983083 secs 2024-04-26T22:58:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15817.983083 secs 2024-04-26T22:58:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:58:58 smithi119 ceph-mon[32752]: pgmap v9119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:58:58 smithi167 ceph-mon[32751]: pgmap v9119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:58:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:58:58 smithi090 ceph-mon[29180]: pgmap v9119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:00 smithi119 ceph-mon[32752]: pgmap v9120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:00 smithi167 ceph-mon[32751]: pgmap v9120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:00 smithi090 ceph-mon[29180]: pgmap v9120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15822.221975 secs 2024-04-26T22:59:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15822.221975 secs 2024-04-26T22:59:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15822.221975 secs 2024-04-26T22:59:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15822.983220 secs 2024-04-26T22:59:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:02 smithi119 ceph-mon[32752]: pgmap v9121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15822.983220 secs 2024-04-26T22:59:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:02 smithi167 ceph-mon[32751]: pgmap v9121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15822.983220 secs 2024-04-26T22:59:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:02 smithi090 ceph-mon[29180]: pgmap v9121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:04 smithi119 ceph-mon[32752]: pgmap v9122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:04 smithi167 ceph-mon[32751]: pgmap v9122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:04 smithi090 ceph-mon[29180]: pgmap v9122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:06 smithi119 ceph-mon[32752]: pgmap v9123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15827.222038 secs 2024-04-26T22:59:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:06 smithi167 ceph-mon[32751]: pgmap v9123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15827.222038 secs 2024-04-26T22:59:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:06 smithi090 ceph-mon[29180]: pgmap v9123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15827.222038 secs 2024-04-26T22:59:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:07 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 15827.983337 secs 2024-04-26T22:59:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:07 smithi119 ceph-mon[32752]: slow request 720.851725 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:59:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:07 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 15827.983337 secs 2024-04-26T22:59:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:07 smithi167 ceph-mon[32751]: slow request 720.851725 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:59:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:07 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 15827.983337 secs 2024-04-26T22:59:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:07 smithi090 ceph-mon[29180]: slow request 720.851725 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T22:59:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:08 smithi119 ceph-mon[32752]: pgmap v9124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:08 smithi167 ceph-mon[32751]: pgmap v9124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:08 smithi090 ceph-mon[29180]: pgmap v9124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:10 smithi119 ceph-mon[32752]: pgmap v9125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:10 smithi167 ceph-mon[32751]: pgmap v9125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:10 smithi090 ceph-mon[29180]: pgmap v9125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15832.222119 secs 2024-04-26T22:59:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15832.222119 secs 2024-04-26T22:59:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15832.222119 secs 2024-04-26T22:59:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15832.983459 secs 2024-04-26T22:59:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:12 smithi119 ceph-mon[32752]: pgmap v9126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15832.983459 secs 2024-04-26T22:59:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:12 smithi167 ceph-mon[32751]: pgmap v9126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15832.983459 secs 2024-04-26T22:59:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:12 smithi090 ceph-mon[29180]: pgmap v9126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:14 smithi119 ceph-mon[32752]: pgmap v9127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:14 smithi167 ceph-mon[32751]: pgmap v9127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:14 smithi090 ceph-mon[29180]: pgmap v9127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:16 smithi167 ceph-mon[32751]: pgmap v9128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15837.222202 secs 2024-04-26T22:59:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:16 smithi090 ceph-mon[29180]: pgmap v9128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15837.222202 secs 2024-04-26T22:59:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:16 smithi119 ceph-mon[32752]: pgmap v9128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15837.222202 secs 2024-04-26T22:59:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15837.983568 secs 2024-04-26T22:59:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15837.983568 secs 2024-04-26T22:59:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15837.983568 secs 2024-04-26T22:59:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:18 smithi167 ceph-mon[32751]: pgmap v9129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:18 smithi090 ceph-mon[29180]: pgmap v9129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:18 smithi119 ceph-mon[32752]: pgmap v9129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:20 smithi167 ceph-mon[32751]: pgmap v9130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:20 smithi090 ceph-mon[29180]: pgmap v9130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:20 smithi119 ceph-mon[32752]: pgmap v9130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15842.222265 secs 2024-04-26T22:59:21.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15842.222265 secs 2024-04-26T22:59:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15842.222265 secs 2024-04-26T22:59:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15842.983649 secs 2024-04-26T22:59:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:22 smithi090 ceph-mon[29180]: pgmap v9131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:22.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15842.983649 secs 2024-04-26T22:59:22.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:22 smithi119 ceph-mon[32752]: pgmap v9131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15842.983649 secs 2024-04-26T22:59:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:22 smithi167 ceph-mon[32751]: pgmap v9131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:24 smithi090 ceph-mon[29180]: pgmap v9132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:24 smithi119 ceph-mon[32752]: pgmap v9132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:24 smithi167 ceph-mon[32751]: pgmap v9132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:26 smithi090 ceph-mon[29180]: pgmap v9133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15847.222346 secs 2024-04-26T22:59:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:26 smithi119 ceph-mon[32752]: pgmap v9133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15847.222346 secs 2024-04-26T22:59:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:26 smithi167 ceph-mon[32751]: pgmap v9133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15847.222346 secs 2024-04-26T22:59:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15847.983765 secs 2024-04-26T22:59:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:59:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15847.983765 secs 2024-04-26T22:59:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:59:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15847.983765 secs 2024-04-26T22:59:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T22:59:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:28 smithi090 ceph-mon[29180]: pgmap v9134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:59:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:59:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:28 smithi119 ceph-mon[32752]: pgmap v9134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:59:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:59:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:28 smithi167 ceph-mon[32751]: pgmap v9134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T22:59:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T22:59:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T22:59:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:29 smithi090 ceph-mon[29180]: pgmap v9135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T22:59:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:29 smithi090 ceph-mon[29180]: pgmap v9136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:29 smithi119 ceph-mon[32752]: pgmap v9135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T22:59:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:29 smithi119 ceph-mon[32752]: pgmap v9136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:29 smithi167 ceph-mon[32751]: pgmap v9135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T22:59:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:29 smithi167 ceph-mon[32751]: pgmap v9136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15852.222427 secs 2024-04-26T22:59:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:31 smithi090 ceph-mon[29180]: pgmap v9137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15852.222427 secs 2024-04-26T22:59:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:31 smithi119 ceph-mon[32752]: pgmap v9137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15852.222427 secs 2024-04-26T22:59:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:31 smithi167 ceph-mon[32751]: pgmap v9137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15852.983880 secs 2024-04-26T22:59:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15852.983880 secs 2024-04-26T22:59:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15852.983880 secs 2024-04-26T22:59:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:33 smithi090 ceph-mon[29180]: pgmap v9138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:33 smithi119 ceph-mon[32752]: pgmap v9138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:33 smithi167 ceph-mon[32751]: pgmap v9138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:35 smithi090 ceph-mon[29180]: pgmap v9139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:35 smithi119 ceph-mon[32752]: pgmap v9139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:35 smithi167 ceph-mon[32751]: pgmap v9139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15857.222511 secs 2024-04-26T22:59:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15857.222511 secs 2024-04-26T22:59:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15857.222511 secs 2024-04-26T22:59:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15857.984016 secs 2024-04-26T22:59:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:37 smithi090 ceph-mon[29180]: pgmap v9140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15857.984016 secs 2024-04-26T22:59:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:37 smithi119 ceph-mon[32752]: pgmap v9140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15857.984016 secs 2024-04-26T22:59:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:37 smithi167 ceph-mon[32751]: pgmap v9140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T22:59:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:38 smithi090 ceph-mon[29180]: pgmap v9141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:59:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:38 smithi119 ceph-mon[32752]: pgmap v9141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:59:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:38 smithi167 ceph-mon[32751]: pgmap v9141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T22:59:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15862.222597 secs 2024-04-26T22:59:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:41 smithi119 ceph-mon[32752]: pgmap v9142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15862.222597 secs 2024-04-26T22:59:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:41 smithi167 ceph-mon[32751]: pgmap v9142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15862.222597 secs 2024-04-26T22:59:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:41 smithi090 ceph-mon[29180]: pgmap v9142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15862.984135 secs 2024-04-26T22:59:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15862.984135 secs 2024-04-26T22:59:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15862.984135 secs 2024-04-26T22:59:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:43 smithi119 ceph-mon[32752]: pgmap v9143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:43 smithi167 ceph-mon[32751]: pgmap v9143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:43 smithi090 ceph-mon[29180]: pgmap v9143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:45 smithi119 ceph-mon[32752]: pgmap v9144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:45 smithi167 ceph-mon[32751]: pgmap v9144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:45 smithi090 ceph-mon[29180]: pgmap v9144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15867.222656 secs 2024-04-26T22:59:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15867.222656 secs 2024-04-26T22:59:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15867.222656 secs 2024-04-26T22:59:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15867.984255 secs 2024-04-26T22:59:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:47 smithi167 ceph-mon[32751]: pgmap v9145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15867.984255 secs 2024-04-26T22:59:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:47 smithi090 ceph-mon[29180]: pgmap v9145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15867.984255 secs 2024-04-26T22:59:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:47 smithi119 ceph-mon[32752]: pgmap v9145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:49 smithi167 ceph-mon[32751]: pgmap v9146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:49 smithi090 ceph-mon[29180]: pgmap v9146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:49 smithi119 ceph-mon[32752]: pgmap v9146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15872.222738 secs 2024-04-26T22:59:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:51 smithi167 ceph-mon[32751]: pgmap v9147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:51.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15872.222738 secs 2024-04-26T22:59:51.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:51 smithi119 ceph-mon[32752]: pgmap v9147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15872.222738 secs 2024-04-26T22:59:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:51 smithi090 ceph-mon[29180]: pgmap v9147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15872.984407 secs 2024-04-26T22:59:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15872.984407 secs 2024-04-26T22:59:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15872.984407 secs 2024-04-26T22:59:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:53 smithi090 ceph-mon[29180]: pgmap v9148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:53 smithi119 ceph-mon[32752]: pgmap v9148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:53 smithi167 ceph-mon[32751]: pgmap v9148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:55 smithi090 ceph-mon[29180]: pgmap v9149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:55 smithi119 ceph-mon[32752]: pgmap v9149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:55 smithi167 ceph-mon[32751]: pgmap v9149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15877.222827 secs 2024-04-26T22:59:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15877.222827 secs 2024-04-26T22:59:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15877.222827 secs 2024-04-26T22:59:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15877.984559 secs 2024-04-26T22:59:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:57 smithi090 ceph-mon[29180]: pgmap v9150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15877.984559 secs 2024-04-26T22:59:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:57 smithi119 ceph-mon[32752]: pgmap v9150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15877.984559 secs 2024-04-26T22:59:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:57 smithi167 ceph-mon[32751]: pgmap v9150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T22:59:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 22:59:59 smithi090 ceph-mon[29180]: pgmap v9151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 22:59:59 smithi119 ceph-mon[32752]: pgmap v9151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T22:59:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 22:59:59 smithi167 ceph-mon[32751]: pgmap v9151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:00:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:00:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:00:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15882.222910 secs 2024-04-26T23:00:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:01 smithi090 ceph-mon[29180]: pgmap v9152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15882.222910 secs 2024-04-26T23:00:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:01 smithi119 ceph-mon[32752]: pgmap v9152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15882.222910 secs 2024-04-26T23:00:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:01 smithi167 ceph-mon[32751]: pgmap v9152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15882.984678 secs 2024-04-26T23:00:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15882.984678 secs 2024-04-26T23:00:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15882.984678 secs 2024-04-26T23:00:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:03 smithi090 ceph-mon[29180]: pgmap v9153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:03 smithi119 ceph-mon[32752]: pgmap v9153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:03 smithi167 ceph-mon[32751]: pgmap v9153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:05 smithi090 ceph-mon[29180]: pgmap v9154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:05 smithi119 ceph-mon[32752]: pgmap v9154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:05 smithi167 ceph-mon[32751]: pgmap v9154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15887.223000 secs 2024-04-26T23:00:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15887.223000 secs 2024-04-26T23:00:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15887.223000 secs 2024-04-26T23:00:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15887.984753 secs 2024-04-26T23:00:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:07 smithi090 ceph-mon[29180]: pgmap v9155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15887.984753 secs 2024-04-26T23:00:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:07 smithi119 ceph-mon[32752]: pgmap v9155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15887.984753 secs 2024-04-26T23:00:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:07 smithi167 ceph-mon[32751]: pgmap v9155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:09 smithi090 ceph-mon[29180]: pgmap v9156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:09 smithi119 ceph-mon[32752]: pgmap v9156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:09 smithi167 ceph-mon[32751]: pgmap v9156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15892.223086 secs 2024-04-26T23:00:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:11 smithi090 ceph-mon[29180]: pgmap v9157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15892.223086 secs 2024-04-26T23:00:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:11 smithi119 ceph-mon[32752]: pgmap v9157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15892.223086 secs 2024-04-26T23:00:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:11 smithi167 ceph-mon[32751]: pgmap v9157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15892.984870 secs 2024-04-26T23:00:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15892.984870 secs 2024-04-26T23:00:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15892.984870 secs 2024-04-26T23:00:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:13 smithi119 ceph-mon[32752]: pgmap v9158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:13 smithi167 ceph-mon[32751]: pgmap v9158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:13 smithi090 ceph-mon[29180]: pgmap v9158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:15 smithi119 ceph-mon[32752]: pgmap v9159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:15 smithi167 ceph-mon[32751]: pgmap v9159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:15 smithi090 ceph-mon[29180]: pgmap v9159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15897.223146 secs 2024-04-26T23:00:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15897.223146 secs 2024-04-26T23:00:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15897.223146 secs 2024-04-26T23:00:17.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15897.984944 secs 2024-04-26T23:00:17.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:17 smithi167 ceph-mon[32751]: pgmap v9160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15897.984944 secs 2024-04-26T23:00:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:17 smithi119 ceph-mon[32752]: pgmap v9160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15897.984944 secs 2024-04-26T23:00:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:17 smithi090 ceph-mon[29180]: pgmap v9160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:19 smithi119 ceph-mon[32752]: pgmap v9161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:19 smithi167 ceph-mon[32751]: pgmap v9161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:19 smithi090 ceph-mon[29180]: pgmap v9161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15902.223208 secs 2024-04-26T23:00:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:21 smithi119 ceph-mon[32752]: pgmap v9162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15902.223208 secs 2024-04-26T23:00:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:21 smithi167 ceph-mon[32751]: pgmap v9162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15902.223208 secs 2024-04-26T23:00:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:21 smithi090 ceph-mon[29180]: pgmap v9162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:22.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15902.985035 secs 2024-04-26T23:00:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15902.985035 secs 2024-04-26T23:00:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15902.985035 secs 2024-04-26T23:00:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:23 smithi119 ceph-mon[32752]: pgmap v9163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:23 smithi167 ceph-mon[32751]: pgmap v9163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:23 smithi090 ceph-mon[29180]: pgmap v9163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:25 smithi119 ceph-mon[32752]: pgmap v9164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:25 smithi167 ceph-mon[32751]: pgmap v9164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:25 smithi090 ceph-mon[29180]: pgmap v9164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:26 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 15907.223292 secs 2024-04-26T23:00:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:26 smithi119 ceph-mon[32752]: slow request 2882.189398 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:00:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:26 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 15907.223292 secs 2024-04-26T23:00:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:26 smithi167 ceph-mon[32751]: slow request 2882.189398 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:00:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:26 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 15907.223292 secs 2024-04-26T23:00:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:26 smithi090 ceph-mon[29180]: slow request 2882.189398 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:00:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15907.985151 secs 2024-04-26T23:00:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:27 smithi119 ceph-mon[32752]: pgmap v9165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15907.985151 secs 2024-04-26T23:00:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:27 smithi167 ceph-mon[32751]: pgmap v9165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15907.985151 secs 2024-04-26T23:00:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:27 smithi090 ceph-mon[29180]: pgmap v9165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:00:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:00:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:00:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:29 smithi119 ceph-mon[32752]: pgmap v9166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:00:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:00:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:00:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:29 smithi167 ceph-mon[32751]: pgmap v9166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:00:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:00:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:00:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:29 smithi090 ceph-mon[29180]: pgmap v9166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:00:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:00:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:00:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:30 smithi119 ceph-mon[32752]: pgmap v9167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:00:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:30 smithi119 ceph-mon[32752]: pgmap v9168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:30 smithi167 ceph-mon[32751]: pgmap v9167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:00:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:30 smithi167 ceph-mon[32751]: pgmap v9168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:30 smithi090 ceph-mon[29180]: pgmap v9167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:00:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:30 smithi090 ceph-mon[29180]: pgmap v9168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15912.223376 secs 2024-04-26T23:00:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15912.223376 secs 2024-04-26T23:00:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15912.223376 secs 2024-04-26T23:00:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15912.985268 secs 2024-04-26T23:00:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:32 smithi119 ceph-mon[32752]: pgmap v9169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15912.985268 secs 2024-04-26T23:00:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:32 smithi167 ceph-mon[32751]: pgmap v9169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:32.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15912.985268 secs 2024-04-26T23:00:32.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:32 smithi090 ceph-mon[29180]: pgmap v9169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:34 smithi119 ceph-mon[32752]: pgmap v9170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:00:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:34 smithi167 ceph-mon[32751]: pgmap v9170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:00:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:34 smithi090 ceph-mon[29180]: pgmap v9170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:00:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15917.223484 secs 2024-04-26T23:00:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:36 smithi119 ceph-mon[32752]: pgmap v9171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15917.223484 secs 2024-04-26T23:00:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:36 smithi167 ceph-mon[32751]: pgmap v9171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15917.223484 secs 2024-04-26T23:00:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:36 smithi090 ceph-mon[29180]: pgmap v9171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:00:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15917.985407 secs 2024-04-26T23:00:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15917.985407 secs 2024-04-26T23:00:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15917.985407 secs 2024-04-26T23:00:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:38 smithi119 ceph-mon[32752]: pgmap v9172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:00:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:38 smithi167 ceph-mon[32751]: pgmap v9172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:00:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:38 smithi090 ceph-mon[29180]: pgmap v9172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:00:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:40 smithi119 ceph-mon[32752]: pgmap v9173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:00:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:40 smithi167 ceph-mon[32751]: pgmap v9173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:00:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:40 smithi090 ceph-mon[29180]: pgmap v9173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:00:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15922.223572 secs 2024-04-26T23:00:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15922.223572 secs 2024-04-26T23:00:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15922.223572 secs 2024-04-26T23:00:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15922.985499 secs 2024-04-26T23:00:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:42 smithi119 ceph-mon[32752]: pgmap v9174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15922.985499 secs 2024-04-26T23:00:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:42 smithi167 ceph-mon[32751]: pgmap v9174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15922.985499 secs 2024-04-26T23:00:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:42 smithi090 ceph-mon[29180]: pgmap v9174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:44 smithi119 ceph-mon[32752]: pgmap v9175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:44 smithi167 ceph-mon[32751]: pgmap v9175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:44 smithi090 ceph-mon[29180]: pgmap v9175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15927.223657 secs 2024-04-26T23:00:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:46 smithi119 ceph-mon[32752]: pgmap v9176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15927.223657 secs 2024-04-26T23:00:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:46 smithi167 ceph-mon[32751]: pgmap v9176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15927.223657 secs 2024-04-26T23:00:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:46 smithi090 ceph-mon[29180]: pgmap v9176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:47.732 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15927.985623 secs 2024-04-26T23:00:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15927.985623 secs 2024-04-26T23:00:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15927.985623 secs 2024-04-26T23:00:48.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:48 smithi090 ceph-mon[29180]: pgmap v9177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:48 smithi119 ceph-mon[32752]: pgmap v9177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:48 smithi167 ceph-mon[32751]: pgmap v9177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:50 smithi119 ceph-mon[32752]: pgmap v9178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:50 smithi167 ceph-mon[32751]: pgmap v9178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:50 smithi090 ceph-mon[29180]: pgmap v9178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15932.223762 secs 2024-04-26T23:00:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15932.223762 secs 2024-04-26T23:00:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15932.223762 secs 2024-04-26T23:00:52.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15932.985697 secs 2024-04-26T23:00:52.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:52 smithi090 ceph-mon[29180]: pgmap v9179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15932.985697 secs 2024-04-26T23:00:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:52 smithi119 ceph-mon[32752]: pgmap v9179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15932.985697 secs 2024-04-26T23:00:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:52 smithi167 ceph-mon[32751]: pgmap v9179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:54 smithi119 ceph-mon[32752]: pgmap v9180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:54 smithi167 ceph-mon[32751]: pgmap v9180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:54 smithi090 ceph-mon[29180]: pgmap v9180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15937.223851 secs 2024-04-26T23:00:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:56 smithi119 ceph-mon[32752]: pgmap v9181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15937.223851 secs 2024-04-26T23:00:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:56 smithi167 ceph-mon[32751]: pgmap v9181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15937.223851 secs 2024-04-26T23:00:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:56 smithi090 ceph-mon[29180]: pgmap v9181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:00:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15937.985759 secs 2024-04-26T23:00:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15937.985759 secs 2024-04-26T23:00:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15937.985759 secs 2024-04-26T23:00:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:00:58 smithi119 ceph-mon[32752]: pgmap v9182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:00:58 smithi167 ceph-mon[32751]: pgmap v9182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:00:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:00:58 smithi090 ceph-mon[29180]: pgmap v9182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:00 smithi119 ceph-mon[32752]: pgmap v9183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:00 smithi167 ceph-mon[32751]: pgmap v9183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:00 smithi090 ceph-mon[29180]: pgmap v9183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15942.223913 secs 2024-04-26T23:01:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15942.223913 secs 2024-04-26T23:01:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15942.223913 secs 2024-04-26T23:01:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15942.985877 secs 2024-04-26T23:01:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:02 smithi119 ceph-mon[32752]: pgmap v9184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15942.985877 secs 2024-04-26T23:01:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:02 smithi167 ceph-mon[32751]: pgmap v9184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15942.985877 secs 2024-04-26T23:01:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:02 smithi090 ceph-mon[29180]: pgmap v9184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:04 smithi119 ceph-mon[32752]: pgmap v9185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:04 smithi167 ceph-mon[32751]: pgmap v9185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:04 smithi090 ceph-mon[29180]: pgmap v9185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15947.224014 secs 2024-04-26T23:01:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:06 smithi119 ceph-mon[32752]: pgmap v9186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15947.224014 secs 2024-04-26T23:01:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:06 smithi167 ceph-mon[32751]: pgmap v9186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15947.224014 secs 2024-04-26T23:01:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:06 smithi090 ceph-mon[29180]: pgmap v9186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15947.986008 secs 2024-04-26T23:01:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15947.986008 secs 2024-04-26T23:01:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15947.986008 secs 2024-04-26T23:01:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:08 smithi119 ceph-mon[32752]: pgmap v9187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:08 smithi167 ceph-mon[32751]: pgmap v9187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:08 smithi090 ceph-mon[29180]: pgmap v9187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:10 smithi119 ceph-mon[32752]: pgmap v9188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:10 smithi167 ceph-mon[32751]: pgmap v9188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:10 smithi090 ceph-mon[29180]: pgmap v9188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15952.224105 secs 2024-04-26T23:01:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15952.224105 secs 2024-04-26T23:01:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15952.224105 secs 2024-04-26T23:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15952.986128 secs 2024-04-26T23:01:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:12 smithi119 ceph-mon[32752]: pgmap v9189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15952.986128 secs 2024-04-26T23:01:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:12 smithi167 ceph-mon[32751]: pgmap v9189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15952.986128 secs 2024-04-26T23:01:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:12 smithi090 ceph-mon[29180]: pgmap v9189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:14 smithi119 ceph-mon[32752]: pgmap v9190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:14 smithi167 ceph-mon[32751]: pgmap v9190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:14 smithi090 ceph-mon[29180]: pgmap v9190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15957.224166 secs 2024-04-26T23:01:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:16 smithi119 ceph-mon[32752]: pgmap v9191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15957.224166 secs 2024-04-26T23:01:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:16 smithi167 ceph-mon[32751]: pgmap v9191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15957.224166 secs 2024-04-26T23:01:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:16 smithi090 ceph-mon[29180]: pgmap v9191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15957.986247 secs 2024-04-26T23:01:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15957.986247 secs 2024-04-26T23:01:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15957.986247 secs 2024-04-26T23:01:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:18 smithi167 ceph-mon[32751]: pgmap v9192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:18 smithi090 ceph-mon[29180]: pgmap v9192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:18 smithi119 ceph-mon[32752]: pgmap v9192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:20 smithi167 ceph-mon[32751]: pgmap v9193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:20 smithi090 ceph-mon[29180]: pgmap v9193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:20 smithi119 ceph-mon[32752]: pgmap v9193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15962.224249 secs 2024-04-26T23:01:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15962.224249 secs 2024-04-26T23:01:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15962.224249 secs 2024-04-26T23:01:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15962.986366 secs 2024-04-26T23:01:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:22 smithi167 ceph-mon[32751]: pgmap v9194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15962.986366 secs 2024-04-26T23:01:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:22 smithi090 ceph-mon[29180]: pgmap v9194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15962.986366 secs 2024-04-26T23:01:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:22 smithi119 ceph-mon[32752]: pgmap v9194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:24 smithi090 ceph-mon[29180]: pgmap v9195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:24 smithi119 ceph-mon[32752]: pgmap v9195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:24 smithi167 ceph-mon[32751]: pgmap v9195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15967.224334 secs 2024-04-26T23:01:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:26 smithi090 ceph-mon[29180]: pgmap v9196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15967.224334 secs 2024-04-26T23:01:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:26 smithi119 ceph-mon[32752]: pgmap v9196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15967.224334 secs 2024-04-26T23:01:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:26 smithi167 ceph-mon[32751]: pgmap v9196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15967.986399 secs 2024-04-26T23:01:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15967.986399 secs 2024-04-26T23:01:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15967.986399 secs 2024-04-26T23:01:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:28 smithi090 ceph-mon[29180]: pgmap v9197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:28 smithi119 ceph-mon[32752]: pgmap v9197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:28 smithi167 ceph-mon[32751]: pgmap v9197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:01:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:01:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:01:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:30 smithi090 ceph-mon[29180]: pgmap v9198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:01:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:01:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:01:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:30 smithi119 ceph-mon[32752]: pgmap v9198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:01:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:01:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:01:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:30 smithi167 ceph-mon[32751]: pgmap v9198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:01:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:01:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:01:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:31 smithi090 ceph-mon[29180]: pgmap v9199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:01:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:31 smithi090 ceph-mon[29180]: pgmap v9200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15972.224417 secs 2024-04-26T23:01:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:31 smithi119 ceph-mon[32752]: pgmap v9199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:01:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:31 smithi119 ceph-mon[32752]: pgmap v9200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15972.224417 secs 2024-04-26T23:01:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:31 smithi167 ceph-mon[32751]: pgmap v9199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:01:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:31 smithi167 ceph-mon[32751]: pgmap v9200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15972.224417 secs 2024-04-26T23:01:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15972.986427 secs 2024-04-26T23:01:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15972.986427 secs 2024-04-26T23:01:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15972.986427 secs 2024-04-26T23:01:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:33 smithi090 ceph-mon[29180]: pgmap v9201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:33 smithi119 ceph-mon[32752]: pgmap v9201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:33 smithi167 ceph-mon[32751]: pgmap v9201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:35 smithi090 ceph-mon[29180]: pgmap v9202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:35 smithi119 ceph-mon[32752]: pgmap v9202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:35 smithi167 ceph-mon[32751]: pgmap v9202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15977.224501 secs 2024-04-26T23:01:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15977.224501 secs 2024-04-26T23:01:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15977.224501 secs 2024-04-26T23:01:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15977.986574 secs 2024-04-26T23:01:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:37 smithi090 ceph-mon[29180]: pgmap v9203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15977.986574 secs 2024-04-26T23:01:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:37 smithi119 ceph-mon[32752]: pgmap v9203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15977.986574 secs 2024-04-26T23:01:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:37 smithi167 ceph-mon[32751]: pgmap v9203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:01:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:39 smithi090 ceph-mon[29180]: pgmap v9204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:01:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:39 smithi119 ceph-mon[32752]: pgmap v9204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:01:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:39 smithi167 ceph-mon[32751]: pgmap v9204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:01:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:41 smithi090 ceph-mon[29180]: pgmap v9205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:01:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15982.224584 secs 2024-04-26T23:01:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:41 smithi119 ceph-mon[32752]: pgmap v9205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:01:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15982.224584 secs 2024-04-26T23:01:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:41 smithi167 ceph-mon[32751]: pgmap v9205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:01:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15982.224584 secs 2024-04-26T23:01:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15982.986711 secs 2024-04-26T23:01:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15982.986711 secs 2024-04-26T23:01:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15982.986711 secs 2024-04-26T23:01:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:43 smithi119 ceph-mon[32752]: pgmap v9206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:43 smithi167 ceph-mon[32751]: pgmap v9206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:43 smithi090 ceph-mon[29180]: pgmap v9206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:44 smithi119 ceph-mon[32752]: pgmap v9207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:44 smithi167 ceph-mon[32751]: pgmap v9207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:44 smithi090 ceph-mon[29180]: pgmap v9207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:45 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15987.224671 secs 2024-04-26T23:01:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:45 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15987.224671 secs 2024-04-26T23:01:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:45 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15987.224671 secs 2024-04-26T23:01:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:46 smithi119 ceph-mon[32752]: pgmap v9208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:46 smithi167 ceph-mon[32751]: pgmap v9208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:46 smithi090 ceph-mon[29180]: pgmap v9208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:47.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15987.986836 secs 2024-04-26T23:01:48.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15987.986836 secs 2024-04-26T23:01:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15987.986836 secs 2024-04-26T23:01:49.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:48 smithi119 ceph-mon[32752]: pgmap v9209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:48 smithi167 ceph-mon[32751]: pgmap v9209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:48 smithi090 ceph-mon[29180]: pgmap v9209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:50 smithi090 ceph-mon[29180]: pgmap v9210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:50 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15992.224756 secs 2024-04-26T23:01:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:50 smithi119 ceph-mon[32752]: pgmap v9210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:50 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15992.224756 secs 2024-04-26T23:01:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:50 smithi167 ceph-mon[32751]: pgmap v9210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:50 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15992.224756 secs 2024-04-26T23:01:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15992.986915 secs 2024-04-26T23:01:52.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15992.986915 secs 2024-04-26T23:01:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15992.986915 secs 2024-04-26T23:01:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:52 smithi090 ceph-mon[29180]: pgmap v9211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:52 smithi119 ceph-mon[32752]: pgmap v9211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:52 smithi167 ceph-mon[32751]: pgmap v9211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:54 smithi090 ceph-mon[29180]: pgmap v9212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:54 smithi119 ceph-mon[32752]: pgmap v9212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:54 smithi167 ceph-mon[32751]: pgmap v9212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:01:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:55 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 15997.224847 secs 2024-04-26T23:01:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:55 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 15997.224847 secs 2024-04-26T23:01:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:55 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 15997.224847 secs 2024-04-26T23:01:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 15997.987028 secs 2024-04-26T23:01:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:56 smithi090 ceph-mon[29180]: pgmap v9213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 15997.987028 secs 2024-04-26T23:01:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:56 smithi119 ceph-mon[32752]: pgmap v9213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 15997.987028 secs 2024-04-26T23:01:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:56 smithi167 ceph-mon[32751]: pgmap v9213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:01:58 smithi090 ceph-mon[29180]: pgmap v9214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:01:58 smithi119 ceph-mon[32752]: pgmap v9214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:01:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:01:58 smithi167 ceph-mon[32751]: pgmap v9214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:00 smithi090 ceph-mon[29180]: pgmap v9215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:00 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16002.224933 secs 2024-04-26T23:02:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:00 smithi119 ceph-mon[32752]: pgmap v9215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:00 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16002.224933 secs 2024-04-26T23:02:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:00 smithi167 ceph-mon[32751]: pgmap v9215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:00 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16002.224933 secs 2024-04-26T23:02:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16002.987178 secs 2024-04-26T23:02:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16002.987178 secs 2024-04-26T23:02:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16002.987178 secs 2024-04-26T23:02:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:02 smithi119 ceph-mon[32752]: pgmap v9216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:02 smithi167 ceph-mon[32751]: pgmap v9216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:02 smithi090 ceph-mon[29180]: pgmap v9216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:04 smithi119 ceph-mon[32752]: pgmap v9217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:04 smithi167 ceph-mon[32751]: pgmap v9217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:04 smithi090 ceph-mon[29180]: pgmap v9217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:05 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 16007.225023 secs 2024-04-26T23:02:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:05 smithi119 ceph-mon[32752]: slow request 11523.006465 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:02:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:05 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 16007.225023 secs 2024-04-26T23:02:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:05 smithi167 ceph-mon[32751]: slow request 11523.006465 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:02:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:05 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 16007.225023 secs 2024-04-26T23:02:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:05 smithi090 ceph-mon[29180]: slow request 11523.006465 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:02:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16007.987265 secs 2024-04-26T23:02:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:06 smithi119 ceph-mon[32752]: pgmap v9218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16007.987265 secs 2024-04-26T23:02:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:06 smithi167 ceph-mon[32751]: pgmap v9218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16007.987265 secs 2024-04-26T23:02:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:06 smithi090 ceph-mon[29180]: pgmap v9218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:08 smithi119 ceph-mon[32752]: pgmap v9219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:08 smithi167 ceph-mon[32751]: pgmap v9219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:08 smithi090 ceph-mon[29180]: pgmap v9219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:10 smithi119 ceph-mon[32752]: pgmap v9220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:10 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16012.225083 secs 2024-04-26T23:02:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:10 smithi167 ceph-mon[32751]: pgmap v9220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:10 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16012.225083 secs 2024-04-26T23:02:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:10 smithi090 ceph-mon[29180]: pgmap v9220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:10 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16012.225083 secs 2024-04-26T23:02:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16012.987401 secs 2024-04-26T23:02:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16012.987401 secs 2024-04-26T23:02:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16012.987401 secs 2024-04-26T23:02:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:12 smithi119 ceph-mon[32752]: pgmap v9221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:12 smithi167 ceph-mon[32751]: pgmap v9221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:12 smithi090 ceph-mon[29180]: pgmap v9221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:14 smithi119 ceph-mon[32752]: pgmap v9222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:14 smithi167 ceph-mon[32751]: pgmap v9222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:14 smithi090 ceph-mon[29180]: pgmap v9222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:15 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16017.225185 secs 2024-04-26T23:02:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:15 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16017.225185 secs 2024-04-26T23:02:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:15 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16017.225185 secs 2024-04-26T23:02:17.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16017.987531 secs 2024-04-26T23:02:17.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:16 smithi167 ceph-mon[32751]: pgmap v9223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16017.987531 secs 2024-04-26T23:02:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:16 smithi119 ceph-mon[32752]: pgmap v9223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16017.987531 secs 2024-04-26T23:02:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:16 smithi090 ceph-mon[29180]: pgmap v9223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:18 smithi119 ceph-mon[32752]: pgmap v9224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:18 smithi167 ceph-mon[32751]: pgmap v9224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:18 smithi090 ceph-mon[29180]: pgmap v9224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:20 smithi119 ceph-mon[32752]: pgmap v9225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:20 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16022.225250 secs 2024-04-26T23:02:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:20 smithi167 ceph-mon[32751]: pgmap v9225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:20 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16022.225250 secs 2024-04-26T23:02:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:20 smithi090 ceph-mon[29180]: pgmap v9225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:20 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16022.225250 secs 2024-04-26T23:02:22.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16022.987668 secs 2024-04-26T23:02:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16022.987668 secs 2024-04-26T23:02:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16022.987668 secs 2024-04-26T23:02:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:22 smithi119 ceph-mon[32752]: pgmap v9226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:22 smithi167 ceph-mon[32751]: pgmap v9226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:22 smithi090 ceph-mon[29180]: pgmap v9226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:24 smithi119 ceph-mon[32752]: pgmap v9227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:24 smithi167 ceph-mon[32751]: pgmap v9227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:24 smithi090 ceph-mon[29180]: pgmap v9227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:25 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16027.225337 secs 2024-04-26T23:02:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:25 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16027.225337 secs 2024-04-26T23:02:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:25 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16027.225337 secs 2024-04-26T23:02:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16027.987799 secs 2024-04-26T23:02:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:26 smithi119 ceph-mon[32752]: pgmap v9228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16027.987799 secs 2024-04-26T23:02:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:26 smithi167 ceph-mon[32751]: pgmap v9228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16027.987799 secs 2024-04-26T23:02:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:26 smithi090 ceph-mon[29180]: pgmap v9228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:28 smithi119 ceph-mon[32752]: pgmap v9229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:28 smithi167 ceph-mon[32751]: pgmap v9229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:28 smithi090 ceph-mon[29180]: pgmap v9229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:02:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:02:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:02:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:30 smithi119 ceph-mon[32752]: pgmap v9230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:30 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16032.225432 secs 2024-04-26T23:02:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:02:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:02:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:30 smithi119 ceph-mon[32752]: pgmap v9231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:02:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:30 smithi119 ceph-mon[32752]: pgmap v9232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:02:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:30 smithi167 ceph-mon[32751]: pgmap v9230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:30 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16032.225432 secs 2024-04-26T23:02:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:02:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:02:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:30 smithi167 ceph-mon[32751]: pgmap v9231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:02:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:30 smithi167 ceph-mon[32751]: pgmap v9232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:02:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:30 smithi090 ceph-mon[29180]: pgmap v9230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:30 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16032.225432 secs 2024-04-26T23:02:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:02:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:02:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:30 smithi090 ceph-mon[29180]: pgmap v9231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:02:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:30 smithi090 ceph-mon[29180]: pgmap v9232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:02:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16032.987856 secs 2024-04-26T23:02:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16032.987856 secs 2024-04-26T23:02:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16032.987856 secs 2024-04-26T23:02:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:32 smithi119 ceph-mon[32752]: pgmap v9233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:02:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:32 smithi167 ceph-mon[32751]: pgmap v9233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:02:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:32 smithi090 ceph-mon[29180]: pgmap v9233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:02:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:35 smithi119 ceph-mon[32752]: pgmap v9234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:35 smithi167 ceph-mon[32751]: pgmap v9234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:35 smithi090 ceph-mon[29180]: pgmap v9234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:37.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16037.225517 secs 2024-04-26T23:02:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16037.225517 secs 2024-04-26T23:02:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16037.225517 secs 2024-04-26T23:02:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16037.987966 secs 2024-04-26T23:02:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:37 smithi119 ceph-mon[32752]: pgmap v9235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16037.987966 secs 2024-04-26T23:02:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:37 smithi167 ceph-mon[32751]: pgmap v9235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16037.987966 secs 2024-04-26T23:02:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:37 smithi090 ceph-mon[29180]: pgmap v9235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:02:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:38 smithi119 ceph-mon[32752]: pgmap v9236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T23:02:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:38 smithi167 ceph-mon[32751]: pgmap v9236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T23:02:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:38 smithi090 ceph-mon[29180]: pgmap v9236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T23:02:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16042.225601 secs 2024-04-26T23:02:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:41 smithi119 ceph-mon[32752]: pgmap v9237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:02:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16042.225601 secs 2024-04-26T23:02:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:41 smithi167 ceph-mon[32751]: pgmap v9237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:02:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16042.225601 secs 2024-04-26T23:02:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:41 smithi090 ceph-mon[29180]: pgmap v9237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:02:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16042.988081 secs 2024-04-26T23:02:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16042.988081 secs 2024-04-26T23:02:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16042.988081 secs 2024-04-26T23:02:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:43 smithi119 ceph-mon[32752]: pgmap v9238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:02:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:43 smithi167 ceph-mon[32751]: pgmap v9238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:02:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:43 smithi090 ceph-mon[29180]: pgmap v9238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:02:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:44 smithi119 ceph-mon[32752]: pgmap v9239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:02:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:44 smithi167 ceph-mon[32751]: pgmap v9239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:02:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:44 smithi090 ceph-mon[29180]: pgmap v9239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:02:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:45 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16047.225685 secs 2024-04-26T23:02:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:45 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16047.225685 secs 2024-04-26T23:02:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:45 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16047.225685 secs 2024-04-26T23:02:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16047.988214 secs 2024-04-26T23:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:46 smithi119 ceph-mon[32752]: pgmap v9240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:02:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16047.988214 secs 2024-04-26T23:02:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:46 smithi167 ceph-mon[32751]: pgmap v9240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16047.988214 secs 2024-04-26T23:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:46 smithi090 ceph-mon[29180]: pgmap v9240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:02:50.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:49 smithi119 ceph-mon[32752]: pgmap v9241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:02:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:49 smithi167 ceph-mon[32751]: pgmap v9241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:02:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:49 smithi090 ceph-mon[29180]: pgmap v9241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:02:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16052.225771 secs 2024-04-26T23:02:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:51 smithi119 ceph-mon[32752]: pgmap v9242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16052.225771 secs 2024-04-26T23:02:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:51 smithi167 ceph-mon[32751]: pgmap v9242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16052.225771 secs 2024-04-26T23:02:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:51 smithi090 ceph-mon[29180]: pgmap v9242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:52.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16052.988330 secs 2024-04-26T23:02:53.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16052.988330 secs 2024-04-26T23:02:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16052.988330 secs 2024-04-26T23:02:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:53 smithi119 ceph-mon[32752]: pgmap v9243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:53 smithi167 ceph-mon[32751]: pgmap v9243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:53 smithi090 ceph-mon[29180]: pgmap v9243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:54 smithi119 ceph-mon[32752]: pgmap v9244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:54 smithi167 ceph-mon[32751]: pgmap v9244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:54 smithi090 ceph-mon[29180]: pgmap v9244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:02:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:55 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16057.225824 secs 2024-04-26T23:02:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:55 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16057.225824 secs 2024-04-26T23:02:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:55 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16057.225824 secs 2024-04-26T23:02:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16057.988424 secs 2024-04-26T23:02:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:56 smithi119 ceph-mon[32752]: pgmap v9245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16057.988424 secs 2024-04-26T23:02:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:56 smithi167 ceph-mon[32751]: pgmap v9245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:02:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16057.988424 secs 2024-04-26T23:02:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:56 smithi090 ceph-mon[29180]: pgmap v9245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:00.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:02:59 smithi119 ceph-mon[32752]: pgmap v9246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:02:59 smithi167 ceph-mon[32751]: pgmap v9246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:02:59 smithi090 ceph-mon[29180]: pgmap v9246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:00 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16062.225886 secs 2024-04-26T23:03:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:00 smithi119 ceph-mon[32752]: pgmap v9247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:00 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16062.225886 secs 2024-04-26T23:03:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:00 smithi167 ceph-mon[32751]: pgmap v9247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:00 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16062.225886 secs 2024-04-26T23:03:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:00 smithi090 ceph-mon[29180]: pgmap v9247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16062.988547 secs 2024-04-26T23:03:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16062.988547 secs 2024-04-26T23:03:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16062.988547 secs 2024-04-26T23:03:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:02 smithi119 ceph-mon[32752]: pgmap v9248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:02 smithi167 ceph-mon[32751]: pgmap v9248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:02 smithi090 ceph-mon[29180]: pgmap v9248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:03:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:03:05.115+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:03:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:03:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:03:05.115+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:03:05.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:03:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:03:05.115+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:03:05.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:03:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:03:05.115+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:03:05.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:03:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:03:05.115+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:03:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:05 smithi119 ceph-mon[32752]: pgmap v9249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:05 smithi167 ceph-mon[32751]: pgmap v9249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:05 smithi090 ceph-mon[29180]: pgmap v9249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 16067.225948 secs 2024-04-26T23:03:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 16067.225948 secs 2024-04-26T23:03:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 16067.225948 secs 2024-04-26T23:03:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16067.988667 secs 2024-04-26T23:03:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:07 smithi119 ceph-mon[32752]: pgmap v9250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16067.988667 secs 2024-04-26T23:03:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:07 smithi167 ceph-mon[32751]: pgmap v9250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16067.988667 secs 2024-04-26T23:03:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:07 smithi090 ceph-mon[29180]: pgmap v9250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:08 smithi090 ceph-mon[29180]: pgmap v9251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:08 smithi119 ceph-mon[32752]: pgmap v9251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:08 smithi167 ceph-mon[32751]: pgmap v9251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:10 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16072.226008 secs 2024-04-26T23:03:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:10 smithi119 ceph-mon[32752]: pgmap v9252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:10 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16072.226008 secs 2024-04-26T23:03:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:10 smithi167 ceph-mon[32751]: pgmap v9252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:10 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16072.226008 secs 2024-04-26T23:03:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:10 smithi090 ceph-mon[29180]: pgmap v9252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16072.988786 secs 2024-04-26T23:03:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16072.988786 secs 2024-04-26T23:03:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16072.988786 secs 2024-04-26T23:03:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:13 smithi090 ceph-mon[29180]: pgmap v9253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:13 smithi119 ceph-mon[32752]: pgmap v9253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:13 smithi167 ceph-mon[32751]: pgmap v9253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:03:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:03:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:15 smithi090 ceph-mon[29180]: pgmap v9254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:03:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:03:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:15 smithi119 ceph-mon[32752]: pgmap v9254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:03:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:03:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:15 smithi167 ceph-mon[32751]: pgmap v9254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:16 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16077.226091 secs 2024-04-26T23:03:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:16 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16077.226091 secs 2024-04-26T23:03:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:16 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16077.226091 secs 2024-04-26T23:03:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16077.988909 secs 2024-04-26T23:03:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:17 smithi090 ceph-mon[29180]: pgmap v9255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16077.988909 secs 2024-04-26T23:03:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:17 smithi119 ceph-mon[32752]: pgmap v9255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16077.988909 secs 2024-04-26T23:03:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:17 smithi167 ceph-mon[32751]: pgmap v9255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:18 smithi119 ceph-mon[32752]: pgmap v9256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:18 smithi167 ceph-mon[32751]: pgmap v9256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:18 smithi090 ceph-mon[29180]: pgmap v9256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:03:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:20 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16082.226184 secs 2024-04-26T23:03:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:20 smithi119 ceph-mon[32752]: pgmap v9257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:20 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16082.226184 secs 2024-04-26T23:03:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:20 smithi167 ceph-mon[32751]: pgmap v9257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:20 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16082.226184 secs 2024-04-26T23:03:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:20 smithi090 ceph-mon[29180]: pgmap v9257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16082.989027 secs 2024-04-26T23:03:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16082.989027 secs 2024-04-26T23:03:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16082.989027 secs 2024-04-26T23:03:23.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:22 smithi090 ceph-mon[29180]: pgmap v9258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:22 smithi119 ceph-mon[32752]: pgmap v9258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:22 smithi167 ceph-mon[32751]: pgmap v9258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:24 smithi119 ceph-mon[32752]: pgmap v9259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:24 smithi167 ceph-mon[32751]: pgmap v9259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:24 smithi090 ceph-mon[29180]: pgmap v9259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:25 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16087.226270 secs 2024-04-26T23:03:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:25 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16087.226270 secs 2024-04-26T23:03:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:25 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16087.226270 secs 2024-04-26T23:03:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:26 smithi119 ceph-mon[32752]: pgmap v9260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:26 smithi167 ceph-mon[32751]: pgmap v9260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:26 smithi090 ceph-mon[29180]: pgmap v9260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16087.989148 secs 2024-04-26T23:03:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16087.989148 secs 2024-04-26T23:03:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16087.989148 secs 2024-04-26T23:03:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:28 smithi119 ceph-mon[32752]: pgmap v9261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:28 smithi167 ceph-mon[32751]: pgmap v9261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:28 smithi090 ceph-mon[29180]: pgmap v9261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:30 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16092.226351 secs 2024-04-26T23:03:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:30 smithi119 ceph-mon[32752]: pgmap v9262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:03:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:30 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16092.226351 secs 2024-04-26T23:03:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:30 smithi167 ceph-mon[32751]: pgmap v9262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:03:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:30 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16092.226351 secs 2024-04-26T23:03:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:30 smithi090 ceph-mon[29180]: pgmap v9262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:03:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:03:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:03:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:31 smithi119 ceph-mon[32752]: pgmap v9263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:03:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:31 smithi119 ceph-mon[32752]: pgmap v9264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:03:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:03:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:03:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:31 smithi167 ceph-mon[32751]: pgmap v9263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:03:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:31 smithi167 ceph-mon[32751]: pgmap v9264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:03:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:03:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:03:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:31 smithi090 ceph-mon[29180]: pgmap v9263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:03:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:31 smithi090 ceph-mon[29180]: pgmap v9264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:03:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16092.989243 secs 2024-04-26T23:03:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16092.989243 secs 2024-04-26T23:03:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16092.989243 secs 2024-04-26T23:03:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:33 smithi119 ceph-mon[32752]: pgmap v9265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:33 smithi167 ceph-mon[32751]: pgmap v9265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:33 smithi090 ceph-mon[29180]: pgmap v9265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:36 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16097.226434 secs 2024-04-26T23:03:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:36 smithi119 ceph-mon[32752]: pgmap v9266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:36 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16097.226434 secs 2024-04-26T23:03:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:36 smithi167 ceph-mon[32751]: pgmap v9266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:36 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16097.226434 secs 2024-04-26T23:03:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:36 smithi090 ceph-mon[29180]: pgmap v9266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16097.989386 secs 2024-04-26T23:03:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16097.989386 secs 2024-04-26T23:03:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16097.989386 secs 2024-04-26T23:03:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:38 smithi119 ceph-mon[32752]: pgmap v9267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:03:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:38 smithi167 ceph-mon[32751]: pgmap v9267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:03:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:38 smithi090 ceph-mon[29180]: pgmap v9267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:03:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:40 smithi119 ceph-mon[32752]: pgmap v9268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:40 smithi167 ceph-mon[32751]: pgmap v9268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:40 smithi090 ceph-mon[29180]: pgmap v9268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:03:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:41 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16102.226520 secs 2024-04-26T23:03:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:41 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16102.226520 secs 2024-04-26T23:03:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:41 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16102.226520 secs 2024-04-26T23:03:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16102.989508 secs 2024-04-26T23:03:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:42 smithi119 ceph-mon[32752]: pgmap v9269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:03:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16102.989508 secs 2024-04-26T23:03:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:42 smithi167 ceph-mon[32751]: pgmap v9269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:03:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16102.989508 secs 2024-04-26T23:03:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:42 smithi090 ceph-mon[29180]: pgmap v9269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:03:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:44 smithi119 ceph-mon[32752]: pgmap v9270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:44 smithi167 ceph-mon[32751]: pgmap v9270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:44 smithi090 ceph-mon[29180]: pgmap v9270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:46 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16107.226602 secs 2024-04-26T23:03:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:46 smithi119 ceph-mon[32752]: pgmap v9271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:46 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16107.226602 secs 2024-04-26T23:03:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:46 smithi167 ceph-mon[32751]: pgmap v9271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:46 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16107.226602 secs 2024-04-26T23:03:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:46 smithi090 ceph-mon[29180]: pgmap v9271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16107.989646 secs 2024-04-26T23:03:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16107.989646 secs 2024-04-26T23:03:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16107.989646 secs 2024-04-26T23:03:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:48 smithi119 ceph-mon[32752]: pgmap v9272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:48 smithi167 ceph-mon[32751]: pgmap v9272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:48 smithi090 ceph-mon[29180]: pgmap v9272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:50 smithi119 ceph-mon[32752]: pgmap v9273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:50 smithi167 ceph-mon[32751]: pgmap v9273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:50 smithi090 ceph-mon[29180]: pgmap v9273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:51 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16112.226685 secs 2024-04-26T23:03:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:51 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16112.226685 secs 2024-04-26T23:03:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:51 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16112.226685 secs 2024-04-26T23:03:52.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16112.989755 secs 2024-04-26T23:03:52.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:52 smithi119 ceph-mon[32752]: pgmap v9274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16112.989755 secs 2024-04-26T23:03:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:52 smithi167 ceph-mon[32751]: pgmap v9274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16112.989755 secs 2024-04-26T23:03:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:52 smithi090 ceph-mon[29180]: pgmap v9274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:54 smithi119 ceph-mon[32752]: pgmap v9275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:54 smithi167 ceph-mon[32751]: pgmap v9275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:54 smithi090 ceph-mon[29180]: pgmap v9275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:03:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:56 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16117.226767 secs 2024-04-26T23:03:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:56 smithi119 ceph-mon[32752]: pgmap v9276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:56 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16117.226767 secs 2024-04-26T23:03:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:56 smithi167 ceph-mon[32751]: pgmap v9276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:56 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16117.226767 secs 2024-04-26T23:03:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:56 smithi090 ceph-mon[29180]: pgmap v9276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16117.989840 secs 2024-04-26T23:03:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16117.989840 secs 2024-04-26T23:03:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16117.989840 secs 2024-04-26T23:03:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:03:58 smithi119 ceph-mon[32752]: pgmap v9277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:03:58 smithi167 ceph-mon[32751]: pgmap v9277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:03:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:03:58 smithi090 ceph-mon[29180]: pgmap v9277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:00 smithi119 ceph-mon[32752]: pgmap v9278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:00 smithi167 ceph-mon[32751]: pgmap v9278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:00 smithi090 ceph-mon[29180]: pgmap v9278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:01 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16122.226826 secs 2024-04-26T23:04:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:01 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16122.226826 secs 2024-04-26T23:04:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:01 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16122.226826 secs 2024-04-26T23:04:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16122.989956 secs 2024-04-26T23:04:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:02 smithi119 ceph-mon[32752]: pgmap v9279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16122.989956 secs 2024-04-26T23:04:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:02 smithi167 ceph-mon[32751]: pgmap v9279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16122.989956 secs 2024-04-26T23:04:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:02 smithi090 ceph-mon[29180]: pgmap v9279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:04 smithi119 ceph-mon[32752]: pgmap v9280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:04 smithi167 ceph-mon[32751]: pgmap v9280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:04 smithi090 ceph-mon[29180]: pgmap v9280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:06 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16127.226888 secs 2024-04-26T23:04:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:06 smithi119 ceph-mon[32752]: pgmap v9281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:06 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16127.226888 secs 2024-04-26T23:04:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:06 smithi167 ceph-mon[32751]: pgmap v9281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:06 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16127.226888 secs 2024-04-26T23:04:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:06 smithi090 ceph-mon[29180]: pgmap v9281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16127.990019 secs 2024-04-26T23:04:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16127.990019 secs 2024-04-26T23:04:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16127.990019 secs 2024-04-26T23:04:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:08 smithi119 ceph-mon[32752]: pgmap v9282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:08 smithi167 ceph-mon[32751]: pgmap v9282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:08 smithi090 ceph-mon[29180]: pgmap v9282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:10 smithi119 ceph-mon[32752]: pgmap v9283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:10 smithi167 ceph-mon[32751]: pgmap v9283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:10 smithi090 ceph-mon[29180]: pgmap v9283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:11 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16132.226968 secs 2024-04-26T23:04:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:11 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16132.226968 secs 2024-04-26T23:04:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:11 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16132.226968 secs 2024-04-26T23:04:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16132.990120 secs 2024-04-26T23:04:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:12 smithi119 ceph-mon[32752]: pgmap v9284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16132.990120 secs 2024-04-26T23:04:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:12 smithi167 ceph-mon[32751]: pgmap v9284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16132.990120 secs 2024-04-26T23:04:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:12 smithi090 ceph-mon[29180]: pgmap v9284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:14 smithi119 ceph-mon[32752]: pgmap v9285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:14 smithi167 ceph-mon[32751]: pgmap v9285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:14 smithi090 ceph-mon[29180]: pgmap v9285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:16 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16137.227031 secs 2024-04-26T23:04:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:16 smithi119 ceph-mon[32752]: pgmap v9286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:16 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16137.227031 secs 2024-04-26T23:04:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:16 smithi167 ceph-mon[32751]: pgmap v9286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:16 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16137.227031 secs 2024-04-26T23:04:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:16 smithi090 ceph-mon[29180]: pgmap v9286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16137.990254 secs 2024-04-26T23:04:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16137.990254 secs 2024-04-26T23:04:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16137.990254 secs 2024-04-26T23:04:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:18 smithi119 ceph-mon[32752]: pgmap v9287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:18 smithi167 ceph-mon[32751]: pgmap v9287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:18 smithi090 ceph-mon[29180]: pgmap v9287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:20 smithi119 ceph-mon[32752]: pgmap v9288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:20 smithi167 ceph-mon[32751]: pgmap v9288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:20 smithi090 ceph-mon[29180]: pgmap v9288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:21 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16142.227092 secs 2024-04-26T23:04:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:21 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16142.227092 secs 2024-04-26T23:04:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:21 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16142.227092 secs 2024-04-26T23:04:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16142.990406 secs 2024-04-26T23:04:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:22 smithi119 ceph-mon[32752]: pgmap v9289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16142.990406 secs 2024-04-26T23:04:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:22 smithi167 ceph-mon[32751]: pgmap v9289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16142.990406 secs 2024-04-26T23:04:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:22 smithi090 ceph-mon[29180]: pgmap v9289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:24 smithi119 ceph-mon[32752]: pgmap v9290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:24 smithi167 ceph-mon[32751]: pgmap v9290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:24 smithi090 ceph-mon[29180]: pgmap v9290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:26 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16147.227168 secs 2024-04-26T23:04:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:26 smithi119 ceph-mon[32752]: pgmap v9291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:26 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16147.227168 secs 2024-04-26T23:04:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:26 smithi167 ceph-mon[32751]: pgmap v9291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:26 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16147.227168 secs 2024-04-26T23:04:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:26 smithi090 ceph-mon[29180]: pgmap v9291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16147.990516 secs 2024-04-26T23:04:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16147.990516 secs 2024-04-26T23:04:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16147.990516 secs 2024-04-26T23:04:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:28 smithi119 ceph-mon[32752]: pgmap v9292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:28 smithi167 ceph-mon[32751]: pgmap v9292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:28 smithi090 ceph-mon[29180]: pgmap v9292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:30 smithi167 ceph-mon[32751]: pgmap v9293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:30 smithi090 ceph-mon[29180]: pgmap v9293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:30 smithi119 ceph-mon[32752]: pgmap v9293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:31 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16152.227229 secs 2024-04-26T23:04:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:04:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:31 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16152.227229 secs 2024-04-26T23:04:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:04:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:31 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16152.227229 secs 2024-04-26T23:04:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:04:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16152.990610 secs 2024-04-26T23:04:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:32 smithi167 ceph-mon[32751]: pgmap v9294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:04:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:04:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:04:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16152.990610 secs 2024-04-26T23:04:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:32 smithi090 ceph-mon[29180]: pgmap v9294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:04:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:04:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:04:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16152.990610 secs 2024-04-26T23:04:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:32 smithi119 ceph-mon[32752]: pgmap v9294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:04:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:04:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:04:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:33 smithi167 ceph-mon[32751]: pgmap v9295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:04:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:33 smithi167 ceph-mon[32751]: pgmap v9296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:04:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:33 smithi090 ceph-mon[29180]: pgmap v9295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:04:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:33 smithi090 ceph-mon[29180]: pgmap v9296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:04:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:33 smithi119 ceph-mon[32752]: pgmap v9295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:04:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:33 smithi119 ceph-mon[32752]: pgmap v9296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:04:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:35 smithi090 ceph-mon[29180]: pgmap v9297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:35 smithi119 ceph-mon[32752]: pgmap v9297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:35 smithi167 ceph-mon[32751]: pgmap v9297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:36 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16157.227290 secs 2024-04-26T23:04:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:36 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16157.227290 secs 2024-04-26T23:04:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:36 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16157.227290 secs 2024-04-26T23:04:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16157.990689 secs 2024-04-26T23:04:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:37 smithi090 ceph-mon[29180]: pgmap v9298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16157.990689 secs 2024-04-26T23:04:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:37 smithi119 ceph-mon[32752]: pgmap v9298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16157.990689 secs 2024-04-26T23:04:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:37 smithi167 ceph-mon[32751]: pgmap v9298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:39 smithi090 ceph-mon[29180]: pgmap v9299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:04:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:39 smithi119 ceph-mon[32752]: pgmap v9299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:04:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:39 smithi167 ceph-mon[32751]: pgmap v9299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:04:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:41 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16162.227375 secs 2024-04-26T23:04:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:41 smithi090 ceph-mon[29180]: pgmap v9300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:41 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16162.227375 secs 2024-04-26T23:04:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:41 smithi119 ceph-mon[32752]: pgmap v9300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:41 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16162.227375 secs 2024-04-26T23:04:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:41 smithi167 ceph-mon[32751]: pgmap v9300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:04:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16162.990839 secs 2024-04-26T23:04:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16162.990839 secs 2024-04-26T23:04:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16162.990839 secs 2024-04-26T23:04:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:43 smithi090 ceph-mon[29180]: pgmap v9301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:04:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:43 smithi119 ceph-mon[32752]: pgmap v9301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:04:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:43 smithi167 ceph-mon[32751]: pgmap v9301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:04:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:45 smithi090 ceph-mon[29180]: pgmap v9302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:45 smithi119 ceph-mon[32752]: pgmap v9302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:45 smithi167 ceph-mon[32751]: pgmap v9302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:46 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16167.227437 secs 2024-04-26T23:04:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:46 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16167.227437 secs 2024-04-26T23:04:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:46 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16167.227437 secs 2024-04-26T23:04:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16167.990902 secs 2024-04-26T23:04:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:47 smithi090 ceph-mon[29180]: pgmap v9303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:48.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16167.990902 secs 2024-04-26T23:04:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:47 smithi119 ceph-mon[32752]: pgmap v9303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16167.990902 secs 2024-04-26T23:04:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:47 smithi167 ceph-mon[32751]: pgmap v9303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:49 smithi090 ceph-mon[29180]: pgmap v9304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:50.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:49 smithi119 ceph-mon[32752]: pgmap v9304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:49 smithi167 ceph-mon[32751]: pgmap v9304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:51.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:51 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16172.227523 secs 2024-04-26T23:04:51.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:51 smithi167 ceph-mon[32751]: pgmap v9305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:51 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16172.227523 secs 2024-04-26T23:04:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:51 smithi090 ceph-mon[29180]: pgmap v9305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:51 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16172.227523 secs 2024-04-26T23:04:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:51 smithi119 ceph-mon[32752]: pgmap v9305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16172.991020 secs 2024-04-26T23:04:52.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16172.991020 secs 2024-04-26T23:04:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16172.991020 secs 2024-04-26T23:04:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:53 smithi090 ceph-mon[29180]: pgmap v9306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:53 smithi119 ceph-mon[32752]: pgmap v9306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:53 smithi167 ceph-mon[32751]: pgmap v9306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:55 smithi090 ceph-mon[29180]: pgmap v9307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:55 smithi119 ceph-mon[32752]: pgmap v9307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:55 smithi167 ceph-mon[32751]: pgmap v9307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:04:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:56 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16177.227582 secs 2024-04-26T23:04:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:56 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16177.227582 secs 2024-04-26T23:04:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:56 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16177.227582 secs 2024-04-26T23:04:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16177.991154 secs 2024-04-26T23:04:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:57 smithi090 ceph-mon[29180]: pgmap v9308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16177.991154 secs 2024-04-26T23:04:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:57 smithi119 ceph-mon[32752]: pgmap v9308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16177.991154 secs 2024-04-26T23:04:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:57 smithi167 ceph-mon[32751]: pgmap v9308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:04:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:04:59 smithi090 ceph-mon[29180]: pgmap v9309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:04:59 smithi119 ceph-mon[32752]: pgmap v9309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:04:59 smithi167 ceph-mon[32751]: pgmap v9309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:01 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16182.227667 secs 2024-04-26T23:05:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:01 smithi119 ceph-mon[32752]: pgmap v9310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:01 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16182.227667 secs 2024-04-26T23:05:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:01 smithi167 ceph-mon[32751]: pgmap v9310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:01 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16182.227667 secs 2024-04-26T23:05:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:01 smithi090 ceph-mon[29180]: pgmap v9310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16182.991269 secs 2024-04-26T23:05:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16182.991269 secs 2024-04-26T23:05:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16182.991269 secs 2024-04-26T23:05:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:03 smithi119 ceph-mon[32752]: pgmap v9311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:03 smithi167 ceph-mon[32751]: pgmap v9311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:03 smithi090 ceph-mon[29180]: pgmap v9311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:05 smithi119 ceph-mon[32752]: pgmap v9312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:05 smithi167 ceph-mon[32751]: pgmap v9312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:05 smithi090 ceph-mon[29180]: pgmap v9312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:06 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16187.227745 secs 2024-04-26T23:05:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:06 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16187.227745 secs 2024-04-26T23:05:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:06 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16187.227745 secs 2024-04-26T23:05:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16187.991345 secs 2024-04-26T23:05:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:07 smithi119 ceph-mon[32752]: pgmap v9313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16187.991345 secs 2024-04-26T23:05:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:07 smithi167 ceph-mon[32751]: pgmap v9313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16187.991345 secs 2024-04-26T23:05:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:07 smithi090 ceph-mon[29180]: pgmap v9313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:09 smithi119 ceph-mon[32752]: pgmap v9314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:09 smithi167 ceph-mon[32751]: pgmap v9314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:09 smithi090 ceph-mon[29180]: pgmap v9314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:11 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16192.227829 secs 2024-04-26T23:05:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:11 smithi119 ceph-mon[32752]: pgmap v9315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:11 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16192.227829 secs 2024-04-26T23:05:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:11 smithi167 ceph-mon[32751]: pgmap v9315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:11 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16192.227829 secs 2024-04-26T23:05:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:11 smithi090 ceph-mon[29180]: pgmap v9315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16192.991456 secs 2024-04-26T23:05:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16192.991456 secs 2024-04-26T23:05:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16192.991456 secs 2024-04-26T23:05:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:13 smithi119 ceph-mon[32752]: pgmap v9316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:13 smithi167 ceph-mon[32751]: pgmap v9316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:13 smithi090 ceph-mon[29180]: pgmap v9316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:14 smithi119 ceph-mon[32752]: pgmap v9317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:14 smithi167 ceph-mon[32751]: pgmap v9317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:14 smithi090 ceph-mon[29180]: pgmap v9317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:15 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16197.227891 secs 2024-04-26T23:05:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:15 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16197.227891 secs 2024-04-26T23:05:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:15 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16197.227891 secs 2024-04-26T23:05:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16197.991577 secs 2024-04-26T23:05:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:16 smithi119 ceph-mon[32752]: pgmap v9318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16197.991577 secs 2024-04-26T23:05:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:16 smithi167 ceph-mon[32751]: pgmap v9318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16197.991577 secs 2024-04-26T23:05:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:16 smithi090 ceph-mon[29180]: pgmap v9318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:19 smithi119 ceph-mon[32752]: pgmap v9319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:19.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:19 smithi090 ceph-mon[29180]: pgmap v9319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:19.663 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:19 smithi167 ceph-mon[32751]: pgmap v9319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:21 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16202.227954 secs 2024-04-26T23:05:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:21 smithi119 ceph-mon[32752]: pgmap v9320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:21 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16202.227954 secs 2024-04-26T23:05:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:21 smithi167 ceph-mon[32751]: pgmap v9320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:21 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16202.227954 secs 2024-04-26T23:05:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:21 smithi090 ceph-mon[29180]: pgmap v9320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16202.991656 secs 2024-04-26T23:05:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16202.991656 secs 2024-04-26T23:05:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16202.991656 secs 2024-04-26T23:05:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:23 smithi119 ceph-mon[32752]: pgmap v9321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:23 smithi167 ceph-mon[32751]: pgmap v9321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:23 smithi090 ceph-mon[29180]: pgmap v9321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:25 smithi119 ceph-mon[32752]: pgmap v9322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:25 smithi167 ceph-mon[32751]: pgmap v9322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:25 smithi090 ceph-mon[29180]: pgmap v9322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:26 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16207.228032 secs 2024-04-26T23:05:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:26 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16207.228032 secs 2024-04-26T23:05:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:26 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16207.228032 secs 2024-04-26T23:05:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16207.991744 secs 2024-04-26T23:05:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:27 smithi119 ceph-mon[32752]: pgmap v9323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16207.991744 secs 2024-04-26T23:05:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:27 smithi167 ceph-mon[32751]: pgmap v9323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16207.991744 secs 2024-04-26T23:05:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:27 smithi090 ceph-mon[29180]: pgmap v9323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:29 smithi119 ceph-mon[32752]: pgmap v9324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:29 smithi167 ceph-mon[32751]: pgmap v9324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:29 smithi090 ceph-mon[29180]: pgmap v9324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:31 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16212.228117 secs 2024-04-26T23:05:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:31 smithi119 ceph-mon[32752]: pgmap v9325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:31 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16212.228117 secs 2024-04-26T23:05:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:31 smithi167 ceph-mon[32751]: pgmap v9325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:31 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16212.228117 secs 2024-04-26T23:05:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:31 smithi090 ceph-mon[29180]: pgmap v9325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16212.991858 secs 2024-04-26T23:05:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16212.991858 secs 2024-04-26T23:05:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16212.991858 secs 2024-04-26T23:05:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:33 smithi119 ceph-mon[32752]: pgmap v9326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:05:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:05:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:05:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:05:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:33 smithi167 ceph-mon[32751]: pgmap v9326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:05:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:05:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:05:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:05:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:33 smithi090 ceph-mon[29180]: pgmap v9326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:05:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:05:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:05:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:05:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:34 smithi119 ceph-mon[32752]: pgmap v9327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:05:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:34 smithi119 ceph-mon[32752]: pgmap v9328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:05:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:34 smithi167 ceph-mon[32751]: pgmap v9327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:05:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:34 smithi167 ceph-mon[32751]: pgmap v9328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:05:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:34 smithi090 ceph-mon[29180]: pgmap v9327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:05:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:34 smithi090 ceph-mon[29180]: pgmap v9328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:05:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:36 smithi119 ceph-mon[32752]: pgmap v9329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:36 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16217.228215 secs 2024-04-26T23:05:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:36 smithi167 ceph-mon[32751]: pgmap v9329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:36 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16217.228215 secs 2024-04-26T23:05:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:36 smithi090 ceph-mon[29180]: pgmap v9329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:36 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16217.228215 secs 2024-04-26T23:05:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16217.991976 secs 2024-04-26T23:05:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16217.991976 secs 2024-04-26T23:05:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16217.991976 secs 2024-04-26T23:05:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:38 smithi119 ceph-mon[32752]: pgmap v9330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:38 smithi167 ceph-mon[32751]: pgmap v9330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:38 smithi090 ceph-mon[29180]: pgmap v9330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:40 smithi119 ceph-mon[32752]: pgmap v9331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:40 smithi167 ceph-mon[32751]: pgmap v9331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:40 smithi090 ceph-mon[29180]: pgmap v9331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:41 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16222.228300 secs 2024-04-26T23:05:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:41 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16222.228300 secs 2024-04-26T23:05:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:41 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16222.228300 secs 2024-04-26T23:05:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16222.992041 secs 2024-04-26T23:05:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:42 smithi119 ceph-mon[32752]: pgmap v9332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16222.992041 secs 2024-04-26T23:05:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:42 smithi167 ceph-mon[32751]: pgmap v9332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16222.992041 secs 2024-04-26T23:05:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:42 smithi090 ceph-mon[29180]: pgmap v9332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:05:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:44 smithi119 ceph-mon[32752]: pgmap v9333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:05:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:44 smithi167 ceph-mon[32751]: pgmap v9333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:05:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:44 smithi090 ceph-mon[29180]: pgmap v9333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:05:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:46 smithi119 ceph-mon[32752]: pgmap v9334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:46 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16227.228382 secs 2024-04-26T23:05:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16227.992103 secs 2024-04-26T23:05:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:46 smithi167 ceph-mon[32751]: pgmap v9334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:46 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16227.228382 secs 2024-04-26T23:05:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16227.992103 secs 2024-04-26T23:05:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:46 smithi090 ceph-mon[29180]: pgmap v9334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:46 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16227.228382 secs 2024-04-26T23:05:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16227.992103 secs 2024-04-26T23:05:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:48 smithi119 ceph-mon[32752]: pgmap v9335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:48 smithi167 ceph-mon[32751]: pgmap v9335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:48 smithi090 ceph-mon[29180]: pgmap v9335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:50 smithi119 ceph-mon[32752]: pgmap v9336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:50 smithi167 ceph-mon[32751]: pgmap v9336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:50 smithi090 ceph-mon[29180]: pgmap v9336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:51 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16232.228466 secs 2024-04-26T23:05:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16232.992214 secs 2024-04-26T23:05:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:51 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16232.228466 secs 2024-04-26T23:05:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16232.992214 secs 2024-04-26T23:05:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:51 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16232.228466 secs 2024-04-26T23:05:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16232.992214 secs 2024-04-26T23:05:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:52 smithi119 ceph-mon[32752]: pgmap v9337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:52 smithi167 ceph-mon[32751]: pgmap v9337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:52 smithi090 ceph-mon[29180]: pgmap v9337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:54 smithi119 ceph-mon[32752]: pgmap v9338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:54 smithi167 ceph-mon[32751]: pgmap v9338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:54 smithi090 ceph-mon[29180]: pgmap v9338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:56 smithi119 ceph-mon[32752]: pgmap v9339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:56 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16237.228551 secs 2024-04-26T23:05:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16237.992347 secs 2024-04-26T23:05:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:56 smithi167 ceph-mon[32751]: pgmap v9339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:56 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16237.228551 secs 2024-04-26T23:05:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16237.992347 secs 2024-04-26T23:05:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:56 smithi090 ceph-mon[29180]: pgmap v9339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:05:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:56 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16237.228551 secs 2024-04-26T23:05:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16237.992347 secs 2024-04-26T23:05:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:05:58 smithi119 ceph-mon[32752]: pgmap v9340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:05:58 smithi167 ceph-mon[32751]: pgmap v9340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:05:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:05:58 smithi090 ceph-mon[29180]: pgmap v9340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:00 smithi119 ceph-mon[32752]: pgmap v9341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:00 smithi167 ceph-mon[32751]: pgmap v9341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:00 smithi090 ceph-mon[29180]: pgmap v9341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:01 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16242.228637 secs 2024-04-26T23:06:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16242.992462 secs 2024-04-26T23:06:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:01 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16242.228637 secs 2024-04-26T23:06:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16242.992462 secs 2024-04-26T23:06:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:01 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16242.228637 secs 2024-04-26T23:06:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16242.992462 secs 2024-04-26T23:06:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:02 smithi119 ceph-mon[32752]: pgmap v9342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:02 smithi167 ceph-mon[32751]: pgmap v9342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:02 smithi090 ceph-mon[29180]: pgmap v9342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:04 smithi119 ceph-mon[32752]: pgmap v9343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:04 smithi167 ceph-mon[32751]: pgmap v9343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:04 smithi090 ceph-mon[29180]: pgmap v9343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:06 smithi167 ceph-mon[32751]: pgmap v9344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:06 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 16247.228723 secs 2024-04-26T23:06:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16247.992551 secs 2024-04-26T23:06:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:06 smithi090 ceph-mon[29180]: pgmap v9344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:06 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 16247.228723 secs 2024-04-26T23:06:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16247.992551 secs 2024-04-26T23:06:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:06 smithi119 ceph-mon[32752]: pgmap v9344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:06 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 16247.228723 secs 2024-04-26T23:06:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16247.992551 secs 2024-04-26T23:06:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:08 smithi167 ceph-mon[32751]: pgmap v9345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:08 smithi090 ceph-mon[29180]: pgmap v9345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:08 smithi119 ceph-mon[32752]: pgmap v9345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:10 smithi167 ceph-mon[32751]: pgmap v9346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:10 smithi090 ceph-mon[29180]: pgmap v9346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:10 smithi119 ceph-mon[32752]: pgmap v9346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:11 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 16252.228825 secs 2024-04-26T23:06:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:11 smithi167 ceph-mon[32751]: slow request 184.856658 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:06:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16252.992630 secs 2024-04-26T23:06:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:11 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 16252.228825 secs 2024-04-26T23:06:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:11 smithi090 ceph-mon[29180]: slow request 184.856658 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:06:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16252.992630 secs 2024-04-26T23:06:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:11 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 16252.228825 secs 2024-04-26T23:06:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:11 smithi119 ceph-mon[32752]: slow request 184.856658 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:06:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16252.992630 secs 2024-04-26T23:06:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:12 smithi090 ceph-mon[29180]: pgmap v9347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:12 smithi119 ceph-mon[32752]: pgmap v9347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:12 smithi167 ceph-mon[32751]: pgmap v9347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:14 smithi090 ceph-mon[29180]: pgmap v9348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:14 smithi119 ceph-mon[32752]: pgmap v9348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:14 smithi167 ceph-mon[32751]: pgmap v9348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:06:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:06:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:06:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:06:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:06:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:06:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:16 smithi090 ceph-mon[29180]: pgmap v9349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:16 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16257.228887 secs 2024-04-26T23:06:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16257.992743 secs 2024-04-26T23:06:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:16 smithi119 ceph-mon[32752]: pgmap v9349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:16 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16257.228887 secs 2024-04-26T23:06:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16257.992743 secs 2024-04-26T23:06:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:16 smithi167 ceph-mon[32751]: pgmap v9349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:16 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16257.228887 secs 2024-04-26T23:06:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16257.992743 secs 2024-04-26T23:06:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:18 smithi090 ceph-mon[29180]: pgmap v9350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:18 smithi119 ceph-mon[32752]: pgmap v9350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:18 smithi167 ceph-mon[32751]: pgmap v9350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:20 smithi090 ceph-mon[29180]: pgmap v9351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:20 smithi119 ceph-mon[32752]: pgmap v9351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:20 smithi167 ceph-mon[32751]: pgmap v9351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:21 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16262.228972 secs 2024-04-26T23:06:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16262.992819 secs 2024-04-26T23:06:21.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:21 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16262.228972 secs 2024-04-26T23:06:21.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16262.992819 secs 2024-04-26T23:06:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:21 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16262.228972 secs 2024-04-26T23:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16262.992819 secs 2024-04-26T23:06:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:22 smithi090 ceph-mon[29180]: pgmap v9352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:22.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:22 smithi119 ceph-mon[32752]: pgmap v9352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:22 smithi167 ceph-mon[32751]: pgmap v9352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:24 smithi090 ceph-mon[29180]: pgmap v9353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:24 smithi119 ceph-mon[32752]: pgmap v9353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:24 smithi167 ceph-mon[32751]: pgmap v9353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:26 smithi090 ceph-mon[29180]: pgmap v9354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:26 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16267.229033 secs 2024-04-26T23:06:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16267.992929 secs 2024-04-26T23:06:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:26 smithi119 ceph-mon[32752]: pgmap v9354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:26 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16267.229033 secs 2024-04-26T23:06:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16267.992929 secs 2024-04-26T23:06:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:26 smithi167 ceph-mon[32751]: pgmap v9354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:26 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16267.229033 secs 2024-04-26T23:06:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16267.992929 secs 2024-04-26T23:06:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:28 smithi090 ceph-mon[29180]: pgmap v9355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:28 smithi119 ceph-mon[32752]: pgmap v9355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:28 smithi167 ceph-mon[32751]: pgmap v9355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:30 smithi090 ceph-mon[29180]: pgmap v9356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:30 smithi119 ceph-mon[32752]: pgmap v9356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:30 smithi167 ceph-mon[32751]: pgmap v9356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:31 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16272.229091 secs 2024-04-26T23:06:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16272.993044 secs 2024-04-26T23:06:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:31 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16272.229091 secs 2024-04-26T23:06:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16272.993044 secs 2024-04-26T23:06:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:31 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16272.229091 secs 2024-04-26T23:06:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16272.993044 secs 2024-04-26T23:06:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:32 smithi090 ceph-mon[29180]: pgmap v9357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:32 smithi119 ceph-mon[32752]: pgmap v9357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:32 smithi167 ceph-mon[32751]: pgmap v9357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:06:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:06:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:06:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:34 smithi119 ceph-mon[32752]: pgmap v9358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:06:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:06:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:06:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:34 smithi167 ceph-mon[32751]: pgmap v9358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:06:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:06:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:06:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:34 smithi090 ceph-mon[29180]: pgmap v9358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:06:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:06:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:06:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:35 smithi119 ceph-mon[32752]: pgmap v9359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:06:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:35 smithi119 ceph-mon[32752]: pgmap v9360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:35 smithi167 ceph-mon[32751]: pgmap v9359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:06:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:35 smithi167 ceph-mon[32751]: pgmap v9360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:35 smithi090 ceph-mon[29180]: pgmap v9359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:06:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:35 smithi090 ceph-mon[29180]: pgmap v9360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:36 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16277.229153 secs 2024-04-26T23:06:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16277.993165 secs 2024-04-26T23:06:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:36 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16277.229153 secs 2024-04-26T23:06:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16277.993165 secs 2024-04-26T23:06:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:36 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16277.229153 secs 2024-04-26T23:06:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16277.993165 secs 2024-04-26T23:06:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:37 smithi119 ceph-mon[32752]: pgmap v9361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:37 smithi167 ceph-mon[32751]: pgmap v9361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:37 smithi090 ceph-mon[29180]: pgmap v9361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:39 smithi119 ceph-mon[32752]: pgmap v9362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:06:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:39 smithi167 ceph-mon[32751]: pgmap v9362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:06:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:39 smithi090 ceph-mon[29180]: pgmap v9362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:06:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:41 smithi119 ceph-mon[32752]: pgmap v9363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:41 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16282.229213 secs 2024-04-26T23:06:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16282.993298 secs 2024-04-26T23:06:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:41 smithi167 ceph-mon[32751]: pgmap v9363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:41 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16282.229213 secs 2024-04-26T23:06:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16282.993298 secs 2024-04-26T23:06:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:41 smithi090 ceph-mon[29180]: pgmap v9363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:41 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16282.229213 secs 2024-04-26T23:06:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16282.993298 secs 2024-04-26T23:06:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:43 smithi119 ceph-mon[32752]: pgmap v9364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:43 smithi167 ceph-mon[32751]: pgmap v9364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:43 smithi090 ceph-mon[29180]: pgmap v9364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:06:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:45 smithi119 ceph-mon[32752]: pgmap v9365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:06:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:45 smithi167 ceph-mon[32751]: pgmap v9365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:06:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:45 smithi090 ceph-mon[29180]: pgmap v9365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:06:46.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:46 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16287.229296 secs 2024-04-26T23:06:46.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16287.993438 secs 2024-04-26T23:06:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:46 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16287.229296 secs 2024-04-26T23:06:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16287.993438 secs 2024-04-26T23:06:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:46 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16287.229296 secs 2024-04-26T23:06:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16287.993438 secs 2024-04-26T23:06:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:47 smithi119 ceph-mon[32752]: pgmap v9366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:47 smithi167 ceph-mon[32751]: pgmap v9366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:47 smithi090 ceph-mon[29180]: pgmap v9366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:49 smithi119 ceph-mon[32752]: pgmap v9367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:49 smithi167 ceph-mon[32751]: pgmap v9367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:49 smithi090 ceph-mon[29180]: pgmap v9367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:51 smithi119 ceph-mon[32752]: pgmap v9368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:51 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16292.229356 secs 2024-04-26T23:06:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16292.993575 secs 2024-04-26T23:06:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:51 smithi167 ceph-mon[32751]: pgmap v9368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:51.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:51 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16292.229356 secs 2024-04-26T23:06:51.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16292.993575 secs 2024-04-26T23:06:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:51 smithi090 ceph-mon[29180]: pgmap v9368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:51 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16292.229356 secs 2024-04-26T23:06:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16292.993575 secs 2024-04-26T23:06:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:53 smithi119 ceph-mon[32752]: pgmap v9369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:53 smithi167 ceph-mon[32751]: pgmap v9369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:53 smithi090 ceph-mon[29180]: pgmap v9369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:55 smithi119 ceph-mon[32752]: pgmap v9370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:55 smithi167 ceph-mon[32751]: pgmap v9370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:55 smithi090 ceph-mon[29180]: pgmap v9370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:06:56.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:56 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16297.229441 secs 2024-04-26T23:06:56.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16297.993687 secs 2024-04-26T23:06:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:56 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16297.229441 secs 2024-04-26T23:06:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16297.993687 secs 2024-04-26T23:06:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:56 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16297.229441 secs 2024-04-26T23:06:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16297.993687 secs 2024-04-26T23:06:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:57 smithi119 ceph-mon[32752]: pgmap v9371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:57 smithi167 ceph-mon[32751]: pgmap v9371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:57 smithi090 ceph-mon[29180]: pgmap v9371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:06:59 smithi119 ceph-mon[32752]: pgmap v9372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:06:59 smithi167 ceph-mon[32751]: pgmap v9372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:06:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:06:59 smithi090 ceph-mon[29180]: pgmap v9372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:01 smithi119 ceph-mon[32752]: pgmap v9373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:01 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16302.229526 secs 2024-04-26T23:07:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16302.993764 secs 2024-04-26T23:07:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:01 smithi167 ceph-mon[32751]: pgmap v9373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:01 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16302.229526 secs 2024-04-26T23:07:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16302.993764 secs 2024-04-26T23:07:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:01 smithi090 ceph-mon[29180]: pgmap v9373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:01 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16302.229526 secs 2024-04-26T23:07:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16302.993764 secs 2024-04-26T23:07:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:03 smithi119 ceph-mon[32752]: pgmap v9374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:03 smithi167 ceph-mon[32751]: pgmap v9374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:03 smithi090 ceph-mon[29180]: pgmap v9374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:05 smithi119 ceph-mon[32752]: pgmap v9375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:05 smithi167 ceph-mon[32751]: pgmap v9375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:05 smithi090 ceph-mon[29180]: pgmap v9375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:06 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 16307.229607 secs 2024-04-26T23:07:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:06 smithi119 ceph-mon[32752]: slow request 1441.108231 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:07:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16307.993869 secs 2024-04-26T23:07:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:06 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 16307.229607 secs 2024-04-26T23:07:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:06 smithi167 ceph-mon[32751]: slow request 1441.108231 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:07:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16307.993869 secs 2024-04-26T23:07:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:06 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 16307.229607 secs 2024-04-26T23:07:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:06 smithi090 ceph-mon[29180]: slow request 1441.108231 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:07:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16307.993869 secs 2024-04-26T23:07:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:07 smithi119 ceph-mon[32752]: pgmap v9376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:07 smithi167 ceph-mon[32751]: pgmap v9376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:07 smithi090 ceph-mon[29180]: pgmap v9376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:09 smithi119 ceph-mon[32752]: pgmap v9377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:09 smithi167 ceph-mon[32751]: pgmap v9377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:09 smithi090 ceph-mon[29180]: pgmap v9377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:11 smithi119 ceph-mon[32752]: pgmap v9378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:11 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16312.229687 secs 2024-04-26T23:07:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16312.993946 secs 2024-04-26T23:07:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:11 smithi167 ceph-mon[32751]: pgmap v9378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:11 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16312.229687 secs 2024-04-26T23:07:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16312.993946 secs 2024-04-26T23:07:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:11 smithi090 ceph-mon[29180]: pgmap v9378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:11 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16312.229687 secs 2024-04-26T23:07:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16312.993946 secs 2024-04-26T23:07:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:13 smithi119 ceph-mon[32752]: pgmap v9379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:13 smithi167 ceph-mon[32751]: pgmap v9379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:13 smithi090 ceph-mon[29180]: pgmap v9379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:15 smithi119 ceph-mon[32752]: pgmap v9380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:15 smithi167 ceph-mon[32751]: pgmap v9380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:15 smithi090 ceph-mon[29180]: pgmap v9380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:16 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16317.229770 secs 2024-04-26T23:07:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16317.994092 secs 2024-04-26T23:07:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:16 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16317.229770 secs 2024-04-26T23:07:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16317.994092 secs 2024-04-26T23:07:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:16 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16317.229770 secs 2024-04-26T23:07:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16317.994092 secs 2024-04-26T23:07:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:17 smithi119 ceph-mon[32752]: pgmap v9381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:17 smithi167 ceph-mon[32751]: pgmap v9381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:17.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:17 smithi090 ceph-mon[29180]: pgmap v9381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:19 smithi119 ceph-mon[32752]: pgmap v9382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:19 smithi167 ceph-mon[32751]: pgmap v9382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:19 smithi090 ceph-mon[29180]: pgmap v9382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:21 smithi119 ceph-mon[32752]: pgmap v9383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:21 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16322.229856 secs 2024-04-26T23:07:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16322.994191 secs 2024-04-26T23:07:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:21 smithi167 ceph-mon[32751]: pgmap v9383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:21 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16322.229856 secs 2024-04-26T23:07:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16322.994191 secs 2024-04-26T23:07:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:21 smithi090 ceph-mon[29180]: pgmap v9383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:21 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16322.229856 secs 2024-04-26T23:07:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16322.994191 secs 2024-04-26T23:07:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:23 smithi119 ceph-mon[32752]: pgmap v9384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:23 smithi167 ceph-mon[32751]: pgmap v9384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:23 smithi090 ceph-mon[29180]: pgmap v9384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:25 smithi119 ceph-mon[32752]: pgmap v9385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:25 smithi167 ceph-mon[32751]: pgmap v9385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:25 smithi090 ceph-mon[29180]: pgmap v9385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:26.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:26 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16327.229935 secs 2024-04-26T23:07:26.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16327.994334 secs 2024-04-26T23:07:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:26 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16327.229935 secs 2024-04-26T23:07:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16327.994334 secs 2024-04-26T23:07:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:26 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16327.229935 secs 2024-04-26T23:07:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16327.994334 secs 2024-04-26T23:07:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:27 smithi119 ceph-mon[32752]: pgmap v9386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:27 smithi167 ceph-mon[32751]: pgmap v9386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:27 smithi090 ceph-mon[29180]: pgmap v9386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:29 smithi119 ceph-mon[32752]: pgmap v9387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:29 smithi167 ceph-mon[32751]: pgmap v9387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:29 smithi090 ceph-mon[29180]: pgmap v9387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:31 smithi119 ceph-mon[32752]: pgmap v9388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:31 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16332.230020 secs 2024-04-26T23:07:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16332.994481 secs 2024-04-26T23:07:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:31 smithi167 ceph-mon[32751]: pgmap v9388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:31 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16332.230020 secs 2024-04-26T23:07:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16332.994481 secs 2024-04-26T23:07:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:31 smithi090 ceph-mon[29180]: pgmap v9388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:31 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16332.230020 secs 2024-04-26T23:07:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16332.994481 secs 2024-04-26T23:07:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:33 smithi119 ceph-mon[32752]: pgmap v9389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:33 smithi167 ceph-mon[32751]: pgmap v9389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:33 smithi090 ceph-mon[29180]: pgmap v9389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:07:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:07:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:07:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:07:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:07:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:07:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:07:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:07:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:07:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:07:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:07:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:07:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:35 smithi119 ceph-mon[32752]: pgmap v9390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:35 smithi119 ceph-mon[32752]: pgmap v9391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:07:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:35 smithi119 ceph-mon[32752]: pgmap v9392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:35 smithi167 ceph-mon[32751]: pgmap v9390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:35 smithi167 ceph-mon[32751]: pgmap v9391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:07:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:35 smithi167 ceph-mon[32751]: pgmap v9392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:35 smithi090 ceph-mon[29180]: pgmap v9390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:35 smithi090 ceph-mon[29180]: pgmap v9391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:07:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:35 smithi090 ceph-mon[29180]: pgmap v9392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:36 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16337.230107 secs 2024-04-26T23:07:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16337.994618 secs 2024-04-26T23:07:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:36 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16337.230107 secs 2024-04-26T23:07:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16337.994618 secs 2024-04-26T23:07:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:36 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16337.230107 secs 2024-04-26T23:07:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16337.994618 secs 2024-04-26T23:07:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:37 smithi119 ceph-mon[32752]: pgmap v9393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:37 smithi167 ceph-mon[32751]: pgmap v9393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:37 smithi090 ceph-mon[29180]: pgmap v9393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:39 smithi119 ceph-mon[32752]: pgmap v9394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:39 smithi167 ceph-mon[32751]: pgmap v9394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:39 smithi090 ceph-mon[29180]: pgmap v9394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:41 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16342.230187 secs 2024-04-26T23:07:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:41 smithi119 ceph-mon[32752]: pgmap v9395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16342.994731 secs 2024-04-26T23:07:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:41 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16342.230187 secs 2024-04-26T23:07:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:41 smithi167 ceph-mon[32751]: pgmap v9395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16342.994731 secs 2024-04-26T23:07:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:41 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16342.230187 secs 2024-04-26T23:07:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:41 smithi090 ceph-mon[29180]: pgmap v9395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:07:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16342.994731 secs 2024-04-26T23:07:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:43 smithi119 ceph-mon[32752]: pgmap v9396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:07:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:43 smithi167 ceph-mon[32751]: pgmap v9396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:07:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:43 smithi090 ceph-mon[29180]: pgmap v9396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:07:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:45 smithi119 ceph-mon[32752]: pgmap v9397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T23:07:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:45 smithi167 ceph-mon[32751]: pgmap v9397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T23:07:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:45 smithi090 ceph-mon[29180]: pgmap v9397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T23:07:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:46 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16347.230269 secs 2024-04-26T23:07:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16347.994826 secs 2024-04-26T23:07:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:46 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16347.230269 secs 2024-04-26T23:07:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16347.994826 secs 2024-04-26T23:07:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:46 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16347.230269 secs 2024-04-26T23:07:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16347.994826 secs 2024-04-26T23:07:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:47 smithi119 ceph-mon[32752]: pgmap v9398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:47 smithi167 ceph-mon[32751]: pgmap v9398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:47 smithi090 ceph-mon[29180]: pgmap v9398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:49 smithi119 ceph-mon[32752]: pgmap v9399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:49 smithi167 ceph-mon[32751]: pgmap v9399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:49 smithi090 ceph-mon[29180]: pgmap v9399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:51 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16352.230328 secs 2024-04-26T23:07:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:51 smithi119 ceph-mon[32752]: pgmap v9400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16352.994943 secs 2024-04-26T23:07:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:51 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16352.230328 secs 2024-04-26T23:07:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:51 smithi167 ceph-mon[32751]: pgmap v9400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16352.994943 secs 2024-04-26T23:07:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:51 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16352.230328 secs 2024-04-26T23:07:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:51 smithi090 ceph-mon[29180]: pgmap v9400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16352.994943 secs 2024-04-26T23:07:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:53 smithi119 ceph-mon[32752]: pgmap v9401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:53 smithi167 ceph-mon[32751]: pgmap v9401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:53 smithi090 ceph-mon[29180]: pgmap v9401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:07:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:55 smithi119 ceph-mon[32752]: pgmap v9402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:55 smithi167 ceph-mon[32751]: pgmap v9402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:55 smithi090 ceph-mon[29180]: pgmap v9402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:56 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16357.230386 secs 2024-04-26T23:07:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16357.995096 secs 2024-04-26T23:07:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:56 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16357.230386 secs 2024-04-26T23:07:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16357.995096 secs 2024-04-26T23:07:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:56 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16357.230386 secs 2024-04-26T23:07:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16357.995096 secs 2024-04-26T23:07:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:57 smithi119 ceph-mon[32752]: pgmap v9403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:57 smithi167 ceph-mon[32751]: pgmap v9403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:57 smithi090 ceph-mon[29180]: pgmap v9403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:07:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:07:59 smithi119 ceph-mon[32752]: pgmap v9404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:07:59 smithi167 ceph-mon[32751]: pgmap v9404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:07:59 smithi090 ceph-mon[29180]: pgmap v9404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:01 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16362.230469 secs 2024-04-26T23:08:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:01 smithi119 ceph-mon[32752]: pgmap v9405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16362.995239 secs 2024-04-26T23:08:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:01 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16362.230469 secs 2024-04-26T23:08:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:01 smithi167 ceph-mon[32751]: pgmap v9405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16362.995239 secs 2024-04-26T23:08:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:01 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16362.230469 secs 2024-04-26T23:08:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:01 smithi090 ceph-mon[29180]: pgmap v9405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16362.995239 secs 2024-04-26T23:08:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:03 smithi167 ceph-mon[32751]: pgmap v9406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:03 smithi090 ceph-mon[29180]: pgmap v9406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:03 smithi119 ceph-mon[32752]: pgmap v9406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:05.557 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:08:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:08:05.238+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:08:05.558 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:08:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:08:05.238+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:08:05.558 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:08:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:08:05.238+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:08:05.558 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:08:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:08:05.238+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:08:05.558 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:08:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:08:05.238+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:08:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:05 smithi167 ceph-mon[32751]: pgmap v9407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:05 smithi090 ceph-mon[29180]: pgmap v9407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:05 smithi119 ceph-mon[32752]: pgmap v9407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:06 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 16367.230552 secs 2024-04-26T23:08:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16367.995396 secs 2024-04-26T23:08:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:06 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 16367.230552 secs 2024-04-26T23:08:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16367.995396 secs 2024-04-26T23:08:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:06 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 16367.230552 secs 2024-04-26T23:08:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16367.995396 secs 2024-04-26T23:08:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:07 smithi167 ceph-mon[32751]: pgmap v9408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:07 smithi090 ceph-mon[29180]: pgmap v9408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:07 smithi119 ceph-mon[32752]: pgmap v9408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:09 smithi090 ceph-mon[29180]: pgmap v9409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:09 smithi119 ceph-mon[32752]: pgmap v9409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:09 smithi167 ceph-mon[32751]: pgmap v9409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16372.230635 secs 2024-04-26T23:08:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:11 smithi090 ceph-mon[29180]: pgmap v9410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16372.995490 secs 2024-04-26T23:08:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16372.230635 secs 2024-04-26T23:08:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:11 smithi119 ceph-mon[32752]: pgmap v9410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16372.995490 secs 2024-04-26T23:08:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16372.230635 secs 2024-04-26T23:08:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:11 smithi167 ceph-mon[32751]: pgmap v9410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16372.995490 secs 2024-04-26T23:08:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:13 smithi090 ceph-mon[29180]: pgmap v9411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:13 smithi119 ceph-mon[32752]: pgmap v9411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:13 smithi167 ceph-mon[32751]: pgmap v9411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:15 smithi090 ceph-mon[29180]: pgmap v9412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:08:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:08:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:15 smithi119 ceph-mon[32752]: pgmap v9412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:08:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:08:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:15 smithi167 ceph-mon[32751]: pgmap v9412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:08:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16377.230715 secs 2024-04-26T23:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16377.995609 secs 2024-04-26T23:08:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16377.230715 secs 2024-04-26T23:08:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16377.995609 secs 2024-04-26T23:08:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16377.230715 secs 2024-04-26T23:08:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16377.995609 secs 2024-04-26T23:08:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:17 smithi090 ceph-mon[29180]: pgmap v9413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:17 smithi119 ceph-mon[32752]: pgmap v9413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:17 smithi167 ceph-mon[32751]: pgmap v9413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:08:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:19 smithi090 ceph-mon[29180]: pgmap v9414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:20.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:19 smithi119 ceph-mon[32752]: pgmap v9414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:19 smithi167 ceph-mon[32751]: pgmap v9414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:08:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16382.230797 secs 2024-04-26T23:08:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:21 smithi167 ceph-mon[32751]: pgmap v9415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16382.995752 secs 2024-04-26T23:08:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16382.230797 secs 2024-04-26T23:08:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:21 smithi090 ceph-mon[29180]: pgmap v9415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16382.995752 secs 2024-04-26T23:08:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16382.230797 secs 2024-04-26T23:08:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:21 smithi119 ceph-mon[32752]: pgmap v9415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16382.995752 secs 2024-04-26T23:08:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:23 smithi090 ceph-mon[29180]: pgmap v9416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:23 smithi119 ceph-mon[32752]: pgmap v9416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:23 smithi167 ceph-mon[32751]: pgmap v9416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:25 smithi090 ceph-mon[29180]: pgmap v9417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:25 smithi119 ceph-mon[32752]: pgmap v9417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:25 smithi167 ceph-mon[32751]: pgmap v9417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16387.230876 secs 2024-04-26T23:08:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16387.995870 secs 2024-04-26T23:08:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16387.230876 secs 2024-04-26T23:08:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16387.995870 secs 2024-04-26T23:08:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16387.230876 secs 2024-04-26T23:08:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16387.995870 secs 2024-04-26T23:08:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:27 smithi090 ceph-mon[29180]: pgmap v9418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:27 smithi119 ceph-mon[32752]: pgmap v9418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:27 smithi167 ceph-mon[32751]: pgmap v9418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:29 smithi090 ceph-mon[29180]: pgmap v9419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:29 smithi119 ceph-mon[32752]: pgmap v9419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:29 smithi167 ceph-mon[32751]: pgmap v9419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16392.230938 secs 2024-04-26T23:08:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:31 smithi090 ceph-mon[29180]: pgmap v9420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16392.995980 secs 2024-04-26T23:08:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16392.230938 secs 2024-04-26T23:08:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:31 smithi119 ceph-mon[32752]: pgmap v9420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:32.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16392.995980 secs 2024-04-26T23:08:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16392.230938 secs 2024-04-26T23:08:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:31 smithi167 ceph-mon[32751]: pgmap v9420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16392.995980 secs 2024-04-26T23:08:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:33 smithi090 ceph-mon[29180]: pgmap v9421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:33 smithi119 ceph-mon[32752]: pgmap v9421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:33 smithi167 ceph-mon[32751]: pgmap v9421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:35.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:08:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:08:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:08:35.896 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:35 smithi119 ceph-mon[32752]: pgmap v9422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:35.897 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:35.897 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:35.897 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:35.897 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:35 smithi090 ceph-mon[29180]: pgmap v9422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:36.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.006 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:35 smithi167 ceph-mon[32751]: pgmap v9422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:36 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 16397.231031 secs 2024-04-26T23:08:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:36 smithi119 ceph-mon[32752]: slow request 2880.565165 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:08:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16397.996083 secs 2024-04-26T23:08:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:36 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 16397.231031 secs 2024-04-26T23:08:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:36 smithi167 ceph-mon[32751]: slow request 2880.565165 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:08:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16397.996083 secs 2024-04-26T23:08:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:36 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 16397.231031 secs 2024-04-26T23:08:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:36 smithi090 ceph-mon[29180]: slow request 2880.565165 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:08:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16397.996083 secs 2024-04-26T23:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: pgmap v9423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: pgmap v9423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: pgmap v9423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:08:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:08:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:08:39.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:08:39.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:08:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:08:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:08:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:08:39.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:08:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:08:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:08:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:08:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:08:39.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:08:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:39 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:08:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:39 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:08:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:39 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:08:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:39 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:08:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:39 smithi119 ceph-mon[32752]: pgmap v9424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-26T23:08:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:39 smithi119 ceph-mon[32752]: pgmap v9425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 537 B/s rd, 1 op/s 2024-04-26T23:08:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:39 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:39 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:39 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:39 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:39 smithi167 ceph-mon[32751]: pgmap v9424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-26T23:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:39 smithi167 ceph-mon[32751]: pgmap v9425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 537 B/s rd, 1 op/s 2024-04-26T23:08:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:39 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:08:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:39 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:08:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:39 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:08:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:39 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:08:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:39 smithi090 ceph-mon[29180]: pgmap v9424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-26T23:08:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:39 smithi090 ceph-mon[29180]: pgmap v9425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 537 B/s rd, 1 op/s 2024-04-26T23:08:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16402.231105 secs 2024-04-26T23:08:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:41 smithi119 ceph-mon[32752]: pgmap v9426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16402.996201 secs 2024-04-26T23:08:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16402.231105 secs 2024-04-26T23:08:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:41 smithi167 ceph-mon[32751]: pgmap v9426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16402.996201 secs 2024-04-26T23:08:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16402.231105 secs 2024-04-26T23:08:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:41 smithi090 ceph-mon[29180]: pgmap v9426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16402.996201 secs 2024-04-26T23:08:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:43 smithi119 ceph-mon[32752]: pgmap v9427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:43 smithi167 ceph-mon[32751]: pgmap v9427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:43 smithi090 ceph-mon[29180]: pgmap v9427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:45 smithi119 ceph-mon[32752]: pgmap v9428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:45 smithi167 ceph-mon[32751]: pgmap v9428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:45 smithi090 ceph-mon[29180]: pgmap v9428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16407.231188 secs 2024-04-26T23:08:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16407.996318 secs 2024-04-26T23:08:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16407.231188 secs 2024-04-26T23:08:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16407.996318 secs 2024-04-26T23:08:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16407.231188 secs 2024-04-26T23:08:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16407.996318 secs 2024-04-26T23:08:48.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:47 smithi119 ceph-mon[32752]: pgmap v9429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:47 smithi167 ceph-mon[32751]: pgmap v9429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:47 smithi090 ceph-mon[29180]: pgmap v9429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:08:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:49 smithi119 ceph-mon[32752]: pgmap v9430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:08:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:49 smithi167 ceph-mon[32751]: pgmap v9430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:08:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:49 smithi090 ceph-mon[29180]: pgmap v9430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:08:51.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16412.231270 secs 2024-04-26T23:08:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:51 smithi090 ceph-mon[29180]: pgmap v9431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16412.996442 secs 2024-04-26T23:08:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16412.231270 secs 2024-04-26T23:08:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:51 smithi119 ceph-mon[32752]: pgmap v9431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16412.996442 secs 2024-04-26T23:08:52.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16412.231270 secs 2024-04-26T23:08:52.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:51 smithi167 ceph-mon[32751]: pgmap v9431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:52.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16412.996442 secs 2024-04-26T23:08:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:53 smithi119 ceph-mon[32752]: pgmap v9432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:53 smithi167 ceph-mon[32751]: pgmap v9432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:53 smithi090 ceph-mon[29180]: pgmap v9432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:55 smithi119 ceph-mon[32752]: pgmap v9433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:55 smithi167 ceph-mon[32751]: pgmap v9433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:55 smithi090 ceph-mon[29180]: pgmap v9433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:08:57.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16417.231335 secs 2024-04-26T23:08:57.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16417.996565 secs 2024-04-26T23:08:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16417.231335 secs 2024-04-26T23:08:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16417.996565 secs 2024-04-26T23:08:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16417.231335 secs 2024-04-26T23:08:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16417.996565 secs 2024-04-26T23:08:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:57 smithi119 ceph-mon[32752]: pgmap v9434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:57 smithi167 ceph-mon[32751]: pgmap v9434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:08:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:57 smithi090 ceph-mon[29180]: pgmap v9434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:00.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:08:59 smithi119 ceph-mon[32752]: pgmap v9435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:08:59 smithi167 ceph-mon[32751]: pgmap v9435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:08:59 smithi090 ceph-mon[29180]: pgmap v9435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:00 smithi119 ceph-mon[32752]: pgmap v9436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:00 smithi167 ceph-mon[32751]: pgmap v9436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:00 smithi090 ceph-mon[29180]: pgmap v9436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16422.231393 secs 2024-04-26T23:09:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16422.996711 secs 2024-04-26T23:09:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16422.231393 secs 2024-04-26T23:09:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16422.996711 secs 2024-04-26T23:09:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16422.231393 secs 2024-04-26T23:09:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16422.996711 secs 2024-04-26T23:09:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:02 smithi119 ceph-mon[32752]: pgmap v9437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:02 smithi167 ceph-mon[32751]: pgmap v9437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:02 smithi090 ceph-mon[29180]: pgmap v9437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:05 smithi090 ceph-mon[29180]: pgmap v9438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:05 smithi119 ceph-mon[32752]: pgmap v9438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:05 smithi167 ceph-mon[32751]: pgmap v9438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16427.231453 secs 2024-04-26T23:09:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16427.996812 secs 2024-04-26T23:09:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16427.231453 secs 2024-04-26T23:09:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16427.996812 secs 2024-04-26T23:09:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16427.231453 secs 2024-04-26T23:09:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16427.996812 secs 2024-04-26T23:09:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:07 smithi090 ceph-mon[29180]: pgmap v9439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:07 smithi119 ceph-mon[32752]: pgmap v9439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:07 smithi167 ceph-mon[32751]: pgmap v9439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:09 smithi090 ceph-mon[29180]: pgmap v9440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:09 smithi119 ceph-mon[32752]: pgmap v9440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:09 smithi167 ceph-mon[32751]: pgmap v9440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:11 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 16432.231535 secs 2024-04-26T23:09:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:11 smithi090 ceph-mon[29180]: slow request 364.859368 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:09:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:11 smithi090 ceph-mon[29180]: pgmap v9441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16432.996951 secs 2024-04-26T23:09:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:11 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 16432.231535 secs 2024-04-26T23:09:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:11 smithi119 ceph-mon[32752]: slow request 364.859368 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:09:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:11 smithi119 ceph-mon[32752]: pgmap v9441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16432.996951 secs 2024-04-26T23:09:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:11 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 16432.231535 secs 2024-04-26T23:09:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:11 smithi167 ceph-mon[32751]: slow request 364.859368 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:09:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:11 smithi167 ceph-mon[32751]: pgmap v9441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16432.996951 secs 2024-04-26T23:09:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:13 smithi090 ceph-mon[29180]: pgmap v9442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:13 smithi119 ceph-mon[32752]: pgmap v9442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:13 smithi167 ceph-mon[32751]: pgmap v9442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:09:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:15 smithi090 ceph-mon[29180]: pgmap v9443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:15 smithi119 ceph-mon[32752]: pgmap v9443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:15 smithi167 ceph-mon[32751]: pgmap v9443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:09:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16437.231617 secs 2024-04-26T23:09:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16437.997086 secs 2024-04-26T23:09:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16437.231617 secs 2024-04-26T23:09:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16437.997086 secs 2024-04-26T23:09:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16437.231617 secs 2024-04-26T23:09:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16437.997086 secs 2024-04-26T23:09:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:17 smithi090 ceph-mon[29180]: pgmap v9444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:17 smithi119 ceph-mon[32752]: pgmap v9444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:17 smithi167 ceph-mon[32751]: pgmap v9444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:19 smithi090 ceph-mon[29180]: pgmap v9445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:19 smithi119 ceph-mon[32752]: pgmap v9445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:19 smithi167 ceph-mon[32751]: pgmap v9445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:21.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16442.231704 secs 2024-04-26T23:09:21.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:21 smithi167 ceph-mon[32751]: pgmap v9446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:21.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16442.997152 secs 2024-04-26T23:09:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16442.231704 secs 2024-04-26T23:09:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:21 smithi090 ceph-mon[29180]: pgmap v9446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16442.997152 secs 2024-04-26T23:09:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16442.231704 secs 2024-04-26T23:09:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:21 smithi119 ceph-mon[32752]: pgmap v9446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16442.997152 secs 2024-04-26T23:09:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:23 smithi090 ceph-mon[29180]: pgmap v9447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:23 smithi119 ceph-mon[32752]: pgmap v9447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:23 smithi167 ceph-mon[32751]: pgmap v9447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:25 smithi090 ceph-mon[29180]: pgmap v9448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:25 smithi119 ceph-mon[32752]: pgmap v9448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:25 smithi167 ceph-mon[32751]: pgmap v9448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16447.231769 secs 2024-04-26T23:09:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16447.997280 secs 2024-04-26T23:09:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16447.231769 secs 2024-04-26T23:09:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16447.997280 secs 2024-04-26T23:09:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16447.231769 secs 2024-04-26T23:09:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16447.997280 secs 2024-04-26T23:09:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:27 smithi119 ceph-mon[32752]: pgmap v9449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:27 smithi167 ceph-mon[32751]: pgmap v9449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:27 smithi090 ceph-mon[29180]: pgmap v9449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:29 smithi119 ceph-mon[32752]: pgmap v9450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:29 smithi167 ceph-mon[32751]: pgmap v9450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:29 smithi090 ceph-mon[29180]: pgmap v9450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16452.231839 secs 2024-04-26T23:09:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:31 smithi119 ceph-mon[32752]: pgmap v9451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16452.997414 secs 2024-04-26T23:09:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16452.231839 secs 2024-04-26T23:09:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:31 smithi167 ceph-mon[32751]: pgmap v9451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16452.997414 secs 2024-04-26T23:09:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16452.231839 secs 2024-04-26T23:09:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:31 smithi090 ceph-mon[29180]: pgmap v9451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16452.997414 secs 2024-04-26T23:09:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:33 smithi119 ceph-mon[32752]: pgmap v9452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:33 smithi167 ceph-mon[32751]: pgmap v9452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:33 smithi090 ceph-mon[29180]: pgmap v9452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:35 smithi119 ceph-mon[32752]: pgmap v9453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:35 smithi167 ceph-mon[32751]: pgmap v9453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:35 smithi090 ceph-mon[29180]: pgmap v9453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16457.231918 secs 2024-04-26T23:09:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16457.997448 secs 2024-04-26T23:09:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16457.231918 secs 2024-04-26T23:09:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16457.997448 secs 2024-04-26T23:09:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16457.231918 secs 2024-04-26T23:09:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16457.997448 secs 2024-04-26T23:09:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:37 smithi119 ceph-mon[32752]: pgmap v9454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:37 smithi167 ceph-mon[32751]: pgmap v9454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:37 smithi090 ceph-mon[29180]: pgmap v9454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:09:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:09:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:09:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:39 smithi119 ceph-mon[32752]: pgmap v9455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:09:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:09:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:39 smithi167 ceph-mon[32751]: pgmap v9455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:09:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:09:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:39 smithi090 ceph-mon[29180]: pgmap v9455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:09:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:09:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:09:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:40 smithi119 ceph-mon[32752]: pgmap v9456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:09:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:40 smithi119 ceph-mon[32752]: pgmap v9457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:40 smithi167 ceph-mon[32751]: pgmap v9456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:09:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:40 smithi167 ceph-mon[32751]: pgmap v9457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:40 smithi090 ceph-mon[29180]: pgmap v9456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:09:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:40 smithi090 ceph-mon[29180]: pgmap v9457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16462.232004 secs 2024-04-26T23:09:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16462.997574 secs 2024-04-26T23:09:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16462.232004 secs 2024-04-26T23:09:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16462.997574 secs 2024-04-26T23:09:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16462.232004 secs 2024-04-26T23:09:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16462.997574 secs 2024-04-26T23:09:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:42 smithi119 ceph-mon[32752]: pgmap v9458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:42 smithi167 ceph-mon[32751]: pgmap v9458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:42 smithi090 ceph-mon[29180]: pgmap v9458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:43 smithi119 ceph-mon[32752]: pgmap v9459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:43 smithi167 ceph-mon[32751]: pgmap v9459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:43 smithi090 ceph-mon[29180]: pgmap v9459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16467.232088 secs 2024-04-26T23:09:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:46 smithi090 ceph-mon[29180]: pgmap v9460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16467.997639 secs 2024-04-26T23:09:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16467.232088 secs 2024-04-26T23:09:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:46 smithi119 ceph-mon[32752]: pgmap v9460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16467.997639 secs 2024-04-26T23:09:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16467.232088 secs 2024-04-26T23:09:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:46 smithi167 ceph-mon[32751]: pgmap v9460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16467.997639 secs 2024-04-26T23:09:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:48 smithi119 ceph-mon[32752]: pgmap v9461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:48 smithi167 ceph-mon[32751]: pgmap v9461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:48 smithi090 ceph-mon[29180]: pgmap v9461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:09:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:50 smithi119 ceph-mon[32752]: pgmap v9462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:09:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:50 smithi167 ceph-mon[32751]: pgmap v9462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:09:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:50 smithi090 ceph-mon[29180]: pgmap v9462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:09:51.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16472.232173 secs 2024-04-26T23:09:51.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16472.997748 secs 2024-04-26T23:09:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16472.232173 secs 2024-04-26T23:09:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16472.997748 secs 2024-04-26T23:09:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16472.232173 secs 2024-04-26T23:09:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16472.997748 secs 2024-04-26T23:09:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:52 smithi119 ceph-mon[32752]: pgmap v9463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:52 smithi167 ceph-mon[32751]: pgmap v9463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:52 smithi090 ceph-mon[29180]: pgmap v9463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:54 smithi119 ceph-mon[32752]: pgmap v9464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:54 smithi167 ceph-mon[32751]: pgmap v9464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:54 smithi090 ceph-mon[29180]: pgmap v9464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:09:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16477.232278 secs 2024-04-26T23:09:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:56 smithi119 ceph-mon[32752]: pgmap v9465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16477.997840 secs 2024-04-26T23:09:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16477.232278 secs 2024-04-26T23:09:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:56 smithi167 ceph-mon[32751]: pgmap v9465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16477.997840 secs 2024-04-26T23:09:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16477.232278 secs 2024-04-26T23:09:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:56 smithi090 ceph-mon[29180]: pgmap v9465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16477.997840 secs 2024-04-26T23:09:58.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:09:58 smithi090 ceph-mon[29180]: pgmap v9466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:09:58 smithi119 ceph-mon[32752]: pgmap v9466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:09:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:09:58 smithi167 ceph-mon[32751]: pgmap v9466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:00 smithi119 ceph-mon[32752]: pgmap v9467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:10:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:00 smithi167 ceph-mon[32751]: pgmap v9467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:10:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:00 smithi090 ceph-mon[29180]: pgmap v9467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:10:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16482.232362 secs 2024-04-26T23:10:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16482.997955 secs 2024-04-26T23:10:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16482.232362 secs 2024-04-26T23:10:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16482.997955 secs 2024-04-26T23:10:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16482.232362 secs 2024-04-26T23:10:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16482.997955 secs 2024-04-26T23:10:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:02 smithi119 ceph-mon[32752]: pgmap v9468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:02 smithi167 ceph-mon[32751]: pgmap v9468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:02 smithi090 ceph-mon[29180]: pgmap v9468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:04 smithi119 ceph-mon[32752]: pgmap v9469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:04 smithi167 ceph-mon[32751]: pgmap v9469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:04 smithi090 ceph-mon[29180]: pgmap v9469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16487.232449 secs 2024-04-26T23:10:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:06 smithi119 ceph-mon[32752]: pgmap v9470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16487.998077 secs 2024-04-26T23:10:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16487.232449 secs 2024-04-26T23:10:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:06 smithi167 ceph-mon[32751]: pgmap v9470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16487.998077 secs 2024-04-26T23:10:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16487.232449 secs 2024-04-26T23:10:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:06 smithi090 ceph-mon[29180]: pgmap v9470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16487.998077 secs 2024-04-26T23:10:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:08 smithi119 ceph-mon[32752]: pgmap v9471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:08 smithi167 ceph-mon[32751]: pgmap v9471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:08.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:08 smithi090 ceph-mon[29180]: pgmap v9471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:10 smithi119 ceph-mon[32752]: pgmap v9472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:10 smithi167 ceph-mon[32751]: pgmap v9472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:10 smithi090 ceph-mon[29180]: pgmap v9472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16492.232531 secs 2024-04-26T23:10:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16492.998199 secs 2024-04-26T23:10:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16492.232531 secs 2024-04-26T23:10:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16492.998199 secs 2024-04-26T23:10:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16492.232531 secs 2024-04-26T23:10:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16492.998199 secs 2024-04-26T23:10:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:12 smithi119 ceph-mon[32752]: pgmap v9473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:12 smithi167 ceph-mon[32751]: pgmap v9473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:12 smithi090 ceph-mon[29180]: pgmap v9473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:14 smithi119 ceph-mon[32752]: pgmap v9474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:14 smithi167 ceph-mon[32751]: pgmap v9474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:14 smithi090 ceph-mon[29180]: pgmap v9474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16497.232616 secs 2024-04-26T23:10:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:16 smithi119 ceph-mon[32752]: pgmap v9475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16497.998263 secs 2024-04-26T23:10:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16497.232616 secs 2024-04-26T23:10:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:16 smithi167 ceph-mon[32751]: pgmap v9475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16497.998263 secs 2024-04-26T23:10:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16497.232616 secs 2024-04-26T23:10:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:16 smithi090 ceph-mon[29180]: pgmap v9475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16497.998263 secs 2024-04-26T23:10:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:18 smithi119 ceph-mon[32752]: pgmap v9476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:18 smithi167 ceph-mon[32751]: pgmap v9476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:18 smithi090 ceph-mon[29180]: pgmap v9476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:20 smithi119 ceph-mon[32752]: pgmap v9477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:20 smithi167 ceph-mon[32751]: pgmap v9477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:20 smithi090 ceph-mon[29180]: pgmap v9477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:21.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16502.232703 secs 2024-04-26T23:10:21.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16502.998368 secs 2024-04-26T23:10:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16502.232703 secs 2024-04-26T23:10:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16502.998368 secs 2024-04-26T23:10:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16502.232703 secs 2024-04-26T23:10:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16502.998368 secs 2024-04-26T23:10:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:22 smithi119 ceph-mon[32752]: pgmap v9478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:22 smithi167 ceph-mon[32751]: pgmap v9478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:22 smithi090 ceph-mon[29180]: pgmap v9478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:24.741 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:24 smithi090 ceph-mon[29180]: pgmap v9479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:24 smithi119 ceph-mon[32752]: pgmap v9479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:24 smithi167 ceph-mon[32751]: pgmap v9479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16507.232780 secs 2024-04-26T23:10:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:26 smithi119 ceph-mon[32752]: pgmap v9480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16507.998473 secs 2024-04-26T23:10:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16507.232780 secs 2024-04-26T23:10:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:26 smithi167 ceph-mon[32751]: pgmap v9480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16507.998473 secs 2024-04-26T23:10:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16507.232780 secs 2024-04-26T23:10:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:26 smithi090 ceph-mon[29180]: pgmap v9480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16507.998473 secs 2024-04-26T23:10:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:28 smithi119 ceph-mon[32752]: pgmap v9481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:28 smithi167 ceph-mon[32751]: pgmap v9481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:28 smithi090 ceph-mon[29180]: pgmap v9481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:30 smithi119 ceph-mon[32752]: pgmap v9482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:30 smithi167 ceph-mon[32751]: pgmap v9482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:30 smithi090 ceph-mon[29180]: pgmap v9482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16512.232861 secs 2024-04-26T23:10:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16512.998595 secs 2024-04-26T23:10:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16512.232861 secs 2024-04-26T23:10:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16512.998595 secs 2024-04-26T23:10:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16512.232861 secs 2024-04-26T23:10:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16512.998595 secs 2024-04-26T23:10:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:32 smithi119 ceph-mon[32752]: pgmap v9483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:32 smithi167 ceph-mon[32751]: pgmap v9483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:32 smithi090 ceph-mon[29180]: pgmap v9483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:34 smithi119 ceph-mon[32752]: pgmap v9484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:34 smithi167 ceph-mon[32751]: pgmap v9484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:34 smithi090 ceph-mon[29180]: pgmap v9484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16517.232962 secs 2024-04-26T23:10:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:36 smithi119 ceph-mon[32752]: pgmap v9485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16517.998716 secs 2024-04-26T23:10:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16517.232962 secs 2024-04-26T23:10:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:36 smithi167 ceph-mon[32751]: pgmap v9485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16517.998716 secs 2024-04-26T23:10:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16517.232962 secs 2024-04-26T23:10:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:36 smithi090 ceph-mon[29180]: pgmap v9485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16517.998716 secs 2024-04-26T23:10:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:38 smithi119 ceph-mon[32752]: pgmap v9486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:38 smithi167 ceph-mon[32751]: pgmap v9486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:38 smithi090 ceph-mon[29180]: pgmap v9486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:10:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:10:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:10:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:40 smithi119 ceph-mon[32752]: pgmap v9487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:10:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:10:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:10:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:40 smithi167 ceph-mon[32751]: pgmap v9487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:10:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:10:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:10:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:40 smithi090 ceph-mon[29180]: pgmap v9487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:10:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:10:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:10:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16522.233054 secs 2024-04-26T23:10:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:41 smithi119 ceph-mon[32752]: pgmap v9488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:10:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:41 smithi119 ceph-mon[32752]: pgmap v9489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16522.998824 secs 2024-04-26T23:10:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16522.233054 secs 2024-04-26T23:10:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:41 smithi167 ceph-mon[32751]: pgmap v9488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:10:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:41 smithi167 ceph-mon[32751]: pgmap v9489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16522.998824 secs 2024-04-26T23:10:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16522.233054 secs 2024-04-26T23:10:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:41 smithi090 ceph-mon[29180]: pgmap v9488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:10:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:41 smithi090 ceph-mon[29180]: pgmap v9489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16522.998824 secs 2024-04-26T23:10:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:43 smithi119 ceph-mon[32752]: pgmap v9490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:10:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:43 smithi167 ceph-mon[32751]: pgmap v9490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:10:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:43 smithi090 ceph-mon[29180]: pgmap v9490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:10:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:45 smithi119 ceph-mon[32752]: pgmap v9491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:45 smithi167 ceph-mon[32751]: pgmap v9491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:45 smithi090 ceph-mon[29180]: pgmap v9491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16527.233139 secs 2024-04-26T23:10:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16527.998950 secs 2024-04-26T23:10:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16527.233139 secs 2024-04-26T23:10:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16527.998950 secs 2024-04-26T23:10:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16527.233139 secs 2024-04-26T23:10:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16527.998950 secs 2024-04-26T23:10:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:47 smithi119 ceph-mon[32752]: pgmap v9492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:47 smithi167 ceph-mon[32751]: pgmap v9492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:47 smithi090 ceph-mon[29180]: pgmap v9492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:10:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:49 smithi119 ceph-mon[32752]: pgmap v9493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:10:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:49 smithi167 ceph-mon[32751]: pgmap v9493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:10:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:49 smithi090 ceph-mon[29180]: pgmap v9493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:10:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16532.233223 secs 2024-04-26T23:10:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:51 smithi119 ceph-mon[32752]: pgmap v9494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:10:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16532.999071 secs 2024-04-26T23:10:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16532.233223 secs 2024-04-26T23:10:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:51 smithi167 ceph-mon[32751]: pgmap v9494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:10:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16532.999071 secs 2024-04-26T23:10:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16532.233223 secs 2024-04-26T23:10:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:51 smithi090 ceph-mon[29180]: pgmap v9494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:10:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16532.999071 secs 2024-04-26T23:10:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:53 smithi119 ceph-mon[32752]: pgmap v9495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:53 smithi167 ceph-mon[32751]: pgmap v9495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:53 smithi090 ceph-mon[29180]: pgmap v9495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:55 smithi119 ceph-mon[32752]: pgmap v9496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:55 smithi167 ceph-mon[32751]: pgmap v9496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:55 smithi090 ceph-mon[29180]: pgmap v9496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:10:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16537.233307 secs 2024-04-26T23:10:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16537.999167 secs 2024-04-26T23:10:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16537.233307 secs 2024-04-26T23:10:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16537.999167 secs 2024-04-26T23:10:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16537.233307 secs 2024-04-26T23:10:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16537.999167 secs 2024-04-26T23:10:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:57 smithi119 ceph-mon[32752]: pgmap v9497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:57 smithi167 ceph-mon[32751]: pgmap v9497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:57 smithi090 ceph-mon[29180]: pgmap v9497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:10:59 smithi119 ceph-mon[32752]: pgmap v9498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:10:59 smithi167 ceph-mon[32751]: pgmap v9498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:10:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:10:59 smithi090 ceph-mon[29180]: pgmap v9498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16542.233391 secs 2024-04-26T23:11:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:01 smithi119 ceph-mon[32752]: pgmap v9499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16542.999308 secs 2024-04-26T23:11:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16542.233391 secs 2024-04-26T23:11:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:01 smithi167 ceph-mon[32751]: pgmap v9499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16542.999308 secs 2024-04-26T23:11:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16542.233391 secs 2024-04-26T23:11:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:01 smithi090 ceph-mon[29180]: pgmap v9499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16542.999308 secs 2024-04-26T23:11:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:03 smithi119 ceph-mon[32752]: pgmap v9500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:03 smithi167 ceph-mon[32751]: pgmap v9500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:03 smithi090 ceph-mon[29180]: pgmap v9500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:05 smithi119 ceph-mon[32752]: pgmap v9501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:05 smithi167 ceph-mon[32751]: pgmap v9501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:05 smithi090 ceph-mon[29180]: pgmap v9501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16547.233479 secs 2024-04-26T23:11:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:06 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 16547.999452 secs 2024-04-26T23:11:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:06 smithi119 ceph-mon[32752]: slow request 1440.867840 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:11:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16547.233479 secs 2024-04-26T23:11:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:06 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 16547.999452 secs 2024-04-26T23:11:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:06 smithi167 ceph-mon[32751]: slow request 1440.867840 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:11:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16547.233479 secs 2024-04-26T23:11:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:06 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 16547.999452 secs 2024-04-26T23:11:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:06 smithi090 ceph-mon[29180]: slow request 1440.867840 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:11:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:07 smithi119 ceph-mon[32752]: pgmap v9502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:07 smithi167 ceph-mon[32751]: pgmap v9502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:07 smithi090 ceph-mon[29180]: pgmap v9502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:09 smithi119 ceph-mon[32752]: pgmap v9503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:09 smithi167 ceph-mon[32751]: pgmap v9503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:09 smithi090 ceph-mon[29180]: pgmap v9503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:11 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 16552.233563 secs 2024-04-26T23:11:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:11 smithi167 ceph-mon[32751]: slow request 184.739576 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:11:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:11 smithi167 ceph-mon[32751]: pgmap v9504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16552.999526 secs 2024-04-26T23:11:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:11 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 16552.233563 secs 2024-04-26T23:11:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:11 smithi090 ceph-mon[29180]: slow request 184.739576 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:11:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:11 smithi090 ceph-mon[29180]: pgmap v9504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16552.999526 secs 2024-04-26T23:11:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:11 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 16552.233563 secs 2024-04-26T23:11:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:11 smithi119 ceph-mon[32752]: slow request 184.739576 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:11:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:11 smithi119 ceph-mon[32752]: pgmap v9504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16552.999526 secs 2024-04-26T23:11:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:13 smithi167 ceph-mon[32751]: pgmap v9505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:13 smithi090 ceph-mon[29180]: pgmap v9505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:13 smithi119 ceph-mon[32752]: pgmap v9505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:15 smithi167 ceph-mon[32751]: pgmap v9506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:11:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:11:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:15 smithi090 ceph-mon[29180]: pgmap v9506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:11:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:11:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:15 smithi119 ceph-mon[32752]: pgmap v9506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:11:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:11:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16557.233666 secs 2024-04-26T23:11:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16557.999602 secs 2024-04-26T23:11:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16557.233666 secs 2024-04-26T23:11:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16557.999602 secs 2024-04-26T23:11:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16557.233666 secs 2024-04-26T23:11:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16557.999602 secs 2024-04-26T23:11:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:17 smithi090 ceph-mon[29180]: pgmap v9507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:17 smithi119 ceph-mon[32752]: pgmap v9507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:17 smithi167 ceph-mon[32751]: pgmap v9507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:19 smithi090 ceph-mon[29180]: pgmap v9508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:20.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:19 smithi119 ceph-mon[32752]: pgmap v9508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:19 smithi167 ceph-mon[32751]: pgmap v9508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16562.233751 secs 2024-04-26T23:11:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:21 smithi167 ceph-mon[32751]: pgmap v9509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16562.999712 secs 2024-04-26T23:11:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16562.233751 secs 2024-04-26T23:11:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:21 smithi090 ceph-mon[29180]: pgmap v9509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16562.999712 secs 2024-04-26T23:11:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16562.233751 secs 2024-04-26T23:11:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:21 smithi119 ceph-mon[32752]: pgmap v9509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16562.999712 secs 2024-04-26T23:11:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:23 smithi090 ceph-mon[29180]: pgmap v9510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:23 smithi119 ceph-mon[32752]: pgmap v9510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:23 smithi167 ceph-mon[32751]: pgmap v9510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:25.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:25 smithi119 ceph-mon[32752]: pgmap v9511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:25 smithi090 ceph-mon[29180]: pgmap v9511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:25 smithi167 ceph-mon[32751]: pgmap v9511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16567.233838 secs 2024-04-26T23:11:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16567.999789 secs 2024-04-26T23:11:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16567.233838 secs 2024-04-26T23:11:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16567.999789 secs 2024-04-26T23:11:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16567.233838 secs 2024-04-26T23:11:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16567.999789 secs 2024-04-26T23:11:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:27 smithi090 ceph-mon[29180]: pgmap v9512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:27 smithi119 ceph-mon[32752]: pgmap v9512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:27 smithi167 ceph-mon[32751]: pgmap v9512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:29 smithi090 ceph-mon[29180]: pgmap v9513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:29 smithi119 ceph-mon[32752]: pgmap v9513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:29 smithi167 ceph-mon[32751]: pgmap v9513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16572.233953 secs 2024-04-26T23:11:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:31 smithi090 ceph-mon[29180]: pgmap v9514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16572.999921 secs 2024-04-26T23:11:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16572.233953 secs 2024-04-26T23:11:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:31 smithi119 ceph-mon[32752]: pgmap v9514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16572.999921 secs 2024-04-26T23:11:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16572.233953 secs 2024-04-26T23:11:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:31 smithi167 ceph-mon[32751]: pgmap v9514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16572.999921 secs 2024-04-26T23:11:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:33 smithi090 ceph-mon[29180]: pgmap v9515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:33 smithi119 ceph-mon[32752]: pgmap v9515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:33 smithi167 ceph-mon[32751]: pgmap v9515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:35 smithi090 ceph-mon[29180]: pgmap v9516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:35 smithi119 ceph-mon[32752]: pgmap v9516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:35 smithi167 ceph-mon[32751]: pgmap v9516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16577.234040 secs 2024-04-26T23:11:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16578.000053 secs 2024-04-26T23:11:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16577.234040 secs 2024-04-26T23:11:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16578.000053 secs 2024-04-26T23:11:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16577.234040 secs 2024-04-26T23:11:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16578.000053 secs 2024-04-26T23:11:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:37 smithi090 ceph-mon[29180]: pgmap v9517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:37 smithi119 ceph-mon[32752]: pgmap v9517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:37 smithi167 ceph-mon[32751]: pgmap v9517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:39 smithi090 ceph-mon[29180]: pgmap v9518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:39 smithi119 ceph-mon[32752]: pgmap v9518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:39 smithi167 ceph-mon[32751]: pgmap v9518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:11:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:11:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:11:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16582.234125 secs 2024-04-26T23:11:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:41 smithi119 ceph-mon[32752]: pgmap v9519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16583.000098 secs 2024-04-26T23:11:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:11:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:11:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:11:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16582.234125 secs 2024-04-26T23:11:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:41 smithi167 ceph-mon[32751]: pgmap v9519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16583.000098 secs 2024-04-26T23:11:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:11:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:11:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:11:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16582.234125 secs 2024-04-26T23:11:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:41 smithi090 ceph-mon[29180]: pgmap v9519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16583.000098 secs 2024-04-26T23:11:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:11:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:11:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:11:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:42 smithi090 ceph-mon[29180]: pgmap v9520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:11:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:42 smithi090 ceph-mon[29180]: pgmap v9521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:42 smithi119 ceph-mon[32752]: pgmap v9520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:11:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:42 smithi119 ceph-mon[32752]: pgmap v9521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:42 smithi167 ceph-mon[32751]: pgmap v9520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:11:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:42 smithi167 ceph-mon[32751]: pgmap v9521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:44 smithi119 ceph-mon[32752]: pgmap v9522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:11:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:44 smithi167 ceph-mon[32751]: pgmap v9522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:11:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:44 smithi090 ceph-mon[29180]: pgmap v9522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:11:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:46 smithi119 ceph-mon[32752]: pgmap v9523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16587.234206 secs 2024-04-26T23:11:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16588.000161 secs 2024-04-26T23:11:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:46 smithi167 ceph-mon[32751]: pgmap v9523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16587.234206 secs 2024-04-26T23:11:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16588.000161 secs 2024-04-26T23:11:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:46 smithi090 ceph-mon[29180]: pgmap v9523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16587.234206 secs 2024-04-26T23:11:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16588.000161 secs 2024-04-26T23:11:49.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:48 smithi119 ceph-mon[32752]: pgmap v9524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:48 smithi167 ceph-mon[32751]: pgmap v9524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:48 smithi090 ceph-mon[29180]: pgmap v9524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:50 smithi119 ceph-mon[32752]: pgmap v9525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:50 smithi167 ceph-mon[32751]: pgmap v9525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:50 smithi090 ceph-mon[29180]: pgmap v9525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:11:51.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16592.234292 secs 2024-04-26T23:11:51.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16593.000302 secs 2024-04-26T23:11:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16592.234292 secs 2024-04-26T23:11:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16593.000302 secs 2024-04-26T23:11:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16592.234292 secs 2024-04-26T23:11:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16593.000302 secs 2024-04-26T23:11:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:52 smithi119 ceph-mon[32752]: pgmap v9526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:11:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:52 smithi167 ceph-mon[32751]: pgmap v9526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:11:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:52 smithi090 ceph-mon[29180]: pgmap v9526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:11:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:54 smithi119 ceph-mon[32752]: pgmap v9527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:54 smithi167 ceph-mon[32751]: pgmap v9527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:55.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:54 smithi090 ceph-mon[29180]: pgmap v9527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:56 smithi119 ceph-mon[32752]: pgmap v9528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16597.234376 secs 2024-04-26T23:11:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16598.000445 secs 2024-04-26T23:11:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:56 smithi167 ceph-mon[32751]: pgmap v9528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16597.234376 secs 2024-04-26T23:11:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16598.000445 secs 2024-04-26T23:11:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:56 smithi090 ceph-mon[29180]: pgmap v9528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:11:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16597.234376 secs 2024-04-26T23:11:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16598.000445 secs 2024-04-26T23:11:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:11:58 smithi119 ceph-mon[32752]: pgmap v9529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:11:58 smithi167 ceph-mon[32751]: pgmap v9529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:11:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:11:58 smithi090 ceph-mon[29180]: pgmap v9529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:00 smithi119 ceph-mon[32752]: pgmap v9530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:00 smithi167 ceph-mon[32751]: pgmap v9530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:00 smithi090 ceph-mon[29180]: pgmap v9530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16602.234459 secs 2024-04-26T23:12:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16603.000559 secs 2024-04-26T23:12:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16602.234459 secs 2024-04-26T23:12:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16603.000559 secs 2024-04-26T23:12:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16602.234459 secs 2024-04-26T23:12:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16603.000559 secs 2024-04-26T23:12:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:02 smithi119 ceph-mon[32752]: pgmap v9531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:02 smithi167 ceph-mon[32751]: pgmap v9531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:02 smithi090 ceph-mon[29180]: pgmap v9531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:04 smithi119 ceph-mon[32752]: pgmap v9532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:04 smithi167 ceph-mon[32751]: pgmap v9532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:04 smithi090 ceph-mon[29180]: pgmap v9532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:06 smithi119 ceph-mon[32752]: pgmap v9533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16607.234542 secs 2024-04-26T23:12:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16608.000689 secs 2024-04-26T23:12:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:06 smithi167 ceph-mon[32751]: pgmap v9533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16607.234542 secs 2024-04-26T23:12:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16608.000689 secs 2024-04-26T23:12:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:06 smithi090 ceph-mon[29180]: pgmap v9533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16607.234542 secs 2024-04-26T23:12:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16608.000689 secs 2024-04-26T23:12:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:08 smithi119 ceph-mon[32752]: pgmap v9534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:08 smithi167 ceph-mon[32751]: pgmap v9534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:08 smithi090 ceph-mon[29180]: pgmap v9534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:10 smithi119 ceph-mon[32752]: pgmap v9535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:10 smithi167 ceph-mon[32751]: pgmap v9535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:10 smithi090 ceph-mon[29180]: pgmap v9535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16612.234624 secs 2024-04-26T23:12:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16613.000791 secs 2024-04-26T23:12:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16612.234624 secs 2024-04-26T23:12:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16613.000791 secs 2024-04-26T23:12:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16612.234624 secs 2024-04-26T23:12:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16613.000791 secs 2024-04-26T23:12:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:12 smithi119 ceph-mon[32752]: pgmap v9536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:12 smithi167 ceph-mon[32751]: pgmap v9536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:12 smithi090 ceph-mon[29180]: pgmap v9536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:14 smithi119 ceph-mon[32752]: pgmap v9537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:14 smithi167 ceph-mon[32751]: pgmap v9537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:14 smithi090 ceph-mon[29180]: pgmap v9537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:16 smithi119 ceph-mon[32752]: pgmap v9538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16617.234710 secs 2024-04-26T23:12:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16618.000934 secs 2024-04-26T23:12:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:16 smithi167 ceph-mon[32751]: pgmap v9538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16617.234710 secs 2024-04-26T23:12:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16618.000934 secs 2024-04-26T23:12:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:16 smithi090 ceph-mon[29180]: pgmap v9538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16617.234710 secs 2024-04-26T23:12:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16618.000934 secs 2024-04-26T23:12:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:18 smithi119 ceph-mon[32752]: pgmap v9539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:18 smithi167 ceph-mon[32751]: pgmap v9539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:18 smithi090 ceph-mon[29180]: pgmap v9539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:20 smithi119 ceph-mon[32752]: pgmap v9540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:20 smithi167 ceph-mon[32751]: pgmap v9540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:20 smithi090 ceph-mon[29180]: pgmap v9540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:21.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16622.234770 secs 2024-04-26T23:12:21.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16623.001053 secs 2024-04-26T23:12:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16622.234770 secs 2024-04-26T23:12:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16623.001053 secs 2024-04-26T23:12:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16622.234770 secs 2024-04-26T23:12:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16623.001053 secs 2024-04-26T23:12:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:22 smithi119 ceph-mon[32752]: pgmap v9541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:22 smithi167 ceph-mon[32751]: pgmap v9541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:22 smithi090 ceph-mon[29180]: pgmap v9541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:23 smithi119 ceph-mon[32752]: pgmap v9542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:23 smithi167 ceph-mon[32751]: pgmap v9542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:23 smithi090 ceph-mon[29180]: pgmap v9542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:25 smithi119 ceph-mon[32752]: pgmap v9543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:25 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16627.234803 secs 2024-04-26T23:12:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:25 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16628.001183 secs 2024-04-26T23:12:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:25 smithi167 ceph-mon[32751]: pgmap v9543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:25 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16627.234803 secs 2024-04-26T23:12:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:25 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16628.001183 secs 2024-04-26T23:12:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:25 smithi090 ceph-mon[29180]: pgmap v9543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:25 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16627.234803 secs 2024-04-26T23:12:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:25 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16628.001183 secs 2024-04-26T23:12:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:27 smithi119 ceph-mon[32752]: pgmap v9544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:27 smithi167 ceph-mon[32751]: pgmap v9544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:27 smithi090 ceph-mon[29180]: pgmap v9544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:29 smithi119 ceph-mon[32752]: pgmap v9545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:29 smithi167 ceph-mon[32751]: pgmap v9545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:29 smithi090 ceph-mon[29180]: pgmap v9545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:30 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16632.234889 secs 2024-04-26T23:12:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:30 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16633.001305 secs 2024-04-26T23:12:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:30 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16632.234889 secs 2024-04-26T23:12:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:30 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16633.001305 secs 2024-04-26T23:12:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:30 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16632.234889 secs 2024-04-26T23:12:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:30 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16633.001305 secs 2024-04-26T23:12:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:31 smithi119 ceph-mon[32752]: pgmap v9546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:31 smithi167 ceph-mon[32751]: pgmap v9546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:31 smithi090 ceph-mon[29180]: pgmap v9546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:33 smithi119 ceph-mon[32752]: pgmap v9547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:33 smithi167 ceph-mon[32751]: pgmap v9547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:33 smithi090 ceph-mon[29180]: pgmap v9547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:35 smithi119 ceph-mon[32752]: pgmap v9548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:35 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16637.234979 secs 2024-04-26T23:12:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16638.001431 secs 2024-04-26T23:12:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:35 smithi167 ceph-mon[32751]: pgmap v9548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:35 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16637.234979 secs 2024-04-26T23:12:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16638.001431 secs 2024-04-26T23:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:35 smithi090 ceph-mon[29180]: pgmap v9548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:35 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16637.234979 secs 2024-04-26T23:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16638.001431 secs 2024-04-26T23:12:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:37 smithi119 ceph-mon[32752]: pgmap v9549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:37 smithi167 ceph-mon[32751]: pgmap v9549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:37 smithi090 ceph-mon[29180]: pgmap v9549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:39 smithi119 ceph-mon[32752]: pgmap v9550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:39 smithi167 ceph-mon[32751]: pgmap v9550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:39 smithi090 ceph-mon[29180]: pgmap v9550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:40 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16642.235065 secs 2024-04-26T23:12:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16643.001494 secs 2024-04-26T23:12:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:40 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16642.235065 secs 2024-04-26T23:12:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16643.001494 secs 2024-04-26T23:12:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:40 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16642.235065 secs 2024-04-26T23:12:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16643.001494 secs 2024-04-26T23:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:41 smithi119 ceph-mon[32752]: pgmap v9551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:41 smithi119 ceph-mon[32752]: pgmap v9552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:41 smithi119 ceph-mon[32752]: pgmap v9553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:12:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:41 smithi167 ceph-mon[32751]: pgmap v9551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:12:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:12:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:12:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:41 smithi167 ceph-mon[32751]: pgmap v9552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:12:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:41 smithi167 ceph-mon[32751]: pgmap v9553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:41 smithi090 ceph-mon[29180]: pgmap v9551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:41 smithi090 ceph-mon[29180]: pgmap v9552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:41 smithi090 ceph-mon[29180]: pgmap v9553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:12:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:44 smithi090 ceph-mon[29180]: pgmap v9554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:44 smithi119 ceph-mon[32752]: pgmap v9554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:44 smithi167 ceph-mon[32751]: pgmap v9554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16647.235145 secs 2024-04-26T23:12:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:46 smithi119 ceph-mon[32752]: pgmap v9555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16648.001627 secs 2024-04-26T23:12:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16647.235145 secs 2024-04-26T23:12:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:46 smithi167 ceph-mon[32751]: pgmap v9555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16648.001627 secs 2024-04-26T23:12:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16647.235145 secs 2024-04-26T23:12:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:46 smithi090 ceph-mon[29180]: pgmap v9555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16648.001627 secs 2024-04-26T23:12:49.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:48 smithi119 ceph-mon[32752]: pgmap v9556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:12:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:48 smithi167 ceph-mon[32751]: pgmap v9556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:12:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:48 smithi090 ceph-mon[29180]: pgmap v9556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:12:50.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:49 smithi119 ceph-mon[32752]: pgmap v9557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:49 smithi167 ceph-mon[32751]: pgmap v9557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:49 smithi090 ceph-mon[29180]: pgmap v9557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:12:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:50 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16652.235238 secs 2024-04-26T23:12:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:50 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16652.235238 secs 2024-04-26T23:12:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:50 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16652.235238 secs 2024-04-26T23:12:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16653.001687 secs 2024-04-26T23:12:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:51 smithi119 ceph-mon[32752]: pgmap v9558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:12:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16653.001687 secs 2024-04-26T23:12:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:51 smithi167 ceph-mon[32751]: pgmap v9558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:12:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16653.001687 secs 2024-04-26T23:12:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:51 smithi090 ceph-mon[29180]: pgmap v9558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:12:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:54 smithi119 ceph-mon[32752]: pgmap v9559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:54 smithi167 ceph-mon[32751]: pgmap v9559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:54 smithi090 ceph-mon[29180]: pgmap v9559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:12:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16657.235321 secs 2024-04-26T23:12:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:56 smithi119 ceph-mon[32752]: pgmap v9560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16658.001770 secs 2024-04-26T23:12:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16657.235321 secs 2024-04-26T23:12:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:56 smithi167 ceph-mon[32751]: pgmap v9560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16658.001770 secs 2024-04-26T23:12:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16657.235321 secs 2024-04-26T23:12:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:56 smithi090 ceph-mon[29180]: pgmap v9560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16658.001770 secs 2024-04-26T23:12:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:12:57 smithi119 ceph-mon[32752]: pgmap v9561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:12:57 smithi167 ceph-mon[32751]: pgmap v9561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:12:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:12:57 smithi090 ceph-mon[29180]: pgmap v9561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:00 smithi119 ceph-mon[32752]: pgmap v9562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:00 smithi167 ceph-mon[32751]: pgmap v9562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:00 smithi090 ceph-mon[29180]: pgmap v9562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16662.235377 secs 2024-04-26T23:13:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16663.001893 secs 2024-04-26T23:13:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16662.235377 secs 2024-04-26T23:13:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16663.001893 secs 2024-04-26T23:13:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16662.235377 secs 2024-04-26T23:13:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16663.001893 secs 2024-04-26T23:13:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:02 smithi119 ceph-mon[32752]: pgmap v9563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:02 smithi167 ceph-mon[32751]: pgmap v9563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:02 smithi090 ceph-mon[29180]: pgmap v9563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:03 smithi119 ceph-mon[32752]: pgmap v9564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:03 smithi167 ceph-mon[32751]: pgmap v9564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:03 smithi090 ceph-mon[29180]: pgmap v9564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:05.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:13:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:13:05.305+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:13:05.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:13:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:13:05.305+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:13:05.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:13:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:13:05.305+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:13:05.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:13:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:13:05.305+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:13:05.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:13:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:13:05.305+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:13:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 16667.235452 secs 2024-04-26T23:13:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:06 smithi119 ceph-mon[32752]: pgmap v9565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16668.001975 secs 2024-04-26T23:13:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 16667.235452 secs 2024-04-26T23:13:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:06 smithi167 ceph-mon[32751]: pgmap v9565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16668.001975 secs 2024-04-26T23:13:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 16667.235452 secs 2024-04-26T23:13:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:06 smithi090 ceph-mon[29180]: pgmap v9565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16668.001975 secs 2024-04-26T23:13:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:08 smithi119 ceph-mon[32752]: pgmap v9566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:08 smithi167 ceph-mon[32751]: pgmap v9566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:08 smithi090 ceph-mon[29180]: pgmap v9566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:09 smithi119 ceph-mon[32752]: pgmap v9567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:09 smithi167 ceph-mon[32751]: pgmap v9567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:09 smithi090 ceph-mon[29180]: pgmap v9567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16672.235552 secs 2024-04-26T23:13:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16672.235552 secs 2024-04-26T23:13:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16672.235552 secs 2024-04-26T23:13:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16673.002087 secs 2024-04-26T23:13:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:11 smithi167 ceph-mon[32751]: pgmap v9568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16673.002087 secs 2024-04-26T23:13:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:11 smithi090 ceph-mon[29180]: pgmap v9568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16673.002087 secs 2024-04-26T23:13:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:11 smithi119 ceph-mon[32752]: pgmap v9568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:14 smithi119 ceph-mon[32752]: pgmap v9569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:14 smithi167 ceph-mon[32751]: pgmap v9569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:14 smithi090 ceph-mon[29180]: pgmap v9569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:13:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:13:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:15 smithi119 ceph-mon[32752]: pgmap v9570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:13:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:13:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:15 smithi167 ceph-mon[32751]: pgmap v9570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:13:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:13:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:15 smithi090 ceph-mon[29180]: pgmap v9570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16677.235612 secs 2024-04-26T23:13:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16678.002150 secs 2024-04-26T23:13:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16677.235612 secs 2024-04-26T23:13:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16678.002150 secs 2024-04-26T23:13:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16677.235612 secs 2024-04-26T23:13:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16678.002150 secs 2024-04-26T23:13:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:17 smithi119 ceph-mon[32752]: pgmap v9571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:17 smithi167 ceph-mon[32751]: pgmap v9571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:17 smithi090 ceph-mon[29180]: pgmap v9571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:13:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:20 smithi119 ceph-mon[32752]: pgmap v9572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:20 smithi167 ceph-mon[32751]: pgmap v9572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:20 smithi090 ceph-mon[29180]: pgmap v9572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:13:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16682.235696 secs 2024-04-26T23:13:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16683.002263 secs 2024-04-26T23:13:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16682.235696 secs 2024-04-26T23:13:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16683.002263 secs 2024-04-26T23:13:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16682.235696 secs 2024-04-26T23:13:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16683.002263 secs 2024-04-26T23:13:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:22 smithi119 ceph-mon[32752]: pgmap v9573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:22 smithi167 ceph-mon[32751]: pgmap v9573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:22 smithi090 ceph-mon[29180]: pgmap v9573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:23 smithi119 ceph-mon[32752]: pgmap v9574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:23 smithi167 ceph-mon[32751]: pgmap v9574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:23 smithi090 ceph-mon[29180]: pgmap v9574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16687.235781 secs 2024-04-26T23:13:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:26 smithi119 ceph-mon[32752]: pgmap v9575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16688.002341 secs 2024-04-26T23:13:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16687.235781 secs 2024-04-26T23:13:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:26 smithi167 ceph-mon[32751]: pgmap v9575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16688.002341 secs 2024-04-26T23:13:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16687.235781 secs 2024-04-26T23:13:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:26 smithi090 ceph-mon[29180]: pgmap v9575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16688.002341 secs 2024-04-26T23:13:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:27 smithi119 ceph-mon[32752]: pgmap v9576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:27 smithi167 ceph-mon[32751]: pgmap v9576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:27 smithi090 ceph-mon[29180]: pgmap v9576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:29 smithi119 ceph-mon[32752]: pgmap v9577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:29 smithi167 ceph-mon[32751]: pgmap v9577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:30.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:29 smithi090 ceph-mon[29180]: pgmap v9577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:31.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16692.235864 secs 2024-04-26T23:13:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16692.235864 secs 2024-04-26T23:13:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16692.235864 secs 2024-04-26T23:13:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16693.002453 secs 2024-04-26T23:13:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:31 smithi090 ceph-mon[29180]: pgmap v9578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16693.002453 secs 2024-04-26T23:13:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:31 smithi119 ceph-mon[32752]: pgmap v9578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16693.002453 secs 2024-04-26T23:13:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:31 smithi167 ceph-mon[32751]: pgmap v9578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:33 smithi119 ceph-mon[32752]: pgmap v9579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:33 smithi167 ceph-mon[32751]: pgmap v9579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:33 smithi090 ceph-mon[29180]: pgmap v9579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16697.235926 secs 2024-04-26T23:13:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:35 smithi119 ceph-mon[32752]: pgmap v9580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16697.235926 secs 2024-04-26T23:13:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:35 smithi167 ceph-mon[32751]: pgmap v9580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16697.235926 secs 2024-04-26T23:13:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:35 smithi090 ceph-mon[29180]: pgmap v9580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16698.002564 secs 2024-04-26T23:13:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16698.002564 secs 2024-04-26T23:13:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16698.002564 secs 2024-04-26T23:13:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:37 smithi119 ceph-mon[32752]: pgmap v9581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:37 smithi167 ceph-mon[32751]: pgmap v9581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:37 smithi090 ceph-mon[29180]: pgmap v9581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:39 smithi119 ceph-mon[32752]: pgmap v9582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:39 smithi167 ceph-mon[32751]: pgmap v9582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:39 smithi090 ceph-mon[29180]: pgmap v9582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16702.236014 secs 2024-04-26T23:13:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16702.236014 secs 2024-04-26T23:13:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16702.236014 secs 2024-04-26T23:13:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16703.002672 secs 2024-04-26T23:13:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:41 smithi119 ceph-mon[32752]: pgmap v9583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:13:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16703.002672 secs 2024-04-26T23:13:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:41 smithi167 ceph-mon[32751]: pgmap v9583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:13:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16703.002672 secs 2024-04-26T23:13:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:41 smithi090 ceph-mon[29180]: pgmap v9583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:13:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:43 smithi119 ceph-mon[32752]: pgmap v9584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:43 smithi167 ceph-mon[32751]: pgmap v9584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:43 smithi090 ceph-mon[29180]: pgmap v9584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:45.937 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16707.236076 secs 2024-04-26T23:13:45.937 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:45 smithi119 ceph-mon[32752]: pgmap v9585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16707.236076 secs 2024-04-26T23:13:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:45 smithi167 ceph-mon[32751]: pgmap v9585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:46.086 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16707.236076 secs 2024-04-26T23:13:46.087 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:45 smithi090 ceph-mon[29180]: pgmap v9585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16708.002743 secs 2024-04-26T23:13:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:13:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:13:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: pgmap v9586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-26T23:13:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: pgmap v9587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16708.002743 secs 2024-04-26T23:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:13:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:13:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: pgmap v9586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-26T23:13:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: pgmap v9587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16708.002743 secs 2024-04-26T23:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:13:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:13:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: pgmap v9586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-26T23:13:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: pgmap v9587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:13:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:49 smithi119 ceph-mon[32752]: pgmap v9588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:13:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:49 smithi167 ceph-mon[32751]: pgmap v9588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:13:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:49 smithi090 ceph-mon[29180]: pgmap v9588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:13:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16712.236160 secs 2024-04-26T23:13:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:51 smithi119 ceph-mon[32752]: pgmap v9589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16713.002860 secs 2024-04-26T23:13:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16712.236160 secs 2024-04-26T23:13:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:51 smithi167 ceph-mon[32751]: pgmap v9589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16713.002860 secs 2024-04-26T23:13:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16712.236160 secs 2024-04-26T23:13:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:51 smithi090 ceph-mon[29180]: pgmap v9589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16713.002860 secs 2024-04-26T23:13:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:53 smithi119 ceph-mon[32752]: pgmap v9590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:13:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:53 smithi167 ceph-mon[32751]: pgmap v9590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:13:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:53 smithi090 ceph-mon[29180]: pgmap v9590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:13:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:55 smithi119 ceph-mon[32752]: pgmap v9591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:55 smithi167 ceph-mon[32751]: pgmap v9591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:55 smithi090 ceph-mon[29180]: pgmap v9591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:13:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16717.236245 secs 2024-04-26T23:13:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16718.002958 secs 2024-04-26T23:13:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16717.236245 secs 2024-04-26T23:13:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16718.002958 secs 2024-04-26T23:13:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16717.236245 secs 2024-04-26T23:13:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16718.002958 secs 2024-04-26T23:13:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:57 smithi119 ceph-mon[32752]: pgmap v9592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:13:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:57 smithi167 ceph-mon[32751]: pgmap v9592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:13:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:57 smithi090 ceph-mon[29180]: pgmap v9592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:13:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:13:59 smithi119 ceph-mon[32752]: pgmap v9593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:13:59 smithi167 ceph-mon[32751]: pgmap v9593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:13:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:13:59 smithi090 ceph-mon[29180]: pgmap v9593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16722.236335 secs 2024-04-26T23:14:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:01 smithi119 ceph-mon[32752]: pgmap v9594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16723.003098 secs 2024-04-26T23:14:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16722.236335 secs 2024-04-26T23:14:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:01 smithi167 ceph-mon[32751]: pgmap v9594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16723.003098 secs 2024-04-26T23:14:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16722.236335 secs 2024-04-26T23:14:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:01 smithi090 ceph-mon[29180]: pgmap v9594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16723.003098 secs 2024-04-26T23:14:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:03 smithi119 ceph-mon[32752]: pgmap v9595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:03 smithi167 ceph-mon[32751]: pgmap v9595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:03 smithi090 ceph-mon[29180]: pgmap v9595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:05 smithi119 ceph-mon[32752]: pgmap v9596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:05 smithi167 ceph-mon[32751]: pgmap v9596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:05 smithi090 ceph-mon[29180]: pgmap v9596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16727.236415 secs 2024-04-26T23:14:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16728.003220 secs 2024-04-26T23:14:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16727.236415 secs 2024-04-26T23:14:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16728.003220 secs 2024-04-26T23:14:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16727.236415 secs 2024-04-26T23:14:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16728.003220 secs 2024-04-26T23:14:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:07 smithi119 ceph-mon[32752]: pgmap v9597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:07 smithi167 ceph-mon[32751]: pgmap v9597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:07 smithi090 ceph-mon[29180]: pgmap v9597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:09 smithi119 ceph-mon[32752]: pgmap v9598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:09 smithi167 ceph-mon[32751]: pgmap v9598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:09 smithi090 ceph-mon[29180]: pgmap v9598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:11 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 16732.236498 secs 2024-04-26T23:14:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:11 smithi119 ceph-mon[32752]: slow request 364.742511 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:14:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:11 smithi119 ceph-mon[32752]: pgmap v9599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16733.003360 secs 2024-04-26T23:14:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:11 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 16732.236498 secs 2024-04-26T23:14:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:11 smithi167 ceph-mon[32751]: slow request 364.742511 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:14:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:11 smithi167 ceph-mon[32751]: pgmap v9599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16733.003360 secs 2024-04-26T23:14:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:11 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 16732.236498 secs 2024-04-26T23:14:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:11 smithi090 ceph-mon[29180]: slow request 364.742511 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:14:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:11 smithi090 ceph-mon[29180]: pgmap v9599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16733.003360 secs 2024-04-26T23:14:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:13 smithi119 ceph-mon[32752]: pgmap v9600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:13 smithi167 ceph-mon[32751]: pgmap v9600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:13 smithi090 ceph-mon[29180]: pgmap v9600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:15 smithi119 ceph-mon[32752]: pgmap v9601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:15 smithi167 ceph-mon[32751]: pgmap v9601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:15 smithi090 ceph-mon[29180]: pgmap v9601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16737.236583 secs 2024-04-26T23:14:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16738.003450 secs 2024-04-26T23:14:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16737.236583 secs 2024-04-26T23:14:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16738.003450 secs 2024-04-26T23:14:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16737.236583 secs 2024-04-26T23:14:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16738.003450 secs 2024-04-26T23:14:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:17 smithi119 ceph-mon[32752]: pgmap v9602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:17 smithi167 ceph-mon[32751]: pgmap v9602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:17 smithi090 ceph-mon[29180]: pgmap v9602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:19 smithi119 ceph-mon[32752]: pgmap v9603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:19 smithi167 ceph-mon[32751]: pgmap v9603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:19 smithi090 ceph-mon[29180]: pgmap v9603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16742.236666 secs 2024-04-26T23:14:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:21 smithi119 ceph-mon[32752]: pgmap v9604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16743.003515 secs 2024-04-26T23:14:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16742.236666 secs 2024-04-26T23:14:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:21 smithi167 ceph-mon[32751]: pgmap v9604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16743.003515 secs 2024-04-26T23:14:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16742.236666 secs 2024-04-26T23:14:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:21 smithi090 ceph-mon[29180]: pgmap v9604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16743.003515 secs 2024-04-26T23:14:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:23 smithi119 ceph-mon[32752]: pgmap v9605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:23 smithi167 ceph-mon[32751]: pgmap v9605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:23 smithi090 ceph-mon[29180]: pgmap v9605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:25 smithi119 ceph-mon[32752]: pgmap v9606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:25 smithi167 ceph-mon[32751]: pgmap v9606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:25 smithi090 ceph-mon[29180]: pgmap v9606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16747.236749 secs 2024-04-26T23:14:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16748.003577 secs 2024-04-26T23:14:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16747.236749 secs 2024-04-26T23:14:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16748.003577 secs 2024-04-26T23:14:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16747.236749 secs 2024-04-26T23:14:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16748.003577 secs 2024-04-26T23:14:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:27 smithi119 ceph-mon[32752]: pgmap v9607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:27 smithi167 ceph-mon[32751]: pgmap v9607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:27 smithi090 ceph-mon[29180]: pgmap v9607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:29 smithi119 ceph-mon[32752]: pgmap v9608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:29 smithi167 ceph-mon[32751]: pgmap v9608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:29 smithi090 ceph-mon[29180]: pgmap v9608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16752.236832 secs 2024-04-26T23:14:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:31 smithi119 ceph-mon[32752]: pgmap v9609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16753.003703 secs 2024-04-26T23:14:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16752.236832 secs 2024-04-26T23:14:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:31 smithi167 ceph-mon[32751]: pgmap v9609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16753.003703 secs 2024-04-26T23:14:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16752.236832 secs 2024-04-26T23:14:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:31 smithi090 ceph-mon[29180]: pgmap v9609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16753.003703 secs 2024-04-26T23:14:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:33 smithi119 ceph-mon[32752]: pgmap v9610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:33 smithi167 ceph-mon[32751]: pgmap v9610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:33.573 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:33 smithi090 ceph-mon[29180]: pgmap v9610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:35 smithi119 ceph-mon[32752]: pgmap v9611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:35 smithi167 ceph-mon[32751]: pgmap v9611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:35 smithi090 ceph-mon[29180]: pgmap v9611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16757.236918 secs 2024-04-26T23:14:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16758.003786 secs 2024-04-26T23:14:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16757.236918 secs 2024-04-26T23:14:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16758.003786 secs 2024-04-26T23:14:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16757.236918 secs 2024-04-26T23:14:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16758.003786 secs 2024-04-26T23:14:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:37 smithi167 ceph-mon[32751]: pgmap v9612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:37 smithi090 ceph-mon[29180]: pgmap v9612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:37 smithi119 ceph-mon[32752]: pgmap v9612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:39 smithi167 ceph-mon[32751]: pgmap v9613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:39 smithi090 ceph-mon[29180]: pgmap v9613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:39 smithi119 ceph-mon[32752]: pgmap v9613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16762.236981 secs 2024-04-26T23:14:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:41 smithi090 ceph-mon[29180]: pgmap v9614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16763.003898 secs 2024-04-26T23:14:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16762.236981 secs 2024-04-26T23:14:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:41 smithi119 ceph-mon[32752]: pgmap v9614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16763.003898 secs 2024-04-26T23:14:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16762.236981 secs 2024-04-26T23:14:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:41 smithi167 ceph-mon[32751]: pgmap v9614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16763.003898 secs 2024-04-26T23:14:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:43 smithi090 ceph-mon[29180]: pgmap v9615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:43 smithi119 ceph-mon[32752]: pgmap v9615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:43 smithi167 ceph-mon[32751]: pgmap v9615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:45 smithi090 ceph-mon[29180]: pgmap v9616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:45 smithi119 ceph-mon[32752]: pgmap v9616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:45 smithi167 ceph-mon[32751]: pgmap v9616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16767.237042 secs 2024-04-26T23:14:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16768.003960 secs 2024-04-26T23:14:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:14:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16767.237042 secs 2024-04-26T23:14:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16768.003960 secs 2024-04-26T23:14:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:14:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16767.237042 secs 2024-04-26T23:14:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16768.003960 secs 2024-04-26T23:14:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:14:47.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:47 smithi167 ceph-mon[32751]: pgmap v9617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:47 smithi090 ceph-mon[29180]: pgmap v9617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:47 smithi119 ceph-mon[32752]: pgmap v9617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:49 smithi167 ceph-mon[32751]: pgmap v9618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:49 smithi090 ceph-mon[29180]: pgmap v9618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:49 smithi119 ceph-mon[32752]: pgmap v9618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:14:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16772.237126 secs 2024-04-26T23:14:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: pgmap v9619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:14:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:14:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: pgmap v9620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16773.004102 secs 2024-04-26T23:14:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16772.237126 secs 2024-04-26T23:14:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: pgmap v9619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:14:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:14:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: pgmap v9620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16773.004102 secs 2024-04-26T23:14:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16772.237126 secs 2024-04-26T23:14:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: pgmap v9619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:14:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:14:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:14:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: pgmap v9620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:14:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16773.004102 secs 2024-04-26T23:14:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:53 smithi167 ceph-mon[32751]: pgmap v9621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:53 smithi090 ceph-mon[29180]: pgmap v9621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:53 smithi119 ceph-mon[32752]: pgmap v9621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:55 smithi167 ceph-mon[32751]: pgmap v9622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:55 smithi090 ceph-mon[29180]: pgmap v9622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:55 smithi119 ceph-mon[32752]: pgmap v9622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16777.237207 secs 2024-04-26T23:14:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16778.004186 secs 2024-04-26T23:14:56.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16777.237207 secs 2024-04-26T23:14:56.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16778.004186 secs 2024-04-26T23:14:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16777.237207 secs 2024-04-26T23:14:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16778.004186 secs 2024-04-26T23:14:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:57 smithi090 ceph-mon[29180]: pgmap v9623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:57 smithi119 ceph-mon[32752]: pgmap v9623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:57 smithi167 ceph-mon[32751]: pgmap v9623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:14:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:14:59 smithi090 ceph-mon[29180]: pgmap v9624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T23:15:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:14:59 smithi119 ceph-mon[32752]: pgmap v9624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T23:15:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:14:59 smithi167 ceph-mon[32751]: pgmap v9624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-26T23:15:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16782.237288 secs 2024-04-26T23:15:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:01 smithi090 ceph-mon[29180]: pgmap v9625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:15:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16783.004330 secs 2024-04-26T23:15:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16782.237288 secs 2024-04-26T23:15:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:01 smithi119 ceph-mon[32752]: pgmap v9625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:15:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16783.004330 secs 2024-04-26T23:15:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16782.237288 secs 2024-04-26T23:15:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:01 smithi167 ceph-mon[32751]: pgmap v9625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 991 B/s rd, 1 op/s 2024-04-26T23:15:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16783.004330 secs 2024-04-26T23:15:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:03 smithi090 ceph-mon[29180]: pgmap v9626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:03 smithi119 ceph-mon[32752]: pgmap v9626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:03 smithi167 ceph-mon[32751]: pgmap v9626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:05 smithi090 ceph-mon[29180]: pgmap v9627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:05 smithi119 ceph-mon[32752]: pgmap v9627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:05 smithi167 ceph-mon[32751]: pgmap v9627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16787.237372 secs 2024-04-26T23:15:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16788.004464 secs 2024-04-26T23:15:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16787.237372 secs 2024-04-26T23:15:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16788.004464 secs 2024-04-26T23:15:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16787.237372 secs 2024-04-26T23:15:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16788.004464 secs 2024-04-26T23:15:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:07 smithi090 ceph-mon[29180]: pgmap v9628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:07 smithi119 ceph-mon[32752]: pgmap v9628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:07 smithi167 ceph-mon[32751]: pgmap v9628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:09 smithi090 ceph-mon[29180]: pgmap v9629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:09 smithi119 ceph-mon[32752]: pgmap v9629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:09 smithi167 ceph-mon[32751]: pgmap v9629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:11 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 16792.237456 secs 2024-04-26T23:15:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:11 smithi090 ceph-mon[29180]: slow request 724.865289 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:15:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:11 smithi090 ceph-mon[29180]: pgmap v9630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16793.004563 secs 2024-04-26T23:15:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:11 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 16792.237456 secs 2024-04-26T23:15:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:11 smithi119 ceph-mon[32752]: slow request 724.865289 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:15:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:11 smithi119 ceph-mon[32752]: pgmap v9630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16793.004563 secs 2024-04-26T23:15:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:11 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 16792.237456 secs 2024-04-26T23:15:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:11 smithi167 ceph-mon[32751]: slow request 724.865289 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:15:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:11 smithi167 ceph-mon[32751]: pgmap v9630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16793.004563 secs 2024-04-26T23:15:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:13 smithi090 ceph-mon[29180]: pgmap v9631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:13 smithi119 ceph-mon[32752]: pgmap v9631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:13 smithi167 ceph-mon[32751]: pgmap v9631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:15 smithi090 ceph-mon[29180]: pgmap v9632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:15 smithi119 ceph-mon[32752]: pgmap v9632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:15 smithi167 ceph-mon[32751]: pgmap v9632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16797.237540 secs 2024-04-26T23:15:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16798.004644 secs 2024-04-26T23:15:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16797.237540 secs 2024-04-26T23:15:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16798.004644 secs 2024-04-26T23:15:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16797.237540 secs 2024-04-26T23:15:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16798.004644 secs 2024-04-26T23:15:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:17 smithi090 ceph-mon[29180]: pgmap v9633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:17 smithi119 ceph-mon[32752]: pgmap v9633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:17 smithi167 ceph-mon[32751]: pgmap v9633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:19 smithi090 ceph-mon[29180]: pgmap v9634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:20.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:19 smithi119 ceph-mon[32752]: pgmap v9634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:19 smithi167 ceph-mon[32751]: pgmap v9634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16802.237601 secs 2024-04-26T23:15:21.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:21 smithi167 ceph-mon[32751]: pgmap v9635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:21.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16803.004709 secs 2024-04-26T23:15:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16802.237601 secs 2024-04-26T23:15:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:21 smithi090 ceph-mon[29180]: pgmap v9635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16803.004709 secs 2024-04-26T23:15:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16802.237601 secs 2024-04-26T23:15:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:21 smithi119 ceph-mon[32752]: pgmap v9635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16803.004709 secs 2024-04-26T23:15:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:23 smithi090 ceph-mon[29180]: pgmap v9636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:23 smithi119 ceph-mon[32752]: pgmap v9636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:23 smithi167 ceph-mon[32751]: pgmap v9636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:25 smithi090 ceph-mon[29180]: pgmap v9637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:25 smithi119 ceph-mon[32752]: pgmap v9637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:25 smithi167 ceph-mon[32751]: pgmap v9637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16807.237686 secs 2024-04-26T23:15:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16808.004858 secs 2024-04-26T23:15:26.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16807.237686 secs 2024-04-26T23:15:26.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16808.004858 secs 2024-04-26T23:15:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16807.237686 secs 2024-04-26T23:15:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16808.004858 secs 2024-04-26T23:15:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:27 smithi119 ceph-mon[32752]: pgmap v9638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:27 smithi167 ceph-mon[32751]: pgmap v9638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:27 smithi090 ceph-mon[29180]: pgmap v9638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:29 smithi119 ceph-mon[32752]: pgmap v9639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:29 smithi167 ceph-mon[32751]: pgmap v9639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:29 smithi090 ceph-mon[29180]: pgmap v9639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:31.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16812.237771 secs 2024-04-26T23:15:31.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:31 smithi090 ceph-mon[29180]: pgmap v9640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:31.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16813.004943 secs 2024-04-26T23:15:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16812.237771 secs 2024-04-26T23:15:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:31 smithi119 ceph-mon[32752]: pgmap v9640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16813.004943 secs 2024-04-26T23:15:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16812.237771 secs 2024-04-26T23:15:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:31 smithi167 ceph-mon[32751]: pgmap v9640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16813.004943 secs 2024-04-26T23:15:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:33 smithi119 ceph-mon[32752]: pgmap v9641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:33 smithi167 ceph-mon[32751]: pgmap v9641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:33 smithi090 ceph-mon[29180]: pgmap v9641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:35 smithi090 ceph-mon[29180]: pgmap v9642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:35 smithi119 ceph-mon[32752]: pgmap v9642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:35 smithi167 ceph-mon[32751]: pgmap v9642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16817.237853 secs 2024-04-26T23:15:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16818.005016 secs 2024-04-26T23:15:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16817.237853 secs 2024-04-26T23:15:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16818.005016 secs 2024-04-26T23:15:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16817.237853 secs 2024-04-26T23:15:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16818.005016 secs 2024-04-26T23:15:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:37 smithi119 ceph-mon[32752]: pgmap v9643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:37 smithi167 ceph-mon[32751]: pgmap v9643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:37 smithi090 ceph-mon[29180]: pgmap v9643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:39 smithi119 ceph-mon[32752]: pgmap v9644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:39 smithi167 ceph-mon[32751]: pgmap v9644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:39 smithi090 ceph-mon[29180]: pgmap v9644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16822.237940 secs 2024-04-26T23:15:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:41 smithi119 ceph-mon[32752]: pgmap v9645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16823.005172 secs 2024-04-26T23:15:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16822.237940 secs 2024-04-26T23:15:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:41 smithi167 ceph-mon[32751]: pgmap v9645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16823.005172 secs 2024-04-26T23:15:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16822.237940 secs 2024-04-26T23:15:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:41 smithi090 ceph-mon[29180]: pgmap v9645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16823.005172 secs 2024-04-26T23:15:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:43 smithi119 ceph-mon[32752]: pgmap v9646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:43 smithi167 ceph-mon[32751]: pgmap v9646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:43 smithi090 ceph-mon[29180]: pgmap v9646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:45 smithi119 ceph-mon[32752]: pgmap v9647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:45 smithi167 ceph-mon[32751]: pgmap v9647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:45 smithi090 ceph-mon[29180]: pgmap v9647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16827.238024 secs 2024-04-26T23:15:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16828.005293 secs 2024-04-26T23:15:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16827.238024 secs 2024-04-26T23:15:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16828.005293 secs 2024-04-26T23:15:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16827.238024 secs 2024-04-26T23:15:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16828.005293 secs 2024-04-26T23:15:48.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:47 smithi119 ceph-mon[32752]: pgmap v9648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:47 smithi167 ceph-mon[32751]: pgmap v9648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:47 smithi090 ceph-mon[29180]: pgmap v9648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:49 smithi119 ceph-mon[32752]: pgmap v9649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:49 smithi167 ceph-mon[32751]: pgmap v9649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:49 smithi090 ceph-mon[29180]: pgmap v9649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:15:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16832.238105 secs 2024-04-26T23:15:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:51 smithi119 ceph-mon[32752]: pgmap v9650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:15:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16833.005345 secs 2024-04-26T23:15:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:15:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:15:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16832.238105 secs 2024-04-26T23:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:51 smithi167 ceph-mon[32751]: pgmap v9650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16833.005345 secs 2024-04-26T23:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:15:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16832.238105 secs 2024-04-26T23:15:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:51 smithi090 ceph-mon[29180]: pgmap v9650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:15:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:15:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16833.005345 secs 2024-04-26T23:15:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:15:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:15:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:15:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:52 smithi119 ceph-mon[32752]: pgmap v9651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 B/s rd, 1 op/s 2024-04-26T23:15:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:52 smithi119 ceph-mon[32752]: pgmap v9652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:52 smithi167 ceph-mon[32751]: pgmap v9651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 B/s rd, 1 op/s 2024-04-26T23:15:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:52 smithi167 ceph-mon[32751]: pgmap v9652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:52 smithi090 ceph-mon[29180]: pgmap v9651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 949 B/s rd, 1 op/s 2024-04-26T23:15:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:52 smithi090 ceph-mon[29180]: pgmap v9652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:54 smithi119 ceph-mon[32752]: pgmap v9653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:54 smithi167 ceph-mon[32751]: pgmap v9653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:54 smithi090 ceph-mon[29180]: pgmap v9653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16837.238185 secs 2024-04-26T23:15:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:56 smithi119 ceph-mon[32752]: pgmap v9654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16838.005451 secs 2024-04-26T23:15:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16837.238185 secs 2024-04-26T23:15:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:56 smithi167 ceph-mon[32751]: pgmap v9654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16838.005451 secs 2024-04-26T23:15:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16837.238185 secs 2024-04-26T23:15:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:56 smithi090 ceph-mon[29180]: pgmap v9654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:15:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16838.005451 secs 2024-04-26T23:15:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:15:58 smithi119 ceph-mon[32752]: pgmap v9655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:15:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:15:58 smithi167 ceph-mon[32751]: pgmap v9655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:15:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:15:58 smithi090 ceph-mon[29180]: pgmap v9655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:16:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:00 smithi119 ceph-mon[32752]: pgmap v9656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:00 smithi167 ceph-mon[32751]: pgmap v9656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:00 smithi090 ceph-mon[29180]: pgmap v9656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16842.238272 secs 2024-04-26T23:16:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16843.005546 secs 2024-04-26T23:16:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16842.238272 secs 2024-04-26T23:16:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16843.005546 secs 2024-04-26T23:16:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16842.238272 secs 2024-04-26T23:16:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16843.005546 secs 2024-04-26T23:16:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:02 smithi119 ceph-mon[32752]: pgmap v9657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:16:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:02 smithi167 ceph-mon[32751]: pgmap v9657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:16:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:02 smithi090 ceph-mon[29180]: pgmap v9657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:16:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:04 smithi119 ceph-mon[32752]: pgmap v9658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:04 smithi167 ceph-mon[32751]: pgmap v9658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:04 smithi090 ceph-mon[29180]: pgmap v9658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 16847.238356 secs 2024-04-26T23:16:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:06 smithi119 ceph-mon[32752]: pgmap v9659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16848.005662 secs 2024-04-26T23:16:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 16847.238356 secs 2024-04-26T23:16:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:06 smithi167 ceph-mon[32751]: pgmap v9659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16848.005662 secs 2024-04-26T23:16:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 16847.238356 secs 2024-04-26T23:16:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:06 smithi090 ceph-mon[29180]: pgmap v9659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16848.005662 secs 2024-04-26T23:16:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:08 smithi119 ceph-mon[32752]: pgmap v9660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:08 smithi167 ceph-mon[32751]: pgmap v9660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:08 smithi090 ceph-mon[29180]: pgmap v9660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:10 smithi119 ceph-mon[32752]: pgmap v9661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:10 smithi167 ceph-mon[32751]: pgmap v9661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:10 smithi090 ceph-mon[29180]: pgmap v9661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:11 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 16852.238458 secs 2024-04-26T23:16:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:11 smithi119 ceph-mon[32752]: slow request 184.677086 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:16:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16853.005781 secs 2024-04-26T23:16:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:11 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 16852.238458 secs 2024-04-26T23:16:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:11 smithi167 ceph-mon[32751]: slow request 184.677086 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:16:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16853.005781 secs 2024-04-26T23:16:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:11 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 16852.238458 secs 2024-04-26T23:16:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:11 smithi090 ceph-mon[29180]: slow request 184.677086 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:16:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16853.005781 secs 2024-04-26T23:16:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:12 smithi119 ceph-mon[32752]: pgmap v9662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:12 smithi167 ceph-mon[32751]: pgmap v9662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:12 smithi090 ceph-mon[29180]: pgmap v9662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:14 smithi119 ceph-mon[32752]: pgmap v9663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:14 smithi167 ceph-mon[32751]: pgmap v9663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:14 smithi090 ceph-mon[29180]: pgmap v9663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:16:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:16:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:16:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:16:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:16:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:16:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 16857.238536 secs 2024-04-26T23:16:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:16 smithi119 ceph-mon[32752]: pgmap v9664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16858.005895 secs 2024-04-26T23:16:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 16857.238536 secs 2024-04-26T23:16:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:16 smithi167 ceph-mon[32751]: pgmap v9664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16858.005895 secs 2024-04-26T23:16:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 16857.238536 secs 2024-04-26T23:16:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:16 smithi090 ceph-mon[29180]: pgmap v9664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16858.005895 secs 2024-04-26T23:16:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:18 smithi119 ceph-mon[32752]: pgmap v9665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:18 smithi167 ceph-mon[32751]: pgmap v9665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:18 smithi090 ceph-mon[29180]: pgmap v9665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:20 smithi119 ceph-mon[32752]: pgmap v9666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:20 smithi167 ceph-mon[32751]: pgmap v9666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:20 smithi090 ceph-mon[29180]: pgmap v9666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 16862.238622 secs 2024-04-26T23:16:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16863.006017 secs 2024-04-26T23:16:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 16862.238622 secs 2024-04-26T23:16:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16863.006017 secs 2024-04-26T23:16:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 16862.238622 secs 2024-04-26T23:16:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16863.006017 secs 2024-04-26T23:16:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:22 smithi119 ceph-mon[32752]: pgmap v9667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:22 smithi167 ceph-mon[32751]: pgmap v9667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:22 smithi090 ceph-mon[29180]: pgmap v9667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:24 smithi119 ceph-mon[32752]: pgmap v9668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:24 smithi167 ceph-mon[32751]: pgmap v9668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:24 smithi090 ceph-mon[29180]: pgmap v9668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 16867.238699 secs 2024-04-26T23:16:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:26 smithi119 ceph-mon[32752]: pgmap v9669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16868.006096 secs 2024-04-26T23:16:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 16867.238699 secs 2024-04-26T23:16:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:26 smithi167 ceph-mon[32751]: pgmap v9669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16868.006096 secs 2024-04-26T23:16:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 16867.238699 secs 2024-04-26T23:16:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:26 smithi090 ceph-mon[29180]: pgmap v9669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16868.006096 secs 2024-04-26T23:16:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:27 smithi119 ceph-mon[32752]: pgmap v9670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:27 smithi167 ceph-mon[32751]: pgmap v9670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:27 smithi090 ceph-mon[29180]: pgmap v9670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:30 smithi090 ceph-mon[29180]: pgmap v9671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:30 smithi119 ceph-mon[32752]: pgmap v9671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:30 smithi167 ceph-mon[32751]: pgmap v9671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 16872.238781 secs 2024-04-26T23:16:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16873.006181 secs 2024-04-26T23:16:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 16872.238781 secs 2024-04-26T23:16:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16873.006181 secs 2024-04-26T23:16:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 16872.238781 secs 2024-04-26T23:16:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16873.006181 secs 2024-04-26T23:16:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:32 smithi090 ceph-mon[29180]: pgmap v9672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:32 smithi119 ceph-mon[32752]: pgmap v9672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:32 smithi167 ceph-mon[32751]: pgmap v9672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:34 smithi119 ceph-mon[32752]: pgmap v9673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:34 smithi167 ceph-mon[32751]: pgmap v9673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:34 smithi090 ceph-mon[29180]: pgmap v9673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 16877.238864 secs 2024-04-26T23:16:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:36 smithi119 ceph-mon[32752]: pgmap v9674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16878.006247 secs 2024-04-26T23:16:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 16877.238864 secs 2024-04-26T23:16:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:36 smithi167 ceph-mon[32751]: pgmap v9674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16878.006247 secs 2024-04-26T23:16:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 16877.238864 secs 2024-04-26T23:16:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:36 smithi090 ceph-mon[29180]: pgmap v9674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16878.006247 secs 2024-04-26T23:16:38.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:38 smithi090 ceph-mon[29180]: pgmap v9675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:38 smithi119 ceph-mon[32752]: pgmap v9675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:38 smithi167 ceph-mon[32751]: pgmap v9675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:40 smithi119 ceph-mon[32752]: pgmap v9676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:40 smithi167 ceph-mon[32751]: pgmap v9676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:40 smithi090 ceph-mon[29180]: pgmap v9676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 16882.238925 secs 2024-04-26T23:16:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16883.006344 secs 2024-04-26T23:16:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 16882.238925 secs 2024-04-26T23:16:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16883.006344 secs 2024-04-26T23:16:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 16882.238925 secs 2024-04-26T23:16:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16883.006344 secs 2024-04-26T23:16:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:42 smithi119 ceph-mon[32752]: pgmap v9677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:42 smithi167 ceph-mon[32751]: pgmap v9677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:42 smithi090 ceph-mon[29180]: pgmap v9677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:44 smithi119 ceph-mon[32752]: pgmap v9678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:44 smithi167 ceph-mon[32751]: pgmap v9678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:44 smithi090 ceph-mon[29180]: pgmap v9678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:46 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 16887.239013 secs 2024-04-26T23:16:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:46 smithi119 ceph-mon[32752]: slow request 180.008344 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:16:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:46 smithi119 ceph-mon[32752]: pgmap v9679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16888.006472 secs 2024-04-26T23:16:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:46 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 16887.239013 secs 2024-04-26T23:16:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:46 smithi167 ceph-mon[32751]: slow request 180.008344 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:16:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:46 smithi167 ceph-mon[32751]: pgmap v9679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16888.006472 secs 2024-04-26T23:16:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:46 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 16887.239013 secs 2024-04-26T23:16:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:46 smithi090 ceph-mon[29180]: slow request 180.008344 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:16:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:46 smithi090 ceph-mon[29180]: pgmap v9679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16888.006472 secs 2024-04-26T23:16:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:16:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:16:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:16:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:16:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:16:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:16:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:48 smithi119 ceph-mon[32752]: pgmap v9680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:48 smithi167 ceph-mon[32751]: pgmap v9680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:48 smithi090 ceph-mon[29180]: pgmap v9680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:50 smithi119 ceph-mon[32752]: pgmap v9681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:50 smithi167 ceph-mon[32751]: pgmap v9681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:50 smithi090 ceph-mon[29180]: pgmap v9681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:16:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16892.239113 secs 2024-04-26T23:16:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16893.006589 secs 2024-04-26T23:16:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:16:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16892.239113 secs 2024-04-26T23:16:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16893.006589 secs 2024-04-26T23:16:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:16:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16892.239113 secs 2024-04-26T23:16:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16893.006589 secs 2024-04-26T23:16:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:16:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:52 smithi119 ceph-mon[32752]: pgmap v9682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:16:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:16:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:16:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:52 smithi167 ceph-mon[32751]: pgmap v9682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:16:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:16:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:16:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:52 smithi090 ceph-mon[29180]: pgmap v9682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:16:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:16:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:16:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:16:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:53 smithi119 ceph-mon[32752]: pgmap v9683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:16:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:53 smithi119 ceph-mon[32752]: pgmap v9684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:16:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:53 smithi167 ceph-mon[32751]: pgmap v9683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:16:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:53 smithi167 ceph-mon[32751]: pgmap v9684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:16:53.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:53 smithi090 ceph-mon[29180]: pgmap v9683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:16:53.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:53 smithi090 ceph-mon[29180]: pgmap v9684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:16:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:55 smithi119 ceph-mon[32752]: pgmap v9685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:55 smithi167 ceph-mon[32751]: pgmap v9685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:55 smithi090 ceph-mon[29180]: pgmap v9685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16897.239195 secs 2024-04-26T23:16:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16898.006717 secs 2024-04-26T23:16:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16897.239195 secs 2024-04-26T23:16:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16898.006717 secs 2024-04-26T23:16:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16897.239195 secs 2024-04-26T23:16:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16898.006717 secs 2024-04-26T23:16:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:57 smithi167 ceph-mon[32751]: pgmap v9686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:57 smithi090 ceph-mon[29180]: pgmap v9686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:57 smithi119 ceph-mon[32752]: pgmap v9686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:16:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:16:59 smithi167 ceph-mon[32751]: pgmap v9687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:16:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:16:59 smithi090 ceph-mon[29180]: pgmap v9687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:17:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:16:59 smithi119 ceph-mon[32752]: pgmap v9687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:17:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16902.239255 secs 2024-04-26T23:17:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:01 smithi167 ceph-mon[32751]: pgmap v9688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16903.006840 secs 2024-04-26T23:17:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16902.239255 secs 2024-04-26T23:17:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:01 smithi090 ceph-mon[29180]: pgmap v9688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16903.006840 secs 2024-04-26T23:17:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16902.239255 secs 2024-04-26T23:17:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:01 smithi119 ceph-mon[32752]: pgmap v9688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16903.006840 secs 2024-04-26T23:17:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:03 smithi167 ceph-mon[32751]: pgmap v9689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:17:03.966 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:03 smithi090 ceph-mon[29180]: pgmap v9689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:17:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:03 smithi119 ceph-mon[32752]: pgmap v9689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:17:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:05 smithi167 ceph-mon[32751]: pgmap v9690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:05 smithi090 ceph-mon[29180]: pgmap v9690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:05 smithi119 ceph-mon[32752]: pgmap v9690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16907.239337 secs 2024-04-26T23:17:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16908.006924 secs 2024-04-26T23:17:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16907.239337 secs 2024-04-26T23:17:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16908.006924 secs 2024-04-26T23:17:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16907.239337 secs 2024-04-26T23:17:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16908.006924 secs 2024-04-26T23:17:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:07 smithi090 ceph-mon[29180]: pgmap v9691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:07 smithi119 ceph-mon[32752]: pgmap v9691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:07 smithi167 ceph-mon[32751]: pgmap v9691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:09 smithi090 ceph-mon[29180]: pgmap v9692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:09 smithi119 ceph-mon[32752]: pgmap v9692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:09 smithi167 ceph-mon[32751]: pgmap v9692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16912.239417 secs 2024-04-26T23:17:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:11 smithi090 ceph-mon[29180]: pgmap v9693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16913.007017 secs 2024-04-26T23:17:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16912.239417 secs 2024-04-26T23:17:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:11 smithi119 ceph-mon[32752]: pgmap v9693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16913.007017 secs 2024-04-26T23:17:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16912.239417 secs 2024-04-26T23:17:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:11 smithi167 ceph-mon[32751]: pgmap v9693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16913.007017 secs 2024-04-26T23:17:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:13 smithi090 ceph-mon[29180]: pgmap v9694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:13 smithi119 ceph-mon[32752]: pgmap v9694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:13 smithi167 ceph-mon[32751]: pgmap v9694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:15 smithi090 ceph-mon[29180]: pgmap v9695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:15 smithi119 ceph-mon[32752]: pgmap v9695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:15 smithi167 ceph-mon[32751]: pgmap v9695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16917.239502 secs 2024-04-26T23:17:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16918.007147 secs 2024-04-26T23:17:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16917.239502 secs 2024-04-26T23:17:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16918.007147 secs 2024-04-26T23:17:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16917.239502 secs 2024-04-26T23:17:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16918.007147 secs 2024-04-26T23:17:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:17 smithi090 ceph-mon[29180]: pgmap v9696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:17 smithi119 ceph-mon[32752]: pgmap v9696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:17 smithi167 ceph-mon[32751]: pgmap v9696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:19 smithi090 ceph-mon[29180]: pgmap v9697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:19 smithi119 ceph-mon[32752]: pgmap v9697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:19 smithi167 ceph-mon[32751]: pgmap v9697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16922.239586 secs 2024-04-26T23:17:21.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:21 smithi167 ceph-mon[32751]: pgmap v9698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:21.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16923.007268 secs 2024-04-26T23:17:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16922.239586 secs 2024-04-26T23:17:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:21 smithi090 ceph-mon[29180]: pgmap v9698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16923.007268 secs 2024-04-26T23:17:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16922.239586 secs 2024-04-26T23:17:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:21 smithi119 ceph-mon[32752]: pgmap v9698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16923.007268 secs 2024-04-26T23:17:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:23 smithi090 ceph-mon[29180]: pgmap v9699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:23 smithi119 ceph-mon[32752]: pgmap v9699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:23 smithi167 ceph-mon[32751]: pgmap v9699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:25 smithi090 ceph-mon[29180]: pgmap v9700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:25 smithi119 ceph-mon[32752]: pgmap v9700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:25 smithi167 ceph-mon[32751]: pgmap v9700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16927.239690 secs 2024-04-26T23:17:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16928.007414 secs 2024-04-26T23:17:26.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16927.239690 secs 2024-04-26T23:17:26.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16928.007414 secs 2024-04-26T23:17:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16927.239690 secs 2024-04-26T23:17:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16928.007414 secs 2024-04-26T23:17:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:27 smithi090 ceph-mon[29180]: pgmap v9701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:27 smithi119 ceph-mon[32752]: pgmap v9701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:27 smithi167 ceph-mon[32751]: pgmap v9701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:29 smithi090 ceph-mon[29180]: pgmap v9702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:29 smithi119 ceph-mon[32752]: pgmap v9702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:29 smithi167 ceph-mon[32751]: pgmap v9702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16932.239773 secs 2024-04-26T23:17:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:31 smithi119 ceph-mon[32752]: pgmap v9703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16933.007475 secs 2024-04-26T23:17:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16932.239773 secs 2024-04-26T23:17:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:31 smithi167 ceph-mon[32751]: pgmap v9703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16933.007475 secs 2024-04-26T23:17:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16932.239773 secs 2024-04-26T23:17:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:31 smithi090 ceph-mon[29180]: pgmap v9703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16933.007475 secs 2024-04-26T23:17:33.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:33 smithi090 ceph-mon[29180]: pgmap v9704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:33 smithi119 ceph-mon[32752]: pgmap v9704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:33 smithi167 ceph-mon[32751]: pgmap v9704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:35 smithi119 ceph-mon[32752]: pgmap v9705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:35 smithi167 ceph-mon[32751]: pgmap v9705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:35 smithi090 ceph-mon[29180]: pgmap v9705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16937.239861 secs 2024-04-26T23:17:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16938.007547 secs 2024-04-26T23:17:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16937.239861 secs 2024-04-26T23:17:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16938.007547 secs 2024-04-26T23:17:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16937.239861 secs 2024-04-26T23:17:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16938.007547 secs 2024-04-26T23:17:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:37 smithi119 ceph-mon[32752]: pgmap v9706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:37 smithi167 ceph-mon[32751]: pgmap v9706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:37 smithi090 ceph-mon[29180]: pgmap v9706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:39 smithi119 ceph-mon[32752]: pgmap v9707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:39 smithi167 ceph-mon[32751]: pgmap v9707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:39 smithi090 ceph-mon[29180]: pgmap v9707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16942.239952 secs 2024-04-26T23:17:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:41 smithi119 ceph-mon[32752]: pgmap v9708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16943.007620 secs 2024-04-26T23:17:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16942.239952 secs 2024-04-26T23:17:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:41 smithi167 ceph-mon[32751]: pgmap v9708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16943.007620 secs 2024-04-26T23:17:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16942.239952 secs 2024-04-26T23:17:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:41 smithi090 ceph-mon[29180]: pgmap v9708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16943.007620 secs 2024-04-26T23:17:44.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:43 smithi119 ceph-mon[32752]: pgmap v9709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:43 smithi167 ceph-mon[32751]: pgmap v9709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:43 smithi090 ceph-mon[29180]: pgmap v9709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:45 smithi119 ceph-mon[32752]: pgmap v9710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:45 smithi167 ceph-mon[32751]: pgmap v9710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:45 smithi090 ceph-mon[29180]: pgmap v9710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:46 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 16947.240014 secs 2024-04-26T23:17:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:46 smithi119 ceph-mon[32752]: slow request 5763.538731 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:17:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16948.007738 secs 2024-04-26T23:17:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:46 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 16947.240014 secs 2024-04-26T23:17:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:46 smithi167 ceph-mon[32751]: slow request 5763.538731 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:17:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16948.007738 secs 2024-04-26T23:17:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:46 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 16947.240014 secs 2024-04-26T23:17:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:46 smithi090 ceph-mon[29180]: slow request 5763.538731 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:17:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16948.007738 secs 2024-04-26T23:17:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:47 smithi119 ceph-mon[32752]: pgmap v9711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:47 smithi167 ceph-mon[32751]: pgmap v9711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:47 smithi090 ceph-mon[29180]: pgmap v9711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:49 smithi119 ceph-mon[32752]: pgmap v9712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:49 smithi167 ceph-mon[32751]: pgmap v9712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:50.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:49 smithi090 ceph-mon[29180]: pgmap v9712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16952.240104 secs 2024-04-26T23:17:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:51 smithi119 ceph-mon[32752]: pgmap v9713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16953.007834 secs 2024-04-26T23:17:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16952.240104 secs 2024-04-26T23:17:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:51 smithi167 ceph-mon[32751]: pgmap v9713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16953.007834 secs 2024-04-26T23:17:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16952.240104 secs 2024-04-26T23:17:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:51 smithi090 ceph-mon[29180]: pgmap v9713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:17:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16953.007834 secs 2024-04-26T23:17:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:17:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:17:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:17:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:53 smithi119 ceph-mon[32752]: pgmap v9714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:17:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:17:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:17:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:53 smithi167 ceph-mon[32751]: pgmap v9714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:17:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:17:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:17:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:53 smithi090 ceph-mon[29180]: pgmap v9714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:17:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:17:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:17:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:17:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:54 smithi119 ceph-mon[32752]: pgmap v9715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:17:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:54 smithi119 ceph-mon[32752]: pgmap v9716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:17:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:54 smithi167 ceph-mon[32751]: pgmap v9715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:17:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:54 smithi167 ceph-mon[32751]: pgmap v9716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:17:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:54 smithi090 ceph-mon[29180]: pgmap v9715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:17:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:54 smithi090 ceph-mon[29180]: pgmap v9716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:17:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:56 smithi119 ceph-mon[32752]: pgmap v9717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16957.240165 secs 2024-04-26T23:17:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16958.007915 secs 2024-04-26T23:17:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:56 smithi167 ceph-mon[32751]: pgmap v9717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16957.240165 secs 2024-04-26T23:17:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16958.007915 secs 2024-04-26T23:17:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:56 smithi090 ceph-mon[29180]: pgmap v9717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16957.240165 secs 2024-04-26T23:17:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16958.007915 secs 2024-04-26T23:17:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:17:58 smithi119 ceph-mon[32752]: pgmap v9718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:17:58 smithi167 ceph-mon[32751]: pgmap v9718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:17:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:17:58 smithi090 ceph-mon[29180]: pgmap v9718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:18:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:00 smithi119 ceph-mon[32752]: pgmap v9719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:18:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:00 smithi167 ceph-mon[32751]: pgmap v9719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:18:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:00 smithi090 ceph-mon[29180]: pgmap v9719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:18:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16962.240266 secs 2024-04-26T23:18:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16963.007995 secs 2024-04-26T23:18:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16962.240266 secs 2024-04-26T23:18:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16963.007995 secs 2024-04-26T23:18:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16962.240266 secs 2024-04-26T23:18:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16963.007995 secs 2024-04-26T23:18:03.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:02 smithi090 ceph-mon[29180]: pgmap v9720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:18:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:02 smithi119 ceph-mon[32752]: pgmap v9720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:18:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:02 smithi167 ceph-mon[32751]: pgmap v9720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:18:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:04 smithi119 ceph-mon[32752]: pgmap v9721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:18:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:04 smithi167 ceph-mon[32751]: pgmap v9721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:18:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:04 smithi090 ceph-mon[29180]: pgmap v9721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:18:05.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:18:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:18:05.430+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:18:05.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:18:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:18:05.430+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:18:05.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:18:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:18:05.430+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:18:05.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:18:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:18:05.430+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:18:05.877 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:18:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:18:05.430+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:18:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:06 smithi119 ceph-mon[32752]: pgmap v9722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16967.240350 secs 2024-04-26T23:18:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16968.008115 secs 2024-04-26T23:18:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:06 smithi167 ceph-mon[32751]: pgmap v9722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16967.240350 secs 2024-04-26T23:18:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16968.008115 secs 2024-04-26T23:18:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:06 smithi090 ceph-mon[29180]: pgmap v9722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16967.240350 secs 2024-04-26T23:18:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16968.008115 secs 2024-04-26T23:18:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:08 smithi119 ceph-mon[32752]: pgmap v9723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:08 smithi167 ceph-mon[32751]: pgmap v9723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:08 smithi090 ceph-mon[29180]: pgmap v9723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:10 smithi119 ceph-mon[32752]: pgmap v9724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:10 smithi167 ceph-mon[32751]: pgmap v9724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:10 smithi090 ceph-mon[29180]: pgmap v9724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:12.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16972.240435 secs 2024-04-26T23:18:12.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16973.008264 secs 2024-04-26T23:18:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16972.240435 secs 2024-04-26T23:18:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16973.008264 secs 2024-04-26T23:18:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16972.240435 secs 2024-04-26T23:18:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16973.008264 secs 2024-04-26T23:18:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:12 smithi119 ceph-mon[32752]: pgmap v9725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:12 smithi167 ceph-mon[32751]: pgmap v9725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:12 smithi090 ceph-mon[29180]: pgmap v9725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:14 smithi119 ceph-mon[32752]: pgmap v9726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:14 smithi167 ceph-mon[32751]: pgmap v9726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:14 smithi090 ceph-mon[29180]: pgmap v9726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:16 smithi119 ceph-mon[32752]: pgmap v9727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16977.240495 secs 2024-04-26T23:18:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16978.008400 secs 2024-04-26T23:18:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:16 smithi167 ceph-mon[32751]: pgmap v9727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16977.240495 secs 2024-04-26T23:18:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16978.008400 secs 2024-04-26T23:18:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:16 smithi090 ceph-mon[29180]: pgmap v9727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16977.240495 secs 2024-04-26T23:18:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16978.008400 secs 2024-04-26T23:18:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:18 smithi119 ceph-mon[32752]: pgmap v9728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:18 smithi167 ceph-mon[32751]: pgmap v9728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:18 smithi090 ceph-mon[29180]: pgmap v9728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-26T23:18:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:20 smithi119 ceph-mon[32752]: pgmap v9729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:20 smithi167 ceph-mon[32751]: pgmap v9729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:20 smithi090 ceph-mon[29180]: pgmap v9729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-26T23:18:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16982.240578 secs 2024-04-26T23:18:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16983.008529 secs 2024-04-26T23:18:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16982.240578 secs 2024-04-26T23:18:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16983.008529 secs 2024-04-26T23:18:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16982.240578 secs 2024-04-26T23:18:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16983.008529 secs 2024-04-26T23:18:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:22 smithi119 ceph-mon[32752]: pgmap v9730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:22 smithi167 ceph-mon[32751]: pgmap v9730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:22 smithi090 ceph-mon[29180]: pgmap v9730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:24 smithi119 ceph-mon[32752]: pgmap v9731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:24 smithi167 ceph-mon[32751]: pgmap v9731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:24 smithi090 ceph-mon[29180]: pgmap v9731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:26 smithi119 ceph-mon[32752]: pgmap v9732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16987.240636 secs 2024-04-26T23:18:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16988.008683 secs 2024-04-26T23:18:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:26 smithi167 ceph-mon[32751]: pgmap v9732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16987.240636 secs 2024-04-26T23:18:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16988.008683 secs 2024-04-26T23:18:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:26 smithi090 ceph-mon[29180]: pgmap v9732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16987.240636 secs 2024-04-26T23:18:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16988.008683 secs 2024-04-26T23:18:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:28 smithi119 ceph-mon[32752]: pgmap v9733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:28 smithi167 ceph-mon[32751]: pgmap v9733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:28 smithi090 ceph-mon[29180]: pgmap v9733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:30 smithi119 ceph-mon[32752]: pgmap v9734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:30 smithi167 ceph-mon[32751]: pgmap v9734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:30 smithi090 ceph-mon[29180]: pgmap v9734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16992.240718 secs 2024-04-26T23:18:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16993.008758 secs 2024-04-26T23:18:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16992.240718 secs 2024-04-26T23:18:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16993.008758 secs 2024-04-26T23:18:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16992.240718 secs 2024-04-26T23:18:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16993.008758 secs 2024-04-26T23:18:33.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:32 smithi090 ceph-mon[29180]: pgmap v9735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:32 smithi119 ceph-mon[32752]: pgmap v9735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:32 smithi167 ceph-mon[32751]: pgmap v9735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:33 smithi090 ceph-mon[29180]: pgmap v9736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:33 smithi119 ceph-mon[32752]: pgmap v9736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:33 smithi167 ceph-mon[32751]: pgmap v9736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:35 smithi119 ceph-mon[32752]: pgmap v9737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:35 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 16997.240802 secs 2024-04-26T23:18:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:35 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 16998.008883 secs 2024-04-26T23:18:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:35 smithi167 ceph-mon[32751]: pgmap v9737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:35 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 16997.240802 secs 2024-04-26T23:18:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:35 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 16998.008883 secs 2024-04-26T23:18:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:35 smithi090 ceph-mon[29180]: pgmap v9737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:35 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 16997.240802 secs 2024-04-26T23:18:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:35 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 16998.008883 secs 2024-04-26T23:18:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:37 smithi119 ceph-mon[32752]: pgmap v9738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:37 smithi167 ceph-mon[32751]: pgmap v9738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:37 smithi090 ceph-mon[29180]: pgmap v9738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:39 smithi119 ceph-mon[32752]: pgmap v9739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:39 smithi167 ceph-mon[32751]: pgmap v9739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:39 smithi090 ceph-mon[29180]: pgmap v9739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:40 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17002.240888 secs 2024-04-26T23:18:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:40 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17003.009004 secs 2024-04-26T23:18:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:40 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17002.240888 secs 2024-04-26T23:18:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:40 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17003.009004 secs 2024-04-26T23:18:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:40 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17002.240888 secs 2024-04-26T23:18:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:40 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17003.009004 secs 2024-04-26T23:18:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:41 smithi119 ceph-mon[32752]: pgmap v9740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:41 smithi167 ceph-mon[32751]: pgmap v9740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:41 smithi090 ceph-mon[29180]: pgmap v9740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:43 smithi119 ceph-mon[32752]: pgmap v9741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:43 smithi167 ceph-mon[32751]: pgmap v9741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:43 smithi090 ceph-mon[29180]: pgmap v9741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:45 smithi119 ceph-mon[32752]: pgmap v9742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:45 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17007.240978 secs 2024-04-26T23:18:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:45 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17008.009092 secs 2024-04-26T23:18:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:45 smithi167 ceph-mon[32751]: pgmap v9742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:45 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17007.240978 secs 2024-04-26T23:18:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:45 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17008.009092 secs 2024-04-26T23:18:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:45 smithi090 ceph-mon[29180]: pgmap v9742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:45 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17007.240978 secs 2024-04-26T23:18:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:45 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17008.009092 secs 2024-04-26T23:18:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:47 smithi119 ceph-mon[32752]: pgmap v9743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:47 smithi167 ceph-mon[32751]: pgmap v9743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:47 smithi090 ceph-mon[29180]: pgmap v9743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:49 smithi119 ceph-mon[32752]: pgmap v9744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:49 smithi167 ceph-mon[32751]: pgmap v9744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:49 smithi090 ceph-mon[29180]: pgmap v9744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:51.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:50 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17012.241067 secs 2024-04-26T23:18:51.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:50 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17013.009214 secs 2024-04-26T23:18:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:50 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17012.241067 secs 2024-04-26T23:18:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:50 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17013.009214 secs 2024-04-26T23:18:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:50 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17012.241067 secs 2024-04-26T23:18:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:50 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17013.009214 secs 2024-04-26T23:18:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:51 smithi119 ceph-mon[32752]: pgmap v9745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:51 smithi167 ceph-mon[32751]: pgmap v9745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:51 smithi090 ceph-mon[29180]: pgmap v9745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:53 smithi167 ceph-mon[32751]: pgmap v9746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:18:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:53 smithi090 ceph-mon[29180]: pgmap v9746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:54.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:18:54.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.296 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:53 smithi119 ceph-mon[32752]: pgmap v9746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:18:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: pgmap v9747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17017.241133 secs 2024-04-26T23:18:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17018.009307 secs 2024-04-26T23:18:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: pgmap v9747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17017.241133 secs 2024-04-26T23:18:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17018.009307 secs 2024-04-26T23:18:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: pgmap v9747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:18:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17017.241133 secs 2024-04-26T23:18:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17018.009307 secs 2024-04-26T23:18:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:18:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:18:57.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:18:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:18:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:18:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:18:57.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:18:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:18:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:58 smithi119 ceph-mon[32752]: pgmap v9748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:58 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:18:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:58 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:18:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:58 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:18:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:58 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:18:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:58 smithi119 ceph-mon[32752]: pgmap v9749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 B/s rd, 1 op/s 2024-04-26T23:18:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:18:58 smithi119 ceph-mon[32752]: pgmap v9750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-26T23:18:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:58 smithi167 ceph-mon[32751]: pgmap v9748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:58 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:18:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:58 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:18:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:58 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:18:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:58 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:18:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:58 smithi167 ceph-mon[32751]: pgmap v9749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 B/s rd, 1 op/s 2024-04-26T23:18:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:18:58 smithi167 ceph-mon[32751]: pgmap v9750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-26T23:18:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:58 smithi090 ceph-mon[29180]: pgmap v9748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:18:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:58 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:18:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:58 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:18:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:58 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:18:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:58 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:18:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:58 smithi090 ceph-mon[29180]: pgmap v9749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 B/s rd, 1 op/s 2024-04-26T23:18:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:18:58 smithi090 ceph-mon[29180]: pgmap v9750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-26T23:19:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:00 smithi119 ceph-mon[32752]: pgmap v9751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:00 smithi167 ceph-mon[32751]: pgmap v9751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:00 smithi090 ceph-mon[29180]: pgmap v9751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17022.241217 secs 2024-04-26T23:19:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17023.009390 secs 2024-04-26T23:19:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17022.241217 secs 2024-04-26T23:19:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17023.009390 secs 2024-04-26T23:19:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17022.241217 secs 2024-04-26T23:19:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17023.009390 secs 2024-04-26T23:19:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:02 smithi119 ceph-mon[32752]: pgmap v9752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:02 smithi167 ceph-mon[32751]: pgmap v9752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:02 smithi090 ceph-mon[29180]: pgmap v9752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:04 smithi167 ceph-mon[32751]: pgmap v9753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-26T23:19:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:04 smithi090 ceph-mon[29180]: pgmap v9753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-26T23:19:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:04 smithi119 ceph-mon[32752]: pgmap v9753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-26T23:19:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:06 smithi167 ceph-mon[32751]: pgmap v9754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T23:19:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17027.241303 secs 2024-04-26T23:19:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:06 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 17028.009484 secs 2024-04-26T23:19:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:06 smithi167 ceph-mon[32751]: slow request 2881.167344 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:19:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:06 smithi090 ceph-mon[29180]: pgmap v9754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T23:19:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17027.241303 secs 2024-04-26T23:19:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:06 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 17028.009484 secs 2024-04-26T23:19:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:06 smithi090 ceph-mon[29180]: slow request 2881.167344 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:19:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:06 smithi119 ceph-mon[32752]: pgmap v9754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-26T23:19:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17027.241303 secs 2024-04-26T23:19:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:06 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 17028.009484 secs 2024-04-26T23:19:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:06 smithi119 ceph-mon[32752]: slow request 2881.167344 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:19:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:08 smithi167 ceph-mon[32751]: pgmap v9755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:08 smithi090 ceph-mon[29180]: pgmap v9755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:08 smithi119 ceph-mon[32752]: pgmap v9755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:10 smithi167 ceph-mon[32751]: pgmap v9756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:10 smithi090 ceph-mon[29180]: pgmap v9756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:10 smithi119 ceph-mon[32752]: pgmap v9756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:11 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17032.241362 secs 2024-04-26T23:19:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:11 smithi090 ceph-mon[29180]: slow request 364.679990 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:19:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17033.009584 secs 2024-04-26T23:19:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:11 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17032.241362 secs 2024-04-26T23:19:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:11 smithi119 ceph-mon[32752]: slow request 364.679990 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:19:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17033.009584 secs 2024-04-26T23:19:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:11 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17032.241362 secs 2024-04-26T23:19:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:11 smithi167 ceph-mon[32751]: slow request 364.679990 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:19:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17033.009584 secs 2024-04-26T23:19:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:12 smithi090 ceph-mon[29180]: pgmap v9757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:19:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:12 smithi119 ceph-mon[32752]: pgmap v9757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:19:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:12 smithi167 ceph-mon[32751]: pgmap v9757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:19:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:14 smithi090 ceph-mon[29180]: pgmap v9758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:19:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:14 smithi119 ceph-mon[32752]: pgmap v9758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:19:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:14 smithi167 ceph-mon[32751]: pgmap v9758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:19:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:16 smithi090 ceph-mon[29180]: pgmap v9759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17037.241447 secs 2024-04-26T23:19:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17038.009701 secs 2024-04-26T23:19:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:16 smithi119 ceph-mon[32752]: pgmap v9759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17037.241447 secs 2024-04-26T23:19:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17038.009701 secs 2024-04-26T23:19:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:16 smithi167 ceph-mon[32751]: pgmap v9759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:19:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17037.241447 secs 2024-04-26T23:19:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17038.009701 secs 2024-04-26T23:19:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:18 smithi090 ceph-mon[29180]: pgmap v9760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:18 smithi119 ceph-mon[32752]: pgmap v9760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:18 smithi167 ceph-mon[32751]: pgmap v9760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:20 smithi090 ceph-mon[29180]: pgmap v9761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:20 smithi119 ceph-mon[32752]: pgmap v9761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:20 smithi167 ceph-mon[32751]: pgmap v9761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17042.241533 secs 2024-04-26T23:19:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17043.009834 secs 2024-04-26T23:19:21.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17042.241533 secs 2024-04-26T23:19:21.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17043.009834 secs 2024-04-26T23:19:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17042.241533 secs 2024-04-26T23:19:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17043.009834 secs 2024-04-26T23:19:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:22 smithi090 ceph-mon[29180]: pgmap v9762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:22 smithi119 ceph-mon[32752]: pgmap v9762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:22 smithi167 ceph-mon[32751]: pgmap v9762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:24 smithi090 ceph-mon[29180]: pgmap v9763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:24 smithi119 ceph-mon[32752]: pgmap v9763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:24 smithi167 ceph-mon[32751]: pgmap v9763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:26.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:26 smithi119 ceph-mon[32752]: pgmap v9764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:26.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17047.241618 secs 2024-04-26T23:19:26.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17048.009950 secs 2024-04-26T23:19:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:26 smithi090 ceph-mon[29180]: pgmap v9764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17047.241618 secs 2024-04-26T23:19:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17048.009950 secs 2024-04-26T23:19:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:26 smithi167 ceph-mon[32751]: pgmap v9764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17047.241618 secs 2024-04-26T23:19:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17048.009950 secs 2024-04-26T23:19:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:28 smithi090 ceph-mon[29180]: pgmap v9765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:28 smithi119 ceph-mon[32752]: pgmap v9765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:28 smithi167 ceph-mon[32751]: pgmap v9765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:30 smithi090 ceph-mon[29180]: pgmap v9766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:30 smithi119 ceph-mon[32752]: pgmap v9766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:30 smithi167 ceph-mon[32751]: pgmap v9766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17052.241702 secs 2024-04-26T23:19:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17053.010100 secs 2024-04-26T23:19:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17052.241702 secs 2024-04-26T23:19:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17053.010100 secs 2024-04-26T23:19:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17052.241702 secs 2024-04-26T23:19:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17053.010100 secs 2024-04-26T23:19:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:32 smithi090 ceph-mon[29180]: pgmap v9767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:32 smithi119 ceph-mon[32752]: pgmap v9767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:32 smithi167 ceph-mon[32751]: pgmap v9767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:34 smithi090 ceph-mon[29180]: pgmap v9768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:34 smithi119 ceph-mon[32752]: pgmap v9768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:34 smithi167 ceph-mon[32751]: pgmap v9768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:36 smithi090 ceph-mon[29180]: pgmap v9769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17057.241811 secs 2024-04-26T23:19:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17058.010201 secs 2024-04-26T23:19:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:36 smithi119 ceph-mon[32752]: pgmap v9769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17057.241811 secs 2024-04-26T23:19:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17058.010201 secs 2024-04-26T23:19:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:36 smithi167 ceph-mon[32751]: pgmap v9769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17057.241811 secs 2024-04-26T23:19:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17058.010201 secs 2024-04-26T23:19:38.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:38 smithi090 ceph-mon[29180]: pgmap v9770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:38 smithi119 ceph-mon[32752]: pgmap v9770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:38 smithi167 ceph-mon[32751]: pgmap v9770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:40 smithi090 ceph-mon[29180]: pgmap v9771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:40 smithi119 ceph-mon[32752]: pgmap v9771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:40 smithi167 ceph-mon[32751]: pgmap v9771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17062.241878 secs 2024-04-26T23:19:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17063.010341 secs 2024-04-26T23:19:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17062.241878 secs 2024-04-26T23:19:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17063.010341 secs 2024-04-26T23:19:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17062.241878 secs 2024-04-26T23:19:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17063.010341 secs 2024-04-26T23:19:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:42 smithi119 ceph-mon[32752]: pgmap v9772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:42 smithi167 ceph-mon[32751]: pgmap v9772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:42 smithi090 ceph-mon[29180]: pgmap v9772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:44 smithi119 ceph-mon[32752]: pgmap v9773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:44 smithi167 ceph-mon[32751]: pgmap v9773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:44.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:44 smithi090 ceph-mon[29180]: pgmap v9773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:46 smithi119 ceph-mon[32752]: pgmap v9774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:46 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17067.241941 secs 2024-04-26T23:19:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:46 smithi119 ceph-mon[32752]: slow request 360.011273 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:19:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17068.010462 secs 2024-04-26T23:19:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:46 smithi167 ceph-mon[32751]: pgmap v9774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:46 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17067.241941 secs 2024-04-26T23:19:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:46 smithi167 ceph-mon[32751]: slow request 360.011273 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:19:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17068.010462 secs 2024-04-26T23:19:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:46 smithi090 ceph-mon[29180]: pgmap v9774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:46 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17067.241941 secs 2024-04-26T23:19:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:46 smithi090 ceph-mon[29180]: slow request 360.011273 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:19:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17068.010462 secs 2024-04-26T23:19:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:48 smithi167 ceph-mon[32751]: pgmap v9775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:48 smithi090 ceph-mon[29180]: pgmap v9775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:48 smithi119 ceph-mon[32752]: pgmap v9775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:50 smithi167 ceph-mon[32751]: pgmap v9776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:50 smithi090 ceph-mon[29180]: pgmap v9776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:50 smithi119 ceph-mon[32752]: pgmap v9776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17072.242002 secs 2024-04-26T23:19:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17073.010586 secs 2024-04-26T23:19:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17072.242002 secs 2024-04-26T23:19:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17073.010586 secs 2024-04-26T23:19:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17072.242002 secs 2024-04-26T23:19:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17073.010586 secs 2024-04-26T23:19:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:52 smithi167 ceph-mon[32751]: pgmap v9777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:52 smithi090 ceph-mon[29180]: pgmap v9777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:52 smithi119 ceph-mon[32752]: pgmap v9777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:54 smithi167 ceph-mon[32751]: pgmap v9778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:54 smithi090 ceph-mon[29180]: pgmap v9778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:54 smithi119 ceph-mon[32752]: pgmap v9778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:56.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:56 smithi119 ceph-mon[32752]: pgmap v9779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:56.857 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17077.242112 secs 2024-04-26T23:19:56.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17078.010708 secs 2024-04-26T23:19:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:56 smithi090 ceph-mon[29180]: pgmap v9779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17077.242112 secs 2024-04-26T23:19:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17078.010708 secs 2024-04-26T23:19:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:56 smithi167 ceph-mon[32751]: pgmap v9779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:19:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17077.242112 secs 2024-04-26T23:19:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17078.010708 secs 2024-04-26T23:19:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:19:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:19:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:19:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:58 smithi090 ceph-mon[29180]: pgmap v9780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:19:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:19:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:58 smithi119 ceph-mon[32752]: pgmap v9780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:19:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:19:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:58 smithi167 ceph-mon[32751]: pgmap v9780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:19:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:19:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:19:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:19:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:59 smithi090 ceph-mon[29180]: pgmap v9781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T23:19:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:19:59 smithi090 ceph-mon[29180]: pgmap v9782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T23:20:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:59 smithi119 ceph-mon[32752]: pgmap v9781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T23:20:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:19:59 smithi119 ceph-mon[32752]: pgmap v9782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T23:20:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:59 smithi167 ceph-mon[32751]: pgmap v9781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T23:20:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:19:59 smithi167 ceph-mon[32751]: pgmap v9782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T23:20:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:20:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:20:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:20:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17082.242173 secs 2024-04-26T23:20:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:01 smithi090 ceph-mon[29180]: pgmap v9783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17083.010788 secs 2024-04-26T23:20:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17082.242173 secs 2024-04-26T23:20:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:01 smithi119 ceph-mon[32752]: pgmap v9783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17083.010788 secs 2024-04-26T23:20:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17082.242173 secs 2024-04-26T23:20:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:01 smithi167 ceph-mon[32751]: pgmap v9783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17083.010788 secs 2024-04-26T23:20:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:03 smithi090 ceph-mon[29180]: pgmap v9784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:03 smithi119 ceph-mon[32752]: pgmap v9784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:03 smithi167 ceph-mon[32751]: pgmap v9784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:05 smithi090 ceph-mon[29180]: pgmap v9785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:05 smithi119 ceph-mon[32752]: pgmap v9785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:05 smithi167 ceph-mon[32751]: pgmap v9785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17087.242234 secs 2024-04-26T23:20:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17088.010911 secs 2024-04-26T23:20:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17087.242234 secs 2024-04-26T23:20:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17088.010911 secs 2024-04-26T23:20:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17087.242234 secs 2024-04-26T23:20:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17088.010911 secs 2024-04-26T23:20:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:07 smithi119 ceph-mon[32752]: pgmap v9786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:07 smithi167 ceph-mon[32751]: pgmap v9786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:07 smithi090 ceph-mon[29180]: pgmap v9786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:20:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:09 smithi119 ceph-mon[32752]: pgmap v9787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:20:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:09 smithi167 ceph-mon[32751]: pgmap v9787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:20:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:09 smithi090 ceph-mon[29180]: pgmap v9787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:20:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:11 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17092.242295 secs 2024-04-26T23:20:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:11 smithi119 ceph-mon[32752]: slow request 724.748308 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:20:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:11 smithi119 ceph-mon[32752]: pgmap v9788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17093.011011 secs 2024-04-26T23:20:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:11 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17092.242295 secs 2024-04-26T23:20:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:11 smithi167 ceph-mon[32751]: slow request 724.748308 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:20:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:11 smithi167 ceph-mon[32751]: pgmap v9788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17093.011011 secs 2024-04-26T23:20:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:11 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17092.242295 secs 2024-04-26T23:20:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:11 smithi090 ceph-mon[29180]: slow request 724.748308 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:20:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:11 smithi090 ceph-mon[29180]: pgmap v9788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17093.011011 secs 2024-04-26T23:20:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:13 smithi119 ceph-mon[32752]: pgmap v9789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:13 smithi167 ceph-mon[32751]: pgmap v9789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:13 smithi090 ceph-mon[29180]: pgmap v9789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:15 smithi119 ceph-mon[32752]: pgmap v9790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:15 smithi167 ceph-mon[32751]: pgmap v9790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:15 smithi090 ceph-mon[29180]: pgmap v9790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17097.242378 secs 2024-04-26T23:20:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17098.011094 secs 2024-04-26T23:20:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17097.242378 secs 2024-04-26T23:20:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17098.011094 secs 2024-04-26T23:20:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17097.242378 secs 2024-04-26T23:20:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17098.011094 secs 2024-04-26T23:20:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:17 smithi119 ceph-mon[32752]: pgmap v9791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:17 smithi167 ceph-mon[32751]: pgmap v9791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:17 smithi090 ceph-mon[29180]: pgmap v9791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:19 smithi119 ceph-mon[32752]: pgmap v9792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:20.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:19 smithi167 ceph-mon[32751]: pgmap v9792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:19 smithi090 ceph-mon[29180]: pgmap v9792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17102.242460 secs 2024-04-26T23:20:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:21 smithi119 ceph-mon[32752]: pgmap v9793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17103.011235 secs 2024-04-26T23:20:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17102.242460 secs 2024-04-26T23:20:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:21 smithi167 ceph-mon[32751]: pgmap v9793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17103.011235 secs 2024-04-26T23:20:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17102.242460 secs 2024-04-26T23:20:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:21 smithi090 ceph-mon[29180]: pgmap v9793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17103.011235 secs 2024-04-26T23:20:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:23 smithi119 ceph-mon[32752]: pgmap v9794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:23 smithi167 ceph-mon[32751]: pgmap v9794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:23 smithi090 ceph-mon[29180]: pgmap v9794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:25 smithi119 ceph-mon[32752]: pgmap v9795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:25 smithi167 ceph-mon[32751]: pgmap v9795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:25 smithi090 ceph-mon[29180]: pgmap v9795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:26.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17107.242522 secs 2024-04-26T23:20:26.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17108.011358 secs 2024-04-26T23:20:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17107.242522 secs 2024-04-26T23:20:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17108.011358 secs 2024-04-26T23:20:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17107.242522 secs 2024-04-26T23:20:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17108.011358 secs 2024-04-26T23:20:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:27 smithi119 ceph-mon[32752]: pgmap v9796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:27 smithi167 ceph-mon[32751]: pgmap v9796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:27 smithi090 ceph-mon[29180]: pgmap v9796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:29 smithi090 ceph-mon[29180]: pgmap v9797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:29 smithi119 ceph-mon[32752]: pgmap v9797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:29 smithi167 ceph-mon[32751]: pgmap v9797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17112.242604 secs 2024-04-26T23:20:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:31 smithi090 ceph-mon[29180]: pgmap v9798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17113.011472 secs 2024-04-26T23:20:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17112.242604 secs 2024-04-26T23:20:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:31 smithi119 ceph-mon[32752]: pgmap v9798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17113.011472 secs 2024-04-26T23:20:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17112.242604 secs 2024-04-26T23:20:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:31 smithi167 ceph-mon[32751]: pgmap v9798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17113.011472 secs 2024-04-26T23:20:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:33 smithi119 ceph-mon[32752]: pgmap v9799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:33 smithi167 ceph-mon[32751]: pgmap v9799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:34.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:33 smithi090 ceph-mon[29180]: pgmap v9799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:35 smithi119 ceph-mon[32752]: pgmap v9800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:35 smithi167 ceph-mon[32751]: pgmap v9800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:35 smithi090 ceph-mon[29180]: pgmap v9800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17117.242690 secs 2024-04-26T23:20:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17118.011593 secs 2024-04-26T23:20:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17117.242690 secs 2024-04-26T23:20:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17118.011593 secs 2024-04-26T23:20:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17117.242690 secs 2024-04-26T23:20:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17118.011593 secs 2024-04-26T23:20:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:37 smithi119 ceph-mon[32752]: pgmap v9801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:37 smithi167 ceph-mon[32751]: pgmap v9801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:37 smithi090 ceph-mon[29180]: pgmap v9801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:39 smithi119 ceph-mon[32752]: pgmap v9802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:39 smithi167 ceph-mon[32751]: pgmap v9802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:39 smithi090 ceph-mon[29180]: pgmap v9802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:20:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17122.242746 secs 2024-04-26T23:20:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:41 smithi119 ceph-mon[32752]: pgmap v9803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17123.011711 secs 2024-04-26T23:20:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17122.242746 secs 2024-04-26T23:20:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:41 smithi167 ceph-mon[32751]: pgmap v9803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17123.011711 secs 2024-04-26T23:20:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17122.242746 secs 2024-04-26T23:20:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:41 smithi090 ceph-mon[29180]: pgmap v9803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17123.011711 secs 2024-04-26T23:20:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:43 smithi119 ceph-mon[32752]: pgmap v9804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:43 smithi167 ceph-mon[32751]: pgmap v9804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:43 smithi090 ceph-mon[29180]: pgmap v9804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:45 smithi119 ceph-mon[32752]: pgmap v9805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:45 smithi167 ceph-mon[32751]: pgmap v9805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:45 smithi090 ceph-mon[29180]: pgmap v9805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:47.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17127.242826 secs 2024-04-26T23:20:47.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17128.011773 secs 2024-04-26T23:20:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17127.242826 secs 2024-04-26T23:20:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17128.011773 secs 2024-04-26T23:20:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17127.242826 secs 2024-04-26T23:20:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17128.011773 secs 2024-04-26T23:20:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:47 smithi119 ceph-mon[32752]: pgmap v9806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:47 smithi167 ceph-mon[32751]: pgmap v9806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:47 smithi090 ceph-mon[29180]: pgmap v9806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:49 smithi119 ceph-mon[32752]: pgmap v9807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:49 smithi167 ceph-mon[32751]: pgmap v9807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:49 smithi090 ceph-mon[29180]: pgmap v9807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17132.242887 secs 2024-04-26T23:20:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:51 smithi119 ceph-mon[32752]: pgmap v9808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17133.011865 secs 2024-04-26T23:20:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17132.242887 secs 2024-04-26T23:20:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:51 smithi167 ceph-mon[32751]: pgmap v9808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17133.011865 secs 2024-04-26T23:20:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17132.242887 secs 2024-04-26T23:20:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:51 smithi090 ceph-mon[29180]: pgmap v9808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17133.011865 secs 2024-04-26T23:20:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:53 smithi119 ceph-mon[32752]: pgmap v9809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:53 smithi167 ceph-mon[32751]: pgmap v9809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:53 smithi090 ceph-mon[29180]: pgmap v9809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:55 smithi119 ceph-mon[32752]: pgmap v9810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:55 smithi167 ceph-mon[32751]: pgmap v9810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:55 smithi090 ceph-mon[29180]: pgmap v9810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:20:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17137.242972 secs 2024-04-26T23:20:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17138.011982 secs 2024-04-26T23:20:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17137.242972 secs 2024-04-26T23:20:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17138.011982 secs 2024-04-26T23:20:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17137.242972 secs 2024-04-26T23:20:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17138.011982 secs 2024-04-26T23:20:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:57 smithi119 ceph-mon[32752]: pgmap v9811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:57 smithi167 ceph-mon[32751]: pgmap v9811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:57 smithi090 ceph-mon[29180]: pgmap v9811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:20:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:20:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:20:59.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:21:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:59 smithi119 ceph-mon[32752]: pgmap v9812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:21:00.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:21:00.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:20:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:21:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:59 smithi167 ceph-mon[32751]: pgmap v9812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:21:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:21:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:20:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:21:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:59 smithi090 ceph-mon[29180]: pgmap v9812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:21:00.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:21:00.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:20:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:21:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:00 smithi119 ceph-mon[32752]: pgmap v9813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:21:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:00 smithi119 ceph-mon[32752]: pgmap v9814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:00 smithi167 ceph-mon[32751]: pgmap v9813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:21:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:00 smithi167 ceph-mon[32751]: pgmap v9814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:00 smithi090 ceph-mon[29180]: pgmap v9813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:21:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:00 smithi090 ceph-mon[29180]: pgmap v9814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17142.243065 secs 2024-04-26T23:21:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17143.012047 secs 2024-04-26T23:21:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17142.243065 secs 2024-04-26T23:21:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17143.012047 secs 2024-04-26T23:21:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17142.243065 secs 2024-04-26T23:21:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17143.012047 secs 2024-04-26T23:21:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:02 smithi119 ceph-mon[32752]: pgmap v9815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:02 smithi167 ceph-mon[32751]: pgmap v9815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:02 smithi090 ceph-mon[29180]: pgmap v9815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:04 smithi119 ceph-mon[32752]: pgmap v9816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:21:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:04 smithi167 ceph-mon[32751]: pgmap v9816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:21:05.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:04 smithi090 ceph-mon[29180]: pgmap v9816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:21:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17147.243149 secs 2024-04-26T23:21:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:06 smithi119 ceph-mon[32752]: pgmap v9817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17148.012184 secs 2024-04-26T23:21:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17147.243149 secs 2024-04-26T23:21:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:06 smithi167 ceph-mon[32751]: pgmap v9817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17148.012184 secs 2024-04-26T23:21:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17147.243149 secs 2024-04-26T23:21:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:06 smithi090 ceph-mon[29180]: pgmap v9817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17148.012184 secs 2024-04-26T23:21:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:08 smithi119 ceph-mon[32752]: pgmap v9818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:08 smithi167 ceph-mon[32751]: pgmap v9818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:08 smithi090 ceph-mon[29180]: pgmap v9818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:21:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:10 smithi119 ceph-mon[32752]: pgmap v9819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:21:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:10 smithi167 ceph-mon[32751]: pgmap v9819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:21:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:10 smithi090 ceph-mon[29180]: pgmap v9819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:21:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17152.243210 secs 2024-04-26T23:21:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17153.012301 secs 2024-04-26T23:21:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17152.243210 secs 2024-04-26T23:21:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17153.012301 secs 2024-04-26T23:21:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17152.243210 secs 2024-04-26T23:21:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17153.012301 secs 2024-04-26T23:21:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:12 smithi119 ceph-mon[32752]: pgmap v9820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:12 smithi167 ceph-mon[32751]: pgmap v9820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:12 smithi090 ceph-mon[29180]: pgmap v9820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:13 smithi119 ceph-mon[32752]: pgmap v9821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:13 smithi167 ceph-mon[32751]: pgmap v9821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:13 smithi090 ceph-mon[29180]: pgmap v9821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17157.243298 secs 2024-04-26T23:21:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:16 smithi090 ceph-mon[29180]: pgmap v9822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17158.012417 secs 2024-04-26T23:21:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17157.243298 secs 2024-04-26T23:21:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:16 smithi119 ceph-mon[32752]: pgmap v9822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17158.012417 secs 2024-04-26T23:21:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17157.243298 secs 2024-04-26T23:21:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:16 smithi167 ceph-mon[32751]: pgmap v9822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17158.012417 secs 2024-04-26T23:21:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:18 smithi090 ceph-mon[29180]: pgmap v9823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:18.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:18 smithi119 ceph-mon[32752]: pgmap v9823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:18 smithi167 ceph-mon[32751]: pgmap v9823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:20 smithi090 ceph-mon[29180]: pgmap v9824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:20 smithi119 ceph-mon[32752]: pgmap v9824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:20 smithi167 ceph-mon[32751]: pgmap v9824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17162.243361 secs 2024-04-26T23:21:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17163.012531 secs 2024-04-26T23:21:21.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17162.243361 secs 2024-04-26T23:21:21.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17163.012531 secs 2024-04-26T23:21:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17162.243361 secs 2024-04-26T23:21:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17163.012531 secs 2024-04-26T23:21:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:22 smithi090 ceph-mon[29180]: pgmap v9825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:22 smithi119 ceph-mon[32752]: pgmap v9825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:22 smithi167 ceph-mon[32751]: pgmap v9825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:24.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:24 smithi090 ceph-mon[29180]: pgmap v9826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:24.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:24 smithi119 ceph-mon[32752]: pgmap v9826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:24.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:24 smithi167 ceph-mon[32751]: pgmap v9826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:26.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17167.243422 secs 2024-04-26T23:21:26.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:26 smithi119 ceph-mon[32752]: pgmap v9827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:26.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17168.012655 secs 2024-04-26T23:21:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17167.243422 secs 2024-04-26T23:21:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:26 smithi090 ceph-mon[29180]: pgmap v9827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17168.012655 secs 2024-04-26T23:21:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17167.243422 secs 2024-04-26T23:21:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:26 smithi167 ceph-mon[32751]: pgmap v9827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17168.012655 secs 2024-04-26T23:21:28.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:28 smithi090 ceph-mon[29180]: pgmap v9828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:28.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:28 smithi119 ceph-mon[32752]: pgmap v9828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:28.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:28 smithi167 ceph-mon[32751]: pgmap v9828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:30 smithi090 ceph-mon[29180]: pgmap v9829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:30 smithi119 ceph-mon[32752]: pgmap v9829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:30 smithi167 ceph-mon[32751]: pgmap v9829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17172.243481 secs 2024-04-26T23:21:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17173.012771 secs 2024-04-26T23:21:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17172.243481 secs 2024-04-26T23:21:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17173.012771 secs 2024-04-26T23:21:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17172.243481 secs 2024-04-26T23:21:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17173.012771 secs 2024-04-26T23:21:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:32 smithi090 ceph-mon[29180]: pgmap v9830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:32 smithi119 ceph-mon[32752]: pgmap v9830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:32 smithi167 ceph-mon[32751]: pgmap v9830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:34.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:34 smithi090 ceph-mon[29180]: pgmap v9831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:34 smithi119 ceph-mon[32752]: pgmap v9831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:34.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:34 smithi167 ceph-mon[32751]: pgmap v9831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17177.243562 secs 2024-04-26T23:21:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:36 smithi090 ceph-mon[29180]: pgmap v9832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17178.012836 secs 2024-04-26T23:21:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17177.243562 secs 2024-04-26T23:21:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:36 smithi119 ceph-mon[32752]: pgmap v9832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17178.012836 secs 2024-04-26T23:21:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17177.243562 secs 2024-04-26T23:21:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:36 smithi167 ceph-mon[32751]: pgmap v9832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17178.012836 secs 2024-04-26T23:21:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:38 smithi090 ceph-mon[29180]: pgmap v9833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:38.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:38 smithi119 ceph-mon[32752]: pgmap v9833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:38.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:38 smithi167 ceph-mon[32751]: pgmap v9833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:40 smithi119 ceph-mon[32752]: pgmap v9834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:40 smithi167 ceph-mon[32751]: pgmap v9834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:40 smithi090 ceph-mon[29180]: pgmap v9834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:41 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17182.243646 secs 2024-04-26T23:21:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:41 smithi119 ceph-mon[32752]: slow request 11522.608355 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:21:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17183.012951 secs 2024-04-26T23:21:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:41 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17182.243646 secs 2024-04-26T23:21:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:41 smithi167 ceph-mon[32751]: slow request 11522.608355 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:21:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17183.012951 secs 2024-04-26T23:21:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:41 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17182.243646 secs 2024-04-26T23:21:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:41 smithi090 ceph-mon[29180]: slow request 11522.608355 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:21:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17183.012951 secs 2024-04-26T23:21:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:42 smithi119 ceph-mon[32752]: pgmap v9835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:42 smithi167 ceph-mon[32751]: pgmap v9835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:42 smithi090 ceph-mon[29180]: pgmap v9835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:44.568 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:44 smithi119 ceph-mon[32752]: pgmap v9836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:44.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:44 smithi167 ceph-mon[32751]: pgmap v9836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:44.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:44 smithi090 ceph-mon[29180]: pgmap v9836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17187.243729 secs 2024-04-26T23:21:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:46 smithi119 ceph-mon[32752]: pgmap v9837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17188.013069 secs 2024-04-26T23:21:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17187.243729 secs 2024-04-26T23:21:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:46 smithi167 ceph-mon[32751]: pgmap v9837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17188.013069 secs 2024-04-26T23:21:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17187.243729 secs 2024-04-26T23:21:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:46 smithi090 ceph-mon[29180]: pgmap v9837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17188.013069 secs 2024-04-26T23:21:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:48 smithi119 ceph-mon[32752]: pgmap v9838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:48 smithi167 ceph-mon[32751]: pgmap v9838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:48 smithi090 ceph-mon[29180]: pgmap v9838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:50 smithi119 ceph-mon[32752]: pgmap v9839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:50 smithi167 ceph-mon[32751]: pgmap v9839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:50 smithi090 ceph-mon[29180]: pgmap v9839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:51.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17192.243811 secs 2024-04-26T23:21:51.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17193.013139 secs 2024-04-26T23:21:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17192.243811 secs 2024-04-26T23:21:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17193.013139 secs 2024-04-26T23:21:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17192.243811 secs 2024-04-26T23:21:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17193.013139 secs 2024-04-26T23:21:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:52 smithi119 ceph-mon[32752]: pgmap v9840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:52 smithi167 ceph-mon[32751]: pgmap v9840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:52 smithi090 ceph-mon[29180]: pgmap v9840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:54 smithi119 ceph-mon[32752]: pgmap v9841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:54.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:54 smithi167 ceph-mon[32751]: pgmap v9841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:54 smithi090 ceph-mon[29180]: pgmap v9841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17197.243897 secs 2024-04-26T23:21:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:56 smithi119 ceph-mon[32752]: pgmap v9842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17198.013257 secs 2024-04-26T23:21:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17197.243897 secs 2024-04-26T23:21:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:56 smithi167 ceph-mon[32751]: pgmap v9842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17198.013257 secs 2024-04-26T23:21:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17197.243897 secs 2024-04-26T23:21:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:56 smithi090 ceph-mon[29180]: pgmap v9842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:21:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17198.013257 secs 2024-04-26T23:21:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:58 smithi119 ceph-mon[32752]: pgmap v9843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:58 smithi167 ceph-mon[32751]: pgmap v9843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:58 smithi090 ceph-mon[29180]: pgmap v9843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:21:59.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:21:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:21:59.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:21:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:21:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:21:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:22:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:00 smithi119 ceph-mon[32752]: pgmap v9844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:22:00.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:22:00.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:22:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:00 smithi167 ceph-mon[32751]: pgmap v9844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:22:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:22:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:22:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:00 smithi090 ceph-mon[29180]: pgmap v9844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:22:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:22:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:22:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:01 smithi119 ceph-mon[32752]: pgmap v9845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:22:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:01 smithi119 ceph-mon[32752]: pgmap v9846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17202.243978 secs 2024-04-26T23:22:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17203.013358 secs 2024-04-26T23:22:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:01 smithi167 ceph-mon[32751]: pgmap v9845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:22:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:01 smithi167 ceph-mon[32751]: pgmap v9846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17202.243978 secs 2024-04-26T23:22:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17203.013358 secs 2024-04-26T23:22:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:01 smithi090 ceph-mon[29180]: pgmap v9845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:22:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:01 smithi090 ceph-mon[29180]: pgmap v9846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17202.243978 secs 2024-04-26T23:22:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17203.013358 secs 2024-04-26T23:22:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:03 smithi119 ceph-mon[32752]: pgmap v9847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:03 smithi167 ceph-mon[32751]: pgmap v9847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:03.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:03 smithi090 ceph-mon[29180]: pgmap v9847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:05 smithi119 ceph-mon[32752]: pgmap v9848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:05 smithi167 ceph-mon[32751]: pgmap v9848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:05 smithi090 ceph-mon[29180]: pgmap v9848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:06.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17207.244036 secs 2024-04-26T23:22:06.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17208.013476 secs 2024-04-26T23:22:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17207.244036 secs 2024-04-26T23:22:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17208.013476 secs 2024-04-26T23:22:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17207.244036 secs 2024-04-26T23:22:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17208.013476 secs 2024-04-26T23:22:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:07 smithi119 ceph-mon[32752]: pgmap v9849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:07 smithi167 ceph-mon[32751]: pgmap v9849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:07 smithi090 ceph-mon[29180]: pgmap v9849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:22:09.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:09 smithi119 ceph-mon[32752]: pgmap v9850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:22:09.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:09 smithi167 ceph-mon[32751]: pgmap v9850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:22:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:09 smithi090 ceph-mon[29180]: pgmap v9850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:22:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:11 smithi119 ceph-mon[32752]: pgmap v9851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:22:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17212.244135 secs 2024-04-26T23:22:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17213.013588 secs 2024-04-26T23:22:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:11 smithi167 ceph-mon[32751]: pgmap v9851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:22:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17212.244135 secs 2024-04-26T23:22:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17213.013588 secs 2024-04-26T23:22:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:11 smithi090 ceph-mon[29180]: pgmap v9851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:22:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17212.244135 secs 2024-04-26T23:22:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17213.013588 secs 2024-04-26T23:22:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:13 smithi119 ceph-mon[32752]: pgmap v9852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:13 smithi167 ceph-mon[32751]: pgmap v9852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:13 smithi090 ceph-mon[29180]: pgmap v9852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:15 smithi119 ceph-mon[32752]: pgmap v9853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:15 smithi167 ceph-mon[32751]: pgmap v9853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:15 smithi090 ceph-mon[29180]: pgmap v9853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17217.244195 secs 2024-04-26T23:22:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17218.013666 secs 2024-04-26T23:22:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17217.244195 secs 2024-04-26T23:22:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17218.013666 secs 2024-04-26T23:22:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17217.244195 secs 2024-04-26T23:22:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17218.013666 secs 2024-04-26T23:22:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:17 smithi119 ceph-mon[32752]: pgmap v9854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:17 smithi167 ceph-mon[32751]: pgmap v9854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:17 smithi090 ceph-mon[29180]: pgmap v9854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:19 smithi119 ceph-mon[32752]: pgmap v9855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:19.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:19 smithi167 ceph-mon[32751]: pgmap v9855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:19.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:19 smithi090 ceph-mon[29180]: pgmap v9855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:21 smithi119 ceph-mon[32752]: pgmap v9856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17222.244259 secs 2024-04-26T23:22:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17223.013740 secs 2024-04-26T23:22:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:21 smithi167 ceph-mon[32751]: pgmap v9856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17222.244259 secs 2024-04-26T23:22:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17223.013740 secs 2024-04-26T23:22:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:21 smithi090 ceph-mon[29180]: pgmap v9856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17222.244259 secs 2024-04-26T23:22:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17223.013740 secs 2024-04-26T23:22:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:23 smithi119 ceph-mon[32752]: pgmap v9857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:23 smithi167 ceph-mon[32751]: pgmap v9857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:23.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:23 smithi090 ceph-mon[29180]: pgmap v9857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:25.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:25 smithi119 ceph-mon[32752]: pgmap v9858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:25 smithi167 ceph-mon[32751]: pgmap v9858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:25 smithi090 ceph-mon[29180]: pgmap v9858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17227.244317 secs 2024-04-26T23:22:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17228.013801 secs 2024-04-26T23:22:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17227.244317 secs 2024-04-26T23:22:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17228.013801 secs 2024-04-26T23:22:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17227.244317 secs 2024-04-26T23:22:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17228.013801 secs 2024-04-26T23:22:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:27 smithi119 ceph-mon[32752]: pgmap v9859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:27 smithi167 ceph-mon[32751]: pgmap v9859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:27 smithi090 ceph-mon[29180]: pgmap v9859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:29.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:29 smithi119 ceph-mon[32752]: pgmap v9860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:29 smithi167 ceph-mon[32751]: pgmap v9860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:29.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:29 smithi090 ceph-mon[29180]: pgmap v9860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:31 smithi119 ceph-mon[32752]: pgmap v9861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17232.244402 secs 2024-04-26T23:22:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17233.013872 secs 2024-04-26T23:22:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:31 smithi167 ceph-mon[32751]: pgmap v9861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17232.244402 secs 2024-04-26T23:22:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17233.013872 secs 2024-04-26T23:22:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:31 smithi090 ceph-mon[29180]: pgmap v9861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17232.244402 secs 2024-04-26T23:22:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17233.013872 secs 2024-04-26T23:22:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:33 smithi119 ceph-mon[32752]: pgmap v9862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:33.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:33 smithi167 ceph-mon[32751]: pgmap v9862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:33 smithi090 ceph-mon[29180]: pgmap v9862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:35 smithi119 ceph-mon[32752]: pgmap v9863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:35.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:35 smithi167 ceph-mon[32751]: pgmap v9863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:35 smithi090 ceph-mon[29180]: pgmap v9863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17237.244484 secs 2024-04-26T23:22:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17238.013988 secs 2024-04-26T23:22:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17237.244484 secs 2024-04-26T23:22:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17238.013988 secs 2024-04-26T23:22:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17237.244484 secs 2024-04-26T23:22:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17238.013988 secs 2024-04-26T23:22:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:37 smithi119 ceph-mon[32752]: pgmap v9864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:37 smithi167 ceph-mon[32751]: pgmap v9864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:37 smithi090 ceph-mon[29180]: pgmap v9864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:39.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:39 smithi119 ceph-mon[32752]: pgmap v9865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:39 smithi167 ceph-mon[32751]: pgmap v9865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:39.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:39 smithi090 ceph-mon[29180]: pgmap v9865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:41 smithi119 ceph-mon[32752]: pgmap v9866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17242.244570 secs 2024-04-26T23:22:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17243.014083 secs 2024-04-26T23:22:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:41 smithi167 ceph-mon[32751]: pgmap v9866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17242.244570 secs 2024-04-26T23:22:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17243.014083 secs 2024-04-26T23:22:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:41 smithi090 ceph-mon[29180]: pgmap v9866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17242.244570 secs 2024-04-26T23:22:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17243.014083 secs 2024-04-26T23:22:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:43 smithi119 ceph-mon[32752]: pgmap v9867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:43.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:43 smithi167 ceph-mon[32751]: pgmap v9867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:43 smithi090 ceph-mon[29180]: pgmap v9867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:45.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:45 smithi119 ceph-mon[32752]: pgmap v9868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:45 smithi167 ceph-mon[32751]: pgmap v9868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:45 smithi090 ceph-mon[29180]: pgmap v9868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17247.244653 secs 2024-04-26T23:22:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17248.014198 secs 2024-04-26T23:22:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17247.244653 secs 2024-04-26T23:22:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17248.014198 secs 2024-04-26T23:22:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17247.244653 secs 2024-04-26T23:22:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17248.014198 secs 2024-04-26T23:22:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:47 smithi119 ceph-mon[32752]: pgmap v9869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:47 smithi167 ceph-mon[32751]: pgmap v9869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:47 smithi090 ceph-mon[29180]: pgmap v9869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:49.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:49 smithi119 ceph-mon[32752]: pgmap v9870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:49 smithi167 ceph-mon[32751]: pgmap v9870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:49 smithi090 ceph-mon[29180]: pgmap v9870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:51 smithi119 ceph-mon[32752]: pgmap v9871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17252.244742 secs 2024-04-26T23:22:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17253.014324 secs 2024-04-26T23:22:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:51 smithi167 ceph-mon[32751]: pgmap v9871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17252.244742 secs 2024-04-26T23:22:51.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17253.014324 secs 2024-04-26T23:22:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:51 smithi090 ceph-mon[29180]: pgmap v9871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17252.244742 secs 2024-04-26T23:22:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17253.014324 secs 2024-04-26T23:22:53.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:53 smithi119 ceph-mon[32752]: pgmap v9872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:53.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:53 smithi167 ceph-mon[32751]: pgmap v9872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:53.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:53 smithi090 ceph-mon[29180]: pgmap v9872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:55 smithi119 ceph-mon[32752]: pgmap v9873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:55.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:55 smithi167 ceph-mon[32751]: pgmap v9873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:55 smithi090 ceph-mon[29180]: pgmap v9873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:22:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17257.244824 secs 2024-04-26T23:22:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17258.014436 secs 2024-04-26T23:22:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17257.244824 secs 2024-04-26T23:22:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17258.014436 secs 2024-04-26T23:22:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17257.244824 secs 2024-04-26T23:22:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17258.014436 secs 2024-04-26T23:22:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:57 smithi090 ceph-mon[29180]: pgmap v9874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:57 smithi119 ceph-mon[32752]: pgmap v9874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:57 smithi167 ceph-mon[32751]: pgmap v9874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:22:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:22:59 smithi090 ceph-mon[29180]: pgmap v9875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:22:59 smithi119 ceph-mon[32752]: pgmap v9875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:22:59 smithi167 ceph-mon[32751]: pgmap v9875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:23:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:23:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:23:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:23:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:23:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:23:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:23:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:23:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:23:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:23:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:23:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:23:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:01 smithi090 ceph-mon[29180]: pgmap v9876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17262.244900 secs 2024-04-26T23:23:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:01 smithi090 ceph-mon[29180]: pgmap v9877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:23:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:01 smithi090 ceph-mon[29180]: pgmap v9878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17263.014495 secs 2024-04-26T23:23:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:01 smithi119 ceph-mon[32752]: pgmap v9876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17262.244900 secs 2024-04-26T23:23:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:01 smithi119 ceph-mon[32752]: pgmap v9877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:23:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:01 smithi119 ceph-mon[32752]: pgmap v9878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17263.014495 secs 2024-04-26T23:23:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:01 smithi167 ceph-mon[32751]: pgmap v9876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17262.244900 secs 2024-04-26T23:23:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:01 smithi167 ceph-mon[32751]: pgmap v9877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:23:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:01 smithi167 ceph-mon[32751]: pgmap v9878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17263.014495 secs 2024-04-26T23:23:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:03 smithi090 ceph-mon[29180]: pgmap v9879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:23:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:03 smithi119 ceph-mon[32752]: pgmap v9879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:23:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:03 smithi167 ceph-mon[32751]: pgmap v9879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:23:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:05 smithi090 ceph-mon[29180]: pgmap v9880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:05 smithi119 ceph-mon[32752]: pgmap v9880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:05 smithi167 ceph-mon[32751]: pgmap v9880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17267.244986 secs 2024-04-26T23:23:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17268.014607 secs 2024-04-26T23:23:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17267.244986 secs 2024-04-26T23:23:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17268.014607 secs 2024-04-26T23:23:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17267.244986 secs 2024-04-26T23:23:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17268.014607 secs 2024-04-26T23:23:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:07 smithi090 ceph-mon[29180]: pgmap v9881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:07 smithi119 ceph-mon[32752]: pgmap v9881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:07 smithi167 ceph-mon[32751]: pgmap v9881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:23:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:09 smithi090 ceph-mon[29180]: pgmap v9882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T23:23:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:09 smithi119 ceph-mon[32752]: pgmap v9882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T23:23:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:09 smithi167 ceph-mon[32751]: pgmap v9882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-26T23:23:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17272.245070 secs 2024-04-26T23:23:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:11 smithi090 ceph-mon[29180]: pgmap v9883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:23:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17273.014719 secs 2024-04-26T23:23:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17272.245070 secs 2024-04-26T23:23:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:11 smithi119 ceph-mon[32752]: pgmap v9883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:23:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17273.014719 secs 2024-04-26T23:23:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17272.245070 secs 2024-04-26T23:23:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:11 smithi167 ceph-mon[32751]: pgmap v9883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:23:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17273.014719 secs 2024-04-26T23:23:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:13 smithi090 ceph-mon[29180]: pgmap v9884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:13 smithi119 ceph-mon[32752]: pgmap v9884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:13 smithi167 ceph-mon[32751]: pgmap v9884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:15 smithi119 ceph-mon[32752]: pgmap v9885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:15 smithi167 ceph-mon[32751]: pgmap v9885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:15 smithi090 ceph-mon[29180]: pgmap v9885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17277.245131 secs 2024-04-26T23:23:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17278.014841 secs 2024-04-26T23:23:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17277.245131 secs 2024-04-26T23:23:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17278.014841 secs 2024-04-26T23:23:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17277.245131 secs 2024-04-26T23:23:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17278.014841 secs 2024-04-26T23:23:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:17 smithi119 ceph-mon[32752]: pgmap v9886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:17 smithi167 ceph-mon[32751]: pgmap v9886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:17 smithi090 ceph-mon[29180]: pgmap v9886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:19 smithi119 ceph-mon[32752]: pgmap v9887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:20.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:19 smithi167 ceph-mon[32751]: pgmap v9887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:19 smithi090 ceph-mon[29180]: pgmap v9887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17282.245191 secs 2024-04-26T23:23:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:21 smithi119 ceph-mon[32752]: pgmap v9888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17283.014959 secs 2024-04-26T23:23:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17282.245191 secs 2024-04-26T23:23:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:21 smithi167 ceph-mon[32751]: pgmap v9888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17283.014959 secs 2024-04-26T23:23:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17282.245191 secs 2024-04-26T23:23:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:21 smithi090 ceph-mon[29180]: pgmap v9888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17283.014959 secs 2024-04-26T23:23:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:23 smithi119 ceph-mon[32752]: pgmap v9889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:23 smithi167 ceph-mon[32751]: pgmap v9889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:23 smithi090 ceph-mon[29180]: pgmap v9889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:25 smithi119 ceph-mon[32752]: pgmap v9890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:25 smithi167 ceph-mon[32751]: pgmap v9890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:25 smithi090 ceph-mon[29180]: pgmap v9890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:26.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17287.245255 secs 2024-04-26T23:23:26.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17288.015073 secs 2024-04-26T23:23:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17287.245255 secs 2024-04-26T23:23:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17288.015073 secs 2024-04-26T23:23:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17287.245255 secs 2024-04-26T23:23:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17288.015073 secs 2024-04-26T23:23:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:27 smithi119 ceph-mon[32752]: pgmap v9891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:27 smithi167 ceph-mon[32751]: pgmap v9891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:27 smithi090 ceph-mon[29180]: pgmap v9891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:29 smithi119 ceph-mon[32752]: pgmap v9892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:29 smithi167 ceph-mon[32751]: pgmap v9892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:29 smithi090 ceph-mon[29180]: pgmap v9892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17292.245313 secs 2024-04-26T23:23:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:31 smithi119 ceph-mon[32752]: pgmap v9893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17293.015209 secs 2024-04-26T23:23:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17292.245313 secs 2024-04-26T23:23:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:31 smithi167 ceph-mon[32751]: pgmap v9893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17293.015209 secs 2024-04-26T23:23:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17292.245313 secs 2024-04-26T23:23:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:31 smithi090 ceph-mon[29180]: pgmap v9893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17293.015209 secs 2024-04-26T23:23:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:33 smithi119 ceph-mon[32752]: pgmap v9894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:33 smithi167 ceph-mon[32751]: pgmap v9894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:34.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:33 smithi090 ceph-mon[29180]: pgmap v9894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:35 smithi090 ceph-mon[29180]: pgmap v9895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:35.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:23:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:23:35.493+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:23:35.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:23:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:23:35.494+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:23:35.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:23:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:23:35.494+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:23:35.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:23:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:23:35.494+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:23:35.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:23:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:23:35.494+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:23:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:35 smithi119 ceph-mon[32752]: pgmap v9895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:35 smithi167 ceph-mon[32751]: pgmap v9895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17297.245397 secs 2024-04-26T23:23:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17298.015345 secs 2024-04-26T23:23:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17297.245397 secs 2024-04-26T23:23:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17298.015345 secs 2024-04-26T23:23:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17297.245397 secs 2024-04-26T23:23:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17298.015345 secs 2024-04-26T23:23:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:37 smithi119 ceph-mon[32752]: pgmap v9896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:37 smithi167 ceph-mon[32751]: pgmap v9896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:37 smithi090 ceph-mon[29180]: pgmap v9896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:39 smithi119 ceph-mon[32752]: pgmap v9897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:39 smithi167 ceph-mon[32751]: pgmap v9897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:39 smithi090 ceph-mon[29180]: pgmap v9897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17302.245481 secs 2024-04-26T23:23:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:41 smithi119 ceph-mon[32752]: pgmap v9898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17303.015456 secs 2024-04-26T23:23:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17302.245481 secs 2024-04-26T23:23:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:41 smithi167 ceph-mon[32751]: pgmap v9898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17303.015456 secs 2024-04-26T23:23:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17302.245481 secs 2024-04-26T23:23:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:41 smithi090 ceph-mon[29180]: pgmap v9898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17303.015456 secs 2024-04-26T23:23:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:43 smithi119 ceph-mon[32752]: pgmap v9899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:43 smithi167 ceph-mon[32751]: pgmap v9899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:43 smithi090 ceph-mon[29180]: pgmap v9899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:45 smithi119 ceph-mon[32752]: pgmap v9900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:45 smithi167 ceph-mon[32751]: pgmap v9900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:45 smithi090 ceph-mon[29180]: pgmap v9900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17307.245565 secs 2024-04-26T23:23:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17308.015580 secs 2024-04-26T23:23:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17307.245565 secs 2024-04-26T23:23:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17308.015580 secs 2024-04-26T23:23:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17307.245565 secs 2024-04-26T23:23:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17308.015580 secs 2024-04-26T23:23:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:47 smithi119 ceph-mon[32752]: pgmap v9901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:47 smithi167 ceph-mon[32751]: pgmap v9901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:47 smithi090 ceph-mon[29180]: pgmap v9901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:23:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:49 smithi119 ceph-mon[32752]: pgmap v9902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:49 smithi167 ceph-mon[32751]: pgmap v9902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:49 smithi090 ceph-mon[29180]: pgmap v9902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:23:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17312.245648 secs 2024-04-26T23:23:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:51 smithi119 ceph-mon[32752]: pgmap v9903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17313.015700 secs 2024-04-26T23:23:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17312.245648 secs 2024-04-26T23:23:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:51 smithi167 ceph-mon[32751]: pgmap v9903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17313.015700 secs 2024-04-26T23:23:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17312.245648 secs 2024-04-26T23:23:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:51 smithi090 ceph-mon[29180]: pgmap v9903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17313.015700 secs 2024-04-26T23:23:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:53 smithi119 ceph-mon[32752]: pgmap v9904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:54.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:53 smithi167 ceph-mon[32751]: pgmap v9904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:53 smithi090 ceph-mon[29180]: pgmap v9904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:56.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:55 smithi119 ceph-mon[32752]: pgmap v9905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:55 smithi167 ceph-mon[32751]: pgmap v9905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:55 smithi090 ceph-mon[29180]: pgmap v9905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:23:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17317.245729 secs 2024-04-26T23:23:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17318.015819 secs 2024-04-26T23:23:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17317.245729 secs 2024-04-26T23:23:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17318.015819 secs 2024-04-26T23:23:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17317.245729 secs 2024-04-26T23:23:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17318.015819 secs 2024-04-26T23:23:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:57 smithi119 ceph-mon[32752]: pgmap v9906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:57 smithi167 ceph-mon[32751]: pgmap v9906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:23:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:57 smithi090 ceph-mon[29180]: pgmap v9906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:00.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:23:59 smithi119 ceph-mon[32752]: pgmap v9907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:23:59 smithi167 ceph-mon[32751]: pgmap v9907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:23:59 smithi090 ceph-mon[29180]: pgmap v9907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:24:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:24:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:24:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17322.245809 secs 2024-04-26T23:24:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:01 smithi119 ceph-mon[32752]: pgmap v9908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17323.015886 secs 2024-04-26T23:24:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:24:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:24:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:24:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17322.245809 secs 2024-04-26T23:24:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:01 smithi167 ceph-mon[32751]: pgmap v9908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17323.015886 secs 2024-04-26T23:24:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:24:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:24:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:24:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17322.245809 secs 2024-04-26T23:24:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:01 smithi090 ceph-mon[29180]: pgmap v9908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17323.015886 secs 2024-04-26T23:24:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:24:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:24:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:24:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:02 smithi119 ceph-mon[32752]: pgmap v9909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T23:24:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:02 smithi119 ceph-mon[32752]: pgmap v9910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:02 smithi167 ceph-mon[32751]: pgmap v9909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T23:24:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:02 smithi167 ceph-mon[32751]: pgmap v9910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:02 smithi090 ceph-mon[29180]: pgmap v9909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T23:24:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:02 smithi090 ceph-mon[29180]: pgmap v9910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:04 smithi119 ceph-mon[32752]: pgmap v9911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:04 smithi167 ceph-mon[32751]: pgmap v9911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:05.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:04 smithi090 ceph-mon[29180]: pgmap v9911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17327.245872 secs 2024-04-26T23:24:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:06 smithi119 ceph-mon[32752]: pgmap v9912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17328.015958 secs 2024-04-26T23:24:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17327.245872 secs 2024-04-26T23:24:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:06 smithi167 ceph-mon[32751]: pgmap v9912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17328.015958 secs 2024-04-26T23:24:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17327.245872 secs 2024-04-26T23:24:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:06 smithi090 ceph-mon[29180]: pgmap v9912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17328.015958 secs 2024-04-26T23:24:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:08 smithi119 ceph-mon[32752]: pgmap v9913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T23:24:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:08 smithi167 ceph-mon[32751]: pgmap v9913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T23:24:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:08 smithi090 ceph-mon[29180]: pgmap v9913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T23:24:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:10 smithi119 ceph-mon[32752]: pgmap v9914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:10 smithi167 ceph-mon[32751]: pgmap v9914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:10 smithi090 ceph-mon[29180]: pgmap v9914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:24:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17332.245933 secs 2024-04-26T23:24:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17333.016090 secs 2024-04-26T23:24:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17332.245933 secs 2024-04-26T23:24:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17333.016090 secs 2024-04-26T23:24:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17332.245933 secs 2024-04-26T23:24:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17333.016090 secs 2024-04-26T23:24:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:12 smithi119 ceph-mon[32752]: pgmap v9915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:24:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:12 smithi167 ceph-mon[32751]: pgmap v9915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:24:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:12 smithi090 ceph-mon[29180]: pgmap v9915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:24:15.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:14 smithi167 ceph-mon[32751]: pgmap v9916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:14 smithi090 ceph-mon[29180]: pgmap v9916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:15.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:14 smithi119 ceph-mon[32752]: pgmap v9916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17337.246017 secs 2024-04-26T23:24:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:16 smithi167 ceph-mon[32751]: pgmap v9917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17338.016163 secs 2024-04-26T23:24:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17337.246017 secs 2024-04-26T23:24:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:16 smithi090 ceph-mon[29180]: pgmap v9917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17338.016163 secs 2024-04-26T23:24:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17337.246017 secs 2024-04-26T23:24:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:16 smithi119 ceph-mon[32752]: pgmap v9917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17338.016163 secs 2024-04-26T23:24:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:18 smithi167 ceph-mon[32751]: pgmap v9918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:18 smithi090 ceph-mon[29180]: pgmap v9918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:19.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:18 smithi119 ceph-mon[32752]: pgmap v9918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:19 smithi090 ceph-mon[29180]: pgmap v9919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:19 smithi119 ceph-mon[32752]: pgmap v9919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:19 smithi167 ceph-mon[32751]: pgmap v9919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:20 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17342.246082 secs 2024-04-26T23:24:21.221 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:20 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17342.246082 secs 2024-04-26T23:24:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:20 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17342.246082 secs 2024-04-26T23:24:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17343.016276 secs 2024-04-26T23:24:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:21 smithi090 ceph-mon[29180]: pgmap v9920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17343.016276 secs 2024-04-26T23:24:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:21 smithi119 ceph-mon[32752]: pgmap v9920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17343.016276 secs 2024-04-26T23:24:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:21 smithi167 ceph-mon[32751]: pgmap v9920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:24.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:24 smithi119 ceph-mon[32752]: pgmap v9921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:24.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:24 smithi167 ceph-mon[32751]: pgmap v9921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:24.830 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:24 smithi090 ceph-mon[29180]: pgmap v9921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17347.246165 secs 2024-04-26T23:24:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:26 smithi119 ceph-mon[32752]: pgmap v9922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17348.016410 secs 2024-04-26T23:24:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17347.246165 secs 2024-04-26T23:24:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:26 smithi167 ceph-mon[32751]: pgmap v9922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17348.016410 secs 2024-04-26T23:24:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17347.246165 secs 2024-04-26T23:24:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:26 smithi090 ceph-mon[29180]: pgmap v9922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17348.016410 secs 2024-04-26T23:24:28.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:28 smithi119 ceph-mon[32752]: pgmap v9923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:28 smithi167 ceph-mon[32751]: pgmap v9923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:28 smithi090 ceph-mon[29180]: pgmap v9923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:30 smithi119 ceph-mon[32752]: pgmap v9924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:30.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:30 smithi167 ceph-mon[32751]: pgmap v9924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:30 smithi090 ceph-mon[29180]: pgmap v9924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17352.246228 secs 2024-04-26T23:24:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17353.016523 secs 2024-04-26T23:24:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17352.246228 secs 2024-04-26T23:24:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17353.016523 secs 2024-04-26T23:24:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17352.246228 secs 2024-04-26T23:24:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17353.016523 secs 2024-04-26T23:24:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:32 smithi119 ceph-mon[32752]: pgmap v9925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:32 smithi167 ceph-mon[32751]: pgmap v9925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:32 smithi090 ceph-mon[29180]: pgmap v9925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:34 smithi119 ceph-mon[32752]: pgmap v9926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:34 smithi167 ceph-mon[32751]: pgmap v9926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:34 smithi090 ceph-mon[29180]: pgmap v9926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17357.246287 secs 2024-04-26T23:24:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:36 smithi119 ceph-mon[32752]: pgmap v9927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17358.016633 secs 2024-04-26T23:24:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17357.246287 secs 2024-04-26T23:24:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:36 smithi167 ceph-mon[32751]: pgmap v9927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17358.016633 secs 2024-04-26T23:24:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17357.246287 secs 2024-04-26T23:24:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:36 smithi090 ceph-mon[29180]: pgmap v9927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17358.016633 secs 2024-04-26T23:24:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:38 smithi119 ceph-mon[32752]: pgmap v9928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:38 smithi167 ceph-mon[32751]: pgmap v9928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:38 smithi090 ceph-mon[29180]: pgmap v9928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:40.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:40 smithi119 ceph-mon[32752]: pgmap v9929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:40 smithi167 ceph-mon[32751]: pgmap v9929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:40 smithi090 ceph-mon[29180]: pgmap v9929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17362.246372 secs 2024-04-26T23:24:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17363.016697 secs 2024-04-26T23:24:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17362.246372 secs 2024-04-26T23:24:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17363.016697 secs 2024-04-26T23:24:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17362.246372 secs 2024-04-26T23:24:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17363.016697 secs 2024-04-26T23:24:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:42 smithi090 ceph-mon[29180]: pgmap v9930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:42 smithi119 ceph-mon[32752]: pgmap v9930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:42 smithi167 ceph-mon[32751]: pgmap v9930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:44.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:44 smithi090 ceph-mon[29180]: pgmap v9931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:44 smithi119 ceph-mon[32752]: pgmap v9931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:44 smithi167 ceph-mon[32751]: pgmap v9931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17367.246461 secs 2024-04-26T23:24:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:46 smithi090 ceph-mon[29180]: pgmap v9932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17368.016808 secs 2024-04-26T23:24:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17367.246461 secs 2024-04-26T23:24:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:46 smithi119 ceph-mon[32752]: pgmap v9932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17368.016808 secs 2024-04-26T23:24:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17367.246461 secs 2024-04-26T23:24:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:46 smithi167 ceph-mon[32751]: pgmap v9932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17368.016808 secs 2024-04-26T23:24:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:48 smithi090 ceph-mon[29180]: pgmap v9933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:49.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:48 smithi119 ceph-mon[32752]: pgmap v9933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:48 smithi167 ceph-mon[32751]: pgmap v9933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:50 smithi090 ceph-mon[29180]: pgmap v9934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:50 smithi119 ceph-mon[32752]: pgmap v9934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:50 smithi167 ceph-mon[32751]: pgmap v9934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:51.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17372.246546 secs 2024-04-26T23:24:51.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17373.016927 secs 2024-04-26T23:24:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17372.246546 secs 2024-04-26T23:24:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17373.016927 secs 2024-04-26T23:24:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17372.246546 secs 2024-04-26T23:24:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17373.016927 secs 2024-04-26T23:24:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:52 smithi119 ceph-mon[32752]: pgmap v9935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:52 smithi167 ceph-mon[32751]: pgmap v9935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:52 smithi090 ceph-mon[29180]: pgmap v9935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:55.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:54 smithi119 ceph-mon[32752]: pgmap v9936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:54 smithi167 ceph-mon[32751]: pgmap v9936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:55.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:54 smithi090 ceph-mon[29180]: pgmap v9936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:24:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17377.246631 secs 2024-04-26T23:24:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:56 smithi119 ceph-mon[32752]: pgmap v9937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17378.017038 secs 2024-04-26T23:24:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17377.246631 secs 2024-04-26T23:24:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:56 smithi167 ceph-mon[32751]: pgmap v9937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17378.017038 secs 2024-04-26T23:24:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17377.246631 secs 2024-04-26T23:24:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:56 smithi090 ceph-mon[29180]: pgmap v9937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17378.017038 secs 2024-04-26T23:24:59.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:24:58 smithi119 ceph-mon[32752]: pgmap v9938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:59.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:24:58 smithi167 ceph-mon[32751]: pgmap v9938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:24:59.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:24:58 smithi090 ceph-mon[29180]: pgmap v9938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:00 smithi119 ceph-mon[32752]: pgmap v9939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:00 smithi167 ceph-mon[32751]: pgmap v9939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:00 smithi090 ceph-mon[29180]: pgmap v9939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17382.246713 secs 2024-04-26T23:25:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17383.017123 secs 2024-04-26T23:25:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:25:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17382.246713 secs 2024-04-26T23:25:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17383.017123 secs 2024-04-26T23:25:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:25:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17382.246713 secs 2024-04-26T23:25:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17383.017123 secs 2024-04-26T23:25:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:25:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:02 smithi119 ceph-mon[32752]: pgmap v9940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:25:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:25:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:25:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:02 smithi167 ceph-mon[32751]: pgmap v9940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:25:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:25:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:25:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:02 smithi090 ceph-mon[29180]: pgmap v9940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:25:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:25:03.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:25:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:03 smithi119 ceph-mon[32752]: pgmap v9941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:25:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:03 smithi119 ceph-mon[32752]: pgmap v9942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:25:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:03 smithi167 ceph-mon[32751]: pgmap v9941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:25:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:03 smithi167 ceph-mon[32751]: pgmap v9942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:25:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:03 smithi090 ceph-mon[29180]: pgmap v9941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:25:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:03 smithi090 ceph-mon[29180]: pgmap v9942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:25:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:05 smithi119 ceph-mon[32752]: pgmap v9943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:06.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:05 smithi167 ceph-mon[32751]: pgmap v9943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:06.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:05 smithi090 ceph-mon[29180]: pgmap v9943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:07.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17387.246799 secs 2024-04-26T23:25:07.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17388.017255 secs 2024-04-26T23:25:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17387.246799 secs 2024-04-26T23:25:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17388.017255 secs 2024-04-26T23:25:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17387.246799 secs 2024-04-26T23:25:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17388.017255 secs 2024-04-26T23:25:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:07 smithi119 ceph-mon[32752]: pgmap v9944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:07 smithi167 ceph-mon[32751]: pgmap v9944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:07 smithi090 ceph-mon[29180]: pgmap v9944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:10.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:09 smithi119 ceph-mon[32752]: pgmap v9945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:25:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:09 smithi167 ceph-mon[32751]: pgmap v9945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:25:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:09 smithi090 ceph-mon[29180]: pgmap v9945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:25:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:11 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17392.246877 secs 2024-04-26T23:25:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:11 smithi090 ceph-mon[29180]: slow request 724.685505 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:25:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:11 smithi090 ceph-mon[29180]: pgmap v9946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17393.017372 secs 2024-04-26T23:25:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:11 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17392.246877 secs 2024-04-26T23:25:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:11 smithi119 ceph-mon[32752]: slow request 724.685505 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:25:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:11 smithi119 ceph-mon[32752]: pgmap v9946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17393.017372 secs 2024-04-26T23:25:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:11 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17392.246877 secs 2024-04-26T23:25:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:11 smithi167 ceph-mon[32751]: slow request 724.685505 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:25:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:11 smithi167 ceph-mon[32751]: pgmap v9946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:25:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17393.017372 secs 2024-04-26T23:25:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:13 smithi119 ceph-mon[32752]: pgmap v9947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:25:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:13 smithi167 ceph-mon[32751]: pgmap v9947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:25:14.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:13 smithi090 ceph-mon[29180]: pgmap v9947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:25:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:15 smithi119 ceph-mon[32752]: pgmap v9948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:15 smithi167 ceph-mon[32751]: pgmap v9948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:15 smithi090 ceph-mon[29180]: pgmap v9948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17397.246960 secs 2024-04-26T23:25:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17398.017506 secs 2024-04-26T23:25:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17397.246960 secs 2024-04-26T23:25:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17398.017506 secs 2024-04-26T23:25:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17397.246960 secs 2024-04-26T23:25:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17398.017506 secs 2024-04-26T23:25:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:17 smithi119 ceph-mon[32752]: pgmap v9949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:18.150 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:17 smithi090 ceph-mon[29180]: pgmap v9949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:18.150 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:17 smithi167 ceph-mon[32751]: pgmap v9949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:20.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:19 smithi119 ceph-mon[32752]: pgmap v9950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:19 smithi167 ceph-mon[32751]: pgmap v9950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:19 smithi090 ceph-mon[29180]: pgmap v9950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17402.247023 secs 2024-04-26T23:25:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:21 smithi119 ceph-mon[32752]: pgmap v9951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17403.017616 secs 2024-04-26T23:25:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17402.247023 secs 2024-04-26T23:25:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:21 smithi167 ceph-mon[32751]: pgmap v9951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17403.017616 secs 2024-04-26T23:25:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17402.247023 secs 2024-04-26T23:25:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:21 smithi090 ceph-mon[29180]: pgmap v9951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17403.017616 secs 2024-04-26T23:25:24.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:23 smithi119 ceph-mon[32752]: pgmap v9952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:23 smithi167 ceph-mon[32751]: pgmap v9952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:23 smithi090 ceph-mon[29180]: pgmap v9952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:25 smithi119 ceph-mon[32752]: pgmap v9953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:25 smithi167 ceph-mon[32751]: pgmap v9953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:25 smithi090 ceph-mon[29180]: pgmap v9953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:27.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17407.247085 secs 2024-04-26T23:25:27.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17408.017734 secs 2024-04-26T23:25:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17407.247085 secs 2024-04-26T23:25:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17408.017734 secs 2024-04-26T23:25:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17407.247085 secs 2024-04-26T23:25:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17408.017734 secs 2024-04-26T23:25:28.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:27 smithi119 ceph-mon[32752]: pgmap v9954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:27 smithi167 ceph-mon[32751]: pgmap v9954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:27 smithi090 ceph-mon[29180]: pgmap v9954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:29 smithi119 ceph-mon[32752]: pgmap v9955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:29 smithi167 ceph-mon[32751]: pgmap v9955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:29 smithi090 ceph-mon[29180]: pgmap v9955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17412.247146 secs 2024-04-26T23:25:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:31 smithi119 ceph-mon[32752]: pgmap v9956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17413.017850 secs 2024-04-26T23:25:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17412.247146 secs 2024-04-26T23:25:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:31 smithi167 ceph-mon[32751]: pgmap v9956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17413.017850 secs 2024-04-26T23:25:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17412.247146 secs 2024-04-26T23:25:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:31 smithi090 ceph-mon[29180]: pgmap v9956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17413.017850 secs 2024-04-26T23:25:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:33 smithi119 ceph-mon[32752]: pgmap v9957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:33 smithi167 ceph-mon[32751]: pgmap v9957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:33 smithi090 ceph-mon[29180]: pgmap v9957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:35 smithi119 ceph-mon[32752]: pgmap v9958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:35 smithi167 ceph-mon[32751]: pgmap v9958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:35 smithi090 ceph-mon[29180]: pgmap v9958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:37.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17417.247207 secs 2024-04-26T23:25:37.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17418.017916 secs 2024-04-26T23:25:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17417.247207 secs 2024-04-26T23:25:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17418.017916 secs 2024-04-26T23:25:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17417.247207 secs 2024-04-26T23:25:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17418.017916 secs 2024-04-26T23:25:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:37 smithi119 ceph-mon[32752]: pgmap v9959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:37 smithi167 ceph-mon[32751]: pgmap v9959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:37 smithi090 ceph-mon[29180]: pgmap v9959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:40.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:39 smithi119 ceph-mon[32752]: pgmap v9960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:39 smithi167 ceph-mon[32751]: pgmap v9960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:40.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:39 smithi090 ceph-mon[29180]: pgmap v9960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17422.247269 secs 2024-04-26T23:25:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:41 smithi119 ceph-mon[32752]: pgmap v9961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17423.018053 secs 2024-04-26T23:25:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17422.247269 secs 2024-04-26T23:25:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:41 smithi167 ceph-mon[32751]: pgmap v9961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17423.018053 secs 2024-04-26T23:25:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17422.247269 secs 2024-04-26T23:25:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:41 smithi090 ceph-mon[29180]: pgmap v9961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17423.018053 secs 2024-04-26T23:25:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:43 smithi119 ceph-mon[32752]: pgmap v9962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:43 smithi167 ceph-mon[32751]: pgmap v9962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:43 smithi090 ceph-mon[29180]: pgmap v9962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:45 smithi119 ceph-mon[32752]: pgmap v9963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:46.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:45 smithi167 ceph-mon[32751]: pgmap v9963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:45 smithi090 ceph-mon[29180]: pgmap v9963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:46 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17427.247372 secs 2024-04-26T23:25:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:46 smithi119 ceph-mon[32752]: slow request 720.016704 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:25:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17428.018173 secs 2024-04-26T23:25:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:46 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17427.247372 secs 2024-04-26T23:25:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:46 smithi167 ceph-mon[32751]: slow request 720.016704 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:25:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17428.018173 secs 2024-04-26T23:25:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:46 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17427.247372 secs 2024-04-26T23:25:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:46 smithi090 ceph-mon[29180]: slow request 720.016704 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:25:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17428.018173 secs 2024-04-26T23:25:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:47 smithi119 ceph-mon[32752]: pgmap v9964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:47 smithi167 ceph-mon[32751]: pgmap v9964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:47 smithi090 ceph-mon[29180]: pgmap v9964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:49 smithi119 ceph-mon[32752]: pgmap v9965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:49 smithi167 ceph-mon[32751]: pgmap v9965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:49 smithi090 ceph-mon[29180]: pgmap v9965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17432.247459 secs 2024-04-26T23:25:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:51 smithi119 ceph-mon[32752]: pgmap v9966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17433.018284 secs 2024-04-26T23:25:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17432.247459 secs 2024-04-26T23:25:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:51 smithi167 ceph-mon[32751]: pgmap v9966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17433.018284 secs 2024-04-26T23:25:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17432.247459 secs 2024-04-26T23:25:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:51 smithi090 ceph-mon[29180]: pgmap v9966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17433.018284 secs 2024-04-26T23:25:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:53 smithi119 ceph-mon[32752]: pgmap v9967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:53 smithi167 ceph-mon[32751]: pgmap v9967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:53 smithi090 ceph-mon[29180]: pgmap v9967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:56.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:55 smithi119 ceph-mon[32752]: pgmap v9968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:56.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:55 smithi167 ceph-mon[32751]: pgmap v9968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:55 smithi090 ceph-mon[29180]: pgmap v9968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:25:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17437.247541 secs 2024-04-26T23:25:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17438.018358 secs 2024-04-26T23:25:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17437.247541 secs 2024-04-26T23:25:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17438.018358 secs 2024-04-26T23:25:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17437.247541 secs 2024-04-26T23:25:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17438.018358 secs 2024-04-26T23:25:58.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:57 smithi119 ceph-mon[32752]: pgmap v9969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:57 smithi167 ceph-mon[32751]: pgmap v9969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:25:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:57 smithi090 ceph-mon[29180]: pgmap v9969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:00.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:25:59 smithi119 ceph-mon[32752]: pgmap v9970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:25:59 smithi167 ceph-mon[32751]: pgmap v9970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:00.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:25:59 smithi090 ceph-mon[29180]: pgmap v9970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17442.247627 secs 2024-04-26T23:26:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:01 smithi119 ceph-mon[32752]: pgmap v9971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17443.018470 secs 2024-04-26T23:26:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17442.247627 secs 2024-04-26T23:26:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:01 smithi167 ceph-mon[32751]: pgmap v9971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17443.018470 secs 2024-04-26T23:26:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17442.247627 secs 2024-04-26T23:26:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:01 smithi090 ceph-mon[29180]: pgmap v9971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17443.018470 secs 2024-04-26T23:26:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:26:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:26:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:26:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:03 smithi119 ceph-mon[32752]: pgmap v9972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:26:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:26:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:26:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:03 smithi167 ceph-mon[32751]: pgmap v9972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:26:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:26:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:26:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:03 smithi090 ceph-mon[29180]: pgmap v9972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:26:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:26:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:26:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:04 smithi119 ceph-mon[32752]: pgmap v9973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:26:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:04 smithi119 ceph-mon[32752]: pgmap v9974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:26:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:04 smithi167 ceph-mon[32751]: pgmap v9973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:26:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:04 smithi167 ceph-mon[32751]: pgmap v9974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:26:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:04 smithi090 ceph-mon[29180]: pgmap v9973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:26:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:04 smithi090 ceph-mon[29180]: pgmap v9974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:26:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:06 smithi119 ceph-mon[32752]: pgmap v9975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17447.247713 secs 2024-04-26T23:26:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17448.018548 secs 2024-04-26T23:26:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:06 smithi167 ceph-mon[32751]: pgmap v9975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17447.247713 secs 2024-04-26T23:26:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17448.018548 secs 2024-04-26T23:26:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:06 smithi090 ceph-mon[29180]: pgmap v9975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17447.247713 secs 2024-04-26T23:26:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17448.018548 secs 2024-04-26T23:26:09.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:08 smithi119 ceph-mon[32752]: pgmap v9976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:08 smithi167 ceph-mon[32751]: pgmap v9976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:08 smithi090 ceph-mon[29180]: pgmap v9976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:10 smithi119 ceph-mon[32752]: pgmap v9977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:10 smithi167 ceph-mon[32751]: pgmap v9977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:10 smithi090 ceph-mon[29180]: pgmap v9977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17452.247799 secs 2024-04-26T23:26:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17453.018664 secs 2024-04-26T23:26:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17452.247799 secs 2024-04-26T23:26:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17453.018664 secs 2024-04-26T23:26:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17452.247799 secs 2024-04-26T23:26:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17453.018664 secs 2024-04-26T23:26:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:12 smithi167 ceph-mon[32751]: pgmap v9978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:12 smithi090 ceph-mon[29180]: pgmap v9978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:12 smithi119 ceph-mon[32752]: pgmap v9978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:26:15.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:14 smithi167 ceph-mon[32751]: pgmap v9979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:26:15.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:14 smithi090 ceph-mon[29180]: pgmap v9979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:26:15.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:14 smithi119 ceph-mon[32752]: pgmap v9979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:26:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:16 smithi167 ceph-mon[32751]: pgmap v9980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17457.247857 secs 2024-04-26T23:26:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17458.018798 secs 2024-04-26T23:26:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:16 smithi090 ceph-mon[29180]: pgmap v9980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17457.247857 secs 2024-04-26T23:26:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17458.018798 secs 2024-04-26T23:26:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:16 smithi119 ceph-mon[32752]: pgmap v9980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17457.247857 secs 2024-04-26T23:26:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17458.018798 secs 2024-04-26T23:26:19.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:18 smithi090 ceph-mon[29180]: pgmap v9981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:19.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:18 smithi119 ceph-mon[32752]: pgmap v9981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:19.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:18 smithi167 ceph-mon[32751]: pgmap v9981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:21.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:20 smithi167 ceph-mon[32751]: pgmap v9982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:20 smithi090 ceph-mon[29180]: pgmap v9982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:20 smithi119 ceph-mon[32752]: pgmap v9982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17462.247940 secs 2024-04-26T23:26:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17463.018868 secs 2024-04-26T23:26:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17462.247940 secs 2024-04-26T23:26:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17463.018868 secs 2024-04-26T23:26:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17462.247940 secs 2024-04-26T23:26:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17463.018868 secs 2024-04-26T23:26:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:22 smithi090 ceph-mon[29180]: pgmap v9983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:23.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:22 smithi119 ceph-mon[32752]: pgmap v9983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:23.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:22 smithi167 ceph-mon[32751]: pgmap v9983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:24 smithi090 ceph-mon[29180]: pgmap v9984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:24 smithi119 ceph-mon[32752]: pgmap v9984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:25.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:24 smithi167 ceph-mon[32751]: pgmap v9984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:25 smithi090 ceph-mon[29180]: pgmap v9985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:25 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17467.248002 secs 2024-04-26T23:26:26.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:25 smithi119 ceph-mon[32752]: pgmap v9985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:26.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:25 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17467.248002 secs 2024-04-26T23:26:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:25 smithi167 ceph-mon[32751]: pgmap v9985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:25 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17467.248002 secs 2024-04-26T23:26:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17468.018948 secs 2024-04-26T23:26:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17468.018948 secs 2024-04-26T23:26:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17468.018948 secs 2024-04-26T23:26:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:27 smithi090 ceph-mon[29180]: pgmap v9986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:28.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:27 smithi119 ceph-mon[32752]: pgmap v9986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:28.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:27 smithi167 ceph-mon[32751]: pgmap v9986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:30 smithi090 ceph-mon[29180]: pgmap v9987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:30.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:30 smithi119 ceph-mon[32752]: pgmap v9987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:30.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:30 smithi167 ceph-mon[32751]: pgmap v9987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17472.248084 secs 2024-04-26T23:26:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17473.019063 secs 2024-04-26T23:26:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17472.248084 secs 2024-04-26T23:26:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17473.019063 secs 2024-04-26T23:26:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17472.248084 secs 2024-04-26T23:26:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17473.019063 secs 2024-04-26T23:26:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:32 smithi119 ceph-mon[32752]: pgmap v9988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:32 smithi167 ceph-mon[32751]: pgmap v9988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:32 smithi090 ceph-mon[29180]: pgmap v9988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:34.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:34 smithi119 ceph-mon[32752]: pgmap v9989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:34.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:34 smithi167 ceph-mon[32751]: pgmap v9989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:34.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:34 smithi090 ceph-mon[29180]: pgmap v9989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:36 smithi119 ceph-mon[32752]: pgmap v9990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17477.248162 secs 2024-04-26T23:26:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17478.019184 secs 2024-04-26T23:26:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:36 smithi167 ceph-mon[32751]: pgmap v9990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17477.248162 secs 2024-04-26T23:26:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17478.019184 secs 2024-04-26T23:26:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:36 smithi090 ceph-mon[29180]: pgmap v9990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17477.248162 secs 2024-04-26T23:26:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17478.019184 secs 2024-04-26T23:26:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:38 smithi119 ceph-mon[32752]: pgmap v9991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:38.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:38 smithi167 ceph-mon[32751]: pgmap v9991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:38.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:38 smithi090 ceph-mon[29180]: pgmap v9991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:40.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:40 smithi119 ceph-mon[32752]: pgmap v9992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:40.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:40 smithi167 ceph-mon[32751]: pgmap v9992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:40.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:40 smithi090 ceph-mon[29180]: pgmap v9992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17482.248240 secs 2024-04-26T23:26:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17483.019317 secs 2024-04-26T23:26:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17482.248240 secs 2024-04-26T23:26:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17483.019317 secs 2024-04-26T23:26:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17482.248240 secs 2024-04-26T23:26:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17483.019317 secs 2024-04-26T23:26:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:42 smithi119 ceph-mon[32752]: pgmap v9993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:42 smithi167 ceph-mon[32751]: pgmap v9993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:42 smithi090 ceph-mon[29180]: pgmap v9993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:44 smithi119 ceph-mon[32752]: pgmap v9994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:44.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:44 smithi167 ceph-mon[32751]: pgmap v9994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:44 smithi090 ceph-mon[29180]: pgmap v9994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:46 smithi167 ceph-mon[32751]: pgmap v9995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17487.248326 secs 2024-04-26T23:26:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17488.019410 secs 2024-04-26T23:26:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:46 smithi090 ceph-mon[29180]: pgmap v9995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17487.248326 secs 2024-04-26T23:26:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17488.019410 secs 2024-04-26T23:26:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:46 smithi119 ceph-mon[32752]: pgmap v9995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17487.248326 secs 2024-04-26T23:26:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17488.019410 secs 2024-04-26T23:26:48.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:48 smithi167 ceph-mon[32751]: pgmap v9996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:48.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:48 smithi090 ceph-mon[29180]: pgmap v9996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:48.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:48 smithi119 ceph-mon[32752]: pgmap v9996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:50 smithi167 ceph-mon[32751]: pgmap v9997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:50 smithi090 ceph-mon[29180]: pgmap v9997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:50.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:50 smithi119 ceph-mon[32752]: pgmap v9997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17492.248412 secs 2024-04-26T23:26:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17493.019544 secs 2024-04-26T23:26:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17492.248412 secs 2024-04-26T23:26:51.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17493.019544 secs 2024-04-26T23:26:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17492.248412 secs 2024-04-26T23:26:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17493.019544 secs 2024-04-26T23:26:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:52 smithi090 ceph-mon[29180]: pgmap v9998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:52 smithi119 ceph-mon[32752]: pgmap v9998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:52 smithi167 ceph-mon[32751]: pgmap v9998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:54 smithi090 ceph-mon[29180]: pgmap v9999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:54.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:54 smithi119 ceph-mon[32752]: pgmap v9999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:54.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:54 smithi167 ceph-mon[32751]: pgmap v9999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:56.358 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:56 smithi119 ceph-mon[32752]: pgmap v10000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:56.358 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17497.248511 secs 2024-04-26T23:26:56.358 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17498.019659 secs 2024-04-26T23:26:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:56 smithi090 ceph-mon[29180]: pgmap v10000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17497.248511 secs 2024-04-26T23:26:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17498.019659 secs 2024-04-26T23:26:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:56 smithi167 ceph-mon[32751]: pgmap v10000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:26:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17497.248511 secs 2024-04-26T23:26:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17498.019659 secs 2024-04-26T23:26:58.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:26:58 smithi090 ceph-mon[29180]: pgmap v10001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:58.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:26:58 smithi119 ceph-mon[32752]: pgmap v10001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:26:58.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:26:58 smithi167 ceph-mon[32751]: pgmap v10001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:00.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:00 smithi090 ceph-mon[29180]: pgmap v10002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:00.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:00 smithi119 ceph-mon[32752]: pgmap v10002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:00.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:00 smithi167 ceph-mon[32751]: pgmap v10002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17502.248596 secs 2024-04-26T23:27:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17503.019801 secs 2024-04-26T23:27:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17502.248596 secs 2024-04-26T23:27:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17503.019801 secs 2024-04-26T23:27:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17502.248596 secs 2024-04-26T23:27:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17503.019801 secs 2024-04-26T23:27:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:02 smithi090 ceph-mon[29180]: pgmap v10003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:02 smithi119 ceph-mon[32752]: pgmap v10003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:02 smithi167 ceph-mon[32751]: pgmap v10003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:04.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:04 smithi090 ceph-mon[29180]: pgmap v10004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:04.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:27:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:27:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:27:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:04 smithi090 ceph-mon[29180]: pgmap v10005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:27:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:04 smithi090 ceph-mon[29180]: pgmap v10006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:27:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:04 smithi119 ceph-mon[32752]: pgmap v10004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:27:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:27:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:27:04.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:04 smithi119 ceph-mon[32752]: pgmap v10005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:27:04.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:04 smithi119 ceph-mon[32752]: pgmap v10006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:04.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:27:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:04 smithi167 ceph-mon[32751]: pgmap v10004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:27:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:27:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:27:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:04 smithi167 ceph-mon[32751]: pgmap v10005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:27:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:04 smithi167 ceph-mon[32751]: pgmap v10006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:04.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:27:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17507.248677 secs 2024-04-26T23:27:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17508.019874 secs 2024-04-26T23:27:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:06 smithi090 ceph-mon[29180]: pgmap v10007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17507.248677 secs 2024-04-26T23:27:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17508.019874 secs 2024-04-26T23:27:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:06 smithi119 ceph-mon[32752]: pgmap v10007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17507.248677 secs 2024-04-26T23:27:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17508.019874 secs 2024-04-26T23:27:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:06 smithi167 ceph-mon[32751]: pgmap v10007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:08 smithi090 ceph-mon[29180]: pgmap v10008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T23:27:09.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:08 smithi119 ceph-mon[32752]: pgmap v10008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T23:27:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:08 smithi167 ceph-mon[32751]: pgmap v10008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-26T23:27:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:10 smithi090 ceph-mon[29180]: pgmap v10009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:10 smithi119 ceph-mon[32752]: pgmap v10009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:10 smithi167 ceph-mon[32751]: pgmap v10009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:11 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17512.248756 secs 2024-04-26T23:27:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:11 smithi090 ceph-mon[29180]: slow request 1444.876589 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:27:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17513.020019 secs 2024-04-26T23:27:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:11 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17512.248756 secs 2024-04-26T23:27:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:11 smithi119 ceph-mon[32752]: slow request 1444.876589 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:27:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17513.020019 secs 2024-04-26T23:27:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:11 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17512.248756 secs 2024-04-26T23:27:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:11 smithi167 ceph-mon[32751]: slow request 1444.876589 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:27:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17513.020019 secs 2024-04-26T23:27:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:12 smithi090 ceph-mon[29180]: pgmap v10010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:12 smithi119 ceph-mon[32752]: pgmap v10010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:13.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:12 smithi167 ceph-mon[32751]: pgmap v10010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:27:14.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:13 smithi090 ceph-mon[29180]: pgmap v10011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:27:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:13 smithi119 ceph-mon[32752]: pgmap v10011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:27:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:13 smithi167 ceph-mon[32751]: pgmap v10011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:27:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17517.248841 secs 2024-04-26T23:27:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17518.020084 secs 2024-04-26T23:27:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:16 smithi090 ceph-mon[29180]: pgmap v10012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17517.248841 secs 2024-04-26T23:27:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17518.020084 secs 2024-04-26T23:27:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:16 smithi119 ceph-mon[32752]: pgmap v10012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17517.248841 secs 2024-04-26T23:27:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17518.020084 secs 2024-04-26T23:27:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:16 smithi167 ceph-mon[32751]: pgmap v10012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:19.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:18 smithi090 ceph-mon[29180]: pgmap v10013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:19.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:18 smithi119 ceph-mon[32752]: pgmap v10013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:19.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:18 smithi167 ceph-mon[32751]: pgmap v10013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:21.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:20 smithi167 ceph-mon[32751]: pgmap v10014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:21.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:20 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17522.248926 secs 2024-04-26T23:27:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:20 smithi090 ceph-mon[29180]: pgmap v10014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:20 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17522.248926 secs 2024-04-26T23:27:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:20 smithi119 ceph-mon[32752]: pgmap v10014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:20 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17522.248926 secs 2024-04-26T23:27:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17523.020200 secs 2024-04-26T23:27:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17523.020200 secs 2024-04-26T23:27:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17523.020200 secs 2024-04-26T23:27:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:22 smithi090 ceph-mon[29180]: pgmap v10015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:23.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:22 smithi119 ceph-mon[32752]: pgmap v10015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:23.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:22 smithi167 ceph-mon[32751]: pgmap v10015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:23 smithi090 ceph-mon[29180]: pgmap v10016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:24.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:23 smithi119 ceph-mon[32752]: pgmap v10016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:24.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:23 smithi167 ceph-mon[32751]: pgmap v10016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17527.249010 secs 2024-04-26T23:27:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17528.020317 secs 2024-04-26T23:27:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:26 smithi090 ceph-mon[29180]: pgmap v10017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17527.249010 secs 2024-04-26T23:27:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17528.020317 secs 2024-04-26T23:27:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:26 smithi119 ceph-mon[32752]: pgmap v10017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17527.249010 secs 2024-04-26T23:27:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17528.020317 secs 2024-04-26T23:27:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:26 smithi167 ceph-mon[32751]: pgmap v10017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:29.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:28 smithi090 ceph-mon[29180]: pgmap v10018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:29.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:28 smithi119 ceph-mon[32752]: pgmap v10018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:29.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:28 smithi167 ceph-mon[32751]: pgmap v10018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:31.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:30 smithi090 ceph-mon[29180]: pgmap v10019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:30 smithi119 ceph-mon[32752]: pgmap v10019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:30 smithi167 ceph-mon[32751]: pgmap v10019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17532.249094 secs 2024-04-26T23:27:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17533.020409 secs 2024-04-26T23:27:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17532.249094 secs 2024-04-26T23:27:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17533.020409 secs 2024-04-26T23:27:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17532.249094 secs 2024-04-26T23:27:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17533.020409 secs 2024-04-26T23:27:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:32 smithi090 ceph-mon[29180]: pgmap v10020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:33.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:32 smithi119 ceph-mon[32752]: pgmap v10020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:33.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:32 smithi167 ceph-mon[32751]: pgmap v10020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:33 smithi090 ceph-mon[29180]: pgmap v10021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:34.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:33 smithi119 ceph-mon[32752]: pgmap v10021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:34.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:33 smithi167 ceph-mon[32751]: pgmap v10021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17537.249177 secs 2024-04-26T23:27:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17538.020527 secs 2024-04-26T23:27:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:36 smithi090 ceph-mon[29180]: pgmap v10022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17537.249177 secs 2024-04-26T23:27:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17538.020527 secs 2024-04-26T23:27:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:36 smithi119 ceph-mon[32752]: pgmap v10022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17537.249177 secs 2024-04-26T23:27:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17538.020527 secs 2024-04-26T23:27:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:36 smithi167 ceph-mon[32751]: pgmap v10022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:39.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:38 smithi090 ceph-mon[29180]: pgmap v10023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:39.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:38 smithi119 ceph-mon[32752]: pgmap v10023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:39.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:38 smithi167 ceph-mon[32751]: pgmap v10023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:41.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:40 smithi090 ceph-mon[29180]: pgmap v10024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:41.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:40 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17542.249264 secs 2024-04-26T23:27:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:40 smithi119 ceph-mon[32752]: pgmap v10024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:40 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17542.249264 secs 2024-04-26T23:27:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:40 smithi167 ceph-mon[32751]: pgmap v10024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:40 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17542.249264 secs 2024-04-26T23:27:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17543.020608 secs 2024-04-26T23:27:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17543.020608 secs 2024-04-26T23:27:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17543.020608 secs 2024-04-26T23:27:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:42 smithi090 ceph-mon[29180]: pgmap v10025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:42 smithi119 ceph-mon[32752]: pgmap v10025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:42 smithi167 ceph-mon[32751]: pgmap v10025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:43 smithi090 ceph-mon[29180]: pgmap v10026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:43 smithi119 ceph-mon[32752]: pgmap v10026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:44.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:43 smithi167 ceph-mon[32751]: pgmap v10026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17547.249348 secs 2024-04-26T23:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17548.020695 secs 2024-04-26T23:27:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:46 smithi090 ceph-mon[29180]: pgmap v10027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17547.249348 secs 2024-04-26T23:27:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17548.020695 secs 2024-04-26T23:27:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:46 smithi119 ceph-mon[32752]: pgmap v10027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17547.249348 secs 2024-04-26T23:27:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17548.020695 secs 2024-04-26T23:27:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:46 smithi167 ceph-mon[32751]: pgmap v10027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:49.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:48 smithi090 ceph-mon[29180]: pgmap v10028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:48 smithi119 ceph-mon[32752]: pgmap v10028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:49.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:48 smithi167 ceph-mon[32751]: pgmap v10028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:51.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:50 smithi167 ceph-mon[32751]: pgmap v10029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:51.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:50 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17552.249419 secs 2024-04-26T23:27:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:50 smithi090 ceph-mon[29180]: pgmap v10029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:50 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17552.249419 secs 2024-04-26T23:27:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:50 smithi119 ceph-mon[32752]: pgmap v10029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:50 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17552.249419 secs 2024-04-26T23:27:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17553.020810 secs 2024-04-26T23:27:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:51 smithi090 ceph-mon[29180]: pgmap v10030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17553.020810 secs 2024-04-26T23:27:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:51 smithi119 ceph-mon[32752]: pgmap v10030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17553.020810 secs 2024-04-26T23:27:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:51 smithi167 ceph-mon[32751]: pgmap v10030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:55.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:54 smithi090 ceph-mon[29180]: pgmap v10031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:55.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:54 smithi119 ceph-mon[32752]: pgmap v10031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:55.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:54 smithi167 ceph-mon[32751]: pgmap v10031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:27:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:55 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17557.249501 secs 2024-04-26T23:27:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:55 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17557.249501 secs 2024-04-26T23:27:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:55 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17557.249501 secs 2024-04-26T23:27:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17558.020959 secs 2024-04-26T23:27:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:56 smithi090 ceph-mon[29180]: pgmap v10032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:57.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17558.020959 secs 2024-04-26T23:27:57.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:56 smithi119 ceph-mon[32752]: pgmap v10032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17558.020959 secs 2024-04-26T23:27:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:56 smithi167 ceph-mon[32751]: pgmap v10032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:27:57 smithi090 ceph-mon[29180]: pgmap v10033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:27:57 smithi119 ceph-mon[32752]: pgmap v10033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:27:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:27:57 smithi167 ceph-mon[32751]: pgmap v10033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:00 smithi090 ceph-mon[29180]: pgmap v10034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:00 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17562.249583 secs 2024-04-26T23:28:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:00 smithi119 ceph-mon[32752]: pgmap v10034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:00 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17562.249583 secs 2024-04-26T23:28:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:00 smithi167 ceph-mon[32751]: pgmap v10034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:00 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17562.249583 secs 2024-04-26T23:28:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17563.021031 secs 2024-04-26T23:28:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17563.021031 secs 2024-04-26T23:28:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17563.021031 secs 2024-04-26T23:28:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:02 smithi090 ceph-mon[29180]: pgmap v10035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:02 smithi119 ceph-mon[32752]: pgmap v10035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:03.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:02 smithi167 ceph-mon[32751]: pgmap v10035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:03 smithi090 ceph-mon[29180]: pgmap v10036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:04.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:03 smithi119 ceph-mon[32752]: pgmap v10036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:04.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:03 smithi167 ceph-mon[32751]: pgmap v10036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:28:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:28:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:28:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:04 smithi090 ceph-mon[29180]: pgmap v10037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:28:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:04 smithi090 ceph-mon[29180]: pgmap v10038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:28:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:28:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:28:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:28:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:04 smithi119 ceph-mon[32752]: pgmap v10037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:28:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:04 smithi119 ceph-mon[32752]: pgmap v10038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:28:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:28:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:28:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:28:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:04 smithi167 ceph-mon[32751]: pgmap v10037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:28:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:04 smithi167 ceph-mon[32751]: pgmap v10038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:05.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:28:06.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:05 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17567.249667 secs 2024-04-26T23:28:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:05 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17567.249667 secs 2024-04-26T23:28:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:05 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17567.249667 secs 2024-04-26T23:28:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17568.021180 secs 2024-04-26T23:28:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:06 smithi090 ceph-mon[29180]: pgmap v10039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17568.021180 secs 2024-04-26T23:28:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:06 smithi119 ceph-mon[32752]: pgmap v10039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17568.021180 secs 2024-04-26T23:28:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:06 smithi167 ceph-mon[32751]: pgmap v10039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:09 smithi090 ceph-mon[29180]: pgmap v10040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:10.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:09 smithi119 ceph-mon[32752]: pgmap v10040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:09 smithi167 ceph-mon[32751]: pgmap v10040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17572.249752 secs 2024-04-26T23:28:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:11 smithi090 ceph-mon[29180]: pgmap v10041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17573.021281 secs 2024-04-26T23:28:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17572.249752 secs 2024-04-26T23:28:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:11 smithi119 ceph-mon[32752]: pgmap v10041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17573.021281 secs 2024-04-26T23:28:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17572.249752 secs 2024-04-26T23:28:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:11 smithi167 ceph-mon[32751]: pgmap v10041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:28:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17573.021281 secs 2024-04-26T23:28:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:13 smithi090 ceph-mon[29180]: pgmap v10042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:28:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:13 smithi119 ceph-mon[32752]: pgmap v10042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:28:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:13 smithi167 ceph-mon[32751]: pgmap v10042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:28:15.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:15 smithi090 ceph-mon[29180]: pgmap v10043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:28:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:15 smithi119 ceph-mon[32752]: pgmap v10043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:28:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:15 smithi167 ceph-mon[32751]: pgmap v10043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:28:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17577.249824 secs 2024-04-26T23:28:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17578.021397 secs 2024-04-26T23:28:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17577.249824 secs 2024-04-26T23:28:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17578.021397 secs 2024-04-26T23:28:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17577.249824 secs 2024-04-26T23:28:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17578.021397 secs 2024-04-26T23:28:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:17 smithi090 ceph-mon[29180]: pgmap v10044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:17 smithi119 ceph-mon[32752]: pgmap v10044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:17 smithi167 ceph-mon[32751]: pgmap v10044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:20.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:19 smithi119 ceph-mon[32752]: pgmap v10045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:19 smithi167 ceph-mon[32751]: pgmap v10045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:19 smithi090 ceph-mon[29180]: pgmap v10045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17582.249883 secs 2024-04-26T23:28:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:21 smithi119 ceph-mon[32752]: pgmap v10046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17583.021490 secs 2024-04-26T23:28:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17582.249883 secs 2024-04-26T23:28:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:21 smithi167 ceph-mon[32751]: pgmap v10046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17583.021490 secs 2024-04-26T23:28:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17582.249883 secs 2024-04-26T23:28:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:21 smithi090 ceph-mon[29180]: pgmap v10046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17583.021490 secs 2024-04-26T23:28:24.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:23 smithi119 ceph-mon[32752]: pgmap v10047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:24.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:23 smithi167 ceph-mon[32751]: pgmap v10047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:23 smithi090 ceph-mon[29180]: pgmap v10047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:25 smithi119 ceph-mon[32752]: pgmap v10048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:25 smithi167 ceph-mon[32751]: pgmap v10048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:25 smithi090 ceph-mon[29180]: pgmap v10048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17587.249945 secs 2024-04-26T23:28:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17588.021571 secs 2024-04-26T23:28:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17587.249945 secs 2024-04-26T23:28:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17588.021571 secs 2024-04-26T23:28:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17587.249945 secs 2024-04-26T23:28:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17588.021571 secs 2024-04-26T23:28:27.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:27 smithi119 ceph-mon[32752]: pgmap v10049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:27 smithi167 ceph-mon[32751]: pgmap v10049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:27 smithi090 ceph-mon[29180]: pgmap v10049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:29 smithi119 ceph-mon[32752]: pgmap v10050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:30.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:29 smithi167 ceph-mon[32751]: pgmap v10050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:29 smithi090 ceph-mon[29180]: pgmap v10050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17592.250031 secs 2024-04-26T23:28:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:31 smithi119 ceph-mon[32752]: pgmap v10051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17593.021657 secs 2024-04-26T23:28:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17592.250031 secs 2024-04-26T23:28:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:31 smithi167 ceph-mon[32751]: pgmap v10051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17593.021657 secs 2024-04-26T23:28:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17592.250031 secs 2024-04-26T23:28:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:31 smithi090 ceph-mon[29180]: pgmap v10051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17593.021657 secs 2024-04-26T23:28:33.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:33 smithi090 ceph-mon[29180]: pgmap v10052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:33 smithi119 ceph-mon[32752]: pgmap v10052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:33 smithi167 ceph-mon[32751]: pgmap v10052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:35 smithi119 ceph-mon[32752]: pgmap v10053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:35 smithi167 ceph-mon[32751]: pgmap v10053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:35 smithi090 ceph-mon[29180]: pgmap v10053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17597.250116 secs 2024-04-26T23:28:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17598.021773 secs 2024-04-26T23:28:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17597.250116 secs 2024-04-26T23:28:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17598.021773 secs 2024-04-26T23:28:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17597.250116 secs 2024-04-26T23:28:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17598.021773 secs 2024-04-26T23:28:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:37 smithi119 ceph-mon[32752]: pgmap v10054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:37 smithi167 ceph-mon[32751]: pgmap v10054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:37 smithi090 ceph-mon[29180]: pgmap v10054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:39.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:39 smithi090 ceph-mon[29180]: pgmap v10055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:40.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:39 smithi119 ceph-mon[32752]: pgmap v10055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:39 smithi167 ceph-mon[32751]: pgmap v10055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17602.250179 secs 2024-04-26T23:28:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:41 smithi119 ceph-mon[32752]: pgmap v10056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17603.021912 secs 2024-04-26T23:28:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17602.250179 secs 2024-04-26T23:28:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:41 smithi167 ceph-mon[32751]: pgmap v10056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17603.021912 secs 2024-04-26T23:28:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17602.250179 secs 2024-04-26T23:28:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:41 smithi090 ceph-mon[29180]: pgmap v10056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17603.021912 secs 2024-04-26T23:28:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:43 smithi119 ceph-mon[32752]: pgmap v10057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:43 smithi167 ceph-mon[32751]: pgmap v10057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:43 smithi090 ceph-mon[29180]: pgmap v10057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:45 smithi119 ceph-mon[32752]: pgmap v10058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:46.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:45 smithi167 ceph-mon[32751]: pgmap v10058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:45 smithi090 ceph-mon[29180]: pgmap v10058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17607.250265 secs 2024-04-26T23:28:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17608.021990 secs 2024-04-26T23:28:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17607.250265 secs 2024-04-26T23:28:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17608.021990 secs 2024-04-26T23:28:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17607.250265 secs 2024-04-26T23:28:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17608.021990 secs 2024-04-26T23:28:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:47 smithi119 ceph-mon[32752]: pgmap v10059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:47 smithi167 ceph-mon[32751]: pgmap v10059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:47 smithi090 ceph-mon[29180]: pgmap v10059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:28:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:49 smithi119 ceph-mon[32752]: pgmap v10060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:49 smithi167 ceph-mon[32751]: pgmap v10060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:49 smithi090 ceph-mon[29180]: pgmap v10060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:28:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17612.250353 secs 2024-04-26T23:28:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:51 smithi119 ceph-mon[32752]: pgmap v10061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17613.022124 secs 2024-04-26T23:28:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17612.250353 secs 2024-04-26T23:28:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:51 smithi167 ceph-mon[32751]: pgmap v10061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17613.022124 secs 2024-04-26T23:28:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17612.250353 secs 2024-04-26T23:28:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:51 smithi090 ceph-mon[29180]: pgmap v10061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17613.022124 secs 2024-04-26T23:28:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:53 smithi119 ceph-mon[32752]: pgmap v10062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:54.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:53 smithi167 ceph-mon[32751]: pgmap v10062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:53 smithi090 ceph-mon[29180]: pgmap v10062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:56.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:55 smithi119 ceph-mon[32752]: pgmap v10063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:56.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:55 smithi167 ceph-mon[32751]: pgmap v10063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:55 smithi090 ceph-mon[29180]: pgmap v10063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:28:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17617.250437 secs 2024-04-26T23:28:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17618.022239 secs 2024-04-26T23:28:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17617.250437 secs 2024-04-26T23:28:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17618.022239 secs 2024-04-26T23:28:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17617.250437 secs 2024-04-26T23:28:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17618.022239 secs 2024-04-26T23:28:58.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:57 smithi119 ceph-mon[32752]: pgmap v10064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:58.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:57 smithi167 ceph-mon[32751]: pgmap v10064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:28:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:57 smithi090 ceph-mon[29180]: pgmap v10064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:00.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:28:59 smithi119 ceph-mon[32752]: pgmap v10065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:00.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:28:59 smithi167 ceph-mon[32751]: pgmap v10065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:00.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:28:59 smithi090 ceph-mon[29180]: pgmap v10065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17622.250521 secs 2024-04-26T23:29:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:01 smithi119 ceph-mon[32752]: pgmap v10066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17623.022356 secs 2024-04-26T23:29:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17622.250521 secs 2024-04-26T23:29:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:01 smithi167 ceph-mon[32751]: pgmap v10066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17623.022356 secs 2024-04-26T23:29:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17622.250521 secs 2024-04-26T23:29:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:01 smithi090 ceph-mon[29180]: pgmap v10066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17623.022356 secs 2024-04-26T23:29:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:03 smithi119 ceph-mon[32752]: pgmap v10067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:03 smithi167 ceph-mon[32751]: pgmap v10067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:03 smithi090 ceph-mon[29180]: pgmap v10067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:29:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:29:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:29:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-mon[29180]: pgmap v10068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:05.880 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:29:05.601+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:29:05.880 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:29:05.601+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:29:05.880 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:29:05.601+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:29:05.880 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:29:05.601+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:29:05.880 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:29:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:29:05.601+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:29:05.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:05 smithi119 ceph-mon[32752]: pgmap v10068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:05.993 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:05.994 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:05.994 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:05.994 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:06.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:05 smithi167 ceph-mon[32751]: pgmap v10068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:06.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17627.250614 secs 2024-04-26T23:29:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17628.022406 secs 2024-04-26T23:29:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17627.250614 secs 2024-04-26T23:29:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17628.022406 secs 2024-04-26T23:29:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17627.250614 secs 2024-04-26T23:29:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17628.022406 secs 2024-04-26T23:29:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: pgmap v10069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: pgmap v10069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: pgmap v10069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:08.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:29:09.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:29:09.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:29:09.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:29:09.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:09.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:29:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:29:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:29:09.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:29:09.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:29:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:29:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:29:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:29:09.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:29:10.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:29:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:09 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:29:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:29:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:09 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:29:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:09 smithi119 ceph-mon[32752]: pgmap v10070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s 2024-04-26T23:29:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:09 smithi119 ceph-mon[32752]: pgmap v10071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 B/s rd, 1 op/s 2024-04-26T23:29:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:29:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:09 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:29:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:29:10.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:09 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:29:10.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:09 smithi167 ceph-mon[32751]: pgmap v10070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s 2024-04-26T23:29:10.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:09 smithi167 ceph-mon[32751]: pgmap v10071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 B/s rd, 1 op/s 2024-04-26T23:29:10.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:29:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:09 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:29:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:29:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:09 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:29:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:09 smithi090 ceph-mon[29180]: pgmap v10070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 895 B/s rd, 1 op/s 2024-04-26T23:29:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:09 smithi090 ceph-mon[29180]: pgmap v10071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 542 B/s rd, 1 op/s 2024-04-26T23:29:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:11 smithi119 ceph-mon[32752]: pgmap v10072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17632.250693 secs 2024-04-26T23:29:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17633.022472 secs 2024-04-26T23:29:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:11 smithi167 ceph-mon[32751]: pgmap v10072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17632.250693 secs 2024-04-26T23:29:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17633.022472 secs 2024-04-26T23:29:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:11 smithi090 ceph-mon[29180]: pgmap v10072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17632.250693 secs 2024-04-26T23:29:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17633.022472 secs 2024-04-26T23:29:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:13 smithi119 ceph-mon[32752]: pgmap v10073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:13 smithi167 ceph-mon[32751]: pgmap v10073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:14.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:13 smithi090 ceph-mon[29180]: pgmap v10073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:15 smithi119 ceph-mon[32752]: pgmap v10074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:15 smithi167 ceph-mon[32751]: pgmap v10074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:15 smithi090 ceph-mon[29180]: pgmap v10074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17637.250772 secs 2024-04-26T23:29:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17638.022623 secs 2024-04-26T23:29:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17637.250772 secs 2024-04-26T23:29:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17638.022623 secs 2024-04-26T23:29:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17637.250772 secs 2024-04-26T23:29:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17638.022623 secs 2024-04-26T23:29:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:17 smithi119 ceph-mon[32752]: pgmap v10075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:17 smithi167 ceph-mon[32751]: pgmap v10075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:17 smithi090 ceph-mon[29180]: pgmap v10075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 434 B/s wr, 16 op/s 2024-04-26T23:29:20.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:19 smithi119 ceph-mon[32752]: pgmap v10076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-26T23:29:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:19 smithi167 ceph-mon[32751]: pgmap v10076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-26T23:29:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:19 smithi090 ceph-mon[29180]: pgmap v10076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-26T23:29:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17642.250830 secs 2024-04-26T23:29:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:21 smithi119 ceph-mon[32752]: pgmap v10077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T23:29:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17643.022757 secs 2024-04-26T23:29:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17642.250830 secs 2024-04-26T23:29:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:21 smithi167 ceph-mon[32751]: pgmap v10077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T23:29:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17643.022757 secs 2024-04-26T23:29:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17642.250830 secs 2024-04-26T23:29:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:21 smithi090 ceph-mon[29180]: pgmap v10077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-26T23:29:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17643.022757 secs 2024-04-26T23:29:24.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:23 smithi119 ceph-mon[32752]: pgmap v10078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:24.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:23 smithi167 ceph-mon[32751]: pgmap v10078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:23 smithi090 ceph-mon[29180]: pgmap v10078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:25 smithi119 ceph-mon[32752]: pgmap v10079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:25 smithi167 ceph-mon[32751]: pgmap v10079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:25 smithi090 ceph-mon[29180]: pgmap v10079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17647.250916 secs 2024-04-26T23:29:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17648.022831 secs 2024-04-26T23:29:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17647.250916 secs 2024-04-26T23:29:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17648.022831 secs 2024-04-26T23:29:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17647.250916 secs 2024-04-26T23:29:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17648.022831 secs 2024-04-26T23:29:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:27 smithi119 ceph-mon[32752]: pgmap v10080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:27 smithi167 ceph-mon[32751]: pgmap v10080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:27 smithi090 ceph-mon[29180]: pgmap v10080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:29 smithi119 ceph-mon[32752]: pgmap v10081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:30.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:29 smithi167 ceph-mon[32751]: pgmap v10081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:29 smithi090 ceph-mon[29180]: pgmap v10081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17652.251000 secs 2024-04-26T23:29:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:31 smithi119 ceph-mon[32752]: pgmap v10082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17653.022927 secs 2024-04-26T23:29:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17652.251000 secs 2024-04-26T23:29:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:31 smithi167 ceph-mon[32751]: pgmap v10082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17653.022927 secs 2024-04-26T23:29:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17652.251000 secs 2024-04-26T23:29:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:31 smithi090 ceph-mon[29180]: pgmap v10082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17653.022927 secs 2024-04-26T23:29:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:33 smithi119 ceph-mon[32752]: pgmap v10083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:33 smithi167 ceph-mon[32751]: pgmap v10083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:33 smithi090 ceph-mon[29180]: pgmap v10083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:36.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:35 smithi119 ceph-mon[32752]: pgmap v10084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:35 smithi167 ceph-mon[32751]: pgmap v10084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:36.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:35 smithi090 ceph-mon[29180]: pgmap v10084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:37.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17657.251084 secs 2024-04-26T23:29:37.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:36 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 17658.023065 secs 2024-04-26T23:29:37.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:36 smithi090 ceph-mon[29180]: slow request 5761.763197 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:29:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17657.251084 secs 2024-04-26T23:29:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:36 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 17658.023065 secs 2024-04-26T23:29:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:36 smithi119 ceph-mon[32752]: slow request 5761.763197 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:29:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17657.251084 secs 2024-04-26T23:29:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:36 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 17658.023065 secs 2024-04-26T23:29:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:36 smithi167 ceph-mon[32751]: slow request 5761.763197 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:29:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:37 smithi119 ceph-mon[32752]: pgmap v10085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:37 smithi167 ceph-mon[32751]: pgmap v10085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:37 smithi090 ceph-mon[29180]: pgmap v10085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:40.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:39 smithi119 ceph-mon[32752]: pgmap v10086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:39 smithi167 ceph-mon[32751]: pgmap v10086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:40.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:39 smithi090 ceph-mon[29180]: pgmap v10086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17662.251160 secs 2024-04-26T23:29:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:41 smithi167 ceph-mon[32751]: pgmap v10087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17663.023171 secs 2024-04-26T23:29:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17662.251160 secs 2024-04-26T23:29:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:41 smithi090 ceph-mon[29180]: pgmap v10087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17663.023171 secs 2024-04-26T23:29:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17662.251160 secs 2024-04-26T23:29:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:41 smithi119 ceph-mon[32752]: pgmap v10087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17663.023171 secs 2024-04-26T23:29:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:43 smithi167 ceph-mon[32751]: pgmap v10088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:43 smithi090 ceph-mon[29180]: pgmap v10088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:43 smithi119 ceph-mon[32752]: pgmap v10088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:46.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:45 smithi167 ceph-mon[32751]: pgmap v10089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:45 smithi090 ceph-mon[29180]: pgmap v10089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:45 smithi119 ceph-mon[32752]: pgmap v10089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17667.251238 secs 2024-04-26T23:29:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17668.023321 secs 2024-04-26T23:29:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17667.251238 secs 2024-04-26T23:29:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17668.023321 secs 2024-04-26T23:29:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17667.251238 secs 2024-04-26T23:29:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17668.023321 secs 2024-04-26T23:29:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:47 smithi090 ceph-mon[29180]: pgmap v10090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:47 smithi119 ceph-mon[32752]: pgmap v10090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:48.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:47 smithi167 ceph-mon[32751]: pgmap v10090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:49 smithi090 ceph-mon[29180]: pgmap v10091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:49 smithi119 ceph-mon[32752]: pgmap v10091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:49 smithi167 ceph-mon[32751]: pgmap v10091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:52.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17672.251323 secs 2024-04-26T23:29:52.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:51 smithi167 ceph-mon[32751]: pgmap v10092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:52.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17673.023433 secs 2024-04-26T23:29:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17672.251323 secs 2024-04-26T23:29:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:51 smithi090 ceph-mon[29180]: pgmap v10092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17673.023433 secs 2024-04-26T23:29:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17672.251323 secs 2024-04-26T23:29:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:51 smithi119 ceph-mon[32752]: pgmap v10092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17673.023433 secs 2024-04-26T23:29:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:53 smithi090 ceph-mon[29180]: pgmap v10093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:53 smithi119 ceph-mon[32752]: pgmap v10093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:53 smithi167 ceph-mon[32751]: pgmap v10093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:55 smithi090 ceph-mon[29180]: pgmap v10094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:55 smithi119 ceph-mon[32752]: pgmap v10094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:55 smithi167 ceph-mon[32751]: pgmap v10094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:29:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17677.251408 secs 2024-04-26T23:29:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17678.023566 secs 2024-04-26T23:29:57.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17677.251408 secs 2024-04-26T23:29:57.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17678.023566 secs 2024-04-26T23:29:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17677.251408 secs 2024-04-26T23:29:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17678.023566 secs 2024-04-26T23:29:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:57 smithi090 ceph-mon[29180]: pgmap v10095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:57 smithi119 ceph-mon[32752]: pgmap v10095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:29:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:57 smithi167 ceph-mon[32751]: pgmap v10095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:00.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:29:59 smithi090 ceph-mon[29180]: pgmap v10096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:00.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:29:59 smithi119 ceph-mon[32752]: pgmap v10096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:00.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:29:59 smithi167 ceph-mon[32751]: pgmap v10096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:30:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:30:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:30:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17682.251494 secs 2024-04-26T23:30:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:01 smithi090 ceph-mon[29180]: pgmap v10097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17683.023678 secs 2024-04-26T23:30:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17682.251494 secs 2024-04-26T23:30:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:01 smithi119 ceph-mon[32752]: pgmap v10097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17683.023678 secs 2024-04-26T23:30:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17682.251494 secs 2024-04-26T23:30:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:01 smithi167 ceph-mon[32751]: pgmap v10097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17683.023678 secs 2024-04-26T23:30:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:03 smithi090 ceph-mon[29180]: pgmap v10098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:04.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:03 smithi119 ceph-mon[32752]: pgmap v10098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:04.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:03 smithi167 ceph-mon[32751]: pgmap v10098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:06.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:05 smithi090 ceph-mon[29180]: pgmap v10099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:05 smithi119 ceph-mon[32752]: pgmap v10099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:05 smithi167 ceph-mon[32751]: pgmap v10099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17687.251575 secs 2024-04-26T23:30:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17688.023807 secs 2024-04-26T23:30:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17687.251575 secs 2024-04-26T23:30:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17688.023807 secs 2024-04-26T23:30:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17687.251575 secs 2024-04-26T23:30:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17688.023807 secs 2024-04-26T23:30:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:07 smithi090 ceph-mon[29180]: pgmap v10100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:30:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:07 smithi119 ceph-mon[32752]: pgmap v10100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:30:08.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:07 smithi167 ceph-mon[32751]: pgmap v10100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:30:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:30:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:09.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:30:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:30:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:10.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:09 smithi090 ceph-mon[29180]: pgmap v10101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:30:10.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:30:10.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:30:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:09 smithi090 ceph-mon[29180]: pgmap v10102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:09 smithi090 ceph-mon[29180]: pgmap v10103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:09 smithi119 ceph-mon[32752]: pgmap v10101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:30:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:30:10.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:30:10.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:09 smithi119 ceph-mon[32752]: pgmap v10102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:10.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:09 smithi119 ceph-mon[32752]: pgmap v10103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:10.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:09 smithi167 ceph-mon[32751]: pgmap v10101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:30:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:30:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:30:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:09 smithi167 ceph-mon[32751]: pgmap v10102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:09 smithi167 ceph-mon[32751]: pgmap v10103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:30:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:10 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17692.251658 secs 2024-04-26T23:30:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:10 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17692.251658 secs 2024-04-26T23:30:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:10 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17692.251658 secs 2024-04-26T23:30:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17693.023925 secs 2024-04-26T23:30:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:11 smithi119 ceph-mon[32752]: pgmap v10104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17693.023925 secs 2024-04-26T23:30:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:11 smithi167 ceph-mon[32751]: pgmap v10104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17693.023925 secs 2024-04-26T23:30:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:11 smithi090 ceph-mon[29180]: pgmap v10104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:30:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:14 smithi090 ceph-mon[29180]: pgmap v10105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:14 smithi119 ceph-mon[32752]: pgmap v10105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:14 smithi167 ceph-mon[32751]: pgmap v10105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17697.251742 secs 2024-04-26T23:30:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:16 smithi090 ceph-mon[29180]: pgmap v10106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17698.024035 secs 2024-04-26T23:30:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17697.251742 secs 2024-04-26T23:30:16.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:16 smithi119 ceph-mon[32752]: pgmap v10106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:16.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17698.024035 secs 2024-04-26T23:30:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17697.251742 secs 2024-04-26T23:30:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:16 smithi167 ceph-mon[32751]: pgmap v10106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17698.024035 secs 2024-04-26T23:30:18.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:18 smithi090 ceph-mon[29180]: pgmap v10107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:18 smithi119 ceph-mon[32752]: pgmap v10107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:18.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:18 smithi167 ceph-mon[32751]: pgmap v10107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:30:20.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:20 smithi090 ceph-mon[29180]: pgmap v10108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:30:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:20 smithi119 ceph-mon[32752]: pgmap v10108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:30:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:20 smithi167 ceph-mon[32751]: pgmap v10108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:30:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17702.251828 secs 2024-04-26T23:30:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17703.024154 secs 2024-04-26T23:30:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17702.251828 secs 2024-04-26T23:30:21.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17703.024154 secs 2024-04-26T23:30:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17702.251828 secs 2024-04-26T23:30:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17703.024154 secs 2024-04-26T23:30:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:22 smithi090 ceph-mon[29180]: pgmap v10109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:22 smithi167 ceph-mon[32751]: pgmap v10109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:22 smithi119 ceph-mon[32752]: pgmap v10109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:24.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:24 smithi090 ceph-mon[29180]: pgmap v10110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:24.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:24 smithi119 ceph-mon[32752]: pgmap v10110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:24 smithi167 ceph-mon[32751]: pgmap v10110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17707.251918 secs 2024-04-26T23:30:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:26 smithi090 ceph-mon[29180]: pgmap v10111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:26.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17708.024269 secs 2024-04-26T23:30:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17707.251918 secs 2024-04-26T23:30:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:26 smithi119 ceph-mon[32752]: pgmap v10111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17708.024269 secs 2024-04-26T23:30:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17707.251918 secs 2024-04-26T23:30:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:26 smithi167 ceph-mon[32751]: pgmap v10111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17708.024269 secs 2024-04-26T23:30:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:28 smithi090 ceph-mon[29180]: pgmap v10112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:28 smithi119 ceph-mon[32752]: pgmap v10112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:28 smithi167 ceph-mon[32751]: pgmap v10112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:30 smithi090 ceph-mon[29180]: pgmap v10113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:30 smithi119 ceph-mon[32752]: pgmap v10113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:30.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:30 smithi167 ceph-mon[32751]: pgmap v10113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17712.252005 secs 2024-04-26T23:30:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17713.024402 secs 2024-04-26T23:30:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17712.252005 secs 2024-04-26T23:30:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17713.024402 secs 2024-04-26T23:30:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17712.252005 secs 2024-04-26T23:30:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17713.024402 secs 2024-04-26T23:30:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:32 smithi119 ceph-mon[32752]: pgmap v10114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:32 smithi167 ceph-mon[32751]: pgmap v10114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:32 smithi090 ceph-mon[29180]: pgmap v10114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:34 smithi119 ceph-mon[32752]: pgmap v10115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:34 smithi167 ceph-mon[32751]: pgmap v10115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:34 smithi090 ceph-mon[29180]: pgmap v10115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17717.252061 secs 2024-04-26T23:30:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:36 smithi119 ceph-mon[32752]: pgmap v10116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17718.024551 secs 2024-04-26T23:30:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17717.252061 secs 2024-04-26T23:30:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:36 smithi167 ceph-mon[32751]: pgmap v10116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17718.024551 secs 2024-04-26T23:30:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17717.252061 secs 2024-04-26T23:30:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:36 smithi090 ceph-mon[29180]: pgmap v10116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17718.024551 secs 2024-04-26T23:30:38.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:38 smithi090 ceph-mon[29180]: pgmap v10117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:38 smithi119 ceph-mon[32752]: pgmap v10117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:38 smithi167 ceph-mon[32751]: pgmap v10117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:40.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:40 smithi119 ceph-mon[32752]: pgmap v10118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:40.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:40 smithi167 ceph-mon[32751]: pgmap v10118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:40.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:40 smithi090 ceph-mon[29180]: pgmap v10118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17722.252143 secs 2024-04-26T23:30:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17723.024650 secs 2024-04-26T23:30:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17722.252143 secs 2024-04-26T23:30:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17723.024650 secs 2024-04-26T23:30:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17722.252143 secs 2024-04-26T23:30:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17723.024650 secs 2024-04-26T23:30:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:42 smithi119 ceph-mon[32752]: pgmap v10119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:42 smithi167 ceph-mon[32751]: pgmap v10119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:42 smithi090 ceph-mon[29180]: pgmap v10119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:44 smithi119 ceph-mon[32752]: pgmap v10120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:45.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:44 smithi167 ceph-mon[32751]: pgmap v10120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:45.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:44 smithi090 ceph-mon[29180]: pgmap v10120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17727.252228 secs 2024-04-26T23:30:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:46 smithi119 ceph-mon[32752]: pgmap v10121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17728.024753 secs 2024-04-26T23:30:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17727.252228 secs 2024-04-26T23:30:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:46 smithi167 ceph-mon[32751]: pgmap v10121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17728.024753 secs 2024-04-26T23:30:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17727.252228 secs 2024-04-26T23:30:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:46 smithi090 ceph-mon[29180]: pgmap v10121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17728.024753 secs 2024-04-26T23:30:49.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:48 smithi119 ceph-mon[32752]: pgmap v10122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:49.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:48 smithi167 ceph-mon[32751]: pgmap v10122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:49.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:48 smithi090 ceph-mon[29180]: pgmap v10122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:50 smithi119 ceph-mon[32752]: pgmap v10123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:50 smithi167 ceph-mon[32751]: pgmap v10123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:50 smithi090 ceph-mon[29180]: pgmap v10123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17732.252310 secs 2024-04-26T23:30:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17733.024874 secs 2024-04-26T23:30:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17732.252310 secs 2024-04-26T23:30:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17733.024874 secs 2024-04-26T23:30:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17732.252310 secs 2024-04-26T23:30:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17733.024874 secs 2024-04-26T23:30:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:52 smithi167 ceph-mon[32751]: pgmap v10124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:52 smithi119 ceph-mon[32752]: pgmap v10124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:52 smithi090 ceph-mon[29180]: pgmap v10124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:54.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:54 smithi090 ceph-mon[29180]: pgmap v10125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:55.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:54 smithi119 ceph-mon[32752]: pgmap v10125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:55.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:54 smithi167 ceph-mon[32751]: pgmap v10125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:30:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17737.252393 secs 2024-04-26T23:30:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:56 smithi119 ceph-mon[32752]: pgmap v10126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17738.024985 secs 2024-04-26T23:30:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17737.252393 secs 2024-04-26T23:30:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:56 smithi167 ceph-mon[32751]: pgmap v10126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17738.024985 secs 2024-04-26T23:30:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17737.252393 secs 2024-04-26T23:30:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:56 smithi090 ceph-mon[29180]: pgmap v10126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17738.024985 secs 2024-04-26T23:30:59.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:30:58 smithi119 ceph-mon[32752]: pgmap v10127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:59.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:30:58 smithi167 ceph-mon[32751]: pgmap v10127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:30:59.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:30:58 smithi090 ceph-mon[29180]: pgmap v10127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:00 smithi119 ceph-mon[32752]: pgmap v10128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:00 smithi167 ceph-mon[32751]: pgmap v10128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:00 smithi090 ceph-mon[29180]: pgmap v10128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17742.252476 secs 2024-04-26T23:31:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17743.025106 secs 2024-04-26T23:31:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17742.252476 secs 2024-04-26T23:31:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17743.025106 secs 2024-04-26T23:31:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17742.252476 secs 2024-04-26T23:31:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17743.025106 secs 2024-04-26T23:31:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:02 smithi119 ceph-mon[32752]: pgmap v10129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:02 smithi167 ceph-mon[32751]: pgmap v10129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:02 smithi090 ceph-mon[29180]: pgmap v10129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:04 smithi119 ceph-mon[32752]: pgmap v10130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:04 smithi167 ceph-mon[32751]: pgmap v10130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:04 smithi090 ceph-mon[29180]: pgmap v10130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:06 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17747.252552 secs 2024-04-26T23:31:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:06 smithi119 ceph-mon[32752]: slow request 2881.131175 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:31:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:06 smithi119 ceph-mon[32752]: pgmap v10131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:06 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17748.025206 secs 2024-04-26T23:31:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:06 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17747.252552 secs 2024-04-26T23:31:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:06 smithi167 ceph-mon[32751]: slow request 2881.131175 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:31:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:06 smithi167 ceph-mon[32751]: pgmap v10131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:06 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17748.025206 secs 2024-04-26T23:31:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:06 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17747.252552 secs 2024-04-26T23:31:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:06 smithi090 ceph-mon[29180]: slow request 2881.131175 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:31:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:06 smithi090 ceph-mon[29180]: pgmap v10131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:06 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17748.025206 secs 2024-04-26T23:31:09.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:08 smithi119 ceph-mon[32752]: pgmap v10132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:08 smithi167 ceph-mon[32751]: pgmap v10132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:08 smithi090 ceph-mon[29180]: pgmap v10132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:10.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:31:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:31:10.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:31:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:10 smithi119 ceph-mon[32752]: pgmap v10133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:31:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:31:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:31:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:10 smithi167 ceph-mon[32751]: pgmap v10133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:31:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:31:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:31:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:10 smithi090 ceph-mon[29180]: pgmap v10133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:31:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:31:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:31:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17752.252639 secs 2024-04-26T23:31:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:11 smithi119 ceph-mon[32752]: pgmap v10134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T23:31:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:11 smithi119 ceph-mon[32752]: pgmap v10135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:11 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17753.025359 secs 2024-04-26T23:31:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17752.252639 secs 2024-04-26T23:31:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:11 smithi167 ceph-mon[32751]: pgmap v10134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T23:31:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:11 smithi167 ceph-mon[32751]: pgmap v10135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:11 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17753.025359 secs 2024-04-26T23:31:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17752.252639 secs 2024-04-26T23:31:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:11 smithi090 ceph-mon[29180]: pgmap v10134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-26T23:31:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:11 smithi090 ceph-mon[29180]: pgmap v10135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:11 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17753.025359 secs 2024-04-26T23:31:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:13 smithi119 ceph-mon[32752]: pgmap v10136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T23:31:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:13 smithi167 ceph-mon[32751]: pgmap v10136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T23:31:14.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:13 smithi090 ceph-mon[29180]: pgmap v10136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T23:31:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:15 smithi119 ceph-mon[32752]: pgmap v10137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:15 smithi167 ceph-mon[32751]: pgmap v10137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:15 smithi090 ceph-mon[29180]: pgmap v10137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17757.252714 secs 2024-04-26T23:31:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17758.025408 secs 2024-04-26T23:31:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17757.252714 secs 2024-04-26T23:31:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17758.025408 secs 2024-04-26T23:31:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17757.252714 secs 2024-04-26T23:31:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17758.025408 secs 2024-04-26T23:31:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:17 smithi119 ceph-mon[32752]: pgmap v10138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:17 smithi167 ceph-mon[32751]: pgmap v10138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:17 smithi090 ceph-mon[29180]: pgmap v10138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:31:20.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:19 smithi119 ceph-mon[32752]: pgmap v10139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T23:31:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:19 smithi167 ceph-mon[32751]: pgmap v10139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T23:31:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:19 smithi090 ceph-mon[29180]: pgmap v10139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-26T23:31:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17762.252802 secs 2024-04-26T23:31:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:21 smithi119 ceph-mon[32752]: pgmap v10140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:31:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17763.025502 secs 2024-04-26T23:31:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17762.252802 secs 2024-04-26T23:31:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:21 smithi167 ceph-mon[32751]: pgmap v10140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:31:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17763.025502 secs 2024-04-26T23:31:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17762.252802 secs 2024-04-26T23:31:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:21 smithi090 ceph-mon[29180]: pgmap v10140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:31:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17763.025502 secs 2024-04-26T23:31:24.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:23 smithi119 ceph-mon[32752]: pgmap v10141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:24.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:23 smithi167 ceph-mon[32751]: pgmap v10141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:23 smithi090 ceph-mon[29180]: pgmap v10141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:25 smithi119 ceph-mon[32752]: pgmap v10142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:25 smithi167 ceph-mon[32751]: pgmap v10142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:25 smithi090 ceph-mon[29180]: pgmap v10142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17767.252864 secs 2024-04-26T23:31:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17768.025568 secs 2024-04-26T23:31:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17767.252864 secs 2024-04-26T23:31:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17768.025568 secs 2024-04-26T23:31:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17767.252864 secs 2024-04-26T23:31:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17768.025568 secs 2024-04-26T23:31:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:27 smithi119 ceph-mon[32752]: pgmap v10143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:27 smithi167 ceph-mon[32751]: pgmap v10143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:27 smithi090 ceph-mon[29180]: pgmap v10143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:29 smithi119 ceph-mon[32752]: pgmap v10144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:30.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:29 smithi167 ceph-mon[32751]: pgmap v10144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:29 smithi090 ceph-mon[29180]: pgmap v10144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17772.252925 secs 2024-04-26T23:31:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:31 smithi167 ceph-mon[32751]: pgmap v10145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:31 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17773.025703 secs 2024-04-26T23:31:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17772.252925 secs 2024-04-26T23:31:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:31 smithi090 ceph-mon[29180]: pgmap v10145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17773.025703 secs 2024-04-26T23:31:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17772.252925 secs 2024-04-26T23:31:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:31 smithi119 ceph-mon[32752]: pgmap v10145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17773.025703 secs 2024-04-26T23:31:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:33 smithi167 ceph-mon[32751]: pgmap v10146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:33 smithi090 ceph-mon[29180]: pgmap v10146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:34.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:33 smithi119 ceph-mon[32752]: pgmap v10146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:35 smithi167 ceph-mon[32751]: pgmap v10147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:35 smithi090 ceph-mon[29180]: pgmap v10147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:35 smithi119 ceph-mon[32752]: pgmap v10147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17777.253007 secs 2024-04-26T23:31:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:36 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17778.025837 secs 2024-04-26T23:31:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17777.253007 secs 2024-04-26T23:31:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:36 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17778.025837 secs 2024-04-26T23:31:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17777.253007 secs 2024-04-26T23:31:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:36 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17778.025837 secs 2024-04-26T23:31:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:37 smithi090 ceph-mon[29180]: pgmap v10148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:37 smithi119 ceph-mon[32752]: pgmap v10148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:38.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:37 smithi167 ceph-mon[32751]: pgmap v10148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:40.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:39 smithi090 ceph-mon[29180]: pgmap v10149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:40.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:39 smithi119 ceph-mon[32752]: pgmap v10149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:40.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:39 smithi167 ceph-mon[32751]: pgmap v10149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17782.253111 secs 2024-04-26T23:31:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:41 smithi090 ceph-mon[29180]: pgmap v10150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:41 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17783.025955 secs 2024-04-26T23:31:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17782.253111 secs 2024-04-26T23:31:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:41 smithi119 ceph-mon[32752]: pgmap v10150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:41 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17783.025955 secs 2024-04-26T23:31:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17782.253111 secs 2024-04-26T23:31:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:41 smithi167 ceph-mon[32751]: pgmap v10150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:41 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17783.025955 secs 2024-04-26T23:31:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:43 smithi090 ceph-mon[29180]: pgmap v10151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:43 smithi119 ceph-mon[32752]: pgmap v10151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:44.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:43 smithi167 ceph-mon[32751]: pgmap v10151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:31:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:45 smithi090 ceph-mon[29180]: pgmap v10152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:45 smithi119 ceph-mon[32752]: pgmap v10152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:45 smithi167 ceph-mon[32751]: pgmap v10152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17787.253196 secs 2024-04-26T23:31:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:46 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17788.026104 secs 2024-04-26T23:31:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17787.253196 secs 2024-04-26T23:31:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:46 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17788.026104 secs 2024-04-26T23:31:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17787.253196 secs 2024-04-26T23:31:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:46 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17788.026104 secs 2024-04-26T23:31:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:47 smithi090 ceph-mon[29180]: pgmap v10153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:47 smithi119 ceph-mon[32752]: pgmap v10153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:48.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:47 smithi167 ceph-mon[32751]: pgmap v10153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:49 smithi090 ceph-mon[29180]: pgmap v10154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:49 smithi119 ceph-mon[32752]: pgmap v10154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:49 smithi167 ceph-mon[32751]: pgmap v10154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17792.253259 secs 2024-04-26T23:31:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:51 smithi090 ceph-mon[29180]: pgmap v10155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:51 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17793.026223 secs 2024-04-26T23:31:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17792.253259 secs 2024-04-26T23:31:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:51 smithi119 ceph-mon[32752]: pgmap v10155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:51 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17793.026223 secs 2024-04-26T23:31:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17792.253259 secs 2024-04-26T23:31:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:51 smithi167 ceph-mon[32751]: pgmap v10155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:51 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17793.026223 secs 2024-04-26T23:31:54.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:53 smithi167 ceph-mon[32751]: pgmap v10156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:53 smithi090 ceph-mon[29180]: pgmap v10156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:53 smithi119 ceph-mon[32752]: pgmap v10156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:55 smithi090 ceph-mon[29180]: pgmap v10157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:55 smithi119 ceph-mon[32752]: pgmap v10157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:55 smithi167 ceph-mon[32751]: pgmap v10157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:31:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17797.253344 secs 2024-04-26T23:31:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:56 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17798.026346 secs 2024-04-26T23:31:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17797.253344 secs 2024-04-26T23:31:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:56 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17798.026346 secs 2024-04-26T23:31:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17797.253344 secs 2024-04-26T23:31:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:56 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17798.026346 secs 2024-04-26T23:31:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:57 smithi090 ceph-mon[29180]: pgmap v10158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:57 smithi119 ceph-mon[32752]: pgmap v10158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:31:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:57 smithi167 ceph-mon[32751]: pgmap v10158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:00.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:31:59 smithi090 ceph-mon[29180]: pgmap v10159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:00.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:31:59 smithi119 ceph-mon[32752]: pgmap v10159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:00.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:31:59 smithi167 ceph-mon[32751]: pgmap v10159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:00 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17802.253431 secs 2024-04-26T23:32:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:00 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17802.253431 secs 2024-04-26T23:32:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:00 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17802.253431 secs 2024-04-26T23:32:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:01 smithi090 ceph-mon[29180]: pgmap v10160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:01 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 17803.026457 secs 2024-04-26T23:32:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:01 smithi119 ceph-mon[32752]: pgmap v10160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:01 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 17803.026457 secs 2024-04-26T23:32:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:01 smithi167 ceph-mon[32751]: pgmap v10160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:01 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 17803.026457 secs 2024-04-26T23:32:04.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:03 smithi119 ceph-mon[32752]: pgmap v10161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:04.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:03 smithi167 ceph-mon[32751]: pgmap v10161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:04.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:03 smithi090 ceph-mon[29180]: pgmap v10161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:04 smithi119 ceph-mon[32752]: pgmap v10162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:04 smithi167 ceph-mon[32751]: pgmap v10162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:05.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:04 smithi090 ceph-mon[29180]: pgmap v10162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:05 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17807.253518 secs 2024-04-26T23:32:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:05 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17807.253518 secs 2024-04-26T23:32:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:05 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17807.253518 secs 2024-04-26T23:32:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:06 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 17808.026570 secs 2024-04-26T23:32:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:06 smithi119 ceph-mon[32752]: slow request 180.166681 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:32:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:06 smithi119 ceph-mon[32752]: pgmap v10163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:06 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 17808.026570 secs 2024-04-26T23:32:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:06 smithi167 ceph-mon[32751]: slow request 180.166681 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:32:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:06 smithi167 ceph-mon[32751]: pgmap v10163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:06 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 17808.026570 secs 2024-04-26T23:32:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:06 smithi090 ceph-mon[29180]: slow request 180.166681 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:32:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:06 smithi090 ceph-mon[29180]: pgmap v10163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T23:32:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:32:08.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T23:32:08.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:32:08.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T23:32:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:32:09.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:08 smithi119 ceph-mon[32752]: pgmap v10164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:08 smithi167 ceph-mon[32751]: pgmap v10164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:09.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:08 smithi090 ceph-mon[29180]: pgmap v10164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 17812.253605 secs 2024-04-26T23:32:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: slow request 1444.759618 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:32:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: pgmap v10165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:32:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17813.026639 secs 2024-04-26T23:32:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:32:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:32:11.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:32:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 17812.253605 secs 2024-04-26T23:32:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: slow request 1444.759618 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:32:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: pgmap v10165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:32:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17813.026639 secs 2024-04-26T23:32:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:32:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:32:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:32:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 17812.253605 secs 2024-04-26T23:32:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: slow request 1444.759618 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:32:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: pgmap v10165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:32:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17813.026639 secs 2024-04-26T23:32:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:32:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:32:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:32:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:12 smithi119 ceph-mon[32752]: pgmap v10166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:32:12.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:12 smithi119 ceph-mon[32752]: pgmap v10167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:12 smithi167 ceph-mon[32751]: pgmap v10166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:32:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:12 smithi167 ceph-mon[32751]: pgmap v10167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:12 smithi090 ceph-mon[29180]: pgmap v10166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:32:12.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:12 smithi090 ceph-mon[29180]: pgmap v10167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:14.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:14 smithi119 ceph-mon[32752]: pgmap v10168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:32:14.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:14 smithi167 ceph-mon[32751]: pgmap v10168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:32:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:14 smithi090 ceph-mon[29180]: pgmap v10168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:32:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:16 smithi119 ceph-mon[32752]: pgmap v10169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17817.253692 secs 2024-04-26T23:32:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17818.026763 secs 2024-04-26T23:32:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:16 smithi167 ceph-mon[32751]: pgmap v10169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17817.253692 secs 2024-04-26T23:32:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17818.026763 secs 2024-04-26T23:32:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:16 smithi090 ceph-mon[29180]: pgmap v10169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17817.253692 secs 2024-04-26T23:32:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17818.026763 secs 2024-04-26T23:32:18.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:18 smithi119 ceph-mon[32752]: pgmap v10170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:18.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:18 smithi167 ceph-mon[32751]: pgmap v10170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:18.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:18 smithi090 ceph-mon[29180]: pgmap v10170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:20.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:20 smithi119 ceph-mon[32752]: pgmap v10171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:20.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:20 smithi167 ceph-mon[32751]: pgmap v10171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:20.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:20 smithi090 ceph-mon[29180]: pgmap v10171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:32:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17822.253778 secs 2024-04-26T23:32:21.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17823.026873 secs 2024-04-26T23:32:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17822.253778 secs 2024-04-26T23:32:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17823.026873 secs 2024-04-26T23:32:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17822.253778 secs 2024-04-26T23:32:21.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17823.026873 secs 2024-04-26T23:32:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:22 smithi119 ceph-mon[32752]: pgmap v10172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:32:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:22 smithi167 ceph-mon[32751]: pgmap v10172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:32:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:22 smithi090 ceph-mon[29180]: pgmap v10172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:32:24.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:24 smithi119 ceph-mon[32752]: pgmap v10173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:24.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:24 smithi167 ceph-mon[32751]: pgmap v10173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:24.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:24 smithi090 ceph-mon[29180]: pgmap v10173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:26 smithi119 ceph-mon[32752]: pgmap v10174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:26.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17827.253866 secs 2024-04-26T23:32:26.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17828.026990 secs 2024-04-26T23:32:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:26 smithi167 ceph-mon[32751]: pgmap v10174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17827.253866 secs 2024-04-26T23:32:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17828.026990 secs 2024-04-26T23:32:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:26 smithi090 ceph-mon[29180]: pgmap v10174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:26.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17827.253866 secs 2024-04-26T23:32:26.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17828.026990 secs 2024-04-26T23:32:28.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:28 smithi119 ceph-mon[32752]: pgmap v10175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:28.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:28 smithi167 ceph-mon[32751]: pgmap v10175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:28 smithi090 ceph-mon[29180]: pgmap v10175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:30.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:30 smithi167 ceph-mon[32751]: pgmap v10176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:30.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:30 smithi119 ceph-mon[32752]: pgmap v10176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:30 smithi090 ceph-mon[29180]: pgmap v10176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17832.253948 secs 2024-04-26T23:32:31.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17833.027104 secs 2024-04-26T23:32:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17832.253948 secs 2024-04-26T23:32:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17833.027104 secs 2024-04-26T23:32:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17832.253948 secs 2024-04-26T23:32:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17833.027104 secs 2024-04-26T23:32:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:32 smithi119 ceph-mon[32752]: pgmap v10177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:32 smithi167 ceph-mon[32751]: pgmap v10177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:32 smithi090 ceph-mon[29180]: pgmap v10177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:34.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:34 smithi119 ceph-mon[32752]: pgmap v10178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:34.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:34 smithi167 ceph-mon[32751]: pgmap v10178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:34 smithi090 ceph-mon[29180]: pgmap v10178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:36.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:36 smithi119 ceph-mon[32752]: pgmap v10179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:36.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17837.254010 secs 2024-04-26T23:32:36.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17838.027237 secs 2024-04-26T23:32:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:36 smithi167 ceph-mon[32751]: pgmap v10179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17837.254010 secs 2024-04-26T23:32:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17838.027237 secs 2024-04-26T23:32:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:36 smithi090 ceph-mon[29180]: pgmap v10179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17837.254010 secs 2024-04-26T23:32:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17838.027237 secs 2024-04-26T23:32:38.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:38 smithi119 ceph-mon[32752]: pgmap v10180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:38.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:38 smithi167 ceph-mon[32751]: pgmap v10180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:38 smithi090 ceph-mon[29180]: pgmap v10180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:40.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:40 smithi119 ceph-mon[32752]: pgmap v10181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:40.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:40 smithi167 ceph-mon[32751]: pgmap v10181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:40.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:40 smithi090 ceph-mon[29180]: pgmap v10181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:41.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17842.254093 secs 2024-04-26T23:32:41.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17843.027400 secs 2024-04-26T23:32:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17842.254093 secs 2024-04-26T23:32:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17843.027400 secs 2024-04-26T23:32:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17842.254093 secs 2024-04-26T23:32:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17843.027400 secs 2024-04-26T23:32:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:42 smithi119 ceph-mon[32752]: pgmap v10182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:42 smithi167 ceph-mon[32751]: pgmap v10182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:42 smithi090 ceph-mon[29180]: pgmap v10182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:44 smithi119 ceph-mon[32752]: pgmap v10183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:45.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:44 smithi167 ceph-mon[32751]: pgmap v10183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:45.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:44 smithi090 ceph-mon[29180]: pgmap v10183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:46 smithi119 ceph-mon[32752]: pgmap v10184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17847.254177 secs 2024-04-26T23:32:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17848.027490 secs 2024-04-26T23:32:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:46 smithi167 ceph-mon[32751]: pgmap v10184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17847.254177 secs 2024-04-26T23:32:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17848.027490 secs 2024-04-26T23:32:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:46 smithi090 ceph-mon[29180]: pgmap v10184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17847.254177 secs 2024-04-26T23:32:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17848.027490 secs 2024-04-26T23:32:49.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:48 smithi119 ceph-mon[32752]: pgmap v10185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:49.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:48 smithi167 ceph-mon[32751]: pgmap v10185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:49.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:48 smithi090 ceph-mon[29180]: pgmap v10185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:50 smithi119 ceph-mon[32752]: pgmap v10186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:50 smithi167 ceph-mon[32751]: pgmap v10186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:50 smithi090 ceph-mon[29180]: pgmap v10186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17852.254261 secs 2024-04-26T23:32:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17853.027641 secs 2024-04-26T23:32:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17852.254261 secs 2024-04-26T23:32:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17853.027641 secs 2024-04-26T23:32:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17852.254261 secs 2024-04-26T23:32:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17853.027641 secs 2024-04-26T23:32:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:52 smithi119 ceph-mon[32752]: pgmap v10187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:53.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:52 smithi167 ceph-mon[32751]: pgmap v10187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:52 smithi090 ceph-mon[29180]: pgmap v10187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:55.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:54 smithi119 ceph-mon[32752]: pgmap v10188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:55.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:54 smithi167 ceph-mon[32751]: pgmap v10188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:55.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:54 smithi090 ceph-mon[29180]: pgmap v10188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:56 smithi119 ceph-mon[32752]: pgmap v10189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17857.254344 secs 2024-04-26T23:32:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17858.027759 secs 2024-04-26T23:32:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:56 smithi167 ceph-mon[32751]: pgmap v10189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17857.254344 secs 2024-04-26T23:32:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17858.027759 secs 2024-04-26T23:32:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:56 smithi090 ceph-mon[29180]: pgmap v10189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:32:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17857.254344 secs 2024-04-26T23:32:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17858.027759 secs 2024-04-26T23:32:59.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:32:58 smithi119 ceph-mon[32752]: pgmap v10190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:59.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:32:58 smithi167 ceph-mon[32751]: pgmap v10190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:32:59.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:32:58 smithi090 ceph-mon[29180]: pgmap v10190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:00 smithi119 ceph-mon[32752]: pgmap v10191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:00 smithi167 ceph-mon[32751]: pgmap v10191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:00 smithi090 ceph-mon[29180]: pgmap v10191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17862.254430 secs 2024-04-26T23:33:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17863.027838 secs 2024-04-26T23:33:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17862.254430 secs 2024-04-26T23:33:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17863.027838 secs 2024-04-26T23:33:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17862.254430 secs 2024-04-26T23:33:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17863.027838 secs 2024-04-26T23:33:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:02 smithi119 ceph-mon[32752]: pgmap v10192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:02 smithi167 ceph-mon[32751]: pgmap v10192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:02 smithi090 ceph-mon[29180]: pgmap v10192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:04 smithi119 ceph-mon[32752]: pgmap v10193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:04 smithi167 ceph-mon[32751]: pgmap v10193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:05.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:04 smithi090 ceph-mon[29180]: pgmap v10193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:06 smithi119 ceph-mon[32752]: pgmap v10194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17867.254516 secs 2024-04-26T23:33:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17868.027955 secs 2024-04-26T23:33:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:06 smithi167 ceph-mon[32751]: pgmap v10194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17867.254516 secs 2024-04-26T23:33:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17868.027955 secs 2024-04-26T23:33:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:06 smithi090 ceph-mon[29180]: pgmap v10194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17867.254516 secs 2024-04-26T23:33:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17868.027955 secs 2024-04-26T23:33:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:08 smithi119 ceph-mon[32752]: pgmap v10195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:08 smithi167 ceph-mon[32751]: pgmap v10195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:08 smithi090 ceph-mon[29180]: pgmap v10195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:10 smithi119 ceph-mon[32752]: pgmap v10196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:10 smithi167 ceph-mon[32751]: pgmap v10196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:10 smithi090 ceph-mon[29180]: pgmap v10196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:12.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17872.254600 secs 2024-04-26T23:33:12.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17873.028091 secs 2024-04-26T23:33:12.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:33:12.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:33:12.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:33:12.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:33:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17872.254600 secs 2024-04-26T23:33:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17873.028091 secs 2024-04-26T23:33:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:33:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:33:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:33:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:33:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17872.254600 secs 2024-04-26T23:33:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17873.028091 secs 2024-04-26T23:33:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:33:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:33:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:33:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:33:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:12 smithi119 ceph-mon[32752]: pgmap v10197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:12 smithi119 ceph-mon[32752]: pgmap v10198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:33:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:12 smithi119 ceph-mon[32752]: pgmap v10199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:12 smithi167 ceph-mon[32751]: pgmap v10197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:12 smithi167 ceph-mon[32751]: pgmap v10198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:33:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:12 smithi167 ceph-mon[32751]: pgmap v10199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:12 smithi090 ceph-mon[29180]: pgmap v10197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:12 smithi090 ceph-mon[29180]: pgmap v10198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:33:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:12 smithi090 ceph-mon[29180]: pgmap v10199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:14 smithi119 ceph-mon[32752]: pgmap v10200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:15.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:14 smithi167 ceph-mon[32751]: pgmap v10200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:15.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:14 smithi090 ceph-mon[29180]: pgmap v10200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17877.254684 secs 2024-04-26T23:33:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:16 smithi119 ceph-mon[32752]: pgmap v10201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17878.028157 secs 2024-04-26T23:33:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17877.254684 secs 2024-04-26T23:33:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:16 smithi167 ceph-mon[32751]: pgmap v10201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17878.028157 secs 2024-04-26T23:33:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17877.254684 secs 2024-04-26T23:33:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:16 smithi090 ceph-mon[29180]: pgmap v10201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17878.028157 secs 2024-04-26T23:33:19.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:18 smithi119 ceph-mon[32752]: pgmap v10202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:33:19.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:18 smithi167 ceph-mon[32751]: pgmap v10202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:33:19.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:18 smithi090 ceph-mon[29180]: pgmap v10202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:33:21.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:20 smithi119 ceph-mon[32752]: pgmap v10203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:21.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:20 smithi167 ceph-mon[32751]: pgmap v10203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:20 smithi090 ceph-mon[29180]: pgmap v10203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:33:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17882.254767 secs 2024-04-26T23:33:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17883.028294 secs 2024-04-26T23:33:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17882.254767 secs 2024-04-26T23:33:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17883.028294 secs 2024-04-26T23:33:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17882.254767 secs 2024-04-26T23:33:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17883.028294 secs 2024-04-26T23:33:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:22 smithi119 ceph-mon[32752]: pgmap v10204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:33:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:22 smithi167 ceph-mon[32751]: pgmap v10204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:33:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:22 smithi090 ceph-mon[29180]: pgmap v10204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:33:25.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:24 smithi119 ceph-mon[32752]: pgmap v10205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:25.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:24 smithi167 ceph-mon[32751]: pgmap v10205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:24 smithi090 ceph-mon[29180]: pgmap v10205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17887.254852 secs 2024-04-26T23:33:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:26 smithi119 ceph-mon[32752]: pgmap v10206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17888.028432 secs 2024-04-26T23:33:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17887.254852 secs 2024-04-26T23:33:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:26 smithi167 ceph-mon[32751]: pgmap v10206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17888.028432 secs 2024-04-26T23:33:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17887.254852 secs 2024-04-26T23:33:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:26 smithi090 ceph-mon[29180]: pgmap v10206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17888.028432 secs 2024-04-26T23:33:29.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:28 smithi119 ceph-mon[32752]: pgmap v10207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:29.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:28 smithi167 ceph-mon[32751]: pgmap v10207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:29.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:28 smithi090 ceph-mon[29180]: pgmap v10207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:30 smithi167 ceph-mon[32751]: pgmap v10208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:31.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:30 smithi090 ceph-mon[29180]: pgmap v10208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:30 smithi119 ceph-mon[32752]: pgmap v10208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17892.254965 secs 2024-04-26T23:33:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17893.028539 secs 2024-04-26T23:33:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17892.254965 secs 2024-04-26T23:33:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17893.028539 secs 2024-04-26T23:33:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17892.254965 secs 2024-04-26T23:33:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17893.028539 secs 2024-04-26T23:33:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:32 smithi167 ceph-mon[32751]: pgmap v10209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:32 smithi090 ceph-mon[29180]: pgmap v10209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:33.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:32 smithi119 ceph-mon[32752]: pgmap v10209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:35.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:34 smithi167 ceph-mon[32751]: pgmap v10210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:34 smithi090 ceph-mon[29180]: pgmap v10210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:35.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:34 smithi119 ceph-mon[32752]: pgmap v10210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17897.255026 secs 2024-04-26T23:33:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:36 smithi090 ceph-mon[29180]: pgmap v10211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17898.028663 secs 2024-04-26T23:33:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17897.255026 secs 2024-04-26T23:33:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:36 smithi119 ceph-mon[32752]: pgmap v10211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17898.028663 secs 2024-04-26T23:33:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17897.255026 secs 2024-04-26T23:33:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:36 smithi167 ceph-mon[32751]: pgmap v10211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17898.028663 secs 2024-04-26T23:33:39.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:38 smithi090 ceph-mon[29180]: pgmap v10212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:39.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:38 smithi119 ceph-mon[32752]: pgmap v10212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:39.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:38 smithi167 ceph-mon[32751]: pgmap v10212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:40 smithi090 ceph-mon[29180]: pgmap v10213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:40 smithi119 ceph-mon[32752]: pgmap v10213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:40 smithi167 ceph-mon[32751]: pgmap v10213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17902.255087 secs 2024-04-26T23:33:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17903.028758 secs 2024-04-26T23:33:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17902.255087 secs 2024-04-26T23:33:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17903.028758 secs 2024-04-26T23:33:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17902.255087 secs 2024-04-26T23:33:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17903.028758 secs 2024-04-26T23:33:43.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:42 smithi090 ceph-mon[29180]: pgmap v10214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:42 smithi119 ceph-mon[32752]: pgmap v10214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:42 smithi167 ceph-mon[32751]: pgmap v10214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:45.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:44 smithi090 ceph-mon[29180]: pgmap v10215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:44 smithi119 ceph-mon[32752]: pgmap v10215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:45.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:44 smithi167 ceph-mon[32751]: pgmap v10215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17907.255170 secs 2024-04-26T23:33:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:46 smithi090 ceph-mon[29180]: pgmap v10216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17908.028868 secs 2024-04-26T23:33:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17907.255170 secs 2024-04-26T23:33:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:46 smithi119 ceph-mon[32752]: pgmap v10216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17908.028868 secs 2024-04-26T23:33:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17907.255170 secs 2024-04-26T23:33:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:46 smithi167 ceph-mon[32751]: pgmap v10216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17908.028868 secs 2024-04-26T23:33:49.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:48 smithi090 ceph-mon[29180]: pgmap v10217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:48 smithi119 ceph-mon[32752]: pgmap v10217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:49.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:48 smithi167 ceph-mon[32751]: pgmap v10217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:51.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:50 smithi090 ceph-mon[29180]: pgmap v10218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:50 smithi119 ceph-mon[32752]: pgmap v10218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:51.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:50 smithi167 ceph-mon[32751]: pgmap v10218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17912.255244 secs 2024-04-26T23:33:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17913.028959 secs 2024-04-26T23:33:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17912.255244 secs 2024-04-26T23:33:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17913.028959 secs 2024-04-26T23:33:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17912.255244 secs 2024-04-26T23:33:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17913.028959 secs 2024-04-26T23:33:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:52 smithi090 ceph-mon[29180]: pgmap v10219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:52 smithi119 ceph-mon[32752]: pgmap v10219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:53.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:52 smithi167 ceph-mon[32751]: pgmap v10219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:55.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:54 smithi167 ceph-mon[32751]: pgmap v10220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:55.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:54 smithi090 ceph-mon[29180]: pgmap v10220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:55.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:54 smithi119 ceph-mon[32752]: pgmap v10220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:33:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17917.255328 secs 2024-04-26T23:33:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:56 smithi090 ceph-mon[29180]: pgmap v10221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17918.029095 secs 2024-04-26T23:33:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17917.255328 secs 2024-04-26T23:33:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:56 smithi119 ceph-mon[32752]: pgmap v10221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17918.029095 secs 2024-04-26T23:33:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17917.255328 secs 2024-04-26T23:33:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:56 smithi167 ceph-mon[32751]: pgmap v10221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17918.029095 secs 2024-04-26T23:33:59.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:33:58 smithi090 ceph-mon[29180]: pgmap v10222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:33:58 smithi119 ceph-mon[32752]: pgmap v10222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:33:59.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:33:58 smithi167 ceph-mon[32751]: pgmap v10222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:01.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:00 smithi090 ceph-mon[29180]: pgmap v10223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:00 smithi119 ceph-mon[32752]: pgmap v10223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:00 smithi167 ceph-mon[32751]: pgmap v10223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17922.255409 secs 2024-04-26T23:34:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17923.029230 secs 2024-04-26T23:34:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17922.255409 secs 2024-04-26T23:34:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17923.029230 secs 2024-04-26T23:34:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17922.255409 secs 2024-04-26T23:34:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17923.029230 secs 2024-04-26T23:34:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:02 smithi090 ceph-mon[29180]: pgmap v10224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:02 smithi119 ceph-mon[32752]: pgmap v10224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:03.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:02 smithi167 ceph-mon[32751]: pgmap v10224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:04 smithi090 ceph-mon[29180]: pgmap v10225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:04 smithi119 ceph-mon[32752]: pgmap v10225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:04 smithi167 ceph-mon[32751]: pgmap v10225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:06.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:34:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:34:05.668+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:34:06.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:34:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:34:05.668+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:34:06.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:34:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:34:05.668+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:34:06.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:34:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:34:05.668+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:34:06.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:34:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:34:05.668+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:34:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17927.255498 secs 2024-04-26T23:34:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:06 smithi090 ceph-mon[29180]: pgmap v10226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17928.029343 secs 2024-04-26T23:34:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17927.255498 secs 2024-04-26T23:34:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:06 smithi119 ceph-mon[32752]: pgmap v10226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17928.029343 secs 2024-04-26T23:34:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17927.255498 secs 2024-04-26T23:34:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:06 smithi167 ceph-mon[32751]: pgmap v10226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17928.029343 secs 2024-04-26T23:34:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:08 smithi119 ceph-mon[32752]: pgmap v10227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:08 smithi167 ceph-mon[32751]: pgmap v10227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:08 smithi090 ceph-mon[29180]: pgmap v10227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:10 smithi119 ceph-mon[32752]: pgmap v10228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:34:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:10 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17932.255579 secs 2024-04-26T23:34:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:10 smithi167 ceph-mon[32751]: pgmap v10228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:34:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:10 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17932.255579 secs 2024-04-26T23:34:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:10 smithi090 ceph-mon[29180]: pgmap v10228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T23:34:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:10 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17932.255579 secs 2024-04-26T23:34:12.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17933.029456 secs 2024-04-26T23:34:12.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:34:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17933.029456 secs 2024-04-26T23:34:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:34:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17933.029456 secs 2024-04-26T23:34:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:34:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:12 smithi119 ceph-mon[32752]: pgmap v10229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:34:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:34:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:34:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:34:13.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:12 smithi167 ceph-mon[32751]: pgmap v10229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:34:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:34:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:34:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:34:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:12 smithi090 ceph-mon[29180]: pgmap v10229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-26T23:34:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:34:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:34:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:34:14.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:13 smithi119 ceph-mon[32752]: pgmap v10230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 570 B/s wr, 2 op/s 2024-04-26T23:34:14.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:13 smithi119 ceph-mon[32752]: pgmap v10231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 B/s rd, 700 B/s wr, 1 op/s 2024-04-26T23:34:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:13 smithi167 ceph-mon[32751]: pgmap v10230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 570 B/s wr, 2 op/s 2024-04-26T23:34:14.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:13 smithi167 ceph-mon[32751]: pgmap v10231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 B/s rd, 700 B/s wr, 1 op/s 2024-04-26T23:34:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:13 smithi090 ceph-mon[29180]: pgmap v10230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 570 B/s wr, 2 op/s 2024-04-26T23:34:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:13 smithi090 ceph-mon[29180]: pgmap v10231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 700 B/s rd, 700 B/s wr, 1 op/s 2024-04-26T23:34:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:14 smithi119 ceph-mon[32752]: pgmap v10232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 700 B/s wr, 3 op/s 2024-04-26T23:34:15.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:14 smithi167 ceph-mon[32751]: pgmap v10232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 700 B/s wr, 3 op/s 2024-04-26T23:34:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:14 smithi090 ceph-mon[29180]: pgmap v10232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 700 B/s wr, 3 op/s 2024-04-26T23:34:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:15 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17937.255662 secs 2024-04-26T23:34:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:15 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17937.255662 secs 2024-04-26T23:34:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:15 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17937.255662 secs 2024-04-26T23:34:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17938.029551 secs 2024-04-26T23:34:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:16 smithi119 ceph-mon[32752]: pgmap v10233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 700 B/s wr, 3 op/s 2024-04-26T23:34:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17938.029551 secs 2024-04-26T23:34:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:16 smithi167 ceph-mon[32751]: pgmap v10233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 700 B/s wr, 3 op/s 2024-04-26T23:34:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17938.029551 secs 2024-04-26T23:34:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:16 smithi090 ceph-mon[29180]: pgmap v10233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 700 B/s wr, 3 op/s 2024-04-26T23:34:19.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:19 smithi119 ceph-mon[32752]: pgmap v10234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:34:19.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:19 smithi167 ceph-mon[32751]: pgmap v10234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:34:19.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:19 smithi090 ceph-mon[29180]: pgmap v10234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:34:21.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17942.255738 secs 2024-04-26T23:34:21.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:21 smithi119 ceph-mon[32752]: pgmap v10235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:34:21.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17943.029669 secs 2024-04-26T23:34:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17942.255738 secs 2024-04-26T23:34:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:21 smithi167 ceph-mon[32751]: pgmap v10235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:34:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17943.029669 secs 2024-04-26T23:34:21.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17942.255738 secs 2024-04-26T23:34:21.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:21 smithi090 ceph-mon[29180]: pgmap v10235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:34:21.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17943.029669 secs 2024-04-26T23:34:23.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:23 smithi119 ceph-mon[32752]: pgmap v10236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:34:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:23 smithi167 ceph-mon[32751]: pgmap v10236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:34:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:23 smithi090 ceph-mon[29180]: pgmap v10236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:34:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:25 smithi119 ceph-mon[32752]: pgmap v10237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:25 smithi167 ceph-mon[32751]: pgmap v10237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:25 smithi090 ceph-mon[29180]: pgmap v10237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17947.255818 secs 2024-04-26T23:34:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17948.029743 secs 2024-04-26T23:34:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17947.255818 secs 2024-04-26T23:34:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17948.029743 secs 2024-04-26T23:34:26.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17947.255818 secs 2024-04-26T23:34:26.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17948.029743 secs 2024-04-26T23:34:27.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:27 smithi119 ceph-mon[32752]: pgmap v10238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:27 smithi167 ceph-mon[32751]: pgmap v10238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:27 smithi090 ceph-mon[29180]: pgmap v10238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:29.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:29 smithi119 ceph-mon[32752]: pgmap v10239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:29.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:29 smithi167 ceph-mon[32751]: pgmap v10239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:29 smithi090 ceph-mon[29180]: pgmap v10239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17952.255881 secs 2024-04-26T23:34:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:31 smithi119 ceph-mon[32752]: pgmap v10240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17953.029857 secs 2024-04-26T23:34:31.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17952.255881 secs 2024-04-26T23:34:31.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:31 smithi167 ceph-mon[32751]: pgmap v10240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:31.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17953.029857 secs 2024-04-26T23:34:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17952.255881 secs 2024-04-26T23:34:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:31 smithi090 ceph-mon[29180]: pgmap v10240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17953.029857 secs 2024-04-26T23:34:33.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:33 smithi119 ceph-mon[32752]: pgmap v10241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:33.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:33 smithi167 ceph-mon[32751]: pgmap v10241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:33.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:33 smithi090 ceph-mon[29180]: pgmap v10241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:35 smithi119 ceph-mon[32752]: pgmap v10242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:35 smithi167 ceph-mon[32751]: pgmap v10242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:35 smithi090 ceph-mon[29180]: pgmap v10242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17957.255966 secs 2024-04-26T23:34:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17958.029979 secs 2024-04-26T23:34:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17957.255966 secs 2024-04-26T23:34:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17958.029979 secs 2024-04-26T23:34:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17957.255966 secs 2024-04-26T23:34:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17958.029979 secs 2024-04-26T23:34:37.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:37 smithi119 ceph-mon[32752]: pgmap v10243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:37 smithi167 ceph-mon[32751]: pgmap v10243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:37 smithi090 ceph-mon[29180]: pgmap v10243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:39 smithi119 ceph-mon[32752]: pgmap v10244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:39.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:39 smithi167 ceph-mon[32751]: pgmap v10244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:39.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:39 smithi090 ceph-mon[29180]: pgmap v10244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17962.256028 secs 2024-04-26T23:34:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:41 smithi119 ceph-mon[32752]: pgmap v10245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17963.030094 secs 2024-04-26T23:34:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17962.256028 secs 2024-04-26T23:34:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:41 smithi167 ceph-mon[32751]: pgmap v10245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17963.030094 secs 2024-04-26T23:34:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17962.256028 secs 2024-04-26T23:34:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:41 smithi090 ceph-mon[29180]: pgmap v10245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17963.030094 secs 2024-04-26T23:34:43.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:43 smithi119 ceph-mon[32752]: pgmap v10246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:43 smithi167 ceph-mon[32751]: pgmap v10246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:43 smithi090 ceph-mon[29180]: pgmap v10246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:45 smithi167 ceph-mon[32751]: pgmap v10247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:45 smithi090 ceph-mon[29180]: pgmap v10247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:46.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:45 smithi119 ceph-mon[32752]: pgmap v10247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17967.256112 secs 2024-04-26T23:34:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17968.030204 secs 2024-04-26T23:34:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17967.256112 secs 2024-04-26T23:34:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17968.030204 secs 2024-04-26T23:34:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17967.256112 secs 2024-04-26T23:34:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17968.030204 secs 2024-04-26T23:34:47.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:47 smithi090 ceph-mon[29180]: pgmap v10248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:47.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:47 smithi167 ceph-mon[32751]: pgmap v10248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:48.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:47 smithi119 ceph-mon[32752]: pgmap v10248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:49 smithi167 ceph-mon[32751]: pgmap v10249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:49 smithi090 ceph-mon[29180]: pgmap v10249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:50.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:49 smithi119 ceph-mon[32752]: pgmap v10249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17972.256171 secs 2024-04-26T23:34:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:51 smithi167 ceph-mon[32751]: pgmap v10250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17973.030316 secs 2024-04-26T23:34:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17972.256171 secs 2024-04-26T23:34:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:51 smithi090 ceph-mon[29180]: pgmap v10250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17973.030316 secs 2024-04-26T23:34:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17972.256171 secs 2024-04-26T23:34:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:51 smithi119 ceph-mon[32752]: pgmap v10250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17973.030316 secs 2024-04-26T23:34:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:53 smithi090 ceph-mon[29180]: pgmap v10251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:54.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:53 smithi119 ceph-mon[32752]: pgmap v10251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:54.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:53 smithi167 ceph-mon[32751]: pgmap v10251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:55.873 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:55 smithi167 ceph-mon[32751]: pgmap v10252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:55 smithi090 ceph-mon[29180]: pgmap v10252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:56.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:55 smithi119 ceph-mon[32752]: pgmap v10252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:34:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17977.256254 secs 2024-04-26T23:34:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17978.030448 secs 2024-04-26T23:34:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17977.256254 secs 2024-04-26T23:34:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17978.030448 secs 2024-04-26T23:34:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17977.256254 secs 2024-04-26T23:34:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17978.030448 secs 2024-04-26T23:34:57.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:57 smithi090 ceph-mon[29180]: pgmap v10253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:58.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:57 smithi119 ceph-mon[32752]: pgmap v10253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:58.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:57 smithi167 ceph-mon[32751]: pgmap v10253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:34:59.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:34:59 smithi090 ceph-mon[29180]: pgmap v10254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:00.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:34:59 smithi119 ceph-mon[32752]: pgmap v10254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:00.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:34:59 smithi167 ceph-mon[32751]: pgmap v10254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:01.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17982.256338 secs 2024-04-26T23:35:01.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:01 smithi119 ceph-mon[32752]: pgmap v10255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:01.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17983.030523 secs 2024-04-26T23:35:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17982.256338 secs 2024-04-26T23:35:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:01 smithi090 ceph-mon[29180]: pgmap v10255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17983.030523 secs 2024-04-26T23:35:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17982.256338 secs 2024-04-26T23:35:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:01 smithi167 ceph-mon[32751]: pgmap v10255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17983.030523 secs 2024-04-26T23:35:03.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:03 smithi090 ceph-mon[29180]: pgmap v10256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:04.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:03 smithi119 ceph-mon[32752]: pgmap v10256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:03 smithi167 ceph-mon[32751]: pgmap v10256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:05 smithi090 ceph-mon[29180]: pgmap v10257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:05 smithi119 ceph-mon[32752]: pgmap v10257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:05 smithi167 ceph-mon[32751]: pgmap v10257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17987.256397 secs 2024-04-26T23:35:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:06 smithi090 ceph-mon[29180]: 19 slow requests, 2 included below; oldest blocked for > 17988.030634 secs 2024-04-26T23:35:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:06 smithi090 ceph-mon[29180]: slow request 2880.899022 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:35:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:06 smithi090 ceph-mon[29180]: slow request 360.170745 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:35:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17987.256397 secs 2024-04-26T23:35:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:06 smithi119 ceph-mon[32752]: 19 slow requests, 2 included below; oldest blocked for > 17988.030634 secs 2024-04-26T23:35:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:06 smithi119 ceph-mon[32752]: slow request 2880.899022 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:35:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:06 smithi119 ceph-mon[32752]: slow request 360.170745 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:35:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17987.256397 secs 2024-04-26T23:35:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:06 smithi167 ceph-mon[32751]: 19 slow requests, 2 included below; oldest blocked for > 17988.030634 secs 2024-04-26T23:35:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:06 smithi167 ceph-mon[32751]: slow request 2880.899022 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:35:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:06 smithi167 ceph-mon[32751]: slow request 360.170745 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:35:07.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:07 smithi090 ceph-mon[29180]: pgmap v10258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:07 smithi119 ceph-mon[32752]: pgmap v10258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:07 smithi167 ceph-mon[32751]: pgmap v10258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:09 smithi090 ceph-mon[29180]: pgmap v10259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:09 smithi119 ceph-mon[32752]: pgmap v10259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:09 smithi167 ceph-mon[32751]: pgmap v10259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17992.256458 secs 2024-04-26T23:35:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:11 smithi090 ceph-mon[29180]: pgmap v10260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17993.030765 secs 2024-04-26T23:35:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17992.256458 secs 2024-04-26T23:35:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:11 smithi119 ceph-mon[32752]: pgmap v10260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17993.030765 secs 2024-04-26T23:35:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17992.256458 secs 2024-04-26T23:35:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:11 smithi167 ceph-mon[32751]: pgmap v10260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17993.030765 secs 2024-04-26T23:35:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:35:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:35:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:35:13.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:13 smithi090 ceph-mon[29180]: pgmap v10261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:13.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:35:13.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:35:13.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:35:14.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:13 smithi119 ceph-mon[32752]: pgmap v10261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:14.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:35:14.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:35:14.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:35:14.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:13 smithi167 ceph-mon[32751]: pgmap v10261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:14.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:35:14.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:35:14.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:35:14.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:14 smithi090 ceph-mon[29180]: pgmap v10262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:35:14.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:14 smithi090 ceph-mon[29180]: pgmap v10263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:15.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:14 smithi119 ceph-mon[32752]: pgmap v10262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:35:15.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:14 smithi119 ceph-mon[32752]: pgmap v10263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:15.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:14 smithi167 ceph-mon[32751]: pgmap v10262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:35:15.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:14 smithi167 ceph-mon[32751]: pgmap v10263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 17997.256541 secs 2024-04-26T23:35:16.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:16 smithi090 ceph-mon[29180]: pgmap v10264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:16.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 17998.030895 secs 2024-04-26T23:35:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 17997.256541 secs 2024-04-26T23:35:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:16 smithi119 ceph-mon[32752]: pgmap v10264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 17998.030895 secs 2024-04-26T23:35:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 17997.256541 secs 2024-04-26T23:35:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:16 smithi167 ceph-mon[32751]: pgmap v10264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 17998.030895 secs 2024-04-26T23:35:18.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:18 smithi090 ceph-mon[29180]: pgmap v10265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 568 B/s rd, 1 op/s 2024-04-26T23:35:19.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:18 smithi119 ceph-mon[32752]: pgmap v10265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 568 B/s rd, 1 op/s 2024-04-26T23:35:19.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:18 smithi167 ceph-mon[32751]: pgmap v10265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 568 B/s rd, 1 op/s 2024-04-26T23:35:21.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:20 smithi119 ceph-mon[32752]: pgmap v10266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:21.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:20 smithi167 ceph-mon[32751]: pgmap v10266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:20 smithi090 ceph-mon[29180]: pgmap v10266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18002.256623 secs 2024-04-26T23:35:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18003.031015 secs 2024-04-26T23:35:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18002.256623 secs 2024-04-26T23:35:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18003.031015 secs 2024-04-26T23:35:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18002.256623 secs 2024-04-26T23:35:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18003.031015 secs 2024-04-26T23:35:23.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:22 smithi119 ceph-mon[32752]: pgmap v10267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:22 smithi167 ceph-mon[32751]: pgmap v10267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:23.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:22 smithi090 ceph-mon[29180]: pgmap v10267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:35:25.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:24 smithi119 ceph-mon[32752]: pgmap v10268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:35:25.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:24 smithi167 ceph-mon[32751]: pgmap v10268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:35:25.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:24 smithi090 ceph-mon[29180]: pgmap v10268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:35:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18007.256707 secs 2024-04-26T23:35:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:26 smithi119 ceph-mon[32752]: pgmap v10269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18008.031147 secs 2024-04-26T23:35:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18007.256707 secs 2024-04-26T23:35:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:26 smithi167 ceph-mon[32751]: pgmap v10269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18008.031147 secs 2024-04-26T23:35:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18007.256707 secs 2024-04-26T23:35:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:26 smithi090 ceph-mon[29180]: pgmap v10269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18008.031147 secs 2024-04-26T23:35:29.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:28 smithi119 ceph-mon[32752]: pgmap v10270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:29.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:28 smithi167 ceph-mon[32751]: pgmap v10270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:29.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:28 smithi090 ceph-mon[29180]: pgmap v10270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:31.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:30 smithi119 ceph-mon[32752]: pgmap v10271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:31.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:30 smithi167 ceph-mon[32751]: pgmap v10271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:31.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:30 smithi090 ceph-mon[29180]: pgmap v10271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:31.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18012.256800 secs 2024-04-26T23:35:31.959 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18013.031244 secs 2024-04-26T23:35:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18012.256800 secs 2024-04-26T23:35:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18013.031244 secs 2024-04-26T23:35:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18012.256800 secs 2024-04-26T23:35:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18013.031244 secs 2024-04-26T23:35:33.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:32 smithi119 ceph-mon[32752]: pgmap v10272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:32 smithi167 ceph-mon[32751]: pgmap v10272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:33.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:32 smithi090 ceph-mon[29180]: pgmap v10272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:35.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:34 smithi119 ceph-mon[32752]: pgmap v10273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:34 smithi167 ceph-mon[32751]: pgmap v10273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:34 smithi090 ceph-mon[29180]: pgmap v10273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18017.256822 secs 2024-04-26T23:35:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:36 smithi119 ceph-mon[32752]: pgmap v10274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18018.031356 secs 2024-04-26T23:35:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18017.256822 secs 2024-04-26T23:35:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:36 smithi167 ceph-mon[32751]: pgmap v10274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18018.031356 secs 2024-04-26T23:35:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18017.256822 secs 2024-04-26T23:35:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:36 smithi090 ceph-mon[29180]: pgmap v10274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18018.031356 secs 2024-04-26T23:35:39.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:38 smithi119 ceph-mon[32752]: pgmap v10275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:39.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:38 smithi167 ceph-mon[32751]: pgmap v10275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:39.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:38 smithi090 ceph-mon[29180]: pgmap v10275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:40 smithi119 ceph-mon[32752]: pgmap v10276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:41.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:40 smithi167 ceph-mon[32751]: pgmap v10276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:40 smithi090 ceph-mon[29180]: pgmap v10276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18022.256879 secs 2024-04-26T23:35:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18023.031432 secs 2024-04-26T23:35:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18022.256879 secs 2024-04-26T23:35:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18023.031432 secs 2024-04-26T23:35:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18022.256879 secs 2024-04-26T23:35:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18023.031432 secs 2024-04-26T23:35:43.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:42 smithi090 ceph-mon[29180]: pgmap v10277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:42 smithi119 ceph-mon[32752]: pgmap v10277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:42 smithi167 ceph-mon[32751]: pgmap v10277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:45.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:44 smithi119 ceph-mon[32752]: pgmap v10278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:45.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:44 smithi167 ceph-mon[32751]: pgmap v10278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:45.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:44 smithi090 ceph-mon[29180]: pgmap v10278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18027.256943 secs 2024-04-26T23:35:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:46 smithi119 ceph-mon[32752]: pgmap v10279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18028.031471 secs 2024-04-26T23:35:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18027.256943 secs 2024-04-26T23:35:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:46 smithi167 ceph-mon[32751]: pgmap v10279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18028.031471 secs 2024-04-26T23:35:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18027.256943 secs 2024-04-26T23:35:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:46 smithi090 ceph-mon[29180]: pgmap v10279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18028.031471 secs 2024-04-26T23:35:49.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:48 smithi119 ceph-mon[32752]: pgmap v10280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:49.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:48 smithi167 ceph-mon[32751]: pgmap v10280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:49.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:48 smithi090 ceph-mon[29180]: pgmap v10280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:50.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:50 smithi090 ceph-mon[29180]: pgmap v10281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:51.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:50 smithi119 ceph-mon[32752]: pgmap v10281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:51.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:50 smithi167 ceph-mon[32751]: pgmap v10281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18032.257022 secs 2024-04-26T23:35:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18033.031543 secs 2024-04-26T23:35:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18032.257022 secs 2024-04-26T23:35:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18033.031543 secs 2024-04-26T23:35:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18032.257022 secs 2024-04-26T23:35:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18033.031543 secs 2024-04-26T23:35:53.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:52 smithi119 ceph-mon[32752]: pgmap v10282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:53.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:52 smithi167 ceph-mon[32751]: pgmap v10282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:52 smithi090 ceph-mon[29180]: pgmap v10282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:55.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:54 smithi119 ceph-mon[32752]: pgmap v10283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:55.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:54 smithi167 ceph-mon[32751]: pgmap v10283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:55.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:54 smithi090 ceph-mon[29180]: pgmap v10283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:35:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18037.257084 secs 2024-04-26T23:35:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:56 smithi119 ceph-mon[32752]: pgmap v10284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18038.031654 secs 2024-04-26T23:35:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18037.257084 secs 2024-04-26T23:35:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:56 smithi167 ceph-mon[32751]: pgmap v10284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18038.031654 secs 2024-04-26T23:35:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18037.257084 secs 2024-04-26T23:35:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:56 smithi090 ceph-mon[29180]: pgmap v10284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18038.031654 secs 2024-04-26T23:35:59.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:35:58 smithi119 ceph-mon[32752]: pgmap v10285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:59.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:35:58 smithi167 ceph-mon[32751]: pgmap v10285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:35:59.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:35:58 smithi090 ceph-mon[29180]: pgmap v10285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:01.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:00 smithi119 ceph-mon[32752]: pgmap v10286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:00 smithi167 ceph-mon[32751]: pgmap v10286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:01.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:00 smithi090 ceph-mon[29180]: pgmap v10286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18042.257167 secs 2024-04-26T23:36:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18043.031788 secs 2024-04-26T23:36:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18042.257167 secs 2024-04-26T23:36:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18043.031788 secs 2024-04-26T23:36:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18042.257167 secs 2024-04-26T23:36:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18043.031788 secs 2024-04-26T23:36:03.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:02 smithi119 ceph-mon[32752]: pgmap v10287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:03.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:02 smithi167 ceph-mon[32751]: pgmap v10287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:03.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:02 smithi090 ceph-mon[29180]: pgmap v10287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:04 smithi119 ceph-mon[32752]: pgmap v10288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:04 smithi167 ceph-mon[32751]: pgmap v10288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:04 smithi090 ceph-mon[29180]: pgmap v10288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18047.257249 secs 2024-04-26T23:36:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:06 smithi119 ceph-mon[32752]: pgmap v10289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18048.031910 secs 2024-04-26T23:36:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18047.257249 secs 2024-04-26T23:36:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:06 smithi167 ceph-mon[32751]: pgmap v10289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:07.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18048.031910 secs 2024-04-26T23:36:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18047.257249 secs 2024-04-26T23:36:07.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:06 smithi090 ceph-mon[29180]: pgmap v10289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:07.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18048.031910 secs 2024-04-26T23:36:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:08 smithi119 ceph-mon[32752]: pgmap v10290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:08 smithi167 ceph-mon[32751]: pgmap v10290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:09.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:08 smithi090 ceph-mon[29180]: pgmap v10290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:10 smithi119 ceph-mon[32752]: pgmap v10291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:10 smithi167 ceph-mon[32751]: pgmap v10291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:10 smithi090 ceph-mon[29180]: pgmap v10291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18052.257329 secs 2024-04-26T23:36:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18053.032008 secs 2024-04-26T23:36:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18052.257329 secs 2024-04-26T23:36:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18053.032008 secs 2024-04-26T23:36:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18052.257329 secs 2024-04-26T23:36:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18053.032008 secs 2024-04-26T23:36:13.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:12 smithi090 ceph-mon[29180]: pgmap v10292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:12 smithi119 ceph-mon[32752]: pgmap v10292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:12 smithi167 ceph-mon[32751]: pgmap v10292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:14.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:36:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:36:14.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:36:15.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:14 smithi119 ceph-mon[32752]: pgmap v10293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:15.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:36:15.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:36:15.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:36:15.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:14 smithi167 ceph-mon[32751]: pgmap v10293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:15.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:36:15.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:36:15.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:36:15.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:14 smithi090 ceph-mon[29180]: pgmap v10293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:15.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:36:15.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:36:15.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:36:16.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:15 smithi119 ceph-mon[32752]: pgmap v10294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:36:16.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:15 smithi119 ceph-mon[32752]: pgmap v10295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:16.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:15 smithi167 ceph-mon[32751]: pgmap v10294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:36:16.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:15 smithi167 ceph-mon[32751]: pgmap v10295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:16.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:15 smithi090 ceph-mon[29180]: pgmap v10294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:36:16.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:15 smithi090 ceph-mon[29180]: pgmap v10295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18057.257413 secs 2024-04-26T23:36:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18058.032128 secs 2024-04-26T23:36:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18057.257413 secs 2024-04-26T23:36:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18058.032128 secs 2024-04-26T23:36:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18057.257413 secs 2024-04-26T23:36:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18058.032128 secs 2024-04-26T23:36:18.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:17 smithi119 ceph-mon[32752]: pgmap v10296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:17 smithi167 ceph-mon[32751]: pgmap v10296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:17 smithi090 ceph-mon[29180]: pgmap v10296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:20.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:19 smithi119 ceph-mon[32752]: pgmap v10297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:19 smithi167 ceph-mon[32751]: pgmap v10297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:20.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:19 smithi090 ceph-mon[29180]: pgmap v10297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18062.257493 secs 2024-04-26T23:36:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:21 smithi119 ceph-mon[32752]: pgmap v10298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18063.032242 secs 2024-04-26T23:36:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18062.257493 secs 2024-04-26T23:36:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:21 smithi167 ceph-mon[32751]: pgmap v10298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18063.032242 secs 2024-04-26T23:36:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18062.257493 secs 2024-04-26T23:36:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:21 smithi090 ceph-mon[29180]: pgmap v10298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:36:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18063.032242 secs 2024-04-26T23:36:24.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:23 smithi119 ceph-mon[32752]: pgmap v10299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:36:24.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:23 smithi167 ceph-mon[32751]: pgmap v10299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:36:24.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:23 smithi090 ceph-mon[29180]: pgmap v10299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:36:26.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:25 smithi119 ceph-mon[32752]: pgmap v10300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:36:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:25 smithi167 ceph-mon[32751]: pgmap v10300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:36:26.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:25 smithi090 ceph-mon[29180]: pgmap v10300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:36:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18067.257570 secs 2024-04-26T23:36:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18068.032367 secs 2024-04-26T23:36:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18067.257570 secs 2024-04-26T23:36:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18068.032367 secs 2024-04-26T23:36:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18067.257570 secs 2024-04-26T23:36:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18068.032367 secs 2024-04-26T23:36:28.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:27 smithi119 ceph-mon[32752]: pgmap v10301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:28.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:27 smithi167 ceph-mon[32751]: pgmap v10301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:28.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:27 smithi090 ceph-mon[29180]: pgmap v10301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:30.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:29 smithi119 ceph-mon[32752]: pgmap v10302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:30.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:29 smithi167 ceph-mon[32751]: pgmap v10302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:30.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:29 smithi090 ceph-mon[29180]: pgmap v10302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18072.257646 secs 2024-04-26T23:36:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:31 smithi119 ceph-mon[32752]: pgmap v10303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18073.032399 secs 2024-04-26T23:36:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18072.257646 secs 2024-04-26T23:36:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:31 smithi167 ceph-mon[32751]: pgmap v10303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18073.032399 secs 2024-04-26T23:36:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18072.257646 secs 2024-04-26T23:36:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:31 smithi090 ceph-mon[29180]: pgmap v10303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18073.032399 secs 2024-04-26T23:36:34.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:33 smithi119 ceph-mon[32752]: pgmap v10304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:34.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:33 smithi167 ceph-mon[32751]: pgmap v10304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:33 smithi090 ceph-mon[29180]: pgmap v10304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:36.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:35 smithi119 ceph-mon[32752]: pgmap v10305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:35 smithi167 ceph-mon[32751]: pgmap v10305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:36.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:35 smithi090 ceph-mon[29180]: pgmap v10305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18077.257727 secs 2024-04-26T23:36:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18078.032516 secs 2024-04-26T23:36:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18077.257727 secs 2024-04-26T23:36:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18078.032516 secs 2024-04-26T23:36:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18077.257727 secs 2024-04-26T23:36:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18078.032516 secs 2024-04-26T23:36:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:37 smithi090 ceph-mon[29180]: pgmap v10306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:38.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:37 smithi119 ceph-mon[32752]: pgmap v10306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:38.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:37 smithi167 ceph-mon[32751]: pgmap v10306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:40.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:39 smithi167 ceph-mon[32751]: pgmap v10307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:40.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:39 smithi090 ceph-mon[29180]: pgmap v10307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:40.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:39 smithi119 ceph-mon[32752]: pgmap v10307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18082.257808 secs 2024-04-26T23:36:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:41 smithi167 ceph-mon[32751]: pgmap v10308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:41 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18083.032591 secs 2024-04-26T23:36:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18082.257808 secs 2024-04-26T23:36:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:41 smithi090 ceph-mon[29180]: pgmap v10308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:41 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18083.032591 secs 2024-04-26T23:36:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18082.257808 secs 2024-04-26T23:36:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:41 smithi119 ceph-mon[32752]: pgmap v10308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:41 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18083.032591 secs 2024-04-26T23:36:44.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:43 smithi167 ceph-mon[32751]: pgmap v10309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:44.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:43 smithi090 ceph-mon[29180]: pgmap v10309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:44.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:43 smithi119 ceph-mon[32752]: pgmap v10309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:46.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:45 smithi167 ceph-mon[32751]: pgmap v10310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:46.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:45 smithi090 ceph-mon[29180]: pgmap v10310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:46.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:45 smithi119 ceph-mon[32752]: pgmap v10310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18087.257865 secs 2024-04-26T23:36:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:46 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18088.032720 secs 2024-04-26T23:36:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18087.257865 secs 2024-04-26T23:36:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:46 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18088.032720 secs 2024-04-26T23:36:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18087.257865 secs 2024-04-26T23:36:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:46 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18088.032720 secs 2024-04-26T23:36:48.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:47 smithi090 ceph-mon[29180]: pgmap v10311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:48.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:47 smithi119 ceph-mon[32752]: pgmap v10311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:48.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:47 smithi167 ceph-mon[32751]: pgmap v10311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:50.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:49 smithi090 ceph-mon[29180]: pgmap v10312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:49 smithi119 ceph-mon[32752]: pgmap v10312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:50.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:49 smithi167 ceph-mon[32751]: pgmap v10312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18092.257950 secs 2024-04-26T23:36:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:51 smithi090 ceph-mon[29180]: pgmap v10313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18093.032842 secs 2024-04-26T23:36:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18092.257950 secs 2024-04-26T23:36:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:51 smithi119 ceph-mon[32752]: pgmap v10313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18093.032842 secs 2024-04-26T23:36:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18092.257950 secs 2024-04-26T23:36:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:51 smithi167 ceph-mon[32751]: pgmap v10313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18093.032842 secs 2024-04-26T23:36:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:53 smithi090 ceph-mon[29180]: pgmap v10314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:53 smithi119 ceph-mon[32752]: pgmap v10314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:53 smithi167 ceph-mon[32751]: pgmap v10314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:56.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:55 smithi167 ceph-mon[32751]: pgmap v10315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:56.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:55 smithi090 ceph-mon[29180]: pgmap v10315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:56.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:55 smithi119 ceph-mon[32752]: pgmap v10315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:36:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18097.258034 secs 2024-04-26T23:36:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18098.032958 secs 2024-04-26T23:36:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18097.258034 secs 2024-04-26T23:36:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18098.032958 secs 2024-04-26T23:36:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18097.258034 secs 2024-04-26T23:36:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18098.032958 secs 2024-04-26T23:36:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:57 smithi090 ceph-mon[29180]: pgmap v10316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:58.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:57 smithi119 ceph-mon[32752]: pgmap v10316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:36:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:57 smithi167 ceph-mon[32751]: pgmap v10316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:00.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:36:59 smithi090 ceph-mon[29180]: pgmap v10317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:00.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:36:59 smithi119 ceph-mon[32752]: pgmap v10317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:00.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:36:59 smithi167 ceph-mon[32751]: pgmap v10317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:02.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18102.258120 secs 2024-04-26T23:37:02.109 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:01 smithi119 ceph-mon[32752]: pgmap v10318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:02.109 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 18103.033031 secs 2024-04-26T23:37:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18102.258120 secs 2024-04-26T23:37:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:01 smithi090 ceph-mon[29180]: pgmap v10318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 18103.033031 secs 2024-04-26T23:37:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18102.258120 secs 2024-04-26T23:37:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:01 smithi167 ceph-mon[32751]: pgmap v10318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 18103.033031 secs 2024-04-26T23:37:04.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:03 smithi090 ceph-mon[29180]: pgmap v10319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:04.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:03 smithi119 ceph-mon[32752]: pgmap v10319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:04.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:03 smithi167 ceph-mon[32751]: pgmap v10319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:05 smithi090 ceph-mon[29180]: pgmap v10320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:05 smithi119 ceph-mon[32752]: pgmap v10320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:05 smithi167 ceph-mon[32751]: pgmap v10320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18107.258205 secs 2024-04-26T23:37:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:06 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 18108.033104 secs 2024-04-26T23:37:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:06 smithi090 ceph-mon[29180]: slow request 180.105661 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:37:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18107.258205 secs 2024-04-26T23:37:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:06 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 18108.033104 secs 2024-04-26T23:37:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:06 smithi119 ceph-mon[32752]: slow request 180.105661 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:37:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18107.258205 secs 2024-04-26T23:37:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:06 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 18108.033104 secs 2024-04-26T23:37:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:06 smithi167 ceph-mon[32751]: slow request 180.105661 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:37:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:07 smithi090 ceph-mon[29180]: pgmap v10321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:07 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T23:37:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:07 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:37:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:07 smithi119 ceph-mon[32752]: pgmap v10321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:07 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T23:37:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:07 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:37:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:07 smithi167 ceph-mon[32751]: pgmap v10321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:07 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-26T23:37:08.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:07 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:37:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:09 smithi090 ceph-mon[29180]: pgmap v10322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:09 smithi119 ceph-mon[32752]: pgmap v10322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:09 smithi167 ceph-mon[32751]: pgmap v10322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:11 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 18112.258266 secs 2024-04-26T23:37:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:11 smithi090 ceph-mon[29180]: slow request 1444.696894 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:37:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:11 smithi090 ceph-mon[29180]: pgmap v10323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18113.033166 secs 2024-04-26T23:37:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:11 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 18112.258266 secs 2024-04-26T23:37:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:11 smithi119 ceph-mon[32752]: slow request 1444.696894 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:37:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:11 smithi119 ceph-mon[32752]: pgmap v10323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18113.033166 secs 2024-04-26T23:37:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:11 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 18112.258266 secs 2024-04-26T23:37:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:11 smithi167 ceph-mon[32751]: slow request 1444.696894 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:37:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:11 smithi167 ceph-mon[32751]: pgmap v10323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18113.033166 secs 2024-04-26T23:37:14.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:13 smithi090 ceph-mon[29180]: pgmap v10324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:14.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:13 smithi119 ceph-mon[32752]: pgmap v10324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:14.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:13 smithi167 ceph-mon[32751]: pgmap v10324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:37:15.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:37:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:37:15.889 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:15 smithi119 ceph-mon[32752]: pgmap v10325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:15.890 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:37:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:37:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:37:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:15 smithi167 ceph-mon[32751]: pgmap v10325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:37:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:37:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:37:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:15 smithi090 ceph-mon[29180]: pgmap v10325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:37:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:37:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:37:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18117.258350 secs 2024-04-26T23:37:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:16 smithi119 ceph-mon[32752]: pgmap v10326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:37:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:16 smithi119 ceph-mon[32752]: pgmap v10327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18118.033227 secs 2024-04-26T23:37:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18117.258350 secs 2024-04-26T23:37:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:16 smithi167 ceph-mon[32751]: pgmap v10326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:37:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:16 smithi167 ceph-mon[32751]: pgmap v10327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18118.033227 secs 2024-04-26T23:37:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18117.258350 secs 2024-04-26T23:37:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:16 smithi090 ceph-mon[29180]: pgmap v10326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:37:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:16 smithi090 ceph-mon[29180]: pgmap v10327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18118.033227 secs 2024-04-26T23:37:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:18 smithi119 ceph-mon[32752]: pgmap v10328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:37:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:18 smithi167 ceph-mon[32751]: pgmap v10328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:37:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:18 smithi090 ceph-mon[29180]: pgmap v10328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:37:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:20 smithi119 ceph-mon[32752]: pgmap v10329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:20 smithi167 ceph-mon[32751]: pgmap v10329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:20 smithi090 ceph-mon[29180]: pgmap v10329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18122.258409 secs 2024-04-26T23:37:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18123.033343 secs 2024-04-26T23:37:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18122.258409 secs 2024-04-26T23:37:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18123.033343 secs 2024-04-26T23:37:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18122.258409 secs 2024-04-26T23:37:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18123.033343 secs 2024-04-26T23:37:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:22 smithi119 ceph-mon[32752]: pgmap v10330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:22 smithi167 ceph-mon[32751]: pgmap v10330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:22 smithi090 ceph-mon[29180]: pgmap v10330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:37:25.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:24 smithi119 ceph-mon[32752]: pgmap v10331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:37:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:24 smithi167 ceph-mon[32751]: pgmap v10331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:37:25.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:24 smithi090 ceph-mon[29180]: pgmap v10331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:37:26.222 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:25 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18127.258494 secs 2024-04-26T23:37:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:25 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18127.258494 secs 2024-04-26T23:37:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:25 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18127.258494 secs 2024-04-26T23:37:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:26 smithi119 ceph-mon[32752]: pgmap v10332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:37:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18128.033469 secs 2024-04-26T23:37:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:26 smithi167 ceph-mon[32751]: pgmap v10332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:37:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18128.033469 secs 2024-04-26T23:37:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:26 smithi090 ceph-mon[29180]: pgmap v10332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:37:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18128.033469 secs 2024-04-26T23:37:29.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:28 smithi119 ceph-mon[32752]: pgmap v10333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:29.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:28 smithi167 ceph-mon[32751]: pgmap v10333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:29.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:28 smithi090 ceph-mon[29180]: pgmap v10333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:30 smithi119 ceph-mon[32752]: pgmap v10334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:31.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:30 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18132.258554 secs 2024-04-26T23:37:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:30 smithi167 ceph-mon[32751]: pgmap v10334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:30 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18132.258554 secs 2024-04-26T23:37:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:30 smithi090 ceph-mon[29180]: pgmap v10334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:30 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18132.258554 secs 2024-04-26T23:37:32.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18133.033581 secs 2024-04-26T23:37:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18133.033581 secs 2024-04-26T23:37:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18133.033581 secs 2024-04-26T23:37:33.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:32 smithi119 ceph-mon[32752]: pgmap v10335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:33.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:32 smithi167 ceph-mon[32751]: pgmap v10335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:33.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:32 smithi090 ceph-mon[29180]: pgmap v10335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:33 smithi119 ceph-mon[32752]: pgmap v10336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:34.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:33 smithi167 ceph-mon[32751]: pgmap v10336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:34.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:33 smithi090 ceph-mon[29180]: pgmap v10336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18137.258653 secs 2024-04-26T23:37:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:36 smithi090 ceph-mon[29180]: pgmap v10337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:36.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18138.033727 secs 2024-04-26T23:37:36.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18137.258653 secs 2024-04-26T23:37:36.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:36 smithi119 ceph-mon[32752]: pgmap v10337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:36.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18138.033727 secs 2024-04-26T23:37:36.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18137.258653 secs 2024-04-26T23:37:36.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:36 smithi167 ceph-mon[32751]: pgmap v10337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:36.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18138.033727 secs 2024-04-26T23:37:38.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:38 smithi090 ceph-mon[29180]: pgmap v10338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:38.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:38 smithi119 ceph-mon[32752]: pgmap v10338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:38.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:38 smithi167 ceph-mon[32751]: pgmap v10338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:40.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:40 smithi090 ceph-mon[29180]: pgmap v10339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:40.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:40 smithi119 ceph-mon[32752]: pgmap v10339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:40.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:40 smithi167 ceph-mon[32751]: pgmap v10339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:41.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18142.258752 secs 2024-04-26T23:37:41.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18143.033864 secs 2024-04-26T23:37:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18142.258752 secs 2024-04-26T23:37:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18143.033864 secs 2024-04-26T23:37:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18142.258752 secs 2024-04-26T23:37:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18143.033864 secs 2024-04-26T23:37:42.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:42 smithi090 ceph-mon[29180]: pgmap v10340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:42.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:42 smithi119 ceph-mon[32752]: pgmap v10340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:42.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:42 smithi167 ceph-mon[32751]: pgmap v10340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:44.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:44 smithi090 ceph-mon[29180]: pgmap v10341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:44.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:44 smithi119 ceph-mon[32752]: pgmap v10341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:44.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:44 smithi167 ceph-mon[32751]: pgmap v10341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:46 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 18147.258822 secs 2024-04-26T23:37:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:46 smithi090 ceph-mon[29180]: slow request 1440.028154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:37:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:46 smithi090 ceph-mon[29180]: pgmap v10342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18148.033975 secs 2024-04-26T23:37:46.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:46 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 18147.258822 secs 2024-04-26T23:37:46.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:46 smithi119 ceph-mon[32752]: slow request 1440.028154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:37:46.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:46 smithi119 ceph-mon[32752]: pgmap v10342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:46.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18148.033975 secs 2024-04-26T23:37:46.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:46 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 18147.258822 secs 2024-04-26T23:37:46.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:46 smithi167 ceph-mon[32751]: slow request 1440.028154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:37:46.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:46 smithi167 ceph-mon[32751]: pgmap v10342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:46.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18148.033975 secs 2024-04-26T23:37:48.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:48 smithi119 ceph-mon[32752]: pgmap v10343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:48.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:48 smithi167 ceph-mon[32751]: pgmap v10343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:48.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:48 smithi090 ceph-mon[29180]: pgmap v10343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:50.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:50 smithi119 ceph-mon[32752]: pgmap v10344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:50.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:50 smithi167 ceph-mon[32751]: pgmap v10344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:50.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:50 smithi090 ceph-mon[29180]: pgmap v10344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:51.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18152.258905 secs 2024-04-26T23:37:51.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18153.034117 secs 2024-04-26T23:37:51.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18152.258905 secs 2024-04-26T23:37:51.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18153.034117 secs 2024-04-26T23:37:51.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18152.258905 secs 2024-04-26T23:37:51.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18153.034117 secs 2024-04-26T23:37:52.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:52 smithi119 ceph-mon[32752]: pgmap v10345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:52.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:52 smithi167 ceph-mon[32751]: pgmap v10345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:52 smithi090 ceph-mon[29180]: pgmap v10345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:54.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:54 smithi090 ceph-mon[29180]: pgmap v10346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:54.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:54 smithi119 ceph-mon[32752]: pgmap v10346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:54.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:54 smithi167 ceph-mon[32751]: pgmap v10346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:56.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18157.258964 secs 2024-04-26T23:37:56.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:56 smithi119 ceph-mon[32752]: pgmap v10347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:56.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18158.034198 secs 2024-04-26T23:37:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18157.258964 secs 2024-04-26T23:37:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:56 smithi167 ceph-mon[32751]: pgmap v10347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18158.034198 secs 2024-04-26T23:37:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18157.258964 secs 2024-04-26T23:37:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:56 smithi090 ceph-mon[29180]: pgmap v10347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:37:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18158.034198 secs 2024-04-26T23:37:58.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:37:58 smithi119 ceph-mon[32752]: pgmap v10348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:58.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:37:58 smithi167 ceph-mon[32751]: pgmap v10348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:37:58.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:37:58 smithi090 ceph-mon[29180]: pgmap v10348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:00.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:00 smithi119 ceph-mon[32752]: pgmap v10349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:00.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:00 smithi167 ceph-mon[32751]: pgmap v10349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:00.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:00 smithi090 ceph-mon[29180]: pgmap v10349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:01.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18162.259024 secs 2024-04-26T23:38:01.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18163.034312 secs 2024-04-26T23:38:01.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18162.259024 secs 2024-04-26T23:38:01.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18163.034312 secs 2024-04-26T23:38:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18162.259024 secs 2024-04-26T23:38:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18163.034312 secs 2024-04-26T23:38:02.458 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:02 smithi119 ceph-mon[32752]: pgmap v10350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:02.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:02 smithi167 ceph-mon[32751]: pgmap v10350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:02.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:02 smithi090 ceph-mon[29180]: pgmap v10350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:04.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:04 smithi119 ceph-mon[32752]: pgmap v10351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:04.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:04 smithi167 ceph-mon[32751]: pgmap v10351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:04.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:04 smithi090 ceph-mon[29180]: pgmap v10351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:06.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18167.259111 secs 2024-04-26T23:38:06.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:06 smithi119 ceph-mon[32752]: pgmap v10352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:06.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18168.034408 secs 2024-04-26T23:38:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18167.259111 secs 2024-04-26T23:38:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:06 smithi167 ceph-mon[32751]: pgmap v10352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18168.034408 secs 2024-04-26T23:38:06.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18167.259111 secs 2024-04-26T23:38:06.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:06 smithi090 ceph-mon[29180]: pgmap v10352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:06.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18168.034408 secs 2024-04-26T23:38:08.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:08 smithi119 ceph-mon[32752]: pgmap v10353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:08.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:08 smithi167 ceph-mon[32751]: pgmap v10353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:08.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:08 smithi090 ceph-mon[29180]: pgmap v10353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:10.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:10 smithi119 ceph-mon[32752]: pgmap v10354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:10.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:10 smithi167 ceph-mon[32751]: pgmap v10354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:10.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:10 smithi090 ceph-mon[29180]: pgmap v10354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18172.259195 secs 2024-04-26T23:38:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18173.034521 secs 2024-04-26T23:38:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18172.259195 secs 2024-04-26T23:38:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18173.034521 secs 2024-04-26T23:38:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18172.259195 secs 2024-04-26T23:38:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18173.034521 secs 2024-04-26T23:38:12.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:12 smithi119 ceph-mon[32752]: pgmap v10355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:12.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:12 smithi167 ceph-mon[32751]: pgmap v10355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:12.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:12 smithi090 ceph-mon[29180]: pgmap v10355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:14.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:14 smithi119 ceph-mon[32752]: pgmap v10356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:14.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:14 smithi167 ceph-mon[32751]: pgmap v10356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:14.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:14 smithi090 ceph-mon[29180]: pgmap v10356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:15.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:38:15.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:38:15.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:38:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18177.259279 secs 2024-04-26T23:38:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: pgmap v10357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18178.034641 secs 2024-04-26T23:38:16.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:38:16.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:38:16.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: pgmap v10358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:38:16.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: pgmap v10359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:16.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:38:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18177.259279 secs 2024-04-26T23:38:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: pgmap v10357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18178.034641 secs 2024-04-26T23:38:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:38:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:38:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: pgmap v10358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:38:16.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: pgmap v10359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:16.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:38:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18177.259279 secs 2024-04-26T23:38:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: pgmap v10357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18178.034641 secs 2024-04-26T23:38:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:38:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:38:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: pgmap v10358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:38:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: pgmap v10359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:38:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:18 smithi119 ceph-mon[32752]: pgmap v10360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:38:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:18 smithi167 ceph-mon[32751]: pgmap v10360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:38:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:18 smithi090 ceph-mon[29180]: pgmap v10360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:38:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:20 smithi119 ceph-mon[32752]: pgmap v10361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:20 smithi167 ceph-mon[32751]: pgmap v10361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:20 smithi090 ceph-mon[29180]: pgmap v10361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18182.259362 secs 2024-04-26T23:38:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18183.034760 secs 2024-04-26T23:38:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18182.259362 secs 2024-04-26T23:38:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18183.034760 secs 2024-04-26T23:38:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18182.259362 secs 2024-04-26T23:38:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18183.034760 secs 2024-04-26T23:38:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:22 smithi119 ceph-mon[32752]: pgmap v10362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:22 smithi167 ceph-mon[32751]: pgmap v10362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:22 smithi090 ceph-mon[29180]: pgmap v10362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:25.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:24 smithi119 ceph-mon[32752]: pgmap v10363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:24 smithi167 ceph-mon[32751]: pgmap v10363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:25.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:24 smithi090 ceph-mon[29180]: pgmap v10363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:38:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18187.259443 secs 2024-04-26T23:38:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18188.034897 secs 2024-04-26T23:38:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:26 smithi119 ceph-mon[32752]: pgmap v10364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:38:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18187.259443 secs 2024-04-26T23:38:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18188.034897 secs 2024-04-26T23:38:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:26 smithi167 ceph-mon[32751]: pgmap v10364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:38:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18187.259443 secs 2024-04-26T23:38:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18188.034897 secs 2024-04-26T23:38:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:26 smithi090 ceph-mon[29180]: pgmap v10364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:38:29.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:28 smithi119 ceph-mon[32752]: pgmap v10365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:29.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:28 smithi167 ceph-mon[32751]: pgmap v10365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:29.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:28 smithi090 ceph-mon[29180]: pgmap v10365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:30 smithi119 ceph-mon[32752]: pgmap v10366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:30 smithi167 ceph-mon[32751]: pgmap v10366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:30 smithi090 ceph-mon[29180]: pgmap v10366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:32.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18192.259528 secs 2024-04-26T23:38:32.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18193.035020 secs 2024-04-26T23:38:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18192.259528 secs 2024-04-26T23:38:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18193.035020 secs 2024-04-26T23:38:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18192.259528 secs 2024-04-26T23:38:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18193.035020 secs 2024-04-26T23:38:33.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:32 smithi119 ceph-mon[32752]: pgmap v10367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:33.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:32 smithi167 ceph-mon[32751]: pgmap v10367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:33.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:32 smithi090 ceph-mon[29180]: pgmap v10367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:34 smithi119 ceph-mon[32752]: pgmap v10368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:34 smithi167 ceph-mon[32751]: pgmap v10368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:35.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:34 smithi090 ceph-mon[29180]: pgmap v10368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:35 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18197.259612 secs 2024-04-26T23:38:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:35 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18197.259612 secs 2024-04-26T23:38:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:35 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18197.259612 secs 2024-04-26T23:38:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18198.035095 secs 2024-04-26T23:38:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:36 smithi119 ceph-mon[32752]: pgmap v10369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18198.035095 secs 2024-04-26T23:38:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:36 smithi167 ceph-mon[32751]: pgmap v10369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18198.035095 secs 2024-04-26T23:38:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:36 smithi090 ceph-mon[29180]: pgmap v10369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:39.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:38 smithi119 ceph-mon[32752]: pgmap v10370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:39.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:38 smithi167 ceph-mon[32751]: pgmap v10370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:39.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:38 smithi090 ceph-mon[29180]: pgmap v10370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:41.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:40 smithi119 ceph-mon[32752]: pgmap v10371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:41.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:40 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18202.259700 secs 2024-04-26T23:38:41.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:40 smithi167 ceph-mon[32751]: pgmap v10371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:41.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:40 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18202.259700 secs 2024-04-26T23:38:41.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:40 smithi090 ceph-mon[29180]: pgmap v10371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:41.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:40 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18202.259700 secs 2024-04-26T23:38:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18203.035229 secs 2024-04-26T23:38:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18203.035229 secs 2024-04-26T23:38:42.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18203.035229 secs 2024-04-26T23:38:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:42 smithi119 ceph-mon[32752]: pgmap v10372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:43.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:42 smithi167 ceph-mon[32751]: pgmap v10372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:43.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:42 smithi090 ceph-mon[29180]: pgmap v10372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:44.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:43 smithi090 ceph-mon[29180]: pgmap v10373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:44.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:43 smithi119 ceph-mon[32752]: pgmap v10373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:43 smithi167 ceph-mon[32751]: pgmap v10373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18207.259778 secs 2024-04-26T23:38:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18208.035319 secs 2024-04-26T23:38:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:46 smithi119 ceph-mon[32752]: pgmap v10374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18207.259778 secs 2024-04-26T23:38:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18208.035319 secs 2024-04-26T23:38:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:46 smithi167 ceph-mon[32751]: pgmap v10374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18207.259778 secs 2024-04-26T23:38:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18208.035319 secs 2024-04-26T23:38:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:46 smithi090 ceph-mon[29180]: pgmap v10374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:48 smithi119 ceph-mon[32752]: pgmap v10375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:49.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:48 smithi167 ceph-mon[32751]: pgmap v10375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:48 smithi090 ceph-mon[29180]: pgmap v10375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:50 smithi119 ceph-mon[32752]: pgmap v10376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:50 smithi167 ceph-mon[32751]: pgmap v10376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:50 smithi090 ceph-mon[29180]: pgmap v10376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18212.259840 secs 2024-04-26T23:38:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18213.035443 secs 2024-04-26T23:38:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18212.259840 secs 2024-04-26T23:38:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18213.035443 secs 2024-04-26T23:38:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18212.259840 secs 2024-04-26T23:38:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18213.035443 secs 2024-04-26T23:38:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:52 smithi119 ceph-mon[32752]: pgmap v10377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:53.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:52 smithi167 ceph-mon[32751]: pgmap v10377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:53.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:52 smithi090 ceph-mon[29180]: pgmap v10377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:55.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:54 smithi119 ceph-mon[32752]: pgmap v10378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:55.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:54 smithi167 ceph-mon[32751]: pgmap v10378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:55.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:54 smithi090 ceph-mon[29180]: pgmap v10378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:38:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18217.259900 secs 2024-04-26T23:38:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18218.035560 secs 2024-04-26T23:38:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:56 smithi090 ceph-mon[29180]: pgmap v10379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18217.259900 secs 2024-04-26T23:38:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18218.035560 secs 2024-04-26T23:38:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:56 smithi119 ceph-mon[32752]: pgmap v10379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18217.259900 secs 2024-04-26T23:38:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18218.035560 secs 2024-04-26T23:38:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:56 smithi167 ceph-mon[32751]: pgmap v10379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:38:58 smithi119 ceph-mon[32752]: pgmap v10380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:59.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:38:58 smithi167 ceph-mon[32751]: pgmap v10380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:38:59.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:38:58 smithi090 ceph-mon[29180]: pgmap v10380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:00 smithi119 ceph-mon[32752]: pgmap v10381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:00 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18222.259960 secs 2024-04-26T23:39:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:00 smithi167 ceph-mon[32751]: pgmap v10381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:00 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18222.259960 secs 2024-04-26T23:39:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:00 smithi090 ceph-mon[29180]: pgmap v10381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:00 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18222.259960 secs 2024-04-26T23:39:02.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18223.035668 secs 2024-04-26T23:39:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18223.035668 secs 2024-04-26T23:39:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18223.035668 secs 2024-04-26T23:39:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:02 smithi119 ceph-mon[32752]: pgmap v10382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:02 smithi167 ceph-mon[32751]: pgmap v10382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:03.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:02 smithi090 ceph-mon[29180]: pgmap v10382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:04 smithi119 ceph-mon[32752]: pgmap v10383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:04 smithi167 ceph-mon[32751]: pgmap v10383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:05.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:04 smithi090 ceph-mon[29180]: pgmap v10383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:05 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18227.260020 secs 2024-04-26T23:39:06.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:39:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:39:05.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:39:06.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:39:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:39:05.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:39:06.130 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:39:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:39:05.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:39:06.130 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:39:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:39:05.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:39:06.130 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:39:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:39:05.795+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:39:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:05 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18227.260020 secs 2024-04-26T23:39:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:05 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18227.260020 secs 2024-04-26T23:39:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18228.035734 secs 2024-04-26T23:39:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:06 smithi119 ceph-mon[32752]: pgmap v10384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18228.035734 secs 2024-04-26T23:39:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:06 smithi167 ceph-mon[32751]: pgmap v10384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18228.035734 secs 2024-04-26T23:39:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:06 smithi090 ceph-mon[29180]: pgmap v10384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:07 smithi119 ceph-mon[32752]: pgmap v10385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:07 smithi167 ceph-mon[32751]: pgmap v10385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:07 smithi090 ceph-mon[29180]: pgmap v10385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:10 smithi119 ceph-mon[32752]: pgmap v10386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:39:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:10 smithi167 ceph-mon[32751]: pgmap v10386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:39:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:10 smithi090 ceph-mon[29180]: pgmap v10386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:39:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18232.260081 secs 2024-04-26T23:39:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18233.035810 secs 2024-04-26T23:39:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18232.260081 secs 2024-04-26T23:39:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18233.035810 secs 2024-04-26T23:39:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18232.260081 secs 2024-04-26T23:39:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18233.035810 secs 2024-04-26T23:39:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:12 smithi090 ceph-mon[29180]: pgmap v10387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:12 smithi119 ceph-mon[32752]: pgmap v10387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:12 smithi167 ceph-mon[32751]: pgmap v10387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:14 smithi119 ceph-mon[32752]: pgmap v10388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:39:15.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:14 smithi167 ceph-mon[32751]: pgmap v10388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:39:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:14 smithi090 ceph-mon[29180]: pgmap v10388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 511 B/s wr, 14 op/s 2024-04-26T23:39:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18237.260165 secs 2024-04-26T23:39:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18238.035919 secs 2024-04-26T23:39:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: pgmap v10389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:39:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.250 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18237.260165 secs 2024-04-26T23:39:17.250 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18238.035919 secs 2024-04-26T23:39:17.250 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: pgmap v10389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:17.250 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:39:17.250 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.251 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18237.260165 secs 2024-04-26T23:39:17.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18238.035919 secs 2024-04-26T23:39:17.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: pgmap v10389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:17.251 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:39:17.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:17.252 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: pgmap v10390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: pgmap v10390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: pgmap v10390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 511 B/s wr, 13 op/s 2024-04-26T23:39:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:19.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:19.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:39:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:39:20.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:39:20.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:39:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:39:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:39:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:20.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:39:20.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:39:20.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:39:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:39:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:20 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:39:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:20 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:39:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:20 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:39:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:20 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:39:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:20 smithi119 ceph-mon[32752]: pgmap v10391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 535 B/s wr, 14 op/s 2024-04-26T23:39:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:20 smithi119 ceph-mon[32752]: pgmap v10392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:20 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18242.260249 secs 2024-04-26T23:39:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:20 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:39:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:20 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:39:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:20 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:39:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:20 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:39:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:20 smithi167 ceph-mon[32751]: pgmap v10391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 535 B/s wr, 14 op/s 2024-04-26T23:39:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:20 smithi167 ceph-mon[32751]: pgmap v10392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:20 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18242.260249 secs 2024-04-26T23:39:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:20 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:39:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:20 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:39:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:20 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:39:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:20 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:39:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:20 smithi090 ceph-mon[29180]: pgmap v10391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 535 B/s wr, 14 op/s 2024-04-26T23:39:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:20 smithi090 ceph-mon[29180]: pgmap v10392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:20 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18242.260249 secs 2024-04-26T23:39:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18243.035989 secs 2024-04-26T23:39:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18243.035989 secs 2024-04-26T23:39:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18243.035989 secs 2024-04-26T23:39:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:22 smithi119 ceph-mon[32752]: pgmap v10393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:22 smithi167 ceph-mon[32751]: pgmap v10393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:22 smithi090 ceph-mon[29180]: pgmap v10393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:25.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:24 smithi119 ceph-mon[32752]: pgmap v10394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:24 smithi167 ceph-mon[32751]: pgmap v10394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:25.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:24 smithi090 ceph-mon[29180]: pgmap v10394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:25 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18247.260335 secs 2024-04-26T23:39:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:25 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18247.260335 secs 2024-04-26T23:39:26.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:25 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18247.260335 secs 2024-04-26T23:39:27.222 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:26 smithi167 ceph-mon[32751]: pgmap v10395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:27.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18248.036107 secs 2024-04-26T23:39:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:26 smithi119 ceph-mon[32752]: pgmap v10395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18248.036107 secs 2024-04-26T23:39:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:26 smithi090 ceph-mon[29180]: pgmap v10395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18248.036107 secs 2024-04-26T23:39:28.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:27 smithi119 ceph-mon[32752]: pgmap v10396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:28.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:27 smithi167 ceph-mon[32751]: pgmap v10396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:28.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:27 smithi090 ceph-mon[29180]: pgmap v10396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:39:30.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:30 smithi090 ceph-mon[29180]: pgmap v10397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:39:30.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:30 smithi119 ceph-mon[32752]: pgmap v10397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:39:30.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:30 smithi167 ceph-mon[32751]: pgmap v10397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:39:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18252.260421 secs 2024-04-26T23:39:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18253.036170 secs 2024-04-26T23:39:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18252.260421 secs 2024-04-26T23:39:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18253.036170 secs 2024-04-26T23:39:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18252.260421 secs 2024-04-26T23:39:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18253.036170 secs 2024-04-26T23:39:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:32 smithi090 ceph-mon[29180]: pgmap v10398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:32.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:32 smithi119 ceph-mon[32752]: pgmap v10398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:32.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:32 smithi167 ceph-mon[32751]: pgmap v10398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:34 smithi090 ceph-mon[29180]: pgmap v10399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:34.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:34 smithi119 ceph-mon[32752]: pgmap v10399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:34.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:34 smithi167 ceph-mon[32751]: pgmap v10399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18257.260506 secs 2024-04-26T23:39:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:36 smithi090 ceph-mon[29180]: pgmap v10400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18258.036284 secs 2024-04-26T23:39:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18257.260506 secs 2024-04-26T23:39:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:36 smithi119 ceph-mon[32752]: pgmap v10400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18258.036284 secs 2024-04-26T23:39:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18257.260506 secs 2024-04-26T23:39:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:36 smithi167 ceph-mon[32751]: pgmap v10400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18258.036284 secs 2024-04-26T23:39:38.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:38 smithi119 ceph-mon[32752]: pgmap v10401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:38.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:38 smithi167 ceph-mon[32751]: pgmap v10401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:38.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:38 smithi090 ceph-mon[29180]: pgmap v10401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:40.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:40 smithi119 ceph-mon[32752]: pgmap v10402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:40.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:40 smithi167 ceph-mon[32751]: pgmap v10402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:40.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:40 smithi090 ceph-mon[29180]: pgmap v10402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18262.260567 secs 2024-04-26T23:39:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18263.036412 secs 2024-04-26T23:39:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18262.260567 secs 2024-04-26T23:39:41.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18263.036412 secs 2024-04-26T23:39:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18262.260567 secs 2024-04-26T23:39:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18263.036412 secs 2024-04-26T23:39:42.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:42 smithi119 ceph-mon[32752]: pgmap v10403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:42.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:42 smithi167 ceph-mon[32751]: pgmap v10403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:42 smithi090 ceph-mon[29180]: pgmap v10403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:44.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:44 smithi090 ceph-mon[29180]: pgmap v10404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:44.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:44 smithi119 ceph-mon[32752]: pgmap v10404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:44.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:44 smithi167 ceph-mon[32751]: pgmap v10404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:46.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18267.260650 secs 2024-04-26T23:39:46.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:46 smithi119 ceph-mon[32752]: pgmap v10405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:46.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18268.036510 secs 2024-04-26T23:39:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18267.260650 secs 2024-04-26T23:39:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:46 smithi167 ceph-mon[32751]: pgmap v10405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18268.036510 secs 2024-04-26T23:39:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18267.260650 secs 2024-04-26T23:39:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:46 smithi090 ceph-mon[29180]: pgmap v10405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18268.036510 secs 2024-04-26T23:39:48.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:48 smithi119 ceph-mon[32752]: pgmap v10406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:48.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:48 smithi167 ceph-mon[32751]: pgmap v10406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:48.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:48 smithi090 ceph-mon[29180]: pgmap v10406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:50.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:50 smithi119 ceph-mon[32752]: pgmap v10407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:50 smithi167 ceph-mon[32751]: pgmap v10407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:50.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:50 smithi090 ceph-mon[29180]: pgmap v10407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:51.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18272.260735 secs 2024-04-26T23:39:51.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18273.036630 secs 2024-04-26T23:39:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18272.260735 secs 2024-04-26T23:39:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18273.036630 secs 2024-04-26T23:39:51.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18272.260735 secs 2024-04-26T23:39:51.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18273.036630 secs 2024-04-26T23:39:52.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:52 smithi119 ceph-mon[32752]: pgmap v10408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:52.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:52 smithi167 ceph-mon[32751]: pgmap v10408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:52.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:52 smithi090 ceph-mon[29180]: pgmap v10408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:54.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:54 smithi119 ceph-mon[32752]: pgmap v10409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:54.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:54 smithi167 ceph-mon[32751]: pgmap v10409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:54.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:54 smithi090 ceph-mon[29180]: pgmap v10409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:39:56.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18277.260841 secs 2024-04-26T23:39:56.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:56 smithi119 ceph-mon[32752]: pgmap v10410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:56.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18278.036748 secs 2024-04-26T23:39:56.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18277.260841 secs 2024-04-26T23:39:56.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:56 smithi167 ceph-mon[32751]: pgmap v10410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:56.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18278.036748 secs 2024-04-26T23:39:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18277.260841 secs 2024-04-26T23:39:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:56 smithi090 ceph-mon[29180]: pgmap v10410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18278.036748 secs 2024-04-26T23:39:58.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:39:58 smithi119 ceph-mon[32752]: pgmap v10411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:58.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:39:58 smithi167 ceph-mon[32751]: pgmap v10411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:39:58.824 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:39:58 smithi090 ceph-mon[29180]: pgmap v10411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:00.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:00 smithi119 ceph-mon[32752]: pgmap v10412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:00.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:40:00.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:00 smithi167 ceph-mon[32751]: pgmap v10412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:00.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:40:00.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:00 smithi090 ceph-mon[29180]: pgmap v10412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:00.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:40:01.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18282.260903 secs 2024-04-26T23:40:01.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18283.036839 secs 2024-04-26T23:40:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18282.260903 secs 2024-04-26T23:40:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18283.036839 secs 2024-04-26T23:40:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18282.260903 secs 2024-04-26T23:40:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18283.036839 secs 2024-04-26T23:40:02.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:02 smithi119 ceph-mon[32752]: pgmap v10413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:02.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:02 smithi167 ceph-mon[32751]: pgmap v10413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:02.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:02 smithi090 ceph-mon[29180]: pgmap v10413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:04.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:04 smithi119 ceph-mon[32752]: pgmap v10414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:40:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:04 smithi167 ceph-mon[32751]: pgmap v10414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:40:04.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:04 smithi090 ceph-mon[29180]: pgmap v10414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:40:06.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18287.260964 secs 2024-04-26T23:40:06.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:06 smithi119 ceph-mon[32752]: pgmap v10415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:06.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:06 smithi119 ceph-mon[32752]: 20 slow requests, 2 included below; oldest blocked for > 18288.036979 secs 2024-04-26T23:40:06.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:06 smithi119 ceph-mon[32752]: slow request 11523.198167 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:40:06.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:06 smithi119 ceph-mon[32752]: slow request 360.109536 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:40:06.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18287.260964 secs 2024-04-26T23:40:06.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:06 smithi167 ceph-mon[32751]: pgmap v10415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:06.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:06 smithi167 ceph-mon[32751]: 20 slow requests, 2 included below; oldest blocked for > 18288.036979 secs 2024-04-26T23:40:06.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:06 smithi167 ceph-mon[32751]: slow request 11523.198167 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:40:06.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:06 smithi167 ceph-mon[32751]: slow request 360.109536 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:40:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18287.260964 secs 2024-04-26T23:40:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:06 smithi090 ceph-mon[29180]: pgmap v10415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:06 smithi090 ceph-mon[29180]: 20 slow requests, 2 included below; oldest blocked for > 18288.036979 secs 2024-04-26T23:40:06.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:06 smithi090 ceph-mon[29180]: slow request 11523.198167 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:40:06.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:06 smithi090 ceph-mon[29180]: slow request 360.109536 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:40:08.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:08 smithi119 ceph-mon[32752]: pgmap v10416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:08.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:08 smithi167 ceph-mon[32751]: pgmap v10416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:08.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:08 smithi090 ceph-mon[29180]: pgmap v10416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:10.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:10 smithi119 ceph-mon[32752]: pgmap v10417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:40:10.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:10 smithi167 ceph-mon[32751]: pgmap v10417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:40:10.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:10 smithi090 ceph-mon[29180]: pgmap v10417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:40:11.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18292.261026 secs 2024-04-26T23:40:11.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18293.037058 secs 2024-04-26T23:40:11.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18292.261026 secs 2024-04-26T23:40:11.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18293.037058 secs 2024-04-26T23:40:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18292.261026 secs 2024-04-26T23:40:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18293.037058 secs 2024-04-26T23:40:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:12 smithi119 ceph-mon[32752]: pgmap v10418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:12 smithi167 ceph-mon[32751]: pgmap v10418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:12 smithi090 ceph-mon[29180]: pgmap v10418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:14.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:14 smithi119 ceph-mon[32752]: pgmap v10419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:40:14.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:14 smithi167 ceph-mon[32751]: pgmap v10419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:40:14.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:14 smithi090 ceph-mon[29180]: pgmap v10419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:40:16.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18297.261087 secs 2024-04-26T23:40:16.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:16 smithi119 ceph-mon[32752]: pgmap v10420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:16.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18298.037156 secs 2024-04-26T23:40:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18297.261087 secs 2024-04-26T23:40:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:16 smithi167 ceph-mon[32751]: pgmap v10420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18298.037156 secs 2024-04-26T23:40:16.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18297.261087 secs 2024-04-26T23:40:16.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:16 smithi090 ceph-mon[29180]: pgmap v10420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:40:16.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18298.037156 secs 2024-04-26T23:40:19.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:18 smithi090 ceph-mon[29180]: pgmap v10421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:18 smithi119 ceph-mon[32752]: pgmap v10421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:18 smithi167 ceph-mon[32751]: pgmap v10421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:20.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:40:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:40:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:40:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:20 smithi090 ceph-mon[29180]: pgmap v10422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:40:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:40:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:20 smithi119 ceph-mon[32752]: pgmap v10422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:40:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:40:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:20 smithi167 ceph-mon[32751]: pgmap v10422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:40:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:40:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:40:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18302.261152 secs 2024-04-26T23:40:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:21 smithi090 ceph-mon[29180]: pgmap v10423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T23:40:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:21 smithi090 ceph-mon[29180]: pgmap v10424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:22.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18303.037282 secs 2024-04-26T23:40:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18302.261152 secs 2024-04-26T23:40:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:21 smithi119 ceph-mon[32752]: pgmap v10423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T23:40:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:21 smithi119 ceph-mon[32752]: pgmap v10424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18303.037282 secs 2024-04-26T23:40:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18302.261152 secs 2024-04-26T23:40:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:21 smithi167 ceph-mon[32751]: pgmap v10423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-26T23:40:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:21 smithi167 ceph-mon[32751]: pgmap v10424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18303.037282 secs 2024-04-26T23:40:24.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:23 smithi090 ceph-mon[29180]: pgmap v10425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T23:40:24.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:23 smithi119 ceph-mon[32752]: pgmap v10425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T23:40:24.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:23 smithi167 ceph-mon[32751]: pgmap v10425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-26T23:40:26.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:25 smithi090 ceph-mon[29180]: pgmap v10426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:25 smithi119 ceph-mon[32752]: pgmap v10426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:25 smithi167 ceph-mon[32751]: pgmap v10426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18307.261214 secs 2024-04-26T23:40:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18308.037399 secs 2024-04-26T23:40:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18307.261214 secs 2024-04-26T23:40:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18308.037399 secs 2024-04-26T23:40:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18307.261214 secs 2024-04-26T23:40:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18308.037399 secs 2024-04-26T23:40:28.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:27 smithi090 ceph-mon[29180]: pgmap v10427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:28.223 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:27 smithi167 ceph-mon[32751]: pgmap v10427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:28.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:27 smithi119 ceph-mon[32752]: pgmap v10427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:30.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:29 smithi090 ceph-mon[29180]: pgmap v10428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:30.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:29 smithi119 ceph-mon[32752]: pgmap v10428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:30.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:29 smithi167 ceph-mon[32751]: pgmap v10428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:40:32.109 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18312.261298 secs 2024-04-26T23:40:32.110 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:31 smithi119 ceph-mon[32752]: pgmap v10429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:40:32.110 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18313.037534 secs 2024-04-26T23:40:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18312.261298 secs 2024-04-26T23:40:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:31 smithi090 ceph-mon[29180]: pgmap v10429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:40:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18313.037534 secs 2024-04-26T23:40:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18312.261298 secs 2024-04-26T23:40:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:31 smithi167 ceph-mon[32751]: pgmap v10429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:40:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18313.037534 secs 2024-04-26T23:40:34.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:33 smithi090 ceph-mon[29180]: pgmap v10430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:33 smithi119 ceph-mon[32752]: pgmap v10430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:34.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:33 smithi167 ceph-mon[32751]: pgmap v10430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:36.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:35 smithi090 ceph-mon[29180]: pgmap v10431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:35 smithi119 ceph-mon[32752]: pgmap v10431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:35 smithi167 ceph-mon[32751]: pgmap v10431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18317.261383 secs 2024-04-26T23:40:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18318.037643 secs 2024-04-26T23:40:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18317.261383 secs 2024-04-26T23:40:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18318.037643 secs 2024-04-26T23:40:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18317.261383 secs 2024-04-26T23:40:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18318.037643 secs 2024-04-26T23:40:38.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:37 smithi090 ceph-mon[29180]: pgmap v10432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:38.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:37 smithi119 ceph-mon[32752]: pgmap v10432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:37 smithi167 ceph-mon[32751]: pgmap v10432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:40.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:39 smithi090 ceph-mon[29180]: pgmap v10433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:40.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:39 smithi119 ceph-mon[32752]: pgmap v10433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:40.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:39 smithi167 ceph-mon[32751]: pgmap v10433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18322.261469 secs 2024-04-26T23:40:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:41 smithi090 ceph-mon[29180]: pgmap v10434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18323.037757 secs 2024-04-26T23:40:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18322.261469 secs 2024-04-26T23:40:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:41 smithi119 ceph-mon[32752]: pgmap v10434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18323.037757 secs 2024-04-26T23:40:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18322.261469 secs 2024-04-26T23:40:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:41 smithi167 ceph-mon[32751]: pgmap v10434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18323.037757 secs 2024-04-26T23:40:44.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:43 smithi119 ceph-mon[32752]: pgmap v10435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:43 smithi167 ceph-mon[32751]: pgmap v10435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:44.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:43 smithi090 ceph-mon[29180]: pgmap v10435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:46.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:45 smithi119 ceph-mon[32752]: pgmap v10436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:45 smithi167 ceph-mon[32751]: pgmap v10436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:45 smithi090 ceph-mon[29180]: pgmap v10436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18327.261553 secs 2024-04-26T23:40:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18328.037877 secs 2024-04-26T23:40:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18327.261553 secs 2024-04-26T23:40:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18328.037877 secs 2024-04-26T23:40:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18327.261553 secs 2024-04-26T23:40:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18328.037877 secs 2024-04-26T23:40:48.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:47 smithi119 ceph-mon[32752]: pgmap v10437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:48.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:47 smithi167 ceph-mon[32751]: pgmap v10437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:48.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:47 smithi090 ceph-mon[29180]: pgmap v10437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:49 smithi119 ceph-mon[32752]: pgmap v10438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:50.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:49 smithi167 ceph-mon[32751]: pgmap v10438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:49 smithi090 ceph-mon[29180]: pgmap v10438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18332.261638 secs 2024-04-26T23:40:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:51 smithi119 ceph-mon[32752]: pgmap v10439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18333.037992 secs 2024-04-26T23:40:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18332.261638 secs 2024-04-26T23:40:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:51 smithi167 ceph-mon[32751]: pgmap v10439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18333.037992 secs 2024-04-26T23:40:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18332.261638 secs 2024-04-26T23:40:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:51 smithi090 ceph-mon[29180]: pgmap v10439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18333.037992 secs 2024-04-26T23:40:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:53 smithi119 ceph-mon[32752]: pgmap v10440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:53 smithi167 ceph-mon[32751]: pgmap v10440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:53 smithi090 ceph-mon[29180]: pgmap v10440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:56.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:55 smithi119 ceph-mon[32752]: pgmap v10441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:55 smithi167 ceph-mon[32751]: pgmap v10441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:55 smithi090 ceph-mon[29180]: pgmap v10441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:40:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18337.261723 secs 2024-04-26T23:40:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18338.038112 secs 2024-04-26T23:40:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18337.261723 secs 2024-04-26T23:40:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18338.038112 secs 2024-04-26T23:40:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18337.261723 secs 2024-04-26T23:40:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18338.038112 secs 2024-04-26T23:40:58.222 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:57 smithi167 ceph-mon[32751]: pgmap v10442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:58.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:57 smithi119 ceph-mon[32752]: pgmap v10442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:40:58.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:57 smithi090 ceph-mon[29180]: pgmap v10442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:00.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:40:59 smithi119 ceph-mon[32752]: pgmap v10443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:00.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:40:59 smithi167 ceph-mon[32751]: pgmap v10443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:00.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:40:59 smithi090 ceph-mon[29180]: pgmap v10443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18342.261825 secs 2024-04-26T23:41:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:01 smithi119 ceph-mon[32752]: pgmap v10444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18343.038255 secs 2024-04-26T23:41:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18342.261825 secs 2024-04-26T23:41:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:01 smithi167 ceph-mon[32751]: pgmap v10444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18343.038255 secs 2024-04-26T23:41:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18342.261825 secs 2024-04-26T23:41:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:01 smithi090 ceph-mon[29180]: pgmap v10444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18343.038255 secs 2024-04-26T23:41:04.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:03 smithi119 ceph-mon[32752]: pgmap v10445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:04.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:03 smithi167 ceph-mon[32751]: pgmap v10445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:03 smithi090 ceph-mon[29180]: pgmap v10445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:05 smithi119 ceph-mon[32752]: pgmap v10446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:05 smithi167 ceph-mon[32751]: pgmap v10446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:06.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:05 smithi090 ceph-mon[29180]: pgmap v10446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18347.261909 secs 2024-04-26T23:41:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:06 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 18348.038368 secs 2024-04-26T23:41:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:06 smithi119 ceph-mon[32752]: slow request 720.178479 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:41:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18347.261909 secs 2024-04-26T23:41:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:06 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 18348.038368 secs 2024-04-26T23:41:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:06 smithi167 ceph-mon[32751]: slow request 720.178479 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:41:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18347.261909 secs 2024-04-26T23:41:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:06 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 18348.038368 secs 2024-04-26T23:41:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:06 smithi090 ceph-mon[29180]: slow request 720.178479 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:41:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:07 smithi119 ceph-mon[32752]: pgmap v10447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:07 smithi167 ceph-mon[32751]: pgmap v10447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:07 smithi090 ceph-mon[29180]: pgmap v10447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:09 smithi119 ceph-mon[32752]: pgmap v10448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:09 smithi167 ceph-mon[32751]: pgmap v10448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:09 smithi090 ceph-mon[29180]: pgmap v10448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:10 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18352.261999 secs 2024-04-26T23:41:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:10 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18352.261999 secs 2024-04-26T23:41:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:10 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18352.261999 secs 2024-04-26T23:41:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:11 smithi119 ceph-mon[32752]: pgmap v10449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18353.038477 secs 2024-04-26T23:41:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:11 smithi167 ceph-mon[32751]: pgmap v10449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18353.038477 secs 2024-04-26T23:41:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:11 smithi090 ceph-mon[29180]: pgmap v10449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18353.038477 secs 2024-04-26T23:41:14.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:13 smithi119 ceph-mon[32752]: pgmap v10450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:14.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:13 smithi167 ceph-mon[32751]: pgmap v10450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:14.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:13 smithi090 ceph-mon[29180]: pgmap v10450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:15 smithi119 ceph-mon[32752]: pgmap v10451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:15 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18357.262083 secs 2024-04-26T23:41:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:15 smithi167 ceph-mon[32751]: pgmap v10451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:15 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18357.262083 secs 2024-04-26T23:41:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:15 smithi090 ceph-mon[29180]: pgmap v10451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:15 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18357.262083 secs 2024-04-26T23:41:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18358.038591 secs 2024-04-26T23:41:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18358.038591 secs 2024-04-26T23:41:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18358.038591 secs 2024-04-26T23:41:18.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:17 smithi119 ceph-mon[32752]: pgmap v10452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:18.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:17 smithi167 ceph-mon[32751]: pgmap v10452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:18.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:17 smithi090 ceph-mon[29180]: pgmap v10452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:19 smithi119 ceph-mon[32752]: pgmap v10453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:19 smithi167 ceph-mon[32751]: pgmap v10453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:19 smithi090 ceph-mon[29180]: pgmap v10453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18362.262146 secs 2024-04-26T23:41:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:21 smithi119 ceph-mon[32752]: pgmap v10454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:41:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18363.038655 secs 2024-04-26T23:41:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:41:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:41:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:41:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18362.262146 secs 2024-04-26T23:41:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:21 smithi167 ceph-mon[32751]: pgmap v10454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:41:22.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18363.038655 secs 2024-04-26T23:41:22.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:41:22.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:41:22.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:41:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18362.262146 secs 2024-04-26T23:41:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:21 smithi090 ceph-mon[29180]: pgmap v10454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:41:22.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18363.038655 secs 2024-04-26T23:41:22.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:41:22.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:41:22.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:41:23.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:22 smithi119 ceph-mon[32752]: pgmap v10455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:23.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:22 smithi167 ceph-mon[32751]: pgmap v10455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:23.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:22 smithi090 ceph-mon[29180]: pgmap v10455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:25.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:24 smithi119 ceph-mon[32752]: pgmap v10456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:41:25.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:24 smithi167 ceph-mon[32751]: pgmap v10456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:41:25.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:24 smithi090 ceph-mon[29180]: pgmap v10456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:41:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18367.262222 secs 2024-04-26T23:41:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:26 smithi119 ceph-mon[32752]: pgmap v10457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18368.038721 secs 2024-04-26T23:41:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18367.262222 secs 2024-04-26T23:41:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:26 smithi167 ceph-mon[32751]: pgmap v10457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18368.038721 secs 2024-04-26T23:41:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18367.262222 secs 2024-04-26T23:41:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:26 smithi090 ceph-mon[29180]: pgmap v10457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:27.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18368.038721 secs 2024-04-26T23:41:29.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:28 smithi119 ceph-mon[32752]: pgmap v10458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:29.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:28 smithi167 ceph-mon[32751]: pgmap v10458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:29.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:28 smithi090 ceph-mon[29180]: pgmap v10458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:31.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:30 smithi119 ceph-mon[32752]: pgmap v10459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:31.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:30 smithi167 ceph-mon[32751]: pgmap v10459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:31.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:30 smithi090 ceph-mon[29180]: pgmap v10459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18372.262304 secs 2024-04-26T23:41:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18373.038800 secs 2024-04-26T23:41:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18372.262304 secs 2024-04-26T23:41:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18373.038800 secs 2024-04-26T23:41:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18372.262304 secs 2024-04-26T23:41:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18373.038800 secs 2024-04-26T23:41:32.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:32 smithi090 ceph-mon[29180]: pgmap v10460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:32.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:32 smithi119 ceph-mon[32752]: pgmap v10460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:33.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:32 smithi167 ceph-mon[32751]: pgmap v10460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:41:35.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:34 smithi119 ceph-mon[32752]: pgmap v10461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:34 smithi167 ceph-mon[32751]: pgmap v10461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:34 smithi090 ceph-mon[29180]: pgmap v10461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18377.262388 secs 2024-04-26T23:41:37.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:36 smithi119 ceph-mon[32752]: pgmap v10462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:37.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18378.038913 secs 2024-04-26T23:41:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18377.262388 secs 2024-04-26T23:41:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:36 smithi167 ceph-mon[32751]: pgmap v10462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18378.038913 secs 2024-04-26T23:41:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18377.262388 secs 2024-04-26T23:41:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:36 smithi090 ceph-mon[29180]: pgmap v10462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18378.038913 secs 2024-04-26T23:41:39.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:38 smithi119 ceph-mon[32752]: pgmap v10463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:39.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:38 smithi167 ceph-mon[32751]: pgmap v10463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:39.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:38 smithi090 ceph-mon[29180]: pgmap v10463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:40 smithi119 ceph-mon[32752]: pgmap v10464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:41.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:40 smithi167 ceph-mon[32751]: pgmap v10464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:40 smithi090 ceph-mon[29180]: pgmap v10464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18382.262474 secs 2024-04-26T23:41:42.127 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18383.039031 secs 2024-04-26T23:41:42.127 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18382.262474 secs 2024-04-26T23:41:42.128 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18383.039031 secs 2024-04-26T23:41:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18382.262474 secs 2024-04-26T23:41:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18383.039031 secs 2024-04-26T23:41:43.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:42 smithi090 ceph-mon[29180]: pgmap v10465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:42 smithi119 ceph-mon[32752]: pgmap v10465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:43.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:42 smithi167 ceph-mon[32751]: pgmap v10465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:45.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:44 smithi090 ceph-mon[29180]: pgmap v10466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:45.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:44 smithi119 ceph-mon[32752]: pgmap v10466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:45.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:44 smithi167 ceph-mon[32751]: pgmap v10466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18387.262558 secs 2024-04-26T23:41:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:46 smithi090 ceph-mon[29180]: pgmap v10467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18388.039168 secs 2024-04-26T23:41:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18387.262558 secs 2024-04-26T23:41:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:46 smithi119 ceph-mon[32752]: pgmap v10467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18388.039168 secs 2024-04-26T23:41:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18387.262558 secs 2024-04-26T23:41:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:46 smithi167 ceph-mon[32751]: pgmap v10467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18388.039168 secs 2024-04-26T23:41:49.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:48 smithi090 ceph-mon[29180]: pgmap v10468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:48 smithi119 ceph-mon[32752]: pgmap v10468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:49.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:48 smithi167 ceph-mon[32751]: pgmap v10468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:51.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:50 smithi090 ceph-mon[29180]: pgmap v10469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:50 smithi119 ceph-mon[32752]: pgmap v10469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:50 smithi167 ceph-mon[32751]: pgmap v10469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18392.262642 secs 2024-04-26T23:41:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18393.039237 secs 2024-04-26T23:41:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18392.262642 secs 2024-04-26T23:41:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18393.039237 secs 2024-04-26T23:41:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18392.262642 secs 2024-04-26T23:41:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18393.039237 secs 2024-04-26T23:41:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:52 smithi090 ceph-mon[29180]: pgmap v10470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:52 smithi119 ceph-mon[32752]: pgmap v10470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:53.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:52 smithi167 ceph-mon[32751]: pgmap v10470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:55.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:54 smithi090 ceph-mon[29180]: pgmap v10471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:55.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:54 smithi119 ceph-mon[32752]: pgmap v10471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:55.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:54 smithi167 ceph-mon[32751]: pgmap v10471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:41:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18397.262727 secs 2024-04-26T23:41:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:56 smithi090 ceph-mon[29180]: pgmap v10472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:57.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18398.039407 secs 2024-04-26T23:41:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18397.262727 secs 2024-04-26T23:41:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:56 smithi119 ceph-mon[32752]: pgmap v10472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18398.039407 secs 2024-04-26T23:41:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18397.262727 secs 2024-04-26T23:41:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:56 smithi167 ceph-mon[32751]: pgmap v10472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18398.039407 secs 2024-04-26T23:41:59.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:41:58 smithi090 ceph-mon[29180]: pgmap v10473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:41:58 smithi119 ceph-mon[32752]: pgmap v10473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:41:59.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:41:58 smithi167 ceph-mon[32751]: pgmap v10473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:00 smithi119 ceph-mon[32752]: pgmap v10474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:00 smithi167 ceph-mon[32751]: pgmap v10474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:00 smithi090 ceph-mon[29180]: pgmap v10474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:02.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18402.262818 secs 2024-04-26T23:42:02.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18403.039515 secs 2024-04-26T23:42:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18402.262818 secs 2024-04-26T23:42:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18403.039515 secs 2024-04-26T23:42:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18402.262818 secs 2024-04-26T23:42:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18403.039515 secs 2024-04-26T23:42:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:02 smithi119 ceph-mon[32752]: pgmap v10475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:02 smithi167 ceph-mon[32751]: pgmap v10475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:03.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:02 smithi090 ceph-mon[29180]: pgmap v10475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:04 smithi119 ceph-mon[32752]: pgmap v10476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:04 smithi167 ceph-mon[32751]: pgmap v10476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:05.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:04 smithi090 ceph-mon[29180]: pgmap v10476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18407.262903 secs 2024-04-26T23:42:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:06 smithi119 ceph-mon[32752]: pgmap v10477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18408.039594 secs 2024-04-26T23:42:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18407.262903 secs 2024-04-26T23:42:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:06 smithi167 ceph-mon[32751]: pgmap v10477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18408.039594 secs 2024-04-26T23:42:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18407.262903 secs 2024-04-26T23:42:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:06 smithi090 ceph-mon[29180]: pgmap v10477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18408.039594 secs 2024-04-26T23:42:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:08 smithi119 ceph-mon[32752]: pgmap v10478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:08 smithi167 ceph-mon[32751]: pgmap v10478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:08 smithi090 ceph-mon[29180]: pgmap v10478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:10 smithi119 ceph-mon[32752]: pgmap v10479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:10 smithi167 ceph-mon[32751]: pgmap v10479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:10 smithi090 ceph-mon[29180]: pgmap v10479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18412.262994 secs 2024-04-26T23:42:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18413.039671 secs 2024-04-26T23:42:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18412.262994 secs 2024-04-26T23:42:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18413.039671 secs 2024-04-26T23:42:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18412.262994 secs 2024-04-26T23:42:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18413.039671 secs 2024-04-26T23:42:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:12 smithi119 ceph-mon[32752]: pgmap v10480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:12 smithi167 ceph-mon[32751]: pgmap v10480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:12 smithi090 ceph-mon[29180]: pgmap v10480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:14 smithi119 ceph-mon[32752]: pgmap v10481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:15.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:14 smithi167 ceph-mon[32751]: pgmap v10481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:14 smithi090 ceph-mon[29180]: pgmap v10481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:17.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18417.263053 secs 2024-04-26T23:42:17.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:16 smithi090 ceph-mon[29180]: pgmap v10482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:17.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18418.039806 secs 2024-04-26T23:42:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18417.263053 secs 2024-04-26T23:42:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:16 smithi119 ceph-mon[32752]: pgmap v10482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18418.039806 secs 2024-04-26T23:42:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18417.263053 secs 2024-04-26T23:42:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:16 smithi167 ceph-mon[32751]: pgmap v10482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18418.039806 secs 2024-04-26T23:42:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:18 smithi119 ceph-mon[32752]: pgmap v10483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:18 smithi167 ceph-mon[32751]: pgmap v10483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:18 smithi090 ceph-mon[29180]: pgmap v10483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:20 smithi119 ceph-mon[32752]: pgmap v10484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:20 smithi167 ceph-mon[32751]: pgmap v10484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:20 smithi090 ceph-mon[29180]: pgmap v10484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18422.263143 secs 2024-04-26T23:42:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18423.039915 secs 2024-04-26T23:42:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:42:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18422.263143 secs 2024-04-26T23:42:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18423.039915 secs 2024-04-26T23:42:22.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:42:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18422.263143 secs 2024-04-26T23:42:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18423.039915 secs 2024-04-26T23:42:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:42:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:22 smithi119 ceph-mon[32752]: pgmap v10485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:42:23.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:42:23.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:42:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:22 smithi167 ceph-mon[32751]: pgmap v10485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:42:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:42:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:42:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:22 smithi090 ceph-mon[29180]: pgmap v10485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:23.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:42:23.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:42:23.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:42:24.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:23 smithi119 ceph-mon[32752]: pgmap v10486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:42:24.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:23 smithi119 ceph-mon[32752]: pgmap v10487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:42:24.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:23 smithi167 ceph-mon[32751]: pgmap v10486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:42:24.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:23 smithi167 ceph-mon[32751]: pgmap v10487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:42:24.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:23 smithi090 ceph-mon[29180]: pgmap v10486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:42:24.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:23 smithi090 ceph-mon[29180]: pgmap v10487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:42:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:25 smithi119 ceph-mon[32752]: pgmap v10488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:26.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:25 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18427.263229 secs 2024-04-26T23:42:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:25 smithi167 ceph-mon[32751]: pgmap v10488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:25 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18427.263229 secs 2024-04-26T23:42:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:25 smithi090 ceph-mon[29180]: pgmap v10488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:25 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18427.263229 secs 2024-04-26T23:42:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18428.040027 secs 2024-04-26T23:42:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18428.040027 secs 2024-04-26T23:42:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18428.040027 secs 2024-04-26T23:42:28.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:27 smithi119 ceph-mon[32752]: pgmap v10489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:28.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:27 smithi167 ceph-mon[32751]: pgmap v10489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:28.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:27 smithi090 ceph-mon[29180]: pgmap v10489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:30.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:29 smithi119 ceph-mon[32752]: pgmap v10490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:42:30.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:29 smithi167 ceph-mon[32751]: pgmap v10490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:42:30.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:29 smithi090 ceph-mon[29180]: pgmap v10490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:42:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:30 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18432.263311 secs 2024-04-26T23:42:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:30 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18432.263311 secs 2024-04-26T23:42:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:30 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18432.263311 secs 2024-04-26T23:42:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:31 smithi119 ceph-mon[32752]: pgmap v10491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18433.040153 secs 2024-04-26T23:42:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:31 smithi167 ceph-mon[32751]: pgmap v10491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18433.040153 secs 2024-04-26T23:42:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:31 smithi090 ceph-mon[29180]: pgmap v10491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:42:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18433.040153 secs 2024-04-26T23:42:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:33 smithi119 ceph-mon[32752]: pgmap v10492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:42:34.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:33 smithi167 ceph-mon[32751]: pgmap v10492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:42:34.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:33 smithi090 ceph-mon[29180]: pgmap v10492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:42:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:35 smithi119 ceph-mon[32752]: pgmap v10493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:35 smithi167 ceph-mon[32751]: pgmap v10493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:35.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:35 smithi090 ceph-mon[29180]: pgmap v10493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18437.263372 secs 2024-04-26T23:42:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18437.263372 secs 2024-04-26T23:42:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18437.263372 secs 2024-04-26T23:42:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:37 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18438.040299 secs 2024-04-26T23:42:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:37 smithi119 ceph-mon[32752]: pgmap v10494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:37 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18438.040299 secs 2024-04-26T23:42:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:37 smithi167 ceph-mon[32751]: pgmap v10494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:37 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18438.040299 secs 2024-04-26T23:42:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:37 smithi090 ceph-mon[29180]: pgmap v10494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:39.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:39 smithi119 ceph-mon[32752]: pgmap v10495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:39.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:39 smithi167 ceph-mon[32751]: pgmap v10495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:39.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:39 smithi090 ceph-mon[29180]: pgmap v10495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18442.263458 secs 2024-04-26T23:42:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:41 smithi119 ceph-mon[32752]: pgmap v10496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:41.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18443.040368 secs 2024-04-26T23:42:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18442.263458 secs 2024-04-26T23:42:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:41 smithi167 ceph-mon[32751]: pgmap v10496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18443.040368 secs 2024-04-26T23:42:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18442.263458 secs 2024-04-26T23:42:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:41 smithi090 ceph-mon[29180]: pgmap v10496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18443.040368 secs 2024-04-26T23:42:43.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:43 smithi119 ceph-mon[32752]: pgmap v10497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:43.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:43 smithi167 ceph-mon[32751]: pgmap v10497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:43.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:43 smithi090 ceph-mon[29180]: pgmap v10497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:45.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:45 smithi119 ceph-mon[32752]: pgmap v10498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:45.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:45 smithi167 ceph-mon[32751]: pgmap v10498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:45 smithi090 ceph-mon[29180]: pgmap v10498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:46.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18447.263541 secs 2024-04-26T23:42:46.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18448.040486 secs 2024-04-26T23:42:46.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18447.263541 secs 2024-04-26T23:42:46.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18448.040486 secs 2024-04-26T23:42:46.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18447.263541 secs 2024-04-26T23:42:46.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18448.040486 secs 2024-04-26T23:42:47.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:47 smithi090 ceph-mon[29180]: pgmap v10499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:47.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:47 smithi119 ceph-mon[32752]: pgmap v10499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:47.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:47 smithi167 ceph-mon[32751]: pgmap v10499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:49.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:49 smithi119 ceph-mon[32752]: pgmap v10500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:49.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:49 smithi167 ceph-mon[32751]: pgmap v10500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:49.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:49 smithi090 ceph-mon[29180]: pgmap v10500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:51.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18452.263625 secs 2024-04-26T23:42:51.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:51 smithi119 ceph-mon[32752]: pgmap v10501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:51.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18453.040565 secs 2024-04-26T23:42:51.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18452.263625 secs 2024-04-26T23:42:51.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:51 smithi167 ceph-mon[32751]: pgmap v10501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:51.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18453.040565 secs 2024-04-26T23:42:51.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18452.263625 secs 2024-04-26T23:42:51.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:51 smithi090 ceph-mon[29180]: pgmap v10501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:51.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18453.040565 secs 2024-04-26T23:42:53.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:53 smithi119 ceph-mon[32752]: pgmap v10502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:53.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:53 smithi167 ceph-mon[32751]: pgmap v10502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:53.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:53 smithi090 ceph-mon[29180]: pgmap v10502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:55.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:55 smithi119 ceph-mon[32752]: pgmap v10503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:55.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:55 smithi167 ceph-mon[32751]: pgmap v10503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:55.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:55 smithi090 ceph-mon[29180]: pgmap v10503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:42:56.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18457.263707 secs 2024-04-26T23:42:56.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18458.040667 secs 2024-04-26T23:42:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18457.263707 secs 2024-04-26T23:42:56.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18458.040667 secs 2024-04-26T23:42:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18457.263707 secs 2024-04-26T23:42:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18458.040667 secs 2024-04-26T23:42:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:57 smithi119 ceph-mon[32752]: pgmap v10504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:57.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:57 smithi167 ceph-mon[32751]: pgmap v10504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:57 smithi090 ceph-mon[29180]: pgmap v10504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:59.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:42:59 smithi119 ceph-mon[32752]: pgmap v10505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:59.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:42:59 smithi167 ceph-mon[32751]: pgmap v10505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:42:59.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:42:59 smithi090 ceph-mon[29180]: pgmap v10505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:01.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18462.263814 secs 2024-04-26T23:43:01.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:01 smithi119 ceph-mon[32752]: pgmap v10506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:01.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18463.040738 secs 2024-04-26T23:43:01.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18462.263814 secs 2024-04-26T23:43:01.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:01 smithi167 ceph-mon[32751]: pgmap v10506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:01.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18463.040738 secs 2024-04-26T23:43:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18462.263814 secs 2024-04-26T23:43:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:01 smithi090 ceph-mon[29180]: pgmap v10506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18463.040738 secs 2024-04-26T23:43:03.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:03 smithi119 ceph-mon[32752]: pgmap v10507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:03.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:03 smithi167 ceph-mon[32751]: pgmap v10507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:03.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:03 smithi090 ceph-mon[29180]: pgmap v10507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:05.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:05 smithi119 ceph-mon[32752]: pgmap v10508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:05.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:05 smithi167 ceph-mon[32751]: pgmap v10508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:05.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:05 smithi090 ceph-mon[29180]: pgmap v10508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:06.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18467.263871 secs 2024-04-26T23:43:06.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18468.040812 secs 2024-04-26T23:43:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18467.263871 secs 2024-04-26T23:43:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18468.040812 secs 2024-04-26T23:43:06.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18467.263871 secs 2024-04-26T23:43:06.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18468.040812 secs 2024-04-26T23:43:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:07 smithi090 ceph-mon[29180]: pgmap v10509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:07 smithi119 ceph-mon[32752]: pgmap v10509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:07 smithi167 ceph-mon[32751]: pgmap v10509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:09 smithi090 ceph-mon[29180]: pgmap v10510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:09 smithi119 ceph-mon[32752]: pgmap v10510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:09 smithi167 ceph-mon[32751]: pgmap v10510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18472.263957 secs 2024-04-26T23:43:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:11 smithi090 ceph-mon[29180]: pgmap v10511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18473.040869 secs 2024-04-26T23:43:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18472.263957 secs 2024-04-26T23:43:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:11 smithi119 ceph-mon[32752]: pgmap v10511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18473.040869 secs 2024-04-26T23:43:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18472.263957 secs 2024-04-26T23:43:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:11 smithi167 ceph-mon[32751]: pgmap v10511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18473.040869 secs 2024-04-26T23:43:14.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:13 smithi090 ceph-mon[29180]: pgmap v10512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:14.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:13 smithi119 ceph-mon[32752]: pgmap v10512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:14.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:13 smithi167 ceph-mon[32751]: pgmap v10512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:16.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:15 smithi090 ceph-mon[29180]: pgmap v10513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:15 smithi119 ceph-mon[32752]: pgmap v10513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:15 smithi167 ceph-mon[32751]: pgmap v10513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18477.264040 secs 2024-04-26T23:43:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18478.040962 secs 2024-04-26T23:43:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18477.264040 secs 2024-04-26T23:43:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18478.040962 secs 2024-04-26T23:43:17.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18477.264040 secs 2024-04-26T23:43:17.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18478.040962 secs 2024-04-26T23:43:18.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:17 smithi119 ceph-mon[32752]: pgmap v10514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:18.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:17 smithi167 ceph-mon[32751]: pgmap v10514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:18.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:17 smithi090 ceph-mon[29180]: pgmap v10514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:19 smithi119 ceph-mon[32752]: pgmap v10515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:19 smithi167 ceph-mon[32751]: pgmap v10515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:20.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:19 smithi090 ceph-mon[29180]: pgmap v10515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18482.264124 secs 2024-04-26T23:43:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:21 smithi119 ceph-mon[32752]: pgmap v10516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18483.041078 secs 2024-04-26T23:43:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18482.264124 secs 2024-04-26T23:43:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:21 smithi167 ceph-mon[32751]: pgmap v10516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18483.041078 secs 2024-04-26T23:43:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18482.264124 secs 2024-04-26T23:43:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:21 smithi090 ceph-mon[29180]: pgmap v10516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:22.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18483.041078 secs 2024-04-26T23:43:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:43:24.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:23 smithi119 ceph-mon[32752]: pgmap v10517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:24.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:43:24.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:43:24.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:23 smithi119 ceph-mon[32752]: pgmap v10518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:24.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:43:24.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:23 smithi167 ceph-mon[32751]: pgmap v10517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:24.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:43:24.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:43:24.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:23 smithi167 ceph-mon[32751]: pgmap v10518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:24.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:43:24.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:23 smithi090 ceph-mon[29180]: pgmap v10517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:24.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:43:24.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:43:24.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:23 smithi090 ceph-mon[29180]: pgmap v10518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:24.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:43:25.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:25 smithi119 ceph-mon[32752]: pgmap v10519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:25 smithi167 ceph-mon[32751]: pgmap v10519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:25.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:25 smithi090 ceph-mon[29180]: pgmap v10519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18487.264183 secs 2024-04-26T23:43:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18487.264183 secs 2024-04-26T23:43:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18487.264183 secs 2024-04-26T23:43:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:27 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18488.041154 secs 2024-04-26T23:43:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:27 smithi119 ceph-mon[32752]: pgmap v10520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:27 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18488.041154 secs 2024-04-26T23:43:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:27 smithi167 ceph-mon[32751]: pgmap v10520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:27 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18488.041154 secs 2024-04-26T23:43:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:27 smithi090 ceph-mon[29180]: pgmap v10520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:29.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:29 smithi119 ceph-mon[32752]: pgmap v10521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:43:29.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:29 smithi167 ceph-mon[32751]: pgmap v10521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:43:29.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:29 smithi090 ceph-mon[29180]: pgmap v10521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:43:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:30 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18492.264269 secs 2024-04-26T23:43:31.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:30 smithi119 ceph-mon[32752]: pgmap v10522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:30 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18492.264269 secs 2024-04-26T23:43:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:30 smithi167 ceph-mon[32751]: pgmap v10522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:30 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18492.264269 secs 2024-04-26T23:43:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:30 smithi090 ceph-mon[29180]: pgmap v10522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18493.041289 secs 2024-04-26T23:43:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18493.041289 secs 2024-04-26T23:43:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18493.041289 secs 2024-04-26T23:43:33.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:33 smithi119 ceph-mon[32752]: pgmap v10523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:33.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:33 smithi167 ceph-mon[32751]: pgmap v10523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:33.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:33 smithi090 ceph-mon[29180]: pgmap v10523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:43:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:35 smithi119 ceph-mon[32752]: pgmap v10524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:35 smithi167 ceph-mon[32751]: pgmap v10524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:35.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:35 smithi090 ceph-mon[29180]: pgmap v10524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18497.264352 secs 2024-04-26T23:43:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18497.264352 secs 2024-04-26T23:43:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18497.264352 secs 2024-04-26T23:43:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:37 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18498.041407 secs 2024-04-26T23:43:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:37 smithi119 ceph-mon[32752]: pgmap v10525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:37 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18498.041407 secs 2024-04-26T23:43:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:37 smithi167 ceph-mon[32751]: pgmap v10525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:37 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18498.041407 secs 2024-04-26T23:43:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:37 smithi090 ceph-mon[29180]: pgmap v10525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:39.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:39 smithi119 ceph-mon[32752]: pgmap v10526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:39.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:39 smithi167 ceph-mon[32751]: pgmap v10526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:39.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:39 smithi090 ceph-mon[29180]: pgmap v10526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:41.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:40 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18502.264432 secs 2024-04-26T23:43:41.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:40 smithi119 ceph-mon[32752]: pgmap v10527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:41.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:40 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18502.264432 secs 2024-04-26T23:43:41.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:40 smithi167 ceph-mon[32751]: pgmap v10527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:41.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:40 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18502.264432 secs 2024-04-26T23:43:41.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:40 smithi090 ceph-mon[29180]: pgmap v10527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18503.041509 secs 2024-04-26T23:43:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:42 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18503.041509 secs 2024-04-26T23:43:42.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18503.041509 secs 2024-04-26T23:43:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:43 smithi119 ceph-mon[32752]: pgmap v10528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:43.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:43 smithi167 ceph-mon[32751]: pgmap v10528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:43.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:43 smithi090 ceph-mon[29180]: pgmap v10528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:45.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:45 smithi119 ceph-mon[32752]: pgmap v10529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:45.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:45 smithi167 ceph-mon[32751]: pgmap v10529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:45.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:45 smithi090 ceph-mon[29180]: pgmap v10529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:46.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18507.264516 secs 2024-04-26T23:43:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18507.264516 secs 2024-04-26T23:43:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18507.264516 secs 2024-04-26T23:43:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:47 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18508.041590 secs 2024-04-26T23:43:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:47 smithi119 ceph-mon[32752]: pgmap v10530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:47 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18508.041590 secs 2024-04-26T23:43:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:47 smithi167 ceph-mon[32751]: pgmap v10530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:47 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18508.041590 secs 2024-04-26T23:43:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:47 smithi090 ceph-mon[29180]: pgmap v10530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:49 smithi119 ceph-mon[32752]: pgmap v10531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:49.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:49 smithi167 ceph-mon[32751]: pgmap v10531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:49 smithi090 ceph-mon[29180]: pgmap v10531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:50 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18512.264577 secs 2024-04-26T23:43:51.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:50 smithi119 ceph-mon[32752]: pgmap v10532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:50 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18512.264577 secs 2024-04-26T23:43:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:50 smithi167 ceph-mon[32751]: pgmap v10532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:50 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18512.264577 secs 2024-04-26T23:43:51.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:50 smithi090 ceph-mon[29180]: pgmap v10532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18513.041653 secs 2024-04-26T23:43:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18513.041653 secs 2024-04-26T23:43:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18513.041653 secs 2024-04-26T23:43:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:53 smithi119 ceph-mon[32752]: pgmap v10533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:53.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:53 smithi167 ceph-mon[32751]: pgmap v10533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:53.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:53 smithi090 ceph-mon[29180]: pgmap v10533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:55.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:55 smithi119 ceph-mon[32752]: pgmap v10534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:55.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:55 smithi167 ceph-mon[32751]: pgmap v10534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:55.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:55 smithi090 ceph-mon[29180]: pgmap v10534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:56.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18517.264636 secs 2024-04-26T23:43:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18517.264636 secs 2024-04-26T23:43:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18517.264636 secs 2024-04-26T23:43:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18518.041773 secs 2024-04-26T23:43:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:57 smithi119 ceph-mon[32752]: pgmap v10535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18518.041773 secs 2024-04-26T23:43:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:57 smithi167 ceph-mon[32751]: pgmap v10535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18518.041773 secs 2024-04-26T23:43:57.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:57 smithi090 ceph-mon[29180]: pgmap v10535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:43:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:43:59 smithi119 ceph-mon[32752]: pgmap v10536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:59.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:43:59 smithi167 ceph-mon[32751]: pgmap v10536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:43:59.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:43:59 smithi090 ceph-mon[29180]: pgmap v10536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:00 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18522.264723 secs 2024-04-26T23:44:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:00 smithi119 ceph-mon[32752]: pgmap v10537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:01.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:00 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18522.264723 secs 2024-04-26T23:44:01.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:00 smithi167 ceph-mon[32751]: pgmap v10537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:00 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18522.264723 secs 2024-04-26T23:44:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:00 smithi090 ceph-mon[29180]: pgmap v10537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18523.041888 secs 2024-04-26T23:44:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18523.041888 secs 2024-04-26T23:44:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18523.041888 secs 2024-04-26T23:44:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:03 smithi119 ceph-mon[32752]: pgmap v10538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:03 smithi167 ceph-mon[32751]: pgmap v10538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:03.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:03 smithi090 ceph-mon[29180]: pgmap v10538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:05 smithi119 ceph-mon[32752]: pgmap v10539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:05 smithi167 ceph-mon[32751]: pgmap v10539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:05.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:05 smithi090 ceph-mon[29180]: pgmap v10539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:06.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18527.264824 secs 2024-04-26T23:44:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18527.264824 secs 2024-04-26T23:44:06.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18527.264824 secs 2024-04-26T23:44:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:07 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18528.042006 secs 2024-04-26T23:44:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:07 smithi119 ceph-mon[32752]: pgmap v10540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:07 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18528.042006 secs 2024-04-26T23:44:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:07 smithi167 ceph-mon[32751]: pgmap v10540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:07 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18528.042006 secs 2024-04-26T23:44:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:07 smithi090 ceph-mon[29180]: pgmap v10540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:09 smithi119 ceph-mon[32752]: pgmap v10541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:09 smithi167 ceph-mon[32751]: pgmap v10541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:09 smithi090 ceph-mon[29180]: pgmap v10541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:10 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18532.264915 secs 2024-04-26T23:44:11.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:10 smithi119 ceph-mon[32752]: pgmap v10542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:10 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18532.264915 secs 2024-04-26T23:44:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:10 smithi167 ceph-mon[32751]: pgmap v10542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:10 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18532.264915 secs 2024-04-26T23:44:11.818 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:10 smithi090 ceph-mon[29180]: pgmap v10542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:12 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18533.042120 secs 2024-04-26T23:44:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:12 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18533.042120 secs 2024-04-26T23:44:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:12 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18533.042120 secs 2024-04-26T23:44:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:13 smithi119 ceph-mon[32752]: pgmap v10543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:13 smithi167 ceph-mon[32751]: pgmap v10543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:13 smithi090 ceph-mon[29180]: pgmap v10543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:15.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:15 smithi119 ceph-mon[32752]: pgmap v10544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:15.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:15 smithi167 ceph-mon[32751]: pgmap v10544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:15 smithi090 ceph-mon[29180]: pgmap v10544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18537.265005 secs 2024-04-26T23:44:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18537.265005 secs 2024-04-26T23:44:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18537.265005 secs 2024-04-26T23:44:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:17 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18538.042195 secs 2024-04-26T23:44:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:17 smithi119 ceph-mon[32752]: pgmap v10545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T23:44:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:17 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18538.042195 secs 2024-04-26T23:44:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:17 smithi167 ceph-mon[32751]: pgmap v10545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T23:44:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:17 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18538.042195 secs 2024-04-26T23:44:17.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:17 smithi090 ceph-mon[29180]: pgmap v10545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-26T23:44:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:19 smithi119 ceph-mon[32752]: pgmap v10546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:19 smithi167 ceph-mon[32751]: pgmap v10546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:19 smithi090 ceph-mon[29180]: pgmap v10546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-26T23:44:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:20 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18542.265067 secs 2024-04-26T23:44:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:20 smithi119 ceph-mon[32752]: pgmap v10547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:44:21.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:20 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18542.265067 secs 2024-04-26T23:44:21.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:20 smithi167 ceph-mon[32751]: pgmap v10547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:44:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:20 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18542.265067 secs 2024-04-26T23:44:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:20 smithi090 ceph-mon[29180]: pgmap v10547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:44:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:22 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18543.042315 secs 2024-04-26T23:44:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:22 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18543.042315 secs 2024-04-26T23:44:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:22 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18543.042315 secs 2024-04-26T23:44:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:23 smithi119 ceph-mon[32752]: pgmap v10548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:44:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:23 smithi167 ceph-mon[32751]: pgmap v10548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:44:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:23 smithi090 ceph-mon[29180]: pgmap v10548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:44:24.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:24.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:24.333 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:44:25.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:25 smithi119 ceph-mon[32752]: pgmap v10549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:25 smithi167 ceph-mon[32751]: pgmap v10549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:25.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:25 smithi090 ceph-mon[29180]: pgmap v10549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:26.029 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 18547.265154 secs 2024-04-26T23:44:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 18547.265154 secs 2024-04-26T23:44:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 18547.265154 secs 2024-04-26T23:44:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:27 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18548.042401 secs 2024-04-26T23:44:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:27 smithi119 ceph-mon[32752]: pgmap v10550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:27 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18548.042401 secs 2024-04-26T23:44:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:27 smithi167 ceph-mon[32751]: pgmap v10550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:27.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:27 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18548.042401 secs 2024-04-26T23:44:27.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:27 smithi090 ceph-mon[29180]: pgmap v10550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:28.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:28.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:28.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: pgmap v10551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-26T23:44:28.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: pgmap v10552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:44:28.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:28.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:28.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: pgmap v10551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-26T23:44:28.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: pgmap v10552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:44:28.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:28.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:44:28.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:44:28.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: pgmap v10551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-26T23:44:28.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: pgmap v10552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-26T23:44:28.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:44:30.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:30 smithi119 ceph-mon[32752]: pgmap v10553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:30.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:30 smithi167 ceph-mon[32751]: pgmap v10553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:30.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:30 smithi090 ceph-mon[29180]: pgmap v10553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18552.265240 secs 2024-04-26T23:44:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18553.042530 secs 2024-04-26T23:44:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18552.265240 secs 2024-04-26T23:44:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18553.042530 secs 2024-04-26T23:44:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18552.265240 secs 2024-04-26T23:44:31.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18553.042530 secs 2024-04-26T23:44:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:32 smithi119 ceph-mon[32752]: pgmap v10554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:32.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:32 smithi167 ceph-mon[32751]: pgmap v10554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:32.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:32 smithi090 ceph-mon[29180]: pgmap v10554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:34.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:34 smithi119 ceph-mon[32752]: pgmap v10555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:34.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:34 smithi167 ceph-mon[32751]: pgmap v10555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:34.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:34 smithi090 ceph-mon[29180]: pgmap v10555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:44:35.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:35 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:44:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:44:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:35 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:44:35.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:44:35.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:35 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:44:36.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:44:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:44:35.860+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:44:36.130 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:44:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:44:35.860+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:44:36.130 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:44:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:44:35.860+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:44:36.130 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:44:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:44:35.860+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:44:36.130 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:44:35 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:44:35.860+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:44:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18557.265325 secs 2024-04-26T23:44:36.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:36 smithi119 ceph-mon[32752]: pgmap v10556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:36.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18558.042627 secs 2024-04-26T23:44:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18557.265325 secs 2024-04-26T23:44:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:36 smithi167 ceph-mon[32751]: pgmap v10556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:36.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18558.042627 secs 2024-04-26T23:44:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18557.265325 secs 2024-04-26T23:44:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:36 smithi090 ceph-mon[29180]: pgmap v10556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:36.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18558.042627 secs 2024-04-26T23:44:38.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:38 smithi119 ceph-mon[32752]: pgmap v10557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:44:38.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:38 smithi167 ceph-mon[32751]: pgmap v10557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:44:38.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:38 smithi090 ceph-mon[29180]: pgmap v10557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:44:40.658 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:40 smithi090 ceph-mon[29180]: pgmap v10558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:40.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:40 smithi119 ceph-mon[32752]: pgmap v10558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:40.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:40 smithi167 ceph-mon[32751]: pgmap v10558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18562.265412 secs 2024-04-26T23:44:41.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18563.042697 secs 2024-04-26T23:44:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18562.265412 secs 2024-04-26T23:44:41.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18563.042697 secs 2024-04-26T23:44:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18562.265412 secs 2024-04-26T23:44:41.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18563.042697 secs 2024-04-26T23:44:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:42 smithi119 ceph-mon[32752]: pgmap v10559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:43.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:42 smithi167 ceph-mon[32751]: pgmap v10559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:43.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:42 smithi090 ceph-mon[29180]: pgmap v10559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:45.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:44 smithi119 ceph-mon[32752]: pgmap v10560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:45.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:44 smithi167 ceph-mon[32751]: pgmap v10560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:45.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:44 smithi090 ceph-mon[29180]: pgmap v10560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:47.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18567.265496 secs 2024-04-26T23:44:47.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:46 smithi090 ceph-mon[29180]: pgmap v10561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:47.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18568.042785 secs 2024-04-26T23:44:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18567.265496 secs 2024-04-26T23:44:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:46 smithi119 ceph-mon[32752]: pgmap v10561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18568.042785 secs 2024-04-26T23:44:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18567.265496 secs 2024-04-26T23:44:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:46 smithi167 ceph-mon[32751]: pgmap v10561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18568.042785 secs 2024-04-26T23:44:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:48 smithi119 ceph-mon[32752]: pgmap v10562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:49.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:48 smithi167 ceph-mon[32751]: pgmap v10562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:48 smithi090 ceph-mon[29180]: pgmap v10562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T23:44:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:50 smithi119 ceph-mon[32752]: pgmap v10563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:50 smithi167 ceph-mon[32751]: pgmap v10563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:50 smithi090 ceph-mon[29180]: pgmap v10563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:44:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18572.265582 secs 2024-04-26T23:44:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18573.042876 secs 2024-04-26T23:44:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18572.265582 secs 2024-04-26T23:44:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18573.042876 secs 2024-04-26T23:44:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18572.265582 secs 2024-04-26T23:44:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18573.042876 secs 2024-04-26T23:44:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:52 smithi119 ceph-mon[32752]: pgmap v10564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:53.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:52 smithi167 ceph-mon[32751]: pgmap v10564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:53.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:52 smithi090 ceph-mon[29180]: pgmap v10564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:55.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:54 smithi090 ceph-mon[29180]: pgmap v10565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:55.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:54 smithi119 ceph-mon[32752]: pgmap v10565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:55.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:54 smithi167 ceph-mon[32751]: pgmap v10565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:44:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18577.265667 secs 2024-04-26T23:44:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:56 smithi119 ceph-mon[32752]: pgmap v10566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18578.042970 secs 2024-04-26T23:44:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18577.265667 secs 2024-04-26T23:44:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:56 smithi167 ceph-mon[32751]: pgmap v10566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18578.042970 secs 2024-04-26T23:44:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18577.265667 secs 2024-04-26T23:44:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:56 smithi090 ceph-mon[29180]: pgmap v10566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:57.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18578.042970 secs 2024-04-26T23:44:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:44:58 smithi119 ceph-mon[32752]: pgmap v10567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:59.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:44:58 smithi167 ceph-mon[32751]: pgmap v10567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:44:59.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:44:58 smithi090 ceph-mon[29180]: pgmap v10567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:00 smithi119 ceph-mon[32752]: pgmap v10568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:00 smithi167 ceph-mon[32751]: pgmap v10568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:00 smithi090 ceph-mon[29180]: pgmap v10568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18582.265752 secs 2024-04-26T23:45:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18583.043039 secs 2024-04-26T23:45:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18582.265752 secs 2024-04-26T23:45:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18583.043039 secs 2024-04-26T23:45:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18582.265752 secs 2024-04-26T23:45:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18583.043039 secs 2024-04-26T23:45:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:02 smithi119 ceph-mon[32752]: pgmap v10569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:02 smithi167 ceph-mon[32751]: pgmap v10569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:03.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:02 smithi090 ceph-mon[29180]: pgmap v10569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:04 smithi119 ceph-mon[32752]: pgmap v10570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:04 smithi167 ceph-mon[32751]: pgmap v10570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:05.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:04 smithi090 ceph-mon[29180]: pgmap v10570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18587.265847 secs 2024-04-26T23:45:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:06 smithi119 ceph-mon[32752]: pgmap v10571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18588.043159 secs 2024-04-26T23:45:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18587.265847 secs 2024-04-26T23:45:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:06 smithi167 ceph-mon[32751]: pgmap v10571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18588.043159 secs 2024-04-26T23:45:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18587.265847 secs 2024-04-26T23:45:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:06 smithi090 ceph-mon[29180]: pgmap v10571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18588.043159 secs 2024-04-26T23:45:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:08 smithi119 ceph-mon[32752]: pgmap v10572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:08 smithi167 ceph-mon[32751]: pgmap v10572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:08 smithi090 ceph-mon[29180]: pgmap v10572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:10 smithi119 ceph-mon[32752]: pgmap v10573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:10 smithi167 ceph-mon[32751]: pgmap v10573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:10 smithi090 ceph-mon[29180]: pgmap v10573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18592.265940 secs 2024-04-26T23:45:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18593.043247 secs 2024-04-26T23:45:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18592.265940 secs 2024-04-26T23:45:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18593.043247 secs 2024-04-26T23:45:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18592.265940 secs 2024-04-26T23:45:12.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18593.043247 secs 2024-04-26T23:45:13.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:12 smithi119 ceph-mon[32752]: pgmap v10574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:12 smithi167 ceph-mon[32751]: pgmap v10574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:13.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:12 smithi090 ceph-mon[29180]: pgmap v10574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:14 smithi119 ceph-mon[32752]: pgmap v10575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:15.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:14 smithi167 ceph-mon[32751]: pgmap v10575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:14 smithi090 ceph-mon[29180]: pgmap v10575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18597.266029 secs 2024-04-26T23:45:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:16 smithi119 ceph-mon[32752]: pgmap v10576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18598.043372 secs 2024-04-26T23:45:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18597.266029 secs 2024-04-26T23:45:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:16 smithi167 ceph-mon[32751]: pgmap v10576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18598.043372 secs 2024-04-26T23:45:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18597.266029 secs 2024-04-26T23:45:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:16 smithi090 ceph-mon[29180]: pgmap v10576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:17.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18598.043372 secs 2024-04-26T23:45:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:18 smithi119 ceph-mon[32752]: pgmap v10577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:18 smithi167 ceph-mon[32751]: pgmap v10577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:18 smithi090 ceph-mon[29180]: pgmap v10577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:20 smithi119 ceph-mon[32752]: pgmap v10578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:20 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18602.266114 secs 2024-04-26T23:45:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:20 smithi167 ceph-mon[32751]: pgmap v10578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:20 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18602.266114 secs 2024-04-26T23:45:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:20 smithi090 ceph-mon[29180]: pgmap v10578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:21.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:20 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18602.266114 secs 2024-04-26T23:45:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18603.043476 secs 2024-04-26T23:45:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18603.043476 secs 2024-04-26T23:45:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18603.043476 secs 2024-04-26T23:45:23.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:22 smithi119 ceph-mon[32752]: pgmap v10579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:22 smithi167 ceph-mon[32751]: pgmap v10579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:23.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:22 smithi090 ceph-mon[29180]: pgmap v10579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:25.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:24 smithi119 ceph-mon[32752]: pgmap v10580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:24 smithi167 ceph-mon[32751]: pgmap v10580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:25.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:24 smithi090 ceph-mon[29180]: pgmap v10580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:25 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18607.266198 secs 2024-04-26T23:45:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:25 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18607.266198 secs 2024-04-26T23:45:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:25 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18607.266198 secs 2024-04-26T23:45:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:26 smithi119 ceph-mon[32752]: pgmap v10581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18608.043609 secs 2024-04-26T23:45:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:26 smithi167 ceph-mon[32751]: pgmap v10581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18608.043609 secs 2024-04-26T23:45:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:26 smithi090 ceph-mon[29180]: pgmap v10581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18608.043609 secs 2024-04-26T23:45:28.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:28.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:28.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:45:29.163 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:28 smithi167 ceph-mon[32751]: pgmap v10582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:29.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:28 smithi119 ceph-mon[32752]: pgmap v10582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:29.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:28 smithi090 ceph-mon[29180]: pgmap v10582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:31.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:30 smithi167 ceph-mon[32751]: pgmap v10583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:31.149 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:30 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18612.266282 secs 2024-04-26T23:45:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:30 smithi119 ceph-mon[32752]: pgmap v10583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:31.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:30 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18612.266282 secs 2024-04-26T23:45:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:30 smithi090 ceph-mon[29180]: pgmap v10583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:31.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:30 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18612.266282 secs 2024-04-26T23:45:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18613.043729 secs 2024-04-26T23:45:32.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:32.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18613.043729 secs 2024-04-26T23:45:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18613.043729 secs 2024-04-26T23:45:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:45:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:45:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:45:33.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:32 smithi119 ceph-mon[32752]: pgmap v10584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:32 smithi119 ceph-mon[32752]: pgmap v10585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s rd, 1 op/s 2024-04-26T23:45:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:32 smithi119 ceph-mon[32752]: pgmap v10586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-26T23:45:33.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:32 smithi167 ceph-mon[32751]: pgmap v10584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:33.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:32 smithi167 ceph-mon[32751]: pgmap v10585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s rd, 1 op/s 2024-04-26T23:45:33.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:32 smithi167 ceph-mon[32751]: pgmap v10586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-26T23:45:33.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:32 smithi090 ceph-mon[29180]: pgmap v10584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:33.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:32 smithi090 ceph-mon[29180]: pgmap v10585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 988 B/s rd, 1 op/s 2024-04-26T23:45:33.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:32 smithi090 ceph-mon[29180]: pgmap v10586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-26T23:45:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:34 smithi119 ceph-mon[32752]: pgmap v10587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:34 smithi167 ceph-mon[32751]: pgmap v10587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:35.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:34 smithi090 ceph-mon[29180]: pgmap v10587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:35 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18617.266367 secs 2024-04-26T23:45:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:35 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18617.266367 secs 2024-04-26T23:45:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:35 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18617.266367 secs 2024-04-26T23:45:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18618.043854 secs 2024-04-26T23:45:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:36 smithi119 ceph-mon[32752]: pgmap v10588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18618.043854 secs 2024-04-26T23:45:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:36 smithi167 ceph-mon[32751]: pgmap v10588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18618.043854 secs 2024-04-26T23:45:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:36 smithi090 ceph-mon[29180]: pgmap v10588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:38.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:38 smithi119 ceph-mon[32752]: pgmap v10589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-26T23:45:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:38 smithi167 ceph-mon[32751]: pgmap v10589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-26T23:45:38.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:38 smithi090 ceph-mon[29180]: pgmap v10589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-26T23:45:41.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:40 smithi119 ceph-mon[32752]: pgmap v10590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:41.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:40 smithi167 ceph-mon[32751]: pgmap v10590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:41.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:40 smithi090 ceph-mon[29180]: pgmap v10590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18622.266452 secs 2024-04-26T23:45:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18623.043948 secs 2024-04-26T23:45:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18622.266452 secs 2024-04-26T23:45:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18623.043948 secs 2024-04-26T23:45:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18622.266452 secs 2024-04-26T23:45:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18623.043948 secs 2024-04-26T23:45:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:42 smithi119 ceph-mon[32752]: pgmap v10591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:45:43.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:42 smithi167 ceph-mon[32751]: pgmap v10591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:45:43.327 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:42 smithi090 ceph-mon[29180]: pgmap v10591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:45:45.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:44 smithi119 ceph-mon[32752]: pgmap v10592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:45.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:44 smithi167 ceph-mon[32751]: pgmap v10592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:45.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:44 smithi090 ceph-mon[29180]: pgmap v10592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:47.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18627.266535 secs 2024-04-26T23:45:47.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18628.044047 secs 2024-04-26T23:45:47.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:46 smithi090 ceph-mon[29180]: pgmap v10593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18627.266535 secs 2024-04-26T23:45:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18628.044047 secs 2024-04-26T23:45:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:46 smithi119 ceph-mon[32752]: pgmap v10593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18627.266535 secs 2024-04-26T23:45:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18628.044047 secs 2024-04-26T23:45:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:46 smithi167 ceph-mon[32751]: pgmap v10593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:48 smithi119 ceph-mon[32752]: pgmap v10594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:49.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:48 smithi167 ceph-mon[32751]: pgmap v10594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:48 smithi090 ceph-mon[29180]: pgmap v10594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:51.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:50 smithi119 ceph-mon[32752]: pgmap v10595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:50 smithi167 ceph-mon[32751]: pgmap v10595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:50 smithi090 ceph-mon[29180]: pgmap v10595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18632.266620 secs 2024-04-26T23:45:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18633.044195 secs 2024-04-26T23:45:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18632.266620 secs 2024-04-26T23:45:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18633.044195 secs 2024-04-26T23:45:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18632.266620 secs 2024-04-26T23:45:52.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18633.044195 secs 2024-04-26T23:45:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:52 smithi090 ceph-mon[29180]: pgmap v10596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:52 smithi119 ceph-mon[32752]: pgmap v10596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:53.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:52 smithi167 ceph-mon[32751]: pgmap v10596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:55.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:54 smithi119 ceph-mon[32752]: pgmap v10597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:55.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:54 smithi167 ceph-mon[32751]: pgmap v10597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:55.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:54 smithi090 ceph-mon[29180]: pgmap v10597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:45:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18637.266705 secs 2024-04-26T23:45:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18638.044324 secs 2024-04-26T23:45:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:56 smithi119 ceph-mon[32752]: pgmap v10598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18637.266705 secs 2024-04-26T23:45:57.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18638.044324 secs 2024-04-26T23:45:57.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:56 smithi167 ceph-mon[32751]: pgmap v10598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:57.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18637.266705 secs 2024-04-26T23:45:57.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18638.044324 secs 2024-04-26T23:45:57.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:56 smithi090 ceph-mon[29180]: pgmap v10598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:45:58 smithi119 ceph-mon[32752]: pgmap v10599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:59.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:45:58 smithi167 ceph-mon[32751]: pgmap v10599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:45:59.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:45:58 smithi090 ceph-mon[29180]: pgmap v10599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:00 smithi119 ceph-mon[32752]: pgmap v10600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:00 smithi167 ceph-mon[32751]: pgmap v10600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:00 smithi090 ceph-mon[29180]: pgmap v10600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18642.266812 secs 2024-04-26T23:46:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18643.044437 secs 2024-04-26T23:46:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18642.266812 secs 2024-04-26T23:46:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18643.044437 secs 2024-04-26T23:46:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18642.266812 secs 2024-04-26T23:46:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18643.044437 secs 2024-04-26T23:46:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:02 smithi119 ceph-mon[32752]: pgmap v10601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:02 smithi167 ceph-mon[32751]: pgmap v10601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:03.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:02 smithi090 ceph-mon[29180]: pgmap v10601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:04 smithi119 ceph-mon[32752]: pgmap v10602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:04 smithi167 ceph-mon[32751]: pgmap v10602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:05.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:04 smithi090 ceph-mon[29180]: pgmap v10602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18647.266883 secs 2024-04-26T23:46:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:06 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 18648.044550 secs 2024-04-26T23:46:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:06 smithi119 ceph-mon[32752]: slow request 720.117108 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:46:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:06 smithi119 ceph-mon[32752]: pgmap v10603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18647.266883 secs 2024-04-26T23:46:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:06 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 18648.044550 secs 2024-04-26T23:46:07.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:06 smithi167 ceph-mon[32751]: slow request 720.117108 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:46:07.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:06 smithi167 ceph-mon[32751]: pgmap v10603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18647.266883 secs 2024-04-26T23:46:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:06 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 18648.044550 secs 2024-04-26T23:46:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:06 smithi090 ceph-mon[29180]: slow request 720.117108 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:46:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:06 smithi090 ceph-mon[29180]: pgmap v10603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:08 smithi119 ceph-mon[32752]: pgmap v10604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:08 smithi167 ceph-mon[32751]: pgmap v10604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:08 smithi090 ceph-mon[29180]: pgmap v10604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:10 smithi119 ceph-mon[32752]: pgmap v10605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:10 smithi167 ceph-mon[32751]: pgmap v10605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:10 smithi090 ceph-mon[29180]: pgmap v10605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18652.266973 secs 2024-04-26T23:46:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18653.044664 secs 2024-04-26T23:46:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18652.266973 secs 2024-04-26T23:46:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18653.044664 secs 2024-04-26T23:46:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18652.266973 secs 2024-04-26T23:46:12.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18653.044664 secs 2024-04-26T23:46:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:12 smithi119 ceph-mon[32752]: pgmap v10606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:12 smithi167 ceph-mon[32751]: pgmap v10606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:12 smithi090 ceph-mon[29180]: pgmap v10606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:14 smithi119 ceph-mon[32752]: pgmap v10607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:15.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:14 smithi167 ceph-mon[32751]: pgmap v10607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:14 smithi090 ceph-mon[29180]: pgmap v10607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:15 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18657.267035 secs 2024-04-26T23:46:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:15 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18657.267035 secs 2024-04-26T23:46:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:15 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18657.267035 secs 2024-04-26T23:46:17.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18658.044781 secs 2024-04-26T23:46:17.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:16 smithi090 ceph-mon[29180]: pgmap v10608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18658.044781 secs 2024-04-26T23:46:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:16 smithi119 ceph-mon[32752]: pgmap v10608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18658.044781 secs 2024-04-26T23:46:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:16 smithi167 ceph-mon[32751]: pgmap v10608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:18 smithi119 ceph-mon[32752]: pgmap v10609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:18 smithi167 ceph-mon[32751]: pgmap v10609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:19.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:18 smithi090 ceph-mon[29180]: pgmap v10609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:20 smithi119 ceph-mon[32752]: pgmap v10610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:20 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18662.267119 secs 2024-04-26T23:46:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:20 smithi167 ceph-mon[32751]: pgmap v10610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:20 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18662.267119 secs 2024-04-26T23:46:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:20 smithi090 ceph-mon[29180]: pgmap v10610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:21.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:20 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18662.267119 secs 2024-04-26T23:46:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18663.044879 secs 2024-04-26T23:46:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18663.044879 secs 2024-04-26T23:46:22.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18663.044879 secs 2024-04-26T23:46:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:22 smithi119 ceph-mon[32752]: pgmap v10611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:22 smithi167 ceph-mon[32751]: pgmap v10611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:22 smithi090 ceph-mon[29180]: pgmap v10611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:25.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:24 smithi119 ceph-mon[32752]: pgmap v10612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:24 smithi167 ceph-mon[32751]: pgmap v10612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:25.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:24 smithi090 ceph-mon[29180]: pgmap v10612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:25 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18667.267206 secs 2024-04-26T23:46:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:25 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18667.267206 secs 2024-04-26T23:46:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:25 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18667.267206 secs 2024-04-26T23:46:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18668.045003 secs 2024-04-26T23:46:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:26 smithi119 ceph-mon[32752]: pgmap v10613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18668.045003 secs 2024-04-26T23:46:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:26 smithi167 ceph-mon[32751]: pgmap v10613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18668.045003 secs 2024-04-26T23:46:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:26 smithi090 ceph-mon[29180]: pgmap v10613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:28.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:28 smithi119 ceph-mon[32752]: pgmap v10614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:28.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:28 smithi167 ceph-mon[32751]: pgmap v10614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:28.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:28 smithi090 ceph-mon[29180]: pgmap v10614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:31.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:30 smithi119 ceph-mon[32752]: pgmap v10615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:30 smithi167 ceph-mon[32751]: pgmap v10615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:30 smithi090 ceph-mon[29180]: pgmap v10615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18672.267292 secs 2024-04-26T23:46:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18673.045119 secs 2024-04-26T23:46:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18672.267292 secs 2024-04-26T23:46:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18673.045119 secs 2024-04-26T23:46:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18672.267292 secs 2024-04-26T23:46:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18673.045119 secs 2024-04-26T23:46:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:32 smithi119 ceph-mon[32752]: pgmap v10616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:46:33.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:32 smithi167 ceph-mon[32751]: pgmap v10616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:33.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:33.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:33.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:33.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:46:33.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:32 smithi090 ceph-mon[29180]: pgmap v10616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:33.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:46:33.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:46:33.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:46:33.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:46:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:33 smithi119 ceph-mon[32752]: pgmap v10617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T23:46:34.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:33 smithi119 ceph-mon[32752]: pgmap v10618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T23:46:34.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:33 smithi167 ceph-mon[32751]: pgmap v10617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T23:46:34.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:33 smithi167 ceph-mon[32751]: pgmap v10618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T23:46:34.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:33 smithi090 ceph-mon[29180]: pgmap v10617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-26T23:46:34.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:33 smithi090 ceph-mon[29180]: pgmap v10618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-26T23:46:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:35 smithi119 ceph-mon[32752]: pgmap v10619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:35 smithi167 ceph-mon[32751]: pgmap v10619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:35 smithi090 ceph-mon[29180]: pgmap v10619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18677.267350 secs 2024-04-26T23:46:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18678.045242 secs 2024-04-26T23:46:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18677.267350 secs 2024-04-26T23:46:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18678.045242 secs 2024-04-26T23:46:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18677.267350 secs 2024-04-26T23:46:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18678.045242 secs 2024-04-26T23:46:38.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:37 smithi119 ceph-mon[32752]: pgmap v10620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:37 smithi167 ceph-mon[32751]: pgmap v10620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:38.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:37 smithi090 ceph-mon[29180]: pgmap v10620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:40.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:39 smithi119 ceph-mon[32752]: pgmap v10621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:40.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:39 smithi167 ceph-mon[32751]: pgmap v10621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:40.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:39 smithi090 ceph-mon[29180]: pgmap v10621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18682.267409 secs 2024-04-26T23:46:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:41 smithi119 ceph-mon[32752]: pgmap v10622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18683.045353 secs 2024-04-26T23:46:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18682.267409 secs 2024-04-26T23:46:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:41 smithi167 ceph-mon[32751]: pgmap v10622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18683.045353 secs 2024-04-26T23:46:42.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18682.267409 secs 2024-04-26T23:46:42.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:41 smithi090 ceph-mon[29180]: pgmap v10622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:46:42.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18683.045353 secs 2024-04-26T23:46:44.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:43 smithi090 ceph-mon[29180]: pgmap v10623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:46:44.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:43 smithi119 ceph-mon[32752]: pgmap v10623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:46:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:43 smithi167 ceph-mon[32751]: pgmap v10623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:46:46.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:45 smithi119 ceph-mon[32752]: pgmap v10624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:45 smithi167 ceph-mon[32751]: pgmap v10624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:45 smithi090 ceph-mon[29180]: pgmap v10624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:47.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18687.267493 secs 2024-04-26T23:46:47.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18688.045455 secs 2024-04-26T23:46:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18687.267493 secs 2024-04-26T23:46:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18688.045455 secs 2024-04-26T23:46:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18687.267493 secs 2024-04-26T23:46:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18688.045455 secs 2024-04-26T23:46:48.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:47 smithi119 ceph-mon[32752]: pgmap v10625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:48.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:47 smithi167 ceph-mon[32751]: pgmap v10625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:48.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:47 smithi090 ceph-mon[29180]: pgmap v10625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:49 smithi119 ceph-mon[32752]: pgmap v10626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:50.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:49 smithi167 ceph-mon[32751]: pgmap v10626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:49 smithi090 ceph-mon[29180]: pgmap v10626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:50 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18692.267577 secs 2024-04-26T23:46:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:50 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18692.267577 secs 2024-04-26T23:46:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:50 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18692.267577 secs 2024-04-26T23:46:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:51 smithi119 ceph-mon[32752]: pgmap v10627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18693.045563 secs 2024-04-26T23:46:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:51 smithi167 ceph-mon[32751]: pgmap v10627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18693.045563 secs 2024-04-26T23:46:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:51 smithi090 ceph-mon[29180]: pgmap v10627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:52.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18693.045563 secs 2024-04-26T23:46:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:53 smithi119 ceph-mon[32752]: pgmap v10628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:53 smithi167 ceph-mon[32751]: pgmap v10628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:53 smithi090 ceph-mon[29180]: pgmap v10628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:56.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:55 smithi119 ceph-mon[32752]: pgmap v10629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:56.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:55 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18697.267660 secs 2024-04-26T23:46:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:55 smithi167 ceph-mon[32751]: pgmap v10629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:55 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18697.267660 secs 2024-04-26T23:46:56.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:55 smithi090 ceph-mon[29180]: pgmap v10629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:46:56.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:55 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18697.267660 secs 2024-04-26T23:46:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18698.045704 secs 2024-04-26T23:46:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18698.045704 secs 2024-04-26T23:46:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18698.045704 secs 2024-04-26T23:46:58.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:57 smithi119 ceph-mon[32752]: pgmap v10630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:58.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:57 smithi167 ceph-mon[32751]: pgmap v10630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:46:58.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:57 smithi090 ceph-mon[29180]: pgmap v10630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:00.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:46:59 smithi119 ceph-mon[32752]: pgmap v10631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:00.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:46:59 smithi167 ceph-mon[32751]: pgmap v10631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:00.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:46:59 smithi090 ceph-mon[29180]: pgmap v10631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:01.249 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:00 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18702.267745 secs 2024-04-26T23:47:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:00 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18702.267745 secs 2024-04-26T23:47:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:00 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18702.267745 secs 2024-04-26T23:47:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:01 smithi119 ceph-mon[32752]: pgmap v10632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18703.045782 secs 2024-04-26T23:47:02.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:01 smithi167 ceph-mon[32751]: pgmap v10632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:02.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18703.045782 secs 2024-04-26T23:47:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:01 smithi090 ceph-mon[29180]: pgmap v10632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18703.045782 secs 2024-04-26T23:47:04.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:04 smithi119 ceph-mon[32752]: pgmap v10633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:04.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:04 smithi167 ceph-mon[32751]: pgmap v10633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:04 smithi090 ceph-mon[29180]: pgmap v10633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:05 smithi119 ceph-mon[32752]: pgmap v10634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:05 smithi167 ceph-mon[32751]: pgmap v10634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:05.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:05 smithi090 ceph-mon[29180]: pgmap v10634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18707.267843 secs 2024-04-26T23:47:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18707.267843 secs 2024-04-26T23:47:06.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18707.267843 secs 2024-04-26T23:47:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:07 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18708.045860 secs 2024-04-26T23:47:07.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:07 smithi167 ceph-mon[32751]: pgmap v10635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:07 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18708.045860 secs 2024-04-26T23:47:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:07 smithi090 ceph-mon[29180]: pgmap v10635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:07.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:07 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18708.045860 secs 2024-04-26T23:47:07.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:07 smithi119 ceph-mon[32752]: pgmap v10635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:10.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:09 smithi119 ceph-mon[32752]: pgmap v10636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:10.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:09 smithi167 ceph-mon[32751]: pgmap v10636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:09 smithi090 ceph-mon[29180]: pgmap v10636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18712.267927 secs 2024-04-26T23:47:12.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:11 smithi119 ceph-mon[32752]: pgmap v10637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:12.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18713.045946 secs 2024-04-26T23:47:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18712.267927 secs 2024-04-26T23:47:12.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:11 smithi167 ceph-mon[32751]: pgmap v10637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:12.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18713.045946 secs 2024-04-26T23:47:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18712.267927 secs 2024-04-26T23:47:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:11 smithi090 ceph-mon[29180]: pgmap v10637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18713.045946 secs 2024-04-26T23:47:14.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:13 smithi119 ceph-mon[32752]: pgmap v10638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:14.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:13 smithi167 ceph-mon[32751]: pgmap v10638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:14.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:13 smithi090 ceph-mon[29180]: pgmap v10638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:15.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:15 smithi090 ceph-mon[29180]: pgmap v10639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:16.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:15 smithi119 ceph-mon[32752]: pgmap v10639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:16.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:15 smithi167 ceph-mon[32751]: pgmap v10639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18717.267989 secs 2024-04-26T23:47:17.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18718.046026 secs 2024-04-26T23:47:17.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18717.267989 secs 2024-04-26T23:47:17.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18718.046026 secs 2024-04-26T23:47:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18717.267989 secs 2024-04-26T23:47:17.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18718.046026 secs 2024-04-26T23:47:18.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:17 smithi119 ceph-mon[32752]: pgmap v10640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:18.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:17 smithi167 ceph-mon[32751]: pgmap v10640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:18.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:17 smithi090 ceph-mon[29180]: pgmap v10640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:20.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:19 smithi119 ceph-mon[32752]: pgmap v10641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:20.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:19 smithi167 ceph-mon[32751]: pgmap v10641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:20.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:19 smithi090 ceph-mon[29180]: pgmap v10641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18722.268048 secs 2024-04-26T23:47:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:21 smithi119 ceph-mon[32752]: pgmap v10642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:22.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18723.046148 secs 2024-04-26T23:47:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18722.268048 secs 2024-04-26T23:47:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:21 smithi167 ceph-mon[32751]: pgmap v10642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18723.046148 secs 2024-04-26T23:47:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18722.268048 secs 2024-04-26T23:47:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:21 smithi090 ceph-mon[29180]: pgmap v10642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:22.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18723.046148 secs 2024-04-26T23:47:24.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:23 smithi119 ceph-mon[32752]: pgmap v10643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:24.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:23 smithi167 ceph-mon[32751]: pgmap v10643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:24.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:23 smithi090 ceph-mon[29180]: pgmap v10643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:26.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:25 smithi119 ceph-mon[32752]: pgmap v10644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:25 smithi167 ceph-mon[32751]: pgmap v10644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:26.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:25 smithi090 ceph-mon[29180]: pgmap v10644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 18727.268110 secs 2024-04-26T23:47:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18728.046297 secs 2024-04-26T23:47:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 18727.268110 secs 2024-04-26T23:47:27.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18728.046297 secs 2024-04-26T23:47:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 18727.268110 secs 2024-04-26T23:47:27.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18728.046297 secs 2024-04-26T23:47:28.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:27 smithi119 ceph-mon[32752]: pgmap v10645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:28.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:27 smithi167 ceph-mon[32751]: pgmap v10645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:28.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:27 smithi090 ceph-mon[29180]: pgmap v10645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:30.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:29 smithi119 ceph-mon[32752]: pgmap v10646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:30.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:29 smithi167 ceph-mon[32751]: pgmap v10646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:30.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:29 smithi090 ceph-mon[29180]: pgmap v10646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:32.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:31 smithi119 ceph-mon[32752]: 27 slow requests, 1 included below; oldest blocked for > 18732.268170 secs 2024-04-26T23:47:32.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:31 smithi119 ceph-mon[32752]: slow request 183.772279 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:47:32.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:31 smithi119 ceph-mon[32752]: pgmap v10647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:32.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18733.046423 secs 2024-04-26T23:47:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:31 smithi167 ceph-mon[32751]: 27 slow requests, 1 included below; oldest blocked for > 18732.268170 secs 2024-04-26T23:47:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:31 smithi167 ceph-mon[32751]: slow request 183.772279 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:47:32.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:31 smithi167 ceph-mon[32751]: pgmap v10647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:32.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18733.046423 secs 2024-04-26T23:47:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:31 smithi090 ceph-mon[29180]: 27 slow requests, 1 included below; oldest blocked for > 18732.268170 secs 2024-04-26T23:47:32.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:31 smithi090 ceph-mon[29180]: slow request 183.772279 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:47:32.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:31 smithi090 ceph-mon[29180]: pgmap v10647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:32.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18733.046423 secs 2024-04-26T23:47:33.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:33.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:33.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:47:34.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:33 smithi119 ceph-mon[32752]: pgmap v10648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:34.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:34.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:34.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:47:34.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:33 smithi167 ceph-mon[32751]: pgmap v10648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:34.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:34.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:34.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:47:34.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:33 smithi090 ceph-mon[29180]: pgmap v10648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:34.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:47:34.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:47:34.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:47:35.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:34 smithi119 ceph-mon[32752]: pgmap v10649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:47:35.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:34 smithi119 ceph-mon[32752]: pgmap v10650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:34 smithi167 ceph-mon[32751]: pgmap v10649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:47:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:34 smithi167 ceph-mon[32751]: pgmap v10650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:35.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:34 smithi090 ceph-mon[29180]: pgmap v10649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:47:35.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:34 smithi090 ceph-mon[29180]: pgmap v10650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:36.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:47:36.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:35 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:47:36.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:47:36.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:35 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:47:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:47:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:35 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:47:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18737.268253 secs 2024-04-26T23:47:37.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:36 smithi167 ceph-mon[32751]: pgmap v10651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:37.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18738.046537 secs 2024-04-26T23:47:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18737.268253 secs 2024-04-26T23:47:37.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:36 smithi090 ceph-mon[29180]: pgmap v10651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:37.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18738.046537 secs 2024-04-26T23:47:37.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18737.268253 secs 2024-04-26T23:47:37.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:36 smithi119 ceph-mon[32752]: pgmap v10651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:37.209 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18738.046537 secs 2024-04-26T23:47:39.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:38 smithi167 ceph-mon[32751]: pgmap v10652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:47:39.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:38 smithi090 ceph-mon[29180]: pgmap v10652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:47:39.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:38 smithi119 ceph-mon[32752]: pgmap v10652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:47:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:40 smithi090 ceph-mon[29180]: pgmap v10653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:41.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:40 smithi119 ceph-mon[32752]: pgmap v10653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:41.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:40 smithi167 ceph-mon[32751]: pgmap v10653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18742.268337 secs 2024-04-26T23:47:42.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18743.046601 secs 2024-04-26T23:47:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18742.268337 secs 2024-04-26T23:47:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18743.046601 secs 2024-04-26T23:47:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18742.268337 secs 2024-04-26T23:47:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18743.046601 secs 2024-04-26T23:47:43.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:42 smithi090 ceph-mon[29180]: pgmap v10654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:42 smithi119 ceph-mon[32752]: pgmap v10654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:43.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:42 smithi167 ceph-mon[32751]: pgmap v10654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:47:45.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:44 smithi090 ceph-mon[29180]: pgmap v10655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:47:45.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:44 smithi119 ceph-mon[32752]: pgmap v10655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:47:45.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:44 smithi167 ceph-mon[32751]: pgmap v10655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:47:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:46 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18747.268419 secs 2024-04-26T23:47:47.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:46 smithi090 ceph-mon[29180]: pgmap v10656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:47.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18748.046726 secs 2024-04-26T23:47:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:46 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18747.268419 secs 2024-04-26T23:47:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:46 smithi119 ceph-mon[32752]: pgmap v10656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18748.046726 secs 2024-04-26T23:47:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:46 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18747.268419 secs 2024-04-26T23:47:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:46 smithi167 ceph-mon[32751]: pgmap v10656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18748.046726 secs 2024-04-26T23:47:49.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:48 smithi090 ceph-mon[29180]: pgmap v10657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:48 smithi119 ceph-mon[32752]: pgmap v10657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:49.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:48 smithi167 ceph-mon[32751]: pgmap v10657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:51.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:50 smithi090 ceph-mon[29180]: pgmap v10658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:50 smithi119 ceph-mon[32752]: pgmap v10658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:50 smithi167 ceph-mon[32751]: pgmap v10658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:51 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18752.268503 secs 2024-04-26T23:47:52.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18753.046788 secs 2024-04-26T23:47:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:51 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18752.268503 secs 2024-04-26T23:47:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18753.046788 secs 2024-04-26T23:47:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:51 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18752.268503 secs 2024-04-26T23:47:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18753.046788 secs 2024-04-26T23:47:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:52 smithi090 ceph-mon[29180]: pgmap v10659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:52 smithi119 ceph-mon[32752]: pgmap v10659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:53.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:52 smithi167 ceph-mon[32751]: pgmap v10659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:55.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:54 smithi090 ceph-mon[29180]: pgmap v10660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:55.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:54 smithi119 ceph-mon[32752]: pgmap v10660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:55.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:54 smithi167 ceph-mon[32751]: pgmap v10660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:47:57.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:56 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18757.268563 secs 2024-04-26T23:47:57.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:56 smithi090 ceph-mon[29180]: pgmap v10661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:57.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18758.046862 secs 2024-04-26T23:47:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:56 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18757.268563 secs 2024-04-26T23:47:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:56 smithi119 ceph-mon[32752]: pgmap v10661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18758.046862 secs 2024-04-26T23:47:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:56 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18757.268563 secs 2024-04-26T23:47:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:56 smithi167 ceph-mon[32751]: pgmap v10661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:57.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18758.046862 secs 2024-04-26T23:47:59.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:47:58 smithi090 ceph-mon[29180]: pgmap v10662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:47:58 smithi119 ceph-mon[32752]: pgmap v10662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:47:59.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:47:58 smithi167 ceph-mon[32751]: pgmap v10662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:01.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:00 smithi167 ceph-mon[32751]: pgmap v10663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:01.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:00 smithi090 ceph-mon[29180]: pgmap v10663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:00 smithi119 ceph-mon[32752]: pgmap v10663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:02.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:01 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18762.268646 secs 2024-04-26T23:48:02.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18763.046988 secs 2024-04-26T23:48:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:01 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18762.268646 secs 2024-04-26T23:48:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18763.046988 secs 2024-04-26T23:48:02.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:01 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18762.268646 secs 2024-04-26T23:48:02.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18763.046988 secs 2024-04-26T23:48:03.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:02 smithi090 ceph-mon[29180]: pgmap v10664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:03.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:02 smithi119 ceph-mon[32752]: pgmap v10664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:02 smithi167 ceph-mon[32751]: pgmap v10664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:04 smithi090 ceph-mon[29180]: pgmap v10665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:04 smithi119 ceph-mon[32752]: pgmap v10665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:04 smithi167 ceph-mon[32751]: pgmap v10665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:07.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:06 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18767.268731 secs 2024-04-26T23:48:07.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:06 smithi090 ceph-mon[29180]: pgmap v10666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:07.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18768.047101 secs 2024-04-26T23:48:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:06 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18767.268731 secs 2024-04-26T23:48:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:06 smithi119 ceph-mon[32752]: pgmap v10666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18768.047101 secs 2024-04-26T23:48:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:06 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18767.268731 secs 2024-04-26T23:48:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:06 smithi167 ceph-mon[32751]: pgmap v10666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18768.047101 secs 2024-04-26T23:48:09.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:08 smithi090 ceph-mon[29180]: pgmap v10667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:08 smithi119 ceph-mon[32752]: pgmap v10667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:09.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:08 smithi167 ceph-mon[32751]: pgmap v10667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:11.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:10 smithi090 ceph-mon[29180]: pgmap v10668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:10 smithi119 ceph-mon[32752]: pgmap v10668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:10 smithi167 ceph-mon[32751]: pgmap v10668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:11 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18772.268816 secs 2024-04-26T23:48:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18773.047181 secs 2024-04-26T23:48:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:11 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18772.268816 secs 2024-04-26T23:48:12.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18773.047181 secs 2024-04-26T23:48:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:11 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18772.268816 secs 2024-04-26T23:48:12.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18773.047181 secs 2024-04-26T23:48:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:12 smithi119 ceph-mon[32752]: pgmap v10669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:12 smithi167 ceph-mon[32751]: pgmap v10669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:13.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:12 smithi090 ceph-mon[29180]: pgmap v10669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:14 smithi119 ceph-mon[32752]: pgmap v10670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:15.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:14 smithi167 ceph-mon[32751]: pgmap v10670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:14 smithi090 ceph-mon[29180]: pgmap v10670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:17.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:16 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18777.268901 secs 2024-04-26T23:48:17.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:16 smithi090 ceph-mon[29180]: pgmap v10671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:17.182 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18778.047263 secs 2024-04-26T23:48:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:16 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18777.268901 secs 2024-04-26T23:48:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:16 smithi119 ceph-mon[32752]: pgmap v10671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18778.047263 secs 2024-04-26T23:48:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:16 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18777.268901 secs 2024-04-26T23:48:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:16 smithi167 ceph-mon[32751]: pgmap v10671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18778.047263 secs 2024-04-26T23:48:19.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:18 smithi119 ceph-mon[32752]: pgmap v10672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:19.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:18 smithi167 ceph-mon[32751]: pgmap v10672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:19.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:18 smithi090 ceph-mon[29180]: pgmap v10672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:20 smithi119 ceph-mon[32752]: pgmap v10673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:20 smithi167 ceph-mon[32751]: pgmap v10673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:21.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:20 smithi090 ceph-mon[29180]: pgmap v10673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:21 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18782.268988 secs 2024-04-26T23:48:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18783.047326 secs 2024-04-26T23:48:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:21 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18782.268988 secs 2024-04-26T23:48:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18783.047326 secs 2024-04-26T23:48:22.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:21 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18782.268988 secs 2024-04-26T23:48:22.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18783.047326 secs 2024-04-26T23:48:23.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:22 smithi119 ceph-mon[32752]: pgmap v10674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:22 smithi167 ceph-mon[32751]: pgmap v10674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:23.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:22 smithi090 ceph-mon[29180]: pgmap v10674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:25.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:24 smithi119 ceph-mon[32752]: pgmap v10675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:25.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:24 smithi167 ceph-mon[32751]: pgmap v10675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:25.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:24 smithi090 ceph-mon[29180]: pgmap v10675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:26 smithi119 ceph-mon[32752]: 27 slow requests, 1 included below; oldest blocked for > 18787.269072 secs 2024-04-26T23:48:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:26 smithi119 ceph-mon[32752]: slow request 5762.235178 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:48:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:26 smithi119 ceph-mon[32752]: pgmap v10676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18788.047448 secs 2024-04-26T23:48:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:26 smithi167 ceph-mon[32751]: 27 slow requests, 1 included below; oldest blocked for > 18787.269072 secs 2024-04-26T23:48:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:26 smithi167 ceph-mon[32751]: slow request 5762.235178 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:48:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:26 smithi167 ceph-mon[32751]: pgmap v10676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18788.047448 secs 2024-04-26T23:48:27.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:26 smithi090 ceph-mon[29180]: 27 slow requests, 1 included below; oldest blocked for > 18787.269072 secs 2024-04-26T23:48:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:26 smithi090 ceph-mon[29180]: slow request 5762.235178 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:48:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:26 smithi090 ceph-mon[29180]: pgmap v10676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18788.047448 secs 2024-04-26T23:48:29.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:28 smithi119 ceph-mon[32752]: pgmap v10677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:29.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:28 smithi167 ceph-mon[32751]: pgmap v10677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:29.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:28 smithi090 ceph-mon[29180]: pgmap v10677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:31.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:30 smithi119 ceph-mon[32752]: pgmap v10678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:31.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:30 smithi167 ceph-mon[32751]: pgmap v10678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:31.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:30 smithi090 ceph-mon[29180]: pgmap v10678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18792.269151 secs 2024-04-26T23:48:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18793.047512 secs 2024-04-26T23:48:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18792.269151 secs 2024-04-26T23:48:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18793.047512 secs 2024-04-26T23:48:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18792.269151 secs 2024-04-26T23:48:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18793.047512 secs 2024-04-26T23:48:33.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:32 smithi090 ceph-mon[29180]: pgmap v10679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:32 smithi119 ceph-mon[32752]: pgmap v10679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:33.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:32 smithi167 ceph-mon[32751]: pgmap v10679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:34.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:34.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:48:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:34 smithi119 ceph-mon[32752]: pgmap v10680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:48:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:34 smithi167 ceph-mon[32751]: pgmap v10680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:35.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:35.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:48:35.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:34 smithi090 ceph-mon[29180]: pgmap v10680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:35.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:48:35.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:48:35.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:48:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:35 smithi119 ceph-mon[32752]: pgmap v10681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:35 smithi167 ceph-mon[32751]: pgmap v10681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:36.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:35 smithi090 ceph-mon[29180]: pgmap v10681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18797.269208 secs 2024-04-26T23:48:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18798.047627 secs 2024-04-26T23:48:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18797.269208 secs 2024-04-26T23:48:37.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18798.047627 secs 2024-04-26T23:48:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18797.269208 secs 2024-04-26T23:48:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18798.047627 secs 2024-04-26T23:48:38.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:37 smithi119 ceph-mon[32752]: pgmap v10682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:37 smithi167 ceph-mon[32751]: pgmap v10682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:38.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:37 smithi090 ceph-mon[29180]: pgmap v10682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:40.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:39 smithi119 ceph-mon[32752]: pgmap v10683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:40.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:39 smithi167 ceph-mon[32751]: pgmap v10683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:39 smithi090 ceph-mon[29180]: pgmap v10683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18802.269266 secs 2024-04-26T23:48:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:41 smithi119 ceph-mon[32752]: pgmap v10684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18803.047749 secs 2024-04-26T23:48:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18802.269266 secs 2024-04-26T23:48:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:41 smithi167 ceph-mon[32751]: pgmap v10684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18803.047749 secs 2024-04-26T23:48:42.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18802.269266 secs 2024-04-26T23:48:42.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:41 smithi090 ceph-mon[29180]: pgmap v10684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:42.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18803.047749 secs 2024-04-26T23:48:44.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:43 smithi119 ceph-mon[32752]: pgmap v10685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:43 smithi167 ceph-mon[32751]: pgmap v10685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:44.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:43 smithi090 ceph-mon[29180]: pgmap v10685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:46.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:45 smithi119 ceph-mon[32752]: pgmap v10686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:45 smithi167 ceph-mon[32751]: pgmap v10686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:46.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:45 smithi090 ceph-mon[29180]: pgmap v10686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:48:47.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:46 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18807.269327 secs 2024-04-26T23:48:47.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18808.047811 secs 2024-04-26T23:48:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:46 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18807.269327 secs 2024-04-26T23:48:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18808.047811 secs 2024-04-26T23:48:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:46 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18807.269327 secs 2024-04-26T23:48:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18808.047811 secs 2024-04-26T23:48:48.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:47 smithi119 ceph-mon[32752]: pgmap v10687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:48.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:47 smithi167 ceph-mon[32751]: pgmap v10687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:48.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:47 smithi090 ceph-mon[29180]: pgmap v10687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:49 smithi119 ceph-mon[32752]: pgmap v10688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:50.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:49 smithi167 ceph-mon[32751]: pgmap v10688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:50.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:49 smithi090 ceph-mon[29180]: pgmap v10688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:50 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18812.269410 secs 2024-04-26T23:48:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:50 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18812.269410 secs 2024-04-26T23:48:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:50 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18812.269410 secs 2024-04-26T23:48:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:51 smithi119 ceph-mon[32752]: pgmap v10689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18813.047888 secs 2024-04-26T23:48:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:51 smithi167 ceph-mon[32751]: pgmap v10689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18813.047888 secs 2024-04-26T23:48:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:51 smithi090 ceph-mon[29180]: pgmap v10689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:52.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18813.047888 secs 2024-04-26T23:48:54.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:53 smithi119 ceph-mon[32752]: pgmap v10690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:54 smithi167 ceph-mon[32751]: pgmap v10690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:54.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:53 smithi090 ceph-mon[29180]: pgmap v10690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:56.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:56 smithi119 ceph-mon[32752]: pgmap v10691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:56.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:56 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18817.269494 secs 2024-04-26T23:48:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:56 smithi167 ceph-mon[32751]: pgmap v10691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:56 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18817.269494 secs 2024-04-26T23:48:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:56 smithi090 ceph-mon[29180]: pgmap v10691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:48:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:56 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18817.269494 secs 2024-04-26T23:48:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18818.048003 secs 2024-04-26T23:48:57.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18818.048003 secs 2024-04-26T23:48:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18818.048003 secs 2024-04-26T23:48:58.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:48:58 smithi119 ceph-mon[32752]: pgmap v10692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:58.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:48:58 smithi167 ceph-mon[32751]: pgmap v10692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:48:58.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:48:58 smithi090 ceph-mon[29180]: pgmap v10692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:00.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:00 smithi119 ceph-mon[32752]: pgmap v10693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:00.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:00 smithi167 ceph-mon[32751]: pgmap v10693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:00.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:00 smithi090 ceph-mon[29180]: pgmap v10693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:01.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:01 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18822.269579 secs 2024-04-26T23:49:01.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:01 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18822.269579 secs 2024-04-26T23:49:01.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:01 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18822.269579 secs 2024-04-26T23:49:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:02 smithi119 ceph-mon[32752]: pgmap v10694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18823.048070 secs 2024-04-26T23:49:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:02 smithi167 ceph-mon[32751]: pgmap v10694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18823.048070 secs 2024-04-26T23:49:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:02 smithi090 ceph-mon[29180]: pgmap v10694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18823.048070 secs 2024-04-26T23:49:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:03 smithi119 ceph-mon[32752]: pgmap v10695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:03 smithi167 ceph-mon[32751]: pgmap v10695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:03.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:03 smithi090 ceph-mon[29180]: pgmap v10695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:05.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:05 smithi119 ceph-mon[32752]: pgmap v10696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:05.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:05 smithi167 ceph-mon[32751]: pgmap v10696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:05.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:05 smithi090 ceph-mon[29180]: pgmap v10696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:06.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:06 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18827.269667 secs 2024-04-26T23:49:06.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18828.048207 secs 2024-04-26T23:49:06.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:06 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18827.269667 secs 2024-04-26T23:49:06.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18828.048207 secs 2024-04-26T23:49:06.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:06 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18827.269667 secs 2024-04-26T23:49:06.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18828.048207 secs 2024-04-26T23:49:07.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:07 smithi167 ceph-mon[32751]: pgmap v10697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:07.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:07 smithi090 ceph-mon[29180]: pgmap v10697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:07.709 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:07 smithi119 ceph-mon[32752]: pgmap v10697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:09.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:09 smithi167 ceph-mon[32751]: pgmap v10698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:09.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:09 smithi090 ceph-mon[29180]: pgmap v10698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:09.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:09 smithi119 ceph-mon[32752]: pgmap v10698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:11 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18832.269749 secs 2024-04-26T23:49:11.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:11 smithi167 ceph-mon[32751]: pgmap v10699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:11.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18833.048340 secs 2024-04-26T23:49:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:11 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18832.269749 secs 2024-04-26T23:49:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:11 smithi090 ceph-mon[29180]: pgmap v10699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18833.048340 secs 2024-04-26T23:49:11.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:11 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18832.269749 secs 2024-04-26T23:49:11.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:11 smithi119 ceph-mon[32752]: pgmap v10699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:11.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18833.048340 secs 2024-04-26T23:49:13.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:13 smithi167 ceph-mon[32751]: pgmap v10700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:13.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:13 smithi090 ceph-mon[29180]: pgmap v10700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:13.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:13 smithi119 ceph-mon[32752]: pgmap v10700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:15.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:15 smithi167 ceph-mon[32751]: pgmap v10701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:15.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:15 smithi090 ceph-mon[29180]: pgmap v10701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:15.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:15 smithi119 ceph-mon[32752]: pgmap v10701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:16 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18837.269846 secs 2024-04-26T23:49:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18838.048478 secs 2024-04-26T23:49:16.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:16 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18837.269846 secs 2024-04-26T23:49:16.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18838.048478 secs 2024-04-26T23:49:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:16 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18837.269846 secs 2024-04-26T23:49:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18838.048478 secs 2024-04-26T23:49:17.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:17 smithi090 ceph-mon[29180]: pgmap v10702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:17.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:17 smithi119 ceph-mon[32752]: pgmap v10702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:17.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:17 smithi167 ceph-mon[32751]: pgmap v10702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:19.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:19 smithi090 ceph-mon[29180]: pgmap v10703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:19.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:19 smithi119 ceph-mon[32752]: pgmap v10703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:19.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:19 smithi167 ceph-mon[32751]: pgmap v10703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:21.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:21 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18842.269932 secs 2024-04-26T23:49:21.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:21 smithi090 ceph-mon[29180]: pgmap v10704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:21.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18843.048594 secs 2024-04-26T23:49:21.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:21 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18842.269932 secs 2024-04-26T23:49:21.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:21 smithi119 ceph-mon[32752]: pgmap v10704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:21.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18843.048594 secs 2024-04-26T23:49:21.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:21 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18842.269932 secs 2024-04-26T23:49:21.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:21 smithi167 ceph-mon[32751]: pgmap v10704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:21.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18843.048594 secs 2024-04-26T23:49:23.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:23 smithi090 ceph-mon[29180]: pgmap v10705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:23.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:23 smithi119 ceph-mon[32752]: pgmap v10705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:23 smithi167 ceph-mon[32751]: pgmap v10705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:25.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:25 smithi090 ceph-mon[29180]: pgmap v10706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:25.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:25 smithi119 ceph-mon[32752]: pgmap v10706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:25.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:25 smithi167 ceph-mon[32751]: pgmap v10706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:26.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:26 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18847.270013 secs 2024-04-26T23:49:26.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18848.048708 secs 2024-04-26T23:49:26.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:26 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18847.270013 secs 2024-04-26T23:49:26.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18848.048708 secs 2024-04-26T23:49:26.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:26 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18847.270013 secs 2024-04-26T23:49:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18848.048708 secs 2024-04-26T23:49:28.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:27 smithi119 ceph-mon[32752]: pgmap v10707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:28.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:27 smithi167 ceph-mon[32751]: pgmap v10707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:28.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:27 smithi090 ceph-mon[29180]: pgmap v10707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:30.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:29 smithi119 ceph-mon[32752]: pgmap v10708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:30.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:29 smithi167 ceph-mon[32751]: pgmap v10708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:30.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:29 smithi090 ceph-mon[29180]: pgmap v10708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18852.270074 secs 2024-04-26T23:49:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:31 smithi119 ceph-mon[32752]: pgmap v10709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18853.048825 secs 2024-04-26T23:49:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18852.270074 secs 2024-04-26T23:49:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:31 smithi167 ceph-mon[32751]: pgmap v10709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18853.048825 secs 2024-04-26T23:49:32.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18852.270074 secs 2024-04-26T23:49:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:31 smithi090 ceph-mon[29180]: pgmap v10709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18853.048825 secs 2024-04-26T23:49:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:33 smithi119 ceph-mon[32752]: pgmap v10710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:34.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:33 smithi167 ceph-mon[32751]: pgmap v10710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:34.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:33 smithi090 ceph-mon[29180]: pgmap v10710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:35.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:35.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:49:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:35.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:35 smithi119 ceph-mon[32752]: pgmap v10711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:35 smithi167 ceph-mon[32751]: pgmap v10711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:36.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:35 smithi090 ceph-mon[29180]: pgmap v10711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18857.270137 secs 2024-04-26T23:49:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18858.048978 secs 2024-04-26T23:49:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18857.270137 secs 2024-04-26T23:49:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18858.048978 secs 2024-04-26T23:49:37.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18857.270137 secs 2024-04-26T23:49:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18858.048978 secs 2024-04-26T23:49:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: pgmap v10712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:38.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: pgmap v10712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:38.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: pgmap v10712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:38.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:38.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:38.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:49:39.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:49:39.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:49:39.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:49:39.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:49:39.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:49:39.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:49:39.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:49:39.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:49:39.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:49:40.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:39 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:49:40.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:39 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:49:40.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:39 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:49:40.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:39 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:49:40.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:39 smithi119 ceph-mon[32752]: pgmap v10713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:40.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:39 smithi119 ceph-mon[32752]: pgmap v10714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:49:40.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:39 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:49:40.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:39 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:49:40.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:39 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:49:40.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:39 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:49:40.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:39 smithi167 ceph-mon[32751]: pgmap v10713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:40.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:39 smithi167 ceph-mon[32751]: pgmap v10714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:49:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:39 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:49:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:39 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:49:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:39 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:49:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:39 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:49:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:39 smithi090 ceph-mon[29180]: pgmap v10713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:39 smithi090 ceph-mon[29180]: pgmap v10714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:49:41.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:40 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18862.270196 secs 2024-04-26T23:49:41.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:40 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18862.270196 secs 2024-04-26T23:49:41.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:40 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18862.270196 secs 2024-04-26T23:49:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:41 smithi119 ceph-mon[32752]: pgmap v10715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18863.049086 secs 2024-04-26T23:49:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:41 smithi167 ceph-mon[32751]: pgmap v10715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18863.049086 secs 2024-04-26T23:49:42.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:41 smithi090 ceph-mon[29180]: pgmap v10715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:42.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18863.049086 secs 2024-04-26T23:49:44.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:43 smithi119 ceph-mon[32752]: pgmap v10716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:43 smithi167 ceph-mon[32751]: pgmap v10716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:44.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:43 smithi090 ceph-mon[29180]: pgmap v10716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:46.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:45 smithi119 ceph-mon[32752]: pgmap v10717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:46.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:45 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18867.270257 secs 2024-04-26T23:49:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:45 smithi167 ceph-mon[32751]: pgmap v10717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:46.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:45 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18867.270257 secs 2024-04-26T23:49:46.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:45 smithi090 ceph-mon[29180]: pgmap v10717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:46.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:45 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18867.270257 secs 2024-04-26T23:49:47.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:47 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18868.049175 secs 2024-04-26T23:49:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:47 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18868.049175 secs 2024-04-26T23:49:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:47 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18868.049175 secs 2024-04-26T23:49:48.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:48 smithi119 ceph-mon[32752]: pgmap v10718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:48.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:48 smithi167 ceph-mon[32751]: pgmap v10718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:48.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:48 smithi090 ceph-mon[29180]: pgmap v10718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:50 smithi119 ceph-mon[32752]: pgmap v10719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:50.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:50 smithi167 ceph-mon[32751]: pgmap v10719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:50.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:50 smithi090 ceph-mon[29180]: pgmap v10719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T23:49:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:51 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18872.270317 secs 2024-04-26T23:49:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:51 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18872.270317 secs 2024-04-26T23:49:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:51 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18872.270317 secs 2024-04-26T23:49:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:52 smithi119 ceph-mon[32752]: pgmap v10720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:49:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18873.049292 secs 2024-04-26T23:49:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:52 smithi167 ceph-mon[32751]: pgmap v10720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:49:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18873.049292 secs 2024-04-26T23:49:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:52 smithi090 ceph-mon[29180]: pgmap v10720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T23:49:52.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18873.049292 secs 2024-04-26T23:49:54.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:54 smithi119 ceph-mon[32752]: pgmap v10721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:54.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:54 smithi167 ceph-mon[32751]: pgmap v10721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:54.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:54 smithi090 ceph-mon[29180]: pgmap v10721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:56.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:56 smithi119 ceph-mon[32752]: pgmap v10722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:56.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:56 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18877.270401 secs 2024-04-26T23:49:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:56 smithi167 ceph-mon[32751]: pgmap v10722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:56.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:56 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18877.270401 secs 2024-04-26T23:49:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:56 smithi090 ceph-mon[29180]: pgmap v10722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:49:56.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:56 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18877.270401 secs 2024-04-26T23:49:57.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18878.049373 secs 2024-04-26T23:49:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18878.049373 secs 2024-04-26T23:49:57.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18878.049373 secs 2024-04-26T23:49:58.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:58 smithi119 ceph-mon[32752]: pgmap v10723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:58.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:58 smithi167 ceph-mon[32751]: pgmap v10723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:58.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:58 smithi090 ceph-mon[29180]: pgmap v10723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:59.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:49:59 smithi090 ceph-mon[29180]: pgmap v10724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:59.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:49:59 smithi119 ceph-mon[32752]: pgmap v10724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:49:59.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:49:59 smithi167 ceph-mon[32751]: pgmap v10724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:00.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:50:00.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:50:00.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-26T23:50:01.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:01 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18882.270480 secs 2024-04-26T23:50:01.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:01 smithi090 ceph-mon[29180]: pgmap v10725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:01.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:01 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18882.270480 secs 2024-04-26T23:50:01.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:01 smithi119 ceph-mon[32752]: pgmap v10725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:01.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:01 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18882.270480 secs 2024-04-26T23:50:01.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:01 smithi167 ceph-mon[32751]: pgmap v10725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18883.049468 secs 2024-04-26T23:50:02.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18883.049468 secs 2024-04-26T23:50:02.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18883.049468 secs 2024-04-26T23:50:03.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:03 smithi090 ceph-mon[29180]: pgmap v10726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:03.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:03 smithi119 ceph-mon[32752]: pgmap v10726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:03.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:03 smithi167 ceph-mon[32751]: pgmap v10726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:05.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:05 smithi167 ceph-mon[32751]: pgmap v10727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:05.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:05 smithi090 ceph-mon[29180]: pgmap v10727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:05.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:05 smithi119 ceph-mon[32752]: pgmap v10727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:06.325 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:50:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:50:05.985+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:50:06.325 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:50:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:50:05.985+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:50:06.325 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:50:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:50:05.985+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:50:06.325 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:50:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:50:05.985+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:50:06.325 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:50:05 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:50:05.985+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:50:06.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:06 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 18887.270563 secs 2024-04-26T23:50:06.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18888.049586 secs 2024-04-26T23:50:06.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:06 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 18887.270563 secs 2024-04-26T23:50:06.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18888.049586 secs 2024-04-26T23:50:06.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:06 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 18887.270563 secs 2024-04-26T23:50:06.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18888.049586 secs 2024-04-26T23:50:07.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:07 smithi090 ceph-mon[29180]: pgmap v10728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:07.709 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:07 smithi119 ceph-mon[32752]: pgmap v10728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:07.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:07 smithi167 ceph-mon[32751]: pgmap v10728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:09.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:09 smithi090 ceph-mon[29180]: pgmap v10729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T23:50:09.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:09 smithi119 ceph-mon[32752]: pgmap v10729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T23:50:09.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:09 smithi167 ceph-mon[32751]: pgmap v10729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T23:50:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18892.270649 secs 2024-04-26T23:50:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:11 smithi090 ceph-mon[29180]: pgmap v10730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18893.049655 secs 2024-04-26T23:50:11.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18892.270649 secs 2024-04-26T23:50:11.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:11 smithi119 ceph-mon[32752]: pgmap v10730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:11.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18893.049655 secs 2024-04-26T23:50:11.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18892.270649 secs 2024-04-26T23:50:11.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:11 smithi167 ceph-mon[32751]: pgmap v10730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:11.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18893.049655 secs 2024-04-26T23:50:13.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:13 smithi090 ceph-mon[29180]: pgmap v10731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:13.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:13 smithi119 ceph-mon[32752]: pgmap v10731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:13.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:13 smithi167 ceph-mon[32751]: pgmap v10731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:15.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:15 smithi090 ceph-mon[29180]: pgmap v10732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:15.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:50:15.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:50:15.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:15 smithi119 ceph-mon[32752]: pgmap v10732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:15.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:50:15.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:50:15.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:15 smithi167 ceph-mon[32751]: pgmap v10732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:15.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:50:15.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:50:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18897.270728 secs 2024-04-26T23:50:16.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18898.049779 secs 2024-04-26T23:50:16.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18897.270728 secs 2024-04-26T23:50:16.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18898.049779 secs 2024-04-26T23:50:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18897.270728 secs 2024-04-26T23:50:16.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18898.049779 secs 2024-04-26T23:50:17.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:17 smithi090 ceph-mon[29180]: pgmap v10733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:17.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:17 smithi119 ceph-mon[32752]: pgmap v10733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:17.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:17 smithi167 ceph-mon[32751]: pgmap v10733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:19.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:19 smithi090 ceph-mon[29180]: pgmap v10734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:19.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:19 smithi119 ceph-mon[32752]: pgmap v10734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:19.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:19 smithi167 ceph-mon[32751]: pgmap v10734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-26T23:50:21.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18902.270813 secs 2024-04-26T23:50:21.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:21 smithi090 ceph-mon[29180]: pgmap v10735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 3 op/s 2024-04-26T23:50:21.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18903.049916 secs 2024-04-26T23:50:21.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18902.270813 secs 2024-04-26T23:50:21.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:21 smithi119 ceph-mon[32752]: pgmap v10735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 3 op/s 2024-04-26T23:50:21.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18903.049916 secs 2024-04-26T23:50:21.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18902.270813 secs 2024-04-26T23:50:21.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:21 smithi167 ceph-mon[32751]: pgmap v10735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 170 B/s wr, 3 op/s 2024-04-26T23:50:21.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18903.049916 secs 2024-04-26T23:50:23.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:23 smithi090 ceph-mon[29180]: pgmap v10736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:23.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:23 smithi119 ceph-mon[32752]: pgmap v10736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:23.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:23 smithi167 ceph-mon[32751]: pgmap v10736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:25.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:25 smithi090 ceph-mon[29180]: pgmap v10737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:25 smithi119 ceph-mon[32752]: pgmap v10737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:25.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:25 smithi167 ceph-mon[32751]: pgmap v10737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:26.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18907.270897 secs 2024-04-26T23:50:26.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18908.050011 secs 2024-04-26T23:50:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18907.270897 secs 2024-04-26T23:50:26.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18908.050011 secs 2024-04-26T23:50:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18907.270897 secs 2024-04-26T23:50:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18908.050011 secs 2024-04-26T23:50:27.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:27 smithi090 ceph-mon[29180]: pgmap v10738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:27.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:27 smithi119 ceph-mon[32752]: pgmap v10738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:27.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:27 smithi167 ceph-mon[32751]: pgmap v10738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:29.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:29 smithi090 ceph-mon[29180]: pgmap v10739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:29.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:29 smithi119 ceph-mon[32752]: pgmap v10739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:29.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:29 smithi167 ceph-mon[32751]: pgmap v10739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:31 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 18912.270957 secs 2024-04-26T23:50:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:31 smithi090 ceph-mon[29180]: slow request 363.775066 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:50:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:31 smithi090 ceph-mon[29180]: pgmap v10740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18913.050129 secs 2024-04-26T23:50:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:31 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 18912.270957 secs 2024-04-26T23:50:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:31 smithi119 ceph-mon[32752]: slow request 363.775066 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:50:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:31 smithi119 ceph-mon[32752]: pgmap v10740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18913.050129 secs 2024-04-26T23:50:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:31 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 18912.270957 secs 2024-04-26T23:50:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:31 smithi167 ceph-mon[32751]: slow request 363.775066 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:50:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:31 smithi167 ceph-mon[32751]: pgmap v10740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:31.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18913.050129 secs 2024-04-26T23:50:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:33 smithi090 ceph-mon[29180]: pgmap v10741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:33.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:33 smithi119 ceph-mon[32752]: pgmap v10741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:33.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:33 smithi167 ceph-mon[32751]: pgmap v10741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:35.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:35 smithi119 ceph-mon[32752]: pgmap v10742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:35 smithi167 ceph-mon[32751]: pgmap v10742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:35.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:35 smithi090 ceph-mon[29180]: pgmap v10742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:36.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18917.271015 secs 2024-04-26T23:50:36.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18918.050240 secs 2024-04-26T23:50:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18917.271015 secs 2024-04-26T23:50:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18918.050240 secs 2024-04-26T23:50:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18917.271015 secs 2024-04-26T23:50:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18918.050240 secs 2024-04-26T23:50:37.709 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:37 smithi119 ceph-mon[32752]: pgmap v10743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:37.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:37 smithi167 ceph-mon[32751]: pgmap v10743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:37 smithi090 ceph-mon[29180]: pgmap v10743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:39.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:39 smithi119 ceph-mon[32752]: pgmap v10744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:39.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:39.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:39.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:39.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:39 smithi167 ceph-mon[32751]: pgmap v10744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:39.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:39.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:39.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:39.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:39 smithi090 ceph-mon[29180]: pgmap v10744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:39.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:50:39.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:39.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:40.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:40.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:40.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:40 smithi119 ceph-mon[32752]: pgmap v10745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:50:40.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:40 smithi119 ceph-mon[32752]: pgmap v10746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:40.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:40.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:40.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:40.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:40 smithi167 ceph-mon[32751]: pgmap v10745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:50:40.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:40 smithi167 ceph-mon[32751]: pgmap v10746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:40.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:40.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:50:40.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:50:40.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:40 smithi090 ceph-mon[29180]: pgmap v10745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:50:40.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:40 smithi090 ceph-mon[29180]: pgmap v10746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:40.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:50:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18922.271098 secs 2024-04-26T23:50:41.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18923.050360 secs 2024-04-26T23:50:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18922.271098 secs 2024-04-26T23:50:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18923.050360 secs 2024-04-26T23:50:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18922.271098 secs 2024-04-26T23:50:41.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18923.050360 secs 2024-04-26T23:50:42.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:42 smithi119 ceph-mon[32752]: pgmap v10747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:42.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:42 smithi167 ceph-mon[32751]: pgmap v10747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:42 smithi090 ceph-mon[29180]: pgmap v10747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:45.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:44 smithi119 ceph-mon[32752]: pgmap v10748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:45.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:44 smithi167 ceph-mon[32751]: pgmap v10748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:45.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:44 smithi090 ceph-mon[29180]: pgmap v10748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18927.271181 secs 2024-04-26T23:50:47.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:46 smithi119 ceph-mon[32752]: pgmap v10749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:47.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18928.050489 secs 2024-04-26T23:50:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18927.271181 secs 2024-04-26T23:50:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:46 smithi167 ceph-mon[32751]: pgmap v10749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18928.050489 secs 2024-04-26T23:50:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18927.271181 secs 2024-04-26T23:50:47.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:46 smithi090 ceph-mon[29180]: pgmap v10749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:47.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18928.050489 secs 2024-04-26T23:50:48.931 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:48 smithi090 ceph-mon[29180]: pgmap v10750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:49.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:48 smithi119 ceph-mon[32752]: pgmap v10750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:49.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:48 smithi167 ceph-mon[32751]: pgmap v10750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:50:51.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:50 smithi119 ceph-mon[32752]: pgmap v10751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:50:51.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:50 smithi167 ceph-mon[32751]: pgmap v10751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:50:51.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:50 smithi090 ceph-mon[29180]: pgmap v10751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:50:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18932.271241 secs 2024-04-26T23:50:52.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18933.050588 secs 2024-04-26T23:50:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18932.271241 secs 2024-04-26T23:50:52.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18933.050588 secs 2024-04-26T23:50:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18932.271241 secs 2024-04-26T23:50:52.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18933.050588 secs 2024-04-26T23:50:53.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:52 smithi090 ceph-mon[29180]: pgmap v10752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:53.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:52 smithi119 ceph-mon[32752]: pgmap v10752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:53.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:52 smithi167 ceph-mon[32751]: pgmap v10752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:55.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:54 smithi119 ceph-mon[32752]: pgmap v10753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:55.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:54 smithi167 ceph-mon[32751]: pgmap v10753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:55.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:54 smithi090 ceph-mon[29180]: pgmap v10753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:50:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18937.271326 secs 2024-04-26T23:50:57.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:56 smithi119 ceph-mon[32752]: pgmap v10754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:57.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18938.050708 secs 2024-04-26T23:50:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18937.271326 secs 2024-04-26T23:50:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:56 smithi167 ceph-mon[32751]: pgmap v10754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18938.050708 secs 2024-04-26T23:50:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18937.271326 secs 2024-04-26T23:50:57.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:56 smithi090 ceph-mon[29180]: pgmap v10754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:57.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18938.050708 secs 2024-04-26T23:50:59.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:50:58 smithi119 ceph-mon[32752]: pgmap v10755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:59.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:50:58 smithi167 ceph-mon[32751]: pgmap v10755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:50:59.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:50:58 smithi090 ceph-mon[29180]: pgmap v10755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:01.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:00 smithi119 ceph-mon[32752]: pgmap v10756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:01.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:00 smithi167 ceph-mon[32751]: pgmap v10756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:01.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:00 smithi090 ceph-mon[29180]: pgmap v10756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18942.271409 secs 2024-04-26T23:51:02.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18943.050783 secs 2024-04-26T23:51:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18942.271409 secs 2024-04-26T23:51:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18943.050783 secs 2024-04-26T23:51:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18942.271409 secs 2024-04-26T23:51:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18943.050783 secs 2024-04-26T23:51:03.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:02 smithi119 ceph-mon[32752]: pgmap v10757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:03.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:02 smithi167 ceph-mon[32751]: pgmap v10757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:03.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:02 smithi090 ceph-mon[29180]: pgmap v10757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:04 smithi119 ceph-mon[32752]: pgmap v10758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:05.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:04 smithi167 ceph-mon[32751]: pgmap v10758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:04 smithi090 ceph-mon[29180]: pgmap v10758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18947.271493 secs 2024-04-26T23:51:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:06 smithi119 ceph-mon[32752]: pgmap v10759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:07.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18948.050901 secs 2024-04-26T23:51:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18947.271493 secs 2024-04-26T23:51:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:06 smithi167 ceph-mon[32751]: pgmap v10759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18948.050901 secs 2024-04-26T23:51:07.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18947.271493 secs 2024-04-26T23:51:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:06 smithi090 ceph-mon[29180]: pgmap v10759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18948.050901 secs 2024-04-26T23:51:08.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:08 smithi090 ceph-mon[29180]: pgmap v10760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:09.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:08 smithi119 ceph-mon[32752]: pgmap v10760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:09.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:08 smithi167 ceph-mon[32751]: pgmap v10760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:10 smithi119 ceph-mon[32752]: pgmap v10761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:10 smithi167 ceph-mon[32751]: pgmap v10761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:10 smithi090 ceph-mon[29180]: pgmap v10761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:11 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 18952.271576 secs 2024-04-26T23:51:12.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:11 smithi119 ceph-mon[32752]: slow request 2884.899409 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:51:12.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18953.050995 secs 2024-04-26T23:51:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:11 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 18952.271576 secs 2024-04-26T23:51:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:11 smithi167 ceph-mon[32751]: slow request 2884.899409 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:51:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18953.050995 secs 2024-04-26T23:51:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:11 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 18952.271576 secs 2024-04-26T23:51:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:11 smithi090 ceph-mon[29180]: slow request 2884.899409 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:51:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18953.050995 secs 2024-04-26T23:51:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:12 smithi119 ceph-mon[32752]: pgmap v10762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:12 smithi167 ceph-mon[32751]: pgmap v10762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:12 smithi090 ceph-mon[29180]: pgmap v10762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-26T23:51:15.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:14 smithi119 ceph-mon[32752]: pgmap v10763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:15.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:14 smithi167 ceph-mon[32751]: pgmap v10763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:15.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:14 smithi090 ceph-mon[29180]: pgmap v10763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T23:51:17.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18957.271661 secs 2024-04-26T23:51:17.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:16 smithi119 ceph-mon[32752]: pgmap v10764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:17.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18958.051082 secs 2024-04-26T23:51:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18957.271661 secs 2024-04-26T23:51:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:16 smithi167 ceph-mon[32751]: pgmap v10764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:17.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18958.051082 secs 2024-04-26T23:51:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18957.271661 secs 2024-04-26T23:51:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:16 smithi090 ceph-mon[29180]: pgmap v10764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:17.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18958.051082 secs 2024-04-26T23:51:19.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:18 smithi119 ceph-mon[32752]: pgmap v10765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:19.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:18 smithi167 ceph-mon[32751]: pgmap v10765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:19.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:18 smithi090 ceph-mon[29180]: pgmap v10765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:21.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:20 smithi119 ceph-mon[32752]: pgmap v10766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:21.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:20 smithi167 ceph-mon[32751]: pgmap v10766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:20 smithi090 ceph-mon[29180]: pgmap v10766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18962.271748 secs 2024-04-26T23:51:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18963.051183 secs 2024-04-26T23:51:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18962.271748 secs 2024-04-26T23:51:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18963.051183 secs 2024-04-26T23:51:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18962.271748 secs 2024-04-26T23:51:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18963.051183 secs 2024-04-26T23:51:23.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:22 smithi119 ceph-mon[32752]: pgmap v10767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:23.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:22 smithi167 ceph-mon[32751]: pgmap v10767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:23.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:22 smithi090 ceph-mon[29180]: pgmap v10767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:25.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:24 smithi119 ceph-mon[32752]: pgmap v10768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:25.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:24 smithi167 ceph-mon[32751]: pgmap v10768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:25.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:24 smithi090 ceph-mon[29180]: pgmap v10768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18967.271831 secs 2024-04-26T23:51:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:26 smithi119 ceph-mon[32752]: pgmap v10769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18968.051258 secs 2024-04-26T23:51:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18967.271831 secs 2024-04-26T23:51:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:26 smithi167 ceph-mon[32751]: pgmap v10769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18968.051258 secs 2024-04-26T23:51:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18967.271831 secs 2024-04-26T23:51:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:26 smithi090 ceph-mon[29180]: pgmap v10769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18968.051258 secs 2024-04-26T23:51:29.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:28 smithi119 ceph-mon[32752]: pgmap v10770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:29.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:28 smithi167 ceph-mon[32751]: pgmap v10770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:29.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:28 smithi090 ceph-mon[29180]: pgmap v10770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:31.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:30 smithi119 ceph-mon[32752]: pgmap v10771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:31.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:30 smithi167 ceph-mon[32751]: pgmap v10771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:31.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:30 smithi090 ceph-mon[29180]: pgmap v10771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18972.271913 secs 2024-04-26T23:51:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18973.051357 secs 2024-04-26T23:51:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18972.271913 secs 2024-04-26T23:51:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18973.051357 secs 2024-04-26T23:51:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18972.271913 secs 2024-04-26T23:51:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18973.051357 secs 2024-04-26T23:51:33.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:32 smithi119 ceph-mon[32752]: pgmap v10772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:33.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:32 smithi167 ceph-mon[32751]: pgmap v10772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:33.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:32 smithi090 ceph-mon[29180]: pgmap v10772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:35.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:34 smithi119 ceph-mon[32752]: pgmap v10773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:34 smithi167 ceph-mon[32751]: pgmap v10773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:34 smithi090 ceph-mon[29180]: pgmap v10773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18977.271975 secs 2024-04-26T23:51:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:36 smithi119 ceph-mon[32752]: pgmap v10774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:37.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18978.051472 secs 2024-04-26T23:51:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18977.271975 secs 2024-04-26T23:51:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:36 smithi167 ceph-mon[32751]: pgmap v10774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18978.051472 secs 2024-04-26T23:51:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18977.271975 secs 2024-04-26T23:51:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:36 smithi090 ceph-mon[29180]: pgmap v10774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18978.051472 secs 2024-04-26T23:51:39.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:38 smithi119 ceph-mon[32752]: pgmap v10775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:39.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:38 smithi167 ceph-mon[32751]: pgmap v10775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:39.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:38 smithi090 ceph-mon[29180]: pgmap v10775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:40.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:40.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:40.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:51:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:40 smithi119 ceph-mon[32752]: pgmap v10776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:51:41.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:40 smithi167 ceph-mon[32751]: pgmap v10776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:41.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:41.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:41.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:51:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:40 smithi090 ceph-mon[29180]: pgmap v10776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:51:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:51:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:51:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18982.272037 secs 2024-04-26T23:51:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:41 smithi119 ceph-mon[32752]: pgmap v10777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:51:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:41 smithi119 ceph-mon[32752]: pgmap v10778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:42.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18983.051578 secs 2024-04-26T23:51:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18982.272037 secs 2024-04-26T23:51:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:41 smithi167 ceph-mon[32751]: pgmap v10777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:51:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:41 smithi167 ceph-mon[32751]: pgmap v10778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18983.051578 secs 2024-04-26T23:51:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18982.272037 secs 2024-04-26T23:51:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:41 smithi090 ceph-mon[29180]: pgmap v10777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:51:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:41 smithi090 ceph-mon[29180]: pgmap v10778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:42.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18983.051578 secs 2024-04-26T23:51:44.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:43 smithi119 ceph-mon[32752]: pgmap v10779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:51:44.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:43 smithi167 ceph-mon[32751]: pgmap v10779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:51:44.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:43 smithi090 ceph-mon[29180]: pgmap v10779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:51:46.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:45 smithi119 ceph-mon[32752]: pgmap v10780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:46.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:45 smithi167 ceph-mon[32751]: pgmap v10780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:46.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:45 smithi090 ceph-mon[29180]: pgmap v10780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18987.272120 secs 2024-04-26T23:51:47.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18988.051700 secs 2024-04-26T23:51:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18987.272120 secs 2024-04-26T23:51:47.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18988.051700 secs 2024-04-26T23:51:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18987.272120 secs 2024-04-26T23:51:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18988.051700 secs 2024-04-26T23:51:48.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:47 smithi119 ceph-mon[32752]: pgmap v10781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:48.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:47 smithi167 ceph-mon[32751]: pgmap v10781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:48.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:47 smithi090 ceph-mon[29180]: pgmap v10781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:50.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:49 smithi119 ceph-mon[32752]: pgmap v10782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:50.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:49 smithi167 ceph-mon[32751]: pgmap v10782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:50.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:49 smithi090 ceph-mon[29180]: pgmap v10782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:51:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18992.272205 secs 2024-04-26T23:51:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:51 smithi119 ceph-mon[32752]: pgmap v10783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:51:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18993.051823 secs 2024-04-26T23:51:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18992.272205 secs 2024-04-26T23:51:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:51 smithi167 ceph-mon[32751]: pgmap v10783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:51:52.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18993.051823 secs 2024-04-26T23:51:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18992.272205 secs 2024-04-26T23:51:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:51 smithi090 ceph-mon[29180]: pgmap v10783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:51:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18993.051823 secs 2024-04-26T23:51:54.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:53 smithi119 ceph-mon[32752]: pgmap v10784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:54.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:53 smithi167 ceph-mon[32751]: pgmap v10784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:53 smithi090 ceph-mon[29180]: pgmap v10784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:55.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:55 smithi090 ceph-mon[29180]: pgmap v10785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:56.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:55 smithi119 ceph-mon[32752]: pgmap v10785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:56.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:55 smithi167 ceph-mon[32751]: pgmap v10785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:51:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 18997.272292 secs 2024-04-26T23:51:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 18998.051906 secs 2024-04-26T23:51:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 18997.272292 secs 2024-04-26T23:51:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 18998.051906 secs 2024-04-26T23:51:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 18997.272292 secs 2024-04-26T23:51:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 18998.051906 secs 2024-04-26T23:51:58.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:57 smithi119 ceph-mon[32752]: pgmap v10786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:58.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:57 smithi167 ceph-mon[32751]: pgmap v10786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:51:58.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:57 smithi090 ceph-mon[29180]: pgmap v10786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:00.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:51:59 smithi119 ceph-mon[32752]: pgmap v10787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:00.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:51:59 smithi167 ceph-mon[32751]: pgmap v10787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:00.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:51:59 smithi090 ceph-mon[29180]: pgmap v10787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19002.272351 secs 2024-04-26T23:52:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:01 smithi167 ceph-mon[32751]: pgmap v10788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19003.051963 secs 2024-04-26T23:52:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19002.272351 secs 2024-04-26T23:52:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:01 smithi090 ceph-mon[29180]: pgmap v10788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:02.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19003.051963 secs 2024-04-26T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19002.272351 secs 2024-04-26T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:01 smithi119 ceph-mon[32752]: pgmap v10788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19003.051963 secs 2024-04-26T23:52:04.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:03 smithi167 ceph-mon[32751]: pgmap v10789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:04.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:03 smithi090 ceph-mon[29180]: pgmap v10789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:04.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:03 smithi119 ceph-mon[32752]: pgmap v10789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:05 smithi167 ceph-mon[32751]: pgmap v10790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:05 smithi090 ceph-mon[29180]: pgmap v10790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:05 smithi119 ceph-mon[32752]: pgmap v10790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19007.272436 secs 2024-04-26T23:52:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19008.052096 secs 2024-04-26T23:52:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19007.272436 secs 2024-04-26T23:52:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19008.052096 secs 2024-04-26T23:52:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19007.272436 secs 2024-04-26T23:52:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19008.052096 secs 2024-04-26T23:52:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:07 smithi090 ceph-mon[29180]: pgmap v10791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:08.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:07 smithi119 ceph-mon[32752]: pgmap v10791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:07 smithi167 ceph-mon[32751]: pgmap v10791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:09 smithi090 ceph-mon[29180]: pgmap v10792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:09 smithi119 ceph-mon[32752]: pgmap v10792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:09 smithi167 ceph-mon[32751]: pgmap v10792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19012.272520 secs 2024-04-26T23:52:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:11 smithi090 ceph-mon[29180]: pgmap v10793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19013.052228 secs 2024-04-26T23:52:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19012.272520 secs 2024-04-26T23:52:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:11 smithi119 ceph-mon[32752]: pgmap v10793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19013.052228 secs 2024-04-26T23:52:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19012.272520 secs 2024-04-26T23:52:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:11 smithi167 ceph-mon[32751]: pgmap v10793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19013.052228 secs 2024-04-26T23:52:14.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:13 smithi090 ceph-mon[29180]: pgmap v10794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:14.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:13 smithi119 ceph-mon[32752]: pgmap v10794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:13 smithi167 ceph-mon[32751]: pgmap v10794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:16.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:15 smithi090 ceph-mon[29180]: pgmap v10795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:15 smithi119 ceph-mon[32752]: pgmap v10795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:15 smithi167 ceph-mon[32751]: pgmap v10795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19017.272602 secs 2024-04-26T23:52:17.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19018.052365 secs 2024-04-26T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19017.272602 secs 2024-04-26T23:52:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19018.052365 secs 2024-04-26T23:52:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19017.272602 secs 2024-04-26T23:52:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19018.052365 secs 2024-04-26T23:52:18.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:17 smithi090 ceph-mon[29180]: pgmap v10796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:18.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:17 smithi119 ceph-mon[32752]: pgmap v10796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:18.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:17 smithi167 ceph-mon[32751]: pgmap v10796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:20.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:19 smithi090 ceph-mon[29180]: pgmap v10797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:19 smithi119 ceph-mon[32752]: pgmap v10797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:19 smithi167 ceph-mon[32751]: pgmap v10797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:21 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 19022.272661 secs 2024-04-26T23:52:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:21 smithi090 ceph-mon[29180]: slow request 11521.331956 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:52:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:21 smithi090 ceph-mon[29180]: pgmap v10798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19023.052475 secs 2024-04-26T23:52:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:21 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 19022.272661 secs 2024-04-26T23:52:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:21 smithi119 ceph-mon[32752]: slow request 11521.331956 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:52:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:21 smithi119 ceph-mon[32752]: pgmap v10798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19023.052475 secs 2024-04-26T23:52:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:21 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 19022.272661 secs 2024-04-26T23:52:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:21 smithi167 ceph-mon[32751]: slow request 11521.331956 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:52:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:21 smithi167 ceph-mon[32751]: pgmap v10798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19023.052475 secs 2024-04-26T23:52:24.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:23 smithi167 ceph-mon[32751]: pgmap v10799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:23 smithi090 ceph-mon[29180]: pgmap v10799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:24.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:23 smithi119 ceph-mon[32752]: pgmap v10799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:26.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:25 smithi090 ceph-mon[29180]: pgmap v10800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:25 smithi119 ceph-mon[32752]: pgmap v10800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:26.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:25 smithi167 ceph-mon[32751]: pgmap v10800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19027.272761 secs 2024-04-26T23:52:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19028.052552 secs 2024-04-26T23:52:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19027.272761 secs 2024-04-26T23:52:27.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19028.052552 secs 2024-04-26T23:52:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19027.272761 secs 2024-04-26T23:52:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19028.052552 secs 2024-04-26T23:52:28.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:27 smithi090 ceph-mon[29180]: pgmap v10801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:28.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:27 smithi119 ceph-mon[32752]: pgmap v10801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:28.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:27 smithi167 ceph-mon[32751]: pgmap v10801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:30.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:29 smithi090 ceph-mon[29180]: pgmap v10802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:30.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:29 smithi119 ceph-mon[32752]: pgmap v10802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:30.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:29 smithi167 ceph-mon[32751]: pgmap v10802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19032.272848 secs 2024-04-26T23:52:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:31 smithi090 ceph-mon[29180]: pgmap v10803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19033.052684 secs 2024-04-26T23:52:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19032.272848 secs 2024-04-26T23:52:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:31 smithi119 ceph-mon[32752]: pgmap v10803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19033.052684 secs 2024-04-26T23:52:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19032.272848 secs 2024-04-26T23:52:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:31 smithi167 ceph-mon[32751]: pgmap v10803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19033.052684 secs 2024-04-26T23:52:34.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:33 smithi090 ceph-mon[29180]: pgmap v10804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:34.161 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:33 smithi167 ceph-mon[32751]: pgmap v10804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:33 smithi119 ceph-mon[32752]: pgmap v10804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:36.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:35 smithi090 ceph-mon[29180]: pgmap v10805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:35 smithi119 ceph-mon[32752]: pgmap v10805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:35 smithi167 ceph-mon[32751]: pgmap v10805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19037.272934 secs 2024-04-26T23:52:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19038.052782 secs 2024-04-26T23:52:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19037.272934 secs 2024-04-26T23:52:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19038.052782 secs 2024-04-26T23:52:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19037.272934 secs 2024-04-26T23:52:37.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19038.052782 secs 2024-04-26T23:52:38.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:37 smithi119 ceph-mon[32752]: pgmap v10806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:37 smithi167 ceph-mon[32751]: pgmap v10806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:38.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:37 smithi090 ceph-mon[29180]: pgmap v10806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:40.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:39 smithi119 ceph-mon[32752]: pgmap v10807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:40.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:39 smithi167 ceph-mon[32751]: pgmap v10807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:40.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:39 smithi090 ceph-mon[29180]: pgmap v10807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:41.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:52:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:52:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:52:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19042.273020 secs 2024-04-26T23:52:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:41 smithi090 ceph-mon[29180]: pgmap v10808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19043.052850 secs 2024-04-26T23:52:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:52:42.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:52:42.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:52:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19042.273020 secs 2024-04-26T23:52:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:41 smithi119 ceph-mon[32752]: pgmap v10808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19043.052850 secs 2024-04-26T23:52:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:52:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:52:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:52:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19042.273020 secs 2024-04-26T23:52:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:41 smithi167 ceph-mon[32751]: pgmap v10808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19043.052850 secs 2024-04-26T23:52:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:52:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:52:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:52:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:42 smithi119 ceph-mon[32752]: pgmap v10809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:52:43.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:42 smithi119 ceph-mon[32752]: pgmap v10810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:43.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:42 smithi167 ceph-mon[32751]: pgmap v10809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:52:43.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:42 smithi167 ceph-mon[32751]: pgmap v10810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:43.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:42 smithi090 ceph-mon[29180]: pgmap v10809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:52:43.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:42 smithi090 ceph-mon[29180]: pgmap v10810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:44 smithi119 ceph-mon[32752]: pgmap v10811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:52:45.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:44 smithi167 ceph-mon[32751]: pgmap v10811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:52:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:44 smithi090 ceph-mon[29180]: pgmap v10811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:52:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19047.273106 secs 2024-04-26T23:52:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:46 smithi119 ceph-mon[32752]: pgmap v10812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19048.052979 secs 2024-04-26T23:52:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19047.273106 secs 2024-04-26T23:52:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:46 smithi167 ceph-mon[32751]: pgmap v10812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19048.052979 secs 2024-04-26T23:52:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19047.273106 secs 2024-04-26T23:52:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:46 smithi090 ceph-mon[29180]: pgmap v10812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19048.052979 secs 2024-04-26T23:52:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:48 smithi119 ceph-mon[32752]: pgmap v10813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:52:49.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:48 smithi167 ceph-mon[32751]: pgmap v10813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:52:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:48 smithi090 ceph-mon[29180]: pgmap v10813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:52:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:50 smithi119 ceph-mon[32752]: pgmap v10814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:51.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:50 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19052.273191 secs 2024-04-26T23:52:51.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:50 smithi167 ceph-mon[32751]: pgmap v10814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:50 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19052.273191 secs 2024-04-26T23:52:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:50 smithi090 ceph-mon[29180]: pgmap v10814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:52:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:50 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19052.273191 secs 2024-04-26T23:52:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19053.053110 secs 2024-04-26T23:52:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19053.053110 secs 2024-04-26T23:52:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19053.053110 secs 2024-04-26T23:52:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:53 smithi119 ceph-mon[32752]: pgmap v10815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:52:53.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:53 smithi167 ceph-mon[32751]: pgmap v10815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:52:53.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:53 smithi090 ceph-mon[29180]: pgmap v10815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:52:55.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:55 smithi119 ceph-mon[32752]: pgmap v10816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:55.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:55 smithi167 ceph-mon[32751]: pgmap v10816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:55.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:55 smithi090 ceph-mon[29180]: pgmap v10816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19057.273257 secs 2024-04-26T23:52:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19057.273257 secs 2024-04-26T23:52:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19057.273257 secs 2024-04-26T23:52:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:57 smithi119 ceph-mon[32752]: pgmap v10817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19058.053244 secs 2024-04-26T23:52:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:57 smithi167 ceph-mon[32751]: pgmap v10817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19058.053244 secs 2024-04-26T23:52:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:57 smithi090 ceph-mon[29180]: pgmap v10817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:52:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19058.053244 secs 2024-04-26T23:52:59.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:52:59 smithi119 ceph-mon[32752]: pgmap v10818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:59.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:52:59 smithi167 ceph-mon[32751]: pgmap v10818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:52:59.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:52:59 smithi090 ceph-mon[29180]: pgmap v10818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:01 smithi119 ceph-mon[32752]: pgmap v10819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19062.273342 secs 2024-04-26T23:53:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:01 smithi167 ceph-mon[32751]: pgmap v10819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19062.273342 secs 2024-04-26T23:53:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:01 smithi090 ceph-mon[29180]: pgmap v10819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19062.273342 secs 2024-04-26T23:53:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19063.053355 secs 2024-04-26T23:53:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19063.053355 secs 2024-04-26T23:53:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19063.053355 secs 2024-04-26T23:53:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:03 smithi119 ceph-mon[32752]: pgmap v10820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:03.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:03 smithi167 ceph-mon[32751]: pgmap v10820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:03 smithi090 ceph-mon[29180]: pgmap v10820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:05 smithi119 ceph-mon[32752]: pgmap v10821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:05 smithi167 ceph-mon[32751]: pgmap v10821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:05.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:05 smithi090 ceph-mon[29180]: pgmap v10821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 19067.273402 secs 2024-04-26T23:53:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 19067.273402 secs 2024-04-26T23:53:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 19067.273402 secs 2024-04-26T23:53:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:07 smithi119 ceph-mon[32752]: pgmap v10822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:07 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 19068.053449 secs 2024-04-26T23:53:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:07 smithi119 ceph-mon[32752]: slow request 1440.193560 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:53:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:07 smithi167 ceph-mon[32751]: pgmap v10822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:07 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 19068.053449 secs 2024-04-26T23:53:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:07 smithi167 ceph-mon[32751]: slow request 1440.193560 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:53:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:07 smithi090 ceph-mon[29180]: pgmap v10822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:07 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 19068.053449 secs 2024-04-26T23:53:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:07 smithi090 ceph-mon[29180]: slow request 1440.193560 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:53:09.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:09 smithi119 ceph-mon[32752]: pgmap v10823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:09 smithi167 ceph-mon[32751]: pgmap v10823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:09 smithi090 ceph-mon[29180]: pgmap v10823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:10 smithi090 ceph-mon[29180]: pgmap v10824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:10.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:10 smithi119 ceph-mon[32752]: pgmap v10824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:10.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:10 smithi167 ceph-mon[32751]: pgmap v10824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:11 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 19072.273462 secs 2024-04-26T23:53:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:11 smithi090 ceph-mon[29180]: slow request 184.035211 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:53:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:11 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 19072.273462 secs 2024-04-26T23:53:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:11 smithi119 ceph-mon[32752]: slow request 184.035211 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:53:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:11 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 19072.273462 secs 2024-04-26T23:53:11.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:11 smithi167 ceph-mon[32751]: slow request 184.035211 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-26T23:53:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:12 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19073.053567 secs 2024-04-26T23:53:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:12 smithi090 ceph-mon[29180]: pgmap v10825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:12 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19073.053567 secs 2024-04-26T23:53:12.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:12 smithi119 ceph-mon[32752]: pgmap v10825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:12.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:12 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19073.053567 secs 2024-04-26T23:53:12.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:12 smithi167 ceph-mon[32751]: pgmap v10825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:14.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:14 smithi119 ceph-mon[32752]: pgmap v10826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:14.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:14 smithi167 ceph-mon[32751]: pgmap v10826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:14 smithi090 ceph-mon[29180]: pgmap v10826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:15.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:53:15.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:53:15.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:53:15.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:53:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-26T23:53:15.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-26T23:53:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19077.273545 secs 2024-04-26T23:53:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:16 smithi119 ceph-mon[32752]: pgmap v10827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19078.053684 secs 2024-04-26T23:53:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19077.273545 secs 2024-04-26T23:53:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:16 smithi167 ceph-mon[32751]: pgmap v10827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:16.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19078.053684 secs 2024-04-26T23:53:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19077.273545 secs 2024-04-26T23:53:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:16 smithi090 ceph-mon[29180]: pgmap v10827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19078.053684 secs 2024-04-26T23:53:18.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:18 smithi119 ceph-mon[32752]: pgmap v10828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:18.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:18 smithi167 ceph-mon[32751]: pgmap v10828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:18.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:18 smithi090 ceph-mon[29180]: pgmap v10828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:20.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:20 smithi119 ceph-mon[32752]: pgmap v10829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:20.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:20 smithi167 ceph-mon[32751]: pgmap v10829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:20.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:20 smithi090 ceph-mon[29180]: pgmap v10829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:21 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19082.273603 secs 2024-04-26T23:53:21.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19083.053808 secs 2024-04-26T23:53:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:21 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19082.273603 secs 2024-04-26T23:53:21.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19083.053808 secs 2024-04-26T23:53:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:21 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19082.273603 secs 2024-04-26T23:53:21.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19083.053808 secs 2024-04-26T23:53:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:22 smithi119 ceph-mon[32752]: pgmap v10830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:22 smithi167 ceph-mon[32751]: pgmap v10830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:22 smithi090 ceph-mon[29180]: pgmap v10830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:24.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:24 smithi119 ceph-mon[32752]: pgmap v10831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:24.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:24 smithi167 ceph-mon[32751]: pgmap v10831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:24.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:24 smithi090 ceph-mon[29180]: pgmap v10831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:26 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19087.273659 secs 2024-04-26T23:53:26.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:26 smithi119 ceph-mon[32752]: pgmap v10832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:26.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19088.053883 secs 2024-04-26T23:53:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:26 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19087.273659 secs 2024-04-26T23:53:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:26 smithi167 ceph-mon[32751]: pgmap v10832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19088.053883 secs 2024-04-26T23:53:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:26 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19087.273659 secs 2024-04-26T23:53:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:26 smithi090 ceph-mon[29180]: pgmap v10832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:26.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19088.053883 secs 2024-04-26T23:53:28.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:28 smithi119 ceph-mon[32752]: pgmap v10833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:28.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:28 smithi167 ceph-mon[32751]: pgmap v10833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:28 smithi090 ceph-mon[29180]: pgmap v10833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:30.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:30 smithi119 ceph-mon[32752]: pgmap v10834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:30 smithi167 ceph-mon[32751]: pgmap v10834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:30 smithi090 ceph-mon[29180]: pgmap v10834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:31 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19092.273745 secs 2024-04-26T23:53:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19093.053994 secs 2024-04-26T23:53:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:31 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19092.273745 secs 2024-04-26T23:53:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19093.053994 secs 2024-04-26T23:53:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:31 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19092.273745 secs 2024-04-26T23:53:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19093.053994 secs 2024-04-26T23:53:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:32 smithi119 ceph-mon[32752]: pgmap v10835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:32 smithi167 ceph-mon[32751]: pgmap v10835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:32 smithi090 ceph-mon[29180]: pgmap v10835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:34.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:34 smithi119 ceph-mon[32752]: pgmap v10836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:34.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:34 smithi167 ceph-mon[32751]: pgmap v10836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:34 smithi090 ceph-mon[29180]: pgmap v10836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:36 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19097.273826 secs 2024-04-26T23:53:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:36 smithi119 ceph-mon[32752]: pgmap v10837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:36.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19098.054118 secs 2024-04-26T23:53:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:36 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19097.273826 secs 2024-04-26T23:53:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:36 smithi167 ceph-mon[32751]: pgmap v10837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19098.054118 secs 2024-04-26T23:53:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:36 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19097.273826 secs 2024-04-26T23:53:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:36 smithi090 ceph-mon[29180]: pgmap v10837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:36.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19098.054118 secs 2024-04-26T23:53:38.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:38 smithi119 ceph-mon[32752]: pgmap v10838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:38.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:38 smithi167 ceph-mon[32751]: pgmap v10838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:38 smithi090 ceph-mon[29180]: pgmap v10838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:40.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:40 smithi119 ceph-mon[32752]: pgmap v10839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:40.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:40 smithi167 ceph-mon[32751]: pgmap v10839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:40.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:40 smithi090 ceph-mon[29180]: pgmap v10839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19102.273907 secs 2024-04-26T23:53:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19103.054240 secs 2024-04-26T23:53:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:53:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19102.273907 secs 2024-04-26T23:53:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19103.054240 secs 2024-04-26T23:53:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:53:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19102.273907 secs 2024-04-26T23:53:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19103.054240 secs 2024-04-26T23:53:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:53:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:42 smithi119 ceph-mon[32752]: pgmap v10840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:42.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:53:42.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:53:42.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:42 smithi119 ceph-mon[32752]: pgmap v10841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:53:42.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:42 smithi119 ceph-mon[32752]: pgmap v10842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:42.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:53:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:42 smithi167 ceph-mon[32751]: pgmap v10840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:42.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:53:42.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:53:42.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:42 smithi167 ceph-mon[32751]: pgmap v10841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:53:42.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:42 smithi167 ceph-mon[32751]: pgmap v10842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:42.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:53:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:42 smithi090 ceph-mon[29180]: pgmap v10840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:53:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:53:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:42 smithi090 ceph-mon[29180]: pgmap v10841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:53:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:42 smithi090 ceph-mon[29180]: pgmap v10842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:42.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:53:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:44 smithi119 ceph-mon[32752]: pgmap v10843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:45.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:44 smithi167 ceph-mon[32751]: pgmap v10843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:44 smithi090 ceph-mon[29180]: pgmap v10843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19107.273992 secs 2024-04-26T23:53:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19108.054373 secs 2024-04-26T23:53:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:46 smithi119 ceph-mon[32752]: pgmap v10844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19107.273992 secs 2024-04-26T23:53:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19108.054373 secs 2024-04-26T23:53:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:46 smithi167 ceph-mon[32751]: pgmap v10844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19107.273992 secs 2024-04-26T23:53:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19108.054373 secs 2024-04-26T23:53:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:46 smithi090 ceph-mon[29180]: pgmap v10844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:48 smithi119 ceph-mon[32752]: pgmap v10845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:53:49.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:48 smithi167 ceph-mon[32751]: pgmap v10845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:53:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:48 smithi090 ceph-mon[29180]: pgmap v10845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:53:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:50 smithi119 ceph-mon[32752]: pgmap v10846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:51.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:50 smithi167 ceph-mon[32751]: pgmap v10846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:51.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:50 smithi090 ceph-mon[29180]: pgmap v10846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:53:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19112.274075 secs 2024-04-26T23:53:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19113.054519 secs 2024-04-26T23:53:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19112.274075 secs 2024-04-26T23:53:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19113.054519 secs 2024-04-26T23:53:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19112.274075 secs 2024-04-26T23:53:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19113.054519 secs 2024-04-26T23:53:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:53 smithi119 ceph-mon[32752]: pgmap v10847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:53:53.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:53 smithi167 ceph-mon[32751]: pgmap v10847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:53:53.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:53 smithi090 ceph-mon[29180]: pgmap v10847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:53:55.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:55 smithi119 ceph-mon[32752]: pgmap v10848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:55.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:55 smithi167 ceph-mon[32751]: pgmap v10848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:55.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:55 smithi090 ceph-mon[29180]: pgmap v10848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:53:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19117.274161 secs 2024-04-26T23:53:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19117.274161 secs 2024-04-26T23:53:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19117.274161 secs 2024-04-26T23:53:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19118.054638 secs 2024-04-26T23:53:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:57 smithi119 ceph-mon[32752]: pgmap v10849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19118.054638 secs 2024-04-26T23:53:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:57 smithi167 ceph-mon[32751]: pgmap v10849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19118.054638 secs 2024-04-26T23:53:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:57 smithi090 ceph-mon[29180]: pgmap v10849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:59.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:53:59 smithi119 ceph-mon[32752]: pgmap v10850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:59.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:53:59 smithi167 ceph-mon[32751]: pgmap v10850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:53:59.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:53:59 smithi090 ceph-mon[29180]: pgmap v10850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:01 smithi119 ceph-mon[32752]: pgmap v10851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19122.274246 secs 2024-04-26T23:54:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:01 smithi167 ceph-mon[32751]: pgmap v10851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19122.274246 secs 2024-04-26T23:54:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:01 smithi090 ceph-mon[29180]: pgmap v10851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19122.274246 secs 2024-04-26T23:54:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19123.054751 secs 2024-04-26T23:54:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19123.054751 secs 2024-04-26T23:54:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19123.054751 secs 2024-04-26T23:54:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:03 smithi119 ceph-mon[32752]: pgmap v10852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:03.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:03 smithi167 ceph-mon[32751]: pgmap v10852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:03 smithi090 ceph-mon[29180]: pgmap v10852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:05 smithi119 ceph-mon[32752]: pgmap v10853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:05 smithi167 ceph-mon[32751]: pgmap v10853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:05.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:05 smithi090 ceph-mon[29180]: pgmap v10853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19127.274330 secs 2024-04-26T23:54:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19127.274330 secs 2024-04-26T23:54:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19127.274330 secs 2024-04-26T23:54:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:07 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19128.054869 secs 2024-04-26T23:54:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:07 smithi119 ceph-mon[32752]: pgmap v10854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:07 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19128.054869 secs 2024-04-26T23:54:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:07 smithi167 ceph-mon[32751]: pgmap v10854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:07 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19128.054869 secs 2024-04-26T23:54:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:07 smithi090 ceph-mon[29180]: pgmap v10854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:09.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:09 smithi119 ceph-mon[32752]: pgmap v10855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:09 smithi167 ceph-mon[32751]: pgmap v10855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:09.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:09 smithi090 ceph-mon[29180]: pgmap v10855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:11 smithi119 ceph-mon[32752]: pgmap v10856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:11 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19132.274416 secs 2024-04-26T23:54:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:11 smithi167 ceph-mon[32751]: pgmap v10856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:11 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19132.274416 secs 2024-04-26T23:54:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:11 smithi090 ceph-mon[29180]: pgmap v10856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:11 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19132.274416 secs 2024-04-26T23:54:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:12 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19133.055005 secs 2024-04-26T23:54:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:12 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19133.055005 secs 2024-04-26T23:54:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:12 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19133.055005 secs 2024-04-26T23:54:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:13 smithi119 ceph-mon[32752]: pgmap v10857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:13.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:13 smithi167 ceph-mon[32751]: pgmap v10857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:13 smithi090 ceph-mon[29180]: pgmap v10857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:15.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:15 smithi167 ceph-mon[32751]: pgmap v10858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:15.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:15 smithi090 ceph-mon[29180]: pgmap v10858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:15.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:15 smithi119 ceph-mon[32752]: pgmap v10858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19137.274501 secs 2024-04-26T23:54:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19137.274501 secs 2024-04-26T23:54:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19137.274501 secs 2024-04-26T23:54:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:17 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19138.055145 secs 2024-04-26T23:54:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:17 smithi167 ceph-mon[32751]: pgmap v10859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:17 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19138.055145 secs 2024-04-26T23:54:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:17 smithi090 ceph-mon[29180]: pgmap v10859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:17 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19138.055145 secs 2024-04-26T23:54:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:17 smithi119 ceph-mon[32752]: pgmap v10859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:18.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:18 smithi090 ceph-mon[29180]: pgmap v10860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:18.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:18 smithi119 ceph-mon[32752]: pgmap v10860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:18.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:18 smithi167 ceph-mon[32751]: pgmap v10860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:20 smithi119 ceph-mon[32752]: pgmap v10861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:21 smithi167 ceph-mon[32751]: pgmap v10861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:20 smithi090 ceph-mon[29180]: pgmap v10861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19142.274588 secs 2024-04-26T23:54:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:22 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19143.055271 secs 2024-04-26T23:54:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19142.274588 secs 2024-04-26T23:54:22.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:22 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19143.055271 secs 2024-04-26T23:54:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19142.274588 secs 2024-04-26T23:54:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:22 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19143.055271 secs 2024-04-26T23:54:23.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:23 smithi119 ceph-mon[32752]: pgmap v10862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:23.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:23 smithi167 ceph-mon[32751]: pgmap v10862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:23.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:23 smithi090 ceph-mon[29180]: pgmap v10862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:25 smithi119 ceph-mon[32752]: pgmap v10863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:25.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:25 smithi167 ceph-mon[32751]: pgmap v10863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:25.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:25 smithi090 ceph-mon[29180]: pgmap v10863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:26 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19147.274673 secs 2024-04-26T23:54:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:26 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19147.274673 secs 2024-04-26T23:54:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:26 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19147.274673 secs 2024-04-26T23:54:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:27 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19148.055417 secs 2024-04-26T23:54:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:27 smithi119 ceph-mon[32752]: pgmap v10864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:27 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19148.055417 secs 2024-04-26T23:54:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:27 smithi167 ceph-mon[32751]: pgmap v10864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:27 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19148.055417 secs 2024-04-26T23:54:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:27 smithi090 ceph-mon[29180]: pgmap v10864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:29.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:29 smithi119 ceph-mon[32752]: pgmap v10865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:29.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:29 smithi167 ceph-mon[32751]: pgmap v10865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:29.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:29 smithi090 ceph-mon[29180]: pgmap v10865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:31 smithi119 ceph-mon[32752]: pgmap v10866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:31 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19152.274760 secs 2024-04-26T23:54:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:31 smithi167 ceph-mon[32751]: pgmap v10866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:31 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19152.274760 secs 2024-04-26T23:54:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:31 smithi090 ceph-mon[29180]: pgmap v10866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:31 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19152.274760 secs 2024-04-26T23:54:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:32 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19153.055503 secs 2024-04-26T23:54:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:32 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19153.055503 secs 2024-04-26T23:54:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:32 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19153.055503 secs 2024-04-26T23:54:33.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:33 smithi119 ceph-mon[32752]: pgmap v10867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:33.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:33 smithi167 ceph-mon[32751]: pgmap v10867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:33.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:33 smithi090 ceph-mon[29180]: pgmap v10867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:35.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:35 smithi119 ceph-mon[32752]: pgmap v10868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:35.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:35 smithi167 ceph-mon[32751]: pgmap v10868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:35.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:35 smithi090 ceph-mon[29180]: pgmap v10868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:36 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19157.274850 secs 2024-04-26T23:54:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:36 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19157.274850 secs 2024-04-26T23:54:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:36 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19157.274850 secs 2024-04-26T23:54:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:37 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19158.055588 secs 2024-04-26T23:54:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:37 smithi119 ceph-mon[32752]: pgmap v10869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:37 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19158.055588 secs 2024-04-26T23:54:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:37 smithi167 ceph-mon[32751]: pgmap v10869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:37 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19158.055588 secs 2024-04-26T23:54:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:37 smithi090 ceph-mon[29180]: pgmap v10869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:39.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:39 smithi119 ceph-mon[32752]: pgmap v10870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:39.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:39 smithi167 ceph-mon[32751]: pgmap v10870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:39.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:39 smithi090 ceph-mon[29180]: pgmap v10870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:41 smithi119 ceph-mon[32752]: pgmap v10871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19162.274933 secs 2024-04-26T23:54:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:41 smithi167 ceph-mon[32751]: pgmap v10871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19162.274933 secs 2024-04-26T23:54:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:41 smithi090 ceph-mon[29180]: pgmap v10871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19162.274933 secs 2024-04-26T23:54:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:42 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19163.055716 secs 2024-04-26T23:54:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:54:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:42 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19163.055716 secs 2024-04-26T23:54:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:54:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:42 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19163.055716 secs 2024-04-26T23:54:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:54:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:43 smithi167 ceph-mon[32751]: pgmap v10872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:54:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:54:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:54:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:43 smithi090 ceph-mon[29180]: pgmap v10872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:54:43.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:54:43.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:54:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:43 smithi119 ceph-mon[32752]: pgmap v10872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:54:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:54:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:54:44.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:44 smithi167 ceph-mon[32751]: pgmap v10873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:54:44.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:44 smithi167 ceph-mon[32751]: pgmap v10874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:54:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:44 smithi090 ceph-mon[29180]: pgmap v10873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:54:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:44 smithi090 ceph-mon[29180]: pgmap v10874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:54:44.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:44 smithi119 ceph-mon[32752]: pgmap v10873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:54:44.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:44 smithi119 ceph-mon[32752]: pgmap v10874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:54:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:46 smithi167 ceph-mon[32751]: pgmap v10875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19167.275020 secs 2024-04-26T23:54:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:46 smithi090 ceph-mon[29180]: pgmap v10875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19167.275020 secs 2024-04-26T23:54:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:46 smithi119 ceph-mon[32752]: pgmap v10875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19167.275020 secs 2024-04-26T23:54:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:47 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19168.055818 secs 2024-04-26T23:54:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:47 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19168.055818 secs 2024-04-26T23:54:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:47 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19168.055818 secs 2024-04-26T23:54:48.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:48 smithi090 ceph-mon[29180]: pgmap v10876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:48.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:48 smithi119 ceph-mon[32752]: pgmap v10876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:48.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:48 smithi167 ceph-mon[32751]: pgmap v10876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:49 smithi090 ceph-mon[29180]: pgmap v10877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:49 smithi119 ceph-mon[32752]: pgmap v10877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:49 smithi167 ceph-mon[32751]: pgmap v10877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19172.275104 secs 2024-04-26T23:54:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:51 smithi119 ceph-mon[32752]: pgmap v10878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19173.055894 secs 2024-04-26T23:54:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19172.275104 secs 2024-04-26T23:54:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:51 smithi167 ceph-mon[32751]: pgmap v10878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19173.055894 secs 2024-04-26T23:54:52.081 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19172.275104 secs 2024-04-26T23:54:52.081 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:51 smithi090 ceph-mon[29180]: pgmap v10878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:54:52.081 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19173.055894 secs 2024-04-26T23:54:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:53 smithi119 ceph-mon[32752]: pgmap v10879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:54:54.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:53 smithi167 ceph-mon[32751]: pgmap v10879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:54:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:53 smithi090 ceph-mon[29180]: pgmap v10879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:54:56.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:55 smithi119 ceph-mon[32752]: pgmap v10880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:56.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:55 smithi167 ceph-mon[32751]: pgmap v10880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:55 smithi090 ceph-mon[29180]: pgmap v10880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:54:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19177.275190 secs 2024-04-26T23:54:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19178.056021 secs 2024-04-26T23:54:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19177.275190 secs 2024-04-26T23:54:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19178.056021 secs 2024-04-26T23:54:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19177.275190 secs 2024-04-26T23:54:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19178.056021 secs 2024-04-26T23:54:58.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:57 smithi119 ceph-mon[32752]: pgmap v10881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:58.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:57 smithi167 ceph-mon[32751]: pgmap v10881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:54:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:57 smithi090 ceph-mon[29180]: pgmap v10881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:00.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:54:59 smithi119 ceph-mon[32752]: pgmap v10882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:54:59 smithi167 ceph-mon[32751]: pgmap v10882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:00.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:54:59 smithi090 ceph-mon[29180]: pgmap v10882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19182.275249 secs 2024-04-26T23:55:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:01 smithi119 ceph-mon[32752]: pgmap v10883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19183.056141 secs 2024-04-26T23:55:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19182.275249 secs 2024-04-26T23:55:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:01 smithi167 ceph-mon[32751]: pgmap v10883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19183.056141 secs 2024-04-26T23:55:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19182.275249 secs 2024-04-26T23:55:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:01 smithi090 ceph-mon[29180]: pgmap v10883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19183.056141 secs 2024-04-26T23:55:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:03 smithi119 ceph-mon[32752]: pgmap v10884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:03 smithi167 ceph-mon[32751]: pgmap v10884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:03 smithi090 ceph-mon[29180]: pgmap v10884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:06.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:05 smithi090 ceph-mon[29180]: pgmap v10885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:06.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:05 smithi119 ceph-mon[32752]: pgmap v10885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:05 smithi167 ceph-mon[32751]: pgmap v10885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:06.378 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:55:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:55:06.052+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:55:06.378 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:55:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:55:06.052+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:55:06.378 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:55:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:55:06.052+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:55:06.378 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:55:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:55:06.052+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:55:06.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 26 23:55:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-26T23:55:06.052+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T23:55:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19187.275334 secs 2024-04-26T23:55:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19188.056251 secs 2024-04-26T23:55:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19187.275334 secs 2024-04-26T23:55:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19188.056251 secs 2024-04-26T23:55:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19187.275334 secs 2024-04-26T23:55:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19188.056251 secs 2024-04-26T23:55:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:07 smithi119 ceph-mon[32752]: pgmap v10886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:07 smithi167 ceph-mon[32751]: pgmap v10886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:07 smithi090 ceph-mon[29180]: pgmap v10886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:09 smithi167 ceph-mon[32751]: pgmap v10887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:10.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:09 smithi090 ceph-mon[29180]: pgmap v10887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:09 smithi119 ceph-mon[32752]: pgmap v10887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:11 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19192.275419 secs 2024-04-26T23:55:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:11 smithi167 ceph-mon[32751]: pgmap v10888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19193.056314 secs 2024-04-26T23:55:12.107 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:11 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19192.275419 secs 2024-04-26T23:55:12.107 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:11 smithi119 ceph-mon[32752]: pgmap v10888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:12.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19193.056314 secs 2024-04-26T23:55:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:11 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19192.275419 secs 2024-04-26T23:55:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:11 smithi090 ceph-mon[29180]: pgmap v10888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19193.056314 secs 2024-04-26T23:55:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:13 smithi167 ceph-mon[32751]: pgmap v10889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:14.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:13 smithi090 ceph-mon[29180]: pgmap v10889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:13 smithi119 ceph-mon[32752]: pgmap v10889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:15 smithi167 ceph-mon[32751]: pgmap v10890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:15 smithi090 ceph-mon[29180]: pgmap v10890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:15 smithi119 ceph-mon[32752]: pgmap v10890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19197.275506 secs 2024-04-26T23:55:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19198.056458 secs 2024-04-26T23:55:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19197.275506 secs 2024-04-26T23:55:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19198.056458 secs 2024-04-26T23:55:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19197.275506 secs 2024-04-26T23:55:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19198.056458 secs 2024-04-26T23:55:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:17 smithi090 ceph-mon[29180]: pgmap v10891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:18.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:17 smithi119 ceph-mon[32752]: pgmap v10891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:18.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:17 smithi167 ceph-mon[32751]: pgmap v10891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:19 smithi090 ceph-mon[29180]: pgmap v10892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:19 smithi119 ceph-mon[32752]: pgmap v10892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:19 smithi167 ceph-mon[32751]: pgmap v10892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T23:55:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:21 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19202.275589 secs 2024-04-26T23:55:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:21 smithi090 ceph-mon[29180]: pgmap v10893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19203.056583 secs 2024-04-26T23:55:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:21 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19202.275589 secs 2024-04-26T23:55:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:21 smithi119 ceph-mon[32752]: pgmap v10893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19203.056583 secs 2024-04-26T23:55:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:21 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19202.275589 secs 2024-04-26T23:55:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:21 smithi167 ceph-mon[32751]: pgmap v10893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19203.056583 secs 2024-04-26T23:55:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:23 smithi090 ceph-mon[29180]: pgmap v10894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:24.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:23 smithi119 ceph-mon[32752]: pgmap v10894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:24.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:23 smithi167 ceph-mon[32751]: pgmap v10894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:25 smithi090 ceph-mon[29180]: pgmap v10895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:25 smithi119 ceph-mon[32752]: pgmap v10895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:25 smithi167 ceph-mon[32751]: pgmap v10895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:26 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19207.275674 secs 2024-04-26T23:55:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19208.056694 secs 2024-04-26T23:55:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:26 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19207.275674 secs 2024-04-26T23:55:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19208.056694 secs 2024-04-26T23:55:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:26 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19207.275674 secs 2024-04-26T23:55:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19208.056694 secs 2024-04-26T23:55:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:27 smithi090 ceph-mon[29180]: pgmap v10896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:28.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:27 smithi119 ceph-mon[32752]: pgmap v10896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:28.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:27 smithi167 ceph-mon[32751]: pgmap v10896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:29 smithi090 ceph-mon[29180]: pgmap v10897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:30.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:29 smithi119 ceph-mon[32752]: pgmap v10897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:30.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:29 smithi167 ceph-mon[32751]: pgmap v10897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:31 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19212.275732 secs 2024-04-26T23:55:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:31 smithi090 ceph-mon[29180]: pgmap v10898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19213.056839 secs 2024-04-26T23:55:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:31 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19212.275732 secs 2024-04-26T23:55:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:31 smithi119 ceph-mon[32752]: pgmap v10898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19213.056839 secs 2024-04-26T23:55:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:31 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19212.275732 secs 2024-04-26T23:55:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:31 smithi167 ceph-mon[32751]: pgmap v10898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19213.056839 secs 2024-04-26T23:55:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:33 smithi090 ceph-mon[29180]: pgmap v10899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:34.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:33 smithi119 ceph-mon[32752]: pgmap v10899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:34.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:33 smithi167 ceph-mon[32751]: pgmap v10899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:36.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:35 smithi167 ceph-mon[32751]: pgmap v10900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:35 smithi090 ceph-mon[29180]: pgmap v10900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:35 smithi119 ceph-mon[32752]: pgmap v10900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:36 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19217.275819 secs 2024-04-26T23:55:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19218.056911 secs 2024-04-26T23:55:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:36 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19217.275819 secs 2024-04-26T23:55:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19218.056911 secs 2024-04-26T23:55:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:36 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19217.275819 secs 2024-04-26T23:55:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19218.056911 secs 2024-04-26T23:55:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:37 smithi090 ceph-mon[29180]: pgmap v10901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:37 smithi119 ceph-mon[32752]: pgmap v10901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:38.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:37 smithi167 ceph-mon[32751]: pgmap v10901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:39 smithi090 ceph-mon[29180]: pgmap v10902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:40.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:39 smithi119 ceph-mon[32752]: pgmap v10902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:40.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:39 smithi167 ceph-mon[32751]: pgmap v10902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:55:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19222.275882 secs 2024-04-26T23:55:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:41 smithi090 ceph-mon[29180]: pgmap v10903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19223.056981 secs 2024-04-26T23:55:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19222.275882 secs 2024-04-26T23:55:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:41 smithi119 ceph-mon[32752]: pgmap v10903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19223.056981 secs 2024-04-26T23:55:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19222.275882 secs 2024-04-26T23:55:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:41 smithi167 ceph-mon[32751]: pgmap v10903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19223.056981 secs 2024-04-26T23:55:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:55:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:55:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:55:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:43 smithi119 ceph-mon[32752]: pgmap v10904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:55:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:55:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:55:44.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:43 smithi167 ceph-mon[32751]: pgmap v10904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:44.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:55:44.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:55:44.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:55:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:43 smithi090 ceph-mon[29180]: pgmap v10904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:55:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:55:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:55:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:44 smithi119 ceph-mon[32752]: pgmap v10905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:55:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:44 smithi119 ceph-mon[32752]: pgmap v10906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:45.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:44 smithi167 ceph-mon[32751]: pgmap v10905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:55:45.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:44 smithi167 ceph-mon[32751]: pgmap v10906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:44 smithi090 ceph-mon[29180]: pgmap v10905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-26T23:55:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:44 smithi090 ceph-mon[29180]: pgmap v10906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19227.275968 secs 2024-04-26T23:55:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:46 smithi119 ceph-mon[32752]: pgmap v10907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19228.057104 secs 2024-04-26T23:55:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19227.275968 secs 2024-04-26T23:55:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:46 smithi167 ceph-mon[32751]: pgmap v10907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19228.057104 secs 2024-04-26T23:55:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19227.275968 secs 2024-04-26T23:55:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:46 smithi090 ceph-mon[29180]: pgmap v10907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19228.057104 secs 2024-04-26T23:55:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:48 smithi119 ceph-mon[32752]: pgmap v10908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:55:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:48 smithi167 ceph-mon[32751]: pgmap v10908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:55:49.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:48 smithi090 ceph-mon[29180]: pgmap v10908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:55:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:50 smithi119 ceph-mon[32752]: pgmap v10909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:50 smithi167 ceph-mon[32751]: pgmap v10909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:50 smithi090 ceph-mon[29180]: pgmap v10909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19232.276071 secs 2024-04-26T23:55:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19233.057174 secs 2024-04-26T23:55:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19232.276071 secs 2024-04-26T23:55:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19233.057174 secs 2024-04-26T23:55:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19232.276071 secs 2024-04-26T23:55:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19233.057174 secs 2024-04-26T23:55:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:53 smithi119 ceph-mon[32752]: pgmap v10910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:53.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:53 smithi167 ceph-mon[32751]: pgmap v10910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:53 smithi090 ceph-mon[29180]: pgmap v10910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:55:55.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:55 smithi119 ceph-mon[32752]: pgmap v10911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:55:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:55 smithi167 ceph-mon[32751]: pgmap v10911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:55:55.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:55 smithi090 ceph-mon[29180]: pgmap v10911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:55:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19237.276154 secs 2024-04-26T23:55:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19237.276154 secs 2024-04-26T23:55:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19237.276154 secs 2024-04-26T23:55:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:57 smithi119 ceph-mon[32752]: pgmap v10912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19238.057321 secs 2024-04-26T23:55:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:57 smithi167 ceph-mon[32751]: pgmap v10912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19238.057321 secs 2024-04-26T23:55:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:57 smithi090 ceph-mon[29180]: pgmap v10912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19238.057321 secs 2024-04-26T23:55:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:55:59 smithi119 ceph-mon[32752]: pgmap v10913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:55:59 smithi167 ceph-mon[32751]: pgmap v10913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:55:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:55:59 smithi090 ceph-mon[29180]: pgmap v10913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:01 smithi119 ceph-mon[32752]: pgmap v10914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19242.276242 secs 2024-04-26T23:56:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:01 smithi167 ceph-mon[32751]: pgmap v10914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19242.276242 secs 2024-04-26T23:56:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:01 smithi090 ceph-mon[29180]: pgmap v10914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19242.276242 secs 2024-04-26T23:56:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19243.057459 secs 2024-04-26T23:56:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19243.057459 secs 2024-04-26T23:56:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19243.057459 secs 2024-04-26T23:56:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:03 smithi119 ceph-mon[32752]: pgmap v10915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:03.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:03 smithi167 ceph-mon[32751]: pgmap v10915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:03 smithi090 ceph-mon[29180]: pgmap v10915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:05.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:05 smithi167 ceph-mon[32751]: pgmap v10916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:05 smithi090 ceph-mon[29180]: pgmap v10916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:05 smithi119 ceph-mon[32752]: pgmap v10916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19247.276303 secs 2024-04-26T23:56:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19247.276303 secs 2024-04-26T23:56:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19247.276303 secs 2024-04-26T23:56:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:07 smithi167 ceph-mon[32751]: pgmap v10917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:07 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19248.057543 secs 2024-04-26T23:56:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:07 smithi090 ceph-mon[29180]: pgmap v10917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:07 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19248.057543 secs 2024-04-26T23:56:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:07 smithi119 ceph-mon[32752]: pgmap v10917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:07 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19248.057543 secs 2024-04-26T23:56:09.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:09 smithi167 ceph-mon[32751]: pgmap v10918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:09.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:09 smithi090 ceph-mon[29180]: pgmap v10918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:09.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:09 smithi119 ceph-mon[32752]: pgmap v10918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:11 smithi090 ceph-mon[29180]: pgmap v10919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:11 smithi090 ceph-mon[29180]: 28 slow requests, 2 included below; oldest blocked for > 19252.276388 secs 2024-04-26T23:56:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:11 smithi090 ceph-mon[29180]: slow request 2884.782401 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:11 smithi090 ceph-mon[29180]: slow request 364.038138 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:11 smithi119 ceph-mon[32752]: pgmap v10919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:11 smithi119 ceph-mon[32752]: 28 slow requests, 2 included below; oldest blocked for > 19252.276388 secs 2024-04-26T23:56:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:11 smithi119 ceph-mon[32752]: slow request 2884.782401 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:11 smithi119 ceph-mon[32752]: slow request 364.038138 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:11 smithi167 ceph-mon[32751]: pgmap v10919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:11 smithi167 ceph-mon[32751]: 28 slow requests, 2 included below; oldest blocked for > 19252.276388 secs 2024-04-26T23:56:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:11 smithi167 ceph-mon[32751]: slow request 2884.782401 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:11 smithi167 ceph-mon[32751]: slow request 364.038138 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:12 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19253.057613 secs 2024-04-26T23:56:12.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:12 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19253.057613 secs 2024-04-26T23:56:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:12 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19253.057613 secs 2024-04-26T23:56:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:13 smithi090 ceph-mon[29180]: pgmap v10920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:13 smithi119 ceph-mon[32752]: pgmap v10920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:13 smithi167 ceph-mon[32751]: pgmap v10920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:14 smithi119 ceph-mon[32752]: pgmap v10921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:14 smithi167 ceph-mon[32751]: pgmap v10921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:14 smithi090 ceph-mon[29180]: pgmap v10921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19257.276472 secs 2024-04-26T23:56:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:16 smithi167 ceph-mon[32751]: pgmap v10922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19258.057738 secs 2024-04-26T23:56:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19257.276472 secs 2024-04-26T23:56:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:16 smithi090 ceph-mon[29180]: pgmap v10922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19258.057738 secs 2024-04-26T23:56:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19257.276472 secs 2024-04-26T23:56:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:16 smithi119 ceph-mon[32752]: pgmap v10922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19258.057738 secs 2024-04-26T23:56:18.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:18 smithi167 ceph-mon[32751]: pgmap v10923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:18.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:18 smithi090 ceph-mon[29180]: pgmap v10923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:18 smithi119 ceph-mon[32752]: pgmap v10923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:20 smithi090 ceph-mon[29180]: pgmap v10924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:20 smithi119 ceph-mon[32752]: pgmap v10924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:20 smithi167 ceph-mon[32751]: pgmap v10924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:21 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19262.276559 secs 2024-04-26T23:56:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19263.057841 secs 2024-04-26T23:56:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:21 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19262.276559 secs 2024-04-26T23:56:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19263.057841 secs 2024-04-26T23:56:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:21 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19262.276559 secs 2024-04-26T23:56:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19263.057841 secs 2024-04-26T23:56:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:22 smithi090 ceph-mon[29180]: pgmap v10925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:22 smithi119 ceph-mon[32752]: pgmap v10925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:22 smithi167 ceph-mon[32751]: pgmap v10925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:24 smithi090 ceph-mon[29180]: pgmap v10926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:24 smithi119 ceph-mon[32752]: pgmap v10926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:24 smithi167 ceph-mon[32751]: pgmap v10926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:26 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19267.276644 secs 2024-04-26T23:56:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:26 smithi090 ceph-mon[29180]: pgmap v10927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19268.057927 secs 2024-04-26T23:56:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:26 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19267.276644 secs 2024-04-26T23:56:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:26 smithi119 ceph-mon[32752]: pgmap v10927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19268.057927 secs 2024-04-26T23:56:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:26 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19267.276644 secs 2024-04-26T23:56:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:26 smithi167 ceph-mon[32751]: pgmap v10927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19268.057927 secs 2024-04-26T23:56:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:28 smithi090 ceph-mon[29180]: pgmap v10928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:28 smithi119 ceph-mon[32752]: pgmap v10928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:28 smithi167 ceph-mon[32751]: pgmap v10928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:30 smithi090 ceph-mon[29180]: pgmap v10929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:30 smithi119 ceph-mon[32752]: pgmap v10929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:30 smithi167 ceph-mon[32751]: pgmap v10929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:31 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 19272.276716 secs 2024-04-26T23:56:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:31 smithi090 ceph-mon[29180]: slow request 723.780824 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:56:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19273.058056 secs 2024-04-26T23:56:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:31 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 19272.276716 secs 2024-04-26T23:56:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:31 smithi119 ceph-mon[32752]: slow request 723.780824 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:56:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19273.058056 secs 2024-04-26T23:56:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:31 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 19272.276716 secs 2024-04-26T23:56:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:31 smithi167 ceph-mon[32751]: slow request 723.780824 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-26T23:56:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19273.058056 secs 2024-04-26T23:56:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:32 smithi090 ceph-mon[29180]: pgmap v10930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:32 smithi119 ceph-mon[32752]: pgmap v10930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:32 smithi167 ceph-mon[32751]: pgmap v10930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:34.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:34 smithi090 ceph-mon[29180]: pgmap v10931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:34 smithi119 ceph-mon[32752]: pgmap v10931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:34 smithi167 ceph-mon[32751]: pgmap v10931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:36.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:36 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 19277.276801 secs 2024-04-26T23:56:36.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:36 smithi167 ceph-mon[32751]: slow request 5760.610935 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:36.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:36 smithi167 ceph-mon[32751]: pgmap v10932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:36.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19278.058201 secs 2024-04-26T23:56:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:36 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 19277.276801 secs 2024-04-26T23:56:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:36 smithi090 ceph-mon[29180]: slow request 5760.610935 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:36 smithi090 ceph-mon[29180]: pgmap v10932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19278.058201 secs 2024-04-26T23:56:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:36 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 19277.276801 secs 2024-04-26T23:56:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:36 smithi119 ceph-mon[32752]: slow request 5760.610935 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:56:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:36 smithi119 ceph-mon[32752]: pgmap v10932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19278.058201 secs 2024-04-26T23:56:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:38 smithi090 ceph-mon[29180]: pgmap v10933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:38 smithi119 ceph-mon[32752]: pgmap v10933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:38 smithi167 ceph-mon[32751]: pgmap v10933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:40 smithi090 ceph-mon[29180]: pgmap v10934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:40 smithi119 ceph-mon[32752]: pgmap v10934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:40 smithi167 ceph-mon[32751]: pgmap v10934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19282.276864 secs 2024-04-26T23:56:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19283.058298 secs 2024-04-26T23:56:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19282.276864 secs 2024-04-26T23:56:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19283.058298 secs 2024-04-26T23:56:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19282.276864 secs 2024-04-26T23:56:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19283.058298 secs 2024-04-26T23:56:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:43 smithi119 ceph-mon[32752]: pgmap v10935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:43 smithi167 ceph-mon[32751]: pgmap v10935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:43 smithi090 ceph-mon[29180]: pgmap v10935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:56:44.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:56:44.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:56:44.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:56:45.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:45 smithi119 ceph-mon[32752]: pgmap v10936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:56:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:56:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:56:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:45 smithi167 ceph-mon[32751]: pgmap v10936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:56:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:56:45.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:56:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:45 smithi090 ceph-mon[29180]: pgmap v10936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:56:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:56:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:56:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:56:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:46 smithi119 ceph-mon[32752]: pgmap v10937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:56:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:46 smithi119 ceph-mon[32752]: pgmap v10938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19287.276922 secs 2024-04-26T23:56:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:46 smithi167 ceph-mon[32751]: pgmap v10937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:56:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:46 smithi167 ceph-mon[32751]: pgmap v10938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19287.276922 secs 2024-04-26T23:56:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:46 smithi090 ceph-mon[29180]: pgmap v10937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:56:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:46 smithi090 ceph-mon[29180]: pgmap v10938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19287.276922 secs 2024-04-26T23:56:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:47 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19288.058418 secs 2024-04-26T23:56:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:47 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19288.058418 secs 2024-04-26T23:56:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:47 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19288.058418 secs 2024-04-26T23:56:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:48 smithi167 ceph-mon[32751]: pgmap v10939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:48 smithi090 ceph-mon[29180]: pgmap v10939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:48 smithi119 ceph-mon[32752]: pgmap v10939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:50 smithi167 ceph-mon[32751]: pgmap v10940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:56:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:50 smithi090 ceph-mon[29180]: pgmap v10940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:56:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:50 smithi119 ceph-mon[32752]: pgmap v10940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:56:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19292.277004 secs 2024-04-26T23:56:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19292.277004 secs 2024-04-26T23:56:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19292.277004 secs 2024-04-26T23:56:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:52 smithi090 ceph-mon[29180]: pgmap v10941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19293.058514 secs 2024-04-26T23:56:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:52 smithi119 ceph-mon[32752]: pgmap v10941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19293.058514 secs 2024-04-26T23:56:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:52 smithi167 ceph-mon[32751]: pgmap v10941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:56:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19293.058514 secs 2024-04-26T23:56:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:54 smithi090 ceph-mon[29180]: pgmap v10942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:56:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:54 smithi119 ceph-mon[32752]: pgmap v10942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:56:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:54 smithi167 ceph-mon[32751]: pgmap v10942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:56:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:56 smithi090 ceph-mon[29180]: pgmap v10943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T23:56:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19297.277084 secs 2024-04-26T23:56:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:56 smithi119 ceph-mon[32752]: pgmap v10943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T23:56:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19297.277084 secs 2024-04-26T23:56:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:56 smithi167 ceph-mon[32751]: pgmap v10943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-26T23:56:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19297.277084 secs 2024-04-26T23:56:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19298.058613 secs 2024-04-26T23:56:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19298.058613 secs 2024-04-26T23:56:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19298.058613 secs 2024-04-26T23:56:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:56:58 smithi090 ceph-mon[29180]: pgmap v10944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:56:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:56:58 smithi119 ceph-mon[32752]: pgmap v10944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:56:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:56:58 smithi167 ceph-mon[32751]: pgmap v10944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:57:00.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:00 smithi090 ceph-mon[29180]: pgmap v10945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:00 smithi119 ceph-mon[32752]: pgmap v10945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:00 smithi167 ceph-mon[32751]: pgmap v10945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19302.277143 secs 2024-04-26T23:57:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:01 smithi090 ceph-mon[29180]: pgmap v10946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19302.277143 secs 2024-04-26T23:57:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:01 smithi119 ceph-mon[32752]: pgmap v10946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19302.277143 secs 2024-04-26T23:57:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:01 smithi167 ceph-mon[32751]: pgmap v10946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19303.058734 secs 2024-04-26T23:57:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19303.058734 secs 2024-04-26T23:57:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19303.058734 secs 2024-04-26T23:57:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:03 smithi090 ceph-mon[29180]: pgmap v10947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:57:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:03 smithi119 ceph-mon[32752]: pgmap v10947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:57:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:03 smithi167 ceph-mon[32751]: pgmap v10947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T23:57:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:05 smithi090 ceph-mon[29180]: pgmap v10948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:05 smithi119 ceph-mon[32752]: pgmap v10948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:05 smithi167 ceph-mon[32751]: pgmap v10948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19307.277226 secs 2024-04-26T23:57:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19308.058799 secs 2024-04-26T23:57:06.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19307.277226 secs 2024-04-26T23:57:06.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19308.058799 secs 2024-04-26T23:57:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19307.277226 secs 2024-04-26T23:57:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19308.058799 secs 2024-04-26T23:57:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:07 smithi090 ceph-mon[29180]: pgmap v10949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:07 smithi119 ceph-mon[32752]: pgmap v10949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:07 smithi167 ceph-mon[32751]: pgmap v10949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:09 smithi090 ceph-mon[29180]: pgmap v10950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:09 smithi119 ceph-mon[32752]: pgmap v10950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:09 smithi167 ceph-mon[32751]: pgmap v10950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:11 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19312.277284 secs 2024-04-26T23:57:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:11 smithi090 ceph-mon[29180]: pgmap v10951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19313.058910 secs 2024-04-26T23:57:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:11 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19312.277284 secs 2024-04-26T23:57:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:11 smithi119 ceph-mon[32752]: pgmap v10951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19313.058910 secs 2024-04-26T23:57:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:11 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19312.277284 secs 2024-04-26T23:57:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:11 smithi167 ceph-mon[32751]: pgmap v10951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19313.058910 secs 2024-04-26T23:57:13.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:13 smithi119 ceph-mon[32752]: pgmap v10952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:13 smithi090 ceph-mon[29180]: pgmap v10952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:13 smithi167 ceph-mon[32751]: pgmap v10952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:15 smithi090 ceph-mon[29180]: pgmap v10953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:15 smithi119 ceph-mon[32752]: pgmap v10953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:15 smithi167 ceph-mon[32751]: pgmap v10953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19317.277346 secs 2024-04-26T23:57:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19318.059043 secs 2024-04-26T23:57:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19317.277346 secs 2024-04-26T23:57:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19318.059043 secs 2024-04-26T23:57:16.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19317.277346 secs 2024-04-26T23:57:16.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19318.059043 secs 2024-04-26T23:57:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:17 smithi090 ceph-mon[29180]: pgmap v10954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:17 smithi119 ceph-mon[32752]: pgmap v10954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:17 smithi167 ceph-mon[32751]: pgmap v10954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:19.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:19 smithi090 ceph-mon[29180]: pgmap v10955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:19 smithi119 ceph-mon[32752]: pgmap v10955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:19 smithi167 ceph-mon[32751]: pgmap v10955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:21 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19322.277429 secs 2024-04-26T23:57:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:21 smithi090 ceph-mon[29180]: pgmap v10956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19323.059161 secs 2024-04-26T23:57:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:21 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19322.277429 secs 2024-04-26T23:57:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:21 smithi119 ceph-mon[32752]: pgmap v10956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19323.059161 secs 2024-04-26T23:57:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:21 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19322.277429 secs 2024-04-26T23:57:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:21 smithi167 ceph-mon[32751]: pgmap v10956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19323.059161 secs 2024-04-26T23:57:23.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:23 smithi090 ceph-mon[29180]: pgmap v10957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:23 smithi119 ceph-mon[32752]: pgmap v10957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:23 smithi167 ceph-mon[32751]: pgmap v10957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:25 smithi119 ceph-mon[32752]: pgmap v10958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:25 smithi167 ceph-mon[32751]: pgmap v10958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:25 smithi090 ceph-mon[29180]: pgmap v10958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:26 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19327.277515 secs 2024-04-26T23:57:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19328.059284 secs 2024-04-26T23:57:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:26 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19327.277515 secs 2024-04-26T23:57:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19328.059284 secs 2024-04-26T23:57:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:26 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19327.277515 secs 2024-04-26T23:57:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19328.059284 secs 2024-04-26T23:57:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:27 smithi119 ceph-mon[32752]: pgmap v10959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:27 smithi167 ceph-mon[32751]: pgmap v10959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:27 smithi090 ceph-mon[29180]: pgmap v10959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:29 smithi119 ceph-mon[32752]: pgmap v10960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:29 smithi167 ceph-mon[32751]: pgmap v10960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:29 smithi090 ceph-mon[29180]: pgmap v10960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:31 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19332.277598 secs 2024-04-26T23:57:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:31 smithi119 ceph-mon[32752]: pgmap v10961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19333.059416 secs 2024-04-26T23:57:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:31 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19332.277598 secs 2024-04-26T23:57:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:31 smithi167 ceph-mon[32751]: pgmap v10961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19333.059416 secs 2024-04-26T23:57:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:31 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19332.277598 secs 2024-04-26T23:57:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:31 smithi090 ceph-mon[29180]: pgmap v10961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19333.059416 secs 2024-04-26T23:57:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:33 smithi119 ceph-mon[32752]: pgmap v10962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:33 smithi167 ceph-mon[32751]: pgmap v10962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:33 smithi090 ceph-mon[29180]: pgmap v10962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:35 smithi119 ceph-mon[32752]: pgmap v10963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:35 smithi167 ceph-mon[32751]: pgmap v10963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:35 smithi090 ceph-mon[29180]: pgmap v10963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:36.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:36 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19337.277682 secs 2024-04-26T23:57:36.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19338.059537 secs 2024-04-26T23:57:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:36 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19337.277682 secs 2024-04-26T23:57:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19338.059537 secs 2024-04-26T23:57:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:36 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19337.277682 secs 2024-04-26T23:57:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19338.059537 secs 2024-04-26T23:57:37.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:37 smithi090 ceph-mon[29180]: pgmap v10964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:37 smithi119 ceph-mon[32752]: pgmap v10964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:37 smithi167 ceph-mon[32751]: pgmap v10964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:39 smithi119 ceph-mon[32752]: pgmap v10965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:39 smithi167 ceph-mon[32751]: pgmap v10965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:39 smithi090 ceph-mon[29180]: pgmap v10965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19342.277766 secs 2024-04-26T23:57:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:41 smithi119 ceph-mon[32752]: pgmap v10966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19343.059653 secs 2024-04-26T23:57:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19342.277766 secs 2024-04-26T23:57:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:41 smithi167 ceph-mon[32751]: pgmap v10966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19343.059653 secs 2024-04-26T23:57:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19342.277766 secs 2024-04-26T23:57:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:41 smithi090 ceph-mon[29180]: pgmap v10966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19343.059653 secs 2024-04-26T23:57:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:43 smithi119 ceph-mon[32752]: pgmap v10967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:43 smithi167 ceph-mon[32751]: pgmap v10967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:43 smithi090 ceph-mon[29180]: pgmap v10967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:44.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:57:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:57:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:57:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:45 smithi119 ceph-mon[32752]: pgmap v10968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:57:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:57:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:45 smithi119 ceph-mon[32752]: pgmap v10969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:57:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:45 smithi119 ceph-mon[32752]: pgmap v10970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:57:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:45 smithi167 ceph-mon[32751]: pgmap v10968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:57:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:57:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:45 smithi167 ceph-mon[32751]: pgmap v10969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:57:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:45 smithi167 ceph-mon[32751]: pgmap v10970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:57:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:45 smithi090 ceph-mon[29180]: pgmap v10968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:57:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:57:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:45 smithi090 ceph-mon[29180]: pgmap v10969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-26T23:57:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:45 smithi090 ceph-mon[29180]: pgmap v10970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:57:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19347.277851 secs 2024-04-26T23:57:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19348.059748 secs 2024-04-26T23:57:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19347.277851 secs 2024-04-26T23:57:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19348.059748 secs 2024-04-26T23:57:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19347.277851 secs 2024-04-26T23:57:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19348.059748 secs 2024-04-26T23:57:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:47 smithi119 ceph-mon[32752]: pgmap v10971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:57:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:47 smithi167 ceph-mon[32751]: pgmap v10971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:57:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:47 smithi090 ceph-mon[29180]: pgmap v10971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:57:49.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:49 smithi119 ceph-mon[32752]: pgmap v10972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:49 smithi167 ceph-mon[32751]: pgmap v10972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:49 smithi090 ceph-mon[29180]: pgmap v10972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19352.277905 secs 2024-04-26T23:57:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19352.277905 secs 2024-04-26T23:57:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19352.277905 secs 2024-04-26T23:57:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19353.059821 secs 2024-04-26T23:57:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:52 smithi090 ceph-mon[29180]: pgmap v10973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19353.059821 secs 2024-04-26T23:57:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:52 smithi119 ceph-mon[32752]: pgmap v10973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19353.059821 secs 2024-04-26T23:57:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:52 smithi167 ceph-mon[32751]: pgmap v10973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:57:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:54 smithi090 ceph-mon[29180]: pgmap v10974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:57:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:54 smithi119 ceph-mon[32752]: pgmap v10974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:57:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:54 smithi167 ceph-mon[32751]: pgmap v10974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-26T23:57:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19357.277967 secs 2024-04-26T23:57:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:56 smithi090 ceph-mon[29180]: pgmap v10975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:57:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19357.277967 secs 2024-04-26T23:57:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:56 smithi119 ceph-mon[32752]: pgmap v10975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:57:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19357.277967 secs 2024-04-26T23:57:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:56 smithi167 ceph-mon[32751]: pgmap v10975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:57:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19358.059941 secs 2024-04-26T23:57:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19358.059941 secs 2024-04-26T23:57:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19358.059941 secs 2024-04-26T23:57:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:58 smithi090 ceph-mon[29180]: pgmap v10976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:58 smithi119 ceph-mon[32752]: pgmap v10976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:58 smithi167 ceph-mon[32751]: pgmap v10976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:57:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:57:59 smithi090 ceph-mon[29180]: pgmap v10977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:59.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:57:59 smithi119 ceph-mon[32752]: pgmap v10977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:57:59.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:57:59 smithi167 ceph-mon[32751]: pgmap v10977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19362.278049 secs 2024-04-26T23:58:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19362.278049 secs 2024-04-26T23:58:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19362.278049 secs 2024-04-26T23:58:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19363.060061 secs 2024-04-26T23:58:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:02 smithi090 ceph-mon[29180]: pgmap v10978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19363.060061 secs 2024-04-26T23:58:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:02 smithi119 ceph-mon[32752]: pgmap v10978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19363.060061 secs 2024-04-26T23:58:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:02 smithi167 ceph-mon[32751]: pgmap v10978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:04 smithi090 ceph-mon[29180]: pgmap v10979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:04 smithi119 ceph-mon[32752]: pgmap v10979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:04 smithi167 ceph-mon[32751]: pgmap v10979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:06.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19367.278110 secs 2024-04-26T23:58:06.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:06 smithi167 ceph-mon[32751]: pgmap v10980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19367.278110 secs 2024-04-26T23:58:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:06 smithi090 ceph-mon[29180]: pgmap v10980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19367.278110 secs 2024-04-26T23:58:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:06 smithi119 ceph-mon[32752]: pgmap v10980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:07 smithi090 ceph-mon[29180]: 20 slow requests, 1 included below; oldest blocked for > 19368.060187 secs 2024-04-26T23:58:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:07 smithi090 ceph-mon[29180]: slow request 1440.132744 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:58:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:07 smithi119 ceph-mon[32752]: 20 slow requests, 1 included below; oldest blocked for > 19368.060187 secs 2024-04-26T23:58:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:07 smithi119 ceph-mon[32752]: slow request 1440.132744 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:58:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:07 smithi167 ceph-mon[32751]: 20 slow requests, 1 included below; oldest blocked for > 19368.060187 secs 2024-04-26T23:58:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:07 smithi167 ceph-mon[32751]: slow request 1440.132744 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-26T23:58:08.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:08 smithi090 ceph-mon[29180]: pgmap v10981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:08.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:08 smithi119 ceph-mon[32752]: pgmap v10981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:08.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:08 smithi167 ceph-mon[32751]: pgmap v10981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:09 smithi090 ceph-mon[29180]: pgmap v10982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:09.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:09 smithi119 ceph-mon[32752]: pgmap v10982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:09.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:09 smithi167 ceph-mon[32751]: pgmap v10982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:11 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19372.278193 secs 2024-04-26T23:58:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:11 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19372.278193 secs 2024-04-26T23:58:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:11 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19372.278193 secs 2024-04-26T23:58:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:12 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19373.060250 secs 2024-04-26T23:58:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:12 smithi090 ceph-mon[29180]: pgmap v10983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:12 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19373.060250 secs 2024-04-26T23:58:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:12 smithi119 ceph-mon[32752]: pgmap v10983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:12 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19373.060250 secs 2024-04-26T23:58:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:12 smithi167 ceph-mon[32751]: pgmap v10983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:14.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:14 smithi119 ceph-mon[32752]: pgmap v10984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:14.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:14 smithi090 ceph-mon[29180]: pgmap v10984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:14 smithi167 ceph-mon[32751]: pgmap v10984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19377.278277 secs 2024-04-26T23:58:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:16 smithi090 ceph-mon[29180]: pgmap v10985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19377.278277 secs 2024-04-26T23:58:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:16 smithi119 ceph-mon[32752]: pgmap v10985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19377.278277 secs 2024-04-26T23:58:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:16 smithi167 ceph-mon[32751]: pgmap v10985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:17 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19378.060369 secs 2024-04-26T23:58:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:17 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19378.060369 secs 2024-04-26T23:58:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:17 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19378.060369 secs 2024-04-26T23:58:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:18 smithi090 ceph-mon[29180]: pgmap v10986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:18.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:18 smithi119 ceph-mon[32752]: pgmap v10986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:18 smithi167 ceph-mon[32751]: pgmap v10986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:19 smithi090 ceph-mon[29180]: pgmap v10987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:19 smithi119 ceph-mon[32752]: pgmap v10987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:19.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:19 smithi167 ceph-mon[32751]: pgmap v10987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:20 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19382.278380 secs 2024-04-26T23:58:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:20 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19382.278380 secs 2024-04-26T23:58:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:20 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19382.278380 secs 2024-04-26T23:58:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:21 smithi090 ceph-mon[29180]: pgmap v10988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:21 smithi119 ceph-mon[32752]: pgmap v10988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:21 smithi167 ceph-mon[32751]: pgmap v10988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:22 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19383.060506 secs 2024-04-26T23:58:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:22 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19383.060506 secs 2024-04-26T23:58:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:22 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19383.060506 secs 2024-04-26T23:58:23.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:23 smithi090 ceph-mon[29180]: pgmap v10989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:23 smithi119 ceph-mon[32752]: pgmap v10989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:23 smithi167 ceph-mon[32751]: pgmap v10989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:25 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19387.278467 secs 2024-04-26T23:58:25.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:25 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19387.278467 secs 2024-04-26T23:58:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:25 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19387.278467 secs 2024-04-26T23:58:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:26 smithi090 ceph-mon[29180]: pgmap v10990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:26 smithi119 ceph-mon[32752]: pgmap v10990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:26 smithi167 ceph-mon[32751]: pgmap v10990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:27 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19388.060579 secs 2024-04-26T23:58:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:27 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19388.060579 secs 2024-04-26T23:58:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:27 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19388.060579 secs 2024-04-26T23:58:28.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:28 smithi090 ceph-mon[29180]: pgmap v10991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:28.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:28 smithi119 ceph-mon[32752]: pgmap v10991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:28.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:28 smithi167 ceph-mon[32751]: pgmap v10991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:30 smithi090 ceph-mon[29180]: pgmap v10992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:30 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19392.278552 secs 2024-04-26T23:58:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:30 smithi119 ceph-mon[32752]: pgmap v10992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:30 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19392.278552 secs 2024-04-26T23:58:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:30 smithi167 ceph-mon[32751]: pgmap v10992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:30 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19392.278552 secs 2024-04-26T23:58:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:31 smithi090 ceph-mon[29180]: pgmap v10993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:31 smithi119 ceph-mon[32752]: pgmap v10993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:31 smithi167 ceph-mon[32751]: pgmap v10993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:32 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19393.060693 secs 2024-04-26T23:58:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:32 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19393.060693 secs 2024-04-26T23:58:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:32 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19393.060693 secs 2024-04-26T23:58:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:33 smithi090 ceph-mon[29180]: pgmap v10994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:33 smithi119 ceph-mon[32752]: pgmap v10994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:33.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:33 smithi167 ceph-mon[32751]: pgmap v10994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:35.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:35 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19397.278634 secs 2024-04-26T23:58:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:35 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19397.278634 secs 2024-04-26T23:58:35.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:35 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19397.278634 secs 2024-04-26T23:58:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:36 smithi090 ceph-mon[29180]: pgmap v10995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:36.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:36 smithi167 ceph-mon[32751]: pgmap v10995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:36 smithi119 ceph-mon[32752]: pgmap v10995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:37 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19398.060812 secs 2024-04-26T23:58:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:37 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19398.060812 secs 2024-04-26T23:58:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:37 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19398.060812 secs 2024-04-26T23:58:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:38 smithi090 ceph-mon[29180]: pgmap v10996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:38.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:38 smithi119 ceph-mon[32752]: pgmap v10996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:38.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:38 smithi167 ceph-mon[32751]: pgmap v10996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:40 smithi090 ceph-mon[29180]: pgmap v10997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:40 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19402.278696 secs 2024-04-26T23:58:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:40 smithi119 ceph-mon[32752]: pgmap v10997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:40.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:40 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19402.278696 secs 2024-04-26T23:58:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:40 smithi167 ceph-mon[32751]: pgmap v10997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:40 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19402.278696 secs 2024-04-26T23:58:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:41 smithi119 ceph-mon[32752]: pgmap v10998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:41 smithi167 ceph-mon[32751]: pgmap v10998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:41 smithi090 ceph-mon[29180]: pgmap v10998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:42 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19403.060930 secs 2024-04-26T23:58:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:42 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19403.060930 secs 2024-04-26T23:58:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:42 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19403.060930 secs 2024-04-26T23:58:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:43 smithi119 ceph-mon[32752]: pgmap v10999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:43.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:43 smithi167 ceph-mon[32751]: pgmap v10999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:43 smithi090 ceph-mon[29180]: pgmap v10999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:45.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:45 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19407.278765 secs 2024-04-26T23:58:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:45 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19407.278765 secs 2024-04-26T23:58:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:45 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19407.278765 secs 2024-04-26T23:58:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:46 smithi090 ceph-mon[29180]: pgmap v11000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:58:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:58:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:58:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:58:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:46 smithi119 ceph-mon[32752]: pgmap v11000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:58:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:58:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:58:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:58:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:46 smithi167 ceph-mon[32751]: pgmap v11000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:58:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:58:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:58:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:58:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:58:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:47 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19408.061080 secs 2024-04-26T23:58:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:47 smithi090 ceph-mon[29180]: pgmap v11001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:58:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:47 smithi090 ceph-mon[29180]: pgmap v11002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:47 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19408.061080 secs 2024-04-26T23:58:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:47 smithi119 ceph-mon[32752]: pgmap v11001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:58:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:47 smithi119 ceph-mon[32752]: pgmap v11002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:47 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19408.061080 secs 2024-04-26T23:58:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:47 smithi167 ceph-mon[32751]: pgmap v11001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-26T23:58:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:47 smithi167 ceph-mon[32751]: pgmap v11002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:49.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:49 smithi090 ceph-mon[29180]: pgmap v11003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:58:49.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:49 smithi119 ceph-mon[32752]: pgmap v11003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:58:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:49 smithi167 ceph-mon[32751]: pgmap v11003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-26T23:58:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:50 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19412.278850 secs 2024-04-26T23:58:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:50 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19412.278850 secs 2024-04-26T23:58:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:50 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19412.278850 secs 2024-04-26T23:58:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:51 smithi090 ceph-mon[29180]: pgmap v11004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:51 smithi119 ceph-mon[32752]: pgmap v11004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:51 smithi167 ceph-mon[32751]: pgmap v11004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19413.061203 secs 2024-04-26T23:58:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:52 smithi090 ceph-mon[29180]: pgmap v11005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19413.061203 secs 2024-04-26T23:58:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:52 smithi119 ceph-mon[32752]: pgmap v11005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19413.061203 secs 2024-04-26T23:58:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:52 smithi167 ceph-mon[32751]: pgmap v11005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:54 smithi119 ceph-mon[32752]: pgmap v11006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:54 smithi167 ceph-mon[32751]: pgmap v11006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:54 smithi090 ceph-mon[29180]: pgmap v11006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-26T23:58:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:55 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19417.278912 secs 2024-04-26T23:58:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:55 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19417.278912 secs 2024-04-26T23:58:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:55 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19417.278912 secs 2024-04-26T23:58:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19418.061348 secs 2024-04-26T23:58:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:56 smithi119 ceph-mon[32752]: pgmap v11007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:58:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19418.061348 secs 2024-04-26T23:58:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:56 smithi167 ceph-mon[32751]: pgmap v11007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:58:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19418.061348 secs 2024-04-26T23:58:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:56 smithi090 ceph-mon[29180]: pgmap v11007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T23:58:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:58:58 smithi119 ceph-mon[32752]: pgmap v11008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:58:58 smithi167 ceph-mon[32751]: pgmap v11008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:58:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:58:58 smithi090 ceph-mon[29180]: pgmap v11008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:00 smithi119 ceph-mon[32752]: pgmap v11009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:00 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19422.278969 secs 2024-04-26T23:59:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:00 smithi167 ceph-mon[32751]: pgmap v11009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:00 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19422.278969 secs 2024-04-26T23:59:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:00 smithi090 ceph-mon[29180]: pgmap v11009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:00 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19422.278969 secs 2024-04-26T23:59:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19423.061459 secs 2024-04-26T23:59:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19423.061459 secs 2024-04-26T23:59:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19423.061459 secs 2024-04-26T23:59:03.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:02 smithi119 ceph-mon[32752]: pgmap v11010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:03.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:02 smithi167 ceph-mon[32751]: pgmap v11010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:02 smithi090 ceph-mon[29180]: pgmap v11010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:05.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:04 smithi119 ceph-mon[32752]: pgmap v11011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:04 smithi167 ceph-mon[32751]: pgmap v11011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:04 smithi090 ceph-mon[29180]: pgmap v11011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:06.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:05 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19427.279054 secs 2024-04-26T23:59:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:05 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19427.279054 secs 2024-04-26T23:59:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:05 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19427.279054 secs 2024-04-26T23:59:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19428.061526 secs 2024-04-26T23:59:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:06 smithi119 ceph-mon[32752]: pgmap v11012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19428.061526 secs 2024-04-26T23:59:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:06 smithi167 ceph-mon[32751]: pgmap v11012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19428.061526 secs 2024-04-26T23:59:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:06 smithi090 ceph-mon[29180]: pgmap v11012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:09.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:08 smithi119 ceph-mon[32752]: pgmap v11013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:09.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:08 smithi167 ceph-mon[32751]: pgmap v11013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:09.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:08 smithi090 ceph-mon[29180]: pgmap v11013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:10 smithi090 ceph-mon[29180]: pgmap v11014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:10 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19432.279115 secs 2024-04-26T23:59:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:10 smithi119 ceph-mon[32752]: pgmap v11014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:10 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19432.279115 secs 2024-04-26T23:59:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:10 smithi167 ceph-mon[32751]: pgmap v11014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:10 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19432.279115 secs 2024-04-26T23:59:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19433.061620 secs 2024-04-26T23:59:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19433.061620 secs 2024-04-26T23:59:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19433.061620 secs 2024-04-26T23:59:13.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:12 smithi119 ceph-mon[32752]: pgmap v11015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:12 smithi167 ceph-mon[32751]: pgmap v11015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:13.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:12 smithi090 ceph-mon[29180]: pgmap v11015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:15.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:14 smithi119 ceph-mon[32752]: pgmap v11016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:14 smithi167 ceph-mon[32751]: pgmap v11016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:14 smithi090 ceph-mon[29180]: pgmap v11016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:15 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19437.279190 secs 2024-04-26T23:59:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:15 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19437.279190 secs 2024-04-26T23:59:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:15 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19437.279190 secs 2024-04-26T23:59:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19438.061738 secs 2024-04-26T23:59:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:16 smithi119 ceph-mon[32752]: pgmap v11017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19438.061738 secs 2024-04-26T23:59:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:16 smithi167 ceph-mon[32751]: pgmap v11017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19438.061738 secs 2024-04-26T23:59:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:16 smithi090 ceph-mon[29180]: pgmap v11017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:18 smithi119 ceph-mon[32752]: pgmap v11018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:19.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:18 smithi167 ceph-mon[32751]: pgmap v11018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:18 smithi090 ceph-mon[29180]: pgmap v11018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:20 smithi119 ceph-mon[32752]: pgmap v11019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:20 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19442.279253 secs 2024-04-26T23:59:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:20 smithi167 ceph-mon[32751]: pgmap v11019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:20 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19442.279253 secs 2024-04-26T23:59:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:20 smithi090 ceph-mon[29180]: pgmap v11019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:20 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19442.279253 secs 2024-04-26T23:59:22.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19443.061812 secs 2024-04-26T23:59:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19443.061812 secs 2024-04-26T23:59:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19443.061812 secs 2024-04-26T23:59:23.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:22 smithi119 ceph-mon[32752]: pgmap v11020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:22 smithi167 ceph-mon[32751]: pgmap v11020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:22 smithi090 ceph-mon[29180]: pgmap v11020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:24 smithi119 ceph-mon[32752]: pgmap v11021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:24 smithi167 ceph-mon[32751]: pgmap v11021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:24 smithi090 ceph-mon[29180]: pgmap v11021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:26.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:25 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19447.279312 secs 2024-04-26T23:59:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:25 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19447.279312 secs 2024-04-26T23:59:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:25 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19447.279312 secs 2024-04-26T23:59:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19448.061907 secs 2024-04-26T23:59:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:26 smithi119 ceph-mon[32752]: pgmap v11022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19448.061907 secs 2024-04-26T23:59:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:26 smithi167 ceph-mon[32751]: pgmap v11022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19448.061907 secs 2024-04-26T23:59:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:26 smithi090 ceph-mon[29180]: pgmap v11022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:28 smithi119 ceph-mon[32752]: pgmap v11023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:28 smithi167 ceph-mon[32751]: pgmap v11023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:28 smithi090 ceph-mon[29180]: pgmap v11023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:30 smithi119 ceph-mon[32752]: pgmap v11024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:30 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19452.279396 secs 2024-04-26T23:59:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:30 smithi167 ceph-mon[32751]: pgmap v11024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:30 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19452.279396 secs 2024-04-26T23:59:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:30 smithi090 ceph-mon[29180]: pgmap v11024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:30 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19452.279396 secs 2024-04-26T23:59:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19453.062015 secs 2024-04-26T23:59:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19453.062015 secs 2024-04-26T23:59:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19453.062015 secs 2024-04-26T23:59:33.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:32 smithi119 ceph-mon[32752]: pgmap v11025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:32 smithi167 ceph-mon[32751]: pgmap v11025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:32 smithi090 ceph-mon[29180]: pgmap v11025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:35.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:34 smithi119 ceph-mon[32752]: pgmap v11026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:34 smithi167 ceph-mon[32751]: pgmap v11026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:35.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:34 smithi090 ceph-mon[29180]: pgmap v11026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:36 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19457.279480 secs 2024-04-26T23:59:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:36 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19457.279480 secs 2024-04-26T23:59:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:36 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19457.279480 secs 2024-04-26T23:59:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:37 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19458.062139 secs 2024-04-26T23:59:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:37 smithi119 ceph-mon[32752]: pgmap v11027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:37 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19458.062139 secs 2024-04-26T23:59:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:37 smithi167 ceph-mon[32751]: pgmap v11027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:37 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19458.062139 secs 2024-04-26T23:59:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:37 smithi090 ceph-mon[29180]: pgmap v11027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:39.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:39 smithi119 ceph-mon[32752]: pgmap v11028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:39 smithi167 ceph-mon[32751]: pgmap v11028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:39 smithi090 ceph-mon[29180]: pgmap v11028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:41 smithi119 ceph-mon[32752]: pgmap v11029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19462.279541 secs 2024-04-26T23:59:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:41 smithi167 ceph-mon[32751]: pgmap v11029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19462.279541 secs 2024-04-26T23:59:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:41 smithi090 ceph-mon[29180]: pgmap v11029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19462.279541 secs 2024-04-26T23:59:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:42 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19463.062233 secs 2024-04-26T23:59:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:42 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19463.062233 secs 2024-04-26T23:59:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:42 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19463.062233 secs 2024-04-26T23:59:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:43 smithi119 ceph-mon[32752]: pgmap v11030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:43.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:43 smithi167 ceph-mon[32751]: pgmap v11030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:43 smithi090 ceph-mon[29180]: pgmap v11030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:45.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:45 smithi119 ceph-mon[32752]: pgmap v11031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:45 smithi167 ceph-mon[32751]: pgmap v11031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:45 smithi090 ceph-mon[29180]: pgmap v11031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T23:59:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19467.279626 secs 2024-04-26T23:59:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:59:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19467.279626 secs 2024-04-26T23:59:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:59:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19467.279626 secs 2024-04-26T23:59:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T23:59:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:47 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19468.062345 secs 2024-04-26T23:59:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:47 smithi119 ceph-mon[32752]: pgmap v11032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:47 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19468.062345 secs 2024-04-26T23:59:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:47 smithi167 ceph-mon[32751]: pgmap v11032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:47 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19468.062345 secs 2024-04-26T23:59:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:47 smithi090 ceph-mon[29180]: pgmap v11032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: pgmap v11033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: pgmap v11033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: pgmap v11033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T23:59:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:49.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:59:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:59:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:59:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:59:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:59:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:59:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:59:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-26T23:59:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T23:59:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T23:59:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-26T23:59:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:51 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:59:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:51 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:59:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:51 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:59:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:51 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:59:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:51 smithi119 ceph-mon[32752]: pgmap v11034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:59:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:51 smithi119 ceph-mon[32752]: pgmap v11035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19472.279691 secs 2024-04-26T23:59:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:51 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:59:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:51 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:59:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:51 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:59:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:51 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:59:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:51 smithi167 ceph-mon[32751]: pgmap v11034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:59:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:51 smithi167 ceph-mon[32751]: pgmap v11035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19472.279691 secs 2024-04-26T23:59:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:51 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-26T23:59:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:51 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-26T23:59:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:51 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-26T23:59:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:51 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T23:59:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:51 smithi090 ceph-mon[29180]: pgmap v11034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-26T23:59:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:51 smithi090 ceph-mon[29180]: pgmap v11035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19472.279691 secs 2024-04-26T23:59:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19473.062451 secs 2024-04-26T23:59:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19473.062451 secs 2024-04-26T23:59:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19473.062451 secs 2024-04-26T23:59:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:53 smithi119 ceph-mon[32752]: pgmap v11036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:53 smithi167 ceph-mon[32751]: pgmap v11036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:53 smithi090 ceph-mon[29180]: pgmap v11036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:55 smithi119 ceph-mon[32752]: pgmap v11037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:55 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19477.279773 secs 2024-04-26T23:59:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:55 smithi167 ceph-mon[32751]: pgmap v11037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:55 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19477.279773 secs 2024-04-26T23:59:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:55 smithi090 ceph-mon[29180]: pgmap v11037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:55 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19477.279773 secs 2024-04-26T23:59:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:57 smithi119 ceph-mon[32752]: pgmap v11038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19478.062524 secs 2024-04-26T23:59:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:57 smithi167 ceph-mon[32751]: pgmap v11038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19478.062524 secs 2024-04-26T23:59:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:57 smithi090 ceph-mon[29180]: pgmap v11038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19478.062524 secs 2024-04-26T23:59:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 26 23:59:59 smithi119 ceph-mon[32752]: pgmap v11039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 26 23:59:59 smithi167 ceph-mon[32751]: pgmap v11039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-26T23:59:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 26 23:59:59 smithi090 ceph-mon[29180]: pgmap v11039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:00:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:00:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:00 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19482.279824 secs 2024-04-27T00:00:00.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:00:00.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:00 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19482.279824 secs 2024-04-27T00:00:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:00:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:00 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19482.279824 secs 2024-04-27T00:00:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:01 smithi167 ceph-mon[32751]: pgmap v11040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:00:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:01 smithi090 ceph-mon[29180]: pgmap v11040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:00:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:01 smithi119 ceph-mon[32752]: pgmap v11040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:00:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:02 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19483.062606 secs 2024-04-27T00:00:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:02 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19483.062606 secs 2024-04-27T00:00:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:02 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19483.062606 secs 2024-04-27T00:00:03.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:03 smithi167 ceph-mon[32751]: pgmap v11041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:03 smithi090 ceph-mon[29180]: pgmap v11041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:03 smithi119 ceph-mon[32752]: pgmap v11041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:05.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:05 smithi167 ceph-mon[32751]: pgmap v11042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:05 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 19487.279886 secs 2024-04-27T00:00:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:05 smithi090 ceph-mon[29180]: pgmap v11042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:05 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 19487.279886 secs 2024-04-27T00:00:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:05 smithi119 ceph-mon[32752]: pgmap v11042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:05 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 19487.279886 secs 2024-04-27T00:00:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:07 smithi090 ceph-mon[29180]: pgmap v11043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:07 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19488.062681 secs 2024-04-27T00:00:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:07 smithi119 ceph-mon[32752]: pgmap v11043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:07 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19488.062681 secs 2024-04-27T00:00:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:07 smithi167 ceph-mon[32751]: pgmap v11043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:07 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19488.062681 secs 2024-04-27T00:00:09.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:09 smithi167 ceph-mon[32751]: pgmap v11044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:09.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:09 smithi090 ceph-mon[29180]: pgmap v11044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:09.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:09 smithi119 ceph-mon[32752]: pgmap v11044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:10.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:10 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19492.279946 secs 2024-04-27T00:00:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:10 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19492.279946 secs 2024-04-27T00:00:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:10 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19492.279946 secs 2024-04-27T00:00:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:11 smithi090 ceph-mon[29180]: pgmap v11045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:11 smithi119 ceph-mon[32752]: pgmap v11045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:11 smithi167 ceph-mon[32751]: pgmap v11045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:12 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19493.062813 secs 2024-04-27T00:00:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:12 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19493.062813 secs 2024-04-27T00:00:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:12 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19493.062813 secs 2024-04-27T00:00:13.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:13 smithi090 ceph-mon[29180]: pgmap v11046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:13 smithi119 ceph-mon[32752]: pgmap v11046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:13 smithi167 ceph-mon[32751]: pgmap v11046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:15.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:15 smithi119 ceph-mon[32752]: pgmap v11047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:15.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:00:15.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:00:15.358 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:15 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19497.280004 secs 2024-04-27T00:00:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:15 smithi090 ceph-mon[29180]: pgmap v11047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:00:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:00:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:15 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19497.280004 secs 2024-04-27T00:00:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:15 smithi167 ceph-mon[32751]: pgmap v11047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:00:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:00:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:15 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19497.280004 secs 2024-04-27T00:00:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:17 smithi090 ceph-mon[29180]: pgmap v11048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:17 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19498.062880 secs 2024-04-27T00:00:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:17 smithi119 ceph-mon[32752]: pgmap v11048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:17 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19498.062880 secs 2024-04-27T00:00:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:17 smithi167 ceph-mon[32751]: pgmap v11048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:17 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19498.062880 secs 2024-04-27T00:00:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:19 smithi090 ceph-mon[29180]: pgmap v11049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:19.402 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:19 smithi167 ceph-mon[32751]: pgmap v11049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:19 smithi119 ceph-mon[32752]: pgmap v11049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:20 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19502.280065 secs 2024-04-27T00:00:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:20 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19502.280065 secs 2024-04-27T00:00:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:20 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19502.280065 secs 2024-04-27T00:00:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:21 smithi090 ceph-mon[29180]: pgmap v11050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:21 smithi119 ceph-mon[32752]: pgmap v11050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:21 smithi167 ceph-mon[32751]: pgmap v11050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:00:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:22 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19503.063016 secs 2024-04-27T00:00:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:22 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19503.063016 secs 2024-04-27T00:00:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:22 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19503.063016 secs 2024-04-27T00:00:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:23 smithi090 ceph-mon[29180]: pgmap v11051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:23 smithi119 ceph-mon[32752]: pgmap v11051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:23 smithi167 ceph-mon[32751]: pgmap v11051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:24.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:24 smithi090 ceph-mon[29180]: pgmap v11052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:24.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:24 smithi119 ceph-mon[32752]: pgmap v11052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:24 smithi167 ceph-mon[32751]: pgmap v11052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:25 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19507.280124 secs 2024-04-27T00:00:25.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:25 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19507.280124 secs 2024-04-27T00:00:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:25 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19507.280124 secs 2024-04-27T00:00:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:26 smithi090 ceph-mon[29180]: pgmap v11053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:26 smithi119 ceph-mon[32752]: pgmap v11053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:26 smithi167 ceph-mon[32751]: pgmap v11053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:27 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19508.063131 secs 2024-04-27T00:00:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:27 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19508.063131 secs 2024-04-27T00:00:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:27 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19508.063131 secs 2024-04-27T00:00:28.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:28 smithi119 ceph-mon[32752]: pgmap v11054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:28.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:28 smithi167 ceph-mon[32751]: pgmap v11054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:28.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:28 smithi090 ceph-mon[29180]: pgmap v11054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:30 smithi119 ceph-mon[32752]: pgmap v11055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:30 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19512.280208 secs 2024-04-27T00:00:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:30 smithi167 ceph-mon[32751]: pgmap v11055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:30 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19512.280208 secs 2024-04-27T00:00:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:30 smithi090 ceph-mon[29180]: pgmap v11055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:30 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19512.280208 secs 2024-04-27T00:00:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19513.063195 secs 2024-04-27T00:00:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19513.063195 secs 2024-04-27T00:00:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19513.063195 secs 2024-04-27T00:00:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:32 smithi119 ceph-mon[32752]: pgmap v11056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:32 smithi167 ceph-mon[32751]: pgmap v11056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:32 smithi090 ceph-mon[29180]: pgmap v11056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:34 smithi119 ceph-mon[32752]: pgmap v11057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:34 smithi167 ceph-mon[32751]: pgmap v11057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:34 smithi090 ceph-mon[29180]: pgmap v11057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:00:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:35 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19517.280295 secs 2024-04-27T00:00:35.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:35 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19517.280295 secs 2024-04-27T00:00:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:35 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19517.280295 secs 2024-04-27T00:00:36.526 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:00:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:00:36.172+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:00:36.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:00:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:00:36.172+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:00:36.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:00:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:00:36.172+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:00:36.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:00:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:00:36.172+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:00:36.527 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:00:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:00:36.172+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:00:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:36 smithi119 ceph-mon[32752]: pgmap v11058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19518.063273 secs 2024-04-27T00:00:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:36 smithi167 ceph-mon[32751]: pgmap v11058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19518.063273 secs 2024-04-27T00:00:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:36 smithi090 ceph-mon[29180]: pgmap v11058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19518.063273 secs 2024-04-27T00:00:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:38 smithi119 ceph-mon[32752]: pgmap v11059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:38 smithi167 ceph-mon[32751]: pgmap v11059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:38 smithi090 ceph-mon[29180]: pgmap v11059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:00:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:40 smithi119 ceph-mon[32752]: pgmap v11060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:40 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19522.280382 secs 2024-04-27T00:00:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:40 smithi167 ceph-mon[32751]: pgmap v11060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:40 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19522.280382 secs 2024-04-27T00:00:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:40 smithi090 ceph-mon[29180]: pgmap v11060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:40 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19522.280382 secs 2024-04-27T00:00:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19523.063409 secs 2024-04-27T00:00:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19523.063409 secs 2024-04-27T00:00:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19523.063409 secs 2024-04-27T00:00:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:43 smithi090 ceph-mon[29180]: pgmap v11061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:43 smithi119 ceph-mon[32752]: pgmap v11061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:43 smithi167 ceph-mon[32751]: pgmap v11061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:45.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:45 smithi119 ceph-mon[32752]: pgmap v11062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:45.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:45 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19527.280465 secs 2024-04-27T00:00:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:45 smithi090 ceph-mon[29180]: pgmap v11062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:45 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19527.280465 secs 2024-04-27T00:00:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:45 smithi167 ceph-mon[32751]: pgmap v11062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:45 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19527.280465 secs 2024-04-27T00:00:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:47 smithi090 ceph-mon[29180]: pgmap v11063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:47 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19528.063529 secs 2024-04-27T00:00:47.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:47 smithi119 ceph-mon[32752]: pgmap v11063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:47.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:47 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19528.063529 secs 2024-04-27T00:00:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:47 smithi167 ceph-mon[32751]: pgmap v11063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:47 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19528.063529 secs 2024-04-27T00:00:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:49 smithi090 ceph-mon[29180]: pgmap v11064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:49.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:49 smithi119 ceph-mon[32752]: pgmap v11064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:49 smithi167 ceph-mon[32751]: pgmap v11064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:00:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:50 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19532.280553 secs 2024-04-27T00:00:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:00:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:50 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19532.280553 secs 2024-04-27T00:00:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:00:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:50 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19532.280553 secs 2024-04-27T00:00:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:51 smithi167 ceph-mon[32751]: pgmap v11065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:00:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:00:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:51 smithi090 ceph-mon[29180]: pgmap v11065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:00:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:00:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:51 smithi119 ceph-mon[32752]: pgmap v11065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:00:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:00:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:00:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:00:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:52 smithi090 ceph-mon[29180]: pgmap v11066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 904 B/s rd, 1 op/s 2024-04-27T00:00:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:52 smithi090 ceph-mon[29180]: pgmap v11067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:52 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19533.063598 secs 2024-04-27T00:00:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:52 smithi119 ceph-mon[32752]: pgmap v11066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 904 B/s rd, 1 op/s 2024-04-27T00:00:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:52 smithi119 ceph-mon[32752]: pgmap v11067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:52 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19533.063598 secs 2024-04-27T00:00:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:52 smithi167 ceph-mon[32751]: pgmap v11066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 904 B/s rd, 1 op/s 2024-04-27T00:00:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:52 smithi167 ceph-mon[32751]: pgmap v11067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:52 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19533.063598 secs 2024-04-27T00:00:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:54 smithi090 ceph-mon[29180]: pgmap v11068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-27T00:00:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:54 smithi119 ceph-mon[32752]: pgmap v11068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-27T00:00:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:54 smithi167 ceph-mon[32751]: pgmap v11068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-27T00:00:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:55 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19537.280639 secs 2024-04-27T00:00:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:55 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19537.280639 secs 2024-04-27T00:00:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:55 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19537.280639 secs 2024-04-27T00:00:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:56 smithi090 ceph-mon[29180]: pgmap v11069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:56 smithi119 ceph-mon[32752]: pgmap v11069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:56 smithi167 ceph-mon[32751]: pgmap v11069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19538.063735 secs 2024-04-27T00:00:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:57 smithi119 ceph-mon[32752]: pgmap v11070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19538.063735 secs 2024-04-27T00:00:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:57 smithi090 ceph-mon[29180]: pgmap v11070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:00:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19538.063735 secs 2024-04-27T00:00:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:57 smithi167 ceph-mon[32751]: pgmap v11070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:00:59 smithi090 ceph-mon[29180]: pgmap v11071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:00:59 smithi119 ceph-mon[32752]: pgmap v11071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:00:59 smithi167 ceph-mon[32751]: pgmap v11071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:00 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19542.280725 secs 2024-04-27T00:01:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:00 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19542.280725 secs 2024-04-27T00:01:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:00 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19542.280725 secs 2024-04-27T00:01:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:01 smithi119 ceph-mon[32752]: pgmap v11072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:01:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19543.063804 secs 2024-04-27T00:01:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:01 smithi167 ceph-mon[32751]: pgmap v11072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:01:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19543.063804 secs 2024-04-27T00:01:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:01 smithi090 ceph-mon[29180]: pgmap v11072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:01:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19543.063804 secs 2024-04-27T00:01:04.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:03 smithi090 ceph-mon[29180]: pgmap v11073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:04.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:03 smithi119 ceph-mon[32752]: pgmap v11073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:03 smithi167 ceph-mon[32751]: pgmap v11073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:05 smithi090 ceph-mon[29180]: pgmap v11074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:05 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19547.280815 secs 2024-04-27T00:01:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:05 smithi119 ceph-mon[32752]: pgmap v11074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:05 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19547.280815 secs 2024-04-27T00:01:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:05 smithi167 ceph-mon[32751]: pgmap v11074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:05 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19547.280815 secs 2024-04-27T00:01:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19548.063880 secs 2024-04-27T00:01:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19548.063880 secs 2024-04-27T00:01:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19548.063880 secs 2024-04-27T00:01:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:07 smithi090 ceph-mon[29180]: pgmap v11075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:08.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:07 smithi167 ceph-mon[32751]: pgmap v11075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:07 smithi119 ceph-mon[32752]: pgmap v11075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:09 smithi090 ceph-mon[29180]: pgmap v11076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:09 smithi119 ceph-mon[32752]: pgmap v11076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:09 smithi167 ceph-mon[32751]: pgmap v11076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:10 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 19552.280906 secs 2024-04-27T00:01:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:10 smithi090 ceph-mon[29180]: slow request 2884.719534 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:01:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:10 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 19552.280906 secs 2024-04-27T00:01:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:10 smithi119 ceph-mon[32752]: slow request 2884.719534 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:01:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:10 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 19552.280906 secs 2024-04-27T00:01:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:10 smithi167 ceph-mon[32751]: slow request 2884.719534 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:01:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:11 smithi090 ceph-mon[29180]: pgmap v11077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19553.063950 secs 2024-04-27T00:01:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:11 smithi119 ceph-mon[32752]: pgmap v11077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19553.063950 secs 2024-04-27T00:01:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:11 smithi167 ceph-mon[32751]: pgmap v11077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19553.063950 secs 2024-04-27T00:01:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:13 smithi090 ceph-mon[29180]: pgmap v11078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:14.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:13 smithi119 ceph-mon[32752]: pgmap v11078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:13 smithi167 ceph-mon[32751]: pgmap v11078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:01:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:15 smithi090 ceph-mon[29180]: pgmap v11079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:15 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19557.280969 secs 2024-04-27T00:01:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:15 smithi119 ceph-mon[32752]: pgmap v11079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:15 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19557.280969 secs 2024-04-27T00:01:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:15 smithi167 ceph-mon[32751]: pgmap v11079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:01:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:15 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19557.280969 secs 2024-04-27T00:01:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19558.064031 secs 2024-04-27T00:01:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19558.064031 secs 2024-04-27T00:01:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19558.064031 secs 2024-04-27T00:01:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:17 smithi090 ceph-mon[29180]: pgmap v11080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:17 smithi119 ceph-mon[32752]: pgmap v11080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:17 smithi167 ceph-mon[32751]: pgmap v11080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:19 smithi119 ceph-mon[32752]: pgmap v11081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:19 smithi167 ceph-mon[32751]: pgmap v11081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:19 smithi090 ceph-mon[29180]: pgmap v11081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:20 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19562.281057 secs 2024-04-27T00:01:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:20 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19562.281057 secs 2024-04-27T00:01:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:20 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19562.281057 secs 2024-04-27T00:01:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:21 smithi119 ceph-mon[32752]: pgmap v11082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:22.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19563.064162 secs 2024-04-27T00:01:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:21 smithi167 ceph-mon[32751]: pgmap v11082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19563.064162 secs 2024-04-27T00:01:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:21 smithi090 ceph-mon[29180]: pgmap v11082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19563.064162 secs 2024-04-27T00:01:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:23 smithi119 ceph-mon[32752]: pgmap v11083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:23 smithi167 ceph-mon[32751]: pgmap v11083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:23 smithi090 ceph-mon[29180]: pgmap v11083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:25 smithi119 ceph-mon[32752]: pgmap v11084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:25 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19567.281142 secs 2024-04-27T00:01:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:25 smithi167 ceph-mon[32751]: pgmap v11084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:25 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19567.281142 secs 2024-04-27T00:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:25 smithi090 ceph-mon[29180]: pgmap v11084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:25 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19567.281142 secs 2024-04-27T00:01:27.197 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19568.064273 secs 2024-04-27T00:01:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19568.064273 secs 2024-04-27T00:01:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19568.064273 secs 2024-04-27T00:01:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:27 smithi119 ceph-mon[32752]: pgmap v11085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:27 smithi167 ceph-mon[32751]: pgmap v11085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:27 smithi090 ceph-mon[29180]: pgmap v11085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:30.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:29 smithi119 ceph-mon[32752]: pgmap v11086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:29 smithi167 ceph-mon[32751]: pgmap v11086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:29 smithi090 ceph-mon[29180]: pgmap v11086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:30 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19572.281223 secs 2024-04-27T00:01:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:30 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19572.281223 secs 2024-04-27T00:01:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:30 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19572.281223 secs 2024-04-27T00:01:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:31 smithi119 ceph-mon[32752]: pgmap v11087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19573.064403 secs 2024-04-27T00:01:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:31 smithi167 ceph-mon[32751]: pgmap v11087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19573.064403 secs 2024-04-27T00:01:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:31 smithi090 ceph-mon[29180]: pgmap v11087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19573.064403 secs 2024-04-27T00:01:34.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:33 smithi119 ceph-mon[32752]: pgmap v11088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:33 smithi167 ceph-mon[32751]: pgmap v11088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:33 smithi090 ceph-mon[29180]: pgmap v11088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:35 smithi119 ceph-mon[32752]: pgmap v11089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:35 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19577.281305 secs 2024-04-27T00:01:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:35 smithi167 ceph-mon[32751]: pgmap v11089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:35 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19577.281305 secs 2024-04-27T00:01:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:35 smithi090 ceph-mon[29180]: pgmap v11089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:35 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19577.281305 secs 2024-04-27T00:01:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19578.064467 secs 2024-04-27T00:01:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19578.064467 secs 2024-04-27T00:01:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19578.064467 secs 2024-04-27T00:01:38.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:37 smithi167 ceph-mon[32751]: pgmap v11090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:37 smithi119 ceph-mon[32752]: pgmap v11090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:37 smithi090 ceph-mon[29180]: pgmap v11090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:39 smithi119 ceph-mon[32752]: pgmap v11091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:39 smithi167 ceph-mon[32751]: pgmap v11091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:39 smithi090 ceph-mon[29180]: pgmap v11091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:40 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19582.281392 secs 2024-04-27T00:01:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:40 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19582.281392 secs 2024-04-27T00:01:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:40 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19582.281392 secs 2024-04-27T00:01:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:41 smithi119 ceph-mon[32752]: pgmap v11092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19583.064588 secs 2024-04-27T00:01:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:41 smithi167 ceph-mon[32751]: pgmap v11092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19583.064588 secs 2024-04-27T00:01:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:41 smithi090 ceph-mon[29180]: pgmap v11092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19583.064588 secs 2024-04-27T00:01:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:43 smithi119 ceph-mon[32752]: pgmap v11093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:43 smithi167 ceph-mon[32751]: pgmap v11093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:44.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:43 smithi090 ceph-mon[29180]: pgmap v11093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:45 smithi119 ceph-mon[32752]: pgmap v11094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:45 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 19587.281469 secs 2024-04-27T00:01:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:45 smithi119 ceph-mon[32752]: slow request 2880.050801 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:01:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:45 smithi167 ceph-mon[32751]: pgmap v11094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:45 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 19587.281469 secs 2024-04-27T00:01:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:45 smithi167 ceph-mon[32751]: slow request 2880.050801 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:01:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:45 smithi090 ceph-mon[29180]: pgmap v11094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:45 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 19587.281469 secs 2024-04-27T00:01:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:45 smithi090 ceph-mon[29180]: slow request 2880.050801 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:01:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19588.064699 secs 2024-04-27T00:01:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19588.064699 secs 2024-04-27T00:01:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19588.064699 secs 2024-04-27T00:01:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:47 smithi119 ceph-mon[32752]: pgmap v11095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:47 smithi167 ceph-mon[32751]: pgmap v11095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:47 smithi090 ceph-mon[29180]: pgmap v11095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:49 smithi119 ceph-mon[32752]: pgmap v11096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:49 smithi167 ceph-mon[32751]: pgmap v11096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:49 smithi090 ceph-mon[29180]: pgmap v11096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:01:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:50 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19592.281556 secs 2024-04-27T00:01:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:01:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:50 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19592.281556 secs 2024-04-27T00:01:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:01:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:50 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19592.281556 secs 2024-04-27T00:01:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:01:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:51 smithi119 ceph-mon[32752]: pgmap v11097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19593.064775 secs 2024-04-27T00:01:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:01:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:01:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:01:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:51 smithi167 ceph-mon[32751]: pgmap v11097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19593.064775 secs 2024-04-27T00:01:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:01:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:01:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:01:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:51 smithi090 ceph-mon[29180]: pgmap v11097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:01:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19593.064775 secs 2024-04-27T00:01:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:01:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:01:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:01:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:53 smithi090 ceph-mon[29180]: pgmap v11098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-27T00:01:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:53 smithi090 ceph-mon[29180]: pgmap v11099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:53 smithi119 ceph-mon[32752]: pgmap v11098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-27T00:01:53.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:53 smithi119 ceph-mon[32752]: pgmap v11099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:53 smithi167 ceph-mon[32751]: pgmap v11098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-27T00:01:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:53 smithi167 ceph-mon[32751]: pgmap v11099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:55 smithi090 ceph-mon[29180]: pgmap v11100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:55 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19597.281635 secs 2024-04-27T00:01:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:55 smithi119 ceph-mon[32752]: pgmap v11100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:55 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19597.281635 secs 2024-04-27T00:01:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:55 smithi167 ceph-mon[32751]: pgmap v11100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:55 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19597.281635 secs 2024-04-27T00:01:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:57 smithi090 ceph-mon[29180]: pgmap v11101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:57 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19598.064906 secs 2024-04-27T00:01:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:57 smithi119 ceph-mon[32752]: pgmap v11101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:57 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19598.064906 secs 2024-04-27T00:01:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:57 smithi167 ceph-mon[32751]: pgmap v11101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:01:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:57 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19598.064906 secs 2024-04-27T00:01:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:01:59 smithi090 ceph-mon[29180]: pgmap v11102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T00:01:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:01:59 smithi119 ceph-mon[32752]: pgmap v11102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T00:01:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:01:59 smithi167 ceph-mon[32751]: pgmap v11102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T00:01:59.877 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192182]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:01:59.877 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192182]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:01:59.877 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192182]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:01:59.878 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192182]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:01:59.878 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192185]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:01:59.878 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192185]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:01:59.878 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192185]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:01:59.878 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192185]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:01:59.979 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159216]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:01:59.979 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159216]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:01:59.979 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159216]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:01:59.979 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159216]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:01:59.979 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159219]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:01:59.980 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159219]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:01:59.980 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159219]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:01:59.980 INFO:journalctl@ceph.osd.1.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159219]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.058 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160609]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.058 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160609]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.058 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160609]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.058 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160609]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.059 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160612]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.059 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160612]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.059 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160612]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.059 INFO:journalctl@ceph.osd.2.smithi167.stdout:Apr 27 00:01:59 smithi167 sudo[160612]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.153 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.153 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.153 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192188]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.153 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192188]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.154 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192191]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.154 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192191]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.154 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192191]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.154 INFO:journalctl@ceph.osd.3.smithi090.stdout:Apr 27 00:01:59 smithi090 sudo[192191]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.242 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159222]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.242 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159222]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.243 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159222]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.243 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:01:59 smithi119 sudo[159222]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.243 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159225]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.243 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159225]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.243 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159225]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.243 INFO:journalctl@ceph.osd.4.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159225]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.318 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160615]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.319 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160615]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.319 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160615]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.319 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160615]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.319 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160618]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.319 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160618]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.319 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160618]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.319 INFO:journalctl@ceph.osd.5.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160618]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.423 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192194]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.423 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192194]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.423 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192194]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.423 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192194]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.424 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192197]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.424 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192197]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.424 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192197]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.424 INFO:journalctl@ceph.osd.6.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192197]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.501 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159228]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.502 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159228]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.502 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159228]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.502 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159228]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.502 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159231]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.502 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159231]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.502 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159231]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.502 INFO:journalctl@ceph.osd.7.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159231]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.594 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160627]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.594 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160627]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.595 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160627]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.595 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160627]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.595 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160621]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.595 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160621]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.595 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160621]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.595 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160621]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.596 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160624]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.596 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160624]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.596 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160624]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.596 INFO:journalctl@ceph.osd.8.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160624]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 ceph-mon[29180]: pgmap v11103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:00.689 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:00.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:00.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:00.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19602.281730 secs 2024-04-27T00:02:00.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.690 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192206]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192206]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.691 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192200]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.691 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192200]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.691 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192200]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.691 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192200]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.691 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192203]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.692 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192203]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.692 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192203]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.692 INFO:journalctl@ceph.osd.9.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192203]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.768 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 ceph-mon[32752]: pgmap v11103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:00.768 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:00.769 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:00.769 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:00.769 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19602.281730 secs 2024-04-27T00:02:00.769 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159234]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:00.769 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159234]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.769 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159234]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.770 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159234]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:00.770 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159237]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:00.770 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159237]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:00.770 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159237]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:00.770 INFO:journalctl@ceph.osd.10.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159237]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159240]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159240]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159240]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159240]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159243]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159243]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159243]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:00 smithi119 sudo[159243]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:01.068 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160630]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:01.068 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160630]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:01.068 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160630]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:01.069 INFO:journalctl@ceph.osd.11.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160630]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 ceph-mon[32751]: pgmap v11103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19602.281730 secs 2024-04-27T00:02:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160633]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-27T00:02:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160633]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160633]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160633]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160636]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160636]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:00 smithi167 sudo[160636]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192209]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-27T00:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192209]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-27T00:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192209]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-27T00:02:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:00 smithi090 sudo[192209]: pam_unix(sudo:session): session closed for user root 2024-04-27T00:02:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19603.065000 secs 2024-04-27T00:02:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19603.065000 secs 2024-04-27T00:02:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-27T00:02:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19603.065000 secs 2024-04-27T00:02:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:02 smithi090 ceph-mon[29180]: pgmap v11104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:02:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:02 smithi119 ceph-mon[32752]: pgmap v11104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:02:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:02 smithi167 ceph-mon[32751]: pgmap v11104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:02:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:04 smithi090 ceph-mon[29180]: pgmap v11105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:04 smithi119 ceph-mon[32752]: pgmap v11105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:04 smithi167 ceph-mon[32751]: pgmap v11105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:05 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19607.281797 secs 2024-04-27T00:02:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:05 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19607.281797 secs 2024-04-27T00:02:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:05 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19607.281797 secs 2024-04-27T00:02:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:06 smithi090 ceph-mon[29180]: pgmap v11106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19608.065119 secs 2024-04-27T00:02:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:06 smithi119 ceph-mon[32752]: pgmap v11106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19608.065119 secs 2024-04-27T00:02:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:06 smithi167 ceph-mon[32751]: pgmap v11106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19608.065119 secs 2024-04-27T00:02:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:08 smithi119 ceph-mon[32752]: pgmap v11107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:08 smithi167 ceph-mon[32751]: pgmap v11107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:08 smithi090 ceph-mon[29180]: pgmap v11107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:10 smithi119 ceph-mon[32752]: pgmap v11108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:10 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 19612.281884 secs 2024-04-27T00:02:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:10 smithi119 ceph-mon[32752]: slow request 724.043633 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:10 smithi167 ceph-mon[32751]: pgmap v11108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:10 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 19612.281884 secs 2024-04-27T00:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:10 smithi167 ceph-mon[32751]: slow request 724.043633 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:02:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:10 smithi090 ceph-mon[29180]: pgmap v11108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:10 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 19612.281884 secs 2024-04-27T00:02:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:10 smithi090 ceph-mon[29180]: slow request 724.043633 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:02:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19613.065237 secs 2024-04-27T00:02:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19613.065237 secs 2024-04-27T00:02:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19613.065237 secs 2024-04-27T00:02:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:12 smithi119 ceph-mon[32752]: pgmap v11109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:12 smithi167 ceph-mon[32751]: pgmap v11109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:12 smithi090 ceph-mon[29180]: pgmap v11109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 12 op/s 2024-04-27T00:02:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:14 smithi119 ceph-mon[32752]: pgmap v11110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:14 smithi167 ceph-mon[32751]: pgmap v11110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:14 smithi090 ceph-mon[29180]: pgmap v11110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 120 KiB/s wr, 13 op/s 2024-04-27T00:02:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:15 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19617.281970 secs 2024-04-27T00:02:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:15 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19617.281970 secs 2024-04-27T00:02:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:15 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19617.281970 secs 2024-04-27T00:02:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:16 smithi119 ceph-mon[32752]: pgmap v11111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:16.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19618.065366 secs 2024-04-27T00:02:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:16 smithi167 ceph-mon[32751]: pgmap v11111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19618.065366 secs 2024-04-27T00:02:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:16 smithi090 ceph-mon[29180]: pgmap v11111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19618.065366 secs 2024-04-27T00:02:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:18 smithi119 ceph-mon[32752]: pgmap v11112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:18 smithi167 ceph-mon[32751]: pgmap v11112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:18 smithi090 ceph-mon[29180]: pgmap v11112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:20 smithi119 ceph-mon[32752]: pgmap v11113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:20 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19622.282048 secs 2024-04-27T00:02:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:20 smithi167 ceph-mon[32751]: pgmap v11113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:20 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19622.282048 secs 2024-04-27T00:02:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:20 smithi090 ceph-mon[29180]: pgmap v11113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:20 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19622.282048 secs 2024-04-27T00:02:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19623.065486 secs 2024-04-27T00:02:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19623.065486 secs 2024-04-27T00:02:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19623.065486 secs 2024-04-27T00:02:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:22 smithi119 ceph-mon[32752]: pgmap v11114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:22 smithi167 ceph-mon[32751]: pgmap v11114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:22 smithi090 ceph-mon[29180]: pgmap v11114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:25.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:24 smithi119 ceph-mon[32752]: pgmap v11115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:25.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:24 smithi167 ceph-mon[32751]: pgmap v11115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:24 smithi090 ceph-mon[29180]: pgmap v11115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:25 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19627.282131 secs 2024-04-27T00:02:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:25 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19627.282131 secs 2024-04-27T00:02:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:25 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19627.282131 secs 2024-04-27T00:02:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:26 smithi090 ceph-mon[29180]: pgmap v11116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19628.065584 secs 2024-04-27T00:02:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:26 smithi119 ceph-mon[32752]: pgmap v11116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19628.065584 secs 2024-04-27T00:02:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:26 smithi167 ceph-mon[32751]: pgmap v11116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19628.065584 secs 2024-04-27T00:02:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:28 smithi119 ceph-mon[32752]: pgmap v11117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:29.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:28 smithi167 ceph-mon[32751]: pgmap v11117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:28 smithi090 ceph-mon[29180]: pgmap v11117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:30 smithi119 ceph-mon[32752]: pgmap v11118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:30 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19632.282216 secs 2024-04-27T00:02:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:30 smithi167 ceph-mon[32751]: pgmap v11118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:30 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19632.282216 secs 2024-04-27T00:02:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:30 smithi090 ceph-mon[29180]: pgmap v11118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:30 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19632.282216 secs 2024-04-27T00:02:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19633.065713 secs 2024-04-27T00:02:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19633.065713 secs 2024-04-27T00:02:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19633.065713 secs 2024-04-27T00:02:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:32 smithi119 ceph-mon[32752]: pgmap v11119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:32 smithi167 ceph-mon[32751]: pgmap v11119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:32 smithi090 ceph-mon[29180]: pgmap v11119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:34 smithi119 ceph-mon[32752]: pgmap v11120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:34 smithi167 ceph-mon[32751]: pgmap v11120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:35.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:34 smithi090 ceph-mon[29180]: pgmap v11120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:35 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19637.282299 secs 2024-04-27T00:02:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:35 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19637.282299 secs 2024-04-27T00:02:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:35 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19637.282299 secs 2024-04-27T00:02:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:36 smithi119 ceph-mon[32752]: pgmap v11121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19638.065808 secs 2024-04-27T00:02:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:36 smithi167 ceph-mon[32751]: pgmap v11121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19638.065808 secs 2024-04-27T00:02:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:36 smithi090 ceph-mon[29180]: pgmap v11121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19638.065808 secs 2024-04-27T00:02:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:38 smithi119 ceph-mon[32752]: pgmap v11122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:38 smithi167 ceph-mon[32751]: pgmap v11122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:39.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:38 smithi090 ceph-mon[29180]: pgmap v11122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:40 smithi119 ceph-mon[32752]: pgmap v11123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:40 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19642.282384 secs 2024-04-27T00:02:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:40 smithi167 ceph-mon[32751]: pgmap v11123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:40 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19642.282384 secs 2024-04-27T00:02:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:40 smithi090 ceph-mon[29180]: pgmap v11123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:40 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19642.282384 secs 2024-04-27T00:02:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:41 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19643.065892 secs 2024-04-27T00:02:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:41 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19643.065892 secs 2024-04-27T00:02:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:41 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19643.065892 secs 2024-04-27T00:02:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:42 smithi119 ceph-mon[32752]: pgmap v11124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:42 smithi167 ceph-mon[32751]: pgmap v11124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:43.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:42 smithi090 ceph-mon[29180]: pgmap v11124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:44 smithi119 ceph-mon[32752]: pgmap v11125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:44 smithi167 ceph-mon[32751]: pgmap v11125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:44 smithi090 ceph-mon[29180]: pgmap v11125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:45 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19647.282465 secs 2024-04-27T00:02:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:45 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19647.282465 secs 2024-04-27T00:02:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:45 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19647.282465 secs 2024-04-27T00:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:46 smithi119 ceph-mon[32752]: pgmap v11126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:46 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19648.066030 secs 2024-04-27T00:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:46 smithi167 ceph-mon[32751]: pgmap v11126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:46 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19648.066030 secs 2024-04-27T00:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:46 smithi090 ceph-mon[29180]: pgmap v11126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:46 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19648.066030 secs 2024-04-27T00:02:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:48 smithi119 ceph-mon[32752]: pgmap v11127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:48 smithi167 ceph-mon[32751]: pgmap v11127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:48 smithi090 ceph-mon[29180]: pgmap v11127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:50 smithi119 ceph-mon[32752]: pgmap v11128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:50 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19652.282545 secs 2024-04-27T00:02:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:50 smithi167 ceph-mon[32751]: pgmap v11128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:50 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19652.282545 secs 2024-04-27T00:02:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:50 smithi090 ceph-mon[29180]: pgmap v11128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:02:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:50 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19652.282545 secs 2024-04-27T00:02:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:51 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19653.066128 secs 2024-04-27T00:02:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:02:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:51 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19653.066128 secs 2024-04-27T00:02:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:02:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:51 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19653.066128 secs 2024-04-27T00:02:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:02:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:52 smithi119 ceph-mon[32752]: pgmap v11129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:02:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:02:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:52 smithi167 ceph-mon[32751]: pgmap v11129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:02:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:02:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:52 smithi090 ceph-mon[29180]: pgmap v11129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:02:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:02:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:02:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:02:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:53 smithi119 ceph-mon[32752]: pgmap v11130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:02:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:53 smithi119 ceph-mon[32752]: pgmap v11131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:02:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:53 smithi167 ceph-mon[32751]: pgmap v11130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:02:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:53 smithi167 ceph-mon[32751]: pgmap v11131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:02:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:53 smithi090 ceph-mon[29180]: pgmap v11130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:02:54.176 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:53 smithi090 ceph-mon[29180]: pgmap v11131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:02:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:55 smithi119 ceph-mon[32752]: pgmap v11132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:55 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19657.282619 secs 2024-04-27T00:02:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:55 smithi167 ceph-mon[32751]: pgmap v11132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:55 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19657.282619 secs 2024-04-27T00:02:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:55 smithi090 ceph-mon[29180]: pgmap v11132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:55 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19657.282619 secs 2024-04-27T00:02:57.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:56 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19658.066265 secs 2024-04-27T00:02:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:56 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19658.066265 secs 2024-04-27T00:02:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:56 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19658.066265 secs 2024-04-27T00:02:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:57 smithi119 ceph-mon[32752]: pgmap v11133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:57 smithi167 ceph-mon[32751]: pgmap v11133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:02:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:57 smithi090 ceph-mon[29180]: pgmap v11133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:02:59 smithi119 ceph-mon[32752]: pgmap v11134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:02:59 smithi167 ceph-mon[32751]: pgmap v11134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:02:59 smithi090 ceph-mon[29180]: pgmap v11134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:00 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19662.282703 secs 2024-04-27T00:03:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:00 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19662.282703 secs 2024-04-27T00:03:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:00 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19662.282703 secs 2024-04-27T00:03:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:01 smithi119 ceph-mon[32752]: pgmap v11135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:01 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19663.066414 secs 2024-04-27T00:03:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:01 smithi167 ceph-mon[32751]: pgmap v11135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:01 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19663.066414 secs 2024-04-27T00:03:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:01 smithi090 ceph-mon[29180]: pgmap v11135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:01 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19663.066414 secs 2024-04-27T00:03:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:03 smithi119 ceph-mon[32752]: pgmap v11136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:03:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:03 smithi167 ceph-mon[32751]: pgmap v11136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:03:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:03 smithi090 ceph-mon[29180]: pgmap v11136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:03:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:05 smithi119 ceph-mon[32752]: pgmap v11137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:05 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 19667.282795 secs 2024-04-27T00:03:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:05 smithi167 ceph-mon[32751]: pgmap v11137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:05 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 19667.282795 secs 2024-04-27T00:03:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:05 smithi090 ceph-mon[29180]: pgmap v11137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:05 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 19667.282795 secs 2024-04-27T00:03:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:06 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19668.066535 secs 2024-04-27T00:03:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:06 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19668.066535 secs 2024-04-27T00:03:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:06 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19668.066535 secs 2024-04-27T00:03:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:07 smithi119 ceph-mon[32752]: pgmap v11138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:07 smithi167 ceph-mon[32751]: pgmap v11138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:07 smithi090 ceph-mon[29180]: pgmap v11138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:09 smithi119 ceph-mon[32752]: pgmap v11139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:09 smithi167 ceph-mon[32751]: pgmap v11139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:09 smithi090 ceph-mon[29180]: pgmap v11139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:10 smithi119 ceph-mon[32752]: 29 slow requests, 1 included below; oldest blocked for > 19672.282858 secs 2024-04-27T00:03:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:10 smithi119 ceph-mon[32752]: slow request 180.917056 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:03:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:10 smithi167 ceph-mon[32751]: 29 slow requests, 1 included below; oldest blocked for > 19672.282858 secs 2024-04-27T00:03:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:10 smithi167 ceph-mon[32751]: slow request 180.917056 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:03:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:10 smithi090 ceph-mon[29180]: 29 slow requests, 1 included below; oldest blocked for > 19672.282858 secs 2024-04-27T00:03:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:10 smithi090 ceph-mon[29180]: slow request 180.917056 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:03:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:11 smithi119 ceph-mon[32752]: pgmap v11140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:11 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19673.066636 secs 2024-04-27T00:03:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:11 smithi167 ceph-mon[32751]: pgmap v11140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:11 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19673.066636 secs 2024-04-27T00:03:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:11 smithi090 ceph-mon[29180]: pgmap v11140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:11 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19673.066636 secs 2024-04-27T00:03:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:13 smithi119 ceph-mon[32752]: pgmap v11141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:13 smithi167 ceph-mon[32751]: pgmap v11141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:13 smithi090 ceph-mon[29180]: pgmap v11141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:15 smithi119 ceph-mon[32752]: pgmap v11142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:03:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:03:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19677.282942 secs 2024-04-27T00:03:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:15 smithi167 ceph-mon[32751]: pgmap v11142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:03:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:03:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19677.282942 secs 2024-04-27T00:03:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:15 smithi090 ceph-mon[29180]: pgmap v11142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:03:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:03:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19677.282942 secs 2024-04-27T00:03:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:16 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19678.066785 secs 2024-04-27T00:03:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:16 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19678.066785 secs 2024-04-27T00:03:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:16 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19678.066785 secs 2024-04-27T00:03:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:17 smithi119 ceph-mon[32752]: pgmap v11143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:17 smithi167 ceph-mon[32751]: pgmap v11143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:17 smithi090 ceph-mon[29180]: pgmap v11143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:19.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:19 smithi090 ceph-mon[29180]: pgmap v11144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:19 smithi167 ceph-mon[32751]: pgmap v11144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:19 smithi119 ceph-mon[32752]: pgmap v11144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19682.283030 secs 2024-04-27T00:03:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19682.283030 secs 2024-04-27T00:03:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19682.283030 secs 2024-04-27T00:03:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:21 smithi167 ceph-mon[32751]: pgmap v11145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:21 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19683.066853 secs 2024-04-27T00:03:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:21 smithi090 ceph-mon[29180]: pgmap v11145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:21 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19683.066853 secs 2024-04-27T00:03:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:21 smithi119 ceph-mon[32752]: pgmap v11145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:21 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19683.066853 secs 2024-04-27T00:03:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:23 smithi167 ceph-mon[32751]: pgmap v11146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:23 smithi090 ceph-mon[29180]: pgmap v11146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:23 smithi119 ceph-mon[32752]: pgmap v11146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:25 smithi090 ceph-mon[29180]: pgmap v11147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19687.283096 secs 2024-04-27T00:03:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:25 smithi119 ceph-mon[32752]: pgmap v11147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19687.283096 secs 2024-04-27T00:03:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:25 smithi167 ceph-mon[32751]: pgmap v11147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19687.283096 secs 2024-04-27T00:03:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:26 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19688.066933 secs 2024-04-27T00:03:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:26 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19688.066933 secs 2024-04-27T00:03:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:26 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19688.066933 secs 2024-04-27T00:03:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:27 smithi090 ceph-mon[29180]: pgmap v11148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:27 smithi119 ceph-mon[32752]: pgmap v11148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:27 smithi167 ceph-mon[32751]: pgmap v11148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:29 smithi090 ceph-mon[29180]: pgmap v11149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:29 smithi119 ceph-mon[32752]: pgmap v11149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:29 smithi167 ceph-mon[32751]: pgmap v11149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:30 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19692.283181 secs 2024-04-27T00:03:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:30 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19692.283181 secs 2024-04-27T00:03:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:30 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19692.283181 secs 2024-04-27T00:03:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:31 smithi090 ceph-mon[29180]: pgmap v11150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:31 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19693.067042 secs 2024-04-27T00:03:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:31 smithi119 ceph-mon[32752]: pgmap v11150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:31 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19693.067042 secs 2024-04-27T00:03:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:31 smithi167 ceph-mon[32751]: pgmap v11150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:31 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19693.067042 secs 2024-04-27T00:03:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:33 smithi090 ceph-mon[29180]: pgmap v11151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:33 smithi119 ceph-mon[32752]: pgmap v11151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:33 smithi167 ceph-mon[32751]: pgmap v11151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:35 smithi090 ceph-mon[29180]: pgmap v11152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19697.283269 secs 2024-04-27T00:03:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:35 smithi119 ceph-mon[32752]: pgmap v11152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19697.283269 secs 2024-04-27T00:03:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:35 smithi167 ceph-mon[32751]: pgmap v11152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19697.283269 secs 2024-04-27T00:03:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:36 smithi090 ceph-mon[29180]: 20 slow requests, 0 included below; oldest blocked for > 19698.067172 secs 2024-04-27T00:03:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:36 smithi119 ceph-mon[32752]: 20 slow requests, 0 included below; oldest blocked for > 19698.067172 secs 2024-04-27T00:03:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:36 smithi167 ceph-mon[32751]: 20 slow requests, 0 included below; oldest blocked for > 19698.067172 secs 2024-04-27T00:03:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:37 smithi090 ceph-mon[29180]: pgmap v11153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:37 smithi119 ceph-mon[32752]: pgmap v11153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:37 smithi167 ceph-mon[32751]: pgmap v11153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:40.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:39 smithi167 ceph-mon[32751]: pgmap v11154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:39 smithi090 ceph-mon[29180]: pgmap v11154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:39 smithi119 ceph-mon[32752]: pgmap v11154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19702.283354 secs 2024-04-27T00:03:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19702.283354 secs 2024-04-27T00:03:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19702.283354 secs 2024-04-27T00:03:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:41 smithi090 ceph-mon[29180]: pgmap v11155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:41 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 19703.067290 secs 2024-04-27T00:03:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:41 smithi090 ceph-mon[29180]: slow request 184.639700 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:03:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:41 smithi119 ceph-mon[32752]: pgmap v11155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:41 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 19703.067290 secs 2024-04-27T00:03:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:41 smithi119 ceph-mon[32752]: slow request 184.639700 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:03:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:41 smithi167 ceph-mon[32751]: pgmap v11155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:41 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 19703.067290 secs 2024-04-27T00:03:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:41 smithi167 ceph-mon[32751]: slow request 184.639700 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:03:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:44 smithi119 ceph-mon[32752]: pgmap v11156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:03:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:44 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:03:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:44 smithi167 ceph-mon[32751]: pgmap v11156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:03:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:44 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:03:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:44 smithi090 ceph-mon[29180]: pgmap v11156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:03:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:44 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:03:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:45 smithi119 ceph-mon[32752]: pgmap v11157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:45.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19707.283438 secs 2024-04-27T00:03:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:45 smithi167 ceph-mon[32751]: pgmap v11157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19707.283438 secs 2024-04-27T00:03:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:45 smithi090 ceph-mon[29180]: pgmap v11157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19707.283438 secs 2024-04-27T00:03:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19708.067408 secs 2024-04-27T00:03:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19708.067408 secs 2024-04-27T00:03:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19708.067408 secs 2024-04-27T00:03:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:47 smithi090 ceph-mon[29180]: pgmap v11158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:47 smithi119 ceph-mon[32752]: pgmap v11158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:47 smithi167 ceph-mon[32751]: pgmap v11158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:49 smithi119 ceph-mon[32752]: pgmap v11159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:49 smithi167 ceph-mon[32751]: pgmap v11159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:49 smithi090 ceph-mon[29180]: pgmap v11159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:03:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19712.283524 secs 2024-04-27T00:03:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19712.283524 secs 2024-04-27T00:03:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19712.283524 secs 2024-04-27T00:03:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:51 smithi119 ceph-mon[32752]: pgmap v11160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19713.067506 secs 2024-04-27T00:03:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:51 smithi167 ceph-mon[32751]: pgmap v11160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19713.067506 secs 2024-04-27T00:03:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:51 smithi090 ceph-mon[29180]: pgmap v11160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19713.067506 secs 2024-04-27T00:03:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:53 smithi119 ceph-mon[32752]: pgmap v11161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:03:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:03:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:03:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:03:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:53 smithi167 ceph-mon[32751]: pgmap v11161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:03:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:03:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:03:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:03:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:53 smithi090 ceph-mon[29180]: pgmap v11161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:03:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:03:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:03:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:03:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:03:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:54 smithi119 ceph-mon[32752]: pgmap v11162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:03:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:54 smithi119 ceph-mon[32752]: pgmap v11163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:03:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:54 smithi167 ceph-mon[32751]: pgmap v11162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:03:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:54 smithi167 ceph-mon[32751]: pgmap v11163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:54 smithi090 ceph-mon[29180]: pgmap v11162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:03:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:54 smithi090 ceph-mon[29180]: pgmap v11163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:03:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19717.283608 secs 2024-04-27T00:03:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19717.283608 secs 2024-04-27T00:03:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19717.283608 secs 2024-04-27T00:03:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:56 smithi119 ceph-mon[32752]: pgmap v11164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19718.067585 secs 2024-04-27T00:03:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:56 smithi167 ceph-mon[32751]: pgmap v11164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19718.067585 secs 2024-04-27T00:03:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:56 smithi090 ceph-mon[29180]: pgmap v11164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:03:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19718.067585 secs 2024-04-27T00:03:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:03:58 smithi119 ceph-mon[32752]: pgmap v11165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:03:58.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:03:58 smithi167 ceph-mon[32751]: pgmap v11165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:03:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:03:58 smithi090 ceph-mon[29180]: pgmap v11165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:00 smithi119 ceph-mon[32752]: pgmap v11166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19722.283693 secs 2024-04-27T00:04:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:00 smithi167 ceph-mon[32751]: pgmap v11166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19722.283693 secs 2024-04-27T00:04:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:00 smithi090 ceph-mon[29180]: pgmap v11166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19722.283693 secs 2024-04-27T00:04:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19723.067723 secs 2024-04-27T00:04:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19723.067723 secs 2024-04-27T00:04:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19723.067723 secs 2024-04-27T00:04:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:02 smithi119 ceph-mon[32752]: pgmap v11167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:02 smithi167 ceph-mon[32751]: pgmap v11167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:02 smithi090 ceph-mon[29180]: pgmap v11167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:04 smithi119 ceph-mon[32752]: pgmap v11168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:04:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:04 smithi167 ceph-mon[32751]: pgmap v11168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:04:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:04 smithi090 ceph-mon[29180]: pgmap v11168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:04:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:05 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19727.283779 secs 2024-04-27T00:04:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:05 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19727.283779 secs 2024-04-27T00:04:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:05 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19727.283779 secs 2024-04-27T00:04:06.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:06 smithi090 ceph-mon[29180]: pgmap v11169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:06.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19728.067858 secs 2024-04-27T00:04:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:06 smithi119 ceph-mon[32752]: pgmap v11169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19728.067858 secs 2024-04-27T00:04:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:06 smithi167 ceph-mon[32751]: pgmap v11169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19728.067858 secs 2024-04-27T00:04:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:08 smithi119 ceph-mon[32752]: pgmap v11170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:08 smithi167 ceph-mon[32751]: pgmap v11170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:08 smithi090 ceph-mon[29180]: pgmap v11170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:10 smithi119 ceph-mon[32752]: pgmap v11171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:10.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:10 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19732.283842 secs 2024-04-27T00:04:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:10 smithi167 ceph-mon[32751]: pgmap v11171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:10 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19732.283842 secs 2024-04-27T00:04:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:10 smithi090 ceph-mon[29180]: pgmap v11171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:10 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19732.283842 secs 2024-04-27T00:04:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19733.067974 secs 2024-04-27T00:04:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19733.067974 secs 2024-04-27T00:04:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19733.067974 secs 2024-04-27T00:04:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:12 smithi119 ceph-mon[32752]: pgmap v11172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:12 smithi167 ceph-mon[32751]: pgmap v11172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:12 smithi090 ceph-mon[29180]: pgmap v11172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:14 smithi119 ceph-mon[32752]: pgmap v11173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:14 smithi167 ceph-mon[32751]: pgmap v11173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:14 smithi090 ceph-mon[29180]: pgmap v11173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19737.283926 secs 2024-04-27T00:04:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19737.283926 secs 2024-04-27T00:04:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19737.283926 secs 2024-04-27T00:04:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:16 smithi119 ceph-mon[32752]: pgmap v11174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19738.068042 secs 2024-04-27T00:04:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:16 smithi167 ceph-mon[32751]: pgmap v11174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19738.068042 secs 2024-04-27T00:04:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:16 smithi090 ceph-mon[29180]: pgmap v11174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19738.068042 secs 2024-04-27T00:04:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:18 smithi119 ceph-mon[32752]: pgmap v11175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:18 smithi167 ceph-mon[32751]: pgmap v11175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:18 smithi090 ceph-mon[29180]: pgmap v11175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:20 smithi119 ceph-mon[32752]: pgmap v11176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19742.283990 secs 2024-04-27T00:04:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:20 smithi167 ceph-mon[32751]: pgmap v11176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19742.283990 secs 2024-04-27T00:04:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:20 smithi090 ceph-mon[29180]: pgmap v11176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19742.283990 secs 2024-04-27T00:04:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19743.068155 secs 2024-04-27T00:04:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19743.068155 secs 2024-04-27T00:04:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19743.068155 secs 2024-04-27T00:04:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:22 smithi119 ceph-mon[32752]: pgmap v11177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:22 smithi167 ceph-mon[32751]: pgmap v11177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:22 smithi090 ceph-mon[29180]: pgmap v11177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:24 smithi119 ceph-mon[32752]: pgmap v11178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:24 smithi167 ceph-mon[32751]: pgmap v11178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:24 smithi090 ceph-mon[29180]: pgmap v11178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19747.284046 secs 2024-04-27T00:04:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19747.284046 secs 2024-04-27T00:04:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19747.284046 secs 2024-04-27T00:04:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:26 smithi167 ceph-mon[32751]: pgmap v11179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19748.068290 secs 2024-04-27T00:04:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:26 smithi090 ceph-mon[29180]: pgmap v11179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19748.068290 secs 2024-04-27T00:04:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:26 smithi119 ceph-mon[32752]: pgmap v11179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19748.068290 secs 2024-04-27T00:04:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:28 smithi167 ceph-mon[32751]: pgmap v11180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:28 smithi090 ceph-mon[29180]: pgmap v11180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:28 smithi119 ceph-mon[32752]: pgmap v11180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:30 smithi167 ceph-mon[32751]: pgmap v11181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:30 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19752.284105 secs 2024-04-27T00:04:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:30 smithi090 ceph-mon[29180]: pgmap v11181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:30 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19752.284105 secs 2024-04-27T00:04:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:30 smithi119 ceph-mon[32752]: pgmap v11181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:30 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19752.284105 secs 2024-04-27T00:04:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19753.068392 secs 2024-04-27T00:04:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19753.068392 secs 2024-04-27T00:04:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19753.068392 secs 2024-04-27T00:04:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:32 smithi090 ceph-mon[29180]: pgmap v11182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:33.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:32 smithi119 ceph-mon[32752]: pgmap v11182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:32 smithi167 ceph-mon[32751]: pgmap v11182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:34 smithi090 ceph-mon[29180]: pgmap v11183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:35.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:34 smithi119 ceph-mon[32752]: pgmap v11183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:35.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:34 smithi167 ceph-mon[32751]: pgmap v11183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19757.284168 secs 2024-04-27T00:04:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19757.284168 secs 2024-04-27T00:04:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19757.284168 secs 2024-04-27T00:04:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:36 smithi090 ceph-mon[29180]: pgmap v11184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19758.068511 secs 2024-04-27T00:04:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:36 smithi119 ceph-mon[32752]: pgmap v11184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19758.068511 secs 2024-04-27T00:04:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:36 smithi167 ceph-mon[32751]: pgmap v11184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19758.068511 secs 2024-04-27T00:04:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:38 smithi090 ceph-mon[29180]: pgmap v11185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:38 smithi119 ceph-mon[32752]: pgmap v11185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:39.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:38 smithi167 ceph-mon[32751]: pgmap v11185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:40.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:40 smithi167 ceph-mon[32751]: pgmap v11186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:40.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19762.284252 secs 2024-04-27T00:04:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:40 smithi090 ceph-mon[29180]: pgmap v11186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19762.284252 secs 2024-04-27T00:04:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:40 smithi119 ceph-mon[32752]: pgmap v11186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19762.284252 secs 2024-04-27T00:04:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19763.068603 secs 2024-04-27T00:04:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19763.068603 secs 2024-04-27T00:04:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19763.068603 secs 2024-04-27T00:04:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:42 smithi090 ceph-mon[29180]: pgmap v11187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:43.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:42 smithi119 ceph-mon[32752]: pgmap v11187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:43.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:42 smithi167 ceph-mon[32751]: pgmap v11187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:44 smithi090 ceph-mon[29180]: pgmap v11188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:44 smithi119 ceph-mon[32752]: pgmap v11188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:45.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:44 smithi167 ceph-mon[32751]: pgmap v11188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19767.284334 secs 2024-04-27T00:04:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19767.284334 secs 2024-04-27T00:04:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19767.284334 secs 2024-04-27T00:04:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:46 smithi090 ceph-mon[29180]: pgmap v11189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19768.068708 secs 2024-04-27T00:04:46.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:46 smithi119 ceph-mon[32752]: pgmap v11189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:46.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19768.068708 secs 2024-04-27T00:04:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:46 smithi167 ceph-mon[32751]: pgmap v11189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19768.068708 secs 2024-04-27T00:04:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:48 smithi090 ceph-mon[29180]: pgmap v11190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:48 smithi119 ceph-mon[32752]: pgmap v11190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:48 smithi167 ceph-mon[32751]: pgmap v11190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:50 smithi090 ceph-mon[29180]: pgmap v11191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19772.284420 secs 2024-04-27T00:04:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:50 smithi119 ceph-mon[32752]: pgmap v11191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19772.284420 secs 2024-04-27T00:04:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:50 smithi167 ceph-mon[32751]: pgmap v11191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:04:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19772.284420 secs 2024-04-27T00:04:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19773.068843 secs 2024-04-27T00:04:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19773.068843 secs 2024-04-27T00:04:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19773.068843 secs 2024-04-27T00:04:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:52 smithi090 ceph-mon[29180]: pgmap v11192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:52 smithi119 ceph-mon[32752]: pgmap v11192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:52 smithi167 ceph-mon[32751]: pgmap v11192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:04:54.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:04:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:04:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:54 smithi119 ceph-mon[32752]: pgmap v11193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:04:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:04:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:54 smithi119 ceph-mon[32752]: pgmap v11194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:04:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:54 smithi119 ceph-mon[32752]: pgmap v11195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:04:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:54 smithi167 ceph-mon[32751]: pgmap v11193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:04:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:04:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:54 smithi167 ceph-mon[32751]: pgmap v11194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:04:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:54 smithi167 ceph-mon[32751]: pgmap v11195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:04:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:54 smithi090 ceph-mon[29180]: pgmap v11193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:04:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:04:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:04:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:54 smithi090 ceph-mon[29180]: pgmap v11194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:04:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:54 smithi090 ceph-mon[29180]: pgmap v11195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:04:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19777.284503 secs 2024-04-27T00:04:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19777.284503 secs 2024-04-27T00:04:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19777.284503 secs 2024-04-27T00:04:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19778.068961 secs 2024-04-27T00:04:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:56 smithi119 ceph-mon[32752]: pgmap v11196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19778.068961 secs 2024-04-27T00:04:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:56 smithi167 ceph-mon[32751]: pgmap v11196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19778.068961 secs 2024-04-27T00:04:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:56 smithi090 ceph-mon[29180]: pgmap v11196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:04:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:04:58 smithi119 ceph-mon[32752]: pgmap v11197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:04:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:04:58 smithi167 ceph-mon[32751]: pgmap v11197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:04:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:04:58 smithi090 ceph-mon[29180]: pgmap v11197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:05:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:00 smithi119 ceph-mon[32752]: pgmap v11198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19782.284587 secs 2024-04-27T00:05:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:00 smithi167 ceph-mon[32751]: pgmap v11198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19782.284587 secs 2024-04-27T00:05:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:00 smithi090 ceph-mon[29180]: pgmap v11198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19782.284587 secs 2024-04-27T00:05:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19783.069110 secs 2024-04-27T00:05:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19783.069110 secs 2024-04-27T00:05:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19783.069110 secs 2024-04-27T00:05:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:03 smithi119 ceph-mon[32752]: pgmap v11199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:03 smithi167 ceph-mon[32751]: pgmap v11199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:03 smithi090 ceph-mon[29180]: pgmap v11199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:05 smithi119 ceph-mon[32752]: pgmap v11200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:05:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:05 smithi167 ceph-mon[32751]: pgmap v11200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:05:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:05 smithi090 ceph-mon[29180]: pgmap v11200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:05:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:06 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19787.284671 secs 2024-04-27T00:05:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:06 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19787.284671 secs 2024-04-27T00:05:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:06 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19787.284671 secs 2024-04-27T00:05:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19788.069227 secs 2024-04-27T00:05:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:07 smithi119 ceph-mon[32752]: pgmap v11201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19788.069227 secs 2024-04-27T00:05:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:07 smithi167 ceph-mon[32751]: pgmap v11201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19788.069227 secs 2024-04-27T00:05:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:07 smithi090 ceph-mon[29180]: pgmap v11201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:09 smithi119 ceph-mon[32752]: pgmap v11202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:09 smithi167 ceph-mon[32751]: pgmap v11202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:09 smithi090 ceph-mon[29180]: pgmap v11202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:11 smithi119 ceph-mon[32752]: pgmap v11203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:11 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19792.284732 secs 2024-04-27T00:05:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:11 smithi167 ceph-mon[32751]: pgmap v11203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:11 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19792.284732 secs 2024-04-27T00:05:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:11 smithi090 ceph-mon[29180]: pgmap v11203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:11 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19792.284732 secs 2024-04-27T00:05:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19793.069346 secs 2024-04-27T00:05:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19793.069346 secs 2024-04-27T00:05:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19793.069346 secs 2024-04-27T00:05:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:13 smithi119 ceph-mon[32752]: pgmap v11204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:13 smithi167 ceph-mon[32751]: pgmap v11204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:13 smithi090 ceph-mon[29180]: pgmap v11204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:15 smithi119 ceph-mon[32752]: pgmap v11205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:15 smithi167 ceph-mon[32751]: pgmap v11205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:15 smithi090 ceph-mon[29180]: pgmap v11205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:16 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19797.284819 secs 2024-04-27T00:05:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:16 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19797.284819 secs 2024-04-27T00:05:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:16 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19797.284819 secs 2024-04-27T00:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19798.069469 secs 2024-04-27T00:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:17 smithi119 ceph-mon[32752]: pgmap v11206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19798.069469 secs 2024-04-27T00:05:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:17 smithi167 ceph-mon[32751]: pgmap v11206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19798.069469 secs 2024-04-27T00:05:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:17 smithi090 ceph-mon[29180]: pgmap v11206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:19 smithi119 ceph-mon[32752]: pgmap v11207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:19 smithi167 ceph-mon[32751]: pgmap v11207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:19 smithi090 ceph-mon[29180]: pgmap v11207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:21 smithi119 ceph-mon[32752]: pgmap v11208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:21 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19802.284880 secs 2024-04-27T00:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:21 smithi167 ceph-mon[32751]: pgmap v11208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:21 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19802.284880 secs 2024-04-27T00:05:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:21 smithi090 ceph-mon[29180]: pgmap v11208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:21 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19802.284880 secs 2024-04-27T00:05:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19803.069533 secs 2024-04-27T00:05:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19803.069533 secs 2024-04-27T00:05:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19803.069533 secs 2024-04-27T00:05:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:23 smithi119 ceph-mon[32752]: pgmap v11209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:23 smithi167 ceph-mon[32751]: pgmap v11209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:23 smithi090 ceph-mon[29180]: pgmap v11209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:25 smithi167 ceph-mon[32751]: pgmap v11210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:25 smithi090 ceph-mon[29180]: pgmap v11210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:25 smithi119 ceph-mon[32752]: pgmap v11210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:26 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19807.284941 secs 2024-04-27T00:05:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:26 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19807.284941 secs 2024-04-27T00:05:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:26 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19807.284941 secs 2024-04-27T00:05:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19808.069645 secs 2024-04-27T00:05:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:27 smithi167 ceph-mon[32751]: pgmap v11211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19808.069645 secs 2024-04-27T00:05:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:27 smithi090 ceph-mon[29180]: pgmap v11211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19808.069645 secs 2024-04-27T00:05:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:27 smithi119 ceph-mon[32752]: pgmap v11211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:29 smithi167 ceph-mon[32751]: pgmap v11212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:29 smithi090 ceph-mon[29180]: pgmap v11212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:29 smithi119 ceph-mon[32752]: pgmap v11212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:31 smithi090 ceph-mon[29180]: pgmap v11213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:31 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19812.285030 secs 2024-04-27T00:05:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:31 smithi119 ceph-mon[32752]: pgmap v11213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:31 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19812.285030 secs 2024-04-27T00:05:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:31 smithi167 ceph-mon[32751]: pgmap v11213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:31 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19812.285030 secs 2024-04-27T00:05:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19813.069765 secs 2024-04-27T00:05:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19813.069765 secs 2024-04-27T00:05:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19813.069765 secs 2024-04-27T00:05:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:33 smithi090 ceph-mon[29180]: pgmap v11214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:33 smithi119 ceph-mon[32752]: pgmap v11214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:33 smithi167 ceph-mon[32751]: pgmap v11214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:35 smithi090 ceph-mon[29180]: pgmap v11215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:35 smithi119 ceph-mon[32752]: pgmap v11215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:35 smithi167 ceph-mon[32751]: pgmap v11215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:36 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19817.285111 secs 2024-04-27T00:05:36.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:05:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:05:36.226+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:05:36.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:05:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:05:36.226+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:05:36.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:05:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:05:36.226+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:05:36.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:05:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:05:36.226+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:05:36.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:05:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:05:36.226+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:05:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:36 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19817.285111 secs 2024-04-27T00:05:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:36 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19817.285111 secs 2024-04-27T00:05:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19818.069906 secs 2024-04-27T00:05:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:37 smithi090 ceph-mon[29180]: pgmap v11216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19818.069906 secs 2024-04-27T00:05:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:37 smithi119 ceph-mon[32752]: pgmap v11216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19818.069906 secs 2024-04-27T00:05:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:37 smithi167 ceph-mon[32751]: pgmap v11216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:39 smithi090 ceph-mon[29180]: pgmap v11217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:39 smithi119 ceph-mon[32752]: pgmap v11217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:39 smithi167 ceph-mon[32751]: pgmap v11217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19822.285198 secs 2024-04-27T00:05:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19822.285198 secs 2024-04-27T00:05:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19822.285198 secs 2024-04-27T00:05:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:41 smithi090 ceph-mon[29180]: pgmap v11218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:41.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:41 smithi167 ceph-mon[32751]: pgmap v11218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:41 smithi119 ceph-mon[32752]: pgmap v11218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19823.070026 secs 2024-04-27T00:05:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19823.070026 secs 2024-04-27T00:05:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19823.070026 secs 2024-04-27T00:05:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:43 smithi090 ceph-mon[29180]: pgmap v11219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:43 smithi119 ceph-mon[32752]: pgmap v11219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:43 smithi167 ceph-mon[32751]: pgmap v11219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:45 smithi090 ceph-mon[29180]: pgmap v11220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19827.285255 secs 2024-04-27T00:05:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:45 smithi119 ceph-mon[32752]: pgmap v11220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19827.285255 secs 2024-04-27T00:05:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:45 smithi167 ceph-mon[32751]: pgmap v11220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19827.285255 secs 2024-04-27T00:05:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19828.070160 secs 2024-04-27T00:05:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:47 smithi090 ceph-mon[29180]: pgmap v11221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19828.070160 secs 2024-04-27T00:05:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:47 smithi119 ceph-mon[32752]: pgmap v11221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19828.070160 secs 2024-04-27T00:05:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:47 smithi167 ceph-mon[32751]: pgmap v11221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:49 smithi090 ceph-mon[29180]: pgmap v11222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:49 smithi119 ceph-mon[32752]: pgmap v11222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:49 smithi167 ceph-mon[32751]: pgmap v11222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T00:05:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19832.285335 secs 2024-04-27T00:05:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19832.285335 secs 2024-04-27T00:05:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19832.285335 secs 2024-04-27T00:05:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:51 smithi090 ceph-mon[29180]: pgmap v11223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:51 smithi119 ceph-mon[32752]: pgmap v11223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:51 smithi167 ceph-mon[32751]: pgmap v11223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T00:05:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19833.070274 secs 2024-04-27T00:05:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19833.070274 secs 2024-04-27T00:05:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19833.070274 secs 2024-04-27T00:05:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:53 smithi119 ceph-mon[32752]: pgmap v11224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:53 smithi167 ceph-mon[32751]: pgmap v11224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:53 smithi090 ceph-mon[29180]: pgmap v11224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:05:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:05:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:05:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:05:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:55 smithi119 ceph-mon[32752]: pgmap v11225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:05:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:05:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:05:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19837.285422 secs 2024-04-27T00:05:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:55 smithi167 ceph-mon[32751]: pgmap v11225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:05:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:05:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:05:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19837.285422 secs 2024-04-27T00:05:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:55 smithi090 ceph-mon[29180]: pgmap v11225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:05:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:05:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:05:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:05:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19837.285422 secs 2024-04-27T00:05:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:56 smithi119 ceph-mon[32752]: pgmap v11226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:05:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:56 smithi119 ceph-mon[32752]: pgmap v11227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:56 smithi167 ceph-mon[32751]: pgmap v11226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:05:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:56 smithi167 ceph-mon[32751]: pgmap v11227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:56 smithi090 ceph-mon[29180]: pgmap v11226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:05:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:56 smithi090 ceph-mon[29180]: pgmap v11227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19838.070363 secs 2024-04-27T00:05:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19838.070363 secs 2024-04-27T00:05:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19838.070363 secs 2024-04-27T00:05:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:05:58 smithi119 ceph-mon[32752]: pgmap v11228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:05:58 smithi167 ceph-mon[32751]: pgmap v11228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:05:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:05:58 smithi090 ceph-mon[29180]: pgmap v11228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:00.428 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:00 smithi090 ceph-mon[29180]: pgmap v11229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:00.428 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19842.285504 secs 2024-04-27T00:06:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:00 smithi119 ceph-mon[32752]: pgmap v11229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19842.285504 secs 2024-04-27T00:06:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:00 smithi167 ceph-mon[32751]: pgmap v11229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19842.285504 secs 2024-04-27T00:06:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:01 smithi119 ceph-mon[32752]: pgmap v11230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:01 smithi167 ceph-mon[32751]: pgmap v11230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:01.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:01 smithi090 ceph-mon[29180]: pgmap v11230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19843.070471 secs 2024-04-27T00:06:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19843.070471 secs 2024-04-27T00:06:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19843.070471 secs 2024-04-27T00:06:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:03 smithi119 ceph-mon[32752]: pgmap v11231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:06:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:03 smithi167 ceph-mon[32751]: pgmap v11231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:06:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:03 smithi090 ceph-mon[29180]: pgmap v11231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:06:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:05 smithi119 ceph-mon[32752]: pgmap v11232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:06:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:05 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19847.285589 secs 2024-04-27T00:06:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:05 smithi167 ceph-mon[32751]: pgmap v11232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:06:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:05 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19847.285589 secs 2024-04-27T00:06:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:05 smithi090 ceph-mon[29180]: pgmap v11232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:06:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:05 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19847.285589 secs 2024-04-27T00:06:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19848.070591 secs 2024-04-27T00:06:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19848.070591 secs 2024-04-27T00:06:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19848.070591 secs 2024-04-27T00:06:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:07 smithi119 ceph-mon[32752]: pgmap v11233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:07 smithi167 ceph-mon[32751]: pgmap v11233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:07 smithi090 ceph-mon[29180]: pgmap v11233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:09 smithi119 ceph-mon[32752]: pgmap v11234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:09 smithi167 ceph-mon[32751]: pgmap v11234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:09 smithi090 ceph-mon[29180]: pgmap v11234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:10 smithi119 ceph-mon[32752]: 29 slow requests, 1 included below; oldest blocked for > 19852.285675 secs 2024-04-27T00:06:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:10 smithi119 ceph-mon[32752]: slow request 360.919874 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:06:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:10 smithi167 ceph-mon[32751]: 29 slow requests, 1 included below; oldest blocked for > 19852.285675 secs 2024-04-27T00:06:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:10 smithi167 ceph-mon[32751]: slow request 360.919874 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:06:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:10 smithi090 ceph-mon[29180]: 29 slow requests, 1 included below; oldest blocked for > 19852.285675 secs 2024-04-27T00:06:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:10 smithi090 ceph-mon[29180]: slow request 360.919874 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:06:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:11 smithi119 ceph-mon[32752]: pgmap v11235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19853.070664 secs 2024-04-27T00:06:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:11 smithi167 ceph-mon[32751]: pgmap v11235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19853.070664 secs 2024-04-27T00:06:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:11 smithi090 ceph-mon[29180]: pgmap v11235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19853.070664 secs 2024-04-27T00:06:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:13 smithi119 ceph-mon[32752]: pgmap v11236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:13 smithi167 ceph-mon[32751]: pgmap v11236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:13 smithi090 ceph-mon[29180]: pgmap v11236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:15 smithi119 ceph-mon[32752]: pgmap v11237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19857.285759 secs 2024-04-27T00:06:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:15 smithi167 ceph-mon[32751]: pgmap v11237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19857.285759 secs 2024-04-27T00:06:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:15 smithi090 ceph-mon[29180]: pgmap v11237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19857.285759 secs 2024-04-27T00:06:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19858.070731 secs 2024-04-27T00:06:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19858.070731 secs 2024-04-27T00:06:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19858.070731 secs 2024-04-27T00:06:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:17 smithi167 ceph-mon[32751]: pgmap v11238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:17 smithi090 ceph-mon[29180]: pgmap v11238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:18.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:17 smithi119 ceph-mon[32752]: pgmap v11238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:19 smithi167 ceph-mon[32751]: pgmap v11239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:19 smithi090 ceph-mon[29180]: pgmap v11239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:19 smithi119 ceph-mon[32752]: pgmap v11239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19862.285844 secs 2024-04-27T00:06:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19862.285844 secs 2024-04-27T00:06:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19862.285844 secs 2024-04-27T00:06:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:21 smithi090 ceph-mon[29180]: pgmap v11240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19863.070809 secs 2024-04-27T00:06:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:21 smithi119 ceph-mon[32752]: pgmap v11240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19863.070809 secs 2024-04-27T00:06:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:21 smithi167 ceph-mon[32751]: pgmap v11240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19863.070809 secs 2024-04-27T00:06:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:23 smithi090 ceph-mon[29180]: pgmap v11241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:23 smithi119 ceph-mon[32752]: pgmap v11241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:23 smithi167 ceph-mon[32751]: pgmap v11241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:25 smithi090 ceph-mon[29180]: pgmap v11242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19867.285930 secs 2024-04-27T00:06:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:25 smithi119 ceph-mon[32752]: pgmap v11242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19867.285930 secs 2024-04-27T00:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:25 smithi167 ceph-mon[32751]: pgmap v11242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19867.285930 secs 2024-04-27T00:06:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19868.070924 secs 2024-04-27T00:06:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19868.070924 secs 2024-04-27T00:06:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19868.070924 secs 2024-04-27T00:06:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:27 smithi090 ceph-mon[29180]: pgmap v11243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:27 smithi119 ceph-mon[32752]: pgmap v11243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:27 smithi167 ceph-mon[32751]: pgmap v11243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:29 smithi090 ceph-mon[29180]: pgmap v11244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:29 smithi119 ceph-mon[32752]: pgmap v11244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:29 smithi167 ceph-mon[32751]: pgmap v11244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:30 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19872.286016 secs 2024-04-27T00:06:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:30 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19872.286016 secs 2024-04-27T00:06:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:30 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19872.286016 secs 2024-04-27T00:06:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:31 smithi090 ceph-mon[29180]: pgmap v11245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19873.071036 secs 2024-04-27T00:06:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:31 smithi119 ceph-mon[32752]: pgmap v11245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19873.071036 secs 2024-04-27T00:06:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:31 smithi167 ceph-mon[32751]: pgmap v11245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19873.071036 secs 2024-04-27T00:06:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:33 smithi090 ceph-mon[29180]: pgmap v11246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:33 smithi119 ceph-mon[32752]: pgmap v11246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:33 smithi167 ceph-mon[32751]: pgmap v11246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:35 smithi090 ceph-mon[29180]: pgmap v11247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19877.286104 secs 2024-04-27T00:06:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:35 smithi119 ceph-mon[32752]: pgmap v11247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19877.286104 secs 2024-04-27T00:06:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:35 smithi167 ceph-mon[32751]: pgmap v11247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19877.286104 secs 2024-04-27T00:06:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19878.071132 secs 2024-04-27T00:06:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19878.071132 secs 2024-04-27T00:06:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19878.071132 secs 2024-04-27T00:06:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:37 smithi090 ceph-mon[29180]: pgmap v11248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:37 smithi119 ceph-mon[32752]: pgmap v11248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:37 smithi167 ceph-mon[32751]: pgmap v11248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:39 smithi090 ceph-mon[29180]: pgmap v11249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:39 smithi119 ceph-mon[32752]: pgmap v11249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:39 smithi167 ceph-mon[32751]: pgmap v11249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19882.286164 secs 2024-04-27T00:06:41.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19882.286164 secs 2024-04-27T00:06:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19882.286164 secs 2024-04-27T00:06:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:41 smithi090 ceph-mon[29180]: pgmap v11250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:41 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 19883.071262 secs 2024-04-27T00:06:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:41 smithi090 ceph-mon[29180]: slow request 364.643673 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:06:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:41 smithi119 ceph-mon[32752]: pgmap v11250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:41 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 19883.071262 secs 2024-04-27T00:06:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:41 smithi119 ceph-mon[32752]: slow request 364.643673 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:06:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:41 smithi167 ceph-mon[32751]: pgmap v11250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:41 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 19883.071262 secs 2024-04-27T00:06:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:41 smithi167 ceph-mon[32751]: slow request 364.643673 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:06:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:43 smithi090 ceph-mon[29180]: pgmap v11251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:43 smithi119 ceph-mon[32752]: pgmap v11251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:43 smithi167 ceph-mon[32751]: pgmap v11251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:45 smithi090 ceph-mon[29180]: pgmap v11252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19887.286249 secs 2024-04-27T00:06:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:45 smithi119 ceph-mon[32752]: pgmap v11252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19887.286249 secs 2024-04-27T00:06:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:45 smithi167 ceph-mon[32751]: pgmap v11252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19887.286249 secs 2024-04-27T00:06:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19888.071401 secs 2024-04-27T00:06:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19888.071401 secs 2024-04-27T00:06:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19888.071401 secs 2024-04-27T00:06:48.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:47 smithi119 ceph-mon[32752]: pgmap v11253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:47 smithi167 ceph-mon[32751]: pgmap v11253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:47 smithi090 ceph-mon[29180]: pgmap v11253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:49 smithi119 ceph-mon[32752]: pgmap v11254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:49 smithi167 ceph-mon[32751]: pgmap v11254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:49 smithi090 ceph-mon[29180]: pgmap v11254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19892.286331 secs 2024-04-27T00:06:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19892.286331 secs 2024-04-27T00:06:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19892.286331 secs 2024-04-27T00:06:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:51 smithi119 ceph-mon[32752]: pgmap v11255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19893.071503 secs 2024-04-27T00:06:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:51 smithi167 ceph-mon[32751]: pgmap v11255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19893.071503 secs 2024-04-27T00:06:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:51 smithi090 ceph-mon[29180]: pgmap v11255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19893.071503 secs 2024-04-27T00:06:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:53 smithi119 ceph-mon[32752]: pgmap v11256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:53 smithi167 ceph-mon[32751]: pgmap v11256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:53 smithi090 ceph-mon[29180]: pgmap v11256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:06:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:06:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:06:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:06:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:55 smithi119 ceph-mon[32752]: pgmap v11257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19897.286422 secs 2024-04-27T00:06:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:06:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:06:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:06:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:55 smithi167 ceph-mon[32751]: pgmap v11257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19897.286422 secs 2024-04-27T00:06:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:06:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:06:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:06:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:55 smithi090 ceph-mon[29180]: pgmap v11257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:06:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19897.286422 secs 2024-04-27T00:06:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:06:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:06:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:06:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:56 smithi119 ceph-mon[32752]: pgmap v11258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:06:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:56 smithi119 ceph-mon[32752]: pgmap v11259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19898.071623 secs 2024-04-27T00:06:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:56 smithi167 ceph-mon[32751]: pgmap v11258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:06:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:56 smithi167 ceph-mon[32751]: pgmap v11259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19898.071623 secs 2024-04-27T00:06:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:56 smithi090 ceph-mon[29180]: pgmap v11258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:06:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:56 smithi090 ceph-mon[29180]: pgmap v11259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:06:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19898.071623 secs 2024-04-27T00:06:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:06:58 smithi119 ceph-mon[32752]: pgmap v11260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:06:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:06:58 smithi167 ceph-mon[32751]: pgmap v11260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:06:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:06:58 smithi090 ceph-mon[29180]: pgmap v11260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:07:01.178 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:00 smithi090 ceph-mon[29180]: pgmap v11261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:01.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19902.286502 secs 2024-04-27T00:07:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:00 smithi119 ceph-mon[32752]: pgmap v11261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19902.286502 secs 2024-04-27T00:07:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:00 smithi167 ceph-mon[32751]: pgmap v11261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19902.286502 secs 2024-04-27T00:07:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19903.071735 secs 2024-04-27T00:07:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19903.071735 secs 2024-04-27T00:07:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19903.071735 secs 2024-04-27T00:07:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:02 smithi119 ceph-mon[32752]: pgmap v11262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:02 smithi167 ceph-mon[32751]: pgmap v11262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:02 smithi090 ceph-mon[29180]: pgmap v11262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:04 smithi119 ceph-mon[32752]: pgmap v11263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:04 smithi167 ceph-mon[32751]: pgmap v11263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:04 smithi090 ceph-mon[29180]: pgmap v11263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:07:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:05 smithi119 ceph-mon[32752]: 29 slow requests, 1 included below; oldest blocked for > 19907.286584 secs 2024-04-27T00:07:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:05 smithi119 ceph-mon[32752]: slow request 11522.048528 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:07:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:05 smithi167 ceph-mon[32751]: 29 slow requests, 1 included below; oldest blocked for > 19907.286584 secs 2024-04-27T00:07:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:05 smithi167 ceph-mon[32751]: slow request 11522.048528 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:07:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:05 smithi090 ceph-mon[29180]: 29 slow requests, 1 included below; oldest blocked for > 19907.286584 secs 2024-04-27T00:07:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:05 smithi090 ceph-mon[29180]: slow request 11522.048528 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:07:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:06 smithi119 ceph-mon[32752]: pgmap v11264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:07:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:06 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 19908.071813 secs 2024-04-27T00:07:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:06 smithi119 ceph-mon[32752]: slow request 5761.229674 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:07:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:06 smithi167 ceph-mon[32751]: pgmap v11264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:07:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:06 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 19908.071813 secs 2024-04-27T00:07:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:06 smithi167 ceph-mon[32751]: slow request 5761.229674 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:07:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:06 smithi090 ceph-mon[29180]: pgmap v11264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:07:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:06 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 19908.071813 secs 2024-04-27T00:07:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:06 smithi090 ceph-mon[29180]: slow request 5761.229674 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:07:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:08 smithi119 ceph-mon[32752]: pgmap v11265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:08 smithi167 ceph-mon[32751]: pgmap v11265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:08 smithi090 ceph-mon[29180]: pgmap v11265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:10 smithi119 ceph-mon[32752]: pgmap v11266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:10 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19912.286667 secs 2024-04-27T00:07:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:10 smithi167 ceph-mon[32751]: pgmap v11266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:10 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19912.286667 secs 2024-04-27T00:07:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:10 smithi090 ceph-mon[29180]: pgmap v11266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:10 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19912.286667 secs 2024-04-27T00:07:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19913.071947 secs 2024-04-27T00:07:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19913.071947 secs 2024-04-27T00:07:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19913.071947 secs 2024-04-27T00:07:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:12 smithi119 ceph-mon[32752]: pgmap v11267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:12 smithi167 ceph-mon[32751]: pgmap v11267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:12 smithi090 ceph-mon[29180]: pgmap v11267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:14 smithi119 ceph-mon[32752]: pgmap v11268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:15.361 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:14 smithi167 ceph-mon[32751]: pgmap v11268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:14 smithi090 ceph-mon[29180]: pgmap v11268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19917.286752 secs 2024-04-27T00:07:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19917.286752 secs 2024-04-27T00:07:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19917.286752 secs 2024-04-27T00:07:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:16 smithi119 ceph-mon[32752]: pgmap v11269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19918.072062 secs 2024-04-27T00:07:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:16 smithi167 ceph-mon[32751]: pgmap v11269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19918.072062 secs 2024-04-27T00:07:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:16 smithi090 ceph-mon[29180]: pgmap v11269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19918.072062 secs 2024-04-27T00:07:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:18 smithi119 ceph-mon[32752]: pgmap v11270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:18 smithi167 ceph-mon[32751]: pgmap v11270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:18 smithi090 ceph-mon[29180]: pgmap v11270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:20 smithi119 ceph-mon[32752]: pgmap v11271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19922.286846 secs 2024-04-27T00:07:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:20 smithi167 ceph-mon[32751]: pgmap v11271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19922.286846 secs 2024-04-27T00:07:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:20 smithi090 ceph-mon[29180]: pgmap v11271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19922.286846 secs 2024-04-27T00:07:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19923.072154 secs 2024-04-27T00:07:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19923.072154 secs 2024-04-27T00:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19923.072154 secs 2024-04-27T00:07:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:22 smithi119 ceph-mon[32752]: pgmap v11272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:22 smithi167 ceph-mon[32751]: pgmap v11272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:22 smithi090 ceph-mon[29180]: pgmap v11272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:24 smithi119 ceph-mon[32752]: pgmap v11273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:24 smithi167 ceph-mon[32751]: pgmap v11273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:24 smithi090 ceph-mon[29180]: pgmap v11273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19927.286936 secs 2024-04-27T00:07:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19927.286936 secs 2024-04-27T00:07:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19927.286936 secs 2024-04-27T00:07:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:26 smithi119 ceph-mon[32752]: pgmap v11274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19928.072291 secs 2024-04-27T00:07:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:26 smithi167 ceph-mon[32751]: pgmap v11274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19928.072291 secs 2024-04-27T00:07:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:26 smithi090 ceph-mon[29180]: pgmap v11274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19928.072291 secs 2024-04-27T00:07:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:28 smithi119 ceph-mon[32752]: pgmap v11275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:28 smithi167 ceph-mon[32751]: pgmap v11275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:28 smithi090 ceph-mon[29180]: pgmap v11275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:31.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:30 smithi090 ceph-mon[29180]: pgmap v11276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:31.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:30 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19932.287024 secs 2024-04-27T00:07:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:30 smithi119 ceph-mon[32752]: pgmap v11276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:30 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19932.287024 secs 2024-04-27T00:07:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:30 smithi167 ceph-mon[32751]: pgmap v11276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:30 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19932.287024 secs 2024-04-27T00:07:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19933.072407 secs 2024-04-27T00:07:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19933.072407 secs 2024-04-27T00:07:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19933.072407 secs 2024-04-27T00:07:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:32 smithi119 ceph-mon[32752]: pgmap v11277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:32 smithi167 ceph-mon[32751]: pgmap v11277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:32 smithi090 ceph-mon[29180]: pgmap v11277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:34 smithi119 ceph-mon[32752]: pgmap v11278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:34 smithi167 ceph-mon[32751]: pgmap v11278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:34 smithi090 ceph-mon[29180]: pgmap v11278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:36 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19937.287109 secs 2024-04-27T00:07:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:36 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19937.287109 secs 2024-04-27T00:07:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:36 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19937.287109 secs 2024-04-27T00:07:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:37 smithi119 ceph-mon[32752]: pgmap v11279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19938.072446 secs 2024-04-27T00:07:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:37 smithi167 ceph-mon[32751]: pgmap v11279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19938.072446 secs 2024-04-27T00:07:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:37 smithi090 ceph-mon[29180]: pgmap v11279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19938.072446 secs 2024-04-27T00:07:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:39 smithi119 ceph-mon[32752]: pgmap v11280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:39 smithi167 ceph-mon[32751]: pgmap v11280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:39 smithi090 ceph-mon[29180]: pgmap v11280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:41 smithi119 ceph-mon[32752]: pgmap v11281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:41 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19942.287190 secs 2024-04-27T00:07:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:41 smithi167 ceph-mon[32751]: pgmap v11281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:41 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19942.287190 secs 2024-04-27T00:07:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:41 smithi090 ceph-mon[29180]: pgmap v11281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:41 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19942.287190 secs 2024-04-27T00:07:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19943.072581 secs 2024-04-27T00:07:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19943.072581 secs 2024-04-27T00:07:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19943.072581 secs 2024-04-27T00:07:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:43 smithi119 ceph-mon[32752]: pgmap v11282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:43 smithi167 ceph-mon[32751]: pgmap v11282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:43 smithi090 ceph-mon[29180]: pgmap v11282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:45 smithi090 ceph-mon[29180]: pgmap v11283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:45 smithi119 ceph-mon[32752]: pgmap v11283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:45 smithi167 ceph-mon[32751]: pgmap v11283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:46 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19947.287274 secs 2024-04-27T00:07:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:46 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19947.287274 secs 2024-04-27T00:07:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:46 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19947.287274 secs 2024-04-27T00:07:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:47 smithi119 ceph-mon[32752]: pgmap v11284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19948.072659 secs 2024-04-27T00:07:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:47 smithi167 ceph-mon[32751]: pgmap v11284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19948.072659 secs 2024-04-27T00:07:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:47 smithi090 ceph-mon[29180]: pgmap v11284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19948.072659 secs 2024-04-27T00:07:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:48 smithi119 ceph-mon[32752]: pgmap v11285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:48 smithi167 ceph-mon[32751]: pgmap v11285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:48 smithi090 ceph-mon[29180]: pgmap v11285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:07:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:50 smithi167 ceph-mon[32751]: pgmap v11286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19952.287359 secs 2024-04-27T00:07:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:50 smithi090 ceph-mon[29180]: pgmap v11286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19952.287359 secs 2024-04-27T00:07:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:50 smithi119 ceph-mon[32752]: pgmap v11286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19952.287359 secs 2024-04-27T00:07:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19953.072767 secs 2024-04-27T00:07:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19953.072767 secs 2024-04-27T00:07:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19953.072767 secs 2024-04-27T00:07:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:52 smithi090 ceph-mon[29180]: pgmap v11287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:07:53.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:52 smithi119 ceph-mon[32752]: pgmap v11287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:07:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:52 smithi167 ceph-mon[32751]: pgmap v11287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:07:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:54 smithi090 ceph-mon[29180]: pgmap v11288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:55.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:54 smithi119 ceph-mon[32752]: pgmap v11288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:55.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:54 smithi167 ceph-mon[32751]: pgmap v11288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:07:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19957.287443 secs 2024-04-27T00:07:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19957.287443 secs 2024-04-27T00:07:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19957.287443 secs 2024-04-27T00:07:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:56 smithi090 ceph-mon[29180]: pgmap v11289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:07:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:07:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19958.072854 secs 2024-04-27T00:07:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:07:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:07:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:07:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:56 smithi119 ceph-mon[32752]: pgmap v11289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:07:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:07:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19958.072854 secs 2024-04-27T00:07:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:07:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:07:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:56 smithi167 ceph-mon[32751]: pgmap v11289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19958.072854 secs 2024-04-27T00:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:07:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:07:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:57 smithi090 ceph-mon[29180]: pgmap v11290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T00:07:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:57 smithi090 ceph-mon[29180]: pgmap v11291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:07:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:57 smithi119 ceph-mon[32752]: pgmap v11290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T00:07:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:57 smithi119 ceph-mon[32752]: pgmap v11291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:07:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:57 smithi167 ceph-mon[32751]: pgmap v11290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T00:07:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:57 smithi167 ceph-mon[32751]: pgmap v11291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:07:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:07:59 smithi090 ceph-mon[29180]: pgmap v11292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:07:59 smithi119 ceph-mon[32752]: pgmap v11292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:07:59 smithi167 ceph-mon[32751]: pgmap v11292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19962.287529 secs 2024-04-27T00:08:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19962.287529 secs 2024-04-27T00:08:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19962.287529 secs 2024-04-27T00:08:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:01 smithi090 ceph-mon[29180]: pgmap v11293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19963.072990 secs 2024-04-27T00:08:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:01 smithi119 ceph-mon[32752]: pgmap v11293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19963.072990 secs 2024-04-27T00:08:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:01 smithi167 ceph-mon[32751]: pgmap v11293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19963.072990 secs 2024-04-27T00:08:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:03 smithi090 ceph-mon[29180]: pgmap v11294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:08:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:03 smithi119 ceph-mon[32752]: pgmap v11294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:08:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:03 smithi167 ceph-mon[32751]: pgmap v11294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:08:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:05 smithi090 ceph-mon[29180]: pgmap v11295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:05 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19967.287613 secs 2024-04-27T00:08:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:05 smithi119 ceph-mon[32752]: pgmap v11295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:05 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19967.287613 secs 2024-04-27T00:08:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:05 smithi167 ceph-mon[32751]: pgmap v11295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:08:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:05 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19967.287613 secs 2024-04-27T00:08:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19968.073108 secs 2024-04-27T00:08:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19968.073108 secs 2024-04-27T00:08:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19968.073108 secs 2024-04-27T00:08:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:07 smithi090 ceph-mon[29180]: pgmap v11296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:08:08.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:07 smithi119 ceph-mon[32752]: pgmap v11296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:08:08.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:07 smithi167 ceph-mon[32751]: pgmap v11296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:08:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:09 smithi090 ceph-mon[29180]: pgmap v11297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:09 smithi119 ceph-mon[32752]: pgmap v11297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:09 smithi167 ceph-mon[32751]: pgmap v11297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:10 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19972.287698 secs 2024-04-27T00:08:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:10 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19972.287698 secs 2024-04-27T00:08:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:10 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19972.287698 secs 2024-04-27T00:08:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:11 smithi090 ceph-mon[29180]: pgmap v11298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19973.073226 secs 2024-04-27T00:08:11.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:11 smithi167 ceph-mon[32751]: pgmap v11298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:11.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19973.073226 secs 2024-04-27T00:08:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:11 smithi119 ceph-mon[32752]: pgmap v11298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19973.073226 secs 2024-04-27T00:08:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:13 smithi119 ceph-mon[32752]: pgmap v11299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:13 smithi167 ceph-mon[32751]: pgmap v11299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:13 smithi090 ceph-mon[29180]: pgmap v11299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:15 smithi119 ceph-mon[32752]: pgmap v11300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19977.287782 secs 2024-04-27T00:08:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:15 smithi167 ceph-mon[32751]: pgmap v11300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19977.287782 secs 2024-04-27T00:08:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:15 smithi090 ceph-mon[29180]: pgmap v11300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19977.287782 secs 2024-04-27T00:08:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19978.073347 secs 2024-04-27T00:08:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19978.073347 secs 2024-04-27T00:08:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19978.073347 secs 2024-04-27T00:08:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:17 smithi119 ceph-mon[32752]: pgmap v11301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:17 smithi167 ceph-mon[32751]: pgmap v11301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:17 smithi090 ceph-mon[29180]: pgmap v11301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:19 smithi119 ceph-mon[32752]: pgmap v11302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:19 smithi167 ceph-mon[32751]: pgmap v11302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:19 smithi090 ceph-mon[29180]: pgmap v11302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19982.287830 secs 2024-04-27T00:08:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19982.287830 secs 2024-04-27T00:08:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19982.287830 secs 2024-04-27T00:08:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:21 smithi119 ceph-mon[32752]: pgmap v11303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:21.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19983.073466 secs 2024-04-27T00:08:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:21 smithi167 ceph-mon[32751]: pgmap v11303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19983.073466 secs 2024-04-27T00:08:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:21 smithi090 ceph-mon[29180]: pgmap v11303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19983.073466 secs 2024-04-27T00:08:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:23 smithi119 ceph-mon[32752]: pgmap v11304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:23 smithi167 ceph-mon[32751]: pgmap v11304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:23 smithi090 ceph-mon[29180]: pgmap v11304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:25 smithi119 ceph-mon[32752]: pgmap v11305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19987.287913 secs 2024-04-27T00:08:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:25 smithi167 ceph-mon[32751]: pgmap v11305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19987.287913 secs 2024-04-27T00:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:25 smithi090 ceph-mon[29180]: pgmap v11305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19987.287913 secs 2024-04-27T00:08:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19988.073558 secs 2024-04-27T00:08:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19988.073558 secs 2024-04-27T00:08:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19988.073558 secs 2024-04-27T00:08:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:27 smithi119 ceph-mon[32752]: pgmap v11306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:27 smithi167 ceph-mon[32751]: pgmap v11306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:27 smithi090 ceph-mon[29180]: pgmap v11306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:29 smithi119 ceph-mon[32752]: pgmap v11307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:29 smithi167 ceph-mon[32751]: pgmap v11307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:29 smithi090 ceph-mon[29180]: pgmap v11307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:31.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:30 smithi090 ceph-mon[29180]: 29 slow requests, 1 included below; oldest blocked for > 19992.287997 secs 2024-04-27T00:08:31.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:30 smithi090 ceph-mon[29180]: slow request 1443.792105 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:08:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:30 smithi119 ceph-mon[32752]: 29 slow requests, 1 included below; oldest blocked for > 19992.287997 secs 2024-04-27T00:08:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:30 smithi119 ceph-mon[32752]: slow request 1443.792105 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:08:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:30 smithi167 ceph-mon[32751]: 29 slow requests, 1 included below; oldest blocked for > 19992.287997 secs 2024-04-27T00:08:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:30 smithi167 ceph-mon[32751]: slow request 1443.792105 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:08:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:31 smithi090 ceph-mon[29180]: pgmap v11308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19993.073675 secs 2024-04-27T00:08:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:31 smithi119 ceph-mon[32752]: pgmap v11308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19993.073675 secs 2024-04-27T00:08:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:31 smithi167 ceph-mon[32751]: pgmap v11308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19993.073675 secs 2024-04-27T00:08:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:33 smithi119 ceph-mon[32752]: pgmap v11309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:33 smithi167 ceph-mon[32751]: pgmap v11309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:33 smithi090 ceph-mon[29180]: pgmap v11309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:35 smithi119 ceph-mon[32752]: pgmap v11310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 19997.288056 secs 2024-04-27T00:08:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:35 smithi167 ceph-mon[32751]: pgmap v11310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 19997.288056 secs 2024-04-27T00:08:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:35 smithi090 ceph-mon[29180]: pgmap v11310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 19997.288056 secs 2024-04-27T00:08:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 19998.073795 secs 2024-04-27T00:08:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 19998.073795 secs 2024-04-27T00:08:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 19998.073795 secs 2024-04-27T00:08:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:37 smithi119 ceph-mon[32752]: pgmap v11311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:37 smithi167 ceph-mon[32751]: pgmap v11311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:37 smithi090 ceph-mon[29180]: pgmap v11311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:39 smithi119 ceph-mon[32752]: pgmap v11312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:39 smithi167 ceph-mon[32751]: pgmap v11312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:39 smithi090 ceph-mon[29180]: pgmap v11312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20002.288138 secs 2024-04-27T00:08:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20002.288138 secs 2024-04-27T00:08:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20002.288138 secs 2024-04-27T00:08:41.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:41 smithi167 ceph-mon[32751]: pgmap v11313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:41.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20003.073869 secs 2024-04-27T00:08:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:41 smithi119 ceph-mon[32752]: pgmap v11313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20003.073869 secs 2024-04-27T00:08:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:41 smithi090 ceph-mon[29180]: pgmap v11313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20003.073869 secs 2024-04-27T00:08:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:43 smithi119 ceph-mon[32752]: pgmap v11314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:43 smithi167 ceph-mon[32751]: pgmap v11314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:43 smithi090 ceph-mon[29180]: pgmap v11314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:45 smithi090 ceph-mon[29180]: pgmap v11315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20007.288221 secs 2024-04-27T00:08:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:45 smithi119 ceph-mon[32752]: pgmap v11315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20007.288221 secs 2024-04-27T00:08:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:45 smithi167 ceph-mon[32751]: pgmap v11315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20007.288221 secs 2024-04-27T00:08:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20008.074004 secs 2024-04-27T00:08:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20008.074004 secs 2024-04-27T00:08:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20008.074004 secs 2024-04-27T00:08:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:47 smithi090 ceph-mon[29180]: pgmap v11316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:47 smithi119 ceph-mon[32752]: pgmap v11316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:47 smithi167 ceph-mon[32751]: pgmap v11316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:49 smithi090 ceph-mon[29180]: pgmap v11317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:49 smithi119 ceph-mon[32752]: pgmap v11317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:49 smithi167 ceph-mon[32751]: pgmap v11317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20012.288306 secs 2024-04-27T00:08:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20012.288306 secs 2024-04-27T00:08:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20012.288306 secs 2024-04-27T00:08:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:51 smithi090 ceph-mon[29180]: pgmap v11318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20013.074112 secs 2024-04-27T00:08:51.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:51 smithi119 ceph-mon[32752]: pgmap v11318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:51.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20013.074112 secs 2024-04-27T00:08:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:51 smithi167 ceph-mon[32751]: pgmap v11318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20013.074112 secs 2024-04-27T00:08:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:53 smithi090 ceph-mon[29180]: pgmap v11319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:53 smithi119 ceph-mon[32752]: pgmap v11319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:53 smithi167 ceph-mon[32751]: pgmap v11319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:55 smithi090 ceph-mon[29180]: pgmap v11320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20017.288393 secs 2024-04-27T00:08:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:55 smithi119 ceph-mon[32752]: pgmap v11320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20017.288393 secs 2024-04-27T00:08:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:55 smithi167 ceph-mon[32751]: pgmap v11320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:08:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20017.288393 secs 2024-04-27T00:08:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20018.074209 secs 2024-04-27T00:08:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20018.074209 secs 2024-04-27T00:08:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20018.074209 secs 2024-04-27T00:08:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:57 smithi119 ceph-mon[32752]: pgmap v11321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:08:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:08:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:08:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:57 smithi119 ceph-mon[32752]: pgmap v11322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:08:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:57 smithi119 ceph-mon[32752]: pgmap v11323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:08:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:08:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:08:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:57 smithi167 ceph-mon[32751]: pgmap v11321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:08:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:08:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:08:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:57 smithi167 ceph-mon[32751]: pgmap v11322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:08:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:57 smithi167 ceph-mon[32751]: pgmap v11323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:08:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:08:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:08:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:57 smithi090 ceph-mon[29180]: pgmap v11321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:08:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:08:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:08:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:08:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:57 smithi090 ceph-mon[29180]: pgmap v11322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:08:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:57 smithi090 ceph-mon[29180]: pgmap v11323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:08:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:08:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:00 smithi090 ceph-mon[29180]: pgmap v11324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20022.288494 secs 2024-04-27T00:09:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:00 smithi119 ceph-mon[32752]: pgmap v11324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20022.288494 secs 2024-04-27T00:09:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:00 smithi167 ceph-mon[32751]: pgmap v11324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20022.288494 secs 2024-04-27T00:09:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20023.074324 secs 2024-04-27T00:09:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:02 smithi090 ceph-mon[29180]: pgmap v11325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20023.074324 secs 2024-04-27T00:09:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:02 smithi119 ceph-mon[32752]: pgmap v11325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20023.074324 secs 2024-04-27T00:09:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:02 smithi167 ceph-mon[32751]: pgmap v11325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:04 smithi090 ceph-mon[29180]: pgmap v11326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:09:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:04 smithi119 ceph-mon[32752]: pgmap v11326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:09:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:04 smithi167 ceph-mon[32751]: pgmap v11326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:09:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:05 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20027.288577 secs 2024-04-27T00:09:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:05 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20027.288577 secs 2024-04-27T00:09:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:05 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20027.288577 secs 2024-04-27T00:09:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:06 smithi090 ceph-mon[29180]: pgmap v11327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:06 smithi119 ceph-mon[32752]: pgmap v11327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:06 smithi167 ceph-mon[32751]: pgmap v11327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:09:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20028.074434 secs 2024-04-27T00:09:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20028.074434 secs 2024-04-27T00:09:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20028.074434 secs 2024-04-27T00:09:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:08 smithi119 ceph-mon[32752]: pgmap v11328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:09:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:08 smithi167 ceph-mon[32751]: pgmap v11328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:09:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:08 smithi090 ceph-mon[29180]: pgmap v11328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:09:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:10 smithi119 ceph-mon[32752]: pgmap v11329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:10 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20032.288656 secs 2024-04-27T00:09:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:10 smithi167 ceph-mon[32751]: pgmap v11329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:10 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20032.288656 secs 2024-04-27T00:09:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:10 smithi090 ceph-mon[29180]: pgmap v11329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:10 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20032.288656 secs 2024-04-27T00:09:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20033.074557 secs 2024-04-27T00:09:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:12 smithi119 ceph-mon[32752]: pgmap v11330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20033.074557 secs 2024-04-27T00:09:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:12 smithi167 ceph-mon[32751]: pgmap v11330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20033.074557 secs 2024-04-27T00:09:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:12 smithi090 ceph-mon[29180]: pgmap v11330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:14 smithi119 ceph-mon[32752]: pgmap v11331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:14 smithi167 ceph-mon[32751]: pgmap v11331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:14 smithi090 ceph-mon[29180]: pgmap v11331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20037.288737 secs 2024-04-27T00:09:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20037.288737 secs 2024-04-27T00:09:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20037.288737 secs 2024-04-27T00:09:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:16 smithi119 ceph-mon[32752]: pgmap v11332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:16 smithi167 ceph-mon[32751]: pgmap v11332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:16 smithi090 ceph-mon[29180]: pgmap v11332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20038.074660 secs 2024-04-27T00:09:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20038.074660 secs 2024-04-27T00:09:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20038.074660 secs 2024-04-27T00:09:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:18 smithi119 ceph-mon[32752]: pgmap v11333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:18.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:18 smithi090 ceph-mon[29180]: pgmap v11333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:18.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:18 smithi167 ceph-mon[32751]: pgmap v11333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:20 smithi119 ceph-mon[32752]: pgmap v11334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20042.288821 secs 2024-04-27T00:09:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:20 smithi167 ceph-mon[32751]: pgmap v11334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20042.288821 secs 2024-04-27T00:09:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:20 smithi090 ceph-mon[29180]: pgmap v11334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20042.288821 secs 2024-04-27T00:09:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20043.074801 secs 2024-04-27T00:09:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:22 smithi119 ceph-mon[32752]: pgmap v11335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20043.074801 secs 2024-04-27T00:09:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:22 smithi167 ceph-mon[32751]: pgmap v11335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20043.074801 secs 2024-04-27T00:09:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:22 smithi090 ceph-mon[29180]: pgmap v11335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:24 smithi119 ceph-mon[32752]: pgmap v11336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:24 smithi167 ceph-mon[32751]: pgmap v11336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:24 smithi090 ceph-mon[29180]: pgmap v11336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20047.288906 secs 2024-04-27T00:09:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20047.288906 secs 2024-04-27T00:09:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20047.288906 secs 2024-04-27T00:09:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:26 smithi119 ceph-mon[32752]: pgmap v11337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:26 smithi167 ceph-mon[32751]: pgmap v11337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:26 smithi090 ceph-mon[29180]: pgmap v11337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20048.074921 secs 2024-04-27T00:09:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20048.074921 secs 2024-04-27T00:09:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20048.074921 secs 2024-04-27T00:09:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:28 smithi119 ceph-mon[32752]: pgmap v11338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:28 smithi167 ceph-mon[32751]: pgmap v11338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:28 smithi090 ceph-mon[29180]: pgmap v11338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:30 smithi119 ceph-mon[32752]: pgmap v11339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:30 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20052.288994 secs 2024-04-27T00:09:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:30 smithi167 ceph-mon[32751]: pgmap v11339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:30 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20052.288994 secs 2024-04-27T00:09:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:30 smithi090 ceph-mon[29180]: pgmap v11339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:30 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20052.288994 secs 2024-04-27T00:09:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20053.075044 secs 2024-04-27T00:09:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:32 smithi119 ceph-mon[32752]: pgmap v11340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20053.075044 secs 2024-04-27T00:09:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:32 smithi167 ceph-mon[32751]: pgmap v11340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20053.075044 secs 2024-04-27T00:09:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:32 smithi090 ceph-mon[29180]: pgmap v11340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:34 smithi119 ceph-mon[32752]: pgmap v11341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:34 smithi167 ceph-mon[32751]: pgmap v11341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:34 smithi090 ceph-mon[29180]: pgmap v11341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20057.289053 secs 2024-04-27T00:09:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20057.289053 secs 2024-04-27T00:09:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20057.289053 secs 2024-04-27T00:09:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:36 smithi119 ceph-mon[32752]: pgmap v11342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:36 smithi167 ceph-mon[32751]: pgmap v11342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:36 smithi090 ceph-mon[29180]: pgmap v11342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20058.075156 secs 2024-04-27T00:09:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:37 smithi119 ceph-mon[32752]: pgmap v11343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20058.075156 secs 2024-04-27T00:09:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:37 smithi167 ceph-mon[32751]: pgmap v11343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:37.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20058.075156 secs 2024-04-27T00:09:37.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:37 smithi090 ceph-mon[29180]: pgmap v11343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:40 smithi090 ceph-mon[29180]: pgmap v11344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20062.289132 secs 2024-04-27T00:09:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:40 smithi119 ceph-mon[32752]: pgmap v11344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20062.289132 secs 2024-04-27T00:09:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:40 smithi167 ceph-mon[32751]: pgmap v11344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20062.289132 secs 2024-04-27T00:09:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20063.075291 secs 2024-04-27T00:09:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:42 smithi090 ceph-mon[29180]: pgmap v11345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20063.075291 secs 2024-04-27T00:09:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:42 smithi119 ceph-mon[32752]: pgmap v11345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20063.075291 secs 2024-04-27T00:09:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:42 smithi167 ceph-mon[32751]: pgmap v11345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:44 smithi119 ceph-mon[32752]: pgmap v11346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:44 smithi167 ceph-mon[32751]: pgmap v11346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:44 smithi090 ceph-mon[29180]: pgmap v11346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20067.289208 secs 2024-04-27T00:09:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20067.289208 secs 2024-04-27T00:09:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20067.289208 secs 2024-04-27T00:09:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:46 smithi119 ceph-mon[32752]: pgmap v11347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:46 smithi167 ceph-mon[32751]: pgmap v11347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:46 smithi090 ceph-mon[29180]: pgmap v11347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20068.075443 secs 2024-04-27T00:09:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20068.075443 secs 2024-04-27T00:09:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20068.075443 secs 2024-04-27T00:09:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:48 smithi119 ceph-mon[32752]: pgmap v11348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:48 smithi167 ceph-mon[32751]: pgmap v11348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:48 smithi090 ceph-mon[29180]: pgmap v11348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:50 smithi119 ceph-mon[32752]: pgmap v11349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20072.289283 secs 2024-04-27T00:09:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:50 smithi167 ceph-mon[32751]: pgmap v11349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20072.289283 secs 2024-04-27T00:09:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:50 smithi090 ceph-mon[29180]: pgmap v11349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20072.289283 secs 2024-04-27T00:09:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20073.075507 secs 2024-04-27T00:09:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:52 smithi119 ceph-mon[32752]: pgmap v11350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20073.075507 secs 2024-04-27T00:09:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:52 smithi167 ceph-mon[32751]: pgmap v11350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20073.075507 secs 2024-04-27T00:09:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:52 smithi090 ceph-mon[29180]: pgmap v11350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:54 smithi119 ceph-mon[32752]: pgmap v11351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:54 smithi167 ceph-mon[32751]: pgmap v11351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:54 smithi090 ceph-mon[29180]: pgmap v11351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20077.289382 secs 2024-04-27T00:09:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20077.289382 secs 2024-04-27T00:09:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20077.289382 secs 2024-04-27T00:09:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:56 smithi119 ceph-mon[32752]: pgmap v11352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:56 smithi167 ceph-mon[32751]: pgmap v11352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:56 smithi090 ceph-mon[29180]: pgmap v11352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:09:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20078.075629 secs 2024-04-27T00:09:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20078.075629 secs 2024-04-27T00:09:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20078.075629 secs 2024-04-27T00:09:58.462 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:58 smithi090 ceph-mon[29180]: pgmap v11353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:58.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:09:58.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.463 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:09:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:58 smithi167 ceph-mon[32751]: pgmap v11353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:58.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:09:58.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:09:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.495 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:58 smithi119 ceph-mon[32752]: pgmap v11353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:09:58.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:09:58.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:09:58.496 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:09:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: pgmap v11354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:10:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20082.289473 secs 2024-04-27T00:10:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: pgmap v11354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:10:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20082.289473 secs 2024-04-27T00:10:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: pgmap v11354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:10:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20082.289473 secs 2024-04-27T00:10:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:10:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:10:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:10:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:10:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:10:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:10:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:10:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:10:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: pgmap v11355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: pgmap v11356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:02.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20083.075744 secs 2024-04-27T00:10:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:10:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:10:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:10:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:10:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:10:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:10:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: pgmap v11355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T00:10:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: pgmap v11356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20083.075744 secs 2024-04-27T00:10:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:10:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:10:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:10:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:10:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:10:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:10:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: pgmap v11355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T00:10:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: pgmap v11356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20083.075744 secs 2024-04-27T00:10:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:04 smithi119 ceph-mon[32752]: pgmap v11357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:04 smithi167 ceph-mon[32751]: pgmap v11357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:04 smithi090 ceph-mon[29180]: pgmap v11357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:05 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20087.289530 secs 2024-04-27T00:10:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:05 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20087.289530 secs 2024-04-27T00:10:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:05 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20087.289530 secs 2024-04-27T00:10:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:06 smithi119 ceph-mon[32752]: pgmap v11358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:06 smithi167 ceph-mon[32751]: pgmap v11358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:06 smithi090 ceph-mon[29180]: pgmap v11358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20088.075856 secs 2024-04-27T00:10:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20088.075856 secs 2024-04-27T00:10:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20088.075856 secs 2024-04-27T00:10:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:08 smithi119 ceph-mon[32752]: pgmap v11359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:08 smithi167 ceph-mon[32751]: pgmap v11359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:08 smithi090 ceph-mon[29180]: pgmap v11359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:10 smithi119 ceph-mon[32752]: pgmap v11360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:10 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20092.289615 secs 2024-04-27T00:10:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:10 smithi167 ceph-mon[32751]: pgmap v11360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:10 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20092.289615 secs 2024-04-27T00:10:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:10 smithi090 ceph-mon[29180]: pgmap v11360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:10:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:10 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20092.289615 secs 2024-04-27T00:10:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:12 smithi119 ceph-mon[32752]: pgmap v11361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:10:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20093.075929 secs 2024-04-27T00:10:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:12 smithi167 ceph-mon[32751]: pgmap v11361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:10:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20093.075929 secs 2024-04-27T00:10:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:12 smithi090 ceph-mon[29180]: pgmap v11361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:10:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20093.075929 secs 2024-04-27T00:10:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:13 smithi119 ceph-mon[32752]: pgmap v11362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:13 smithi167 ceph-mon[32751]: pgmap v11362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:13 smithi090 ceph-mon[29180]: pgmap v11362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:15 smithi090 ceph-mon[29180]: pgmap v11363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20097.289699 secs 2024-04-27T00:10:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:15 smithi119 ceph-mon[32752]: pgmap v11363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20097.289699 secs 2024-04-27T00:10:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:15 smithi167 ceph-mon[32751]: pgmap v11363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20097.289699 secs 2024-04-27T00:10:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20098.076053 secs 2024-04-27T00:10:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20098.076053 secs 2024-04-27T00:10:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20098.076053 secs 2024-04-27T00:10:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:17 smithi090 ceph-mon[29180]: pgmap v11364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:17 smithi119 ceph-mon[32752]: pgmap v11364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:17 smithi167 ceph-mon[32751]: pgmap v11364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:19 smithi090 ceph-mon[29180]: pgmap v11365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:19 smithi119 ceph-mon[32752]: pgmap v11365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:19 smithi167 ceph-mon[32751]: pgmap v11365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20102.289784 secs 2024-04-27T00:10:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20102.289784 secs 2024-04-27T00:10:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20102.289784 secs 2024-04-27T00:10:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:21 smithi119 ceph-mon[32752]: pgmap v11366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20103.076187 secs 2024-04-27T00:10:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:21 smithi167 ceph-mon[32751]: pgmap v11366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20103.076187 secs 2024-04-27T00:10:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:21 smithi090 ceph-mon[29180]: pgmap v11366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20103.076187 secs 2024-04-27T00:10:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:23 smithi119 ceph-mon[32752]: pgmap v11367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:23 smithi167 ceph-mon[32751]: pgmap v11367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:23 smithi090 ceph-mon[29180]: pgmap v11367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:25 smithi119 ceph-mon[32752]: pgmap v11368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20107.289871 secs 2024-04-27T00:10:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:25 smithi167 ceph-mon[32751]: pgmap v11368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20107.289871 secs 2024-04-27T00:10:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:25 smithi090 ceph-mon[29180]: pgmap v11368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20107.289871 secs 2024-04-27T00:10:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20108.076303 secs 2024-04-27T00:10:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20108.076303 secs 2024-04-27T00:10:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20108.076303 secs 2024-04-27T00:10:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:27 smithi119 ceph-mon[32752]: pgmap v11369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:27 smithi167 ceph-mon[32751]: pgmap v11369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:27 smithi090 ceph-mon[29180]: pgmap v11369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:29 smithi119 ceph-mon[32752]: pgmap v11370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:29 smithi167 ceph-mon[32751]: pgmap v11370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:29 smithi090 ceph-mon[29180]: pgmap v11370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:30 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20112.289956 secs 2024-04-27T00:10:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:30 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20112.289956 secs 2024-04-27T00:10:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:30 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20112.289956 secs 2024-04-27T00:10:32.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:31 smithi090 ceph-mon[29180]: pgmap v11371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:32.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20113.076423 secs 2024-04-27T00:10:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:31 smithi119 ceph-mon[32752]: pgmap v11371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20113.076423 secs 2024-04-27T00:10:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:31 smithi167 ceph-mon[32751]: pgmap v11371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20113.076423 secs 2024-04-27T00:10:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:33 smithi119 ceph-mon[32752]: pgmap v11372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:33 smithi167 ceph-mon[32751]: pgmap v11372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:33 smithi090 ceph-mon[29180]: pgmap v11372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:35.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:35 smithi090 ceph-mon[29180]: pgmap v11373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:35.928 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20117.290017 secs 2024-04-27T00:10:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:35 smithi119 ceph-mon[32752]: pgmap v11373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20117.290017 secs 2024-04-27T00:10:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:35 smithi167 ceph-mon[32751]: pgmap v11373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20117.290017 secs 2024-04-27T00:10:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20118.076542 secs 2024-04-27T00:10:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20118.076542 secs 2024-04-27T00:10:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20118.076542 secs 2024-04-27T00:10:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:37 smithi119 ceph-mon[32752]: pgmap v11374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:37 smithi167 ceph-mon[32751]: pgmap v11374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:37 smithi090 ceph-mon[29180]: pgmap v11374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:39 smithi119 ceph-mon[32752]: pgmap v11375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:39 smithi167 ceph-mon[32751]: pgmap v11375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:39 smithi090 ceph-mon[29180]: pgmap v11375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20122.290103 secs 2024-04-27T00:10:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20122.290103 secs 2024-04-27T00:10:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20122.290103 secs 2024-04-27T00:10:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:41 smithi119 ceph-mon[32752]: pgmap v11376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20123.076662 secs 2024-04-27T00:10:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:41 smithi167 ceph-mon[32751]: pgmap v11376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20123.076662 secs 2024-04-27T00:10:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:41 smithi090 ceph-mon[29180]: pgmap v11376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20123.076662 secs 2024-04-27T00:10:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:43 smithi119 ceph-mon[32752]: pgmap v11377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:43 smithi167 ceph-mon[32751]: pgmap v11377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:43 smithi090 ceph-mon[29180]: pgmap v11377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:45 smithi119 ceph-mon[32752]: pgmap v11378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20127.290189 secs 2024-04-27T00:10:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:45 smithi167 ceph-mon[32751]: pgmap v11378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20127.290189 secs 2024-04-27T00:10:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:45 smithi090 ceph-mon[29180]: pgmap v11378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20127.290189 secs 2024-04-27T00:10:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20128.076827 secs 2024-04-27T00:10:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20128.076827 secs 2024-04-27T00:10:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20128.076827 secs 2024-04-27T00:10:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:47 smithi119 ceph-mon[32752]: pgmap v11379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:47 smithi167 ceph-mon[32751]: pgmap v11379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:47 smithi090 ceph-mon[29180]: pgmap v11379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:10:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:49 smithi119 ceph-mon[32752]: pgmap v11380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:49 smithi167 ceph-mon[32751]: pgmap v11380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:49 smithi090 ceph-mon[29180]: pgmap v11380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:10:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20132.290292 secs 2024-04-27T00:10:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20132.290292 secs 2024-04-27T00:10:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20132.290292 secs 2024-04-27T00:10:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:51 smithi119 ceph-mon[32752]: pgmap v11381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20133.076902 secs 2024-04-27T00:10:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:51 smithi167 ceph-mon[32751]: pgmap v11381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20133.076902 secs 2024-04-27T00:10:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:51 smithi090 ceph-mon[29180]: pgmap v11381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20133.076902 secs 2024-04-27T00:10:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:53 smithi119 ceph-mon[32752]: pgmap v11382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:53 smithi167 ceph-mon[32751]: pgmap v11382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:53 smithi090 ceph-mon[29180]: pgmap v11382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:55 smithi119 ceph-mon[32752]: pgmap v11383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:55 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20137.290377 secs 2024-04-27T00:10:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:55 smithi167 ceph-mon[32751]: pgmap v11383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:55 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20137.290377 secs 2024-04-27T00:10:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:55 smithi090 ceph-mon[29180]: pgmap v11383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:10:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:55 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20137.290377 secs 2024-04-27T00:10:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20138.077023 secs 2024-04-27T00:10:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20138.077023 secs 2024-04-27T00:10:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20138.077023 secs 2024-04-27T00:10:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:57 smithi119 ceph-mon[32752]: pgmap v11384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:57 smithi167 ceph-mon[32751]: pgmap v11384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:10:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:57 smithi090 ceph-mon[29180]: pgmap v11384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:10:59 smithi119 ceph-mon[32752]: pgmap v11385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:10:59 smithi167 ceph-mon[32751]: pgmap v11385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:10:59 smithi090 ceph-mon[29180]: pgmap v11385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:00 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20142.290463 secs 2024-04-27T00:11:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:11:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:00 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20142.290463 secs 2024-04-27T00:11:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:11:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:00 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20142.290463 secs 2024-04-27T00:11:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:02 smithi119 ceph-mon[32752]: pgmap v11386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20143.077127 secs 2024-04-27T00:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:11:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:02 smithi167 ceph-mon[32751]: pgmap v11386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20143.077127 secs 2024-04-27T00:11:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:11:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:11:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:02 smithi090 ceph-mon[29180]: pgmap v11386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20143.077127 secs 2024-04-27T00:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:11:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:11:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:03 smithi119 ceph-mon[32752]: pgmap v11387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T00:11:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:03 smithi119 ceph-mon[32752]: pgmap v11388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:03 smithi167 ceph-mon[32751]: pgmap v11387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T00:11:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:03 smithi167 ceph-mon[32751]: pgmap v11388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:03 smithi090 ceph-mon[29180]: pgmap v11387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T00:11:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:03 smithi090 ceph-mon[29180]: pgmap v11388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:05 smithi119 ceph-mon[32752]: pgmap v11389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:05 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 20147.290547 secs 2024-04-27T00:11:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:05 smithi167 ceph-mon[32751]: pgmap v11389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:05 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 20147.290547 secs 2024-04-27T00:11:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:05 smithi090 ceph-mon[29180]: pgmap v11389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:05 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 20147.290547 secs 2024-04-27T00:11:06.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:11:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:11:06.337+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:11:06.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:11:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:11:06.337+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:11:06.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:11:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:11:06.337+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:11:06.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:11:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:11:06.337+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:11:06.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:11:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:11:06.337+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:11:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:07 smithi119 ceph-mon[32752]: pgmap v11390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20148.077200 secs 2024-04-27T00:11:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:07 smithi167 ceph-mon[32751]: pgmap v11390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20148.077200 secs 2024-04-27T00:11:07.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:07 smithi090 ceph-mon[29180]: pgmap v11390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:11:07.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20148.077200 secs 2024-04-27T00:11:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:09 smithi119 ceph-mon[32752]: pgmap v11391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T00:11:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:09 smithi167 ceph-mon[32751]: pgmap v11391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T00:11:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:09 smithi090 ceph-mon[29180]: pgmap v11391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T00:11:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:10 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20152.290633 secs 2024-04-27T00:11:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:10 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20152.290633 secs 2024-04-27T00:11:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:10 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20152.290633 secs 2024-04-27T00:11:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:11 smithi119 ceph-mon[32752]: pgmap v11392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 443 B/s wr, 17 op/s 2024-04-27T00:11:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:11 smithi167 ceph-mon[32751]: pgmap v11392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 443 B/s wr, 17 op/s 2024-04-27T00:11:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:11 smithi090 ceph-mon[29180]: pgmap v11392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 443 B/s wr, 17 op/s 2024-04-27T00:11:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20153.077310 secs 2024-04-27T00:11:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:12 smithi119 ceph-mon[32752]: pgmap v11393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-27T00:11:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20153.077310 secs 2024-04-27T00:11:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:12 smithi167 ceph-mon[32751]: pgmap v11393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-27T00:11:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20153.077310 secs 2024-04-27T00:11:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:12 smithi090 ceph-mon[29180]: pgmap v11393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-27T00:11:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:14 smithi090 ceph-mon[29180]: pgmap v11394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:11:15.271 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:14 smithi167 ceph-mon[32751]: pgmap v11394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:11:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:14 smithi119 ceph-mon[32752]: pgmap v11394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:11:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:11:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:11:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:15 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20157.290696 secs 2024-04-27T00:11:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:11:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:11:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:15 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20157.290696 secs 2024-04-27T00:11:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:11:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:11:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:15 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20157.290696 secs 2024-04-27T00:11:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:16 smithi119 ceph-mon[32752]: pgmap v11395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:11:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20158.077446 secs 2024-04-27T00:11:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:16 smithi167 ceph-mon[32751]: pgmap v11395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:11:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20158.077446 secs 2024-04-27T00:11:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:16 smithi090 ceph-mon[29180]: pgmap v11395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:11:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20158.077446 secs 2024-04-27T00:11:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:18 smithi119 ceph-mon[32752]: pgmap v11396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:11:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:18 smithi167 ceph-mon[32751]: pgmap v11396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:11:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:18 smithi090 ceph-mon[29180]: pgmap v11396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:11:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:20 smithi119 ceph-mon[32752]: pgmap v11397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:11:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:20 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20162.290783 secs 2024-04-27T00:11:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:20 smithi167 ceph-mon[32751]: pgmap v11397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:11:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:20 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20162.290783 secs 2024-04-27T00:11:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:20 smithi090 ceph-mon[29180]: pgmap v11397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:11:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:20 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20162.290783 secs 2024-04-27T00:11:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20163.077581 secs 2024-04-27T00:11:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20163.077581 secs 2024-04-27T00:11:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20163.077581 secs 2024-04-27T00:11:23.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:22 smithi119 ceph-mon[32752]: pgmap v11398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:22 smithi167 ceph-mon[32751]: pgmap v11398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:22 smithi090 ceph-mon[29180]: pgmap v11398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:24 smithi119 ceph-mon[32752]: pgmap v11399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:24 smithi167 ceph-mon[32751]: pgmap v11399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:24 smithi090 ceph-mon[29180]: pgmap v11399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:25 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20167.290873 secs 2024-04-27T00:11:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:25 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20167.290873 secs 2024-04-27T00:11:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:25 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20167.290873 secs 2024-04-27T00:11:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:26 smithi119 ceph-mon[32752]: pgmap v11400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20168.077656 secs 2024-04-27T00:11:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:26 smithi167 ceph-mon[32751]: pgmap v11400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20168.077656 secs 2024-04-27T00:11:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:26 smithi090 ceph-mon[29180]: pgmap v11400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20168.077656 secs 2024-04-27T00:11:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:28 smithi119 ceph-mon[32752]: pgmap v11401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:28 smithi167 ceph-mon[32751]: pgmap v11401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:28 smithi090 ceph-mon[29180]: pgmap v11401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:30 smithi119 ceph-mon[32752]: pgmap v11402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:30 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20172.290963 secs 2024-04-27T00:11:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:30 smithi167 ceph-mon[32751]: pgmap v11402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:30 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20172.290963 secs 2024-04-27T00:11:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:30 smithi090 ceph-mon[29180]: pgmap v11402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:30 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20172.290963 secs 2024-04-27T00:11:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20173.077804 secs 2024-04-27T00:11:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20173.077804 secs 2024-04-27T00:11:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20173.077804 secs 2024-04-27T00:11:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:32 smithi119 ceph-mon[32752]: pgmap v11403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:32 smithi167 ceph-mon[32751]: pgmap v11403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:32 smithi090 ceph-mon[29180]: pgmap v11403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:34 smithi119 ceph-mon[32752]: pgmap v11404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:34 smithi167 ceph-mon[32751]: pgmap v11404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:34 smithi090 ceph-mon[29180]: pgmap v11404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:35 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20177.291022 secs 2024-04-27T00:11:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:35 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20177.291022 secs 2024-04-27T00:11:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:35 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20177.291022 secs 2024-04-27T00:11:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:36 smithi119 ceph-mon[32752]: pgmap v11405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20178.077878 secs 2024-04-27T00:11:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:36 smithi167 ceph-mon[32751]: pgmap v11405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20178.077878 secs 2024-04-27T00:11:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:36 smithi090 ceph-mon[29180]: pgmap v11405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20178.077878 secs 2024-04-27T00:11:39.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:38 smithi119 ceph-mon[32752]: pgmap v11406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:38 smithi167 ceph-mon[32751]: pgmap v11406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:38 smithi090 ceph-mon[29180]: pgmap v11406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:40 smithi119 ceph-mon[32752]: pgmap v11407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:40 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20182.291106 secs 2024-04-27T00:11:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:40 smithi167 ceph-mon[32751]: pgmap v11407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:40 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20182.291106 secs 2024-04-27T00:11:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:40 smithi090 ceph-mon[29180]: pgmap v11407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:40 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20182.291106 secs 2024-04-27T00:11:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20183.077947 secs 2024-04-27T00:11:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20183.077947 secs 2024-04-27T00:11:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20183.077947 secs 2024-04-27T00:11:43.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:42 smithi119 ceph-mon[32752]: pgmap v11408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:42 smithi167 ceph-mon[32751]: pgmap v11408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:42 smithi090 ceph-mon[29180]: pgmap v11408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:44 smithi119 ceph-mon[32752]: pgmap v11409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:44 smithi167 ceph-mon[32751]: pgmap v11409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:44 smithi090 ceph-mon[29180]: pgmap v11409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:45 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20187.291190 secs 2024-04-27T00:11:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:45 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20187.291190 secs 2024-04-27T00:11:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:45 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20187.291190 secs 2024-04-27T00:11:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:46 smithi119 ceph-mon[32752]: pgmap v11410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20188.078068 secs 2024-04-27T00:11:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:46 smithi167 ceph-mon[32751]: pgmap v11410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20188.078068 secs 2024-04-27T00:11:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:46 smithi090 ceph-mon[29180]: pgmap v11410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20188.078068 secs 2024-04-27T00:11:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:48 smithi119 ceph-mon[32752]: pgmap v11411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:48 smithi167 ceph-mon[32751]: pgmap v11411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:48 smithi090 ceph-mon[29180]: pgmap v11411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:50 smithi119 ceph-mon[32752]: pgmap v11412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:50 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20192.291274 secs 2024-04-27T00:11:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:50 smithi167 ceph-mon[32751]: pgmap v11412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:50 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20192.291274 secs 2024-04-27T00:11:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:50 smithi090 ceph-mon[29180]: pgmap v11412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:50 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20192.291274 secs 2024-04-27T00:11:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20193.078193 secs 2024-04-27T00:11:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20193.078193 secs 2024-04-27T00:11:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20193.078193 secs 2024-04-27T00:11:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:52 smithi119 ceph-mon[32752]: pgmap v11413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:52 smithi167 ceph-mon[32751]: pgmap v11413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:52 smithi090 ceph-mon[29180]: pgmap v11413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:54 smithi119 ceph-mon[32752]: pgmap v11414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:54 smithi167 ceph-mon[32751]: pgmap v11414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:54 smithi090 ceph-mon[29180]: pgmap v11414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:11:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:55 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20197.291359 secs 2024-04-27T00:11:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:55 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20197.291359 secs 2024-04-27T00:11:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:55 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20197.291359 secs 2024-04-27T00:11:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:56 smithi119 ceph-mon[32752]: pgmap v11415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20198.078333 secs 2024-04-27T00:11:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:56 smithi167 ceph-mon[32751]: pgmap v11415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20198.078333 secs 2024-04-27T00:11:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:56 smithi090 ceph-mon[29180]: pgmap v11415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20198.078333 secs 2024-04-27T00:11:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:11:58 smithi119 ceph-mon[32752]: pgmap v11416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:11:58 smithi167 ceph-mon[32751]: pgmap v11416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:11:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:11:58 smithi090 ceph-mon[29180]: pgmap v11416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:00 smithi119 ceph-mon[32752]: pgmap v11417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:00 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20202.291444 secs 2024-04-27T00:12:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:00 smithi167 ceph-mon[32751]: pgmap v11417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:00 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20202.291444 secs 2024-04-27T00:12:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:00 smithi090 ceph-mon[29180]: pgmap v11417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:00 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20202.291444 secs 2024-04-27T00:12:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20203.078426 secs 2024-04-27T00:12:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:12:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20203.078426 secs 2024-04-27T00:12:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:12:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20203.078426 secs 2024-04-27T00:12:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:12:03.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:02 smithi119 ceph-mon[32752]: pgmap v11418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:12:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:12:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:12:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:02 smithi167 ceph-mon[32751]: pgmap v11418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:12:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:12:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:12:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:02 smithi090 ceph-mon[29180]: pgmap v11418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:12:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:12:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:12:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:03 smithi119 ceph-mon[32752]: pgmap v11419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:12:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:03 smithi119 ceph-mon[32752]: pgmap v11420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:12:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:03 smithi167 ceph-mon[32751]: pgmap v11419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:12:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:03 smithi167 ceph-mon[32751]: pgmap v11420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:12:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:03 smithi090 ceph-mon[29180]: pgmap v11419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:12:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:03 smithi090 ceph-mon[29180]: pgmap v11420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:12:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:06 smithi119 ceph-mon[32752]: pgmap v11421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20207.291528 secs 2024-04-27T00:12:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:06 smithi167 ceph-mon[32751]: pgmap v11421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20207.291528 secs 2024-04-27T00:12:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:06 smithi090 ceph-mon[29180]: pgmap v11421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20207.291528 secs 2024-04-27T00:12:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:07 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 20208.078516 secs 2024-04-27T00:12:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:07 smithi119 ceph-mon[32752]: slow request 11522.715283 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:12:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:07 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 20208.078516 secs 2024-04-27T00:12:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:07 smithi167 ceph-mon[32751]: slow request 11522.715283 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:12:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:07 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 20208.078516 secs 2024-04-27T00:12:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:07 smithi090 ceph-mon[29180]: slow request 11522.715283 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:12:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:08 smithi119 ceph-mon[32752]: pgmap v11422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:08 smithi167 ceph-mon[32751]: pgmap v11422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:08 smithi090 ceph-mon[29180]: pgmap v11422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:10 smithi119 ceph-mon[32752]: pgmap v11423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:10 smithi167 ceph-mon[32751]: pgmap v11423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:10 smithi090 ceph-mon[29180]: pgmap v11423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:11 smithi119 ceph-mon[32752]: 25 slow requests, 1 included below; oldest blocked for > 20212.291612 secs 2024-04-27T00:12:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:11 smithi119 ceph-mon[32752]: slow request 720.925810 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:12:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:11 smithi167 ceph-mon[32751]: 25 slow requests, 1 included below; oldest blocked for > 20212.291612 secs 2024-04-27T00:12:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:11 smithi167 ceph-mon[32751]: slow request 720.925810 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:12:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:11 smithi090 ceph-mon[29180]: 25 slow requests, 1 included below; oldest blocked for > 20212.291612 secs 2024-04-27T00:12:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:11 smithi090 ceph-mon[29180]: slow request 720.925810 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:12:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:12 smithi119 ceph-mon[32752]: pgmap v11424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20213.078634 secs 2024-04-27T00:12:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:12 smithi167 ceph-mon[32751]: pgmap v11424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20213.078634 secs 2024-04-27T00:12:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:12 smithi090 ceph-mon[29180]: pgmap v11424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20213.078634 secs 2024-04-27T00:12:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:14 smithi119 ceph-mon[32752]: pgmap v11425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:14 smithi167 ceph-mon[32751]: pgmap v11425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:14 smithi090 ceph-mon[29180]: pgmap v11425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:16 smithi119 ceph-mon[32752]: pgmap v11426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20217.291697 secs 2024-04-27T00:12:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:16 smithi167 ceph-mon[32751]: pgmap v11426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20217.291697 secs 2024-04-27T00:12:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:16 smithi090 ceph-mon[29180]: pgmap v11426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:12:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20217.291697 secs 2024-04-27T00:12:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20218.078751 secs 2024-04-27T00:12:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20218.078751 secs 2024-04-27T00:12:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20218.078751 secs 2024-04-27T00:12:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:18 smithi119 ceph-mon[32752]: pgmap v11427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:18 smithi167 ceph-mon[32751]: pgmap v11427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:18 smithi090 ceph-mon[29180]: pgmap v11427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:20 smithi119 ceph-mon[32752]: pgmap v11428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:20 smithi167 ceph-mon[32751]: pgmap v11428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:20 smithi090 ceph-mon[29180]: pgmap v11428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20222.291760 secs 2024-04-27T00:12:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20222.291760 secs 2024-04-27T00:12:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20222.291760 secs 2024-04-27T00:12:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:22 smithi119 ceph-mon[32752]: pgmap v11429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20223.078863 secs 2024-04-27T00:12:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:22 smithi167 ceph-mon[32751]: pgmap v11429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20223.078863 secs 2024-04-27T00:12:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:22 smithi090 ceph-mon[29180]: pgmap v11429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20223.078863 secs 2024-04-27T00:12:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:24 smithi119 ceph-mon[32752]: pgmap v11430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:24 smithi167 ceph-mon[32751]: pgmap v11430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:24 smithi090 ceph-mon[29180]: pgmap v11430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:26 smithi119 ceph-mon[32752]: pgmap v11431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20227.291847 secs 2024-04-27T00:12:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:26 smithi167 ceph-mon[32751]: pgmap v11431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20227.291847 secs 2024-04-27T00:12:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:26 smithi090 ceph-mon[29180]: pgmap v11431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20227.291847 secs 2024-04-27T00:12:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20228.078933 secs 2024-04-27T00:12:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20228.078933 secs 2024-04-27T00:12:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20228.078933 secs 2024-04-27T00:12:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:28 smithi167 ceph-mon[32751]: pgmap v11432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:28 smithi090 ceph-mon[29180]: pgmap v11432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:28 smithi119 ceph-mon[32752]: pgmap v11432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:30 smithi090 ceph-mon[29180]: pgmap v11433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:30 smithi119 ceph-mon[32752]: pgmap v11433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:30 smithi167 ceph-mon[32751]: pgmap v11433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20232.291935 secs 2024-04-27T00:12:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20232.291935 secs 2024-04-27T00:12:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20232.291935 secs 2024-04-27T00:12:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:32 smithi090 ceph-mon[29180]: pgmap v11434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20233.079047 secs 2024-04-27T00:12:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:32 smithi119 ceph-mon[32752]: pgmap v11434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20233.079047 secs 2024-04-27T00:12:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:32 smithi167 ceph-mon[32751]: pgmap v11434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20233.079047 secs 2024-04-27T00:12:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:34 smithi090 ceph-mon[29180]: pgmap v11435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:34 smithi119 ceph-mon[32752]: pgmap v11435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:34 smithi167 ceph-mon[32751]: pgmap v11435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:36 smithi090 ceph-mon[29180]: pgmap v11436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20237.292021 secs 2024-04-27T00:12:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:36 smithi119 ceph-mon[32752]: pgmap v11436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20237.292021 secs 2024-04-27T00:12:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:36 smithi167 ceph-mon[32751]: pgmap v11436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20237.292021 secs 2024-04-27T00:12:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20238.079111 secs 2024-04-27T00:12:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20238.079111 secs 2024-04-27T00:12:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20238.079111 secs 2024-04-27T00:12:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:38 smithi090 ceph-mon[29180]: pgmap v11437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:38 smithi119 ceph-mon[32752]: pgmap v11437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:38 smithi167 ceph-mon[32751]: pgmap v11437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:40 smithi090 ceph-mon[29180]: pgmap v11438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:40 smithi119 ceph-mon[32752]: pgmap v11438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:40 smithi167 ceph-mon[32751]: pgmap v11438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20242.292106 secs 2024-04-27T00:12:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20242.292106 secs 2024-04-27T00:12:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20242.292106 secs 2024-04-27T00:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:42 smithi090 ceph-mon[29180]: pgmap v11439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:42 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 20243.079218 secs 2024-04-27T00:12:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:42 smithi090 ceph-mon[29180]: slow request 724.651629 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:12:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:42 smithi119 ceph-mon[32752]: pgmap v11439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:42 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 20243.079218 secs 2024-04-27T00:12:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:42 smithi119 ceph-mon[32752]: slow request 724.651629 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:12:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:42 smithi167 ceph-mon[32751]: pgmap v11439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:42 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 20243.079218 secs 2024-04-27T00:12:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:42 smithi167 ceph-mon[32751]: slow request 724.651629 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:12:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:43 smithi119 ceph-mon[32752]: pgmap v11440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:43 smithi167 ceph-mon[32751]: pgmap v11440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:43 smithi090 ceph-mon[29180]: pgmap v11440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:45 smithi119 ceph-mon[32752]: pgmap v11441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:45 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20247.292168 secs 2024-04-27T00:12:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:45 smithi167 ceph-mon[32751]: pgmap v11441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:45 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20247.292168 secs 2024-04-27T00:12:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:45 smithi090 ceph-mon[29180]: pgmap v11441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:45 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20247.292168 secs 2024-04-27T00:12:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20248.079319 secs 2024-04-27T00:12:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20248.079319 secs 2024-04-27T00:12:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20248.079319 secs 2024-04-27T00:12:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:47 smithi119 ceph-mon[32752]: pgmap v11442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:47 smithi167 ceph-mon[32751]: pgmap v11442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:47 smithi090 ceph-mon[29180]: pgmap v11442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:50.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:49 smithi119 ceph-mon[32752]: pgmap v11443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:49 smithi167 ceph-mon[32751]: pgmap v11443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:49 smithi090 ceph-mon[29180]: pgmap v11443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:50 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20252.292248 secs 2024-04-27T00:12:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:50 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20252.292248 secs 2024-04-27T00:12:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:50 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20252.292248 secs 2024-04-27T00:12:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:51 smithi119 ceph-mon[32752]: pgmap v11444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20253.079455 secs 2024-04-27T00:12:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:51 smithi167 ceph-mon[32751]: pgmap v11444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20253.079455 secs 2024-04-27T00:12:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:51 smithi090 ceph-mon[29180]: pgmap v11444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20253.079455 secs 2024-04-27T00:12:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:53 smithi119 ceph-mon[32752]: pgmap v11445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:53 smithi167 ceph-mon[32751]: pgmap v11445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:53 smithi090 ceph-mon[29180]: pgmap v11445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:55 smithi119 ceph-mon[32752]: pgmap v11446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:55 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20257.292332 secs 2024-04-27T00:12:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:55 smithi167 ceph-mon[32751]: pgmap v11446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:55 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20257.292332 secs 2024-04-27T00:12:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:55 smithi090 ceph-mon[29180]: pgmap v11446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:12:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:55 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20257.292332 secs 2024-04-27T00:12:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20258.079535 secs 2024-04-27T00:12:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20258.079535 secs 2024-04-27T00:12:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20258.079535 secs 2024-04-27T00:12:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:57 smithi119 ceph-mon[32752]: pgmap v11447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:57 smithi167 ceph-mon[32751]: pgmap v11447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:12:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:57 smithi090 ceph-mon[29180]: pgmap v11447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:00.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:12:59 smithi119 ceph-mon[32752]: pgmap v11448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:12:59 smithi167 ceph-mon[32751]: pgmap v11448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:12:59 smithi090 ceph-mon[29180]: pgmap v11448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:00 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20262.292414 secs 2024-04-27T00:13:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:00 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20262.292414 secs 2024-04-27T00:13:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:00 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20262.292414 secs 2024-04-27T00:13:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:01 smithi090 ceph-mon[29180]: pgmap v11449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20263.079608 secs 2024-04-27T00:13:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:01 smithi119 ceph-mon[32752]: pgmap v11449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20263.079608 secs 2024-04-27T00:13:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:01 smithi167 ceph-mon[32751]: pgmap v11449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20263.079608 secs 2024-04-27T00:13:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:13:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:13:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:13:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:03 smithi119 ceph-mon[32752]: pgmap v11450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:13:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:13:04.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:13:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:03 smithi167 ceph-mon[32751]: pgmap v11450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:13:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:13:04.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:13:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:03 smithi090 ceph-mon[29180]: pgmap v11450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:13:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:13:04.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:13:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:04 smithi119 ceph-mon[32752]: pgmap v11451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:13:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:04 smithi119 ceph-mon[32752]: pgmap v11452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:04 smithi167 ceph-mon[32751]: pgmap v11451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:13:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:04 smithi167 ceph-mon[32751]: pgmap v11452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:04 smithi090 ceph-mon[29180]: pgmap v11451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:13:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:04 smithi090 ceph-mon[29180]: pgmap v11452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:05 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20267.292499 secs 2024-04-27T00:13:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:05 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20267.292499 secs 2024-04-27T00:13:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:05 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20267.292499 secs 2024-04-27T00:13:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:06 smithi119 ceph-mon[32752]: pgmap v11453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20268.079753 secs 2024-04-27T00:13:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:06 smithi167 ceph-mon[32751]: pgmap v11453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20268.079753 secs 2024-04-27T00:13:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:06 smithi090 ceph-mon[29180]: pgmap v11453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20268.079753 secs 2024-04-27T00:13:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:08 smithi119 ceph-mon[32752]: pgmap v11454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:13:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:08 smithi167 ceph-mon[32751]: pgmap v11454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:13:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:08 smithi090 ceph-mon[29180]: pgmap v11454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:13:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:10 smithi119 ceph-mon[32752]: pgmap v11455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:10 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20272.292559 secs 2024-04-27T00:13:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:10 smithi167 ceph-mon[32751]: pgmap v11455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:10 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20272.292559 secs 2024-04-27T00:13:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:10 smithi090 ceph-mon[29180]: pgmap v11455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:10 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20272.292559 secs 2024-04-27T00:13:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20273.079832 secs 2024-04-27T00:13:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20273.079832 secs 2024-04-27T00:13:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20273.079832 secs 2024-04-27T00:13:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:12 smithi119 ceph-mon[32752]: pgmap v11456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:12 smithi167 ceph-mon[32751]: pgmap v11456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:12 smithi090 ceph-mon[29180]: pgmap v11456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:13:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:14 smithi119 ceph-mon[32752]: pgmap v11457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:13:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:14 smithi167 ceph-mon[32751]: pgmap v11457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:13:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:14 smithi090 ceph-mon[29180]: pgmap v11457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:13:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:15 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20277.292645 secs 2024-04-27T00:13:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:15 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20277.292645 secs 2024-04-27T00:13:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:15 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20277.292645 secs 2024-04-27T00:13:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:16 smithi119 ceph-mon[32752]: pgmap v11458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20278.079950 secs 2024-04-27T00:13:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:16 smithi167 ceph-mon[32751]: pgmap v11458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20278.079950 secs 2024-04-27T00:13:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:16 smithi090 ceph-mon[29180]: pgmap v11458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20278.079950 secs 2024-04-27T00:13:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:18 smithi119 ceph-mon[32752]: pgmap v11459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:18 smithi167 ceph-mon[32751]: pgmap v11459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:18 smithi090 ceph-mon[29180]: pgmap v11459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:20 smithi119 ceph-mon[32752]: pgmap v11460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:20 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20282.292706 secs 2024-04-27T00:13:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:20 smithi167 ceph-mon[32751]: pgmap v11460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:20 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20282.292706 secs 2024-04-27T00:13:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:20 smithi090 ceph-mon[29180]: pgmap v11460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:20 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20282.292706 secs 2024-04-27T00:13:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20283.080090 secs 2024-04-27T00:13:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20283.080090 secs 2024-04-27T00:13:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20283.080090 secs 2024-04-27T00:13:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:22 smithi119 ceph-mon[32752]: pgmap v11461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:22 smithi167 ceph-mon[32751]: pgmap v11461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:22 smithi090 ceph-mon[29180]: pgmap v11461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:24 smithi119 ceph-mon[32752]: pgmap v11462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:24 smithi167 ceph-mon[32751]: pgmap v11462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:24 smithi090 ceph-mon[29180]: pgmap v11462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:25 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20287.292813 secs 2024-04-27T00:13:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:25 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20287.292813 secs 2024-04-27T00:13:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:25 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20287.292813 secs 2024-04-27T00:13:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:26 smithi119 ceph-mon[32752]: pgmap v11463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20288.080209 secs 2024-04-27T00:13:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:26 smithi167 ceph-mon[32751]: pgmap v11463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20288.080209 secs 2024-04-27T00:13:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:26 smithi090 ceph-mon[29180]: pgmap v11463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20288.080209 secs 2024-04-27T00:13:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:28 smithi119 ceph-mon[32752]: pgmap v11464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:28 smithi167 ceph-mon[32751]: pgmap v11464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:29.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:28 smithi090 ceph-mon[29180]: pgmap v11464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:30 smithi119 ceph-mon[32752]: pgmap v11465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:30 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20292.292900 secs 2024-04-27T00:13:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:30 smithi167 ceph-mon[32751]: pgmap v11465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:30 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20292.292900 secs 2024-04-27T00:13:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:30 smithi090 ceph-mon[29180]: pgmap v11465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:30 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20292.292900 secs 2024-04-27T00:13:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20293.080354 secs 2024-04-27T00:13:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20293.080354 secs 2024-04-27T00:13:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20293.080354 secs 2024-04-27T00:13:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:32 smithi119 ceph-mon[32752]: pgmap v11466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:32 smithi167 ceph-mon[32751]: pgmap v11466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:32 smithi090 ceph-mon[29180]: pgmap v11466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:35.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:34 smithi119 ceph-mon[32752]: pgmap v11467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:34 smithi167 ceph-mon[32751]: pgmap v11467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:34 smithi090 ceph-mon[29180]: pgmap v11467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:35 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20297.292989 secs 2024-04-27T00:13:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:35 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20297.292989 secs 2024-04-27T00:13:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:35 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20297.292989 secs 2024-04-27T00:13:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:36 smithi119 ceph-mon[32752]: pgmap v11468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20298.080430 secs 2024-04-27T00:13:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:36 smithi167 ceph-mon[32751]: pgmap v11468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20298.080430 secs 2024-04-27T00:13:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:36 smithi090 ceph-mon[29180]: pgmap v11468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20298.080430 secs 2024-04-27T00:13:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:38 smithi167 ceph-mon[32751]: pgmap v11469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:38 smithi090 ceph-mon[29180]: pgmap v11469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:38 smithi119 ceph-mon[32752]: pgmap v11469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:40 smithi167 ceph-mon[32751]: pgmap v11470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:40 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20302.293071 secs 2024-04-27T00:13:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:40 smithi090 ceph-mon[29180]: pgmap v11470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:40 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20302.293071 secs 2024-04-27T00:13:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:40 smithi119 ceph-mon[32752]: pgmap v11470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:40 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20302.293071 secs 2024-04-27T00:13:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20303.080583 secs 2024-04-27T00:13:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20303.080583 secs 2024-04-27T00:13:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20303.080583 secs 2024-04-27T00:13:43.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:42 smithi090 ceph-mon[29180]: pgmap v11471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:42 smithi167 ceph-mon[32751]: pgmap v11471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:42 smithi119 ceph-mon[32752]: pgmap v11471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:44 smithi167 ceph-mon[32751]: pgmap v11472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:44 smithi090 ceph-mon[29180]: pgmap v11472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:44 smithi119 ceph-mon[32752]: pgmap v11472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:45 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20307.293155 secs 2024-04-27T00:13:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:45 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20307.293155 secs 2024-04-27T00:13:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:45 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20307.293155 secs 2024-04-27T00:13:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:46 smithi090 ceph-mon[29180]: pgmap v11473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20308.080709 secs 2024-04-27T00:13:47.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:46 smithi167 ceph-mon[32751]: pgmap v11473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:47.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20308.080709 secs 2024-04-27T00:13:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:46 smithi119 ceph-mon[32752]: pgmap v11473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20308.080709 secs 2024-04-27T00:13:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:48 smithi119 ceph-mon[32752]: pgmap v11474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:48 smithi167 ceph-mon[32751]: pgmap v11474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:48 smithi090 ceph-mon[29180]: pgmap v11474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:50 smithi119 ceph-mon[32752]: pgmap v11475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:50 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20312.293233 secs 2024-04-27T00:13:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:50 smithi167 ceph-mon[32751]: pgmap v11475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:50 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20312.293233 secs 2024-04-27T00:13:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:50 smithi090 ceph-mon[29180]: pgmap v11475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:50 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20312.293233 secs 2024-04-27T00:13:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20313.080790 secs 2024-04-27T00:13:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20313.080790 secs 2024-04-27T00:13:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20313.080790 secs 2024-04-27T00:13:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:52 smithi119 ceph-mon[32752]: pgmap v11476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:52 smithi167 ceph-mon[32751]: pgmap v11476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:52 smithi090 ceph-mon[29180]: pgmap v11476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:54 smithi119 ceph-mon[32752]: pgmap v11477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:54 smithi167 ceph-mon[32751]: pgmap v11477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:54 smithi090 ceph-mon[29180]: pgmap v11477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:13:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:55 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20317.293317 secs 2024-04-27T00:13:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:55 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20317.293317 secs 2024-04-27T00:13:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:55 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20317.293317 secs 2024-04-27T00:13:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:56 smithi119 ceph-mon[32752]: pgmap v11478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20318.080871 secs 2024-04-27T00:13:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:56 smithi167 ceph-mon[32751]: pgmap v11478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20318.080871 secs 2024-04-27T00:13:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:56 smithi090 ceph-mon[29180]: pgmap v11478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20318.080871 secs 2024-04-27T00:13:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:13:58 smithi119 ceph-mon[32752]: pgmap v11479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:13:58 smithi167 ceph-mon[32751]: pgmap v11479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:13:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:13:58 smithi090 ceph-mon[29180]: pgmap v11479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:00 smithi119 ceph-mon[32752]: pgmap v11480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:00 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20322.293402 secs 2024-04-27T00:14:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:00 smithi167 ceph-mon[32751]: pgmap v11480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:00 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20322.293402 secs 2024-04-27T00:14:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:00 smithi090 ceph-mon[29180]: pgmap v11480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:00 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20322.293402 secs 2024-04-27T00:14:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20323.080988 secs 2024-04-27T00:14:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20323.080988 secs 2024-04-27T00:14:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20323.080988 secs 2024-04-27T00:14:02.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:02 smithi090 ceph-mon[29180]: pgmap v11481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:02 smithi119 ceph-mon[32752]: pgmap v11481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:02 smithi167 ceph-mon[32751]: pgmap v11481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:04 smithi119 ceph-mon[32752]: pgmap v11482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:14:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:14:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:14:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:04 smithi119 ceph-mon[32752]: pgmap v11483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:14:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:04 smithi119 ceph-mon[32752]: pgmap v11484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:14:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:04 smithi167 ceph-mon[32751]: pgmap v11482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:14:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:14:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:14:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:04 smithi167 ceph-mon[32751]: pgmap v11483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:14:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:04 smithi167 ceph-mon[32751]: pgmap v11484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:04 smithi090 ceph-mon[29180]: pgmap v11482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:04 smithi090 ceph-mon[29180]: pgmap v11483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:04 smithi090 ceph-mon[29180]: pgmap v11484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:14:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:05 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 20327.293486 secs 2024-04-27T00:14:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:05 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 20327.293486 secs 2024-04-27T00:14:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:05 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 20327.293486 secs 2024-04-27T00:14:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20328.081107 secs 2024-04-27T00:14:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:06 smithi119 ceph-mon[32752]: pgmap v11485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20328.081107 secs 2024-04-27T00:14:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:06 smithi167 ceph-mon[32751]: pgmap v11485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20328.081107 secs 2024-04-27T00:14:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:06 smithi090 ceph-mon[29180]: pgmap v11485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:09 smithi119 ceph-mon[32752]: pgmap v11486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:14:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:09 smithi167 ceph-mon[32751]: pgmap v11486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:14:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:09 smithi090 ceph-mon[29180]: pgmap v11486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:14:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:10 smithi119 ceph-mon[32752]: 30 slow requests, 2 included below; oldest blocked for > 20332.293571 secs 2024-04-27T00:14:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:10 smithi119 ceph-mon[32752]: slow request 1444.055320 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:14:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:10 smithi119 ceph-mon[32752]: slow request 183.701017 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:14:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:10 smithi167 ceph-mon[32751]: 30 slow requests, 2 included below; oldest blocked for > 20332.293571 secs 2024-04-27T00:14:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:10 smithi167 ceph-mon[32751]: slow request 1444.055320 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:14:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:10 smithi167 ceph-mon[32751]: slow request 183.701017 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:14:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:10 smithi090 ceph-mon[29180]: 30 slow requests, 2 included below; oldest blocked for > 20332.293571 secs 2024-04-27T00:14:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:10 smithi090 ceph-mon[29180]: slow request 1444.055320 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:14:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:10 smithi090 ceph-mon[29180]: slow request 183.701017 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:14:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:11 smithi119 ceph-mon[32752]: pgmap v11487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:11 smithi167 ceph-mon[32751]: pgmap v11487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:11 smithi090 ceph-mon[29180]: pgmap v11487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:14:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20333.081185 secs 2024-04-27T00:14:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20333.081185 secs 2024-04-27T00:14:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20333.081185 secs 2024-04-27T00:14:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:13 smithi119 ceph-mon[32752]: pgmap v11488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:14:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:13 smithi167 ceph-mon[32751]: pgmap v11488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:14:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:13 smithi090 ceph-mon[29180]: pgmap v11488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:14:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:15 smithi119 ceph-mon[32752]: pgmap v11489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:14:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:14:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:14:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:15 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20337.293656 secs 2024-04-27T00:14:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:15 smithi167 ceph-mon[32751]: pgmap v11489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:14:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:14:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:14:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:15 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20337.293656 secs 2024-04-27T00:14:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:15 smithi090 ceph-mon[29180]: pgmap v11489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:14:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:14:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:14:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:15 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20337.293656 secs 2024-04-27T00:14:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20338.081304 secs 2024-04-27T00:14:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:17 smithi119 ceph-mon[32752]: pgmap v11490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20338.081304 secs 2024-04-27T00:14:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:17 smithi167 ceph-mon[32751]: pgmap v11490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20338.081304 secs 2024-04-27T00:14:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:17 smithi090 ceph-mon[29180]: pgmap v11490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:18 smithi119 ceph-mon[32752]: pgmap v11491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:18 smithi167 ceph-mon[32751]: pgmap v11491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:18 smithi090 ceph-mon[29180]: pgmap v11491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:20 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20342.293741 secs 2024-04-27T00:14:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:20 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20342.293741 secs 2024-04-27T00:14:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:20 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20342.293741 secs 2024-04-27T00:14:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:21 smithi119 ceph-mon[32752]: pgmap v11492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:21 smithi167 ceph-mon[32751]: pgmap v11492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:21 smithi090 ceph-mon[29180]: pgmap v11492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20343.081446 secs 2024-04-27T00:14:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20343.081446 secs 2024-04-27T00:14:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20343.081446 secs 2024-04-27T00:14:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:23 smithi119 ceph-mon[32752]: pgmap v11493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:23 smithi167 ceph-mon[32751]: pgmap v11493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:23 smithi090 ceph-mon[29180]: pgmap v11493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:25 smithi119 ceph-mon[32752]: pgmap v11494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:25 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20347.293846 secs 2024-04-27T00:14:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:25 smithi167 ceph-mon[32751]: pgmap v11494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:25 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20347.293846 secs 2024-04-27T00:14:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:25 smithi090 ceph-mon[29180]: pgmap v11494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:25 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20347.293846 secs 2024-04-27T00:14:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20348.081526 secs 2024-04-27T00:14:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:27 smithi119 ceph-mon[32752]: pgmap v11495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20348.081526 secs 2024-04-27T00:14:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:27 smithi167 ceph-mon[32751]: pgmap v11495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20348.081526 secs 2024-04-27T00:14:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:27 smithi090 ceph-mon[29180]: pgmap v11495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:29 smithi119 ceph-mon[32752]: pgmap v11496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:29 smithi167 ceph-mon[32751]: pgmap v11496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:29 smithi090 ceph-mon[29180]: pgmap v11496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:30 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20352.293935 secs 2024-04-27T00:14:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:30 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20352.293935 secs 2024-04-27T00:14:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:30 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20352.293935 secs 2024-04-27T00:14:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:31 smithi119 ceph-mon[32752]: pgmap v11497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:31 smithi167 ceph-mon[32751]: pgmap v11497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:31 smithi090 ceph-mon[29180]: pgmap v11497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20353.081588 secs 2024-04-27T00:14:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:32 smithi090 ceph-mon[29180]: pgmap v11498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20353.081588 secs 2024-04-27T00:14:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:32 smithi119 ceph-mon[32752]: pgmap v11498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20353.081588 secs 2024-04-27T00:14:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:32 smithi167 ceph-mon[32751]: pgmap v11498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:35 smithi119 ceph-mon[32752]: pgmap v11499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:35 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20357.294019 secs 2024-04-27T00:14:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:35 smithi167 ceph-mon[32751]: pgmap v11499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:35 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20357.294019 secs 2024-04-27T00:14:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:35 smithi090 ceph-mon[29180]: pgmap v11499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:35 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20357.294019 secs 2024-04-27T00:14:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20358.081686 secs 2024-04-27T00:14:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:37 smithi119 ceph-mon[32752]: pgmap v11500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20358.081686 secs 2024-04-27T00:14:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:37 smithi167 ceph-mon[32751]: pgmap v11500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20358.081686 secs 2024-04-27T00:14:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:37 smithi090 ceph-mon[29180]: pgmap v11500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:39 smithi119 ceph-mon[32752]: pgmap v11501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:39 smithi167 ceph-mon[32751]: pgmap v11501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:39 smithi090 ceph-mon[29180]: pgmap v11501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:40 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20362.294099 secs 2024-04-27T00:14:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:40 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20362.294099 secs 2024-04-27T00:14:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:40 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20362.294099 secs 2024-04-27T00:14:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:41 smithi119 ceph-mon[32752]: pgmap v11502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:41 smithi167 ceph-mon[32751]: pgmap v11502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:41 smithi090 ceph-mon[29180]: pgmap v11502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20363.081804 secs 2024-04-27T00:14:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20363.081804 secs 2024-04-27T00:14:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20363.081804 secs 2024-04-27T00:14:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:43 smithi119 ceph-mon[32752]: pgmap v11503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:43 smithi167 ceph-mon[32751]: pgmap v11503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:43 smithi090 ceph-mon[29180]: pgmap v11503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:44 smithi119 ceph-mon[32752]: pgmap v11504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:44 smithi167 ceph-mon[32751]: pgmap v11504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:44 smithi090 ceph-mon[29180]: pgmap v11504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:45 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20367.294190 secs 2024-04-27T00:14:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:45 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20367.294190 secs 2024-04-27T00:14:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:45 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20367.294190 secs 2024-04-27T00:14:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:46 smithi119 ceph-mon[32752]: pgmap v11505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:46 smithi167 ceph-mon[32751]: pgmap v11505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:46 smithi090 ceph-mon[29180]: pgmap v11505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20368.081923 secs 2024-04-27T00:14:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20368.081923 secs 2024-04-27T00:14:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20368.081923 secs 2024-04-27T00:14:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:48 smithi119 ceph-mon[32752]: pgmap v11506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:48 smithi167 ceph-mon[32751]: pgmap v11506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:48 smithi090 ceph-mon[29180]: pgmap v11506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:50 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20372.294274 secs 2024-04-27T00:14:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:50 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20372.294274 secs 2024-04-27T00:14:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:50 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20372.294274 secs 2024-04-27T00:14:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:51 smithi119 ceph-mon[32752]: pgmap v11507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:51 smithi167 ceph-mon[32751]: pgmap v11507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:51 smithi090 ceph-mon[29180]: pgmap v11507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20373.082035 secs 2024-04-27T00:14:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20373.082035 secs 2024-04-27T00:14:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20373.082035 secs 2024-04-27T00:14:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:53 smithi119 ceph-mon[32752]: pgmap v11508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:53 smithi167 ceph-mon[32751]: pgmap v11508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:53 smithi090 ceph-mon[29180]: pgmap v11508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:55 smithi119 ceph-mon[32752]: pgmap v11509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:55 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20377.294358 secs 2024-04-27T00:14:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:55 smithi167 ceph-mon[32751]: pgmap v11509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:55 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20377.294358 secs 2024-04-27T00:14:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:55 smithi090 ceph-mon[29180]: pgmap v11509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:14:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:55 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20377.294358 secs 2024-04-27T00:14:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20378.082158 secs 2024-04-27T00:14:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:57 smithi119 ceph-mon[32752]: pgmap v11510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20378.082158 secs 2024-04-27T00:14:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:57 smithi167 ceph-mon[32751]: pgmap v11510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20378.082158 secs 2024-04-27T00:14:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:57 smithi090 ceph-mon[29180]: pgmap v11510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:14:58 smithi119 ceph-mon[32752]: pgmap v11511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:14:58 smithi167 ceph-mon[32751]: pgmap v11511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:14:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:14:58 smithi090 ceph-mon[29180]: pgmap v11511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:00 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20382.294444 secs 2024-04-27T00:15:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:00 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20382.294444 secs 2024-04-27T00:15:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:00 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20382.294444 secs 2024-04-27T00:15:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:01 smithi119 ceph-mon[32752]: pgmap v11512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:01 smithi167 ceph-mon[32751]: pgmap v11512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:01.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:01 smithi090 ceph-mon[29180]: pgmap v11512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:02.506 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20383.082292 secs 2024-04-27T00:15:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20383.082292 secs 2024-04-27T00:15:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20383.082292 secs 2024-04-27T00:15:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:03 smithi119 ceph-mon[32752]: pgmap v11513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:03 smithi167 ceph-mon[32751]: pgmap v11513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:03 smithi090 ceph-mon[29180]: pgmap v11513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:05.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:05 smithi090 ceph-mon[29180]: pgmap v11514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:05.532 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:15:05.532 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:05 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 20387.294536 secs 2024-04-27T00:15:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:05 smithi119 ceph-mon[32752]: pgmap v11514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:15:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:05 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 20387.294536 secs 2024-04-27T00:15:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:05 smithi167 ceph-mon[32751]: pgmap v11514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:15:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:05 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 20387.294536 secs 2024-04-27T00:15:06.535 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:06 smithi090 ceph-mon[29180]: pgmap v11515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:06 smithi119 ceph-mon[32752]: pgmap v11515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:06 smithi167 ceph-mon[32751]: pgmap v11515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20388.082393 secs 2024-04-27T00:15:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20388.082393 secs 2024-04-27T00:15:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20388.082393 secs 2024-04-27T00:15:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:08 smithi119 ceph-mon[32752]: pgmap v11516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:08 smithi167 ceph-mon[32751]: pgmap v11516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:08.577 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:08 smithi090 ceph-mon[29180]: pgmap v11516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:15:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:15:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: pgmap v11517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:15:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: pgmap v11518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:15:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:15:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: pgmap v11517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:15:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: pgmap v11518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:10.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:15:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:15:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: pgmap v11517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:15:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: pgmap v11518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:15:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:10 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20392.294620 secs 2024-04-27T00:15:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:10 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20392.294620 secs 2024-04-27T00:15:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:10 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20392.294620 secs 2024-04-27T00:15:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:11 smithi119 ceph-mon[32752]: pgmap v11519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20393.082474 secs 2024-04-27T00:15:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:11 smithi167 ceph-mon[32751]: pgmap v11519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20393.082474 secs 2024-04-27T00:15:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:11 smithi090 ceph-mon[29180]: pgmap v11519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20393.082474 secs 2024-04-27T00:15:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:13 smithi119 ceph-mon[32752]: pgmap v11520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T00:15:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:13 smithi167 ceph-mon[32751]: pgmap v11520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T00:15:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:13 smithi090 ceph-mon[29180]: pgmap v11520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T00:15:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:15 smithi119 ceph-mon[32752]: pgmap v11521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:15:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:15:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:15 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20397.294704 secs 2024-04-27T00:15:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:15 smithi167 ceph-mon[32751]: pgmap v11521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:15:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:15:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:15 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20397.294704 secs 2024-04-27T00:15:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:15 smithi090 ceph-mon[29180]: pgmap v11521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:15:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:15:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:15 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20397.294704 secs 2024-04-27T00:15:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20398.082588 secs 2024-04-27T00:15:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20398.082588 secs 2024-04-27T00:15:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20398.082588 secs 2024-04-27T00:15:17.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:17 smithi167 ceph-mon[32751]: pgmap v11522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:17 smithi119 ceph-mon[32752]: pgmap v11522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:18.084 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:17 smithi090 ceph-mon[29180]: pgmap v11522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:20.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:19 smithi119 ceph-mon[32752]: pgmap v11523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:15:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:19 smithi167 ceph-mon[32751]: pgmap v11523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:15:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:19 smithi090 ceph-mon[29180]: pgmap v11523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:15:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:20 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20402.294794 secs 2024-04-27T00:15:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:20 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20402.294794 secs 2024-04-27T00:15:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:20 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20402.294794 secs 2024-04-27T00:15:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:21 smithi119 ceph-mon[32752]: pgmap v11524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20403.082706 secs 2024-04-27T00:15:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:21 smithi167 ceph-mon[32751]: pgmap v11524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20403.082706 secs 2024-04-27T00:15:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:21 smithi090 ceph-mon[29180]: pgmap v11524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20403.082706 secs 2024-04-27T00:15:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:23 smithi119 ceph-mon[32752]: pgmap v11525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:23 smithi167 ceph-mon[32751]: pgmap v11525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:24.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:23 smithi090 ceph-mon[29180]: pgmap v11525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:25 smithi119 ceph-mon[32752]: pgmap v11526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:25 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20407.294882 secs 2024-04-27T00:15:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:25 smithi167 ceph-mon[32751]: pgmap v11526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:25 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20407.294882 secs 2024-04-27T00:15:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:25 smithi090 ceph-mon[29180]: pgmap v11526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:25 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20407.294882 secs 2024-04-27T00:15:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20408.082826 secs 2024-04-27T00:15:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20408.082826 secs 2024-04-27T00:15:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20408.082826 secs 2024-04-27T00:15:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:27 smithi090 ceph-mon[29180]: pgmap v11527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:27 smithi119 ceph-mon[32752]: pgmap v11527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:27 smithi167 ceph-mon[32751]: pgmap v11527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:29 smithi119 ceph-mon[32752]: pgmap v11528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:29 smithi167 ceph-mon[32751]: pgmap v11528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:30.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:29 smithi090 ceph-mon[29180]: pgmap v11528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:30 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20412.294968 secs 2024-04-27T00:15:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:30 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20412.294968 secs 2024-04-27T00:15:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:30 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20412.294968 secs 2024-04-27T00:15:32.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:31 smithi090 ceph-mon[29180]: pgmap v11529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:32.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20413.082967 secs 2024-04-27T00:15:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:31 smithi119 ceph-mon[32752]: pgmap v11529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20413.082967 secs 2024-04-27T00:15:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:31 smithi167 ceph-mon[32751]: pgmap v11529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20413.082967 secs 2024-04-27T00:15:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:33 smithi119 ceph-mon[32752]: pgmap v11530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:33 smithi167 ceph-mon[32751]: pgmap v11530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:33 smithi090 ceph-mon[29180]: pgmap v11530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:35 smithi119 ceph-mon[32752]: pgmap v11531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:35 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20417.295056 secs 2024-04-27T00:15:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:35 smithi167 ceph-mon[32751]: pgmap v11531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:35 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20417.295056 secs 2024-04-27T00:15:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:35 smithi090 ceph-mon[29180]: pgmap v11531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:35 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20417.295056 secs 2024-04-27T00:15:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20418.083086 secs 2024-04-27T00:15:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20418.083086 secs 2024-04-27T00:15:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20418.083086 secs 2024-04-27T00:15:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:37 smithi119 ceph-mon[32752]: pgmap v11532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:37 smithi167 ceph-mon[32751]: pgmap v11532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:37 smithi090 ceph-mon[29180]: pgmap v11532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:39 smithi119 ceph-mon[32752]: pgmap v11533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:39 smithi167 ceph-mon[32751]: pgmap v11533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:39 smithi090 ceph-mon[29180]: pgmap v11533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:40 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20422.295142 secs 2024-04-27T00:15:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:40 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20422.295142 secs 2024-04-27T00:15:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:40 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20422.295142 secs 2024-04-27T00:15:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:41 smithi119 ceph-mon[32752]: pgmap v11534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20423.083206 secs 2024-04-27T00:15:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:41 smithi167 ceph-mon[32751]: pgmap v11534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20423.083206 secs 2024-04-27T00:15:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:41 smithi090 ceph-mon[29180]: pgmap v11534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20423.083206 secs 2024-04-27T00:15:44.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:43 smithi119 ceph-mon[32752]: pgmap v11535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:43 smithi167 ceph-mon[32751]: pgmap v11535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:43 smithi090 ceph-mon[29180]: pgmap v11535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:45 smithi119 ceph-mon[32752]: pgmap v11536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:45 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20427.295226 secs 2024-04-27T00:15:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:45 smithi167 ceph-mon[32751]: pgmap v11536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:45 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20427.295226 secs 2024-04-27T00:15:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:45 smithi090 ceph-mon[29180]: pgmap v11536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:45 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20427.295226 secs 2024-04-27T00:15:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20428.083295 secs 2024-04-27T00:15:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20428.083295 secs 2024-04-27T00:15:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20428.083295 secs 2024-04-27T00:15:48.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:47 smithi090 ceph-mon[29180]: pgmap v11537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:48.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:47 smithi119 ceph-mon[32752]: pgmap v11537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:47 smithi167 ceph-mon[32751]: pgmap v11537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:50.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:49 smithi119 ceph-mon[32752]: pgmap v11538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:49 smithi167 ceph-mon[32751]: pgmap v11538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:49 smithi090 ceph-mon[29180]: pgmap v11538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:50 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20432.295289 secs 2024-04-27T00:15:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:50 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20432.295289 secs 2024-04-27T00:15:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:50 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20432.295289 secs 2024-04-27T00:15:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:51 smithi119 ceph-mon[32752]: pgmap v11539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20433.083418 secs 2024-04-27T00:15:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:51 smithi167 ceph-mon[32751]: pgmap v11539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20433.083418 secs 2024-04-27T00:15:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:51 smithi090 ceph-mon[29180]: pgmap v11539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20433.083418 secs 2024-04-27T00:15:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:53 smithi119 ceph-mon[32752]: pgmap v11540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:53 smithi167 ceph-mon[32751]: pgmap v11540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:53 smithi090 ceph-mon[29180]: pgmap v11540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:55 smithi119 ceph-mon[32752]: pgmap v11541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:55 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20437.295352 secs 2024-04-27T00:15:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:55 smithi167 ceph-mon[32751]: pgmap v11541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:55 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20437.295352 secs 2024-04-27T00:15:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:55 smithi090 ceph-mon[29180]: pgmap v11541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:15:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:55 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20437.295352 secs 2024-04-27T00:15:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20438.083540 secs 2024-04-27T00:15:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20438.083540 secs 2024-04-27T00:15:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20438.083540 secs 2024-04-27T00:15:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:57 smithi119 ceph-mon[32752]: pgmap v11542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:57 smithi167 ceph-mon[32751]: pgmap v11542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:15:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:57 smithi090 ceph-mon[29180]: pgmap v11542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:00.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:15:59 smithi119 ceph-mon[32752]: pgmap v11543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:15:59 smithi167 ceph-mon[32751]: pgmap v11543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:15:59 smithi090 ceph-mon[29180]: pgmap v11543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:00 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20442.295438 secs 2024-04-27T00:16:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:00 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20442.295438 secs 2024-04-27T00:16:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:00 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20442.295438 secs 2024-04-27T00:16:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:01 smithi119 ceph-mon[32752]: pgmap v11544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20443.083632 secs 2024-04-27T00:16:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:01 smithi167 ceph-mon[32751]: pgmap v11544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20443.083632 secs 2024-04-27T00:16:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:01 smithi090 ceph-mon[29180]: pgmap v11544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20443.083632 secs 2024-04-27T00:16:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:03 smithi167 ceph-mon[32751]: pgmap v11545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:03 smithi090 ceph-mon[29180]: pgmap v11545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:03 smithi119 ceph-mon[32752]: pgmap v11545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:05 smithi167 ceph-mon[32751]: pgmap v11546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:05 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20447.295523 secs 2024-04-27T00:16:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:05 smithi090 ceph-mon[29180]: pgmap v11546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:05 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20447.295523 secs 2024-04-27T00:16:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:05 smithi119 ceph-mon[32752]: pgmap v11546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:05 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20447.295523 secs 2024-04-27T00:16:06.819 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:16:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:16:06.408+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:16:06.819 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:16:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:16:06.408+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:16:06.820 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:16:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:16:06.408+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:16:06.820 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:16:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:16:06.408+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:16:06.820 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:16:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:16:06.408+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:16:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20448.083733 secs 2024-04-27T00:16:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20448.083733 secs 2024-04-27T00:16:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20448.083733 secs 2024-04-27T00:16:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:07 smithi090 ceph-mon[29180]: pgmap v11547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:07 smithi119 ceph-mon[32752]: pgmap v11547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:07 smithi167 ceph-mon[32751]: pgmap v11547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:16:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:16:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:16:10.086 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:09 smithi167 ceph-mon[32751]: pgmap v11548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:09 smithi090 ceph-mon[29180]: pgmap v11548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:10.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:09 smithi119 ceph-mon[32752]: pgmap v11548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:10 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20452.295605 secs 2024-04-27T00:16:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:10 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20452.295605 secs 2024-04-27T00:16:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:10 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20452.295605 secs 2024-04-27T00:16:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:11 smithi090 ceph-mon[29180]: pgmap v11549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20453.083832 secs 2024-04-27T00:16:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:11 smithi119 ceph-mon[32752]: pgmap v11549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20453.083832 secs 2024-04-27T00:16:12.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:11 smithi167 ceph-mon[32751]: pgmap v11549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:12.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20453.083832 secs 2024-04-27T00:16:14.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: pgmap v11550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:16:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:16:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: pgmap v11551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 396 B/s wr, 2 op/s 2024-04-27T00:16:14.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: pgmap v11552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 B/s rd, 491 B/s wr, 1 op/s 2024-04-27T00:16:14.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: pgmap v11550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:16:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:16:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: pgmap v11551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 396 B/s wr, 2 op/s 2024-04-27T00:16:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: pgmap v11552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 B/s rd, 491 B/s wr, 1 op/s 2024-04-27T00:16:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: pgmap v11550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:16:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:16:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:16:14.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: pgmap v11551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 396 B/s wr, 2 op/s 2024-04-27T00:16:14.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: pgmap v11552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 B/s rd, 491 B/s wr, 1 op/s 2024-04-27T00:16:14.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:16:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:16 smithi090 ceph-mon[29180]: pgmap v11553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 491 B/s wr, 3 op/s 2024-04-27T00:16:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:16 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20457.295688 secs 2024-04-27T00:16:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:16 smithi119 ceph-mon[32752]: pgmap v11553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 491 B/s wr, 3 op/s 2024-04-27T00:16:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:16 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20457.295688 secs 2024-04-27T00:16:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:16 smithi167 ceph-mon[32751]: pgmap v11553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 491 B/s wr, 3 op/s 2024-04-27T00:16:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:16 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20457.295688 secs 2024-04-27T00:16:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20458.083951 secs 2024-04-27T00:16:17.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20458.083951 secs 2024-04-27T00:16:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20458.083951 secs 2024-04-27T00:16:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:18 smithi090 ceph-mon[29180]: pgmap v11554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-27T00:16:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:18 smithi119 ceph-mon[32752]: pgmap v11554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-27T00:16:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:18 smithi167 ceph-mon[32751]: pgmap v11554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 B/s rd, 1 op/s 2024-04-27T00:16:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:20 smithi090 ceph-mon[29180]: pgmap v11555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:20 smithi119 ceph-mon[32752]: pgmap v11555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:20 smithi167 ceph-mon[32751]: pgmap v11555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:21 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20462.295772 secs 2024-04-27T00:16:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:21 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20462.295772 secs 2024-04-27T00:16:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:21 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20462.295772 secs 2024-04-27T00:16:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20463.084042 secs 2024-04-27T00:16:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:22 smithi090 ceph-mon[29180]: pgmap v11556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20463.084042 secs 2024-04-27T00:16:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:22 smithi119 ceph-mon[32752]: pgmap v11556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20463.084042 secs 2024-04-27T00:16:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:22 smithi167 ceph-mon[32751]: pgmap v11556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:24 smithi090 ceph-mon[29180]: pgmap v11557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:16:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:24 smithi119 ceph-mon[32752]: pgmap v11557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:16:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:24 smithi167 ceph-mon[32751]: pgmap v11557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:16:26.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:26 smithi119 ceph-mon[32752]: pgmap v11558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:26.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:26 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20467.295852 secs 2024-04-27T00:16:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:26 smithi090 ceph-mon[29180]: pgmap v11558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:26 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20467.295852 secs 2024-04-27T00:16:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:26 smithi167 ceph-mon[32751]: pgmap v11558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:26 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20467.295852 secs 2024-04-27T00:16:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20468.084121 secs 2024-04-27T00:16:27.522 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20468.084121 secs 2024-04-27T00:16:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20468.084121 secs 2024-04-27T00:16:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:28 smithi090 ceph-mon[29180]: pgmap v11559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:28 smithi119 ceph-mon[32752]: pgmap v11559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:28 smithi167 ceph-mon[32751]: pgmap v11559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:30 smithi090 ceph-mon[29180]: pgmap v11560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:30 smithi119 ceph-mon[32752]: pgmap v11560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:30 smithi167 ceph-mon[32751]: pgmap v11560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20472.295913 secs 2024-04-27T00:16:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20472.295913 secs 2024-04-27T00:16:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20472.295913 secs 2024-04-27T00:16:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20473.084232 secs 2024-04-27T00:16:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:32 smithi090 ceph-mon[29180]: pgmap v11561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20473.084232 secs 2024-04-27T00:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:32 smithi119 ceph-mon[32752]: pgmap v11561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20473.084232 secs 2024-04-27T00:16:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:32 smithi167 ceph-mon[32751]: pgmap v11561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:34 smithi090 ceph-mon[29180]: pgmap v11562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:34 smithi119 ceph-mon[32752]: pgmap v11562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:34 smithi167 ceph-mon[32751]: pgmap v11562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:36 smithi090 ceph-mon[29180]: pgmap v11563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20477.295973 secs 2024-04-27T00:16:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:36 smithi119 ceph-mon[32752]: pgmap v11563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20477.295973 secs 2024-04-27T00:16:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:36 smithi167 ceph-mon[32751]: pgmap v11563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20477.295973 secs 2024-04-27T00:16:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20478.084367 secs 2024-04-27T00:16:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20478.084367 secs 2024-04-27T00:16:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20478.084367 secs 2024-04-27T00:16:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:38 smithi119 ceph-mon[32752]: pgmap v11564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:38 smithi167 ceph-mon[32751]: pgmap v11564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:38 smithi090 ceph-mon[29180]: pgmap v11564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:40 smithi119 ceph-mon[32752]: pgmap v11565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:40 smithi167 ceph-mon[32751]: pgmap v11565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:40 smithi090 ceph-mon[29180]: pgmap v11565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20482.296035 secs 2024-04-27T00:16:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20482.296035 secs 2024-04-27T00:16:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20482.296035 secs 2024-04-27T00:16:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20483.084488 secs 2024-04-27T00:16:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:42 smithi119 ceph-mon[32752]: pgmap v11566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20483.084488 secs 2024-04-27T00:16:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:42 smithi167 ceph-mon[32751]: pgmap v11566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20483.084488 secs 2024-04-27T00:16:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:42 smithi090 ceph-mon[29180]: pgmap v11566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:43 smithi167 ceph-mon[32751]: pgmap v11567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:43 smithi090 ceph-mon[29180]: pgmap v11567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:43 smithi119 ceph-mon[32752]: pgmap v11567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:46 smithi090 ceph-mon[29180]: pgmap v11568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:46 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20487.296118 secs 2024-04-27T00:16:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:46 smithi119 ceph-mon[32752]: pgmap v11568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:46 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20487.296118 secs 2024-04-27T00:16:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:46 smithi167 ceph-mon[32751]: pgmap v11568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:46 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20487.296118 secs 2024-04-27T00:16:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20488.084569 secs 2024-04-27T00:16:47.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20488.084569 secs 2024-04-27T00:16:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20488.084569 secs 2024-04-27T00:16:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:48 smithi090 ceph-mon[29180]: pgmap v11569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:48 smithi119 ceph-mon[32752]: pgmap v11569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:48.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:48 smithi167 ceph-mon[32751]: pgmap v11569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:50 smithi090 ceph-mon[29180]: pgmap v11570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:50 smithi119 ceph-mon[32752]: pgmap v11570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:50 smithi167 ceph-mon[32751]: pgmap v11570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:51 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20492.296180 secs 2024-04-27T00:16:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:51 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20492.296180 secs 2024-04-27T00:16:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:51 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20492.296180 secs 2024-04-27T00:16:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20493.084693 secs 2024-04-27T00:16:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:52 smithi090 ceph-mon[29180]: pgmap v11571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20493.084693 secs 2024-04-27T00:16:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:52 smithi119 ceph-mon[32752]: pgmap v11571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20493.084693 secs 2024-04-27T00:16:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:52 smithi167 ceph-mon[32751]: pgmap v11571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:54 smithi090 ceph-mon[29180]: pgmap v11572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:54 smithi119 ceph-mon[32752]: pgmap v11572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:54 smithi167 ceph-mon[32751]: pgmap v11572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:56 smithi090 ceph-mon[29180]: pgmap v11573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:56 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20497.296263 secs 2024-04-27T00:16:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:56 smithi119 ceph-mon[32752]: pgmap v11573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:56 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20497.296263 secs 2024-04-27T00:16:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:56 smithi167 ceph-mon[32751]: pgmap v11573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:16:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:56 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20497.296263 secs 2024-04-27T00:16:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20498.084813 secs 2024-04-27T00:16:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20498.084813 secs 2024-04-27T00:16:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20498.084813 secs 2024-04-27T00:16:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:16:58 smithi090 ceph-mon[29180]: pgmap v11574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:16:58 smithi119 ceph-mon[32752]: pgmap v11574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:16:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:16:58 smithi167 ceph-mon[32751]: pgmap v11574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:00 smithi090 ceph-mon[29180]: pgmap v11575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:00 smithi119 ceph-mon[32752]: pgmap v11575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:00 smithi167 ceph-mon[32751]: pgmap v11575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:01 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20502.296326 secs 2024-04-27T00:17:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:01 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20502.296326 secs 2024-04-27T00:17:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:01 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20502.296326 secs 2024-04-27T00:17:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20503.084932 secs 2024-04-27T00:17:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:02 smithi090 ceph-mon[29180]: pgmap v11576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20503.084932 secs 2024-04-27T00:17:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:02 smithi119 ceph-mon[32752]: pgmap v11576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20503.084932 secs 2024-04-27T00:17:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:02 smithi167 ceph-mon[32751]: pgmap v11576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:04 smithi119 ceph-mon[32752]: pgmap v11577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:04 smithi167 ceph-mon[32751]: pgmap v11577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:04 smithi090 ceph-mon[29180]: pgmap v11577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:06 smithi119 ceph-mon[32752]: pgmap v11578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:06 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20507.296406 secs 2024-04-27T00:17:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:06 smithi167 ceph-mon[32751]: pgmap v11578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:06 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20507.296406 secs 2024-04-27T00:17:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:06 smithi090 ceph-mon[29180]: pgmap v11578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:06 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20507.296406 secs 2024-04-27T00:17:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:07 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 20508.085055 secs 2024-04-27T00:17:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:07 smithi119 ceph-mon[32752]: slow request 2880.225166 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:17:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:07 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 20508.085055 secs 2024-04-27T00:17:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:07 smithi167 ceph-mon[32751]: slow request 2880.225166 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:17:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:07 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 20508.085055 secs 2024-04-27T00:17:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:07 smithi090 ceph-mon[29180]: slow request 2880.225166 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:17:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:08 smithi119 ceph-mon[32752]: pgmap v11579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:08 smithi167 ceph-mon[32751]: pgmap v11579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:08 smithi090 ceph-mon[29180]: pgmap v11579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:10 smithi119 ceph-mon[32752]: pgmap v11580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:10 smithi167 ceph-mon[32751]: pgmap v11580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:10 smithi090 ceph-mon[29180]: pgmap v11580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:11 smithi119 ceph-mon[32752]: 27 slow requests, 1 included below; oldest blocked for > 20512.296492 secs 2024-04-27T00:17:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:11 smithi119 ceph-mon[32752]: slow request 363.703938 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:17:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:11 smithi167 ceph-mon[32751]: 27 slow requests, 1 included below; oldest blocked for > 20512.296492 secs 2024-04-27T00:17:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:11 smithi167 ceph-mon[32751]: slow request 363.703938 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:17:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:11 smithi090 ceph-mon[29180]: 27 slow requests, 1 included below; oldest blocked for > 20512.296492 secs 2024-04-27T00:17:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:11 smithi090 ceph-mon[29180]: slow request 363.703938 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:17:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20513.085123 secs 2024-04-27T00:17:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:12 smithi119 ceph-mon[32752]: pgmap v11581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20513.085123 secs 2024-04-27T00:17:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:12 smithi167 ceph-mon[32751]: pgmap v11581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20513.085123 secs 2024-04-27T00:17:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:12 smithi090 ceph-mon[29180]: pgmap v11581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:14 smithi119 ceph-mon[32752]: pgmap v11582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:17:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:17:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:17:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:17:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:14 smithi167 ceph-mon[32751]: pgmap v11582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:17:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:17:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:17:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:17:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:14 smithi090 ceph-mon[29180]: pgmap v11582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:17:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:17:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:17:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:17:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:15 smithi119 ceph-mon[32752]: pgmap v11583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:17:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:15 smithi119 ceph-mon[32752]: pgmap v11584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:15 smithi167 ceph-mon[32751]: pgmap v11583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:17:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:15 smithi167 ceph-mon[32751]: pgmap v11584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:15 smithi090 ceph-mon[29180]: pgmap v11583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:17:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:15 smithi090 ceph-mon[29180]: pgmap v11584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:16 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20517.296573 secs 2024-04-27T00:17:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:16 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20517.296573 secs 2024-04-27T00:17:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:16 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20517.296573 secs 2024-04-27T00:17:17.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20518.085240 secs 2024-04-27T00:17:17.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:17 smithi167 ceph-mon[32751]: pgmap v11585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20518.085240 secs 2024-04-27T00:17:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:17 smithi119 ceph-mon[32752]: pgmap v11585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20518.085240 secs 2024-04-27T00:17:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:17 smithi090 ceph-mon[29180]: pgmap v11585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:19 smithi119 ceph-mon[32752]: pgmap v11586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T00:17:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:19 smithi167 ceph-mon[32751]: pgmap v11586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T00:17:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:19 smithi090 ceph-mon[29180]: pgmap v11586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T00:17:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:20 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20522.296657 secs 2024-04-27T00:17:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:20 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20522.296657 secs 2024-04-27T00:17:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:20 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20522.296657 secs 2024-04-27T00:17:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:21 smithi119 ceph-mon[32752]: pgmap v11587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:21 smithi167 ceph-mon[32751]: pgmap v11587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:21 smithi090 ceph-mon[29180]: pgmap v11587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20523.085392 secs 2024-04-27T00:17:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20523.085392 secs 2024-04-27T00:17:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20523.085392 secs 2024-04-27T00:17:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:23 smithi119 ceph-mon[32752]: pgmap v11588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:23 smithi167 ceph-mon[32751]: pgmap v11588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:23 smithi090 ceph-mon[29180]: pgmap v11588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:17:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:25 smithi119 ceph-mon[32752]: pgmap v11589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:17:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:25 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20527.296741 secs 2024-04-27T00:17:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:25 smithi167 ceph-mon[32751]: pgmap v11589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:17:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:25 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20527.296741 secs 2024-04-27T00:17:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:25 smithi090 ceph-mon[29180]: pgmap v11589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:17:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:25 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20527.296741 secs 2024-04-27T00:17:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20528.085510 secs 2024-04-27T00:17:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:27 smithi119 ceph-mon[32752]: pgmap v11590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20528.085510 secs 2024-04-27T00:17:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:27 smithi167 ceph-mon[32751]: pgmap v11590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20528.085510 secs 2024-04-27T00:17:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:27 smithi090 ceph-mon[29180]: pgmap v11590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:29 smithi119 ceph-mon[32752]: pgmap v11591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:29 smithi167 ceph-mon[32751]: pgmap v11591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:29 smithi090 ceph-mon[29180]: pgmap v11591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:30 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20532.296825 secs 2024-04-27T00:17:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:30 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20532.296825 secs 2024-04-27T00:17:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:30 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20532.296825 secs 2024-04-27T00:17:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:31 smithi119 ceph-mon[32752]: pgmap v11592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:31 smithi167 ceph-mon[32751]: pgmap v11592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:31 smithi090 ceph-mon[29180]: pgmap v11592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20533.085597 secs 2024-04-27T00:17:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20533.085597 secs 2024-04-27T00:17:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20533.085597 secs 2024-04-27T00:17:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:33 smithi119 ceph-mon[32752]: pgmap v11593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:33 smithi167 ceph-mon[32751]: pgmap v11593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:33 smithi090 ceph-mon[29180]: pgmap v11593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:35 smithi119 ceph-mon[32752]: pgmap v11594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:35 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20537.296914 secs 2024-04-27T00:17:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:35 smithi167 ceph-mon[32751]: pgmap v11594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:35 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20537.296914 secs 2024-04-27T00:17:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:35 smithi090 ceph-mon[29180]: pgmap v11594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:35 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20537.296914 secs 2024-04-27T00:17:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20538.085688 secs 2024-04-27T00:17:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:37 smithi119 ceph-mon[32752]: pgmap v11595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20538.085688 secs 2024-04-27T00:17:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:37 smithi167 ceph-mon[32751]: pgmap v11595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20538.085688 secs 2024-04-27T00:17:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:37 smithi090 ceph-mon[29180]: pgmap v11595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:39 smithi119 ceph-mon[32752]: pgmap v11596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:39 smithi167 ceph-mon[32751]: pgmap v11596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:39 smithi090 ceph-mon[29180]: pgmap v11596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:40 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20542.296975 secs 2024-04-27T00:17:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:40 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20542.296975 secs 2024-04-27T00:17:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:40 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20542.296975 secs 2024-04-27T00:17:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:41 smithi119 ceph-mon[32752]: pgmap v11597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:41 smithi167 ceph-mon[32751]: pgmap v11597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:41 smithi090 ceph-mon[29180]: pgmap v11597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20543.085808 secs 2024-04-27T00:17:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20543.085808 secs 2024-04-27T00:17:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20543.085808 secs 2024-04-27T00:17:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:43 smithi119 ceph-mon[32752]: pgmap v11598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:43 smithi167 ceph-mon[32751]: pgmap v11598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:43 smithi090 ceph-mon[29180]: pgmap v11598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:45 smithi119 ceph-mon[32752]: pgmap v11599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:45 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20547.297034 secs 2024-04-27T00:17:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:45 smithi167 ceph-mon[32751]: pgmap v11599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:45 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20547.297034 secs 2024-04-27T00:17:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:45 smithi090 ceph-mon[29180]: pgmap v11599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:45 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20547.297034 secs 2024-04-27T00:17:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20548.085893 secs 2024-04-27T00:17:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:47 smithi119 ceph-mon[32752]: pgmap v11600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20548.085893 secs 2024-04-27T00:17:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:47 smithi167 ceph-mon[32751]: pgmap v11600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20548.085893 secs 2024-04-27T00:17:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:47 smithi090 ceph-mon[29180]: pgmap v11600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:49 smithi119 ceph-mon[32752]: pgmap v11601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:49 smithi167 ceph-mon[32751]: pgmap v11601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:49 smithi090 ceph-mon[29180]: pgmap v11601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:50 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20552.297095 secs 2024-04-27T00:17:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:50 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20552.297095 secs 2024-04-27T00:17:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:50 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20552.297095 secs 2024-04-27T00:17:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:51 smithi119 ceph-mon[32752]: pgmap v11602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:51 smithi167 ceph-mon[32751]: pgmap v11602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:51 smithi090 ceph-mon[29180]: pgmap v11602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20553.086025 secs 2024-04-27T00:17:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20553.086025 secs 2024-04-27T00:17:52.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20553.086025 secs 2024-04-27T00:17:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:53 smithi119 ceph-mon[32752]: pgmap v11603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:53 smithi167 ceph-mon[32751]: pgmap v11603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:53 smithi090 ceph-mon[29180]: pgmap v11603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:55 smithi119 ceph-mon[32752]: pgmap v11604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:55 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20557.297154 secs 2024-04-27T00:17:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:55 smithi167 ceph-mon[32751]: pgmap v11604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:55 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20557.297154 secs 2024-04-27T00:17:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:55 smithi090 ceph-mon[29180]: pgmap v11604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:17:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:55 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20557.297154 secs 2024-04-27T00:17:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:56 smithi167 ceph-mon[32751]: pgmap v11605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:56 smithi090 ceph-mon[29180]: pgmap v11605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:56 smithi119 ceph-mon[32752]: pgmap v11605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20558.086161 secs 2024-04-27T00:17:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20558.086161 secs 2024-04-27T00:17:57.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20558.086161 secs 2024-04-27T00:17:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:17:58 smithi167 ceph-mon[32751]: pgmap v11606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:17:58 smithi090 ceph-mon[29180]: pgmap v11606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:17:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:17:58 smithi119 ceph-mon[32752]: pgmap v11606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:00 smithi119 ceph-mon[32752]: pgmap v11607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:00 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20562.297240 secs 2024-04-27T00:18:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:00 smithi167 ceph-mon[32751]: pgmap v11607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:00 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20562.297240 secs 2024-04-27T00:18:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:00 smithi090 ceph-mon[29180]: pgmap v11607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:00 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20562.297240 secs 2024-04-27T00:18:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20563.086293 secs 2024-04-27T00:18:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20563.086293 secs 2024-04-27T00:18:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20563.086293 secs 2024-04-27T00:18:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:02 smithi119 ceph-mon[32752]: pgmap v11608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:02 smithi167 ceph-mon[32751]: pgmap v11608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:02 smithi090 ceph-mon[29180]: pgmap v11608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:04 smithi119 ceph-mon[32752]: pgmap v11609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:04 smithi167 ceph-mon[32751]: pgmap v11609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:04 smithi090 ceph-mon[29180]: pgmap v11609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:05 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 20567.297322 secs 2024-04-27T00:18:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:05 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 20567.297322 secs 2024-04-27T00:18:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:05 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 20567.297322 secs 2024-04-27T00:18:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20568.086399 secs 2024-04-27T00:18:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:06 smithi119 ceph-mon[32752]: pgmap v11610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20568.086399 secs 2024-04-27T00:18:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:06 smithi167 ceph-mon[32751]: pgmap v11610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20568.086399 secs 2024-04-27T00:18:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:06 smithi090 ceph-mon[29180]: pgmap v11610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:08 smithi119 ceph-mon[32752]: pgmap v11611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:08 smithi167 ceph-mon[32751]: pgmap v11611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:08 smithi090 ceph-mon[29180]: pgmap v11611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:10 smithi119 ceph-mon[32752]: pgmap v11612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:10 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 20572.297429 secs 2024-04-27T00:18:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:10 smithi119 ceph-mon[32752]: slow request 182.502465 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:18:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:10 smithi167 ceph-mon[32751]: pgmap v11612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:10 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 20572.297429 secs 2024-04-27T00:18:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:10 smithi167 ceph-mon[32751]: slow request 182.502465 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:18:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:10 smithi090 ceph-mon[29180]: pgmap v11612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:10 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 20572.297429 secs 2024-04-27T00:18:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:10 smithi090 ceph-mon[29180]: slow request 182.502465 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:18:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20573.086513 secs 2024-04-27T00:18:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20573.086513 secs 2024-04-27T00:18:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20573.086513 secs 2024-04-27T00:18:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:12 smithi119 ceph-mon[32752]: pgmap v11613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:12 smithi167 ceph-mon[32751]: pgmap v11613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:12 smithi090 ceph-mon[29180]: pgmap v11613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:14 smithi119 ceph-mon[32752]: pgmap v11614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:18:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:14 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:18:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:14 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:18:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:18:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:18:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:14 smithi167 ceph-mon[32751]: pgmap v11614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:18:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:14 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:18:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:14 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:18:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:18:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:18:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:18:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:14 smithi090 ceph-mon[29180]: pgmap v11614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:14 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:14 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:18:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:15 smithi119 ceph-mon[32752]: pgmap v11615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:18:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:15 smithi119 ceph-mon[32752]: pgmap v11616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20577.297515 secs 2024-04-27T00:18:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:15 smithi167 ceph-mon[32751]: pgmap v11615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:18:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:15 smithi167 ceph-mon[32751]: pgmap v11616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20577.297515 secs 2024-04-27T00:18:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:15 smithi090 ceph-mon[29180]: pgmap v11615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:18:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:15 smithi090 ceph-mon[29180]: pgmap v11616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20577.297515 secs 2024-04-27T00:18:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20578.086641 secs 2024-04-27T00:18:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20578.086641 secs 2024-04-27T00:18:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20578.086641 secs 2024-04-27T00:18:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:17 smithi119 ceph-mon[32752]: pgmap v11617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:17 smithi167 ceph-mon[32751]: pgmap v11617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:17 smithi090 ceph-mon[29180]: pgmap v11617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:19 smithi119 ceph-mon[32752]: pgmap v11618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:19 smithi167 ceph-mon[32751]: pgmap v11618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:19 smithi090 ceph-mon[29180]: pgmap v11618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20582.297600 secs 2024-04-27T00:18:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20582.297600 secs 2024-04-27T00:18:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20582.297600 secs 2024-04-27T00:18:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:21 smithi119 ceph-mon[32752]: pgmap v11619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20583.086705 secs 2024-04-27T00:18:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:21 smithi167 ceph-mon[32751]: pgmap v11619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20583.086705 secs 2024-04-27T00:18:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:21 smithi090 ceph-mon[29180]: pgmap v11619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:18:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20583.086705 secs 2024-04-27T00:18:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:23 smithi119 ceph-mon[32752]: pgmap v11620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:18:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:23 smithi167 ceph-mon[32751]: pgmap v11620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:18:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:23 smithi090 ceph-mon[29180]: pgmap v11620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:18:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:25 smithi119 ceph-mon[32752]: pgmap v11621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:18:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20587.297684 secs 2024-04-27T00:18:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:25 smithi167 ceph-mon[32751]: pgmap v11621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:18:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20587.297684 secs 2024-04-27T00:18:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:25 smithi090 ceph-mon[29180]: pgmap v11621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:18:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20587.297684 secs 2024-04-27T00:18:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20588.086780 secs 2024-04-27T00:18:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20588.086780 secs 2024-04-27T00:18:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20588.086780 secs 2024-04-27T00:18:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:28 smithi119 ceph-mon[32752]: pgmap v11622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:28 smithi167 ceph-mon[32751]: pgmap v11622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:28 smithi090 ceph-mon[29180]: pgmap v11622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:30 smithi119 ceph-mon[32752]: pgmap v11623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:30 smithi167 ceph-mon[32751]: pgmap v11623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:30 smithi090 ceph-mon[29180]: pgmap v11623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20592.297771 secs 2024-04-27T00:18:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20592.297771 secs 2024-04-27T00:18:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20592.297771 secs 2024-04-27T00:18:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:32 smithi090 ceph-mon[29180]: pgmap v11624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20593.086874 secs 2024-04-27T00:18:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:32 smithi119 ceph-mon[32752]: pgmap v11624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20593.086874 secs 2024-04-27T00:18:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:32 smithi167 ceph-mon[32751]: pgmap v11624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20593.086874 secs 2024-04-27T00:18:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:34 smithi119 ceph-mon[32752]: pgmap v11625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:34 smithi167 ceph-mon[32751]: pgmap v11625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:34 smithi090 ceph-mon[29180]: pgmap v11625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:36 smithi119 ceph-mon[32752]: pgmap v11626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20597.297855 secs 2024-04-27T00:18:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:36 smithi167 ceph-mon[32751]: pgmap v11626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20597.297855 secs 2024-04-27T00:18:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:36 smithi090 ceph-mon[29180]: pgmap v11626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20597.297855 secs 2024-04-27T00:18:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20598.086995 secs 2024-04-27T00:18:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20598.086995 secs 2024-04-27T00:18:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20598.086995 secs 2024-04-27T00:18:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:38 smithi119 ceph-mon[32752]: pgmap v11627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:38 smithi167 ceph-mon[32751]: pgmap v11627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:38 smithi090 ceph-mon[29180]: pgmap v11627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:40 smithi119 ceph-mon[32752]: pgmap v11628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:40 smithi167 ceph-mon[32751]: pgmap v11628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:40 smithi090 ceph-mon[29180]: pgmap v11628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20602.297917 secs 2024-04-27T00:18:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20602.297917 secs 2024-04-27T00:18:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20602.297917 secs 2024-04-27T00:18:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:42 smithi119 ceph-mon[32752]: pgmap v11629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20603.087124 secs 2024-04-27T00:18:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:42 smithi167 ceph-mon[32751]: pgmap v11629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20603.087124 secs 2024-04-27T00:18:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:42 smithi090 ceph-mon[29180]: pgmap v11629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20603.087124 secs 2024-04-27T00:18:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:44 smithi119 ceph-mon[32752]: pgmap v11630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:44 smithi167 ceph-mon[32751]: pgmap v11630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:44 smithi090 ceph-mon[29180]: pgmap v11630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:46 smithi119 ceph-mon[32752]: pgmap v11631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20607.297980 secs 2024-04-27T00:18:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:46 smithi167 ceph-mon[32751]: pgmap v11631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20607.297980 secs 2024-04-27T00:18:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:46 smithi090 ceph-mon[29180]: pgmap v11631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20607.297980 secs 2024-04-27T00:18:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20608.087259 secs 2024-04-27T00:18:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20608.087259 secs 2024-04-27T00:18:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20608.087259 secs 2024-04-27T00:18:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:48 smithi119 ceph-mon[32752]: pgmap v11632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:48 smithi167 ceph-mon[32751]: pgmap v11632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:48 smithi090 ceph-mon[29180]: pgmap v11632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:50 smithi167 ceph-mon[32751]: pgmap v11633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:50 smithi090 ceph-mon[29180]: pgmap v11633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:50 smithi119 ceph-mon[32752]: pgmap v11633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20612.298086 secs 2024-04-27T00:18:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20612.298086 secs 2024-04-27T00:18:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20612.298086 secs 2024-04-27T00:18:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:52 smithi167 ceph-mon[32751]: pgmap v11634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20613.087329 secs 2024-04-27T00:18:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:52 smithi090 ceph-mon[29180]: pgmap v11634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20613.087329 secs 2024-04-27T00:18:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:52 smithi119 ceph-mon[32752]: pgmap v11634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20613.087329 secs 2024-04-27T00:18:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:54 smithi167 ceph-mon[32751]: pgmap v11635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:54 smithi090 ceph-mon[29180]: pgmap v11635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:54 smithi119 ceph-mon[32752]: pgmap v11635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:56 smithi090 ceph-mon[29180]: pgmap v11636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20617.298169 secs 2024-04-27T00:18:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:56 smithi119 ceph-mon[32752]: pgmap v11636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20617.298169 secs 2024-04-27T00:18:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:56 smithi167 ceph-mon[32751]: pgmap v11636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:18:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20617.298169 secs 2024-04-27T00:18:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20618.087445 secs 2024-04-27T00:18:57.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20618.087445 secs 2024-04-27T00:18:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20618.087445 secs 2024-04-27T00:18:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:18:58 smithi090 ceph-mon[29180]: pgmap v11637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:18:58 smithi119 ceph-mon[32752]: pgmap v11637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:18:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:18:58 smithi167 ceph-mon[32751]: pgmap v11637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:00 smithi090 ceph-mon[29180]: pgmap v11638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:00 smithi119 ceph-mon[32752]: pgmap v11638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:00 smithi167 ceph-mon[32751]: pgmap v11638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20622.298256 secs 2024-04-27T00:19:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20622.298256 secs 2024-04-27T00:19:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20622.298256 secs 2024-04-27T00:19:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:02 smithi090 ceph-mon[29180]: pgmap v11639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20623.087559 secs 2024-04-27T00:19:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:02 smithi119 ceph-mon[32752]: pgmap v11639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20623.087559 secs 2024-04-27T00:19:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:02 smithi167 ceph-mon[32751]: pgmap v11639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20623.087559 secs 2024-04-27T00:19:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:04 smithi090 ceph-mon[29180]: pgmap v11640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:04 smithi119 ceph-mon[32752]: pgmap v11640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:04 smithi167 ceph-mon[32751]: pgmap v11640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:06 smithi090 ceph-mon[29180]: pgmap v11641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:06 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 20627.298338 secs 2024-04-27T00:19:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:06 smithi090 ceph-mon[29180]: slow request 5761.176961 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:19:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:06 smithi119 ceph-mon[32752]: pgmap v11641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:06 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 20627.298338 secs 2024-04-27T00:19:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:06 smithi119 ceph-mon[32752]: slow request 5761.176961 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:19:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:06 smithi167 ceph-mon[32751]: pgmap v11641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:06 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 20627.298338 secs 2024-04-27T00:19:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:06 smithi167 ceph-mon[32751]: slow request 5761.176961 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:19:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20628.087629 secs 2024-04-27T00:19:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20628.087629 secs 2024-04-27T00:19:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20628.087629 secs 2024-04-27T00:19:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:08 smithi090 ceph-mon[29180]: pgmap v11642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:08 smithi119 ceph-mon[32752]: pgmap v11642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:08 smithi167 ceph-mon[32751]: pgmap v11642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:10 smithi090 ceph-mon[29180]: pgmap v11643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:10 smithi119 ceph-mon[32752]: pgmap v11643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:10 smithi167 ceph-mon[32751]: pgmap v11643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20632.298397 secs 2024-04-27T00:19:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20632.298397 secs 2024-04-27T00:19:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20632.298397 secs 2024-04-27T00:19:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:12 smithi090 ceph-mon[29180]: pgmap v11644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20633.087759 secs 2024-04-27T00:19:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:12 smithi119 ceph-mon[32752]: pgmap v11644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20633.087759 secs 2024-04-27T00:19:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:12 smithi167 ceph-mon[32751]: pgmap v11644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20633.087759 secs 2024-04-27T00:19:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:14 smithi090 ceph-mon[29180]: pgmap v11645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:14 smithi119 ceph-mon[32752]: pgmap v11645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:14 smithi167 ceph-mon[32751]: pgmap v11645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:19:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:19:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:19:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:16 smithi119 ceph-mon[32752]: pgmap v11646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20637.298489 secs 2024-04-27T00:19:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:19:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:19:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:19:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:16 smithi167 ceph-mon[32751]: pgmap v11646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20637.298489 secs 2024-04-27T00:19:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:19:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:19:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:19:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:16 smithi090 ceph-mon[29180]: pgmap v11646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20637.298489 secs 2024-04-27T00:19:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:19:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:19:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:19:17.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:17 smithi167 ceph-mon[32751]: pgmap v11647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:19:17.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:17 smithi167 ceph-mon[32751]: pgmap v11648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:17.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20638.087857 secs 2024-04-27T00:19:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:17 smithi119 ceph-mon[32752]: pgmap v11647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:19:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:17 smithi119 ceph-mon[32752]: pgmap v11648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20638.087857 secs 2024-04-27T00:19:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:17 smithi090 ceph-mon[29180]: pgmap v11647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:19:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:17 smithi090 ceph-mon[29180]: pgmap v11648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20638.087857 secs 2024-04-27T00:19:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:19 smithi119 ceph-mon[32752]: pgmap v11649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:19:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:19 smithi167 ceph-mon[32751]: pgmap v11649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:19:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:19 smithi090 ceph-mon[29180]: pgmap v11649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:19:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:21 smithi119 ceph-mon[32752]: pgmap v11650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20642.298576 secs 2024-04-27T00:19:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:21 smithi167 ceph-mon[32751]: pgmap v11650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20642.298576 secs 2024-04-27T00:19:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:21 smithi090 ceph-mon[29180]: pgmap v11650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20642.298576 secs 2024-04-27T00:19:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20643.087973 secs 2024-04-27T00:19:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20643.087973 secs 2024-04-27T00:19:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20643.087973 secs 2024-04-27T00:19:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:23 smithi119 ceph-mon[32752]: pgmap v11651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:23 smithi167 ceph-mon[32751]: pgmap v11651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:23 smithi090 ceph-mon[29180]: pgmap v11651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:25 smithi119 ceph-mon[32752]: pgmap v11652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:25 smithi167 ceph-mon[32751]: pgmap v11652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:25 smithi090 ceph-mon[29180]: pgmap v11652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:19:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20647.298661 secs 2024-04-27T00:19:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20647.298661 secs 2024-04-27T00:19:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20647.298661 secs 2024-04-27T00:19:27.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:27 smithi119 ceph-mon[32752]: pgmap v11653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:19:27.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20648.088112 secs 2024-04-27T00:19:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:27 smithi167 ceph-mon[32751]: pgmap v11653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:19:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20648.088112 secs 2024-04-27T00:19:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:27 smithi090 ceph-mon[29180]: pgmap v11653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:19:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20648.088112 secs 2024-04-27T00:19:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:29 smithi119 ceph-mon[32752]: pgmap v11654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:29 smithi167 ceph-mon[32751]: pgmap v11654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:29 smithi090 ceph-mon[29180]: pgmap v11654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:31 smithi119 ceph-mon[32752]: pgmap v11655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20652.298746 secs 2024-04-27T00:19:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:31 smithi167 ceph-mon[32751]: pgmap v11655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20652.298746 secs 2024-04-27T00:19:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:31 smithi090 ceph-mon[29180]: pgmap v11655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20652.298746 secs 2024-04-27T00:19:32.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20653.088227 secs 2024-04-27T00:19:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20653.088227 secs 2024-04-27T00:19:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20653.088227 secs 2024-04-27T00:19:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:33 smithi119 ceph-mon[32752]: pgmap v11656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:33 smithi167 ceph-mon[32751]: pgmap v11656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:33 smithi090 ceph-mon[29180]: pgmap v11656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:35 smithi119 ceph-mon[32752]: pgmap v11657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20657.298845 secs 2024-04-27T00:19:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:35 smithi167 ceph-mon[32751]: pgmap v11657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20657.298845 secs 2024-04-27T00:19:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:35 smithi090 ceph-mon[29180]: pgmap v11657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20657.298845 secs 2024-04-27T00:19:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:37 smithi119 ceph-mon[32752]: pgmap v11658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20658.088329 secs 2024-04-27T00:19:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:37 smithi167 ceph-mon[32751]: pgmap v11658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20658.088329 secs 2024-04-27T00:19:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:37 smithi090 ceph-mon[29180]: pgmap v11658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20658.088329 secs 2024-04-27T00:19:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:38 smithi090 ceph-mon[29180]: pgmap v11659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:38 smithi119 ceph-mon[32752]: pgmap v11659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:38 smithi167 ceph-mon[32751]: pgmap v11659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:40 smithi119 ceph-mon[32752]: pgmap v11660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20662.298909 secs 2024-04-27T00:19:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:40 smithi167 ceph-mon[32751]: pgmap v11660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20662.298909 secs 2024-04-27T00:19:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:40 smithi090 ceph-mon[29180]: pgmap v11660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20662.298909 secs 2024-04-27T00:19:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20663.088471 secs 2024-04-27T00:19:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20663.088471 secs 2024-04-27T00:19:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20663.088471 secs 2024-04-27T00:19:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:42 smithi119 ceph-mon[32752]: pgmap v11661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:42 smithi167 ceph-mon[32751]: pgmap v11661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:42 smithi090 ceph-mon[29180]: pgmap v11661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:44 smithi119 ceph-mon[32752]: pgmap v11662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:44 smithi167 ceph-mon[32751]: pgmap v11662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:44 smithi090 ceph-mon[29180]: pgmap v11662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20667.298997 secs 2024-04-27T00:19:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20667.298997 secs 2024-04-27T00:19:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20667.298997 secs 2024-04-27T00:19:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:46 smithi119 ceph-mon[32752]: pgmap v11663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20668.088597 secs 2024-04-27T00:19:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:46 smithi167 ceph-mon[32751]: pgmap v11663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20668.088597 secs 2024-04-27T00:19:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:46 smithi090 ceph-mon[29180]: pgmap v11663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20668.088597 secs 2024-04-27T00:19:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:48 smithi119 ceph-mon[32752]: pgmap v11664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:48 smithi167 ceph-mon[32751]: pgmap v11664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:48 smithi090 ceph-mon[29180]: pgmap v11664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:50 smithi119 ceph-mon[32752]: pgmap v11665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20672.299058 secs 2024-04-27T00:19:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:50 smithi167 ceph-mon[32751]: pgmap v11665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20672.299058 secs 2024-04-27T00:19:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:50 smithi090 ceph-mon[29180]: pgmap v11665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20672.299058 secs 2024-04-27T00:19:51.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20673.088711 secs 2024-04-27T00:19:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20673.088711 secs 2024-04-27T00:19:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20673.088711 secs 2024-04-27T00:19:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:52 smithi119 ceph-mon[32752]: pgmap v11666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:52 smithi167 ceph-mon[32751]: pgmap v11666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:52 smithi090 ceph-mon[29180]: pgmap v11666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:54 smithi119 ceph-mon[32752]: pgmap v11667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:54 smithi167 ceph-mon[32751]: pgmap v11667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:54 smithi090 ceph-mon[29180]: pgmap v11667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:19:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20677.299143 secs 2024-04-27T00:19:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20677.299143 secs 2024-04-27T00:19:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20677.299143 secs 2024-04-27T00:19:56.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:56 smithi090 ceph-mon[29180]: pgmap v11668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:56.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20678.088831 secs 2024-04-27T00:19:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:56 smithi119 ceph-mon[32752]: pgmap v11668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20678.088831 secs 2024-04-27T00:19:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:56 smithi167 ceph-mon[32751]: pgmap v11668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20678.088831 secs 2024-04-27T00:19:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:19:58 smithi119 ceph-mon[32752]: pgmap v11669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:19:58 smithi167 ceph-mon[32751]: pgmap v11669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:19:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:19:58 smithi090 ceph-mon[29180]: pgmap v11669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:00 smithi119 ceph-mon[32752]: pgmap v11670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:20:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20682.299230 secs 2024-04-27T00:20:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:00 smithi167 ceph-mon[32751]: pgmap v11670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:20:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20682.299230 secs 2024-04-27T00:20:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:00 smithi090 ceph-mon[29180]: pgmap v11670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:20:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20682.299230 secs 2024-04-27T00:20:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20683.088951 secs 2024-04-27T00:20:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20683.088951 secs 2024-04-27T00:20:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20683.088951 secs 2024-04-27T00:20:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:02 smithi119 ceph-mon[32752]: pgmap v11671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:02 smithi167 ceph-mon[32751]: pgmap v11671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:02 smithi090 ceph-mon[29180]: pgmap v11671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:04 smithi119 ceph-mon[32752]: pgmap v11672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:04 smithi167 ceph-mon[32751]: pgmap v11672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:04 smithi090 ceph-mon[29180]: pgmap v11672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20687.299316 secs 2024-04-27T00:20:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20687.299316 secs 2024-04-27T00:20:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20687.299316 secs 2024-04-27T00:20:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:06 smithi167 ceph-mon[32751]: pgmap v11673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20688.089084 secs 2024-04-27T00:20:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:06 smithi090 ceph-mon[29180]: pgmap v11673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20688.089084 secs 2024-04-27T00:20:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:06 smithi119 ceph-mon[32752]: pgmap v11673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20688.089084 secs 2024-04-27T00:20:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:08 smithi167 ceph-mon[32751]: pgmap v11674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:08 smithi090 ceph-mon[29180]: pgmap v11674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:08 smithi119 ceph-mon[32752]: pgmap v11674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:10 smithi167 ceph-mon[32751]: pgmap v11675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20692.299400 secs 2024-04-27T00:20:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:10 smithi090 ceph-mon[29180]: pgmap v11675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20692.299400 secs 2024-04-27T00:20:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:10 smithi119 ceph-mon[32752]: pgmap v11675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20692.299400 secs 2024-04-27T00:20:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20693.089199 secs 2024-04-27T00:20:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20693.089199 secs 2024-04-27T00:20:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20693.089199 secs 2024-04-27T00:20:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:12 smithi090 ceph-mon[29180]: pgmap v11676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:12 smithi119 ceph-mon[32752]: pgmap v11676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:12 smithi167 ceph-mon[32751]: pgmap v11676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:14 smithi090 ceph-mon[29180]: pgmap v11677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:14 smithi119 ceph-mon[32752]: pgmap v11677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:14 smithi167 ceph-mon[32751]: pgmap v11677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20697.299485 secs 2024-04-27T00:20:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:20:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20697.299485 secs 2024-04-27T00:20:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:20:16.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20697.299485 secs 2024-04-27T00:20:16.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:20:17.124 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:17 smithi119 ceph-mon[32752]: pgmap v11678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:17.124 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20698.089285 secs 2024-04-27T00:20:17.124 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.124 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.125 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.125 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:17 smithi090 ceph-mon[29180]: pgmap v11678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20698.089285 secs 2024-04-27T00:20:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:17 smithi167 ceph-mon[32751]: pgmap v11678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20698.089285 secs 2024-04-27T00:20:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:18 smithi167 ceph-mon[32751]: pgmap v11679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:18 smithi090 ceph-mon[29180]: pgmap v11679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:18 smithi119 ceph-mon[32752]: pgmap v11679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:20:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:20:20.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:20:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:20:20.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:20:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:20:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:20:20.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:20:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:20:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:20:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:20:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:20:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: pgmap v11680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:20:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:20:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: pgmap v11681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:20:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: pgmap v11682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20702.299570 secs 2024-04-27T00:20:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: pgmap v11680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: pgmap v11681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: pgmap v11682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20702.299570 secs 2024-04-27T00:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: pgmap v11680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: pgmap v11681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:20:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: pgmap v11682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20702.299570 secs 2024-04-27T00:20:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20703.089401 secs 2024-04-27T00:20:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20703.089401 secs 2024-04-27T00:20:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20703.089401 secs 2024-04-27T00:20:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:22 smithi090 ceph-mon[29180]: pgmap v11683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T00:20:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:22 smithi119 ceph-mon[32752]: pgmap v11683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T00:20:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:22 smithi167 ceph-mon[32751]: pgmap v11683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T00:20:25.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:24 smithi090 ceph-mon[29180]: pgmap v11684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:24 smithi119 ceph-mon[32752]: pgmap v11684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:24 smithi167 ceph-mon[32751]: pgmap v11684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20707.299657 secs 2024-04-27T00:20:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20707.299657 secs 2024-04-27T00:20:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20707.299657 secs 2024-04-27T00:20:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:26 smithi119 ceph-mon[32752]: pgmap v11685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20708.089517 secs 2024-04-27T00:20:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:26 smithi167 ceph-mon[32751]: pgmap v11685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20708.089517 secs 2024-04-27T00:20:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:26 smithi090 ceph-mon[29180]: pgmap v11685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20708.089517 secs 2024-04-27T00:20:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:28 smithi119 ceph-mon[32752]: pgmap v11686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T00:20:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:28 smithi167 ceph-mon[32751]: pgmap v11686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T00:20:29.329 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:28 smithi090 ceph-mon[29180]: pgmap v11686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T00:20:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:30 smithi119 ceph-mon[32752]: pgmap v11687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:20:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20712.299741 secs 2024-04-27T00:20:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:30 smithi167 ceph-mon[32751]: pgmap v11687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:20:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20712.299741 secs 2024-04-27T00:20:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:30 smithi090 ceph-mon[29180]: pgmap v11687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:20:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20712.299741 secs 2024-04-27T00:20:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20713.089630 secs 2024-04-27T00:20:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20713.089630 secs 2024-04-27T00:20:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20713.089630 secs 2024-04-27T00:20:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:32 smithi119 ceph-mon[32752]: pgmap v11688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:32 smithi167 ceph-mon[32751]: pgmap v11688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:32 smithi090 ceph-mon[29180]: pgmap v11688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:34 smithi119 ceph-mon[32752]: pgmap v11689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:34 smithi167 ceph-mon[32751]: pgmap v11689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:34 smithi090 ceph-mon[29180]: pgmap v11689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20717.299848 secs 2024-04-27T00:20:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20717.299848 secs 2024-04-27T00:20:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20717.299848 secs 2024-04-27T00:20:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:36 smithi119 ceph-mon[32752]: pgmap v11690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20718.089757 secs 2024-04-27T00:20:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:36 smithi167 ceph-mon[32751]: pgmap v11690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20718.089757 secs 2024-04-27T00:20:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:36 smithi090 ceph-mon[29180]: pgmap v11690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20718.089757 secs 2024-04-27T00:20:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:38 smithi090 ceph-mon[29180]: pgmap v11691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:38 smithi119 ceph-mon[32752]: pgmap v11691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:38 smithi167 ceph-mon[32751]: pgmap v11691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:40 smithi119 ceph-mon[32752]: pgmap v11692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20722.299912 secs 2024-04-27T00:20:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:40 smithi167 ceph-mon[32751]: pgmap v11692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20722.299912 secs 2024-04-27T00:20:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:40 smithi090 ceph-mon[29180]: pgmap v11692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20722.299912 secs 2024-04-27T00:20:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20723.089851 secs 2024-04-27T00:20:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20723.089851 secs 2024-04-27T00:20:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20723.089851 secs 2024-04-27T00:20:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:42 smithi119 ceph-mon[32752]: pgmap v11693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:42 smithi167 ceph-mon[32751]: pgmap v11693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:42 smithi090 ceph-mon[29180]: pgmap v11693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:20:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:44 smithi119 ceph-mon[32752]: pgmap v11694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:44 smithi167 ceph-mon[32751]: pgmap v11694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:44 smithi090 ceph-mon[29180]: pgmap v11694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20727.300001 secs 2024-04-27T00:20:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20727.300001 secs 2024-04-27T00:20:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20727.300001 secs 2024-04-27T00:20:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:46 smithi119 ceph-mon[32752]: pgmap v11695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20728.089942 secs 2024-04-27T00:20:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:46 smithi167 ceph-mon[32751]: pgmap v11695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20728.089942 secs 2024-04-27T00:20:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:46 smithi090 ceph-mon[29180]: pgmap v11695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20728.089942 secs 2024-04-27T00:20:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:48 smithi119 ceph-mon[32752]: pgmap v11696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:48 smithi167 ceph-mon[32751]: pgmap v11696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:48 smithi090 ceph-mon[29180]: pgmap v11696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:50 smithi119 ceph-mon[32752]: pgmap v11697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20732.300087 secs 2024-04-27T00:20:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:50 smithi167 ceph-mon[32751]: pgmap v11697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20732.300087 secs 2024-04-27T00:20:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:50 smithi090 ceph-mon[29180]: pgmap v11697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20732.300087 secs 2024-04-27T00:20:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20733.090018 secs 2024-04-27T00:20:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20733.090018 secs 2024-04-27T00:20:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20733.090018 secs 2024-04-27T00:20:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:52 smithi119 ceph-mon[32752]: pgmap v11698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:52 smithi167 ceph-mon[32751]: pgmap v11698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:52 smithi090 ceph-mon[29180]: pgmap v11698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:54 smithi119 ceph-mon[32752]: pgmap v11699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:54 smithi167 ceph-mon[32751]: pgmap v11699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:54 smithi090 ceph-mon[29180]: pgmap v11699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:20:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20737.300146 secs 2024-04-27T00:20:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20737.300146 secs 2024-04-27T00:20:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20737.300146 secs 2024-04-27T00:20:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:56 smithi119 ceph-mon[32752]: pgmap v11700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20738.090140 secs 2024-04-27T00:20:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:56 smithi167 ceph-mon[32751]: pgmap v11700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20738.090140 secs 2024-04-27T00:20:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:56 smithi090 ceph-mon[29180]: pgmap v11700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20738.090140 secs 2024-04-27T00:20:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:20:58 smithi119 ceph-mon[32752]: pgmap v11701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:20:58 smithi167 ceph-mon[32751]: pgmap v11701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:20:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:20:58 smithi090 ceph-mon[29180]: pgmap v11701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:00 smithi119 ceph-mon[32752]: pgmap v11702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20742.300231 secs 2024-04-27T00:21:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:00 smithi167 ceph-mon[32751]: pgmap v11702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20742.300231 secs 2024-04-27T00:21:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:00 smithi090 ceph-mon[29180]: pgmap v11702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20742.300231 secs 2024-04-27T00:21:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20743.090255 secs 2024-04-27T00:21:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20743.090255 secs 2024-04-27T00:21:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20743.090255 secs 2024-04-27T00:21:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:02 smithi119 ceph-mon[32752]: pgmap v11703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:02 smithi167 ceph-mon[32751]: pgmap v11703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:02 smithi090 ceph-mon[29180]: pgmap v11703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:04 smithi119 ceph-mon[32752]: pgmap v11704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:04 smithi167 ceph-mon[32751]: pgmap v11704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:04 smithi090 ceph-mon[29180]: pgmap v11704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20747.300292 secs 2024-04-27T00:21:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20747.300292 secs 2024-04-27T00:21:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20747.300292 secs 2024-04-27T00:21:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:06 smithi119 ceph-mon[32752]: pgmap v11705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20748.090385 secs 2024-04-27T00:21:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:06 smithi167 ceph-mon[32751]: pgmap v11705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20748.090385 secs 2024-04-27T00:21:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:06 smithi090 ceph-mon[29180]: pgmap v11705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20748.090385 secs 2024-04-27T00:21:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:08 smithi119 ceph-mon[32752]: pgmap v11706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:08 smithi167 ceph-mon[32751]: pgmap v11706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:08 smithi090 ceph-mon[29180]: pgmap v11706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:10 smithi119 ceph-mon[32752]: pgmap v11707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:10 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 20752.300380 secs 2024-04-27T00:21:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:10 smithi119 ceph-mon[32752]: slow request 362.505416 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:21:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:10 smithi167 ceph-mon[32751]: pgmap v11707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:10 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 20752.300380 secs 2024-04-27T00:21:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:10 smithi167 ceph-mon[32751]: slow request 362.505416 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:21:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:10 smithi090 ceph-mon[29180]: pgmap v11707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:10 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 20752.300380 secs 2024-04-27T00:21:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:10 smithi090 ceph-mon[29180]: slow request 362.505416 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:21:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20753.090394 secs 2024-04-27T00:21:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20753.090394 secs 2024-04-27T00:21:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20753.090394 secs 2024-04-27T00:21:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:12 smithi119 ceph-mon[32752]: pgmap v11708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:12 smithi167 ceph-mon[32751]: pgmap v11708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:12 smithi090 ceph-mon[29180]: pgmap v11708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:14 smithi119 ceph-mon[32752]: pgmap v11709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:14 smithi167 ceph-mon[32751]: pgmap v11709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:14 smithi090 ceph-mon[29180]: pgmap v11709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20757.300464 secs 2024-04-27T00:21:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20757.300464 secs 2024-04-27T00:21:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20757.300464 secs 2024-04-27T00:21:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:17 smithi119 ceph-mon[32752]: pgmap v11710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20758.090472 secs 2024-04-27T00:21:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:17 smithi167 ceph-mon[32751]: pgmap v11710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20758.090472 secs 2024-04-27T00:21:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:17 smithi090 ceph-mon[29180]: pgmap v11710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20758.090472 secs 2024-04-27T00:21:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:19 smithi119 ceph-mon[32752]: pgmap v11711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:19 smithi167 ceph-mon[32751]: pgmap v11711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:19 smithi090 ceph-mon[29180]: pgmap v11711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:21:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:21:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:21:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:21 smithi090 ceph-mon[29180]: pgmap v11712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20762.300559 secs 2024-04-27T00:21:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:21:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:21:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:21 smithi119 ceph-mon[32752]: pgmap v11712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20762.300559 secs 2024-04-27T00:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:21:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:21 smithi167 ceph-mon[32751]: pgmap v11712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20762.300559 secs 2024-04-27T00:21:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:21:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:21:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:22 smithi090 ceph-mon[29180]: pgmap v11713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:21:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:22 smithi090 ceph-mon[29180]: pgmap v11714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20763.090540 secs 2024-04-27T00:21:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:22 smithi119 ceph-mon[32752]: pgmap v11713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:21:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:22 smithi119 ceph-mon[32752]: pgmap v11714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20763.090540 secs 2024-04-27T00:21:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:22 smithi167 ceph-mon[32751]: pgmap v11713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:21:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:22 smithi167 ceph-mon[32751]: pgmap v11714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20763.090540 secs 2024-04-27T00:21:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:24 smithi090 ceph-mon[29180]: pgmap v11715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:21:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:24 smithi119 ceph-mon[32752]: pgmap v11715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:21:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:24 smithi167 ceph-mon[32751]: pgmap v11715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:21:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:26 smithi090 ceph-mon[29180]: pgmap v11716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20767.300644 secs 2024-04-27T00:21:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:26 smithi119 ceph-mon[32752]: pgmap v11716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20767.300644 secs 2024-04-27T00:21:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:26 smithi167 ceph-mon[32751]: pgmap v11716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20767.300644 secs 2024-04-27T00:21:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20768.090672 secs 2024-04-27T00:21:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20768.090672 secs 2024-04-27T00:21:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20768.090672 secs 2024-04-27T00:21:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:28 smithi119 ceph-mon[32752]: pgmap v11717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:28 smithi167 ceph-mon[32751]: pgmap v11717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:28 smithi090 ceph-mon[29180]: pgmap v11717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:30 smithi119 ceph-mon[32752]: pgmap v11718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:30 smithi167 ceph-mon[32751]: pgmap v11718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:30 smithi090 ceph-mon[29180]: pgmap v11718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:21:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20772.300729 secs 2024-04-27T00:21:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20772.300729 secs 2024-04-27T00:21:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20772.300729 secs 2024-04-27T00:21:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:32 smithi090 ceph-mon[29180]: pgmap v11719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:21:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20773.090771 secs 2024-04-27T00:21:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:32 smithi119 ceph-mon[32752]: pgmap v11719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:21:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20773.090771 secs 2024-04-27T00:21:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:32 smithi167 ceph-mon[32751]: pgmap v11719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:21:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20773.090771 secs 2024-04-27T00:21:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:34 smithi119 ceph-mon[32752]: pgmap v11720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:34 smithi167 ceph-mon[32751]: pgmap v11720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:34 smithi090 ceph-mon[29180]: pgmap v11720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:36.523 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:36 smithi090 ceph-mon[29180]: pgmap v11721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:36.523 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20777.300815 secs 2024-04-27T00:21:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:36 smithi119 ceph-mon[32752]: pgmap v11721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20777.300815 secs 2024-04-27T00:21:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:36 smithi167 ceph-mon[32751]: pgmap v11721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20777.300815 secs 2024-04-27T00:21:36.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:21:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:21:36.521+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:21:36.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:21:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:21:36.521+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:21:36.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:21:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:21:36.521+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:21:36.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:21:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:21:36.521+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:21:36.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:21:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:21:36.521+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:21:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20778.090905 secs 2024-04-27T00:21:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20778.090905 secs 2024-04-27T00:21:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20778.090905 secs 2024-04-27T00:21:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:38 smithi119 ceph-mon[32752]: pgmap v11722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:38 smithi167 ceph-mon[32751]: pgmap v11722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:38 smithi090 ceph-mon[29180]: pgmap v11722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:40 smithi119 ceph-mon[32752]: pgmap v11723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:40 smithi167 ceph-mon[32751]: pgmap v11723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:40 smithi090 ceph-mon[29180]: pgmap v11723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20782.300900 secs 2024-04-27T00:21:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20782.300900 secs 2024-04-27T00:21:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20782.300900 secs 2024-04-27T00:21:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:42 smithi119 ceph-mon[32752]: pgmap v11724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20783.091036 secs 2024-04-27T00:21:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:42 smithi167 ceph-mon[32751]: pgmap v11724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20783.091036 secs 2024-04-27T00:21:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:42 smithi090 ceph-mon[29180]: pgmap v11724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20783.091036 secs 2024-04-27T00:21:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:44 smithi119 ceph-mon[32752]: pgmap v11725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:44 smithi167 ceph-mon[32751]: pgmap v11725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:44 smithi090 ceph-mon[29180]: pgmap v11725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:46 smithi119 ceph-mon[32752]: pgmap v11726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20787.300960 secs 2024-04-27T00:21:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:46 smithi167 ceph-mon[32751]: pgmap v11726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20787.300960 secs 2024-04-27T00:21:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:46 smithi090 ceph-mon[29180]: pgmap v11726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20787.300960 secs 2024-04-27T00:21:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20788.091165 secs 2024-04-27T00:21:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20788.091165 secs 2024-04-27T00:21:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20788.091165 secs 2024-04-27T00:21:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:48 smithi119 ceph-mon[32752]: pgmap v11727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:48 smithi167 ceph-mon[32751]: pgmap v11727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:48 smithi090 ceph-mon[29180]: pgmap v11727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:50 smithi119 ceph-mon[32752]: pgmap v11728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:50 smithi167 ceph-mon[32751]: pgmap v11728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:50 smithi090 ceph-mon[29180]: pgmap v11728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:21:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20792.301024 secs 2024-04-27T00:21:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20792.301024 secs 2024-04-27T00:21:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20792.301024 secs 2024-04-27T00:21:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:52 smithi119 ceph-mon[32752]: pgmap v11729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20793.091283 secs 2024-04-27T00:21:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:52 smithi167 ceph-mon[32751]: pgmap v11729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20793.091283 secs 2024-04-27T00:21:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:52 smithi090 ceph-mon[29180]: pgmap v11729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20793.091283 secs 2024-04-27T00:21:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:54 smithi119 ceph-mon[32752]: pgmap v11730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:54 smithi167 ceph-mon[32751]: pgmap v11730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:54 smithi090 ceph-mon[29180]: pgmap v11730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20797.301087 secs 2024-04-27T00:21:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20797.301087 secs 2024-04-27T00:21:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20797.301087 secs 2024-04-27T00:21:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:56 smithi119 ceph-mon[32752]: pgmap v11731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:56 smithi167 ceph-mon[32751]: pgmap v11731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:56 smithi090 ceph-mon[29180]: pgmap v11731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:21:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20798.091402 secs 2024-04-27T00:21:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20798.091402 secs 2024-04-27T00:21:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20798.091402 secs 2024-04-27T00:21:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:21:58 smithi119 ceph-mon[32752]: pgmap v11732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:21:58 smithi167 ceph-mon[32751]: pgmap v11732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:21:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:21:58 smithi090 ceph-mon[29180]: pgmap v11732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:00 smithi119 ceph-mon[32752]: pgmap v11733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20802.301148 secs 2024-04-27T00:22:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:00 smithi167 ceph-mon[32751]: pgmap v11733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20802.301148 secs 2024-04-27T00:22:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:00 smithi090 ceph-mon[29180]: pgmap v11733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20802.301148 secs 2024-04-27T00:22:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:02 smithi119 ceph-mon[32752]: pgmap v11734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20803.091485 secs 2024-04-27T00:22:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:02 smithi167 ceph-mon[32751]: pgmap v11734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20803.091485 secs 2024-04-27T00:22:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:02 smithi090 ceph-mon[29180]: pgmap v11734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20803.091485 secs 2024-04-27T00:22:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:04 smithi119 ceph-mon[32752]: pgmap v11735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:04 smithi167 ceph-mon[32751]: pgmap v11735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:04 smithi090 ceph-mon[29180]: pgmap v11735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20807.301232 secs 2024-04-27T00:22:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20807.301232 secs 2024-04-27T00:22:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20807.301232 secs 2024-04-27T00:22:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:06 smithi119 ceph-mon[32752]: pgmap v11736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:06 smithi167 ceph-mon[32751]: pgmap v11736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:06 smithi090 ceph-mon[29180]: pgmap v11736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:07 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 20808.091603 secs 2024-04-27T00:22:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:07 smithi119 ceph-mon[32752]: slow request 2880.164160 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:22:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:07 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 20808.091603 secs 2024-04-27T00:22:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:07 smithi167 ceph-mon[32751]: slow request 2880.164160 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:22:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:07 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 20808.091603 secs 2024-04-27T00:22:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:07 smithi090 ceph-mon[29180]: slow request 2880.164160 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:22:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:08 smithi119 ceph-mon[32752]: pgmap v11737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:08 smithi167 ceph-mon[32751]: pgmap v11737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:08 smithi090 ceph-mon[29180]: pgmap v11737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:10 smithi119 ceph-mon[32752]: pgmap v11738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20812.301316 secs 2024-04-27T00:22:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:10 smithi167 ceph-mon[32751]: pgmap v11738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20812.301316 secs 2024-04-27T00:22:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:10 smithi090 ceph-mon[29180]: pgmap v11738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20812.301316 secs 2024-04-27T00:22:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:12 smithi119 ceph-mon[32752]: pgmap v11739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20813.091719 secs 2024-04-27T00:22:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:12 smithi167 ceph-mon[32751]: pgmap v11739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20813.091719 secs 2024-04-27T00:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:12 smithi090 ceph-mon[29180]: pgmap v11739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20813.091719 secs 2024-04-27T00:22:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:14 smithi119 ceph-mon[32752]: pgmap v11740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:14 smithi167 ceph-mon[32751]: pgmap v11740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:14 smithi090 ceph-mon[29180]: pgmap v11740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:22:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20817.301398 secs 2024-04-27T00:22:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20817.301398 secs 2024-04-27T00:22:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20817.301398 secs 2024-04-27T00:22:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:16 smithi119 ceph-mon[32752]: pgmap v11741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:16 smithi167 ceph-mon[32751]: pgmap v11741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:16 smithi090 ceph-mon[29180]: pgmap v11741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:22:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20818.091829 secs 2024-04-27T00:22:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20818.091829 secs 2024-04-27T00:22:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20818.091829 secs 2024-04-27T00:22:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:18 smithi119 ceph-mon[32752]: pgmap v11742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:18 smithi167 ceph-mon[32751]: pgmap v11742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:18 smithi090 ceph-mon[29180]: pgmap v11742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:20 smithi119 ceph-mon[32752]: pgmap v11743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20822.301482 secs 2024-04-27T00:22:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:20 smithi167 ceph-mon[32751]: pgmap v11743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20822.301482 secs 2024-04-27T00:22:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:20 smithi090 ceph-mon[29180]: pgmap v11743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20822.301482 secs 2024-04-27T00:22:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:22:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:22:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:22:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:22 smithi119 ceph-mon[32752]: pgmap v11744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20823.091974 secs 2024-04-27T00:22:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:22:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:22:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:22:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:22 smithi167 ceph-mon[32751]: pgmap v11744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20823.091974 secs 2024-04-27T00:22:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:22:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:22:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:22:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:22 smithi090 ceph-mon[29180]: pgmap v11744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20823.091974 secs 2024-04-27T00:22:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:22:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:22:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:22:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:23 smithi119 ceph-mon[32752]: pgmap v11745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:22:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:23 smithi119 ceph-mon[32752]: pgmap v11746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:23 smithi167 ceph-mon[32751]: pgmap v11745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:22:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:23 smithi167 ceph-mon[32751]: pgmap v11746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:23 smithi090 ceph-mon[29180]: pgmap v11745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:22:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:23 smithi090 ceph-mon[29180]: pgmap v11746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:25 smithi119 ceph-mon[32752]: pgmap v11747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20827.301566 secs 2024-04-27T00:22:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:25 smithi167 ceph-mon[32751]: pgmap v11747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20827.301566 secs 2024-04-27T00:22:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:25 smithi090 ceph-mon[29180]: pgmap v11747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20827.301566 secs 2024-04-27T00:22:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:27 smithi119 ceph-mon[32752]: pgmap v11748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20828.092092 secs 2024-04-27T00:22:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:27 smithi167 ceph-mon[32751]: pgmap v11748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20828.092092 secs 2024-04-27T00:22:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:27 smithi090 ceph-mon[29180]: pgmap v11748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20828.092092 secs 2024-04-27T00:22:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:29 smithi119 ceph-mon[32752]: pgmap v11749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:22:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:29 smithi167 ceph-mon[32751]: pgmap v11749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:22:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:29 smithi090 ceph-mon[29180]: pgmap v11749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:22:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20832.301647 secs 2024-04-27T00:22:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20832.301647 secs 2024-04-27T00:22:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20832.301647 secs 2024-04-27T00:22:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:31 smithi119 ceph-mon[32752]: pgmap v11750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:31 smithi167 ceph-mon[32751]: pgmap v11750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:31 smithi090 ceph-mon[29180]: pgmap v11750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:22:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20833.092232 secs 2024-04-27T00:22:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:32 smithi119 ceph-mon[32752]: pgmap v11751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:22:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20833.092232 secs 2024-04-27T00:22:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:32 smithi167 ceph-mon[32751]: pgmap v11751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:22:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20833.092232 secs 2024-04-27T00:22:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:32 smithi090 ceph-mon[29180]: pgmap v11751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:22:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:34 smithi090 ceph-mon[29180]: pgmap v11752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:35.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:34 smithi119 ceph-mon[32752]: pgmap v11752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:34 smithi167 ceph-mon[32751]: pgmap v11752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20837.301708 secs 2024-04-27T00:22:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20837.301708 secs 2024-04-27T00:22:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20837.301708 secs 2024-04-27T00:22:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:36 smithi090 ceph-mon[29180]: pgmap v11753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20838.092357 secs 2024-04-27T00:22:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:36 smithi119 ceph-mon[32752]: pgmap v11753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20838.092357 secs 2024-04-27T00:22:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:36 smithi167 ceph-mon[32751]: pgmap v11753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20838.092357 secs 2024-04-27T00:22:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:38 smithi090 ceph-mon[29180]: pgmap v11754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:38 smithi119 ceph-mon[32752]: pgmap v11754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:38 smithi167 ceph-mon[32751]: pgmap v11754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:40 smithi090 ceph-mon[29180]: pgmap v11755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20842.301815 secs 2024-04-27T00:22:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:40 smithi119 ceph-mon[32752]: pgmap v11755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20842.301815 secs 2024-04-27T00:22:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:40 smithi167 ceph-mon[32751]: pgmap v11755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20842.301815 secs 2024-04-27T00:22:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20843.092495 secs 2024-04-27T00:22:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20843.092495 secs 2024-04-27T00:22:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20843.092495 secs 2024-04-27T00:22:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:42 smithi090 ceph-mon[29180]: pgmap v11756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:42 smithi119 ceph-mon[32752]: pgmap v11756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:42 smithi167 ceph-mon[32751]: pgmap v11756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:44 smithi090 ceph-mon[29180]: pgmap v11757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:44 smithi119 ceph-mon[32752]: pgmap v11757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:44 smithi167 ceph-mon[32751]: pgmap v11757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20847.301875 secs 2024-04-27T00:22:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20847.301875 secs 2024-04-27T00:22:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20847.301875 secs 2024-04-27T00:22:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:46 smithi090 ceph-mon[29180]: pgmap v11758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20848.092625 secs 2024-04-27T00:22:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:46 smithi119 ceph-mon[32752]: pgmap v11758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20848.092625 secs 2024-04-27T00:22:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:46 smithi167 ceph-mon[32751]: pgmap v11758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20848.092625 secs 2024-04-27T00:22:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:48 smithi090 ceph-mon[29180]: pgmap v11759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:48 smithi119 ceph-mon[32752]: pgmap v11759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:48 smithi167 ceph-mon[32751]: pgmap v11759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:50.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:50 smithi167 ceph-mon[32751]: pgmap v11760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:50.871 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20852.301938 secs 2024-04-27T00:22:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:50 smithi090 ceph-mon[29180]: pgmap v11760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20852.301938 secs 2024-04-27T00:22:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:50 smithi119 ceph-mon[32752]: pgmap v11760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20852.301938 secs 2024-04-27T00:22:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20853.092753 secs 2024-04-27T00:22:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20853.092753 secs 2024-04-27T00:22:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20853.092753 secs 2024-04-27T00:22:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:52 smithi090 ceph-mon[29180]: pgmap v11761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:52 smithi119 ceph-mon[32752]: pgmap v11761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:52 smithi167 ceph-mon[32751]: pgmap v11761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:54 smithi090 ceph-mon[29180]: pgmap v11762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:54 smithi119 ceph-mon[32752]: pgmap v11762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:54 smithi167 ceph-mon[32751]: pgmap v11762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:22:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20857.301999 secs 2024-04-27T00:22:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20857.301999 secs 2024-04-27T00:22:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20857.301999 secs 2024-04-27T00:22:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:56 smithi119 ceph-mon[32752]: pgmap v11763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20858.092856 secs 2024-04-27T00:22:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:56 smithi167 ceph-mon[32751]: pgmap v11763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20858.092856 secs 2024-04-27T00:22:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:56 smithi090 ceph-mon[29180]: pgmap v11763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20858.092856 secs 2024-04-27T00:22:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:22:58 smithi119 ceph-mon[32752]: pgmap v11764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:22:58 smithi167 ceph-mon[32751]: pgmap v11764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:22:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:22:58 smithi090 ceph-mon[29180]: pgmap v11764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:00 smithi119 ceph-mon[32752]: pgmap v11765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:00 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 20862.302083 secs 2024-04-27T00:23:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:00 smithi119 ceph-mon[32752]: slow request 11520.300080 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:23:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:00 smithi167 ceph-mon[32751]: pgmap v11765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:00 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 20862.302083 secs 2024-04-27T00:23:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:00 smithi167 ceph-mon[32751]: slow request 11520.300080 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:23:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:00 smithi090 ceph-mon[29180]: pgmap v11765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:00 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 20862.302083 secs 2024-04-27T00:23:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:00 smithi090 ceph-mon[29180]: slow request 11520.300080 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:23:02.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20863.092976 secs 2024-04-27T00:23:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20863.092976 secs 2024-04-27T00:23:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20863.092976 secs 2024-04-27T00:23:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:02 smithi119 ceph-mon[32752]: pgmap v11766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:02 smithi167 ceph-mon[32751]: pgmap v11766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:02 smithi090 ceph-mon[29180]: pgmap v11766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:04 smithi119 ceph-mon[32752]: pgmap v11767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:04 smithi167 ceph-mon[32751]: pgmap v11767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:05.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:04 smithi090 ceph-mon[29180]: pgmap v11767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20867.302144 secs 2024-04-27T00:23:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20867.302144 secs 2024-04-27T00:23:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20867.302144 secs 2024-04-27T00:23:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:06 smithi119 ceph-mon[32752]: pgmap v11768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:06 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 20868.093099 secs 2024-04-27T00:23:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:06 smithi119 ceph-mon[32752]: slow request 5760.961487 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:23:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:06 smithi167 ceph-mon[32751]: pgmap v11768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:06 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 20868.093099 secs 2024-04-27T00:23:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:06 smithi167 ceph-mon[32751]: slow request 5760.961487 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:23:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:06 smithi090 ceph-mon[29180]: pgmap v11768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:06 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 20868.093099 secs 2024-04-27T00:23:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:06 smithi090 ceph-mon[29180]: slow request 5760.961487 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:23:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:08 smithi119 ceph-mon[32752]: pgmap v11769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:08 smithi167 ceph-mon[32751]: pgmap v11769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:09.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:08 smithi090 ceph-mon[29180]: pgmap v11769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:10 smithi119 ceph-mon[32752]: pgmap v11770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:10 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 20872.302238 secs 2024-04-27T00:23:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:10 smithi119 ceph-mon[32752]: slow request 723.709684 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:23:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:10 smithi167 ceph-mon[32751]: pgmap v11770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:10 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 20872.302238 secs 2024-04-27T00:23:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:10 smithi167 ceph-mon[32751]: slow request 723.709684 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:23:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:10 smithi090 ceph-mon[29180]: pgmap v11770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:10 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 20872.302238 secs 2024-04-27T00:23:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:10 smithi090 ceph-mon[29180]: slow request 723.709684 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:23:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20873.093211 secs 2024-04-27T00:23:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20873.093211 secs 2024-04-27T00:23:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20873.093211 secs 2024-04-27T00:23:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:12 smithi119 ceph-mon[32752]: pgmap v11771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:12 smithi167 ceph-mon[32751]: pgmap v11771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:12 smithi090 ceph-mon[29180]: pgmap v11771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:14 smithi119 ceph-mon[32752]: pgmap v11772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:14 smithi167 ceph-mon[32751]: pgmap v11772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:14 smithi090 ceph-mon[29180]: pgmap v11772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20877.302281 secs 2024-04-27T00:23:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20877.302281 secs 2024-04-27T00:23:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20877.302281 secs 2024-04-27T00:23:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:16 smithi119 ceph-mon[32752]: pgmap v11773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20878.093318 secs 2024-04-27T00:23:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:16 smithi167 ceph-mon[32751]: pgmap v11773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20878.093318 secs 2024-04-27T00:23:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:16 smithi090 ceph-mon[29180]: pgmap v11773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20878.093318 secs 2024-04-27T00:23:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:18 smithi119 ceph-mon[32752]: pgmap v11774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:18 smithi167 ceph-mon[32751]: pgmap v11774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:18 smithi090 ceph-mon[29180]: pgmap v11774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:20 smithi119 ceph-mon[32752]: pgmap v11775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20882.302342 secs 2024-04-27T00:23:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:20 smithi167 ceph-mon[32751]: pgmap v11775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20882.302342 secs 2024-04-27T00:23:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:20 smithi090 ceph-mon[29180]: pgmap v11775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20882.302342 secs 2024-04-27T00:23:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20883.093440 secs 2024-04-27T00:23:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:23:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20883.093440 secs 2024-04-27T00:23:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:23:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20883.093440 secs 2024-04-27T00:23:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:23:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:22 smithi119 ceph-mon[32752]: pgmap v11776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:23:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:23:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:22 smithi119 ceph-mon[32752]: pgmap v11777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:23:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:22 smithi119 ceph-mon[32752]: pgmap v11778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:23:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:23:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:22 smithi167 ceph-mon[32751]: pgmap v11776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:23:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:23:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:22 smithi167 ceph-mon[32751]: pgmap v11777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:23:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:22 smithi167 ceph-mon[32751]: pgmap v11778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:23:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:23:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:22 smithi090 ceph-mon[29180]: pgmap v11776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:23:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:23:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:22 smithi090 ceph-mon[29180]: pgmap v11777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:23:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:22 smithi090 ceph-mon[29180]: pgmap v11778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:23:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:23:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:24 smithi090 ceph-mon[29180]: pgmap v11779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:24 smithi119 ceph-mon[32752]: pgmap v11779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:24 smithi167 ceph-mon[32751]: pgmap v11779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20887.302429 secs 2024-04-27T00:23:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20887.302429 secs 2024-04-27T00:23:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20887.302429 secs 2024-04-27T00:23:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:26 smithi090 ceph-mon[29180]: pgmap v11780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:26 smithi119 ceph-mon[32752]: pgmap v11780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:26 smithi167 ceph-mon[32751]: pgmap v11780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20888.093515 secs 2024-04-27T00:23:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20888.093515 secs 2024-04-27T00:23:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20888.093515 secs 2024-04-27T00:23:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:28 smithi090 ceph-mon[29180]: pgmap v11781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:23:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:28 smithi119 ceph-mon[32752]: pgmap v11781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:23:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:28 smithi167 ceph-mon[32751]: pgmap v11781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:23:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20892.302513 secs 2024-04-27T00:23:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20892.302513 secs 2024-04-27T00:23:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20892.302513 secs 2024-04-27T00:23:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:31 smithi119 ceph-mon[32752]: pgmap v11782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:31 smithi167 ceph-mon[32751]: pgmap v11782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:31 smithi090 ceph-mon[29180]: pgmap v11782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:23:32.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20893.093595 secs 2024-04-27T00:23:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20893.093595 secs 2024-04-27T00:23:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20893.093595 secs 2024-04-27T00:23:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:33 smithi119 ceph-mon[32752]: pgmap v11783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:23:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:33 smithi167 ceph-mon[32751]: pgmap v11783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:23:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:33 smithi090 ceph-mon[29180]: pgmap v11783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:23:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:34 smithi090 ceph-mon[29180]: pgmap v11784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:34 smithi119 ceph-mon[32752]: pgmap v11784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:34 smithi167 ceph-mon[32751]: pgmap v11784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20897.302598 secs 2024-04-27T00:23:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20897.302598 secs 2024-04-27T00:23:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20897.302598 secs 2024-04-27T00:23:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:36 smithi090 ceph-mon[29180]: pgmap v11785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:36 smithi119 ceph-mon[32752]: pgmap v11785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:36 smithi167 ceph-mon[32751]: pgmap v11785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20898.093695 secs 2024-04-27T00:23:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20898.093695 secs 2024-04-27T00:23:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20898.093695 secs 2024-04-27T00:23:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:38 smithi090 ceph-mon[29180]: pgmap v11786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:38 smithi119 ceph-mon[32752]: pgmap v11786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:38 smithi167 ceph-mon[32751]: pgmap v11786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20902.302666 secs 2024-04-27T00:23:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20902.302666 secs 2024-04-27T00:23:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20902.302666 secs 2024-04-27T00:23:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:41 smithi119 ceph-mon[32752]: pgmap v11787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:41 smithi167 ceph-mon[32751]: pgmap v11787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:41 smithi090 ceph-mon[29180]: pgmap v11787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20903.093811 secs 2024-04-27T00:23:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20903.093811 secs 2024-04-27T00:23:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20903.093811 secs 2024-04-27T00:23:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:43 smithi119 ceph-mon[32752]: pgmap v11788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:43 smithi167 ceph-mon[32751]: pgmap v11788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:43 smithi090 ceph-mon[29180]: pgmap v11788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:44 smithi090 ceph-mon[29180]: pgmap v11789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:44 smithi119 ceph-mon[32752]: pgmap v11789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:44 smithi167 ceph-mon[32751]: pgmap v11789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20907.302753 secs 2024-04-27T00:23:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20907.302753 secs 2024-04-27T00:23:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20907.302753 secs 2024-04-27T00:23:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:46 smithi090 ceph-mon[29180]: pgmap v11790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:46 smithi119 ceph-mon[32752]: pgmap v11790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:46 smithi167 ceph-mon[32751]: pgmap v11790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20908.093933 secs 2024-04-27T00:23:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20908.093933 secs 2024-04-27T00:23:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20908.093933 secs 2024-04-27T00:23:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:48 smithi090 ceph-mon[29180]: pgmap v11791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:48 smithi119 ceph-mon[32752]: pgmap v11791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:48 smithi167 ceph-mon[32751]: pgmap v11791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:50.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20912.302838 secs 2024-04-27T00:23:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20912.302838 secs 2024-04-27T00:23:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20912.302838 secs 2024-04-27T00:23:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:51 smithi119 ceph-mon[32752]: pgmap v11792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:51 smithi167 ceph-mon[32751]: pgmap v11792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:51 smithi090 ceph-mon[29180]: pgmap v11792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20913.094002 secs 2024-04-27T00:23:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20913.094002 secs 2024-04-27T00:23:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20913.094002 secs 2024-04-27T00:23:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:53 smithi119 ceph-mon[32752]: pgmap v11793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:53 smithi167 ceph-mon[32751]: pgmap v11793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:53 smithi090 ceph-mon[29180]: pgmap v11793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:54 smithi090 ceph-mon[29180]: pgmap v11794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:54 smithi119 ceph-mon[32752]: pgmap v11794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:54 smithi167 ceph-mon[32751]: pgmap v11794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:23:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20917.302916 secs 2024-04-27T00:23:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20917.302916 secs 2024-04-27T00:23:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20917.302916 secs 2024-04-27T00:23:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:56 smithi090 ceph-mon[29180]: pgmap v11795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:56 smithi119 ceph-mon[32752]: pgmap v11795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:56 smithi167 ceph-mon[32751]: pgmap v11795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20918.094119 secs 2024-04-27T00:23:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20918.094119 secs 2024-04-27T00:23:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20918.094119 secs 2024-04-27T00:23:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:23:59 smithi090 ceph-mon[29180]: pgmap v11796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:23:59 smithi119 ceph-mon[32752]: pgmap v11796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:23:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:23:59 smithi167 ceph-mon[32751]: pgmap v11796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:00.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20922.302979 secs 2024-04-27T00:24:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20922.302979 secs 2024-04-27T00:24:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20922.302979 secs 2024-04-27T00:24:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:01 smithi119 ceph-mon[32752]: pgmap v11797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:01 smithi167 ceph-mon[32751]: pgmap v11797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:01 smithi090 ceph-mon[29180]: pgmap v11797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:02.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20923.094181 secs 2024-04-27T00:24:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20923.094181 secs 2024-04-27T00:24:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20923.094181 secs 2024-04-27T00:24:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:03 smithi119 ceph-mon[32752]: pgmap v11798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:03 smithi167 ceph-mon[32751]: pgmap v11798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:03 smithi090 ceph-mon[29180]: pgmap v11798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:05 smithi119 ceph-mon[32752]: pgmap v11799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20927.303066 secs 2024-04-27T00:24:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:05 smithi167 ceph-mon[32751]: pgmap v11799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20927.303066 secs 2024-04-27T00:24:05.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:05 smithi090 ceph-mon[29180]: pgmap v11799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:05.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20927.303066 secs 2024-04-27T00:24:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20928.094319 secs 2024-04-27T00:24:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:07 smithi119 ceph-mon[32752]: pgmap v11800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20928.094319 secs 2024-04-27T00:24:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:07 smithi167 ceph-mon[32751]: pgmap v11800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20928.094319 secs 2024-04-27T00:24:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:07 smithi090 ceph-mon[29180]: pgmap v11800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:09 smithi119 ceph-mon[32752]: pgmap v11801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:09 smithi167 ceph-mon[32751]: pgmap v11801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:09 smithi090 ceph-mon[29180]: pgmap v11801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:10 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 20932.303153 secs 2024-04-27T00:24:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:10 smithi119 ceph-mon[32752]: slow request 1440.937351 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:24:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:10 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 20932.303153 secs 2024-04-27T00:24:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:10 smithi167 ceph-mon[32751]: slow request 1440.937351 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:24:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:10 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 20932.303153 secs 2024-04-27T00:24:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:10 smithi090 ceph-mon[29180]: slow request 1440.937351 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:24:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:11 smithi119 ceph-mon[32752]: pgmap v11802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:11 smithi167 ceph-mon[32751]: pgmap v11802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:11 smithi090 ceph-mon[29180]: pgmap v11802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20933.094450 secs 2024-04-27T00:24:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20933.094450 secs 2024-04-27T00:24:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20933.094450 secs 2024-04-27T00:24:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:13 smithi119 ceph-mon[32752]: pgmap v11803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:13 smithi167 ceph-mon[32751]: pgmap v11803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:13 smithi090 ceph-mon[29180]: pgmap v11803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:15 smithi119 ceph-mon[32752]: pgmap v11804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20937.303237 secs 2024-04-27T00:24:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:15 smithi167 ceph-mon[32751]: pgmap v11804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20937.303237 secs 2024-04-27T00:24:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:15 smithi090 ceph-mon[29180]: pgmap v11804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20937.303237 secs 2024-04-27T00:24:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20938.094545 secs 2024-04-27T00:24:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:17 smithi119 ceph-mon[32752]: pgmap v11805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20938.094545 secs 2024-04-27T00:24:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:17 smithi167 ceph-mon[32751]: pgmap v11805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20938.094545 secs 2024-04-27T00:24:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:17 smithi090 ceph-mon[29180]: pgmap v11805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:19 smithi119 ceph-mon[32752]: pgmap v11806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:19 smithi167 ceph-mon[32751]: pgmap v11806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:19 smithi090 ceph-mon[29180]: pgmap v11806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:20.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20942.303323 secs 2024-04-27T00:24:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20942.303323 secs 2024-04-27T00:24:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20942.303323 secs 2024-04-27T00:24:21.732 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:21 smithi090 ceph-mon[29180]: pgmap v11807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:21 smithi119 ceph-mon[32752]: pgmap v11807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:21 smithi167 ceph-mon[32751]: pgmap v11807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20943.094662 secs 2024-04-27T00:24:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:24:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20943.094662 secs 2024-04-27T00:24:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:24:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20943.094662 secs 2024-04-27T00:24:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:24:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:23 smithi119 ceph-mon[32752]: pgmap v11808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:24:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:24:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:23 smithi119 ceph-mon[32752]: pgmap v11809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:24:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:23 smithi119 ceph-mon[32752]: pgmap v11810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:24:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:24:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:23 smithi167 ceph-mon[32751]: pgmap v11808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:24:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:24:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:23 smithi167 ceph-mon[32751]: pgmap v11809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:24:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:23 smithi167 ceph-mon[32751]: pgmap v11810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:24:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:24:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:23 smithi090 ceph-mon[29180]: pgmap v11808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:24:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:24:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:23 smithi090 ceph-mon[29180]: pgmap v11809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:24:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:23 smithi090 ceph-mon[29180]: pgmap v11810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:24:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:24:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20947.303406 secs 2024-04-27T00:24:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:26 smithi119 ceph-mon[32752]: pgmap v11811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20947.303406 secs 2024-04-27T00:24:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:26 smithi167 ceph-mon[32751]: pgmap v11811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20947.303406 secs 2024-04-27T00:24:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:26 smithi090 ceph-mon[29180]: pgmap v11811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20948.094775 secs 2024-04-27T00:24:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20948.094775 secs 2024-04-27T00:24:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20948.094775 secs 2024-04-27T00:24:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:28 smithi119 ceph-mon[32752]: pgmap v11812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:28 smithi167 ceph-mon[32751]: pgmap v11812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:28 smithi090 ceph-mon[29180]: pgmap v11812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:30 smithi119 ceph-mon[32752]: pgmap v11813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:30 smithi167 ceph-mon[32751]: pgmap v11813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:30 smithi090 ceph-mon[29180]: pgmap v11813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20952.303485 secs 2024-04-27T00:24:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20952.303485 secs 2024-04-27T00:24:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20952.303485 secs 2024-04-27T00:24:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20953.094881 secs 2024-04-27T00:24:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:32 smithi090 ceph-mon[29180]: pgmap v11814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20953.094881 secs 2024-04-27T00:24:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:32 smithi119 ceph-mon[32752]: pgmap v11814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20953.094881 secs 2024-04-27T00:24:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:32 smithi167 ceph-mon[32751]: pgmap v11814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:24:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:34 smithi119 ceph-mon[32752]: pgmap v11815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:24:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:34 smithi167 ceph-mon[32751]: pgmap v11815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:24:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:34 smithi090 ceph-mon[29180]: pgmap v11815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:24:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20957.303574 secs 2024-04-27T00:24:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:36 smithi119 ceph-mon[32752]: pgmap v11816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20957.303574 secs 2024-04-27T00:24:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:36 smithi167 ceph-mon[32751]: pgmap v11816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20957.303574 secs 2024-04-27T00:24:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:36 smithi090 ceph-mon[29180]: pgmap v11816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20958.094998 secs 2024-04-27T00:24:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20958.094998 secs 2024-04-27T00:24:37.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20958.094998 secs 2024-04-27T00:24:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:38 smithi119 ceph-mon[32752]: pgmap v11817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:38 smithi167 ceph-mon[32751]: pgmap v11817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:38 smithi090 ceph-mon[29180]: pgmap v11817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:40 smithi119 ceph-mon[32752]: pgmap v11818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:40 smithi167 ceph-mon[32751]: pgmap v11818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:40 smithi090 ceph-mon[29180]: pgmap v11818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20962.303634 secs 2024-04-27T00:24:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20962.303634 secs 2024-04-27T00:24:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20962.303634 secs 2024-04-27T00:24:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:42 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 20963.095118 secs 2024-04-27T00:24:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:42 smithi119 ceph-mon[32752]: slow request 1444.667528 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:24:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:42 smithi119 ceph-mon[32752]: pgmap v11819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:42 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 20963.095118 secs 2024-04-27T00:24:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:42 smithi167 ceph-mon[32751]: slow request 1444.667528 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:24:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:42 smithi167 ceph-mon[32751]: pgmap v11819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:42 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 20963.095118 secs 2024-04-27T00:24:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:42 smithi090 ceph-mon[29180]: slow request 1444.667528 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:24:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:42 smithi090 ceph-mon[29180]: pgmap v11819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:43 smithi119 ceph-mon[32752]: pgmap v11820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:43 smithi167 ceph-mon[32751]: pgmap v11820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:43 smithi090 ceph-mon[29180]: pgmap v11820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20967.303718 secs 2024-04-27T00:24:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:46 smithi119 ceph-mon[32752]: pgmap v11821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20967.303718 secs 2024-04-27T00:24:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:46 smithi167 ceph-mon[32751]: pgmap v11821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20967.303718 secs 2024-04-27T00:24:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:46 smithi090 ceph-mon[29180]: pgmap v11821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20968.095256 secs 2024-04-27T00:24:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20968.095256 secs 2024-04-27T00:24:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20968.095256 secs 2024-04-27T00:24:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:48 smithi119 ceph-mon[32752]: pgmap v11822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:48 smithi167 ceph-mon[32751]: pgmap v11822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:48 smithi090 ceph-mon[29180]: pgmap v11822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:50 smithi119 ceph-mon[32752]: pgmap v11823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:50 smithi167 ceph-mon[32751]: pgmap v11823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:50 smithi090 ceph-mon[29180]: pgmap v11823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20972.303821 secs 2024-04-27T00:24:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20972.303821 secs 2024-04-27T00:24:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20972.303821 secs 2024-04-27T00:24:52.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20973.095342 secs 2024-04-27T00:24:52.453 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:52 smithi090 ceph-mon[29180]: pgmap v11824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20973.095342 secs 2024-04-27T00:24:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:52 smithi119 ceph-mon[32752]: pgmap v11824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20973.095342 secs 2024-04-27T00:24:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:52 smithi167 ceph-mon[32751]: pgmap v11824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:54 smithi119 ceph-mon[32752]: pgmap v11825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:54 smithi167 ceph-mon[32751]: pgmap v11825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:54 smithi090 ceph-mon[29180]: pgmap v11825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20977.303910 secs 2024-04-27T00:24:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:56 smithi119 ceph-mon[32752]: pgmap v11826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20977.303910 secs 2024-04-27T00:24:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:56 smithi167 ceph-mon[32751]: pgmap v11826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20977.303910 secs 2024-04-27T00:24:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:56 smithi090 ceph-mon[29180]: pgmap v11826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:24:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20978.095472 secs 2024-04-27T00:24:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20978.095472 secs 2024-04-27T00:24:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20978.095472 secs 2024-04-27T00:24:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:24:58 smithi119 ceph-mon[32752]: pgmap v11827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:24:58 smithi167 ceph-mon[32751]: pgmap v11827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:24:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:24:58 smithi090 ceph-mon[29180]: pgmap v11827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:00 smithi119 ceph-mon[32752]: pgmap v11828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:00 smithi167 ceph-mon[32751]: pgmap v11828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:00 smithi090 ceph-mon[29180]: pgmap v11828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20982.303969 secs 2024-04-27T00:25:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20982.303969 secs 2024-04-27T00:25:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20982.303969 secs 2024-04-27T00:25:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20983.095610 secs 2024-04-27T00:25:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:02 smithi090 ceph-mon[29180]: pgmap v11829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20983.095610 secs 2024-04-27T00:25:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:02 smithi119 ceph-mon[32752]: pgmap v11829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20983.095610 secs 2024-04-27T00:25:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:02 smithi167 ceph-mon[32751]: pgmap v11829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:04 smithi119 ceph-mon[32752]: pgmap v11830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:04 smithi167 ceph-mon[32751]: pgmap v11830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:04 smithi090 ceph-mon[29180]: pgmap v11830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20987.304053 secs 2024-04-27T00:25:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:06 smithi119 ceph-mon[32752]: pgmap v11831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20987.304053 secs 2024-04-27T00:25:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:06 smithi167 ceph-mon[32751]: pgmap v11831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20987.304053 secs 2024-04-27T00:25:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:06 smithi090 ceph-mon[29180]: pgmap v11831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20988.095734 secs 2024-04-27T00:25:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20988.095734 secs 2024-04-27T00:25:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20988.095734 secs 2024-04-27T00:25:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:08 smithi119 ceph-mon[32752]: pgmap v11832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:08.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:08 smithi167 ceph-mon[32751]: pgmap v11832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:08 smithi090 ceph-mon[29180]: pgmap v11832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:10 smithi119 ceph-mon[32752]: pgmap v11833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:10 smithi167 ceph-mon[32751]: pgmap v11833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:10 smithi090 ceph-mon[29180]: pgmap v11833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20992.304136 secs 2024-04-27T00:25:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20992.304136 secs 2024-04-27T00:25:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20992.304136 secs 2024-04-27T00:25:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20993.095849 secs 2024-04-27T00:25:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:12 smithi119 ceph-mon[32752]: pgmap v11834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20993.095849 secs 2024-04-27T00:25:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:12 smithi167 ceph-mon[32751]: pgmap v11834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20993.095849 secs 2024-04-27T00:25:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:12 smithi090 ceph-mon[29180]: pgmap v11834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:14 smithi119 ceph-mon[32752]: pgmap v11835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:14 smithi167 ceph-mon[32751]: pgmap v11835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:14 smithi090 ceph-mon[29180]: pgmap v11835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 20997.304221 secs 2024-04-27T00:25:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:16 smithi119 ceph-mon[32752]: pgmap v11836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 20997.304221 secs 2024-04-27T00:25:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:16 smithi167 ceph-mon[32751]: pgmap v11836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 20997.304221 secs 2024-04-27T00:25:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:16 smithi090 ceph-mon[29180]: pgmap v11836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 20998.095951 secs 2024-04-27T00:25:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 20998.095951 secs 2024-04-27T00:25:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 20998.095951 secs 2024-04-27T00:25:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:18 smithi119 ceph-mon[32752]: pgmap v11837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:18 smithi167 ceph-mon[32751]: pgmap v11837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:18 smithi090 ceph-mon[29180]: pgmap v11837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:20 smithi119 ceph-mon[32752]: pgmap v11838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21002.304282 secs 2024-04-27T00:25:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:20 smithi167 ceph-mon[32751]: pgmap v11838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21002.304282 secs 2024-04-27T00:25:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:20 smithi090 ceph-mon[29180]: pgmap v11838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21002.304282 secs 2024-04-27T00:25:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21003.096020 secs 2024-04-27T00:25:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:22 smithi119 ceph-mon[32752]: pgmap v11839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21003.096020 secs 2024-04-27T00:25:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:22 smithi167 ceph-mon[32751]: pgmap v11839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21003.096020 secs 2024-04-27T00:25:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:22 smithi090 ceph-mon[29180]: pgmap v11839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:23 smithi119 ceph-mon[32752]: pgmap v11840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:25:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:23 smithi167 ceph-mon[32751]: pgmap v11840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:25:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:23 smithi090 ceph-mon[29180]: pgmap v11840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:25:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:25:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:25:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:24 smithi119 ceph-mon[32752]: pgmap v11841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:25:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:24 smithi119 ceph-mon[32752]: pgmap v11842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:25:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:25:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:25:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:24 smithi167 ceph-mon[32751]: pgmap v11841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:25:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:24 smithi167 ceph-mon[32751]: pgmap v11842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:25:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:25:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:25:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:24 smithi090 ceph-mon[29180]: pgmap v11841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:25:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:24 smithi090 ceph-mon[29180]: pgmap v11842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:25:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21007.304366 secs 2024-04-27T00:25:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21007.304366 secs 2024-04-27T00:25:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21007.304366 secs 2024-04-27T00:25:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:26 smithi119 ceph-mon[32752]: pgmap v11843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:26 smithi167 ceph-mon[32751]: pgmap v11843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:26 smithi090 ceph-mon[29180]: pgmap v11843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21008.096118 secs 2024-04-27T00:25:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21008.096118 secs 2024-04-27T00:25:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21008.096118 secs 2024-04-27T00:25:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:28 smithi119 ceph-mon[32752]: pgmap v11844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:25:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:28 smithi167 ceph-mon[32751]: pgmap v11844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:25:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:28 smithi090 ceph-mon[29180]: pgmap v11844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:25:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:30 smithi119 ceph-mon[32752]: pgmap v11845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21012.304450 secs 2024-04-27T00:25:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:30 smithi167 ceph-mon[32751]: pgmap v11845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21012.304450 secs 2024-04-27T00:25:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:30 smithi090 ceph-mon[29180]: pgmap v11845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21012.304450 secs 2024-04-27T00:25:32.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21013.096236 secs 2024-04-27T00:25:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21013.096236 secs 2024-04-27T00:25:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21013.096236 secs 2024-04-27T00:25:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:32 smithi119 ceph-mon[32752]: pgmap v11846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:32 smithi167 ceph-mon[32751]: pgmap v11846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:32 smithi090 ceph-mon[29180]: pgmap v11846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:25:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:34 smithi119 ceph-mon[32752]: pgmap v11847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:25:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:34 smithi167 ceph-mon[32751]: pgmap v11847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:25:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:34 smithi090 ceph-mon[29180]: pgmap v11847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:25:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21017.304536 secs 2024-04-27T00:25:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21017.304536 secs 2024-04-27T00:25:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21017.304536 secs 2024-04-27T00:25:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21018.096372 secs 2024-04-27T00:25:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:36 smithi119 ceph-mon[32752]: pgmap v11848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21018.096372 secs 2024-04-27T00:25:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:36 smithi167 ceph-mon[32751]: pgmap v11848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21018.096372 secs 2024-04-27T00:25:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:36 smithi090 ceph-mon[29180]: pgmap v11848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:38 smithi119 ceph-mon[32752]: pgmap v11849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:38 smithi167 ceph-mon[32751]: pgmap v11849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:38 smithi090 ceph-mon[29180]: pgmap v11849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:40 smithi119 ceph-mon[32752]: pgmap v11850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21022.304622 secs 2024-04-27T00:25:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:40 smithi167 ceph-mon[32751]: pgmap v11850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21022.304622 secs 2024-04-27T00:25:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:40 smithi090 ceph-mon[29180]: pgmap v11850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21022.304622 secs 2024-04-27T00:25:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21023.096489 secs 2024-04-27T00:25:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21023.096489 secs 2024-04-27T00:25:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21023.096489 secs 2024-04-27T00:25:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:42 smithi119 ceph-mon[32752]: pgmap v11851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:42 smithi167 ceph-mon[32751]: pgmap v11851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:42 smithi090 ceph-mon[29180]: pgmap v11851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:44 smithi119 ceph-mon[32752]: pgmap v11852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:44 smithi167 ceph-mon[32751]: pgmap v11852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:44 smithi090 ceph-mon[29180]: pgmap v11852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21027.304707 secs 2024-04-27T00:25:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21027.304707 secs 2024-04-27T00:25:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21027.304707 secs 2024-04-27T00:25:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21028.096607 secs 2024-04-27T00:25:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:46 smithi119 ceph-mon[32752]: pgmap v11853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21028.096607 secs 2024-04-27T00:25:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:46 smithi167 ceph-mon[32751]: pgmap v11853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21028.096607 secs 2024-04-27T00:25:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:46 smithi090 ceph-mon[29180]: pgmap v11853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:48 smithi119 ceph-mon[32752]: pgmap v11854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:48 smithi167 ceph-mon[32751]: pgmap v11854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:48 smithi090 ceph-mon[29180]: pgmap v11854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:50 smithi119 ceph-mon[32752]: pgmap v11855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21032.304800 secs 2024-04-27T00:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:50 smithi167 ceph-mon[32751]: pgmap v11855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21032.304800 secs 2024-04-27T00:25:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:50 smithi090 ceph-mon[29180]: pgmap v11855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21032.304800 secs 2024-04-27T00:25:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21033.096722 secs 2024-04-27T00:25:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21033.096722 secs 2024-04-27T00:25:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21033.096722 secs 2024-04-27T00:25:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:52 smithi119 ceph-mon[32752]: pgmap v11856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:52 smithi167 ceph-mon[32751]: pgmap v11856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:52 smithi090 ceph-mon[29180]: pgmap v11856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:54 smithi119 ceph-mon[32752]: pgmap v11857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:54 smithi167 ceph-mon[32751]: pgmap v11857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:54 smithi090 ceph-mon[29180]: pgmap v11857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:25:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21037.304887 secs 2024-04-27T00:25:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21037.304887 secs 2024-04-27T00:25:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21037.304887 secs 2024-04-27T00:25:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21038.096788 secs 2024-04-27T00:25:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:56 smithi119 ceph-mon[32752]: pgmap v11858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21038.096788 secs 2024-04-27T00:25:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:56 smithi167 ceph-mon[32751]: pgmap v11858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21038.096788 secs 2024-04-27T00:25:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:56 smithi090 ceph-mon[29180]: pgmap v11858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:25:58 smithi119 ceph-mon[32752]: pgmap v11859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:25:58 smithi167 ceph-mon[32751]: pgmap v11859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:25:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:25:58 smithi090 ceph-mon[29180]: pgmap v11859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:00 smithi119 ceph-mon[32752]: pgmap v11860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21042.304949 secs 2024-04-27T00:26:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:00 smithi167 ceph-mon[32751]: pgmap v11860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21042.304949 secs 2024-04-27T00:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:00 smithi090 ceph-mon[29180]: pgmap v11860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21042.304949 secs 2024-04-27T00:26:02.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21043.096925 secs 2024-04-27T00:26:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21043.096925 secs 2024-04-27T00:26:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21043.096925 secs 2024-04-27T00:26:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:03 smithi119 ceph-mon[32752]: pgmap v11861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:03 smithi167 ceph-mon[32751]: pgmap v11861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:03 smithi090 ceph-mon[29180]: pgmap v11861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:05 smithi119 ceph-mon[32752]: pgmap v11862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:05 smithi167 ceph-mon[32751]: pgmap v11862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:05 smithi090 ceph-mon[29180]: pgmap v11862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21047.305037 secs 2024-04-27T00:26:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21047.305037 secs 2024-04-27T00:26:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21047.305037 secs 2024-04-27T00:26:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21048.097067 secs 2024-04-27T00:26:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:07 smithi119 ceph-mon[32752]: pgmap v11863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21048.097067 secs 2024-04-27T00:26:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:07 smithi167 ceph-mon[32751]: pgmap v11863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21048.097067 secs 2024-04-27T00:26:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:07 smithi090 ceph-mon[29180]: pgmap v11863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:09 smithi119 ceph-mon[32752]: pgmap v11864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:09 smithi167 ceph-mon[32751]: pgmap v11864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:09 smithi090 ceph-mon[29180]: pgmap v11864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:11 smithi119 ceph-mon[32752]: pgmap v11865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21052.305142 secs 2024-04-27T00:26:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:11 smithi167 ceph-mon[32751]: pgmap v11865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21052.305142 secs 2024-04-27T00:26:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:11 smithi090 ceph-mon[29180]: pgmap v11865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21052.305142 secs 2024-04-27T00:26:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21053.097203 secs 2024-04-27T00:26:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21053.097203 secs 2024-04-27T00:26:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21053.097203 secs 2024-04-27T00:26:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:13 smithi119 ceph-mon[32752]: pgmap v11866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:13 smithi167 ceph-mon[32751]: pgmap v11866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:13 smithi090 ceph-mon[29180]: pgmap v11866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:15 smithi119 ceph-mon[32752]: pgmap v11867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:15 smithi167 ceph-mon[32751]: pgmap v11867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:15 smithi090 ceph-mon[29180]: pgmap v11867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21057.305231 secs 2024-04-27T00:26:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21057.305231 secs 2024-04-27T00:26:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21057.305231 secs 2024-04-27T00:26:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21058.097317 secs 2024-04-27T00:26:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:17 smithi119 ceph-mon[32752]: pgmap v11868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21058.097317 secs 2024-04-27T00:26:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:17 smithi167 ceph-mon[32751]: pgmap v11868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21058.097317 secs 2024-04-27T00:26:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:17 smithi090 ceph-mon[29180]: pgmap v11868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:19 smithi119 ceph-mon[32752]: pgmap v11869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:19 smithi167 ceph-mon[32751]: pgmap v11869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:19 smithi090 ceph-mon[29180]: pgmap v11869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:21 smithi119 ceph-mon[32752]: pgmap v11870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21062.305312 secs 2024-04-27T00:26:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:21 smithi167 ceph-mon[32751]: pgmap v11870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21062.305312 secs 2024-04-27T00:26:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:21 smithi090 ceph-mon[29180]: pgmap v11870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21062.305312 secs 2024-04-27T00:26:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21063.097461 secs 2024-04-27T00:26:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21063.097461 secs 2024-04-27T00:26:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21063.097461 secs 2024-04-27T00:26:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:23 smithi119 ceph-mon[32752]: pgmap v11871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:23 smithi167 ceph-mon[32751]: pgmap v11871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:23 smithi090 ceph-mon[29180]: pgmap v11871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:26:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:26:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:26:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:25 smithi167 ceph-mon[32751]: pgmap v11872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:26:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:26:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:26:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:25 smithi090 ceph-mon[29180]: pgmap v11872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:26:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:26:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:26:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:25 smithi119 ceph-mon[32752]: pgmap v11872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:26:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:26:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:26:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:26 smithi167 ceph-mon[32751]: pgmap v11873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21067.305373 secs 2024-04-27T00:26:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:26 smithi090 ceph-mon[29180]: pgmap v11873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21067.305373 secs 2024-04-27T00:26:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:26 smithi119 ceph-mon[32752]: pgmap v11873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21067.305373 secs 2024-04-27T00:26:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21068.097538 secs 2024-04-27T00:26:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21068.097538 secs 2024-04-27T00:26:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21068.097538 secs 2024-04-27T00:26:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:28 smithi090 ceph-mon[29180]: pgmap v11874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:28 smithi119 ceph-mon[32752]: pgmap v11874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:28 smithi167 ceph-mon[32751]: pgmap v11874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:30 smithi090 ceph-mon[29180]: pgmap v11875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:26:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:30 smithi119 ceph-mon[32752]: pgmap v11875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:26:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:30 smithi167 ceph-mon[32751]: pgmap v11875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:26:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21072.305450 secs 2024-04-27T00:26:31.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21072.305450 secs 2024-04-27T00:26:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21072.305450 secs 2024-04-27T00:26:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:32 smithi090 ceph-mon[29180]: pgmap v11876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21073.097654 secs 2024-04-27T00:26:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:32 smithi119 ceph-mon[32752]: pgmap v11876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21073.097654 secs 2024-04-27T00:26:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:32 smithi167 ceph-mon[32751]: pgmap v11876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21073.097654 secs 2024-04-27T00:26:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:34 smithi090 ceph-mon[29180]: pgmap v11877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:34 smithi119 ceph-mon[32752]: pgmap v11877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:34 smithi167 ceph-mon[32751]: pgmap v11877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:36 smithi090 ceph-mon[29180]: pgmap v11878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21077.305508 secs 2024-04-27T00:26:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:36 smithi119 ceph-mon[32752]: pgmap v11878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21077.305508 secs 2024-04-27T00:26:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:36 smithi167 ceph-mon[32751]: pgmap v11878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:26:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21077.305508 secs 2024-04-27T00:26:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21078.097788 secs 2024-04-27T00:26:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21078.097788 secs 2024-04-27T00:26:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21078.097788 secs 2024-04-27T00:26:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:38 smithi090 ceph-mon[29180]: pgmap v11879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:38 smithi119 ceph-mon[32752]: pgmap v11879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:38 smithi167 ceph-mon[32751]: pgmap v11879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:40 smithi090 ceph-mon[29180]: pgmap v11880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:40 smithi119 ceph-mon[32752]: pgmap v11880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:40 smithi167 ceph-mon[32751]: pgmap v11880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21082.305591 secs 2024-04-27T00:26:42.121 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21082.305591 secs 2024-04-27T00:26:42.122 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21082.305591 secs 2024-04-27T00:26:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:42 smithi090 ceph-mon[29180]: pgmap v11881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21083.097905 secs 2024-04-27T00:26:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:42 smithi119 ceph-mon[32752]: pgmap v11881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21083.097905 secs 2024-04-27T00:26:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:42 smithi167 ceph-mon[32751]: pgmap v11881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21083.097905 secs 2024-04-27T00:26:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:44 smithi090 ceph-mon[29180]: pgmap v11882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:44 smithi119 ceph-mon[32752]: pgmap v11882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:44 smithi167 ceph-mon[32751]: pgmap v11882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:46 smithi090 ceph-mon[29180]: pgmap v11883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21087.305670 secs 2024-04-27T00:26:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:46 smithi119 ceph-mon[32752]: pgmap v11883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21087.305670 secs 2024-04-27T00:26:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:46 smithi167 ceph-mon[32751]: pgmap v11883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21087.305670 secs 2024-04-27T00:26:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21088.097982 secs 2024-04-27T00:26:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21088.097982 secs 2024-04-27T00:26:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21088.097982 secs 2024-04-27T00:26:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:48 smithi090 ceph-mon[29180]: pgmap v11884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:48 smithi119 ceph-mon[32752]: pgmap v11884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:48 smithi167 ceph-mon[32751]: pgmap v11884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T00:26:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:50 smithi090 ceph-mon[29180]: pgmap v11885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:50 smithi119 ceph-mon[32752]: pgmap v11885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:50 smithi167 ceph-mon[32751]: pgmap v11885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:26:51.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21092.305750 secs 2024-04-27T00:26:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21092.305750 secs 2024-04-27T00:26:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21092.305750 secs 2024-04-27T00:26:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:52 smithi119 ceph-mon[32752]: pgmap v11886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21093.098103 secs 2024-04-27T00:26:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:52 smithi167 ceph-mon[32751]: pgmap v11886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21093.098103 secs 2024-04-27T00:26:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:52 smithi090 ceph-mon[29180]: pgmap v11886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21093.098103 secs 2024-04-27T00:26:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:54 smithi119 ceph-mon[32752]: pgmap v11887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:54 smithi167 ceph-mon[32751]: pgmap v11887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:54 smithi090 ceph-mon[29180]: pgmap v11887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:56 smithi119 ceph-mon[32752]: pgmap v11888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21097.305829 secs 2024-04-27T00:26:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:56 smithi167 ceph-mon[32751]: pgmap v11888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21097.305829 secs 2024-04-27T00:26:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:56 smithi090 ceph-mon[29180]: pgmap v11888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:26:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21097.305829 secs 2024-04-27T00:26:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21098.098225 secs 2024-04-27T00:26:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21098.098225 secs 2024-04-27T00:26:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21098.098225 secs 2024-04-27T00:26:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:26:58 smithi119 ceph-mon[32752]: pgmap v11889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:26:58 smithi167 ceph-mon[32751]: pgmap v11889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:26:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:26:58 smithi090 ceph-mon[29180]: pgmap v11889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:00 smithi119 ceph-mon[32752]: pgmap v11890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:00 smithi167 ceph-mon[32751]: pgmap v11890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:00 smithi090 ceph-mon[29180]: pgmap v11890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:01.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21102.305915 secs 2024-04-27T00:27:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21102.305915 secs 2024-04-27T00:27:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21102.305915 secs 2024-04-27T00:27:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:02 smithi090 ceph-mon[29180]: pgmap v11891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21103.098345 secs 2024-04-27T00:27:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:02 smithi119 ceph-mon[32752]: pgmap v11891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21103.098345 secs 2024-04-27T00:27:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:02 smithi167 ceph-mon[32751]: pgmap v11891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21103.098345 secs 2024-04-27T00:27:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:04 smithi119 ceph-mon[32752]: pgmap v11892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:04 smithi167 ceph-mon[32751]: pgmap v11892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:04 smithi090 ceph-mon[29180]: pgmap v11892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:06 smithi119 ceph-mon[32752]: pgmap v11893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21107.305987 secs 2024-04-27T00:27:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:06 smithi167 ceph-mon[32751]: pgmap v11893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21107.305987 secs 2024-04-27T00:27:06.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:06 smithi090 ceph-mon[29180]: pgmap v11893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:06.588 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21107.305987 secs 2024-04-27T00:27:06.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:27:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:27:06.588+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:27:06.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:27:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:27:06.588+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:27:06.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:27:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:27:06.588+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:27:06.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:27:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:27:06.588+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:27:06.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:27:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:27:06.588+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:27:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21108.098410 secs 2024-04-27T00:27:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21108.098410 secs 2024-04-27T00:27:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21108.098410 secs 2024-04-27T00:27:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:08 smithi119 ceph-mon[32752]: pgmap v11894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:08 smithi167 ceph-mon[32751]: pgmap v11894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:08 smithi090 ceph-mon[29180]: pgmap v11894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:10 smithi119 ceph-mon[32752]: pgmap v11895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:10 smithi167 ceph-mon[32751]: pgmap v11895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:10 smithi090 ceph-mon[29180]: pgmap v11895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:11 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 21112.306076 secs 2024-04-27T00:27:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:11 smithi119 ceph-mon[32752]: slow request 722.511112 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:27:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:11 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 21112.306076 secs 2024-04-27T00:27:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:11 smithi167 ceph-mon[32751]: slow request 722.511112 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:27:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:11 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 21112.306076 secs 2024-04-27T00:27:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:11 smithi090 ceph-mon[29180]: slow request 722.511112 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:27:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:12 smithi119 ceph-mon[32752]: pgmap v11896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21113.098488 secs 2024-04-27T00:27:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:12 smithi167 ceph-mon[32751]: pgmap v11896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21113.098488 secs 2024-04-27T00:27:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:12 smithi090 ceph-mon[29180]: pgmap v11896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21113.098488 secs 2024-04-27T00:27:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:14 smithi119 ceph-mon[32752]: pgmap v11897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:14 smithi167 ceph-mon[32751]: pgmap v11897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:14 smithi090 ceph-mon[29180]: pgmap v11897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:16 smithi119 ceph-mon[32752]: pgmap v11898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21117.306160 secs 2024-04-27T00:27:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:16 smithi167 ceph-mon[32751]: pgmap v11898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21117.306160 secs 2024-04-27T00:27:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:16 smithi090 ceph-mon[29180]: pgmap v11898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21117.306160 secs 2024-04-27T00:27:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21118.098614 secs 2024-04-27T00:27:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21118.098614 secs 2024-04-27T00:27:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21118.098614 secs 2024-04-27T00:27:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:18 smithi119 ceph-mon[32752]: pgmap v11899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:18 smithi167 ceph-mon[32751]: pgmap v11899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:18 smithi090 ceph-mon[29180]: pgmap v11899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:20 smithi119 ceph-mon[32752]: pgmap v11900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:20 smithi167 ceph-mon[32751]: pgmap v11900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:20 smithi090 ceph-mon[29180]: pgmap v11900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:21.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21122.306242 secs 2024-04-27T00:27:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21122.306242 secs 2024-04-27T00:27:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21122.306242 secs 2024-04-27T00:27:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:22 smithi119 ceph-mon[32752]: pgmap v11901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21123.098712 secs 2024-04-27T00:27:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:22 smithi167 ceph-mon[32751]: pgmap v11901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21123.098712 secs 2024-04-27T00:27:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:22 smithi090 ceph-mon[29180]: pgmap v11901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21123.098712 secs 2024-04-27T00:27:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:24 smithi119 ceph-mon[32752]: pgmap v11902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:24 smithi167 ceph-mon[32751]: pgmap v11902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:24 smithi090 ceph-mon[29180]: pgmap v11902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:27:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:27:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:27:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:26 smithi119 ceph-mon[32752]: pgmap v11903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21127.306331 secs 2024-04-27T00:27:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:27:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:27:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:27:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:26 smithi167 ceph-mon[32751]: pgmap v11903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21127.306331 secs 2024-04-27T00:27:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:27:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:27:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:27:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:26 smithi090 ceph-mon[29180]: pgmap v11903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21127.306331 secs 2024-04-27T00:27:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:27:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:27:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:27:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:27 smithi119 ceph-mon[32752]: pgmap v11904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:27:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:27 smithi119 ceph-mon[32752]: pgmap v11905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21128.098782 secs 2024-04-27T00:27:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:27 smithi167 ceph-mon[32751]: pgmap v11904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:27:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:27 smithi167 ceph-mon[32751]: pgmap v11905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21128.098782 secs 2024-04-27T00:27:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:27 smithi090 ceph-mon[29180]: pgmap v11904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:27:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:27 smithi090 ceph-mon[29180]: pgmap v11905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21128.098782 secs 2024-04-27T00:27:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:29 smithi119 ceph-mon[32752]: pgmap v11906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:27:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:29 smithi167 ceph-mon[32751]: pgmap v11906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:27:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:29 smithi090 ceph-mon[29180]: pgmap v11906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:27:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:31 smithi119 ceph-mon[32752]: pgmap v11907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21132.306415 secs 2024-04-27T00:27:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:31 smithi167 ceph-mon[32751]: pgmap v11907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21132.306415 secs 2024-04-27T00:27:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:31 smithi090 ceph-mon[29180]: pgmap v11907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21132.306415 secs 2024-04-27T00:27:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21133.098903 secs 2024-04-27T00:27:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21133.098903 secs 2024-04-27T00:27:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21133.098903 secs 2024-04-27T00:27:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:33 smithi119 ceph-mon[32752]: pgmap v11908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:33 smithi167 ceph-mon[32751]: pgmap v11908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:33 smithi090 ceph-mon[29180]: pgmap v11908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:35 smithi119 ceph-mon[32752]: pgmap v11909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21137.306498 secs 2024-04-27T00:27:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:35 smithi167 ceph-mon[32751]: pgmap v11909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21137.306498 secs 2024-04-27T00:27:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:35 smithi090 ceph-mon[29180]: pgmap v11909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:27:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21137.306498 secs 2024-04-27T00:27:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:37 smithi119 ceph-mon[32752]: pgmap v11910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:27:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21138.099015 secs 2024-04-27T00:27:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:37 smithi167 ceph-mon[32751]: pgmap v11910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:27:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21138.099015 secs 2024-04-27T00:27:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:37 smithi090 ceph-mon[29180]: pgmap v11910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:27:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21138.099015 secs 2024-04-27T00:27:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:39 smithi119 ceph-mon[32752]: pgmap v11911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:39 smithi167 ceph-mon[32751]: pgmap v11911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:39 smithi090 ceph-mon[29180]: pgmap v11911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21142.306583 secs 2024-04-27T00:27:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21142.306583 secs 2024-04-27T00:27:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21142.306583 secs 2024-04-27T00:27:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:41 smithi119 ceph-mon[32752]: pgmap v11912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:41 smithi167 ceph-mon[32751]: pgmap v11912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:41 smithi090 ceph-mon[29180]: pgmap v11912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21143.099132 secs 2024-04-27T00:27:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21143.099132 secs 2024-04-27T00:27:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21143.099132 secs 2024-04-27T00:27:43.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:43 smithi090 ceph-mon[29180]: pgmap v11913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:43 smithi119 ceph-mon[32752]: pgmap v11913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:43 smithi167 ceph-mon[32751]: pgmap v11913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:45 smithi119 ceph-mon[32752]: pgmap v11914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21147.306643 secs 2024-04-27T00:27:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:45 smithi167 ceph-mon[32751]: pgmap v11914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21147.306643 secs 2024-04-27T00:27:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:45 smithi090 ceph-mon[29180]: pgmap v11914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21147.306643 secs 2024-04-27T00:27:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:47 smithi119 ceph-mon[32752]: pgmap v11915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21148.099280 secs 2024-04-27T00:27:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:47 smithi167 ceph-mon[32751]: pgmap v11915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21148.099280 secs 2024-04-27T00:27:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:47 smithi090 ceph-mon[29180]: pgmap v11915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21148.099280 secs 2024-04-27T00:27:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:49 smithi119 ceph-mon[32752]: pgmap v11916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:49 smithi167 ceph-mon[32751]: pgmap v11916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:49 smithi090 ceph-mon[29180]: pgmap v11916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21152.306729 secs 2024-04-27T00:27:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21152.306729 secs 2024-04-27T00:27:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21152.306729 secs 2024-04-27T00:27:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:51 smithi119 ceph-mon[32752]: pgmap v11917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:51 smithi167 ceph-mon[32751]: pgmap v11917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:51 smithi090 ceph-mon[29180]: pgmap v11917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21153.099402 secs 2024-04-27T00:27:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21153.099402 secs 2024-04-27T00:27:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21153.099402 secs 2024-04-27T00:27:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:53 smithi119 ceph-mon[32752]: pgmap v11918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:53 smithi167 ceph-mon[32751]: pgmap v11918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:53 smithi090 ceph-mon[29180]: pgmap v11918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:55 smithi119 ceph-mon[32752]: pgmap v11919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21157.306829 secs 2024-04-27T00:27:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:55 smithi167 ceph-mon[32751]: pgmap v11919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21157.306829 secs 2024-04-27T00:27:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:55 smithi090 ceph-mon[29180]: pgmap v11919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:27:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21157.306829 secs 2024-04-27T00:27:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:57 smithi119 ceph-mon[32752]: pgmap v11920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21158.099464 secs 2024-04-27T00:27:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:57 smithi167 ceph-mon[32751]: pgmap v11920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21158.099464 secs 2024-04-27T00:27:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:57 smithi090 ceph-mon[29180]: pgmap v11920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21158.099464 secs 2024-04-27T00:27:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:27:59 smithi119 ceph-mon[32752]: pgmap v11921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:27:59 smithi167 ceph-mon[32751]: pgmap v11921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:27:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:27:59 smithi090 ceph-mon[29180]: pgmap v11921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21162.306892 secs 2024-04-27T00:28:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21162.306892 secs 2024-04-27T00:28:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21162.306892 secs 2024-04-27T00:28:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:01 smithi119 ceph-mon[32752]: pgmap v11922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:01 smithi167 ceph-mon[32751]: pgmap v11922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:01 smithi090 ceph-mon[29180]: pgmap v11922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21163.099582 secs 2024-04-27T00:28:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21163.099582 secs 2024-04-27T00:28:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21163.099582 secs 2024-04-27T00:28:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:03 smithi119 ceph-mon[32752]: pgmap v11923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:03 smithi167 ceph-mon[32751]: pgmap v11923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:03 smithi090 ceph-mon[29180]: pgmap v11923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:05 smithi119 ceph-mon[32752]: pgmap v11924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:05 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 21167.306978 secs 2024-04-27T00:28:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:05 smithi119 ceph-mon[32752]: slow request 11521.715222 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:28:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:05 smithi167 ceph-mon[32751]: pgmap v11924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:05 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 21167.306978 secs 2024-04-27T00:28:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:05 smithi167 ceph-mon[32751]: slow request 11521.715222 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:28:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:05 smithi090 ceph-mon[29180]: pgmap v11924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:05 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 21167.306978 secs 2024-04-27T00:28:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:05 smithi090 ceph-mon[29180]: slow request 11521.715222 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:28:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:07 smithi167 ceph-mon[32751]: pgmap v11925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21168.099698 secs 2024-04-27T00:28:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:07 smithi090 ceph-mon[29180]: pgmap v11925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21168.099698 secs 2024-04-27T00:28:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:07 smithi119 ceph-mon[32752]: pgmap v11925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21168.099698 secs 2024-04-27T00:28:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:09 smithi167 ceph-mon[32751]: pgmap v11926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:09 smithi090 ceph-mon[29180]: pgmap v11926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:09 smithi119 ceph-mon[32752]: pgmap v11926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21172.307083 secs 2024-04-27T00:28:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21172.307083 secs 2024-04-27T00:28:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21172.307083 secs 2024-04-27T00:28:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:11 smithi167 ceph-mon[32751]: pgmap v11927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:11 smithi090 ceph-mon[29180]: pgmap v11927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:11 smithi119 ceph-mon[32752]: pgmap v11927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21173.099775 secs 2024-04-27T00:28:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21173.099775 secs 2024-04-27T00:28:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21173.099775 secs 2024-04-27T00:28:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:13 smithi090 ceph-mon[29180]: pgmap v11928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:13 smithi119 ceph-mon[32752]: pgmap v11928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:13 smithi167 ceph-mon[32751]: pgmap v11928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:15 smithi090 ceph-mon[29180]: pgmap v11929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21177.307166 secs 2024-04-27T00:28:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:15 smithi119 ceph-mon[32752]: pgmap v11929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21177.307166 secs 2024-04-27T00:28:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:15 smithi167 ceph-mon[32751]: pgmap v11929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21177.307166 secs 2024-04-27T00:28:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:17 smithi090 ceph-mon[29180]: pgmap v11930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21178.099923 secs 2024-04-27T00:28:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:17 smithi119 ceph-mon[32752]: pgmap v11930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21178.099923 secs 2024-04-27T00:28:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:17 smithi167 ceph-mon[32751]: pgmap v11930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21178.099923 secs 2024-04-27T00:28:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:19 smithi090 ceph-mon[29180]: pgmap v11931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:19 smithi119 ceph-mon[32752]: pgmap v11931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:19 smithi167 ceph-mon[32751]: pgmap v11931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21182.307228 secs 2024-04-27T00:28:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21182.307228 secs 2024-04-27T00:28:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21182.307228 secs 2024-04-27T00:28:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:21 smithi090 ceph-mon[29180]: pgmap v11932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:21.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:21 smithi167 ceph-mon[32751]: pgmap v11932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:21 smithi119 ceph-mon[32752]: pgmap v11932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21183.100019 secs 2024-04-27T00:28:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21183.100019 secs 2024-04-27T00:28:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21183.100019 secs 2024-04-27T00:28:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:23 smithi090 ceph-mon[29180]: pgmap v11933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:23 smithi119 ceph-mon[32752]: pgmap v11933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:23 smithi167 ceph-mon[32751]: pgmap v11933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:24 smithi090 ceph-mon[29180]: pgmap v11934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:24 smithi119 ceph-mon[32752]: pgmap v11934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:24 smithi167 ceph-mon[32751]: pgmap v11934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21187.307311 secs 2024-04-27T00:28:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21187.307311 secs 2024-04-27T00:28:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21187.307311 secs 2024-04-27T00:28:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:26 smithi090 ceph-mon[29180]: pgmap v11935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:28:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:28:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:28:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:26 smithi090 ceph-mon[29180]: pgmap v11936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:28:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:26 smithi090 ceph-mon[29180]: pgmap v11937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:28:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:26 smithi119 ceph-mon[32752]: pgmap v11935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:26 smithi119 ceph-mon[32752]: pgmap v11936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:26 smithi119 ceph-mon[32752]: pgmap v11937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:28:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:26 smithi167 ceph-mon[32751]: pgmap v11935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:28:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:28:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:28:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:26 smithi167 ceph-mon[32751]: pgmap v11936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:28:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:26 smithi167 ceph-mon[32751]: pgmap v11937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:28:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21188.100086 secs 2024-04-27T00:28:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21188.100086 secs 2024-04-27T00:28:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21188.100086 secs 2024-04-27T00:28:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:28 smithi090 ceph-mon[29180]: pgmap v11938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:28:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:28 smithi119 ceph-mon[32752]: pgmap v11938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:28:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:28 smithi167 ceph-mon[32751]: pgmap v11938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:28:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21192.307373 secs 2024-04-27T00:28:30.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21192.307373 secs 2024-04-27T00:28:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21192.307373 secs 2024-04-27T00:28:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:31 smithi119 ceph-mon[32752]: pgmap v11939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:31 smithi167 ceph-mon[32751]: pgmap v11939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:31 smithi090 ceph-mon[29180]: pgmap v11939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21193.100187 secs 2024-04-27T00:28:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21193.100187 secs 2024-04-27T00:28:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21193.100187 secs 2024-04-27T00:28:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:33 smithi119 ceph-mon[32752]: pgmap v11940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:28:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:33 smithi167 ceph-mon[32751]: pgmap v11940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:28:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:33 smithi090 ceph-mon[29180]: pgmap v11940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:28:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:35 smithi119 ceph-mon[32752]: pgmap v11941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21197.307455 secs 2024-04-27T00:28:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:35 smithi167 ceph-mon[32751]: pgmap v11941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21197.307455 secs 2024-04-27T00:28:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:35 smithi090 ceph-mon[29180]: pgmap v11941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:28:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21197.307455 secs 2024-04-27T00:28:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21198.100281 secs 2024-04-27T00:28:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:37 smithi119 ceph-mon[32752]: pgmap v11942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:28:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21198.100281 secs 2024-04-27T00:28:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:37 smithi167 ceph-mon[32751]: pgmap v11942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:28:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21198.100281 secs 2024-04-27T00:28:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:37 smithi090 ceph-mon[29180]: pgmap v11942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:28:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:39 smithi119 ceph-mon[32752]: pgmap v11943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:39 smithi167 ceph-mon[32751]: pgmap v11943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:39 smithi090 ceph-mon[29180]: pgmap v11943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21202.307558 secs 2024-04-27T00:28:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21202.307558 secs 2024-04-27T00:28:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21202.307558 secs 2024-04-27T00:28:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:41 smithi167 ceph-mon[32751]: pgmap v11944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:41 smithi090 ceph-mon[29180]: pgmap v11944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:41 smithi119 ceph-mon[32752]: pgmap v11944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21203.100432 secs 2024-04-27T00:28:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21203.100432 secs 2024-04-27T00:28:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21203.100432 secs 2024-04-27T00:28:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:43 smithi167 ceph-mon[32751]: pgmap v11945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:43 smithi090 ceph-mon[29180]: pgmap v11945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:43 smithi119 ceph-mon[32752]: pgmap v11945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:45 smithi090 ceph-mon[29180]: pgmap v11946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21207.307642 secs 2024-04-27T00:28:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:45 smithi119 ceph-mon[32752]: pgmap v11946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21207.307642 secs 2024-04-27T00:28:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:45 smithi167 ceph-mon[32751]: pgmap v11946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21207.307642 secs 2024-04-27T00:28:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21208.100526 secs 2024-04-27T00:28:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:47 smithi090 ceph-mon[29180]: pgmap v11947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21208.100526 secs 2024-04-27T00:28:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:47 smithi119 ceph-mon[32752]: pgmap v11947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21208.100526 secs 2024-04-27T00:28:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:47 smithi167 ceph-mon[32751]: pgmap v11947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:49 smithi090 ceph-mon[29180]: pgmap v11948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:49 smithi119 ceph-mon[32752]: pgmap v11948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:49 smithi167 ceph-mon[32751]: pgmap v11948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21212.307728 secs 2024-04-27T00:28:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21212.307728 secs 2024-04-27T00:28:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21212.307728 secs 2024-04-27T00:28:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:51 smithi090 ceph-mon[29180]: pgmap v11949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T00:28:51.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:51 smithi167 ceph-mon[32751]: pgmap v11949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T00:28:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:51 smithi119 ceph-mon[32752]: pgmap v11949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T00:28:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21213.100617 secs 2024-04-27T00:28:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21213.100617 secs 2024-04-27T00:28:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21213.100617 secs 2024-04-27T00:28:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:53 smithi090 ceph-mon[29180]: pgmap v11950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:53 smithi119 ceph-mon[32752]: pgmap v11950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:53 smithi167 ceph-mon[32751]: pgmap v11950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:55 smithi090 ceph-mon[29180]: pgmap v11951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21217.307813 secs 2024-04-27T00:28:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:55 smithi119 ceph-mon[32752]: pgmap v11951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21217.307813 secs 2024-04-27T00:28:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:55 smithi167 ceph-mon[32751]: pgmap v11951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21217.307813 secs 2024-04-27T00:28:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21218.100753 secs 2024-04-27T00:28:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:57 smithi090 ceph-mon[29180]: pgmap v11952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21218.100753 secs 2024-04-27T00:28:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:57 smithi119 ceph-mon[32752]: pgmap v11952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21218.100753 secs 2024-04-27T00:28:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:57 smithi167 ceph-mon[32751]: pgmap v11952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:28:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:28:59 smithi090 ceph-mon[29180]: pgmap v11953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:28:59 smithi119 ceph-mon[32752]: pgmap v11953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:28:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:28:59 smithi167 ceph-mon[32751]: pgmap v11953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21222.307897 secs 2024-04-27T00:29:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:00 smithi090 ceph-mon[29180]: pgmap v11954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T00:29:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21222.307897 secs 2024-04-27T00:29:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:00 smithi119 ceph-mon[32752]: pgmap v11954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T00:29:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21222.307897 secs 2024-04-27T00:29:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:00 smithi167 ceph-mon[32751]: pgmap v11954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T00:29:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21223.100867 secs 2024-04-27T00:29:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21223.100867 secs 2024-04-27T00:29:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21223.100867 secs 2024-04-27T00:29:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:02 smithi167 ceph-mon[32751]: pgmap v11955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:02 smithi090 ceph-mon[29180]: pgmap v11955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:02 smithi119 ceph-mon[32752]: pgmap v11955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:05 smithi119 ceph-mon[32752]: pgmap v11956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21227.307958 secs 2024-04-27T00:29:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:05 smithi167 ceph-mon[32751]: pgmap v11956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21227.307958 secs 2024-04-27T00:29:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:05 smithi090 ceph-mon[29180]: pgmap v11956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21227.307958 secs 2024-04-27T00:29:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21228.101012 secs 2024-04-27T00:29:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:07 smithi119 ceph-mon[32752]: pgmap v11957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21228.101012 secs 2024-04-27T00:29:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:07 smithi167 ceph-mon[32751]: pgmap v11957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21228.101012 secs 2024-04-27T00:29:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:07 smithi090 ceph-mon[29180]: pgmap v11957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:09 smithi119 ceph-mon[32752]: pgmap v11958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:09 smithi167 ceph-mon[32751]: pgmap v11958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:09 smithi090 ceph-mon[29180]: pgmap v11958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21232.308018 secs 2024-04-27T00:29:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21232.308018 secs 2024-04-27T00:29:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21232.308018 secs 2024-04-27T00:29:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:11 smithi119 ceph-mon[32752]: pgmap v11959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:11 smithi167 ceph-mon[32751]: pgmap v11959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:11 smithi090 ceph-mon[29180]: pgmap v11959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21233.101117 secs 2024-04-27T00:29:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21233.101117 secs 2024-04-27T00:29:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21233.101117 secs 2024-04-27T00:29:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:13 smithi119 ceph-mon[32752]: pgmap v11960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:13 smithi167 ceph-mon[32751]: pgmap v11960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:13 smithi090 ceph-mon[29180]: pgmap v11960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:15 smithi119 ceph-mon[32752]: pgmap v11961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21237.308095 secs 2024-04-27T00:29:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:15 smithi167 ceph-mon[32751]: pgmap v11961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21237.308095 secs 2024-04-27T00:29:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:15 smithi090 ceph-mon[29180]: pgmap v11961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21237.308095 secs 2024-04-27T00:29:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21238.101248 secs 2024-04-27T00:29:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:17 smithi119 ceph-mon[32752]: pgmap v11962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21238.101248 secs 2024-04-27T00:29:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:17 smithi167 ceph-mon[32751]: pgmap v11962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21238.101248 secs 2024-04-27T00:29:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:17 smithi090 ceph-mon[29180]: pgmap v11962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:19 smithi119 ceph-mon[32752]: pgmap v11963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:19 smithi167 ceph-mon[32751]: pgmap v11963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:19 smithi090 ceph-mon[29180]: pgmap v11963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21242.308167 secs 2024-04-27T00:29:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21242.308167 secs 2024-04-27T00:29:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21242.308167 secs 2024-04-27T00:29:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:21 smithi167 ceph-mon[32751]: pgmap v11964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:21 smithi090 ceph-mon[29180]: pgmap v11964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:21 smithi119 ceph-mon[32752]: pgmap v11964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21243.101402 secs 2024-04-27T00:29:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21243.101402 secs 2024-04-27T00:29:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21243.101402 secs 2024-04-27T00:29:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:23 smithi167 ceph-mon[32751]: pgmap v11965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:23 smithi090 ceph-mon[29180]: pgmap v11965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:23 smithi119 ceph-mon[32752]: pgmap v11965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:25 smithi090 ceph-mon[29180]: pgmap v11966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21247.308250 secs 2024-04-27T00:29:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:25 smithi119 ceph-mon[32752]: pgmap v11966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21247.308250 secs 2024-04-27T00:29:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:25 smithi167 ceph-mon[32751]: pgmap v11966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21247.308250 secs 2024-04-27T00:29:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:29:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:29:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:29:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21248.101494 secs 2024-04-27T00:29:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:27 smithi090 ceph-mon[29180]: pgmap v11967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:29:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:29:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:29:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21248.101494 secs 2024-04-27T00:29:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:27 smithi119 ceph-mon[32752]: pgmap v11967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:29:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:29:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:29:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21248.101494 secs 2024-04-27T00:29:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:27 smithi167 ceph-mon[32751]: pgmap v11967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:29:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:29:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:29:28.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:28 smithi090 ceph-mon[29180]: pgmap v11968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:29:28.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:28 smithi090 ceph-mon[29180]: pgmap v11969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:29:28.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:28 smithi119 ceph-mon[32752]: pgmap v11968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:29:28.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:28 smithi119 ceph-mon[32752]: pgmap v11969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:29:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:28 smithi167 ceph-mon[32751]: pgmap v11968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:29:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:28 smithi167 ceph-mon[32751]: pgmap v11969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:29:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:30 smithi090 ceph-mon[29180]: pgmap v11970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21252.308334 secs 2024-04-27T00:29:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:30 smithi119 ceph-mon[32752]: pgmap v11970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21252.308334 secs 2024-04-27T00:29:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:30 smithi167 ceph-mon[32751]: pgmap v11970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21252.308334 secs 2024-04-27T00:29:32.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21253.101553 secs 2024-04-27T00:29:32.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:32 smithi119 ceph-mon[32752]: pgmap v11971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21253.101553 secs 2024-04-27T00:29:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:32 smithi090 ceph-mon[29180]: pgmap v11971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21253.101553 secs 2024-04-27T00:29:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:32 smithi167 ceph-mon[32751]: pgmap v11971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:34 smithi090 ceph-mon[29180]: pgmap v11972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:29:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:34 smithi119 ceph-mon[32752]: pgmap v11972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:29:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:34 smithi167 ceph-mon[32751]: pgmap v11972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:29:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21257.308424 secs 2024-04-27T00:29:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21257.308424 secs 2024-04-27T00:29:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21257.308424 secs 2024-04-27T00:29:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:36 smithi090 ceph-mon[29180]: pgmap v11973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:36 smithi119 ceph-mon[32752]: pgmap v11973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:36 smithi167 ceph-mon[32751]: pgmap v11973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:29:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21258.101677 secs 2024-04-27T00:29:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:37 smithi090 ceph-mon[29180]: pgmap v11974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:29:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21258.101677 secs 2024-04-27T00:29:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:37 smithi119 ceph-mon[32752]: pgmap v11974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:29:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21258.101677 secs 2024-04-27T00:29:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:37 smithi167 ceph-mon[32751]: pgmap v11974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:29:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:40 smithi119 ceph-mon[32752]: pgmap v11975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:40 smithi167 ceph-mon[32751]: pgmap v11975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:40 smithi090 ceph-mon[29180]: pgmap v11975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21262.308526 secs 2024-04-27T00:29:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21262.308526 secs 2024-04-27T00:29:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21262.308526 secs 2024-04-27T00:29:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21263.101817 secs 2024-04-27T00:29:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:42 smithi119 ceph-mon[32752]: pgmap v11976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21263.101817 secs 2024-04-27T00:29:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:42 smithi167 ceph-mon[32751]: pgmap v11976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21263.101817 secs 2024-04-27T00:29:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:42 smithi090 ceph-mon[29180]: pgmap v11976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:43 smithi119 ceph-mon[32752]: pgmap v11977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:43 smithi167 ceph-mon[32751]: pgmap v11977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:43 smithi090 ceph-mon[29180]: pgmap v11977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:46 smithi119 ceph-mon[32752]: pgmap v11978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21267.308588 secs 2024-04-27T00:29:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:46 smithi167 ceph-mon[32751]: pgmap v11978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21267.308588 secs 2024-04-27T00:29:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:46 smithi090 ceph-mon[29180]: pgmap v11978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21267.308588 secs 2024-04-27T00:29:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21268.101899 secs 2024-04-27T00:29:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21268.101899 secs 2024-04-27T00:29:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21268.101899 secs 2024-04-27T00:29:48.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:48 smithi090 ceph-mon[29180]: pgmap v11979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:48 smithi119 ceph-mon[32752]: pgmap v11979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:48 smithi167 ceph-mon[32751]: pgmap v11979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:50 smithi119 ceph-mon[32752]: pgmap v11980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:50 smithi167 ceph-mon[32751]: pgmap v11980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:50 smithi090 ceph-mon[29180]: pgmap v11980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21272.308673 secs 2024-04-27T00:29:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21272.308673 secs 2024-04-27T00:29:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21272.308673 secs 2024-04-27T00:29:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21273.101969 secs 2024-04-27T00:29:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:52 smithi167 ceph-mon[32751]: pgmap v11981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21273.101969 secs 2024-04-27T00:29:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:52 smithi090 ceph-mon[29180]: pgmap v11981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21273.101969 secs 2024-04-27T00:29:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:52 smithi119 ceph-mon[32752]: pgmap v11981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:54 smithi167 ceph-mon[32751]: pgmap v11982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:54 smithi090 ceph-mon[29180]: pgmap v11982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:54 smithi119 ceph-mon[32752]: pgmap v11982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:56 smithi167 ceph-mon[32751]: pgmap v11983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21277.308754 secs 2024-04-27T00:29:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:56 smithi090 ceph-mon[29180]: pgmap v11983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21277.308754 secs 2024-04-27T00:29:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:56 smithi119 ceph-mon[32752]: pgmap v11983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:29:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21277.308754 secs 2024-04-27T00:29:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21278.102083 secs 2024-04-27T00:29:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21278.102083 secs 2024-04-27T00:29:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21278.102083 secs 2024-04-27T00:29:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:29:58 smithi090 ceph-mon[29180]: pgmap v11984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:29:58 smithi119 ceph-mon[32752]: pgmap v11984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:29:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:29:58 smithi167 ceph-mon[32751]: pgmap v11984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:00 smithi090 ceph-mon[29180]: pgmap v11985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:30:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:00 smithi119 ceph-mon[32752]: pgmap v11985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:30:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:00 smithi167 ceph-mon[32751]: pgmap v11985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:30:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21282.308848 secs 2024-04-27T00:30:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21282.308848 secs 2024-04-27T00:30:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21282.308848 secs 2024-04-27T00:30:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21283.102144 secs 2024-04-27T00:30:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:02 smithi090 ceph-mon[29180]: pgmap v11986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:02.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21283.102144 secs 2024-04-27T00:30:02.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:02 smithi119 ceph-mon[32752]: pgmap v11986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21283.102144 secs 2024-04-27T00:30:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:02 smithi167 ceph-mon[32751]: pgmap v11986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:04.362 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:04 smithi119 ceph-mon[32752]: pgmap v11987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:04 smithi090 ceph-mon[29180]: pgmap v11987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:04 smithi167 ceph-mon[32751]: pgmap v11987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:06 smithi090 ceph-mon[29180]: pgmap v11988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21287.308932 secs 2024-04-27T00:30:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:06 smithi119 ceph-mon[32752]: pgmap v11988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21287.308932 secs 2024-04-27T00:30:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:06 smithi167 ceph-mon[32751]: pgmap v11988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21287.308932 secs 2024-04-27T00:30:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21288.102234 secs 2024-04-27T00:30:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21288.102234 secs 2024-04-27T00:30:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21288.102234 secs 2024-04-27T00:30:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:08 smithi090 ceph-mon[29180]: pgmap v11989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:08 smithi119 ceph-mon[32752]: pgmap v11989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:08 smithi167 ceph-mon[32751]: pgmap v11989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:10 smithi090 ceph-mon[29180]: pgmap v11990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:10 smithi119 ceph-mon[32752]: pgmap v11990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:10 smithi167 ceph-mon[32751]: pgmap v11990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21292.309016 secs 2024-04-27T00:30:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21292.309016 secs 2024-04-27T00:30:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21292.309016 secs 2024-04-27T00:30:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21293.102347 secs 2024-04-27T00:30:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:12 smithi090 ceph-mon[29180]: pgmap v11991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21293.102347 secs 2024-04-27T00:30:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:12 smithi119 ceph-mon[32752]: pgmap v11991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21293.102347 secs 2024-04-27T00:30:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:12 smithi167 ceph-mon[32751]: pgmap v11991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:14 smithi090 ceph-mon[29180]: pgmap v11992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:14 smithi119 ceph-mon[32752]: pgmap v11992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:14 smithi167 ceph-mon[32751]: pgmap v11992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:16 smithi090 ceph-mon[29180]: pgmap v11993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21297.309092 secs 2024-04-27T00:30:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:16 smithi119 ceph-mon[32752]: pgmap v11993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21297.309092 secs 2024-04-27T00:30:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:16 smithi167 ceph-mon[32751]: pgmap v11993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21297.309092 secs 2024-04-27T00:30:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21298.102470 secs 2024-04-27T00:30:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21298.102470 secs 2024-04-27T00:30:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21298.102470 secs 2024-04-27T00:30:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:18 smithi090 ceph-mon[29180]: pgmap v11994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:18 smithi119 ceph-mon[32752]: pgmap v11994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:18 smithi167 ceph-mon[32751]: pgmap v11994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:20 smithi090 ceph-mon[29180]: pgmap v11995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:20 smithi119 ceph-mon[32752]: pgmap v11995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:20 smithi167 ceph-mon[32751]: pgmap v11995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:21.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21302.309171 secs 2024-04-27T00:30:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21302.309171 secs 2024-04-27T00:30:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21302.309171 secs 2024-04-27T00:30:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21303.102558 secs 2024-04-27T00:30:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:22 smithi119 ceph-mon[32752]: pgmap v11996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21303.102558 secs 2024-04-27T00:30:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:22 smithi167 ceph-mon[32751]: pgmap v11996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21303.102558 secs 2024-04-27T00:30:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:22 smithi090 ceph-mon[29180]: pgmap v11996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:24 smithi119 ceph-mon[32752]: pgmap v11997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:24 smithi167 ceph-mon[32751]: pgmap v11997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:24 smithi090 ceph-mon[29180]: pgmap v11997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:26 smithi119 ceph-mon[32752]: pgmap v11998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21307.309249 secs 2024-04-27T00:30:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:26 smithi167 ceph-mon[32751]: pgmap v11998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21307.309249 secs 2024-04-27T00:30:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:26 smithi090 ceph-mon[29180]: pgmap v11998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21307.309249 secs 2024-04-27T00:30:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21308.102629 secs 2024-04-27T00:30:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:30:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21308.102629 secs 2024-04-27T00:30:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:30:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21308.102629 secs 2024-04-27T00:30:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:30:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:28 smithi167 ceph-mon[32751]: pgmap v11999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:28 smithi090 ceph-mon[29180]: pgmap v11999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:28.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.414 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:28 smithi119 ceph-mon[32752]: pgmap v11999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:28.414 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.414 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.414 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:28.415 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: pgmap v12000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21312.309316 secs 2024-04-27T00:30:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: pgmap v12000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21312.309316 secs 2024-04-27T00:30:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: pgmap v12000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21312.309316 secs 2024-04-27T00:30:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:30:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:30:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:30:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:30:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:30:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:30:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:30:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:30:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:30:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:30:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:30:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:30:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:30:32.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:30:32.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:30:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:30:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:30:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:32 smithi090 ceph-mon[29180]: pgmap v12001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T00:30:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:32 smithi090 ceph-mon[29180]: pgmap v12002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21313.102767 secs 2024-04-27T00:30:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:30:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:30:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:30:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:30:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:32 smithi119 ceph-mon[32752]: pgmap v12001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T00:30:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:32 smithi119 ceph-mon[32752]: pgmap v12002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21313.102767 secs 2024-04-27T00:30:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:30:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:30:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:30:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:30:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:32 smithi167 ceph-mon[32751]: pgmap v12001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T00:30:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:32 smithi167 ceph-mon[32751]: pgmap v12002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21313.102767 secs 2024-04-27T00:30:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:34 smithi119 ceph-mon[32752]: pgmap v12003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:34 smithi167 ceph-mon[32751]: pgmap v12003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:34 smithi090 ceph-mon[29180]: pgmap v12003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21317.309397 secs 2024-04-27T00:30:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21317.309397 secs 2024-04-27T00:30:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21317.309397 secs 2024-04-27T00:30:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:36 smithi119 ceph-mon[32752]: pgmap v12004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:36 smithi167 ceph-mon[32751]: pgmap v12004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:36 smithi090 ceph-mon[29180]: pgmap v12004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21318.102878 secs 2024-04-27T00:30:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21318.102878 secs 2024-04-27T00:30:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21318.102878 secs 2024-04-27T00:30:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:38 smithi119 ceph-mon[32752]: pgmap v12005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:38 smithi167 ceph-mon[32751]: pgmap v12005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:38 smithi090 ceph-mon[29180]: pgmap v12005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:40 smithi119 ceph-mon[32752]: pgmap v12006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21322.309469 secs 2024-04-27T00:30:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:40 smithi167 ceph-mon[32751]: pgmap v12006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21322.309469 secs 2024-04-27T00:30:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:40 smithi090 ceph-mon[29180]: pgmap v12006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:30:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21322.309469 secs 2024-04-27T00:30:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:42 smithi119 ceph-mon[32752]: pgmap v12007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:30:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21323.102956 secs 2024-04-27T00:30:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:42 smithi167 ceph-mon[32751]: pgmap v12007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:30:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21323.102956 secs 2024-04-27T00:30:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:42 smithi090 ceph-mon[29180]: pgmap v12007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:30:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21323.102956 secs 2024-04-27T00:30:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:44 smithi119 ceph-mon[32752]: pgmap v12008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:44 smithi167 ceph-mon[32751]: pgmap v12008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:44 smithi090 ceph-mon[29180]: pgmap v12008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21327.309553 secs 2024-04-27T00:30:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21327.309553 secs 2024-04-27T00:30:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21327.309553 secs 2024-04-27T00:30:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:46 smithi167 ceph-mon[32751]: pgmap v12009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:46 smithi090 ceph-mon[29180]: pgmap v12009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:46 smithi119 ceph-mon[32752]: pgmap v12009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21328.103077 secs 2024-04-27T00:30:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21328.103077 secs 2024-04-27T00:30:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21328.103077 secs 2024-04-27T00:30:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:48 smithi167 ceph-mon[32751]: pgmap v12010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:48 smithi090 ceph-mon[29180]: pgmap v12010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:48 smithi119 ceph-mon[32752]: pgmap v12010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:50.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:50 smithi090 ceph-mon[29180]: pgmap v12011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:50.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21332.309635 secs 2024-04-27T00:30:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:50 smithi119 ceph-mon[32752]: pgmap v12011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21332.309635 secs 2024-04-27T00:30:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:50 smithi167 ceph-mon[32751]: pgmap v12011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21332.309635 secs 2024-04-27T00:30:52.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:52 smithi167 ceph-mon[32751]: pgmap v12012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:52.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21333.103168 secs 2024-04-27T00:30:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:52 smithi090 ceph-mon[29180]: pgmap v12012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21333.103168 secs 2024-04-27T00:30:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:52 smithi119 ceph-mon[32752]: pgmap v12012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21333.103168 secs 2024-04-27T00:30:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:54 smithi090 ceph-mon[29180]: pgmap v12013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:54 smithi119 ceph-mon[32752]: pgmap v12013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:54.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:54 smithi167 ceph-mon[32751]: pgmap v12013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21337.309717 secs 2024-04-27T00:30:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21337.309717 secs 2024-04-27T00:30:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21337.309717 secs 2024-04-27T00:30:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:56 smithi090 ceph-mon[29180]: pgmap v12014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:56 smithi119 ceph-mon[32752]: pgmap v12014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:56 smithi167 ceph-mon[32751]: pgmap v12014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:30:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21338.103305 secs 2024-04-27T00:30:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21338.103305 secs 2024-04-27T00:30:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21338.103305 secs 2024-04-27T00:30:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:30:58 smithi090 ceph-mon[29180]: pgmap v12015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:30:58 smithi119 ceph-mon[32752]: pgmap v12015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:30:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:30:58 smithi167 ceph-mon[32751]: pgmap v12015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:00 smithi090 ceph-mon[29180]: pgmap v12016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21342.309802 secs 2024-04-27T00:31:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:00 smithi119 ceph-mon[32752]: pgmap v12016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21342.309802 secs 2024-04-27T00:31:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:00 smithi167 ceph-mon[32751]: pgmap v12016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21342.309802 secs 2024-04-27T00:31:02.605 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:02 smithi119 ceph-mon[32752]: pgmap v12017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:02.606 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21343.103448 secs 2024-04-27T00:31:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:02 smithi090 ceph-mon[29180]: pgmap v12017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21343.103448 secs 2024-04-27T00:31:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:02 smithi167 ceph-mon[32751]: pgmap v12017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21343.103448 secs 2024-04-27T00:31:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:04 smithi090 ceph-mon[29180]: pgmap v12018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:04 smithi119 ceph-mon[32752]: pgmap v12018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:04 smithi167 ceph-mon[32751]: pgmap v12018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21347.309890 secs 2024-04-27T00:31:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21347.309890 secs 2024-04-27T00:31:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21347.309890 secs 2024-04-27T00:31:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:06 smithi090 ceph-mon[29180]: pgmap v12019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:06 smithi119 ceph-mon[32752]: pgmap v12019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:06 smithi167 ceph-mon[32751]: pgmap v12019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21348.103563 secs 2024-04-27T00:31:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21348.103563 secs 2024-04-27T00:31:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21348.103563 secs 2024-04-27T00:31:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:08 smithi090 ceph-mon[29180]: pgmap v12020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:08 smithi119 ceph-mon[32752]: pgmap v12020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:08 smithi167 ceph-mon[32751]: pgmap v12020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:10 smithi090 ceph-mon[29180]: pgmap v12021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21352.309979 secs 2024-04-27T00:31:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:10 smithi119 ceph-mon[32752]: pgmap v12021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21352.309979 secs 2024-04-27T00:31:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:10 smithi167 ceph-mon[32751]: pgmap v12021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21352.309979 secs 2024-04-27T00:31:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:12 smithi090 ceph-mon[29180]: pgmap v12022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21353.103682 secs 2024-04-27T00:31:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:12 smithi119 ceph-mon[32752]: pgmap v12022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21353.103682 secs 2024-04-27T00:31:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:12 smithi167 ceph-mon[32751]: pgmap v12022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21353.103682 secs 2024-04-27T00:31:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:13 smithi090 ceph-mon[29180]: pgmap v12023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:13 smithi119 ceph-mon[32752]: pgmap v12023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:13 smithi167 ceph-mon[32751]: pgmap v12023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:15 smithi119 ceph-mon[32752]: pgmap v12024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21357.310084 secs 2024-04-27T00:31:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:15 smithi167 ceph-mon[32751]: pgmap v12024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21357.310084 secs 2024-04-27T00:31:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:15 smithi090 ceph-mon[29180]: pgmap v12024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21357.310084 secs 2024-04-27T00:31:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21358.103797 secs 2024-04-27T00:31:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21358.103797 secs 2024-04-27T00:31:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21358.103797 secs 2024-04-27T00:31:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:17 smithi167 ceph-mon[32751]: pgmap v12025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:17 smithi090 ceph-mon[29180]: pgmap v12025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:17 smithi119 ceph-mon[32752]: pgmap v12025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:19 smithi167 ceph-mon[32751]: pgmap v12026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:19 smithi090 ceph-mon[29180]: pgmap v12026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:20.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:19 smithi119 ceph-mon[32752]: pgmap v12026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21362.310146 secs 2024-04-27T00:31:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21362.310146 secs 2024-04-27T00:31:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21362.310146 secs 2024-04-27T00:31:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:21 smithi090 ceph-mon[29180]: pgmap v12027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21363.103877 secs 2024-04-27T00:31:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:21 smithi119 ceph-mon[32752]: pgmap v12027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21363.103877 secs 2024-04-27T00:31:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:21 smithi167 ceph-mon[32751]: pgmap v12027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21363.103877 secs 2024-04-27T00:31:23.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:23 smithi167 ceph-mon[32751]: pgmap v12028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:23 smithi090 ceph-mon[29180]: pgmap v12028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:23 smithi119 ceph-mon[32752]: pgmap v12028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:25 smithi090 ceph-mon[29180]: pgmap v12029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21367.310227 secs 2024-04-27T00:31:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:25 smithi119 ceph-mon[32752]: pgmap v12029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21367.310227 secs 2024-04-27T00:31:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:25 smithi167 ceph-mon[32751]: pgmap v12029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21367.310227 secs 2024-04-27T00:31:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21368.103998 secs 2024-04-27T00:31:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21368.103998 secs 2024-04-27T00:31:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21368.103998 secs 2024-04-27T00:31:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:27 smithi090 ceph-mon[29180]: pgmap v12030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:27 smithi119 ceph-mon[32752]: pgmap v12030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:27 smithi167 ceph-mon[32751]: pgmap v12030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:29 smithi090 ceph-mon[29180]: pgmap v12031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:29 smithi119 ceph-mon[32752]: pgmap v12031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:29 smithi167 ceph-mon[32751]: pgmap v12031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21372.310329 secs 2024-04-27T00:31:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21372.310329 secs 2024-04-27T00:31:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21372.310329 secs 2024-04-27T00:31:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:31 smithi090 ceph-mon[29180]: pgmap v12032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:31:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21373.104070 secs 2024-04-27T00:31:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:31 smithi119 ceph-mon[32752]: pgmap v12032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:31:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21373.104070 secs 2024-04-27T00:31:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:31 smithi167 ceph-mon[32751]: pgmap v12032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:31:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21373.104070 secs 2024-04-27T00:31:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:31:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:31:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:32 smithi090 ceph-mon[29180]: pgmap v12033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:31:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:32 smithi090 ceph-mon[29180]: pgmap v12034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:32.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:31:32.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:31:32.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:32 smithi119 ceph-mon[32752]: pgmap v12033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:31:32.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:32 smithi119 ceph-mon[32752]: pgmap v12034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:32.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:31:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:31:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:32 smithi167 ceph-mon[32751]: pgmap v12033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:31:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:32 smithi167 ceph-mon[32751]: pgmap v12034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:31:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:34 smithi090 ceph-mon[29180]: pgmap v12035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:34 smithi119 ceph-mon[32752]: pgmap v12035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:34 smithi167 ceph-mon[32751]: pgmap v12035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21377.310413 secs 2024-04-27T00:31:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21377.310413 secs 2024-04-27T00:31:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21377.310413 secs 2024-04-27T00:31:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:36 smithi090 ceph-mon[29180]: pgmap v12036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21378.104208 secs 2024-04-27T00:31:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:36 smithi119 ceph-mon[32752]: pgmap v12036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21378.104208 secs 2024-04-27T00:31:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:36 smithi167 ceph-mon[32751]: pgmap v12036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21378.104208 secs 2024-04-27T00:31:39.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:38 smithi090 ceph-mon[29180]: pgmap v12037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:31:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:38 smithi119 ceph-mon[32752]: pgmap v12037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:31:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:38 smithi167 ceph-mon[32751]: pgmap v12037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:31:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:40 smithi090 ceph-mon[29180]: pgmap v12038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21382.310501 secs 2024-04-27T00:31:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:40 smithi119 ceph-mon[32752]: pgmap v12038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21382.310501 secs 2024-04-27T00:31:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:40 smithi167 ceph-mon[32751]: pgmap v12038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:31:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21382.310501 secs 2024-04-27T00:31:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21383.104328 secs 2024-04-27T00:31:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21383.104328 secs 2024-04-27T00:31:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21383.104328 secs 2024-04-27T00:31:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:42 smithi090 ceph-mon[29180]: pgmap v12039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:31:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:42 smithi119 ceph-mon[32752]: pgmap v12039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:31:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:42 smithi167 ceph-mon[32751]: pgmap v12039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:31:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:44 smithi090 ceph-mon[29180]: pgmap v12040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:44 smithi119 ceph-mon[32752]: pgmap v12040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:44 smithi167 ceph-mon[32751]: pgmap v12040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21387.310560 secs 2024-04-27T00:31:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21387.310560 secs 2024-04-27T00:31:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21387.310560 secs 2024-04-27T00:31:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:46 smithi090 ceph-mon[29180]: pgmap v12041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21388.104448 secs 2024-04-27T00:31:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:46 smithi119 ceph-mon[32752]: pgmap v12041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21388.104448 secs 2024-04-27T00:31:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:46 smithi167 ceph-mon[32751]: pgmap v12041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21388.104448 secs 2024-04-27T00:31:49.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:48 smithi090 ceph-mon[29180]: pgmap v12042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:49.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:48 smithi119 ceph-mon[32752]: pgmap v12042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:49.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:48 smithi167 ceph-mon[32751]: pgmap v12042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:50 smithi090 ceph-mon[29180]: pgmap v12043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21392.310648 secs 2024-04-27T00:31:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:50 smithi119 ceph-mon[32752]: pgmap v12043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21392.310648 secs 2024-04-27T00:31:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:50 smithi167 ceph-mon[32751]: pgmap v12043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21392.310648 secs 2024-04-27T00:31:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21393.104568 secs 2024-04-27T00:31:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21393.104568 secs 2024-04-27T00:31:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21393.104568 secs 2024-04-27T00:31:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:52 smithi090 ceph-mon[29180]: pgmap v12044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:53.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:52 smithi167 ceph-mon[32751]: pgmap v12044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:52 smithi119 ceph-mon[32752]: pgmap v12044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:31:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:54 smithi119 ceph-mon[32752]: pgmap v12045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:54 smithi167 ceph-mon[32751]: pgmap v12045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:54 smithi090 ceph-mon[29180]: pgmap v12045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:31:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21397.310733 secs 2024-04-27T00:31:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21397.310733 secs 2024-04-27T00:31:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21397.310733 secs 2024-04-27T00:31:56.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:56 smithi090 ceph-mon[29180]: pgmap v12046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:56.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21398.104666 secs 2024-04-27T00:31:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:56 smithi119 ceph-mon[32752]: pgmap v12046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21398.104666 secs 2024-04-27T00:31:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:56 smithi167 ceph-mon[32751]: pgmap v12046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21398.104666 secs 2024-04-27T00:31:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:31:58 smithi119 ceph-mon[32752]: pgmap v12047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:31:58 smithi167 ceph-mon[32751]: pgmap v12047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:31:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:31:58 smithi090 ceph-mon[29180]: pgmap v12047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:00 smithi119 ceph-mon[32752]: pgmap v12048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21402.310828 secs 2024-04-27T00:32:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:00 smithi167 ceph-mon[32751]: pgmap v12048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21402.310828 secs 2024-04-27T00:32:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:00 smithi090 ceph-mon[29180]: pgmap v12048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21402.310828 secs 2024-04-27T00:32:02.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21403.104794 secs 2024-04-27T00:32:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21403.104794 secs 2024-04-27T00:32:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21403.104794 secs 2024-04-27T00:32:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:02 smithi119 ceph-mon[32752]: pgmap v12049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:02 smithi167 ceph-mon[32751]: pgmap v12049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:02 smithi090 ceph-mon[29180]: pgmap v12049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:04 smithi119 ceph-mon[32752]: pgmap v12050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:04 smithi167 ceph-mon[32751]: pgmap v12050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:04 smithi090 ceph-mon[29180]: pgmap v12050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21407.310914 secs 2024-04-27T00:32:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21407.310914 secs 2024-04-27T00:32:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21407.310914 secs 2024-04-27T00:32:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:06 smithi119 ceph-mon[32752]: pgmap v12051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21408.104930 secs 2024-04-27T00:32:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:06 smithi167 ceph-mon[32751]: pgmap v12051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21408.104930 secs 2024-04-27T00:32:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:06 smithi090 ceph-mon[29180]: pgmap v12051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21408.104930 secs 2024-04-27T00:32:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:08 smithi119 ceph-mon[32752]: pgmap v12052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:08 smithi167 ceph-mon[32751]: pgmap v12052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:08 smithi090 ceph-mon[29180]: pgmap v12052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:10 smithi119 ceph-mon[32752]: pgmap v12053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21412.310976 secs 2024-04-27T00:32:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:10 smithi167 ceph-mon[32751]: pgmap v12053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21412.310976 secs 2024-04-27T00:32:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:10 smithi090 ceph-mon[29180]: pgmap v12053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21412.310976 secs 2024-04-27T00:32:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21413.105065 secs 2024-04-27T00:32:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21413.105065 secs 2024-04-27T00:32:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21413.105065 secs 2024-04-27T00:32:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:12 smithi119 ceph-mon[32752]: pgmap v12054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:12 smithi167 ceph-mon[32751]: pgmap v12054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:12 smithi090 ceph-mon[29180]: pgmap v12054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:14 smithi119 ceph-mon[32752]: pgmap v12055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:14 smithi167 ceph-mon[32751]: pgmap v12055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:14 smithi090 ceph-mon[29180]: pgmap v12055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21417.311080 secs 2024-04-27T00:32:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21417.311080 secs 2024-04-27T00:32:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21417.311080 secs 2024-04-27T00:32:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:16 smithi119 ceph-mon[32752]: pgmap v12056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21418.105140 secs 2024-04-27T00:32:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:16 smithi167 ceph-mon[32751]: pgmap v12056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21418.105140 secs 2024-04-27T00:32:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:16 smithi090 ceph-mon[29180]: pgmap v12056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21418.105140 secs 2024-04-27T00:32:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:18 smithi119 ceph-mon[32752]: pgmap v12057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:18 smithi167 ceph-mon[32751]: pgmap v12057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:18 smithi090 ceph-mon[29180]: pgmap v12057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:20 smithi119 ceph-mon[32752]: pgmap v12058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21422.311149 secs 2024-04-27T00:32:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:20 smithi167 ceph-mon[32751]: pgmap v12058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21422.311149 secs 2024-04-27T00:32:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:20 smithi090 ceph-mon[29180]: pgmap v12058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21422.311149 secs 2024-04-27T00:32:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21423.105273 secs 2024-04-27T00:32:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21423.105273 secs 2024-04-27T00:32:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21423.105273 secs 2024-04-27T00:32:23.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:22 smithi167 ceph-mon[32751]: pgmap v12059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:22 smithi119 ceph-mon[32752]: pgmap v12059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:22 smithi090 ceph-mon[29180]: pgmap v12059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:24 smithi119 ceph-mon[32752]: pgmap v12060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:25.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:24 smithi167 ceph-mon[32751]: pgmap v12060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:24 smithi090 ceph-mon[29180]: pgmap v12060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21427.311234 secs 2024-04-27T00:32:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21427.311234 secs 2024-04-27T00:32:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21427.311234 secs 2024-04-27T00:32:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:26 smithi119 ceph-mon[32752]: pgmap v12061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21428.105414 secs 2024-04-27T00:32:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:26 smithi167 ceph-mon[32751]: pgmap v12061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21428.105414 secs 2024-04-27T00:32:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:26 smithi090 ceph-mon[29180]: pgmap v12061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21428.105414 secs 2024-04-27T00:32:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:28 smithi119 ceph-mon[32752]: pgmap v12062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:28 smithi167 ceph-mon[32751]: pgmap v12062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:28 smithi090 ceph-mon[29180]: pgmap v12062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:30 smithi119 ceph-mon[32752]: pgmap v12063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:30 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 21432.311318 secs 2024-04-27T00:32:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:30 smithi119 ceph-mon[32752]: slow request 2883.815426 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:32:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:30 smithi167 ceph-mon[32751]: pgmap v12063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:30 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 21432.311318 secs 2024-04-27T00:32:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:30 smithi167 ceph-mon[32751]: slow request 2883.815426 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:32:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:30 smithi090 ceph-mon[29180]: pgmap v12063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:30 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 21432.311318 secs 2024-04-27T00:32:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:30 smithi090 ceph-mon[29180]: slow request 2883.815426 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:32:32.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21433.105532 secs 2024-04-27T00:32:32.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:32:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21433.105532 secs 2024-04-27T00:32:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:32:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21433.105532 secs 2024-04-27T00:32:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:32:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:32 smithi119 ceph-mon[32752]: pgmap v12064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:32:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:32:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:32:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:32 smithi167 ceph-mon[32751]: pgmap v12064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:32:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:32:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:32:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:32 smithi090 ceph-mon[29180]: pgmap v12064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:32:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:32:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:32:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:33 smithi119 ceph-mon[32752]: pgmap v12065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:32:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:33 smithi119 ceph-mon[32752]: pgmap v12066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:32:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:33 smithi167 ceph-mon[32751]: pgmap v12065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:32:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:33 smithi167 ceph-mon[32751]: pgmap v12066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:32:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:33 smithi090 ceph-mon[29180]: pgmap v12065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:32:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:33 smithi090 ceph-mon[29180]: pgmap v12066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:32:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:35 smithi119 ceph-mon[32752]: pgmap v12067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:32:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21437.311402 secs 2024-04-27T00:32:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:35 smithi167 ceph-mon[32751]: pgmap v12067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:32:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21437.311402 secs 2024-04-27T00:32:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:35 smithi090 ceph-mon[29180]: pgmap v12067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:32:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21437.311402 secs 2024-04-27T00:32:36.993 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:32:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:32:36.701+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:32:36.993 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:32:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:32:36.701+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:32:36.993 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:32:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:32:36.701+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:32:36.994 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:32:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:32:36.701+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:32:36.994 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:32:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:32:36.701+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:32:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21438.105647 secs 2024-04-27T00:32:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21438.105647 secs 2024-04-27T00:32:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21438.105647 secs 2024-04-27T00:32:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:37 smithi119 ceph-mon[32752]: pgmap v12068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:32:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:37 smithi167 ceph-mon[32751]: pgmap v12068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:32:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:37 smithi090 ceph-mon[29180]: pgmap v12068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:32:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:39 smithi119 ceph-mon[32752]: pgmap v12069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-27T00:32:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:39 smithi167 ceph-mon[32751]: pgmap v12069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-27T00:32:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:39 smithi090 ceph-mon[29180]: pgmap v12069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-27T00:32:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21442.311487 secs 2024-04-27T00:32:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21442.311487 secs 2024-04-27T00:32:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21442.311487 secs 2024-04-27T00:32:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:42 smithi119 ceph-mon[32752]: pgmap v12070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-27T00:32:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21443.105761 secs 2024-04-27T00:32:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:42 smithi167 ceph-mon[32751]: pgmap v12070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-27T00:32:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21443.105761 secs 2024-04-27T00:32:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:42 smithi090 ceph-mon[29180]: pgmap v12070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 233 B/s wr, 17 op/s 2024-04-27T00:32:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21443.105761 secs 2024-04-27T00:32:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:43 smithi119 ceph-mon[32752]: pgmap v12071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 204 B/s wr, 15 op/s 2024-04-27T00:32:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:43 smithi167 ceph-mon[32751]: pgmap v12071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 204 B/s wr, 15 op/s 2024-04-27T00:32:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:43 smithi090 ceph-mon[29180]: pgmap v12071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 204 B/s wr, 15 op/s 2024-04-27T00:32:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:45 smithi090 ceph-mon[29180]: pgmap v12072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21447.311570 secs 2024-04-27T00:32:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:45 smithi119 ceph-mon[32752]: pgmap v12072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21447.311570 secs 2024-04-27T00:32:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:45 smithi167 ceph-mon[32751]: pgmap v12072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21447.311570 secs 2024-04-27T00:32:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21448.105843 secs 2024-04-27T00:32:47.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21448.105843 secs 2024-04-27T00:32:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21448.105843 secs 2024-04-27T00:32:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:47 smithi090 ceph-mon[29180]: pgmap v12073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:48.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:47 smithi119 ceph-mon[32752]: pgmap v12073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:47 smithi167 ceph-mon[32751]: pgmap v12073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:49 smithi090 ceph-mon[29180]: pgmap v12074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:50.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:49 smithi119 ceph-mon[32752]: pgmap v12074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:49 smithi167 ceph-mon[32751]: pgmap v12074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:32:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21452.311654 secs 2024-04-27T00:32:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21452.311654 secs 2024-04-27T00:32:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21452.311654 secs 2024-04-27T00:32:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:51 smithi119 ceph-mon[32752]: pgmap v12075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:52.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21453.105963 secs 2024-04-27T00:32:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:51 smithi167 ceph-mon[32751]: pgmap v12075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21453.105963 secs 2024-04-27T00:32:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:51 smithi090 ceph-mon[29180]: pgmap v12075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21453.105963 secs 2024-04-27T00:32:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:53 smithi119 ceph-mon[32752]: pgmap v12076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:53 smithi167 ceph-mon[32751]: pgmap v12076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:53 smithi090 ceph-mon[29180]: pgmap v12076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:55 smithi119 ceph-mon[32752]: pgmap v12077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21457.311739 secs 2024-04-27T00:32:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:55 smithi167 ceph-mon[32751]: pgmap v12077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21457.311739 secs 2024-04-27T00:32:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:55 smithi090 ceph-mon[29180]: pgmap v12077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:32:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21457.311739 secs 2024-04-27T00:32:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21458.106063 secs 2024-04-27T00:32:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21458.106063 secs 2024-04-27T00:32:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21458.106063 secs 2024-04-27T00:32:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:57 smithi119 ceph-mon[32752]: pgmap v12078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:58.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:57 smithi167 ceph-mon[32751]: pgmap v12078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:32:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:57 smithi090 ceph-mon[29180]: pgmap v12078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:00.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:32:59 smithi119 ceph-mon[32752]: pgmap v12079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:32:59 smithi167 ceph-mon[32751]: pgmap v12079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:32:59 smithi090 ceph-mon[29180]: pgmap v12079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21462.311826 secs 2024-04-27T00:33:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21462.311826 secs 2024-04-27T00:33:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21462.311826 secs 2024-04-27T00:33:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:01 smithi090 ceph-mon[29180]: pgmap v12080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:02.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21463.106140 secs 2024-04-27T00:33:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:01 smithi119 ceph-mon[32752]: pgmap v12080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21463.106140 secs 2024-04-27T00:33:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:01 smithi167 ceph-mon[32751]: pgmap v12080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21463.106140 secs 2024-04-27T00:33:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:03 smithi119 ceph-mon[32752]: pgmap v12081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:04.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:03 smithi167 ceph-mon[32751]: pgmap v12081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:03 smithi090 ceph-mon[29180]: pgmap v12081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:05 smithi119 ceph-mon[32752]: pgmap v12082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:05 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 21467.311909 secs 2024-04-27T00:33:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:05 smithi119 ceph-mon[32752]: slow request 11521.580327 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:33:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:05 smithi167 ceph-mon[32751]: pgmap v12082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:05 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 21467.311909 secs 2024-04-27T00:33:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:05 smithi167 ceph-mon[32751]: slow request 11521.580327 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:33:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:05 smithi090 ceph-mon[29180]: pgmap v12082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:05 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 21467.311909 secs 2024-04-27T00:33:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:05 smithi090 ceph-mon[29180]: slow request 11521.580327 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:33:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21468.106213 secs 2024-04-27T00:33:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21468.106213 secs 2024-04-27T00:33:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21468.106213 secs 2024-04-27T00:33:08.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:07 smithi119 ceph-mon[32752]: pgmap v12083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:07 smithi167 ceph-mon[32751]: pgmap v12083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:07 smithi090 ceph-mon[29180]: pgmap v12083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:10.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:09 smithi119 ceph-mon[32752]: pgmap v12084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:10.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:09 smithi167 ceph-mon[32751]: pgmap v12084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:10.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:09 smithi090 ceph-mon[29180]: pgmap v12084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21472.311971 secs 2024-04-27T00:33:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21472.311971 secs 2024-04-27T00:33:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21472.311971 secs 2024-04-27T00:33:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:11 smithi119 ceph-mon[32752]: pgmap v12085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21473.106327 secs 2024-04-27T00:33:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:11 smithi167 ceph-mon[32751]: pgmap v12085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21473.106327 secs 2024-04-27T00:33:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:11 smithi090 ceph-mon[29180]: pgmap v12085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21473.106327 secs 2024-04-27T00:33:14.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:13 smithi119 ceph-mon[32752]: pgmap v12086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:14.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:13 smithi167 ceph-mon[32751]: pgmap v12086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:13 smithi090 ceph-mon[29180]: pgmap v12086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:33:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:15 smithi119 ceph-mon[32752]: pgmap v12087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:15 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 21477.312032 secs 2024-04-27T00:33:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:15 smithi119 ceph-mon[32752]: slow request 11521.543767 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-27T00:33:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:15 smithi167 ceph-mon[32751]: pgmap v12087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:15 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 21477.312032 secs 2024-04-27T00:33:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:15 smithi167 ceph-mon[32751]: slow request 11521.543767 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-27T00:33:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:15 smithi090 ceph-mon[29180]: pgmap v12087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:33:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:15 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 21477.312032 secs 2024-04-27T00:33:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:15 smithi090 ceph-mon[29180]: slow request 11521.543767 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-27T00:33:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21478.106448 secs 2024-04-27T00:33:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21478.106448 secs 2024-04-27T00:33:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21478.106448 secs 2024-04-27T00:33:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:17 smithi119 ceph-mon[32752]: pgmap v12088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:17 smithi167 ceph-mon[32751]: pgmap v12088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:17 smithi090 ceph-mon[29180]: pgmap v12088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:20.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:19 smithi119 ceph-mon[32752]: pgmap v12089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:19 smithi167 ceph-mon[32751]: pgmap v12089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:19 smithi090 ceph-mon[29180]: pgmap v12089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21482.312109 secs 2024-04-27T00:33:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21482.312109 secs 2024-04-27T00:33:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21482.312109 secs 2024-04-27T00:33:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:21 smithi119 ceph-mon[32752]: pgmap v12090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21483.106515 secs 2024-04-27T00:33:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:21 smithi167 ceph-mon[32751]: pgmap v12090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21483.106515 secs 2024-04-27T00:33:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:21 smithi090 ceph-mon[29180]: pgmap v12090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21483.106515 secs 2024-04-27T00:33:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:23 smithi119 ceph-mon[32752]: pgmap v12091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:23 smithi167 ceph-mon[32751]: pgmap v12091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:24.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:23 smithi090 ceph-mon[29180]: pgmap v12091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:25 smithi119 ceph-mon[32752]: pgmap v12092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21487.312170 secs 2024-04-27T00:33:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:25 smithi167 ceph-mon[32751]: pgmap v12092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21487.312170 secs 2024-04-27T00:33:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:25 smithi090 ceph-mon[29180]: pgmap v12092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21487.312170 secs 2024-04-27T00:33:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21488.106638 secs 2024-04-27T00:33:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21488.106638 secs 2024-04-27T00:33:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21488.106638 secs 2024-04-27T00:33:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:27 smithi119 ceph-mon[32752]: pgmap v12093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:27 smithi167 ceph-mon[32751]: pgmap v12093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:28.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:27 smithi090 ceph-mon[29180]: pgmap v12093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:29 smithi119 ceph-mon[32752]: pgmap v12094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:29 smithi167 ceph-mon[32751]: pgmap v12094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:30.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:29 smithi090 ceph-mon[29180]: pgmap v12094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21492.312255 secs 2024-04-27T00:33:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21492.312255 secs 2024-04-27T00:33:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21492.312255 secs 2024-04-27T00:33:32.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:31 smithi090 ceph-mon[29180]: pgmap v12095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:32.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21493.106748 secs 2024-04-27T00:33:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:31 smithi119 ceph-mon[32752]: pgmap v12095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21493.106748 secs 2024-04-27T00:33:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:31 smithi167 ceph-mon[32751]: pgmap v12095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21493.106748 secs 2024-04-27T00:33:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:33:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:33:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:33:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:33 smithi119 ceph-mon[32752]: pgmap v12096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:33:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:33:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:33 smithi119 ceph-mon[32752]: pgmap v12097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:33:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:33 smithi119 ceph-mon[32752]: pgmap v12098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:33:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:33 smithi167 ceph-mon[32751]: pgmap v12096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:33:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:33:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:33 smithi167 ceph-mon[32751]: pgmap v12097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:33:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:33 smithi167 ceph-mon[32751]: pgmap v12098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:33:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:33 smithi090 ceph-mon[29180]: pgmap v12096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:33:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:33:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:33 smithi090 ceph-mon[29180]: pgmap v12097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:33:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:33 smithi090 ceph-mon[29180]: pgmap v12098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:33:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21497.312340 secs 2024-04-27T00:33:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21497.312340 secs 2024-04-27T00:33:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21497.312340 secs 2024-04-27T00:33:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:36 smithi090 ceph-mon[29180]: pgmap v12099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:36 smithi119 ceph-mon[32752]: pgmap v12099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:36 smithi167 ceph-mon[32751]: pgmap v12099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21498.106857 secs 2024-04-27T00:33:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21498.106857 secs 2024-04-27T00:33:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21498.106857 secs 2024-04-27T00:33:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:38 smithi090 ceph-mon[29180]: pgmap v12100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:33:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:38 smithi119 ceph-mon[32752]: pgmap v12100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:33:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:38 smithi167 ceph-mon[32751]: pgmap v12100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:33:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:40 smithi090 ceph-mon[29180]: pgmap v12101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21502.312426 secs 2024-04-27T00:33:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:40 smithi119 ceph-mon[32752]: pgmap v12101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21502.312426 secs 2024-04-27T00:33:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:40 smithi167 ceph-mon[32751]: pgmap v12101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21502.312426 secs 2024-04-27T00:33:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21503.106970 secs 2024-04-27T00:33:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:42 smithi090 ceph-mon[29180]: pgmap v12102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21503.106970 secs 2024-04-27T00:33:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:42 smithi119 ceph-mon[32752]: pgmap v12102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21503.106970 secs 2024-04-27T00:33:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:42 smithi167 ceph-mon[32751]: pgmap v12102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:33:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:44 smithi090 ceph-mon[29180]: pgmap v12103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:33:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:44 smithi119 ceph-mon[32752]: pgmap v12103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:33:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:44 smithi167 ceph-mon[32751]: pgmap v12103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:33:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21507.312484 secs 2024-04-27T00:33:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21507.312484 secs 2024-04-27T00:33:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21507.312484 secs 2024-04-27T00:33:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:46 smithi090 ceph-mon[29180]: pgmap v12104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:46 smithi119 ceph-mon[32752]: pgmap v12104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:46 smithi167 ceph-mon[32751]: pgmap v12104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21508.107119 secs 2024-04-27T00:33:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21508.107119 secs 2024-04-27T00:33:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21508.107119 secs 2024-04-27T00:33:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:48 smithi090 ceph-mon[29180]: pgmap v12105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:48 smithi119 ceph-mon[32752]: pgmap v12105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:48 smithi167 ceph-mon[32751]: pgmap v12105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:50 smithi090 ceph-mon[29180]: pgmap v12106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21512.312542 secs 2024-04-27T00:33:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:50 smithi119 ceph-mon[32752]: pgmap v12106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21512.312542 secs 2024-04-27T00:33:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:50 smithi167 ceph-mon[32751]: pgmap v12106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21512.312542 secs 2024-04-27T00:33:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21513.107226 secs 2024-04-27T00:33:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:52 smithi090 ceph-mon[29180]: pgmap v12107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21513.107226 secs 2024-04-27T00:33:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:52 smithi119 ceph-mon[32752]: pgmap v12107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21513.107226 secs 2024-04-27T00:33:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:52 smithi167 ceph-mon[32751]: pgmap v12107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:54.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:54 smithi167 ceph-mon[32751]: pgmap v12108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:54 smithi090 ceph-mon[29180]: pgmap v12108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:54 smithi119 ceph-mon[32752]: pgmap v12108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:33:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21517.312623 secs 2024-04-27T00:33:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21517.312623 secs 2024-04-27T00:33:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21517.312623 secs 2024-04-27T00:33:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:56 smithi090 ceph-mon[29180]: pgmap v12109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:56 smithi119 ceph-mon[32752]: pgmap v12109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:56 smithi167 ceph-mon[32751]: pgmap v12109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21518.107288 secs 2024-04-27T00:33:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21518.107288 secs 2024-04-27T00:33:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21518.107288 secs 2024-04-27T00:33:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:33:58 smithi090 ceph-mon[29180]: pgmap v12110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:33:58 smithi119 ceph-mon[32752]: pgmap v12110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:33:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:33:58 smithi167 ceph-mon[32751]: pgmap v12110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:00 smithi119 ceph-mon[32752]: pgmap v12111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21522.312708 secs 2024-04-27T00:34:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:00 smithi167 ceph-mon[32751]: pgmap v12111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21522.312708 secs 2024-04-27T00:34:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:00 smithi090 ceph-mon[29180]: pgmap v12111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21522.312708 secs 2024-04-27T00:34:02.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21523.107403 secs 2024-04-27T00:34:02.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:02 smithi090 ceph-mon[29180]: pgmap v12112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21523.107403 secs 2024-04-27T00:34:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:02 smithi119 ceph-mon[32752]: pgmap v12112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21523.107403 secs 2024-04-27T00:34:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:02 smithi167 ceph-mon[32751]: pgmap v12112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:04 smithi119 ceph-mon[32752]: pgmap v12113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:04 smithi167 ceph-mon[32751]: pgmap v12113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:04 smithi090 ceph-mon[29180]: pgmap v12113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21527.312813 secs 2024-04-27T00:34:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21527.312813 secs 2024-04-27T00:34:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21527.312813 secs 2024-04-27T00:34:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:06 smithi119 ceph-mon[32752]: pgmap v12114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:06 smithi167 ceph-mon[32751]: pgmap v12114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:06 smithi090 ceph-mon[29180]: pgmap v12114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21528.107500 secs 2024-04-27T00:34:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:07 smithi119 ceph-mon[32752]: pgmap v12115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21528.107500 secs 2024-04-27T00:34:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:07 smithi167 ceph-mon[32751]: pgmap v12115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21528.107500 secs 2024-04-27T00:34:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:07 smithi090 ceph-mon[29180]: pgmap v12115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:09 smithi119 ceph-mon[32752]: pgmap v12116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:09 smithi167 ceph-mon[32751]: pgmap v12116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:09 smithi090 ceph-mon[29180]: pgmap v12116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21532.312884 secs 2024-04-27T00:34:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21532.312884 secs 2024-04-27T00:34:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21532.312884 secs 2024-04-27T00:34:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:11 smithi119 ceph-mon[32752]: pgmap v12117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:11 smithi167 ceph-mon[32751]: pgmap v12117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:11 smithi090 ceph-mon[29180]: pgmap v12117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21533.107585 secs 2024-04-27T00:34:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21533.107585 secs 2024-04-27T00:34:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21533.107585 secs 2024-04-27T00:34:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:13 smithi119 ceph-mon[32752]: pgmap v12118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:13 smithi167 ceph-mon[32751]: pgmap v12118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:13 smithi090 ceph-mon[29180]: pgmap v12118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21537.312972 secs 2024-04-27T00:34:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21537.312972 secs 2024-04-27T00:34:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21537.312972 secs 2024-04-27T00:34:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:16 smithi090 ceph-mon[29180]: pgmap v12119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:16 smithi119 ceph-mon[32752]: pgmap v12119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:16 smithi167 ceph-mon[32751]: pgmap v12119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21538.107701 secs 2024-04-27T00:34:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21538.107701 secs 2024-04-27T00:34:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21538.107701 secs 2024-04-27T00:34:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:18 smithi090 ceph-mon[29180]: pgmap v12120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:18 smithi119 ceph-mon[32752]: pgmap v12120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:18 smithi167 ceph-mon[32751]: pgmap v12120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:19 smithi119 ceph-mon[32752]: pgmap v12121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:19 smithi167 ceph-mon[32751]: pgmap v12121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:19 smithi090 ceph-mon[29180]: pgmap v12121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21542.313035 secs 2024-04-27T00:34:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21542.313035 secs 2024-04-27T00:34:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21542.313035 secs 2024-04-27T00:34:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:21 smithi119 ceph-mon[32752]: pgmap v12122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:21 smithi167 ceph-mon[32751]: pgmap v12122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:21 smithi090 ceph-mon[29180]: pgmap v12122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21543.107814 secs 2024-04-27T00:34:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21543.107814 secs 2024-04-27T00:34:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21543.107814 secs 2024-04-27T00:34:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:23 smithi119 ceph-mon[32752]: pgmap v12123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:23 smithi167 ceph-mon[32751]: pgmap v12123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:23 smithi090 ceph-mon[29180]: pgmap v12123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:25.619 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21547.313118 secs 2024-04-27T00:34:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21547.313118 secs 2024-04-27T00:34:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21547.313118 secs 2024-04-27T00:34:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:26 smithi090 ceph-mon[29180]: pgmap v12124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:26 smithi119 ceph-mon[32752]: pgmap v12124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:26 smithi167 ceph-mon[32751]: pgmap v12124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21548.107950 secs 2024-04-27T00:34:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21548.107950 secs 2024-04-27T00:34:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21548.107950 secs 2024-04-27T00:34:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:28 smithi090 ceph-mon[29180]: pgmap v12125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:28 smithi119 ceph-mon[32752]: pgmap v12125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:28 smithi167 ceph-mon[32751]: pgmap v12125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:29 smithi119 ceph-mon[32752]: pgmap v12126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:29 smithi167 ceph-mon[32751]: pgmap v12126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:29 smithi090 ceph-mon[29180]: pgmap v12126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21552.313200 secs 2024-04-27T00:34:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21552.313200 secs 2024-04-27T00:34:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21552.313200 secs 2024-04-27T00:34:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:31 smithi119 ceph-mon[32752]: pgmap v12127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:31 smithi167 ceph-mon[32751]: pgmap v12127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:31 smithi090 ceph-mon[29180]: pgmap v12127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21553.108080 secs 2024-04-27T00:34:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21553.108080 secs 2024-04-27T00:34:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21553.108080 secs 2024-04-27T00:34:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:33 smithi119 ceph-mon[32752]: pgmap v12128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:34:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:33 smithi167 ceph-mon[32751]: pgmap v12128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:34:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:33 smithi090 ceph-mon[29180]: pgmap v12128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:34:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:34:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:34:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:34 smithi119 ceph-mon[32752]: pgmap v12129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:34:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:34 smithi119 ceph-mon[32752]: pgmap v12130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:34:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:34:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:34:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:34 smithi167 ceph-mon[32751]: pgmap v12129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:34:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:34 smithi167 ceph-mon[32751]: pgmap v12130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:34:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:34:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:34:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:34 smithi090 ceph-mon[29180]: pgmap v12129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:34:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:34 smithi090 ceph-mon[29180]: pgmap v12130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:34:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21557.313273 secs 2024-04-27T00:34:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21557.313273 secs 2024-04-27T00:34:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21557.313273 secs 2024-04-27T00:34:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:36 smithi119 ceph-mon[32752]: pgmap v12131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:36 smithi167 ceph-mon[32751]: pgmap v12131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:36.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:36 smithi090 ceph-mon[29180]: pgmap v12131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21558.108198 secs 2024-04-27T00:34:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21558.108198 secs 2024-04-27T00:34:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21558.108198 secs 2024-04-27T00:34:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:38 smithi119 ceph-mon[32752]: pgmap v12132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:34:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:38 smithi167 ceph-mon[32751]: pgmap v12132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:34:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:38 smithi090 ceph-mon[29180]: pgmap v12132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:34:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21562.313358 secs 2024-04-27T00:34:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:41 smithi119 ceph-mon[32752]: pgmap v12133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21562.313358 secs 2024-04-27T00:34:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:41 smithi167 ceph-mon[32751]: pgmap v12133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21562.313358 secs 2024-04-27T00:34:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:41 smithi090 ceph-mon[29180]: pgmap v12133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:34:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:42 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21563.108268 secs 2024-04-27T00:34:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:42 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21563.108268 secs 2024-04-27T00:34:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:42 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21563.108268 secs 2024-04-27T00:34:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:43 smithi119 ceph-mon[32752]: pgmap v12134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:34:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:43 smithi167 ceph-mon[32751]: pgmap v12134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:34:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:43 smithi090 ceph-mon[29180]: pgmap v12134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:34:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:45 smithi119 ceph-mon[32752]: pgmap v12135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:34:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:45 smithi167 ceph-mon[32751]: pgmap v12135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:34:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:45 smithi090 ceph-mon[29180]: pgmap v12135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:34:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21567.313442 secs 2024-04-27T00:34:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21567.313442 secs 2024-04-27T00:34:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21567.313442 secs 2024-04-27T00:34:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:47 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21568.108406 secs 2024-04-27T00:34:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:47 smithi119 ceph-mon[32752]: pgmap v12136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:47 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21568.108406 secs 2024-04-27T00:34:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:47 smithi167 ceph-mon[32751]: pgmap v12136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:47 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21568.108406 secs 2024-04-27T00:34:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:47 smithi090 ceph-mon[29180]: pgmap v12136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:49 smithi119 ceph-mon[32752]: pgmap v12137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:49 smithi167 ceph-mon[32751]: pgmap v12137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:49 smithi090 ceph-mon[29180]: pgmap v12137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21572.313521 secs 2024-04-27T00:34:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:51 smithi119 ceph-mon[32752]: pgmap v12138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21572.313521 secs 2024-04-27T00:34:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:51 smithi167 ceph-mon[32751]: pgmap v12138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21572.313521 secs 2024-04-27T00:34:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:51 smithi090 ceph-mon[29180]: pgmap v12138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:52 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21573.108493 secs 2024-04-27T00:34:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:52 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21573.108493 secs 2024-04-27T00:34:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:52 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21573.108493 secs 2024-04-27T00:34:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:53 smithi119 ceph-mon[32752]: pgmap v12139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:53 smithi167 ceph-mon[32751]: pgmap v12139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:53 smithi090 ceph-mon[29180]: pgmap v12139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:55 smithi119 ceph-mon[32752]: pgmap v12140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:55 smithi167 ceph-mon[32751]: pgmap v12140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:55 smithi090 ceph-mon[29180]: pgmap v12140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:34:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21577.313601 secs 2024-04-27T00:34:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21577.313601 secs 2024-04-27T00:34:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21577.313601 secs 2024-04-27T00:34:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:57 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21578.108636 secs 2024-04-27T00:34:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:57 smithi119 ceph-mon[32752]: pgmap v12141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:57 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21578.108636 secs 2024-04-27T00:34:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:57 smithi167 ceph-mon[32751]: pgmap v12141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:57 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21578.108636 secs 2024-04-27T00:34:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:57 smithi090 ceph-mon[29180]: pgmap v12141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:34:59 smithi119 ceph-mon[32752]: pgmap v12142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:34:59 smithi167 ceph-mon[32751]: pgmap v12142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:34:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:34:59 smithi090 ceph-mon[29180]: pgmap v12142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21582.313685 secs 2024-04-27T00:35:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:01 smithi119 ceph-mon[32752]: pgmap v12143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21582.313685 secs 2024-04-27T00:35:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:01 smithi167 ceph-mon[32751]: pgmap v12143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21582.313685 secs 2024-04-27T00:35:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:01 smithi090 ceph-mon[29180]: pgmap v12143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:02 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21583.108734 secs 2024-04-27T00:35:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:02 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21583.108734 secs 2024-04-27T00:35:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:02 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21583.108734 secs 2024-04-27T00:35:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:03 smithi119 ceph-mon[32752]: pgmap v12144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:03 smithi167 ceph-mon[32751]: pgmap v12144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:03 smithi090 ceph-mon[29180]: pgmap v12144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:05 smithi119 ceph-mon[32752]: pgmap v12145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:05 smithi167 ceph-mon[32751]: pgmap v12145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:05 smithi090 ceph-mon[29180]: pgmap v12145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21587.313766 secs 2024-04-27T00:35:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21587.313766 secs 2024-04-27T00:35:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21587.313766 secs 2024-04-27T00:35:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:07 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21588.108795 secs 2024-04-27T00:35:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:07 smithi119 ceph-mon[32752]: pgmap v12146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:07 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21588.108795 secs 2024-04-27T00:35:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:07 smithi167 ceph-mon[32751]: pgmap v12146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:07 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21588.108795 secs 2024-04-27T00:35:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:07 smithi090 ceph-mon[29180]: pgmap v12146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:09 smithi119 ceph-mon[32752]: pgmap v12147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:09 smithi167 ceph-mon[32751]: pgmap v12147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:09 smithi090 ceph-mon[29180]: pgmap v12147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:11 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 21592.313852 secs 2024-04-27T00:35:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:11 smithi119 ceph-mon[32752]: slow request 1443.721297 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:35:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:11 smithi119 ceph-mon[32752]: pgmap v12148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:11 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 21592.313852 secs 2024-04-27T00:35:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:11 smithi167 ceph-mon[32751]: slow request 1443.721297 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:35:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:11 smithi167 ceph-mon[32751]: pgmap v12148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:11 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 21592.313852 secs 2024-04-27T00:35:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:11 smithi090 ceph-mon[29180]: slow request 1443.721297 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:35:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:11 smithi090 ceph-mon[29180]: pgmap v12148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:12 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21593.108855 secs 2024-04-27T00:35:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:12 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21593.108855 secs 2024-04-27T00:35:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:12 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21593.108855 secs 2024-04-27T00:35:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:13 smithi119 ceph-mon[32752]: pgmap v12149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:13 smithi167 ceph-mon[32751]: pgmap v12149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:13 smithi090 ceph-mon[29180]: pgmap v12149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:15 smithi119 ceph-mon[32752]: pgmap v12150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:15.557 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:15 smithi090 ceph-mon[29180]: pgmap v12150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:15 smithi167 ceph-mon[32751]: pgmap v12150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21597.313930 secs 2024-04-27T00:35:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21597.313930 secs 2024-04-27T00:35:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21597.313930 secs 2024-04-27T00:35:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:17 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21598.108937 secs 2024-04-27T00:35:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:17 smithi119 ceph-mon[32752]: pgmap v12151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:17 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21598.108937 secs 2024-04-27T00:35:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:17 smithi167 ceph-mon[32751]: pgmap v12151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:17 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21598.108937 secs 2024-04-27T00:35:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:17 smithi090 ceph-mon[29180]: pgmap v12151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:19 smithi119 ceph-mon[32752]: pgmap v12152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:19 smithi167 ceph-mon[32751]: pgmap v12152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:19 smithi090 ceph-mon[29180]: pgmap v12152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21602.314013 secs 2024-04-27T00:35:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:21 smithi119 ceph-mon[32752]: pgmap v12153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21602.314013 secs 2024-04-27T00:35:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:21 smithi167 ceph-mon[32751]: pgmap v12153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21602.314013 secs 2024-04-27T00:35:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:21 smithi090 ceph-mon[29180]: pgmap v12153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:22 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21603.109073 secs 2024-04-27T00:35:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:22 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21603.109073 secs 2024-04-27T00:35:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:22 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21603.109073 secs 2024-04-27T00:35:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:23 smithi119 ceph-mon[32752]: pgmap v12154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:23 smithi167 ceph-mon[32751]: pgmap v12154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:23 smithi090 ceph-mon[29180]: pgmap v12154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:25 smithi119 ceph-mon[32752]: pgmap v12155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:25 smithi167 ceph-mon[32751]: pgmap v12155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:25 smithi090 ceph-mon[29180]: pgmap v12155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21607.314118 secs 2024-04-27T00:35:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21607.314118 secs 2024-04-27T00:35:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21607.314118 secs 2024-04-27T00:35:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:27 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21608.109161 secs 2024-04-27T00:35:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:27 smithi119 ceph-mon[32752]: pgmap v12156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:27 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21608.109161 secs 2024-04-27T00:35:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:27 smithi167 ceph-mon[32751]: pgmap v12156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:27 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21608.109161 secs 2024-04-27T00:35:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:27 smithi090 ceph-mon[29180]: pgmap v12156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:29 smithi119 ceph-mon[32752]: pgmap v12157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:29 smithi167 ceph-mon[32751]: pgmap v12157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:29 smithi090 ceph-mon[29180]: pgmap v12157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21612.314203 secs 2024-04-27T00:35:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21612.314203 secs 2024-04-27T00:35:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21612.314203 secs 2024-04-27T00:35:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:31 smithi119 ceph-mon[32752]: pgmap v12158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:31 smithi167 ceph-mon[32751]: pgmap v12158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:31 smithi090 ceph-mon[29180]: pgmap v12158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:32 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21613.109278 secs 2024-04-27T00:35:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:32 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21613.109278 secs 2024-04-27T00:35:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:32 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21613.109278 secs 2024-04-27T00:35:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:33 smithi167 ceph-mon[32751]: pgmap v12159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:33 smithi090 ceph-mon[29180]: pgmap v12159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:33 smithi119 ceph-mon[32752]: pgmap v12159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:35:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:35:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:35:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:35 smithi167 ceph-mon[32751]: pgmap v12160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:35:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:35:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:35:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21617.314288 secs 2024-04-27T00:35:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:35 smithi090 ceph-mon[29180]: pgmap v12160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:35:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:35:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:35:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21617.314288 secs 2024-04-27T00:35:35.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:35 smithi119 ceph-mon[32752]: pgmap v12160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:35:35.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21617.314288 secs 2024-04-27T00:35:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:36 smithi090 ceph-mon[29180]: pgmap v12161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:35:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:36 smithi090 ceph-mon[29180]: pgmap v12162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:36 smithi119 ceph-mon[32752]: pgmap v12161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:35:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:36 smithi119 ceph-mon[32752]: pgmap v12162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:36 smithi167 ceph-mon[32751]: pgmap v12161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:35:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:36 smithi167 ceph-mon[32751]: pgmap v12162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:37 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 21618.109416 secs 2024-04-27T00:35:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:37 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 21618.109416 secs 2024-04-27T00:35:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:37 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 21618.109416 secs 2024-04-27T00:35:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:38 smithi090 ceph-mon[29180]: pgmap v12163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:38 smithi119 ceph-mon[32752]: pgmap v12163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:38 smithi167 ceph-mon[32751]: pgmap v12163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:40 smithi090 ceph-mon[29180]: pgmap v12164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21622.314372 secs 2024-04-27T00:35:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:40 smithi119 ceph-mon[32752]: pgmap v12164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21622.314372 secs 2024-04-27T00:35:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:40 smithi167 ceph-mon[32751]: pgmap v12164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21622.314372 secs 2024-04-27T00:35:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:42 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 21623.109523 secs 2024-04-27T00:35:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:42 smithi090 ceph-mon[29180]: slow request 184.152271 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:35:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:42 smithi090 ceph-mon[29180]: pgmap v12165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:42 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 21623.109523 secs 2024-04-27T00:35:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:42 smithi119 ceph-mon[32752]: slow request 184.152271 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:35:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:42 smithi119 ceph-mon[32752]: pgmap v12165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:42 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 21623.109523 secs 2024-04-27T00:35:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:42 smithi167 ceph-mon[32751]: slow request 184.152271 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:35:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:42 smithi167 ceph-mon[32751]: pgmap v12165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:35:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:44 smithi090 ceph-mon[29180]: pgmap v12166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:35:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:35:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:44 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:35:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:44 smithi119 ceph-mon[32752]: pgmap v12166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:35:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:35:44.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:44 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:35:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:44 smithi167 ceph-mon[32751]: pgmap v12166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:35:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:35:44.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:44 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:35:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21627.314452 secs 2024-04-27T00:35:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21627.314452 secs 2024-04-27T00:35:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21627.314452 secs 2024-04-27T00:35:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:46 smithi090 ceph-mon[29180]: pgmap v12167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:35:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:46 smithi119 ceph-mon[32752]: pgmap v12167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:35:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:46 smithi167 ceph-mon[32751]: pgmap v12167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:35:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:47 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21628.109602 secs 2024-04-27T00:35:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:47 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21628.109602 secs 2024-04-27T00:35:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:47 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21628.109602 secs 2024-04-27T00:35:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:48 smithi090 ceph-mon[29180]: pgmap v12168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:48 smithi119 ceph-mon[32752]: pgmap v12168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:48 smithi167 ceph-mon[32751]: pgmap v12168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:50 smithi090 ceph-mon[29180]: pgmap v12169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21632.314535 secs 2024-04-27T00:35:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:50 smithi119 ceph-mon[32752]: pgmap v12169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21632.314535 secs 2024-04-27T00:35:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:50 smithi167 ceph-mon[32751]: pgmap v12169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21632.314535 secs 2024-04-27T00:35:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:52 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21633.109693 secs 2024-04-27T00:35:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:52 smithi090 ceph-mon[29180]: pgmap v12170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:52 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21633.109693 secs 2024-04-27T00:35:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:52 smithi119 ceph-mon[32752]: pgmap v12170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:52 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21633.109693 secs 2024-04-27T00:35:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:52 smithi167 ceph-mon[32751]: pgmap v12170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:54 smithi090 ceph-mon[29180]: pgmap v12171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:54 smithi119 ceph-mon[32752]: pgmap v12171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:54 smithi167 ceph-mon[32751]: pgmap v12171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21637.314617 secs 2024-04-27T00:35:55.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21637.314617 secs 2024-04-27T00:35:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21637.314617 secs 2024-04-27T00:35:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:56 smithi090 ceph-mon[29180]: pgmap v12172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:56 smithi119 ceph-mon[32752]: pgmap v12172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:56 smithi167 ceph-mon[32751]: pgmap v12172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:35:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:57 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21638.109756 secs 2024-04-27T00:35:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:57 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21638.109756 secs 2024-04-27T00:35:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:57 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21638.109756 secs 2024-04-27T00:35:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:35:58 smithi119 ceph-mon[32752]: pgmap v12173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:35:58 smithi167 ceph-mon[32751]: pgmap v12173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:35:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:35:58 smithi090 ceph-mon[29180]: pgmap v12173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:00 smithi119 ceph-mon[32752]: pgmap v12174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21642.314702 secs 2024-04-27T00:36:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:00 smithi167 ceph-mon[32751]: pgmap v12174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21642.314702 secs 2024-04-27T00:36:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:00 smithi090 ceph-mon[29180]: pgmap v12174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21642.314702 secs 2024-04-27T00:36:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:02 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21643.109850 secs 2024-04-27T00:36:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:02 smithi119 ceph-mon[32752]: pgmap v12175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:02 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21643.109850 secs 2024-04-27T00:36:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:02 smithi167 ceph-mon[32751]: pgmap v12175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:02 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21643.109850 secs 2024-04-27T00:36:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:02 smithi090 ceph-mon[29180]: pgmap v12175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:04 smithi119 ceph-mon[32752]: pgmap v12176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:04 smithi167 ceph-mon[32751]: pgmap v12176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:04 smithi090 ceph-mon[29180]: pgmap v12176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:05.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21647.314810 secs 2024-04-27T00:36:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21647.314810 secs 2024-04-27T00:36:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21647.314810 secs 2024-04-27T00:36:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:06 smithi119 ceph-mon[32752]: pgmap v12177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:06 smithi167 ceph-mon[32751]: pgmap v12177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:06 smithi090 ceph-mon[29180]: pgmap v12177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:07 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21648.110000 secs 2024-04-27T00:36:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:07 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21648.110000 secs 2024-04-27T00:36:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:07 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21648.110000 secs 2024-04-27T00:36:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:08 smithi119 ceph-mon[32752]: pgmap v12178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:08 smithi167 ceph-mon[32751]: pgmap v12178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:08 smithi090 ceph-mon[29180]: pgmap v12178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:10 smithi119 ceph-mon[32752]: pgmap v12179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21652.314876 secs 2024-04-27T00:36:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:10 smithi167 ceph-mon[32751]: pgmap v12179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21652.314876 secs 2024-04-27T00:36:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:10 smithi090 ceph-mon[29180]: pgmap v12179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21652.314876 secs 2024-04-27T00:36:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:11 smithi119 ceph-mon[32752]: pgmap v12180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:11 smithi167 ceph-mon[32751]: pgmap v12180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:11 smithi090 ceph-mon[29180]: pgmap v12180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:12 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21653.110078 secs 2024-04-27T00:36:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:12 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21653.110078 secs 2024-04-27T00:36:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:12 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21653.110078 secs 2024-04-27T00:36:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:13 smithi119 ceph-mon[32752]: pgmap v12181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:13 smithi167 ceph-mon[32751]: pgmap v12181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:13 smithi090 ceph-mon[29180]: pgmap v12181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:15 smithi119 ceph-mon[32752]: pgmap v12182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21657.314960 secs 2024-04-27T00:36:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:15 smithi167 ceph-mon[32751]: pgmap v12182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21657.314960 secs 2024-04-27T00:36:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:15 smithi090 ceph-mon[29180]: pgmap v12182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21657.314960 secs 2024-04-27T00:36:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21658.110195 secs 2024-04-27T00:36:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21658.110195 secs 2024-04-27T00:36:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21658.110195 secs 2024-04-27T00:36:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:17 smithi119 ceph-mon[32752]: pgmap v12183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:17 smithi167 ceph-mon[32751]: pgmap v12183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:17 smithi090 ceph-mon[29180]: pgmap v12183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:19 smithi119 ceph-mon[32752]: pgmap v12184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:19 smithi167 ceph-mon[32751]: pgmap v12184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:19 smithi090 ceph-mon[29180]: pgmap v12184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21662.315021 secs 2024-04-27T00:36:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21662.315021 secs 2024-04-27T00:36:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21662.315021 secs 2024-04-27T00:36:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21663.110264 secs 2024-04-27T00:36:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:21 smithi119 ceph-mon[32752]: pgmap v12185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21663.110264 secs 2024-04-27T00:36:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:21 smithi167 ceph-mon[32751]: pgmap v12185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21663.110264 secs 2024-04-27T00:36:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:21 smithi090 ceph-mon[29180]: pgmap v12185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:23 smithi119 ceph-mon[32752]: pgmap v12186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:23 smithi167 ceph-mon[32751]: pgmap v12186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:23 smithi090 ceph-mon[29180]: pgmap v12186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:25 smithi119 ceph-mon[32752]: pgmap v12187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21667.315107 secs 2024-04-27T00:36:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:25 smithi167 ceph-mon[32751]: pgmap v12187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21667.315107 secs 2024-04-27T00:36:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:25 smithi090 ceph-mon[29180]: pgmap v12187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21667.315107 secs 2024-04-27T00:36:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21668.110416 secs 2024-04-27T00:36:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21668.110416 secs 2024-04-27T00:36:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21668.110416 secs 2024-04-27T00:36:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:27 smithi119 ceph-mon[32752]: pgmap v12188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:27 smithi167 ceph-mon[32751]: pgmap v12188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:27 smithi090 ceph-mon[29180]: pgmap v12188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:29 smithi119 ceph-mon[32752]: pgmap v12189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:29 smithi167 ceph-mon[32751]: pgmap v12189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:29 smithi090 ceph-mon[29180]: pgmap v12189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21672.315186 secs 2024-04-27T00:36:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21672.315186 secs 2024-04-27T00:36:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21672.315186 secs 2024-04-27T00:36:32.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21673.110574 secs 2024-04-27T00:36:32.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:31 smithi090 ceph-mon[29180]: pgmap v12190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21673.110574 secs 2024-04-27T00:36:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:31 smithi119 ceph-mon[32752]: pgmap v12190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21673.110574 secs 2024-04-27T00:36:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:31 smithi167 ceph-mon[32751]: pgmap v12190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:33 smithi119 ceph-mon[32752]: pgmap v12191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:33 smithi167 ceph-mon[32751]: pgmap v12191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:33 smithi090 ceph-mon[29180]: pgmap v12191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:36:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:36:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:36:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:36 smithi119 ceph-mon[32752]: pgmap v12192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21677.315248 secs 2024-04-27T00:36:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:36:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:36:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:36:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:36 smithi167 ceph-mon[32751]: pgmap v12192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21677.315248 secs 2024-04-27T00:36:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:36:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:36:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:36:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:36 smithi090 ceph-mon[29180]: pgmap v12192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21677.315248 secs 2024-04-27T00:36:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:36:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:36:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:36:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:37 smithi119 ceph-mon[32752]: pgmap v12193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:36:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:37 smithi119 ceph-mon[32752]: pgmap v12194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:37 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21678.110657 secs 2024-04-27T00:36:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:37 smithi167 ceph-mon[32751]: pgmap v12193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:36:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:37 smithi167 ceph-mon[32751]: pgmap v12194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:37 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21678.110657 secs 2024-04-27T00:36:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:37 smithi090 ceph-mon[29180]: pgmap v12193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:36:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:37 smithi090 ceph-mon[29180]: pgmap v12194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:37 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21678.110657 secs 2024-04-27T00:36:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:39 smithi119 ceph-mon[32752]: pgmap v12195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:36:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:39 smithi167 ceph-mon[32751]: pgmap v12195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:36:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:39 smithi090 ceph-mon[29180]: pgmap v12195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:36:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:41 smithi119 ceph-mon[32752]: pgmap v12196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21682.315328 secs 2024-04-27T00:36:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:41 smithi167 ceph-mon[32751]: pgmap v12196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21682.315328 secs 2024-04-27T00:36:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:41 smithi090 ceph-mon[29180]: pgmap v12196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21682.315328 secs 2024-04-27T00:36:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:42 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21683.110740 secs 2024-04-27T00:36:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:42 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21683.110740 secs 2024-04-27T00:36:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:42 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21683.110740 secs 2024-04-27T00:36:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:43 smithi119 ceph-mon[32752]: pgmap v12197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:43 smithi167 ceph-mon[32751]: pgmap v12197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:43 smithi090 ceph-mon[29180]: pgmap v12197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:45 smithi119 ceph-mon[32752]: pgmap v12198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:45 smithi167 ceph-mon[32751]: pgmap v12198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:45 smithi090 ceph-mon[29180]: pgmap v12198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:36:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21687.315414 secs 2024-04-27T00:36:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21687.315414 secs 2024-04-27T00:36:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21687.315414 secs 2024-04-27T00:36:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:47 smithi119 ceph-mon[32752]: pgmap v12199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:36:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:47 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21688.110851 secs 2024-04-27T00:36:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:47 smithi167 ceph-mon[32751]: pgmap v12199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:36:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:47 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21688.110851 secs 2024-04-27T00:36:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:47 smithi090 ceph-mon[29180]: pgmap v12199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:36:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:47 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21688.110851 secs 2024-04-27T00:36:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:49 smithi119 ceph-mon[32752]: pgmap v12200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:49 smithi167 ceph-mon[32751]: pgmap v12200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:49 smithi090 ceph-mon[29180]: pgmap v12200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:51 smithi119 ceph-mon[32752]: pgmap v12201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21692.315497 secs 2024-04-27T00:36:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:51 smithi167 ceph-mon[32751]: pgmap v12201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21692.315497 secs 2024-04-27T00:36:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:51 smithi090 ceph-mon[29180]: pgmap v12201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21692.315497 secs 2024-04-27T00:36:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:52 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21693.110926 secs 2024-04-27T00:36:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:52 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21693.110926 secs 2024-04-27T00:36:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:52 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21693.110926 secs 2024-04-27T00:36:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:53 smithi167 ceph-mon[32751]: pgmap v12202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:53 smithi090 ceph-mon[29180]: pgmap v12202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:53 smithi119 ceph-mon[32752]: pgmap v12202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:55 smithi167 ceph-mon[32751]: pgmap v12203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:55 smithi090 ceph-mon[29180]: pgmap v12203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:55 smithi119 ceph-mon[32752]: pgmap v12203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:36:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21697.315583 secs 2024-04-27T00:36:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21697.315583 secs 2024-04-27T00:36:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21697.315583 secs 2024-04-27T00:36:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:57 smithi090 ceph-mon[29180]: pgmap v12204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:57 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21698.111046 secs 2024-04-27T00:36:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:57 smithi119 ceph-mon[32752]: pgmap v12204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:57 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21698.111046 secs 2024-04-27T00:36:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:57 smithi167 ceph-mon[32751]: pgmap v12204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:57 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21698.111046 secs 2024-04-27T00:36:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:36:58 smithi090 ceph-mon[29180]: pgmap v12205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:36:58 smithi119 ceph-mon[32752]: pgmap v12205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:36:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:36:58 smithi167 ceph-mon[32751]: pgmap v12205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:01.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:00 smithi119 ceph-mon[32752]: pgmap v12206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21702.315665 secs 2024-04-27T00:37:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:00 smithi167 ceph-mon[32751]: pgmap v12206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:01.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21702.315665 secs 2024-04-27T00:37:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:00 smithi090 ceph-mon[29180]: pgmap v12206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21702.315665 secs 2024-04-27T00:37:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21703.111166 secs 2024-04-27T00:37:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21703.111166 secs 2024-04-27T00:37:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21703.111166 secs 2024-04-27T00:37:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:02 smithi119 ceph-mon[32752]: pgmap v12207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:02 smithi167 ceph-mon[32751]: pgmap v12207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:02 smithi090 ceph-mon[29180]: pgmap v12207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:04 smithi090 ceph-mon[29180]: pgmap v12208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:04 smithi119 ceph-mon[32752]: pgmap v12208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:04 smithi167 ceph-mon[32751]: pgmap v12208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21707.315726 secs 2024-04-27T00:37:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21707.315726 secs 2024-04-27T00:37:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21707.315726 secs 2024-04-27T00:37:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:06 smithi119 ceph-mon[32752]: pgmap v12209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21708.111288 secs 2024-04-27T00:37:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:06 smithi167 ceph-mon[32751]: pgmap v12209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21708.111288 secs 2024-04-27T00:37:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:06 smithi090 ceph-mon[29180]: pgmap v12209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21708.111288 secs 2024-04-27T00:37:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:08 smithi119 ceph-mon[32752]: pgmap v12210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:08 smithi167 ceph-mon[32751]: pgmap v12210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:08 smithi090 ceph-mon[29180]: pgmap v12210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:10 smithi119 ceph-mon[32752]: pgmap v12211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21712.315812 secs 2024-04-27T00:37:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:10 smithi167 ceph-mon[32751]: pgmap v12211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21712.315812 secs 2024-04-27T00:37:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:10 smithi090 ceph-mon[29180]: pgmap v12211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21712.315812 secs 2024-04-27T00:37:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21713.111422 secs 2024-04-27T00:37:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21713.111422 secs 2024-04-27T00:37:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21713.111422 secs 2024-04-27T00:37:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:12 smithi119 ceph-mon[32752]: pgmap v12212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:12 smithi167 ceph-mon[32751]: pgmap v12212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:12 smithi090 ceph-mon[29180]: pgmap v12212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:14 smithi119 ceph-mon[32752]: pgmap v12213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:14 smithi167 ceph-mon[32751]: pgmap v12213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:15.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:14 smithi090 ceph-mon[29180]: pgmap v12213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21717.315896 secs 2024-04-27T00:37:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21717.315896 secs 2024-04-27T00:37:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21717.315896 secs 2024-04-27T00:37:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:16 smithi119 ceph-mon[32752]: pgmap v12214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21718.111522 secs 2024-04-27T00:37:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:16 smithi167 ceph-mon[32751]: pgmap v12214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21718.111522 secs 2024-04-27T00:37:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:16 smithi090 ceph-mon[29180]: pgmap v12214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21718.111522 secs 2024-04-27T00:37:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:18 smithi119 ceph-mon[32752]: pgmap v12215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:18 smithi167 ceph-mon[32751]: pgmap v12215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:18 smithi090 ceph-mon[29180]: pgmap v12215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:21.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:20 smithi119 ceph-mon[32752]: pgmap v12216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21722.315980 secs 2024-04-27T00:37:21.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:20 smithi167 ceph-mon[32751]: pgmap v12216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21722.315980 secs 2024-04-27T00:37:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:20 smithi090 ceph-mon[29180]: pgmap v12216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21722.315980 secs 2024-04-27T00:37:22.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21723.111642 secs 2024-04-27T00:37:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21723.111642 secs 2024-04-27T00:37:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21723.111642 secs 2024-04-27T00:37:23.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:22 smithi119 ceph-mon[32752]: pgmap v12217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:22 smithi167 ceph-mon[32751]: pgmap v12217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:23.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:22 smithi090 ceph-mon[29180]: pgmap v12217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:24 smithi119 ceph-mon[32752]: pgmap v12218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:24 smithi167 ceph-mon[32751]: pgmap v12218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:25.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:24 smithi090 ceph-mon[29180]: pgmap v12218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21727.316064 secs 2024-04-27T00:37:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21727.316064 secs 2024-04-27T00:37:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21727.316064 secs 2024-04-27T00:37:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:26 smithi119 ceph-mon[32752]: pgmap v12219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21728.111756 secs 2024-04-27T00:37:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:26 smithi167 ceph-mon[32751]: pgmap v12219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21728.111756 secs 2024-04-27T00:37:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:26 smithi090 ceph-mon[29180]: pgmap v12219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21728.111756 secs 2024-04-27T00:37:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:28 smithi119 ceph-mon[32752]: pgmap v12220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:28 smithi167 ceph-mon[32751]: pgmap v12220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:29.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:28 smithi090 ceph-mon[29180]: pgmap v12220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:30 smithi119 ceph-mon[32752]: pgmap v12221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21732.316148 secs 2024-04-27T00:37:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:30 smithi167 ceph-mon[32751]: pgmap v12221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21732.316148 secs 2024-04-27T00:37:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:30 smithi090 ceph-mon[29180]: pgmap v12221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21732.316148 secs 2024-04-27T00:37:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21733.111871 secs 2024-04-27T00:37:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21733.111871 secs 2024-04-27T00:37:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21733.111871 secs 2024-04-27T00:37:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:32 smithi119 ceph-mon[32752]: pgmap v12222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:33.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:32 smithi167 ceph-mon[32751]: pgmap v12222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:33.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:32 smithi090 ceph-mon[29180]: pgmap v12222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:35.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:34 smithi119 ceph-mon[32752]: pgmap v12223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:34 smithi167 ceph-mon[32751]: pgmap v12223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:35.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:34 smithi090 ceph-mon[29180]: pgmap v12223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21737.316235 secs 2024-04-27T00:37:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:37:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21737.316235 secs 2024-04-27T00:37:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:37:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21737.316235 secs 2024-04-27T00:37:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:37:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-mon[29180]: pgmap v12224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21738.111966 secs 2024-04-27T00:37:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:37:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:37:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:37:37.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:37:36.753+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:37:37.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:37:36.753+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:37:37.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:37:36.753+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:37:37.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:37:36.753+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:37:37.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:37:36 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:37:36.753+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:37:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:36 smithi119 ceph-mon[32752]: pgmap v12224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21738.111966 secs 2024-04-27T00:37:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:37:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:37:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:37:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:36 smithi167 ceph-mon[32751]: pgmap v12224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21738.111966 secs 2024-04-27T00:37:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:37:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:37:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:37:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:37 smithi090 ceph-mon[29180]: pgmap v12225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:37:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:37 smithi090 ceph-mon[29180]: pgmap v12226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:37:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:37 smithi119 ceph-mon[32752]: pgmap v12225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:37:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:37 smithi119 ceph-mon[32752]: pgmap v12226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:37:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:37 smithi167 ceph-mon[32751]: pgmap v12225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:37:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:37 smithi167 ceph-mon[32751]: pgmap v12226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:37:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:39 smithi090 ceph-mon[29180]: pgmap v12227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:39 smithi119 ceph-mon[32752]: pgmap v12227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:39 smithi167 ceph-mon[32751]: pgmap v12227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21742.316319 secs 2024-04-27T00:37:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21742.316319 secs 2024-04-27T00:37:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21742.316319 secs 2024-04-27T00:37:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:41 smithi090 ceph-mon[29180]: pgmap v12228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21743.112042 secs 2024-04-27T00:37:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:41 smithi119 ceph-mon[32752]: pgmap v12228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21743.112042 secs 2024-04-27T00:37:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:41 smithi167 ceph-mon[32751]: pgmap v12228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21743.112042 secs 2024-04-27T00:37:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:43 smithi090 ceph-mon[29180]: pgmap v12229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T00:37:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:43 smithi119 ceph-mon[32752]: pgmap v12229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T00:37:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:43 smithi167 ceph-mon[32751]: pgmap v12229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T00:37:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:45 smithi090 ceph-mon[29180]: pgmap v12230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21747.316404 secs 2024-04-27T00:37:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:45 smithi119 ceph-mon[32752]: pgmap v12230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21747.316404 secs 2024-04-27T00:37:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:45 smithi167 ceph-mon[32751]: pgmap v12230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T00:37:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21747.316404 secs 2024-04-27T00:37:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21748.112154 secs 2024-04-27T00:37:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21748.112154 secs 2024-04-27T00:37:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21748.112154 secs 2024-04-27T00:37:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:47 smithi090 ceph-mon[29180]: pgmap v12231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T00:37:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:47 smithi119 ceph-mon[32752]: pgmap v12231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T00:37:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:47 smithi167 ceph-mon[32751]: pgmap v12231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T00:37:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:49 smithi090 ceph-mon[29180]: pgmap v12232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:37:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:49 smithi119 ceph-mon[32752]: pgmap v12232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:37:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:49 smithi167 ceph-mon[32751]: pgmap v12232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T00:37:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21752.316489 secs 2024-04-27T00:37:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21752.316489 secs 2024-04-27T00:37:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21752.316489 secs 2024-04-27T00:37:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:51 smithi090 ceph-mon[29180]: pgmap v12233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21753.112247 secs 2024-04-27T00:37:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:51 smithi119 ceph-mon[32752]: pgmap v12233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21753.112247 secs 2024-04-27T00:37:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:51 smithi167 ceph-mon[32751]: pgmap v12233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21753.112247 secs 2024-04-27T00:37:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:53 smithi090 ceph-mon[29180]: pgmap v12234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:53 smithi119 ceph-mon[32752]: pgmap v12234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:53 smithi167 ceph-mon[32751]: pgmap v12234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:55 smithi090 ceph-mon[29180]: pgmap v12235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21757.316578 secs 2024-04-27T00:37:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:55 smithi119 ceph-mon[32752]: pgmap v12235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21757.316578 secs 2024-04-27T00:37:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:55 smithi167 ceph-mon[32751]: pgmap v12235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:37:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21757.316578 secs 2024-04-27T00:37:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21758.112427 secs 2024-04-27T00:37:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21758.112427 secs 2024-04-27T00:37:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21758.112427 secs 2024-04-27T00:37:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:57 smithi119 ceph-mon[32752]: pgmap v12236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:57 smithi167 ceph-mon[32751]: pgmap v12236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:37:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:57 smithi090 ceph-mon[29180]: pgmap v12236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:37:59 smithi119 ceph-mon[32752]: pgmap v12237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:37:59 smithi167 ceph-mon[32751]: pgmap v12237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:37:59 smithi090 ceph-mon[29180]: pgmap v12237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21762.316661 secs 2024-04-27T00:38:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21762.316661 secs 2024-04-27T00:38:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21762.316661 secs 2024-04-27T00:38:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:01 smithi090 ceph-mon[29180]: pgmap v12238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21763.112518 secs 2024-04-27T00:38:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:01 smithi119 ceph-mon[32752]: pgmap v12238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21763.112518 secs 2024-04-27T00:38:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:01 smithi167 ceph-mon[32751]: pgmap v12238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21763.112518 secs 2024-04-27T00:38:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:03 smithi119 ceph-mon[32752]: pgmap v12239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:03 smithi167 ceph-mon[32751]: pgmap v12239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:03 smithi090 ceph-mon[29180]: pgmap v12239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:05 smithi119 ceph-mon[32752]: pgmap v12240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21767.316747 secs 2024-04-27T00:38:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:05 smithi167 ceph-mon[32751]: pgmap v12240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21767.316747 secs 2024-04-27T00:38:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:05 smithi090 ceph-mon[29180]: pgmap v12240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:06.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21767.316747 secs 2024-04-27T00:38:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21768.112656 secs 2024-04-27T00:38:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21768.112656 secs 2024-04-27T00:38:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21768.112656 secs 2024-04-27T00:38:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:07 smithi119 ceph-mon[32752]: pgmap v12241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:07 smithi167 ceph-mon[32751]: pgmap v12241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:07 smithi090 ceph-mon[29180]: pgmap v12241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:10.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:09 smithi119 ceph-mon[32752]: pgmap v12242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:09 smithi167 ceph-mon[32751]: pgmap v12242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:09 smithi090 ceph-mon[29180]: pgmap v12242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:10 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 21772.316847 secs 2024-04-27T00:38:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:10 smithi119 ceph-mon[32752]: slow request 2884.078596 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:38:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:10 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 21772.316847 secs 2024-04-27T00:38:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:10 smithi167 ceph-mon[32751]: slow request 2884.078596 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:38:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:10 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 21772.316847 secs 2024-04-27T00:38:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:10 smithi090 ceph-mon[29180]: slow request 2884.078596 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:38:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:11 smithi119 ceph-mon[32752]: pgmap v12243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21773.112724 secs 2024-04-27T00:38:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:11 smithi167 ceph-mon[32751]: pgmap v12243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21773.112724 secs 2024-04-27T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:11 smithi090 ceph-mon[29180]: pgmap v12243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21773.112724 secs 2024-04-27T00:38:14.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:13 smithi119 ceph-mon[32752]: pgmap v12244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:13 smithi167 ceph-mon[32751]: pgmap v12244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:14.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:13 smithi090 ceph-mon[29180]: pgmap v12244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:15 smithi119 ceph-mon[32752]: pgmap v12245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21777.316939 secs 2024-04-27T00:38:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:15 smithi167 ceph-mon[32751]: pgmap v12245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21777.316939 secs 2024-04-27T00:38:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:15 smithi090 ceph-mon[29180]: pgmap v12245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21777.316939 secs 2024-04-27T00:38:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21778.112816 secs 2024-04-27T00:38:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21778.112816 secs 2024-04-27T00:38:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21778.112816 secs 2024-04-27T00:38:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:17 smithi119 ceph-mon[32752]: pgmap v12246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:17 smithi167 ceph-mon[32751]: pgmap v12246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:17 smithi090 ceph-mon[29180]: pgmap v12246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:19 smithi119 ceph-mon[32752]: pgmap v12247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:19 smithi167 ceph-mon[32751]: pgmap v12247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:19 smithi090 ceph-mon[29180]: pgmap v12247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21782.317029 secs 2024-04-27T00:38:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21782.317029 secs 2024-04-27T00:38:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21782.317029 secs 2024-04-27T00:38:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:21 smithi119 ceph-mon[32752]: pgmap v12248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21783.112950 secs 2024-04-27T00:38:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:21 smithi167 ceph-mon[32751]: pgmap v12248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21783.112950 secs 2024-04-27T00:38:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:21 smithi090 ceph-mon[29180]: pgmap v12248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21783.112950 secs 2024-04-27T00:38:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:23 smithi119 ceph-mon[32752]: pgmap v12249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:23 smithi167 ceph-mon[32751]: pgmap v12249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:23 smithi090 ceph-mon[29180]: pgmap v12249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:25 smithi119 ceph-mon[32752]: pgmap v12250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21787.317115 secs 2024-04-27T00:38:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:25 smithi167 ceph-mon[32751]: pgmap v12250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21787.317115 secs 2024-04-27T00:38:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:25 smithi090 ceph-mon[29180]: pgmap v12250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21787.317115 secs 2024-04-27T00:38:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21788.113029 secs 2024-04-27T00:38:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21788.113029 secs 2024-04-27T00:38:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21788.113029 secs 2024-04-27T00:38:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:27 smithi119 ceph-mon[32752]: pgmap v12251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:27 smithi167 ceph-mon[32751]: pgmap v12251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:27 smithi090 ceph-mon[29180]: pgmap v12251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:29 smithi119 ceph-mon[32752]: pgmap v12252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:29 smithi167 ceph-mon[32751]: pgmap v12252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:29 smithi090 ceph-mon[29180]: pgmap v12252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21792.317201 secs 2024-04-27T00:38:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21792.317201 secs 2024-04-27T00:38:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21792.317201 secs 2024-04-27T00:38:32.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:31 smithi090 ceph-mon[29180]: pgmap v12253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:32.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21793.113163 secs 2024-04-27T00:38:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:31 smithi119 ceph-mon[32752]: pgmap v12253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21793.113163 secs 2024-04-27T00:38:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:31 smithi167 ceph-mon[32751]: pgmap v12253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21793.113163 secs 2024-04-27T00:38:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:33 smithi119 ceph-mon[32752]: pgmap v12254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:33 smithi167 ceph-mon[32751]: pgmap v12254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:33 smithi090 ceph-mon[29180]: pgmap v12254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:35 smithi119 ceph-mon[32752]: pgmap v12255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21797.317260 secs 2024-04-27T00:38:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:35 smithi167 ceph-mon[32751]: pgmap v12255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21797.317260 secs 2024-04-27T00:38:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:35 smithi090 ceph-mon[29180]: pgmap v12255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21797.317260 secs 2024-04-27T00:38:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21798.113244 secs 2024-04-27T00:38:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:38:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21798.113244 secs 2024-04-27T00:38:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:38:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21798.113244 secs 2024-04-27T00:38:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:38:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:37 smithi119 ceph-mon[32752]: pgmap v12256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:37 smithi119 ceph-mon[32752]: pgmap v12257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:37 smithi119 ceph-mon[32752]: pgmap v12258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:38:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:38:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:37 smithi167 ceph-mon[32751]: pgmap v12256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:37 smithi167 ceph-mon[32751]: pgmap v12257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:37 smithi167 ceph-mon[32751]: pgmap v12258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:38:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:37 smithi090 ceph-mon[29180]: pgmap v12256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:37 smithi090 ceph-mon[29180]: pgmap v12257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:37 smithi090 ceph-mon[29180]: pgmap v12258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:38:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:38:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:40 smithi119 ceph-mon[32752]: pgmap v12259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:40.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:40 smithi167 ceph-mon[32751]: pgmap v12259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:40.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:40 smithi090 ceph-mon[29180]: pgmap v12259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21802.317346 secs 2024-04-27T00:38:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21802.317346 secs 2024-04-27T00:38:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21802.317346 secs 2024-04-27T00:38:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:42 smithi119 ceph-mon[32752]: 22 slow requests, 1 included below; oldest blocked for > 21803.113305 secs 2024-04-27T00:38:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:42 smithi119 ceph-mon[32752]: slow request 364.156054 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:38:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:42 smithi119 ceph-mon[32752]: pgmap v12260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:42 smithi167 ceph-mon[32751]: 22 slow requests, 1 included below; oldest blocked for > 21803.113305 secs 2024-04-27T00:38:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:42 smithi167 ceph-mon[32751]: slow request 364.156054 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:38:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:42 smithi167 ceph-mon[32751]: pgmap v12260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:42 smithi090 ceph-mon[29180]: 22 slow requests, 1 included below; oldest blocked for > 21803.113305 secs 2024-04-27T00:38:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:42 smithi090 ceph-mon[29180]: slow request 364.156054 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:38:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:42 smithi090 ceph-mon[29180]: pgmap v12260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:44 smithi119 ceph-mon[32752]: pgmap v12261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:38:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:44 smithi167 ceph-mon[32751]: pgmap v12261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:38:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:44 smithi090 ceph-mon[29180]: pgmap v12261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:38:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21807.317430 secs 2024-04-27T00:38:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:46 smithi119 ceph-mon[32752]: pgmap v12262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21807.317430 secs 2024-04-27T00:38:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:46 smithi167 ceph-mon[32751]: pgmap v12262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21807.317430 secs 2024-04-27T00:38:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:46 smithi090 ceph-mon[29180]: pgmap v12262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:38:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:47 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21808.113424 secs 2024-04-27T00:38:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:47 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21808.113424 secs 2024-04-27T00:38:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:47 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21808.113424 secs 2024-04-27T00:38:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:48 smithi119 ceph-mon[32752]: pgmap v12263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:38:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:48 smithi167 ceph-mon[32751]: pgmap v12263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:38:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:48 smithi090 ceph-mon[29180]: pgmap v12263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:38:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:50 smithi119 ceph-mon[32752]: pgmap v12264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:50 smithi167 ceph-mon[32751]: pgmap v12264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:50 smithi090 ceph-mon[29180]: pgmap v12264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21812.317516 secs 2024-04-27T00:38:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21812.317516 secs 2024-04-27T00:38:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21812.317516 secs 2024-04-27T00:38:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:52 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21813.113538 secs 2024-04-27T00:38:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:52 smithi119 ceph-mon[32752]: pgmap v12265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:52 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21813.113538 secs 2024-04-27T00:38:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:52 smithi167 ceph-mon[32751]: pgmap v12265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:52 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21813.113538 secs 2024-04-27T00:38:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:52 smithi090 ceph-mon[29180]: pgmap v12265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:54 smithi119 ceph-mon[32752]: pgmap v12266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:54 smithi167 ceph-mon[32751]: pgmap v12266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:54 smithi090 ceph-mon[29180]: pgmap v12266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21817.317600 secs 2024-04-27T00:38:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21817.317600 secs 2024-04-27T00:38:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21817.317600 secs 2024-04-27T00:38:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:56 smithi167 ceph-mon[32751]: pgmap v12267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:56 smithi090 ceph-mon[29180]: pgmap v12267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:56 smithi119 ceph-mon[32752]: pgmap v12267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:38:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:57 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21818.113654 secs 2024-04-27T00:38:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:57 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21818.113654 secs 2024-04-27T00:38:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:57 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21818.113654 secs 2024-04-27T00:38:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:38:58 smithi090 ceph-mon[29180]: pgmap v12268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:38:58 smithi119 ceph-mon[32752]: pgmap v12268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:38:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:38:58 smithi167 ceph-mon[32751]: pgmap v12268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:00 smithi090 ceph-mon[29180]: pgmap v12269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21822.317685 secs 2024-04-27T00:39:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:00 smithi119 ceph-mon[32752]: pgmap v12269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21822.317685 secs 2024-04-27T00:39:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:00 smithi167 ceph-mon[32751]: pgmap v12269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21822.317685 secs 2024-04-27T00:39:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:02 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21823.113729 secs 2024-04-27T00:39:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:02 smithi090 ceph-mon[29180]: pgmap v12270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:02 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21823.113729 secs 2024-04-27T00:39:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:02 smithi119 ceph-mon[32752]: pgmap v12270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:02 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21823.113729 secs 2024-04-27T00:39:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:02 smithi167 ceph-mon[32751]: pgmap v12270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:04 smithi090 ceph-mon[29180]: pgmap v12271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:04 smithi119 ceph-mon[32752]: pgmap v12271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:04 smithi167 ceph-mon[32751]: pgmap v12271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21827.317770 secs 2024-04-27T00:39:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21827.317770 secs 2024-04-27T00:39:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21827.317770 secs 2024-04-27T00:39:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:06 smithi090 ceph-mon[29180]: pgmap v12272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:06.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:06 smithi119 ceph-mon[32752]: pgmap v12272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:06 smithi167 ceph-mon[32751]: pgmap v12272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:07 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21828.113790 secs 2024-04-27T00:39:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:07 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21828.113790 secs 2024-04-27T00:39:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:07 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21828.113790 secs 2024-04-27T00:39:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:08 smithi090 ceph-mon[29180]: pgmap v12273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:08 smithi119 ceph-mon[32752]: pgmap v12273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:08 smithi167 ceph-mon[32751]: pgmap v12273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:10 smithi090 ceph-mon[29180]: pgmap v12274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:10 smithi090 ceph-mon[29180]: 31 slow requests, 2 included below; oldest blocked for > 21832.317826 secs 2024-04-27T00:39:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:10 smithi090 ceph-mon[29180]: slow request 5764.945659 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:39:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:10 smithi090 ceph-mon[29180]: slow request 1442.522862 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:39:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:10 smithi119 ceph-mon[32752]: pgmap v12274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:10 smithi119 ceph-mon[32752]: 31 slow requests, 2 included below; oldest blocked for > 21832.317826 secs 2024-04-27T00:39:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:10 smithi119 ceph-mon[32752]: slow request 5764.945659 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:39:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:10 smithi119 ceph-mon[32752]: slow request 1442.522862 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:39:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:10 smithi167 ceph-mon[32751]: pgmap v12274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:10 smithi167 ceph-mon[32751]: 31 slow requests, 2 included below; oldest blocked for > 21832.317826 secs 2024-04-27T00:39:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:10 smithi167 ceph-mon[32751]: slow request 5764.945659 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:39:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:10 smithi167 ceph-mon[32751]: slow request 1442.522862 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:39:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:12 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21833.113869 secs 2024-04-27T00:39:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:12 smithi090 ceph-mon[29180]: pgmap v12275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:12 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21833.113869 secs 2024-04-27T00:39:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:12 smithi119 ceph-mon[32752]: pgmap v12275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:12 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21833.113869 secs 2024-04-27T00:39:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:12 smithi167 ceph-mon[32751]: pgmap v12275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:14 smithi090 ceph-mon[29180]: pgmap v12276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:14.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:14 smithi119 ceph-mon[32752]: pgmap v12276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:14 smithi167 ceph-mon[32751]: pgmap v12276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21837.317885 secs 2024-04-27T00:39:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21837.317885 secs 2024-04-27T00:39:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21837.317885 secs 2024-04-27T00:39:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:16 smithi090 ceph-mon[29180]: pgmap v12277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:16 smithi119 ceph-mon[32752]: pgmap v12277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:16 smithi167 ceph-mon[32751]: pgmap v12277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:17 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21838.113951 secs 2024-04-27T00:39:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:17 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21838.113951 secs 2024-04-27T00:39:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:17 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21838.113951 secs 2024-04-27T00:39:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:18 smithi090 ceph-mon[29180]: pgmap v12278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:18 smithi119 ceph-mon[32752]: pgmap v12278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:18 smithi167 ceph-mon[32751]: pgmap v12278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:20 smithi090 ceph-mon[29180]: pgmap v12279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21842.317949 secs 2024-04-27T00:39:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:20 smithi119 ceph-mon[32752]: pgmap v12279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21842.317949 secs 2024-04-27T00:39:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:20 smithi167 ceph-mon[32751]: pgmap v12279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21842.317949 secs 2024-04-27T00:39:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:22 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21843.114079 secs 2024-04-27T00:39:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:22 smithi119 ceph-mon[32752]: pgmap v12280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:22 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21843.114079 secs 2024-04-27T00:39:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:22 smithi167 ceph-mon[32751]: pgmap v12280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:22 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21843.114079 secs 2024-04-27T00:39:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:22 smithi090 ceph-mon[29180]: pgmap v12280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:24 smithi119 ceph-mon[32752]: pgmap v12281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:24 smithi167 ceph-mon[32751]: pgmap v12281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:24 smithi090 ceph-mon[29180]: pgmap v12281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21847.318036 secs 2024-04-27T00:39:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21847.318036 secs 2024-04-27T00:39:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21847.318036 secs 2024-04-27T00:39:26.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:26 smithi167 ceph-mon[32751]: pgmap v12282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:26 smithi119 ceph-mon[32752]: pgmap v12282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:26 smithi090 ceph-mon[29180]: pgmap v12282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:27 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21848.114174 secs 2024-04-27T00:39:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:27 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21848.114174 secs 2024-04-27T00:39:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:27 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21848.114174 secs 2024-04-27T00:39:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:28 smithi119 ceph-mon[32752]: pgmap v12283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:28 smithi167 ceph-mon[32751]: pgmap v12283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:28 smithi090 ceph-mon[29180]: pgmap v12283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:30 smithi119 ceph-mon[32752]: pgmap v12284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21852.318122 secs 2024-04-27T00:39:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:30 smithi167 ceph-mon[32751]: pgmap v12284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21852.318122 secs 2024-04-27T00:39:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:30 smithi090 ceph-mon[29180]: pgmap v12284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21852.318122 secs 2024-04-27T00:39:32.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:32 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21853.114292 secs 2024-04-27T00:39:32.677 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:32 smithi090 ceph-mon[29180]: pgmap v12285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:32 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21853.114292 secs 2024-04-27T00:39:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:32 smithi119 ceph-mon[32752]: pgmap v12285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:32 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21853.114292 secs 2024-04-27T00:39:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:32 smithi167 ceph-mon[32751]: pgmap v12285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:34 smithi119 ceph-mon[32752]: pgmap v12286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:34 smithi167 ceph-mon[32751]: pgmap v12286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:34.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:34 smithi090 ceph-mon[29180]: pgmap v12286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21857.318181 secs 2024-04-27T00:39:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21857.318181 secs 2024-04-27T00:39:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21857.318181 secs 2024-04-27T00:39:36.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:36 smithi119 ceph-mon[32752]: pgmap v12287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:36 smithi167 ceph-mon[32751]: pgmap v12287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:36 smithi090 ceph-mon[29180]: pgmap v12287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:37 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21858.114438 secs 2024-04-27T00:39:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:39:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:37 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21858.114438 secs 2024-04-27T00:39:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:39:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:37 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21858.114438 secs 2024-04-27T00:39:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:39:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:38 smithi119 ceph-mon[32752]: pgmap v12288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:39:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:39:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:39:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:38 smithi167 ceph-mon[32751]: pgmap v12288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:39:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:39:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:39:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:38 smithi090 ceph-mon[29180]: pgmap v12288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:39:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:39:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:39:39.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:39 smithi090 ceph-mon[29180]: pgmap v12289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:39:39.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:39 smithi090 ceph-mon[29180]: pgmap v12290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:39:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:39 smithi119 ceph-mon[32752]: pgmap v12289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:39:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:39 smithi119 ceph-mon[32752]: pgmap v12290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:39:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:39 smithi167 ceph-mon[32751]: pgmap v12289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:39:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:39 smithi167 ceph-mon[32751]: pgmap v12290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:39:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21862.318268 secs 2024-04-27T00:39:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21862.318268 secs 2024-04-27T00:39:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21862.318268 secs 2024-04-27T00:39:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:41 smithi119 ceph-mon[32752]: pgmap v12291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:41 smithi167 ceph-mon[32751]: pgmap v12291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:41 smithi090 ceph-mon[29180]: pgmap v12291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:42 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21863.114514 secs 2024-04-27T00:39:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:42 smithi119 ceph-mon[32752]: pgmap v12292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:42 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21863.114514 secs 2024-04-27T00:39:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:42 smithi167 ceph-mon[32751]: pgmap v12292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:42 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21863.114514 secs 2024-04-27T00:39:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:42 smithi090 ceph-mon[29180]: pgmap v12292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:45 smithi119 ceph-mon[32752]: pgmap v12293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:45 smithi167 ceph-mon[32751]: pgmap v12293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:45 smithi090 ceph-mon[29180]: pgmap v12293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21867.318353 secs 2024-04-27T00:39:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21867.318353 secs 2024-04-27T00:39:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21867.318353 secs 2024-04-27T00:39:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:47 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21868.114591 secs 2024-04-27T00:39:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:47 smithi167 ceph-mon[32751]: pgmap v12294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:47 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21868.114591 secs 2024-04-27T00:39:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:47 smithi090 ceph-mon[29180]: pgmap v12294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:47 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21868.114591 secs 2024-04-27T00:39:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:47 smithi119 ceph-mon[32752]: pgmap v12294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:39:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:49 smithi167 ceph-mon[32751]: pgmap v12295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:39:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:49 smithi090 ceph-mon[29180]: pgmap v12295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:39:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:49 smithi119 ceph-mon[32752]: pgmap v12295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:39:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:51 smithi167 ceph-mon[32751]: pgmap v12296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21872.318437 secs 2024-04-27T00:39:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:51 smithi090 ceph-mon[29180]: pgmap v12296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21872.318437 secs 2024-04-27T00:39:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:51 smithi119 ceph-mon[32752]: pgmap v12296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21872.318437 secs 2024-04-27T00:39:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:52 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21873.114703 secs 2024-04-27T00:39:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:52 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21873.114703 secs 2024-04-27T00:39:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:52 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21873.114703 secs 2024-04-27T00:39:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:53 smithi090 ceph-mon[29180]: pgmap v12297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:53 smithi119 ceph-mon[32752]: pgmap v12297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:53 smithi167 ceph-mon[32751]: pgmap v12297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:55 smithi090 ceph-mon[29180]: pgmap v12298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:55 smithi119 ceph-mon[32752]: pgmap v12298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:55 smithi167 ceph-mon[32751]: pgmap v12298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:39:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21877.318497 secs 2024-04-27T00:39:56.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21877.318497 secs 2024-04-27T00:39:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21877.318497 secs 2024-04-27T00:39:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:57 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21878.114819 secs 2024-04-27T00:39:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:57 smithi090 ceph-mon[29180]: pgmap v12299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:57 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21878.114819 secs 2024-04-27T00:39:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:57 smithi119 ceph-mon[32752]: pgmap v12299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:57 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21878.114819 secs 2024-04-27T00:39:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:57 smithi167 ceph-mon[32751]: pgmap v12299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:39:59 smithi090 ceph-mon[29180]: pgmap v12300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:39:59 smithi119 ceph-mon[32752]: pgmap v12300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:39:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:39:59 smithi167 ceph-mon[32751]: pgmap v12300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:40:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:40:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:40:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:01 smithi090 ceph-mon[29180]: pgmap v12301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21882.318585 secs 2024-04-27T00:40:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:01 smithi119 ceph-mon[32752]: pgmap v12301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21882.318585 secs 2024-04-27T00:40:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:01 smithi167 ceph-mon[32751]: pgmap v12301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21882.318585 secs 2024-04-27T00:40:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:02 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21883.114912 secs 2024-04-27T00:40:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:02 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21883.114912 secs 2024-04-27T00:40:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:02 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21883.114912 secs 2024-04-27T00:40:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:03 smithi090 ceph-mon[29180]: pgmap v12302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:03 smithi119 ceph-mon[32752]: pgmap v12302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:03 smithi167 ceph-mon[32751]: pgmap v12302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:05 smithi090 ceph-mon[29180]: pgmap v12303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:05 smithi119 ceph-mon[32752]: pgmap v12303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:05 smithi167 ceph-mon[32751]: pgmap v12303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21887.318672 secs 2024-04-27T00:40:06.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21887.318672 secs 2024-04-27T00:40:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21887.318672 secs 2024-04-27T00:40:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:07 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21888.115047 secs 2024-04-27T00:40:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:07 smithi090 ceph-mon[29180]: pgmap v12304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:07 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21888.115047 secs 2024-04-27T00:40:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:07 smithi119 ceph-mon[32752]: pgmap v12304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:07 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21888.115047 secs 2024-04-27T00:40:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:07 smithi167 ceph-mon[32751]: pgmap v12304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:09 smithi090 ceph-mon[29180]: pgmap v12305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:09 smithi119 ceph-mon[32752]: pgmap v12305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:09 smithi167 ceph-mon[32751]: pgmap v12305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:11 smithi090 ceph-mon[29180]: pgmap v12306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21892.318758 secs 2024-04-27T00:40:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:11 smithi119 ceph-mon[32752]: pgmap v12306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21892.318758 secs 2024-04-27T00:40:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:11 smithi167 ceph-mon[32751]: pgmap v12306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21892.318758 secs 2024-04-27T00:40:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:12 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21893.115166 secs 2024-04-27T00:40:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:12 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21893.115166 secs 2024-04-27T00:40:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:12 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21893.115166 secs 2024-04-27T00:40:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:13 smithi090 ceph-mon[29180]: pgmap v12307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:13 smithi119 ceph-mon[32752]: pgmap v12307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:13 smithi167 ceph-mon[32751]: pgmap v12307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:15.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:15 smithi090 ceph-mon[29180]: pgmap v12308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:15 smithi119 ceph-mon[32752]: pgmap v12308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:15 smithi167 ceph-mon[32751]: pgmap v12308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21897.318843 secs 2024-04-27T00:40:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21897.318843 secs 2024-04-27T00:40:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21897.318843 secs 2024-04-27T00:40:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:17 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21898.115307 secs 2024-04-27T00:40:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:17 smithi119 ceph-mon[32752]: pgmap v12309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:17 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21898.115307 secs 2024-04-27T00:40:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:17 smithi167 ceph-mon[32751]: pgmap v12309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:17 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21898.115307 secs 2024-04-27T00:40:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:17 smithi090 ceph-mon[29180]: pgmap v12309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:19 smithi119 ceph-mon[32752]: pgmap v12310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:19 smithi167 ceph-mon[32751]: pgmap v12310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:19 smithi090 ceph-mon[29180]: pgmap v12310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:21 smithi119 ceph-mon[32752]: pgmap v12311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21902.318907 secs 2024-04-27T00:40:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:21 smithi167 ceph-mon[32751]: pgmap v12311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21902.318907 secs 2024-04-27T00:40:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:21 smithi090 ceph-mon[29180]: pgmap v12311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21902.318907 secs 2024-04-27T00:40:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:22 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21903.115442 secs 2024-04-27T00:40:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:22 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21903.115442 secs 2024-04-27T00:40:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:22 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21903.115442 secs 2024-04-27T00:40:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:23 smithi119 ceph-mon[32752]: pgmap v12312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:23 smithi167 ceph-mon[32751]: pgmap v12312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:23 smithi090 ceph-mon[29180]: pgmap v12312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:25 smithi119 ceph-mon[32752]: pgmap v12313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:25 smithi167 ceph-mon[32751]: pgmap v12313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:25 smithi090 ceph-mon[29180]: pgmap v12313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:26.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21907.318992 secs 2024-04-27T00:40:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21907.318992 secs 2024-04-27T00:40:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21907.318992 secs 2024-04-27T00:40:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:27 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21908.115557 secs 2024-04-27T00:40:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:27 smithi119 ceph-mon[32752]: pgmap v12314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:27 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21908.115557 secs 2024-04-27T00:40:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:27 smithi167 ceph-mon[32751]: pgmap v12314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:27 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21908.115557 secs 2024-04-27T00:40:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:27 smithi090 ceph-mon[29180]: pgmap v12314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:29.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:29 smithi090 ceph-mon[29180]: pgmap v12315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:29 smithi119 ceph-mon[32752]: pgmap v12315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:29 smithi167 ceph-mon[32751]: pgmap v12315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:31 smithi119 ceph-mon[32752]: pgmap v12316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21912.319096 secs 2024-04-27T00:40:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:31 smithi167 ceph-mon[32751]: pgmap v12316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21912.319096 secs 2024-04-27T00:40:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:31 smithi090 ceph-mon[29180]: pgmap v12316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21912.319096 secs 2024-04-27T00:40:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:32 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21913.115673 secs 2024-04-27T00:40:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:32 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21913.115673 secs 2024-04-27T00:40:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:32 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21913.115673 secs 2024-04-27T00:40:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:33 smithi119 ceph-mon[32752]: pgmap v12317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:33 smithi167 ceph-mon[32751]: pgmap v12317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:33 smithi090 ceph-mon[29180]: pgmap v12317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:35 smithi119 ceph-mon[32752]: pgmap v12318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:35 smithi167 ceph-mon[32751]: pgmap v12318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:35 smithi090 ceph-mon[29180]: pgmap v12318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:36.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21917.319177 secs 2024-04-27T00:40:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21917.319177 secs 2024-04-27T00:40:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21917.319177 secs 2024-04-27T00:40:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:37 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21918.115784 secs 2024-04-27T00:40:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:37 smithi119 ceph-mon[32752]: pgmap v12319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:37 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21918.115784 secs 2024-04-27T00:40:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:37 smithi167 ceph-mon[32751]: pgmap v12319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:37 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21918.115784 secs 2024-04-27T00:40:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:37 smithi090 ceph-mon[29180]: pgmap v12319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:40:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:40:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:40:39.489 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:39 smithi090 ceph-mon[29180]: pgmap v12320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:39.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.490 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:39 smithi119 ceph-mon[32752]: pgmap v12320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:39 smithi167 ceph-mon[32751]: pgmap v12320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: pgmap v12321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21922.319242 secs 2024-04-27T00:40:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: pgmap v12321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21922.319242 secs 2024-04-27T00:40:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: pgmap v12321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21922.319242 secs 2024-04-27T00:40:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21923.115874 secs 2024-04-27T00:40:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:40:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:40:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:40:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:40:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21923.115874 secs 2024-04-27T00:40:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:40:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:40:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:40:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:40:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21923.115874 secs 2024-04-27T00:40:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:40:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:40:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:40:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:40:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:40:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:43 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:40:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:43 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:40:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:43 smithi167 ceph-mon[32751]: pgmap v12322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 893 B/s rd, 1 op/s 2024-04-27T00:40:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:43 smithi167 ceph-mon[32751]: pgmap v12323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:43 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:40:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:43 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:40:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:43 smithi090 ceph-mon[29180]: pgmap v12322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 893 B/s rd, 1 op/s 2024-04-27T00:40:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:43 smithi090 ceph-mon[29180]: pgmap v12323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:43 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:40:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:43 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:40:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:43 smithi119 ceph-mon[32752]: pgmap v12322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 893 B/s rd, 1 op/s 2024-04-27T00:40:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:43 smithi119 ceph-mon[32752]: pgmap v12323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:45 smithi167 ceph-mon[32751]: pgmap v12324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21927.319327 secs 2024-04-27T00:40:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:45 smithi090 ceph-mon[29180]: pgmap v12324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21927.319327 secs 2024-04-27T00:40:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:45 smithi119 ceph-mon[32752]: pgmap v12324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21927.319327 secs 2024-04-27T00:40:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:47 smithi090 ceph-mon[29180]: pgmap v12325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:47 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21928.115993 secs 2024-04-27T00:40:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:47 smithi119 ceph-mon[32752]: pgmap v12325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:47 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21928.115993 secs 2024-04-27T00:40:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:47 smithi167 ceph-mon[32751]: pgmap v12325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:47 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21928.115993 secs 2024-04-27T00:40:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:49 smithi090 ceph-mon[29180]: pgmap v12326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:49 smithi119 ceph-mon[32752]: pgmap v12326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:49 smithi167 ceph-mon[32751]: pgmap v12326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21932.319413 secs 2024-04-27T00:40:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21932.319413 secs 2024-04-27T00:40:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21932.319413 secs 2024-04-27T00:40:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:51 smithi090 ceph-mon[29180]: pgmap v12327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:51 smithi119 ceph-mon[32752]: pgmap v12327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:51 smithi167 ceph-mon[32751]: pgmap v12327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:40:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:52 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21933.116109 secs 2024-04-27T00:40:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:52 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21933.116109 secs 2024-04-27T00:40:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:52 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21933.116109 secs 2024-04-27T00:40:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:53 smithi090 ceph-mon[29180]: pgmap v12328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:40:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:53 smithi119 ceph-mon[32752]: pgmap v12328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:40:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:53 smithi167 ceph-mon[32751]: pgmap v12328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:40:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:55 smithi090 ceph-mon[29180]: pgmap v12329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21937.319499 secs 2024-04-27T00:40:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:55 smithi119 ceph-mon[32752]: pgmap v12329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21937.319499 secs 2024-04-27T00:40:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:55 smithi167 ceph-mon[32751]: pgmap v12329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:40:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21937.319499 secs 2024-04-27T00:40:57.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:57 smithi167 ceph-mon[32751]: pgmap v12330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:57.620 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:57 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21938.116241 secs 2024-04-27T00:40:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:57 smithi090 ceph-mon[29180]: pgmap v12330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:57 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21938.116241 secs 2024-04-27T00:40:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:57 smithi119 ceph-mon[32752]: pgmap v12330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:57 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21938.116241 secs 2024-04-27T00:40:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:40:59 smithi090 ceph-mon[29180]: pgmap v12331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:40:59 smithi119 ceph-mon[32752]: pgmap v12331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:40:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:40:59 smithi167 ceph-mon[32751]: pgmap v12331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21942.319584 secs 2024-04-27T00:41:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21942.319584 secs 2024-04-27T00:41:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21942.319584 secs 2024-04-27T00:41:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:01 smithi090 ceph-mon[29180]: pgmap v12332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:01 smithi119 ceph-mon[32752]: pgmap v12332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:01 smithi167 ceph-mon[32751]: pgmap v12332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:02 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21943.116354 secs 2024-04-27T00:41:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:02 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21943.116354 secs 2024-04-27T00:41:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:02 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21943.116354 secs 2024-04-27T00:41:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:03 smithi090 ceph-mon[29180]: pgmap v12333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:03 smithi119 ceph-mon[32752]: pgmap v12333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:03 smithi167 ceph-mon[32751]: pgmap v12333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:05 smithi090 ceph-mon[29180]: pgmap v12334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21947.319671 secs 2024-04-27T00:41:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:05 smithi119 ceph-mon[32752]: pgmap v12334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21947.319671 secs 2024-04-27T00:41:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:05 smithi167 ceph-mon[32751]: pgmap v12334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21947.319671 secs 2024-04-27T00:41:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:07 smithi090 ceph-mon[29180]: pgmap v12335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:07 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21948.116419 secs 2024-04-27T00:41:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:07 smithi119 ceph-mon[32752]: pgmap v12335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:07 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21948.116419 secs 2024-04-27T00:41:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:07 smithi167 ceph-mon[32751]: pgmap v12335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:07 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21948.116419 secs 2024-04-27T00:41:09.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:09 smithi090 ceph-mon[29180]: pgmap v12336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:09 smithi119 ceph-mon[32752]: pgmap v12336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:09 smithi167 ceph-mon[32751]: pgmap v12336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21952.319756 secs 2024-04-27T00:41:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21952.319756 secs 2024-04-27T00:41:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21952.319756 secs 2024-04-27T00:41:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:11 smithi119 ceph-mon[32752]: pgmap v12337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:11 smithi167 ceph-mon[32751]: pgmap v12337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:11 smithi090 ceph-mon[29180]: pgmap v12337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:12.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:12 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21953.116490 secs 2024-04-27T00:41:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:12 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21953.116490 secs 2024-04-27T00:41:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:12 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21953.116490 secs 2024-04-27T00:41:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:13 smithi119 ceph-mon[32752]: pgmap v12338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:13 smithi167 ceph-mon[32751]: pgmap v12338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:13.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:13 smithi090 ceph-mon[29180]: pgmap v12338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:15 smithi119 ceph-mon[32752]: pgmap v12339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21957.319841 secs 2024-04-27T00:41:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:15 smithi167 ceph-mon[32751]: pgmap v12339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21957.319841 secs 2024-04-27T00:41:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:15 smithi090 ceph-mon[29180]: pgmap v12339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21957.319841 secs 2024-04-27T00:41:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:17 smithi119 ceph-mon[32752]: pgmap v12340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:17 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21958.116607 secs 2024-04-27T00:41:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:17 smithi167 ceph-mon[32751]: pgmap v12340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:17 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21958.116607 secs 2024-04-27T00:41:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:17 smithi090 ceph-mon[29180]: pgmap v12340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:17 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21958.116607 secs 2024-04-27T00:41:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:19 smithi119 ceph-mon[32752]: pgmap v12341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:19 smithi167 ceph-mon[32751]: pgmap v12341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:19 smithi090 ceph-mon[29180]: pgmap v12341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21962.319932 secs 2024-04-27T00:41:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21962.319932 secs 2024-04-27T00:41:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21962.319932 secs 2024-04-27T00:41:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:21 smithi119 ceph-mon[32752]: pgmap v12342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:21 smithi167 ceph-mon[32751]: pgmap v12342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:21 smithi090 ceph-mon[29180]: pgmap v12342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:22 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21963.116674 secs 2024-04-27T00:41:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:22 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21963.116674 secs 2024-04-27T00:41:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:22 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21963.116674 secs 2024-04-27T00:41:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:23 smithi119 ceph-mon[32752]: pgmap v12343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:23 smithi167 ceph-mon[32751]: pgmap v12343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:23 smithi090 ceph-mon[29180]: pgmap v12343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:25 smithi119 ceph-mon[32752]: pgmap v12344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21967.320023 secs 2024-04-27T00:41:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:25 smithi167 ceph-mon[32751]: pgmap v12344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21967.320023 secs 2024-04-27T00:41:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:25 smithi090 ceph-mon[29180]: pgmap v12344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21967.320023 secs 2024-04-27T00:41:27.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:27 smithi090 ceph-mon[29180]: pgmap v12345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:27.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:27 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21968.116792 secs 2024-04-27T00:41:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:27 smithi119 ceph-mon[32752]: pgmap v12345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:27 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21968.116792 secs 2024-04-27T00:41:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:27 smithi167 ceph-mon[32751]: pgmap v12345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:27 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21968.116792 secs 2024-04-27T00:41:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:29 smithi119 ceph-mon[32752]: pgmap v12346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:29 smithi167 ceph-mon[32751]: pgmap v12346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:29 smithi090 ceph-mon[29180]: pgmap v12346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21972.320110 secs 2024-04-27T00:41:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21972.320110 secs 2024-04-27T00:41:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21972.320110 secs 2024-04-27T00:41:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:31 smithi119 ceph-mon[32752]: pgmap v12347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:31 smithi167 ceph-mon[32751]: pgmap v12347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:31 smithi090 ceph-mon[29180]: pgmap v12347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:32 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21973.116902 secs 2024-04-27T00:41:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:32 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21973.116902 secs 2024-04-27T00:41:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:32 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21973.116902 secs 2024-04-27T00:41:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:33 smithi119 ceph-mon[32752]: pgmap v12348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:33 smithi167 ceph-mon[32751]: pgmap v12348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:33 smithi090 ceph-mon[29180]: pgmap v12348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:35 smithi119 ceph-mon[32752]: pgmap v12349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21977.320171 secs 2024-04-27T00:41:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:35 smithi167 ceph-mon[32751]: pgmap v12349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21977.320171 secs 2024-04-27T00:41:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:35 smithi090 ceph-mon[29180]: pgmap v12349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21977.320171 secs 2024-04-27T00:41:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:37 smithi119 ceph-mon[32752]: pgmap v12350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:37 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21978.117011 secs 2024-04-27T00:41:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:37 smithi167 ceph-mon[32751]: pgmap v12350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:37 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21978.117011 secs 2024-04-27T00:41:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:37 smithi090 ceph-mon[29180]: pgmap v12350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:37 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21978.117011 secs 2024-04-27T00:41:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:39 smithi119 ceph-mon[32752]: pgmap v12351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:39 smithi167 ceph-mon[32751]: pgmap v12351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:39 smithi090 ceph-mon[29180]: pgmap v12351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21982.320245 secs 2024-04-27T00:41:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21982.320245 secs 2024-04-27T00:41:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21982.320245 secs 2024-04-27T00:41:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:41 smithi119 ceph-mon[32752]: pgmap v12352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:41 smithi167 ceph-mon[32751]: pgmap v12352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:41 smithi090 ceph-mon[29180]: pgmap v12352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:42 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21983.117125 secs 2024-04-27T00:41:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:41:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:42 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21983.117125 secs 2024-04-27T00:41:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:41:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:42 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21983.117125 secs 2024-04-27T00:41:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:41:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:43.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:43 smithi090 ceph-mon[29180]: pgmap v12353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:43.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:41:43.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:41:43.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:43 smithi090 ceph-mon[29180]: pgmap v12354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T00:41:43.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:43 smithi090 ceph-mon[29180]: pgmap v12355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T00:41:43.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:43 smithi119 ceph-mon[32752]: pgmap v12353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:43 smithi119 ceph-mon[32752]: pgmap v12354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T00:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:43 smithi119 ceph-mon[32752]: pgmap v12355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T00:41:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:43 smithi167 ceph-mon[32751]: pgmap v12353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:43 smithi167 ceph-mon[32751]: pgmap v12354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T00:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:43 smithi167 ceph-mon[32751]: pgmap v12355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T00:41:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:41:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:45 smithi090 ceph-mon[29180]: pgmap v12356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21987.320302 secs 2024-04-27T00:41:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:45 smithi119 ceph-mon[32752]: pgmap v12356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21987.320302 secs 2024-04-27T00:41:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:45 smithi167 ceph-mon[32751]: pgmap v12356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21987.320302 secs 2024-04-27T00:41:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:46 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21988.117248 secs 2024-04-27T00:41:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:46 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21988.117248 secs 2024-04-27T00:41:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:46 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21988.117248 secs 2024-04-27T00:41:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:47 smithi090 ceph-mon[29180]: pgmap v12357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:47 smithi119 ceph-mon[32752]: pgmap v12357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:47 smithi167 ceph-mon[32751]: pgmap v12357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:49 smithi090 ceph-mon[29180]: pgmap v12358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:49 smithi119 ceph-mon[32752]: pgmap v12358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:49 smithi167 ceph-mon[32751]: pgmap v12358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21992.320352 secs 2024-04-27T00:41:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21992.320352 secs 2024-04-27T00:41:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21992.320352 secs 2024-04-27T00:41:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:51 smithi090 ceph-mon[29180]: pgmap v12359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:51 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21993.117357 secs 2024-04-27T00:41:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:51 smithi119 ceph-mon[32752]: pgmap v12359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:51 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21993.117357 secs 2024-04-27T00:41:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:51 smithi167 ceph-mon[32751]: pgmap v12359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:41:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:51 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21993.117357 secs 2024-04-27T00:41:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:53 smithi090 ceph-mon[29180]: pgmap v12360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:41:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:53 smithi119 ceph-mon[32752]: pgmap v12360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:41:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:53 smithi167 ceph-mon[32751]: pgmap v12360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:41:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:55 smithi090 ceph-mon[29180]: pgmap v12361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 21997.320440 secs 2024-04-27T00:41:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:55 smithi119 ceph-mon[32752]: pgmap v12361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 21997.320440 secs 2024-04-27T00:41:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:55 smithi167 ceph-mon[32751]: pgmap v12361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:41:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 21997.320440 secs 2024-04-27T00:41:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:56 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 21998.117480 secs 2024-04-27T00:41:57.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:56 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 21998.117480 secs 2024-04-27T00:41:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:56 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 21998.117480 secs 2024-04-27T00:41:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:57 smithi090 ceph-mon[29180]: pgmap v12362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:57 smithi119 ceph-mon[32752]: pgmap v12362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:41:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:57 smithi167 ceph-mon[32751]: pgmap v12362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:41:59 smithi090 ceph-mon[29180]: pgmap v12363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:41:59 smithi119 ceph-mon[32752]: pgmap v12363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:41:59 smithi167 ceph-mon[32751]: pgmap v12363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22002.320528 secs 2024-04-27T00:42:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22002.320528 secs 2024-04-27T00:42:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22002.320528 secs 2024-04-27T00:42:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:01 smithi090 ceph-mon[29180]: pgmap v12364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:01 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22003.117551 secs 2024-04-27T00:42:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:01 smithi119 ceph-mon[32752]: pgmap v12364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:01 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22003.117551 secs 2024-04-27T00:42:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:01 smithi167 ceph-mon[32751]: pgmap v12364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:01 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22003.117551 secs 2024-04-27T00:42:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:03 smithi090 ceph-mon[29180]: pgmap v12365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:03 smithi119 ceph-mon[32752]: pgmap v12365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:03 smithi167 ceph-mon[32751]: pgmap v12365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:05 smithi090 ceph-mon[29180]: pgmap v12366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22007.320611 secs 2024-04-27T00:42:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:05 smithi119 ceph-mon[32752]: pgmap v12366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22007.320611 secs 2024-04-27T00:42:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:05 smithi167 ceph-mon[32751]: pgmap v12366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22007.320611 secs 2024-04-27T00:42:07.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:06 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22008.117632 secs 2024-04-27T00:42:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:06 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22008.117632 secs 2024-04-27T00:42:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:06 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22008.117632 secs 2024-04-27T00:42:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:07 smithi119 ceph-mon[32752]: pgmap v12367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:07 smithi167 ceph-mon[32751]: pgmap v12367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:08.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:07 smithi090 ceph-mon[29180]: pgmap v12367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:10.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:09 smithi119 ceph-mon[32752]: pgmap v12368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:09 smithi167 ceph-mon[32751]: pgmap v12368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:09 smithi090 ceph-mon[29180]: pgmap v12368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22012.320697 secs 2024-04-27T00:42:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22012.320697 secs 2024-04-27T00:42:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22012.320697 secs 2024-04-27T00:42:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:11 smithi119 ceph-mon[32752]: pgmap v12369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:11 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22013.117744 secs 2024-04-27T00:42:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:11 smithi167 ceph-mon[32751]: pgmap v12369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:11 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22013.117744 secs 2024-04-27T00:42:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:11 smithi090 ceph-mon[29180]: pgmap v12369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:11 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22013.117744 secs 2024-04-27T00:42:14.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:13 smithi119 ceph-mon[32752]: pgmap v12370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:14.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:13 smithi167 ceph-mon[32751]: pgmap v12370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:13 smithi090 ceph-mon[29180]: pgmap v12370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:15 smithi119 ceph-mon[32752]: pgmap v12371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22017.320781 secs 2024-04-27T00:42:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:15 smithi167 ceph-mon[32751]: pgmap v12371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22017.320781 secs 2024-04-27T00:42:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:15 smithi090 ceph-mon[29180]: pgmap v12371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22017.320781 secs 2024-04-27T00:42:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:16 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22018.117826 secs 2024-04-27T00:42:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:16 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22018.117826 secs 2024-04-27T00:42:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:16 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22018.117826 secs 2024-04-27T00:42:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:17 smithi119 ceph-mon[32752]: pgmap v12372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:17 smithi167 ceph-mon[32751]: pgmap v12372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:17 smithi090 ceph-mon[29180]: pgmap v12372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:19 smithi119 ceph-mon[32752]: pgmap v12373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:19 smithi167 ceph-mon[32751]: pgmap v12373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:19 smithi090 ceph-mon[29180]: pgmap v12373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22022.320870 secs 2024-04-27T00:42:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22022.320870 secs 2024-04-27T00:42:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22022.320870 secs 2024-04-27T00:42:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:21 smithi119 ceph-mon[32752]: pgmap v12374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:21 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22023.117905 secs 2024-04-27T00:42:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:21 smithi167 ceph-mon[32751]: pgmap v12374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:21 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22023.117905 secs 2024-04-27T00:42:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:21 smithi090 ceph-mon[29180]: pgmap v12374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:21 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22023.117905 secs 2024-04-27T00:42:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:23 smithi119 ceph-mon[32752]: pgmap v12375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:23 smithi167 ceph-mon[32751]: pgmap v12375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:23 smithi090 ceph-mon[29180]: pgmap v12375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:25 smithi119 ceph-mon[32752]: pgmap v12376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22027.320960 secs 2024-04-27T00:42:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:25 smithi167 ceph-mon[32751]: pgmap v12376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22027.320960 secs 2024-04-27T00:42:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:25 smithi090 ceph-mon[29180]: pgmap v12376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22027.320960 secs 2024-04-27T00:42:27.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:26 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22028.118041 secs 2024-04-27T00:42:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:26 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22028.118041 secs 2024-04-27T00:42:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:26 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22028.118041 secs 2024-04-27T00:42:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:27 smithi119 ceph-mon[32752]: pgmap v12377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:27 smithi167 ceph-mon[32751]: pgmap v12377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:27 smithi090 ceph-mon[29180]: pgmap v12377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:29 smithi119 ceph-mon[32752]: pgmap v12378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:29 smithi167 ceph-mon[32751]: pgmap v12378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:29 smithi090 ceph-mon[29180]: pgmap v12378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22032.321020 secs 2024-04-27T00:42:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22032.321020 secs 2024-04-27T00:42:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22032.321020 secs 2024-04-27T00:42:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:31 smithi119 ceph-mon[32752]: pgmap v12379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:31 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22033.118167 secs 2024-04-27T00:42:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:31 smithi167 ceph-mon[32751]: pgmap v12379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:31 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22033.118167 secs 2024-04-27T00:42:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:31 smithi090 ceph-mon[29180]: pgmap v12379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:31 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22033.118167 secs 2024-04-27T00:42:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:33 smithi119 ceph-mon[32752]: pgmap v12380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:33 smithi167 ceph-mon[32751]: pgmap v12380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:33 smithi090 ceph-mon[29180]: pgmap v12380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:35 smithi119 ceph-mon[32752]: pgmap v12381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22037.321080 secs 2024-04-27T00:42:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:35 smithi167 ceph-mon[32751]: pgmap v12381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22037.321080 secs 2024-04-27T00:42:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:35 smithi090 ceph-mon[29180]: pgmap v12381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22037.321080 secs 2024-04-27T00:42:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:36 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22038.118288 secs 2024-04-27T00:42:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:36 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22038.118288 secs 2024-04-27T00:42:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:36 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22038.118288 secs 2024-04-27T00:42:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:37 smithi119 ceph-mon[32752]: pgmap v12382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:37 smithi167 ceph-mon[32751]: pgmap v12382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:37 smithi090 ceph-mon[29180]: pgmap v12382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:39 smithi119 ceph-mon[32752]: pgmap v12383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:39 smithi167 ceph-mon[32751]: pgmap v12383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:39 smithi090 ceph-mon[29180]: pgmap v12383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:42:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22042.321142 secs 2024-04-27T00:42:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22042.321142 secs 2024-04-27T00:42:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22042.321142 secs 2024-04-27T00:42:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:41 smithi119 ceph-mon[32752]: pgmap v12384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:41 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22043.118445 secs 2024-04-27T00:42:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:41 smithi167 ceph-mon[32751]: pgmap v12384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:41 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22043.118445 secs 2024-04-27T00:42:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:41 smithi090 ceph-mon[29180]: pgmap v12384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:41 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22043.118445 secs 2024-04-27T00:42:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:42:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:42:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:42:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:43 smithi119 ceph-mon[32752]: pgmap v12385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:42:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:42:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:42:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:43 smithi167 ceph-mon[32751]: pgmap v12385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:42:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:42:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:42:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:43 smithi090 ceph-mon[29180]: pgmap v12385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:42:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:42:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:42:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:44 smithi119 ceph-mon[32752]: pgmap v12386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:42:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:44 smithi119 ceph-mon[32752]: pgmap v12387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:44 smithi167 ceph-mon[32751]: pgmap v12386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:42:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:44 smithi167 ceph-mon[32751]: pgmap v12387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:44 smithi090 ceph-mon[29180]: pgmap v12386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T00:42:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:44 smithi090 ceph-mon[29180]: pgmap v12387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22047.321201 secs 2024-04-27T00:42:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22047.321201 secs 2024-04-27T00:42:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22047.321201 secs 2024-04-27T00:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:47 smithi119 ceph-mon[32752]: pgmap v12388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:47 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22048.118524 secs 2024-04-27T00:42:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:47 smithi167 ceph-mon[32751]: pgmap v12388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:47 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22048.118524 secs 2024-04-27T00:42:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:47 smithi090 ceph-mon[29180]: pgmap v12388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:47 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22048.118524 secs 2024-04-27T00:42:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:49 smithi119 ceph-mon[32752]: pgmap v12389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:42:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:49 smithi167 ceph-mon[32751]: pgmap v12389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:42:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:49 smithi090 ceph-mon[29180]: pgmap v12389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:42:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:51 smithi119 ceph-mon[32752]: pgmap v12390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22052.321262 secs 2024-04-27T00:42:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:51 smithi167 ceph-mon[32751]: pgmap v12390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22052.321262 secs 2024-04-27T00:42:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:51 smithi090 ceph-mon[29180]: pgmap v12390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22052.321262 secs 2024-04-27T00:42:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:52 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22053.118629 secs 2024-04-27T00:42:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:52 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22053.118629 secs 2024-04-27T00:42:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:52 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22053.118629 secs 2024-04-27T00:42:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:53 smithi119 ceph-mon[32752]: pgmap v12391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:53 smithi167 ceph-mon[32751]: pgmap v12391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:53 smithi090 ceph-mon[29180]: pgmap v12391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:42:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:55 smithi119 ceph-mon[32752]: pgmap v12392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:42:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:55 smithi167 ceph-mon[32751]: pgmap v12392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:42:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:55 smithi090 ceph-mon[29180]: pgmap v12392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:42:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22057.321349 secs 2024-04-27T00:42:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22057.321349 secs 2024-04-27T00:42:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22057.321349 secs 2024-04-27T00:42:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:57 smithi119 ceph-mon[32752]: pgmap v12393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:57 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22058.118702 secs 2024-04-27T00:42:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:57 smithi167 ceph-mon[32751]: pgmap v12393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:57 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22058.118702 secs 2024-04-27T00:42:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:57 smithi090 ceph-mon[29180]: pgmap v12393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:57 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22058.118702 secs 2024-04-27T00:42:59.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:42:59 smithi119 ceph-mon[32752]: pgmap v12394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:42:59 smithi167 ceph-mon[32751]: pgmap v12394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:42:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:42:59 smithi090 ceph-mon[29180]: pgmap v12394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:01 smithi119 ceph-mon[32752]: pgmap v12395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22062.321435 secs 2024-04-27T00:43:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:01 smithi167 ceph-mon[32751]: pgmap v12395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22062.321435 secs 2024-04-27T00:43:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:01 smithi090 ceph-mon[29180]: pgmap v12395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22062.321435 secs 2024-04-27T00:43:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:02 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22063.118813 secs 2024-04-27T00:43:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:02 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22063.118813 secs 2024-04-27T00:43:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:02 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22063.118813 secs 2024-04-27T00:43:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:03 smithi119 ceph-mon[32752]: pgmap v12396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:03 smithi167 ceph-mon[32751]: pgmap v12396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:03 smithi090 ceph-mon[29180]: pgmap v12396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:05.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:05 smithi119 ceph-mon[32752]: pgmap v12397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:05 smithi167 ceph-mon[32751]: pgmap v12397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:05 smithi090 ceph-mon[29180]: pgmap v12397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22067.321524 secs 2024-04-27T00:43:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22067.321524 secs 2024-04-27T00:43:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22067.321524 secs 2024-04-27T00:43:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:07 smithi090 ceph-mon[29180]: pgmap v12398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:07 smithi090 ceph-mon[29180]: 22 slow requests, 0 included below; oldest blocked for > 22068.118927 secs 2024-04-27T00:43:07.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:43:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:43:06.863+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:43:07.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:43:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:43:06.863+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:43:07.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:43:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:43:06.863+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:43:07.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:43:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:43:06.863+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:43:07.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:43:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:43:06.863+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:43:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:07 smithi119 ceph-mon[32752]: pgmap v12398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:07 smithi119 ceph-mon[32752]: 22 slow requests, 0 included below; oldest blocked for > 22068.118927 secs 2024-04-27T00:43:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:07 smithi167 ceph-mon[32751]: pgmap v12398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:07 smithi167 ceph-mon[32751]: 22 slow requests, 0 included below; oldest blocked for > 22068.118927 secs 2024-04-27T00:43:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:09 smithi167 ceph-mon[32751]: pgmap v12399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:09 smithi090 ceph-mon[29180]: pgmap v12399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:09 smithi119 ceph-mon[32752]: pgmap v12399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:11 smithi167 ceph-mon[32751]: pgmap v12400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22072.321609 secs 2024-04-27T00:43:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:11 smithi090 ceph-mon[29180]: pgmap v12400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22072.321609 secs 2024-04-27T00:43:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:11 smithi119 ceph-mon[32752]: pgmap v12400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22072.321609 secs 2024-04-27T00:43:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:12 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22073.118998 secs 2024-04-27T00:43:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:43:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:43:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:12 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22073.118998 secs 2024-04-27T00:43:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:43:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:43:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:12 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22073.118998 secs 2024-04-27T00:43:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:43:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:43:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:13 smithi090 ceph-mon[29180]: pgmap v12401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:13 smithi119 ceph-mon[32752]: pgmap v12401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:13 smithi167 ceph-mon[32751]: pgmap v12401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:15 smithi090 ceph-mon[29180]: pgmap v12402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:15 smithi119 ceph-mon[32752]: pgmap v12402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:15 smithi167 ceph-mon[32751]: pgmap v12402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22077.321666 secs 2024-04-27T00:43:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22077.321666 secs 2024-04-27T00:43:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22077.321666 secs 2024-04-27T00:43:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:17 smithi090 ceph-mon[29180]: pgmap v12403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:17 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22078.119118 secs 2024-04-27T00:43:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:17 smithi119 ceph-mon[32752]: pgmap v12403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:17 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22078.119118 secs 2024-04-27T00:43:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:17 smithi167 ceph-mon[32751]: pgmap v12403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:17 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22078.119118 secs 2024-04-27T00:43:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:19 smithi090 ceph-mon[29180]: pgmap v12404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:19 smithi119 ceph-mon[32752]: pgmap v12404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:19 smithi167 ceph-mon[32751]: pgmap v12404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:21 smithi090 ceph-mon[29180]: pgmap v12405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22082.321751 secs 2024-04-27T00:43:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:21 smithi119 ceph-mon[32752]: pgmap v12405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22082.321751 secs 2024-04-27T00:43:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:21 smithi167 ceph-mon[32751]: pgmap v12405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T00:43:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22082.321751 secs 2024-04-27T00:43:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:22 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22083.119189 secs 2024-04-27T00:43:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:22 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22083.119189 secs 2024-04-27T00:43:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:22 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22083.119189 secs 2024-04-27T00:43:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:23 smithi090 ceph-mon[29180]: pgmap v12406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:23 smithi119 ceph-mon[32752]: pgmap v12406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:23 smithi167 ceph-mon[32751]: pgmap v12406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:25 smithi090 ceph-mon[29180]: pgmap v12407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:25 smithi119 ceph-mon[32752]: pgmap v12407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:25 smithi167 ceph-mon[32751]: pgmap v12407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22087.321820 secs 2024-04-27T00:43:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22087.321820 secs 2024-04-27T00:43:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22087.321820 secs 2024-04-27T00:43:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:27 smithi090 ceph-mon[29180]: pgmap v12408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:27 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22088.119305 secs 2024-04-27T00:43:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:27 smithi167 ceph-mon[32751]: pgmap v12408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:27 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22088.119305 secs 2024-04-27T00:43:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:27 smithi119 ceph-mon[32752]: pgmap v12408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:27 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22088.119305 secs 2024-04-27T00:43:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:29 smithi090 ceph-mon[29180]: pgmap v12409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:29 smithi119 ceph-mon[32752]: pgmap v12409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:29 smithi167 ceph-mon[32751]: pgmap v12409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:31 smithi090 ceph-mon[29180]: pgmap v12410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22092.321909 secs 2024-04-27T00:43:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:31 smithi119 ceph-mon[32752]: pgmap v12410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22092.321909 secs 2024-04-27T00:43:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:31 smithi167 ceph-mon[32751]: pgmap v12410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22092.321909 secs 2024-04-27T00:43:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:32 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22093.119396 secs 2024-04-27T00:43:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:32 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22093.119396 secs 2024-04-27T00:43:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:32 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22093.119396 secs 2024-04-27T00:43:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:33 smithi119 ceph-mon[32752]: pgmap v12411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:33 smithi167 ceph-mon[32751]: pgmap v12411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:33 smithi090 ceph-mon[29180]: pgmap v12411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:35 smithi119 ceph-mon[32752]: pgmap v12412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:35 smithi167 ceph-mon[32751]: pgmap v12412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:35 smithi090 ceph-mon[29180]: pgmap v12412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22097.321991 secs 2024-04-27T00:43:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22097.321991 secs 2024-04-27T00:43:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22097.321991 secs 2024-04-27T00:43:37.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:37 smithi119 ceph-mon[32752]: pgmap v12413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:37.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:37 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22098.119429 secs 2024-04-27T00:43:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:37 smithi167 ceph-mon[32751]: pgmap v12413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:37 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22098.119429 secs 2024-04-27T00:43:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:37 smithi090 ceph-mon[29180]: pgmap v12413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:37 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22098.119429 secs 2024-04-27T00:43:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:39 smithi119 ceph-mon[32752]: pgmap v12414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:39 smithi167 ceph-mon[32751]: pgmap v12414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:39 smithi090 ceph-mon[29180]: pgmap v12414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:41 smithi119 ceph-mon[32752]: pgmap v12415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22102.322099 secs 2024-04-27T00:43:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:41 smithi167 ceph-mon[32751]: pgmap v12415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22102.322099 secs 2024-04-27T00:43:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:41 smithi090 ceph-mon[29180]: pgmap v12415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22102.322099 secs 2024-04-27T00:43:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:42 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22103.119454 secs 2024-04-27T00:43:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:42 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22103.119454 secs 2024-04-27T00:43:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:42 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22103.119454 secs 2024-04-27T00:43:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:43 smithi119 ceph-mon[32752]: pgmap v12416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:43 smithi167 ceph-mon[32751]: pgmap v12416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:43 smithi090 ceph-mon[29180]: pgmap v12416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:43:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:43:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:43:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:45 smithi119 ceph-mon[32752]: pgmap v12417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:43:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:43:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:45 smithi119 ceph-mon[32752]: pgmap v12418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:43:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:45 smithi119 ceph-mon[32752]: pgmap v12419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:43:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:45 smithi167 ceph-mon[32751]: pgmap v12417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:43:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:43:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:45 smithi167 ceph-mon[32751]: pgmap v12418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:43:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:45 smithi167 ceph-mon[32751]: pgmap v12419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:43:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:45 smithi090 ceph-mon[29180]: pgmap v12417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:45 smithi090 ceph-mon[29180]: pgmap v12418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:45 smithi090 ceph-mon[29180]: pgmap v12419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:43:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22107.322157 secs 2024-04-27T00:43:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22107.322157 secs 2024-04-27T00:43:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22107.322157 secs 2024-04-27T00:43:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:47 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22108.119532 secs 2024-04-27T00:43:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:47 smithi119 ceph-mon[32752]: pgmap v12420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:47 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22108.119532 secs 2024-04-27T00:43:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:47 smithi167 ceph-mon[32751]: pgmap v12420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:47 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22108.119532 secs 2024-04-27T00:43:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:47 smithi090 ceph-mon[29180]: pgmap v12420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:49 smithi119 ceph-mon[32752]: pgmap v12421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:49 smithi167 ceph-mon[32751]: pgmap v12421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:49 smithi090 ceph-mon[29180]: pgmap v12421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22112.322242 secs 2024-04-27T00:43:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22112.322242 secs 2024-04-27T00:43:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22112.322242 secs 2024-04-27T00:43:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:51 smithi119 ceph-mon[32752]: pgmap v12422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:51 smithi167 ceph-mon[32751]: pgmap v12422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:51 smithi090 ceph-mon[29180]: pgmap v12422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:43:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:52 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22113.119609 secs 2024-04-27T00:43:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:52 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22113.119609 secs 2024-04-27T00:43:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:52 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22113.119609 secs 2024-04-27T00:43:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:53 smithi119 ceph-mon[32752]: pgmap v12423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:43:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:53 smithi167 ceph-mon[32751]: pgmap v12423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:43:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:53 smithi090 ceph-mon[29180]: pgmap v12423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:43:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:55 smithi119 ceph-mon[32752]: pgmap v12424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:43:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22117.322318 secs 2024-04-27T00:43:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:55 smithi167 ceph-mon[32751]: pgmap v12424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:43:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22117.322318 secs 2024-04-27T00:43:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:55 smithi090 ceph-mon[29180]: pgmap v12424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:43:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22117.322318 secs 2024-04-27T00:43:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:57 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22118.119696 secs 2024-04-27T00:43:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:57 smithi119 ceph-mon[32752]: pgmap v12425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:57 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22118.119696 secs 2024-04-27T00:43:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:57 smithi167 ceph-mon[32751]: pgmap v12425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:57 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22118.119696 secs 2024-04-27T00:43:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:57 smithi090 ceph-mon[29180]: pgmap v12425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:43:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:43:59 smithi119 ceph-mon[32752]: pgmap v12426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:43:59 smithi167 ceph-mon[32751]: pgmap v12426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:43:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:43:59 smithi090 ceph-mon[29180]: pgmap v12426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22122.322402 secs 2024-04-27T00:44:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22122.322402 secs 2024-04-27T00:44:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22122.322402 secs 2024-04-27T00:44:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:01 smithi119 ceph-mon[32752]: pgmap v12427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:01 smithi167 ceph-mon[32751]: pgmap v12427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:01 smithi090 ceph-mon[29180]: pgmap v12427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:02.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:02 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22123.119766 secs 2024-04-27T00:44:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:02 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22123.119766 secs 2024-04-27T00:44:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:02 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22123.119766 secs 2024-04-27T00:44:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:03 smithi119 ceph-mon[32752]: pgmap v12428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:03 smithi167 ceph-mon[32751]: pgmap v12428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:03 smithi090 ceph-mon[29180]: pgmap v12428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:05 smithi119 ceph-mon[32752]: pgmap v12429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22127.322485 secs 2024-04-27T00:44:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:05 smithi167 ceph-mon[32751]: pgmap v12429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22127.322485 secs 2024-04-27T00:44:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:05 smithi090 ceph-mon[29180]: pgmap v12429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22127.322485 secs 2024-04-27T00:44:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:07 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22128.119883 secs 2024-04-27T00:44:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:07 smithi119 ceph-mon[32752]: pgmap v12430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:07 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22128.119883 secs 2024-04-27T00:44:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:07 smithi167 ceph-mon[32751]: pgmap v12430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:07 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22128.119883 secs 2024-04-27T00:44:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:07 smithi090 ceph-mon[29180]: pgmap v12430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:09 smithi119 ceph-mon[32752]: pgmap v12431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:09 smithi167 ceph-mon[32751]: pgmap v12431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:09 smithi090 ceph-mon[29180]: pgmap v12431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:10 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 22132.322568 secs 2024-04-27T00:44:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:10 smithi119 ceph-mon[32752]: slow request 5764.828580 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:44:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:10 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 22132.322568 secs 2024-04-27T00:44:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:10 smithi167 ceph-mon[32751]: slow request 5764.828580 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:44:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:10 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 22132.322568 secs 2024-04-27T00:44:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:10 smithi090 ceph-mon[29180]: slow request 5764.828580 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:44:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:11 smithi119 ceph-mon[32752]: pgmap v12432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:11 smithi167 ceph-mon[32751]: pgmap v12432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:11 smithi090 ceph-mon[29180]: pgmap v12432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:12 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22133.119942 secs 2024-04-27T00:44:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:12 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22133.119942 secs 2024-04-27T00:44:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:12 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22133.119942 secs 2024-04-27T00:44:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:13 smithi119 ceph-mon[32752]: pgmap v12433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:13 smithi167 ceph-mon[32751]: pgmap v12433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:13 smithi090 ceph-mon[29180]: pgmap v12433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:44:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:15 smithi119 ceph-mon[32752]: pgmap v12434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22137.322647 secs 2024-04-27T00:44:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:15 smithi167 ceph-mon[32751]: pgmap v12434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22137.322647 secs 2024-04-27T00:44:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:15 smithi090 ceph-mon[29180]: pgmap v12434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:44:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22137.322647 secs 2024-04-27T00:44:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:17 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22138.120081 secs 2024-04-27T00:44:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:17 smithi119 ceph-mon[32752]: pgmap v12435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:17 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22138.120081 secs 2024-04-27T00:44:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:17 smithi167 ceph-mon[32751]: pgmap v12435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:17 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22138.120081 secs 2024-04-27T00:44:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:17 smithi090 ceph-mon[29180]: pgmap v12435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:19 smithi119 ceph-mon[32752]: pgmap v12436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:19.813 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:19 smithi090 ceph-mon[29180]: pgmap v12436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:19 smithi167 ceph-mon[32751]: pgmap v12436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22142.322733 secs 2024-04-27T00:44:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22142.322733 secs 2024-04-27T00:44:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22142.322733 secs 2024-04-27T00:44:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:21 smithi119 ceph-mon[32752]: pgmap v12437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:21 smithi167 ceph-mon[32751]: pgmap v12437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:21 smithi090 ceph-mon[29180]: pgmap v12437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:22 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22143.120220 secs 2024-04-27T00:44:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:22 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22143.120220 secs 2024-04-27T00:44:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:22 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22143.120220 secs 2024-04-27T00:44:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:23 smithi119 ceph-mon[32752]: pgmap v12438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:23 smithi167 ceph-mon[32751]: pgmap v12438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:23 smithi090 ceph-mon[29180]: pgmap v12438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:24 smithi119 ceph-mon[32752]: pgmap v12439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:24 smithi167 ceph-mon[32751]: pgmap v12439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:24 smithi090 ceph-mon[29180]: pgmap v12439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22147.322833 secs 2024-04-27T00:44:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22147.322833 secs 2024-04-27T00:44:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22147.322833 secs 2024-04-27T00:44:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:26 smithi119 ceph-mon[32752]: pgmap v12440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:26 smithi167 ceph-mon[32751]: pgmap v12440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:26 smithi090 ceph-mon[29180]: pgmap v12440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:27 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22148.120357 secs 2024-04-27T00:44:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:27 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22148.120357 secs 2024-04-27T00:44:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:27 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22148.120357 secs 2024-04-27T00:44:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:28 smithi119 ceph-mon[32752]: pgmap v12441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:28 smithi167 ceph-mon[32751]: pgmap v12441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:28 smithi090 ceph-mon[29180]: pgmap v12441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22152.322916 secs 2024-04-27T00:44:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22152.322916 secs 2024-04-27T00:44:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22152.322916 secs 2024-04-27T00:44:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:31 smithi119 ceph-mon[32752]: pgmap v12442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:31 smithi167 ceph-mon[32751]: pgmap v12442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:31 smithi090 ceph-mon[29180]: pgmap v12442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:32 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22153.120500 secs 2024-04-27T00:44:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:32 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22153.120500 secs 2024-04-27T00:44:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:32 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22153.120500 secs 2024-04-27T00:44:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:33 smithi119 ceph-mon[32752]: pgmap v12443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:33 smithi167 ceph-mon[32751]: pgmap v12443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:33 smithi090 ceph-mon[29180]: pgmap v12443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:44:35.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:35 smithi090 ceph-mon[29180]: pgmap v12444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:35.685 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22157.322988 secs 2024-04-27T00:44:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:35 smithi119 ceph-mon[32752]: pgmap v12444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22157.322988 secs 2024-04-27T00:44:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:35 smithi167 ceph-mon[32751]: pgmap v12444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22157.322988 secs 2024-04-27T00:44:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:37 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22158.120575 secs 2024-04-27T00:44:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:37 smithi119 ceph-mon[32752]: pgmap v12445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:37 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22158.120575 secs 2024-04-27T00:44:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:37 smithi167 ceph-mon[32751]: pgmap v12445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:37 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22158.120575 secs 2024-04-27T00:44:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:37 smithi090 ceph-mon[29180]: pgmap v12445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:39 smithi119 ceph-mon[32752]: pgmap v12446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:39 smithi167 ceph-mon[32751]: pgmap v12446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:39 smithi090 ceph-mon[29180]: pgmap v12446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22162.323093 secs 2024-04-27T00:44:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22162.323093 secs 2024-04-27T00:44:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22162.323093 secs 2024-04-27T00:44:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:41 smithi119 ceph-mon[32752]: pgmap v12447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:41 smithi167 ceph-mon[32751]: pgmap v12447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:41 smithi090 ceph-mon[29180]: pgmap v12447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:42 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22163.120682 secs 2024-04-27T00:44:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:42 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22163.120682 secs 2024-04-27T00:44:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:42 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22163.120682 secs 2024-04-27T00:44:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:43 smithi119 ceph-mon[32752]: pgmap v12448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:43 smithi167 ceph-mon[32751]: pgmap v12448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:43 smithi090 ceph-mon[29180]: pgmap v12448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:45 smithi119 ceph-mon[32752]: pgmap v12449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22167.323158 secs 2024-04-27T00:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:44:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:44:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:45 smithi167 ceph-mon[32751]: pgmap v12449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22167.323158 secs 2024-04-27T00:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:44:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:44:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:45 smithi090 ceph-mon[29180]: pgmap v12449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22167.323158 secs 2024-04-27T00:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:44:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:44:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:46 smithi119 ceph-mon[32752]: pgmap v12450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:44:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:46 smithi119 ceph-mon[32752]: pgmap v12451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:46 smithi167 ceph-mon[32751]: pgmap v12450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:44:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:46 smithi167 ceph-mon[32751]: pgmap v12451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:46 smithi090 ceph-mon[29180]: pgmap v12450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:44:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:46 smithi090 ceph-mon[29180]: pgmap v12451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:47 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22168.120793 secs 2024-04-27T00:44:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:47 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22168.120793 secs 2024-04-27T00:44:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:47 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22168.120793 secs 2024-04-27T00:44:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:48 smithi119 ceph-mon[32752]: pgmap v12452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:44:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:48 smithi167 ceph-mon[32751]: pgmap v12452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:44:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:48 smithi090 ceph-mon[29180]: pgmap v12452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:44:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:50 smithi119 ceph-mon[32752]: pgmap v12453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22172.323242 secs 2024-04-27T00:44:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:50 smithi167 ceph-mon[32751]: pgmap v12453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22172.323242 secs 2024-04-27T00:44:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:50 smithi090 ceph-mon[29180]: pgmap v12453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22172.323242 secs 2024-04-27T00:44:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:52 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22173.120883 secs 2024-04-27T00:44:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:52 smithi119 ceph-mon[32752]: pgmap v12454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:52 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22173.120883 secs 2024-04-27T00:44:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:52 smithi167 ceph-mon[32751]: pgmap v12454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:52 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22173.120883 secs 2024-04-27T00:44:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:52 smithi090 ceph-mon[29180]: pgmap v12454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:44:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:54 smithi119 ceph-mon[32752]: pgmap v12455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:44:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:54 smithi167 ceph-mon[32751]: pgmap v12455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:44:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:54 smithi090 ceph-mon[29180]: pgmap v12455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:44:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22177.323326 secs 2024-04-27T00:44:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22177.323326 secs 2024-04-27T00:44:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22177.323326 secs 2024-04-27T00:44:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:56 smithi119 ceph-mon[32752]: pgmap v12456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:44:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:56 smithi167 ceph-mon[32751]: pgmap v12456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:44:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:56 smithi090 ceph-mon[29180]: pgmap v12456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:44:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:57 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22178.120956 secs 2024-04-27T00:44:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:57 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22178.120956 secs 2024-04-27T00:44:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:57 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22178.120956 secs 2024-04-27T00:44:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:58 smithi119 ceph-mon[32752]: pgmap v12457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:58 smithi167 ceph-mon[32751]: pgmap v12457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:58 smithi090 ceph-mon[29180]: pgmap v12457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:44:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:44:59 smithi119 ceph-mon[32752]: pgmap v12458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:44:59 smithi167 ceph-mon[32751]: pgmap v12458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:44:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:44:59 smithi090 ceph-mon[29180]: pgmap v12458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22182.323412 secs 2024-04-27T00:45:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22182.323412 secs 2024-04-27T00:45:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22182.323412 secs 2024-04-27T00:45:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:01 smithi119 ceph-mon[32752]: pgmap v12459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:01 smithi167 ceph-mon[32751]: pgmap v12459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:01 smithi090 ceph-mon[29180]: pgmap v12459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:02 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22183.121062 secs 2024-04-27T00:45:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:02 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22183.121062 secs 2024-04-27T00:45:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:02 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22183.121062 secs 2024-04-27T00:45:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:03 smithi119 ceph-mon[32752]: pgmap v12460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:03 smithi167 ceph-mon[32751]: pgmap v12460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:03 smithi090 ceph-mon[29180]: pgmap v12460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22187.323494 secs 2024-04-27T00:45:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:06 smithi119 ceph-mon[32752]: pgmap v12461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22187.323494 secs 2024-04-27T00:45:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:06 smithi167 ceph-mon[32751]: pgmap v12461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22187.323494 secs 2024-04-27T00:45:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:06 smithi090 ceph-mon[29180]: pgmap v12461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:07 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22188.121169 secs 2024-04-27T00:45:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:07 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22188.121169 secs 2024-04-27T00:45:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:07 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22188.121169 secs 2024-04-27T00:45:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:08 smithi119 ceph-mon[32752]: pgmap v12462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:08 smithi167 ceph-mon[32751]: pgmap v12462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:08 smithi090 ceph-mon[29180]: pgmap v12462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:10 smithi119 ceph-mon[32752]: pgmap v12463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:10 smithi167 ceph-mon[32751]: pgmap v12463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:10 smithi090 ceph-mon[29180]: pgmap v12463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22192.323580 secs 2024-04-27T00:45:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22192.323580 secs 2024-04-27T00:45:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22192.323580 secs 2024-04-27T00:45:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:12 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22193.121289 secs 2024-04-27T00:45:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:12 smithi119 ceph-mon[32752]: pgmap v12464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:12 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22193.121289 secs 2024-04-27T00:45:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:12 smithi167 ceph-mon[32751]: pgmap v12464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:12 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22193.121289 secs 2024-04-27T00:45:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:12 smithi090 ceph-mon[29180]: pgmap v12464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:14.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:14 smithi119 ceph-mon[32752]: pgmap v12465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:14 smithi167 ceph-mon[32751]: pgmap v12465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:14 smithi090 ceph-mon[29180]: pgmap v12465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22197.323663 secs 2024-04-27T00:45:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:16 smithi119 ceph-mon[32752]: pgmap v12466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22197.323663 secs 2024-04-27T00:45:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:16 smithi167 ceph-mon[32751]: pgmap v12466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22197.323663 secs 2024-04-27T00:45:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:16 smithi090 ceph-mon[29180]: pgmap v12466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:17 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22198.121403 secs 2024-04-27T00:45:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:17 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22198.121403 secs 2024-04-27T00:45:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:17 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22198.121403 secs 2024-04-27T00:45:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:18 smithi119 ceph-mon[32752]: pgmap v12467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:18 smithi167 ceph-mon[32751]: pgmap v12467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:18 smithi090 ceph-mon[29180]: pgmap v12467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:20 smithi119 ceph-mon[32752]: pgmap v12468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:20 smithi167 ceph-mon[32751]: pgmap v12468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:20 smithi090 ceph-mon[29180]: pgmap v12468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22202.323749 secs 2024-04-27T00:45:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22202.323749 secs 2024-04-27T00:45:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22202.323749 secs 2024-04-27T00:45:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:22 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22203.121469 secs 2024-04-27T00:45:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:22 smithi119 ceph-mon[32752]: pgmap v12469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:22 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22203.121469 secs 2024-04-27T00:45:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:22 smithi167 ceph-mon[32751]: pgmap v12469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:22 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22203.121469 secs 2024-04-27T00:45:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:22 smithi090 ceph-mon[29180]: pgmap v12469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:24 smithi119 ceph-mon[32752]: pgmap v12470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:24 smithi167 ceph-mon[32751]: pgmap v12470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:24 smithi090 ceph-mon[29180]: pgmap v12470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22207.323832 secs 2024-04-27T00:45:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:26 smithi119 ceph-mon[32752]: pgmap v12471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22207.323832 secs 2024-04-27T00:45:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:26 smithi167 ceph-mon[32751]: pgmap v12471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22207.323832 secs 2024-04-27T00:45:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:26 smithi090 ceph-mon[29180]: pgmap v12471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:27 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22208.121549 secs 2024-04-27T00:45:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:27 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22208.121549 secs 2024-04-27T00:45:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:27 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22208.121549 secs 2024-04-27T00:45:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:28 smithi119 ceph-mon[32752]: pgmap v12472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:28 smithi167 ceph-mon[32751]: pgmap v12472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:28 smithi090 ceph-mon[29180]: pgmap v12472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:30 smithi119 ceph-mon[32752]: pgmap v12473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:30 smithi167 ceph-mon[32751]: pgmap v12473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:30 smithi090 ceph-mon[29180]: pgmap v12473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22212.323926 secs 2024-04-27T00:45:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22212.323926 secs 2024-04-27T00:45:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22212.323926 secs 2024-04-27T00:45:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:32 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22213.121689 secs 2024-04-27T00:45:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:32 smithi090 ceph-mon[29180]: pgmap v12474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:32 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22213.121689 secs 2024-04-27T00:45:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:32 smithi119 ceph-mon[32752]: pgmap v12474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:32 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22213.121689 secs 2024-04-27T00:45:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:32 smithi167 ceph-mon[32751]: pgmap v12474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:34 smithi119 ceph-mon[32752]: pgmap v12475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:34 smithi167 ceph-mon[32751]: pgmap v12475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:34 smithi090 ceph-mon[29180]: pgmap v12475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22217.324010 secs 2024-04-27T00:45:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:36 smithi119 ceph-mon[32752]: pgmap v12476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22217.324010 secs 2024-04-27T00:45:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:36 smithi167 ceph-mon[32751]: pgmap v12476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22217.324010 secs 2024-04-27T00:45:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:36 smithi090 ceph-mon[29180]: pgmap v12476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:37 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22218.121825 secs 2024-04-27T00:45:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:37 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22218.121825 secs 2024-04-27T00:45:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:37 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22218.121825 secs 2024-04-27T00:45:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:38 smithi119 ceph-mon[32752]: pgmap v12477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:38 smithi167 ceph-mon[32751]: pgmap v12477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:38.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:38 smithi090 ceph-mon[29180]: pgmap v12477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:40 smithi119 ceph-mon[32752]: pgmap v12478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:40 smithi167 ceph-mon[32751]: pgmap v12478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:40 smithi090 ceph-mon[29180]: pgmap v12478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22222.324112 secs 2024-04-27T00:45:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22222.324112 secs 2024-04-27T00:45:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22222.324112 secs 2024-04-27T00:45:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:42 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22223.121904 secs 2024-04-27T00:45:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:42 smithi119 ceph-mon[32752]: pgmap v12479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:42 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22223.121904 secs 2024-04-27T00:45:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:42 smithi167 ceph-mon[32751]: pgmap v12479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:42 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22223.121904 secs 2024-04-27T00:45:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:42 smithi090 ceph-mon[29180]: pgmap v12479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:44 smithi119 ceph-mon[32752]: pgmap v12480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:44 smithi167 ceph-mon[32751]: pgmap v12480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:44.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:44 smithi090 ceph-mon[29180]: pgmap v12480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:45:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:45:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:45:46.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 22227.324195 secs 2024-04-27T00:45:46.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:46 smithi119 ceph-mon[32752]: pgmap v12481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 22227.324195 secs 2024-04-27T00:45:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:46 smithi167 ceph-mon[32751]: pgmap v12481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:46.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 22227.324195 secs 2024-04-27T00:45:46.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:46 smithi090 ceph-mon[29180]: pgmap v12481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:47.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:47 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22228.121963 secs 2024-04-27T00:45:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:47 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22228.121963 secs 2024-04-27T00:45:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:47 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22228.121963 secs 2024-04-27T00:45:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:48 smithi119 ceph-mon[32752]: pgmap v12482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:48 smithi167 ceph-mon[32751]: pgmap v12482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:48.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:48 smithi090 ceph-mon[29180]: pgmap v12482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:45:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: pgmap v12483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:45:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:45:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: pgmap v12483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:45:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: pgmap v12483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:45:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:45:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:45:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:51 smithi119 ceph-mon[32752]: pgmap v12484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T00:45:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:51 smithi119 ceph-mon[32752]: pgmap v12485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:51 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22232.324252 secs 2024-04-27T00:45:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:51 smithi167 ceph-mon[32751]: pgmap v12484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T00:45:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:51 smithi167 ceph-mon[32751]: pgmap v12485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:51 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22232.324252 secs 2024-04-27T00:45:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:51 smithi090 ceph-mon[29180]: pgmap v12484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T00:45:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:51 smithi090 ceph-mon[29180]: pgmap v12485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:51 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22232.324252 secs 2024-04-27T00:45:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:52 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22233.122035 secs 2024-04-27T00:45:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:52 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22233.122035 secs 2024-04-27T00:45:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:52 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22233.122035 secs 2024-04-27T00:45:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:53 smithi119 ceph-mon[32752]: pgmap v12486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:53 smithi167 ceph-mon[32751]: pgmap v12486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:53 smithi090 ceph-mon[29180]: pgmap v12486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:55 smithi119 ceph-mon[32752]: pgmap v12487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:45:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:55 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:45:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:55 smithi167 ceph-mon[32751]: pgmap v12487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:45:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:55 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:45:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:55 smithi090 ceph-mon[29180]: pgmap v12487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:45:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:55 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:45:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:56 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22237.324308 secs 2024-04-27T00:45:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:56 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22237.324308 secs 2024-04-27T00:45:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:56 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22237.324308 secs 2024-04-27T00:45:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:57 smithi167 ceph-mon[32751]: pgmap v12488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:57 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22238.122146 secs 2024-04-27T00:45:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:57 smithi090 ceph-mon[29180]: pgmap v12488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:57 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22238.122146 secs 2024-04-27T00:45:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:57 smithi119 ceph-mon[32752]: pgmap v12488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:45:57.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:57 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22238.122146 secs 2024-04-27T00:45:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:45:59 smithi167 ceph-mon[32751]: pgmap v12489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 603 B/s rd, 1 op/s 2024-04-27T00:45:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:45:59 smithi090 ceph-mon[29180]: pgmap v12489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 603 B/s rd, 1 op/s 2024-04-27T00:45:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:45:59 smithi119 ceph-mon[32752]: pgmap v12489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 603 B/s rd, 1 op/s 2024-04-27T00:46:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:01 smithi167 ceph-mon[32751]: pgmap v12490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:46:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:01 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22242.324377 secs 2024-04-27T00:46:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:01 smithi090 ceph-mon[29180]: pgmap v12490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:46:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:01 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22242.324377 secs 2024-04-27T00:46:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:01 smithi119 ceph-mon[32752]: pgmap v12490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:46:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:01 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22242.324377 secs 2024-04-27T00:46:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:02 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22243.122265 secs 2024-04-27T00:46:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:02 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22243.122265 secs 2024-04-27T00:46:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:02 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22243.122265 secs 2024-04-27T00:46:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:03 smithi090 ceph-mon[29180]: pgmap v12491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:03 smithi119 ceph-mon[32752]: pgmap v12491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:03 smithi167 ceph-mon[32751]: pgmap v12491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:05 smithi090 ceph-mon[29180]: pgmap v12492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:05 smithi119 ceph-mon[32752]: pgmap v12492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:05 smithi167 ceph-mon[32751]: pgmap v12492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:06 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22247.324456 secs 2024-04-27T00:46:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:06 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22247.324456 secs 2024-04-27T00:46:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:06 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22247.324456 secs 2024-04-27T00:46:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:07 smithi090 ceph-mon[29180]: pgmap v12493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:07 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 22248.122406 secs 2024-04-27T00:46:07.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:07 smithi119 ceph-mon[32752]: pgmap v12493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:07.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:07 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 22248.122406 secs 2024-04-27T00:46:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:07 smithi167 ceph-mon[32751]: pgmap v12493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:07 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 22248.122406 secs 2024-04-27T00:46:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:09 smithi090 ceph-mon[29180]: pgmap v12494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:09 smithi119 ceph-mon[32752]: pgmap v12494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:09 smithi167 ceph-mon[32751]: pgmap v12494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:11 smithi090 ceph-mon[29180]: pgmap v12495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:11 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22252.324537 secs 2024-04-27T00:46:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:11 smithi119 ceph-mon[32752]: pgmap v12495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:11 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22252.324537 secs 2024-04-27T00:46:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:11 smithi167 ceph-mon[32751]: pgmap v12495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:11 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22252.324537 secs 2024-04-27T00:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:12 smithi090 ceph-mon[29180]: 23 slow requests, 2 included below; oldest blocked for > 22253.122516 secs 2024-04-27T00:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:12 smithi090 ceph-mon[29180]: slow request 184.003649 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:12 smithi090 ceph-mon[29180]: slow request 814.165265 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:46:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:12 smithi119 ceph-mon[32752]: 23 slow requests, 2 included below; oldest blocked for > 22253.122516 secs 2024-04-27T00:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:12 smithi119 ceph-mon[32752]: slow request 184.003649 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:12 smithi119 ceph-mon[32752]: slow request 814.165265 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:46:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:46:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:12 smithi167 ceph-mon[32751]: 23 slow requests, 2 included below; oldest blocked for > 22253.122516 secs 2024-04-27T00:46:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:12 smithi167 ceph-mon[32751]: slow request 184.003649 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:46:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:12 smithi167 ceph-mon[32751]: slow request 814.165265 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:46:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:46:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:46:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:13 smithi090 ceph-mon[29180]: pgmap v12496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:13 smithi119 ceph-mon[32752]: pgmap v12496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:13 smithi167 ceph-mon[32751]: pgmap v12496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:15 smithi090 ceph-mon[29180]: pgmap v12497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:15 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22257.324622 secs 2024-04-27T00:46:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:15 smithi119 ceph-mon[32752]: pgmap v12497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:15 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22257.324622 secs 2024-04-27T00:46:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:15 smithi167 ceph-mon[32751]: pgmap v12497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:15 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22257.324622 secs 2024-04-27T00:46:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:17 smithi090 ceph-mon[29180]: pgmap v12498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:17 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22258.122591 secs 2024-04-27T00:46:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:17 smithi119 ceph-mon[32752]: pgmap v12498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:17 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22258.122591 secs 2024-04-27T00:46:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:17 smithi167 ceph-mon[32751]: pgmap v12498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:17 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22258.122591 secs 2024-04-27T00:46:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:19 smithi090 ceph-mon[29180]: pgmap v12499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:19 smithi119 ceph-mon[32752]: pgmap v12499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:19 smithi167 ceph-mon[32751]: pgmap v12499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:20.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:20 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22262.324709 secs 2024-04-27T00:46:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:20 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22262.324709 secs 2024-04-27T00:46:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:20 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22262.324709 secs 2024-04-27T00:46:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:21 smithi090 ceph-mon[29180]: pgmap v12500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:21 smithi119 ceph-mon[32752]: pgmap v12500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:21 smithi167 ceph-mon[32751]: pgmap v12500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:22 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22263.122665 secs 2024-04-27T00:46:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:22 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22263.122665 secs 2024-04-27T00:46:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:22 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22263.122665 secs 2024-04-27T00:46:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:23 smithi090 ceph-mon[29180]: pgmap v12501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:23 smithi119 ceph-mon[32752]: pgmap v12501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:23 smithi167 ceph-mon[32751]: pgmap v12501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:25 smithi119 ceph-mon[32752]: pgmap v12502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:25 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22267.324815 secs 2024-04-27T00:46:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:25 smithi167 ceph-mon[32751]: pgmap v12502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:25 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22267.324815 secs 2024-04-27T00:46:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:25 smithi090 ceph-mon[29180]: pgmap v12502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:25 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22267.324815 secs 2024-04-27T00:46:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:27 smithi119 ceph-mon[32752]: pgmap v12503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:27 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22268.122801 secs 2024-04-27T00:46:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:27 smithi167 ceph-mon[32751]: pgmap v12503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:27 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22268.122801 secs 2024-04-27T00:46:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:27 smithi090 ceph-mon[29180]: pgmap v12503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:27 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22268.122801 secs 2024-04-27T00:46:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:29 smithi119 ceph-mon[32752]: pgmap v12504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:29 smithi167 ceph-mon[32751]: pgmap v12504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:29 smithi090 ceph-mon[29180]: pgmap v12504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:30 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22272.324877 secs 2024-04-27T00:46:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:30 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22272.324877 secs 2024-04-27T00:46:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:30 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22272.324877 secs 2024-04-27T00:46:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:31 smithi119 ceph-mon[32752]: pgmap v12505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:31 smithi167 ceph-mon[32751]: pgmap v12505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:31 smithi090 ceph-mon[29180]: pgmap v12505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:32 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22273.122871 secs 2024-04-27T00:46:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:32 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22273.122871 secs 2024-04-27T00:46:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:32 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22273.122871 secs 2024-04-27T00:46:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:33 smithi119 ceph-mon[32752]: pgmap v12506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:33 smithi167 ceph-mon[32751]: pgmap v12506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:33 smithi090 ceph-mon[29180]: pgmap v12506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:35 smithi119 ceph-mon[32752]: pgmap v12507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22277.324939 secs 2024-04-27T00:46:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:35 smithi167 ceph-mon[32751]: pgmap v12507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22277.324939 secs 2024-04-27T00:46:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:35 smithi090 ceph-mon[29180]: pgmap v12507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22277.324939 secs 2024-04-27T00:46:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:37 smithi119 ceph-mon[32752]: pgmap v12508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:37 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22278.122992 secs 2024-04-27T00:46:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:37 smithi167 ceph-mon[32751]: pgmap v12508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:37 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22278.122992 secs 2024-04-27T00:46:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:37 smithi090 ceph-mon[29180]: pgmap v12508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:37 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22278.122992 secs 2024-04-27T00:46:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:39 smithi119 ceph-mon[32752]: pgmap v12509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:39 smithi167 ceph-mon[32751]: pgmap v12509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:39 smithi090 ceph-mon[29180]: pgmap v12509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22282.325025 secs 2024-04-27T00:46:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22282.325025 secs 2024-04-27T00:46:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22282.325025 secs 2024-04-27T00:46:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:41 smithi119 ceph-mon[32752]: pgmap v12510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:41 smithi167 ceph-mon[32751]: pgmap v12510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:41 smithi090 ceph-mon[29180]: pgmap v12510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:42 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22283.123106 secs 2024-04-27T00:46:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:42 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22283.123106 secs 2024-04-27T00:46:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:42 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22283.123106 secs 2024-04-27T00:46:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:43 smithi119 ceph-mon[32752]: pgmap v12511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:43 smithi167 ceph-mon[32751]: pgmap v12511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:43 smithi090 ceph-mon[29180]: pgmap v12511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:45 smithi119 ceph-mon[32752]: pgmap v12512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:45 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22287.325109 secs 2024-04-27T00:46:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:45 smithi167 ceph-mon[32751]: pgmap v12512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:45 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22287.325109 secs 2024-04-27T00:46:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:45 smithi090 ceph-mon[29180]: pgmap v12512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:45 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22287.325109 secs 2024-04-27T00:46:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:47 smithi119 ceph-mon[32752]: pgmap v12513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:47 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22288.123246 secs 2024-04-27T00:46:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:47 smithi167 ceph-mon[32751]: pgmap v12513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:47 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22288.123246 secs 2024-04-27T00:46:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:47 smithi090 ceph-mon[29180]: pgmap v12513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:47 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22288.123246 secs 2024-04-27T00:46:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:49 smithi119 ceph-mon[32752]: pgmap v12514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:49 smithi167 ceph-mon[32751]: pgmap v12514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:49 smithi090 ceph-mon[29180]: pgmap v12514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:46:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:50 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22292.325172 secs 2024-04-27T00:46:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:46:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:50 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22292.325172 secs 2024-04-27T00:46:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:46:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:50 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22292.325172 secs 2024-04-27T00:46:51.681 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:51 smithi167 ceph-mon[32751]: pgmap v12515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:51 smithi119 ceph-mon[32752]: pgmap v12515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:51 smithi090 ceph-mon[29180]: pgmap v12515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:52.648 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:52 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22293.123329 secs 2024-04-27T00:46:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:52 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22293.123329 secs 2024-04-27T00:46:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:52 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22293.123329 secs 2024-04-27T00:46:53.734 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:53 smithi167 ceph-mon[32751]: pgmap v12516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:53 smithi119 ceph-mon[32752]: pgmap v12516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:53 smithi090 ceph-mon[29180]: pgmap v12516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: pgmap v12517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:46:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:46:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: pgmap v12518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-27T00:46:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: pgmap v12519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: pgmap v12517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:46:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: pgmap v12518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-27T00:46:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: pgmap v12519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: pgmap v12517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:46:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:46:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: pgmap v12518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-27T00:46:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: pgmap v12519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:46:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:56 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22297.325232 secs 2024-04-27T00:46:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:56 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22297.325232 secs 2024-04-27T00:46:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:56 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22297.325232 secs 2024-04-27T00:46:57.156 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:57 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22298.123398 secs 2024-04-27T00:46:57.157 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:57 smithi090 ceph-mon[29180]: pgmap v12520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:57 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22298.123398 secs 2024-04-27T00:46:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:57 smithi119 ceph-mon[32752]: pgmap v12520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:57 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22298.123398 secs 2024-04-27T00:46:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:57 smithi167 ceph-mon[32751]: pgmap v12520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:46:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:46:59 smithi090 ceph-mon[29180]: pgmap v12521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T00:46:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:46:59 smithi119 ceph-mon[32752]: pgmap v12521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T00:46:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:46:59 smithi167 ceph-mon[32751]: pgmap v12521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T00:47:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:01 smithi119 ceph-mon[32752]: pgmap v12522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:01 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22302.325316 secs 2024-04-27T00:47:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:01 smithi167 ceph-mon[32751]: pgmap v12522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:01 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22302.325316 secs 2024-04-27T00:47:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:01 smithi090 ceph-mon[29180]: pgmap v12522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:01 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22302.325316 secs 2024-04-27T00:47:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:02 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22303.123492 secs 2024-04-27T00:47:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:02 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22303.123492 secs 2024-04-27T00:47:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:02 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22303.123492 secs 2024-04-27T00:47:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:03 smithi119 ceph-mon[32752]: pgmap v12523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T00:47:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:03 smithi167 ceph-mon[32751]: pgmap v12523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T00:47:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:03 smithi090 ceph-mon[29180]: pgmap v12523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T00:47:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:05 smithi119 ceph-mon[32752]: pgmap v12524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:47:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:05 smithi167 ceph-mon[32751]: pgmap v12524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:47:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:05 smithi090 ceph-mon[29180]: pgmap v12524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:47:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:06 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22307.325397 secs 2024-04-27T00:47:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:06 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22307.325397 secs 2024-04-27T00:47:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:06 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22307.325397 secs 2024-04-27T00:47:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:07 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22308.123631 secs 2024-04-27T00:47:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:07 smithi119 ceph-mon[32752]: pgmap v12525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:07 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22308.123631 secs 2024-04-27T00:47:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:07 smithi167 ceph-mon[32751]: pgmap v12525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:07.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:07 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22308.123631 secs 2024-04-27T00:47:07.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:07 smithi090 ceph-mon[29180]: pgmap v12525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:09 smithi119 ceph-mon[32752]: pgmap v12526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:09 smithi167 ceph-mon[32751]: pgmap v12526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:09 smithi090 ceph-mon[29180]: pgmap v12526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:11 smithi119 ceph-mon[32752]: pgmap v12527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:11 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22312.325482 secs 2024-04-27T00:47:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:11 smithi167 ceph-mon[32751]: pgmap v12527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:11 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22312.325482 secs 2024-04-27T00:47:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:11 smithi090 ceph-mon[29180]: pgmap v12527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:11 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22312.325482 secs 2024-04-27T00:47:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:12 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22313.123711 secs 2024-04-27T00:47:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:12 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22313.123711 secs 2024-04-27T00:47:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:12 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22313.123711 secs 2024-04-27T00:47:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:13 smithi119 ceph-mon[32752]: pgmap v12528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:13 smithi167 ceph-mon[32751]: pgmap v12528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:13 smithi090 ceph-mon[29180]: pgmap v12528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:15 smithi119 ceph-mon[32752]: pgmap v12529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:15 smithi167 ceph-mon[32751]: pgmap v12529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:15 smithi090 ceph-mon[29180]: pgmap v12529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:16 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22317.325573 secs 2024-04-27T00:47:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:16 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22317.325573 secs 2024-04-27T00:47:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:16 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22317.325573 secs 2024-04-27T00:47:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:17 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22318.123797 secs 2024-04-27T00:47:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:17 smithi119 ceph-mon[32752]: pgmap v12530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:17 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22318.123797 secs 2024-04-27T00:47:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:17 smithi167 ceph-mon[32751]: pgmap v12530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:17 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22318.123797 secs 2024-04-27T00:47:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:17 smithi090 ceph-mon[29180]: pgmap v12530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:19 smithi119 ceph-mon[32752]: pgmap v12531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:19 smithi167 ceph-mon[32751]: pgmap v12531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:19 smithi090 ceph-mon[29180]: pgmap v12531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:21 smithi119 ceph-mon[32752]: pgmap v12532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:21 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22322.325657 secs 2024-04-27T00:47:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:21 smithi167 ceph-mon[32751]: pgmap v12532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:21 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22322.325657 secs 2024-04-27T00:47:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:21 smithi090 ceph-mon[29180]: pgmap v12532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:21 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22322.325657 secs 2024-04-27T00:47:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:22 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22323.123868 secs 2024-04-27T00:47:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:22 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22323.123868 secs 2024-04-27T00:47:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:22 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22323.123868 secs 2024-04-27T00:47:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:23 smithi119 ceph-mon[32752]: pgmap v12533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:23 smithi167 ceph-mon[32751]: pgmap v12533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:23 smithi090 ceph-mon[29180]: pgmap v12533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:25 smithi119 ceph-mon[32752]: pgmap v12534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:25 smithi167 ceph-mon[32751]: pgmap v12534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:25 smithi090 ceph-mon[29180]: pgmap v12534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:26 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22327.325739 secs 2024-04-27T00:47:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:26 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22327.325739 secs 2024-04-27T00:47:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:26 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22327.325739 secs 2024-04-27T00:47:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:27 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22328.123975 secs 2024-04-27T00:47:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:27 smithi167 ceph-mon[32751]: pgmap v12535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:27 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22328.123975 secs 2024-04-27T00:47:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:27 smithi119 ceph-mon[32752]: pgmap v12535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:27 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22328.123975 secs 2024-04-27T00:47:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:27 smithi090 ceph-mon[29180]: pgmap v12535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:29 smithi119 ceph-mon[32752]: pgmap v12536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:29 smithi167 ceph-mon[32751]: pgmap v12536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:29 smithi090 ceph-mon[29180]: pgmap v12536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:31 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22332.325822 secs 2024-04-27T00:47:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:31 smithi119 ceph-mon[32752]: pgmap v12537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:31 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22332.325822 secs 2024-04-27T00:47:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:31 smithi167 ceph-mon[32751]: pgmap v12537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:31 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22332.325822 secs 2024-04-27T00:47:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:31 smithi090 ceph-mon[29180]: pgmap v12537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:32 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22333.124084 secs 2024-04-27T00:47:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:32 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22333.124084 secs 2024-04-27T00:47:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:32 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22333.124084 secs 2024-04-27T00:47:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:33 smithi119 ceph-mon[32752]: pgmap v12538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:33 smithi167 ceph-mon[32751]: pgmap v12538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:33 smithi090 ceph-mon[29180]: pgmap v12538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:35 smithi119 ceph-mon[32752]: pgmap v12539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:35 smithi167 ceph-mon[32751]: pgmap v12539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:35 smithi090 ceph-mon[29180]: pgmap v12539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:36 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22337.325883 secs 2024-04-27T00:47:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:36 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22337.325883 secs 2024-04-27T00:47:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:36 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22337.325883 secs 2024-04-27T00:47:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:37 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22338.124204 secs 2024-04-27T00:47:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:37 smithi119 ceph-mon[32752]: pgmap v12540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:37 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22338.124204 secs 2024-04-27T00:47:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:37 smithi167 ceph-mon[32751]: pgmap v12540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:37 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22338.124204 secs 2024-04-27T00:47:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:37 smithi090 ceph-mon[29180]: pgmap v12540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:39 smithi119 ceph-mon[32752]: pgmap v12541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:39 smithi167 ceph-mon[32751]: pgmap v12541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:39 smithi090 ceph-mon[29180]: pgmap v12541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:41 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22342.325945 secs 2024-04-27T00:47:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:41 smithi119 ceph-mon[32752]: pgmap v12542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:41 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22342.325945 secs 2024-04-27T00:47:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:41 smithi167 ceph-mon[32751]: pgmap v12542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:41 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22342.325945 secs 2024-04-27T00:47:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:41 smithi090 ceph-mon[29180]: pgmap v12542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:42 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22343.124292 secs 2024-04-27T00:47:42.565 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:42 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22343.124292 secs 2024-04-27T00:47:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:42 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22343.124292 secs 2024-04-27T00:47:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:43 smithi119 ceph-mon[32752]: pgmap v12543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:43 smithi167 ceph-mon[32751]: pgmap v12543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:43 smithi090 ceph-mon[29180]: pgmap v12543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:45 smithi119 ceph-mon[32752]: pgmap v12544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:45 smithi167 ceph-mon[32751]: pgmap v12544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:45 smithi090 ceph-mon[29180]: pgmap v12544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:46 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22347.326028 secs 2024-04-27T00:47:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:46 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22347.326028 secs 2024-04-27T00:47:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:46 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22347.326028 secs 2024-04-27T00:47:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:47 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22348.124411 secs 2024-04-27T00:47:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:47 smithi119 ceph-mon[32752]: pgmap v12545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:47 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22348.124411 secs 2024-04-27T00:47:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:47 smithi167 ceph-mon[32751]: pgmap v12545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:47 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22348.124411 secs 2024-04-27T00:47:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:47 smithi090 ceph-mon[29180]: pgmap v12545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:49 smithi119 ceph-mon[32752]: pgmap v12546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:49 smithi167 ceph-mon[32751]: pgmap v12546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:49.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:49 smithi090 ceph-mon[29180]: pgmap v12546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:51 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22352.326117 secs 2024-04-27T00:47:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:51 smithi119 ceph-mon[32752]: pgmap v12547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:51 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22352.326117 secs 2024-04-27T00:47:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:51 smithi167 ceph-mon[32751]: pgmap v12547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:51 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22352.326117 secs 2024-04-27T00:47:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:51 smithi090 ceph-mon[29180]: pgmap v12547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:52 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22353.124526 secs 2024-04-27T00:47:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:52 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22353.124526 secs 2024-04-27T00:47:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:52 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22353.124526 secs 2024-04-27T00:47:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:53 smithi119 ceph-mon[32752]: pgmap v12548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:53 smithi167 ceph-mon[32751]: pgmap v12548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:53 smithi090 ceph-mon[29180]: pgmap v12548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:47:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:55 smithi119 ceph-mon[32752]: pgmap v12549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:47:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:47:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:47:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:47:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:55 smithi167 ceph-mon[32751]: pgmap v12549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:47:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:47:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:47:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:47:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:55 smithi090 ceph-mon[29180]: pgmap v12549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:47:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:47:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:47:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:47:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:47:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:56 smithi119 ceph-mon[32752]: pgmap v12550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-27T00:47:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:56 smithi119 ceph-mon[32752]: pgmap v12551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:47:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:56 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22357.326182 secs 2024-04-27T00:47:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:56 smithi167 ceph-mon[32751]: pgmap v12550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-27T00:47:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:56 smithi167 ceph-mon[32751]: pgmap v12551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:47:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:56 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22357.326182 secs 2024-04-27T00:47:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:56 smithi090 ceph-mon[29180]: pgmap v12550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-27T00:47:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:56 smithi090 ceph-mon[29180]: pgmap v12551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:47:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:56 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22357.326182 secs 2024-04-27T00:47:57.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:57 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22358.124642 secs 2024-04-27T00:47:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:57 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22358.124642 secs 2024-04-27T00:47:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:57 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22358.124642 secs 2024-04-27T00:47:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:47:58 smithi119 ceph-mon[32752]: pgmap v12552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:47:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:47:58 smithi167 ceph-mon[32751]: pgmap v12552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:47:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:47:58 smithi090 ceph-mon[29180]: pgmap v12552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:00 smithi119 ceph-mon[32752]: pgmap v12553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:00 smithi167 ceph-mon[32751]: pgmap v12553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:00 smithi090 ceph-mon[29180]: pgmap v12553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:01 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22362.326268 secs 2024-04-27T00:48:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:01 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22362.326268 secs 2024-04-27T00:48:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:01 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22362.326268 secs 2024-04-27T00:48:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:02 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22363.124756 secs 2024-04-27T00:48:02.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:02 smithi090 ceph-mon[29180]: pgmap v12554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:02 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22363.124756 secs 2024-04-27T00:48:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:02 smithi119 ceph-mon[32752]: pgmap v12554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:02 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22363.124756 secs 2024-04-27T00:48:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:02 smithi167 ceph-mon[32751]: pgmap v12554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:04 smithi119 ceph-mon[32752]: pgmap v12555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-27T00:48:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:04 smithi167 ceph-mon[32751]: pgmap v12555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-27T00:48:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:04 smithi090 ceph-mon[29180]: pgmap v12555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 577 B/s rd, 1 op/s 2024-04-27T00:48:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:06 smithi119 ceph-mon[32752]: pgmap v12556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:48:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:06 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22367.326349 secs 2024-04-27T00:48:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:06 smithi167 ceph-mon[32751]: pgmap v12556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:48:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:06 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22367.326349 secs 2024-04-27T00:48:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:06 smithi090 ceph-mon[29180]: pgmap v12556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:48:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:06 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22367.326349 secs 2024-04-27T00:48:07.288 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:48:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:48:06.916+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:48:07.288 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:48:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:48:06.916+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:48:07.288 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:48:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:48:06.916+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:48:07.289 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:48:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:48:06.916+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:48:07.289 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:48:06 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:48:06.916+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:48:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:07 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22368.124851 secs 2024-04-27T00:48:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:07 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22368.124851 secs 2024-04-27T00:48:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:07 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22368.124851 secs 2024-04-27T00:48:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:08 smithi119 ceph-mon[32752]: pgmap v12557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:08 smithi167 ceph-mon[32751]: pgmap v12557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:08 smithi090 ceph-mon[29180]: pgmap v12557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:10 smithi119 ceph-mon[32752]: pgmap v12558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:10 smithi167 ceph-mon[32751]: pgmap v12558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:10 smithi090 ceph-mon[29180]: pgmap v12558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:11 smithi119 ceph-mon[32752]: 29 slow requests, 1 included below; oldest blocked for > 22372.326434 secs 2024-04-27T00:48:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:11 smithi119 ceph-mon[32752]: slow request 2880.960632 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:48:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:11 smithi167 ceph-mon[32751]: 29 slow requests, 1 included below; oldest blocked for > 22372.326434 secs 2024-04-27T00:48:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:11 smithi167 ceph-mon[32751]: slow request 2880.960632 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:48:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:11 smithi090 ceph-mon[29180]: 29 slow requests, 1 included below; oldest blocked for > 22372.326434 secs 2024-04-27T00:48:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:11 smithi090 ceph-mon[29180]: slow request 2880.960632 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:48:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:12 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22373.124972 secs 2024-04-27T00:48:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:12 smithi119 ceph-mon[32752]: pgmap v12559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:12 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22373.124972 secs 2024-04-27T00:48:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:12 smithi167 ceph-mon[32751]: pgmap v12559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:12 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22373.124972 secs 2024-04-27T00:48:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:12 smithi090 ceph-mon[29180]: pgmap v12559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:14 smithi119 ceph-mon[32752]: pgmap v12560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:14 smithi167 ceph-mon[32751]: pgmap v12560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:14 smithi090 ceph-mon[29180]: pgmap v12560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:16 smithi167 ceph-mon[32751]: pgmap v12561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:16 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22377.326516 secs 2024-04-27T00:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:16 smithi090 ceph-mon[29180]: pgmap v12561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:16 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22377.326516 secs 2024-04-27T00:48:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:16 smithi119 ceph-mon[32752]: pgmap v12561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:16 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22377.326516 secs 2024-04-27T00:48:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:17 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22378.125091 secs 2024-04-27T00:48:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:17 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22378.125091 secs 2024-04-27T00:48:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:17 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22378.125091 secs 2024-04-27T00:48:18.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:18 smithi167 ceph-mon[32751]: pgmap v12562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:18 smithi090 ceph-mon[29180]: pgmap v12562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:18 smithi119 ceph-mon[32752]: pgmap v12562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:20 smithi090 ceph-mon[29180]: pgmap v12563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:20 smithi119 ceph-mon[32752]: pgmap v12563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:20 smithi167 ceph-mon[32751]: pgmap v12563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:48:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:21 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22382.326601 secs 2024-04-27T00:48:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:21 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22382.326601 secs 2024-04-27T00:48:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:21 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22382.326601 secs 2024-04-27T00:48:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:22 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22383.125185 secs 2024-04-27T00:48:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:22 smithi090 ceph-mon[29180]: pgmap v12564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:22 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22383.125185 secs 2024-04-27T00:48:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:22 smithi119 ceph-mon[32752]: pgmap v12564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:22 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22383.125185 secs 2024-04-27T00:48:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:22 smithi167 ceph-mon[32751]: pgmap v12564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:24 smithi090 ceph-mon[29180]: pgmap v12565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:24 smithi119 ceph-mon[32752]: pgmap v12565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:24 smithi167 ceph-mon[32751]: pgmap v12565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:26 smithi090 ceph-mon[29180]: pgmap v12566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:26 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22387.326684 secs 2024-04-27T00:48:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:26 smithi119 ceph-mon[32752]: pgmap v12566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:26 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22387.326684 secs 2024-04-27T00:48:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:26 smithi167 ceph-mon[32751]: pgmap v12566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:26 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22387.326684 secs 2024-04-27T00:48:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:27 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22388.125308 secs 2024-04-27T00:48:27.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:27 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22388.125308 secs 2024-04-27T00:48:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:27 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22388.125308 secs 2024-04-27T00:48:28.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:28 smithi090 ceph-mon[29180]: pgmap v12567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:28 smithi119 ceph-mon[32752]: pgmap v12567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:28 smithi167 ceph-mon[32751]: pgmap v12567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:30.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:30 smithi090 ceph-mon[29180]: pgmap v12568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:30 smithi119 ceph-mon[32752]: pgmap v12568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:30 smithi167 ceph-mon[32751]: pgmap v12568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:31 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22392.326768 secs 2024-04-27T00:48:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:31 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22392.326768 secs 2024-04-27T00:48:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:31 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22392.326768 secs 2024-04-27T00:48:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:32 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22393.125420 secs 2024-04-27T00:48:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:32 smithi090 ceph-mon[29180]: pgmap v12569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:32 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22393.125420 secs 2024-04-27T00:48:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:32 smithi119 ceph-mon[32752]: pgmap v12569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:32 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22393.125420 secs 2024-04-27T00:48:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:32 smithi167 ceph-mon[32751]: pgmap v12569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:34 smithi090 ceph-mon[29180]: pgmap v12570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:34 smithi119 ceph-mon[32752]: pgmap v12570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:34 smithi167 ceph-mon[32751]: pgmap v12570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:35 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22397.326823 secs 2024-04-27T00:48:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:35 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22397.326823 secs 2024-04-27T00:48:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:35 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22397.326823 secs 2024-04-27T00:48:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:36 smithi090 ceph-mon[29180]: pgmap v12571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:36 smithi119 ceph-mon[32752]: pgmap v12571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:36 smithi167 ceph-mon[32751]: pgmap v12571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:37 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22398.125533 secs 2024-04-27T00:48:37.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:37 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22398.125533 secs 2024-04-27T00:48:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:37 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22398.125533 secs 2024-04-27T00:48:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:38 smithi119 ceph-mon[32752]: pgmap v12572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:38 smithi167 ceph-mon[32751]: pgmap v12572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:38.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:38 smithi090 ceph-mon[29180]: pgmap v12572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:40 smithi119 ceph-mon[32752]: pgmap v12573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:40 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22402.326885 secs 2024-04-27T00:48:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:40 smithi167 ceph-mon[32751]: pgmap v12573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:40 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22402.326885 secs 2024-04-27T00:48:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:40 smithi090 ceph-mon[29180]: pgmap v12573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:40 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22402.326885 secs 2024-04-27T00:48:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:42 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 22403.125614 secs 2024-04-27T00:48:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:42 smithi119 ceph-mon[32752]: slow request 2884.698024 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:48:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:42 smithi119 ceph-mon[32752]: pgmap v12574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:42 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 22403.125614 secs 2024-04-27T00:48:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:42 smithi167 ceph-mon[32751]: slow request 2884.698024 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:48:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:42 smithi167 ceph-mon[32751]: pgmap v12574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:42 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 22403.125614 secs 2024-04-27T00:48:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:42 smithi090 ceph-mon[29180]: slow request 2884.698024 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:48:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:42 smithi090 ceph-mon[29180]: pgmap v12574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:43 smithi119 ceph-mon[32752]: pgmap v12575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:43 smithi167 ceph-mon[32751]: pgmap v12575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:43 smithi090 ceph-mon[29180]: pgmap v12575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:46 smithi119 ceph-mon[32752]: pgmap v12576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:46 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 22407.326949 secs 2024-04-27T00:48:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:46 smithi167 ceph-mon[32751]: pgmap v12576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:46 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 22407.326949 secs 2024-04-27T00:48:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:46 smithi090 ceph-mon[29180]: pgmap v12576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:46 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 22407.326949 secs 2024-04-27T00:48:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:47 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22408.125726 secs 2024-04-27T00:48:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:47 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22408.125726 secs 2024-04-27T00:48:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:47 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22408.125726 secs 2024-04-27T00:48:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:48 smithi119 ceph-mon[32752]: pgmap v12577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:48 smithi167 ceph-mon[32751]: pgmap v12577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:48 smithi090 ceph-mon[29180]: pgmap v12577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:50 smithi119 ceph-mon[32752]: pgmap v12578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:50 smithi167 ceph-mon[32751]: pgmap v12578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:50 smithi090 ceph-mon[29180]: pgmap v12578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:51 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 22412.327011 secs 2024-04-27T00:48:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:51 smithi119 ceph-mon[32752]: slow request 181.701099 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:48:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:51 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 22412.327011 secs 2024-04-27T00:48:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:51 smithi167 ceph-mon[32751]: slow request 181.701099 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:48:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:51 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 22412.327011 secs 2024-04-27T00:48:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:51 smithi090 ceph-mon[29180]: slow request 181.701099 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:48:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:52 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22413.125859 secs 2024-04-27T00:48:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:52 smithi119 ceph-mon[32752]: pgmap v12579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:52 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22413.125859 secs 2024-04-27T00:48:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:52 smithi167 ceph-mon[32751]: pgmap v12579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:52 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22413.125859 secs 2024-04-27T00:48:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:52 smithi090 ceph-mon[29180]: pgmap v12579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:54 smithi119 ceph-mon[32752]: pgmap v12580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:54 smithi167 ceph-mon[32751]: pgmap v12580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:54 smithi090 ceph-mon[29180]: pgmap v12580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:48:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:55 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:48:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:55 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:48:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:55 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:48:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:55 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:48:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:55 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T00:48:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:55 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:48:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:56 smithi167 ceph-mon[32751]: pgmap v12581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22417.327075 secs 2024-04-27T00:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:48:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:48:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:56 smithi090 ceph-mon[29180]: pgmap v12581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22417.327075 secs 2024-04-27T00:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:48:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:48:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:56 smithi119 ceph-mon[32752]: pgmap v12581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:48:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:48:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22417.327075 secs 2024-04-27T00:48:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:48:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:48:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:48:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:57 smithi167 ceph-mon[32751]: pgmap v12582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:48:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:57 smithi167 ceph-mon[32751]: pgmap v12583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:57 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22418.126005 secs 2024-04-27T00:48:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:57 smithi090 ceph-mon[29180]: pgmap v12582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:48:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:57 smithi090 ceph-mon[29180]: pgmap v12583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:57 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22418.126005 secs 2024-04-27T00:48:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:57 smithi119 ceph-mon[32752]: pgmap v12582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:48:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:57 smithi119 ceph-mon[32752]: pgmap v12583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:48:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:57 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22418.126005 secs 2024-04-27T00:48:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:48:59 smithi167 ceph-mon[32751]: pgmap v12584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:48:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:48:59 smithi090 ceph-mon[29180]: pgmap v12584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:48:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:48:59 smithi119 ceph-mon[32752]: pgmap v12584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:49:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:01 smithi167 ceph-mon[32751]: pgmap v12585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22422.327136 secs 2024-04-27T00:49:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:01 smithi090 ceph-mon[29180]: pgmap v12585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22422.327136 secs 2024-04-27T00:49:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:01 smithi119 ceph-mon[32752]: pgmap v12585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22422.327136 secs 2024-04-27T00:49:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:02 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22423.126118 secs 2024-04-27T00:49:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:02 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22423.126118 secs 2024-04-27T00:49:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:02 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22423.126118 secs 2024-04-27T00:49:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:03 smithi090 ceph-mon[29180]: pgmap v12586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:03 smithi119 ceph-mon[32752]: pgmap v12586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:03 smithi167 ceph-mon[32751]: pgmap v12586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:05 smithi090 ceph-mon[29180]: pgmap v12587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:05 smithi119 ceph-mon[32752]: pgmap v12587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:05 smithi167 ceph-mon[32751]: pgmap v12587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22427.327195 secs 2024-04-27T00:49:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22427.327195 secs 2024-04-27T00:49:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22427.327195 secs 2024-04-27T00:49:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:07 smithi090 ceph-mon[29180]: pgmap v12588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:49:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:07 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22428.126218 secs 2024-04-27T00:49:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:07 smithi119 ceph-mon[32752]: pgmap v12588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:49:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:07 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22428.126218 secs 2024-04-27T00:49:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:07 smithi167 ceph-mon[32751]: pgmap v12588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:49:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:07 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22428.126218 secs 2024-04-27T00:49:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:09 smithi090 ceph-mon[29180]: pgmap v12589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:09 smithi119 ceph-mon[32752]: pgmap v12589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:09 smithi167 ceph-mon[32751]: pgmap v12589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:11 smithi090 ceph-mon[29180]: pgmap v12590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:11 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 22432.327279 secs 2024-04-27T00:49:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:11 smithi090 ceph-mon[29180]: slow request 5764.765907 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:49:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:11 smithi119 ceph-mon[32752]: pgmap v12590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:11 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 22432.327279 secs 2024-04-27T00:49:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:11 smithi119 ceph-mon[32752]: slow request 5764.765907 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:49:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:11 smithi167 ceph-mon[32751]: pgmap v12590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:11 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 22432.327279 secs 2024-04-27T00:49:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:11 smithi167 ceph-mon[32751]: slow request 5764.765907 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:49:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:12 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 22433.126289 secs 2024-04-27T00:49:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:12 smithi090 ceph-mon[29180]: slow request 364.007422 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:49:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:12 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 22433.126289 secs 2024-04-27T00:49:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:12 smithi119 ceph-mon[32752]: slow request 364.007422 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:49:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:12 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 22433.126289 secs 2024-04-27T00:49:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:12 smithi167 ceph-mon[32751]: slow request 364.007422 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:49:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:13 smithi090 ceph-mon[29180]: pgmap v12591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:13 smithi119 ceph-mon[32752]: pgmap v12591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:13 smithi167 ceph-mon[32751]: pgmap v12591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:15 smithi090 ceph-mon[29180]: pgmap v12592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:15 smithi119 ceph-mon[32752]: pgmap v12592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:15 smithi167 ceph-mon[32751]: pgmap v12592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22437.327361 secs 2024-04-27T00:49:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22437.327361 secs 2024-04-27T00:49:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22437.327361 secs 2024-04-27T00:49:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:17 smithi090 ceph-mon[29180]: pgmap v12593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:17 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22438.126406 secs 2024-04-27T00:49:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:17 smithi119 ceph-mon[32752]: pgmap v12593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:17 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22438.126406 secs 2024-04-27T00:49:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:17 smithi167 ceph-mon[32751]: pgmap v12593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:17 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22438.126406 secs 2024-04-27T00:49:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:19 smithi090 ceph-mon[29180]: pgmap v12594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:19 smithi119 ceph-mon[32752]: pgmap v12594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:19 smithi167 ceph-mon[32751]: pgmap v12594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:21 smithi090 ceph-mon[29180]: pgmap v12595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22442.327444 secs 2024-04-27T00:49:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:21 smithi119 ceph-mon[32752]: pgmap v12595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22442.327444 secs 2024-04-27T00:49:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:21 smithi167 ceph-mon[32751]: pgmap v12595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22442.327444 secs 2024-04-27T00:49:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:22 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22443.126492 secs 2024-04-27T00:49:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:22 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22443.126492 secs 2024-04-27T00:49:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:22 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22443.126492 secs 2024-04-27T00:49:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:23 smithi090 ceph-mon[29180]: pgmap v12596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:23 smithi119 ceph-mon[32752]: pgmap v12596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:23 smithi167 ceph-mon[32751]: pgmap v12596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:25 smithi119 ceph-mon[32752]: pgmap v12597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:25 smithi167 ceph-mon[32751]: pgmap v12597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:25 smithi090 ceph-mon[29180]: pgmap v12597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22447.327525 secs 2024-04-27T00:49:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22447.327525 secs 2024-04-27T00:49:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22447.327525 secs 2024-04-27T00:49:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:27 smithi167 ceph-mon[32751]: pgmap v12598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:27 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22448.126559 secs 2024-04-27T00:49:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:27 smithi119 ceph-mon[32752]: pgmap v12598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:27 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22448.126559 secs 2024-04-27T00:49:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:27 smithi090 ceph-mon[29180]: pgmap v12598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:27 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22448.126559 secs 2024-04-27T00:49:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:29 smithi119 ceph-mon[32752]: pgmap v12599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:29 smithi167 ceph-mon[32751]: pgmap v12599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:29 smithi090 ceph-mon[29180]: pgmap v12599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:31 smithi119 ceph-mon[32752]: pgmap v12600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22452.327609 secs 2024-04-27T00:49:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:31 smithi167 ceph-mon[32751]: pgmap v12600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22452.327609 secs 2024-04-27T00:49:31.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:31 smithi090 ceph-mon[29180]: pgmap v12600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:31.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22452.327609 secs 2024-04-27T00:49:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:32 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22453.126640 secs 2024-04-27T00:49:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:32 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22453.126640 secs 2024-04-27T00:49:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:32 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22453.126640 secs 2024-04-27T00:49:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:33 smithi119 ceph-mon[32752]: pgmap v12601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:33 smithi167 ceph-mon[32751]: pgmap v12601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:33 smithi090 ceph-mon[29180]: pgmap v12601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:35 smithi119 ceph-mon[32752]: pgmap v12602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:35 smithi167 ceph-mon[32751]: pgmap v12602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:35 smithi090 ceph-mon[29180]: pgmap v12602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22457.327692 secs 2024-04-27T00:49:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22457.327692 secs 2024-04-27T00:49:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22457.327692 secs 2024-04-27T00:49:37.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:37 smithi119 ceph-mon[32752]: pgmap v12603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:37.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:37 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22458.126756 secs 2024-04-27T00:49:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:37 smithi167 ceph-mon[32751]: pgmap v12603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:37 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22458.126756 secs 2024-04-27T00:49:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:37 smithi090 ceph-mon[29180]: pgmap v12603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:37 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22458.126756 secs 2024-04-27T00:49:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:39 smithi119 ceph-mon[32752]: pgmap v12604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:39 smithi167 ceph-mon[32751]: pgmap v12604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:39 smithi090 ceph-mon[29180]: pgmap v12604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:41 smithi119 ceph-mon[32752]: pgmap v12605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22462.327777 secs 2024-04-27T00:49:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:41 smithi167 ceph-mon[32751]: pgmap v12605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22462.327777 secs 2024-04-27T00:49:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:41 smithi090 ceph-mon[29180]: pgmap v12605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22462.327777 secs 2024-04-27T00:49:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:42 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22463.126886 secs 2024-04-27T00:49:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:42 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22463.126886 secs 2024-04-27T00:49:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:42 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22463.126886 secs 2024-04-27T00:49:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:43 smithi119 ceph-mon[32752]: pgmap v12606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:43 smithi167 ceph-mon[32751]: pgmap v12606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:43 smithi090 ceph-mon[29180]: pgmap v12606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:45 smithi119 ceph-mon[32752]: pgmap v12607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:45 smithi167 ceph-mon[32751]: pgmap v12607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:45 smithi090 ceph-mon[29180]: pgmap v12607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:46.469 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:46 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 22467.327822 secs 2024-04-27T00:49:46.470 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:46 smithi090 ceph-mon[29180]: slow request 5760.097154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:49:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:46 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 22467.327822 secs 2024-04-27T00:49:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:46 smithi119 ceph-mon[32752]: slow request 5760.097154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:49:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:46 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 22467.327822 secs 2024-04-27T00:49:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:46 smithi167 ceph-mon[32751]: slow request 5760.097154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:49:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:47 smithi119 ceph-mon[32752]: pgmap v12608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:47 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22468.127003 secs 2024-04-27T00:49:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:47 smithi167 ceph-mon[32751]: pgmap v12608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:47 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22468.127003 secs 2024-04-27T00:49:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:47 smithi090 ceph-mon[29180]: pgmap v12608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:47 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22468.127003 secs 2024-04-27T00:49:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:49 smithi119 ceph-mon[32752]: pgmap v12609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:49 smithi167 ceph-mon[32751]: pgmap v12609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:49 smithi090 ceph-mon[29180]: pgmap v12609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:51 smithi119 ceph-mon[32752]: pgmap v12610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22472.327908 secs 2024-04-27T00:49:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:51 smithi167 ceph-mon[32751]: pgmap v12610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22472.327908 secs 2024-04-27T00:49:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:51 smithi090 ceph-mon[29180]: pgmap v12610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22472.327908 secs 2024-04-27T00:49:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:52 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22473.127115 secs 2024-04-27T00:49:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:52 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22473.127115 secs 2024-04-27T00:49:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:52 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22473.127115 secs 2024-04-27T00:49:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:53 smithi119 ceph-mon[32752]: pgmap v12611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:53 smithi167 ceph-mon[32751]: pgmap v12611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:53 smithi090 ceph-mon[29180]: pgmap v12611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:55 smithi119 ceph-mon[32752]: pgmap v12612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:55 smithi167 ceph-mon[32751]: pgmap v12612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:55 smithi090 ceph-mon[29180]: pgmap v12612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:49:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22477.327996 secs 2024-04-27T00:49:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:49:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22477.327996 secs 2024-04-27T00:49:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:49:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22477.327996 secs 2024-04-27T00:49:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:49:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:57 smithi167 ceph-mon[32751]: pgmap v12613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:57 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22478.127184 secs 2024-04-27T00:49:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:49:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:49:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:57 smithi167 ceph-mon[32751]: pgmap v12614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:49:57.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:57 smithi167 ceph-mon[32751]: pgmap v12615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:57.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:49:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:57 smithi119 ceph-mon[32752]: pgmap v12613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:57 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22478.127184 secs 2024-04-27T00:49:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:49:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:49:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:57 smithi119 ceph-mon[32752]: pgmap v12614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:49:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:57 smithi119 ceph-mon[32752]: pgmap v12615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:57 smithi090 ceph-mon[29180]: pgmap v12613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:57 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22478.127184 secs 2024-04-27T00:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:57 smithi090 ceph-mon[29180]: pgmap v12614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:57 smithi090 ceph-mon[29180]: pgmap v12615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:49:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:49:58 smithi167 ceph-mon[32751]: pgmap v12616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:49:58 smithi090 ceph-mon[29180]: pgmap v12616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:49:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:49:58 smithi119 ceph-mon[32752]: pgmap v12616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:50:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:50:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:50:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T00:50:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22482.328059 secs 2024-04-27T00:50:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:01 smithi167 ceph-mon[32751]: pgmap v12617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:50:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22482.328059 secs 2024-04-27T00:50:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:01 smithi090 ceph-mon[29180]: pgmap v12617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:50:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22482.328059 secs 2024-04-27T00:50:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:01 smithi119 ceph-mon[32752]: pgmap v12617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:50:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:02 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22483.127303 secs 2024-04-27T00:50:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:02 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22483.127303 secs 2024-04-27T00:50:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:02 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22483.127303 secs 2024-04-27T00:50:03.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:03 smithi090 ceph-mon[29180]: pgmap v12618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:50:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:03 smithi119 ceph-mon[32752]: pgmap v12618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:50:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:03 smithi167 ceph-mon[32751]: pgmap v12618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:50:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:04 smithi119 ceph-mon[32752]: pgmap v12619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:50:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:04 smithi167 ceph-mon[32751]: pgmap v12619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:50:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:04 smithi090 ceph-mon[29180]: pgmap v12619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:50:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22487.328145 secs 2024-04-27T00:50:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22487.328145 secs 2024-04-27T00:50:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22487.328145 secs 2024-04-27T00:50:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:06 smithi090 ceph-mon[29180]: pgmap v12620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:50:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:06 smithi119 ceph-mon[32752]: pgmap v12620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:50:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:06 smithi167 ceph-mon[32751]: pgmap v12620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:50:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:07 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22488.127454 secs 2024-04-27T00:50:07.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:07 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22488.127454 secs 2024-04-27T00:50:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:07 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22488.127454 secs 2024-04-27T00:50:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:08 smithi090 ceph-mon[29180]: pgmap v12621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:08 smithi119 ceph-mon[32752]: pgmap v12621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:08 smithi167 ceph-mon[32751]: pgmap v12621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22492.328228 secs 2024-04-27T00:50:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22492.328228 secs 2024-04-27T00:50:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22492.328228 secs 2024-04-27T00:50:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:11 smithi119 ceph-mon[32752]: pgmap v12622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:11 smithi167 ceph-mon[32751]: pgmap v12622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:11 smithi090 ceph-mon[29180]: pgmap v12622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:12 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22493.127517 secs 2024-04-27T00:50:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:12 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22493.127517 secs 2024-04-27T00:50:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:12 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22493.127517 secs 2024-04-27T00:50:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:13 smithi119 ceph-mon[32752]: pgmap v12623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:13 smithi167 ceph-mon[32751]: pgmap v12623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:13 smithi090 ceph-mon[29180]: pgmap v12623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:14 smithi090 ceph-mon[29180]: pgmap v12624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:14 smithi119 ceph-mon[32752]: pgmap v12624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:14 smithi167 ceph-mon[32751]: pgmap v12624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22497.328289 secs 2024-04-27T00:50:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22497.328289 secs 2024-04-27T00:50:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22497.328289 secs 2024-04-27T00:50:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:16 smithi090 ceph-mon[29180]: pgmap v12625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:16 smithi119 ceph-mon[32752]: pgmap v12625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:16 smithi167 ceph-mon[32751]: pgmap v12625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:17 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22498.127586 secs 2024-04-27T00:50:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:17 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22498.127586 secs 2024-04-27T00:50:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:17 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22498.127586 secs 2024-04-27T00:50:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:18 smithi090 ceph-mon[29180]: pgmap v12626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:19.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:18 smithi119 ceph-mon[32752]: pgmap v12626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:18 smithi167 ceph-mon[32751]: pgmap v12626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22502.328371 secs 2024-04-27T00:50:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22502.328371 secs 2024-04-27T00:50:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22502.328371 secs 2024-04-27T00:50:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:21 smithi119 ceph-mon[32752]: pgmap v12627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:21 smithi167 ceph-mon[32751]: pgmap v12627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:21 smithi090 ceph-mon[29180]: pgmap v12627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:22 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22503.127665 secs 2024-04-27T00:50:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:22 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22503.127665 secs 2024-04-27T00:50:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:22 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22503.127665 secs 2024-04-27T00:50:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:23 smithi119 ceph-mon[32752]: pgmap v12628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:23 smithi167 ceph-mon[32751]: pgmap v12628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:23 smithi090 ceph-mon[29180]: pgmap v12628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:24 smithi090 ceph-mon[29180]: pgmap v12629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:24 smithi119 ceph-mon[32752]: pgmap v12629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:24 smithi167 ceph-mon[32751]: pgmap v12629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22507.328456 secs 2024-04-27T00:50:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22507.328456 secs 2024-04-27T00:50:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22507.328456 secs 2024-04-27T00:50:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:26 smithi090 ceph-mon[29180]: pgmap v12630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:26 smithi119 ceph-mon[32752]: pgmap v12630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:26 smithi167 ceph-mon[32751]: pgmap v12630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:27 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22508.127728 secs 2024-04-27T00:50:27.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:27 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22508.127728 secs 2024-04-27T00:50:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:27 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22508.127728 secs 2024-04-27T00:50:28.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:28 smithi090 ceph-mon[29180]: pgmap v12631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:29.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:28 smithi119 ceph-mon[32752]: pgmap v12631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:28 smithi167 ceph-mon[32751]: pgmap v12631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22512.328537 secs 2024-04-27T00:50:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22512.328537 secs 2024-04-27T00:50:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22512.328537 secs 2024-04-27T00:50:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:31 smithi119 ceph-mon[32752]: pgmap v12632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:31 smithi167 ceph-mon[32751]: pgmap v12632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:31 smithi090 ceph-mon[29180]: pgmap v12632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:32 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22513.127797 secs 2024-04-27T00:50:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:32 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22513.127797 secs 2024-04-27T00:50:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:32 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22513.127797 secs 2024-04-27T00:50:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:33 smithi119 ceph-mon[32752]: pgmap v12633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:33 smithi167 ceph-mon[32751]: pgmap v12633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:33 smithi090 ceph-mon[29180]: pgmap v12633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:34 smithi090 ceph-mon[29180]: pgmap v12634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:35.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:34 smithi119 ceph-mon[32752]: pgmap v12634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:34 smithi167 ceph-mon[32751]: pgmap v12634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22517.328619 secs 2024-04-27T00:50:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22517.328619 secs 2024-04-27T00:50:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22517.328619 secs 2024-04-27T00:50:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:36 smithi090 ceph-mon[29180]: pgmap v12635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:36 smithi119 ceph-mon[32752]: pgmap v12635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:36 smithi167 ceph-mon[32751]: pgmap v12635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:37 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22518.127879 secs 2024-04-27T00:50:37.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:37 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22518.127879 secs 2024-04-27T00:50:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:37 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22518.127879 secs 2024-04-27T00:50:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:38 smithi090 ceph-mon[29180]: pgmap v12636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:39.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:38 smithi119 ceph-mon[32752]: pgmap v12636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:38 smithi167 ceph-mon[32751]: pgmap v12636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22522.328704 secs 2024-04-27T00:50:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22522.328704 secs 2024-04-27T00:50:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22522.328704 secs 2024-04-27T00:50:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:41 smithi119 ceph-mon[32752]: pgmap v12637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:41 smithi167 ceph-mon[32751]: pgmap v12637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:41 smithi090 ceph-mon[29180]: pgmap v12637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:42 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22523.127955 secs 2024-04-27T00:50:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:42 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22523.127955 secs 2024-04-27T00:50:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:42 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22523.127955 secs 2024-04-27T00:50:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:43 smithi119 ceph-mon[32752]: pgmap v12638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:43 smithi167 ceph-mon[32751]: pgmap v12638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:43 smithi090 ceph-mon[29180]: pgmap v12638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:44 smithi090 ceph-mon[29180]: pgmap v12639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:45.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:44 smithi119 ceph-mon[32752]: pgmap v12639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:44 smithi167 ceph-mon[32751]: pgmap v12639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22527.328794 secs 2024-04-27T00:50:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22527.328794 secs 2024-04-27T00:50:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22527.328794 secs 2024-04-27T00:50:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:47 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22528.128072 secs 2024-04-27T00:50:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:47 smithi090 ceph-mon[29180]: pgmap v12640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:47 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22528.128072 secs 2024-04-27T00:50:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:47 smithi119 ceph-mon[32752]: pgmap v12640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:47 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22528.128072 secs 2024-04-27T00:50:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:47 smithi167 ceph-mon[32751]: pgmap v12640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:49.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:49 smithi090 ceph-mon[29180]: pgmap v12641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:49 smithi119 ceph-mon[32752]: pgmap v12641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:50.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:49 smithi167 ceph-mon[32751]: pgmap v12641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22532.328826 secs 2024-04-27T00:50:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22532.328826 secs 2024-04-27T00:50:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22532.328826 secs 2024-04-27T00:50:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:51 smithi090 ceph-mon[29180]: pgmap v12642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:51 smithi119 ceph-mon[32752]: pgmap v12642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:51 smithi167 ceph-mon[32751]: pgmap v12642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:52 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22533.128182 secs 2024-04-27T00:50:53.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:52 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22533.128182 secs 2024-04-27T00:50:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:52 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22533.128182 secs 2024-04-27T00:50:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:53 smithi090 ceph-mon[29180]: pgmap v12643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:54.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:53 smithi119 ceph-mon[32752]: pgmap v12643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:54.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:53 smithi167 ceph-mon[32751]: pgmap v12643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:55 smithi090 ceph-mon[29180]: pgmap v12644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22537.328887 secs 2024-04-27T00:50:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:55 smithi119 ceph-mon[32752]: pgmap v12644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22537.328887 secs 2024-04-27T00:50:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:55 smithi167 ceph-mon[32751]: pgmap v12644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:50:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22537.328887 secs 2024-04-27T00:50:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:57 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22538.128300 secs 2024-04-27T00:50:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:57 smithi167 ceph-mon[32751]: pgmap v12645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:50:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:57 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22538.128300 secs 2024-04-27T00:50:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:57 smithi090 ceph-mon[29180]: pgmap v12645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:50:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:58.020 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:57 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22538.128300 secs 2024-04-27T00:50:58.020 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:57 smithi119 ceph-mon[32752]: pgmap v12645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:50:58.021 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:50:58.021 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:58.021 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:58.021 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:50:58.021 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:59 smithi119 ceph-mon[32752]: pgmap v12646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:50:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:59 smithi167 ceph-mon[32751]: pgmap v12646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:50:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:59 smithi090 ceph-mon[29180]: pgmap v12646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:50:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22542.328954 secs 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:51:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:51:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:51:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: pgmap v12647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T00:51:01.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: pgmap v12648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:01.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22542.328954 secs 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:51:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:51:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:51:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:51:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: pgmap v12647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T00:51:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: pgmap v12648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22542.328954 secs 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T00:51:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T00:51:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: pgmap v12647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T00:51:01.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: pgmap v12648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:01.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:51:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:02 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22543.128416 secs 2024-04-27T00:51:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:02 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22543.128416 secs 2024-04-27T00:51:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:02 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22543.128416 secs 2024-04-27T00:51:03.781 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:03 smithi090 ceph-mon[29180]: pgmap v12649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:03.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:03 smithi119 ceph-mon[32752]: pgmap v12649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:03 smithi167 ceph-mon[32751]: pgmap v12649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:04 smithi090 ceph-mon[29180]: pgmap v12650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:05.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:04 smithi119 ceph-mon[32752]: pgmap v12650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:05.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:04 smithi167 ceph-mon[32751]: pgmap v12650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22547.329039 secs 2024-04-27T00:51:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22547.329039 secs 2024-04-27T00:51:06.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22547.329039 secs 2024-04-27T00:51:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:06 smithi090 ceph-mon[29180]: pgmap v12651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:07.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:06 smithi119 ceph-mon[32752]: pgmap v12651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:06 smithi167 ceph-mon[32751]: pgmap v12651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:07 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22548.128497 secs 2024-04-27T00:51:07.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:07 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22548.128497 secs 2024-04-27T00:51:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:07 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22548.128497 secs 2024-04-27T00:51:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:08 smithi090 ceph-mon[29180]: pgmap v12652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:08.966 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:08 smithi119 ceph-mon[32752]: pgmap v12652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:08 smithi167 ceph-mon[32751]: pgmap v12652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:51:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22552.329133 secs 2024-04-27T00:51:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22552.329133 secs 2024-04-27T00:51:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22552.329133 secs 2024-04-27T00:51:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:11 smithi119 ceph-mon[32752]: pgmap v12653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:51:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:11 smithi167 ceph-mon[32751]: pgmap v12653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:51:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:11 smithi090 ceph-mon[29180]: pgmap v12653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:51:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:12 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22553.128574 secs 2024-04-27T00:51:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:12 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22553.128574 secs 2024-04-27T00:51:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:12 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22553.128574 secs 2024-04-27T00:51:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:13 smithi119 ceph-mon[32752]: pgmap v12654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:13 smithi167 ceph-mon[32751]: pgmap v12654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:13 smithi090 ceph-mon[29180]: pgmap v12654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:14 smithi090 ceph-mon[29180]: pgmap v12655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:15.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:14 smithi119 ceph-mon[32752]: pgmap v12655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:15.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:14 smithi167 ceph-mon[32751]: pgmap v12655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22557.329217 secs 2024-04-27T00:51:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22557.329217 secs 2024-04-27T00:51:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22557.329217 secs 2024-04-27T00:51:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:16 smithi090 ceph-mon[29180]: pgmap v12656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:17.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:16 smithi119 ceph-mon[32752]: pgmap v12656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:17.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:16 smithi167 ceph-mon[32751]: pgmap v12656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:17 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22558.128654 secs 2024-04-27T00:51:18.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:17 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22558.128654 secs 2024-04-27T00:51:18.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:17 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22558.128654 secs 2024-04-27T00:51:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:18 smithi090 ceph-mon[29180]: pgmap v12657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:19.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:18 smithi119 ceph-mon[32752]: pgmap v12657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:19.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:18 smithi167 ceph-mon[32751]: pgmap v12657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22562.329277 secs 2024-04-27T00:51:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22562.329277 secs 2024-04-27T00:51:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22562.329277 secs 2024-04-27T00:51:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:21 smithi119 ceph-mon[32752]: pgmap v12658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:21 smithi167 ceph-mon[32751]: pgmap v12658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:21 smithi090 ceph-mon[29180]: pgmap v12658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:22 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22563.128733 secs 2024-04-27T00:51:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:22 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22563.128733 secs 2024-04-27T00:51:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:22 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22563.128733 secs 2024-04-27T00:51:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:23 smithi119 ceph-mon[32752]: pgmap v12659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:23 smithi167 ceph-mon[32751]: pgmap v12659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:23 smithi090 ceph-mon[29180]: pgmap v12659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:24.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:24 smithi090 ceph-mon[29180]: pgmap v12660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:25.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:24 smithi119 ceph-mon[32752]: pgmap v12660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:25.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:24 smithi167 ceph-mon[32751]: pgmap v12660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22567.329362 secs 2024-04-27T00:51:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22567.329362 secs 2024-04-27T00:51:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22567.329362 secs 2024-04-27T00:51:26.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:26 smithi090 ceph-mon[29180]: pgmap v12661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:26 smithi119 ceph-mon[32752]: pgmap v12661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:26 smithi167 ceph-mon[32751]: pgmap v12661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:27 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22568.128795 secs 2024-04-27T00:51:27.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:27 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22568.128795 secs 2024-04-27T00:51:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:27 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22568.128795 secs 2024-04-27T00:51:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:28 smithi090 ceph-mon[29180]: pgmap v12662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:29.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:28 smithi119 ceph-mon[32752]: pgmap v12662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:29.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:28 smithi167 ceph-mon[32751]: pgmap v12662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22572.329444 secs 2024-04-27T00:51:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22572.329444 secs 2024-04-27T00:51:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22572.329444 secs 2024-04-27T00:51:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:31 smithi119 ceph-mon[32752]: pgmap v12663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:31 smithi167 ceph-mon[32751]: pgmap v12663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:31 smithi090 ceph-mon[29180]: pgmap v12663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:32 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22573.128876 secs 2024-04-27T00:51:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:32 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22573.128876 secs 2024-04-27T00:51:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:32 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22573.128876 secs 2024-04-27T00:51:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:33 smithi119 ceph-mon[32752]: pgmap v12664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:33 smithi167 ceph-mon[32751]: pgmap v12664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:33 smithi090 ceph-mon[29180]: pgmap v12664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:34 smithi090 ceph-mon[29180]: pgmap v12665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:35.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:34 smithi119 ceph-mon[32752]: pgmap v12665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:35.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:34 smithi167 ceph-mon[32751]: pgmap v12665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:35.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22577.329529 secs 2024-04-27T00:51:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22577.329529 secs 2024-04-27T00:51:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22577.329529 secs 2024-04-27T00:51:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:36 smithi090 ceph-mon[29180]: pgmap v12666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:36 smithi119 ceph-mon[32752]: pgmap v12666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:36 smithi167 ceph-mon[32751]: pgmap v12666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:37 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22578.129002 secs 2024-04-27T00:51:37.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:37 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22578.129002 secs 2024-04-27T00:51:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:37 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22578.129002 secs 2024-04-27T00:51:38.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:38 smithi090 ceph-mon[29180]: pgmap v12667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:39.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:38 smithi119 ceph-mon[32752]: pgmap v12667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:39.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:38 smithi167 ceph-mon[32751]: pgmap v12667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22582.329622 secs 2024-04-27T00:51:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22582.329622 secs 2024-04-27T00:51:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22582.329622 secs 2024-04-27T00:51:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:41 smithi119 ceph-mon[32752]: pgmap v12668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:41 smithi167 ceph-mon[32751]: pgmap v12668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:41 smithi090 ceph-mon[29180]: pgmap v12668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:42 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22583.129134 secs 2024-04-27T00:51:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:42 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22583.129134 secs 2024-04-27T00:51:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:42 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22583.129134 secs 2024-04-27T00:51:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:43 smithi119 ceph-mon[32752]: pgmap v12669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:43 smithi167 ceph-mon[32751]: pgmap v12669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:43.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:43 smithi090 ceph-mon[29180]: pgmap v12669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:44.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:44 smithi090 ceph-mon[29180]: pgmap v12670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:44 smithi119 ceph-mon[32752]: pgmap v12670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:44 smithi167 ceph-mon[32751]: pgmap v12670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22587.329708 secs 2024-04-27T00:51:46.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22587.329708 secs 2024-04-27T00:51:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22587.329708 secs 2024-04-27T00:51:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:46 smithi090 ceph-mon[29180]: pgmap v12671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:46 smithi119 ceph-mon[32752]: pgmap v12671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:47.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:46 smithi167 ceph-mon[32751]: pgmap v12671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:47 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22588.129271 secs 2024-04-27T00:51:48.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:47 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22588.129271 secs 2024-04-27T00:51:48.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:47 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22588.129271 secs 2024-04-27T00:51:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:48 smithi090 ceph-mon[29180]: pgmap v12672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:49.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:48 smithi119 ceph-mon[32752]: pgmap v12672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:48 smithi167 ceph-mon[32751]: pgmap v12672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:50 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 22592.329813 secs 2024-04-27T00:51:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:50 smithi119 ceph-mon[32752]: slow request 361.703901 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:51:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:50 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 22592.329813 secs 2024-04-27T00:51:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:50 smithi167 ceph-mon[32751]: slow request 361.703901 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:51:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:50 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 22592.329813 secs 2024-04-27T00:51:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:50 smithi090 ceph-mon[29180]: slow request 361.703901 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:51:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:51 smithi119 ceph-mon[32752]: pgmap v12673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:51 smithi167 ceph-mon[32751]: pgmap v12673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:51 smithi090 ceph-mon[29180]: pgmap v12673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:52 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22593.129343 secs 2024-04-27T00:51:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:52 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22593.129343 secs 2024-04-27T00:51:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:52 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22593.129343 secs 2024-04-27T00:51:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:53 smithi119 ceph-mon[32752]: pgmap v12674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:53 smithi167 ceph-mon[32751]: pgmap v12674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:53 smithi090 ceph-mon[29180]: pgmap v12674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:54 smithi090 ceph-mon[29180]: pgmap v12675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:55.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:54 smithi119 ceph-mon[32752]: pgmap v12675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:55.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:54 smithi167 ceph-mon[32751]: pgmap v12675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22597.329895 secs 2024-04-27T00:51:56.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22597.329895 secs 2024-04-27T00:51:56.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22597.329895 secs 2024-04-27T00:51:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:56 smithi090 ceph-mon[29180]: pgmap v12676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:57.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:56 smithi119 ceph-mon[32752]: pgmap v12676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:57.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:56 smithi167 ceph-mon[32751]: pgmap v12676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:51:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:57 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22598.129418 secs 2024-04-27T00:51:57.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:57 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22598.129418 secs 2024-04-27T00:51:58.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:57 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22598.129418 secs 2024-04-27T00:51:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:51:58 smithi090 ceph-mon[29180]: pgmap v12677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:59.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:51:58 smithi119 ceph-mon[32752]: pgmap v12677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:51:59.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:51:58 smithi167 ceph-mon[32751]: pgmap v12677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22602.329957 secs 2024-04-27T00:52:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22602.329957 secs 2024-04-27T00:52:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22602.329957 secs 2024-04-27T00:52:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:01 smithi119 ceph-mon[32752]: pgmap v12678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:52:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:01 smithi167 ceph-mon[32751]: pgmap v12678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:01 smithi090 ceph-mon[29180]: pgmap v12678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:52:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:02.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:02 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22603.129482 secs 2024-04-27T00:52:02.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:52:02.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:52:02.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:02 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22603.129482 secs 2024-04-27T00:52:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:52:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:52:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:02 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22603.129482 secs 2024-04-27T00:52:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:52:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:52:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:52:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:03 smithi119 ceph-mon[32752]: pgmap v12679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:52:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:03 smithi119 ceph-mon[32752]: pgmap v12680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:03 smithi167 ceph-mon[32751]: pgmap v12679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:52:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:03 smithi167 ceph-mon[32751]: pgmap v12680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:03 smithi090 ceph-mon[29180]: pgmap v12679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T00:52:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:03 smithi090 ceph-mon[29180]: pgmap v12680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:05 smithi119 ceph-mon[32752]: pgmap v12681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22607.330019 secs 2024-04-27T00:52:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:05 smithi167 ceph-mon[32751]: pgmap v12681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22607.330019 secs 2024-04-27T00:52:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:05 smithi090 ceph-mon[29180]: pgmap v12681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22607.330019 secs 2024-04-27T00:52:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:07 smithi119 ceph-mon[32752]: pgmap v12682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:07 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22608.129550 secs 2024-04-27T00:52:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:07 smithi167 ceph-mon[32751]: pgmap v12682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:07 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22608.129550 secs 2024-04-27T00:52:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:07 smithi090 ceph-mon[29180]: pgmap v12682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:07 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22608.129550 secs 2024-04-27T00:52:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:09 smithi119 ceph-mon[32752]: pgmap v12683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:52:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:09 smithi167 ceph-mon[32751]: pgmap v12683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:52:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:09 smithi090 ceph-mon[29180]: pgmap v12683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T00:52:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22612.330079 secs 2024-04-27T00:52:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22612.330079 secs 2024-04-27T00:52:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22612.330079 secs 2024-04-27T00:52:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:11 smithi119 ceph-mon[32752]: pgmap v12684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:11 smithi167 ceph-mon[32751]: pgmap v12684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:11 smithi090 ceph-mon[29180]: pgmap v12684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:52:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:12 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22613.129614 secs 2024-04-27T00:52:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:12 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22613.129614 secs 2024-04-27T00:52:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:12 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22613.129614 secs 2024-04-27T00:52:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:13 smithi119 ceph-mon[32752]: pgmap v12685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:52:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:13 smithi167 ceph-mon[32751]: pgmap v12685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:52:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:13 smithi090 ceph-mon[29180]: pgmap v12685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:52:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:15 smithi119 ceph-mon[32752]: pgmap v12686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22617.330140 secs 2024-04-27T00:52:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:15 smithi167 ceph-mon[32751]: pgmap v12686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22617.330140 secs 2024-04-27T00:52:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:15 smithi090 ceph-mon[29180]: pgmap v12686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22617.330140 secs 2024-04-27T00:52:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:17 smithi119 ceph-mon[32752]: pgmap v12687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:17 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22618.129694 secs 2024-04-27T00:52:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:17 smithi167 ceph-mon[32751]: pgmap v12687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:17 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22618.129694 secs 2024-04-27T00:52:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:17 smithi090 ceph-mon[29180]: pgmap v12687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:17 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22618.129694 secs 2024-04-27T00:52:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:19 smithi119 ceph-mon[32752]: pgmap v12688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:19.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:19 smithi167 ceph-mon[32751]: pgmap v12688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:19 smithi090 ceph-mon[29180]: pgmap v12688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22622.330200 secs 2024-04-27T00:52:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22622.330200 secs 2024-04-27T00:52:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22622.330200 secs 2024-04-27T00:52:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:21 smithi119 ceph-mon[32752]: pgmap v12689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:21 smithi167 ceph-mon[32751]: pgmap v12689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:21 smithi090 ceph-mon[29180]: pgmap v12689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:22 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22623.129770 secs 2024-04-27T00:52:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:22 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22623.129770 secs 2024-04-27T00:52:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:22 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22623.129770 secs 2024-04-27T00:52:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:23 smithi119 ceph-mon[32752]: pgmap v12690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:23 smithi167 ceph-mon[32751]: pgmap v12690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:23 smithi090 ceph-mon[29180]: pgmap v12690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:25 smithi119 ceph-mon[32752]: pgmap v12691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22627.330260 secs 2024-04-27T00:52:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:25 smithi167 ceph-mon[32751]: pgmap v12691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22627.330260 secs 2024-04-27T00:52:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:25 smithi090 ceph-mon[29180]: pgmap v12691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22627.330260 secs 2024-04-27T00:52:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:27 smithi119 ceph-mon[32752]: pgmap v12692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:27 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22628.129852 secs 2024-04-27T00:52:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:27 smithi167 ceph-mon[32751]: pgmap v12692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:27 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22628.129852 secs 2024-04-27T00:52:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:27 smithi090 ceph-mon[29180]: pgmap v12692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:27 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22628.129852 secs 2024-04-27T00:52:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:29 smithi119 ceph-mon[32752]: pgmap v12693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:29 smithi167 ceph-mon[32751]: pgmap v12693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:29 smithi090 ceph-mon[29180]: pgmap v12693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22632.330344 secs 2024-04-27T00:52:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22632.330344 secs 2024-04-27T00:52:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22632.330344 secs 2024-04-27T00:52:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:31 smithi119 ceph-mon[32752]: pgmap v12694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:31 smithi167 ceph-mon[32751]: pgmap v12694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:31 smithi090 ceph-mon[29180]: pgmap v12694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:32.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:32 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22633.129928 secs 2024-04-27T00:52:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:32 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22633.129928 secs 2024-04-27T00:52:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:32 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22633.129928 secs 2024-04-27T00:52:33.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:33 smithi119 ceph-mon[32752]: pgmap v12695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:33 smithi167 ceph-mon[32751]: pgmap v12695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:33 smithi090 ceph-mon[29180]: pgmap v12695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:35 smithi119 ceph-mon[32752]: pgmap v12696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22637.330428 secs 2024-04-27T00:52:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:35 smithi167 ceph-mon[32751]: pgmap v12696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22637.330428 secs 2024-04-27T00:52:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:35 smithi090 ceph-mon[29180]: pgmap v12696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22637.330428 secs 2024-04-27T00:52:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:37 smithi119 ceph-mon[32752]: pgmap v12697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:37 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22638.130041 secs 2024-04-27T00:52:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:37 smithi167 ceph-mon[32751]: pgmap v12697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:37 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22638.130041 secs 2024-04-27T00:52:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:37 smithi090 ceph-mon[29180]: pgmap v12697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:37 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22638.130041 secs 2024-04-27T00:52:39.789 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:39 smithi090 ceph-mon[29180]: pgmap v12698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:39.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:39 smithi119 ceph-mon[32752]: pgmap v12698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:39 smithi167 ceph-mon[32751]: pgmap v12698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22642.330517 secs 2024-04-27T00:52:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22642.330517 secs 2024-04-27T00:52:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22642.330517 secs 2024-04-27T00:52:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:41 smithi119 ceph-mon[32752]: pgmap v12699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:41 smithi167 ceph-mon[32751]: pgmap v12699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:41 smithi090 ceph-mon[29180]: pgmap v12699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:42 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22643.130123 secs 2024-04-27T00:52:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:42 smithi119 ceph-mon[32752]: pgmap v12700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:42 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22643.130123 secs 2024-04-27T00:52:42.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:42 smithi167 ceph-mon[32751]: pgmap v12700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:42 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22643.130123 secs 2024-04-27T00:52:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:42 smithi090 ceph-mon[29180]: pgmap v12700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:45.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:44 smithi119 ceph-mon[32752]: pgmap v12701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:45.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:44 smithi167 ceph-mon[32751]: pgmap v12701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:44 smithi090 ceph-mon[29180]: pgmap v12701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22647.330601 secs 2024-04-27T00:52:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22647.330601 secs 2024-04-27T00:52:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22647.330601 secs 2024-04-27T00:52:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:46 smithi167 ceph-mon[32751]: pgmap v12702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:46 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22648.130265 secs 2024-04-27T00:52:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:46 smithi090 ceph-mon[29180]: pgmap v12702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:46 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22648.130265 secs 2024-04-27T00:52:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:46 smithi119 ceph-mon[32752]: pgmap v12702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:46 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22648.130265 secs 2024-04-27T00:52:49.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:48 smithi167 ceph-mon[32751]: pgmap v12703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:49.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:48 smithi090 ceph-mon[29180]: pgmap v12703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:48 smithi119 ceph-mon[32752]: pgmap v12703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:50 smithi167 ceph-mon[32751]: pgmap v12704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22652.330685 secs 2024-04-27T00:52:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:50 smithi090 ceph-mon[29180]: pgmap v12704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22652.330685 secs 2024-04-27T00:52:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:50 smithi119 ceph-mon[32752]: pgmap v12704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22652.330685 secs 2024-04-27T00:52:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:51 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22653.130420 secs 2024-04-27T00:52:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:51 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22653.130420 secs 2024-04-27T00:52:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:51 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22653.130420 secs 2024-04-27T00:52:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:52 smithi090 ceph-mon[29180]: pgmap v12705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:52 smithi119 ceph-mon[32752]: pgmap v12705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:52 smithi167 ceph-mon[32751]: pgmap v12705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:54 smithi090 ceph-mon[29180]: pgmap v12706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:54 smithi119 ceph-mon[32752]: pgmap v12706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:54 smithi167 ceph-mon[32751]: pgmap v12706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:52:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22657.330769 secs 2024-04-27T00:52:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22657.330769 secs 2024-04-27T00:52:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22657.330769 secs 2024-04-27T00:52:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:56 smithi090 ceph-mon[29180]: pgmap v12707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:56 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22658.130530 secs 2024-04-27T00:52:57.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:56 smithi167 ceph-mon[32751]: pgmap v12707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:57.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:56 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22658.130530 secs 2024-04-27T00:52:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:56 smithi119 ceph-mon[32752]: pgmap v12707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:56 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22658.130530 secs 2024-04-27T00:52:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:52:58 smithi090 ceph-mon[29180]: pgmap v12708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:52:58 smithi119 ceph-mon[32752]: pgmap v12708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:52:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:52:58 smithi167 ceph-mon[32751]: pgmap v12708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:00 smithi090 ceph-mon[29180]: pgmap v12709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22662.330859 secs 2024-04-27T00:53:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:00 smithi119 ceph-mon[32752]: pgmap v12709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22662.330859 secs 2024-04-27T00:53:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:00 smithi167 ceph-mon[32751]: pgmap v12709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22662.330859 secs 2024-04-27T00:53:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:01 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22663.130640 secs 2024-04-27T00:53:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:01 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22663.130640 secs 2024-04-27T00:53:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:01 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22663.130640 secs 2024-04-27T00:53:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:02 smithi090 ceph-mon[29180]: pgmap v12710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:53:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:53:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:53:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:53:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:02 smithi119 ceph-mon[32752]: pgmap v12710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:53:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:53:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:53:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:53:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:02 smithi167 ceph-mon[32751]: pgmap v12710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:53:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:53:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:53:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:53:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:03 smithi090 ceph-mon[29180]: pgmap v12711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:53:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:03 smithi090 ceph-mon[29180]: pgmap v12712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:53:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:03 smithi119 ceph-mon[32752]: pgmap v12711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:53:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:03 smithi119 ceph-mon[32752]: pgmap v12712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:53:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:03 smithi167 ceph-mon[32751]: pgmap v12711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:53:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:03 smithi167 ceph-mon[32751]: pgmap v12712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:53:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:05 smithi090 ceph-mon[29180]: pgmap v12713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:53:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22667.330920 secs 2024-04-27T00:53:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:05 smithi119 ceph-mon[32752]: pgmap v12713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:53:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22667.330920 secs 2024-04-27T00:53:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:05 smithi167 ceph-mon[32751]: pgmap v12713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:53:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22667.330920 secs 2024-04-27T00:53:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:06 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22668.130722 secs 2024-04-27T00:53:07.204 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:06 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22668.130722 secs 2024-04-27T00:53:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:06 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22668.130722 secs 2024-04-27T00:53:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:07 smithi090 ceph-mon[29180]: pgmap v12714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:53:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:07 smithi119 ceph-mon[32752]: pgmap v12714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:53:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:07 smithi167 ceph-mon[32751]: pgmap v12714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:53:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:09 smithi090 ceph-mon[29180]: pgmap v12715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T00:53:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:09 smithi119 ceph-mon[32752]: pgmap v12715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T00:53:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:09 smithi167 ceph-mon[32751]: pgmap v12715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T00:53:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22672.330982 secs 2024-04-27T00:53:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22672.330982 secs 2024-04-27T00:53:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22672.330982 secs 2024-04-27T00:53:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:11 smithi119 ceph-mon[32752]: pgmap v12716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T00:53:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:11 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22673.130785 secs 2024-04-27T00:53:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:11 smithi167 ceph-mon[32751]: pgmap v12716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T00:53:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:11 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22673.130785 secs 2024-04-27T00:53:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:11 smithi090 ceph-mon[29180]: pgmap v12716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T00:53:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:11 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22673.130785 secs 2024-04-27T00:53:14.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:13 smithi119 ceph-mon[32752]: pgmap v12717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T00:53:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:13 smithi167 ceph-mon[32751]: pgmap v12717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T00:53:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:13 smithi090 ceph-mon[29180]: pgmap v12717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T00:53:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:15 smithi119 ceph-mon[32752]: pgmap v12718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22677.331044 secs 2024-04-27T00:53:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:15 smithi167 ceph-mon[32751]: pgmap v12718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22677.331044 secs 2024-04-27T00:53:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:15 smithi090 ceph-mon[29180]: pgmap v12718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22677.331044 secs 2024-04-27T00:53:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:16 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22678.130862 secs 2024-04-27T00:53:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:16 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22678.130862 secs 2024-04-27T00:53:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:16 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22678.130862 secs 2024-04-27T00:53:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:17 smithi119 ceph-mon[32752]: pgmap v12719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:17 smithi167 ceph-mon[32751]: pgmap v12719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:17 smithi090 ceph-mon[29180]: pgmap v12719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:19 smithi119 ceph-mon[32752]: pgmap v12720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:19 smithi167 ceph-mon[32751]: pgmap v12720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:19 smithi090 ceph-mon[29180]: pgmap v12720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T00:53:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22682.331107 secs 2024-04-27T00:53:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22682.331107 secs 2024-04-27T00:53:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22682.331107 secs 2024-04-27T00:53:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:21 smithi119 ceph-mon[32752]: pgmap v12721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:21 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22683.130969 secs 2024-04-27T00:53:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:21 smithi167 ceph-mon[32751]: pgmap v12721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:21 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22683.130969 secs 2024-04-27T00:53:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:21 smithi090 ceph-mon[29180]: pgmap v12721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:21 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22683.130969 secs 2024-04-27T00:53:24.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:23 smithi119 ceph-mon[32752]: pgmap v12722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:23 smithi167 ceph-mon[32751]: pgmap v12722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:23 smithi090 ceph-mon[29180]: pgmap v12722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:25 smithi119 ceph-mon[32752]: pgmap v12723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22687.331168 secs 2024-04-27T00:53:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:25 smithi167 ceph-mon[32751]: pgmap v12723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22687.331168 secs 2024-04-27T00:53:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:25 smithi090 ceph-mon[29180]: pgmap v12723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22687.331168 secs 2024-04-27T00:53:27.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:26 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22688.131079 secs 2024-04-27T00:53:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:26 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22688.131079 secs 2024-04-27T00:53:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:26 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22688.131079 secs 2024-04-27T00:53:28.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:27 smithi119 ceph-mon[32752]: pgmap v12724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:27 smithi167 ceph-mon[32751]: pgmap v12724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:27 smithi090 ceph-mon[29180]: pgmap v12724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:30.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:29 smithi119 ceph-mon[32752]: pgmap v12725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:29 smithi167 ceph-mon[32751]: pgmap v12725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:29 smithi090 ceph-mon[29180]: pgmap v12725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22692.331253 secs 2024-04-27T00:53:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22692.331253 secs 2024-04-27T00:53:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22692.331253 secs 2024-04-27T00:53:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:31 smithi090 ceph-mon[29180]: pgmap v12726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:32.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:31 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22693.131186 secs 2024-04-27T00:53:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:31 smithi119 ceph-mon[32752]: pgmap v12726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:31 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22693.131186 secs 2024-04-27T00:53:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:31 smithi167 ceph-mon[32751]: pgmap v12726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:31 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22693.131186 secs 2024-04-27T00:53:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:33 smithi119 ceph-mon[32752]: pgmap v12727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:33 smithi167 ceph-mon[32751]: pgmap v12727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:33 smithi090 ceph-mon[29180]: pgmap v12727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:35 smithi119 ceph-mon[32752]: pgmap v12728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22697.331339 secs 2024-04-27T00:53:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:35 smithi167 ceph-mon[32751]: pgmap v12728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22697.331339 secs 2024-04-27T00:53:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:35 smithi090 ceph-mon[29180]: pgmap v12728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22697.331339 secs 2024-04-27T00:53:37.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:36 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 22698.131300 secs 2024-04-27T00:53:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:36 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 22698.131300 secs 2024-04-27T00:53:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:36 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 22698.131300 secs 2024-04-27T00:53:37.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:53:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:53:37.046+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:53:37.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:53:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:53:37.046+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:53:37.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:53:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:53:37.046+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:53:37.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:53:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:53:37.046+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:53:37.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:53:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:53:37.046+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:53:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:37 smithi119 ceph-mon[32752]: pgmap v12729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:37 smithi167 ceph-mon[32751]: pgmap v12729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:37 smithi090 ceph-mon[29180]: pgmap v12729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:39 smithi119 ceph-mon[32752]: pgmap v12730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:39 smithi167 ceph-mon[32751]: pgmap v12730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:39 smithi090 ceph-mon[29180]: pgmap v12730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22702.331400 secs 2024-04-27T00:53:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22702.331400 secs 2024-04-27T00:53:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22702.331400 secs 2024-04-27T00:53:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:41 smithi119 ceph-mon[32752]: pgmap v12731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:41 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22703.131451 secs 2024-04-27T00:53:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:41 smithi167 ceph-mon[32751]: pgmap v12731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:41 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22703.131451 secs 2024-04-27T00:53:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:41 smithi090 ceph-mon[29180]: pgmap v12731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:41 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22703.131451 secs 2024-04-27T00:53:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:43 smithi119 ceph-mon[32752]: pgmap v12732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:43 smithi167 ceph-mon[32751]: pgmap v12732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:43 smithi090 ceph-mon[29180]: pgmap v12732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:53:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:44 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:53:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:53:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:44 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:53:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:53:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:44 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:53:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:45 smithi119 ceph-mon[32752]: pgmap v12733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:45 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 22707.331493 secs 2024-04-27T00:53:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:45 smithi119 ceph-mon[32752]: slow request 11523.630210 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:53:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:45 smithi167 ceph-mon[32751]: pgmap v12733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:45 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 22707.331493 secs 2024-04-27T00:53:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:45 smithi167 ceph-mon[32751]: slow request 11523.630210 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:53:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:45 smithi090 ceph-mon[29180]: pgmap v12733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:45 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 22707.331493 secs 2024-04-27T00:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:45 smithi090 ceph-mon[29180]: slow request 11523.630210 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:53:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:46 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22708.131563 secs 2024-04-27T00:53:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:46 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22708.131563 secs 2024-04-27T00:53:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:46 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22708.131563 secs 2024-04-27T00:53:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:47 smithi119 ceph-mon[32752]: pgmap v12734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:47 smithi167 ceph-mon[32751]: pgmap v12734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:47 smithi090 ceph-mon[29180]: pgmap v12734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T00:53:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:49 smithi119 ceph-mon[32752]: pgmap v12735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:49 smithi167 ceph-mon[32751]: pgmap v12735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:49 smithi090 ceph-mon[29180]: pgmap v12735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T00:53:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22712.331583 secs 2024-04-27T00:53:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22712.331583 secs 2024-04-27T00:53:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22712.331583 secs 2024-04-27T00:53:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:51 smithi119 ceph-mon[32752]: pgmap v12736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:51 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22713.131659 secs 2024-04-27T00:53:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:51 smithi167 ceph-mon[32751]: pgmap v12736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:51 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22713.131659 secs 2024-04-27T00:53:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:51 smithi090 ceph-mon[29180]: pgmap v12736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:51 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22713.131659 secs 2024-04-27T00:53:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:53 smithi119 ceph-mon[32752]: pgmap v12737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:53 smithi167 ceph-mon[32751]: pgmap v12737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:53 smithi090 ceph-mon[29180]: pgmap v12737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:55 smithi090 ceph-mon[29180]: pgmap v12738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22717.331673 secs 2024-04-27T00:53:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:55 smithi119 ceph-mon[32752]: pgmap v12738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22717.331673 secs 2024-04-27T00:53:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:55 smithi167 ceph-mon[32751]: pgmap v12738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:53:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22717.331673 secs 2024-04-27T00:53:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:57 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22718.131740 secs 2024-04-27T00:53:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:56 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22718.131740 secs 2024-04-27T00:53:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:56 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22718.131740 secs 2024-04-27T00:53:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:53:58 smithi119 ceph-mon[32752]: pgmap v12739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:53:58 smithi167 ceph-mon[32751]: pgmap v12739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:53:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:53:58 smithi090 ceph-mon[29180]: pgmap v12739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:00 smithi119 ceph-mon[32752]: pgmap v12740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:00 smithi167 ceph-mon[32751]: pgmap v12740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:00 smithi090 ceph-mon[29180]: pgmap v12740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22722.331762 secs 2024-04-27T00:54:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22722.331762 secs 2024-04-27T00:54:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22722.331762 secs 2024-04-27T00:54:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:02 smithi090 ceph-mon[29180]: pgmap v12741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:02 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22723.131855 secs 2024-04-27T00:54:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:02 smithi119 ceph-mon[32752]: pgmap v12741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:02 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22723.131855 secs 2024-04-27T00:54:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:02 smithi167 ceph-mon[32751]: pgmap v12741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:02 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22723.131855 secs 2024-04-27T00:54:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:54:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:54:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:54:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:04 smithi119 ceph-mon[32752]: pgmap v12742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:54:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:54:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:04 smithi119 ceph-mon[32752]: pgmap v12743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-27T00:54:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:04 smithi119 ceph-mon[32752]: pgmap v12744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:54:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:04 smithi167 ceph-mon[32751]: pgmap v12742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:54:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:54:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:04 smithi167 ceph-mon[32751]: pgmap v12743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-27T00:54:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:04 smithi167 ceph-mon[32751]: pgmap v12744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:54:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:04 smithi090 ceph-mon[29180]: pgmap v12742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:54:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:54:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:04 smithi090 ceph-mon[29180]: pgmap v12743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-27T00:54:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:04 smithi090 ceph-mon[29180]: pgmap v12744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:54:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22727.331846 secs 2024-04-27T00:54:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:06 smithi119 ceph-mon[32752]: pgmap v12745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22727.331846 secs 2024-04-27T00:54:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:06 smithi167 ceph-mon[32751]: pgmap v12745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22727.331846 secs 2024-04-27T00:54:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:06 smithi090 ceph-mon[29180]: pgmap v12745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:07.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:07 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22728.131978 secs 2024-04-27T00:54:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:07 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22728.131978 secs 2024-04-27T00:54:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:07 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22728.131978 secs 2024-04-27T00:54:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:08 smithi119 ceph-mon[32752]: pgmap v12746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:54:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:08 smithi167 ceph-mon[32751]: pgmap v12746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:54:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:08 smithi090 ceph-mon[29180]: pgmap v12746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T00:54:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:10 smithi119 ceph-mon[32752]: pgmap v12747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22732.331929 secs 2024-04-27T00:54:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:10 smithi167 ceph-mon[32751]: pgmap v12747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22732.331929 secs 2024-04-27T00:54:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:10 smithi090 ceph-mon[29180]: pgmap v12747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22732.331929 secs 2024-04-27T00:54:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:12 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22733.132097 secs 2024-04-27T00:54:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:12 smithi119 ceph-mon[32752]: pgmap v12748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:12 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22733.132097 secs 2024-04-27T00:54:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:12 smithi167 ceph-mon[32751]: pgmap v12748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:12 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22733.132097 secs 2024-04-27T00:54:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:12 smithi090 ceph-mon[29180]: pgmap v12748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T00:54:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:14 smithi119 ceph-mon[32752]: pgmap v12749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:54:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:14 smithi167 ceph-mon[32751]: pgmap v12749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:54:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:14 smithi090 ceph-mon[29180]: pgmap v12749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:54:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22737.332003 secs 2024-04-27T00:54:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22737.332003 secs 2024-04-27T00:54:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22737.332003 secs 2024-04-27T00:54:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:16 smithi119 ceph-mon[32752]: pgmap v12750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:16 smithi167 ceph-mon[32751]: pgmap v12750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:16 smithi090 ceph-mon[29180]: pgmap v12750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:17 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22738.132231 secs 2024-04-27T00:54:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:17 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22738.132231 secs 2024-04-27T00:54:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:17 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22738.132231 secs 2024-04-27T00:54:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:18 smithi119 ceph-mon[32752]: pgmap v12751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:18 smithi167 ceph-mon[32751]: pgmap v12751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:18 smithi090 ceph-mon[29180]: pgmap v12751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:20 smithi119 ceph-mon[32752]: pgmap v12752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22742.332062 secs 2024-04-27T00:54:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:20 smithi167 ceph-mon[32751]: pgmap v12752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22742.332062 secs 2024-04-27T00:54:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:20 smithi090 ceph-mon[29180]: pgmap v12752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22742.332062 secs 2024-04-27T00:54:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:22 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22743.132349 secs 2024-04-27T00:54:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:22 smithi119 ceph-mon[32752]: pgmap v12753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:22 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22743.132349 secs 2024-04-27T00:54:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:22 smithi167 ceph-mon[32751]: pgmap v12753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:22 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22743.132349 secs 2024-04-27T00:54:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:22 smithi090 ceph-mon[29180]: pgmap v12753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:24.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:24 smithi119 ceph-mon[32752]: pgmap v12754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:24 smithi167 ceph-mon[32751]: pgmap v12754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:24 smithi090 ceph-mon[29180]: pgmap v12754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22747.332116 secs 2024-04-27T00:54:25.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22747.332116 secs 2024-04-27T00:54:25.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22747.332116 secs 2024-04-27T00:54:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:26 smithi119 ceph-mon[32752]: pgmap v12755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:26 smithi167 ceph-mon[32751]: pgmap v12755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:26 smithi090 ceph-mon[29180]: pgmap v12755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:27.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:27 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22748.132436 secs 2024-04-27T00:54:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:27 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22748.132436 secs 2024-04-27T00:54:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:27 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22748.132436 secs 2024-04-27T00:54:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:28 smithi119 ceph-mon[32752]: pgmap v12756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:28 smithi167 ceph-mon[32751]: pgmap v12756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:28 smithi090 ceph-mon[29180]: pgmap v12756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:30 smithi119 ceph-mon[32752]: pgmap v12757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22752.332174 secs 2024-04-27T00:54:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:30 smithi167 ceph-mon[32751]: pgmap v12757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22752.332174 secs 2024-04-27T00:54:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:30 smithi090 ceph-mon[29180]: pgmap v12757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22752.332174 secs 2024-04-27T00:54:32.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:32 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22753.132517 secs 2024-04-27T00:54:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:32 smithi119 ceph-mon[32752]: pgmap v12758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:32 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22753.132517 secs 2024-04-27T00:54:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:32 smithi167 ceph-mon[32751]: pgmap v12758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:32 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22753.132517 secs 2024-04-27T00:54:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:32 smithi090 ceph-mon[29180]: pgmap v12758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:34 smithi119 ceph-mon[32752]: pgmap v12759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:34.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:34 smithi167 ceph-mon[32751]: pgmap v12759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:34 smithi090 ceph-mon[29180]: pgmap v12759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:35.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22757.332231 secs 2024-04-27T00:54:35.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22757.332231 secs 2024-04-27T00:54:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22757.332231 secs 2024-04-27T00:54:36.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:36 smithi119 ceph-mon[32752]: pgmap v12760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:36 smithi167 ceph-mon[32751]: pgmap v12760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:36 smithi090 ceph-mon[29180]: pgmap v12760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:37.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:37 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22758.132601 secs 2024-04-27T00:54:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:37 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22758.132601 secs 2024-04-27T00:54:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:37 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22758.132601 secs 2024-04-27T00:54:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:38 smithi119 ceph-mon[32752]: pgmap v12761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:38 smithi167 ceph-mon[32751]: pgmap v12761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:38.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:38 smithi090 ceph-mon[29180]: pgmap v12761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:40 smithi119 ceph-mon[32752]: pgmap v12762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22762.332320 secs 2024-04-27T00:54:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:40 smithi167 ceph-mon[32751]: pgmap v12762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22762.332320 secs 2024-04-27T00:54:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:40 smithi090 ceph-mon[29180]: pgmap v12762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22762.332320 secs 2024-04-27T00:54:42.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:42 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22763.132678 secs 2024-04-27T00:54:42.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:42 smithi090 ceph-mon[29180]: pgmap v12763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:42 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22763.132678 secs 2024-04-27T00:54:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:42 smithi119 ceph-mon[32752]: pgmap v12763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:42 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22763.132678 secs 2024-04-27T00:54:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:42 smithi167 ceph-mon[32751]: pgmap v12763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:44 smithi119 ceph-mon[32752]: pgmap v12764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:44 smithi167 ceph-mon[32751]: pgmap v12764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:44.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:44 smithi090 ceph-mon[29180]: pgmap v12764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22767.332400 secs 2024-04-27T00:54:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22767.332400 secs 2024-04-27T00:54:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22767.332400 secs 2024-04-27T00:54:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:46 smithi119 ceph-mon[32752]: pgmap v12765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:46 smithi167 ceph-mon[32751]: pgmap v12765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:46 smithi090 ceph-mon[29180]: pgmap v12765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:47 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22768.132745 secs 2024-04-27T00:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:47 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22768.132745 secs 2024-04-27T00:54:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:47 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22768.132745 secs 2024-04-27T00:54:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:48 smithi119 ceph-mon[32752]: pgmap v12766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:48 smithi167 ceph-mon[32751]: pgmap v12766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:48 smithi090 ceph-mon[29180]: pgmap v12766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:50 smithi119 ceph-mon[32752]: pgmap v12767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:50.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22772.332490 secs 2024-04-27T00:54:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:50 smithi167 ceph-mon[32751]: pgmap v12767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:50.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22772.332490 secs 2024-04-27T00:54:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:50 smithi090 ceph-mon[29180]: pgmap v12767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22772.332490 secs 2024-04-27T00:54:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:52 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22773.132806 secs 2024-04-27T00:54:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:52 smithi119 ceph-mon[32752]: pgmap v12768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:52 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22773.132806 secs 2024-04-27T00:54:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:52 smithi167 ceph-mon[32751]: pgmap v12768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:52 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22773.132806 secs 2024-04-27T00:54:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:52 smithi090 ceph-mon[29180]: pgmap v12768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:54 smithi119 ceph-mon[32752]: pgmap v12769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:54 smithi167 ceph-mon[32751]: pgmap v12769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:54 smithi090 ceph-mon[29180]: pgmap v12769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:54:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22777.332579 secs 2024-04-27T00:54:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22777.332579 secs 2024-04-27T00:54:55.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22777.332579 secs 2024-04-27T00:54:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:56 smithi119 ceph-mon[32752]: pgmap v12770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:56 smithi167 ceph-mon[32751]: pgmap v12770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:56 smithi090 ceph-mon[29180]: pgmap v12770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:57 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22778.132896 secs 2024-04-27T00:54:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:57 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22778.132896 secs 2024-04-27T00:54:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:57 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22778.132896 secs 2024-04-27T00:54:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:54:58 smithi119 ceph-mon[32752]: pgmap v12771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:54:58 smithi167 ceph-mon[32751]: pgmap v12771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:54:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:54:58 smithi090 ceph-mon[29180]: pgmap v12771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:00 smithi119 ceph-mon[32752]: pgmap v12772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22782.332665 secs 2024-04-27T00:55:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:00 smithi167 ceph-mon[32751]: pgmap v12772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22782.332665 secs 2024-04-27T00:55:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:00 smithi090 ceph-mon[29180]: pgmap v12772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22782.332665 secs 2024-04-27T00:55:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:02 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22783.133041 secs 2024-04-27T00:55:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:02 smithi119 ceph-mon[32752]: pgmap v12773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:02 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22783.133041 secs 2024-04-27T00:55:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:02 smithi167 ceph-mon[32751]: pgmap v12773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:02 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22783.133041 secs 2024-04-27T00:55:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:02 smithi090 ceph-mon[29180]: pgmap v12773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:55:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:55:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:55:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:04 smithi119 ceph-mon[32752]: pgmap v12774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:55:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:55:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:55:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:04 smithi167 ceph-mon[32751]: pgmap v12774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:55:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:55:04.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:55:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:04 smithi090 ceph-mon[29180]: pgmap v12774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:55:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:55:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:55:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:05 smithi119 ceph-mon[32752]: pgmap v12775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:55:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:05 smithi119 ceph-mon[32752]: pgmap v12776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22787.332721 secs 2024-04-27T00:55:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:05 smithi167 ceph-mon[32751]: pgmap v12775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:55:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:05 smithi167 ceph-mon[32751]: pgmap v12776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22787.332721 secs 2024-04-27T00:55:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:05 smithi090 ceph-mon[29180]: pgmap v12775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:55:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:05 smithi090 ceph-mon[29180]: pgmap v12776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22787.332721 secs 2024-04-27T00:55:07.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:07 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22788.133175 secs 2024-04-27T00:55:07.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:07 smithi119 ceph-mon[32752]: pgmap v12777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:07 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22788.133175 secs 2024-04-27T00:55:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:07 smithi167 ceph-mon[32751]: pgmap v12777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:07 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22788.133175 secs 2024-04-27T00:55:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:07 smithi090 ceph-mon[29180]: pgmap v12777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:09 smithi119 ceph-mon[32752]: pgmap v12778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:55:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:09 smithi167 ceph-mon[32751]: pgmap v12778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:55:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:09 smithi090 ceph-mon[29180]: pgmap v12778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:55:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22792.332781 secs 2024-04-27T00:55:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22792.332781 secs 2024-04-27T00:55:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22792.332781 secs 2024-04-27T00:55:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:11 smithi167 ceph-mon[32751]: pgmap v12779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:11 smithi090 ceph-mon[29180]: pgmap v12779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:11 smithi119 ceph-mon[32752]: pgmap v12779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:55:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:12 smithi167 ceph-mon[32751]: 3 slow requests, 1 included below; oldest blocked for > 22793.133307 secs 2024-04-27T00:55:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:12 smithi167 ceph-mon[32751]: slow request 724.014440 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:55:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:12 smithi167 ceph-mon[32751]: pgmap v12780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:55:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:12 smithi090 ceph-mon[29180]: 3 slow requests, 1 included below; oldest blocked for > 22793.133307 secs 2024-04-27T00:55:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:12 smithi090 ceph-mon[29180]: slow request 724.014440 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:55:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:12 smithi090 ceph-mon[29180]: pgmap v12780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:55:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:12 smithi119 ceph-mon[32752]: 3 slow requests, 1 included below; oldest blocked for > 22793.133307 secs 2024-04-27T00:55:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:12 smithi119 ceph-mon[32752]: slow request 724.014440 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:55:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:12 smithi119 ceph-mon[32752]: pgmap v12780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:55:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:15 smithi119 ceph-mon[32752]: pgmap v12781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:55:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:15 smithi167 ceph-mon[32751]: pgmap v12781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:55:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:15 smithi090 ceph-mon[29180]: pgmap v12781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:55:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22797.332838 secs 2024-04-27T00:55:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22797.332838 secs 2024-04-27T00:55:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22797.332838 secs 2024-04-27T00:55:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:17 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22798.133424 secs 2024-04-27T00:55:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:17 smithi119 ceph-mon[32752]: pgmap v12782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:17 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22798.133424 secs 2024-04-27T00:55:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:17 smithi167 ceph-mon[32751]: pgmap v12782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:17 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22798.133424 secs 2024-04-27T00:55:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:17 smithi090 ceph-mon[29180]: pgmap v12782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:19 smithi119 ceph-mon[32752]: pgmap v12783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:19 smithi167 ceph-mon[32751]: pgmap v12783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:19.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:19 smithi090 ceph-mon[29180]: pgmap v12783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22802.332914 secs 2024-04-27T00:55:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:21 smithi119 ceph-mon[32752]: pgmap v12784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22802.332914 secs 2024-04-27T00:55:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:21 smithi167 ceph-mon[32751]: pgmap v12784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22802.332914 secs 2024-04-27T00:55:21.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:21 smithi090 ceph-mon[29180]: pgmap v12784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:22 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22803.133520 secs 2024-04-27T00:55:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:22 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22803.133520 secs 2024-04-27T00:55:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:22 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22803.133520 secs 2024-04-27T00:55:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:23 smithi119 ceph-mon[32752]: pgmap v12785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:23 smithi167 ceph-mon[32751]: pgmap v12785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:23 smithi090 ceph-mon[29180]: pgmap v12785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:25 smithi119 ceph-mon[32752]: pgmap v12786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:25 smithi167 ceph-mon[32751]: pgmap v12786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:25 smithi090 ceph-mon[29180]: pgmap v12786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22807.332974 secs 2024-04-27T00:55:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22807.332974 secs 2024-04-27T00:55:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22807.332974 secs 2024-04-27T00:55:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:27 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22808.133612 secs 2024-04-27T00:55:27.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:27 smithi167 ceph-mon[32751]: pgmap v12787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:27 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22808.133612 secs 2024-04-27T00:55:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:27 smithi119 ceph-mon[32752]: pgmap v12787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:27 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22808.133612 secs 2024-04-27T00:55:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:27 smithi090 ceph-mon[29180]: pgmap v12787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:29.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:29 smithi090 ceph-mon[29180]: pgmap v12788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:29 smithi119 ceph-mon[32752]: pgmap v12788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:29 smithi167 ceph-mon[32751]: pgmap v12788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22812.333063 secs 2024-04-27T00:55:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:31 smithi119 ceph-mon[32752]: pgmap v12789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22812.333063 secs 2024-04-27T00:55:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:31 smithi167 ceph-mon[32751]: pgmap v12789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22812.333063 secs 2024-04-27T00:55:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:31 smithi090 ceph-mon[29180]: pgmap v12789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:32.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:32 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22813.133726 secs 2024-04-27T00:55:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:32 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22813.133726 secs 2024-04-27T00:55:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:32 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22813.133726 secs 2024-04-27T00:55:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:33 smithi119 ceph-mon[32752]: pgmap v12790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:33 smithi167 ceph-mon[32751]: pgmap v12790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:33 smithi090 ceph-mon[29180]: pgmap v12790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:35 smithi119 ceph-mon[32752]: pgmap v12791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:35 smithi167 ceph-mon[32751]: pgmap v12791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:35 smithi090 ceph-mon[29180]: pgmap v12791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22817.333123 secs 2024-04-27T00:55:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22817.333123 secs 2024-04-27T00:55:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22817.333123 secs 2024-04-27T00:55:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:37 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22818.133865 secs 2024-04-27T00:55:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:37 smithi119 ceph-mon[32752]: pgmap v12792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:37 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22818.133865 secs 2024-04-27T00:55:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:37 smithi167 ceph-mon[32751]: pgmap v12792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:37 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22818.133865 secs 2024-04-27T00:55:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:37 smithi090 ceph-mon[29180]: pgmap v12792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:39 smithi119 ceph-mon[32752]: pgmap v12793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:39 smithi167 ceph-mon[32751]: pgmap v12793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:39 smithi090 ceph-mon[29180]: pgmap v12793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22822.333185 secs 2024-04-27T00:55:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:41 smithi119 ceph-mon[32752]: pgmap v12794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22822.333185 secs 2024-04-27T00:55:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:41 smithi167 ceph-mon[32751]: pgmap v12794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22822.333185 secs 2024-04-27T00:55:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:41 smithi090 ceph-mon[29180]: pgmap v12794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:42 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22823.133982 secs 2024-04-27T00:55:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:42 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22823.133982 secs 2024-04-27T00:55:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:42 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22823.133982 secs 2024-04-27T00:55:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:43 smithi119 ceph-mon[32752]: pgmap v12795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:43 smithi167 ceph-mon[32751]: pgmap v12795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:43 smithi090 ceph-mon[29180]: pgmap v12795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:45 smithi119 ceph-mon[32752]: pgmap v12796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:45 smithi167 ceph-mon[32751]: pgmap v12796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:45 smithi090 ceph-mon[29180]: pgmap v12796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22827.333273 secs 2024-04-27T00:55:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22827.333273 secs 2024-04-27T00:55:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22827.333273 secs 2024-04-27T00:55:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:47 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22828.134099 secs 2024-04-27T00:55:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:47 smithi119 ceph-mon[32752]: pgmap v12797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:47 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22828.134099 secs 2024-04-27T00:55:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:47 smithi167 ceph-mon[32751]: pgmap v12797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:47 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22828.134099 secs 2024-04-27T00:55:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:47 smithi090 ceph-mon[29180]: pgmap v12797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:49 smithi119 ceph-mon[32752]: pgmap v12798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:49 smithi167 ceph-mon[32751]: pgmap v12798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:49 smithi090 ceph-mon[29180]: pgmap v12798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22832.333359 secs 2024-04-27T00:55:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:51 smithi119 ceph-mon[32752]: pgmap v12799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22832.333359 secs 2024-04-27T00:55:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:51 smithi167 ceph-mon[32751]: pgmap v12799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22832.333359 secs 2024-04-27T00:55:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:51 smithi090 ceph-mon[29180]: pgmap v12799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:52 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22833.134198 secs 2024-04-27T00:55:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:52 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22833.134198 secs 2024-04-27T00:55:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:52 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22833.134198 secs 2024-04-27T00:55:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:53 smithi119 ceph-mon[32752]: pgmap v12800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:53.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:53 smithi167 ceph-mon[32751]: pgmap v12800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:53.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:53 smithi090 ceph-mon[29180]: pgmap v12800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:55 smithi119 ceph-mon[32752]: pgmap v12801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:55.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:55 smithi167 ceph-mon[32751]: pgmap v12801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:55.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:55 smithi090 ceph-mon[29180]: pgmap v12801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:55:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22837.333445 secs 2024-04-27T00:55:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22837.333445 secs 2024-04-27T00:55:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22837.333445 secs 2024-04-27T00:55:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:57 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22838.134320 secs 2024-04-27T00:55:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:57 smithi119 ceph-mon[32752]: pgmap v12802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:57 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22838.134320 secs 2024-04-27T00:55:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:57 smithi167 ceph-mon[32751]: pgmap v12802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:57 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22838.134320 secs 2024-04-27T00:55:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:57 smithi090 ceph-mon[29180]: pgmap v12802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:55:59 smithi119 ceph-mon[32752]: pgmap v12803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:55:59 smithi167 ceph-mon[32751]: pgmap v12803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:55:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:55:59 smithi090 ceph-mon[29180]: pgmap v12803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22842.333531 secs 2024-04-27T00:56:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:01 smithi119 ceph-mon[32752]: pgmap v12804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22842.333531 secs 2024-04-27T00:56:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:01 smithi167 ceph-mon[32751]: pgmap v12804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22842.333531 secs 2024-04-27T00:56:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:01 smithi090 ceph-mon[29180]: pgmap v12804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:02 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22843.134440 secs 2024-04-27T00:56:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:02 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22843.134440 secs 2024-04-27T00:56:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:02 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22843.134440 secs 2024-04-27T00:56:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:03 smithi119 ceph-mon[32752]: pgmap v12805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:03.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:03 smithi167 ceph-mon[32751]: pgmap v12805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:03 smithi090 ceph-mon[29180]: pgmap v12805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:56:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:56:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:56:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:05 smithi119 ceph-mon[32752]: pgmap v12806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:56:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:56:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:56:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:05 smithi167 ceph-mon[32751]: pgmap v12806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:56:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:56:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:56:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:05 smithi090 ceph-mon[29180]: pgmap v12806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:56:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:56:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:56:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:06 smithi119 ceph-mon[32752]: pgmap v12807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:56:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:06 smithi119 ceph-mon[32752]: pgmap v12808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22847.333634 secs 2024-04-27T00:56:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:06 smithi167 ceph-mon[32751]: pgmap v12807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:56:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:06 smithi167 ceph-mon[32751]: pgmap v12808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22847.333634 secs 2024-04-27T00:56:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:06 smithi090 ceph-mon[29180]: pgmap v12807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:56:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:06 smithi090 ceph-mon[29180]: pgmap v12808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22847.333634 secs 2024-04-27T00:56:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:07 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22848.134505 secs 2024-04-27T00:56:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:07 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22848.134505 secs 2024-04-27T00:56:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:07 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22848.134505 secs 2024-04-27T00:56:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:08 smithi119 ceph-mon[32752]: pgmap v12809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:08 smithi167 ceph-mon[32751]: pgmap v12809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:08 smithi090 ceph-mon[29180]: pgmap v12809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:10 smithi167 ceph-mon[32751]: pgmap v12810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:10 smithi090 ceph-mon[29180]: pgmap v12810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:10 smithi119 ceph-mon[32752]: pgmap v12810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22852.333734 secs 2024-04-27T00:56:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22852.333734 secs 2024-04-27T00:56:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22852.333734 secs 2024-04-27T00:56:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:12 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22853.134644 secs 2024-04-27T00:56:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:12 smithi167 ceph-mon[32751]: pgmap v12811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:12 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22853.134644 secs 2024-04-27T00:56:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:12 smithi090 ceph-mon[29180]: pgmap v12811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:12 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22853.134644 secs 2024-04-27T00:56:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:12 smithi119 ceph-mon[32752]: pgmap v12811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:56:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:14 smithi167 ceph-mon[32751]: pgmap v12812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:56:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:14 smithi090 ceph-mon[29180]: pgmap v12812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:56:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:14 smithi119 ceph-mon[32752]: pgmap v12812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:56:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:16 smithi090 ceph-mon[29180]: pgmap v12813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:56:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22857.333813 secs 2024-04-27T00:56:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:16 smithi119 ceph-mon[32752]: pgmap v12813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:56:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22857.333813 secs 2024-04-27T00:56:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:16 smithi167 ceph-mon[32751]: pgmap v12813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:56:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22857.333813 secs 2024-04-27T00:56:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:17 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22858.134753 secs 2024-04-27T00:56:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:17 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22858.134753 secs 2024-04-27T00:56:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:17 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22858.134753 secs 2024-04-27T00:56:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:18 smithi090 ceph-mon[29180]: pgmap v12814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:18 smithi119 ceph-mon[32752]: pgmap v12814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:18 smithi167 ceph-mon[32751]: pgmap v12814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:20 smithi090 ceph-mon[29180]: pgmap v12815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:20 smithi119 ceph-mon[32752]: pgmap v12815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:20 smithi167 ceph-mon[32751]: pgmap v12815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22862.333875 secs 2024-04-27T00:56:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22862.333875 secs 2024-04-27T00:56:21.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22862.333875 secs 2024-04-27T00:56:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:22 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22863.134873 secs 2024-04-27T00:56:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:22 smithi090 ceph-mon[29180]: pgmap v12816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:22 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22863.134873 secs 2024-04-27T00:56:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:22 smithi119 ceph-mon[32752]: pgmap v12816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:22 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22863.134873 secs 2024-04-27T00:56:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:22 smithi167 ceph-mon[32751]: pgmap v12816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:24 smithi090 ceph-mon[29180]: pgmap v12817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:24 smithi119 ceph-mon[32752]: pgmap v12817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:24.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:24 smithi167 ceph-mon[32751]: pgmap v12817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:26 smithi090 ceph-mon[29180]: pgmap v12818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22867.333965 secs 2024-04-27T00:56:26.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:26 smithi119 ceph-mon[32752]: pgmap v12818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22867.333965 secs 2024-04-27T00:56:26.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:26 smithi167 ceph-mon[32751]: pgmap v12818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22867.333965 secs 2024-04-27T00:56:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:27 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22868.135008 secs 2024-04-27T00:56:27.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:27 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22868.135008 secs 2024-04-27T00:56:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:27 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22868.135008 secs 2024-04-27T00:56:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:28 smithi090 ceph-mon[29180]: pgmap v12819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:28.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:28 smithi119 ceph-mon[32752]: pgmap v12819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:28.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:28 smithi167 ceph-mon[32751]: pgmap v12819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:30 smithi090 ceph-mon[29180]: pgmap v12820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:30.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:30 smithi119 ceph-mon[32752]: pgmap v12820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:30 smithi167 ceph-mon[32751]: pgmap v12820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22872.334039 secs 2024-04-27T00:56:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22872.334039 secs 2024-04-27T00:56:31.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22872.334039 secs 2024-04-27T00:56:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:32 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22873.135117 secs 2024-04-27T00:56:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:32 smithi090 ceph-mon[29180]: pgmap v12821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:32 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22873.135117 secs 2024-04-27T00:56:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:32 smithi119 ceph-mon[32752]: pgmap v12821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:32.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:32 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22873.135117 secs 2024-04-27T00:56:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:32 smithi167 ceph-mon[32751]: pgmap v12821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:34.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:34 smithi090 ceph-mon[29180]: pgmap v12822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:34 smithi119 ceph-mon[32752]: pgmap v12822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:34 smithi167 ceph-mon[32751]: pgmap v12822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:36 smithi090 ceph-mon[29180]: pgmap v12823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22877.334121 secs 2024-04-27T00:56:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:36 smithi119 ceph-mon[32752]: pgmap v12823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22877.334121 secs 2024-04-27T00:56:36.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:36 smithi167 ceph-mon[32751]: pgmap v12823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22877.334121 secs 2024-04-27T00:56:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:37 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 22878.135267 secs 2024-04-27T00:56:37.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:37 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 22878.135267 secs 2024-04-27T00:56:37.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:37 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 22878.135267 secs 2024-04-27T00:56:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:38 smithi090 ceph-mon[29180]: pgmap v12824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:38.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:38 smithi119 ceph-mon[32752]: pgmap v12824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:38.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:38 smithi167 ceph-mon[32751]: pgmap v12824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:40 smithi090 ceph-mon[29180]: pgmap v12825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:40.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:40 smithi119 ceph-mon[32752]: pgmap v12825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:40.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:40 smithi167 ceph-mon[32751]: pgmap v12825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22882.334206 secs 2024-04-27T00:56:41.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22882.334206 secs 2024-04-27T00:56:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22882.334206 secs 2024-04-27T00:56:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:42 smithi119 ceph-mon[32752]: 24 slow requests, 2 included below; oldest blocked for > 22883.135401 secs 2024-04-27T00:56:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:42 smithi119 ceph-mon[32752]: slow request 183.830688 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:56:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:42 smithi119 ceph-mon[32752]: slow request 1444.178150 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:56:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:42 smithi119 ceph-mon[32752]: pgmap v12826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:42 smithi167 ceph-mon[32751]: 24 slow requests, 2 included below; oldest blocked for > 22883.135401 secs 2024-04-27T00:56:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:42 smithi167 ceph-mon[32751]: slow request 183.830688 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:56:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:42 smithi167 ceph-mon[32751]: slow request 1444.178150 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:56:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:42 smithi167 ceph-mon[32751]: pgmap v12826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:42 smithi090 ceph-mon[29180]: 24 slow requests, 2 included below; oldest blocked for > 22883.135401 secs 2024-04-27T00:56:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:42 smithi090 ceph-mon[29180]: slow request 183.830688 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T00:56:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:42 smithi090 ceph-mon[29180]: slow request 1444.178150 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:56:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:42 smithi090 ceph-mon[29180]: pgmap v12826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:43 smithi090 ceph-mon[29180]: pgmap v12827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:43 smithi119 ceph-mon[32752]: pgmap v12827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:43 smithi167 ceph-mon[32751]: pgmap v12827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:56:45.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:44 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:56:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:56:45.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:44 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:56:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T00:56:45.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:44 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T00:56:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:45 smithi090 ceph-mon[29180]: pgmap v12828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22887.334289 secs 2024-04-27T00:56:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:45 smithi119 ceph-mon[32752]: pgmap v12828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22887.334289 secs 2024-04-27T00:56:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:45 smithi167 ceph-mon[32751]: pgmap v12828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22887.334289 secs 2024-04-27T00:56:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22888.135537 secs 2024-04-27T00:56:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22888.135537 secs 2024-04-27T00:56:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22888.135537 secs 2024-04-27T00:56:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:47 smithi090 ceph-mon[29180]: pgmap v12829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:47 smithi119 ceph-mon[32752]: pgmap v12829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:47 smithi167 ceph-mon[32751]: pgmap v12829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:49 smithi119 ceph-mon[32752]: pgmap v12830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:49 smithi167 ceph-mon[32751]: pgmap v12830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:49 smithi090 ceph-mon[29180]: pgmap v12830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22892.334373 secs 2024-04-27T00:56:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22892.334373 secs 2024-04-27T00:56:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22892.334373 secs 2024-04-27T00:56:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22893.135616 secs 2024-04-27T00:56:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:51 smithi119 ceph-mon[32752]: pgmap v12831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22893.135616 secs 2024-04-27T00:56:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:51 smithi167 ceph-mon[32751]: pgmap v12831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22893.135616 secs 2024-04-27T00:56:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:51 smithi090 ceph-mon[29180]: pgmap v12831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:53 smithi119 ceph-mon[32752]: pgmap v12832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:53 smithi167 ceph-mon[32751]: pgmap v12832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:54.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:53 smithi090 ceph-mon[29180]: pgmap v12832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:55 smithi119 ceph-mon[32752]: pgmap v12833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22897.334459 secs 2024-04-27T00:56:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:55 smithi167 ceph-mon[32751]: pgmap v12833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22897.334459 secs 2024-04-27T00:56:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:55 smithi090 ceph-mon[29180]: pgmap v12833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:56:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22897.334459 secs 2024-04-27T00:56:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22898.135724 secs 2024-04-27T00:56:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22898.135724 secs 2024-04-27T00:56:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22898.135724 secs 2024-04-27T00:56:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:57 smithi119 ceph-mon[32752]: pgmap v12834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:58.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:57 smithi167 ceph-mon[32751]: pgmap v12834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:56:58.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:57 smithi090 ceph-mon[29180]: pgmap v12834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:56:59 smithi119 ceph-mon[32752]: pgmap v12835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:56:59 smithi167 ceph-mon[32751]: pgmap v12835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:00.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:56:59 smithi090 ceph-mon[29180]: pgmap v12835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22902.334543 secs 2024-04-27T00:57:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22902.334543 secs 2024-04-27T00:57:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22902.334543 secs 2024-04-27T00:57:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22903.135830 secs 2024-04-27T00:57:02.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:01 smithi090 ceph-mon[29180]: pgmap v12836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:57:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22903.135830 secs 2024-04-27T00:57:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:01 smithi119 ceph-mon[32752]: pgmap v12836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:57:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22903.135830 secs 2024-04-27T00:57:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:01 smithi167 ceph-mon[32751]: pgmap v12836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:57:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:03 smithi119 ceph-mon[32752]: pgmap v12837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:57:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:03 smithi167 ceph-mon[32751]: pgmap v12837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:57:04.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:03 smithi090 ceph-mon[29180]: pgmap v12837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T00:57:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:05 smithi119 ceph-mon[32752]: pgmap v12838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:57:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22907.334638 secs 2024-04-27T00:57:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:57:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:57:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:57:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:05 smithi167 ceph-mon[32751]: pgmap v12838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:57:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22907.334638 secs 2024-04-27T00:57:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:57:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:57:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:57:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:05 smithi090 ceph-mon[29180]: pgmap v12838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:57:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22907.334638 secs 2024-04-27T00:57:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:57:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:57:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:57:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:06 smithi119 ceph-mon[32752]: pgmap v12839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 B/s rd, 1 op/s 2024-04-27T00:57:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:06 smithi119 ceph-mon[32752]: pgmap v12840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:57:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22908.135945 secs 2024-04-27T00:57:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:06 smithi167 ceph-mon[32751]: pgmap v12839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 B/s rd, 1 op/s 2024-04-27T00:57:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:06 smithi167 ceph-mon[32751]: pgmap v12840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:57:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22908.135945 secs 2024-04-27T00:57:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:06 smithi090 ceph-mon[29180]: pgmap v12839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 B/s rd, 1 op/s 2024-04-27T00:57:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:06 smithi090 ceph-mon[29180]: pgmap v12840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T00:57:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22908.135945 secs 2024-04-27T00:57:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:08 smithi119 ceph-mon[32752]: pgmap v12841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:57:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:08 smithi167 ceph-mon[32751]: pgmap v12841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:57:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:08 smithi090 ceph-mon[29180]: pgmap v12841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:57:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:11 smithi119 ceph-mon[32752]: pgmap v12842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22912.334722 secs 2024-04-27T00:57:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:11 smithi167 ceph-mon[32751]: pgmap v12842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22912.334722 secs 2024-04-27T00:57:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:11 smithi090 ceph-mon[29180]: pgmap v12842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22912.334722 secs 2024-04-27T00:57:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22913.136021 secs 2024-04-27T00:57:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22913.136021 secs 2024-04-27T00:57:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22913.136021 secs 2024-04-27T00:57:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:13 smithi119 ceph-mon[32752]: pgmap v12843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:13 smithi167 ceph-mon[32751]: pgmap v12843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:13 smithi090 ceph-mon[29180]: pgmap v12843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:15 smithi119 ceph-mon[32752]: pgmap v12844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:15 smithi167 ceph-mon[32751]: pgmap v12844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:15 smithi090 ceph-mon[29180]: pgmap v12844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:57:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22917.334809 secs 2024-04-27T00:57:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22917.334809 secs 2024-04-27T00:57:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22917.334809 secs 2024-04-27T00:57:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:17 smithi119 ceph-mon[32752]: pgmap v12845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:57:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22918.136138 secs 2024-04-27T00:57:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:17 smithi167 ceph-mon[32751]: pgmap v12845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:57:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22918.136138 secs 2024-04-27T00:57:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:17 smithi090 ceph-mon[29180]: pgmap v12845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:57:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22918.136138 secs 2024-04-27T00:57:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:19 smithi119 ceph-mon[32752]: pgmap v12846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:19 smithi167 ceph-mon[32751]: pgmap v12846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:19 smithi090 ceph-mon[29180]: pgmap v12846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:21 smithi119 ceph-mon[32752]: pgmap v12847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22922.334871 secs 2024-04-27T00:57:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:21 smithi167 ceph-mon[32751]: pgmap v12847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22922.334871 secs 2024-04-27T00:57:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:21 smithi090 ceph-mon[29180]: pgmap v12847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22922.334871 secs 2024-04-27T00:57:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22923.136253 secs 2024-04-27T00:57:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22923.136253 secs 2024-04-27T00:57:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22923.136253 secs 2024-04-27T00:57:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:23 smithi119 ceph-mon[32752]: pgmap v12848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:23 smithi167 ceph-mon[32751]: pgmap v12848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:23 smithi090 ceph-mon[29180]: pgmap v12848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:25 smithi119 ceph-mon[32752]: pgmap v12849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:25 smithi167 ceph-mon[32751]: pgmap v12849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:25 smithi090 ceph-mon[29180]: pgmap v12849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22927.334980 secs 2024-04-27T00:57:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22927.334980 secs 2024-04-27T00:57:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22927.334980 secs 2024-04-27T00:57:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:27 smithi119 ceph-mon[32752]: pgmap v12850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22928.136429 secs 2024-04-27T00:57:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:27 smithi167 ceph-mon[32751]: pgmap v12850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22928.136429 secs 2024-04-27T00:57:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:27 smithi090 ceph-mon[29180]: pgmap v12850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22928.136429 secs 2024-04-27T00:57:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:29 smithi119 ceph-mon[32752]: pgmap v12851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:29 smithi167 ceph-mon[32751]: pgmap v12851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:29 smithi090 ceph-mon[29180]: pgmap v12851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:31 smithi119 ceph-mon[32752]: pgmap v12852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22932.335070 secs 2024-04-27T00:57:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:31 smithi167 ceph-mon[32751]: pgmap v12852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22932.335070 secs 2024-04-27T00:57:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:31 smithi090 ceph-mon[29180]: pgmap v12852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22932.335070 secs 2024-04-27T00:57:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22933.136550 secs 2024-04-27T00:57:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22933.136550 secs 2024-04-27T00:57:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22933.136550 secs 2024-04-27T00:57:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:33 smithi167 ceph-mon[32751]: pgmap v12853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:33 smithi090 ceph-mon[29180]: pgmap v12853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:33 smithi119 ceph-mon[32752]: pgmap v12853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:35.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:35 smithi167 ceph-mon[32751]: pgmap v12854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:35.408 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:35 smithi090 ceph-mon[29180]: pgmap v12854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:35 smithi119 ceph-mon[32752]: pgmap v12854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22937.335161 secs 2024-04-27T00:57:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22937.335161 secs 2024-04-27T00:57:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22937.335161 secs 2024-04-27T00:57:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:37 smithi167 ceph-mon[32751]: pgmap v12855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22938.136667 secs 2024-04-27T00:57:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:37 smithi090 ceph-mon[29180]: pgmap v12855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22938.136667 secs 2024-04-27T00:57:37.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:37 smithi119 ceph-mon[32752]: pgmap v12855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:37.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22938.136667 secs 2024-04-27T00:57:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:39 smithi090 ceph-mon[29180]: pgmap v12856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:39 smithi119 ceph-mon[32752]: pgmap v12856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:39 smithi167 ceph-mon[32751]: pgmap v12856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:41 smithi090 ceph-mon[29180]: pgmap v12857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22942.335253 secs 2024-04-27T00:57:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:41 smithi119 ceph-mon[32752]: pgmap v12857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22942.335253 secs 2024-04-27T00:57:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:41 smithi167 ceph-mon[32751]: pgmap v12857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22942.335253 secs 2024-04-27T00:57:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22943.136785 secs 2024-04-27T00:57:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22943.136785 secs 2024-04-27T00:57:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22943.136785 secs 2024-04-27T00:57:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:43 smithi090 ceph-mon[29180]: pgmap v12858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:43 smithi119 ceph-mon[32752]: pgmap v12858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:43 smithi167 ceph-mon[32751]: pgmap v12858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:45 smithi090 ceph-mon[29180]: pgmap v12859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:45 smithi119 ceph-mon[32752]: pgmap v12859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:45 smithi167 ceph-mon[32751]: pgmap v12859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22947.335343 secs 2024-04-27T00:57:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22947.335343 secs 2024-04-27T00:57:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22947.335343 secs 2024-04-27T00:57:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:47 smithi090 ceph-mon[29180]: pgmap v12860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22948.136933 secs 2024-04-27T00:57:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:47 smithi119 ceph-mon[32752]: pgmap v12860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22948.136933 secs 2024-04-27T00:57:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:47 smithi167 ceph-mon[32751]: pgmap v12860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22948.136933 secs 2024-04-27T00:57:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:49 smithi090 ceph-mon[29180]: pgmap v12861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:49 smithi119 ceph-mon[32752]: pgmap v12861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:49 smithi167 ceph-mon[32751]: pgmap v12861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:51 smithi090 ceph-mon[29180]: pgmap v12862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:51 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 22952.335432 secs 2024-04-27T00:57:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:51 smithi090 ceph-mon[29180]: slow request 721.709520 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:57:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:51 smithi119 ceph-mon[32752]: pgmap v12862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:51 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 22952.335432 secs 2024-04-27T00:57:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:51 smithi119 ceph-mon[32752]: slow request 721.709520 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:57:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:51 smithi167 ceph-mon[32751]: pgmap v12862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:51 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 22952.335432 secs 2024-04-27T00:57:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:51 smithi167 ceph-mon[32751]: slow request 721.709520 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T00:57:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22953.137011 secs 2024-04-27T00:57:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22953.137011 secs 2024-04-27T00:57:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22953.137011 secs 2024-04-27T00:57:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:53 smithi090 ceph-mon[29180]: pgmap v12863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:53.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:53 smithi119 ceph-mon[32752]: pgmap v12863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:53.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:53 smithi167 ceph-mon[32751]: pgmap v12863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:55 smithi090 ceph-mon[29180]: pgmap v12864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:55.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:55 smithi119 ceph-mon[32752]: pgmap v12864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:55 smithi167 ceph-mon[32751]: pgmap v12864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:57:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22957.335536 secs 2024-04-27T00:57:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22957.335536 secs 2024-04-27T00:57:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22957.335536 secs 2024-04-27T00:57:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:57 smithi090 ceph-mon[29180]: pgmap v12865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22958.137090 secs 2024-04-27T00:57:57.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:57 smithi167 ceph-mon[32751]: pgmap v12865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:57.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22958.137090 secs 2024-04-27T00:57:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:57 smithi119 ceph-mon[32752]: pgmap v12865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22958.137090 secs 2024-04-27T00:57:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:57:59 smithi090 ceph-mon[29180]: pgmap v12866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:57:59 smithi119 ceph-mon[32752]: pgmap v12866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:57:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:57:59 smithi167 ceph-mon[32751]: pgmap v12866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:01 smithi090 ceph-mon[29180]: pgmap v12867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22962.335597 secs 2024-04-27T00:58:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:01 smithi119 ceph-mon[32752]: pgmap v12867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22962.335597 secs 2024-04-27T00:58:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:01 smithi167 ceph-mon[32751]: pgmap v12867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22962.335597 secs 2024-04-27T00:58:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22963.137203 secs 2024-04-27T00:58:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22963.137203 secs 2024-04-27T00:58:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22963.137203 secs 2024-04-27T00:58:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:03 smithi119 ceph-mon[32752]: pgmap v12868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:03 smithi167 ceph-mon[32751]: pgmap v12868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:03 smithi090 ceph-mon[29180]: pgmap v12868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:05 smithi119 ceph-mon[32752]: pgmap v12869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:05 smithi167 ceph-mon[32751]: pgmap v12869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:05 smithi090 ceph-mon[29180]: pgmap v12869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22967.335684 secs 2024-04-27T00:58:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:58:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22967.335684 secs 2024-04-27T00:58:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:58:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22967.335684 secs 2024-04-27T00:58:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:58:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:07 smithi119 ceph-mon[32752]: pgmap v12870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22968.137291 secs 2024-04-27T00:58:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:58:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:58:07.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:07 smithi119 ceph-mon[32752]: pgmap v12871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:58:07.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:07 smithi119 ceph-mon[32752]: pgmap v12872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:07.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:07 smithi167 ceph-mon[32751]: pgmap v12870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22968.137291 secs 2024-04-27T00:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:07 smithi167 ceph-mon[32751]: pgmap v12871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:58:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:07 smithi167 ceph-mon[32751]: pgmap v12872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:58:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:07 smithi090 ceph-mon[29180]: pgmap v12870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22968.137291 secs 2024-04-27T00:58:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:58:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:58:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:07 smithi090 ceph-mon[29180]: pgmap v12871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T00:58:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:07 smithi090 ceph-mon[29180]: pgmap v12872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:58:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:09 smithi119 ceph-mon[32752]: pgmap v12873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:09 smithi167 ceph-mon[32751]: pgmap v12873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:09 smithi090 ceph-mon[29180]: pgmap v12873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22972.335768 secs 2024-04-27T00:58:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22972.335768 secs 2024-04-27T00:58:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22972.335768 secs 2024-04-27T00:58:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:11 smithi119 ceph-mon[32752]: pgmap v12874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:11 smithi167 ceph-mon[32751]: pgmap v12874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:11 smithi090 ceph-mon[29180]: pgmap v12874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22973.137442 secs 2024-04-27T00:58:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22973.137442 secs 2024-04-27T00:58:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22973.137442 secs 2024-04-27T00:58:13.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:13 smithi119 ceph-mon[32752]: pgmap v12875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:58:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:13 smithi167 ceph-mon[32751]: pgmap v12875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:58:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:13 smithi090 ceph-mon[29180]: pgmap v12875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T00:58:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:15 smithi119 ceph-mon[32752]: pgmap v12876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22977.335856 secs 2024-04-27T00:58:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:15 smithi167 ceph-mon[32751]: pgmap v12876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22977.335856 secs 2024-04-27T00:58:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:15 smithi090 ceph-mon[29180]: pgmap v12876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:58:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22977.335856 secs 2024-04-27T00:58:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22978.137520 secs 2024-04-27T00:58:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:17 smithi119 ceph-mon[32752]: pgmap v12877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:58:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22978.137520 secs 2024-04-27T00:58:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:17 smithi167 ceph-mon[32751]: pgmap v12877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:58:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22978.137520 secs 2024-04-27T00:58:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:17 smithi090 ceph-mon[29180]: pgmap v12877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:58:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:19 smithi119 ceph-mon[32752]: pgmap v12878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:19 smithi167 ceph-mon[32751]: pgmap v12878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:19 smithi090 ceph-mon[29180]: pgmap v12878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:20.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22982.335920 secs 2024-04-27T00:58:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22982.335920 secs 2024-04-27T00:58:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22982.335920 secs 2024-04-27T00:58:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:21 smithi119 ceph-mon[32752]: pgmap v12879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:21 smithi167 ceph-mon[32751]: pgmap v12879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:21 smithi090 ceph-mon[29180]: pgmap v12879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22983.137629 secs 2024-04-27T00:58:22.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22983.137629 secs 2024-04-27T00:58:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22983.137629 secs 2024-04-27T00:58:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:23 smithi119 ceph-mon[32752]: pgmap v12880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:23 smithi167 ceph-mon[32751]: pgmap v12880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:23 smithi090 ceph-mon[29180]: pgmap v12880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:25.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:25 smithi119 ceph-mon[32752]: pgmap v12881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22987.336004 secs 2024-04-27T00:58:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:25 smithi167 ceph-mon[32751]: pgmap v12881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22987.336004 secs 2024-04-27T00:58:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:25 smithi090 ceph-mon[29180]: pgmap v12881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22987.336004 secs 2024-04-27T00:58:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22988.137754 secs 2024-04-27T00:58:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:27 smithi119 ceph-mon[32752]: pgmap v12882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:27.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22988.137754 secs 2024-04-27T00:58:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:27 smithi167 ceph-mon[32751]: pgmap v12882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22988.137754 secs 2024-04-27T00:58:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:27 smithi090 ceph-mon[29180]: pgmap v12882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:29.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:29 smithi119 ceph-mon[32752]: pgmap v12883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:29.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:29 smithi167 ceph-mon[32751]: pgmap v12883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:29.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:29 smithi090 ceph-mon[29180]: pgmap v12883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:30.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22992.336109 secs 2024-04-27T00:58:30.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22992.336109 secs 2024-04-27T00:58:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22992.336109 secs 2024-04-27T00:58:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:31 smithi167 ceph-mon[32751]: pgmap v12884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:31.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:31 smithi090 ceph-mon[29180]: pgmap v12884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:31 smithi119 ceph-mon[32752]: pgmap v12884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22993.137865 secs 2024-04-27T00:58:32.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22993.137865 secs 2024-04-27T00:58:33.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22993.137865 secs 2024-04-27T00:58:33.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:33 smithi167 ceph-mon[32751]: pgmap v12885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:33.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:33 smithi090 ceph-mon[29180]: pgmap v12885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:33 smithi119 ceph-mon[32752]: pgmap v12885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:35.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:35 smithi090 ceph-mon[29180]: pgmap v12886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 22997.336193 secs 2024-04-27T00:58:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:35 smithi119 ceph-mon[32752]: pgmap v12886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 22997.336193 secs 2024-04-27T00:58:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:35 smithi167 ceph-mon[32751]: pgmap v12886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 22997.336193 secs 2024-04-27T00:58:37.375 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:58:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:58:37.118+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:58:37.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:58:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:58:37.118+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:58:37.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:58:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:58:37.118+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:58:37.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:58:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:58:37.118+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:58:37.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 00:58:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T00:58:37.118+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T00:58:37.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 22998.138011 secs 2024-04-27T00:58:37.855 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:37 smithi119 ceph-mon[32752]: pgmap v12887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:37.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 22998.138011 secs 2024-04-27T00:58:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:37 smithi090 ceph-mon[29180]: pgmap v12887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 22998.138011 secs 2024-04-27T00:58:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:37 smithi167 ceph-mon[32751]: pgmap v12887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:39 smithi090 ceph-mon[29180]: pgmap v12888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:39 smithi119 ceph-mon[32752]: pgmap v12888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:39 smithi167 ceph-mon[32751]: pgmap v12888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:40.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23002.336282 secs 2024-04-27T00:58:41.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23002.336282 secs 2024-04-27T00:58:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23002.336282 secs 2024-04-27T00:58:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:41 smithi090 ceph-mon[29180]: pgmap v12889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:42.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:41 smithi119 ceph-mon[32752]: pgmap v12889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:41 smithi167 ceph-mon[32751]: pgmap v12889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23003.138126 secs 2024-04-27T00:58:42.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:42 smithi090 ceph-mon[29180]: pgmap v12890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23003.138126 secs 2024-04-27T00:58:43.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:42 smithi119 ceph-mon[32752]: pgmap v12890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23003.138126 secs 2024-04-27T00:58:43.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:42 smithi167 ceph-mon[32751]: pgmap v12890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:45 smithi119 ceph-mon[32752]: pgmap v12891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:45.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23007.336367 secs 2024-04-27T00:58:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:45 smithi167 ceph-mon[32751]: pgmap v12891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:45.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23007.336367 secs 2024-04-27T00:58:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:45 smithi090 ceph-mon[29180]: pgmap v12891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:45.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23007.336367 secs 2024-04-27T00:58:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23008.138191 secs 2024-04-27T00:58:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:47 smithi119 ceph-mon[32752]: pgmap v12892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23008.138191 secs 2024-04-27T00:58:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:47 smithi167 ceph-mon[32751]: pgmap v12892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23008.138191 secs 2024-04-27T00:58:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:47 smithi090 ceph-mon[29180]: pgmap v12892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:49.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:49 smithi119 ceph-mon[32752]: pgmap v12893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:49.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:49 smithi167 ceph-mon[32751]: pgmap v12893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:49.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:49 smithi090 ceph-mon[29180]: pgmap v12893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T00:58:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23012.336446 secs 2024-04-27T00:58:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23012.336446 secs 2024-04-27T00:58:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23012.336446 secs 2024-04-27T00:58:51.776 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:51 smithi090 ceph-mon[29180]: pgmap v12894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:51 smithi119 ceph-mon[32752]: pgmap v12894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:51 smithi167 ceph-mon[32751]: pgmap v12894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23013.138310 secs 2024-04-27T00:58:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23013.138310 secs 2024-04-27T00:58:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23013.138310 secs 2024-04-27T00:58:53.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:53 smithi119 ceph-mon[32752]: pgmap v12895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:53.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:53 smithi167 ceph-mon[32751]: pgmap v12895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:53.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:53 smithi090 ceph-mon[29180]: pgmap v12895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:55 smithi119 ceph-mon[32752]: pgmap v12896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:55.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23017.336531 secs 2024-04-27T00:58:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:55 smithi167 ceph-mon[32751]: pgmap v12896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:55.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23017.336531 secs 2024-04-27T00:58:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:55 smithi090 ceph-mon[29180]: pgmap v12896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:55.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23017.336531 secs 2024-04-27T00:58:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23018.138451 secs 2024-04-27T00:58:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:57 smithi119 ceph-mon[32752]: pgmap v12897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23018.138451 secs 2024-04-27T00:58:57.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:57 smithi167 ceph-mon[32751]: pgmap v12897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23018.138451 secs 2024-04-27T00:58:57.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:57 smithi090 ceph-mon[29180]: pgmap v12897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:58:59.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:58:59 smithi119 ceph-mon[32752]: pgmap v12898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:59.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:58:59 smithi167 ceph-mon[32751]: pgmap v12898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:58:59.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:58:59 smithi090 ceph-mon[29180]: pgmap v12898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23022.336591 secs 2024-04-27T00:59:00.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23022.336591 secs 2024-04-27T00:59:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23022.336591 secs 2024-04-27T00:59:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:01 smithi119 ceph-mon[32752]: pgmap v12899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:01.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:01 smithi167 ceph-mon[32751]: pgmap v12899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:01 smithi090 ceph-mon[29180]: pgmap v12899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23023.138566 secs 2024-04-27T00:59:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23023.138566 secs 2024-04-27T00:59:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23023.138566 secs 2024-04-27T00:59:03.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:03 smithi167 ceph-mon[32751]: pgmap v12900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:03.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:03 smithi090 ceph-mon[29180]: pgmap v12900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:03 smithi119 ceph-mon[32752]: pgmap v12900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:05 smithi167 ceph-mon[32751]: pgmap v12901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:05.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23027.336676 secs 2024-04-27T00:59:05.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:05 smithi090 ceph-mon[29180]: pgmap v12901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:05.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23027.336676 secs 2024-04-27T00:59:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:05 smithi119 ceph-mon[32752]: pgmap v12901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23027.336676 secs 2024-04-27T00:59:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:59:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:59:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T00:59:07.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23028.138679 secs 2024-04-27T00:59:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:07 smithi090 ceph-mon[29180]: pgmap v12902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:59:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:59:07.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:59:07.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23028.138679 secs 2024-04-27T00:59:07.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:07 smithi119 ceph-mon[32752]: pgmap v12902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:07.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:59:07.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:59:07.955 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:59:08.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23028.138679 secs 2024-04-27T00:59:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:07 smithi167 ceph-mon[32751]: pgmap v12902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T00:59:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T00:59:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T00:59:08.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:08 smithi090 ceph-mon[29180]: pgmap v12903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:59:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:08 smithi090 ceph-mon[29180]: pgmap v12904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:59:09.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:08 smithi119 ceph-mon[32752]: pgmap v12903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:59:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:08 smithi119 ceph-mon[32752]: pgmap v12904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:59:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:08 smithi167 ceph-mon[32751]: pgmap v12903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T00:59:09.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:08 smithi167 ceph-mon[32751]: pgmap v12904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T00:59:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:10 smithi090 ceph-mon[29180]: pgmap v12905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:10 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 23032.336763 secs 2024-04-27T00:59:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:10 smithi090 ceph-mon[29180]: slow request 2883.744209 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:59:11.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:10 smithi119 ceph-mon[32752]: pgmap v12905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:10 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 23032.336763 secs 2024-04-27T00:59:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:10 smithi119 ceph-mon[32752]: slow request 2883.744209 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:59:11.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:10 smithi167 ceph-mon[32751]: pgmap v12905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:10 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 23032.336763 secs 2024-04-27T00:59:11.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:10 smithi167 ceph-mon[32751]: slow request 2883.744209 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:59:12.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23033.138750 secs 2024-04-27T00:59:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:12 smithi090 ceph-mon[29180]: pgmap v12906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:13.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23033.138750 secs 2024-04-27T00:59:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:12 smithi119 ceph-mon[32752]: pgmap v12906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:13.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23033.138750 secs 2024-04-27T00:59:13.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:12 smithi167 ceph-mon[32751]: pgmap v12906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:14 smithi090 ceph-mon[29180]: pgmap v12907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:15.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:14 smithi119 ceph-mon[32752]: pgmap v12907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:15.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:14 smithi167 ceph-mon[32751]: pgmap v12907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:15.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23037.336854 secs 2024-04-27T00:59:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:15 smithi090 ceph-mon[29180]: pgmap v12908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:16.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23037.336854 secs 2024-04-27T00:59:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:15 smithi119 ceph-mon[32752]: pgmap v12908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:16.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23037.336854 secs 2024-04-27T00:59:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:15 smithi167 ceph-mon[32751]: pgmap v12908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T00:59:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23038.138862 secs 2024-04-27T00:59:17.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23038.138862 secs 2024-04-27T00:59:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23038.138862 secs 2024-04-27T00:59:18.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:17 smithi090 ceph-mon[29180]: pgmap v12909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:59:18.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:17 smithi119 ceph-mon[32752]: pgmap v12909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:59:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:17 smithi167 ceph-mon[32751]: pgmap v12909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T00:59:20.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:20 smithi119 ceph-mon[32752]: pgmap v12910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:20.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:20 smithi167 ceph-mon[32751]: pgmap v12910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:20.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:20 smithi090 ceph-mon[29180]: pgmap v12910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:21.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:21 smithi119 ceph-mon[32752]: 32 slow requests, 4 included below; oldest blocked for > 23042.336943 secs 2024-04-27T00:59:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:21 smithi119 ceph-mon[32752]: slow request 23042.336942 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-27T00:59:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:21 smithi119 ceph-mon[32752]: slow request 23042.334048 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:21 smithi119 ceph-mon[32752]: slow request 23042.328437 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:21 smithi119 ceph-mon[32752]: slow request 23040.361712 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-27T00:59:21.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:21 smithi167 ceph-mon[32751]: 32 slow requests, 4 included below; oldest blocked for > 23042.336943 secs 2024-04-27T00:59:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:21 smithi167 ceph-mon[32751]: slow request 23042.336942 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-27T00:59:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:21 smithi167 ceph-mon[32751]: slow request 23042.334048 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:21 smithi167 ceph-mon[32751]: slow request 23042.328437 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:21 smithi167 ceph-mon[32751]: slow request 23040.361712 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-27T00:59:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:21 smithi090 ceph-mon[29180]: 32 slow requests, 4 included below; oldest blocked for > 23042.336943 secs 2024-04-27T00:59:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:21 smithi090 ceph-mon[29180]: slow request 23042.336942 seconds old, received at 2024-04-26T18:35:17.753404+0000: client_request(client.15213:304 rmsnap #0x1//scheduled-2024-04-26-18_29_00_UTC 2024-04-26T18:35:17.753197+0000 caller_uid=0, caller_gid=0{}) currently failed to xlock, waiting 2024-04-27T00:59:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:21 smithi090 ceph-mon[29180]: slow request 23042.334048 seconds old, received at 2024-04-26T18:35:17.756299+0000: client_request(client.24733:267878 rename #0x1000000217f/@@@CDRW.TMP #0x1000000217f/GRAPH1.CDR 2024-04-26T18:35:17.755936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:21 smithi090 ceph-mon[29180]: slow request 23042.328437 seconds old, received at 2024-04-26T18:35:17.761910+0000: client_request(client.24733:267874 lookup #0x1000000212e/TIPS 2024-04-26T18:35:17.751936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:21 smithi090 ceph-mon[29180]: slow request 23040.361712 seconds old, received at 2024-04-26T18:35:19.728635+0000: internal op exportdir:mds.0:186 currently failed to acquire snap lock 2024-04-27T00:59:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:22 smithi119 ceph-mon[32752]: 24 slow requests, 5 included below; oldest blocked for > 23043.138959 secs 2024-04-27T00:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:22 smithi119 ceph-mon[32752]: slow request 23043.138959 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:22 smithi119 ceph-mon[32752]: slow request 23043.137398 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:22 smithi119 ceph-mon[32752]: slow request 23043.135929 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:22 smithi119 ceph-mon[32752]: slow request 23043.135458 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:22 smithi119 ceph-mon[32752]: slow request 23043.123193 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:22 smithi119 ceph-mon[32752]: pgmap v12911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:22 smithi167 ceph-mon[32751]: 24 slow requests, 5 included below; oldest blocked for > 23043.138959 secs 2024-04-27T00:59:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:22 smithi167 ceph-mon[32751]: slow request 23043.138959 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:22 smithi167 ceph-mon[32751]: slow request 23043.137398 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:22 smithi167 ceph-mon[32751]: slow request 23043.135929 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:22 smithi167 ceph-mon[32751]: slow request 23043.135458 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:22 smithi167 ceph-mon[32751]: slow request 23043.123193 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:22 smithi167 ceph-mon[32751]: pgmap v12911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:22.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:22 smithi090 ceph-mon[29180]: 24 slow requests, 5 included below; oldest blocked for > 23043.138959 secs 2024-04-27T00:59:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:22 smithi090 ceph-mon[29180]: slow request 23043.138959 seconds old, received at 2024-04-26T18:35:17.750645+0000: client_request(client.24733:267868 lookup #0x100000021d6/NEWS1_1B.PRN 2024-04-26T18:35:17.744936+0000 FWD=1 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:22 smithi090 ceph-mon[29180]: slow request 23043.137398 seconds old, received at 2024-04-26T18:35:17.752206+0000: client_request(client.24733:267873 create owner_uid=1000, owner_gid=1316 #0x10000000229/__40D6B.DB 2024-04-26T18:35:17.751936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:22 smithi090 ceph-mon[29180]: slow request 23043.135929 seconds old, received at 2024-04-26T18:35:17.753675+0000: client_request(client.24733:267875 readdir #0x1000000217e GRAPH1.CDR 2024-04-26T18:35:17.752936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:22 smithi090 ceph-mon[29180]: slow request 23043.135458 seconds old, received at 2024-04-26T18:35:17.754146+0000: client_request(client.24733:267877 readdir #0x10000002199 ARTISTIC.ACL 2024-04-26T18:35:17.753936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:22 smithi090 ceph-mon[29180]: slow request 23043.123193 seconds old, received at 2024-04-26T18:35:17.766411+0000: client_request(client.24733:267881 readdir #0x10000002189 CORELDRW.CDT 2024-04-26T18:35:17.765936+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,}) currently failed to acquire snap lock 2024-04-27T00:59:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:22 smithi090 ceph-mon[29180]: pgmap v12911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:24.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:24 smithi119 ceph-mon[32752]: pgmap v12912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:24.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:24 smithi167 ceph-mon[32751]: pgmap v12912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:24.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:24 smithi090 ceph-mon[29180]: pgmap v12912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:26.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23047.337031 secs 2024-04-27T00:59:26.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:26 smithi167 ceph-mon[32751]: pgmap v12913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:26.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23047.337031 secs 2024-04-27T00:59:26.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:26 smithi090 ceph-mon[29180]: pgmap v12913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:26.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23047.337031 secs 2024-04-27T00:59:26.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:26 smithi119 ceph-mon[32752]: pgmap v12913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:27 smithi167 ceph-mon[32751]: 24 slow requests, 4 included below; oldest blocked for > 23048.139097 secs 2024-04-27T00:59:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:27 smithi167 ceph-mon[32751]: slow request 23044.554460 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-27T00:59:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:27 smithi167 ceph-mon[32751]: slow request 23044.554124 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-27T00:59:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:27 smithi167 ceph-mon[32751]: slow request 23044.445289 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-27T00:59:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:27 smithi167 ceph-mon[32751]: slow request 23044.441001 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-27T00:59:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:27 smithi090 ceph-mon[29180]: 24 slow requests, 4 included below; oldest blocked for > 23048.139097 secs 2024-04-27T00:59:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:27 smithi090 ceph-mon[29180]: slow request 23044.554460 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-27T00:59:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:27 smithi090 ceph-mon[29180]: slow request 23044.554124 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-27T00:59:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:27 smithi090 ceph-mon[29180]: slow request 23044.445289 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-27T00:59:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:27 smithi090 ceph-mon[29180]: slow request 23044.441001 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-27T00:59:27.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:27 smithi119 ceph-mon[32752]: 24 slow requests, 4 included below; oldest blocked for > 23048.139097 secs 2024-04-27T00:59:27.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:27 smithi119 ceph-mon[32752]: slow request 23044.554460 seconds old, received at 2024-04-26T18:35:21.335281+0000: internal op exportdir:mds.1:12 currently failed to acquire snap lock 2024-04-27T00:59:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:27 smithi119 ceph-mon[32752]: slow request 23044.554124 seconds old, received at 2024-04-26T18:35:21.335617+0000: internal op exportdir:mds.1:13 currently failed to acquire snap lock 2024-04-27T00:59:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:27 smithi119 ceph-mon[32752]: slow request 23044.445289 seconds old, received at 2024-04-26T18:35:21.444452+0000: peer_request:mds.2:2 currently dispatched 2024-04-27T00:59:27.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:27 smithi119 ceph-mon[32752]: slow request 23044.441001 seconds old, received at 2024-04-26T18:35:21.448741+0000: peer_request:mds.2:3 currently dispatched 2024-04-27T00:59:28.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:28 smithi167 ceph-mon[32751]: pgmap v12914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:28.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:28 smithi090 ceph-mon[29180]: pgmap v12914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:28.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:28 smithi119 ceph-mon[32752]: pgmap v12914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:30.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:30 smithi167 ceph-mon[32751]: pgmap v12915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:30.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:30 smithi090 ceph-mon[29180]: pgmap v12915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:30.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:30 smithi119 ceph-mon[32752]: pgmap v12915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23052.337118 secs 2024-04-27T00:59:31.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23052.337118 secs 2024-04-27T00:59:31.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23052.337118 secs 2024-04-27T00:59:32.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23053.139169 secs 2024-04-27T00:59:32.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:32 smithi090 ceph-mon[29180]: pgmap v12916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:32.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23053.139169 secs 2024-04-27T00:59:32.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:32 smithi119 ceph-mon[32752]: pgmap v12916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:32.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23053.139169 secs 2024-04-27T00:59:32.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:32 smithi167 ceph-mon[32751]: pgmap v12916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:34 smithi090 ceph-mon[29180]: pgmap v12917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:34.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:34 smithi119 ceph-mon[32752]: pgmap v12917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:34.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:34 smithi167 ceph-mon[32751]: pgmap v12917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:36.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23057.337204 secs 2024-04-27T00:59:36.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:36 smithi090 ceph-mon[29180]: pgmap v12918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:36.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23057.337204 secs 2024-04-27T00:59:36.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:36 smithi119 ceph-mon[32752]: pgmap v12918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:36.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23057.337204 secs 2024-04-27T00:59:36.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:36 smithi167 ceph-mon[32751]: pgmap v12918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:37.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23058.139345 secs 2024-04-27T00:59:37.703 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23058.139345 secs 2024-04-27T00:59:37.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23058.139345 secs 2024-04-27T00:59:38.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:38 smithi090 ceph-mon[29180]: pgmap v12919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:38.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:38 smithi119 ceph-mon[32752]: pgmap v12919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:38.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:38 smithi167 ceph-mon[32751]: pgmap v12919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:40.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:40 smithi090 ceph-mon[29180]: pgmap v12920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:40.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:40 smithi119 ceph-mon[32752]: pgmap v12920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:40.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:40 smithi167 ceph-mon[32751]: pgmap v12920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23062.337290 secs 2024-04-27T00:59:41.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23062.337290 secs 2024-04-27T00:59:41.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23062.337290 secs 2024-04-27T00:59:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:42 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 23063.139408 secs 2024-04-27T00:59:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:42 smithi090 ceph-mon[29180]: slow request 363.834696 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:59:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:42 smithi090 ceph-mon[29180]: pgmap v12921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:42.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:42 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 23063.139408 secs 2024-04-27T00:59:42.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:42 smithi119 ceph-mon[32752]: slow request 363.834696 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:59:42.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:42 smithi119 ceph-mon[32752]: pgmap v12921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:42.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:42 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 23063.139408 secs 2024-04-27T00:59:42.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:42 smithi167 ceph-mon[32751]: slow request 363.834696 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T00:59:42.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:42 smithi167 ceph-mon[32751]: pgmap v12921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:44 smithi090 ceph-mon[29180]: pgmap v12922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:44.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:44 smithi119 ceph-mon[32752]: pgmap v12922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:44.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:44 smithi167 ceph-mon[32751]: pgmap v12922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:46.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23067.337372 secs 2024-04-27T00:59:46.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:46 smithi090 ceph-mon[29180]: pgmap v12923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:46.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23067.337372 secs 2024-04-27T00:59:46.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:46 smithi119 ceph-mon[32752]: pgmap v12923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:46.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23067.337372 secs 2024-04-27T00:59:46.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:46 smithi167 ceph-mon[32751]: pgmap v12923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23068.139503 secs 2024-04-27T00:59:47.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23068.139503 secs 2024-04-27T00:59:47.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23068.139503 secs 2024-04-27T00:59:48.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:48 smithi090 ceph-mon[29180]: pgmap v12924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:48.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:48 smithi119 ceph-mon[32752]: pgmap v12924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:48.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:48 smithi167 ceph-mon[32751]: pgmap v12924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:50.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:50 smithi090 ceph-mon[29180]: pgmap v12925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:50.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:50 smithi119 ceph-mon[32752]: pgmap v12925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:50.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:50 smithi167 ceph-mon[32751]: pgmap v12925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:51.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23072.337433 secs 2024-04-27T00:59:51.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23072.337433 secs 2024-04-27T00:59:51.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23072.337433 secs 2024-04-27T00:59:52.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23073.139626 secs 2024-04-27T00:59:52.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:52 smithi090 ceph-mon[29180]: pgmap v12926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:52.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23073.139626 secs 2024-04-27T00:59:52.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:52 smithi119 ceph-mon[32752]: pgmap v12926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:52.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23073.139626 secs 2024-04-27T00:59:52.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:52 smithi167 ceph-mon[32751]: pgmap v12926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:54.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:54 smithi090 ceph-mon[29180]: pgmap v12927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:54.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:54 smithi119 ceph-mon[32752]: pgmap v12927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:54.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:54 smithi167 ceph-mon[32751]: pgmap v12927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T00:59:56.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23077.337521 secs 2024-04-27T00:59:56.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:56 smithi090 ceph-mon[29180]: pgmap v12928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:56.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23077.337521 secs 2024-04-27T00:59:56.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:56 smithi119 ceph-mon[32752]: pgmap v12928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:56.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23077.337521 secs 2024-04-27T00:59:56.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:56 smithi167 ceph-mon[32751]: pgmap v12928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:57.717 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23078.139750 secs 2024-04-27T00:59:57.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23078.139750 secs 2024-04-27T00:59:57.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23078.139750 secs 2024-04-27T00:59:58.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 00:59:58 smithi119 ceph-mon[32752]: pgmap v12929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:58.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 00:59:58 smithi167 ceph-mon[32751]: pgmap v12929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T00:59:58.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 00:59:58 smithi090 ceph-mon[29180]: pgmap v12929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:00.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:00 smithi119 ceph-mon[32752]: pgmap v12930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:00.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:00:00.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:00 smithi167 ceph-mon[32751]: pgmap v12930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:00.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:00:00.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:00 smithi090 ceph-mon[29180]: pgmap v12930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:00.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:00:01.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23082.337605 secs 2024-04-27T01:00:01.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23082.337605 secs 2024-04-27T01:00:01.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23082.337605 secs 2024-04-27T01:00:02.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23083.139872 secs 2024-04-27T01:00:02.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:02 smithi090 ceph-mon[29180]: pgmap v12931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:02.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23083.139872 secs 2024-04-27T01:00:02.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:02 smithi119 ceph-mon[32752]: pgmap v12931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:02.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23083.139872 secs 2024-04-27T01:00:02.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:02 smithi167 ceph-mon[32751]: pgmap v12931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:04.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:03 smithi119 ceph-mon[32752]: pgmap v12932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:04.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:03 smithi167 ceph-mon[32751]: pgmap v12932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:04.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:03 smithi090 ceph-mon[29180]: pgmap v12932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:06.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:06 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 23087.337691 secs 2024-04-27T01:00:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:06 smithi167 ceph-mon[32751]: slow request 23044.333083 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:00:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:06 smithi167 ceph-mon[32751]: pgmap v12933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:06.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:06 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 23087.337691 secs 2024-04-27T01:00:06.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:06 smithi090 ceph-mon[29180]: slow request 23044.333083 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:00:06.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:06 smithi090 ceph-mon[29180]: pgmap v12933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:06.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:06 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 23087.337691 secs 2024-04-27T01:00:06.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:06 smithi119 ceph-mon[32752]: slow request 23044.333083 seconds old, received at 2024-04-26T18:36:00.758012+0000: client_request(client.17043:1 getattr pAsLsXsFs #0x1 2024-04-26T18:36:00.754102+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:00:06.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:06 smithi119 ceph-mon[32752]: pgmap v12933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:07.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23088.139953 secs 2024-04-27T01:00:07.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23088.139953 secs 2024-04-27T01:00:07.702 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23088.139953 secs 2024-04-27T01:00:08.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:08 smithi167 ceph-mon[32751]: pgmap v12934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:08.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:00:08.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:00:08.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:00:08.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:00:08.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:08 smithi090 ceph-mon[29180]: pgmap v12934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:08.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:00:08.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:00:08.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:00:08.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:00:08.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:08 smithi119 ceph-mon[32752]: pgmap v12934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:08.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:00:08.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:00:08.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:00:08.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:00:09.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:09 smithi167 ceph-mon[32751]: pgmap v12935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:00:09.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:09 smithi167 ceph-mon[32751]: pgmap v12936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:00:09.570 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:09 smithi090 ceph-mon[29180]: pgmap v12935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:00:09.571 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:09 smithi090 ceph-mon[29180]: pgmap v12936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:00:09.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:09 smithi119 ceph-mon[32752]: pgmap v12935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:00:09.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:09 smithi119 ceph-mon[32752]: pgmap v12936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:00:11.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:11 smithi090 ceph-mon[29180]: pgmap v12937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:11.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23092.337777 secs 2024-04-27T01:00:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:11 smithi119 ceph-mon[32752]: pgmap v12937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23092.337777 secs 2024-04-27T01:00:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:11 smithi167 ceph-mon[32751]: pgmap v12937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23092.337777 secs 2024-04-27T01:00:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23093.140094 secs 2024-04-27T01:00:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23093.140094 secs 2024-04-27T01:00:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23093.140094 secs 2024-04-27T01:00:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:13 smithi090 ceph-mon[29180]: pgmap v12938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:00:13.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:13 smithi119 ceph-mon[32752]: pgmap v12938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:00:13.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:13 smithi167 ceph-mon[32751]: pgmap v12938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:00:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:15 smithi090 ceph-mon[29180]: pgmap v12939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:15.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:15 smithi119 ceph-mon[32752]: pgmap v12939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:15.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:15 smithi167 ceph-mon[32751]: pgmap v12939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23097.337862 secs 2024-04-27T01:00:16.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23097.337862 secs 2024-04-27T01:00:16.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23097.337862 secs 2024-04-27T01:00:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23098.140219 secs 2024-04-27T01:00:17.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:17 smithi090 ceph-mon[29180]: pgmap v12940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:17.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23098.140219 secs 2024-04-27T01:00:17.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:17 smithi119 ceph-mon[32752]: pgmap v12940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:17.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23098.140219 secs 2024-04-27T01:00:17.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:17 smithi167 ceph-mon[32751]: pgmap v12940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:00:19.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:19 smithi090 ceph-mon[29180]: pgmap v12941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:00:19.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:19 smithi119 ceph-mon[32752]: pgmap v12941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:00:19.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:19 smithi167 ceph-mon[32751]: pgmap v12941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:00:21.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:21 smithi090 ceph-mon[29180]: pgmap v12942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:21.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23102.337946 secs 2024-04-27T01:00:21.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:21 smithi119 ceph-mon[32752]: pgmap v12942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:21.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23102.337946 secs 2024-04-27T01:00:21.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:21 smithi167 ceph-mon[32751]: pgmap v12942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:21.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23102.337946 secs 2024-04-27T01:00:22.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23103.140341 secs 2024-04-27T01:00:22.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23103.140341 secs 2024-04-27T01:00:22.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23103.140341 secs 2024-04-27T01:00:23.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:23 smithi090 ceph-mon[29180]: pgmap v12943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:23.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:23 smithi119 ceph-mon[32752]: pgmap v12943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:23.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:23 smithi167 ceph-mon[32751]: pgmap v12943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:25.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:25 smithi090 ceph-mon[29180]: pgmap v12944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:25.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:25 smithi119 ceph-mon[32752]: pgmap v12944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:25.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:25 smithi167 ceph-mon[32751]: pgmap v12944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:26.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23107.338048 secs 2024-04-27T01:00:26.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23107.338048 secs 2024-04-27T01:00:26.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23107.338048 secs 2024-04-27T01:00:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23108.140436 secs 2024-04-27T01:00:27.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:27 smithi090 ceph-mon[29180]: pgmap v12945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23108.140436 secs 2024-04-27T01:00:27.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:27 smithi167 ceph-mon[32751]: pgmap v12945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:27.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23108.140436 secs 2024-04-27T01:00:27.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:27 smithi119 ceph-mon[32752]: pgmap v12945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:29.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:29 smithi090 ceph-mon[29180]: pgmap v12946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:29.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:29 smithi119 ceph-mon[32752]: pgmap v12946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:29.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:29 smithi167 ceph-mon[32751]: pgmap v12946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:31 smithi090 ceph-mon[29180]: pgmap v12947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23112.338132 secs 2024-04-27T01:00:31.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:31 smithi119 ceph-mon[32752]: pgmap v12947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:31.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23112.338132 secs 2024-04-27T01:00:31.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:31 smithi167 ceph-mon[32751]: pgmap v12947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:31.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23112.338132 secs 2024-04-27T01:00:32.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23113.140570 secs 2024-04-27T01:00:32.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23113.140570 secs 2024-04-27T01:00:32.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23113.140570 secs 2024-04-27T01:00:33.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:33 smithi119 ceph-mon[32752]: pgmap v12948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:33.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:33 smithi167 ceph-mon[32751]: pgmap v12948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:33.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:33 smithi090 ceph-mon[29180]: pgmap v12948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:35.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:35 smithi119 ceph-mon[32752]: pgmap v12949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:35.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:35 smithi167 ceph-mon[32751]: pgmap v12949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:35.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:35 smithi090 ceph-mon[29180]: pgmap v12949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:36.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23117.338215 secs 2024-04-27T01:00:36.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23117.338215 secs 2024-04-27T01:00:36.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23117.338215 secs 2024-04-27T01:00:37.702 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23118.140706 secs 2024-04-27T01:00:37.702 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:37 smithi119 ceph-mon[32752]: pgmap v12950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:37.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23118.140706 secs 2024-04-27T01:00:37.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:37 smithi167 ceph-mon[32751]: pgmap v12950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:37.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23118.140706 secs 2024-04-27T01:00:37.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:37 smithi090 ceph-mon[29180]: pgmap v12950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:39.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:39 smithi090 ceph-mon[29180]: pgmap v12951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:39.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:39 smithi119 ceph-mon[32752]: pgmap v12951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:39.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:39 smithi167 ceph-mon[32751]: pgmap v12951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:41.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:41 smithi119 ceph-mon[32752]: pgmap v12952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:41.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23122.338300 secs 2024-04-27T01:00:41.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:41 smithi167 ceph-mon[32751]: pgmap v12952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:41.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23122.338300 secs 2024-04-27T01:00:41.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:41 smithi090 ceph-mon[29180]: pgmap v12952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:41.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23122.338300 secs 2024-04-27T01:00:42.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23123.140845 secs 2024-04-27T01:00:42.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23123.140845 secs 2024-04-27T01:00:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23123.140845 secs 2024-04-27T01:00:43.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:43 smithi119 ceph-mon[32752]: pgmap v12953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:43.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:43 smithi167 ceph-mon[32751]: pgmap v12953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:43.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:43 smithi090 ceph-mon[29180]: pgmap v12953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:45.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:45 smithi119 ceph-mon[32752]: pgmap v12954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:45.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:45 smithi167 ceph-mon[32751]: pgmap v12954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:45.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:45 smithi090 ceph-mon[29180]: pgmap v12954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:46.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23127.338384 secs 2024-04-27T01:00:46.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23127.338384 secs 2024-04-27T01:00:46.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23127.338384 secs 2024-04-27T01:00:47.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23128.140988 secs 2024-04-27T01:00:47.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:47 smithi119 ceph-mon[32752]: pgmap v12955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:47.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23128.140988 secs 2024-04-27T01:00:47.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:47 smithi167 ceph-mon[32751]: pgmap v12955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:47.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23128.140988 secs 2024-04-27T01:00:47.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:47 smithi090 ceph-mon[29180]: pgmap v12955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:49.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:49 smithi119 ceph-mon[32752]: pgmap v12956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:49.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:49 smithi167 ceph-mon[32751]: pgmap v12956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:49.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:49 smithi090 ceph-mon[29180]: pgmap v12956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:51.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:51 smithi119 ceph-mon[32752]: pgmap v12957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:51.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23132.338467 secs 2024-04-27T01:00:51.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:51 smithi167 ceph-mon[32751]: pgmap v12957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:51.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23132.338467 secs 2024-04-27T01:00:51.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:51 smithi090 ceph-mon[29180]: pgmap v12957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:51.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23132.338467 secs 2024-04-27T01:00:52.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23133.141094 secs 2024-04-27T01:00:52.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23133.141094 secs 2024-04-27T01:00:52.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23133.141094 secs 2024-04-27T01:00:53.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:53 smithi119 ceph-mon[32752]: pgmap v12958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:53.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:53 smithi167 ceph-mon[32751]: pgmap v12958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:53.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:53 smithi090 ceph-mon[29180]: pgmap v12958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:55.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:55 smithi119 ceph-mon[32752]: pgmap v12959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:55.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:55 smithi167 ceph-mon[32751]: pgmap v12959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:55.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:55 smithi090 ceph-mon[29180]: pgmap v12959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:00:56.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23137.338526 secs 2024-04-27T01:00:56.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23137.338526 secs 2024-04-27T01:00:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23137.338526 secs 2024-04-27T01:00:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23138.141210 secs 2024-04-27T01:00:57.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:57 smithi167 ceph-mon[32751]: pgmap v12960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:57.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23138.141210 secs 2024-04-27T01:00:57.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:57 smithi119 ceph-mon[32752]: pgmap v12960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:57.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23138.141210 secs 2024-04-27T01:00:57.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:57 smithi090 ceph-mon[29180]: pgmap v12960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:59.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:00:59 smithi119 ceph-mon[32752]: pgmap v12961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:59.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:00:59 smithi167 ceph-mon[32751]: pgmap v12961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:00:59.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:00:59 smithi090 ceph-mon[29180]: pgmap v12961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:00.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23142.338610 secs 2024-04-27T01:01:00.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23142.338610 secs 2024-04-27T01:01:00.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23142.338610 secs 2024-04-27T01:01:01.723 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:01 smithi119 ceph-mon[32752]: pgmap v12962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:01.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:01 smithi167 ceph-mon[32751]: pgmap v12962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:01.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:01 smithi090 ceph-mon[29180]: pgmap v12962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:02.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23143.141333 secs 2024-04-27T01:01:02.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23143.141333 secs 2024-04-27T01:01:02.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23143.141333 secs 2024-04-27T01:01:03.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:03 smithi119 ceph-mon[32752]: pgmap v12963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:03.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:03 smithi167 ceph-mon[32751]: pgmap v12963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:03.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:03 smithi090 ceph-mon[29180]: pgmap v12963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:05.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:05 smithi119 ceph-mon[32752]: pgmap v12964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:05.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23147.338694 secs 2024-04-27T01:01:05.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:05 smithi167 ceph-mon[32751]: pgmap v12964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:05.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23147.338694 secs 2024-04-27T01:01:05.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:05 smithi090 ceph-mon[29180]: pgmap v12964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:05.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23147.338694 secs 2024-04-27T01:01:07.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23148.141451 secs 2024-04-27T01:01:07.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:07 smithi119 ceph-mon[32752]: pgmap v12965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:07.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23148.141451 secs 2024-04-27T01:01:07.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:07 smithi167 ceph-mon[32751]: pgmap v12965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:07.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23148.141451 secs 2024-04-27T01:01:07.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:07 smithi090 ceph-mon[29180]: pgmap v12965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:08.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:01:08.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:01:08.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:01:09.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:09 smithi119 ceph-mon[32752]: pgmap v12966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:09.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:09 smithi167 ceph-mon[32751]: pgmap v12966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:09.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:09 smithi090 ceph-mon[29180]: pgmap v12966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:09.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:09.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: pgmap v12967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23152.338768 secs 2024-04-27T01:01:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: pgmap v12967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23152.338768 secs 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: pgmap v12967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:11.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23152.338768 secs 2024-04-27T01:01:11.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:11.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:11.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23153.141518 secs 2024-04-27T01:01:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:01:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:01:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:01:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:01:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:01:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:01:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:01:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:01:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: pgmap v12968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-27T01:01:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: pgmap v12969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:12.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23153.141518 secs 2024-04-27T01:01:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:01:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:01:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:01:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:01:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: pgmap v12968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-27T01:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: pgmap v12969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:12.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23153.141518 secs 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:01:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: pgmap v12968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-27T01:01:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: pgmap v12969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:01:14.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:14 smithi167 ceph-mon[32751]: pgmap v12970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:14.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:14 smithi090 ceph-mon[29180]: pgmap v12970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:15.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:14 smithi119 ceph-mon[32752]: pgmap v12970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:15.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23157.338855 secs 2024-04-27T01:01:16.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23157.338855 secs 2024-04-27T01:01:16.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23157.338855 secs 2024-04-27T01:01:16.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:16 smithi090 ceph-mon[29180]: pgmap v12971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:17.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:16 smithi119 ceph-mon[32752]: pgmap v12971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:17.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:16 smithi167 ceph-mon[32751]: pgmap v12971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:17.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23158.141678 secs 2024-04-27T01:01:18.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23158.141678 secs 2024-04-27T01:01:18.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23158.141678 secs 2024-04-27T01:01:18.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:18 smithi090 ceph-mon[29180]: pgmap v12972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:19.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:18 smithi119 ceph-mon[32752]: pgmap v12972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:19.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:18 smithi167 ceph-mon[32751]: pgmap v12972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:20.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:20 smithi090 ceph-mon[29180]: pgmap v12973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:20.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23162.338916 secs 2024-04-27T01:01:21.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:20 smithi119 ceph-mon[32752]: pgmap v12973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:21.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23162.338916 secs 2024-04-27T01:01:21.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:20 smithi167 ceph-mon[32751]: pgmap v12973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:01:21.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23162.338916 secs 2024-04-27T01:01:22.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23163.141763 secs 2024-04-27T01:01:22.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:22 smithi090 ceph-mon[29180]: pgmap v12974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:01:23.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23163.141763 secs 2024-04-27T01:01:23.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:22 smithi119 ceph-mon[32752]: pgmap v12974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:01:23.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23163.141763 secs 2024-04-27T01:01:23.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:22 smithi167 ceph-mon[32751]: pgmap v12974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:01:24.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:24 smithi090 ceph-mon[29180]: pgmap v12975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:25.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:24 smithi119 ceph-mon[32752]: pgmap v12975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:25.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:24 smithi167 ceph-mon[32751]: pgmap v12975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:25.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23167.339006 secs 2024-04-27T01:01:26.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23167.339006 secs 2024-04-27T01:01:26.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23167.339006 secs 2024-04-27T01:01:26.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:26 smithi090 ceph-mon[29180]: pgmap v12976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:27.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:26 smithi119 ceph-mon[32752]: pgmap v12976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:27.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:26 smithi167 ceph-mon[32751]: pgmap v12976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:27.646 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23168.141904 secs 2024-04-27T01:01:27.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23168.141904 secs 2024-04-27T01:01:28.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23168.141904 secs 2024-04-27T01:01:28.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:28 smithi090 ceph-mon[29180]: pgmap v12977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:29.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:28 smithi119 ceph-mon[32752]: pgmap v12977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:29.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:28 smithi167 ceph-mon[32751]: pgmap v12977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:29.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:29 smithi090 ceph-mon[29180]: pgmap v12978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:30.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:29 smithi119 ceph-mon[32752]: pgmap v12978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:30.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:29 smithi167 ceph-mon[32751]: pgmap v12978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:30.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23172.339091 secs 2024-04-27T01:01:31.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23172.339091 secs 2024-04-27T01:01:31.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23172.339091 secs 2024-04-27T01:01:31.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:31 smithi090 ceph-mon[29180]: pgmap v12979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:32.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:31 smithi119 ceph-mon[32752]: pgmap v12979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:32.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:31 smithi167 ceph-mon[32751]: pgmap v12979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:32.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23173.141980 secs 2024-04-27T01:01:33.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23173.141980 secs 2024-04-27T01:01:33.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23173.141980 secs 2024-04-27T01:01:34.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:33 smithi119 ceph-mon[32752]: pgmap v12980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:34.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:33 smithi167 ceph-mon[32751]: pgmap v12980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:34.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:33 smithi090 ceph-mon[29180]: pgmap v12980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:35.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:35 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 23177.339149 secs 2024-04-27T01:01:35.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:35 smithi119 ceph-mon[32752]: slow request 23041.602501 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-27T01:01:35.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:35 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 23177.339149 secs 2024-04-27T01:01:35.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:35 smithi167 ceph-mon[32751]: slow request 23041.602501 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-27T01:01:35.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:35 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 23177.339149 secs 2024-04-27T01:01:35.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:35 smithi090 ceph-mon[29180]: slow request 23041.602501 seconds old, received at 2024-04-26T18:37:33.490052+0000: client_request(client.24733:267882 getattr AsXsFs #0x100000001fb 2024-04-26T18:37:33.488896+0000 caller_uid=0, caller_gid=0{6,}) currently failed to acquire snap lock 2024-04-27T01:01:36.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:36 smithi119 ceph-mon[32752]: pgmap v12981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:36.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:36 smithi167 ceph-mon[32751]: pgmap v12981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:36.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:36 smithi090 ceph-mon[29180]: pgmap v12981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:37.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23178.142063 secs 2024-04-27T01:01:37.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23178.142063 secs 2024-04-27T01:01:37.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23178.142063 secs 2024-04-27T01:01:38.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:38 smithi119 ceph-mon[32752]: pgmap v12982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:38.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:38 smithi167 ceph-mon[32751]: pgmap v12982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:38.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:38 smithi090 ceph-mon[29180]: pgmap v12982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:40.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:40 smithi090 ceph-mon[29180]: pgmap v12983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:40.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23182.339233 secs 2024-04-27T01:01:41.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:40 smithi119 ceph-mon[32752]: pgmap v12983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:41.079 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23182.339233 secs 2024-04-27T01:01:41.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:40 smithi167 ceph-mon[32751]: pgmap v12983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:41.080 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23182.339233 secs 2024-04-27T01:01:42.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23183.142202 secs 2024-04-27T01:01:42.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:42 smithi090 ceph-mon[29180]: pgmap v12984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:43.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23183.142202 secs 2024-04-27T01:01:43.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:42 smithi119 ceph-mon[32752]: pgmap v12984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:43.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23183.142202 secs 2024-04-27T01:01:43.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:42 smithi167 ceph-mon[32751]: pgmap v12984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:44.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:44 smithi090 ceph-mon[29180]: pgmap v12985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:45.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:44 smithi119 ceph-mon[32752]: pgmap v12985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:45.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:44 smithi167 ceph-mon[32751]: pgmap v12985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:45.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23187.339319 secs 2024-04-27T01:01:46.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23187.339319 secs 2024-04-27T01:01:46.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23187.339319 secs 2024-04-27T01:01:46.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:46 smithi090 ceph-mon[29180]: pgmap v12986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:47.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:46 smithi119 ceph-mon[32752]: pgmap v12986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:47.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:46 smithi167 ceph-mon[32751]: pgmap v12986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:47.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23188.142359 secs 2024-04-27T01:01:48.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23188.142359 secs 2024-04-27T01:01:48.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23188.142359 secs 2024-04-27T01:01:48.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:48 smithi090 ceph-mon[29180]: pgmap v12987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:49.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:48 smithi119 ceph-mon[32752]: pgmap v12987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:49.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:48 smithi167 ceph-mon[32751]: pgmap v12987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:50.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:50 smithi090 ceph-mon[29180]: pgmap v12988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:50.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23192.339406 secs 2024-04-27T01:01:51.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:50 smithi119 ceph-mon[32752]: pgmap v12988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:51.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23192.339406 secs 2024-04-27T01:01:51.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:50 smithi167 ceph-mon[32751]: pgmap v12988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:51.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23192.339406 secs 2024-04-27T01:01:52.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23193.142404 secs 2024-04-27T01:01:52.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:52 smithi090 ceph-mon[29180]: pgmap v12989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:53.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23193.142404 secs 2024-04-27T01:01:53.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:52 smithi119 ceph-mon[32752]: pgmap v12989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:53.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23193.142404 secs 2024-04-27T01:01:53.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:52 smithi167 ceph-mon[32751]: pgmap v12989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:54.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:54 smithi090 ceph-mon[29180]: pgmap v12990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:55.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:54 smithi119 ceph-mon[32752]: pgmap v12990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:55.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:54 smithi167 ceph-mon[32751]: pgmap v12990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:01:55.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23197.339491 secs 2024-04-27T01:01:56.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23197.339491 secs 2024-04-27T01:01:56.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23197.339491 secs 2024-04-27T01:01:56.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:56 smithi090 ceph-mon[29180]: pgmap v12991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:57.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:56 smithi119 ceph-mon[32752]: pgmap v12991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:57.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:56 smithi167 ceph-mon[32751]: pgmap v12991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:57.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23198.142525 secs 2024-04-27T01:01:57.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:01:57 smithi090 ceph-mon[29180]: pgmap v12992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:57.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23198.142525 secs 2024-04-27T01:01:57.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:01:57 smithi167 ceph-mon[32751]: pgmap v12992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:01:58.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23198.142525 secs 2024-04-27T01:01:58.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:01:57 smithi119 ceph-mon[32752]: pgmap v12992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:00.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:00 smithi090 ceph-mon[29180]: pgmap v12993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:00.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23202.339556 secs 2024-04-27T01:02:01.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:00 smithi119 ceph-mon[32752]: pgmap v12993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:01.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23202.339556 secs 2024-04-27T01:02:01.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:00 smithi167 ceph-mon[32751]: pgmap v12993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:01.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23202.339556 secs 2024-04-27T01:02:02.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23203.142635 secs 2024-04-27T01:02:02.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:02 smithi090 ceph-mon[29180]: pgmap v12994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:03.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23203.142635 secs 2024-04-27T01:02:03.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:02 smithi119 ceph-mon[32752]: pgmap v12994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:03.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23203.142635 secs 2024-04-27T01:02:03.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:02 smithi167 ceph-mon[32751]: pgmap v12994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:04.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:04 smithi090 ceph-mon[29180]: pgmap v12995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:05.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:04 smithi119 ceph-mon[32752]: pgmap v12995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:05.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:04 smithi167 ceph-mon[32751]: pgmap v12995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:05.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23207.339644 secs 2024-04-27T01:02:06.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23207.339644 secs 2024-04-27T01:02:06.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23207.339644 secs 2024-04-27T01:02:06.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:06 smithi090 ceph-mon[29180]: pgmap v12996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:07.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:06 smithi119 ceph-mon[32752]: pgmap v12996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:07.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:06 smithi167 ceph-mon[32751]: pgmap v12996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:07.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23208.142762 secs 2024-04-27T01:02:07.950 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23208.142762 secs 2024-04-27T01:02:08.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23208.142762 secs 2024-04-27T01:02:08.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:08 smithi090 ceph-mon[29180]: pgmap v12997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:09.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:08 smithi119 ceph-mon[32752]: pgmap v12997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:09.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:08 smithi167 ceph-mon[32751]: pgmap v12997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:10.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:10 smithi090 ceph-mon[29180]: pgmap v12998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:10.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23212.339730 secs 2024-04-27T01:02:11.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:10 smithi119 ceph-mon[32752]: pgmap v12998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:11.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23212.339730 secs 2024-04-27T01:02:11.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:10 smithi167 ceph-mon[32751]: pgmap v12998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:11.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23212.339730 secs 2024-04-27T01:02:12.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23213.142869 secs 2024-04-27T01:02:12.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:12 smithi090 ceph-mon[29180]: pgmap v12999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:12.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:02:12.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:12.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:13.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23213.142869 secs 2024-04-27T01:02:13.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:12 smithi119 ceph-mon[32752]: pgmap v12999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:13.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:02:13.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:13.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:13.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23213.142869 secs 2024-04-27T01:02:13.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:12 smithi167 ceph-mon[32751]: pgmap v12999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:13.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:02:13.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:13.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:13.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:02:13.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:02:13.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:14.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:02:14.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:02:14.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:14.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:02:14.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:02:14.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:02:14.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:14 smithi090 ceph-mon[29180]: pgmap v13000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T01:02:14.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:14 smithi090 ceph-mon[29180]: pgmap v13001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T01:02:15.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:14 smithi119 ceph-mon[32752]: pgmap v13000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T01:02:15.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:14 smithi119 ceph-mon[32752]: pgmap v13001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T01:02:15.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:14 smithi167 ceph-mon[32751]: pgmap v13000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T01:02:15.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:14 smithi167 ceph-mon[32751]: pgmap v13001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T01:02:15.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:15 smithi090 ceph-mon[29180]: pgmap v13002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:15.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23217.339818 secs 2024-04-27T01:02:16.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:15 smithi119 ceph-mon[32752]: pgmap v13002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:16.049 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23217.339818 secs 2024-04-27T01:02:16.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:15 smithi167 ceph-mon[32751]: pgmap v13002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:16.063 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23217.339818 secs 2024-04-27T01:02:17.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23218.142998 secs 2024-04-27T01:02:17.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23218.142998 secs 2024-04-27T01:02:17.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23218.142998 secs 2024-04-27T01:02:18.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:17 smithi090 ceph-mon[29180]: pgmap v13003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:18.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:17 smithi119 ceph-mon[32752]: pgmap v13003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:18.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:17 smithi167 ceph-mon[32751]: pgmap v13003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:20.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:19 smithi090 ceph-mon[29180]: pgmap v13004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:20.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:19 smithi119 ceph-mon[32752]: pgmap v13004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:20.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:19 smithi167 ceph-mon[32751]: pgmap v13004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:21.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23222.339903 secs 2024-04-27T01:02:21.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23222.339903 secs 2024-04-27T01:02:21.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23222.339903 secs 2024-04-27T01:02:22.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:21 smithi119 ceph-mon[32752]: pgmap v13005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:22.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23223.143126 secs 2024-04-27T01:02:22.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:21 smithi167 ceph-mon[32751]: pgmap v13005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:22.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23223.143126 secs 2024-04-27T01:02:22.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:21 smithi090 ceph-mon[29180]: pgmap v13005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:02:22.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23223.143126 secs 2024-04-27T01:02:24.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:23 smithi119 ceph-mon[32752]: pgmap v13006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:02:24.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:23 smithi167 ceph-mon[32751]: pgmap v13006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:02:24.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:23 smithi090 ceph-mon[29180]: pgmap v13006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:02:26.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:25 smithi119 ceph-mon[32752]: pgmap v13007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:26.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23227.339991 secs 2024-04-27T01:02:26.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:25 smithi167 ceph-mon[32751]: pgmap v13007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:26.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23227.339991 secs 2024-04-27T01:02:26.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:25 smithi090 ceph-mon[29180]: pgmap v13007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:26.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23227.339991 secs 2024-04-27T01:02:27.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23228.143245 secs 2024-04-27T01:02:27.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23228.143245 secs 2024-04-27T01:02:27.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23228.143245 secs 2024-04-27T01:02:28.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:27 smithi119 ceph-mon[32752]: pgmap v13008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:28.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:27 smithi167 ceph-mon[32751]: pgmap v13008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:28.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:27 smithi090 ceph-mon[29180]: pgmap v13008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:30.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:29 smithi090 ceph-mon[29180]: pgmap v13009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:30.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:29 smithi119 ceph-mon[32752]: pgmap v13009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:30.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:29 smithi167 ceph-mon[32751]: pgmap v13009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:31.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23232.340079 secs 2024-04-27T01:02:31.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23232.340079 secs 2024-04-27T01:02:31.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23232.340079 secs 2024-04-27T01:02:32.272 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:31 smithi090 ceph-mon[29180]: pgmap v13010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:32.273 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23233.143368 secs 2024-04-27T01:02:32.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:31 smithi119 ceph-mon[32752]: pgmap v13010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:32.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23233.143368 secs 2024-04-27T01:02:32.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:31 smithi167 ceph-mon[32751]: pgmap v13010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:32.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23233.143368 secs 2024-04-27T01:02:34.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:33 smithi119 ceph-mon[32752]: pgmap v13011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:34.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:33 smithi167 ceph-mon[32751]: pgmap v13011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:34.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:33 smithi090 ceph-mon[29180]: pgmap v13011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:35 smithi119 ceph-mon[32752]: pgmap v13012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:36.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23237.340167 secs 2024-04-27T01:02:36.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:35 smithi167 ceph-mon[32751]: pgmap v13012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:36.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23237.340167 secs 2024-04-27T01:02:36.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:35 smithi090 ceph-mon[29180]: pgmap v13012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:36.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23237.340167 secs 2024-04-27T01:02:37.200 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23238.143411 secs 2024-04-27T01:02:37.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23238.143411 secs 2024-04-27T01:02:37.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23238.143411 secs 2024-04-27T01:02:38.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:37 smithi119 ceph-mon[32752]: pgmap v13013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:38.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:37 smithi167 ceph-mon[32751]: pgmap v13013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:38.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:37 smithi090 ceph-mon[29180]: pgmap v13013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:40.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:39 smithi119 ceph-mon[32752]: pgmap v13014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:40.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:39 smithi167 ceph-mon[32751]: pgmap v13014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:40.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:39 smithi090 ceph-mon[29180]: pgmap v13014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:41.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23242.340228 secs 2024-04-27T01:02:41.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23242.340228 secs 2024-04-27T01:02:41.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23242.340228 secs 2024-04-27T01:02:42.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:41 smithi119 ceph-mon[32752]: pgmap v13015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:42.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23243.143530 secs 2024-04-27T01:02:42.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:41 smithi167 ceph-mon[32751]: pgmap v13015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:42.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23243.143530 secs 2024-04-27T01:02:42.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:41 smithi090 ceph-mon[29180]: pgmap v13015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:42.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23243.143530 secs 2024-04-27T01:02:44.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:43 smithi119 ceph-mon[32752]: pgmap v13016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:44.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:43 smithi167 ceph-mon[32751]: pgmap v13016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:44.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:43 smithi090 ceph-mon[29180]: pgmap v13016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:46.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:45 smithi090 ceph-mon[29180]: pgmap v13017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:46.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23247.340317 secs 2024-04-27T01:02:46.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:45 smithi119 ceph-mon[32752]: pgmap v13017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:46.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23247.340317 secs 2024-04-27T01:02:46.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:45 smithi167 ceph-mon[32751]: pgmap v13017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:46.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23247.340317 secs 2024-04-27T01:02:47.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23248.143602 secs 2024-04-27T01:02:47.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23248.143602 secs 2024-04-27T01:02:47.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23248.143602 secs 2024-04-27T01:02:48.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:47 smithi119 ceph-mon[32752]: pgmap v13018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:48.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:47 smithi167 ceph-mon[32751]: pgmap v13018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:48.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:47 smithi090 ceph-mon[29180]: pgmap v13018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:50.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:49 smithi119 ceph-mon[32752]: pgmap v13019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:50.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:49 smithi167 ceph-mon[32751]: pgmap v13019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:50.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:49 smithi090 ceph-mon[29180]: pgmap v13019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:51.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23252.340405 secs 2024-04-27T01:02:51.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23252.340405 secs 2024-04-27T01:02:51.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23252.340405 secs 2024-04-27T01:02:52.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:51 smithi119 ceph-mon[32752]: pgmap v13020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:52.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23253.143678 secs 2024-04-27T01:02:52.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:51 smithi167 ceph-mon[32751]: pgmap v13020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:52.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23253.143678 secs 2024-04-27T01:02:52.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:51 smithi090 ceph-mon[29180]: pgmap v13020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:52.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23253.143678 secs 2024-04-27T01:02:54.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:53 smithi119 ceph-mon[32752]: pgmap v13021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:54.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:53 smithi167 ceph-mon[32751]: pgmap v13021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:54.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:53 smithi090 ceph-mon[29180]: pgmap v13021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:56.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:55 smithi119 ceph-mon[32752]: pgmap v13022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:56.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23257.340493 secs 2024-04-27T01:02:56.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:55 smithi167 ceph-mon[32751]: pgmap v13022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:56.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23257.340493 secs 2024-04-27T01:02:56.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:55 smithi090 ceph-mon[29180]: pgmap v13022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:02:56.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23257.340493 secs 2024-04-27T01:02:57.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23258.143814 secs 2024-04-27T01:02:57.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23258.143814 secs 2024-04-27T01:02:57.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23258.143814 secs 2024-04-27T01:02:58.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:57 smithi119 ceph-mon[32752]: pgmap v13023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:58.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:57 smithi167 ceph-mon[32751]: pgmap v13023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:02:58.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:57 smithi090 ceph-mon[29180]: pgmap v13023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:00.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:02:59 smithi119 ceph-mon[32752]: pgmap v13024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:00.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:02:59 smithi167 ceph-mon[32751]: pgmap v13024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:00.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:02:59 smithi090 ceph-mon[29180]: pgmap v13024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:01.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23262.340582 secs 2024-04-27T01:03:01.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23262.340582 secs 2024-04-27T01:03:01.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23262.340582 secs 2024-04-27T01:03:02.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:01 smithi119 ceph-mon[32752]: pgmap v13025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:02.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23263.143930 secs 2024-04-27T01:03:02.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:01 smithi167 ceph-mon[32751]: pgmap v13025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:02.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23263.143930 secs 2024-04-27T01:03:02.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:01 smithi090 ceph-mon[29180]: pgmap v13025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:02.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23263.143930 secs 2024-04-27T01:03:04.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:03 smithi119 ceph-mon[32752]: pgmap v13026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:04.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:03 smithi167 ceph-mon[32751]: pgmap v13026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:04.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:03 smithi090 ceph-mon[29180]: pgmap v13026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:06.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:05 smithi119 ceph-mon[32752]: pgmap v13027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:06.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23267.340672 secs 2024-04-27T01:03:06.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:05 smithi167 ceph-mon[32751]: pgmap v13027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:06.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23267.340672 secs 2024-04-27T01:03:06.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:05 smithi090 ceph-mon[29180]: pgmap v13027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:06.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23267.340672 secs 2024-04-27T01:03:07.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23268.144046 secs 2024-04-27T01:03:07.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23268.144046 secs 2024-04-27T01:03:07.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23268.144046 secs 2024-04-27T01:03:08.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:08 smithi119 ceph-mon[32752]: pgmap v13028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:08.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:08 smithi167 ceph-mon[32751]: pgmap v13028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:08.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:08 smithi090 ceph-mon[29180]: pgmap v13028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:10.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:10 smithi119 ceph-mon[32752]: pgmap v13029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:10.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:10 smithi167 ceph-mon[32751]: pgmap v13029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:10.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:10 smithi090 ceph-mon[29180]: pgmap v13029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:11.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:11 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 23272.340762 secs 2024-04-27T01:03:11.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:11 smithi119 ceph-mon[32752]: slow request 2882.545798 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:03:11.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:11 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 23272.340762 secs 2024-04-27T01:03:11.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:11 smithi167 ceph-mon[32751]: slow request 2882.545798 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:03:11.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:11 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 23272.340762 secs 2024-04-27T01:03:11.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:11 smithi090 ceph-mon[29180]: slow request 2882.545798 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:03:12.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:12 smithi119 ceph-mon[32752]: pgmap v13030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:12.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23273.144156 secs 2024-04-27T01:03:12.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:12 smithi167 ceph-mon[32751]: pgmap v13030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:12.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23273.144156 secs 2024-04-27T01:03:12.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:12 smithi090 ceph-mon[29180]: pgmap v13030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:12.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23273.144156 secs 2024-04-27T01:03:13.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:03:13.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:03:13.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:03:14.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:14 smithi119 ceph-mon[32752]: pgmap v13031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:14.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:03:14.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:03:14.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:03:14.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:14 smithi167 ceph-mon[32751]: pgmap v13031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:14.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:03:14.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:03:14.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:03:14.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:14 smithi090 ceph-mon[29180]: pgmap v13031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:14.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:03:14.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:03:14.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:03:15.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:15 smithi119 ceph-mon[32752]: pgmap v13032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:03:15.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:15 smithi119 ceph-mon[32752]: pgmap v13033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:15.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:15 smithi167 ceph-mon[32751]: pgmap v13032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:03:15.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:15 smithi167 ceph-mon[32751]: pgmap v13033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:15.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:15 smithi090 ceph-mon[29180]: pgmap v13032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:03:15.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:15 smithi090 ceph-mon[29180]: pgmap v13033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:16.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23277.340831 secs 2024-04-27T01:03:16.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23277.340831 secs 2024-04-27T01:03:16.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23277.340831 secs 2024-04-27T01:03:17.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:17 smithi119 ceph-mon[32752]: pgmap v13034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:17.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23278.144272 secs 2024-04-27T01:03:17.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:17 smithi167 ceph-mon[32751]: pgmap v13034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:17.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23278.144272 secs 2024-04-27T01:03:17.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:17 smithi090 ceph-mon[29180]: pgmap v13034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:17.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23278.144272 secs 2024-04-27T01:03:19.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:19 smithi119 ceph-mon[32752]: pgmap v13035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T01:03:19.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:19 smithi167 ceph-mon[32751]: pgmap v13035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T01:03:19.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:19 smithi090 ceph-mon[29180]: pgmap v13035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T01:03:21.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:21 smithi119 ceph-mon[32752]: pgmap v13036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:21.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23282.340894 secs 2024-04-27T01:03:21.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:21 smithi167 ceph-mon[32751]: pgmap v13036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:21.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23282.340894 secs 2024-04-27T01:03:21.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:21 smithi090 ceph-mon[29180]: pgmap v13036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:21.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23282.340894 secs 2024-04-27T01:03:22.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23283.144391 secs 2024-04-27T01:03:22.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23283.144391 secs 2024-04-27T01:03:22.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23283.144391 secs 2024-04-27T01:03:23.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:23 smithi167 ceph-mon[32751]: pgmap v13037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:23.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:23 smithi090 ceph-mon[29180]: pgmap v13037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:23.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:23 smithi119 ceph-mon[32752]: pgmap v13037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:03:25.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:25 smithi167 ceph-mon[32751]: pgmap v13038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:03:25.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:25 smithi090 ceph-mon[29180]: pgmap v13038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:03:25.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:25 smithi119 ceph-mon[32752]: pgmap v13038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:03:26.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23287.340985 secs 2024-04-27T01:03:26.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23287.340985 secs 2024-04-27T01:03:26.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23287.340985 secs 2024-04-27T01:03:27.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:27 smithi167 ceph-mon[32751]: pgmap v13039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:27.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23288.144502 secs 2024-04-27T01:03:27.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:27 smithi090 ceph-mon[29180]: pgmap v13039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:27.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23288.144502 secs 2024-04-27T01:03:27.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:27 smithi119 ceph-mon[32752]: pgmap v13039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:27.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23288.144502 secs 2024-04-27T01:03:29.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:29 smithi167 ceph-mon[32751]: pgmap v13040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:29.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:29 smithi090 ceph-mon[29180]: pgmap v13040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:29.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:29 smithi119 ceph-mon[32752]: pgmap v13040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:31.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:31 smithi090 ceph-mon[29180]: pgmap v13041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:31.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23292.341072 secs 2024-04-27T01:03:31.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:31 smithi119 ceph-mon[32752]: pgmap v13041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:31.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23292.341072 secs 2024-04-27T01:03:31.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:31 smithi167 ceph-mon[32751]: pgmap v13041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:31.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23292.341072 secs 2024-04-27T01:03:32.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23293.144620 secs 2024-04-27T01:03:32.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23293.144620 secs 2024-04-27T01:03:32.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23293.144620 secs 2024-04-27T01:03:33.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:33 smithi090 ceph-mon[29180]: pgmap v13042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:33.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:33 smithi119 ceph-mon[32752]: pgmap v13042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:33.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:33 smithi167 ceph-mon[32751]: pgmap v13042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:35.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:35 smithi090 ceph-mon[29180]: pgmap v13043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:35.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:35 smithi119 ceph-mon[32752]: pgmap v13043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:35.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:35 smithi167 ceph-mon[32751]: pgmap v13043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:36.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23297.341136 secs 2024-04-27T01:03:36.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23297.341136 secs 2024-04-27T01:03:36.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23297.341136 secs 2024-04-27T01:03:37.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:37 smithi090 ceph-mon[29180]: pgmap v13044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:37.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23298.144736 secs 2024-04-27T01:03:37.449 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:37 smithi119 ceph-mon[32752]: pgmap v13044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:37.449 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23298.144736 secs 2024-04-27T01:03:37.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:37 smithi167 ceph-mon[32751]: pgmap v13044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:37.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23298.144736 secs 2024-04-27T01:03:39.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:39 smithi090 ceph-mon[29180]: pgmap v13045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:39.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:39 smithi119 ceph-mon[32752]: pgmap v13045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:39.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:39 smithi167 ceph-mon[32751]: pgmap v13045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:41.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:41 smithi090 ceph-mon[29180]: pgmap v13046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:41.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23302.341197 secs 2024-04-27T01:03:41.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:41 smithi119 ceph-mon[32752]: pgmap v13046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:41.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23302.341197 secs 2024-04-27T01:03:41.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:41 smithi167 ceph-mon[32751]: pgmap v13046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:41.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23302.341197 secs 2024-04-27T01:03:42.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23303.144850 secs 2024-04-27T01:03:42.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23303.144850 secs 2024-04-27T01:03:42.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23303.144850 secs 2024-04-27T01:03:43.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:43 smithi090 ceph-mon[29180]: pgmap v13047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:43.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:43 smithi119 ceph-mon[32752]: pgmap v13047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:43.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:43 smithi167 ceph-mon[32751]: pgmap v13047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:45.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:45 smithi090 ceph-mon[29180]: pgmap v13048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:45.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:45 smithi119 ceph-mon[32752]: pgmap v13048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:45.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:45 smithi167 ceph-mon[32751]: pgmap v13048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:46.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23307.341258 secs 2024-04-27T01:03:46.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23307.341258 secs 2024-04-27T01:03:46.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23307.341258 secs 2024-04-27T01:03:47.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:47 smithi090 ceph-mon[29180]: pgmap v13049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:47.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23308.144983 secs 2024-04-27T01:03:47.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:47 smithi119 ceph-mon[32752]: pgmap v13049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:47.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23308.144983 secs 2024-04-27T01:03:47.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:47 smithi167 ceph-mon[32751]: pgmap v13049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:47.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23308.144983 secs 2024-04-27T01:03:49.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:49 smithi090 ceph-mon[29180]: pgmap v13050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:49.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:49 smithi119 ceph-mon[32752]: pgmap v13050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:49.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:49 smithi167 ceph-mon[32751]: pgmap v13050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:03:51.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:51 smithi090 ceph-mon[29180]: pgmap v13051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:51.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23312.341344 secs 2024-04-27T01:03:51.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:51 smithi119 ceph-mon[32752]: pgmap v13051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:51.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23312.341344 secs 2024-04-27T01:03:51.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:51 smithi167 ceph-mon[32751]: pgmap v13051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:03:51.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23312.341344 secs 2024-04-27T01:03:52.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23313.145128 secs 2024-04-27T01:03:52.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23313.145128 secs 2024-04-27T01:03:52.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23313.145128 secs 2024-04-27T01:03:53.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:53 smithi119 ceph-mon[32752]: pgmap v13052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:53.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:53 smithi167 ceph-mon[32751]: pgmap v13052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:53.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:53 smithi090 ceph-mon[29180]: pgmap v13052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:55.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:55 smithi119 ceph-mon[32752]: pgmap v13053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:55.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:55 smithi167 ceph-mon[32751]: pgmap v13053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:55.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:55 smithi090 ceph-mon[29180]: pgmap v13053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:03:56.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23317.341429 secs 2024-04-27T01:03:56.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23317.341429 secs 2024-04-27T01:03:56.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23317.341429 secs 2024-04-27T01:03:57.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:57 smithi167 ceph-mon[32751]: pgmap v13054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:57.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23318.145244 secs 2024-04-27T01:03:57.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:57 smithi119 ceph-mon[32752]: pgmap v13054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:57.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23318.145244 secs 2024-04-27T01:03:57.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:57 smithi090 ceph-mon[29180]: pgmap v13054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:57.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23318.145244 secs 2024-04-27T01:03:59.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:03:59 smithi119 ceph-mon[32752]: pgmap v13055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:59.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:03:59 smithi167 ceph-mon[32751]: pgmap v13055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:03:59.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:03:59 smithi090 ceph-mon[29180]: pgmap v13055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:01.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:01 smithi119 ceph-mon[32752]: pgmap v13056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:01.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23322.341516 secs 2024-04-27T01:04:01.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:01 smithi167 ceph-mon[32751]: pgmap v13056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:01.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23322.341516 secs 2024-04-27T01:04:01.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:01 smithi090 ceph-mon[29180]: pgmap v13056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:01.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23322.341516 secs 2024-04-27T01:04:02.521 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23323.145359 secs 2024-04-27T01:04:02.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23323.145359 secs 2024-04-27T01:04:02.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23323.145359 secs 2024-04-27T01:04:03.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:03 smithi119 ceph-mon[32752]: pgmap v13057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:03.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:03 smithi090 ceph-mon[29180]: pgmap v13057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:03.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:03 smithi167 ceph-mon[32751]: pgmap v13057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:05.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:05 smithi119 ceph-mon[32752]: pgmap v13058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:05.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:05 smithi167 ceph-mon[32751]: pgmap v13058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:05.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:05 smithi090 ceph-mon[29180]: pgmap v13058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:06.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23327.341604 secs 2024-04-27T01:04:06.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23327.341604 secs 2024-04-27T01:04:06.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23327.341604 secs 2024-04-27T01:04:07.448 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:07 smithi119 ceph-mon[32752]: pgmap v13059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:07.449 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23328.145478 secs 2024-04-27T01:04:07.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:07 smithi167 ceph-mon[32751]: pgmap v13059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:07.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23328.145478 secs 2024-04-27T01:04:07.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:07 smithi090 ceph-mon[29180]: pgmap v13059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:07.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23328.145478 secs 2024-04-27T01:04:07.620 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:04:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:04:07.229+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:04:07.620 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:04:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:04:07.229+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:04:07.620 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:04:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:04:07.229+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:04:07.620 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:04:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:04:07.229+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:04:07.621 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:04:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:04:07.229+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:04:09.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:09 smithi119 ceph-mon[32752]: pgmap v13060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:09.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:09 smithi167 ceph-mon[32751]: pgmap v13060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:09.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:09 smithi090 ceph-mon[29180]: pgmap v13060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:11.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:11 smithi119 ceph-mon[32752]: pgmap v13061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T01:04:11.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23332.341687 secs 2024-04-27T01:04:11.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:11 smithi167 ceph-mon[32751]: pgmap v13061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T01:04:11.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23332.341687 secs 2024-04-27T01:04:11.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:11 smithi090 ceph-mon[29180]: pgmap v13061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T01:04:11.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23332.341687 secs 2024-04-27T01:04:12.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23333.145590 secs 2024-04-27T01:04:12.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23333.145590 secs 2024-04-27T01:04:12.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23333.145590 secs 2024-04-27T01:04:13.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:13 smithi119 ceph-mon[32752]: pgmap v13062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T01:04:13.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:13 smithi167 ceph-mon[32751]: pgmap v13062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T01:04:13.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:13 smithi090 ceph-mon[29180]: pgmap v13062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T01:04:14.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:04:14.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:04:14.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:04:15.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:15 smithi119 ceph-mon[32752]: pgmap v13063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T01:04:15.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:04:15.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:04:15.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:15 smithi119 ceph-mon[32752]: pgmap v13064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T01:04:15.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:15 smithi119 ceph-mon[32752]: pgmap v13065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T01:04:15.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:04:15.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:15 smithi167 ceph-mon[32751]: pgmap v13063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T01:04:15.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:04:15.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:04:15.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:15 smithi167 ceph-mon[32751]: pgmap v13064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T01:04:15.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:15 smithi167 ceph-mon[32751]: pgmap v13065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T01:04:15.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:04:15.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:15 smithi090 ceph-mon[29180]: pgmap v13063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T01:04:15.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:04:15.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:04:15.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:15 smithi090 ceph-mon[29180]: pgmap v13064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T01:04:15.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:15 smithi090 ceph-mon[29180]: pgmap v13065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T01:04:15.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:04:16.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23337.341769 secs 2024-04-27T01:04:16.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23337.341769 secs 2024-04-27T01:04:16.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23337.341769 secs 2024-04-27T01:04:17.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23338.145730 secs 2024-04-27T01:04:17.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:17 smithi119 ceph-mon[32752]: pgmap v13066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T01:04:17.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23338.145730 secs 2024-04-27T01:04:17.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:17 smithi167 ceph-mon[32751]: pgmap v13066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T01:04:17.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23338.145730 secs 2024-04-27T01:04:17.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:17 smithi090 ceph-mon[29180]: pgmap v13066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T01:04:19.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:19 smithi119 ceph-mon[32752]: pgmap v13067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:04:19.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:19 smithi167 ceph-mon[32751]: pgmap v13067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:04:19.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:19 smithi090 ceph-mon[29180]: pgmap v13067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:04:20.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23342.341865 secs 2024-04-27T01:04:20.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23342.341865 secs 2024-04-27T01:04:20.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23342.341865 secs 2024-04-27T01:04:21.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:21 smithi119 ceph-mon[32752]: pgmap v13068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:04:21.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:21 smithi167 ceph-mon[32751]: pgmap v13068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:04:21.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:21 smithi090 ceph-mon[29180]: pgmap v13068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:04:22.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23343.145838 secs 2024-04-27T01:04:22.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23343.145838 secs 2024-04-27T01:04:22.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23343.145838 secs 2024-04-27T01:04:23.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:23 smithi119 ceph-mon[32752]: pgmap v13069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:04:23.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:23 smithi167 ceph-mon[32751]: pgmap v13069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:04:23.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:23 smithi090 ceph-mon[29180]: pgmap v13069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:04:25.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:25 smithi119 ceph-mon[32752]: pgmap v13070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:04:25.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23347.341966 secs 2024-04-27T01:04:25.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:25 smithi167 ceph-mon[32751]: pgmap v13070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:04:25.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23347.341966 secs 2024-04-27T01:04:25.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:25 smithi090 ceph-mon[29180]: pgmap v13070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:04:25.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23347.341966 secs 2024-04-27T01:04:27.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23348.145982 secs 2024-04-27T01:04:27.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:27 smithi119 ceph-mon[32752]: pgmap v13071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:27.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23348.145982 secs 2024-04-27T01:04:27.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:27 smithi167 ceph-mon[32751]: pgmap v13071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:27.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23348.145982 secs 2024-04-27T01:04:27.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:27 smithi090 ceph-mon[29180]: pgmap v13071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:29.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:29 smithi119 ceph-mon[32752]: pgmap v13072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:29.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:29 smithi167 ceph-mon[32751]: pgmap v13072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:29.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:29 smithi090 ceph-mon[29180]: pgmap v13072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:30.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23352.342051 secs 2024-04-27T01:04:30.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23352.342051 secs 2024-04-27T01:04:30.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23352.342051 secs 2024-04-27T01:04:31.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:31 smithi119 ceph-mon[32752]: pgmap v13073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:31.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:31 smithi167 ceph-mon[32751]: pgmap v13073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:31.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:31 smithi090 ceph-mon[29180]: pgmap v13073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:32.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23353.146101 secs 2024-04-27T01:04:32.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23353.146101 secs 2024-04-27T01:04:32.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23353.146101 secs 2024-04-27T01:04:33.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:33 smithi090 ceph-mon[29180]: pgmap v13074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:33.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:33 smithi119 ceph-mon[32752]: pgmap v13074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:33.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:33 smithi167 ceph-mon[32751]: pgmap v13074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:35 smithi119 ceph-mon[32752]: pgmap v13075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23357.342134 secs 2024-04-27T01:04:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:35 smithi167 ceph-mon[32751]: pgmap v13075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23357.342134 secs 2024-04-27T01:04:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:35 smithi090 ceph-mon[29180]: pgmap v13075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23357.342134 secs 2024-04-27T01:04:37.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23358.146228 secs 2024-04-27T01:04:37.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:37 smithi119 ceph-mon[32752]: pgmap v13076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:37.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23358.146228 secs 2024-04-27T01:04:37.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:37 smithi167 ceph-mon[32751]: pgmap v13076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:37.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23358.146228 secs 2024-04-27T01:04:37.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:37 smithi090 ceph-mon[29180]: pgmap v13076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:39.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:39 smithi119 ceph-mon[32752]: pgmap v13077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:39.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:39 smithi167 ceph-mon[32751]: pgmap v13077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:39.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:39 smithi090 ceph-mon[29180]: pgmap v13077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:40.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23362.342196 secs 2024-04-27T01:04:40.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23362.342196 secs 2024-04-27T01:04:40.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23362.342196 secs 2024-04-27T01:04:41.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:41 smithi119 ceph-mon[32752]: pgmap v13078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:41.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:41 smithi167 ceph-mon[32751]: pgmap v13078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:41.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:41 smithi090 ceph-mon[29180]: pgmap v13078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:42.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23363.146350 secs 2024-04-27T01:04:42.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23363.146350 secs 2024-04-27T01:04:42.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23363.146350 secs 2024-04-27T01:04:43.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:43 smithi119 ceph-mon[32752]: pgmap v13079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:43.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:43 smithi167 ceph-mon[32751]: pgmap v13079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:43.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:43 smithi090 ceph-mon[29180]: pgmap v13079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:45.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:45 smithi119 ceph-mon[32752]: pgmap v13080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:45.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23367.342280 secs 2024-04-27T01:04:45.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:45 smithi167 ceph-mon[32751]: pgmap v13080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:45.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23367.342280 secs 2024-04-27T01:04:45.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:45 smithi090 ceph-mon[29180]: pgmap v13080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:45.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23367.342280 secs 2024-04-27T01:04:47.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23368.146471 secs 2024-04-27T01:04:47.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:47 smithi119 ceph-mon[32752]: pgmap v13081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:47.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23368.146471 secs 2024-04-27T01:04:47.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:47 smithi167 ceph-mon[32751]: pgmap v13081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:47.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23368.146471 secs 2024-04-27T01:04:47.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:47 smithi090 ceph-mon[29180]: pgmap v13081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:49.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:49 smithi119 ceph-mon[32752]: pgmap v13082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:49.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:49 smithi167 ceph-mon[32751]: pgmap v13082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:49.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:49 smithi090 ceph-mon[29180]: pgmap v13082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:50.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23372.342364 secs 2024-04-27T01:04:50.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23372.342364 secs 2024-04-27T01:04:50.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23372.342364 secs 2024-04-27T01:04:51.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:51 smithi119 ceph-mon[32752]: pgmap v13083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:51.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:51 smithi167 ceph-mon[32751]: pgmap v13083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:51.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:51 smithi090 ceph-mon[29180]: pgmap v13083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:52.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23373.146596 secs 2024-04-27T01:04:52.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23373.146596 secs 2024-04-27T01:04:52.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23373.146596 secs 2024-04-27T01:04:53.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:53 smithi167 ceph-mon[32751]: pgmap v13084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:53.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:53 smithi090 ceph-mon[29180]: pgmap v13084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:54.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:53 smithi119 ceph-mon[32752]: pgmap v13084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:55.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:55 smithi167 ceph-mon[32751]: pgmap v13085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:55.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23377.342451 secs 2024-04-27T01:04:55.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:55 smithi090 ceph-mon[29180]: pgmap v13085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:55.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23377.342451 secs 2024-04-27T01:04:56.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:55 smithi119 ceph-mon[32752]: pgmap v13085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:56.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23377.342451 secs 2024-04-27T01:04:57.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23378.146751 secs 2024-04-27T01:04:57.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:57 smithi167 ceph-mon[32751]: pgmap v13086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:57.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23378.146751 secs 2024-04-27T01:04:57.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:57 smithi090 ceph-mon[29180]: pgmap v13086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:58.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23378.146751 secs 2024-04-27T01:04:58.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:57 smithi119 ceph-mon[32752]: pgmap v13086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:04:59.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:04:59 smithi167 ceph-mon[32751]: pgmap v13087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:04:59.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:04:59 smithi090 ceph-mon[29180]: pgmap v13087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:00.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:04:59 smithi119 ceph-mon[32752]: pgmap v13087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:00.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23382.342534 secs 2024-04-27T01:05:01.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23382.342534 secs 2024-04-27T01:05:01.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23382.342534 secs 2024-04-27T01:05:01.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:01 smithi090 ceph-mon[29180]: pgmap v13088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:02.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:01 smithi119 ceph-mon[32752]: pgmap v13088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:02.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:01 smithi167 ceph-mon[32751]: pgmap v13088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:02.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23383.146813 secs 2024-04-27T01:05:03.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23383.146813 secs 2024-04-27T01:05:03.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23383.146813 secs 2024-04-27T01:05:03.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:03 smithi090 ceph-mon[29180]: pgmap v13089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:04.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:03 smithi119 ceph-mon[32752]: pgmap v13089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:04.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:03 smithi167 ceph-mon[32751]: pgmap v13089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:05.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:05 smithi090 ceph-mon[29180]: pgmap v13090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:05.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23387.342624 secs 2024-04-27T01:05:06.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:05 smithi119 ceph-mon[32752]: pgmap v13090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:06.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23387.342624 secs 2024-04-27T01:05:06.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:05 smithi167 ceph-mon[32751]: pgmap v13090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:06.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23387.342624 secs 2024-04-27T01:05:07.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:07 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 23388.146950 secs 2024-04-27T01:05:07.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:07 smithi119 ceph-mon[32752]: slow request 5760.287062 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:07.849 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:07 smithi119 ceph-mon[32752]: pgmap v13091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:07.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:07 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 23388.146950 secs 2024-04-27T01:05:07.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:07 smithi090 ceph-mon[29180]: slow request 5760.287062 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:07.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:07 smithi090 ceph-mon[29180]: pgmap v13091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:08.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:07 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 23388.146950 secs 2024-04-27T01:05:08.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:07 smithi167 ceph-mon[32751]: slow request 5760.287062 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:08.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:07 smithi167 ceph-mon[32751]: pgmap v13091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:09.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:09 smithi090 ceph-mon[29180]: pgmap v13092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:10.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:09 smithi119 ceph-mon[32752]: pgmap v13092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:10.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:09 smithi167 ceph-mon[32751]: pgmap v13092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:10.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23392.342706 secs 2024-04-27T01:05:11.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23392.342706 secs 2024-04-27T01:05:11.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23392.342706 secs 2024-04-27T01:05:11.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:11 smithi090 ceph-mon[29180]: pgmap v13093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:12.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:11 smithi119 ceph-mon[32752]: pgmap v13093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:12.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:11 smithi167 ceph-mon[32751]: pgmap v13093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:12.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23393.147097 secs 2024-04-27T01:05:13.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23393.147097 secs 2024-04-27T01:05:13.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23393.147097 secs 2024-04-27T01:05:13.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:13 smithi090 ceph-mon[29180]: pgmap v13094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:14.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:13 smithi119 ceph-mon[32752]: pgmap v13094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:14.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:13 smithi167 ceph-mon[32751]: pgmap v13094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:05:14.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:05:15.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:05:15.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:05:15.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:15 smithi090 ceph-mon[29180]: pgmap v13095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:15.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23397.342815 secs 2024-04-27T01:05:15.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:05:15.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:05:15.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:05:16.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:15 smithi119 ceph-mon[32752]: pgmap v13095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:16.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23397.342815 secs 2024-04-27T01:05:16.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:05:16.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:05:16.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:05:16.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:15 smithi167 ceph-mon[32751]: pgmap v13095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:05:16.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23397.342815 secs 2024-04-27T01:05:16.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:05:16.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:05:16.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:05:16.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:16 smithi090 ceph-mon[29180]: pgmap v13096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:05:16.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:16 smithi090 ceph-mon[29180]: pgmap v13097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:17.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:16 smithi119 ceph-mon[32752]: pgmap v13096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:05:17.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:16 smithi119 ceph-mon[32752]: pgmap v13097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:17.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:16 smithi167 ceph-mon[32751]: pgmap v13096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:05:17.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:16 smithi167 ceph-mon[32751]: pgmap v13097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:17.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23398.147221 secs 2024-04-27T01:05:18.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23398.147221 secs 2024-04-27T01:05:18.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23398.147221 secs 2024-04-27T01:05:18.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:18 smithi090 ceph-mon[29180]: pgmap v13098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:05:19.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:18 smithi119 ceph-mon[32752]: pgmap v13098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:05:19.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:18 smithi167 ceph-mon[32751]: pgmap v13098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:05:20.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:20 smithi090 ceph-mon[29180]: pgmap v13099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:20.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23402.342878 secs 2024-04-27T01:05:21.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:20 smithi119 ceph-mon[32752]: pgmap v13099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:21.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23402.342878 secs 2024-04-27T01:05:21.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:20 smithi167 ceph-mon[32751]: pgmap v13099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:21.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23402.342878 secs 2024-04-27T01:05:23.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23403.147301 secs 2024-04-27T01:05:23.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:22 smithi119 ceph-mon[32752]: pgmap v13100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:23.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23403.147301 secs 2024-04-27T01:05:23.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:22 smithi167 ceph-mon[32751]: pgmap v13100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:23.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23403.147301 secs 2024-04-27T01:05:23.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:22 smithi090 ceph-mon[29180]: pgmap v13100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:05:25.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:24 smithi119 ceph-mon[32752]: pgmap v13101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:05:25.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:24 smithi167 ceph-mon[32751]: pgmap v13101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:05:25.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:24 smithi090 ceph-mon[29180]: pgmap v13101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:05:26.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23407.342937 secs 2024-04-27T01:05:26.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23407.342937 secs 2024-04-27T01:05:26.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23407.342937 secs 2024-04-27T01:05:27.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:26 smithi119 ceph-mon[32752]: pgmap v13102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:05:27.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:26 smithi167 ceph-mon[32751]: pgmap v13102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:05:27.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:26 smithi090 ceph-mon[29180]: pgmap v13102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:05:27.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23408.147427 secs 2024-04-27T01:05:27.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:27 smithi167 ceph-mon[32751]: pgmap v13103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:28.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23408.147427 secs 2024-04-27T01:05:28.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:27 smithi119 ceph-mon[32752]: pgmap v13103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:28.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23408.147427 secs 2024-04-27T01:05:28.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:27 smithi090 ceph-mon[29180]: pgmap v13103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:30.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:30 smithi119 ceph-mon[32752]: pgmap v13104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:30.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:30 smithi167 ceph-mon[32751]: pgmap v13104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:30.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:30 smithi090 ceph-mon[29180]: pgmap v13104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:31.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23412.342994 secs 2024-04-27T01:05:31.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23412.342994 secs 2024-04-27T01:05:31.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23412.342994 secs 2024-04-27T01:05:32.520 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:32 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 23413.147536 secs 2024-04-27T01:05:32.520 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:32 smithi090 ceph-mon[29180]: slow request 23040.016549 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:32.520 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:32 smithi090 ceph-mon[29180]: pgmap v13105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:32.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:32 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 23413.147536 secs 2024-04-27T01:05:32.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:32 smithi119 ceph-mon[32752]: slow request 23040.016549 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:32.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:32 smithi119 ceph-mon[32752]: pgmap v13105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:32.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:32 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 23413.147536 secs 2024-04-27T01:05:32.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:32 smithi167 ceph-mon[32751]: slow request 23040.016549 seconds old, received at 2024-04-26T18:41:30.881632+0000: client_request(client.17049:1 getattr pAsLsXsFs #0x1 2024-04-26T18:41:30.877933+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:32.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:32 smithi167 ceph-mon[32751]: pgmap v13105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:34.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:34 smithi119 ceph-mon[32752]: pgmap v13106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:34.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:34 smithi167 ceph-mon[32751]: pgmap v13106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:34.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:34 smithi090 ceph-mon[29180]: pgmap v13106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:36.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23417.343053 secs 2024-04-27T01:05:36.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:36 smithi119 ceph-mon[32752]: pgmap v13107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:36.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23417.343053 secs 2024-04-27T01:05:36.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:36 smithi167 ceph-mon[32751]: pgmap v13107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:36.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23417.343053 secs 2024-04-27T01:05:36.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:36 smithi090 ceph-mon[29180]: pgmap v13107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:37.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:37 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 23418.147633 secs 2024-04-27T01:05:37.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:37 smithi119 ceph-mon[32752]: slow request 11521.887766 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:37.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:37 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 23418.147633 secs 2024-04-27T01:05:37.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:37 smithi167 ceph-mon[32751]: slow request 11521.887766 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:37.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:37 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 23418.147633 secs 2024-04-27T01:05:37.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:37 smithi090 ceph-mon[29180]: slow request 11521.887766 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:38.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:38 smithi119 ceph-mon[32752]: pgmap v13108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:38.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:38 smithi167 ceph-mon[32751]: pgmap v13108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:38.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:38 smithi090 ceph-mon[29180]: pgmap v13108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:40.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:40 smithi119 ceph-mon[32752]: pgmap v13109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:40.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:40 smithi167 ceph-mon[32751]: pgmap v13109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:40.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:40 smithi090 ceph-mon[29180]: pgmap v13109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:41.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23422.343111 secs 2024-04-27T01:05:41.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23422.343111 secs 2024-04-27T01:05:41.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23422.343111 secs 2024-04-27T01:05:42.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:42 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 23423.147750 secs 2024-04-27T01:05:42.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:42 smithi119 ceph-mon[32752]: slow request 723.843037 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:42.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:42 smithi119 ceph-mon[32752]: pgmap v13110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:42.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:42 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 23423.147750 secs 2024-04-27T01:05:42.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:42 smithi167 ceph-mon[32751]: slow request 723.843037 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:42.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:42 smithi167 ceph-mon[32751]: pgmap v13110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:42.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:42 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 23423.147750 secs 2024-04-27T01:05:42.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:42 smithi090 ceph-mon[29180]: slow request 723.843037 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:05:42.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:42 smithi090 ceph-mon[29180]: pgmap v13110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:44.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:44 smithi119 ceph-mon[32752]: pgmap v13111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:44.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:44 smithi167 ceph-mon[32751]: pgmap v13111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:44.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:44 smithi090 ceph-mon[29180]: pgmap v13111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:46.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23427.343156 secs 2024-04-27T01:05:46.547 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:46 smithi119 ceph-mon[32752]: pgmap v13112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:46.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23427.343156 secs 2024-04-27T01:05:46.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:46 smithi167 ceph-mon[32751]: pgmap v13112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:46.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23427.343156 secs 2024-04-27T01:05:46.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:46 smithi090 ceph-mon[29180]: pgmap v13112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:47.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23428.147907 secs 2024-04-27T01:05:47.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23428.147907 secs 2024-04-27T01:05:47.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23428.147907 secs 2024-04-27T01:05:48.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:48 smithi119 ceph-mon[32752]: pgmap v13113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:48.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:48 smithi167 ceph-mon[32751]: pgmap v13113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:48.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:48 smithi090 ceph-mon[29180]: pgmap v13113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:50.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:50 smithi119 ceph-mon[32752]: pgmap v13114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:50.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:50 smithi167 ceph-mon[32751]: pgmap v13114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:50.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:50 smithi090 ceph-mon[29180]: pgmap v13114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:51.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23432.343245 secs 2024-04-27T01:05:51.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23432.343245 secs 2024-04-27T01:05:51.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23432.343245 secs 2024-04-27T01:05:52.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23433.148028 secs 2024-04-27T01:05:52.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:52 smithi167 ceph-mon[32751]: pgmap v13115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:52.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23433.148028 secs 2024-04-27T01:05:52.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:52 smithi090 ceph-mon[29180]: pgmap v13115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:52.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23433.148028 secs 2024-04-27T01:05:52.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:52 smithi119 ceph-mon[32752]: pgmap v13115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:54.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:53 smithi119 ceph-mon[32752]: pgmap v13116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:54.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:53 smithi167 ceph-mon[32751]: pgmap v13116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:54.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:53 smithi090 ceph-mon[29180]: pgmap v13116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:56.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23437.343330 secs 2024-04-27T01:05:56.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:56 smithi119 ceph-mon[32752]: pgmap v13117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:56.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23437.343330 secs 2024-04-27T01:05:56.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:56 smithi167 ceph-mon[32751]: pgmap v13117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:56.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23437.343330 secs 2024-04-27T01:05:56.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:56 smithi090 ceph-mon[29180]: pgmap v13117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:05:57.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23438.148156 secs 2024-04-27T01:05:57.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23438.148156 secs 2024-04-27T01:05:57.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23438.148156 secs 2024-04-27T01:05:58.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:05:58 smithi119 ceph-mon[32752]: pgmap v13118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:58.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:05:58 smithi167 ceph-mon[32751]: pgmap v13118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:05:58.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:05:58 smithi090 ceph-mon[29180]: pgmap v13118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:00.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:00 smithi119 ceph-mon[32752]: pgmap v13119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:00.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:00 smithi167 ceph-mon[32751]: pgmap v13119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:00.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:00 smithi090 ceph-mon[29180]: pgmap v13119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:01.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23442.343419 secs 2024-04-27T01:06:01.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23442.343419 secs 2024-04-27T01:06:01.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23442.343419 secs 2024-04-27T01:06:02.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23443.148257 secs 2024-04-27T01:06:02.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:02 smithi119 ceph-mon[32752]: pgmap v13120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:02.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23443.148257 secs 2024-04-27T01:06:02.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:02 smithi167 ceph-mon[32751]: pgmap v13120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:02.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23443.148257 secs 2024-04-27T01:06:02.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:02 smithi090 ceph-mon[29180]: pgmap v13120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:04.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:04 smithi119 ceph-mon[32752]: pgmap v13121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:04.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:04 smithi167 ceph-mon[32751]: pgmap v13121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:04.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:04 smithi090 ceph-mon[29180]: pgmap v13121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:06.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23447.343524 secs 2024-04-27T01:06:06.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:06 smithi119 ceph-mon[32752]: pgmap v13122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:06.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23447.343524 secs 2024-04-27T01:06:06.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:06 smithi167 ceph-mon[32751]: pgmap v13122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:06.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23447.343524 secs 2024-04-27T01:06:06.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:06 smithi090 ceph-mon[29180]: pgmap v13122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:07.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23448.148396 secs 2024-04-27T01:06:07.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23448.148396 secs 2024-04-27T01:06:07.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23448.148396 secs 2024-04-27T01:06:08.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:08 smithi119 ceph-mon[32752]: pgmap v13123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:08.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:08 smithi167 ceph-mon[32751]: pgmap v13123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:08.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:08 smithi090 ceph-mon[29180]: pgmap v13123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:10.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:10 smithi119 ceph-mon[32752]: pgmap v13124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:10.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:10 smithi167 ceph-mon[32751]: pgmap v13124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:10.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:10 smithi090 ceph-mon[29180]: pgmap v13124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:11.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23452.343630 secs 2024-04-27T01:06:11.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23452.343630 secs 2024-04-27T01:06:11.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23452.343630 secs 2024-04-27T01:06:12.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23453.148502 secs 2024-04-27T01:06:12.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:12 smithi119 ceph-mon[32752]: pgmap v13125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:12.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23453.148502 secs 2024-04-27T01:06:12.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:12 smithi167 ceph-mon[32751]: pgmap v13125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:12.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23453.148502 secs 2024-04-27T01:06:12.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:12 smithi090 ceph-mon[29180]: pgmap v13125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:14.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:14 smithi167 ceph-mon[32751]: pgmap v13126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:14.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:14 smithi090 ceph-mon[29180]: pgmap v13126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:14.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:14 smithi119 ceph-mon[32752]: pgmap v13126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:15.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:06:15.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:06:15.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:06:16.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23457.343739 secs 2024-04-27T01:06:16.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:16 smithi167 ceph-mon[32751]: pgmap v13127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:16.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:06:16.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:06:16.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:06:16.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23457.343739 secs 2024-04-27T01:06:16.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:16 smithi090 ceph-mon[29180]: pgmap v13127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:16.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:06:16.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:06:16.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:06:16.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23457.343739 secs 2024-04-27T01:06:16.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:16 smithi119 ceph-mon[32752]: pgmap v13127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:16.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:06:16.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:06:16.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:06:17.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23458.148568 secs 2024-04-27T01:06:17.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:17 smithi090 ceph-mon[29180]: pgmap v13128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:06:17.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:17 smithi090 ceph-mon[29180]: pgmap v13129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:17.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23458.148568 secs 2024-04-27T01:06:17.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:17 smithi119 ceph-mon[32752]: pgmap v13128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:06:17.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:17 smithi119 ceph-mon[32752]: pgmap v13129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:17.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23458.148568 secs 2024-04-27T01:06:17.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:17 smithi167 ceph-mon[32751]: pgmap v13128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:06:17.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:17 smithi167 ceph-mon[32751]: pgmap v13129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:19.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:19 smithi090 ceph-mon[29180]: pgmap v13130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:06:19.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:19 smithi119 ceph-mon[32752]: pgmap v13130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:06:19.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:19 smithi167 ceph-mon[32751]: pgmap v13130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:06:21.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:21 smithi090 ceph-mon[29180]: pgmap v13131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:21.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23462.343849 secs 2024-04-27T01:06:21.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:21 smithi119 ceph-mon[32752]: pgmap v13131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:21.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23462.343849 secs 2024-04-27T01:06:21.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:21 smithi167 ceph-mon[32751]: pgmap v13131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:21.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23462.343849 secs 2024-04-27T01:06:22.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23463.148687 secs 2024-04-27T01:06:22.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23463.148687 secs 2024-04-27T01:06:22.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23463.148687 secs 2024-04-27T01:06:23.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:23 smithi090 ceph-mon[29180]: pgmap v13132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:23.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:23 smithi119 ceph-mon[32752]: pgmap v13132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:23.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:23 smithi167 ceph-mon[32751]: pgmap v13132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:25.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:25 smithi090 ceph-mon[29180]: pgmap v13133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:25.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:25 smithi119 ceph-mon[32752]: pgmap v13133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:25.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:25 smithi167 ceph-mon[32751]: pgmap v13133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:06:26.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23467.343935 secs 2024-04-27T01:06:26.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23467.343935 secs 2024-04-27T01:06:26.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23467.343935 secs 2024-04-27T01:06:27.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23468.148765 secs 2024-04-27T01:06:27.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:27 smithi090 ceph-mon[29180]: pgmap v13134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:06:27.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23468.148765 secs 2024-04-27T01:06:27.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:27 smithi167 ceph-mon[32751]: pgmap v13134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:06:27.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23468.148765 secs 2024-04-27T01:06:27.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:27 smithi119 ceph-mon[32752]: pgmap v13134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:06:29.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:29 smithi090 ceph-mon[29180]: pgmap v13135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:29.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:29 smithi119 ceph-mon[32752]: pgmap v13135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:29.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:29 smithi167 ceph-mon[32751]: pgmap v13135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:31.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:31 smithi090 ceph-mon[29180]: pgmap v13136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:31.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23472.344022 secs 2024-04-27T01:06:31.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:31 smithi119 ceph-mon[32752]: pgmap v13136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:31.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23472.344022 secs 2024-04-27T01:06:31.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:31 smithi167 ceph-mon[32751]: pgmap v13136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:31.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23472.344022 secs 2024-04-27T01:06:32.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23473.148878 secs 2024-04-27T01:06:32.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23473.148878 secs 2024-04-27T01:06:32.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23473.148878 secs 2024-04-27T01:06:33.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:33 smithi090 ceph-mon[29180]: pgmap v13137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:33.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:33 smithi119 ceph-mon[32752]: pgmap v13137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:33.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:33 smithi167 ceph-mon[32751]: pgmap v13137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:35.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:35 smithi090 ceph-mon[29180]: pgmap v13138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:35.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:35 smithi119 ceph-mon[32752]: pgmap v13138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:35.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:35 smithi167 ceph-mon[32751]: pgmap v13138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:36.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23477.344079 secs 2024-04-27T01:06:36.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23477.344079 secs 2024-04-27T01:06:36.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23477.344079 secs 2024-04-27T01:06:37.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23478.148998 secs 2024-04-27T01:06:37.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:37 smithi090 ceph-mon[29180]: pgmap v13139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:37.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23478.148998 secs 2024-04-27T01:06:37.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:37 smithi119 ceph-mon[32752]: pgmap v13139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:37.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23478.148998 secs 2024-04-27T01:06:37.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:37 smithi167 ceph-mon[32751]: pgmap v13139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:39.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:39 smithi090 ceph-mon[29180]: pgmap v13140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:39.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:39 smithi119 ceph-mon[32752]: pgmap v13140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:39.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:39 smithi167 ceph-mon[32751]: pgmap v13140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:41.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:41 smithi090 ceph-mon[29180]: pgmap v13141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:41.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23482.344145 secs 2024-04-27T01:06:41.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:41 smithi119 ceph-mon[32752]: pgmap v13141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:41.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23482.344145 secs 2024-04-27T01:06:41.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:41 smithi167 ceph-mon[32751]: pgmap v13141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:41.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23482.344145 secs 2024-04-27T01:06:42.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23483.149148 secs 2024-04-27T01:06:42.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23483.149148 secs 2024-04-27T01:06:42.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23483.149148 secs 2024-04-27T01:06:43.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:43 smithi119 ceph-mon[32752]: pgmap v13142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:43.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:43 smithi167 ceph-mon[32751]: pgmap v13142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:43.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:43 smithi090 ceph-mon[29180]: pgmap v13142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:45.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:45 smithi119 ceph-mon[32752]: pgmap v13143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:45.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:45 smithi167 ceph-mon[32751]: pgmap v13143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:45.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:45 smithi090 ceph-mon[29180]: pgmap v13143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:46.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23487.344236 secs 2024-04-27T01:06:46.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23487.344236 secs 2024-04-27T01:06:46.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23487.344236 secs 2024-04-27T01:06:47.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23488.149270 secs 2024-04-27T01:06:47.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:47 smithi119 ceph-mon[32752]: pgmap v13144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:47.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23488.149270 secs 2024-04-27T01:06:47.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:47 smithi167 ceph-mon[32751]: pgmap v13144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:47.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23488.149270 secs 2024-04-27T01:06:47.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:47 smithi090 ceph-mon[29180]: pgmap v13144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:49.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:49 smithi119 ceph-mon[32752]: pgmap v13145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:49.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:49 smithi167 ceph-mon[32751]: pgmap v13145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:49.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:49 smithi090 ceph-mon[29180]: pgmap v13145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:51.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:51 smithi119 ceph-mon[32752]: pgmap v13146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:51.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23492.344324 secs 2024-04-27T01:06:51.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:51 smithi167 ceph-mon[32751]: pgmap v13146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:51.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23492.344324 secs 2024-04-27T01:06:51.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:51 smithi090 ceph-mon[29180]: pgmap v13146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:51.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23492.344324 secs 2024-04-27T01:06:52.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23493.149426 secs 2024-04-27T01:06:52.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23493.149426 secs 2024-04-27T01:06:52.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23493.149426 secs 2024-04-27T01:06:53.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:53 smithi119 ceph-mon[32752]: pgmap v13147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:53.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:53 smithi167 ceph-mon[32751]: pgmap v13147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:53.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:53 smithi090 ceph-mon[29180]: pgmap v13147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:55.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:55 smithi119 ceph-mon[32752]: pgmap v13148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:55.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:55 smithi167 ceph-mon[32751]: pgmap v13148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:55.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:55 smithi090 ceph-mon[29180]: pgmap v13148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:06:56.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23497.344424 secs 2024-04-27T01:06:56.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23497.344424 secs 2024-04-27T01:06:56.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23497.344424 secs 2024-04-27T01:06:57.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23498.149530 secs 2024-04-27T01:06:57.711 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:57 smithi167 ceph-mon[32751]: pgmap v13149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:57.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23498.149530 secs 2024-04-27T01:06:57.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:57 smithi119 ceph-mon[32752]: pgmap v13149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:57.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23498.149530 secs 2024-04-27T01:06:57.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:57 smithi090 ceph-mon[29180]: pgmap v13149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:59.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:06:59 smithi119 ceph-mon[32752]: pgmap v13150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:59.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:06:59 smithi167 ceph-mon[32751]: pgmap v13150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:06:59.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:06:59 smithi090 ceph-mon[29180]: pgmap v13150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:01.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:01 smithi119 ceph-mon[32752]: pgmap v13151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:01.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23502.344508 secs 2024-04-27T01:07:01.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:01 smithi167 ceph-mon[32751]: pgmap v13151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:01.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23502.344508 secs 2024-04-27T01:07:01.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:01 smithi090 ceph-mon[29180]: pgmap v13151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:01.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23502.344508 secs 2024-04-27T01:07:02.768 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23503.149591 secs 2024-04-27T01:07:02.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23503.149591 secs 2024-04-27T01:07:02.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23503.149591 secs 2024-04-27T01:07:03.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:03 smithi119 ceph-mon[32752]: pgmap v13152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:03.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:03 smithi167 ceph-mon[32751]: pgmap v13152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:03.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:03 smithi090 ceph-mon[29180]: pgmap v13152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:05.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:05 smithi119 ceph-mon[32752]: pgmap v13153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:05.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:05 smithi167 ceph-mon[32751]: pgmap v13153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:05.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:05 smithi090 ceph-mon[29180]: pgmap v13153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:06.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23507.344594 secs 2024-04-27T01:07:06.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23507.344594 secs 2024-04-27T01:07:06.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23507.344594 secs 2024-04-27T01:07:07.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23508.149694 secs 2024-04-27T01:07:07.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:07 smithi119 ceph-mon[32752]: pgmap v13154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:07.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23508.149694 secs 2024-04-27T01:07:07.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:07 smithi167 ceph-mon[32751]: pgmap v13154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:07.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23508.149694 secs 2024-04-27T01:07:07.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:07 smithi090 ceph-mon[29180]: pgmap v13154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:09.670 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:09 smithi090 ceph-mon[29180]: pgmap v13155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:09.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:09 smithi119 ceph-mon[32752]: pgmap v13155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:09.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:09 smithi167 ceph-mon[32751]: pgmap v13155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:11.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:11 smithi119 ceph-mon[32752]: pgmap v13156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:11.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23512.344675 secs 2024-04-27T01:07:11.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:11 smithi167 ceph-mon[32751]: pgmap v13156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:11.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23512.344675 secs 2024-04-27T01:07:11.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:11 smithi090 ceph-mon[29180]: pgmap v13156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:11.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23512.344675 secs 2024-04-27T01:07:12.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:12 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 23513.149817 secs 2024-04-27T01:07:12.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:12 smithi119 ceph-mon[32752]: slow request 1444.030950 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:07:12.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:12 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 23513.149817 secs 2024-04-27T01:07:12.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:12 smithi167 ceph-mon[32751]: slow request 1444.030950 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:07:12.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:12 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 23513.149817 secs 2024-04-27T01:07:12.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:12 smithi090 ceph-mon[29180]: slow request 1444.030950 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:07:13.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:13 smithi119 ceph-mon[32752]: pgmap v13157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:13.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:13 smithi167 ceph-mon[32751]: pgmap v13157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:13.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:13 smithi090 ceph-mon[29180]: pgmap v13157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:15.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:15 smithi119 ceph-mon[32752]: pgmap v13158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:15.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:15 smithi167 ceph-mon[32751]: pgmap v13158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:15.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:15 smithi090 ceph-mon[29180]: pgmap v13158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:16.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23517.344761 secs 2024-04-27T01:07:16.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:07:16.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23517.344761 secs 2024-04-27T01:07:16.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:07:16.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23517.344761 secs 2024-04-27T01:07:16.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:07:17.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23518.149962 secs 2024-04-27T01:07:17.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:17 smithi119 ceph-mon[32752]: pgmap v13159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:17.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:07:17.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:07:17.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:07:17.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23518.149962 secs 2024-04-27T01:07:17.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:17 smithi167 ceph-mon[32751]: pgmap v13159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:17.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:07:17.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:07:17.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:07:17.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23518.149962 secs 2024-04-27T01:07:17.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:17 smithi090 ceph-mon[29180]: pgmap v13159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:17.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:07:17.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:07:17.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:07:18.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:18 smithi119 ceph-mon[32752]: pgmap v13160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:07:18.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:18 smithi119 ceph-mon[32752]: pgmap v13161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:18.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:18 smithi167 ceph-mon[32751]: pgmap v13160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:07:18.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:18 smithi167 ceph-mon[32751]: pgmap v13161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:18.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:18 smithi090 ceph-mon[29180]: pgmap v13160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:07:18.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:18 smithi090 ceph-mon[29180]: pgmap v13161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:20.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:20 smithi119 ceph-mon[32752]: pgmap v13162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:20.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:20 smithi167 ceph-mon[32751]: pgmap v13162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:20.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:20 smithi090 ceph-mon[29180]: pgmap v13162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:21.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23522.344852 secs 2024-04-27T01:07:21.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23522.344852 secs 2024-04-27T01:07:21.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23522.344852 secs 2024-04-27T01:07:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:22 smithi119 ceph-mon[32752]: pgmap v13163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23523.150100 secs 2024-04-27T01:07:22.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:22 smithi167 ceph-mon[32751]: pgmap v13163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:22.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23523.150100 secs 2024-04-27T01:07:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:22 smithi090 ceph-mon[29180]: pgmap v13163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23523.150100 secs 2024-04-27T01:07:24.765 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:24 smithi090 ceph-mon[29180]: pgmap v13164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:07:24.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:24 smithi119 ceph-mon[32752]: pgmap v13164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:07:24.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:24 smithi167 ceph-mon[32751]: pgmap v13164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:07:26.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:26 smithi119 ceph-mon[32752]: pgmap v13165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:26.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23527.344942 secs 2024-04-27T01:07:26.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:26 smithi167 ceph-mon[32751]: pgmap v13165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:26.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23527.344942 secs 2024-04-27T01:07:26.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:26 smithi090 ceph-mon[29180]: pgmap v13165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:07:26.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23527.344942 secs 2024-04-27T01:07:27.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23528.150248 secs 2024-04-27T01:07:27.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23528.150248 secs 2024-04-27T01:07:27.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23528.150248 secs 2024-04-27T01:07:28.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:28 smithi119 ceph-mon[32752]: pgmap v13166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:07:28.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:28 smithi167 ceph-mon[32751]: pgmap v13166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:07:28.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:28 smithi090 ceph-mon[29180]: pgmap v13166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:07:30.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:30 smithi119 ceph-mon[32752]: pgmap v13167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:30.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23532.345029 secs 2024-04-27T01:07:30.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:30 smithi167 ceph-mon[32751]: pgmap v13167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:30.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23532.345029 secs 2024-04-27T01:07:30.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:30 smithi090 ceph-mon[29180]: pgmap v13167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:30.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23532.345029 secs 2024-04-27T01:07:32.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:32 smithi119 ceph-mon[32752]: pgmap v13168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:32.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23533.150349 secs 2024-04-27T01:07:32.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:32 smithi167 ceph-mon[32751]: pgmap v13168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:32.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23533.150349 secs 2024-04-27T01:07:32.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:32 smithi090 ceph-mon[29180]: pgmap v13168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:32.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23533.150349 secs 2024-04-27T01:07:34.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:34 smithi119 ceph-mon[32752]: pgmap v13169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:34.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:34 smithi167 ceph-mon[32751]: pgmap v13169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:34.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:34 smithi090 ceph-mon[29180]: pgmap v13169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:35.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23537.345134 secs 2024-04-27T01:07:35.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23537.345134 secs 2024-04-27T01:07:35.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23537.345134 secs 2024-04-27T01:07:36.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:36 smithi119 ceph-mon[32752]: pgmap v13170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:36.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:36 smithi167 ceph-mon[32751]: pgmap v13170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:36.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:36 smithi090 ceph-mon[29180]: pgmap v13170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:37.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23538.150479 secs 2024-04-27T01:07:37.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23538.150479 secs 2024-04-27T01:07:37.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23538.150479 secs 2024-04-27T01:07:38.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:38 smithi119 ceph-mon[32752]: pgmap v13171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:38.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:38 smithi167 ceph-mon[32751]: pgmap v13171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:38.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:38 smithi090 ceph-mon[29180]: pgmap v13171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:40.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:40 smithi119 ceph-mon[32752]: pgmap v13172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:40.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23542.345237 secs 2024-04-27T01:07:40.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:40 smithi167 ceph-mon[32751]: pgmap v13172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:40.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23542.345237 secs 2024-04-27T01:07:40.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:40 smithi090 ceph-mon[29180]: pgmap v13172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:40.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23542.345237 secs 2024-04-27T01:07:42.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:42 smithi119 ceph-mon[32752]: pgmap v13173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:42.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23543.150645 secs 2024-04-27T01:07:42.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:42 smithi167 ceph-mon[32751]: pgmap v13173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:42.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23543.150645 secs 2024-04-27T01:07:42.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:42 smithi090 ceph-mon[29180]: pgmap v13173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:42.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23543.150645 secs 2024-04-27T01:07:44.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:44 smithi119 ceph-mon[32752]: pgmap v13174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:44.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:44 smithi167 ceph-mon[32751]: pgmap v13174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:44.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:44 smithi090 ceph-mon[29180]: pgmap v13174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:45.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23547.345321 secs 2024-04-27T01:07:45.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23547.345321 secs 2024-04-27T01:07:45.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23547.345321 secs 2024-04-27T01:07:46.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:46 smithi119 ceph-mon[32752]: pgmap v13175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:46.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:46 smithi167 ceph-mon[32751]: pgmap v13175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:46.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:46 smithi090 ceph-mon[29180]: pgmap v13175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:47.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23548.150777 secs 2024-04-27T01:07:47.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23548.150777 secs 2024-04-27T01:07:47.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23548.150777 secs 2024-04-27T01:07:48.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:48 smithi119 ceph-mon[32752]: pgmap v13176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:48.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:48 smithi167 ceph-mon[32751]: pgmap v13176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:48.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:48 smithi090 ceph-mon[29180]: pgmap v13176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:50.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:50 smithi119 ceph-mon[32752]: pgmap v13177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:50.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23552.345407 secs 2024-04-27T01:07:50.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:50 smithi167 ceph-mon[32751]: pgmap v13177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:50.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23552.345407 secs 2024-04-27T01:07:50.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:50 smithi090 ceph-mon[29180]: pgmap v13177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:50.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23552.345407 secs 2024-04-27T01:07:52.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:52 smithi119 ceph-mon[32752]: pgmap v13178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:52.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23553.150891 secs 2024-04-27T01:07:52.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:52 smithi167 ceph-mon[32751]: pgmap v13178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:52.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23553.150891 secs 2024-04-27T01:07:52.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:52 smithi090 ceph-mon[29180]: pgmap v13178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:52.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23553.150891 secs 2024-04-27T01:07:54.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:54 smithi119 ceph-mon[32752]: pgmap v13179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:54 smithi167 ceph-mon[32751]: pgmap v13179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:54.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:54 smithi090 ceph-mon[29180]: pgmap v13179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:55.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23557.345493 secs 2024-04-27T01:07:55.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23557.345493 secs 2024-04-27T01:07:56.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23557.345493 secs 2024-04-27T01:07:56.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:56 smithi090 ceph-mon[29180]: pgmap v13180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:57.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:56 smithi119 ceph-mon[32752]: pgmap v13180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:57.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:56 smithi167 ceph-mon[32751]: pgmap v13180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:07:57.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23558.150989 secs 2024-04-27T01:07:57.960 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23558.150989 secs 2024-04-27T01:07:58.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23558.150989 secs 2024-04-27T01:07:58.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:07:58 smithi090 ceph-mon[29180]: pgmap v13181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:59.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:07:58 smithi119 ceph-mon[32752]: pgmap v13181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:07:59.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:07:58 smithi167 ceph-mon[32751]: pgmap v13181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:00.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:00 smithi090 ceph-mon[29180]: pgmap v13182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:00.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23562.345578 secs 2024-04-27T01:08:01.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:00 smithi119 ceph-mon[32752]: pgmap v13182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:01.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23562.345578 secs 2024-04-27T01:08:01.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:00 smithi167 ceph-mon[32751]: pgmap v13182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:01.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23562.345578 secs 2024-04-27T01:08:02.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:02 smithi090 ceph-mon[29180]: pgmap v13183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:02.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23563.151125 secs 2024-04-27T01:08:03.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:02 smithi119 ceph-mon[32752]: pgmap v13183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:03.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23563.151125 secs 2024-04-27T01:08:03.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:02 smithi167 ceph-mon[32751]: pgmap v13183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:03.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23563.151125 secs 2024-04-27T01:08:04.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:04 smithi090 ceph-mon[29180]: pgmap v13184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:05.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:04 smithi119 ceph-mon[32752]: pgmap v13184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:05.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:04 smithi167 ceph-mon[32751]: pgmap v13184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:05.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23567.345681 secs 2024-04-27T01:08:06.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23567.345681 secs 2024-04-27T01:08:06.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23567.345681 secs 2024-04-27T01:08:06.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:06 smithi090 ceph-mon[29180]: pgmap v13185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:07.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:06 smithi119 ceph-mon[32752]: pgmap v13185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:07.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:06 smithi167 ceph-mon[32751]: pgmap v13185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:07.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23568.151242 secs 2024-04-27T01:08:07.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23568.151242 secs 2024-04-27T01:08:08.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23568.151242 secs 2024-04-27T01:08:08.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:08 smithi090 ceph-mon[29180]: pgmap v13186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:09.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:08 smithi119 ceph-mon[32752]: pgmap v13186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:09.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:08 smithi167 ceph-mon[32751]: pgmap v13186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:10.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:10 smithi090 ceph-mon[29180]: pgmap v13187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:10.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23572.345760 secs 2024-04-27T01:08:11.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:10 smithi119 ceph-mon[32752]: pgmap v13187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:11.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23572.345760 secs 2024-04-27T01:08:11.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:10 smithi167 ceph-mon[32751]: pgmap v13187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:11.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23572.345760 secs 2024-04-27T01:08:12.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:12 smithi090 ceph-mon[29180]: pgmap v13188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:12.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23573.151357 secs 2024-04-27T01:08:13.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:12 smithi119 ceph-mon[32752]: pgmap v13188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:13.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23573.151357 secs 2024-04-27T01:08:13.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:12 smithi167 ceph-mon[32751]: pgmap v13188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:13.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23573.151357 secs 2024-04-27T01:08:14.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:14 smithi090 ceph-mon[29180]: pgmap v13189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:15.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:14 smithi119 ceph-mon[32752]: pgmap v13189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:15.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:14 smithi167 ceph-mon[32751]: pgmap v13189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:15.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23577.345851 secs 2024-04-27T01:08:16.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23577.345851 secs 2024-04-27T01:08:16.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23577.345851 secs 2024-04-27T01:08:16.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:16 smithi090 ceph-mon[29180]: pgmap v13190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:17.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:16 smithi119 ceph-mon[32752]: pgmap v13190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:17.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:16 smithi167 ceph-mon[32751]: pgmap v13190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:17.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23578.151475 secs 2024-04-27T01:08:17.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:08:17.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:08:17.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:08:17.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:08:18.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23578.151475 secs 2024-04-27T01:08:18.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:08:18.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:08:18.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:08:18.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:08:18.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23578.151475 secs 2024-04-27T01:08:18.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:08:18.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:08:18.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:08:18.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:08:18.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:18 smithi090 ceph-mon[29180]: pgmap v13191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:08:18.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:18 smithi090 ceph-mon[29180]: pgmap v13192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T01:08:18.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:18 smithi090 ceph-mon[29180]: pgmap v13193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T01:08:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:18 smithi119 ceph-mon[32752]: pgmap v13191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:08:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:18 smithi119 ceph-mon[32752]: pgmap v13192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T01:08:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:18 smithi119 ceph-mon[32752]: pgmap v13193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T01:08:19.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:18 smithi167 ceph-mon[32751]: pgmap v13191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:08:19.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:18 smithi167 ceph-mon[32751]: pgmap v13192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T01:08:19.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:18 smithi167 ceph-mon[32751]: pgmap v13193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T01:08:21.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:20 smithi119 ceph-mon[32752]: pgmap v13194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:08:21.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:20 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23582.345942 secs 2024-04-27T01:08:21.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:20 smithi167 ceph-mon[32751]: pgmap v13194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:08:21.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:20 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23582.345942 secs 2024-04-27T01:08:21.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:20 smithi090 ceph-mon[29180]: pgmap v13194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:08:21.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:20 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23582.345942 secs 2024-04-27T01:08:23.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23583.151583 secs 2024-04-27T01:08:23.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:22 smithi119 ceph-mon[32752]: pgmap v13195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:08:23.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23583.151583 secs 2024-04-27T01:08:23.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:22 smithi167 ceph-mon[32751]: pgmap v13195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:08:23.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23583.151583 secs 2024-04-27T01:08:23.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:22 smithi090 ceph-mon[29180]: pgmap v13195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:08:25.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:24 smithi119 ceph-mon[32752]: pgmap v13196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:08:25.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:24 smithi167 ceph-mon[32751]: pgmap v13196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:08:25.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:24 smithi090 ceph-mon[29180]: pgmap v13196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:08:26.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23587.346029 secs 2024-04-27T01:08:26.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23587.346029 secs 2024-04-27T01:08:26.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23587.346029 secs 2024-04-27T01:08:27.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:26 smithi119 ceph-mon[32752]: pgmap v13197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:08:27.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:26 smithi167 ceph-mon[32751]: pgmap v13197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:08:27.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:26 smithi090 ceph-mon[29180]: pgmap v13197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:08:27.960 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23588.151695 secs 2024-04-27T01:08:28.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23588.151695 secs 2024-04-27T01:08:28.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23588.151695 secs 2024-04-27T01:08:29.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:28 smithi119 ceph-mon[32752]: pgmap v13198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:08:29.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:28 smithi167 ceph-mon[32751]: pgmap v13198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:08:29.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:28 smithi090 ceph-mon[29180]: pgmap v13198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:08:31.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:30 smithi119 ceph-mon[32752]: pgmap v13199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:31.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23592.346108 secs 2024-04-27T01:08:31.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:30 smithi167 ceph-mon[32751]: pgmap v13199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:31.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23592.346108 secs 2024-04-27T01:08:31.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:30 smithi090 ceph-mon[29180]: pgmap v13199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:31.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23592.346108 secs 2024-04-27T01:08:32.918 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23593.151804 secs 2024-04-27T01:08:32.919 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:32 smithi090 ceph-mon[29180]: pgmap v13200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:33.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23593.151804 secs 2024-04-27T01:08:33.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:32 smithi119 ceph-mon[32752]: pgmap v13200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:33.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23593.151804 secs 2024-04-27T01:08:33.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:32 smithi167 ceph-mon[32751]: pgmap v13200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:35.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:34 smithi119 ceph-mon[32752]: pgmap v13201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:35.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:34 smithi167 ceph-mon[32751]: pgmap v13201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:35.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:34 smithi090 ceph-mon[29180]: pgmap v13201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:36.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:35 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23597.346170 secs 2024-04-27T01:08:36.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:35 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23597.346170 secs 2024-04-27T01:08:36.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:35 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23597.346170 secs 2024-04-27T01:08:37.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:36 smithi119 ceph-mon[32752]: pgmap v13202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:37.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:36 smithi167 ceph-mon[32751]: pgmap v13202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:37.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:36 smithi090 ceph-mon[29180]: pgmap v13202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:37.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23598.151946 secs 2024-04-27T01:08:37.946 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:37 smithi119 ceph-mon[32752]: pgmap v13203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:38.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23598.151946 secs 2024-04-27T01:08:38.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:37 smithi167 ceph-mon[32751]: pgmap v13203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:38.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23598.151946 secs 2024-04-27T01:08:38.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:37 smithi090 ceph-mon[29180]: pgmap v13203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:40.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:40 smithi090 ceph-mon[29180]: pgmap v13204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:40.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:40 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23602.346233 secs 2024-04-27T01:08:41.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:40 smithi119 ceph-mon[32752]: pgmap v13204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:41.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:40 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23602.346233 secs 2024-04-27T01:08:41.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:40 smithi167 ceph-mon[32751]: pgmap v13204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:41.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:40 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23602.346233 secs 2024-04-27T01:08:42.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23603.152057 secs 2024-04-27T01:08:42.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:42 smithi090 ceph-mon[29180]: pgmap v13205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:43.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23603.152057 secs 2024-04-27T01:08:43.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:42 smithi119 ceph-mon[32752]: pgmap v13205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:43.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23603.152057 secs 2024-04-27T01:08:43.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:42 smithi167 ceph-mon[32751]: pgmap v13205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:44.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:44 smithi119 ceph-mon[32752]: pgmap v13206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:44.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:44 smithi167 ceph-mon[32751]: pgmap v13206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:44.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:44 smithi090 ceph-mon[29180]: pgmap v13206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:45.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:45 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23607.346317 secs 2024-04-27T01:08:45.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:45 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23607.346317 secs 2024-04-27T01:08:45.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:45 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23607.346317 secs 2024-04-27T01:08:46.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:46 smithi090 ceph-mon[29180]: pgmap v13207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:47.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:46 smithi119 ceph-mon[32752]: pgmap v13207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:47.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:46 smithi167 ceph-mon[32751]: pgmap v13207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:47.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23608.152194 secs 2024-04-27T01:08:48.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23608.152194 secs 2024-04-27T01:08:48.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23608.152194 secs 2024-04-27T01:08:48.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:48 smithi090 ceph-mon[29180]: pgmap v13208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:49.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:48 smithi119 ceph-mon[32752]: pgmap v13208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:49.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:48 smithi167 ceph-mon[32751]: pgmap v13208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:50.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:50 smithi090 ceph-mon[29180]: pgmap v13209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:50.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:50 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23612.346398 secs 2024-04-27T01:08:51.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:50 smithi119 ceph-mon[32752]: pgmap v13209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:51.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:50 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23612.346398 secs 2024-04-27T01:08:51.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:50 smithi167 ceph-mon[32751]: pgmap v13209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:51.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:50 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23612.346398 secs 2024-04-27T01:08:52.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23613.152331 secs 2024-04-27T01:08:52.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:52 smithi090 ceph-mon[29180]: pgmap v13210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:53.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23613.152331 secs 2024-04-27T01:08:53.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:52 smithi119 ceph-mon[32752]: pgmap v13210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:53.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23613.152331 secs 2024-04-27T01:08:53.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:52 smithi167 ceph-mon[32751]: pgmap v13210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:54.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:54 smithi090 ceph-mon[29180]: pgmap v13211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:55.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:54 smithi119 ceph-mon[32752]: pgmap v13211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:55.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:54 smithi167 ceph-mon[32751]: pgmap v13211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:08:55.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:55 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23617.346483 secs 2024-04-27T01:08:56.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:55 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23617.346483 secs 2024-04-27T01:08:56.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:55 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23617.346483 secs 2024-04-27T01:08:56.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:56 smithi090 ceph-mon[29180]: pgmap v13212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:57.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:56 smithi119 ceph-mon[32752]: pgmap v13212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:57.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:56 smithi167 ceph-mon[32751]: pgmap v13212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:57.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23618.152461 secs 2024-04-27T01:08:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23618.152461 secs 2024-04-27T01:08:58.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23618.152461 secs 2024-04-27T01:08:58.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:08:58 smithi090 ceph-mon[29180]: pgmap v13213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:59.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:08:58 smithi119 ceph-mon[32752]: pgmap v13213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:08:59.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:08:58 smithi167 ceph-mon[32751]: pgmap v13213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:00.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:00 smithi090 ceph-mon[29180]: pgmap v13214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:00.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:00 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23622.346566 secs 2024-04-27T01:09:01.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:00 smithi119 ceph-mon[32752]: pgmap v13214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:01.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:00 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23622.346566 secs 2024-04-27T01:09:01.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:00 smithi167 ceph-mon[32751]: pgmap v13214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:01.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:00 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23622.346566 secs 2024-04-27T01:09:03.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23623.152582 secs 2024-04-27T01:09:03.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:02 smithi119 ceph-mon[32752]: pgmap v13215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:03.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23623.152582 secs 2024-04-27T01:09:03.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:02 smithi167 ceph-mon[32751]: pgmap v13215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:03.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23623.152582 secs 2024-04-27T01:09:03.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:02 smithi090 ceph-mon[29180]: pgmap v13215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:05.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:04 smithi119 ceph-mon[32752]: pgmap v13216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:05.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:04 smithi167 ceph-mon[32751]: pgmap v13216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:05.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:04 smithi090 ceph-mon[29180]: pgmap v13216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:06.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:05 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23627.346650 secs 2024-04-27T01:09:06.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:05 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23627.346650 secs 2024-04-27T01:09:06.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:05 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23627.346650 secs 2024-04-27T01:09:07.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:06 smithi119 ceph-mon[32752]: pgmap v13217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:07.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:06 smithi167 ceph-mon[32751]: pgmap v13217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:07.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:06 smithi090 ceph-mon[29180]: pgmap v13217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:07.945 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23628.152706 secs 2024-04-27T01:09:08.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23628.152706 secs 2024-04-27T01:09:08.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23628.152706 secs 2024-04-27T01:09:09.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:08 smithi119 ceph-mon[32752]: pgmap v13218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:09.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:08 smithi167 ceph-mon[32751]: pgmap v13218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:09.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:08 smithi090 ceph-mon[29180]: pgmap v13218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:11.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:10 smithi119 ceph-mon[32752]: pgmap v13219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:11.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:10 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23632.346737 secs 2024-04-27T01:09:11.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:10 smithi167 ceph-mon[32751]: pgmap v13219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:11.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:10 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23632.346737 secs 2024-04-27T01:09:11.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:10 smithi090 ceph-mon[29180]: pgmap v13219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:11.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:10 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23632.346737 secs 2024-04-27T01:09:13.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23633.152798 secs 2024-04-27T01:09:13.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:12 smithi119 ceph-mon[32752]: pgmap v13220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:13.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23633.152798 secs 2024-04-27T01:09:13.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:12 smithi167 ceph-mon[32751]: pgmap v13220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:13.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23633.152798 secs 2024-04-27T01:09:13.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:12 smithi090 ceph-mon[29180]: pgmap v13220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:15.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:14 smithi119 ceph-mon[32752]: pgmap v13221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:15.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:14 smithi167 ceph-mon[32751]: pgmap v13221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:15.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:14 smithi090 ceph-mon[29180]: pgmap v13221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:16.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:15 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23637.346801 secs 2024-04-27T01:09:16.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:15 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23637.346801 secs 2024-04-27T01:09:16.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:15 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23637.346801 secs 2024-04-27T01:09:17.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:16 smithi119 ceph-mon[32752]: pgmap v13222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:17.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:16 smithi167 ceph-mon[32751]: pgmap v13222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:17.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:16 smithi090 ceph-mon[29180]: pgmap v13222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:18.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23638.152888 secs 2024-04-27T01:09:18.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:09:18.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23638.152888 secs 2024-04-27T01:09:18.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:09:18.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23638.152888 secs 2024-04-27T01:09:18.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:09:19.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:18 smithi119 ceph-mon[32752]: pgmap v13223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:09:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:09:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:18 smithi119 ceph-mon[32752]: pgmap v13224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:09:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:18 smithi119 ceph-mon[32752]: pgmap v13225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:19.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:09:19.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:18 smithi167 ceph-mon[32751]: pgmap v13223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:19.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:09:19.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:09:19.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:18 smithi167 ceph-mon[32751]: pgmap v13224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:09:19.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:18 smithi167 ceph-mon[32751]: pgmap v13225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:19.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:09:19.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:18 smithi090 ceph-mon[29180]: pgmap v13223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:19.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:09:19.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:09:19.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:18 smithi090 ceph-mon[29180]: pgmap v13224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:09:19.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:18 smithi090 ceph-mon[29180]: pgmap v13225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:19.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:09:21.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23642.346890 secs 2024-04-27T01:09:21.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:21 smithi090 ceph-mon[29180]: pgmap v13226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:21.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23642.346890 secs 2024-04-27T01:09:21.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:21 smithi119 ceph-mon[32752]: pgmap v13226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:21.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23642.346890 secs 2024-04-27T01:09:21.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:21 smithi167 ceph-mon[32751]: pgmap v13226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:22.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23643.153023 secs 2024-04-27T01:09:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23643.153023 secs 2024-04-27T01:09:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23643.153023 secs 2024-04-27T01:09:23.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:23 smithi090 ceph-mon[29180]: pgmap v13227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:09:23.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:23 smithi119 ceph-mon[32752]: pgmap v13227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:09:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:23 smithi167 ceph-mon[32751]: pgmap v13227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:09:25.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:25 smithi090 ceph-mon[29180]: pgmap v13228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:25.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:25 smithi119 ceph-mon[32752]: pgmap v13228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:25.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:25 smithi167 ceph-mon[32751]: pgmap v13228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:26.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23647.346975 secs 2024-04-27T01:09:26.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23647.346975 secs 2024-04-27T01:09:26.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23647.346975 secs 2024-04-27T01:09:27.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23648.153147 secs 2024-04-27T01:09:27.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:27 smithi167 ceph-mon[32751]: pgmap v13229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:27.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23648.153147 secs 2024-04-27T01:09:27.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:27 smithi119 ceph-mon[32752]: pgmap v13229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:27.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23648.153147 secs 2024-04-27T01:09:27.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:27 smithi090 ceph-mon[29180]: pgmap v13229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:09:29.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:29 smithi090 ceph-mon[29180]: pgmap v13230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:09:29.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:29 smithi119 ceph-mon[32752]: pgmap v13230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:09:29.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:29 smithi167 ceph-mon[32751]: pgmap v13230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:09:31.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23652.347076 secs 2024-04-27T01:09:31.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:31 smithi119 ceph-mon[32752]: pgmap v13231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:31.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23652.347076 secs 2024-04-27T01:09:31.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:31 smithi167 ceph-mon[32751]: pgmap v13231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:31.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23652.347076 secs 2024-04-27T01:09:31.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:31 smithi090 ceph-mon[29180]: pgmap v13231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:32.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23653.153211 secs 2024-04-27T01:09:32.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23653.153211 secs 2024-04-27T01:09:32.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23653.153211 secs 2024-04-27T01:09:33.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:33 smithi119 ceph-mon[32752]: pgmap v13232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:33.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:33 smithi167 ceph-mon[32751]: pgmap v13232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:33.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:33 smithi090 ceph-mon[29180]: pgmap v13232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:35.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:35 smithi119 ceph-mon[32752]: pgmap v13233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:35.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:35 smithi167 ceph-mon[32751]: pgmap v13233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:35.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:35 smithi090 ceph-mon[29180]: pgmap v13233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:36.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 23657.347160 secs 2024-04-27T01:09:36.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 23657.347160 secs 2024-04-27T01:09:36.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 23657.347160 secs 2024-04-27T01:09:37.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23658.153340 secs 2024-04-27T01:09:37.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:37 smithi090 ceph-mon[29180]: pgmap v13234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:37.617 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:09:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:09:37.281+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:09:37.617 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:09:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:09:37.281+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:09:37.617 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:09:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:09:37.281+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:09:37.617 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:09:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:09:37.281+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:09:37.617 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:09:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:09:37.281+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:09:37.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23658.153340 secs 2024-04-27T01:09:37.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:37 smithi119 ceph-mon[32752]: pgmap v13234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:37.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23658.153340 secs 2024-04-27T01:09:37.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:37 smithi167 ceph-mon[32751]: pgmap v13234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:39.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:39 smithi119 ceph-mon[32752]: pgmap v13235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:39.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:39 smithi167 ceph-mon[32751]: pgmap v13235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:39.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:39 smithi090 ceph-mon[29180]: pgmap v13235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:41.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23662.347243 secs 2024-04-27T01:09:41.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:41 smithi119 ceph-mon[32752]: pgmap v13236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:41.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23662.347243 secs 2024-04-27T01:09:41.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:41 smithi167 ceph-mon[32751]: pgmap v13236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:41.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23662.347243 secs 2024-04-27T01:09:41.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:41 smithi090 ceph-mon[29180]: pgmap v13236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:42.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23663.153476 secs 2024-04-27T01:09:42.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23663.153476 secs 2024-04-27T01:09:42.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23663.153476 secs 2024-04-27T01:09:43.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:43 smithi119 ceph-mon[32752]: pgmap v13237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:43.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:09:43.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:09:43.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:43 smithi167 ceph-mon[32751]: pgmap v13237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:43.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:09:43.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:09:43.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:43 smithi090 ceph-mon[29180]: pgmap v13237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:43.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:09:43.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:09:45.668 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:45 smithi090 ceph-mon[29180]: pgmap v13238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:45.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:45 smithi119 ceph-mon[32752]: pgmap v13238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:45.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:45 smithi167 ceph-mon[32751]: pgmap v13238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:46.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23667.347323 secs 2024-04-27T01:09:46.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23667.347323 secs 2024-04-27T01:09:46.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23667.347323 secs 2024-04-27T01:09:47.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23668.153600 secs 2024-04-27T01:09:47.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:47 smithi119 ceph-mon[32752]: pgmap v13239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:47.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23668.153600 secs 2024-04-27T01:09:47.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:47 smithi167 ceph-mon[32751]: pgmap v13239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:47.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23668.153600 secs 2024-04-27T01:09:47.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:47 smithi090 ceph-mon[29180]: pgmap v13239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:09:49.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:49 smithi119 ceph-mon[32752]: pgmap v13240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:49.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:49 smithi167 ceph-mon[32751]: pgmap v13240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:49.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:49 smithi090 ceph-mon[29180]: pgmap v13240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:09:51.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:51 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 23672.347408 secs 2024-04-27T01:09:51.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:51 smithi119 ceph-mon[32752]: slow request 1441.721496 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:09:51.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:51 smithi119 ceph-mon[32752]: pgmap v13241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:51.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:51 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 23672.347408 secs 2024-04-27T01:09:51.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:51 smithi167 ceph-mon[32751]: slow request 1441.721496 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:09:51.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:51 smithi167 ceph-mon[32751]: pgmap v13241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:51.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:51 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 23672.347408 secs 2024-04-27T01:09:51.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:51 smithi090 ceph-mon[29180]: slow request 1441.721496 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:09:51.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:51 smithi090 ceph-mon[29180]: pgmap v13241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:52.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23673.153679 secs 2024-04-27T01:09:52.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23673.153679 secs 2024-04-27T01:09:52.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23673.153679 secs 2024-04-27T01:09:53.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:53 smithi119 ceph-mon[32752]: pgmap v13242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:53.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:53 smithi167 ceph-mon[32751]: pgmap v13242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:53.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:53 smithi090 ceph-mon[29180]: pgmap v13242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:55.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:55 smithi119 ceph-mon[32752]: pgmap v13243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:55.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:55 smithi167 ceph-mon[32751]: pgmap v13243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:55.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:55 smithi090 ceph-mon[29180]: pgmap v13243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:56.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23677.347464 secs 2024-04-27T01:09:56.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23677.347464 secs 2024-04-27T01:09:56.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23677.347464 secs 2024-04-27T01:09:57.710 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23678.153775 secs 2024-04-27T01:09:57.710 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:57 smithi167 ceph-mon[32751]: pgmap v13244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:57.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23678.153775 secs 2024-04-27T01:09:57.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:57 smithi119 ceph-mon[32752]: pgmap v13244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:57.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23678.153775 secs 2024-04-27T01:09:57.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:57 smithi090 ceph-mon[29180]: pgmap v13244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:09:59.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:09:59 smithi119 ceph-mon[32752]: pgmap v13245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:59.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:09:59 smithi167 ceph-mon[32751]: pgmap v13245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:09:59.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:09:59 smithi090 ceph-mon[29180]: pgmap v13245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:00.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:10:00.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:10:00.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:10:01.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23682.347522 secs 2024-04-27T01:10:01.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:01 smithi119 ceph-mon[32752]: pgmap v13246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:01.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23682.347522 secs 2024-04-27T01:10:01.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:01 smithi167 ceph-mon[32751]: pgmap v13246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:01.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23682.347522 secs 2024-04-27T01:10:01.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:01 smithi090 ceph-mon[29180]: pgmap v13246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:02.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23683.153874 secs 2024-04-27T01:10:02.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23683.153874 secs 2024-04-27T01:10:02.872 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23683.153874 secs 2024-04-27T01:10:03.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:03 smithi119 ceph-mon[32752]: pgmap v13247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:03.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:03 smithi167 ceph-mon[32751]: pgmap v13247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:03.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:03 smithi090 ceph-mon[29180]: pgmap v13247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:05.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:05 smithi119 ceph-mon[32752]: pgmap v13248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:05.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:05 smithi167 ceph-mon[32751]: pgmap v13248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:05.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:05 smithi090 ceph-mon[29180]: pgmap v13248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:06.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23687.347580 secs 2024-04-27T01:10:06.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23687.347580 secs 2024-04-27T01:10:06.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23687.347580 secs 2024-04-27T01:10:07.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:07 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 23688.154004 secs 2024-04-27T01:10:07.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:07 smithi119 ceph-mon[32752]: slow request 5760.226561 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:10:07.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:07 smithi119 ceph-mon[32752]: pgmap v13249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:07.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:07 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 23688.154004 secs 2024-04-27T01:10:07.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:07 smithi167 ceph-mon[32751]: slow request 5760.226561 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:10:07.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:07 smithi167 ceph-mon[32751]: pgmap v13249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:07.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:07 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 23688.154004 secs 2024-04-27T01:10:07.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:07 smithi090 ceph-mon[29180]: slow request 5760.226561 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:10:07.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:07 smithi090 ceph-mon[29180]: pgmap v13249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:09.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:09 smithi119 ceph-mon[32752]: pgmap v13250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:09.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:09 smithi167 ceph-mon[32751]: pgmap v13250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:09.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:09 smithi090 ceph-mon[29180]: pgmap v13250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:11.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23692.347636 secs 2024-04-27T01:10:11.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:11 smithi119 ceph-mon[32752]: pgmap v13251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:11.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23692.347636 secs 2024-04-27T01:10:11.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:11 smithi167 ceph-mon[32751]: pgmap v13251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:11.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23692.347636 secs 2024-04-27T01:10:11.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:11 smithi090 ceph-mon[29180]: pgmap v13251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:12.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23693.154140 secs 2024-04-27T01:10:12.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23693.154140 secs 2024-04-27T01:10:12.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23693.154140 secs 2024-04-27T01:10:13.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:13 smithi119 ceph-mon[32752]: pgmap v13252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:13.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:13 smithi167 ceph-mon[32751]: pgmap v13252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:13.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:13 smithi090 ceph-mon[29180]: pgmap v13252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:15.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:15 smithi090 ceph-mon[29180]: pgmap v13253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:15.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:15 smithi119 ceph-mon[32752]: pgmap v13253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:15.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:15 smithi167 ceph-mon[32751]: pgmap v13253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:16.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23697.347692 secs 2024-04-27T01:10:16.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23697.347692 secs 2024-04-27T01:10:16.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23697.347692 secs 2024-04-27T01:10:17.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23698.154283 secs 2024-04-27T01:10:17.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:17 smithi119 ceph-mon[32752]: pgmap v13254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:17.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23698.154283 secs 2024-04-27T01:10:17.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:17 smithi167 ceph-mon[32751]: pgmap v13254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:17.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23698.154283 secs 2024-04-27T01:10:17.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:17 smithi090 ceph-mon[29180]: pgmap v13254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:18.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:10:18.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:10:18.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:10:19.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:19 smithi119 ceph-mon[32752]: pgmap v13255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:19.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:10:19.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:10:19.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:10:19.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:19 smithi167 ceph-mon[32751]: pgmap v13255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:19.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:10:19.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:10:19.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:10:19.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:19 smithi090 ceph-mon[29180]: pgmap v13255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:19.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:10:19.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:10:19.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:10:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:20 smithi119 ceph-mon[32752]: pgmap v13256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:10:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:20 smithi119 ceph-mon[32752]: pgmap v13257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23702.347750 secs 2024-04-27T01:10:20.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:20 smithi167 ceph-mon[32751]: pgmap v13256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:10:20.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:20 smithi167 ceph-mon[32751]: pgmap v13257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:20.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23702.347750 secs 2024-04-27T01:10:20.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:20 smithi090 ceph-mon[29180]: pgmap v13256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:10:20.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:20 smithi090 ceph-mon[29180]: pgmap v13257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:20.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23702.347750 secs 2024-04-27T01:10:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23703.154431 secs 2024-04-27T01:10:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:22 smithi119 ceph-mon[32752]: pgmap v13258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:22.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23703.154431 secs 2024-04-27T01:10:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:22 smithi167 ceph-mon[32751]: pgmap v13258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:22.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23703.154431 secs 2024-04-27T01:10:22.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:22 smithi090 ceph-mon[29180]: pgmap v13258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:24.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:24 smithi119 ceph-mon[32752]: pgmap v13259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:10:24.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:24 smithi167 ceph-mon[32751]: pgmap v13259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:10:24.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:24 smithi090 ceph-mon[29180]: pgmap v13259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:10:25.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23707.347798 secs 2024-04-27T01:10:25.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23707.347798 secs 2024-04-27T01:10:25.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23707.347798 secs 2024-04-27T01:10:26.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:26 smithi119 ceph-mon[32752]: pgmap v13260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:26.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:26 smithi167 ceph-mon[32751]: pgmap v13260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:26.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:26 smithi090 ceph-mon[29180]: pgmap v13260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:10:27.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23708.154550 secs 2024-04-27T01:10:27.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23708.154550 secs 2024-04-27T01:10:27.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23708.154550 secs 2024-04-27T01:10:28.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:28 smithi119 ceph-mon[32752]: pgmap v13261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:10:28.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:28 smithi167 ceph-mon[32751]: pgmap v13261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:10:28.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:28 smithi090 ceph-mon[29180]: pgmap v13261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:10:30.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:30 smithi119 ceph-mon[32752]: pgmap v13262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:10:30.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23712.347855 secs 2024-04-27T01:10:30.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:30 smithi167 ceph-mon[32751]: pgmap v13262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:10:30.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23712.347855 secs 2024-04-27T01:10:30.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:30 smithi090 ceph-mon[29180]: pgmap v13262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:10:30.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23712.347855 secs 2024-04-27T01:10:32.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23713.154617 secs 2024-04-27T01:10:32.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:32 smithi119 ceph-mon[32752]: pgmap v13263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:32.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23713.154617 secs 2024-04-27T01:10:32.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:32 smithi167 ceph-mon[32751]: pgmap v13263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:32.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23713.154617 secs 2024-04-27T01:10:32.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:32 smithi090 ceph-mon[29180]: pgmap v13263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:34.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:34 smithi167 ceph-mon[32751]: pgmap v13264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:34.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:34 smithi090 ceph-mon[29180]: pgmap v13264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:35.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:34 smithi119 ceph-mon[32752]: pgmap v13264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:35.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:35 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 23717.347911 secs 2024-04-27T01:10:35.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:35 smithi167 ceph-mon[32751]: slow request 23044.165296 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:10:35.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:35 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 23717.347911 secs 2024-04-27T01:10:35.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:35 smithi090 ceph-mon[29180]: slow request 23044.165296 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:10:36.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:35 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 23717.347911 secs 2024-04-27T01:10:36.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:35 smithi119 ceph-mon[32752]: slow request 23044.165296 seconds old, received at 2024-04-26T18:46:30.936019+0000: client_request(client.26372:1 getattr pAsLsXsFs #0x1 2024-04-26T18:46:30.932226+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:10:36.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:36 smithi167 ceph-mon[32751]: pgmap v13265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:36.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:36 smithi090 ceph-mon[29180]: pgmap v13265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:37.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:36 smithi119 ceph-mon[32752]: pgmap v13265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:37.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23718.154694 secs 2024-04-27T01:10:37.944 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23718.154694 secs 2024-04-27T01:10:38.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23718.154694 secs 2024-04-27T01:10:38.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:38 smithi090 ceph-mon[29180]: pgmap v13266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:39.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:38 smithi119 ceph-mon[32752]: pgmap v13266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:39.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:38 smithi167 ceph-mon[32751]: pgmap v13266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:40.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:40 smithi090 ceph-mon[29180]: pgmap v13267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:40.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23722.347970 secs 2024-04-27T01:10:41.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:40 smithi119 ceph-mon[32752]: pgmap v13267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:41.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23722.347970 secs 2024-04-27T01:10:41.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:40 smithi167 ceph-mon[32751]: pgmap v13267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:41.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23722.347970 secs 2024-04-27T01:10:42.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23723.154788 secs 2024-04-27T01:10:42.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:42 smithi090 ceph-mon[29180]: pgmap v13268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:43.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23723.154788 secs 2024-04-27T01:10:43.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:42 smithi119 ceph-mon[32752]: pgmap v13268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:43.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23723.154788 secs 2024-04-27T01:10:43.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:42 smithi167 ceph-mon[32751]: pgmap v13268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:44.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:44 smithi090 ceph-mon[29180]: pgmap v13269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:45.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:44 smithi119 ceph-mon[32752]: pgmap v13269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:45.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:44 smithi167 ceph-mon[32751]: pgmap v13269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:45.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23727.348027 secs 2024-04-27T01:10:46.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23727.348027 secs 2024-04-27T01:10:46.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23727.348027 secs 2024-04-27T01:10:46.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:46 smithi090 ceph-mon[29180]: pgmap v13270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:47.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:46 smithi119 ceph-mon[32752]: pgmap v13270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:47.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:46 smithi167 ceph-mon[32751]: pgmap v13270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:47.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23728.154854 secs 2024-04-27T01:10:48.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23728.154854 secs 2024-04-27T01:10:48.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23728.154854 secs 2024-04-27T01:10:48.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:48 smithi090 ceph-mon[29180]: pgmap v13271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:49.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:48 smithi119 ceph-mon[32752]: pgmap v13271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:49.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:48 smithi167 ceph-mon[32751]: pgmap v13271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:50.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:50 smithi090 ceph-mon[29180]: pgmap v13272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:50.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23732.348084 secs 2024-04-27T01:10:51.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:50 smithi119 ceph-mon[32752]: pgmap v13272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:51.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23732.348084 secs 2024-04-27T01:10:51.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:50 smithi167 ceph-mon[32751]: pgmap v13272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:51.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23732.348084 secs 2024-04-27T01:10:52.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23733.154995 secs 2024-04-27T01:10:52.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:52 smithi090 ceph-mon[29180]: pgmap v13273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:53.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23733.154995 secs 2024-04-27T01:10:53.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:52 smithi119 ceph-mon[32752]: pgmap v13273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:53.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23733.154995 secs 2024-04-27T01:10:53.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:52 smithi167 ceph-mon[32751]: pgmap v13273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:54.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:54 smithi090 ceph-mon[29180]: pgmap v13274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:55.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:54 smithi119 ceph-mon[32752]: pgmap v13274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:55.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:54 smithi167 ceph-mon[32751]: pgmap v13274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:55.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23737.348139 secs 2024-04-27T01:10:56.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23737.348139 secs 2024-04-27T01:10:56.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23737.348139 secs 2024-04-27T01:10:56.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:56 smithi090 ceph-mon[29180]: pgmap v13275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:57.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:56 smithi119 ceph-mon[32752]: pgmap v13275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:57.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:56 smithi167 ceph-mon[32751]: pgmap v13275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:10:57.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23738.155138 secs 2024-04-27T01:10:57.959 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23738.155138 secs 2024-04-27T01:10:58.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23738.155138 secs 2024-04-27T01:10:58.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:10:58 smithi090 ceph-mon[29180]: pgmap v13276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:59.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:10:58 smithi119 ceph-mon[32752]: pgmap v13276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:10:59.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:10:58 smithi167 ceph-mon[32751]: pgmap v13276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:00.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:00 smithi090 ceph-mon[29180]: pgmap v13277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:00.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23742.348197 secs 2024-04-27T01:11:01.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:00 smithi119 ceph-mon[32752]: pgmap v13277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:01.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23742.348197 secs 2024-04-27T01:11:01.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:00 smithi167 ceph-mon[32751]: pgmap v13277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:01.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23742.348197 secs 2024-04-27T01:11:02.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23743.155258 secs 2024-04-27T01:11:02.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:02 smithi090 ceph-mon[29180]: pgmap v13278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:03.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23743.155258 secs 2024-04-27T01:11:03.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:02 smithi119 ceph-mon[32752]: pgmap v13278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:03.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23743.155258 secs 2024-04-27T01:11:03.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:02 smithi167 ceph-mon[32751]: pgmap v13278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:04.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:04 smithi090 ceph-mon[29180]: pgmap v13279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:05.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:04 smithi119 ceph-mon[32752]: pgmap v13279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:05.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:04 smithi167 ceph-mon[32751]: pgmap v13279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:06.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23747.348256 secs 2024-04-27T01:11:06.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23747.348256 secs 2024-04-27T01:11:06.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23747.348256 secs 2024-04-27T01:11:07.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:06 smithi119 ceph-mon[32752]: pgmap v13280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:07.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:06 smithi167 ceph-mon[32751]: pgmap v13280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:07.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:06 smithi090 ceph-mon[29180]: pgmap v13280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:07.944 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23748.155407 secs 2024-04-27T01:11:08.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23748.155407 secs 2024-04-27T01:11:08.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23748.155407 secs 2024-04-27T01:11:09.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:08 smithi119 ceph-mon[32752]: pgmap v13281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:09.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:08 smithi167 ceph-mon[32751]: pgmap v13281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:09.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:08 smithi090 ceph-mon[29180]: pgmap v13281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:11.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:10 smithi119 ceph-mon[32752]: pgmap v13282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:11.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23752.348315 secs 2024-04-27T01:11:11.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:10 smithi167 ceph-mon[32751]: pgmap v13282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:11.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23752.348315 secs 2024-04-27T01:11:11.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:10 smithi090 ceph-mon[29180]: pgmap v13282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:11.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23752.348315 secs 2024-04-27T01:11:13.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23753.155474 secs 2024-04-27T01:11:13.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:12 smithi119 ceph-mon[32752]: pgmap v13283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:13.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23753.155474 secs 2024-04-27T01:11:13.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:12 smithi167 ceph-mon[32751]: pgmap v13283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:13.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23753.155474 secs 2024-04-27T01:11:13.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:12 smithi090 ceph-mon[29180]: pgmap v13283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:15.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:14 smithi119 ceph-mon[32752]: pgmap v13284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:15.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:14 smithi167 ceph-mon[32751]: pgmap v13284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:15.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:14 smithi090 ceph-mon[29180]: pgmap v13284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:16.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23757.348402 secs 2024-04-27T01:11:16.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23757.348402 secs 2024-04-27T01:11:16.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23757.348402 secs 2024-04-27T01:11:17.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:16 smithi119 ceph-mon[32752]: pgmap v13285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:17.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:16 smithi167 ceph-mon[32751]: pgmap v13285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:17.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:16 smithi090 ceph-mon[29180]: pgmap v13285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:18.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23758.155592 secs 2024-04-27T01:11:18.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23758.155592 secs 2024-04-27T01:11:18.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23758.155592 secs 2024-04-27T01:11:19.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:18 smithi090 ceph-mon[29180]: pgmap v13286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:19.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:18 smithi119 ceph-mon[32752]: pgmap v13286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:19.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:18 smithi167 ceph-mon[32751]: pgmap v13286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:19.997 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:11:20.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:11:20.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:11:20.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:20 smithi119 ceph-mon[32752]: pgmap v13287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23762.348488 secs 2024-04-27T01:11:20.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:20 smithi090 ceph-mon[29180]: pgmap v13287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:20.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:20.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23762.348488 secs 2024-04-27T01:11:21.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:20 smithi167 ceph-mon[32751]: pgmap v13287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:21.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:21.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:21.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:21.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:21.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23762.348488 secs 2024-04-27T01:11:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23763.155665 secs 2024-04-27T01:11:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: pgmap v13288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23763.155665 secs 2024-04-27T01:11:22.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: pgmap v13288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:22.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23763.155665 secs 2024-04-27T01:11:22.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: pgmap v13288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:22.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:22.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:22.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:11:23.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:11:23.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:11:23.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:11:23.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:11:23.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:11:23.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: pgmap v13289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-27T01:11:23.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: pgmap v13290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:23.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:23.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:11:23.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:11:23.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:11:23.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:11:23.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:11:23.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:11:23.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:11:23.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:11:23.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: pgmap v13289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-27T01:11:23.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: pgmap v13290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:23.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:23.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:11:23.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:11:23.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:11:23.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:11:23.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:11:23.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:11:23.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:11:23.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:11:23.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: pgmap v13289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 887 B/s rd, 1 op/s 2024-04-27T01:11:23.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: pgmap v13290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:23.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:11:25.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:24 smithi119 ceph-mon[32752]: pgmap v13291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:25.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:24 smithi167 ceph-mon[32751]: pgmap v13291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:25.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:24 smithi090 ceph-mon[29180]: pgmap v13291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:26.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23767.348579 secs 2024-04-27T01:11:26.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23767.348579 secs 2024-04-27T01:11:26.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23767.348579 secs 2024-04-27T01:11:27.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:26 smithi119 ceph-mon[32752]: pgmap v13292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:27.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:26 smithi167 ceph-mon[32751]: pgmap v13292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:27.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:26 smithi090 ceph-mon[29180]: pgmap v13292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:28.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23768.155787 secs 2024-04-27T01:11:28.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23768.155787 secs 2024-04-27T01:11:28.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23768.155787 secs 2024-04-27T01:11:29.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:28 smithi119 ceph-mon[32752]: pgmap v13293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:29.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:28 smithi167 ceph-mon[32751]: pgmap v13293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:29.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:28 smithi090 ceph-mon[29180]: pgmap v13293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:30.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23772.348666 secs 2024-04-27T01:11:30.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23772.348666 secs 2024-04-27T01:11:31.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23772.348666 secs 2024-04-27T01:11:31.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:31 smithi167 ceph-mon[32751]: pgmap v13294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:31.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:31 smithi090 ceph-mon[29180]: pgmap v13294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:32.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:31 smithi119 ceph-mon[32752]: pgmap v13294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:11:32.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23773.155911 secs 2024-04-27T01:11:33.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23773.155911 secs 2024-04-27T01:11:33.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23773.155911 secs 2024-04-27T01:11:33.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:33 smithi090 ceph-mon[29180]: pgmap v13295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:11:34.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:33 smithi119 ceph-mon[32752]: pgmap v13295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:11:34.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:33 smithi167 ceph-mon[32751]: pgmap v13295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:11:35.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:34 smithi119 ceph-mon[32752]: pgmap v13296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:35.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:34 smithi167 ceph-mon[32751]: pgmap v13296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:35.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:34 smithi090 ceph-mon[29180]: pgmap v13296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:36.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23777.348766 secs 2024-04-27T01:11:36.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23777.348766 secs 2024-04-27T01:11:36.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23777.348766 secs 2024-04-27T01:11:37.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:36 smithi119 ceph-mon[32752]: pgmap v13297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:37.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:36 smithi167 ceph-mon[32751]: pgmap v13297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:37.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:36 smithi090 ceph-mon[29180]: pgmap v13297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:38.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23778.156065 secs 2024-04-27T01:11:38.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23778.156065 secs 2024-04-27T01:11:38.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23778.156065 secs 2024-04-27T01:11:39.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:38 smithi119 ceph-mon[32752]: pgmap v13298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:39.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:38 smithi167 ceph-mon[32751]: pgmap v13298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:39.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:38 smithi090 ceph-mon[29180]: pgmap v13298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:40.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:40 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23782.348845 secs 2024-04-27T01:11:40.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:40 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23782.348845 secs 2024-04-27T01:11:41.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:40 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23782.348845 secs 2024-04-27T01:11:41.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:41 smithi090 ceph-mon[29180]: pgmap v13299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:42.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:41 smithi119 ceph-mon[32752]: pgmap v13299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:42.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:41 smithi167 ceph-mon[32751]: pgmap v13299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:42.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23783.156176 secs 2024-04-27T01:11:43.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23783.156176 secs 2024-04-27T01:11:43.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23783.156176 secs 2024-04-27T01:11:43.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:43 smithi090 ceph-mon[29180]: pgmap v13300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:44.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:43 smithi119 ceph-mon[32752]: pgmap v13300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:44.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:43 smithi167 ceph-mon[32751]: pgmap v13300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:45.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:44 smithi119 ceph-mon[32752]: pgmap v13301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:45.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:44 smithi167 ceph-mon[32751]: pgmap v13301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:45.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:44 smithi090 ceph-mon[29180]: pgmap v13301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:46.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:45 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23787.348910 secs 2024-04-27T01:11:46.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:45 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23787.348910 secs 2024-04-27T01:11:46.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:45 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23787.348910 secs 2024-04-27T01:11:47.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:46 smithi119 ceph-mon[32752]: pgmap v13302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:47.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:46 smithi167 ceph-mon[32751]: pgmap v13302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:47.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:46 smithi090 ceph-mon[29180]: pgmap v13302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:48.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23788.156279 secs 2024-04-27T01:11:48.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23788.156279 secs 2024-04-27T01:11:48.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23788.156279 secs 2024-04-27T01:11:49.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:48 smithi119 ceph-mon[32752]: pgmap v13303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:49.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:48 smithi167 ceph-mon[32751]: pgmap v13303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:49.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:48 smithi090 ceph-mon[29180]: pgmap v13303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:50.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:50 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23792.348977 secs 2024-04-27T01:11:50.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:50 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23792.348977 secs 2024-04-27T01:11:51.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:50 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23792.348977 secs 2024-04-27T01:11:51.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:51 smithi090 ceph-mon[29180]: pgmap v13304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:52.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:51 smithi119 ceph-mon[32752]: pgmap v13304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:52.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:51 smithi167 ceph-mon[32751]: pgmap v13304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:52.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23793.156418 secs 2024-04-27T01:11:53.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23793.156418 secs 2024-04-27T01:11:53.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23793.156418 secs 2024-04-27T01:11:53.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:53 smithi090 ceph-mon[29180]: pgmap v13305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:54.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:53 smithi119 ceph-mon[32752]: pgmap v13305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:54.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:53 smithi167 ceph-mon[32751]: pgmap v13305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:55.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:54 smithi119 ceph-mon[32752]: pgmap v13306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:55.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:54 smithi167 ceph-mon[32751]: pgmap v13306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:55.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:54 smithi090 ceph-mon[29180]: pgmap v13306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:56.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:55 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23797.349041 secs 2024-04-27T01:11:56.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:55 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23797.349041 secs 2024-04-27T01:11:56.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:55 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23797.349041 secs 2024-04-27T01:11:57.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:56 smithi119 ceph-mon[32752]: pgmap v13307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:57.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:56 smithi167 ceph-mon[32751]: pgmap v13307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:57.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:56 smithi090 ceph-mon[29180]: pgmap v13307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:11:58.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23798.156553 secs 2024-04-27T01:11:58.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23798.156553 secs 2024-04-27T01:11:58.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23798.156553 secs 2024-04-27T01:11:59.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:11:58 smithi119 ceph-mon[32752]: pgmap v13308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:59.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:11:58 smithi167 ceph-mon[32751]: pgmap v13308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:11:59.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:11:58 smithi090 ceph-mon[29180]: pgmap v13308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:00.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:00 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23802.349103 secs 2024-04-27T01:12:00.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:00 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23802.349103 secs 2024-04-27T01:12:01.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:00 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23802.349103 secs 2024-04-27T01:12:01.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:01 smithi090 ceph-mon[29180]: pgmap v13309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:02.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:01 smithi119 ceph-mon[32752]: pgmap v13309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:02.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:01 smithi167 ceph-mon[32751]: pgmap v13309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:02.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23803.156643 secs 2024-04-27T01:12:03.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23803.156643 secs 2024-04-27T01:12:03.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23803.156643 secs 2024-04-27T01:12:03.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:03 smithi090 ceph-mon[29180]: pgmap v13310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:04.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:03 smithi119 ceph-mon[32752]: pgmap v13310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:04.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:03 smithi167 ceph-mon[32751]: pgmap v13310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:05.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:04 smithi119 ceph-mon[32752]: pgmap v13311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:05.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:04 smithi167 ceph-mon[32751]: pgmap v13311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:05.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:04 smithi090 ceph-mon[29180]: pgmap v13311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:06.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:05 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23807.349164 secs 2024-04-27T01:12:06.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:05 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23807.349164 secs 2024-04-27T01:12:06.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:05 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23807.349164 secs 2024-04-27T01:12:07.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:06 smithi119 ceph-mon[32752]: pgmap v13312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:07.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:06 smithi167 ceph-mon[32751]: pgmap v13312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:07.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:06 smithi090 ceph-mon[29180]: pgmap v13312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:08.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23808.156780 secs 2024-04-27T01:12:08.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23808.156780 secs 2024-04-27T01:12:08.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23808.156780 secs 2024-04-27T01:12:09.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:08 smithi119 ceph-mon[32752]: pgmap v13313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:09.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:08 smithi167 ceph-mon[32751]: pgmap v13313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:09.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:08 smithi090 ceph-mon[29180]: pgmap v13313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:10.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:10 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23812.349223 secs 2024-04-27T01:12:11.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:10 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23812.349223 secs 2024-04-27T01:12:11.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:10 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23812.349223 secs 2024-04-27T01:12:11.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:11 smithi090 ceph-mon[29180]: pgmap v13314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:12.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:11 smithi119 ceph-mon[32752]: pgmap v13314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:12.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:11 smithi167 ceph-mon[32751]: pgmap v13314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:12.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23813.156878 secs 2024-04-27T01:12:13.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23813.156878 secs 2024-04-27T01:12:13.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23813.156878 secs 2024-04-27T01:12:13.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:13 smithi090 ceph-mon[29180]: pgmap v13315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:14.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:13 smithi119 ceph-mon[32752]: pgmap v13315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:14.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:13 smithi167 ceph-mon[32751]: pgmap v13315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:15.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:14 smithi119 ceph-mon[32752]: pgmap v13316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:15.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:14 smithi167 ceph-mon[32751]: pgmap v13316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:15.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:14 smithi090 ceph-mon[29180]: pgmap v13316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:16.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:15 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23817.349309 secs 2024-04-27T01:12:16.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:15 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23817.349309 secs 2024-04-27T01:12:16.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:15 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23817.349309 secs 2024-04-27T01:12:17.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:16 smithi119 ceph-mon[32752]: pgmap v13317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:17.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:16 smithi167 ceph-mon[32751]: pgmap v13317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:17.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:16 smithi090 ceph-mon[29180]: pgmap v13317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:18.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23818.156947 secs 2024-04-27T01:12:18.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23818.156947 secs 2024-04-27T01:12:18.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23818.156947 secs 2024-04-27T01:12:19.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:18 smithi119 ceph-mon[32752]: pgmap v13318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:19.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:18 smithi167 ceph-mon[32751]: pgmap v13318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:19.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:18 smithi090 ceph-mon[29180]: pgmap v13318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:20.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:20 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23822.349395 secs 2024-04-27T01:12:21.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:20 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23822.349395 secs 2024-04-27T01:12:21.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:20 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23822.349395 secs 2024-04-27T01:12:21.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:21 smithi090 ceph-mon[29180]: pgmap v13319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:22.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:21 smithi119 ceph-mon[32752]: pgmap v13319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:22.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:21 smithi167 ceph-mon[32751]: pgmap v13319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:22.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23823.157075 secs 2024-04-27T01:12:23.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23823.157075 secs 2024-04-27T01:12:23.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23823.157075 secs 2024-04-27T01:12:23.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:23 smithi090 ceph-mon[29180]: pgmap v13320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:23.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:12:23.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:23.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:24.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:23 smithi119 ceph-mon[32752]: pgmap v13320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:24.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:12:24.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:24.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:24.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:23 smithi167 ceph-mon[32751]: pgmap v13320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:24.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:12:24.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:24.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:24.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:12:24.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:12:24.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:25.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:12:25.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:12:25.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:25.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:12:25.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:12:25.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:12:25.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:25 smithi090 ceph-mon[29180]: pgmap v13321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:12:25.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:25 smithi090 ceph-mon[29180]: pgmap v13322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:25.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:25 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23827.349484 secs 2024-04-27T01:12:26.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:25 smithi119 ceph-mon[32752]: pgmap v13321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:12:26.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:25 smithi119 ceph-mon[32752]: pgmap v13322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:26.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:25 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23827.349484 secs 2024-04-27T01:12:26.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:25 smithi167 ceph-mon[32751]: pgmap v13321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:12:26.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:25 smithi167 ceph-mon[32751]: pgmap v13322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:26.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:25 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23827.349484 secs 2024-04-27T01:12:27.859 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23828.157199 secs 2024-04-27T01:12:27.860 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:27 smithi167 ceph-mon[32751]: pgmap v13323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:27.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23828.157199 secs 2024-04-27T01:12:27.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:27 smithi090 ceph-mon[29180]: pgmap v13323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:28.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23828.157199 secs 2024-04-27T01:12:28.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:27 smithi119 ceph-mon[32752]: pgmap v13323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:29.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:29 smithi090 ceph-mon[29180]: pgmap v13324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T01:12:30.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:29 smithi119 ceph-mon[32752]: pgmap v13324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T01:12:30.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:29 smithi167 ceph-mon[32751]: pgmap v13324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T01:12:30.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:30 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23832.349567 secs 2024-04-27T01:12:31.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:30 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23832.349567 secs 2024-04-27T01:12:31.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:30 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23832.349567 secs 2024-04-27T01:12:31.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:31 smithi090 ceph-mon[29180]: pgmap v13325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:32.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:31 smithi119 ceph-mon[32752]: pgmap v13325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:31 smithi167 ceph-mon[32751]: pgmap v13325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:32.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23833.157318 secs 2024-04-27T01:12:33.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23833.157318 secs 2024-04-27T01:12:33.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23833.157318 secs 2024-04-27T01:12:33.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:33 smithi090 ceph-mon[29180]: pgmap v13326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:34.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:33 smithi119 ceph-mon[32752]: pgmap v13326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:34.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:33 smithi167 ceph-mon[32751]: pgmap v13326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:12:35.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:35 smithi090 ceph-mon[29180]: pgmap v13327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:12:35.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:35 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 23837.349648 secs 2024-04-27T01:12:36.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:35 smithi119 ceph-mon[32752]: pgmap v13327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:12:36.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:35 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 23837.349648 secs 2024-04-27T01:12:36.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:35 smithi167 ceph-mon[32751]: pgmap v13327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:12:36.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:35 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 23837.349648 secs 2024-04-27T01:12:37.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23838.157412 secs 2024-04-27T01:12:37.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:37 smithi090 ceph-mon[29180]: pgmap v13328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:38.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23838.157412 secs 2024-04-27T01:12:38.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:37 smithi119 ceph-mon[32752]: pgmap v13328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:38.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23838.157412 secs 2024-04-27T01:12:38.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:37 smithi167 ceph-mon[32751]: pgmap v13328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:40.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:39 smithi119 ceph-mon[32752]: pgmap v13329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:40.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:39 smithi167 ceph-mon[32751]: pgmap v13329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:40.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:39 smithi090 ceph-mon[29180]: pgmap v13329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:41.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:40 smithi119 ceph-mon[32752]: 33 slow requests, 1 included below; oldest blocked for > 23842.349737 secs 2024-04-27T01:12:41.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:40 smithi119 ceph-mon[32752]: slow request 182.815769 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:12:41.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:40 smithi167 ceph-mon[32751]: 33 slow requests, 1 included below; oldest blocked for > 23842.349737 secs 2024-04-27T01:12:41.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:40 smithi167 ceph-mon[32751]: slow request 182.815769 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:12:41.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:40 smithi090 ceph-mon[29180]: 33 slow requests, 1 included below; oldest blocked for > 23842.349737 secs 2024-04-27T01:12:41.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:40 smithi090 ceph-mon[29180]: slow request 182.815769 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:12:42.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:41 smithi119 ceph-mon[32752]: pgmap v13330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:42.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:41 smithi167 ceph-mon[32751]: pgmap v13330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:42.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:41 smithi090 ceph-mon[29180]: pgmap v13330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:43.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23843.157557 secs 2024-04-27T01:12:43.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23843.157557 secs 2024-04-27T01:12:43.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23843.157557 secs 2024-04-27T01:12:44.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:43 smithi119 ceph-mon[32752]: pgmap v13331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:44.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:12:44.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:12:44.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:43 smithi167 ceph-mon[32751]: pgmap v13331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:44.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:12:44.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:12:44.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:43 smithi090 ceph-mon[29180]: pgmap v13331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:44.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:12:44.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:12:46.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:45 smithi119 ceph-mon[32752]: pgmap v13332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:46.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:45 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23847.349802 secs 2024-04-27T01:12:46.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:45 smithi167 ceph-mon[32751]: pgmap v13332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:46.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:45 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23847.349802 secs 2024-04-27T01:12:46.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:45 smithi090 ceph-mon[29180]: pgmap v13332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:46.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:45 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23847.349802 secs 2024-04-27T01:12:48.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23848.157636 secs 2024-04-27T01:12:48.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:47 smithi119 ceph-mon[32752]: pgmap v13333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:48.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23848.157636 secs 2024-04-27T01:12:48.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:47 smithi167 ceph-mon[32751]: pgmap v13333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:48.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23848.157636 secs 2024-04-27T01:12:48.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:47 smithi090 ceph-mon[29180]: pgmap v13333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:50.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:49 smithi119 ceph-mon[32752]: pgmap v13334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:50.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:49 smithi167 ceph-mon[32751]: pgmap v13334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:50.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:49 smithi090 ceph-mon[29180]: pgmap v13334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:51.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:50 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23852.349862 secs 2024-04-27T01:12:51.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:50 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23852.349862 secs 2024-04-27T01:12:51.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:50 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23852.349862 secs 2024-04-27T01:12:52.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:51 smithi119 ceph-mon[32752]: pgmap v13335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:52.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:51 smithi167 ceph-mon[32751]: pgmap v13335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:52.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:51 smithi090 ceph-mon[29180]: pgmap v13335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:52.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23853.157748 secs 2024-04-27T01:12:53.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23853.157748 secs 2024-04-27T01:12:53.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23853.157748 secs 2024-04-27T01:12:54.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:53 smithi119 ceph-mon[32752]: pgmap v13336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:54.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:53 smithi167 ceph-mon[32751]: pgmap v13336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:54.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:53 smithi090 ceph-mon[29180]: pgmap v13336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:56.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:55 smithi119 ceph-mon[32752]: pgmap v13337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:56.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:55 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23857.349951 secs 2024-04-27T01:12:56.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:55 smithi167 ceph-mon[32751]: pgmap v13337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:56.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:55 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23857.349951 secs 2024-04-27T01:12:56.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:55 smithi090 ceph-mon[29180]: pgmap v13337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:12:56.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:55 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23857.349951 secs 2024-04-27T01:12:58.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23858.157868 secs 2024-04-27T01:12:58.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:57 smithi119 ceph-mon[32752]: pgmap v13338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:58.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23858.157868 secs 2024-04-27T01:12:58.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:57 smithi167 ceph-mon[32751]: pgmap v13338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:12:58.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23858.157868 secs 2024-04-27T01:12:58.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:57 smithi090 ceph-mon[29180]: pgmap v13338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:00.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:12:59 smithi119 ceph-mon[32752]: pgmap v13339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:00.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:12:59 smithi167 ceph-mon[32751]: pgmap v13339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:00.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:12:59 smithi090 ceph-mon[29180]: pgmap v13339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:01.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:00 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23862.350035 secs 2024-04-27T01:13:01.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:00 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23862.350035 secs 2024-04-27T01:13:01.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:00 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23862.350035 secs 2024-04-27T01:13:02.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:01 smithi090 ceph-mon[29180]: pgmap v13340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:02.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:01 smithi119 ceph-mon[32752]: pgmap v13340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:02.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:01 smithi167 ceph-mon[32751]: pgmap v13340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:03.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23863.157983 secs 2024-04-27T01:13:03.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23863.157983 secs 2024-04-27T01:13:03.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23863.157983 secs 2024-04-27T01:13:04.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:03 smithi119 ceph-mon[32752]: pgmap v13341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:04.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:03 smithi167 ceph-mon[32751]: pgmap v13341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:04.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:03 smithi090 ceph-mon[29180]: pgmap v13341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:06.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:05 smithi119 ceph-mon[32752]: pgmap v13342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:06.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:05 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23867.350098 secs 2024-04-27T01:13:06.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:05 smithi167 ceph-mon[32751]: pgmap v13342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:06.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:05 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23867.350098 secs 2024-04-27T01:13:06.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:05 smithi090 ceph-mon[29180]: pgmap v13342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:06.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:05 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23867.350098 secs 2024-04-27T01:13:08.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23868.158075 secs 2024-04-27T01:13:08.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:07 smithi119 ceph-mon[32752]: pgmap v13343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:08.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23868.158075 secs 2024-04-27T01:13:08.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:07 smithi167 ceph-mon[32751]: pgmap v13343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:08.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23868.158075 secs 2024-04-27T01:13:08.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:07 smithi090 ceph-mon[29180]: pgmap v13343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:10.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:09 smithi119 ceph-mon[32752]: pgmap v13344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:10.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:09 smithi167 ceph-mon[32751]: pgmap v13344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:10.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:09 smithi090 ceph-mon[29180]: pgmap v13344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:11.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:10 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23872.350160 secs 2024-04-27T01:13:11.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:10 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23872.350160 secs 2024-04-27T01:13:11.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:10 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23872.350160 secs 2024-04-27T01:13:12.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:11 smithi119 ceph-mon[32752]: pgmap v13345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:12.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:11 smithi167 ceph-mon[32751]: pgmap v13345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:12.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:11 smithi090 ceph-mon[29180]: pgmap v13345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:13.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23873.158214 secs 2024-04-27T01:13:13.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:12 smithi119 ceph-mon[32752]: pgmap v13346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:13.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23873.158214 secs 2024-04-27T01:13:13.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:12 smithi167 ceph-mon[32751]: pgmap v13346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:13.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23873.158214 secs 2024-04-27T01:13:13.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:12 smithi090 ceph-mon[29180]: pgmap v13346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:15.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:15 smithi119 ceph-mon[32752]: pgmap v13347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:15.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:15 smithi167 ceph-mon[32751]: pgmap v13347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:15.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:15 smithi090 ceph-mon[29180]: pgmap v13347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:16.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:16 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23877.350240 secs 2024-04-27T01:13:16.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:16 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23877.350240 secs 2024-04-27T01:13:16.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:16 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23877.350240 secs 2024-04-27T01:13:17.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23878.158327 secs 2024-04-27T01:13:17.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:17 smithi119 ceph-mon[32752]: pgmap v13348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:17.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23878.158327 secs 2024-04-27T01:13:17.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:17 smithi167 ceph-mon[32751]: pgmap v13348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:17.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23878.158327 secs 2024-04-27T01:13:17.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:17 smithi090 ceph-mon[29180]: pgmap v13348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:19.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:19 smithi119 ceph-mon[32752]: pgmap v13349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:19.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:19 smithi167 ceph-mon[32751]: pgmap v13349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:19.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:19 smithi090 ceph-mon[29180]: pgmap v13349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:21.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:21 smithi119 ceph-mon[32752]: pgmap v13350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:21.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:21 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23882.350302 secs 2024-04-27T01:13:21.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:21 smithi167 ceph-mon[32751]: pgmap v13350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:21.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:21 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23882.350302 secs 2024-04-27T01:13:21.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:21 smithi090 ceph-mon[29180]: pgmap v13350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:21.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:21 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23882.350302 secs 2024-04-27T01:13:22.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23883.158466 secs 2024-04-27T01:13:22.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23883.158466 secs 2024-04-27T01:13:22.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23883.158466 secs 2024-04-27T01:13:23.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:23 smithi119 ceph-mon[32752]: pgmap v13351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:23.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:23 smithi167 ceph-mon[32751]: pgmap v13351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:23.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:23 smithi090 ceph-mon[29180]: pgmap v13351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:25.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:25 smithi119 ceph-mon[32752]: pgmap v13352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:25.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:13:25.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:13:25.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:13:25.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:13:25.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:25 smithi167 ceph-mon[32751]: pgmap v13352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:25.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:13:25.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:13:25.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:13:25.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:13:25.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:25 smithi090 ceph-mon[29180]: pgmap v13352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:25.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:13:25.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:13:25.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:13:25.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:13:26.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:26 smithi119 ceph-mon[32752]: pgmap v13353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:13:26.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:26 smithi119 ceph-mon[32752]: pgmap v13354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:26.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:26 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23887.350388 secs 2024-04-27T01:13:26.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:26 smithi167 ceph-mon[32751]: pgmap v13353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:13:26.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:26 smithi167 ceph-mon[32751]: pgmap v13354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:26.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:26 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23887.350388 secs 2024-04-27T01:13:26.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:26 smithi090 ceph-mon[29180]: pgmap v13353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:13:26.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:26 smithi090 ceph-mon[29180]: pgmap v13354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:26.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:26 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23887.350388 secs 2024-04-27T01:13:27.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23888.158554 secs 2024-04-27T01:13:27.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23888.158554 secs 2024-04-27T01:13:27.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23888.158554 secs 2024-04-27T01:13:28.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:28 smithi119 ceph-mon[32752]: pgmap v13355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:28.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:28 smithi167 ceph-mon[32751]: pgmap v13355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:28.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:28 smithi090 ceph-mon[29180]: pgmap v13355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:30.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:30 smithi119 ceph-mon[32752]: pgmap v13356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:30.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:30 smithi167 ceph-mon[32751]: pgmap v13356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:30.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:30 smithi090 ceph-mon[29180]: pgmap v13356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:31.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:31 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23892.350472 secs 2024-04-27T01:13:31.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:31 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23892.350472 secs 2024-04-27T01:13:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:31 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23892.350472 secs 2024-04-27T01:13:32.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:32 smithi167 ceph-mon[32751]: pgmap v13357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:32.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23893.158703 secs 2024-04-27T01:13:32.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:32 smithi090 ceph-mon[29180]: pgmap v13357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:32.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23893.158703 secs 2024-04-27T01:13:32.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:32 smithi119 ceph-mon[32752]: pgmap v13357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:13:32.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23893.158703 secs 2024-04-27T01:13:34.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:34 smithi167 ceph-mon[32751]: pgmap v13358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:13:34.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:34 smithi090 ceph-mon[29180]: pgmap v13358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:13:34.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:34 smithi119 ceph-mon[32752]: pgmap v13358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:13:36.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:36 smithi167 ceph-mon[32751]: pgmap v13359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:13:36.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:36 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23897.350556 secs 2024-04-27T01:13:36.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:36 smithi090 ceph-mon[29180]: pgmap v13359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:13:36.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:36 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23897.350556 secs 2024-04-27T01:13:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:36 smithi119 ceph-mon[32752]: pgmap v13359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:13:36.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:36 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23897.350556 secs 2024-04-27T01:13:37.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23898.158774 secs 2024-04-27T01:13:37.443 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23898.158774 secs 2024-04-27T01:13:37.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23898.158774 secs 2024-04-27T01:13:38.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:38 smithi090 ceph-mon[29180]: pgmap v13360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:38.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:38 smithi119 ceph-mon[32752]: pgmap v13360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:38.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:38 smithi167 ceph-mon[32751]: pgmap v13360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:40.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:40 smithi090 ceph-mon[29180]: pgmap v13361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:40.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:40 smithi119 ceph-mon[32752]: pgmap v13361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:40.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:40 smithi167 ceph-mon[32751]: pgmap v13361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:41.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:41 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23902.350615 secs 2024-04-27T01:13:41.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:41 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23902.350615 secs 2024-04-27T01:13:41.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:41 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23902.350615 secs 2024-04-27T01:13:42.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:42 smithi090 ceph-mon[29180]: pgmap v13362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:42.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:42 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23903.158907 secs 2024-04-27T01:13:42.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:42 smithi119 ceph-mon[32752]: pgmap v13362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:42.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:42 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23903.158907 secs 2024-04-27T01:13:42.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:42 smithi167 ceph-mon[32751]: pgmap v13362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:42.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:42 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23903.158907 secs 2024-04-27T01:13:44.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:44 smithi090 ceph-mon[29180]: pgmap v13363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:44.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:44 smithi119 ceph-mon[32752]: pgmap v13363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:44.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:44 smithi167 ceph-mon[32751]: pgmap v13363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:46.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:46 smithi090 ceph-mon[29180]: pgmap v13364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:46.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:46 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23907.350701 secs 2024-04-27T01:13:46.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:46 smithi119 ceph-mon[32752]: pgmap v13364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:46.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:46 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23907.350701 secs 2024-04-27T01:13:46.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:46 smithi167 ceph-mon[32751]: pgmap v13364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:46.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:46 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23907.350701 secs 2024-04-27T01:13:47.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:47 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23908.159041 secs 2024-04-27T01:13:47.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:47 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23908.159041 secs 2024-04-27T01:13:47.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:47 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23908.159041 secs 2024-04-27T01:13:48.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:48 smithi090 ceph-mon[29180]: pgmap v13365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:48.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:48 smithi119 ceph-mon[32752]: pgmap v13365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:48.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:48 smithi167 ceph-mon[32751]: pgmap v13365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:50.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:50 smithi090 ceph-mon[29180]: pgmap v13366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:50.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:50 smithi119 ceph-mon[32752]: pgmap v13366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:50.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:50 smithi167 ceph-mon[32751]: pgmap v13366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:51.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:51 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23912.350789 secs 2024-04-27T01:13:51.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:51 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23912.350789 secs 2024-04-27T01:13:51.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:51 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23912.350789 secs 2024-04-27T01:13:52.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:52 smithi090 ceph-mon[29180]: pgmap v13367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:52.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:52 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23913.159153 secs 2024-04-27T01:13:52.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:52 smithi119 ceph-mon[32752]: pgmap v13367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:52.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:52 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23913.159153 secs 2024-04-27T01:13:52.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:52 smithi167 ceph-mon[32751]: pgmap v13367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:52.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:52 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23913.159153 secs 2024-04-27T01:13:54.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:54 smithi090 ceph-mon[29180]: pgmap v13368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:54.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:54 smithi119 ceph-mon[32752]: pgmap v13368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:54.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:54 smithi167 ceph-mon[32751]: pgmap v13368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:56.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:56 smithi090 ceph-mon[29180]: pgmap v13369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:56.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:56 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23917.350800 secs 2024-04-27T01:13:56.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:56 smithi119 ceph-mon[32752]: pgmap v13369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:56.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:56 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23917.350800 secs 2024-04-27T01:13:56.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:56 smithi167 ceph-mon[32751]: pgmap v13369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:13:56.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:56 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23917.350800 secs 2024-04-27T01:13:57.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:57 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23918.159215 secs 2024-04-27T01:13:57.457 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:57 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23918.159215 secs 2024-04-27T01:13:57.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:57 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23918.159215 secs 2024-04-27T01:13:58.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:13:58 smithi119 ceph-mon[32752]: pgmap v13370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:58.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:13:58 smithi167 ceph-mon[32751]: pgmap v13370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:13:58.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:13:58 smithi090 ceph-mon[29180]: pgmap v13370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:00.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:00 smithi119 ceph-mon[32752]: pgmap v13371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:00.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:00 smithi167 ceph-mon[32751]: pgmap v13371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:00.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:00 smithi090 ceph-mon[29180]: pgmap v13371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:01.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:01 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23922.350860 secs 2024-04-27T01:14:01.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:01 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23922.350860 secs 2024-04-27T01:14:01.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:01 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23922.350860 secs 2024-04-27T01:14:02.515 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:02 smithi090 ceph-mon[29180]: pgmap v13372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:02.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23923.159306 secs 2024-04-27T01:14:02.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:02 smithi119 ceph-mon[32752]: pgmap v13372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:02.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23923.159306 secs 2024-04-27T01:14:02.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:02 smithi167 ceph-mon[32751]: pgmap v13372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:02.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23923.159306 secs 2024-04-27T01:14:04.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:04 smithi119 ceph-mon[32752]: pgmap v13373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:04.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:04 smithi167 ceph-mon[32751]: pgmap v13373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:04.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:04 smithi090 ceph-mon[29180]: pgmap v13373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:06.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:06 smithi119 ceph-mon[32752]: pgmap v13374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:06.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:06 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23927.350921 secs 2024-04-27T01:14:06.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:06 smithi167 ceph-mon[32751]: pgmap v13374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:06.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:06 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23927.350921 secs 2024-04-27T01:14:06.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:06 smithi090 ceph-mon[29180]: pgmap v13374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:06.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:06 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23927.350921 secs 2024-04-27T01:14:07.443 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23928.159369 secs 2024-04-27T01:14:07.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23928.159369 secs 2024-04-27T01:14:07.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23928.159369 secs 2024-04-27T01:14:08.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:08 smithi119 ceph-mon[32752]: pgmap v13375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:08.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:08 smithi167 ceph-mon[32751]: pgmap v13375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:08.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:08 smithi090 ceph-mon[29180]: pgmap v13375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:10.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:10 smithi119 ceph-mon[32752]: pgmap v13376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:10.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:10 smithi167 ceph-mon[32751]: pgmap v13376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:10.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:10 smithi090 ceph-mon[29180]: pgmap v13376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:11.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:11 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23932.350982 secs 2024-04-27T01:14:11.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:11 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23932.350982 secs 2024-04-27T01:14:11.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:11 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23932.350982 secs 2024-04-27T01:14:12.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:12 smithi119 ceph-mon[32752]: pgmap v13377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:12.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23933.159410 secs 2024-04-27T01:14:12.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:12 smithi167 ceph-mon[32751]: pgmap v13377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:12.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23933.159410 secs 2024-04-27T01:14:12.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:12 smithi090 ceph-mon[29180]: pgmap v13377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:12.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23933.159410 secs 2024-04-27T01:14:14.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:14 smithi119 ceph-mon[32752]: pgmap v13378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:14.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:14 smithi167 ceph-mon[32751]: pgmap v13378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:14.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:14 smithi090 ceph-mon[29180]: pgmap v13378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:16.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:16 smithi119 ceph-mon[32752]: pgmap v13379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:16.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:16 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23937.351042 secs 2024-04-27T01:14:16.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:16 smithi167 ceph-mon[32751]: pgmap v13379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:16.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:16 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23937.351042 secs 2024-04-27T01:14:16.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:16 smithi090 ceph-mon[29180]: pgmap v13379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:16.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:16 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23937.351042 secs 2024-04-27T01:14:17.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23938.159447 secs 2024-04-27T01:14:17.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23938.159447 secs 2024-04-27T01:14:17.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23938.159447 secs 2024-04-27T01:14:18.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:18 smithi119 ceph-mon[32752]: pgmap v13380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:18.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:18 smithi167 ceph-mon[32751]: pgmap v13380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:18.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:18 smithi090 ceph-mon[29180]: pgmap v13380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:20.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:20 smithi119 ceph-mon[32752]: pgmap v13381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:20.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:20 smithi167 ceph-mon[32751]: pgmap v13381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:20.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:20 smithi090 ceph-mon[29180]: pgmap v13381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:21.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:21 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23942.351126 secs 2024-04-27T01:14:21.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:21 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23942.351126 secs 2024-04-27T01:14:21.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:21 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23942.351126 secs 2024-04-27T01:14:22.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:22 smithi119 ceph-mon[32752]: pgmap v13382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:22.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23943.159557 secs 2024-04-27T01:14:22.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:22 smithi167 ceph-mon[32751]: pgmap v13382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:22.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23943.159557 secs 2024-04-27T01:14:22.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:22 smithi090 ceph-mon[29180]: pgmap v13382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:22.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23943.159557 secs 2024-04-27T01:14:24.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:24 smithi119 ceph-mon[32752]: pgmap v13383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:24.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:24 smithi167 ceph-mon[32751]: pgmap v13383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:24.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:24 smithi090 ceph-mon[29180]: pgmap v13383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:25.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:14:25.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:14:25.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:14:26.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:26 smithi119 ceph-mon[32752]: pgmap v13384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:26.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:26 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23947.351192 secs 2024-04-27T01:14:26.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:14:26.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:14:26.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:26 smithi119 ceph-mon[32752]: pgmap v13385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:14:26.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:26 smithi119 ceph-mon[32752]: pgmap v13386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:26.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:14:26.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:26 smithi167 ceph-mon[32751]: pgmap v13384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:26.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:26 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23947.351192 secs 2024-04-27T01:14:26.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:14:26.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:14:26.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:26 smithi167 ceph-mon[32751]: pgmap v13385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:14:26.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:26 smithi167 ceph-mon[32751]: pgmap v13386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:26.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:14:26.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:26 smithi090 ceph-mon[29180]: pgmap v13384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:26.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:26 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23947.351192 secs 2024-04-27T01:14:26.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:14:26.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:14:26.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:26 smithi090 ceph-mon[29180]: pgmap v13385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:14:26.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:26 smithi090 ceph-mon[29180]: pgmap v13386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:26.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:14:27.457 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:27 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23948.159637 secs 2024-04-27T01:14:27.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:27 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23948.159637 secs 2024-04-27T01:14:27.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:27 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23948.159637 secs 2024-04-27T01:14:28.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:28 smithi119 ceph-mon[32752]: pgmap v13387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:14:28.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:28 smithi167 ceph-mon[32751]: pgmap v13387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:14:28.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:28 smithi090 ceph-mon[29180]: pgmap v13387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:14:30.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:30 smithi167 ceph-mon[32751]: pgmap v13388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:30.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:30 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23952.351274 secs 2024-04-27T01:14:30.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:30 smithi090 ceph-mon[29180]: pgmap v13388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:30.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:30 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23952.351274 secs 2024-04-27T01:14:31.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:30 smithi119 ceph-mon[32752]: pgmap v13388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:31.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:30 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23952.351274 secs 2024-04-27T01:14:32.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:32 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23953.159747 secs 2024-04-27T01:14:32.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:32 smithi167 ceph-mon[32751]: pgmap v13389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:32.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:32 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23953.159747 secs 2024-04-27T01:14:32.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:32 smithi090 ceph-mon[29180]: pgmap v13389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:33.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:32 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23953.159747 secs 2024-04-27T01:14:33.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:32 smithi119 ceph-mon[32752]: pgmap v13389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:34.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:34 smithi167 ceph-mon[32751]: pgmap v13390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:34.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:34 smithi090 ceph-mon[29180]: pgmap v13390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:35.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:34 smithi119 ceph-mon[32752]: pgmap v13390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:14:35.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:35 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23957.351356 secs 2024-04-27T01:14:36.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:35 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23957.351356 secs 2024-04-27T01:14:36.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:35 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23957.351356 secs 2024-04-27T01:14:36.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:36 smithi090 ceph-mon[29180]: pgmap v13391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:14:37.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:36 smithi119 ceph-mon[32752]: pgmap v13391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:14:37.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:36 smithi167 ceph-mon[32751]: pgmap v13391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:14:37.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:37 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23958.159827 secs 2024-04-27T01:14:37.865 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:14:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:14:37.411+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:14:37.865 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:14:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:14:37.411+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:14:37.865 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:14:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:14:37.411+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:14:37.865 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:14:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:14:37.411+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:14:37.865 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:14:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:14:37.411+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:14:37.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:37 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23958.159827 secs 2024-04-27T01:14:38.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:37 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23958.159827 secs 2024-04-27T01:14:38.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:38 smithi090 ceph-mon[29180]: pgmap v13392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:39.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:38 smithi119 ceph-mon[32752]: pgmap v13392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:39.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:38 smithi167 ceph-mon[32751]: pgmap v13392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:40.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:40 smithi090 ceph-mon[29180]: pgmap v13393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:40.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:40 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23962.351418 secs 2024-04-27T01:14:41.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:40 smithi119 ceph-mon[32752]: pgmap v13393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:41.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:40 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23962.351418 secs 2024-04-27T01:14:41.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:40 smithi167 ceph-mon[32751]: pgmap v13393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:41.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:40 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23962.351418 secs 2024-04-27T01:14:41.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23963.159960 secs 2024-04-27T01:14:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23963.159960 secs 2024-04-27T01:14:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23963.159960 secs 2024-04-27T01:14:42.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:42 smithi090 ceph-mon[29180]: pgmap v13394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:43.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:42 smithi119 ceph-mon[32752]: pgmap v13394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:43.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:42 smithi167 ceph-mon[32751]: pgmap v13394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:44.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:44 smithi090 ceph-mon[29180]: pgmap v13395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:45.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:44 smithi119 ceph-mon[32752]: pgmap v13395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:45.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:44 smithi167 ceph-mon[32751]: pgmap v13395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:45.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:45 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23967.351499 secs 2024-04-27T01:14:46.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:45 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23967.351499 secs 2024-04-27T01:14:46.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:45 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23967.351499 secs 2024-04-27T01:14:46.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:46 smithi090 ceph-mon[29180]: pgmap v13396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:46.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23968.160071 secs 2024-04-27T01:14:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:46 smithi119 ceph-mon[32752]: pgmap v13396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23968.160071 secs 2024-04-27T01:14:47.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:46 smithi167 ceph-mon[32751]: pgmap v13396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:47.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23968.160071 secs 2024-04-27T01:14:48.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:48 smithi090 ceph-mon[29180]: pgmap v13397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:49.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:48 smithi119 ceph-mon[32752]: pgmap v13397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:49.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:48 smithi167 ceph-mon[32751]: pgmap v13397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:50.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:50 smithi090 ceph-mon[29180]: pgmap v13398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:50.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:50 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23972.351584 secs 2024-04-27T01:14:51.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:50 smithi119 ceph-mon[32752]: pgmap v13398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:51.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:50 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23972.351584 secs 2024-04-27T01:14:51.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:50 smithi167 ceph-mon[32751]: pgmap v13398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:14:51.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:50 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23972.351584 secs 2024-04-27T01:14:51.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23973.160221 secs 2024-04-27T01:14:52.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23973.160221 secs 2024-04-27T01:14:52.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23973.160221 secs 2024-04-27T01:14:52.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:52 smithi090 ceph-mon[29180]: pgmap v13399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:53.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:52 smithi119 ceph-mon[32752]: pgmap v13399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:53.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:52 smithi167 ceph-mon[32751]: pgmap v13399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:54.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:54 smithi090 ceph-mon[29180]: pgmap v13400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:55.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:54 smithi119 ceph-mon[32752]: pgmap v13400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:55.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:54 smithi167 ceph-mon[32751]: pgmap v13400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:14:55.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:55 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23977.351667 secs 2024-04-27T01:14:56.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:55 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23977.351667 secs 2024-04-27T01:14:56.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:55 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23977.351667 secs 2024-04-27T01:14:56.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:56 smithi090 ceph-mon[29180]: pgmap v13401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:56.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23978.160321 secs 2024-04-27T01:14:57.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:56 smithi119 ceph-mon[32752]: pgmap v13401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:57.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23978.160321 secs 2024-04-27T01:14:57.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:56 smithi167 ceph-mon[32751]: pgmap v13401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:57.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23978.160321 secs 2024-04-27T01:14:58.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:14:58 smithi090 ceph-mon[29180]: pgmap v13402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:59.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:14:58 smithi119 ceph-mon[32752]: pgmap v13402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:14:59.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:14:58 smithi167 ceph-mon[32751]: pgmap v13402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:00.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:00 smithi090 ceph-mon[29180]: pgmap v13403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:00.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:00 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23982.351752 secs 2024-04-27T01:15:01.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:00 smithi119 ceph-mon[32752]: pgmap v13403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:01.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:00 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23982.351752 secs 2024-04-27T01:15:01.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:00 smithi167 ceph-mon[32751]: pgmap v13403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:01.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:00 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23982.351752 secs 2024-04-27T01:15:02.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23983.160435 secs 2024-04-27T01:15:02.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23983.160435 secs 2024-04-27T01:15:02.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23983.160435 secs 2024-04-27T01:15:03.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:02 smithi090 ceph-mon[29180]: pgmap v13404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:03.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:02 smithi119 ceph-mon[32752]: pgmap v13404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:03.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:02 smithi167 ceph-mon[32751]: pgmap v13404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:05.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:04 smithi119 ceph-mon[32752]: pgmap v13405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:05.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:04 smithi167 ceph-mon[32751]: pgmap v13405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:05.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:04 smithi090 ceph-mon[29180]: pgmap v13405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:06.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:05 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23987.351843 secs 2024-04-27T01:15:06.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:05 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23987.351843 secs 2024-04-27T01:15:06.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:05 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23987.351843 secs 2024-04-27T01:15:07.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:06 smithi119 ceph-mon[32752]: pgmap v13406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:07.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23988.160558 secs 2024-04-27T01:15:07.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:06 smithi167 ceph-mon[32751]: pgmap v13406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:07.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23988.160558 secs 2024-04-27T01:15:07.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:06 smithi090 ceph-mon[29180]: pgmap v13406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:07.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23988.160558 secs 2024-04-27T01:15:09.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:08 smithi119 ceph-mon[32752]: pgmap v13407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:09.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:08 smithi167 ceph-mon[32751]: pgmap v13407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:09.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:08 smithi090 ceph-mon[29180]: pgmap v13407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:11.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:10 smithi119 ceph-mon[32752]: pgmap v13408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:11.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:10 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23992.351904 secs 2024-04-27T01:15:11.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:10 smithi167 ceph-mon[32751]: pgmap v13408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:11.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:10 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23992.351904 secs 2024-04-27T01:15:11.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:10 smithi090 ceph-mon[29180]: pgmap v13408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:11.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:10 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23992.351904 secs 2024-04-27T01:15:12.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23993.160629 secs 2024-04-27T01:15:12.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23993.160629 secs 2024-04-27T01:15:12.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23993.160629 secs 2024-04-27T01:15:13.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:12 smithi119 ceph-mon[32752]: pgmap v13409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:13.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:12 smithi167 ceph-mon[32751]: pgmap v13409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:13.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:12 smithi090 ceph-mon[29180]: pgmap v13409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:15:15.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:14 smithi119 ceph-mon[32752]: pgmap v13410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:15.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:14 smithi167 ceph-mon[32751]: pgmap v13410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:15.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:14 smithi090 ceph-mon[29180]: pgmap v13410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:15:16.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:15 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 23997.351968 secs 2024-04-27T01:15:16.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:15 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 23997.351968 secs 2024-04-27T01:15:16.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:15 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 23997.351968 secs 2024-04-27T01:15:17.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:16 smithi119 ceph-mon[32752]: pgmap v13411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:17.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 23998.160763 secs 2024-04-27T01:15:17.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:16 smithi167 ceph-mon[32751]: pgmap v13411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:17.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 23998.160763 secs 2024-04-27T01:15:17.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:16 smithi090 ceph-mon[29180]: pgmap v13411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:17.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 23998.160763 secs 2024-04-27T01:15:19.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:18 smithi119 ceph-mon[32752]: pgmap v13412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:19.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:18 smithi167 ceph-mon[32751]: pgmap v13412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:19.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:18 smithi090 ceph-mon[29180]: pgmap v13412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:21.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:20 smithi119 ceph-mon[32752]: pgmap v13413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:21.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:20 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24002.352028 secs 2024-04-27T01:15:21.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:20 smithi167 ceph-mon[32751]: pgmap v13413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:21.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:20 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24002.352028 secs 2024-04-27T01:15:21.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:20 smithi090 ceph-mon[29180]: pgmap v13413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:21.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:20 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24002.352028 secs 2024-04-27T01:15:22.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24003.160884 secs 2024-04-27T01:15:22.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24003.160884 secs 2024-04-27T01:15:22.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24003.160884 secs 2024-04-27T01:15:23.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:22 smithi119 ceph-mon[32752]: pgmap v13414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:23.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:22 smithi167 ceph-mon[32751]: pgmap v13414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:22 smithi090 ceph-mon[29180]: pgmap v13414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:25.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:24 smithi119 ceph-mon[32752]: pgmap v13415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:25.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:24 smithi167 ceph-mon[32751]: pgmap v13415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:25.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:24 smithi090 ceph-mon[29180]: pgmap v13415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:26.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:25 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24007.352113 secs 2024-04-27T01:15:26.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:15:26.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:25 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24007.352113 secs 2024-04-27T01:15:26.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:15:26.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:25 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24007.352113 secs 2024-04-27T01:15:26.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:15:27.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:26 smithi119 ceph-mon[32752]: pgmap v13416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:27.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24008.160973 secs 2024-04-27T01:15:27.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:15:27.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:15:27.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:15:27.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:26 smithi167 ceph-mon[32751]: pgmap v13416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:27.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24008.160973 secs 2024-04-27T01:15:27.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:15:27.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:15:27.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:15:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:26 smithi090 ceph-mon[29180]: pgmap v13416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24008.160973 secs 2024-04-27T01:15:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:15:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:15:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:15:28.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:27 smithi119 ceph-mon[32752]: pgmap v13417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:15:28.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:27 smithi119 ceph-mon[32752]: pgmap v13418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:28.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:27 smithi167 ceph-mon[32751]: pgmap v13417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:15:28.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:27 smithi167 ceph-mon[32751]: pgmap v13418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:28.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:27 smithi090 ceph-mon[29180]: pgmap v13417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:15:28.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:27 smithi090 ceph-mon[29180]: pgmap v13418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:30.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:29 smithi119 ceph-mon[32752]: pgmap v13419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:30.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:29 smithi167 ceph-mon[32751]: pgmap v13419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:30.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:29 smithi090 ceph-mon[29180]: pgmap v13419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:31.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:30 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24012.352171 secs 2024-04-27T01:15:31.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:30 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24012.352171 secs 2024-04-27T01:15:31.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:30 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24012.352171 secs 2024-04-27T01:15:32.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:31 smithi119 ceph-mon[32752]: pgmap v13420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:32.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24013.161144 secs 2024-04-27T01:15:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:31 smithi167 ceph-mon[32751]: pgmap v13420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24013.161144 secs 2024-04-27T01:15:32.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:31 smithi090 ceph-mon[29180]: pgmap v13420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:32.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24013.161144 secs 2024-04-27T01:15:34.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:33 smithi119 ceph-mon[32752]: pgmap v13421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:15:34.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:33 smithi167 ceph-mon[32751]: pgmap v13421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:15:34.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:33 smithi090 ceph-mon[29180]: pgmap v13421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:15:35.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:34 smithi119 ceph-mon[32752]: pgmap v13422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:35.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:34 smithi167 ceph-mon[32751]: pgmap v13422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:35.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:34 smithi090 ceph-mon[29180]: pgmap v13422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:15:36.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:35 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24017.352258 secs 2024-04-27T01:15:36.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:35 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24017.352258 secs 2024-04-27T01:15:36.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:35 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24017.352258 secs 2024-04-27T01:15:37.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:36 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 24018.161259 secs 2024-04-27T01:15:37.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:36 smithi119 ceph-mon[32752]: slow request 23044.853024 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:15:37.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:36 smithi119 ceph-mon[32752]: pgmap v13423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:15:37.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:36 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 24018.161259 secs 2024-04-27T01:15:37.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:36 smithi167 ceph-mon[32751]: slow request 23044.853024 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:15:37.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:36 smithi167 ceph-mon[32751]: pgmap v13423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:15:37.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:36 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 24018.161259 secs 2024-04-27T01:15:37.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:36 smithi090 ceph-mon[29180]: slow request 23044.853024 seconds old, received at 2024-04-26T18:51:31.058880+0000: client_request(client.17061:1 getattr pAsLsXsFs #0x1 2024-04-26T18:51:31.055159+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:15:37.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:36 smithi090 ceph-mon[29180]: pgmap v13423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:15:39.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:39 smithi090 ceph-mon[29180]: pgmap v13424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:39.745 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:39 smithi119 ceph-mon[32752]: pgmap v13424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:39.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:39 smithi167 ceph-mon[32751]: pgmap v13424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:41.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:41 smithi090 ceph-mon[29180]: 33 slow requests, 1 included below; oldest blocked for > 24022.352341 secs 2024-04-27T01:15:41.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:41 smithi090 ceph-mon[29180]: slow request 362.818374 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:15:41.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:41 smithi090 ceph-mon[29180]: pgmap v13425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:41.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24023.161403 secs 2024-04-27T01:15:41.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:41 smithi119 ceph-mon[32752]: 33 slow requests, 1 included below; oldest blocked for > 24022.352341 secs 2024-04-27T01:15:41.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:41 smithi119 ceph-mon[32752]: slow request 362.818374 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:15:41.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:41 smithi119 ceph-mon[32752]: pgmap v13425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:41.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24023.161403 secs 2024-04-27T01:15:41.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:41 smithi167 ceph-mon[32751]: 33 slow requests, 1 included below; oldest blocked for > 24022.352341 secs 2024-04-27T01:15:41.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:41 smithi167 ceph-mon[32751]: slow request 362.818374 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:15:41.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:41 smithi167 ceph-mon[32751]: pgmap v13425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:41.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24023.161403 secs 2024-04-27T01:15:43.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:43 smithi090 ceph-mon[29180]: pgmap v13426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:43.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:43 smithi119 ceph-mon[32752]: pgmap v13426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:43.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:43 smithi167 ceph-mon[32751]: pgmap v13426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:45.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:45 smithi090 ceph-mon[29180]: pgmap v13427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:45.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:45 smithi119 ceph-mon[32752]: pgmap v13427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:45.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:45 smithi167 ceph-mon[32751]: pgmap v13427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:46.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:46 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24027.352423 secs 2024-04-27T01:15:46.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24028.161522 secs 2024-04-27T01:15:46.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:46 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24027.352423 secs 2024-04-27T01:15:46.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24028.161522 secs 2024-04-27T01:15:46.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:46 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24027.352423 secs 2024-04-27T01:15:46.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24028.161522 secs 2024-04-27T01:15:47.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:47 smithi090 ceph-mon[29180]: pgmap v13428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:47.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:47 smithi119 ceph-mon[32752]: pgmap v13428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:47.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:47 smithi167 ceph-mon[32751]: pgmap v13428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:49.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:49 smithi090 ceph-mon[29180]: pgmap v13429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:49.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:49 smithi119 ceph-mon[32752]: pgmap v13429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:49.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:49 smithi167 ceph-mon[32751]: pgmap v13429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:51.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:51 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24032.352509 secs 2024-04-27T01:15:51.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:51 smithi090 ceph-mon[29180]: pgmap v13430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:51.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24033.161636 secs 2024-04-27T01:15:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:51 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24032.352509 secs 2024-04-27T01:15:51.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:51 smithi119 ceph-mon[32752]: pgmap v13430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:51.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24033.161636 secs 2024-04-27T01:15:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:51 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24032.352509 secs 2024-04-27T01:15:51.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:51 smithi167 ceph-mon[32751]: pgmap v13430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:51.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24033.161636 secs 2024-04-27T01:15:53.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:53 smithi090 ceph-mon[29180]: pgmap v13431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:53.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:53 smithi119 ceph-mon[32752]: pgmap v13431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:53.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:53 smithi167 ceph-mon[32751]: pgmap v13431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:55.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:55 smithi090 ceph-mon[29180]: pgmap v13432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:55.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:55 smithi119 ceph-mon[32752]: pgmap v13432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:55.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:55 smithi167 ceph-mon[32751]: pgmap v13432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:56.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:56 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24037.352593 secs 2024-04-27T01:15:56.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24038.161756 secs 2024-04-27T01:15:56.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:56 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24037.352593 secs 2024-04-27T01:15:56.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24038.161756 secs 2024-04-27T01:15:56.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:56 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24037.352593 secs 2024-04-27T01:15:56.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24038.161756 secs 2024-04-27T01:15:57.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:57 smithi119 ceph-mon[32752]: pgmap v13433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:57.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:57 smithi167 ceph-mon[32751]: pgmap v13433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:57.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:57 smithi090 ceph-mon[29180]: pgmap v13433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:15:59.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:15:59 smithi119 ceph-mon[32752]: pgmap v13434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:59.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:15:59 smithi167 ceph-mon[32751]: pgmap v13434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:15:59.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:15:59 smithi090 ceph-mon[29180]: pgmap v13434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:01.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:01 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24042.352655 secs 2024-04-27T01:16:01.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:01 smithi119 ceph-mon[32752]: pgmap v13435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:01.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24043.161875 secs 2024-04-27T01:16:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:01 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24042.352655 secs 2024-04-27T01:16:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:01 smithi167 ceph-mon[32751]: pgmap v13435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:01.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24043.161875 secs 2024-04-27T01:16:01.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:01 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24042.352655 secs 2024-04-27T01:16:01.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:01 smithi090 ceph-mon[29180]: pgmap v13435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:01.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24043.161875 secs 2024-04-27T01:16:03.666 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:03 smithi090 ceph-mon[29180]: pgmap v13436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:03.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:03 smithi119 ceph-mon[32752]: pgmap v13436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:03.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:03 smithi167 ceph-mon[32751]: pgmap v13436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:05.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:05 smithi119 ceph-mon[32752]: pgmap v13437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:05.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:05 smithi167 ceph-mon[32751]: pgmap v13437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:05.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:05 smithi090 ceph-mon[29180]: pgmap v13437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:06.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:06 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24047.352740 secs 2024-04-27T01:16:06.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24048.162010 secs 2024-04-27T01:16:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:06 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24047.352740 secs 2024-04-27T01:16:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24048.162010 secs 2024-04-27T01:16:06.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:06 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24047.352740 secs 2024-04-27T01:16:06.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24048.162010 secs 2024-04-27T01:16:07.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:07 smithi119 ceph-mon[32752]: pgmap v13438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:07.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:07 smithi167 ceph-mon[32751]: pgmap v13438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:07.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:07 smithi090 ceph-mon[29180]: pgmap v13438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:09.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:09 smithi119 ceph-mon[32752]: pgmap v13439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:09.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:09 smithi167 ceph-mon[32751]: pgmap v13439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:09.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:09 smithi090 ceph-mon[29180]: pgmap v13439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:11 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24052.352801 secs 2024-04-27T01:16:11.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:11 smithi119 ceph-mon[32752]: pgmap v13440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:11.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24053.162082 secs 2024-04-27T01:16:11.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:11 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24052.352801 secs 2024-04-27T01:16:11.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:11 smithi167 ceph-mon[32751]: pgmap v13440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:11.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24053.162082 secs 2024-04-27T01:16:11.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:11 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24052.352801 secs 2024-04-27T01:16:11.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:11 smithi090 ceph-mon[29180]: pgmap v13440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:11.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24053.162082 secs 2024-04-27T01:16:13.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:13 smithi119 ceph-mon[32752]: pgmap v13441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:13.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:13 smithi167 ceph-mon[32751]: pgmap v13441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:13.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:13 smithi090 ceph-mon[29180]: pgmap v13441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:15.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:15 smithi119 ceph-mon[32752]: pgmap v13442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:15.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:15 smithi167 ceph-mon[32751]: pgmap v13442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:15.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:15 smithi090 ceph-mon[29180]: pgmap v13442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:16.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:16 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24057.352855 secs 2024-04-27T01:16:16.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24058.162144 secs 2024-04-27T01:16:16.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:16 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24057.352855 secs 2024-04-27T01:16:16.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24058.162144 secs 2024-04-27T01:16:16.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:16 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24057.352855 secs 2024-04-27T01:16:16.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24058.162144 secs 2024-04-27T01:16:17.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:17 smithi119 ceph-mon[32752]: pgmap v13443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:17.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:17 smithi167 ceph-mon[32751]: pgmap v13443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:17.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:17 smithi090 ceph-mon[29180]: pgmap v13443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:19.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:19 smithi119 ceph-mon[32752]: pgmap v13444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:19.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:19 smithi167 ceph-mon[32751]: pgmap v13444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:19.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:19 smithi090 ceph-mon[29180]: pgmap v13444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:21.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:21 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24062.352914 secs 2024-04-27T01:16:21.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:21 smithi119 ceph-mon[32752]: pgmap v13445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:21.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24063.162264 secs 2024-04-27T01:16:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:21 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24062.352914 secs 2024-04-27T01:16:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:21 smithi167 ceph-mon[32751]: pgmap v13445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:21.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24063.162264 secs 2024-04-27T01:16:21.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:21 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24062.352914 secs 2024-04-27T01:16:21.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:21 smithi090 ceph-mon[29180]: pgmap v13445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:21.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24063.162264 secs 2024-04-27T01:16:23.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:23 smithi119 ceph-mon[32752]: pgmap v13446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:23.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:23 smithi167 ceph-mon[32751]: pgmap v13446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:23.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:23 smithi090 ceph-mon[29180]: pgmap v13446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:25.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:25 smithi119 ceph-mon[32752]: pgmap v13447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:25.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:25 smithi167 ceph-mon[32751]: pgmap v13447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:25.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:25 smithi090 ceph-mon[29180]: pgmap v13447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:26.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:26 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24067.352967 secs 2024-04-27T01:16:26.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24068.162361 secs 2024-04-27T01:16:26.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:16:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:26 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24067.352967 secs 2024-04-27T01:16:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24068.162361 secs 2024-04-27T01:16:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:16:26.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:26 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24067.352967 secs 2024-04-27T01:16:26.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24068.162361 secs 2024-04-27T01:16:26.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:16:27.674 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:27 smithi090 ceph-mon[29180]: pgmap v13448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:27.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:27 smithi119 ceph-mon[32752]: pgmap v13448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:27.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:27 smithi167 ceph-mon[32751]: pgmap v13448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:29.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:29 smithi119 ceph-mon[32752]: pgmap v13449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:29.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:29 smithi167 ceph-mon[32751]: pgmap v13449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:29.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:29 smithi090 ceph-mon[29180]: pgmap v13449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:31.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24072.353060 secs 2024-04-27T01:16:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: pgmap v13450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:16:31.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:16:31.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24073.162448 secs 2024-04-27T01:16:31.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24072.353060 secs 2024-04-27T01:16:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: pgmap v13450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:16:31.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:16:31.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.615 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24073.162448 secs 2024-04-27T01:16:31.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24072.353060 secs 2024-04-27T01:16:31.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: pgmap v13450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:31.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:16:31.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:16:31.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:16:31.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24073.162448 secs 2024-04-27T01:16:32.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:32 smithi119 ceph-mon[32752]: pgmap v13451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-27T01:16:32.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:32 smithi119 ceph-mon[32752]: pgmap v13452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:32.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:32 smithi167 ceph-mon[32751]: pgmap v13451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-27T01:16:32.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:32 smithi167 ceph-mon[32751]: pgmap v13452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:32.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:32 smithi090 ceph-mon[29180]: pgmap v13451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-27T01:16:32.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:32 smithi090 ceph-mon[29180]: pgmap v13452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:34.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:34 smithi119 ceph-mon[32752]: pgmap v13453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T01:16:34.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:34 smithi167 ceph-mon[32751]: pgmap v13453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T01:16:34.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:34 smithi090 ceph-mon[29180]: pgmap v13453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T01:16:35.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:16:35.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:35 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:16:35.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:16:35.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:35 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:16:35.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:16:35.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:35 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:16:36.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:36 smithi119 ceph-mon[32752]: pgmap v13454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:36.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24077.353146 secs 2024-04-27T01:16:36.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24078.162515 secs 2024-04-27T01:16:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:36 smithi167 ceph-mon[32751]: pgmap v13454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24077.353146 secs 2024-04-27T01:16:36.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24078.162515 secs 2024-04-27T01:16:36.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:36 smithi090 ceph-mon[29180]: pgmap v13454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:36.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24077.353146 secs 2024-04-27T01:16:36.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24078.162515 secs 2024-04-27T01:16:38.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:38 smithi119 ceph-mon[32752]: pgmap v13455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T01:16:38.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:38 smithi167 ceph-mon[32751]: pgmap v13455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T01:16:38.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:38 smithi090 ceph-mon[29180]: pgmap v13455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T01:16:40.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:40 smithi119 ceph-mon[32752]: pgmap v13456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:40.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:40 smithi167 ceph-mon[32751]: pgmap v13456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:40.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:40 smithi090 ceph-mon[29180]: pgmap v13456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24082.353208 secs 2024-04-27T01:16:41.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24083.162605 secs 2024-04-27T01:16:41.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24082.353208 secs 2024-04-27T01:16:41.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24083.162605 secs 2024-04-27T01:16:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24082.353208 secs 2024-04-27T01:16:41.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24083.162605 secs 2024-04-27T01:16:42.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:42 smithi119 ceph-mon[32752]: pgmap v13457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:16:42.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:42 smithi167 ceph-mon[32751]: pgmap v13457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:16:42.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:42 smithi090 ceph-mon[29180]: pgmap v13457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:16:44.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:44 smithi119 ceph-mon[32752]: pgmap v13458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:44.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:44 smithi167 ceph-mon[32751]: pgmap v13458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:44.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:44 smithi090 ceph-mon[29180]: pgmap v13458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:46.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:46 smithi119 ceph-mon[32752]: pgmap v13459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:46.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24087.353299 secs 2024-04-27T01:16:46.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24088.162703 secs 2024-04-27T01:16:46.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:46 smithi167 ceph-mon[32751]: pgmap v13459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:46.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24087.353299 secs 2024-04-27T01:16:46.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24088.162703 secs 2024-04-27T01:16:46.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:46 smithi090 ceph-mon[29180]: pgmap v13459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:46.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24087.353299 secs 2024-04-27T01:16:46.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24088.162703 secs 2024-04-27T01:16:48.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:48 smithi119 ceph-mon[32752]: pgmap v13460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:48.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:48 smithi167 ceph-mon[32751]: pgmap v13460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:48.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:48 smithi090 ceph-mon[29180]: pgmap v13460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:50.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:50 smithi119 ceph-mon[32752]: pgmap v13461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:50.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:50 smithi167 ceph-mon[32751]: pgmap v13461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:50.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:50 smithi090 ceph-mon[29180]: pgmap v13461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24092.353383 secs 2024-04-27T01:16:51.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24093.162781 secs 2024-04-27T01:16:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24092.353383 secs 2024-04-27T01:16:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24093.162781 secs 2024-04-27T01:16:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24092.353383 secs 2024-04-27T01:16:51.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24093.162781 secs 2024-04-27T01:16:52.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:52 smithi119 ceph-mon[32752]: pgmap v13462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:52.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:52 smithi167 ceph-mon[32751]: pgmap v13462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:52.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:52 smithi090 ceph-mon[29180]: pgmap v13462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:54.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:54 smithi119 ceph-mon[32752]: pgmap v13463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:54.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:54 smithi167 ceph-mon[32751]: pgmap v13463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:54.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:54 smithi090 ceph-mon[29180]: pgmap v13463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:56 smithi119 ceph-mon[32752]: pgmap v13464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24097.353455 secs 2024-04-27T01:16:56.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24098.162896 secs 2024-04-27T01:16:56.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:56 smithi167 ceph-mon[32751]: pgmap v13464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:56.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24097.353455 secs 2024-04-27T01:16:56.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24098.162896 secs 2024-04-27T01:16:56.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:56 smithi090 ceph-mon[29180]: pgmap v13464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:16:56.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24097.353455 secs 2024-04-27T01:16:56.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24098.162896 secs 2024-04-27T01:16:58.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:16:58 smithi119 ceph-mon[32752]: pgmap v13465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:58.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:16:58 smithi167 ceph-mon[32751]: pgmap v13465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:16:58.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:16:58 smithi090 ceph-mon[29180]: pgmap v13465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:00.761 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:00 smithi090 ceph-mon[29180]: pgmap v13466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:00.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:00 smithi119 ceph-mon[32752]: pgmap v13466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:00.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:00 smithi167 ceph-mon[32751]: pgmap v13466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:01.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24102.353564 secs 2024-04-27T01:17:01.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24103.163016 secs 2024-04-27T01:17:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24102.353564 secs 2024-04-27T01:17:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24103.163016 secs 2024-04-27T01:17:01.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24102.353564 secs 2024-04-27T01:17:01.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24103.163016 secs 2024-04-27T01:17:02.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:02 smithi119 ceph-mon[32752]: pgmap v13467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:02.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:02 smithi167 ceph-mon[32751]: pgmap v13467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:02.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:02 smithi090 ceph-mon[29180]: pgmap v13467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:04.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:04 smithi119 ceph-mon[32752]: pgmap v13468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:04.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:04 smithi167 ceph-mon[32751]: pgmap v13468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:04.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:04 smithi090 ceph-mon[29180]: pgmap v13468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:06.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:06 smithi119 ceph-mon[32752]: pgmap v13469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:06.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24107.353634 secs 2024-04-27T01:17:06.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24108.163136 secs 2024-04-27T01:17:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:06 smithi167 ceph-mon[32751]: pgmap v13469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24107.353634 secs 2024-04-27T01:17:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24108.163136 secs 2024-04-27T01:17:06.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:06 smithi090 ceph-mon[29180]: pgmap v13469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:06.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24107.353634 secs 2024-04-27T01:17:06.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24108.163136 secs 2024-04-27T01:17:08.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:08 smithi119 ceph-mon[32752]: pgmap v13470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:08.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:08 smithi167 ceph-mon[32751]: pgmap v13470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:08.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:08 smithi090 ceph-mon[29180]: pgmap v13470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:10.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:10 smithi119 ceph-mon[32752]: pgmap v13471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:10.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:10 smithi167 ceph-mon[32751]: pgmap v13471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:10.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:10 smithi090 ceph-mon[29180]: pgmap v13471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24112.353695 secs 2024-04-27T01:17:11.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24113.163269 secs 2024-04-27T01:17:11.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24112.353695 secs 2024-04-27T01:17:11.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24113.163269 secs 2024-04-27T01:17:11.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24112.353695 secs 2024-04-27T01:17:11.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24113.163269 secs 2024-04-27T01:17:12.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:12 smithi119 ceph-mon[32752]: pgmap v13472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:12.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:12 smithi167 ceph-mon[32751]: pgmap v13472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:12.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:12 smithi090 ceph-mon[29180]: pgmap v13472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:14.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:14 smithi119 ceph-mon[32752]: pgmap v13473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:14.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:14 smithi167 ceph-mon[32751]: pgmap v13473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:14.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:14 smithi090 ceph-mon[29180]: pgmap v13473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:16.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:16 smithi119 ceph-mon[32752]: pgmap v13474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:16.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24117.353772 secs 2024-04-27T01:17:16.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24118.163411 secs 2024-04-27T01:17:16.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:16 smithi167 ceph-mon[32751]: pgmap v13474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:16.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24117.353772 secs 2024-04-27T01:17:16.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24118.163411 secs 2024-04-27T01:17:16.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:16 smithi090 ceph-mon[29180]: pgmap v13474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:16.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24117.353772 secs 2024-04-27T01:17:16.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24118.163411 secs 2024-04-27T01:17:18.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:18 smithi119 ceph-mon[32752]: pgmap v13475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:18.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:18 smithi167 ceph-mon[32751]: pgmap v13475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:18.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:18 smithi090 ceph-mon[29180]: pgmap v13475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:20.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:20 smithi119 ceph-mon[32752]: pgmap v13476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:20.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:20 smithi167 ceph-mon[32751]: pgmap v13476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:20.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:20 smithi090 ceph-mon[29180]: pgmap v13476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:21.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24122.353804 secs 2024-04-27T01:17:21.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24123.163503 secs 2024-04-27T01:17:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24122.353804 secs 2024-04-27T01:17:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24123.163503 secs 2024-04-27T01:17:21.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24122.353804 secs 2024-04-27T01:17:21.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24123.163503 secs 2024-04-27T01:17:22.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:22 smithi119 ceph-mon[32752]: pgmap v13477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:22.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:22 smithi167 ceph-mon[32751]: pgmap v13477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:22.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:22 smithi090 ceph-mon[29180]: pgmap v13477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:24.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:24 smithi119 ceph-mon[32752]: pgmap v13478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:24.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:24 smithi167 ceph-mon[32751]: pgmap v13478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:24.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:24 smithi090 ceph-mon[29180]: pgmap v13478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:25.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:25 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24127.353892 secs 2024-04-27T01:17:25.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:25 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24127.353892 secs 2024-04-27T01:17:26.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:25 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24127.353892 secs 2024-04-27T01:17:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:26 smithi167 ceph-mon[32751]: pgmap v13479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24128.163580 secs 2024-04-27T01:17:26.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:26 smithi090 ceph-mon[29180]: pgmap v13479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:26.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24128.163580 secs 2024-04-27T01:17:27.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:26 smithi119 ceph-mon[32752]: pgmap v13479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:27.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24128.163580 secs 2024-04-27T01:17:28.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:28 smithi167 ceph-mon[32751]: pgmap v13480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:28.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:28 smithi090 ceph-mon[29180]: pgmap v13480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:29.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:28 smithi119 ceph-mon[32752]: pgmap v13480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:30.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:30 smithi167 ceph-mon[32751]: pgmap v13481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:30.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:30 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24132.353975 secs 2024-04-27T01:17:30.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:30 smithi090 ceph-mon[29180]: pgmap v13481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:30.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:30 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24132.353975 secs 2024-04-27T01:17:31.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:30 smithi119 ceph-mon[32752]: pgmap v13481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:31.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:30 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24132.353975 secs 2024-04-27T01:17:31.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24133.163697 secs 2024-04-27T01:17:31.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:17:32.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24133.163697 secs 2024-04-27T01:17:32.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:17:32.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24133.163697 secs 2024-04-27T01:17:32.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:17:32.764 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:32 smithi090 ceph-mon[29180]: pgmap v13482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:32.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:32 smithi167 ceph-mon[32751]: pgmap v13482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:33.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:32 smithi119 ceph-mon[32752]: pgmap v13482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:34.742 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:34 smithi167 ceph-mon[32751]: pgmap v13483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:34.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:34 smithi090 ceph-mon[29180]: pgmap v13483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:35.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:34 smithi119 ceph-mon[32752]: pgmap v13483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:36.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: pgmap v13484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:36.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24137.354057 secs 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: pgmap v13485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: pgmap v13486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24138.163816 secs 2024-04-27T01:17:36.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: pgmap v13484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:36.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24137.354057 secs 2024-04-27T01:17:36.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:17:36.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:17:36.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: pgmap v13485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-27T01:17:36.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: pgmap v13486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:36.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24138.163816 secs 2024-04-27T01:17:36.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: pgmap v13484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:36.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24137.354057 secs 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: pgmap v13485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 981 B/s rd, 1 op/s 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: pgmap v13486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:17:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24138.163816 secs 2024-04-27T01:17:38.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:38 smithi119 ceph-mon[32752]: pgmap v13487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T01:17:38.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:38 smithi167 ceph-mon[32751]: pgmap v13487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T01:17:38.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:38 smithi090 ceph-mon[29180]: pgmap v13487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T01:17:40.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:40 smithi119 ceph-mon[32752]: pgmap v13488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:40.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:40 smithi167 ceph-mon[32751]: pgmap v13488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:40.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:40 smithi090 ceph-mon[29180]: pgmap v13488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:41.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24142.354143 secs 2024-04-27T01:17:41.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:41 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 24143.163885 secs 2024-04-27T01:17:41.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:41 smithi167 ceph-mon[32751]: slow request 1443.859172 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:17:41.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24142.354143 secs 2024-04-27T01:17:41.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:41 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 24143.163885 secs 2024-04-27T01:17:41.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:41 smithi090 ceph-mon[29180]: slow request 1443.859172 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:17:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24142.354143 secs 2024-04-27T01:17:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:41 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 24143.163885 secs 2024-04-27T01:17:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:41 smithi119 ceph-mon[32752]: slow request 1443.859172 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:17:42.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:42 smithi167 ceph-mon[32751]: pgmap v13489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:42.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:42 smithi090 ceph-mon[29180]: pgmap v13489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:42.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:42 smithi119 ceph-mon[32752]: pgmap v13489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:44.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:44 smithi119 ceph-mon[32752]: pgmap v13490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T01:17:44.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:44 smithi167 ceph-mon[32751]: pgmap v13490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T01:17:44.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:44 smithi090 ceph-mon[29180]: pgmap v13490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T01:17:46.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24147.354226 secs 2024-04-27T01:17:46.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:46 smithi119 ceph-mon[32752]: pgmap v13491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:17:46.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24148.164033 secs 2024-04-27T01:17:46.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24147.354226 secs 2024-04-27T01:17:46.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:46 smithi167 ceph-mon[32751]: pgmap v13491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:17:46.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24148.164033 secs 2024-04-27T01:17:46.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24147.354226 secs 2024-04-27T01:17:46.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:46 smithi090 ceph-mon[29180]: pgmap v13491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:17:46.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24148.164033 secs 2024-04-27T01:17:48.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:48 smithi119 ceph-mon[32752]: pgmap v13492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:48.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:48 smithi167 ceph-mon[32751]: pgmap v13492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:48.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:48 smithi090 ceph-mon[29180]: pgmap v13492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:50.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:50 smithi167 ceph-mon[32751]: pgmap v13493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:50.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:50 smithi090 ceph-mon[29180]: pgmap v13493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:50.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:50 smithi119 ceph-mon[32752]: pgmap v13493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:51.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24152.354311 secs 2024-04-27T01:17:51.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24153.164121 secs 2024-04-27T01:17:51.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24152.354311 secs 2024-04-27T01:17:51.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24153.164121 secs 2024-04-27T01:17:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24152.354311 secs 2024-04-27T01:17:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24153.164121 secs 2024-04-27T01:17:52.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:52 smithi167 ceph-mon[32751]: pgmap v13494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:52.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:52 smithi090 ceph-mon[29180]: pgmap v13494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:52.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:52 smithi119 ceph-mon[32752]: pgmap v13494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:54.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:54 smithi167 ceph-mon[32751]: pgmap v13495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:54.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:54 smithi090 ceph-mon[29180]: pgmap v13495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:54.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:54 smithi119 ceph-mon[32752]: pgmap v13495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:56.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24157.354393 secs 2024-04-27T01:17:56.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:56 smithi090 ceph-mon[29180]: pgmap v13496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:56.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24158.164194 secs 2024-04-27T01:17:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24157.354393 secs 2024-04-27T01:17:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:56 smithi119 ceph-mon[32752]: pgmap v13496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24158.164194 secs 2024-04-27T01:17:56.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24157.354393 secs 2024-04-27T01:17:56.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:56 smithi167 ceph-mon[32751]: pgmap v13496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:17:56.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24158.164194 secs 2024-04-27T01:17:58.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:17:58 smithi090 ceph-mon[29180]: pgmap v13497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:58.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:17:58 smithi119 ceph-mon[32752]: pgmap v13497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:17:58.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:17:58 smithi167 ceph-mon[32751]: pgmap v13497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:00.607 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:00 smithi167 ceph-mon[32751]: pgmap v13498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:00.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:00 smithi090 ceph-mon[29180]: pgmap v13498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:00.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:00 smithi119 ceph-mon[32752]: pgmap v13498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:01.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24162.354454 secs 2024-04-27T01:18:01.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24163.164304 secs 2024-04-27T01:18:01.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24162.354454 secs 2024-04-27T01:18:01.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24163.164304 secs 2024-04-27T01:18:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24162.354454 secs 2024-04-27T01:18:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24163.164304 secs 2024-04-27T01:18:02.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:02 smithi090 ceph-mon[29180]: pgmap v13499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:02.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:02 smithi119 ceph-mon[32752]: pgmap v13499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:02.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:02 smithi167 ceph-mon[32751]: pgmap v13499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:04.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:04 smithi090 ceph-mon[29180]: pgmap v13500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:04.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:04 smithi119 ceph-mon[32752]: pgmap v13500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:04.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:04 smithi167 ceph-mon[32751]: pgmap v13500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:06.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24167.354538 secs 2024-04-27T01:18:06.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:06 smithi090 ceph-mon[29180]: pgmap v13501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:06.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24168.164432 secs 2024-04-27T01:18:06.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24167.354538 secs 2024-04-27T01:18:06.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:06 smithi119 ceph-mon[32752]: pgmap v13501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:06.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24168.164432 secs 2024-04-27T01:18:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24167.354538 secs 2024-04-27T01:18:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:06 smithi167 ceph-mon[32751]: pgmap v13501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:06.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24168.164432 secs 2024-04-27T01:18:08.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:08 smithi090 ceph-mon[29180]: pgmap v13502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:08.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:08 smithi119 ceph-mon[32752]: pgmap v13502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:08.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:08 smithi167 ceph-mon[32751]: pgmap v13502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:10.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:10 smithi119 ceph-mon[32752]: pgmap v13503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:10.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:10 smithi090 ceph-mon[29180]: pgmap v13503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:10.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:10 smithi167 ceph-mon[32751]: pgmap v13503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:11.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24172.354597 secs 2024-04-27T01:18:11.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24173.164502 secs 2024-04-27T01:18:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24172.354597 secs 2024-04-27T01:18:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24173.164502 secs 2024-04-27T01:18:11.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24172.354597 secs 2024-04-27T01:18:11.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24173.164502 secs 2024-04-27T01:18:12.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:12 smithi090 ceph-mon[29180]: pgmap v13504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:12.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:12 smithi119 ceph-mon[32752]: pgmap v13504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:12.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:12 smithi167 ceph-mon[32751]: pgmap v13504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:14.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:14 smithi090 ceph-mon[29180]: pgmap v13505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:14.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:14 smithi119 ceph-mon[32752]: pgmap v13505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:14.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:14 smithi167 ceph-mon[32751]: pgmap v13505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24177.354656 secs 2024-04-27T01:18:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:16 smithi090 ceph-mon[29180]: pgmap v13506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:16.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24178.164582 secs 2024-04-27T01:18:16.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24177.354656 secs 2024-04-27T01:18:16.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:16 smithi119 ceph-mon[32752]: pgmap v13506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:16.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24178.164582 secs 2024-04-27T01:18:16.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24177.354656 secs 2024-04-27T01:18:16.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:16 smithi167 ceph-mon[32751]: pgmap v13506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:16.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24178.164582 secs 2024-04-27T01:18:18.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:18 smithi090 ceph-mon[29180]: pgmap v13507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:18.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:18 smithi119 ceph-mon[32752]: pgmap v13507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:18.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:18 smithi167 ceph-mon[32751]: pgmap v13507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:20.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:20 smithi090 ceph-mon[29180]: pgmap v13508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:20.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:20 smithi119 ceph-mon[32752]: pgmap v13508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:20.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:20 smithi167 ceph-mon[32751]: pgmap v13508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:21.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24182.354739 secs 2024-04-27T01:18:21.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24183.164680 secs 2024-04-27T01:18:21.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24182.354739 secs 2024-04-27T01:18:21.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24183.164680 secs 2024-04-27T01:18:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24182.354739 secs 2024-04-27T01:18:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24183.164680 secs 2024-04-27T01:18:22.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:22 smithi090 ceph-mon[29180]: pgmap v13509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:22.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:22 smithi119 ceph-mon[32752]: pgmap v13509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:22.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:22 smithi167 ceph-mon[32751]: pgmap v13509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:24.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:24 smithi090 ceph-mon[29180]: pgmap v13510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:24.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:24 smithi119 ceph-mon[32752]: pgmap v13510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:24.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:24 smithi167 ceph-mon[32751]: pgmap v13510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:26.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24187.354822 secs 2024-04-27T01:18:26.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:26 smithi119 ceph-mon[32752]: pgmap v13511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:26.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24188.164796 secs 2024-04-27T01:18:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24187.354822 secs 2024-04-27T01:18:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:26 smithi167 ceph-mon[32751]: pgmap v13511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24188.164796 secs 2024-04-27T01:18:26.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24187.354822 secs 2024-04-27T01:18:26.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:26 smithi090 ceph-mon[29180]: pgmap v13511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:26.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24188.164796 secs 2024-04-27T01:18:28.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:28 smithi119 ceph-mon[32752]: pgmap v13512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:28.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:28 smithi167 ceph-mon[32751]: pgmap v13512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:28.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:28 smithi090 ceph-mon[29180]: pgmap v13512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:30.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:30 smithi119 ceph-mon[32752]: pgmap v13513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:30.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:30 smithi167 ceph-mon[32751]: pgmap v13513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:30.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:30 smithi090 ceph-mon[29180]: pgmap v13513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:31.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24192.354885 secs 2024-04-27T01:18:31.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24193.164857 secs 2024-04-27T01:18:31.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24192.354885 secs 2024-04-27T01:18:31.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24193.164857 secs 2024-04-27T01:18:31.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24192.354885 secs 2024-04-27T01:18:31.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24193.164857 secs 2024-04-27T01:18:32.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:32 smithi119 ceph-mon[32752]: pgmap v13514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:32.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:32 smithi167 ceph-mon[32751]: pgmap v13514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:32.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:32 smithi090 ceph-mon[29180]: pgmap v13514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:34.665 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:34 smithi090 ceph-mon[29180]: pgmap v13515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:34.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:34 smithi119 ceph-mon[32752]: pgmap v13515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:34.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:34 smithi167 ceph-mon[32751]: pgmap v13515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:36.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24197.354956 secs 2024-04-27T01:18:36.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:36 smithi119 ceph-mon[32752]: pgmap v13516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:36.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:18:36.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24198.164923 secs 2024-04-27T01:18:36.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:18:36.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:18:36.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:18:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24197.354956 secs 2024-04-27T01:18:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:36 smithi167 ceph-mon[32751]: pgmap v13516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:18:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24198.164923 secs 2024-04-27T01:18:36.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:18:36.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:18:36.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:18:36.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24197.354956 secs 2024-04-27T01:18:36.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:36 smithi090 ceph-mon[29180]: pgmap v13516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:36.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:18:36.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24198.164923 secs 2024-04-27T01:18:36.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:18:36.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:18:36.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:18:37.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:37 smithi119 ceph-mon[32752]: pgmap v13517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T01:18:37.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:37 smithi119 ceph-mon[32752]: pgmap v13518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:37.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:37 smithi167 ceph-mon[32751]: pgmap v13517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T01:18:37.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:37 smithi167 ceph-mon[32751]: pgmap v13518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:37.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:37 smithi090 ceph-mon[29180]: pgmap v13517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T01:18:37.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:37 smithi090 ceph-mon[29180]: pgmap v13518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:39.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:39 smithi119 ceph-mon[32752]: pgmap v13519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T01:18:39.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:39 smithi167 ceph-mon[32751]: pgmap v13519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T01:18:39.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:39 smithi090 ceph-mon[29180]: pgmap v13519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T01:18:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:41 smithi119 ceph-mon[32752]: pgmap v13520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:41 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24202.355017 secs 2024-04-27T01:18:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24203.165016 secs 2024-04-27T01:18:41.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:41 smithi167 ceph-mon[32751]: pgmap v13520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:41.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:41 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24202.355017 secs 2024-04-27T01:18:41.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24203.165016 secs 2024-04-27T01:18:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:41 smithi090 ceph-mon[29180]: pgmap v13520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:41 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24202.355017 secs 2024-04-27T01:18:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24203.165016 secs 2024-04-27T01:18:43.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:43 smithi119 ceph-mon[32752]: pgmap v13521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:43.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:43 smithi167 ceph-mon[32751]: pgmap v13521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:43.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:43 smithi090 ceph-mon[29180]: pgmap v13521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:45.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:45 smithi119 ceph-mon[32752]: pgmap v13522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:45.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:45 smithi167 ceph-mon[32751]: pgmap v13522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:45.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:45 smithi090 ceph-mon[29180]: pgmap v13522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:18:46.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:46 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24207.355098 secs 2024-04-27T01:18:46.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24208.165132 secs 2024-04-27T01:18:46.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:46 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24207.355098 secs 2024-04-27T01:18:46.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24208.165132 secs 2024-04-27T01:18:46.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:46 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24207.355098 secs 2024-04-27T01:18:46.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24208.165132 secs 2024-04-27T01:18:47.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:47 smithi119 ceph-mon[32752]: pgmap v13523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:18:47.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:47 smithi167 ceph-mon[32751]: pgmap v13523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:18:47.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:47 smithi090 ceph-mon[29180]: pgmap v13523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:18:49.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:49 smithi119 ceph-mon[32752]: pgmap v13524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:49.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:49 smithi167 ceph-mon[32751]: pgmap v13524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:49.816 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:49 smithi090 ceph-mon[29180]: pgmap v13524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:51 smithi119 ceph-mon[32752]: pgmap v13525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24212.355180 secs 2024-04-27T01:18:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24213.165201 secs 2024-04-27T01:18:51.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:51 smithi167 ceph-mon[32751]: pgmap v13525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24212.355180 secs 2024-04-27T01:18:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24213.165201 secs 2024-04-27T01:18:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:51 smithi090 ceph-mon[29180]: pgmap v13525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24212.355180 secs 2024-04-27T01:18:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24213.165201 secs 2024-04-27T01:18:53.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:53 smithi119 ceph-mon[32752]: pgmap v13526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:53.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:53 smithi167 ceph-mon[32751]: pgmap v13526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:53.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:53 smithi090 ceph-mon[29180]: pgmap v13526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:55.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:55 smithi119 ceph-mon[32752]: pgmap v13527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:55.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:55 smithi167 ceph-mon[32751]: pgmap v13527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:55.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:55 smithi090 ceph-mon[29180]: pgmap v13527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:18:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24217.355261 secs 2024-04-27T01:18:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24218.165267 secs 2024-04-27T01:18:56.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24217.355261 secs 2024-04-27T01:18:56.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24218.165267 secs 2024-04-27T01:18:56.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24217.355261 secs 2024-04-27T01:18:56.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24218.165267 secs 2024-04-27T01:18:57.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:57 smithi119 ceph-mon[32752]: pgmap v13528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:57.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:57 smithi167 ceph-mon[32751]: pgmap v13528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:57.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:57 smithi090 ceph-mon[29180]: pgmap v13528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:59.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:18:59 smithi119 ceph-mon[32752]: pgmap v13529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:59.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:18:59 smithi167 ceph-mon[32751]: pgmap v13529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:18:59.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:18:59 smithi090 ceph-mon[29180]: pgmap v13529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:01.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:01 smithi119 ceph-mon[32752]: pgmap v13530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:01.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24222.355320 secs 2024-04-27T01:19:01.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24223.165344 secs 2024-04-27T01:19:01.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:01 smithi167 ceph-mon[32751]: pgmap v13530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24222.355320 secs 2024-04-27T01:19:01.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24223.165344 secs 2024-04-27T01:19:01.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:01 smithi090 ceph-mon[29180]: pgmap v13530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:01.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24222.355320 secs 2024-04-27T01:19:01.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24223.165344 secs 2024-04-27T01:19:03.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:03 smithi119 ceph-mon[32752]: pgmap v13531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:03.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:03 smithi167 ceph-mon[32751]: pgmap v13531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:03.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:03 smithi090 ceph-mon[29180]: pgmap v13531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:05.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:05 smithi119 ceph-mon[32752]: pgmap v13532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:05.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:05 smithi167 ceph-mon[32751]: pgmap v13532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:05.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:05 smithi090 ceph-mon[29180]: pgmap v13532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:06.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24227.355401 secs 2024-04-27T01:19:06.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24228.165460 secs 2024-04-27T01:19:06.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24227.355401 secs 2024-04-27T01:19:06.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24228.165460 secs 2024-04-27T01:19:06.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24227.355401 secs 2024-04-27T01:19:06.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24228.165460 secs 2024-04-27T01:19:07.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:07 smithi119 ceph-mon[32752]: pgmap v13533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:07.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:07 smithi167 ceph-mon[32751]: pgmap v13533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:07.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:07 smithi090 ceph-mon[29180]: pgmap v13533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:09.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:09 smithi119 ceph-mon[32752]: pgmap v13534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:09.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:09 smithi167 ceph-mon[32751]: pgmap v13534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:09.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:09 smithi090 ceph-mon[29180]: pgmap v13534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:11.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:11 smithi119 ceph-mon[32752]: pgmap v13535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:11 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24232.355484 secs 2024-04-27T01:19:11.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24233.165570 secs 2024-04-27T01:19:11.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:11 smithi167 ceph-mon[32751]: pgmap v13535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:11.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:11 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24232.355484 secs 2024-04-27T01:19:11.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24233.165570 secs 2024-04-27T01:19:11.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:11 smithi090 ceph-mon[29180]: pgmap v13535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:11.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:11 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24232.355484 secs 2024-04-27T01:19:11.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24233.165570 secs 2024-04-27T01:19:13.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:13 smithi119 ceph-mon[32752]: pgmap v13536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:13.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:13 smithi167 ceph-mon[32751]: pgmap v13536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:13.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:13 smithi090 ceph-mon[29180]: pgmap v13536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:15.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:15 smithi119 ceph-mon[32752]: pgmap v13537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:15.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:15 smithi167 ceph-mon[32751]: pgmap v13537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:15.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:15 smithi090 ceph-mon[29180]: pgmap v13537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:16.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24237.355539 secs 2024-04-27T01:19:16.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24238.165663 secs 2024-04-27T01:19:16.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24237.355539 secs 2024-04-27T01:19:16.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24238.165663 secs 2024-04-27T01:19:16.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24237.355539 secs 2024-04-27T01:19:16.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24238.165663 secs 2024-04-27T01:19:17.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:17 smithi119 ceph-mon[32752]: pgmap v13538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:17.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:17 smithi167 ceph-mon[32751]: pgmap v13538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:17.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:17 smithi090 ceph-mon[29180]: pgmap v13538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:19.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:19 smithi119 ceph-mon[32752]: pgmap v13539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:19.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:19 smithi167 ceph-mon[32751]: pgmap v13539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:19.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:19 smithi090 ceph-mon[29180]: pgmap v13539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:21.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:21 smithi119 ceph-mon[32752]: pgmap v13540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:21.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24242.355603 secs 2024-04-27T01:19:21.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24243.165782 secs 2024-04-27T01:19:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:21 smithi167 ceph-mon[32751]: pgmap v13540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24242.355603 secs 2024-04-27T01:19:21.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24243.165782 secs 2024-04-27T01:19:21.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:21 smithi090 ceph-mon[29180]: pgmap v13540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:21.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24242.355603 secs 2024-04-27T01:19:21.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24243.165782 secs 2024-04-27T01:19:23.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:23 smithi119 ceph-mon[32752]: pgmap v13541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:23.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:23 smithi167 ceph-mon[32751]: pgmap v13541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:23.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:23 smithi090 ceph-mon[29180]: pgmap v13541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:25.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:25 smithi119 ceph-mon[32752]: pgmap v13542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:25.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:25 smithi167 ceph-mon[32751]: pgmap v13542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:25.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:25 smithi090 ceph-mon[29180]: pgmap v13542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:26.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 24247.355686 secs 2024-04-27T01:19:26.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24248.165855 secs 2024-04-27T01:19:26.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 24247.355686 secs 2024-04-27T01:19:26.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24248.165855 secs 2024-04-27T01:19:26.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 24247.355686 secs 2024-04-27T01:19:26.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24248.165855 secs 2024-04-27T01:19:27.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:27 smithi119 ceph-mon[32752]: pgmap v13543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:27.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:27 smithi167 ceph-mon[32751]: pgmap v13543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:27.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:27 smithi090 ceph-mon[29180]: pgmap v13543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:29.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:29 smithi119 ceph-mon[32752]: pgmap v13544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:29.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:29 smithi167 ceph-mon[32751]: pgmap v13544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:29.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:29 smithi090 ceph-mon[29180]: pgmap v13544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:31.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:31 smithi119 ceph-mon[32752]: pgmap v13545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:31.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:31 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 24252.355750 secs 2024-04-27T01:19:31.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:31 smithi119 ceph-mon[32752]: slow request 180.457435 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:19:31.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24253.165960 secs 2024-04-27T01:19:31.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:31 smithi167 ceph-mon[32751]: pgmap v13545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:31.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:31 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 24252.355750 secs 2024-04-27T01:19:31.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:31 smithi167 ceph-mon[32751]: slow request 180.457435 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:19:31.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24253.165960 secs 2024-04-27T01:19:31.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:31 smithi090 ceph-mon[29180]: pgmap v13545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:31.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:31 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 24252.355750 secs 2024-04-27T01:19:31.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:31 smithi090 ceph-mon[29180]: slow request 180.457435 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:19:31.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24253.165960 secs 2024-04-27T01:19:33.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:33 smithi119 ceph-mon[32752]: pgmap v13546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:33.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:33 smithi167 ceph-mon[32751]: pgmap v13546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:33.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:33 smithi090 ceph-mon[29180]: pgmap v13546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:35.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:35 smithi119 ceph-mon[32752]: pgmap v13547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:35.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:35 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:19:35.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:35 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:19:35.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:35 smithi167 ceph-mon[32751]: pgmap v13547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:35.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:35 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:19:35.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:35 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:19:35.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:35 smithi090 ceph-mon[29180]: pgmap v13547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:35.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:35 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:19:35.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:35 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:19:36.764 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:36 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24257.355828 secs 2024-04-27T01:19:36.764 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24258.166070 secs 2024-04-27T01:19:36.764 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:19:36.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:36 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24257.355828 secs 2024-04-27T01:19:36.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24258.166070 secs 2024-04-27T01:19:36.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:19:36.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:36 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24257.355828 secs 2024-04-27T01:19:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24258.166070 secs 2024-04-27T01:19:36.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:19:37.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:37 smithi119 ceph-mon[32752]: pgmap v13548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:37.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:19:37.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:19:37.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:19:37.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:37 smithi167 ceph-mon[32751]: pgmap v13548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:37.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:19:37.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:19:37.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:19:37.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:37 smithi090 ceph-mon[29180]: pgmap v13548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:37.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:19:37.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:19:37.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:19:38.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:38 smithi119 ceph-mon[32752]: pgmap v13549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:19:38.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:38 smithi119 ceph-mon[32752]: pgmap v13550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:38.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:38 smithi167 ceph-mon[32751]: pgmap v13549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:19:38.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:38 smithi167 ceph-mon[32751]: pgmap v13550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:38.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:38 smithi090 ceph-mon[29180]: pgmap v13549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:19:38.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:38 smithi090 ceph-mon[29180]: pgmap v13550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:40.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:40 smithi119 ceph-mon[32752]: pgmap v13551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:40.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:40 smithi167 ceph-mon[32751]: pgmap v13551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:40.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:40 smithi090 ceph-mon[29180]: pgmap v13551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:41.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:41 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24262.355916 secs 2024-04-27T01:19:41.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24263.166216 secs 2024-04-27T01:19:41.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:41 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24262.355916 secs 2024-04-27T01:19:41.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24263.166216 secs 2024-04-27T01:19:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:41 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24262.355916 secs 2024-04-27T01:19:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24263.166216 secs 2024-04-27T01:19:42.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:42 smithi119 ceph-mon[32752]: pgmap v13552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:42.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:42 smithi167 ceph-mon[32751]: pgmap v13552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:42.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:42 smithi090 ceph-mon[29180]: pgmap v13552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:44.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:44 smithi119 ceph-mon[32752]: pgmap v13553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:19:44.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:44 smithi167 ceph-mon[32751]: pgmap v13553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:19:44.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:44 smithi090 ceph-mon[29180]: pgmap v13553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:19:46.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:46 smithi167 ceph-mon[32751]: pgmap v13554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:46.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:46 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24267.355992 secs 2024-04-27T01:19:46.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24268.166327 secs 2024-04-27T01:19:46.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:46 smithi090 ceph-mon[29180]: pgmap v13554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:46.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:46 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24267.355992 secs 2024-04-27T01:19:46.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24268.166327 secs 2024-04-27T01:19:47.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:46 smithi119 ceph-mon[32752]: pgmap v13554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:19:47.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:46 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24267.355992 secs 2024-04-27T01:19:47.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24268.166327 secs 2024-04-27T01:19:48.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:48 smithi167 ceph-mon[32751]: pgmap v13555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:19:48.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:48 smithi090 ceph-mon[29180]: pgmap v13555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:19:49.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:48 smithi119 ceph-mon[32752]: pgmap v13555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:19:50.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:50 smithi167 ceph-mon[32751]: pgmap v13556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:50.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:50 smithi090 ceph-mon[29180]: pgmap v13556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:51.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:50 smithi119 ceph-mon[32752]: pgmap v13556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:51.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:51 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24272.356078 secs 2024-04-27T01:19:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24273.166396 secs 2024-04-27T01:19:52.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:51 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24272.356078 secs 2024-04-27T01:19:52.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24273.166396 secs 2024-04-27T01:19:52.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:51 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24272.356078 secs 2024-04-27T01:19:52.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24273.166396 secs 2024-04-27T01:19:52.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:52 smithi090 ceph-mon[29180]: pgmap v13557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:53.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:52 smithi119 ceph-mon[32752]: pgmap v13557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:53.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:52 smithi167 ceph-mon[32751]: pgmap v13557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:54.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:54 smithi090 ceph-mon[29180]: pgmap v13558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:55.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:54 smithi119 ceph-mon[32752]: pgmap v13558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:55.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:54 smithi167 ceph-mon[32751]: pgmap v13558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:55.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:55 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24277.356159 secs 2024-04-27T01:19:56.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:55 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24277.356159 secs 2024-04-27T01:19:56.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:55 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24277.356159 secs 2024-04-27T01:19:56.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:56 smithi090 ceph-mon[29180]: pgmap v13559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:56.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24278.166516 secs 2024-04-27T01:19:57.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:56 smithi119 ceph-mon[32752]: pgmap v13559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:57.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24278.166516 secs 2024-04-27T01:19:57.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:56 smithi167 ceph-mon[32751]: pgmap v13559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:19:57.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24278.166516 secs 2024-04-27T01:19:58.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:19:58 smithi090 ceph-mon[29180]: pgmap v13560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:59.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:19:58 smithi119 ceph-mon[32752]: pgmap v13560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:19:59.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:19:58 smithi167 ceph-mon[32751]: pgmap v13560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:00.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:00 smithi090 ceph-mon[29180]: pgmap v13561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:00.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:20:00.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:00 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24282.356299 secs 2024-04-27T01:20:01.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:00 smithi119 ceph-mon[32752]: pgmap v13561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:01.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:20:01.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:00 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24282.356299 secs 2024-04-27T01:20:01.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:00 smithi167 ceph-mon[32751]: pgmap v13561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:01.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:20:01.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:00 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24282.356299 secs 2024-04-27T01:20:01.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24283.166587 secs 2024-04-27T01:20:01.956 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24283.166587 secs 2024-04-27T01:20:02.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24283.166587 secs 2024-04-27T01:20:02.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:02 smithi090 ceph-mon[29180]: pgmap v13562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:03.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:02 smithi119 ceph-mon[32752]: pgmap v13562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:03.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:02 smithi167 ceph-mon[32751]: pgmap v13562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:04.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:04 smithi119 ceph-mon[32752]: pgmap v13563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:04.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:04 smithi090 ceph-mon[29180]: pgmap v13563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:05.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:04 smithi167 ceph-mon[32751]: pgmap v13563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:05.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:05 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24287.356381 secs 2024-04-27T01:20:06.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:05 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24287.356381 secs 2024-04-27T01:20:06.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:05 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24287.356381 secs 2024-04-27T01:20:06.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:06 smithi090 ceph-mon[29180]: pgmap v13564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:06.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24288.166696 secs 2024-04-27T01:20:07.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:06 smithi119 ceph-mon[32752]: pgmap v13564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:07.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24288.166696 secs 2024-04-27T01:20:07.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:06 smithi167 ceph-mon[32751]: pgmap v13564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:07.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24288.166696 secs 2024-04-27T01:20:07.764 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:20:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:20:07.463+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:20:07.764 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:20:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:20:07.463+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:20:07.765 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:20:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:20:07.463+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:20:07.765 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:20:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:20:07.463+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:20:07.765 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:20:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:20:07.463+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:20:08.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:08 smithi090 ceph-mon[29180]: pgmap v13565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:09.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:08 smithi119 ceph-mon[32752]: pgmap v13565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:09.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:08 smithi167 ceph-mon[32751]: pgmap v13565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:10.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:10 smithi090 ceph-mon[29180]: pgmap v13566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:10.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:10 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24292.356464 secs 2024-04-27T01:20:11.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:10 smithi119 ceph-mon[32752]: pgmap v13566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:11.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:10 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24292.356464 secs 2024-04-27T01:20:11.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:10 smithi167 ceph-mon[32751]: pgmap v13566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:11.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:10 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24292.356464 secs 2024-04-27T01:20:11.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24293.166805 secs 2024-04-27T01:20:11.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24293.166805 secs 2024-04-27T01:20:12.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24293.166805 secs 2024-04-27T01:20:12.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:12 smithi090 ceph-mon[29180]: pgmap v13567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:13.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:12 smithi119 ceph-mon[32752]: pgmap v13567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:13.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:12 smithi167 ceph-mon[32751]: pgmap v13567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:14.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:14 smithi090 ceph-mon[29180]: pgmap v13568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:15.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:14 smithi119 ceph-mon[32752]: pgmap v13568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:15.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:14 smithi167 ceph-mon[32751]: pgmap v13568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:16.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:15 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24297.356547 secs 2024-04-27T01:20:16.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:15 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24297.356547 secs 2024-04-27T01:20:16.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:15 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24297.356547 secs 2024-04-27T01:20:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:16 smithi119 ceph-mon[32752]: pgmap v13569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24298.166917 secs 2024-04-27T01:20:17.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:16 smithi167 ceph-mon[32751]: pgmap v13569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:17.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24298.166917 secs 2024-04-27T01:20:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:16 smithi090 ceph-mon[29180]: pgmap v13569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:17.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24298.166917 secs 2024-04-27T01:20:19.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:18 smithi119 ceph-mon[32752]: pgmap v13570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:19.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:18 smithi167 ceph-mon[32751]: pgmap v13570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:19.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:18 smithi090 ceph-mon[29180]: pgmap v13570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:21.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:20 smithi119 ceph-mon[32752]: pgmap v13571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:21.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:20 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24302.356647 secs 2024-04-27T01:20:21.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:20 smithi167 ceph-mon[32751]: pgmap v13571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:21.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:20 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24302.356647 secs 2024-04-27T01:20:21.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:20 smithi090 ceph-mon[29180]: pgmap v13571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:21.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:20 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24302.356647 secs 2024-04-27T01:20:22.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24303.167060 secs 2024-04-27T01:20:22.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24303.167060 secs 2024-04-27T01:20:22.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24303.167060 secs 2024-04-27T01:20:23.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:22 smithi119 ceph-mon[32752]: pgmap v13572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:23.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:22 smithi167 ceph-mon[32751]: pgmap v13572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:23.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:22 smithi090 ceph-mon[29180]: pgmap v13572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:25.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:24 smithi119 ceph-mon[32752]: pgmap v13573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:25.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:24 smithi167 ceph-mon[32751]: pgmap v13573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:25.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:24 smithi090 ceph-mon[29180]: pgmap v13573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:26.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:25 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24307.356732 secs 2024-04-27T01:20:26.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:25 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24307.356732 secs 2024-04-27T01:20:26.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:25 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24307.356732 secs 2024-04-27T01:20:27.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:26 smithi119 ceph-mon[32752]: pgmap v13574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:27.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24308.167177 secs 2024-04-27T01:20:27.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:26 smithi167 ceph-mon[32751]: pgmap v13574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:27.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24308.167177 secs 2024-04-27T01:20:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:26 smithi090 ceph-mon[29180]: pgmap v13574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:27.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24308.167177 secs 2024-04-27T01:20:29.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:28 smithi119 ceph-mon[32752]: pgmap v13575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:29.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:28 smithi167 ceph-mon[32751]: pgmap v13575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:29.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:28 smithi090 ceph-mon[29180]: pgmap v13575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:31.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:30 smithi119 ceph-mon[32752]: pgmap v13576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:31.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:30 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 24312.356814 secs 2024-04-27T01:20:31.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:30 smithi119 ceph-mon[32752]: slow request 5763.860923 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:20:31.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:30 smithi167 ceph-mon[32751]: pgmap v13576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:31.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:30 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 24312.356814 secs 2024-04-27T01:20:31.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:30 smithi167 ceph-mon[32751]: slow request 5763.860923 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:20:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:30 smithi090 ceph-mon[29180]: pgmap v13576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:30 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 24312.356814 secs 2024-04-27T01:20:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:30 smithi090 ceph-mon[29180]: slow request 5763.860923 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:20:31.956 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24313.167263 secs 2024-04-27T01:20:32.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24313.167263 secs 2024-04-27T01:20:32.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24313.167263 secs 2024-04-27T01:20:33.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:32 smithi119 ceph-mon[32752]: pgmap v13577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:33.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:32 smithi167 ceph-mon[32751]: pgmap v13577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:33.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:32 smithi090 ceph-mon[29180]: pgmap v13577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:35.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:34 smithi119 ceph-mon[32752]: pgmap v13578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:35.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:34 smithi167 ceph-mon[32751]: pgmap v13578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:35.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:34 smithi090 ceph-mon[29180]: pgmap v13578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:36.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:35 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24317.356900 secs 2024-04-27T01:20:36.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:35 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24317.356900 secs 2024-04-27T01:20:36.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:35 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24317.356900 secs 2024-04-27T01:20:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:36 smithi119 ceph-mon[32752]: pgmap v13579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:37.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:36 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 24318.167408 secs 2024-04-27T01:20:37.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:36 smithi119 ceph-mon[32752]: slow request 23044.804272 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:20:37.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:36 smithi167 ceph-mon[32751]: pgmap v13579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:37.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:36 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 24318.167408 secs 2024-04-27T01:20:37.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:36 smithi167 ceph-mon[32751]: slow request 23044.804272 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:20:37.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:36 smithi090 ceph-mon[29180]: pgmap v13579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:37.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:36 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 24318.167408 secs 2024-04-27T01:20:37.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:36 smithi090 ceph-mon[29180]: slow request 23044.804272 seconds old, received at 2024-04-26T18:56:31.113781+0000: client_request(client.26384:1 getattr pAsLsXsFs #0x1 2024-04-26T18:56:31.109375+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:20:38.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:20:38.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:20:38.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:20:38.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:20:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:20:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:20:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:20:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:20:38.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:20:38.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:20:38.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:20:38.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:20:39.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:38 smithi090 ceph-mon[29180]: pgmap v13580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:39.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:38 smithi090 ceph-mon[29180]: pgmap v13581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:20:39.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:38 smithi090 ceph-mon[29180]: pgmap v13582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:20:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:38 smithi119 ceph-mon[32752]: pgmap v13580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:38 smithi119 ceph-mon[32752]: pgmap v13581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:20:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:38 smithi119 ceph-mon[32752]: pgmap v13582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:20:39.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:38 smithi167 ceph-mon[32751]: pgmap v13580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:39.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:38 smithi167 ceph-mon[32751]: pgmap v13581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:20:39.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:38 smithi167 ceph-mon[32751]: pgmap v13582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:20:41.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:40 smithi119 ceph-mon[32752]: pgmap v13583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:41.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:40 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24322.357155 secs 2024-04-27T01:20:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:40 smithi167 ceph-mon[32751]: pgmap v13583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:40 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24322.357155 secs 2024-04-27T01:20:41.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:40 smithi090 ceph-mon[29180]: pgmap v13583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:41.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:40 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24322.357155 secs 2024-04-27T01:20:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:41 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 24323.167549 secs 2024-04-27T01:20:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:41 smithi119 ceph-mon[32752]: slow request 2884.210298 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:20:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:41 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 24323.167549 secs 2024-04-27T01:20:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:41 smithi167 ceph-mon[32751]: slow request 2884.210298 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:20:42.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:41 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 24323.167549 secs 2024-04-27T01:20:42.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:41 smithi090 ceph-mon[29180]: slow request 2884.210298 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:20:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:42 smithi119 ceph-mon[32752]: pgmap v13584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:42 smithi167 ceph-mon[32751]: pgmap v13584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:42 smithi090 ceph-mon[29180]: pgmap v13584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:44.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:44 smithi119 ceph-mon[32752]: pgmap v13585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:44.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:44 smithi167 ceph-mon[32751]: pgmap v13585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:44.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:44 smithi090 ceph-mon[29180]: pgmap v13585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:45.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:45 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24327.357214 secs 2024-04-27T01:20:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:45 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24327.357214 secs 2024-04-27T01:20:46.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:45 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24327.357214 secs 2024-04-27T01:20:47.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:46 smithi119 ceph-mon[32752]: pgmap v13586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:47.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24328.167671 secs 2024-04-27T01:20:47.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:46 smithi167 ceph-mon[32751]: pgmap v13586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:47.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24328.167671 secs 2024-04-27T01:20:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:46 smithi090 ceph-mon[29180]: pgmap v13586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:20:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24328.167671 secs 2024-04-27T01:20:49.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:48 smithi119 ceph-mon[32752]: pgmap v13587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:20:49.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:48 smithi167 ceph-mon[32751]: pgmap v13587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:20:49.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:48 smithi090 ceph-mon[29180]: pgmap v13587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:20:51.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:50 smithi119 ceph-mon[32752]: pgmap v13588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:51.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:50 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24332.357302 secs 2024-04-27T01:20:51.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:50 smithi167 ceph-mon[32751]: pgmap v13588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:51.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:50 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24332.357302 secs 2024-04-27T01:20:51.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:50 smithi090 ceph-mon[29180]: pgmap v13588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:51.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:50 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24332.357302 secs 2024-04-27T01:20:52.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24333.167800 secs 2024-04-27T01:20:52.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24333.167800 secs 2024-04-27T01:20:52.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24333.167800 secs 2024-04-27T01:20:53.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:52 smithi119 ceph-mon[32752]: pgmap v13589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:53.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:52 smithi167 ceph-mon[32751]: pgmap v13589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:52 smithi090 ceph-mon[29180]: pgmap v13589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:55.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:54 smithi119 ceph-mon[32752]: pgmap v13590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:55.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:54 smithi167 ceph-mon[32751]: pgmap v13590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:55.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:54 smithi090 ceph-mon[29180]: pgmap v13590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:20:56.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:55 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24337.357390 secs 2024-04-27T01:20:56.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:55 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24337.357390 secs 2024-04-27T01:20:56.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:55 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24337.357390 secs 2024-04-27T01:20:57.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:56 smithi119 ceph-mon[32752]: pgmap v13591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:57.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24338.167912 secs 2024-04-27T01:20:57.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:56 smithi167 ceph-mon[32751]: pgmap v13591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:57.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24338.167912 secs 2024-04-27T01:20:57.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:56 smithi090 ceph-mon[29180]: pgmap v13591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:57.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24338.167912 secs 2024-04-27T01:20:59.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:20:58 smithi119 ceph-mon[32752]: pgmap v13592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:59.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:20:58 smithi167 ceph-mon[32751]: pgmap v13592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:20:59.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:20:58 smithi090 ceph-mon[29180]: pgmap v13592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:01.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:00 smithi119 ceph-mon[32752]: pgmap v13593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:01.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:00 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24342.357476 secs 2024-04-27T01:21:01.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:00 smithi167 ceph-mon[32751]: pgmap v13593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:01.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:00 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24342.357476 secs 2024-04-27T01:21:01.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:00 smithi090 ceph-mon[29180]: pgmap v13593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:01.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:00 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24342.357476 secs 2024-04-27T01:21:02.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24343.168002 secs 2024-04-27T01:21:02.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24343.168002 secs 2024-04-27T01:21:02.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24343.168002 secs 2024-04-27T01:21:03.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:02 smithi119 ceph-mon[32752]: pgmap v13594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:03.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:02 smithi167 ceph-mon[32751]: pgmap v13594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:03.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:02 smithi090 ceph-mon[29180]: pgmap v13594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:05.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:04 smithi119 ceph-mon[32752]: pgmap v13595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:05.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:04 smithi167 ceph-mon[32751]: pgmap v13595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:05.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:04 smithi090 ceph-mon[29180]: pgmap v13595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:06.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:05 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24347.357529 secs 2024-04-27T01:21:06.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:05 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24347.357529 secs 2024-04-27T01:21:06.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:05 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24347.357529 secs 2024-04-27T01:21:07.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:06 smithi119 ceph-mon[32752]: pgmap v13596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:07.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24348.168120 secs 2024-04-27T01:21:07.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:06 smithi167 ceph-mon[32751]: pgmap v13596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:07.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24348.168120 secs 2024-04-27T01:21:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:06 smithi090 ceph-mon[29180]: pgmap v13596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:07.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24348.168120 secs 2024-04-27T01:21:09.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:08 smithi119 ceph-mon[32752]: pgmap v13597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:09.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:08 smithi167 ceph-mon[32751]: pgmap v13597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:09.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:08 smithi090 ceph-mon[29180]: pgmap v13597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:21:10.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:09 smithi167 ceph-mon[32751]: pgmap v13598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:10.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:09 smithi090 ceph-mon[29180]: pgmap v13598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:10.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:09 smithi119 ceph-mon[32752]: pgmap v13598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:11.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:10 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24352.357587 secs 2024-04-27T01:21:11.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:10 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24352.357587 secs 2024-04-27T01:21:11.191 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:10 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24352.357587 secs 2024-04-27T01:21:12.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24353.168249 secs 2024-04-27T01:21:12.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:11 smithi090 ceph-mon[29180]: pgmap v13599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24353.168249 secs 2024-04-27T01:21:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:11 smithi119 ceph-mon[32752]: pgmap v13599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:12.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24353.168249 secs 2024-04-27T01:21:12.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:11 smithi167 ceph-mon[32751]: pgmap v13599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:15.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:14 smithi119 ceph-mon[32752]: pgmap v13600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:15.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:14 smithi167 ceph-mon[32751]: pgmap v13600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:15.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:14 smithi090 ceph-mon[29180]: pgmap v13600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:16.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:15 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24357.357642 secs 2024-04-27T01:21:16.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:15 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24357.357642 secs 2024-04-27T01:21:16.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:15 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24357.357642 secs 2024-04-27T01:21:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:16 smithi119 ceph-mon[32752]: pgmap v13601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24358.168400 secs 2024-04-27T01:21:17.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:16 smithi167 ceph-mon[32751]: pgmap v13601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:17.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24358.168400 secs 2024-04-27T01:21:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:16 smithi090 ceph-mon[29180]: pgmap v13601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24358.168400 secs 2024-04-27T01:21:19.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:18 smithi119 ceph-mon[32752]: pgmap v13602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:19.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:18 smithi167 ceph-mon[32751]: pgmap v13602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:19.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:18 smithi090 ceph-mon[29180]: pgmap v13602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:21.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:20 smithi119 ceph-mon[32752]: pgmap v13603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:21.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:20 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24362.357705 secs 2024-04-27T01:21:21.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:20 smithi167 ceph-mon[32751]: pgmap v13603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:21.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:20 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24362.357705 secs 2024-04-27T01:21:21.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:20 smithi090 ceph-mon[29180]: pgmap v13603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:21.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:20 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24362.357705 secs 2024-04-27T01:21:22.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24363.168515 secs 2024-04-27T01:21:22.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24363.168515 secs 2024-04-27T01:21:22.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24363.168515 secs 2024-04-27T01:21:23.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:22 smithi119 ceph-mon[32752]: pgmap v13604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:23.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:22 smithi167 ceph-mon[32751]: pgmap v13604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:23.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:22 smithi090 ceph-mon[29180]: pgmap v13604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:25.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:24 smithi119 ceph-mon[32752]: pgmap v13605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:25.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:24 smithi167 ceph-mon[32751]: pgmap v13605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:25.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:24 smithi090 ceph-mon[29180]: pgmap v13605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:26.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:25 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24367.357799 secs 2024-04-27T01:21:26.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:25 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24367.357799 secs 2024-04-27T01:21:26.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:25 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24367.357799 secs 2024-04-27T01:21:27.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:26 smithi119 ceph-mon[32752]: pgmap v13606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:27.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24368.168609 secs 2024-04-27T01:21:27.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:26 smithi167 ceph-mon[32751]: pgmap v13606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:27.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24368.168609 secs 2024-04-27T01:21:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:26 smithi090 ceph-mon[29180]: pgmap v13606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24368.168609 secs 2024-04-27T01:21:28.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:27 smithi119 ceph-mon[32752]: pgmap v13607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:28.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:27 smithi167 ceph-mon[32751]: pgmap v13607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:28.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:27 smithi090 ceph-mon[29180]: pgmap v13607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:31.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:30 smithi119 ceph-mon[32752]: pgmap v13608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:31.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:30 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24372.357826 secs 2024-04-27T01:21:31.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:30 smithi167 ceph-mon[32751]: pgmap v13608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:31.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:30 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24372.357826 secs 2024-04-27T01:21:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:30 smithi090 ceph-mon[29180]: pgmap v13608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:30 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24372.357826 secs 2024-04-27T01:21:32.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24373.168759 secs 2024-04-27T01:21:32.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24373.168759 secs 2024-04-27T01:21:32.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24373.168759 secs 2024-04-27T01:21:33.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:32 smithi119 ceph-mon[32752]: pgmap v13609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:33.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:32 smithi167 ceph-mon[32751]: pgmap v13609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:33.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:32 smithi090 ceph-mon[29180]: pgmap v13609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:35.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:34 smithi119 ceph-mon[32752]: pgmap v13610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:35.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:34 smithi167 ceph-mon[32751]: pgmap v13610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:35.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:34 smithi090 ceph-mon[29180]: pgmap v13610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:36.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:35 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24377.357915 secs 2024-04-27T01:21:36.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:35 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24377.357915 secs 2024-04-27T01:21:36.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:35 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24377.357915 secs 2024-04-27T01:21:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:36 smithi119 ceph-mon[32752]: pgmap v13611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24378.168884 secs 2024-04-27T01:21:37.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:36 smithi167 ceph-mon[32751]: pgmap v13611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:37.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24378.168884 secs 2024-04-27T01:21:37.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:36 smithi090 ceph-mon[29180]: pgmap v13611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:37.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24378.168884 secs 2024-04-27T01:21:38.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:38 smithi167 ceph-mon[32751]: pgmap v13612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:38.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:21:38.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:38.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:38.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:38.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:38 smithi119 ceph-mon[32752]: pgmap v13612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:21:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.043 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:38 smithi090 ceph-mon[29180]: pgmap v13612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:39.043 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:21:39.043 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.043 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.044 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:39.044 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: pgmap v13613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:41.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 24382.357979 secs 2024-04-27T01:21:41.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: slow request 722.824011 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:21:41.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: pgmap v13613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 24382.357979 secs 2024-04-27T01:21:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: slow request 722.824011 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:21:41.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: pgmap v13613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:41.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 24382.357979 secs 2024-04-27T01:21:41.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: slow request 722.824011 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:21:41.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:41.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24383.168950 secs 2024-04-27T01:21:42.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:21:42.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:21:42.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24383.168950 secs 2024-04-27T01:21:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:21:42.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:21:42.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24383.168950 secs 2024-04-27T01:21:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:21:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:21:42.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:21:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:21:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:21:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:21:43.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:21:43.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:21:43.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:21:43.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: pgmap v13614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:43.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:42 smithi119 ceph-mon[32752]: pgmap v13615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:21:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:21:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:21:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:21:43.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:21:43.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:21:43.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: pgmap v13614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:43.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:42 smithi167 ceph-mon[32751]: pgmap v13615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:21:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:21:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:21:43.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:21:43.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:21:43.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:21:43.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: pgmap v13614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:43.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:42 smithi090 ceph-mon[29180]: pgmap v13615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:45.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:44 smithi119 ceph-mon[32752]: pgmap v13616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:45.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:44 smithi167 ceph-mon[32751]: pgmap v13616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:45.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:44 smithi090 ceph-mon[29180]: pgmap v13616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:45 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24387.358064 secs 2024-04-27T01:21:46.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:45 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24387.358064 secs 2024-04-27T01:21:46.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:45 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24387.358064 secs 2024-04-27T01:21:47.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:46 smithi119 ceph-mon[32752]: pgmap v13617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:47.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24388.169091 secs 2024-04-27T01:21:47.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:46 smithi167 ceph-mon[32751]: pgmap v13617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:47.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24388.169091 secs 2024-04-27T01:21:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:46 smithi090 ceph-mon[29180]: pgmap v13617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24388.169091 secs 2024-04-27T01:21:48.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:47 smithi167 ceph-mon[32751]: pgmap v13618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:48.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:47 smithi090 ceph-mon[29180]: pgmap v13618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:48.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:47 smithi119 ceph-mon[32752]: pgmap v13618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:51.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:50 smithi119 ceph-mon[32752]: pgmap v13619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:51.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:50 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24392.358154 secs 2024-04-27T01:21:51.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:50 smithi167 ceph-mon[32751]: pgmap v13619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:51.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:50 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24392.358154 secs 2024-04-27T01:21:51.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:50 smithi090 ceph-mon[29180]: pgmap v13619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:51.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:50 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24392.358154 secs 2024-04-27T01:21:52.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24393.169221 secs 2024-04-27T01:21:52.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24393.169221 secs 2024-04-27T01:21:52.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24393.169221 secs 2024-04-27T01:21:53.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:52 smithi119 ceph-mon[32752]: pgmap v13620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:53.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:52 smithi167 ceph-mon[32751]: pgmap v13620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:52 smithi090 ceph-mon[29180]: pgmap v13620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:21:55.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:54 smithi119 ceph-mon[32752]: pgmap v13621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:55.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:54 smithi167 ceph-mon[32751]: pgmap v13621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:55.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:54 smithi090 ceph-mon[29180]: pgmap v13621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:21:56.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:55 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24397.358240 secs 2024-04-27T01:21:56.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:55 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24397.358240 secs 2024-04-27T01:21:56.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:55 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24397.358240 secs 2024-04-27T01:21:57.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:56 smithi119 ceph-mon[32752]: pgmap v13622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:57.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24398.169352 secs 2024-04-27T01:21:57.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:56 smithi167 ceph-mon[32751]: pgmap v13622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:57.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24398.169352 secs 2024-04-27T01:21:57.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:56 smithi090 ceph-mon[29180]: pgmap v13622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:57.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24398.169352 secs 2024-04-27T01:21:59.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:21:58 smithi119 ceph-mon[32752]: pgmap v13623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:59.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:21:58 smithi167 ceph-mon[32751]: pgmap v13623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:21:59.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:21:58 smithi090 ceph-mon[29180]: pgmap v13623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:01.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:00 smithi119 ceph-mon[32752]: pgmap v13624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:01.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:00 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24402.358341 secs 2024-04-27T01:22:01.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:00 smithi167 ceph-mon[32751]: pgmap v13624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:01.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:00 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24402.358341 secs 2024-04-27T01:22:01.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:00 smithi090 ceph-mon[29180]: pgmap v13624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:01.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:00 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24402.358341 secs 2024-04-27T01:22:02.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24403.169505 secs 2024-04-27T01:22:02.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24403.169505 secs 2024-04-27T01:22:02.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24403.169505 secs 2024-04-27T01:22:03.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:02 smithi119 ceph-mon[32752]: pgmap v13625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:03.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:02 smithi167 ceph-mon[32751]: pgmap v13625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:03.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:02 smithi090 ceph-mon[29180]: pgmap v13625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:05.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:04 smithi119 ceph-mon[32752]: pgmap v13626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:04 smithi167 ceph-mon[32751]: pgmap v13626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:05.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:04 smithi090 ceph-mon[29180]: pgmap v13626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:06.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:05 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24407.358422 secs 2024-04-27T01:22:06.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:05 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24407.358422 secs 2024-04-27T01:22:06.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:05 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24407.358422 secs 2024-04-27T01:22:07.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:06 smithi119 ceph-mon[32752]: pgmap v13627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:07.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24408.169590 secs 2024-04-27T01:22:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:06 smithi167 ceph-mon[32751]: pgmap v13627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:07.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24408.169590 secs 2024-04-27T01:22:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:06 smithi090 ceph-mon[29180]: pgmap v13627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24408.169590 secs 2024-04-27T01:22:08.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:07 smithi119 ceph-mon[32752]: pgmap v13628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:08.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:07 smithi167 ceph-mon[32751]: pgmap v13628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:08.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:07 smithi090 ceph-mon[29180]: pgmap v13628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:11.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:10 smithi119 ceph-mon[32752]: pgmap v13629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:11.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:10 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24412.358505 secs 2024-04-27T01:22:11.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:10 smithi167 ceph-mon[32751]: pgmap v13629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:11.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:10 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24412.358505 secs 2024-04-27T01:22:11.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:10 smithi090 ceph-mon[29180]: pgmap v13629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:11.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:10 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24412.358505 secs 2024-04-27T01:22:12.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24413.169726 secs 2024-04-27T01:22:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24413.169726 secs 2024-04-27T01:22:12.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24413.169726 secs 2024-04-27T01:22:13.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:12 smithi119 ceph-mon[32752]: pgmap v13630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:13.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:12 smithi167 ceph-mon[32751]: pgmap v13630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:13.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:12 smithi090 ceph-mon[29180]: pgmap v13630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:15.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:14 smithi119 ceph-mon[32752]: pgmap v13631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:15.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:14 smithi167 ceph-mon[32751]: pgmap v13631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:15.056 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:14 smithi090 ceph-mon[29180]: pgmap v13631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:16.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:15 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24417.358588 secs 2024-04-27T01:22:16.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:15 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24417.358588 secs 2024-04-27T01:22:16.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:15 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24417.358588 secs 2024-04-27T01:22:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:16 smithi119 ceph-mon[32752]: pgmap v13632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24418.169818 secs 2024-04-27T01:22:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:16 smithi167 ceph-mon[32751]: pgmap v13632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:17.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24418.169818 secs 2024-04-27T01:22:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:16 smithi090 ceph-mon[29180]: pgmap v13632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24418.169818 secs 2024-04-27T01:22:19.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:18 smithi119 ceph-mon[32752]: pgmap v13633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:19.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:18 smithi167 ceph-mon[32751]: pgmap v13633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:19.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:18 smithi090 ceph-mon[29180]: pgmap v13633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:21.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:20 smithi119 ceph-mon[32752]: pgmap v13634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:21.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:20 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24422.358674 secs 2024-04-27T01:22:21.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:20 smithi167 ceph-mon[32751]: pgmap v13634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:21.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:20 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24422.358674 secs 2024-04-27T01:22:21.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:20 smithi090 ceph-mon[29180]: pgmap v13634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:21.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:20 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24422.358674 secs 2024-04-27T01:22:22.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24423.169913 secs 2024-04-27T01:22:22.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24423.169913 secs 2024-04-27T01:22:22.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24423.169913 secs 2024-04-27T01:22:23.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:22 smithi119 ceph-mon[32752]: pgmap v13635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:23.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:22 smithi167 ceph-mon[32751]: pgmap v13635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:23.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:22 smithi090 ceph-mon[29180]: pgmap v13635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:24.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:24 smithi090 ceph-mon[29180]: pgmap v13636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:25.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:24 smithi119 ceph-mon[32752]: pgmap v13636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:25.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:24 smithi167 ceph-mon[32751]: pgmap v13636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:26.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:25 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24427.358758 secs 2024-04-27T01:22:26.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:25 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24427.358758 secs 2024-04-27T01:22:26.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:25 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24427.358758 secs 2024-04-27T01:22:27.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:26 smithi119 ceph-mon[32752]: pgmap v13637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:27.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24428.170056 secs 2024-04-27T01:22:27.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:26 smithi167 ceph-mon[32751]: pgmap v13637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:27.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24428.170056 secs 2024-04-27T01:22:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:26 smithi090 ceph-mon[29180]: pgmap v13637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24428.170056 secs 2024-04-27T01:22:28.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:27 smithi119 ceph-mon[32752]: pgmap v13638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:28.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:27 smithi167 ceph-mon[32751]: pgmap v13638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:28.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:27 smithi090 ceph-mon[29180]: pgmap v13638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:31.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:30 smithi119 ceph-mon[32752]: pgmap v13639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:31.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:30 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 24432.358849 secs 2024-04-27T01:22:31.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:30 smithi119 ceph-mon[32752]: slow request 360.460535 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:22:31.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:30 smithi167 ceph-mon[32751]: pgmap v13639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:31.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:30 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 24432.358849 secs 2024-04-27T01:22:31.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:30 smithi167 ceph-mon[32751]: slow request 360.460535 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:22:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:30 smithi090 ceph-mon[29180]: pgmap v13639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:30 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 24432.358849 secs 2024-04-27T01:22:31.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:30 smithi090 ceph-mon[29180]: slow request 360.460535 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:22:32.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24433.170151 secs 2024-04-27T01:22:32.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24433.170151 secs 2024-04-27T01:22:32.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24433.170151 secs 2024-04-27T01:22:33.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:32 smithi119 ceph-mon[32752]: pgmap v13640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:33.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:32 smithi167 ceph-mon[32751]: pgmap v13640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:33.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:32 smithi090 ceph-mon[29180]: pgmap v13640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:35.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:34 smithi119 ceph-mon[32752]: pgmap v13641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:35.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:34 smithi167 ceph-mon[32751]: pgmap v13641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:35.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:34 smithi090 ceph-mon[29180]: pgmap v13641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:36.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:35 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24437.358933 secs 2024-04-27T01:22:36.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:35 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24437.358933 secs 2024-04-27T01:22:36.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:35 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24437.358933 secs 2024-04-27T01:22:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:36 smithi119 ceph-mon[32752]: pgmap v13642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24438.170283 secs 2024-04-27T01:22:37.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:36 smithi167 ceph-mon[32751]: pgmap v13642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:37.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24438.170283 secs 2024-04-27T01:22:37.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:36 smithi090 ceph-mon[29180]: pgmap v13642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:37.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24438.170283 secs 2024-04-27T01:22:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:38 smithi119 ceph-mon[32752]: pgmap v13643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:39.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:38 smithi167 ceph-mon[32751]: pgmap v13643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:39.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:38 smithi090 ceph-mon[29180]: pgmap v13643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:41.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:40 smithi119 ceph-mon[32752]: pgmap v13644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:41.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:40 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24442.359009 secs 2024-04-27T01:22:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:40 smithi167 ceph-mon[32751]: pgmap v13644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:41.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:40 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24442.359009 secs 2024-04-27T01:22:41.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:40 smithi090 ceph-mon[29180]: pgmap v13644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:41.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:40 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24442.359009 secs 2024-04-27T01:22:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24443.170422 secs 2024-04-27T01:22:42.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24443.170422 secs 2024-04-27T01:22:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24443.170422 secs 2024-04-27T01:22:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:42 smithi119 ceph-mon[32752]: pgmap v13645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:22:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:43.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:42 smithi167 ceph-mon[32751]: pgmap v13645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:43.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:22:43.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:42 smithi090 ceph-mon[29180]: pgmap v13645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:22:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:43.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:44.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:22:44.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:22:44.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:44.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:22:44.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:22:44.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:44.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:22:44.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:22:44.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:22:45.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:44 smithi119 ceph-mon[32752]: pgmap v13646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T01:22:45.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:44 smithi119 ceph-mon[32752]: pgmap v13647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s rd, 1 op/s 2024-04-27T01:22:45.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:44 smithi167 ceph-mon[32751]: pgmap v13646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T01:22:45.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:44 smithi167 ceph-mon[32751]: pgmap v13647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s rd, 1 op/s 2024-04-27T01:22:45.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:44 smithi090 ceph-mon[29180]: pgmap v13646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T01:22:45.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:44 smithi090 ceph-mon[29180]: pgmap v13647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s rd, 1 op/s 2024-04-27T01:22:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:45 smithi119 ceph-mon[32752]: pgmap v13648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:45 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24447.359095 secs 2024-04-27T01:22:46.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:45 smithi167 ceph-mon[32751]: pgmap v13648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:46.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:45 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24447.359095 secs 2024-04-27T01:22:46.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:45 smithi090 ceph-mon[29180]: pgmap v13648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:46.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:45 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24447.359095 secs 2024-04-27T01:22:47.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24448.170517 secs 2024-04-27T01:22:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24448.170517 secs 2024-04-27T01:22:47.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24448.170517 secs 2024-04-27T01:22:48.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:47 smithi119 ceph-mon[32752]: pgmap v13649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:48.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:47 smithi167 ceph-mon[32751]: pgmap v13649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:48.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:47 smithi090 ceph-mon[29180]: pgmap v13649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:50.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:50 smithi119 ceph-mon[32752]: pgmap v13650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:50.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:50 smithi167 ceph-mon[32751]: pgmap v13650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:50.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:50 smithi090 ceph-mon[29180]: pgmap v13650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:51.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:51 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24452.359181 secs 2024-04-27T01:22:51.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24453.170635 secs 2024-04-27T01:22:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:51 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24452.359181 secs 2024-04-27T01:22:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24453.170635 secs 2024-04-27T01:22:51.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:51 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24452.359181 secs 2024-04-27T01:22:51.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24453.170635 secs 2024-04-27T01:22:52.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:52 smithi119 ceph-mon[32752]: pgmap v13651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:52.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:52 smithi167 ceph-mon[32751]: pgmap v13651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:52.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:52 smithi090 ceph-mon[29180]: pgmap v13651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:22:54.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:54 smithi119 ceph-mon[32752]: pgmap v13652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:22:54.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:54 smithi167 ceph-mon[32751]: pgmap v13652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:22:54.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:54 smithi090 ceph-mon[29180]: pgmap v13652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:22:56.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:56 smithi119 ceph-mon[32752]: pgmap v13653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:56.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:56 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24457.359282 secs 2024-04-27T01:22:56.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24458.170733 secs 2024-04-27T01:22:56.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:56 smithi167 ceph-mon[32751]: pgmap v13653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:56.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:56 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24457.359282 secs 2024-04-27T01:22:56.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24458.170733 secs 2024-04-27T01:22:56.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:56 smithi090 ceph-mon[29180]: pgmap v13653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:22:56.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:56 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24457.359282 secs 2024-04-27T01:22:56.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24458.170733 secs 2024-04-27T01:22:58.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:22:58 smithi119 ceph-mon[32752]: pgmap v13654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:58.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:22:58 smithi167 ceph-mon[32751]: pgmap v13654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:22:58.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:22:58 smithi090 ceph-mon[29180]: pgmap v13654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:00.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:00 smithi119 ceph-mon[32752]: pgmap v13655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:00.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:00 smithi167 ceph-mon[32751]: pgmap v13655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:00.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:00 smithi090 ceph-mon[29180]: pgmap v13655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:01.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:01 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24462.359371 secs 2024-04-27T01:23:01.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24463.170828 secs 2024-04-27T01:23:01.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:01 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24462.359371 secs 2024-04-27T01:23:01.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24463.170828 secs 2024-04-27T01:23:01.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:01 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24462.359371 secs 2024-04-27T01:23:01.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24463.170828 secs 2024-04-27T01:23:02.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:02 smithi167 ceph-mon[32751]: pgmap v13656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:02.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:02 smithi090 ceph-mon[29180]: pgmap v13656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:02.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:02 smithi119 ceph-mon[32752]: pgmap v13656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:04.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:04 smithi167 ceph-mon[32751]: pgmap v13657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:04.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:04 smithi090 ceph-mon[29180]: pgmap v13657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:04.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:04 smithi119 ceph-mon[32752]: pgmap v13657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:06.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:06 smithi090 ceph-mon[29180]: pgmap v13658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:06.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:06 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24467.359455 secs 2024-04-27T01:23:06.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24468.170946 secs 2024-04-27T01:23:06.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:06 smithi119 ceph-mon[32752]: pgmap v13658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:06.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:06 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24467.359455 secs 2024-04-27T01:23:06.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24468.170946 secs 2024-04-27T01:23:06.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:06 smithi167 ceph-mon[32751]: pgmap v13658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:06.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:06 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24467.359455 secs 2024-04-27T01:23:06.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24468.170946 secs 2024-04-27T01:23:08.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:08 smithi090 ceph-mon[29180]: pgmap v13659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:08.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:08 smithi119 ceph-mon[32752]: pgmap v13659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:08.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:08 smithi167 ceph-mon[32751]: pgmap v13659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:10.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:10 smithi090 ceph-mon[29180]: pgmap v13660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:10.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:10 smithi119 ceph-mon[32752]: pgmap v13660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:10.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:10 smithi167 ceph-mon[32751]: pgmap v13660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:11.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:11 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24472.359541 secs 2024-04-27T01:23:11.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24473.171058 secs 2024-04-27T01:23:11.441 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:11 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24472.359541 secs 2024-04-27T01:23:11.441 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24473.171058 secs 2024-04-27T01:23:11.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:11 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24472.359541 secs 2024-04-27T01:23:11.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24473.171058 secs 2024-04-27T01:23:12.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:12 smithi090 ceph-mon[29180]: pgmap v13661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:12.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:12 smithi119 ceph-mon[32752]: pgmap v13661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:12.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:12 smithi167 ceph-mon[32751]: pgmap v13661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:14.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:14 smithi090 ceph-mon[29180]: pgmap v13662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:14.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:14 smithi119 ceph-mon[32752]: pgmap v13662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:14.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:14 smithi167 ceph-mon[32751]: pgmap v13662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:16.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:16 smithi090 ceph-mon[29180]: pgmap v13663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:16.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:16 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24477.359625 secs 2024-04-27T01:23:16.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24478.171179 secs 2024-04-27T01:23:16.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:16 smithi119 ceph-mon[32752]: pgmap v13663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:16.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:16 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24477.359625 secs 2024-04-27T01:23:16.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24478.171179 secs 2024-04-27T01:23:16.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:16 smithi167 ceph-mon[32751]: pgmap v13663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:16.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:16 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24477.359625 secs 2024-04-27T01:23:16.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24478.171179 secs 2024-04-27T01:23:18.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:18 smithi090 ceph-mon[29180]: pgmap v13664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:18.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:18 smithi119 ceph-mon[32752]: pgmap v13664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:18.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:18 smithi167 ceph-mon[32751]: pgmap v13664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:20.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:20 smithi090 ceph-mon[29180]: pgmap v13665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:20.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:20 smithi119 ceph-mon[32752]: pgmap v13665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:20.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:20 smithi167 ceph-mon[32751]: pgmap v13665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:21.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:21 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24482.359712 secs 2024-04-27T01:23:21.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24483.171291 secs 2024-04-27T01:23:21.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:21 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24482.359712 secs 2024-04-27T01:23:21.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24483.171291 secs 2024-04-27T01:23:21.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:21 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24482.359712 secs 2024-04-27T01:23:21.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24483.171291 secs 2024-04-27T01:23:22.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:22 smithi090 ceph-mon[29180]: pgmap v13666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:22.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:22 smithi119 ceph-mon[32752]: pgmap v13666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:22.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:22 smithi167 ceph-mon[32751]: pgmap v13666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:24.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:24 smithi090 ceph-mon[29180]: pgmap v13667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:24.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:24 smithi119 ceph-mon[32752]: pgmap v13667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:24.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:24 smithi167 ceph-mon[32751]: pgmap v13667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:26.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:26 smithi090 ceph-mon[29180]: pgmap v13668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:26.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:26 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24487.359800 secs 2024-04-27T01:23:26.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24488.171406 secs 2024-04-27T01:23:26.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:26 smithi119 ceph-mon[32752]: pgmap v13668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:26.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:26 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24487.359800 secs 2024-04-27T01:23:26.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24488.171406 secs 2024-04-27T01:23:26.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:26 smithi167 ceph-mon[32751]: pgmap v13668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:26.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:26 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24487.359800 secs 2024-04-27T01:23:26.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24488.171406 secs 2024-04-27T01:23:28.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:28 smithi090 ceph-mon[29180]: pgmap v13669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:28.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:28 smithi119 ceph-mon[32752]: pgmap v13669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:28.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:28 smithi167 ceph-mon[32751]: pgmap v13669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:30.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:30 smithi090 ceph-mon[29180]: pgmap v13670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:30.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:30 smithi119 ceph-mon[32752]: pgmap v13670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:30.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:30 smithi167 ceph-mon[32751]: pgmap v13670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:31.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:31 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24492.359867 secs 2024-04-27T01:23:31.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24493.171522 secs 2024-04-27T01:23:31.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:31 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24492.359867 secs 2024-04-27T01:23:31.455 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24493.171522 secs 2024-04-27T01:23:31.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:31 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24492.359867 secs 2024-04-27T01:23:31.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24493.171522 secs 2024-04-27T01:23:32.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:32 smithi090 ceph-mon[29180]: pgmap v13671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:32.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:32 smithi119 ceph-mon[32752]: pgmap v13671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:32.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:32 smithi167 ceph-mon[32751]: pgmap v13671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:34.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:34 smithi090 ceph-mon[29180]: pgmap v13672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:34.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:34 smithi119 ceph-mon[32752]: pgmap v13672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:34.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:34 smithi167 ceph-mon[32751]: pgmap v13672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:36.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:36 smithi119 ceph-mon[32752]: pgmap v13673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:36.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:36 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24497.359955 secs 2024-04-27T01:23:36.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24498.171605 secs 2024-04-27T01:23:36.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:36 smithi167 ceph-mon[32751]: pgmap v13673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:36.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:36 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24497.359955 secs 2024-04-27T01:23:36.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24498.171605 secs 2024-04-27T01:23:36.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:36 smithi090 ceph-mon[29180]: pgmap v13673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:36.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:36 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24497.359955 secs 2024-04-27T01:23:36.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24498.171605 secs 2024-04-27T01:23:38.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:38 smithi119 ceph-mon[32752]: pgmap v13674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:38.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:38 smithi167 ceph-mon[32751]: pgmap v13674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:38.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:38 smithi090 ceph-mon[29180]: pgmap v13674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:40.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:40 smithi119 ceph-mon[32752]: pgmap v13675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:40.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:40 smithi167 ceph-mon[32751]: pgmap v13675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:40.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:40 smithi090 ceph-mon[29180]: pgmap v13675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:23:41.441 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:41 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24502.360039 secs 2024-04-27T01:23:41.441 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24503.171679 secs 2024-04-27T01:23:41.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:41 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24502.360039 secs 2024-04-27T01:23:41.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24503.171679 secs 2024-04-27T01:23:41.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:41 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24502.360039 secs 2024-04-27T01:23:41.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24503.171679 secs 2024-04-27T01:23:42.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:42 smithi119 ceph-mon[32752]: pgmap v13676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:42.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:42 smithi167 ceph-mon[32751]: pgmap v13676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:42.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:42 smithi090 ceph-mon[29180]: pgmap v13676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:23:43.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:23:43.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:23:44.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:44 smithi119 ceph-mon[32752]: pgmap v13677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:44.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:23:44.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:23:44.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:44 smithi119 ceph-mon[32752]: pgmap v13678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:23:44.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:44 smithi119 ceph-mon[32752]: pgmap v13679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:44.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:23:44.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:44 smithi167 ceph-mon[32751]: pgmap v13677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:44.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:23:44.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:23:44.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:44 smithi167 ceph-mon[32751]: pgmap v13678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:23:44.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:44 smithi167 ceph-mon[32751]: pgmap v13679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:44.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:23:44.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:44 smithi090 ceph-mon[29180]: pgmap v13677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:44.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:23:44.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:23:44.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:44 smithi090 ceph-mon[29180]: pgmap v13678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:23:44.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:44 smithi090 ceph-mon[29180]: pgmap v13679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:44.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:23:46.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:45 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24507.360130 secs 2024-04-27T01:23:46.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:45 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24507.360130 secs 2024-04-27T01:23:46.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:45 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24507.360130 secs 2024-04-27T01:23:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:46 smithi119 ceph-mon[32752]: pgmap v13680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24508.171796 secs 2024-04-27T01:23:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:46 smithi167 ceph-mon[32751]: pgmap v13680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24508.171796 secs 2024-04-27T01:23:47.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:46 smithi090 ceph-mon[29180]: pgmap v13680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24508.171796 secs 2024-04-27T01:23:48.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:47 smithi090 ceph-mon[29180]: pgmap v13681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:23:48.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:47 smithi119 ceph-mon[32752]: pgmap v13681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:23:48.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:47 smithi167 ceph-mon[32751]: pgmap v13681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:23:50.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:49 smithi090 ceph-mon[29180]: pgmap v13682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:50.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:49 smithi119 ceph-mon[32752]: pgmap v13682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:50.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:49 smithi167 ceph-mon[32751]: pgmap v13682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:51.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:50 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24512.360215 secs 2024-04-27T01:23:51.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:50 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24512.360215 secs 2024-04-27T01:23:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:50 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24512.360215 secs 2024-04-27T01:23:52.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24513.171926 secs 2024-04-27T01:23:52.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:51 smithi090 ceph-mon[29180]: pgmap v13683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:52.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24513.171926 secs 2024-04-27T01:23:52.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:51 smithi119 ceph-mon[32752]: pgmap v13683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:52.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24513.171926 secs 2024-04-27T01:23:52.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:51 smithi167 ceph-mon[32751]: pgmap v13683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:23:54.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:53 smithi090 ceph-mon[29180]: pgmap v13684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:23:54.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:53 smithi119 ceph-mon[32752]: pgmap v13684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:23:54.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:53 smithi167 ceph-mon[32751]: pgmap v13684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:23:56.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:55 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24517.360302 secs 2024-04-27T01:23:56.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:55 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24517.360302 secs 2024-04-27T01:23:56.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:55 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24517.360302 secs 2024-04-27T01:23:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:56 smithi119 ceph-mon[32752]: pgmap v13685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:57.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24518.172058 secs 2024-04-27T01:23:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:56 smithi167 ceph-mon[32751]: pgmap v13685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24518.172058 secs 2024-04-27T01:23:57.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:56 smithi090 ceph-mon[29180]: pgmap v13685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:57.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24518.172058 secs 2024-04-27T01:23:58.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:57 smithi090 ceph-mon[29180]: pgmap v13686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:58.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:57 smithi119 ceph-mon[32752]: pgmap v13686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:23:58.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:57 smithi167 ceph-mon[32751]: pgmap v13686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:00.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:23:59 smithi090 ceph-mon[29180]: pgmap v13687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:00.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:23:59 smithi119 ceph-mon[32752]: pgmap v13687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:00.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:23:59 smithi167 ceph-mon[32751]: pgmap v13687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:01.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:00 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24522.360408 secs 2024-04-27T01:24:01.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:00 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24522.360408 secs 2024-04-27T01:24:01.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:00 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24522.360408 secs 2024-04-27T01:24:02.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24523.172179 secs 2024-04-27T01:24:02.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:01 smithi090 ceph-mon[29180]: pgmap v13688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:02.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24523.172179 secs 2024-04-27T01:24:02.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:01 smithi119 ceph-mon[32752]: pgmap v13688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:02.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24523.172179 secs 2024-04-27T01:24:02.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:01 smithi167 ceph-mon[32751]: pgmap v13688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:04.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:04 smithi119 ceph-mon[32752]: pgmap v13689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:04.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:04 smithi167 ceph-mon[32751]: pgmap v13689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:04.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:04 smithi090 ceph-mon[29180]: pgmap v13689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:06.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:05 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24527.360469 secs 2024-04-27T01:24:06.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:05 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24527.360469 secs 2024-04-27T01:24:06.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:05 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24527.360469 secs 2024-04-27T01:24:07.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:06 smithi119 ceph-mon[32752]: pgmap v13690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:07.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24528.172299 secs 2024-04-27T01:24:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:06 smithi167 ceph-mon[32751]: pgmap v13690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24528.172299 secs 2024-04-27T01:24:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:06 smithi090 ceph-mon[29180]: pgmap v13690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24528.172299 secs 2024-04-27T01:24:08.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:07 smithi090 ceph-mon[29180]: pgmap v13691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:08.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:07 smithi119 ceph-mon[32752]: pgmap v13691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:08.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:07 smithi167 ceph-mon[32751]: pgmap v13691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:10.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:09 smithi090 ceph-mon[29180]: pgmap v13692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:10.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:09 smithi119 ceph-mon[32752]: pgmap v13692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:10.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:09 smithi167 ceph-mon[32751]: pgmap v13692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:11.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:10 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24532.360558 secs 2024-04-27T01:24:11.190 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:10 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24532.360558 secs 2024-04-27T01:24:11.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:10 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24532.360558 secs 2024-04-27T01:24:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24533.172413 secs 2024-04-27T01:24:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:11 smithi090 ceph-mon[29180]: pgmap v13693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:12.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24533.172413 secs 2024-04-27T01:24:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:11 smithi119 ceph-mon[32752]: pgmap v13693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:12.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24533.172413 secs 2024-04-27T01:24:12.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:11 smithi167 ceph-mon[32751]: pgmap v13693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:14.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:13 smithi090 ceph-mon[29180]: pgmap v13694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:14.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:13 smithi119 ceph-mon[32752]: pgmap v13694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:14.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:13 smithi167 ceph-mon[32751]: pgmap v13694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:16.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:15 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24537.360642 secs 2024-04-27T01:24:16.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:15 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24537.360642 secs 2024-04-27T01:24:16.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:15 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24537.360642 secs 2024-04-27T01:24:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:16 smithi119 ceph-mon[32752]: pgmap v13695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24538.172515 secs 2024-04-27T01:24:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:16 smithi167 ceph-mon[32751]: pgmap v13695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24538.172515 secs 2024-04-27T01:24:17.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:16 smithi090 ceph-mon[29180]: pgmap v13695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24538.172515 secs 2024-04-27T01:24:18.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:17 smithi090 ceph-mon[29180]: pgmap v13696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:18.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:17 smithi119 ceph-mon[32752]: pgmap v13696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:18.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:17 smithi167 ceph-mon[32751]: pgmap v13696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:20.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:19 smithi090 ceph-mon[29180]: pgmap v13697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:20.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:19 smithi119 ceph-mon[32752]: pgmap v13697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:20.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:19 smithi167 ceph-mon[32751]: pgmap v13697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:21.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:20 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24542.360721 secs 2024-04-27T01:24:21.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:20 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24542.360721 secs 2024-04-27T01:24:21.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:20 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24542.360721 secs 2024-04-27T01:24:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24543.172640 secs 2024-04-27T01:24:22.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:21 smithi090 ceph-mon[29180]: pgmap v13698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:22.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24543.172640 secs 2024-04-27T01:24:22.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:21 smithi119 ceph-mon[32752]: pgmap v13698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:22.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24543.172640 secs 2024-04-27T01:24:22.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:21 smithi167 ceph-mon[32751]: pgmap v13698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:24.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:23 smithi090 ceph-mon[29180]: pgmap v13699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:24.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:23 smithi119 ceph-mon[32752]: pgmap v13699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:24.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:23 smithi167 ceph-mon[32751]: pgmap v13699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:26.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:25 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 24547.360806 secs 2024-04-27T01:24:26.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:25 smithi119 ceph-mon[32752]: slow request 11522.326912 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:24:26.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:25 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 24547.360806 secs 2024-04-27T01:24:26.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:25 smithi167 ceph-mon[32751]: slow request 11522.326912 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:24:26.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:25 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 24547.360806 secs 2024-04-27T01:24:26.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:25 smithi090 ceph-mon[29180]: slow request 11522.326912 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:24:27.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:26 smithi119 ceph-mon[32752]: pgmap v13700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:27.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24548.172765 secs 2024-04-27T01:24:27.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:26 smithi167 ceph-mon[32751]: pgmap v13700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:27.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24548.172765 secs 2024-04-27T01:24:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:26 smithi090 ceph-mon[29180]: pgmap v13700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24548.172765 secs 2024-04-27T01:24:28.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:27 smithi090 ceph-mon[29180]: pgmap v13701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:28.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:27 smithi119 ceph-mon[32752]: pgmap v13701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:28.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:27 smithi167 ceph-mon[32751]: pgmap v13701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:30.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:29 smithi090 ceph-mon[29180]: pgmap v13702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:30.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:29 smithi119 ceph-mon[32752]: pgmap v13702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:30.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:29 smithi167 ceph-mon[32751]: pgmap v13702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:31.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:30 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24552.360893 secs 2024-04-27T01:24:31.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:30 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24552.360893 secs 2024-04-27T01:24:31.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:30 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24552.360893 secs 2024-04-27T01:24:32.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24553.172829 secs 2024-04-27T01:24:32.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:31 smithi090 ceph-mon[29180]: pgmap v13703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:32.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24553.172829 secs 2024-04-27T01:24:32.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:31 smithi119 ceph-mon[32752]: pgmap v13703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:32.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24553.172829 secs 2024-04-27T01:24:32.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:31 smithi167 ceph-mon[32751]: pgmap v13703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:34.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:33 smithi090 ceph-mon[29180]: pgmap v13704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:34.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:33 smithi119 ceph-mon[32752]: pgmap v13704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:34.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:33 smithi167 ceph-mon[32751]: pgmap v13704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:36.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:35 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24557.360972 secs 2024-04-27T01:24:36.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:35 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24557.360972 secs 2024-04-27T01:24:36.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:35 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24557.360972 secs 2024-04-27T01:24:37.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:36 smithi090 ceph-mon[29180]: pgmap v13705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:37.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24558.172947 secs 2024-04-27T01:24:37.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:36 smithi119 ceph-mon[32752]: pgmap v13705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24558.172947 secs 2024-04-27T01:24:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:36 smithi167 ceph-mon[32751]: pgmap v13705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24558.172947 secs 2024-04-27T01:24:38.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:37 smithi090 ceph-mon[29180]: pgmap v13706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:38.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:37 smithi119 ceph-mon[32752]: pgmap v13706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:38.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:37 smithi167 ceph-mon[32751]: pgmap v13706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:40.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:39 smithi090 ceph-mon[29180]: pgmap v13707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:40.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:39 smithi119 ceph-mon[32752]: pgmap v13707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:40.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:39 smithi167 ceph-mon[32751]: pgmap v13707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:41.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:40 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24562.361080 secs 2024-04-27T01:24:41.190 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:40 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24562.361080 secs 2024-04-27T01:24:41.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:40 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24562.361080 secs 2024-04-27T01:24:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24563.173065 secs 2024-04-27T01:24:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:41 smithi090 ceph-mon[29180]: pgmap v13708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:42.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24563.173065 secs 2024-04-27T01:24:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:41 smithi119 ceph-mon[32752]: pgmap v13708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:42.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24563.173065 secs 2024-04-27T01:24:42.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:41 smithi167 ceph-mon[32751]: pgmap v13708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:44.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:44 smithi119 ceph-mon[32752]: pgmap v13709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:44.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:24:44.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:44 smithi167 ceph-mon[32751]: pgmap v13709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:44.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:24:44.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:44 smithi090 ceph-mon[29180]: pgmap v13709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:24:44.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:24:45.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:24:45.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:24:45.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:45 smithi119 ceph-mon[32752]: pgmap v13710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:24:45.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:45 smithi119 ceph-mon[32752]: pgmap v13711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:45.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:24:45.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:24:45.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:24:45.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:45 smithi167 ceph-mon[32751]: pgmap v13710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:24:45.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:45 smithi167 ceph-mon[32751]: pgmap v13711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:45.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:24:45.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:24:45.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:24:45.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:45 smithi090 ceph-mon[29180]: pgmap v13710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:24:45.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:45 smithi090 ceph-mon[29180]: pgmap v13711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:45.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:24:46.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:46 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24567.361165 secs 2024-04-27T01:24:46.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24568.173202 secs 2024-04-27T01:24:46.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:46 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24567.361165 secs 2024-04-27T01:24:46.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24568.173202 secs 2024-04-27T01:24:46.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:46 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24567.361165 secs 2024-04-27T01:24:46.612 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24568.173202 secs 2024-04-27T01:24:47.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:47 smithi119 ceph-mon[32752]: pgmap v13712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:47.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:47 smithi167 ceph-mon[32751]: pgmap v13712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:47.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:47 smithi090 ceph-mon[29180]: pgmap v13712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:49.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:49 smithi090 ceph-mon[29180]: pgmap v13713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:50.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:49 smithi119 ceph-mon[32752]: pgmap v13713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:50.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:49 smithi167 ceph-mon[32751]: pgmap v13713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:51.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:51 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24572.361249 secs 2024-04-27T01:24:51.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:51 smithi090 ceph-mon[29180]: pgmap v13714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:51.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24573.173309 secs 2024-04-27T01:24:52.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:51 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24572.361249 secs 2024-04-27T01:24:52.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:51 smithi119 ceph-mon[32752]: pgmap v13714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:52.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24573.173309 secs 2024-04-27T01:24:52.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:51 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24572.361249 secs 2024-04-27T01:24:52.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:51 smithi167 ceph-mon[32751]: pgmap v13714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:24:52.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24573.173309 secs 2024-04-27T01:24:53.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:53 smithi090 ceph-mon[29180]: pgmap v13715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:24:54.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:53 smithi119 ceph-mon[32752]: pgmap v13715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:24:54.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:53 smithi167 ceph-mon[32751]: pgmap v13715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:24:55.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:55 smithi090 ceph-mon[29180]: pgmap v13716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:24:55.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:55 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24577.361308 secs 2024-04-27T01:24:56.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:55 smithi119 ceph-mon[32752]: pgmap v13716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:24:56.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:55 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24577.361308 secs 2024-04-27T01:24:56.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:55 smithi167 ceph-mon[32751]: pgmap v13716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:24:56.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:55 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24577.361308 secs 2024-04-27T01:24:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24578.173435 secs 2024-04-27T01:24:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24578.173435 secs 2024-04-27T01:24:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24578.173435 secs 2024-04-27T01:24:57.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:57 smithi090 ceph-mon[29180]: pgmap v13717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:58.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:57 smithi119 ceph-mon[32752]: pgmap v13717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:58.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:57 smithi167 ceph-mon[32751]: pgmap v13717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:24:59.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:24:59 smithi090 ceph-mon[29180]: pgmap v13718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:00.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:24:59 smithi119 ceph-mon[32752]: pgmap v13718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:00.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:24:59 smithi167 ceph-mon[32751]: pgmap v13718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:00.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:00 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24582.361395 secs 2024-04-27T01:25:01.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:00 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24582.361395 secs 2024-04-27T01:25:01.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:00 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24582.361395 secs 2024-04-27T01:25:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:01 smithi090 ceph-mon[29180]: pgmap v13719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24583.173557 secs 2024-04-27T01:25:01.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:01 smithi167 ceph-mon[32751]: pgmap v13719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:01.955 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24583.173557 secs 2024-04-27T01:25:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:01 smithi119 ceph-mon[32752]: pgmap v13719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:02.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24583.173557 secs 2024-04-27T01:25:03.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:03 smithi090 ceph-mon[29180]: pgmap v13720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:04.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:03 smithi119 ceph-mon[32752]: pgmap v13720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:04.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:03 smithi167 ceph-mon[32751]: pgmap v13720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:06.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:05 smithi119 ceph-mon[32752]: pgmap v13721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:06.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:05 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24587.361478 secs 2024-04-27T01:25:06.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:05 smithi167 ceph-mon[32751]: pgmap v13721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:06.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:05 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24587.361478 secs 2024-04-27T01:25:06.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:05 smithi090 ceph-mon[29180]: pgmap v13721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:06.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:05 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24587.361478 secs 2024-04-27T01:25:07.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24588.173678 secs 2024-04-27T01:25:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24588.173678 secs 2024-04-27T01:25:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24588.173678 secs 2024-04-27T01:25:08.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:07 smithi119 ceph-mon[32752]: pgmap v13722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:08.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:07 smithi167 ceph-mon[32751]: pgmap v13722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:08.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:07 smithi090 ceph-mon[29180]: pgmap v13722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:10.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:09 smithi119 ceph-mon[32752]: pgmap v13723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:10.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:09 smithi167 ceph-mon[32751]: pgmap v13723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:10.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:09 smithi090 ceph-mon[29180]: pgmap v13723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:11.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:10 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24592.361562 secs 2024-04-27T01:25:11.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:10 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24592.361562 secs 2024-04-27T01:25:11.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:10 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24592.361562 secs 2024-04-27T01:25:11.940 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:11 smithi119 ceph-mon[32752]: pgmap v13724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:11.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24593.173794 secs 2024-04-27T01:25:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:11 smithi167 ceph-mon[32751]: pgmap v13724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24593.173794 secs 2024-04-27T01:25:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:11 smithi090 ceph-mon[29180]: pgmap v13724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24593.173794 secs 2024-04-27T01:25:14.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:13 smithi119 ceph-mon[32752]: pgmap v13725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:14.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:13 smithi167 ceph-mon[32751]: pgmap v13725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:14.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:13 smithi090 ceph-mon[29180]: pgmap v13725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:16.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:15 smithi119 ceph-mon[32752]: pgmap v13726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:16.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:15 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24597.361622 secs 2024-04-27T01:25:16.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:15 smithi167 ceph-mon[32751]: pgmap v13726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:16.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:15 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24597.361622 secs 2024-04-27T01:25:16.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:15 smithi090 ceph-mon[29180]: pgmap v13726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:16.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:15 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24597.361622 secs 2024-04-27T01:25:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24598.173912 secs 2024-04-27T01:25:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24598.173912 secs 2024-04-27T01:25:17.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24598.173912 secs 2024-04-27T01:25:18.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:17 smithi119 ceph-mon[32752]: pgmap v13727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:18.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:17 smithi167 ceph-mon[32751]: pgmap v13727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:18.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:17 smithi090 ceph-mon[29180]: pgmap v13727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:19.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:19 smithi090 ceph-mon[29180]: pgmap v13728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:20.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:19 smithi119 ceph-mon[32752]: pgmap v13728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:20.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:19 smithi167 ceph-mon[32751]: pgmap v13728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:21.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:20 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24602.361707 secs 2024-04-27T01:25:21.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:20 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24602.361707 secs 2024-04-27T01:25:21.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:20 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24602.361707 secs 2024-04-27T01:25:22.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:21 smithi119 ceph-mon[32752]: pgmap v13729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:22.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24603.174010 secs 2024-04-27T01:25:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:21 smithi167 ceph-mon[32751]: pgmap v13729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24603.174010 secs 2024-04-27T01:25:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:21 smithi090 ceph-mon[29180]: pgmap v13729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24603.174010 secs 2024-04-27T01:25:24.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:23 smithi119 ceph-mon[32752]: pgmap v13730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:24.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:23 smithi167 ceph-mon[32751]: pgmap v13730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:24.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:23 smithi090 ceph-mon[29180]: pgmap v13730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:26.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:25 smithi119 ceph-mon[32752]: pgmap v13731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:26.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:25 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24607.361814 secs 2024-04-27T01:25:26.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:25 smithi167 ceph-mon[32751]: pgmap v13731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:26.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:25 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24607.361814 secs 2024-04-27T01:25:26.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:25 smithi090 ceph-mon[29180]: pgmap v13731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:26.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:25 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24607.361814 secs 2024-04-27T01:25:27.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24608.174134 secs 2024-04-27T01:25:27.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24608.174134 secs 2024-04-27T01:25:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24608.174134 secs 2024-04-27T01:25:28.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:27 smithi119 ceph-mon[32752]: pgmap v13732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:28.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:27 smithi167 ceph-mon[32751]: pgmap v13732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:28.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:27 smithi090 ceph-mon[29180]: pgmap v13732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:30.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:29 smithi119 ceph-mon[32752]: pgmap v13733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:30.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:29 smithi167 ceph-mon[32751]: pgmap v13733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:30.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:29 smithi090 ceph-mon[29180]: pgmap v13733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:31.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:30 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24612.361901 secs 2024-04-27T01:25:31.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:30 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24612.361901 secs 2024-04-27T01:25:31.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:30 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24612.361901 secs 2024-04-27T01:25:31.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:31 smithi167 ceph-mon[32751]: pgmap v13734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:31.955 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24613.174224 secs 2024-04-27T01:25:32.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:31 smithi119 ceph-mon[32752]: pgmap v13734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:32.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24613.174224 secs 2024-04-27T01:25:32.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:31 smithi090 ceph-mon[29180]: pgmap v13734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:32.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24613.174224 secs 2024-04-27T01:25:34.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:33 smithi119 ceph-mon[32752]: pgmap v13735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:34.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:33 smithi167 ceph-mon[32751]: pgmap v13735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:34.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:33 smithi090 ceph-mon[29180]: pgmap v13735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:36.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:35 smithi119 ceph-mon[32752]: pgmap v13736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:36.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:35 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 24617.361988 secs 2024-04-27T01:25:36.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:35 smithi167 ceph-mon[32751]: pgmap v13736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:36.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:35 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 24617.361988 secs 2024-04-27T01:25:36.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:35 smithi090 ceph-mon[29180]: pgmap v13736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:25:36.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:35 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 24617.361988 secs 2024-04-27T01:25:37.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:36 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 24618.174399 secs 2024-04-27T01:25:37.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:36 smithi090 ceph-mon[29180]: slow request 23044.699748 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:25:37.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:36 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 24618.174399 secs 2024-04-27T01:25:37.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:36 smithi119 ceph-mon[32752]: slow request 23044.699748 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:25:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:36 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 24618.174399 secs 2024-04-27T01:25:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:36 smithi167 ceph-mon[32751]: slow request 23044.699748 seconds old, received at 2024-04-26T19:01:31.225296+0000: client_request(client.26390:1 getattr pAsLsXsFs #0x1 2024-04-26T19:01:31.218725+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:25:37.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:37 smithi090 ceph-mon[29180]: pgmap v13737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:37.862 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:25:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:25:37.575+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:25:37.863 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:25:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:25:37.575+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:25:37.863 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:25:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:25:37.575+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:25:37.863 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:25:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:25:37.575+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:25:37.863 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:25:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:25:37.575+0000 7f823c7a7700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:25:38.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:37 smithi119 ceph-mon[32752]: pgmap v13737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:38.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:37 smithi167 ceph-mon[32751]: pgmap v13737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:40.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:39 smithi119 ceph-mon[32752]: pgmap v13738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:25:40.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:39 smithi167 ceph-mon[32751]: pgmap v13738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:25:40.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:39 smithi090 ceph-mon[29180]: pgmap v13738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:25:41.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:40 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24622.362092 secs 2024-04-27T01:25:41.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:40 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24622.362092 secs 2024-04-27T01:25:41.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:40 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24622.362092 secs 2024-04-27T01:25:42.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:41 smithi119 ceph-mon[32752]: pgmap v13739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:25:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24623.174445 secs 2024-04-27T01:25:42.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:41 smithi167 ceph-mon[32751]: pgmap v13739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:25:42.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24623.174445 secs 2024-04-27T01:25:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:41 smithi090 ceph-mon[29180]: pgmap v13739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:25:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24623.174445 secs 2024-04-27T01:25:44.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:43 smithi119 ceph-mon[32752]: pgmap v13740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:25:44.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:25:44.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:25:44.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:43 smithi167 ceph-mon[32751]: pgmap v13740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:25:44.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:25:44.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:25:44.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:43 smithi090 ceph-mon[29180]: pgmap v13740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:25:44.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:25:44.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:25:45.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:25:45.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:25:45.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:25:46.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:45 smithi119 ceph-mon[32752]: pgmap v13741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:25:46.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:45 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24627.362159 secs 2024-04-27T01:25:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:25:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:25:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:25:46.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:45 smithi167 ceph-mon[32751]: pgmap v13741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:25:46.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:45 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24627.362159 secs 2024-04-27T01:25:46.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:25:46.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:25:46.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:25:46.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:45 smithi090 ceph-mon[29180]: pgmap v13741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:25:46.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:45 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24627.362159 secs 2024-04-27T01:25:46.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:25:46.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:25:46.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:25:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:46 smithi119 ceph-mon[32752]: pgmap v13742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-27T01:25:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:46 smithi119 ceph-mon[32752]: pgmap v13743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-27T01:25:47.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24628.174532 secs 2024-04-27T01:25:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:46 smithi167 ceph-mon[32751]: pgmap v13742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-27T01:25:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:46 smithi167 ceph-mon[32751]: pgmap v13743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-27T01:25:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24628.174532 secs 2024-04-27T01:25:47.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:46 smithi090 ceph-mon[29180]: pgmap v13742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-27T01:25:47.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:46 smithi090 ceph-mon[29180]: pgmap v13743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-27T01:25:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24628.174532 secs 2024-04-27T01:25:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:48 smithi119 ceph-mon[32752]: pgmap v13744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:25:49.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:48 smithi167 ceph-mon[32751]: pgmap v13744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:25:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:48 smithi090 ceph-mon[29180]: pgmap v13744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:25:51.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:50 smithi119 ceph-mon[32752]: pgmap v13745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:25:51.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:50 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24632.362241 secs 2024-04-27T01:25:51.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:50 smithi167 ceph-mon[32751]: pgmap v13745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:25:51.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:50 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24632.362241 secs 2024-04-27T01:25:51.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:50 smithi090 ceph-mon[29180]: pgmap v13745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:25:51.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:50 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24632.362241 secs 2024-04-27T01:25:52.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24633.174661 secs 2024-04-27T01:25:52.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24633.174661 secs 2024-04-27T01:25:52.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24633.174661 secs 2024-04-27T01:25:53.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:52 smithi119 ceph-mon[32752]: pgmap v13746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:25:53.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:52 smithi167 ceph-mon[32751]: pgmap v13746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:25:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:52 smithi090 ceph-mon[29180]: pgmap v13746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:25:55.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:54 smithi119 ceph-mon[32752]: pgmap v13747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:25:55.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:54 smithi167 ceph-mon[32751]: pgmap v13747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:25:55.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:54 smithi090 ceph-mon[29180]: pgmap v13747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:25:56.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:55 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24637.362328 secs 2024-04-27T01:25:56.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:55 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24637.362328 secs 2024-04-27T01:25:56.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:55 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24637.362328 secs 2024-04-27T01:25:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:56 smithi119 ceph-mon[32752]: pgmap v13748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:25:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24638.174727 secs 2024-04-27T01:25:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:56 smithi167 ceph-mon[32751]: pgmap v13748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:25:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24638.174727 secs 2024-04-27T01:25:57.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:56 smithi090 ceph-mon[29180]: pgmap v13748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:25:57.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24638.174727 secs 2024-04-27T01:25:59.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:25:58 smithi119 ceph-mon[32752]: pgmap v13749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:59.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:25:58 smithi167 ceph-mon[32751]: pgmap v13749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:25:59.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:25:58 smithi090 ceph-mon[29180]: pgmap v13749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:01.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:00 smithi119 ceph-mon[32752]: pgmap v13750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:01.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:00 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24642.362412 secs 2024-04-27T01:26:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:00 smithi167 ceph-mon[32751]: pgmap v13750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:01.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:00 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24642.362412 secs 2024-04-27T01:26:01.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:00 smithi090 ceph-mon[29180]: pgmap v13750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:01.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:00 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24642.362412 secs 2024-04-27T01:26:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24643.174791 secs 2024-04-27T01:26:02.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24643.174791 secs 2024-04-27T01:26:02.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24643.174791 secs 2024-04-27T01:26:03.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:02 smithi119 ceph-mon[32752]: pgmap v13751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:03.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:02 smithi167 ceph-mon[32751]: pgmap v13751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:03.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:02 smithi090 ceph-mon[29180]: pgmap v13751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:04 smithi119 ceph-mon[32752]: pgmap v13752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:04 smithi167 ceph-mon[32751]: pgmap v13752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:05.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:04 smithi090 ceph-mon[29180]: pgmap v13752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:06.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:05 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24647.362496 secs 2024-04-27T01:26:06.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:05 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24647.362496 secs 2024-04-27T01:26:06.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:05 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24647.362496 secs 2024-04-27T01:26:06.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:06 smithi090 ceph-mon[29180]: pgmap v13753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:26:06.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24648.174908 secs 2024-04-27T01:26:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:06 smithi119 ceph-mon[32752]: pgmap v13753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:26:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24648.174908 secs 2024-04-27T01:26:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:06 smithi167 ceph-mon[32751]: pgmap v13753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:26:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24648.174908 secs 2024-04-27T01:26:09.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:08 smithi119 ceph-mon[32752]: pgmap v13754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:09.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:08 smithi167 ceph-mon[32751]: pgmap v13754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:09.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:08 smithi090 ceph-mon[29180]: pgmap v13754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:11.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:10 smithi119 ceph-mon[32752]: pgmap v13755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:26:11.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:10 smithi119 ceph-mon[32752]: 33 slow requests, 1 included below; oldest blocked for > 24652.362588 secs 2024-04-27T01:26:11.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:10 smithi119 ceph-mon[32752]: slow request 5764.124337 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:26:11.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:10 smithi167 ceph-mon[32751]: pgmap v13755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:26:11.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:10 smithi167 ceph-mon[32751]: 33 slow requests, 1 included below; oldest blocked for > 24652.362588 secs 2024-04-27T01:26:11.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:10 smithi167 ceph-mon[32751]: slow request 5764.124337 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:26:11.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:10 smithi090 ceph-mon[29180]: pgmap v13755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:26:11.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:10 smithi090 ceph-mon[29180]: 33 slow requests, 1 included below; oldest blocked for > 24652.362588 secs 2024-04-27T01:26:11.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:10 smithi090 ceph-mon[29180]: slow request 5764.124337 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:26:12.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24653.175043 secs 2024-04-27T01:26:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24653.175043 secs 2024-04-27T01:26:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24653.175043 secs 2024-04-27T01:26:13.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:12 smithi119 ceph-mon[32752]: pgmap v13756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:13.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:12 smithi167 ceph-mon[32751]: pgmap v13756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:13.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:12 smithi090 ceph-mon[29180]: pgmap v13756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:15.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:14 smithi119 ceph-mon[32752]: pgmap v13757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:15.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:14 smithi167 ceph-mon[32751]: pgmap v13757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:15.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:14 smithi090 ceph-mon[29180]: pgmap v13757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:26:16.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:15 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24657.362674 secs 2024-04-27T01:26:16.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:15 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24657.362674 secs 2024-04-27T01:26:16.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:15 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24657.362674 secs 2024-04-27T01:26:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:16 smithi119 ceph-mon[32752]: pgmap v13758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24658.175177 secs 2024-04-27T01:26:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:16 smithi167 ceph-mon[32751]: pgmap v13758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24658.175177 secs 2024-04-27T01:26:17.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:16 smithi090 ceph-mon[29180]: pgmap v13758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:17.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24658.175177 secs 2024-04-27T01:26:18.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:17 smithi167 ceph-mon[32751]: pgmap v13759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:18.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:17 smithi090 ceph-mon[29180]: pgmap v13759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:18.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:17 smithi119 ceph-mon[32752]: pgmap v13759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:20.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:20 smithi119 ceph-mon[32752]: pgmap v13760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:20.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:20 smithi167 ceph-mon[32751]: pgmap v13760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:20.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:20 smithi090 ceph-mon[29180]: pgmap v13760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:21.762 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:21 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24662.362761 secs 2024-04-27T01:26:21.762 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24663.175315 secs 2024-04-27T01:26:21.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:21 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24662.362761 secs 2024-04-27T01:26:21.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24663.175315 secs 2024-04-27T01:26:21.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:21 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24662.362761 secs 2024-04-27T01:26:21.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24663.175315 secs 2024-04-27T01:26:22.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:22 smithi119 ceph-mon[32752]: pgmap v13761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:22.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:22 smithi167 ceph-mon[32751]: pgmap v13761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:22.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:22 smithi090 ceph-mon[29180]: pgmap v13761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:24.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:24 smithi119 ceph-mon[32752]: pgmap v13762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:24.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:24 smithi167 ceph-mon[32751]: pgmap v13762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:24.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:24 smithi090 ceph-mon[29180]: pgmap v13762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:26.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:26 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24667.362818 secs 2024-04-27T01:26:26.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:26 smithi119 ceph-mon[32752]: pgmap v13763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:26.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24668.175396 secs 2024-04-27T01:26:26.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:26 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24667.362818 secs 2024-04-27T01:26:26.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:26 smithi167 ceph-mon[32751]: pgmap v13763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:26.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24668.175396 secs 2024-04-27T01:26:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:26 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24667.362818 secs 2024-04-27T01:26:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:26 smithi090 ceph-mon[29180]: pgmap v13763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24668.175396 secs 2024-04-27T01:26:28.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:28 smithi119 ceph-mon[32752]: pgmap v13764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:28.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:28 smithi167 ceph-mon[32751]: pgmap v13764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:28.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:28 smithi090 ceph-mon[29180]: pgmap v13764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:30.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:30 smithi119 ceph-mon[32752]: pgmap v13765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:30.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:30 smithi167 ceph-mon[32751]: pgmap v13765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:30.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:30 smithi090 ceph-mon[29180]: pgmap v13765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:31.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:31 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24672.362881 secs 2024-04-27T01:26:31.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24673.175518 secs 2024-04-27T01:26:31.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:31 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24672.362881 secs 2024-04-27T01:26:31.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24673.175518 secs 2024-04-27T01:26:31.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:31 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24672.362881 secs 2024-04-27T01:26:31.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24673.175518 secs 2024-04-27T01:26:32.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:32 smithi119 ceph-mon[32752]: pgmap v13766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:32.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:32 smithi167 ceph-mon[32751]: pgmap v13766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:32.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:32 smithi090 ceph-mon[29180]: pgmap v13766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:34.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:34 smithi119 ceph-mon[32752]: pgmap v13767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:34.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:34 smithi167 ceph-mon[32751]: pgmap v13767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:34.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:34 smithi090 ceph-mon[29180]: pgmap v13767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:36.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:36 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24677.362965 secs 2024-04-27T01:26:36.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:36 smithi090 ceph-mon[29180]: pgmap v13768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:36.759 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24678.175644 secs 2024-04-27T01:26:36.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:36 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24677.362965 secs 2024-04-27T01:26:36.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:36 smithi119 ceph-mon[32752]: pgmap v13768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:36.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24678.175644 secs 2024-04-27T01:26:36.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:36 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24677.362965 secs 2024-04-27T01:26:36.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:36 smithi167 ceph-mon[32751]: pgmap v13768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:36.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24678.175644 secs 2024-04-27T01:26:38.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:38 smithi119 ceph-mon[32752]: pgmap v13769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:38.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:38 smithi167 ceph-mon[32751]: pgmap v13769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:38.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:38 smithi090 ceph-mon[29180]: pgmap v13769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:40.663 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:40 smithi090 ceph-mon[29180]: pgmap v13770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:40.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:40 smithi119 ceph-mon[32752]: pgmap v13770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:40.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:40 smithi167 ceph-mon[32751]: pgmap v13770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:41.690 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:41 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24682.363056 secs 2024-04-27T01:26:41.690 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24683.175763 secs 2024-04-27T01:26:41.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:41 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24682.363056 secs 2024-04-27T01:26:41.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24683.175763 secs 2024-04-27T01:26:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:41 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24682.363056 secs 2024-04-27T01:26:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24683.175763 secs 2024-04-27T01:26:42.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:42 smithi119 ceph-mon[32752]: pgmap v13771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:42.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:42 smithi167 ceph-mon[32751]: pgmap v13771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:42.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:42 smithi090 ceph-mon[29180]: pgmap v13771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:44.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:44 smithi119 ceph-mon[32752]: pgmap v13772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:44.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:44 smithi167 ceph-mon[32751]: pgmap v13772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:44.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:44 smithi090 ceph-mon[29180]: pgmap v13772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:26:45.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:26:45.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:26:45.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:26:46.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:46 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24687.363117 secs 2024-04-27T01:26:46.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:46 smithi119 ceph-mon[32752]: pgmap v13773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:46.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24688.175832 secs 2024-04-27T01:26:46.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:26:46.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:26:46.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:26:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:46 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24687.363117 secs 2024-04-27T01:26:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:46 smithi167 ceph-mon[32751]: pgmap v13773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24688.175832 secs 2024-04-27T01:26:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:26:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:26:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:26:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:46 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24687.363117 secs 2024-04-27T01:26:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:46 smithi090 ceph-mon[29180]: pgmap v13773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24688.175832 secs 2024-04-27T01:26:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:26:46.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:26:46.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:26:47.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:47 smithi119 ceph-mon[32752]: pgmap v13774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:26:47.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:47 smithi119 ceph-mon[32752]: pgmap v13775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:47.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:47 smithi167 ceph-mon[32751]: pgmap v13774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:26:47.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:47 smithi167 ceph-mon[32751]: pgmap v13775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:47 smithi090 ceph-mon[29180]: pgmap v13774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:26:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:47 smithi090 ceph-mon[29180]: pgmap v13775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:49.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:49 smithi119 ceph-mon[32752]: pgmap v13776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:26:49.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:49 smithi167 ceph-mon[32751]: pgmap v13776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:26:49.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:49 smithi090 ceph-mon[29180]: pgmap v13776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:26:51.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:51 smithi119 ceph-mon[32752]: pgmap v13777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:51.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:51 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24692.363180 secs 2024-04-27T01:26:51.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24693.175986 secs 2024-04-27T01:26:51.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:51 smithi167 ceph-mon[32751]: pgmap v13777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:51.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:51 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24692.363180 secs 2024-04-27T01:26:51.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24693.175986 secs 2024-04-27T01:26:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:51 smithi090 ceph-mon[29180]: pgmap v13777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:51 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24692.363180 secs 2024-04-27T01:26:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24693.175986 secs 2024-04-27T01:26:53.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:53 smithi119 ceph-mon[32752]: pgmap v13778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:26:53.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:53 smithi167 ceph-mon[32751]: pgmap v13778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:26:53.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:53 smithi090 ceph-mon[29180]: pgmap v13778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:26:55.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:55 smithi119 ceph-mon[32752]: pgmap v13779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:55.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:55 smithi167 ceph-mon[32751]: pgmap v13779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:55.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:55 smithi090 ceph-mon[29180]: pgmap v13779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:26:56.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:56 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24697.363265 secs 2024-04-27T01:26:56.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24698.176099 secs 2024-04-27T01:26:56.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:56 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24697.363265 secs 2024-04-27T01:26:56.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24698.176099 secs 2024-04-27T01:26:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:56 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24697.363265 secs 2024-04-27T01:26:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24698.176099 secs 2024-04-27T01:26:57.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:57 smithi119 ceph-mon[32752]: pgmap v13780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:26:57.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:57 smithi167 ceph-mon[32751]: pgmap v13780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:26:57.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:57 smithi090 ceph-mon[29180]: pgmap v13780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:26:59.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:26:59 smithi119 ceph-mon[32752]: pgmap v13781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:59.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:26:59 smithi167 ceph-mon[32751]: pgmap v13781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:26:59.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:26:59 smithi090 ceph-mon[29180]: pgmap v13781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:01.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:01 smithi119 ceph-mon[32752]: pgmap v13782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:01.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:01 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24702.363352 secs 2024-04-27T01:27:01.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24703.176210 secs 2024-04-27T01:27:01.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:01 smithi167 ceph-mon[32751]: pgmap v13782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:01.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:01 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24702.363352 secs 2024-04-27T01:27:01.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24703.176210 secs 2024-04-27T01:27:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:01 smithi090 ceph-mon[29180]: pgmap v13782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:01 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24702.363352 secs 2024-04-27T01:27:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24703.176210 secs 2024-04-27T01:27:03.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:03 smithi119 ceph-mon[32752]: pgmap v13783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:03.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:03 smithi167 ceph-mon[32751]: pgmap v13783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:03.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:03 smithi090 ceph-mon[29180]: pgmap v13783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:05.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:05 smithi119 ceph-mon[32752]: pgmap v13784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:05.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:05 smithi167 ceph-mon[32751]: pgmap v13784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:05.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:05 smithi090 ceph-mon[29180]: pgmap v13784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:06.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:06 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24707.363435 secs 2024-04-27T01:27:06.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24708.176324 secs 2024-04-27T01:27:06.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:06 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24707.363435 secs 2024-04-27T01:27:06.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24708.176324 secs 2024-04-27T01:27:06.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:06 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24707.363435 secs 2024-04-27T01:27:06.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24708.176324 secs 2024-04-27T01:27:07.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:07 smithi119 ceph-mon[32752]: pgmap v13785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:07.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:07 smithi167 ceph-mon[32751]: pgmap v13785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:07.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:07 smithi090 ceph-mon[29180]: pgmap v13785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:09.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:09 smithi119 ceph-mon[32752]: pgmap v13786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:09.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:09 smithi167 ceph-mon[32751]: pgmap v13786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:09.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:09 smithi090 ceph-mon[29180]: pgmap v13786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:11.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:11 smithi119 ceph-mon[32752]: pgmap v13787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:11.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:11 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24712.363523 secs 2024-04-27T01:27:11.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24713.176448 secs 2024-04-27T01:27:11.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:11 smithi167 ceph-mon[32751]: pgmap v13787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:11.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:11 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24712.363523 secs 2024-04-27T01:27:11.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24713.176448 secs 2024-04-27T01:27:11.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:11 smithi090 ceph-mon[29180]: pgmap v13787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:11.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:11 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24712.363523 secs 2024-04-27T01:27:11.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24713.176448 secs 2024-04-27T01:27:13.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:13 smithi119 ceph-mon[32752]: pgmap v13788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:13.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:13 smithi167 ceph-mon[32751]: pgmap v13788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:13.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:13 smithi090 ceph-mon[29180]: pgmap v13788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:15.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:15 smithi119 ceph-mon[32752]: pgmap v13789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:15.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:15 smithi167 ceph-mon[32751]: pgmap v13789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:15.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:15 smithi090 ceph-mon[29180]: pgmap v13789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:16.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:16 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24717.363607 secs 2024-04-27T01:27:16.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24718.176585 secs 2024-04-27T01:27:16.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:16 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24717.363607 secs 2024-04-27T01:27:16.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24718.176585 secs 2024-04-27T01:27:16.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:16 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24717.363607 secs 2024-04-27T01:27:16.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24718.176585 secs 2024-04-27T01:27:17.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:17 smithi119 ceph-mon[32752]: pgmap v13790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:17.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:17 smithi167 ceph-mon[32751]: pgmap v13790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:17.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:17 smithi090 ceph-mon[29180]: pgmap v13790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:19.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:19 smithi119 ceph-mon[32752]: pgmap v13791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:19.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:19 smithi167 ceph-mon[32751]: pgmap v13791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:19.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:19 smithi090 ceph-mon[29180]: pgmap v13791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:21.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:21 smithi119 ceph-mon[32752]: pgmap v13792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:21.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:21 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24722.363699 secs 2024-04-27T01:27:21.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24723.176717 secs 2024-04-27T01:27:21.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:21 smithi167 ceph-mon[32751]: pgmap v13792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:21.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:21 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24722.363699 secs 2024-04-27T01:27:21.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24723.176717 secs 2024-04-27T01:27:21.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:21 smithi090 ceph-mon[29180]: pgmap v13792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:21.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:21 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24722.363699 secs 2024-04-27T01:27:21.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24723.176717 secs 2024-04-27T01:27:23.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:23 smithi119 ceph-mon[32752]: pgmap v13793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:23.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:23 smithi167 ceph-mon[32751]: pgmap v13793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:23.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:23 smithi090 ceph-mon[29180]: pgmap v13793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:25.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:25 smithi119 ceph-mon[32752]: pgmap v13794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:25.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:25 smithi167 ceph-mon[32751]: pgmap v13794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:25.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:25 smithi090 ceph-mon[29180]: pgmap v13794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:26.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:26 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24727.363749 secs 2024-04-27T01:27:26.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24728.176871 secs 2024-04-27T01:27:26.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:26 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24727.363749 secs 2024-04-27T01:27:26.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24728.176871 secs 2024-04-27T01:27:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:26 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24727.363749 secs 2024-04-27T01:27:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24728.176871 secs 2024-04-27T01:27:27.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:27 smithi119 ceph-mon[32752]: pgmap v13795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:27.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:27 smithi167 ceph-mon[32751]: pgmap v13795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:27.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:27 smithi090 ceph-mon[29180]: pgmap v13795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:29.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:29 smithi119 ceph-mon[32752]: pgmap v13796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:29.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:29 smithi167 ceph-mon[32751]: pgmap v13796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:29.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:29 smithi090 ceph-mon[29180]: pgmap v13796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:31.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:31 smithi119 ceph-mon[32752]: pgmap v13797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:31.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:31 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24732.363846 secs 2024-04-27T01:27:31.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24733.176995 secs 2024-04-27T01:27:31.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:31 smithi167 ceph-mon[32751]: pgmap v13797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:31.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:31 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24732.363846 secs 2024-04-27T01:27:31.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24733.176995 secs 2024-04-27T01:27:31.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:31 smithi090 ceph-mon[29180]: pgmap v13797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:31.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:31 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24732.363846 secs 2024-04-27T01:27:31.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24733.176995 secs 2024-04-27T01:27:33.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:33 smithi119 ceph-mon[32752]: pgmap v13798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:33.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:33 smithi167 ceph-mon[32751]: pgmap v13798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:33.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:33 smithi090 ceph-mon[29180]: pgmap v13798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:35.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:35 smithi119 ceph-mon[32752]: pgmap v13799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:35.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:35 smithi167 ceph-mon[32751]: pgmap v13799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:35.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:35 smithi090 ceph-mon[29180]: pgmap v13799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:36.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:36 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24737.363932 secs 2024-04-27T01:27:36.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24738.177123 secs 2024-04-27T01:27:36.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:36 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24737.363932 secs 2024-04-27T01:27:36.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24738.177123 secs 2024-04-27T01:27:36.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:36 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24737.363932 secs 2024-04-27T01:27:36.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24738.177123 secs 2024-04-27T01:27:37.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:37 smithi119 ceph-mon[32752]: pgmap v13800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:37.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:37 smithi167 ceph-mon[32751]: pgmap v13800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:37.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:37 smithi090 ceph-mon[29180]: pgmap v13800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:39.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:39 smithi119 ceph-mon[32752]: pgmap v13801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:39.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:39 smithi167 ceph-mon[32751]: pgmap v13801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:39.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:39 smithi090 ceph-mon[29180]: pgmap v13801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:41.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:41 smithi119 ceph-mon[32752]: pgmap v13802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:41.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:41 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24742.364023 secs 2024-04-27T01:27:41.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24743.177219 secs 2024-04-27T01:27:41.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:41 smithi167 ceph-mon[32751]: pgmap v13802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:41.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:41 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24742.364023 secs 2024-04-27T01:27:41.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24743.177219 secs 2024-04-27T01:27:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:41 smithi090 ceph-mon[29180]: pgmap v13802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:41 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24742.364023 secs 2024-04-27T01:27:41.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24743.177219 secs 2024-04-27T01:27:43.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:43 smithi119 ceph-mon[32752]: pgmap v13803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:43.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:43 smithi167 ceph-mon[32751]: pgmap v13803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:43.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:43 smithi090 ceph-mon[29180]: pgmap v13803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:45.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:45 smithi167 ceph-mon[32751]: pgmap v13804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:45.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:45 smithi090 ceph-mon[29180]: pgmap v13804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:46.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:45 smithi119 ceph-mon[32752]: pgmap v13804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:27:46.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:46 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24747.364098 secs 2024-04-27T01:27:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24748.177345 secs 2024-04-27T01:27:46.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:27:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:46 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24747.364098 secs 2024-04-27T01:27:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24748.177345 secs 2024-04-27T01:27:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:27:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:46 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24747.364098 secs 2024-04-27T01:27:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24748.177345 secs 2024-04-27T01:27:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:27:47.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:47 smithi167 ceph-mon[32751]: pgmap v13805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:47.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:27:47.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:27:47.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:27:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:47 smithi090 ceph-mon[29180]: pgmap v13805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:27:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:27:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:27:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:47 smithi119 ceph-mon[32752]: pgmap v13805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:27:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:27:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:27:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:27:48.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:48 smithi167 ceph-mon[32751]: pgmap v13806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:27:48.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:48 smithi167 ceph-mon[32751]: pgmap v13807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:48.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:48 smithi090 ceph-mon[29180]: pgmap v13806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:27:48.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:48 smithi090 ceph-mon[29180]: pgmap v13807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:48 smithi119 ceph-mon[32752]: pgmap v13806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:27:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:48 smithi119 ceph-mon[32752]: pgmap v13807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:50.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:50 smithi090 ceph-mon[29180]: pgmap v13808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:51.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:50 smithi119 ceph-mon[32752]: pgmap v13808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:51.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:50 smithi167 ceph-mon[32751]: pgmap v13808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:51.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:51 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24752.364184 secs 2024-04-27T01:27:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24753.177408 secs 2024-04-27T01:27:52.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:51 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24752.364184 secs 2024-04-27T01:27:52.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24753.177408 secs 2024-04-27T01:27:52.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:51 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24752.364184 secs 2024-04-27T01:27:52.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24753.177408 secs 2024-04-27T01:27:52.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:52 smithi090 ceph-mon[29180]: pgmap v13809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:53.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:52 smithi119 ceph-mon[32752]: pgmap v13809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:53.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:52 smithi167 ceph-mon[32751]: pgmap v13809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:54.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:54 smithi090 ceph-mon[29180]: pgmap v13810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:27:55.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:54 smithi119 ceph-mon[32752]: pgmap v13810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:27:55.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:54 smithi167 ceph-mon[32751]: pgmap v13810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:27:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:56 smithi090 ceph-mon[29180]: pgmap v13811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:56 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24757.364270 secs 2024-04-27T01:27:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24758.177531 secs 2024-04-27T01:27:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:56 smithi119 ceph-mon[32752]: pgmap v13811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:56 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24757.364270 secs 2024-04-27T01:27:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24758.177531 secs 2024-04-27T01:27:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:56 smithi167 ceph-mon[32751]: pgmap v13811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:27:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:56 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24757.364270 secs 2024-04-27T01:27:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24758.177531 secs 2024-04-27T01:27:58.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:27:58 smithi090 ceph-mon[29180]: pgmap v13812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:27:59.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:27:58 smithi119 ceph-mon[32752]: pgmap v13812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:27:59.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:27:58 smithi167 ceph-mon[32751]: pgmap v13812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:28:00.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:00 smithi090 ceph-mon[29180]: pgmap v13813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:01.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:00 smithi119 ceph-mon[32752]: pgmap v13813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:00 smithi167 ceph-mon[32751]: pgmap v13813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:01 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24762.364357 secs 2024-04-27T01:28:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24763.177647 secs 2024-04-27T01:28:01.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:01 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24762.364357 secs 2024-04-27T01:28:01.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24763.177647 secs 2024-04-27T01:28:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:01 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24762.364357 secs 2024-04-27T01:28:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24763.177647 secs 2024-04-27T01:28:02.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:02 smithi090 ceph-mon[29180]: pgmap v13814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:03.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:02 smithi119 ceph-mon[32752]: pgmap v13814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:03.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:02 smithi167 ceph-mon[32751]: pgmap v13814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:04 smithi090 ceph-mon[29180]: pgmap v13815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:04 smithi119 ceph-mon[32752]: pgmap v13815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:04 smithi167 ceph-mon[32751]: pgmap v13815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:06.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:06 smithi090 ceph-mon[29180]: pgmap v13816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:06.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:06 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24767.364443 secs 2024-04-27T01:28:06.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24768.177724 secs 2024-04-27T01:28:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:06 smithi119 ceph-mon[32752]: pgmap v13816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:06 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24767.364443 secs 2024-04-27T01:28:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24768.177724 secs 2024-04-27T01:28:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:06 smithi167 ceph-mon[32751]: pgmap v13816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:06 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24767.364443 secs 2024-04-27T01:28:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24768.177724 secs 2024-04-27T01:28:08.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:08 smithi090 ceph-mon[29180]: pgmap v13817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:09.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:08 smithi119 ceph-mon[32752]: pgmap v13817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:09.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:08 smithi167 ceph-mon[32751]: pgmap v13817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:10.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:10 smithi090 ceph-mon[29180]: pgmap v13818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:11.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:10 smithi119 ceph-mon[32752]: pgmap v13818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:11.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:10 smithi167 ceph-mon[32751]: pgmap v13818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:11.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:11 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24772.364530 secs 2024-04-27T01:28:11.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24773.177835 secs 2024-04-27T01:28:11.940 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:11 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24772.364530 secs 2024-04-27T01:28:11.940 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24773.177835 secs 2024-04-27T01:28:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:11 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24772.364530 secs 2024-04-27T01:28:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24773.177835 secs 2024-04-27T01:28:12.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:12 smithi090 ceph-mon[29180]: pgmap v13819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:13.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:12 smithi119 ceph-mon[32752]: pgmap v13819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:13.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:12 smithi167 ceph-mon[32751]: pgmap v13819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:15.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:14 smithi119 ceph-mon[32752]: pgmap v13820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:15.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:14 smithi167 ceph-mon[32751]: pgmap v13820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:15.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:14 smithi090 ceph-mon[29180]: pgmap v13820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:16.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:15 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24777.364614 secs 2024-04-27T01:28:16.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:15 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24777.364614 secs 2024-04-27T01:28:16.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:15 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24777.364614 secs 2024-04-27T01:28:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:16 smithi119 ceph-mon[32752]: pgmap v13821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24778.177950 secs 2024-04-27T01:28:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:16 smithi167 ceph-mon[32751]: pgmap v13821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24778.177950 secs 2024-04-27T01:28:17.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:16 smithi090 ceph-mon[29180]: pgmap v13821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:17.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24778.177950 secs 2024-04-27T01:28:19.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:18 smithi119 ceph-mon[32752]: pgmap v13822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:19.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:18 smithi167 ceph-mon[32751]: pgmap v13822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:19.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:18 smithi090 ceph-mon[29180]: pgmap v13822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:21.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:20 smithi119 ceph-mon[32752]: pgmap v13823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:21.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:20 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24782.364700 secs 2024-04-27T01:28:21.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:20 smithi167 ceph-mon[32751]: pgmap v13823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:21.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:20 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24782.364700 secs 2024-04-27T01:28:21.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:20 smithi090 ceph-mon[29180]: pgmap v13823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:21.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:20 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24782.364700 secs 2024-04-27T01:28:22.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24783.178039 secs 2024-04-27T01:28:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24783.178039 secs 2024-04-27T01:28:22.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24783.178039 secs 2024-04-27T01:28:23.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:22 smithi119 ceph-mon[32752]: pgmap v13824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:23.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:22 smithi167 ceph-mon[32751]: pgmap v13824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:23.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:22 smithi090 ceph-mon[29180]: pgmap v13824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:25.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:24 smithi119 ceph-mon[32752]: pgmap v13825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:25.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:24 smithi167 ceph-mon[32751]: pgmap v13825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:25.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:24 smithi090 ceph-mon[29180]: pgmap v13825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:26.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:25 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24787.364784 secs 2024-04-27T01:28:26.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:25 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24787.364784 secs 2024-04-27T01:28:26.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:25 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24787.364784 secs 2024-04-27T01:28:27.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:26 smithi119 ceph-mon[32752]: pgmap v13826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:27.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24788.178101 secs 2024-04-27T01:28:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:26 smithi167 ceph-mon[32751]: pgmap v13826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:27.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24788.178101 secs 2024-04-27T01:28:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:26 smithi090 ceph-mon[29180]: pgmap v13826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24788.178101 secs 2024-04-27T01:28:29.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:28 smithi119 ceph-mon[32752]: pgmap v13827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:29.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:28 smithi167 ceph-mon[32751]: pgmap v13827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:29.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:28 smithi090 ceph-mon[29180]: pgmap v13827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:31.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:30 smithi119 ceph-mon[32752]: pgmap v13828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:31.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:30 smithi119 ceph-mon[32752]: 33 slow requests, 1 included below; oldest blocked for > 24792.364851 secs 2024-04-27T01:28:31.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:30 smithi119 ceph-mon[32752]: slow request 720.466536 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:28:31.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:30 smithi167 ceph-mon[32751]: pgmap v13828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:31.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:30 smithi167 ceph-mon[32751]: 33 slow requests, 1 included below; oldest blocked for > 24792.364851 secs 2024-04-27T01:28:31.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:30 smithi167 ceph-mon[32751]: slow request 720.466536 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:28:31.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:30 smithi090 ceph-mon[29180]: pgmap v13828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:31.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:30 smithi090 ceph-mon[29180]: 33 slow requests, 1 included below; oldest blocked for > 24792.364851 secs 2024-04-27T01:28:31.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:30 smithi090 ceph-mon[29180]: slow request 720.466536 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:28:31.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24793.178212 secs 2024-04-27T01:28:32.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24793.178212 secs 2024-04-27T01:28:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24793.178212 secs 2024-04-27T01:28:33.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:32 smithi119 ceph-mon[32752]: pgmap v13829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:33.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:32 smithi167 ceph-mon[32751]: pgmap v13829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:33.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:32 smithi090 ceph-mon[29180]: pgmap v13829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:35.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:34 smithi119 ceph-mon[32752]: pgmap v13830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:35.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:34 smithi167 ceph-mon[32751]: pgmap v13830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:35.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:34 smithi090 ceph-mon[29180]: pgmap v13830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:36.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:35 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 24797.364937 secs 2024-04-27T01:28:36.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:35 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 24797.364937 secs 2024-04-27T01:28:36.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:35 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 24797.364937 secs 2024-04-27T01:28:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:36 smithi119 ceph-mon[32752]: pgmap v13831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:37.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24798.178330 secs 2024-04-27T01:28:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:36 smithi167 ceph-mon[32751]: pgmap v13831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24798.178330 secs 2024-04-27T01:28:37.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:36 smithi090 ceph-mon[29180]: pgmap v13831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:37.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24798.178330 secs 2024-04-27T01:28:39.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:38 smithi119 ceph-mon[32752]: pgmap v13832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:39.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:38 smithi167 ceph-mon[32751]: pgmap v13832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:39.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:38 smithi090 ceph-mon[29180]: pgmap v13832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:41.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:40 smithi119 ceph-mon[32752]: pgmap v13833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:41.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:40 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 24802.365023 secs 2024-04-27T01:28:41.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:40 smithi119 ceph-mon[32752]: slow request 182.533634 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:28:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:40 smithi167 ceph-mon[32751]: pgmap v13833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:40 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 24802.365023 secs 2024-04-27T01:28:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:40 smithi167 ceph-mon[32751]: slow request 182.533634 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:28:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:40 smithi090 ceph-mon[29180]: pgmap v13833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:41.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:40 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 24802.365023 secs 2024-04-27T01:28:41.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:40 smithi090 ceph-mon[29180]: slow request 182.533634 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:28:41.940 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24803.178397 secs 2024-04-27T01:28:42.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24803.178397 secs 2024-04-27T01:28:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24803.178397 secs 2024-04-27T01:28:43.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:42 smithi119 ceph-mon[32752]: pgmap v13834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:43.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:42 smithi167 ceph-mon[32751]: pgmap v13834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:43.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:42 smithi090 ceph-mon[29180]: pgmap v13834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:44.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:28:44.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:28:44.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:28:44.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:28:44.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:28:44.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:28:45.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:44 smithi119 ceph-mon[32752]: pgmap v13835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:45.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:44 smithi167 ceph-mon[32751]: pgmap v13835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:45.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:44 smithi090 ceph-mon[29180]: pgmap v13835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:46.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24807.365117 secs 2024-04-27T01:28:46.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24807.365117 secs 2024-04-27T01:28:46.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24807.365117 secs 2024-04-27T01:28:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:46 smithi119 ceph-mon[32752]: pgmap v13836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24808.178498 secs 2024-04-27T01:28:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:46 smithi167 ceph-mon[32751]: pgmap v13836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24808.178498 secs 2024-04-27T01:28:47.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:46 smithi090 ceph-mon[29180]: pgmap v13836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:28:47.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24808.178498 secs 2024-04-27T01:28:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:28:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:28:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:28:48.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:28:48.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:28:48.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:28:48.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:28:48.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:28:48.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:28:48.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:28:48.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:28:48.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:28:49.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:48 smithi119 ceph-mon[32752]: pgmap v13837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:48 smithi119 ceph-mon[32752]: pgmap v13838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:28:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:48 smithi119 ceph-mon[32752]: pgmap v13839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:28:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:48 smithi167 ceph-mon[32751]: pgmap v13837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:48 smithi167 ceph-mon[32751]: pgmap v13838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:28:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:48 smithi167 ceph-mon[32751]: pgmap v13839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:28:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:48 smithi090 ceph-mon[29180]: pgmap v13837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:28:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:48 smithi090 ceph-mon[29180]: pgmap v13838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:28:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:48 smithi090 ceph-mon[29180]: pgmap v13839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:28:51.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:50 smithi119 ceph-mon[32752]: pgmap v13840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:51.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24812.365176 secs 2024-04-27T01:28:51.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:50 smithi167 ceph-mon[32751]: pgmap v13840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:51.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24812.365176 secs 2024-04-27T01:28:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:50 smithi090 ceph-mon[29180]: pgmap v13840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24812.365176 secs 2024-04-27T01:28:52.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24813.178566 secs 2024-04-27T01:28:52.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24813.178566 secs 2024-04-27T01:28:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24813.178566 secs 2024-04-27T01:28:53.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:52 smithi119 ceph-mon[32752]: pgmap v13841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:53.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:52 smithi167 ceph-mon[32751]: pgmap v13841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:53.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:52 smithi090 ceph-mon[29180]: pgmap v13841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:55.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:54 smithi119 ceph-mon[32752]: pgmap v13842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:55.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:54 smithi167 ceph-mon[32751]: pgmap v13842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:55.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:54 smithi090 ceph-mon[29180]: pgmap v13842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:56.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24817.365264 secs 2024-04-27T01:28:56.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24817.365264 secs 2024-04-27T01:28:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24817.365264 secs 2024-04-27T01:28:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:56 smithi119 ceph-mon[32752]: pgmap v13843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24818.178687 secs 2024-04-27T01:28:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:56 smithi167 ceph-mon[32751]: pgmap v13843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24818.178687 secs 2024-04-27T01:28:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:56 smithi090 ceph-mon[29180]: pgmap v13843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:28:57.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24818.178687 secs 2024-04-27T01:28:59.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:28:58 smithi119 ceph-mon[32752]: pgmap v13844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:28:59.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:28:58 smithi167 ceph-mon[32751]: pgmap v13844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:28:59.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:28:58 smithi090 ceph-mon[29180]: pgmap v13844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:29:01.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:00 smithi119 ceph-mon[32752]: pgmap v13845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:01.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24822.365324 secs 2024-04-27T01:29:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:00 smithi167 ceph-mon[32751]: pgmap v13845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24822.365324 secs 2024-04-27T01:29:01.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:00 smithi090 ceph-mon[29180]: pgmap v13845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:01.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24822.365324 secs 2024-04-27T01:29:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24823.178756 secs 2024-04-27T01:29:02.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24823.178756 secs 2024-04-27T01:29:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24823.178756 secs 2024-04-27T01:29:03.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:02 smithi119 ceph-mon[32752]: pgmap v13846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:03.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:02 smithi167 ceph-mon[32751]: pgmap v13846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:03.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:02 smithi090 ceph-mon[29180]: pgmap v13846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:04 smithi119 ceph-mon[32752]: pgmap v13847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:04 smithi167 ceph-mon[32751]: pgmap v13847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:05.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:04 smithi090 ceph-mon[29180]: pgmap v13847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:06.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24827.365426 secs 2024-04-27T01:29:06.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24827.365426 secs 2024-04-27T01:29:06.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24827.365426 secs 2024-04-27T01:29:07.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:06 smithi119 ceph-mon[32752]: pgmap v13848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24828.178883 secs 2024-04-27T01:29:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:06 smithi167 ceph-mon[32751]: pgmap v13848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24828.178883 secs 2024-04-27T01:29:07.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:06 smithi090 ceph-mon[29180]: pgmap v13848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:07.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24828.178883 secs 2024-04-27T01:29:09.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:08 smithi119 ceph-mon[32752]: pgmap v13849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:09.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:08 smithi167 ceph-mon[32751]: pgmap v13849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:09.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:08 smithi090 ceph-mon[29180]: pgmap v13849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:11.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:10 smithi119 ceph-mon[32752]: pgmap v13850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:11.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24832.365513 secs 2024-04-27T01:29:11.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:10 smithi167 ceph-mon[32751]: pgmap v13850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:11.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24832.365513 secs 2024-04-27T01:29:11.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:10 smithi090 ceph-mon[29180]: pgmap v13850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:11.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24832.365513 secs 2024-04-27T01:29:12.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24833.178959 secs 2024-04-27T01:29:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24833.178959 secs 2024-04-27T01:29:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24833.178959 secs 2024-04-27T01:29:13.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:12 smithi119 ceph-mon[32752]: pgmap v13851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:13.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:12 smithi167 ceph-mon[32751]: pgmap v13851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:13.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:12 smithi090 ceph-mon[29180]: pgmap v13851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:15.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:14 smithi119 ceph-mon[32752]: pgmap v13852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:15.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:14 smithi167 ceph-mon[32751]: pgmap v13852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:15.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:14 smithi090 ceph-mon[29180]: pgmap v13852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:16.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24837.365597 secs 2024-04-27T01:29:16.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24837.365597 secs 2024-04-27T01:29:16.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24837.365597 secs 2024-04-27T01:29:17.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:16 smithi119 ceph-mon[32752]: pgmap v13853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24838.179078 secs 2024-04-27T01:29:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:16 smithi167 ceph-mon[32751]: pgmap v13853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24838.179078 secs 2024-04-27T01:29:17.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:16 smithi090 ceph-mon[29180]: pgmap v13853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:17.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24838.179078 secs 2024-04-27T01:29:19.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:18 smithi119 ceph-mon[32752]: pgmap v13854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:19.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:18 smithi167 ceph-mon[32751]: pgmap v13854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:19.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:18 smithi090 ceph-mon[29180]: pgmap v13854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:21.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:20 smithi167 ceph-mon[32751]: pgmap v13855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:21.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24842.365683 secs 2024-04-27T01:29:21.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:20 smithi090 ceph-mon[29180]: pgmap v13855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:21.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24842.365683 secs 2024-04-27T01:29:21.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:20 smithi119 ceph-mon[32752]: pgmap v13855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:21.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24842.365683 secs 2024-04-27T01:29:22.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24843.179195 secs 2024-04-27T01:29:22.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24843.179195 secs 2024-04-27T01:29:22.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24843.179195 secs 2024-04-27T01:29:23.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:22 smithi167 ceph-mon[32751]: pgmap v13856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:23.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:22 smithi090 ceph-mon[29180]: pgmap v13856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:23.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:22 smithi119 ceph-mon[32752]: pgmap v13856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:25.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:24 smithi167 ceph-mon[32751]: pgmap v13857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:25.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:24 smithi090 ceph-mon[29180]: pgmap v13857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:25.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:24 smithi119 ceph-mon[32752]: pgmap v13857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:26.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24847.365742 secs 2024-04-27T01:29:26.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24847.365742 secs 2024-04-27T01:29:26.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24847.365742 secs 2024-04-27T01:29:27.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:26 smithi090 ceph-mon[29180]: pgmap v13858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24848.179321 secs 2024-04-27T01:29:27.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:26 smithi119 ceph-mon[32752]: pgmap v13858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:27.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24848.179321 secs 2024-04-27T01:29:27.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:26 smithi167 ceph-mon[32751]: pgmap v13858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:27.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24848.179321 secs 2024-04-27T01:29:29.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:28 smithi090 ceph-mon[29180]: pgmap v13859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:29.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:28 smithi119 ceph-mon[32752]: pgmap v13859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:29.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:28 smithi167 ceph-mon[32751]: pgmap v13859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:31.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:30 smithi167 ceph-mon[32751]: pgmap v13860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:31.105 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24852.365829 secs 2024-04-27T01:29:31.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:30 smithi090 ceph-mon[29180]: pgmap v13860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:31.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24852.365829 secs 2024-04-27T01:29:31.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:30 smithi119 ceph-mon[32752]: pgmap v13860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:31.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24852.365829 secs 2024-04-27T01:29:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24853.179440 secs 2024-04-27T01:29:32.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24853.179440 secs 2024-04-27T01:29:32.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24853.179440 secs 2024-04-27T01:29:33.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:32 smithi090 ceph-mon[29180]: pgmap v13861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:33.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:32 smithi119 ceph-mon[32752]: pgmap v13861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:33.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:32 smithi167 ceph-mon[32751]: pgmap v13861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:34.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:33 smithi090 ceph-mon[29180]: pgmap v13862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:34.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:33 smithi119 ceph-mon[32752]: pgmap v13862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:34.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:33 smithi167 ceph-mon[32751]: pgmap v13862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:36.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24857.365889 secs 2024-04-27T01:29:36.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24857.365889 secs 2024-04-27T01:29:36.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24857.365889 secs 2024-04-27T01:29:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:36 smithi119 ceph-mon[32752]: pgmap v13863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:37.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24858.179528 secs 2024-04-27T01:29:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:36 smithi167 ceph-mon[32751]: pgmap v13863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24858.179528 secs 2024-04-27T01:29:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:36 smithi090 ceph-mon[29180]: pgmap v13863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24858.179528 secs 2024-04-27T01:29:39.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:38 smithi119 ceph-mon[32752]: pgmap v13864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:39.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:38 smithi167 ceph-mon[32751]: pgmap v13864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:39.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:38 smithi090 ceph-mon[29180]: pgmap v13864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:41.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:40 smithi119 ceph-mon[32752]: pgmap v13865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:41.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24862.365975 secs 2024-04-27T01:29:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:40 smithi167 ceph-mon[32751]: pgmap v13865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24862.365975 secs 2024-04-27T01:29:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:40 smithi090 ceph-mon[29180]: pgmap v13865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:41.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24862.365975 secs 2024-04-27T01:29:42.012 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24863.179628 secs 2024-04-27T01:29:42.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24863.179628 secs 2024-04-27T01:29:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24863.179628 secs 2024-04-27T01:29:43.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:42 smithi119 ceph-mon[32752]: pgmap v13866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:43.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:42 smithi167 ceph-mon[32751]: pgmap v13866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:43.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:42 smithi090 ceph-mon[29180]: pgmap v13866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:45.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:44 smithi119 ceph-mon[32752]: pgmap v13867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:45.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:44 smithi167 ceph-mon[32751]: pgmap v13867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:45.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:44 smithi090 ceph-mon[29180]: pgmap v13867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:29:46.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24867.366081 secs 2024-04-27T01:29:46.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24867.366081 secs 2024-04-27T01:29:46.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24867.366081 secs 2024-04-27T01:29:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:46 smithi119 ceph-mon[32752]: pgmap v13868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24868.179707 secs 2024-04-27T01:29:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:46 smithi167 ceph-mon[32751]: pgmap v13868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24868.179707 secs 2024-04-27T01:29:47.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:46 smithi090 ceph-mon[29180]: pgmap v13868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:47.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24868.179707 secs 2024-04-27T01:29:49.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:48 smithi119 ceph-mon[32752]: pgmap v13869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:29:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:29:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:29:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:29:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:48 smithi167 ceph-mon[32751]: pgmap v13869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:29:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:29:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:29:49.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:29:49.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:48 smithi090 ceph-mon[29180]: pgmap v13869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:29:49.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:29:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:29:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:29:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:29:50.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:49 smithi119 ceph-mon[32752]: pgmap v13870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:29:50.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:49 smithi119 ceph-mon[32752]: pgmap v13871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:50.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:49 smithi167 ceph-mon[32751]: pgmap v13870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:29:50.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:49 smithi167 ceph-mon[32751]: pgmap v13871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:50.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:49 smithi090 ceph-mon[29180]: pgmap v13870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:29:50.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:49 smithi090 ceph-mon[29180]: pgmap v13871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:51.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24872.366139 secs 2024-04-27T01:29:51.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24872.366139 secs 2024-04-27T01:29:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24872.366139 secs 2024-04-27T01:29:52.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:51 smithi119 ceph-mon[32752]: pgmap v13872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:52.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24873.179823 secs 2024-04-27T01:29:52.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:51 smithi167 ceph-mon[32751]: pgmap v13872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:52.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24873.179823 secs 2024-04-27T01:29:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:51 smithi090 ceph-mon[29180]: pgmap v13872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24873.179823 secs 2024-04-27T01:29:54.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:53 smithi119 ceph-mon[32752]: pgmap v13873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:29:54.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:53 smithi167 ceph-mon[32751]: pgmap v13873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:29:54.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:53 smithi090 ceph-mon[29180]: pgmap v13873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:29:56.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:55 smithi119 ceph-mon[32752]: pgmap v13874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:56.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24877.366225 secs 2024-04-27T01:29:56.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:55 smithi167 ceph-mon[32751]: pgmap v13874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:56.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24877.366225 secs 2024-04-27T01:29:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:55 smithi090 ceph-mon[29180]: pgmap v13874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:56.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24877.366225 secs 2024-04-27T01:29:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24878.179961 secs 2024-04-27T01:29:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24878.179961 secs 2024-04-27T01:29:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24878.179961 secs 2024-04-27T01:29:58.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:57 smithi119 ceph-mon[32752]: pgmap v13875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:58.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:57 smithi167 ceph-mon[32751]: pgmap v13875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:58.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:57 smithi090 ceph-mon[29180]: pgmap v13875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:29:59.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:29:59 smithi090 ceph-mon[29180]: pgmap v13876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:30:00.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:29:59 smithi119 ceph-mon[32752]: pgmap v13876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:30:00.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:29:59 smithi167 ceph-mon[32751]: pgmap v13876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:30:01.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:30:01.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24882.366309 secs 2024-04-27T01:30:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:30:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24882.366309 secs 2024-04-27T01:30:01.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:30:01.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24882.366309 secs 2024-04-27T01:30:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:01 smithi119 ceph-mon[32752]: pgmap v13877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24883.180080 secs 2024-04-27T01:30:02.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:01 smithi167 ceph-mon[32751]: pgmap v13877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:02.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24883.180080 secs 2024-04-27T01:30:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:01 smithi090 ceph-mon[29180]: pgmap v13877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:02.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24883.180080 secs 2024-04-27T01:30:04.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:03 smithi119 ceph-mon[32752]: pgmap v13878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:04.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:03 smithi167 ceph-mon[32751]: pgmap v13878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:04.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:03 smithi090 ceph-mon[29180]: pgmap v13878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:06.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:05 smithi119 ceph-mon[32752]: pgmap v13879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:06.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24887.366398 secs 2024-04-27T01:30:06.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:05 smithi167 ceph-mon[32751]: pgmap v13879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:06.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24887.366398 secs 2024-04-27T01:30:06.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:05 smithi090 ceph-mon[29180]: pgmap v13879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:06.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24887.366398 secs 2024-04-27T01:30:07.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24888.180202 secs 2024-04-27T01:30:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24888.180202 secs 2024-04-27T01:30:07.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24888.180202 secs 2024-04-27T01:30:08.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:07 smithi119 ceph-mon[32752]: pgmap v13880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:08.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:07 smithi167 ceph-mon[32751]: pgmap v13880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:08.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:07 smithi090 ceph-mon[29180]: pgmap v13880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:10.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:09 smithi119 ceph-mon[32752]: pgmap v13881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:10.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:09 smithi167 ceph-mon[32751]: pgmap v13881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:10.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:09 smithi090 ceph-mon[29180]: pgmap v13881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:11.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24892.366482 secs 2024-04-27T01:30:11.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24892.366482 secs 2024-04-27T01:30:11.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24892.366482 secs 2024-04-27T01:30:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:11 smithi167 ceph-mon[32751]: pgmap v13882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24893.180313 secs 2024-04-27T01:30:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:11 smithi090 ceph-mon[29180]: pgmap v13882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24893.180313 secs 2024-04-27T01:30:12.189 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:11 smithi119 ceph-mon[32752]: pgmap v13882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:12.190 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24893.180313 secs 2024-04-27T01:30:14.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:13 smithi167 ceph-mon[32751]: pgmap v13883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:14.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:13 smithi090 ceph-mon[29180]: pgmap v13883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:14.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:13 smithi119 ceph-mon[32752]: pgmap v13883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:16.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:15 smithi167 ceph-mon[32751]: pgmap v13884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:16.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24897.366543 secs 2024-04-27T01:30:16.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:15 smithi090 ceph-mon[29180]: pgmap v13884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:16.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24897.366543 secs 2024-04-27T01:30:16.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:15 smithi119 ceph-mon[32752]: pgmap v13884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:16.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24897.366543 secs 2024-04-27T01:30:17.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24898.180431 secs 2024-04-27T01:30:17.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24898.180431 secs 2024-04-27T01:30:17.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24898.180431 secs 2024-04-27T01:30:18.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:17 smithi090 ceph-mon[29180]: pgmap v13885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:18.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:17 smithi119 ceph-mon[32752]: pgmap v13885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:18.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:17 smithi167 ceph-mon[32751]: pgmap v13885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:20.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:19 smithi090 ceph-mon[29180]: pgmap v13886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:20.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:19 smithi119 ceph-mon[32752]: pgmap v13886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:20.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:19 smithi167 ceph-mon[32751]: pgmap v13886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:21.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24902.366627 secs 2024-04-27T01:30:21.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24902.366627 secs 2024-04-27T01:30:21.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24902.366627 secs 2024-04-27T01:30:22.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:21 smithi090 ceph-mon[29180]: pgmap v13887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24903.180500 secs 2024-04-27T01:30:22.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:21 smithi119 ceph-mon[32752]: pgmap v13887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:22.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24903.180500 secs 2024-04-27T01:30:22.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:21 smithi167 ceph-mon[32751]: pgmap v13887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:22.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24903.180500 secs 2024-04-27T01:30:24.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:23 smithi090 ceph-mon[29180]: pgmap v13888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:24.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:23 smithi119 ceph-mon[32752]: pgmap v13888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:24.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:23 smithi167 ceph-mon[32751]: pgmap v13888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:26.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:25 smithi090 ceph-mon[29180]: pgmap v13889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:26.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24907.366713 secs 2024-04-27T01:30:26.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:25 smithi119 ceph-mon[32752]: pgmap v13889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:26.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24907.366713 secs 2024-04-27T01:30:26.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:25 smithi167 ceph-mon[32751]: pgmap v13889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:26.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24907.366713 secs 2024-04-27T01:30:27.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24908.180583 secs 2024-04-27T01:30:27.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24908.180583 secs 2024-04-27T01:30:27.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24908.180583 secs 2024-04-27T01:30:28.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:27 smithi090 ceph-mon[29180]: pgmap v13890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:28.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:27 smithi119 ceph-mon[32752]: pgmap v13890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:28.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:27 smithi167 ceph-mon[32751]: pgmap v13890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:29.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:28 smithi090 ceph-mon[29180]: pgmap v13891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:29.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:28 smithi119 ceph-mon[32752]: pgmap v13891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:29.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:28 smithi167 ceph-mon[32751]: pgmap v13891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:31.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24912.366805 secs 2024-04-27T01:30:31.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:31 smithi119 ceph-mon[32752]: pgmap v13892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:31.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24913.180647 secs 2024-04-27T01:30:31.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24912.366805 secs 2024-04-27T01:30:31.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:31 smithi167 ceph-mon[32751]: pgmap v13892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:31.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24913.180647 secs 2024-04-27T01:30:31.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24912.366805 secs 2024-04-27T01:30:31.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:31 smithi090 ceph-mon[29180]: pgmap v13892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:31.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24913.180647 secs 2024-04-27T01:30:33.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:33 smithi119 ceph-mon[32752]: pgmap v13893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:33.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:33 smithi167 ceph-mon[32751]: pgmap v13893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:33.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:33 smithi090 ceph-mon[29180]: pgmap v13893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:35.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:35 smithi119 ceph-mon[32752]: pgmap v13894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:35.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:35 smithi167 ceph-mon[32751]: pgmap v13894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:35.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:35 smithi090 ceph-mon[29180]: pgmap v13894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:30:36.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24917.366864 secs 2024-04-27T01:30:36.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24918.180760 secs 2024-04-27T01:30:36.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24917.366864 secs 2024-04-27T01:30:36.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24918.180760 secs 2024-04-27T01:30:36.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24917.366864 secs 2024-04-27T01:30:36.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24918.180760 secs 2024-04-27T01:30:37.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:37 smithi119 ceph-mon[32752]: pgmap v13895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:37.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:37 smithi167 ceph-mon[32751]: pgmap v13895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:37.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:37 smithi090 ceph-mon[29180]: pgmap v13895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:30:37.861 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:30:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:30:37.646+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:30:37.861 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:30:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:30:37.646+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:30:37.861 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:30:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:30:37.646+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:30:37.861 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:30:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:30:37.646+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:30:37.861 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:30:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:30:37.646+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:30:39.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:39 smithi119 ceph-mon[32752]: pgmap v13896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:39.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:39 smithi167 ceph-mon[32751]: pgmap v13896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:39.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:39 smithi090 ceph-mon[29180]: pgmap v13896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:41.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24922.366923 secs 2024-04-27T01:30:41.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:41 smithi119 ceph-mon[32752]: pgmap v13897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:41.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24923.180854 secs 2024-04-27T01:30:41.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24922.366923 secs 2024-04-27T01:30:41.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:41 smithi167 ceph-mon[32751]: pgmap v13897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:41.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24923.180854 secs 2024-04-27T01:30:41.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24922.366923 secs 2024-04-27T01:30:41.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:41 smithi090 ceph-mon[29180]: pgmap v13897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:41.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24923.180854 secs 2024-04-27T01:30:43.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:43 smithi119 ceph-mon[32752]: pgmap v13898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:43.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:43 smithi167 ceph-mon[32751]: pgmap v13898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:43.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:43 smithi090 ceph-mon[29180]: pgmap v13898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:45.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:45 smithi119 ceph-mon[32752]: pgmap v13899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:45.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:45 smithi167 ceph-mon[32751]: pgmap v13899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:45.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:45 smithi090 ceph-mon[29180]: pgmap v13899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:46.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24927.366984 secs 2024-04-27T01:30:46.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24928.180979 secs 2024-04-27T01:30:46.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24927.366984 secs 2024-04-27T01:30:46.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24928.180979 secs 2024-04-27T01:30:46.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24927.366984 secs 2024-04-27T01:30:46.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24928.180979 secs 2024-04-27T01:30:47.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:47 smithi119 ceph-mon[32752]: pgmap v13900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:47.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:47 smithi167 ceph-mon[32751]: pgmap v13900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:47.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:47 smithi090 ceph-mon[29180]: pgmap v13900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:49.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:49 smithi119 ceph-mon[32752]: pgmap v13901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:49.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:30:49.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:30:49.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:30:49.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:30:49.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:49 smithi167 ceph-mon[32751]: pgmap v13901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:49.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:30:49.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:30:49.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:30:49.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:30:49.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:49 smithi090 ceph-mon[29180]: pgmap v13901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:30:49.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:30:49.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:30:49.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:30:49.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:30:50.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:50 smithi119 ceph-mon[32752]: pgmap v13902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:30:50.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:50 smithi119 ceph-mon[32752]: pgmap v13903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:50.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:50 smithi167 ceph-mon[32751]: pgmap v13902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:30:50.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:50 smithi167 ceph-mon[32751]: pgmap v13903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:50.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:50 smithi090 ceph-mon[29180]: pgmap v13902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:30:50.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:50 smithi090 ceph-mon[29180]: pgmap v13903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:51.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24932.367060 secs 2024-04-27T01:30:51.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24933.181100 secs 2024-04-27T01:30:51.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24932.367060 secs 2024-04-27T01:30:51.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24933.181100 secs 2024-04-27T01:30:51.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24932.367060 secs 2024-04-27T01:30:51.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24933.181100 secs 2024-04-27T01:30:52.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:52 smithi119 ceph-mon[32752]: pgmap v13904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:52.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:52 smithi167 ceph-mon[32751]: pgmap v13904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:52.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:52 smithi090 ceph-mon[29180]: pgmap v13904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:54.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:54 smithi167 ceph-mon[32751]: pgmap v13905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:30:54.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:54 smithi090 ceph-mon[29180]: pgmap v13905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:30:55.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:54 smithi119 ceph-mon[32752]: pgmap v13905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:30:56.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24937.367119 secs 2024-04-27T01:30:56.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:56 smithi167 ceph-mon[32751]: pgmap v13906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:56.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24938.181236 secs 2024-04-27T01:30:56.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24937.367119 secs 2024-04-27T01:30:56.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:56 smithi090 ceph-mon[29180]: pgmap v13906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:56.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24938.181236 secs 2024-04-27T01:30:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24937.367119 secs 2024-04-27T01:30:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:56 smithi119 ceph-mon[32752]: pgmap v13906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:30:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24938.181236 secs 2024-04-27T01:30:58.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:30:58 smithi167 ceph-mon[32751]: pgmap v13907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:30:58.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:30:58 smithi090 ceph-mon[29180]: pgmap v13907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:30:59.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:30:58 smithi119 ceph-mon[32752]: pgmap v13907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:31:00.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:00 smithi167 ceph-mon[32751]: pgmap v13908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:31:00.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:00 smithi090 ceph-mon[29180]: pgmap v13908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:31:01.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:00 smithi119 ceph-mon[32752]: pgmap v13908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:31:01.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24942.367208 secs 2024-04-27T01:31:01.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24943.181371 secs 2024-04-27T01:31:01.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24942.367208 secs 2024-04-27T01:31:01.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24943.181371 secs 2024-04-27T01:31:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24942.367208 secs 2024-04-27T01:31:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24943.181371 secs 2024-04-27T01:31:02.761 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:02 smithi090 ceph-mon[29180]: pgmap v13909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:02.901 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:02 smithi167 ceph-mon[32751]: pgmap v13909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:03.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:02 smithi119 ceph-mon[32752]: pgmap v13909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:04 smithi090 ceph-mon[29180]: pgmap v13910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:04 smithi119 ceph-mon[32752]: pgmap v13910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:04 smithi167 ceph-mon[32751]: pgmap v13910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:06.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24947.367293 secs 2024-04-27T01:31:06.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:06 smithi090 ceph-mon[29180]: pgmap v13911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:06.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24948.181428 secs 2024-04-27T01:31:07.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24947.367293 secs 2024-04-27T01:31:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:06 smithi119 ceph-mon[32752]: pgmap v13911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24948.181428 secs 2024-04-27T01:31:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24947.367293 secs 2024-04-27T01:31:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:06 smithi167 ceph-mon[32751]: pgmap v13911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24948.181428 secs 2024-04-27T01:31:08.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:08 smithi090 ceph-mon[29180]: pgmap v13912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:09.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:08 smithi119 ceph-mon[32752]: pgmap v13912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:09.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:08 smithi167 ceph-mon[32751]: pgmap v13912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:10.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:10 smithi090 ceph-mon[29180]: pgmap v13913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:11.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:10 smithi119 ceph-mon[32752]: pgmap v13913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:11.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:10 smithi167 ceph-mon[32751]: pgmap v13913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:11.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24952.367380 secs 2024-04-27T01:31:11.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:11 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 24953.181564 secs 2024-04-27T01:31:11.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:11 smithi090 ceph-mon[29180]: slow request 2884.062697 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:31:12.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24952.367380 secs 2024-04-27T01:31:12.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:11 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 24953.181564 secs 2024-04-27T01:31:12.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:11 smithi119 ceph-mon[32752]: slow request 2884.062697 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:31:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24952.367380 secs 2024-04-27T01:31:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:11 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 24953.181564 secs 2024-04-27T01:31:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:11 smithi167 ceph-mon[32751]: slow request 2884.062697 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:31:12.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:12 smithi167 ceph-mon[32751]: pgmap v13914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:12.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:12 smithi090 ceph-mon[29180]: pgmap v13914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:13.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:12 smithi119 ceph-mon[32752]: pgmap v13914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:14.840 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:14 smithi119 ceph-mon[32752]: pgmap v13915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:14.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:14 smithi090 ceph-mon[29180]: pgmap v13915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:15.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:14 smithi167 ceph-mon[32751]: pgmap v13915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:16.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24957.367467 secs 2024-04-27T01:31:16.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:16 smithi090 ceph-mon[29180]: pgmap v13916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:31:16.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24958.181679 secs 2024-04-27T01:31:17.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24957.367467 secs 2024-04-27T01:31:17.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:16 smithi119 ceph-mon[32752]: pgmap v13916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:31:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24958.181679 secs 2024-04-27T01:31:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24957.367467 secs 2024-04-27T01:31:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:16 smithi167 ceph-mon[32751]: pgmap v13916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:31:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24958.181679 secs 2024-04-27T01:31:18.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:18 smithi090 ceph-mon[29180]: pgmap v13917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:19.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:18 smithi119 ceph-mon[32752]: pgmap v13917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:19.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:18 smithi167 ceph-mon[32751]: pgmap v13917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:20.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:20 smithi090 ceph-mon[29180]: pgmap v13918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:21.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:20 smithi119 ceph-mon[32752]: pgmap v13918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:21.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:20 smithi167 ceph-mon[32751]: pgmap v13918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:21.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24962.367552 secs 2024-04-27T01:31:21.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24963.181748 secs 2024-04-27T01:31:22.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24962.367552 secs 2024-04-27T01:31:22.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24963.181748 secs 2024-04-27T01:31:22.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24962.367552 secs 2024-04-27T01:31:22.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24963.181748 secs 2024-04-27T01:31:22.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:22 smithi090 ceph-mon[29180]: pgmap v13919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:23.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:22 smithi119 ceph-mon[32752]: pgmap v13919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:23.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:22 smithi167 ceph-mon[32751]: pgmap v13919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:24.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:24 smithi090 ceph-mon[29180]: pgmap v13920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:25.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:24 smithi119 ceph-mon[32752]: pgmap v13920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:25.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:24 smithi167 ceph-mon[32751]: pgmap v13920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:26.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24967.367639 secs 2024-04-27T01:31:26.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:26 smithi090 ceph-mon[29180]: pgmap v13921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:31:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24968.181811 secs 2024-04-27T01:31:27.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24967.367639 secs 2024-04-27T01:31:27.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:26 smithi119 ceph-mon[32752]: pgmap v13921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:31:27.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24968.181811 secs 2024-04-27T01:31:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24967.367639 secs 2024-04-27T01:31:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:26 smithi167 ceph-mon[32751]: pgmap v13921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:31:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24968.181811 secs 2024-04-27T01:31:28.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:28 smithi090 ceph-mon[29180]: pgmap v13922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:29.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:28 smithi119 ceph-mon[32752]: pgmap v13922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:29.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:28 smithi167 ceph-mon[32751]: pgmap v13922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:31.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:30 smithi119 ceph-mon[32752]: pgmap v13923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:31.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:30 smithi167 ceph-mon[32751]: pgmap v13923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:31.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:30 smithi090 ceph-mon[29180]: pgmap v13923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:31.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24972.367728 secs 2024-04-27T01:31:31.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24973.181931 secs 2024-04-27T01:31:32.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24972.367728 secs 2024-04-27T01:31:32.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24973.181931 secs 2024-04-27T01:31:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24972.367728 secs 2024-04-27T01:31:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24973.181931 secs 2024-04-27T01:31:33.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:32 smithi119 ceph-mon[32752]: pgmap v13924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:33.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:32 smithi167 ceph-mon[32751]: pgmap v13924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:33.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:32 smithi090 ceph-mon[29180]: pgmap v13924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:35.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:34 smithi119 ceph-mon[32752]: pgmap v13925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:35.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:34 smithi167 ceph-mon[32751]: pgmap v13925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:35.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:34 smithi090 ceph-mon[29180]: pgmap v13925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24977.367815 secs 2024-04-27T01:31:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:36 smithi119 ceph-mon[32752]: pgmap v13926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24978.182077 secs 2024-04-27T01:31:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24977.367815 secs 2024-04-27T01:31:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:36 smithi167 ceph-mon[32751]: pgmap v13926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24978.182077 secs 2024-04-27T01:31:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24977.367815 secs 2024-04-27T01:31:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:36 smithi090 ceph-mon[29180]: pgmap v13926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24978.182077 secs 2024-04-27T01:31:39.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:38 smithi119 ceph-mon[32752]: pgmap v13927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:39.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:38 smithi167 ceph-mon[32751]: pgmap v13927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:39.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:38 smithi090 ceph-mon[29180]: pgmap v13927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:41.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:40 smithi119 ceph-mon[32752]: pgmap v13928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:41.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:40 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 24982.367902 secs 2024-04-27T01:31:41.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:40 smithi119 ceph-mon[32752]: slow request 362.536513 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:31:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:40 smithi167 ceph-mon[32751]: pgmap v13928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:40 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 24982.367902 secs 2024-04-27T01:31:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:40 smithi167 ceph-mon[32751]: slow request 362.536513 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:31:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:40 smithi090 ceph-mon[29180]: pgmap v13928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:40 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 24982.367902 secs 2024-04-27T01:31:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:40 smithi090 ceph-mon[29180]: slow request 362.536513 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:31:42.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24983.182187 secs 2024-04-27T01:31:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24983.182187 secs 2024-04-27T01:31:42.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24983.182187 secs 2024-04-27T01:31:43.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:42 smithi119 ceph-mon[32752]: pgmap v13929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:43.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:42 smithi167 ceph-mon[32751]: pgmap v13929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:43.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:42 smithi090 ceph-mon[29180]: pgmap v13929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:44.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:44 smithi090 ceph-mon[29180]: pgmap v13930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:45.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:44 smithi119 ceph-mon[32752]: pgmap v13930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:45.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:44 smithi167 ceph-mon[32751]: pgmap v13930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:46.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24987.367989 secs 2024-04-27T01:31:46.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24987.367989 secs 2024-04-27T01:31:46.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24987.367989 secs 2024-04-27T01:31:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:46 smithi119 ceph-mon[32752]: pgmap v13931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24988.182297 secs 2024-04-27T01:31:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:46 smithi167 ceph-mon[32751]: pgmap v13931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24988.182297 secs 2024-04-27T01:31:47.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:46 smithi090 ceph-mon[29180]: pgmap v13931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:47.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24988.182297 secs 2024-04-27T01:31:49.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:48 smithi119 ceph-mon[32752]: pgmap v13932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:49.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:48 smithi167 ceph-mon[32751]: pgmap v13932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:49.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:48 smithi090 ceph-mon[29180]: pgmap v13932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:49.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:31:50.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:31:50.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:31:50.904 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:50 smithi119 ceph-mon[32752]: pgmap v13933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:50.905 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:50.905 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:50.905 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:50.905 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:50.905 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24992.368060 secs 2024-04-27T01:31:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:50 smithi090 ceph-mon[29180]: pgmap v13933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24992.368060 secs 2024-04-27T01:31:51.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:50 smithi167 ceph-mon[32751]: pgmap v13933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:31:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:51.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24992.368060 secs 2024-04-27T01:31:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24993.182357 secs 2024-04-27T01:31:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:52.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24993.182357 secs 2024-04-27T01:31:52.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:52.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:52.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24993.182357 secs 2024-04-27T01:31:52.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:52.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: pgmap v13934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:53.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:31:53.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:31:53.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:31:53.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:31:53.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:31:53.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: pgmap v13935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T01:31:53.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: pgmap v13936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-27T01:31:53.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: pgmap v13934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:31:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:31:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:31:53.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:31:53.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:31:53.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: pgmap v13935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T01:31:53.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: pgmap v13936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-27T01:31:53.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: pgmap v13934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:31:53.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:31:53.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:31:53.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:31:53.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:31:53.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: pgmap v13935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T01:31:53.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: pgmap v13936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-27T01:31:53.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:31:56.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:55 smithi119 ceph-mon[32752]: pgmap v13937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:31:56.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 24997.368145 secs 2024-04-27T01:31:56.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:55 smithi167 ceph-mon[32751]: pgmap v13937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:31:56.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 24997.368145 secs 2024-04-27T01:31:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:55 smithi090 ceph-mon[29180]: pgmap v13937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:31:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 24997.368145 secs 2024-04-27T01:31:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 24998.182452 secs 2024-04-27T01:31:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 24998.182452 secs 2024-04-27T01:31:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 24998.182452 secs 2024-04-27T01:31:58.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:57 smithi119 ceph-mon[32752]: pgmap v13938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:31:58.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:57 smithi167 ceph-mon[32751]: pgmap v13938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:31:58.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:57 smithi090 ceph-mon[29180]: pgmap v13938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:00.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:31:59 smithi119 ceph-mon[32752]: pgmap v13939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:00.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:31:59 smithi167 ceph-mon[32751]: pgmap v13939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:00.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:31:59 smithi090 ceph-mon[29180]: pgmap v13939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:01.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25002.368233 secs 2024-04-27T01:32:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25002.368233 secs 2024-04-27T01:32:01.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25002.368233 secs 2024-04-27T01:32:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:01 smithi119 ceph-mon[32752]: pgmap v13940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25003.182570 secs 2024-04-27T01:32:02.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:01 smithi167 ceph-mon[32751]: pgmap v13940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:02.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25003.182570 secs 2024-04-27T01:32:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:01 smithi090 ceph-mon[29180]: pgmap v13940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25003.182570 secs 2024-04-27T01:32:04.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:03 smithi119 ceph-mon[32752]: pgmap v13941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:32:04.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:03 smithi167 ceph-mon[32751]: pgmap v13941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:32:04.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:03 smithi090 ceph-mon[29180]: pgmap v13941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:32:06.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:05 smithi119 ceph-mon[32752]: pgmap v13942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:06.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25007.368322 secs 2024-04-27T01:32:06.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:05 smithi167 ceph-mon[32751]: pgmap v13942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:06.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25007.368322 secs 2024-04-27T01:32:06.071 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:05 smithi090 ceph-mon[29180]: pgmap v13942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:06.072 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25007.368322 secs 2024-04-27T01:32:07.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25008.182673 secs 2024-04-27T01:32:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25008.182673 secs 2024-04-27T01:32:07.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25008.182673 secs 2024-04-27T01:32:08.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:07 smithi119 ceph-mon[32752]: pgmap v13943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:08.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:07 smithi167 ceph-mon[32751]: pgmap v13943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:08.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:07 smithi090 ceph-mon[29180]: pgmap v13943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:10.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:09 smithi167 ceph-mon[32751]: pgmap v13944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:10.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:09 smithi090 ceph-mon[29180]: pgmap v13944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:10.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:09 smithi119 ceph-mon[32752]: pgmap v13944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:11.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25012.368411 secs 2024-04-27T01:32:11.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25012.368411 secs 2024-04-27T01:32:11.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25012.368411 secs 2024-04-27T01:32:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:11 smithi167 ceph-mon[32751]: pgmap v13945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25013.182733 secs 2024-04-27T01:32:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:11 smithi090 ceph-mon[29180]: pgmap v13945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25013.182733 secs 2024-04-27T01:32:12.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:11 smithi119 ceph-mon[32752]: pgmap v13945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:12.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25013.182733 secs 2024-04-27T01:32:14.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:13 smithi167 ceph-mon[32751]: pgmap v13946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:14.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:13 smithi119 ceph-mon[32752]: pgmap v13946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:14.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:13 smithi090 ceph-mon[29180]: pgmap v13946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:32:15.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:15 smithi090 ceph-mon[29180]: pgmap v13947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:15.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:15 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25017.368499 secs 2024-04-27T01:32:15.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:15 smithi090 ceph-mon[29180]: slow request 23040.638634 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:32:16.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:15 smithi119 ceph-mon[32752]: pgmap v13947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:16.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:15 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25017.368499 secs 2024-04-27T01:32:16.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:15 smithi119 ceph-mon[32752]: slow request 23040.638634 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:32:16.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:15 smithi167 ceph-mon[32751]: pgmap v13947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:16.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:15 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25017.368499 secs 2024-04-27T01:32:16.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:15 smithi167 ceph-mon[32751]: slow request 23040.638634 seconds old, received at 2024-04-26T19:08:14.483269+0000: client_request(client.24733:267883 getattr AsXsFs #0x100000001fb 2024-04-26T19:08:14.482659+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:32:17.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25018.182850 secs 2024-04-27T01:32:17.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25018.182850 secs 2024-04-27T01:32:17.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25018.182850 secs 2024-04-27T01:32:18.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:17 smithi090 ceph-mon[29180]: pgmap v13948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:18.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:17 smithi119 ceph-mon[32752]: pgmap v13948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:18.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:17 smithi167 ceph-mon[32751]: pgmap v13948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:20.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:19 smithi090 ceph-mon[29180]: pgmap v13949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:20.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:19 smithi119 ceph-mon[32752]: pgmap v13949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:20.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:19 smithi167 ceph-mon[32751]: pgmap v13949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:21.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25022.368586 secs 2024-04-27T01:32:21.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25022.368586 secs 2024-04-27T01:32:21.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25022.368586 secs 2024-04-27T01:32:22.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:21 smithi090 ceph-mon[29180]: pgmap v13950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:22.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25023.182972 secs 2024-04-27T01:32:22.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:21 smithi119 ceph-mon[32752]: pgmap v13950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:22.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25023.182972 secs 2024-04-27T01:32:22.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:21 smithi167 ceph-mon[32751]: pgmap v13950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:22.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25023.182972 secs 2024-04-27T01:32:24.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:23 smithi090 ceph-mon[29180]: pgmap v13951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:24.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:23 smithi119 ceph-mon[32752]: pgmap v13951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:24.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:23 smithi167 ceph-mon[32751]: pgmap v13951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:26.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:25 smithi090 ceph-mon[29180]: pgmap v13952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:26.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25027.368673 secs 2024-04-27T01:32:26.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:25 smithi119 ceph-mon[32752]: pgmap v13952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:26.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25027.368673 secs 2024-04-27T01:32:26.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:25 smithi167 ceph-mon[32751]: pgmap v13952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:26.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25027.368673 secs 2024-04-27T01:32:27.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25028.183086 secs 2024-04-27T01:32:27.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25028.183086 secs 2024-04-27T01:32:27.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25028.183086 secs 2024-04-27T01:32:28.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:27 smithi090 ceph-mon[29180]: pgmap v13953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:28.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:27 smithi119 ceph-mon[32752]: pgmap v13953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:28.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:27 smithi167 ceph-mon[32751]: pgmap v13953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:30.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:29 smithi090 ceph-mon[29180]: pgmap v13954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:30.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:29 smithi119 ceph-mon[32752]: pgmap v13954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:30.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:29 smithi167 ceph-mon[32751]: pgmap v13954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:31.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25032.368731 secs 2024-04-27T01:32:31.203 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25032.368731 secs 2024-04-27T01:32:31.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25032.368731 secs 2024-04-27T01:32:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:31 smithi090 ceph-mon[29180]: pgmap v13955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25033.183182 secs 2024-04-27T01:32:32.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:31 smithi119 ceph-mon[32752]: pgmap v13955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:32.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25033.183182 secs 2024-04-27T01:32:32.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:31 smithi167 ceph-mon[32751]: pgmap v13955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:32.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25033.183182 secs 2024-04-27T01:32:33.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:32 smithi090 ceph-mon[29180]: pgmap v13956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:33.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:32 smithi119 ceph-mon[32752]: pgmap v13956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:33.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:32 smithi167 ceph-mon[32751]: pgmap v13956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:36.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:35 smithi119 ceph-mon[32752]: pgmap v13957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:36.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:35 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25037.368837 secs 2024-04-27T01:32:36.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:35 smithi119 ceph-mon[32752]: slow request 11520.702970 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:32:36.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:35 smithi167 ceph-mon[32751]: pgmap v13957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:36.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:35 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25037.368837 secs 2024-04-27T01:32:36.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:35 smithi167 ceph-mon[32751]: slow request 11520.702970 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:32:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:35 smithi090 ceph-mon[29180]: pgmap v13957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:36.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:35 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25037.368837 secs 2024-04-27T01:32:36.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:35 smithi090 ceph-mon[29180]: slow request 11520.702970 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:32:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25038.183304 secs 2024-04-27T01:32:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25038.183304 secs 2024-04-27T01:32:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25038.183304 secs 2024-04-27T01:32:38.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:37 smithi119 ceph-mon[32752]: pgmap v13958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:38.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:37 smithi167 ceph-mon[32751]: pgmap v13958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:38.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:37 smithi090 ceph-mon[29180]: pgmap v13958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:40.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:39 smithi119 ceph-mon[32752]: pgmap v13959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:40.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:39 smithi167 ceph-mon[32751]: pgmap v13959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:40.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:39 smithi090 ceph-mon[29180]: pgmap v13959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:41.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25042.368927 secs 2024-04-27T01:32:41.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25042.368927 secs 2024-04-27T01:32:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25042.368927 secs 2024-04-27T01:32:42.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:41 smithi119 ceph-mon[32752]: pgmap v13960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:42.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25043.183431 secs 2024-04-27T01:32:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:41 smithi167 ceph-mon[32751]: pgmap v13960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25043.183431 secs 2024-04-27T01:32:42.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:41 smithi090 ceph-mon[29180]: pgmap v13960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:42.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25043.183431 secs 2024-04-27T01:32:44.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:43 smithi119 ceph-mon[32752]: pgmap v13961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:44.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:43 smithi167 ceph-mon[32751]: pgmap v13961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:44.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:43 smithi090 ceph-mon[29180]: pgmap v13961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:46.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:45 smithi167 ceph-mon[32751]: pgmap v13962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:46.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25047.368986 secs 2024-04-27T01:32:46.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:45 smithi090 ceph-mon[29180]: pgmap v13962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:46.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25047.368986 secs 2024-04-27T01:32:46.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:45 smithi119 ceph-mon[32752]: pgmap v13962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:46.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25047.368986 secs 2024-04-27T01:32:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25048.183532 secs 2024-04-27T01:32:47.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25048.183532 secs 2024-04-27T01:32:47.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25048.183532 secs 2024-04-27T01:32:48.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:47 smithi167 ceph-mon[32751]: pgmap v13963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:48.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:47 smithi090 ceph-mon[29180]: pgmap v13963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:48.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:47 smithi119 ceph-mon[32752]: pgmap v13963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:50.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:49 smithi167 ceph-mon[32751]: pgmap v13964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:50.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:49 smithi090 ceph-mon[29180]: pgmap v13964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:50.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:49 smithi119 ceph-mon[32752]: pgmap v13964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:32:51.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25052.369072 secs 2024-04-27T01:32:51.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25052.369072 secs 2024-04-27T01:32:51.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25052.369072 secs 2024-04-27T01:32:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:51 smithi090 ceph-mon[29180]: pgmap v13965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:52.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25053.183654 secs 2024-04-27T01:32:52.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:51 smithi119 ceph-mon[32752]: pgmap v13965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:52.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25053.183654 secs 2024-04-27T01:32:52.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:51 smithi167 ceph-mon[32751]: pgmap v13965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:52.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25053.183654 secs 2024-04-27T01:32:53.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:32:53.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:32:53.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:32:54.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:53 smithi090 ceph-mon[29180]: pgmap v13966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:54.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:54.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:54.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:53 smithi119 ceph-mon[32752]: pgmap v13966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:54.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:54.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:54.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:53 smithi167 ceph-mon[32751]: pgmap v13966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:32:54.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:54.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:55.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:32:55.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:32:55.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:55.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:32:55.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:32:55.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:55.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:32:55.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:32:55.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:32:56.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:55 smithi090 ceph-mon[29180]: pgmap v13967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:32:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:55 smithi090 ceph-mon[29180]: pgmap v13968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25057.369133 secs 2024-04-27T01:32:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:55 smithi119 ceph-mon[32752]: pgmap v13967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:32:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:55 smithi119 ceph-mon[32752]: pgmap v13968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25057.369133 secs 2024-04-27T01:32:56.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:55 smithi167 ceph-mon[32751]: pgmap v13967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:32:56.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:55 smithi167 ceph-mon[32751]: pgmap v13968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:56.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25057.369133 secs 2024-04-27T01:32:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25058.183775 secs 2024-04-27T01:32:57.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25058.183775 secs 2024-04-27T01:32:57.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25058.183775 secs 2024-04-27T01:32:58.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:57 smithi090 ceph-mon[29180]: pgmap v13969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:58.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:57 smithi119 ceph-mon[32752]: pgmap v13969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:32:58.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:57 smithi167 ceph-mon[32751]: pgmap v13969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:00.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:32:59 smithi090 ceph-mon[29180]: pgmap v13970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T01:33:00.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:32:59 smithi119 ceph-mon[32752]: pgmap v13970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T01:33:00.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:32:59 smithi167 ceph-mon[32751]: pgmap v13970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T01:33:01.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25062.369217 secs 2024-04-27T01:33:01.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25062.369217 secs 2024-04-27T01:33:01.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25062.369217 secs 2024-04-27T01:33:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:01 smithi090 ceph-mon[29180]: pgmap v13971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25063.183869 secs 2024-04-27T01:33:02.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:01 smithi119 ceph-mon[32752]: pgmap v13971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:02.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25063.183869 secs 2024-04-27T01:33:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:01 smithi167 ceph-mon[32751]: pgmap v13971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:02.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25063.183869 secs 2024-04-27T01:33:03.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:02 smithi119 ceph-mon[32752]: pgmap v13972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:03.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:02 smithi167 ceph-mon[32751]: pgmap v13972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:03.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:02 smithi090 ceph-mon[29180]: pgmap v13972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:05.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:05 smithi119 ceph-mon[32752]: pgmap v13973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:33:05.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:05 smithi167 ceph-mon[32751]: pgmap v13973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:33:05.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:05 smithi090 ceph-mon[29180]: pgmap v13973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:33:06.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25067.369303 secs 2024-04-27T01:33:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25067.369303 secs 2024-04-27T01:33:06.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25067.369303 secs 2024-04-27T01:33:07.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:07 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25068.184014 secs 2024-04-27T01:33:07.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:07 smithi119 ceph-mon[32752]: pgmap v13974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:07.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:07 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25068.184014 secs 2024-04-27T01:33:07.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:07 smithi167 ceph-mon[32751]: pgmap v13974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:07.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:07 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25068.184014 secs 2024-04-27T01:33:07.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:07 smithi090 ceph-mon[29180]: pgmap v13974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:09.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:09 smithi119 ceph-mon[32752]: pgmap v13975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:09.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:09 smithi167 ceph-mon[32751]: pgmap v13975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:09.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:09 smithi090 ceph-mon[29180]: pgmap v13975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:11.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:11 smithi119 ceph-mon[32752]: pgmap v13976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:11.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25072.369362 secs 2024-04-27T01:33:11.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:11 smithi167 ceph-mon[32751]: pgmap v13976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:11.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25072.369362 secs 2024-04-27T01:33:11.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:11 smithi090 ceph-mon[29180]: pgmap v13976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:11.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25072.369362 secs 2024-04-27T01:33:12.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:12 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25073.184107 secs 2024-04-27T01:33:12.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:12 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25073.184107 secs 2024-04-27T01:33:12.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:12 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25073.184107 secs 2024-04-27T01:33:13.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:13 smithi167 ceph-mon[32751]: pgmap v13977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:13.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:13 smithi090 ceph-mon[29180]: pgmap v13977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:13.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:13 smithi119 ceph-mon[32752]: pgmap v13977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:15.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:15 smithi167 ceph-mon[32751]: pgmap v13978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:15.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:15 smithi119 ceph-mon[32752]: pgmap v13978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:15.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:15 smithi090 ceph-mon[29180]: pgmap v13978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:16.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25077.369450 secs 2024-04-27T01:33:16.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25077.369450 secs 2024-04-27T01:33:16.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25077.369450 secs 2024-04-27T01:33:17.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:17 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25078.184216 secs 2024-04-27T01:33:17.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:17 smithi167 ceph-mon[32751]: pgmap v13979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:17 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25078.184216 secs 2024-04-27T01:33:17.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:17 smithi090 ceph-mon[29180]: pgmap v13979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:17.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:17 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25078.184216 secs 2024-04-27T01:33:17.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:17 smithi119 ceph-mon[32752]: pgmap v13979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:19.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:19 smithi090 ceph-mon[29180]: pgmap v13980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:19.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:19 smithi119 ceph-mon[32752]: pgmap v13980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:19.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:19 smithi167 ceph-mon[32751]: pgmap v13980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:21.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:21 smithi090 ceph-mon[29180]: pgmap v13981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:21.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25082.369511 secs 2024-04-27T01:33:21.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:21 smithi119 ceph-mon[32752]: pgmap v13981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:21.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25082.369511 secs 2024-04-27T01:33:21.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:21 smithi167 ceph-mon[32751]: pgmap v13981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:21.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25082.369511 secs 2024-04-27T01:33:22.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25083.184335 secs 2024-04-27T01:33:22.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25083.184335 secs 2024-04-27T01:33:22.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25083.184335 secs 2024-04-27T01:33:23.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:23 smithi090 ceph-mon[29180]: pgmap v13982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:23.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:23 smithi119 ceph-mon[32752]: pgmap v13982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:23 smithi167 ceph-mon[32751]: pgmap v13982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:25.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:25 smithi090 ceph-mon[29180]: pgmap v13983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:25.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:25 smithi119 ceph-mon[32752]: pgmap v13983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:25.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:25 smithi167 ceph-mon[32751]: pgmap v13983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:26.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25087.369598 secs 2024-04-27T01:33:26.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25088.184471 secs 2024-04-27T01:33:26.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25087.369598 secs 2024-04-27T01:33:26.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25088.184471 secs 2024-04-27T01:33:26.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25087.369598 secs 2024-04-27T01:33:26.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25088.184471 secs 2024-04-27T01:33:27.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:27 smithi090 ceph-mon[29180]: pgmap v13984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:27.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:27 smithi119 ceph-mon[32752]: pgmap v13984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:27.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:27 smithi167 ceph-mon[32751]: pgmap v13984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:29.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:29 smithi090 ceph-mon[29180]: pgmap v13985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:29.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:29 smithi119 ceph-mon[32752]: pgmap v13985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:29.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:29 smithi167 ceph-mon[32751]: pgmap v13985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:31.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:31 smithi167 ceph-mon[32751]: pgmap v13986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:31.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25092.369656 secs 2024-04-27T01:33:31.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25093.184589 secs 2024-04-27T01:33:31.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:31 smithi090 ceph-mon[29180]: pgmap v13986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:31.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25092.369656 secs 2024-04-27T01:33:31.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25093.184589 secs 2024-04-27T01:33:31.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:31 smithi119 ceph-mon[32752]: pgmap v13986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:31.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25092.369656 secs 2024-04-27T01:33:31.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25093.184589 secs 2024-04-27T01:33:33.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:33 smithi090 ceph-mon[29180]: pgmap v13987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:33.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:33 smithi119 ceph-mon[32752]: pgmap v13987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:33.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:33 smithi167 ceph-mon[32751]: pgmap v13987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:35.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:35 smithi090 ceph-mon[29180]: pgmap v13988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:35.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:35 smithi119 ceph-mon[32752]: pgmap v13988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:35.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:35 smithi167 ceph-mon[32751]: pgmap v13988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:36.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25097.369742 secs 2024-04-27T01:33:36.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25098.184659 secs 2024-04-27T01:33:36.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25097.369742 secs 2024-04-27T01:33:36.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25098.184659 secs 2024-04-27T01:33:36.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25097.369742 secs 2024-04-27T01:33:36.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25098.184659 secs 2024-04-27T01:33:37.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:37 smithi090 ceph-mon[29180]: pgmap v13989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:37.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:37 smithi119 ceph-mon[32752]: pgmap v13989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:37.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:37 smithi167 ceph-mon[32751]: pgmap v13989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:39.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:39 smithi090 ceph-mon[29180]: pgmap v13990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:39.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:39 smithi119 ceph-mon[32752]: pgmap v13990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:39.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:39 smithi167 ceph-mon[32751]: pgmap v13990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:41.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:41 smithi090 ceph-mon[29180]: pgmap v13991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:41.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:41 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25102.369828 secs 2024-04-27T01:33:41.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:41 smithi090 ceph-mon[29180]: slow request 1442.835860 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:33:41.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25103.184768 secs 2024-04-27T01:33:41.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:41 smithi119 ceph-mon[32752]: pgmap v13991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:41.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:41 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25102.369828 secs 2024-04-27T01:33:41.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:41 smithi119 ceph-mon[32752]: slow request 1442.835860 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:33:41.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25103.184768 secs 2024-04-27T01:33:41.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:41 smithi167 ceph-mon[32751]: pgmap v13991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:41.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:41 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25102.369828 secs 2024-04-27T01:33:41.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:41 smithi167 ceph-mon[32751]: slow request 1442.835860 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:33:41.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25103.184768 secs 2024-04-27T01:33:43.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:43 smithi090 ceph-mon[29180]: pgmap v13992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:43.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:43 smithi119 ceph-mon[32752]: pgmap v13992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:43.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:43 smithi167 ceph-mon[32751]: pgmap v13992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:45.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:45 smithi090 ceph-mon[29180]: pgmap v13993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:45.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:45 smithi119 ceph-mon[32752]: pgmap v13993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:45.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:45 smithi167 ceph-mon[32751]: pgmap v13993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:46.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25107.369890 secs 2024-04-27T01:33:46.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25108.184877 secs 2024-04-27T01:33:46.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25107.369890 secs 2024-04-27T01:33:46.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25108.184877 secs 2024-04-27T01:33:46.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25107.369890 secs 2024-04-27T01:33:46.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25108.184877 secs 2024-04-27T01:33:47.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:47 smithi119 ceph-mon[32752]: pgmap v13994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:47.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:47 smithi167 ceph-mon[32751]: pgmap v13994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:47.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:47 smithi090 ceph-mon[29180]: pgmap v13994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:49.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:49 smithi119 ceph-mon[32752]: pgmap v13995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:49.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:49 smithi167 ceph-mon[32751]: pgmap v13995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:49.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:49 smithi090 ceph-mon[29180]: pgmap v13995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:51.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:51 smithi119 ceph-mon[32752]: pgmap v13996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:51.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:51 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25112.369951 secs 2024-04-27T01:33:51.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:51 smithi119 ceph-mon[32752]: slow request 2881.744039 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:33:51.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25113.184953 secs 2024-04-27T01:33:51.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:51 smithi167 ceph-mon[32751]: pgmap v13996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:51.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:51 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25112.369951 secs 2024-04-27T01:33:51.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:51 smithi167 ceph-mon[32751]: slow request 2881.744039 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:33:51.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25113.184953 secs 2024-04-27T01:33:51.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:51 smithi090 ceph-mon[29180]: pgmap v13996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:51.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:51 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25112.369951 secs 2024-04-27T01:33:51.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:51 smithi090 ceph-mon[29180]: slow request 2881.744039 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:33:51.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25113.184953 secs 2024-04-27T01:33:53.493 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:53 smithi090 ceph-mon[29180]: pgmap v13997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:53.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:53 smithi119 ceph-mon[32752]: pgmap v13997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:53.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:53 smithi167 ceph-mon[32751]: pgmap v13997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:33:54.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:33:54.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:33:54.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:33:55.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:55 smithi119 ceph-mon[32752]: pgmap v13998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:55.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:33:55.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:33:55.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:55 smithi119 ceph-mon[32752]: pgmap v13999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:33:55.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:55 smithi119 ceph-mon[32752]: pgmap v14000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:55.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:33:55.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:55 smithi167 ceph-mon[32751]: pgmap v13998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:55.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:33:55.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:33:55.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:55 smithi167 ceph-mon[32751]: pgmap v13999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:33:55.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:55 smithi167 ceph-mon[32751]: pgmap v14000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:55.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:33:55.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:55 smithi090 ceph-mon[29180]: pgmap v13998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:33:55.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:33:55.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:33:55.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:55 smithi090 ceph-mon[29180]: pgmap v13999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:33:55.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:55 smithi090 ceph-mon[29180]: pgmap v14000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:55.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:33:56.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25117.370013 secs 2024-04-27T01:33:56.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25118.185102 secs 2024-04-27T01:33:56.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25117.370013 secs 2024-04-27T01:33:56.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25118.185102 secs 2024-04-27T01:33:56.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25117.370013 secs 2024-04-27T01:33:56.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25118.185102 secs 2024-04-27T01:33:57.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:57 smithi119 ceph-mon[32752]: pgmap v14001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:57.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:57 smithi167 ceph-mon[32751]: pgmap v14001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:33:57.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:57 smithi090 ceph-mon[29180]: pgmap v14001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:00.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:33:59 smithi167 ceph-mon[32751]: pgmap v14002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:00.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:33:59 smithi090 ceph-mon[29180]: pgmap v14002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:00.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:33:59 smithi119 ceph-mon[32752]: pgmap v14002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:01.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25122.370115 secs 2024-04-27T01:34:01.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25122.370115 secs 2024-04-27T01:34:01.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25122.370115 secs 2024-04-27T01:34:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:01 smithi167 ceph-mon[32751]: pgmap v14003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:02.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25123.185222 secs 2024-04-27T01:34:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:01 smithi090 ceph-mon[29180]: pgmap v14003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25123.185222 secs 2024-04-27T01:34:02.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:01 smithi119 ceph-mon[32752]: pgmap v14003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:02.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25123.185222 secs 2024-04-27T01:34:04.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:03 smithi090 ceph-mon[29180]: pgmap v14004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:34:04.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:03 smithi119 ceph-mon[32752]: pgmap v14004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:34:04.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:03 smithi167 ceph-mon[32751]: pgmap v14004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:34:06.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:05 smithi090 ceph-mon[29180]: pgmap v14005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:34:06.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25127.370175 secs 2024-04-27T01:34:06.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:05 smithi119 ceph-mon[32752]: pgmap v14005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:34:06.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25127.370175 secs 2024-04-27T01:34:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:05 smithi167 ceph-mon[32751]: pgmap v14005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:34:06.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25127.370175 secs 2024-04-27T01:34:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25128.185360 secs 2024-04-27T01:34:07.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25128.185360 secs 2024-04-27T01:34:07.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25128.185360 secs 2024-04-27T01:34:08.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:07 smithi090 ceph-mon[29180]: pgmap v14006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:08.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:07 smithi119 ceph-mon[32752]: pgmap v14006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:08.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:07 smithi167 ceph-mon[32751]: pgmap v14006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:10.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:09 smithi090 ceph-mon[29180]: pgmap v14007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:10.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:09 smithi119 ceph-mon[32752]: pgmap v14007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:10.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:09 smithi167 ceph-mon[32751]: pgmap v14007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:11.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25132.370260 secs 2024-04-27T01:34:11.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25132.370260 secs 2024-04-27T01:34:11.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25132.370260 secs 2024-04-27T01:34:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:11 smithi090 ceph-mon[29180]: pgmap v14008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25133.185486 secs 2024-04-27T01:34:12.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:11 smithi119 ceph-mon[32752]: pgmap v14008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:12.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25133.185486 secs 2024-04-27T01:34:12.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:11 smithi167 ceph-mon[32751]: pgmap v14008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:12.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25133.185486 secs 2024-04-27T01:34:14.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:13 smithi090 ceph-mon[29180]: pgmap v14009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:14.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:13 smithi119 ceph-mon[32752]: pgmap v14009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:14.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:13 smithi167 ceph-mon[32751]: pgmap v14009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:16.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:15 smithi119 ceph-mon[32752]: pgmap v14010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:16.090 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25137.370344 secs 2024-04-27T01:34:16.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:15 smithi090 ceph-mon[29180]: pgmap v14010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:16.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25137.370344 secs 2024-04-27T01:34:16.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:15 smithi167 ceph-mon[32751]: pgmap v14010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:16.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25137.370344 secs 2024-04-27T01:34:17.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25138.185554 secs 2024-04-27T01:34:17.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25138.185554 secs 2024-04-27T01:34:17.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25138.185554 secs 2024-04-27T01:34:18.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:17 smithi090 ceph-mon[29180]: pgmap v14011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:18.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:17 smithi119 ceph-mon[32752]: pgmap v14011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:18.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:17 smithi167 ceph-mon[32751]: pgmap v14011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:20.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:19 smithi090 ceph-mon[29180]: pgmap v14012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:20.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:19 smithi119 ceph-mon[32752]: pgmap v14012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:20.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:19 smithi167 ceph-mon[32751]: pgmap v14012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:21.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25142.370428 secs 2024-04-27T01:34:21.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25142.370428 secs 2024-04-27T01:34:21.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25142.370428 secs 2024-04-27T01:34:22.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:21 smithi090 ceph-mon[29180]: pgmap v14013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:22.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25143.185637 secs 2024-04-27T01:34:22.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:21 smithi119 ceph-mon[32752]: pgmap v14013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:22.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25143.185637 secs 2024-04-27T01:34:22.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:21 smithi167 ceph-mon[32751]: pgmap v14013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:22.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25143.185637 secs 2024-04-27T01:34:24.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:23 smithi090 ceph-mon[29180]: pgmap v14014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:24.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:23 smithi119 ceph-mon[32752]: pgmap v14014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:24.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:23 smithi167 ceph-mon[32751]: pgmap v14014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:25 smithi090 ceph-mon[29180]: pgmap v14015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25147.370515 secs 2024-04-27T01:34:26.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:25 smithi119 ceph-mon[32752]: pgmap v14015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:26.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25147.370515 secs 2024-04-27T01:34:26.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:25 smithi167 ceph-mon[32751]: pgmap v14015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:26.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25147.370515 secs 2024-04-27T01:34:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25148.185704 secs 2024-04-27T01:34:27.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25148.185704 secs 2024-04-27T01:34:27.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25148.185704 secs 2024-04-27T01:34:28.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:27 smithi090 ceph-mon[29180]: pgmap v14016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:28.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:27 smithi119 ceph-mon[32752]: pgmap v14016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:28.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:27 smithi167 ceph-mon[32751]: pgmap v14016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:29.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:28 smithi119 ceph-mon[32752]: pgmap v14017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:29.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:28 smithi167 ceph-mon[32751]: pgmap v14017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:29.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:28 smithi090 ceph-mon[29180]: pgmap v14017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:31.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25152.370600 secs 2024-04-27T01:34:31.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25152.370600 secs 2024-04-27T01:34:31.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25152.370600 secs 2024-04-27T01:34:32.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:31 smithi119 ceph-mon[32752]: pgmap v14018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:32.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25153.185825 secs 2024-04-27T01:34:32.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:31 smithi167 ceph-mon[32751]: pgmap v14018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:32.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25153.185825 secs 2024-04-27T01:34:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:31 smithi090 ceph-mon[29180]: pgmap v14018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25153.185825 secs 2024-04-27T01:34:34.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:33 smithi090 ceph-mon[29180]: pgmap v14019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:34.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:33 smithi119 ceph-mon[32752]: pgmap v14019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:33 smithi167 ceph-mon[32751]: pgmap v14019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:35 smithi090 ceph-mon[29180]: pgmap v14020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25157.370688 secs 2024-04-27T01:34:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:35 smithi119 ceph-mon[32752]: pgmap v14020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:36.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25157.370688 secs 2024-04-27T01:34:36.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:35 smithi167 ceph-mon[32751]: pgmap v14020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:36.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25157.370688 secs 2024-04-27T01:34:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25158.185946 secs 2024-04-27T01:34:37.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25158.185946 secs 2024-04-27T01:34:37.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25158.185946 secs 2024-04-27T01:34:38.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:37 smithi090 ceph-mon[29180]: pgmap v14021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:38.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:37 smithi119 ceph-mon[32752]: pgmap v14021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:38.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:37 smithi167 ceph-mon[32751]: pgmap v14021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:40.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:39 smithi090 ceph-mon[29180]: pgmap v14022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:40.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:39 smithi119 ceph-mon[32752]: pgmap v14022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:40.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:39 smithi167 ceph-mon[32751]: pgmap v14022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:40.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25162.370770 secs 2024-04-27T01:34:41.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25162.370770 secs 2024-04-27T01:34:41.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25162.370770 secs 2024-04-27T01:34:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:41 smithi090 ceph-mon[29180]: pgmap v14023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:42.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25163.186065 secs 2024-04-27T01:34:42.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:41 smithi119 ceph-mon[32752]: pgmap v14023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:42.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25163.186065 secs 2024-04-27T01:34:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:41 smithi167 ceph-mon[32751]: pgmap v14023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25163.186065 secs 2024-04-27T01:34:44.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:43 smithi090 ceph-mon[29180]: pgmap v14024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:44.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:43 smithi119 ceph-mon[32752]: pgmap v14024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:44.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:43 smithi167 ceph-mon[32751]: pgmap v14024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:46.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:45 smithi090 ceph-mon[29180]: pgmap v14025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:46.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25167.370845 secs 2024-04-27T01:34:46.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:45 smithi119 ceph-mon[32752]: pgmap v14025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:46.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25167.370845 secs 2024-04-27T01:34:46.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:45 smithi167 ceph-mon[32751]: pgmap v14025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:46.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25167.370845 secs 2024-04-27T01:34:47.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25168.186212 secs 2024-04-27T01:34:47.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25168.186212 secs 2024-04-27T01:34:47.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25168.186212 secs 2024-04-27T01:34:48.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:47 smithi090 ceph-mon[29180]: pgmap v14026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:48.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:47 smithi119 ceph-mon[32752]: pgmap v14026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:48.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:47 smithi167 ceph-mon[32751]: pgmap v14026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:50.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:49 smithi090 ceph-mon[29180]: pgmap v14027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:50.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:49 smithi119 ceph-mon[32752]: pgmap v14027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:50.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:49 smithi167 ceph-mon[32751]: pgmap v14027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:51.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25172.370936 secs 2024-04-27T01:34:51.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25172.370936 secs 2024-04-27T01:34:51.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25172.370936 secs 2024-04-27T01:34:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:51 smithi090 ceph-mon[29180]: pgmap v14028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25173.186332 secs 2024-04-27T01:34:52.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:51 smithi119 ceph-mon[32752]: pgmap v14028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:52.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25173.186332 secs 2024-04-27T01:34:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:51 smithi167 ceph-mon[32751]: pgmap v14028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25173.186332 secs 2024-04-27T01:34:53.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:52 smithi119 ceph-mon[32752]: pgmap v14029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:53.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:52 smithi167 ceph-mon[32751]: pgmap v14029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:53.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:52 smithi090 ceph-mon[29180]: pgmap v14029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:34:56.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:55 smithi090 ceph-mon[29180]: pgmap v14030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:34:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25177.371003 secs 2024-04-27T01:34:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:34:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:34:56.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:34:56.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:55 smithi119 ceph-mon[32752]: pgmap v14030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:34:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25177.371003 secs 2024-04-27T01:34:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:34:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:34:56.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:34:56.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:55 smithi167 ceph-mon[32751]: pgmap v14030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:34:56.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:34:56.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25177.371003 secs 2024-04-27T01:34:56.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:34:56.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:34:56.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:34:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:56 smithi090 ceph-mon[29180]: pgmap v14031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:34:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:56 smithi090 ceph-mon[29180]: pgmap v14032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25178.186477 secs 2024-04-27T01:34:57.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:56 smithi119 ceph-mon[32752]: pgmap v14031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:34:57.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:56 smithi119 ceph-mon[32752]: pgmap v14032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:57.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25178.186477 secs 2024-04-27T01:34:57.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:56 smithi167 ceph-mon[32751]: pgmap v14031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:34:57.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:56 smithi167 ceph-mon[32751]: pgmap v14032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:34:57.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25178.186477 secs 2024-04-27T01:34:59.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:34:58 smithi090 ceph-mon[29180]: pgmap v14033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:34:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:34:58 smithi119 ceph-mon[32752]: pgmap v14033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:34:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:34:58 smithi167 ceph-mon[32751]: pgmap v14033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:35:01.103 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:00 smithi167 ceph-mon[32751]: pgmap v14034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:01.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25182.371111 secs 2024-04-27T01:35:01.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:00 smithi090 ceph-mon[29180]: pgmap v14034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:01.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25182.371111 secs 2024-04-27T01:35:01.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:00 smithi119 ceph-mon[32752]: pgmap v14034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:01.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25182.371111 secs 2024-04-27T01:35:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25183.186579 secs 2024-04-27T01:35:02.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25183.186579 secs 2024-04-27T01:35:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25183.186579 secs 2024-04-27T01:35:03.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:02 smithi090 ceph-mon[29180]: pgmap v14035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:03.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:02 smithi119 ceph-mon[32752]: pgmap v14035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:03.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:02 smithi167 ceph-mon[32751]: pgmap v14035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:05.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:04 smithi090 ceph-mon[29180]: pgmap v14036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:05.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:04 smithi119 ceph-mon[32752]: pgmap v14036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:05.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:04 smithi167 ceph-mon[32751]: pgmap v14036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:06.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25187.371198 secs 2024-04-27T01:35:06.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25187.371198 secs 2024-04-27T01:35:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25187.371198 secs 2024-04-27T01:35:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:06 smithi090 ceph-mon[29180]: pgmap v14037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:35:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25188.186725 secs 2024-04-27T01:35:07.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:06 smithi119 ceph-mon[32752]: pgmap v14037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:35:07.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25188.186725 secs 2024-04-27T01:35:07.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:06 smithi167 ceph-mon[32751]: pgmap v14037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:35:07.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25188.186725 secs 2024-04-27T01:35:09.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:08 smithi090 ceph-mon[29180]: pgmap v14038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:09.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:08 smithi119 ceph-mon[32752]: pgmap v14038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:09.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:08 smithi167 ceph-mon[32751]: pgmap v14038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:11.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:10 smithi090 ceph-mon[29180]: pgmap v14039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:11.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25192.371279 secs 2024-04-27T01:35:11.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:10 smithi119 ceph-mon[32752]: pgmap v14039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:11.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25192.371279 secs 2024-04-27T01:35:11.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:10 smithi167 ceph-mon[32751]: pgmap v14039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:11.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25192.371279 secs 2024-04-27T01:35:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25193.186802 secs 2024-04-27T01:35:12.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25193.186802 secs 2024-04-27T01:35:12.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25193.186802 secs 2024-04-27T01:35:13.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:12 smithi090 ceph-mon[29180]: pgmap v14040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:13.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:12 smithi119 ceph-mon[32752]: pgmap v14040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:13.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:12 smithi167 ceph-mon[32751]: pgmap v14040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:15.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:14 smithi119 ceph-mon[32752]: pgmap v14041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:15.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:14 smithi167 ceph-mon[32751]: pgmap v14041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:15.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:14 smithi090 ceph-mon[29180]: pgmap v14041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:16.189 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25197.371365 secs 2024-04-27T01:35:16.189 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:15 smithi119 ceph-mon[32752]: pgmap v14042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:16.260 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25197.371365 secs 2024-04-27T01:35:16.260 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:15 smithi090 ceph-mon[29180]: pgmap v14042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:16.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25197.371365 secs 2024-04-27T01:35:16.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:15 smithi167 ceph-mon[32751]: pgmap v14042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:17.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25198.186924 secs 2024-04-27T01:35:17.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25198.186924 secs 2024-04-27T01:35:17.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25198.186924 secs 2024-04-27T01:35:18.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:17 smithi119 ceph-mon[32752]: pgmap v14043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:18.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:17 smithi167 ceph-mon[32751]: pgmap v14043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:18.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:17 smithi090 ceph-mon[29180]: pgmap v14043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:20.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:20 smithi090 ceph-mon[29180]: pgmap v14044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:21.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:20 smithi119 ceph-mon[32752]: pgmap v14044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:21.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:20 smithi167 ceph-mon[32751]: pgmap v14044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25202.371452 secs 2024-04-27T01:35:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25203.187036 secs 2024-04-27T01:35:22.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25202.371452 secs 2024-04-27T01:35:22.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25203.187036 secs 2024-04-27T01:35:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25202.371452 secs 2024-04-27T01:35:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25203.187036 secs 2024-04-27T01:35:22.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:22 smithi090 ceph-mon[29180]: pgmap v14045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:23.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:22 smithi119 ceph-mon[32752]: pgmap v14045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:23.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:22 smithi167 ceph-mon[32751]: pgmap v14045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:24.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:24 smithi090 ceph-mon[29180]: pgmap v14046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:25.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:24 smithi119 ceph-mon[32752]: pgmap v14046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:25.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:24 smithi167 ceph-mon[32751]: pgmap v14046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:26.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25207.371509 secs 2024-04-27T01:35:26.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:26 smithi090 ceph-mon[29180]: pgmap v14047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:26.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25208.187168 secs 2024-04-27T01:35:27.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25207.371509 secs 2024-04-27T01:35:27.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:26 smithi119 ceph-mon[32752]: pgmap v14047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:27.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25208.187168 secs 2024-04-27T01:35:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25207.371509 secs 2024-04-27T01:35:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:26 smithi167 ceph-mon[32751]: pgmap v14047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25208.187168 secs 2024-04-27T01:35:28.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:28 smithi090 ceph-mon[29180]: pgmap v14048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:29.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:28 smithi119 ceph-mon[32752]: pgmap v14048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:29.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:28 smithi167 ceph-mon[32751]: pgmap v14048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:30.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:30 smithi090 ceph-mon[29180]: pgmap v14049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:31.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:30 smithi119 ceph-mon[32752]: pgmap v14049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:31.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:30 smithi167 ceph-mon[32751]: pgmap v14049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:31.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25212.371593 secs 2024-04-27T01:35:31.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25213.187278 secs 2024-04-27T01:35:31.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25212.371593 secs 2024-04-27T01:35:31.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25213.187278 secs 2024-04-27T01:35:32.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25212.371593 secs 2024-04-27T01:35:32.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25213.187278 secs 2024-04-27T01:35:32.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:32 smithi090 ceph-mon[29180]: pgmap v14050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:33.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:32 smithi119 ceph-mon[32752]: pgmap v14050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:33.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:32 smithi167 ceph-mon[32751]: pgmap v14050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:35.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:34 smithi119 ceph-mon[32752]: pgmap v14051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:35.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:34 smithi167 ceph-mon[32751]: pgmap v14051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:35.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:34 smithi090 ceph-mon[29180]: pgmap v14051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:35:37.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25217.371678 secs 2024-04-27T01:35:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:36 smithi119 ceph-mon[32752]: pgmap v14052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25218.187414 secs 2024-04-27T01:35:37.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25217.371678 secs 2024-04-27T01:35:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:36 smithi167 ceph-mon[32751]: pgmap v14052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25218.187414 secs 2024-04-27T01:35:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25217.371678 secs 2024-04-27T01:35:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:36 smithi090 ceph-mon[29180]: pgmap v14052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25218.187414 secs 2024-04-27T01:35:39.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:38 smithi119 ceph-mon[32752]: pgmap v14053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:39.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:38 smithi167 ceph-mon[32751]: pgmap v14053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:39.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:38 smithi090 ceph-mon[29180]: pgmap v14053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:41.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:40 smithi119 ceph-mon[32752]: pgmap v14054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:41.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:40 smithi167 ceph-mon[32751]: pgmap v14054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:41.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:40 smithi090 ceph-mon[29180]: pgmap v14054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:42.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25222.371764 secs 2024-04-27T01:35:42.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25223.187529 secs 2024-04-27T01:35:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25222.371764 secs 2024-04-27T01:35:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25223.187529 secs 2024-04-27T01:35:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25222.371764 secs 2024-04-27T01:35:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25223.187529 secs 2024-04-27T01:35:42.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:42 smithi090 ceph-mon[29180]: pgmap v14055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:43.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:42 smithi119 ceph-mon[32752]: pgmap v14055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:43.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:42 smithi167 ceph-mon[32751]: pgmap v14055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:44.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:44 smithi119 ceph-mon[32752]: pgmap v14056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:44.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:44 smithi167 ceph-mon[32751]: pgmap v14056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:44.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:44 smithi090 ceph-mon[29180]: pgmap v14056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:46.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25227.371821 secs 2024-04-27T01:35:46.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:46 smithi090 ceph-mon[29180]: pgmap v14057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:46.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25228.187595 secs 2024-04-27T01:35:47.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25227.371821 secs 2024-04-27T01:35:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:46 smithi119 ceph-mon[32752]: pgmap v14057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25228.187595 secs 2024-04-27T01:35:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25227.371821 secs 2024-04-27T01:35:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:46 smithi167 ceph-mon[32751]: pgmap v14057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:47.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25228.187595 secs 2024-04-27T01:35:48.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:48 smithi090 ceph-mon[29180]: pgmap v14058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:49.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:48 smithi119 ceph-mon[32752]: pgmap v14058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:49.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:48 smithi167 ceph-mon[32751]: pgmap v14058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:35:50.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:50 smithi090 ceph-mon[29180]: pgmap v14059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:51.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:50 smithi119 ceph-mon[32752]: pgmap v14059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:51.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:50 smithi167 ceph-mon[32751]: pgmap v14059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:35:51.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25232.371881 secs 2024-04-27T01:35:51.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25233.187654 secs 2024-04-27T01:35:52.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25232.371881 secs 2024-04-27T01:35:52.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25233.187654 secs 2024-04-27T01:35:52.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25232.371881 secs 2024-04-27T01:35:52.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25233.187654 secs 2024-04-27T01:35:52.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:52 smithi090 ceph-mon[29180]: pgmap v14060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:35:53.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:52 smithi119 ceph-mon[32752]: pgmap v14060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:35:53.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:52 smithi167 ceph-mon[32751]: pgmap v14060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:35:54.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:54 smithi090 ceph-mon[29180]: pgmap v14061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:35:55.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:54 smithi119 ceph-mon[32752]: pgmap v14061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:35:55.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:54 smithi167 ceph-mon[32751]: pgmap v14061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:35:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25237.371942 secs 2024-04-27T01:35:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:56 smithi119 ceph-mon[32752]: pgmap v14062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:35:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25238.187727 secs 2024-04-27T01:35:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:35:57.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:35:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:35:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25237.371942 secs 2024-04-27T01:35:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:56 smithi167 ceph-mon[32751]: pgmap v14062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:35:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25238.187727 secs 2024-04-27T01:35:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:35:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:35:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:35:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25237.371942 secs 2024-04-27T01:35:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:56 smithi090 ceph-mon[29180]: pgmap v14062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:35:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:35:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25238.187727 secs 2024-04-27T01:35:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:35:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:35:57.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:35:57.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:57 smithi090 ceph-mon[29180]: pgmap v14063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:35:57.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:57 smithi090 ceph-mon[29180]: pgmap v14064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:58.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:57 smithi119 ceph-mon[32752]: pgmap v14063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:35:58.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:57 smithi119 ceph-mon[32752]: pgmap v14064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:35:58.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:57 smithi167 ceph-mon[32751]: pgmap v14063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:35:58.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:57 smithi167 ceph-mon[32751]: pgmap v14064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:00.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:35:59 smithi119 ceph-mon[32752]: pgmap v14065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:00.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:35:59 smithi167 ceph-mon[32751]: pgmap v14065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:00.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:35:59 smithi090 ceph-mon[29180]: pgmap v14065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25242.372035 secs 2024-04-27T01:36:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:01 smithi119 ceph-mon[32752]: pgmap v14066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25243.187838 secs 2024-04-27T01:36:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25242.372035 secs 2024-04-27T01:36:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:01 smithi167 ceph-mon[32751]: pgmap v14066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25243.187838 secs 2024-04-27T01:36:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25242.372035 secs 2024-04-27T01:36:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:01 smithi090 ceph-mon[29180]: pgmap v14066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25243.187838 secs 2024-04-27T01:36:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:03 smithi119 ceph-mon[32752]: pgmap v14067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:36:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:03 smithi167 ceph-mon[32751]: pgmap v14067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:36:04.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:03 smithi090 ceph-mon[29180]: pgmap v14067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:36:06.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:05 smithi119 ceph-mon[32752]: pgmap v14068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:06.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:05 smithi167 ceph-mon[32751]: pgmap v14068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:06.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:05 smithi090 ceph-mon[29180]: pgmap v14068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:36:07.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25247.372117 secs 2024-04-27T01:36:07.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25248.187934 secs 2024-04-27T01:36:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25247.372117 secs 2024-04-27T01:36:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25248.187934 secs 2024-04-27T01:36:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25247.372117 secs 2024-04-27T01:36:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25248.187934 secs 2024-04-27T01:36:08.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:07 smithi119 ceph-mon[32752]: pgmap v14069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:36:08.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:07 smithi167 ceph-mon[32751]: pgmap v14069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:36:08.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:07 smithi090 ceph-mon[29180]: pgmap v14069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:36:08.109 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:36:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:36:07.774+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:36:08.110 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:36:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:36:07.774+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:36:08.110 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:36:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:36:07.774+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:36:08.110 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:36:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:36:07.774+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:36:08.110 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:36:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:36:07.774+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:36:10.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:09 smithi119 ceph-mon[32752]: pgmap v14070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T01:36:10.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:09 smithi167 ceph-mon[32751]: pgmap v14070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T01:36:10.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:09 smithi090 ceph-mon[29180]: pgmap v14070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T01:36:12.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:11 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25252.372179 secs 2024-04-27T01:36:12.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:11 smithi119 ceph-mon[32752]: slow request 5761.006377 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:36:12.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:11 smithi119 ceph-mon[32752]: pgmap v14071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:12.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25253.188042 secs 2024-04-27T01:36:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:11 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25252.372179 secs 2024-04-27T01:36:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:11 smithi167 ceph-mon[32751]: slow request 5761.006377 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:36:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:11 smithi167 ceph-mon[32751]: pgmap v14071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25253.188042 secs 2024-04-27T01:36:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:11 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25252.372179 secs 2024-04-27T01:36:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:11 smithi090 ceph-mon[29180]: slow request 5761.006377 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:36:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:11 smithi090 ceph-mon[29180]: pgmap v14071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25253.188042 secs 2024-04-27T01:36:14.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:13 smithi119 ceph-mon[32752]: pgmap v14072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:14.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:13 smithi167 ceph-mon[32751]: pgmap v14072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:14.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:13 smithi090 ceph-mon[29180]: pgmap v14072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:16.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:15 smithi119 ceph-mon[32752]: pgmap v14073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:36:16.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:15 smithi167 ceph-mon[32751]: pgmap v14073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:36:16.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:15 smithi090 ceph-mon[29180]: pgmap v14073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:36:16.939 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25257.372238 secs 2024-04-27T01:36:16.939 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25258.188183 secs 2024-04-27T01:36:17.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25257.372238 secs 2024-04-27T01:36:17.011 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25258.188183 secs 2024-04-27T01:36:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25257.372238 secs 2024-04-27T01:36:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25258.188183 secs 2024-04-27T01:36:18.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:17 smithi119 ceph-mon[32752]: pgmap v14074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:18.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:17 smithi167 ceph-mon[32751]: pgmap v14074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:18.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:17 smithi090 ceph-mon[29180]: pgmap v14074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T01:36:20.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:19 smithi119 ceph-mon[32752]: pgmap v14075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:36:20.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:19 smithi167 ceph-mon[32751]: pgmap v14075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:36:20.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:19 smithi090 ceph-mon[29180]: pgmap v14075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T01:36:21.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25262.372322 secs 2024-04-27T01:36:21.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25262.372322 secs 2024-04-27T01:36:21.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25262.372322 secs 2024-04-27T01:36:22.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:21 smithi119 ceph-mon[32752]: pgmap v14076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T01:36:22.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25263.188318 secs 2024-04-27T01:36:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:21 smithi167 ceph-mon[32751]: pgmap v14076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T01:36:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25263.188318 secs 2024-04-27T01:36:22.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:21 smithi090 ceph-mon[29180]: pgmap v14076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T01:36:22.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25263.188318 secs 2024-04-27T01:36:24.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:23 smithi119 ceph-mon[32752]: pgmap v14077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:36:24.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:23 smithi167 ceph-mon[32751]: pgmap v14077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:36:24.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:23 smithi090 ceph-mon[29180]: pgmap v14077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:36:26.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:25 smithi119 ceph-mon[32752]: pgmap v14078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:36:26.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25267.372419 secs 2024-04-27T01:36:26.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:25 smithi167 ceph-mon[32751]: pgmap v14078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:36:26.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25267.372419 secs 2024-04-27T01:36:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:25 smithi090 ceph-mon[29180]: pgmap v14078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:36:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25267.372419 secs 2024-04-27T01:36:27.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25268.188444 secs 2024-04-27T01:36:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25268.188444 secs 2024-04-27T01:36:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25268.188444 secs 2024-04-27T01:36:28.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:27 smithi119 ceph-mon[32752]: pgmap v14079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:28.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:27 smithi167 ceph-mon[32751]: pgmap v14079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:28.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:27 smithi090 ceph-mon[29180]: pgmap v14079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:30.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:29 smithi119 ceph-mon[32752]: pgmap v14080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:30.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:29 smithi167 ceph-mon[32751]: pgmap v14080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:30.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:29 smithi090 ceph-mon[29180]: pgmap v14080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:31.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25272.372519 secs 2024-04-27T01:36:31.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25272.372519 secs 2024-04-27T01:36:31.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25272.372519 secs 2024-04-27T01:36:31.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:31 smithi167 ceph-mon[32751]: pgmap v14081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:31.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25273.188568 secs 2024-04-27T01:36:32.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:31 smithi119 ceph-mon[32752]: pgmap v14081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:32.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25273.188568 secs 2024-04-27T01:36:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:31 smithi090 ceph-mon[29180]: pgmap v14081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25273.188568 secs 2024-04-27T01:36:34.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:33 smithi119 ceph-mon[32752]: pgmap v14082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:34.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:33 smithi167 ceph-mon[32751]: pgmap v14082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:34.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:33 smithi090 ceph-mon[29180]: pgmap v14082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:36.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:35 smithi119 ceph-mon[32752]: pgmap v14083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:36.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25277.372621 secs 2024-04-27T01:36:36.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:35 smithi167 ceph-mon[32751]: pgmap v14083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:36.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25277.372621 secs 2024-04-27T01:36:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:35 smithi090 ceph-mon[29180]: pgmap v14083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25277.372621 secs 2024-04-27T01:36:37.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25278.188670 secs 2024-04-27T01:36:37.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25278.188670 secs 2024-04-27T01:36:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25278.188670 secs 2024-04-27T01:36:38.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:37 smithi119 ceph-mon[32752]: pgmap v14084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:38.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:37 smithi167 ceph-mon[32751]: pgmap v14084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:38.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:37 smithi090 ceph-mon[29180]: pgmap v14084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:40.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:39 smithi119 ceph-mon[32752]: pgmap v14085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:40.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:39 smithi167 ceph-mon[32751]: pgmap v14085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:40.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:39 smithi090 ceph-mon[29180]: pgmap v14085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:41.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25282.372724 secs 2024-04-27T01:36:41.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25282.372724 secs 2024-04-27T01:36:41.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25282.372724 secs 2024-04-27T01:36:42.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:41 smithi119 ceph-mon[32752]: pgmap v14086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:42.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:41 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 25283.188778 secs 2024-04-27T01:36:42.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:41 smithi119 ceph-mon[32752]: slow request 5764.761188 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:36:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:41 smithi167 ceph-mon[32751]: pgmap v14086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:41 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 25283.188778 secs 2024-04-27T01:36:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:41 smithi167 ceph-mon[32751]: slow request 5764.761188 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:36:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:41 smithi090 ceph-mon[29180]: pgmap v14086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:41 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 25283.188778 secs 2024-04-27T01:36:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:41 smithi090 ceph-mon[29180]: slow request 5764.761188 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:36:44.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:43 smithi119 ceph-mon[32752]: pgmap v14087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:44.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:43 smithi167 ceph-mon[32751]: pgmap v14087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:44.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:43 smithi090 ceph-mon[29180]: pgmap v14087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:46.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:45 smithi119 ceph-mon[32752]: pgmap v14088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:46.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25287.372825 secs 2024-04-27T01:36:46.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:45 smithi167 ceph-mon[32751]: pgmap v14088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:46.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25287.372825 secs 2024-04-27T01:36:46.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:45 smithi090 ceph-mon[29180]: pgmap v14088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:46.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25287.372825 secs 2024-04-27T01:36:47.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25288.188920 secs 2024-04-27T01:36:47.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25288.188920 secs 2024-04-27T01:36:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25288.188920 secs 2024-04-27T01:36:48.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:47 smithi119 ceph-mon[32752]: pgmap v14089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:48.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:47 smithi167 ceph-mon[32751]: pgmap v14089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:48.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:47 smithi090 ceph-mon[29180]: pgmap v14089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:50.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:49 smithi119 ceph-mon[32752]: pgmap v14090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:50.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:49 smithi167 ceph-mon[32751]: pgmap v14090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:50.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:49 smithi090 ceph-mon[29180]: pgmap v14090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:51.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25292.372909 secs 2024-04-27T01:36:51.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25292.372909 secs 2024-04-27T01:36:51.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25292.372909 secs 2024-04-27T01:36:52.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:51 smithi119 ceph-mon[32752]: pgmap v14091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:52.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25293.189036 secs 2024-04-27T01:36:52.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:51 smithi167 ceph-mon[32751]: pgmap v14091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:52.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25293.189036 secs 2024-04-27T01:36:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:51 smithi090 ceph-mon[29180]: pgmap v14091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25293.189036 secs 2024-04-27T01:36:54.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:53 smithi119 ceph-mon[32752]: pgmap v14092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:54.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:53 smithi167 ceph-mon[32751]: pgmap v14092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:54.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:53 smithi090 ceph-mon[29180]: pgmap v14092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:56.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:55 smithi119 ceph-mon[32752]: pgmap v14093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:56.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25297.372997 secs 2024-04-27T01:36:56.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:55 smithi167 ceph-mon[32751]: pgmap v14093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:56.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25297.372997 secs 2024-04-27T01:36:56.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:55 smithi090 ceph-mon[29180]: pgmap v14093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:36:56.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25297.372997 secs 2024-04-27T01:36:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25298.189178 secs 2024-04-27T01:36:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:36:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25298.189178 secs 2024-04-27T01:36:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:36:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25298.189178 secs 2024-04-27T01:36:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:36:58.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:57 smithi119 ceph-mon[32752]: pgmap v14094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:58.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:36:58.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:36:58.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:36:58.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:57 smithi167 ceph-mon[32751]: pgmap v14094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:58.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:36:58.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:36:58.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:36:58.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:57 smithi090 ceph-mon[29180]: pgmap v14094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:36:58.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:36:58.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:36:58.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:36:59.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:58 smithi119 ceph-mon[32752]: pgmap v14095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:36:59.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:36:58 smithi119 ceph-mon[32752]: pgmap v14096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:36:59.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:58 smithi167 ceph-mon[32751]: pgmap v14095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:36:59.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:36:58 smithi167 ceph-mon[32751]: pgmap v14096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:36:59.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:58 smithi090 ceph-mon[29180]: pgmap v14095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:36:59.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:36:58 smithi090 ceph-mon[29180]: pgmap v14096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:37:01.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:00 smithi119 ceph-mon[32752]: pgmap v14097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:37:01.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25302.373060 secs 2024-04-27T01:37:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:00 smithi167 ceph-mon[32751]: pgmap v14097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:37:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25302.373060 secs 2024-04-27T01:37:01.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:00 smithi090 ceph-mon[29180]: pgmap v14097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:37:01.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25302.373060 secs 2024-04-27T01:37:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25303.189317 secs 2024-04-27T01:37:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25303.189317 secs 2024-04-27T01:37:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25303.189317 secs 2024-04-27T01:37:03.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:02 smithi119 ceph-mon[32752]: pgmap v14098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:37:03.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:02 smithi167 ceph-mon[32751]: pgmap v14098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:37:03.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:02 smithi090 ceph-mon[29180]: pgmap v14098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:37:05.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:04 smithi119 ceph-mon[32752]: pgmap v14099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:37:05.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:04 smithi167 ceph-mon[32751]: pgmap v14099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:37:05.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:04 smithi090 ceph-mon[29180]: pgmap v14099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:37:06.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25307.373138 secs 2024-04-27T01:37:06.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25307.373138 secs 2024-04-27T01:37:06.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25307.373138 secs 2024-04-27T01:37:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:06 smithi167 ceph-mon[32751]: pgmap v14100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25308.189450 secs 2024-04-27T01:37:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:06 smithi090 ceph-mon[29180]: pgmap v14100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:07.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25308.189450 secs 2024-04-27T01:37:07.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:06 smithi119 ceph-mon[32752]: pgmap v14100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:07.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25308.189450 secs 2024-04-27T01:37:09.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:08 smithi167 ceph-mon[32751]: pgmap v14101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:09.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:08 smithi090 ceph-mon[29180]: pgmap v14101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:09.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:08 smithi119 ceph-mon[32752]: pgmap v14101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:11.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:10 smithi167 ceph-mon[32751]: pgmap v14102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:11.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25312.373243 secs 2024-04-27T01:37:11.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:10 smithi090 ceph-mon[29180]: pgmap v14102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:11.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25312.373243 secs 2024-04-27T01:37:11.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:10 smithi119 ceph-mon[32752]: pgmap v14102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:11.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25312.373243 secs 2024-04-27T01:37:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25313.189556 secs 2024-04-27T01:37:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25313.189556 secs 2024-04-27T01:37:12.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25313.189556 secs 2024-04-27T01:37:13.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:12 smithi090 ceph-mon[29180]: pgmap v14103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:37:13.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:12 smithi119 ceph-mon[32752]: pgmap v14103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:37:13.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:12 smithi167 ceph-mon[32751]: pgmap v14103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:37:15.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:14 smithi090 ceph-mon[29180]: pgmap v14104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:37:15.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:14 smithi119 ceph-mon[32752]: pgmap v14104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:37:15.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:14 smithi167 ceph-mon[32751]: pgmap v14104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:37:16.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25317.373340 secs 2024-04-27T01:37:16.188 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25317.373340 secs 2024-04-27T01:37:16.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25317.373340 secs 2024-04-27T01:37:17.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:16 smithi090 ceph-mon[29180]: pgmap v14105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:17.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25318.189676 secs 2024-04-27T01:37:17.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:16 smithi119 ceph-mon[32752]: pgmap v14105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:17.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25318.189676 secs 2024-04-27T01:37:17.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:16 smithi167 ceph-mon[32751]: pgmap v14105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:37:17.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25318.189676 secs 2024-04-27T01:37:19.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:18 smithi090 ceph-mon[29180]: pgmap v14106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:19.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:18 smithi119 ceph-mon[32752]: pgmap v14106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:19.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:18 smithi167 ceph-mon[32751]: pgmap v14106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:21.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:20 smithi090 ceph-mon[29180]: pgmap v14107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:21.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25322.373440 secs 2024-04-27T01:37:21.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:20 smithi119 ceph-mon[32752]: pgmap v14107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:21.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25322.373440 secs 2024-04-27T01:37:21.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:20 smithi167 ceph-mon[32751]: pgmap v14107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:21.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25322.373440 secs 2024-04-27T01:37:22.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25323.189752 secs 2024-04-27T01:37:22.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25323.189752 secs 2024-04-27T01:37:22.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25323.189752 secs 2024-04-27T01:37:23.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:22 smithi090 ceph-mon[29180]: pgmap v14108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:23.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:22 smithi119 ceph-mon[32752]: pgmap v14108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:23.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:22 smithi167 ceph-mon[32751]: pgmap v14108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:25.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:24 smithi090 ceph-mon[29180]: pgmap v14109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:25.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:24 smithi119 ceph-mon[32752]: pgmap v14109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:25.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:24 smithi167 ceph-mon[32751]: pgmap v14109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25327.373501 secs 2024-04-27T01:37:26.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25327.373501 secs 2024-04-27T01:37:26.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25327.373501 secs 2024-04-27T01:37:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:26 smithi090 ceph-mon[29180]: pgmap v14110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25328.189872 secs 2024-04-27T01:37:27.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:26 smithi119 ceph-mon[32752]: pgmap v14110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:27.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25328.189872 secs 2024-04-27T01:37:27.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:26 smithi167 ceph-mon[32751]: pgmap v14110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:27.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25328.189872 secs 2024-04-27T01:37:29.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:28 smithi090 ceph-mon[29180]: pgmap v14111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:29.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:28 smithi119 ceph-mon[32752]: pgmap v14111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:29.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:28 smithi167 ceph-mon[32751]: pgmap v14111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:31.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:30 smithi167 ceph-mon[32751]: pgmap v14112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:31.104 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25332.373585 secs 2024-04-27T01:37:31.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:30 smithi090 ceph-mon[29180]: pgmap v14112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:31.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25332.373585 secs 2024-04-27T01:37:31.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:30 smithi119 ceph-mon[32752]: pgmap v14112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:31.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25332.373585 secs 2024-04-27T01:37:31.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25333.189972 secs 2024-04-27T01:37:32.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25333.189972 secs 2024-04-27T01:37:32.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25333.189972 secs 2024-04-27T01:37:33.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:32 smithi090 ceph-mon[29180]: pgmap v14113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:33.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:32 smithi119 ceph-mon[32752]: pgmap v14113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:33.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:32 smithi167 ceph-mon[32751]: pgmap v14113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:35.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:34 smithi090 ceph-mon[29180]: pgmap v14114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:35.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:34 smithi119 ceph-mon[32752]: pgmap v14114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:35.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:34 smithi167 ceph-mon[32751]: pgmap v14114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25337.373669 secs 2024-04-27T01:37:36.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25337.373669 secs 2024-04-27T01:37:36.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25337.373669 secs 2024-04-27T01:37:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:36 smithi090 ceph-mon[29180]: pgmap v14115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25338.190114 secs 2024-04-27T01:37:37.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:36 smithi119 ceph-mon[32752]: pgmap v14115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:37.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25338.190114 secs 2024-04-27T01:37:37.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:36 smithi167 ceph-mon[32751]: pgmap v14115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:37.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25338.190114 secs 2024-04-27T01:37:39.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:38 smithi119 ceph-mon[32752]: pgmap v14116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:39.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:38 smithi167 ceph-mon[32751]: pgmap v14116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:39.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:38 smithi090 ceph-mon[29180]: pgmap v14116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:41.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:40 smithi119 ceph-mon[32752]: pgmap v14117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:41.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:40 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25342.373750 secs 2024-04-27T01:37:41.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:40 smithi119 ceph-mon[32752]: slow request 722.542361 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:37:41.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:40 smithi167 ceph-mon[32751]: pgmap v14117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:41.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:40 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25342.373750 secs 2024-04-27T01:37:41.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:40 smithi167 ceph-mon[32751]: slow request 722.542361 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:37:41.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:40 smithi090 ceph-mon[29180]: pgmap v14117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:41.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:40 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25342.373750 secs 2024-04-27T01:37:41.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:40 smithi090 ceph-mon[29180]: slow request 722.542361 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:37:42.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25343.190250 secs 2024-04-27T01:37:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25343.190250 secs 2024-04-27T01:37:42.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25343.190250 secs 2024-04-27T01:37:43.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:42 smithi119 ceph-mon[32752]: pgmap v14118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:43.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:42 smithi167 ceph-mon[32751]: pgmap v14118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:43.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:42 smithi090 ceph-mon[29180]: pgmap v14118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:45.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:44 smithi119 ceph-mon[32752]: pgmap v14119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:45.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:44 smithi167 ceph-mon[32751]: pgmap v14119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:45.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:44 smithi090 ceph-mon[29180]: pgmap v14119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:46.188 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25347.373848 secs 2024-04-27T01:37:46.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25347.373848 secs 2024-04-27T01:37:46.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25347.373848 secs 2024-04-27T01:37:47.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:46 smithi119 ceph-mon[32752]: pgmap v14120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:47.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25348.190406 secs 2024-04-27T01:37:47.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:46 smithi167 ceph-mon[32751]: pgmap v14120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:47.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25348.190406 secs 2024-04-27T01:37:47.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:46 smithi090 ceph-mon[29180]: pgmap v14120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:47.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25348.190406 secs 2024-04-27T01:37:49.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:48 smithi119 ceph-mon[32752]: pgmap v14121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:49.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:48 smithi167 ceph-mon[32751]: pgmap v14121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:49.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:48 smithi090 ceph-mon[29180]: pgmap v14121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:50.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:49 smithi119 ceph-mon[32752]: pgmap v14122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:50.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:49 smithi167 ceph-mon[32751]: pgmap v14122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:50.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:49 smithi090 ceph-mon[29180]: pgmap v14122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:51.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25352.373909 secs 2024-04-27T01:37:51.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25352.373909 secs 2024-04-27T01:37:51.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25352.373909 secs 2024-04-27T01:37:52.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25353.190448 secs 2024-04-27T01:37:52.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:51 smithi119 ceph-mon[32752]: pgmap v14123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25353.190448 secs 2024-04-27T01:37:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:51 smithi167 ceph-mon[32751]: pgmap v14123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:52.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25353.190448 secs 2024-04-27T01:37:52.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:51 smithi090 ceph-mon[29180]: pgmap v14123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:54.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:54 smithi119 ceph-mon[32752]: pgmap v14124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:54.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:54 smithi167 ceph-mon[32751]: pgmap v14124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:54.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:54 smithi090 ceph-mon[29180]: pgmap v14124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:56.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:56 smithi119 ceph-mon[32752]: pgmap v14125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:56.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25357.373991 secs 2024-04-27T01:37:56.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25358.190531 secs 2024-04-27T01:37:56.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:56 smithi167 ceph-mon[32751]: pgmap v14125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:56.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25357.373991 secs 2024-04-27T01:37:56.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25358.190531 secs 2024-04-27T01:37:56.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:56 smithi090 ceph-mon[29180]: pgmap v14125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:37:56.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25357.373991 secs 2024-04-27T01:37:56.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25358.190531 secs 2024-04-27T01:37:58.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:58 smithi119 ceph-mon[32752]: pgmap v14126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:58.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:37:58.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:37:58.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:37:58.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:58 smithi119 ceph-mon[32752]: pgmap v14127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:37:58.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:58 smithi119 ceph-mon[32752]: pgmap v14128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:37:58.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:37:58.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:58 smithi167 ceph-mon[32751]: pgmap v14126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:58.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:37:58.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:37:58.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:37:58.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:58 smithi167 ceph-mon[32751]: pgmap v14127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:37:58.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:58 smithi167 ceph-mon[32751]: pgmap v14128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:37:58.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:37:58.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:58 smithi090 ceph-mon[29180]: pgmap v14126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:37:58.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:37:58.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:37:58.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:37:58.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:58 smithi090 ceph-mon[29180]: pgmap v14127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:37:58.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:58 smithi090 ceph-mon[29180]: pgmap v14128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:37:58.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:38:00.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:37:59 smithi119 ceph-mon[32752]: pgmap v14129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:00.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:37:59 smithi167 ceph-mon[32751]: pgmap v14129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:00.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:37:59 smithi090 ceph-mon[29180]: pgmap v14129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:01.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25362.374094 secs 2024-04-27T01:38:01.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25362.374094 secs 2024-04-27T01:38:01.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25362.374094 secs 2024-04-27T01:38:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25363.190643 secs 2024-04-27T01:38:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:01 smithi090 ceph-mon[29180]: pgmap v14130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:02.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25363.190643 secs 2024-04-27T01:38:02.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:01 smithi119 ceph-mon[32752]: pgmap v14130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25363.190643 secs 2024-04-27T01:38:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:01 smithi167 ceph-mon[32751]: pgmap v14130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:04.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:03 smithi119 ceph-mon[32752]: pgmap v14131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:04.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:03 smithi167 ceph-mon[32751]: pgmap v14131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:04.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:03 smithi090 ceph-mon[29180]: pgmap v14131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:06.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25367.374155 secs 2024-04-27T01:38:06.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25367.374155 secs 2024-04-27T01:38:06.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25367.374155 secs 2024-04-27T01:38:07.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:06 smithi119 ceph-mon[32752]: pgmap v14132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:07.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25368.190714 secs 2024-04-27T01:38:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:06 smithi167 ceph-mon[32751]: pgmap v14132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25368.190714 secs 2024-04-27T01:38:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:06 smithi090 ceph-mon[29180]: pgmap v14132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25368.190714 secs 2024-04-27T01:38:08.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:07 smithi119 ceph-mon[32752]: pgmap v14133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:38:08.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:07 smithi167 ceph-mon[32751]: pgmap v14133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:38:08.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:07 smithi090 ceph-mon[29180]: pgmap v14133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:38:10.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:09 smithi119 ceph-mon[32752]: pgmap v14134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:10.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:09 smithi167 ceph-mon[32751]: pgmap v14134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:10.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:09 smithi090 ceph-mon[29180]: pgmap v14134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:11.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25372.374218 secs 2024-04-27T01:38:11.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25372.374218 secs 2024-04-27T01:38:11.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25372.374218 secs 2024-04-27T01:38:12.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25373.190796 secs 2024-04-27T01:38:12.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:11 smithi119 ceph-mon[32752]: pgmap v14135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:12.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25373.190796 secs 2024-04-27T01:38:12.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:11 smithi167 ceph-mon[32751]: pgmap v14135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:12.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25373.190796 secs 2024-04-27T01:38:12.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:11 smithi090 ceph-mon[29180]: pgmap v14135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:14.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:13 smithi119 ceph-mon[32752]: pgmap v14136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:14.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:13 smithi167 ceph-mon[32751]: pgmap v14136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:14.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:13 smithi090 ceph-mon[29180]: pgmap v14136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:16.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25377.374300 secs 2024-04-27T01:38:16.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25377.374300 secs 2024-04-27T01:38:16.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25377.374300 secs 2024-04-27T01:38:17.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:16 smithi090 ceph-mon[29180]: pgmap v14137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:17.011 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25378.190859 secs 2024-04-27T01:38:17.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:16 smithi119 ceph-mon[32752]: pgmap v14137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:17.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25378.190859 secs 2024-04-27T01:38:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:16 smithi167 ceph-mon[32751]: pgmap v14137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25378.190859 secs 2024-04-27T01:38:18.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:17 smithi119 ceph-mon[32752]: pgmap v14138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:18.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:17 smithi167 ceph-mon[32751]: pgmap v14138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:18.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:17 smithi090 ceph-mon[29180]: pgmap v14138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:20.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:19 smithi119 ceph-mon[32752]: pgmap v14139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:20.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:19 smithi167 ceph-mon[32751]: pgmap v14139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:20.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:19 smithi090 ceph-mon[29180]: pgmap v14139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:21.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25382.374384 secs 2024-04-27T01:38:21.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25382.374384 secs 2024-04-27T01:38:21.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25382.374384 secs 2024-04-27T01:38:22.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25383.190931 secs 2024-04-27T01:38:22.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:21 smithi119 ceph-mon[32752]: pgmap v14140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:22.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25383.190931 secs 2024-04-27T01:38:22.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:21 smithi167 ceph-mon[32751]: pgmap v14140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:22.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25383.190931 secs 2024-04-27T01:38:22.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:21 smithi090 ceph-mon[29180]: pgmap v14140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:24.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:23 smithi119 ceph-mon[32752]: pgmap v14141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:24.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:23 smithi167 ceph-mon[32751]: pgmap v14141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:24.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:23 smithi090 ceph-mon[29180]: pgmap v14141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:26.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25387.374441 secs 2024-04-27T01:38:26.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25387.374441 secs 2024-04-27T01:38:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25387.374441 secs 2024-04-27T01:38:27.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:26 smithi119 ceph-mon[32752]: pgmap v14142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:27.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25388.191044 secs 2024-04-27T01:38:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:26 smithi167 ceph-mon[32751]: pgmap v14142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25388.191044 secs 2024-04-27T01:38:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:26 smithi090 ceph-mon[29180]: pgmap v14142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25388.191044 secs 2024-04-27T01:38:28.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:27 smithi119 ceph-mon[32752]: pgmap v14143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:28.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:27 smithi167 ceph-mon[32751]: pgmap v14143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:28.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:27 smithi090 ceph-mon[29180]: pgmap v14143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:30.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:29 smithi119 ceph-mon[32752]: pgmap v14144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:30.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:29 smithi167 ceph-mon[32751]: pgmap v14144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:30.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:29 smithi090 ceph-mon[29180]: pgmap v14144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:31.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25392.374526 secs 2024-04-27T01:38:31.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25392.374526 secs 2024-04-27T01:38:31.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25392.374526 secs 2024-04-27T01:38:32.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25393.191158 secs 2024-04-27T01:38:32.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:31 smithi119 ceph-mon[32752]: pgmap v14145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:32.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25393.191158 secs 2024-04-27T01:38:32.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:31 smithi167 ceph-mon[32751]: pgmap v14145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:32.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25393.191158 secs 2024-04-27T01:38:32.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:31 smithi090 ceph-mon[29180]: pgmap v14145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:34.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:33 smithi119 ceph-mon[32752]: pgmap v14146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:33 smithi167 ceph-mon[32751]: pgmap v14146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:34.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:33 smithi090 ceph-mon[29180]: pgmap v14146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:36.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25397.374603 secs 2024-04-27T01:38:36.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25397.374603 secs 2024-04-27T01:38:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25397.374603 secs 2024-04-27T01:38:37.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:36 smithi119 ceph-mon[32752]: pgmap v14147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:37.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25398.191292 secs 2024-04-27T01:38:37.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:36 smithi167 ceph-mon[32751]: pgmap v14147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:37.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25398.191292 secs 2024-04-27T01:38:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:36 smithi090 ceph-mon[29180]: pgmap v14147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25398.191292 secs 2024-04-27T01:38:38.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:37 smithi119 ceph-mon[32752]: pgmap v14148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:38.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:37 smithi167 ceph-mon[32751]: pgmap v14148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:38.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:37 smithi090 ceph-mon[29180]: pgmap v14148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:40.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:39 smithi119 ceph-mon[32752]: pgmap v14149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:40.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:39 smithi167 ceph-mon[32751]: pgmap v14149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:40.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:39 smithi090 ceph-mon[29180]: pgmap v14149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:41.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25402.374688 secs 2024-04-27T01:38:41.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25402.374688 secs 2024-04-27T01:38:41.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25402.374688 secs 2024-04-27T01:38:42.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25403.191414 secs 2024-04-27T01:38:42.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:41 smithi119 ceph-mon[32752]: pgmap v14150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25403.191414 secs 2024-04-27T01:38:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:41 smithi167 ceph-mon[32751]: pgmap v14150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:42.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25403.191414 secs 2024-04-27T01:38:42.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:41 smithi090 ceph-mon[29180]: pgmap v14150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:44.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:43 smithi119 ceph-mon[32752]: pgmap v14151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:44.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:43 smithi167 ceph-mon[32751]: pgmap v14151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:44.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:43 smithi090 ceph-mon[29180]: pgmap v14151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:46.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25407.374771 secs 2024-04-27T01:38:46.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25407.374771 secs 2024-04-27T01:38:46.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25407.374771 secs 2024-04-27T01:38:47.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:46 smithi090 ceph-mon[29180]: pgmap v14152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:47.010 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25408.191495 secs 2024-04-27T01:38:47.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:46 smithi119 ceph-mon[32752]: pgmap v14152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:47.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25408.191495 secs 2024-04-27T01:38:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:46 smithi167 ceph-mon[32751]: pgmap v14152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25408.191495 secs 2024-04-27T01:38:48.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:47 smithi119 ceph-mon[32752]: pgmap v14153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:48.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:47 smithi167 ceph-mon[32751]: pgmap v14153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:48.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:47 smithi090 ceph-mon[29180]: pgmap v14153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:50.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:49 smithi119 ceph-mon[32752]: pgmap v14154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:50.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:49 smithi167 ceph-mon[32751]: pgmap v14154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:50.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:49 smithi090 ceph-mon[29180]: pgmap v14154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:51.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25412.374862 secs 2024-04-27T01:38:51.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25412.374862 secs 2024-04-27T01:38:51.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25412.374862 secs 2024-04-27T01:38:52.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25413.191553 secs 2024-04-27T01:38:52.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:51 smithi119 ceph-mon[32752]: pgmap v14155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25413.191553 secs 2024-04-27T01:38:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:51 smithi167 ceph-mon[32751]: pgmap v14155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:52.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25413.191553 secs 2024-04-27T01:38:52.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:51 smithi090 ceph-mon[29180]: pgmap v14155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:54.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:53 smithi119 ceph-mon[32752]: pgmap v14156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:54.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:53 smithi167 ceph-mon[32751]: pgmap v14156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:54.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:53 smithi090 ceph-mon[29180]: pgmap v14156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:38:56.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25417.374954 secs 2024-04-27T01:38:56.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25417.374954 secs 2024-04-27T01:38:56.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25417.374954 secs 2024-04-27T01:38:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:56 smithi119 ceph-mon[32752]: pgmap v14157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25418.191649 secs 2024-04-27T01:38:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:56 smithi167 ceph-mon[32751]: pgmap v14157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25418.191649 secs 2024-04-27T01:38:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:56 smithi090 ceph-mon[29180]: pgmap v14157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25418.191649 secs 2024-04-27T01:38:58.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:57 smithi119 ceph-mon[32752]: pgmap v14158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:58.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:57 smithi167 ceph-mon[32751]: pgmap v14158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:58.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:57 smithi090 ceph-mon[29180]: pgmap v14158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:38:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:38:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:38:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:38:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:58 smithi119 ceph-mon[32752]: pgmap v14159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:38:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:58 smithi119 ceph-mon[32752]: pgmap v14160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:38:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:38:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:38:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:38:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:38:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:58 smithi167 ceph-mon[32751]: pgmap v14159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:38:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:58 smithi167 ceph-mon[32751]: pgmap v14160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:38:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:38:59.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:38:59.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:38:59.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:38:59.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:58 smithi090 ceph-mon[29180]: pgmap v14159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:38:59.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:58 smithi090 ceph-mon[29180]: pgmap v14160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:38:59.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:38:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:39:01.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25422.375042 secs 2024-04-27T01:39:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25422.375042 secs 2024-04-27T01:39:01.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25422.375042 secs 2024-04-27T01:39:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:01 smithi119 ceph-mon[32752]: pgmap v14161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25423.191765 secs 2024-04-27T01:39:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:01 smithi167 ceph-mon[32751]: pgmap v14161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25423.191765 secs 2024-04-27T01:39:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:01 smithi090 ceph-mon[29180]: pgmap v14161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:02.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25423.191765 secs 2024-04-27T01:39:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:03 smithi119 ceph-mon[32752]: pgmap v14162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:39:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:03 smithi167 ceph-mon[32751]: pgmap v14162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:39:04.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:03 smithi090 ceph-mon[29180]: pgmap v14162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:39:06.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:05 smithi119 ceph-mon[32752]: pgmap v14163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:06.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25427.375127 secs 2024-04-27T01:39:06.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:05 smithi167 ceph-mon[32751]: pgmap v14163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:06.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25427.375127 secs 2024-04-27T01:39:06.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:05 smithi090 ceph-mon[29180]: pgmap v14163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:06.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25427.375127 secs 2024-04-27T01:39:07.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25428.191885 secs 2024-04-27T01:39:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25428.191885 secs 2024-04-27T01:39:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25428.191885 secs 2024-04-27T01:39:08.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:07 smithi119 ceph-mon[32752]: pgmap v14164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:08.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:07 smithi167 ceph-mon[32751]: pgmap v14164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:08.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:07 smithi090 ceph-mon[29180]: pgmap v14164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:39:10.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:09 smithi119 ceph-mon[32752]: pgmap v14165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:39:10.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:09 smithi167 ceph-mon[32751]: pgmap v14165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:39:10.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:09 smithi090 ceph-mon[29180]: pgmap v14165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:39:11.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25432.375211 secs 2024-04-27T01:39:11.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25432.375211 secs 2024-04-27T01:39:11.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25432.375211 secs 2024-04-27T01:39:12.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:11 smithi119 ceph-mon[32752]: pgmap v14166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:12.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25433.192000 secs 2024-04-27T01:39:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:11 smithi167 ceph-mon[32751]: pgmap v14166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25433.192000 secs 2024-04-27T01:39:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:11 smithi090 ceph-mon[29180]: pgmap v14166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:12.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25433.192000 secs 2024-04-27T01:39:14.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:13 smithi119 ceph-mon[32752]: pgmap v14167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:14.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:13 smithi167 ceph-mon[32751]: pgmap v14167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:14.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:13 smithi090 ceph-mon[29180]: pgmap v14167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:16.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:15 smithi119 ceph-mon[32752]: pgmap v14168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:16.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25437.375295 secs 2024-04-27T01:39:16.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:15 smithi167 ceph-mon[32751]: pgmap v14168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:16.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25437.375295 secs 2024-04-27T01:39:16.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:15 smithi090 ceph-mon[29180]: pgmap v14168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:16.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25437.375295 secs 2024-04-27T01:39:17.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25438.192135 secs 2024-04-27T01:39:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25438.192135 secs 2024-04-27T01:39:17.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25438.192135 secs 2024-04-27T01:39:18.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:17 smithi119 ceph-mon[32752]: pgmap v14169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:18.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:17 smithi167 ceph-mon[32751]: pgmap v14169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:18.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:17 smithi090 ceph-mon[29180]: pgmap v14169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:19.911 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:19 smithi090 ceph-mon[29180]: pgmap v14170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:20.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:19 smithi119 ceph-mon[32752]: pgmap v14170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:20.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:19 smithi167 ceph-mon[32751]: pgmap v14170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:21.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25442.375379 secs 2024-04-27T01:39:21.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25442.375379 secs 2024-04-27T01:39:21.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25442.375379 secs 2024-04-27T01:39:22.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:21 smithi119 ceph-mon[32752]: pgmap v14171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:22.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25443.192253 secs 2024-04-27T01:39:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:21 smithi167 ceph-mon[32751]: pgmap v14171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25443.192253 secs 2024-04-27T01:39:22.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:21 smithi090 ceph-mon[29180]: pgmap v14171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:22.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25443.192253 secs 2024-04-27T01:39:24.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:23 smithi119 ceph-mon[32752]: pgmap v14172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:24.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:23 smithi167 ceph-mon[32751]: pgmap v14172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:24.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:23 smithi090 ceph-mon[29180]: pgmap v14172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:26.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:25 smithi119 ceph-mon[32752]: pgmap v14173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:26.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25447.375465 secs 2024-04-27T01:39:26.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:25 smithi167 ceph-mon[32751]: pgmap v14173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:26.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25447.375465 secs 2024-04-27T01:39:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:25 smithi090 ceph-mon[29180]: pgmap v14173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:26.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25447.375465 secs 2024-04-27T01:39:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25448.192323 secs 2024-04-27T01:39:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25448.192323 secs 2024-04-27T01:39:27.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25448.192323 secs 2024-04-27T01:39:28.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:27 smithi167 ceph-mon[32751]: pgmap v14174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:28.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:27 smithi090 ceph-mon[29180]: pgmap v14174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:28.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:27 smithi119 ceph-mon[32752]: pgmap v14174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:30.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:29 smithi167 ceph-mon[32751]: pgmap v14175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:30.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:29 smithi090 ceph-mon[29180]: pgmap v14175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:30.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:29 smithi119 ceph-mon[32752]: pgmap v14175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:31.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25452.375546 secs 2024-04-27T01:39:31.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25452.375546 secs 2024-04-27T01:39:31.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25452.375546 secs 2024-04-27T01:39:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:31 smithi090 ceph-mon[29180]: pgmap v14176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25453.192479 secs 2024-04-27T01:39:32.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:31 smithi119 ceph-mon[32752]: pgmap v14176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:32.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25453.192479 secs 2024-04-27T01:39:32.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:31 smithi167 ceph-mon[32751]: pgmap v14176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:32.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25453.192479 secs 2024-04-27T01:39:34.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:33 smithi090 ceph-mon[29180]: pgmap v14177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:34.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:33 smithi119 ceph-mon[32752]: pgmap v14177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:33 smithi167 ceph-mon[32751]: pgmap v14177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:35 smithi090 ceph-mon[29180]: pgmap v14178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:36.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25457.375604 secs 2024-04-27T01:39:36.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:35 smithi119 ceph-mon[32752]: pgmap v14178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:36.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25457.375604 secs 2024-04-27T01:39:36.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:35 smithi167 ceph-mon[32751]: pgmap v14178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:36.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25457.375604 secs 2024-04-27T01:39:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25458.192590 secs 2024-04-27T01:39:37.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25458.192590 secs 2024-04-27T01:39:37.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25458.192590 secs 2024-04-27T01:39:38.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:37 smithi090 ceph-mon[29180]: pgmap v14179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:38.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:37 smithi119 ceph-mon[32752]: pgmap v14179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:38.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:37 smithi167 ceph-mon[32751]: pgmap v14179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:40.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:39 smithi090 ceph-mon[29180]: pgmap v14180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:40.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:39 smithi119 ceph-mon[32752]: pgmap v14180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:40.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:39 smithi167 ceph-mon[32751]: pgmap v14180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:41.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25462.375664 secs 2024-04-27T01:39:41.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25462.375664 secs 2024-04-27T01:39:41.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25462.375664 secs 2024-04-27T01:39:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:41 smithi090 ceph-mon[29180]: pgmap v14181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25463.192703 secs 2024-04-27T01:39:42.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:41 smithi119 ceph-mon[32752]: pgmap v14181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:42.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25463.192703 secs 2024-04-27T01:39:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:41 smithi167 ceph-mon[32751]: pgmap v14181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25463.192703 secs 2024-04-27T01:39:44.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:43 smithi090 ceph-mon[29180]: pgmap v14182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:44.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:43 smithi119 ceph-mon[32752]: pgmap v14182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:44.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:43 smithi167 ceph-mon[32751]: pgmap v14182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:46.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:45 smithi119 ceph-mon[32752]: pgmap v14183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:46.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25467.375725 secs 2024-04-27T01:39:46.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:45 smithi090 ceph-mon[29180]: pgmap v14183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:46.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25467.375725 secs 2024-04-27T01:39:46.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:45 smithi167 ceph-mon[32751]: pgmap v14183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:46.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25467.375725 secs 2024-04-27T01:39:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25468.192827 secs 2024-04-27T01:39:47.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25468.192827 secs 2024-04-27T01:39:47.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25468.192827 secs 2024-04-27T01:39:48.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:47 smithi090 ceph-mon[29180]: pgmap v14184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:48.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:47 smithi119 ceph-mon[32752]: pgmap v14184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:48.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:47 smithi167 ceph-mon[32751]: pgmap v14184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:50.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:49 smithi090 ceph-mon[29180]: pgmap v14185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:50.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:49 smithi119 ceph-mon[32752]: pgmap v14185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:50.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:49 smithi167 ceph-mon[32751]: pgmap v14185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:51.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25472.375810 secs 2024-04-27T01:39:51.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25472.375810 secs 2024-04-27T01:39:51.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25472.375810 secs 2024-04-27T01:39:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:51 smithi090 ceph-mon[29180]: pgmap v14186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25473.192976 secs 2024-04-27T01:39:52.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:51 smithi119 ceph-mon[32752]: pgmap v14186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:52.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25473.192976 secs 2024-04-27T01:39:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:51 smithi167 ceph-mon[32751]: pgmap v14186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25473.192976 secs 2024-04-27T01:39:54.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:53 smithi090 ceph-mon[29180]: pgmap v14187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:54.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:53 smithi119 ceph-mon[32752]: pgmap v14187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:54.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:53 smithi167 ceph-mon[32751]: pgmap v14187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:56.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:55 smithi090 ceph-mon[29180]: pgmap v14188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:56.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25477.375823 secs 2024-04-27T01:39:56.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:55 smithi119 ceph-mon[32752]: pgmap v14188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:56.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25477.375823 secs 2024-04-27T01:39:56.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:55 smithi167 ceph-mon[32751]: pgmap v14188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:39:56.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25477.375823 secs 2024-04-27T01:39:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25478.193117 secs 2024-04-27T01:39:57.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25478.193117 secs 2024-04-27T01:39:57.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25478.193117 secs 2024-04-27T01:39:58.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:57 smithi119 ceph-mon[32752]: pgmap v14189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:58.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:57 smithi167 ceph-mon[32751]: pgmap v14189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:58.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:57 smithi090 ceph-mon[29180]: pgmap v14189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:39:59.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:39:59.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:39:59.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:40:00.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:59 smithi119 ceph-mon[32752]: pgmap v14190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:00.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:40:00.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:40:00.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:39:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:40:00.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:59 smithi167 ceph-mon[32751]: pgmap v14190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:00.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:40:00.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:40:00.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:39:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:40:00.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:59 smithi090 ceph-mon[29180]: pgmap v14190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:00.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:40:00.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:40:00.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:39:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:40:01.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:00 smithi167 ceph-mon[32751]: pgmap v14191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:40:01.203 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:00 smithi167 ceph-mon[32751]: pgmap v14192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:01.203 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:40:01.203 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25482.375913 secs 2024-04-27T01:40:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:00 smithi119 ceph-mon[32752]: pgmap v14191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:40:01.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:00 smithi119 ceph-mon[32752]: pgmap v14192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:01.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:40:01.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25482.375913 secs 2024-04-27T01:40:01.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:00 smithi090 ceph-mon[29180]: pgmap v14191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:40:01.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:00 smithi090 ceph-mon[29180]: pgmap v14192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:01.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:40:01.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25482.375913 secs 2024-04-27T01:40:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25483.193241 secs 2024-04-27T01:40:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25483.193241 secs 2024-04-27T01:40:02.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25483.193241 secs 2024-04-27T01:40:03.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:02 smithi119 ceph-mon[32752]: pgmap v14193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:03.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:02 smithi167 ceph-mon[32751]: pgmap v14193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:03.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:02 smithi090 ceph-mon[29180]: pgmap v14193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:05.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:04 smithi119 ceph-mon[32752]: pgmap v14194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:05.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:04 smithi167 ceph-mon[32751]: pgmap v14194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:05.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:04 smithi090 ceph-mon[29180]: pgmap v14194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:06.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25487.376005 secs 2024-04-27T01:40:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25487.376005 secs 2024-04-27T01:40:06.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25487.376005 secs 2024-04-27T01:40:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:06 smithi090 ceph-mon[29180]: pgmap v14195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:07.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25488.193352 secs 2024-04-27T01:40:07.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:06 smithi119 ceph-mon[32752]: pgmap v14195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:07.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25488.193352 secs 2024-04-27T01:40:07.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:06 smithi167 ceph-mon[32751]: pgmap v14195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:40:07.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25488.193352 secs 2024-04-27T01:40:09.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:08 smithi119 ceph-mon[32752]: pgmap v14196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:40:09.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:08 smithi167 ceph-mon[32751]: pgmap v14196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:40:09.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:08 smithi090 ceph-mon[29180]: pgmap v14196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:40:11.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:10 smithi119 ceph-mon[32752]: pgmap v14197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:40:11.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25492.376113 secs 2024-04-27T01:40:11.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:10 smithi167 ceph-mon[32751]: pgmap v14197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:40:11.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25492.376113 secs 2024-04-27T01:40:11.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:10 smithi090 ceph-mon[29180]: pgmap v14197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:40:11.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25492.376113 secs 2024-04-27T01:40:12.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25493.193475 secs 2024-04-27T01:40:12.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25493.193475 secs 2024-04-27T01:40:12.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25493.193475 secs 2024-04-27T01:40:13.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:12 smithi119 ceph-mon[32752]: pgmap v14198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:13.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:12 smithi167 ceph-mon[32751]: pgmap v14198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:13.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:12 smithi090 ceph-mon[29180]: pgmap v14198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:14.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:13 smithi119 ceph-mon[32752]: pgmap v14199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:14.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:13 smithi167 ceph-mon[32751]: pgmap v14199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:14.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:13 smithi090 ceph-mon[29180]: pgmap v14199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:16.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25497.376212 secs 2024-04-27T01:40:16.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:16 smithi119 ceph-mon[32752]: pgmap v14200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:16.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25498.193602 secs 2024-04-27T01:40:16.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25497.376212 secs 2024-04-27T01:40:16.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:16 smithi167 ceph-mon[32751]: pgmap v14200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:16.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25498.193602 secs 2024-04-27T01:40:16.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25497.376212 secs 2024-04-27T01:40:16.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:16 smithi090 ceph-mon[29180]: pgmap v14200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:16.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25498.193602 secs 2024-04-27T01:40:18.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:18 smithi119 ceph-mon[32752]: pgmap v14201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:18.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:18 smithi167 ceph-mon[32751]: pgmap v14201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:18.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:18 smithi090 ceph-mon[29180]: pgmap v14201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:20.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:20 smithi119 ceph-mon[32752]: pgmap v14202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:20.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:20 smithi167 ceph-mon[32751]: pgmap v14202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:20.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:20 smithi090 ceph-mon[29180]: pgmap v14202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:21.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25502.376310 secs 2024-04-27T01:40:21.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25503.193680 secs 2024-04-27T01:40:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25502.376310 secs 2024-04-27T01:40:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25503.193680 secs 2024-04-27T01:40:21.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25502.376310 secs 2024-04-27T01:40:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25503.193680 secs 2024-04-27T01:40:22.760 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:22 smithi090 ceph-mon[29180]: pgmap v14203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:22.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:22 smithi119 ceph-mon[32752]: pgmap v14203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:22.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:22 smithi167 ceph-mon[32751]: pgmap v14203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:24.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:24 smithi119 ceph-mon[32752]: pgmap v14204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:24.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:24 smithi167 ceph-mon[32751]: pgmap v14204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:24.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:24 smithi090 ceph-mon[29180]: pgmap v14204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:26.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25507.376413 secs 2024-04-27T01:40:26.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:26 smithi119 ceph-mon[32752]: pgmap v14205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:26.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25508.193808 secs 2024-04-27T01:40:26.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25507.376413 secs 2024-04-27T01:40:26.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:26 smithi167 ceph-mon[32751]: pgmap v14205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:26.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25508.193808 secs 2024-04-27T01:40:26.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25507.376413 secs 2024-04-27T01:40:26.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:26 smithi090 ceph-mon[29180]: pgmap v14205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:26.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25508.193808 secs 2024-04-27T01:40:28.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:28 smithi119 ceph-mon[32752]: pgmap v14206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:28.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:28 smithi167 ceph-mon[32751]: pgmap v14206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:28.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:28 smithi090 ceph-mon[29180]: pgmap v14206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:30.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:30 smithi119 ceph-mon[32752]: pgmap v14207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:30.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:30 smithi167 ceph-mon[32751]: pgmap v14207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:30.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:30 smithi090 ceph-mon[29180]: pgmap v14207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:31.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:31 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25512.376517 secs 2024-04-27T01:40:31.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:31 smithi119 ceph-mon[32752]: slow request 1440.478202 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:40:31.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25513.193944 secs 2024-04-27T01:40:31.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:31 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25512.376517 secs 2024-04-27T01:40:31.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:31 smithi167 ceph-mon[32751]: slow request 1440.478202 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:40:31.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25513.193944 secs 2024-04-27T01:40:31.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:31 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25512.376517 secs 2024-04-27T01:40:31.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:31 smithi090 ceph-mon[29180]: slow request 1440.478202 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:40:31.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25513.193944 secs 2024-04-27T01:40:32.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:32 smithi119 ceph-mon[32752]: pgmap v14208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:32.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:32 smithi167 ceph-mon[32751]: pgmap v14208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:32.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:32 smithi090 ceph-mon[29180]: pgmap v14208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:34.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:34 smithi119 ceph-mon[32752]: pgmap v14209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:34.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:34 smithi167 ceph-mon[32751]: pgmap v14209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:34.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:34 smithi090 ceph-mon[29180]: pgmap v14209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:36.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25517.376615 secs 2024-04-27T01:40:36.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:36 smithi119 ceph-mon[32752]: pgmap v14210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:36.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25518.194061 secs 2024-04-27T01:40:36.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25517.376615 secs 2024-04-27T01:40:36.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:36 smithi167 ceph-mon[32751]: pgmap v14210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:36.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25518.194061 secs 2024-04-27T01:40:36.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25517.376615 secs 2024-04-27T01:40:36.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:36 smithi090 ceph-mon[29180]: pgmap v14210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:36.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25518.194061 secs 2024-04-27T01:40:38.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:38 smithi119 ceph-mon[32752]: pgmap v14211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:38.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:38 smithi167 ceph-mon[32751]: pgmap v14211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:38.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:38 smithi090 ceph-mon[29180]: pgmap v14211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:40.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:40 smithi119 ceph-mon[32752]: pgmap v14212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:40.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:40 smithi167 ceph-mon[32751]: pgmap v14212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:40.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:40 smithi090 ceph-mon[29180]: pgmap v14212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:41.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25522.376712 secs 2024-04-27T01:40:41.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25523.194177 secs 2024-04-27T01:40:41.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25522.376712 secs 2024-04-27T01:40:41.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25523.194177 secs 2024-04-27T01:40:41.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25522.376712 secs 2024-04-27T01:40:41.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25523.194177 secs 2024-04-27T01:40:42.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:42 smithi119 ceph-mon[32752]: pgmap v14213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:42.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:42 smithi167 ceph-mon[32751]: pgmap v14213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:42.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:42 smithi090 ceph-mon[29180]: pgmap v14213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:44.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:44 smithi119 ceph-mon[32752]: pgmap v14214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:44.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:44 smithi167 ceph-mon[32751]: pgmap v14214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:44.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:44 smithi090 ceph-mon[29180]: pgmap v14214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:46.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25527.376820 secs 2024-04-27T01:40:46.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:46 smithi119 ceph-mon[32752]: pgmap v14215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:46.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25528.194297 secs 2024-04-27T01:40:46.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25527.376820 secs 2024-04-27T01:40:46.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:46 smithi167 ceph-mon[32751]: pgmap v14215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:46.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25528.194297 secs 2024-04-27T01:40:46.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25527.376820 secs 2024-04-27T01:40:46.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:46 smithi090 ceph-mon[29180]: pgmap v14215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:46.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25528.194297 secs 2024-04-27T01:40:48.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:48 smithi119 ceph-mon[32752]: pgmap v14216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:48.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:48 smithi167 ceph-mon[32751]: pgmap v14216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:48.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:48 smithi090 ceph-mon[29180]: pgmap v14216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:50.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:50 smithi119 ceph-mon[32752]: pgmap v14217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:50.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:50 smithi167 ceph-mon[32751]: pgmap v14217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:50.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:50 smithi090 ceph-mon[29180]: pgmap v14217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:51.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25532.376924 secs 2024-04-27T01:40:51.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25533.194413 secs 2024-04-27T01:40:51.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25532.376924 secs 2024-04-27T01:40:51.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25533.194413 secs 2024-04-27T01:40:52.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25532.376924 secs 2024-04-27T01:40:52.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25533.194413 secs 2024-04-27T01:40:52.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:52 smithi167 ceph-mon[32751]: pgmap v14218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:52.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:52 smithi090 ceph-mon[29180]: pgmap v14218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:53.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:52 smithi119 ceph-mon[32752]: pgmap v14218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:54.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:54 smithi167 ceph-mon[32751]: pgmap v14219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:54.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:54 smithi090 ceph-mon[29180]: pgmap v14219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:55.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:54 smithi119 ceph-mon[32752]: pgmap v14219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:40:56.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25537.377028 secs 2024-04-27T01:40:56.868 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:56 smithi167 ceph-mon[32751]: pgmap v14220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:56.868 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25538.194556 secs 2024-04-27T01:40:56.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25537.377028 secs 2024-04-27T01:40:56.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:56 smithi090 ceph-mon[29180]: pgmap v14220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:56.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25538.194556 secs 2024-04-27T01:40:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25537.377028 secs 2024-04-27T01:40:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:56 smithi119 ceph-mon[32752]: pgmap v14220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25538.194556 secs 2024-04-27T01:40:58.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:58 smithi090 ceph-mon[29180]: pgmap v14221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:59.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:58 smithi119 ceph-mon[32752]: pgmap v14221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:59.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:58 smithi167 ceph-mon[32751]: pgmap v14221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:40:59.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:40:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:41:00.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:40:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:41:00.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:40:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:41:00.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:00 smithi090 ceph-mon[29180]: pgmap v14222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:00.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:41:00.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:41:00.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:41:01.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:00 smithi119 ceph-mon[32752]: pgmap v14222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:01.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:41:01.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:41:01.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:41:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:00 smithi167 ceph-mon[32751]: pgmap v14222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:41:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:41:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:41:01.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25542.377227 secs 2024-04-27T01:41:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:01 smithi090 ceph-mon[29180]: pgmap v14223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:41:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:01 smithi090 ceph-mon[29180]: pgmap v14224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25543.194657 secs 2024-04-27T01:41:01.952 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25542.377227 secs 2024-04-27T01:41:01.953 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:01 smithi167 ceph-mon[32751]: pgmap v14223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:41:01.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:01 smithi167 ceph-mon[32751]: pgmap v14224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:01.954 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25543.194657 secs 2024-04-27T01:41:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25542.377227 secs 2024-04-27T01:41:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:01 smithi119 ceph-mon[32752]: pgmap v14223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:41:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:01 smithi119 ceph-mon[32752]: pgmap v14224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25543.194657 secs 2024-04-27T01:41:03.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:03 smithi090 ceph-mon[29180]: pgmap v14225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:41:04.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:03 smithi119 ceph-mon[32752]: pgmap v14225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:41:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:03 smithi167 ceph-mon[32751]: pgmap v14225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:41:05.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:05 smithi090 ceph-mon[29180]: pgmap v14226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:06.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:05 smithi119 ceph-mon[32752]: pgmap v14226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:06.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:05 smithi167 ceph-mon[32751]: pgmap v14226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:06.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25547.377314 secs 2024-04-27T01:41:06.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25548.194794 secs 2024-04-27T01:41:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25547.377314 secs 2024-04-27T01:41:07.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25548.194794 secs 2024-04-27T01:41:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25547.377314 secs 2024-04-27T01:41:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25548.194794 secs 2024-04-27T01:41:07.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:07 smithi090 ceph-mon[29180]: pgmap v14227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:08.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:07 smithi119 ceph-mon[32752]: pgmap v14227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:08.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:07 smithi167 ceph-mon[32751]: pgmap v14227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:41:09.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:09 smithi090 ceph-mon[29180]: pgmap v14228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:41:10.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:09 smithi119 ceph-mon[32752]: pgmap v14228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:41:10.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:09 smithi167 ceph-mon[32751]: pgmap v14228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:41:11.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25552.377401 secs 2024-04-27T01:41:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:11 smithi090 ceph-mon[29180]: pgmap v14229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T01:41:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25553.194901 secs 2024-04-27T01:41:12.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25552.377401 secs 2024-04-27T01:41:12.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:11 smithi119 ceph-mon[32752]: pgmap v14229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T01:41:12.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25553.194901 secs 2024-04-27T01:41:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25552.377401 secs 2024-04-27T01:41:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:11 smithi167 ceph-mon[32751]: pgmap v14229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T01:41:12.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25553.194901 secs 2024-04-27T01:41:13.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:13 smithi090 ceph-mon[29180]: pgmap v14230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:14.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:13 smithi119 ceph-mon[32752]: pgmap v14230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:14.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:13 smithi167 ceph-mon[32751]: pgmap v14230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:15.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:15 smithi090 ceph-mon[29180]: pgmap v14231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:16.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:15 smithi119 ceph-mon[32752]: pgmap v14231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:16.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:15 smithi167 ceph-mon[32751]: pgmap v14231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:16.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25557.377489 secs 2024-04-27T01:41:16.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25558.195000 secs 2024-04-27T01:41:16.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25557.377489 secs 2024-04-27T01:41:16.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25558.195000 secs 2024-04-27T01:41:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25557.377489 secs 2024-04-27T01:41:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25558.195000 secs 2024-04-27T01:41:17.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:17 smithi090 ceph-mon[29180]: pgmap v14232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:18.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:17 smithi119 ceph-mon[32752]: pgmap v14232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:18.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:17 smithi167 ceph-mon[32751]: pgmap v14232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:19.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:19 smithi090 ceph-mon[29180]: pgmap v14233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:20.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:19 smithi119 ceph-mon[32752]: pgmap v14233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:20.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:19 smithi167 ceph-mon[32751]: pgmap v14233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:21.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25562.377577 secs 2024-04-27T01:41:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:21 smithi090 ceph-mon[29180]: pgmap v14234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25563.195095 secs 2024-04-27T01:41:22.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25562.377577 secs 2024-04-27T01:41:22.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:21 smithi119 ceph-mon[32752]: pgmap v14234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:22.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25563.195095 secs 2024-04-27T01:41:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25562.377577 secs 2024-04-27T01:41:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:21 smithi167 ceph-mon[32751]: pgmap v14234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:41:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25563.195095 secs 2024-04-27T01:41:24.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:23 smithi119 ceph-mon[32752]: pgmap v14235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:24.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:23 smithi167 ceph-mon[32751]: pgmap v14235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:24.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:23 smithi090 ceph-mon[29180]: pgmap v14235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:25.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:25 smithi090 ceph-mon[29180]: pgmap v14236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:26.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:25 smithi119 ceph-mon[32752]: pgmap v14236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:26.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:25 smithi167 ceph-mon[32751]: pgmap v14236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25567.377638 secs 2024-04-27T01:41:27.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25568.195222 secs 2024-04-27T01:41:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25567.377638 secs 2024-04-27T01:41:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25568.195222 secs 2024-04-27T01:41:27.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25567.377638 secs 2024-04-27T01:41:27.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25568.195222 secs 2024-04-27T01:41:28.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:27 smithi119 ceph-mon[32752]: pgmap v14237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:28.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:27 smithi167 ceph-mon[32751]: pgmap v14237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:28.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:27 smithi090 ceph-mon[29180]: pgmap v14237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:30.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:29 smithi119 ceph-mon[32752]: pgmap v14238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:30.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:29 smithi167 ceph-mon[32751]: pgmap v14238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:30.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:29 smithi090 ceph-mon[29180]: pgmap v14238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25572.377725 secs 2024-04-27T01:41:32.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:31 smithi119 ceph-mon[32752]: pgmap v14239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:32.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25573.195346 secs 2024-04-27T01:41:32.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25572.377725 secs 2024-04-27T01:41:32.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:31 smithi167 ceph-mon[32751]: pgmap v14239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:32.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25573.195346 secs 2024-04-27T01:41:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25572.377725 secs 2024-04-27T01:41:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:31 smithi090 ceph-mon[29180]: pgmap v14239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:32.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25573.195346 secs 2024-04-27T01:41:34.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:33 smithi119 ceph-mon[32752]: pgmap v14240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:34.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:33 smithi167 ceph-mon[32751]: pgmap v14240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:34.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:33 smithi090 ceph-mon[29180]: pgmap v14240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:36.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:35 smithi119 ceph-mon[32752]: pgmap v14241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:36.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:35 smithi167 ceph-mon[32751]: pgmap v14241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:36.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:35 smithi090 ceph-mon[29180]: pgmap v14241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25577.377806 secs 2024-04-27T01:41:37.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25578.195430 secs 2024-04-27T01:41:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25577.377806 secs 2024-04-27T01:41:37.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25578.195430 secs 2024-04-27T01:41:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25577.377806 secs 2024-04-27T01:41:37.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25578.195430 secs 2024-04-27T01:41:38.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:37 smithi119 ceph-mon[32752]: pgmap v14242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:38.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:37 smithi167 ceph-mon[32751]: pgmap v14242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:38.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:37 smithi090 ceph-mon[29180]: pgmap v14242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:38.109 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:41:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:41:37.839+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:41:38.109 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:41:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:41:37.839+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:41:38.109 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:41:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:41:37.839+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:41:38.109 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:41:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:41:37.839+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:41:38.109 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:41:37 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:41:37.839+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:41:40.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:39 smithi119 ceph-mon[32752]: pgmap v14243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:40.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:39 smithi167 ceph-mon[32751]: pgmap v14243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:40.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:39 smithi090 ceph-mon[29180]: pgmap v14243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:42.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25582.377890 secs 2024-04-27T01:41:42.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:41 smithi119 ceph-mon[32752]: pgmap v14244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:42.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:41 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 25583.195542 secs 2024-04-27T01:41:42.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:41 smithi119 ceph-mon[32752]: slow request 2883.890829 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:41:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25582.377890 secs 2024-04-27T01:41:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:41 smithi167 ceph-mon[32751]: pgmap v14244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:41 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 25583.195542 secs 2024-04-27T01:41:42.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:41 smithi167 ceph-mon[32751]: slow request 2883.890829 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:41:42.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25582.377890 secs 2024-04-27T01:41:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:41 smithi090 ceph-mon[29180]: pgmap v14244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:41 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 25583.195542 secs 2024-04-27T01:41:42.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:41 smithi090 ceph-mon[29180]: slow request 2883.890829 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:41:44.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:43 smithi119 ceph-mon[32752]: pgmap v14245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:44.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:43 smithi167 ceph-mon[32751]: pgmap v14245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:44.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:43 smithi090 ceph-mon[29180]: pgmap v14245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:46.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:45 smithi119 ceph-mon[32752]: pgmap v14246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:46.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:45 smithi167 ceph-mon[32751]: pgmap v14246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:46.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:45 smithi090 ceph-mon[29180]: pgmap v14246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:46.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25587.377976 secs 2024-04-27T01:41:46.938 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25588.195678 secs 2024-04-27T01:41:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25587.377976 secs 2024-04-27T01:41:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25588.195678 secs 2024-04-27T01:41:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25587.377976 secs 2024-04-27T01:41:47.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25588.195678 secs 2024-04-27T01:41:48.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:47 smithi119 ceph-mon[32752]: pgmap v14247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:48.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:47 smithi167 ceph-mon[32751]: pgmap v14247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:48.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:47 smithi090 ceph-mon[29180]: pgmap v14247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:50.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:49 smithi119 ceph-mon[32752]: pgmap v14248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:50.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:49 smithi167 ceph-mon[32751]: pgmap v14248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:50.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:49 smithi090 ceph-mon[29180]: pgmap v14248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:41:52.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25592.378060 secs 2024-04-27T01:41:52.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:51 smithi119 ceph-mon[32752]: pgmap v14249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:52.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25593.195751 secs 2024-04-27T01:41:52.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25592.378060 secs 2024-04-27T01:41:52.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:51 smithi167 ceph-mon[32751]: pgmap v14249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:52.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25593.195751 secs 2024-04-27T01:41:52.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25592.378060 secs 2024-04-27T01:41:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:51 smithi090 ceph-mon[29180]: pgmap v14249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:41:52.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25593.195751 secs 2024-04-27T01:41:54.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:53 smithi119 ceph-mon[32752]: pgmap v14250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:54.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:53 smithi167 ceph-mon[32751]: pgmap v14250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:54.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:53 smithi090 ceph-mon[29180]: pgmap v14250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:56.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:55 smithi119 ceph-mon[32752]: pgmap v14251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:56.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:55 smithi167 ceph-mon[32751]: pgmap v14251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:56.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:55 smithi090 ceph-mon[29180]: pgmap v14251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:41:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25597.378147 secs 2024-04-27T01:41:57.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25598.195831 secs 2024-04-27T01:41:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25597.378147 secs 2024-04-27T01:41:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25598.195831 secs 2024-04-27T01:41:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25597.378147 secs 2024-04-27T01:41:57.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25598.195831 secs 2024-04-27T01:41:58.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:57 smithi119 ceph-mon[32752]: pgmap v14252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:58.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:57 smithi167 ceph-mon[32751]: pgmap v14252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:41:58.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:57 smithi090 ceph-mon[29180]: pgmap v14252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:00.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:41:59 smithi119 ceph-mon[32752]: pgmap v14253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:42:00.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:41:59 smithi167 ceph-mon[32751]: pgmap v14253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:42:00.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:41:59 smithi090 ceph-mon[29180]: pgmap v14253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:42:01.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25602.378209 secs 2024-04-27T01:42:01.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:42:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25602.378209 secs 2024-04-27T01:42:01.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:42:01.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25602.378209 secs 2024-04-27T01:42:01.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:42:01.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:01 smithi090 ceph-mon[29180]: pgmap v14254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:01.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25603.195895 secs 2024-04-27T01:42:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:01 smithi119 ceph-mon[32752]: pgmap v14254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25603.195895 secs 2024-04-27T01:42:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:01 smithi167 ceph-mon[32751]: pgmap v14254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:02.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25603.195895 secs 2024-04-27T01:42:03.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: pgmap v14255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:03.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:03.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:03.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: pgmap v14255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: pgmap v14255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:04.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:42:04.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:42:04.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:42:04.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:42:04.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:42:04.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: pgmap v14256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:42:04.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: pgmap v14257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:04.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:05.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:42:05.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:42:05.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:42:05.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:42:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:42:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:42:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: pgmap v14256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:42:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: pgmap v14257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:05.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:42:05.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:42:05.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:42:05.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:42:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:42:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:42:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: pgmap v14256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:42:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: pgmap v14257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:42:06.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25607.378295 secs 2024-04-27T01:42:06.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25607.378295 secs 2024-04-27T01:42:06.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25607.378295 secs 2024-04-27T01:42:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:06 smithi119 ceph-mon[32752]: pgmap v14258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:07.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25608.196041 secs 2024-04-27T01:42:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:06 smithi167 ceph-mon[32751]: pgmap v14258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:07.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25608.196041 secs 2024-04-27T01:42:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:06 smithi090 ceph-mon[29180]: pgmap v14258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25608.196041 secs 2024-04-27T01:42:08.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:07 smithi119 ceph-mon[32752]: pgmap v14259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T01:42:08.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:07 smithi167 ceph-mon[32751]: pgmap v14259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T01:42:08.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:07 smithi090 ceph-mon[29180]: pgmap v14259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T01:42:10.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:09 smithi119 ceph-mon[32752]: pgmap v14260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:10.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:09 smithi167 ceph-mon[32751]: pgmap v14260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:10.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:09 smithi090 ceph-mon[29180]: pgmap v14260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:11.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25612.378377 secs 2024-04-27T01:42:11.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25612.378377 secs 2024-04-27T01:42:11.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25612.378377 secs 2024-04-27T01:42:12.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25613.196102 secs 2024-04-27T01:42:12.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:11 smithi119 ceph-mon[32752]: pgmap v14261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:12.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25613.196102 secs 2024-04-27T01:42:12.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:11 smithi167 ceph-mon[32751]: pgmap v14261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:12.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25613.196102 secs 2024-04-27T01:42:12.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:11 smithi090 ceph-mon[29180]: pgmap v14261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:42:14.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:13 smithi119 ceph-mon[32752]: pgmap v14262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:42:14.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:13 smithi167 ceph-mon[32751]: pgmap v14262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:42:14.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:13 smithi090 ceph-mon[29180]: pgmap v14262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:42:16.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25617.378460 secs 2024-04-27T01:42:16.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25617.378460 secs 2024-04-27T01:42:16.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25617.378460 secs 2024-04-27T01:42:17.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:16 smithi119 ceph-mon[32752]: pgmap v14263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:17.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25618.196199 secs 2024-04-27T01:42:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:16 smithi167 ceph-mon[32751]: pgmap v14263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25618.196199 secs 2024-04-27T01:42:17.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:16 smithi090 ceph-mon[29180]: pgmap v14263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:17.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25618.196199 secs 2024-04-27T01:42:18.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:17 smithi119 ceph-mon[32752]: pgmap v14264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:18.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:17 smithi167 ceph-mon[32751]: pgmap v14264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:18.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:17 smithi090 ceph-mon[29180]: pgmap v14264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:20.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:19 smithi119 ceph-mon[32752]: pgmap v14265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:20.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:19 smithi167 ceph-mon[32751]: pgmap v14265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:20.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:19 smithi090 ceph-mon[29180]: pgmap v14265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:21.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25622.378548 secs 2024-04-27T01:42:21.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25622.378548 secs 2024-04-27T01:42:21.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25622.378548 secs 2024-04-27T01:42:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:22 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25623.196285 secs 2024-04-27T01:42:22.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:22 smithi119 ceph-mon[32752]: pgmap v14266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:22.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:22 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25623.196285 secs 2024-04-27T01:42:22.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:22 smithi167 ceph-mon[32751]: pgmap v14266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:22.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:22 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25623.196285 secs 2024-04-27T01:42:22.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:22 smithi090 ceph-mon[29180]: pgmap v14266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:24.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:23 smithi119 ceph-mon[32752]: pgmap v14267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:24.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:23 smithi167 ceph-mon[32751]: pgmap v14267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:24.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:23 smithi090 ceph-mon[29180]: pgmap v14267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:26.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25627.378628 secs 2024-04-27T01:42:26.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25627.378628 secs 2024-04-27T01:42:26.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25627.378628 secs 2024-04-27T01:42:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:26 smithi119 ceph-mon[32752]: pgmap v14268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:27.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25628.196427 secs 2024-04-27T01:42:27.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:26 smithi167 ceph-mon[32751]: pgmap v14268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25628.196427 secs 2024-04-27T01:42:27.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:26 smithi090 ceph-mon[29180]: pgmap v14268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:27.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25628.196427 secs 2024-04-27T01:42:28.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:27 smithi119 ceph-mon[32752]: pgmap v14269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:28.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:27 smithi167 ceph-mon[32751]: pgmap v14269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:28.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:27 smithi090 ceph-mon[29180]: pgmap v14269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:30.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:29 smithi119 ceph-mon[32752]: pgmap v14270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:30.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:29 smithi167 ceph-mon[32751]: pgmap v14270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:30.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:29 smithi090 ceph-mon[29180]: pgmap v14270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:31.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25632.378716 secs 2024-04-27T01:42:31.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25632.378716 secs 2024-04-27T01:42:31.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25632.378716 secs 2024-04-27T01:42:32.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25633.196508 secs 2024-04-27T01:42:32.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:31 smithi119 ceph-mon[32752]: pgmap v14271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:32.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25633.196508 secs 2024-04-27T01:42:32.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:32 smithi167 ceph-mon[32751]: pgmap v14271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:32.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25633.196508 secs 2024-04-27T01:42:32.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:31 smithi090 ceph-mon[29180]: pgmap v14271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:34.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:33 smithi119 ceph-mon[32752]: pgmap v14272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:33 smithi167 ceph-mon[32751]: pgmap v14272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:33 smithi090 ceph-mon[29180]: pgmap v14272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:36.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25637.378815 secs 2024-04-27T01:42:36.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25637.378815 secs 2024-04-27T01:42:36.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25637.378815 secs 2024-04-27T01:42:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:36 smithi119 ceph-mon[32752]: pgmap v14273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25638.196599 secs 2024-04-27T01:42:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:36 smithi167 ceph-mon[32751]: pgmap v14273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:37.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25638.196599 secs 2024-04-27T01:42:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:36 smithi090 ceph-mon[29180]: pgmap v14273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25638.196599 secs 2024-04-27T01:42:38.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:37 smithi119 ceph-mon[32752]: pgmap v14274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:38.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:37 smithi167 ceph-mon[32751]: pgmap v14274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:38.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:37 smithi090 ceph-mon[29180]: pgmap v14274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:40.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:39 smithi119 ceph-mon[32752]: pgmap v14275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:40.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:39 smithi167 ceph-mon[32751]: pgmap v14275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:40.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:39 smithi090 ceph-mon[29180]: pgmap v14275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:41.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25642.378903 secs 2024-04-27T01:42:41.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25642.378903 secs 2024-04-27T01:42:41.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25642.378903 secs 2024-04-27T01:42:42.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25643.196689 secs 2024-04-27T01:42:42.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:42 smithi119 ceph-mon[32752]: pgmap v14276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25643.196689 secs 2024-04-27T01:42:42.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:41 smithi167 ceph-mon[32751]: pgmap v14276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:42.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25643.196689 secs 2024-04-27T01:42:42.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:41 smithi090 ceph-mon[29180]: pgmap v14276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:44.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:44 smithi119 ceph-mon[32752]: pgmap v14277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:44.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:44 smithi167 ceph-mon[32751]: pgmap v14277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:44.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:44 smithi090 ceph-mon[29180]: pgmap v14277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:46.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25647.378987 secs 2024-04-27T01:42:46.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25647.378987 secs 2024-04-27T01:42:46.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25647.378987 secs 2024-04-27T01:42:47.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:46 smithi119 ceph-mon[32752]: pgmap v14278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:47.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25648.196806 secs 2024-04-27T01:42:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:46 smithi167 ceph-mon[32751]: pgmap v14278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25648.196806 secs 2024-04-27T01:42:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:46 smithi090 ceph-mon[29180]: pgmap v14278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25648.196806 secs 2024-04-27T01:42:48.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:47 smithi119 ceph-mon[32752]: pgmap v14279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:48.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:47 smithi167 ceph-mon[32751]: pgmap v14279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:48.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:47 smithi090 ceph-mon[29180]: pgmap v14279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:50.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:49 smithi119 ceph-mon[32752]: pgmap v14280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:50.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:49 smithi167 ceph-mon[32751]: pgmap v14280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:50.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:49 smithi090 ceph-mon[29180]: pgmap v14280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:51.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25652.379092 secs 2024-04-27T01:42:51.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25652.379092 secs 2024-04-27T01:42:51.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25652.379092 secs 2024-04-27T01:42:52.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25653.196914 secs 2024-04-27T01:42:52.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:51 smithi119 ceph-mon[32752]: pgmap v14281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25653.196914 secs 2024-04-27T01:42:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:51 smithi167 ceph-mon[32751]: pgmap v14281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:52.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25653.196914 secs 2024-04-27T01:42:52.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:51 smithi090 ceph-mon[29180]: pgmap v14281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:54.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:53 smithi119 ceph-mon[32752]: pgmap v14282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:54.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:53 smithi167 ceph-mon[32751]: pgmap v14282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:54.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:53 smithi090 ceph-mon[29180]: pgmap v14282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:42:56.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25657.379153 secs 2024-04-27T01:42:56.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25657.379153 secs 2024-04-27T01:42:56.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25657.379153 secs 2024-04-27T01:42:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:56 smithi119 ceph-mon[32752]: pgmap v14283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25658.197062 secs 2024-04-27T01:42:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:56 smithi167 ceph-mon[32751]: pgmap v14283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25658.197062 secs 2024-04-27T01:42:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:56 smithi090 ceph-mon[29180]: pgmap v14283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25658.197062 secs 2024-04-27T01:42:58.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:57 smithi090 ceph-mon[29180]: pgmap v14284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:58.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:57 smithi119 ceph-mon[32752]: pgmap v14284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:42:58.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:57 smithi167 ceph-mon[32751]: pgmap v14284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:00.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:42:59 smithi119 ceph-mon[32752]: pgmap v14285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:00.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:42:59 smithi167 ceph-mon[32751]: pgmap v14285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:00.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:42:59 smithi090 ceph-mon[29180]: pgmap v14285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25662.379215 secs 2024-04-27T01:43:01.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25662.379215 secs 2024-04-27T01:43:01.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25662.379215 secs 2024-04-27T01:43:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:02 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25663.197123 secs 2024-04-27T01:43:02.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:02 smithi119 ceph-mon[32752]: pgmap v14286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:02.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:02 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25663.197123 secs 2024-04-27T01:43:02.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:02 smithi167 ceph-mon[32751]: pgmap v14286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:02.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:02 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25663.197123 secs 2024-04-27T01:43:02.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:02 smithi090 ceph-mon[29180]: pgmap v14286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:04.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:03 smithi119 ceph-mon[32752]: pgmap v14287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:04.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:03 smithi167 ceph-mon[32751]: pgmap v14287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:04.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:03 smithi090 ceph-mon[29180]: pgmap v14287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:05.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:43:05.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:05.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:05.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:43:05.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:05.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:05.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:43:05.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:05.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:06.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25667.379301 secs 2024-04-27T01:43:06.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:43:06.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:43:06.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:06 smithi119 ceph-mon[32752]: pgmap v14288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T01:43:06.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:06 smithi119 ceph-mon[32752]: pgmap v14289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:06.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25667.379301 secs 2024-04-27T01:43:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:43:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:43:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:06 smithi167 ceph-mon[32751]: pgmap v14288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T01:43:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:06 smithi167 ceph-mon[32751]: pgmap v14289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:06.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:06.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25667.379301 secs 2024-04-27T01:43:06.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:43:06.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:43:06.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:06 smithi090 ceph-mon[29180]: pgmap v14288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T01:43:06.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:06 smithi090 ceph-mon[29180]: pgmap v14289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:06.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:43:07.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:07 smithi119 ceph-mon[32752]: 24 slow requests, 1 included below; oldest blocked for > 25668.197215 secs 2024-04-27T01:43:07.287 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:07 smithi119 ceph-mon[32752]: slow request 11521.355075 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:43:07.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:07 smithi167 ceph-mon[32751]: 24 slow requests, 1 included below; oldest blocked for > 25668.197215 secs 2024-04-27T01:43:07.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:07 smithi167 ceph-mon[32751]: slow request 11521.355075 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:43:07.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:07 smithi090 ceph-mon[29180]: 24 slow requests, 1 included below; oldest blocked for > 25668.197215 secs 2024-04-27T01:43:07.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:07 smithi090 ceph-mon[29180]: slow request 11521.355075 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:43:08.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:08 smithi119 ceph-mon[32752]: pgmap v14290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:08.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:08 smithi167 ceph-mon[32751]: pgmap v14290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:08.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:08 smithi090 ceph-mon[29180]: pgmap v14290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:10.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:10 smithi119 ceph-mon[32752]: pgmap v14291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:10.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:10 smithi167 ceph-mon[32751]: pgmap v14291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:10.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:10 smithi090 ceph-mon[29180]: pgmap v14291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:11.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25672.379361 secs 2024-04-27T01:43:11.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25673.197351 secs 2024-04-27T01:43:11.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25672.379361 secs 2024-04-27T01:43:11.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25673.197351 secs 2024-04-27T01:43:11.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25672.379361 secs 2024-04-27T01:43:11.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25673.197351 secs 2024-04-27T01:43:12.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:12 smithi119 ceph-mon[32752]: pgmap v14292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:12.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:12 smithi167 ceph-mon[32751]: pgmap v14292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:12.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:12 smithi090 ceph-mon[29180]: pgmap v14292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:43:14.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:14 smithi119 ceph-mon[32752]: pgmap v14293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T01:43:14.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:14 smithi167 ceph-mon[32751]: pgmap v14293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T01:43:14.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:14 smithi090 ceph-mon[29180]: pgmap v14293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T01:43:16.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25677.379444 secs 2024-04-27T01:43:16.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:16 smithi119 ceph-mon[32752]: pgmap v14294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:43:16.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25678.197471 secs 2024-04-27T01:43:16.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25677.379444 secs 2024-04-27T01:43:16.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:16 smithi167 ceph-mon[32751]: pgmap v14294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:43:16.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25678.197471 secs 2024-04-27T01:43:16.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25677.379444 secs 2024-04-27T01:43:16.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:16 smithi090 ceph-mon[29180]: pgmap v14294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:43:16.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25678.197471 secs 2024-04-27T01:43:18.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:18 smithi119 ceph-mon[32752]: pgmap v14295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:18.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:18 smithi167 ceph-mon[32751]: pgmap v14295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:18.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:18 smithi090 ceph-mon[29180]: pgmap v14295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:20.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:20 smithi119 ceph-mon[32752]: pgmap v14296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:20.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:20 smithi167 ceph-mon[32751]: pgmap v14296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:20.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:20 smithi090 ceph-mon[29180]: pgmap v14296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:21.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25682.379526 secs 2024-04-27T01:43:21.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25683.197597 secs 2024-04-27T01:43:21.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25682.379526 secs 2024-04-27T01:43:21.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25683.197597 secs 2024-04-27T01:43:21.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25682.379526 secs 2024-04-27T01:43:21.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25683.197597 secs 2024-04-27T01:43:22.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:22 smithi119 ceph-mon[32752]: pgmap v14297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:22.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:22 smithi167 ceph-mon[32751]: pgmap v14297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:22.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:22 smithi090 ceph-mon[29180]: pgmap v14297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:24.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:24 smithi119 ceph-mon[32752]: pgmap v14298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:24.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:24 smithi167 ceph-mon[32751]: pgmap v14298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:24.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:24 smithi090 ceph-mon[29180]: pgmap v14298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:26.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25687.379610 secs 2024-04-27T01:43:26.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:26 smithi119 ceph-mon[32752]: pgmap v14299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:26.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25688.197681 secs 2024-04-27T01:43:26.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25687.379610 secs 2024-04-27T01:43:26.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:26 smithi167 ceph-mon[32751]: pgmap v14299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:26.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25688.197681 secs 2024-04-27T01:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25687.379610 secs 2024-04-27T01:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:26 smithi090 ceph-mon[29180]: pgmap v14299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:26.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25688.197681 secs 2024-04-27T01:43:28.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:28 smithi119 ceph-mon[32752]: pgmap v14300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:28.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:28 smithi167 ceph-mon[32751]: pgmap v14300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:28.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:28 smithi090 ceph-mon[29180]: pgmap v14300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:30.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:30 smithi119 ceph-mon[32752]: pgmap v14301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:30.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:30 smithi167 ceph-mon[32751]: pgmap v14301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:30.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:30 smithi090 ceph-mon[29180]: pgmap v14301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:31.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25692.379696 secs 2024-04-27T01:43:31.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25693.197776 secs 2024-04-27T01:43:31.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25692.379696 secs 2024-04-27T01:43:31.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25693.197776 secs 2024-04-27T01:43:31.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25692.379696 secs 2024-04-27T01:43:31.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25693.197776 secs 2024-04-27T01:43:32.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:32 smithi119 ceph-mon[32752]: pgmap v14302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:32.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:32 smithi167 ceph-mon[32751]: pgmap v14302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:32.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:32 smithi090 ceph-mon[29180]: pgmap v14302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:34.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:34 smithi119 ceph-mon[32752]: pgmap v14303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:34.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:34 smithi167 ceph-mon[32751]: pgmap v14303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:34.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:34 smithi090 ceph-mon[29180]: pgmap v14303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:36.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25697.379780 secs 2024-04-27T01:43:36.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:36 smithi167 ceph-mon[32751]: pgmap v14304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:36.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25698.197849 secs 2024-04-27T01:43:36.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25697.379780 secs 2024-04-27T01:43:36.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:36 smithi090 ceph-mon[29180]: pgmap v14304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:36.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25698.197849 secs 2024-04-27T01:43:36.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25697.379780 secs 2024-04-27T01:43:36.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:36 smithi119 ceph-mon[32752]: pgmap v14304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:36.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25698.197849 secs 2024-04-27T01:43:38.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:38 smithi167 ceph-mon[32751]: pgmap v14305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:38.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:38 smithi090 ceph-mon[29180]: pgmap v14305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:38.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:38 smithi119 ceph-mon[32752]: pgmap v14305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:40.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:40 smithi167 ceph-mon[32751]: pgmap v14306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:40.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:40 smithi090 ceph-mon[29180]: pgmap v14306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:40.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:40 smithi119 ceph-mon[32752]: pgmap v14306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:41.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25702.379865 secs 2024-04-27T01:43:41.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25703.197983 secs 2024-04-27T01:43:41.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25702.379865 secs 2024-04-27T01:43:41.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25703.197983 secs 2024-04-27T01:43:41.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25702.379865 secs 2024-04-27T01:43:41.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25703.197983 secs 2024-04-27T01:43:42.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:42 smithi167 ceph-mon[32751]: pgmap v14307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:42.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:42 smithi090 ceph-mon[29180]: pgmap v14307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:42.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:42 smithi119 ceph-mon[32752]: pgmap v14307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:44.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:44 smithi090 ceph-mon[29180]: pgmap v14308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:44.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:44 smithi119 ceph-mon[32752]: pgmap v14308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:44.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:44 smithi167 ceph-mon[32751]: pgmap v14308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:46.588 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25707.379950 secs 2024-04-27T01:43:46.588 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:46 smithi119 ceph-mon[32752]: pgmap v14309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:46.589 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25708.198128 secs 2024-04-27T01:43:46.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25707.379950 secs 2024-04-27T01:43:46.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:46 smithi090 ceph-mon[29180]: pgmap v14309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:46.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25708.198128 secs 2024-04-27T01:43:46.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25707.379950 secs 2024-04-27T01:43:46.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:46 smithi167 ceph-mon[32751]: pgmap v14309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:46.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25708.198128 secs 2024-04-27T01:43:48.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:48 smithi090 ceph-mon[29180]: pgmap v14310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:48.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:48 smithi119 ceph-mon[32752]: pgmap v14310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:48.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:48 smithi167 ceph-mon[32751]: pgmap v14310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:50.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:50 smithi090 ceph-mon[29180]: pgmap v14311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:50.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:50 smithi119 ceph-mon[32752]: pgmap v14311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:50.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:50 smithi167 ceph-mon[32751]: pgmap v14311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:51.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25712.380034 secs 2024-04-27T01:43:51.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25713.198242 secs 2024-04-27T01:43:51.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25712.380034 secs 2024-04-27T01:43:51.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25713.198242 secs 2024-04-27T01:43:51.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25712.380034 secs 2024-04-27T01:43:51.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25713.198242 secs 2024-04-27T01:43:52.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:52 smithi090 ceph-mon[29180]: pgmap v14312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:52.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:52 smithi119 ceph-mon[32752]: pgmap v14312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:52.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:52 smithi167 ceph-mon[32751]: pgmap v14312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:54.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:54 smithi119 ceph-mon[32752]: pgmap v14313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:54.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:54 smithi167 ceph-mon[32751]: pgmap v14313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:54.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:54 smithi090 ceph-mon[29180]: pgmap v14313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:56.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25717.380094 secs 2024-04-27T01:43:56.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:56 smithi119 ceph-mon[32752]: pgmap v14314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:56.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25718.198336 secs 2024-04-27T01:43:56.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25717.380094 secs 2024-04-27T01:43:56.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:56 smithi167 ceph-mon[32751]: pgmap v14314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:56.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25718.198336 secs 2024-04-27T01:43:56.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25717.380094 secs 2024-04-27T01:43:56.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:56 smithi090 ceph-mon[29180]: pgmap v14314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:43:56.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25718.198336 secs 2024-04-27T01:43:58.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:43:58 smithi119 ceph-mon[32752]: pgmap v14315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:58.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:43:58 smithi167 ceph-mon[32751]: pgmap v14315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:43:58.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:43:58 smithi090 ceph-mon[29180]: pgmap v14315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:00.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:00 smithi119 ceph-mon[32752]: pgmap v14316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:00.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:00 smithi167 ceph-mon[32751]: pgmap v14316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:00.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:00 smithi090 ceph-mon[29180]: pgmap v14316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:01.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25722.380153 secs 2024-04-27T01:44:01.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25723.198395 secs 2024-04-27T01:44:01.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25722.380153 secs 2024-04-27T01:44:01.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25723.198395 secs 2024-04-27T01:44:01.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25722.380153 secs 2024-04-27T01:44:01.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25723.198395 secs 2024-04-27T01:44:02.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:02 smithi119 ceph-mon[32752]: pgmap v14317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:02.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:02 smithi167 ceph-mon[32751]: pgmap v14317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:02.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:02 smithi090 ceph-mon[29180]: pgmap v14317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:04.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:04 smithi119 ceph-mon[32752]: pgmap v14318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:04.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:04 smithi167 ceph-mon[32751]: pgmap v14318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:04.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:04 smithi090 ceph-mon[29180]: pgmap v14318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:05.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:44:05.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:44:05.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:44:06.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25727.380236 secs 2024-04-27T01:44:06.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:06 smithi119 ceph-mon[32752]: pgmap v14319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:06.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25728.198438 secs 2024-04-27T01:44:06.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:44:06.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:44:06.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:44:06.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25727.380236 secs 2024-04-27T01:44:06.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:06 smithi167 ceph-mon[32751]: pgmap v14319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:06.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25728.198438 secs 2024-04-27T01:44:06.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:44:06.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:44:06.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:44:06.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25727.380236 secs 2024-04-27T01:44:06.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:06 smithi090 ceph-mon[29180]: pgmap v14319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:06.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25728.198438 secs 2024-04-27T01:44:06.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:44:06.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:44:06.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:44:07.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:07 smithi119 ceph-mon[32752]: pgmap v14320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:44:07.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:07 smithi119 ceph-mon[32752]: pgmap v14321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:07.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:07 smithi167 ceph-mon[32751]: pgmap v14320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:44:07.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:07 smithi167 ceph-mon[32751]: pgmap v14321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:07.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:07 smithi090 ceph-mon[29180]: pgmap v14320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:44:07.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:07 smithi090 ceph-mon[29180]: pgmap v14321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:09.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:09 smithi119 ceph-mon[32752]: pgmap v14322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:44:09.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:09 smithi167 ceph-mon[32751]: pgmap v14322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:44:09.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:09 smithi090 ceph-mon[29180]: pgmap v14322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:44:11.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:11 smithi119 ceph-mon[32752]: pgmap v14323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:11.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25732.380318 secs 2024-04-27T01:44:11.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25733.198546 secs 2024-04-27T01:44:11.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:11 smithi167 ceph-mon[32751]: pgmap v14323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:11.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25732.380318 secs 2024-04-27T01:44:11.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25733.198546 secs 2024-04-27T01:44:11.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:11 smithi090 ceph-mon[29180]: pgmap v14323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:11.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25732.380318 secs 2024-04-27T01:44:11.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25733.198546 secs 2024-04-27T01:44:13.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:13 smithi119 ceph-mon[32752]: pgmap v14324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:13.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:13 smithi167 ceph-mon[32751]: pgmap v14324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:13.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:13 smithi090 ceph-mon[29180]: pgmap v14324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:15.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:15 smithi119 ceph-mon[32752]: pgmap v14325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:15.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:15 smithi167 ceph-mon[32751]: pgmap v14325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:15.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:15 smithi090 ceph-mon[29180]: pgmap v14325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:44:16.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25737.380398 secs 2024-04-27T01:44:16.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25738.198689 secs 2024-04-27T01:44:16.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25737.380398 secs 2024-04-27T01:44:16.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25738.198689 secs 2024-04-27T01:44:16.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25737.380398 secs 2024-04-27T01:44:16.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25738.198689 secs 2024-04-27T01:44:17.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:17 smithi119 ceph-mon[32752]: pgmap v14326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:44:17.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:17 smithi167 ceph-mon[32751]: pgmap v14326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:44:17.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:17 smithi090 ceph-mon[29180]: pgmap v14326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:44:19.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:19 smithi119 ceph-mon[32752]: pgmap v14327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:19.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:19 smithi167 ceph-mon[32751]: pgmap v14327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:19.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:19 smithi090 ceph-mon[29180]: pgmap v14327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:21.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:21 smithi119 ceph-mon[32752]: pgmap v14328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:21.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25742.380455 secs 2024-04-27T01:44:21.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25743.198754 secs 2024-04-27T01:44:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:21 smithi167 ceph-mon[32751]: pgmap v14328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25742.380455 secs 2024-04-27T01:44:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25743.198754 secs 2024-04-27T01:44:21.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:21 smithi090 ceph-mon[29180]: pgmap v14328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25742.380455 secs 2024-04-27T01:44:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25743.198754 secs 2024-04-27T01:44:23.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:23 smithi119 ceph-mon[32752]: pgmap v14329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:23.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:23 smithi167 ceph-mon[32751]: pgmap v14329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:23.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:23 smithi090 ceph-mon[29180]: pgmap v14329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:25.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:25 smithi167 ceph-mon[32751]: pgmap v14330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:25.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:25 smithi090 ceph-mon[29180]: pgmap v14330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:26.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:25 smithi119 ceph-mon[32752]: pgmap v14330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:26.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25747.380534 secs 2024-04-27T01:44:26.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25748.198891 secs 2024-04-27T01:44:26.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25747.380534 secs 2024-04-27T01:44:26.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25748.198891 secs 2024-04-27T01:44:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25747.380534 secs 2024-04-27T01:44:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25748.198891 secs 2024-04-27T01:44:27.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:27 smithi167 ceph-mon[32751]: pgmap v14331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:27.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:27 smithi090 ceph-mon[29180]: pgmap v14331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:28.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:27 smithi119 ceph-mon[32752]: pgmap v14331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:29.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:29 smithi167 ceph-mon[32751]: pgmap v14332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:29.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:29 smithi090 ceph-mon[29180]: pgmap v14332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:30.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:29 smithi119 ceph-mon[32752]: pgmap v14332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:31.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:31 smithi167 ceph-mon[32751]: pgmap v14333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:31.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25752.380609 secs 2024-04-27T01:44:31.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25753.199001 secs 2024-04-27T01:44:31.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:31 smithi090 ceph-mon[29180]: pgmap v14333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:31.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25752.380609 secs 2024-04-27T01:44:31.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25753.199001 secs 2024-04-27T01:44:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:31 smithi119 ceph-mon[32752]: pgmap v14333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25752.380609 secs 2024-04-27T01:44:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25753.199001 secs 2024-04-27T01:44:33.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:33 smithi090 ceph-mon[29180]: pgmap v14334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:34.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:33 smithi119 ceph-mon[32752]: pgmap v14334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:34.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:33 smithi167 ceph-mon[32751]: pgmap v14334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:35.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:35 smithi090 ceph-mon[29180]: pgmap v14335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:36.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:35 smithi119 ceph-mon[32752]: pgmap v14335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:36.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:35 smithi167 ceph-mon[32751]: pgmap v14335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:36.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25757.380691 secs 2024-04-27T01:44:36.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25758.199128 secs 2024-04-27T01:44:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25757.380691 secs 2024-04-27T01:44:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25758.199128 secs 2024-04-27T01:44:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25757.380691 secs 2024-04-27T01:44:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25758.199128 secs 2024-04-27T01:44:37.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:37 smithi090 ceph-mon[29180]: pgmap v14336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:38.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:37 smithi119 ceph-mon[32752]: pgmap v14336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:38.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:37 smithi167 ceph-mon[32751]: pgmap v14336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:39.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:39 smithi090 ceph-mon[29180]: pgmap v14337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:40.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:39 smithi119 ceph-mon[32752]: pgmap v14337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:40.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:39 smithi167 ceph-mon[32751]: pgmap v14337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:41.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:41 smithi090 ceph-mon[29180]: pgmap v14338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:41.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25762.380774 secs 2024-04-27T01:44:41.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25763.199248 secs 2024-04-27T01:44:42.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:41 smithi119 ceph-mon[32752]: pgmap v14338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:42.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25762.380774 secs 2024-04-27T01:44:42.076 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25763.199248 secs 2024-04-27T01:44:42.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:41 smithi167 ceph-mon[32751]: pgmap v14338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:42.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25762.380774 secs 2024-04-27T01:44:42.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25763.199248 secs 2024-04-27T01:44:43.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:43 smithi090 ceph-mon[29180]: pgmap v14339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:44.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:43 smithi119 ceph-mon[32752]: pgmap v14339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:44.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:43 smithi167 ceph-mon[32751]: pgmap v14339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:45.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:45 smithi090 ceph-mon[29180]: pgmap v14340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:46.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:45 smithi119 ceph-mon[32752]: pgmap v14340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:46.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:45 smithi167 ceph-mon[32751]: pgmap v14340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:46.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25767.380842 secs 2024-04-27T01:44:46.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25768.199374 secs 2024-04-27T01:44:46.937 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25767.380842 secs 2024-04-27T01:44:46.937 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25768.199374 secs 2024-04-27T01:44:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25767.380842 secs 2024-04-27T01:44:47.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25768.199374 secs 2024-04-27T01:44:47.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:47 smithi090 ceph-mon[29180]: pgmap v14341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:48.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:47 smithi119 ceph-mon[32752]: pgmap v14341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:48.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:47 smithi167 ceph-mon[32751]: pgmap v14341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:49.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:49 smithi090 ceph-mon[29180]: pgmap v14342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:50.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:49 smithi119 ceph-mon[32752]: pgmap v14342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:50.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:49 smithi167 ceph-mon[32751]: pgmap v14342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:44:51.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:51 smithi090 ceph-mon[29180]: pgmap v14343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:51.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25772.380926 secs 2024-04-27T01:44:51.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25773.199518 secs 2024-04-27T01:44:52.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:51 smithi119 ceph-mon[32752]: pgmap v14343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:52.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25772.380926 secs 2024-04-27T01:44:52.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25773.199518 secs 2024-04-27T01:44:52.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:51 smithi167 ceph-mon[32751]: pgmap v14343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:52.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25772.380926 secs 2024-04-27T01:44:52.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25773.199518 secs 2024-04-27T01:44:53.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:53 smithi090 ceph-mon[29180]: pgmap v14344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:44:54.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:53 smithi119 ceph-mon[32752]: pgmap v14344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:44:54.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:53 smithi167 ceph-mon[32751]: pgmap v14344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:44:55.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:55 smithi090 ceph-mon[29180]: pgmap v14345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:56.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:55 smithi119 ceph-mon[32752]: pgmap v14345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:56.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:55 smithi167 ceph-mon[32751]: pgmap v14345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:44:56.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25777.381006 secs 2024-04-27T01:44:56.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25778.199645 secs 2024-04-27T01:44:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25777.381006 secs 2024-04-27T01:44:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25778.199645 secs 2024-04-27T01:44:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25777.381006 secs 2024-04-27T01:44:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25778.199645 secs 2024-04-27T01:44:57.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:57 smithi090 ceph-mon[29180]: pgmap v14346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:44:58.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:57 smithi119 ceph-mon[32752]: pgmap v14346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:44:58.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:57 smithi167 ceph-mon[32751]: pgmap v14346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:45:00.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:44:59 smithi119 ceph-mon[32752]: pgmap v14347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:45:00.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:44:59 smithi167 ceph-mon[32751]: pgmap v14347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:45:00.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:44:59 smithi090 ceph-mon[29180]: pgmap v14347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:45:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:01 smithi119 ceph-mon[32752]: pgmap v14348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25782.381111 secs 2024-04-27T01:45:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25783.199778 secs 2024-04-27T01:45:02.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:01 smithi167 ceph-mon[32751]: pgmap v14348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:02.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25782.381111 secs 2024-04-27T01:45:02.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25783.199778 secs 2024-04-27T01:45:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:01 smithi090 ceph-mon[29180]: pgmap v14348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25782.381111 secs 2024-04-27T01:45:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25783.199778 secs 2024-04-27T01:45:04.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:03 smithi119 ceph-mon[32752]: pgmap v14349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:03 smithi167 ceph-mon[32751]: pgmap v14349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:04.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:03 smithi090 ceph-mon[29180]: pgmap v14349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:06.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:05 smithi119 ceph-mon[32752]: pgmap v14350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:06.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:05 smithi167 ceph-mon[32751]: pgmap v14350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:06.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:05 smithi090 ceph-mon[29180]: pgmap v14350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25787.381189 secs 2024-04-27T01:45:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25788.199889 secs 2024-04-27T01:45:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:45:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25787.381189 secs 2024-04-27T01:45:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25788.199889 secs 2024-04-27T01:45:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:45:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25787.381189 secs 2024-04-27T01:45:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25788.199889 secs 2024-04-27T01:45:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:45:08.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:07 smithi119 ceph-mon[32752]: pgmap v14351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:08.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:45:08.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:45:08.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:07 smithi119 ceph-mon[32752]: pgmap v14352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:45:08.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:07 smithi119 ceph-mon[32752]: pgmap v14353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:08.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:45:08.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:07 smithi167 ceph-mon[32751]: pgmap v14351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:08.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:45:08.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:45:08.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:07 smithi167 ceph-mon[32751]: pgmap v14352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:45:08.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:07 smithi167 ceph-mon[32751]: pgmap v14353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:08.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:45:08.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:07 smithi090 ceph-mon[29180]: pgmap v14351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:08.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:45:08.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:45:08.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:07 smithi090 ceph-mon[29180]: pgmap v14352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:45:08.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:07 smithi090 ceph-mon[29180]: pgmap v14353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:08.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:45:10.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:09 smithi167 ceph-mon[32751]: pgmap v14354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:10.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:09 smithi090 ceph-mon[29180]: pgmap v14354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:10.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:09 smithi119 ceph-mon[32752]: pgmap v14354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:11.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25792.381251 secs 2024-04-27T01:45:11.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25792.381251 secs 2024-04-27T01:45:11.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25792.381251 secs 2024-04-27T01:45:12.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:11 smithi090 ceph-mon[29180]: pgmap v14355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:12.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25793.199971 secs 2024-04-27T01:45:12.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:11 smithi119 ceph-mon[32752]: pgmap v14355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:12.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25793.199971 secs 2024-04-27T01:45:12.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:11 smithi167 ceph-mon[32751]: pgmap v14355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:12.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25793.199971 secs 2024-04-27T01:45:14.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:13 smithi090 ceph-mon[29180]: pgmap v14356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:45:14.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:13 smithi119 ceph-mon[32752]: pgmap v14356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:45:14.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:13 smithi167 ceph-mon[32751]: pgmap v14356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:45:16.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:15 smithi119 ceph-mon[32752]: pgmap v14357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:16.089 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:15 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25797.381333 secs 2024-04-27T01:45:16.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:15 smithi090 ceph-mon[29180]: pgmap v14357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:16.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:15 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25797.381333 secs 2024-04-27T01:45:16.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:15 smithi167 ceph-mon[32751]: pgmap v14357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:45:16.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:15 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25797.381333 secs 2024-04-27T01:45:17.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25798.200098 secs 2024-04-27T01:45:17.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25798.200098 secs 2024-04-27T01:45:17.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25798.200098 secs 2024-04-27T01:45:18.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:17 smithi090 ceph-mon[29180]: pgmap v14358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:45:18.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:17 smithi119 ceph-mon[32752]: pgmap v14358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:45:18.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:17 smithi167 ceph-mon[32751]: pgmap v14358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:45:20.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:19 smithi090 ceph-mon[29180]: pgmap v14359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:20.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:19 smithi119 ceph-mon[32752]: pgmap v14359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:20.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:19 smithi167 ceph-mon[32751]: pgmap v14359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:21.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:20 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25802.381417 secs 2024-04-27T01:45:21.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:20 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25802.381417 secs 2024-04-27T01:45:21.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:20 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25802.381417 secs 2024-04-27T01:45:22.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:21 smithi090 ceph-mon[29180]: pgmap v14360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:22.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25803.200234 secs 2024-04-27T01:45:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:21 smithi119 ceph-mon[32752]: pgmap v14360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25803.200234 secs 2024-04-27T01:45:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:21 smithi167 ceph-mon[32751]: pgmap v14360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25803.200234 secs 2024-04-27T01:45:24.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:23 smithi090 ceph-mon[29180]: pgmap v14361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:24.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:23 smithi119 ceph-mon[32752]: pgmap v14361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:24.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:23 smithi167 ceph-mon[32751]: pgmap v14361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:26.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:25 smithi090 ceph-mon[29180]: pgmap v14362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:26.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:25 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25807.381475 secs 2024-04-27T01:45:26.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:25 smithi119 ceph-mon[32752]: pgmap v14362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:26.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:25 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25807.381475 secs 2024-04-27T01:45:26.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:25 smithi167 ceph-mon[32751]: pgmap v14362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:26.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:25 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25807.381475 secs 2024-04-27T01:45:27.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25808.200330 secs 2024-04-27T01:45:27.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25808.200330 secs 2024-04-27T01:45:27.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25808.200330 secs 2024-04-27T01:45:28.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:27 smithi090 ceph-mon[29180]: pgmap v14363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:28.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:27 smithi119 ceph-mon[32752]: pgmap v14363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:28.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:27 smithi167 ceph-mon[32751]: pgmap v14363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:30.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:29 smithi090 ceph-mon[29180]: pgmap v14364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:30.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:29 smithi119 ceph-mon[32752]: pgmap v14364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:30.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:29 smithi167 ceph-mon[32751]: pgmap v14364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:31.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:30 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25812.381559 secs 2024-04-27T01:45:31.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:30 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25812.381559 secs 2024-04-27T01:45:31.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:30 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25812.381559 secs 2024-04-27T01:45:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:31 smithi090 ceph-mon[29180]: pgmap v14365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25813.200411 secs 2024-04-27T01:45:32.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:31 smithi119 ceph-mon[32752]: pgmap v14365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:32.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25813.200411 secs 2024-04-27T01:45:32.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:31 smithi167 ceph-mon[32751]: pgmap v14365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:32.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25813.200411 secs 2024-04-27T01:45:34.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:33 smithi119 ceph-mon[32752]: pgmap v14366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:34.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:33 smithi167 ceph-mon[32751]: pgmap v14366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:33 smithi090 ceph-mon[29180]: pgmap v14366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:35 smithi119 ceph-mon[32752]: pgmap v14367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:35 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25817.381615 secs 2024-04-27T01:45:36.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:35 smithi167 ceph-mon[32751]: pgmap v14367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:36.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:35 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25817.381615 secs 2024-04-27T01:45:36.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:35 smithi090 ceph-mon[29180]: pgmap v14367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:36.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:35 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25817.381615 secs 2024-04-27T01:45:37.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25818.200477 secs 2024-04-27T01:45:37.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25818.200477 secs 2024-04-27T01:45:37.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25818.200477 secs 2024-04-27T01:45:38.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:37 smithi119 ceph-mon[32752]: pgmap v14368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:38.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:37 smithi167 ceph-mon[32751]: pgmap v14368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:38.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:37 smithi090 ceph-mon[29180]: pgmap v14368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:40.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:39 smithi119 ceph-mon[32752]: pgmap v14369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:40.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:39 smithi167 ceph-mon[32751]: pgmap v14369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:40.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:39 smithi090 ceph-mon[29180]: pgmap v14369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:41.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:40 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25822.381673 secs 2024-04-27T01:45:41.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:40 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25822.381673 secs 2024-04-27T01:45:41.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:40 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25822.381673 secs 2024-04-27T01:45:42.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:41 smithi119 ceph-mon[32752]: pgmap v14370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:42.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25823.200603 secs 2024-04-27T01:45:42.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:41 smithi167 ceph-mon[32751]: pgmap v14370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:42.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25823.200603 secs 2024-04-27T01:45:42.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:41 smithi090 ceph-mon[29180]: pgmap v14370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:42.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25823.200603 secs 2024-04-27T01:45:44.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:43 smithi119 ceph-mon[32752]: pgmap v14371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:44.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:43 smithi167 ceph-mon[32751]: pgmap v14371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:44.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:43 smithi090 ceph-mon[29180]: pgmap v14371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:46.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:45 smithi119 ceph-mon[32752]: pgmap v14372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:46.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:45 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25827.381730 secs 2024-04-27T01:45:46.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:45 smithi167 ceph-mon[32751]: pgmap v14372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:46.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:45 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25827.381730 secs 2024-04-27T01:45:46.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:45 smithi090 ceph-mon[29180]: pgmap v14372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:46.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:45 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25827.381730 secs 2024-04-27T01:45:47.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25828.200686 secs 2024-04-27T01:45:47.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25828.200686 secs 2024-04-27T01:45:47.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25828.200686 secs 2024-04-27T01:45:48.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:47 smithi119 ceph-mon[32752]: pgmap v14373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:48.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:47 smithi167 ceph-mon[32751]: pgmap v14373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:48.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:47 smithi090 ceph-mon[29180]: pgmap v14373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:50.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:49 smithi119 ceph-mon[32752]: pgmap v14374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:50.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:49 smithi167 ceph-mon[32751]: pgmap v14374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:50.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:49 smithi090 ceph-mon[29180]: pgmap v14374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:51.259 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:50 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25832.381809 secs 2024-04-27T01:45:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:50 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25832.381809 secs 2024-04-27T01:45:51.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:50 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25832.381809 secs 2024-04-27T01:45:52.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:51 smithi119 ceph-mon[32752]: pgmap v14375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:52.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25833.200766 secs 2024-04-27T01:45:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:51 smithi167 ceph-mon[32751]: pgmap v14375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:52.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25833.200766 secs 2024-04-27T01:45:52.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:51 smithi090 ceph-mon[29180]: pgmap v14375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:52.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25833.200766 secs 2024-04-27T01:45:54.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:53 smithi119 ceph-mon[32752]: pgmap v14376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:54.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:53 smithi167 ceph-mon[32751]: pgmap v14376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:54.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:53 smithi090 ceph-mon[29180]: pgmap v14376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:56.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:55 smithi119 ceph-mon[32752]: pgmap v14377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:56.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:55 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25837.381881 secs 2024-04-27T01:45:56.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:55 smithi167 ceph-mon[32751]: pgmap v14377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:56.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:55 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25837.381881 secs 2024-04-27T01:45:56.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:55 smithi090 ceph-mon[29180]: pgmap v14377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:45:56.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:55 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25837.381881 secs 2024-04-27T01:45:57.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25838.200877 secs 2024-04-27T01:45:57.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25838.200877 secs 2024-04-27T01:45:57.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25838.200877 secs 2024-04-27T01:45:58.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:57 smithi119 ceph-mon[32752]: pgmap v14378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:58.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:57 smithi167 ceph-mon[32751]: pgmap v14378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:45:58.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:57 smithi090 ceph-mon[29180]: pgmap v14378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:00.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:45:59 smithi119 ceph-mon[32752]: pgmap v14379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:00.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:45:59 smithi167 ceph-mon[32751]: pgmap v14379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:00.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:45:59 smithi090 ceph-mon[29180]: pgmap v14379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:01.202 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:00 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25842.381962 secs 2024-04-27T01:46:01.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:00 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25842.381962 secs 2024-04-27T01:46:01.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:00 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25842.381962 secs 2024-04-27T01:46:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:01 smithi119 ceph-mon[32752]: pgmap v14380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25843.200948 secs 2024-04-27T01:46:02.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:01 smithi167 ceph-mon[32751]: pgmap v14380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:02.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25843.200948 secs 2024-04-27T01:46:02.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:01 smithi090 ceph-mon[29180]: pgmap v14380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:02.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25843.200948 secs 2024-04-27T01:46:04.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:03 smithi119 ceph-mon[32752]: pgmap v14381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:04.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:03 smithi167 ceph-mon[32751]: pgmap v14381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:04.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:03 smithi090 ceph-mon[29180]: pgmap v14381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:06.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:05 smithi119 ceph-mon[32752]: pgmap v14382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:06.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:05 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25847.382044 secs 2024-04-27T01:46:06.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:05 smithi167 ceph-mon[32751]: pgmap v14382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:06.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:05 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25847.382044 secs 2024-04-27T01:46:06.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:05 smithi090 ceph-mon[29180]: pgmap v14382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:06.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:05 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25847.382044 secs 2024-04-27T01:46:07.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25848.201011 secs 2024-04-27T01:46:07.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:46:07.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25848.201011 secs 2024-04-27T01:46:07.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:46:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25848.201011 secs 2024-04-27T01:46:07.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:46:08.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:07 smithi119 ceph-mon[32752]: pgmap v14383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:08.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:46:08.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:46:08.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:46:08.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:07 smithi167 ceph-mon[32751]: pgmap v14383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:08.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:46:08.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:46:08.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:46:08.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:07 smithi090 ceph-mon[29180]: pgmap v14383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:08.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:46:08.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:46:08.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:46:09.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:08 smithi119 ceph-mon[32752]: pgmap v14384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:46:09.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:08 smithi119 ceph-mon[32752]: pgmap v14385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:46:09.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:08 smithi167 ceph-mon[32751]: pgmap v14384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:46:09.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:08 smithi167 ceph-mon[32751]: pgmap v14385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:46:09.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:08 smithi090 ceph-mon[29180]: pgmap v14384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:46:09.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:08 smithi090 ceph-mon[29180]: pgmap v14385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:46:11.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:10 smithi119 ceph-mon[32752]: pgmap v14386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:11.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:10 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25852.382126 secs 2024-04-27T01:46:11.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:10 smithi167 ceph-mon[32751]: pgmap v14386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:11.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:10 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25852.382126 secs 2024-04-27T01:46:11.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:10 smithi090 ceph-mon[29180]: pgmap v14386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:11.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:10 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25852.382126 secs 2024-04-27T01:46:12.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:11 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25853.201131 secs 2024-04-27T01:46:12.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:11 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25853.201131 secs 2024-04-27T01:46:12.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:11 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25853.201131 secs 2024-04-27T01:46:13.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:12 smithi119 ceph-mon[32752]: pgmap v14387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:13.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:12 smithi167 ceph-mon[32751]: pgmap v14387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:13.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:12 smithi090 ceph-mon[29180]: pgmap v14387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:14.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:13 smithi119 ceph-mon[32752]: pgmap v14388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:14.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:13 smithi167 ceph-mon[32751]: pgmap v14388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:14.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:13 smithi090 ceph-mon[29180]: pgmap v14388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:16.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25857.382208 secs 2024-04-27T01:46:16.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:16 smithi090 ceph-mon[29180]: pgmap v14389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:16.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:16 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25858.201248 secs 2024-04-27T01:46:16.988 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25857.382208 secs 2024-04-27T01:46:16.989 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:16 smithi119 ceph-mon[32752]: pgmap v14389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:16.989 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:16 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25858.201248 secs 2024-04-27T01:46:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25857.382208 secs 2024-04-27T01:46:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:16 smithi167 ceph-mon[32751]: pgmap v14389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:46:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:16 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25858.201248 secs 2024-04-27T01:46:18.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:18 smithi090 ceph-mon[29180]: pgmap v14390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:46:19.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:18 smithi119 ceph-mon[32752]: pgmap v14390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:46:19.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:18 smithi167 ceph-mon[32751]: pgmap v14390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:46:20.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:20 smithi090 ceph-mon[29180]: pgmap v14391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:21.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:20 smithi119 ceph-mon[32752]: pgmap v14391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:21.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:20 smithi167 ceph-mon[32751]: pgmap v14391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:21.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25862.382273 secs 2024-04-27T01:46:21.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:21 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25863.201441 secs 2024-04-27T01:46:22.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25862.382273 secs 2024-04-27T01:46:22.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:21 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25863.201441 secs 2024-04-27T01:46:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25862.382273 secs 2024-04-27T01:46:22.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:21 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25863.201441 secs 2024-04-27T01:46:22.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:22 smithi090 ceph-mon[29180]: pgmap v14392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:23.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:22 smithi119 ceph-mon[32752]: pgmap v14392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:23.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:22 smithi167 ceph-mon[32751]: pgmap v14392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:24.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:24 smithi090 ceph-mon[29180]: pgmap v14393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:25.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:24 smithi119 ceph-mon[32752]: pgmap v14393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:25.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:24 smithi167 ceph-mon[32751]: pgmap v14393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:26.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25867.382328 secs 2024-04-27T01:46:26.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:26 smithi090 ceph-mon[29180]: pgmap v14394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:26.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:26 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25868.201569 secs 2024-04-27T01:46:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25867.382328 secs 2024-04-27T01:46:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:26 smithi119 ceph-mon[32752]: pgmap v14394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:26 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25868.201569 secs 2024-04-27T01:46:27.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25867.382328 secs 2024-04-27T01:46:27.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:26 smithi167 ceph-mon[32751]: pgmap v14394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:27.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:26 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25868.201569 secs 2024-04-27T01:46:28.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:28 smithi090 ceph-mon[29180]: pgmap v14395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:29.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:28 smithi119 ceph-mon[32752]: pgmap v14395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:29.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:28 smithi167 ceph-mon[32751]: pgmap v14395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:31.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:30 smithi119 ceph-mon[32752]: pgmap v14396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:31.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:30 smithi167 ceph-mon[32751]: pgmap v14396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:31.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:30 smithi090 ceph-mon[29180]: pgmap v14396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:31.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25872.382386 secs 2024-04-27T01:46:31.952 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:31 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25873.201676 secs 2024-04-27T01:46:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25872.382386 secs 2024-04-27T01:46:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:31 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25873.201676 secs 2024-04-27T01:46:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25872.382386 secs 2024-04-27T01:46:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:31 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25873.201676 secs 2024-04-27T01:46:33.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:32 smithi119 ceph-mon[32752]: pgmap v14397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:33.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:32 smithi167 ceph-mon[32751]: pgmap v14397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:33.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:32 smithi090 ceph-mon[29180]: pgmap v14397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:35.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:34 smithi119 ceph-mon[32752]: pgmap v14398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:35.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:34 smithi167 ceph-mon[32751]: pgmap v14398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:35.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:34 smithi090 ceph-mon[29180]: pgmap v14398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:46:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25877.382442 secs 2024-04-27T01:46:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:36 smithi119 ceph-mon[32752]: pgmap v14399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:36 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25878.201802 secs 2024-04-27T01:46:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25877.382442 secs 2024-04-27T01:46:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:36 smithi167 ceph-mon[32751]: pgmap v14399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:36 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25878.201802 secs 2024-04-27T01:46:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25877.382442 secs 2024-04-27T01:46:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:36 smithi090 ceph-mon[29180]: pgmap v14399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:36 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25878.201802 secs 2024-04-27T01:46:39.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:38 smithi119 ceph-mon[32752]: pgmap v14400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:39.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:38 smithi167 ceph-mon[32751]: pgmap v14400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:39.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:38 smithi090 ceph-mon[29180]: pgmap v14400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:41.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:40 smithi119 ceph-mon[32752]: pgmap v14401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:41.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:40 smithi167 ceph-mon[32751]: pgmap v14401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:41.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:40 smithi090 ceph-mon[29180]: pgmap v14401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:42.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25882.382498 secs 2024-04-27T01:46:42.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:41 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25883.201864 secs 2024-04-27T01:46:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25882.382498 secs 2024-04-27T01:46:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:41 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25883.201864 secs 2024-04-27T01:46:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25882.382498 secs 2024-04-27T01:46:42.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:41 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25883.201864 secs 2024-04-27T01:46:43.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:42 smithi119 ceph-mon[32752]: pgmap v14402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:43.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:42 smithi167 ceph-mon[32751]: pgmap v14402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:43.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:42 smithi090 ceph-mon[29180]: pgmap v14402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:45.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:44 smithi119 ceph-mon[32752]: pgmap v14403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:45.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:44 smithi167 ceph-mon[32751]: pgmap v14403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:45.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:44 smithi090 ceph-mon[29180]: pgmap v14403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:47.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25887.382556 secs 2024-04-27T01:46:47.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:46 smithi119 ceph-mon[32752]: pgmap v14404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:47.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:46 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25888.202001 secs 2024-04-27T01:46:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25887.382556 secs 2024-04-27T01:46:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:46 smithi167 ceph-mon[32751]: pgmap v14404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:46 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25888.202001 secs 2024-04-27T01:46:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25887.382556 secs 2024-04-27T01:46:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:46 smithi090 ceph-mon[29180]: pgmap v14404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:46 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25888.202001 secs 2024-04-27T01:46:49.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:48 smithi119 ceph-mon[32752]: pgmap v14405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:49.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:48 smithi167 ceph-mon[32751]: pgmap v14405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:49.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:48 smithi090 ceph-mon[29180]: pgmap v14405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T01:46:51.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:50 smithi119 ceph-mon[32752]: pgmap v14406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:51.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:50 smithi167 ceph-mon[32751]: pgmap v14406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:51.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:50 smithi090 ceph-mon[29180]: pgmap v14406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T01:46:52.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25892.382615 secs 2024-04-27T01:46:52.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:51 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25893.202114 secs 2024-04-27T01:46:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25892.382615 secs 2024-04-27T01:46:52.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:51 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25893.202114 secs 2024-04-27T01:46:52.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25892.382615 secs 2024-04-27T01:46:52.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:51 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25893.202114 secs 2024-04-27T01:46:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:52 smithi119 ceph-mon[32752]: pgmap v14407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:46:53.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:52 smithi167 ceph-mon[32751]: pgmap v14407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:46:53.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:52 smithi090 ceph-mon[29180]: pgmap v14407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:46:55.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:54 smithi119 ceph-mon[32752]: pgmap v14408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:46:55.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:54 smithi167 ceph-mon[32751]: pgmap v14408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:46:55.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:54 smithi090 ceph-mon[29180]: pgmap v14408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:46:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25897.382662 secs 2024-04-27T01:46:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:56 smithi119 ceph-mon[32752]: pgmap v14409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:56 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25898.202233 secs 2024-04-27T01:46:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25897.382662 secs 2024-04-27T01:46:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:56 smithi167 ceph-mon[32751]: pgmap v14409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:57.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:56 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25898.202233 secs 2024-04-27T01:46:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25897.382662 secs 2024-04-27T01:46:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:56 smithi090 ceph-mon[29180]: pgmap v14409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:56 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25898.202233 secs 2024-04-27T01:46:59.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:46:58 smithi119 ceph-mon[32752]: pgmap v14410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:59.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:46:58 smithi167 ceph-mon[32751]: pgmap v14410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:46:59.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:46:58 smithi090 ceph-mon[29180]: pgmap v14410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:01.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:00 smithi119 ceph-mon[32752]: pgmap v14411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:01.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:00 smithi167 ceph-mon[32751]: pgmap v14411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:00 smithi090 ceph-mon[29180]: pgmap v14411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:01.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25902.382720 secs 2024-04-27T01:47:01.952 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:01 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25903.202373 secs 2024-04-27T01:47:02.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25902.382720 secs 2024-04-27T01:47:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:01 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25903.202373 secs 2024-04-27T01:47:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25902.382720 secs 2024-04-27T01:47:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:01 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25903.202373 secs 2024-04-27T01:47:03.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:02 smithi119 ceph-mon[32752]: pgmap v14412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:03.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:02 smithi167 ceph-mon[32751]: pgmap v14412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:03.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:02 smithi090 ceph-mon[29180]: pgmap v14412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:05.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:04 smithi119 ceph-mon[32752]: pgmap v14413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:05.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:04 smithi167 ceph-mon[32751]: pgmap v14413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:05.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:04 smithi090 ceph-mon[29180]: pgmap v14413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 25907.382776 secs 2024-04-27T01:47:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:06 smithi119 ceph-mon[32752]: pgmap v14414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:06 smithi119 ceph-mon[32752]: 24 slow requests, 0 included below; oldest blocked for > 25908.202492 secs 2024-04-27T01:47:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 25907.382776 secs 2024-04-27T01:47:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:06 smithi167 ceph-mon[32751]: pgmap v14414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:06 smithi167 ceph-mon[32751]: 24 slow requests, 0 included below; oldest blocked for > 25908.202492 secs 2024-04-27T01:47:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 25907.382776 secs 2024-04-27T01:47:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:06 smithi090 ceph-mon[29180]: pgmap v14414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:07.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:06 smithi090 ceph-mon[29180]: 24 slow requests, 0 included below; oldest blocked for > 25908.202492 secs 2024-04-27T01:47:07.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:47:08.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:47:08.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:47:08.357 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:47:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:47:07.962+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:47:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:47:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:47:07.962+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:47:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:47:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:47:07.962+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:47:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:47:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:47:07.962+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:47:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:47:07 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:47:07.962+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:47:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:08 smithi119 ceph-mon[32752]: pgmap v14415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:09.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:08 smithi167 ceph-mon[32751]: pgmap v14415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:09.051 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:08 smithi090 ceph-mon[29180]: pgmap v14415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:10.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:10 smithi119 ceph-mon[32752]: pgmap v14416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:47:11.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:10 smithi167 ceph-mon[32751]: pgmap v14416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:47:11.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:10 smithi090 ceph-mon[29180]: pgmap v14416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:47:11.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:11 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 25912.382802 secs 2024-04-27T01:47:11.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:11 smithi119 ceph-mon[32752]: slow request 5763.790248 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:47:11.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:11 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25913.202590 secs 2024-04-27T01:47:11.990 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:11 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 25912.382802 secs 2024-04-27T01:47:11.990 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:11 smithi090 ceph-mon[29180]: slow request 5763.790248 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:47:11.990 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:11 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25913.202590 secs 2024-04-27T01:47:12.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:11 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 25912.382802 secs 2024-04-27T01:47:12.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:11 smithi167 ceph-mon[32751]: slow request 5763.790248 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:47:12.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:11 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25913.202590 secs 2024-04-27T01:47:12.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: pgmap v14417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:47:12.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:47:12.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:47:12.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:47:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:47:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: pgmap v14417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:47:13.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:47:13.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:47:13.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:47:13.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:47:13.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: pgmap v14417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:47:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:47:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:47:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:13.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:47:13.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:47:13.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:47:14.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:13 smithi119 ceph-mon[32752]: pgmap v14418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 195 B/s wr, 14 op/s 2024-04-27T01:47:14.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:13 smithi119 ceph-mon[32752]: pgmap v14419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 241 B/s wr, 17 op/s 2024-04-27T01:47:14.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:13 smithi167 ceph-mon[32751]: pgmap v14418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 195 B/s wr, 14 op/s 2024-04-27T01:47:14.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:13 smithi167 ceph-mon[32751]: pgmap v14419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 241 B/s wr, 17 op/s 2024-04-27T01:47:14.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:13 smithi090 ceph-mon[29180]: pgmap v14418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 195 B/s wr, 14 op/s 2024-04-27T01:47:14.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:13 smithi090 ceph-mon[29180]: pgmap v14419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 241 B/s wr, 17 op/s 2024-04-27T01:47:16.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:15 smithi119 ceph-mon[32752]: pgmap v14420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 B/s wr, 18 op/s 2024-04-27T01:47:16.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:15 smithi167 ceph-mon[32751]: pgmap v14420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 B/s wr, 18 op/s 2024-04-27T01:47:16.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:15 smithi090 ceph-mon[29180]: pgmap v14420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 B/s wr, 18 op/s 2024-04-27T01:47:17.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:16 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25917.382864 secs 2024-04-27T01:47:17.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:16 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25918.202689 secs 2024-04-27T01:47:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:16 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25917.382864 secs 2024-04-27T01:47:17.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:16 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25918.202689 secs 2024-04-27T01:47:17.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:16 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25917.382864 secs 2024-04-27T01:47:17.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:16 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25918.202689 secs 2024-04-27T01:47:18.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:17 smithi119 ceph-mon[32752]: pgmap v14421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 B/s wr, 18 op/s 2024-04-27T01:47:18.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:17 smithi167 ceph-mon[32751]: pgmap v14421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 B/s wr, 18 op/s 2024-04-27T01:47:18.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:17 smithi090 ceph-mon[29180]: pgmap v14421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 241 B/s wr, 18 op/s 2024-04-27T01:47:20.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:19 smithi119 ceph-mon[32752]: pgmap v14422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 603 B/s rd, 1 op/s 2024-04-27T01:47:20.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:19 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:47:20.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:19 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:47:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:19 smithi167 ceph-mon[32751]: pgmap v14422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 603 B/s rd, 1 op/s 2024-04-27T01:47:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:19 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:47:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:19 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:47:20.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:19 smithi090 ceph-mon[29180]: pgmap v14422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 603 B/s rd, 1 op/s 2024-04-27T01:47:20.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:19 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:47:20.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:19 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:47:22.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:21 smithi119 ceph-mon[32752]: pgmap v14423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:22.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:21 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25922.382949 secs 2024-04-27T01:47:22.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:21 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25923.202806 secs 2024-04-27T01:47:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:21 smithi167 ceph-mon[32751]: pgmap v14423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:21 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25922.382949 secs 2024-04-27T01:47:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:21 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25923.202806 secs 2024-04-27T01:47:22.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:21 smithi090 ceph-mon[29180]: pgmap v14423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:22.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:21 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25922.382949 secs 2024-04-27T01:47:22.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:21 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25923.202806 secs 2024-04-27T01:47:24.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:23 smithi090 ceph-mon[29180]: pgmap v14424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:47:24.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:23 smithi119 ceph-mon[32752]: pgmap v14424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:47:24.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:23 smithi167 ceph-mon[32751]: pgmap v14424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:47:26.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:25 smithi119 ceph-mon[32752]: pgmap v14425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:26.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:25 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25927.383036 secs 2024-04-27T01:47:26.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:25 smithi167 ceph-mon[32751]: pgmap v14425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:26.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:25 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25927.383036 secs 2024-04-27T01:47:26.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:25 smithi090 ceph-mon[29180]: pgmap v14425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:26.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:25 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25927.383036 secs 2024-04-27T01:47:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:26 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25928.202920 secs 2024-04-27T01:47:27.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:26 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25928.202920 secs 2024-04-27T01:47:27.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:26 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25928.202920 secs 2024-04-27T01:47:28.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:27 smithi119 ceph-mon[32752]: pgmap v14426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:28.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:27 smithi167 ceph-mon[32751]: pgmap v14426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:28.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:27 smithi090 ceph-mon[29180]: pgmap v14426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:30.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:29 smithi119 ceph-mon[32752]: pgmap v14427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:30.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:29 smithi167 ceph-mon[32751]: pgmap v14427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:30.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:29 smithi090 ceph-mon[29180]: pgmap v14427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:30 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25932.383124 secs 2024-04-27T01:47:31.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:30 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25932.383124 secs 2024-04-27T01:47:31.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:30 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25932.383124 secs 2024-04-27T01:47:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:31 smithi119 ceph-mon[32752]: pgmap v14428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:31 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25933.203036 secs 2024-04-27T01:47:32.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:31 smithi167 ceph-mon[32751]: pgmap v14428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:32.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:31 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25933.203036 secs 2024-04-27T01:47:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:31 smithi090 ceph-mon[29180]: pgmap v14428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:31 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25933.203036 secs 2024-04-27T01:47:34.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:33 smithi119 ceph-mon[32752]: pgmap v14429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:34.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:33 smithi167 ceph-mon[32751]: pgmap v14429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:34.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:33 smithi090 ceph-mon[29180]: pgmap v14429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:36.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:35 smithi119 ceph-mon[32752]: pgmap v14430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:36.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:35 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25937.383184 secs 2024-04-27T01:47:36.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:35 smithi167 ceph-mon[32751]: pgmap v14430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:36.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:35 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25937.383184 secs 2024-04-27T01:47:36.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:35 smithi090 ceph-mon[29180]: pgmap v14430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:36.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:35 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25937.383184 secs 2024-04-27T01:47:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:36 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25938.203185 secs 2024-04-27T01:47:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:36 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25938.203185 secs 2024-04-27T01:47:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:36 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25938.203185 secs 2024-04-27T01:47:38.023 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:37 smithi090 ceph-mon[29180]: pgmap v14431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:38.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:37 smithi119 ceph-mon[32752]: pgmap v14431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:38.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:37 smithi167 ceph-mon[32751]: pgmap v14431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:40.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:39 smithi119 ceph-mon[32752]: pgmap v14432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:40.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:39 smithi167 ceph-mon[32751]: pgmap v14432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:40.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:39 smithi090 ceph-mon[29180]: pgmap v14432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:41.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:40 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25942.383246 secs 2024-04-27T01:47:41.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:40 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25942.383246 secs 2024-04-27T01:47:41.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:40 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25942.383246 secs 2024-04-27T01:47:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:41 smithi119 ceph-mon[32752]: pgmap v14433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:42.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:41 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25943.203304 secs 2024-04-27T01:47:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:41 smithi167 ceph-mon[32751]: pgmap v14433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:41 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25943.203304 secs 2024-04-27T01:47:42.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:41 smithi090 ceph-mon[29180]: pgmap v14433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:42.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:41 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25943.203304 secs 2024-04-27T01:47:44.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:43 smithi119 ceph-mon[32752]: pgmap v14434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:44.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:43 smithi167 ceph-mon[32751]: pgmap v14434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:44.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:43 smithi090 ceph-mon[29180]: pgmap v14434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:46.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:45 smithi167 ceph-mon[32751]: pgmap v14435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:46.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:45 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25947.383306 secs 2024-04-27T01:47:46.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:45 smithi090 ceph-mon[29180]: pgmap v14435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:46.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:45 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25947.383306 secs 2024-04-27T01:47:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:45 smithi119 ceph-mon[32752]: pgmap v14435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:46.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:45 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25947.383306 secs 2024-04-27T01:47:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:46 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25948.203471 secs 2024-04-27T01:47:47.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:46 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25948.203471 secs 2024-04-27T01:47:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:46 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25948.203471 secs 2024-04-27T01:47:48.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:47 smithi167 ceph-mon[32751]: pgmap v14436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:48.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:47 smithi090 ceph-mon[29180]: pgmap v14436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:48.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:47 smithi119 ceph-mon[32752]: pgmap v14436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:50.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:49 smithi167 ceph-mon[32751]: pgmap v14437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:50.088 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:49 smithi119 ceph-mon[32752]: pgmap v14437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:50.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:49 smithi090 ceph-mon[29180]: pgmap v14437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:50 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25952.383368 secs 2024-04-27T01:47:51.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:50 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25952.383368 secs 2024-04-27T01:47:51.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:50 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25952.383368 secs 2024-04-27T01:47:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:51 smithi090 ceph-mon[29180]: pgmap v14438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:52.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:51 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25953.203601 secs 2024-04-27T01:47:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:51 smithi119 ceph-mon[32752]: pgmap v14438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:52.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:51 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25953.203601 secs 2024-04-27T01:47:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:51 smithi167 ceph-mon[32751]: pgmap v14438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:51 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25953.203601 secs 2024-04-27T01:47:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:53 smithi090 ceph-mon[29180]: pgmap v14439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:53 smithi119 ceph-mon[32752]: pgmap v14439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:54.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:53 smithi167 ceph-mon[32751]: pgmap v14439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:56.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:55 smithi090 ceph-mon[29180]: pgmap v14440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:56.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:55 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25957.383449 secs 2024-04-27T01:47:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:55 smithi119 ceph-mon[32752]: pgmap v14440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:56.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:55 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25957.383449 secs 2024-04-27T01:47:56.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:55 smithi167 ceph-mon[32751]: pgmap v14440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:47:56.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:55 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25957.383449 secs 2024-04-27T01:47:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:56 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25958.203741 secs 2024-04-27T01:47:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:56 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25958.203741 secs 2024-04-27T01:47:57.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:56 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25958.203741 secs 2024-04-27T01:47:58.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:57 smithi090 ceph-mon[29180]: pgmap v14441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:57 smithi119 ceph-mon[32752]: pgmap v14441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:47:58.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:57 smithi167 ceph-mon[32751]: pgmap v14441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:00.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:47:59 smithi090 ceph-mon[29180]: pgmap v14442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:47:59 smithi119 ceph-mon[32752]: pgmap v14442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:00.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:47:59 smithi167 ceph-mon[32751]: pgmap v14442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:00 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25962.383535 secs 2024-04-27T01:48:01.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:00 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25962.383535 secs 2024-04-27T01:48:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:00 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25962.383535 secs 2024-04-27T01:48:02.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:01 smithi090 ceph-mon[29180]: pgmap v14443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:01 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25963.203878 secs 2024-04-27T01:48:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:01 smithi119 ceph-mon[32752]: pgmap v14443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:02.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:01 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25963.203878 secs 2024-04-27T01:48:02.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:01 smithi167 ceph-mon[32751]: pgmap v14443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:02.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:01 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25963.203878 secs 2024-04-27T01:48:04.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:03 smithi090 ceph-mon[29180]: pgmap v14444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:04.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:03 smithi119 ceph-mon[32752]: pgmap v14444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:04.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:03 smithi167 ceph-mon[32751]: pgmap v14444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:06.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:05 smithi090 ceph-mon[29180]: pgmap v14445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:06.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:05 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25967.383619 secs 2024-04-27T01:48:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:05 smithi119 ceph-mon[32752]: pgmap v14445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:06.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:05 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25967.383619 secs 2024-04-27T01:48:06.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:05 smithi167 ceph-mon[32751]: pgmap v14445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:06.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:05 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25967.383619 secs 2024-04-27T01:48:07.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:06 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25968.203942 secs 2024-04-27T01:48:07.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:06 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25968.203942 secs 2024-04-27T01:48:07.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:06 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25968.203942 secs 2024-04-27T01:48:08.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:07 smithi090 ceph-mon[29180]: pgmap v14446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:08.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:07 smithi119 ceph-mon[32752]: pgmap v14446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:08.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:07 smithi167 ceph-mon[32751]: pgmap v14446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:10.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:09 smithi090 ceph-mon[29180]: pgmap v14447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:10.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:09 smithi119 ceph-mon[32752]: pgmap v14447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:10.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:09 smithi167 ceph-mon[32751]: pgmap v14447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:11.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:10 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25972.383702 secs 2024-04-27T01:48:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:10 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25972.383702 secs 2024-04-27T01:48:11.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:10 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25972.383702 secs 2024-04-27T01:48:12.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:11 smithi090 ceph-mon[29180]: pgmap v14448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:12.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:11 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25973.204077 secs 2024-04-27T01:48:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:11 smithi119 ceph-mon[32752]: pgmap v14448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:12.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:11 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25973.204077 secs 2024-04-27T01:48:12.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:11 smithi167 ceph-mon[32751]: pgmap v14448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:12.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:11 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25973.204077 secs 2024-04-27T01:48:13.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:48:13.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:48:13.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:48:14.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:13 smithi119 ceph-mon[32752]: pgmap v14449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:14.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:13 smithi167 ceph-mon[32751]: pgmap v14449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:14.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:13 smithi090 ceph-mon[29180]: pgmap v14449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:48:16.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:15 smithi167 ceph-mon[32751]: pgmap v14450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:16.051 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:15 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25977.383784 secs 2024-04-27T01:48:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:15 smithi119 ceph-mon[32752]: pgmap v14450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:16.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:15 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25977.383784 secs 2024-04-27T01:48:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:15 smithi090 ceph-mon[29180]: pgmap v14450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:48:16.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:15 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25977.383784 secs 2024-04-27T01:48:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:16 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25978.204228 secs 2024-04-27T01:48:17.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:48:17.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:48:17.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:16 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25978.204228 secs 2024-04-27T01:48:17.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:48:17.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:48:17.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:16 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25978.204228 secs 2024-04-27T01:48:17.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:17.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:48:17.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:48:17.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:48:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:17 smithi119 ceph-mon[32752]: pgmap v14451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:18.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:17 smithi119 ceph-mon[32752]: pgmap v14452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 1 op/s 2024-04-27T01:48:18.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:17 smithi119 ceph-mon[32752]: pgmap v14453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:18.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:17 smithi167 ceph-mon[32751]: pgmap v14451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:18.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:17 smithi167 ceph-mon[32751]: pgmap v14452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 1 op/s 2024-04-27T01:48:18.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:17 smithi167 ceph-mon[32751]: pgmap v14453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:17 smithi090 ceph-mon[29180]: pgmap v14451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:18.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:17 smithi090 ceph-mon[29180]: pgmap v14452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 1 op/s 2024-04-27T01:48:18.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:17 smithi090 ceph-mon[29180]: pgmap v14453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:19 smithi119 ceph-mon[32752]: pgmap v14454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:20.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:19 smithi167 ceph-mon[32751]: pgmap v14454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:20.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:19 smithi090 ceph-mon[29180]: pgmap v14454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:21.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:20 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25982.383842 secs 2024-04-27T01:48:21.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:20 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25982.383842 secs 2024-04-27T01:48:21.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:20 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25982.383842 secs 2024-04-27T01:48:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:21 smithi119 ceph-mon[32752]: pgmap v14455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:21 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25983.204357 secs 2024-04-27T01:48:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:21 smithi167 ceph-mon[32751]: pgmap v14455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:21 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25983.204357 secs 2024-04-27T01:48:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:21 smithi090 ceph-mon[29180]: pgmap v14455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:22.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:21 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25983.204357 secs 2024-04-27T01:48:24.160 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:23 smithi090 ceph-mon[29180]: pgmap v14456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T01:48:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:23 smithi119 ceph-mon[32752]: pgmap v14456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T01:48:24.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:23 smithi167 ceph-mon[32751]: pgmap v14456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T01:48:26.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:25 smithi167 ceph-mon[32751]: pgmap v14457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:26.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:25 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25987.383926 secs 2024-04-27T01:48:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:25 smithi119 ceph-mon[32752]: pgmap v14457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:26.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:25 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25987.383926 secs 2024-04-27T01:48:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:25 smithi090 ceph-mon[29180]: pgmap v14457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:26.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:25 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25987.383926 secs 2024-04-27T01:48:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:26 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25988.204480 secs 2024-04-27T01:48:27.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:26 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25988.204480 secs 2024-04-27T01:48:27.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:26 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25988.204480 secs 2024-04-27T01:48:28.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:27 smithi119 ceph-mon[32752]: pgmap v14458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:48:28.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:27 smithi167 ceph-mon[32751]: pgmap v14458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:48:28.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:27 smithi090 ceph-mon[29180]: pgmap v14458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:48:30.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:29 smithi119 ceph-mon[32752]: pgmap v14459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:30.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:29 smithi167 ceph-mon[32751]: pgmap v14459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:30.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:29 smithi090 ceph-mon[29180]: pgmap v14459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:31.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:30 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25992.383988 secs 2024-04-27T01:48:31.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:30 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25992.383988 secs 2024-04-27T01:48:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:30 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25992.383988 secs 2024-04-27T01:48:32.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:31 smithi167 ceph-mon[32751]: pgmap v14460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:32.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:31 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25993.204550 secs 2024-04-27T01:48:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:31 smithi119 ceph-mon[32752]: pgmap v14460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:31 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25993.204550 secs 2024-04-27T01:48:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:31 smithi090 ceph-mon[29180]: pgmap v14460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:32.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:31 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25993.204550 secs 2024-04-27T01:48:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:33 smithi119 ceph-mon[32752]: pgmap v14461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:33 smithi167 ceph-mon[32751]: pgmap v14461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:33 smithi090 ceph-mon[29180]: pgmap v14461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:35 smithi119 ceph-mon[32752]: pgmap v14462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:36.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:35 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 25997.384039 secs 2024-04-27T01:48:36.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:35 smithi167 ceph-mon[32751]: pgmap v14462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:36.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:35 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 25997.384039 secs 2024-04-27T01:48:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:35 smithi090 ceph-mon[29180]: pgmap v14462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:36.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:35 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 25997.384039 secs 2024-04-27T01:48:37.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:36 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 25998.204663 secs 2024-04-27T01:48:37.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:36 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 25998.204663 secs 2024-04-27T01:48:37.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:36 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 25998.204663 secs 2024-04-27T01:48:38.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:37 smithi119 ceph-mon[32752]: pgmap v14463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:38.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:37 smithi167 ceph-mon[32751]: pgmap v14463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:38.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:37 smithi090 ceph-mon[29180]: pgmap v14463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:40.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:39 smithi090 ceph-mon[29180]: pgmap v14464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:39 smithi119 ceph-mon[32752]: pgmap v14464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:40.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:39 smithi167 ceph-mon[32751]: pgmap v14464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:40 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26002.384119 secs 2024-04-27T01:48:41.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:40 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26002.384119 secs 2024-04-27T01:48:41.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:40 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26002.384119 secs 2024-04-27T01:48:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:41 smithi119 ceph-mon[32752]: pgmap v14465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:42.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:41 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26003.204724 secs 2024-04-27T01:48:42.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:41 smithi167 ceph-mon[32751]: pgmap v14465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:42.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:41 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26003.204724 secs 2024-04-27T01:48:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:41 smithi090 ceph-mon[29180]: pgmap v14465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:42.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:41 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26003.204724 secs 2024-04-27T01:48:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:43 smithi119 ceph-mon[32752]: pgmap v14466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:44.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:43 smithi167 ceph-mon[32751]: pgmap v14466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:44.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:43 smithi090 ceph-mon[29180]: pgmap v14466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:45 smithi119 ceph-mon[32752]: pgmap v14467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:46.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:45 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26007.384204 secs 2024-04-27T01:48:46.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:45 smithi167 ceph-mon[32751]: pgmap v14467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:46.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:45 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26007.384204 secs 2024-04-27T01:48:46.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:45 smithi090 ceph-mon[29180]: pgmap v14467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:46.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:45 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26007.384204 secs 2024-04-27T01:48:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:46 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26008.204815 secs 2024-04-27T01:48:47.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:46 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26008.204815 secs 2024-04-27T01:48:47.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:46 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26008.204815 secs 2024-04-27T01:48:48.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:47 smithi119 ceph-mon[32752]: pgmap v14468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:48.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:47 smithi167 ceph-mon[32751]: pgmap v14468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:48.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:47 smithi090 ceph-mon[29180]: pgmap v14468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:50.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:49 smithi119 ceph-mon[32752]: pgmap v14469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:50.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:49 smithi167 ceph-mon[32751]: pgmap v14469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:49 smithi090 ceph-mon[29180]: pgmap v14469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:50 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26012.384266 secs 2024-04-27T01:48:51.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:50 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26012.384266 secs 2024-04-27T01:48:51.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:50 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26012.384266 secs 2024-04-27T01:48:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:51 smithi119 ceph-mon[32752]: pgmap v14470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:52.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:51 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26013.204896 secs 2024-04-27T01:48:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:51 smithi167 ceph-mon[32751]: pgmap v14470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:51 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26013.204896 secs 2024-04-27T01:48:52.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:51 smithi090 ceph-mon[29180]: pgmap v14470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:52.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:51 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26013.204896 secs 2024-04-27T01:48:53.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:52 smithi119 ceph-mon[32752]: pgmap v14471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:53.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:52 smithi167 ceph-mon[32751]: pgmap v14471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:53.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:52 smithi090 ceph-mon[29180]: pgmap v14471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:55.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:55 smithi090 ceph-mon[29180]: pgmap v14472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:56.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:55 smithi119 ceph-mon[32752]: pgmap v14472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:56.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:55 smithi167 ceph-mon[32751]: pgmap v14472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:48:56.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:56 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26017.384343 secs 2024-04-27T01:48:56.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:56 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26018.205014 secs 2024-04-27T01:48:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:56 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26017.384343 secs 2024-04-27T01:48:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:56 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26018.205014 secs 2024-04-27T01:48:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:56 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26017.384343 secs 2024-04-27T01:48:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:56 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26018.205014 secs 2024-04-27T01:48:57.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:57 smithi090 ceph-mon[29180]: pgmap v14473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:58.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:57 smithi119 ceph-mon[32752]: pgmap v14473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:58.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:57 smithi167 ceph-mon[32751]: pgmap v14473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:48:59.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:48:59 smithi090 ceph-mon[29180]: pgmap v14474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:00.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:48:59 smithi119 ceph-mon[32752]: pgmap v14474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:00.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:48:59 smithi167 ceph-mon[32751]: pgmap v14474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:01.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:01 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26022.384404 secs 2024-04-27T01:49:01.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:01 smithi167 ceph-mon[32751]: pgmap v14475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:01.853 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:01 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26023.205138 secs 2024-04-27T01:49:01.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:01 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26022.384404 secs 2024-04-27T01:49:01.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:01 smithi090 ceph-mon[29180]: pgmap v14475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:01.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:01 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26023.205138 secs 2024-04-27T01:49:02.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:01 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26022.384404 secs 2024-04-27T01:49:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:01 smithi119 ceph-mon[32752]: pgmap v14475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:01 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26023.205138 secs 2024-04-27T01:49:03.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:03 smithi090 ceph-mon[29180]: pgmap v14476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:04.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:03 smithi119 ceph-mon[32752]: pgmap v14476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:04.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:03 smithi167 ceph-mon[32751]: pgmap v14476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:05.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:05 smithi090 ceph-mon[29180]: pgmap v14477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:06.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:05 smithi119 ceph-mon[32752]: pgmap v14477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:06.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:05 smithi167 ceph-mon[32751]: pgmap v14477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:06.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:06 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26027.384485 secs 2024-04-27T01:49:06.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:06 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26028.205253 secs 2024-04-27T01:49:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:06 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26027.384485 secs 2024-04-27T01:49:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:06 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26028.205253 secs 2024-04-27T01:49:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:06 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26027.384485 secs 2024-04-27T01:49:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:06 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26028.205253 secs 2024-04-27T01:49:07.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:07 smithi090 ceph-mon[29180]: pgmap v14478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:08.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:07 smithi119 ceph-mon[32752]: pgmap v14478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:08.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:07 smithi167 ceph-mon[32751]: pgmap v14478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:09.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:09 smithi090 ceph-mon[29180]: pgmap v14479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:10.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:09 smithi119 ceph-mon[32752]: pgmap v14479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:10.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:09 smithi167 ceph-mon[32751]: pgmap v14479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:11.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:11 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26032.384571 secs 2024-04-27T01:49:11.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:11 smithi090 ceph-mon[29180]: pgmap v14480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:11.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:11 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26033.205358 secs 2024-04-27T01:49:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:11 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26032.384571 secs 2024-04-27T01:49:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:11 smithi119 ceph-mon[32752]: pgmap v14480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:12.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:11 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26033.205358 secs 2024-04-27T01:49:12.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:11 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26032.384571 secs 2024-04-27T01:49:12.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:11 smithi167 ceph-mon[32751]: pgmap v14480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:12.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:11 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26033.205358 secs 2024-04-27T01:49:13.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:13 smithi090 ceph-mon[29180]: pgmap v14481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:14.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:13 smithi119 ceph-mon[32752]: pgmap v14481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:14.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:13 smithi167 ceph-mon[32751]: pgmap v14481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:16.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:15 smithi119 ceph-mon[32752]: pgmap v14482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:16.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:15 smithi167 ceph-mon[32751]: pgmap v14482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:16.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:15 smithi090 ceph-mon[29180]: pgmap v14482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:17.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:16 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26037.384647 secs 2024-04-27T01:49:17.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:16 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26038.205481 secs 2024-04-27T01:49:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:16 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26037.384647 secs 2024-04-27T01:49:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:16 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26038.205481 secs 2024-04-27T01:49:17.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:16 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26037.384647 secs 2024-04-27T01:49:17.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:16 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26038.205481 secs 2024-04-27T01:49:18.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:17 smithi119 ceph-mon[32752]: pgmap v14483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:18.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:49:18.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:49:18.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:49:18.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:49:18.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:17 smithi167 ceph-mon[32751]: pgmap v14483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:18.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:49:18.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:49:18.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:49:18.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:49:18.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:17 smithi090 ceph-mon[29180]: pgmap v14483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:18.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:49:18.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:49:18.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:49:18.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:49:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:18 smithi119 ceph-mon[32752]: pgmap v14484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T01:49:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:18 smithi119 ceph-mon[32752]: pgmap v14485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T01:49:19.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:18 smithi167 ceph-mon[32751]: pgmap v14484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T01:49:19.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:18 smithi167 ceph-mon[32751]: pgmap v14485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T01:49:19.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:18 smithi090 ceph-mon[29180]: pgmap v14484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T01:49:19.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:18 smithi090 ceph-mon[29180]: pgmap v14485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T01:49:21.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:20 smithi119 ceph-mon[32752]: pgmap v14486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:21.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:20 smithi167 ceph-mon[32751]: pgmap v14486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:21.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:20 smithi090 ceph-mon[29180]: pgmap v14486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:22.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:21 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26042.384734 secs 2024-04-27T01:49:22.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:21 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26043.205621 secs 2024-04-27T01:49:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:21 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26042.384734 secs 2024-04-27T01:49:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:21 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26043.205621 secs 2024-04-27T01:49:22.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:21 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26042.384734 secs 2024-04-27T01:49:22.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:21 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26043.205621 secs 2024-04-27T01:49:23.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:22 smithi119 ceph-mon[32752]: pgmap v14487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:23.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:22 smithi167 ceph-mon[32751]: pgmap v14487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:23.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:22 smithi090 ceph-mon[29180]: pgmap v14487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:25.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:24 smithi119 ceph-mon[32752]: pgmap v14488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:25.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:24 smithi167 ceph-mon[32751]: pgmap v14488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:25.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:24 smithi090 ceph-mon[29180]: pgmap v14488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:26.910 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:26 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26047.384813 secs 2024-04-27T01:49:26.910 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:26 smithi090 ceph-mon[29180]: pgmap v14489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:26.910 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:26 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26048.205730 secs 2024-04-27T01:49:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:26 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26047.384813 secs 2024-04-27T01:49:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:26 smithi119 ceph-mon[32752]: pgmap v14489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:27.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:26 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26048.205730 secs 2024-04-27T01:49:27.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:26 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26047.384813 secs 2024-04-27T01:49:27.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:26 smithi167 ceph-mon[32751]: pgmap v14489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:49:27.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:26 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26048.205730 secs 2024-04-27T01:49:29.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:28 smithi119 ceph-mon[32752]: pgmap v14490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:49:29.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:28 smithi167 ceph-mon[32751]: pgmap v14490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:49:29.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:28 smithi090 ceph-mon[29180]: pgmap v14490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:49:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:30 smithi119 ceph-mon[32752]: pgmap v14491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:31.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:30 smithi167 ceph-mon[32751]: pgmap v14491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:31.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:30 smithi090 ceph-mon[29180]: pgmap v14491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:31.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:31 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26052.384870 secs 2024-04-27T01:49:31.951 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:31 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26053.205843 secs 2024-04-27T01:49:32.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:31 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26052.384870 secs 2024-04-27T01:49:32.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:31 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26053.205843 secs 2024-04-27T01:49:32.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:31 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26052.384870 secs 2024-04-27T01:49:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:31 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26053.205843 secs 2024-04-27T01:49:33.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:32 smithi119 ceph-mon[32752]: pgmap v14492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:33.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:32 smithi167 ceph-mon[32751]: pgmap v14492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:33.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:32 smithi090 ceph-mon[29180]: pgmap v14492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:35.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:34 smithi119 ceph-mon[32752]: pgmap v14493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:35.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:34 smithi167 ceph-mon[32751]: pgmap v14493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:35.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:34 smithi090 ceph-mon[29180]: pgmap v14493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:37.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:36 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26057.384922 secs 2024-04-27T01:49:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:36 smithi119 ceph-mon[32752]: pgmap v14494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:37.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:36 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26058.205966 secs 2024-04-27T01:49:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:36 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26057.384922 secs 2024-04-27T01:49:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:36 smithi167 ceph-mon[32751]: pgmap v14494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:36 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26058.205966 secs 2024-04-27T01:49:37.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:36 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26057.384922 secs 2024-04-27T01:49:37.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:36 smithi090 ceph-mon[29180]: pgmap v14494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:37.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:36 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26058.205966 secs 2024-04-27T01:49:39.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:38 smithi119 ceph-mon[32752]: pgmap v14495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:39.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:38 smithi167 ceph-mon[32751]: pgmap v14495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:39.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:38 smithi090 ceph-mon[29180]: pgmap v14495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:41.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:40 smithi119 ceph-mon[32752]: pgmap v14496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:41.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:40 smithi167 ceph-mon[32751]: pgmap v14496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:41.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:40 smithi090 ceph-mon[29180]: pgmap v14496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:41 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 26062.385003 secs 2024-04-27T01:49:42.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:41 smithi119 ceph-mon[32752]: slow request 1442.553614 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:49:42.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:41 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26063.206076 secs 2024-04-27T01:49:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:41 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 26062.385003 secs 2024-04-27T01:49:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:41 smithi167 ceph-mon[32751]: slow request 1442.553614 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:49:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:41 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26063.206076 secs 2024-04-27T01:49:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:41 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 26062.385003 secs 2024-04-27T01:49:42.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:41 smithi090 ceph-mon[29180]: slow request 1442.553614 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:49:42.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:41 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26063.206076 secs 2024-04-27T01:49:43.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:42 smithi090 ceph-mon[29180]: pgmap v14497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:43.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:42 smithi119 ceph-mon[32752]: pgmap v14497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:43.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:42 smithi167 ceph-mon[32751]: pgmap v14497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:45.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:44 smithi119 ceph-mon[32752]: pgmap v14498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:45.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:44 smithi167 ceph-mon[32751]: pgmap v14498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:45.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:44 smithi090 ceph-mon[29180]: pgmap v14498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:46 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26067.385087 secs 2024-04-27T01:49:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:46 smithi119 ceph-mon[32752]: pgmap v14499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:47.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:46 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26068.206193 secs 2024-04-27T01:49:47.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:46 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26067.385087 secs 2024-04-27T01:49:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:46 smithi167 ceph-mon[32751]: pgmap v14499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:47.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:46 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26068.206193 secs 2024-04-27T01:49:47.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:46 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26067.385087 secs 2024-04-27T01:49:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:46 smithi090 ceph-mon[29180]: pgmap v14499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:47.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:46 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26068.206193 secs 2024-04-27T01:49:49.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:48 smithi119 ceph-mon[32752]: pgmap v14500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:49.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:48 smithi167 ceph-mon[32751]: pgmap v14500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:49.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:48 smithi090 ceph-mon[29180]: pgmap v14500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:51.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:50 smithi119 ceph-mon[32752]: pgmap v14501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:51.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:50 smithi167 ceph-mon[32751]: pgmap v14501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:50 smithi090 ceph-mon[29180]: pgmap v14501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:51 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26072.385169 secs 2024-04-27T01:49:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:51 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26073.206260 secs 2024-04-27T01:49:52.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:51 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26072.385169 secs 2024-04-27T01:49:52.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:51 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26073.206260 secs 2024-04-27T01:49:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:51 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26072.385169 secs 2024-04-27T01:49:52.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:51 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26073.206260 secs 2024-04-27T01:49:53.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:52 smithi119 ceph-mon[32752]: pgmap v14502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:53.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:52 smithi167 ceph-mon[32751]: pgmap v14502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:53.109 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:52 smithi090 ceph-mon[29180]: pgmap v14502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:55.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:54 smithi119 ceph-mon[32752]: pgmap v14503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:55.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:54 smithi167 ceph-mon[32751]: pgmap v14503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:55.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:54 smithi090 ceph-mon[29180]: pgmap v14503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:49:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:56 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26077.385232 secs 2024-04-27T01:49:57.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:56 smithi119 ceph-mon[32752]: pgmap v14504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:57.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:56 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26078.206389 secs 2024-04-27T01:49:57.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:56 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26077.385232 secs 2024-04-27T01:49:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:56 smithi167 ceph-mon[32751]: pgmap v14504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:57.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:56 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26078.206389 secs 2024-04-27T01:49:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:56 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26077.385232 secs 2024-04-27T01:49:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:56 smithi090 ceph-mon[29180]: pgmap v14504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:57.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:56 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26078.206389 secs 2024-04-27T01:49:59.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:49:58 smithi119 ceph-mon[32752]: pgmap v14505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:59.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:49:58 smithi167 ceph-mon[32751]: pgmap v14505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:49:59.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:49:58 smithi090 ceph-mon[29180]: pgmap v14505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:01.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:00 smithi119 ceph-mon[32752]: pgmap v14506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:01.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:50:01.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:00 smithi167 ceph-mon[32751]: pgmap v14506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:01.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:50:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:00 smithi090 ceph-mon[29180]: pgmap v14506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:01.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T01:50:02.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:01 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26082.385293 secs 2024-04-27T01:50:02.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:01 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26083.206466 secs 2024-04-27T01:50:02.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:01 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26082.385293 secs 2024-04-27T01:50:02.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:01 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26083.206466 secs 2024-04-27T01:50:02.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:01 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26082.385293 secs 2024-04-27T01:50:02.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:01 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26083.206466 secs 2024-04-27T01:50:03.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:02 smithi119 ceph-mon[32752]: pgmap v14507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:03.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:02 smithi167 ceph-mon[32751]: pgmap v14507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:03.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:02 smithi090 ceph-mon[29180]: pgmap v14507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:05.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:04 smithi119 ceph-mon[32752]: pgmap v14508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:05.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:04 smithi167 ceph-mon[32751]: pgmap v14508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:05.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:04 smithi090 ceph-mon[29180]: pgmap v14508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:07.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:06 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26087.385356 secs 2024-04-27T01:50:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:06 smithi119 ceph-mon[32752]: pgmap v14509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:07.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:06 smithi119 ceph-mon[32752]: 4 slow requests, 0 included below; oldest blocked for > 26088.206583 secs 2024-04-27T01:50:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:06 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26087.385356 secs 2024-04-27T01:50:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:06 smithi167 ceph-mon[32751]: pgmap v14509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:07.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:06 smithi167 ceph-mon[32751]: 4 slow requests, 0 included below; oldest blocked for > 26088.206583 secs 2024-04-27T01:50:07.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:06 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26087.385356 secs 2024-04-27T01:50:07.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:06 smithi090 ceph-mon[29180]: pgmap v14509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:07.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:06 smithi090 ceph-mon[29180]: 4 slow requests, 0 included below; oldest blocked for > 26088.206583 secs 2024-04-27T01:50:09.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:08 smithi119 ceph-mon[32752]: pgmap v14510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:09.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:08 smithi167 ceph-mon[32751]: pgmap v14510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:09.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:08 smithi090 ceph-mon[29180]: pgmap v14510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:11.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:10 smithi119 ceph-mon[32752]: pgmap v14511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:11.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:10 smithi167 ceph-mon[32751]: pgmap v14511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:11.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:10 smithi090 ceph-mon[29180]: pgmap v14511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:11 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 26092.385435 secs 2024-04-27T01:50:12.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:11 smithi119 ceph-mon[32752]: 25 slow requests, 1 included below; oldest blocked for > 26093.206697 secs 2024-04-27T01:50:12.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:11 smithi119 ceph-mon[32752]: slow request 182.988747 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:50:12.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:11 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 26092.385435 secs 2024-04-27T01:50:12.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:11 smithi167 ceph-mon[32751]: 25 slow requests, 1 included below; oldest blocked for > 26093.206697 secs 2024-04-27T01:50:12.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:11 smithi167 ceph-mon[32751]: slow request 182.988747 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:50:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:11 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 26092.385435 secs 2024-04-27T01:50:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:11 smithi090 ceph-mon[29180]: 25 slow requests, 1 included below; oldest blocked for > 26093.206697 secs 2024-04-27T01:50:12.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:11 smithi090 ceph-mon[29180]: slow request 182.988747 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T01:50:13.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:12 smithi119 ceph-mon[32752]: pgmap v14512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:13.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:50:13.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:50:13.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:12 smithi167 ceph-mon[32751]: pgmap v14512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:13.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:50:13.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:50:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:12 smithi090 ceph-mon[29180]: pgmap v14512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:13.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:50:13.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:50:14.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:14 smithi090 ceph-mon[29180]: pgmap v14513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:15.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:14 smithi119 ceph-mon[32752]: pgmap v14513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:15.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:14 smithi167 ceph-mon[32751]: pgmap v14513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:17.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:16 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26097.385519 secs 2024-04-27T01:50:17.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:16 smithi119 ceph-mon[32752]: slow request 184.262300 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:50:17.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:16 smithi119 ceph-mon[32752]: pgmap v14514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:17.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26098.206771 secs 2024-04-27T01:50:17.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:16 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26097.385519 secs 2024-04-27T01:50:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:16 smithi167 ceph-mon[32751]: slow request 184.262300 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:50:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:16 smithi167 ceph-mon[32751]: pgmap v14514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:17.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26098.206771 secs 2024-04-27T01:50:17.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:16 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26097.385519 secs 2024-04-27T01:50:17.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:16 smithi090 ceph-mon[29180]: slow request 184.262300 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:50:17.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:16 smithi090 ceph-mon[29180]: pgmap v14514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:17.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26098.206771 secs 2024-04-27T01:50:18.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:50:18.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:50:18.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:50:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:18 smithi119 ceph-mon[32752]: pgmap v14515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:19.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:50:19.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:50:19.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:50:19.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:18 smithi167 ceph-mon[32751]: pgmap v14515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:19.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:50:19.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:50:19.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:50:19.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:18 smithi090 ceph-mon[29180]: pgmap v14515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:19.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:50:19.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:50:19.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:50:20.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:19 smithi119 ceph-mon[32752]: pgmap v14516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:50:20.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:19 smithi119 ceph-mon[32752]: pgmap v14517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:50:20.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:19 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:50:20.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:19 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:50:20.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:19 smithi167 ceph-mon[32751]: pgmap v14516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:50:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:19 smithi167 ceph-mon[32751]: pgmap v14517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:50:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:19 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:50:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:19 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:50:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:19 smithi090 ceph-mon[29180]: pgmap v14516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:50:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:19 smithi090 ceph-mon[29180]: pgmap v14517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:50:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:19 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T01:50:20.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:19 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:50:22.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:21 smithi119 ceph-mon[32752]: pgmap v14518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:22.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26102.385605 secs 2024-04-27T01:50:22.036 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26103.206883 secs 2024-04-27T01:50:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:21 smithi167 ceph-mon[32751]: pgmap v14518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26102.385605 secs 2024-04-27T01:50:22.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26103.206883 secs 2024-04-27T01:50:22.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:21 smithi090 ceph-mon[29180]: pgmap v14518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:22.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26102.385605 secs 2024-04-27T01:50:22.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26103.206883 secs 2024-04-27T01:50:24.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:23 smithi119 ceph-mon[32752]: pgmap v14519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:50:24.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:23 smithi167 ceph-mon[32751]: pgmap v14519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:50:24.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:23 smithi090 ceph-mon[29180]: pgmap v14519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:50:26.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:25 smithi119 ceph-mon[32752]: pgmap v14520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:26.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26107.385684 secs 2024-04-27T01:50:26.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:25 smithi167 ceph-mon[32751]: pgmap v14520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:26.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26107.385684 secs 2024-04-27T01:50:26.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:25 smithi090 ceph-mon[29180]: pgmap v14520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:26.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26107.385684 secs 2024-04-27T01:50:27.009 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26108.207004 secs 2024-04-27T01:50:27.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26108.207004 secs 2024-04-27T01:50:27.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26108.207004 secs 2024-04-27T01:50:28.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:27 smithi119 ceph-mon[32752]: pgmap v14521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:28.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:27 smithi167 ceph-mon[32751]: pgmap v14521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:28.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:27 smithi090 ceph-mon[29180]: pgmap v14521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:50:30.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:29 smithi119 ceph-mon[32752]: pgmap v14522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:50:30.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:29 smithi167 ceph-mon[32751]: pgmap v14522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:50:30.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:29 smithi090 ceph-mon[29180]: pgmap v14522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:50:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26112.385795 secs 2024-04-27T01:50:31.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26112.385795 secs 2024-04-27T01:50:31.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26112.385795 secs 2024-04-27T01:50:32.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:31 smithi119 ceph-mon[32752]: pgmap v14523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:32.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26113.207117 secs 2024-04-27T01:50:32.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:31 smithi167 ceph-mon[32751]: pgmap v14523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:32.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26113.207117 secs 2024-04-27T01:50:32.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:31 smithi090 ceph-mon[29180]: pgmap v14523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:32.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26113.207117 secs 2024-04-27T01:50:34.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:33 smithi119 ceph-mon[32752]: pgmap v14524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:34.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:33 smithi167 ceph-mon[32751]: pgmap v14524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:34.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:33 smithi090 ceph-mon[29180]: pgmap v14524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:36.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:35 smithi119 ceph-mon[32752]: pgmap v14525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:36.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26117.385880 secs 2024-04-27T01:50:36.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:35 smithi167 ceph-mon[32751]: pgmap v14525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:36.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26117.385880 secs 2024-04-27T01:50:36.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:35 smithi090 ceph-mon[29180]: pgmap v14525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:36.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26117.385880 secs 2024-04-27T01:50:37.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26118.207234 secs 2024-04-27T01:50:37.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26118.207234 secs 2024-04-27T01:50:37.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26118.207234 secs 2024-04-27T01:50:38.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:37 smithi119 ceph-mon[32752]: pgmap v14526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:38.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:37 smithi167 ceph-mon[32751]: pgmap v14526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:38.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:37 smithi090 ceph-mon[29180]: pgmap v14526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:40.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:39 smithi167 ceph-mon[32751]: pgmap v14527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:40.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:39 smithi090 ceph-mon[29180]: pgmap v14527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:39 smithi119 ceph-mon[32752]: pgmap v14527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:41.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26122.385962 secs 2024-04-27T01:50:41.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26122.385962 secs 2024-04-27T01:50:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26122.385962 secs 2024-04-27T01:50:42.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:41 smithi167 ceph-mon[32751]: pgmap v14528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:42.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26123.207333 secs 2024-04-27T01:50:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:41 smithi090 ceph-mon[29180]: pgmap v14528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26123.207333 secs 2024-04-27T01:50:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:41 smithi119 ceph-mon[32752]: pgmap v14528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26123.207333 secs 2024-04-27T01:50:44.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:43 smithi167 ceph-mon[32751]: pgmap v14529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:44.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:43 smithi090 ceph-mon[29180]: pgmap v14529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:43 smithi119 ceph-mon[32752]: pgmap v14529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:46.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:45 smithi167 ceph-mon[32751]: pgmap v14530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:46.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26127.386050 secs 2024-04-27T01:50:46.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:45 smithi090 ceph-mon[29180]: pgmap v14530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:46.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26127.386050 secs 2024-04-27T01:50:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:45 smithi119 ceph-mon[32752]: pgmap v14530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26127.386050 secs 2024-04-27T01:50:47.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26128.207453 secs 2024-04-27T01:50:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26128.207453 secs 2024-04-27T01:50:47.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26128.207453 secs 2024-04-27T01:50:48.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:47 smithi090 ceph-mon[29180]: pgmap v14531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:48.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:47 smithi119 ceph-mon[32752]: pgmap v14531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:48.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:47 smithi167 ceph-mon[32751]: pgmap v14531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:50.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:49 smithi090 ceph-mon[29180]: pgmap v14532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:50.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:49 smithi119 ceph-mon[32752]: pgmap v14532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:50.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:49 smithi167 ceph-mon[32751]: pgmap v14532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26132.386115 secs 2024-04-27T01:50:51.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26132.386115 secs 2024-04-27T01:50:51.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26132.386115 secs 2024-04-27T01:50:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:51 smithi090 ceph-mon[29180]: pgmap v14533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26133.207567 secs 2024-04-27T01:50:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:51 smithi119 ceph-mon[32752]: pgmap v14533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26133.207567 secs 2024-04-27T01:50:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:51 smithi167 ceph-mon[32751]: pgmap v14533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26133.207567 secs 2024-04-27T01:50:54.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:53 smithi090 ceph-mon[29180]: pgmap v14534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:53 smithi119 ceph-mon[32752]: pgmap v14534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:54.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:53 smithi167 ceph-mon[32751]: pgmap v14534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:56.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:55 smithi090 ceph-mon[29180]: pgmap v14535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:56.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:55 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26137.386172 secs 2024-04-27T01:50:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:55 smithi119 ceph-mon[32752]: pgmap v14535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:55 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26137.386172 secs 2024-04-27T01:50:56.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:55 smithi167 ceph-mon[32751]: pgmap v14535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:50:56.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:55 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26137.386172 secs 2024-04-27T01:50:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26138.207695 secs 2024-04-27T01:50:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26138.207695 secs 2024-04-27T01:50:57.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26138.207695 secs 2024-04-27T01:50:58.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:57 smithi090 ceph-mon[29180]: pgmap v14536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:57 smithi119 ceph-mon[32752]: pgmap v14536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:50:58.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:57 smithi167 ceph-mon[32751]: pgmap v14536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:00.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:50:59 smithi090 ceph-mon[29180]: pgmap v14537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:00.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:50:59 smithi119 ceph-mon[32752]: pgmap v14537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:00.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:50:59 smithi167 ceph-mon[32751]: pgmap v14537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26142.386231 secs 2024-04-27T01:51:01.200 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26142.386231 secs 2024-04-27T01:51:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26142.386231 secs 2024-04-27T01:51:02.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:01 smithi090 ceph-mon[29180]: pgmap v14538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:02.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:01 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26143.207813 secs 2024-04-27T01:51:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:01 smithi119 ceph-mon[32752]: pgmap v14538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:01 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26143.207813 secs 2024-04-27T01:51:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:01 smithi167 ceph-mon[32751]: pgmap v14538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:01 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26143.207813 secs 2024-04-27T01:51:04.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:03 smithi090 ceph-mon[29180]: pgmap v14539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:04.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:03 smithi119 ceph-mon[32752]: pgmap v14539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:04.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:03 smithi167 ceph-mon[32751]: pgmap v14539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:06.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:05 smithi090 ceph-mon[29180]: pgmap v14540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:06.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26147.386315 secs 2024-04-27T01:51:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:05 smithi119 ceph-mon[32752]: pgmap v14540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26147.386315 secs 2024-04-27T01:51:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:05 smithi167 ceph-mon[32751]: pgmap v14540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:06.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26147.386315 secs 2024-04-27T01:51:07.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26148.207933 secs 2024-04-27T01:51:07.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26148.207933 secs 2024-04-27T01:51:07.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26148.207933 secs 2024-04-27T01:51:08.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:07 smithi090 ceph-mon[29180]: pgmap v14541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:08.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:07 smithi119 ceph-mon[32752]: pgmap v14541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:08.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:07 smithi167 ceph-mon[32751]: pgmap v14541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:10.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:09 smithi090 ceph-mon[29180]: pgmap v14542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:10.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:09 smithi119 ceph-mon[32752]: pgmap v14542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:10.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:09 smithi167 ceph-mon[32751]: pgmap v14542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:10 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26152.386402 secs 2024-04-27T01:51:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:10 smithi119 ceph-mon[32752]: slow request 5762.591437 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:51:11.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:10 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26152.386402 secs 2024-04-27T01:51:11.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:10 smithi167 ceph-mon[32751]: slow request 5762.591437 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:51:11.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:10 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26152.386402 secs 2024-04-27T01:51:11.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:10 smithi090 ceph-mon[29180]: slow request 5762.591437 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:51:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:11 smithi119 ceph-mon[32752]: pgmap v14543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26153.208052 secs 2024-04-27T01:51:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:11 smithi167 ceph-mon[32751]: pgmap v14543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:12.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26153.208052 secs 2024-04-27T01:51:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:11 smithi090 ceph-mon[29180]: pgmap v14543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26153.208052 secs 2024-04-27T01:51:14.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:13 smithi119 ceph-mon[32752]: pgmap v14544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:14.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:13 smithi167 ceph-mon[32751]: pgmap v14544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:14.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:13 smithi090 ceph-mon[29180]: pgmap v14544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:15 smithi119 ceph-mon[32752]: pgmap v14545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26157.386488 secs 2024-04-27T01:51:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:15 smithi167 ceph-mon[32751]: pgmap v14545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26157.386488 secs 2024-04-27T01:51:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:15 smithi090 ceph-mon[29180]: pgmap v14545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26157.386488 secs 2024-04-27T01:51:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26158.208137 secs 2024-04-27T01:51:17.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26158.208137 secs 2024-04-27T01:51:17.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26158.208137 secs 2024-04-27T01:51:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:17 smithi119 ceph-mon[32752]: pgmap v14546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:18.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:17 smithi167 ceph-mon[32751]: pgmap v14546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:17 smithi090 ceph-mon[29180]: pgmap v14546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:19.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:51:19.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:51:19.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:51:19.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:51:19.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:51:19.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:51:19.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:51:19.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:51:19.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:51:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:19 smithi119 ceph-mon[32752]: pgmap v14547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:19 smithi119 ceph-mon[32752]: pgmap v14548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:51:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:19 smithi119 ceph-mon[32752]: pgmap v14549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:20.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:51:20.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:19 smithi167 ceph-mon[32751]: pgmap v14547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:20.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:19 smithi167 ceph-mon[32751]: pgmap v14548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:51:20.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:19 smithi167 ceph-mon[32751]: pgmap v14549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:20.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:51:20.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:19 smithi090 ceph-mon[29180]: pgmap v14547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:20.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:19 smithi090 ceph-mon[29180]: pgmap v14548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:51:20.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:19 smithi090 ceph-mon[29180]: pgmap v14549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:20.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:51:21.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26162.386572 secs 2024-04-27T01:51:21.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26162.386572 secs 2024-04-27T01:51:21.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26162.386572 secs 2024-04-27T01:51:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:21 smithi119 ceph-mon[32752]: pgmap v14550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26163.208277 secs 2024-04-27T01:51:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:21 smithi167 ceph-mon[32751]: pgmap v14550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26163.208277 secs 2024-04-27T01:51:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:21 smithi090 ceph-mon[29180]: pgmap v14550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26163.208277 secs 2024-04-27T01:51:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:23 smithi119 ceph-mon[32752]: pgmap v14551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:51:24.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:23 smithi167 ceph-mon[32751]: pgmap v14551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:51:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:23 smithi090 ceph-mon[29180]: pgmap v14551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:51:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:25 smithi119 ceph-mon[32752]: pgmap v14552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26167.386667 secs 2024-04-27T01:51:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:25 smithi167 ceph-mon[32751]: pgmap v14552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:26.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26167.386667 secs 2024-04-27T01:51:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:25 smithi090 ceph-mon[29180]: pgmap v14552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26167.386667 secs 2024-04-27T01:51:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26168.208342 secs 2024-04-27T01:51:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26168.208342 secs 2024-04-27T01:51:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26168.208342 secs 2024-04-27T01:51:28.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:27 smithi119 ceph-mon[32752]: pgmap v14553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:28.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:27 smithi167 ceph-mon[32751]: pgmap v14553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:28.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:27 smithi090 ceph-mon[29180]: pgmap v14553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:51:30.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:29 smithi119 ceph-mon[32752]: pgmap v14554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:51:30.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:29 smithi167 ceph-mon[32751]: pgmap v14554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:51:30.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:29 smithi090 ceph-mon[29180]: pgmap v14554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:51:31.201 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26172.386746 secs 2024-04-27T01:51:31.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26172.386746 secs 2024-04-27T01:51:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26172.386746 secs 2024-04-27T01:51:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:31 smithi090 ceph-mon[29180]: pgmap v14555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:32.108 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26173.208463 secs 2024-04-27T01:51:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:31 smithi119 ceph-mon[32752]: pgmap v14555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26173.208463 secs 2024-04-27T01:51:32.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:31 smithi167 ceph-mon[32751]: pgmap v14555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:32.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26173.208463 secs 2024-04-27T01:51:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:33 smithi119 ceph-mon[32752]: pgmap v14556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:33 smithi167 ceph-mon[32751]: pgmap v14556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:33 smithi090 ceph-mon[29180]: pgmap v14556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:35 smithi119 ceph-mon[32752]: pgmap v14557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26177.386826 secs 2024-04-27T01:51:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:35 smithi167 ceph-mon[32751]: pgmap v14557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:36.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26177.386826 secs 2024-04-27T01:51:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:35 smithi090 ceph-mon[29180]: pgmap v14557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26177.386826 secs 2024-04-27T01:51:37.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26178.208589 secs 2024-04-27T01:51:37.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26178.208589 secs 2024-04-27T01:51:37.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26178.208589 secs 2024-04-27T01:51:38.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:37 smithi119 ceph-mon[32752]: pgmap v14558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:38.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:37 smithi167 ceph-mon[32751]: pgmap v14558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:38.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:37 smithi090 ceph-mon[29180]: pgmap v14558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:39 smithi119 ceph-mon[32752]: pgmap v14559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:40.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:39 smithi167 ceph-mon[32751]: pgmap v14559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:40.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:39 smithi090 ceph-mon[29180]: pgmap v14559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26182.386883 secs 2024-04-27T01:51:41.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26182.386883 secs 2024-04-27T01:51:41.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26182.386883 secs 2024-04-27T01:51:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:41 smithi119 ceph-mon[32752]: pgmap v14560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26183.208663 secs 2024-04-27T01:51:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:41 smithi167 ceph-mon[32751]: pgmap v14560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26183.208663 secs 2024-04-27T01:51:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:41 smithi090 ceph-mon[29180]: pgmap v14560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26183.208663 secs 2024-04-27T01:51:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:43 smithi119 ceph-mon[32752]: pgmap v14561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:44.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:43 smithi167 ceph-mon[32751]: pgmap v14561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:44.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:43 smithi090 ceph-mon[29180]: pgmap v14561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:45 smithi119 ceph-mon[32752]: pgmap v14562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26187.386978 secs 2024-04-27T01:51:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:45 smithi167 ceph-mon[32751]: pgmap v14562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:46.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26187.386978 secs 2024-04-27T01:51:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:45 smithi090 ceph-mon[29180]: pgmap v14562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26187.386978 secs 2024-04-27T01:51:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26188.208793 secs 2024-04-27T01:51:47.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26188.208793 secs 2024-04-27T01:51:47.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26188.208793 secs 2024-04-27T01:51:48.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:47 smithi119 ceph-mon[32752]: pgmap v14563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:48.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:47 smithi167 ceph-mon[32751]: pgmap v14563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:48.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:47 smithi090 ceph-mon[29180]: pgmap v14563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:50.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:49 smithi119 ceph-mon[32752]: pgmap v14564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:50.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:49 smithi167 ceph-mon[32751]: pgmap v14564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:50.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:49 smithi090 ceph-mon[29180]: pgmap v14564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26192.387064 secs 2024-04-27T01:51:51.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26192.387064 secs 2024-04-27T01:51:51.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26192.387064 secs 2024-04-27T01:51:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:52 smithi119 ceph-mon[32752]: pgmap v14565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:52 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26193.208887 secs 2024-04-27T01:51:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:52 smithi167 ceph-mon[32751]: pgmap v14565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:52.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:52 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26193.208887 secs 2024-04-27T01:51:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:52 smithi090 ceph-mon[29180]: pgmap v14565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:52 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26193.208887 secs 2024-04-27T01:51:54.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:54 smithi119 ceph-mon[32752]: pgmap v14566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:54.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:54 smithi167 ceph-mon[32751]: pgmap v14566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:54.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:54 smithi090 ceph-mon[29180]: pgmap v14566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:56 smithi119 ceph-mon[32752]: pgmap v14567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26197.387123 secs 2024-04-27T01:51:56.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:56 smithi167 ceph-mon[32751]: pgmap v14567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:56.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26197.387123 secs 2024-04-27T01:51:56.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:56 smithi090 ceph-mon[29180]: pgmap v14567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:56.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26197.387123 secs 2024-04-27T01:51:57.259 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:57 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26198.208998 secs 2024-04-27T01:51:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:57 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26198.208998 secs 2024-04-27T01:51:57.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:57 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26198.208998 secs 2024-04-27T01:51:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:58 smithi119 ceph-mon[32752]: pgmap v14568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:58.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:58 smithi167 ceph-mon[32751]: pgmap v14568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:58.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:58 smithi090 ceph-mon[29180]: pgmap v14568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:51:59.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:51:59 smithi119 ceph-mon[32752]: pgmap v14569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:59.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:51:59 smithi167 ceph-mon[32751]: pgmap v14569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:51:59.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:51:59 smithi090 ceph-mon[29180]: pgmap v14569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:01.102 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26202.387208 secs 2024-04-27T01:52:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26202.387208 secs 2024-04-27T01:52:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26202.387208 secs 2024-04-27T01:52:02.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:01 smithi090 ceph-mon[29180]: pgmap v14570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:02.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:01 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26203.209126 secs 2024-04-27T01:52:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:01 smithi119 ceph-mon[32752]: pgmap v14570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:01 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26203.209126 secs 2024-04-27T01:52:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:01 smithi167 ceph-mon[32751]: pgmap v14570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:01 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26203.209126 secs 2024-04-27T01:52:04.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:03 smithi119 ceph-mon[32752]: pgmap v14571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:04.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:03 smithi167 ceph-mon[32751]: pgmap v14571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:04.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:03 smithi090 ceph-mon[29180]: pgmap v14571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:05 smithi119 ceph-mon[32752]: pgmap v14572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26207.387294 secs 2024-04-27T01:52:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:05 smithi167 ceph-mon[32751]: pgmap v14572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26207.387294 secs 2024-04-27T01:52:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:05 smithi090 ceph-mon[29180]: pgmap v14572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26207.387294 secs 2024-04-27T01:52:07.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26208.209222 secs 2024-04-27T01:52:07.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26208.209222 secs 2024-04-27T01:52:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26208.209222 secs 2024-04-27T01:52:08.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:07 smithi119 ceph-mon[32752]: pgmap v14573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:08.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:07 smithi167 ceph-mon[32751]: pgmap v14573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:07 smithi090 ceph-mon[29180]: pgmap v14573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:52:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:52:08.013+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:52:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:52:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:52:08.013+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:52:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:52:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:52:08.013+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:52:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:52:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:52:08.013+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:52:08.358 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:52:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:52:08.013+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:52:09.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:09 smithi090 ceph-mon[29180]: pgmap v14574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:10.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:09 smithi119 ceph-mon[32752]: pgmap v14574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:10.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:09 smithi167 ceph-mon[32751]: pgmap v14574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:11.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26212.387386 secs 2024-04-27T01:52:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26212.387386 secs 2024-04-27T01:52:11.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26212.387386 secs 2024-04-27T01:52:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:11 smithi090 ceph-mon[29180]: pgmap v14575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26213.209337 secs 2024-04-27T01:52:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:11 smithi119 ceph-mon[32752]: pgmap v14575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26213.209337 secs 2024-04-27T01:52:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:11 smithi167 ceph-mon[32751]: pgmap v14575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26213.209337 secs 2024-04-27T01:52:14.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:13 smithi119 ceph-mon[32752]: pgmap v14576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:14.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:13 smithi167 ceph-mon[32751]: pgmap v14576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:14.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:13 smithi090 ceph-mon[29180]: pgmap v14576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:15 smithi119 ceph-mon[32752]: pgmap v14577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26217.387446 secs 2024-04-27T01:52:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:15 smithi167 ceph-mon[32751]: pgmap v14577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26217.387446 secs 2024-04-27T01:52:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:15 smithi090 ceph-mon[29180]: pgmap v14577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26217.387446 secs 2024-04-27T01:52:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26218.209476 secs 2024-04-27T01:52:17.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26218.209476 secs 2024-04-27T01:52:17.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26218.209476 secs 2024-04-27T01:52:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:17 smithi119 ceph-mon[32752]: pgmap v14578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:18.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:17 smithi167 ceph-mon[32751]: pgmap v14578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:17 smithi090 ceph-mon[29180]: pgmap v14578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:20.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:19 smithi167 ceph-mon[32751]: pgmap v14579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:52:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.050 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:19 smithi090 ceph-mon[29180]: pgmap v14579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:20.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:52:20.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:19 smithi119 ceph-mon[32752]: pgmap v14579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T01:52:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:52:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:21.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26222.387509 secs 2024-04-27T01:52:21.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26222.387509 secs 2024-04-27T01:52:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26222.387509 secs 2024-04-27T01:52:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: pgmap v14580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:52:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26223.209568 secs 2024-04-27T01:52:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: pgmap v14580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:52:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26223.209568 secs 2024-04-27T01:52:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: pgmap v14580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:52:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26223.209568 secs 2024-04-27T01:52:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:22.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:52:23.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:52:23.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:52:23.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:52:23.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:52:23.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: pgmap v14581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-27T01:52:23.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: pgmap v14582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:52:23.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:52:23.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:52:23.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:52:23.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:52:23.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:52:23.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: pgmap v14581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-27T01:52:23.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: pgmap v14582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:52:23.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T01:52:23.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T01:52:23.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T01:52:23.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:23.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T01:52:23.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T01:52:23.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T01:52:23.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:52:23.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:52:23.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: pgmap v14581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 942 B/s rd, 1 op/s 2024-04-27T01:52:23.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: pgmap v14582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:52:23.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:52:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:25 smithi119 ceph-mon[32752]: pgmap v14583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26227.387594 secs 2024-04-27T01:52:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:25 smithi167 ceph-mon[32751]: pgmap v14583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26227.387594 secs 2024-04-27T01:52:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:25 smithi090 ceph-mon[29180]: pgmap v14583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26227.387594 secs 2024-04-27T01:52:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26228.209642 secs 2024-04-27T01:52:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26228.209642 secs 2024-04-27T01:52:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26228.209642 secs 2024-04-27T01:52:28.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:27 smithi119 ceph-mon[32752]: pgmap v14584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:28.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:27 smithi167 ceph-mon[32751]: pgmap v14584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:28.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:27 smithi090 ceph-mon[29180]: pgmap v14584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:30.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:29 smithi119 ceph-mon[32752]: pgmap v14585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:30.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:29 smithi167 ceph-mon[32751]: pgmap v14585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:30.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:29 smithi090 ceph-mon[29180]: pgmap v14585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:31.200 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26232.387680 secs 2024-04-27T01:52:31.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26232.387680 secs 2024-04-27T01:52:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26232.387680 secs 2024-04-27T01:52:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:31 smithi119 ceph-mon[32752]: pgmap v14586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26233.209760 secs 2024-04-27T01:52:32.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:31 smithi167 ceph-mon[32751]: pgmap v14586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:32.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26233.209760 secs 2024-04-27T01:52:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:31 smithi090 ceph-mon[29180]: pgmap v14586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T01:52:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26233.209760 secs 2024-04-27T01:52:34.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:33 smithi090 ceph-mon[29180]: pgmap v14587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:52:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:33 smithi119 ceph-mon[32752]: pgmap v14587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:52:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:33 smithi167 ceph-mon[32751]: pgmap v14587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:52:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:35 smithi119 ceph-mon[32752]: pgmap v14588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:35 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26237.387766 secs 2024-04-27T01:52:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:35 smithi119 ceph-mon[32752]: slow request 23043.524457 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:52:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:35 smithi167 ceph-mon[32751]: pgmap v14588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:35 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26237.387766 secs 2024-04-27T01:52:36.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:35 smithi167 ceph-mon[32751]: slow request 23043.524457 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:52:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:35 smithi090 ceph-mon[29180]: pgmap v14588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:35 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26237.387766 secs 2024-04-27T01:52:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:35 smithi090 ceph-mon[29180]: slow request 23043.524457 seconds old, received at 2024-04-26T19:28:31.616713+0000: client_request(client.26420:1 getattr pAsLsXsFs #0x1 2024-04-26T19:28:31.612604+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:52:37.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26238.209859 secs 2024-04-27T01:52:37.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26238.209859 secs 2024-04-27T01:52:37.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26238.209859 secs 2024-04-27T01:52:38.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:37 smithi119 ceph-mon[32752]: pgmap v14589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:38.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:37 smithi167 ceph-mon[32751]: pgmap v14589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:38.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:37 smithi090 ceph-mon[29180]: pgmap v14589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:39 smithi119 ceph-mon[32752]: pgmap v14590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:40.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:39 smithi167 ceph-mon[32751]: pgmap v14590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:40.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:39 smithi090 ceph-mon[29180]: pgmap v14590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26242.387851 secs 2024-04-27T01:52:41.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26242.387851 secs 2024-04-27T01:52:41.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26242.387851 secs 2024-04-27T01:52:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:41 smithi119 ceph-mon[32752]: pgmap v14591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26243.209924 secs 2024-04-27T01:52:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:41 smithi167 ceph-mon[32751]: pgmap v14591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26243.209924 secs 2024-04-27T01:52:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:41 smithi090 ceph-mon[29180]: pgmap v14591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26243.209924 secs 2024-04-27T01:52:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:43 smithi119 ceph-mon[32752]: pgmap v14592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:44.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:43 smithi167 ceph-mon[32751]: pgmap v14592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:44.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:43 smithi090 ceph-mon[29180]: pgmap v14592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:45 smithi119 ceph-mon[32752]: pgmap v14593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26247.387944 secs 2024-04-27T01:52:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:45 smithi167 ceph-mon[32751]: pgmap v14593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26247.387944 secs 2024-04-27T01:52:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:45 smithi090 ceph-mon[29180]: pgmap v14593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26247.387944 secs 2024-04-27T01:52:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26248.210035 secs 2024-04-27T01:52:47.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26248.210035 secs 2024-04-27T01:52:47.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26248.210035 secs 2024-04-27T01:52:48.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:47 smithi119 ceph-mon[32752]: pgmap v14594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:48.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:47 smithi167 ceph-mon[32751]: pgmap v14594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:48.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:47 smithi090 ceph-mon[29180]: pgmap v14594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:50.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:49 smithi119 ceph-mon[32752]: pgmap v14595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:50.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:49 smithi167 ceph-mon[32751]: pgmap v14595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:49 smithi090 ceph-mon[29180]: pgmap v14595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:51.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26252.388033 secs 2024-04-27T01:52:51.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26252.388033 secs 2024-04-27T01:52:51.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26252.388033 secs 2024-04-27T01:52:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:51 smithi119 ceph-mon[32752]: pgmap v14596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26253.210173 secs 2024-04-27T01:52:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:51 smithi167 ceph-mon[32751]: pgmap v14596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26253.210173 secs 2024-04-27T01:52:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:51 smithi090 ceph-mon[29180]: pgmap v14596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26253.210173 secs 2024-04-27T01:52:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:53 smithi119 ceph-mon[32752]: pgmap v14597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:54.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:53 smithi167 ceph-mon[32751]: pgmap v14597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:54.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:53 smithi090 ceph-mon[29180]: pgmap v14597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:55 smithi119 ceph-mon[32752]: pgmap v14598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:55 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26257.388121 secs 2024-04-27T01:52:56.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:55 smithi167 ceph-mon[32751]: pgmap v14598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:56.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:55 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26257.388121 secs 2024-04-27T01:52:56.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:55 smithi090 ceph-mon[29180]: pgmap v14598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:52:56.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:55 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26257.388121 secs 2024-04-27T01:52:57.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26258.210303 secs 2024-04-27T01:52:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26258.210303 secs 2024-04-27T01:52:57.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26258.210303 secs 2024-04-27T01:52:58.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:57 smithi119 ceph-mon[32752]: pgmap v14599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:58.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:57 smithi167 ceph-mon[32751]: pgmap v14599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:52:58.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:57 smithi090 ceph-mon[29180]: pgmap v14599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:52:59 smithi119 ceph-mon[32752]: pgmap v14600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:00.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:52:59 smithi167 ceph-mon[32751]: pgmap v14600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:00.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:52:59 smithi090 ceph-mon[29180]: pgmap v14600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26262.388208 secs 2024-04-27T01:53:01.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26262.388208 secs 2024-04-27T01:53:01.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26262.388208 secs 2024-04-27T01:53:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:02 smithi119 ceph-mon[32752]: pgmap v14601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:02 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26263.210425 secs 2024-04-27T01:53:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:02 smithi167 ceph-mon[32751]: pgmap v14601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:02 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26263.210425 secs 2024-04-27T01:53:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:02 smithi090 ceph-mon[29180]: pgmap v14601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:02 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26263.210425 secs 2024-04-27T01:53:04.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:04 smithi119 ceph-mon[32752]: pgmap v14602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:04.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:04 smithi167 ceph-mon[32751]: pgmap v14602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:04.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:04 smithi090 ceph-mon[29180]: pgmap v14602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:06 smithi119 ceph-mon[32752]: pgmap v14603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26267.388298 secs 2024-04-27T01:53:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:06 smithi167 ceph-mon[32751]: pgmap v14603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26267.388298 secs 2024-04-27T01:53:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:06 smithi090 ceph-mon[29180]: pgmap v14603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26267.388298 secs 2024-04-27T01:53:07.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:07 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26268.210544 secs 2024-04-27T01:53:07.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:07 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26268.210544 secs 2024-04-27T01:53:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:07 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26268.210544 secs 2024-04-27T01:53:08.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:08 smithi119 ceph-mon[32752]: pgmap v14604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:08.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:08 smithi167 ceph-mon[32751]: pgmap v14604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:08.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:08 smithi090 ceph-mon[29180]: pgmap v14604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:09 smithi119 ceph-mon[32752]: pgmap v14605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:09.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:09 smithi167 ceph-mon[32751]: pgmap v14605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:09.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:09 smithi090 ceph-mon[29180]: pgmap v14605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:11.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26272.388385 secs 2024-04-27T01:53:11.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26272.388385 secs 2024-04-27T01:53:11.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26272.388385 secs 2024-04-27T01:53:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:11 smithi090 ceph-mon[29180]: pgmap v14606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:11 smithi090 ceph-mon[29180]: 25 slow requests, 1 included below; oldest blocked for > 26273.210664 secs 2024-04-27T01:53:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:11 smithi090 ceph-mon[29180]: slow request 362.992714 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:53:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:11 smithi119 ceph-mon[32752]: pgmap v14606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:11 smithi119 ceph-mon[32752]: 25 slow requests, 1 included below; oldest blocked for > 26273.210664 secs 2024-04-27T01:53:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:11 smithi119 ceph-mon[32752]: slow request 362.992714 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:53:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:11 smithi167 ceph-mon[32751]: pgmap v14606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:11 smithi167 ceph-mon[32751]: 25 slow requests, 1 included below; oldest blocked for > 26273.210664 secs 2024-04-27T01:53:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:11 smithi167 ceph-mon[32751]: slow request 362.992714 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:53:14.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:13 smithi119 ceph-mon[32752]: pgmap v14607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:14.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:13 smithi167 ceph-mon[32751]: pgmap v14607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:14.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:13 smithi090 ceph-mon[29180]: pgmap v14607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:15 smithi119 ceph-mon[32752]: pgmap v14608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:15 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26277.388472 secs 2024-04-27T01:53:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:15 smithi119 ceph-mon[32752]: slow request 364.265253 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:53:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:15 smithi167 ceph-mon[32751]: pgmap v14608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:15 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26277.388472 secs 2024-04-27T01:53:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:15 smithi167 ceph-mon[32751]: slow request 364.265253 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:53:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:15 smithi090 ceph-mon[29180]: pgmap v14608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:15 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26277.388472 secs 2024-04-27T01:53:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:15 smithi090 ceph-mon[29180]: slow request 364.265253 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:53:17.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26278.210744 secs 2024-04-27T01:53:17.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26278.210744 secs 2024-04-27T01:53:17.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26278.210744 secs 2024-04-27T01:53:18.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:17 smithi119 ceph-mon[32752]: pgmap v14609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:18.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:17 smithi167 ceph-mon[32751]: pgmap v14609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:17 smithi090 ceph-mon[29180]: pgmap v14609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:20.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:19 smithi119 ceph-mon[32752]: pgmap v14610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:20.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:19 smithi167 ceph-mon[32751]: pgmap v14610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:19 smithi090 ceph-mon[29180]: pgmap v14610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:21.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26282.388557 secs 2024-04-27T01:53:21.186 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26282.388557 secs 2024-04-27T01:53:21.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26282.388557 secs 2024-04-27T01:53:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:21 smithi119 ceph-mon[32752]: pgmap v14611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26283.210888 secs 2024-04-27T01:53:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:21 smithi167 ceph-mon[32751]: pgmap v14611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26283.210888 secs 2024-04-27T01:53:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:21 smithi090 ceph-mon[29180]: pgmap v14611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26283.210888 secs 2024-04-27T01:53:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:23 smithi119 ceph-mon[32752]: pgmap v14612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:53:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:24.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:23 smithi167 ceph-mon[32751]: pgmap v14612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:24.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:53:24.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:24.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:23 smithi090 ceph-mon[29180]: pgmap v14612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:53:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:25.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:53:25.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:53:25.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:25.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:53:25.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:53:25.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:25.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:53:25.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:53:25.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:53:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:25 smithi119 ceph-mon[32752]: pgmap v14613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:53:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:25 smithi119 ceph-mon[32752]: pgmap v14614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26287.388619 secs 2024-04-27T01:53:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:25 smithi167 ceph-mon[32751]: pgmap v14613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:53:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:25 smithi167 ceph-mon[32751]: pgmap v14614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26287.388619 secs 2024-04-27T01:53:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:25 smithi090 ceph-mon[29180]: pgmap v14613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:53:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:25 smithi090 ceph-mon[29180]: pgmap v14614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26287.388619 secs 2024-04-27T01:53:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26288.210953 secs 2024-04-27T01:53:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26288.210953 secs 2024-04-27T01:53:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26288.210953 secs 2024-04-27T01:53:28.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:27 smithi090 ceph-mon[29180]: pgmap v14615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:27 smithi119 ceph-mon[32752]: pgmap v14615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:28.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:27 smithi167 ceph-mon[32751]: pgmap v14615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:30.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:29 smithi119 ceph-mon[32752]: pgmap v14616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T01:53:30.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:29 smithi167 ceph-mon[32751]: pgmap v14616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T01:53:30.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:29 smithi090 ceph-mon[29180]: pgmap v14616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T01:53:31.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26292.388679 secs 2024-04-27T01:53:31.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26292.388679 secs 2024-04-27T01:53:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26292.388679 secs 2024-04-27T01:53:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:31 smithi119 ceph-mon[32752]: pgmap v14617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26293.211063 secs 2024-04-27T01:53:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:31 smithi167 ceph-mon[32751]: pgmap v14617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26293.211063 secs 2024-04-27T01:53:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:31 smithi090 ceph-mon[29180]: pgmap v14617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26293.211063 secs 2024-04-27T01:53:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:33 smithi119 ceph-mon[32752]: pgmap v14618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:33 smithi167 ceph-mon[32751]: pgmap v14618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:33 smithi090 ceph-mon[29180]: pgmap v14618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:53:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:35 smithi119 ceph-mon[32752]: pgmap v14619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:53:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26297.388741 secs 2024-04-27T01:53:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:35 smithi167 ceph-mon[32751]: pgmap v14619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:53:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26297.388741 secs 2024-04-27T01:53:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:35 smithi090 ceph-mon[29180]: pgmap v14619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:53:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26297.388741 secs 2024-04-27T01:53:37.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26298.211187 secs 2024-04-27T01:53:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26298.211187 secs 2024-04-27T01:53:37.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26298.211187 secs 2024-04-27T01:53:38.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:37 smithi119 ceph-mon[32752]: pgmap v14620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:38.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:37 smithi167 ceph-mon[32751]: pgmap v14620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:38.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:37 smithi090 ceph-mon[29180]: pgmap v14620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:40.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:39 smithi119 ceph-mon[32752]: pgmap v14621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:40.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:39 smithi167 ceph-mon[32751]: pgmap v14621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:40.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:39 smithi090 ceph-mon[29180]: pgmap v14621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26302.388799 secs 2024-04-27T01:53:41.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26302.388799 secs 2024-04-27T01:53:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26302.388799 secs 2024-04-27T01:53:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:42 smithi119 ceph-mon[32752]: pgmap v14622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:42 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26303.211322 secs 2024-04-27T01:53:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:42 smithi167 ceph-mon[32751]: pgmap v14622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:42 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26303.211322 secs 2024-04-27T01:53:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:42 smithi090 ceph-mon[29180]: pgmap v14622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:42 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26303.211322 secs 2024-04-27T01:53:44.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:44 smithi119 ceph-mon[32752]: pgmap v14623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:44.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:44 smithi167 ceph-mon[32751]: pgmap v14623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:44.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:44 smithi090 ceph-mon[29180]: pgmap v14623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:46 smithi119 ceph-mon[32752]: pgmap v14624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26307.388859 secs 2024-04-27T01:53:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:46 smithi167 ceph-mon[32751]: pgmap v14624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26307.388859 secs 2024-04-27T01:53:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:46 smithi090 ceph-mon[29180]: pgmap v14624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26307.388859 secs 2024-04-27T01:53:47.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:47 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26308.211433 secs 2024-04-27T01:53:47.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:47 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26308.211433 secs 2024-04-27T01:53:47.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:47 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26308.211433 secs 2024-04-27T01:53:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:48 smithi119 ceph-mon[32752]: pgmap v14625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:48.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:48 smithi167 ceph-mon[32751]: pgmap v14625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:48 smithi090 ceph-mon[29180]: pgmap v14625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:50 smithi119 ceph-mon[32752]: pgmap v14626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:50.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:50 smithi167 ceph-mon[32751]: pgmap v14626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:50.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:50 smithi090 ceph-mon[29180]: pgmap v14626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:51.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26312.388918 secs 2024-04-27T01:53:51.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26312.388918 secs 2024-04-27T01:53:51.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26312.388918 secs 2024-04-27T01:53:52.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:52 smithi119 ceph-mon[32752]: pgmap v14627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:52 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26313.211557 secs 2024-04-27T01:53:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:52 smithi167 ceph-mon[32751]: pgmap v14627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:52 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26313.211557 secs 2024-04-27T01:53:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:52 smithi090 ceph-mon[29180]: pgmap v14627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:52 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26313.211557 secs 2024-04-27T01:53:53.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:53 smithi119 ceph-mon[32752]: pgmap v14628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:53.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:53 smithi167 ceph-mon[32751]: pgmap v14628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:53.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:53 smithi090 ceph-mon[29180]: pgmap v14628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:55.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:55 smithi119 ceph-mon[32752]: pgmap v14629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:55.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:55 smithi167 ceph-mon[32751]: pgmap v14629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:55.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:55 smithi090 ceph-mon[29180]: pgmap v14629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:53:56.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26317.388961 secs 2024-04-27T01:53:56.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26317.388961 secs 2024-04-27T01:53:56.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26317.388961 secs 2024-04-27T01:53:57.508 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:57 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26318.211669 secs 2024-04-27T01:53:57.508 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:57 smithi090 ceph-mon[29180]: pgmap v14630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:57.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:57 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26318.211669 secs 2024-04-27T01:53:57.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:57 smithi119 ceph-mon[32752]: pgmap v14630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:57.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:57 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26318.211669 secs 2024-04-27T01:53:57.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:57 smithi167 ceph-mon[32751]: pgmap v14630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:59.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:53:59 smithi119 ceph-mon[32752]: pgmap v14631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:59.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:53:59 smithi167 ceph-mon[32751]: pgmap v14631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:53:59.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:53:59 smithi090 ceph-mon[29180]: pgmap v14631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26322.389022 secs 2024-04-27T01:54:01.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:01 smithi119 ceph-mon[32752]: pgmap v14632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:01.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26322.389022 secs 2024-04-27T01:54:01.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:01 smithi167 ceph-mon[32751]: pgmap v14632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:01.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26322.389022 secs 2024-04-27T01:54:01.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:01 smithi090 ceph-mon[29180]: pgmap v14632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:02.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:02 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26323.211786 secs 2024-04-27T01:54:02.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:02 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26323.211786 secs 2024-04-27T01:54:02.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:02 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26323.211786 secs 2024-04-27T01:54:03.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:03 smithi119 ceph-mon[32752]: pgmap v14633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:03.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:03 smithi167 ceph-mon[32751]: pgmap v14633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:03.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:03 smithi090 ceph-mon[29180]: pgmap v14633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:05.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:05 smithi119 ceph-mon[32752]: pgmap v14634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:05.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:05 smithi167 ceph-mon[32751]: pgmap v14634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:05.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:05 smithi090 ceph-mon[29180]: pgmap v14634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:06.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26327.389081 secs 2024-04-27T01:54:06.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26327.389081 secs 2024-04-27T01:54:06.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26327.389081 secs 2024-04-27T01:54:07.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:07 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26328.211914 secs 2024-04-27T01:54:07.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:07 smithi119 ceph-mon[32752]: pgmap v14635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:07.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:07 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26328.211914 secs 2024-04-27T01:54:07.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:07 smithi167 ceph-mon[32751]: pgmap v14635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:07.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:07 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26328.211914 secs 2024-04-27T01:54:07.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:07 smithi090 ceph-mon[29180]: pgmap v14635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:09.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:09 smithi119 ceph-mon[32752]: pgmap v14636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:09.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:09 smithi167 ceph-mon[32751]: pgmap v14636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:09.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:09 smithi090 ceph-mon[29180]: pgmap v14636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:11.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26332.389158 secs 2024-04-27T01:54:11.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:11 smithi119 ceph-mon[32752]: pgmap v14637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:11.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26332.389158 secs 2024-04-27T01:54:11.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:11 smithi167 ceph-mon[32751]: pgmap v14637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:11.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26332.389158 secs 2024-04-27T01:54:11.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:11 smithi090 ceph-mon[29180]: pgmap v14637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:12.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:12 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26333.212064 secs 2024-04-27T01:54:12.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:12 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26333.212064 secs 2024-04-27T01:54:12.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:12 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26333.212064 secs 2024-04-27T01:54:13.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:13 smithi119 ceph-mon[32752]: pgmap v14638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:13.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:13 smithi167 ceph-mon[32751]: pgmap v14638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:13.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:13 smithi090 ceph-mon[29180]: pgmap v14638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:15.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:15 smithi119 ceph-mon[32752]: pgmap v14639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:15.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:15 smithi167 ceph-mon[32751]: pgmap v14639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:15.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:15 smithi090 ceph-mon[29180]: pgmap v14639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:16.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26337.389217 secs 2024-04-27T01:54:16.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26337.389217 secs 2024-04-27T01:54:16.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26337.389217 secs 2024-04-27T01:54:17.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:17 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26338.212204 secs 2024-04-27T01:54:17.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:17 smithi119 ceph-mon[32752]: pgmap v14640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:17.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:17 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26338.212204 secs 2024-04-27T01:54:17.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:17 smithi167 ceph-mon[32751]: pgmap v14640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:17.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:17 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26338.212204 secs 2024-04-27T01:54:17.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:17 smithi090 ceph-mon[29180]: pgmap v14640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:19.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:19 smithi119 ceph-mon[32752]: pgmap v14641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:19.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:19 smithi167 ceph-mon[32751]: pgmap v14641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:19.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:19 smithi090 ceph-mon[29180]: pgmap v14641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26342.389306 secs 2024-04-27T01:54:21.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:21 smithi119 ceph-mon[32752]: pgmap v14642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:21.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26342.389306 secs 2024-04-27T01:54:21.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:21 smithi167 ceph-mon[32751]: pgmap v14642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:21.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26342.389306 secs 2024-04-27T01:54:21.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:21 smithi090 ceph-mon[29180]: pgmap v14642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:22 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26343.212340 secs 2024-04-27T01:54:22.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:22 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26343.212340 secs 2024-04-27T01:54:22.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:22 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26343.212340 secs 2024-04-27T01:54:23.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:23 smithi119 ceph-mon[32752]: pgmap v14643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:23.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:23 smithi167 ceph-mon[32751]: pgmap v14643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:23.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:23 smithi090 ceph-mon[29180]: pgmap v14643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:54:24.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:54:24.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:54:25.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:25 smithi119 ceph-mon[32752]: pgmap v14644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:25.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:54:25.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:54:25.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:25 smithi119 ceph-mon[32752]: pgmap v14645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:54:25.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:25 smithi119 ceph-mon[32752]: pgmap v14646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:25.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:54:25.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:25 smithi167 ceph-mon[32751]: pgmap v14644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:25.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:54:25.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:54:25.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:25 smithi167 ceph-mon[32751]: pgmap v14645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:54:25.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:25 smithi167 ceph-mon[32751]: pgmap v14646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:25.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:54:25.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:25 smithi090 ceph-mon[29180]: pgmap v14644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:25.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:54:25.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:54:25.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:25 smithi090 ceph-mon[29180]: pgmap v14645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T01:54:25.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:25 smithi090 ceph-mon[29180]: pgmap v14646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:25.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:54:26.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26347.389391 secs 2024-04-27T01:54:26.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26348.212413 secs 2024-04-27T01:54:26.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26347.389391 secs 2024-04-27T01:54:26.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26348.212413 secs 2024-04-27T01:54:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26347.389391 secs 2024-04-27T01:54:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26348.212413 secs 2024-04-27T01:54:27.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:27 smithi119 ceph-mon[32752]: pgmap v14647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:27.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:27 smithi167 ceph-mon[32751]: pgmap v14647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:27.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:27 smithi090 ceph-mon[29180]: pgmap v14647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:29 smithi119 ceph-mon[32752]: pgmap v14648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:30.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:29 smithi167 ceph-mon[32751]: pgmap v14648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:29 smithi090 ceph-mon[29180]: pgmap v14648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26352.389476 secs 2024-04-27T01:54:31.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26352.389476 secs 2024-04-27T01:54:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26352.389476 secs 2024-04-27T01:54:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:31 smithi119 ceph-mon[32752]: pgmap v14649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26353.212489 secs 2024-04-27T01:54:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:31 smithi167 ceph-mon[32751]: pgmap v14649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26353.212489 secs 2024-04-27T01:54:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:31 smithi090 ceph-mon[29180]: pgmap v14649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:54:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26353.212489 secs 2024-04-27T01:54:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:33 smithi119 ceph-mon[32752]: pgmap v14650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:54:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:33 smithi167 ceph-mon[32751]: pgmap v14650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:54:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:33 smithi090 ceph-mon[29180]: pgmap v14650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T01:54:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:36 smithi119 ceph-mon[32752]: pgmap v14651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:54:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26357.389561 secs 2024-04-27T01:54:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:35 smithi167 ceph-mon[32751]: pgmap v14651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:54:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26357.389561 secs 2024-04-27T01:54:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:35 smithi090 ceph-mon[29180]: pgmap v14651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:54:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26357.389561 secs 2024-04-27T01:54:37.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:37 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26358.212587 secs 2024-04-27T01:54:37.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:37 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26358.212587 secs 2024-04-27T01:54:37.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:37 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26358.212587 secs 2024-04-27T01:54:38.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:38 smithi119 ceph-mon[32752]: pgmap v14652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:38.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:38 smithi167 ceph-mon[32751]: pgmap v14652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:38 smithi090 ceph-mon[29180]: pgmap v14652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:40.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:40 smithi119 ceph-mon[32752]: pgmap v14653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:40.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:40 smithi167 ceph-mon[32751]: pgmap v14653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:40.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:40 smithi090 ceph-mon[29180]: pgmap v14653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:41.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26362.389645 secs 2024-04-27T01:54:41.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26362.389645 secs 2024-04-27T01:54:41.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26362.389645 secs 2024-04-27T01:54:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:42 smithi119 ceph-mon[32752]: pgmap v14654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:42 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26363.212666 secs 2024-04-27T01:54:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:42 smithi167 ceph-mon[32751]: pgmap v14654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:42 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26363.212666 secs 2024-04-27T01:54:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:42 smithi090 ceph-mon[29180]: pgmap v14654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:42 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26363.212666 secs 2024-04-27T01:54:44.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:44 smithi119 ceph-mon[32752]: pgmap v14655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:44.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:44 smithi167 ceph-mon[32751]: pgmap v14655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:44.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:44 smithi090 ceph-mon[29180]: pgmap v14655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:46 smithi119 ceph-mon[32752]: pgmap v14656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26367.389730 secs 2024-04-27T01:54:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:46 smithi167 ceph-mon[32751]: pgmap v14656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26367.389730 secs 2024-04-27T01:54:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:46 smithi090 ceph-mon[29180]: pgmap v14656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26367.389730 secs 2024-04-27T01:54:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:47 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26368.212786 secs 2024-04-27T01:54:47.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:47 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26368.212786 secs 2024-04-27T01:54:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:47 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26368.212786 secs 2024-04-27T01:54:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:48 smithi119 ceph-mon[32752]: pgmap v14657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:48.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:48 smithi167 ceph-mon[32751]: pgmap v14657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:48 smithi090 ceph-mon[29180]: pgmap v14657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:49.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:49 smithi119 ceph-mon[32752]: pgmap v14658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:49.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:49 smithi167 ceph-mon[32751]: pgmap v14658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:49 smithi090 ceph-mon[29180]: pgmap v14658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:51.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26372.389811 secs 2024-04-27T01:54:51.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26372.389811 secs 2024-04-27T01:54:51.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26372.389811 secs 2024-04-27T01:54:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:51 smithi090 ceph-mon[29180]: pgmap v14659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26373.212901 secs 2024-04-27T01:54:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:51 smithi119 ceph-mon[32752]: pgmap v14659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26373.212901 secs 2024-04-27T01:54:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:51 smithi167 ceph-mon[32751]: pgmap v14659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26373.212901 secs 2024-04-27T01:54:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:53 smithi119 ceph-mon[32752]: pgmap v14660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:54.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:53 smithi167 ceph-mon[32751]: pgmap v14660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:53 smithi090 ceph-mon[29180]: pgmap v14660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:55 smithi119 ceph-mon[32752]: pgmap v14661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:56.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:55 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26377.389866 secs 2024-04-27T01:54:56.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:55 smithi167 ceph-mon[32751]: pgmap v14661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:56.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:55 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26377.389866 secs 2024-04-27T01:54:56.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:55 smithi090 ceph-mon[29180]: pgmap v14661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:54:56.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:55 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26377.389866 secs 2024-04-27T01:54:57.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26378.212968 secs 2024-04-27T01:54:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26378.212968 secs 2024-04-27T01:54:57.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26378.212968 secs 2024-04-27T01:54:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:54:58 smithi119 ceph-mon[32752]: pgmap v14662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:58.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:54:58 smithi167 ceph-mon[32751]: pgmap v14662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:54:58.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:54:58 smithi090 ceph-mon[29180]: pgmap v14662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:00 smithi119 ceph-mon[32752]: pgmap v14663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:00.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:00 smithi167 ceph-mon[32751]: pgmap v14663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:00.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:00 smithi090 ceph-mon[29180]: pgmap v14663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26382.389947 secs 2024-04-27T01:55:01.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26382.389947 secs 2024-04-27T01:55:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26382.389947 secs 2024-04-27T01:55:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:02 smithi119 ceph-mon[32752]: pgmap v14664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:02.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:02 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26383.213104 secs 2024-04-27T01:55:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:02 smithi167 ceph-mon[32751]: pgmap v14664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:02 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26383.213104 secs 2024-04-27T01:55:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:02 smithi090 ceph-mon[29180]: pgmap v14664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:02 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26383.213104 secs 2024-04-27T01:55:04.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:04 smithi119 ceph-mon[32752]: pgmap v14665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:04.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:04 smithi167 ceph-mon[32751]: pgmap v14665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:04.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:04 smithi090 ceph-mon[29180]: pgmap v14665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:06 smithi119 ceph-mon[32752]: pgmap v14666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:06 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26387.390008 secs 2024-04-27T01:55:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:06 smithi119 ceph-mon[32752]: slow request 11521.268631 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:55:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:06 smithi167 ceph-mon[32751]: pgmap v14666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:06 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26387.390008 secs 2024-04-27T01:55:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:06 smithi167 ceph-mon[32751]: slow request 11521.268631 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:55:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:06 smithi090 ceph-mon[29180]: pgmap v14666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:06 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26387.390008 secs 2024-04-27T01:55:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:06 smithi090 ceph-mon[29180]: slow request 11521.268631 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:55:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:07 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26388.213217 secs 2024-04-27T01:55:07.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:07 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26388.213217 secs 2024-04-27T01:55:07.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:07 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26388.213217 secs 2024-04-27T01:55:08.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:08 smithi119 ceph-mon[32752]: pgmap v14667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:08.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:08 smithi167 ceph-mon[32751]: pgmap v14667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:08 smithi090 ceph-mon[29180]: pgmap v14667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:09.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:09 smithi119 ceph-mon[32752]: pgmap v14668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:09.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:09 smithi167 ceph-mon[32751]: pgmap v14668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:09.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:09 smithi090 ceph-mon[29180]: pgmap v14668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26392.390086 secs 2024-04-27T01:55:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26392.390086 secs 2024-04-27T01:55:11.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26392.390086 secs 2024-04-27T01:55:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:11 smithi090 ceph-mon[29180]: pgmap v14669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:12.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26393.213338 secs 2024-04-27T01:55:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:11 smithi119 ceph-mon[32752]: pgmap v14669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26393.213338 secs 2024-04-27T01:55:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:11 smithi167 ceph-mon[32751]: pgmap v14669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26393.213338 secs 2024-04-27T01:55:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:14 smithi119 ceph-mon[32752]: pgmap v14670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:14.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:14 smithi167 ceph-mon[32751]: pgmap v14670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:14.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:14 smithi090 ceph-mon[29180]: pgmap v14670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:16 smithi119 ceph-mon[32752]: pgmap v14671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:16.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26397.390164 secs 2024-04-27T01:55:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:16 smithi167 ceph-mon[32751]: pgmap v14671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26397.390164 secs 2024-04-27T01:55:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:16 smithi090 ceph-mon[29180]: pgmap v14671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26397.390164 secs 2024-04-27T01:55:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:17 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26398.213458 secs 2024-04-27T01:55:17.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:17 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26398.213458 secs 2024-04-27T01:55:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:17 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26398.213458 secs 2024-04-27T01:55:18.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:18 smithi119 ceph-mon[32752]: pgmap v14672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:18.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:18 smithi167 ceph-mon[32751]: pgmap v14672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:18.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:18 smithi090 ceph-mon[29180]: pgmap v14672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:20.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:20 smithi119 ceph-mon[32752]: pgmap v14673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:20.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:20 smithi167 ceph-mon[32751]: pgmap v14673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:20.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:20 smithi090 ceph-mon[29180]: pgmap v14673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26402.390255 secs 2024-04-27T01:55:21.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26402.390255 secs 2024-04-27T01:55:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26402.390255 secs 2024-04-27T01:55:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:22 smithi119 ceph-mon[32752]: pgmap v14674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:22 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26403.213519 secs 2024-04-27T01:55:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:22 smithi167 ceph-mon[32751]: pgmap v14674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:22 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26403.213519 secs 2024-04-27T01:55:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:22 smithi090 ceph-mon[29180]: pgmap v14674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:22 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26403.213519 secs 2024-04-27T01:55:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:24 smithi119 ceph-mon[32752]: pgmap v14675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:24.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:24 smithi167 ceph-mon[32751]: pgmap v14675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:24 smithi090 ceph-mon[29180]: pgmap v14675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:26 smithi119 ceph-mon[32752]: pgmap v14676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:55:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26407.390328 secs 2024-04-27T01:55:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:55:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:55:26.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:55:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:26 smithi167 ceph-mon[32751]: pgmap v14676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:55:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26407.390328 secs 2024-04-27T01:55:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:55:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:55:26.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:55:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:26 smithi090 ceph-mon[29180]: pgmap v14676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:55:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26407.390328 secs 2024-04-27T01:55:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:55:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:55:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:55:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:27 smithi119 ceph-mon[32752]: pgmap v14677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:55:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:27 smithi119 ceph-mon[32752]: pgmap v14678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:27 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26408.213626 secs 2024-04-27T01:55:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:27 smithi167 ceph-mon[32751]: pgmap v14677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:55:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:27 smithi167 ceph-mon[32751]: pgmap v14678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:27 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26408.213626 secs 2024-04-27T01:55:27.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:27 smithi090 ceph-mon[29180]: pgmap v14677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:55:27.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:27 smithi090 ceph-mon[29180]: pgmap v14678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:27.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:27 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26408.213626 secs 2024-04-27T01:55:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:29 smithi119 ceph-mon[32752]: pgmap v14679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:55:29.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:29 smithi167 ceph-mon[32751]: pgmap v14679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:55:29.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:29 smithi090 ceph-mon[29180]: pgmap v14679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:55:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:31 smithi119 ceph-mon[32752]: pgmap v14680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:31.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26412.390385 secs 2024-04-27T01:55:31.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:31 smithi167 ceph-mon[32751]: pgmap v14680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:31.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26412.390385 secs 2024-04-27T01:55:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:31 smithi090 ceph-mon[29180]: pgmap v14680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26412.390385 secs 2024-04-27T01:55:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:32 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26413.213740 secs 2024-04-27T01:55:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:32 smithi119 ceph-mon[32752]: pgmap v14681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:32 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26413.213740 secs 2024-04-27T01:55:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:32 smithi167 ceph-mon[32751]: pgmap v14681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:32 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26413.213740 secs 2024-04-27T01:55:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:32 smithi090 ceph-mon[29180]: pgmap v14681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:35.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:34 smithi119 ceph-mon[32752]: pgmap v14682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:35.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:34 smithi167 ceph-mon[32751]: pgmap v14682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:35.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:34 smithi090 ceph-mon[29180]: pgmap v14682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:55:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26417.390445 secs 2024-04-27T01:55:37.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:36 smithi119 ceph-mon[32752]: pgmap v14683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:55:37.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26418.213890 secs 2024-04-27T01:55:37.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26417.390445 secs 2024-04-27T01:55:37.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:36 smithi167 ceph-mon[32751]: pgmap v14683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:55:37.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26418.213890 secs 2024-04-27T01:55:37.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26417.390445 secs 2024-04-27T01:55:37.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:36 smithi090 ceph-mon[29180]: pgmap v14683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:55:37.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26418.213890 secs 2024-04-27T01:55:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:38 smithi119 ceph-mon[32752]: pgmap v14684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:39.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:38 smithi167 ceph-mon[32751]: pgmap v14684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:38 smithi090 ceph-mon[29180]: pgmap v14684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:41.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:40 smithi119 ceph-mon[32752]: pgmap v14685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:41.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:40 smithi167 ceph-mon[32751]: pgmap v14685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:41.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:40 smithi090 ceph-mon[29180]: pgmap v14685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:42.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26422.390503 secs 2024-04-27T01:55:42.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26423.213999 secs 2024-04-27T01:55:42.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26422.390503 secs 2024-04-27T01:55:42.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26423.213999 secs 2024-04-27T01:55:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26422.390503 secs 2024-04-27T01:55:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26423.213999 secs 2024-04-27T01:55:43.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:42 smithi119 ceph-mon[32752]: pgmap v14686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:43.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:42 smithi167 ceph-mon[32751]: pgmap v14686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:43.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:42 smithi090 ceph-mon[29180]: pgmap v14686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:45.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:44 smithi119 ceph-mon[32752]: pgmap v14687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:45.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:44 smithi167 ceph-mon[32751]: pgmap v14687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:45.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:44 smithi090 ceph-mon[29180]: pgmap v14687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:47.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26427.390552 secs 2024-04-27T01:55:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:46 smithi119 ceph-mon[32752]: pgmap v14688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:47.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26428.214114 secs 2024-04-27T01:55:47.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26427.390552 secs 2024-04-27T01:55:47.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:46 smithi167 ceph-mon[32751]: pgmap v14688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:47.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26428.214114 secs 2024-04-27T01:55:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26427.390552 secs 2024-04-27T01:55:47.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:46 smithi090 ceph-mon[29180]: pgmap v14688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:47.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26428.214114 secs 2024-04-27T01:55:49.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:48 smithi119 ceph-mon[32752]: pgmap v14689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:49.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:48 smithi167 ceph-mon[32751]: pgmap v14689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:49.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:48 smithi090 ceph-mon[29180]: pgmap v14689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:51.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:50 smithi119 ceph-mon[32752]: pgmap v14690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:51.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:50 smithi167 ceph-mon[32751]: pgmap v14690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:51.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:50 smithi090 ceph-mon[29180]: pgmap v14690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:52.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26432.390635 secs 2024-04-27T01:55:52.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26433.214229 secs 2024-04-27T01:55:52.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26432.390635 secs 2024-04-27T01:55:52.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26433.214229 secs 2024-04-27T01:55:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26432.390635 secs 2024-04-27T01:55:52.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26433.214229 secs 2024-04-27T01:55:53.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:52 smithi119 ceph-mon[32752]: pgmap v14691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:53.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:52 smithi167 ceph-mon[32751]: pgmap v14691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:53.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:52 smithi090 ceph-mon[29180]: pgmap v14691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:55.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:54 smithi119 ceph-mon[32752]: pgmap v14692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:55.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:54 smithi167 ceph-mon[32751]: pgmap v14692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:55.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:54 smithi090 ceph-mon[29180]: pgmap v14692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:55:56.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:55 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26437.390694 secs 2024-04-27T01:55:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:55 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26437.390694 secs 2024-04-27T01:55:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:55 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26437.390694 secs 2024-04-27T01:55:57.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:56 smithi167 ceph-mon[32751]: pgmap v14693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:57.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26438.214353 secs 2024-04-27T01:55:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:56 smithi090 ceph-mon[29180]: pgmap v14693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:57.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26438.214353 secs 2024-04-27T01:55:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:56 smithi119 ceph-mon[32752]: pgmap v14693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26438.214353 secs 2024-04-27T01:55:59.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:55:58 smithi167 ceph-mon[32751]: pgmap v14694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:59.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:55:58 smithi090 ceph-mon[29180]: pgmap v14694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:55:59.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:55:58 smithi119 ceph-mon[32752]: pgmap v14694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:01.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:00 smithi167 ceph-mon[32751]: pgmap v14695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:01.049 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26442.390777 secs 2024-04-27T01:56:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:00 smithi090 ceph-mon[29180]: pgmap v14695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:01.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26442.390777 secs 2024-04-27T01:56:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:00 smithi119 ceph-mon[32752]: pgmap v14695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:01.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26442.390777 secs 2024-04-27T01:56:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:01 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26443.214468 secs 2024-04-27T01:56:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:01 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26443.214468 secs 2024-04-27T01:56:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:01 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26443.214468 secs 2024-04-27T01:56:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:02 smithi090 ceph-mon[29180]: pgmap v14696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:03.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:02 smithi119 ceph-mon[32752]: pgmap v14696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:03.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:02 smithi167 ceph-mon[32751]: pgmap v14696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:04 smithi090 ceph-mon[29180]: pgmap v14697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:05.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:04 smithi119 ceph-mon[32752]: pgmap v14697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:05.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:04 smithi167 ceph-mon[32751]: pgmap v14697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:06.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26447.390862 secs 2024-04-27T01:56:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26447.390862 secs 2024-04-27T01:56:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26447.390862 secs 2024-04-27T01:56:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:06 smithi090 ceph-mon[29180]: pgmap v14698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:07.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26448.214543 secs 2024-04-27T01:56:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:06 smithi119 ceph-mon[32752]: pgmap v14698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:07.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26448.214543 secs 2024-04-27T01:56:07.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:06 smithi167 ceph-mon[32751]: pgmap v14698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:07.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26448.214543 secs 2024-04-27T01:56:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:08 smithi090 ceph-mon[29180]: pgmap v14699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:09.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:08 smithi119 ceph-mon[32752]: pgmap v14699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:09.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:08 smithi167 ceph-mon[32751]: pgmap v14699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:10 smithi090 ceph-mon[29180]: pgmap v14700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:11.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26452.390949 secs 2024-04-27T01:56:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:10 smithi119 ceph-mon[32752]: pgmap v14700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26452.390949 secs 2024-04-27T01:56:11.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:10 smithi167 ceph-mon[32751]: pgmap v14700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:11.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26452.390949 secs 2024-04-27T01:56:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26453.214650 secs 2024-04-27T01:56:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26453.214650 secs 2024-04-27T01:56:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26453.214650 secs 2024-04-27T01:56:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:12 smithi090 ceph-mon[29180]: pgmap v14701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:13.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:12 smithi119 ceph-mon[32752]: pgmap v14701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:13.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:12 smithi167 ceph-mon[32751]: pgmap v14701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:14 smithi090 ceph-mon[29180]: pgmap v14702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:15.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:14 smithi119 ceph-mon[32752]: pgmap v14702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:15.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:14 smithi167 ceph-mon[32751]: pgmap v14702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:16.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26457.391034 secs 2024-04-27T01:56:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26457.391034 secs 2024-04-27T01:56:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26457.391034 secs 2024-04-27T01:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:16 smithi090 ceph-mon[29180]: pgmap v14703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26458.214770 secs 2024-04-27T01:56:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:16 smithi119 ceph-mon[32752]: pgmap v14703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26458.214770 secs 2024-04-27T01:56:17.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:16 smithi167 ceph-mon[32751]: pgmap v14703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:17.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26458.214770 secs 2024-04-27T01:56:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:18 smithi090 ceph-mon[29180]: pgmap v14704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:19.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:18 smithi119 ceph-mon[32752]: pgmap v14704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:19.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:18 smithi167 ceph-mon[32751]: pgmap v14704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:20 smithi090 ceph-mon[29180]: pgmap v14705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:21.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26462.391116 secs 2024-04-27T01:56:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:20 smithi119 ceph-mon[32752]: pgmap v14705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:21.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26462.391116 secs 2024-04-27T01:56:21.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:20 smithi167 ceph-mon[32751]: pgmap v14705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:21.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26462.391116 secs 2024-04-27T01:56:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26463.214911 secs 2024-04-27T01:56:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26463.214911 secs 2024-04-27T01:56:22.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26463.214911 secs 2024-04-27T01:56:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:22 smithi090 ceph-mon[29180]: pgmap v14706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:23.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:22 smithi119 ceph-mon[32752]: pgmap v14706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:23.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:22 smithi167 ceph-mon[32751]: pgmap v14706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:24 smithi090 ceph-mon[29180]: pgmap v14707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:25.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:24 smithi119 ceph-mon[32752]: pgmap v14707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:25.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:24 smithi167 ceph-mon[32751]: pgmap v14707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26467.391198 secs 2024-04-27T01:56:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:56:26.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26467.391198 secs 2024-04-27T01:56:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:56:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26467.391198 secs 2024-04-27T01:56:26.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:26 smithi090 ceph-mon[29180]: pgmap v14708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26468.215005 secs 2024-04-27T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:56:27.258 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:56:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:26 smithi119 ceph-mon[32752]: pgmap v14708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26468.215005 secs 2024-04-27T01:56:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:56:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:56:27.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:56:27.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:26 smithi167 ceph-mon[32751]: pgmap v14708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26468.215005 secs 2024-04-27T01:56:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:56:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:56:27.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:56:28.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:27 smithi090 ceph-mon[29180]: pgmap v14709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:56:28.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:27 smithi090 ceph-mon[29180]: pgmap v14710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:27 smithi119 ceph-mon[32752]: pgmap v14709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:56:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:27 smithi119 ceph-mon[32752]: pgmap v14710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:28.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:27 smithi167 ceph-mon[32751]: pgmap v14709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:56:28.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:27 smithi167 ceph-mon[32751]: pgmap v14710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:29 smithi119 ceph-mon[32752]: pgmap v14711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:30.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:29 smithi167 ceph-mon[32751]: pgmap v14711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:29 smithi090 ceph-mon[29180]: pgmap v14711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:31.200 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26472.391301 secs 2024-04-27T01:56:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26472.391301 secs 2024-04-27T01:56:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26472.391301 secs 2024-04-27T01:56:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:31 smithi119 ceph-mon[32752]: pgmap v14712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:32.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26473.215123 secs 2024-04-27T01:56:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:31 smithi167 ceph-mon[32751]: pgmap v14712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26473.215123 secs 2024-04-27T01:56:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:31 smithi090 ceph-mon[29180]: pgmap v14712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26473.215123 secs 2024-04-27T01:56:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:33 smithi119 ceph-mon[32752]: pgmap v14713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:56:34.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:33 smithi167 ceph-mon[32751]: pgmap v14713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:56:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:33 smithi090 ceph-mon[29180]: pgmap v14713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:56:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:35 smithi119 ceph-mon[32752]: pgmap v14714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26477.391379 secs 2024-04-27T01:56:36.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:35 smithi167 ceph-mon[32751]: pgmap v14714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:36.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26477.391379 secs 2024-04-27T01:56:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:35 smithi090 ceph-mon[29180]: pgmap v14714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:56:36.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26477.391379 secs 2024-04-27T01:56:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26478.215195 secs 2024-04-27T01:56:37.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26478.215195 secs 2024-04-27T01:56:37.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26478.215195 secs 2024-04-27T01:56:38.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:37 smithi119 ceph-mon[32752]: pgmap v14715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:56:38.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:37 smithi167 ceph-mon[32751]: pgmap v14715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:56:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:37 smithi090 ceph-mon[29180]: pgmap v14715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:56:40.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:39 smithi119 ceph-mon[32752]: pgmap v14716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:40.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:39 smithi167 ceph-mon[32751]: pgmap v14716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:39 smithi090 ceph-mon[29180]: pgmap v14716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:41.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26482.391463 secs 2024-04-27T01:56:41.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26482.391463 secs 2024-04-27T01:56:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26482.391463 secs 2024-04-27T01:56:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:41 smithi119 ceph-mon[32752]: pgmap v14717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:42.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26483.215304 secs 2024-04-27T01:56:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:41 smithi167 ceph-mon[32751]: pgmap v14717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26483.215304 secs 2024-04-27T01:56:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:41 smithi090 ceph-mon[29180]: pgmap v14717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26483.215304 secs 2024-04-27T01:56:44.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:43 smithi119 ceph-mon[32752]: pgmap v14718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:44.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:43 smithi167 ceph-mon[32751]: pgmap v14718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:43 smithi090 ceph-mon[29180]: pgmap v14718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:45 smithi119 ceph-mon[32752]: pgmap v14719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:46.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26487.391549 secs 2024-04-27T01:56:46.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:45 smithi167 ceph-mon[32751]: pgmap v14719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:46.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26487.391549 secs 2024-04-27T01:56:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:45 smithi090 ceph-mon[29180]: pgmap v14719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:46.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26487.391549 secs 2024-04-27T01:56:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26488.215421 secs 2024-04-27T01:56:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26488.215421 secs 2024-04-27T01:56:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26488.215421 secs 2024-04-27T01:56:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:47 smithi119 ceph-mon[32752]: pgmap v14720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:48.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:47 smithi167 ceph-mon[32751]: pgmap v14720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:47 smithi090 ceph-mon[29180]: pgmap v14720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:49 smithi119 ceph-mon[32752]: pgmap v14721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:50.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:49 smithi167 ceph-mon[32751]: pgmap v14721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:49 smithi090 ceph-mon[29180]: pgmap v14721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:51.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26492.391633 secs 2024-04-27T01:56:51.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26492.391633 secs 2024-04-27T01:56:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26492.391633 secs 2024-04-27T01:56:52.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:51 smithi119 ceph-mon[32752]: pgmap v14722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:52.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26493.215549 secs 2024-04-27T01:56:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:51 smithi167 ceph-mon[32751]: pgmap v14722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:52.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26493.215549 secs 2024-04-27T01:56:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:51 smithi090 ceph-mon[29180]: pgmap v14722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:52.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26493.215549 secs 2024-04-27T01:56:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:53 smithi119 ceph-mon[32752]: pgmap v14723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:54.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:53 smithi167 ceph-mon[32751]: pgmap v14723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:54.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:53 smithi090 ceph-mon[29180]: pgmap v14723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:55 smithi119 ceph-mon[32752]: pgmap v14724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:55 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26497.391689 secs 2024-04-27T01:56:56.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:55 smithi167 ceph-mon[32751]: pgmap v14724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:56.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:55 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26497.391689 secs 2024-04-27T01:56:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:55 smithi090 ceph-mon[29180]: pgmap v14724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:56:56.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:55 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26497.391689 secs 2024-04-27T01:56:57.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26498.215691 secs 2024-04-27T01:56:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26498.215691 secs 2024-04-27T01:56:57.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26498.215691 secs 2024-04-27T01:56:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:57 smithi119 ceph-mon[32752]: pgmap v14725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:58.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:57 smithi167 ceph-mon[32751]: pgmap v14725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:56:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:57 smithi090 ceph-mon[29180]: pgmap v14725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:56:59 smithi119 ceph-mon[32752]: pgmap v14726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:00.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:56:59 smithi167 ceph-mon[32751]: pgmap v14726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:56:59 smithi090 ceph-mon[29180]: pgmap v14726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:01.200 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26502.391751 secs 2024-04-27T01:57:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26502.391751 secs 2024-04-27T01:57:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26502.391751 secs 2024-04-27T01:57:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:01 smithi119 ceph-mon[32752]: pgmap v14727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:01 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26503.215784 secs 2024-04-27T01:57:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:01 smithi167 ceph-mon[32751]: pgmap v14727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:02.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:01 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26503.215784 secs 2024-04-27T01:57:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:01 smithi090 ceph-mon[29180]: pgmap v14727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:02.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:01 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26503.215784 secs 2024-04-27T01:57:04.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:03 smithi119 ceph-mon[32752]: pgmap v14728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:04.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:03 smithi167 ceph-mon[32751]: pgmap v14728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:03 smithi090 ceph-mon[29180]: pgmap v14728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:05 smithi119 ceph-mon[32752]: pgmap v14729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:06.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26507.391809 secs 2024-04-27T01:57:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:05 smithi167 ceph-mon[32751]: pgmap v14729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:06.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26507.391809 secs 2024-04-27T01:57:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:05 smithi090 ceph-mon[29180]: pgmap v14729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:06.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26507.391809 secs 2024-04-27T01:57:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26508.215899 secs 2024-04-27T01:57:07.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26508.215899 secs 2024-04-27T01:57:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26508.215899 secs 2024-04-27T01:57:08.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:07 smithi119 ceph-mon[32752]: pgmap v14730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:08.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:07 smithi167 ceph-mon[32751]: pgmap v14730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:07 smithi090 ceph-mon[29180]: pgmap v14730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T01:57:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:09 smithi119 ceph-mon[32752]: pgmap v14731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:10.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:09 smithi167 ceph-mon[32751]: pgmap v14731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:09 smithi090 ceph-mon[29180]: pgmap v14731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26512.391869 secs 2024-04-27T01:57:11.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26512.391869 secs 2024-04-27T01:57:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26512.391869 secs 2024-04-27T01:57:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:11 smithi119 ceph-mon[32752]: pgmap v14732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-27T01:57:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26513.216013 secs 2024-04-27T01:57:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:11 smithi167 ceph-mon[32751]: pgmap v14732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-27T01:57:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26513.216013 secs 2024-04-27T01:57:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:11 smithi090 ceph-mon[29180]: pgmap v14732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-27T01:57:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26513.216013 secs 2024-04-27T01:57:14.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:13 smithi090 ceph-mon[29180]: pgmap v14733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-27T01:57:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:13 smithi119 ceph-mon[32752]: pgmap v14733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-27T01:57:14.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:13 smithi167 ceph-mon[32751]: pgmap v14733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 341 B/s wr, 1 op/s 2024-04-27T01:57:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:15 smithi119 ceph-mon[32752]: pgmap v14734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26517.391953 secs 2024-04-27T01:57:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:15 smithi167 ceph-mon[32751]: pgmap v14734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26517.391953 secs 2024-04-27T01:57:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:15 smithi090 ceph-mon[29180]: pgmap v14734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26517.391953 secs 2024-04-27T01:57:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26518.216078 secs 2024-04-27T01:57:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26518.216078 secs 2024-04-27T01:57:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26518.216078 secs 2024-04-27T01:57:18.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:17 smithi119 ceph-mon[32752]: pgmap v14735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:18.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:17 smithi167 ceph-mon[32751]: pgmap v14735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:17 smithi090 ceph-mon[29180]: pgmap v14735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:20.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:20 smithi119 ceph-mon[32752]: pgmap v14736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:20.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:20 smithi167 ceph-mon[32751]: pgmap v14736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:20 smithi090 ceph-mon[29180]: pgmap v14736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T01:57:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26522.392034 secs 2024-04-27T01:57:21.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26522.392034 secs 2024-04-27T01:57:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26522.392034 secs 2024-04-27T01:57:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:22 smithi119 ceph-mon[32752]: pgmap v14737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:57:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:22 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26523.216193 secs 2024-04-27T01:57:22.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:22 smithi167 ceph-mon[32751]: pgmap v14737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:57:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:22 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26523.216193 secs 2024-04-27T01:57:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:22 smithi090 ceph-mon[29180]: pgmap v14737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:57:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:22 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26523.216193 secs 2024-04-27T01:57:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:24 smithi119 ceph-mon[32752]: pgmap v14738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:57:24.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:24 smithi167 ceph-mon[32751]: pgmap v14738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:57:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:24 smithi090 ceph-mon[29180]: pgmap v14738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-27T01:57:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:26 smithi119 ceph-mon[32752]: pgmap v14739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:57:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26527.392126 secs 2024-04-27T01:57:26.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:26 smithi167 ceph-mon[32751]: pgmap v14739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:57:26.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26527.392126 secs 2024-04-27T01:57:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:26 smithi090 ceph-mon[29180]: pgmap v14739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:57:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26527.392126 secs 2024-04-27T01:57:27.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:27 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26528.216310 secs 2024-04-27T01:57:27.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:57:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:27 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26528.216310 secs 2024-04-27T01:57:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:57:27.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:27 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26528.216310 secs 2024-04-27T01:57:27.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:57:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:28 smithi119 ceph-mon[32752]: pgmap v14740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:57:28.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:57:28.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:57:28.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:28 smithi167 ceph-mon[32751]: pgmap v14740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:28.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:57:28.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:57:28.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:57:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:28 smithi090 ceph-mon[29180]: pgmap v14740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:57:28.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:57:28.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:57:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:29 smithi119 ceph-mon[32752]: pgmap v14741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:57:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:29 smithi119 ceph-mon[32752]: pgmap v14742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:57:29.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:29 smithi167 ceph-mon[32751]: pgmap v14741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:57:29.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:29 smithi167 ceph-mon[32751]: pgmap v14742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:57:29.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:29 smithi090 ceph-mon[29180]: pgmap v14741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:57:29.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:29 smithi090 ceph-mon[29180]: pgmap v14742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:57:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:31 smithi119 ceph-mon[32752]: pgmap v14743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26532.392178 secs 2024-04-27T01:57:31.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:31 smithi167 ceph-mon[32751]: pgmap v14743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:31.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26532.392178 secs 2024-04-27T01:57:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:31 smithi090 ceph-mon[29180]: pgmap v14743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:31.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26532.392178 secs 2024-04-27T01:57:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:32 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26533.216441 secs 2024-04-27T01:57:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:32 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26533.216441 secs 2024-04-27T01:57:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:32 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26533.216441 secs 2024-04-27T01:57:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:33 smithi119 ceph-mon[32752]: pgmap v14744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:33.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:33 smithi167 ceph-mon[32751]: pgmap v14744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:33.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:33 smithi090 ceph-mon[29180]: pgmap v14744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:35 smithi119 ceph-mon[32752]: pgmap v14745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T01:57:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:35 smithi167 ceph-mon[32751]: pgmap v14745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T01:57:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:35 smithi090 ceph-mon[29180]: pgmap v14745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T01:57:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26537.392258 secs 2024-04-27T01:57:36.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26537.392258 secs 2024-04-27T01:57:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26537.392258 secs 2024-04-27T01:57:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:37 smithi119 ceph-mon[32752]: pgmap v14746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:37 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 26538.216592 secs 2024-04-27T01:57:37.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:37 smithi167 ceph-mon[32751]: pgmap v14746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:37.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:37 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 26538.216592 secs 2024-04-27T01:57:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:37 smithi090 ceph-mon[29180]: pgmap v14746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:57:37.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:37 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 26538.216592 secs 2024-04-27T01:57:38.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:57:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:57:38.138+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:57:38.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:57:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:57:38.138+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:57:38.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:57:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:57:38.138+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:57:38.607 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:57:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:57:38.138+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:57:38.608 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 01:57:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T01:57:38.138+0000 7f823bfa6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T01:57:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:39 smithi119 ceph-mon[32752]: pgmap v14747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:57:39.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:39 smithi167 ceph-mon[32751]: pgmap v14747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:57:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:39 smithi090 ceph-mon[29180]: pgmap v14747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:57:40.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:40 smithi090 ceph-mon[29180]: pgmap v14748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:40.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:40 smithi119 ceph-mon[32752]: pgmap v14748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:40.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:40 smithi167 ceph-mon[32751]: pgmap v14748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:41 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26542.392342 secs 2024-04-27T01:57:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:41 smithi090 ceph-mon[29180]: slow request 2882.858375 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:57:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:41 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26542.392342 secs 2024-04-27T01:57:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:41 smithi119 ceph-mon[32752]: slow request 2882.858375 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:57:41.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:41 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26542.392342 secs 2024-04-27T01:57:41.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:41 smithi167 ceph-mon[32751]: slow request 2882.858375 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:57:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:42 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26543.216728 secs 2024-04-27T01:57:42.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:42 smithi090 ceph-mon[29180]: pgmap v14749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:42.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:42 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26543.216728 secs 2024-04-27T01:57:42.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:42 smithi119 ceph-mon[32752]: pgmap v14749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:42.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:42 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26543.216728 secs 2024-04-27T01:57:42.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:42 smithi167 ceph-mon[32751]: pgmap v14749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:44 smithi090 ceph-mon[29180]: pgmap v14750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:44.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:57:44.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:44 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:57:44.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:44 smithi119 ceph-mon[32752]: pgmap v14750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:44.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:57:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:44 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:57:44.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:44 smithi167 ceph-mon[32751]: pgmap v14750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:44.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T01:57:44.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:44 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T01:57:46.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26547.392424 secs 2024-04-27T01:57:46.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:46 smithi090 ceph-mon[29180]: pgmap v14751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:46.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:46 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26548.216844 secs 2024-04-27T01:57:46.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26547.392424 secs 2024-04-27T01:57:46.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:46 smithi119 ceph-mon[32752]: pgmap v14751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:46.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:46 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26548.216844 secs 2024-04-27T01:57:46.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26547.392424 secs 2024-04-27T01:57:46.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:46 smithi167 ceph-mon[32751]: pgmap v14751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:46.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:46 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26548.216844 secs 2024-04-27T01:57:48.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:48 smithi090 ceph-mon[29180]: pgmap v14752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:48.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:48 smithi119 ceph-mon[32752]: pgmap v14752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:48.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:48 smithi167 ceph-mon[32751]: pgmap v14752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:50.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:50 smithi090 ceph-mon[29180]: pgmap v14753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:50.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:50 smithi119 ceph-mon[32752]: pgmap v14753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:50.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:50 smithi167 ceph-mon[32751]: pgmap v14753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T01:57:51.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26552.392511 secs 2024-04-27T01:57:51.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:51 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26553.216903 secs 2024-04-27T01:57:51.685 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26552.392511 secs 2024-04-27T01:57:51.686 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:51 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26553.216903 secs 2024-04-27T01:57:51.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26552.392511 secs 2024-04-27T01:57:51.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:51 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26553.216903 secs 2024-04-27T01:57:52.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:52 smithi090 ceph-mon[29180]: pgmap v14754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:52.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:52 smithi119 ceph-mon[32752]: pgmap v14754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:52.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:52 smithi167 ceph-mon[32751]: pgmap v14754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:54.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:54 smithi090 ceph-mon[29180]: pgmap v14755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:54.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:54 smithi119 ceph-mon[32752]: pgmap v14755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:54.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:54 smithi167 ceph-mon[32751]: pgmap v14755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:57:56.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26557.392593 secs 2024-04-27T01:57:56.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:56 smithi090 ceph-mon[29180]: pgmap v14756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:57:56.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:56 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26558.217043 secs 2024-04-27T01:57:56.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26557.392593 secs 2024-04-27T01:57:56.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:56 smithi119 ceph-mon[32752]: pgmap v14756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:57:56.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:56 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26558.217043 secs 2024-04-27T01:57:56.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26557.392593 secs 2024-04-27T01:57:56.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:56 smithi167 ceph-mon[32751]: pgmap v14756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T01:57:56.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:56 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26558.217043 secs 2024-04-27T01:57:58.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:57:58 smithi090 ceph-mon[29180]: pgmap v14757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:58.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:57:58 smithi119 ceph-mon[32752]: pgmap v14757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:57:58.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:57:58 smithi167 ceph-mon[32751]: pgmap v14757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:00.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:00 smithi090 ceph-mon[29180]: pgmap v14758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:00.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:00 smithi119 ceph-mon[32752]: pgmap v14758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:00.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:00 smithi167 ceph-mon[32751]: pgmap v14758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:01.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26562.392680 secs 2024-04-27T01:58:01.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:01 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26563.217153 secs 2024-04-27T01:58:01.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26562.392680 secs 2024-04-27T01:58:01.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:01 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26563.217153 secs 2024-04-27T01:58:01.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26562.392680 secs 2024-04-27T01:58:01.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:01 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26563.217153 secs 2024-04-27T01:58:02.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:02 smithi090 ceph-mon[29180]: pgmap v14759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:02.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:02 smithi119 ceph-mon[32752]: pgmap v14759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:02.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:02 smithi167 ceph-mon[32751]: pgmap v14759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:04.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:04 smithi090 ceph-mon[29180]: pgmap v14760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:58:04.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:04 smithi119 ceph-mon[32752]: pgmap v14760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:58:04.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:04 smithi167 ceph-mon[32751]: pgmap v14760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T01:58:06.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26567.392763 secs 2024-04-27T01:58:06.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:06 smithi090 ceph-mon[29180]: pgmap v14761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:06.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:06 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26568.217293 secs 2024-04-27T01:58:06.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26567.392763 secs 2024-04-27T01:58:06.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:06 smithi119 ceph-mon[32752]: pgmap v14761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:06.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:06 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26568.217293 secs 2024-04-27T01:58:06.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26567.392763 secs 2024-04-27T01:58:06.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:06 smithi167 ceph-mon[32751]: pgmap v14761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:06.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:06 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26568.217293 secs 2024-04-27T01:58:08.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:08 smithi090 ceph-mon[29180]: pgmap v14762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:08.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:08 smithi119 ceph-mon[32752]: pgmap v14762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:08.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:08 smithi167 ceph-mon[32751]: pgmap v14762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:10.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:10 smithi090 ceph-mon[29180]: pgmap v14763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:10.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:10 smithi119 ceph-mon[32752]: pgmap v14763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:10.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:10 smithi167 ceph-mon[32751]: pgmap v14763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26572.392825 secs 2024-04-27T01:58:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:11 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26573.217412 secs 2024-04-27T01:58:11.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26572.392825 secs 2024-04-27T01:58:11.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:11 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26573.217412 secs 2024-04-27T01:58:11.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26572.392825 secs 2024-04-27T01:58:11.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:11 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26573.217412 secs 2024-04-27T01:58:12.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:12 smithi119 ceph-mon[32752]: pgmap v14764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:12.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:12 smithi167 ceph-mon[32751]: pgmap v14764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:12.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:12 smithi090 ceph-mon[29180]: pgmap v14764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:14.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:14 smithi119 ceph-mon[32752]: pgmap v14765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:14.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:14 smithi167 ceph-mon[32751]: pgmap v14765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:14.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:14 smithi090 ceph-mon[29180]: pgmap v14765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T01:58:16.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26577.392907 secs 2024-04-27T01:58:16.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:16 smithi119 ceph-mon[32752]: pgmap v14766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:16.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:16 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26578.217526 secs 2024-04-27T01:58:16.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26577.392907 secs 2024-04-27T01:58:16.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:16 smithi167 ceph-mon[32751]: pgmap v14766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:16.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:16 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26578.217526 secs 2024-04-27T01:58:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26577.392907 secs 2024-04-27T01:58:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:16 smithi090 ceph-mon[29180]: pgmap v14766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T01:58:16.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:16 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26578.217526 secs 2024-04-27T01:58:18.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:18 smithi119 ceph-mon[32752]: pgmap v14767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:18.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:18 smithi167 ceph-mon[32751]: pgmap v14767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:18.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:18 smithi090 ceph-mon[29180]: pgmap v14767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:20.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:20 smithi119 ceph-mon[32752]: pgmap v14768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:20.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:20 smithi167 ceph-mon[32751]: pgmap v14768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:20.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:20 smithi090 ceph-mon[29180]: pgmap v14768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:21.685 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26582.392992 secs 2024-04-27T01:58:21.686 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:21 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26583.217595 secs 2024-04-27T01:58:21.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26582.392992 secs 2024-04-27T01:58:21.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:21 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26583.217595 secs 2024-04-27T01:58:21.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26582.392992 secs 2024-04-27T01:58:21.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:21 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26583.217595 secs 2024-04-27T01:58:22.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:22 smithi119 ceph-mon[32752]: pgmap v14769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:22.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:22 smithi167 ceph-mon[32751]: pgmap v14769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:22.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:22 smithi090 ceph-mon[29180]: pgmap v14769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:24.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:24 smithi119 ceph-mon[32752]: pgmap v14770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:24.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:24 smithi167 ceph-mon[32751]: pgmap v14770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:24.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:24 smithi090 ceph-mon[29180]: pgmap v14770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:26.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26587.393073 secs 2024-04-27T01:58:26.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:26 smithi119 ceph-mon[32752]: pgmap v14771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:26.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:26 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26588.217714 secs 2024-04-27T01:58:26.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26587.393073 secs 2024-04-27T01:58:26.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:26 smithi167 ceph-mon[32751]: pgmap v14771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:26.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:26 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26588.217714 secs 2024-04-27T01:58:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26587.393073 secs 2024-04-27T01:58:26.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:26 smithi090 ceph-mon[29180]: pgmap v14771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:26.857 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:26 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26588.217714 secs 2024-04-27T01:58:27.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:58:27.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:58:27.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:58:28.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:28 smithi090 ceph-mon[29180]: pgmap v14772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:28.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:58:28.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:58:28.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:28 smithi090 ceph-mon[29180]: pgmap v14773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:58:28.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:28 smithi090 ceph-mon[29180]: pgmap v14774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:58:28.758 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:58:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:28 smithi119 ceph-mon[32752]: pgmap v14772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:58:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:58:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:28 smithi119 ceph-mon[32752]: pgmap v14773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:58:28.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:28 smithi119 ceph-mon[32752]: pgmap v14774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:58:28.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:58:28.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:28 smithi167 ceph-mon[32751]: pgmap v14772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:28.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:58:28.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:58:28.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:28 smithi167 ceph-mon[32751]: pgmap v14773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T01:58:28.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:28 smithi167 ceph-mon[32751]: pgmap v14774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:58:28.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:58:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:30 smithi090 ceph-mon[29180]: pgmap v14775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:30.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:30 smithi119 ceph-mon[32752]: pgmap v14775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:30.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:30 smithi167 ceph-mon[32751]: pgmap v14775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26592.393160 secs 2024-04-27T01:58:31.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26592.393160 secs 2024-04-27T01:58:31.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26592.393160 secs 2024-04-27T01:58:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:32 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26593.217847 secs 2024-04-27T01:58:32.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:32 smithi090 ceph-mon[29180]: pgmap v14776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:58:32.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:32 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26593.217847 secs 2024-04-27T01:58:32.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:32 smithi119 ceph-mon[32752]: pgmap v14776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:58:32.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:32 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26593.217847 secs 2024-04-27T01:58:32.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:32 smithi167 ceph-mon[32751]: pgmap v14776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:58:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:34 smithi090 ceph-mon[29180]: pgmap v14777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:34.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:34 smithi119 ceph-mon[32752]: pgmap v14777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:34.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:34 smithi167 ceph-mon[32751]: pgmap v14777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26597.393242 secs 2024-04-27T01:58:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26597.393242 secs 2024-04-27T01:58:36.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26597.393242 secs 2024-04-27T01:58:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:36 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26598.217984 secs 2024-04-27T01:58:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:36 smithi090 ceph-mon[29180]: pgmap v14778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:36 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26598.217984 secs 2024-04-27T01:58:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:36 smithi119 ceph-mon[32752]: pgmap v14778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:37.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:36 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26598.217984 secs 2024-04-27T01:58:37.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:36 smithi167 ceph-mon[32751]: pgmap v14778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:58:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:38 smithi090 ceph-mon[29180]: pgmap v14779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:58:38.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:38 smithi119 ceph-mon[32752]: pgmap v14779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:58:38.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:38 smithi167 ceph-mon[32751]: pgmap v14779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:58:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:40 smithi090 ceph-mon[29180]: pgmap v14780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:40.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:40 smithi119 ceph-mon[32752]: pgmap v14780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:40.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:40 smithi167 ceph-mon[32751]: pgmap v14780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26602.393325 secs 2024-04-27T01:58:41.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26602.393325 secs 2024-04-27T01:58:41.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26602.393325 secs 2024-04-27T01:58:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:42 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26603.218093 secs 2024-04-27T01:58:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:42 smithi090 ceph-mon[29180]: pgmap v14781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:42.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:42 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26603.218093 secs 2024-04-27T01:58:42.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:42 smithi119 ceph-mon[32752]: pgmap v14781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:42.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:42 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26603.218093 secs 2024-04-27T01:58:42.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:42 smithi167 ceph-mon[32751]: pgmap v14781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:44 smithi090 ceph-mon[29180]: pgmap v14782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:44.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:44 smithi119 ceph-mon[32752]: pgmap v14782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:44.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:44 smithi167 ceph-mon[32751]: pgmap v14782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26607.393409 secs 2024-04-27T01:58:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26607.393409 secs 2024-04-27T01:58:46.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26607.393409 secs 2024-04-27T01:58:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:46 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26608.218216 secs 2024-04-27T01:58:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:46 smithi090 ceph-mon[29180]: pgmap v14783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:46 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26608.218216 secs 2024-04-27T01:58:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:46 smithi119 ceph-mon[32752]: pgmap v14783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:46 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26608.218216 secs 2024-04-27T01:58:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:46 smithi167 ceph-mon[32751]: pgmap v14783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:48 smithi090 ceph-mon[29180]: pgmap v14784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:48.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:48 smithi119 ceph-mon[32752]: pgmap v14784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:48.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:48 smithi167 ceph-mon[32751]: pgmap v14784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:50 smithi090 ceph-mon[29180]: pgmap v14785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:50.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:50 smithi119 ceph-mon[32752]: pgmap v14785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:50.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:50 smithi167 ceph-mon[32751]: pgmap v14785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26612.393497 secs 2024-04-27T01:58:51.434 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26612.393497 secs 2024-04-27T01:58:51.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26612.393497 secs 2024-04-27T01:58:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:52 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26613.218351 secs 2024-04-27T01:58:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:52 smithi090 ceph-mon[29180]: pgmap v14786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:52.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:52 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26613.218351 secs 2024-04-27T01:58:52.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:52 smithi119 ceph-mon[32752]: pgmap v14786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:52.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:52 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26613.218351 secs 2024-04-27T01:58:52.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:52 smithi167 ceph-mon[32751]: pgmap v14786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:54 smithi090 ceph-mon[29180]: pgmap v14787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:54.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:54 smithi119 ceph-mon[32752]: pgmap v14787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:54.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:54 smithi167 ceph-mon[32751]: pgmap v14787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:58:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:55 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26617.393578 secs 2024-04-27T01:58:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:55 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26617.393578 secs 2024-04-27T01:58:56.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:55 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26617.393578 secs 2024-04-27T01:58:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:56 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26618.218427 secs 2024-04-27T01:58:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:56 smithi119 ceph-mon[32752]: pgmap v14788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:57.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:56 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26618.218427 secs 2024-04-27T01:58:57.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:56 smithi167 ceph-mon[32751]: pgmap v14788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:56 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26618.218427 secs 2024-04-27T01:58:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:56 smithi090 ceph-mon[29180]: pgmap v14788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:58.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:58:58 smithi090 ceph-mon[29180]: pgmap v14789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:58.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:58:58 smithi119 ceph-mon[32752]: pgmap v14789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:58:58.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:58:58 smithi167 ceph-mon[32751]: pgmap v14789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:00 smithi090 ceph-mon[29180]: pgmap v14790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:00.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:00 smithi119 ceph-mon[32752]: pgmap v14790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:00.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:00 smithi167 ceph-mon[32751]: pgmap v14790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26622.393663 secs 2024-04-27T01:59:01.450 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26622.393663 secs 2024-04-27T01:59:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26622.393663 secs 2024-04-27T01:59:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:02 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26623.218530 secs 2024-04-27T01:59:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:02 smithi090 ceph-mon[29180]: pgmap v14791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:02.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:02 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26623.218530 secs 2024-04-27T01:59:02.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:02 smithi119 ceph-mon[32752]: pgmap v14791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:02.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:02 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26623.218530 secs 2024-04-27T01:59:02.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:02 smithi167 ceph-mon[32751]: pgmap v14791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:04 smithi090 ceph-mon[29180]: pgmap v14792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:04.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:04 smithi119 ceph-mon[32752]: pgmap v14792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:04.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:04 smithi167 ceph-mon[32751]: pgmap v14792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:06.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26627.393744 secs 2024-04-27T01:59:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26627.393744 secs 2024-04-27T01:59:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26627.393744 secs 2024-04-27T01:59:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:06 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26628.218646 secs 2024-04-27T01:59:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:06 smithi119 ceph-mon[32752]: pgmap v14793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:07.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:06 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26628.218646 secs 2024-04-27T01:59:07.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:06 smithi167 ceph-mon[32751]: pgmap v14793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:06 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26628.218646 secs 2024-04-27T01:59:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:06 smithi090 ceph-mon[29180]: pgmap v14793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:08 smithi090 ceph-mon[29180]: pgmap v14794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:08.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:08 smithi119 ceph-mon[32752]: pgmap v14794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:08.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:08 smithi167 ceph-mon[32751]: pgmap v14794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:10 smithi090 ceph-mon[29180]: pgmap v14795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:10.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:10 smithi119 ceph-mon[32752]: pgmap v14795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:10.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:10 smithi167 ceph-mon[32751]: pgmap v14795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26632.393850 secs 2024-04-27T01:59:11.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26632.393850 secs 2024-04-27T01:59:11.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26632.393850 secs 2024-04-27T01:59:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:12 smithi090 ceph-mon[29180]: 6 slow requests, 1 included below; oldest blocked for > 26633.218743 secs 2024-04-27T01:59:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:12 smithi090 ceph-mon[29180]: slow request 723.000793 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:59:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:12 smithi090 ceph-mon[29180]: pgmap v14796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:12.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:12 smithi119 ceph-mon[32752]: 6 slow requests, 1 included below; oldest blocked for > 26633.218743 secs 2024-04-27T01:59:12.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:12 smithi119 ceph-mon[32752]: slow request 723.000793 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:59:12.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:12 smithi119 ceph-mon[32752]: pgmap v14796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:12.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:12 smithi167 ceph-mon[32751]: 6 slow requests, 1 included below; oldest blocked for > 26633.218743 secs 2024-04-27T01:59:12.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:12 smithi167 ceph-mon[32751]: slow request 723.000793 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T01:59:12.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:12 smithi167 ceph-mon[32751]: pgmap v14796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:14 smithi090 ceph-mon[29180]: pgmap v14797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:14.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:14 smithi119 ceph-mon[32752]: pgmap v14797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:14.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:14 smithi167 ceph-mon[32751]: pgmap v14797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:16.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:15 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26637.393935 secs 2024-04-27T01:59:16.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:15 smithi090 ceph-mon[29180]: slow request 724.270716 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:59:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:15 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26637.393935 secs 2024-04-27T01:59:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:15 smithi119 ceph-mon[32752]: slow request 724.270716 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:59:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:15 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26637.393935 secs 2024-04-27T01:59:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:15 smithi167 ceph-mon[32751]: slow request 724.270716 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T01:59:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:16 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26638.218813 secs 2024-04-27T01:59:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:16 smithi090 ceph-mon[29180]: pgmap v14798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:16 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26638.218813 secs 2024-04-27T01:59:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:16 smithi119 ceph-mon[32752]: pgmap v14798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:16 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26638.218813 secs 2024-04-27T01:59:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:16 smithi167 ceph-mon[32751]: pgmap v14798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:18 smithi090 ceph-mon[29180]: pgmap v14799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:18.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:18 smithi119 ceph-mon[32752]: pgmap v14799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:18.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:18 smithi167 ceph-mon[32751]: pgmap v14799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:20 smithi090 ceph-mon[29180]: pgmap v14800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:20.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:20 smithi119 ceph-mon[32752]: pgmap v14800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:20.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:20 smithi167 ceph-mon[32751]: pgmap v14800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26642.394022 secs 2024-04-27T01:59:21.435 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26642.394022 secs 2024-04-27T01:59:21.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26642.394022 secs 2024-04-27T01:59:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:22 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26643.218947 secs 2024-04-27T01:59:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:22 smithi090 ceph-mon[29180]: pgmap v14801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:22 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26643.218947 secs 2024-04-27T01:59:22.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:22 smithi119 ceph-mon[32752]: pgmap v14801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:22.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:22 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26643.218947 secs 2024-04-27T01:59:22.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:22 smithi167 ceph-mon[32751]: pgmap v14801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:24 smithi090 ceph-mon[29180]: pgmap v14802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:24.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:24 smithi119 ceph-mon[32752]: pgmap v14802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:24.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:24 smithi167 ceph-mon[32751]: pgmap v14802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:26.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26647.394106 secs 2024-04-27T01:59:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26647.394106 secs 2024-04-27T01:59:26.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26647.394106 secs 2024-04-27T01:59:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:26 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26648.219064 secs 2024-04-27T01:59:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:26 smithi119 ceph-mon[32752]: pgmap v14803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:27.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:26 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26648.219064 secs 2024-04-27T01:59:27.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:26 smithi167 ceph-mon[32751]: pgmap v14803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:26 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26648.219064 secs 2024-04-27T01:59:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:26 smithi090 ceph-mon[29180]: pgmap v14803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:28.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:28 smithi090 ceph-mon[29180]: pgmap v14804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:28.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:28 smithi119 ceph-mon[32752]: pgmap v14804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:28.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:28 smithi167 ceph-mon[32751]: pgmap v14804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:59:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:59:29.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:59:29.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:29 smithi119 ceph-mon[32752]: pgmap v14805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:59:29.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:29 smithi119 ceph-mon[32752]: pgmap v14806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:29.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:59:29.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:59:29.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:59:29.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:59:29.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:29 smithi167 ceph-mon[32751]: pgmap v14805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:59:29.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:29 smithi167 ceph-mon[32751]: pgmap v14806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:29.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:59:29.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T01:59:29.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T01:59:29.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T01:59:29.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:29 smithi090 ceph-mon[29180]: pgmap v14805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T01:59:29.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:29 smithi090 ceph-mon[29180]: pgmap v14806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:29.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T01:59:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26652.394167 secs 2024-04-27T01:59:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26652.394167 secs 2024-04-27T01:59:31.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26652.394167 secs 2024-04-27T01:59:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:31 smithi119 ceph-mon[32752]: pgmap v14807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:31 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26653.219162 secs 2024-04-27T01:59:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:31 smithi167 ceph-mon[32751]: pgmap v14807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:31 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26653.219162 secs 2024-04-27T01:59:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:31 smithi090 ceph-mon[29180]: pgmap v14807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:31 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26653.219162 secs 2024-04-27T01:59:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:33 smithi119 ceph-mon[32752]: pgmap v14808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:59:34.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:33 smithi167 ceph-mon[32751]: pgmap v14808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:59:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:33 smithi090 ceph-mon[29180]: pgmap v14808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T01:59:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:35 smithi119 ceph-mon[32752]: pgmap v14809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:36.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26657.394250 secs 2024-04-27T01:59:36.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:35 smithi167 ceph-mon[32751]: pgmap v14809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:36.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26657.394250 secs 2024-04-27T01:59:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:35 smithi090 ceph-mon[29180]: pgmap v14809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26657.394250 secs 2024-04-27T01:59:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:36 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26658.219285 secs 2024-04-27T01:59:37.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:36 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26658.219285 secs 2024-04-27T01:59:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:36 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26658.219285 secs 2024-04-27T01:59:38.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:37 smithi119 ceph-mon[32752]: pgmap v14810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:38.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:37 smithi167 ceph-mon[32751]: pgmap v14810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:37 smithi090 ceph-mon[29180]: pgmap v14810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T01:59:40.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:39 smithi119 ceph-mon[32752]: pgmap v14811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:59:40.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:39 smithi167 ceph-mon[32751]: pgmap v14811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:59:40.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:39 smithi090 ceph-mon[29180]: pgmap v14811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T01:59:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26662.394334 secs 2024-04-27T01:59:41.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26662.394334 secs 2024-04-27T01:59:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26662.394334 secs 2024-04-27T01:59:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:41 smithi119 ceph-mon[32752]: pgmap v14812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:41 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26663.219403 secs 2024-04-27T01:59:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:41 smithi167 ceph-mon[32751]: pgmap v14812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:41 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26663.219403 secs 2024-04-27T01:59:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:41 smithi090 ceph-mon[29180]: pgmap v14812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:41 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26663.219403 secs 2024-04-27T01:59:44.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:43 smithi119 ceph-mon[32752]: pgmap v14813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:44.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:43 smithi167 ceph-mon[32751]: pgmap v14813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:44.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:43 smithi090 ceph-mon[29180]: pgmap v14813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:45 smithi119 ceph-mon[32752]: pgmap v14814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:46.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26667.394417 secs 2024-04-27T01:59:46.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:45 smithi167 ceph-mon[32751]: pgmap v14814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:46.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26667.394417 secs 2024-04-27T01:59:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:45 smithi090 ceph-mon[29180]: pgmap v14814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26667.394417 secs 2024-04-27T01:59:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:46 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26668.219527 secs 2024-04-27T01:59:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:46 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26668.219527 secs 2024-04-27T01:59:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:46 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26668.219527 secs 2024-04-27T01:59:48.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:47 smithi119 ceph-mon[32752]: pgmap v14815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:48.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:47 smithi167 ceph-mon[32751]: pgmap v14815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:48.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:47 smithi090 ceph-mon[29180]: pgmap v14815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:50.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:49 smithi119 ceph-mon[32752]: pgmap v14816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:50.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:49 smithi167 ceph-mon[32751]: pgmap v14816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:50.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:49 smithi090 ceph-mon[29180]: pgmap v14816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:51.185 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26672.394503 secs 2024-04-27T01:59:51.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26672.394503 secs 2024-04-27T01:59:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26672.394503 secs 2024-04-27T01:59:52.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:51 smithi119 ceph-mon[32752]: pgmap v14817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:52.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:51 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26673.219644 secs 2024-04-27T01:59:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:51 smithi167 ceph-mon[32751]: pgmap v14817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:51 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26673.219644 secs 2024-04-27T01:59:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:51 smithi090 ceph-mon[29180]: pgmap v14817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:51 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26673.219644 secs 2024-04-27T01:59:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:53 smithi119 ceph-mon[32752]: pgmap v14818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:54.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:53 smithi167 ceph-mon[32751]: pgmap v14818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:53 smithi090 ceph-mon[29180]: pgmap v14818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:55 smithi119 ceph-mon[32752]: pgmap v14819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:56.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:55 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26677.394585 secs 2024-04-27T01:59:56.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:55 smithi167 ceph-mon[32751]: pgmap v14819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:56.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:55 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26677.394585 secs 2024-04-27T01:59:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:55 smithi090 ceph-mon[29180]: pgmap v14819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T01:59:56.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:55 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26677.394585 secs 2024-04-27T01:59:57.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:56 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26678.219708 secs 2024-04-27T01:59:57.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:56 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26678.219708 secs 2024-04-27T01:59:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:56 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26678.219708 secs 2024-04-27T01:59:58.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:57 smithi090 ceph-mon[29180]: pgmap v14820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:58.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:57 smithi119 ceph-mon[32752]: pgmap v14820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T01:59:58.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:57 smithi167 ceph-mon[32751]: pgmap v14820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:00.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 01:59:59 smithi119 ceph-mon[32752]: pgmap v14821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:00.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 01:59:59 smithi167 ceph-mon[32751]: pgmap v14821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:00.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 01:59:59 smithi090 ceph-mon[29180]: pgmap v14821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:01.200 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:00:01.200 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26682.394670 secs 2024-04-27T02:00:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:00:01.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26682.394670 secs 2024-04-27T02:00:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:00:01.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26682.394670 secs 2024-04-27T02:00:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:01 smithi119 ceph-mon[32752]: pgmap v14822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:01 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26683.219824 secs 2024-04-27T02:00:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:01 smithi167 ceph-mon[32751]: pgmap v14822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:01 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26683.219824 secs 2024-04-27T02:00:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:01 smithi090 ceph-mon[29180]: pgmap v14822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:01 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26683.219824 secs 2024-04-27T02:00:04.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:03 smithi119 ceph-mon[32752]: pgmap v14823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:04.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:03 smithi167 ceph-mon[32751]: pgmap v14823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:03 smithi090 ceph-mon[29180]: pgmap v14823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:06.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:05 smithi090 ceph-mon[29180]: pgmap v14824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:06.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26687.394754 secs 2024-04-27T02:00:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:05 smithi119 ceph-mon[32752]: pgmap v14824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26687.394754 secs 2024-04-27T02:00:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:05 smithi167 ceph-mon[32751]: pgmap v14824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26687.394754 secs 2024-04-27T02:00:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:06 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26688.219897 secs 2024-04-27T02:00:07.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:06 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26688.219897 secs 2024-04-27T02:00:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:06 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26688.219897 secs 2024-04-27T02:00:08.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:07 smithi119 ceph-mon[32752]: pgmap v14825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:08.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:07 smithi167 ceph-mon[32751]: pgmap v14825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:07 smithi090 ceph-mon[29180]: pgmap v14825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:09 smithi119 ceph-mon[32752]: pgmap v14826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:10.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:09 smithi167 ceph-mon[32751]: pgmap v14826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:09 smithi090 ceph-mon[29180]: pgmap v14826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26692.394848 secs 2024-04-27T02:00:11.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26692.394848 secs 2024-04-27T02:00:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26692.394848 secs 2024-04-27T02:00:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:11 smithi119 ceph-mon[32752]: pgmap v14827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:11 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26693.219959 secs 2024-04-27T02:00:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:11 smithi167 ceph-mon[32751]: pgmap v14827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:11 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26693.219959 secs 2024-04-27T02:00:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:11 smithi090 ceph-mon[29180]: pgmap v14827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:11 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26693.219959 secs 2024-04-27T02:00:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:13 smithi119 ceph-mon[32752]: pgmap v14828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:14.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:13 smithi167 ceph-mon[32751]: pgmap v14828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:13 smithi090 ceph-mon[29180]: pgmap v14828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:15 smithi119 ceph-mon[32752]: pgmap v14829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26697.394937 secs 2024-04-27T02:00:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:15 smithi167 ceph-mon[32751]: pgmap v14829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26697.394937 secs 2024-04-27T02:00:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:15 smithi090 ceph-mon[29180]: pgmap v14829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26697.394937 secs 2024-04-27T02:00:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:16 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26698.220075 secs 2024-04-27T02:00:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:16 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26698.220075 secs 2024-04-27T02:00:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:16 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26698.220075 secs 2024-04-27T02:00:18.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:17 smithi119 ceph-mon[32752]: pgmap v14830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:18.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:17 smithi167 ceph-mon[32751]: pgmap v14830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:17 smithi090 ceph-mon[29180]: pgmap v14830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:20.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:19 smithi119 ceph-mon[32752]: pgmap v14831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:20.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:19 smithi167 ceph-mon[32751]: pgmap v14831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:19 smithi090 ceph-mon[29180]: pgmap v14831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:21.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26702.394997 secs 2024-04-27T02:00:21.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26702.394997 secs 2024-04-27T02:00:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26702.394997 secs 2024-04-27T02:00:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:21 smithi119 ceph-mon[32752]: pgmap v14832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:21 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26703.220188 secs 2024-04-27T02:00:22.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:21 smithi167 ceph-mon[32751]: pgmap v14832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:22.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:21 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26703.220188 secs 2024-04-27T02:00:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:21 smithi090 ceph-mon[29180]: pgmap v14832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:21 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26703.220188 secs 2024-04-27T02:00:24.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:24 smithi119 ceph-mon[32752]: pgmap v14833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:24.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:24 smithi167 ceph-mon[32751]: pgmap v14833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:24 smithi090 ceph-mon[29180]: pgmap v14833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:26 smithi119 ceph-mon[32752]: pgmap v14834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26707.395081 secs 2024-04-27T02:00:26.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:26 smithi167 ceph-mon[32751]: pgmap v14834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:26.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26707.395081 secs 2024-04-27T02:00:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:26 smithi090 ceph-mon[29180]: pgmap v14834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26707.395081 secs 2024-04-27T02:00:27.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:27 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26708.220322 secs 2024-04-27T02:00:27.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:27 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26708.220322 secs 2024-04-27T02:00:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:27 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26708.220322 secs 2024-04-27T02:00:28.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:28 smithi090 ceph-mon[29180]: pgmap v14835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:28.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:28 smithi119 ceph-mon[32752]: pgmap v14835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:28.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:28 smithi167 ceph-mon[32751]: pgmap v14835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:29.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:00:29.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:00:29.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:00:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:30 smithi119 ceph-mon[32752]: pgmap v14836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:00:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:00:30.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:00:30.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:30 smithi167 ceph-mon[32751]: pgmap v14836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:30.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:00:30.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:00:30.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:00:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:30 smithi090 ceph-mon[29180]: pgmap v14836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:00:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:00:30.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:00:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:31 smithi119 ceph-mon[32752]: pgmap v14837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:00:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:31 smithi119 ceph-mon[32752]: pgmap v14838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:31.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26712.395166 secs 2024-04-27T02:00:31.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:31 smithi167 ceph-mon[32751]: pgmap v14837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:00:31.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:31 smithi167 ceph-mon[32751]: pgmap v14838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:31.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26712.395166 secs 2024-04-27T02:00:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:31 smithi090 ceph-mon[29180]: pgmap v14837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:00:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:31 smithi090 ceph-mon[29180]: pgmap v14838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26712.395166 secs 2024-04-27T02:00:32.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:32 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26713.220433 secs 2024-04-27T02:00:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:32 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26713.220433 secs 2024-04-27T02:00:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:32 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26713.220433 secs 2024-04-27T02:00:33.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:33 smithi119 ceph-mon[32752]: pgmap v14839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:33.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:33 smithi167 ceph-mon[32751]: pgmap v14839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:33.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:33 smithi090 ceph-mon[29180]: pgmap v14839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:35 smithi119 ceph-mon[32752]: pgmap v14840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:35 smithi167 ceph-mon[32751]: pgmap v14840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:35 smithi090 ceph-mon[29180]: pgmap v14840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:36.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26717.395254 secs 2024-04-27T02:00:36.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26717.395254 secs 2024-04-27T02:00:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26717.395254 secs 2024-04-27T02:00:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:37 smithi119 ceph-mon[32752]: pgmap v14841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:37 smithi119 ceph-mon[32752]: 6 slow requests, 0 included below; oldest blocked for > 26718.220528 secs 2024-04-27T02:00:37.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:37 smithi167 ceph-mon[32751]: pgmap v14841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:37.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:37 smithi167 ceph-mon[32751]: 6 slow requests, 0 included below; oldest blocked for > 26718.220528 secs 2024-04-27T02:00:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:37 smithi090 ceph-mon[29180]: pgmap v14841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:00:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:37 smithi090 ceph-mon[29180]: 6 slow requests, 0 included below; oldest blocked for > 26718.220528 secs 2024-04-27T02:00:39.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:39 smithi119 ceph-mon[32752]: pgmap v14842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:00:39.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:39 smithi167 ceph-mon[32751]: pgmap v14842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:00:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:39 smithi090 ceph-mon[29180]: pgmap v14842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:00:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:41 smithi119 ceph-mon[32752]: pgmap v14843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:00:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26722.395342 secs 2024-04-27T02:00:41.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:41 smithi167 ceph-mon[32751]: pgmap v14843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:00:41.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26722.395342 secs 2024-04-27T02:00:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:41 smithi090 ceph-mon[29180]: pgmap v14843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:00:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26722.395342 secs 2024-04-27T02:00:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:42 smithi119 ceph-mon[32752]: 26 slow requests, 2 included below; oldest blocked for > 26723.220625 secs 2024-04-27T02:00:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:42 smithi119 ceph-mon[32752]: slow request 182.826642 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:00:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:42 smithi119 ceph-mon[32752]: slow request 11616.089013 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:00:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:42 smithi167 ceph-mon[32751]: 26 slow requests, 2 included below; oldest blocked for > 26723.220625 secs 2024-04-27T02:00:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:42 smithi167 ceph-mon[32751]: slow request 182.826642 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:00:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:42 smithi167 ceph-mon[32751]: slow request 11616.089013 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:00:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:42 smithi090 ceph-mon[29180]: 26 slow requests, 2 included below; oldest blocked for > 26723.220625 secs 2024-04-27T02:00:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:42 smithi090 ceph-mon[29180]: slow request 182.826642 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:00:42.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:42 smithi090 ceph-mon[29180]: slow request 11616.089013 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:00:43.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:43 smithi167 ceph-mon[32751]: pgmap v14844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:43.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:43 smithi090 ceph-mon[29180]: pgmap v14844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:43.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:43 smithi119 ceph-mon[32752]: pgmap v14844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:45.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:45 smithi167 ceph-mon[32751]: pgmap v14845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:45.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:00:45.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:00:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:45 smithi090 ceph-mon[29180]: pgmap v14845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:00:45.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:00:45.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:45 smithi119 ceph-mon[32752]: pgmap v14845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:45.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:00:45.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:00:46.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26727.395430 secs 2024-04-27T02:00:46.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26727.395430 secs 2024-04-27T02:00:46.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26727.395430 secs 2024-04-27T02:00:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:47 smithi090 ceph-mon[29180]: pgmap v14846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26728.220707 secs 2024-04-27T02:00:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:47 smithi119 ceph-mon[32752]: pgmap v14846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:47.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26728.220707 secs 2024-04-27T02:00:47.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:47 smithi167 ceph-mon[32751]: pgmap v14846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:47.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26728.220707 secs 2024-04-27T02:00:49.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:49 smithi090 ceph-mon[29180]: pgmap v14847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:49.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:49 smithi119 ceph-mon[32752]: pgmap v14847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:49.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:49 smithi167 ceph-mon[32751]: pgmap v14847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:51.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:51 smithi167 ceph-mon[32751]: pgmap v14848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:51.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26732.395518 secs 2024-04-27T02:00:51.336 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:51 smithi119 ceph-mon[32752]: pgmap v14848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:51.336 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26732.395518 secs 2024-04-27T02:00:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:51 smithi090 ceph-mon[29180]: pgmap v14848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26732.395518 secs 2024-04-27T02:00:52.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26733.220832 secs 2024-04-27T02:00:52.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26733.220832 secs 2024-04-27T02:00:52.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26733.220832 secs 2024-04-27T02:00:53.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:53 smithi090 ceph-mon[29180]: pgmap v14849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:53.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:53 smithi119 ceph-mon[32752]: pgmap v14849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:53.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:53 smithi167 ceph-mon[32751]: pgmap v14849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:54.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:54 smithi090 ceph-mon[29180]: pgmap v14850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:54.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:54 smithi119 ceph-mon[32752]: pgmap v14850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:54.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:54 smithi167 ceph-mon[32751]: pgmap v14850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:00:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26737.395599 secs 2024-04-27T02:00:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:56 smithi119 ceph-mon[32752]: pgmap v14851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:57.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26738.220951 secs 2024-04-27T02:00:57.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26737.395599 secs 2024-04-27T02:00:57.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:56 smithi167 ceph-mon[32751]: pgmap v14851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:57.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26738.220951 secs 2024-04-27T02:00:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26737.395599 secs 2024-04-27T02:00:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:56 smithi090 ceph-mon[29180]: pgmap v14851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26738.220951 secs 2024-04-27T02:00:58.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:00:58 smithi090 ceph-mon[29180]: pgmap v14852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:59.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:00:58 smithi119 ceph-mon[32752]: pgmap v14852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:00:59.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:00:58 smithi167 ceph-mon[32751]: pgmap v14852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:01.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:00 smithi119 ceph-mon[32752]: pgmap v14853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:01.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:00 smithi167 ceph-mon[32751]: pgmap v14853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:00 smithi090 ceph-mon[29180]: pgmap v14853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:01.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26742.395684 secs 2024-04-27T02:01:01.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26743.221088 secs 2024-04-27T02:01:01.950 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26742.395684 secs 2024-04-27T02:01:01.950 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26743.221088 secs 2024-04-27T02:01:02.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26742.395684 secs 2024-04-27T02:01:02.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26743.221088 secs 2024-04-27T02:01:03.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:02 smithi119 ceph-mon[32752]: pgmap v14854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:03.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:02 smithi167 ceph-mon[32751]: pgmap v14854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:03.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:02 smithi090 ceph-mon[29180]: pgmap v14854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:05.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:04 smithi119 ceph-mon[32752]: pgmap v14855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:05.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:04 smithi167 ceph-mon[32751]: pgmap v14855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:05.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:04 smithi090 ceph-mon[29180]: pgmap v14855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26747.395767 secs 2024-04-27T02:01:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:06 smithi119 ceph-mon[32752]: pgmap v14856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:07.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26748.221150 secs 2024-04-27T02:01:07.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26747.395767 secs 2024-04-27T02:01:07.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:06 smithi167 ceph-mon[32751]: pgmap v14856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:07.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26748.221150 secs 2024-04-27T02:01:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26747.395767 secs 2024-04-27T02:01:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:06 smithi090 ceph-mon[29180]: pgmap v14856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:07.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26748.221150 secs 2024-04-27T02:01:09.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:08 smithi119 ceph-mon[32752]: pgmap v14857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:09.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:08 smithi167 ceph-mon[32751]: pgmap v14857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:09.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:08 smithi090 ceph-mon[29180]: pgmap v14857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:11.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:10 smithi119 ceph-mon[32752]: pgmap v14858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:11.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:10 smithi167 ceph-mon[32751]: pgmap v14858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:11.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:10 smithi090 ceph-mon[29180]: pgmap v14858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:12.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26752.395852 secs 2024-04-27T02:01:12.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26753.221258 secs 2024-04-27T02:01:12.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26752.395852 secs 2024-04-27T02:01:12.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26753.221258 secs 2024-04-27T02:01:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26752.395852 secs 2024-04-27T02:01:12.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26753.221258 secs 2024-04-27T02:01:13.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:12 smithi119 ceph-mon[32752]: pgmap v14859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:13.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:12 smithi167 ceph-mon[32751]: pgmap v14859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:13.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:12 smithi090 ceph-mon[29180]: pgmap v14859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:15.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:14 smithi119 ceph-mon[32752]: pgmap v14860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:15.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:14 smithi167 ceph-mon[32751]: pgmap v14860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:15.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:14 smithi090 ceph-mon[29180]: pgmap v14860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26757.395932 secs 2024-04-27T02:01:17.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:16 smithi119 ceph-mon[32752]: pgmap v14861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:17.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26758.221357 secs 2024-04-27T02:01:17.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26757.395932 secs 2024-04-27T02:01:17.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:16 smithi167 ceph-mon[32751]: pgmap v14861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:17.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26758.221357 secs 2024-04-27T02:01:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26757.395932 secs 2024-04-27T02:01:17.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:16 smithi090 ceph-mon[29180]: pgmap v14861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:17.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26758.221357 secs 2024-04-27T02:01:19.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:18 smithi119 ceph-mon[32752]: pgmap v14862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:19.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:18 smithi167 ceph-mon[32751]: pgmap v14862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:19.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:18 smithi090 ceph-mon[29180]: pgmap v14862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:21.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:20 smithi119 ceph-mon[32752]: pgmap v14863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:21.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:20 smithi167 ceph-mon[32751]: pgmap v14863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:21.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:20 smithi090 ceph-mon[29180]: pgmap v14863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:22.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26762.395990 secs 2024-04-27T02:01:22.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26763.221501 secs 2024-04-27T02:01:22.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26762.395990 secs 2024-04-27T02:01:22.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26763.221501 secs 2024-04-27T02:01:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26762.395990 secs 2024-04-27T02:01:22.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26763.221501 secs 2024-04-27T02:01:23.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:22 smithi119 ceph-mon[32752]: pgmap v14864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:23.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:22 smithi167 ceph-mon[32751]: pgmap v14864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:23.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:22 smithi090 ceph-mon[29180]: pgmap v14864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:25.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:24 smithi119 ceph-mon[32752]: pgmap v14865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:25.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:24 smithi167 ceph-mon[32751]: pgmap v14865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:25.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:24 smithi090 ceph-mon[29180]: pgmap v14865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26767.396075 secs 2024-04-27T02:01:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:26 smithi119 ceph-mon[32752]: pgmap v14866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:27.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26768.221624 secs 2024-04-27T02:01:27.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26767.396075 secs 2024-04-27T02:01:27.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:26 smithi167 ceph-mon[32751]: pgmap v14866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:27.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26768.221624 secs 2024-04-27T02:01:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26767.396075 secs 2024-04-27T02:01:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:26 smithi090 ceph-mon[29180]: pgmap v14866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:27.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26768.221624 secs 2024-04-27T02:01:29.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:28 smithi119 ceph-mon[32752]: pgmap v14867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:29.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:28 smithi167 ceph-mon[32751]: pgmap v14867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:29.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:28 smithi090 ceph-mon[29180]: pgmap v14867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:30.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:01:30.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:01:30.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:01:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:30 smithi119 ceph-mon[32752]: pgmap v14868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:01:31.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:01:31.035 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:01:31.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:30 smithi167 ceph-mon[32751]: pgmap v14868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:31.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:01:31.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:01:31.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:01:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:30 smithi090 ceph-mon[29180]: pgmap v14868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:01:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:01:31.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:01:32.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26772.396142 secs 2024-04-27T02:01:32.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:31 smithi119 ceph-mon[32752]: pgmap v14869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:01:32.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:31 smithi119 ceph-mon[32752]: pgmap v14870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:32.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26773.221697 secs 2024-04-27T02:01:32.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26772.396142 secs 2024-04-27T02:01:32.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:31 smithi167 ceph-mon[32751]: pgmap v14869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:01:32.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:31 smithi167 ceph-mon[32751]: pgmap v14870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:32.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26773.221697 secs 2024-04-27T02:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26772.396142 secs 2024-04-27T02:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:31 smithi090 ceph-mon[29180]: pgmap v14869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:31 smithi090 ceph-mon[29180]: pgmap v14870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26773.221697 secs 2024-04-27T02:01:34.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:33 smithi119 ceph-mon[32752]: pgmap v14871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:01:34.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:33 smithi167 ceph-mon[32751]: pgmap v14871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:01:34.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:33 smithi090 ceph-mon[29180]: pgmap v14871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:01:36.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:35 smithi119 ceph-mon[32752]: pgmap v14872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:36.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:35 smithi167 ceph-mon[32751]: pgmap v14872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:36.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:35 smithi090 ceph-mon[29180]: pgmap v14872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26777.396228 secs 2024-04-27T02:01:37.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26778.221826 secs 2024-04-27T02:01:37.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26777.396228 secs 2024-04-27T02:01:37.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26778.221826 secs 2024-04-27T02:01:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26777.396228 secs 2024-04-27T02:01:37.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26778.221826 secs 2024-04-27T02:01:38.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:37 smithi119 ceph-mon[32752]: pgmap v14873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:38.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:37 smithi167 ceph-mon[32751]: pgmap v14873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:38.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:37 smithi090 ceph-mon[29180]: pgmap v14873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:39.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:39 smithi090 ceph-mon[29180]: pgmap v14874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:40.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:39 smithi119 ceph-mon[32752]: pgmap v14874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:40.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:39 smithi167 ceph-mon[32751]: pgmap v14874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:01:42.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26782.396307 secs 2024-04-27T02:01:42.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:41 smithi167 ceph-mon[32751]: pgmap v14875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:01:42.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26783.221938 secs 2024-04-27T02:01:42.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26782.396307 secs 2024-04-27T02:01:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:41 smithi090 ceph-mon[29180]: pgmap v14875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:01:42.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26783.221938 secs 2024-04-27T02:01:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26782.396307 secs 2024-04-27T02:01:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:41 smithi119 ceph-mon[32752]: pgmap v14875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:01:42.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26783.221938 secs 2024-04-27T02:01:44.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:43 smithi167 ceph-mon[32751]: pgmap v14876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:44.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:43 smithi090 ceph-mon[29180]: pgmap v14876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:44.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:43 smithi119 ceph-mon[32752]: pgmap v14876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:46.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:45 smithi167 ceph-mon[32751]: pgmap v14877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:46.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:45 smithi090 ceph-mon[29180]: pgmap v14877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:46.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:45 smithi119 ceph-mon[32752]: pgmap v14877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26787.396390 secs 2024-04-27T02:01:47.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26788.222060 secs 2024-04-27T02:01:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26787.396390 secs 2024-04-27T02:01:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26788.222060 secs 2024-04-27T02:01:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26787.396390 secs 2024-04-27T02:01:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26788.222060 secs 2024-04-27T02:01:48.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:47 smithi090 ceph-mon[29180]: pgmap v14878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:48.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:47 smithi119 ceph-mon[32752]: pgmap v14878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:48.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:47 smithi167 ceph-mon[32751]: pgmap v14878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:50.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:49 smithi090 ceph-mon[29180]: pgmap v14879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:50.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:49 smithi119 ceph-mon[32752]: pgmap v14879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:50.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:49 smithi167 ceph-mon[32751]: pgmap v14879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:52.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26792.396474 secs 2024-04-27T02:01:52.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:51 smithi119 ceph-mon[32752]: pgmap v14880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:52.086 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26793.222174 secs 2024-04-27T02:01:52.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26792.396474 secs 2024-04-27T02:01:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:51 smithi090 ceph-mon[29180]: pgmap v14880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26793.222174 secs 2024-04-27T02:01:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26792.396474 secs 2024-04-27T02:01:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:51 smithi167 ceph-mon[32751]: pgmap v14880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26793.222174 secs 2024-04-27T02:01:54.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:53 smithi090 ceph-mon[29180]: pgmap v14881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:54.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:53 smithi119 ceph-mon[32752]: pgmap v14881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:54.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:53 smithi167 ceph-mon[32751]: pgmap v14881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:56.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:55 smithi090 ceph-mon[29180]: pgmap v14882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:56.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:55 smithi119 ceph-mon[32752]: pgmap v14882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:56.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:55 smithi167 ceph-mon[32751]: pgmap v14882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:01:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26797.396559 secs 2024-04-27T02:01:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26798.222305 secs 2024-04-27T02:01:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26797.396559 secs 2024-04-27T02:01:57.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26798.222305 secs 2024-04-27T02:01:57.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26797.396559 secs 2024-04-27T02:01:57.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26798.222305 secs 2024-04-27T02:01:58.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:57 smithi090 ceph-mon[29180]: pgmap v14883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:58.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:57 smithi119 ceph-mon[32752]: pgmap v14883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:01:58.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:57 smithi167 ceph-mon[32751]: pgmap v14883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:00.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:01:59 smithi090 ceph-mon[29180]: pgmap v14884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:00.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:01:59 smithi119 ceph-mon[32752]: pgmap v14884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:00.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:01:59 smithi167 ceph-mon[32751]: pgmap v14884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:01.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26802.396640 secs 2024-04-27T02:02:01.198 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26802.396640 secs 2024-04-27T02:02:01.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26802.396640 secs 2024-04-27T02:02:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:01 smithi090 ceph-mon[29180]: pgmap v14885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:02.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26803.222368 secs 2024-04-27T02:02:02.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:01 smithi119 ceph-mon[32752]: pgmap v14885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26803.222368 secs 2024-04-27T02:02:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:01 smithi167 ceph-mon[32751]: pgmap v14885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26803.222368 secs 2024-04-27T02:02:04.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:03 smithi090 ceph-mon[29180]: pgmap v14886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:04.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:03 smithi119 ceph-mon[32752]: pgmap v14886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:04.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:03 smithi167 ceph-mon[32751]: pgmap v14886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:06.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:05 smithi090 ceph-mon[29180]: pgmap v14887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:06.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26807.396726 secs 2024-04-27T02:02:06.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:05 smithi119 ceph-mon[32752]: pgmap v14887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26807.396726 secs 2024-04-27T02:02:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:05 smithi167 ceph-mon[32751]: pgmap v14887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26807.396726 secs 2024-04-27T02:02:07.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26808.222490 secs 2024-04-27T02:02:07.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26808.222490 secs 2024-04-27T02:02:07.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26808.222490 secs 2024-04-27T02:02:08.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:07 smithi090 ceph-mon[29180]: pgmap v14888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:08.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:07 smithi119 ceph-mon[32752]: pgmap v14888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:08.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:07 smithi167 ceph-mon[32751]: pgmap v14888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:10.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:09 smithi090 ceph-mon[29180]: pgmap v14889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:09 smithi119 ceph-mon[32752]: pgmap v14889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:10.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:09 smithi167 ceph-mon[32751]: pgmap v14889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26812.396825 secs 2024-04-27T02:02:11.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26812.396825 secs 2024-04-27T02:02:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26812.396825 secs 2024-04-27T02:02:12.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:11 smithi119 ceph-mon[32752]: pgmap v14890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26813.222552 secs 2024-04-27T02:02:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:11 smithi167 ceph-mon[32751]: pgmap v14890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26813.222552 secs 2024-04-27T02:02:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:11 smithi090 ceph-mon[29180]: pgmap v14890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26813.222552 secs 2024-04-27T02:02:14.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:13 smithi119 ceph-mon[32752]: pgmap v14891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:14.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:13 smithi167 ceph-mon[32751]: pgmap v14891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:13 smithi090 ceph-mon[29180]: pgmap v14891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:16.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:15 smithi119 ceph-mon[32752]: pgmap v14892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26817.396911 secs 2024-04-27T02:02:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:15 smithi167 ceph-mon[32751]: pgmap v14892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26817.396911 secs 2024-04-27T02:02:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:15 smithi090 ceph-mon[29180]: pgmap v14892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26817.396911 secs 2024-04-27T02:02:17.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26818.222635 secs 2024-04-27T02:02:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26818.222635 secs 2024-04-27T02:02:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26818.222635 secs 2024-04-27T02:02:18.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:17 smithi119 ceph-mon[32752]: pgmap v14893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:18.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:17 smithi167 ceph-mon[32751]: pgmap v14893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:17 smithi090 ceph-mon[29180]: pgmap v14893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:20.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:19 smithi119 ceph-mon[32752]: pgmap v14894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:19 smithi167 ceph-mon[32751]: pgmap v14894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:20.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:19 smithi090 ceph-mon[29180]: pgmap v14894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:21.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26822.396994 secs 2024-04-27T02:02:21.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26822.396994 secs 2024-04-27T02:02:21.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26822.396994 secs 2024-04-27T02:02:22.184 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:21 smithi119 ceph-mon[32752]: pgmap v14895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:22.184 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26823.222719 secs 2024-04-27T02:02:22.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:21 smithi167 ceph-mon[32751]: pgmap v14895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:22.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26823.222719 secs 2024-04-27T02:02:22.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:21 smithi090 ceph-mon[29180]: pgmap v14895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26823.222719 secs 2024-04-27T02:02:24.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:23 smithi119 ceph-mon[32752]: pgmap v14896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:24.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:23 smithi167 ceph-mon[32751]: pgmap v14896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:24.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:23 smithi090 ceph-mon[29180]: pgmap v14896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:25 smithi119 ceph-mon[32752]: pgmap v14897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:26.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26827.397054 secs 2024-04-27T02:02:26.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:25 smithi167 ceph-mon[32751]: pgmap v14897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:26.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26827.397054 secs 2024-04-27T02:02:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:25 smithi090 ceph-mon[29180]: pgmap v14897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:26.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26827.397054 secs 2024-04-27T02:02:27.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26828.222832 secs 2024-04-27T02:02:27.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26828.222832 secs 2024-04-27T02:02:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26828.222832 secs 2024-04-27T02:02:28.257 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:27 smithi090 ceph-mon[29180]: pgmap v14898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:28.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:27 smithi119 ceph-mon[32752]: pgmap v14898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:28.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:27 smithi167 ceph-mon[32751]: pgmap v14898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:30.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:29 smithi119 ceph-mon[32752]: pgmap v14899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:30.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:29 smithi167 ceph-mon[32751]: pgmap v14899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:30.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:29 smithi090 ceph-mon[29180]: pgmap v14899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:31.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26832.397140 secs 2024-04-27T02:02:31.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:02:31.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26832.397140 secs 2024-04-27T02:02:31.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:02:31.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26832.397140 secs 2024-04-27T02:02:31.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:02:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:32 smithi090 ceph-mon[29180]: pgmap v14900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26833.222920 secs 2024-04-27T02:02:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:32 smithi119 ceph-mon[32752]: pgmap v14900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:32.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26833.222920 secs 2024-04-27T02:02:32.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:32 smithi167 ceph-mon[32751]: pgmap v14900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26833.222920 secs 2024-04-27T02:02:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:32.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: pgmap v14901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: pgmap v14901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:34.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.301 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: pgmap v14901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:34.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:02:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:02:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:02:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:02:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:02:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:02:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:02:35.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:02:35.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: pgmap v14902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:02:35.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: pgmap v14903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:02:35.286 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:02:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:02:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:02:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:02:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:02:35.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:02:35.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:02:35.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:02:35.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: pgmap v14902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:02:35.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: pgmap v14903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:02:35.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:02:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:02:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:02:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:02:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:02:35.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:02:35.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:02:35.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:02:35.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:02:35.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: pgmap v14902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:02:35.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: pgmap v14903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:02:35.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:02:36.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26837.397236 secs 2024-04-27T02:02:36.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26837.397236 secs 2024-04-27T02:02:36.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26837.397236 secs 2024-04-27T02:02:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26838.223028 secs 2024-04-27T02:02:37.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:36 smithi119 ceph-mon[32752]: pgmap v14904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:02:37.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26838.223028 secs 2024-04-27T02:02:37.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:36 smithi167 ceph-mon[32751]: pgmap v14904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:02:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26838.223028 secs 2024-04-27T02:02:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:36 smithi090 ceph-mon[29180]: pgmap v14904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:02:38.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:02:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:02:38.188+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:02:38.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:02:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:02:38.188+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:02:38.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:02:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:02:38.188+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:02:38.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:02:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:02:38.188+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:02:38.606 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:02:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:02:38.188+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:02:39.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:39 smithi119 ceph-mon[32752]: pgmap v14905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 B/s rd, 1 op/s 2024-04-27T02:02:39.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:39 smithi167 ceph-mon[32751]: pgmap v14905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 B/s rd, 1 op/s 2024-04-27T02:02:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:39 smithi090 ceph-mon[29180]: pgmap v14905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 B/s rd, 1 op/s 2024-04-27T02:02:41.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:41 smithi119 ceph-mon[32752]: pgmap v14906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-27T02:02:41.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26842.397295 secs 2024-04-27T02:02:41.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:41 smithi167 ceph-mon[32751]: pgmap v14906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-27T02:02:41.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26842.397295 secs 2024-04-27T02:02:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:41 smithi090 ceph-mon[29180]: pgmap v14906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-27T02:02:41.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26842.397295 secs 2024-04-27T02:02:42.138 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26843.223089 secs 2024-04-27T02:02:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26843.223089 secs 2024-04-27T02:02:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26843.223089 secs 2024-04-27T02:02:43.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:43 smithi119 ceph-mon[32752]: pgmap v14907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-27T02:02:43.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:43 smithi167 ceph-mon[32751]: pgmap v14907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-27T02:02:43.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:43 smithi090 ceph-mon[29180]: pgmap v14907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 213 B/s wr, 2 op/s 2024-04-27T02:02:45.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:45 smithi119 ceph-mon[32752]: pgmap v14908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T02:02:45.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:45 smithi167 ceph-mon[32751]: pgmap v14908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T02:02:45.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:45 smithi090 ceph-mon[29180]: pgmap v14908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T02:02:46.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26847.397381 secs 2024-04-27T02:02:46.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26847.397381 secs 2024-04-27T02:02:46.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26847.397381 secs 2024-04-27T02:02:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26848.223227 secs 2024-04-27T02:02:47.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:47 smithi119 ceph-mon[32752]: pgmap v14909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:02:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26848.223227 secs 2024-04-27T02:02:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:47 smithi167 ceph-mon[32751]: pgmap v14909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:02:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26848.223227 secs 2024-04-27T02:02:47.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:47 smithi090 ceph-mon[29180]: pgmap v14909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:02:49.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:49 smithi167 ceph-mon[32751]: pgmap v14910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:02:49.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:49 smithi090 ceph-mon[29180]: pgmap v14910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:02:49.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:49 smithi119 ceph-mon[32752]: pgmap v14910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:02:51.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:51 smithi167 ceph-mon[32751]: pgmap v14911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:02:51.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26852.397468 secs 2024-04-27T02:02:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:51 smithi090 ceph-mon[29180]: pgmap v14911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:02:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26852.397468 secs 2024-04-27T02:02:51.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:51 smithi119 ceph-mon[32752]: pgmap v14911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:02:51.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26852.397468 secs 2024-04-27T02:02:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26853.223300 secs 2024-04-27T02:02:52.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26853.223300 secs 2024-04-27T02:02:52.435 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26853.223300 secs 2024-04-27T02:02:53.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:53 smithi167 ceph-mon[32751]: pgmap v14912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:53.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:53 smithi090 ceph-mon[29180]: pgmap v14912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:53.459 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:53 smithi119 ceph-mon[32752]: pgmap v14912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:55.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:55 smithi090 ceph-mon[29180]: pgmap v14913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:55.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:55 smithi119 ceph-mon[32752]: pgmap v14913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:55.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:55 smithi167 ceph-mon[32751]: pgmap v14913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:02:56.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26857.397552 secs 2024-04-27T02:02:56.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26857.397552 secs 2024-04-27T02:02:56.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26857.397552 secs 2024-04-27T02:02:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26858.223400 secs 2024-04-27T02:02:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:57 smithi090 ceph-mon[29180]: pgmap v14914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:57.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26858.223400 secs 2024-04-27T02:02:57.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:57 smithi119 ceph-mon[32752]: pgmap v14914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:57.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26858.223400 secs 2024-04-27T02:02:57.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:57 smithi167 ceph-mon[32751]: pgmap v14914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:02:59 smithi090 ceph-mon[29180]: pgmap v14915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:59.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:02:59 smithi119 ceph-mon[32752]: pgmap v14915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:02:59.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:02:59 smithi167 ceph-mon[32751]: pgmap v14915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:01.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:01 smithi167 ceph-mon[32751]: pgmap v14916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:01.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:01 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26862.397637 secs 2024-04-27T02:03:01.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:01 smithi167 ceph-mon[32751]: slow request 23044.443268 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:03:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:01 smithi090 ceph-mon[29180]: pgmap v14916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:01 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26862.397637 secs 2024-04-27T02:03:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:01 smithi090 ceph-mon[29180]: slow request 23044.443268 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:03:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:01 smithi119 ceph-mon[32752]: pgmap v14916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:01 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26862.397637 secs 2024-04-27T02:03:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:01 smithi119 ceph-mon[32752]: slow request 23044.443268 seconds old, received at 2024-04-26T19:38:55.707773+0000: client_request(client.24733:267884 getattr AsXsFs #0x100000001fb 2024-04-26T19:38:55.706260+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:03:02.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26863.223468 secs 2024-04-27T02:03:02.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26863.223468 secs 2024-04-27T02:03:02.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26863.223468 secs 2024-04-27T02:03:03.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:03 smithi090 ceph-mon[29180]: pgmap v14917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:03.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:03 smithi119 ceph-mon[32752]: pgmap v14917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:03.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:03 smithi167 ceph-mon[32751]: pgmap v14917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:05.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:05 smithi090 ceph-mon[29180]: pgmap v14918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:05.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:05 smithi119 ceph-mon[32752]: pgmap v14918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:05.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:05 smithi167 ceph-mon[32751]: pgmap v14918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:06.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26867.397722 secs 2024-04-27T02:03:06.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26867.397722 secs 2024-04-27T02:03:06.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26867.397722 secs 2024-04-27T02:03:07.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26868.223607 secs 2024-04-27T02:03:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:07 smithi090 ceph-mon[29180]: pgmap v14919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:07.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26868.223607 secs 2024-04-27T02:03:07.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:07 smithi119 ceph-mon[32752]: pgmap v14919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:07.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26868.223607 secs 2024-04-27T02:03:07.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:07 smithi167 ceph-mon[32751]: pgmap v14919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:08.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:08 smithi090 ceph-mon[29180]: pgmap v14920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:08.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:08 smithi119 ceph-mon[32752]: pgmap v14920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:08.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:08 smithi167 ceph-mon[32751]: pgmap v14920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:11 smithi119 ceph-mon[32752]: pgmap v14921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:11.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26872.397826 secs 2024-04-27T02:03:11.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:11 smithi167 ceph-mon[32751]: pgmap v14921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:11.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26872.397826 secs 2024-04-27T02:03:11.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:11 smithi090 ceph-mon[29180]: pgmap v14921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26872.397826 secs 2024-04-27T02:03:12.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26873.223725 secs 2024-04-27T02:03:12.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26873.223725 secs 2024-04-27T02:03:12.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26873.223725 secs 2024-04-27T02:03:13.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:13 smithi119 ceph-mon[32752]: pgmap v14922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:13.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:13 smithi167 ceph-mon[32751]: pgmap v14922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:13.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:13 smithi090 ceph-mon[29180]: pgmap v14922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:15.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:15 smithi167 ceph-mon[32751]: pgmap v14923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:15.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:15 smithi090 ceph-mon[29180]: pgmap v14923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:15.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:15 smithi119 ceph-mon[32752]: pgmap v14923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26877.397887 secs 2024-04-27T02:03:16.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26877.397887 secs 2024-04-27T02:03:16.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26877.397887 secs 2024-04-27T02:03:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26878.223836 secs 2024-04-27T02:03:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:17 smithi167 ceph-mon[32751]: pgmap v14924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:17.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26878.223836 secs 2024-04-27T02:03:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:17 smithi090 ceph-mon[29180]: pgmap v14924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:17.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26878.223836 secs 2024-04-27T02:03:17.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:17 smithi119 ceph-mon[32752]: pgmap v14924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:19.300 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:19 smithi167 ceph-mon[32751]: pgmap v14925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:19.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:19 smithi090 ceph-mon[29180]: pgmap v14925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:19.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:19 smithi119 ceph-mon[32752]: pgmap v14925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:21.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:21 smithi167 ceph-mon[32751]: pgmap v14926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:21.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26882.397977 secs 2024-04-27T02:03:21.358 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:21 smithi090 ceph-mon[29180]: pgmap v14926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:21.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26882.397977 secs 2024-04-27T02:03:21.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:21 smithi119 ceph-mon[32752]: pgmap v14926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26882.397977 secs 2024-04-27T02:03:22.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26883.223951 secs 2024-04-27T02:03:22.434 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26883.223951 secs 2024-04-27T02:03:22.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26883.223951 secs 2024-04-27T02:03:23.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:23 smithi090 ceph-mon[29180]: pgmap v14927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:23.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:23 smithi119 ceph-mon[32752]: pgmap v14927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:23.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:23 smithi167 ceph-mon[32751]: pgmap v14927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:25.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:25 smithi090 ceph-mon[29180]: pgmap v14928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:25.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:25 smithi119 ceph-mon[32752]: pgmap v14928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:25.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:25 smithi167 ceph-mon[32751]: pgmap v14928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:26.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26887.398040 secs 2024-04-27T02:03:26.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26887.398040 secs 2024-04-27T02:03:26.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26887.398040 secs 2024-04-27T02:03:27.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26888.224016 secs 2024-04-27T02:03:27.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:27 smithi090 ceph-mon[29180]: pgmap v14929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:27.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26888.224016 secs 2024-04-27T02:03:27.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:27 smithi119 ceph-mon[32752]: pgmap v14929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26888.224016 secs 2024-04-27T02:03:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:27 smithi167 ceph-mon[32751]: pgmap v14929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:29.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:29 smithi090 ceph-mon[29180]: pgmap v14930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:29.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:29 smithi119 ceph-mon[32752]: pgmap v14930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:29.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:29 smithi167 ceph-mon[32751]: pgmap v14930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:31.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:31 smithi167 ceph-mon[32751]: pgmap v14931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:31.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26892.398125 secs 2024-04-27T02:03:31.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:31 smithi090 ceph-mon[29180]: pgmap v14931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:31.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26892.398125 secs 2024-04-27T02:03:31.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:31 smithi119 ceph-mon[32752]: pgmap v14931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:31.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26892.398125 secs 2024-04-27T02:03:32.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26893.224129 secs 2024-04-27T02:03:32.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26893.224129 secs 2024-04-27T02:03:32.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26893.224129 secs 2024-04-27T02:03:33.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:33 smithi090 ceph-mon[29180]: pgmap v14932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:33.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:33 smithi119 ceph-mon[32752]: pgmap v14932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:33.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:33 smithi167 ceph-mon[32751]: pgmap v14932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:34.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:03:34.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:03:34.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:03:35.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:35 smithi090 ceph-mon[29180]: pgmap v14933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:35.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:35.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:35 smithi119 ceph-mon[32752]: pgmap v14933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:35.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:35.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:35.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:35 smithi167 ceph-mon[32751]: pgmap v14933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:35.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:35.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:36.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26897.398212 secs 2024-04-27T02:03:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:03:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:03:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:36 smithi090 ceph-mon[29180]: pgmap v14934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T02:03:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:36 smithi090 ceph-mon[29180]: pgmap v14935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:36.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:36.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26897.398212 secs 2024-04-27T02:03:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:03:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:03:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:36 smithi119 ceph-mon[32752]: pgmap v14934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T02:03:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:36 smithi119 ceph-mon[32752]: pgmap v14935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:36.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:36.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26897.398212 secs 2024-04-27T02:03:36.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:03:36.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:03:36.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:36 smithi167 ceph-mon[32751]: pgmap v14934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T02:03:36.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:36 smithi167 ceph-mon[32751]: pgmap v14935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:36.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:03:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26898.224252 secs 2024-04-27T02:03:37.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26898.224252 secs 2024-04-27T02:03:37.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26898.224252 secs 2024-04-27T02:03:38.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:38 smithi090 ceph-mon[29180]: pgmap v14936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:38.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:38 smithi119 ceph-mon[32752]: pgmap v14936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:38 smithi167 ceph-mon[32751]: pgmap v14936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:40.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:40 smithi119 ceph-mon[32752]: pgmap v14937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:40.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:40 smithi167 ceph-mon[32751]: pgmap v14937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:40.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:40 smithi090 ceph-mon[29180]: pgmap v14937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:41.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26902.398298 secs 2024-04-27T02:03:41.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26902.398298 secs 2024-04-27T02:03:41.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26902.398298 secs 2024-04-27T02:03:42.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:42 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 26903.224285 secs 2024-04-27T02:03:42.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:42 smithi119 ceph-mon[32752]: slow request 362.830302 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:03:42.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:42 smithi119 ceph-mon[32752]: pgmap v14938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:42.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:42 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 26903.224285 secs 2024-04-27T02:03:42.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:42 smithi167 ceph-mon[32751]: slow request 362.830302 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:03:42.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:42 smithi167 ceph-mon[32751]: pgmap v14938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:42.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:42 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 26903.224285 secs 2024-04-27T02:03:42.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:42 smithi090 ceph-mon[29180]: slow request 362.830302 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:03:42.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:42 smithi090 ceph-mon[29180]: pgmap v14938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:03:44.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:44 smithi119 ceph-mon[32752]: pgmap v14939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T02:03:44.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:44 smithi167 ceph-mon[32751]: pgmap v14939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T02:03:44.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:44 smithi090 ceph-mon[29180]: pgmap v14939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T02:03:46.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26907.398381 secs 2024-04-27T02:03:46.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:46 smithi119 ceph-mon[32752]: pgmap v14940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:03:46.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26907.398381 secs 2024-04-27T02:03:46.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:46 smithi167 ceph-mon[32751]: pgmap v14940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:03:46.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26907.398381 secs 2024-04-27T02:03:46.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:46 smithi090 ceph-mon[29180]: pgmap v14940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:03:47.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26908.224424 secs 2024-04-27T02:03:47.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26908.224424 secs 2024-04-27T02:03:47.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26908.224424 secs 2024-04-27T02:03:48.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:48 smithi167 ceph-mon[32751]: pgmap v14941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:48.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:48 smithi090 ceph-mon[29180]: pgmap v14941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:48.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:48 smithi119 ceph-mon[32752]: pgmap v14941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:50.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:50 smithi167 ceph-mon[32751]: pgmap v14942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:50.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:50 smithi090 ceph-mon[29180]: pgmap v14942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:50.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:50 smithi119 ceph-mon[32752]: pgmap v14942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:51.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26912.398465 secs 2024-04-27T02:03:51.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26912.398465 secs 2024-04-27T02:03:51.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26912.398465 secs 2024-04-27T02:03:52.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26913.224560 secs 2024-04-27T02:03:52.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:52 smithi090 ceph-mon[29180]: pgmap v14943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:52.684 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26913.224560 secs 2024-04-27T02:03:52.684 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:52 smithi119 ceph-mon[32752]: pgmap v14943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:52.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26913.224560 secs 2024-04-27T02:03:52.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:52 smithi167 ceph-mon[32751]: pgmap v14943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:54.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:54 smithi090 ceph-mon[29180]: pgmap v14944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:54.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:54 smithi119 ceph-mon[32752]: pgmap v14944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:54.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:54 smithi167 ceph-mon[32751]: pgmap v14944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:03:56.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26917.398551 secs 2024-04-27T02:03:56.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:56 smithi090 ceph-mon[29180]: pgmap v14945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:03:56.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26917.398551 secs 2024-04-27T02:03:56.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:56 smithi119 ceph-mon[32752]: pgmap v14945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:03:56.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26917.398551 secs 2024-04-27T02:03:56.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:56 smithi167 ceph-mon[32751]: pgmap v14945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:03:57.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26918.224678 secs 2024-04-27T02:03:57.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26918.224678 secs 2024-04-27T02:03:57.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26918.224678 secs 2024-04-27T02:03:58.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:03:58 smithi090 ceph-mon[29180]: pgmap v14946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:58.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:03:58 smithi119 ceph-mon[32752]: pgmap v14946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:03:58.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:03:58 smithi167 ceph-mon[32751]: pgmap v14946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:00.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:00 smithi090 ceph-mon[29180]: pgmap v14947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:00.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:00 smithi119 ceph-mon[32752]: pgmap v14947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:00.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:00 smithi167 ceph-mon[32751]: pgmap v14947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:01.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26922.398638 secs 2024-04-27T02:04:01.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26922.398638 secs 2024-04-27T02:04:01.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26922.398638 secs 2024-04-27T02:04:02.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26923.224770 secs 2024-04-27T02:04:02.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:02 smithi090 ceph-mon[29180]: pgmap v14948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:02.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26923.224770 secs 2024-04-27T02:04:02.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:02 smithi119 ceph-mon[32752]: pgmap v14948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:02.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26923.224770 secs 2024-04-27T02:04:02.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:02 smithi167 ceph-mon[32751]: pgmap v14948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:04.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:04 smithi090 ceph-mon[29180]: pgmap v14949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:04.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:04 smithi119 ceph-mon[32752]: pgmap v14949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:04.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:04 smithi167 ceph-mon[32751]: pgmap v14949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:06.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26927.398701 secs 2024-04-27T02:04:06.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:06 smithi090 ceph-mon[29180]: pgmap v14950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:06.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26927.398701 secs 2024-04-27T02:04:06.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:06 smithi119 ceph-mon[32752]: pgmap v14950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:06.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26927.398701 secs 2024-04-27T02:04:06.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:06 smithi167 ceph-mon[32751]: pgmap v14950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:07.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26928.224904 secs 2024-04-27T02:04:07.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26928.224904 secs 2024-04-27T02:04:07.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26928.224904 secs 2024-04-27T02:04:08.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:08 smithi090 ceph-mon[29180]: pgmap v14951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:08.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:08 smithi119 ceph-mon[32752]: pgmap v14951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:08.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:08 smithi167 ceph-mon[32751]: pgmap v14951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:10.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:10 smithi090 ceph-mon[29180]: pgmap v14952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:10.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:10 smithi119 ceph-mon[32752]: pgmap v14952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:10.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:10 smithi167 ceph-mon[32751]: pgmap v14952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:11.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26932.398792 secs 2024-04-27T02:04:11.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26933.225040 secs 2024-04-27T02:04:11.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26932.398792 secs 2024-04-27T02:04:11.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26933.225040 secs 2024-04-27T02:04:11.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26932.398792 secs 2024-04-27T02:04:11.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26933.225040 secs 2024-04-27T02:04:12.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:12 smithi090 ceph-mon[29180]: pgmap v14953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:12.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:12 smithi119 ceph-mon[32752]: pgmap v14953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:12.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:12 smithi167 ceph-mon[32751]: pgmap v14953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:14.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:14 smithi090 ceph-mon[29180]: pgmap v14954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:14.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:14 smithi119 ceph-mon[32752]: pgmap v14954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:14.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:14 smithi167 ceph-mon[32751]: pgmap v14954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:16.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26937.398870 secs 2024-04-27T02:04:16.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:16 smithi090 ceph-mon[29180]: pgmap v14955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:16.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26938.225157 secs 2024-04-27T02:04:16.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26937.398870 secs 2024-04-27T02:04:16.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:16 smithi119 ceph-mon[32752]: pgmap v14955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:16.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26938.225157 secs 2024-04-27T02:04:16.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26937.398870 secs 2024-04-27T02:04:16.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:16 smithi167 ceph-mon[32751]: pgmap v14955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:16.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26938.225157 secs 2024-04-27T02:04:18.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:18 smithi090 ceph-mon[29180]: pgmap v14956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:18.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:18 smithi119 ceph-mon[32752]: pgmap v14956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:18.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:18 smithi167 ceph-mon[32751]: pgmap v14956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:20.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:20 smithi119 ceph-mon[32752]: pgmap v14957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:20.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:20 smithi167 ceph-mon[32751]: pgmap v14957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:20.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:20 smithi090 ceph-mon[29180]: pgmap v14957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:21.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26942.398931 secs 2024-04-27T02:04:21.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26943.225279 secs 2024-04-27T02:04:21.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26942.398931 secs 2024-04-27T02:04:21.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26943.225279 secs 2024-04-27T02:04:21.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26942.398931 secs 2024-04-27T02:04:21.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26943.225279 secs 2024-04-27T02:04:22.684 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:22 smithi119 ceph-mon[32752]: pgmap v14958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:22.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:22 smithi167 ceph-mon[32751]: pgmap v14958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:22.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:22 smithi090 ceph-mon[29180]: pgmap v14958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:24.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:24 smithi119 ceph-mon[32752]: pgmap v14959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:24.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:24 smithi167 ceph-mon[32751]: pgmap v14959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:24.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:24 smithi090 ceph-mon[29180]: pgmap v14959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:26.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26947.398988 secs 2024-04-27T02:04:26.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:26 smithi119 ceph-mon[32752]: pgmap v14960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:26.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26948.225429 secs 2024-04-27T02:04:26.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26947.398988 secs 2024-04-27T02:04:26.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:26 smithi167 ceph-mon[32751]: pgmap v14960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:26.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26948.225429 secs 2024-04-27T02:04:26.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26947.398988 secs 2024-04-27T02:04:26.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:26 smithi090 ceph-mon[29180]: pgmap v14960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:26.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26948.225429 secs 2024-04-27T02:04:28.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:28 smithi119 ceph-mon[32752]: pgmap v14961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:28.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:28 smithi167 ceph-mon[32751]: pgmap v14961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:28.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:28 smithi090 ceph-mon[29180]: pgmap v14961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:30.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:30 smithi090 ceph-mon[29180]: pgmap v14962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:30.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:30 smithi119 ceph-mon[32752]: pgmap v14962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:30.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:30 smithi167 ceph-mon[32751]: pgmap v14962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:31.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:31 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 26952.399051 secs 2024-04-27T02:04:31.699 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:31 smithi167 ceph-mon[32751]: slow request 2880.500737 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:04:31.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26953.225533 secs 2024-04-27T02:04:31.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:31 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 26952.399051 secs 2024-04-27T02:04:31.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:31 smithi119 ceph-mon[32752]: slow request 2880.500737 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:04:31.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26953.225533 secs 2024-04-27T02:04:31.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:31 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 26952.399051 secs 2024-04-27T02:04:31.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:31 smithi090 ceph-mon[29180]: slow request 2880.500737 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:04:31.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26953.225533 secs 2024-04-27T02:04:32.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:32 smithi119 ceph-mon[32752]: pgmap v14963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:32.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:32 smithi167 ceph-mon[32751]: pgmap v14963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:32.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:32 smithi090 ceph-mon[29180]: pgmap v14963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:34.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:34 smithi119 ceph-mon[32752]: pgmap v14964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:34.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:34 smithi167 ceph-mon[32751]: pgmap v14964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:34.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:34 smithi090 ceph-mon[29180]: pgmap v14964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:35.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:04:35.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:04:35.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:04:36.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26957.399134 secs 2024-04-27T02:04:36.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: pgmap v14965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26958.225643 secs 2024-04-27T02:04:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:04:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:04:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: pgmap v14966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:04:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: pgmap v14967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:36.784 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:04:36.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26957.399134 secs 2024-04-27T02:04:36.797 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: pgmap v14965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:36.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26958.225643 secs 2024-04-27T02:04:36.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:04:36.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:04:36.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: pgmap v14966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:04:36.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: pgmap v14967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:36.798 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:04:36.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26957.399134 secs 2024-04-27T02:04:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: pgmap v14965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:04:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26958.225643 secs 2024-04-27T02:04:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:04:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:04:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: pgmap v14966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:04:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: pgmap v14967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:36.856 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:04:39.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:39 smithi167 ceph-mon[32751]: pgmap v14968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:04:39.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:39 smithi090 ceph-mon[29180]: pgmap v14968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:04:39.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:39 smithi119 ceph-mon[32752]: pgmap v14968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:04:41.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:41 smithi167 ceph-mon[32751]: pgmap v14969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:41.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26962.399221 secs 2024-04-27T02:04:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:41 smithi090 ceph-mon[29180]: pgmap v14969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26962.399221 secs 2024-04-27T02:04:41.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:41 smithi119 ceph-mon[32752]: pgmap v14969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:41.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26962.399221 secs 2024-04-27T02:04:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26963.225734 secs 2024-04-27T02:04:42.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26963.225734 secs 2024-04-27T02:04:42.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26963.225734 secs 2024-04-27T02:04:43.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:43 smithi090 ceph-mon[29180]: pgmap v14970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:04:43.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:43 smithi119 ceph-mon[32752]: pgmap v14970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:04:43.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:43 smithi167 ceph-mon[32751]: pgmap v14970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:04:45.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:44 smithi119 ceph-mon[32752]: pgmap v14971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:45.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:44 smithi167 ceph-mon[32751]: pgmap v14971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:44 smithi090 ceph-mon[29180]: pgmap v14971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:04:46.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26967.399306 secs 2024-04-27T02:04:46.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26967.399306 secs 2024-04-27T02:04:46.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26967.399306 secs 2024-04-27T02:04:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26968.225857 secs 2024-04-27T02:04:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:46 smithi119 ceph-mon[32752]: pgmap v14972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:04:47.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26968.225857 secs 2024-04-27T02:04:47.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:46 smithi167 ceph-mon[32751]: pgmap v14972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:04:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26968.225857 secs 2024-04-27T02:04:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:46 smithi090 ceph-mon[29180]: pgmap v14972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:04:49.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:49 smithi090 ceph-mon[29180]: pgmap v14973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:49.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:49 smithi119 ceph-mon[32752]: pgmap v14973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:49.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:49 smithi167 ceph-mon[32751]: pgmap v14973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:51.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:51 smithi090 ceph-mon[29180]: pgmap v14974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:51.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26972.399391 secs 2024-04-27T02:04:51.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:51 smithi119 ceph-mon[32752]: pgmap v14974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:51.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26972.399391 secs 2024-04-27T02:04:51.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:51 smithi167 ceph-mon[32751]: pgmap v14974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:51.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26972.399391 secs 2024-04-27T02:04:52.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26973.225944 secs 2024-04-27T02:04:52.434 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26973.225944 secs 2024-04-27T02:04:52.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26973.225944 secs 2024-04-27T02:04:53.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:53 smithi090 ceph-mon[29180]: pgmap v14975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:53.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:53 smithi119 ceph-mon[32752]: pgmap v14975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:53.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:53 smithi167 ceph-mon[32751]: pgmap v14975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:55.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:55 smithi090 ceph-mon[29180]: pgmap v14976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:55.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:55 smithi119 ceph-mon[32752]: pgmap v14976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:55.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:55 smithi167 ceph-mon[32751]: pgmap v14976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:04:56.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26977.399451 secs 2024-04-27T02:04:56.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26977.399451 secs 2024-04-27T02:04:56.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26977.399451 secs 2024-04-27T02:04:57.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26978.226045 secs 2024-04-27T02:04:57.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:57 smithi090 ceph-mon[29180]: pgmap v14977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:57.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26978.226045 secs 2024-04-27T02:04:57.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:57 smithi119 ceph-mon[32752]: pgmap v14977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:57.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26978.226045 secs 2024-04-27T02:04:57.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:57 smithi167 ceph-mon[32751]: pgmap v14977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:04:59 smithi090 ceph-mon[29180]: pgmap v14978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:59.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:04:59 smithi119 ceph-mon[32752]: pgmap v14978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:04:59.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:04:59 smithi167 ceph-mon[32751]: pgmap v14978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:01.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:01 smithi167 ceph-mon[32751]: pgmap v14979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:01.350 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26982.399536 secs 2024-04-27T02:05:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:01 smithi090 ceph-mon[29180]: pgmap v14979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26982.399536 secs 2024-04-27T02:05:01.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:01 smithi119 ceph-mon[32752]: pgmap v14979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:01.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26982.399536 secs 2024-04-27T02:05:02.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26983.226161 secs 2024-04-27T02:05:02.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26983.226161 secs 2024-04-27T02:05:02.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26983.226161 secs 2024-04-27T02:05:03.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:03 smithi090 ceph-mon[29180]: pgmap v14980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:03.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:03 smithi119 ceph-mon[32752]: pgmap v14980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:03.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:03 smithi167 ceph-mon[32751]: pgmap v14980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:05.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:05 smithi090 ceph-mon[29180]: pgmap v14981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:05.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:05 smithi119 ceph-mon[32752]: pgmap v14981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:05.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:05 smithi167 ceph-mon[32751]: pgmap v14981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:06.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26987.399619 secs 2024-04-27T02:05:06.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26987.399619 secs 2024-04-27T02:05:06.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26987.399619 secs 2024-04-27T02:05:07.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26988.226300 secs 2024-04-27T02:05:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:07 smithi090 ceph-mon[29180]: pgmap v14982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:07.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26988.226300 secs 2024-04-27T02:05:07.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:07 smithi119 ceph-mon[32752]: pgmap v14982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:07.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26988.226300 secs 2024-04-27T02:05:07.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:07 smithi167 ceph-mon[32751]: pgmap v14982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:08.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:08 smithi090 ceph-mon[29180]: pgmap v14983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:08.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:08 smithi119 ceph-mon[32752]: pgmap v14983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:08.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:08 smithi167 ceph-mon[32751]: pgmap v14983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:11.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:11 smithi090 ceph-mon[29180]: pgmap v14984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26992.399677 secs 2024-04-27T02:05:11.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:11 smithi119 ceph-mon[32752]: pgmap v14984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:11.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26992.399677 secs 2024-04-27T02:05:11.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:11 smithi167 ceph-mon[32751]: pgmap v14984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:11.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26992.399677 secs 2024-04-27T02:05:12.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26993.226440 secs 2024-04-27T02:05:12.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26993.226440 secs 2024-04-27T02:05:12.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26993.226440 secs 2024-04-27T02:05:13.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:13 smithi090 ceph-mon[29180]: pgmap v14985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:13 smithi119 ceph-mon[32752]: pgmap v14985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:13.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:13 smithi167 ceph-mon[32751]: pgmap v14985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:15.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:15 smithi090 ceph-mon[29180]: pgmap v14986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:15.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:15 smithi119 ceph-mon[32752]: pgmap v14986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:15.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:15 smithi167 ceph-mon[32751]: pgmap v14986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:16.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 26997.399758 secs 2024-04-27T02:05:16.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 26997.399758 secs 2024-04-27T02:05:16.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 26997.399758 secs 2024-04-27T02:05:17.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 26998.226570 secs 2024-04-27T02:05:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:17 smithi090 ceph-mon[29180]: pgmap v14987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:17.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 26998.226570 secs 2024-04-27T02:05:17.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:17 smithi119 ceph-mon[32752]: pgmap v14987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:17.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 26998.226570 secs 2024-04-27T02:05:17.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:17 smithi167 ceph-mon[32751]: pgmap v14987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:19.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:19 smithi090 ceph-mon[29180]: pgmap v14988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:19.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:19 smithi119 ceph-mon[32752]: pgmap v14988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:19.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:19 smithi167 ceph-mon[32751]: pgmap v14988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:21.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:21 smithi090 ceph-mon[29180]: pgmap v14989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:21.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27002.399849 secs 2024-04-27T02:05:21.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:21 smithi119 ceph-mon[32752]: pgmap v14989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:21.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27002.399849 secs 2024-04-27T02:05:21.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:21 smithi167 ceph-mon[32751]: pgmap v14989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:21.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27002.399849 secs 2024-04-27T02:05:22.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27003.226685 secs 2024-04-27T02:05:22.434 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27003.226685 secs 2024-04-27T02:05:22.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27003.226685 secs 2024-04-27T02:05:23.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:23 smithi090 ceph-mon[29180]: pgmap v14990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:23.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:23 smithi119 ceph-mon[32752]: pgmap v14990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:23.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:23 smithi167 ceph-mon[32751]: pgmap v14990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:25.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:25 smithi090 ceph-mon[29180]: pgmap v14991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:25.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:25 smithi119 ceph-mon[32752]: pgmap v14991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:25.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:25 smithi167 ceph-mon[32751]: pgmap v14991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:26.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27007.399912 secs 2024-04-27T02:05:26.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27007.399912 secs 2024-04-27T02:05:26.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27007.399912 secs 2024-04-27T02:05:27.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27008.226790 secs 2024-04-27T02:05:27.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:27 smithi090 ceph-mon[29180]: pgmap v14992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:27.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27008.226790 secs 2024-04-27T02:05:27.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:27 smithi119 ceph-mon[32752]: pgmap v14992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27008.226790 secs 2024-04-27T02:05:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:27 smithi167 ceph-mon[32751]: pgmap v14992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:28.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:28 smithi119 ceph-mon[32752]: pgmap v14993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:28.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:28 smithi167 ceph-mon[32751]: pgmap v14993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:28.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:28 smithi090 ceph-mon[29180]: pgmap v14993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:31.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:31 smithi090 ceph-mon[29180]: pgmap v14994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:31.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27012.399996 secs 2024-04-27T02:05:31.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:31 smithi119 ceph-mon[32752]: pgmap v14994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:31.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27012.399996 secs 2024-04-27T02:05:31.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:31 smithi167 ceph-mon[32751]: pgmap v14994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:31.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27012.399996 secs 2024-04-27T02:05:32.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27013.226905 secs 2024-04-27T02:05:32.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27013.226905 secs 2024-04-27T02:05:32.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27013.226905 secs 2024-04-27T02:05:33.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:33 smithi090 ceph-mon[29180]: pgmap v14995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:33.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:33 smithi119 ceph-mon[32752]: pgmap v14995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:33.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:33 smithi167 ceph-mon[32751]: pgmap v14995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:35.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:35 smithi090 ceph-mon[29180]: pgmap v14996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:35.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:35 smithi119 ceph-mon[32752]: pgmap v14996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:35.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:35 smithi167 ceph-mon[32751]: pgmap v14996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:36.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27017.400084 secs 2024-04-27T02:05:36.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27017.400084 secs 2024-04-27T02:05:36.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27017.400084 secs 2024-04-27T02:05:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27018.227029 secs 2024-04-27T02:05:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:37 smithi090 ceph-mon[29180]: pgmap v14997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:05:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:05:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:05:37.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:05:37.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27018.227029 secs 2024-04-27T02:05:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:37 smithi119 ceph-mon[32752]: pgmap v14997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:05:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:05:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:05:37.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27018.227029 secs 2024-04-27T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:37 smithi167 ceph-mon[32751]: pgmap v14997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:05:37.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:05:37.548 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:05:38.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:38 smithi090 ceph-mon[29180]: pgmap v14998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:05:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:38 smithi090 ceph-mon[29180]: pgmap v14999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:38.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:38 smithi119 ceph-mon[32752]: pgmap v14998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:05:38.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:38 smithi119 ceph-mon[32752]: pgmap v14999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:38 smithi167 ceph-mon[32751]: pgmap v14998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:05:38.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:38 smithi167 ceph-mon[32751]: pgmap v14999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:40.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:40 smithi090 ceph-mon[29180]: pgmap v15000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:40.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:40 smithi119 ceph-mon[32752]: pgmap v15000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:40.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:40 smithi167 ceph-mon[32751]: pgmap v15000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27022.400144 secs 2024-04-27T02:05:41.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27022.400144 secs 2024-04-27T02:05:41.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27022.400144 secs 2024-04-27T02:05:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:42 smithi090 ceph-mon[29180]: pgmap v15001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27023.227138 secs 2024-04-27T02:05:42.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:42 smithi119 ceph-mon[32752]: pgmap v15001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:42.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27023.227138 secs 2024-04-27T02:05:42.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:42 smithi167 ceph-mon[32751]: pgmap v15001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:42.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27023.227138 secs 2024-04-27T02:05:44.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:44 smithi090 ceph-mon[29180]: pgmap v15002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:05:44.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:44 smithi119 ceph-mon[32752]: pgmap v15002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:05:44.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:44 smithi167 ceph-mon[32751]: pgmap v15002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:05:46.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:46 smithi090 ceph-mon[29180]: pgmap v15003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:46.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27027.400230 secs 2024-04-27T02:05:46.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:46 smithi119 ceph-mon[32752]: pgmap v15003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:46.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27027.400230 secs 2024-04-27T02:05:46.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:46 smithi167 ceph-mon[32751]: pgmap v15003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:05:46.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27027.400230 secs 2024-04-27T02:05:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27028.227261 secs 2024-04-27T02:05:47.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27028.227261 secs 2024-04-27T02:05:47.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27028.227261 secs 2024-04-27T02:05:48.255 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:48 smithi090 ceph-mon[29180]: pgmap v15004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:05:48.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:48 smithi119 ceph-mon[32752]: pgmap v15004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:05:48.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:48 smithi167 ceph-mon[32751]: pgmap v15004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:05:49.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:49 smithi119 ceph-mon[32752]: pgmap v15005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:49.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:49 smithi167 ceph-mon[32751]: pgmap v15005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:49.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:49 smithi090 ceph-mon[29180]: pgmap v15005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:52.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27032.400312 secs 2024-04-27T02:05:52.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:51 smithi090 ceph-mon[29180]: pgmap v15006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:52.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27033.227340 secs 2024-04-27T02:05:52.236 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27032.400312 secs 2024-04-27T02:05:52.236 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:51 smithi119 ceph-mon[32752]: pgmap v15006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:52.236 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27033.227340 secs 2024-04-27T02:05:52.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27032.400312 secs 2024-04-27T02:05:52.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:51 smithi167 ceph-mon[32751]: pgmap v15006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:52.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27033.227340 secs 2024-04-27T02:05:54.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:53 smithi090 ceph-mon[29180]: pgmap v15007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:54.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:53 smithi119 ceph-mon[32752]: pgmap v15007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:54.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:53 smithi167 ceph-mon[32751]: pgmap v15007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:56.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:55 smithi090 ceph-mon[29180]: pgmap v15008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:56.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:55 smithi119 ceph-mon[32752]: pgmap v15008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:56.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:55 smithi167 ceph-mon[32751]: pgmap v15008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:05:57.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27037.400397 secs 2024-04-27T02:05:57.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27038.227494 secs 2024-04-27T02:05:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27037.400397 secs 2024-04-27T02:05:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27038.227494 secs 2024-04-27T02:05:57.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27037.400397 secs 2024-04-27T02:05:57.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27038.227494 secs 2024-04-27T02:05:58.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:57 smithi090 ceph-mon[29180]: pgmap v15009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:58.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:57 smithi119 ceph-mon[32752]: pgmap v15009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:05:58.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:57 smithi167 ceph-mon[32751]: pgmap v15009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:00.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:05:59 smithi090 ceph-mon[29180]: pgmap v15010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:00.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:05:59 smithi119 ceph-mon[32752]: pgmap v15010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:00.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:05:59 smithi167 ceph-mon[32751]: pgmap v15010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:01.199 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27042.400485 secs 2024-04-27T02:06:01.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27042.400485 secs 2024-04-27T02:06:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27042.400485 secs 2024-04-27T02:06:02.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:01 smithi119 ceph-mon[32752]: pgmap v15011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:02.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27043.227645 secs 2024-04-27T02:06:02.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:01 smithi167 ceph-mon[32751]: pgmap v15011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:02.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27043.227645 secs 2024-04-27T02:06:02.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:01 smithi090 ceph-mon[29180]: pgmap v15011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:02.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27043.227645 secs 2024-04-27T02:06:04.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:03 smithi090 ceph-mon[29180]: pgmap v15012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:04.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:03 smithi119 ceph-mon[32752]: pgmap v15012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:04.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:03 smithi167 ceph-mon[32751]: pgmap v15012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:06.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:05 smithi119 ceph-mon[32752]: pgmap v15013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:06.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:05 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27047.400569 secs 2024-04-27T02:06:06.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:05 smithi167 ceph-mon[32751]: pgmap v15013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:06.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:05 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27047.400569 secs 2024-04-27T02:06:06.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:05 smithi090 ceph-mon[29180]: pgmap v15013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:06.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:05 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27047.400569 secs 2024-04-27T02:06:07.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27048.227776 secs 2024-04-27T02:06:07.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27048.227776 secs 2024-04-27T02:06:07.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27048.227776 secs 2024-04-27T02:06:08.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:07 smithi119 ceph-mon[32752]: pgmap v15014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:08.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:07 smithi167 ceph-mon[32751]: pgmap v15014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:08.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:07 smithi090 ceph-mon[29180]: pgmap v15014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:10.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:09 smithi119 ceph-mon[32752]: pgmap v15015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:10.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:09 smithi167 ceph-mon[32751]: pgmap v15015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:10.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:09 smithi090 ceph-mon[29180]: pgmap v15015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:11.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27052.400653 secs 2024-04-27T02:06:11.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27052.400653 secs 2024-04-27T02:06:11.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27052.400653 secs 2024-04-27T02:06:12.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:11 smithi119 ceph-mon[32752]: pgmap v15016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:12.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27053.227890 secs 2024-04-27T02:06:12.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:11 smithi167 ceph-mon[32751]: pgmap v15016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:12.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27053.227890 secs 2024-04-27T02:06:12.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:11 smithi090 ceph-mon[29180]: pgmap v15016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:12.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27053.227890 secs 2024-04-27T02:06:14.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:13 smithi119 ceph-mon[32752]: pgmap v15017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:14.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:13 smithi167 ceph-mon[32751]: pgmap v15017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:14.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:13 smithi090 ceph-mon[29180]: pgmap v15017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:16.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:15 smithi119 ceph-mon[32752]: pgmap v15018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:16.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27057.400739 secs 2024-04-27T02:06:16.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:15 smithi167 ceph-mon[32751]: pgmap v15018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:16.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27057.400739 secs 2024-04-27T02:06:16.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:15 smithi090 ceph-mon[29180]: pgmap v15018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:16.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27057.400739 secs 2024-04-27T02:06:17.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27058.228046 secs 2024-04-27T02:06:17.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27058.228046 secs 2024-04-27T02:06:17.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27058.228046 secs 2024-04-27T02:06:18.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:17 smithi119 ceph-mon[32752]: pgmap v15019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:18.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:17 smithi167 ceph-mon[32751]: pgmap v15019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:18.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:17 smithi090 ceph-mon[29180]: pgmap v15019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:20.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:19 smithi119 ceph-mon[32752]: pgmap v15020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:20.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:19 smithi167 ceph-mon[32751]: pgmap v15020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:20.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:19 smithi090 ceph-mon[29180]: pgmap v15020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:21.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27062.400823 secs 2024-04-27T02:06:21.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27062.400823 secs 2024-04-27T02:06:21.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27062.400823 secs 2024-04-27T02:06:22.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:21 smithi119 ceph-mon[32752]: pgmap v15021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:22.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27063.228184 secs 2024-04-27T02:06:22.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:21 smithi167 ceph-mon[32751]: pgmap v15021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:22.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27063.228184 secs 2024-04-27T02:06:22.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:21 smithi090 ceph-mon[29180]: pgmap v15021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:22.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27063.228184 secs 2024-04-27T02:06:24.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:23 smithi119 ceph-mon[32752]: pgmap v15022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:24.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:23 smithi167 ceph-mon[32751]: pgmap v15022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:24.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:23 smithi090 ceph-mon[29180]: pgmap v15022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:26.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:25 smithi119 ceph-mon[32752]: pgmap v15023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:26.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27067.400911 secs 2024-04-27T02:06:26.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:25 smithi167 ceph-mon[32751]: pgmap v15023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:26.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27067.400911 secs 2024-04-27T02:06:26.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:25 smithi090 ceph-mon[29180]: pgmap v15023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:26.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27067.400911 secs 2024-04-27T02:06:27.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27068.228297 secs 2024-04-27T02:06:27.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27068.228297 secs 2024-04-27T02:06:27.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27068.228297 secs 2024-04-27T02:06:28.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:27 smithi119 ceph-mon[32752]: pgmap v15024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:28.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:27 smithi167 ceph-mon[32751]: pgmap v15024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:28.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:27 smithi090 ceph-mon[29180]: pgmap v15024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:30.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:29 smithi119 ceph-mon[32752]: pgmap v15025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:30.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:29 smithi167 ceph-mon[32751]: pgmap v15025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:30.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:29 smithi090 ceph-mon[29180]: pgmap v15025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:31.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27072.400996 secs 2024-04-27T02:06:31.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27072.400996 secs 2024-04-27T02:06:31.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27072.400996 secs 2024-04-27T02:06:32.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:31 smithi119 ceph-mon[32752]: pgmap v15026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:32.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27073.228413 secs 2024-04-27T02:06:32.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:31 smithi167 ceph-mon[32751]: pgmap v15026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:32.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27073.228413 secs 2024-04-27T02:06:32.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:31 smithi090 ceph-mon[29180]: pgmap v15026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:32.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27073.228413 secs 2024-04-27T02:06:34.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:33 smithi119 ceph-mon[32752]: pgmap v15027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:34.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:33 smithi167 ceph-mon[32751]: pgmap v15027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:34.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:33 smithi090 ceph-mon[29180]: pgmap v15027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:36.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:35 smithi119 ceph-mon[32752]: pgmap v15028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:36.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27077.401078 secs 2024-04-27T02:06:36.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:35 smithi167 ceph-mon[32751]: pgmap v15028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:36.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27077.401078 secs 2024-04-27T02:06:36.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:35 smithi090 ceph-mon[29180]: pgmap v15028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:36.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27077.401078 secs 2024-04-27T02:06:37.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27078.228474 secs 2024-04-27T02:06:37.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:06:37.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27078.228474 secs 2024-04-27T02:06:37.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:06:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27078.228474 secs 2024-04-27T02:06:37.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:06:38.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:38 smithi119 ceph-mon[32752]: pgmap v15029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:38.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:06:38.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:06:38.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:06:38.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:38 smithi167 ceph-mon[32751]: pgmap v15029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:38.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:06:38.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:06:38.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:06:38.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:38 smithi090 ceph-mon[29180]: pgmap v15029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:38.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:06:38.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:06:38.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:06:39.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:39 smithi119 ceph-mon[32752]: pgmap v15030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:06:39.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:39 smithi119 ceph-mon[32752]: pgmap v15031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:06:39.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:39 smithi167 ceph-mon[32751]: pgmap v15030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:06:39.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:39 smithi167 ceph-mon[32751]: pgmap v15031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:06:39.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:39 smithi090 ceph-mon[29180]: pgmap v15030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:06:39.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:39 smithi090 ceph-mon[29180]: pgmap v15031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:06:41.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:41 smithi119 ceph-mon[32752]: pgmap v15032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:41.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27082.401163 secs 2024-04-27T02:06:41.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:41 smithi167 ceph-mon[32751]: pgmap v15032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:41.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27082.401163 secs 2024-04-27T02:06:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:41 smithi090 ceph-mon[29180]: pgmap v15032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27082.401163 secs 2024-04-27T02:06:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27083.228587 secs 2024-04-27T02:06:42.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27083.228587 secs 2024-04-27T02:06:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27083.228587 secs 2024-04-27T02:06:43.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:43 smithi119 ceph-mon[32752]: pgmap v15033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:43.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:43 smithi167 ceph-mon[32751]: pgmap v15033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:43.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:43 smithi090 ceph-mon[29180]: pgmap v15033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:45.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:45 smithi119 ceph-mon[32752]: pgmap v15034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:45.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:45 smithi167 ceph-mon[32751]: pgmap v15034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:45.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:45 smithi090 ceph-mon[29180]: pgmap v15034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:46.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27087.401246 secs 2024-04-27T02:06:46.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27087.401246 secs 2024-04-27T02:06:46.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27087.401246 secs 2024-04-27T02:06:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:47 smithi119 ceph-mon[32752]: pgmap v15035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27088.228704 secs 2024-04-27T02:06:47.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:47 smithi167 ceph-mon[32751]: pgmap v15035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:47.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27088.228704 secs 2024-04-27T02:06:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:47 smithi090 ceph-mon[29180]: pgmap v15035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:06:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27088.228704 secs 2024-04-27T02:06:49.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:49 smithi119 ceph-mon[32752]: pgmap v15036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:06:49.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:49 smithi167 ceph-mon[32751]: pgmap v15036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:06:49.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:49 smithi090 ceph-mon[29180]: pgmap v15036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:06:51.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:51 smithi119 ceph-mon[32752]: pgmap v15037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:51.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27092.401332 secs 2024-04-27T02:06:51.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:51 smithi167 ceph-mon[32751]: pgmap v15037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:51.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27092.401332 secs 2024-04-27T02:06:51.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:51 smithi090 ceph-mon[29180]: pgmap v15037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:51.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27092.401332 secs 2024-04-27T02:06:52.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27093.228765 secs 2024-04-27T02:06:52.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27093.228765 secs 2024-04-27T02:06:52.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27093.228765 secs 2024-04-27T02:06:53.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:53 smithi119 ceph-mon[32752]: pgmap v15038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:53.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:53 smithi167 ceph-mon[32751]: pgmap v15038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:53.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:53 smithi090 ceph-mon[29180]: pgmap v15038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:55.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:55 smithi119 ceph-mon[32752]: pgmap v15039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:55.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:55 smithi167 ceph-mon[32751]: pgmap v15039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:55.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:55 smithi090 ceph-mon[29180]: pgmap v15039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:06:56.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27097.401418 secs 2024-04-27T02:06:56.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27097.401418 secs 2024-04-27T02:06:56.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27097.401418 secs 2024-04-27T02:06:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:57 smithi119 ceph-mon[32752]: pgmap v15040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27098.228886 secs 2024-04-27T02:06:57.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:57 smithi167 ceph-mon[32751]: pgmap v15040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:57.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27098.228886 secs 2024-04-27T02:06:57.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:57 smithi090 ceph-mon[29180]: pgmap v15040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:57.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27098.228886 secs 2024-04-27T02:06:59.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:06:59 smithi167 ceph-mon[32751]: pgmap v15041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:59.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:06:59 smithi090 ceph-mon[29180]: pgmap v15041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:06:59.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:06:59 smithi119 ceph-mon[32752]: pgmap v15041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:01.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:01 smithi167 ceph-mon[32751]: pgmap v15042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:01.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27102.401501 secs 2024-04-27T02:07:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:01 smithi090 ceph-mon[29180]: pgmap v15042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:01.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27102.401501 secs 2024-04-27T02:07:01.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:01 smithi119 ceph-mon[32752]: pgmap v15042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:01.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27102.401501 secs 2024-04-27T02:07:02.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27103.229019 secs 2024-04-27T02:07:02.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27103.229019 secs 2024-04-27T02:07:02.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27103.229019 secs 2024-04-27T02:07:03.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:03 smithi167 ceph-mon[32751]: pgmap v15043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:03.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:03 smithi090 ceph-mon[29180]: pgmap v15043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:03.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:03 smithi119 ceph-mon[32752]: pgmap v15043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:05.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:05 smithi090 ceph-mon[29180]: pgmap v15044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:05.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:05 smithi119 ceph-mon[32752]: pgmap v15044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:05.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:05 smithi167 ceph-mon[32751]: pgmap v15044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:06.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27107.401585 secs 2024-04-27T02:07:06.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27107.401585 secs 2024-04-27T02:07:06.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27107.401585 secs 2024-04-27T02:07:07.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:07 smithi090 ceph-mon[29180]: pgmap v15045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:07.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27108.229159 secs 2024-04-27T02:07:07.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:07 smithi119 ceph-mon[32752]: pgmap v15045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:07.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27108.229159 secs 2024-04-27T02:07:07.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:07 smithi167 ceph-mon[32751]: pgmap v15045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:07.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27108.229159 secs 2024-04-27T02:07:09.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:09 smithi090 ceph-mon[29180]: pgmap v15046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:09.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:09 smithi119 ceph-mon[32752]: pgmap v15046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:09.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:09 smithi167 ceph-mon[32751]: pgmap v15046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:11.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:11 smithi090 ceph-mon[29180]: pgmap v15047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:11.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27112.401671 secs 2024-04-27T02:07:11.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:11 smithi119 ceph-mon[32752]: pgmap v15047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:11.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27112.401671 secs 2024-04-27T02:07:11.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:11 smithi167 ceph-mon[32751]: pgmap v15047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:11.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27112.401671 secs 2024-04-27T02:07:12.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27113.229276 secs 2024-04-27T02:07:12.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27113.229276 secs 2024-04-27T02:07:12.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27113.229276 secs 2024-04-27T02:07:13.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:13 smithi090 ceph-mon[29180]: pgmap v15048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:13.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:13 smithi119 ceph-mon[32752]: pgmap v15048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:13.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:13 smithi167 ceph-mon[32751]: pgmap v15048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:15.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:15 smithi090 ceph-mon[29180]: pgmap v15049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:15.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:15 smithi119 ceph-mon[32752]: pgmap v15049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:15.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:15 smithi167 ceph-mon[32751]: pgmap v15049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:16.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27117.401755 secs 2024-04-27T02:07:16.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27117.401755 secs 2024-04-27T02:07:16.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27117.401755 secs 2024-04-27T02:07:17.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:17 smithi090 ceph-mon[29180]: pgmap v15050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:17.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27118.229373 secs 2024-04-27T02:07:17.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:17 smithi119 ceph-mon[32752]: pgmap v15050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:17.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27118.229373 secs 2024-04-27T02:07:17.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:17 smithi167 ceph-mon[32751]: pgmap v15050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:17.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27118.229373 secs 2024-04-27T02:07:19.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:19 smithi090 ceph-mon[29180]: pgmap v15051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:19.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:19 smithi119 ceph-mon[32752]: pgmap v15051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:19.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:19 smithi167 ceph-mon[32751]: pgmap v15051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:21.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:21 smithi090 ceph-mon[29180]: pgmap v15052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:21.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27122.401850 secs 2024-04-27T02:07:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:21 smithi119 ceph-mon[32752]: pgmap v15052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:21.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27122.401850 secs 2024-04-27T02:07:21.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:21 smithi167 ceph-mon[32751]: pgmap v15052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:21.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27122.401850 secs 2024-04-27T02:07:22.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27123.229480 secs 2024-04-27T02:07:22.433 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27123.229480 secs 2024-04-27T02:07:22.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27123.229480 secs 2024-04-27T02:07:23.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:23 smithi090 ceph-mon[29180]: pgmap v15053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:23.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:23 smithi119 ceph-mon[32752]: pgmap v15053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:23.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:23 smithi167 ceph-mon[32751]: pgmap v15053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:25.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:25 smithi090 ceph-mon[29180]: pgmap v15054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:25.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:25 smithi119 ceph-mon[32752]: pgmap v15054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:25.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:25 smithi167 ceph-mon[32751]: pgmap v15054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:26.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27127.401912 secs 2024-04-27T02:07:26.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27127.401912 secs 2024-04-27T02:07:26.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27127.401912 secs 2024-04-27T02:07:27.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:27 smithi090 ceph-mon[29180]: pgmap v15055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:27.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27128.229604 secs 2024-04-27T02:07:27.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:27 smithi119 ceph-mon[32752]: pgmap v15055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:27.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27128.229604 secs 2024-04-27T02:07:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:27 smithi167 ceph-mon[32751]: pgmap v15055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:27.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27128.229604 secs 2024-04-27T02:07:29.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:29 smithi119 ceph-mon[32752]: pgmap v15056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:29.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:29 smithi167 ceph-mon[32751]: pgmap v15056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:29.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:29 smithi090 ceph-mon[29180]: pgmap v15056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:31.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:31 smithi090 ceph-mon[29180]: pgmap v15057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:31.407 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27132.401996 secs 2024-04-27T02:07:31.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:31 smithi119 ceph-mon[32752]: pgmap v15057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:31.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27132.401996 secs 2024-04-27T02:07:31.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:31 smithi167 ceph-mon[32751]: pgmap v15057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:31.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27132.401996 secs 2024-04-27T02:07:32.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27133.229710 secs 2024-04-27T02:07:32.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27133.229710 secs 2024-04-27T02:07:32.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27133.229710 secs 2024-04-27T02:07:33.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:33 smithi119 ceph-mon[32752]: pgmap v15058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:33.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:33 smithi167 ceph-mon[32751]: pgmap v15058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:33.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:33 smithi090 ceph-mon[29180]: pgmap v15058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:34.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:34 smithi119 ceph-mon[32752]: pgmap v15059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:34.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:34 smithi167 ceph-mon[32751]: pgmap v15059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:34.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:34 smithi090 ceph-mon[29180]: pgmap v15059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:36.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27137.402100 secs 2024-04-27T02:07:36.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:36 smithi090 ceph-mon[29180]: pgmap v15060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:36.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27138.229829 secs 2024-04-27T02:07:37.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27137.402100 secs 2024-04-27T02:07:37.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:36 smithi119 ceph-mon[32752]: pgmap v15060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:37.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27138.229829 secs 2024-04-27T02:07:37.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27137.402100 secs 2024-04-27T02:07:37.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:36 smithi167 ceph-mon[32751]: pgmap v15060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:37.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27138.229829 secs 2024-04-27T02:07:39.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:38 smithi119 ceph-mon[32752]: pgmap v15061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:39.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:07:39.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:07:39.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:07:39.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:07:39.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:38 smithi167 ceph-mon[32751]: pgmap v15061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:39.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:07:39.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:07:39.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:07:39.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:07:39.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:38 smithi090 ceph-mon[29180]: pgmap v15061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:39.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:07:39.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:07:39.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:07:39.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:07:40.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:39 smithi119 ceph-mon[32752]: pgmap v15062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:07:40.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:39 smithi119 ceph-mon[32752]: pgmap v15063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:40.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:39 smithi167 ceph-mon[32751]: pgmap v15062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:07:40.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:39 smithi167 ceph-mon[32751]: pgmap v15063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:39 smithi090 ceph-mon[29180]: pgmap v15062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:07:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:39 smithi090 ceph-mon[29180]: pgmap v15063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:42.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27142.402185 secs 2024-04-27T02:07:42.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:41 smithi119 ceph-mon[32752]: pgmap v15064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:42.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27143.229941 secs 2024-04-27T02:07:42.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27142.402185 secs 2024-04-27T02:07:42.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:41 smithi167 ceph-mon[32751]: pgmap v15064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:42.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27143.229941 secs 2024-04-27T02:07:42.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27142.402185 secs 2024-04-27T02:07:42.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:41 smithi090 ceph-mon[29180]: pgmap v15064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:42.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27143.229941 secs 2024-04-27T02:07:44.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:43 smithi119 ceph-mon[32752]: pgmap v15065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:07:44.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:43 smithi167 ceph-mon[32751]: pgmap v15065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:07:44.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:43 smithi090 ceph-mon[29180]: pgmap v15065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:07:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:45 smithi119 ceph-mon[32752]: pgmap v15066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:45 smithi167 ceph-mon[32751]: pgmap v15066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:45 smithi090 ceph-mon[29180]: pgmap v15066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:47.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27147.402232 secs 2024-04-27T02:07:47.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27148.230060 secs 2024-04-27T02:07:47.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27147.402232 secs 2024-04-27T02:07:47.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27148.230060 secs 2024-04-27T02:07:47.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27147.402232 secs 2024-04-27T02:07:47.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27148.230060 secs 2024-04-27T02:07:48.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:47 smithi119 ceph-mon[32752]: pgmap v15067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:48.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:47 smithi167 ceph-mon[32751]: pgmap v15067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:48.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:47 smithi090 ceph-mon[29180]: pgmap v15067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:07:50.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:49 smithi119 ceph-mon[32752]: pgmap v15068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:07:50.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:49 smithi167 ceph-mon[32751]: pgmap v15068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:07:50.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:49 smithi090 ceph-mon[29180]: pgmap v15068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:07:51.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27152.402317 secs 2024-04-27T02:07:51.907 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:51 smithi090 ceph-mon[29180]: pgmap v15069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:51.908 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27153.230162 secs 2024-04-27T02:07:52.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27152.402317 secs 2024-04-27T02:07:52.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:51 smithi119 ceph-mon[32752]: pgmap v15069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:52.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27153.230162 secs 2024-04-27T02:07:52.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27152.402317 secs 2024-04-27T02:07:52.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:51 smithi167 ceph-mon[32751]: pgmap v15069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:52.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27153.230162 secs 2024-04-27T02:07:54.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:53 smithi119 ceph-mon[32752]: pgmap v15070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:54.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:53 smithi167 ceph-mon[32751]: pgmap v15070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:54.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:53 smithi090 ceph-mon[29180]: pgmap v15070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:56.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:55 smithi119 ceph-mon[32752]: pgmap v15071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:56.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:55 smithi167 ceph-mon[32751]: pgmap v15071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:56.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:55 smithi090 ceph-mon[29180]: pgmap v15071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:07:57.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27157.402401 secs 2024-04-27T02:07:57.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27158.230298 secs 2024-04-27T02:07:57.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27157.402401 secs 2024-04-27T02:07:57.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27158.230298 secs 2024-04-27T02:07:57.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27157.402401 secs 2024-04-27T02:07:57.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27158.230298 secs 2024-04-27T02:07:58.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:57 smithi119 ceph-mon[32752]: pgmap v15072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:58.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:57 smithi167 ceph-mon[32751]: pgmap v15072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:07:58.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:57 smithi090 ceph-mon[29180]: pgmap v15072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:00.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:07:59 smithi119 ceph-mon[32752]: pgmap v15073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:00.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:07:59 smithi167 ceph-mon[32751]: pgmap v15073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:00.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:07:59 smithi090 ceph-mon[29180]: pgmap v15073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:02.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27162.402505 secs 2024-04-27T02:08:02.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:01 smithi119 ceph-mon[32752]: pgmap v15074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:02.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27163.230367 secs 2024-04-27T02:08:02.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27162.402505 secs 2024-04-27T02:08:02.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:01 smithi167 ceph-mon[32751]: pgmap v15074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:02.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27163.230367 secs 2024-04-27T02:08:02.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27162.402505 secs 2024-04-27T02:08:02.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:01 smithi090 ceph-mon[29180]: pgmap v15074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:02.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27163.230367 secs 2024-04-27T02:08:04.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:03 smithi119 ceph-mon[32752]: pgmap v15075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:04.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:03 smithi167 ceph-mon[32751]: pgmap v15075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:04.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:03 smithi090 ceph-mon[29180]: pgmap v15075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:06.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:05 smithi119 ceph-mon[32752]: pgmap v15076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:06.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:05 smithi167 ceph-mon[32751]: pgmap v15076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:06.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:05 smithi090 ceph-mon[29180]: pgmap v15076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:07.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27167.402609 secs 2024-04-27T02:08:07.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27168.230487 secs 2024-04-27T02:08:07.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27167.402609 secs 2024-04-27T02:08:07.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27168.230487 secs 2024-04-27T02:08:07.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27167.402609 secs 2024-04-27T02:08:07.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27168.230487 secs 2024-04-27T02:08:08.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:07 smithi119 ceph-mon[32752]: pgmap v15077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:08.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:07 smithi167 ceph-mon[32751]: pgmap v15077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:08.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:07 smithi090 ceph-mon[29180]: pgmap v15077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:08.605 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:08:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:08:08.297+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:08:08.605 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:08:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:08:08.297+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:08:08.605 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:08:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:08:08.297+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:08:08.605 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:08:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:08:08.297+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:08:08.605 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:08:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:08:08.297+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:08:10.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:09 smithi119 ceph-mon[32752]: pgmap v15078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-27T02:08:10.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:09 smithi167 ceph-mon[32751]: pgmap v15078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-27T02:08:10.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:09 smithi090 ceph-mon[29180]: pgmap v15078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-27T02:08:12.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27172.402712 secs 2024-04-27T02:08:12.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:11 smithi119 ceph-mon[32752]: pgmap v15079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:12.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27173.230620 secs 2024-04-27T02:08:12.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27172.402712 secs 2024-04-27T02:08:12.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:11 smithi167 ceph-mon[32751]: pgmap v15079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:12.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27173.230620 secs 2024-04-27T02:08:12.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27172.402712 secs 2024-04-27T02:08:12.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:11 smithi090 ceph-mon[29180]: pgmap v15079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:12.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27173.230620 secs 2024-04-27T02:08:14.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:13 smithi119 ceph-mon[32752]: pgmap v15080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:14.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:13 smithi167 ceph-mon[32751]: pgmap v15080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:14.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:13 smithi090 ceph-mon[29180]: pgmap v15080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:16.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:15 smithi119 ceph-mon[32752]: pgmap v15081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:16.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:15 smithi167 ceph-mon[32751]: pgmap v15081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:16.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:15 smithi090 ceph-mon[29180]: pgmap v15081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:17.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27177.402817 secs 2024-04-27T02:08:17.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27178.230735 secs 2024-04-27T02:08:17.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27177.402817 secs 2024-04-27T02:08:17.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27178.230735 secs 2024-04-27T02:08:17.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27177.402817 secs 2024-04-27T02:08:17.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27178.230735 secs 2024-04-27T02:08:18.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:17 smithi119 ceph-mon[32752]: pgmap v15082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:18.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:17 smithi167 ceph-mon[32751]: pgmap v15082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:18.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:17 smithi090 ceph-mon[29180]: pgmap v15082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:20.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:19 smithi119 ceph-mon[32752]: pgmap v15083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:20.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:19 smithi167 ceph-mon[32751]: pgmap v15083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:20.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:19 smithi090 ceph-mon[29180]: pgmap v15083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:08:22.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27182.402908 secs 2024-04-27T02:08:22.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:21 smithi119 ceph-mon[32752]: pgmap v15084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:22.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27183.230841 secs 2024-04-27T02:08:22.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27182.402908 secs 2024-04-27T02:08:22.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:21 smithi167 ceph-mon[32751]: pgmap v15084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:22.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27183.230841 secs 2024-04-27T02:08:22.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27182.402908 secs 2024-04-27T02:08:22.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:21 smithi090 ceph-mon[29180]: pgmap v15084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-27T02:08:22.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27183.230841 secs 2024-04-27T02:08:23.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:23 smithi090 ceph-mon[29180]: pgmap v15085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:08:24.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:23 smithi119 ceph-mon[32752]: pgmap v15085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:08:24.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:23 smithi167 ceph-mon[32751]: pgmap v15085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:08:26.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:25 smithi119 ceph-mon[32752]: pgmap v15086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:08:26.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:25 smithi167 ceph-mon[32751]: pgmap v15086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:08:26.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:25 smithi090 ceph-mon[29180]: pgmap v15086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:08:27.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27187.403010 secs 2024-04-27T02:08:27.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27188.230957 secs 2024-04-27T02:08:27.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27187.403010 secs 2024-04-27T02:08:27.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27188.230957 secs 2024-04-27T02:08:27.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27187.403010 secs 2024-04-27T02:08:27.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27188.230957 secs 2024-04-27T02:08:28.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:27 smithi119 ceph-mon[32752]: pgmap v15087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:28.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:27 smithi167 ceph-mon[32751]: pgmap v15087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:28.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:27 smithi090 ceph-mon[29180]: pgmap v15087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:30.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:29 smithi119 ceph-mon[32752]: pgmap v15088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:30.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:29 smithi167 ceph-mon[32751]: pgmap v15088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:29 smithi090 ceph-mon[29180]: pgmap v15088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:32.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27192.403098 secs 2024-04-27T02:08:32.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:31 smithi119 ceph-mon[32752]: pgmap v15089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:32.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27193.231024 secs 2024-04-27T02:08:32.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27192.403098 secs 2024-04-27T02:08:32.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:31 smithi167 ceph-mon[32751]: pgmap v15089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:32.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27193.231024 secs 2024-04-27T02:08:32.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27192.403098 secs 2024-04-27T02:08:32.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:31 smithi090 ceph-mon[29180]: pgmap v15089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:32.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27193.231024 secs 2024-04-27T02:08:34.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:33 smithi119 ceph-mon[32752]: pgmap v15090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:34.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:33 smithi167 ceph-mon[32751]: pgmap v15090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:34.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:33 smithi090 ceph-mon[29180]: pgmap v15090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:36.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:35 smithi119 ceph-mon[32752]: pgmap v15091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:36.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:35 smithi167 ceph-mon[32751]: pgmap v15091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:36.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:35 smithi090 ceph-mon[29180]: pgmap v15091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:37.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27197.403201 secs 2024-04-27T02:08:37.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27198.231141 secs 2024-04-27T02:08:37.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27197.403201 secs 2024-04-27T02:08:37.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27198.231141 secs 2024-04-27T02:08:37.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27197.403201 secs 2024-04-27T02:08:37.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27198.231141 secs 2024-04-27T02:08:38.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:37 smithi119 ceph-mon[32752]: pgmap v15092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:38.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:37 smithi167 ceph-mon[32751]: pgmap v15092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:38.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:37 smithi090 ceph-mon[29180]: pgmap v15092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:39.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:08:39.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:08:39.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:08:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:39 smithi090 ceph-mon[29180]: pgmap v15093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:08:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:08:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:39 smithi090 ceph-mon[29180]: pgmap v15094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:08:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:39 smithi090 ceph-mon[29180]: pgmap v15095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:08:40.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:08:40.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:39 smithi119 ceph-mon[32752]: pgmap v15093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:40.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:08:40.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:08:40.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:39 smithi119 ceph-mon[32752]: pgmap v15094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:08:40.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:39 smithi119 ceph-mon[32752]: pgmap v15095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:08:40.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:08:40.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:39 smithi167 ceph-mon[32751]: pgmap v15093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:40.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:08:40.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:08:40.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:39 smithi167 ceph-mon[32751]: pgmap v15094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:08:40.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:39 smithi167 ceph-mon[32751]: pgmap v15095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:08:40.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:08:41.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27202.403304 secs 2024-04-27T02:08:41.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27202.403304 secs 2024-04-27T02:08:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27202.403304 secs 2024-04-27T02:08:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:41 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 27203.231235 secs 2024-04-27T02:08:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:41 smithi119 ceph-mon[32752]: slow request 5764.273984 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:08:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:41 smithi119 ceph-mon[32752]: pgmap v15096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:08:42.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:41 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 27203.231235 secs 2024-04-27T02:08:42.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:41 smithi167 ceph-mon[32751]: slow request 5764.273984 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:08:42.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:41 smithi167 ceph-mon[32751]: pgmap v15096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:08:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:41 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 27203.231235 secs 2024-04-27T02:08:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:41 smithi090 ceph-mon[29180]: slow request 5764.273984 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:08:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:41 smithi090 ceph-mon[29180]: pgmap v15096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:08:43.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:43 smithi119 ceph-mon[32752]: pgmap v15097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:08:43.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:43 smithi167 ceph-mon[32751]: pgmap v15097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:08:43.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:43 smithi090 ceph-mon[29180]: pgmap v15097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:08:45.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:45 smithi119 ceph-mon[32752]: pgmap v15098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:08:45.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:45 smithi167 ceph-mon[32751]: pgmap v15098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:08:45.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:45 smithi090 ceph-mon[29180]: pgmap v15098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:08:46.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27207.403407 secs 2024-04-27T02:08:46.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27207.403407 secs 2024-04-27T02:08:46.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27207.403407 secs 2024-04-27T02:08:47.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27208.231308 secs 2024-04-27T02:08:47.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:47 smithi119 ceph-mon[32752]: pgmap v15099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T02:08:47.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27208.231308 secs 2024-04-27T02:08:47.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:47 smithi167 ceph-mon[32751]: pgmap v15099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T02:08:47.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27208.231308 secs 2024-04-27T02:08:47.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:47 smithi090 ceph-mon[29180]: pgmap v15099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T02:08:49.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:49 smithi119 ceph-mon[32752]: pgmap v15100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T02:08:49.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:49 smithi167 ceph-mon[32751]: pgmap v15100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T02:08:49.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:49 smithi090 ceph-mon[29180]: pgmap v15100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T02:08:51.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27212.403465 secs 2024-04-27T02:08:51.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27212.403465 secs 2024-04-27T02:08:51.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27212.403465 secs 2024-04-27T02:08:52.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27213.231421 secs 2024-04-27T02:08:52.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:51 smithi119 ceph-mon[32752]: pgmap v15101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:08:52.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27213.231421 secs 2024-04-27T02:08:52.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:51 smithi167 ceph-mon[32751]: pgmap v15101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:08:52.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27213.231421 secs 2024-04-27T02:08:52.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:51 smithi090 ceph-mon[29180]: pgmap v15101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:08:53.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:53 smithi119 ceph-mon[32752]: pgmap v15102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:08:53.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:53 smithi167 ceph-mon[32751]: pgmap v15102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:08:53.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:53 smithi090 ceph-mon[29180]: pgmap v15102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:08:55.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:55 smithi119 ceph-mon[32752]: pgmap v15103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:55.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:55 smithi167 ceph-mon[32751]: pgmap v15103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:55.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:55 smithi090 ceph-mon[29180]: pgmap v15103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:56.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27217.403549 secs 2024-04-27T02:08:56.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27217.403549 secs 2024-04-27T02:08:56.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27217.403549 secs 2024-04-27T02:08:57.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27218.231536 secs 2024-04-27T02:08:57.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:57 smithi119 ceph-mon[32752]: pgmap v15104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:57.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27218.231536 secs 2024-04-27T02:08:57.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:57 smithi167 ceph-mon[32751]: pgmap v15104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:57.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27218.231536 secs 2024-04-27T02:08:57.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:57 smithi090 ceph-mon[29180]: pgmap v15104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:08:59.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:08:59 smithi119 ceph-mon[32752]: pgmap v15105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:59.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:08:59 smithi167 ceph-mon[32751]: pgmap v15105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:08:59.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:08:59 smithi090 ceph-mon[29180]: pgmap v15105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:01.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:00 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27222.403649 secs 2024-04-27T02:09:01.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:00 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27222.403649 secs 2024-04-27T02:09:01.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:00 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27222.403649 secs 2024-04-27T02:09:02.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27223.231615 secs 2024-04-27T02:09:02.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:01 smithi119 ceph-mon[32752]: pgmap v15106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:02.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27223.231615 secs 2024-04-27T02:09:02.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:01 smithi167 ceph-mon[32751]: pgmap v15106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:02.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27223.231615 secs 2024-04-27T02:09:02.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:01 smithi090 ceph-mon[29180]: pgmap v15106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:03.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:03 smithi119 ceph-mon[32752]: pgmap v15107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:03.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:03 smithi167 ceph-mon[32751]: pgmap v15107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:03.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:03 smithi090 ceph-mon[29180]: pgmap v15107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:05.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:05 smithi119 ceph-mon[32752]: pgmap v15108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:05.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:05 smithi167 ceph-mon[32751]: pgmap v15108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:05.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:05 smithi090 ceph-mon[29180]: pgmap v15108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:06.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27227.403748 secs 2024-04-27T02:09:06.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27227.403748 secs 2024-04-27T02:09:06.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27227.403748 secs 2024-04-27T02:09:07.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27228.231711 secs 2024-04-27T02:09:07.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:07 smithi119 ceph-mon[32752]: pgmap v15109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:07.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27228.231711 secs 2024-04-27T02:09:07.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:07 smithi167 ceph-mon[32751]: pgmap v15109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:07.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27228.231711 secs 2024-04-27T02:09:07.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:07 smithi090 ceph-mon[29180]: pgmap v15109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:09.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:09 smithi119 ceph-mon[32752]: pgmap v15110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:09.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:09 smithi167 ceph-mon[32751]: pgmap v15110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:09.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:09 smithi090 ceph-mon[29180]: pgmap v15110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:11.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27232.403816 secs 2024-04-27T02:09:11.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27232.403816 secs 2024-04-27T02:09:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27232.403816 secs 2024-04-27T02:09:12.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27233.231869 secs 2024-04-27T02:09:12.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:11 smithi119 ceph-mon[32752]: pgmap v15111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:12.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27233.231869 secs 2024-04-27T02:09:12.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:11 smithi167 ceph-mon[32751]: pgmap v15111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:12.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27233.231869 secs 2024-04-27T02:09:12.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:11 smithi090 ceph-mon[29180]: pgmap v15111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:13.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:13 smithi119 ceph-mon[32752]: pgmap v15112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:13.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:13 smithi167 ceph-mon[32751]: pgmap v15112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:13.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:13 smithi090 ceph-mon[29180]: pgmap v15112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:09:15.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:15 smithi119 ceph-mon[32752]: pgmap v15113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:15.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:15 smithi167 ceph-mon[32751]: pgmap v15113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:15.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:15 smithi090 ceph-mon[29180]: pgmap v15113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:09:16.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27237.403874 secs 2024-04-27T02:09:16.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27237.403874 secs 2024-04-27T02:09:16.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27237.403874 secs 2024-04-27T02:09:17.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27238.231990 secs 2024-04-27T02:09:17.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:17 smithi119 ceph-mon[32752]: pgmap v15114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:17.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27238.231990 secs 2024-04-27T02:09:17.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:17 smithi167 ceph-mon[32751]: pgmap v15114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:17.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27238.231990 secs 2024-04-27T02:09:17.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:17 smithi090 ceph-mon[29180]: pgmap v15114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:19.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:19 smithi119 ceph-mon[32752]: pgmap v15115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:19.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:19 smithi167 ceph-mon[32751]: pgmap v15115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:19.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:19 smithi090 ceph-mon[29180]: pgmap v15115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:21.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27242.403963 secs 2024-04-27T02:09:21.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27242.403963 secs 2024-04-27T02:09:21.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27242.403963 secs 2024-04-27T02:09:22.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27243.232110 secs 2024-04-27T02:09:22.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:21 smithi119 ceph-mon[32752]: pgmap v15116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:22.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27243.232110 secs 2024-04-27T02:09:22.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:21 smithi167 ceph-mon[32751]: pgmap v15116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:22.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27243.232110 secs 2024-04-27T02:09:22.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:21 smithi090 ceph-mon[29180]: pgmap v15116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:23.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:23 smithi119 ceph-mon[32752]: pgmap v15117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:23.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:23 smithi167 ceph-mon[32751]: pgmap v15117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:23.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:23 smithi090 ceph-mon[29180]: pgmap v15117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:25.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:25 smithi119 ceph-mon[32752]: pgmap v15118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:25.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:25 smithi167 ceph-mon[32751]: pgmap v15118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:25.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:25 smithi090 ceph-mon[29180]: pgmap v15118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:26.480 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27247.404049 secs 2024-04-27T02:09:26.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27247.404049 secs 2024-04-27T02:09:26.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27247.404049 secs 2024-04-27T02:09:27.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27248.232226 secs 2024-04-27T02:09:27.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:27 smithi119 ceph-mon[32752]: pgmap v15119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:27.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27248.232226 secs 2024-04-27T02:09:27.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:27 smithi167 ceph-mon[32751]: pgmap v15119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:27.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27248.232226 secs 2024-04-27T02:09:27.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:27 smithi090 ceph-mon[29180]: pgmap v15119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:29.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:29 smithi119 ceph-mon[32752]: pgmap v15120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:29.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:29 smithi167 ceph-mon[32751]: pgmap v15120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:29.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:29 smithi090 ceph-mon[29180]: pgmap v15120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:31.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27252.404137 secs 2024-04-27T02:09:31.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27252.404137 secs 2024-04-27T02:09:31.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27252.404137 secs 2024-04-27T02:09:32.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27253.232365 secs 2024-04-27T02:09:32.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:31 smithi119 ceph-mon[32752]: pgmap v15121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:32.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27253.232365 secs 2024-04-27T02:09:32.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:31 smithi167 ceph-mon[32751]: pgmap v15121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:32.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27253.232365 secs 2024-04-27T02:09:32.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:31 smithi090 ceph-mon[29180]: pgmap v15121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:33.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:33 smithi119 ceph-mon[32752]: pgmap v15122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:33.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:33 smithi167 ceph-mon[32751]: pgmap v15122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:33.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:33 smithi090 ceph-mon[29180]: pgmap v15122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:35.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:35 smithi119 ceph-mon[32752]: pgmap v15123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:35.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:35 smithi167 ceph-mon[32751]: pgmap v15123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:35.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:35 smithi090 ceph-mon[29180]: pgmap v15123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:36.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27257.404238 secs 2024-04-27T02:09:36.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27257.404238 secs 2024-04-27T02:09:36.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27257.404238 secs 2024-04-27T02:09:37.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27258.232499 secs 2024-04-27T02:09:37.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:37 smithi119 ceph-mon[32752]: pgmap v15124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:37.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27258.232499 secs 2024-04-27T02:09:37.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:37 smithi167 ceph-mon[32751]: pgmap v15124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:37.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27258.232499 secs 2024-04-27T02:09:37.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:37 smithi090 ceph-mon[29180]: pgmap v15124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:39.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:39 smithi119 ceph-mon[32752]: pgmap v15125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:39.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:39 smithi167 ceph-mon[32751]: pgmap v15125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:39.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:39 smithi090 ceph-mon[29180]: pgmap v15125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:40.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:09:40.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:09:40.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:09:40.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:40 smithi119 ceph-mon[32752]: pgmap v15126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:09:40.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:40 smithi119 ceph-mon[32752]: pgmap v15127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:40.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:09:40.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:09:40.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:09:40.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:09:40.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:40 smithi167 ceph-mon[32751]: pgmap v15126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:09:40.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:40 smithi167 ceph-mon[32751]: pgmap v15127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:40.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:09:40.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:09:40.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:09:40.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:09:40.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:40 smithi090 ceph-mon[29180]: pgmap v15126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:09:40.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:40 smithi090 ceph-mon[29180]: pgmap v15127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:40.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:09:41.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27262.404325 secs 2024-04-27T02:09:41.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27262.404325 secs 2024-04-27T02:09:41.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27262.404325 secs 2024-04-27T02:09:42.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:42 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 27263.232592 secs 2024-04-27T02:09:42.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:42 smithi119 ceph-mon[32752]: slow request 722.838609 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:09:42.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:42 smithi119 ceph-mon[32752]: pgmap v15128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:42.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:42 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 27263.232592 secs 2024-04-27T02:09:42.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:42 smithi167 ceph-mon[32751]: slow request 722.838609 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:09:42.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:42 smithi167 ceph-mon[32751]: pgmap v15128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:42.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:42 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 27263.232592 secs 2024-04-27T02:09:42.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:42 smithi090 ceph-mon[29180]: slow request 722.838609 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:09:42.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:42 smithi090 ceph-mon[29180]: pgmap v15128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:45.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:44 smithi119 ceph-mon[32752]: pgmap v15129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:45.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:44 smithi167 ceph-mon[32751]: pgmap v15129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:45.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:44 smithi090 ceph-mon[29180]: pgmap v15129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:46.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:45 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27267.404417 secs 2024-04-27T02:09:46.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:45 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27267.404417 secs 2024-04-27T02:09:46.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:45 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27267.404417 secs 2024-04-27T02:09:47.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:46 smithi119 ceph-mon[32752]: pgmap v15130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:47.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27268.232680 secs 2024-04-27T02:09:47.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:46 smithi167 ceph-mon[32751]: pgmap v15130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:47.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27268.232680 secs 2024-04-27T02:09:47.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:46 smithi090 ceph-mon[29180]: pgmap v15130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:09:47.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27268.232680 secs 2024-04-27T02:09:49.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:48 smithi119 ceph-mon[32752]: pgmap v15131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:09:49.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:48 smithi167 ceph-mon[32751]: pgmap v15131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:09:49.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:48 smithi090 ceph-mon[29180]: pgmap v15131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:09:51.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:50 smithi119 ceph-mon[32752]: pgmap v15132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:09:51.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27272.404498 secs 2024-04-27T02:09:51.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:50 smithi167 ceph-mon[32751]: pgmap v15132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:09:51.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27272.404498 secs 2024-04-27T02:09:51.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:50 smithi090 ceph-mon[29180]: pgmap v15132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:09:51.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27272.404498 secs 2024-04-27T02:09:52.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27273.232742 secs 2024-04-27T02:09:52.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27273.232742 secs 2024-04-27T02:09:52.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27273.232742 secs 2024-04-27T02:09:53.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:52 smithi119 ceph-mon[32752]: pgmap v15133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:53.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:52 smithi167 ceph-mon[32751]: pgmap v15133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:53.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:52 smithi090 ceph-mon[29180]: pgmap v15133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:55.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:54 smithi119 ceph-mon[32752]: pgmap v15134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:55.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:54 smithi167 ceph-mon[32751]: pgmap v15134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:55.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:54 smithi090 ceph-mon[29180]: pgmap v15134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:09:56.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27277.404583 secs 2024-04-27T02:09:56.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27277.404583 secs 2024-04-27T02:09:56.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27277.404583 secs 2024-04-27T02:09:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:57 smithi119 ceph-mon[32752]: pgmap v15135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27278.232834 secs 2024-04-27T02:09:57.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:57 smithi167 ceph-mon[32751]: pgmap v15135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:57.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27278.232834 secs 2024-04-27T02:09:57.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:57 smithi090 ceph-mon[29180]: pgmap v15135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:57.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27278.232834 secs 2024-04-27T02:09:59.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:09:59 smithi119 ceph-mon[32752]: pgmap v15136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:59.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:09:59 smithi167 ceph-mon[32751]: pgmap v15136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:09:59.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:09:59 smithi090 ceph-mon[29180]: pgmap v15136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:00.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:10:00.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:10:00.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:10:01.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:01 smithi090 ceph-mon[29180]: pgmap v15137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:01.256 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27282.404669 secs 2024-04-27T02:10:01.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:01 smithi119 ceph-mon[32752]: pgmap v15137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:01.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27282.404669 secs 2024-04-27T02:10:01.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:01 smithi167 ceph-mon[32751]: pgmap v15137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:01.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27282.404669 secs 2024-04-27T02:10:02.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27283.232940 secs 2024-04-27T02:10:02.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27283.232940 secs 2024-04-27T02:10:02.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27283.232940 secs 2024-04-27T02:10:03.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:03 smithi119 ceph-mon[32752]: pgmap v15138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:03.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:03 smithi167 ceph-mon[32751]: pgmap v15138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:03.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:03 smithi090 ceph-mon[29180]: pgmap v15138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:05.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:05 smithi119 ceph-mon[32752]: pgmap v15139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:05.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:05 smithi167 ceph-mon[32751]: pgmap v15139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:05.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:05 smithi090 ceph-mon[29180]: pgmap v15139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:06.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27287.404755 secs 2024-04-27T02:10:06.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27287.404755 secs 2024-04-27T02:10:06.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27287.404755 secs 2024-04-27T02:10:07.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:07 smithi119 ceph-mon[32752]: pgmap v15140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:07.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27288.233079 secs 2024-04-27T02:10:07.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:07 smithi167 ceph-mon[32751]: pgmap v15140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:07.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27288.233079 secs 2024-04-27T02:10:07.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:07 smithi090 ceph-mon[29180]: pgmap v15140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:07.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27288.233079 secs 2024-04-27T02:10:09.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:09 smithi119 ceph-mon[32752]: pgmap v15141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:09.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:09 smithi167 ceph-mon[32751]: pgmap v15141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:09.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:09 smithi090 ceph-mon[29180]: pgmap v15141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:11.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:11 smithi119 ceph-mon[32752]: pgmap v15142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:11.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27292.404800 secs 2024-04-27T02:10:11.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:11 smithi167 ceph-mon[32751]: pgmap v15142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:11.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27292.404800 secs 2024-04-27T02:10:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:11 smithi090 ceph-mon[29180]: pgmap v15142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:11.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27292.404800 secs 2024-04-27T02:10:12.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27293.233199 secs 2024-04-27T02:10:12.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27293.233199 secs 2024-04-27T02:10:12.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27293.233199 secs 2024-04-27T02:10:13.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:13 smithi119 ceph-mon[32752]: pgmap v15143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:13.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:13 smithi167 ceph-mon[32751]: pgmap v15143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:13.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:13 smithi090 ceph-mon[29180]: pgmap v15143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:15.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:15 smithi119 ceph-mon[32752]: pgmap v15144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:15.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:15 smithi167 ceph-mon[32751]: pgmap v15144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:15.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:15 smithi090 ceph-mon[29180]: pgmap v15144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:16.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27297.404883 secs 2024-04-27T02:10:16.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27297.404883 secs 2024-04-27T02:10:16.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27297.404883 secs 2024-04-27T02:10:17.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:17 smithi167 ceph-mon[32751]: pgmap v15145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:17.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27298.233335 secs 2024-04-27T02:10:17.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:17 smithi090 ceph-mon[29180]: pgmap v15145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:17.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27298.233335 secs 2024-04-27T02:10:17.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:17 smithi119 ceph-mon[32752]: pgmap v15145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:17.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27298.233335 secs 2024-04-27T02:10:19.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:19 smithi167 ceph-mon[32751]: pgmap v15146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:19.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:19 smithi090 ceph-mon[29180]: pgmap v15146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:19.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:19 smithi119 ceph-mon[32752]: pgmap v15146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:21.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:21 smithi167 ceph-mon[32751]: pgmap v15147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:21.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27302.404964 secs 2024-04-27T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:21 smithi090 ceph-mon[29180]: pgmap v15147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:21.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27302.404964 secs 2024-04-27T02:10:21.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:21 smithi119 ceph-mon[32752]: pgmap v15147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:21.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27302.404964 secs 2024-04-27T02:10:22.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27303.233460 secs 2024-04-27T02:10:22.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27303.233460 secs 2024-04-27T02:10:22.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27303.233460 secs 2024-04-27T02:10:23.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:23 smithi090 ceph-mon[29180]: pgmap v15148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:23.433 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:23 smithi119 ceph-mon[32752]: pgmap v15148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:23.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:23 smithi167 ceph-mon[32751]: pgmap v15148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:25.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:25 smithi090 ceph-mon[29180]: pgmap v15149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:25.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:25 smithi119 ceph-mon[32752]: pgmap v15149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:25.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:25 smithi167 ceph-mon[32751]: pgmap v15149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:26.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27307.405052 secs 2024-04-27T02:10:26.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27307.405052 secs 2024-04-27T02:10:26.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27307.405052 secs 2024-04-27T02:10:27.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:27 smithi090 ceph-mon[29180]: pgmap v15150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:27.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27308.233578 secs 2024-04-27T02:10:27.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:27 smithi119 ceph-mon[32752]: pgmap v15150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:27.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27308.233578 secs 2024-04-27T02:10:27.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:27 smithi167 ceph-mon[32751]: pgmap v15150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:27.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27308.233578 secs 2024-04-27T02:10:29.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:29 smithi090 ceph-mon[29180]: pgmap v15151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:29.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:29 smithi119 ceph-mon[32752]: pgmap v15151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:29.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:29 smithi167 ceph-mon[32751]: pgmap v15151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:31.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:31 smithi167 ceph-mon[32751]: pgmap v15152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:31.349 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27312.405143 secs 2024-04-27T02:10:31.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:31 smithi090 ceph-mon[29180]: pgmap v15152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:31.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27312.405143 secs 2024-04-27T02:10:31.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:31 smithi119 ceph-mon[32752]: pgmap v15152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:31.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27312.405143 secs 2024-04-27T02:10:32.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27313.233704 secs 2024-04-27T02:10:32.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27313.233704 secs 2024-04-27T02:10:32.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27313.233704 secs 2024-04-27T02:10:33.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:33 smithi090 ceph-mon[29180]: pgmap v15153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:33.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:33 smithi119 ceph-mon[32752]: pgmap v15153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:33.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:33 smithi167 ceph-mon[32751]: pgmap v15153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:35.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:35 smithi090 ceph-mon[29180]: pgmap v15154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:35.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:35 smithi119 ceph-mon[32752]: pgmap v15154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:35.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:35 smithi167 ceph-mon[32751]: pgmap v15154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:36.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27317.405230 secs 2024-04-27T02:10:36.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27317.405230 secs 2024-04-27T02:10:36.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27317.405230 secs 2024-04-27T02:10:37.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:37 smithi090 ceph-mon[29180]: pgmap v15155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:37.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27318.233823 secs 2024-04-27T02:10:37.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:37 smithi119 ceph-mon[32752]: pgmap v15155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:37.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27318.233823 secs 2024-04-27T02:10:37.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:37 smithi167 ceph-mon[32751]: pgmap v15155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:37.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27318.233823 secs 2024-04-27T02:10:39.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:39 smithi090 ceph-mon[29180]: pgmap v15156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:39.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:39 smithi119 ceph-mon[32752]: pgmap v15156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:39.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:39 smithi167 ceph-mon[32751]: pgmap v15156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:40.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:10:40.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:10:40.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:10:41.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:41 smithi090 ceph-mon[29180]: pgmap v15157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:41.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27322.405301 secs 2024-04-27T02:10:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:10:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:10:41.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:10:41.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:41 smithi119 ceph-mon[32752]: pgmap v15157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:41.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27322.405301 secs 2024-04-27T02:10:41.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:10:41.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:10:41.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:10:41.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:41 smithi167 ceph-mon[32751]: pgmap v15157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:41.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27322.405301 secs 2024-04-27T02:10:41.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:10:41.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:10:41.547 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:10:42.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:42 smithi090 ceph-mon[29180]: pgmap v15158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:10:42.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:42 smithi090 ceph-mon[29180]: pgmap v15159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27323.233879 secs 2024-04-27T02:10:42.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:42 smithi119 ceph-mon[32752]: pgmap v15158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:10:42.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:42 smithi119 ceph-mon[32752]: pgmap v15159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:42.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27323.233879 secs 2024-04-27T02:10:42.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:42 smithi167 ceph-mon[32751]: pgmap v15158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:10:42.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:42 smithi167 ceph-mon[32751]: pgmap v15159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:42.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27323.233879 secs 2024-04-27T02:10:44.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:44 smithi090 ceph-mon[29180]: pgmap v15160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:10:44.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:44 smithi119 ceph-mon[32752]: pgmap v15160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:10:44.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:44 smithi167 ceph-mon[32751]: pgmap v15160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:10:46.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:46 smithi090 ceph-mon[29180]: pgmap v15161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:46.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27327.405379 secs 2024-04-27T02:10:46.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:46 smithi119 ceph-mon[32752]: pgmap v15161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:46.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27327.405379 secs 2024-04-27T02:10:46.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:46 smithi167 ceph-mon[32751]: pgmap v15161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:46.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27327.405379 secs 2024-04-27T02:10:47.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27328.233944 secs 2024-04-27T02:10:47.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27328.233944 secs 2024-04-27T02:10:47.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27328.233944 secs 2024-04-27T02:10:48.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:48 smithi119 ceph-mon[32752]: pgmap v15162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:48.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:48 smithi167 ceph-mon[32751]: pgmap v15162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:48.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:48 smithi090 ceph-mon[29180]: pgmap v15162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:50.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:50 smithi119 ceph-mon[32752]: pgmap v15163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:50.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:50 smithi167 ceph-mon[32751]: pgmap v15163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:50.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:50 smithi090 ceph-mon[29180]: pgmap v15163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:10:51.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27332.405464 secs 2024-04-27T02:10:51.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27332.405464 secs 2024-04-27T02:10:51.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27332.405464 secs 2024-04-27T02:10:52.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:52 smithi119 ceph-mon[32752]: pgmap v15164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:10:52.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27333.234038 secs 2024-04-27T02:10:52.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:52 smithi167 ceph-mon[32751]: pgmap v15164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:10:52.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27333.234038 secs 2024-04-27T02:10:52.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:52 smithi090 ceph-mon[29180]: pgmap v15164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:10:52.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27333.234038 secs 2024-04-27T02:10:54.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:54 smithi119 ceph-mon[32752]: pgmap v15165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:54.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:54 smithi167 ceph-mon[32751]: pgmap v15165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:54.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:54 smithi090 ceph-mon[29180]: pgmap v15165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:56.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:56 smithi119 ceph-mon[32752]: pgmap v15166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:56.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27337.405546 secs 2024-04-27T02:10:56.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:56 smithi167 ceph-mon[32751]: pgmap v15166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:56.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27337.405546 secs 2024-04-27T02:10:56.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:56 smithi090 ceph-mon[29180]: pgmap v15166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:10:56.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27337.405546 secs 2024-04-27T02:10:57.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27338.234156 secs 2024-04-27T02:10:57.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27338.234156 secs 2024-04-27T02:10:57.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27338.234156 secs 2024-04-27T02:10:58.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:10:58 smithi119 ceph-mon[32752]: pgmap v15167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:58.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:10:58 smithi167 ceph-mon[32751]: pgmap v15167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:10:58.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:10:58 smithi090 ceph-mon[29180]: pgmap v15167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:00.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:00 smithi119 ceph-mon[32752]: pgmap v15168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:00.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:00 smithi167 ceph-mon[32751]: pgmap v15168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:00.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:00 smithi090 ceph-mon[29180]: pgmap v15168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:01.448 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27342.405630 secs 2024-04-27T02:11:01.506 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27342.405630 secs 2024-04-27T02:11:01.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27342.405630 secs 2024-04-27T02:11:02.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:02 smithi119 ceph-mon[32752]: pgmap v15169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:02.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27343.234298 secs 2024-04-27T02:11:02.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:02 smithi167 ceph-mon[32751]: pgmap v15169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:02.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27343.234298 secs 2024-04-27T02:11:02.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:02 smithi090 ceph-mon[29180]: pgmap v15169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:02.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27343.234298 secs 2024-04-27T02:11:03.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:03 smithi119 ceph-mon[32752]: pgmap v15170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:03.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:03 smithi167 ceph-mon[32751]: pgmap v15170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:03.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:03 smithi090 ceph-mon[29180]: pgmap v15170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:06.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:05 smithi119 ceph-mon[32752]: pgmap v15171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:06.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:05 smithi167 ceph-mon[32751]: pgmap v15171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:06.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:05 smithi090 ceph-mon[29180]: pgmap v15171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:07.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27347.405712 secs 2024-04-27T02:11:07.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27348.234432 secs 2024-04-27T02:11:07.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27347.405712 secs 2024-04-27T02:11:07.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27348.234432 secs 2024-04-27T02:11:07.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27347.405712 secs 2024-04-27T02:11:07.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27348.234432 secs 2024-04-27T02:11:08.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:07 smithi119 ceph-mon[32752]: pgmap v15172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:08.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:07 smithi167 ceph-mon[32751]: pgmap v15172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:08.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:07 smithi090 ceph-mon[29180]: pgmap v15172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:10.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:09 smithi119 ceph-mon[32752]: pgmap v15173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:10.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:09 smithi167 ceph-mon[32751]: pgmap v15173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:10.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:09 smithi090 ceph-mon[29180]: pgmap v15173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:12.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27352.405802 secs 2024-04-27T02:11:12.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:11 smithi119 ceph-mon[32752]: pgmap v15174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:12.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:11 smithi119 ceph-mon[32752]: 26 slow requests, 1 included below; oldest blocked for > 27353.234536 secs 2024-04-27T02:11:12.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:11 smithi119 ceph-mon[32752]: slow request 1443.016586 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:11:12.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27352.405802 secs 2024-04-27T02:11:12.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:11 smithi167 ceph-mon[32751]: pgmap v15174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:12.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:11 smithi167 ceph-mon[32751]: 26 slow requests, 1 included below; oldest blocked for > 27353.234536 secs 2024-04-27T02:11:12.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:11 smithi167 ceph-mon[32751]: slow request 1443.016586 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:11:12.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27352.405802 secs 2024-04-27T02:11:12.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:11 smithi090 ceph-mon[29180]: pgmap v15174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:12.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:11 smithi090 ceph-mon[29180]: 26 slow requests, 1 included below; oldest blocked for > 27353.234536 secs 2024-04-27T02:11:12.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:11 smithi090 ceph-mon[29180]: slow request 1443.016586 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:11:14.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:13 smithi119 ceph-mon[32752]: pgmap v15175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:14.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:13 smithi167 ceph-mon[32751]: pgmap v15175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:14.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:13 smithi090 ceph-mon[29180]: pgmap v15175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:15.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:15 smithi090 ceph-mon[29180]: pgmap v15176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:16.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:15 smithi119 ceph-mon[32752]: pgmap v15176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:16.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:15 smithi167 ceph-mon[32751]: pgmap v15176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:17.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:16 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 27357.405887 secs 2024-04-27T02:11:17.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:16 smithi119 ceph-mon[32752]: slow request 1444.282668 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:11:17.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27358.234658 secs 2024-04-27T02:11:17.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:16 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 27357.405887 secs 2024-04-27T02:11:17.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:16 smithi167 ceph-mon[32751]: slow request 1444.282668 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:11:17.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27358.234658 secs 2024-04-27T02:11:17.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:16 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 27357.405887 secs 2024-04-27T02:11:17.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:16 smithi090 ceph-mon[29180]: slow request 1444.282668 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:11:17.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27358.234658 secs 2024-04-27T02:11:18.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:17 smithi119 ceph-mon[32752]: pgmap v15177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:18.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:17 smithi167 ceph-mon[32751]: pgmap v15177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:18.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:17 smithi090 ceph-mon[29180]: pgmap v15177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:20.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:19 smithi119 ceph-mon[32752]: pgmap v15178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:20.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:19 smithi167 ceph-mon[32751]: pgmap v15178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:20.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:19 smithi090 ceph-mon[29180]: pgmap v15178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:22.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27362.405982 secs 2024-04-27T02:11:22.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:21 smithi119 ceph-mon[32752]: pgmap v15179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:22.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27363.234748 secs 2024-04-27T02:11:22.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27362.405982 secs 2024-04-27T02:11:22.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:21 smithi167 ceph-mon[32751]: pgmap v15179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:22.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27363.234748 secs 2024-04-27T02:11:22.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27362.405982 secs 2024-04-27T02:11:22.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:21 smithi090 ceph-mon[29180]: pgmap v15179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:22.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27363.234748 secs 2024-04-27T02:11:24.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:23 smithi119 ceph-mon[32752]: pgmap v15180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:24.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:23 smithi167 ceph-mon[32751]: pgmap v15180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:24.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:23 smithi090 ceph-mon[29180]: pgmap v15180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:26.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:25 smithi119 ceph-mon[32752]: pgmap v15181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:26.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:25 smithi167 ceph-mon[32751]: pgmap v15181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:26.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:25 smithi090 ceph-mon[29180]: pgmap v15181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:27.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27367.406042 secs 2024-04-27T02:11:27.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27368.234877 secs 2024-04-27T02:11:27.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27367.406042 secs 2024-04-27T02:11:27.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27368.234877 secs 2024-04-27T02:11:27.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27367.406042 secs 2024-04-27T02:11:27.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27368.234877 secs 2024-04-27T02:11:28.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:27 smithi119 ceph-mon[32752]: pgmap v15182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:28.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:27 smithi167 ceph-mon[32751]: pgmap v15182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:28.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:27 smithi090 ceph-mon[29180]: pgmap v15182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:30.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:29 smithi119 ceph-mon[32752]: pgmap v15183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:30.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:29 smithi167 ceph-mon[32751]: pgmap v15183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:30.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:29 smithi090 ceph-mon[29180]: pgmap v15183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:32.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27372.406141 secs 2024-04-27T02:11:32.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:31 smithi167 ceph-mon[32751]: pgmap v15184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:32.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27373.234933 secs 2024-04-27T02:11:32.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27372.406141 secs 2024-04-27T02:11:32.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:31 smithi090 ceph-mon[29180]: pgmap v15184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:32.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27373.234933 secs 2024-04-27T02:11:32.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27372.406141 secs 2024-04-27T02:11:32.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:31 smithi119 ceph-mon[32752]: pgmap v15184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:32.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27373.234933 secs 2024-04-27T02:11:34.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:33 smithi167 ceph-mon[32751]: pgmap v15185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:34.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:33 smithi090 ceph-mon[29180]: pgmap v15185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:34.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:33 smithi119 ceph-mon[32752]: pgmap v15185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:36.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:35 smithi167 ceph-mon[32751]: pgmap v15186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:36.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:35 smithi090 ceph-mon[29180]: pgmap v15186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:36.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:35 smithi119 ceph-mon[32752]: pgmap v15186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:37.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27377.406237 secs 2024-04-27T02:11:37.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27378.235048 secs 2024-04-27T02:11:37.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27377.406237 secs 2024-04-27T02:11:37.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27378.235048 secs 2024-04-27T02:11:37.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27377.406237 secs 2024-04-27T02:11:37.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27378.235048 secs 2024-04-27T02:11:38.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:37 smithi090 ceph-mon[29180]: pgmap v15187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:38.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:37 smithi119 ceph-mon[32752]: pgmap v15187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:38.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:37 smithi167 ceph-mon[32751]: pgmap v15187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:40.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:39 smithi090 ceph-mon[29180]: pgmap v15188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:40.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:39 smithi119 ceph-mon[32752]: pgmap v15188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:40.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:39 smithi167 ceph-mon[32751]: pgmap v15188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:41.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:11:41.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:11:41.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:11:42.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27382.406336 secs 2024-04-27T02:11:42.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:41 smithi090 ceph-mon[29180]: pgmap v15189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:42.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27383.235152 secs 2024-04-27T02:11:42.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:11:42.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:11:42.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:11:42.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27382.406336 secs 2024-04-27T02:11:42.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:41 smithi119 ceph-mon[32752]: pgmap v15189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:42.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27383.235152 secs 2024-04-27T02:11:42.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:11:42.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:11:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:11:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27382.406336 secs 2024-04-27T02:11:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:41 smithi167 ceph-mon[32751]: pgmap v15189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27383.235152 secs 2024-04-27T02:11:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:11:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:11:42.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:11:43.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:42 smithi090 ceph-mon[29180]: pgmap v15190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:11:43.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:42 smithi090 ceph-mon[29180]: pgmap v15191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:43.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:42 smithi119 ceph-mon[32752]: pgmap v15190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:11:43.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:42 smithi119 ceph-mon[32752]: pgmap v15191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:43.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:42 smithi167 ceph-mon[32751]: pgmap v15190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:11:43.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:42 smithi167 ceph-mon[32751]: pgmap v15191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:45.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:44 smithi090 ceph-mon[29180]: pgmap v15192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:45.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:44 smithi119 ceph-mon[32752]: pgmap v15192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:45.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:44 smithi167 ceph-mon[32751]: pgmap v15192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:47.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27387.406437 secs 2024-04-27T02:11:47.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:46 smithi090 ceph-mon[29180]: pgmap v15193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:47.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27388.235285 secs 2024-04-27T02:11:47.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27387.406437 secs 2024-04-27T02:11:47.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:46 smithi119 ceph-mon[32752]: pgmap v15193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:47.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27388.235285 secs 2024-04-27T02:11:47.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27387.406437 secs 2024-04-27T02:11:47.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:46 smithi167 ceph-mon[32751]: pgmap v15193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:47.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27388.235285 secs 2024-04-27T02:11:49.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:48 smithi090 ceph-mon[29180]: pgmap v15194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:11:49.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:48 smithi119 ceph-mon[32752]: pgmap v15194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:11:49.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:48 smithi167 ceph-mon[32751]: pgmap v15194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:11:51.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:50 smithi090 ceph-mon[29180]: pgmap v15195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:51.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:50 smithi119 ceph-mon[32752]: pgmap v15195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:51.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:50 smithi167 ceph-mon[32751]: pgmap v15195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:11:52.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27392.406538 secs 2024-04-27T02:11:52.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27393.235399 secs 2024-04-27T02:11:52.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27392.406538 secs 2024-04-27T02:11:52.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27393.235399 secs 2024-04-27T02:11:52.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27392.406538 secs 2024-04-27T02:11:52.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27393.235399 secs 2024-04-27T02:11:53.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:52 smithi090 ceph-mon[29180]: pgmap v15196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:11:53.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:52 smithi119 ceph-mon[32752]: pgmap v15196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:11:53.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:52 smithi167 ceph-mon[32751]: pgmap v15196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:11:55.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:54 smithi090 ceph-mon[29180]: pgmap v15197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:55.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:54 smithi119 ceph-mon[32752]: pgmap v15197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:55.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:54 smithi167 ceph-mon[32751]: pgmap v15197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:11:57.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27397.406622 secs 2024-04-27T02:11:57.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:56 smithi090 ceph-mon[29180]: pgmap v15198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:57.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27398.235519 secs 2024-04-27T02:11:57.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27397.406622 secs 2024-04-27T02:11:57.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:56 smithi119 ceph-mon[32752]: pgmap v15198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:57.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27398.235519 secs 2024-04-27T02:11:57.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27397.406622 secs 2024-04-27T02:11:57.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:56 smithi167 ceph-mon[32751]: pgmap v15198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:57.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27398.235519 secs 2024-04-27T02:11:59.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:11:58 smithi090 ceph-mon[29180]: pgmap v15199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:59.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:11:58 smithi119 ceph-mon[32752]: pgmap v15199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:11:59.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:11:58 smithi167 ceph-mon[32751]: pgmap v15199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:01.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:00 smithi090 ceph-mon[29180]: pgmap v15200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:01.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:00 smithi119 ceph-mon[32752]: pgmap v15200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:01.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:00 smithi167 ceph-mon[32751]: pgmap v15200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:02.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27402.406702 secs 2024-04-27T02:12:02.005 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27403.235631 secs 2024-04-27T02:12:02.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27402.406702 secs 2024-04-27T02:12:02.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27403.235631 secs 2024-04-27T02:12:02.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27402.406702 secs 2024-04-27T02:12:02.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27403.235631 secs 2024-04-27T02:12:03.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:02 smithi119 ceph-mon[32752]: pgmap v15201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:03.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:02 smithi167 ceph-mon[32751]: pgmap v15201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:03.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:02 smithi090 ceph-mon[29180]: pgmap v15201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:05.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:04 smithi119 ceph-mon[32752]: pgmap v15202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:05.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:04 smithi167 ceph-mon[32751]: pgmap v15202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:05.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:04 smithi090 ceph-mon[29180]: pgmap v15202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:07.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27407.406784 secs 2024-04-27T02:12:07.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:06 smithi119 ceph-mon[32752]: pgmap v15203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:07.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27408.235776 secs 2024-04-27T02:12:07.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27407.406784 secs 2024-04-27T02:12:07.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:06 smithi167 ceph-mon[32751]: pgmap v15203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:07.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27408.235776 secs 2024-04-27T02:12:07.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27407.406784 secs 2024-04-27T02:12:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:06 smithi090 ceph-mon[29180]: pgmap v15203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27408.235776 secs 2024-04-27T02:12:09.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:08 smithi119 ceph-mon[32752]: pgmap v15204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:09.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:08 smithi167 ceph-mon[32751]: pgmap v15204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:09.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:08 smithi090 ceph-mon[29180]: pgmap v15204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:11.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:10 smithi119 ceph-mon[32752]: pgmap v15205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:11.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:10 smithi167 ceph-mon[32751]: pgmap v15205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:11.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:10 smithi090 ceph-mon[29180]: pgmap v15205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:12.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27412.406869 secs 2024-04-27T02:12:12.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27413.235919 secs 2024-04-27T02:12:12.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27412.406869 secs 2024-04-27T02:12:12.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27413.235919 secs 2024-04-27T02:12:12.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27412.406869 secs 2024-04-27T02:12:12.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27413.235919 secs 2024-04-27T02:12:13.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:12 smithi119 ceph-mon[32752]: pgmap v15206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:13.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:12 smithi167 ceph-mon[32751]: pgmap v15206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:13.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:12 smithi090 ceph-mon[29180]: pgmap v15206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:15.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:14 smithi119 ceph-mon[32752]: pgmap v15207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:15.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:14 smithi167 ceph-mon[32751]: pgmap v15207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:15.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:14 smithi090 ceph-mon[29180]: pgmap v15207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:17.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27417.406953 secs 2024-04-27T02:12:17.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:16 smithi119 ceph-mon[32752]: pgmap v15208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:17.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27418.236013 secs 2024-04-27T02:12:17.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27417.406953 secs 2024-04-27T02:12:17.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:16 smithi167 ceph-mon[32751]: pgmap v15208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:17.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27418.236013 secs 2024-04-27T02:12:17.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27417.406953 secs 2024-04-27T02:12:17.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:16 smithi090 ceph-mon[29180]: pgmap v15208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:17.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27418.236013 secs 2024-04-27T02:12:19.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:18 smithi119 ceph-mon[32752]: pgmap v15209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:19.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:18 smithi167 ceph-mon[32751]: pgmap v15209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:19.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:18 smithi090 ceph-mon[29180]: pgmap v15209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:21.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:20 smithi119 ceph-mon[32752]: pgmap v15210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:21.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27422.407041 secs 2024-04-27T02:12:21.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:20 smithi167 ceph-mon[32751]: pgmap v15210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:21.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27422.407041 secs 2024-04-27T02:12:21.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:20 smithi090 ceph-mon[29180]: pgmap v15210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:21.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27422.407041 secs 2024-04-27T02:12:22.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27423.236123 secs 2024-04-27T02:12:22.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27423.236123 secs 2024-04-27T02:12:22.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27423.236123 secs 2024-04-27T02:12:23.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:22 smithi119 ceph-mon[32752]: pgmap v15211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:23.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:22 smithi167 ceph-mon[32751]: pgmap v15211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:23.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:22 smithi090 ceph-mon[29180]: pgmap v15211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:25.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:24 smithi119 ceph-mon[32752]: pgmap v15212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:25.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:24 smithi167 ceph-mon[32751]: pgmap v15212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:25.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:24 smithi090 ceph-mon[29180]: pgmap v15212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:26.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27427.407124 secs 2024-04-27T02:12:26.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27427.407124 secs 2024-04-27T02:12:26.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27427.407124 secs 2024-04-27T02:12:27.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:26 smithi119 ceph-mon[32752]: pgmap v15213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:27.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27428.236253 secs 2024-04-27T02:12:27.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:26 smithi167 ceph-mon[32751]: pgmap v15213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:27.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27428.236253 secs 2024-04-27T02:12:27.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:26 smithi090 ceph-mon[29180]: pgmap v15213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:27.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27428.236253 secs 2024-04-27T02:12:29.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:28 smithi119 ceph-mon[32752]: pgmap v15214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:29.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:28 smithi167 ceph-mon[32751]: pgmap v15214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:29.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:28 smithi090 ceph-mon[29180]: pgmap v15214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:31.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:30 smithi119 ceph-mon[32752]: pgmap v15215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:31.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27432.407212 secs 2024-04-27T02:12:31.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:30 smithi167 ceph-mon[32751]: pgmap v15215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:31.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27432.407212 secs 2024-04-27T02:12:31.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:30 smithi090 ceph-mon[29180]: pgmap v15215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:31.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27432.407212 secs 2024-04-27T02:12:32.255 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27433.236413 secs 2024-04-27T02:12:32.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27433.236413 secs 2024-04-27T02:12:32.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27433.236413 secs 2024-04-27T02:12:33.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:32 smithi090 ceph-mon[29180]: pgmap v15216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:33.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:32 smithi119 ceph-mon[32752]: pgmap v15216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:33.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:32 smithi167 ceph-mon[32751]: pgmap v15216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:35.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:34 smithi119 ceph-mon[32752]: pgmap v15217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:35.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:34 smithi167 ceph-mon[32751]: pgmap v15217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:35.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:34 smithi090 ceph-mon[29180]: pgmap v15217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:36.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:35 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27437.407296 secs 2024-04-27T02:12:36.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:35 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27437.407296 secs 2024-04-27T02:12:36.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:35 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27437.407296 secs 2024-04-27T02:12:37.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:36 smithi119 ceph-mon[32752]: pgmap v15218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:37.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:36 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27438.236488 secs 2024-04-27T02:12:37.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:36 smithi167 ceph-mon[32751]: pgmap v15218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:37.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:36 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27438.236488 secs 2024-04-27T02:12:37.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:36 smithi090 ceph-mon[29180]: pgmap v15218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:37.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:36 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27438.236488 secs 2024-04-27T02:12:39.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:38 smithi119 ceph-mon[32752]: pgmap v15219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:39.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:38 smithi167 ceph-mon[32751]: pgmap v15219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:39.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:38 smithi090 ceph-mon[29180]: pgmap v15219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:41.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:40 smithi119 ceph-mon[32752]: pgmap v15220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:41.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:40 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27442.407355 secs 2024-04-27T02:12:41.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:40 smithi167 ceph-mon[32751]: pgmap v15220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:41.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:40 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27442.407355 secs 2024-04-27T02:12:41.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:40 smithi090 ceph-mon[29180]: pgmap v15220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:41.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:40 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27442.407355 secs 2024-04-27T02:12:42.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:41 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27443.236602 secs 2024-04-27T02:12:42.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:12:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:41 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27443.236602 secs 2024-04-27T02:12:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:12:42.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:41 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27443.236602 secs 2024-04-27T02:12:42.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:12:43.154 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:43 smithi090 ceph-mon[29180]: pgmap v15221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:43.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.155 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.172 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:43 smithi119 ceph-mon[32752]: pgmap v15221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:43.172 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.172 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.172 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.172 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:43 smithi167 ceph-mon[32751]: pgmap v15221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:43.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:43.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: pgmap v15222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:45.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: pgmap v15222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:45.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: pgmap v15222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:12:45.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:45.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:46.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:12:46.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:12:46.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:12:46.033 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:12:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:12:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:12:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:12:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: pgmap v15223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T02:12:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: pgmap v15224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:46.034 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:46.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:12:46.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:12:46.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:12:46.047 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:12:46.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:12:46.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:12:46.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:12:46.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: pgmap v15223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T02:12:46.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: pgmap v15224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:46.048 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:12:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:12:46.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.105 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:12:46.106 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:12:46.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: pgmap v15223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T02:12:46.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: pgmap v15224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:46.107 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:12:47.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27447.407444 secs 2024-04-27T02:12:47.032 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:46 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27448.236700 secs 2024-04-27T02:12:47.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27447.407444 secs 2024-04-27T02:12:47.046 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:46 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27448.236700 secs 2024-04-27T02:12:47.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27447.407444 secs 2024-04-27T02:12:47.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:46 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27448.236700 secs 2024-04-27T02:12:48.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:47 smithi119 ceph-mon[32752]: pgmap v15225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:48.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:47 smithi167 ceph-mon[32751]: pgmap v15225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:48.104 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:47 smithi090 ceph-mon[29180]: pgmap v15225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:50.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:50 smithi119 ceph-mon[32752]: pgmap v15226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:50.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:50 smithi167 ceph-mon[32751]: pgmap v15226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:50 smithi090 ceph-mon[29180]: pgmap v15226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:51.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27452.407503 secs 2024-04-27T02:12:51.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27452.407503 secs 2024-04-27T02:12:51.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27452.407503 secs 2024-04-27T02:12:52.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:52 smithi119 ceph-mon[32752]: pgmap v15227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:52.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27453.236817 secs 2024-04-27T02:12:52.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:52 smithi167 ceph-mon[32751]: pgmap v15227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:52.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27453.236817 secs 2024-04-27T02:12:52.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:52 smithi090 ceph-mon[29180]: pgmap v15227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:12:52.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27453.236817 secs 2024-04-27T02:12:54.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:54 smithi119 ceph-mon[32752]: pgmap v15228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T02:12:54.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:54 smithi167 ceph-mon[32751]: pgmap v15228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T02:12:54.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:54 smithi090 ceph-mon[29180]: pgmap v15228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T02:12:56.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:56 smithi119 ceph-mon[32752]: pgmap v15229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:12:56.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27457.407558 secs 2024-04-27T02:12:56.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:56 smithi167 ceph-mon[32751]: pgmap v15229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:12:56.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27457.407558 secs 2024-04-27T02:12:56.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:56 smithi090 ceph-mon[29180]: pgmap v15229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:12:56.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27457.407558 secs 2024-04-27T02:12:57.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27458.236934 secs 2024-04-27T02:12:57.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27458.236934 secs 2024-04-27T02:12:57.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27458.236934 secs 2024-04-27T02:12:58.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:12:58 smithi119 ceph-mon[32752]: pgmap v15230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:58.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:12:58 smithi167 ceph-mon[32751]: pgmap v15230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:12:58.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:12:58 smithi090 ceph-mon[29180]: pgmap v15230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:00.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:00 smithi119 ceph-mon[32752]: pgmap v15231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:00.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:00 smithi167 ceph-mon[32751]: pgmap v15231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:00.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:00 smithi090 ceph-mon[29180]: pgmap v15231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:01.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27462.407642 secs 2024-04-27T02:13:01.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27462.407642 secs 2024-04-27T02:13:01.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27462.407642 secs 2024-04-27T02:13:02.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:02 smithi167 ceph-mon[32751]: pgmap v15232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:02.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27463.237015 secs 2024-04-27T02:13:02.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:02 smithi090 ceph-mon[29180]: pgmap v15232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:02.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27463.237015 secs 2024-04-27T02:13:02.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:02 smithi119 ceph-mon[32752]: pgmap v15232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:02.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27463.237015 secs 2024-04-27T02:13:04.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:04 smithi167 ceph-mon[32751]: pgmap v15233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:04.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:04 smithi090 ceph-mon[29180]: pgmap v15233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:04 smithi119 ceph-mon[32752]: pgmap v15233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:06.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:06 smithi167 ceph-mon[32751]: pgmap v15234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:06.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27467.407701 secs 2024-04-27T02:13:06.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:06 smithi090 ceph-mon[29180]: pgmap v15234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:06.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27467.407701 secs 2024-04-27T02:13:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:06 smithi119 ceph-mon[32752]: pgmap v15234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:06.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27467.407701 secs 2024-04-27T02:13:07.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27468.237162 secs 2024-04-27T02:13:07.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27468.237162 secs 2024-04-27T02:13:07.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27468.237162 secs 2024-04-27T02:13:08.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:08 smithi119 ceph-mon[32752]: pgmap v15235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:08 smithi090 ceph-mon[29180]: pgmap v15235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:08.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:08 smithi167 ceph-mon[32751]: pgmap v15235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:10.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:10 smithi090 ceph-mon[29180]: pgmap v15236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:10.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:10 smithi119 ceph-mon[32752]: pgmap v15236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:10.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:10 smithi167 ceph-mon[32751]: pgmap v15236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27472.407783 secs 2024-04-27T02:13:11.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27472.407783 secs 2024-04-27T02:13:11.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27472.407783 secs 2024-04-27T02:13:12.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:12 smithi090 ceph-mon[29180]: pgmap v15237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:12.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27473.237295 secs 2024-04-27T02:13:12.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:12 smithi119 ceph-mon[32752]: pgmap v15237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:12.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27473.237295 secs 2024-04-27T02:13:12.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:12 smithi167 ceph-mon[32751]: pgmap v15237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:12.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27473.237295 secs 2024-04-27T02:13:14.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:14 smithi090 ceph-mon[29180]: pgmap v15238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:14.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:14 smithi119 ceph-mon[32752]: pgmap v15238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:14.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:14 smithi167 ceph-mon[32751]: pgmap v15238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:16.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:16 smithi090 ceph-mon[29180]: pgmap v15239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:16.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27477.407871 secs 2024-04-27T02:13:16.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:16 smithi119 ceph-mon[32752]: pgmap v15239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:16.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27477.407871 secs 2024-04-27T02:13:16.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:16 smithi167 ceph-mon[32751]: pgmap v15239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:16.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27477.407871 secs 2024-04-27T02:13:17.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27478.237439 secs 2024-04-27T02:13:17.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27478.237439 secs 2024-04-27T02:13:17.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27478.237439 secs 2024-04-27T02:13:18.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:18 smithi090 ceph-mon[29180]: pgmap v15240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:18.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:18 smithi119 ceph-mon[32752]: pgmap v15240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:18.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:18 smithi167 ceph-mon[32751]: pgmap v15240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:20.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:20 smithi090 ceph-mon[29180]: pgmap v15241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:20.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:20 smithi119 ceph-mon[32752]: pgmap v15241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:20.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:20 smithi167 ceph-mon[32751]: pgmap v15241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:21.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27482.407958 secs 2024-04-27T02:13:21.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27482.407958 secs 2024-04-27T02:13:21.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27482.407958 secs 2024-04-27T02:13:22.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:22 smithi090 ceph-mon[29180]: pgmap v15242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:22.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27483.237548 secs 2024-04-27T02:13:22.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:22 smithi119 ceph-mon[32752]: pgmap v15242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:22.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27483.237548 secs 2024-04-27T02:13:22.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:22 smithi167 ceph-mon[32751]: pgmap v15242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:22.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27483.237548 secs 2024-04-27T02:13:24.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:24 smithi090 ceph-mon[29180]: pgmap v15243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:24.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:24 smithi119 ceph-mon[32752]: pgmap v15243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:24.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:24 smithi167 ceph-mon[32751]: pgmap v15243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:13:26.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:26 smithi090 ceph-mon[29180]: pgmap v15244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:26.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27487.408045 secs 2024-04-27T02:13:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:26 smithi119 ceph-mon[32752]: pgmap v15244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:26.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27487.408045 secs 2024-04-27T02:13:26.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:26 smithi167 ceph-mon[32751]: pgmap v15244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:13:26.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27487.408045 secs 2024-04-27T02:13:27.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27488.237617 secs 2024-04-27T02:13:27.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27488.237617 secs 2024-04-27T02:13:27.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27488.237617 secs 2024-04-27T02:13:28.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:28 smithi090 ceph-mon[29180]: pgmap v15245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:28.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:28 smithi119 ceph-mon[32752]: pgmap v15245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:28.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:28 smithi167 ceph-mon[32751]: pgmap v15245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:30.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:29 smithi119 ceph-mon[32752]: pgmap v15246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:30.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:29 smithi167 ceph-mon[32751]: pgmap v15246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:30.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:29 smithi090 ceph-mon[29180]: pgmap v15246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:31.197 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27492.408133 secs 2024-04-27T02:13:31.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27492.408133 secs 2024-04-27T02:13:31.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27492.408133 secs 2024-04-27T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:31 smithi090 ceph-mon[29180]: pgmap v15247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:32.254 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27493.237683 secs 2024-04-27T02:13:32.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:31 smithi119 ceph-mon[32752]: pgmap v15247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:32.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27493.237683 secs 2024-04-27T02:13:32.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:31 smithi167 ceph-mon[32751]: pgmap v15247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:32.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27493.237683 secs 2024-04-27T02:13:34.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:34 smithi119 ceph-mon[32752]: pgmap v15248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:34.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:34 smithi167 ceph-mon[32751]: pgmap v15248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:34.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:34 smithi090 ceph-mon[29180]: pgmap v15248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:36.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:36 smithi119 ceph-mon[32752]: pgmap v15249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:36.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27497.408215 secs 2024-04-27T02:13:36.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:36 smithi167 ceph-mon[32751]: pgmap v15249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:36.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27497.408215 secs 2024-04-27T02:13:36.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:36 smithi090 ceph-mon[29180]: pgmap v15249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:36.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27497.408215 secs 2024-04-27T02:13:37.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27498.237822 secs 2024-04-27T02:13:37.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27498.237822 secs 2024-04-27T02:13:37.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27498.237822 secs 2024-04-27T02:13:38.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:38 smithi119 ceph-mon[32752]: pgmap v15250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:38.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:38 smithi167 ceph-mon[32751]: pgmap v15250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:38.340 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:38 smithi090 ceph-mon[29180]: pgmap v15250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:38.603 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:13:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:13:38.361+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:13:38.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:13:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:13:38.361+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:13:38.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:13:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:13:38.361+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:13:38.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:13:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:13:38.361+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:13:38.604 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:13:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:13:38.361+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:13:40.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:40 smithi167 ceph-mon[32751]: pgmap v15251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:40.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:40 smithi090 ceph-mon[29180]: pgmap v15251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:40.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:40 smithi119 ceph-mon[32752]: pgmap v15251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:41.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:41 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 27502.408275 secs 2024-04-27T02:13:41.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:41 smithi167 ceph-mon[32751]: slow request 2882.576886 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:13:41.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:41 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 27502.408275 secs 2024-04-27T02:13:41.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:41 smithi090 ceph-mon[29180]: slow request 2882.576886 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:13:41.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:41 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 27502.408275 secs 2024-04-27T02:13:41.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:41 smithi119 ceph-mon[32752]: slow request 2882.576886 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:13:42.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27503.237962 secs 2024-04-27T02:13:42.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:42 smithi167 ceph-mon[32751]: pgmap v15252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:42.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27503.237962 secs 2024-04-27T02:13:42.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:42 smithi090 ceph-mon[29180]: pgmap v15252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:42.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27503.237962 secs 2024-04-27T02:13:42.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:42 smithi119 ceph-mon[32752]: pgmap v15252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:44.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:44 smithi167 ceph-mon[32751]: pgmap v15253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:44.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:44 smithi090 ceph-mon[29180]: pgmap v15253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:44.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:44 smithi119 ceph-mon[32752]: pgmap v15253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:13:46.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:46 smithi090 ceph-mon[29180]: pgmap v15254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:46.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:13:46.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27507.408363 secs 2024-04-27T02:13:46.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:46.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:46.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:46 smithi119 ceph-mon[32752]: pgmap v15254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:46.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:13:46.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27507.408363 secs 2024-04-27T02:13:46.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:46.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:46.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:46 smithi167 ceph-mon[32751]: pgmap v15254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:13:46.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:13:46.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27507.408363 secs 2024-04-27T02:13:46.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:46.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:47.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27508.238110 secs 2024-04-27T02:13:47.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:13:47.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:13:47.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:47.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27508.238110 secs 2024-04-27T02:13:47.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:13:47.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:13:47.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:47.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27508.238110 secs 2024-04-27T02:13:47.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:13:47.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:13:47.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:13:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:48 smithi090 ceph-mon[29180]: pgmap v15255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T02:13:48.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:48 smithi090 ceph-mon[29180]: pgmap v15256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:48.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:48 smithi119 ceph-mon[32752]: pgmap v15255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T02:13:48.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:48 smithi119 ceph-mon[32752]: pgmap v15256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:48.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:48 smithi167 ceph-mon[32751]: pgmap v15255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T02:13:48.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:48 smithi167 ceph-mon[32751]: pgmap v15256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:50 smithi090 ceph-mon[29180]: pgmap v15257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:13:50.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:50 smithi119 ceph-mon[32752]: pgmap v15257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:13:50.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:50 smithi167 ceph-mon[32751]: pgmap v15257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:13:51.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27512.408451 secs 2024-04-27T02:13:51.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27512.408451 secs 2024-04-27T02:13:51.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27512.408451 secs 2024-04-27T02:13:52.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:52 smithi090 ceph-mon[29180]: pgmap v15258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:52.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27513.238223 secs 2024-04-27T02:13:52.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:52 smithi119 ceph-mon[32752]: pgmap v15258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:52.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27513.238223 secs 2024-04-27T02:13:52.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:52 smithi167 ceph-mon[32751]: pgmap v15258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:52.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27513.238223 secs 2024-04-27T02:13:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:54 smithi090 ceph-mon[29180]: pgmap v15259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:54.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:54 smithi119 ceph-mon[32752]: pgmap v15259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:54.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:54 smithi167 ceph-mon[32751]: pgmap v15259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:56.333 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:56 smithi119 ceph-mon[32752]: pgmap v15260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:56.334 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27517.408558 secs 2024-04-27T02:13:56.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:56 smithi090 ceph-mon[29180]: pgmap v15260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:56.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27517.408558 secs 2024-04-27T02:13:56.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:56 smithi167 ceph-mon[32751]: pgmap v15260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:13:56.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27517.408558 secs 2024-04-27T02:13:57.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27518.238406 secs 2024-04-27T02:13:57.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27518.238406 secs 2024-04-27T02:13:57.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27518.238406 secs 2024-04-27T02:13:58.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:13:58 smithi090 ceph-mon[29180]: pgmap v15261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:13:58.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:13:58 smithi119 ceph-mon[32752]: pgmap v15261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:13:58.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:13:58 smithi167 ceph-mon[32751]: pgmap v15261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:14:00.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:00 smithi090 ceph-mon[29180]: pgmap v15262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:00.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:00 smithi119 ceph-mon[32752]: pgmap v15262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:00.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:00 smithi167 ceph-mon[32751]: pgmap v15262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:01.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27522.408648 secs 2024-04-27T02:14:01.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27522.408648 secs 2024-04-27T02:14:01.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27522.408648 secs 2024-04-27T02:14:02.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:02 smithi090 ceph-mon[29180]: pgmap v15263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:02.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27523.238528 secs 2024-04-27T02:14:02.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:02 smithi119 ceph-mon[32752]: pgmap v15263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:02.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27523.238528 secs 2024-04-27T02:14:02.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:02 smithi167 ceph-mon[32751]: pgmap v15263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:02.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27523.238528 secs 2024-04-27T02:14:04.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:04 smithi090 ceph-mon[29180]: pgmap v15264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:04 smithi119 ceph-mon[32752]: pgmap v15264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:04 smithi167 ceph-mon[32751]: pgmap v15264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:06.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:06 smithi090 ceph-mon[29180]: pgmap v15265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:06.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:06 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 27527.408750 secs 2024-04-27T02:14:06.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:06 smithi090 ceph-mon[29180]: slow request 23043.190192 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:14:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:06 smithi119 ceph-mon[32752]: pgmap v15265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:06 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 27527.408750 secs 2024-04-27T02:14:06.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:06 smithi119 ceph-mon[32752]: slow request 23043.190192 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:14:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:06 smithi167 ceph-mon[32751]: pgmap v15265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:06.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:06 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 27527.408750 secs 2024-04-27T02:14:06.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:06 smithi167 ceph-mon[32751]: slow request 23043.190192 seconds old, received at 2024-04-26T19:50:01.971962+0000: client_request(client.17127:1 getattr pAsLsXsFs #0x1 2024-04-26T19:50:01.968136+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:14:07.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27528.238622 secs 2024-04-27T02:14:07.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27528.238622 secs 2024-04-27T02:14:07.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27528.238622 secs 2024-04-27T02:14:08.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:08 smithi090 ceph-mon[29180]: pgmap v15266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:08.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:08 smithi119 ceph-mon[32752]: pgmap v15266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:08.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:08 smithi167 ceph-mon[32751]: pgmap v15266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:10.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:10 smithi090 ceph-mon[29180]: pgmap v15267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:10.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:10 smithi119 ceph-mon[32752]: pgmap v15267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:10.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:10 smithi167 ceph-mon[32751]: pgmap v15267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:11.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27532.408850 secs 2024-04-27T02:14:11.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27532.408850 secs 2024-04-27T02:14:11.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27532.408850 secs 2024-04-27T02:14:12.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:12 smithi119 ceph-mon[32752]: pgmap v15268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:12.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27533.238747 secs 2024-04-27T02:14:12.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:12 smithi167 ceph-mon[32751]: pgmap v15268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:12.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27533.238747 secs 2024-04-27T02:14:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:12 smithi090 ceph-mon[29180]: pgmap v15268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27533.238747 secs 2024-04-27T02:14:14.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:14 smithi119 ceph-mon[32752]: pgmap v15269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:14.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:14 smithi167 ceph-mon[32751]: pgmap v15269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:14.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:14 smithi090 ceph-mon[29180]: pgmap v15269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:16.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:16 smithi119 ceph-mon[32752]: pgmap v15270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:16.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27537.408948 secs 2024-04-27T02:14:16.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:16 smithi167 ceph-mon[32751]: pgmap v15270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:16.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27537.408948 secs 2024-04-27T02:14:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:16 smithi090 ceph-mon[29180]: pgmap v15270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27537.408948 secs 2024-04-27T02:14:17.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27538.238840 secs 2024-04-27T02:14:17.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27538.238840 secs 2024-04-27T02:14:17.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27538.238840 secs 2024-04-27T02:14:18.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:18 smithi119 ceph-mon[32752]: pgmap v15271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:18.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:18 smithi167 ceph-mon[32751]: pgmap v15271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:18.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:18 smithi090 ceph-mon[29180]: pgmap v15271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:20.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:20 smithi119 ceph-mon[32752]: pgmap v15272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:20.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:20 smithi167 ceph-mon[32751]: pgmap v15272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:20.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:20 smithi090 ceph-mon[29180]: pgmap v15272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:21.391 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27542.409038 secs 2024-04-27T02:14:21.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27542.409038 secs 2024-04-27T02:14:21.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27542.409038 secs 2024-04-27T02:14:22.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:22 smithi119 ceph-mon[32752]: pgmap v15273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:22.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27543.238962 secs 2024-04-27T02:14:22.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:22 smithi167 ceph-mon[32751]: pgmap v15273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:22.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27543.238962 secs 2024-04-27T02:14:22.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:22 smithi090 ceph-mon[29180]: pgmap v15273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:22.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27543.238962 secs 2024-04-27T02:14:24.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:24 smithi119 ceph-mon[32752]: pgmap v15274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:24.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:24 smithi167 ceph-mon[32751]: pgmap v15274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:24 smithi090 ceph-mon[29180]: pgmap v15274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:26 smithi119 ceph-mon[32752]: pgmap v15275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27547.409128 secs 2024-04-27T02:14:26.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:26 smithi167 ceph-mon[32751]: pgmap v15275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:26.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27547.409128 secs 2024-04-27T02:14:26.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:26 smithi090 ceph-mon[29180]: pgmap v15275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:26.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27547.409128 secs 2024-04-27T02:14:27.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27548.239094 secs 2024-04-27T02:14:27.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27548.239094 secs 2024-04-27T02:14:27.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27548.239094 secs 2024-04-27T02:14:28.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:28 smithi119 ceph-mon[32752]: pgmap v15276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:28.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:28 smithi167 ceph-mon[32751]: pgmap v15276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:28.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:28 smithi090 ceph-mon[29180]: pgmap v15276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:30.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:30 smithi119 ceph-mon[32752]: pgmap v15277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:30.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:30 smithi167 ceph-mon[32751]: pgmap v15277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:30.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:30 smithi090 ceph-mon[29180]: pgmap v15277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:31.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27552.409226 secs 2024-04-27T02:14:31.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27552.409226 secs 2024-04-27T02:14:31.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27552.409226 secs 2024-04-27T02:14:32.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:32 smithi090 ceph-mon[29180]: pgmap v15278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:32.505 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27553.239236 secs 2024-04-27T02:14:32.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:32 smithi119 ceph-mon[32752]: pgmap v15278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:32.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27553.239236 secs 2024-04-27T02:14:32.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:32 smithi167 ceph-mon[32751]: pgmap v15278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:32.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27553.239236 secs 2024-04-27T02:14:34.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:34 smithi119 ceph-mon[32752]: pgmap v15279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:34.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:34 smithi167 ceph-mon[32751]: pgmap v15279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:34.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:34 smithi090 ceph-mon[29180]: pgmap v15279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:36.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:36 smithi119 ceph-mon[32752]: pgmap v15280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:36.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27557.409321 secs 2024-04-27T02:14:36.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:36 smithi167 ceph-mon[32751]: pgmap v15280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:36.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27557.409321 secs 2024-04-27T02:14:36.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:36 smithi090 ceph-mon[29180]: pgmap v15280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:36.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27557.409321 secs 2024-04-27T02:14:37.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27558.239397 secs 2024-04-27T02:14:37.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27558.239397 secs 2024-04-27T02:14:37.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27558.239397 secs 2024-04-27T02:14:38.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:38 smithi119 ceph-mon[32752]: pgmap v15281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:38.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:38 smithi167 ceph-mon[32751]: pgmap v15281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:38.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:38 smithi090 ceph-mon[29180]: pgmap v15281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:40.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:40 smithi119 ceph-mon[32752]: pgmap v15282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:40.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:40 smithi167 ceph-mon[32751]: pgmap v15282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:40.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:40 smithi090 ceph-mon[29180]: pgmap v15282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:14:41.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27562.409422 secs 2024-04-27T02:14:41.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27562.409422 secs 2024-04-27T02:14:41.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27562.409422 secs 2024-04-27T02:14:42.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:42 smithi119 ceph-mon[32752]: pgmap v15283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:42.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27563.239447 secs 2024-04-27T02:14:42.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:42 smithi167 ceph-mon[32751]: pgmap v15283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:42.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27563.239447 secs 2024-04-27T02:14:42.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:42 smithi090 ceph-mon[29180]: pgmap v15283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:42.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27563.239447 secs 2024-04-27T02:14:43.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:43 smithi119 ceph-mon[32752]: pgmap v15284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:43.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:43 smithi167 ceph-mon[32751]: pgmap v15284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:43.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:43 smithi090 ceph-mon[29180]: pgmap v15284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:45.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:45 smithi090 ceph-mon[29180]: pgmap v15285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:45.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:45 smithi119 ceph-mon[32752]: pgmap v15285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:45.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:45 smithi167 ceph-mon[32751]: pgmap v15285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:14:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27567.409522 secs 2024-04-27T02:14:46.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27567.409522 secs 2024-04-27T02:14:46.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27567.409522 secs 2024-04-27T02:14:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27568.239540 secs 2024-04-27T02:14:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: pgmap v15286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:14:47.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:14:47.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:14:47.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: pgmap v15287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:14:47.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: pgmap v15288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T02:14:47.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:14:47.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27568.239540 secs 2024-04-27T02:14:47.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: pgmap v15286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:47.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:14:47.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:14:47.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:14:47.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: pgmap v15287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:14:47.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: pgmap v15288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T02:14:47.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:14:47.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27568.239540 secs 2024-04-27T02:14:47.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: pgmap v15286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:14:47.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:14:47.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:14:47.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:14:47.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: pgmap v15287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:14:47.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: pgmap v15288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T02:14:47.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:14:50.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:49 smithi119 ceph-mon[32752]: pgmap v15289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:50.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:49 smithi167 ceph-mon[32751]: pgmap v15289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:49 smithi090 ceph-mon[29180]: pgmap v15289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:51.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:50 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27572.409609 secs 2024-04-27T02:14:51.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:50 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27572.409609 secs 2024-04-27T02:14:51.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:50 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27572.409609 secs 2024-04-27T02:14:52.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:51 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27573.239651 secs 2024-04-27T02:14:52.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:51 smithi119 ceph-mon[32752]: pgmap v15290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:52.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:51 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27573.239651 secs 2024-04-27T02:14:52.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:51 smithi167 ceph-mon[32751]: pgmap v15290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:52.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:51 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27573.239651 secs 2024-04-27T02:14:52.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:51 smithi090 ceph-mon[29180]: pgmap v15290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:54.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:54 smithi119 ceph-mon[32752]: pgmap v15291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T02:14:54.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:54 smithi167 ceph-mon[32751]: pgmap v15291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T02:14:54.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:54 smithi090 ceph-mon[29180]: pgmap v15291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T02:14:56.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:56 smithi119 ceph-mon[32752]: pgmap v15292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:56.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27577.409697 secs 2024-04-27T02:14:56.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:56 smithi167 ceph-mon[32751]: pgmap v15292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:56.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27577.409697 secs 2024-04-27T02:14:56.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:56 smithi090 ceph-mon[29180]: pgmap v15292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:14:56.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27577.409697 secs 2024-04-27T02:14:57.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27578.239790 secs 2024-04-27T02:14:57.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27578.239790 secs 2024-04-27T02:14:57.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27578.239790 secs 2024-04-27T02:14:58.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:14:58 smithi119 ceph-mon[32752]: pgmap v15293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:14:58.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:14:58 smithi167 ceph-mon[32751]: pgmap v15293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:14:58.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:14:58 smithi090 ceph-mon[29180]: pgmap v15293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:15:00.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:00 smithi119 ceph-mon[32752]: pgmap v15294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:00.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:00 smithi167 ceph-mon[32751]: pgmap v15294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:00.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:00 smithi090 ceph-mon[29180]: pgmap v15294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:01.447 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27582.409794 secs 2024-04-27T02:15:01.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27582.409794 secs 2024-04-27T02:15:01.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27582.409794 secs 2024-04-27T02:15:02.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27583.239865 secs 2024-04-27T02:15:02.505 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:02 smithi090 ceph-mon[29180]: pgmap v15295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:02.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27583.239865 secs 2024-04-27T02:15:02.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:02 smithi119 ceph-mon[32752]: pgmap v15295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:02.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27583.239865 secs 2024-04-27T02:15:02.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:02 smithi167 ceph-mon[32751]: pgmap v15295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:04 smithi119 ceph-mon[32752]: pgmap v15296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:04 smithi167 ceph-mon[32751]: pgmap v15296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:04 smithi090 ceph-mon[29180]: pgmap v15296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:06 smithi119 ceph-mon[32752]: pgmap v15297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:06 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 27587.409841 secs 2024-04-27T02:15:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:06 smithi119 ceph-mon[32752]: slow request 11520.037674 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:15:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:06 smithi167 ceph-mon[32751]: pgmap v15297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:06 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 27587.409841 secs 2024-04-27T02:15:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:06 smithi167 ceph-mon[32751]: slow request 11520.037674 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:15:06.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:06 smithi090 ceph-mon[29180]: pgmap v15297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:06.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:06 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 27587.409841 secs 2024-04-27T02:15:06.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:06 smithi090 ceph-mon[29180]: slow request 11520.037674 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:15:07.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27588.239982 secs 2024-04-27T02:15:07.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27588.239982 secs 2024-04-27T02:15:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27588.239982 secs 2024-04-27T02:15:08.494 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:08 smithi090 ceph-mon[29180]: pgmap v15298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:08.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:08 smithi119 ceph-mon[32752]: pgmap v15298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:08.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:08 smithi167 ceph-mon[32751]: pgmap v15298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:10.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:10 smithi119 ceph-mon[32752]: pgmap v15299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:10.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:10 smithi167 ceph-mon[32751]: pgmap v15299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:10.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:10 smithi090 ceph-mon[29180]: pgmap v15299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:11.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27592.409917 secs 2024-04-27T02:15:11.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27592.409917 secs 2024-04-27T02:15:11.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27592.409917 secs 2024-04-27T02:15:12.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27593.240114 secs 2024-04-27T02:15:12.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:12 smithi119 ceph-mon[32752]: pgmap v15300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:12.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27593.240114 secs 2024-04-27T02:15:12.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:12 smithi167 ceph-mon[32751]: pgmap v15300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27593.240114 secs 2024-04-27T02:15:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:12 smithi090 ceph-mon[29180]: pgmap v15300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:14.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:14 smithi119 ceph-mon[32752]: pgmap v15301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:14.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:14 smithi167 ceph-mon[32751]: pgmap v15301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:14.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:14 smithi090 ceph-mon[29180]: pgmap v15301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:16.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:16 smithi119 ceph-mon[32752]: pgmap v15302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:16.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27597.409999 secs 2024-04-27T02:15:16.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:16 smithi167 ceph-mon[32751]: pgmap v15302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:16.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27597.409999 secs 2024-04-27T02:15:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:16 smithi090 ceph-mon[29180]: pgmap v15302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27597.409999 secs 2024-04-27T02:15:17.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27598.240253 secs 2024-04-27T02:15:17.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27598.240253 secs 2024-04-27T02:15:17.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27598.240253 secs 2024-04-27T02:15:18.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:18 smithi119 ceph-mon[32752]: pgmap v15303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:18.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:18 smithi167 ceph-mon[32751]: pgmap v15303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:18.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:18 smithi090 ceph-mon[29180]: pgmap v15303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:19.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:19 smithi119 ceph-mon[32752]: pgmap v15304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:19.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:19 smithi167 ceph-mon[32751]: pgmap v15304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:19.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:19 smithi090 ceph-mon[29180]: pgmap v15304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:21.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27602.410060 secs 2024-04-27T02:15:21.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27602.410060 secs 2024-04-27T02:15:21.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27602.410060 secs 2024-04-27T02:15:22.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27603.240415 secs 2024-04-27T02:15:22.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:21 smithi119 ceph-mon[32752]: pgmap v15305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:22.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27603.240415 secs 2024-04-27T02:15:22.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:21 smithi167 ceph-mon[32751]: pgmap v15305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:22.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27603.240415 secs 2024-04-27T02:15:22.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:21 smithi090 ceph-mon[29180]: pgmap v15305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:24.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:24 smithi119 ceph-mon[32752]: pgmap v15306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:24.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:24 smithi167 ceph-mon[32751]: pgmap v15306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:24.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:24 smithi090 ceph-mon[29180]: pgmap v15306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:26 smithi119 ceph-mon[32752]: pgmap v15307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27607.410121 secs 2024-04-27T02:15:26.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:26 smithi167 ceph-mon[32751]: pgmap v15307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:26.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27607.410121 secs 2024-04-27T02:15:26.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:26 smithi090 ceph-mon[29180]: pgmap v15307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:26.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27607.410121 secs 2024-04-27T02:15:27.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27608.240523 secs 2024-04-27T02:15:27.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27608.240523 secs 2024-04-27T02:15:27.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27608.240523 secs 2024-04-27T02:15:28.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:28 smithi119 ceph-mon[32752]: pgmap v15308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:28.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:28 smithi167 ceph-mon[32751]: pgmap v15308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:28.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:28 smithi090 ceph-mon[29180]: pgmap v15308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:30.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:30 smithi119 ceph-mon[32752]: pgmap v15309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:30.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:30 smithi167 ceph-mon[32751]: pgmap v15309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:30.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:30 smithi090 ceph-mon[29180]: pgmap v15309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:31.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27612.410182 secs 2024-04-27T02:15:31.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27612.410182 secs 2024-04-27T02:15:31.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27612.410182 secs 2024-04-27T02:15:32.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27613.240600 secs 2024-04-27T02:15:32.505 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:32 smithi090 ceph-mon[29180]: pgmap v15310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:32.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27613.240600 secs 2024-04-27T02:15:32.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:32 smithi119 ceph-mon[32752]: pgmap v15310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:32.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27613.240600 secs 2024-04-27T02:15:32.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:32 smithi167 ceph-mon[32751]: pgmap v15310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:34.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:34 smithi119 ceph-mon[32752]: pgmap v15311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:34.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:34 smithi167 ceph-mon[32751]: pgmap v15311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:34.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:34 smithi090 ceph-mon[29180]: pgmap v15311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:36.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:36 smithi119 ceph-mon[32752]: pgmap v15312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:36.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27617.410241 secs 2024-04-27T02:15:36.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:36 smithi167 ceph-mon[32751]: pgmap v15312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:36.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27617.410241 secs 2024-04-27T02:15:36.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:36 smithi090 ceph-mon[29180]: pgmap v15312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:36.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27617.410241 secs 2024-04-27T02:15:37.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27618.240716 secs 2024-04-27T02:15:37.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27618.240716 secs 2024-04-27T02:15:37.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27618.240716 secs 2024-04-27T02:15:38.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:38 smithi119 ceph-mon[32752]: pgmap v15313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:38.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:38 smithi167 ceph-mon[32751]: pgmap v15313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:38.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:38 smithi090 ceph-mon[29180]: pgmap v15313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:40.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:40 smithi119 ceph-mon[32752]: pgmap v15314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:40.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:40 smithi167 ceph-mon[32751]: pgmap v15314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:40.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:40 smithi090 ceph-mon[29180]: pgmap v15314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:41.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27622.410325 secs 2024-04-27T02:15:41.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27622.410325 secs 2024-04-27T02:15:41.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27622.410325 secs 2024-04-27T02:15:42.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27623.240812 secs 2024-04-27T02:15:42.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:42 smithi119 ceph-mon[32752]: pgmap v15315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:42.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27623.240812 secs 2024-04-27T02:15:42.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:42 smithi167 ceph-mon[32751]: pgmap v15315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:42.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27623.240812 secs 2024-04-27T02:15:42.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:42 smithi090 ceph-mon[29180]: pgmap v15315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:44.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:44 smithi119 ceph-mon[32752]: pgmap v15316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:44.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:44 smithi167 ceph-mon[32751]: pgmap v15316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:44.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:44 smithi090 ceph-mon[29180]: pgmap v15316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:46.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:46 smithi119 ceph-mon[32752]: pgmap v15317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:46.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27627.410407 secs 2024-04-27T02:15:46.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:46 smithi167 ceph-mon[32751]: pgmap v15317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:46.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27627.410407 secs 2024-04-27T02:15:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:46 smithi090 ceph-mon[29180]: pgmap v15317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:15:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27627.410407 secs 2024-04-27T02:15:47.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27628.240959 secs 2024-04-27T02:15:47.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:15:47.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27628.240959 secs 2024-04-27T02:15:47.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:15:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27628.240959 secs 2024-04-27T02:15:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:15:48.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:48 smithi119 ceph-mon[32752]: pgmap v15318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:48.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:15:48.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:15:48.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:15:48.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:48 smithi167 ceph-mon[32751]: pgmap v15318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:48.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:15:48.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:15:48.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:15:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:48 smithi090 ceph-mon[29180]: pgmap v15318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:15:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:15:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:15:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:15:49.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:49 smithi119 ceph-mon[32752]: pgmap v15319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:15:49.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:49 smithi119 ceph-mon[32752]: pgmap v15320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:15:49.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:49 smithi167 ceph-mon[32751]: pgmap v15319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:15:49.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:49 smithi167 ceph-mon[32751]: pgmap v15320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:15:49.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:49 smithi090 ceph-mon[29180]: pgmap v15319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:15:49.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:49 smithi090 ceph-mon[29180]: pgmap v15320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:15:50.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:50 smithi119 ceph-mon[32752]: pgmap v15321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:50.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:50 smithi167 ceph-mon[32751]: pgmap v15321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:50.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:50 smithi090 ceph-mon[29180]: pgmap v15321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:51.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27632.410490 secs 2024-04-27T02:15:51.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27632.410490 secs 2024-04-27T02:15:51.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27632.410490 secs 2024-04-27T02:15:52.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27633.241037 secs 2024-04-27T02:15:52.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:52 smithi119 ceph-mon[32752]: pgmap v15322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:52.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27633.241037 secs 2024-04-27T02:15:52.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:52 smithi167 ceph-mon[32751]: pgmap v15322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:52.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27633.241037 secs 2024-04-27T02:15:52.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:52 smithi090 ceph-mon[29180]: pgmap v15322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:55.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:54 smithi119 ceph-mon[32752]: pgmap v15323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:55.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:54 smithi167 ceph-mon[32751]: pgmap v15323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:55.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:54 smithi090 ceph-mon[29180]: pgmap v15323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:57.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27637.410569 secs 2024-04-27T02:15:57.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:56 smithi119 ceph-mon[32752]: pgmap v15324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:57.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:56 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27638.241196 secs 2024-04-27T02:15:57.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27637.410569 secs 2024-04-27T02:15:57.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:56 smithi167 ceph-mon[32751]: pgmap v15324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:57.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:56 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27638.241196 secs 2024-04-27T02:15:57.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27637.410569 secs 2024-04-27T02:15:57.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:56 smithi090 ceph-mon[29180]: pgmap v15324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:15:57.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:56 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27638.241196 secs 2024-04-27T02:15:59.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:15:58 smithi119 ceph-mon[32752]: pgmap v15325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:15:59.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:15:58 smithi167 ceph-mon[32751]: pgmap v15325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:15:59.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:15:58 smithi090 ceph-mon[29180]: pgmap v15325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:16:01.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:00 smithi119 ceph-mon[32752]: pgmap v15326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:01.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:00 smithi167 ceph-mon[32751]: pgmap v15326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:01.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:00 smithi090 ceph-mon[29180]: pgmap v15326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:02.254 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27642.410654 secs 2024-04-27T02:16:02.255 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27643.241312 secs 2024-04-27T02:16:02.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27642.410654 secs 2024-04-27T02:16:02.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27643.241312 secs 2024-04-27T02:16:02.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27642.410654 secs 2024-04-27T02:16:02.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27643.241312 secs 2024-04-27T02:16:03.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:02 smithi119 ceph-mon[32752]: pgmap v15327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:03.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:02 smithi167 ceph-mon[32751]: pgmap v15327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:03.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:02 smithi090 ceph-mon[29180]: pgmap v15327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:05.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:04 smithi119 ceph-mon[32752]: pgmap v15328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:05.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:04 smithi167 ceph-mon[32751]: pgmap v15328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:05.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:04 smithi090 ceph-mon[29180]: pgmap v15328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:07.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27647.410763 secs 2024-04-27T02:16:07.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:06 smithi119 ceph-mon[32752]: pgmap v15329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:07.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27648.241427 secs 2024-04-27T02:16:07.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27647.410763 secs 2024-04-27T02:16:07.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:06 smithi167 ceph-mon[32751]: pgmap v15329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:07.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27648.241427 secs 2024-04-27T02:16:07.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27647.410763 secs 2024-04-27T02:16:07.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:06 smithi090 ceph-mon[29180]: pgmap v15329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:07.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27648.241427 secs 2024-04-27T02:16:09.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:08 smithi119 ceph-mon[32752]: pgmap v15330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:09.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:08 smithi167 ceph-mon[32751]: pgmap v15330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:09.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:08 smithi090 ceph-mon[29180]: pgmap v15330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:11.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:10 smithi119 ceph-mon[32752]: pgmap v15331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:11.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27652.410825 secs 2024-04-27T02:16:11.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:10 smithi167 ceph-mon[32751]: pgmap v15331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:11.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27652.410825 secs 2024-04-27T02:16:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:10 smithi090 ceph-mon[29180]: pgmap v15331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27652.410825 secs 2024-04-27T02:16:12.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:11 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27653.241545 secs 2024-04-27T02:16:12.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:11 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27653.241545 secs 2024-04-27T02:16:12.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:11 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27653.241545 secs 2024-04-27T02:16:13.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:12 smithi119 ceph-mon[32752]: pgmap v15332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:13.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:12 smithi167 ceph-mon[32751]: pgmap v15332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:13.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:12 smithi090 ceph-mon[29180]: pgmap v15332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:15.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:14 smithi119 ceph-mon[32752]: pgmap v15333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:15.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:14 smithi167 ceph-mon[32751]: pgmap v15333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:15.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:14 smithi090 ceph-mon[29180]: pgmap v15333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:16.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:15 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27657.410889 secs 2024-04-27T02:16:16.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:15 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27657.410889 secs 2024-04-27T02:16:16.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:15 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27657.410889 secs 2024-04-27T02:16:17.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:16 smithi119 ceph-mon[32752]: pgmap v15334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:17.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:16 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27658.241665 secs 2024-04-27T02:16:17.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:16 smithi167 ceph-mon[32751]: pgmap v15334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:17.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:16 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27658.241665 secs 2024-04-27T02:16:17.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:16 smithi090 ceph-mon[29180]: pgmap v15334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:17.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:16 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27658.241665 secs 2024-04-27T02:16:19.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:18 smithi119 ceph-mon[32752]: pgmap v15335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:19.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:18 smithi167 ceph-mon[32751]: pgmap v15335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:19.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:18 smithi090 ceph-mon[29180]: pgmap v15335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:21.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:20 smithi119 ceph-mon[32752]: pgmap v15336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:21.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:20 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27662.410952 secs 2024-04-27T02:16:21.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:20 smithi167 ceph-mon[32751]: pgmap v15336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:21.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:20 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27662.410952 secs 2024-04-27T02:16:21.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:20 smithi090 ceph-mon[29180]: pgmap v15336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:21.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:20 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27662.410952 secs 2024-04-27T02:16:22.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:21 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27663.241781 secs 2024-04-27T02:16:22.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:21 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27663.241781 secs 2024-04-27T02:16:22.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:21 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27663.241781 secs 2024-04-27T02:16:23.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:22 smithi119 ceph-mon[32752]: pgmap v15337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:23.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:22 smithi167 ceph-mon[32751]: pgmap v15337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:23.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:22 smithi090 ceph-mon[29180]: pgmap v15337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:25.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:24 smithi119 ceph-mon[32752]: pgmap v15338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:25.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:24 smithi167 ceph-mon[32751]: pgmap v15338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:25.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:24 smithi090 ceph-mon[29180]: pgmap v15338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:26.103 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:25 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27667.411040 secs 2024-04-27T02:16:26.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:25 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27667.411040 secs 2024-04-27T02:16:26.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:25 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27667.411040 secs 2024-04-27T02:16:27.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:26 smithi119 ceph-mon[32752]: pgmap v15339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:27.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:26 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27668.241863 secs 2024-04-27T02:16:27.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:26 smithi167 ceph-mon[32751]: pgmap v15339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:27.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:26 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27668.241863 secs 2024-04-27T02:16:27.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:26 smithi090 ceph-mon[29180]: pgmap v15339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:27.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:26 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27668.241863 secs 2024-04-27T02:16:29.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:28 smithi119 ceph-mon[32752]: pgmap v15340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:29.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:28 smithi167 ceph-mon[32751]: pgmap v15340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:29.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:28 smithi090 ceph-mon[29180]: pgmap v15340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:31.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:30 smithi119 ceph-mon[32752]: pgmap v15341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:31.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27672.411102 secs 2024-04-27T02:16:31.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:30 smithi167 ceph-mon[32751]: pgmap v15341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:31.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27672.411102 secs 2024-04-27T02:16:31.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:30 smithi090 ceph-mon[29180]: pgmap v15341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:31.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27672.411102 secs 2024-04-27T02:16:32.254 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:31 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27673.241939 secs 2024-04-27T02:16:32.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:31 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27673.241939 secs 2024-04-27T02:16:32.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:31 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27673.241939 secs 2024-04-27T02:16:33.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:33 smithi119 ceph-mon[32752]: pgmap v15342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:33.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:33 smithi167 ceph-mon[32751]: pgmap v15342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:33.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:33 smithi090 ceph-mon[29180]: pgmap v15342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:35.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:35 smithi119 ceph-mon[32752]: pgmap v15343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:35.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:35 smithi167 ceph-mon[32751]: pgmap v15343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:35.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:35 smithi090 ceph-mon[29180]: pgmap v15343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:36.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27677.411186 secs 2024-04-27T02:16:36.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27677.411186 secs 2024-04-27T02:16:36.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27677.411186 secs 2024-04-27T02:16:37.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:37 smithi119 ceph-mon[32752]: pgmap v15344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:37.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27678.242045 secs 2024-04-27T02:16:37.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:37 smithi167 ceph-mon[32751]: pgmap v15344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:37.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27678.242045 secs 2024-04-27T02:16:37.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:37 smithi090 ceph-mon[29180]: pgmap v15344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:37.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27678.242045 secs 2024-04-27T02:16:39.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:39 smithi119 ceph-mon[32752]: pgmap v15345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:39.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:39 smithi167 ceph-mon[32751]: pgmap v15345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:39.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:39 smithi090 ceph-mon[29180]: pgmap v15345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:41.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:41 smithi119 ceph-mon[32752]: pgmap v15346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:41.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27682.411287 secs 2024-04-27T02:16:41.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:41 smithi167 ceph-mon[32751]: pgmap v15346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:41.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27682.411287 secs 2024-04-27T02:16:41.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:41 smithi090 ceph-mon[29180]: pgmap v15346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:41.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27682.411287 secs 2024-04-27T02:16:42.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27683.242181 secs 2024-04-27T02:16:42.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27683.242181 secs 2024-04-27T02:16:42.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27683.242181 secs 2024-04-27T02:16:43.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:43 smithi119 ceph-mon[32752]: pgmap v15347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:43.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:43 smithi167 ceph-mon[32751]: pgmap v15347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:43.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:43 smithi090 ceph-mon[29180]: pgmap v15347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:45.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:45 smithi119 ceph-mon[32752]: pgmap v15348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:45.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:45 smithi167 ceph-mon[32751]: pgmap v15348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:45.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:45 smithi090 ceph-mon[29180]: pgmap v15348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:16:46.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27687.411370 secs 2024-04-27T02:16:46.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27687.411370 secs 2024-04-27T02:16:46.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27687.411370 secs 2024-04-27T02:16:47.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:47 smithi119 ceph-mon[32752]: pgmap v15349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:47.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27688.242325 secs 2024-04-27T02:16:47.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:47 smithi167 ceph-mon[32751]: pgmap v15349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:47.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27688.242325 secs 2024-04-27T02:16:47.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:47 smithi090 ceph-mon[29180]: pgmap v15349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:47.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27688.242325 secs 2024-04-27T02:16:48.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:16:48.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:16:48.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:16:49.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:49 smithi119 ceph-mon[32752]: pgmap v15350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:49.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:16:49.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:16:49.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:16:49.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:49 smithi167 ceph-mon[32751]: pgmap v15350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:49.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:16:49.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:16:49.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:16:49.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:49 smithi090 ceph-mon[29180]: pgmap v15350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:16:49.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:16:49.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:16:49.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:16:50.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:50 smithi119 ceph-mon[32752]: pgmap v15351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:16:50.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:50 smithi119 ceph-mon[32752]: pgmap v15352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:50.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:50 smithi167 ceph-mon[32751]: pgmap v15351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:16:50.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:50 smithi167 ceph-mon[32751]: pgmap v15352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:50 smithi090 ceph-mon[29180]: pgmap v15351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:16:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:50 smithi090 ceph-mon[29180]: pgmap v15352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:51.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27692.411452 secs 2024-04-27T02:16:51.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27692.411452 secs 2024-04-27T02:16:51.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27692.411452 secs 2024-04-27T02:16:52.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:52 smithi167 ceph-mon[32751]: pgmap v15353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:52.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27693.242407 secs 2024-04-27T02:16:52.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:52 smithi090 ceph-mon[29180]: pgmap v15353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:52.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27693.242407 secs 2024-04-27T02:16:52.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:52 smithi119 ceph-mon[32752]: pgmap v15353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:52.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27693.242407 secs 2024-04-27T02:16:54.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:54 smithi167 ceph-mon[32751]: pgmap v15354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:16:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:54 smithi090 ceph-mon[29180]: pgmap v15354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:16:54.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:54 smithi119 ceph-mon[32752]: pgmap v15354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:16:56.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:56 smithi167 ceph-mon[32751]: pgmap v15355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:56.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27697.411513 secs 2024-04-27T02:16:56.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:56 smithi090 ceph-mon[29180]: pgmap v15355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:56.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27697.411513 secs 2024-04-27T02:16:56.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:56 smithi119 ceph-mon[32752]: pgmap v15355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:56.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27697.411513 secs 2024-04-27T02:16:57.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27698.242515 secs 2024-04-27T02:16:57.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27698.242515 secs 2024-04-27T02:16:57.432 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27698.242515 secs 2024-04-27T02:16:58.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:16:58 smithi167 ceph-mon[32751]: pgmap v15356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:58.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:16:58 smithi090 ceph-mon[29180]: pgmap v15356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:16:58.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:16:58 smithi119 ceph-mon[32752]: pgmap v15356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:17:00.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:00 smithi090 ceph-mon[29180]: pgmap v15357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:17:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:00 smithi119 ceph-mon[32752]: pgmap v15357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:17:00.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:00 smithi167 ceph-mon[32751]: pgmap v15357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:17:01.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27702.411574 secs 2024-04-27T02:17:01.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27702.411574 secs 2024-04-27T02:17:01.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27702.411574 secs 2024-04-27T02:17:02.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:02 smithi090 ceph-mon[29180]: pgmap v15358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:02.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27703.242593 secs 2024-04-27T02:17:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:02 smithi119 ceph-mon[32752]: pgmap v15358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:02.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27703.242593 secs 2024-04-27T02:17:02.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:02 smithi167 ceph-mon[32751]: pgmap v15358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:02.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27703.242593 secs 2024-04-27T02:17:04.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:04 smithi090 ceph-mon[29180]: pgmap v15359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:04 smithi119 ceph-mon[32752]: pgmap v15359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:04 smithi167 ceph-mon[32751]: pgmap v15359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:06.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:06 smithi090 ceph-mon[29180]: pgmap v15360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:06.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27707.411659 secs 2024-04-27T02:17:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:06 smithi119 ceph-mon[32752]: pgmap v15360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27707.411659 secs 2024-04-27T02:17:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:06 smithi167 ceph-mon[32751]: pgmap v15360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27707.411659 secs 2024-04-27T02:17:07.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27708.242673 secs 2024-04-27T02:17:07.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27708.242673 secs 2024-04-27T02:17:07.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27708.242673 secs 2024-04-27T02:17:08.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:08 smithi090 ceph-mon[29180]: pgmap v15361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:08.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:08 smithi119 ceph-mon[32752]: pgmap v15361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:08.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:08 smithi167 ceph-mon[32751]: pgmap v15361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:10.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:10 smithi090 ceph-mon[29180]: pgmap v15362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:10.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:10 smithi119 ceph-mon[32752]: pgmap v15362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:10.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:10 smithi167 ceph-mon[32751]: pgmap v15362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27712.411744 secs 2024-04-27T02:17:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27712.411744 secs 2024-04-27T02:17:11.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27712.411744 secs 2024-04-27T02:17:12.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:12 smithi090 ceph-mon[29180]: pgmap v15363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:12.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27713.242750 secs 2024-04-27T02:17:12.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:12 smithi119 ceph-mon[32752]: pgmap v15363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:12.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27713.242750 secs 2024-04-27T02:17:12.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:12 smithi167 ceph-mon[32751]: pgmap v15363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:12.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27713.242750 secs 2024-04-27T02:17:14.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:14 smithi090 ceph-mon[29180]: pgmap v15364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:14.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:14 smithi119 ceph-mon[32752]: pgmap v15364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:14.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:14 smithi167 ceph-mon[32751]: pgmap v15364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:16.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:16 smithi090 ceph-mon[29180]: pgmap v15365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:16.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27717.411823 secs 2024-04-27T02:17:16.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:16 smithi119 ceph-mon[32752]: pgmap v15365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:16.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27717.411823 secs 2024-04-27T02:17:16.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:16 smithi167 ceph-mon[32751]: pgmap v15365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:16.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27717.411823 secs 2024-04-27T02:17:17.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27718.242890 secs 2024-04-27T02:17:17.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27718.242890 secs 2024-04-27T02:17:17.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27718.242890 secs 2024-04-27T02:17:18.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:18 smithi090 ceph-mon[29180]: pgmap v15366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:18.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:18 smithi119 ceph-mon[32752]: pgmap v15366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:18.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:18 smithi167 ceph-mon[32751]: pgmap v15366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:20.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:20 smithi090 ceph-mon[29180]: pgmap v15367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:20.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:20 smithi119 ceph-mon[32752]: pgmap v15367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:20.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:20 smithi167 ceph-mon[32751]: pgmap v15367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:21.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27722.411887 secs 2024-04-27T02:17:21.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27722.411887 secs 2024-04-27T02:17:21.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27722.411887 secs 2024-04-27T02:17:22.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:22 smithi119 ceph-mon[32752]: pgmap v15368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:22.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27723.243024 secs 2024-04-27T02:17:22.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:22 smithi167 ceph-mon[32751]: pgmap v15368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:22.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27723.243024 secs 2024-04-27T02:17:22.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:22 smithi090 ceph-mon[29180]: pgmap v15368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:22.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27723.243024 secs 2024-04-27T02:17:24.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:24 smithi119 ceph-mon[32752]: pgmap v15369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:24.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:24 smithi167 ceph-mon[32751]: pgmap v15369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:24.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:24 smithi090 ceph-mon[29180]: pgmap v15369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:26 smithi119 ceph-mon[32752]: pgmap v15370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27727.411974 secs 2024-04-27T02:17:26.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:26 smithi167 ceph-mon[32751]: pgmap v15370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:26.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27727.411974 secs 2024-04-27T02:17:26.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:26 smithi090 ceph-mon[29180]: pgmap v15370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:26.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27727.411974 secs 2024-04-27T02:17:27.432 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27728.243140 secs 2024-04-27T02:17:27.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27728.243140 secs 2024-04-27T02:17:27.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27728.243140 secs 2024-04-27T02:17:28.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:28 smithi119 ceph-mon[32752]: pgmap v15371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:28.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:28 smithi167 ceph-mon[32751]: pgmap v15371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:28.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:28 smithi090 ceph-mon[29180]: pgmap v15371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:30.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:30 smithi119 ceph-mon[32752]: pgmap v15372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:30.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:30 smithi167 ceph-mon[32751]: pgmap v15372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:30.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:30 smithi090 ceph-mon[29180]: pgmap v15372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:31.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27732.412037 secs 2024-04-27T02:17:31.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27732.412037 secs 2024-04-27T02:17:31.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27732.412037 secs 2024-04-27T02:17:32.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:32 smithi090 ceph-mon[29180]: pgmap v15373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:32.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27733.243218 secs 2024-04-27T02:17:32.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:32 smithi119 ceph-mon[32752]: pgmap v15373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:32.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27733.243218 secs 2024-04-27T02:17:32.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:32 smithi167 ceph-mon[32751]: pgmap v15373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:32.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27733.243218 secs 2024-04-27T02:17:34.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:34 smithi119 ceph-mon[32752]: pgmap v15374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:34.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:34 smithi167 ceph-mon[32751]: pgmap v15374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:34.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:34 smithi090 ceph-mon[29180]: pgmap v15374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:36.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:36 smithi119 ceph-mon[32752]: pgmap v15375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:36.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27737.412093 secs 2024-04-27T02:17:36.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:36 smithi167 ceph-mon[32751]: pgmap v15375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:36.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27737.412093 secs 2024-04-27T02:17:36.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:36 smithi090 ceph-mon[29180]: pgmap v15375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:36.640 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27737.412093 secs 2024-04-27T02:17:37.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27738.243337 secs 2024-04-27T02:17:37.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27738.243337 secs 2024-04-27T02:17:37.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27738.243337 secs 2024-04-27T02:17:38.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:38 smithi119 ceph-mon[32752]: pgmap v15376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:38.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:38 smithi167 ceph-mon[32751]: pgmap v15376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:38.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:38 smithi090 ceph-mon[29180]: pgmap v15376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:40.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:40 smithi119 ceph-mon[32752]: pgmap v15377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:40.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:40 smithi167 ceph-mon[32751]: pgmap v15377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:40.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:40 smithi090 ceph-mon[29180]: pgmap v15377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:41.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27742.412174 secs 2024-04-27T02:17:41.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27742.412174 secs 2024-04-27T02:17:41.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27742.412174 secs 2024-04-27T02:17:42.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:42 smithi119 ceph-mon[32752]: pgmap v15378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:42.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27743.243393 secs 2024-04-27T02:17:42.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:42 smithi167 ceph-mon[32751]: pgmap v15378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:42.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27743.243393 secs 2024-04-27T02:17:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:42 smithi090 ceph-mon[29180]: pgmap v15378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:42.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27743.243393 secs 2024-04-27T02:17:44.534 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:44 smithi119 ceph-mon[32752]: pgmap v15379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:44.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:44 smithi167 ceph-mon[32751]: pgmap v15379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:44 smithi090 ceph-mon[29180]: pgmap v15379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:46.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:46 smithi119 ceph-mon[32752]: pgmap v15380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:46.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27747.412243 secs 2024-04-27T02:17:46.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:46 smithi167 ceph-mon[32751]: pgmap v15380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:46.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27747.412243 secs 2024-04-27T02:17:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:46 smithi090 ceph-mon[29180]: pgmap v15380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27747.412243 secs 2024-04-27T02:17:47.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27748.243480 secs 2024-04-27T02:17:47.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27748.243480 secs 2024-04-27T02:17:47.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27748.243480 secs 2024-04-27T02:17:48.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:48 smithi119 ceph-mon[32752]: pgmap v15381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:48.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:48 smithi167 ceph-mon[32751]: pgmap v15381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:48.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:48 smithi090 ceph-mon[29180]: pgmap v15381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:49.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:17:49.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:17:49.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:17:50.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:50 smithi119 ceph-mon[32752]: pgmap v15382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:50.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:50 smithi090 ceph-mon[29180]: pgmap v15382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:50.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:50 smithi167 ceph-mon[32751]: pgmap v15382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:51.264 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27752.412336 secs 2024-04-27T02:17:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27752.412336 secs 2024-04-27T02:17:51.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27752.412336 secs 2024-04-27T02:17:52.505 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:52 smithi119 ceph-mon[32752]: pgmap v15383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:52.505 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27753.243550 secs 2024-04-27T02:17:52.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:52 smithi167 ceph-mon[32751]: pgmap v15383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:52.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27753.243550 secs 2024-04-27T02:17:52.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:52 smithi090 ceph-mon[29180]: pgmap v15383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:52.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27753.243550 secs 2024-04-27T02:17:54.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: pgmap v15384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:54.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:17:54.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:17:54.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: pgmap v15385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-27T02:17:54.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: pgmap v15386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T02:17:54.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: pgmap v15384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:54.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:17:54.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:17:54.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: pgmap v15385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-27T02:17:54.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: pgmap v15386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T02:17:54.296 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: pgmap v15384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:17:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:54.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:17:54.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:17:54.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: pgmap v15385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 B/s rd, 1 op/s 2024-04-27T02:17:54.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: pgmap v15386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T02:17:54.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:17:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:56 smithi119 ceph-mon[32752]: pgmap v15387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:56.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27757.412393 secs 2024-04-27T02:17:56.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:56 smithi167 ceph-mon[32751]: pgmap v15387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:56.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27757.412393 secs 2024-04-27T02:17:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:56 smithi090 ceph-mon[29180]: pgmap v15387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:17:56.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27757.412393 secs 2024-04-27T02:17:57.432 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27758.243646 secs 2024-04-27T02:17:57.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27758.243646 secs 2024-04-27T02:17:57.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27758.243646 secs 2024-04-27T02:17:58.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:17:58 smithi119 ceph-mon[32752]: pgmap v15388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T02:17:58.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:17:58 smithi167 ceph-mon[32751]: pgmap v15388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T02:17:58.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:17:58 smithi090 ceph-mon[29180]: pgmap v15388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 602 B/s rd, 1 op/s 2024-04-27T02:18:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:00 smithi119 ceph-mon[32752]: pgmap v15389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:00 smithi167 ceph-mon[32751]: pgmap v15389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:00 smithi090 ceph-mon[29180]: pgmap v15389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:01.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27762.412454 secs 2024-04-27T02:18:01.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27762.412454 secs 2024-04-27T02:18:01.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27762.412454 secs 2024-04-27T02:18:02.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:02 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27763.243760 secs 2024-04-27T02:18:02.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:02 smithi090 ceph-mon[29180]: pgmap v15390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:02 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27763.243760 secs 2024-04-27T02:18:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:02 smithi119 ceph-mon[32752]: pgmap v15390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:02.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:02 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27763.243760 secs 2024-04-27T02:18:02.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:02 smithi167 ceph-mon[32751]: pgmap v15390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:04.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:04 smithi119 ceph-mon[32752]: pgmap v15391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:18:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:04 smithi167 ceph-mon[32751]: pgmap v15391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:18:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:04 smithi090 ceph-mon[29180]: pgmap v15391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:18:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:06 smithi119 ceph-mon[32752]: pgmap v15392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:06.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27767.412538 secs 2024-04-27T02:18:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:06 smithi167 ceph-mon[32751]: pgmap v15392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27767.412538 secs 2024-04-27T02:18:06.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:06 smithi090 ceph-mon[29180]: pgmap v15392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:06.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27767.412538 secs 2024-04-27T02:18:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:07 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27768.243878 secs 2024-04-27T02:18:07.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:07 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27768.243878 secs 2024-04-27T02:18:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:07 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27768.243878 secs 2024-04-27T02:18:08.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:08 smithi119 ceph-mon[32752]: pgmap v15393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:08.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:08 smithi167 ceph-mon[32751]: pgmap v15393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:08.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:08 smithi090 ceph-mon[29180]: pgmap v15393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:09.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:09 smithi119 ceph-mon[32752]: pgmap v15394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:09.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:09 smithi167 ceph-mon[32751]: pgmap v15394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:09.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:09 smithi090 ceph-mon[29180]: pgmap v15394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:11.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:10 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27772.412621 secs 2024-04-27T02:18:11.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:10 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27772.412621 secs 2024-04-27T02:18:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:10 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27772.412621 secs 2024-04-27T02:18:12.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:12 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27773.243986 secs 2024-04-27T02:18:12.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:12 smithi119 ceph-mon[32752]: pgmap v15395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:12.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:12 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27773.243986 secs 2024-04-27T02:18:12.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:12 smithi167 ceph-mon[32751]: pgmap v15395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:12.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:12 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27773.243986 secs 2024-04-27T02:18:12.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:12 smithi090 ceph-mon[29180]: pgmap v15395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:14.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:14 smithi119 ceph-mon[32752]: pgmap v15396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:14.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:14 smithi167 ceph-mon[32751]: pgmap v15396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:14.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:14 smithi090 ceph-mon[29180]: pgmap v15396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:16.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:16 smithi119 ceph-mon[32752]: pgmap v15397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:16.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27777.412704 secs 2024-04-27T02:18:16.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:16 smithi167 ceph-mon[32751]: pgmap v15397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:16.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27777.412704 secs 2024-04-27T02:18:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:16 smithi090 ceph-mon[29180]: pgmap v15397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:16.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27777.412704 secs 2024-04-27T02:18:17.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:17 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27778.244051 secs 2024-04-27T02:18:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:17 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27778.244051 secs 2024-04-27T02:18:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:17 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27778.244051 secs 2024-04-27T02:18:18.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:18 smithi119 ceph-mon[32752]: pgmap v15398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:18.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:18 smithi167 ceph-mon[32751]: pgmap v15398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:18.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:18 smithi090 ceph-mon[29180]: pgmap v15398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:20.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:20 smithi119 ceph-mon[32752]: pgmap v15399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:20.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:20 smithi167 ceph-mon[32751]: pgmap v15399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:20 smithi090 ceph-mon[29180]: pgmap v15399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:21.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27782.412796 secs 2024-04-27T02:18:21.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27782.412796 secs 2024-04-27T02:18:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27782.412796 secs 2024-04-27T02:18:22.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:22 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27783.244186 secs 2024-04-27T02:18:22.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:22 smithi119 ceph-mon[32752]: pgmap v15400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:22.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:22 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27783.244186 secs 2024-04-27T02:18:22.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:22 smithi167 ceph-mon[32751]: pgmap v15400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:22 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27783.244186 secs 2024-04-27T02:18:22.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:22 smithi090 ceph-mon[29180]: pgmap v15400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:24.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:24 smithi119 ceph-mon[32752]: pgmap v15401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:24.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:24 smithi167 ceph-mon[32751]: pgmap v15401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:24.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:24 smithi090 ceph-mon[29180]: pgmap v15401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:26.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:26 smithi119 ceph-mon[32752]: pgmap v15402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:26.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27787.412871 secs 2024-04-27T02:18:26.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:26 smithi167 ceph-mon[32751]: pgmap v15402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:26.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27787.412871 secs 2024-04-27T02:18:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:26 smithi090 ceph-mon[29180]: pgmap v15402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:26.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27787.412871 secs 2024-04-27T02:18:27.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:27 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27788.244307 secs 2024-04-27T02:18:27.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:27 smithi119 ceph-mon[32752]: pgmap v15403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:27.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:27 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27788.244307 secs 2024-04-27T02:18:27.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:27 smithi167 ceph-mon[32751]: pgmap v15403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:27 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27788.244307 secs 2024-04-27T02:18:27.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:27 smithi090 ceph-mon[29180]: pgmap v15403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:30.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:30 smithi119 ceph-mon[32752]: pgmap v15404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:30.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:30 smithi167 ceph-mon[32751]: pgmap v15404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:30.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:30 smithi090 ceph-mon[29180]: pgmap v15404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:31.446 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27792.412930 secs 2024-04-27T02:18:31.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27792.412930 secs 2024-04-27T02:18:31.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27792.412930 secs 2024-04-27T02:18:32.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:32 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27793.244405 secs 2024-04-27T02:18:32.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:32 smithi090 ceph-mon[29180]: pgmap v15405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:32.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:32 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27793.244405 secs 2024-04-27T02:18:32.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:32 smithi119 ceph-mon[32752]: pgmap v15405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:32.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:32 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27793.244405 secs 2024-04-27T02:18:32.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:32 smithi167 ceph-mon[32751]: pgmap v15405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:34.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:34 smithi119 ceph-mon[32752]: pgmap v15406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:34.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:34 smithi167 ceph-mon[32751]: pgmap v15406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:34.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:34 smithi090 ceph-mon[29180]: pgmap v15406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:36.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:36 smithi119 ceph-mon[32752]: pgmap v15407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:36.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27797.412989 secs 2024-04-27T02:18:36.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:36 smithi167 ceph-mon[32751]: pgmap v15407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:36.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27797.412989 secs 2024-04-27T02:18:36.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:36 smithi090 ceph-mon[29180]: pgmap v15407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:36.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27797.412989 secs 2024-04-27T02:18:37.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:37 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27798.244522 secs 2024-04-27T02:18:37.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:37 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27798.244522 secs 2024-04-27T02:18:37.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:37 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27798.244522 secs 2024-04-27T02:18:38.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:38 smithi119 ceph-mon[32752]: pgmap v15408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:38.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:38 smithi167 ceph-mon[32751]: pgmap v15408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:38.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:38 smithi090 ceph-mon[29180]: pgmap v15408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:39.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:39 smithi119 ceph-mon[32752]: pgmap v15409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:39.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:39 smithi167 ceph-mon[32751]: pgmap v15409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:39.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:39 smithi090 ceph-mon[29180]: pgmap v15409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:41.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27802.413073 secs 2024-04-27T02:18:41.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27802.413073 secs 2024-04-27T02:18:41.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27802.413073 secs 2024-04-27T02:18:42.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:42 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27803.244661 secs 2024-04-27T02:18:42.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:42 smithi119 ceph-mon[32752]: pgmap v15410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:42.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:42 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27803.244661 secs 2024-04-27T02:18:42.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:42 smithi167 ceph-mon[32751]: pgmap v15410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:42.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:42 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27803.244661 secs 2024-04-27T02:18:42.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:42 smithi090 ceph-mon[29180]: pgmap v15410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:44.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:44 smithi119 ceph-mon[32752]: pgmap v15411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:44.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:44 smithi167 ceph-mon[32751]: pgmap v15411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:44 smithi090 ceph-mon[29180]: pgmap v15411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:46.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:46 smithi119 ceph-mon[32752]: pgmap v15412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:46.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27807.413155 secs 2024-04-27T02:18:46.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:46 smithi167 ceph-mon[32751]: pgmap v15412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:46.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27807.413155 secs 2024-04-27T02:18:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:46 smithi090 ceph-mon[29180]: pgmap v15412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:46.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27807.413155 secs 2024-04-27T02:18:47.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:47 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27808.244821 secs 2024-04-27T02:18:47.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:47 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27808.244821 secs 2024-04-27T02:18:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:47 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27808.244821 secs 2024-04-27T02:18:48.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:48 smithi119 ceph-mon[32752]: pgmap v15413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:48.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:48 smithi167 ceph-mon[32751]: pgmap v15413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:48 smithi090 ceph-mon[29180]: pgmap v15413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:50.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:50 smithi119 ceph-mon[32752]: pgmap v15414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:50.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:50 smithi167 ceph-mon[32751]: pgmap v15414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:50.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:50 smithi090 ceph-mon[29180]: pgmap v15414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:51.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27812.413228 secs 2024-04-27T02:18:51.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27812.413228 secs 2024-04-27T02:18:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27812.413228 secs 2024-04-27T02:18:52.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:52 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27813.244889 secs 2024-04-27T02:18:52.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:52 smithi119 ceph-mon[32752]: pgmap v15415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:52.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:52 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27813.244889 secs 2024-04-27T02:18:52.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:52 smithi167 ceph-mon[32751]: pgmap v15415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:52.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:52 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27813.244889 secs 2024-04-27T02:18:52.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:52 smithi090 ceph-mon[29180]: pgmap v15415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:53.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:53 smithi119 ceph-mon[32752]: pgmap v15416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:53.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:53 smithi167 ceph-mon[32751]: pgmap v15416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:53.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:53 smithi090 ceph-mon[29180]: pgmap v15416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:54.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:18:54.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:18:54.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:18:55.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:55 smithi119 ceph-mon[32752]: pgmap v15417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:55.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:55 smithi167 ceph-mon[32751]: pgmap v15417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:55.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:55 smithi090 ceph-mon[29180]: pgmap v15417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:18:56.501 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27817.413311 secs 2024-04-27T02:18:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27817.413311 secs 2024-04-27T02:18:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27817.413311 secs 2024-04-27T02:18:57.377 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:57 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27818.244962 secs 2024-04-27T02:18:57.378 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:57 smithi167 ceph-mon[32751]: pgmap v15418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:57.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:57 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27818.244962 secs 2024-04-27T02:18:57.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:57 smithi119 ceph-mon[32752]: pgmap v15418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:57.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:57 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27818.244962 secs 2024-04-27T02:18:57.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:57 smithi090 ceph-mon[29180]: pgmap v15418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:18:58.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:18:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:18:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:58 smithi119 ceph-mon[32752]: pgmap v15419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 992 B/s rd, 1 op/s 2024-04-27T02:18:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:58 smithi119 ceph-mon[32752]: pgmap v15420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-27T02:18:58.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:18:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:18:58.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:18:58.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:58 smithi167 ceph-mon[32751]: pgmap v15419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 992 B/s rd, 1 op/s 2024-04-27T02:18:58.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:58 smithi167 ceph-mon[32751]: pgmap v15420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-27T02:18:58.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:18:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:18:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:18:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:18:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:58 smithi090 ceph-mon[29180]: pgmap v15419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 992 B/s rd, 1 op/s 2024-04-27T02:18:58.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:58 smithi090 ceph-mon[29180]: pgmap v15420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 B/s rd, 1 op/s 2024-04-27T02:18:58.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:18:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:19:00.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:00 smithi119 ceph-mon[32752]: pgmap v15421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:00.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:00 smithi167 ceph-mon[32751]: pgmap v15421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:00.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:00 smithi090 ceph-mon[29180]: pgmap v15421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:01.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27822.413396 secs 2024-04-27T02:19:01.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:01 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27823.245021 secs 2024-04-27T02:19:01.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27822.413396 secs 2024-04-27T02:19:01.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:01 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27823.245021 secs 2024-04-27T02:19:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27822.413396 secs 2024-04-27T02:19:01.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:01 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27823.245021 secs 2024-04-27T02:19:02.754 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:02 smithi090 ceph-mon[29180]: pgmap v15422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:02.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:02 smithi119 ceph-mon[32752]: pgmap v15422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:02.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:02 smithi167 ceph-mon[32751]: pgmap v15422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:04.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:04 smithi119 ceph-mon[32752]: pgmap v15423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:04.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:04 smithi167 ceph-mon[32751]: pgmap v15423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:04.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:04 smithi090 ceph-mon[29180]: pgmap v15423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:06.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27827.413477 secs 2024-04-27T02:19:06.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:06 smithi119 ceph-mon[32752]: pgmap v15424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:06.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:06 smithi119 ceph-mon[32752]: 26 slow requests, 0 included below; oldest blocked for > 27828.245118 secs 2024-04-27T02:19:06.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27827.413477 secs 2024-04-27T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:06 smithi167 ceph-mon[32751]: pgmap v15424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:06.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:06 smithi167 ceph-mon[32751]: 26 slow requests, 0 included below; oldest blocked for > 27828.245118 secs 2024-04-27T02:19:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27827.413477 secs 2024-04-27T02:19:06.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:06 smithi090 ceph-mon[29180]: pgmap v15424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:06.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:06 smithi090 ceph-mon[29180]: 26 slow requests, 0 included below; oldest blocked for > 27828.245118 secs 2024-04-27T02:19:08.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:08 smithi119 ceph-mon[32752]: pgmap v15425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:19:08.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:08 smithi167 ceph-mon[32751]: pgmap v15425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:19:08.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:08 smithi090 ceph-mon[29180]: pgmap v15425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:19:08.853 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:19:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:19:08.473+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:19:08.853 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:19:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:19:08.473+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:19:08.853 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:19:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:19:08.473+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:19:08.853 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:19:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:19:08.473+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:19:08.853 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:19:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:19:08.473+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:19:10.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:10 smithi119 ceph-mon[32752]: pgmap v15426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:10.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:10 smithi167 ceph-mon[32751]: pgmap v15426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:10.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:10 smithi090 ceph-mon[29180]: pgmap v15426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:11.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:11 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27832.413561 secs 2024-04-27T02:19:11.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:11 smithi167 ceph-mon[32751]: 7 slow requests, 1 included below; oldest blocked for > 27833.245233 secs 2024-04-27T02:19:11.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:11 smithi167 ceph-mon[32751]: slow request 5764.126367 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:19:11.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:11 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27832.413561 secs 2024-04-27T02:19:11.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:11 smithi090 ceph-mon[29180]: 7 slow requests, 1 included below; oldest blocked for > 27833.245233 secs 2024-04-27T02:19:11.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:11 smithi090 ceph-mon[29180]: slow request 5764.126367 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:19:12.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:11 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27832.413561 secs 2024-04-27T02:19:12.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:11 smithi119 ceph-mon[32752]: 7 slow requests, 1 included below; oldest blocked for > 27833.245233 secs 2024-04-27T02:19:12.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:11 smithi119 ceph-mon[32752]: slow request 5764.126367 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:19:12.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:12 smithi167 ceph-mon[32751]: pgmap v15427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:12.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:19:12.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:19:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:12 smithi090 ceph-mon[29180]: pgmap v15427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:19:12.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:19:13.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:12 smithi119 ceph-mon[32752]: pgmap v15427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:13.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:19:13.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:19:14.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:14 smithi167 ceph-mon[32751]: pgmap v15428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:14.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:14 smithi090 ceph-mon[29180]: pgmap v15428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:15.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:14 smithi119 ceph-mon[32752]: pgmap v15428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:16.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27837.413644 secs 2024-04-27T02:19:16.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:16 smithi167 ceph-mon[32751]: pgmap v15429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:16.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:16 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27838.245298 secs 2024-04-27T02:19:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27837.413644 secs 2024-04-27T02:19:16.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:16 smithi090 ceph-mon[29180]: pgmap v15429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:16.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:16 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27838.245298 secs 2024-04-27T02:19:17.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27837.413644 secs 2024-04-27T02:19:17.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:16 smithi119 ceph-mon[32752]: pgmap v15429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:17.031 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:16 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27838.245298 secs 2024-04-27T02:19:18.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:18 smithi167 ceph-mon[32751]: pgmap v15430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:18.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:18 smithi090 ceph-mon[29180]: pgmap v15430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:19.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:18 smithi119 ceph-mon[32752]: pgmap v15430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:20.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:20 smithi167 ceph-mon[32751]: pgmap v15431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:20.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:20 smithi090 ceph-mon[29180]: pgmap v15431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:21.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:20 smithi119 ceph-mon[32752]: pgmap v15431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27842.413730 secs 2024-04-27T02:19:21.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:21 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27843.245418 secs 2024-04-27T02:19:22.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27842.413730 secs 2024-04-27T02:19:22.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:21 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27843.245418 secs 2024-04-27T02:19:22.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27842.413730 secs 2024-04-27T02:19:22.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:21 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27843.245418 secs 2024-04-27T02:19:22.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:22 smithi090 ceph-mon[29180]: pgmap v15432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:23.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:22 smithi119 ceph-mon[32752]: pgmap v15432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:23.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:22 smithi167 ceph-mon[32751]: pgmap v15432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:24.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:24 smithi090 ceph-mon[29180]: pgmap v15433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:25.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:24 smithi119 ceph-mon[32752]: pgmap v15433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:25.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:24 smithi167 ceph-mon[32751]: pgmap v15433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:19:26.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27847.413813 secs 2024-04-27T02:19:26.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:26 smithi090 ceph-mon[29180]: pgmap v15434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:26.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:26 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27848.245526 secs 2024-04-27T02:19:27.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27847.413813 secs 2024-04-27T02:19:27.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:26 smithi119 ceph-mon[32752]: pgmap v15434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:27.030 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:26 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27848.245526 secs 2024-04-27T02:19:27.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27847.413813 secs 2024-04-27T02:19:27.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:26 smithi167 ceph-mon[32751]: pgmap v15434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:27.045 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:26 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27848.245526 secs 2024-04-27T02:19:28.832 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:28 smithi119 ceph-mon[32752]: pgmap v15435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:28.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:28 smithi090 ceph-mon[29180]: pgmap v15435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:29.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:28 smithi167 ceph-mon[32751]: pgmap v15435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:31.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:30 smithi119 ceph-mon[32752]: pgmap v15436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:31.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:30 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27852.413896 secs 2024-04-27T02:19:31.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:30 smithi167 ceph-mon[32751]: pgmap v15436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:31.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:30 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27852.413896 secs 2024-04-27T02:19:31.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:30 smithi090 ceph-mon[29180]: pgmap v15436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:31.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:30 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27852.413896 secs 2024-04-27T02:19:32.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:31 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27853.245676 secs 2024-04-27T02:19:32.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:31 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27853.245676 secs 2024-04-27T02:19:32.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:31 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27853.245676 secs 2024-04-27T02:19:33.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:32 smithi119 ceph-mon[32752]: pgmap v15437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:33.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:32 smithi167 ceph-mon[32751]: pgmap v15437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:33.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:32 smithi090 ceph-mon[29180]: pgmap v15437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:35.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:34 smithi119 ceph-mon[32752]: pgmap v15438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:35.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:34 smithi167 ceph-mon[32751]: pgmap v15438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:35.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:34 smithi090 ceph-mon[29180]: pgmap v15438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:36.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27857.413987 secs 2024-04-27T02:19:36.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27857.413987 secs 2024-04-27T02:19:36.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27857.413987 secs 2024-04-27T02:19:37.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:37 smithi119 ceph-mon[32752]: pgmap v15439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:37.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:37 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27858.245777 secs 2024-04-27T02:19:37.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:37 smithi167 ceph-mon[32751]: pgmap v15439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:37.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:37 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27858.245777 secs 2024-04-27T02:19:37.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:37 smithi090 ceph-mon[29180]: pgmap v15439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:37.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:37 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27858.245777 secs 2024-04-27T02:19:39.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:39 smithi119 ceph-mon[32752]: pgmap v15440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:39.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:39 smithi167 ceph-mon[32751]: pgmap v15440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:39.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:39 smithi090 ceph-mon[29180]: pgmap v15440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:41.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:41 smithi119 ceph-mon[32752]: pgmap v15441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:41.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27862.414073 secs 2024-04-27T02:19:41.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:41 smithi167 ceph-mon[32751]: pgmap v15441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:41.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27862.414073 secs 2024-04-27T02:19:41.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:41 smithi090 ceph-mon[29180]: pgmap v15441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:41.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27862.414073 secs 2024-04-27T02:19:42.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:42 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27863.245845 secs 2024-04-27T02:19:42.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:42 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27863.245845 secs 2024-04-27T02:19:42.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:42 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27863.245845 secs 2024-04-27T02:19:43.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:43 smithi119 ceph-mon[32752]: pgmap v15442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:43.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:43 smithi167 ceph-mon[32751]: pgmap v15442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:43.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:43 smithi090 ceph-mon[29180]: pgmap v15442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:45.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:45 smithi119 ceph-mon[32752]: pgmap v15443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:45.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:45 smithi167 ceph-mon[32751]: pgmap v15443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:45.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:45 smithi090 ceph-mon[29180]: pgmap v15443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:46.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27867.414159 secs 2024-04-27T02:19:46.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27867.414159 secs 2024-04-27T02:19:46.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27867.414159 secs 2024-04-27T02:19:47.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:47 smithi119 ceph-mon[32752]: pgmap v15444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:47.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:47 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27868.245941 secs 2024-04-27T02:19:47.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:47 smithi167 ceph-mon[32751]: pgmap v15444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:47.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:47 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27868.245941 secs 2024-04-27T02:19:47.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:47 smithi090 ceph-mon[29180]: pgmap v15444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:47.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:47 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27868.245941 secs 2024-04-27T02:19:49.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:49 smithi119 ceph-mon[32752]: pgmap v15445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:49.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:49 smithi167 ceph-mon[32751]: pgmap v15445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:49.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:49 smithi090 ceph-mon[29180]: pgmap v15445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:51.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:51 smithi119 ceph-mon[32752]: pgmap v15446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:51.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27872.414261 secs 2024-04-27T02:19:51.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:51 smithi167 ceph-mon[32751]: pgmap v15446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:51.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27872.414261 secs 2024-04-27T02:19:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:51 smithi090 ceph-mon[29180]: pgmap v15446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27872.414261 secs 2024-04-27T02:19:52.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:52 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27873.246075 secs 2024-04-27T02:19:52.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:52 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27873.246075 secs 2024-04-27T02:19:52.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:52 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27873.246075 secs 2024-04-27T02:19:53.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:53 smithi119 ceph-mon[32752]: pgmap v15447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:53.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:53 smithi167 ceph-mon[32751]: pgmap v15447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:53.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:53 smithi090 ceph-mon[29180]: pgmap v15447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:55.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:55 smithi119 ceph-mon[32752]: pgmap v15448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:55.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:55 smithi167 ceph-mon[32751]: pgmap v15448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:55.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:55 smithi090 ceph-mon[29180]: pgmap v15448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:19:56.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:56 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27877.414343 secs 2024-04-27T02:19:56.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:56 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27877.414343 secs 2024-04-27T02:19:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:56 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27877.414343 secs 2024-04-27T02:19:57.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:57 smithi167 ceph-mon[32751]: pgmap v15449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:57.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:57 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27878.246208 secs 2024-04-27T02:19:57.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:57 smithi090 ceph-mon[29180]: pgmap v15449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:57.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:57 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27878.246208 secs 2024-04-27T02:19:57.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:57 smithi119 ceph-mon[32752]: pgmap v15449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:57.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:57 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27878.246208 secs 2024-04-27T02:19:58.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:19:58.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:19:58.431 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:19:59.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:59 smithi167 ceph-mon[32751]: pgmap v15450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:59.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:19:59.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:19:59.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:19:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:19:59.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:59 smithi090 ceph-mon[29180]: pgmap v15450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:59.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:19:59.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:19:59.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:19:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:19:59.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:59 smithi119 ceph-mon[32752]: pgmap v15450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:19:59.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:19:59.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:19:59.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:19:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:20:00.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:00 smithi167 ceph-mon[32751]: pgmap v15451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:20:00.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:00 smithi167 ceph-mon[32751]: pgmap v15452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:20:00.295 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:20:00.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:00 smithi090 ceph-mon[29180]: pgmap v15451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:20:00.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:00 smithi090 ceph-mon[29180]: pgmap v15452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:20:00.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:20:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:00 smithi119 ceph-mon[32752]: pgmap v15451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:20:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:00 smithi119 ceph-mon[32752]: pgmap v15452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:20:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:20:01.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:01 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27882.414427 secs 2024-04-27T02:20:01.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:01 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27882.414427 secs 2024-04-27T02:20:01.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:01 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27882.414427 secs 2024-04-27T02:20:02.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:02 smithi090 ceph-mon[29180]: pgmap v15453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:20:02.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:02 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27883.246299 secs 2024-04-27T02:20:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:02 smithi119 ceph-mon[32752]: pgmap v15453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:20:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:02 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27883.246299 secs 2024-04-27T02:20:02.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:02 smithi167 ceph-mon[32751]: pgmap v15453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:20:02.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:02 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27883.246299 secs 2024-04-27T02:20:04.340 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:04 smithi119 ceph-mon[32752]: pgmap v15454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:20:04.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:04 smithi090 ceph-mon[29180]: pgmap v15454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:20:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:04 smithi167 ceph-mon[32751]: pgmap v15454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:20:06.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:06 smithi090 ceph-mon[29180]: pgmap v15455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:06.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:06 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27887.414513 secs 2024-04-27T02:20:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:06 smithi119 ceph-mon[32752]: pgmap v15455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:06 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27887.414513 secs 2024-04-27T02:20:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:06 smithi167 ceph-mon[32751]: pgmap v15455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:06 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27887.414513 secs 2024-04-27T02:20:07.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:07 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27888.246419 secs 2024-04-27T02:20:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:07 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27888.246419 secs 2024-04-27T02:20:07.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:07 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27888.246419 secs 2024-04-27T02:20:08.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:08 smithi090 ceph-mon[29180]: pgmap v15456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:08.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:08 smithi119 ceph-mon[32752]: pgmap v15456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:08.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:08 smithi167 ceph-mon[32751]: pgmap v15456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:10.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:10 smithi090 ceph-mon[29180]: pgmap v15457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:10.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:10 smithi119 ceph-mon[32752]: pgmap v15457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:10.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:10 smithi167 ceph-mon[32751]: pgmap v15457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:11 smithi090 ceph-mon[29180]: 36 slow requests, 1 included below; oldest blocked for > 27892.414598 secs 2024-04-27T02:20:11.353 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:11 smithi090 ceph-mon[29180]: slow request 11524.920611 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:20:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:11 smithi119 ceph-mon[32752]: 36 slow requests, 1 included below; oldest blocked for > 27892.414598 secs 2024-04-27T02:20:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:11 smithi119 ceph-mon[32752]: slow request 11524.920611 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:20:11.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:11 smithi167 ceph-mon[32751]: 36 slow requests, 1 included below; oldest blocked for > 27892.414598 secs 2024-04-27T02:20:11.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:11 smithi167 ceph-mon[32751]: slow request 11524.920611 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:20:12.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:12 smithi090 ceph-mon[29180]: pgmap v15458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:20:12.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:12 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27893.246532 secs 2024-04-27T02:20:12.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:12 smithi119 ceph-mon[32752]: pgmap v15458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:20:12.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:12 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27893.246532 secs 2024-04-27T02:20:12.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:12 smithi167 ceph-mon[32751]: pgmap v15458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:20:12.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:12 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27893.246532 secs 2024-04-27T02:20:14.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:14 smithi090 ceph-mon[29180]: pgmap v15459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:20:14.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:14 smithi119 ceph-mon[32752]: pgmap v15459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:20:14.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:14 smithi167 ceph-mon[32751]: pgmap v15459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:20:16.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:16 smithi090 ceph-mon[29180]: pgmap v15460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:16.359 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:16 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27897.414678 secs 2024-04-27T02:20:16.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:16 smithi119 ceph-mon[32752]: pgmap v15460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:16.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:16 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27897.414678 secs 2024-04-27T02:20:16.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:16 smithi167 ceph-mon[32751]: pgmap v15460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:20:16.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:16 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27897.414678 secs 2024-04-27T02:20:17.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:17 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27898.246630 secs 2024-04-27T02:20:17.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:17 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27898.246630 secs 2024-04-27T02:20:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:17 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27898.246630 secs 2024-04-27T02:20:18.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:18 smithi090 ceph-mon[29180]: pgmap v15461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:18.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:18 smithi119 ceph-mon[32752]: pgmap v15461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:18.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:18 smithi167 ceph-mon[32751]: pgmap v15461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:20.354 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:20 smithi090 ceph-mon[29180]: pgmap v15462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:20.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:20 smithi119 ceph-mon[32752]: pgmap v15462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:20.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:20 smithi167 ceph-mon[32751]: pgmap v15462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:21.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:21 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27902.414765 secs 2024-04-27T02:20:21.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:21 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27902.414765 secs 2024-04-27T02:20:21.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:21 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27902.414765 secs 2024-04-27T02:20:22.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:22 smithi090 ceph-mon[29180]: pgmap v15463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:22.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:22 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27903.246728 secs 2024-04-27T02:20:22.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:22 smithi119 ceph-mon[32752]: pgmap v15463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:22.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:22 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27903.246728 secs 2024-04-27T02:20:22.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:22 smithi167 ceph-mon[32751]: pgmap v15463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:22.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:22 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27903.246728 secs 2024-04-27T02:20:24.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:24 smithi119 ceph-mon[32752]: pgmap v15464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:24.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:24 smithi167 ceph-mon[32751]: pgmap v15464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:24.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:24 smithi090 ceph-mon[29180]: pgmap v15464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:26.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:26 smithi119 ceph-mon[32752]: pgmap v15465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:26 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27907.414825 secs 2024-04-27T02:20:26.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:26 smithi167 ceph-mon[32751]: pgmap v15465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:26.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:26 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27907.414825 secs 2024-04-27T02:20:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:26 smithi090 ceph-mon[29180]: pgmap v15465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:26 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27907.414825 secs 2024-04-27T02:20:27.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:27 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27908.246814 secs 2024-04-27T02:20:27.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:27 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27908.246814 secs 2024-04-27T02:20:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:27 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27908.246814 secs 2024-04-27T02:20:28.431 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:28 smithi119 ceph-mon[32752]: pgmap v15466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:28.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:28 smithi167 ceph-mon[32751]: pgmap v15466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:28.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:28 smithi090 ceph-mon[29180]: pgmap v15466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:30.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:30 smithi119 ceph-mon[32752]: pgmap v15467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:30.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:30 smithi167 ceph-mon[32751]: pgmap v15467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:30.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:30 smithi090 ceph-mon[29180]: pgmap v15467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:31.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:31 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27912.414886 secs 2024-04-27T02:20:31.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:31 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27912.414886 secs 2024-04-27T02:20:31.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:31 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27912.414886 secs 2024-04-27T02:20:32.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:32 smithi090 ceph-mon[29180]: pgmap v15468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:32.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:32 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27913.246875 secs 2024-04-27T02:20:32.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:32 smithi119 ceph-mon[32752]: pgmap v15468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:32.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:32 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27913.246875 secs 2024-04-27T02:20:32.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:32 smithi167 ceph-mon[32751]: pgmap v15468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:32.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:32 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27913.246875 secs 2024-04-27T02:20:34.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:34 smithi119 ceph-mon[32752]: pgmap v15469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:34.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:34 smithi167 ceph-mon[32751]: pgmap v15469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:34.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:34 smithi090 ceph-mon[29180]: pgmap v15469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:36.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:36 smithi119 ceph-mon[32752]: pgmap v15470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:36.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:36 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27917.414949 secs 2024-04-27T02:20:36.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:36 smithi167 ceph-mon[32751]: pgmap v15470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:36.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:36 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27917.414949 secs 2024-04-27T02:20:36.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:36 smithi090 ceph-mon[29180]: pgmap v15470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:36.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:36 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27917.414949 secs 2024-04-27T02:20:37.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:37 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27918.246951 secs 2024-04-27T02:20:37.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:37 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27918.246951 secs 2024-04-27T02:20:37.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:37 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27918.246951 secs 2024-04-27T02:20:38.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:38 smithi119 ceph-mon[32752]: pgmap v15471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:38.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:38 smithi167 ceph-mon[32751]: pgmap v15471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:38.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:38 smithi090 ceph-mon[29180]: pgmap v15471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:40.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:40 smithi119 ceph-mon[32752]: pgmap v15472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:40.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:40 smithi167 ceph-mon[32751]: pgmap v15472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:40.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:40 smithi090 ceph-mon[29180]: pgmap v15472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:41.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:41 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27922.415012 secs 2024-04-27T02:20:41.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:41 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27922.415012 secs 2024-04-27T02:20:41.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:41 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27922.415012 secs 2024-04-27T02:20:42.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:42 smithi119 ceph-mon[32752]: pgmap v15473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:42.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:42 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27923.247081 secs 2024-04-27T02:20:42.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:42 smithi167 ceph-mon[32751]: pgmap v15473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:42.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:42 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27923.247081 secs 2024-04-27T02:20:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:42 smithi090 ceph-mon[29180]: pgmap v15473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:42 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27923.247081 secs 2024-04-27T02:20:44.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:44 smithi119 ceph-mon[32752]: pgmap v15474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:44.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:44 smithi167 ceph-mon[32751]: pgmap v15474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:44 smithi090 ceph-mon[29180]: pgmap v15474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:46.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:46 smithi119 ceph-mon[32752]: pgmap v15475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:46.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:46 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27927.415072 secs 2024-04-27T02:20:46.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:46 smithi167 ceph-mon[32751]: pgmap v15475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:46.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:46 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27927.415072 secs 2024-04-27T02:20:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:46 smithi090 ceph-mon[29180]: pgmap v15475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:46 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27927.415072 secs 2024-04-27T02:20:47.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:47 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27928.247197 secs 2024-04-27T02:20:47.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:47 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27928.247197 secs 2024-04-27T02:20:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:47 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27928.247197 secs 2024-04-27T02:20:48.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:48 smithi119 ceph-mon[32752]: pgmap v15476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:48.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:48 smithi167 ceph-mon[32751]: pgmap v15476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:48.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:48 smithi090 ceph-mon[29180]: pgmap v15476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:50.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:50 smithi119 ceph-mon[32752]: pgmap v15477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:50.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:50 smithi167 ceph-mon[32751]: pgmap v15477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:50.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:50 smithi090 ceph-mon[29180]: pgmap v15477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:51.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:51 smithi119 ceph-mon[32752]: 36 slow requests, 0 included below; oldest blocked for > 27932.415160 secs 2024-04-27T02:20:51.533 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:51 smithi090 ceph-mon[29180]: 36 slow requests, 0 included below; oldest blocked for > 27932.415160 secs 2024-04-27T02:20:51.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:51 smithi167 ceph-mon[32751]: 36 slow requests, 0 included below; oldest blocked for > 27932.415160 secs 2024-04-27T02:20:52.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:52 smithi119 ceph-mon[32752]: pgmap v15478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:52.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:52 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27933.247293 secs 2024-04-27T02:20:52.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:52 smithi167 ceph-mon[32751]: pgmap v15478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:52.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:52 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27933.247293 secs 2024-04-27T02:20:52.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:52 smithi090 ceph-mon[29180]: pgmap v15478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:52.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:52 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27933.247293 secs 2024-04-27T02:20:54.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:54 smithi119 ceph-mon[32752]: pgmap v15479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:54.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:54 smithi167 ceph-mon[32751]: pgmap v15479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:54.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:54 smithi090 ceph-mon[29180]: pgmap v15479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:56 smithi119 ceph-mon[32752]: pgmap v15480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:56 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 27937.415217 secs 2024-04-27T02:20:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:56 smithi119 ceph-mon[32752]: slow request 183.185910 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:20:56.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:56 smithi167 ceph-mon[32751]: pgmap v15480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:56.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:56 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 27937.415217 secs 2024-04-27T02:20:56.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:56 smithi167 ceph-mon[32751]: slow request 183.185910 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:20:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:56 smithi090 ceph-mon[29180]: pgmap v15480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:20:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:56 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 27937.415217 secs 2024-04-27T02:20:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:56 smithi090 ceph-mon[29180]: slow request 183.185910 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:20:57.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:57 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27938.247356 secs 2024-04-27T02:20:57.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:57 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27938.247356 secs 2024-04-27T02:20:57.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:57 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27938.247356 secs 2024-04-27T02:20:58.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:58 smithi119 ceph-mon[32752]: pgmap v15481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:58.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:58 smithi167 ceph-mon[32751]: pgmap v15481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:58.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:58 smithi090 ceph-mon[29180]: pgmap v15481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:20:59.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:59 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:20:59.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:59 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:20:59.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:20:59.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:20:59.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:20:59.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:20:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:20:59.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:59 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:20:59.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:59 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:20:59.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:20:59.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:20:59.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:20:59.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:20:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:20:59.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:59 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:20:59.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:59 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:20:59.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:20:59.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:20:59.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:20:59.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:20:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:21:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:00 smithi119 ceph-mon[32752]: pgmap v15482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:00 smithi119 ceph-mon[32752]: pgmap v15483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:21:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:00 smithi119 ceph-mon[32752]: pgmap v15484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:00 smithi167 ceph-mon[32751]: pgmap v15482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:00 smithi167 ceph-mon[32751]: pgmap v15483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:21:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:00 smithi167 ceph-mon[32751]: pgmap v15484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:00 smithi090 ceph-mon[29180]: pgmap v15482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:00 smithi090 ceph-mon[29180]: pgmap v15483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:21:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:00 smithi090 ceph-mon[29180]: pgmap v15484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:01.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27942.415277 secs 2024-04-27T02:21:01.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27942.415277 secs 2024-04-27T02:21:01.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27942.415277 secs 2024-04-27T02:21:02.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:02 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27943.247470 secs 2024-04-27T02:21:02.504 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:02 smithi090 ceph-mon[29180]: pgmap v15485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:02 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27943.247470 secs 2024-04-27T02:21:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:02 smithi119 ceph-mon[32752]: pgmap v15485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:02.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:02 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27943.247470 secs 2024-04-27T02:21:02.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:02 smithi167 ceph-mon[32751]: pgmap v15485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:04.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:04 smithi119 ceph-mon[32752]: pgmap v15486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:21:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:04 smithi167 ceph-mon[32751]: pgmap v15486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:21:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:04 smithi090 ceph-mon[29180]: pgmap v15486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:21:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:06 smithi119 ceph-mon[32752]: pgmap v15487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27947.415355 secs 2024-04-27T02:21:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:06 smithi167 ceph-mon[32751]: pgmap v15487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27947.415355 secs 2024-04-27T02:21:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:06 smithi090 ceph-mon[29180]: pgmap v15487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:21:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27947.415355 secs 2024-04-27T02:21:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:07 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27948.247585 secs 2024-04-27T02:21:07.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:07 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27948.247585 secs 2024-04-27T02:21:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:07 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27948.247585 secs 2024-04-27T02:21:08.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:08 smithi119 ceph-mon[32752]: pgmap v15488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:21:08.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:08 smithi167 ceph-mon[32751]: pgmap v15488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:21:08.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:08 smithi090 ceph-mon[29180]: pgmap v15488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:21:09.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:09 smithi119 ceph-mon[32752]: pgmap v15489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:21:09.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:09 smithi167 ceph-mon[32751]: pgmap v15489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:21:09.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:09 smithi090 ceph-mon[29180]: pgmap v15489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:21:11.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27952.415438 secs 2024-04-27T02:21:11.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27952.415438 secs 2024-04-27T02:21:11.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27952.415438 secs 2024-04-27T02:21:12.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:12 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27953.247699 secs 2024-04-27T02:21:12.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:12 smithi119 ceph-mon[32752]: pgmap v15490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:12.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:12 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27953.247699 secs 2024-04-27T02:21:12.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:12 smithi167 ceph-mon[32751]: pgmap v15490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:12.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:12 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27953.247699 secs 2024-04-27T02:21:12.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:12 smithi090 ceph-mon[29180]: pgmap v15490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:14.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:14 smithi119 ceph-mon[32752]: pgmap v15491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:14.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:14 smithi167 ceph-mon[32751]: pgmap v15491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:14.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:14 smithi090 ceph-mon[29180]: pgmap v15491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:16.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:16 smithi119 ceph-mon[32752]: pgmap v15492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:16.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27957.415497 secs 2024-04-27T02:21:16.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:16 smithi167 ceph-mon[32751]: pgmap v15492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:16.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27957.415497 secs 2024-04-27T02:21:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:16 smithi090 ceph-mon[29180]: pgmap v15492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27957.415497 secs 2024-04-27T02:21:17.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:17 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27958.247814 secs 2024-04-27T02:21:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:17 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27958.247814 secs 2024-04-27T02:21:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:17 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27958.247814 secs 2024-04-27T02:21:18.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:18 smithi119 ceph-mon[32752]: pgmap v15493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:18.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:18 smithi167 ceph-mon[32751]: pgmap v15493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:18.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:18 smithi090 ceph-mon[29180]: pgmap v15493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:20.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:20 smithi119 ceph-mon[32752]: pgmap v15494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:20.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:20 smithi167 ceph-mon[32751]: pgmap v15494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:20.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:20 smithi090 ceph-mon[29180]: pgmap v15494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:21.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27962.415580 secs 2024-04-27T02:21:21.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27962.415580 secs 2024-04-27T02:21:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27962.415580 secs 2024-04-27T02:21:22.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:22 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27963.247928 secs 2024-04-27T02:21:22.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:22 smithi119 ceph-mon[32752]: pgmap v15495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:22.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:22 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27963.247928 secs 2024-04-27T02:21:22.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:22 smithi167 ceph-mon[32751]: pgmap v15495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:22 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27963.247928 secs 2024-04-27T02:21:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:22 smithi090 ceph-mon[29180]: pgmap v15495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:24.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:24 smithi119 ceph-mon[32752]: pgmap v15496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:24.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:24 smithi167 ceph-mon[32751]: pgmap v15496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:24.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:24 smithi090 ceph-mon[29180]: pgmap v15496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:26.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:26 smithi119 ceph-mon[32752]: pgmap v15497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:26.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27967.415663 secs 2024-04-27T02:21:26.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:26 smithi167 ceph-mon[32751]: pgmap v15497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:26.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27967.415663 secs 2024-04-27T02:21:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:26 smithi090 ceph-mon[29180]: pgmap v15497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27967.415663 secs 2024-04-27T02:21:27.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:27 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27968.248047 secs 2024-04-27T02:21:27.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:27 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27968.248047 secs 2024-04-27T02:21:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:27 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27968.248047 secs 2024-04-27T02:21:28.431 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:28 smithi119 ceph-mon[32752]: pgmap v15498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:28.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:28 smithi167 ceph-mon[32751]: pgmap v15498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:28.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:28 smithi090 ceph-mon[29180]: pgmap v15498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:30.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:30 smithi119 ceph-mon[32752]: pgmap v15499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:30.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:30 smithi167 ceph-mon[32751]: pgmap v15499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:30.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:30 smithi090 ceph-mon[29180]: pgmap v15499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:31.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27972.415742 secs 2024-04-27T02:21:31.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27972.415742 secs 2024-04-27T02:21:31.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27972.415742 secs 2024-04-27T02:21:32.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:32 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27973.248153 secs 2024-04-27T02:21:32.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:32 smithi090 ceph-mon[29180]: pgmap v15500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:32.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:32 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27973.248153 secs 2024-04-27T02:21:32.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:32 smithi119 ceph-mon[32752]: pgmap v15500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:32.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:32 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27973.248153 secs 2024-04-27T02:21:32.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:32 smithi167 ceph-mon[32751]: pgmap v15500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:34.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:34 smithi119 ceph-mon[32752]: pgmap v15501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:34.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:34 smithi167 ceph-mon[32751]: pgmap v15501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:34.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:34 smithi090 ceph-mon[29180]: pgmap v15501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:36.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:36 smithi119 ceph-mon[32752]: pgmap v15502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:36.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27977.415800 secs 2024-04-27T02:21:36.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:36 smithi167 ceph-mon[32751]: pgmap v15502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:36.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27977.415800 secs 2024-04-27T02:21:36.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:36 smithi090 ceph-mon[29180]: pgmap v15502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:36.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27977.415800 secs 2024-04-27T02:21:37.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:37 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27978.248267 secs 2024-04-27T02:21:37.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:37 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27978.248267 secs 2024-04-27T02:21:37.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:37 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27978.248267 secs 2024-04-27T02:21:38.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:38 smithi119 ceph-mon[32752]: pgmap v15503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:38.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:38 smithi167 ceph-mon[32751]: pgmap v15503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:38.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:38 smithi090 ceph-mon[29180]: pgmap v15503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:21:40.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:40 smithi119 ceph-mon[32752]: pgmap v15504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:40.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:40 smithi167 ceph-mon[32751]: pgmap v15504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:40.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:40 smithi090 ceph-mon[29180]: pgmap v15504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:41.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27982.415855 secs 2024-04-27T02:21:41.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27982.415855 secs 2024-04-27T02:21:41.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27982.415855 secs 2024-04-27T02:21:42.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:42 smithi119 ceph-mon[32752]: 7 slow requests, 1 included below; oldest blocked for > 27983.248401 secs 2024-04-27T02:21:42.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:42 smithi119 ceph-mon[32752]: slow request 1442.854418 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:21:42.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:42 smithi119 ceph-mon[32752]: pgmap v15505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:42.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:42 smithi167 ceph-mon[32751]: 7 slow requests, 1 included below; oldest blocked for > 27983.248401 secs 2024-04-27T02:21:42.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:42 smithi167 ceph-mon[32751]: slow request 1442.854418 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:21:42.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:42 smithi167 ceph-mon[32751]: pgmap v15505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:42 smithi090 ceph-mon[29180]: 7 slow requests, 1 included below; oldest blocked for > 27983.248401 secs 2024-04-27T02:21:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:42 smithi090 ceph-mon[29180]: slow request 1442.854418 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:21:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:42 smithi090 ceph-mon[29180]: pgmap v15505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:44.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:44 smithi119 ceph-mon[32752]: pgmap v15506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:44.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:44 smithi167 ceph-mon[32751]: pgmap v15506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:44.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:44 smithi090 ceph-mon[29180]: pgmap v15506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:46.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:46 smithi119 ceph-mon[32752]: pgmap v15507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:46.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27987.415935 secs 2024-04-27T02:21:46.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:46 smithi167 ceph-mon[32751]: pgmap v15507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:46.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27987.415935 secs 2024-04-27T02:21:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:46 smithi090 ceph-mon[29180]: pgmap v15507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27987.415935 secs 2024-04-27T02:21:47.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:47 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27988.248518 secs 2024-04-27T02:21:47.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:47 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27988.248518 secs 2024-04-27T02:21:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:47 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27988.248518 secs 2024-04-27T02:21:48.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:48 smithi119 ceph-mon[32752]: pgmap v15508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:48.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:48 smithi167 ceph-mon[32751]: pgmap v15508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:48.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:48 smithi090 ceph-mon[29180]: pgmap v15508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:49.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:49 smithi119 ceph-mon[32752]: pgmap v15509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:49.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:49 smithi167 ceph-mon[32751]: pgmap v15509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:49.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:49 smithi090 ceph-mon[29180]: pgmap v15509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:51.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:51 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 27992.416009 secs 2024-04-27T02:21:51.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:51 smithi119 ceph-mon[32752]: slow request 5761.790097 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:21:51.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:51 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 27992.416009 secs 2024-04-27T02:21:51.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:51 smithi167 ceph-mon[32751]: slow request 5761.790097 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:21:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:51 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 27992.416009 secs 2024-04-27T02:21:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:51 smithi090 ceph-mon[29180]: slow request 5761.790097 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:21:52.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:52 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27993.248624 secs 2024-04-27T02:21:52.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:52 smithi119 ceph-mon[32752]: pgmap v15510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:52.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:52 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27993.248624 secs 2024-04-27T02:21:52.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:52 smithi167 ceph-mon[32751]: pgmap v15510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:52.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:52 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27993.248624 secs 2024-04-27T02:21:52.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:52 smithi090 ceph-mon[29180]: pgmap v15510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:54.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:54 smithi119 ceph-mon[32752]: pgmap v15511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:54.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:54 smithi167 ceph-mon[32751]: pgmap v15511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:54.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:54 smithi090 ceph-mon[29180]: pgmap v15511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:56 smithi119 ceph-mon[32752]: pgmap v15512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 27997.416056 secs 2024-04-27T02:21:56.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:56 smithi167 ceph-mon[32751]: pgmap v15512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:56.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 27997.416056 secs 2024-04-27T02:21:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:56 smithi090 ceph-mon[29180]: pgmap v15512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:21:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 27997.416056 secs 2024-04-27T02:21:57.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:57 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 27998.248708 secs 2024-04-27T02:21:57.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:57 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 27998.248708 secs 2024-04-27T02:21:57.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:57 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 27998.248708 secs 2024-04-27T02:21:58.431 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:21:58 smithi119 ceph-mon[32752]: pgmap v15513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:58.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:21:58 smithi167 ceph-mon[32751]: pgmap v15513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:21:58.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:21:58 smithi090 ceph-mon[29180]: pgmap v15513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:00 smithi119 ceph-mon[32752]: pgmap v15514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:22:00.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:22:00.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:22:00.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:22:00.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:00 smithi167 ceph-mon[32751]: pgmap v15514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:22:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:22:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:22:00.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:22:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:00 smithi090 ceph-mon[29180]: pgmap v15514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:22:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:22:00.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:22:00.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:22:01.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:01 smithi167 ceph-mon[32751]: pgmap v15515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:22:01.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:01 smithi167 ceph-mon[32751]: pgmap v15516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:01.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28002.416137 secs 2024-04-27T02:22:01.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:01 smithi119 ceph-mon[32752]: pgmap v15515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:22:01.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:01 smithi119 ceph-mon[32752]: pgmap v15516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:01.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28002.416137 secs 2024-04-27T02:22:01.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:01 smithi090 ceph-mon[29180]: pgmap v15515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:22:01.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:01 smithi090 ceph-mon[29180]: pgmap v15516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:01.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28002.416137 secs 2024-04-27T02:22:02.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:02 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 28003.248766 secs 2024-04-27T02:22:02.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:02 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 28003.248766 secs 2024-04-27T02:22:02.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:02 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 28003.248766 secs 2024-04-27T02:22:03.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:03 smithi119 ceph-mon[32752]: pgmap v15517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:03 smithi167 ceph-mon[32751]: pgmap v15517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:03.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:03 smithi090 ceph-mon[29180]: pgmap v15517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:05.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:05 smithi119 ceph-mon[32752]: pgmap v15518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:05.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:05 smithi167 ceph-mon[32751]: pgmap v15518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:05.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:05 smithi090 ceph-mon[29180]: pgmap v15518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28007.416197 secs 2024-04-27T02:22:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28007.416197 secs 2024-04-27T02:22:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28007.416197 secs 2024-04-27T02:22:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:07 smithi119 ceph-mon[32752]: pgmap v15519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:07 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 28008.248852 secs 2024-04-27T02:22:07.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:07 smithi167 ceph-mon[32751]: pgmap v15519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:07.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:07 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 28008.248852 secs 2024-04-27T02:22:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:07 smithi090 ceph-mon[29180]: pgmap v15519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:22:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:07 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 28008.248852 secs 2024-04-27T02:22:09.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:09 smithi119 ceph-mon[32752]: pgmap v15520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:22:09.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:09 smithi167 ceph-mon[32751]: pgmap v15520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:22:09.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:09 smithi090 ceph-mon[29180]: pgmap v15520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:22:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:11 smithi119 ceph-mon[32752]: pgmap v15521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:22:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28012.416283 secs 2024-04-27T02:22:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:11 smithi167 ceph-mon[32751]: pgmap v15521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:22:11.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28012.416283 secs 2024-04-27T02:22:11.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:11 smithi090 ceph-mon[29180]: pgmap v15521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:22:11.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28012.416283 secs 2024-04-27T02:22:12.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:12 smithi119 ceph-mon[32752]: 27 slow requests, 1 included below; oldest blocked for > 28013.248958 secs 2024-04-27T02:22:12.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:12 smithi119 ceph-mon[32752]: slow request 182.519542 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:22:12.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:12 smithi167 ceph-mon[32751]: 27 slow requests, 1 included below; oldest blocked for > 28013.248958 secs 2024-04-27T02:22:12.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:12 smithi167 ceph-mon[32751]: slow request 182.519542 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:22:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:12 smithi090 ceph-mon[29180]: 27 slow requests, 1 included below; oldest blocked for > 28013.248958 secs 2024-04-27T02:22:12.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:12 smithi090 ceph-mon[29180]: slow request 182.519542 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:22:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:13 smithi119 ceph-mon[32752]: pgmap v15522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:13 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:22:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:13 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:22:13.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:13 smithi167 ceph-mon[32751]: pgmap v15522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:13.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:13 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:22:13.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:13 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:22:13.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:13 smithi090 ceph-mon[29180]: pgmap v15522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:13.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:13 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:22:13.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:13 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:22:15.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:15 smithi119 ceph-mon[32752]: pgmap v15523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:15.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:15 smithi167 ceph-mon[32751]: pgmap v15523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:15 smithi090 ceph-mon[29180]: pgmap v15523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:16.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28017.416334 secs 2024-04-27T02:22:16.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28017.416334 secs 2024-04-27T02:22:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28017.416334 secs 2024-04-27T02:22:17.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:17 smithi119 ceph-mon[32752]: pgmap v15524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:17.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28018.249077 secs 2024-04-27T02:22:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:17 smithi167 ceph-mon[32751]: pgmap v15524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28018.249077 secs 2024-04-27T02:22:17.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:17 smithi090 ceph-mon[29180]: pgmap v15524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28018.249077 secs 2024-04-27T02:22:19.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:19 smithi119 ceph-mon[32752]: pgmap v15525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:19.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:19 smithi167 ceph-mon[32751]: pgmap v15525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:19.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:19 smithi090 ceph-mon[29180]: pgmap v15525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:21.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:21 smithi119 ceph-mon[32752]: pgmap v15526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:21.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28022.416392 secs 2024-04-27T02:22:21.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:21 smithi167 ceph-mon[32751]: pgmap v15526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:21.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28022.416392 secs 2024-04-27T02:22:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:21 smithi090 ceph-mon[29180]: pgmap v15526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28022.416392 secs 2024-04-27T02:22:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28023.249208 secs 2024-04-27T02:22:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28023.249208 secs 2024-04-27T02:22:22.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28023.249208 secs 2024-04-27T02:22:23.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:23 smithi119 ceph-mon[32752]: pgmap v15527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:23.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:23 smithi167 ceph-mon[32751]: pgmap v15527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:23.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:23 smithi090 ceph-mon[29180]: pgmap v15527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:25.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:25 smithi119 ceph-mon[32752]: pgmap v15528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:25.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:25 smithi167 ceph-mon[32751]: pgmap v15528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:25.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:25 smithi090 ceph-mon[29180]: pgmap v15528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28027.416476 secs 2024-04-27T02:22:26.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28027.416476 secs 2024-04-27T02:22:26.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28027.416476 secs 2024-04-27T02:22:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:27 smithi119 ceph-mon[32752]: pgmap v15529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:27.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28028.249327 secs 2024-04-27T02:22:27.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:27 smithi167 ceph-mon[32751]: pgmap v15529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:27.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28028.249327 secs 2024-04-27T02:22:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:27 smithi090 ceph-mon[29180]: pgmap v15529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28028.249327 secs 2024-04-27T02:22:28.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:28 smithi119 ceph-mon[32752]: pgmap v15530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:28.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:28 smithi167 ceph-mon[32751]: pgmap v15530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:28.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:28 smithi090 ceph-mon[29180]: pgmap v15530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:31.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:30 smithi167 ceph-mon[32751]: pgmap v15531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:31.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:30 smithi119 ceph-mon[32752]: pgmap v15531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:31.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:30 smithi090 ceph-mon[29180]: pgmap v15531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:32.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28032.416565 secs 2024-04-27T02:22:32.253 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28033.249446 secs 2024-04-27T02:22:32.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28032.416565 secs 2024-04-27T02:22:32.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28033.249446 secs 2024-04-27T02:22:32.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28032.416565 secs 2024-04-27T02:22:32.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28033.249446 secs 2024-04-27T02:22:33.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:32 smithi119 ceph-mon[32752]: pgmap v15532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:33.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:32 smithi167 ceph-mon[32751]: pgmap v15532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:33.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:32 smithi090 ceph-mon[29180]: pgmap v15532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:35.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:34 smithi119 ceph-mon[32752]: pgmap v15533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:35.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:34 smithi167 ceph-mon[32751]: pgmap v15533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:35.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:34 smithi090 ceph-mon[29180]: pgmap v15533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:37.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28037.416648 secs 2024-04-27T02:22:37.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:36 smithi119 ceph-mon[32752]: pgmap v15534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:37.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28038.249551 secs 2024-04-27T02:22:37.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28037.416648 secs 2024-04-27T02:22:37.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:36 smithi167 ceph-mon[32751]: pgmap v15534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:37.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28038.249551 secs 2024-04-27T02:22:37.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28037.416648 secs 2024-04-27T02:22:37.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:36 smithi090 ceph-mon[29180]: pgmap v15534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:37.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28038.249551 secs 2024-04-27T02:22:39.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:38 smithi119 ceph-mon[32752]: pgmap v15535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:39.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:38 smithi167 ceph-mon[32751]: pgmap v15535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:39.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:38 smithi090 ceph-mon[29180]: pgmap v15535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:41.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:40 smithi119 ceph-mon[32752]: pgmap v15536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:41.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:40 smithi167 ceph-mon[32751]: pgmap v15536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:41.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:40 smithi090 ceph-mon[29180]: pgmap v15536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:42.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28042.416732 secs 2024-04-27T02:22:42.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28043.249685 secs 2024-04-27T02:22:42.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28042.416732 secs 2024-04-27T02:22:42.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28043.249685 secs 2024-04-27T02:22:42.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28042.416732 secs 2024-04-27T02:22:42.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28043.249685 secs 2024-04-27T02:22:43.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:42 smithi119 ceph-mon[32752]: pgmap v15537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:43.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:42 smithi167 ceph-mon[32751]: pgmap v15537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:43.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:42 smithi090 ceph-mon[29180]: pgmap v15537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:45.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:44 smithi119 ceph-mon[32752]: pgmap v15538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:45.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:44 smithi167 ceph-mon[32751]: pgmap v15538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:45.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:44 smithi090 ceph-mon[29180]: pgmap v15538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:46.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:45 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28047.416800 secs 2024-04-27T02:22:46.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:45 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28047.416800 secs 2024-04-27T02:22:46.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:45 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28047.416800 secs 2024-04-27T02:22:47.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:46 smithi119 ceph-mon[32752]: pgmap v15539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:47.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:46 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28048.249762 secs 2024-04-27T02:22:47.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:46 smithi167 ceph-mon[32751]: pgmap v15539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:47.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:46 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28048.249762 secs 2024-04-27T02:22:47.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:46 smithi090 ceph-mon[29180]: pgmap v15539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:47.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:46 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28048.249762 secs 2024-04-27T02:22:49.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:48 smithi119 ceph-mon[32752]: pgmap v15540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:49.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:48 smithi167 ceph-mon[32751]: pgmap v15540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:49.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:48 smithi090 ceph-mon[29180]: pgmap v15540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:51.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:51 smithi119 ceph-mon[32752]: pgmap v15541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:51.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28052.416884 secs 2024-04-27T02:22:51.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:51 smithi167 ceph-mon[32751]: pgmap v15541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:51.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28052.416884 secs 2024-04-27T02:22:51.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:51 smithi090 ceph-mon[29180]: pgmap v15541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:51.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28052.416884 secs 2024-04-27T02:22:52.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:52 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28053.249823 secs 2024-04-27T02:22:52.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:52 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28053.249823 secs 2024-04-27T02:22:52.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:52 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28053.249823 secs 2024-04-27T02:22:53.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:53 smithi119 ceph-mon[32752]: pgmap v15542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:53.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:53 smithi167 ceph-mon[32751]: pgmap v15542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:53.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:53 smithi090 ceph-mon[29180]: pgmap v15542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:55.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:55 smithi119 ceph-mon[32752]: pgmap v15543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:55.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:55 smithi167 ceph-mon[32751]: pgmap v15543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:55.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:55 smithi090 ceph-mon[29180]: pgmap v15543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:22:56.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28057.416973 secs 2024-04-27T02:22:56.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28057.416973 secs 2024-04-27T02:22:56.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28057.416973 secs 2024-04-27T02:22:57.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:57 smithi119 ceph-mon[32752]: pgmap v15544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:57.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:57 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28058.249944 secs 2024-04-27T02:22:57.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:57 smithi167 ceph-mon[32751]: pgmap v15544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:57.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:57 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28058.249944 secs 2024-04-27T02:22:57.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:57 smithi090 ceph-mon[29180]: pgmap v15544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:57.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:57 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28058.249944 secs 2024-04-27T02:22:59.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:22:59 smithi119 ceph-mon[32752]: pgmap v15545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:59.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:22:59 smithi167 ceph-mon[32751]: pgmap v15545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:22:59.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:22:59 smithi090 ceph-mon[29180]: pgmap v15545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:00.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:23:00.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:23:00.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:23:01.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:01 smithi167 ceph-mon[32751]: pgmap v15546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:01.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28062.417041 secs 2024-04-27T02:23:01.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:01 smithi119 ceph-mon[32752]: pgmap v15546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:01.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28062.417041 secs 2024-04-27T02:23:01.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:01 smithi090 ceph-mon[29180]: pgmap v15546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:01.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28062.417041 secs 2024-04-27T02:23:01.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:01.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:02.011 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:02 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28063.250010 secs 2024-04-27T02:23:02.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:02 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28063.250010 secs 2024-04-27T02:23:02.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:02 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28063.250010 secs 2024-04-27T02:23:03.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:03 smithi090 ceph-mon[29180]: pgmap v15547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:03 smithi119 ceph-mon[32752]: pgmap v15547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:03.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:03 smithi167 ceph-mon[32751]: pgmap v15547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:03.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.532 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:23:04.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:23:04.533 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:23:04.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:23:04.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:23:04.605 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:23:05.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:05 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:23:05.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:05 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:23:05.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:05 smithi119 ceph-mon[32752]: pgmap v15548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:05.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:05 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:23:05.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:05 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:23:05.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:05 smithi119 ceph-mon[32752]: pgmap v15549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 B/s rd, 1 op/s 2024-04-27T02:23:05.531 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:05 smithi119 ceph-mon[32752]: pgmap v15550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:05.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:05 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:23:05.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:05 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:23:05.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:05 smithi167 ceph-mon[32751]: pgmap v15548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:05.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:05 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:23:05.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:05 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:23:05.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:05 smithi167 ceph-mon[32751]: pgmap v15549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 B/s rd, 1 op/s 2024-04-27T02:23:05.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:05 smithi167 ceph-mon[32751]: pgmap v15550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:05.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:05 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:23:05.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:05 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:23:05.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:05 smithi090 ceph-mon[29180]: pgmap v15548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:05.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:05 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:23:05.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:05 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:23:05.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:05 smithi090 ceph-mon[29180]: pgmap v15549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1012 B/s rd, 1 op/s 2024-04-27T02:23:05.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:05 smithi090 ceph-mon[29180]: pgmap v15550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:06.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28067.417123 secs 2024-04-27T02:23:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28067.417123 secs 2024-04-27T02:23:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28067.417123 secs 2024-04-27T02:23:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:07 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28068.250088 secs 2024-04-27T02:23:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:07 smithi119 ceph-mon[32752]: pgmap v15551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:07 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28068.250088 secs 2024-04-27T02:23:07.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:07 smithi167 ceph-mon[32751]: pgmap v15551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:07 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28068.250088 secs 2024-04-27T02:23:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:07 smithi090 ceph-mon[29180]: pgmap v15551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:09.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:09 smithi119 ceph-mon[32752]: pgmap v15552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 B/s rd, 1 op/s 2024-04-27T02:23:09.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:09 smithi167 ceph-mon[32751]: pgmap v15552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 B/s rd, 1 op/s 2024-04-27T02:23:09.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:09 smithi090 ceph-mon[29180]: pgmap v15552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 B/s rd, 1 op/s 2024-04-27T02:23:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:11 smithi119 ceph-mon[32752]: pgmap v15553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28072.417207 secs 2024-04-27T02:23:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:11 smithi167 ceph-mon[32751]: pgmap v15553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:11.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28072.417207 secs 2024-04-27T02:23:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:11 smithi090 ceph-mon[29180]: pgmap v15553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:11.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28072.417207 secs 2024-04-27T02:23:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:12 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28073.250165 secs 2024-04-27T02:23:12.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:12 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28073.250165 secs 2024-04-27T02:23:12.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:12 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28073.250165 secs 2024-04-27T02:23:13.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:13 smithi119 ceph-mon[32752]: pgmap v15554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 B/s rd, 1 op/s 2024-04-27T02:23:13.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:13 smithi167 ceph-mon[32751]: pgmap v15554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 B/s rd, 1 op/s 2024-04-27T02:23:13.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:13 smithi090 ceph-mon[29180]: pgmap v15554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 B/s rd, 1 op/s 2024-04-27T02:23:15.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:15 smithi119 ceph-mon[32752]: pgmap v15555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:23:15.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:15 smithi167 ceph-mon[32751]: pgmap v15555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:23:15.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:15 smithi090 ceph-mon[29180]: pgmap v15555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:23:16.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28077.417286 secs 2024-04-27T02:23:16.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28077.417286 secs 2024-04-27T02:23:16.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28077.417286 secs 2024-04-27T02:23:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28078.250300 secs 2024-04-27T02:23:17.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:17 smithi119 ceph-mon[32752]: pgmap v15556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28078.250300 secs 2024-04-27T02:23:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:17 smithi167 ceph-mon[32751]: pgmap v15556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28078.250300 secs 2024-04-27T02:23:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:17 smithi090 ceph-mon[29180]: pgmap v15556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:19.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:19 smithi119 ceph-mon[32752]: pgmap v15557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:19.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:19 smithi167 ceph-mon[32751]: pgmap v15557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:19.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:19 smithi090 ceph-mon[29180]: pgmap v15557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:21 smithi119 ceph-mon[32752]: pgmap v15558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:21.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28082.417388 secs 2024-04-27T02:23:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:21 smithi167 ceph-mon[32751]: pgmap v15558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:21.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28082.417388 secs 2024-04-27T02:23:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:21 smithi090 ceph-mon[29180]: pgmap v15558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:21.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28082.417388 secs 2024-04-27T02:23:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28083.250417 secs 2024-04-27T02:23:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28083.250417 secs 2024-04-27T02:23:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28083.250417 secs 2024-04-27T02:23:23.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:23 smithi119 ceph-mon[32752]: pgmap v15559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:23.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:23 smithi167 ceph-mon[32751]: pgmap v15559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:23.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:23 smithi090 ceph-mon[29180]: pgmap v15559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:25.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:25 smithi119 ceph-mon[32752]: pgmap v15560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:25.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:25 smithi167 ceph-mon[32751]: pgmap v15560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:25.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:25 smithi090 ceph-mon[29180]: pgmap v15560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28087.417447 secs 2024-04-27T02:23:26.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28087.417447 secs 2024-04-27T02:23:26.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28087.417447 secs 2024-04-27T02:23:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28088.250539 secs 2024-04-27T02:23:27.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:27 smithi119 ceph-mon[32752]: pgmap v15561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28088.250539 secs 2024-04-27T02:23:27.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:27 smithi167 ceph-mon[32751]: pgmap v15561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28088.250539 secs 2024-04-27T02:23:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:27 smithi090 ceph-mon[29180]: pgmap v15561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:28.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:28 smithi119 ceph-mon[32752]: pgmap v15562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:28.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:28 smithi167 ceph-mon[32751]: pgmap v15562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:28.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:28 smithi090 ceph-mon[29180]: pgmap v15562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:31.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:31 smithi090 ceph-mon[29180]: pgmap v15563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:31.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28092.417550 secs 2024-04-27T02:23:31.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:31 smithi167 ceph-mon[32751]: pgmap v15563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:31.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28092.417550 secs 2024-04-27T02:23:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:31 smithi119 ceph-mon[32752]: pgmap v15563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:31.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28092.417550 secs 2024-04-27T02:23:32.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:32 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28093.250691 secs 2024-04-27T02:23:32.448 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:32 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28093.250691 secs 2024-04-27T02:23:32.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:32 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28093.250691 secs 2024-04-27T02:23:33.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:33 smithi090 ceph-mon[29180]: pgmap v15564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:33.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:33 smithi119 ceph-mon[32752]: pgmap v15564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:33.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:33 smithi167 ceph-mon[32751]: pgmap v15564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:35.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:35 smithi090 ceph-mon[29180]: pgmap v15565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:35.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:35 smithi119 ceph-mon[32752]: pgmap v15565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:35.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:35 smithi167 ceph-mon[32751]: pgmap v15565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:36.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28097.417636 secs 2024-04-27T02:23:36.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28097.417636 secs 2024-04-27T02:23:36.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28097.417636 secs 2024-04-27T02:23:37.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:37 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28098.250760 secs 2024-04-27T02:23:37.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:37 smithi090 ceph-mon[29180]: pgmap v15566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:37 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28098.250760 secs 2024-04-27T02:23:37.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:37 smithi119 ceph-mon[32752]: pgmap v15566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:37 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28098.250760 secs 2024-04-27T02:23:37.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:37 smithi167 ceph-mon[32751]: pgmap v15566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:39.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:39 smithi090 ceph-mon[29180]: pgmap v15567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:39.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:39 smithi119 ceph-mon[32752]: pgmap v15567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:39.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:39 smithi167 ceph-mon[32751]: pgmap v15567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:41.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:41 smithi090 ceph-mon[29180]: pgmap v15568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:41.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28102.417724 secs 2024-04-27T02:23:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:41 smithi119 ceph-mon[32752]: pgmap v15568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:41.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28102.417724 secs 2024-04-27T02:23:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:41 smithi167 ceph-mon[32751]: pgmap v15568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:41.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28102.417724 secs 2024-04-27T02:23:42.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:42 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28103.250822 secs 2024-04-27T02:23:42.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:42 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28103.250822 secs 2024-04-27T02:23:42.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:42 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28103.250822 secs 2024-04-27T02:23:43.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:43 smithi090 ceph-mon[29180]: pgmap v15569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:43.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:43 smithi119 ceph-mon[32752]: pgmap v15569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:43.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:43 smithi167 ceph-mon[32751]: pgmap v15569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:45.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:45 smithi090 ceph-mon[29180]: pgmap v15570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:45.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:45 smithi119 ceph-mon[32752]: pgmap v15570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:45.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:45 smithi167 ceph-mon[32751]: pgmap v15570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:46.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28107.417824 secs 2024-04-27T02:23:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28107.417824 secs 2024-04-27T02:23:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28107.417824 secs 2024-04-27T02:23:47.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:47 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28108.250962 secs 2024-04-27T02:23:47.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:47 smithi119 ceph-mon[32752]: pgmap v15571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:47 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28108.250962 secs 2024-04-27T02:23:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:47 smithi167 ceph-mon[32751]: pgmap v15571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:47 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28108.250962 secs 2024-04-27T02:23:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:47 smithi090 ceph-mon[29180]: pgmap v15571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:49.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:49 smithi119 ceph-mon[32752]: pgmap v15572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:49.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:49 smithi167 ceph-mon[32751]: pgmap v15572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:49.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:49 smithi090 ceph-mon[29180]: pgmap v15572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:51.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:51 smithi119 ceph-mon[32752]: pgmap v15573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:51.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28112.417912 secs 2024-04-27T02:23:51.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:51 smithi167 ceph-mon[32751]: pgmap v15573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:51.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28112.417912 secs 2024-04-27T02:23:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:51 smithi090 ceph-mon[29180]: pgmap v15573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28112.417912 secs 2024-04-27T02:23:52.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:52 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28113.251073 secs 2024-04-27T02:23:52.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:52 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28113.251073 secs 2024-04-27T02:23:52.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:52 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28113.251073 secs 2024-04-27T02:23:53.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:53 smithi119 ceph-mon[32752]: pgmap v15574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:53.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:53 smithi167 ceph-mon[32751]: pgmap v15574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:53.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:53 smithi090 ceph-mon[29180]: pgmap v15574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:55.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:55 smithi119 ceph-mon[32752]: pgmap v15575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:55.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:55 smithi167 ceph-mon[32751]: pgmap v15575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:55.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:55 smithi090 ceph-mon[29180]: pgmap v15575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:23:56.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:56 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 28117.418001 secs 2024-04-27T02:23:56.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:56 smithi119 ceph-mon[32752]: slow request 363.188694 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:23:56.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:56 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 28117.418001 secs 2024-04-27T02:23:56.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:56 smithi167 ceph-mon[32751]: slow request 363.188694 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:23:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:56 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 28117.418001 secs 2024-04-27T02:23:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:56 smithi090 ceph-mon[29180]: slow request 363.188694 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:23:57.389 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:57 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28118.251191 secs 2024-04-27T02:23:57.389 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:57 smithi090 ceph-mon[29180]: pgmap v15576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:57.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:57 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28118.251191 secs 2024-04-27T02:23:57.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:57 smithi119 ceph-mon[32752]: pgmap v15576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:57.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:57 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28118.251191 secs 2024-04-27T02:23:57.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:57 smithi167 ceph-mon[32751]: pgmap v15576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:59.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:23:59 smithi119 ceph-mon[32752]: pgmap v15577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:59.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:23:59 smithi167 ceph-mon[32751]: pgmap v15577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:23:59.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:23:59 smithi090 ceph-mon[29180]: pgmap v15577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:01.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:01 smithi119 ceph-mon[32752]: pgmap v15578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:01.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28122.418109 secs 2024-04-27T02:24:01.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:01 smithi167 ceph-mon[32751]: pgmap v15578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:01.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28122.418109 secs 2024-04-27T02:24:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:01 smithi090 ceph-mon[29180]: pgmap v15578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28122.418109 secs 2024-04-27T02:24:02.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:02 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28123.251303 secs 2024-04-27T02:24:02.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:02 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28123.251303 secs 2024-04-27T02:24:02.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:02 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28123.251303 secs 2024-04-27T02:24:03.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:03 smithi119 ceph-mon[32752]: pgmap v15579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:03.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:03 smithi167 ceph-mon[32751]: pgmap v15579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:03.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:03 smithi090 ceph-mon[29180]: pgmap v15579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:04.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:24:04.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:24:04.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:24:05.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:05 smithi119 ceph-mon[32752]: pgmap v15580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:05.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:05.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:05.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:05 smithi167 ceph-mon[32751]: pgmap v15580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:05.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:05.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:05.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:05 smithi090 ceph-mon[29180]: pgmap v15580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:05.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:05.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:06.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28127.418190 secs 2024-04-27T02:24:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:24:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:24:06.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28127.418190 secs 2024-04-27T02:24:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:24:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:24:06.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28127.418190 secs 2024-04-27T02:24:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:24:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:24:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:24:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:07 smithi119 ceph-mon[32752]: pgmap v15581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T02:24:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:07 smithi119 ceph-mon[32752]: pgmap v15582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:07 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28128.251400 secs 2024-04-27T02:24:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:07 smithi167 ceph-mon[32751]: pgmap v15581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T02:24:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:07 smithi167 ceph-mon[32751]: pgmap v15582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:07 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28128.251400 secs 2024-04-27T02:24:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:07 smithi090 ceph-mon[29180]: pgmap v15581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T02:24:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:07 smithi090 ceph-mon[29180]: pgmap v15582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:07 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28128.251400 secs 2024-04-27T02:24:08.851 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:24:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:24:08.527+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:24:08.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:24:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:24:08.527+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:24:08.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:24:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:24:08.527+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:24:08.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:24:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:24:08.527+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:24:08.852 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:24:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:24:08.527+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:24:09.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:09 smithi119 ceph-mon[32752]: pgmap v15583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:09.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:09 smithi167 ceph-mon[32751]: pgmap v15583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:09.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:09 smithi090 ceph-mon[29180]: pgmap v15583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:11 smithi119 ceph-mon[32752]: pgmap v15584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28132.418273 secs 2024-04-27T02:24:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:11 smithi167 ceph-mon[32751]: pgmap v15584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28132.418273 secs 2024-04-27T02:24:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:11 smithi090 ceph-mon[29180]: pgmap v15584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:11.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28132.418273 secs 2024-04-27T02:24:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:12 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28133.251508 secs 2024-04-27T02:24:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:12 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28133.251508 secs 2024-04-27T02:24:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:12 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28133.251508 secs 2024-04-27T02:24:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:13 smithi119 ceph-mon[32752]: pgmap v15585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:13.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:13 smithi167 ceph-mon[32751]: pgmap v15585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:13.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:13 smithi090 ceph-mon[29180]: pgmap v15585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:24:15.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:15 smithi119 ceph-mon[32752]: pgmap v15586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 B/s rd, 110 B/s wr, 1 op/s 2024-04-27T02:24:15.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:15 smithi167 ceph-mon[32751]: pgmap v15586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 B/s rd, 110 B/s wr, 1 op/s 2024-04-27T02:24:15.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:15 smithi090 ceph-mon[29180]: pgmap v15586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 B/s rd, 110 B/s wr, 1 op/s 2024-04-27T02:24:16.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28137.418357 secs 2024-04-27T02:24:16.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28137.418357 secs 2024-04-27T02:24:16.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28137.418357 secs 2024-04-27T02:24:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:17 smithi119 ceph-mon[32752]: pgmap v15587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T02:24:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28138.251625 secs 2024-04-27T02:24:17.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:17 smithi167 ceph-mon[32751]: pgmap v15587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T02:24:17.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28138.251625 secs 2024-04-27T02:24:17.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:17 smithi090 ceph-mon[29180]: pgmap v15587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 409 B/s wr, 2 op/s 2024-04-27T02:24:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28138.251625 secs 2024-04-27T02:24:19.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:19 smithi119 ceph-mon[32752]: pgmap v15588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:19.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:19 smithi167 ceph-mon[32751]: pgmap v15588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:19.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:19 smithi090 ceph-mon[29180]: pgmap v15588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:21 smithi119 ceph-mon[32752]: pgmap v15589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28142.418442 secs 2024-04-27T02:24:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:21 smithi167 ceph-mon[32751]: pgmap v15589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28142.418442 secs 2024-04-27T02:24:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:21 smithi090 ceph-mon[29180]: pgmap v15589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28142.418442 secs 2024-04-27T02:24:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28143.251694 secs 2024-04-27T02:24:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28143.251694 secs 2024-04-27T02:24:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28143.251694 secs 2024-04-27T02:24:23.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:23 smithi119 ceph-mon[32752]: pgmap v15590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:23.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:23 smithi167 ceph-mon[32751]: pgmap v15590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:23.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:23 smithi090 ceph-mon[29180]: pgmap v15590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:25.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:25 smithi119 ceph-mon[32752]: pgmap v15591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:25.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:25 smithi167 ceph-mon[32751]: pgmap v15591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:25.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:25 smithi090 ceph-mon[29180]: pgmap v15591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:24:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28147.418523 secs 2024-04-27T02:24:26.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28147.418523 secs 2024-04-27T02:24:26.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28147.418523 secs 2024-04-27T02:24:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:27 smithi119 ceph-mon[32752]: pgmap v15592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:24:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28148.251814 secs 2024-04-27T02:24:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:27 smithi167 ceph-mon[32751]: pgmap v15592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:24:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28148.251814 secs 2024-04-27T02:24:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:27 smithi090 ceph-mon[29180]: pgmap v15592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:24:27.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28148.251814 secs 2024-04-27T02:24:29.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:29 smithi119 ceph-mon[32752]: pgmap v15593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:29.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:29 smithi167 ceph-mon[32751]: pgmap v15593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:29.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:29 smithi090 ceph-mon[29180]: pgmap v15593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:31 smithi119 ceph-mon[32752]: pgmap v15594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28152.418603 secs 2024-04-27T02:24:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:31 smithi167 ceph-mon[32751]: pgmap v15594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28152.418603 secs 2024-04-27T02:24:31.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:31 smithi090 ceph-mon[29180]: pgmap v15594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:31.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28152.418603 secs 2024-04-27T02:24:32.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:32 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28153.251927 secs 2024-04-27T02:24:32.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:32 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28153.251927 secs 2024-04-27T02:24:32.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:32 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28153.251927 secs 2024-04-27T02:24:33.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:33 smithi119 ceph-mon[32752]: pgmap v15595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:33.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:33 smithi167 ceph-mon[32751]: pgmap v15595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:33.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:33 smithi090 ceph-mon[29180]: pgmap v15595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:35.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:35 smithi119 ceph-mon[32752]: pgmap v15596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:35.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:35 smithi167 ceph-mon[32751]: pgmap v15596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:35.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:35 smithi090 ceph-mon[29180]: pgmap v15596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:36.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28157.418677 secs 2024-04-27T02:24:36.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28157.418677 secs 2024-04-27T02:24:36.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28157.418677 secs 2024-04-27T02:24:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:37 smithi119 ceph-mon[32752]: pgmap v15597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:37.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:37 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28158.252049 secs 2024-04-27T02:24:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:37 smithi167 ceph-mon[32751]: pgmap v15597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:37 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28158.252049 secs 2024-04-27T02:24:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:37 smithi090 ceph-mon[29180]: pgmap v15597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:37.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:37 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28158.252049 secs 2024-04-27T02:24:39.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:39 smithi119 ceph-mon[32752]: pgmap v15598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:39.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:39 smithi167 ceph-mon[32751]: pgmap v15598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:39.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:39 smithi090 ceph-mon[29180]: pgmap v15598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:41 smithi119 ceph-mon[32752]: pgmap v15599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28162.418765 secs 2024-04-27T02:24:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:41 smithi167 ceph-mon[32751]: pgmap v15599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28162.418765 secs 2024-04-27T02:24:41.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:41 smithi090 ceph-mon[29180]: pgmap v15599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:41.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28162.418765 secs 2024-04-27T02:24:42.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:42 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28163.252170 secs 2024-04-27T02:24:42.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:42 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28163.252170 secs 2024-04-27T02:24:42.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:42 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28163.252170 secs 2024-04-27T02:24:43.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:43 smithi119 ceph-mon[32752]: pgmap v15600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:43.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:43 smithi167 ceph-mon[32751]: pgmap v15600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:43.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:43 smithi090 ceph-mon[29180]: pgmap v15600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:45.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:45 smithi119 ceph-mon[32752]: pgmap v15601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:45.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:45 smithi167 ceph-mon[32751]: pgmap v15601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:45.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:45 smithi090 ceph-mon[29180]: pgmap v15601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28167.418855 secs 2024-04-27T02:24:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28167.418855 secs 2024-04-27T02:24:46.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28167.418855 secs 2024-04-27T02:24:47.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:47 smithi119 ceph-mon[32752]: pgmap v15602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:47.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:47 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28168.252297 secs 2024-04-27T02:24:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:47 smithi167 ceph-mon[32751]: pgmap v15602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:47 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28168.252297 secs 2024-04-27T02:24:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:47 smithi090 ceph-mon[29180]: pgmap v15602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:47 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28168.252297 secs 2024-04-27T02:24:49.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:49 smithi119 ceph-mon[32752]: pgmap v15603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:49.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:49 smithi167 ceph-mon[32751]: pgmap v15603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:49.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:49 smithi090 ceph-mon[29180]: pgmap v15603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:50.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:50 smithi090 ceph-mon[29180]: pgmap v15604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:50.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:50 smithi119 ceph-mon[32752]: pgmap v15604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:50.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:50 smithi167 ceph-mon[32751]: pgmap v15604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:51.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28172.418942 secs 2024-04-27T02:24:51.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28172.418942 secs 2024-04-27T02:24:51.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28172.418942 secs 2024-04-27T02:24:52.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:52 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28173.252362 secs 2024-04-27T02:24:52.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:52 smithi090 ceph-mon[29180]: pgmap v15605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:52.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:52 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28173.252362 secs 2024-04-27T02:24:52.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:52 smithi119 ceph-mon[32752]: pgmap v15605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:52.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:52 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28173.252362 secs 2024-04-27T02:24:52.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:52 smithi167 ceph-mon[32751]: pgmap v15605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:54.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:54 smithi090 ceph-mon[29180]: pgmap v15606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:54.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:54 smithi119 ceph-mon[32752]: pgmap v15606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:54.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:54 smithi167 ceph-mon[32751]: pgmap v15606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T02:24:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28177.419026 secs 2024-04-27T02:24:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:56 smithi090 ceph-mon[29180]: pgmap v15607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:56.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28177.419026 secs 2024-04-27T02:24:56.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:56 smithi119 ceph-mon[32752]: pgmap v15607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:56.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28177.419026 secs 2024-04-27T02:24:56.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:56 smithi167 ceph-mon[32751]: pgmap v15607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:24:57.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:57 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28178.252485 secs 2024-04-27T02:24:57.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:57 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28178.252485 secs 2024-04-27T02:24:57.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:57 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28178.252485 secs 2024-04-27T02:24:58.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:24:58 smithi090 ceph-mon[29180]: pgmap v15608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:58.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:24:58 smithi119 ceph-mon[32752]: pgmap v15608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:24:58.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:24:58 smithi167 ceph-mon[32751]: pgmap v15608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:00.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:00 smithi090 ceph-mon[29180]: pgmap v15609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:00.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:00 smithi119 ceph-mon[32752]: pgmap v15609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:00.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:00 smithi167 ceph-mon[32751]: pgmap v15609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28182.419104 secs 2024-04-27T02:25:01.680 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28182.419104 secs 2024-04-27T02:25:01.695 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28182.419104 secs 2024-04-27T02:25:02.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:02 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28183.252599 secs 2024-04-27T02:25:02.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:02 smithi090 ceph-mon[29180]: pgmap v15610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:02.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:02 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28183.252599 secs 2024-04-27T02:25:02.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:02 smithi119 ceph-mon[32752]: pgmap v15610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:02.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:02 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28183.252599 secs 2024-04-27T02:25:02.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:02 smithi167 ceph-mon[32751]: pgmap v15610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:04.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:04 smithi090 ceph-mon[29180]: pgmap v15611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:04.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:04 smithi119 ceph-mon[32752]: pgmap v15611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:04.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:04 smithi167 ceph-mon[32751]: pgmap v15611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28187.419192 secs 2024-04-27T02:25:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: pgmap v15612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:25:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:25:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:25:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: pgmap v15613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:25:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: pgmap v15614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:25:06.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28187.419192 secs 2024-04-27T02:25:06.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: pgmap v15612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:06.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:25:06.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:25:06.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:25:06.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: pgmap v15613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:25:06.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: pgmap v15614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:06.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:25:06.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28187.419192 secs 2024-04-27T02:25:06.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: pgmap v15612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:06.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:25:06.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:25:06.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:25:06.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: pgmap v15613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:25:06.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: pgmap v15614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:06.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:25:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:07 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28188.252707 secs 2024-04-27T02:25:07.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:07 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28188.252707 secs 2024-04-27T02:25:07.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:07 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28188.252707 secs 2024-04-27T02:25:08.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:08 smithi119 ceph-mon[32752]: pgmap v15615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:25:08.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:08 smithi167 ceph-mon[32751]: pgmap v15615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:25:08.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:08 smithi090 ceph-mon[29180]: pgmap v15615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:25:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:11 smithi119 ceph-mon[32752]: pgmap v15616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:11 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 28192.419254 secs 2024-04-27T02:25:11.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:11 smithi119 ceph-mon[32752]: slow request 11524.857882 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:25:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:11 smithi167 ceph-mon[32751]: pgmap v15616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:11 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 28192.419254 secs 2024-04-27T02:25:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:11 smithi167 ceph-mon[32751]: slow request 11524.857882 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:25:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:11 smithi090 ceph-mon[29180]: pgmap v15616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:11.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:11 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 28192.419254 secs 2024-04-27T02:25:11.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:11 smithi090 ceph-mon[29180]: slow request 11524.857882 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:25:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:12 smithi119 ceph-mon[32752]: 27 slow requests, 1 included below; oldest blocked for > 28193.252800 secs 2024-04-27T02:25:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:12 smithi119 ceph-mon[32752]: slow request 362.523384 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:25:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:12 smithi167 ceph-mon[32751]: 27 slow requests, 1 included below; oldest blocked for > 28193.252800 secs 2024-04-27T02:25:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:12 smithi167 ceph-mon[32751]: slow request 362.523384 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:25:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:12 smithi090 ceph-mon[29180]: 27 slow requests, 1 included below; oldest blocked for > 28193.252800 secs 2024-04-27T02:25:12.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:12 smithi090 ceph-mon[29180]: slow request 362.523384 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:25:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:13 smithi119 ceph-mon[32752]: pgmap v15617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:25:13.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:13 smithi167 ceph-mon[32751]: pgmap v15617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:25:13.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:13 smithi090 ceph-mon[29180]: pgmap v15617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:25:15.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:15 smithi119 ceph-mon[32752]: pgmap v15618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:15.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:15 smithi167 ceph-mon[32751]: pgmap v15618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:15.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:15 smithi090 ceph-mon[29180]: pgmap v15618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:25:16.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28197.419341 secs 2024-04-27T02:25:16.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28197.419341 secs 2024-04-27T02:25:16.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28197.419341 secs 2024-04-27T02:25:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28198.252873 secs 2024-04-27T02:25:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:17 smithi119 ceph-mon[32752]: pgmap v15619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:25:17.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28198.252873 secs 2024-04-27T02:25:17.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:17 smithi167 ceph-mon[32751]: pgmap v15619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:25:17.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28198.252873 secs 2024-04-27T02:25:17.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:17 smithi090 ceph-mon[29180]: pgmap v15619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:25:19.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:19 smithi119 ceph-mon[32752]: pgmap v15620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:19.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:19 smithi167 ceph-mon[32751]: pgmap v15620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:19.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:19 smithi090 ceph-mon[29180]: pgmap v15620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:21 smithi119 ceph-mon[32752]: pgmap v15621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28202.419404 secs 2024-04-27T02:25:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:21 smithi167 ceph-mon[32751]: pgmap v15621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28202.419404 secs 2024-04-27T02:25:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:21 smithi090 ceph-mon[29180]: pgmap v15621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28202.419404 secs 2024-04-27T02:25:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28203.253005 secs 2024-04-27T02:25:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28203.253005 secs 2024-04-27T02:25:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28203.253005 secs 2024-04-27T02:25:23.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:23 smithi119 ceph-mon[32752]: pgmap v15622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:23.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:23 smithi167 ceph-mon[32751]: pgmap v15622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:23.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:23 smithi090 ceph-mon[29180]: pgmap v15622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:25.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:25 smithi090 ceph-mon[29180]: pgmap v15623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:25.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:25 smithi119 ceph-mon[32752]: pgmap v15623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:25.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:25 smithi167 ceph-mon[32751]: pgmap v15623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:26.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28207.419495 secs 2024-04-27T02:25:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28207.419495 secs 2024-04-27T02:25:26.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28207.419495 secs 2024-04-27T02:25:27.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28208.253144 secs 2024-04-27T02:25:27.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:27 smithi090 ceph-mon[29180]: pgmap v15624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28208.253144 secs 2024-04-27T02:25:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:27 smithi119 ceph-mon[32752]: pgmap v15624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28208.253144 secs 2024-04-27T02:25:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:27 smithi167 ceph-mon[32751]: pgmap v15624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:29.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:29 smithi119 ceph-mon[32752]: pgmap v15625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:29.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:29 smithi167 ceph-mon[32751]: pgmap v15625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:29.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:29 smithi090 ceph-mon[29180]: pgmap v15625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:31 smithi119 ceph-mon[32752]: pgmap v15626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28212.419580 secs 2024-04-27T02:25:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:31 smithi167 ceph-mon[32751]: pgmap v15626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28212.419580 secs 2024-04-27T02:25:31.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:31 smithi090 ceph-mon[29180]: pgmap v15626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:31.597 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28212.419580 secs 2024-04-27T02:25:32.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:32 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28213.253260 secs 2024-04-27T02:25:32.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:32 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28213.253260 secs 2024-04-27T02:25:32.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:32 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28213.253260 secs 2024-04-27T02:25:33.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:33 smithi119 ceph-mon[32752]: pgmap v15627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:33.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:33 smithi167 ceph-mon[32751]: pgmap v15627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:33.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:33 smithi090 ceph-mon[29180]: pgmap v15627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:35.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:35 smithi119 ceph-mon[32752]: pgmap v15628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:35.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:35 smithi167 ceph-mon[32751]: pgmap v15628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:35.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:35 smithi090 ceph-mon[29180]: pgmap v15628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:36.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28217.419670 secs 2024-04-27T02:25:36.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28217.419670 secs 2024-04-27T02:25:36.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28217.419670 secs 2024-04-27T02:25:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:37 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28218.253373 secs 2024-04-27T02:25:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:37 smithi119 ceph-mon[32752]: pgmap v15629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:37 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28218.253373 secs 2024-04-27T02:25:37.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:37 smithi167 ceph-mon[32751]: pgmap v15629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:37 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28218.253373 secs 2024-04-27T02:25:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:37 smithi090 ceph-mon[29180]: pgmap v15629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:39.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:39 smithi119 ceph-mon[32752]: pgmap v15630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:39.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:39 smithi167 ceph-mon[32751]: pgmap v15630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:39.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:39 smithi090 ceph-mon[29180]: pgmap v15630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:41 smithi119 ceph-mon[32752]: pgmap v15631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28222.419727 secs 2024-04-27T02:25:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:41 smithi167 ceph-mon[32751]: pgmap v15631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28222.419727 secs 2024-04-27T02:25:41.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:41 smithi090 ceph-mon[29180]: pgmap v15631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:41.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28222.419727 secs 2024-04-27T02:25:42.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:42 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28223.253487 secs 2024-04-27T02:25:42.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:42 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28223.253487 secs 2024-04-27T02:25:42.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:42 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28223.253487 secs 2024-04-27T02:25:43.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:43 smithi119 ceph-mon[32752]: pgmap v15632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:43.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:43 smithi167 ceph-mon[32751]: pgmap v15632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:43.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:43 smithi090 ceph-mon[29180]: pgmap v15632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:45.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:45 smithi119 ceph-mon[32752]: pgmap v15633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:45.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:45 smithi167 ceph-mon[32751]: pgmap v15633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:45.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:45 smithi090 ceph-mon[29180]: pgmap v15633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:46.444 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:46 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 28227.419822 secs 2024-04-27T02:25:46.445 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:46 smithi090 ceph-mon[29180]: slow request 11520.189154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:25:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:46 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 28227.419822 secs 2024-04-27T02:25:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:46 smithi119 ceph-mon[32752]: slow request 11520.189154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:25:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:46 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 28227.419822 secs 2024-04-27T02:25:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:46 smithi167 ceph-mon[32751]: slow request 11520.189154 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:25:47.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:47 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28228.253607 secs 2024-04-27T02:25:47.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:47 smithi119 ceph-mon[32752]: pgmap v15634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:47 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28228.253607 secs 2024-04-27T02:25:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:47 smithi167 ceph-mon[32751]: pgmap v15634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:47 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28228.253607 secs 2024-04-27T02:25:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:47 smithi090 ceph-mon[29180]: pgmap v15634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:49.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:49 smithi119 ceph-mon[32752]: pgmap v15635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:49.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:49 smithi167 ceph-mon[32751]: pgmap v15635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:49.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:49 smithi090 ceph-mon[29180]: pgmap v15635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:51.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:51 smithi119 ceph-mon[32752]: pgmap v15636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:51.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28232.419883 secs 2024-04-27T02:25:51.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:51 smithi167 ceph-mon[32751]: pgmap v15636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:51.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28232.419883 secs 2024-04-27T02:25:51.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:51 smithi090 ceph-mon[29180]: pgmap v15636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:51.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28232.419883 secs 2024-04-27T02:25:52.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:52 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28233.253716 secs 2024-04-27T02:25:52.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:52 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28233.253716 secs 2024-04-27T02:25:52.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:52 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28233.253716 secs 2024-04-27T02:25:53.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:53 smithi119 ceph-mon[32752]: pgmap v15637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:53.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:53 smithi167 ceph-mon[32751]: pgmap v15637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:53.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:53 smithi090 ceph-mon[29180]: pgmap v15637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:55.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:55 smithi119 ceph-mon[32752]: pgmap v15638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:55.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:55 smithi167 ceph-mon[32751]: pgmap v15638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:55.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:55 smithi090 ceph-mon[29180]: pgmap v15638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:25:56.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28237.419964 secs 2024-04-27T02:25:56.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28237.419964 secs 2024-04-27T02:25:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28237.419964 secs 2024-04-27T02:25:57.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:57 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28238.253831 secs 2024-04-27T02:25:57.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:57 smithi119 ceph-mon[32752]: pgmap v15639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:57.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:57 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28238.253831 secs 2024-04-27T02:25:57.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:57 smithi167 ceph-mon[32751]: pgmap v15639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:57.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:57 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28238.253831 secs 2024-04-27T02:25:57.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:57 smithi090 ceph-mon[29180]: pgmap v15639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:59.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:25:59 smithi119 ceph-mon[32752]: pgmap v15640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:59.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:25:59 smithi167 ceph-mon[32751]: pgmap v15640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:25:59.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:25:59 smithi090 ceph-mon[29180]: pgmap v15640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:01.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:01 smithi119 ceph-mon[32752]: pgmap v15641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:01.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28242.420022 secs 2024-04-27T02:26:01.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:01 smithi167 ceph-mon[32751]: pgmap v15641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:01.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28242.420022 secs 2024-04-27T02:26:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:01 smithi090 ceph-mon[29180]: pgmap v15641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28242.420022 secs 2024-04-27T02:26:02.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:02 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28243.253917 secs 2024-04-27T02:26:02.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:02 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28243.253917 secs 2024-04-27T02:26:02.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:02 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28243.253917 secs 2024-04-27T02:26:03.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:03 smithi119 ceph-mon[32752]: pgmap v15642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:03.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:03 smithi167 ceph-mon[32751]: pgmap v15642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:03.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:03 smithi090 ceph-mon[29180]: pgmap v15642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:05.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:05 smithi119 ceph-mon[32752]: pgmap v15643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:05.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:05 smithi167 ceph-mon[32751]: pgmap v15643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:05.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:05 smithi090 ceph-mon[29180]: pgmap v15643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:06.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28247.420107 secs 2024-04-27T02:26:06.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:26:06.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28247.420107 secs 2024-04-27T02:26:06.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:26:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28247.420107 secs 2024-04-27T02:26:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:26:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:07 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28248.254039 secs 2024-04-27T02:26:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:07 smithi119 ceph-mon[32752]: pgmap v15644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:26:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:26:07.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:26:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:07 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28248.254039 secs 2024-04-27T02:26:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:07 smithi167 ceph-mon[32751]: pgmap v15644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:26:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:26:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:26:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:07 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28248.254039 secs 2024-04-27T02:26:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:07 smithi090 ceph-mon[29180]: pgmap v15644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:26:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:26:07.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:26:08.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:08 smithi119 ceph-mon[32752]: pgmap v15645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:26:08.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:08 smithi119 ceph-mon[32752]: pgmap v15646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:08.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:08 smithi167 ceph-mon[32751]: pgmap v15645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:26:08.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:08 smithi167 ceph-mon[32751]: pgmap v15646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:08.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:08 smithi090 ceph-mon[29180]: pgmap v15645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:26:08.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:08 smithi090 ceph-mon[29180]: pgmap v15646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:10.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:10 smithi119 ceph-mon[32752]: pgmap v15647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:10.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:10 smithi167 ceph-mon[32751]: pgmap v15647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:10 smithi090 ceph-mon[29180]: pgmap v15647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28252.420166 secs 2024-04-27T02:26:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28252.420166 secs 2024-04-27T02:26:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28252.420166 secs 2024-04-27T02:26:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:12 smithi119 ceph-mon[32752]: pgmap v15648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:12 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28253.254154 secs 2024-04-27T02:26:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:12 smithi167 ceph-mon[32751]: pgmap v15648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:12 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28253.254154 secs 2024-04-27T02:26:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:12 smithi090 ceph-mon[29180]: pgmap v15648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:12.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:12 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28253.254154 secs 2024-04-27T02:26:14.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:14 smithi119 ceph-mon[32752]: pgmap v15649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:26:14.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:14 smithi167 ceph-mon[32751]: pgmap v15649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:26:14.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:14 smithi090 ceph-mon[29180]: pgmap v15649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:26:16.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:16 smithi119 ceph-mon[32752]: pgmap v15650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:16.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28257.420248 secs 2024-04-27T02:26:16.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:16 smithi167 ceph-mon[32751]: pgmap v15650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:16.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28257.420248 secs 2024-04-27T02:26:16.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:16 smithi090 ceph-mon[29180]: pgmap v15650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:26:16.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28257.420248 secs 2024-04-27T02:26:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28258.254310 secs 2024-04-27T02:26:17.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28258.254310 secs 2024-04-27T02:26:17.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28258.254310 secs 2024-04-27T02:26:18.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:18 smithi119 ceph-mon[32752]: pgmap v15651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:26:18.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:18 smithi167 ceph-mon[32751]: pgmap v15651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:26:18.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:18 smithi090 ceph-mon[29180]: pgmap v15651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:26:20.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:20 smithi119 ceph-mon[32752]: pgmap v15652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:20.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:20 smithi167 ceph-mon[32751]: pgmap v15652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:20.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:20 smithi090 ceph-mon[29180]: pgmap v15652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28262.420322 secs 2024-04-27T02:26:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28262.420322 secs 2024-04-27T02:26:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28262.420322 secs 2024-04-27T02:26:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:22 smithi119 ceph-mon[32752]: pgmap v15653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28263.254427 secs 2024-04-27T02:26:22.545 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:22 smithi167 ceph-mon[32751]: pgmap v15653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:22.546 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28263.254427 secs 2024-04-27T02:26:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:22 smithi090 ceph-mon[29180]: pgmap v15653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28263.254427 secs 2024-04-27T02:26:24.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:24 smithi119 ceph-mon[32752]: pgmap v15654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:24.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:24 smithi167 ceph-mon[32751]: pgmap v15654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:24.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:24 smithi090 ceph-mon[29180]: pgmap v15654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:26 smithi119 ceph-mon[32752]: pgmap v15655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28267.420404 secs 2024-04-27T02:26:26.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:26 smithi167 ceph-mon[32751]: pgmap v15655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:26.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28267.420404 secs 2024-04-27T02:26:26.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:26 smithi090 ceph-mon[29180]: pgmap v15655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:26.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28267.420404 secs 2024-04-27T02:26:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28268.254546 secs 2024-04-27T02:26:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28268.254546 secs 2024-04-27T02:26:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28268.254546 secs 2024-04-27T02:26:28.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:28 smithi119 ceph-mon[32752]: pgmap v15656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:28.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:28 smithi167 ceph-mon[32751]: pgmap v15656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:28.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:28 smithi090 ceph-mon[29180]: pgmap v15656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:29.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:29 smithi167 ceph-mon[32751]: pgmap v15657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:29.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:29 smithi090 ceph-mon[29180]: pgmap v15657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:29.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:29 smithi119 ceph-mon[32752]: pgmap v15657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:32.246 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28272.420490 secs 2024-04-27T02:26:32.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:31 smithi167 ceph-mon[32751]: pgmap v15658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:32.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28273.254663 secs 2024-04-27T02:26:32.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28272.420490 secs 2024-04-27T02:26:32.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:31 smithi119 ceph-mon[32752]: pgmap v15658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:32.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28273.254663 secs 2024-04-27T02:26:32.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28272.420490 secs 2024-04-27T02:26:32.304 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:31 smithi090 ceph-mon[29180]: pgmap v15658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:32.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28273.254663 secs 2024-04-27T02:26:34.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:33 smithi119 ceph-mon[32752]: pgmap v15659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:34.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:33 smithi167 ceph-mon[32751]: pgmap v15659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:34.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:33 smithi090 ceph-mon[29180]: pgmap v15659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:36.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:35 smithi119 ceph-mon[32752]: pgmap v15660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:36.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:35 smithi167 ceph-mon[32751]: pgmap v15660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:36.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:35 smithi090 ceph-mon[29180]: pgmap v15660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:37.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28277.420576 secs 2024-04-27T02:26:37.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28278.254715 secs 2024-04-27T02:26:37.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28277.420576 secs 2024-04-27T02:26:37.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28278.254715 secs 2024-04-27T02:26:37.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28277.420576 secs 2024-04-27T02:26:37.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28278.254715 secs 2024-04-27T02:26:38.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:37 smithi119 ceph-mon[32752]: pgmap v15661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:38.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:37 smithi167 ceph-mon[32751]: pgmap v15661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:38.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:37 smithi090 ceph-mon[29180]: pgmap v15661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:40.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:39 smithi119 ceph-mon[32752]: pgmap v15662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:40.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:39 smithi167 ceph-mon[32751]: pgmap v15662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:40.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:39 smithi090 ceph-mon[29180]: pgmap v15662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:42.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28282.420661 secs 2024-04-27T02:26:42.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:41 smithi119 ceph-mon[32752]: pgmap v15663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:42.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28283.254794 secs 2024-04-27T02:26:42.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28282.420661 secs 2024-04-27T02:26:42.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:41 smithi167 ceph-mon[32751]: pgmap v15663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:42.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28283.254794 secs 2024-04-27T02:26:42.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28282.420661 secs 2024-04-27T02:26:42.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:41 smithi090 ceph-mon[29180]: pgmap v15663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:42.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28283.254794 secs 2024-04-27T02:26:44.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:43 smithi119 ceph-mon[32752]: pgmap v15664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:44.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:43 smithi167 ceph-mon[32751]: pgmap v15664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:44.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:43 smithi090 ceph-mon[29180]: pgmap v15664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:46.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:46 smithi090 ceph-mon[29180]: pgmap v15665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:46.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28287.420746 secs 2024-04-27T02:26:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:46 smithi119 ceph-mon[32752]: pgmap v15665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28287.420746 secs 2024-04-27T02:26:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:46 smithi167 ceph-mon[32751]: pgmap v15665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28287.420746 secs 2024-04-27T02:26:47.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:47 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28288.254932 secs 2024-04-27T02:26:47.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:47 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28288.254932 secs 2024-04-27T02:26:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:47 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28288.254932 secs 2024-04-27T02:26:48.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:48 smithi090 ceph-mon[29180]: pgmap v15666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:48.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:48 smithi119 ceph-mon[32752]: pgmap v15666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:48.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:48 smithi167 ceph-mon[32751]: pgmap v15666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:50.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:50 smithi090 ceph-mon[29180]: pgmap v15667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:50.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:50 smithi119 ceph-mon[32752]: pgmap v15667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:50.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:50 smithi167 ceph-mon[32751]: pgmap v15667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:51.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28292.420832 secs 2024-04-27T02:26:51.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28292.420832 secs 2024-04-27T02:26:51.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28292.420832 secs 2024-04-27T02:26:52.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:52 smithi090 ceph-mon[29180]: pgmap v15668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:52.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:52 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28293.255041 secs 2024-04-27T02:26:52.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:52 smithi119 ceph-mon[32752]: pgmap v15668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:52.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:52 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28293.255041 secs 2024-04-27T02:26:52.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:52 smithi167 ceph-mon[32751]: pgmap v15668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:52.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:52 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28293.255041 secs 2024-04-27T02:26:54.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:54 smithi090 ceph-mon[29180]: pgmap v15669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:54.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:54 smithi119 ceph-mon[32752]: pgmap v15669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:54.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:54 smithi167 ceph-mon[32751]: pgmap v15669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:56.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:56 smithi090 ceph-mon[29180]: pgmap v15670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:56.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28297.420921 secs 2024-04-27T02:26:56.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:56 smithi119 ceph-mon[32752]: pgmap v15670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:56.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28297.420921 secs 2024-04-27T02:26:56.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:56 smithi167 ceph-mon[32751]: pgmap v15670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:26:56.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28297.420921 secs 2024-04-27T02:26:57.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:57 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28298.255156 secs 2024-04-27T02:26:57.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:57 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28298.255156 secs 2024-04-27T02:26:57.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:57 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28298.255156 secs 2024-04-27T02:26:58.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:26:58 smithi090 ceph-mon[29180]: pgmap v15671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:58.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:26:58 smithi119 ceph-mon[32752]: pgmap v15671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:26:58.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:26:58 smithi167 ceph-mon[32751]: pgmap v15671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:00.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:00 smithi090 ceph-mon[29180]: pgmap v15672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:00.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:00 smithi119 ceph-mon[32752]: pgmap v15672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:00.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:00 smithi167 ceph-mon[32751]: pgmap v15672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:01.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28302.421013 secs 2024-04-27T02:27:01.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28302.421013 secs 2024-04-27T02:27:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28302.421013 secs 2024-04-27T02:27:02.430 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:02 smithi119 ceph-mon[32752]: pgmap v15673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:02.430 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:02 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28303.255249 secs 2024-04-27T02:27:02.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:02 smithi167 ceph-mon[32751]: pgmap v15673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:02.445 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:02 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28303.255249 secs 2024-04-27T02:27:02.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:02 smithi090 ceph-mon[29180]: pgmap v15673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:02.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:02 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28303.255249 secs 2024-04-27T02:27:04.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:04 smithi119 ceph-mon[32752]: pgmap v15674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:04.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:04 smithi167 ceph-mon[32751]: pgmap v15674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:04.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:04 smithi090 ceph-mon[29180]: pgmap v15674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:06.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:06 smithi119 ceph-mon[32752]: pgmap v15675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:06.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28307.421101 secs 2024-04-27T02:27:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:06 smithi167 ceph-mon[32751]: pgmap v15675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28307.421101 secs 2024-04-27T02:27:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:06 smithi090 ceph-mon[29180]: pgmap v15675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:06.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28307.421101 secs 2024-04-27T02:27:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:07 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28308.255318 secs 2024-04-27T02:27:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:27:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:07 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28308.255318 secs 2024-04-27T02:27:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:27:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:07 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28308.255318 secs 2024-04-27T02:27:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:27:08.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:08 smithi119 ceph-mon[32752]: pgmap v15676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:08.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:27:08.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:27:08.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:27:08.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:08 smithi167 ceph-mon[32751]: pgmap v15676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:08.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:27:08.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:27:08.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:27:08.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:08 smithi090 ceph-mon[29180]: pgmap v15676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:08.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:27:08.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:27:08.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:27:09.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:09 smithi119 ceph-mon[32752]: pgmap v15677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:09.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:09 smithi167 ceph-mon[32751]: pgmap v15677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:09.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:09 smithi090 ceph-mon[29180]: pgmap v15677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:11 smithi119 ceph-mon[32752]: pgmap v15678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28312.421163 secs 2024-04-27T02:27:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:11 smithi167 ceph-mon[32751]: pgmap v15678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28312.421163 secs 2024-04-27T02:27:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:11 smithi090 ceph-mon[29180]: pgmap v15678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:11.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28312.421163 secs 2024-04-27T02:27:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:12 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28313.255416 secs 2024-04-27T02:27:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:12 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28313.255416 secs 2024-04-27T02:27:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:12 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28313.255416 secs 2024-04-27T02:27:13.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:13 smithi119 ceph-mon[32752]: pgmap v15679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:13.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:13 smithi167 ceph-mon[32751]: pgmap v15679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:13.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:13 smithi090 ceph-mon[29180]: pgmap v15679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:15.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:15 smithi119 ceph-mon[32752]: pgmap v15680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:27:15.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:15 smithi167 ceph-mon[32751]: pgmap v15680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:27:15.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:15 smithi090 ceph-mon[29180]: pgmap v15680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:27:16.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28317.421225 secs 2024-04-27T02:27:16.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28317.421225 secs 2024-04-27T02:27:16.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28317.421225 secs 2024-04-27T02:27:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:17 smithi119 ceph-mon[32752]: pgmap v15681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:17.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28318.255540 secs 2024-04-27T02:27:17.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:17 smithi167 ceph-mon[32751]: pgmap v15681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:17.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28318.255540 secs 2024-04-27T02:27:17.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:17 smithi090 ceph-mon[29180]: pgmap v15681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:17.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28318.255540 secs 2024-04-27T02:27:19.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:19 smithi119 ceph-mon[32752]: pgmap v15682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:19.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:19 smithi167 ceph-mon[32751]: pgmap v15682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:19.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:19 smithi090 ceph-mon[29180]: pgmap v15682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:27:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:21 smithi119 ceph-mon[32752]: pgmap v15683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28322.421284 secs 2024-04-27T02:27:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:21 smithi167 ceph-mon[32751]: pgmap v15683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:21.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28322.421284 secs 2024-04-27T02:27:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:21 smithi090 ceph-mon[29180]: pgmap v15683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:21.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28322.421284 secs 2024-04-27T02:27:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28323.255665 secs 2024-04-27T02:27:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28323.255665 secs 2024-04-27T02:27:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28323.255665 secs 2024-04-27T02:27:23.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:23 smithi119 ceph-mon[32752]: pgmap v15684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:23.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:23 smithi167 ceph-mon[32751]: pgmap v15684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:23.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:23 smithi090 ceph-mon[29180]: pgmap v15684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:25.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:25 smithi119 ceph-mon[32752]: pgmap v15685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:25.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:25 smithi167 ceph-mon[32751]: pgmap v15685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:25.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:25 smithi090 ceph-mon[29180]: pgmap v15685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28327.421365 secs 2024-04-27T02:27:26.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28327.421365 secs 2024-04-27T02:27:26.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28327.421365 secs 2024-04-27T02:27:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:27 smithi119 ceph-mon[32752]: pgmap v15686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28328.255784 secs 2024-04-27T02:27:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:27 smithi167 ceph-mon[32751]: pgmap v15686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28328.255784 secs 2024-04-27T02:27:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:27 smithi090 ceph-mon[29180]: pgmap v15686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28328.255784 secs 2024-04-27T02:27:29.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:29 smithi119 ceph-mon[32752]: pgmap v15687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:29.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:29 smithi167 ceph-mon[32751]: pgmap v15687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:29.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:29 smithi090 ceph-mon[29180]: pgmap v15687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:31 smithi119 ceph-mon[32752]: pgmap v15688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28332.421448 secs 2024-04-27T02:27:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:31 smithi167 ceph-mon[32751]: pgmap v15688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28332.421448 secs 2024-04-27T02:27:31.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:31 smithi090 ceph-mon[29180]: pgmap v15688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:31.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28332.421448 secs 2024-04-27T02:27:32.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:32 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28333.255891 secs 2024-04-27T02:27:32.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:32 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28333.255891 secs 2024-04-27T02:27:32.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:32 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28333.255891 secs 2024-04-27T02:27:33.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:33 smithi119 ceph-mon[32752]: pgmap v15689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:33.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:33 smithi167 ceph-mon[32751]: pgmap v15689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:33.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:33 smithi090 ceph-mon[29180]: pgmap v15689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:35.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:35 smithi090 ceph-mon[29180]: pgmap v15690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:35.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:35 smithi119 ceph-mon[32752]: pgmap v15690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:35.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:35 smithi167 ceph-mon[32751]: pgmap v15690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:36.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28337.421529 secs 2024-04-27T02:27:36.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28337.421529 secs 2024-04-27T02:27:36.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28337.421529 secs 2024-04-27T02:27:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:37 smithi119 ceph-mon[32752]: pgmap v15691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:37 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28338.256005 secs 2024-04-27T02:27:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:37 smithi167 ceph-mon[32751]: pgmap v15691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:37 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28338.256005 secs 2024-04-27T02:27:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:37 smithi090 ceph-mon[29180]: pgmap v15691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:37 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28338.256005 secs 2024-04-27T02:27:39.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:39 smithi119 ceph-mon[32752]: pgmap v15692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:39.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:39 smithi167 ceph-mon[32751]: pgmap v15692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:39.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:39 smithi090 ceph-mon[29180]: pgmap v15692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:41 smithi119 ceph-mon[32752]: pgmap v15693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28342.421617 secs 2024-04-27T02:27:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:41 smithi167 ceph-mon[32751]: pgmap v15693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28342.421617 secs 2024-04-27T02:27:41.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:41 smithi090 ceph-mon[29180]: pgmap v15693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:41.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28342.421617 secs 2024-04-27T02:27:42.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:42 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28343.256114 secs 2024-04-27T02:27:42.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:42 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28343.256114 secs 2024-04-27T02:27:42.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:42 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28343.256114 secs 2024-04-27T02:27:43.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:43 smithi119 ceph-mon[32752]: pgmap v15694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:43.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:43 smithi167 ceph-mon[32751]: pgmap v15694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:43.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:43 smithi090 ceph-mon[29180]: pgmap v15694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:45.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:45 smithi119 ceph-mon[32752]: pgmap v15695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:45.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:45 smithi167 ceph-mon[32751]: pgmap v15695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:45.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:45 smithi090 ceph-mon[29180]: pgmap v15695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28347.421701 secs 2024-04-27T02:27:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28347.421701 secs 2024-04-27T02:27:46.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28347.421701 secs 2024-04-27T02:27:47.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:47 smithi119 ceph-mon[32752]: pgmap v15696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:47.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:47 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28348.256229 secs 2024-04-27T02:27:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:47 smithi167 ceph-mon[32751]: pgmap v15696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:47.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:47 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28348.256229 secs 2024-04-27T02:27:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:47 smithi090 ceph-mon[29180]: pgmap v15696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:47 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28348.256229 secs 2024-04-27T02:27:49.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:49 smithi119 ceph-mon[32752]: pgmap v15697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:49.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:49 smithi167 ceph-mon[32751]: pgmap v15697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:49.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:49 smithi090 ceph-mon[29180]: pgmap v15697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:51.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:51 smithi119 ceph-mon[32752]: pgmap v15698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:51.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28352.421791 secs 2024-04-27T02:27:51.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:51 smithi167 ceph-mon[32751]: pgmap v15698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:51.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28352.421791 secs 2024-04-27T02:27:51.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:51 smithi090 ceph-mon[29180]: pgmap v15698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:51.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28352.421791 secs 2024-04-27T02:27:52.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:52 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28353.256299 secs 2024-04-27T02:27:52.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:52 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28353.256299 secs 2024-04-27T02:27:52.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:52 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28353.256299 secs 2024-04-27T02:27:53.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:53 smithi119 ceph-mon[32752]: pgmap v15699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:53.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:53 smithi167 ceph-mon[32751]: pgmap v15699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:53.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:53 smithi090 ceph-mon[29180]: pgmap v15699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:55.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:55 smithi119 ceph-mon[32752]: pgmap v15700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:55.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:55 smithi167 ceph-mon[32751]: pgmap v15700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:55.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:55 smithi090 ceph-mon[29180]: pgmap v15700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:27:56.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28357.421865 secs 2024-04-27T02:27:56.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28357.421865 secs 2024-04-27T02:27:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28357.421865 secs 2024-04-27T02:27:57.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:57 smithi119 ceph-mon[32752]: pgmap v15701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:57.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:57 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28358.256432 secs 2024-04-27T02:27:57.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:57 smithi167 ceph-mon[32751]: pgmap v15701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:57.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:57 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28358.256432 secs 2024-04-27T02:27:57.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:57 smithi090 ceph-mon[29180]: pgmap v15701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:57.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:57 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28358.256432 secs 2024-04-27T02:27:59.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:27:59 smithi119 ceph-mon[32752]: pgmap v15702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:59.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:27:59 smithi167 ceph-mon[32751]: pgmap v15702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:27:59.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:27:59 smithi090 ceph-mon[29180]: pgmap v15702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:01.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:01 smithi119 ceph-mon[32752]: pgmap v15703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:01.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28362.421944 secs 2024-04-27T02:28:01.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:01 smithi167 ceph-mon[32751]: pgmap v15703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:01.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28362.421944 secs 2024-04-27T02:28:01.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:01 smithi090 ceph-mon[29180]: pgmap v15703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28362.421944 secs 2024-04-27T02:28:02.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:02 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28363.256572 secs 2024-04-27T02:28:02.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:02 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28363.256572 secs 2024-04-27T02:28:02.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:02 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28363.256572 secs 2024-04-27T02:28:03.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:03 smithi119 ceph-mon[32752]: pgmap v15704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:03.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:03 smithi167 ceph-mon[32751]: pgmap v15704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:03.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:03 smithi090 ceph-mon[29180]: pgmap v15704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:05.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:05 smithi119 ceph-mon[32752]: pgmap v15705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:05.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:05 smithi167 ceph-mon[32751]: pgmap v15705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:05.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:05 smithi090 ceph-mon[29180]: pgmap v15705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:06.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28367.422031 secs 2024-04-27T02:28:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28367.422031 secs 2024-04-27T02:28:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28367.422031 secs 2024-04-27T02:28:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:07 smithi119 ceph-mon[32752]: pgmap v15706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:07.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:07 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28368.256689 secs 2024-04-27T02:28:07.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:07 smithi167 ceph-mon[32751]: pgmap v15706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:07.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:07 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28368.256689 secs 2024-04-27T02:28:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:07 smithi090 ceph-mon[29180]: pgmap v15706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:07.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:07 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28368.256689 secs 2024-04-27T02:28:08.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:28:08.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:28:08.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:28:09.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:09 smithi119 ceph-mon[32752]: pgmap v15707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:09.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:28:09.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:28:09.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:28:09.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:09 smithi167 ceph-mon[32751]: pgmap v15707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:09.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:28:09.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:28:09.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:28:09.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:09 smithi090 ceph-mon[29180]: pgmap v15707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:09.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:28:09.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:28:09.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:28:10.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:10 smithi167 ceph-mon[32751]: pgmap v15708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:28:10.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:10 smithi167 ceph-mon[32751]: pgmap v15709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:10 smithi090 ceph-mon[29180]: pgmap v15708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:28:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:10 smithi090 ceph-mon[29180]: pgmap v15709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:10.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:10 smithi119 ceph-mon[32752]: pgmap v15708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:28:10.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:10 smithi119 ceph-mon[32752]: pgmap v15709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:11.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28372.422115 secs 2024-04-27T02:28:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28372.422115 secs 2024-04-27T02:28:11.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28372.422115 secs 2024-04-27T02:28:12.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:12 smithi167 ceph-mon[32751]: pgmap v15710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:12 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28373.256805 secs 2024-04-27T02:28:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:12 smithi090 ceph-mon[29180]: pgmap v15710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:12 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28373.256805 secs 2024-04-27T02:28:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:12 smithi119 ceph-mon[32752]: pgmap v15710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:12.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:12 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28373.256805 secs 2024-04-27T02:28:13.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:13 smithi167 ceph-mon[32751]: pgmap v15711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:28:13.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:13 smithi090 ceph-mon[29180]: pgmap v15711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:28:13.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:13 smithi119 ceph-mon[32752]: pgmap v15711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:28:15.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:15 smithi119 ceph-mon[32752]: pgmap v15712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:15.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:15 smithi167 ceph-mon[32751]: pgmap v15712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:15.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:15 smithi090 ceph-mon[29180]: pgmap v15712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:16.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28377.422197 secs 2024-04-27T02:28:16.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28377.422197 secs 2024-04-27T02:28:16.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28377.422197 secs 2024-04-27T02:28:17.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28378.256941 secs 2024-04-27T02:28:17.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:17 smithi119 ceph-mon[32752]: pgmap v15713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:17.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28378.256941 secs 2024-04-27T02:28:17.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:17 smithi167 ceph-mon[32751]: pgmap v15713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:17.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28378.256941 secs 2024-04-27T02:28:17.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:17 smithi090 ceph-mon[29180]: pgmap v15713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:28:19.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:19 smithi119 ceph-mon[32752]: pgmap v15714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:28:19.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:19 smithi167 ceph-mon[32751]: pgmap v15714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:28:19.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:19 smithi090 ceph-mon[29180]: pgmap v15714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:28:21.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28382.422277 secs 2024-04-27T02:28:21.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:21 smithi119 ceph-mon[32752]: pgmap v15715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:21.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28382.422277 secs 2024-04-27T02:28:21.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:21 smithi167 ceph-mon[32751]: pgmap v15715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:21.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28382.422277 secs 2024-04-27T02:28:21.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:21 smithi090 ceph-mon[29180]: pgmap v15715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:22.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28383.257010 secs 2024-04-27T02:28:22.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28383.257010 secs 2024-04-27T02:28:22.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28383.257010 secs 2024-04-27T02:28:23.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:23 smithi119 ceph-mon[32752]: pgmap v15716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:23.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:23 smithi167 ceph-mon[32751]: pgmap v15716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:23.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:23 smithi090 ceph-mon[29180]: pgmap v15716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:25.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:25 smithi119 ceph-mon[32752]: pgmap v15717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:25.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:25 smithi167 ceph-mon[32751]: pgmap v15717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:25.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:25 smithi090 ceph-mon[29180]: pgmap v15717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:26.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28387.422353 secs 2024-04-27T02:28:26.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28387.422353 secs 2024-04-27T02:28:26.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28387.422353 secs 2024-04-27T02:28:27.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28388.257125 secs 2024-04-27T02:28:27.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:27 smithi119 ceph-mon[32752]: pgmap v15718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:27.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28388.257125 secs 2024-04-27T02:28:27.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:27 smithi167 ceph-mon[32751]: pgmap v15718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:27.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28388.257125 secs 2024-04-27T02:28:27.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:27 smithi090 ceph-mon[29180]: pgmap v15718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:29.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:29 smithi119 ceph-mon[32752]: pgmap v15719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:29.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:29 smithi167 ceph-mon[32751]: pgmap v15719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:29.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:29 smithi090 ceph-mon[29180]: pgmap v15719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:31.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28392.422439 secs 2024-04-27T02:28:31.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:31 smithi119 ceph-mon[32752]: pgmap v15720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:31.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28392.422439 secs 2024-04-27T02:28:31.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:31 smithi167 ceph-mon[32751]: pgmap v15720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:31.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28392.422439 secs 2024-04-27T02:28:31.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:31 smithi090 ceph-mon[29180]: pgmap v15720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:32.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:32 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28393.257243 secs 2024-04-27T02:28:32.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:32 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28393.257243 secs 2024-04-27T02:28:32.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:32 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28393.257243 secs 2024-04-27T02:28:33.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:33 smithi119 ceph-mon[32752]: pgmap v15721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:33.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:33 smithi167 ceph-mon[32751]: pgmap v15721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:33.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:33 smithi090 ceph-mon[29180]: pgmap v15721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:35.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:35 smithi119 ceph-mon[32752]: pgmap v15722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:35.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:35 smithi167 ceph-mon[32751]: pgmap v15722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:35.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:35 smithi090 ceph-mon[29180]: pgmap v15722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:36.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28397.422514 secs 2024-04-27T02:28:36.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28397.422514 secs 2024-04-27T02:28:36.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28397.422514 secs 2024-04-27T02:28:37.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:37 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28398.257364 secs 2024-04-27T02:28:37.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:37 smithi119 ceph-mon[32752]: pgmap v15723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:37.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:37 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28398.257364 secs 2024-04-27T02:28:37.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:37 smithi167 ceph-mon[32751]: pgmap v15723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:37.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:37 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28398.257364 secs 2024-04-27T02:28:37.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:37 smithi090 ceph-mon[29180]: pgmap v15723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:39.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:39 smithi119 ceph-mon[32752]: pgmap v15724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:39.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:39 smithi167 ceph-mon[32751]: pgmap v15724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:39.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:39 smithi090 ceph-mon[29180]: pgmap v15724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:41.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28402.422614 secs 2024-04-27T02:28:41.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:41 smithi119 ceph-mon[32752]: pgmap v15725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:41.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28402.422614 secs 2024-04-27T02:28:41.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:41 smithi167 ceph-mon[32751]: pgmap v15725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:41.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28402.422614 secs 2024-04-27T02:28:41.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:41 smithi090 ceph-mon[29180]: pgmap v15725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:42.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:42 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28403.257429 secs 2024-04-27T02:28:42.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:42 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28403.257429 secs 2024-04-27T02:28:42.851 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:42 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28403.257429 secs 2024-04-27T02:28:43.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:43 smithi119 ceph-mon[32752]: pgmap v15726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:43.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:43 smithi167 ceph-mon[32751]: pgmap v15726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:43.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:43 smithi090 ceph-mon[29180]: pgmap v15726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:46.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:46 smithi090 ceph-mon[29180]: pgmap v15727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:46.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28407.422702 secs 2024-04-27T02:28:46.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:46 smithi119 ceph-mon[32752]: pgmap v15727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:46.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28407.422702 secs 2024-04-27T02:28:46.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:46 smithi167 ceph-mon[32751]: pgmap v15727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:46.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28407.422702 secs 2024-04-27T02:28:47.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:47 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28408.257550 secs 2024-04-27T02:28:47.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:47 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28408.257550 secs 2024-04-27T02:28:47.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:47 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28408.257550 secs 2024-04-27T02:28:48.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:48 smithi090 ceph-mon[29180]: pgmap v15728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:48.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:48 smithi119 ceph-mon[32752]: pgmap v15728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:48.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:48 smithi167 ceph-mon[32751]: pgmap v15728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:50.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:50 smithi090 ceph-mon[29180]: pgmap v15729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:50.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:50 smithi119 ceph-mon[32752]: pgmap v15729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:50.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:50 smithi167 ceph-mon[32751]: pgmap v15729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:51.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28412.422802 secs 2024-04-27T02:28:51.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28412.422802 secs 2024-04-27T02:28:51.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28412.422802 secs 2024-04-27T02:28:52.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:52 smithi119 ceph-mon[32752]: pgmap v15730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:52.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:52 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28413.257663 secs 2024-04-27T02:28:52.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:52 smithi167 ceph-mon[32751]: pgmap v15730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:52.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:52 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28413.257663 secs 2024-04-27T02:28:52.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:52 smithi090 ceph-mon[29180]: pgmap v15730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:52.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:52 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28413.257663 secs 2024-04-27T02:28:54.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:54 smithi119 ceph-mon[32752]: pgmap v15731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:54.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:54 smithi167 ceph-mon[32751]: pgmap v15731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:54.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:54 smithi090 ceph-mon[29180]: pgmap v15731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:56.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:56 smithi119 ceph-mon[32752]: pgmap v15732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:56.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28417.422890 secs 2024-04-27T02:28:56.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:56 smithi167 ceph-mon[32751]: pgmap v15732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:56.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28417.422890 secs 2024-04-27T02:28:56.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:56 smithi090 ceph-mon[29180]: pgmap v15732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:28:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28417.422890 secs 2024-04-27T02:28:57.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:57 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28418.257752 secs 2024-04-27T02:28:57.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:57 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28418.257752 secs 2024-04-27T02:28:57.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:57 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28418.257752 secs 2024-04-27T02:28:58.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:28:58 smithi119 ceph-mon[32752]: pgmap v15733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:58.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:28:58 smithi167 ceph-mon[32751]: pgmap v15733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:28:58.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:28:58 smithi090 ceph-mon[29180]: pgmap v15733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:00.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:00 smithi119 ceph-mon[32752]: pgmap v15734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:00.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:00 smithi167 ceph-mon[32751]: pgmap v15734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:00.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:00 smithi090 ceph-mon[29180]: pgmap v15734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:01.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28422.422976 secs 2024-04-27T02:29:01.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28422.422976 secs 2024-04-27T02:29:01.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28422.422976 secs 2024-04-27T02:29:02.429 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:02 smithi119 ceph-mon[32752]: pgmap v15735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:02.430 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:02 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28423.257875 secs 2024-04-27T02:29:02.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:02 smithi167 ceph-mon[32751]: pgmap v15735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:02.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:02 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28423.257875 secs 2024-04-27T02:29:02.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:02 smithi090 ceph-mon[29180]: pgmap v15735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:02.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:02 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28423.257875 secs 2024-04-27T02:29:04.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:04 smithi119 ceph-mon[32752]: pgmap v15736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:04.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:04 smithi167 ceph-mon[32751]: pgmap v15736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:04.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:04 smithi090 ceph-mon[29180]: pgmap v15736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:06.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:06 smithi119 ceph-mon[32752]: pgmap v15737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:06.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28427.423036 secs 2024-04-27T02:29:06.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:06 smithi167 ceph-mon[32751]: pgmap v15737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28427.423036 secs 2024-04-27T02:29:06.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:06 smithi090 ceph-mon[29180]: pgmap v15737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:06.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28427.423036 secs 2024-04-27T02:29:07.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:07 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28428.258003 secs 2024-04-27T02:29:07.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:07 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28428.258003 secs 2024-04-27T02:29:07.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:07 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28428.258003 secs 2024-04-27T02:29:08.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:08 smithi119 ceph-mon[32752]: pgmap v15738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:08.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:08 smithi167 ceph-mon[32751]: pgmap v15738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:08.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:08 smithi090 ceph-mon[29180]: pgmap v15738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:09.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:29:09.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:29:09.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:29:10.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:10 smithi119 ceph-mon[32752]: pgmap v15739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:10.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:29:10.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:29:10.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:10 smithi119 ceph-mon[32752]: pgmap v15740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:29:10.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:10 smithi119 ceph-mon[32752]: pgmap v15741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:29:10.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:29:10.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:10 smithi167 ceph-mon[32751]: pgmap v15739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:10.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:29:10.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:29:10.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:10 smithi167 ceph-mon[32751]: pgmap v15740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:29:10.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:10 smithi167 ceph-mon[32751]: pgmap v15741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:29:10.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:29:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:10 smithi090 ceph-mon[29180]: pgmap v15739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:29:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:29:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:10 smithi090 ceph-mon[29180]: pgmap v15740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:29:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:10 smithi090 ceph-mon[29180]: pgmap v15741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:29:10.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:29:11.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28432.423096 secs 2024-04-27T02:29:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28432.423096 secs 2024-04-27T02:29:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28432.423096 secs 2024-04-27T02:29:12.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:12 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28433.258123 secs 2024-04-27T02:29:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:12 smithi119 ceph-mon[32752]: pgmap v15742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:29:12.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:12 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28433.258123 secs 2024-04-27T02:29:12.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:12 smithi167 ceph-mon[32751]: pgmap v15742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:29:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:12 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28433.258123 secs 2024-04-27T02:29:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:12 smithi090 ceph-mon[29180]: pgmap v15742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:29:14.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:14 smithi119 ceph-mon[32752]: pgmap v15743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:29:14.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:14 smithi167 ceph-mon[32751]: pgmap v15743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:29:14.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:14 smithi090 ceph-mon[29180]: pgmap v15743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:29:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28437.423178 secs 2024-04-27T02:29:16.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:16 smithi119 ceph-mon[32752]: pgmap v15744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T02:29:16.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28437.423178 secs 2024-04-27T02:29:16.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:16 smithi167 ceph-mon[32751]: pgmap v15744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T02:29:16.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28437.423178 secs 2024-04-27T02:29:16.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:16 smithi090 ceph-mon[29180]: pgmap v15744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T02:29:17.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:17 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28438.258247 secs 2024-04-27T02:29:17.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:17 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28438.258247 secs 2024-04-27T02:29:17.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:17 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28438.258247 secs 2024-04-27T02:29:18.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:18 smithi119 ceph-mon[32752]: pgmap v15745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T02:29:18.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:18 smithi167 ceph-mon[32751]: pgmap v15745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T02:29:18.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:18 smithi090 ceph-mon[29180]: pgmap v15745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T02:29:20.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:20 smithi119 ceph-mon[32752]: pgmap v15746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-27T02:29:20.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:20 smithi167 ceph-mon[32751]: pgmap v15746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-27T02:29:20.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:20 smithi090 ceph-mon[29180]: pgmap v15746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-27T02:29:21.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28442.423267 secs 2024-04-27T02:29:21.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28442.423267 secs 2024-04-27T02:29:21.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28442.423267 secs 2024-04-27T02:29:22.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:22 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28443.258401 secs 2024-04-27T02:29:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:22 smithi119 ceph-mon[32752]: pgmap v15747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:29:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:22 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28443.258401 secs 2024-04-27T02:29:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:22 smithi167 ceph-mon[32751]: pgmap v15747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:29:22.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:22 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28443.258401 secs 2024-04-27T02:29:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:22 smithi090 ceph-mon[29180]: pgmap v15747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:29:24.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:24 smithi119 ceph-mon[32752]: pgmap v15748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:29:24.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:24 smithi167 ceph-mon[32751]: pgmap v15748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:29:24.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:24 smithi090 ceph-mon[29180]: pgmap v15748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:29:26.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28447.423350 secs 2024-04-27T02:29:26.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:26 smithi119 ceph-mon[32752]: pgmap v15749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:29:26.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28447.423350 secs 2024-04-27T02:29:26.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:26 smithi167 ceph-mon[32751]: pgmap v15749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:29:26.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28447.423350 secs 2024-04-27T02:29:26.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:26 smithi090 ceph-mon[29180]: pgmap v15749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:29:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:27 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28448.258554 secs 2024-04-27T02:29:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:27 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28448.258554 secs 2024-04-27T02:29:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:27 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28448.258554 secs 2024-04-27T02:29:28.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:28 smithi119 ceph-mon[32752]: pgmap v15750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:28.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:28 smithi167 ceph-mon[32751]: pgmap v15750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:28.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:28 smithi090 ceph-mon[29180]: pgmap v15750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:30.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:30 smithi119 ceph-mon[32752]: pgmap v15751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:30.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:30 smithi167 ceph-mon[32751]: pgmap v15751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:30.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:30 smithi090 ceph-mon[29180]: pgmap v15751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:31.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28452.423436 secs 2024-04-27T02:29:31.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28452.423436 secs 2024-04-27T02:29:31.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28452.423436 secs 2024-04-27T02:29:32.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:32 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28453.258661 secs 2024-04-27T02:29:32.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:32 smithi119 ceph-mon[32752]: pgmap v15752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:32.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:32 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28453.258661 secs 2024-04-27T02:29:32.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:32 smithi167 ceph-mon[32751]: pgmap v15752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:32.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:32 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28453.258661 secs 2024-04-27T02:29:32.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:32 smithi090 ceph-mon[29180]: pgmap v15752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:34.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:34 smithi167 ceph-mon[32751]: pgmap v15753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:34.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:34 smithi090 ceph-mon[29180]: pgmap v15753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:34.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:34 smithi119 ceph-mon[32752]: pgmap v15753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:36.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28457.423508 secs 2024-04-27T02:29:36.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:36 smithi167 ceph-mon[32751]: pgmap v15754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:36.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28457.423508 secs 2024-04-27T02:29:36.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:36 smithi090 ceph-mon[29180]: pgmap v15754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:36.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28457.423508 secs 2024-04-27T02:29:36.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:36 smithi119 ceph-mon[32752]: pgmap v15754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:37.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:37 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 28458.258779 secs 2024-04-27T02:29:37.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:37 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 28458.258779 secs 2024-04-27T02:29:37.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:37 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 28458.258779 secs 2024-04-27T02:29:38.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:38 smithi167 ceph-mon[32751]: pgmap v15755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:38.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:38 smithi090 ceph-mon[29180]: pgmap v15755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:38.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:38 smithi119 ceph-mon[32752]: pgmap v15755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:39.100 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:29:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:29:38.653+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:29:39.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:29:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:29:38.653+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:29:39.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:29:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:29:38.653+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:29:39.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:29:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:29:38.653+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:29:39.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:29:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:29:38.653+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:29:40.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:40 smithi090 ceph-mon[29180]: pgmap v15756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:40.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:40 smithi119 ceph-mon[32752]: pgmap v15756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:40.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:40 smithi167 ceph-mon[32751]: pgmap v15756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:29:41.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28462.423591 secs 2024-04-27T02:29:41.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28462.423591 secs 2024-04-27T02:29:41.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28462.423591 secs 2024-04-27T02:29:42.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:42 smithi090 ceph-mon[29180]: 9 slow requests, 1 included below; oldest blocked for > 28463.258892 secs 2024-04-27T02:29:42.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:42 smithi090 ceph-mon[29180]: slow request 5763.954179 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:29:42.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:42 smithi090 ceph-mon[29180]: pgmap v15757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:42.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:42 smithi119 ceph-mon[32752]: 9 slow requests, 1 included below; oldest blocked for > 28463.258892 secs 2024-04-27T02:29:42.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:42 smithi119 ceph-mon[32752]: slow request 5763.954179 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:29:42.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:42 smithi119 ceph-mon[32752]: pgmap v15757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:42.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:42 smithi167 ceph-mon[32751]: 9 slow requests, 1 included below; oldest blocked for > 28463.258892 secs 2024-04-27T02:29:42.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:42 smithi167 ceph-mon[32751]: slow request 5763.954179 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:29:42.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:42 smithi167 ceph-mon[32751]: pgmap v15757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:43.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:43 smithi090 ceph-mon[29180]: pgmap v15758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:29:43.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:43 smithi119 ceph-mon[32752]: pgmap v15758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:29:43.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:43 smithi167 ceph-mon[32751]: pgmap v15758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-27T02:29:45.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:29:45.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:29:45.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:45 smithi090 ceph-mon[29180]: pgmap v15759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:45.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:29:45.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:29:45.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:45 smithi119 ceph-mon[32752]: pgmap v15759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:45.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:29:45.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:29:45.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:45 smithi167 ceph-mon[32751]: pgmap v15759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28467.423673 secs 2024-04-27T02:29:46.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28467.423673 secs 2024-04-27T02:29:46.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28467.423673 secs 2024-04-27T02:29:47.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:47 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28468.258993 secs 2024-04-27T02:29:47.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:47 smithi090 ceph-mon[29180]: pgmap v15760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:47.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:47 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28468.258993 secs 2024-04-27T02:29:47.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:47 smithi119 ceph-mon[32752]: pgmap v15760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:47.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:47 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28468.258993 secs 2024-04-27T02:29:47.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:47 smithi167 ceph-mon[32751]: pgmap v15760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:50.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:50 smithi119 ceph-mon[32752]: pgmap v15761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:50.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:50 smithi167 ceph-mon[32751]: pgmap v15761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:50.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:50 smithi090 ceph-mon[29180]: pgmap v15761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:51.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28472.423757 secs 2024-04-27T02:29:51.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28472.423757 secs 2024-04-27T02:29:51.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28472.423757 secs 2024-04-27T02:29:52.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:52 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28473.259134 secs 2024-04-27T02:29:52.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:52 smithi119 ceph-mon[32752]: pgmap v15762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:52.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:52 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28473.259134 secs 2024-04-27T02:29:52.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:52 smithi167 ceph-mon[32751]: pgmap v15762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:52.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:52 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28473.259134 secs 2024-04-27T02:29:52.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:52 smithi090 ceph-mon[29180]: pgmap v15762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:54.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:54 smithi119 ceph-mon[32752]: pgmap v15763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:54.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:54 smithi167 ceph-mon[32751]: pgmap v15763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:54.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:54 smithi090 ceph-mon[29180]: pgmap v15763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T02:29:56.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:56 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 28477.423841 secs 2024-04-27T02:29:56.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:56 smithi119 ceph-mon[32752]: slow request 723.194534 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:29:56.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:56 smithi119 ceph-mon[32752]: pgmap v15764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T02:29:56.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:56 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 28477.423841 secs 2024-04-27T02:29:56.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:56 smithi167 ceph-mon[32751]: slow request 723.194534 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:29:56.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:56 smithi167 ceph-mon[32751]: pgmap v15764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T02:29:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:56 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 28477.423841 secs 2024-04-27T02:29:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:56 smithi090 ceph-mon[29180]: slow request 723.194534 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:29:56.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:56 smithi090 ceph-mon[29180]: pgmap v15764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T02:29:57.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:57 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28478.259207 secs 2024-04-27T02:29:57.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:57 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28478.259207 secs 2024-04-27T02:29:57.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:57 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28478.259207 secs 2024-04-27T02:29:58.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:29:58 smithi119 ceph-mon[32752]: pgmap v15765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:58.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:29:58 smithi167 ceph-mon[32751]: pgmap v15765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:29:58.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:29:58 smithi090 ceph-mon[29180]: pgmap v15765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:00.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:00 smithi119 ceph-mon[32752]: pgmap v15766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:00.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:30:00.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:00 smithi167 ceph-mon[32751]: pgmap v15766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:00.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:30:00.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:00 smithi090 ceph-mon[29180]: pgmap v15766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:00.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:30:01.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28482.423930 secs 2024-04-27T02:30:01.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28482.423930 secs 2024-04-27T02:30:01.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28482.423930 secs 2024-04-27T02:30:02.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:02 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28483.259324 secs 2024-04-27T02:30:02.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:02 smithi167 ceph-mon[32751]: pgmap v15767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:02.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:02 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28483.259324 secs 2024-04-27T02:30:02.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:02 smithi090 ceph-mon[29180]: pgmap v15767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:02.679 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:02 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28483.259324 secs 2024-04-27T02:30:02.680 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:02 smithi119 ceph-mon[32752]: pgmap v15767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:04.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:04 smithi167 ceph-mon[32751]: pgmap v15768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:30:04.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:04 smithi090 ceph-mon[29180]: pgmap v15768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:30:04.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:04 smithi119 ceph-mon[32752]: pgmap v15768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:30:06.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28487.424012 secs 2024-04-27T02:30:06.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:06 smithi167 ceph-mon[32751]: pgmap v15769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:06.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28487.424012 secs 2024-04-27T02:30:06.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:06 smithi090 ceph-mon[29180]: pgmap v15769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:06.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28487.424012 secs 2024-04-27T02:30:06.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:06 smithi119 ceph-mon[32752]: pgmap v15769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:07.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:07 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28488.259453 secs 2024-04-27T02:30:07.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:07 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28488.259453 secs 2024-04-27T02:30:07.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:07 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28488.259453 secs 2024-04-27T02:30:08.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:08 smithi090 ceph-mon[29180]: pgmap v15770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:30:08.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:08 smithi119 ceph-mon[32752]: pgmap v15770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:30:08.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:08 smithi167 ceph-mon[32751]: pgmap v15770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:30:09.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:30:09.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:30:09.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:30:10.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:10 smithi090 ceph-mon[29180]: pgmap v15771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:30:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:30:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:10 smithi090 ceph-mon[29180]: pgmap v15772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:10 smithi090 ceph-mon[29180]: pgmap v15773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:30:10.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:10 smithi119 ceph-mon[32752]: pgmap v15771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:30:10.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:30:10.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:10 smithi119 ceph-mon[32752]: pgmap v15772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:10 smithi119 ceph-mon[32752]: pgmap v15773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:30:10.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:10 smithi167 ceph-mon[32751]: pgmap v15771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:30:10.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:30:10.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:10 smithi167 ceph-mon[32751]: pgmap v15772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:10 smithi167 ceph-mon[32751]: pgmap v15773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:10.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:30:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28492.424100 secs 2024-04-27T02:30:11.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28492.424100 secs 2024-04-27T02:30:11.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28492.424100 secs 2024-04-27T02:30:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:12 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28493.259541 secs 2024-04-27T02:30:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:12 smithi090 ceph-mon[29180]: pgmap v15774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:12 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28493.259541 secs 2024-04-27T02:30:12.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:12 smithi119 ceph-mon[32752]: pgmap v15774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:12 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28493.259541 secs 2024-04-27T02:30:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:12 smithi167 ceph-mon[32751]: pgmap v15774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:30:15.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:15 smithi119 ceph-mon[32752]: pgmap v15775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:30:15.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:15 smithi167 ceph-mon[32751]: pgmap v15775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:30:15.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:15 smithi090 ceph-mon[29180]: pgmap v15775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:30:16.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28497.424186 secs 2024-04-27T02:30:16.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28497.424186 secs 2024-04-27T02:30:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28497.424186 secs 2024-04-27T02:30:17.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:17 smithi167 ceph-mon[32751]: pgmap v15776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:30:17.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:17 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28498.259612 secs 2024-04-27T02:30:17.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:17 smithi090 ceph-mon[29180]: pgmap v15776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:30:17.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:17 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28498.259612 secs 2024-04-27T02:30:17.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:17 smithi119 ceph-mon[32752]: pgmap v15776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:30:17.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:17 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28498.259612 secs 2024-04-27T02:30:19.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:19 smithi167 ceph-mon[32751]: pgmap v15777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:30:19.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:19 smithi090 ceph-mon[29180]: pgmap v15777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:30:19.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:19 smithi119 ceph-mon[32752]: pgmap v15777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:30:21.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:21 smithi167 ceph-mon[32751]: pgmap v15778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:30:21.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28502.424264 secs 2024-04-27T02:30:21.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:21 smithi090 ceph-mon[29180]: pgmap v15778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:30:21.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28502.424264 secs 2024-04-27T02:30:21.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:21 smithi119 ceph-mon[32752]: pgmap v15778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:30:21.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28502.424264 secs 2024-04-27T02:30:22.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:22 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28503.259728 secs 2024-04-27T02:30:22.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:22 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28503.259728 secs 2024-04-27T02:30:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:22 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28503.259728 secs 2024-04-27T02:30:23.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:23 smithi090 ceph-mon[29180]: pgmap v15779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:23.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:23 smithi119 ceph-mon[32752]: pgmap v15779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:23.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:23 smithi167 ceph-mon[32751]: pgmap v15779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:25.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:25 smithi090 ceph-mon[29180]: pgmap v15780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:25.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:25 smithi119 ceph-mon[32752]: pgmap v15780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:25.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:25 smithi167 ceph-mon[32751]: pgmap v15780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:26.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28507.424356 secs 2024-04-27T02:30:26.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28507.424356 secs 2024-04-27T02:30:26.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28507.424356 secs 2024-04-27T02:30:27.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:27 smithi090 ceph-mon[29180]: pgmap v15781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:27.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:27 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28508.259799 secs 2024-04-27T02:30:27.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:27 smithi119 ceph-mon[32752]: pgmap v15781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:27.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:27 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28508.259799 secs 2024-04-27T02:30:27.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:27 smithi167 ceph-mon[32751]: pgmap v15781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:27.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:27 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28508.259799 secs 2024-04-27T02:30:29.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:29 smithi090 ceph-mon[29180]: pgmap v15782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:29.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:29 smithi119 ceph-mon[32752]: pgmap v15782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:29.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:29 smithi167 ceph-mon[32751]: pgmap v15782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:31.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:31 smithi090 ceph-mon[29180]: pgmap v15783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:31.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28512.424441 secs 2024-04-27T02:30:31.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:31 smithi119 ceph-mon[32752]: pgmap v15783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:31.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28512.424441 secs 2024-04-27T02:30:31.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:31 smithi167 ceph-mon[32751]: pgmap v15783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:31.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28512.424441 secs 2024-04-27T02:30:32.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:32 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28513.259916 secs 2024-04-27T02:30:32.429 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:32 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28513.259916 secs 2024-04-27T02:30:32.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:32 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28513.259916 secs 2024-04-27T02:30:33.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:33 smithi090 ceph-mon[29180]: pgmap v15784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:33.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:33 smithi119 ceph-mon[32752]: pgmap v15784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:33.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:33 smithi167 ceph-mon[32751]: pgmap v15784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:35.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:35 smithi090 ceph-mon[29180]: pgmap v15785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:35.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:35 smithi119 ceph-mon[32752]: pgmap v15785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:35.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:35 smithi167 ceph-mon[32751]: pgmap v15785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:36.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28517.424524 secs 2024-04-27T02:30:36.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28517.424524 secs 2024-04-27T02:30:36.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28517.424524 secs 2024-04-27T02:30:37.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:37 smithi090 ceph-mon[29180]: pgmap v15786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:37.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:37 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28518.260036 secs 2024-04-27T02:30:37.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:37 smithi119 ceph-mon[32752]: pgmap v15786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:37.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:37 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28518.260036 secs 2024-04-27T02:30:37.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:37 smithi167 ceph-mon[32751]: pgmap v15786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:37.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:37 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28518.260036 secs 2024-04-27T02:30:39.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:39 smithi090 ceph-mon[29180]: pgmap v15787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:39.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:39 smithi119 ceph-mon[32752]: pgmap v15787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:39.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:39 smithi167 ceph-mon[32751]: pgmap v15787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:41.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:41 smithi090 ceph-mon[29180]: pgmap v15788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:41.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28522.424604 secs 2024-04-27T02:30:41.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:41 smithi119 ceph-mon[32752]: pgmap v15788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:41.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28522.424604 secs 2024-04-27T02:30:41.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:41 smithi167 ceph-mon[32751]: pgmap v15788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:41.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28522.424604 secs 2024-04-27T02:30:42.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:42 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28523.260179 secs 2024-04-27T02:30:42.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:42 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28523.260179 secs 2024-04-27T02:30:42.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:42 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28523.260179 secs 2024-04-27T02:30:43.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:43 smithi090 ceph-mon[29180]: pgmap v15789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:43.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:43 smithi119 ceph-mon[32752]: pgmap v15789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:43.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:43 smithi167 ceph-mon[32751]: pgmap v15789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:45.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:45 smithi090 ceph-mon[29180]: pgmap v15790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:45.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:45 smithi119 ceph-mon[32752]: pgmap v15790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:45.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:45 smithi167 ceph-mon[32751]: pgmap v15790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:46.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28527.424685 secs 2024-04-27T02:30:46.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28527.424685 secs 2024-04-27T02:30:46.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28527.424685 secs 2024-04-27T02:30:47.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:47 smithi119 ceph-mon[32752]: pgmap v15791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:47.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:47 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28528.260296 secs 2024-04-27T02:30:47.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:47 smithi167 ceph-mon[32751]: pgmap v15791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:47.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:47 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28528.260296 secs 2024-04-27T02:30:47.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:47 smithi090 ceph-mon[29180]: pgmap v15791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:47.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:47 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28528.260296 secs 2024-04-27T02:30:49.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:49 smithi119 ceph-mon[32752]: pgmap v15792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:49.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:49 smithi167 ceph-mon[32751]: pgmap v15792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:49.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:49 smithi090 ceph-mon[29180]: pgmap v15792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:51.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:51 smithi119 ceph-mon[32752]: pgmap v15793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:51.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28532.424771 secs 2024-04-27T02:30:51.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:51 smithi167 ceph-mon[32751]: pgmap v15793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:51.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28532.424771 secs 2024-04-27T02:30:51.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:51 smithi090 ceph-mon[29180]: pgmap v15793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:51.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28532.424771 secs 2024-04-27T02:30:52.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:52 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28533.260409 secs 2024-04-27T02:30:52.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:52 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28533.260409 secs 2024-04-27T02:30:52.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:52 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28533.260409 secs 2024-04-27T02:30:53.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:53 smithi119 ceph-mon[32752]: pgmap v15794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:53.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:53 smithi167 ceph-mon[32751]: pgmap v15794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:53.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:53 smithi090 ceph-mon[29180]: pgmap v15794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:55.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:55 smithi119 ceph-mon[32752]: pgmap v15795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:55.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:55 smithi167 ceph-mon[32751]: pgmap v15795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:55.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:55 smithi090 ceph-mon[29180]: pgmap v15795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:30:56.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28537.424852 secs 2024-04-27T02:30:56.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28537.424852 secs 2024-04-27T02:30:56.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28537.424852 secs 2024-04-27T02:30:57.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:57 smithi119 ceph-mon[32752]: pgmap v15796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:57.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:57 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28538.260476 secs 2024-04-27T02:30:57.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:57 smithi167 ceph-mon[32751]: pgmap v15796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:57.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:57 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28538.260476 secs 2024-04-27T02:30:57.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:57 smithi090 ceph-mon[29180]: pgmap v15796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:57.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:57 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28538.260476 secs 2024-04-27T02:30:59.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:30:59 smithi119 ceph-mon[32752]: pgmap v15797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:59.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:30:59 smithi167 ceph-mon[32751]: pgmap v15797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:30:59.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:30:59 smithi090 ceph-mon[29180]: pgmap v15797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:01.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:01 smithi119 ceph-mon[32752]: pgmap v15798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:01.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28542.424942 secs 2024-04-27T02:31:01.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:01 smithi167 ceph-mon[32751]: pgmap v15798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:01.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28542.424942 secs 2024-04-27T02:31:01.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:01 smithi090 ceph-mon[29180]: pgmap v15798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:01.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28542.424942 secs 2024-04-27T02:31:02.429 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:02 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28543.260558 secs 2024-04-27T02:31:02.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:02 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28543.260558 secs 2024-04-27T02:31:02.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:02 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28543.260558 secs 2024-04-27T02:31:03.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:03 smithi119 ceph-mon[32752]: pgmap v15799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:03.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:03 smithi167 ceph-mon[32751]: pgmap v15799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:03.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:03 smithi090 ceph-mon[29180]: pgmap v15799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:05.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:05 smithi119 ceph-mon[32752]: pgmap v15800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:05.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:05 smithi167 ceph-mon[32751]: pgmap v15800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:05.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:05 smithi090 ceph-mon[29180]: pgmap v15800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:06.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28547.425002 secs 2024-04-27T02:31:06.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28547.425002 secs 2024-04-27T02:31:06.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28547.425002 secs 2024-04-27T02:31:07.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:07 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28548.260688 secs 2024-04-27T02:31:07.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:07 smithi119 ceph-mon[32752]: pgmap v15801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:07.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:07 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28548.260688 secs 2024-04-27T02:31:07.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:07 smithi167 ceph-mon[32751]: pgmap v15801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:07.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:07 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28548.260688 secs 2024-04-27T02:31:07.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:07 smithi090 ceph-mon[29180]: pgmap v15801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:09.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:09 smithi119 ceph-mon[32752]: pgmap v15802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:09.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:09 smithi167 ceph-mon[32751]: pgmap v15802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:09.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:09 smithi090 ceph-mon[29180]: pgmap v15802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:10.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:31:10.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:31:10.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:31:11.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:11 smithi119 ceph-mon[32752]: pgmap v15803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:11.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28552.425073 secs 2024-04-27T02:31:11.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:31:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:31:11.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:31:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:11 smithi167 ceph-mon[32751]: pgmap v15803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28552.425073 secs 2024-04-27T02:31:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:31:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:31:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:31:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:11 smithi090 ceph-mon[29180]: pgmap v15803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28552.425073 secs 2024-04-27T02:31:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:31:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:31:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:31:12.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:12 smithi119 ceph-mon[32752]: pgmap v15804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:31:12.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:12 smithi119 ceph-mon[32752]: pgmap v15805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:12 smithi119 ceph-mon[32752]: 9 slow requests, 1 included below; oldest blocked for > 28553.260797 secs 2024-04-27T02:31:12.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:12 smithi119 ceph-mon[32752]: slow request 722.531381 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:31:12.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:12 smithi167 ceph-mon[32751]: pgmap v15804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:31:12.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:12 smithi167 ceph-mon[32751]: pgmap v15805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:12.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:12 smithi167 ceph-mon[32751]: 9 slow requests, 1 included below; oldest blocked for > 28553.260797 secs 2024-04-27T02:31:12.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:12 smithi167 ceph-mon[32751]: slow request 722.531381 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:31:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:12 smithi090 ceph-mon[29180]: pgmap v15804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:31:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:12 smithi090 ceph-mon[29180]: pgmap v15805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:12 smithi090 ceph-mon[29180]: 9 slow requests, 1 included below; oldest blocked for > 28553.260797 secs 2024-04-27T02:31:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:12 smithi090 ceph-mon[29180]: slow request 722.531381 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:31:14.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:14 smithi119 ceph-mon[32752]: pgmap v15806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:31:14.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:14 smithi167 ceph-mon[32751]: pgmap v15806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:31:14.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:14 smithi090 ceph-mon[29180]: pgmap v15806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:31:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:16 smithi119 ceph-mon[32752]: pgmap v15807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28557.425157 secs 2024-04-27T02:31:16.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:16 smithi167 ceph-mon[32751]: pgmap v15807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:16.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28557.425157 secs 2024-04-27T02:31:16.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:16 smithi090 ceph-mon[29180]: pgmap v15807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:16.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28557.425157 secs 2024-04-27T02:31:17.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:17 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28558.260912 secs 2024-04-27T02:31:17.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:17 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28558.260912 secs 2024-04-27T02:31:17.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:17 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28558.260912 secs 2024-04-27T02:31:18.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:18 smithi119 ceph-mon[32752]: pgmap v15808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:18.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:18 smithi167 ceph-mon[32751]: pgmap v15808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:18.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:18 smithi090 ceph-mon[29180]: pgmap v15808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:20.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:20 smithi119 ceph-mon[32752]: pgmap v15809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:20.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:20 smithi167 ceph-mon[32751]: pgmap v15809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:20.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:20 smithi090 ceph-mon[29180]: pgmap v15809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:31:21.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28562.425242 secs 2024-04-27T02:31:21.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28562.425242 secs 2024-04-27T02:31:21.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28562.425242 secs 2024-04-27T02:31:22.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:22 smithi119 ceph-mon[32752]: pgmap v15810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:31:22.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:22 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28563.261011 secs 2024-04-27T02:31:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:22 smithi167 ceph-mon[32751]: pgmap v15810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:31:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:22 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28563.261011 secs 2024-04-27T02:31:22.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:22 smithi090 ceph-mon[29180]: pgmap v15810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:31:22.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:22 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28563.261011 secs 2024-04-27T02:31:24.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:24 smithi119 ceph-mon[32752]: pgmap v15811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:24.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:24 smithi167 ceph-mon[32751]: pgmap v15811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:24.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:24 smithi090 ceph-mon[29180]: pgmap v15811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:26.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:26 smithi119 ceph-mon[32752]: pgmap v15812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:26.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28567.425326 secs 2024-04-27T02:31:26.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:26 smithi167 ceph-mon[32751]: pgmap v15812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:26.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28567.425326 secs 2024-04-27T02:31:26.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:26 smithi090 ceph-mon[29180]: pgmap v15812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:26.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28567.425326 secs 2024-04-27T02:31:27.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:27 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28568.261102 secs 2024-04-27T02:31:27.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:27 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28568.261102 secs 2024-04-27T02:31:27.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:27 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28568.261102 secs 2024-04-27T02:31:28.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:28 smithi119 ceph-mon[32752]: pgmap v15813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:28.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:28 smithi167 ceph-mon[32751]: pgmap v15813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:28.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:28 smithi090 ceph-mon[29180]: pgmap v15813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:30.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:30 smithi119 ceph-mon[32752]: pgmap v15814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:30.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:30 smithi167 ceph-mon[32751]: pgmap v15814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:30.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:30 smithi090 ceph-mon[29180]: pgmap v15814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:31.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28572.425412 secs 2024-04-27T02:31:31.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28572.425412 secs 2024-04-27T02:31:31.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28572.425412 secs 2024-04-27T02:31:32.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:32 smithi090 ceph-mon[29180]: pgmap v15815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:32.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:32 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28573.261235 secs 2024-04-27T02:31:32.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:32 smithi119 ceph-mon[32752]: pgmap v15815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:32.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:32 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28573.261235 secs 2024-04-27T02:31:32.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:32 smithi167 ceph-mon[32751]: pgmap v15815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:32.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:32 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28573.261235 secs 2024-04-27T02:31:34.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:34 smithi119 ceph-mon[32752]: pgmap v15816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:34.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:34 smithi167 ceph-mon[32751]: pgmap v15816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:34.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:34 smithi090 ceph-mon[29180]: pgmap v15816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:36.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:36 smithi119 ceph-mon[32752]: pgmap v15817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:36.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28577.425495 secs 2024-04-27T02:31:36.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:36 smithi167 ceph-mon[32751]: pgmap v15817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:36.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28577.425495 secs 2024-04-27T02:31:36.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:36 smithi090 ceph-mon[29180]: pgmap v15817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:36.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28577.425495 secs 2024-04-27T02:31:37.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:37 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28578.261345 secs 2024-04-27T02:31:37.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:37 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28578.261345 secs 2024-04-27T02:31:37.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:37 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28578.261345 secs 2024-04-27T02:31:38.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:38 smithi119 ceph-mon[32752]: pgmap v15818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:38.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:38 smithi167 ceph-mon[32751]: pgmap v15818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:38.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:38 smithi090 ceph-mon[29180]: pgmap v15818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:40.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:40 smithi119 ceph-mon[32752]: pgmap v15819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:40.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:40 smithi167 ceph-mon[32751]: pgmap v15819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:40.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:40 smithi090 ceph-mon[29180]: pgmap v15819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:41.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28582.425580 secs 2024-04-27T02:31:41.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28582.425580 secs 2024-04-27T02:31:41.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28582.425580 secs 2024-04-27T02:31:42.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:42 smithi119 ceph-mon[32752]: pgmap v15820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:42.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:42 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28583.261477 secs 2024-04-27T02:31:42.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:42 smithi167 ceph-mon[32751]: pgmap v15820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:42.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:42 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28583.261477 secs 2024-04-27T02:31:42.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:42 smithi090 ceph-mon[29180]: pgmap v15820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:42.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:42 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28583.261477 secs 2024-04-27T02:31:44.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:44 smithi119 ceph-mon[32752]: pgmap v15821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:44.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:44 smithi167 ceph-mon[32751]: pgmap v15821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:44.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:44 smithi090 ceph-mon[29180]: pgmap v15821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:46.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:46 smithi119 ceph-mon[32752]: pgmap v15822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:46.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28587.425662 secs 2024-04-27T02:31:46.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:46 smithi167 ceph-mon[32751]: pgmap v15822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:46.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28587.425662 secs 2024-04-27T02:31:46.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:46 smithi090 ceph-mon[29180]: pgmap v15822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:46.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28587.425662 secs 2024-04-27T02:31:47.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:47 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28588.261558 secs 2024-04-27T02:31:47.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:47 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28588.261558 secs 2024-04-27T02:31:47.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:47 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28588.261558 secs 2024-04-27T02:31:48.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:48 smithi119 ceph-mon[32752]: pgmap v15823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:48.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:48 smithi167 ceph-mon[32751]: pgmap v15823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:48.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:48 smithi090 ceph-mon[29180]: pgmap v15823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:50.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:50 smithi119 ceph-mon[32752]: pgmap v15824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:50.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:50 smithi167 ceph-mon[32751]: pgmap v15824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:50.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:50 smithi090 ceph-mon[29180]: pgmap v15824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:51.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28592.425748 secs 2024-04-27T02:31:51.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28592.425748 secs 2024-04-27T02:31:51.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28592.425748 secs 2024-04-27T02:31:52.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:52 smithi119 ceph-mon[32752]: pgmap v15825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:52.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:52 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28593.261618 secs 2024-04-27T02:31:52.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:52 smithi167 ceph-mon[32751]: pgmap v15825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:52.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:52 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28593.261618 secs 2024-04-27T02:31:52.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:52 smithi090 ceph-mon[29180]: pgmap v15825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:52.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:52 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28593.261618 secs 2024-04-27T02:31:54.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:54 smithi119 ceph-mon[32752]: pgmap v15826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:54.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:54 smithi167 ceph-mon[32751]: pgmap v15826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:54.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:54 smithi090 ceph-mon[29180]: pgmap v15826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:56.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:56 smithi167 ceph-mon[32751]: pgmap v15827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:56.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28597.425826 secs 2024-04-27T02:31:56.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:56 smithi090 ceph-mon[29180]: pgmap v15827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:56.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28597.425826 secs 2024-04-27T02:31:56.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:56 smithi119 ceph-mon[32752]: pgmap v15827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:31:56.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28597.425826 secs 2024-04-27T02:31:57.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:57 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28598.261743 secs 2024-04-27T02:31:57.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:57 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28598.261743 secs 2024-04-27T02:31:57.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:57 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28598.261743 secs 2024-04-27T02:31:58.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:31:58 smithi167 ceph-mon[32751]: pgmap v15828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:58.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:31:58 smithi090 ceph-mon[29180]: pgmap v15828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:31:58.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:31:58 smithi119 ceph-mon[32752]: pgmap v15828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:00.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:00 smithi167 ceph-mon[32751]: pgmap v15829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:00.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:00 smithi090 ceph-mon[29180]: pgmap v15829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:00.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:00 smithi119 ceph-mon[32752]: pgmap v15829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:01.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28602.425888 secs 2024-04-27T02:32:01.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28602.425888 secs 2024-04-27T02:32:01.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28602.425888 secs 2024-04-27T02:32:02.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:02 smithi090 ceph-mon[29180]: pgmap v15830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:02.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:02 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28603.261881 secs 2024-04-27T02:32:02.679 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:02 smithi119 ceph-mon[32752]: pgmap v15830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:02.679 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:02 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28603.261881 secs 2024-04-27T02:32:02.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:02 smithi167 ceph-mon[32751]: pgmap v15830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:02.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:02 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28603.261881 secs 2024-04-27T02:32:04.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:04 smithi090 ceph-mon[29180]: pgmap v15831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:04.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:04 smithi119 ceph-mon[32752]: pgmap v15831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:04.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:04 smithi167 ceph-mon[32751]: pgmap v15831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:06.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:06 smithi090 ceph-mon[29180]: pgmap v15832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:06.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28607.425949 secs 2024-04-27T02:32:06.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:06 smithi119 ceph-mon[32752]: pgmap v15832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:06.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28607.425949 secs 2024-04-27T02:32:06.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:06 smithi167 ceph-mon[32751]: pgmap v15832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:06.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28607.425949 secs 2024-04-27T02:32:07.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:07 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28608.261980 secs 2024-04-27T02:32:07.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:07 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28608.261980 secs 2024-04-27T02:32:07.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:07 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28608.261980 secs 2024-04-27T02:32:08.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:08 smithi090 ceph-mon[29180]: pgmap v15833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:08.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:08 smithi119 ceph-mon[32752]: pgmap v15833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:08.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:08 smithi167 ceph-mon[32751]: pgmap v15833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:10.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:10 smithi090 ceph-mon[29180]: pgmap v15834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:10.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:10 smithi119 ceph-mon[32752]: pgmap v15834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:10.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:10 smithi167 ceph-mon[32751]: pgmap v15834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28612.426012 secs 2024-04-27T02:32:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:11 smithi090 ceph-mon[29180]: pgmap v15835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:32:11.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28612.426012 secs 2024-04-27T02:32:11.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:11 smithi119 ceph-mon[32752]: pgmap v15835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:11.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:32:11.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28612.426012 secs 2024-04-27T02:32:11.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:11 smithi167 ceph-mon[32751]: pgmap v15835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:11.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:32:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:12 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28613.262074 secs 2024-04-27T02:32:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:32:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:32:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:12 smithi090 ceph-mon[29180]: pgmap v15836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:32:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:12 smithi090 ceph-mon[29180]: pgmap v15837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:12.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:32:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:12 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28613.262074 secs 2024-04-27T02:32:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:32:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:32:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:12 smithi119 ceph-mon[32752]: pgmap v15836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:32:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:12 smithi119 ceph-mon[32752]: pgmap v15837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:12.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:32:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:12 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28613.262074 secs 2024-04-27T02:32:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:32:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:32:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:12 smithi167 ceph-mon[32751]: pgmap v15836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:32:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:12 smithi167 ceph-mon[32751]: pgmap v15837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:12.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:32:14.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:14 smithi090 ceph-mon[29180]: pgmap v15838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:15.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:14 smithi119 ceph-mon[32752]: pgmap v15838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:15.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:14 smithi167 ceph-mon[32751]: pgmap v15838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:16.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28617.426074 secs 2024-04-27T02:32:16.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:16 smithi090 ceph-mon[29180]: pgmap v15839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:16.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:16 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28618.262189 secs 2024-04-27T02:32:17.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28617.426074 secs 2024-04-27T02:32:17.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:16 smithi119 ceph-mon[32752]: pgmap v15839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:17.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:16 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28618.262189 secs 2024-04-27T02:32:17.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28617.426074 secs 2024-04-27T02:32:17.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:16 smithi167 ceph-mon[32751]: pgmap v15839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:17.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:16 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28618.262189 secs 2024-04-27T02:32:18.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:18 smithi090 ceph-mon[29180]: pgmap v15840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:32:19.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:18 smithi119 ceph-mon[32752]: pgmap v15840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:32:19.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:18 smithi167 ceph-mon[32751]: pgmap v15840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:32:20.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:20 smithi090 ceph-mon[29180]: pgmap v15841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:21.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:20 smithi119 ceph-mon[32752]: pgmap v15841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:21.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:20 smithi167 ceph-mon[32751]: pgmap v15841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:32:22.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28622.426131 secs 2024-04-27T02:32:22.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:21 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28623.262296 secs 2024-04-27T02:32:22.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28622.426131 secs 2024-04-27T02:32:22.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:21 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28623.262296 secs 2024-04-27T02:32:22.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28622.426131 secs 2024-04-27T02:32:22.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:21 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28623.262296 secs 2024-04-27T02:32:23.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:22 smithi119 ceph-mon[32752]: pgmap v15842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:32:23.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:22 smithi167 ceph-mon[32751]: pgmap v15842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:32:23.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:22 smithi090 ceph-mon[29180]: pgmap v15842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:32:25.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:24 smithi119 ceph-mon[32752]: pgmap v15843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:25.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:24 smithi167 ceph-mon[32751]: pgmap v15843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:25.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:24 smithi090 ceph-mon[29180]: pgmap v15843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:27.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28627.426216 secs 2024-04-27T02:32:27.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:26 smithi119 ceph-mon[32752]: pgmap v15844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:27.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:26 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28628.262422 secs 2024-04-27T02:32:27.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28627.426216 secs 2024-04-27T02:32:27.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:26 smithi167 ceph-mon[32751]: pgmap v15844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:27.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:26 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28628.262422 secs 2024-04-27T02:32:27.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28627.426216 secs 2024-04-27T02:32:27.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:26 smithi090 ceph-mon[29180]: pgmap v15844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:27.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:26 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28628.262422 secs 2024-04-27T02:32:29.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:28 smithi119 ceph-mon[32752]: pgmap v15845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:29.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:28 smithi167 ceph-mon[32751]: pgmap v15845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:29.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:28 smithi090 ceph-mon[29180]: pgmap v15845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:31.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:30 smithi119 ceph-mon[32752]: pgmap v15846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:31.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:30 smithi167 ceph-mon[32751]: pgmap v15846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:31.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:30 smithi090 ceph-mon[29180]: pgmap v15846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:31.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28632.426298 secs 2024-04-27T02:32:31.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:31 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28633.262537 secs 2024-04-27T02:32:32.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28632.426298 secs 2024-04-27T02:32:32.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:31 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28633.262537 secs 2024-04-27T02:32:32.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28632.426298 secs 2024-04-27T02:32:32.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:31 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28633.262537 secs 2024-04-27T02:32:32.929 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:32 smithi119 ceph-mon[32752]: pgmap v15847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:32.943 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:32 smithi167 ceph-mon[32751]: pgmap v15847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:33.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:32 smithi090 ceph-mon[29180]: pgmap v15847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:32:35.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:34 smithi119 ceph-mon[32752]: pgmap v15848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:35.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:34 smithi167 ceph-mon[32751]: pgmap v15848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:35.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:34 smithi090 ceph-mon[29180]: pgmap v15848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:37.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28637.426384 secs 2024-04-27T02:32:37.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:36 smithi119 ceph-mon[32752]: pgmap v15849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:37.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:36 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 28638.262668 secs 2024-04-27T02:32:37.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28637.426384 secs 2024-04-27T02:32:37.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:36 smithi167 ceph-mon[32751]: pgmap v15849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:37.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:36 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 28638.262668 secs 2024-04-27T02:32:37.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28637.426384 secs 2024-04-27T02:32:37.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:36 smithi090 ceph-mon[29180]: pgmap v15849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:37.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:36 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 28638.262668 secs 2024-04-27T02:32:39.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:38 smithi119 ceph-mon[32752]: pgmap v15850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:39.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:38 smithi167 ceph-mon[32751]: pgmap v15850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:39.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:38 smithi090 ceph-mon[29180]: pgmap v15850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:41.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:40 smithi119 ceph-mon[32752]: pgmap v15851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:41.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:40 smithi167 ceph-mon[32751]: pgmap v15851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:41.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:40 smithi090 ceph-mon[29180]: pgmap v15851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:42.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28642.426443 secs 2024-04-27T02:32:42.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:41 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 28643.262781 secs 2024-04-27T02:32:42.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:41 smithi119 ceph-mon[32752]: slow request 182.353905 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:32:42.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28642.426443 secs 2024-04-27T02:32:42.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:41 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 28643.262781 secs 2024-04-27T02:32:42.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:41 smithi167 ceph-mon[32751]: slow request 182.353905 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:32:42.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28642.426443 secs 2024-04-27T02:32:42.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:41 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 28643.262781 secs 2024-04-27T02:32:42.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:41 smithi090 ceph-mon[29180]: slow request 182.353905 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:32:43.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:42 smithi119 ceph-mon[32752]: pgmap v15852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:43.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:42 smithi167 ceph-mon[32751]: pgmap v15852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:43.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:42 smithi090 ceph-mon[29180]: pgmap v15852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:45.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:44 smithi119 ceph-mon[32752]: pgmap v15853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:45.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:44 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:32:45.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:44 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:32:45.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:44 smithi167 ceph-mon[32751]: pgmap v15853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:45.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:44 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:32:45.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:44 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:32:45.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:44 smithi090 ceph-mon[29180]: pgmap v15853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:45.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:44 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:32:45.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:44 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:32:47.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28647.426529 secs 2024-04-27T02:32:47.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:46 smithi119 ceph-mon[32752]: pgmap v15854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:47.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28648.262919 secs 2024-04-27T02:32:47.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28647.426529 secs 2024-04-27T02:32:47.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:46 smithi167 ceph-mon[32751]: pgmap v15854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:47.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28648.262919 secs 2024-04-27T02:32:47.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28647.426529 secs 2024-04-27T02:32:47.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:46 smithi090 ceph-mon[29180]: pgmap v15854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:47.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28648.262919 secs 2024-04-27T02:32:49.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:48 smithi119 ceph-mon[32752]: pgmap v15855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:49.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:48 smithi167 ceph-mon[32751]: pgmap v15855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:49.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:48 smithi090 ceph-mon[29180]: pgmap v15855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:51.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:50 smithi119 ceph-mon[32752]: pgmap v15856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:51.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:50 smithi167 ceph-mon[32751]: pgmap v15856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:51.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:50 smithi090 ceph-mon[29180]: pgmap v15856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:52.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28652.426614 secs 2024-04-27T02:32:52.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28653.263035 secs 2024-04-27T02:32:52.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28652.426614 secs 2024-04-27T02:32:52.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28653.263035 secs 2024-04-27T02:32:52.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28652.426614 secs 2024-04-27T02:32:52.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28653.263035 secs 2024-04-27T02:32:53.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:52 smithi119 ceph-mon[32752]: pgmap v15857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:53.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:52 smithi167 ceph-mon[32751]: pgmap v15857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:53.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:52 smithi090 ceph-mon[29180]: pgmap v15857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:55.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:54 smithi119 ceph-mon[32752]: pgmap v15858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:55.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:54 smithi167 ceph-mon[32751]: pgmap v15858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:55.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:54 smithi090 ceph-mon[29180]: pgmap v15858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:32:57.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28657.426700 secs 2024-04-27T02:32:57.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:56 smithi119 ceph-mon[32752]: pgmap v15859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:57.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28658.263131 secs 2024-04-27T02:32:57.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28657.426700 secs 2024-04-27T02:32:57.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:56 smithi167 ceph-mon[32751]: pgmap v15859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:57.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28658.263131 secs 2024-04-27T02:32:57.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28657.426700 secs 2024-04-27T02:32:57.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:56 smithi090 ceph-mon[29180]: pgmap v15859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:57.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28658.263131 secs 2024-04-27T02:32:59.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:32:58 smithi119 ceph-mon[32752]: pgmap v15860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:59.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:32:58 smithi167 ceph-mon[32751]: pgmap v15860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:32:59.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:32:58 smithi090 ceph-mon[29180]: pgmap v15860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:01.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:00 smithi119 ceph-mon[32752]: pgmap v15861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:01.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:00 smithi167 ceph-mon[32751]: pgmap v15861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:01.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:00 smithi090 ceph-mon[29180]: pgmap v15861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:02.000 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28662.426793 secs 2024-04-27T02:33:02.001 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28663.263243 secs 2024-04-27T02:33:02.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28662.426793 secs 2024-04-27T02:33:02.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28663.263243 secs 2024-04-27T02:33:02.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28662.426793 secs 2024-04-27T02:33:02.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28663.263243 secs 2024-04-27T02:33:02.943 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:02 smithi167 ceph-mon[32751]: pgmap v15862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:03.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:02 smithi119 ceph-mon[32752]: pgmap v15862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:03.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:02 smithi090 ceph-mon[29180]: pgmap v15862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:05.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:04 smithi119 ceph-mon[32752]: pgmap v15863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:05.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:04 smithi167 ceph-mon[32751]: pgmap v15863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:05.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:04 smithi090 ceph-mon[29180]: pgmap v15863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:07.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28667.426875 secs 2024-04-27T02:33:07.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:06 smithi119 ceph-mon[32752]: pgmap v15864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:07.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28668.263401 secs 2024-04-27T02:33:07.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28667.426875 secs 2024-04-27T02:33:07.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:06 smithi167 ceph-mon[32751]: pgmap v15864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:07.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28668.263401 secs 2024-04-27T02:33:07.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28667.426875 secs 2024-04-27T02:33:07.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:06 smithi090 ceph-mon[29180]: pgmap v15864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:07.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28668.263401 secs 2024-04-27T02:33:09.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:08 smithi119 ceph-mon[32752]: pgmap v15865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:09.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:08 smithi167 ceph-mon[32751]: pgmap v15865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:09.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:08 smithi090 ceph-mon[29180]: pgmap v15865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:11.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:10 smithi119 ceph-mon[32752]: pgmap v15866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:11.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:10 smithi167 ceph-mon[32751]: pgmap v15866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:11.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:10 smithi090 ceph-mon[29180]: pgmap v15866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:12.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28672.426936 secs 2024-04-27T02:33:12.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:11 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28673.263508 secs 2024-04-27T02:33:12.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:33:12.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28672.426936 secs 2024-04-27T02:33:12.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:11 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28673.263508 secs 2024-04-27T02:33:12.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:33:12.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28672.426936 secs 2024-04-27T02:33:12.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:11 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28673.263508 secs 2024-04-27T02:33:12.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:33:13.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:13 smithi090 ceph-mon[29180]: pgmap v15867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:13.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.473 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:13 smithi119 ceph-mon[32752]: pgmap v15867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:13.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.474 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:13 smithi167 ceph-mon[32751]: pgmap v15867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:13.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:13.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: pgmap v15868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:15.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.281 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: pgmap v15868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:15.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: pgmap v15868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:15.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:15.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:15.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:16.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:33:16.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:33:16.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:33:16.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:33:16.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:33:16.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:33:16.280 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:16.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:33:16.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:33:16.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:33:16.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:33:16.294 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:16.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:33:16.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:33:16.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:33:16.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:33:16.352 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:33:17.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28677.427001 secs 2024-04-27T02:33:17.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:16 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:33:17.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:16 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:33:17.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:16 smithi119 ceph-mon[32752]: pgmap v15869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 868 B/s rd, 1 op/s 2024-04-27T02:33:17.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:16 smithi119 ceph-mon[32752]: pgmap v15870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:17.279 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28678.263601 secs 2024-04-27T02:33:17.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28677.427001 secs 2024-04-27T02:33:17.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:16 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:33:17.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:16 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:33:17.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:16 smithi167 ceph-mon[32751]: pgmap v15869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 868 B/s rd, 1 op/s 2024-04-27T02:33:17.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:16 smithi167 ceph-mon[32751]: pgmap v15870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:17.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28678.263601 secs 2024-04-27T02:33:17.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28677.427001 secs 2024-04-27T02:33:17.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:16 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:33:17.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:16 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:33:17.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:16 smithi090 ceph-mon[29180]: pgmap v15869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 868 B/s rd, 1 op/s 2024-04-27T02:33:17.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:16 smithi090 ceph-mon[29180]: pgmap v15870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:17.351 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28678.263601 secs 2024-04-27T02:33:19.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:18 smithi090 ceph-mon[29180]: pgmap v15871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:19.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:18 smithi119 ceph-mon[32752]: pgmap v15871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:19.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:18 smithi167 ceph-mon[32751]: pgmap v15871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:21.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:20 smithi119 ceph-mon[32752]: pgmap v15872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:21.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:20 smithi167 ceph-mon[32751]: pgmap v15872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:21.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:20 smithi090 ceph-mon[29180]: pgmap v15872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:22.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28682.427060 secs 2024-04-27T02:33:22.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:21 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28683.263717 secs 2024-04-27T02:33:22.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28682.427060 secs 2024-04-27T02:33:22.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:21 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28683.263717 secs 2024-04-27T02:33:22.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28682.427060 secs 2024-04-27T02:33:22.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:21 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28683.263717 secs 2024-04-27T02:33:23.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:22 smithi119 ceph-mon[32752]: pgmap v15873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:23.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:22 smithi167 ceph-mon[32751]: pgmap v15873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:23.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:22 smithi090 ceph-mon[29180]: pgmap v15873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T02:33:25.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:25 smithi119 ceph-mon[32752]: pgmap v15874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:33:25.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:25 smithi167 ceph-mon[32751]: pgmap v15874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:33:25.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:25 smithi090 ceph-mon[29180]: pgmap v15874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:33:26.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28687.427146 secs 2024-04-27T02:33:26.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28687.427146 secs 2024-04-27T02:33:26.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28687.427146 secs 2024-04-27T02:33:27.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:27 smithi119 ceph-mon[32752]: pgmap v15875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:33:27.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28688.263827 secs 2024-04-27T02:33:27.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:27 smithi167 ceph-mon[32751]: pgmap v15875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:33:27.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28688.263827 secs 2024-04-27T02:33:27.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:27 smithi090 ceph-mon[29180]: pgmap v15875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:33:27.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28688.263827 secs 2024-04-27T02:33:29.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:29 smithi119 ceph-mon[32752]: pgmap v15876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:29.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:29 smithi167 ceph-mon[32751]: pgmap v15876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:29.546 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:29 smithi090 ceph-mon[29180]: pgmap v15876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:31.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:31 smithi119 ceph-mon[32752]: pgmap v15877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:31.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28692.427230 secs 2024-04-27T02:33:31.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:31 smithi167 ceph-mon[32751]: pgmap v15877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:31.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28692.427230 secs 2024-04-27T02:33:31.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:31 smithi090 ceph-mon[29180]: pgmap v15877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:31.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28692.427230 secs 2024-04-27T02:33:32.429 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28693.263941 secs 2024-04-27T02:33:32.443 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28693.263941 secs 2024-04-27T02:33:32.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28693.263941 secs 2024-04-27T02:33:33.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:33 smithi119 ceph-mon[32752]: pgmap v15878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:33.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:33 smithi167 ceph-mon[32751]: pgmap v15878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:33.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:33 smithi090 ceph-mon[29180]: pgmap v15878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:35.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:35 smithi119 ceph-mon[32752]: pgmap v15879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:35.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:35 smithi167 ceph-mon[32751]: pgmap v15879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:35.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:35 smithi090 ceph-mon[29180]: pgmap v15879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:36.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28697.427310 secs 2024-04-27T02:33:36.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28697.427310 secs 2024-04-27T02:33:36.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28697.427310 secs 2024-04-27T02:33:37.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:37 smithi119 ceph-mon[32752]: pgmap v15880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:37.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28698.264081 secs 2024-04-27T02:33:37.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:37 smithi167 ceph-mon[32751]: pgmap v15880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:37.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28698.264081 secs 2024-04-27T02:33:37.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:37 smithi090 ceph-mon[29180]: pgmap v15880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:37.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28698.264081 secs 2024-04-27T02:33:39.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:39 smithi119 ceph-mon[32752]: pgmap v15881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:39.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:39 smithi167 ceph-mon[32751]: pgmap v15881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:39.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:39 smithi090 ceph-mon[29180]: pgmap v15881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:41.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:41 smithi119 ceph-mon[32752]: pgmap v15882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:41.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:41 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 28702.427371 secs 2024-04-27T02:33:41.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:41 smithi119 ceph-mon[32752]: slow request 23042.792079 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:33:41.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:41 smithi167 ceph-mon[32751]: pgmap v15882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:41.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:41 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 28702.427371 secs 2024-04-27T02:33:41.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:41 smithi167 ceph-mon[32751]: slow request 23042.792079 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:33:41.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:41 smithi090 ceph-mon[29180]: pgmap v15882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:41.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:41 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 28702.427371 secs 2024-04-27T02:33:41.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:41 smithi090 ceph-mon[29180]: slow request 23042.792079 seconds old, received at 2024-04-26T20:09:37.388696+0000: client_request(client.24733:267885 getattr AsXsFs #0x100000001fb 2024-04-26T20:09:37.387615+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:33:42.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28703.264215 secs 2024-04-27T02:33:42.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28703.264215 secs 2024-04-27T02:33:42.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28703.264215 secs 2024-04-27T02:33:43.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:43 smithi119 ceph-mon[32752]: pgmap v15883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:43.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:43 smithi167 ceph-mon[32751]: pgmap v15883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:43.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:43 smithi090 ceph-mon[29180]: pgmap v15883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:45.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:45 smithi119 ceph-mon[32752]: pgmap v15884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:45.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:45 smithi167 ceph-mon[32751]: pgmap v15884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:45.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:45 smithi090 ceph-mon[29180]: pgmap v15884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:46.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28707.427454 secs 2024-04-27T02:33:46.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28707.427454 secs 2024-04-27T02:33:46.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28707.427454 secs 2024-04-27T02:33:47.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:47 smithi119 ceph-mon[32752]: pgmap v15885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:47.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28708.264329 secs 2024-04-27T02:33:47.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:47 smithi167 ceph-mon[32751]: pgmap v15885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:47.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28708.264329 secs 2024-04-27T02:33:47.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:47 smithi090 ceph-mon[29180]: pgmap v15885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:47.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28708.264329 secs 2024-04-27T02:33:49.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:49 smithi119 ceph-mon[32752]: pgmap v15886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:49.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:49 smithi167 ceph-mon[32751]: pgmap v15886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:49.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:49 smithi090 ceph-mon[29180]: pgmap v15886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:51.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:51 smithi119 ceph-mon[32752]: pgmap v15887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:51.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28712.427536 secs 2024-04-27T02:33:51.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:51 smithi167 ceph-mon[32751]: pgmap v15887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:51.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28712.427536 secs 2024-04-27T02:33:51.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:51 smithi090 ceph-mon[29180]: pgmap v15887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:51.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28712.427536 secs 2024-04-27T02:33:52.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28713.264478 secs 2024-04-27T02:33:52.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28713.264478 secs 2024-04-27T02:33:52.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28713.264478 secs 2024-04-27T02:33:53.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:53 smithi119 ceph-mon[32752]: pgmap v15888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:53.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:53 smithi167 ceph-mon[32751]: pgmap v15888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:53.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:53 smithi090 ceph-mon[29180]: pgmap v15888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:55.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:55 smithi119 ceph-mon[32752]: pgmap v15889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:55.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:55 smithi167 ceph-mon[32751]: pgmap v15889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:55.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:55 smithi090 ceph-mon[29180]: pgmap v15889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:33:56.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28717.427618 secs 2024-04-27T02:33:56.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28717.427618 secs 2024-04-27T02:33:56.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28717.427618 secs 2024-04-27T02:33:57.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:57 smithi119 ceph-mon[32752]: pgmap v15890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:57.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28718.264598 secs 2024-04-27T02:33:57.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:57 smithi167 ceph-mon[32751]: pgmap v15890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:57.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28718.264598 secs 2024-04-27T02:33:57.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:57 smithi090 ceph-mon[29180]: pgmap v15890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:57.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28718.264598 secs 2024-04-27T02:33:59.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:33:59 smithi119 ceph-mon[32752]: pgmap v15891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:59.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:33:59 smithi167 ceph-mon[32751]: pgmap v15891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:33:59.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:33:59 smithi090 ceph-mon[29180]: pgmap v15891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:01.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:01 smithi119 ceph-mon[32752]: pgmap v15892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:01.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28722.427703 secs 2024-04-27T02:34:01.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:01 smithi167 ceph-mon[32751]: pgmap v15892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:01.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28722.427703 secs 2024-04-27T02:34:01.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:01 smithi090 ceph-mon[29180]: pgmap v15892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:01.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28722.427703 secs 2024-04-27T02:34:02.500 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28723.264736 secs 2024-04-27T02:34:02.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28723.264736 secs 2024-04-27T02:34:02.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28723.264736 secs 2024-04-27T02:34:03.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:03 smithi119 ceph-mon[32752]: pgmap v15893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:03.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:03 smithi167 ceph-mon[32751]: pgmap v15893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:03.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:03 smithi090 ceph-mon[29180]: pgmap v15893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:05.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:05 smithi119 ceph-mon[32752]: pgmap v15894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:05.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:05 smithi167 ceph-mon[32751]: pgmap v15894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:05.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:05 smithi090 ceph-mon[29180]: pgmap v15894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:06.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28727.427815 secs 2024-04-27T02:34:06.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28727.427815 secs 2024-04-27T02:34:06.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28727.427815 secs 2024-04-27T02:34:07.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:07 smithi119 ceph-mon[32752]: pgmap v15895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:07.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28728.264861 secs 2024-04-27T02:34:07.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:07 smithi167 ceph-mon[32751]: pgmap v15895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:07.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28728.264861 secs 2024-04-27T02:34:07.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:07 smithi090 ceph-mon[29180]: pgmap v15895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:07.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28728.264861 secs 2024-04-27T02:34:09.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:09 smithi119 ceph-mon[32752]: pgmap v15896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:09.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:09 smithi167 ceph-mon[32751]: pgmap v15896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:09.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:09 smithi090 ceph-mon[29180]: pgmap v15896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:11.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:11 smithi119 ceph-mon[32752]: pgmap v15897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:11.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28732.427900 secs 2024-04-27T02:34:11.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:11 smithi167 ceph-mon[32751]: pgmap v15897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28732.427900 secs 2024-04-27T02:34:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:11 smithi090 ceph-mon[29180]: pgmap v15897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28732.427900 secs 2024-04-27T02:34:12.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28733.264936 secs 2024-04-27T02:34:12.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28733.264936 secs 2024-04-27T02:34:12.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28733.264936 secs 2024-04-27T02:34:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:13 smithi119 ceph-mon[32752]: pgmap v15898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:13.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:13 smithi167 ceph-mon[32751]: pgmap v15898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:13.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:13 smithi090 ceph-mon[29180]: pgmap v15898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:15.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:15 smithi119 ceph-mon[32752]: pgmap v15899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:15.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:15 smithi167 ceph-mon[32751]: pgmap v15899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:15.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:15 smithi090 ceph-mon[29180]: pgmap v15899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28737.428002 secs 2024-04-27T02:34:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:34:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:16.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:16.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28737.428002 secs 2024-04-27T02:34:16.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:34:16.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:16.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:16.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28737.428002 secs 2024-04-27T02:34:16.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:34:16.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:16.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:17.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:17 smithi167 ceph-mon[32751]: pgmap v15900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:17.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28738.265001 secs 2024-04-27T02:34:17.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:34:17.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:34:17.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:17.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:17 smithi090 ceph-mon[29180]: pgmap v15900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:17.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28738.265001 secs 2024-04-27T02:34:17.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:34:17.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:34:17.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:17.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:17 smithi119 ceph-mon[32752]: pgmap v15900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:17.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28738.265001 secs 2024-04-27T02:34:17.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:34:17.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:34:17.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:34:18.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:18 smithi167 ceph-mon[32751]: pgmap v15901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T02:34:18.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:18 smithi167 ceph-mon[32751]: pgmap v15902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:18.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:18 smithi090 ceph-mon[29180]: pgmap v15901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T02:34:18.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:18 smithi090 ceph-mon[29180]: pgmap v15902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:18.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:18 smithi119 ceph-mon[32752]: pgmap v15901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T02:34:18.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:18 smithi119 ceph-mon[32752]: pgmap v15902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:20.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:20 smithi167 ceph-mon[32751]: pgmap v15903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:20.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:20 smithi090 ceph-mon[29180]: pgmap v15903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:20.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:20 smithi119 ceph-mon[32752]: pgmap v15903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:21.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28742.428091 secs 2024-04-27T02:34:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28742.428091 secs 2024-04-27T02:34:21.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28742.428091 secs 2024-04-27T02:34:22.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:22 smithi167 ceph-mon[32751]: pgmap v15904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28743.265129 secs 2024-04-27T02:34:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:22 smithi090 ceph-mon[29180]: pgmap v15904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:22.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28743.265129 secs 2024-04-27T02:34:22.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:22 smithi119 ceph-mon[32752]: pgmap v15904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:22.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28743.265129 secs 2024-04-27T02:34:24.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:24 smithi090 ceph-mon[29180]: pgmap v15905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:34:24.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:24 smithi119 ceph-mon[32752]: pgmap v15905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:34:24.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:24 smithi167 ceph-mon[32751]: pgmap v15905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:34:26.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:26 smithi090 ceph-mon[29180]: pgmap v15906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:26.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28747.428197 secs 2024-04-27T02:34:26.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:26 smithi119 ceph-mon[32752]: pgmap v15906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:26.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28747.428197 secs 2024-04-27T02:34:26.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:26 smithi167 ceph-mon[32751]: pgmap v15906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:34:26.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28747.428197 secs 2024-04-27T02:34:27.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28748.265205 secs 2024-04-27T02:34:27.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28748.265205 secs 2024-04-27T02:34:27.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28748.265205 secs 2024-04-27T02:34:28.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:28 smithi090 ceph-mon[29180]: pgmap v15907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:34:28.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:28 smithi119 ceph-mon[32752]: pgmap v15907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:34:28.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:28 smithi167 ceph-mon[32751]: pgmap v15907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:34:30.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:30 smithi090 ceph-mon[29180]: pgmap v15908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:30.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:30 smithi119 ceph-mon[32752]: pgmap v15908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:30.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:30 smithi167 ceph-mon[32751]: pgmap v15908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:31.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28752.428302 secs 2024-04-27T02:34:31.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28752.428302 secs 2024-04-27T02:34:31.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28752.428302 secs 2024-04-27T02:34:32.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:32 smithi090 ceph-mon[29180]: pgmap v15909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:32.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28753.265310 secs 2024-04-27T02:34:32.679 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:32 smithi119 ceph-mon[32752]: pgmap v15909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:32.679 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28753.265310 secs 2024-04-27T02:34:32.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:32 smithi167 ceph-mon[32751]: pgmap v15909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:32.693 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28753.265310 secs 2024-04-27T02:34:34.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:34 smithi090 ceph-mon[29180]: pgmap v15910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:34.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:34 smithi119 ceph-mon[32752]: pgmap v15910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:34.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:34 smithi167 ceph-mon[32751]: pgmap v15910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:36.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:36 smithi090 ceph-mon[29180]: pgmap v15911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:36.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28757.428393 secs 2024-04-27T02:34:36.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:36 smithi119 ceph-mon[32752]: pgmap v15911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:36.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28757.428393 secs 2024-04-27T02:34:36.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:36 smithi167 ceph-mon[32751]: pgmap v15911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:36.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28757.428393 secs 2024-04-27T02:34:37.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28758.265426 secs 2024-04-27T02:34:37.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28758.265426 secs 2024-04-27T02:34:37.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28758.265426 secs 2024-04-27T02:34:38.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:38 smithi090 ceph-mon[29180]: pgmap v15912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:38.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:38 smithi119 ceph-mon[32752]: pgmap v15912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:38.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:38 smithi167 ceph-mon[32751]: pgmap v15912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:39.100 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:34:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:34:38.706+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:34:39.100 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:34:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:34:38.706+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:34:39.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:34:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:34:38.706+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:34:39.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:34:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:34:38.706+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:34:39.101 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:34:38 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:34:38.706+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:34:40.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:40 smithi090 ceph-mon[29180]: pgmap v15913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:40.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:40 smithi119 ceph-mon[32752]: pgmap v15913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:40.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:40 smithi167 ceph-mon[32751]: pgmap v15913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:34:41.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28762.428492 secs 2024-04-27T02:34:41.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28762.428492 secs 2024-04-27T02:34:41.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28762.428492 secs 2024-04-27T02:34:42.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:42 smithi090 ceph-mon[29180]: pgmap v15914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:42.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28763.265561 secs 2024-04-27T02:34:42.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:42 smithi119 ceph-mon[32752]: pgmap v15914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:42.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28763.265561 secs 2024-04-27T02:34:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:42 smithi167 ceph-mon[32751]: pgmap v15914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28763.265561 secs 2024-04-27T02:34:43.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:43 smithi090 ceph-mon[29180]: pgmap v15915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:43.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:43 smithi119 ceph-mon[32752]: pgmap v15915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:43.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:43 smithi167 ceph-mon[32751]: pgmap v15915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:46.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:45 smithi119 ceph-mon[32752]: pgmap v15916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:46.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:45 smithi167 ceph-mon[32751]: pgmap v15916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:46.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:45 smithi090 ceph-mon[29180]: pgmap v15916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:47.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28767.428602 secs 2024-04-27T02:34:47.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28768.265678 secs 2024-04-27T02:34:47.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28767.428602 secs 2024-04-27T02:34:47.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28768.265678 secs 2024-04-27T02:34:47.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28767.428602 secs 2024-04-27T02:34:47.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28768.265678 secs 2024-04-27T02:34:48.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:47 smithi119 ceph-mon[32752]: pgmap v15917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:48.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:47 smithi167 ceph-mon[32751]: pgmap v15917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:48.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:47 smithi090 ceph-mon[29180]: pgmap v15917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:50.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:49 smithi119 ceph-mon[32752]: pgmap v15918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:50.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:49 smithi167 ceph-mon[32751]: pgmap v15918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:50.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:49 smithi090 ceph-mon[29180]: pgmap v15918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:52.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28772.428710 secs 2024-04-27T02:34:52.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:51 smithi119 ceph-mon[32752]: pgmap v15919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:52.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28773.265785 secs 2024-04-27T02:34:52.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28772.428710 secs 2024-04-27T02:34:52.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:51 smithi167 ceph-mon[32751]: pgmap v15919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:52.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28773.265785 secs 2024-04-27T02:34:52.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28772.428710 secs 2024-04-27T02:34:52.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:51 smithi090 ceph-mon[29180]: pgmap v15919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:52.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28773.265785 secs 2024-04-27T02:34:54.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:53 smithi119 ceph-mon[32752]: pgmap v15920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:54.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:53 smithi167 ceph-mon[32751]: pgmap v15920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:54.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:53 smithi090 ceph-mon[29180]: pgmap v15920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-27T02:34:56.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:55 smithi119 ceph-mon[32752]: pgmap v15921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:56.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:55 smithi167 ceph-mon[32751]: pgmap v15921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:56.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:55 smithi090 ceph-mon[29180]: pgmap v15921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-27T02:34:57.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28777.428821 secs 2024-04-27T02:34:57.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28778.265865 secs 2024-04-27T02:34:57.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28777.428821 secs 2024-04-27T02:34:57.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28778.265865 secs 2024-04-27T02:34:57.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28777.428821 secs 2024-04-27T02:34:57.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28778.265865 secs 2024-04-27T02:34:58.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:57 smithi119 ceph-mon[32752]: pgmap v15922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:58.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:57 smithi167 ceph-mon[32751]: pgmap v15922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:34:58.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:57 smithi090 ceph-mon[29180]: pgmap v15922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:00.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:34:59 smithi119 ceph-mon[32752]: pgmap v15923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:00.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:34:59 smithi167 ceph-mon[32751]: pgmap v15923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:00.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:34:59 smithi090 ceph-mon[29180]: pgmap v15923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:02.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28782.428902 secs 2024-04-27T02:35:02.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:01 smithi119 ceph-mon[32752]: pgmap v15924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:02.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28783.265983 secs 2024-04-27T02:35:02.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28782.428902 secs 2024-04-27T02:35:02.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:01 smithi167 ceph-mon[32751]: pgmap v15924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:02.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28783.265983 secs 2024-04-27T02:35:02.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28782.428902 secs 2024-04-27T02:35:02.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:01 smithi090 ceph-mon[29180]: pgmap v15924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:02.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28783.265983 secs 2024-04-27T02:35:04.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:03 smithi119 ceph-mon[32752]: pgmap v15925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:04.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:03 smithi167 ceph-mon[32751]: pgmap v15925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:04.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:03 smithi090 ceph-mon[29180]: pgmap v15925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:06.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:05 smithi119 ceph-mon[32752]: pgmap v15926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:06.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:05 smithi167 ceph-mon[32751]: pgmap v15926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:06.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:05 smithi090 ceph-mon[29180]: pgmap v15926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:07.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28787.428978 secs 2024-04-27T02:35:07.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28788.266122 secs 2024-04-27T02:35:07.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28787.428978 secs 2024-04-27T02:35:07.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28788.266122 secs 2024-04-27T02:35:07.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28787.428978 secs 2024-04-27T02:35:07.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28788.266122 secs 2024-04-27T02:35:07.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:07 smithi090 ceph-mon[29180]: pgmap v15927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:08.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:07 smithi119 ceph-mon[32752]: pgmap v15927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:08.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:07 smithi167 ceph-mon[32751]: pgmap v15927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:10.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:09 smithi119 ceph-mon[32752]: pgmap v15928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:10.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:09 smithi167 ceph-mon[32751]: pgmap v15928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:10.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:09 smithi090 ceph-mon[29180]: pgmap v15928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:12.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28792.429062 secs 2024-04-27T02:35:12.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:11 smithi119 ceph-mon[32752]: pgmap v15929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:12.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:11 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 28793.266258 secs 2024-04-27T02:35:12.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:11 smithi119 ceph-mon[32752]: slow request 2883.048308 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:35:12.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28792.429062 secs 2024-04-27T02:35:12.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:11 smithi167 ceph-mon[32751]: pgmap v15929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:12.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:11 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 28793.266258 secs 2024-04-27T02:35:12.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:11 smithi167 ceph-mon[32751]: slow request 2883.048308 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:35:12.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28792.429062 secs 2024-04-27T02:35:12.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:11 smithi090 ceph-mon[29180]: pgmap v15929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:12.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:11 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 28793.266258 secs 2024-04-27T02:35:12.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:11 smithi090 ceph-mon[29180]: slow request 2883.048308 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:35:14.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:13 smithi119 ceph-mon[32752]: pgmap v15930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:14.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:13 smithi167 ceph-mon[32751]: pgmap v15930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:14.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:13 smithi090 ceph-mon[29180]: pgmap v15930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:16.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:15 smithi119 ceph-mon[32752]: pgmap v15931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:16.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:15 smithi167 ceph-mon[32751]: pgmap v15931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:16.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:15 smithi090 ceph-mon[29180]: pgmap v15931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:17.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:16 smithi119 ceph-mon[32752]: 37 slow requests, 1 included below; oldest blocked for > 28797.429141 secs 2024-04-27T02:35:17.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:16 smithi119 ceph-mon[32752]: slow request 2884.305922 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:35:17.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28798.266355 secs 2024-04-27T02:35:17.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:35:17.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:16 smithi167 ceph-mon[32751]: 37 slow requests, 1 included below; oldest blocked for > 28797.429141 secs 2024-04-27T02:35:17.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:16 smithi167 ceph-mon[32751]: slow request 2884.305922 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:35:17.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28798.266355 secs 2024-04-27T02:35:17.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:35:17.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:16 smithi090 ceph-mon[29180]: 37 slow requests, 1 included below; oldest blocked for > 28797.429141 secs 2024-04-27T02:35:17.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:16 smithi090 ceph-mon[29180]: slow request 2884.305922 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:35:17.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28798.266355 secs 2024-04-27T02:35:17.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:35:18.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:17 smithi119 ceph-mon[32752]: pgmap v15932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:18.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:35:18.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:35:18.028 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:35:18.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:17 smithi167 ceph-mon[32751]: pgmap v15932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:18.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:35:18.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:35:18.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:35:18.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:17 smithi090 ceph-mon[29180]: pgmap v15932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:18.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:35:18.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:35:18.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:35:19.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:18 smithi119 ceph-mon[32752]: pgmap v15933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:35:19.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:18 smithi119 ceph-mon[32752]: pgmap v15934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:35:19.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:18 smithi167 ceph-mon[32751]: pgmap v15933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:35:19.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:18 smithi167 ceph-mon[32751]: pgmap v15934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:35:19.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:18 smithi090 ceph-mon[29180]: pgmap v15933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:35:19.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:18 smithi090 ceph-mon[29180]: pgmap v15934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:35:21.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:20 smithi119 ceph-mon[32752]: pgmap v15935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:21.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:20 smithi167 ceph-mon[32751]: pgmap v15935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:21.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:20 smithi090 ceph-mon[29180]: pgmap v15935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28802.429227 secs 2024-04-27T02:35:22.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:21 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28803.266478 secs 2024-04-27T02:35:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28802.429227 secs 2024-04-27T02:35:22.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:21 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28803.266478 secs 2024-04-27T02:35:22.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28802.429227 secs 2024-04-27T02:35:22.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:21 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28803.266478 secs 2024-04-27T02:35:23.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:22 smithi119 ceph-mon[32752]: pgmap v15936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:23.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:22 smithi167 ceph-mon[32751]: pgmap v15936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:23.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:22 smithi090 ceph-mon[29180]: pgmap v15936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:25.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:24 smithi167 ceph-mon[32751]: pgmap v15937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:25.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:24 smithi090 ceph-mon[29180]: pgmap v15937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:25.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:24 smithi119 ceph-mon[32752]: pgmap v15937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:27.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28807.429310 secs 2024-04-27T02:35:27.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:26 smithi167 ceph-mon[32751]: pgmap v15938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:27.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:26 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28808.266619 secs 2024-04-27T02:35:27.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28807.429310 secs 2024-04-27T02:35:27.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:26 smithi090 ceph-mon[29180]: pgmap v15938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:27.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:26 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28808.266619 secs 2024-04-27T02:35:27.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28807.429310 secs 2024-04-27T02:35:27.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:26 smithi119 ceph-mon[32752]: pgmap v15938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:35:27.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:26 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28808.266619 secs 2024-04-27T02:35:29.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:28 smithi167 ceph-mon[32751]: pgmap v15939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:35:29.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:28 smithi090 ceph-mon[29180]: pgmap v15939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:35:29.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:28 smithi119 ceph-mon[32752]: pgmap v15939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:35:31.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:30 smithi167 ceph-mon[32751]: pgmap v15940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:31.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:30 smithi090 ceph-mon[29180]: pgmap v15940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:31.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:30 smithi119 ceph-mon[32752]: pgmap v15940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:32.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28812.429391 secs 2024-04-27T02:35:32.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:31 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28813.266717 secs 2024-04-27T02:35:32.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28812.429391 secs 2024-04-27T02:35:32.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:31 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28813.266717 secs 2024-04-27T02:35:32.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28812.429391 secs 2024-04-27T02:35:32.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:31 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28813.266717 secs 2024-04-27T02:35:33.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:32 smithi090 ceph-mon[29180]: pgmap v15941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:33.178 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:32 smithi119 ceph-mon[32752]: pgmap v15941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:33.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:32 smithi167 ceph-mon[32751]: pgmap v15941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:35.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:34 smithi090 ceph-mon[29180]: pgmap v15942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:35.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:34 smithi119 ceph-mon[32752]: pgmap v15942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:35.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:34 smithi167 ceph-mon[32751]: pgmap v15942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:37.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28817.429474 secs 2024-04-27T02:35:37.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:36 smithi090 ceph-mon[29180]: pgmap v15943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:37.101 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:36 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28818.266791 secs 2024-04-27T02:35:37.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28817.429474 secs 2024-04-27T02:35:37.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:36 smithi119 ceph-mon[32752]: pgmap v15943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:37.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:36 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28818.266791 secs 2024-04-27T02:35:37.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28817.429474 secs 2024-04-27T02:35:37.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:36 smithi167 ceph-mon[32751]: pgmap v15943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:37.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:36 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28818.266791 secs 2024-04-27T02:35:39.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:38 smithi090 ceph-mon[29180]: pgmap v15944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:39.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:38 smithi119 ceph-mon[32752]: pgmap v15944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:39.293 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:38 smithi167 ceph-mon[32751]: pgmap v15944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:41.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:40 smithi090 ceph-mon[29180]: pgmap v15945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:41.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:40 smithi119 ceph-mon[32752]: pgmap v15945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:41.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:40 smithi167 ceph-mon[32751]: pgmap v15945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:42.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28822.429548 secs 2024-04-27T02:35:42.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:41 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 28823.266928 secs 2024-04-27T02:35:42.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:41 smithi090 ceph-mon[29180]: slow request 362.358052 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:35:42.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28822.429548 secs 2024-04-27T02:35:42.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:41 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 28823.266928 secs 2024-04-27T02:35:42.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:41 smithi119 ceph-mon[32752]: slow request 362.358052 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:35:42.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28822.429548 secs 2024-04-27T02:35:42.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:41 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 28823.266928 secs 2024-04-27T02:35:42.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:41 smithi167 ceph-mon[32751]: slow request 362.358052 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:35:43.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:42 smithi090 ceph-mon[29180]: pgmap v15946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:43.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:42 smithi119 ceph-mon[32752]: pgmap v15946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:43.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:42 smithi167 ceph-mon[32751]: pgmap v15946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:45.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:44 smithi090 ceph-mon[29180]: pgmap v15947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:45.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:44 smithi119 ceph-mon[32752]: pgmap v15947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:45.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:44 smithi167 ceph-mon[32751]: pgmap v15947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:47.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28827.429627 secs 2024-04-27T02:35:47.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:46 smithi090 ceph-mon[29180]: pgmap v15948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:47.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28828.267032 secs 2024-04-27T02:35:47.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28827.429627 secs 2024-04-27T02:35:47.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:46 smithi119 ceph-mon[32752]: pgmap v15948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:47.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28828.267032 secs 2024-04-27T02:35:47.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28827.429627 secs 2024-04-27T02:35:47.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:46 smithi167 ceph-mon[32751]: pgmap v15948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:47.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28828.267032 secs 2024-04-27T02:35:49.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:48 smithi090 ceph-mon[29180]: pgmap v15949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:49.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:48 smithi119 ceph-mon[32752]: pgmap v15949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:49.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:48 smithi167 ceph-mon[32751]: pgmap v15949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:51.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:50 smithi090 ceph-mon[29180]: pgmap v15950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:51.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:50 smithi119 ceph-mon[32752]: pgmap v15950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:51.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:50 smithi167 ceph-mon[32751]: pgmap v15950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:52.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28832.429706 secs 2024-04-27T02:35:52.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28833.267160 secs 2024-04-27T02:35:52.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28832.429706 secs 2024-04-27T02:35:52.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28833.267160 secs 2024-04-27T02:35:52.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28832.429706 secs 2024-04-27T02:35:52.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28833.267160 secs 2024-04-27T02:35:53.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:52 smithi090 ceph-mon[29180]: pgmap v15951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:53.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:52 smithi119 ceph-mon[32752]: pgmap v15951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:53.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:52 smithi167 ceph-mon[32751]: pgmap v15951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:55.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:54 smithi090 ceph-mon[29180]: pgmap v15952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:55.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:54 smithi119 ceph-mon[32752]: pgmap v15952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:55.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:54 smithi167 ceph-mon[32751]: pgmap v15952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:35:57.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28837.429799 secs 2024-04-27T02:35:57.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:56 smithi090 ceph-mon[29180]: pgmap v15953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:57.100 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28838.267257 secs 2024-04-27T02:35:57.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28837.429799 secs 2024-04-27T02:35:57.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:56 smithi119 ceph-mon[32752]: pgmap v15953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:57.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28838.267257 secs 2024-04-27T02:35:57.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28837.429799 secs 2024-04-27T02:35:57.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:56 smithi167 ceph-mon[32751]: pgmap v15953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:57.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28838.267257 secs 2024-04-27T02:35:59.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:35:58 smithi090 ceph-mon[29180]: pgmap v15954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:59.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:35:58 smithi119 ceph-mon[32752]: pgmap v15954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:35:59.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:35:58 smithi167 ceph-mon[32751]: pgmap v15954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:01.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:00 smithi090 ceph-mon[29180]: pgmap v15955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:01.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:00 smithi119 ceph-mon[32752]: pgmap v15955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:01.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:00 smithi167 ceph-mon[32751]: pgmap v15955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:02.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28842.429881 secs 2024-04-27T02:36:02.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28843.267328 secs 2024-04-27T02:36:02.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28842.429881 secs 2024-04-27T02:36:02.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28843.267328 secs 2024-04-27T02:36:02.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28842.429881 secs 2024-04-27T02:36:02.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28843.267328 secs 2024-04-27T02:36:03.178 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:02 smithi119 ceph-mon[32752]: pgmap v15956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:03.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:02 smithi167 ceph-mon[32751]: pgmap v15956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:03.250 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:02 smithi090 ceph-mon[29180]: pgmap v15956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:05.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:04 smithi119 ceph-mon[32752]: pgmap v15957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:05.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:04 smithi167 ceph-mon[32751]: pgmap v15957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:05.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:04 smithi090 ceph-mon[29180]: pgmap v15957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:07.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28847.429970 secs 2024-04-27T02:36:07.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:06 smithi119 ceph-mon[32752]: pgmap v15958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:07.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:06 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28848.267453 secs 2024-04-27T02:36:07.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28847.429970 secs 2024-04-27T02:36:07.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:06 smithi167 ceph-mon[32751]: pgmap v15958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:07.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:06 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28848.267453 secs 2024-04-27T02:36:07.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28847.429970 secs 2024-04-27T02:36:07.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:06 smithi090 ceph-mon[29180]: pgmap v15958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:07.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:06 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28848.267453 secs 2024-04-27T02:36:09.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:08 smithi119 ceph-mon[32752]: pgmap v15959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:09.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:08 smithi167 ceph-mon[32751]: pgmap v15959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:09.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:08 smithi090 ceph-mon[29180]: pgmap v15959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:11.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:10 smithi119 ceph-mon[32752]: pgmap v15960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:11.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:10 smithi167 ceph-mon[32751]: pgmap v15960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:11.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:10 smithi090 ceph-mon[29180]: pgmap v15960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:12.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28852.430056 secs 2024-04-27T02:36:12.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:11 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28853.267560 secs 2024-04-27T02:36:12.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28852.430056 secs 2024-04-27T02:36:12.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:11 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28853.267560 secs 2024-04-27T02:36:12.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28852.430056 secs 2024-04-27T02:36:12.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:11 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28853.267560 secs 2024-04-27T02:36:13.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:12 smithi119 ceph-mon[32752]: pgmap v15961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:13.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:12 smithi167 ceph-mon[32751]: pgmap v15961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:13.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:12 smithi090 ceph-mon[29180]: pgmap v15961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:15.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:14 smithi119 ceph-mon[32752]: pgmap v15962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:15.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:14 smithi167 ceph-mon[32751]: pgmap v15962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:15.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:14 smithi090 ceph-mon[29180]: pgmap v15962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:17.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28857.430138 secs 2024-04-27T02:36:17.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:16 smithi119 ceph-mon[32752]: pgmap v15963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:17.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:16 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28858.267679 secs 2024-04-27T02:36:17.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28857.430138 secs 2024-04-27T02:36:17.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:16 smithi167 ceph-mon[32751]: pgmap v15963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:17.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:16 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28858.267679 secs 2024-04-27T02:36:17.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28857.430138 secs 2024-04-27T02:36:17.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:16 smithi090 ceph-mon[29180]: pgmap v15963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:17.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:16 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28858.267679 secs 2024-04-27T02:36:18.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:36:18.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:36:18.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:36:19.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:18 smithi119 ceph-mon[32752]: pgmap v15964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:19.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:36:19.277 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:36:19.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:18 smithi119 ceph-mon[32752]: pgmap v15965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:36:19.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:18 smithi119 ceph-mon[32752]: pgmap v15966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:36:19.278 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:36:19.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:18 smithi167 ceph-mon[32751]: pgmap v15964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:19.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:36:19.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:36:19.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:18 smithi167 ceph-mon[32751]: pgmap v15965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:36:19.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:18 smithi167 ceph-mon[32751]: pgmap v15966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:36:19.292 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:36:19.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:18 smithi090 ceph-mon[29180]: pgmap v15964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:19.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:36:19.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:36:19.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:18 smithi090 ceph-mon[29180]: pgmap v15965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:36:19.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:18 smithi090 ceph-mon[29180]: pgmap v15966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:36:19.350 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:36:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28862.430222 secs 2024-04-27T02:36:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:21 smithi119 ceph-mon[32752]: pgmap v15967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28862.430222 secs 2024-04-27T02:36:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:21 smithi167 ceph-mon[32751]: pgmap v15967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28862.430222 secs 2024-04-27T02:36:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:21 smithi090 ceph-mon[29180]: pgmap v15967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:22.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28863.267812 secs 2024-04-27T02:36:22.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28863.267812 secs 2024-04-27T02:36:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28863.267812 secs 2024-04-27T02:36:23.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:23 smithi119 ceph-mon[32752]: pgmap v15968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:36:23.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:23 smithi167 ceph-mon[32751]: pgmap v15968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:36:23.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:23 smithi090 ceph-mon[29180]: pgmap v15968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:36:25.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:25 smithi119 ceph-mon[32752]: pgmap v15969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:25.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:25 smithi167 ceph-mon[32751]: pgmap v15969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:25.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:25 smithi090 ceph-mon[29180]: pgmap v15969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:26.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28867.430306 secs 2024-04-27T02:36:26.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28867.430306 secs 2024-04-27T02:36:26.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28867.430306 secs 2024-04-27T02:36:27.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28868.267952 secs 2024-04-27T02:36:27.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:27 smithi119 ceph-mon[32752]: pgmap v15970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:27.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28868.267952 secs 2024-04-27T02:36:27.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:27 smithi167 ceph-mon[32751]: pgmap v15970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:27.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28868.267952 secs 2024-04-27T02:36:27.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:27 smithi090 ceph-mon[29180]: pgmap v15970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:36:29.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:29 smithi119 ceph-mon[32752]: pgmap v15971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:36:29.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:29 smithi167 ceph-mon[32751]: pgmap v15971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:36:29.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:29 smithi090 ceph-mon[29180]: pgmap v15971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:36:31.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28872.430390 secs 2024-04-27T02:36:31.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:31 smithi119 ceph-mon[32752]: pgmap v15972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:31.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28872.430390 secs 2024-04-27T02:36:31.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:31 smithi167 ceph-mon[32751]: pgmap v15972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:31.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28872.430390 secs 2024-04-27T02:36:31.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:31 smithi090 ceph-mon[29180]: pgmap v15972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:32.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28873.268099 secs 2024-04-27T02:36:32.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28873.268099 secs 2024-04-27T02:36:32.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28873.268099 secs 2024-04-27T02:36:33.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:33 smithi167 ceph-mon[32751]: pgmap v15973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:33.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:33 smithi090 ceph-mon[29180]: pgmap v15973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:33.678 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:33 smithi119 ceph-mon[32752]: pgmap v15973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:35.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:35 smithi167 ceph-mon[32751]: pgmap v15974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:35.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:35 smithi090 ceph-mon[29180]: pgmap v15974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:35.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:35 smithi119 ceph-mon[32752]: pgmap v15974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:36.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28877.430449 secs 2024-04-27T02:36:36.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28877.430449 secs 2024-04-27T02:36:36.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28877.430449 secs 2024-04-27T02:36:37.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28878.268230 secs 2024-04-27T02:36:37.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:37 smithi090 ceph-mon[29180]: pgmap v15975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:37.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28878.268230 secs 2024-04-27T02:36:37.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:37 smithi119 ceph-mon[32752]: pgmap v15975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:37.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28878.268230 secs 2024-04-27T02:36:37.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:37 smithi167 ceph-mon[32751]: pgmap v15975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:39.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:39 smithi090 ceph-mon[29180]: pgmap v15976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:39.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:39 smithi119 ceph-mon[32752]: pgmap v15976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:39.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:39 smithi167 ceph-mon[32751]: pgmap v15976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:41.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28882.430532 secs 2024-04-27T02:36:41.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:41 smithi090 ceph-mon[29180]: pgmap v15977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:41.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28882.430532 secs 2024-04-27T02:36:41.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:41 smithi119 ceph-mon[32752]: pgmap v15977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:41.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28882.430532 secs 2024-04-27T02:36:41.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:41 smithi167 ceph-mon[32751]: pgmap v15977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:42.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28883.268336 secs 2024-04-27T02:36:42.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28883.268336 secs 2024-04-27T02:36:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28883.268336 secs 2024-04-27T02:36:43.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:43 smithi090 ceph-mon[29180]: pgmap v15978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:43.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:43 smithi119 ceph-mon[32752]: pgmap v15978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:43.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:43 smithi167 ceph-mon[32751]: pgmap v15978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:45.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:45 smithi119 ceph-mon[32752]: pgmap v15979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:45.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:45 smithi167 ceph-mon[32751]: pgmap v15979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:45.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:45 smithi090 ceph-mon[29180]: pgmap v15979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:46.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28887.430614 secs 2024-04-27T02:36:46.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28887.430614 secs 2024-04-27T02:36:46.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28887.430614 secs 2024-04-27T02:36:47.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28888.268455 secs 2024-04-27T02:36:47.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:47 smithi119 ceph-mon[32752]: pgmap v15980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:47.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28888.268455 secs 2024-04-27T02:36:47.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:47 smithi167 ceph-mon[32751]: pgmap v15980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:47.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28888.268455 secs 2024-04-27T02:36:47.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:47 smithi090 ceph-mon[29180]: pgmap v15980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:49.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:49 smithi119 ceph-mon[32752]: pgmap v15981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:49.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:49 smithi167 ceph-mon[32751]: pgmap v15981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:49.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:49 smithi090 ceph-mon[29180]: pgmap v15981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:51.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28892.430696 secs 2024-04-27T02:36:51.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:51 smithi119 ceph-mon[32752]: pgmap v15982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:51.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28892.430696 secs 2024-04-27T02:36:51.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:51 smithi167 ceph-mon[32751]: pgmap v15982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:51.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28892.430696 secs 2024-04-27T02:36:51.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:51 smithi090 ceph-mon[29180]: pgmap v15982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:52.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28893.268544 secs 2024-04-27T02:36:52.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28893.268544 secs 2024-04-27T02:36:52.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28893.268544 secs 2024-04-27T02:36:53.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:53 smithi119 ceph-mon[32752]: pgmap v15983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:53.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:53 smithi167 ceph-mon[32751]: pgmap v15983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:53.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:53 smithi090 ceph-mon[29180]: pgmap v15983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:55.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:55 smithi119 ceph-mon[32752]: pgmap v15984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:55.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:55 smithi167 ceph-mon[32751]: pgmap v15984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:55.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:55 smithi090 ceph-mon[29180]: pgmap v15984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:36:56.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28897.430778 secs 2024-04-27T02:36:56.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28897.430778 secs 2024-04-27T02:36:56.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28897.430778 secs 2024-04-27T02:36:57.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28898.268648 secs 2024-04-27T02:36:57.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:57 smithi119 ceph-mon[32752]: pgmap v15985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:57.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28898.268648 secs 2024-04-27T02:36:57.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:57 smithi167 ceph-mon[32751]: pgmap v15985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:57.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28898.268648 secs 2024-04-27T02:36:57.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:57 smithi090 ceph-mon[29180]: pgmap v15985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:59.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:36:59 smithi119 ceph-mon[32752]: pgmap v15986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:59.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:36:59 smithi167 ceph-mon[32751]: pgmap v15986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:36:59.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:36:59 smithi090 ceph-mon[29180]: pgmap v15986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:01.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28902.430862 secs 2024-04-27T02:37:01.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:01 smithi119 ceph-mon[32752]: pgmap v15987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:01.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28902.430862 secs 2024-04-27T02:37:01.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:01 smithi167 ceph-mon[32751]: pgmap v15987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:01.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28902.430862 secs 2024-04-27T02:37:01.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:01 smithi090 ceph-mon[29180]: pgmap v15987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:02.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28903.268764 secs 2024-04-27T02:37:02.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28903.268764 secs 2024-04-27T02:37:02.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28903.268764 secs 2024-04-27T02:37:03.501 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:03 smithi090 ceph-mon[29180]: pgmap v15988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:03.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:03 smithi119 ceph-mon[32752]: pgmap v15988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:03.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:03 smithi167 ceph-mon[32751]: pgmap v15988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:05.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:05 smithi119 ceph-mon[32752]: pgmap v15989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:05.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:05 smithi167 ceph-mon[32751]: pgmap v15989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:05.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:05 smithi090 ceph-mon[29180]: pgmap v15989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:06.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28907.430950 secs 2024-04-27T02:37:06.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28907.430950 secs 2024-04-27T02:37:06.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28907.430950 secs 2024-04-27T02:37:07.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28908.268856 secs 2024-04-27T02:37:07.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:07 smithi119 ceph-mon[32752]: pgmap v15990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:07.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28908.268856 secs 2024-04-27T02:37:07.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:07 smithi167 ceph-mon[32751]: pgmap v15990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:07.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28908.268856 secs 2024-04-27T02:37:07.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:07 smithi090 ceph-mon[29180]: pgmap v15990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:09.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:09 smithi119 ceph-mon[32752]: pgmap v15991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:09.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:09 smithi167 ceph-mon[32751]: pgmap v15991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:09.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:09 smithi090 ceph-mon[29180]: pgmap v15991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:11.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28912.431036 secs 2024-04-27T02:37:11.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:11 smithi119 ceph-mon[32752]: pgmap v15992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:11.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28912.431036 secs 2024-04-27T02:37:11.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:11 smithi167 ceph-mon[32751]: pgmap v15992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:11.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28912.431036 secs 2024-04-27T02:37:11.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:11 smithi090 ceph-mon[29180]: pgmap v15992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:12.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28913.268914 secs 2024-04-27T02:37:12.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28913.268914 secs 2024-04-27T02:37:12.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28913.268914 secs 2024-04-27T02:37:13.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:13 smithi119 ceph-mon[32752]: pgmap v15993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:13.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:13 smithi167 ceph-mon[32751]: pgmap v15993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:13.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:13 smithi090 ceph-mon[29180]: pgmap v15993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:15.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:15 smithi119 ceph-mon[32752]: pgmap v15994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:15.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:15 smithi167 ceph-mon[32751]: pgmap v15994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:15.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:15 smithi090 ceph-mon[29180]: pgmap v15994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:16.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28917.431118 secs 2024-04-27T02:37:16.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28917.431118 secs 2024-04-27T02:37:16.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28917.431118 secs 2024-04-27T02:37:17.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28918.269035 secs 2024-04-27T02:37:17.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:17 smithi119 ceph-mon[32752]: pgmap v15995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:17.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28918.269035 secs 2024-04-27T02:37:17.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:17 smithi167 ceph-mon[32751]: pgmap v15995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:17.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28918.269035 secs 2024-04-27T02:37:17.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:17 smithi090 ceph-mon[29180]: pgmap v15995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:19.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:19 smithi167 ceph-mon[32751]: pgmap v15996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:19.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:37:19.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:37:19.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:37:19.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:37:19.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:19 smithi090 ceph-mon[29180]: pgmap v15996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:19.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:37:19.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:37:19.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:37:19.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:37:19.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:19 smithi119 ceph-mon[32752]: pgmap v15996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:19.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:37:19.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:37:19.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:37:19.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:37:20.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:20 smithi167 ceph-mon[32751]: pgmap v15997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:37:20.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:20 smithi167 ceph-mon[32751]: pgmap v15998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:20.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:20 smithi090 ceph-mon[29180]: pgmap v15997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:37:20.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:20 smithi090 ceph-mon[29180]: pgmap v15998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:20.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:20 smithi119 ceph-mon[32752]: pgmap v15997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:37:20.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:20 smithi119 ceph-mon[32752]: pgmap v15998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28922.431178 secs 2024-04-27T02:37:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28922.431178 secs 2024-04-27T02:37:21.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28922.431178 secs 2024-04-27T02:37:22.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:22 smithi167 ceph-mon[32751]: pgmap v15999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:22.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28923.269144 secs 2024-04-27T02:37:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:22 smithi090 ceph-mon[29180]: pgmap v15999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28923.269144 secs 2024-04-27T02:37:22.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:22 smithi119 ceph-mon[32752]: pgmap v15999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:22.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28923.269144 secs 2024-04-27T02:37:24.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:24 smithi090 ceph-mon[29180]: pgmap v16000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:37:24.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:24 smithi119 ceph-mon[32752]: pgmap v16000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:37:24.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:24 smithi167 ceph-mon[32751]: pgmap v16000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:37:26.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:26 smithi090 ceph-mon[29180]: pgmap v16001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:26.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28927.431258 secs 2024-04-27T02:37:26.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:26 smithi119 ceph-mon[32752]: pgmap v16001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:26.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28927.431258 secs 2024-04-27T02:37:26.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:26 smithi167 ceph-mon[32751]: pgmap v16001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:26.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28927.431258 secs 2024-04-27T02:37:27.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28928.269262 secs 2024-04-27T02:37:27.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28928.269262 secs 2024-04-27T02:37:27.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28928.269262 secs 2024-04-27T02:37:28.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:28 smithi090 ceph-mon[29180]: pgmap v16002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:28.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:28 smithi119 ceph-mon[32752]: pgmap v16002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:28.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:28 smithi167 ceph-mon[32751]: pgmap v16002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:37:30.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:30 smithi090 ceph-mon[29180]: pgmap v16003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:37:30.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:30 smithi119 ceph-mon[32752]: pgmap v16003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:37:30.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:30 smithi167 ceph-mon[32751]: pgmap v16003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:37:31.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28932.431338 secs 2024-04-27T02:37:31.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28932.431338 secs 2024-04-27T02:37:31.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28932.431338 secs 2024-04-27T02:37:32.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:32 smithi090 ceph-mon[29180]: pgmap v16004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:32.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28933.269367 secs 2024-04-27T02:37:32.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:32 smithi119 ceph-mon[32752]: pgmap v16004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:32.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28933.269367 secs 2024-04-27T02:37:32.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:32 smithi167 ceph-mon[32751]: pgmap v16004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:32.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28933.269367 secs 2024-04-27T02:37:34.579 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:34 smithi119 ceph-mon[32752]: pgmap v16005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:34.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:34 smithi167 ceph-mon[32751]: pgmap v16005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:34.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:34 smithi090 ceph-mon[29180]: pgmap v16005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:36.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:36 smithi090 ceph-mon[29180]: pgmap v16006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:36.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28937.431424 secs 2024-04-27T02:37:36.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:36 smithi119 ceph-mon[32752]: pgmap v16006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:36.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28937.431424 secs 2024-04-27T02:37:36.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:36 smithi167 ceph-mon[32751]: pgmap v16006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:36.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28937.431424 secs 2024-04-27T02:37:37.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28938.269462 secs 2024-04-27T02:37:37.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28938.269462 secs 2024-04-27T02:37:37.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28938.269462 secs 2024-04-27T02:37:38.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:38 smithi090 ceph-mon[29180]: pgmap v16007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:38.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:38 smithi119 ceph-mon[32752]: pgmap v16007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:38.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:38 smithi167 ceph-mon[32751]: pgmap v16007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:40.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:40 smithi090 ceph-mon[29180]: pgmap v16008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:40.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:40 smithi119 ceph-mon[32752]: pgmap v16008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:40.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:40 smithi167 ceph-mon[32751]: pgmap v16008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:41.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28942.431506 secs 2024-04-27T02:37:41.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28942.431506 secs 2024-04-27T02:37:41.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28942.431506 secs 2024-04-27T02:37:42.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:42 smithi090 ceph-mon[29180]: pgmap v16009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:42.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28943.269576 secs 2024-04-27T02:37:42.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:42 smithi119 ceph-mon[32752]: pgmap v16009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:42.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28943.269576 secs 2024-04-27T02:37:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:42 smithi167 ceph-mon[32751]: pgmap v16009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28943.269576 secs 2024-04-27T02:37:44.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:44 smithi090 ceph-mon[29180]: pgmap v16010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:44.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:44 smithi119 ceph-mon[32752]: pgmap v16010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:44.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:44 smithi167 ceph-mon[32751]: pgmap v16010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:46.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:46 smithi090 ceph-mon[29180]: pgmap v16011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:46.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28947.431588 secs 2024-04-27T02:37:46.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:46 smithi119 ceph-mon[32752]: pgmap v16011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:46.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28947.431588 secs 2024-04-27T02:37:46.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:46 smithi167 ceph-mon[32751]: pgmap v16011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:46.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28947.431588 secs 2024-04-27T02:37:47.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28948.269693 secs 2024-04-27T02:37:47.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28948.269693 secs 2024-04-27T02:37:47.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28948.269693 secs 2024-04-27T02:37:48.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:48 smithi090 ceph-mon[29180]: pgmap v16012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:48.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:48 smithi119 ceph-mon[32752]: pgmap v16012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:48.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:48 smithi167 ceph-mon[32751]: pgmap v16012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:50.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:50 smithi119 ceph-mon[32752]: pgmap v16013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:50.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:50 smithi167 ceph-mon[32751]: pgmap v16013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:50.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:50 smithi090 ceph-mon[29180]: pgmap v16013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:51.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28952.431668 secs 2024-04-27T02:37:51.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28952.431668 secs 2024-04-27T02:37:51.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28952.431668 secs 2024-04-27T02:37:52.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:52 smithi119 ceph-mon[32752]: pgmap v16014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:52.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28953.269806 secs 2024-04-27T02:37:52.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:52 smithi167 ceph-mon[32751]: pgmap v16014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:52.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28953.269806 secs 2024-04-27T02:37:52.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:52 smithi090 ceph-mon[29180]: pgmap v16014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:52.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28953.269806 secs 2024-04-27T02:37:53.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:53 smithi119 ceph-mon[32752]: pgmap v16015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:53.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:53 smithi167 ceph-mon[32751]: pgmap v16015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:53.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:53 smithi090 ceph-mon[29180]: pgmap v16015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:56.291 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:56 smithi167 ceph-mon[32751]: pgmap v16016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:56.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:56 smithi090 ceph-mon[29180]: pgmap v16016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:56.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:56 smithi119 ceph-mon[32752]: pgmap v16016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:37:57.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:57 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28957.431749 secs 2024-04-27T02:37:57.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28958.269957 secs 2024-04-27T02:37:57.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:57 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28957.431749 secs 2024-04-27T02:37:57.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28958.269957 secs 2024-04-27T02:37:57.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:57 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28957.431749 secs 2024-04-27T02:37:57.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28958.269957 secs 2024-04-27T02:37:58.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:37:58 smithi090 ceph-mon[29180]: pgmap v16017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:58.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:37:58 smithi119 ceph-mon[32752]: pgmap v16017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:37:58.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:37:58 smithi167 ceph-mon[32751]: pgmap v16017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:00.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:00 smithi090 ceph-mon[29180]: pgmap v16018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:00.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:00 smithi119 ceph-mon[32752]: pgmap v16018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:00.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:00 smithi167 ceph-mon[32751]: pgmap v16018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:01.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28962.431832 secs 2024-04-27T02:38:01.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28962.431832 secs 2024-04-27T02:38:01.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28962.431832 secs 2024-04-27T02:38:02.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:02 smithi090 ceph-mon[29180]: pgmap v16019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:02.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28963.270057 secs 2024-04-27T02:38:02.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:02 smithi119 ceph-mon[32752]: pgmap v16019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:02.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28963.270057 secs 2024-04-27T02:38:02.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:02 smithi167 ceph-mon[32751]: pgmap v16019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:02.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28963.270057 secs 2024-04-27T02:38:04.329 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:04 smithi119 ceph-mon[32752]: pgmap v16020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:04.343 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:04 smithi167 ceph-mon[32751]: pgmap v16020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:04.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:04 smithi090 ceph-mon[29180]: pgmap v16020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:06.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:06 smithi090 ceph-mon[29180]: pgmap v16021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:06.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28967.431893 secs 2024-04-27T02:38:06.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:06 smithi119 ceph-mon[32752]: pgmap v16021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:06.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28967.431893 secs 2024-04-27T02:38:06.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:06 smithi167 ceph-mon[32751]: pgmap v16021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:06.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28967.431893 secs 2024-04-27T02:38:07.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28968.270190 secs 2024-04-27T02:38:07.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28968.270190 secs 2024-04-27T02:38:07.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28968.270190 secs 2024-04-27T02:38:08.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:08 smithi090 ceph-mon[29180]: pgmap v16022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:08.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:08 smithi119 ceph-mon[32752]: pgmap v16022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:08.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:08 smithi167 ceph-mon[32751]: pgmap v16022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:10.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:10 smithi090 ceph-mon[29180]: pgmap v16023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:10.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:10 smithi119 ceph-mon[32752]: pgmap v16023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:10.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:10 smithi167 ceph-mon[32751]: pgmap v16023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:11.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28972.431972 secs 2024-04-27T02:38:11.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28972.431972 secs 2024-04-27T02:38:11.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28972.431972 secs 2024-04-27T02:38:12.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:12 smithi090 ceph-mon[29180]: pgmap v16024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:12.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28973.270300 secs 2024-04-27T02:38:12.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:12 smithi119 ceph-mon[32752]: pgmap v16024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:12.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28973.270300 secs 2024-04-27T02:38:12.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:12 smithi167 ceph-mon[32751]: pgmap v16024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:12.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28973.270300 secs 2024-04-27T02:38:14.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:14 smithi090 ceph-mon[29180]: pgmap v16025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:14.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:14 smithi119 ceph-mon[32752]: pgmap v16025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:14.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:14 smithi167 ceph-mon[32751]: pgmap v16025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:16.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:16 smithi090 ceph-mon[29180]: pgmap v16026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:16.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28977.432052 secs 2024-04-27T02:38:16.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:16 smithi119 ceph-mon[32752]: pgmap v16026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:16.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28977.432052 secs 2024-04-27T02:38:16.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:16 smithi167 ceph-mon[32751]: pgmap v16026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:16.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28977.432052 secs 2024-04-27T02:38:17.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28978.270459 secs 2024-04-27T02:38:17.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28978.270459 secs 2024-04-27T02:38:17.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28978.270459 secs 2024-04-27T02:38:18.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:18 smithi090 ceph-mon[29180]: pgmap v16027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:18.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:18 smithi119 ceph-mon[32752]: pgmap v16027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:18.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:18 smithi167 ceph-mon[32751]: pgmap v16027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:19.349 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:38:19.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:38:19.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:38:20.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:20 smithi119 ceph-mon[32752]: pgmap v16028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:20.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:38:20.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:38:20.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:38:20.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:20 smithi167 ceph-mon[32751]: pgmap v16028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:20.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:38:20.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:38:20.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:38:20.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:20 smithi090 ceph-mon[29180]: pgmap v16028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:20.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:38:20.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:38:20.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:38:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:21 smithi119 ceph-mon[32752]: pgmap v16029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:38:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:21 smithi119 ceph-mon[32752]: pgmap v16030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28982.432110 secs 2024-04-27T02:38:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:21 smithi167 ceph-mon[32751]: pgmap v16029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:38:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:21 smithi167 ceph-mon[32751]: pgmap v16030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28982.432110 secs 2024-04-27T02:38:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:21 smithi090 ceph-mon[29180]: pgmap v16029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:38:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:21 smithi090 ceph-mon[29180]: pgmap v16030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28982.432110 secs 2024-04-27T02:38:22.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28983.270520 secs 2024-04-27T02:38:22.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28983.270520 secs 2024-04-27T02:38:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28983.270520 secs 2024-04-27T02:38:23.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:23 smithi119 ceph-mon[32752]: pgmap v16031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:23.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:23 smithi167 ceph-mon[32751]: pgmap v16031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:23.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:23 smithi090 ceph-mon[29180]: pgmap v16031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:25.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:25 smithi119 ceph-mon[32752]: pgmap v16032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:25.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:25 smithi167 ceph-mon[32751]: pgmap v16032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:25.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:25 smithi090 ceph-mon[29180]: pgmap v16032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:26.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28987.432193 secs 2024-04-27T02:38:26.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28987.432193 secs 2024-04-27T02:38:26.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28987.432193 secs 2024-04-27T02:38:27.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:27 smithi119 ceph-mon[32752]: pgmap v16033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:27.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28988.270639 secs 2024-04-27T02:38:27.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:27 smithi167 ceph-mon[32751]: pgmap v16033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:27.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28988.270639 secs 2024-04-27T02:38:27.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:27 smithi090 ceph-mon[29180]: pgmap v16033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:38:27.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28988.270639 secs 2024-04-27T02:38:29.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:29 smithi119 ceph-mon[32752]: pgmap v16034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:38:29.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:29 smithi167 ceph-mon[32751]: pgmap v16034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:38:29.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:29 smithi090 ceph-mon[29180]: pgmap v16034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:38:31.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:31 smithi119 ceph-mon[32752]: pgmap v16035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:38:31.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28992.432250 secs 2024-04-27T02:38:31.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:31 smithi167 ceph-mon[32751]: pgmap v16035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:38:31.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28992.432250 secs 2024-04-27T02:38:31.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:31 smithi090 ceph-mon[29180]: pgmap v16035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:38:31.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28992.432250 secs 2024-04-27T02:38:32.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28993.270735 secs 2024-04-27T02:38:32.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28993.270735 secs 2024-04-27T02:38:32.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28993.270735 secs 2024-04-27T02:38:33.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:33 smithi119 ceph-mon[32752]: pgmap v16036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:33.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:33 smithi167 ceph-mon[32751]: pgmap v16036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:33.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:33 smithi090 ceph-mon[29180]: pgmap v16036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:35.401 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:35 smithi090 ceph-mon[29180]: pgmap v16037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:35.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:35 smithi119 ceph-mon[32752]: pgmap v16037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:35.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:35 smithi167 ceph-mon[32751]: pgmap v16037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:36.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 28997.432335 secs 2024-04-27T02:38:36.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 28997.432335 secs 2024-04-27T02:38:36.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 28997.432335 secs 2024-04-27T02:38:37.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:37 smithi119 ceph-mon[32752]: pgmap v16038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:37.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 28998.270797 secs 2024-04-27T02:38:37.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:37 smithi167 ceph-mon[32751]: pgmap v16038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:37.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 28998.270797 secs 2024-04-27T02:38:37.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:37 smithi090 ceph-mon[29180]: pgmap v16038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:37.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 28998.270797 secs 2024-04-27T02:38:39.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:39 smithi119 ceph-mon[32752]: pgmap v16039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:39.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:39 smithi167 ceph-mon[32751]: pgmap v16039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:39.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:39 smithi090 ceph-mon[29180]: pgmap v16039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:41.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:41 smithi119 ceph-mon[32752]: pgmap v16040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:41.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29002.432420 secs 2024-04-27T02:38:41.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:41 smithi167 ceph-mon[32751]: pgmap v16040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:41.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29002.432420 secs 2024-04-27T02:38:41.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:41 smithi090 ceph-mon[29180]: pgmap v16040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:41.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29002.432420 secs 2024-04-27T02:38:42.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29003.270919 secs 2024-04-27T02:38:42.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29003.270919 secs 2024-04-27T02:38:42.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29003.270919 secs 2024-04-27T02:38:43.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:43 smithi119 ceph-mon[32752]: pgmap v16041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:43.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:43 smithi167 ceph-mon[32751]: pgmap v16041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:43.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:43 smithi090 ceph-mon[29180]: pgmap v16041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:45.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:45 smithi119 ceph-mon[32752]: pgmap v16042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:45.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:45 smithi167 ceph-mon[32751]: pgmap v16042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:45.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:45 smithi090 ceph-mon[29180]: pgmap v16042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:46.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29007.432504 secs 2024-04-27T02:38:46.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29007.432504 secs 2024-04-27T02:38:46.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29007.432504 secs 2024-04-27T02:38:47.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:47 smithi119 ceph-mon[32752]: pgmap v16043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:47.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29008.271037 secs 2024-04-27T02:38:47.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:47 smithi167 ceph-mon[32751]: pgmap v16043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:47.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29008.271037 secs 2024-04-27T02:38:47.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:47 smithi090 ceph-mon[29180]: pgmap v16043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:47.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29008.271037 secs 2024-04-27T02:38:49.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:49 smithi119 ceph-mon[32752]: pgmap v16044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:49.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:49 smithi167 ceph-mon[32751]: pgmap v16044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:49.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:49 smithi090 ceph-mon[29180]: pgmap v16044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:51.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:51 smithi119 ceph-mon[32752]: pgmap v16045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:51.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29012.432590 secs 2024-04-27T02:38:51.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:51 smithi167 ceph-mon[32751]: pgmap v16045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:51.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29012.432590 secs 2024-04-27T02:38:51.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:51 smithi090 ceph-mon[29180]: pgmap v16045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:51.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29012.432590 secs 2024-04-27T02:38:52.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29013.271172 secs 2024-04-27T02:38:52.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29013.271172 secs 2024-04-27T02:38:52.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29013.271172 secs 2024-04-27T02:38:53.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:53 smithi119 ceph-mon[32752]: pgmap v16046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:53.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:53 smithi167 ceph-mon[32751]: pgmap v16046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:53.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:53 smithi090 ceph-mon[29180]: pgmap v16046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:55.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:55 smithi119 ceph-mon[32752]: pgmap v16047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:55.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:55 smithi167 ceph-mon[32751]: pgmap v16047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:55.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:55 smithi090 ceph-mon[29180]: pgmap v16047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:38:56.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29017.432671 secs 2024-04-27T02:38:56.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29017.432671 secs 2024-04-27T02:38:56.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29017.432671 secs 2024-04-27T02:38:57.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:57 smithi119 ceph-mon[32752]: pgmap v16048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:57.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29018.271285 secs 2024-04-27T02:38:57.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:57 smithi167 ceph-mon[32751]: pgmap v16048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:57.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29018.271285 secs 2024-04-27T02:38:57.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:57 smithi090 ceph-mon[29180]: pgmap v16048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:57.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29018.271285 secs 2024-04-27T02:38:59.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:38:59 smithi119 ceph-mon[32752]: pgmap v16049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:59.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:38:59 smithi167 ceph-mon[32751]: pgmap v16049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:38:59.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:38:59 smithi090 ceph-mon[29180]: pgmap v16049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:01.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:01 smithi119 ceph-mon[32752]: pgmap v16050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:01.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29022.432748 secs 2024-04-27T02:39:01.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:01 smithi167 ceph-mon[32751]: pgmap v16050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:01.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29022.432748 secs 2024-04-27T02:39:01.578 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:01 smithi090 ceph-mon[29180]: pgmap v16050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:01.579 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29022.432748 secs 2024-04-27T02:39:02.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29023.271401 secs 2024-04-27T02:39:02.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29023.271401 secs 2024-04-27T02:39:02.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29023.271401 secs 2024-04-27T02:39:03.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:03 smithi119 ceph-mon[32752]: pgmap v16051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:03.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:03 smithi167 ceph-mon[32751]: pgmap v16051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:03.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:03 smithi090 ceph-mon[29180]: pgmap v16051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:05.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:05 smithi119 ceph-mon[32752]: pgmap v16052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:05.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:05 smithi167 ceph-mon[32751]: pgmap v16052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:05.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:05 smithi090 ceph-mon[29180]: pgmap v16052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:06.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29027.432844 secs 2024-04-27T02:39:06.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29027.432844 secs 2024-04-27T02:39:06.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29027.432844 secs 2024-04-27T02:39:07.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:07 smithi119 ceph-mon[32752]: pgmap v16053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:07.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29028.271495 secs 2024-04-27T02:39:07.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:07 smithi167 ceph-mon[32751]: pgmap v16053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:07.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29028.271495 secs 2024-04-27T02:39:07.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:07 smithi090 ceph-mon[29180]: pgmap v16053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:07.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29028.271495 secs 2024-04-27T02:39:09.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:09 smithi119 ceph-mon[32752]: pgmap v16054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:09.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:09 smithi167 ceph-mon[32751]: pgmap v16054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:09.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:09 smithi090 ceph-mon[29180]: pgmap v16054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:11.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:11 smithi119 ceph-mon[32752]: pgmap v16055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:11.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:11 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29032.432931 secs 2024-04-27T02:39:11.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:11 smithi167 ceph-mon[32751]: pgmap v16055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:11.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:11 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29032.432931 secs 2024-04-27T02:39:11.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:11 smithi090 ceph-mon[29180]: pgmap v16055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:11 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29032.432931 secs 2024-04-27T02:39:12.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29033.271610 secs 2024-04-27T02:39:12.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29033.271610 secs 2024-04-27T02:39:12.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29033.271610 secs 2024-04-27T02:39:13.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:13 smithi119 ceph-mon[32752]: pgmap v16056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:13.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:13 smithi167 ceph-mon[32751]: pgmap v16056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:13.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:13 smithi090 ceph-mon[29180]: pgmap v16056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:15.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:15 smithi119 ceph-mon[32752]: pgmap v16057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:15.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:15 smithi167 ceph-mon[32751]: pgmap v16057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:15.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:15 smithi090 ceph-mon[29180]: pgmap v16057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:16.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:16 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29037.433016 secs 2024-04-27T02:39:16.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:16 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29037.433016 secs 2024-04-27T02:39:16.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:16 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29037.433016 secs 2024-04-27T02:39:17.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:17 smithi119 ceph-mon[32752]: pgmap v16058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:17.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29038.271711 secs 2024-04-27T02:39:17.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:17 smithi167 ceph-mon[32751]: pgmap v16058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:17.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29038.271711 secs 2024-04-27T02:39:17.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:17 smithi090 ceph-mon[29180]: pgmap v16058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:17.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29038.271711 secs 2024-04-27T02:39:19.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:19 smithi119 ceph-mon[32752]: pgmap v16059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:19.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:19 smithi167 ceph-mon[32751]: pgmap v16059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:19.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:19 smithi090 ceph-mon[29180]: pgmap v16059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:20.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:39:20.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:39:20.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:39:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:21 smithi119 ceph-mon[32752]: pgmap v16060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:21 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29042.433081 secs 2024-04-27T02:39:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:39:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:39:21.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:39:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:21 smithi167 ceph-mon[32751]: pgmap v16060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:21 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29042.433081 secs 2024-04-27T02:39:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:39:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:39:21.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:39:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:21 smithi090 ceph-mon[29180]: pgmap v16060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:21 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29042.433081 secs 2024-04-27T02:39:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:39:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:39:21.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:39:22.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:22 smithi119 ceph-mon[32752]: pgmap v16061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:39:22.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:22 smithi119 ceph-mon[32752]: pgmap v16062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:22.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29043.271860 secs 2024-04-27T02:39:22.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:22 smithi167 ceph-mon[32751]: pgmap v16061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:39:22.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:22 smithi167 ceph-mon[32751]: pgmap v16062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:22.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29043.271860 secs 2024-04-27T02:39:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:22 smithi090 ceph-mon[29180]: pgmap v16061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:39:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:22 smithi090 ceph-mon[29180]: pgmap v16062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:22.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29043.271860 secs 2024-04-27T02:39:24.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:24 smithi119 ceph-mon[32752]: pgmap v16063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:39:24.540 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:24 smithi167 ceph-mon[32751]: pgmap v16063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:39:24.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:24 smithi090 ceph-mon[29180]: pgmap v16063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:39:26.526 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:26 smithi119 ceph-mon[32752]: pgmap v16064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:26.527 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:26 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29047.433164 secs 2024-04-27T02:39:26.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:26 smithi167 ceph-mon[32751]: pgmap v16064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:26.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:26 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29047.433164 secs 2024-04-27T02:39:26.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:26 smithi090 ceph-mon[29180]: pgmap v16064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:26.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:26 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29047.433164 secs 2024-04-27T02:39:27.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29048.271996 secs 2024-04-27T02:39:27.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29048.271996 secs 2024-04-27T02:39:27.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29048.271996 secs 2024-04-27T02:39:28.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:28 smithi167 ceph-mon[32751]: pgmap v16065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:28.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:28 smithi090 ceph-mon[29180]: pgmap v16065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:28.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:28 smithi119 ceph-mon[32752]: pgmap v16065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:30.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:30 smithi167 ceph-mon[32751]: pgmap v16066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:30.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:30 smithi090 ceph-mon[29180]: pgmap v16066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:30.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:30 smithi119 ceph-mon[32752]: pgmap v16066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:39:31.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:31 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29052.433248 secs 2024-04-27T02:39:31.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:31 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29052.433248 secs 2024-04-27T02:39:31.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:31 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29052.433248 secs 2024-04-27T02:39:32.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:32 smithi090 ceph-mon[29180]: pgmap v16067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:39:32.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29053.272065 secs 2024-04-27T02:39:32.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:32 smithi119 ceph-mon[32752]: pgmap v16067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:39:32.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29053.272065 secs 2024-04-27T02:39:32.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:32 smithi167 ceph-mon[32751]: pgmap v16067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:39:32.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29053.272065 secs 2024-04-27T02:39:34.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:34 smithi090 ceph-mon[29180]: pgmap v16068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:34.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:34 smithi119 ceph-mon[32752]: pgmap v16068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:34.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:34 smithi167 ceph-mon[32751]: pgmap v16068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:36.579 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:36 smithi119 ceph-mon[32752]: pgmap v16069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:36.579 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:36 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29057.433330 secs 2024-04-27T02:39:36.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:36 smithi167 ceph-mon[32751]: pgmap v16069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:36.594 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:36 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29057.433330 secs 2024-04-27T02:39:36.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:36 smithi090 ceph-mon[29180]: pgmap v16069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:36.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:36 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29057.433330 secs 2024-04-27T02:39:37.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29058.272168 secs 2024-04-27T02:39:37.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29058.272168 secs 2024-04-27T02:39:37.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29058.272168 secs 2024-04-27T02:39:38.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:38 smithi090 ceph-mon[29180]: pgmap v16070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:38.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:38 smithi119 ceph-mon[32752]: pgmap v16070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:38.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:38 smithi167 ceph-mon[32751]: pgmap v16070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:40.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:40 smithi090 ceph-mon[29180]: pgmap v16071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:40.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:40 smithi119 ceph-mon[32752]: pgmap v16071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:40 smithi167 ceph-mon[32751]: pgmap v16071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:39:41.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:41 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29062.433392 secs 2024-04-27T02:39:41.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:41 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29062.433392 secs 2024-04-27T02:39:41.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:41 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29062.433392 secs 2024-04-27T02:39:42.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:42 smithi090 ceph-mon[29180]: pgmap v16072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:42.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29063.272282 secs 2024-04-27T02:39:42.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:42 smithi119 ceph-mon[32752]: pgmap v16072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:42.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29063.272282 secs 2024-04-27T02:39:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:42 smithi167 ceph-mon[32751]: pgmap v16072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29063.272282 secs 2024-04-27T02:39:44.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:44 smithi090 ceph-mon[29180]: pgmap v16073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:44.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:44 smithi119 ceph-mon[32752]: pgmap v16073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:44.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:44 smithi167 ceph-mon[32751]: pgmap v16073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:46.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:46 smithi090 ceph-mon[29180]: pgmap v16074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:46.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:46 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29067.433449 secs 2024-04-27T02:39:46.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:46 smithi119 ceph-mon[32752]: pgmap v16074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:46.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:46 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29067.433449 secs 2024-04-27T02:39:46.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:46 smithi167 ceph-mon[32751]: pgmap v16074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:46.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:46 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29067.433449 secs 2024-04-27T02:39:47.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29068.272373 secs 2024-04-27T02:39:47.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29068.272373 secs 2024-04-27T02:39:47.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29068.272373 secs 2024-04-27T02:39:48.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:48 smithi090 ceph-mon[29180]: pgmap v16075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:48.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:48 smithi119 ceph-mon[32752]: pgmap v16075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:48.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:48 smithi167 ceph-mon[32751]: pgmap v16075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:50.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:50 smithi090 ceph-mon[29180]: pgmap v16076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:50.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:50 smithi119 ceph-mon[32752]: pgmap v16076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:50.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:50 smithi167 ceph-mon[32751]: pgmap v16076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:51.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:51 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29072.433533 secs 2024-04-27T02:39:51.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:51 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29072.433533 secs 2024-04-27T02:39:51.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:51 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29072.433533 secs 2024-04-27T02:39:52.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:52 smithi090 ceph-mon[29180]: pgmap v16077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:52.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29073.272469 secs 2024-04-27T02:39:52.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:52 smithi119 ceph-mon[32752]: pgmap v16077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:52.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29073.272469 secs 2024-04-27T02:39:52.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:52 smithi167 ceph-mon[32751]: pgmap v16077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:52.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29073.272469 secs 2024-04-27T02:39:54.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:54 smithi090 ceph-mon[29180]: pgmap v16078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:54.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:54 smithi119 ceph-mon[32752]: pgmap v16078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:54.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:54 smithi167 ceph-mon[32751]: pgmap v16078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T02:39:56.598 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:56 smithi090 ceph-mon[29180]: pgmap v16079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:56.599 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:56 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29077.433600 secs 2024-04-27T02:39:56.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:56 smithi119 ceph-mon[32752]: pgmap v16079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:56.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:56 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29077.433600 secs 2024-04-27T02:39:56.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:56 smithi167 ceph-mon[32751]: pgmap v16079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:39:56.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:56 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29077.433600 secs 2024-04-27T02:39:57.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29078.272594 secs 2024-04-27T02:39:57.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29078.272594 secs 2024-04-27T02:39:57.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29078.272594 secs 2024-04-27T02:39:58.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:39:58 smithi119 ceph-mon[32752]: pgmap v16080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:58.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:39:58 smithi167 ceph-mon[32751]: pgmap v16080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:39:58.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:39:58 smithi090 ceph-mon[29180]: pgmap v16080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:00.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:00 smithi119 ceph-mon[32752]: pgmap v16081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:00.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:40:00.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:00 smithi167 ceph-mon[32751]: pgmap v16081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:00.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:40:00.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:00 smithi090 ceph-mon[29180]: pgmap v16081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:00.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:40:01.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:01 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29082.433658 secs 2024-04-27T02:40:01.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:01 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29082.433658 secs 2024-04-27T02:40:01.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:01 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29082.433658 secs 2024-04-27T02:40:02.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:02 smithi119 ceph-mon[32752]: pgmap v16082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:02.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29083.272733 secs 2024-04-27T02:40:02.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:02 smithi167 ceph-mon[32751]: pgmap v16082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:02.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29083.272733 secs 2024-04-27T02:40:02.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:02 smithi090 ceph-mon[29180]: pgmap v16082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:02.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29083.272733 secs 2024-04-27T02:40:04.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:04 smithi119 ceph-mon[32752]: pgmap v16083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:04.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:04 smithi167 ceph-mon[32751]: pgmap v16083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:04.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:04 smithi090 ceph-mon[29180]: pgmap v16083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:06.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:06 smithi090 ceph-mon[29180]: pgmap v16084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:06.651 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:06 smithi090 ceph-mon[29180]: 37 slow requests, 0 included below; oldest blocked for > 29087.433741 secs 2024-04-27T02:40:06.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:06 smithi119 ceph-mon[32752]: pgmap v16084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:06.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:06 smithi119 ceph-mon[32752]: 37 slow requests, 0 included below; oldest blocked for > 29087.433741 secs 2024-04-27T02:40:06.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:06 smithi167 ceph-mon[32751]: pgmap v16084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:06.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:06 smithi167 ceph-mon[32751]: 37 slow requests, 0 included below; oldest blocked for > 29087.433741 secs 2024-04-27T02:40:07.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29088.272892 secs 2024-04-27T02:40:07.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29088.272892 secs 2024-04-27T02:40:07.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29088.272892 secs 2024-04-27T02:40:08.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:08 smithi119 ceph-mon[32752]: pgmap v16085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:08.790 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:08 smithi090 ceph-mon[29180]: pgmap v16085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:08.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:08 smithi167 ceph-mon[32751]: pgmap v16085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:09.099 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:40:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:40:08.817+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:40:09.099 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:40:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:40:08.817+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:40:09.099 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:40:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:40:08.817+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:40:09.099 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:40:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:40:08.817+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:40:09.099 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:40:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:40:08.817+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:40:09.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:09 smithi119 ceph-mon[32752]: pgmap v16086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:09.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:09 smithi167 ceph-mon[32751]: pgmap v16086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:09.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:09 smithi090 ceph-mon[29180]: pgmap v16086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:12.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:11 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29092.433822 secs 2024-04-27T02:40:12.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:11 smithi119 ceph-mon[32752]: pgmap v16087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:12.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:11 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29092.433822 secs 2024-04-27T02:40:12.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:11 smithi167 ceph-mon[32751]: pgmap v16087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:12.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:11 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29092.433822 secs 2024-04-27T02:40:12.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:11 smithi090 ceph-mon[29180]: pgmap v16087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:13.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29093.272974 secs 2024-04-27T02:40:13.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29093.272974 secs 2024-04-27T02:40:13.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29093.272974 secs 2024-04-27T02:40:14.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:13 smithi119 ceph-mon[32752]: pgmap v16088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:14.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:13 smithi167 ceph-mon[32751]: pgmap v16088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:14.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:13 smithi090 ceph-mon[29180]: pgmap v16088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:16.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:15 smithi119 ceph-mon[32752]: pgmap v16089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T02:40:16.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:40:16.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:40:16.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:15 smithi167 ceph-mon[32751]: pgmap v16089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T02:40:16.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:40:16.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:40:16.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:15 smithi090 ceph-mon[29180]: pgmap v16089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T02:40:16.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:40:16.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:40:17.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:16 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29097.433906 secs 2024-04-27T02:40:17.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:16 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29097.433906 secs 2024-04-27T02:40:17.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:16 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29097.433906 secs 2024-04-27T02:40:18.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29098.273101 secs 2024-04-27T02:40:18.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:17 smithi119 ceph-mon[32752]: pgmap v16090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T02:40:18.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29098.273101 secs 2024-04-27T02:40:18.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:17 smithi167 ceph-mon[32751]: pgmap v16090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T02:40:18.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29098.273101 secs 2024-04-27T02:40:18.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:17 smithi090 ceph-mon[29180]: pgmap v16090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T02:40:20.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:19 smithi119 ceph-mon[32752]: pgmap v16091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T02:40:20.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:19 smithi167 ceph-mon[32751]: pgmap v16091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T02:40:20.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:19 smithi090 ceph-mon[29180]: pgmap v16091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T02:40:22.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29102.433989 secs 2024-04-27T02:40:22.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: pgmap v16092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T02:40:22.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:40:22.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:40:22.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:40:22.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: pgmap v16093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-27T02:40:22.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: pgmap v16094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-27T02:40:22.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:40:22.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29102.433989 secs 2024-04-27T02:40:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: pgmap v16092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T02:40:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:40:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:40:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:40:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: pgmap v16093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-27T02:40:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: pgmap v16094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-27T02:40:22.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:40:22.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29102.433989 secs 2024-04-27T02:40:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: pgmap v16092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T02:40:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:40:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:40:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:40:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: pgmap v16093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 380 B/s wr, 14 op/s 2024-04-27T02:40:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: pgmap v16094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 467 B/s wr, 18 op/s 2024-04-27T02:40:22.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:40:23.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29103.273180 secs 2024-04-27T02:40:23.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29103.273180 secs 2024-04-27T02:40:23.098 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29103.273180 secs 2024-04-27T02:40:24.026 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:23 smithi119 ceph-mon[32752]: pgmap v16095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:24.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:23 smithi167 ceph-mon[32751]: pgmap v16095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:24.099 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:23 smithi090 ceph-mon[29180]: pgmap v16095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:26.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:26 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29107.434052 secs 2024-04-27T02:40:26.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:26 smithi119 ceph-mon[32752]: pgmap v16096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:26.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:26 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29107.434052 secs 2024-04-27T02:40:26.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:26 smithi167 ceph-mon[32751]: pgmap v16096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:26.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:26 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29107.434052 secs 2024-04-27T02:40:26.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:26 smithi090 ceph-mon[29180]: pgmap v16096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:27.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29108.273305 secs 2024-04-27T02:40:27.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29108.273305 secs 2024-04-27T02:40:27.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29108.273305 secs 2024-04-27T02:40:28.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:28 smithi119 ceph-mon[32752]: pgmap v16097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:40:28.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:28 smithi167 ceph-mon[32751]: pgmap v16097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:40:28.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:28 smithi090 ceph-mon[29180]: pgmap v16097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:40:30.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:30 smithi119 ceph-mon[32752]: pgmap v16098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:30.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:30 smithi167 ceph-mon[32751]: pgmap v16098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:30.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:30 smithi090 ceph-mon[29180]: pgmap v16098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:40:31.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:31 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29112.434131 secs 2024-04-27T02:40:31.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:31 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29112.434131 secs 2024-04-27T02:40:31.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:31 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29112.434131 secs 2024-04-27T02:40:32.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29113.273461 secs 2024-04-27T02:40:32.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:32 smithi119 ceph-mon[32752]: pgmap v16099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:40:32.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29113.273461 secs 2024-04-27T02:40:32.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:32 smithi167 ceph-mon[32751]: pgmap v16099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:40:32.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29113.273461 secs 2024-04-27T02:40:32.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:32 smithi090 ceph-mon[29180]: pgmap v16099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:40:33.738 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:33 smithi090 ceph-mon[29180]: pgmap v16100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:33.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:33 smithi119 ceph-mon[32752]: pgmap v16100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:33.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:33 smithi167 ceph-mon[32751]: pgmap v16100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:36.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:36 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29117.434210 secs 2024-04-27T02:40:36.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:36 smithi119 ceph-mon[32752]: pgmap v16101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:36.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:36 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29117.434210 secs 2024-04-27T02:40:36.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:36 smithi167 ceph-mon[32751]: pgmap v16101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:36.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:36 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29117.434210 secs 2024-04-27T02:40:36.803 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:36 smithi090 ceph-mon[29180]: pgmap v16101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:37.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29118.273555 secs 2024-04-27T02:40:37.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29118.273555 secs 2024-04-27T02:40:37.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29118.273555 secs 2024-04-27T02:40:38.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:38 smithi119 ceph-mon[32752]: pgmap v16102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:38.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:38 smithi167 ceph-mon[32751]: pgmap v16102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:38.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:38 smithi090 ceph-mon[29180]: pgmap v16102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:40.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:40 smithi119 ceph-mon[32752]: pgmap v16103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:40.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:40 smithi167 ceph-mon[32751]: pgmap v16103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:40.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:40 smithi090 ceph-mon[29180]: pgmap v16103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:41.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:41 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29122.434267 secs 2024-04-27T02:40:41.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:41 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29122.434267 secs 2024-04-27T02:40:41.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:41 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29122.434267 secs 2024-04-27T02:40:42.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29123.273609 secs 2024-04-27T02:40:42.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:42 smithi119 ceph-mon[32752]: pgmap v16104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:42.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29123.273609 secs 2024-04-27T02:40:42.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:42 smithi167 ceph-mon[32751]: pgmap v16104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:42.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29123.273609 secs 2024-04-27T02:40:42.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:42 smithi090 ceph-mon[29180]: pgmap v16104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:43.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:43 smithi119 ceph-mon[32752]: pgmap v16105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:43.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:43 smithi167 ceph-mon[32751]: pgmap v16105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:43.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:43 smithi090 ceph-mon[29180]: pgmap v16105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:46.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:46 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29127.434349 secs 2024-04-27T02:40:46.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:46 smithi119 ceph-mon[32752]: pgmap v16106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:46.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:46 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29127.434349 secs 2024-04-27T02:40:46.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:46 smithi167 ceph-mon[32751]: pgmap v16106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:46.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:46 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29127.434349 secs 2024-04-27T02:40:46.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:46 smithi090 ceph-mon[29180]: pgmap v16106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:47.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29128.273736 secs 2024-04-27T02:40:47.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29128.273736 secs 2024-04-27T02:40:47.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29128.273736 secs 2024-04-27T02:40:48.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:48 smithi119 ceph-mon[32752]: pgmap v16107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:48.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:48 smithi167 ceph-mon[32751]: pgmap v16107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:48.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:48 smithi090 ceph-mon[29180]: pgmap v16107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:50.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:50 smithi119 ceph-mon[32752]: pgmap v16108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:50.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:50 smithi167 ceph-mon[32751]: pgmap v16108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:50.848 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:50 smithi090 ceph-mon[29180]: pgmap v16108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:51.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:51 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29132.434433 secs 2024-04-27T02:40:51.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:51 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29132.434433 secs 2024-04-27T02:40:51.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:51 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29132.434433 secs 2024-04-27T02:40:52.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29133.273834 secs 2024-04-27T02:40:52.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:52 smithi119 ceph-mon[32752]: pgmap v16109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:52.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29133.273834 secs 2024-04-27T02:40:52.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:52 smithi167 ceph-mon[32751]: pgmap v16109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:52.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29133.273834 secs 2024-04-27T02:40:52.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:52 smithi090 ceph-mon[29180]: pgmap v16109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:53.776 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:53 smithi119 ceph-mon[32752]: pgmap v16110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:53.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:53 smithi167 ceph-mon[32751]: pgmap v16110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:53.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:53 smithi090 ceph-mon[29180]: pgmap v16110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:40:56.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:56 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29137.434517 secs 2024-04-27T02:40:56.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:56 smithi119 ceph-mon[32752]: pgmap v16111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:56.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:56 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29137.434517 secs 2024-04-27T02:40:56.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:56 smithi167 ceph-mon[32751]: pgmap v16111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:56.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:56 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29137.434517 secs 2024-04-27T02:40:56.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:56 smithi090 ceph-mon[29180]: pgmap v16111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:57.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29138.274007 secs 2024-04-27T02:40:57.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29138.274007 secs 2024-04-27T02:40:57.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29138.274007 secs 2024-04-27T02:40:58.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:40:58 smithi119 ceph-mon[32752]: pgmap v16112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:58.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:40:58 smithi167 ceph-mon[32751]: pgmap v16112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:40:58.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:40:58 smithi090 ceph-mon[29180]: pgmap v16112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:00.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:00 smithi119 ceph-mon[32752]: pgmap v16113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:00.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:00 smithi167 ceph-mon[32751]: pgmap v16113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:00.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:00 smithi090 ceph-mon[29180]: pgmap v16113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:01.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:01 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29142.434603 secs 2024-04-27T02:41:01.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:01 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29142.434603 secs 2024-04-27T02:41:01.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:01 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29142.434603 secs 2024-04-27T02:41:02.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29143.274119 secs 2024-04-27T02:41:02.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:02 smithi119 ceph-mon[32752]: pgmap v16114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:02.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29143.274119 secs 2024-04-27T02:41:02.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:02 smithi167 ceph-mon[32751]: pgmap v16114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:02.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29143.274119 secs 2024-04-27T02:41:02.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:02 smithi090 ceph-mon[29180]: pgmap v16114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:03.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:03 smithi119 ceph-mon[32752]: pgmap v16115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:41:03.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:03 smithi167 ceph-mon[32751]: pgmap v16115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:41:03.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:03 smithi090 ceph-mon[29180]: pgmap v16115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T02:41:06.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:06 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29147.434681 secs 2024-04-27T02:41:06.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:06 smithi090 ceph-mon[29180]: pgmap v16116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:06.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:06 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29147.434681 secs 2024-04-27T02:41:06.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:06 smithi119 ceph-mon[32752]: pgmap v16116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:06.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:06 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29147.434681 secs 2024-04-27T02:41:06.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:06 smithi167 ceph-mon[32751]: pgmap v16116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:07.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:07 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 29148.274239 secs 2024-04-27T02:41:07.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:07 smithi119 ceph-mon[32752]: slow request 11520.414350 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:41:07.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:07 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 29148.274239 secs 2024-04-27T02:41:07.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:07 smithi167 ceph-mon[32751]: slow request 11520.414350 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:41:07.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:07 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 29148.274239 secs 2024-04-27T02:41:07.849 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:07 smithi090 ceph-mon[29180]: slow request 11520.414350 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:41:08.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:08 smithi119 ceph-mon[32752]: pgmap v16117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:08.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:08 smithi167 ceph-mon[32751]: pgmap v16117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:08.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:08 smithi090 ceph-mon[29180]: pgmap v16117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:09.777 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:09 smithi119 ceph-mon[32752]: pgmap v16118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:41:09.791 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:09 smithi167 ceph-mon[32751]: pgmap v16118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:41:09.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:09 smithi090 ceph-mon[29180]: pgmap v16118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:41:11.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:11 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29152.434765 secs 2024-04-27T02:41:11.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:11 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29152.434765 secs 2024-04-27T02:41:11.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:11 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29152.434765 secs 2024-04-27T02:41:12.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29153.274363 secs 2024-04-27T02:41:12.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:12 smithi119 ceph-mon[32752]: pgmap v16119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:12.541 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29153.274363 secs 2024-04-27T02:41:12.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:12 smithi167 ceph-mon[32751]: pgmap v16119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29153.274363 secs 2024-04-27T02:41:12.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:12 smithi090 ceph-mon[29180]: pgmap v16119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:14.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:14 smithi119 ceph-mon[32752]: pgmap v16120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:41:14.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:14 smithi167 ceph-mon[32751]: pgmap v16120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:41:14.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:14 smithi090 ceph-mon[29180]: pgmap v16120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:41:16.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:16 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29157.434846 secs 2024-04-27T02:41:16.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:16 smithi119 ceph-mon[32752]: pgmap v16121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:16.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:16 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29157.434846 secs 2024-04-27T02:41:16.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:16 smithi167 ceph-mon[32751]: pgmap v16121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:16.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:16 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29157.434846 secs 2024-04-27T02:41:16.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:16 smithi090 ceph-mon[29180]: pgmap v16121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:41:17.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29158.274524 secs 2024-04-27T02:41:17.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29158.274524 secs 2024-04-27T02:41:17.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29158.274524 secs 2024-04-27T02:41:18.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:18 smithi119 ceph-mon[32752]: pgmap v16122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:18.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:18 smithi167 ceph-mon[32751]: pgmap v16122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:18.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:18 smithi090 ceph-mon[29180]: pgmap v16122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:19.778 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:19 smithi119 ceph-mon[32752]: pgmap v16123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:19.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:19 smithi167 ceph-mon[32751]: pgmap v16123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:19.850 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:19 smithi090 ceph-mon[29180]: pgmap v16123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:21.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:21 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29162.434900 secs 2024-04-27T02:41:21.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:21 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29162.434900 secs 2024-04-27T02:41:21.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:21 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29162.434900 secs 2024-04-27T02:41:22.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29163.274591 secs 2024-04-27T02:41:22.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:22 smithi119 ceph-mon[32752]: pgmap v16124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:41:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:41:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:41:22.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:41:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29163.274591 secs 2024-04-27T02:41:22.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:22 smithi167 ceph-mon[32751]: pgmap v16124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:41:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:41:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:41:22.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:41:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29163.274591 secs 2024-04-27T02:41:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:22 smithi090 ceph-mon[29180]: pgmap v16124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:41:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:41:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:41:22.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:41:23.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:23 smithi119 ceph-mon[32752]: pgmap v16125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:41:23.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:23 smithi119 ceph-mon[32752]: pgmap v16126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:41:23.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:23 smithi167 ceph-mon[32751]: pgmap v16125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:41:23.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:23 smithi167 ceph-mon[32751]: pgmap v16126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:41:23.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:23 smithi090 ceph-mon[29180]: pgmap v16125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:41:23.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:23 smithi090 ceph-mon[29180]: pgmap v16126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:41:25.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:25 smithi119 ceph-mon[32752]: pgmap v16127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:25.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:25 smithi167 ceph-mon[32751]: pgmap v16127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:25.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:25 smithi090 ceph-mon[29180]: pgmap v16127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:26.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:26 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29167.434990 secs 2024-04-27T02:41:26.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:26 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29167.434990 secs 2024-04-27T02:41:26.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:26 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29167.434990 secs 2024-04-27T02:41:27.528 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29168.274669 secs 2024-04-27T02:41:27.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:27 smithi119 ceph-mon[32752]: pgmap v16128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:27.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29168.274669 secs 2024-04-27T02:41:27.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:27 smithi167 ceph-mon[32751]: pgmap v16128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:27.600 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29168.274669 secs 2024-04-27T02:41:27.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:27 smithi090 ceph-mon[29180]: pgmap v16128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:29.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:29 smithi119 ceph-mon[32752]: pgmap v16129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:41:29.542 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:29 smithi167 ceph-mon[32751]: pgmap v16129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:41:29.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:29 smithi090 ceph-mon[29180]: pgmap v16129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:41:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:31 smithi119 ceph-mon[32752]: pgmap v16130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:31.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:31 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29172.435069 secs 2024-04-27T02:41:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:31 smithi167 ceph-mon[32751]: pgmap v16130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:31.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:31 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29172.435069 secs 2024-04-27T02:41:31.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:31 smithi090 ceph-mon[29180]: pgmap v16130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:41:31.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:31 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29172.435069 secs 2024-04-27T02:41:32.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29173.274788 secs 2024-04-27T02:41:32.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29173.274788 secs 2024-04-27T02:41:32.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29173.274788 secs 2024-04-27T02:41:33.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:33 smithi119 ceph-mon[32752]: pgmap v16131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:41:33.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:33 smithi167 ceph-mon[32751]: pgmap v16131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:41:33.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:33 smithi090 ceph-mon[29180]: pgmap v16131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:41:35.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:35 smithi119 ceph-mon[32752]: pgmap v16132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:35.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:35 smithi167 ceph-mon[32751]: pgmap v16132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:35.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:35 smithi090 ceph-mon[29180]: pgmap v16132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:36.430 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:36 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29177.435134 secs 2024-04-27T02:41:36.444 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:36 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29177.435134 secs 2024-04-27T02:41:36.502 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:36 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29177.435134 secs 2024-04-27T02:41:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29178.274885 secs 2024-04-27T02:41:37.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:37 smithi119 ceph-mon[32752]: pgmap v16133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29178.274885 secs 2024-04-27T02:41:37.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:37 smithi167 ceph-mon[32751]: pgmap v16133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29178.274885 secs 2024-04-27T02:41:37.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:37 smithi090 ceph-mon[29180]: pgmap v16133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:39.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:39 smithi119 ceph-mon[32752]: pgmap v16134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:39.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:39 smithi167 ceph-mon[32751]: pgmap v16134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:39.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:39 smithi090 ceph-mon[29180]: pgmap v16134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:41.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:41 smithi119 ceph-mon[32752]: pgmap v16135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:41.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:41 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29182.435216 secs 2024-04-27T02:41:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:41 smithi167 ceph-mon[32751]: pgmap v16135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:41.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:41 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29182.435216 secs 2024-04-27T02:41:41.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:41 smithi090 ceph-mon[29180]: pgmap v16135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:41.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:41 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29182.435216 secs 2024-04-27T02:41:42.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:42 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 29183.274963 secs 2024-04-27T02:41:42.530 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:42 smithi119 ceph-mon[32752]: slow request 722.366087 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:41:42.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:42 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 29183.274963 secs 2024-04-27T02:41:42.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:42 smithi167 ceph-mon[32751]: slow request 722.366087 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:41:42.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:42 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 29183.274963 secs 2024-04-27T02:41:42.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:42 smithi090 ceph-mon[29180]: slow request 722.366087 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:41:43.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:43 smithi119 ceph-mon[32752]: pgmap v16136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:43.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:43 smithi167 ceph-mon[32751]: pgmap v16136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:43.601 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:43 smithi090 ceph-mon[29180]: pgmap v16136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:45.529 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:45 smithi119 ceph-mon[32752]: pgmap v16137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:45.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:45 smithi167 ceph-mon[32751]: pgmap v16137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:45.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:45 smithi090 ceph-mon[29180]: pgmap v16137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:46.543 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:46 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29187.435274 secs 2024-04-27T02:41:46.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:46 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29187.435274 secs 2024-04-27T02:41:46.779 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:46 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29187.435274 secs 2024-04-27T02:41:47.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29188.275096 secs 2024-04-27T02:41:47.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:47 smithi167 ceph-mon[32751]: pgmap v16138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29188.275096 secs 2024-04-27T02:41:47.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:47 smithi090 ceph-mon[29180]: pgmap v16138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:47.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29188.275096 secs 2024-04-27T02:41:47.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:47 smithi119 ceph-mon[32752]: pgmap v16138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:49.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:49 smithi167 ceph-mon[32751]: pgmap v16139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:49.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:49 smithi090 ceph-mon[29180]: pgmap v16139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:49.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:49 smithi119 ceph-mon[32752]: pgmap v16139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:51.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:51 smithi167 ceph-mon[32751]: pgmap v16140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:51.544 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:51 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29192.435359 secs 2024-04-27T02:41:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:51 smithi090 ceph-mon[29180]: pgmap v16140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:51.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:51 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29192.435359 secs 2024-04-27T02:41:51.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:51 smithi119 ceph-mon[32752]: pgmap v16140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:51.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:51 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29192.435359 secs 2024-04-27T02:41:52.589 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29193.275220 secs 2024-04-27T02:41:52.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29193.275220 secs 2024-04-27T02:41:52.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29193.275220 secs 2024-04-27T02:41:53.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:53 smithi090 ceph-mon[29180]: pgmap v16141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:53.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:53 smithi119 ceph-mon[32752]: pgmap v16141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:53.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:53 smithi167 ceph-mon[32751]: pgmap v16141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:55.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:55 smithi090 ceph-mon[29180]: pgmap v16142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:55.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:55 smithi119 ceph-mon[32752]: pgmap v16142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:55.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:55 smithi167 ceph-mon[32751]: pgmap v16142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:41:56.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:56 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29197.435435 secs 2024-04-27T02:41:56.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:56 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29197.435435 secs 2024-04-27T02:41:56.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:56 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29197.435435 secs 2024-04-27T02:41:57.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29198.275353 secs 2024-04-27T02:41:57.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:57 smithi090 ceph-mon[29180]: pgmap v16143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:57.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29198.275353 secs 2024-04-27T02:41:57.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:57 smithi119 ceph-mon[32752]: pgmap v16143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:57.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29198.275353 secs 2024-04-27T02:41:57.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:57 smithi167 ceph-mon[32751]: pgmap v16143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:59.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:41:59 smithi090 ceph-mon[29180]: pgmap v16144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:59.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:41:59 smithi119 ceph-mon[32752]: pgmap v16144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:41:59.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:41:59 smithi167 ceph-mon[32751]: pgmap v16144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:01.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:01 smithi090 ceph-mon[29180]: pgmap v16145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:01.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:01 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29202.435521 secs 2024-04-27T02:42:01.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:01 smithi119 ceph-mon[32752]: pgmap v16145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:01.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:01 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29202.435521 secs 2024-04-27T02:42:01.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:01 smithi167 ceph-mon[32751]: pgmap v16145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:01.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:01 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29202.435521 secs 2024-04-27T02:42:02.602 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29203.275490 secs 2024-04-27T02:42:02.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29203.275490 secs 2024-04-27T02:42:02.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29203.275490 secs 2024-04-27T02:42:03.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:03 smithi090 ceph-mon[29180]: pgmap v16146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:03.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:03 smithi119 ceph-mon[32752]: pgmap v16146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:03.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:03 smithi167 ceph-mon[32751]: pgmap v16146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:05.606 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:05 smithi090 ceph-mon[29180]: pgmap v16147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:05.780 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:05 smithi119 ceph-mon[32752]: pgmap v16147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:05.794 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:05 smithi167 ceph-mon[32751]: pgmap v16147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:06.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:06 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29207.435604 secs 2024-04-27T02:42:06.682 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:06 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29207.435604 secs 2024-04-27T02:42:06.696 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:06 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29207.435604 secs 2024-04-27T02:42:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29208.275564 secs 2024-04-27T02:42:07.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:07 smithi090 ceph-mon[29180]: pgmap v16148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:07.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29208.275564 secs 2024-04-27T02:42:07.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:07 smithi119 ceph-mon[32752]: pgmap v16148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:07.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29208.275564 secs 2024-04-27T02:42:07.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:07 smithi167 ceph-mon[32751]: pgmap v16148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:09.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:09 smithi090 ceph-mon[29180]: pgmap v16149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:09.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:09 smithi119 ceph-mon[32752]: pgmap v16149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:09.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:09 smithi167 ceph-mon[32751]: pgmap v16149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:11.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:11 smithi090 ceph-mon[29180]: pgmap v16150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:11.604 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:11 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29212.435688 secs 2024-04-27T02:42:11.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:11 smithi119 ceph-mon[32752]: pgmap v16150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:11.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:11 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29212.435688 secs 2024-04-27T02:42:11.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:11 smithi167 ceph-mon[32751]: pgmap v16150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:11.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:11 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29212.435688 secs 2024-04-27T02:42:12.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29213.275681 secs 2024-04-27T02:42:12.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29213.275681 secs 2024-04-27T02:42:12.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29213.275681 secs 2024-04-27T02:42:13.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:13 smithi090 ceph-mon[29180]: pgmap v16151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:13.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:13 smithi119 ceph-mon[32752]: pgmap v16151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:13.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:13 smithi167 ceph-mon[32751]: pgmap v16151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:15.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:15 smithi090 ceph-mon[29180]: pgmap v16152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:15.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:15 smithi119 ceph-mon[32752]: pgmap v16152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:15.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:15 smithi167 ceph-mon[32751]: pgmap v16152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:16.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:16 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29217.435769 secs 2024-04-27T02:42:16.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:16 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29217.435769 secs 2024-04-27T02:42:16.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:16 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29217.435769 secs 2024-04-27T02:42:17.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29218.275802 secs 2024-04-27T02:42:17.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:17 smithi119 ceph-mon[32752]: pgmap v16153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:17.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29218.275802 secs 2024-04-27T02:42:17.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:17 smithi167 ceph-mon[32751]: pgmap v16153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:17.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29218.275802 secs 2024-04-27T02:42:17.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:17 smithi090 ceph-mon[29180]: pgmap v16153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:19.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:19 smithi119 ceph-mon[32752]: pgmap v16154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:19.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:19 smithi167 ceph-mon[32751]: pgmap v16154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:19.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:19 smithi090 ceph-mon[29180]: pgmap v16154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:21.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:21 smithi119 ceph-mon[32752]: pgmap v16155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:21.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:21 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29222.435852 secs 2024-04-27T02:42:21.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:21 smithi167 ceph-mon[32751]: pgmap v16155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:21.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:21 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29222.435852 secs 2024-04-27T02:42:21.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:21 smithi090 ceph-mon[29180]: pgmap v16155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:21.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:21 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29222.435852 secs 2024-04-27T02:42:22.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29223.275954 secs 2024-04-27T02:42:22.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:42:22.795 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29223.275954 secs 2024-04-27T02:42:22.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:42:22.799 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29223.275954 secs 2024-04-27T02:42:22.799 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:42:23.781 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:23 smithi119 ceph-mon[32752]: pgmap v16156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:23.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:42:23.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:42:23.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:42:23.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:23 smithi167 ceph-mon[32751]: pgmap v16156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:23.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:42:23.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:42:23.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:42:23.853 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:23 smithi090 ceph-mon[29180]: pgmap v16156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:23.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:42:23.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:42:23.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:42:24.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:24 smithi119 ceph-mon[32752]: pgmap v16157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:42:24.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:24 smithi119 ceph-mon[32752]: pgmap v16158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:42:24.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:24 smithi167 ceph-mon[32751]: pgmap v16157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:42:24.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:24 smithi167 ceph-mon[32751]: pgmap v16158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:42:24.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:24 smithi090 ceph-mon[29180]: pgmap v16157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:42:24.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:24 smithi090 ceph-mon[29180]: pgmap v16158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:42:26.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:26 smithi119 ceph-mon[32752]: pgmap v16159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:26.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:26 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29227.435936 secs 2024-04-27T02:42:26.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:26 smithi167 ceph-mon[32751]: pgmap v16159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:26.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:26 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29227.435936 secs 2024-04-27T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:26 smithi090 ceph-mon[29180]: pgmap v16159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:26.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:26 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29227.435936 secs 2024-04-27T02:42:27.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29228.276098 secs 2024-04-27T02:42:27.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29228.276098 secs 2024-04-27T02:42:27.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29228.276098 secs 2024-04-27T02:42:28.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:28 smithi119 ceph-mon[32752]: pgmap v16160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:28.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:28 smithi167 ceph-mon[32751]: pgmap v16160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:28.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:28 smithi090 ceph-mon[29180]: pgmap v16160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:30.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:30 smithi119 ceph-mon[32752]: pgmap v16161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:30.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:30 smithi167 ceph-mon[32751]: pgmap v16161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:30.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:30 smithi090 ceph-mon[29180]: pgmap v16161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:31.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:31 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29232.435996 secs 2024-04-27T02:42:31.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:31 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29232.435996 secs 2024-04-27T02:42:31.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:31 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29232.435996 secs 2024-04-27T02:42:32.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:32 smithi119 ceph-mon[32752]: pgmap v16162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:32.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29233.276239 secs 2024-04-27T02:42:32.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:32 smithi167 ceph-mon[32751]: pgmap v16162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:32.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29233.276239 secs 2024-04-27T02:42:32.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:32 smithi090 ceph-mon[29180]: pgmap v16162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:42:32.855 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29233.276239 secs 2024-04-27T02:42:34.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:34 smithi119 ceph-mon[32752]: pgmap v16163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:42:34.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:34 smithi167 ceph-mon[32751]: pgmap v16163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:42:34.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:34 smithi090 ceph-mon[29180]: pgmap v16163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:42:36.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:36 smithi090 ceph-mon[29180]: pgmap v16164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:36.657 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:36 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29237.436079 secs 2024-04-27T02:42:36.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:36 smithi119 ceph-mon[32752]: pgmap v16164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:36.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:36 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29237.436079 secs 2024-04-27T02:42:36.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:36 smithi167 ceph-mon[32751]: pgmap v16164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:36.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:36 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29237.436079 secs 2024-04-27T02:42:37.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29238.276371 secs 2024-04-27T02:42:37.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29238.276371 secs 2024-04-27T02:42:37.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29238.276371 secs 2024-04-27T02:42:38.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:38 smithi090 ceph-mon[29180]: pgmap v16165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:38.782 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:38 smithi119 ceph-mon[32752]: pgmap v16165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:38.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:38 smithi167 ceph-mon[32751]: pgmap v16165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:39.783 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:39 smithi119 ceph-mon[32752]: pgmap v16166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:39.796 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:39 smithi167 ceph-mon[32751]: pgmap v16166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:39.854 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:39 smithi090 ceph-mon[29180]: pgmap v16166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:42.282 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:41 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29242.436159 secs 2024-04-27T02:42:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:41 smithi119 ceph-mon[32752]: pgmap v16167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:42.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29243.276418 secs 2024-04-27T02:42:42.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:41 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29242.436159 secs 2024-04-27T02:42:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:41 smithi167 ceph-mon[32751]: pgmap v16167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:42.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29243.276418 secs 2024-04-27T02:42:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:41 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29242.436159 secs 2024-04-27T02:42:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:41 smithi090 ceph-mon[29180]: pgmap v16167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:42.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29243.276418 secs 2024-04-27T02:42:44.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:43 smithi119 ceph-mon[32752]: pgmap v16168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:44.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:43 smithi167 ceph-mon[32751]: pgmap v16168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:44.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:43 smithi090 ceph-mon[29180]: pgmap v16168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:46.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:45 smithi119 ceph-mon[32752]: pgmap v16169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:46.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:45 smithi167 ceph-mon[32751]: pgmap v16169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:46.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:45 smithi090 ceph-mon[29180]: pgmap v16169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:46 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29247.436218 secs 2024-04-27T02:42:47.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:46 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29248.276564 secs 2024-04-27T02:42:47.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:46 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29247.436218 secs 2024-04-27T02:42:47.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:46 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29248.276564 secs 2024-04-27T02:42:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:46 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29247.436218 secs 2024-04-27T02:42:47.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:46 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29248.276564 secs 2024-04-27T02:42:48.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:47 smithi119 ceph-mon[32752]: pgmap v16170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:48.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:47 smithi167 ceph-mon[32751]: pgmap v16170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:48.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:47 smithi090 ceph-mon[29180]: pgmap v16170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:50.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:49 smithi119 ceph-mon[32752]: pgmap v16171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:50.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:49 smithi167 ceph-mon[32751]: pgmap v16171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:50.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:49 smithi090 ceph-mon[29180]: pgmap v16171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:52.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:51 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29252.436300 secs 2024-04-27T02:42:52.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:51 smithi119 ceph-mon[32752]: pgmap v16172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:52.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:51 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29253.276672 secs 2024-04-27T02:42:52.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:51 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29252.436300 secs 2024-04-27T02:42:52.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:51 smithi167 ceph-mon[32751]: pgmap v16172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:52.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:51 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29253.276672 secs 2024-04-27T02:42:52.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:51 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29252.436300 secs 2024-04-27T02:42:52.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:51 smithi090 ceph-mon[29180]: pgmap v16172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:52.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:51 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29253.276672 secs 2024-04-27T02:42:54.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:53 smithi119 ceph-mon[32752]: pgmap v16173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:54.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:53 smithi167 ceph-mon[32751]: pgmap v16173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:54.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:53 smithi090 ceph-mon[29180]: pgmap v16173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:56.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:55 smithi119 ceph-mon[32752]: pgmap v16174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:56.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:55 smithi167 ceph-mon[32751]: pgmap v16174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:56.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:55 smithi090 ceph-mon[29180]: pgmap v16174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:42:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:56 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29257.436380 secs 2024-04-27T02:42:57.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:56 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29258.276762 secs 2024-04-27T02:42:57.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:56 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29257.436380 secs 2024-04-27T02:42:57.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:56 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29258.276762 secs 2024-04-27T02:42:57.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:56 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29257.436380 secs 2024-04-27T02:42:57.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:56 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29258.276762 secs 2024-04-27T02:42:58.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:57 smithi119 ceph-mon[32752]: pgmap v16175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:58.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:57 smithi167 ceph-mon[32751]: pgmap v16175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:42:58.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:57 smithi090 ceph-mon[29180]: pgmap v16175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:00.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:42:59 smithi119 ceph-mon[32752]: pgmap v16176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:00.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:42:59 smithi167 ceph-mon[32751]: pgmap v16176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:00.355 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:42:59 smithi090 ceph-mon[29180]: pgmap v16176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:02.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:01 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29262.436466 secs 2024-04-27T02:43:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:01 smithi119 ceph-mon[32752]: pgmap v16177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:02.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:01 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29263.276844 secs 2024-04-27T02:43:02.297 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:01 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29262.436466 secs 2024-04-27T02:43:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:01 smithi167 ceph-mon[32751]: pgmap v16177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:02.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:01 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29263.276844 secs 2024-04-27T02:43:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:01 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29262.436466 secs 2024-04-27T02:43:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:01 smithi090 ceph-mon[29180]: pgmap v16177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:02.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:01 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29263.276844 secs 2024-04-27T02:43:04.283 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:03 smithi119 ceph-mon[32752]: pgmap v16178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:04.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:03 smithi167 ceph-mon[32751]: pgmap v16178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:04.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:03 smithi090 ceph-mon[29180]: pgmap v16178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:06.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:06 smithi119 ceph-mon[32752]: pgmap v16179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:06.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:06 smithi167 ceph-mon[32751]: pgmap v16179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:06.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:06 smithi090 ceph-mon[29180]: pgmap v16179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:07 smithi119 ceph-mon[32752]: 1 slow requests, 0 included below; oldest blocked for > 29267.436550 secs 2024-04-27T02:43:07.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29268.276926 secs 2024-04-27T02:43:07.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:07 smithi167 ceph-mon[32751]: 1 slow requests, 0 included below; oldest blocked for > 29267.436550 secs 2024-04-27T02:43:07.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29268.276926 secs 2024-04-27T02:43:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:07 smithi090 ceph-mon[29180]: 1 slow requests, 0 included below; oldest blocked for > 29267.436550 secs 2024-04-27T02:43:07.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29268.276926 secs 2024-04-27T02:43:08.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:08 smithi119 ceph-mon[32752]: pgmap v16180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:08.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:08 smithi167 ceph-mon[32751]: pgmap v16180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:08.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:08 smithi090 ceph-mon[29180]: pgmap v16180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:10.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:10 smithi119 ceph-mon[32752]: pgmap v16181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:10.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:10 smithi167 ceph-mon[32751]: pgmap v16181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:10.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:10 smithi090 ceph-mon[29180]: pgmap v16181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:12 smithi119 ceph-mon[32752]: 38 slow requests, 2 included below; oldest blocked for > 29272.436634 secs 2024-04-27T02:43:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:12 smithi119 ceph-mon[32752]: slow request 181.364053 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:43:12.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:12 smithi119 ceph-mon[32752]: slow request 1518.207327 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:43:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:12 smithi119 ceph-mon[32752]: pgmap v16182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:12 smithi119 ceph-mon[32752]: 28 slow requests, 1 included below; oldest blocked for > 29273.276999 secs 2024-04-27T02:43:12.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:12 smithi119 ceph-mon[32752]: slow request 1442.547583 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:43:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:12 smithi167 ceph-mon[32751]: 38 slow requests, 2 included below; oldest blocked for > 29272.436634 secs 2024-04-27T02:43:12.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:12 smithi167 ceph-mon[32751]: slow request 181.364053 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:43:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:12 smithi167 ceph-mon[32751]: slow request 1518.207327 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:43:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:12 smithi167 ceph-mon[32751]: pgmap v16182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:12 smithi167 ceph-mon[32751]: 28 slow requests, 1 included below; oldest blocked for > 29273.276999 secs 2024-04-27T02:43:12.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:12 smithi167 ceph-mon[32751]: slow request 1442.547583 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:43:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:12 smithi090 ceph-mon[29180]: 38 slow requests, 2 included below; oldest blocked for > 29272.436634 secs 2024-04-27T02:43:12.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:12 smithi090 ceph-mon[29180]: slow request 181.364053 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:43:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:12 smithi090 ceph-mon[29180]: slow request 1518.207327 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:43:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:12 smithi090 ceph-mon[29180]: pgmap v16182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:12 smithi090 ceph-mon[29180]: 28 slow requests, 1 included below; oldest blocked for > 29273.276999 secs 2024-04-27T02:43:12.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:12 smithi090 ceph-mon[29180]: slow request 1442.547583 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:43:14.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:14 smithi119 ceph-mon[32752]: pgmap v16183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:14.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:14 smithi167 ceph-mon[32751]: pgmap v16183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:14.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:14 smithi090 ceph-mon[29180]: pgmap v16183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:16 smithi119 ceph-mon[32752]: pgmap v16184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:16 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:43:16.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:16 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:43:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:16 smithi167 ceph-mon[32751]: pgmap v16184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:16.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:16 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:43:16.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:16 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:43:16.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:16 smithi090 ceph-mon[29180]: pgmap v16184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:16 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:43:16.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:16 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:43:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:17 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29277.436718 secs 2024-04-27T02:43:17.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29278.277068 secs 2024-04-27T02:43:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:17 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29277.436718 secs 2024-04-27T02:43:17.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29278.277068 secs 2024-04-27T02:43:17.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:17 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29277.436718 secs 2024-04-27T02:43:17.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29278.277068 secs 2024-04-27T02:43:18.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:18 smithi119 ceph-mon[32752]: pgmap v16185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:18.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:18 smithi167 ceph-mon[32751]: pgmap v16185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:18.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:18 smithi090 ceph-mon[29180]: pgmap v16185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:20.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:20 smithi119 ceph-mon[32752]: pgmap v16186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:20.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:20 smithi167 ceph-mon[32751]: pgmap v16186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:20.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:20 smithi090 ceph-mon[29180]: pgmap v16186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:22.284 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:22 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29282.436822 secs 2024-04-27T02:43:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:22 smithi119 ceph-mon[32752]: pgmap v16187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:22.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29283.277134 secs 2024-04-27T02:43:22.298 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:22 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29282.436822 secs 2024-04-27T02:43:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:22 smithi167 ceph-mon[32751]: pgmap v16187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:22.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29283.277134 secs 2024-04-27T02:43:22.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:22 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29282.436822 secs 2024-04-27T02:43:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:22 smithi090 ceph-mon[29180]: pgmap v16187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:22.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29283.277134 secs 2024-04-27T02:43:23.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:43:23.299 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:43:23.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:43:24.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:24 smithi167 ceph-mon[32751]: pgmap v16188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:24.041 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.042 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:24 smithi119 ceph-mon[32752]: pgmap v16188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.285 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.356 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:24 smithi090 ceph-mon[29180]: pgmap v16188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:24.357 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: pgmap v16189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:26.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29287.436897 secs 2024-04-27T02:43:26.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: pgmap v16189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:26.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29287.436897 secs 2024-04-27T02:43:26.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: pgmap v16189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29287.436897 secs 2024-04-27T02:43:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:26.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:27.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:43:27.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:43:27.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29288.277223 secs 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.536 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:43:27.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:43:27.537 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:27.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:43:27.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:43:27.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29288.277223 secs 2024-04-27T02:43:27.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.550 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:43:27.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:43:27.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:43:27.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:43:27.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:27.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:43:27.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:43:27.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29288.277223 secs 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:43:27.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:43:27.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:43:27.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:43:28.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:28 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:43:28.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:28 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:43:28.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:28 smithi119 ceph-mon[32752]: pgmap v16190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T02:43:28.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:28 smithi119 ceph-mon[32752]: pgmap v16191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:28.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:28 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:43:28.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:28 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:43:28.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:28 smithi167 ceph-mon[32751]: pgmap v16190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T02:43:28.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:28 smithi167 ceph-mon[32751]: pgmap v16191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:28.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:28 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:43:28.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:28 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:43:28.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:28 smithi090 ceph-mon[29180]: pgmap v16190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T02:43:28.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:28 smithi090 ceph-mon[29180]: pgmap v16191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:30.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:30 smithi119 ceph-mon[32752]: pgmap v16192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:30.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:30 smithi167 ceph-mon[32751]: pgmap v16192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:30.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:30 smithi090 ceph-mon[29180]: pgmap v16192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:31.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:31 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29292.436969 secs 2024-04-27T02:43:31.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:31 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29292.436969 secs 2024-04-27T02:43:31.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:31 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29292.436969 secs 2024-04-27T02:43:32.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:32 smithi119 ceph-mon[32752]: pgmap v16193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:32.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29293.277296 secs 2024-04-27T02:43:32.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:32 smithi167 ceph-mon[32751]: pgmap v16193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:32.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29293.277296 secs 2024-04-27T02:43:32.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:32 smithi090 ceph-mon[29180]: pgmap v16193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:32.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29293.277296 secs 2024-04-27T02:43:34.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:34 smithi119 ceph-mon[32752]: pgmap v16194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:34.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:34 smithi167 ceph-mon[32751]: pgmap v16194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:34.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:34 smithi090 ceph-mon[29180]: pgmap v16194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:36.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:36 smithi119 ceph-mon[32752]: pgmap v16195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:36.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:36 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29297.437049 secs 2024-04-27T02:43:36.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:36 smithi167 ceph-mon[32751]: pgmap v16195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:36.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:36 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29297.437049 secs 2024-04-27T02:43:36.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:36 smithi090 ceph-mon[29180]: pgmap v16195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T02:43:36.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:36 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29297.437049 secs 2024-04-27T02:43:37.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:37 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29298.277442 secs 2024-04-27T02:43:37.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:37 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29298.277442 secs 2024-04-27T02:43:37.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:37 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29298.277442 secs 2024-04-27T02:43:38.535 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:38 smithi119 ceph-mon[32752]: pgmap v16196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:43:38.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:38 smithi167 ceph-mon[32751]: pgmap v16196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:43:38.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:38 smithi090 ceph-mon[29180]: pgmap v16196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:43:40.549 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:40 smithi167 ceph-mon[32751]: pgmap v16197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:40.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:40 smithi090 ceph-mon[29180]: pgmap v16197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:40.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:40 smithi119 ceph-mon[32752]: pgmap v16197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:41.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:41 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29302.437133 secs 2024-04-27T02:43:41.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:41 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29302.437133 secs 2024-04-27T02:43:41.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:41 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29302.437133 secs 2024-04-27T02:43:42.607 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:42 smithi090 ceph-mon[29180]: pgmap v16198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:42.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:42 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29303.277510 secs 2024-04-27T02:43:42.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:42 smithi119 ceph-mon[32752]: pgmap v16198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:42.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:42 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29303.277510 secs 2024-04-27T02:43:42.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:42 smithi167 ceph-mon[32751]: pgmap v16198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:42.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:42 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29303.277510 secs 2024-04-27T02:43:44.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:44 smithi090 ceph-mon[29180]: pgmap v16199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:44.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:44 smithi119 ceph-mon[32752]: pgmap v16199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:44.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:44 smithi167 ceph-mon[32751]: pgmap v16199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:46.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:46 smithi090 ceph-mon[29180]: pgmap v16200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:46.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:46 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29307.437192 secs 2024-04-27T02:43:46.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:46 smithi119 ceph-mon[32752]: pgmap v16200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:46.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:46 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29307.437192 secs 2024-04-27T02:43:46.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:46 smithi167 ceph-mon[32751]: pgmap v16200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:46.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:46 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29307.437192 secs 2024-04-27T02:43:47.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29308.277575 secs 2024-04-27T02:43:47.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29308.277575 secs 2024-04-27T02:43:47.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29308.277575 secs 2024-04-27T02:43:48.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:48 smithi090 ceph-mon[29180]: pgmap v16201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:48.785 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:48 smithi119 ceph-mon[32752]: pgmap v16201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:48.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:48 smithi167 ceph-mon[32751]: pgmap v16201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:50.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:50 smithi090 ceph-mon[29180]: pgmap v16202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:50.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:50 smithi119 ceph-mon[32752]: pgmap v16202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:50.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:50 smithi167 ceph-mon[32751]: pgmap v16202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:51.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:51 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29312.437256 secs 2024-04-27T02:43:51.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:51 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29312.437256 secs 2024-04-27T02:43:51.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:51 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29312.437256 secs 2024-04-27T02:43:52.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:52 smithi090 ceph-mon[29180]: pgmap v16203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:52.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29313.277668 secs 2024-04-27T02:43:52.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:52 smithi119 ceph-mon[32752]: pgmap v16203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:52.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29313.277668 secs 2024-04-27T02:43:52.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:52 smithi167 ceph-mon[32751]: pgmap v16203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:52.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29313.277668 secs 2024-04-27T02:43:54.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:54 smithi090 ceph-mon[29180]: pgmap v16204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:54.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:54 smithi119 ceph-mon[32752]: pgmap v16204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:54.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:54 smithi167 ceph-mon[32751]: pgmap v16204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:56.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:56 smithi090 ceph-mon[29180]: pgmap v16205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:56.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:56 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29317.437338 secs 2024-04-27T02:43:56.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:56 smithi119 ceph-mon[32752]: pgmap v16205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:56.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:56 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29317.437338 secs 2024-04-27T02:43:56.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:56 smithi167 ceph-mon[32751]: pgmap v16205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:43:56.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:56 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29317.437338 secs 2024-04-27T02:43:57.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29318.277785 secs 2024-04-27T02:43:57.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29318.277785 secs 2024-04-27T02:43:57.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29318.277785 secs 2024-04-27T02:43:58.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:43:58 smithi090 ceph-mon[29180]: pgmap v16206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:58.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:43:58 smithi119 ceph-mon[32752]: pgmap v16206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:43:58.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:43:58 smithi167 ceph-mon[32751]: pgmap v16206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:00.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:00 smithi090 ceph-mon[29180]: pgmap v16207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:00.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:00 smithi119 ceph-mon[32752]: pgmap v16207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:00.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:00 smithi167 ceph-mon[32751]: pgmap v16207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:01.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:01 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29322.437420 secs 2024-04-27T02:44:01.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:01 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29322.437420 secs 2024-04-27T02:44:01.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:01 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29322.437420 secs 2024-04-27T02:44:02.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:02 smithi090 ceph-mon[29180]: pgmap v16208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:02.609 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29323.277848 secs 2024-04-27T02:44:02.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:02 smithi119 ceph-mon[32752]: pgmap v16208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:02.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29323.277848 secs 2024-04-27T02:44:02.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:02 smithi167 ceph-mon[32751]: pgmap v16208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:02.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29323.277848 secs 2024-04-27T02:44:04.608 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:04 smithi090 ceph-mon[29180]: pgmap v16209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:04.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:04 smithi119 ceph-mon[32752]: pgmap v16209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:04.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:04 smithi167 ceph-mon[32751]: pgmap v16209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:06.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:06 smithi119 ceph-mon[32752]: pgmap v16210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:06.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:06 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29327.437502 secs 2024-04-27T02:44:06.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:06 smithi167 ceph-mon[32751]: pgmap v16210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:06.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:06 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29327.437502 secs 2024-04-27T02:44:06.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:06 smithi090 ceph-mon[29180]: pgmap v16210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:06.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:06 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29327.437502 secs 2024-04-27T02:44:07.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29328.277915 secs 2024-04-27T02:44:07.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29328.277915 secs 2024-04-27T02:44:07.858 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29328.277915 secs 2024-04-27T02:44:08.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:08 smithi119 ceph-mon[32752]: pgmap v16211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:08.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:08 smithi167 ceph-mon[32751]: pgmap v16211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:08.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:08 smithi090 ceph-mon[29180]: pgmap v16211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:10.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:10 smithi119 ceph-mon[32752]: pgmap v16212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:10.800 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:10 smithi167 ceph-mon[32751]: pgmap v16212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:10.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:10 smithi090 ceph-mon[29180]: pgmap v16212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:11.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:11 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29332.437584 secs 2024-04-27T02:44:11.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:11 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29332.437584 secs 2024-04-27T02:44:11.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:11 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29332.437584 secs 2024-04-27T02:44:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:12 smithi119 ceph-mon[32752]: pgmap v16213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:12.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:12 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29333.278000 secs 2024-04-27T02:44:12.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:12 smithi167 ceph-mon[32751]: pgmap v16213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:12.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:12 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29333.278000 secs 2024-04-27T02:44:12.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:12 smithi090 ceph-mon[29180]: pgmap v16213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:12.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:12 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29333.278000 secs 2024-04-27T02:44:14.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:14 smithi119 ceph-mon[32752]: pgmap v16214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:14.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:14 smithi167 ceph-mon[32751]: pgmap v16214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:14.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:14 smithi090 ceph-mon[29180]: pgmap v16214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:16.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:16 smithi119 ceph-mon[32752]: pgmap v16215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:16.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:16 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29337.437664 secs 2024-04-27T02:44:16.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:16 smithi167 ceph-mon[32751]: pgmap v16215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:16.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:16 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29337.437664 secs 2024-04-27T02:44:16.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:16 smithi090 ceph-mon[29180]: pgmap v16215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:16.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:16 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29337.437664 secs 2024-04-27T02:44:17.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:17 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29338.278147 secs 2024-04-27T02:44:17.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:17 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29338.278147 secs 2024-04-27T02:44:17.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:17 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29338.278147 secs 2024-04-27T02:44:18.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:18 smithi119 ceph-mon[32752]: pgmap v16216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:18.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:18 smithi167 ceph-mon[32751]: pgmap v16216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:18.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:18 smithi090 ceph-mon[29180]: pgmap v16216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:20.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:20 smithi119 ceph-mon[32752]: pgmap v16217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:20.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:20 smithi167 ceph-mon[32751]: pgmap v16217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:20.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:20 smithi090 ceph-mon[29180]: pgmap v16217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:21.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:21 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29342.437750 secs 2024-04-27T02:44:21.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:21 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29342.437750 secs 2024-04-27T02:44:21.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:21 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29342.437750 secs 2024-04-27T02:44:22.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:22 smithi119 ceph-mon[32752]: pgmap v16218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:22.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:22 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29343.278244 secs 2024-04-27T02:44:22.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:22 smithi167 ceph-mon[32751]: pgmap v16218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:22.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:22 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29343.278244 secs 2024-04-27T02:44:22.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:22 smithi090 ceph-mon[29180]: pgmap v16218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:22.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:22 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29343.278244 secs 2024-04-27T02:44:24.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:24 smithi119 ceph-mon[32752]: pgmap v16219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:24.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:24 smithi167 ceph-mon[32751]: pgmap v16219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:24.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:24 smithi090 ceph-mon[29180]: pgmap v16219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:26.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:26 smithi119 ceph-mon[32752]: pgmap v16220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:26.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:26 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29347.437845 secs 2024-04-27T02:44:26.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:26 smithi167 ceph-mon[32751]: pgmap v16220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:26.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:26 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29347.437845 secs 2024-04-27T02:44:26.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:26 smithi090 ceph-mon[29180]: pgmap v16220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:26.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:26 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29347.437845 secs 2024-04-27T02:44:27.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:27 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29348.278398 secs 2024-04-27T02:44:27.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:44:27.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:27.733 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:27.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:27 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29348.278398 secs 2024-04-27T02:44:27.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:44:27.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:27.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:27.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:27 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29348.278398 secs 2024-04-27T02:44:27.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:44:27.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:27.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:28.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:28 smithi119 ceph-mon[32752]: pgmap v16221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:28.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:44:28.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:44:28.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:28.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:28 smithi167 ceph-mon[32751]: pgmap v16221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:28.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:44:28.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:44:28.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:28.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:28 smithi090 ceph-mon[29180]: pgmap v16221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:28.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:44:28.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:44:28.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:44:29.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:29 smithi119 ceph-mon[32752]: pgmap v16222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T02:44:29.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:29 smithi119 ceph-mon[32752]: pgmap v16223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:44:29.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:29 smithi167 ceph-mon[32751]: pgmap v16222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T02:44:29.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:29 smithi167 ceph-mon[32751]: pgmap v16223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:44:29.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:29 smithi090 ceph-mon[29180]: pgmap v16222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T02:44:29.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:29 smithi090 ceph-mon[29180]: pgmap v16223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T02:44:30.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:30 smithi119 ceph-mon[32752]: pgmap v16224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:30.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:30 smithi167 ceph-mon[32751]: pgmap v16224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:30.859 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:30 smithi090 ceph-mon[29180]: pgmap v16224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:31.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:31 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29352.437934 secs 2024-04-27T02:44:31.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:31 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29352.437934 secs 2024-04-27T02:44:31.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:31 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29352.437934 secs 2024-04-27T02:44:32.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:32 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29353.278464 secs 2024-04-27T02:44:32.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:32 smithi119 ceph-mon[32752]: pgmap v16225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:32.801 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:32 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29353.278464 secs 2024-04-27T02:44:32.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:32 smithi167 ceph-mon[32751]: pgmap v16225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:32.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:32 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29353.278464 secs 2024-04-27T02:44:32.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:32 smithi090 ceph-mon[29180]: pgmap v16225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:35.037 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:34 smithi119 ceph-mon[32752]: pgmap v16226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:35.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:34 smithi167 ceph-mon[32751]: pgmap v16226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:35.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:34 smithi090 ceph-mon[29180]: pgmap v16226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:36 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29357.438019 secs 2024-04-27T02:44:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:36 smithi119 ceph-mon[32752]: pgmap v16227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:36 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29358.278566 secs 2024-04-27T02:44:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:36 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29357.438019 secs 2024-04-27T02:44:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:36 smithi167 ceph-mon[32751]: pgmap v16227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:36 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29358.278566 secs 2024-04-27T02:44:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:36 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29357.438019 secs 2024-04-27T02:44:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:36 smithi090 ceph-mon[29180]: pgmap v16227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:44:37.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:36 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29358.278566 secs 2024-04-27T02:44:39.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:38 smithi119 ceph-mon[32752]: pgmap v16228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:44:39.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:38 smithi167 ceph-mon[32751]: pgmap v16228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:44:39.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:38 smithi090 ceph-mon[29180]: pgmap v16228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:44:41.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:40 smithi167 ceph-mon[32751]: pgmap v16229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:41.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:40 smithi090 ceph-mon[29180]: pgmap v16229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:41.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:40 smithi119 ceph-mon[32752]: pgmap v16229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:41 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29362.438101 secs 2024-04-27T02:44:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:41 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29363.278676 secs 2024-04-27T02:44:42.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:41 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29362.438101 secs 2024-04-27T02:44:42.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:41 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29363.278676 secs 2024-04-27T02:44:42.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:41 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29362.438101 secs 2024-04-27T02:44:42.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:41 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29363.278676 secs 2024-04-27T02:44:43.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:42 smithi167 ceph-mon[32751]: pgmap v16230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:43.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:42 smithi090 ceph-mon[29180]: pgmap v16230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:43.288 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:42 smithi119 ceph-mon[32752]: pgmap v16230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:45.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:45 smithi090 ceph-mon[29180]: pgmap v16231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:45.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:45 smithi119 ceph-mon[32752]: pgmap v16231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:45.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:45 smithi167 ceph-mon[32751]: pgmap v16231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:46.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:46 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29367.438185 secs 2024-04-27T02:44:46.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:46 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29367.438185 secs 2024-04-27T02:44:46.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:46 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29367.438185 secs 2024-04-27T02:44:47.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:47 smithi090 ceph-mon[29180]: pgmap v16232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:47.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:47 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29368.278758 secs 2024-04-27T02:44:47.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:47 smithi119 ceph-mon[32752]: pgmap v16232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:47.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:47 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29368.278758 secs 2024-04-27T02:44:47.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:47 smithi167 ceph-mon[32751]: pgmap v16232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:47.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:47 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29368.278758 secs 2024-04-27T02:44:49.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:49 smithi090 ceph-mon[29180]: pgmap v16233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:49.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:49 smithi119 ceph-mon[32752]: pgmap v16233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:49.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:49 smithi167 ceph-mon[32751]: pgmap v16233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:51.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:51 smithi090 ceph-mon[29180]: pgmap v16234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:51.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:51 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29372.438266 secs 2024-04-27T02:44:51.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:51 smithi119 ceph-mon[32752]: pgmap v16234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:51.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:51 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29372.438266 secs 2024-04-27T02:44:51.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:51 smithi167 ceph-mon[32751]: pgmap v16234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:51.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:51 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29372.438266 secs 2024-04-27T02:44:52.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:52 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29373.278832 secs 2024-04-27T02:44:52.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:52 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29373.278832 secs 2024-04-27T02:44:52.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:52 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29373.278832 secs 2024-04-27T02:44:53.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:53 smithi090 ceph-mon[29180]: pgmap v16235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:53.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:53 smithi119 ceph-mon[32752]: pgmap v16235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:53.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:53 smithi167 ceph-mon[32751]: pgmap v16235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:55.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:55 smithi090 ceph-mon[29180]: pgmap v16236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:55.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:55 smithi119 ceph-mon[32752]: pgmap v16236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:55.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:55 smithi167 ceph-mon[32751]: pgmap v16236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:44:56.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:56 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29377.438346 secs 2024-04-27T02:44:56.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:56 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29377.438346 secs 2024-04-27T02:44:56.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:56 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29377.438346 secs 2024-04-27T02:44:57.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:57 smithi090 ceph-mon[29180]: pgmap v16237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:57.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:57 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29378.278915 secs 2024-04-27T02:44:57.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:57 smithi119 ceph-mon[32752]: pgmap v16237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:57.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:57 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29378.278915 secs 2024-04-27T02:44:57.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:57 smithi167 ceph-mon[32751]: pgmap v16237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:57.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:57 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29378.278915 secs 2024-04-27T02:44:59.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:44:59 smithi090 ceph-mon[29180]: pgmap v16238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:59.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:44:59 smithi119 ceph-mon[32752]: pgmap v16238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:44:59.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:44:59 smithi167 ceph-mon[32751]: pgmap v16238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:01.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:01 smithi090 ceph-mon[29180]: pgmap v16239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:01.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:01 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29382.438426 secs 2024-04-27T02:45:01.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:01 smithi119 ceph-mon[32752]: pgmap v16239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:01.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:01 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29382.438426 secs 2024-04-27T02:45:01.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:01 smithi167 ceph-mon[32751]: pgmap v16239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:01.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:01 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29382.438426 secs 2024-04-27T02:45:02.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:02 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29383.279024 secs 2024-04-27T02:45:02.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:02 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29383.279024 secs 2024-04-27T02:45:02.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:02 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29383.279024 secs 2024-04-27T02:45:03.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:03 smithi119 ceph-mon[32752]: pgmap v16240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:03.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:03 smithi167 ceph-mon[32751]: pgmap v16240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:03.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:03 smithi090 ceph-mon[29180]: pgmap v16240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:05.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:05 smithi119 ceph-mon[32752]: pgmap v16241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:05.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:05 smithi167 ceph-mon[32751]: pgmap v16241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:05.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:05 smithi090 ceph-mon[29180]: pgmap v16241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:06.690 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:06 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29387.438506 secs 2024-04-27T02:45:06.704 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:06 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29387.438506 secs 2024-04-27T02:45:06.765 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:06 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29387.438506 secs 2024-04-27T02:45:07.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:07 smithi119 ceph-mon[32752]: pgmap v16242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:07.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:07 smithi119 ceph-mon[32752]: 28 slow requests, 0 included below; oldest blocked for > 29388.279146 secs 2024-04-27T02:45:07.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:07 smithi167 ceph-mon[32751]: pgmap v16242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:07.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:07 smithi167 ceph-mon[32751]: 28 slow requests, 0 included below; oldest blocked for > 29388.279146 secs 2024-04-27T02:45:07.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:07 smithi090 ceph-mon[29180]: pgmap v16242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:07.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:07 smithi090 ceph-mon[29180]: 28 slow requests, 0 included below; oldest blocked for > 29388.279146 secs 2024-04-27T02:45:09.361 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:45:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:45:08.885+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:45:09.361 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:45:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:45:08.885+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:45:09.362 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:45:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:45:08.885+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:45:09.362 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:45:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:45:08.885+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:45:09.362 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:45:08 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:45:08.885+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:45:09.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:09 smithi119 ceph-mon[32752]: pgmap v16243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:09.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:09 smithi167 ceph-mon[32751]: pgmap v16243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:09.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:09 smithi090 ceph-mon[29180]: pgmap v16243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:11.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:11 smithi119 ceph-mon[32752]: pgmap v16244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:11.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:11 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29392.438592 secs 2024-04-27T02:45:11.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:11 smithi167 ceph-mon[32751]: pgmap v16244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:11.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:11 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29392.438592 secs 2024-04-27T02:45:11.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:11 smithi090 ceph-mon[29180]: pgmap v16244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:11.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:11 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29392.438592 secs 2024-04-27T02:45:12.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:12 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29393.279216 secs 2024-04-27T02:45:12.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:12 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29393.279216 secs 2024-04-27T02:45:12.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:12 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29393.279216 secs 2024-04-27T02:45:13.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:13 smithi119 ceph-mon[32752]: pgmap v16245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:13.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:13 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:45:13.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:13 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:45:13.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:13 smithi167 ceph-mon[32751]: pgmap v16245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:13.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:13 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:45:13.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:13 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:45:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:13 smithi090 ceph-mon[29180]: pgmap v16245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:45:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:13 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:45:13.815 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:13 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:45:15.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:15 smithi119 ceph-mon[32752]: pgmap v16246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:15.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:15 smithi167 ceph-mon[32751]: pgmap v16246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:15.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:15 smithi090 ceph-mon[29180]: pgmap v16246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:16.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:16 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29397.438673 secs 2024-04-27T02:45:16.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:16 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29397.438673 secs 2024-04-27T02:45:16.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:16 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29397.438673 secs 2024-04-27T02:45:17.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:17 smithi119 ceph-mon[32752]: pgmap v16247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:17.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:17 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29398.279331 secs 2024-04-27T02:45:17.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:17 smithi167 ceph-mon[32751]: pgmap v16247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:17.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:17 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29398.279331 secs 2024-04-27T02:45:17.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:17 smithi090 ceph-mon[29180]: pgmap v16247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:17.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:17 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29398.279331 secs 2024-04-27T02:45:19.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:19 smithi119 ceph-mon[32752]: pgmap v16248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:19.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:19 smithi167 ceph-mon[32751]: pgmap v16248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:19.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:19 smithi090 ceph-mon[29180]: pgmap v16248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:21.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:21 smithi119 ceph-mon[32752]: pgmap v16249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:21.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:21 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29402.438753 secs 2024-04-27T02:45:21.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:21 smithi167 ceph-mon[32751]: pgmap v16249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:21.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:21 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29402.438753 secs 2024-04-27T02:45:21.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:21 smithi090 ceph-mon[29180]: pgmap v16249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:21.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:21 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29402.438753 secs 2024-04-27T02:45:22.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:22 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29403.279457 secs 2024-04-27T02:45:22.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:22 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29403.279457 secs 2024-04-27T02:45:22.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:22 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29403.279457 secs 2024-04-27T02:45:23.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:23 smithi119 ceph-mon[32752]: pgmap v16250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:23.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:23 smithi167 ceph-mon[32751]: pgmap v16250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:23.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:23 smithi090 ceph-mon[29180]: pgmap v16250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:25.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:25 smithi119 ceph-mon[32752]: pgmap v16251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:25.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:25 smithi167 ceph-mon[32751]: pgmap v16251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:25.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:25 smithi090 ceph-mon[29180]: pgmap v16251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T02:45:26.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:26 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29407.438845 secs 2024-04-27T02:45:26.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:26 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29407.438845 secs 2024-04-27T02:45:26.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:26 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29407.438845 secs 2024-04-27T02:45:27.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:27 smithi119 ceph-mon[32752]: pgmap v16252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:27.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:27 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29408.279550 secs 2024-04-27T02:45:27.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:27 smithi167 ceph-mon[32751]: pgmap v16252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:27.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:27 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29408.279550 secs 2024-04-27T02:45:27.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:27 smithi090 ceph-mon[29180]: pgmap v16252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:27.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:27 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29408.279550 secs 2024-04-27T02:45:28.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:45:28.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:45:28.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:45:29.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:29 smithi090 ceph-mon[29180]: pgmap v16253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:29.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:45:29.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:45:29.688 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:45:29.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:29 smithi119 ceph-mon[32752]: pgmap v16253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:29.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:45:29.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:45:29.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:45:29.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:29 smithi167 ceph-mon[32751]: pgmap v16253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:29.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:45:29.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:45:29.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:45:30.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:30 smithi119 ceph-mon[32752]: pgmap v16254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:45:30.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:30 smithi119 ceph-mon[32752]: pgmap v16255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:30.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:30 smithi167 ceph-mon[32751]: pgmap v16254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:45:30.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:30 smithi167 ceph-mon[32751]: pgmap v16255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:30.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:30 smithi090 ceph-mon[29180]: pgmap v16254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:45:30.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:30 smithi090 ceph-mon[29180]: pgmap v16255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:31.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:31 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29412.438906 secs 2024-04-27T02:45:31.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:31 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29412.438906 secs 2024-04-27T02:45:31.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:31 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29412.438906 secs 2024-04-27T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:32 smithi119 ceph-mon[32752]: pgmap v16256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:32.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:32 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29413.279616 secs 2024-04-27T02:45:32.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:32 smithi167 ceph-mon[32751]: pgmap v16256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:32.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:32 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29413.279616 secs 2024-04-27T02:45:32.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:32 smithi090 ceph-mon[29180]: pgmap v16256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:32.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:32 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29413.279616 secs 2024-04-27T02:45:33.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:33 smithi119 ceph-mon[32752]: pgmap v16257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:45:33.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:33 smithi167 ceph-mon[32751]: pgmap v16257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:45:33.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:33 smithi090 ceph-mon[29180]: pgmap v16257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:45:35.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:35 smithi090 ceph-mon[29180]: pgmap v16258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:36.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:35 smithi119 ceph-mon[32752]: pgmap v16258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:36.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:35 smithi167 ceph-mon[32751]: pgmap v16258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:36.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:36 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29417.438993 secs 2024-04-27T02:45:36.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:36 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29417.438993 secs 2024-04-27T02:45:36.955 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:36 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29417.438993 secs 2024-04-27T02:45:37.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:37 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29418.279718 secs 2024-04-27T02:45:37.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:37 smithi090 ceph-mon[29180]: pgmap v16259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:38.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:37 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29418.279718 secs 2024-04-27T02:45:38.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:37 smithi119 ceph-mon[32752]: pgmap v16259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:37 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29418.279718 secs 2024-04-27T02:45:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:37 smithi167 ceph-mon[32751]: pgmap v16259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:45:39.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:39 smithi090 ceph-mon[29180]: pgmap v16260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:45:40.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:39 smithi119 ceph-mon[32752]: pgmap v16260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:45:40.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:39 smithi167 ceph-mon[32751]: pgmap v16260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:45:41.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:41 smithi090 ceph-mon[29180]: 38 slow requests, 1 included below; oldest blocked for > 29422.439055 secs 2024-04-27T02:45:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:41 smithi090 ceph-mon[29180]: slow request 5762.905087 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:45:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:41 smithi090 ceph-mon[29180]: pgmap v16261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:42.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:41 smithi119 ceph-mon[32752]: 38 slow requests, 1 included below; oldest blocked for > 29422.439055 secs 2024-04-27T02:45:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:41 smithi119 ceph-mon[32752]: slow request 5762.905087 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:45:42.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:41 smithi119 ceph-mon[32752]: pgmap v16261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:41 smithi167 ceph-mon[32751]: 38 slow requests, 1 included below; oldest blocked for > 29422.439055 secs 2024-04-27T02:45:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:41 smithi167 ceph-mon[32751]: slow request 5762.905087 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:45:42.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:41 smithi167 ceph-mon[32751]: pgmap v16261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:42.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:42 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 29423.279830 secs 2024-04-27T02:45:42.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:42 smithi090 ceph-mon[29180]: slow request 2882.885847 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:45:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:42 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 29423.279830 secs 2024-04-27T02:45:43.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:42 smithi119 ceph-mon[32752]: slow request 2882.885847 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:45:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:42 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 29423.279830 secs 2024-04-27T02:45:43.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:42 smithi167 ceph-mon[32751]: slow request 2882.885847 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:45:43.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:43 smithi090 ceph-mon[29180]: pgmap v16262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:44.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:43 smithi119 ceph-mon[32752]: pgmap v16262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:44.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:43 smithi167 ceph-mon[32751]: pgmap v16262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:45.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:45 smithi090 ceph-mon[29180]: pgmap v16263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:46.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:45 smithi119 ceph-mon[32752]: pgmap v16263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:46.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:45 smithi167 ceph-mon[32751]: pgmap v16263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:46.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:46 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29427.439132 secs 2024-04-27T02:45:47.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:46 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29427.439132 secs 2024-04-27T02:45:47.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:46 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29427.439132 secs 2024-04-27T02:45:47.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:47 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29428.279968 secs 2024-04-27T02:45:47.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:47 smithi090 ceph-mon[29180]: pgmap v16264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:48.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:47 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29428.279968 secs 2024-04-27T02:45:48.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:47 smithi119 ceph-mon[32752]: pgmap v16264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:48.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:47 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29428.279968 secs 2024-04-27T02:45:48.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:47 smithi167 ceph-mon[32751]: pgmap v16264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:49.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:49 smithi090 ceph-mon[29180]: pgmap v16265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:50.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:49 smithi119 ceph-mon[32752]: pgmap v16265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:50.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:49 smithi167 ceph-mon[32751]: pgmap v16265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:51 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29432.439212 secs 2024-04-27T02:45:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:51 smithi090 ceph-mon[29180]: pgmap v16266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:52.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:51 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29432.439212 secs 2024-04-27T02:45:52.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:51 smithi119 ceph-mon[32752]: pgmap v16266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:52.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:51 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29432.439212 secs 2024-04-27T02:45:52.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:51 smithi167 ceph-mon[32751]: pgmap v16266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:52.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:52 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29433.280083 secs 2024-04-27T02:45:53.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:52 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29433.280083 secs 2024-04-27T02:45:53.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:52 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29433.280083 secs 2024-04-27T02:45:53.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:53 smithi090 ceph-mon[29180]: pgmap v16267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:54.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:53 smithi119 ceph-mon[32752]: pgmap v16267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:54.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:53 smithi167 ceph-mon[32751]: pgmap v16267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:55.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:55 smithi090 ceph-mon[29180]: pgmap v16268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:56.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:55 smithi119 ceph-mon[32752]: pgmap v16268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:56.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:55 smithi167 ceph-mon[32751]: pgmap v16268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:45:57.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:56 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29437.439293 secs 2024-04-27T02:45:57.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:56 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29437.439293 secs 2024-04-27T02:45:57.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:56 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29437.439293 secs 2024-04-27T02:45:58.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:57 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29438.280163 secs 2024-04-27T02:45:58.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:57 smithi119 ceph-mon[32752]: pgmap v16269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:58.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:57 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29438.280163 secs 2024-04-27T02:45:58.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:57 smithi167 ceph-mon[32751]: pgmap v16269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:45:58.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:57 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29438.280163 secs 2024-04-27T02:45:58.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:57 smithi090 ceph-mon[29180]: pgmap v16269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:00.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:45:59 smithi119 ceph-mon[32752]: pgmap v16270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:00.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:45:59 smithi167 ceph-mon[32751]: pgmap v16270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:00.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:45:59 smithi090 ceph-mon[29180]: pgmap v16270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:02.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:01 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29442.439378 secs 2024-04-27T02:46:02.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:01 smithi119 ceph-mon[32752]: pgmap v16271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:02.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:01 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29442.439378 secs 2024-04-27T02:46:02.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:01 smithi167 ceph-mon[32751]: pgmap v16271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:02.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:01 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29442.439378 secs 2024-04-27T02:46:02.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:01 smithi090 ceph-mon[29180]: pgmap v16271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:03.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:02 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29443.280270 secs 2024-04-27T02:46:03.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:02 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29443.280270 secs 2024-04-27T02:46:03.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:02 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29443.280270 secs 2024-04-27T02:46:04.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:03 smithi119 ceph-mon[32752]: pgmap v16272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:04.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:03 smithi167 ceph-mon[32751]: pgmap v16272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:04.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:03 smithi090 ceph-mon[29180]: pgmap v16272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:06.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:05 smithi119 ceph-mon[32752]: pgmap v16273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:06.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:05 smithi167 ceph-mon[32751]: pgmap v16273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:06.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:05 smithi090 ceph-mon[29180]: pgmap v16273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:06.942 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:06 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29447.439460 secs 2024-04-27T02:46:06.956 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:06 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29447.439460 secs 2024-04-27T02:46:07.014 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:06 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29447.439460 secs 2024-04-27T02:46:08.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:07 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29448.280396 secs 2024-04-27T02:46:08.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:07 smithi119 ceph-mon[32752]: pgmap v16274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:08.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:07 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29448.280396 secs 2024-04-27T02:46:08.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:07 smithi167 ceph-mon[32751]: pgmap v16274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:08.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:07 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29448.280396 secs 2024-04-27T02:46:08.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:07 smithi090 ceph-mon[29180]: pgmap v16274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:10.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:09 smithi119 ceph-mon[32752]: pgmap v16275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:10.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:09 smithi167 ceph-mon[32751]: pgmap v16275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:10.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:09 smithi090 ceph-mon[29180]: pgmap v16275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:12.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:11 smithi119 ceph-mon[32752]: 38 slow requests, 1 included below; oldest blocked for > 29452.439544 secs 2024-04-27T02:46:12.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:11 smithi119 ceph-mon[32752]: slow request 361.366963 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:46:12.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:11 smithi119 ceph-mon[32752]: pgmap v16276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:12.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:11 smithi167 ceph-mon[32751]: 38 slow requests, 1 included below; oldest blocked for > 29452.439544 secs 2024-04-27T02:46:12.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:11 smithi167 ceph-mon[32751]: slow request 361.366963 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:46:12.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:11 smithi167 ceph-mon[32751]: pgmap v16276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:12.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:11 smithi090 ceph-mon[29180]: 38 slow requests, 1 included below; oldest blocked for > 29452.439544 secs 2024-04-27T02:46:12.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:11 smithi090 ceph-mon[29180]: slow request 361.366963 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:46:12.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:11 smithi090 ceph-mon[29180]: pgmap v16276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:13.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:12 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29453.280509 secs 2024-04-27T02:46:13.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:12 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29453.280509 secs 2024-04-27T02:46:13.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:12 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29453.280509 secs 2024-04-27T02:46:14.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:13 smithi119 ceph-mon[32752]: pgmap v16277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:14.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:13 smithi167 ceph-mon[32751]: pgmap v16277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:14.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:13 smithi090 ceph-mon[29180]: pgmap v16277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:15.916 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:15 smithi090 ceph-mon[29180]: pgmap v16278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:16.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:15 smithi119 ceph-mon[32752]: pgmap v16278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:16.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:15 smithi167 ceph-mon[32751]: pgmap v16278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:17.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:16 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29457.439622 secs 2024-04-27T02:46:17.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:16 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29457.439622 secs 2024-04-27T02:46:17.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:16 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29457.439622 secs 2024-04-27T02:46:18.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:17 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29458.280630 secs 2024-04-27T02:46:18.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:17 smithi119 ceph-mon[32752]: pgmap v16279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:18.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:17 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29458.280630 secs 2024-04-27T02:46:18.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:17 smithi167 ceph-mon[32751]: pgmap v16279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:18.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:17 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29458.280630 secs 2024-04-27T02:46:18.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:17 smithi090 ceph-mon[29180]: pgmap v16279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:20.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:19 smithi119 ceph-mon[32752]: pgmap v16280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:20.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:19 smithi167 ceph-mon[32751]: pgmap v16280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:20.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:19 smithi090 ceph-mon[29180]: pgmap v16280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:22.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:21 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29462.439708 secs 2024-04-27T02:46:22.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:21 smithi119 ceph-mon[32752]: pgmap v16281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:22.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:21 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29462.439708 secs 2024-04-27T02:46:22.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:21 smithi167 ceph-mon[32751]: pgmap v16281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:22.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:21 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29462.439708 secs 2024-04-27T02:46:22.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:21 smithi090 ceph-mon[29180]: pgmap v16281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:23.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:22 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29463.280694 secs 2024-04-27T02:46:23.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:22 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29463.280694 secs 2024-04-27T02:46:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:22 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29463.280694 secs 2024-04-27T02:46:24.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:23 smithi119 ceph-mon[32752]: pgmap v16282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:24.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:23 smithi167 ceph-mon[32751]: pgmap v16282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:24.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:23 smithi090 ceph-mon[29180]: pgmap v16282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:26.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:25 smithi119 ceph-mon[32752]: pgmap v16283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:26.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:25 smithi167 ceph-mon[32751]: pgmap v16283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:26.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:25 smithi090 ceph-mon[29180]: pgmap v16283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:27.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:26 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29467.439814 secs 2024-04-27T02:46:27.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:26 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29467.439814 secs 2024-04-27T02:46:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:26 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29467.439814 secs 2024-04-27T02:46:28.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:27 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29468.280811 secs 2024-04-27T02:46:28.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:27 smithi119 ceph-mon[32752]: pgmap v16284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:28.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:27 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29468.280811 secs 2024-04-27T02:46:28.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:27 smithi167 ceph-mon[32751]: pgmap v16284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:28.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:27 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29468.280811 secs 2024-04-27T02:46:28.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:27 smithi090 ceph-mon[29180]: pgmap v16284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:29.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:46:29.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:46:29.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:46:30.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:29 smithi119 ceph-mon[32752]: pgmap v16285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:30.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:46:30.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:46:30.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:29 smithi119 ceph-mon[32752]: pgmap v16286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:46:30.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:29 smithi119 ceph-mon[32752]: pgmap v16287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:30.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:46:30.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:29 smithi167 ceph-mon[32751]: pgmap v16285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:30.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:46:30.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:46:30.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:29 smithi167 ceph-mon[32751]: pgmap v16286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:46:30.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:29 smithi167 ceph-mon[32751]: pgmap v16287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:30.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:46:30.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:29 smithi090 ceph-mon[29180]: pgmap v16285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:30.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:46:30.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:46:30.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:29 smithi090 ceph-mon[29180]: pgmap v16286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:46:30.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:29 smithi090 ceph-mon[29180]: pgmap v16287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:30.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:46:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:31 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29472.439876 secs 2024-04-27T02:46:31.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:31 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29472.439876 secs 2024-04-27T02:46:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:31 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29472.439876 secs 2024-04-27T02:46:32.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:32 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29473.280945 secs 2024-04-27T02:46:32.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:32 smithi119 ceph-mon[32752]: pgmap v16288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:32.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:32 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29473.280945 secs 2024-04-27T02:46:32.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:32 smithi167 ceph-mon[32751]: pgmap v16288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:32.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:32 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29473.280945 secs 2024-04-27T02:46:32.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:32 smithi090 ceph-mon[29180]: pgmap v16288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:34.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:34 smithi090 ceph-mon[29180]: pgmap v16289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:34.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:34 smithi119 ceph-mon[32752]: pgmap v16289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:34.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:34 smithi167 ceph-mon[32751]: pgmap v16289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:36.608 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:36 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29477.439953 secs 2024-04-27T02:46:36.609 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:36 smithi167 ceph-mon[32751]: pgmap v16290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:36 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29477.439953 secs 2024-04-27T02:46:36.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:36 smithi090 ceph-mon[29180]: pgmap v16290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:36.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:36 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29477.439953 secs 2024-04-27T02:46:36.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:36 smithi119 ceph-mon[32752]: pgmap v16290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:46:37.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:37 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29478.281062 secs 2024-04-27T02:46:37.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:37 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29478.281062 secs 2024-04-27T02:46:37.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:37 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29478.281062 secs 2024-04-27T02:46:38.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:38 smithi090 ceph-mon[29180]: pgmap v16291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:46:38.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:38 smithi119 ceph-mon[32752]: pgmap v16291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:46:38.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:38 smithi167 ceph-mon[32751]: pgmap v16291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:46:40.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:40 smithi090 ceph-mon[29180]: pgmap v16292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:46:40.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:40 smithi119 ceph-mon[32752]: pgmap v16292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:46:40.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:40 smithi167 ceph-mon[32751]: pgmap v16292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:46:41.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:41 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29482.440043 secs 2024-04-27T02:46:41.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:41 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29482.440043 secs 2024-04-27T02:46:41.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:41 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29482.440043 secs 2024-04-27T02:46:42.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:42 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29483.281196 secs 2024-04-27T02:46:42.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:42 smithi119 ceph-mon[32752]: pgmap v16293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:42.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:42 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29483.281196 secs 2024-04-27T02:46:42.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:42 smithi167 ceph-mon[32751]: pgmap v16293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:42.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:42 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29483.281196 secs 2024-04-27T02:46:42.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:42 smithi090 ceph-mon[29180]: pgmap v16293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:44.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:44 smithi119 ceph-mon[32752]: pgmap v16294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:44.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:44 smithi167 ceph-mon[32751]: pgmap v16294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:44.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:44 smithi090 ceph-mon[29180]: pgmap v16294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:46.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:46 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29487.440123 secs 2024-04-27T02:46:46.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:46 smithi119 ceph-mon[32752]: pgmap v16295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:46.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:46 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29487.440123 secs 2024-04-27T02:46:46.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:46 smithi167 ceph-mon[32751]: pgmap v16295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:46.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:46 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29487.440123 secs 2024-04-27T02:46:46.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:46 smithi090 ceph-mon[29180]: pgmap v16295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:47.766 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:47 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29488.281318 secs 2024-04-27T02:46:47.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:47 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29488.281318 secs 2024-04-27T02:46:47.806 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:47 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29488.281318 secs 2024-04-27T02:46:48.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:48 smithi119 ceph-mon[32752]: pgmap v16296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:48.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:48 smithi167 ceph-mon[32751]: pgmap v16296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:48.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:48 smithi090 ceph-mon[29180]: pgmap v16296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:50.792 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:50 smithi119 ceph-mon[32752]: pgmap v16297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:50.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:50 smithi167 ceph-mon[32751]: pgmap v16297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:50.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:50 smithi090 ceph-mon[29180]: pgmap v16297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:51.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:51 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29492.440204 secs 2024-04-27T02:46:51.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:51 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29492.440204 secs 2024-04-27T02:46:51.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:51 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29492.440204 secs 2024-04-27T02:46:52.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:52 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29493.281451 secs 2024-04-27T02:46:52.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:52 smithi119 ceph-mon[32752]: pgmap v16298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:52.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:52 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29493.281451 secs 2024-04-27T02:46:52.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:52 smithi167 ceph-mon[32751]: pgmap v16298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:52.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:52 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29493.281451 secs 2024-04-27T02:46:52.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:52 smithi090 ceph-mon[29180]: pgmap v16298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:54.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:54 smithi119 ceph-mon[32752]: pgmap v16299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:54.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:54 smithi167 ceph-mon[32751]: pgmap v16299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:54.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:54 smithi090 ceph-mon[29180]: pgmap v16299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:46:56.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:56 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29497.440286 secs 2024-04-27T02:46:56.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:56 smithi119 ceph-mon[32752]: pgmap v16300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:56.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:56 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29497.440286 secs 2024-04-27T02:46:56.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:56 smithi167 ceph-mon[32751]: pgmap v16300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:56.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:56 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29497.440286 secs 2024-04-27T02:46:56.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:56 smithi090 ceph-mon[29180]: pgmap v16300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:57.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:57 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29498.281545 secs 2024-04-27T02:46:57.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:57 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29498.281545 secs 2024-04-27T02:46:57.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:57 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29498.281545 secs 2024-04-27T02:46:58.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:46:58 smithi119 ceph-mon[32752]: pgmap v16301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:58.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:46:58 smithi167 ceph-mon[32751]: pgmap v16301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:46:58.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:46:58 smithi090 ceph-mon[29180]: pgmap v16301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:00.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:00 smithi119 ceph-mon[32752]: pgmap v16302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:00.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:00 smithi167 ceph-mon[32751]: pgmap v16302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:00.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:00 smithi090 ceph-mon[29180]: pgmap v16302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:01.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:01 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29502.440370 secs 2024-04-27T02:47:01.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:01 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29502.440370 secs 2024-04-27T02:47:01.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:01 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29502.440370 secs 2024-04-27T02:47:02.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:02 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29503.281608 secs 2024-04-27T02:47:02.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:02 smithi119 ceph-mon[32752]: pgmap v16303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:02.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:02 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29503.281608 secs 2024-04-27T02:47:02.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:02 smithi167 ceph-mon[32751]: pgmap v16303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:02.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:02 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29503.281608 secs 2024-04-27T02:47:02.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:02 smithi090 ceph-mon[29180]: pgmap v16303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:04.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:04 smithi119 ceph-mon[32752]: pgmap v16304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:04.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:04 smithi167 ceph-mon[32751]: pgmap v16304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:04.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:04 smithi090 ceph-mon[29180]: pgmap v16304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:06.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:06 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29507.440450 secs 2024-04-27T02:47:06.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:06 smithi119 ceph-mon[32752]: pgmap v16305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:06.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:06 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29507.440450 secs 2024-04-27T02:47:06.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:06 smithi167 ceph-mon[32751]: pgmap v16305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:06.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:06 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29507.440450 secs 2024-04-27T02:47:06.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:06 smithi090 ceph-mon[29180]: pgmap v16305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:07.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:07 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29508.281738 secs 2024-04-27T02:47:07.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:07 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29508.281738 secs 2024-04-27T02:47:07.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:07 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29508.281738 secs 2024-04-27T02:47:08.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:08 smithi119 ceph-mon[32752]: pgmap v16306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:08.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:08 smithi167 ceph-mon[32751]: pgmap v16306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:08.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:08 smithi090 ceph-mon[29180]: pgmap v16306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:10.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:10 smithi119 ceph-mon[32752]: pgmap v16307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:10.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:10 smithi167 ceph-mon[32751]: pgmap v16307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:10.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:10 smithi090 ceph-mon[29180]: pgmap v16307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:11.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:11 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29512.440535 secs 2024-04-27T02:47:11.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:11 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29512.440535 secs 2024-04-27T02:47:11.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:11 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29512.440535 secs 2024-04-27T02:47:12.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:12 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29513.281875 secs 2024-04-27T02:47:12.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:12 smithi119 ceph-mon[32752]: pgmap v16308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:12.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:12 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29513.281875 secs 2024-04-27T02:47:12.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:12 smithi167 ceph-mon[32751]: pgmap v16308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:12.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:12 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29513.281875 secs 2024-04-27T02:47:12.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:12 smithi090 ceph-mon[29180]: pgmap v16308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:14.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:14 smithi119 ceph-mon[32752]: pgmap v16309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:14.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:14 smithi167 ceph-mon[32751]: pgmap v16309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:14.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:14 smithi090 ceph-mon[29180]: pgmap v16309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:16.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:16 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29517.440618 secs 2024-04-27T02:47:16.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:16 smithi119 ceph-mon[32752]: pgmap v16310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:16.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:16 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29517.440618 secs 2024-04-27T02:47:16.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:16 smithi167 ceph-mon[32751]: pgmap v16310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:16.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:16 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29517.440618 secs 2024-04-27T02:47:16.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:16 smithi090 ceph-mon[29180]: pgmap v16310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:17.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:17 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29518.281992 secs 2024-04-27T02:47:17.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:17 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29518.281992 secs 2024-04-27T02:47:17.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:17 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29518.281992 secs 2024-04-27T02:47:18.747 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:18 smithi090 ceph-mon[29180]: pgmap v16311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:18.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:18 smithi119 ceph-mon[32752]: pgmap v16311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:18.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:18 smithi167 ceph-mon[32751]: pgmap v16311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:19.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:19 smithi119 ceph-mon[32752]: pgmap v16312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:19.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:19 smithi167 ceph-mon[32751]: pgmap v16312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:19.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:19 smithi090 ceph-mon[29180]: pgmap v16312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:21.543 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:21 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29522.440704 secs 2024-04-27T02:47:21.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:21 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29522.440704 secs 2024-04-27T02:47:21.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:21 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29522.440704 secs 2024-04-27T02:47:22.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:22 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29523.282109 secs 2024-04-27T02:47:22.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:22 smithi119 ceph-mon[32752]: pgmap v16313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:22.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:22 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29523.282109 secs 2024-04-27T02:47:22.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:22 smithi167 ceph-mon[32751]: pgmap v16313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:22.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:22 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29523.282109 secs 2024-04-27T02:47:22.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:22 smithi090 ceph-mon[29180]: pgmap v16313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:24.793 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:24 smithi119 ceph-mon[32752]: pgmap v16314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:24.807 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:24 smithi167 ceph-mon[32751]: pgmap v16314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:24.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:24 smithi090 ceph-mon[29180]: pgmap v16314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:26.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:26 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29527.440811 secs 2024-04-27T02:47:26.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:26 smithi119 ceph-mon[32752]: pgmap v16315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:26.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:26 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29527.440811 secs 2024-04-27T02:47:26.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:26 smithi167 ceph-mon[32751]: pgmap v16315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:26.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:26 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29527.440811 secs 2024-04-27T02:47:26.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:26 smithi090 ceph-mon[29180]: pgmap v16315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:27.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:27 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29528.282224 secs 2024-04-27T02:47:27.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:27 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29528.282224 secs 2024-04-27T02:47:27.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:27 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29528.282224 secs 2024-04-27T02:47:28.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:28 smithi119 ceph-mon[32752]: pgmap v16316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:28.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:28 smithi167 ceph-mon[32751]: pgmap v16316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:28.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:28 smithi090 ceph-mon[29180]: pgmap v16316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:30.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:30 smithi119 ceph-mon[32752]: pgmap v16317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:30.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:47:30.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:47:30.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:47:30.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:47:30.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:30 smithi167 ceph-mon[32751]: pgmap v16317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:30.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:47:30.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:47:30.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:47:30.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:47:30.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:30 smithi090 ceph-mon[29180]: pgmap v16317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:30.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:47:30.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:47:30.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:47:30.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:47:31.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:31 smithi119 ceph-mon[32752]: pgmap v16318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:47:31.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:31 smithi119 ceph-mon[32752]: pgmap v16319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:31.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:31 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29532.440849 secs 2024-04-27T02:47:31.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:31 smithi167 ceph-mon[32751]: pgmap v16318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:47:31.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:31 smithi167 ceph-mon[32751]: pgmap v16319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:31.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:31 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29532.440849 secs 2024-04-27T02:47:31.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:31 smithi090 ceph-mon[29180]: pgmap v16318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:47:31.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:31 smithi090 ceph-mon[29180]: pgmap v16319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:31.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:31 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29532.440849 secs 2024-04-27T02:47:32.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:32 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29533.282341 secs 2024-04-27T02:47:32.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:32 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29533.282341 secs 2024-04-27T02:47:32.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:32 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29533.282341 secs 2024-04-27T02:47:33.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:33 smithi119 ceph-mon[32752]: pgmap v16320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:47:33.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:33 smithi167 ceph-mon[32751]: pgmap v16320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:47:33.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:33 smithi090 ceph-mon[29180]: pgmap v16320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:47:35.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:35 smithi119 ceph-mon[32752]: pgmap v16321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:35.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:35 smithi167 ceph-mon[32751]: pgmap v16321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:35.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:35 smithi090 ceph-mon[29180]: pgmap v16321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:36.695 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:36 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29537.440909 secs 2024-04-27T02:47:36.709 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:36 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29537.440909 secs 2024-04-27T02:47:36.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:36 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29537.440909 secs 2024-04-27T02:47:37.767 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:37 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29538.282474 secs 2024-04-27T02:47:37.768 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:37 smithi090 ceph-mon[29180]: pgmap v16322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:37.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:37 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29538.282474 secs 2024-04-27T02:47:37.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:37 smithi119 ceph-mon[32752]: pgmap v16322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:37.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:37 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29538.282474 secs 2024-04-27T02:47:37.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:37 smithi167 ceph-mon[32751]: pgmap v16322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:47:39.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:39 smithi119 ceph-mon[32752]: pgmap v16323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:47:39.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:39 smithi167 ceph-mon[32751]: pgmap v16323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:47:39.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:39 smithi090 ceph-mon[29180]: pgmap v16323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:47:41.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:41 smithi119 ceph-mon[32752]: pgmap v16324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:47:41.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:41 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29542.440972 secs 2024-04-27T02:47:41.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:41 smithi167 ceph-mon[32751]: pgmap v16324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:47:41.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:41 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29542.440972 secs 2024-04-27T02:47:41.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:41 smithi090 ceph-mon[29180]: pgmap v16324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:47:41.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:41 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29542.440972 secs 2024-04-27T02:47:42.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:42 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29543.282543 secs 2024-04-27T02:47:42.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:42 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29543.282543 secs 2024-04-27T02:47:42.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:42 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29543.282543 secs 2024-04-27T02:47:43.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:43 smithi119 ceph-mon[32752]: pgmap v16325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:43.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:43 smithi167 ceph-mon[32751]: pgmap v16325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:43.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:43 smithi090 ceph-mon[29180]: pgmap v16325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:45.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:45 smithi119 ceph-mon[32752]: pgmap v16326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:45.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:45 smithi167 ceph-mon[32751]: pgmap v16326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:45.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:45 smithi090 ceph-mon[29180]: pgmap v16326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:46.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:46 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29547.441055 secs 2024-04-27T02:47:46.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:46 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29547.441055 secs 2024-04-27T02:47:46.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:46 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29547.441055 secs 2024-04-27T02:47:47.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:47 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29548.282695 secs 2024-04-27T02:47:47.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:47 smithi119 ceph-mon[32752]: pgmap v16327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:47.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:47 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29548.282695 secs 2024-04-27T02:47:47.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:47 smithi167 ceph-mon[32751]: pgmap v16327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:47.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:47 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29548.282695 secs 2024-04-27T02:47:47.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:47 smithi090 ceph-mon[29180]: pgmap v16327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:49.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:49 smithi119 ceph-mon[32752]: pgmap v16328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:49.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:49 smithi167 ceph-mon[32751]: pgmap v16328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:49.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:49 smithi090 ceph-mon[29180]: pgmap v16328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:51.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:51 smithi119 ceph-mon[32752]: pgmap v16329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:51.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:51 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29552.441160 secs 2024-04-27T02:47:51.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:51 smithi167 ceph-mon[32751]: pgmap v16329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:51.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:51 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29552.441160 secs 2024-04-27T02:47:51.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:51 smithi090 ceph-mon[29180]: pgmap v16329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:51.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:51 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29552.441160 secs 2024-04-27T02:47:52.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:52 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29553.282832 secs 2024-04-27T02:47:52.808 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:52 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29553.282832 secs 2024-04-27T02:47:52.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:52 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29553.282832 secs 2024-04-27T02:47:53.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:53 smithi119 ceph-mon[32752]: pgmap v16330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:53.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:53 smithi167 ceph-mon[32751]: pgmap v16330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:53.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:53 smithi090 ceph-mon[29180]: pgmap v16330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:55.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:55 smithi119 ceph-mon[32752]: pgmap v16331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:55.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:55 smithi167 ceph-mon[32751]: pgmap v16331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:55.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:55 smithi090 ceph-mon[29180]: pgmap v16331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:47:56.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:56 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29557.441219 secs 2024-04-27T02:47:56.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:56 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29557.441219 secs 2024-04-27T02:47:56.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:56 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29557.441219 secs 2024-04-27T02:47:57.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:57 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29558.282951 secs 2024-04-27T02:47:57.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:57 smithi119 ceph-mon[32752]: pgmap v16332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:57.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:57 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29558.282951 secs 2024-04-27T02:47:57.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:57 smithi167 ceph-mon[32751]: pgmap v16332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:57.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:57 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29558.282951 secs 2024-04-27T02:47:57.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:57 smithi090 ceph-mon[29180]: pgmap v16332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:59.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:47:59 smithi119 ceph-mon[32752]: pgmap v16333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:59.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:47:59 smithi167 ceph-mon[32751]: pgmap v16333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:47:59.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:47:59 smithi090 ceph-mon[29180]: pgmap v16333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:01.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:01 smithi119 ceph-mon[32752]: pgmap v16334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:01.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:01 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29562.441304 secs 2024-04-27T02:48:01.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:01 smithi167 ceph-mon[32751]: pgmap v16334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:01.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:01 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29562.441304 secs 2024-04-27T02:48:01.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:01 smithi090 ceph-mon[29180]: pgmap v16334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:01.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:01 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29562.441304 secs 2024-04-27T02:48:02.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:02 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29563.283078 secs 2024-04-27T02:48:02.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:02 smithi119 ceph-mon[32752]: pgmap v16335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:02.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:02 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29563.283078 secs 2024-04-27T02:48:02.809 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:02 smithi167 ceph-mon[32751]: pgmap v16335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:02.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:02 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29563.283078 secs 2024-04-27T02:48:02.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:02 smithi090 ceph-mon[29180]: pgmap v16335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:05.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:05 smithi119 ceph-mon[32752]: pgmap v16336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:05.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:05 smithi167 ceph-mon[32751]: pgmap v16336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:05.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:05 smithi090 ceph-mon[29180]: pgmap v16336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:06.446 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:06 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29567.441367 secs 2024-04-27T02:48:06.460 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:06 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29567.441367 secs 2024-04-27T02:48:06.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:06 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29567.441367 secs 2024-04-27T02:48:07.518 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:07 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 29568.283170 secs 2024-04-27T02:48:07.519 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:07 smithi090 ceph-mon[29180]: pgmap v16337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:07.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:07 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 29568.283170 secs 2024-04-27T02:48:07.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:07 smithi119 ceph-mon[32752]: pgmap v16337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:07.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:07 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 29568.283170 secs 2024-04-27T02:48:07.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:07 smithi167 ceph-mon[32751]: pgmap v16337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:09.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:09 smithi119 ceph-mon[32752]: pgmap v16338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:09.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:09 smithi167 ceph-mon[32751]: pgmap v16338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:09.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:09 smithi090 ceph-mon[29180]: pgmap v16338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:11.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:11 smithi119 ceph-mon[32752]: pgmap v16339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:11.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:11 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29572.441451 secs 2024-04-27T02:48:11.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:11 smithi167 ceph-mon[32751]: pgmap v16339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:11.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:11 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29572.441451 secs 2024-04-27T02:48:11.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:11 smithi090 ceph-mon[29180]: pgmap v16339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:11.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:11 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29572.441451 secs 2024-04-27T02:48:12.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:12 smithi119 ceph-mon[32752]: 29 slow requests, 2 included below; oldest blocked for > 29573.283297 secs 2024-04-27T02:48:12.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:12 smithi119 ceph-mon[32752]: slow request 182.141586 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:48:12.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:12 smithi119 ceph-mon[32752]: slow request 11645.355854 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:48:12.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:12 smithi167 ceph-mon[32751]: 29 slow requests, 2 included below; oldest blocked for > 29573.283297 secs 2024-04-27T02:48:12.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:12 smithi167 ceph-mon[32751]: slow request 182.141586 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:48:12.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:12 smithi167 ceph-mon[32751]: slow request 11645.355854 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:48:12.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:12 smithi090 ceph-mon[29180]: 29 slow requests, 2 included below; oldest blocked for > 29573.283297 secs 2024-04-27T02:48:12.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:12 smithi090 ceph-mon[29180]: slow request 182.141586 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:48:12.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:12 smithi090 ceph-mon[29180]: slow request 11645.355854 seconds old, received at 2024-04-26T23:34:05.678088+0000: client_request(client.17445:1 getattr pAsLsXsFs #0x1 2024-04-26T23:34:05.671094+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:48:13.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:13 smithi119 ceph-mon[32752]: pgmap v16340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:13.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:13 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:48:13.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:13 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:48:13.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:13 smithi167 ceph-mon[32751]: pgmap v16340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:13.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:13 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:48:13.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:13 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:48:13.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:13 smithi090 ceph-mon[29180]: pgmap v16340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:13.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:13 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:48:13.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:13 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:48:15.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:15 smithi119 ceph-mon[32752]: pgmap v16341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:15.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:15 smithi167 ceph-mon[32751]: pgmap v16341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:15.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:15 smithi090 ceph-mon[29180]: pgmap v16341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:16.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:16 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29577.441534 secs 2024-04-27T02:48:16.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:16 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29577.441534 secs 2024-04-27T02:48:16.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:16 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29577.441534 secs 2024-04-27T02:48:17.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:17 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29578.283436 secs 2024-04-27T02:48:17.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:17 smithi119 ceph-mon[32752]: pgmap v16342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:17.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:17 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29578.283436 secs 2024-04-27T02:48:17.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:17 smithi167 ceph-mon[32751]: pgmap v16342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:17.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:17 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29578.283436 secs 2024-04-27T02:48:17.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:17 smithi090 ceph-mon[29180]: pgmap v16342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:19.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:19 smithi119 ceph-mon[32752]: pgmap v16343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:19.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:19 smithi167 ceph-mon[32751]: pgmap v16343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:19.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:19 smithi090 ceph-mon[29180]: pgmap v16343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:21.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:21 smithi119 ceph-mon[32752]: pgmap v16344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:21.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:21 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29582.441617 secs 2024-04-27T02:48:21.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:21 smithi167 ceph-mon[32751]: pgmap v16344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:21.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:21 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29582.441617 secs 2024-04-27T02:48:21.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:21 smithi090 ceph-mon[29180]: pgmap v16344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:21.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:21 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29582.441617 secs 2024-04-27T02:48:22.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:22 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29583.283553 secs 2024-04-27T02:48:22.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:22 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29583.283553 secs 2024-04-27T02:48:22.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:22 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29583.283553 secs 2024-04-27T02:48:23.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:23 smithi119 ceph-mon[32752]: pgmap v16345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:23.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:23 smithi167 ceph-mon[32751]: pgmap v16345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:23.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:23 smithi090 ceph-mon[29180]: pgmap v16345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:25.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:25 smithi119 ceph-mon[32752]: pgmap v16346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:25.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:25 smithi167 ceph-mon[32751]: pgmap v16346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:25.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:25 smithi090 ceph-mon[29180]: pgmap v16346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:26.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:26 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29587.441698 secs 2024-04-27T02:48:26.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:26 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29587.441698 secs 2024-04-27T02:48:26.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:26 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29587.441698 secs 2024-04-27T02:48:27.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:27 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29588.283704 secs 2024-04-27T02:48:27.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:27 smithi119 ceph-mon[32752]: pgmap v16347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:27.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:27 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29588.283704 secs 2024-04-27T02:48:27.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:27 smithi167 ceph-mon[32751]: pgmap v16347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:27.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:27 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29588.283704 secs 2024-04-27T02:48:27.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:27 smithi090 ceph-mon[29180]: pgmap v16347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:29.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:29 smithi119 ceph-mon[32752]: pgmap v16348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:29.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:29 smithi167 ceph-mon[32751]: pgmap v16348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:29.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:29 smithi090 ceph-mon[29180]: pgmap v16348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:30.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:48:30.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:48:30.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:48:31.480 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:31 smithi119 ceph-mon[32752]: pgmap v16349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:31.481 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:31 smithi119 ceph-mon[32752]: 38 slow requests, 0 included below; oldest blocked for > 29592.441808 secs 2024-04-27T02:48:31.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:31 smithi167 ceph-mon[32751]: pgmap v16349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:31.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:31 smithi167 ceph-mon[32751]: 38 slow requests, 0 included below; oldest blocked for > 29592.441808 secs 2024-04-27T02:48:31.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:31 smithi090 ceph-mon[29180]: pgmap v16349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:31.583 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:31 smithi090 ceph-mon[29180]: 38 slow requests, 0 included below; oldest blocked for > 29592.441808 secs 2024-04-27T02:48:32.487 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:32 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29593.283765 secs 2024-04-27T02:48:32.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:32 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29593.283765 secs 2024-04-27T02:48:32.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:32 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29593.283765 secs 2024-04-27T02:48:33.546 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:33 smithi119 ceph-mon[32752]: pgmap v16350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:33.560 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:33 smithi167 ceph-mon[32751]: pgmap v16350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:33.618 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:33 smithi090 ceph-mon[29180]: pgmap v16350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:35.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: pgmap v16351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:35.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:48:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:48:35.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: pgmap v16351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:35.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:48:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:48:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: pgmap v16351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:35.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:48:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:48:35.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:48:36.697 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:36 smithi119 ceph-mon[32752]: pgmap v16352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:48:36.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:36 smithi119 ceph-mon[32752]: pgmap v16353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:36.698 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:36 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29597.441898 secs 2024-04-27T02:48:36.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:36 smithi167 ceph-mon[32751]: pgmap v16352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:48:36.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:36 smithi167 ceph-mon[32751]: pgmap v16353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:36.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:36 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29597.441898 secs 2024-04-27T02:48:36.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:36 smithi090 ceph-mon[29180]: pgmap v16352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:48:36.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:36 smithi090 ceph-mon[29180]: pgmap v16353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:36.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:36 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29597.441898 secs 2024-04-27T02:48:37.769 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:37 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29598.283914 secs 2024-04-27T02:48:37.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:37 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29598.283914 secs 2024-04-27T02:48:37.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:37 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29598.283914 secs 2024-04-27T02:48:38.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:38 smithi090 ceph-mon[29180]: pgmap v16354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:38.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:38 smithi119 ceph-mon[32752]: pgmap v16354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:38.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:38 smithi167 ceph-mon[32751]: pgmap v16354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:39.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:48:39.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:48:39.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:48:39.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:48:39.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:48:39.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:48:40.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:40 smithi119 ceph-mon[32752]: pgmap v16355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:40.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:40 smithi167 ceph-mon[32751]: pgmap v16355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:40.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:40 smithi090 ceph-mon[29180]: pgmap v16355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:41.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:41 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29602.441985 secs 2024-04-27T02:48:41.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:41 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29602.441985 secs 2024-04-27T02:48:41.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:41 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29602.441985 secs 2024-04-27T02:48:42.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:42 smithi119 ceph-mon[32752]: pgmap v16356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:42.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:42 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29603.284038 secs 2024-04-27T02:48:42.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:42 smithi167 ceph-mon[32751]: pgmap v16356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:42.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:42 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29603.284038 secs 2024-04-27T02:48:42.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:42 smithi090 ceph-mon[29180]: pgmap v16356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:48:42.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:42 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29603.284038 secs 2024-04-27T02:48:44.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:44 smithi119 ceph-mon[32752]: pgmap v16357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T02:48:44.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:44 smithi167 ceph-mon[32751]: pgmap v16357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T02:48:44.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:44 smithi090 ceph-mon[29180]: pgmap v16357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T02:48:46.749 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:46 smithi119 ceph-mon[32752]: pgmap v16358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:48:46.750 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:46 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29607.442089 secs 2024-04-27T02:48:46.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:46 smithi167 ceph-mon[32751]: pgmap v16358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:48:46.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:46 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29607.442089 secs 2024-04-27T02:48:46.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:46 smithi090 ceph-mon[29180]: pgmap v16358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:48:46.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:46 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29607.442089 secs 2024-04-27T02:48:47.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:47 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29608.284165 secs 2024-04-27T02:48:47.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:47 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29608.284165 secs 2024-04-27T02:48:47.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:47 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29608.284165 secs 2024-04-27T02:48:48.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:48 smithi119 ceph-mon[32752]: pgmap v16359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:48.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:48 smithi167 ceph-mon[32751]: pgmap v16359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:48.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:48 smithi090 ceph-mon[29180]: pgmap v16359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:50.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:50 smithi119 ceph-mon[32752]: pgmap v16360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:50.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:50 smithi167 ceph-mon[32751]: pgmap v16360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:50.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:50 smithi090 ceph-mon[29180]: pgmap v16360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:51.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:51 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29612.442174 secs 2024-04-27T02:48:51.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:51 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29612.442174 secs 2024-04-27T02:48:51.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:51 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29612.442174 secs 2024-04-27T02:48:52.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:52 smithi090 ceph-mon[29180]: pgmap v16361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:52.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:52 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29613.284315 secs 2024-04-27T02:48:52.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:52 smithi119 ceph-mon[32752]: pgmap v16361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:52.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:52 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29613.284315 secs 2024-04-27T02:48:52.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:52 smithi167 ceph-mon[32751]: pgmap v16361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:52.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:52 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29613.284315 secs 2024-04-27T02:48:54.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:54 smithi119 ceph-mon[32752]: pgmap v16362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:54.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:54 smithi167 ceph-mon[32751]: pgmap v16362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:54.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:54 smithi090 ceph-mon[29180]: pgmap v16362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:56.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:56 smithi119 ceph-mon[32752]: pgmap v16363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:56.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:56 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29617.442256 secs 2024-04-27T02:48:56.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:56 smithi167 ceph-mon[32751]: pgmap v16363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:56.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:56 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29617.442256 secs 2024-04-27T02:48:56.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:56 smithi090 ceph-mon[29180]: pgmap v16363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:48:56.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:56 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29617.442256 secs 2024-04-27T02:48:57.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:57 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29618.284442 secs 2024-04-27T02:48:57.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:57 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29618.284442 secs 2024-04-27T02:48:57.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:57 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29618.284442 secs 2024-04-27T02:48:58.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:48:58 smithi119 ceph-mon[32752]: pgmap v16364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:58.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:48:58 smithi167 ceph-mon[32751]: pgmap v16364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:48:58.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:48:58 smithi090 ceph-mon[29180]: pgmap v16364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:00.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:00 smithi119 ceph-mon[32752]: pgmap v16365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:00.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:00 smithi167 ceph-mon[32751]: pgmap v16365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:00.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:00 smithi090 ceph-mon[29180]: pgmap v16365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:01.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:01 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29622.442315 secs 2024-04-27T02:49:01.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:01 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29622.442315 secs 2024-04-27T02:49:01.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:01 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29622.442315 secs 2024-04-27T02:49:02.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:02 smithi119 ceph-mon[32752]: pgmap v16366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:02.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:02 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29623.284543 secs 2024-04-27T02:49:02.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:02 smithi167 ceph-mon[32751]: pgmap v16366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:02.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:02 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29623.284543 secs 2024-04-27T02:49:02.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:02 smithi090 ceph-mon[29180]: pgmap v16366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:02.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:02 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29623.284543 secs 2024-04-27T02:49:04.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:04 smithi119 ceph-mon[32752]: pgmap v16367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:04.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:04 smithi167 ceph-mon[32751]: pgmap v16367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:04.868 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:04 smithi090 ceph-mon[29180]: pgmap v16367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:06.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:06 smithi119 ceph-mon[32752]: pgmap v16368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:06.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:06 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29627.442397 secs 2024-04-27T02:49:06.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:06 smithi167 ceph-mon[32751]: pgmap v16368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:06.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:06 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29627.442397 secs 2024-04-27T02:49:06.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:06 smithi090 ceph-mon[29180]: pgmap v16368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:06.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:06 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29627.442397 secs 2024-04-27T02:49:07.770 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:07 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29628.284625 secs 2024-04-27T02:49:07.796 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:07 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29628.284625 secs 2024-04-27T02:49:07.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:07 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29628.284625 secs 2024-04-27T02:49:08.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:08 smithi119 ceph-mon[32752]: pgmap v16369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:08.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:08 smithi167 ceph-mon[32751]: pgmap v16369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:08.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:08 smithi090 ceph-mon[29180]: pgmap v16369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:09.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:09 smithi119 ceph-mon[32752]: pgmap v16370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:09.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:09 smithi167 ceph-mon[32751]: pgmap v16370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:09.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:09 smithi090 ceph-mon[29180]: pgmap v16370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:12.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:11 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29632.442477 secs 2024-04-27T02:49:12.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:11 smithi119 ceph-mon[32752]: pgmap v16371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:12.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:11 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29633.284748 secs 2024-04-27T02:49:12.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:11 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29632.442477 secs 2024-04-27T02:49:12.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:11 smithi167 ceph-mon[32751]: pgmap v16371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:12.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:11 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29633.284748 secs 2024-04-27T02:49:12.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:11 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29632.442477 secs 2024-04-27T02:49:12.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:11 smithi090 ceph-mon[29180]: pgmap v16371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:12.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:11 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29633.284748 secs 2024-04-27T02:49:14.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:13 smithi119 ceph-mon[32752]: pgmap v16372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:14.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:13 smithi167 ceph-mon[32751]: pgmap v16372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:14.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:13 smithi090 ceph-mon[29180]: pgmap v16372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:16.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:15 smithi119 ceph-mon[32752]: pgmap v16373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:16.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:15 smithi167 ceph-mon[32751]: pgmap v16373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:16.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:15 smithi090 ceph-mon[29180]: pgmap v16373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:17.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:16 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29637.442563 secs 2024-04-27T02:49:17.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:16 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29638.284835 secs 2024-04-27T02:49:17.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:16 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29637.442563 secs 2024-04-27T02:49:17.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:16 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29638.284835 secs 2024-04-27T02:49:17.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:16 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29637.442563 secs 2024-04-27T02:49:17.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:16 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29638.284835 secs 2024-04-27T02:49:18.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:17 smithi119 ceph-mon[32752]: pgmap v16374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:18.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:17 smithi167 ceph-mon[32751]: pgmap v16374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:18.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:17 smithi090 ceph-mon[29180]: pgmap v16374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:20.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:19 smithi119 ceph-mon[32752]: pgmap v16375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:20.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:19 smithi167 ceph-mon[32751]: pgmap v16375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:20.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:19 smithi090 ceph-mon[29180]: pgmap v16375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:22.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:21 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29642.442649 secs 2024-04-27T02:49:22.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:21 smithi119 ceph-mon[32752]: pgmap v16376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:22.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:21 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29643.284964 secs 2024-04-27T02:49:22.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:21 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29642.442649 secs 2024-04-27T02:49:22.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:21 smithi167 ceph-mon[32751]: pgmap v16376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:22.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:21 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29643.284964 secs 2024-04-27T02:49:22.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:21 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29642.442649 secs 2024-04-27T02:49:22.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:21 smithi090 ceph-mon[29180]: pgmap v16376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:22.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:21 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29643.284964 secs 2024-04-27T02:49:24.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:23 smithi119 ceph-mon[32752]: pgmap v16377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:24.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:23 smithi167 ceph-mon[32751]: pgmap v16377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:24.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:23 smithi090 ceph-mon[29180]: pgmap v16377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:26.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:25 smithi119 ceph-mon[32752]: pgmap v16378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:26.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:25 smithi167 ceph-mon[32751]: pgmap v16378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:26.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:25 smithi090 ceph-mon[29180]: pgmap v16378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:27.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:26 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29647.442734 secs 2024-04-27T02:49:27.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:26 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29648.285091 secs 2024-04-27T02:49:27.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:26 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29647.442734 secs 2024-04-27T02:49:27.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:26 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29648.285091 secs 2024-04-27T02:49:27.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:26 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29647.442734 secs 2024-04-27T02:49:27.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:26 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29648.285091 secs 2024-04-27T02:49:28.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:28 smithi119 ceph-mon[32752]: pgmap v16379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:28.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:28 smithi167 ceph-mon[32751]: pgmap v16379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:28.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:27 smithi090 ceph-mon[29180]: pgmap v16379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:30.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:30 smithi119 ceph-mon[32752]: pgmap v16380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:30.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:30 smithi167 ceph-mon[32751]: pgmap v16380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:30.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:30 smithi090 ceph-mon[29180]: pgmap v16380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:32.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:32 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29652.442823 secs 2024-04-27T02:49:32.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:32 smithi119 ceph-mon[32752]: pgmap v16381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:32.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:32 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29653.285192 secs 2024-04-27T02:49:32.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:32 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29652.442823 secs 2024-04-27T02:49:32.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:32 smithi167 ceph-mon[32751]: pgmap v16381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:32.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:32 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29653.285192 secs 2024-04-27T02:49:32.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:32 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29652.442823 secs 2024-04-27T02:49:32.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:32 smithi090 ceph-mon[29180]: pgmap v16381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:32.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:32 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29653.285192 secs 2024-04-27T02:49:34.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:34 smithi119 ceph-mon[32752]: pgmap v16382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:34.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:34 smithi167 ceph-mon[32751]: pgmap v16382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:34.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:34 smithi090 ceph-mon[29180]: pgmap v16382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:35.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:49:35.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:49:35.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:49:36.699 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:36 smithi119 ceph-mon[32752]: pgmap v16383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:36.699 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:36 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29657.442913 secs 2024-04-27T02:49:36.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:36 smithi167 ceph-mon[32751]: pgmap v16383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:36.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:36 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29657.442913 secs 2024-04-27T02:49:36.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:36 smithi090 ceph-mon[29180]: pgmap v16383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:36.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:36 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29657.442913 secs 2024-04-27T02:49:37.646 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:37 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29658.285311 secs 2024-04-27T02:49:37.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:37 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29658.285311 secs 2024-04-27T02:49:37.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:37 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29658.285311 secs 2024-04-27T02:49:38.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:38 smithi119 ceph-mon[32752]: pgmap v16384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:38.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:38 smithi167 ceph-mon[32751]: pgmap v16384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:38.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:38 smithi090 ceph-mon[29180]: pgmap v16384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:40.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: pgmap v16385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:40.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:49:40.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:49:40.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: pgmap v16386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-27T02:49:40.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: pgmap v16387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:40.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: pgmap v16385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:40.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:49:40.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:49:40.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: pgmap v16386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-27T02:49:40.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: pgmap v16387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:40.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: pgmap v16385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:40.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:40.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:49:40.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:49:40.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: pgmap v16386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-27T02:49:40.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: pgmap v16387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:40.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:49:41.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:41 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29662.442996 secs 2024-04-27T02:49:41.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:41 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29662.442996 secs 2024-04-27T02:49:41.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:41 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29662.442996 secs 2024-04-27T02:49:42.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:42 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29663.285445 secs 2024-04-27T02:49:42.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:42 smithi090 ceph-mon[29180]: pgmap v16388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:42.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:42 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29663.285445 secs 2024-04-27T02:49:42.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:42 smithi119 ceph-mon[32752]: pgmap v16388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:42.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:42 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29663.285445 secs 2024-04-27T02:49:42.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:42 smithi167 ceph-mon[32751]: pgmap v16388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:44.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:44 smithi119 ceph-mon[32752]: pgmap v16389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:44.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:44 smithi167 ceph-mon[32751]: pgmap v16389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:44.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:44 smithi090 ceph-mon[29180]: pgmap v16389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:46.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:46 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29667.443083 secs 2024-04-27T02:49:46.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:46 smithi119 ceph-mon[32752]: pgmap v16390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:46.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:46 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29667.443083 secs 2024-04-27T02:49:46.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:46 smithi167 ceph-mon[32751]: pgmap v16390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:46.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:46 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29667.443083 secs 2024-04-27T02:49:46.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:46 smithi090 ceph-mon[29180]: pgmap v16390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:47.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:47 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29668.285522 secs 2024-04-27T02:49:47.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:47 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29668.285522 secs 2024-04-27T02:49:47.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:47 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29668.285522 secs 2024-04-27T02:49:48.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:48 smithi119 ceph-mon[32752]: pgmap v16391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-27T02:49:48.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:48 smithi167 ceph-mon[32751]: pgmap v16391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-27T02:49:48.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:48 smithi090 ceph-mon[29180]: pgmap v16391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-27T02:49:50.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:50 smithi119 ceph-mon[32752]: pgmap v16392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:49:50.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:50 smithi167 ceph-mon[32751]: pgmap v16392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:49:50.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:50 smithi090 ceph-mon[29180]: pgmap v16392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:49:51.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:51 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29672.443182 secs 2024-04-27T02:49:51.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:51 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29672.443182 secs 2024-04-27T02:49:51.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:51 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29672.443182 secs 2024-04-27T02:49:52.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:52 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29673.285658 secs 2024-04-27T02:49:52.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:52 smithi119 ceph-mon[32752]: pgmap v16393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:52.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:52 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29673.285658 secs 2024-04-27T02:49:52.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:52 smithi167 ceph-mon[32751]: pgmap v16393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:52.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:52 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29673.285658 secs 2024-04-27T02:49:52.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:52 smithi090 ceph-mon[29180]: pgmap v16393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:54.672 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:54 smithi090 ceph-mon[29180]: pgmap v16394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:54.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:54 smithi119 ceph-mon[32752]: pgmap v16394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:54.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:54 smithi167 ceph-mon[32751]: pgmap v16394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:49:56.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:56 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29677.443279 secs 2024-04-27T02:49:56.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:56 smithi119 ceph-mon[32752]: pgmap v16395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:56.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:56 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29677.443279 secs 2024-04-27T02:49:56.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:56 smithi167 ceph-mon[32751]: pgmap v16395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:56.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:56 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29677.443279 secs 2024-04-27T02:49:56.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:56 smithi090 ceph-mon[29180]: pgmap v16395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:57.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:57 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29678.285794 secs 2024-04-27T02:49:57.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:57 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29678.285794 secs 2024-04-27T02:49:57.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:57 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29678.285794 secs 2024-04-27T02:49:58.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:49:58 smithi119 ceph-mon[32752]: pgmap v16396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:58.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:49:58 smithi167 ceph-mon[32751]: pgmap v16396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:49:58.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:49:58 smithi090 ceph-mon[29180]: pgmap v16396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:00.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:00 smithi119 ceph-mon[32752]: pgmap v16397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:00.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:50:00.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:00 smithi167 ceph-mon[32751]: pgmap v16397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:00.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:50:00.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:00 smithi090 ceph-mon[29180]: pgmap v16397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:00.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T02:50:01.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:01 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29682.443338 secs 2024-04-27T02:50:01.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:01 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29682.443338 secs 2024-04-27T02:50:01.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:01 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29682.443338 secs 2024-04-27T02:50:02.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:02 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29683.285916 secs 2024-04-27T02:50:02.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:02 smithi119 ceph-mon[32752]: pgmap v16398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:02.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:02 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29683.285916 secs 2024-04-27T02:50:02.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:02 smithi167 ceph-mon[32751]: pgmap v16398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:02.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:02 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29683.285916 secs 2024-04-27T02:50:02.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:02 smithi090 ceph-mon[29180]: pgmap v16398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:04.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:04 smithi119 ceph-mon[32752]: pgmap v16399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:04.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:04 smithi167 ceph-mon[32751]: pgmap v16399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:04.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:04 smithi090 ceph-mon[29180]: pgmap v16399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:06.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:06 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29687.443443 secs 2024-04-27T02:50:06.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:06 smithi119 ceph-mon[32752]: pgmap v16400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:06.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:06 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29687.443443 secs 2024-04-27T02:50:06.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:06 smithi167 ceph-mon[32751]: pgmap v16400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:06.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:06 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29687.443443 secs 2024-04-27T02:50:06.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:06 smithi090 ceph-mon[29180]: pgmap v16400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:07.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:07 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 29688.286041 secs 2024-04-27T02:50:07.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:07 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 29688.286041 secs 2024-04-27T02:50:07.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:07 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 29688.286041 secs 2024-04-27T02:50:08.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:08 smithi119 ceph-mon[32752]: pgmap v16401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:08.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:08 smithi167 ceph-mon[32751]: pgmap v16401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:08.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:08 smithi090 ceph-mon[29180]: pgmap v16401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:09.370 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:50:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:50:09.001+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:50:09.371 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:50:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:50:09.001+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:50:09.371 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:50:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:50:09.001+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:50:09.371 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:50:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:50:09.001+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:50:09.371 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:50:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:50:09.001+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:50:10.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:10 smithi119 ceph-mon[32752]: pgmap v16402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:10.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:10 smithi167 ceph-mon[32751]: pgmap v16402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:10.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:10 smithi090 ceph-mon[29180]: pgmap v16402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:11.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:11 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29692.443545 secs 2024-04-27T02:50:11.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:11 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29692.443545 secs 2024-04-27T02:50:11.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:11 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29692.443545 secs 2024-04-27T02:50:12.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:12 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29693.286160 secs 2024-04-27T02:50:12.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:12 smithi119 ceph-mon[32752]: pgmap v16403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:12.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:12 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29693.286160 secs 2024-04-27T02:50:12.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:12 smithi167 ceph-mon[32751]: pgmap v16403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:12.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:12 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29693.286160 secs 2024-04-27T02:50:12.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:12 smithi090 ceph-mon[29180]: pgmap v16403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:13.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:13 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:50:13.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:13 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:50:13.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:13 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:50:13.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:13 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:50:13.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:13 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:50:13.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:13 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:50:14.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:14 smithi119 ceph-mon[32752]: pgmap v16404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T02:50:14.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:14 smithi167 ceph-mon[32751]: pgmap v16404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T02:50:14.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:14 smithi090 ceph-mon[29180]: pgmap v16404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 85 B/s wr, 13 op/s 2024-04-27T02:50:16.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:16 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29697.443639 secs 2024-04-27T02:50:16.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:16 smithi119 ceph-mon[32752]: pgmap v16405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:16.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:16 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29697.443639 secs 2024-04-27T02:50:16.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:16 smithi167 ceph-mon[32751]: pgmap v16405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:16.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:16 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29697.443639 secs 2024-04-27T02:50:16.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:16 smithi090 ceph-mon[29180]: pgmap v16405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:17.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:17 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29698.286279 secs 2024-04-27T02:50:17.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:17 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29698.286279 secs 2024-04-27T02:50:17.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:17 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29698.286279 secs 2024-04-27T02:50:18.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:18 smithi119 ceph-mon[32752]: pgmap v16406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:18.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:18 smithi167 ceph-mon[32751]: pgmap v16406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:18.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:18 smithi090 ceph-mon[29180]: pgmap v16406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:20.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:20 smithi119 ceph-mon[32752]: pgmap v16407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:20.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:20 smithi167 ceph-mon[32751]: pgmap v16407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:20.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:20 smithi090 ceph-mon[29180]: pgmap v16407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:21.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:21 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29702.443743 secs 2024-04-27T02:50:21.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:21 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29702.443743 secs 2024-04-27T02:50:21.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:21 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29702.443743 secs 2024-04-27T02:50:22.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:22 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29703.286415 secs 2024-04-27T02:50:22.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:22 smithi119 ceph-mon[32752]: pgmap v16408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:22.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:22 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29703.286415 secs 2024-04-27T02:50:22.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:22 smithi167 ceph-mon[32751]: pgmap v16408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:22.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:22 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29703.286415 secs 2024-04-27T02:50:22.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:22 smithi090 ceph-mon[29180]: pgmap v16408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:23.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:23 smithi119 ceph-mon[32752]: pgmap v16409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:23.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:23 smithi167 ceph-mon[32751]: pgmap v16409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:23 smithi090 ceph-mon[29180]: pgmap v16409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-27T02:50:26.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:26 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29707.443799 secs 2024-04-27T02:50:26.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:26 smithi119 ceph-mon[32752]: pgmap v16410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:50:26.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:26 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29707.443799 secs 2024-04-27T02:50:26.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:26 smithi167 ceph-mon[32751]: pgmap v16410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:50:26.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:26 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29707.443799 secs 2024-04-27T02:50:26.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:26 smithi090 ceph-mon[29180]: pgmap v16410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T02:50:27.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:27 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29708.286526 secs 2024-04-27T02:50:27.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:27 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29708.286526 secs 2024-04-27T02:50:27.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:27 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29708.286526 secs 2024-04-27T02:50:28.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:28 smithi119 ceph-mon[32752]: pgmap v16411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:28.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:28 smithi167 ceph-mon[32751]: pgmap v16411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:28.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:28 smithi090 ceph-mon[29180]: pgmap v16411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:30.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:30 smithi119 ceph-mon[32752]: pgmap v16412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:30.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:30 smithi167 ceph-mon[32751]: pgmap v16412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:30.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:30 smithi090 ceph-mon[29180]: pgmap v16412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:31.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:31 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29712.443891 secs 2024-04-27T02:50:31.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:31 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29712.443891 secs 2024-04-27T02:50:31.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:31 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29712.443891 secs 2024-04-27T02:50:32.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:32 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29713.286639 secs 2024-04-27T02:50:32.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:32 smithi119 ceph-mon[32752]: pgmap v16413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:32.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:32 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29713.286639 secs 2024-04-27T02:50:32.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:32 smithi167 ceph-mon[32751]: pgmap v16413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:32.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:32 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29713.286639 secs 2024-04-27T02:50:32.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:32 smithi090 ceph-mon[29180]: pgmap v16413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:34.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:34 smithi119 ceph-mon[32752]: pgmap v16414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:34.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:34 smithi167 ceph-mon[32751]: pgmap v16414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:34.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:34 smithi090 ceph-mon[29180]: pgmap v16414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:36.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:36 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29717.443990 secs 2024-04-27T02:50:36.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:36 smithi119 ceph-mon[32752]: pgmap v16415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:36.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:36 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29717.443990 secs 2024-04-27T02:50:36.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:36 smithi167 ceph-mon[32751]: pgmap v16415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:36.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:36 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29717.443990 secs 2024-04-27T02:50:36.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:36 smithi090 ceph-mon[29180]: pgmap v16415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:37.772 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:37 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29718.286750 secs 2024-04-27T02:50:37.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:37 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29718.286750 secs 2024-04-27T02:50:37.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:37 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29718.286750 secs 2024-04-27T02:50:38.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:38 smithi119 ceph-mon[32752]: pgmap v16416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:38.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:38 smithi167 ceph-mon[32751]: pgmap v16416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:38.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:38 smithi090 ceph-mon[29180]: pgmap v16416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:40.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:40 smithi119 ceph-mon[32752]: pgmap v16417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:40.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:50:40.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:50:40.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:50:40.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:50:40.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:40 smithi167 ceph-mon[32751]: pgmap v16417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:40.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:50:40.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:50:40.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:50:40.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:50:40.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:40 smithi090 ceph-mon[29180]: pgmap v16417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:40.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:50:40.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:50:40.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:50:40.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:50:41.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:41 smithi119 ceph-mon[32752]: pgmap v16418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T02:50:41.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:41 smithi119 ceph-mon[32752]: pgmap v16419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:41.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:41 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29722.444069 secs 2024-04-27T02:50:41.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:41 smithi167 ceph-mon[32751]: pgmap v16418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T02:50:41.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:41 smithi167 ceph-mon[32751]: pgmap v16419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:41.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:41 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29722.444069 secs 2024-04-27T02:50:41.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:41 smithi090 ceph-mon[29180]: pgmap v16418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T02:50:41.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:41 smithi090 ceph-mon[29180]: pgmap v16419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:41.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:41 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29722.444069 secs 2024-04-27T02:50:42.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:42 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29723.286862 secs 2024-04-27T02:50:42.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:42 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29723.286862 secs 2024-04-27T02:50:42.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:42 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29723.286862 secs 2024-04-27T02:50:43.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:43 smithi119 ceph-mon[32752]: pgmap v16420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:50:43.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:43 smithi167 ceph-mon[32751]: pgmap v16420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:50:43.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:43 smithi090 ceph-mon[29180]: pgmap v16420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:50:45.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:45 smithi119 ceph-mon[32752]: pgmap v16421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:45.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:45 smithi167 ceph-mon[32751]: pgmap v16421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:45.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:45 smithi090 ceph-mon[29180]: pgmap v16421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:46.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:46 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29727.444170 secs 2024-04-27T02:50:46.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:46 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29727.444170 secs 2024-04-27T02:50:46.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:46 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29727.444170 secs 2024-04-27T02:50:47.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:47 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29728.286956 secs 2024-04-27T02:50:47.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:47 smithi119 ceph-mon[32752]: pgmap v16422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:47.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:47 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29728.286956 secs 2024-04-27T02:50:47.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:47 smithi167 ceph-mon[32751]: pgmap v16422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:47.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:47 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29728.286956 secs 2024-04-27T02:50:47.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:47 smithi090 ceph-mon[29180]: pgmap v16422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:50:49.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:49 smithi119 ceph-mon[32752]: pgmap v16423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:50:49.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:49 smithi167 ceph-mon[32751]: pgmap v16423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:50:49.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:49 smithi090 ceph-mon[29180]: pgmap v16423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T02:50:51.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:51 smithi119 ceph-mon[32752]: pgmap v16424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:50:51.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:51 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29732.444274 secs 2024-04-27T02:50:51.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:51 smithi167 ceph-mon[32751]: pgmap v16424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:50:51.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:51 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29732.444274 secs 2024-04-27T02:50:51.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:51 smithi090 ceph-mon[29180]: pgmap v16424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:50:51.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:51 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29732.444274 secs 2024-04-27T02:50:52.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:52 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29733.287064 secs 2024-04-27T02:50:52.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:52 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29733.287064 secs 2024-04-27T02:50:52.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:52 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29733.287064 secs 2024-04-27T02:50:53.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:53 smithi119 ceph-mon[32752]: pgmap v16425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:53.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:53 smithi167 ceph-mon[32751]: pgmap v16425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:53.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:53 smithi090 ceph-mon[29180]: pgmap v16425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:55.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:55 smithi119 ceph-mon[32752]: pgmap v16426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:55.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:55 smithi167 ceph-mon[32751]: pgmap v16426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:55.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:55 smithi090 ceph-mon[29180]: pgmap v16426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:50:56.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:56 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29737.444334 secs 2024-04-27T02:50:56.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:56 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29737.444334 secs 2024-04-27T02:50:56.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:56 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29737.444334 secs 2024-04-27T02:50:57.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:57 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29738.287178 secs 2024-04-27T02:50:57.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:57 smithi119 ceph-mon[32752]: pgmap v16427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:57.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:57 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29738.287178 secs 2024-04-27T02:50:57.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:57 smithi167 ceph-mon[32751]: pgmap v16427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:57.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:57 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29738.287178 secs 2024-04-27T02:50:57.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:57 smithi090 ceph-mon[29180]: pgmap v16427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:59.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:50:59 smithi119 ceph-mon[32752]: pgmap v16428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:59.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:50:59 smithi167 ceph-mon[32751]: pgmap v16428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:50:59.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:50:59 smithi090 ceph-mon[29180]: pgmap v16428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:01.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:01 smithi119 ceph-mon[32752]: pgmap v16429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:01.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:01 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29742.444418 secs 2024-04-27T02:51:01.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:01 smithi167 ceph-mon[32751]: pgmap v16429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:01.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:01 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29742.444418 secs 2024-04-27T02:51:01.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:01 smithi090 ceph-mon[29180]: pgmap v16429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:01.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:01 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29742.444418 secs 2024-04-27T02:51:02.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:02 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29743.287288 secs 2024-04-27T02:51:02.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:02 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29743.287288 secs 2024-04-27T02:51:02.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:02 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29743.287288 secs 2024-04-27T02:51:03.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:03 smithi119 ceph-mon[32752]: pgmap v16430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:03.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:03 smithi167 ceph-mon[32751]: pgmap v16430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:03.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:03 smithi090 ceph-mon[29180]: pgmap v16430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:04.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:04 smithi119 ceph-mon[32752]: pgmap v16431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:04.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:04 smithi167 ceph-mon[32751]: pgmap v16431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:04.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:04 smithi090 ceph-mon[29180]: pgmap v16431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:06.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:06 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29747.444498 secs 2024-04-27T02:51:06.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:06 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29747.444498 secs 2024-04-27T02:51:06.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:06 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29747.444498 secs 2024-04-27T02:51:07.522 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:07 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29748.287409 secs 2024-04-27T02:51:07.523 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:07 smithi090 ceph-mon[29180]: pgmap v16432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:07.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:07 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29748.287409 secs 2024-04-27T02:51:07.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:07 smithi119 ceph-mon[32752]: pgmap v16432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:07.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:07 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29748.287409 secs 2024-04-27T02:51:07.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:07 smithi167 ceph-mon[32751]: pgmap v16432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:09.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:09 smithi119 ceph-mon[32752]: pgmap v16433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:09.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:09 smithi167 ceph-mon[32751]: pgmap v16433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:09.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:09 smithi090 ceph-mon[29180]: pgmap v16433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:11.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:11 smithi119 ceph-mon[32752]: pgmap v16434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:11.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:11 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29752.444583 secs 2024-04-27T02:51:11.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:11 smithi167 ceph-mon[32751]: pgmap v16434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:11.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:11 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29752.444583 secs 2024-04-27T02:51:11.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:11 smithi090 ceph-mon[29180]: pgmap v16434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:11.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:11 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29752.444583 secs 2024-04-27T02:51:12.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:12 smithi119 ceph-mon[32752]: 12 slow requests, 1 included below; oldest blocked for > 29753.287503 secs 2024-04-27T02:51:12.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:12 smithi119 ceph-mon[32752]: slow request 362.145793 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:51:12.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:12 smithi167 ceph-mon[32751]: 12 slow requests, 1 included below; oldest blocked for > 29753.287503 secs 2024-04-27T02:51:12.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:12 smithi167 ceph-mon[32751]: slow request 362.145793 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:51:12.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:12 smithi090 ceph-mon[29180]: 12 slow requests, 1 included below; oldest blocked for > 29753.287503 secs 2024-04-27T02:51:12.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:12 smithi090 ceph-mon[29180]: slow request 362.145793 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:51:13.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:13 smithi119 ceph-mon[32752]: pgmap v16435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:13.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:13 smithi167 ceph-mon[32751]: pgmap v16435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:13.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:13 smithi090 ceph-mon[29180]: pgmap v16435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T02:51:15.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:15 smithi119 ceph-mon[32752]: pgmap v16436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:15.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:15 smithi167 ceph-mon[32751]: pgmap v16436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:15.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:15 smithi090 ceph-mon[29180]: pgmap v16436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T02:51:16.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:16 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29757.444685 secs 2024-04-27T02:51:16.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:16 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29757.444685 secs 2024-04-27T02:51:16.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:16 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29757.444685 secs 2024-04-27T02:51:17.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:17 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29758.287633 secs 2024-04-27T02:51:17.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:17 smithi119 ceph-mon[32752]: pgmap v16437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:17.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:17 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29758.287633 secs 2024-04-27T02:51:17.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:17 smithi167 ceph-mon[32751]: pgmap v16437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:17 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29758.287633 secs 2024-04-27T02:51:17.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:17 smithi090 ceph-mon[29180]: pgmap v16437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:19.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:19 smithi119 ceph-mon[32752]: pgmap v16438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:19.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:19 smithi167 ceph-mon[32751]: pgmap v16438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:19.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:19 smithi090 ceph-mon[29180]: pgmap v16438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:21.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:21 smithi119 ceph-mon[32752]: pgmap v16439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:21.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:21 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29762.444772 secs 2024-04-27T02:51:21.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:21 smithi167 ceph-mon[32751]: pgmap v16439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:21.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:21 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29762.444772 secs 2024-04-27T02:51:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:21 smithi090 ceph-mon[29180]: pgmap v16439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:21.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:21 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29762.444772 secs 2024-04-27T02:51:22.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:22 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29763.287763 secs 2024-04-27T02:51:22.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:22 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29763.287763 secs 2024-04-27T02:51:22.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:22 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29763.287763 secs 2024-04-27T02:51:23.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:23 smithi119 ceph-mon[32752]: pgmap v16440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:23.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:23 smithi167 ceph-mon[32751]: pgmap v16440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:23.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:23 smithi090 ceph-mon[29180]: pgmap v16440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:25.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:25 smithi119 ceph-mon[32752]: pgmap v16441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:25.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:25 smithi167 ceph-mon[32751]: pgmap v16441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:25.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:25 smithi090 ceph-mon[29180]: pgmap v16441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:26.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:26 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29767.444828 secs 2024-04-27T02:51:26.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:26 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29767.444828 secs 2024-04-27T02:51:26.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:26 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29767.444828 secs 2024-04-27T02:51:27.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:27 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29768.287886 secs 2024-04-27T02:51:27.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:27 smithi119 ceph-mon[32752]: pgmap v16442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:27.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:27 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29768.287886 secs 2024-04-27T02:51:27.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:27 smithi167 ceph-mon[32751]: pgmap v16442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:27.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:27 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29768.287886 secs 2024-04-27T02:51:27.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:27 smithi090 ceph-mon[29180]: pgmap v16442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:29.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:29 smithi119 ceph-mon[32752]: pgmap v16443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:29.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:29 smithi167 ceph-mon[32751]: pgmap v16443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:29.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:29 smithi090 ceph-mon[29180]: pgmap v16443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:31.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:31 smithi167 ceph-mon[32751]: pgmap v16444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:31.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:31 smithi167 ceph-mon[32751]: 3 slow requests, 0 included below; oldest blocked for > 29772.444912 secs 2024-04-27T02:51:31.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:31 smithi090 ceph-mon[29180]: pgmap v16444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:31.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:31 smithi090 ceph-mon[29180]: 3 slow requests, 0 included below; oldest blocked for > 29772.444912 secs 2024-04-27T02:51:31.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:31 smithi119 ceph-mon[32752]: pgmap v16444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:31.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:31 smithi119 ceph-mon[32752]: 3 slow requests, 0 included below; oldest blocked for > 29772.444912 secs 2024-04-27T02:51:32.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:32 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29773.287958 secs 2024-04-27T02:51:32.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:32 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29773.287958 secs 2024-04-27T02:51:32.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:32 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29773.287958 secs 2024-04-27T02:51:33.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:33 smithi167 ceph-mon[32751]: pgmap v16445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:33.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:33 smithi090 ceph-mon[29180]: pgmap v16445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:33.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:33 smithi119 ceph-mon[32752]: pgmap v16445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:35.564 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:35 smithi167 ceph-mon[32751]: pgmap v16446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:35.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:35 smithi090 ceph-mon[29180]: pgmap v16446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:35.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:35 smithi119 ceph-mon[32752]: pgmap v16446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:36.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:36 smithi090 ceph-mon[29180]: 39 slow requests, 1 included below; oldest blocked for > 29777.444996 secs 2024-04-27T02:51:36.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:36 smithi090 ceph-mon[29180]: slow request 181.685337 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:51:36.701 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:36 smithi119 ceph-mon[32752]: 39 slow requests, 1 included below; oldest blocked for > 29777.444996 secs 2024-04-27T02:51:36.701 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:36 smithi119 ceph-mon[32752]: slow request 181.685337 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:51:36.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:36 smithi167 ceph-mon[32751]: 39 slow requests, 1 included below; oldest blocked for > 29777.444996 secs 2024-04-27T02:51:36.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:36 smithi167 ceph-mon[32751]: slow request 181.685337 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:51:37.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:37 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29778.288019 secs 2024-04-27T02:51:37.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:37 smithi090 ceph-mon[29180]: pgmap v16447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:37.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:37 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29778.288019 secs 2024-04-27T02:51:37.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:37 smithi119 ceph-mon[32752]: pgmap v16447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:37.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:37 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29778.288019 secs 2024-04-27T02:51:37.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:37 smithi167 ceph-mon[32751]: pgmap v16447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:39.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:39 smithi090 ceph-mon[29180]: pgmap v16448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:39 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:51:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:39 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:51:39.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:39 smithi119 ceph-mon[32752]: pgmap v16448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:39.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:39 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:51:39.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:39 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:51:39.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:39 smithi167 ceph-mon[32751]: pgmap v16448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:39.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:39 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:51:39.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:39 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:51:40.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:51:40.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:51:40.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:51:41.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:41 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29782.445059 secs 2024-04-27T02:51:41.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:41 smithi090 ceph-mon[29180]: pgmap v16449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:51:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:51:41.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:51:41.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:41 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29782.445059 secs 2024-04-27T02:51:41.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:41 smithi119 ceph-mon[32752]: pgmap v16449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:41.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:51:41.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:51:41.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:51:41.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:41 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29782.445059 secs 2024-04-27T02:51:41.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:41 smithi167 ceph-mon[32751]: pgmap v16449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:41.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:51:41.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:51:41.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:51:42.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:42 smithi090 ceph-mon[29180]: pgmap v16450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:51:42.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:42 smithi090 ceph-mon[29180]: pgmap v16451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:42.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:42 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29783.288142 secs 2024-04-27T02:51:42.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:42 smithi119 ceph-mon[32752]: pgmap v16450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:51:42.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:42 smithi119 ceph-mon[32752]: pgmap v16451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:42.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:42 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29783.288142 secs 2024-04-27T02:51:42.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:42 smithi167 ceph-mon[32751]: pgmap v16450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:51:42.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:42 smithi167 ceph-mon[32751]: pgmap v16451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:42.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:42 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29783.288142 secs 2024-04-27T02:51:44.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:44 smithi090 ceph-mon[29180]: pgmap v16452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:51:44.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:44 smithi119 ceph-mon[32752]: pgmap v16452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:51:44.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:44 smithi167 ceph-mon[32751]: pgmap v16452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:51:46.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:46 smithi090 ceph-mon[29180]: pgmap v16453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:46.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:46 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29787.445118 secs 2024-04-27T02:51:46.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:46 smithi119 ceph-mon[32752]: pgmap v16453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:46.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:46 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29787.445118 secs 2024-04-27T02:51:46.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:46 smithi167 ceph-mon[32751]: pgmap v16453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:46.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:46 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29787.445118 secs 2024-04-27T02:51:47.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:47 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29788.288270 secs 2024-04-27T02:51:47.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:47 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29788.288270 secs 2024-04-27T02:51:47.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:47 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29788.288270 secs 2024-04-27T02:51:48.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:48 smithi090 ceph-mon[29180]: pgmap v16454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:48.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:48 smithi119 ceph-mon[32752]: pgmap v16454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:48.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:48 smithi167 ceph-mon[32751]: pgmap v16454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:50 smithi090 ceph-mon[29180]: pgmap v16455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:50.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:50 smithi119 ceph-mon[32752]: pgmap v16455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:50.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:50 smithi167 ceph-mon[32751]: pgmap v16455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:51:51.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:51 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29792.445201 secs 2024-04-27T02:51:51.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:51 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29792.445201 secs 2024-04-27T02:51:51.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:51 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29792.445201 secs 2024-04-27T02:51:52.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:52 smithi090 ceph-mon[29180]: pgmap v16456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:51:52.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:52 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29793.288423 secs 2024-04-27T02:51:52.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:52 smithi119 ceph-mon[32752]: pgmap v16456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:51:52.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:52 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29793.288423 secs 2024-04-27T02:51:52.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:52 smithi167 ceph-mon[32751]: pgmap v16456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:51:52.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:52 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29793.288423 secs 2024-04-27T02:51:54.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:54 smithi090 ceph-mon[29180]: pgmap v16457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:54.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:54 smithi119 ceph-mon[32752]: pgmap v16457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:54.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:54 smithi167 ceph-mon[32751]: pgmap v16457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:56.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:56 smithi090 ceph-mon[29180]: pgmap v16458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:56.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:56 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29797.445280 secs 2024-04-27T02:51:56.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:56 smithi119 ceph-mon[32752]: pgmap v16458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:56.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:56 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29797.445280 secs 2024-04-27T02:51:56.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:56 smithi167 ceph-mon[32751]: pgmap v16458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:51:56.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:56 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29797.445280 secs 2024-04-27T02:51:57.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:57 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29798.288491 secs 2024-04-27T02:51:57.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:57 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29798.288491 secs 2024-04-27T02:51:57.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:57 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29798.288491 secs 2024-04-27T02:51:58.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:51:58 smithi090 ceph-mon[29180]: pgmap v16459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:58.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:51:58 smithi119 ceph-mon[32752]: pgmap v16459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:51:58.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:51:58 smithi167 ceph-mon[32751]: pgmap v16459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:00.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:00 smithi119 ceph-mon[32752]: pgmap v16460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:00.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:00 smithi167 ceph-mon[32751]: pgmap v16460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:00.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:00 smithi090 ceph-mon[29180]: pgmap v16460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:01.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:01 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29802.445363 secs 2024-04-27T02:52:01.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:01 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29802.445363 secs 2024-04-27T02:52:01.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:01 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29802.445363 secs 2024-04-27T02:52:02.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:02 smithi119 ceph-mon[32752]: pgmap v16461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:02.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:02 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29803.288623 secs 2024-04-27T02:52:02.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:02 smithi167 ceph-mon[32751]: pgmap v16461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:02.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:02 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29803.288623 secs 2024-04-27T02:52:02.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:02 smithi090 ceph-mon[29180]: pgmap v16461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:02.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:02 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29803.288623 secs 2024-04-27T02:52:04.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:04 smithi119 ceph-mon[32752]: pgmap v16462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:04.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:04 smithi167 ceph-mon[32751]: pgmap v16462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:04.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:04 smithi090 ceph-mon[29180]: pgmap v16462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:06.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:06 smithi119 ceph-mon[32752]: pgmap v16463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:06.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:06 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29807.445446 secs 2024-04-27T02:52:06.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:06 smithi167 ceph-mon[32751]: pgmap v16463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:06.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:06 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29807.445446 secs 2024-04-27T02:52:06.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:06 smithi090 ceph-mon[29180]: pgmap v16463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:06.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:06 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29807.445446 secs 2024-04-27T02:52:07.774 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:07 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29808.288734 secs 2024-04-27T02:52:07.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:07 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29808.288734 secs 2024-04-27T02:52:07.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:07 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29808.288734 secs 2024-04-27T02:52:08.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:08 smithi119 ceph-mon[32752]: pgmap v16464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:08.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:08 smithi167 ceph-mon[32751]: pgmap v16464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:08.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:08 smithi090 ceph-mon[29180]: pgmap v16464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:10.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:10 smithi119 ceph-mon[32752]: pgmap v16465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:10.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:10 smithi167 ceph-mon[32751]: pgmap v16465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:10.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:10 smithi090 ceph-mon[29180]: pgmap v16465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:11 smithi119 ceph-mon[32752]: 39 slow requests, 1 included below; oldest blocked for > 29812.445505 secs 2024-04-27T02:52:11.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:11 smithi119 ceph-mon[32752]: slow request 721.372924 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:52:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:11 smithi167 ceph-mon[32751]: 39 slow requests, 1 included below; oldest blocked for > 29812.445505 secs 2024-04-27T02:52:11.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:11 smithi167 ceph-mon[32751]: slow request 721.372924 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:52:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:11 smithi090 ceph-mon[29180]: 39 slow requests, 1 included below; oldest blocked for > 29812.445505 secs 2024-04-27T02:52:11.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:11 smithi090 ceph-mon[29180]: slow request 721.372924 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:52:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:12 smithi119 ceph-mon[32752]: pgmap v16466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:12.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:12 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29813.288796 secs 2024-04-27T02:52:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:12 smithi167 ceph-mon[32751]: pgmap v16466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:12.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:12 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29813.288796 secs 2024-04-27T02:52:12.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:12 smithi090 ceph-mon[29180]: pgmap v16466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:12.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:12 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29813.288796 secs 2024-04-27T02:52:14.691 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:14 smithi090 ceph-mon[29180]: pgmap v16467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:14.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:14 smithi119 ceph-mon[32752]: pgmap v16467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:14.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:14 smithi167 ceph-mon[32751]: pgmap v16467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:16.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:16 smithi119 ceph-mon[32752]: pgmap v16468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:16.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:16 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29817.445584 secs 2024-04-27T02:52:16.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:16 smithi167 ceph-mon[32751]: pgmap v16468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:16.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:16 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29817.445584 secs 2024-04-27T02:52:16.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:16 smithi090 ceph-mon[29180]: pgmap v16468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:16.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:16 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29817.445584 secs 2024-04-27T02:52:17.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:17 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29818.288941 secs 2024-04-27T02:52:17.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:17 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29818.288941 secs 2024-04-27T02:52:17.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:17 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29818.288941 secs 2024-04-27T02:52:18.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:18 smithi119 ceph-mon[32752]: pgmap v16469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:18.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:18 smithi167 ceph-mon[32751]: pgmap v16469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:18.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:18 smithi090 ceph-mon[29180]: pgmap v16469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:20.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:20 smithi119 ceph-mon[32752]: pgmap v16470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:20.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:20 smithi167 ceph-mon[32751]: pgmap v16470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:20.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:20 smithi090 ceph-mon[29180]: pgmap v16470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:21.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:21 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29822.445643 secs 2024-04-27T02:52:21.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:21 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29822.445643 secs 2024-04-27T02:52:21.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:21 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29822.445643 secs 2024-04-27T02:52:22.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:22 smithi119 ceph-mon[32752]: pgmap v16471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:22.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:22 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29823.289036 secs 2024-04-27T02:52:22.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:22 smithi167 ceph-mon[32751]: pgmap v16471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:22.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:22 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29823.289036 secs 2024-04-27T02:52:22.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:22 smithi090 ceph-mon[29180]: pgmap v16471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:22.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:22 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29823.289036 secs 2024-04-27T02:52:24.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:24 smithi119 ceph-mon[32752]: pgmap v16472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:24.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:24 smithi167 ceph-mon[32751]: pgmap v16472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:24.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:24 smithi090 ceph-mon[29180]: pgmap v16472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:26.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:26 smithi119 ceph-mon[32752]: pgmap v16473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:26.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:26 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29827.445724 secs 2024-04-27T02:52:26.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:26 smithi167 ceph-mon[32751]: pgmap v16473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:26.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:26 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29827.445724 secs 2024-04-27T02:52:26.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:26 smithi090 ceph-mon[29180]: pgmap v16473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:26.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:26 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29827.445724 secs 2024-04-27T02:52:27.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:27 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29828.289130 secs 2024-04-27T02:52:27.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:27 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29828.289130 secs 2024-04-27T02:52:27.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:27 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29828.289130 secs 2024-04-27T02:52:28.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:28 smithi119 ceph-mon[32752]: pgmap v16474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:28.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:28 smithi167 ceph-mon[32751]: pgmap v16474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:28.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:28 smithi090 ceph-mon[29180]: pgmap v16474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:30.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:30 smithi119 ceph-mon[32752]: pgmap v16475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:30.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:30 smithi167 ceph-mon[32751]: pgmap v16475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:30.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:30 smithi090 ceph-mon[29180]: pgmap v16475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:31.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:31 smithi119 ceph-mon[32752]: 39 slow requests, 1 included below; oldest blocked for > 29832.445807 secs 2024-04-27T02:52:31.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:31 smithi119 ceph-mon[32752]: slow request 5760.547492 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:52:31.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:31 smithi167 ceph-mon[32751]: 39 slow requests, 1 included below; oldest blocked for > 29832.445807 secs 2024-04-27T02:52:31.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:31 smithi167 ceph-mon[32751]: slow request 5760.547492 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:52:31.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:31 smithi090 ceph-mon[29180]: 39 slow requests, 1 included below; oldest blocked for > 29832.445807 secs 2024-04-27T02:52:31.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:31 smithi090 ceph-mon[29180]: slow request 5760.547492 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:52:32.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:32 smithi119 ceph-mon[32752]: pgmap v16476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:32.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:32 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29833.289214 secs 2024-04-27T02:52:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:32 smithi167 ceph-mon[32751]: pgmap v16476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:32 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29833.289214 secs 2024-04-27T02:52:32.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:32 smithi090 ceph-mon[29180]: pgmap v16476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:32.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:32 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29833.289214 secs 2024-04-27T02:52:34.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:34 smithi119 ceph-mon[32752]: pgmap v16477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:34.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:34 smithi167 ceph-mon[32751]: pgmap v16477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:34.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:34 smithi090 ceph-mon[29180]: pgmap v16477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:36.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:36 smithi119 ceph-mon[32752]: pgmap v16478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:36.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:36 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29837.445890 secs 2024-04-27T02:52:36.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:36 smithi167 ceph-mon[32751]: pgmap v16478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:36.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:36 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29837.445890 secs 2024-04-27T02:52:36.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:36 smithi090 ceph-mon[29180]: pgmap v16478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:36.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:36 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29837.445890 secs 2024-04-27T02:52:37.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:37 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29838.289289 secs 2024-04-27T02:52:37.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:37 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29838.289289 secs 2024-04-27T02:52:37.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:37 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29838.289289 secs 2024-04-27T02:52:38.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:38 smithi119 ceph-mon[32752]: pgmap v16479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:38.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:38 smithi167 ceph-mon[32751]: pgmap v16479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:38.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:38 smithi090 ceph-mon[29180]: pgmap v16479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:40.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:40 smithi119 ceph-mon[32752]: pgmap v16480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:40.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:40 smithi167 ceph-mon[32751]: pgmap v16480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:40.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:40 smithi090 ceph-mon[29180]: pgmap v16480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:41.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:41 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29842.445975 secs 2024-04-27T02:52:41.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:52:41.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:41 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29842.445975 secs 2024-04-27T02:52:41.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:52:41.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:41 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29842.445975 secs 2024-04-27T02:52:41.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:52:42.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:42 smithi119 ceph-mon[32752]: pgmap v16481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:42.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:42 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29843.289396 secs 2024-04-27T02:52:42.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:52:42.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:52:42.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:52:42.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:42 smithi167 ceph-mon[32751]: pgmap v16481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:42.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:42 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29843.289396 secs 2024-04-27T02:52:42.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:52:42.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:52:42.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:52:42.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:42 smithi090 ceph-mon[29180]: pgmap v16481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:42 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29843.289396 secs 2024-04-27T02:52:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:52:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:52:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:52:43.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:43 smithi119 ceph-mon[32752]: pgmap v16482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 B/s rd, 1 op/s 2024-04-27T02:52:43.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:43 smithi119 ceph-mon[32752]: pgmap v16483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:43.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:43 smithi167 ceph-mon[32751]: pgmap v16482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 B/s rd, 1 op/s 2024-04-27T02:52:43.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:43 smithi167 ceph-mon[32751]: pgmap v16483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:43.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:43 smithi090 ceph-mon[29180]: pgmap v16482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 955 B/s rd, 1 op/s 2024-04-27T02:52:43.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:43 smithi090 ceph-mon[29180]: pgmap v16483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:45.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:45 smithi119 ceph-mon[32752]: pgmap v16484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:45.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:45 smithi167 ceph-mon[32751]: pgmap v16484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:45.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:45 smithi090 ceph-mon[29180]: pgmap v16484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:46.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:46 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29847.446056 secs 2024-04-27T02:52:46.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:46 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29847.446056 secs 2024-04-27T02:52:46.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:46 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29847.446056 secs 2024-04-27T02:52:47.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:47 smithi119 ceph-mon[32752]: pgmap v16485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:47.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:47 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29848.289520 secs 2024-04-27T02:52:47.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:47 smithi167 ceph-mon[32751]: pgmap v16485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:47.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:47 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29848.289520 secs 2024-04-27T02:52:47.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:47 smithi090 ceph-mon[29180]: pgmap v16485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:47.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:47 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29848.289520 secs 2024-04-27T02:52:49.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:49 smithi119 ceph-mon[32752]: pgmap v16486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-27T02:52:49.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:49 smithi167 ceph-mon[32751]: pgmap v16486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-27T02:52:49.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:49 smithi090 ceph-mon[29180]: pgmap v16486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 587 B/s rd, 1 op/s 2024-04-27T02:52:51.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:51 smithi119 ceph-mon[32752]: pgmap v16487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:51.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:51 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29852.446139 secs 2024-04-27T02:52:51.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:51 smithi167 ceph-mon[32751]: pgmap v16487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:51.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:51 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29852.446139 secs 2024-04-27T02:52:51.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:51 smithi090 ceph-mon[29180]: pgmap v16487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:52:51.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:51 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29852.446139 secs 2024-04-27T02:52:52.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:52 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29853.289643 secs 2024-04-27T02:52:52.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:52 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29853.289643 secs 2024-04-27T02:52:52.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:52 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29853.289643 secs 2024-04-27T02:52:53.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:53 smithi119 ceph-mon[32752]: pgmap v16488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:52:53.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:53 smithi167 ceph-mon[32751]: pgmap v16488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:52:53.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:53 smithi090 ceph-mon[29180]: pgmap v16488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:52:55.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:55 smithi119 ceph-mon[32752]: pgmap v16489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:55.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:55 smithi167 ceph-mon[32751]: pgmap v16489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:55.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:55 smithi090 ceph-mon[29180]: pgmap v16489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:52:56.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:56 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29857.446221 secs 2024-04-27T02:52:56.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:56 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29857.446221 secs 2024-04-27T02:52:56.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:56 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29857.446221 secs 2024-04-27T02:52:57.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:57 smithi119 ceph-mon[32752]: pgmap v16490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:57.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:57 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29858.289767 secs 2024-04-27T02:52:57.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:57 smithi167 ceph-mon[32751]: pgmap v16490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:57.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:57 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29858.289767 secs 2024-04-27T02:52:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:57 smithi090 ceph-mon[29180]: pgmap v16490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:57.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:57 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29858.289767 secs 2024-04-27T02:52:58.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:52:58 smithi119 ceph-mon[32752]: pgmap v16491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:58.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:52:58 smithi167 ceph-mon[32751]: pgmap v16491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:52:58.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:52:58 smithi090 ceph-mon[29180]: pgmap v16491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:01.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:00 smithi119 ceph-mon[32752]: pgmap v16492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:01.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:00 smithi167 ceph-mon[32751]: pgmap v16492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:01.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:00 smithi090 ceph-mon[29180]: pgmap v16492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:01 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29862.446306 secs 2024-04-27T02:53:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:01 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29862.446306 secs 2024-04-27T02:53:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:01 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29862.446306 secs 2024-04-27T02:53:03.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:02 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29863.289891 secs 2024-04-27T02:53:03.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:02 smithi119 ceph-mon[32752]: pgmap v16493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:03.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:02 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29863.289891 secs 2024-04-27T02:53:03.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:02 smithi167 ceph-mon[32751]: pgmap v16493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:03.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:02 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29863.289891 secs 2024-04-27T02:53:03.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:02 smithi090 ceph-mon[29180]: pgmap v16493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:05.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:04 smithi119 ceph-mon[32752]: pgmap v16494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:05.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:04 smithi167 ceph-mon[32751]: pgmap v16494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:04 smithi090 ceph-mon[29180]: pgmap v16494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:06 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29867.446394 secs 2024-04-27T02:53:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:06 smithi119 ceph-mon[32752]: pgmap v16495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:07.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:06 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29867.446394 secs 2024-04-27T02:53:07.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:06 smithi167 ceph-mon[32751]: pgmap v16495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:07.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:06 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29867.446394 secs 2024-04-27T02:53:07.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:06 smithi090 ceph-mon[29180]: pgmap v16495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:08.024 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:07 smithi090 ceph-mon[29180]: 12 slow requests, 0 included below; oldest blocked for > 29868.290018 secs 2024-04-27T02:53:08.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:07 smithi119 ceph-mon[32752]: 12 slow requests, 0 included below; oldest blocked for > 29868.290018 secs 2024-04-27T02:53:08.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:07 smithi167 ceph-mon[32751]: 12 slow requests, 0 included below; oldest blocked for > 29868.290018 secs 2024-04-27T02:53:09.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:08 smithi119 ceph-mon[32752]: pgmap v16496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:09.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:08 smithi167 ceph-mon[32751]: pgmap v16496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:09.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:08 smithi090 ceph-mon[29180]: pgmap v16496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:11.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:10 smithi119 ceph-mon[32752]: pgmap v16497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:11.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:10 smithi167 ceph-mon[32751]: pgmap v16497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:11.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:10 smithi090 ceph-mon[29180]: pgmap v16497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:12.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:11 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29872.446477 secs 2024-04-27T02:53:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:11 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29872.446477 secs 2024-04-27T02:53:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:11 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29872.446477 secs 2024-04-27T02:53:13.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:12 smithi119 ceph-mon[32752]: 30 slow requests, 2 included below; oldest blocked for > 29873.290144 secs 2024-04-27T02:53:13.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:12 smithi119 ceph-mon[32752]: slow request 182.033574 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:53:13.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:12 smithi119 ceph-mon[32752]: slow request 23108.451332 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:53:13.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:12 smithi119 ceph-mon[32752]: pgmap v16498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:53:13.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:53:13.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:12 smithi167 ceph-mon[32751]: 30 slow requests, 2 included below; oldest blocked for > 29873.290144 secs 2024-04-27T02:53:13.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:12 smithi167 ceph-mon[32751]: slow request 182.033574 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:53:13.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:12 smithi167 ceph-mon[32751]: slow request 23108.451332 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:53:13.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:12 smithi167 ceph-mon[32751]: pgmap v16498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:53:13.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:53:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:12 smithi090 ceph-mon[29180]: 30 slow requests, 2 included below; oldest blocked for > 29873.290144 secs 2024-04-27T02:53:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:12 smithi090 ceph-mon[29180]: slow request 182.033574 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:53:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:12 smithi090 ceph-mon[29180]: slow request 23108.451332 seconds old, received at 2024-04-26T20:28:02.589457+0000: client_request(client.26486:1 getattr pAsLsXsFs #0x1 2024-04-26T20:28:02.582550+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:53:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:12 smithi090 ceph-mon[29180]: pgmap v16498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:13.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T02:53:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:53:15.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:14 smithi119 ceph-mon[32752]: pgmap v16499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:15.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:14 smithi167 ceph-mon[32751]: pgmap v16499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:15.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:14 smithi090 ceph-mon[29180]: pgmap v16499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:16.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:16 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29877.446559 secs 2024-04-27T02:53:16.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:16 smithi119 ceph-mon[32752]: pgmap v16500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:17.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:16 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29877.446559 secs 2024-04-27T02:53:17.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:16 smithi167 ceph-mon[32751]: pgmap v16500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:16 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29877.446559 secs 2024-04-27T02:53:17.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:16 smithi090 ceph-mon[29180]: pgmap v16500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:18.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29878.290270 secs 2024-04-27T02:53:18.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29878.290270 secs 2024-04-27T02:53:18.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29878.290270 secs 2024-04-27T02:53:19.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:18 smithi119 ceph-mon[32752]: pgmap v16501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:19.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:18 smithi167 ceph-mon[32751]: pgmap v16501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:19.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:18 smithi090 ceph-mon[29180]: pgmap v16501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:21.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:20 smithi119 ceph-mon[32752]: pgmap v16502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:21.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:20 smithi167 ceph-mon[32751]: pgmap v16502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:20 smithi090 ceph-mon[29180]: pgmap v16502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:22.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:21 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29882.446642 secs 2024-04-27T02:53:22.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:21 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29882.446642 secs 2024-04-27T02:53:22.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:21 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29882.446642 secs 2024-04-27T02:53:23.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29883.290426 secs 2024-04-27T02:53:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:22 smithi167 ceph-mon[32751]: pgmap v16503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:23.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29883.290426 secs 2024-04-27T02:53:23.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:22 smithi090 ceph-mon[29180]: pgmap v16503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:23.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29883.290426 secs 2024-04-27T02:53:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:22 smithi119 ceph-mon[32752]: pgmap v16503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:25.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:24 smithi167 ceph-mon[32751]: pgmap v16504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:25.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:24 smithi090 ceph-mon[29180]: pgmap v16504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:25.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:24 smithi119 ceph-mon[32752]: pgmap v16504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:27.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:26 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29887.446726 secs 2024-04-27T02:53:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:26 smithi167 ceph-mon[32751]: pgmap v16505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29888.290575 secs 2024-04-27T02:53:27.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:26 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29887.446726 secs 2024-04-27T02:53:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:26 smithi090 ceph-mon[29180]: pgmap v16505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29888.290575 secs 2024-04-27T02:53:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:26 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29887.446726 secs 2024-04-27T02:53:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:26 smithi119 ceph-mon[32752]: pgmap v16505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29888.290575 secs 2024-04-27T02:53:29.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:28 smithi167 ceph-mon[32751]: pgmap v16506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:29.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:28 smithi090 ceph-mon[29180]: pgmap v16506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:29.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:28 smithi119 ceph-mon[32752]: pgmap v16506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:30 smithi167 ceph-mon[32751]: pgmap v16507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:31.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:30 smithi090 ceph-mon[29180]: pgmap v16507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:31.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:30 smithi119 ceph-mon[32752]: pgmap v16507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:31 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29892.446807 secs 2024-04-27T02:53:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29893.290663 secs 2024-04-27T02:53:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:31 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29892.446807 secs 2024-04-27T02:53:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29893.290663 secs 2024-04-27T02:53:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:31 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29892.446807 secs 2024-04-27T02:53:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29893.290663 secs 2024-04-27T02:53:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:32 smithi090 ceph-mon[29180]: pgmap v16508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:33.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:32 smithi119 ceph-mon[32752]: pgmap v16508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:33.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:32 smithi167 ceph-mon[32751]: pgmap v16508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:35.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:34 smithi090 ceph-mon[29180]: pgmap v16509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:35.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:34 smithi119 ceph-mon[32752]: pgmap v16509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:35.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:34 smithi167 ceph-mon[32751]: pgmap v16509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:37.104 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:36 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29897.446893 secs 2024-04-27T02:53:37.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:36 smithi119 ceph-mon[32752]: pgmap v16510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:37.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29898.290792 secs 2024-04-27T02:53:37.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:36 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29897.446893 secs 2024-04-27T02:53:37.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:36 smithi167 ceph-mon[32751]: pgmap v16510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:37.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29898.290792 secs 2024-04-27T02:53:37.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:36 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29897.446893 secs 2024-04-27T02:53:37.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:36 smithi090 ceph-mon[29180]: pgmap v16510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:37.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29898.290792 secs 2024-04-27T02:53:39.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:38 smithi090 ceph-mon[29180]: pgmap v16511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:39.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:38 smithi119 ceph-mon[32752]: pgmap v16511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:39.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:38 smithi167 ceph-mon[32751]: pgmap v16511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:41.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:40 smithi090 ceph-mon[29180]: pgmap v16512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:41.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:40 smithi119 ceph-mon[32752]: pgmap v16512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:41.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:40 smithi167 ceph-mon[32751]: pgmap v16512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:42.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:41 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29902.446976 secs 2024-04-27T02:53:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:41 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 29903.290915 secs 2024-04-27T02:53:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:41 smithi090 ceph-mon[29180]: slow request 1442.382038 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:53:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:53:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:41 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29902.446976 secs 2024-04-27T02:53:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:41 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 29903.290915 secs 2024-04-27T02:53:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:41 smithi119 ceph-mon[32752]: slow request 1442.382038 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:53:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:53:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:41 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29902.446976 secs 2024-04-27T02:53:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:41 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 29903.290915 secs 2024-04-27T02:53:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:41 smithi167 ceph-mon[32751]: slow request 1442.382038 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:53:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:53:43.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:43 smithi090 ceph-mon[29180]: pgmap v16513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.411 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:43 smithi119 ceph-mon[32752]: pgmap v16513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:43.411 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.412 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.412 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.412 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:43 smithi167 ceph-mon[32751]: pgmap v16513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:43.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:45 smithi090 ceph-mon[29180]: pgmap v16514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:45 smithi119 ceph-mon[32752]: pgmap v16514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:45 smithi167 ceph-mon[32751]: pgmap v16514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:53:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:53:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:53:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:53:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:53:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:53:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:53:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:53:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29907.447049 secs 2024-04-27T02:53:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:53:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:53:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: pgmap v16515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:53:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:53:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: pgmap v16516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-27T02:53:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: pgmap v16517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29908.291045 secs 2024-04-27T02:53:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29907.447049 secs 2024-04-27T02:53:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:53:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:53:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: pgmap v16515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:53:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:53:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: pgmap v16516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-27T02:53:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: pgmap v16517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29908.291045 secs 2024-04-27T02:53:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29907.447049 secs 2024-04-27T02:53:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T02:53:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T02:53:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: pgmap v16515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T02:53:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T02:53:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: pgmap v16516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-27T02:53:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: pgmap v16517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29908.291045 secs 2024-04-27T02:53:49.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:49 smithi090 ceph-mon[29180]: pgmap v16518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T02:53:49.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:49 smithi119 ceph-mon[32752]: pgmap v16518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T02:53:49.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:49 smithi167 ceph-mon[32751]: pgmap v16518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T02:53:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:51 smithi119 ceph-mon[32752]: pgmap v16519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:51.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:51 smithi167 ceph-mon[32751]: pgmap v16519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:51 smithi090 ceph-mon[29180]: pgmap v16519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:52 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29912.447108 secs 2024-04-27T02:53:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29913.291169 secs 2024-04-27T02:53:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:52 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29912.447108 secs 2024-04-27T02:53:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29913.291169 secs 2024-04-27T02:53:52.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:52 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29912.447108 secs 2024-04-27T02:53:52.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29913.291169 secs 2024-04-27T02:53:53.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:53 smithi119 ceph-mon[32752]: pgmap v16520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T02:53:53.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:53 smithi167 ceph-mon[32751]: pgmap v16520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T02:53:53.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:53 smithi090 ceph-mon[29180]: pgmap v16520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T02:53:55.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:55 smithi119 ceph-mon[32752]: pgmap v16521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:55.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:55 smithi167 ceph-mon[32751]: pgmap v16521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:55 smithi090 ceph-mon[29180]: pgmap v16521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:53:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:57 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29917.447190 secs 2024-04-27T02:53:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:57 smithi119 ceph-mon[32752]: pgmap v16522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:53:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29918.291299 secs 2024-04-27T02:53:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:57 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29917.447190 secs 2024-04-27T02:53:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:57 smithi167 ceph-mon[32751]: pgmap v16522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:53:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29918.291299 secs 2024-04-27T02:53:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:57 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29917.447190 secs 2024-04-27T02:53:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:57 smithi090 ceph-mon[29180]: pgmap v16522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:53:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29918.291299 secs 2024-04-27T02:53:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:53:59 smithi119 ceph-mon[32752]: pgmap v16523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:59.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:53:59 smithi167 ceph-mon[32751]: pgmap v16523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:53:59.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:53:59 smithi090 ceph-mon[29180]: pgmap v16523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:01 smithi119 ceph-mon[32752]: pgmap v16524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:01 smithi167 ceph-mon[32751]: pgmap v16524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:01 smithi090 ceph-mon[29180]: pgmap v16524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:02 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29922.447231 secs 2024-04-27T02:54:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29923.291438 secs 2024-04-27T02:54:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:02 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29922.447231 secs 2024-04-27T02:54:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29923.291438 secs 2024-04-27T02:54:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:02 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29922.447231 secs 2024-04-27T02:54:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29923.291438 secs 2024-04-27T02:54:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:03 smithi119 ceph-mon[32752]: pgmap v16525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:03.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:03 smithi167 ceph-mon[32751]: pgmap v16525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:03 smithi090 ceph-mon[29180]: pgmap v16525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:05 smithi119 ceph-mon[32752]: pgmap v16526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:05 smithi167 ceph-mon[32751]: pgmap v16526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:05 smithi090 ceph-mon[29180]: pgmap v16526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:07 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29927.447315 secs 2024-04-27T02:54:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:07 smithi119 ceph-mon[32752]: pgmap v16527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29928.291589 secs 2024-04-27T02:54:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:07 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29927.447315 secs 2024-04-27T02:54:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:07 smithi167 ceph-mon[32751]: pgmap v16527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29928.291589 secs 2024-04-27T02:54:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:07 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29927.447315 secs 2024-04-27T02:54:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:07 smithi090 ceph-mon[29180]: pgmap v16527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29928.291589 secs 2024-04-27T02:54:09.427 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:09 smithi090 ceph-mon[29180]: pgmap v16528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:09 smithi119 ceph-mon[32752]: pgmap v16528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:09 smithi167 ceph-mon[32751]: pgmap v16528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:11 smithi119 ceph-mon[32752]: pgmap v16529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:11 smithi167 ceph-mon[32751]: pgmap v16529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:11 smithi090 ceph-mon[29180]: pgmap v16529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:12 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29932.447397 secs 2024-04-27T02:54:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29933.291715 secs 2024-04-27T02:54:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:12 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29932.447397 secs 2024-04-27T02:54:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29933.291715 secs 2024-04-27T02:54:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:12 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29932.447397 secs 2024-04-27T02:54:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29933.291715 secs 2024-04-27T02:54:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:13 smithi119 ceph-mon[32752]: pgmap v16530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:13 smithi167 ceph-mon[32751]: pgmap v16530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:13 smithi090 ceph-mon[29180]: pgmap v16530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:15 smithi119 ceph-mon[32752]: pgmap v16531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:15 smithi167 ceph-mon[32751]: pgmap v16531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:15.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:15 smithi090 ceph-mon[29180]: pgmap v16531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:16 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29937.447440 secs 2024-04-27T02:54:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:16 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29937.447440 secs 2024-04-27T02:54:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:16 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29937.447440 secs 2024-04-27T02:54:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:17 smithi119 ceph-mon[32752]: pgmap v16532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29938.291859 secs 2024-04-27T02:54:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:17 smithi167 ceph-mon[32751]: pgmap v16532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29938.291859 secs 2024-04-27T02:54:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:17 smithi090 ceph-mon[29180]: pgmap v16532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29938.291859 secs 2024-04-27T02:54:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:19 smithi119 ceph-mon[32752]: pgmap v16533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:19 smithi167 ceph-mon[32751]: pgmap v16533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:19.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:19 smithi090 ceph-mon[29180]: pgmap v16533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:21 smithi119 ceph-mon[32752]: pgmap v16534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:21 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29942.447524 secs 2024-04-27T02:54:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:21 smithi167 ceph-mon[32751]: pgmap v16534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:21 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29942.447524 secs 2024-04-27T02:54:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:21 smithi090 ceph-mon[29180]: pgmap v16534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:21 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29942.447524 secs 2024-04-27T02:54:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29943.291985 secs 2024-04-27T02:54:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29943.291985 secs 2024-04-27T02:54:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29943.291985 secs 2024-04-27T02:54:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:23 smithi119 ceph-mon[32752]: pgmap v16535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:23 smithi167 ceph-mon[32751]: pgmap v16535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:23.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:23 smithi090 ceph-mon[29180]: pgmap v16535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:25 smithi119 ceph-mon[32752]: pgmap v16536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:25 smithi167 ceph-mon[32751]: pgmap v16536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:25 smithi090 ceph-mon[29180]: pgmap v16536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:26 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29947.447599 secs 2024-04-27T02:54:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:26 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29947.447599 secs 2024-04-27T02:54:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:26 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29947.447599 secs 2024-04-27T02:54:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:27 smithi119 ceph-mon[32752]: pgmap v16537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29948.292059 secs 2024-04-27T02:54:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:27 smithi167 ceph-mon[32751]: pgmap v16537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29948.292059 secs 2024-04-27T02:54:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:27 smithi090 ceph-mon[29180]: pgmap v16537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29948.292059 secs 2024-04-27T02:54:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:29 smithi119 ceph-mon[32752]: pgmap v16538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:29 smithi167 ceph-mon[32751]: pgmap v16538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:29 smithi090 ceph-mon[29180]: pgmap v16538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:31 smithi119 ceph-mon[32752]: pgmap v16539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:31 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29952.447684 secs 2024-04-27T02:54:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:31 smithi167 ceph-mon[32751]: pgmap v16539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:31 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29952.447684 secs 2024-04-27T02:54:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:31 smithi090 ceph-mon[29180]: pgmap v16539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:31 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29952.447684 secs 2024-04-27T02:54:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29953.292138 secs 2024-04-27T02:54:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29953.292138 secs 2024-04-27T02:54:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29953.292138 secs 2024-04-27T02:54:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:33 smithi119 ceph-mon[32752]: pgmap v16540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:33 smithi167 ceph-mon[32751]: pgmap v16540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:33 smithi090 ceph-mon[29180]: pgmap v16540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:35 smithi119 ceph-mon[32752]: pgmap v16541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:35 smithi167 ceph-mon[32751]: pgmap v16541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:35 smithi090 ceph-mon[29180]: pgmap v16541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:36 smithi119 ceph-mon[32752]: 39 slow requests, 1 included below; oldest blocked for > 29957.447766 secs 2024-04-27T02:54:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:36 smithi119 ceph-mon[32752]: slow request 361.688107 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:54:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:36 smithi167 ceph-mon[32751]: 39 slow requests, 1 included below; oldest blocked for > 29957.447766 secs 2024-04-27T02:54:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:36 smithi167 ceph-mon[32751]: slow request 361.688107 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:54:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:36 smithi090 ceph-mon[29180]: 39 slow requests, 1 included below; oldest blocked for > 29957.447766 secs 2024-04-27T02:54:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:36 smithi090 ceph-mon[29180]: slow request 361.688107 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:54:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:37 smithi119 ceph-mon[32752]: pgmap v16542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29958.292212 secs 2024-04-27T02:54:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:37 smithi167 ceph-mon[32751]: pgmap v16542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29958.292212 secs 2024-04-27T02:54:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:37 smithi090 ceph-mon[29180]: pgmap v16542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29958.292212 secs 2024-04-27T02:54:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:39 smithi119 ceph-mon[32752]: pgmap v16543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:39 smithi167 ceph-mon[32751]: pgmap v16543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:39 smithi090 ceph-mon[29180]: pgmap v16543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:41 smithi119 ceph-mon[32752]: pgmap v16544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:41 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29962.447846 secs 2024-04-27T02:54:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:41 smithi167 ceph-mon[32751]: pgmap v16544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:41 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29962.447846 secs 2024-04-27T02:54:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:41 smithi090 ceph-mon[29180]: pgmap v16544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:41 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29962.447846 secs 2024-04-27T02:54:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29963.292342 secs 2024-04-27T02:54:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29963.292342 secs 2024-04-27T02:54:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29963.292342 secs 2024-04-27T02:54:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:43 smithi119 ceph-mon[32752]: pgmap v16545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:43 smithi167 ceph-mon[32751]: pgmap v16545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:43 smithi090 ceph-mon[29180]: pgmap v16545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:45 smithi119 ceph-mon[32752]: pgmap v16546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:45 smithi167 ceph-mon[32751]: pgmap v16546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:45 smithi090 ceph-mon[29180]: pgmap v16546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:54:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:46 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29967.447933 secs 2024-04-27T02:54:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:54:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:46 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29967.447933 secs 2024-04-27T02:54:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:54:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:46 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29967.447933 secs 2024-04-27T02:54:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:54:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:47 smithi119 ceph-mon[32752]: pgmap v16547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29968.292430 secs 2024-04-27T02:54:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:54:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:54:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:47 smithi167 ceph-mon[32751]: pgmap v16547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29968.292430 secs 2024-04-27T02:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:54:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:47 smithi090 ceph-mon[29180]: pgmap v16547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:54:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29968.292430 secs 2024-04-27T02:54:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:54:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:54:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:54:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:48 smithi119 ceph-mon[32752]: pgmap v16548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T02:54:48.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:48 smithi119 ceph-mon[32752]: pgmap v16549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:48 smithi167 ceph-mon[32751]: pgmap v16548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T02:54:48.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:48 smithi167 ceph-mon[32751]: pgmap v16549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:48 smithi090 ceph-mon[29180]: pgmap v16548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T02:54:48.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:48 smithi090 ceph-mon[29180]: pgmap v16549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:50 smithi119 ceph-mon[32752]: pgmap v16550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:50 smithi167 ceph-mon[32751]: pgmap v16550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:50.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:50 smithi090 ceph-mon[29180]: pgmap v16550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:51 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29972.448014 secs 2024-04-27T02:54:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:51 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29972.448014 secs 2024-04-27T02:54:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:51 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29972.448014 secs 2024-04-27T02:54:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:52 smithi119 ceph-mon[32752]: pgmap v16551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:52.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29973.292554 secs 2024-04-27T02:54:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:52 smithi167 ceph-mon[32751]: pgmap v16551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:52.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29973.292554 secs 2024-04-27T02:54:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:52 smithi090 ceph-mon[29180]: pgmap v16551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:52.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29973.292554 secs 2024-04-27T02:54:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:54 smithi119 ceph-mon[32752]: pgmap v16552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T02:54:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:54 smithi167 ceph-mon[32751]: pgmap v16552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T02:54:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:54 smithi090 ceph-mon[29180]: pgmap v16552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T02:54:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:56 smithi119 ceph-mon[32752]: pgmap v16553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:56 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29977.448089 secs 2024-04-27T02:54:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:56 smithi167 ceph-mon[32751]: pgmap v16553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:56 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29977.448089 secs 2024-04-27T02:54:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:56 smithi090 ceph-mon[29180]: pgmap v16553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:54:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:56 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29977.448089 secs 2024-04-27T02:54:57.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29978.292620 secs 2024-04-27T02:54:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29978.292620 secs 2024-04-27T02:54:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29978.292620 secs 2024-04-27T02:54:58.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:54:58 smithi119 ceph-mon[32752]: pgmap v16554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:54:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:54:58 smithi167 ceph-mon[32751]: pgmap v16554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:54:58.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:54:58 smithi090 ceph-mon[29180]: pgmap v16554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:55:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:00 smithi119 ceph-mon[32752]: pgmap v16555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:00 smithi167 ceph-mon[32751]: pgmap v16555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:00 smithi090 ceph-mon[29180]: pgmap v16555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:01 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29982.448172 secs 2024-04-27T02:55:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:01 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29982.448172 secs 2024-04-27T02:55:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:01 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29982.448172 secs 2024-04-27T02:55:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:02 smithi119 ceph-mon[32752]: pgmap v16556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29983.292702 secs 2024-04-27T02:55:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:02 smithi167 ceph-mon[32751]: pgmap v16556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29983.292702 secs 2024-04-27T02:55:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:02 smithi090 ceph-mon[29180]: pgmap v16556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29983.292702 secs 2024-04-27T02:55:04.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:04 smithi119 ceph-mon[32752]: pgmap v16557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:04 smithi167 ceph-mon[32751]: pgmap v16557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:04.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:04 smithi090 ceph-mon[29180]: pgmap v16557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:06 smithi119 ceph-mon[32752]: pgmap v16558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:06 smithi119 ceph-mon[32752]: 39 slow requests, 0 included below; oldest blocked for > 29987.448256 secs 2024-04-27T02:55:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:06 smithi167 ceph-mon[32751]: pgmap v16558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:06 smithi167 ceph-mon[32751]: 39 slow requests, 0 included below; oldest blocked for > 29987.448256 secs 2024-04-27T02:55:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:06 smithi090 ceph-mon[29180]: pgmap v16558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:06 smithi090 ceph-mon[29180]: 39 slow requests, 0 included below; oldest blocked for > 29987.448256 secs 2024-04-27T02:55:07.705 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29988.292762 secs 2024-04-27T02:55:07.719 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29988.292762 secs 2024-04-27T02:55:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29988.292762 secs 2024-04-27T02:55:08.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:08 smithi119 ceph-mon[32752]: pgmap v16559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:08.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:08 smithi167 ceph-mon[32751]: pgmap v16559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:08.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:08 smithi090 ceph-mon[29180]: pgmap v16559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:09.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:55:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:55:09.055+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:55:09.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:55:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:55:09.055+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:55:09.376 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:55:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:55:09.055+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:55:09.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:55:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:55:09.055+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:55:09.377 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 02:55:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T02:55:09.055+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T02:55:10.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:10 smithi119 ceph-mon[32752]: pgmap v16560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:10.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:10 smithi167 ceph-mon[32751]: pgmap v16560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:10.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:10 smithi090 ceph-mon[29180]: pgmap v16560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:11 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 29992.448355 secs 2024-04-27T02:55:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:11 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 29992.448355 secs 2024-04-27T02:55:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:11 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 29992.448355 secs 2024-04-27T02:55:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:12 smithi119 ceph-mon[32752]: pgmap v16561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29993.292869 secs 2024-04-27T02:55:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:12 smithi167 ceph-mon[32751]: pgmap v16561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29993.292869 secs 2024-04-27T02:55:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:12 smithi090 ceph-mon[29180]: pgmap v16561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29993.292869 secs 2024-04-27T02:55:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:14 smithi119 ceph-mon[32752]: pgmap v16562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:14.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:14 smithi167 ceph-mon[32751]: pgmap v16562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:14.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:14 smithi090 ceph-mon[29180]: pgmap v16562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:55:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:55:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:55:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:55:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:55:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:55:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:16 smithi119 ceph-mon[32752]: pgmap v16563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:16 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 29997.448439 secs 2024-04-27T02:55:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:16 smithi167 ceph-mon[32751]: pgmap v16563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:16 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 29997.448439 secs 2024-04-27T02:55:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:16 smithi090 ceph-mon[29180]: pgmap v16563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:16 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 29997.448439 secs 2024-04-27T02:55:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 29998.292985 secs 2024-04-27T02:55:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 29998.292985 secs 2024-04-27T02:55:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 29998.292985 secs 2024-04-27T02:55:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:18 smithi119 ceph-mon[32752]: pgmap v16564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:18 smithi167 ceph-mon[32751]: pgmap v16564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:18.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:18 smithi090 ceph-mon[29180]: pgmap v16564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:20 smithi119 ceph-mon[32752]: pgmap v16565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:20 smithi167 ceph-mon[32751]: pgmap v16565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:20 smithi090 ceph-mon[29180]: pgmap v16565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:21 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30002.448499 secs 2024-04-27T02:55:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:21 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30002.448499 secs 2024-04-27T02:55:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:21 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30002.448499 secs 2024-04-27T02:55:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:22 smithi119 ceph-mon[32752]: pgmap v16566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30003.293095 secs 2024-04-27T02:55:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:22 smithi167 ceph-mon[32751]: pgmap v16566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30003.293095 secs 2024-04-27T02:55:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:22 smithi090 ceph-mon[29180]: pgmap v16566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:22.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30003.293095 secs 2024-04-27T02:55:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:24 smithi119 ceph-mon[32752]: pgmap v16567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:24 smithi167 ceph-mon[32751]: pgmap v16567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:24.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:24 smithi090 ceph-mon[29180]: pgmap v16567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:26 smithi119 ceph-mon[32752]: pgmap v16568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:26 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30007.448599 secs 2024-04-27T02:55:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:26 smithi167 ceph-mon[32751]: pgmap v16568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:26 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30007.448599 secs 2024-04-27T02:55:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:26 smithi090 ceph-mon[29180]: pgmap v16568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T02:55:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:26 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30007.448599 secs 2024-04-27T02:55:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30008.293194 secs 2024-04-27T02:55:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30008.293194 secs 2024-04-27T02:55:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30008.293194 secs 2024-04-27T02:55:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:28 smithi119 ceph-mon[32752]: pgmap v16569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:28 smithi167 ceph-mon[32751]: pgmap v16569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:28.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:28 smithi090 ceph-mon[29180]: pgmap v16569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:30.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:30 smithi119 ceph-mon[32752]: pgmap v16570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:30.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:30 smithi167 ceph-mon[32751]: pgmap v16570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:30.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:30 smithi090 ceph-mon[29180]: pgmap v16570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:31 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30012.448682 secs 2024-04-27T02:55:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:31 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30012.448682 secs 2024-04-27T02:55:31.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:31 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30012.448682 secs 2024-04-27T02:55:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:32 smithi119 ceph-mon[32752]: pgmap v16571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:32.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30013.293283 secs 2024-04-27T02:55:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:32 smithi167 ceph-mon[32751]: pgmap v16571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:32.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30013.293283 secs 2024-04-27T02:55:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:32 smithi090 ceph-mon[29180]: pgmap v16571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:32.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30013.293283 secs 2024-04-27T02:55:34.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:34 smithi119 ceph-mon[32752]: pgmap v16572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:34.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:34 smithi167 ceph-mon[32751]: pgmap v16572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:34.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:34 smithi090 ceph-mon[29180]: pgmap v16572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:36.678 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:36 smithi090 ceph-mon[29180]: pgmap v16573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:36.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:36 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30017.448762 secs 2024-04-27T02:55:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:36 smithi119 ceph-mon[32752]: pgmap v16573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:36 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30017.448762 secs 2024-04-27T02:55:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:36 smithi167 ceph-mon[32751]: pgmap v16573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:36 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30017.448762 secs 2024-04-27T02:55:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30018.293412 secs 2024-04-27T02:55:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30018.293412 secs 2024-04-27T02:55:37.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30018.293412 secs 2024-04-27T02:55:38.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:38 smithi119 ceph-mon[32752]: pgmap v16574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:38.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:38 smithi167 ceph-mon[32751]: pgmap v16574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:38 smithi090 ceph-mon[29180]: pgmap v16574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:40.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:40 smithi119 ceph-mon[32752]: pgmap v16575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:40.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:40 smithi167 ceph-mon[32751]: pgmap v16575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:40.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:40 smithi090 ceph-mon[29180]: pgmap v16575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:41.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:41 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30022.448819 secs 2024-04-27T02:55:41.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:41 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30022.448819 secs 2024-04-27T02:55:41.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:41 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30022.448819 secs 2024-04-27T02:55:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30023.293530 secs 2024-04-27T02:55:42.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:42 smithi119 ceph-mon[32752]: pgmap v16576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:42.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30023.293530 secs 2024-04-27T02:55:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:42 smithi167 ceph-mon[32751]: pgmap v16576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:42.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30023.293530 secs 2024-04-27T02:55:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:42 smithi090 ceph-mon[29180]: pgmap v16576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:43 smithi119 ceph-mon[32752]: pgmap v16577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:43 smithi167 ceph-mon[32751]: pgmap v16577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:43 smithi090 ceph-mon[29180]: pgmap v16577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:46 smithi090 ceph-mon[29180]: pgmap v16578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:46 smithi119 ceph-mon[32752]: pgmap v16578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:46 smithi167 ceph-mon[32751]: pgmap v16578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:55:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:47 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30027.448892 secs 2024-04-27T02:55:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30028.293609 secs 2024-04-27T02:55:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:47 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30027.448892 secs 2024-04-27T02:55:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30028.293609 secs 2024-04-27T02:55:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:47 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30027.448892 secs 2024-04-27T02:55:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30028.293609 secs 2024-04-27T02:55:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:48 smithi090 ceph-mon[29180]: pgmap v16579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:55:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:55:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:55:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:55:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:48 smithi119 ceph-mon[32752]: pgmap v16579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:55:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:55:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:55:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:55:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:48 smithi167 ceph-mon[32751]: pgmap v16579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:55:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:55:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:55:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:55:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:55:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:49 smithi090 ceph-mon[29180]: pgmap v16580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:55:49.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:49 smithi090 ceph-mon[29180]: pgmap v16581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T02:55:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:49 smithi119 ceph-mon[32752]: pgmap v16580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:55:49.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:49 smithi119 ceph-mon[32752]: pgmap v16581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T02:55:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:49 smithi167 ceph-mon[32751]: pgmap v16580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T02:55:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:49 smithi167 ceph-mon[32751]: pgmap v16581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T02:55:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:51 smithi090 ceph-mon[29180]: pgmap v16582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:51 smithi119 ceph-mon[32752]: pgmap v16582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:51 smithi167 ceph-mon[32751]: pgmap v16582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:52 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30032.448956 secs 2024-04-27T02:55:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30033.293728 secs 2024-04-27T02:55:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:52 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30032.448956 secs 2024-04-27T02:55:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30033.293728 secs 2024-04-27T02:55:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:52 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30032.448956 secs 2024-04-27T02:55:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30033.293728 secs 2024-04-27T02:55:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:53 smithi119 ceph-mon[32752]: pgmap v16583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:53 smithi167 ceph-mon[32751]: pgmap v16583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:53 smithi090 ceph-mon[29180]: pgmap v16583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:55 smithi119 ceph-mon[32752]: pgmap v16584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:55 smithi167 ceph-mon[32751]: pgmap v16584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:55 smithi090 ceph-mon[29180]: pgmap v16584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:57 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30037.449045 secs 2024-04-27T02:55:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:57 smithi119 ceph-mon[32752]: pgmap v16585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30038.293801 secs 2024-04-27T02:55:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:57 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30037.449045 secs 2024-04-27T02:55:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:57 smithi167 ceph-mon[32751]: pgmap v16585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30038.293801 secs 2024-04-27T02:55:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:57 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30037.449045 secs 2024-04-27T02:55:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:57 smithi090 ceph-mon[29180]: pgmap v16585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:55:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30038.293801 secs 2024-04-27T02:55:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:55:59 smithi119 ceph-mon[32752]: pgmap v16586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:55:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:55:59 smithi167 ceph-mon[32751]: pgmap v16586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:55:59.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:55:59 smithi090 ceph-mon[29180]: pgmap v16586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:56:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:01 smithi119 ceph-mon[32752]: pgmap v16587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:01 smithi167 ceph-mon[32751]: pgmap v16587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:01 smithi090 ceph-mon[29180]: pgmap v16587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:02 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30042.449127 secs 2024-04-27T02:56:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30043.293874 secs 2024-04-27T02:56:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:02 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30042.449127 secs 2024-04-27T02:56:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30043.293874 secs 2024-04-27T02:56:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:02 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30042.449127 secs 2024-04-27T02:56:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30043.293874 secs 2024-04-27T02:56:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:03 smithi119 ceph-mon[32752]: pgmap v16588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:03 smithi167 ceph-mon[32751]: pgmap v16588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:03.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:03 smithi090 ceph-mon[29180]: pgmap v16588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:05 smithi119 ceph-mon[32752]: pgmap v16589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:05 smithi167 ceph-mon[32751]: pgmap v16589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:05 smithi090 ceph-mon[29180]: pgmap v16589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:07 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30047.449189 secs 2024-04-27T02:56:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:07 smithi119 ceph-mon[32752]: pgmap v16590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30048.294002 secs 2024-04-27T02:56:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:07 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30047.449189 secs 2024-04-27T02:56:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:07 smithi167 ceph-mon[32751]: pgmap v16590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30048.294002 secs 2024-04-27T02:56:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:07 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30047.449189 secs 2024-04-27T02:56:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:07 smithi090 ceph-mon[29180]: pgmap v16590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30048.294002 secs 2024-04-27T02:56:09.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:09 smithi119 ceph-mon[32752]: pgmap v16591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:09 smithi167 ceph-mon[32751]: pgmap v16591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:09 smithi090 ceph-mon[29180]: pgmap v16591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:11.428 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:11 smithi090 ceph-mon[29180]: pgmap v16592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:11 smithi119 ceph-mon[32752]: pgmap v16592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:11 smithi167 ceph-mon[32751]: pgmap v16592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:12 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30052.449274 secs 2024-04-27T02:56:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:12 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 30053.294104 secs 2024-04-27T02:56:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:12 smithi119 ceph-mon[32752]: slow request 362.037535 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:56:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:12 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30052.449274 secs 2024-04-27T02:56:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:12 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 30053.294104 secs 2024-04-27T02:56:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:12 smithi167 ceph-mon[32751]: slow request 362.037535 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:56:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:12 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30052.449274 secs 2024-04-27T02:56:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:12 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 30053.294104 secs 2024-04-27T02:56:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:12 smithi090 ceph-mon[29180]: slow request 362.037535 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:56:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:13 smithi119 ceph-mon[32752]: pgmap v16593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:13 smithi167 ceph-mon[32751]: pgmap v16593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:13 smithi090 ceph-mon[29180]: pgmap v16593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:15 smithi119 ceph-mon[32752]: pgmap v16594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:15 smithi167 ceph-mon[32751]: pgmap v16594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:15 smithi090 ceph-mon[29180]: pgmap v16594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:17 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30057.449360 secs 2024-04-27T02:56:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:17 smithi119 ceph-mon[32752]: pgmap v16595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30058.294176 secs 2024-04-27T02:56:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:17 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30057.449360 secs 2024-04-27T02:56:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:17 smithi167 ceph-mon[32751]: pgmap v16595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30058.294176 secs 2024-04-27T02:56:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:17 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30057.449360 secs 2024-04-27T02:56:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:17 smithi090 ceph-mon[29180]: pgmap v16595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30058.294176 secs 2024-04-27T02:56:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:19 smithi119 ceph-mon[32752]: pgmap v16596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:19.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:19 smithi167 ceph-mon[32751]: pgmap v16596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:19.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:19 smithi090 ceph-mon[29180]: pgmap v16596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:21 smithi119 ceph-mon[32752]: pgmap v16597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:21 smithi167 ceph-mon[32751]: pgmap v16597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:21 smithi090 ceph-mon[29180]: pgmap v16597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:22 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30062.449445 secs 2024-04-27T02:56:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30063.294287 secs 2024-04-27T02:56:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:22 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30062.449445 secs 2024-04-27T02:56:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30063.294287 secs 2024-04-27T02:56:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:22 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30062.449445 secs 2024-04-27T02:56:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30063.294287 secs 2024-04-27T02:56:23.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:23 smithi090 ceph-mon[29180]: pgmap v16598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:23 smithi119 ceph-mon[32752]: pgmap v16598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:23 smithi167 ceph-mon[32751]: pgmap v16598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:25.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:25 smithi119 ceph-mon[32752]: pgmap v16599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:25 smithi167 ceph-mon[32751]: pgmap v16599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:25 smithi090 ceph-mon[29180]: pgmap v16599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:27 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30067.449531 secs 2024-04-27T02:56:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:27 smithi119 ceph-mon[32752]: pgmap v16600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30068.294412 secs 2024-04-27T02:56:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:27 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30067.449531 secs 2024-04-27T02:56:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:27 smithi167 ceph-mon[32751]: pgmap v16600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30068.294412 secs 2024-04-27T02:56:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:27 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30067.449531 secs 2024-04-27T02:56:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:27 smithi090 ceph-mon[29180]: pgmap v16600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30068.294412 secs 2024-04-27T02:56:29.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:29 smithi119 ceph-mon[32752]: pgmap v16601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:29 smithi167 ceph-mon[32751]: pgmap v16601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:29.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:29 smithi090 ceph-mon[29180]: pgmap v16601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:31 smithi119 ceph-mon[32752]: pgmap v16602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:31 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30072.449619 secs 2024-04-27T02:56:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:31 smithi167 ceph-mon[32751]: pgmap v16602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:31 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30072.449619 secs 2024-04-27T02:56:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:31 smithi090 ceph-mon[29180]: pgmap v16602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:31 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30072.449619 secs 2024-04-27T02:56:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30073.294586 secs 2024-04-27T02:56:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30073.294586 secs 2024-04-27T02:56:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30073.294586 secs 2024-04-27T02:56:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:33 smithi119 ceph-mon[32752]: pgmap v16603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:33.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:33 smithi167 ceph-mon[32751]: pgmap v16603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:33 smithi090 ceph-mon[29180]: pgmap v16603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:35 smithi119 ceph-mon[32752]: pgmap v16604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:35.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:35 smithi167 ceph-mon[32751]: pgmap v16604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:35 smithi090 ceph-mon[29180]: pgmap v16604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:36 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30077.449703 secs 2024-04-27T02:56:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:36 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30077.449703 secs 2024-04-27T02:56:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:36 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30077.449703 secs 2024-04-27T02:56:37.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:37 smithi167 ceph-mon[32751]: pgmap v16605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:37.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30078.294705 secs 2024-04-27T02:56:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:37 smithi119 ceph-mon[32752]: pgmap v16605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30078.294705 secs 2024-04-27T02:56:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:37 smithi090 ceph-mon[29180]: pgmap v16605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30078.294705 secs 2024-04-27T02:56:39.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:39 smithi119 ceph-mon[32752]: pgmap v16606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:39 smithi167 ceph-mon[32751]: pgmap v16606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:39.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:39 smithi090 ceph-mon[29180]: pgmap v16606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:41 smithi119 ceph-mon[32752]: pgmap v16607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:41.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:41 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30082.449815 secs 2024-04-27T02:56:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:41 smithi167 ceph-mon[32751]: pgmap v16607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:41 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30082.449815 secs 2024-04-27T02:56:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:41 smithi090 ceph-mon[29180]: pgmap v16607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:41 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30082.449815 secs 2024-04-27T02:56:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30083.294818 secs 2024-04-27T02:56:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30083.294818 secs 2024-04-27T02:56:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30083.294818 secs 2024-04-27T02:56:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:43 smithi119 ceph-mon[32752]: pgmap v16608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:43.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:43 smithi167 ceph-mon[32751]: pgmap v16608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:43 smithi090 ceph-mon[29180]: pgmap v16608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:45 smithi119 ceph-mon[32752]: pgmap v16609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:45 smithi167 ceph-mon[32751]: pgmap v16609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:45 smithi090 ceph-mon[29180]: pgmap v16609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:56:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:46 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30087.449875 secs 2024-04-27T02:56:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:46 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30087.449875 secs 2024-04-27T02:56:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:46 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30087.449875 secs 2024-04-27T02:56:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:47 smithi119 ceph-mon[32752]: pgmap v16610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30088.294954 secs 2024-04-27T02:56:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:47 smithi167 ceph-mon[32751]: pgmap v16610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30088.294954 secs 2024-04-27T02:56:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:47 smithi090 ceph-mon[29180]: pgmap v16610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30088.294954 secs 2024-04-27T02:56:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:56:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:56:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:56:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:49 smithi119 ceph-mon[32752]: pgmap v16611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:56:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:56:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:56:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:49 smithi167 ceph-mon[32751]: pgmap v16611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:56:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:56:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:56:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:49 smithi090 ceph-mon[29180]: pgmap v16611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:56:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:56:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:56:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:56:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:50 smithi119 ceph-mon[32752]: pgmap v16612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:56:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:50 smithi119 ceph-mon[32752]: pgmap v16613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:50 smithi167 ceph-mon[32751]: pgmap v16612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:56:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:50 smithi167 ceph-mon[32751]: pgmap v16613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:50 smithi090 ceph-mon[29180]: pgmap v16612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T02:56:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:50 smithi090 ceph-mon[29180]: pgmap v16613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:51 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30092.449962 secs 2024-04-27T02:56:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:51 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30092.449962 secs 2024-04-27T02:56:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:51 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30092.449962 secs 2024-04-27T02:56:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:52 smithi119 ceph-mon[32752]: pgmap v16614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30093.295024 secs 2024-04-27T02:56:52.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:52 smithi167 ceph-mon[32751]: pgmap v16614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:52.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30093.295024 secs 2024-04-27T02:56:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:52 smithi090 ceph-mon[29180]: pgmap v16614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30093.295024 secs 2024-04-27T02:56:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:54 smithi119 ceph-mon[32752]: pgmap v16615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:56:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:54 smithi167 ceph-mon[32751]: pgmap v16615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:56:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:54 smithi090 ceph-mon[29180]: pgmap v16615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:56:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:56 smithi119 ceph-mon[32752]: pgmap v16616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:56 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30097.450022 secs 2024-04-27T02:56:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:56 smithi167 ceph-mon[32751]: pgmap v16616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:56 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30097.450022 secs 2024-04-27T02:56:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:56 smithi090 ceph-mon[29180]: pgmap v16616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:56 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30097.450022 secs 2024-04-27T02:56:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30098.295125 secs 2024-04-27T02:56:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30098.295125 secs 2024-04-27T02:56:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30098.295125 secs 2024-04-27T02:56:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:56:58 smithi119 ceph-mon[32752]: pgmap v16617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:56:58 smithi167 ceph-mon[32751]: pgmap v16617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:56:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:56:58 smithi090 ceph-mon[29180]: pgmap v16617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:00 smithi119 ceph-mon[32752]: pgmap v16618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T02:57:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:00 smithi167 ceph-mon[32751]: pgmap v16618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T02:57:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:00 smithi090 ceph-mon[29180]: pgmap v16618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T02:57:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:01 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30102.450082 secs 2024-04-27T02:57:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:01 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30102.450082 secs 2024-04-27T02:57:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:01 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30102.450082 secs 2024-04-27T02:57:02.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:02 smithi090 ceph-mon[29180]: pgmap v16619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:02.552 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30103.295264 secs 2024-04-27T02:57:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:02 smithi119 ceph-mon[32752]: pgmap v16619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30103.295264 secs 2024-04-27T02:57:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:02 smithi167 ceph-mon[32751]: pgmap v16619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30103.295264 secs 2024-04-27T02:57:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:03 smithi167 ceph-mon[32751]: pgmap v16620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:03 smithi090 ceph-mon[29180]: pgmap v16620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:03 smithi119 ceph-mon[32752]: pgmap v16620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:05 smithi119 ceph-mon[32752]: pgmap v16621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:05 smithi167 ceph-mon[32751]: pgmap v16621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:05 smithi090 ceph-mon[29180]: pgmap v16621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:06 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30107.450164 secs 2024-04-27T02:57:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:06 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30107.450164 secs 2024-04-27T02:57:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:06 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30107.450164 secs 2024-04-27T02:57:07.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30108.295422 secs 2024-04-27T02:57:07.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:07 smithi119 ceph-mon[32752]: pgmap v16622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:07.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30108.295422 secs 2024-04-27T02:57:07.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:07 smithi167 ceph-mon[32751]: pgmap v16622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30108.295422 secs 2024-04-27T02:57:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:07 smithi090 ceph-mon[29180]: pgmap v16622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T02:57:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:09 smithi119 ceph-mon[32752]: pgmap v16623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:09 smithi167 ceph-mon[32751]: pgmap v16623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:10.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:09 smithi090 ceph-mon[29180]: pgmap v16623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:11 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30112.450251 secs 2024-04-27T02:57:12.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:11 smithi119 ceph-mon[32752]: pgmap v16624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:11 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30112.450251 secs 2024-04-27T02:57:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:11 smithi167 ceph-mon[32751]: pgmap v16624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:11 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30112.450251 secs 2024-04-27T02:57:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:11 smithi090 ceph-mon[29180]: pgmap v16624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:12 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 30113.295476 secs 2024-04-27T02:57:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:12 smithi119 ceph-mon[32752]: slow request 722.153766 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:57:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:12 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 30113.295476 secs 2024-04-27T02:57:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:12 smithi167 ceph-mon[32751]: slow request 722.153766 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:57:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:12 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 30113.295476 secs 2024-04-27T02:57:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:12 smithi090 ceph-mon[29180]: slow request 722.153766 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T02:57:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:13 smithi119 ceph-mon[32752]: pgmap v16625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:13 smithi167 ceph-mon[32751]: pgmap v16625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:13 smithi090 ceph-mon[29180]: pgmap v16625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:15 smithi119 ceph-mon[32752]: pgmap v16626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:15 smithi167 ceph-mon[32751]: pgmap v16626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:15 smithi090 ceph-mon[29180]: pgmap v16626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:16 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30117.450335 secs 2024-04-27T02:57:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:16 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30117.450335 secs 2024-04-27T02:57:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:16 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30117.450335 secs 2024-04-27T02:57:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30118.295594 secs 2024-04-27T02:57:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:17 smithi119 ceph-mon[32752]: pgmap v16627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30118.295594 secs 2024-04-27T02:57:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:17 smithi167 ceph-mon[32751]: pgmap v16627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30118.295594 secs 2024-04-27T02:57:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:17 smithi090 ceph-mon[29180]: pgmap v16627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:19 smithi119 ceph-mon[32752]: pgmap v16628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:19 smithi167 ceph-mon[32751]: pgmap v16628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:19 smithi090 ceph-mon[29180]: pgmap v16628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:21 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30122.450421 secs 2024-04-27T02:57:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:21 smithi119 ceph-mon[32752]: pgmap v16629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:21 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30122.450421 secs 2024-04-27T02:57:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:21 smithi167 ceph-mon[32751]: pgmap v16629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:21 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30122.450421 secs 2024-04-27T02:57:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:21 smithi090 ceph-mon[29180]: pgmap v16629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30123.295732 secs 2024-04-27T02:57:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30123.295732 secs 2024-04-27T02:57:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30123.295732 secs 2024-04-27T02:57:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:23 smithi119 ceph-mon[32752]: pgmap v16630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:23 smithi167 ceph-mon[32751]: pgmap v16630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:23 smithi090 ceph-mon[29180]: pgmap v16630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:25 smithi119 ceph-mon[32752]: pgmap v16631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:25 smithi167 ceph-mon[32751]: pgmap v16631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:25 smithi090 ceph-mon[29180]: pgmap v16631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:26 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30127.450485 secs 2024-04-27T02:57:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:26 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30127.450485 secs 2024-04-27T02:57:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:26 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30127.450485 secs 2024-04-27T02:57:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30128.295837 secs 2024-04-27T02:57:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:27 smithi119 ceph-mon[32752]: pgmap v16632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30128.295837 secs 2024-04-27T02:57:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:27 smithi167 ceph-mon[32751]: pgmap v16632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30128.295837 secs 2024-04-27T02:57:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:27 smithi090 ceph-mon[29180]: pgmap v16632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:29 smithi119 ceph-mon[32752]: pgmap v16633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:30.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:29 smithi167 ceph-mon[32751]: pgmap v16633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:29 smithi090 ceph-mon[29180]: pgmap v16633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:31 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30132.450569 secs 2024-04-27T02:57:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:31 smithi119 ceph-mon[32752]: pgmap v16634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:31 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30132.450569 secs 2024-04-27T02:57:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:31 smithi167 ceph-mon[32751]: pgmap v16634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:31 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30132.450569 secs 2024-04-27T02:57:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:31 smithi090 ceph-mon[29180]: pgmap v16634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30133.295954 secs 2024-04-27T02:57:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30133.295954 secs 2024-04-27T02:57:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30133.295954 secs 2024-04-27T02:57:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:33 smithi119 ceph-mon[32752]: pgmap v16635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:33 smithi167 ceph-mon[32751]: pgmap v16635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:33 smithi090 ceph-mon[29180]: pgmap v16635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:35 smithi119 ceph-mon[32752]: pgmap v16636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:35 smithi167 ceph-mon[32751]: pgmap v16636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:35 smithi090 ceph-mon[29180]: pgmap v16636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:36 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30137.450650 secs 2024-04-27T02:57:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:36 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30137.450650 secs 2024-04-27T02:57:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:36 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30137.450650 secs 2024-04-27T02:57:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30138.296070 secs 2024-04-27T02:57:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:37 smithi119 ceph-mon[32752]: pgmap v16637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30138.296070 secs 2024-04-27T02:57:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:37 smithi167 ceph-mon[32751]: pgmap v16637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30138.296070 secs 2024-04-27T02:57:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:37 smithi090 ceph-mon[29180]: pgmap v16637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:40.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:39 smithi119 ceph-mon[32752]: pgmap v16638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:39 smithi167 ceph-mon[32751]: pgmap v16638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:40.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:39 smithi090 ceph-mon[29180]: pgmap v16638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:41 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30142.450738 secs 2024-04-27T02:57:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:41 smithi119 ceph-mon[32752]: pgmap v16639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:41 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30142.450738 secs 2024-04-27T02:57:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:41 smithi167 ceph-mon[32751]: pgmap v16639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:41 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30142.450738 secs 2024-04-27T02:57:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:41 smithi090 ceph-mon[29180]: pgmap v16639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30143.296164 secs 2024-04-27T02:57:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30143.296164 secs 2024-04-27T02:57:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30143.296164 secs 2024-04-27T02:57:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:43 smithi119 ceph-mon[32752]: pgmap v16640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:43 smithi167 ceph-mon[32751]: pgmap v16640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:43 smithi090 ceph-mon[29180]: pgmap v16640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:45 smithi119 ceph-mon[32752]: pgmap v16641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:46.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:45 smithi167 ceph-mon[32751]: pgmap v16641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:45 smithi090 ceph-mon[29180]: pgmap v16641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:46 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30147.450822 secs 2024-04-27T02:57:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:46 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30147.450822 secs 2024-04-27T02:57:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:46 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30147.450822 secs 2024-04-27T02:57:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30148.296285 secs 2024-04-27T02:57:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:47 smithi119 ceph-mon[32752]: pgmap v16642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30148.296285 secs 2024-04-27T02:57:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:47 smithi167 ceph-mon[32751]: pgmap v16642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30148.296285 secs 2024-04-27T02:57:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:47 smithi090 ceph-mon[29180]: pgmap v16642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:57:49.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:57:49.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:57:49.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:57:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:49 smithi119 ceph-mon[32752]: pgmap v16643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:57:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:57:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:49 smithi119 ceph-mon[32752]: pgmap v16644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:57:50.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:49 smithi119 ceph-mon[32752]: pgmap v16645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:50.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:57:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:49 smithi167 ceph-mon[32751]: pgmap v16643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:57:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:57:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:49 smithi167 ceph-mon[32751]: pgmap v16644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:57:50.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:49 smithi167 ceph-mon[32751]: pgmap v16645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:50.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:57:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:49 smithi090 ceph-mon[29180]: pgmap v16643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:57:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:57:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:57:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:49 smithi090 ceph-mon[29180]: pgmap v16644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:57:50.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:49 smithi090 ceph-mon[29180]: pgmap v16645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:50.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:57:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:51 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30152.450919 secs 2024-04-27T02:57:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:51 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30152.450919 secs 2024-04-27T02:57:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:51 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30152.450919 secs 2024-04-27T02:57:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30153.296411 secs 2024-04-27T02:57:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:52 smithi119 ceph-mon[32752]: pgmap v16646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30153.296411 secs 2024-04-27T02:57:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:52 smithi167 ceph-mon[32751]: pgmap v16646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30153.296411 secs 2024-04-27T02:57:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:52 smithi090 ceph-mon[29180]: pgmap v16646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:54 smithi119 ceph-mon[32752]: pgmap v16647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:54 smithi167 ceph-mon[32751]: pgmap v16647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:54 smithi090 ceph-mon[29180]: pgmap v16647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:56 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30157.450977 secs 2024-04-27T02:57:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:56 smithi119 ceph-mon[32752]: pgmap v16648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:56 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30157.450977 secs 2024-04-27T02:57:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:56 smithi167 ceph-mon[32751]: pgmap v16648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:56 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30157.450977 secs 2024-04-27T02:57:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:56 smithi090 ceph-mon[29180]: pgmap v16648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:57:57.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30158.296519 secs 2024-04-27T02:57:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30158.296519 secs 2024-04-27T02:57:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30158.296519 secs 2024-04-27T02:57:58.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:57:58 smithi119 ceph-mon[32752]: pgmap v16649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:57:58.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:57:58 smithi167 ceph-mon[32751]: pgmap v16649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:57:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:57:58 smithi090 ceph-mon[29180]: pgmap v16649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:58:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:00 smithi119 ceph-mon[32752]: pgmap v16650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:58:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:00 smithi167 ceph-mon[32751]: pgmap v16650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:58:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:00 smithi090 ceph-mon[29180]: pgmap v16650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:58:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:01 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30162.451063 secs 2024-04-27T02:58:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:01 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30162.451063 secs 2024-04-27T02:58:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:01 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30162.451063 secs 2024-04-27T02:58:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30163.296597 secs 2024-04-27T02:58:02.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:02 smithi119 ceph-mon[32752]: pgmap v16651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30163.296597 secs 2024-04-27T02:58:02.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:02 smithi167 ceph-mon[32751]: pgmap v16651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30163.296597 secs 2024-04-27T02:58:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:02 smithi090 ceph-mon[29180]: pgmap v16651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:04 smithi119 ceph-mon[32752]: pgmap v16652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:04 smithi167 ceph-mon[32751]: pgmap v16652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:04 smithi090 ceph-mon[29180]: pgmap v16652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:06 smithi119 ceph-mon[32752]: 5 slow requests, 0 included below; oldest blocked for > 30167.451123 secs 2024-04-27T02:58:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:06 smithi119 ceph-mon[32752]: pgmap v16653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:06 smithi167 ceph-mon[32751]: 5 slow requests, 0 included below; oldest blocked for > 30167.451123 secs 2024-04-27T02:58:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:06 smithi167 ceph-mon[32751]: pgmap v16653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:06 smithi090 ceph-mon[29180]: 5 slow requests, 0 included below; oldest blocked for > 30167.451123 secs 2024-04-27T02:58:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:06 smithi090 ceph-mon[29180]: pgmap v16653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30168.296707 secs 2024-04-27T02:58:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30168.296707 secs 2024-04-27T02:58:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30168.296707 secs 2024-04-27T02:58:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:08 smithi119 ceph-mon[32752]: pgmap v16654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:08 smithi167 ceph-mon[32751]: pgmap v16654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:08.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:08 smithi090 ceph-mon[29180]: pgmap v16654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:10 smithi119 ceph-mon[32752]: pgmap v16655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:10 smithi167 ceph-mon[32751]: pgmap v16655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:10 smithi090 ceph-mon[29180]: pgmap v16655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:11.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:11 smithi090 ceph-mon[29180]: 40 slow requests, 2 included below; oldest blocked for > 30172.451205 secs 2024-04-27T02:58:11.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:11 smithi090 ceph-mon[29180]: slow request 181.142809 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:58:11.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:11 smithi090 ceph-mon[29180]: slow request 11623.955313 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:58:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:11 smithi119 ceph-mon[32752]: 40 slow requests, 2 included below; oldest blocked for > 30172.451205 secs 2024-04-27T02:58:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:11 smithi119 ceph-mon[32752]: slow request 181.142809 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:58:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:11 smithi119 ceph-mon[32752]: slow request 11623.955313 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:58:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:11 smithi167 ceph-mon[32751]: 40 slow requests, 2 included below; oldest blocked for > 30172.451205 secs 2024-04-27T02:58:11.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:11 smithi167 ceph-mon[32751]: slow request 181.142809 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T02:58:11.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:11 smithi167 ceph-mon[32751]: slow request 11623.955313 seconds old, received at 2024-04-26T23:44:26.249296+0000: client_request(client.24733:267894 getattr AsXsFs #0x100000001fb 2024-04-26T23:44:26.248679+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T02:58:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30173.296783 secs 2024-04-27T02:58:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:12 smithi119 ceph-mon[32752]: pgmap v16656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30173.296783 secs 2024-04-27T02:58:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:12 smithi167 ceph-mon[32751]: pgmap v16656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30173.296783 secs 2024-04-27T02:58:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:12 smithi090 ceph-mon[29180]: pgmap v16656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:14 smithi119 ceph-mon[32752]: pgmap v16657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:14 smithi167 ceph-mon[32751]: pgmap v16657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:14 smithi090 ceph-mon[29180]: pgmap v16657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:58:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:58:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:58:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:58:15.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T02:58:15.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T02:58:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:16 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30177.451286 secs 2024-04-27T02:58:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:16 smithi119 ceph-mon[32752]: pgmap v16658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:16 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30177.451286 secs 2024-04-27T02:58:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:16 smithi167 ceph-mon[32751]: pgmap v16658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:16 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30177.451286 secs 2024-04-27T02:58:16.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:16 smithi090 ceph-mon[29180]: pgmap v16658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30178.296856 secs 2024-04-27T02:58:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30178.296856 secs 2024-04-27T02:58:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30178.296856 secs 2024-04-27T02:58:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:18 smithi119 ceph-mon[32752]: pgmap v16659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:18.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:18 smithi167 ceph-mon[32751]: pgmap v16659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:18 smithi090 ceph-mon[29180]: pgmap v16659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:20 smithi119 ceph-mon[32752]: pgmap v16660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:20 smithi167 ceph-mon[32751]: pgmap v16660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:20 smithi090 ceph-mon[29180]: pgmap v16660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:21 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30182.451371 secs 2024-04-27T02:58:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:21 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30182.451371 secs 2024-04-27T02:58:21.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:21 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30182.451371 secs 2024-04-27T02:58:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30183.296969 secs 2024-04-27T02:58:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:22 smithi119 ceph-mon[32752]: pgmap v16661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:22.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30183.296969 secs 2024-04-27T02:58:22.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:22 smithi167 ceph-mon[32751]: pgmap v16661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30183.296969 secs 2024-04-27T02:58:22.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:22 smithi090 ceph-mon[29180]: pgmap v16661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:24.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:24 smithi119 ceph-mon[32752]: pgmap v16662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:24 smithi167 ceph-mon[32751]: pgmap v16662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:24 smithi090 ceph-mon[29180]: pgmap v16662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:26 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30187.451449 secs 2024-04-27T02:58:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:26 smithi119 ceph-mon[32752]: pgmap v16663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:26 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30187.451449 secs 2024-04-27T02:58:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:26 smithi167 ceph-mon[32751]: pgmap v16663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:26 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30187.451449 secs 2024-04-27T02:58:26.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:26 smithi090 ceph-mon[29180]: pgmap v16663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30188.297114 secs 2024-04-27T02:58:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30188.297114 secs 2024-04-27T02:58:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30188.297114 secs 2024-04-27T02:58:28.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:28 smithi167 ceph-mon[32751]: pgmap v16664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:28 smithi090 ceph-mon[29180]: pgmap v16664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:29.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:28 smithi119 ceph-mon[32752]: pgmap v16664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:29.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:29 smithi167 ceph-mon[32751]: pgmap v16665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:29 smithi090 ceph-mon[29180]: pgmap v16665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:29 smithi119 ceph-mon[32752]: pgmap v16665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:31 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30192.451535 secs 2024-04-27T02:58:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:31 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30192.451535 secs 2024-04-27T02:58:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:31 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30192.451535 secs 2024-04-27T02:58:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30193.297235 secs 2024-04-27T02:58:32.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:32 smithi119 ceph-mon[32752]: pgmap v16666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30193.297235 secs 2024-04-27T02:58:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:32 smithi167 ceph-mon[32751]: pgmap v16666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30193.297235 secs 2024-04-27T02:58:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:32 smithi090 ceph-mon[29180]: pgmap v16666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:34 smithi119 ceph-mon[32752]: pgmap v16667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:34 smithi167 ceph-mon[32751]: pgmap v16667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:34 smithi090 ceph-mon[29180]: pgmap v16667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:36 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30197.451612 secs 2024-04-27T02:58:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:36 smithi119 ceph-mon[32752]: pgmap v16668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:36 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30197.451612 secs 2024-04-27T02:58:36.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:36 smithi167 ceph-mon[32751]: pgmap v16668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:36 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30197.451612 secs 2024-04-27T02:58:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:36 smithi090 ceph-mon[29180]: pgmap v16668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:37.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30198.297329 secs 2024-04-27T02:58:37.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30198.297329 secs 2024-04-27T02:58:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30198.297329 secs 2024-04-27T02:58:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:38 smithi119 ceph-mon[32752]: pgmap v16669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:38 smithi167 ceph-mon[32751]: pgmap v16669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:38.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:38 smithi090 ceph-mon[29180]: pgmap v16669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:40.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:40 smithi119 ceph-mon[32752]: pgmap v16670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:40.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:40 smithi167 ceph-mon[32751]: pgmap v16670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:40.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:40 smithi090 ceph-mon[29180]: pgmap v16670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:41.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:41 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30202.451696 secs 2024-04-27T02:58:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:41 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30202.451696 secs 2024-04-27T02:58:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:41 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30202.451696 secs 2024-04-27T02:58:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30203.297434 secs 2024-04-27T02:58:42.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:42 smithi119 ceph-mon[32752]: pgmap v16671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30203.297434 secs 2024-04-27T02:58:42.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:42 smithi167 ceph-mon[32751]: pgmap v16671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30203.297434 secs 2024-04-27T02:58:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:42 smithi090 ceph-mon[29180]: pgmap v16671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:44 smithi119 ceph-mon[32752]: pgmap v16672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:44.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:44 smithi167 ceph-mon[32751]: pgmap v16672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:44.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:44 smithi090 ceph-mon[29180]: pgmap v16672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:46.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:46 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30207.451781 secs 2024-04-27T02:58:46.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:46 smithi119 ceph-mon[32752]: pgmap v16673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:46 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30207.451781 secs 2024-04-27T02:58:46.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:46 smithi167 ceph-mon[32751]: pgmap v16673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:46 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30207.451781 secs 2024-04-27T02:58:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:46 smithi090 ceph-mon[29180]: pgmap v16673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30208.297587 secs 2024-04-27T02:58:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30208.297587 secs 2024-04-27T02:58:47.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30208.297587 secs 2024-04-27T02:58:48.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:48 smithi119 ceph-mon[32752]: pgmap v16674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:48.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:48 smithi167 ceph-mon[32751]: pgmap v16674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:48.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:48 smithi090 ceph-mon[29180]: pgmap v16674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:58:50.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:50 smithi119 ceph-mon[32752]: pgmap v16675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:50.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:58:50.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:58:50.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:58:50.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:58:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:50 smithi167 ceph-mon[32751]: pgmap v16675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:58:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:58:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:58:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:58:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:50 smithi090 ceph-mon[29180]: pgmap v16675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:58:50.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:58:50.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:58:50.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:58:50.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:58:51.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:51 smithi119 ceph-mon[32752]: pgmap v16676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:58:51.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:51 smithi119 ceph-mon[32752]: pgmap v16677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:51.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:51 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30212.451866 secs 2024-04-27T02:58:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:51 smithi167 ceph-mon[32751]: pgmap v16676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:58:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:51 smithi167 ceph-mon[32751]: pgmap v16677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:51 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30212.451866 secs 2024-04-27T02:58:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:51 smithi090 ceph-mon[29180]: pgmap v16676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T02:58:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:51 smithi090 ceph-mon[29180]: pgmap v16677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:51 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30212.451866 secs 2024-04-27T02:58:52.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30213.297740 secs 2024-04-27T02:58:52.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30213.297740 secs 2024-04-27T02:58:52.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30213.297740 secs 2024-04-27T02:58:53.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:53 smithi119 ceph-mon[32752]: pgmap v16678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:58:53.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:53 smithi167 ceph-mon[32751]: pgmap v16678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:58:53.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:53 smithi090 ceph-mon[29180]: pgmap v16678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:58:55.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:55 smithi119 ceph-mon[32752]: pgmap v16679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:55.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:55 smithi167 ceph-mon[32751]: pgmap v16679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:55.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:55 smithi090 ceph-mon[29180]: pgmap v16679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:56.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:56 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30217.451923 secs 2024-04-27T02:58:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:56 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30217.451923 secs 2024-04-27T02:58:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:56 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30217.451923 secs 2024-04-27T02:58:57.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30218.297871 secs 2024-04-27T02:58:57.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:57 smithi119 ceph-mon[32752]: pgmap v16680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30218.297871 secs 2024-04-27T02:58:57.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:57 smithi167 ceph-mon[32751]: pgmap v16680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:57.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30218.297871 secs 2024-04-27T02:58:57.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:57 smithi090 ceph-mon[29180]: pgmap v16680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:58:59.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:58:59 smithi119 ceph-mon[32752]: pgmap v16681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:58:59.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:58:59 smithi167 ceph-mon[32751]: pgmap v16681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:58:59.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:58:59 smithi090 ceph-mon[29180]: pgmap v16681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T02:59:00.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:00 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30222.452009 secs 2024-04-27T02:59:00.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:00 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30222.452009 secs 2024-04-27T02:59:00.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:00 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30222.452009 secs 2024-04-27T02:59:01.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:01 smithi119 ceph-mon[32752]: pgmap v16682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:59:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:01 smithi167 ceph-mon[32751]: pgmap v16682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:59:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:01 smithi090 ceph-mon[29180]: pgmap v16682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T02:59:02.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30223.297989 secs 2024-04-27T02:59:02.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30223.297989 secs 2024-04-27T02:59:02.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30223.297989 secs 2024-04-27T02:59:03.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:03 smithi119 ceph-mon[32752]: pgmap v16683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:03.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:03 smithi167 ceph-mon[32751]: pgmap v16683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:03.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:03 smithi090 ceph-mon[29180]: pgmap v16683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:05 smithi167 ceph-mon[32751]: pgmap v16684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:05.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:05 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30227.452108 secs 2024-04-27T02:59:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:05 smithi090 ceph-mon[29180]: pgmap v16684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:05.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:05 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30227.452108 secs 2024-04-27T02:59:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:05 smithi119 ceph-mon[32752]: pgmap v16684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:05 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30227.452108 secs 2024-04-27T02:59:07.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30228.298088 secs 2024-04-27T02:59:07.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:07 smithi167 ceph-mon[32751]: pgmap v16685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:07.858 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30228.298088 secs 2024-04-27T02:59:07.859 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:07 smithi119 ceph-mon[32752]: pgmap v16685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:07.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30228.298088 secs 2024-04-27T02:59:07.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:07 smithi090 ceph-mon[29180]: pgmap v16685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:09.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:09 smithi167 ceph-mon[32751]: pgmap v16686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:09.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:09 smithi090 ceph-mon[29180]: pgmap v16686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:09 smithi119 ceph-mon[32752]: pgmap v16686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:10.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:10 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30232.452192 secs 2024-04-27T02:59:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:10 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30232.452192 secs 2024-04-27T02:59:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:10 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30232.452192 secs 2024-04-27T02:59:11.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:11 smithi090 ceph-mon[29180]: pgmap v16687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:11 smithi119 ceph-mon[32752]: pgmap v16687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:11 smithi167 ceph-mon[32751]: pgmap v16687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30233.298178 secs 2024-04-27T02:59:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:12 smithi090 ceph-mon[29180]: pgmap v16688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30233.298178 secs 2024-04-27T02:59:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:12 smithi119 ceph-mon[32752]: pgmap v16688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30233.298178 secs 2024-04-27T02:59:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:12 smithi167 ceph-mon[32751]: pgmap v16688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:15.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:15 smithi119 ceph-mon[32752]: pgmap v16689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:15.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:15 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30237.452270 secs 2024-04-27T02:59:15.560 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:15 smithi090 ceph-mon[29180]: pgmap v16689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:15.560 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:15 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30237.452270 secs 2024-04-27T02:59:15.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:15 smithi167 ceph-mon[32751]: pgmap v16689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:15.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:15 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30237.452270 secs 2024-04-27T02:59:17.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30238.298299 secs 2024-04-27T02:59:17.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:17 smithi119 ceph-mon[32752]: pgmap v16690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:17.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30238.298299 secs 2024-04-27T02:59:17.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:17 smithi167 ceph-mon[32751]: pgmap v16690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:17.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30238.298299 secs 2024-04-27T02:59:17.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:17 smithi090 ceph-mon[29180]: pgmap v16690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:19.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:19 smithi119 ceph-mon[32752]: pgmap v16691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:19.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:19 smithi167 ceph-mon[32751]: pgmap v16691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:19.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:19 smithi090 ceph-mon[29180]: pgmap v16691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:20.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:20 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30242.452352 secs 2024-04-27T02:59:20.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:20 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30242.452352 secs 2024-04-27T02:59:20.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:20 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30242.452352 secs 2024-04-27T02:59:21.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:21 smithi119 ceph-mon[32752]: pgmap v16692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:21.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:21 smithi167 ceph-mon[32751]: pgmap v16692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:21.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:21 smithi090 ceph-mon[29180]: pgmap v16692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:22.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30243.298430 secs 2024-04-27T02:59:22.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30243.298430 secs 2024-04-27T02:59:22.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30243.298430 secs 2024-04-27T02:59:23.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:23 smithi119 ceph-mon[32752]: pgmap v16693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:23.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:23 smithi167 ceph-mon[32751]: pgmap v16693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:23.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:23 smithi090 ceph-mon[29180]: pgmap v16693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:25.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:25 smithi119 ceph-mon[32752]: pgmap v16694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:25.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:25 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30247.452439 secs 2024-04-27T02:59:25.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:25 smithi167 ceph-mon[32751]: pgmap v16694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:25.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:25 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30247.452439 secs 2024-04-27T02:59:25.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:25 smithi090 ceph-mon[29180]: pgmap v16694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:25.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:25 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30247.452439 secs 2024-04-27T02:59:27.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30248.298585 secs 2024-04-27T02:59:27.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:27 smithi119 ceph-mon[32752]: pgmap v16695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:27.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30248.298585 secs 2024-04-27T02:59:27.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:27 smithi167 ceph-mon[32751]: pgmap v16695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:27.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30248.298585 secs 2024-04-27T02:59:27.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:27 smithi090 ceph-mon[29180]: pgmap v16695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:29.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:29 smithi119 ceph-mon[32752]: pgmap v16696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:29.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:29 smithi167 ceph-mon[32751]: pgmap v16696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:29.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:29 smithi090 ceph-mon[29180]: pgmap v16696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:30.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:30 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30252.452522 secs 2024-04-27T02:59:30.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:30 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30252.452522 secs 2024-04-27T02:59:30.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:30 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30252.452522 secs 2024-04-27T02:59:31.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:31 smithi119 ceph-mon[32752]: pgmap v16697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:31.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:31 smithi167 ceph-mon[32751]: pgmap v16697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:31 smithi090 ceph-mon[29180]: pgmap v16697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:32.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30253.298687 secs 2024-04-27T02:59:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30253.298687 secs 2024-04-27T02:59:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30253.298687 secs 2024-04-27T02:59:33.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:33 smithi119 ceph-mon[32752]: pgmap v16698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:33.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:33 smithi167 ceph-mon[32751]: pgmap v16698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:33 smithi090 ceph-mon[29180]: pgmap v16698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:35.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:35 smithi119 ceph-mon[32752]: pgmap v16699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:35.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:35 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30257.452602 secs 2024-04-27T02:59:35.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:35 smithi167 ceph-mon[32751]: pgmap v16699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:35.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:35 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30257.452602 secs 2024-04-27T02:59:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:35 smithi090 ceph-mon[29180]: pgmap v16699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:35.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:35 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30257.452602 secs 2024-04-27T02:59:37.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30258.298798 secs 2024-04-27T02:59:37.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:37 smithi167 ceph-mon[32751]: pgmap v16700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:37.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30258.298798 secs 2024-04-27T02:59:37.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:37 smithi090 ceph-mon[29180]: pgmap v16700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:37.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30258.298798 secs 2024-04-27T02:59:37.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:37 smithi119 ceph-mon[32752]: pgmap v16700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:39.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:39 smithi167 ceph-mon[32751]: pgmap v16701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:39.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:39 smithi090 ceph-mon[29180]: pgmap v16701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:39.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:39 smithi119 ceph-mon[32752]: pgmap v16701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:40.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:40 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30262.452686 secs 2024-04-27T02:59:40.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:40 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30262.452686 secs 2024-04-27T02:59:40.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:40 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30262.452686 secs 2024-04-27T02:59:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:41 smithi167 ceph-mon[32751]: pgmap v16702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:41 smithi090 ceph-mon[29180]: pgmap v16702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:41.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:41 smithi119 ceph-mon[32752]: pgmap v16702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30263.298861 secs 2024-04-27T02:59:42.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30263.298861 secs 2024-04-27T02:59:42.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30263.298861 secs 2024-04-27T02:59:43.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:43 smithi090 ceph-mon[29180]: pgmap v16703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:43.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:43 smithi119 ceph-mon[32752]: pgmap v16703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:43.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:43 smithi167 ceph-mon[32751]: pgmap v16703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:45.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:45 smithi119 ceph-mon[32752]: pgmap v16704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:45.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:45 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30267.452767 secs 2024-04-27T02:59:45.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:45 smithi167 ceph-mon[32751]: pgmap v16704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:45.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:45 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30267.452767 secs 2024-04-27T02:59:45.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:45 smithi090 ceph-mon[29180]: pgmap v16704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:45.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:45 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30267.452767 secs 2024-04-27T02:59:47.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30268.299004 secs 2024-04-27T02:59:47.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:47 smithi119 ceph-mon[32752]: pgmap v16705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:47.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30268.299004 secs 2024-04-27T02:59:47.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:47 smithi167 ceph-mon[32751]: pgmap v16705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:47.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30268.299004 secs 2024-04-27T02:59:47.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:47 smithi090 ceph-mon[29180]: pgmap v16705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:49.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:49 smithi119 ceph-mon[32752]: pgmap v16706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:49.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:49 smithi167 ceph-mon[32751]: pgmap v16706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:49.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:49 smithi090 ceph-mon[29180]: pgmap v16706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T02:59:50.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:50 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30272.452851 secs 2024-04-27T02:59:50.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:59:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:50 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30272.452851 secs 2024-04-27T02:59:50.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:59:50.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:50 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30272.452851 secs 2024-04-27T02:59:50.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T02:59:51.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:51 smithi119 ceph-mon[32752]: pgmap v16707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:51.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:59:51.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:59:51.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:59:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:51 smithi167 ceph-mon[32751]: pgmap v16707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:59:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:59:51.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:59:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:51 smithi090 ceph-mon[29180]: pgmap v16707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T02:59:51.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T02:59:51.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T02:59:51.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T02:59:52.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:52 smithi119 ceph-mon[32752]: pgmap v16708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:59:52.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:52 smithi119 ceph-mon[32752]: pgmap v16709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:52.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30273.299108 secs 2024-04-27T02:59:52.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:52 smithi167 ceph-mon[32751]: pgmap v16708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:59:52.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:52 smithi167 ceph-mon[32751]: pgmap v16709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:52.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30273.299108 secs 2024-04-27T02:59:52.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:52 smithi090 ceph-mon[29180]: pgmap v16708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T02:59:52.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:52 smithi090 ceph-mon[29180]: pgmap v16709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:52.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30273.299108 secs 2024-04-27T02:59:54.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:54 smithi119 ceph-mon[32752]: pgmap v16710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:59:54.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:54 smithi167 ceph-mon[32751]: pgmap v16710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:59:54.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:54 smithi090 ceph-mon[29180]: pgmap v16710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T02:59:55.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:55 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30277.452908 secs 2024-04-27T02:59:55.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:55 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30277.452908 secs 2024-04-27T02:59:56.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:55 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30277.452908 secs 2024-04-27T02:59:56.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:56 smithi167 ceph-mon[32751]: pgmap v16711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:56 smithi090 ceph-mon[29180]: pgmap v16711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:56 smithi119 ceph-mon[32752]: pgmap v16711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:57.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30278.299246 secs 2024-04-27T02:59:58.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30278.299246 secs 2024-04-27T02:59:58.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30278.299246 secs 2024-04-27T02:59:58.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 02:59:58 smithi090 ceph-mon[29180]: pgmap v16712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:59.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 02:59:58 smithi119 ceph-mon[32752]: pgmap v16712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T02:59:59.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 02:59:58 smithi167 ceph-mon[32751]: pgmap v16712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:00.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:00 smithi090 ceph-mon[29180]: pgmap v16713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:00.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:00:00.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:00 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30282.452990 secs 2024-04-27T03:00:01.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:00 smithi119 ceph-mon[32752]: pgmap v16713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:01.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:00:01.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:00 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30282.452990 secs 2024-04-27T03:00:01.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:00 smithi167 ceph-mon[32751]: pgmap v16713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:01.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:00:01.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:00 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30282.452990 secs 2024-04-27T03:00:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:01 smithi090 ceph-mon[29180]: pgmap v16714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:00:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:01 smithi119 ceph-mon[32752]: pgmap v16714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:00:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:01 smithi167 ceph-mon[32751]: pgmap v16714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:00:02.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30283.299319 secs 2024-04-27T03:00:03.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30283.299319 secs 2024-04-27T03:00:03.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30283.299319 secs 2024-04-27T03:00:04.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:03 smithi119 ceph-mon[32752]: pgmap v16715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:04.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:03 smithi167 ceph-mon[32751]: pgmap v16715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:04.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:03 smithi090 ceph-mon[29180]: pgmap v16715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:06 smithi119 ceph-mon[32752]: pgmap v16716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:06.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:06 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30287.453051 secs 2024-04-27T03:00:06.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:06 smithi167 ceph-mon[32751]: pgmap v16716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:06.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:06 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30287.453051 secs 2024-04-27T03:00:06.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:06 smithi090 ceph-mon[29180]: pgmap v16716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:06.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:06 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30287.453051 secs 2024-04-27T03:00:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30288.299423 secs 2024-04-27T03:00:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30288.299423 secs 2024-04-27T03:00:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30288.299423 secs 2024-04-27T03:00:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:08 smithi119 ceph-mon[32752]: pgmap v16717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:08.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:08 smithi167 ceph-mon[32751]: pgmap v16717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:08 smithi090 ceph-mon[29180]: pgmap v16717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:09.630 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:00:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:00:09.183+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:00:09.630 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:00:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:00:09.183+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:00:09.630 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:00:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:00:09.183+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:00:09.630 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:00:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:00:09.183+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:00:09.630 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:00:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:00:09.183+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:00:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:10 smithi119 ceph-mon[32752]: pgmap v16718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:10 smithi167 ceph-mon[32751]: pgmap v16718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:10 smithi090 ceph-mon[29180]: pgmap v16718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:11 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30292.453112 secs 2024-04-27T03:00:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:11 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30292.453112 secs 2024-04-27T03:00:11.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:11 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30292.453112 secs 2024-04-27T03:00:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:12 smithi119 ceph-mon[32752]: pgmap v16719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30293.299520 secs 2024-04-27T03:00:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:12 smithi167 ceph-mon[32751]: pgmap v16719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:12.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30293.299520 secs 2024-04-27T03:00:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:12 smithi090 ceph-mon[29180]: pgmap v16719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30293.299520 secs 2024-04-27T03:00:14.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:14 smithi167 ceph-mon[32751]: pgmap v16720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:14.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:14 smithi090 ceph-mon[29180]: pgmap v16720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:14.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:14 smithi119 ceph-mon[32752]: pgmap v16720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:16.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:16 smithi167 ceph-mon[32751]: pgmap v16721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:16.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:16 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30297.453196 secs 2024-04-27T03:00:16.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:16 smithi090 ceph-mon[29180]: pgmap v16721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:16.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:16 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30297.453196 secs 2024-04-27T03:00:16.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:16 smithi119 ceph-mon[32752]: pgmap v16721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:16.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:16 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30297.453196 secs 2024-04-27T03:00:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30298.299618 secs 2024-04-27T03:00:17.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30298.299618 secs 2024-04-27T03:00:17.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30298.299618 secs 2024-04-27T03:00:18.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:18 smithi167 ceph-mon[32751]: pgmap v16722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:18.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:18 smithi090 ceph-mon[29180]: pgmap v16722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:18.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:18 smithi119 ceph-mon[32752]: pgmap v16722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:20 smithi090 ceph-mon[29180]: pgmap v16723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:20.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:20 smithi119 ceph-mon[32752]: pgmap v16723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:20.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:20 smithi167 ceph-mon[32751]: pgmap v16723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:21.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:21 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30302.453281 secs 2024-04-27T03:00:21.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:21 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30302.453281 secs 2024-04-27T03:00:21.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:21 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30302.453281 secs 2024-04-27T03:00:22.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:22 smithi090 ceph-mon[29180]: pgmap v16724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:22.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30303.299722 secs 2024-04-27T03:00:22.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:22 smithi119 ceph-mon[32752]: pgmap v16724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:22.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30303.299722 secs 2024-04-27T03:00:22.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:22 smithi167 ceph-mon[32751]: pgmap v16724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:22.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30303.299722 secs 2024-04-27T03:00:24.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:24 smithi090 ceph-mon[29180]: pgmap v16725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:24.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:24 smithi119 ceph-mon[32752]: pgmap v16725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:24.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:24 smithi167 ceph-mon[32751]: pgmap v16725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:26.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:26 smithi090 ceph-mon[29180]: pgmap v16726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:26.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:26 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30307.453350 secs 2024-04-27T03:00:26.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:26 smithi119 ceph-mon[32752]: pgmap v16726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:26.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:26 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30307.453350 secs 2024-04-27T03:00:26.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:26 smithi167 ceph-mon[32751]: pgmap v16726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T03:00:26.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:26 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30307.453350 secs 2024-04-27T03:00:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30308.299785 secs 2024-04-27T03:00:27.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30308.299785 secs 2024-04-27T03:00:27.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30308.299785 secs 2024-04-27T03:00:28.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:28 smithi090 ceph-mon[29180]: pgmap v16727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:28.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:28 smithi119 ceph-mon[32752]: pgmap v16727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:28.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:28 smithi167 ceph-mon[32751]: pgmap v16727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:30.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:30 smithi090 ceph-mon[29180]: pgmap v16728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:30.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:30 smithi119 ceph-mon[32752]: pgmap v16728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:30.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:30 smithi167 ceph-mon[32751]: pgmap v16728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:31.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:31 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30312.453433 secs 2024-04-27T03:00:31.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:31 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30312.453433 secs 2024-04-27T03:00:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:31 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30312.453433 secs 2024-04-27T03:00:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:32 smithi090 ceph-mon[29180]: pgmap v16729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30313.299885 secs 2024-04-27T03:00:32.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:32 smithi119 ceph-mon[32752]: pgmap v16729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:32.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30313.299885 secs 2024-04-27T03:00:32.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:32 smithi167 ceph-mon[32751]: pgmap v16729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:32.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30313.299885 secs 2024-04-27T03:00:34.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:34 smithi090 ceph-mon[29180]: pgmap v16730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:34.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:34 smithi119 ceph-mon[32752]: pgmap v16730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:34.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:34 smithi167 ceph-mon[32751]: pgmap v16730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:36.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:36 smithi090 ceph-mon[29180]: pgmap v16731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:36.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:36 smithi090 ceph-mon[29180]: 40 slow requests, 1 included below; oldest blocked for > 30317.453516 secs 2024-04-27T03:00:36.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:36 smithi090 ceph-mon[29180]: slow request 721.693857 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:00:36.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:36 smithi119 ceph-mon[32752]: pgmap v16731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:36.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:36 smithi119 ceph-mon[32752]: 40 slow requests, 1 included below; oldest blocked for > 30317.453516 secs 2024-04-27T03:00:36.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:36 smithi119 ceph-mon[32752]: slow request 721.693857 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:00:36.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:36 smithi167 ceph-mon[32751]: pgmap v16731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:36.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:36 smithi167 ceph-mon[32751]: 40 slow requests, 1 included below; oldest blocked for > 30317.453516 secs 2024-04-27T03:00:36.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:36 smithi167 ceph-mon[32751]: slow request 721.693857 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:00:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30318.299983 secs 2024-04-27T03:00:37.461 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30318.299983 secs 2024-04-27T03:00:37.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30318.299983 secs 2024-04-27T03:00:38.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:38 smithi090 ceph-mon[29180]: pgmap v16732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:38.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:38 smithi119 ceph-mon[32752]: pgmap v16732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:38.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:38 smithi167 ceph-mon[32751]: pgmap v16732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:40 smithi090 ceph-mon[29180]: pgmap v16733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:40.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:40 smithi119 ceph-mon[32752]: pgmap v16733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:40.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:40 smithi167 ceph-mon[32751]: pgmap v16733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:41.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:41 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30322.453599 secs 2024-04-27T03:00:41.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:41 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30322.453599 secs 2024-04-27T03:00:41.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:41 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30322.453599 secs 2024-04-27T03:00:42.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:42 smithi119 ceph-mon[32752]: pgmap v16734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:42.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30323.300104 secs 2024-04-27T03:00:42.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:42 smithi167 ceph-mon[32751]: pgmap v16734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:42.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30323.300104 secs 2024-04-27T03:00:42.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:42 smithi090 ceph-mon[29180]: pgmap v16734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:42.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30323.300104 secs 2024-04-27T03:00:44.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:44 smithi119 ceph-mon[32752]: pgmap v16735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:44.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:44 smithi167 ceph-mon[32751]: pgmap v16735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:44.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:44 smithi090 ceph-mon[29180]: pgmap v16735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:45.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:45 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30327.453687 secs 2024-04-27T03:00:45.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:45 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30327.453687 secs 2024-04-27T03:00:45.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:45 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30327.453687 secs 2024-04-27T03:00:46.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:46 smithi119 ceph-mon[32752]: pgmap v16736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:46.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:46 smithi167 ceph-mon[32751]: pgmap v16736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:46.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:46 smithi090 ceph-mon[29180]: pgmap v16736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:47.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30328.300264 secs 2024-04-27T03:00:47.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30328.300264 secs 2024-04-27T03:00:48.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30328.300264 secs 2024-04-27T03:00:48.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:48 smithi167 ceph-mon[32751]: pgmap v16737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:48.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:48 smithi090 ceph-mon[29180]: pgmap v16737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:49.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:48 smithi119 ceph-mon[32752]: pgmap v16737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:50.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:50 smithi167 ceph-mon[32751]: pgmap v16738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:50.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:50 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30332.453771 secs 2024-04-27T03:00:50.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:50 smithi090 ceph-mon[29180]: pgmap v16738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:50.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:50 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30332.453771 secs 2024-04-27T03:00:51.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:50 smithi119 ceph-mon[32752]: pgmap v16738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:00:51.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:50 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30332.453771 secs 2024-04-27T03:00:51.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:00:51.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:00:52.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:00:52.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:52 smithi090 ceph-mon[29180]: pgmap v16739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:52.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30333.300424 secs 2024-04-27T03:00:52.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:00:52.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:00:52.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:00:53.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:52 smithi119 ceph-mon[32752]: pgmap v16739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:53.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30333.300424 secs 2024-04-27T03:00:53.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:00:53.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:00:53.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:00:53.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:52 smithi167 ceph-mon[32751]: pgmap v16739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:00:53.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30333.300424 secs 2024-04-27T03:00:53.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:00:53.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:00:53.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:00:53.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:53 smithi090 ceph-mon[29180]: pgmap v16740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:00:53.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:53 smithi090 ceph-mon[29180]: pgmap v16741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:54.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:53 smithi119 ceph-mon[32752]: pgmap v16740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:00:54.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:53 smithi119 ceph-mon[32752]: pgmap v16741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:54.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:53 smithi167 ceph-mon[32751]: pgmap v16740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:00:54.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:53 smithi167 ceph-mon[32751]: pgmap v16741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:54.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:54 smithi090 ceph-mon[29180]: pgmap v16742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:55.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:54 smithi119 ceph-mon[32752]: pgmap v16742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:55.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:54 smithi167 ceph-mon[32751]: pgmap v16742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:55.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:55 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30337.453859 secs 2024-04-27T03:00:56.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:55 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30337.453859 secs 2024-04-27T03:00:56.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:55 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30337.453859 secs 2024-04-27T03:00:56.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:56 smithi090 ceph-mon[29180]: pgmap v16743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:57.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:56 smithi119 ceph-mon[32752]: pgmap v16743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:57.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:56 smithi167 ceph-mon[32751]: pgmap v16743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:00:57.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30338.300561 secs 2024-04-27T03:00:58.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30338.300561 secs 2024-04-27T03:00:58.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30338.300561 secs 2024-04-27T03:00:58.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:00:58 smithi090 ceph-mon[29180]: pgmap v16744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:00:59.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:00:58 smithi119 ceph-mon[32752]: pgmap v16744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:00:59.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:00:58 smithi167 ceph-mon[32751]: pgmap v16744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:01:01.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:00 smithi167 ceph-mon[32751]: pgmap v16745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:01.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:00 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30342.453948 secs 2024-04-27T03:01:01.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:00 smithi090 ceph-mon[29180]: pgmap v16745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:01.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:00 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30342.453948 secs 2024-04-27T03:01:01.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:00 smithi119 ceph-mon[32752]: pgmap v16745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:01.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:00 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30342.453948 secs 2024-04-27T03:01:03.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30343.300690 secs 2024-04-27T03:01:03.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:02 smithi167 ceph-mon[32751]: pgmap v16746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:01:03.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30343.300690 secs 2024-04-27T03:01:03.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:02 smithi090 ceph-mon[29180]: pgmap v16746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:01:03.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30343.300690 secs 2024-04-27T03:01:03.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:02 smithi119 ceph-mon[32752]: pgmap v16746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:01:04.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:04 smithi090 ceph-mon[29180]: pgmap v16747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:05.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:04 smithi119 ceph-mon[32752]: pgmap v16747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:05.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:04 smithi167 ceph-mon[32751]: pgmap v16747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:05 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30347.454030 secs 2024-04-27T03:01:06.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:05 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30347.454030 secs 2024-04-27T03:01:06.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:05 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30347.454030 secs 2024-04-27T03:01:07.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:06 smithi090 ceph-mon[29180]: pgmap v16748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:07.210 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:06 smithi119 ceph-mon[32752]: pgmap v16748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:07.224 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:06 smithi167 ceph-mon[32751]: pgmap v16748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:08.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30348.300833 secs 2024-04-27T03:01:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30348.300833 secs 2024-04-27T03:01:08.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30348.300833 secs 2024-04-27T03:01:09.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:08 smithi090 ceph-mon[29180]: pgmap v16749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:08 smithi119 ceph-mon[32752]: pgmap v16749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:08 smithi167 ceph-mon[32751]: pgmap v16749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:11.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:10 smithi090 ceph-mon[29180]: pgmap v16750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:11.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:10 smithi090 ceph-mon[29180]: 40 slow requests, 1 included below; oldest blocked for > 30352.454116 secs 2024-04-27T03:01:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:10 smithi090 ceph-mon[29180]: slow request 361.145720 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:01:11.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:10 smithi119 ceph-mon[32752]: pgmap v16750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:11.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:10 smithi119 ceph-mon[32752]: 40 slow requests, 1 included below; oldest blocked for > 30352.454116 secs 2024-04-27T03:01:11.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:10 smithi119 ceph-mon[32752]: slow request 361.145720 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:01:11.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:10 smithi167 ceph-mon[32751]: pgmap v16750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:11.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:10 smithi167 ceph-mon[32751]: 40 slow requests, 1 included below; oldest blocked for > 30352.454116 secs 2024-04-27T03:01:11.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:10 smithi167 ceph-mon[32751]: slow request 361.145720 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:01:13.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30353.300950 secs 2024-04-27T03:01:13.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:12 smithi090 ceph-mon[29180]: pgmap v16751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:13.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30353.300950 secs 2024-04-27T03:01:13.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:12 smithi119 ceph-mon[32752]: pgmap v16751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:13.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30353.300950 secs 2024-04-27T03:01:13.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:12 smithi167 ceph-mon[32751]: pgmap v16751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:01:15.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:14 smithi090 ceph-mon[29180]: pgmap v16752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:15.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:14 smithi119 ceph-mon[32752]: pgmap v16752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:15.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:14 smithi167 ceph-mon[32751]: pgmap v16752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:01:16.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:15 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30357.454199 secs 2024-04-27T03:01:16.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:15 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30357.454199 secs 2024-04-27T03:01:16.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:15 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30357.454199 secs 2024-04-27T03:01:17.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:16 smithi090 ceph-mon[29180]: pgmap v16753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:16 smithi119 ceph-mon[32752]: pgmap v16753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:16 smithi167 ceph-mon[32751]: pgmap v16753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:18.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30358.301062 secs 2024-04-27T03:01:18.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30358.301062 secs 2024-04-27T03:01:18.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30358.301062 secs 2024-04-27T03:01:19.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:18 smithi090 ceph-mon[29180]: pgmap v16754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:19.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:18 smithi119 ceph-mon[32752]: pgmap v16754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:19.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:18 smithi167 ceph-mon[32751]: pgmap v16754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:20 smithi090 ceph-mon[29180]: pgmap v16755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:20 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30362.454260 secs 2024-04-27T03:01:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:20 smithi119 ceph-mon[32752]: pgmap v16755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:20 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30362.454260 secs 2024-04-27T03:01:21.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:20 smithi167 ceph-mon[32751]: pgmap v16755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:21.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:20 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30362.454260 secs 2024-04-27T03:01:23.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30363.301133 secs 2024-04-27T03:01:23.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:22 smithi090 ceph-mon[29180]: pgmap v16756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:23.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30363.301133 secs 2024-04-27T03:01:23.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:22 smithi119 ceph-mon[32752]: pgmap v16756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:23.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30363.301133 secs 2024-04-27T03:01:23.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:22 smithi167 ceph-mon[32751]: pgmap v16756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:25.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:24 smithi090 ceph-mon[29180]: pgmap v16757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:25.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:24 smithi119 ceph-mon[32752]: pgmap v16757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:25.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:24 smithi167 ceph-mon[32751]: pgmap v16757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:26.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:25 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30367.454320 secs 2024-04-27T03:01:26.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:25 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30367.454320 secs 2024-04-27T03:01:26.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:25 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30367.454320 secs 2024-04-27T03:01:27.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:26 smithi119 ceph-mon[32752]: pgmap v16758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:27.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30368.301273 secs 2024-04-27T03:01:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:26 smithi167 ceph-mon[32751]: pgmap v16758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30368.301273 secs 2024-04-27T03:01:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:26 smithi090 ceph-mon[29180]: pgmap v16758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:27.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30368.301273 secs 2024-04-27T03:01:29.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:28 smithi119 ceph-mon[32752]: pgmap v16759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:29.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:28 smithi167 ceph-mon[32751]: pgmap v16759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:29.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:28 smithi090 ceph-mon[29180]: pgmap v16759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:30.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:30 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30372.454405 secs 2024-04-27T03:01:30.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:30 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30372.454405 secs 2024-04-27T03:01:31.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:30 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30372.454405 secs 2024-04-27T03:01:31.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:31 smithi167 ceph-mon[32751]: pgmap v16760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:31.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:31 smithi090 ceph-mon[29180]: pgmap v16760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:32.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:31 smithi119 ceph-mon[32752]: pgmap v16760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:32.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30373.301412 secs 2024-04-27T03:01:33.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30373.301412 secs 2024-04-27T03:01:33.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30373.301412 secs 2024-04-27T03:01:33.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:33 smithi090 ceph-mon[29180]: pgmap v16761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:34.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:33 smithi119 ceph-mon[32752]: pgmap v16761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:33 smithi167 ceph-mon[32751]: pgmap v16761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:35 smithi090 ceph-mon[29180]: pgmap v16762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:35 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30377.454481 secs 2024-04-27T03:01:36.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:35 smithi119 ceph-mon[32752]: pgmap v16762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:36.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:35 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30377.454481 secs 2024-04-27T03:01:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:35 smithi167 ceph-mon[32751]: pgmap v16762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:35 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30377.454481 secs 2024-04-27T03:01:37.860 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:37 smithi119 ceph-mon[32752]: pgmap v16763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:37.861 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30378.301507 secs 2024-04-27T03:01:37.874 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:37 smithi167 ceph-mon[32751]: pgmap v16763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:37.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30378.301507 secs 2024-04-27T03:01:37.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:37 smithi090 ceph-mon[29180]: pgmap v16763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:37.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30378.301507 secs 2024-04-27T03:01:39.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:39 smithi090 ceph-mon[29180]: pgmap v16764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:40.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:39 smithi119 ceph-mon[32752]: pgmap v16764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:40.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:39 smithi167 ceph-mon[32751]: pgmap v16764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:40 smithi090 ceph-mon[29180]: 40 slow requests, 1 included below; oldest blocked for > 30382.454565 secs 2024-04-27T03:01:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:40 smithi090 ceph-mon[29180]: slow request 5762.623176 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:01:41.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:40 smithi119 ceph-mon[32752]: 40 slow requests, 1 included below; oldest blocked for > 30382.454565 secs 2024-04-27T03:01:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:40 smithi119 ceph-mon[32752]: slow request 5762.623176 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:01:41.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:40 smithi167 ceph-mon[32751]: 40 slow requests, 1 included below; oldest blocked for > 30382.454565 secs 2024-04-27T03:01:41.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:40 smithi167 ceph-mon[32751]: slow request 5762.623176 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:01:41.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:41 smithi090 ceph-mon[29180]: pgmap v16765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:41 smithi119 ceph-mon[32752]: pgmap v16765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:42.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:41 smithi167 ceph-mon[32751]: pgmap v16765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:42.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30383.301615 secs 2024-04-27T03:01:42.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:42 smithi090 ceph-mon[29180]: pgmap v16766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:43.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30383.301615 secs 2024-04-27T03:01:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:42 smithi119 ceph-mon[32752]: pgmap v16766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:43.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30383.301615 secs 2024-04-27T03:01:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:42 smithi167 ceph-mon[32751]: pgmap v16766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:45.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:44 smithi090 ceph-mon[29180]: pgmap v16767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:45.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:44 smithi119 ceph-mon[32752]: pgmap v16767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:45.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:44 smithi167 ceph-mon[32751]: pgmap v16767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:46.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:45 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30387.454640 secs 2024-04-27T03:01:46.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:45 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30387.454640 secs 2024-04-27T03:01:46.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:45 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30387.454640 secs 2024-04-27T03:01:47.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:46 smithi090 ceph-mon[29180]: pgmap v16768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:46 smithi119 ceph-mon[32752]: pgmap v16768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:47.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:46 smithi167 ceph-mon[32751]: pgmap v16768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:48.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30388.301681 secs 2024-04-27T03:01:48.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30388.301681 secs 2024-04-27T03:01:48.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30388.301681 secs 2024-04-27T03:01:49.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:48 smithi090 ceph-mon[29180]: pgmap v16769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:49.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:48 smithi119 ceph-mon[32752]: pgmap v16769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:49.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:48 smithi167 ceph-mon[32751]: pgmap v16769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:51.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:50 smithi090 ceph-mon[29180]: pgmap v16770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:50 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30392.454723 secs 2024-04-27T03:01:51.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:50 smithi119 ceph-mon[32752]: pgmap v16770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:51.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:50 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30392.454723 secs 2024-04-27T03:01:51.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:50 smithi167 ceph-mon[32751]: pgmap v16770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:01:51.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:50 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30392.454723 secs 2024-04-27T03:01:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30393.301757 secs 2024-04-27T03:01:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: pgmap v16771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:01:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:01:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:01:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: pgmap v16772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:01:53.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: pgmap v16773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:01:53.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:01:53.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30393.301757 secs 2024-04-27T03:01:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: pgmap v16771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:01:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:01:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:01:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: pgmap v16772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:01:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: pgmap v16773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:01:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:01:53.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30393.301757 secs 2024-04-27T03:01:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: pgmap v16771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:01:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:01:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:01:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:01:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: pgmap v16772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:01:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: pgmap v16773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:01:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:01:55.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:54 smithi119 ceph-mon[32752]: pgmap v16774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:55.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:54 smithi167 ceph-mon[32751]: pgmap v16774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:55.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:54 smithi090 ceph-mon[29180]: pgmap v16774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:56.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:55 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30397.454806 secs 2024-04-27T03:01:56.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:55 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30397.454806 secs 2024-04-27T03:01:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:55 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30397.454806 secs 2024-04-27T03:01:57.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:56 smithi119 ceph-mon[32752]: pgmap v16775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:56 smithi167 ceph-mon[32751]: pgmap v16775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:56 smithi090 ceph-mon[29180]: pgmap v16775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:58.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30398.301878 secs 2024-04-27T03:01:58.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30398.301878 secs 2024-04-27T03:01:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30398.301878 secs 2024-04-27T03:01:59.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:01:58 smithi119 ceph-mon[32752]: pgmap v16776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:59.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:01:58 smithi167 ceph-mon[32751]: pgmap v16776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:01:59.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:01:58 smithi090 ceph-mon[29180]: pgmap v16776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:00.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:00 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30402.454891 secs 2024-04-27T03:02:00.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:00 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30402.454891 secs 2024-04-27T03:02:00.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:00 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30402.454891 secs 2024-04-27T03:02:01.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:01 smithi119 ceph-mon[32752]: pgmap v16777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:01.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:01 smithi167 ceph-mon[32751]: pgmap v16777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:01.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:01 smithi090 ceph-mon[29180]: pgmap v16777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:02.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30403.302003 secs 2024-04-27T03:02:02.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30403.302003 secs 2024-04-27T03:02:02.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30403.302003 secs 2024-04-27T03:02:03.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:03 smithi119 ceph-mon[32752]: pgmap v16778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:02:03.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:03 smithi167 ceph-mon[32751]: pgmap v16778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:02:03.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:03 smithi090 ceph-mon[29180]: pgmap v16778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:02:05.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:04 smithi119 ceph-mon[32752]: pgmap v16779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:05.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:04 smithi167 ceph-mon[32751]: pgmap v16779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:05.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:04 smithi090 ceph-mon[29180]: pgmap v16779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:06.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:05 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30407.454950 secs 2024-04-27T03:02:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:05 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30407.454950 secs 2024-04-27T03:02:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:05 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30407.454950 secs 2024-04-27T03:02:06.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:06 smithi090 ceph-mon[29180]: pgmap v16780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:07.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:06 smithi119 ceph-mon[32752]: pgmap v16780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:06 smithi167 ceph-mon[32751]: pgmap v16780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:07.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30408.302111 secs 2024-04-27T03:02:07.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30408.302111 secs 2024-04-27T03:02:08.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30408.302111 secs 2024-04-27T03:02:09.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:08 smithi119 ceph-mon[32752]: pgmap v16781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:09.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:08 smithi167 ceph-mon[32751]: pgmap v16781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:09.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:08 smithi090 ceph-mon[29180]: pgmap v16781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:10.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:10 smithi119 ceph-mon[32752]: 40 slow requests, 1 included below; oldest blocked for > 30412.455034 secs 2024-04-27T03:02:10.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:10 smithi119 ceph-mon[32752]: slow request 11524.216783 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:02:10.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:10 smithi167 ceph-mon[32751]: 40 slow requests, 1 included below; oldest blocked for > 30412.455034 secs 2024-04-27T03:02:10.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:10 smithi167 ceph-mon[32751]: slow request 11524.216783 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:02:10.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:10 smithi090 ceph-mon[29180]: 40 slow requests, 1 included below; oldest blocked for > 30412.455034 secs 2024-04-27T03:02:10.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:10 smithi090 ceph-mon[29180]: slow request 11524.216783 seconds old, received at 2024-04-26T23:50:05.991655+0000: client_request(client.17463:1 getattr pAsLsXsFs #0x1 2024-04-26T23:50:05.988144+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:02:11.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:11 smithi119 ceph-mon[32752]: pgmap v16782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:11.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:11 smithi167 ceph-mon[32751]: pgmap v16782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:11.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:11 smithi090 ceph-mon[29180]: pgmap v16782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:12.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:12 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 30413.302225 secs 2024-04-27T03:02:12.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:12 smithi119 ceph-mon[32752]: slow request 722.045655 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:02:12.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:12 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 30413.302225 secs 2024-04-27T03:02:12.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:12 smithi167 ceph-mon[32751]: slow request 722.045655 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:02:12.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:12 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 30413.302225 secs 2024-04-27T03:02:12.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:12 smithi090 ceph-mon[29180]: slow request 722.045655 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:02:13.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:13 smithi119 ceph-mon[32752]: pgmap v16783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:13.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:13 smithi167 ceph-mon[32751]: pgmap v16783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:13.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:13 smithi090 ceph-mon[29180]: pgmap v16783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:15.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:14 smithi119 ceph-mon[32752]: pgmap v16784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:15.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:14 smithi167 ceph-mon[32751]: pgmap v16784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:15.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:14 smithi090 ceph-mon[29180]: pgmap v16784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:16.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:15 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30417.455109 secs 2024-04-27T03:02:16.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:15 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30417.455109 secs 2024-04-27T03:02:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:15 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30417.455109 secs 2024-04-27T03:02:17.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:16 smithi119 ceph-mon[32752]: pgmap v16785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:16 smithi167 ceph-mon[32751]: pgmap v16785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:16 smithi090 ceph-mon[29180]: pgmap v16785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:18.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30418.302335 secs 2024-04-27T03:02:18.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30418.302335 secs 2024-04-27T03:02:18.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30418.302335 secs 2024-04-27T03:02:19.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:18 smithi119 ceph-mon[32752]: pgmap v16786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:19.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:18 smithi167 ceph-mon[32751]: pgmap v16786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:19.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:18 smithi090 ceph-mon[29180]: pgmap v16786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:20.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:20 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30422.455193 secs 2024-04-27T03:02:20.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:20 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30422.455193 secs 2024-04-27T03:02:20.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:20 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30422.455193 secs 2024-04-27T03:02:21.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:21 smithi119 ceph-mon[32752]: pgmap v16787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:21.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:21 smithi167 ceph-mon[32751]: pgmap v16787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:21.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:21 smithi090 ceph-mon[29180]: pgmap v16787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:22.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30423.302497 secs 2024-04-27T03:02:22.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30423.302497 secs 2024-04-27T03:02:22.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30423.302497 secs 2024-04-27T03:02:23.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:23 smithi119 ceph-mon[32752]: pgmap v16788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:23.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:23 smithi167 ceph-mon[32751]: pgmap v16788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:23.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:23 smithi090 ceph-mon[29180]: pgmap v16788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:25.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:24 smithi119 ceph-mon[32752]: pgmap v16789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:25.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:24 smithi167 ceph-mon[32751]: pgmap v16789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:25.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:24 smithi090 ceph-mon[29180]: pgmap v16789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:25 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30427.455272 secs 2024-04-27T03:02:26.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:25 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30427.455272 secs 2024-04-27T03:02:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:25 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30427.455272 secs 2024-04-27T03:02:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:26 smithi119 ceph-mon[32752]: pgmap v16790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:26 smithi167 ceph-mon[32751]: pgmap v16790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:26 smithi090 ceph-mon[29180]: pgmap v16790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:28.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30428.302649 secs 2024-04-27T03:02:28.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30428.302649 secs 2024-04-27T03:02:28.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30428.302649 secs 2024-04-27T03:02:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:28 smithi119 ceph-mon[32752]: pgmap v16791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:29.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:28 smithi167 ceph-mon[32751]: pgmap v16791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:29.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:28 smithi090 ceph-mon[29180]: pgmap v16791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:30.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:30 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30432.455355 secs 2024-04-27T03:02:30.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:30 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30432.455355 secs 2024-04-27T03:02:30.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:30 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30432.455355 secs 2024-04-27T03:02:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:31 smithi119 ceph-mon[32752]: pgmap v16792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:31 smithi167 ceph-mon[32751]: pgmap v16792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:31.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:31 smithi090 ceph-mon[29180]: pgmap v16792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:32.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30433.302791 secs 2024-04-27T03:02:32.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30433.302791 secs 2024-04-27T03:02:32.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30433.302791 secs 2024-04-27T03:02:33.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:33 smithi119 ceph-mon[32752]: pgmap v16793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:33.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:33 smithi167 ceph-mon[32751]: pgmap v16793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:33 smithi090 ceph-mon[29180]: pgmap v16793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:34 smithi119 ceph-mon[32752]: pgmap v16794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:34 smithi167 ceph-mon[32751]: pgmap v16794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:34 smithi090 ceph-mon[29180]: pgmap v16794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:35 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30437.455439 secs 2024-04-27T03:02:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:35 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30437.455439 secs 2024-04-27T03:02:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:35 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30437.455439 secs 2024-04-27T03:02:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:36 smithi119 ceph-mon[32752]: pgmap v16795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:36 smithi167 ceph-mon[32751]: pgmap v16795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:36 smithi090 ceph-mon[29180]: pgmap v16795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:37.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30438.302943 secs 2024-04-27T03:02:37.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30438.302943 secs 2024-04-27T03:02:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30438.302943 secs 2024-04-27T03:02:39.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:38 smithi119 ceph-mon[32752]: pgmap v16796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:39.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:38 smithi167 ceph-mon[32751]: pgmap v16796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:39.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:38 smithi090 ceph-mon[29180]: pgmap v16796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:40.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:40 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30442.455528 secs 2024-04-27T03:02:40.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:40 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30442.455528 secs 2024-04-27T03:02:40.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:40 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30442.455528 secs 2024-04-27T03:02:41.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:41 smithi119 ceph-mon[32752]: pgmap v16797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:41.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:41 smithi167 ceph-mon[32751]: pgmap v16797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:41.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:41 smithi090 ceph-mon[29180]: pgmap v16797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:42.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30443.303016 secs 2024-04-27T03:02:42.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30443.303016 secs 2024-04-27T03:02:42.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30443.303016 secs 2024-04-27T03:02:43.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:43 smithi119 ceph-mon[32752]: pgmap v16798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:43.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:43 smithi167 ceph-mon[32751]: pgmap v16798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:43.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:43 smithi090 ceph-mon[29180]: pgmap v16798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:45.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:44 smithi119 ceph-mon[32752]: pgmap v16799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:45.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:44 smithi167 ceph-mon[32751]: pgmap v16799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:45.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:44 smithi090 ceph-mon[29180]: pgmap v16799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:45 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30447.455613 secs 2024-04-27T03:02:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:45 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30447.455613 secs 2024-04-27T03:02:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:45 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30447.455613 secs 2024-04-27T03:02:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:46 smithi119 ceph-mon[32752]: pgmap v16800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:46 smithi167 ceph-mon[32751]: pgmap v16800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:46 smithi090 ceph-mon[29180]: pgmap v16800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:48.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30448.303158 secs 2024-04-27T03:02:48.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30448.303158 secs 2024-04-27T03:02:48.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30448.303158 secs 2024-04-27T03:02:49.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:48 smithi119 ceph-mon[32752]: pgmap v16801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:49.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:48 smithi167 ceph-mon[32751]: pgmap v16801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:49.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:48 smithi090 ceph-mon[29180]: pgmap v16801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:02:50.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:50 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30452.455697 secs 2024-04-27T03:02:50.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:50 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30452.455697 secs 2024-04-27T03:02:50.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:50 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30452.455697 secs 2024-04-27T03:02:51.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:51 smithi119 ceph-mon[32752]: pgmap v16802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:51.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:51 smithi167 ceph-mon[32751]: pgmap v16802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:51.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:51 smithi090 ceph-mon[29180]: pgmap v16802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:52.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30453.303311 secs 2024-04-27T03:02:52.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30453.303311 secs 2024-04-27T03:02:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30453.303311 secs 2024-04-27T03:02:53.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:53 smithi119 ceph-mon[32752]: pgmap v16803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:53.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:02:53.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:53 smithi167 ceph-mon[32751]: pgmap v16803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:53.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:02:53.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:53 smithi090 ceph-mon[29180]: pgmap v16803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:02:53.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:02:54.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:02:54.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:02:54.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:54 smithi119 ceph-mon[32752]: pgmap v16804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:02:54.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:54 smithi119 ceph-mon[32752]: pgmap v16805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:54.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:02:54.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:02:54.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:02:54.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:54 smithi167 ceph-mon[32751]: pgmap v16804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:02:54.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:54 smithi167 ceph-mon[32751]: pgmap v16805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:54.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:02:54.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:02:54.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:02:54.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:54 smithi090 ceph-mon[29180]: pgmap v16804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:02:54.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:54 smithi090 ceph-mon[29180]: pgmap v16805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:54.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:02:55.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:55 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30457.455781 secs 2024-04-27T03:02:55.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:55 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30457.455781 secs 2024-04-27T03:02:55.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:55 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30457.455781 secs 2024-04-27T03:02:56.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:56 smithi167 ceph-mon[32751]: pgmap v16806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:56.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:56 smithi090 ceph-mon[29180]: pgmap v16806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:56.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:56 smithi119 ceph-mon[32752]: pgmap v16806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:02:57.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30458.303408 secs 2024-04-27T03:02:57.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30458.303408 secs 2024-04-27T03:02:57.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30458.303408 secs 2024-04-27T03:02:58.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:02:58 smithi167 ceph-mon[32751]: pgmap v16807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:02:58.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:02:58 smithi090 ceph-mon[29180]: pgmap v16807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:02:58.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:02:58 smithi119 ceph-mon[32752]: pgmap v16807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:03:00.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:00 smithi090 ceph-mon[29180]: pgmap v16808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:03:00.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:00 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30462.455869 secs 2024-04-27T03:03:00.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:00 smithi119 ceph-mon[32752]: pgmap v16808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:03:00.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:00 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30462.455869 secs 2024-04-27T03:03:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:00 smithi167 ceph-mon[32751]: pgmap v16808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:03:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:00 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30462.455869 secs 2024-04-27T03:03:02.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30463.303513 secs 2024-04-27T03:03:02.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:02 smithi090 ceph-mon[29180]: pgmap v16809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:03:02.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30463.303513 secs 2024-04-27T03:03:02.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:02 smithi119 ceph-mon[32752]: pgmap v16809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:03:02.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30463.303513 secs 2024-04-27T03:03:02.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:02 smithi167 ceph-mon[32751]: pgmap v16809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:03:04.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:04 smithi119 ceph-mon[32752]: pgmap v16810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:03:04.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:04 smithi167 ceph-mon[32751]: pgmap v16810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:03:04.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:04 smithi090 ceph-mon[29180]: pgmap v16810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:03:05.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:05 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30467.455926 secs 2024-04-27T03:03:05.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:05 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30467.455926 secs 2024-04-27T03:03:05.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:05 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30467.455926 secs 2024-04-27T03:03:06.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:06 smithi119 ceph-mon[32752]: pgmap v16811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:06.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:06 smithi167 ceph-mon[32751]: pgmap v16811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:06.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:06 smithi090 ceph-mon[29180]: pgmap v16811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:07.710 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30468.303634 secs 2024-04-27T03:03:07.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30468.303634 secs 2024-04-27T03:03:07.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30468.303634 secs 2024-04-27T03:03:08.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:08 smithi119 ceph-mon[32752]: pgmap v16812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:08.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:08 smithi167 ceph-mon[32751]: pgmap v16812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:08.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:08 smithi090 ceph-mon[29180]: pgmap v16812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:10.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:10 smithi119 ceph-mon[32752]: pgmap v16813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:10.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:10 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30472.456013 secs 2024-04-27T03:03:10.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:10 smithi167 ceph-mon[32751]: pgmap v16813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:10.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:10 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30472.456013 secs 2024-04-27T03:03:10.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:10 smithi090 ceph-mon[29180]: pgmap v16813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:10.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:10 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30472.456013 secs 2024-04-27T03:03:12.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30473.303757 secs 2024-04-27T03:03:12.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:12 smithi090 ceph-mon[29180]: pgmap v16814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30473.303757 secs 2024-04-27T03:03:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:12 smithi119 ceph-mon[32752]: pgmap v16814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:12.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30473.303757 secs 2024-04-27T03:03:12.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:12 smithi167 ceph-mon[32751]: pgmap v16814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:14.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:14 smithi119 ceph-mon[32752]: pgmap v16815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:14.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:14 smithi167 ceph-mon[32751]: pgmap v16815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:14.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:14 smithi090 ceph-mon[29180]: pgmap v16815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:15.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:15 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30477.456099 secs 2024-04-27T03:03:15.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:15 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30477.456099 secs 2024-04-27T03:03:15.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:15 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30477.456099 secs 2024-04-27T03:03:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:16 smithi119 ceph-mon[32752]: pgmap v16816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:16.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:16 smithi167 ceph-mon[32751]: pgmap v16816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:16.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:16 smithi090 ceph-mon[29180]: pgmap v16816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:17.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30478.303885 secs 2024-04-27T03:03:17.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30478.303885 secs 2024-04-27T03:03:17.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30478.303885 secs 2024-04-27T03:03:18.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:18 smithi119 ceph-mon[32752]: pgmap v16817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:18.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:18 smithi167 ceph-mon[32751]: pgmap v16817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:18.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:18 smithi090 ceph-mon[29180]: pgmap v16817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:20 smithi119 ceph-mon[32752]: pgmap v16818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:20 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30482.456187 secs 2024-04-27T03:03:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:20 smithi167 ceph-mon[32751]: pgmap v16818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:20 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30482.456187 secs 2024-04-27T03:03:20.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:20 smithi090 ceph-mon[29180]: pgmap v16818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:20 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30482.456187 secs 2024-04-27T03:03:22.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30483.303974 secs 2024-04-27T03:03:22.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:22 smithi119 ceph-mon[32752]: pgmap v16819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:22.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30483.303974 secs 2024-04-27T03:03:22.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:22 smithi167 ceph-mon[32751]: pgmap v16819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:22.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30483.303974 secs 2024-04-27T03:03:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:22 smithi090 ceph-mon[29180]: pgmap v16819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:24.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:24 smithi119 ceph-mon[32752]: pgmap v16820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:24.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:24 smithi167 ceph-mon[32751]: pgmap v16820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:24.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:24 smithi090 ceph-mon[29180]: pgmap v16820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:25.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:25 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30487.456245 secs 2024-04-27T03:03:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:25 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30487.456245 secs 2024-04-27T03:03:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:25 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30487.456245 secs 2024-04-27T03:03:26.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:26 smithi119 ceph-mon[32752]: pgmap v16821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:26 smithi167 ceph-mon[32751]: pgmap v16821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:26 smithi090 ceph-mon[29180]: pgmap v16821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:27.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30488.304101 secs 2024-04-27T03:03:27.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30488.304101 secs 2024-04-27T03:03:27.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30488.304101 secs 2024-04-27T03:03:28.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:28 smithi119 ceph-mon[32752]: pgmap v16822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:28.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:28 smithi167 ceph-mon[32751]: pgmap v16822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:28.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:28 smithi090 ceph-mon[29180]: pgmap v16822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:30 smithi119 ceph-mon[32752]: pgmap v16823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:30.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:30 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30492.456331 secs 2024-04-27T03:03:30.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:30 smithi167 ceph-mon[32751]: pgmap v16823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:30 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30492.456331 secs 2024-04-27T03:03:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:30 smithi090 ceph-mon[29180]: pgmap v16823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:30 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30492.456331 secs 2024-04-27T03:03:32.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30493.304235 secs 2024-04-27T03:03:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:32 smithi119 ceph-mon[32752]: pgmap v16824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30493.304235 secs 2024-04-27T03:03:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:32 smithi167 ceph-mon[32751]: pgmap v16824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:32.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30493.304235 secs 2024-04-27T03:03:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:32 smithi090 ceph-mon[29180]: pgmap v16824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:34.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:34 smithi119 ceph-mon[32752]: pgmap v16825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:34.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:34 smithi167 ceph-mon[32751]: pgmap v16825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:34.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:34 smithi090 ceph-mon[29180]: pgmap v16825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:35.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:35 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30497.456411 secs 2024-04-27T03:03:35.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:35 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30497.456411 secs 2024-04-27T03:03:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:35 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30497.456411 secs 2024-04-27T03:03:36.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:36 smithi119 ceph-mon[32752]: pgmap v16826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:36.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:36 smithi167 ceph-mon[32751]: pgmap v16826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:36 smithi090 ceph-mon[29180]: pgmap v16826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:37.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30498.304373 secs 2024-04-27T03:03:37.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30498.304373 secs 2024-04-27T03:03:37.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30498.304373 secs 2024-04-27T03:03:38.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:38 smithi119 ceph-mon[32752]: pgmap v16827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:38.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:38 smithi167 ceph-mon[32751]: pgmap v16827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:38.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:38 smithi090 ceph-mon[29180]: pgmap v16827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:40.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:40 smithi119 ceph-mon[32752]: pgmap v16828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:40.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:40 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30502.456500 secs 2024-04-27T03:03:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:40 smithi167 ceph-mon[32751]: pgmap v16828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:40 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30502.456500 secs 2024-04-27T03:03:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:40 smithi090 ceph-mon[29180]: pgmap v16828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:40 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30502.456500 secs 2024-04-27T03:03:42.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30503.304487 secs 2024-04-27T03:03:42.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:42 smithi119 ceph-mon[32752]: pgmap v16829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:42.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30503.304487 secs 2024-04-27T03:03:42.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:42 smithi167 ceph-mon[32751]: pgmap v16829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:42.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30503.304487 secs 2024-04-27T03:03:42.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:42 smithi090 ceph-mon[29180]: pgmap v16829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:44.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:44 smithi119 ceph-mon[32752]: pgmap v16830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:44.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:44 smithi167 ceph-mon[32751]: pgmap v16830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:44.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:44 smithi090 ceph-mon[29180]: pgmap v16830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:45.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:45 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30507.456598 secs 2024-04-27T03:03:45.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:45 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30507.456598 secs 2024-04-27T03:03:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:45 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30507.456598 secs 2024-04-27T03:03:46.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:46 smithi119 ceph-mon[32752]: pgmap v16831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:46.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:46 smithi167 ceph-mon[32751]: pgmap v16831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:46 smithi090 ceph-mon[29180]: pgmap v16831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:47.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30508.304558 secs 2024-04-27T03:03:47.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30508.304558 secs 2024-04-27T03:03:47.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30508.304558 secs 2024-04-27T03:03:48.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:48 smithi119 ceph-mon[32752]: pgmap v16832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:48.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:48 smithi167 ceph-mon[32751]: pgmap v16832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:48.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:48 smithi090 ceph-mon[29180]: pgmap v16832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:50 smithi119 ceph-mon[32752]: pgmap v16833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:50.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:50 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30512.456700 secs 2024-04-27T03:03:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:50 smithi167 ceph-mon[32751]: pgmap v16833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:50.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:50 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30512.456700 secs 2024-04-27T03:03:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:50 smithi090 ceph-mon[29180]: pgmap v16833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:50 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30512.456700 secs 2024-04-27T03:03:52.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30513.304622 secs 2024-04-27T03:03:52.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:52 smithi119 ceph-mon[32752]: pgmap v16834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:52.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30513.304622 secs 2024-04-27T03:03:52.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:52 smithi167 ceph-mon[32751]: pgmap v16834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30513.304622 secs 2024-04-27T03:03:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:52 smithi090 ceph-mon[29180]: pgmap v16834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:53.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:03:53.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:03:53.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:03:54.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:54 smithi167 ceph-mon[32751]: pgmap v16835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:54.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:54 smithi119 ceph-mon[32752]: pgmap v16835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:54.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:54 smithi090 ceph-mon[29180]: pgmap v16835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:03:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:55.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:55 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30517.456765 secs 2024-04-27T03:03:55.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:55 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30517.456765 secs 2024-04-27T03:03:55.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:55 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30517.456765 secs 2024-04-27T03:03:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: pgmap v16836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: pgmap v16836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: pgmap v16836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:03:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:56.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:56.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30518.304689 secs 2024-04-27T03:03:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:03:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:03:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:03:57.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:03:57.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30518.304689 secs 2024-04-27T03:03:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:03:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:03:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:03:57.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:03:57.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30518.304689 secs 2024-04-27T03:03:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:03:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:03:57.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:03:57.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:03:57.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:03:58.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:58 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:03:58.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:58 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:03:58.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:58 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:03:58.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:58 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:03:58.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:58 smithi119 ceph-mon[32752]: pgmap v16837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-27T03:03:58.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:03:58 smithi119 ceph-mon[32752]: pgmap v16838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:03:58.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:58 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:03:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:58 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:03:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:58 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:03:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:58 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:03:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:58 smithi167 ceph-mon[32751]: pgmap v16837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-27T03:03:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:03:58 smithi167 ceph-mon[32751]: pgmap v16838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:03:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:58 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:03:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:58 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:03:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:58 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:03:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:58 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:03:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:58 smithi090 ceph-mon[29180]: pgmap v16837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-27T03:03:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:03:58 smithi090 ceph-mon[29180]: pgmap v16838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:00.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:00 smithi119 ceph-mon[32752]: pgmap v16839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:00.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:00 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30522.456855 secs 2024-04-27T03:04:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:00 smithi167 ceph-mon[32751]: pgmap v16839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:00 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30522.456855 secs 2024-04-27T03:04:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:00 smithi090 ceph-mon[29180]: pgmap v16839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:00 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30522.456855 secs 2024-04-27T03:04:02.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:02 smithi119 ceph-mon[32752]: pgmap v16840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:02.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30523.304803 secs 2024-04-27T03:04:02.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:02 smithi167 ceph-mon[32751]: pgmap v16840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:02.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30523.304803 secs 2024-04-27T03:04:02.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:02 smithi090 ceph-mon[29180]: pgmap v16840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:02.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30523.304803 secs 2024-04-27T03:04:04.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:04 smithi119 ceph-mon[32752]: pgmap v16841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 B/s rd, 1 op/s 2024-04-27T03:04:04.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:04 smithi167 ceph-mon[32751]: pgmap v16841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 B/s rd, 1 op/s 2024-04-27T03:04:04.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:04 smithi090 ceph-mon[29180]: pgmap v16841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 534 B/s rd, 1 op/s 2024-04-27T03:04:05.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:05 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30527.456914 secs 2024-04-27T03:04:05.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:05 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30527.456914 secs 2024-04-27T03:04:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:05 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30527.456914 secs 2024-04-27T03:04:06.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:06 smithi119 ceph-mon[32752]: pgmap v16842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:06.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:06 smithi167 ceph-mon[32751]: pgmap v16842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:06 smithi090 ceph-mon[29180]: pgmap v16842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:04:07.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30528.304920 secs 2024-04-27T03:04:07.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30528.304920 secs 2024-04-27T03:04:07.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30528.304920 secs 2024-04-27T03:04:08.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:08 smithi119 ceph-mon[32752]: pgmap v16843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:04:08.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:08 smithi167 ceph-mon[32751]: pgmap v16843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:04:08.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:08 smithi090 ceph-mon[29180]: pgmap v16843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:04:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:10 smithi119 ceph-mon[32752]: pgmap v16844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:10 smithi119 ceph-mon[32752]: 40 slow requests, 1 included below; oldest blocked for > 30532.457019 secs 2024-04-27T03:04:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:10 smithi119 ceph-mon[32752]: slow request 1441.384437 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:04:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:10 smithi167 ceph-mon[32751]: pgmap v16844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:10 smithi167 ceph-mon[32751]: 40 slow requests, 1 included below; oldest blocked for > 30532.457019 secs 2024-04-27T03:04:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:10 smithi167 ceph-mon[32751]: slow request 1441.384437 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:04:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:10 smithi090 ceph-mon[29180]: pgmap v16844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:10 smithi090 ceph-mon[29180]: 40 slow requests, 1 included below; oldest blocked for > 30532.457019 secs 2024-04-27T03:04:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:10 smithi090 ceph-mon[29180]: slow request 1441.384437 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:04:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:12 smithi167 ceph-mon[32751]: pgmap v16845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30533.305035 secs 2024-04-27T03:04:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:12 smithi090 ceph-mon[29180]: pgmap v16845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30533.305035 secs 2024-04-27T03:04:13.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:12 smithi119 ceph-mon[32752]: pgmap v16845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:13.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30533.305035 secs 2024-04-27T03:04:14.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:14 smithi167 ceph-mon[32751]: pgmap v16846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:14.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:14 smithi090 ceph-mon[29180]: pgmap v16846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:15.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:14 smithi119 ceph-mon[32752]: pgmap v16846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:15 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30537.457117 secs 2024-04-27T03:04:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:15 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30537.457117 secs 2024-04-27T03:04:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:15 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30537.457117 secs 2024-04-27T03:04:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:16 smithi090 ceph-mon[29180]: pgmap v16847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:17.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:16 smithi119 ceph-mon[32752]: pgmap v16847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:17.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:16 smithi167 ceph-mon[32751]: pgmap v16847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:17.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30538.305181 secs 2024-04-27T03:04:18.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30538.305181 secs 2024-04-27T03:04:18.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30538.305181 secs 2024-04-27T03:04:18.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:18 smithi119 ceph-mon[32752]: pgmap v16848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:18.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:18 smithi167 ceph-mon[32751]: pgmap v16848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:18.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:18 smithi090 ceph-mon[29180]: pgmap v16848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:20 smithi090 ceph-mon[29180]: pgmap v16849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:20 smithi090 ceph-mon[29180]: 40 slow requests, 1 included below; oldest blocked for > 30542.457221 secs 2024-04-27T03:04:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:20 smithi090 ceph-mon[29180]: slow request 23041.516516 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:04:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:20 smithi119 ceph-mon[32752]: pgmap v16849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:20 smithi119 ceph-mon[32752]: 40 slow requests, 1 included below; oldest blocked for > 30542.457221 secs 2024-04-27T03:04:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:20 smithi119 ceph-mon[32752]: slow request 23041.516516 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:04:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:20 smithi167 ceph-mon[32751]: pgmap v16849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:20 smithi167 ceph-mon[32751]: 40 slow requests, 1 included below; oldest blocked for > 30542.457221 secs 2024-04-27T03:04:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:20 smithi167 ceph-mon[32751]: slow request 23041.516516 seconds old, received at 2024-04-26T20:40:18.694109+0000: client_request(client.24733:267886 getattr AsXsFs #0x100000001fb 2024-04-26T20:40:18.692185+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:04:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:22 smithi090 ceph-mon[29180]: pgmap v16850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30543.305319 secs 2024-04-27T03:04:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:22 smithi119 ceph-mon[32752]: pgmap v16850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30543.305319 secs 2024-04-27T03:04:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:22 smithi167 ceph-mon[32751]: pgmap v16850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30543.305319 secs 2024-04-27T03:04:24.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:24 smithi090 ceph-mon[29180]: pgmap v16851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:24 smithi119 ceph-mon[32752]: pgmap v16851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:24 smithi167 ceph-mon[32751]: pgmap v16851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:25 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30547.457307 secs 2024-04-27T03:04:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:25 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30547.457307 secs 2024-04-27T03:04:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:25 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30547.457307 secs 2024-04-27T03:04:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:26 smithi090 ceph-mon[29180]: pgmap v16852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:26 smithi119 ceph-mon[32752]: pgmap v16852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:26 smithi167 ceph-mon[32751]: pgmap v16852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:27.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30548.305412 secs 2024-04-27T03:04:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30548.305412 secs 2024-04-27T03:04:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30548.305412 secs 2024-04-27T03:04:28.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:28 smithi090 ceph-mon[29180]: pgmap v16853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:28 smithi119 ceph-mon[32752]: pgmap v16853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:28 smithi167 ceph-mon[32751]: pgmap v16853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:30 smithi090 ceph-mon[29180]: pgmap v16854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:30 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30552.457409 secs 2024-04-27T03:04:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:30 smithi119 ceph-mon[32752]: pgmap v16854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:30 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30552.457409 secs 2024-04-27T03:04:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:30 smithi167 ceph-mon[32751]: pgmap v16854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:30 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30552.457409 secs 2024-04-27T03:04:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:32 smithi090 ceph-mon[29180]: pgmap v16855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30553.305487 secs 2024-04-27T03:04:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:32 smithi119 ceph-mon[32752]: pgmap v16855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30553.305487 secs 2024-04-27T03:04:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:32 smithi167 ceph-mon[32751]: pgmap v16855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30553.305487 secs 2024-04-27T03:04:34.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:34 smithi090 ceph-mon[29180]: pgmap v16856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:35.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:34 smithi119 ceph-mon[32752]: pgmap v16856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:34 smithi167 ceph-mon[32751]: pgmap v16856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:35.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:35 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30557.457507 secs 2024-04-27T03:04:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:35 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30557.457507 secs 2024-04-27T03:04:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:35 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30557.457507 secs 2024-04-27T03:04:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:36 smithi090 ceph-mon[29180]: pgmap v16857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:37.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:36 smithi119 ceph-mon[32752]: pgmap v16857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:36 smithi167 ceph-mon[32751]: pgmap v16857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:37.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30558.305626 secs 2024-04-27T03:04:37.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:37 smithi090 ceph-mon[29180]: pgmap v16858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:37.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30558.305626 secs 2024-04-27T03:04:37.961 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:37 smithi119 ceph-mon[32752]: pgmap v16858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:37.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30558.305626 secs 2024-04-27T03:04:37.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:37 smithi167 ceph-mon[32751]: pgmap v16858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:40.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:39 smithi119 ceph-mon[32752]: pgmap v16859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:40.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:39 smithi167 ceph-mon[32751]: pgmap v16859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:40.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:39 smithi090 ceph-mon[29180]: pgmap v16859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:41.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:40 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30562.457610 secs 2024-04-27T03:04:41.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:40 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30562.457610 secs 2024-04-27T03:04:41.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:40 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30562.457610 secs 2024-04-27T03:04:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:41 smithi119 ceph-mon[32752]: pgmap v16860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30563.305786 secs 2024-04-27T03:04:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:41 smithi167 ceph-mon[32751]: pgmap v16860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30563.305786 secs 2024-04-27T03:04:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:41 smithi090 ceph-mon[29180]: pgmap v16860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30563.305786 secs 2024-04-27T03:04:44.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:43 smithi119 ceph-mon[32752]: pgmap v16861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:44.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:43 smithi167 ceph-mon[32751]: pgmap v16861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:44.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:43 smithi090 ceph-mon[29180]: pgmap v16861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:45 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30567.457669 secs 2024-04-27T03:04:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:45 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30567.457669 secs 2024-04-27T03:04:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:45 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30567.457669 secs 2024-04-27T03:04:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:46 smithi090 ceph-mon[29180]: pgmap v16862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:47.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:46 smithi119 ceph-mon[32752]: pgmap v16862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:47.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:46 smithi167 ceph-mon[32751]: pgmap v16862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:47.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30568.305917 secs 2024-04-27T03:04:48.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30568.305917 secs 2024-04-27T03:04:48.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30568.305917 secs 2024-04-27T03:04:48.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:48 smithi090 ceph-mon[29180]: pgmap v16863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:49.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:48 smithi119 ceph-mon[32752]: pgmap v16863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:49.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:48 smithi167 ceph-mon[32751]: pgmap v16863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:50 smithi090 ceph-mon[29180]: pgmap v16864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:50 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30572.457772 secs 2024-04-27T03:04:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:50 smithi119 ceph-mon[32752]: pgmap v16864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:50 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30572.457772 secs 2024-04-27T03:04:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:50 smithi167 ceph-mon[32751]: pgmap v16864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:50 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30572.457772 secs 2024-04-27T03:04:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:51 smithi119 ceph-mon[32752]: pgmap v16865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:51 smithi167 ceph-mon[32751]: pgmap v16865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:51 smithi090 ceph-mon[29180]: pgmap v16865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:53.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30573.306034 secs 2024-04-27T03:04:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30573.306034 secs 2024-04-27T03:04:53.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30573.306034 secs 2024-04-27T03:04:54.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:53 smithi119 ceph-mon[32752]: pgmap v16866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:54.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:53 smithi167 ceph-mon[32751]: pgmap v16866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:54.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:53 smithi090 ceph-mon[29180]: pgmap v16866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:56.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:55 smithi119 ceph-mon[32752]: pgmap v16867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:56.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:55 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30577.457858 secs 2024-04-27T03:04:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:55 smithi167 ceph-mon[32751]: pgmap v16867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:55 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30577.457858 secs 2024-04-27T03:04:56.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:55 smithi090 ceph-mon[29180]: pgmap v16867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:04:56.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:55 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30577.457858 secs 2024-04-27T03:04:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30578.306171 secs 2024-04-27T03:04:57.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30578.306171 secs 2024-04-27T03:04:57.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30578.306171 secs 2024-04-27T03:04:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:57 smithi119 ceph-mon[32752]: pgmap v16868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:04:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:57 smithi167 ceph-mon[32751]: pgmap v16868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:04:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:57 smithi090 ceph-mon[29180]: pgmap v16868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:04:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:04:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:59.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:04:59.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:04:59.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:58 smithi119 ceph-mon[32752]: pgmap v16869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T03:04:59.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:58 smithi119 ceph-mon[32752]: pgmap v16870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T03:04:59.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:04:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:04:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:04:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:58 smithi167 ceph-mon[32751]: pgmap v16869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T03:04:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:58 smithi167 ceph-mon[32751]: pgmap v16870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T03:04:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:04:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:04:59.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:04:59.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:04:59.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:58 smithi090 ceph-mon[29180]: pgmap v16869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 907 B/s rd, 1 op/s 2024-04-27T03:04:59.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:58 smithi090 ceph-mon[29180]: pgmap v16870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T03:04:59.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:04:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:05:00.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:00 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30582.457965 secs 2024-04-27T03:05:00.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:00 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30582.457965 secs 2024-04-27T03:05:00.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:00 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30582.457965 secs 2024-04-27T03:05:01.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:01 smithi119 ceph-mon[32752]: pgmap v16871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:01.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:01 smithi167 ceph-mon[32751]: pgmap v16871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:01.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:01 smithi090 ceph-mon[29180]: pgmap v16871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:02.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30583.306292 secs 2024-04-27T03:05:02.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30583.306292 secs 2024-04-27T03:05:02.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30583.306292 secs 2024-04-27T03:05:03.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:03 smithi167 ceph-mon[32751]: pgmap v16872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:03.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:03 smithi090 ceph-mon[29180]: pgmap v16872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:03.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:03 smithi119 ceph-mon[32752]: pgmap v16872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:05.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:05 smithi167 ceph-mon[32751]: pgmap v16873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:05.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:05 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30587.458023 secs 2024-04-27T03:05:05.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:05 smithi090 ceph-mon[29180]: pgmap v16873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:05.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:05 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30587.458023 secs 2024-04-27T03:05:05.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:05 smithi119 ceph-mon[32752]: pgmap v16873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:05.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:05 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30587.458023 secs 2024-04-27T03:05:07.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30588.306412 secs 2024-04-27T03:05:07.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:07 smithi167 ceph-mon[32751]: pgmap v16874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:07.612 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30588.306412 secs 2024-04-27T03:05:07.612 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:07 smithi119 ceph-mon[32752]: pgmap v16874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:07.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30588.306412 secs 2024-04-27T03:05:07.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:07 smithi090 ceph-mon[29180]: pgmap v16874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:05:09.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:09 smithi167 ceph-mon[32751]: pgmap v16875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:05:09.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:09 smithi090 ceph-mon[29180]: pgmap v16875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:05:09.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:09 smithi119 ceph-mon[32752]: pgmap v16875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:05:10.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:10 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30592.458110 secs 2024-04-27T03:05:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:10 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30592.458110 secs 2024-04-27T03:05:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:10 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30592.458110 secs 2024-04-27T03:05:11.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:11 smithi090 ceph-mon[29180]: pgmap v16876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:11 smithi119 ceph-mon[32752]: pgmap v16876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:11 smithi167 ceph-mon[32751]: pgmap v16876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:12.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30593.306508 secs 2024-04-27T03:05:12.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30593.306508 secs 2024-04-27T03:05:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30593.306508 secs 2024-04-27T03:05:13.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:13 smithi090 ceph-mon[29180]: pgmap v16877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:13.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:13 smithi119 ceph-mon[32752]: pgmap v16877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:13.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:13 smithi167 ceph-mon[32751]: pgmap v16877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:15.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:15 smithi090 ceph-mon[29180]: pgmap v16878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:15.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:15 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30597.458187 secs 2024-04-27T03:05:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:15 smithi119 ceph-mon[32752]: pgmap v16878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:15 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30597.458187 secs 2024-04-27T03:05:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:15 smithi167 ceph-mon[32751]: pgmap v16878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:15 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30597.458187 secs 2024-04-27T03:05:17.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30598.306601 secs 2024-04-27T03:05:17.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:17 smithi090 ceph-mon[29180]: pgmap v16879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:17.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30598.306601 secs 2024-04-27T03:05:17.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:17 smithi119 ceph-mon[32752]: pgmap v16879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:17.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30598.306601 secs 2024-04-27T03:05:17.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:17 smithi167 ceph-mon[32751]: pgmap v16879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:19.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:19 smithi090 ceph-mon[29180]: pgmap v16880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:19.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:19 smithi119 ceph-mon[32752]: pgmap v16880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:19.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:19 smithi167 ceph-mon[32751]: pgmap v16880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:20.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:20 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30602.458248 secs 2024-04-27T03:05:20.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:20 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30602.458248 secs 2024-04-27T03:05:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:20 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30602.458248 secs 2024-04-27T03:05:21.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:21 smithi090 ceph-mon[29180]: pgmap v16881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:21.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:21 smithi119 ceph-mon[32752]: pgmap v16881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:21 smithi167 ceph-mon[32751]: pgmap v16881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:22.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30603.306742 secs 2024-04-27T03:05:22.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30603.306742 secs 2024-04-27T03:05:22.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30603.306742 secs 2024-04-27T03:05:23.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:23 smithi090 ceph-mon[29180]: pgmap v16882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:23.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:23 smithi119 ceph-mon[32752]: pgmap v16882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:23.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:23 smithi167 ceph-mon[32751]: pgmap v16882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:25.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:25 smithi090 ceph-mon[29180]: pgmap v16883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:25.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:25 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30607.458307 secs 2024-04-27T03:05:25.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:25 smithi119 ceph-mon[32752]: pgmap v16883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:25.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:25 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30607.458307 secs 2024-04-27T03:05:25.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:25 smithi167 ceph-mon[32751]: pgmap v16883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:25.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:25 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30607.458307 secs 2024-04-27T03:05:27.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30608.306851 secs 2024-04-27T03:05:27.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:27 smithi090 ceph-mon[29180]: pgmap v16884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:27.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30608.306851 secs 2024-04-27T03:05:27.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:27 smithi119 ceph-mon[32752]: pgmap v16884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:27.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30608.306851 secs 2024-04-27T03:05:27.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:27 smithi167 ceph-mon[32751]: pgmap v16884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:29.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:29 smithi090 ceph-mon[29180]: pgmap v16885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:29.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:29 smithi119 ceph-mon[32752]: pgmap v16885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:29.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:29 smithi167 ceph-mon[32751]: pgmap v16885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:30.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:30 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30612.458396 secs 2024-04-27T03:05:30.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:30 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30612.458396 secs 2024-04-27T03:05:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:30 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30612.458396 secs 2024-04-27T03:05:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:31 smithi090 ceph-mon[29180]: pgmap v16886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:31 smithi119 ceph-mon[32752]: pgmap v16886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:31.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:31 smithi167 ceph-mon[32751]: pgmap v16886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:32.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30613.306972 secs 2024-04-27T03:05:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30613.306972 secs 2024-04-27T03:05:32.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30613.306972 secs 2024-04-27T03:05:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:33 smithi090 ceph-mon[29180]: pgmap v16887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:33.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:33 smithi119 ceph-mon[32752]: pgmap v16887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:33 smithi167 ceph-mon[32751]: pgmap v16887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:35.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:35 smithi119 ceph-mon[32752]: pgmap v16888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:35.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:35 smithi119 ceph-mon[32752]: 40 slow requests, 0 included below; oldest blocked for > 30617.458465 secs 2024-04-27T03:05:35.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:35 smithi167 ceph-mon[32751]: pgmap v16888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:35.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:35 smithi167 ceph-mon[32751]: 40 slow requests, 0 included below; oldest blocked for > 30617.458465 secs 2024-04-27T03:05:35.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:35 smithi090 ceph-mon[29180]: pgmap v16888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:35.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:35 smithi090 ceph-mon[29180]: 40 slow requests, 0 included below; oldest blocked for > 30617.458465 secs 2024-04-27T03:05:37.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30618.307049 secs 2024-04-27T03:05:37.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:37 smithi119 ceph-mon[32752]: pgmap v16889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:37.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30618.307049 secs 2024-04-27T03:05:37.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:37 smithi167 ceph-mon[32751]: pgmap v16889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:37.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30618.307049 secs 2024-04-27T03:05:37.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:37 smithi090 ceph-mon[29180]: pgmap v16889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:39.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:39 smithi090 ceph-mon[29180]: pgmap v16890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:39.633 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:05:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:05:39.233+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:05:39.633 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:05:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:05:39.233+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:05:39.633 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:05:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:05:39.233+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:05:39.633 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:05:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:05:39.233+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:05:39.634 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:05:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:05:39.233+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:05:39.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:39 smithi119 ceph-mon[32752]: pgmap v16890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:39.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:39 smithi167 ceph-mon[32751]: pgmap v16890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:40.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30622.458523 secs 2024-04-27T03:05:40.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30622.458523 secs 2024-04-27T03:05:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30622.458523 secs 2024-04-27T03:05:41.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:41 smithi119 ceph-mon[32752]: pgmap v16891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:41.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:41 smithi167 ceph-mon[32751]: pgmap v16891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:41 smithi090 ceph-mon[29180]: pgmap v16891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:05:42.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30623.307151 secs 2024-04-27T03:05:42.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30623.307151 secs 2024-04-27T03:05:42.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30623.307151 secs 2024-04-27T03:05:43.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:43 smithi090 ceph-mon[29180]: pgmap v16892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:43.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:05:43.660 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:05:43.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:43 smithi119 ceph-mon[32752]: pgmap v16892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:43.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:05:43.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:05:43.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:43 smithi167 ceph-mon[32751]: pgmap v16892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:43.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:05:43.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:05:45.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:45 smithi119 ceph-mon[32752]: pgmap v16893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:45.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30627.458606 secs 2024-04-27T03:05:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:45 smithi167 ceph-mon[32751]: pgmap v16893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30627.458606 secs 2024-04-27T03:05:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:45 smithi090 ceph-mon[29180]: pgmap v16893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:45.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30627.458606 secs 2024-04-27T03:05:47.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30628.307286 secs 2024-04-27T03:05:47.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:47 smithi119 ceph-mon[32752]: pgmap v16894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:47.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30628.307286 secs 2024-04-27T03:05:47.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:47 smithi167 ceph-mon[32751]: pgmap v16894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:47.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30628.307286 secs 2024-04-27T03:05:47.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:47 smithi090 ceph-mon[29180]: pgmap v16894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:49.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:49 smithi119 ceph-mon[32752]: pgmap v16895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:49.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:49 smithi167 ceph-mon[32751]: pgmap v16895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:49.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:49 smithi090 ceph-mon[29180]: pgmap v16895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:50.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30632.458664 secs 2024-04-27T03:05:50.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30632.458664 secs 2024-04-27T03:05:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30632.458664 secs 2024-04-27T03:05:51.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:51 smithi119 ceph-mon[32752]: pgmap v16896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:51 smithi167 ceph-mon[32751]: pgmap v16896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:51 smithi090 ceph-mon[29180]: pgmap v16896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:52.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30633.307410 secs 2024-04-27T03:05:52.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30633.307410 secs 2024-04-27T03:05:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30633.307410 secs 2024-04-27T03:05:53.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:53 smithi119 ceph-mon[32752]: pgmap v16897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:53.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:53 smithi167 ceph-mon[32751]: pgmap v16897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:53.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:53 smithi090 ceph-mon[29180]: pgmap v16897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:55.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:55 smithi119 ceph-mon[32752]: pgmap v16898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:55.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30637.458740 secs 2024-04-27T03:05:55.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:55 smithi167 ceph-mon[32751]: pgmap v16898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:55.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30637.458740 secs 2024-04-27T03:05:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:55 smithi090 ceph-mon[29180]: pgmap v16898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:05:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30637.458740 secs 2024-04-27T03:05:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30638.307526 secs 2024-04-27T03:05:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:57 smithi119 ceph-mon[32752]: pgmap v16899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30638.307526 secs 2024-04-27T03:05:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:57 smithi167 ceph-mon[32751]: pgmap v16899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30638.307526 secs 2024-04-27T03:05:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:57 smithi090 ceph-mon[29180]: pgmap v16899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:58.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:05:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:05:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:05:59.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:59 smithi090 ceph-mon[29180]: pgmap v16900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:59.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:05:59.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:05:59.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:05:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:05:59.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:59 smithi119 ceph-mon[32752]: pgmap v16900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:59.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:05:59.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:05:59.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:05:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:05:59.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:59 smithi167 ceph-mon[32751]: pgmap v16900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:05:59.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:05:59.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:05:59.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:05:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:06:00.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:00 smithi119 ceph-mon[32752]: pgmap v16901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:06:00.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:00 smithi119 ceph-mon[32752]: pgmap v16902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:00.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30642.458825 secs 2024-04-27T03:06:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:00 smithi167 ceph-mon[32751]: pgmap v16901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:06:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:00 smithi167 ceph-mon[32751]: pgmap v16902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:00.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30642.458825 secs 2024-04-27T03:06:00.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:00 smithi090 ceph-mon[29180]: pgmap v16901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:06:00.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:00 smithi090 ceph-mon[29180]: pgmap v16902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:00.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30642.458825 secs 2024-04-27T03:06:02.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:02 smithi119 ceph-mon[32752]: pgmap v16903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:02.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30643.307633 secs 2024-04-27T03:06:02.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:02 smithi167 ceph-mon[32751]: pgmap v16903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:02.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30643.307633 secs 2024-04-27T03:06:02.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:02 smithi090 ceph-mon[29180]: pgmap v16903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:02.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30643.307633 secs 2024-04-27T03:06:04.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:04 smithi119 ceph-mon[32752]: pgmap v16904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:06:04.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:04 smithi167 ceph-mon[32751]: pgmap v16904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:06:04.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:04 smithi090 ceph-mon[29180]: pgmap v16904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:06:05.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30647.458883 secs 2024-04-27T03:06:05.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30647.458883 secs 2024-04-27T03:06:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30647.458883 secs 2024-04-27T03:06:06.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:06 smithi119 ceph-mon[32752]: pgmap v16905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:06.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:06 smithi167 ceph-mon[32751]: pgmap v16905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:06.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:06 smithi090 ceph-mon[29180]: pgmap v16905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:07.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30648.307726 secs 2024-04-27T03:06:07.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30648.307726 secs 2024-04-27T03:06:07.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30648.307726 secs 2024-04-27T03:06:08.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:08 smithi119 ceph-mon[32752]: pgmap v16906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:08.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:08 smithi167 ceph-mon[32751]: pgmap v16906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:08.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:08 smithi090 ceph-mon[29180]: pgmap v16906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:06:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:10 smithi119 ceph-mon[32752]: pgmap v16907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:06:10.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30652.458943 secs 2024-04-27T03:06:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:10 smithi167 ceph-mon[32751]: pgmap v16907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:06:10.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30652.458943 secs 2024-04-27T03:06:10.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:10 smithi090 ceph-mon[29180]: pgmap v16907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:06:10.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30652.458943 secs 2024-04-27T03:06:12.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:12 smithi119 ceph-mon[32752]: pgmap v16908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:12.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30653.307836 secs 2024-04-27T03:06:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:12 smithi167 ceph-mon[32751]: pgmap v16908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:12.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30653.307836 secs 2024-04-27T03:06:12.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:12 smithi090 ceph-mon[29180]: pgmap v16908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:12.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30653.307836 secs 2024-04-27T03:06:14.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:14 smithi119 ceph-mon[32752]: pgmap v16909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:14.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:14 smithi167 ceph-mon[32751]: pgmap v16909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:14.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:14 smithi090 ceph-mon[29180]: pgmap v16909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30657.459029 secs 2024-04-27T03:06:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30657.459029 secs 2024-04-27T03:06:15.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30657.459029 secs 2024-04-27T03:06:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:16 smithi119 ceph-mon[32752]: pgmap v16910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:16 smithi167 ceph-mon[32751]: pgmap v16910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:16.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:16 smithi090 ceph-mon[29180]: pgmap v16910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:17.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30658.307980 secs 2024-04-27T03:06:17.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30658.307980 secs 2024-04-27T03:06:17.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30658.307980 secs 2024-04-27T03:06:18.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:18 smithi119 ceph-mon[32752]: pgmap v16911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:18.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:18 smithi167 ceph-mon[32751]: pgmap v16911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:18.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:18 smithi090 ceph-mon[29180]: pgmap v16911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:20.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:20 smithi119 ceph-mon[32752]: pgmap v16912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:20.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30662.459117 secs 2024-04-27T03:06:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:20 smithi167 ceph-mon[32751]: pgmap v16912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:20.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30662.459117 secs 2024-04-27T03:06:20.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:20 smithi090 ceph-mon[29180]: pgmap v16912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:20.887 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30662.459117 secs 2024-04-27T03:06:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:22 smithi119 ceph-mon[32752]: pgmap v16913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30663.308043 secs 2024-04-27T03:06:22.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:22 smithi167 ceph-mon[32751]: pgmap v16913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:22.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30663.308043 secs 2024-04-27T03:06:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:22 smithi090 ceph-mon[29180]: pgmap v16913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30663.308043 secs 2024-04-27T03:06:24.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:24 smithi119 ceph-mon[32752]: pgmap v16914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:24.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:24 smithi167 ceph-mon[32751]: pgmap v16914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:24.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:24 smithi090 ceph-mon[29180]: pgmap v16914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30667.459199 secs 2024-04-27T03:06:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30667.459199 secs 2024-04-27T03:06:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30667.459199 secs 2024-04-27T03:06:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:26 smithi119 ceph-mon[32752]: pgmap v16915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:26 smithi167 ceph-mon[32751]: pgmap v16915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:26 smithi090 ceph-mon[29180]: pgmap v16915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30668.308167 secs 2024-04-27T03:06:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:27 smithi119 ceph-mon[32752]: pgmap v16916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30668.308167 secs 2024-04-27T03:06:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:27 smithi167 ceph-mon[32751]: pgmap v16916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30668.308167 secs 2024-04-27T03:06:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:27 smithi090 ceph-mon[29180]: pgmap v16916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:30.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:30 smithi119 ceph-mon[32752]: pgmap v16917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:30.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:30 smithi167 ceph-mon[32751]: pgmap v16917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:30.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:30 smithi090 ceph-mon[29180]: pgmap v16917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:31.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:31 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30672.459286 secs 2024-04-27T03:06:31.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:31 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30672.459286 secs 2024-04-27T03:06:31.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:31 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30672.459286 secs 2024-04-27T03:06:32.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:32 smithi167 ceph-mon[32751]: pgmap v16918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:32.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30673.308257 secs 2024-04-27T03:06:32.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:32 smithi090 ceph-mon[29180]: pgmap v16918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:32.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30673.308257 secs 2024-04-27T03:06:32.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:32 smithi119 ceph-mon[32752]: pgmap v16918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:32.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30673.308257 secs 2024-04-27T03:06:34.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:34 smithi167 ceph-mon[32751]: pgmap v16919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:34.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:34 smithi090 ceph-mon[29180]: pgmap v16919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:34.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:34 smithi119 ceph-mon[32752]: pgmap v16919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:36.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:36 smithi090 ceph-mon[29180]: pgmap v16920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:36.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:36 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30677.459345 secs 2024-04-27T03:06:36.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:36 smithi119 ceph-mon[32752]: pgmap v16920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:36.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:36 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30677.459345 secs 2024-04-27T03:06:36.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:36 smithi167 ceph-mon[32751]: pgmap v16920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:36.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:36 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30677.459345 secs 2024-04-27T03:06:37.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30678.308407 secs 2024-04-27T03:06:37.461 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30678.308407 secs 2024-04-27T03:06:37.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30678.308407 secs 2024-04-27T03:06:38.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:38 smithi090 ceph-mon[29180]: pgmap v16921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:38.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:38 smithi119 ceph-mon[32752]: pgmap v16921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:38.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:38 smithi167 ceph-mon[32751]: pgmap v16921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:40.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:40 smithi090 ceph-mon[29180]: pgmap v16922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:40.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:40 smithi119 ceph-mon[32752]: pgmap v16922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:40.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:40 smithi167 ceph-mon[32751]: pgmap v16922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:41.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:41 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30682.459436 secs 2024-04-27T03:06:41.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:41 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30682.459436 secs 2024-04-27T03:06:41.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:41 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30682.459436 secs 2024-04-27T03:06:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:42 smithi090 ceph-mon[29180]: pgmap v16923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:42.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30683.308541 secs 2024-04-27T03:06:42.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:42 smithi119 ceph-mon[32752]: pgmap v16923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:42.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30683.308541 secs 2024-04-27T03:06:42.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:42 smithi167 ceph-mon[32751]: pgmap v16923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:42.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30683.308541 secs 2024-04-27T03:06:44.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:44 smithi090 ceph-mon[29180]: pgmap v16924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:44.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:44 smithi119 ceph-mon[32752]: pgmap v16924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:44.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:44 smithi167 ceph-mon[32751]: pgmap v16924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:46.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:46 smithi090 ceph-mon[29180]: pgmap v16925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:46.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:46 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30687.459498 secs 2024-04-27T03:06:46.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:46 smithi119 ceph-mon[32752]: pgmap v16925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:46.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:46 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30687.459498 secs 2024-04-27T03:06:46.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:46 smithi167 ceph-mon[32751]: pgmap v16925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:46.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:46 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30687.459498 secs 2024-04-27T03:06:47.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30688.308652 secs 2024-04-27T03:06:47.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30688.308652 secs 2024-04-27T03:06:47.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30688.308652 secs 2024-04-27T03:06:48.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:48 smithi090 ceph-mon[29180]: pgmap v16926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:48.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:48 smithi119 ceph-mon[32752]: pgmap v16926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:48.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:48 smithi167 ceph-mon[32751]: pgmap v16926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:50.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:50 smithi090 ceph-mon[29180]: pgmap v16927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:50.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:50 smithi119 ceph-mon[32752]: pgmap v16927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:50.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:50 smithi167 ceph-mon[32751]: pgmap v16927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:51 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30692.459584 secs 2024-04-27T03:06:51.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:51 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30692.459584 secs 2024-04-27T03:06:51.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:51 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30692.459584 secs 2024-04-27T03:06:52.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:52 smithi090 ceph-mon[29180]: pgmap v16928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:52.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30693.308771 secs 2024-04-27T03:06:52.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:52 smithi119 ceph-mon[32752]: pgmap v16928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:52.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30693.308771 secs 2024-04-27T03:06:52.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:52 smithi167 ceph-mon[32751]: pgmap v16928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:52.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30693.308771 secs 2024-04-27T03:06:54.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:54 smithi090 ceph-mon[29180]: pgmap v16929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:54.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:54 smithi119 ceph-mon[32752]: pgmap v16929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:54.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:54 smithi167 ceph-mon[32751]: pgmap v16929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:56.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:56 smithi090 ceph-mon[29180]: pgmap v16930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:56.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:56 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30697.459667 secs 2024-04-27T03:06:56.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:56 smithi119 ceph-mon[32752]: pgmap v16930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:56.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:56 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30697.459667 secs 2024-04-27T03:06:56.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:56 smithi167 ceph-mon[32751]: pgmap v16930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:06:56.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:56 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30697.459667 secs 2024-04-27T03:06:57.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30698.308842 secs 2024-04-27T03:06:57.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30698.308842 secs 2024-04-27T03:06:57.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30698.308842 secs 2024-04-27T03:06:58.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:58 smithi119 ceph-mon[32752]: pgmap v16931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:58.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:58 smithi167 ceph-mon[32751]: pgmap v16931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:58.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:58 smithi090 ceph-mon[29180]: pgmap v16931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:06:59.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:06:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:06:59.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:06:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:06:59.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:06:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:07:00.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:00 smithi119 ceph-mon[32752]: pgmap v16932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:00.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:07:00.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:07:00.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:07:00.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:00 smithi167 ceph-mon[32751]: pgmap v16932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:00.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:07:00.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:07:00.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:07:00.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:00 smithi090 ceph-mon[29180]: pgmap v16932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:00.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:07:00.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:07:00.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:07:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:01 smithi119 ceph-mon[32752]: pgmap v16933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:07:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:01 smithi119 ceph-mon[32752]: pgmap v16934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:01.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:01 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30702.459757 secs 2024-04-27T03:07:01.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:01 smithi167 ceph-mon[32751]: pgmap v16933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:07:01.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:01 smithi167 ceph-mon[32751]: pgmap v16934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:01.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:01 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30702.459757 secs 2024-04-27T03:07:01.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:01 smithi090 ceph-mon[29180]: pgmap v16933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:07:01.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:01 smithi090 ceph-mon[29180]: pgmap v16934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:01.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:01 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30702.459757 secs 2024-04-27T03:07:02.508 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30703.308978 secs 2024-04-27T03:07:02.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30703.308978 secs 2024-04-27T03:07:02.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30703.308978 secs 2024-04-27T03:07:03.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:03 smithi119 ceph-mon[32752]: pgmap v16935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:03.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:03 smithi167 ceph-mon[32751]: pgmap v16935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:03.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:03 smithi090 ceph-mon[29180]: pgmap v16935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:06.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:05 smithi119 ceph-mon[32752]: pgmap v16936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:06.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30707.459846 secs 2024-04-27T03:07:06.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:05 smithi167 ceph-mon[32751]: pgmap v16936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:06.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30707.459846 secs 2024-04-27T03:07:06.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:05 smithi090 ceph-mon[29180]: pgmap v16936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:06.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30707.459846 secs 2024-04-27T03:07:07.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:06 smithi119 ceph-mon[32752]: pgmap v16937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:07.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:06 smithi167 ceph-mon[32751]: pgmap v16937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:07.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:06 smithi090 ceph-mon[29180]: pgmap v16937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:07:07.962 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30708.309095 secs 2024-04-27T03:07:07.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30708.309095 secs 2024-04-27T03:07:08.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30708.309095 secs 2024-04-27T03:07:09.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:08 smithi119 ceph-mon[32752]: pgmap v16938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:07:09.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:08 smithi167 ceph-mon[32751]: pgmap v16938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:07:09.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:08 smithi090 ceph-mon[29180]: pgmap v16938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:07:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:10 smithi090 ceph-mon[29180]: pgmap v16939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:07:11.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:10 smithi090 ceph-mon[29180]: 7 slow requests, 1 included below; oldest blocked for > 30712.459939 secs 2024-04-27T03:07:11.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:10 smithi090 ceph-mon[29180]: slow request 721.151542 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:07:11.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:10 smithi119 ceph-mon[32752]: pgmap v16939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:07:11.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:10 smithi119 ceph-mon[32752]: 7 slow requests, 1 included below; oldest blocked for > 30712.459939 secs 2024-04-27T03:07:11.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:10 smithi119 ceph-mon[32752]: slow request 721.151542 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:07:11.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:10 smithi167 ceph-mon[32751]: pgmap v16939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:07:11.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:10 smithi167 ceph-mon[32751]: 7 slow requests, 1 included below; oldest blocked for > 30712.459939 secs 2024-04-27T03:07:11.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:10 smithi167 ceph-mon[32751]: slow request 721.151542 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:07:13.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:12 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 30713.309229 secs 2024-04-27T03:07:13.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:12 smithi090 ceph-mon[29180]: slow request 2882.579813 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:07:13.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:12 smithi090 ceph-mon[29180]: pgmap v16940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:13.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:12 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 30713.309229 secs 2024-04-27T03:07:13.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:12 smithi119 ceph-mon[32752]: slow request 2882.579813 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:07:13.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:12 smithi119 ceph-mon[32752]: pgmap v16940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:13.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:12 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 30713.309229 secs 2024-04-27T03:07:13.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:12 smithi167 ceph-mon[32751]: slow request 2882.579813 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:07:13.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:12 smithi167 ceph-mon[32751]: pgmap v16940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:15.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:14 smithi090 ceph-mon[29180]: pgmap v16941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:15.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:14 smithi119 ceph-mon[32752]: pgmap v16941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:15.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:14 smithi167 ceph-mon[32751]: pgmap v16941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:16.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30717.460012 secs 2024-04-27T03:07:16.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30717.460012 secs 2024-04-27T03:07:16.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30717.460012 secs 2024-04-27T03:07:17.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:16 smithi090 ceph-mon[29180]: pgmap v16942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:17.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:16 smithi119 ceph-mon[32752]: pgmap v16942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:17.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:16 smithi167 ceph-mon[32751]: pgmap v16942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:18.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30718.309345 secs 2024-04-27T03:07:18.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30718.309345 secs 2024-04-27T03:07:18.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30718.309345 secs 2024-04-27T03:07:19.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:18 smithi090 ceph-mon[29180]: pgmap v16943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:19.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:18 smithi119 ceph-mon[32752]: pgmap v16943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:19.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:18 smithi167 ceph-mon[32751]: pgmap v16943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:21.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:20 smithi090 ceph-mon[29180]: pgmap v16944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:21.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30722.460108 secs 2024-04-27T03:07:21.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:20 smithi119 ceph-mon[32752]: pgmap v16944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:21.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30722.460108 secs 2024-04-27T03:07:21.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:20 smithi167 ceph-mon[32751]: pgmap v16944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:21.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30722.460108 secs 2024-04-27T03:07:23.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30723.309466 secs 2024-04-27T03:07:23.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:22 smithi090 ceph-mon[29180]: pgmap v16945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:23.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30723.309466 secs 2024-04-27T03:07:23.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:22 smithi119 ceph-mon[32752]: pgmap v16945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:23.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30723.309466 secs 2024-04-27T03:07:23.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:22 smithi167 ceph-mon[32751]: pgmap v16945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:25.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:24 smithi090 ceph-mon[29180]: pgmap v16946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:25.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:24 smithi119 ceph-mon[32752]: pgmap v16946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:25.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:24 smithi167 ceph-mon[32751]: pgmap v16946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30727.460169 secs 2024-04-27T03:07:26.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30727.460169 secs 2024-04-27T03:07:26.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30727.460169 secs 2024-04-27T03:07:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:26 smithi090 ceph-mon[29180]: pgmap v16947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:27.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:26 smithi119 ceph-mon[32752]: pgmap v16947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:27.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:26 smithi167 ceph-mon[32751]: pgmap v16947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30728.309606 secs 2024-04-27T03:07:28.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30728.309606 secs 2024-04-27T03:07:28.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30728.309606 secs 2024-04-27T03:07:29.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:28 smithi119 ceph-mon[32752]: pgmap v16948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:29.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:28 smithi167 ceph-mon[32751]: pgmap v16948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:29.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:28 smithi090 ceph-mon[29180]: pgmap v16948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:31.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:30 smithi119 ceph-mon[32752]: pgmap v16949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:31.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30732.460229 secs 2024-04-27T03:07:31.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:30 smithi167 ceph-mon[32751]: pgmap v16949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:31.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30732.460229 secs 2024-04-27T03:07:31.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:30 smithi090 ceph-mon[29180]: pgmap v16949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:31.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30732.460229 secs 2024-04-27T03:07:33.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30733.309720 secs 2024-04-27T03:07:33.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:32 smithi119 ceph-mon[32752]: pgmap v16950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:33.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30733.309720 secs 2024-04-27T03:07:33.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:32 smithi167 ceph-mon[32751]: pgmap v16950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:33.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30733.309720 secs 2024-04-27T03:07:33.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:32 smithi090 ceph-mon[29180]: pgmap v16950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:35.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:34 smithi119 ceph-mon[32752]: pgmap v16951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:35.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:34 smithi167 ceph-mon[32751]: pgmap v16951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:35.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:34 smithi090 ceph-mon[29180]: pgmap v16951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:36.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30737.460291 secs 2024-04-27T03:07:36.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30737.460291 secs 2024-04-27T03:07:36.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30737.460291 secs 2024-04-27T03:07:37.212 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:36 smithi119 ceph-mon[32752]: pgmap v16952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:37.227 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:36 smithi167 ceph-mon[32751]: pgmap v16952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:37.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:36 smithi090 ceph-mon[29180]: pgmap v16952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:38.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30738.309832 secs 2024-04-27T03:07:38.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30738.309832 secs 2024-04-27T03:07:38.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30738.309832 secs 2024-04-27T03:07:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:38 smithi119 ceph-mon[32752]: pgmap v16953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:39.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:38 smithi167 ceph-mon[32751]: pgmap v16953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:39.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:38 smithi090 ceph-mon[29180]: pgmap v16953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:41.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:40 smithi119 ceph-mon[32752]: pgmap v16954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:41.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:40 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30742.460354 secs 2024-04-27T03:07:41.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:40 smithi167 ceph-mon[32751]: pgmap v16954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:41.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:40 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30742.460354 secs 2024-04-27T03:07:41.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:40 smithi090 ceph-mon[29180]: pgmap v16954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:41.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:40 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30742.460354 secs 2024-04-27T03:07:43.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30743.309944 secs 2024-04-27T03:07:43.186 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:42 smithi090 ceph-mon[29180]: pgmap v16955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:43.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30743.309944 secs 2024-04-27T03:07:43.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:42 smithi119 ceph-mon[32752]: pgmap v16955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:43.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30743.309944 secs 2024-04-27T03:07:43.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:42 smithi167 ceph-mon[32751]: pgmap v16955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:45.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:44 smithi119 ceph-mon[32752]: pgmap v16956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:45.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:44 smithi167 ceph-mon[32751]: pgmap v16956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:45.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:44 smithi090 ceph-mon[29180]: pgmap v16956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:46.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:45 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30747.460435 secs 2024-04-27T03:07:46.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:45 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30747.460435 secs 2024-04-27T03:07:46.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:45 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30747.460435 secs 2024-04-27T03:07:47.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:46 smithi119 ceph-mon[32752]: pgmap v16957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:47.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:46 smithi167 ceph-mon[32751]: pgmap v16957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:47.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:46 smithi090 ceph-mon[29180]: pgmap v16957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:48.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30748.310042 secs 2024-04-27T03:07:48.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30748.310042 secs 2024-04-27T03:07:48.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30748.310042 secs 2024-04-27T03:07:49.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:48 smithi119 ceph-mon[32752]: pgmap v16958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:49.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:48 smithi167 ceph-mon[32751]: pgmap v16958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:49.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:48 smithi090 ceph-mon[29180]: pgmap v16958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:51.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:50 smithi119 ceph-mon[32752]: pgmap v16959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:51.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:50 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30752.460519 secs 2024-04-27T03:07:51.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:50 smithi167 ceph-mon[32751]: pgmap v16959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:51.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:50 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30752.460519 secs 2024-04-27T03:07:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:50 smithi090 ceph-mon[29180]: pgmap v16959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:51.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:50 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30752.460519 secs 2024-04-27T03:07:52.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30753.310183 secs 2024-04-27T03:07:52.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30753.310183 secs 2024-04-27T03:07:52.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30753.310183 secs 2024-04-27T03:07:53.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:52 smithi119 ceph-mon[32752]: pgmap v16960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:53.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:52 smithi167 ceph-mon[32751]: pgmap v16960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:53.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:52 smithi090 ceph-mon[29180]: pgmap v16960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:55.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:54 smithi119 ceph-mon[32752]: pgmap v16961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:55.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:54 smithi167 ceph-mon[32751]: pgmap v16961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:55.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:54 smithi090 ceph-mon[29180]: pgmap v16961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:07:56.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:55 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30757.460603 secs 2024-04-27T03:07:56.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:55 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30757.460603 secs 2024-04-27T03:07:56.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:55 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30757.460603 secs 2024-04-27T03:07:57.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:56 smithi119 ceph-mon[32752]: pgmap v16962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:57.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30758.310322 secs 2024-04-27T03:07:57.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:56 smithi167 ceph-mon[32751]: pgmap v16962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:57.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30758.310322 secs 2024-04-27T03:07:57.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:56 smithi090 ceph-mon[29180]: pgmap v16962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:57.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30758.310322 secs 2024-04-27T03:07:59.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:58 smithi119 ceph-mon[32752]: pgmap v16963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:59.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:58 smithi167 ceph-mon[32751]: pgmap v16963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:07:59.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:58 smithi090 ceph-mon[29180]: pgmap v16963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:00.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:07:59 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:08:00.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:07:59 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:08:00.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:07:59 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:08:01.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:00 smithi119 ceph-mon[32752]: pgmap v16964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:01.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:00 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30762.460667 secs 2024-04-27T03:08:01.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:08:01.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:08:01.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:00 smithi119 ceph-mon[32752]: pgmap v16965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:08:01.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:00 smithi119 ceph-mon[32752]: pgmap v16966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:01.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:08:01.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:00 smithi167 ceph-mon[32751]: pgmap v16964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:01.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:00 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30762.460667 secs 2024-04-27T03:08:01.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:08:01.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:08:01.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:00 smithi167 ceph-mon[32751]: pgmap v16965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:08:01.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:00 smithi167 ceph-mon[32751]: pgmap v16966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:01.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:08:01.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:00 smithi090 ceph-mon[29180]: pgmap v16964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:00 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30762.460667 secs 2024-04-27T03:08:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:08:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:08:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:00 smithi090 ceph-mon[29180]: pgmap v16965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:08:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:00 smithi090 ceph-mon[29180]: pgmap v16966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:08:02.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30763.310416 secs 2024-04-27T03:08:02.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30763.310416 secs 2024-04-27T03:08:02.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30763.310416 secs 2024-04-27T03:08:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:02 smithi119 ceph-mon[32752]: pgmap v16967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:08:03.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:02 smithi167 ceph-mon[32751]: pgmap v16967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:08:03.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:02 smithi090 ceph-mon[29180]: pgmap v16967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:08:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:05 smithi090 ceph-mon[29180]: pgmap v16968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:05 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30767.460729 secs 2024-04-27T03:08:06.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:05 smithi119 ceph-mon[32752]: pgmap v16968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:06.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:05 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30767.460729 secs 2024-04-27T03:08:06.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:05 smithi167 ceph-mon[32751]: pgmap v16968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:06.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:05 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30767.460729 secs 2024-04-27T03:08:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30768.310511 secs 2024-04-27T03:08:07.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:07 smithi167 ceph-mon[32751]: pgmap v16969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:07.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30768.310511 secs 2024-04-27T03:08:07.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:07 smithi090 ceph-mon[29180]: pgmap v16969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:08.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30768.310511 secs 2024-04-27T03:08:08.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:07 smithi119 ceph-mon[32752]: pgmap v16969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:09.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:09 smithi090 ceph-mon[29180]: pgmap v16970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:10.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:09 smithi119 ceph-mon[32752]: pgmap v16970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:10.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:09 smithi167 ceph-mon[32751]: pgmap v16970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:08:11.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:10 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30772.460831 secs 2024-04-27T03:08:11.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:10 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30772.460831 secs 2024-04-27T03:08:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:10 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30772.460831 secs 2024-04-27T03:08:12.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:11 smithi119 ceph-mon[32752]: pgmap v16971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:08:12.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:11 smithi167 ceph-mon[32751]: pgmap v16971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:08:12.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:11 smithi090 ceph-mon[29180]: pgmap v16971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:08:13.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30773.310635 secs 2024-04-27T03:08:13.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30773.310635 secs 2024-04-27T03:08:13.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30773.310635 secs 2024-04-27T03:08:14.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:13 smithi119 ceph-mon[32752]: pgmap v16972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:14.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:13 smithi167 ceph-mon[32751]: pgmap v16972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:14.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:13 smithi090 ceph-mon[29180]: pgmap v16972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:16.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:15 smithi119 ceph-mon[32752]: pgmap v16973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:16.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:15 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30777.460898 secs 2024-04-27T03:08:16.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:15 smithi167 ceph-mon[32751]: pgmap v16973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:16.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:15 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30777.460898 secs 2024-04-27T03:08:16.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:15 smithi090 ceph-mon[29180]: pgmap v16973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:16.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:15 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30777.460898 secs 2024-04-27T03:08:18.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30778.310749 secs 2024-04-27T03:08:18.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:17 smithi119 ceph-mon[32752]: pgmap v16974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:18.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30778.310749 secs 2024-04-27T03:08:18.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:17 smithi167 ceph-mon[32751]: pgmap v16974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:18.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30778.310749 secs 2024-04-27T03:08:18.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:17 smithi090 ceph-mon[29180]: pgmap v16974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:19.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:18 smithi119 ceph-mon[32752]: pgmap v16975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:19.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:18 smithi167 ceph-mon[32751]: pgmap v16975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:19.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:18 smithi090 ceph-mon[29180]: pgmap v16975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:20.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:20 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30782.460974 secs 2024-04-27T03:08:20.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:20 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30782.460974 secs 2024-04-27T03:08:20.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:20 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30782.460974 secs 2024-04-27T03:08:21.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:21 smithi090 ceph-mon[29180]: pgmap v16976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:21.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:21 smithi119 ceph-mon[32752]: pgmap v16976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:21.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:21 smithi167 ceph-mon[32751]: pgmap v16976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:22.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30783.310828 secs 2024-04-27T03:08:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30783.310828 secs 2024-04-27T03:08:22.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30783.310828 secs 2024-04-27T03:08:23.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:23 smithi090 ceph-mon[29180]: pgmap v16977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:23.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:23 smithi119 ceph-mon[32752]: pgmap v16977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:23.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:23 smithi167 ceph-mon[32751]: pgmap v16977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:25.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:25 smithi090 ceph-mon[29180]: pgmap v16978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:25.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:25 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30787.461049 secs 2024-04-27T03:08:26.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:25 smithi119 ceph-mon[32752]: pgmap v16978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:26.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:25 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30787.461049 secs 2024-04-27T03:08:26.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:25 smithi167 ceph-mon[32751]: pgmap v16978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:26.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:25 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30787.461049 secs 2024-04-27T03:08:27.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30788.310941 secs 2024-04-27T03:08:27.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:27 smithi090 ceph-mon[29180]: pgmap v16979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30788.310941 secs 2024-04-27T03:08:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:27 smithi119 ceph-mon[32752]: pgmap v16979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30788.310941 secs 2024-04-27T03:08:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:27 smithi167 ceph-mon[32751]: pgmap v16979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:30.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:29 smithi119 ceph-mon[32752]: pgmap v16980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:30.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:29 smithi167 ceph-mon[32751]: pgmap v16980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:30.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:29 smithi090 ceph-mon[29180]: pgmap v16980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:31.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:30 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30792.461136 secs 2024-04-27T03:08:31.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:30 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30792.461136 secs 2024-04-27T03:08:31.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:30 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30792.461136 secs 2024-04-27T03:08:32.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:31 smithi119 ceph-mon[32752]: pgmap v16981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:32.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:31 smithi167 ceph-mon[32751]: pgmap v16981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:32.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:31 smithi090 ceph-mon[29180]: pgmap v16981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:33.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30793.311050 secs 2024-04-27T03:08:33.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30793.311050 secs 2024-04-27T03:08:33.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30793.311050 secs 2024-04-27T03:08:34.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:33 smithi119 ceph-mon[32752]: pgmap v16982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:34.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:33 smithi167 ceph-mon[32751]: pgmap v16982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:34.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:33 smithi090 ceph-mon[29180]: pgmap v16982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:35.936 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:35 smithi090 ceph-mon[29180]: pgmap v16983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:35.936 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:35 smithi090 ceph-mon[29180]: 7 slow requests, 0 included below; oldest blocked for > 30797.461219 secs 2024-04-27T03:08:36.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:35 smithi119 ceph-mon[32752]: pgmap v16983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:36.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:35 smithi119 ceph-mon[32752]: 7 slow requests, 0 included below; oldest blocked for > 30797.461219 secs 2024-04-27T03:08:36.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:35 smithi167 ceph-mon[32751]: pgmap v16983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:36.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:35 smithi167 ceph-mon[32751]: 7 slow requests, 0 included below; oldest blocked for > 30797.461219 secs 2024-04-27T03:08:37.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:36 smithi119 ceph-mon[32752]: pgmap v16984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:37.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:36 smithi167 ceph-mon[32751]: pgmap v16984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:37.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:36 smithi090 ceph-mon[29180]: pgmap v16984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:37.962 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30798.311171 secs 2024-04-27T03:08:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30798.311171 secs 2024-04-27T03:08:38.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30798.311171 secs 2024-04-27T03:08:39.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:38 smithi119 ceph-mon[32752]: pgmap v16985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:39.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:38 smithi167 ceph-mon[32751]: pgmap v16985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:39.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:38 smithi090 ceph-mon[29180]: pgmap v16985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:40.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:40 smithi090 ceph-mon[29180]: 41 slow requests, 2 included below; oldest blocked for > 30802.461304 secs 2024-04-27T03:08:40.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:40 smithi090 ceph-mon[29180]: slow request 180.970827 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:08:40.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:40 smithi090 ceph-mon[29180]: slow request 3048.231997 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:08:40.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:40 smithi119 ceph-mon[32752]: 41 slow requests, 2 included below; oldest blocked for > 30802.461304 secs 2024-04-27T03:08:40.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:40 smithi119 ceph-mon[32752]: slow request 180.970827 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:08:40.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:40 smithi119 ceph-mon[32752]: slow request 3048.231997 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:08:40.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:40 smithi167 ceph-mon[32751]: 41 slow requests, 2 included below; oldest blocked for > 30802.461304 secs 2024-04-27T03:08:40.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:40 smithi167 ceph-mon[32751]: slow request 180.970827 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:08:40.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:40 smithi167 ceph-mon[32751]: slow request 3048.231997 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:08:41.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:41 smithi090 ceph-mon[29180]: pgmap v16986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:41.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:41 smithi119 ceph-mon[32752]: pgmap v16986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:41.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:41 smithi167 ceph-mon[32751]: pgmap v16986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:42.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30803.311287 secs 2024-04-27T03:08:42.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30803.311287 secs 2024-04-27T03:08:42.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30803.311287 secs 2024-04-27T03:08:43.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:08:43.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:08:43.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:43 smithi090 ceph-mon[29180]: pgmap v16987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:43.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:08:43.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:08:43.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:43 smithi119 ceph-mon[32752]: pgmap v16987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:43.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:08:43.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:08:43.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:43 smithi167 ceph-mon[32751]: pgmap v16987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:45 smithi119 ceph-mon[32752]: pgmap v16988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:45 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30807.461366 secs 2024-04-27T03:08:46.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:45 smithi167 ceph-mon[32751]: pgmap v16988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:46.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:45 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30807.461366 secs 2024-04-27T03:08:46.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:45 smithi090 ceph-mon[29180]: pgmap v16988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:46.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:45 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30807.461366 secs 2024-04-27T03:08:48.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30808.311410 secs 2024-04-27T03:08:48.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:47 smithi119 ceph-mon[32752]: pgmap v16989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:48.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30808.311410 secs 2024-04-27T03:08:48.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:47 smithi167 ceph-mon[32751]: pgmap v16989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:48.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30808.311410 secs 2024-04-27T03:08:48.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:47 smithi090 ceph-mon[29180]: pgmap v16989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:08:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:49 smithi119 ceph-mon[32752]: pgmap v16990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:49 smithi167 ceph-mon[32751]: pgmap v16990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:50.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:49 smithi090 ceph-mon[29180]: pgmap v16990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:51.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:50 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30812.461449 secs 2024-04-27T03:08:51.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:50 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30812.461449 secs 2024-04-27T03:08:51.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:50 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30812.461449 secs 2024-04-27T03:08:51.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:51 smithi090 ceph-mon[29180]: pgmap v16991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:52.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:51 smithi119 ceph-mon[32752]: pgmap v16991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:52.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:51 smithi167 ceph-mon[32751]: pgmap v16991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:53.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30813.311496 secs 2024-04-27T03:08:53.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30813.311496 secs 2024-04-27T03:08:53.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30813.311496 secs 2024-04-27T03:08:54.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:53 smithi119 ceph-mon[32752]: pgmap v16992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:54.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:53 smithi167 ceph-mon[32751]: pgmap v16992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:54.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:53 smithi090 ceph-mon[29180]: pgmap v16992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:56.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:55 smithi119 ceph-mon[32752]: pgmap v16993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:56.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:55 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30817.461525 secs 2024-04-27T03:08:56.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:55 smithi167 ceph-mon[32751]: pgmap v16993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:56.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:55 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30817.461525 secs 2024-04-27T03:08:56.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:55 smithi090 ceph-mon[29180]: pgmap v16993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:56.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:55 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30817.461525 secs 2024-04-27T03:08:57.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:56 smithi119 ceph-mon[32752]: pgmap v16994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:57.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:56 smithi167 ceph-mon[32751]: pgmap v16994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:57.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:56 smithi090 ceph-mon[29180]: pgmap v16994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:08:58.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30818.311607 secs 2024-04-27T03:08:58.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30818.311607 secs 2024-04-27T03:08:58.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30818.311607 secs 2024-04-27T03:08:59.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:08:58 smithi119 ceph-mon[32752]: pgmap v16995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:59.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:08:58 smithi167 ceph-mon[32751]: pgmap v16995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:08:59.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:08:58 smithi090 ceph-mon[29180]: pgmap v16995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:00.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:00 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30822.461610 secs 2024-04-27T03:09:00.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:00 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30822.461610 secs 2024-04-27T03:09:00.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:00 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30822.461610 secs 2024-04-27T03:09:01.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:01 smithi090 ceph-mon[29180]: pgmap v16996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:01.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:09:01.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:01 smithi119 ceph-mon[32752]: pgmap v16996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:01.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:09:01.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:01 smithi167 ceph-mon[32751]: pgmap v16996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:01.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:09:02.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30823.311698 secs 2024-04-27T03:09:02.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:09:02.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:09:02.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:02 smithi090 ceph-mon[29180]: pgmap v16997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:09:02.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:02 smithi090 ceph-mon[29180]: pgmap v16998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:02.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:09:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30823.311698 secs 2024-04-27T03:09:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:09:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:09:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:02 smithi119 ceph-mon[32752]: pgmap v16997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:09:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:02 smithi119 ceph-mon[32752]: pgmap v16998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:09:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30823.311698 secs 2024-04-27T03:09:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:09:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:09:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:02 smithi167 ceph-mon[32751]: pgmap v16997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:09:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:02 smithi167 ceph-mon[32751]: pgmap v16998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:09:04.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:04 smithi119 ceph-mon[32752]: pgmap v16999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:04.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:04 smithi167 ceph-mon[32751]: pgmap v16999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:04.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:04 smithi090 ceph-mon[29180]: pgmap v16999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:05.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:05 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30827.461693 secs 2024-04-27T03:09:05.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:05 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30827.461693 secs 2024-04-27T03:09:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:05 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30827.461693 secs 2024-04-27T03:09:06.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:06 smithi090 ceph-mon[29180]: pgmap v17000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:06.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:06 smithi119 ceph-mon[32752]: pgmap v17000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:06.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:06 smithi167 ceph-mon[32751]: pgmap v17000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:07.712 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30828.311774 secs 2024-04-27T03:09:07.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30828.311774 secs 2024-04-27T03:09:07.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30828.311774 secs 2024-04-27T03:09:08.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:08 smithi119 ceph-mon[32752]: pgmap v17001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:09:08.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:08 smithi167 ceph-mon[32751]: pgmap v17001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:09:08.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:08 smithi090 ceph-mon[29180]: pgmap v17001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:09:10.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:10 smithi119 ceph-mon[32752]: pgmap v17002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:10.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:10 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30832.461777 secs 2024-04-27T03:09:10.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:10 smithi167 ceph-mon[32751]: pgmap v17002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:10.838 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:10 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30832.461777 secs 2024-04-27T03:09:10.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:10 smithi090 ceph-mon[29180]: pgmap v17002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:09:10.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:10 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30832.461777 secs 2024-04-27T03:09:12.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:12 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 30833.311893 secs 2024-04-27T03:09:12.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:12 smithi119 ceph-mon[32752]: slow request 1442.170183 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:09:12.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:12 smithi119 ceph-mon[32752]: pgmap v17003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:09:12.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:12 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 30833.311893 secs 2024-04-27T03:09:12.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:12 smithi167 ceph-mon[32751]: slow request 1442.170183 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:09:12.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:12 smithi167 ceph-mon[32751]: pgmap v17003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:09:12.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:12 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 30833.311893 secs 2024-04-27T03:09:12.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:12 smithi090 ceph-mon[29180]: slow request 1442.170183 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:09:12.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:12 smithi090 ceph-mon[29180]: pgmap v17003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:09:14.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:14 smithi090 ceph-mon[29180]: pgmap v17004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:14.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:14 smithi119 ceph-mon[32752]: pgmap v17004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:14.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:14 smithi167 ceph-mon[32751]: pgmap v17004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:15.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:15 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30837.461866 secs 2024-04-27T03:09:15.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:15 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30837.461866 secs 2024-04-27T03:09:15.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:15 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30837.461866 secs 2024-04-27T03:09:16.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:16 smithi119 ceph-mon[32752]: pgmap v17005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:16.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:16 smithi167 ceph-mon[32751]: pgmap v17005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:16.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:16 smithi090 ceph-mon[29180]: pgmap v17005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:17.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30838.312009 secs 2024-04-27T03:09:17.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30838.312009 secs 2024-04-27T03:09:17.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30838.312009 secs 2024-04-27T03:09:18.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:18 smithi119 ceph-mon[32752]: pgmap v17006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:18.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:18 smithi167 ceph-mon[32751]: pgmap v17006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:18.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:18 smithi090 ceph-mon[29180]: pgmap v17006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:20.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:20 smithi119 ceph-mon[32752]: pgmap v17007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:20.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:20 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30842.461928 secs 2024-04-27T03:09:20.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:20 smithi167 ceph-mon[32751]: pgmap v17007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:20.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:20 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30842.461928 secs 2024-04-27T03:09:20.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:20 smithi090 ceph-mon[29180]: pgmap v17007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:20.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:20 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30842.461928 secs 2024-04-27T03:09:22.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30843.312083 secs 2024-04-27T03:09:22.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:22 smithi119 ceph-mon[32752]: pgmap v17008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:22.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30843.312083 secs 2024-04-27T03:09:22.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:22 smithi167 ceph-mon[32751]: pgmap v17008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:22.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30843.312083 secs 2024-04-27T03:09:22.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:22 smithi090 ceph-mon[29180]: pgmap v17008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:24.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:24 smithi119 ceph-mon[32752]: pgmap v17009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:24.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:24 smithi167 ceph-mon[32751]: pgmap v17009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:24.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:24 smithi090 ceph-mon[29180]: pgmap v17009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:25.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:25 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30847.462004 secs 2024-04-27T03:09:25.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:25 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30847.462004 secs 2024-04-27T03:09:25.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:25 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30847.462004 secs 2024-04-27T03:09:26.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:26 smithi119 ceph-mon[32752]: pgmap v17010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:26.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:26 smithi167 ceph-mon[32751]: pgmap v17010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:26.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:26 smithi090 ceph-mon[29180]: pgmap v17010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:27.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30848.312185 secs 2024-04-27T03:09:27.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30848.312185 secs 2024-04-27T03:09:27.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30848.312185 secs 2024-04-27T03:09:28.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:28 smithi119 ceph-mon[32752]: pgmap v17011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:28.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:28 smithi167 ceph-mon[32751]: pgmap v17011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:28.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:28 smithi090 ceph-mon[29180]: pgmap v17011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:30.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:30 smithi119 ceph-mon[32752]: pgmap v17012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:30.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:30 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30852.462103 secs 2024-04-27T03:09:30.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:30 smithi167 ceph-mon[32751]: pgmap v17012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:30.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:30 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30852.462103 secs 2024-04-27T03:09:30.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:30 smithi090 ceph-mon[29180]: pgmap v17012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:30.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:30 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30852.462103 secs 2024-04-27T03:09:32.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30853.312315 secs 2024-04-27T03:09:32.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:32 smithi119 ceph-mon[32752]: pgmap v17013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:32.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30853.312315 secs 2024-04-27T03:09:32.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:32 smithi167 ceph-mon[32751]: pgmap v17013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:32.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30853.312315 secs 2024-04-27T03:09:32.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:32 smithi090 ceph-mon[29180]: pgmap v17013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:34.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:34 smithi119 ceph-mon[32752]: pgmap v17014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:34.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:34 smithi167 ceph-mon[32751]: pgmap v17014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:34.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:34 smithi090 ceph-mon[29180]: pgmap v17014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:35.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:35 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30857.462196 secs 2024-04-27T03:09:35.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:35 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30857.462196 secs 2024-04-27T03:09:35.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:35 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30857.462196 secs 2024-04-27T03:09:36.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:36 smithi119 ceph-mon[32752]: pgmap v17015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:36.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:36 smithi167 ceph-mon[32751]: pgmap v17015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:36.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:36 smithi090 ceph-mon[29180]: pgmap v17015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:37.785 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30858.312433 secs 2024-04-27T03:09:37.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30858.312433 secs 2024-04-27T03:09:37.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30858.312433 secs 2024-04-27T03:09:38.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:38 smithi119 ceph-mon[32752]: pgmap v17016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:38.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:38 smithi167 ceph-mon[32751]: pgmap v17016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:38.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:38 smithi090 ceph-mon[29180]: pgmap v17016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:40.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:40 smithi119 ceph-mon[32752]: pgmap v17017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:40.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:40 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30862.462284 secs 2024-04-27T03:09:40.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:40 smithi167 ceph-mon[32751]: pgmap v17017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:40.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:40 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30862.462284 secs 2024-04-27T03:09:40.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:40 smithi090 ceph-mon[29180]: pgmap v17017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:40.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:40 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30862.462284 secs 2024-04-27T03:09:42.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30863.312568 secs 2024-04-27T03:09:42.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:42 smithi119 ceph-mon[32752]: pgmap v17018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:42.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30863.312568 secs 2024-04-27T03:09:42.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:42 smithi167 ceph-mon[32751]: pgmap v17018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:42.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30863.312568 secs 2024-04-27T03:09:42.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:42 smithi090 ceph-mon[29180]: pgmap v17018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:44.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:44 smithi119 ceph-mon[32752]: pgmap v17019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:44.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:44 smithi167 ceph-mon[32751]: pgmap v17019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:44.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:44 smithi090 ceph-mon[29180]: pgmap v17019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:45.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:45 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30867.462358 secs 2024-04-27T03:09:45.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:45 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30867.462358 secs 2024-04-27T03:09:45.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:45 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30867.462358 secs 2024-04-27T03:09:46.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:46 smithi119 ceph-mon[32752]: pgmap v17020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:46.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:46 smithi167 ceph-mon[32751]: pgmap v17020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:46.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:46 smithi090 ceph-mon[29180]: pgmap v17020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:47.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30868.312666 secs 2024-04-27T03:09:47.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30868.312666 secs 2024-04-27T03:09:47.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30868.312666 secs 2024-04-27T03:09:48.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:48 smithi119 ceph-mon[32752]: pgmap v17021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:48.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:48 smithi167 ceph-mon[32751]: pgmap v17021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:48.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:48 smithi090 ceph-mon[29180]: pgmap v17021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:50.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:50 smithi119 ceph-mon[32752]: pgmap v17022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:50.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:50 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30872.462416 secs 2024-04-27T03:09:50.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:50 smithi167 ceph-mon[32751]: pgmap v17022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:50.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:50 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30872.462416 secs 2024-04-27T03:09:50.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:50 smithi090 ceph-mon[29180]: pgmap v17022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:50.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:50 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30872.462416 secs 2024-04-27T03:09:52.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30873.312814 secs 2024-04-27T03:09:52.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:52 smithi119 ceph-mon[32752]: pgmap v17023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:52.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30873.312814 secs 2024-04-27T03:09:52.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:52 smithi167 ceph-mon[32751]: pgmap v17023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:52.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30873.312814 secs 2024-04-27T03:09:52.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:52 smithi090 ceph-mon[29180]: pgmap v17023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:54.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:54 smithi119 ceph-mon[32752]: pgmap v17024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:54.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:54 smithi167 ceph-mon[32751]: pgmap v17024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:54.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:54 smithi090 ceph-mon[29180]: pgmap v17024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:09:55.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:55 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30877.462496 secs 2024-04-27T03:09:55.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:55 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30877.462496 secs 2024-04-27T03:09:55.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:55 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30877.462496 secs 2024-04-27T03:09:56.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:56 smithi119 ceph-mon[32752]: pgmap v17025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:56.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:56 smithi167 ceph-mon[32751]: pgmap v17025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:56.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:56 smithi090 ceph-mon[29180]: pgmap v17025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:57.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30878.312930 secs 2024-04-27T03:09:57.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30878.312930 secs 2024-04-27T03:09:57.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30878.312930 secs 2024-04-27T03:09:58.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:09:58 smithi119 ceph-mon[32752]: pgmap v17026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:58.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:09:58 smithi167 ceph-mon[32751]: pgmap v17026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:09:58.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:09:58 smithi090 ceph-mon[29180]: pgmap v17026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:00.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:00 smithi119 ceph-mon[32752]: pgmap v17027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:00.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:10:00.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:00 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30882.462555 secs 2024-04-27T03:10:00.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:00 smithi167 ceph-mon[32751]: pgmap v17027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:00.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:10:00.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:00 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30882.462555 secs 2024-04-27T03:10:00.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:00 smithi090 ceph-mon[29180]: pgmap v17027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:00.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:10:00.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:00 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30882.462555 secs 2024-04-27T03:10:01.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:10:01.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:10:01.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:10:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30883.313060 secs 2024-04-27T03:10:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:02 smithi119 ceph-mon[32752]: pgmap v17028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:02.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:10:02.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:10:02.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:10:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30883.313060 secs 2024-04-27T03:10:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:02 smithi167 ceph-mon[32751]: pgmap v17028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:10:02.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:10:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:10:02.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30883.313060 secs 2024-04-27T03:10:02.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:02 smithi090 ceph-mon[29180]: pgmap v17028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:02.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:10:02.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:10:02.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:10:03.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:03 smithi119 ceph-mon[32752]: pgmap v17029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:10:03.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:03 smithi119 ceph-mon[32752]: pgmap v17030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:10:03.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:03 smithi167 ceph-mon[32751]: pgmap v17029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:10:03.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:03 smithi167 ceph-mon[32751]: pgmap v17030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:10:03.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:03 smithi090 ceph-mon[29180]: pgmap v17029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:10:03.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:03 smithi090 ceph-mon[29180]: pgmap v17030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:10:05.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:05 smithi119 ceph-mon[32752]: pgmap v17031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:05.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:05 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30887.462614 secs 2024-04-27T03:10:05.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:05 smithi167 ceph-mon[32751]: pgmap v17031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:05.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:05 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30887.462614 secs 2024-04-27T03:10:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:05 smithi090 ceph-mon[29180]: pgmap v17031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:05 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30887.462614 secs 2024-04-27T03:10:07.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30888.313209 secs 2024-04-27T03:10:07.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:07 smithi167 ceph-mon[32751]: pgmap v17032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:07.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30888.313209 secs 2024-04-27T03:10:07.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:07 smithi090 ceph-mon[29180]: pgmap v17032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:08.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30888.313209 secs 2024-04-27T03:10:08.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:07 smithi119 ceph-mon[32752]: pgmap v17032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:09.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:09 smithi167 ceph-mon[32751]: pgmap v17033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:10:09.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:09 smithi090 ceph-mon[29180]: pgmap v17033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:10:10.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:09 smithi119 ceph-mon[32752]: pgmap v17033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:10:10.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:10 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30892.462671 secs 2024-04-27T03:10:10.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:10 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30892.462671 secs 2024-04-27T03:10:11.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:10 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30892.462671 secs 2024-04-27T03:10:11.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:11 smithi167 ceph-mon[32751]: pgmap v17034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:11.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:11 smithi090 ceph-mon[29180]: pgmap v17034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:12.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:11 smithi119 ceph-mon[32752]: pgmap v17034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:10:12.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30893.313321 secs 2024-04-27T03:10:13.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30893.313321 secs 2024-04-27T03:10:13.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30893.313321 secs 2024-04-27T03:10:13.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:13 smithi090 ceph-mon[29180]: pgmap v17035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:10:14.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:13 smithi119 ceph-mon[32752]: pgmap v17035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:10:14.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:13 smithi167 ceph-mon[32751]: pgmap v17035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:10:15.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:15 smithi090 ceph-mon[29180]: pgmap v17036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:15 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30897.462753 secs 2024-04-27T03:10:16.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:15 smithi119 ceph-mon[32752]: pgmap v17036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:16.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:15 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30897.462753 secs 2024-04-27T03:10:16.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:15 smithi167 ceph-mon[32751]: pgmap v17036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:16.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:15 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30897.462753 secs 2024-04-27T03:10:17.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30898.313438 secs 2024-04-27T03:10:17.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:17 smithi090 ceph-mon[29180]: pgmap v17037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:18.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30898.313438 secs 2024-04-27T03:10:18.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:17 smithi119 ceph-mon[32752]: pgmap v17037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:18.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30898.313438 secs 2024-04-27T03:10:18.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:17 smithi167 ceph-mon[32751]: pgmap v17037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:19.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:19 smithi090 ceph-mon[29180]: pgmap v17038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:20.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:19 smithi119 ceph-mon[32752]: pgmap v17038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:20.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:19 smithi167 ceph-mon[32751]: pgmap v17038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:20.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:20 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30902.462835 secs 2024-04-27T03:10:21.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:20 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30902.462835 secs 2024-04-27T03:10:21.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:20 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30902.462835 secs 2024-04-27T03:10:21.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:21 smithi090 ceph-mon[29180]: pgmap v17039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:22.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:21 smithi119 ceph-mon[32752]: pgmap v17039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:22.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:21 smithi167 ceph-mon[32751]: pgmap v17039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:22.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30903.313544 secs 2024-04-27T03:10:23.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30903.313544 secs 2024-04-27T03:10:23.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30903.313544 secs 2024-04-27T03:10:23.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:23 smithi090 ceph-mon[29180]: pgmap v17040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:24.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:23 smithi119 ceph-mon[32752]: pgmap v17040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:24.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:23 smithi167 ceph-mon[32751]: pgmap v17040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:25.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:25 smithi090 ceph-mon[29180]: pgmap v17041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:25.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:25 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30907.462917 secs 2024-04-27T03:10:26.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:25 smithi119 ceph-mon[32752]: pgmap v17041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:26.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:25 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30907.462917 secs 2024-04-27T03:10:26.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:25 smithi167 ceph-mon[32751]: pgmap v17041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:26.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:25 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30907.462917 secs 2024-04-27T03:10:27.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30908.313616 secs 2024-04-27T03:10:27.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:27 smithi090 ceph-mon[29180]: pgmap v17042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30908.313616 secs 2024-04-27T03:10:28.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:27 smithi119 ceph-mon[32752]: pgmap v17042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30908.313616 secs 2024-04-27T03:10:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:27 smithi167 ceph-mon[32751]: pgmap v17042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:29.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:29 smithi090 ceph-mon[29180]: pgmap v17043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:30.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:29 smithi119 ceph-mon[32752]: pgmap v17043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:30.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:29 smithi167 ceph-mon[32751]: pgmap v17043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:30.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:30 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30912.462999 secs 2024-04-27T03:10:31.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:30 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30912.462999 secs 2024-04-27T03:10:31.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:30 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30912.462999 secs 2024-04-27T03:10:31.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:31 smithi090 ceph-mon[29180]: pgmap v17044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:32.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:31 smithi119 ceph-mon[32752]: pgmap v17044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:32.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:31 smithi167 ceph-mon[32751]: pgmap v17044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:32.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30913.313712 secs 2024-04-27T03:10:33.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30913.313712 secs 2024-04-27T03:10:33.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30913.313712 secs 2024-04-27T03:10:33.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:33 smithi090 ceph-mon[29180]: pgmap v17045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:34.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:33 smithi119 ceph-mon[32752]: pgmap v17045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:34.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:33 smithi167 ceph-mon[32751]: pgmap v17045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:36.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:35 smithi119 ceph-mon[32752]: pgmap v17046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:36.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:35 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30917.463058 secs 2024-04-27T03:10:36.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:35 smithi167 ceph-mon[32751]: pgmap v17046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:36.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:35 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30917.463058 secs 2024-04-27T03:10:36.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:35 smithi090 ceph-mon[29180]: pgmap v17046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:36.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:35 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30917.463058 secs 2024-04-27T03:10:38.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30918.313842 secs 2024-04-27T03:10:38.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:37 smithi119 ceph-mon[32752]: pgmap v17047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:38.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30918.313842 secs 2024-04-27T03:10:38.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:37 smithi167 ceph-mon[32751]: pgmap v17047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:38.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30918.313842 secs 2024-04-27T03:10:38.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:37 smithi090 ceph-mon[29180]: pgmap v17047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:39.635 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:10:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:10:39.362+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:10:39.635 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:10:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:10:39.362+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:10:39.635 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:10:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:10:39.362+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:10:39.635 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:10:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:10:39.362+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:10:39.636 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:10:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:10:39.362+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:10:40.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:39 smithi119 ceph-mon[32752]: pgmap v17048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:40.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:39 smithi167 ceph-mon[32751]: pgmap v17048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:40.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:39 smithi090 ceph-mon[29180]: pgmap v17048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:40.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:40 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30922.463142 secs 2024-04-27T03:10:41.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:40 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30922.463142 secs 2024-04-27T03:10:41.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:40 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30922.463142 secs 2024-04-27T03:10:42.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:41 smithi119 ceph-mon[32752]: pgmap v17049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:42.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:41 smithi167 ceph-mon[32751]: pgmap v17049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:42.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:41 smithi090 ceph-mon[29180]: pgmap v17049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:10:43.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30923.313958 secs 2024-04-27T03:10:43.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30923.313958 secs 2024-04-27T03:10:43.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30923.313958 secs 2024-04-27T03:10:44.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:43 smithi119 ceph-mon[32752]: pgmap v17050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:44.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:43 smithi167 ceph-mon[32751]: pgmap v17050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:44.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:43 smithi090 ceph-mon[29180]: pgmap v17050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:45.936 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:45 smithi090 ceph-mon[29180]: pgmap v17051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:45.937 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:45 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30927.463203 secs 2024-04-27T03:10:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:45 smithi119 ceph-mon[32752]: pgmap v17051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:45 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30927.463203 secs 2024-04-27T03:10:46.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:45 smithi167 ceph-mon[32751]: pgmap v17051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:46.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:45 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30927.463203 secs 2024-04-27T03:10:48.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30928.314075 secs 2024-04-27T03:10:48.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:47 smithi119 ceph-mon[32752]: pgmap v17052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:48.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30928.314075 secs 2024-04-27T03:10:48.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:47 smithi167 ceph-mon[32751]: pgmap v17052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:48.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30928.314075 secs 2024-04-27T03:10:48.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:47 smithi090 ceph-mon[29180]: pgmap v17052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:49 smithi119 ceph-mon[32752]: pgmap v17053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:49 smithi167 ceph-mon[32751]: pgmap v17053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:50.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:49 smithi090 ceph-mon[29180]: pgmap v17053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:51.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:50 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30932.463286 secs 2024-04-27T03:10:51.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:50 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30932.463286 secs 2024-04-27T03:10:51.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:50 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30932.463286 secs 2024-04-27T03:10:52.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:51 smithi119 ceph-mon[32752]: pgmap v17054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:52.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:51 smithi167 ceph-mon[32751]: pgmap v17054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:52.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:51 smithi090 ceph-mon[29180]: pgmap v17054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:53.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30933.314147 secs 2024-04-27T03:10:53.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:52 smithi119 ceph-mon[32752]: pgmap v17055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:53.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30933.314147 secs 2024-04-27T03:10:53.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:52 smithi167 ceph-mon[32751]: pgmap v17055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:53.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30933.314147 secs 2024-04-27T03:10:53.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:52 smithi090 ceph-mon[29180]: pgmap v17055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:10:55.470 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:55 smithi090 ceph-mon[29180]: pgmap v17056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:55.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:55 smithi119 ceph-mon[32752]: pgmap v17056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:55.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:55 smithi167 ceph-mon[32751]: pgmap v17056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:10:56.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:56 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30937.463370 secs 2024-04-27T03:10:56.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:56 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30937.463370 secs 2024-04-27T03:10:56.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:56 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30937.463370 secs 2024-04-27T03:10:57.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30938.314247 secs 2024-04-27T03:10:57.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:57 smithi119 ceph-mon[32752]: pgmap v17057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:57.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30938.314247 secs 2024-04-27T03:10:57.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:57 smithi167 ceph-mon[32751]: pgmap v17057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:57.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30938.314247 secs 2024-04-27T03:10:57.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:57 smithi090 ceph-mon[29180]: pgmap v17057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:59.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:10:59 smithi119 ceph-mon[32752]: pgmap v17058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:59.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:10:59 smithi167 ceph-mon[32751]: pgmap v17058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:10:59.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:10:59 smithi090 ceph-mon[29180]: pgmap v17058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:01.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:01 smithi119 ceph-mon[32752]: pgmap v17059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:01.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:01 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30942.463455 secs 2024-04-27T03:11:01.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:01 smithi167 ceph-mon[32751]: pgmap v17059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:01.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:01 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30942.463455 secs 2024-04-27T03:11:01.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:01 smithi090 ceph-mon[29180]: pgmap v17059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:01.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:01 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30942.463455 secs 2024-04-27T03:11:02.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30943.314360 secs 2024-04-27T03:11:02.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30943.314360 secs 2024-04-27T03:11:02.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30943.314360 secs 2024-04-27T03:11:03.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:03 smithi119 ceph-mon[32752]: pgmap v17060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:03.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:11:03.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:11:03.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:11:03.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:11:03.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:03 smithi167 ceph-mon[32751]: pgmap v17060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:03.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:11:03.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:11:03.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:11:03.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:11:03.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:03 smithi090 ceph-mon[29180]: pgmap v17060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:03.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:11:03.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:11:03.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:11:03.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:11:04.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:04 smithi119 ceph-mon[32752]: pgmap v17061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:11:04.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:04 smithi119 ceph-mon[32752]: pgmap v17062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:11:04.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:04 smithi167 ceph-mon[32751]: pgmap v17061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:11:04.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:04 smithi167 ceph-mon[32751]: pgmap v17062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:11:04.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:04 smithi090 ceph-mon[29180]: pgmap v17061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:11:04.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:04 smithi090 ceph-mon[29180]: pgmap v17062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:11:06.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:06 smithi119 ceph-mon[32752]: pgmap v17063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:06.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:06 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30947.463538 secs 2024-04-27T03:11:06.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:06 smithi167 ceph-mon[32751]: pgmap v17063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:06.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:06 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30947.463538 secs 2024-04-27T03:11:06.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:06 smithi090 ceph-mon[29180]: pgmap v17063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:06.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:06 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30947.463538 secs 2024-04-27T03:11:07.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30948.314476 secs 2024-04-27T03:11:07.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30948.314476 secs 2024-04-27T03:11:07.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30948.314476 secs 2024-04-27T03:11:08.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:08 smithi119 ceph-mon[32752]: pgmap v17064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:08.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:08 smithi167 ceph-mon[32751]: pgmap v17064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:08.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:08 smithi090 ceph-mon[29180]: pgmap v17064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:10.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:10 smithi119 ceph-mon[32752]: pgmap v17065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:10.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:10 smithi167 ceph-mon[32751]: pgmap v17065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:10.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:10 smithi090 ceph-mon[29180]: pgmap v17065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:11.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:11 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30952.463621 secs 2024-04-27T03:11:11.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:11 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30952.463621 secs 2024-04-27T03:11:11.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:11 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30952.463621 secs 2024-04-27T03:11:12.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:12 smithi119 ceph-mon[32752]: pgmap v17066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:12.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30953.314589 secs 2024-04-27T03:11:12.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:12 smithi167 ceph-mon[32751]: pgmap v17066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:12.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30953.314589 secs 2024-04-27T03:11:12.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:12 smithi090 ceph-mon[29180]: pgmap v17066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:12.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30953.314589 secs 2024-04-27T03:11:14.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:14 smithi119 ceph-mon[32752]: pgmap v17067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:14.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:14 smithi167 ceph-mon[32751]: pgmap v17067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:14.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:14 smithi090 ceph-mon[29180]: pgmap v17067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:16.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:16 smithi119 ceph-mon[32752]: pgmap v17068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:16.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:16 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30957.463701 secs 2024-04-27T03:11:16.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:16 smithi167 ceph-mon[32751]: pgmap v17068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:16.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:16 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30957.463701 secs 2024-04-27T03:11:16.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:16 smithi090 ceph-mon[29180]: pgmap v17068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:11:16.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:16 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30957.463701 secs 2024-04-27T03:11:17.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30958.314668 secs 2024-04-27T03:11:17.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30958.314668 secs 2024-04-27T03:11:17.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30958.314668 secs 2024-04-27T03:11:18.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:18 smithi119 ceph-mon[32752]: pgmap v17069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:18.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:18 smithi167 ceph-mon[32751]: pgmap v17069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:18.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:18 smithi090 ceph-mon[29180]: pgmap v17069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:20.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:20 smithi119 ceph-mon[32752]: pgmap v17070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:20.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:20 smithi167 ceph-mon[32751]: pgmap v17070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:20.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:20 smithi090 ceph-mon[29180]: pgmap v17070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:21.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:21 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30962.463789 secs 2024-04-27T03:11:21.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:21 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30962.463789 secs 2024-04-27T03:11:21.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:21 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30962.463789 secs 2024-04-27T03:11:22.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:22 smithi119 ceph-mon[32752]: pgmap v17071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:22.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30963.314750 secs 2024-04-27T03:11:22.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:22 smithi167 ceph-mon[32751]: pgmap v17071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:22.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30963.314750 secs 2024-04-27T03:11:22.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:22 smithi090 ceph-mon[29180]: pgmap v17071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:22.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30963.314750 secs 2024-04-27T03:11:24.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:24 smithi119 ceph-mon[32752]: pgmap v17072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:24.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:24 smithi167 ceph-mon[32751]: pgmap v17072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:24.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:24 smithi090 ceph-mon[29180]: pgmap v17072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:26.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:26 smithi119 ceph-mon[32752]: pgmap v17073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:26.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:26 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30967.463843 secs 2024-04-27T03:11:26.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:26 smithi167 ceph-mon[32751]: pgmap v17073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:26.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:26 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30967.463843 secs 2024-04-27T03:11:26.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:26 smithi090 ceph-mon[29180]: pgmap v17073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:26.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:26 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30967.463843 secs 2024-04-27T03:11:27.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30968.314861 secs 2024-04-27T03:11:27.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30968.314861 secs 2024-04-27T03:11:27.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30968.314861 secs 2024-04-27T03:11:28.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:28 smithi119 ceph-mon[32752]: pgmap v17074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:28.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:28 smithi167 ceph-mon[32751]: pgmap v17074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:28.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:28 smithi090 ceph-mon[29180]: pgmap v17074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:30.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:30 smithi119 ceph-mon[32752]: pgmap v17075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:30.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:30 smithi167 ceph-mon[32751]: pgmap v17075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:30 smithi090 ceph-mon[29180]: pgmap v17075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:31.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:31 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30972.463912 secs 2024-04-27T03:11:31.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:31 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30972.463912 secs 2024-04-27T03:11:31.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:31 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30972.463912 secs 2024-04-27T03:11:32.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:32 smithi119 ceph-mon[32752]: pgmap v17076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:32.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30973.314998 secs 2024-04-27T03:11:32.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:32 smithi167 ceph-mon[32751]: pgmap v17076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:32.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30973.314998 secs 2024-04-27T03:11:32.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:32 smithi090 ceph-mon[29180]: pgmap v17076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:32.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30973.314998 secs 2024-04-27T03:11:34.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:34 smithi119 ceph-mon[32752]: pgmap v17077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:34.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:34 smithi167 ceph-mon[32751]: pgmap v17077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:34.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:34 smithi090 ceph-mon[29180]: pgmap v17077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:35.576 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:35 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30977.463995 secs 2024-04-27T03:11:35.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:35 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30977.463995 secs 2024-04-27T03:11:35.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:35 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30977.463995 secs 2024-04-27T03:11:36.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:36 smithi167 ceph-mon[32751]: pgmap v17078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:36.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:36 smithi090 ceph-mon[29180]: pgmap v17078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:36.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:36 smithi119 ceph-mon[32752]: pgmap v17078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:37.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30978.315123 secs 2024-04-27T03:11:37.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30978.315123 secs 2024-04-27T03:11:37.714 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30978.315123 secs 2024-04-27T03:11:38.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:38 smithi167 ceph-mon[32751]: pgmap v17079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:38.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:38 smithi090 ceph-mon[29180]: pgmap v17079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:38.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:38 smithi119 ceph-mon[32752]: pgmap v17079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:40.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:40 smithi090 ceph-mon[29180]: pgmap v17080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:40.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:40 smithi090 ceph-mon[29180]: 41 slow requests, 1 included below; oldest blocked for > 30982.464054 secs 2024-04-27T03:11:40.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:40 smithi090 ceph-mon[29180]: slow request 360.973577 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:11:40.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:40 smithi119 ceph-mon[32752]: pgmap v17080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:40.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:40 smithi119 ceph-mon[32752]: 41 slow requests, 1 included below; oldest blocked for > 30982.464054 secs 2024-04-27T03:11:40.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:40 smithi119 ceph-mon[32752]: slow request 360.973577 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:11:40.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:40 smithi167 ceph-mon[32751]: pgmap v17080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:40.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:40 smithi167 ceph-mon[32751]: 41 slow requests, 1 included below; oldest blocked for > 30982.464054 secs 2024-04-27T03:11:40.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:40 smithi167 ceph-mon[32751]: slow request 360.973577 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:11:42.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:42 smithi090 ceph-mon[29180]: pgmap v17081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:42.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30983.315274 secs 2024-04-27T03:11:42.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:42 smithi119 ceph-mon[32752]: pgmap v17081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:42.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30983.315274 secs 2024-04-27T03:11:42.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:42 smithi167 ceph-mon[32751]: pgmap v17081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:42.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30983.315274 secs 2024-04-27T03:11:44.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:44 smithi090 ceph-mon[29180]: pgmap v17082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:44.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:44 smithi119 ceph-mon[32752]: pgmap v17082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:44.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:44 smithi167 ceph-mon[32751]: pgmap v17082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:45.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:45 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30987.464135 secs 2024-04-27T03:11:45.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:45 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30987.464135 secs 2024-04-27T03:11:45.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:45 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30987.464135 secs 2024-04-27T03:11:46.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:46 smithi090 ceph-mon[29180]: pgmap v17083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:46.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:46 smithi119 ceph-mon[32752]: pgmap v17083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:46.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:46 smithi167 ceph-mon[32751]: pgmap v17083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:47.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30988.315401 secs 2024-04-27T03:11:47.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30988.315401 secs 2024-04-27T03:11:47.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30988.315401 secs 2024-04-27T03:11:48.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:48 smithi090 ceph-mon[29180]: pgmap v17084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:48.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:48 smithi119 ceph-mon[32752]: pgmap v17084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:48.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:48 smithi167 ceph-mon[32751]: pgmap v17084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:50.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:50 smithi090 ceph-mon[29180]: pgmap v17085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:50.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:50 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30992.464219 secs 2024-04-27T03:11:50.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:50 smithi119 ceph-mon[32752]: pgmap v17085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:50.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:50 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30992.464219 secs 2024-04-27T03:11:50.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:50 smithi167 ceph-mon[32751]: pgmap v17085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:50.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:50 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30992.464219 secs 2024-04-27T03:11:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:52 smithi090 ceph-mon[29180]: pgmap v17086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:52.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30993.315530 secs 2024-04-27T03:11:52.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:52 smithi119 ceph-mon[32752]: pgmap v17086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:52.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30993.315530 secs 2024-04-27T03:11:52.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:52 smithi167 ceph-mon[32751]: pgmap v17086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:52.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30993.315530 secs 2024-04-27T03:11:54.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:54 smithi090 ceph-mon[29180]: pgmap v17087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:54.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:54 smithi119 ceph-mon[32752]: pgmap v17087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:54.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:54 smithi167 ceph-mon[32751]: pgmap v17087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:55.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:55 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 30997.464279 secs 2024-04-27T03:11:55.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:55 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 30997.464279 secs 2024-04-27T03:11:55.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:55 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 30997.464279 secs 2024-04-27T03:11:56.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:56 smithi090 ceph-mon[29180]: pgmap v17088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:56.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:56 smithi119 ceph-mon[32752]: pgmap v17088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:56.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:56 smithi167 ceph-mon[32751]: pgmap v17088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:11:57.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 30998.315661 secs 2024-04-27T03:11:57.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 30998.315661 secs 2024-04-27T03:11:57.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 30998.315661 secs 2024-04-27T03:11:58.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:11:58 smithi090 ceph-mon[29180]: pgmap v17089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:58.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:11:58 smithi119 ceph-mon[32752]: pgmap v17089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:11:58.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:11:58 smithi167 ceph-mon[32751]: pgmap v17089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:00 smithi090 ceph-mon[29180]: pgmap v17090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:00.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:00 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31002.464365 secs 2024-04-27T03:12:00.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:00 smithi119 ceph-mon[32752]: pgmap v17090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:00.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:00 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31002.464365 secs 2024-04-27T03:12:00.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:00 smithi167 ceph-mon[32751]: pgmap v17090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:00.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:00 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31002.464365 secs 2024-04-27T03:12:02.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:02 smithi090 ceph-mon[29180]: pgmap v17091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:02.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31003.315807 secs 2024-04-27T03:12:02.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:02 smithi119 ceph-mon[32752]: pgmap v17091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:02.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31003.315807 secs 2024-04-27T03:12:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:02 smithi167 ceph-mon[32751]: pgmap v17091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31003.315807 secs 2024-04-27T03:12:03.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:12:03.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:12:03.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:12:04.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:04 smithi119 ceph-mon[32752]: pgmap v17092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:04.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:12:04.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:12:04.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:12:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:04 smithi167 ceph-mon[32751]: pgmap v17092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:12:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:12:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:12:04.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:04 smithi090 ceph-mon[29180]: pgmap v17092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:04.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:12:04.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:12:04.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:12:05.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:05 smithi119 ceph-mon[32752]: pgmap v17093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:12:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:05 smithi119 ceph-mon[32752]: pgmap v17094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:05 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31007.464422 secs 2024-04-27T03:12:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:05 smithi167 ceph-mon[32751]: pgmap v17093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:12:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:05 smithi167 ceph-mon[32751]: pgmap v17094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:05 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31007.464422 secs 2024-04-27T03:12:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:05 smithi090 ceph-mon[29180]: pgmap v17093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:12:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:05 smithi090 ceph-mon[29180]: pgmap v17094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:05 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31007.464422 secs 2024-04-27T03:12:07.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:07 smithi119 ceph-mon[32752]: pgmap v17095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:07.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31008.315960 secs 2024-04-27T03:12:07.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:07 smithi167 ceph-mon[32751]: pgmap v17095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:07.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31008.315960 secs 2024-04-27T03:12:07.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:07 smithi090 ceph-mon[29180]: pgmap v17095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:07.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31008.315960 secs 2024-04-27T03:12:09.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:09 smithi119 ceph-mon[32752]: pgmap v17096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:12:09.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:09 smithi167 ceph-mon[32751]: pgmap v17096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:12:09.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:09 smithi090 ceph-mon[29180]: pgmap v17096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:12:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:10 smithi119 ceph-mon[32752]: 41 slow requests, 1 included below; oldest blocked for > 31012.464506 secs 2024-04-27T03:12:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:10 smithi119 ceph-mon[32752]: slow request 11521.098705 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:12:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:10 smithi167 ceph-mon[32751]: 41 slow requests, 1 included below; oldest blocked for > 31012.464506 secs 2024-04-27T03:12:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:10 smithi167 ceph-mon[32751]: slow request 11521.098705 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:12:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:10 smithi090 ceph-mon[29180]: 41 slow requests, 1 included below; oldest blocked for > 31012.464506 secs 2024-04-27T03:12:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:10 smithi090 ceph-mon[29180]: slow request 11521.098705 seconds old, received at 2024-04-27T00:00:09.119206+0000: client_request(client.24733:267895 getattr AsXsFs #0x100000001fb 2024-04-27T00:00:09.118165+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:12:11.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:11 smithi119 ceph-mon[32752]: pgmap v17097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:11.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:11 smithi167 ceph-mon[32751]: pgmap v17097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:11.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:11 smithi090 ceph-mon[29180]: pgmap v17097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:12.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31013.316088 secs 2024-04-27T03:12:12.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31013.316088 secs 2024-04-27T03:12:12.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31013.316088 secs 2024-04-27T03:12:13.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:13 smithi119 ceph-mon[32752]: pgmap v17098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:13.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:13 smithi167 ceph-mon[32751]: pgmap v17098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:13.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:13 smithi090 ceph-mon[29180]: pgmap v17098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:12:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:15 smithi119 ceph-mon[32752]: pgmap v17099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:12:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:15 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31017.464587 secs 2024-04-27T03:12:15.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:15 smithi167 ceph-mon[32751]: pgmap v17099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:12:15.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:15 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31017.464587 secs 2024-04-27T03:12:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:15 smithi090 ceph-mon[29180]: pgmap v17099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:12:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:15 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31017.464587 secs 2024-04-27T03:12:17.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:17 smithi090 ceph-mon[29180]: pgmap v17100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:17.687 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31018.316230 secs 2024-04-27T03:12:17.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:17 smithi119 ceph-mon[32752]: pgmap v17100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:17.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31018.316230 secs 2024-04-27T03:12:17.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:17 smithi167 ceph-mon[32751]: pgmap v17100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:17.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31018.316230 secs 2024-04-27T03:12:19.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:19 smithi119 ceph-mon[32752]: pgmap v17101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:19.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:19 smithi167 ceph-mon[32751]: pgmap v17101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:19.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:19 smithi090 ceph-mon[29180]: pgmap v17101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:20.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:20 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31022.464672 secs 2024-04-27T03:12:20.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:20 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31022.464672 secs 2024-04-27T03:12:20.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:20 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31022.464672 secs 2024-04-27T03:12:21.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:21 smithi119 ceph-mon[32752]: pgmap v17102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:21.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:21 smithi167 ceph-mon[32751]: pgmap v17102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:21.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:21 smithi090 ceph-mon[29180]: pgmap v17102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:22.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31023.316361 secs 2024-04-27T03:12:22.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31023.316361 secs 2024-04-27T03:12:22.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31023.316361 secs 2024-04-27T03:12:23.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:23 smithi119 ceph-mon[32752]: pgmap v17103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:23.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:23 smithi167 ceph-mon[32751]: pgmap v17103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:23.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:23 smithi090 ceph-mon[29180]: pgmap v17103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:25.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:25 smithi119 ceph-mon[32752]: pgmap v17104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:25.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:25 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31027.464753 secs 2024-04-27T03:12:25.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:25 smithi167 ceph-mon[32751]: pgmap v17104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:25.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:25 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31027.464753 secs 2024-04-27T03:12:25.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:25 smithi090 ceph-mon[29180]: pgmap v17104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:25.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:25 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31027.464753 secs 2024-04-27T03:12:27.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:27 smithi119 ceph-mon[32752]: pgmap v17105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:27.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31028.316477 secs 2024-04-27T03:12:27.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:27 smithi167 ceph-mon[32751]: pgmap v17105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:27.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31028.316477 secs 2024-04-27T03:12:27.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:27 smithi090 ceph-mon[29180]: pgmap v17105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:27.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31028.316477 secs 2024-04-27T03:12:29.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:29 smithi119 ceph-mon[32752]: pgmap v17106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:29.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:29 smithi167 ceph-mon[32751]: pgmap v17106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:29.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:29 smithi090 ceph-mon[29180]: pgmap v17106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:30.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:30 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31032.464828 secs 2024-04-27T03:12:30.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:30 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31032.464828 secs 2024-04-27T03:12:30.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:30 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31032.464828 secs 2024-04-27T03:12:31.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:31 smithi119 ceph-mon[32752]: pgmap v17107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:31.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:31 smithi167 ceph-mon[32751]: pgmap v17107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:31.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:31 smithi090 ceph-mon[29180]: pgmap v17107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:32.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31033.316580 secs 2024-04-27T03:12:32.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31033.316580 secs 2024-04-27T03:12:32.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31033.316580 secs 2024-04-27T03:12:33.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:33 smithi119 ceph-mon[32752]: pgmap v17108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:33.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:33 smithi167 ceph-mon[32751]: pgmap v17108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:33.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:33 smithi090 ceph-mon[29180]: pgmap v17108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:35.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:35 smithi119 ceph-mon[32752]: pgmap v17109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:35.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:35 smithi119 ceph-mon[32752]: 41 slow requests, 1 included below; oldest blocked for > 31037.464912 secs 2024-04-27T03:12:35.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:35 smithi119 ceph-mon[32752]: slow request 1441.705252 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:12:35.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:35 smithi167 ceph-mon[32751]: pgmap v17109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:35.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:35 smithi167 ceph-mon[32751]: 41 slow requests, 1 included below; oldest blocked for > 31037.464912 secs 2024-04-27T03:12:35.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:35 smithi167 ceph-mon[32751]: slow request 1441.705252 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:12:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:35 smithi090 ceph-mon[29180]: pgmap v17109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:35 smithi090 ceph-mon[29180]: 41 slow requests, 1 included below; oldest blocked for > 31037.464912 secs 2024-04-27T03:12:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:35 smithi090 ceph-mon[29180]: slow request 1441.705252 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:12:37.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:37 smithi119 ceph-mon[32752]: pgmap v17110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:37.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31038.316643 secs 2024-04-27T03:12:37.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:37 smithi167 ceph-mon[32751]: pgmap v17110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:37.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31038.316643 secs 2024-04-27T03:12:37.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:37 smithi090 ceph-mon[29180]: pgmap v17110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:37.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31038.316643 secs 2024-04-27T03:12:39.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:39 smithi119 ceph-mon[32752]: pgmap v17111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:39.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:39 smithi167 ceph-mon[32751]: pgmap v17111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:39.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:39 smithi090 ceph-mon[29180]: pgmap v17111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:40.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:40 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31042.464994 secs 2024-04-27T03:12:40.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:40 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31042.464994 secs 2024-04-27T03:12:40.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:40 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31042.464994 secs 2024-04-27T03:12:41.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:41 smithi119 ceph-mon[32752]: pgmap v17112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:41.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:41 smithi167 ceph-mon[32751]: pgmap v17112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:41.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:41 smithi090 ceph-mon[29180]: pgmap v17112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:42.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:42 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 31043.316781 secs 2024-04-27T03:12:42.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:42 smithi119 ceph-mon[32752]: slow request 11524.889191 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:12:42.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:42 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 31043.316781 secs 2024-04-27T03:12:42.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:42 smithi167 ceph-mon[32751]: slow request 11524.889191 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:12:42.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:42 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 31043.316781 secs 2024-04-27T03:12:42.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:42 smithi090 ceph-mon[29180]: slow request 11524.889191 seconds old, received at 2024-04-27T00:00:36.178235+0000: client_request(client.17475:1 getattr pAsLsXsFs #0x1 2024-04-27T00:00:36.174407+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:12:43.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:43 smithi119 ceph-mon[32752]: pgmap v17113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:43.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:43 smithi167 ceph-mon[32751]: pgmap v17113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:43.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:43 smithi090 ceph-mon[29180]: pgmap v17113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:45.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:45 smithi119 ceph-mon[32752]: pgmap v17114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:45.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:45 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31047.465073 secs 2024-04-27T03:12:45.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:45 smithi167 ceph-mon[32751]: pgmap v17114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:45.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:45 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31047.465073 secs 2024-04-27T03:12:45.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:45 smithi090 ceph-mon[29180]: pgmap v17114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:45.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:45 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31047.465073 secs 2024-04-27T03:12:47.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:47 smithi119 ceph-mon[32752]: pgmap v17115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:47.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31048.316872 secs 2024-04-27T03:12:47.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:47 smithi167 ceph-mon[32751]: pgmap v17115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:47.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31048.316872 secs 2024-04-27T03:12:47.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:47 smithi090 ceph-mon[29180]: pgmap v17115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:47.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31048.316872 secs 2024-04-27T03:12:49.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:49 smithi119 ceph-mon[32752]: pgmap v17116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:49.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:49 smithi167 ceph-mon[32751]: pgmap v17116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:49.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:49 smithi090 ceph-mon[29180]: pgmap v17116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:50.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:50 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31052.465135 secs 2024-04-27T03:12:50.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:50 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31052.465135 secs 2024-04-27T03:12:50.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:50 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31052.465135 secs 2024-04-27T03:12:51.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:51 smithi119 ceph-mon[32752]: pgmap v17117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:51.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:51 smithi167 ceph-mon[32751]: pgmap v17117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:51.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:51 smithi090 ceph-mon[29180]: pgmap v17117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:52.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31053.316989 secs 2024-04-27T03:12:52.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31053.316989 secs 2024-04-27T03:12:52.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31053.316989 secs 2024-04-27T03:12:53.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:53 smithi119 ceph-mon[32752]: pgmap v17118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:53.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:53 smithi167 ceph-mon[32751]: pgmap v17118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:53.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:53 smithi090 ceph-mon[29180]: pgmap v17118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:55.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:55 smithi119 ceph-mon[32752]: pgmap v17119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:55.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:55 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31057.465216 secs 2024-04-27T03:12:55.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:55 smithi167 ceph-mon[32751]: pgmap v17119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:55.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:55 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31057.465216 secs 2024-04-27T03:12:55.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:55 smithi090 ceph-mon[29180]: pgmap v17119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:12:55.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:55 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31057.465216 secs 2024-04-27T03:12:57.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:57 smithi119 ceph-mon[32752]: pgmap v17120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:57.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31058.317055 secs 2024-04-27T03:12:57.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:57 smithi167 ceph-mon[32751]: pgmap v17120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:57.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31058.317055 secs 2024-04-27T03:12:57.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:57 smithi090 ceph-mon[29180]: pgmap v17120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:57.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31058.317055 secs 2024-04-27T03:12:59.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:12:59 smithi119 ceph-mon[32752]: pgmap v17121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:59.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:12:59 smithi167 ceph-mon[32751]: pgmap v17121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:12:59.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:12:59 smithi090 ceph-mon[29180]: pgmap v17121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:00.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:00 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31062.465301 secs 2024-04-27T03:13:00.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:00 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31062.465301 secs 2024-04-27T03:13:00.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:00 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31062.465301 secs 2024-04-27T03:13:01.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:01 smithi119 ceph-mon[32752]: pgmap v17122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:01.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:01 smithi167 ceph-mon[32751]: pgmap v17122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:01.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:01 smithi090 ceph-mon[29180]: pgmap v17122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:02.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31063.317117 secs 2024-04-27T03:13:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31063.317117 secs 2024-04-27T03:13:02.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31063.317117 secs 2024-04-27T03:13:03.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:03 smithi119 ceph-mon[32752]: pgmap v17123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:03.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:03 smithi167 ceph-mon[32751]: pgmap v17123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:03.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:03 smithi090 ceph-mon[29180]: pgmap v17123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:04.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:13:04.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:13:04.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:13:04.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:13:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:13:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:13:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:13:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:13:04.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:13:04.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:13:04.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:13:04.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:13:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:05 smithi119 ceph-mon[32752]: pgmap v17124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:05 smithi119 ceph-mon[32752]: pgmap v17125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:13:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:05 smithi119 ceph-mon[32752]: pgmap v17126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:05.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:05 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31067.465382 secs 2024-04-27T03:13:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:05 smithi167 ceph-mon[32751]: pgmap v17124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:05 smithi167 ceph-mon[32751]: pgmap v17125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:13:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:05 smithi167 ceph-mon[32751]: pgmap v17126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:05 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31067.465382 secs 2024-04-27T03:13:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:05 smithi090 ceph-mon[29180]: pgmap v17124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:05 smithi090 ceph-mon[29180]: pgmap v17125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:13:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:05 smithi090 ceph-mon[29180]: pgmap v17126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:05.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:05 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31067.465382 secs 2024-04-27T03:13:07.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31068.317217 secs 2024-04-27T03:13:07.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:07 smithi119 ceph-mon[32752]: pgmap v17127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:07.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31068.317217 secs 2024-04-27T03:13:07.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:07 smithi167 ceph-mon[32751]: pgmap v17127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:07.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31068.317217 secs 2024-04-27T03:13:07.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:07 smithi090 ceph-mon[29180]: pgmap v17127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:09.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:09 smithi119 ceph-mon[32752]: pgmap v17128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:09.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:09 smithi167 ceph-mon[32751]: pgmap v17128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:09.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:09 smithi090 ceph-mon[29180]: pgmap v17128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:10 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31072.465462 secs 2024-04-27T03:13:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:10 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31072.465462 secs 2024-04-27T03:13:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:10 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31072.465462 secs 2024-04-27T03:13:11.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:11 smithi119 ceph-mon[32752]: pgmap v17129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:11.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:11 smithi167 ceph-mon[32751]: pgmap v17129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:11.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:11 smithi090 ceph-mon[29180]: pgmap v17129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:13:12.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31073.317289 secs 2024-04-27T03:13:12.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31073.317289 secs 2024-04-27T03:13:13.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31073.317289 secs 2024-04-27T03:13:13.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:13 smithi167 ceph-mon[32751]: pgmap v17130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:13:13.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:13 smithi090 ceph-mon[29180]: pgmap v17130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:13:14.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:13 smithi119 ceph-mon[32752]: pgmap v17130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:13:15.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:15 smithi167 ceph-mon[32751]: pgmap v17131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:13:15.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:15 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31077.465540 secs 2024-04-27T03:13:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:15 smithi090 ceph-mon[29180]: pgmap v17131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:13:15.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:15 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31077.465540 secs 2024-04-27T03:13:16.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:15 smithi119 ceph-mon[32752]: pgmap v17131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:13:16.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:15 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31077.465540 secs 2024-04-27T03:13:17.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31078.317429 secs 2024-04-27T03:13:17.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:17 smithi167 ceph-mon[32751]: pgmap v17132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:17.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31078.317429 secs 2024-04-27T03:13:17.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:17 smithi090 ceph-mon[29180]: pgmap v17132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:18.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31078.317429 secs 2024-04-27T03:13:18.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:17 smithi119 ceph-mon[32752]: pgmap v17132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:19.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:19 smithi090 ceph-mon[29180]: pgmap v17133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:20.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:19 smithi119 ceph-mon[32752]: pgmap v17133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:20.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:19 smithi167 ceph-mon[32751]: pgmap v17133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:20.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:20 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31082.465624 secs 2024-04-27T03:13:21.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:20 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31082.465624 secs 2024-04-27T03:13:21.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:20 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31082.465624 secs 2024-04-27T03:13:21.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:21 smithi090 ceph-mon[29180]: pgmap v17134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:22.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:21 smithi119 ceph-mon[32752]: pgmap v17134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:22.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:21 smithi167 ceph-mon[32751]: pgmap v17134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:22.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31083.317550 secs 2024-04-27T03:13:23.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31083.317550 secs 2024-04-27T03:13:23.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31083.317550 secs 2024-04-27T03:13:23.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:23 smithi090 ceph-mon[29180]: pgmap v17135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:24.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:23 smithi119 ceph-mon[32752]: pgmap v17135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:24.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:23 smithi167 ceph-mon[32751]: pgmap v17135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:25.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:25 smithi090 ceph-mon[29180]: pgmap v17136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:25.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:25 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31087.465704 secs 2024-04-27T03:13:26.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:25 smithi119 ceph-mon[32752]: pgmap v17136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:26.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:25 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31087.465704 secs 2024-04-27T03:13:26.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:25 smithi167 ceph-mon[32751]: pgmap v17136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:26.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:25 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31087.465704 secs 2024-04-27T03:13:27.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31088.317665 secs 2024-04-27T03:13:27.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:27 smithi090 ceph-mon[29180]: pgmap v17137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:28.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31088.317665 secs 2024-04-27T03:13:28.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:27 smithi119 ceph-mon[32752]: pgmap v17137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:28.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31088.317665 secs 2024-04-27T03:13:28.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:27 smithi167 ceph-mon[32751]: pgmap v17137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:29.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:29 smithi090 ceph-mon[29180]: pgmap v17138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:30.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:29 smithi119 ceph-mon[32752]: pgmap v17138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:30.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:29 smithi167 ceph-mon[32751]: pgmap v17138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:30.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:30 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31092.465811 secs 2024-04-27T03:13:31.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:30 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31092.465811 secs 2024-04-27T03:13:31.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:30 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31092.465811 secs 2024-04-27T03:13:31.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:31 smithi090 ceph-mon[29180]: pgmap v17139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:32.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:31 smithi119 ceph-mon[32752]: pgmap v17139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:32.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:31 smithi167 ceph-mon[32751]: pgmap v17139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:32.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31093.317754 secs 2024-04-27T03:13:33.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31093.317754 secs 2024-04-27T03:13:33.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31093.317754 secs 2024-04-27T03:13:33.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:33 smithi090 ceph-mon[29180]: pgmap v17140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:34.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:33 smithi119 ceph-mon[32752]: pgmap v17140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:34.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:33 smithi167 ceph-mon[32751]: pgmap v17140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:35 smithi090 ceph-mon[29180]: pgmap v17141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:35.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:35 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31097.465896 secs 2024-04-27T03:13:36.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:35 smithi119 ceph-mon[32752]: pgmap v17141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:36.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:35 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31097.465896 secs 2024-04-27T03:13:36.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:35 smithi167 ceph-mon[32751]: pgmap v17141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:36.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:35 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31097.465896 secs 2024-04-27T03:13:37.880 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31098.317876 secs 2024-04-27T03:13:37.880 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:37 smithi167 ceph-mon[32751]: pgmap v17142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:37.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31098.317876 secs 2024-04-27T03:13:37.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:37 smithi090 ceph-mon[29180]: pgmap v17142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:38.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31098.317876 secs 2024-04-27T03:13:38.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:37 smithi119 ceph-mon[32752]: pgmap v17142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:39.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:39 smithi090 ceph-mon[29180]: pgmap v17143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:40.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:39 smithi119 ceph-mon[32752]: pgmap v17143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:40.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:39 smithi167 ceph-mon[32751]: pgmap v17143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:40.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:40 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31102.465959 secs 2024-04-27T03:13:41.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:40 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31102.465959 secs 2024-04-27T03:13:41.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:40 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31102.465959 secs 2024-04-27T03:13:41.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:41 smithi090 ceph-mon[29180]: pgmap v17144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:42.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:41 smithi119 ceph-mon[32752]: pgmap v17144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:42.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:41 smithi167 ceph-mon[32751]: pgmap v17144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:43.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31103.317990 secs 2024-04-27T03:13:43.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31103.317990 secs 2024-04-27T03:13:43.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31103.317990 secs 2024-04-27T03:13:44.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:43 smithi119 ceph-mon[32752]: pgmap v17145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:44.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:43 smithi167 ceph-mon[32751]: pgmap v17145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:44.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:43 smithi090 ceph-mon[29180]: pgmap v17145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:46.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:45 smithi119 ceph-mon[32752]: pgmap v17146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:46.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:45 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31107.466017 secs 2024-04-27T03:13:46.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:45 smithi167 ceph-mon[32751]: pgmap v17146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:46.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:45 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31107.466017 secs 2024-04-27T03:13:46.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:45 smithi090 ceph-mon[29180]: pgmap v17146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:46.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:45 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31107.466017 secs 2024-04-27T03:13:48.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31108.318113 secs 2024-04-27T03:13:48.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:47 smithi119 ceph-mon[32752]: pgmap v17147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:48.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31108.318113 secs 2024-04-27T03:13:48.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:47 smithi167 ceph-mon[32751]: pgmap v17147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:48.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31108.318113 secs 2024-04-27T03:13:48.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:47 smithi090 ceph-mon[29180]: pgmap v17147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:50.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:49 smithi119 ceph-mon[32752]: pgmap v17148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:50.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:49 smithi167 ceph-mon[32751]: pgmap v17148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:50.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:49 smithi090 ceph-mon[29180]: pgmap v17148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:51.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:50 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31112.466079 secs 2024-04-27T03:13:51.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:50 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31112.466079 secs 2024-04-27T03:13:51.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:50 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31112.466079 secs 2024-04-27T03:13:52.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:51 smithi119 ceph-mon[32752]: pgmap v17149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:52.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:51 smithi167 ceph-mon[32751]: pgmap v17149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:52.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:51 smithi090 ceph-mon[29180]: pgmap v17149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:53.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31113.318224 secs 2024-04-27T03:13:53.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31113.318224 secs 2024-04-27T03:13:53.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31113.318224 secs 2024-04-27T03:13:54.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:53 smithi119 ceph-mon[32752]: pgmap v17150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:54.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:53 smithi167 ceph-mon[32751]: pgmap v17150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:54.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:53 smithi090 ceph-mon[29180]: pgmap v17150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:56.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:55 smithi119 ceph-mon[32752]: pgmap v17151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:56.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:55 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31117.466137 secs 2024-04-27T03:13:56.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:55 smithi167 ceph-mon[32751]: pgmap v17151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:56.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:55 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31117.466137 secs 2024-04-27T03:13:56.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:55 smithi090 ceph-mon[29180]: pgmap v17151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:13:56.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:55 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31117.466137 secs 2024-04-27T03:13:58.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31118.318341 secs 2024-04-27T03:13:58.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:57 smithi119 ceph-mon[32752]: pgmap v17152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:58.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31118.318341 secs 2024-04-27T03:13:58.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:57 smithi167 ceph-mon[32751]: pgmap v17152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:13:58.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31118.318341 secs 2024-04-27T03:13:58.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:57 smithi090 ceph-mon[29180]: pgmap v17152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:00.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:13:59 smithi119 ceph-mon[32752]: pgmap v17153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:00.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:13:59 smithi167 ceph-mon[32751]: pgmap v17153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:00.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:13:59 smithi090 ceph-mon[29180]: pgmap v17153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:01.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:00 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31122.466198 secs 2024-04-27T03:14:01.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:00 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31122.466198 secs 2024-04-27T03:14:01.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:00 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31122.466198 secs 2024-04-27T03:14:02.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:01 smithi119 ceph-mon[32752]: pgmap v17154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:02.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:01 smithi167 ceph-mon[32751]: pgmap v17154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:02.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:01 smithi090 ceph-mon[29180]: pgmap v17154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:03.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31123.318472 secs 2024-04-27T03:14:03.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31123.318472 secs 2024-04-27T03:14:03.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31123.318472 secs 2024-04-27T03:14:04.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:03 smithi119 ceph-mon[32752]: pgmap v17155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:04.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:03 smithi167 ceph-mon[32751]: pgmap v17155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:04.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:03 smithi090 ceph-mon[29180]: pgmap v17155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:05.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:14:05.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:14:05.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:14:05.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:05 smithi090 ceph-mon[29180]: pgmap v17156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:05.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:05.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:05.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:05.992 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:05.992 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:05 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31127.466277 secs 2024-04-27T03:14:06.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:05 smithi119 ceph-mon[32752]: pgmap v17156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:06.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:05 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31127.466277 secs 2024-04-27T03:14:06.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:05 smithi167 ceph-mon[32751]: pgmap v17156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:06.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:06.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:05 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31127.466277 secs 2024-04-27T03:14:08.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31128.318539 secs 2024-04-27T03:14:08.166 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: pgmap v17157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:08.166 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.167 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.168 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.168 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31128.318539 secs 2024-04-27T03:14:08.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: pgmap v17157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:08.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.169 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.170 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31128.318539 secs 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: pgmap v17157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.171 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:08.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:08.172 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:14:09.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:14:09.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.065 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:14:09.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:14:09.066 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:09.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:14:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:14:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:14:09.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:14:09.079 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:09.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:14:09.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:14:09.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:14:09.137 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:14:10.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:14:10.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:09 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:14:10.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:14:10.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:09 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:14:10.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:09 smithi119 ceph-mon[32752]: pgmap v17158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T03:14:10.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:09 smithi119 ceph-mon[32752]: pgmap v17159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-27T03:14:10.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:14:10.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:09 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:14:10.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:14:10.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:09 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:14:10.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:09 smithi167 ceph-mon[32751]: pgmap v17158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T03:14:10.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:09 smithi167 ceph-mon[32751]: pgmap v17159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-27T03:14:10.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:14:10.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:09 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:14:10.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:14:10.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:09 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:14:10.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:09 smithi090 ceph-mon[29180]: pgmap v17158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T03:14:10.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:09 smithi090 ceph-mon[29180]: pgmap v17159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 B/s rd, 1 op/s 2024-04-27T03:14:11.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:10 smithi119 ceph-mon[32752]: pgmap v17160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:11.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:10 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31132.466353 secs 2024-04-27T03:14:11.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:10 smithi167 ceph-mon[32751]: pgmap v17160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:11.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:10 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31132.466353 secs 2024-04-27T03:14:11.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:10 smithi090 ceph-mon[29180]: pgmap v17160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:11.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:10 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31132.466353 secs 2024-04-27T03:14:12.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:12 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 31133.318603 secs 2024-04-27T03:14:12.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:12 smithi119 ceph-mon[32752]: slow request 1442.062033 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:14:12.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:12 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 31133.318603 secs 2024-04-27T03:14:12.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:12 smithi167 ceph-mon[32751]: slow request 1442.062033 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:14:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:12 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 31133.318603 secs 2024-04-27T03:14:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:12 smithi090 ceph-mon[29180]: slow request 1442.062033 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:14:13.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:13 smithi119 ceph-mon[32752]: pgmap v17161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:13.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:13 smithi167 ceph-mon[32751]: pgmap v17161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:13.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:13 smithi090 ceph-mon[29180]: pgmap v17161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:15.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:15 smithi119 ceph-mon[32752]: pgmap v17162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:15.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:15 smithi167 ceph-mon[32751]: pgmap v17162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:15.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:15 smithi090 ceph-mon[29180]: pgmap v17162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:16.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:16 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31137.466437 secs 2024-04-27T03:14:16.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:16 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31137.466437 secs 2024-04-27T03:14:16.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:16 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31137.466437 secs 2024-04-27T03:14:17.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:17 smithi119 ceph-mon[32752]: pgmap v17163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:17.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31138.318710 secs 2024-04-27T03:14:17.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:17 smithi167 ceph-mon[32751]: pgmap v17163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:17.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31138.318710 secs 2024-04-27T03:14:17.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:17 smithi090 ceph-mon[29180]: pgmap v17163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:14:17.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31138.318710 secs 2024-04-27T03:14:19.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:19 smithi119 ceph-mon[32752]: pgmap v17164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:14:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:19 smithi167 ceph-mon[32751]: pgmap v17164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:14:19.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:19 smithi090 ceph-mon[29180]: pgmap v17164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:14:21.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:21 smithi167 ceph-mon[32751]: pgmap v17165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:21.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:21 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31142.466521 secs 2024-04-27T03:14:21.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:21 smithi090 ceph-mon[29180]: pgmap v17165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:21.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:21 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31142.466521 secs 2024-04-27T03:14:21.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:21 smithi119 ceph-mon[32752]: pgmap v17165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:21.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:21 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31142.466521 secs 2024-04-27T03:14:22.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31143.318826 secs 2024-04-27T03:14:22.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31143.318826 secs 2024-04-27T03:14:22.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31143.318826 secs 2024-04-27T03:14:23.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:23 smithi167 ceph-mon[32751]: pgmap v17166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:23.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:23 smithi090 ceph-mon[29180]: pgmap v17166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:23.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:23 smithi119 ceph-mon[32752]: pgmap v17166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:25.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:25 smithi090 ceph-mon[29180]: pgmap v17167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:25.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:25 smithi119 ceph-mon[32752]: pgmap v17167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:25.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:25 smithi167 ceph-mon[32751]: pgmap v17167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:26.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:26 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31147.466603 secs 2024-04-27T03:14:26.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:26 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31147.466603 secs 2024-04-27T03:14:26.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:26 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31147.466603 secs 2024-04-27T03:14:27.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:27 smithi090 ceph-mon[29180]: pgmap v17168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:27.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31148.318948 secs 2024-04-27T03:14:27.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:27 smithi119 ceph-mon[32752]: pgmap v17168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:27.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31148.318948 secs 2024-04-27T03:14:27.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:27 smithi167 ceph-mon[32751]: pgmap v17168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:27.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31148.318948 secs 2024-04-27T03:14:29.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:29 smithi090 ceph-mon[29180]: pgmap v17169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:29.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:29 smithi119 ceph-mon[32752]: pgmap v17169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:29.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:29 smithi167 ceph-mon[32751]: pgmap v17169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:31.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:31 smithi090 ceph-mon[29180]: pgmap v17170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:31.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:31 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31152.466682 secs 2024-04-27T03:14:31.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:31 smithi119 ceph-mon[32752]: pgmap v17170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:31.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:31 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31152.466682 secs 2024-04-27T03:14:31.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:31 smithi167 ceph-mon[32751]: pgmap v17170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:31.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:31 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31152.466682 secs 2024-04-27T03:14:32.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31153.319059 secs 2024-04-27T03:14:32.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31153.319059 secs 2024-04-27T03:14:32.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31153.319059 secs 2024-04-27T03:14:33.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:33 smithi090 ceph-mon[29180]: pgmap v17171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:33.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:33 smithi119 ceph-mon[32752]: pgmap v17171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:33.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:33 smithi167 ceph-mon[32751]: pgmap v17171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:35.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:35 smithi090 ceph-mon[29180]: pgmap v17172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:35.565 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:35 smithi119 ceph-mon[32752]: pgmap v17172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:35.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:35 smithi167 ceph-mon[32751]: pgmap v17172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:36.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:36 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31157.466767 secs 2024-04-27T03:14:36.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:36 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31157.466767 secs 2024-04-27T03:14:36.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:36 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31157.466767 secs 2024-04-27T03:14:37.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:37 smithi090 ceph-mon[29180]: pgmap v17173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:37.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31158.319150 secs 2024-04-27T03:14:37.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:37 smithi167 ceph-mon[32751]: pgmap v17173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:37.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31158.319150 secs 2024-04-27T03:14:37.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:37 smithi119 ceph-mon[32752]: pgmap v17173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:37.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31158.319150 secs 2024-04-27T03:14:39.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:39 smithi119 ceph-mon[32752]: pgmap v17174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:39.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:39 smithi090 ceph-mon[29180]: pgmap v17174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:39.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:39 smithi167 ceph-mon[32751]: pgmap v17174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:41.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:40 smithi119 ceph-mon[32752]: pgmap v17175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:41.064 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:40 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31162.466830 secs 2024-04-27T03:14:41.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:40 smithi167 ceph-mon[32751]: pgmap v17175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:41.078 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:40 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31162.466830 secs 2024-04-27T03:14:41.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:40 smithi090 ceph-mon[29180]: pgmap v17175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:41.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:40 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31162.466830 secs 2024-04-27T03:14:42.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31163.319269 secs 2024-04-27T03:14:42.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31163.319269 secs 2024-04-27T03:14:42.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31163.319269 secs 2024-04-27T03:14:43.313 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:43 smithi119 ceph-mon[32752]: pgmap v17176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:43.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:43 smithi167 ceph-mon[32751]: pgmap v17176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:43.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:43 smithi090 ceph-mon[29180]: pgmap v17176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:45.314 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:45 smithi119 ceph-mon[32752]: pgmap v17177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:45.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:45 smithi167 ceph-mon[32751]: pgmap v17177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:45.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:45 smithi090 ceph-mon[29180]: pgmap v17177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:46.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:46 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31167.466888 secs 2024-04-27T03:14:46.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:46 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31167.466888 secs 2024-04-27T03:14:46.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:46 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31167.466888 secs 2024-04-27T03:14:47.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:47 smithi167 ceph-mon[32751]: pgmap v17178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:47.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31168.319394 secs 2024-04-27T03:14:47.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:47 smithi090 ceph-mon[29180]: pgmap v17178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:47.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31168.319394 secs 2024-04-27T03:14:47.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:47 smithi119 ceph-mon[32752]: pgmap v17178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:47.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31168.319394 secs 2024-04-27T03:14:49.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:49 smithi167 ceph-mon[32751]: pgmap v17179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:49.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:49 smithi090 ceph-mon[29180]: pgmap v17179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:49.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:49 smithi119 ceph-mon[32752]: pgmap v17179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:51.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:51 smithi167 ceph-mon[32751]: pgmap v17180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:51.328 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:51 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31172.466974 secs 2024-04-27T03:14:51.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:51 smithi090 ceph-mon[29180]: pgmap v17180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:51.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:51 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31172.466974 secs 2024-04-27T03:14:51.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:51 smithi119 ceph-mon[32752]: pgmap v17180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:51.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:51 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31172.466974 secs 2024-04-27T03:14:52.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31173.319468 secs 2024-04-27T03:14:52.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31173.319468 secs 2024-04-27T03:14:52.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31173.319468 secs 2024-04-27T03:14:53.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:53 smithi090 ceph-mon[29180]: pgmap v17181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:53.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:53 smithi119 ceph-mon[32752]: pgmap v17181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:53.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:53 smithi167 ceph-mon[32751]: pgmap v17181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:55.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:55 smithi090 ceph-mon[29180]: pgmap v17182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:55.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:55 smithi119 ceph-mon[32752]: pgmap v17182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:55.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:55 smithi167 ceph-mon[32751]: pgmap v17182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:14:56.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:56 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31177.467035 secs 2024-04-27T03:14:56.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:56 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31177.467035 secs 2024-04-27T03:14:56.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:56 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31177.467035 secs 2024-04-27T03:14:57.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:57 smithi090 ceph-mon[29180]: pgmap v17183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:57.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31178.319592 secs 2024-04-27T03:14:57.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:57 smithi119 ceph-mon[32752]: pgmap v17183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:57.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31178.319592 secs 2024-04-27T03:14:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:57 smithi167 ceph-mon[32751]: pgmap v17183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:57.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31178.319592 secs 2024-04-27T03:14:59.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:14:59 smithi090 ceph-mon[29180]: pgmap v17184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:59.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:14:59 smithi119 ceph-mon[32752]: pgmap v17184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:14:59.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:14:59 smithi167 ceph-mon[32751]: pgmap v17184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:01.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:01 smithi090 ceph-mon[29180]: pgmap v17185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:01.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:01 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31182.467094 secs 2024-04-27T03:15:01.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:01 smithi119 ceph-mon[32752]: pgmap v17185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:01.566 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:01 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31182.467094 secs 2024-04-27T03:15:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:01 smithi167 ceph-mon[32751]: pgmap v17185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:01.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:01 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31182.467094 secs 2024-04-27T03:15:02.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31183.319711 secs 2024-04-27T03:15:02.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31183.319711 secs 2024-04-27T03:15:02.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31183.319711 secs 2024-04-27T03:15:03.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:03 smithi090 ceph-mon[29180]: pgmap v17186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:03.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:03 smithi119 ceph-mon[32752]: pgmap v17186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:03.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:03 smithi167 ceph-mon[32751]: pgmap v17186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:05.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:05 smithi090 ceph-mon[29180]: pgmap v17187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:05.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:05 smithi119 ceph-mon[32752]: pgmap v17187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:05.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:05 smithi167 ceph-mon[32751]: pgmap v17187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:06.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:06 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31187.467181 secs 2024-04-27T03:15:06.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:06 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31187.467181 secs 2024-04-27T03:15:06.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:06 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31187.467181 secs 2024-04-27T03:15:07.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:07 smithi090 ceph-mon[29180]: pgmap v17188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:07.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31188.319825 secs 2024-04-27T03:15:07.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:07 smithi167 ceph-mon[32751]: pgmap v17188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:07.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31188.319825 secs 2024-04-27T03:15:07.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:07 smithi119 ceph-mon[32752]: pgmap v17188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:07.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31188.319825 secs 2024-04-27T03:15:09.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:09 smithi119 ceph-mon[32752]: pgmap v17189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:09.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:15:09.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:09.366 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:09.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:09 smithi090 ceph-mon[29180]: pgmap v17189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:09.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:15:09.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:09.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:09 smithi167 ceph-mon[32751]: pgmap v17189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:09.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:15:09.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:09.579 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:10.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:15:10.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:15:10.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:10 smithi090 ceph-mon[29180]: pgmap v17190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:15:10.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:10 smithi090 ceph-mon[29180]: pgmap v17191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:10.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:10.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:15:10.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:15:10.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:10 smithi119 ceph-mon[32752]: pgmap v17190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:15:10.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:10 smithi119 ceph-mon[32752]: pgmap v17191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:10.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:15:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:15:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:10 smithi167 ceph-mon[32751]: pgmap v17190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:15:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:10 smithi167 ceph-mon[32751]: pgmap v17191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:10.578 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:15:11.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:11 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31192.467270 secs 2024-04-27T03:15:11.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:11 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31192.467270 secs 2024-04-27T03:15:11.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:11 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31192.467270 secs 2024-04-27T03:15:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31193.319941 secs 2024-04-27T03:15:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:12 smithi090 ceph-mon[29180]: pgmap v17192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:12.563 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31193.319941 secs 2024-04-27T03:15:12.564 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:12 smithi119 ceph-mon[32752]: pgmap v17192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:12.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31193.319941 secs 2024-04-27T03:15:12.577 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:12 smithi167 ceph-mon[32751]: pgmap v17192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:14.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:14 smithi090 ceph-mon[29180]: pgmap v17193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T03:15:14.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:14 smithi119 ceph-mon[32752]: pgmap v17193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T03:15:14.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:14 smithi167 ceph-mon[32751]: pgmap v17193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T03:15:15.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:15 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31197.467347 secs 2024-04-27T03:15:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:15 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31197.467347 secs 2024-04-27T03:15:15.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:15 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31197.467347 secs 2024-04-27T03:15:16.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:16 smithi090 ceph-mon[29180]: pgmap v17194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:16.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:16 smithi119 ceph-mon[32752]: pgmap v17194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:16.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:16 smithi167 ceph-mon[32751]: pgmap v17194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:17.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31198.320077 secs 2024-04-27T03:15:17.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31198.320077 secs 2024-04-27T03:15:17.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31198.320077 secs 2024-04-27T03:15:18.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:18 smithi090 ceph-mon[29180]: pgmap v17195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:18.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:18 smithi119 ceph-mon[32752]: pgmap v17195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:18.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:18 smithi167 ceph-mon[32751]: pgmap v17195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:15:20.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:20 smithi090 ceph-mon[29180]: pgmap v17196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:15:20.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:20 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31202.467431 secs 2024-04-27T03:15:20.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:20 smithi119 ceph-mon[32752]: pgmap v17196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:15:20.814 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:20 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31202.467431 secs 2024-04-27T03:15:20.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:20 smithi167 ceph-mon[32751]: pgmap v17196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:15:20.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:20 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31202.467431 secs 2024-04-27T03:15:22.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31203.320173 secs 2024-04-27T03:15:22.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:22 smithi090 ceph-mon[29180]: pgmap v17197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:22.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31203.320173 secs 2024-04-27T03:15:22.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:22 smithi119 ceph-mon[32752]: pgmap v17197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:22.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31203.320173 secs 2024-04-27T03:15:22.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:22 smithi167 ceph-mon[32751]: pgmap v17197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:24.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:24 smithi090 ceph-mon[29180]: pgmap v17198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:24.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:24 smithi119 ceph-mon[32752]: pgmap v17198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:24.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:24 smithi167 ceph-mon[32751]: pgmap v17198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:25.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:25 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31207.467513 secs 2024-04-27T03:15:25.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:25 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31207.467513 secs 2024-04-27T03:15:25.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:25 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31207.467513 secs 2024-04-27T03:15:26.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:26 smithi090 ceph-mon[29180]: pgmap v17199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:26.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:26 smithi119 ceph-mon[32752]: pgmap v17199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:26.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:26 smithi167 ceph-mon[32751]: pgmap v17199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:27.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31208.320311 secs 2024-04-27T03:15:27.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31208.320311 secs 2024-04-27T03:15:27.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31208.320311 secs 2024-04-27T03:15:28.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:28 smithi090 ceph-mon[29180]: pgmap v17200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:28.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:28 smithi119 ceph-mon[32752]: pgmap v17200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:28.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:28 smithi167 ceph-mon[32751]: pgmap v17200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:30 smithi090 ceph-mon[29180]: pgmap v17201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:30.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:30 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31212.467573 secs 2024-04-27T03:15:30.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:30 smithi119 ceph-mon[32752]: pgmap v17201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:30.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:30 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31212.467573 secs 2024-04-27T03:15:30.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:30 smithi167 ceph-mon[32751]: pgmap v17201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:30.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:30 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31212.467573 secs 2024-04-27T03:15:32.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31213.320429 secs 2024-04-27T03:15:32.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:32 smithi090 ceph-mon[29180]: pgmap v17202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:32.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31213.320429 secs 2024-04-27T03:15:32.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:32 smithi119 ceph-mon[32752]: pgmap v17202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:32.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31213.320429 secs 2024-04-27T03:15:32.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:32 smithi167 ceph-mon[32751]: pgmap v17202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:34.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:34 smithi119 ceph-mon[32752]: pgmap v17203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:34.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:34 smithi167 ceph-mon[32751]: pgmap v17203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:34.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:34 smithi090 ceph-mon[29180]: pgmap v17203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:35.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:35 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31217.467646 secs 2024-04-27T03:15:35.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:35 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31217.467646 secs 2024-04-27T03:15:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:35 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31217.467646 secs 2024-04-27T03:15:36.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:36 smithi119 ceph-mon[32752]: pgmap v17204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:36.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:36 smithi167 ceph-mon[32751]: pgmap v17204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:36.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:36 smithi090 ceph-mon[29180]: pgmap v17204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:37.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31218.320575 secs 2024-04-27T03:15:37.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31218.320575 secs 2024-04-27T03:15:37.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31218.320575 secs 2024-04-27T03:15:38.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:38 smithi119 ceph-mon[32752]: pgmap v17205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:38.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:38 smithi167 ceph-mon[32751]: pgmap v17205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:38.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:38 smithi090 ceph-mon[29180]: pgmap v17205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:40.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:40 smithi119 ceph-mon[32752]: pgmap v17206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:40.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:40 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31222.467738 secs 2024-04-27T03:15:40.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:40 smithi167 ceph-mon[32751]: pgmap v17206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:40.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:40 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31222.467738 secs 2024-04-27T03:15:40.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:40 smithi090 ceph-mon[29180]: pgmap v17206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:40.886 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:40 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31222.467738 secs 2024-04-27T03:15:42.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31223.320708 secs 2024-04-27T03:15:42.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:42 smithi119 ceph-mon[32752]: pgmap v17207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:42.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31223.320708 secs 2024-04-27T03:15:42.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:42 smithi167 ceph-mon[32751]: pgmap v17207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:42.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31223.320708 secs 2024-04-27T03:15:42.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:42 smithi090 ceph-mon[29180]: pgmap v17207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:44.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:44 smithi119 ceph-mon[32752]: pgmap v17208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:44.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:44 smithi167 ceph-mon[32751]: pgmap v17208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:44.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:44 smithi090 ceph-mon[29180]: pgmap v17208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:45.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:45 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31227.467819 secs 2024-04-27T03:15:45.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:45 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31227.467819 secs 2024-04-27T03:15:45.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:45 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31227.467819 secs 2024-04-27T03:15:46.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:46 smithi119 ceph-mon[32752]: pgmap v17209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:46.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:46 smithi167 ceph-mon[32751]: pgmap v17209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:46.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:46 smithi090 ceph-mon[29180]: pgmap v17209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:47.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31228.320787 secs 2024-04-27T03:15:47.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31228.320787 secs 2024-04-27T03:15:47.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31228.320787 secs 2024-04-27T03:15:48.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:48 smithi119 ceph-mon[32752]: pgmap v17210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:48.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:48 smithi167 ceph-mon[32751]: pgmap v17210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:48.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:48 smithi090 ceph-mon[29180]: pgmap v17210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:50.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:50 smithi119 ceph-mon[32752]: pgmap v17211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:50.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:50 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31232.467904 secs 2024-04-27T03:15:50.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:50 smithi167 ceph-mon[32751]: pgmap v17211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:50.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:50 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31232.467904 secs 2024-04-27T03:15:50.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:50 smithi090 ceph-mon[29180]: pgmap v17211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:50.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:50 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31232.467904 secs 2024-04-27T03:15:52.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31233.320906 secs 2024-04-27T03:15:52.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:52 smithi119 ceph-mon[32752]: pgmap v17212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:52.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31233.320906 secs 2024-04-27T03:15:52.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:52 smithi167 ceph-mon[32751]: pgmap v17212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:52.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31233.320906 secs 2024-04-27T03:15:52.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:52 smithi090 ceph-mon[29180]: pgmap v17212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:54.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:54 smithi119 ceph-mon[32752]: pgmap v17213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:54.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:54 smithi167 ceph-mon[32751]: pgmap v17213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:54.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:54 smithi090 ceph-mon[29180]: pgmap v17213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:15:55.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:55 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31237.467986 secs 2024-04-27T03:15:55.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:55 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31237.467986 secs 2024-04-27T03:15:55.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:55 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31237.467986 secs 2024-04-27T03:15:56.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:56 smithi119 ceph-mon[32752]: pgmap v17214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:56.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:56 smithi167 ceph-mon[32751]: pgmap v17214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:56.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:56 smithi090 ceph-mon[29180]: pgmap v17214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:15:57.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31238.321038 secs 2024-04-27T03:15:57.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31238.321038 secs 2024-04-27T03:15:57.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31238.321038 secs 2024-04-27T03:15:58.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:15:58 smithi119 ceph-mon[32752]: pgmap v17215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:58.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:15:58 smithi167 ceph-mon[32751]: pgmap v17215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:15:58.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:15:58 smithi090 ceph-mon[29180]: pgmap v17215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:00.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:00 smithi119 ceph-mon[32752]: pgmap v17216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:00.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:00 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31242.468048 secs 2024-04-27T03:16:00.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:00 smithi167 ceph-mon[32751]: pgmap v17216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:00.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:00 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31242.468048 secs 2024-04-27T03:16:00.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:00 smithi090 ceph-mon[29180]: pgmap v17216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:00.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:00 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31242.468048 secs 2024-04-27T03:16:02.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31243.321109 secs 2024-04-27T03:16:02.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:02 smithi119 ceph-mon[32752]: pgmap v17217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31243.321109 secs 2024-04-27T03:16:02.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:02 smithi167 ceph-mon[32751]: pgmap v17217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:02.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31243.321109 secs 2024-04-27T03:16:02.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:02 smithi090 ceph-mon[29180]: pgmap v17217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:04.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:04 smithi119 ceph-mon[32752]: pgmap v17218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:04.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:04 smithi167 ceph-mon[32751]: pgmap v17218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:04.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:04 smithi090 ceph-mon[29180]: pgmap v17218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:05.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:05 smithi119 ceph-mon[32752]: 41 slow requests, 0 included below; oldest blocked for > 31247.468105 secs 2024-04-27T03:16:05.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:05 smithi167 ceph-mon[32751]: 41 slow requests, 0 included below; oldest blocked for > 31247.468105 secs 2024-04-27T03:16:05.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:05 smithi090 ceph-mon[29180]: 41 slow requests, 0 included below; oldest blocked for > 31247.468105 secs 2024-04-27T03:16:06.766 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:06 smithi090 ceph-mon[29180]: pgmap v17219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:06.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:06 smithi119 ceph-mon[32752]: pgmap v17219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:06.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:06 smithi167 ceph-mon[32751]: pgmap v17219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:07.728 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31248.321220 secs 2024-04-27T03:16:07.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31248.321220 secs 2024-04-27T03:16:07.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31248.321220 secs 2024-04-27T03:16:08.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:08 smithi119 ceph-mon[32752]: pgmap v17220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:08.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:08 smithi167 ceph-mon[32751]: pgmap v17220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:08.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:08 smithi090 ceph-mon[29180]: pgmap v17220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:09.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:16:09.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:16:09.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:16:09.885 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:16:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:16:09.413+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:16:09.886 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:16:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:16:09.413+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:16:09.886 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:16:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:16:09.413+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:16:09.886 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:16:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:16:09.413+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:16:09.886 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:16:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:16:09.413+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:16:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:10 smithi119 ceph-mon[32752]: pgmap v17221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:16:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:16:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:16:10.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31252.468194 secs 2024-04-27T03:16:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:10 smithi167 ceph-mon[32751]: pgmap v17221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:16:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:16:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:16:10.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31252.468194 secs 2024-04-27T03:16:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:10 smithi090 ceph-mon[29180]: pgmap v17221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:16:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:16:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:16:10.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31252.468194 secs 2024-04-27T03:16:11.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:11 smithi119 ceph-mon[32752]: pgmap v17222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T03:16:11.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:11 smithi119 ceph-mon[32752]: pgmap v17223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:16:11.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:11 smithi167 ceph-mon[32751]: pgmap v17222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T03:16:11.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:11 smithi167 ceph-mon[32751]: pgmap v17223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:16:11.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:11 smithi090 ceph-mon[29180]: pgmap v17222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T03:16:11.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:11 smithi090 ceph-mon[29180]: pgmap v17223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:16:12.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31253.321332 secs 2024-04-27T03:16:12.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31253.321332 secs 2024-04-27T03:16:12.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31253.321332 secs 2024-04-27T03:16:13.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:13 smithi119 ceph-mon[32752]: pgmap v17224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T03:16:13.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:13 smithi167 ceph-mon[32751]: pgmap v17224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T03:16:13.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:13 smithi090 ceph-mon[29180]: pgmap v17224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T03:16:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:15 smithi119 ceph-mon[32752]: pgmap v17225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-27T03:16:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:16:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:16:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31257.468271 secs 2024-04-27T03:16:15.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:15 smithi167 ceph-mon[32751]: pgmap v17225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-27T03:16:15.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:16:15.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:16:15.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31257.468271 secs 2024-04-27T03:16:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:15 smithi090 ceph-mon[29180]: pgmap v17225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-27T03:16:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:16:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:16:15.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31257.468271 secs 2024-04-27T03:16:17.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31258.321450 secs 2024-04-27T03:16:17.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:17 smithi119 ceph-mon[32752]: pgmap v17226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-27T03:16:17.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31258.321450 secs 2024-04-27T03:16:17.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:17 smithi167 ceph-mon[32751]: pgmap v17226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-27T03:16:17.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31258.321450 secs 2024-04-27T03:16:17.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:17 smithi090 ceph-mon[29180]: pgmap v17226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 232 B/s wr, 2 op/s 2024-04-27T03:16:19.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:19 smithi119 ceph-mon[32752]: pgmap v17227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 698 B/s rd, 232 B/s wr, 1 op/s 2024-04-27T03:16:19.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:19 smithi167 ceph-mon[32751]: pgmap v17227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 698 B/s rd, 232 B/s wr, 1 op/s 2024-04-27T03:16:19.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:19 smithi090 ceph-mon[29180]: pgmap v17227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 698 B/s rd, 232 B/s wr, 1 op/s 2024-04-27T03:16:20.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31262.468353 secs 2024-04-27T03:16:20.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31262.468353 secs 2024-04-27T03:16:20.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31262.468353 secs 2024-04-27T03:16:21.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:21 smithi119 ceph-mon[32752]: pgmap v17228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-27T03:16:21.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:21 smithi167 ceph-mon[32751]: pgmap v17228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-27T03:16:21.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:21 smithi090 ceph-mon[29180]: pgmap v17228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-27T03:16:22.786 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31263.321549 secs 2024-04-27T03:16:22.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31263.321549 secs 2024-04-27T03:16:22.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31263.321549 secs 2024-04-27T03:16:23.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:23 smithi119 ceph-mon[32752]: pgmap v17229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T03:16:23.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:23 smithi167 ceph-mon[32751]: pgmap v17229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T03:16:23.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:23 smithi090 ceph-mon[29180]: pgmap v17229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T03:16:25.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:25 smithi119 ceph-mon[32752]: pgmap v17230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T03:16:25.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31267.468435 secs 2024-04-27T03:16:25.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:25 smithi167 ceph-mon[32751]: pgmap v17230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T03:16:25.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31267.468435 secs 2024-04-27T03:16:25.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:25 smithi090 ceph-mon[29180]: pgmap v17230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T03:16:25.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31267.468435 secs 2024-04-27T03:16:27.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31268.321660 secs 2024-04-27T03:16:27.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:27 smithi119 ceph-mon[32752]: pgmap v17231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:27.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31268.321660 secs 2024-04-27T03:16:27.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:27 smithi167 ceph-mon[32751]: pgmap v17231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:27.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31268.321660 secs 2024-04-27T03:16:27.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:27 smithi090 ceph-mon[29180]: pgmap v17231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:29.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:29 smithi119 ceph-mon[32752]: pgmap v17232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:29.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:29 smithi167 ceph-mon[32751]: pgmap v17232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:29.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:29 smithi090 ceph-mon[29180]: pgmap v17232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:30.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31272.468522 secs 2024-04-27T03:16:30.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31272.468522 secs 2024-04-27T03:16:30.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31272.468522 secs 2024-04-27T03:16:31.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:31 smithi119 ceph-mon[32752]: pgmap v17233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:31.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:31 smithi167 ceph-mon[32751]: pgmap v17233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:31.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:31 smithi090 ceph-mon[29180]: pgmap v17233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:32.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31273.321776 secs 2024-04-27T03:16:32.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31273.321776 secs 2024-04-27T03:16:32.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31273.321776 secs 2024-04-27T03:16:33.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:33 smithi119 ceph-mon[32752]: pgmap v17234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:33.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:33 smithi167 ceph-mon[32751]: pgmap v17234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:33.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:33 smithi090 ceph-mon[29180]: pgmap v17234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:35.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:35 smithi119 ceph-mon[32752]: pgmap v17235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:35.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31277.468605 secs 2024-04-27T03:16:35.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:35 smithi167 ceph-mon[32751]: pgmap v17235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:35.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31277.468605 secs 2024-04-27T03:16:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:35 smithi090 ceph-mon[29180]: pgmap v17235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:35.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31277.468605 secs 2024-04-27T03:16:37.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31278.321854 secs 2024-04-27T03:16:37.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:37 smithi119 ceph-mon[32752]: pgmap v17236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:37.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31278.321854 secs 2024-04-27T03:16:37.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:37 smithi167 ceph-mon[32751]: pgmap v17236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:37.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31278.321854 secs 2024-04-27T03:16:37.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:37 smithi090 ceph-mon[29180]: pgmap v17236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:39.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:39 smithi119 ceph-mon[32752]: pgmap v17237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:39.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:39 smithi167 ceph-mon[32751]: pgmap v17237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:39.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:39 smithi090 ceph-mon[29180]: pgmap v17237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:40.812 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31282.468689 secs 2024-04-27T03:16:40.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31282.468689 secs 2024-04-27T03:16:40.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31282.468689 secs 2024-04-27T03:16:41.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:41 smithi167 ceph-mon[32751]: pgmap v17238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:41.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:41 smithi090 ceph-mon[29180]: pgmap v17238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:42.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:41 smithi119 ceph-mon[32752]: pgmap v17238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:42.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31283.321972 secs 2024-04-27T03:16:42.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31283.321972 secs 2024-04-27T03:16:43.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31283.321972 secs 2024-04-27T03:16:43.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:43 smithi167 ceph-mon[32751]: pgmap v17239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:43.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:43 smithi090 ceph-mon[29180]: pgmap v17239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:44.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:43 smithi119 ceph-mon[32752]: pgmap v17239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:45.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:45 smithi167 ceph-mon[32751]: pgmap v17240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:45.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31287.468769 secs 2024-04-27T03:16:45.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:45 smithi090 ceph-mon[29180]: pgmap v17240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:45.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31287.468769 secs 2024-04-27T03:16:46.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:45 smithi119 ceph-mon[32752]: pgmap v17240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:46.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31287.468769 secs 2024-04-27T03:16:47.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31288.322095 secs 2024-04-27T03:16:47.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:47 smithi090 ceph-mon[29180]: pgmap v17241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:48.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31288.322095 secs 2024-04-27T03:16:48.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:47 smithi119 ceph-mon[32752]: pgmap v17241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:48.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31288.322095 secs 2024-04-27T03:16:48.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:47 smithi167 ceph-mon[32751]: pgmap v17241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:49 smithi090 ceph-mon[29180]: pgmap v17242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:49 smithi119 ceph-mon[32752]: pgmap v17242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:49 smithi167 ceph-mon[32751]: pgmap v17242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:50.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31292.468823 secs 2024-04-27T03:16:51.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31292.468823 secs 2024-04-27T03:16:51.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31292.468823 secs 2024-04-27T03:16:51.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:51 smithi090 ceph-mon[29180]: pgmap v17243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:52.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:51 smithi119 ceph-mon[32752]: pgmap v17243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:52.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:51 smithi167 ceph-mon[32751]: pgmap v17243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:52.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31293.322211 secs 2024-04-27T03:16:53.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31293.322211 secs 2024-04-27T03:16:53.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31293.322211 secs 2024-04-27T03:16:53.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:53 smithi090 ceph-mon[29180]: pgmap v17244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:54.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:53 smithi119 ceph-mon[32752]: pgmap v17244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:54.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:53 smithi167 ceph-mon[32751]: pgmap v17244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:55.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:55 smithi090 ceph-mon[29180]: pgmap v17245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:55.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31297.468882 secs 2024-04-27T03:16:56.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:55 smithi119 ceph-mon[32752]: pgmap v17245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:56.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31297.468882 secs 2024-04-27T03:16:56.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:55 smithi167 ceph-mon[32751]: pgmap v17245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:16:56.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31297.468882 secs 2024-04-27T03:16:57.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31298.322339 secs 2024-04-27T03:16:57.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:57 smithi090 ceph-mon[29180]: pgmap v17246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:58.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31298.322339 secs 2024-04-27T03:16:58.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:57 smithi119 ceph-mon[32752]: pgmap v17246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:58.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31298.322339 secs 2024-04-27T03:16:58.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:57 smithi167 ceph-mon[32751]: pgmap v17246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:16:59.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:16:59 smithi090 ceph-mon[29180]: pgmap v17247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:00.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:16:59 smithi119 ceph-mon[32752]: pgmap v17247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:00.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:16:59 smithi167 ceph-mon[32751]: pgmap v17247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:00.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31302.468943 secs 2024-04-27T03:17:01.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31302.468943 secs 2024-04-27T03:17:01.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31302.468943 secs 2024-04-27T03:17:01.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:01 smithi090 ceph-mon[29180]: pgmap v17248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:02.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:01 smithi119 ceph-mon[32752]: pgmap v17248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:02.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:01 smithi167 ceph-mon[32751]: pgmap v17248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:02.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31303.322459 secs 2024-04-27T03:17:03.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31303.322459 secs 2024-04-27T03:17:03.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31303.322459 secs 2024-04-27T03:17:03.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:03 smithi090 ceph-mon[29180]: pgmap v17249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:04.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:03 smithi119 ceph-mon[32752]: pgmap v17249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:04.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:03 smithi167 ceph-mon[32751]: pgmap v17249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:05 smithi090 ceph-mon[29180]: pgmap v17250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:05.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31307.469000 secs 2024-04-27T03:17:06.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:05 smithi119 ceph-mon[32752]: pgmap v17250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:06.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31307.469000 secs 2024-04-27T03:17:06.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:05 smithi167 ceph-mon[32751]: pgmap v17250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:06.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31307.469000 secs 2024-04-27T03:17:07.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31308.322538 secs 2024-04-27T03:17:07.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:07 smithi090 ceph-mon[29180]: pgmap v17251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:08.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31308.322538 secs 2024-04-27T03:17:08.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:07 smithi119 ceph-mon[32752]: pgmap v17251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:08.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31308.322538 secs 2024-04-27T03:17:08.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:07 smithi167 ceph-mon[32751]: pgmap v17251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:09.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:09 smithi090 ceph-mon[29180]: pgmap v17252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:10.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:09 smithi119 ceph-mon[32752]: pgmap v17252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:10.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:09 smithi167 ceph-mon[32751]: pgmap v17252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:10.963 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:17:10.963 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31312.469070 secs 2024-04-27T03:17:11.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:17:11.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31312.469070 secs 2024-04-27T03:17:11.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:17:11.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31312.469070 secs 2024-04-27T03:17:12.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:11 smithi119 ceph-mon[32752]: pgmap v17253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:12.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:17:12.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:17:12.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:17:12.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:11 smithi167 ceph-mon[32751]: pgmap v17253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:12.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:17:12.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:17:12.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:17:12.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:11 smithi090 ceph-mon[29180]: pgmap v17253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:12.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:17:12.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:17:12.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:17:13.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:12 smithi119 ceph-mon[32752]: pgmap v17254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:17:13.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:12 smithi119 ceph-mon[32752]: pgmap v17255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:13.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31313.322647 secs 2024-04-27T03:17:13.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:12 smithi167 ceph-mon[32751]: pgmap v17254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:17:13.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:12 smithi167 ceph-mon[32751]: pgmap v17255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:13.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31313.322647 secs 2024-04-27T03:17:13.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:12 smithi090 ceph-mon[29180]: pgmap v17254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:17:13.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:12 smithi090 ceph-mon[29180]: pgmap v17255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:13.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31313.322647 secs 2024-04-27T03:17:15.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:14 smithi119 ceph-mon[32752]: pgmap v17256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:17:15.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:14 smithi167 ceph-mon[32751]: pgmap v17256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:17:15.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:14 smithi090 ceph-mon[29180]: pgmap v17256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:17:16.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31317.469150 secs 2024-04-27T03:17:16.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31317.469150 secs 2024-04-27T03:17:16.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31317.469150 secs 2024-04-27T03:17:17.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:16 smithi090 ceph-mon[29180]: pgmap v17257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:17.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:16 smithi119 ceph-mon[32752]: pgmap v17257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:17.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:16 smithi167 ceph-mon[32751]: pgmap v17257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:18.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31318.322765 secs 2024-04-27T03:17:18.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31318.322765 secs 2024-04-27T03:17:18.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31318.322765 secs 2024-04-27T03:17:19.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:18 smithi119 ceph-mon[32752]: pgmap v17258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:19.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:18 smithi167 ceph-mon[32751]: pgmap v17258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:19.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:18 smithi090 ceph-mon[29180]: pgmap v17258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:21.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:20 smithi119 ceph-mon[32752]: pgmap v17259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:21.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31322.469211 secs 2024-04-27T03:17:21.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:20 smithi167 ceph-mon[32751]: pgmap v17259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:21.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31322.469211 secs 2024-04-27T03:17:21.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:20 smithi090 ceph-mon[29180]: pgmap v17259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:17:21.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31322.469211 secs 2024-04-27T03:17:23.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:22 smithi119 ceph-mon[32752]: pgmap v17260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:17:23.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31323.322866 secs 2024-04-27T03:17:23.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:22 smithi167 ceph-mon[32751]: pgmap v17260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:17:23.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31323.322866 secs 2024-04-27T03:17:23.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:22 smithi090 ceph-mon[29180]: pgmap v17260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:17:23.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31323.322866 secs 2024-04-27T03:17:25.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:24 smithi119 ceph-mon[32752]: pgmap v17261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:25.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:24 smithi167 ceph-mon[32751]: pgmap v17261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:25.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:24 smithi090 ceph-mon[29180]: pgmap v17261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:25.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31327.469291 secs 2024-04-27T03:17:26.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31327.469291 secs 2024-04-27T03:17:26.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31327.469291 secs 2024-04-27T03:17:27.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:26 smithi119 ceph-mon[32752]: pgmap v17262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:27.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:26 smithi167 ceph-mon[32751]: pgmap v17262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:27.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:26 smithi090 ceph-mon[29180]: pgmap v17262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31328.322980 secs 2024-04-27T03:17:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31328.322980 secs 2024-04-27T03:17:28.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31328.322980 secs 2024-04-27T03:17:29.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:28 smithi119 ceph-mon[32752]: pgmap v17263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:29.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:28 smithi167 ceph-mon[32751]: pgmap v17263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:29.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:28 smithi090 ceph-mon[29180]: pgmap v17263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:31.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:30 smithi119 ceph-mon[32752]: pgmap v17264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:31.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31332.469376 secs 2024-04-27T03:17:31.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:30 smithi167 ceph-mon[32751]: pgmap v17264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:31.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31332.469376 secs 2024-04-27T03:17:31.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:30 smithi090 ceph-mon[29180]: pgmap v17264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:31.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31332.469376 secs 2024-04-27T03:17:33.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:32 smithi119 ceph-mon[32752]: pgmap v17265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:33.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31333.323094 secs 2024-04-27T03:17:33.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:32 smithi167 ceph-mon[32751]: pgmap v17265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:33.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31333.323094 secs 2024-04-27T03:17:33.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:32 smithi090 ceph-mon[29180]: pgmap v17265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:33.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31333.323094 secs 2024-04-27T03:17:35.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:34 smithi119 ceph-mon[32752]: pgmap v17266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:35.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:34 smithi167 ceph-mon[32751]: pgmap v17266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:35.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:34 smithi090 ceph-mon[29180]: pgmap v17266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:36.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31337.469458 secs 2024-04-27T03:17:36.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31337.469458 secs 2024-04-27T03:17:36.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31337.469458 secs 2024-04-27T03:17:37.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:36 smithi119 ceph-mon[32752]: pgmap v17267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:37.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:36 smithi167 ceph-mon[32751]: pgmap v17267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:37.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:36 smithi090 ceph-mon[29180]: pgmap v17267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:37.977 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31338.323203 secs 2024-04-27T03:17:38.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31338.323203 secs 2024-04-27T03:17:38.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31338.323203 secs 2024-04-27T03:17:39.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:38 smithi119 ceph-mon[32752]: pgmap v17268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:39.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:38 smithi167 ceph-mon[32751]: pgmap v17268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:39.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:38 smithi090 ceph-mon[29180]: pgmap v17268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:41.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:40 smithi119 ceph-mon[32752]: pgmap v17269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:41.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:40 smithi119 ceph-mon[32752]: 9 slow requests, 1 included below; oldest blocked for > 31342.469538 secs 2024-04-27T03:17:41.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:40 smithi119 ceph-mon[32752]: slow request 720.979061 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:17:41.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:40 smithi167 ceph-mon[32751]: pgmap v17269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:41.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:40 smithi167 ceph-mon[32751]: 9 slow requests, 1 included below; oldest blocked for > 31342.469538 secs 2024-04-27T03:17:41.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:40 smithi167 ceph-mon[32751]: slow request 720.979061 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:17:41.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:40 smithi090 ceph-mon[29180]: pgmap v17269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:41.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:40 smithi090 ceph-mon[29180]: 9 slow requests, 1 included below; oldest blocked for > 31342.469538 secs 2024-04-27T03:17:41.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:40 smithi090 ceph-mon[29180]: slow request 720.979061 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:17:43.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:42 smithi119 ceph-mon[32752]: pgmap v17270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:43.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:42 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 31343.323292 secs 2024-04-27T03:17:43.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:42 smithi119 ceph-mon[32752]: slow request 2882.414417 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:17:43.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:42 smithi167 ceph-mon[32751]: pgmap v17270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:43.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:42 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 31343.323292 secs 2024-04-27T03:17:43.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:42 smithi167 ceph-mon[32751]: slow request 2882.414417 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:17:43.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:42 smithi090 ceph-mon[29180]: pgmap v17270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:43.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:42 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 31343.323292 secs 2024-04-27T03:17:43.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:42 smithi090 ceph-mon[29180]: slow request 2882.414417 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:17:45.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:44 smithi119 ceph-mon[32752]: pgmap v17271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:45.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:44 smithi167 ceph-mon[32751]: pgmap v17271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:45.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:44 smithi090 ceph-mon[29180]: pgmap v17271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:46.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31347.469608 secs 2024-04-27T03:17:46.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31347.469608 secs 2024-04-27T03:17:46.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31347.469608 secs 2024-04-27T03:17:47.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:46 smithi090 ceph-mon[29180]: pgmap v17272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:47.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:46 smithi119 ceph-mon[32752]: pgmap v17272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:47.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:46 smithi167 ceph-mon[32751]: pgmap v17272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:48.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31348.323416 secs 2024-04-27T03:17:48.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31348.323416 secs 2024-04-27T03:17:48.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31348.323416 secs 2024-04-27T03:17:49.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:48 smithi119 ceph-mon[32752]: pgmap v17273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:49.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:48 smithi167 ceph-mon[32751]: pgmap v17273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:49.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:48 smithi090 ceph-mon[29180]: pgmap v17273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:49 smithi119 ceph-mon[32752]: pgmap v17274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:49 smithi167 ceph-mon[32751]: pgmap v17274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:50.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:49 smithi090 ceph-mon[29180]: pgmap v17274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:51.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31352.469676 secs 2024-04-27T03:17:51.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31352.469676 secs 2024-04-27T03:17:51.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31352.469676 secs 2024-04-27T03:17:52.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:51 smithi119 ceph-mon[32752]: pgmap v17275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:52.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:51 smithi167 ceph-mon[32751]: pgmap v17275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:52.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:51 smithi090 ceph-mon[29180]: pgmap v17275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:53.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31353.323530 secs 2024-04-27T03:17:53.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31353.323530 secs 2024-04-27T03:17:53.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31353.323530 secs 2024-04-27T03:17:54.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:53 smithi119 ceph-mon[32752]: pgmap v17276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:54.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:53 smithi167 ceph-mon[32751]: pgmap v17276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:54.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:53 smithi090 ceph-mon[29180]: pgmap v17276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:56.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:55 smithi090 ceph-mon[29180]: pgmap v17277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:56.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31357.469760 secs 2024-04-27T03:17:56.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:55 smithi119 ceph-mon[32752]: pgmap v17277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:56.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31357.469760 secs 2024-04-27T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:55 smithi167 ceph-mon[32751]: pgmap v17277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:17:56.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31357.469760 secs 2024-04-27T03:17:58.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31358.323713 secs 2024-04-27T03:17:58.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:57 smithi119 ceph-mon[32752]: pgmap v17278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:58.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31358.323713 secs 2024-04-27T03:17:58.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:57 smithi167 ceph-mon[32751]: pgmap v17278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:17:58.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31358.323713 secs 2024-04-27T03:17:58.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:57 smithi090 ceph-mon[29180]: pgmap v17278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:00.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:17:59 smithi119 ceph-mon[32752]: pgmap v17279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:00.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:17:59 smithi167 ceph-mon[32751]: pgmap v17279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:00.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:17:59 smithi090 ceph-mon[29180]: pgmap v17279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:01.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31362.469824 secs 2024-04-27T03:18:01.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31362.469824 secs 2024-04-27T03:18:01.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31362.469824 secs 2024-04-27T03:18:02.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:01 smithi119 ceph-mon[32752]: pgmap v17280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:02.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:01 smithi167 ceph-mon[32751]: pgmap v17280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:02.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:01 smithi090 ceph-mon[29180]: pgmap v17280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31363.323838 secs 2024-04-27T03:18:03.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31363.323838 secs 2024-04-27T03:18:03.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31363.323838 secs 2024-04-27T03:18:04.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:03 smithi119 ceph-mon[32752]: pgmap v17281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:04.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:03 smithi167 ceph-mon[32751]: pgmap v17281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:04.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:03 smithi090 ceph-mon[29180]: pgmap v17281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:06.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:05 smithi119 ceph-mon[32752]: pgmap v17282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:06.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31367.469885 secs 2024-04-27T03:18:06.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:05 smithi167 ceph-mon[32751]: pgmap v17282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:06.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31367.469885 secs 2024-04-27T03:18:06.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:05 smithi090 ceph-mon[29180]: pgmap v17282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:06.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31367.469885 secs 2024-04-27T03:18:08.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31368.323911 secs 2024-04-27T03:18:08.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:07 smithi119 ceph-mon[32752]: pgmap v17283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:08.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31368.323911 secs 2024-04-27T03:18:08.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:07 smithi167 ceph-mon[32751]: pgmap v17283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:08.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31368.323911 secs 2024-04-27T03:18:08.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:07 smithi090 ceph-mon[29180]: pgmap v17283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:10.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:09 smithi119 ceph-mon[32752]: pgmap v17284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:10.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:09 smithi167 ceph-mon[32751]: pgmap v17284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:10.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:09 smithi090 ceph-mon[29180]: pgmap v17284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:11.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:10 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31372.469947 secs 2024-04-27T03:18:11.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:10 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31372.469947 secs 2024-04-27T03:18:11.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:10 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31372.469947 secs 2024-04-27T03:18:12.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:11 smithi119 ceph-mon[32752]: pgmap v17285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:12.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:18:12.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:18:12.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:18:12.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:11 smithi119 ceph-mon[32752]: pgmap v17286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:18:12.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:11 smithi119 ceph-mon[32752]: pgmap v17287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:12.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:18:12.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:11 smithi167 ceph-mon[32751]: pgmap v17285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:12.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:18:12.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:18:12.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:18:12.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:11 smithi167 ceph-mon[32751]: pgmap v17286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:18:12.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:11 smithi167 ceph-mon[32751]: pgmap v17287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:12.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:18:12.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:11 smithi090 ceph-mon[29180]: pgmap v17285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:12.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:18:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:18:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:18:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:11 smithi090 ceph-mon[29180]: pgmap v17286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:18:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:11 smithi090 ceph-mon[29180]: pgmap v17287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:12.386 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:18:13.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31373.323999 secs 2024-04-27T03:18:13.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31373.323999 secs 2024-04-27T03:18:13.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31373.323999 secs 2024-04-27T03:18:14.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:13 smithi119 ceph-mon[32752]: pgmap v17288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:14.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:13 smithi167 ceph-mon[32751]: pgmap v17288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:14.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:13 smithi090 ceph-mon[29180]: pgmap v17288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:15.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:15 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31377.470007 secs 2024-04-27T03:18:15.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:15 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31377.470007 secs 2024-04-27T03:18:15.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:15 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31377.470007 secs 2024-04-27T03:18:16.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:16 smithi119 ceph-mon[32752]: pgmap v17289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:16.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:16 smithi167 ceph-mon[32751]: pgmap v17289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:16.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:16 smithi090 ceph-mon[29180]: pgmap v17289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:17.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31378.324108 secs 2024-04-27T03:18:17.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31378.324108 secs 2024-04-27T03:18:17.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31378.324108 secs 2024-04-27T03:18:18.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:18 smithi119 ceph-mon[32752]: pgmap v17290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:18:18.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:18 smithi167 ceph-mon[32751]: pgmap v17290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:18:18.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:18 smithi090 ceph-mon[29180]: pgmap v17290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:18:21.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:20 smithi119 ceph-mon[32752]: pgmap v17291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:21.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:20 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31382.470071 secs 2024-04-27T03:18:21.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:20 smithi167 ceph-mon[32751]: pgmap v17291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:21.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:20 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31382.470071 secs 2024-04-27T03:18:21.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:20 smithi090 ceph-mon[29180]: pgmap v17291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:18:21.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:20 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31382.470071 secs 2024-04-27T03:18:23.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31383.324225 secs 2024-04-27T03:18:23.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:22 smithi119 ceph-mon[32752]: pgmap v17292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:18:23.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31383.324225 secs 2024-04-27T03:18:23.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:22 smithi167 ceph-mon[32751]: pgmap v17292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:18:23.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31383.324225 secs 2024-04-27T03:18:23.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:22 smithi090 ceph-mon[29180]: pgmap v17292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:18:25.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:24 smithi119 ceph-mon[32752]: pgmap v17293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:25.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:24 smithi167 ceph-mon[32751]: pgmap v17293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:25.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:24 smithi090 ceph-mon[29180]: pgmap v17293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:26.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:25 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31387.470153 secs 2024-04-27T03:18:26.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:25 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31387.470153 secs 2024-04-27T03:18:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:25 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31387.470153 secs 2024-04-27T03:18:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:26 smithi119 ceph-mon[32752]: pgmap v17294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:26 smithi167 ceph-mon[32751]: pgmap v17294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:26 smithi090 ceph-mon[29180]: pgmap v17294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:27.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31388.324342 secs 2024-04-27T03:18:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31388.324342 secs 2024-04-27T03:18:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31388.324342 secs 2024-04-27T03:18:29.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:28 smithi119 ceph-mon[32752]: pgmap v17295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:29.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:28 smithi167 ceph-mon[32751]: pgmap v17295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:29.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:28 smithi090 ceph-mon[29180]: pgmap v17295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:31.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:30 smithi119 ceph-mon[32752]: pgmap v17296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:31.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:30 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31392.470233 secs 2024-04-27T03:18:31.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:30 smithi167 ceph-mon[32751]: pgmap v17296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:31.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:30 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31392.470233 secs 2024-04-27T03:18:31.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:30 smithi090 ceph-mon[29180]: pgmap v17296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:31.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:30 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31392.470233 secs 2024-04-27T03:18:32.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:31 smithi119 ceph-mon[32752]: pgmap v17297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:32.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:31 smithi167 ceph-mon[32751]: pgmap v17297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:32.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:31 smithi090 ceph-mon[29180]: pgmap v17297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:33.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31393.324466 secs 2024-04-27T03:18:33.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31393.324466 secs 2024-04-27T03:18:33.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31393.324466 secs 2024-04-27T03:18:34.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:33 smithi119 ceph-mon[32752]: pgmap v17298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:34.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:33 smithi167 ceph-mon[32751]: pgmap v17298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:34.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:33 smithi090 ceph-mon[29180]: pgmap v17298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:35.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:35 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31397.470312 secs 2024-04-27T03:18:35.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:35 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31397.470312 secs 2024-04-27T03:18:35.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:35 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31397.470312 secs 2024-04-27T03:18:36.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:36 smithi119 ceph-mon[32752]: pgmap v17299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:36.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:36 smithi167 ceph-mon[32751]: pgmap v17299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:36.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:36 smithi090 ceph-mon[29180]: pgmap v17299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:37.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31398.324531 secs 2024-04-27T03:18:37.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31398.324531 secs 2024-04-27T03:18:37.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31398.324531 secs 2024-04-27T03:18:38.727 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:38 smithi167 ceph-mon[32751]: pgmap v17300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:38.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:38 smithi119 ceph-mon[32752]: pgmap v17300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:38.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:38 smithi090 ceph-mon[29180]: pgmap v17300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:41.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:40 smithi119 ceph-mon[32752]: pgmap v17301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:41.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:40 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31402.470385 secs 2024-04-27T03:18:41.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:40 smithi167 ceph-mon[32751]: pgmap v17301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:41.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:40 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31402.470385 secs 2024-04-27T03:18:41.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:40 smithi090 ceph-mon[29180]: pgmap v17301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:41.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:40 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31402.470385 secs 2024-04-27T03:18:43.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31403.324601 secs 2024-04-27T03:18:43.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:42 smithi119 ceph-mon[32752]: pgmap v17302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:43.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31403.324601 secs 2024-04-27T03:18:43.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:42 smithi167 ceph-mon[32751]: pgmap v17302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:43.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31403.324601 secs 2024-04-27T03:18:43.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:42 smithi090 ceph-mon[29180]: pgmap v17302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:45.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:44 smithi119 ceph-mon[32752]: pgmap v17303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:45.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:44 smithi167 ceph-mon[32751]: pgmap v17303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:45.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:44 smithi090 ceph-mon[29180]: pgmap v17303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:46.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:45 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31407.470457 secs 2024-04-27T03:18:46.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:45 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31407.470457 secs 2024-04-27T03:18:46.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:45 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31407.470457 secs 2024-04-27T03:18:47.035 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:46 smithi090 ceph-mon[29180]: pgmap v17304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:47.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:46 smithi119 ceph-mon[32752]: pgmap v17304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:47.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:46 smithi167 ceph-mon[32751]: pgmap v17304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:48.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31408.324714 secs 2024-04-27T03:18:48.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31408.324714 secs 2024-04-27T03:18:48.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31408.324714 secs 2024-04-27T03:18:49.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:48 smithi119 ceph-mon[32752]: pgmap v17305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:49.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:48 smithi167 ceph-mon[32751]: pgmap v17305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:49.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:48 smithi090 ceph-mon[29180]: pgmap v17305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:49 smithi119 ceph-mon[32752]: pgmap v17306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:49 smithi167 ceph-mon[32751]: pgmap v17306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:50.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:49 smithi090 ceph-mon[29180]: pgmap v17306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:51.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:50 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31412.470540 secs 2024-04-27T03:18:51.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:50 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31412.470540 secs 2024-04-27T03:18:51.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:50 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31412.470540 secs 2024-04-27T03:18:52.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:51 smithi119 ceph-mon[32752]: pgmap v17307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:52.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:51 smithi167 ceph-mon[32751]: pgmap v17307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:52.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:51 smithi090 ceph-mon[29180]: pgmap v17307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:53.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31413.324858 secs 2024-04-27T03:18:53.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31413.324858 secs 2024-04-27T03:18:53.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31413.324858 secs 2024-04-27T03:18:54.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:53 smithi119 ceph-mon[32752]: pgmap v17308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:54.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:53 smithi167 ceph-mon[32751]: pgmap v17308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:54.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:53 smithi090 ceph-mon[29180]: pgmap v17308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:18:55.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:55 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31417.470624 secs 2024-04-27T03:18:55.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:55 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31417.470624 secs 2024-04-27T03:18:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:55 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31417.470624 secs 2024-04-27T03:18:56.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:56 smithi119 ceph-mon[32752]: pgmap v17309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:56.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:56 smithi167 ceph-mon[32751]: pgmap v17309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:56.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:56 smithi090 ceph-mon[29180]: pgmap v17309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:57.773 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31418.324996 secs 2024-04-27T03:18:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31418.324996 secs 2024-04-27T03:18:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31418.324996 secs 2024-04-27T03:18:58.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:18:58 smithi119 ceph-mon[32752]: pgmap v17310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:58.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:18:58 smithi167 ceph-mon[32751]: pgmap v17310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:18:58.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:18:58 smithi090 ceph-mon[29180]: pgmap v17310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:01.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:00 smithi119 ceph-mon[32752]: pgmap v17311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:01.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:00 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31422.470705 secs 2024-04-27T03:19:01.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:00 smithi167 ceph-mon[32751]: pgmap v17311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:01.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:00 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31422.470705 secs 2024-04-27T03:19:01.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:00 smithi090 ceph-mon[29180]: pgmap v17311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:01.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:00 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31422.470705 secs 2024-04-27T03:19:03.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31423.325116 secs 2024-04-27T03:19:03.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:02 smithi119 ceph-mon[32752]: pgmap v17312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:03.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31423.325116 secs 2024-04-27T03:19:03.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:02 smithi167 ceph-mon[32751]: pgmap v17312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:03.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31423.325116 secs 2024-04-27T03:19:03.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:02 smithi090 ceph-mon[29180]: pgmap v17312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:05.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:04 smithi119 ceph-mon[32752]: pgmap v17313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:05.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:04 smithi167 ceph-mon[32751]: pgmap v17313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:05.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:04 smithi090 ceph-mon[29180]: pgmap v17313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:06.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:05 smithi119 ceph-mon[32752]: 9 slow requests, 0 included below; oldest blocked for > 31427.470811 secs 2024-04-27T03:19:06.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:05 smithi167 ceph-mon[32751]: 9 slow requests, 0 included below; oldest blocked for > 31427.470811 secs 2024-04-27T03:19:06.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:05 smithi090 ceph-mon[29180]: 9 slow requests, 0 included below; oldest blocked for > 31427.470811 secs 2024-04-27T03:19:07.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:06 smithi119 ceph-mon[32752]: pgmap v17314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:07.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:06 smithi167 ceph-mon[32751]: pgmap v17314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:07.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:06 smithi090 ceph-mon[29180]: pgmap v17314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:08.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31428.325268 secs 2024-04-27T03:19:08.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31428.325268 secs 2024-04-27T03:19:08.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31428.325268 secs 2024-04-27T03:19:09.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:08 smithi119 ceph-mon[32752]: pgmap v17315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:09.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:08 smithi167 ceph-mon[32751]: pgmap v17315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:09.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:08 smithi090 ceph-mon[29180]: pgmap v17315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:10.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:09 smithi119 ceph-mon[32752]: pgmap v17316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:10.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:09 smithi167 ceph-mon[32751]: pgmap v17316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:10.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:09 smithi090 ceph-mon[29180]: pgmap v17316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:11.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:10 smithi119 ceph-mon[32752]: 42 slow requests, 3 included below; oldest blocked for > 31432.470895 secs 2024-04-27T03:19:11.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:10 smithi119 ceph-mon[32752]: slow request 1441.162498 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:19:11.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:10 smithi119 ceph-mon[32752]: slow request 180.803304 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:19:11.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:10 smithi119 ceph-mon[32752]: slow request 23047.232838 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:19:11.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:10 smithi167 ceph-mon[32751]: 42 slow requests, 3 included below; oldest blocked for > 31432.470895 secs 2024-04-27T03:19:11.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:10 smithi167 ceph-mon[32751]: slow request 1441.162498 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:19:11.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:10 smithi167 ceph-mon[32751]: slow request 180.803304 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:19:11.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:10 smithi167 ceph-mon[32751]: slow request 23047.232838 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:19:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:10 smithi090 ceph-mon[29180]: 42 slow requests, 3 included below; oldest blocked for > 31432.470895 secs 2024-04-27T03:19:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:10 smithi090 ceph-mon[29180]: slow request 1441.162498 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:19:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:10 smithi090 ceph-mon[29180]: slow request 180.803304 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:19:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:10 smithi090 ceph-mon[29180]: slow request 23047.232838 seconds old, received at 2024-04-26T20:55:02.991460+0000: client_request(client.26516:1 getattr pAsLsXsFs #0x1 2024-04-26T20:55:02.983926+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:19:12.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:11 smithi119 ceph-mon[32752]: pgmap v17317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:12.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:11 smithi167 ceph-mon[32751]: pgmap v17317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:12.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:11 smithi090 ceph-mon[29180]: pgmap v17317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:13.036 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31433.325404 secs 2024-04-27T03:19:13.037 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:19:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31433.325404 secs 2024-04-27T03:19:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:19:13.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31433.325404 secs 2024-04-27T03:19:13.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:19:14.040 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:13 smithi090 ceph-mon[29180]: pgmap v17318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:14.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:13 smithi119 ceph-mon[32752]: pgmap v17318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:14.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:13 smithi167 ceph-mon[32751]: pgmap v17318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:15.967 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:19:15.967 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:19:15.968 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31437.470960 secs 2024-04-27T03:19:15.968 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:15 smithi119 ceph-mon[32752]: pgmap v17319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:16.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:19:16.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:19:16.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31437.470960 secs 2024-04-27T03:19:16.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:15 smithi167 ceph-mon[32751]: pgmap v17319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:16.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:19:16.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:19:16.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31437.470960 secs 2024-04-27T03:19:16.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:15 smithi090 ceph-mon[29180]: pgmap v17319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:17.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:19:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:19:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: pgmap v17320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-27T03:19:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: pgmap v17321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:17.312 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:19:17.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:19:17.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: pgmap v17320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-27T03:19:17.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: pgmap v17321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:17.329 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:17.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:19:17.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:19:17.384 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: pgmap v17320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 961 B/s rd, 1 op/s 2024-04-27T03:19:17.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: pgmap v17321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:17.385 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:19:18.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:18 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31438.325481 secs 2024-04-27T03:19:18.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:18 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31438.325481 secs 2024-04-27T03:19:18.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:18 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31438.325481 secs 2024-04-27T03:19:19.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:19 smithi119 ceph-mon[32752]: pgmap v17322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:19.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:19 smithi167 ceph-mon[32751]: pgmap v17322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:19.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:19 smithi090 ceph-mon[29180]: pgmap v17322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:20.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:20 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:19:20.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:20 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:19:20.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:20 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:19:20.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:20 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:19:20.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:20 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:19:20.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:20 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:19:21.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:21 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31442.471020 secs 2024-04-27T03:19:21.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:21 smithi119 ceph-mon[32752]: pgmap v17323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:21.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:21 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31442.471020 secs 2024-04-27T03:19:21.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:21 smithi167 ceph-mon[32751]: pgmap v17323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:21.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:21 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31442.471020 secs 2024-04-27T03:19:21.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:21 smithi090 ceph-mon[29180]: pgmap v17323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:22.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31443.325599 secs 2024-04-27T03:19:22.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31443.325599 secs 2024-04-27T03:19:22.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31443.325599 secs 2024-04-27T03:19:23.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:23 smithi090 ceph-mon[29180]: pgmap v17324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 B/s rd, 1 op/s 2024-04-27T03:19:23.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:23 smithi119 ceph-mon[32752]: pgmap v17324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 B/s rd, 1 op/s 2024-04-27T03:19:23.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:23 smithi167 ceph-mon[32751]: pgmap v17324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 591 B/s rd, 1 op/s 2024-04-27T03:19:25.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:25 smithi090 ceph-mon[29180]: pgmap v17325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:25.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31447.471079 secs 2024-04-27T03:19:25.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:25 smithi119 ceph-mon[32752]: pgmap v17325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:25.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31447.471079 secs 2024-04-27T03:19:25.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:25 smithi167 ceph-mon[32751]: pgmap v17325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:25.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31447.471079 secs 2024-04-27T03:19:27.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31448.325718 secs 2024-04-27T03:19:27.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:27 smithi090 ceph-mon[29180]: pgmap v17326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:19:27.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31448.325718 secs 2024-04-27T03:19:27.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:27 smithi119 ceph-mon[32752]: pgmap v17326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:19:27.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31448.325718 secs 2024-04-27T03:19:27.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:27 smithi167 ceph-mon[32751]: pgmap v17326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:19:29.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:29 smithi119 ceph-mon[32752]: pgmap v17327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:29.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:29 smithi167 ceph-mon[32751]: pgmap v17327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:29.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:29 smithi090 ceph-mon[29180]: pgmap v17327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:30.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31452.471141 secs 2024-04-27T03:19:30.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31452.471141 secs 2024-04-27T03:19:30.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31452.471141 secs 2024-04-27T03:19:31.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:31 smithi119 ceph-mon[32752]: pgmap v17328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:31.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:31 smithi167 ceph-mon[32751]: pgmap v17328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:31.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:31 smithi090 ceph-mon[29180]: pgmap v17328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:32.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31453.325784 secs 2024-04-27T03:19:32.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31453.325784 secs 2024-04-27T03:19:32.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31453.325784 secs 2024-04-27T03:19:33.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:33 smithi119 ceph-mon[32752]: pgmap v17329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:33.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:33 smithi167 ceph-mon[32751]: pgmap v17329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:33.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:33 smithi090 ceph-mon[29180]: pgmap v17329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:35.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:35 smithi119 ceph-mon[32752]: pgmap v17330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:35.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31457.471197 secs 2024-04-27T03:19:35.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:35 smithi167 ceph-mon[32751]: pgmap v17330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:35.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31457.471197 secs 2024-04-27T03:19:35.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:35 smithi090 ceph-mon[29180]: pgmap v17330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:35.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31457.471197 secs 2024-04-27T03:19:37.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31458.325909 secs 2024-04-27T03:19:37.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:37 smithi119 ceph-mon[32752]: pgmap v17331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:37.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31458.325909 secs 2024-04-27T03:19:37.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:37 smithi167 ceph-mon[32751]: pgmap v17331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:37.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31458.325909 secs 2024-04-27T03:19:37.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:37 smithi090 ceph-mon[29180]: pgmap v17331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:39.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:39 smithi119 ceph-mon[32752]: pgmap v17332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:39.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:39 smithi167 ceph-mon[32751]: pgmap v17332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:39.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:39 smithi090 ceph-mon[29180]: pgmap v17332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:40.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31462.471250 secs 2024-04-27T03:19:40.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31462.471250 secs 2024-04-27T03:19:40.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31462.471250 secs 2024-04-27T03:19:41.712 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:41 smithi119 ceph-mon[32752]: pgmap v17333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:41.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:41 smithi167 ceph-mon[32751]: pgmap v17333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:41.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:41 smithi090 ceph-mon[29180]: pgmap v17333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:42.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31463.325981 secs 2024-04-27T03:19:42.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31463.325981 secs 2024-04-27T03:19:42.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31463.325981 secs 2024-04-27T03:19:43.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:43 smithi119 ceph-mon[32752]: pgmap v17334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:43.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:43 smithi167 ceph-mon[32751]: pgmap v17334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:43.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:43 smithi090 ceph-mon[29180]: pgmap v17334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:45.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:45 smithi119 ceph-mon[32752]: pgmap v17335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:45.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31467.471307 secs 2024-04-27T03:19:45.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:45 smithi167 ceph-mon[32751]: pgmap v17335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:45.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31467.471307 secs 2024-04-27T03:19:45.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:45 smithi090 ceph-mon[29180]: pgmap v17335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:45.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31467.471307 secs 2024-04-27T03:19:47.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31468.326100 secs 2024-04-27T03:19:47.686 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:47 smithi090 ceph-mon[29180]: pgmap v17336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:47.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31468.326100 secs 2024-04-27T03:19:47.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:47 smithi119 ceph-mon[32752]: pgmap v17336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:47.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31468.326100 secs 2024-04-27T03:19:47.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:47 smithi167 ceph-mon[32751]: pgmap v17336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:49.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:49 smithi119 ceph-mon[32752]: pgmap v17337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:49.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:49 smithi167 ceph-mon[32751]: pgmap v17337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:49 smithi090 ceph-mon[29180]: pgmap v17337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:50.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31472.471357 secs 2024-04-27T03:19:50.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31472.471357 secs 2024-04-27T03:19:50.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31472.471357 secs 2024-04-27T03:19:51.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:51 smithi119 ceph-mon[32752]: pgmap v17338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:51.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:51 smithi167 ceph-mon[32751]: pgmap v17338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:51.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:51 smithi090 ceph-mon[29180]: pgmap v17338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:52.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31473.326196 secs 2024-04-27T03:19:52.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31473.326196 secs 2024-04-27T03:19:52.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31473.326196 secs 2024-04-27T03:19:53.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:53 smithi119 ceph-mon[32752]: pgmap v17339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:53.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:53 smithi167 ceph-mon[32751]: pgmap v17339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:53.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:53 smithi090 ceph-mon[29180]: pgmap v17339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:55.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:55 smithi119 ceph-mon[32752]: pgmap v17340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:55.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31477.471414 secs 2024-04-27T03:19:55.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:55 smithi167 ceph-mon[32751]: pgmap v17340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:55.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31477.471414 secs 2024-04-27T03:19:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:55 smithi090 ceph-mon[29180]: pgmap v17340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31477.471414 secs 2024-04-27T03:19:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31478.326336 secs 2024-04-27T03:19:57.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:57 smithi119 ceph-mon[32752]: pgmap v17341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31478.326336 secs 2024-04-27T03:19:57.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:57 smithi167 ceph-mon[32751]: pgmap v17341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31478.326336 secs 2024-04-27T03:19:57.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:57 smithi090 ceph-mon[29180]: pgmap v17341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:19:59.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:19:59 smithi119 ceph-mon[32752]: pgmap v17342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:59.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:19:59 smithi167 ceph-mon[32751]: pgmap v17342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:19:59.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:19:59 smithi090 ceph-mon[29180]: pgmap v17342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:00.748 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:20:00.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31482.471476 secs 2024-04-27T03:20:00.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:20:00.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31482.471476 secs 2024-04-27T03:20:00.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:20:00.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31482.471476 secs 2024-04-27T03:20:01.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:01 smithi119 ceph-mon[32752]: pgmap v17343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:01.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:01 smithi167 ceph-mon[32751]: pgmap v17343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:01.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:01 smithi090 ceph-mon[29180]: pgmap v17343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:02.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31483.326457 secs 2024-04-27T03:20:02.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31483.326457 secs 2024-04-27T03:20:02.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31483.326457 secs 2024-04-27T03:20:03.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:03 smithi119 ceph-mon[32752]: pgmap v17344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:03.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:03 smithi167 ceph-mon[32751]: pgmap v17344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:03.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:03 smithi090 ceph-mon[29180]: pgmap v17344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:05.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:05 smithi119 ceph-mon[32752]: pgmap v17345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:05.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31487.471535 secs 2024-04-27T03:20:05.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:05 smithi167 ceph-mon[32751]: pgmap v17345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:05.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31487.471535 secs 2024-04-27T03:20:05.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:05 smithi090 ceph-mon[29180]: pgmap v17345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:05.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31487.471535 secs 2024-04-27T03:20:07.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31488.326582 secs 2024-04-27T03:20:07.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:07 smithi119 ceph-mon[32752]: pgmap v17346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:07.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31488.326582 secs 2024-04-27T03:20:07.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:07 smithi167 ceph-mon[32751]: pgmap v17346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:07.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31488.326582 secs 2024-04-27T03:20:07.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:07 smithi090 ceph-mon[29180]: pgmap v17346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:09.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:09 smithi119 ceph-mon[32752]: pgmap v17347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:09.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:09 smithi167 ceph-mon[32751]: pgmap v17347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:09.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:09 smithi090 ceph-mon[29180]: pgmap v17347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:10.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31492.471597 secs 2024-04-27T03:20:10.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31492.471597 secs 2024-04-27T03:20:10.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31492.471597 secs 2024-04-27T03:20:11.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:11 smithi119 ceph-mon[32752]: pgmap v17348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:11.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:11 smithi167 ceph-mon[32751]: pgmap v17348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:11.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:11 smithi090 ceph-mon[29180]: pgmap v17348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:12.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31493.326710 secs 2024-04-27T03:20:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31493.326710 secs 2024-04-27T03:20:12.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31493.326710 secs 2024-04-27T03:20:13.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:13 smithi119 ceph-mon[32752]: pgmap v17349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:13.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:13 smithi167 ceph-mon[32751]: pgmap v17349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:13.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:13 smithi090 ceph-mon[29180]: pgmap v17349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:15 smithi119 ceph-mon[32752]: pgmap v17350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:15.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31497.471652 secs 2024-04-27T03:20:15.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:15 smithi167 ceph-mon[32751]: pgmap v17350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:15.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31497.471652 secs 2024-04-27T03:20:15.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:15 smithi090 ceph-mon[29180]: pgmap v17350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:15.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31497.471652 secs 2024-04-27T03:20:16.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:20:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:20:16.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:20:17.768 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31498.326827 secs 2024-04-27T03:20:17.769 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:17 smithi167 ceph-mon[32751]: pgmap v17351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:17.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31498.326827 secs 2024-04-27T03:20:17.784 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:17 smithi090 ceph-mon[29180]: pgmap v17351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:17.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31498.326827 secs 2024-04-27T03:20:17.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:17 smithi119 ceph-mon[32752]: pgmap v17351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:19.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:19 smithi119 ceph-mon[32752]: pgmap v17352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:19.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:19 smithi167 ceph-mon[32751]: pgmap v17352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:19.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:19 smithi090 ceph-mon[29180]: pgmap v17352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:20.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31502.471713 secs 2024-04-27T03:20:20.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31502.471713 secs 2024-04-27T03:20:20.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31502.471713 secs 2024-04-27T03:20:22.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: pgmap v17353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:22.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:20:22.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:20:22.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: pgmap v17354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 987 B/s rd, 1 op/s 2024-04-27T03:20:22.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: pgmap v17355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:22.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: pgmap v17353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:22.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:20:22.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:20:22.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: pgmap v17354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 987 B/s rd, 1 op/s 2024-04-27T03:20:22.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: pgmap v17355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:22.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: pgmap v17353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:22.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:20:22.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:20:22.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: pgmap v17354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 987 B/s rd, 1 op/s 2024-04-27T03:20:22.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: pgmap v17355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:22.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:20:22.992 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31503.326942 secs 2024-04-27T03:20:23.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31503.326942 secs 2024-04-27T03:20:23.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31503.326942 secs 2024-04-27T03:20:24.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:23 smithi119 ceph-mon[32752]: pgmap v17356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-27T03:20:24.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:23 smithi167 ceph-mon[32751]: pgmap v17356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-27T03:20:24.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:23 smithi090 ceph-mon[29180]: pgmap v17356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-27T03:20:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:25 smithi119 ceph-mon[32752]: pgmap v17357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:26.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31507.471770 secs 2024-04-27T03:20:26.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:25 smithi167 ceph-mon[32751]: pgmap v17357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:26.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31507.471770 secs 2024-04-27T03:20:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:25 smithi090 ceph-mon[29180]: pgmap v17357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:26.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31507.471770 secs 2024-04-27T03:20:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31508.327014 secs 2024-04-27T03:20:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:27 smithi119 ceph-mon[32752]: pgmap v17358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-27T03:20:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31508.327014 secs 2024-04-27T03:20:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:27 smithi167 ceph-mon[32751]: pgmap v17358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-27T03:20:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31508.327014 secs 2024-04-27T03:20:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:27 smithi090 ceph-mon[29180]: pgmap v17358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 612 B/s rd, 1 op/s 2024-04-27T03:20:29.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:29 smithi090 ceph-mon[29180]: pgmap v17359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:30.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:29 smithi119 ceph-mon[32752]: pgmap v17359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:30.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:29 smithi167 ceph-mon[32751]: pgmap v17359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31512.471799 secs 2024-04-27T03:20:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31512.471799 secs 2024-04-27T03:20:31.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31512.471799 secs 2024-04-27T03:20:32.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:31 smithi119 ceph-mon[32752]: pgmap v17360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:20:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:31 smithi167 ceph-mon[32751]: pgmap v17360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:20:32.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:31 smithi090 ceph-mon[29180]: pgmap v17360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:20:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31513.327146 secs 2024-04-27T03:20:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31513.327146 secs 2024-04-27T03:20:33.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31513.327146 secs 2024-04-27T03:20:34.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:33 smithi119 ceph-mon[32752]: pgmap v17361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:34.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:33 smithi167 ceph-mon[32751]: pgmap v17361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:34.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:33 smithi090 ceph-mon[29180]: pgmap v17361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:35 smithi119 ceph-mon[32752]: pgmap v17362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:36.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31517.471854 secs 2024-04-27T03:20:36.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:35 smithi167 ceph-mon[32751]: pgmap v17362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:36.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31517.471854 secs 2024-04-27T03:20:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:35 smithi090 ceph-mon[29180]: pgmap v17362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31517.471854 secs 2024-04-27T03:20:38.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31518.327266 secs 2024-04-27T03:20:38.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:37 smithi119 ceph-mon[32752]: pgmap v17363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31518.327266 secs 2024-04-27T03:20:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:37 smithi167 ceph-mon[32751]: pgmap v17363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31518.327266 secs 2024-04-27T03:20:38.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:37 smithi090 ceph-mon[29180]: pgmap v17363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:40.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:39 smithi119 ceph-mon[32752]: pgmap v17364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:40.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:39 smithi167 ceph-mon[32751]: pgmap v17364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:40.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:39 smithi090 ceph-mon[29180]: pgmap v17364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31522.471918 secs 2024-04-27T03:20:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31522.471918 secs 2024-04-27T03:20:41.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31522.471918 secs 2024-04-27T03:20:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:41 smithi119 ceph-mon[32752]: pgmap v17365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:41 smithi167 ceph-mon[32751]: pgmap v17365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:42.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:41 smithi090 ceph-mon[29180]: pgmap v17365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:43.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31523.327402 secs 2024-04-27T03:20:43.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31523.327402 secs 2024-04-27T03:20:43.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31523.327402 secs 2024-04-27T03:20:44.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:43 smithi119 ceph-mon[32752]: pgmap v17366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:43 smithi167 ceph-mon[32751]: pgmap v17366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:44.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:43 smithi090 ceph-mon[29180]: pgmap v17366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:45.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:44 smithi119 ceph-mon[32752]: pgmap v17367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:44 smithi167 ceph-mon[32751]: pgmap v17367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:45.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:44 smithi090 ceph-mon[29180]: pgmap v17367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31527.471999 secs 2024-04-27T03:20:46.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31527.471999 secs 2024-04-27T03:20:46.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31527.471999 secs 2024-04-27T03:20:47.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:46 smithi167 ceph-mon[32751]: pgmap v17368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:47.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:46 smithi090 ceph-mon[29180]: pgmap v17368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:47.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:46 smithi119 ceph-mon[32752]: pgmap v17368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:48.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31528.327518 secs 2024-04-27T03:20:48.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31528.327518 secs 2024-04-27T03:20:48.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31528.327518 secs 2024-04-27T03:20:49.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:48 smithi090 ceph-mon[29180]: pgmap v17369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:49.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:48 smithi119 ceph-mon[32752]: pgmap v17369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:49.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:48 smithi167 ceph-mon[32751]: pgmap v17369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:50.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31532.472059 secs 2024-04-27T03:20:50.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31532.472059 secs 2024-04-27T03:20:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31532.472059 secs 2024-04-27T03:20:51.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:51 smithi119 ceph-mon[32752]: pgmap v17370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:51 smithi167 ceph-mon[32751]: pgmap v17370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:51 smithi090 ceph-mon[29180]: pgmap v17370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:52.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31533.327630 secs 2024-04-27T03:20:52.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31533.327630 secs 2024-04-27T03:20:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31533.327630 secs 2024-04-27T03:20:53.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:53 smithi119 ceph-mon[32752]: pgmap v17371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:53.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:53 smithi167 ceph-mon[32751]: pgmap v17371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:53.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:53 smithi090 ceph-mon[29180]: pgmap v17371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:55.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:54 smithi119 ceph-mon[32752]: pgmap v17372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:55.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:54 smithi167 ceph-mon[32751]: pgmap v17372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:55.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:54 smithi090 ceph-mon[29180]: pgmap v17372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31537.472140 secs 2024-04-27T03:20:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31537.472140 secs 2024-04-27T03:20:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31537.472140 secs 2024-04-27T03:20:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:56 smithi167 ceph-mon[32751]: pgmap v17373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:56 smithi090 ceph-mon[29180]: pgmap v17373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:56 smithi119 ceph-mon[32752]: pgmap v17373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:20:58.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31538.327782 secs 2024-04-27T03:20:58.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31538.327782 secs 2024-04-27T03:20:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31538.327782 secs 2024-04-27T03:20:59.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:20:58 smithi090 ceph-mon[29180]: pgmap v17374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:59.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:20:58 smithi119 ceph-mon[32752]: pgmap v17374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:20:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:20:58 smithi167 ceph-mon[32751]: pgmap v17374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:00.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31542.472223 secs 2024-04-27T03:21:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31542.472223 secs 2024-04-27T03:21:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31542.472223 secs 2024-04-27T03:21:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:01 smithi119 ceph-mon[32752]: pgmap v17375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:01.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:01 smithi167 ceph-mon[32751]: pgmap v17375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:01.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:01 smithi090 ceph-mon[29180]: pgmap v17375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31543.327895 secs 2024-04-27T03:21:02.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31543.327895 secs 2024-04-27T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31543.327895 secs 2024-04-27T03:21:03.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:03 smithi119 ceph-mon[32752]: pgmap v17376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:03 smithi167 ceph-mon[32751]: pgmap v17376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:03.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:03 smithi090 ceph-mon[29180]: pgmap v17376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:04 smithi119 ceph-mon[32752]: pgmap v17377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:05.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:04 smithi167 ceph-mon[32751]: pgmap v17377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:05.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:04 smithi090 ceph-mon[29180]: pgmap v17377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31547.472303 secs 2024-04-27T03:21:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31547.472303 secs 2024-04-27T03:21:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31547.472303 secs 2024-04-27T03:21:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:06 smithi167 ceph-mon[32751]: pgmap v17378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:07.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:06 smithi090 ceph-mon[29180]: pgmap v17378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:06 smithi119 ceph-mon[32752]: pgmap v17378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:08.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31548.327991 secs 2024-04-27T03:21:08.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31548.327991 secs 2024-04-27T03:21:08.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31548.327991 secs 2024-04-27T03:21:09.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:08 smithi167 ceph-mon[32751]: pgmap v17379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:09.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:08 smithi090 ceph-mon[29180]: pgmap v17379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:09.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:08 smithi119 ceph-mon[32752]: pgmap v17379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:09.883 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:21:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:21:09.545+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:21:09.883 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:21:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:21:09.545+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:21:09.883 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:21:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:21:09.545+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:21:09.883 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:21:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:21:09.545+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:21:09.883 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:21:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:21:09.545+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:21:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:10 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31552.472383 secs 2024-04-27T03:21:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:10 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31552.472383 secs 2024-04-27T03:21:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:10 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31552.472383 secs 2024-04-27T03:21:11.711 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:11 smithi119 ceph-mon[32752]: pgmap v17380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:11 smithi167 ceph-mon[32751]: pgmap v17380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:11 smithi090 ceph-mon[29180]: pgmap v17380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:12.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31553.328117 secs 2024-04-27T03:21:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31553.328117 secs 2024-04-27T03:21:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31553.328117 secs 2024-04-27T03:21:13.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:13 smithi119 ceph-mon[32752]: pgmap v17381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:13.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:13 smithi167 ceph-mon[32751]: pgmap v17381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:13.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:13 smithi090 ceph-mon[29180]: pgmap v17381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:15.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:14 smithi167 ceph-mon[32751]: pgmap v17382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:21:15.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:14 smithi090 ceph-mon[29180]: pgmap v17382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:21:15.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:14 smithi119 ceph-mon[32752]: pgmap v17382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:21:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:15 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31557.472466 secs 2024-04-27T03:21:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:15 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31557.472466 secs 2024-04-27T03:21:16.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:15 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31557.472466 secs 2024-04-27T03:21:17.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:16 smithi167 ceph-mon[32751]: pgmap v17383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:21:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:16 smithi090 ceph-mon[29180]: pgmap v17383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:21:17.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:16 smithi119 ceph-mon[32752]: pgmap v17383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:21:18.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31558.328214 secs 2024-04-27T03:21:18.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31558.328214 secs 2024-04-27T03:21:18.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31558.328214 secs 2024-04-27T03:21:19.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:18 smithi090 ceph-mon[29180]: pgmap v17384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:21:19.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:18 smithi119 ceph-mon[32752]: pgmap v17384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:21:19.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:18 smithi167 ceph-mon[32751]: pgmap v17384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:21:20.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:20 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31562.472548 secs 2024-04-27T03:21:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:20 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31562.472548 secs 2024-04-27T03:21:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:20 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31562.472548 secs 2024-04-27T03:21:21.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:21 smithi119 ceph-mon[32752]: pgmap v17385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:21:21.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:21:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:21 smithi167 ceph-mon[32751]: pgmap v17385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:21:21.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:21:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:21 smithi090 ceph-mon[29180]: pgmap v17385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:21:21.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:21:22.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31563.328306 secs 2024-04-27T03:21:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:21:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:21:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:22 smithi119 ceph-mon[32752]: pgmap v17386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 472 B/s wr, 14 op/s 2024-04-27T03:21:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:22 smithi119 ceph-mon[32752]: pgmap v17387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 579 B/s wr, 18 op/s 2024-04-27T03:21:22.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:21:22.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31563.328306 secs 2024-04-27T03:21:22.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:21:22.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:21:22.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:22 smithi167 ceph-mon[32751]: pgmap v17386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 472 B/s wr, 14 op/s 2024-04-27T03:21:22.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:22 smithi167 ceph-mon[32751]: pgmap v17387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 579 B/s wr, 18 op/s 2024-04-27T03:21:22.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:21:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31563.328306 secs 2024-04-27T03:21:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:21:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:21:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:22 smithi090 ceph-mon[29180]: pgmap v17386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 472 B/s wr, 14 op/s 2024-04-27T03:21:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:22 smithi090 ceph-mon[29180]: pgmap v17387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 579 B/s wr, 18 op/s 2024-04-27T03:21:22.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:21:24.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:24 smithi090 ceph-mon[29180]: pgmap v17388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:24 smithi119 ceph-mon[32752]: pgmap v17388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:24 smithi167 ceph-mon[32751]: pgmap v17388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:25 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31567.472635 secs 2024-04-27T03:21:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:25 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31567.472635 secs 2024-04-27T03:21:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:25 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31567.472635 secs 2024-04-27T03:21:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:26 smithi090 ceph-mon[29180]: pgmap v17389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:26 smithi119 ceph-mon[32752]: pgmap v17389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:26 smithi167 ceph-mon[32751]: pgmap v17389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:27.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31568.328441 secs 2024-04-27T03:21:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31568.328441 secs 2024-04-27T03:21:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31568.328441 secs 2024-04-27T03:21:28.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:28 smithi090 ceph-mon[29180]: pgmap v17390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 463 B/s rd, 1 op/s 2024-04-27T03:21:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:28 smithi119 ceph-mon[32752]: pgmap v17390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 463 B/s rd, 1 op/s 2024-04-27T03:21:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:28 smithi167 ceph-mon[32751]: pgmap v17390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 463 B/s rd, 1 op/s 2024-04-27T03:21:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:30 smithi090 ceph-mon[29180]: pgmap v17391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:30 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31572.472718 secs 2024-04-27T03:21:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:30 smithi119 ceph-mon[32752]: pgmap v17391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:30 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31572.472718 secs 2024-04-27T03:21:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:30 smithi167 ceph-mon[32751]: pgmap v17391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:21:31.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:30 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31572.472718 secs 2024-04-27T03:21:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31573.328563 secs 2024-04-27T03:21:32.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:32 smithi090 ceph-mon[29180]: pgmap v17392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T03:21:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31573.328563 secs 2024-04-27T03:21:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:32 smithi119 ceph-mon[32752]: pgmap v17392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T03:21:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31573.328563 secs 2024-04-27T03:21:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:32 smithi167 ceph-mon[32751]: pgmap v17392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 921 B/s rd, 1 op/s 2024-04-27T03:21:34.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:34 smithi090 ceph-mon[29180]: pgmap v17393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:35.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:34 smithi119 ceph-mon[32752]: pgmap v17393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:34 smithi167 ceph-mon[32751]: pgmap v17393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:35 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31577.472800 secs 2024-04-27T03:21:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:35 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31577.472800 secs 2024-04-27T03:21:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:35 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31577.472800 secs 2024-04-27T03:21:37.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:36 smithi119 ceph-mon[32752]: pgmap v17394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:36 smithi167 ceph-mon[32751]: pgmap v17394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:36 smithi090 ceph-mon[29180]: pgmap v17394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:38.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31578.328687 secs 2024-04-27T03:21:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31578.328687 secs 2024-04-27T03:21:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31578.328687 secs 2024-04-27T03:21:39.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:38 smithi119 ceph-mon[32752]: pgmap v17395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:39.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:38 smithi167 ceph-mon[32751]: pgmap v17395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:38 smithi090 ceph-mon[29180]: pgmap v17395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:40 smithi119 ceph-mon[32752]: pgmap v17396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:40 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31582.472824 secs 2024-04-27T03:21:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:40 smithi167 ceph-mon[32751]: pgmap v17396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:40 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31582.472824 secs 2024-04-27T03:21:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:40 smithi090 ceph-mon[29180]: pgmap v17396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:40 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31582.472824 secs 2024-04-27T03:21:43.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31583.328826 secs 2024-04-27T03:21:43.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:42 smithi119 ceph-mon[32752]: pgmap v17397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:43.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31583.328826 secs 2024-04-27T03:21:43.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:42 smithi167 ceph-mon[32751]: pgmap v17397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31583.328826 secs 2024-04-27T03:21:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:42 smithi090 ceph-mon[29180]: pgmap v17397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:45.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:44 smithi119 ceph-mon[32752]: pgmap v17398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:44 smithi167 ceph-mon[32751]: pgmap v17398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:45.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:44 smithi090 ceph-mon[29180]: pgmap v17398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:45 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31587.472883 secs 2024-04-27T03:21:46.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:45 smithi090 ceph-mon[29180]: pgmap v17399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:46.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:45 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31587.472883 secs 2024-04-27T03:21:46.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:45 smithi119 ceph-mon[32752]: pgmap v17399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:46.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:45 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31587.472883 secs 2024-04-27T03:21:46.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:45 smithi167 ceph-mon[32751]: pgmap v17399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:47.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31588.328955 secs 2024-04-27T03:21:47.561 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31588.328955 secs 2024-04-27T03:21:47.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31588.328955 secs 2024-04-27T03:21:48.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:48 smithi090 ceph-mon[29180]: pgmap v17400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:48.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:48 smithi119 ceph-mon[32752]: pgmap v17400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:48.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:48 smithi167 ceph-mon[32751]: pgmap v17400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:50.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:50 smithi090 ceph-mon[29180]: pgmap v17401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:50.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:50 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31592.472946 secs 2024-04-27T03:21:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:50 smithi119 ceph-mon[32752]: pgmap v17401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:50 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31592.472946 secs 2024-04-27T03:21:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:50 smithi167 ceph-mon[32751]: pgmap v17401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:50 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31592.472946 secs 2024-04-27T03:21:52.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31593.329058 secs 2024-04-27T03:21:52.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:52 smithi090 ceph-mon[29180]: pgmap v17402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:53.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31593.329058 secs 2024-04-27T03:21:53.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:52 smithi119 ceph-mon[32752]: pgmap v17402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31593.329058 secs 2024-04-27T03:21:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:52 smithi167 ceph-mon[32751]: pgmap v17402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:54 smithi090 ceph-mon[29180]: pgmap v17403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:55.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:54 smithi119 ceph-mon[32752]: pgmap v17403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:55.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:54 smithi167 ceph-mon[32751]: pgmap v17403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:21:55.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:55 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31597.473027 secs 2024-04-27T03:21:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:55 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31597.473027 secs 2024-04-27T03:21:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:55 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31597.473027 secs 2024-04-27T03:21:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:56 smithi090 ceph-mon[29180]: pgmap v17404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:57.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:56 smithi119 ceph-mon[32752]: pgmap v17404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:56 smithi167 ceph-mon[32751]: pgmap v17404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31598.329206 secs 2024-04-27T03:21:58.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31598.329206 secs 2024-04-27T03:21:58.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31598.329206 secs 2024-04-27T03:21:59.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:21:58 smithi119 ceph-mon[32752]: pgmap v17405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:59.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:21:58 smithi167 ceph-mon[32751]: pgmap v17405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:21:59.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:21:58 smithi090 ceph-mon[29180]: pgmap v17405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:00 smithi119 ceph-mon[32752]: pgmap v17406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:00 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31602.473088 secs 2024-04-27T03:22:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:00 smithi167 ceph-mon[32751]: pgmap v17406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:00 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31602.473088 secs 2024-04-27T03:22:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:00 smithi090 ceph-mon[29180]: pgmap v17406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:00 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31602.473088 secs 2024-04-27T03:22:03.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31603.329348 secs 2024-04-27T03:22:03.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:02 smithi119 ceph-mon[32752]: pgmap v17407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:03.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31603.329348 secs 2024-04-27T03:22:03.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:02 smithi167 ceph-mon[32751]: pgmap v17407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:03.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31603.329348 secs 2024-04-27T03:22:03.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:02 smithi090 ceph-mon[29180]: pgmap v17407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:04.934 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:04 smithi090 ceph-mon[29180]: pgmap v17408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:04 smithi119 ceph-mon[32752]: pgmap v17408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:05.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:04 smithi167 ceph-mon[32751]: pgmap v17408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:05 smithi119 ceph-mon[32752]: 11 slow requests, 0 included below; oldest blocked for > 31607.473168 secs 2024-04-27T03:22:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:05 smithi167 ceph-mon[32751]: 11 slow requests, 0 included below; oldest blocked for > 31607.473168 secs 2024-04-27T03:22:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:05 smithi090 ceph-mon[29180]: 11 slow requests, 0 included below; oldest blocked for > 31607.473168 secs 2024-04-27T03:22:07.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:06 smithi119 ceph-mon[32752]: pgmap v17409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:06 smithi167 ceph-mon[32751]: pgmap v17409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:06 smithi090 ceph-mon[29180]: pgmap v17409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:08.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31608.329422 secs 2024-04-27T03:22:08.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31608.329422 secs 2024-04-27T03:22:08.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31608.329422 secs 2024-04-27T03:22:09.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:08 smithi119 ceph-mon[32752]: pgmap v17410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:09.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:08 smithi167 ceph-mon[32751]: pgmap v17410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:09.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:08 smithi090 ceph-mon[29180]: pgmap v17410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:10 smithi119 ceph-mon[32752]: pgmap v17411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:10 smithi119 ceph-mon[32752]: 11 slow requests, 1 included below; oldest blocked for > 31612.473230 secs 2024-04-27T03:22:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:10 smithi119 ceph-mon[32752]: slow request 360.805639 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:22:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:10 smithi167 ceph-mon[32751]: pgmap v17411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:10 smithi167 ceph-mon[32751]: 11 slow requests, 1 included below; oldest blocked for > 31612.473230 secs 2024-04-27T03:22:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:10 smithi167 ceph-mon[32751]: slow request 360.805639 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:22:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:10 smithi090 ceph-mon[29180]: pgmap v17411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:10 smithi090 ceph-mon[29180]: 11 slow requests, 1 included below; oldest blocked for > 31612.473230 secs 2024-04-27T03:22:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:10 smithi090 ceph-mon[29180]: slow request 360.805639 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:22:13.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31613.329540 secs 2024-04-27T03:22:13.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:12 smithi119 ceph-mon[32752]: pgmap v17412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:13.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31613.329540 secs 2024-04-27T03:22:13.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:12 smithi167 ceph-mon[32751]: pgmap v17412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:13.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31613.329540 secs 2024-04-27T03:22:13.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:12 smithi090 ceph-mon[29180]: pgmap v17412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:22:15.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:14 smithi119 ceph-mon[32752]: pgmap v17413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:15.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:14 smithi167 ceph-mon[32751]: pgmap v17413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:15.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:14 smithi090 ceph-mon[29180]: pgmap v17413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:22:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:15 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 31617.473309 secs 2024-04-27T03:22:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:15 smithi119 ceph-mon[32752]: slow request 180.276855 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:22:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:15 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 31617.473309 secs 2024-04-27T03:22:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:15 smithi167 ceph-mon[32751]: slow request 180.276855 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:22:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:15 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 31617.473309 secs 2024-04-27T03:22:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:15 smithi090 ceph-mon[29180]: slow request 180.276855 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:22:17.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:16 smithi090 ceph-mon[29180]: pgmap v17414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:17.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:16 smithi119 ceph-mon[32752]: pgmap v17414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:17.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:16 smithi167 ceph-mon[32751]: pgmap v17414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:18.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31618.329652 secs 2024-04-27T03:22:18.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31618.329652 secs 2024-04-27T03:22:18.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31618.329652 secs 2024-04-27T03:22:19.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:18 smithi119 ceph-mon[32752]: pgmap v17415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:19.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:18 smithi167 ceph-mon[32751]: pgmap v17415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:19.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:18 smithi090 ceph-mon[29180]: pgmap v17415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:20.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:19 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:22:20.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:19 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:22:20.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:19 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:22:20.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:19 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:22:20.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:19 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:22:20.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:19 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:22:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:20 smithi090 ceph-mon[29180]: pgmap v17416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31622.473381 secs 2024-04-27T03:22:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:20 smithi119 ceph-mon[32752]: pgmap v17416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31622.473381 secs 2024-04-27T03:22:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:20 smithi167 ceph-mon[32751]: pgmap v17416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31622.473381 secs 2024-04-27T03:22:22.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:22:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:22:22.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:22:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31623.329754 secs 2024-04-27T03:22:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:22 smithi119 ceph-mon[32752]: pgmap v17417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:22:23.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:22:23.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:22:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31623.329754 secs 2024-04-27T03:22:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:22 smithi167 ceph-mon[32751]: pgmap v17417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:22:23.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:22:23.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:22:23.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31623.329754 secs 2024-04-27T03:22:23.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:22 smithi090 ceph-mon[29180]: pgmap v17417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:23.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:22:23.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:22:23.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:22:24.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:23 smithi119 ceph-mon[32752]: pgmap v17418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:22:24.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:23 smithi119 ceph-mon[32752]: pgmap v17419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:22:24.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:23 smithi167 ceph-mon[32751]: pgmap v17418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:22:24.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:23 smithi167 ceph-mon[32751]: pgmap v17419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:22:24.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:23 smithi090 ceph-mon[29180]: pgmap v17418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:22:24.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:23 smithi090 ceph-mon[29180]: pgmap v17419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:22:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:25 smithi119 ceph-mon[32752]: pgmap v17420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31627.473463 secs 2024-04-27T03:22:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:25 smithi167 ceph-mon[32751]: pgmap v17420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31627.473463 secs 2024-04-27T03:22:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:25 smithi090 ceph-mon[29180]: pgmap v17420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31627.473463 secs 2024-04-27T03:22:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31628.329888 secs 2024-04-27T03:22:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:27 smithi119 ceph-mon[32752]: pgmap v17421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31628.329888 secs 2024-04-27T03:22:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:27 smithi167 ceph-mon[32751]: pgmap v17421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:28.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31628.329888 secs 2024-04-27T03:22:28.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:27 smithi090 ceph-mon[29180]: pgmap v17421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:30.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:29 smithi119 ceph-mon[32752]: pgmap v17422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:30.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:29 smithi167 ceph-mon[32751]: pgmap v17422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:30.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:29 smithi090 ceph-mon[29180]: pgmap v17422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31632.473544 secs 2024-04-27T03:22:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31632.473544 secs 2024-04-27T03:22:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31632.473544 secs 2024-04-27T03:22:32.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:31 smithi119 ceph-mon[32752]: pgmap v17423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:31 smithi167 ceph-mon[32751]: pgmap v17423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:32.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:31 smithi090 ceph-mon[29180]: pgmap v17423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:22:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31633.330022 secs 2024-04-27T03:22:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31633.330022 secs 2024-04-27T03:22:33.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31633.330022 secs 2024-04-27T03:22:34.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:33 smithi119 ceph-mon[32752]: pgmap v17424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:22:34.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:33 smithi167 ceph-mon[32751]: pgmap v17424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:22:34.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:33 smithi090 ceph-mon[29180]: pgmap v17424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:22:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:35 smithi119 ceph-mon[32752]: pgmap v17425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31637.473626 secs 2024-04-27T03:22:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:35 smithi167 ceph-mon[32751]: pgmap v17425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31637.473626 secs 2024-04-27T03:22:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:35 smithi090 ceph-mon[29180]: pgmap v17425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31637.473626 secs 2024-04-27T03:22:38.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31638.330138 secs 2024-04-27T03:22:38.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:37 smithi119 ceph-mon[32752]: pgmap v17426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31638.330138 secs 2024-04-27T03:22:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:37 smithi167 ceph-mon[32751]: pgmap v17426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31638.330138 secs 2024-04-27T03:22:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:37 smithi090 ceph-mon[29180]: pgmap v17426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:40.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:39 smithi119 ceph-mon[32752]: pgmap v17427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:40.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:39 smithi167 ceph-mon[32751]: pgmap v17427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:40.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:39 smithi090 ceph-mon[29180]: pgmap v17427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31642.473706 secs 2024-04-27T03:22:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31642.473706 secs 2024-04-27T03:22:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31642.473706 secs 2024-04-27T03:22:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:41 smithi119 ceph-mon[32752]: pgmap v17428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:41 smithi167 ceph-mon[32751]: pgmap v17428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:41 smithi090 ceph-mon[29180]: pgmap v17428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:43.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31643.330206 secs 2024-04-27T03:22:43.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31643.330206 secs 2024-04-27T03:22:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31643.330206 secs 2024-04-27T03:22:44.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:43 smithi119 ceph-mon[32752]: pgmap v17429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:43 smithi167 ceph-mon[32751]: pgmap v17429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:44.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:43 smithi090 ceph-mon[29180]: pgmap v17429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:45 smithi119 ceph-mon[32752]: pgmap v17430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31647.473763 secs 2024-04-27T03:22:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:45 smithi167 ceph-mon[32751]: pgmap v17430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31647.473763 secs 2024-04-27T03:22:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:45 smithi090 ceph-mon[29180]: pgmap v17430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31647.473763 secs 2024-04-27T03:22:48.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31648.330323 secs 2024-04-27T03:22:48.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:47 smithi119 ceph-mon[32752]: pgmap v17431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:48.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31648.330323 secs 2024-04-27T03:22:48.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:47 smithi167 ceph-mon[32751]: pgmap v17431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:48.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31648.330323 secs 2024-04-27T03:22:48.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:47 smithi090 ceph-mon[29180]: pgmap v17431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:50.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:49 smithi119 ceph-mon[32752]: pgmap v17432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:50.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:49 smithi167 ceph-mon[32751]: pgmap v17432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:50.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:49 smithi090 ceph-mon[29180]: pgmap v17432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31652.473851 secs 2024-04-27T03:22:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:50 smithi090 ceph-mon[29180]: pgmap v17433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31652.473851 secs 2024-04-27T03:22:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:50 smithi119 ceph-mon[32752]: pgmap v17433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31652.473851 secs 2024-04-27T03:22:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:50 smithi167 ceph-mon[32751]: pgmap v17433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:52.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31653.330442 secs 2024-04-27T03:22:52.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31653.330442 secs 2024-04-27T03:22:52.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31653.330442 secs 2024-04-27T03:22:53.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:53 smithi090 ceph-mon[29180]: pgmap v17434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:53.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:53 smithi119 ceph-mon[32752]: pgmap v17434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:53.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:53 smithi167 ceph-mon[32751]: pgmap v17434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:55.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:55 smithi119 ceph-mon[32752]: pgmap v17435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:55.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31657.473931 secs 2024-04-27T03:22:55.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:55 smithi167 ceph-mon[32751]: pgmap v17435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:55.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31657.473931 secs 2024-04-27T03:22:55.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:55 smithi090 ceph-mon[29180]: pgmap v17435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:55.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31657.473931 secs 2024-04-27T03:22:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31658.330562 secs 2024-04-27T03:22:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:57 smithi119 ceph-mon[32752]: pgmap v17436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31658.330562 secs 2024-04-27T03:22:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:57 smithi167 ceph-mon[32751]: pgmap v17436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31658.330562 secs 2024-04-27T03:22:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:57 smithi090 ceph-mon[29180]: pgmap v17436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:22:59.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:22:59 smithi119 ceph-mon[32752]: pgmap v17437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:59.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:22:59 smithi167 ceph-mon[32751]: pgmap v17437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:22:59.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:22:59 smithi090 ceph-mon[29180]: pgmap v17437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:00.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31662.473995 secs 2024-04-27T03:23:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31662.473995 secs 2024-04-27T03:23:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31662.473995 secs 2024-04-27T03:23:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:01 smithi119 ceph-mon[32752]: pgmap v17438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:01.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:01 smithi167 ceph-mon[32751]: pgmap v17438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:01.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:01 smithi090 ceph-mon[29180]: pgmap v17438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:02.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31663.330655 secs 2024-04-27T03:23:02.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31663.330655 secs 2024-04-27T03:23:02.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31663.330655 secs 2024-04-27T03:23:03.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:03 smithi119 ceph-mon[32752]: pgmap v17439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:03.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:03 smithi167 ceph-mon[32751]: pgmap v17439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:03.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:03 smithi090 ceph-mon[29180]: pgmap v17439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:05.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:05 smithi119 ceph-mon[32752]: pgmap v17440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:05.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31667.474055 secs 2024-04-27T03:23:05.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:05 smithi167 ceph-mon[32751]: pgmap v17440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:05.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31667.474055 secs 2024-04-27T03:23:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:05 smithi090 ceph-mon[29180]: pgmap v17440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31667.474055 secs 2024-04-27T03:23:07.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31668.330788 secs 2024-04-27T03:23:07.684 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:07 smithi090 ceph-mon[29180]: pgmap v17441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:07.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31668.330788 secs 2024-04-27T03:23:07.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:07 smithi119 ceph-mon[32752]: pgmap v17441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:07.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31668.330788 secs 2024-04-27T03:23:07.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:07 smithi167 ceph-mon[32751]: pgmap v17441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:09.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:09 smithi119 ceph-mon[32752]: pgmap v17442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:09.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:09 smithi167 ceph-mon[32751]: pgmap v17442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:09.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:09 smithi090 ceph-mon[29180]: pgmap v17442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:10.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:10 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 31672.474114 secs 2024-04-27T03:23:10.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:10 smithi167 ceph-mon[32751]: slow request 11523.881560 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:23:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:10 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 31672.474114 secs 2024-04-27T03:23:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:10 smithi119 ceph-mon[32752]: slow request 11523.881560 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:23:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:10 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 31672.474114 secs 2024-04-27T03:23:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:10 smithi090 ceph-mon[29180]: slow request 11523.881560 seconds old, received at 2024-04-27T00:11:06.345958+0000: client_request(client.26807:1 getattr pAsLsXsFs #0x1 2024-04-27T00:11:06.339305+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:23:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:11 smithi119 ceph-mon[32752]: pgmap v17443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:11 smithi167 ceph-mon[32751]: pgmap v17443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:11 smithi090 ceph-mon[29180]: pgmap v17443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:12.711 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:12 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 31673.330883 secs 2024-04-27T03:23:12.711 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:12 smithi119 ceph-mon[32752]: slow request 5763.112933 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:23:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:12 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 31673.330883 secs 2024-04-27T03:23:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:12 smithi167 ceph-mon[32751]: slow request 5763.112933 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:23:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:12 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 31673.330883 secs 2024-04-27T03:23:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:12 smithi090 ceph-mon[29180]: slow request 5763.112933 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:23:13.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:13 smithi119 ceph-mon[32752]: pgmap v17444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:13.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:13 smithi167 ceph-mon[32751]: pgmap v17444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:13.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:13 smithi090 ceph-mon[29180]: pgmap v17444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:15 smithi119 ceph-mon[32752]: pgmap v17445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:15 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 31677.474195 secs 2024-04-27T03:23:15.813 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:15 smithi119 ceph-mon[32752]: slow request 5764.350976 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:23:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:15 smithi167 ceph-mon[32751]: pgmap v17445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:15 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 31677.474195 secs 2024-04-27T03:23:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:15 smithi167 ceph-mon[32751]: slow request 5764.350976 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:23:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:15 smithi090 ceph-mon[29180]: pgmap v17445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:15 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 31677.474195 secs 2024-04-27T03:23:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:15 smithi090 ceph-mon[29180]: slow request 5764.350976 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:23:17.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31678.331003 secs 2024-04-27T03:23:17.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:17 smithi119 ceph-mon[32752]: pgmap v17446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:17.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31678.331003 secs 2024-04-27T03:23:17.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:17 smithi167 ceph-mon[32751]: pgmap v17446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:17.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31678.331003 secs 2024-04-27T03:23:17.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:17 smithi090 ceph-mon[29180]: pgmap v17446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:19.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:19 smithi119 ceph-mon[32752]: pgmap v17447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:19.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:19 smithi167 ceph-mon[32751]: pgmap v17447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:19.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:19 smithi090 ceph-mon[29180]: pgmap v17447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:20.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31682.474246 secs 2024-04-27T03:23:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31682.474246 secs 2024-04-27T03:23:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31682.474246 secs 2024-04-27T03:23:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:21 smithi119 ceph-mon[32752]: pgmap v17448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:21 smithi167 ceph-mon[32751]: pgmap v17448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:21 smithi090 ceph-mon[29180]: pgmap v17448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:22.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31683.331134 secs 2024-04-27T03:23:22.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:23:22.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31683.331134 secs 2024-04-27T03:23:22.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:23:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31683.331134 secs 2024-04-27T03:23:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:23:23.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:23 smithi119 ceph-mon[32752]: pgmap v17449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:23.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:23:23.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:23:23.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:23:23.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:23 smithi167 ceph-mon[32751]: pgmap v17449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:23.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:23:23.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:23:23.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:23:23.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:23 smithi090 ceph-mon[29180]: pgmap v17449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:23.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:23:23.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:23:23.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:23:24.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:24 smithi119 ceph-mon[32752]: pgmap v17450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:23:24.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:24 smithi119 ceph-mon[32752]: pgmap v17451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:23:24.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:24 smithi167 ceph-mon[32751]: pgmap v17450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:23:24.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:24 smithi167 ceph-mon[32751]: pgmap v17451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:23:24.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:24 smithi090 ceph-mon[29180]: pgmap v17450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:23:24.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:24 smithi090 ceph-mon[29180]: pgmap v17451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:23:25.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31687.474329 secs 2024-04-27T03:23:25.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31687.474329 secs 2024-04-27T03:23:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31687.474329 secs 2024-04-27T03:23:26.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:26 smithi119 ceph-mon[32752]: pgmap v17452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:26 smithi167 ceph-mon[32751]: pgmap v17452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:26.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:26 smithi090 ceph-mon[29180]: pgmap v17452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:27.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31688.331254 secs 2024-04-27T03:23:27.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31688.331254 secs 2024-04-27T03:23:27.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31688.331254 secs 2024-04-27T03:23:28.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:28 smithi119 ceph-mon[32752]: pgmap v17453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:23:28.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:28 smithi167 ceph-mon[32751]: pgmap v17453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:23:28.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:28 smithi090 ceph-mon[29180]: pgmap v17453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:23:30.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:30 smithi119 ceph-mon[32752]: pgmap v17454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:30.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31692.474414 secs 2024-04-27T03:23:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:30 smithi167 ceph-mon[32751]: pgmap v17454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31692.474414 secs 2024-04-27T03:23:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:30 smithi090 ceph-mon[29180]: pgmap v17454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31692.474414 secs 2024-04-27T03:23:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31693.331413 secs 2024-04-27T03:23:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:32 smithi119 ceph-mon[32752]: pgmap v17455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:32.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31693.331413 secs 2024-04-27T03:23:32.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:32 smithi167 ceph-mon[32751]: pgmap v17455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31693.331413 secs 2024-04-27T03:23:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:32 smithi090 ceph-mon[29180]: pgmap v17455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:23:34.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:34 smithi119 ceph-mon[32752]: pgmap v17456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:23:34.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:34 smithi167 ceph-mon[32751]: pgmap v17456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:23:34.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:34 smithi090 ceph-mon[29180]: pgmap v17456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:23:35.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31697.474494 secs 2024-04-27T03:23:35.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31697.474494 secs 2024-04-27T03:23:35.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31697.474494 secs 2024-04-27T03:23:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:36 smithi119 ceph-mon[32752]: pgmap v17457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:36.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:36 smithi167 ceph-mon[32751]: pgmap v17457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:36 smithi090 ceph-mon[29180]: pgmap v17457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:37.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31698.331535 secs 2024-04-27T03:23:37.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31698.331535 secs 2024-04-27T03:23:37.835 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31698.331535 secs 2024-04-27T03:23:38.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:38 smithi119 ceph-mon[32752]: pgmap v17458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:38.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:38 smithi167 ceph-mon[32751]: pgmap v17458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:38.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:38 smithi090 ceph-mon[29180]: pgmap v17458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:40.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:40 smithi119 ceph-mon[32752]: pgmap v17459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:40.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31702.474575 secs 2024-04-27T03:23:40.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:40 smithi167 ceph-mon[32751]: pgmap v17459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:40.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31702.474575 secs 2024-04-27T03:23:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:40 smithi090 ceph-mon[29180]: pgmap v17459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31702.474575 secs 2024-04-27T03:23:42.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31703.331642 secs 2024-04-27T03:23:42.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:42 smithi119 ceph-mon[32752]: pgmap v17460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:42.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31703.331642 secs 2024-04-27T03:23:42.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:42 smithi167 ceph-mon[32751]: pgmap v17460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:42.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31703.331642 secs 2024-04-27T03:23:42.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:42 smithi090 ceph-mon[29180]: pgmap v17460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:44.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:44 smithi119 ceph-mon[32752]: pgmap v17461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:44.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:44 smithi167 ceph-mon[32751]: pgmap v17461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:44.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:44 smithi090 ceph-mon[29180]: pgmap v17461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:45.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31707.474655 secs 2024-04-27T03:23:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31707.474655 secs 2024-04-27T03:23:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31707.474655 secs 2024-04-27T03:23:46.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:46 smithi119 ceph-mon[32752]: pgmap v17462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:46 smithi167 ceph-mon[32751]: pgmap v17462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:46 smithi090 ceph-mon[29180]: pgmap v17462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:47.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31708.331760 secs 2024-04-27T03:23:47.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31708.331760 secs 2024-04-27T03:23:47.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31708.331760 secs 2024-04-27T03:23:48.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:48 smithi119 ceph-mon[32752]: pgmap v17463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:48.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:48 smithi167 ceph-mon[32751]: pgmap v17463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:48.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:48 smithi090 ceph-mon[29180]: pgmap v17463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:50.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:50 smithi119 ceph-mon[32752]: pgmap v17464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:50.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31712.474738 secs 2024-04-27T03:23:50.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:50 smithi167 ceph-mon[32751]: pgmap v17464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:50.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31712.474738 secs 2024-04-27T03:23:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:50 smithi090 ceph-mon[29180]: pgmap v17464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31712.474738 secs 2024-04-27T03:23:52.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31713.331878 secs 2024-04-27T03:23:52.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:52 smithi119 ceph-mon[32752]: pgmap v17465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:52.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31713.331878 secs 2024-04-27T03:23:52.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:52 smithi167 ceph-mon[32751]: pgmap v17465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31713.331878 secs 2024-04-27T03:23:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:52 smithi090 ceph-mon[29180]: pgmap v17465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:54.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:54 smithi119 ceph-mon[32752]: pgmap v17466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:54.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:54 smithi167 ceph-mon[32751]: pgmap v17466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:54 smithi090 ceph-mon[29180]: pgmap v17466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:55.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31717.474834 secs 2024-04-27T03:23:55.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31717.474834 secs 2024-04-27T03:23:55.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31717.474834 secs 2024-04-27T03:23:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:56 smithi167 ceph-mon[32751]: pgmap v17467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:56 smithi090 ceph-mon[29180]: pgmap v17467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:57.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:56 smithi119 ceph-mon[32752]: pgmap v17467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:23:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31718.331993 secs 2024-04-27T03:23:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31718.331993 secs 2024-04-27T03:23:58.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31718.331993 secs 2024-04-27T03:23:58.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:23:58 smithi167 ceph-mon[32751]: pgmap v17468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:23:58 smithi090 ceph-mon[29180]: pgmap v17468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:23:59.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:23:58 smithi119 ceph-mon[32752]: pgmap v17468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:00 smithi167 ceph-mon[32751]: pgmap v17469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31722.474894 secs 2024-04-27T03:24:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:00 smithi090 ceph-mon[29180]: pgmap v17469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31722.474894 secs 2024-04-27T03:24:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:00 smithi119 ceph-mon[32752]: pgmap v17469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31722.474894 secs 2024-04-27T03:24:02.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31723.332084 secs 2024-04-27T03:24:02.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:02 smithi090 ceph-mon[29180]: pgmap v17470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:03.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31723.332084 secs 2024-04-27T03:24:03.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:02 smithi119 ceph-mon[32752]: pgmap v17470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:03.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31723.332084 secs 2024-04-27T03:24:03.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:02 smithi167 ceph-mon[32751]: pgmap v17470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:04.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:04 smithi090 ceph-mon[29180]: pgmap v17471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:05.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:04 smithi119 ceph-mon[32752]: pgmap v17471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:05.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:04 smithi167 ceph-mon[32751]: pgmap v17471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31727.474948 secs 2024-04-27T03:24:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31727.474948 secs 2024-04-27T03:24:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31727.474948 secs 2024-04-27T03:24:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:06 smithi090 ceph-mon[29180]: pgmap v17472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:06 smithi119 ceph-mon[32752]: pgmap v17472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:06 smithi167 ceph-mon[32751]: pgmap v17472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:07.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:07 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 31728.332164 secs 2024-04-27T03:24:07.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:07 smithi090 ceph-mon[29180]: slow request 23042.968931 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:24:08.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:07 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 31728.332164 secs 2024-04-27T03:24:08.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:07 smithi119 ceph-mon[32752]: slow request 23042.968931 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:24:08.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:07 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 31728.332164 secs 2024-04-27T03:24:08.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:07 smithi167 ceph-mon[32751]: slow request 23042.968931 seconds old, received at 2024-04-26T21:00:03.113877+0000: client_request(client.17205:1 getattr pAsLsXsFs #0x1 2024-04-26T21:00:03.110209+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:24:08.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:08 smithi090 ceph-mon[29180]: pgmap v17473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:09.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:08 smithi119 ceph-mon[32752]: pgmap v17473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:09.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:08 smithi167 ceph-mon[32751]: pgmap v17473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:10.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:10 smithi167 ceph-mon[32751]: pgmap v17474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:10.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31732.475009 secs 2024-04-27T03:24:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:10 smithi090 ceph-mon[29180]: pgmap v17474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31732.475009 secs 2024-04-27T03:24:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:10 smithi119 ceph-mon[32752]: pgmap v17474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31732.475009 secs 2024-04-27T03:24:12.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31733.332288 secs 2024-04-27T03:24:12.862 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:12 smithi119 ceph-mon[32752]: pgmap v17475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31733.332288 secs 2024-04-27T03:24:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:12 smithi090 ceph-mon[29180]: pgmap v17475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:13.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31733.332288 secs 2024-04-27T03:24:13.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:12 smithi167 ceph-mon[32751]: pgmap v17475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:14.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:14 smithi090 ceph-mon[29180]: pgmap v17476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:15.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:14 smithi119 ceph-mon[32752]: pgmap v17476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:15.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:14 smithi167 ceph-mon[32751]: pgmap v17476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31737.475110 secs 2024-04-27T03:24:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31737.475110 secs 2024-04-27T03:24:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31737.475110 secs 2024-04-27T03:24:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:16 smithi090 ceph-mon[29180]: pgmap v17477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:16 smithi119 ceph-mon[32752]: pgmap v17477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:16 smithi167 ceph-mon[32751]: pgmap v17477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:17.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31738.332392 secs 2024-04-27T03:24:18.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31738.332392 secs 2024-04-27T03:24:18.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31738.332392 secs 2024-04-27T03:24:18.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:18 smithi090 ceph-mon[29180]: pgmap v17478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:19.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:18 smithi119 ceph-mon[32752]: pgmap v17478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:19.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:18 smithi167 ceph-mon[32751]: pgmap v17478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:20 smithi090 ceph-mon[29180]: pgmap v17479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31742.475172 secs 2024-04-27T03:24:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:20 smithi119 ceph-mon[32752]: pgmap v17479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31742.475172 secs 2024-04-27T03:24:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:20 smithi167 ceph-mon[32751]: pgmap v17479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31742.475172 secs 2024-04-27T03:24:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31743.332453 secs 2024-04-27T03:24:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:22 smithi090 ceph-mon[29180]: pgmap v17480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31743.332453 secs 2024-04-27T03:24:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:22 smithi119 ceph-mon[32752]: pgmap v17480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31743.332453 secs 2024-04-27T03:24:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:22 smithi167 ceph-mon[32751]: pgmap v17480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:23.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:24:24.030 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:24:24.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:24:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:24 smithi119 ceph-mon[32752]: pgmap v17481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:24 smithi167 ceph-mon[32751]: pgmap v17481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:24 smithi090 ceph-mon[29180]: pgmap v17481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:25.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:25.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31747.475240 secs 2024-04-27T03:24:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31747.475240 secs 2024-04-27T03:24:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31747.475240 secs 2024-04-27T03:24:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: pgmap v17482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: pgmap v17482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: pgmap v17482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31748.332522 secs 2024-04-27T03:24:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:24:28.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:24:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:24:28.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31748.332522 secs 2024-04-27T03:24:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:24:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:24:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:24:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:24:28.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31748.332522 secs 2024-04-27T03:24:28.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:24:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:24:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:28.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:24:28.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:24:28.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:24:28.136 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:24:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:28 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:24:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:28 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:24:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:28 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:24:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:28 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:24:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:28 smithi119 ceph-mon[32752]: pgmap v17483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:28 smithi119 ceph-mon[32752]: pgmap v17484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:24:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:28 smithi119 ceph-mon[32752]: pgmap v17485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:24:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:28 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:24:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:28 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:24:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:28 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:24:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:28 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:24:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:28 smithi167 ceph-mon[32751]: pgmap v17483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:28 smithi167 ceph-mon[32751]: pgmap v17484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:24:29.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:28 smithi167 ceph-mon[32751]: pgmap v17485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:24:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:28 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:24:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:28 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:24:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:28 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:24:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:28 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:24:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:28 smithi090 ceph-mon[29180]: pgmap v17483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:28 smithi090 ceph-mon[29180]: pgmap v17484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:24:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:28 smithi090 ceph-mon[29180]: pgmap v17485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:24:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:30 smithi119 ceph-mon[32752]: pgmap v17486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31752.475300 secs 2024-04-27T03:24:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:30 smithi167 ceph-mon[32751]: pgmap v17486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31752.475300 secs 2024-04-27T03:24:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:30 smithi090 ceph-mon[29180]: pgmap v17486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31752.475300 secs 2024-04-27T03:24:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:32 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31753.332626 secs 2024-04-27T03:24:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:32 smithi119 ceph-mon[32752]: pgmap v17487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:32 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31753.332626 secs 2024-04-27T03:24:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:32 smithi167 ceph-mon[32751]: pgmap v17487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:33.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:32 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31753.332626 secs 2024-04-27T03:24:33.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:32 smithi090 ceph-mon[29180]: pgmap v17487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:34 smithi119 ceph-mon[32752]: pgmap v17488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:24:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:34 smithi167 ceph-mon[32751]: pgmap v17488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:24:35.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:34 smithi090 ceph-mon[29180]: pgmap v17488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:24:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31757.475382 secs 2024-04-27T03:24:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31757.475382 secs 2024-04-27T03:24:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31757.475382 secs 2024-04-27T03:24:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:36 smithi119 ceph-mon[32752]: pgmap v17489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:36 smithi167 ceph-mon[32751]: pgmap v17489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:36 smithi090 ceph-mon[29180]: pgmap v17489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:38.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:37 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31758.332744 secs 2024-04-27T03:24:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:37 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31758.332744 secs 2024-04-27T03:24:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:37 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31758.332744 secs 2024-04-27T03:24:39.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:38 smithi119 ceph-mon[32752]: pgmap v17490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:24:39.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:38 smithi167 ceph-mon[32751]: pgmap v17490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:24:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:38 smithi090 ceph-mon[29180]: pgmap v17490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:24:41.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:40 smithi090 ceph-mon[29180]: pgmap v17491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:41.048 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31762.475466 secs 2024-04-27T03:24:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:40 smithi119 ceph-mon[32752]: pgmap v17491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31762.475466 secs 2024-04-27T03:24:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:40 smithi167 ceph-mon[32751]: pgmap v17491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31762.475466 secs 2024-04-27T03:24:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:42 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31763.332814 secs 2024-04-27T03:24:43.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:42 smithi119 ceph-mon[32752]: pgmap v17492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:42 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31763.332814 secs 2024-04-27T03:24:43.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:42 smithi167 ceph-mon[32751]: pgmap v17492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:42 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31763.332814 secs 2024-04-27T03:24:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:42 smithi090 ceph-mon[29180]: pgmap v17492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:44.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:43 smithi119 ceph-mon[32752]: pgmap v17493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:43 smithi167 ceph-mon[32751]: pgmap v17493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:44.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:43 smithi090 ceph-mon[29180]: pgmap v17493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:45 smithi090 ceph-mon[29180]: pgmap v17494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31767.475547 secs 2024-04-27T03:24:46.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:45 smithi119 ceph-mon[32752]: pgmap v17494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:46.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31767.475547 secs 2024-04-27T03:24:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:45 smithi167 ceph-mon[32751]: pgmap v17494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:46.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31767.475547 secs 2024-04-27T03:24:47.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31768.332931 secs 2024-04-27T03:24:47.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31768.332931 secs 2024-04-27T03:24:47.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31768.332931 secs 2024-04-27T03:24:48.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:48 smithi119 ceph-mon[32752]: pgmap v17495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:48.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:48 smithi167 ceph-mon[32751]: pgmap v17495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:48.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:48 smithi090 ceph-mon[29180]: pgmap v17495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:50.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:50 smithi119 ceph-mon[32752]: pgmap v17496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:50.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:50 smithi167 ceph-mon[32751]: pgmap v17496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:50.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:50 smithi090 ceph-mon[29180]: pgmap v17496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:51.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:51 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31772.475631 secs 2024-04-27T03:24:51.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:51 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31772.475631 secs 2024-04-27T03:24:51.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:51 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31772.475631 secs 2024-04-27T03:24:52.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:52 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31773.333031 secs 2024-04-27T03:24:52.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:52 smithi119 ceph-mon[32752]: pgmap v17497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:52.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:52 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31773.333031 secs 2024-04-27T03:24:52.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:52 smithi167 ceph-mon[32751]: pgmap v17497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:52.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:52 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31773.333031 secs 2024-04-27T03:24:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:52 smithi090 ceph-mon[29180]: pgmap v17497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:54.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:54 smithi119 ceph-mon[32752]: pgmap v17498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:54.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:54 smithi167 ceph-mon[32751]: pgmap v17498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:54.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:54 smithi090 ceph-mon[29180]: pgmap v17498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:56.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:56 smithi119 ceph-mon[32752]: pgmap v17499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:56.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:56 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31777.475715 secs 2024-04-27T03:24:56.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:56 smithi167 ceph-mon[32751]: pgmap v17499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:56.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:56 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31777.475715 secs 2024-04-27T03:24:56.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:56 smithi090 ceph-mon[29180]: pgmap v17499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:24:56.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:56 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31777.475715 secs 2024-04-27T03:24:57.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:57 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31778.333171 secs 2024-04-27T03:24:57.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:57 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31778.333171 secs 2024-04-27T03:24:57.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:57 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31778.333171 secs 2024-04-27T03:24:58.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:24:58 smithi119 ceph-mon[32752]: pgmap v17500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:58.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:24:58 smithi167 ceph-mon[32751]: pgmap v17500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:24:58.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:24:58 smithi090 ceph-mon[29180]: pgmap v17500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:00.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:00 smithi119 ceph-mon[32752]: pgmap v17501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:00.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:00 smithi167 ceph-mon[32751]: pgmap v17501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:00.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:00 smithi090 ceph-mon[29180]: pgmap v17501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:01.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:01 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31782.475799 secs 2024-04-27T03:25:01.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:01 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31782.475799 secs 2024-04-27T03:25:01.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:01 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31782.475799 secs 2024-04-27T03:25:02.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31783.333250 secs 2024-04-27T03:25:02.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:02 smithi119 ceph-mon[32752]: pgmap v17502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:02.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31783.333250 secs 2024-04-27T03:25:02.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:02 smithi167 ceph-mon[32751]: pgmap v17502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:02.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31783.333250 secs 2024-04-27T03:25:02.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:02 smithi090 ceph-mon[29180]: pgmap v17502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:04.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:04 smithi119 ceph-mon[32752]: pgmap v17503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:04.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:04 smithi167 ceph-mon[32751]: pgmap v17503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:04.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:04 smithi090 ceph-mon[29180]: pgmap v17503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:06.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:06 smithi119 ceph-mon[32752]: pgmap v17504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:06.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:06 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31787.475883 secs 2024-04-27T03:25:06.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31788.333413 secs 2024-04-27T03:25:06.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:06 smithi167 ceph-mon[32751]: pgmap v17504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:06.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:06 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31787.475883 secs 2024-04-27T03:25:06.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31788.333413 secs 2024-04-27T03:25:06.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:06 smithi090 ceph-mon[29180]: pgmap v17504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:06.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:06 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31787.475883 secs 2024-04-27T03:25:06.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31788.333413 secs 2024-04-27T03:25:08.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:08 smithi119 ceph-mon[32752]: pgmap v17505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:08.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:08 smithi167 ceph-mon[32751]: pgmap v17505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:08.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:08 smithi090 ceph-mon[29180]: pgmap v17505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:10.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:10 smithi119 ceph-mon[32752]: pgmap v17506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:10.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:10 smithi167 ceph-mon[32751]: pgmap v17506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:10.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:10 smithi090 ceph-mon[29180]: pgmap v17506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:11.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:11 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31792.475966 secs 2024-04-27T03:25:11.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31793.333490 secs 2024-04-27T03:25:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:11 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31792.475966 secs 2024-04-27T03:25:11.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31793.333490 secs 2024-04-27T03:25:11.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:11 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31792.475966 secs 2024-04-27T03:25:11.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31793.333490 secs 2024-04-27T03:25:12.461 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:12 smithi119 ceph-mon[32752]: pgmap v17507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:12.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:12 smithi167 ceph-mon[32751]: pgmap v17507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:12.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:12 smithi090 ceph-mon[29180]: pgmap v17507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:14.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:14 smithi119 ceph-mon[32752]: pgmap v17508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:14.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:14 smithi167 ceph-mon[32751]: pgmap v17508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:14.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:14 smithi090 ceph-mon[29180]: pgmap v17508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:16.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:16 smithi119 ceph-mon[32752]: pgmap v17509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:16.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:16 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 31797.476028 secs 2024-04-27T03:25:16.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:16 smithi119 ceph-mon[32752]: slow request 360.279574 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:25:16.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31798.333606 secs 2024-04-27T03:25:16.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:16 smithi167 ceph-mon[32751]: pgmap v17509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:16.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:16 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 31797.476028 secs 2024-04-27T03:25:16.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:16 smithi167 ceph-mon[32751]: slow request 360.279574 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:25:16.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31798.333606 secs 2024-04-27T03:25:16.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:16 smithi090 ceph-mon[29180]: pgmap v17509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:16.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:16 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 31797.476028 secs 2024-04-27T03:25:16.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:16 smithi090 ceph-mon[29180]: slow request 360.279574 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:25:16.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31798.333606 secs 2024-04-27T03:25:18.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:18 smithi119 ceph-mon[32752]: pgmap v17510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:18.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:18 smithi167 ceph-mon[32751]: pgmap v17510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:18.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:18 smithi090 ceph-mon[29180]: pgmap v17510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:20.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:20 smithi119 ceph-mon[32752]: pgmap v17511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:20.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:20 smithi167 ceph-mon[32751]: pgmap v17511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:20.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:20 smithi090 ceph-mon[29180]: pgmap v17511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:21.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:21 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31802.476102 secs 2024-04-27T03:25:21.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31803.333716 secs 2024-04-27T03:25:21.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:21 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31802.476102 secs 2024-04-27T03:25:21.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31803.333716 secs 2024-04-27T03:25:21.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:21 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31802.476102 secs 2024-04-27T03:25:21.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31803.333716 secs 2024-04-27T03:25:22.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:22 smithi119 ceph-mon[32752]: pgmap v17512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:22.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:22 smithi167 ceph-mon[32751]: pgmap v17512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:22.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:22 smithi090 ceph-mon[29180]: pgmap v17512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:24.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:24 smithi119 ceph-mon[32752]: pgmap v17513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:24.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:24 smithi167 ceph-mon[32751]: pgmap v17513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:24.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:24 smithi090 ceph-mon[29180]: pgmap v17513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:26.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:26 smithi119 ceph-mon[32752]: pgmap v17514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:26.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:26 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31807.476167 secs 2024-04-27T03:25:26.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31808.333835 secs 2024-04-27T03:25:26.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:26 smithi167 ceph-mon[32751]: pgmap v17514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:26.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:26 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31807.476167 secs 2024-04-27T03:25:26.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31808.333835 secs 2024-04-27T03:25:26.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:26 smithi090 ceph-mon[29180]: pgmap v17514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:26.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:26 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31807.476167 secs 2024-04-27T03:25:26.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31808.333835 secs 2024-04-27T03:25:28.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:28 smithi119 ceph-mon[32752]: pgmap v17515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:28.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:25:28.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:28.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:28.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:28 smithi167 ceph-mon[32751]: pgmap v17515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:28.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:25:28.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:28.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:28.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:28 smithi090 ceph-mon[29180]: pgmap v17515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:28.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:25:28.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:28.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:29.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:25:29.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:25:29.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:29 smithi119 ceph-mon[32752]: pgmap v17516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:25:29.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:29 smithi119 ceph-mon[32752]: pgmap v17517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:29.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:29.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:25:29.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:25:29.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:29 smithi167 ceph-mon[32751]: pgmap v17516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:25:29.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:29 smithi167 ceph-mon[32751]: pgmap v17517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:29.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:29.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:25:29.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:25:29.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:29 smithi090 ceph-mon[29180]: pgmap v17516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:25:29.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:29 smithi090 ceph-mon[29180]: pgmap v17517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:29.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:25:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31812.476251 secs 2024-04-27T03:25:30.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31812.476251 secs 2024-04-27T03:25:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31812.476251 secs 2024-04-27T03:25:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:31 smithi119 ceph-mon[32752]: pgmap v17518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31813.333969 secs 2024-04-27T03:25:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:31 smithi167 ceph-mon[32751]: pgmap v17518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31813.333969 secs 2024-04-27T03:25:31.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:31 smithi090 ceph-mon[29180]: pgmap v17518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:31.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31813.333969 secs 2024-04-27T03:25:33.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:33 smithi119 ceph-mon[32752]: pgmap v17519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:33.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:33 smithi167 ceph-mon[32751]: pgmap v17519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:33.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:33 smithi090 ceph-mon[29180]: pgmap v17519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:35.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:35 smithi119 ceph-mon[32752]: pgmap v17520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:35.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31817.476349 secs 2024-04-27T03:25:35.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:35 smithi167 ceph-mon[32751]: pgmap v17520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:35.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31817.476349 secs 2024-04-27T03:25:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:35 smithi090 ceph-mon[29180]: pgmap v17520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:35.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31817.476349 secs 2024-04-27T03:25:36.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31818.334088 secs 2024-04-27T03:25:36.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31818.334088 secs 2024-04-27T03:25:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31818.334088 secs 2024-04-27T03:25:37.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:37 smithi119 ceph-mon[32752]: pgmap v17521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:37.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:37 smithi167 ceph-mon[32751]: pgmap v17521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:37.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:37 smithi090 ceph-mon[29180]: pgmap v17521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:25:39.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:39 smithi119 ceph-mon[32752]: pgmap v17522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:25:39.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:39 smithi167 ceph-mon[32751]: pgmap v17522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:25:39.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:39 smithi090 ceph-mon[29180]: pgmap v17522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:25:40.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31822.476409 secs 2024-04-27T03:25:40.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31822.476409 secs 2024-04-27T03:25:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31822.476409 secs 2024-04-27T03:25:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:41 smithi119 ceph-mon[32752]: pgmap v17523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31823.334152 secs 2024-04-27T03:25:41.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:41 smithi167 ceph-mon[32751]: pgmap v17523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:41.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31823.334152 secs 2024-04-27T03:25:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:41 smithi090 ceph-mon[29180]: pgmap v17523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31823.334152 secs 2024-04-27T03:25:43.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:43 smithi119 ceph-mon[32752]: pgmap v17524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:43.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:43 smithi167 ceph-mon[32751]: pgmap v17524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:43.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:43 smithi090 ceph-mon[29180]: pgmap v17524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:45.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:45 smithi119 ceph-mon[32752]: pgmap v17525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:45.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31827.476489 secs 2024-04-27T03:25:45.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:45 smithi167 ceph-mon[32751]: pgmap v17525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:45.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31827.476489 secs 2024-04-27T03:25:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:45 smithi090 ceph-mon[29180]: pgmap v17525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31827.476489 secs 2024-04-27T03:25:46.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31828.334227 secs 2024-04-27T03:25:46.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31828.334227 secs 2024-04-27T03:25:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31828.334227 secs 2024-04-27T03:25:47.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:47 smithi119 ceph-mon[32752]: pgmap v17526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:47.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:47 smithi167 ceph-mon[32751]: pgmap v17526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:47.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:47 smithi090 ceph-mon[29180]: pgmap v17526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:49.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:49 smithi119 ceph-mon[32752]: pgmap v17527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:49.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:49 smithi167 ceph-mon[32751]: pgmap v17527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:49.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:49 smithi090 ceph-mon[29180]: pgmap v17527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31832.476575 secs 2024-04-27T03:25:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31832.476575 secs 2024-04-27T03:25:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31832.476575 secs 2024-04-27T03:25:51.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:51 smithi119 ceph-mon[32752]: pgmap v17528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:51.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31833.334305 secs 2024-04-27T03:25:51.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:51 smithi167 ceph-mon[32751]: pgmap v17528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31833.334305 secs 2024-04-27T03:25:51.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:51 smithi090 ceph-mon[29180]: pgmap v17528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31833.334305 secs 2024-04-27T03:25:53.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:53 smithi119 ceph-mon[32752]: pgmap v17529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:53.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:53 smithi167 ceph-mon[32751]: pgmap v17529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:53.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:53 smithi090 ceph-mon[29180]: pgmap v17529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:55.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:55 smithi119 ceph-mon[32752]: pgmap v17530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:55.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31837.476659 secs 2024-04-27T03:25:55.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:55 smithi167 ceph-mon[32751]: pgmap v17530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:55.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31837.476659 secs 2024-04-27T03:25:55.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:55 smithi090 ceph-mon[29180]: pgmap v17530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:55.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31837.476659 secs 2024-04-27T03:25:56.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31838.334439 secs 2024-04-27T03:25:56.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31838.334439 secs 2024-04-27T03:25:56.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31838.334439 secs 2024-04-27T03:25:57.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:57 smithi119 ceph-mon[32752]: pgmap v17531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:57.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:57 smithi167 ceph-mon[32751]: pgmap v17531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:57.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:57 smithi090 ceph-mon[29180]: pgmap v17531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:25:59.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:25:59 smithi119 ceph-mon[32752]: pgmap v17532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:59.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:25:59 smithi167 ceph-mon[32751]: pgmap v17532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:25:59.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:25:59 smithi090 ceph-mon[29180]: pgmap v17532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:00.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31842.476742 secs 2024-04-27T03:26:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31842.476742 secs 2024-04-27T03:26:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31842.476742 secs 2024-04-27T03:26:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:01 smithi119 ceph-mon[32752]: pgmap v17533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31843.334521 secs 2024-04-27T03:26:01.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:01 smithi167 ceph-mon[32751]: pgmap v17533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:01.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31843.334521 secs 2024-04-27T03:26:01.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:01 smithi090 ceph-mon[29180]: pgmap v17533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:01.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31843.334521 secs 2024-04-27T03:26:03.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:03 smithi119 ceph-mon[32752]: pgmap v17534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:03.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:03 smithi167 ceph-mon[32751]: pgmap v17534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:03.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:03 smithi090 ceph-mon[29180]: pgmap v17534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:05.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:05 smithi119 ceph-mon[32752]: pgmap v17535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:05.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31847.476799 secs 2024-04-27T03:26:05.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:05 smithi167 ceph-mon[32751]: pgmap v17535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:05.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31847.476799 secs 2024-04-27T03:26:05.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:05 smithi090 ceph-mon[29180]: pgmap v17535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31847.476799 secs 2024-04-27T03:26:06.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31848.334590 secs 2024-04-27T03:26:06.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31848.334590 secs 2024-04-27T03:26:06.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31848.334590 secs 2024-04-27T03:26:07.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:07 smithi119 ceph-mon[32752]: pgmap v17536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:07.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:07 smithi167 ceph-mon[32751]: pgmap v17536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:07.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:07 smithi090 ceph-mon[29180]: pgmap v17536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:09.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:09 smithi119 ceph-mon[32752]: pgmap v17537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:09.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:09 smithi167 ceph-mon[32751]: pgmap v17537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:09.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:09 smithi090 ceph-mon[29180]: pgmap v17537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:10.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31852.476889 secs 2024-04-27T03:26:10.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31852.476889 secs 2024-04-27T03:26:10.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31852.476889 secs 2024-04-27T03:26:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:11 smithi119 ceph-mon[32752]: pgmap v17538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31853.334667 secs 2024-04-27T03:26:11.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:11 smithi167 ceph-mon[32751]: pgmap v17538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:11.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31853.334667 secs 2024-04-27T03:26:11.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:11 smithi090 ceph-mon[29180]: pgmap v17538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:11.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31853.334667 secs 2024-04-27T03:26:13.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:13 smithi119 ceph-mon[32752]: pgmap v17539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:13.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:13 smithi167 ceph-mon[32751]: pgmap v17539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:13.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:13 smithi090 ceph-mon[29180]: pgmap v17539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:15.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:15 smithi119 ceph-mon[32752]: pgmap v17540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31857.476976 secs 2024-04-27T03:26:15.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:15 smithi167 ceph-mon[32751]: pgmap v17540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31857.476976 secs 2024-04-27T03:26:15.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:15 smithi090 ceph-mon[29180]: pgmap v17540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31857.476976 secs 2024-04-27T03:26:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31858.334751 secs 2024-04-27T03:26:16.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31858.334751 secs 2024-04-27T03:26:16.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31858.334751 secs 2024-04-27T03:26:17.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:17 smithi119 ceph-mon[32752]: pgmap v17541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:17.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:17 smithi167 ceph-mon[32751]: pgmap v17541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:17.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:17 smithi090 ceph-mon[29180]: pgmap v17541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:19.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:19 smithi119 ceph-mon[32752]: pgmap v17542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:19.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:19 smithi167 ceph-mon[32751]: pgmap v17542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:19.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:19 smithi090 ceph-mon[29180]: pgmap v17542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31862.477040 secs 2024-04-27T03:26:20.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31862.477040 secs 2024-04-27T03:26:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31862.477040 secs 2024-04-27T03:26:21.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:21 smithi167 ceph-mon[32751]: pgmap v17543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31863.334832 secs 2024-04-27T03:26:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:21 smithi090 ceph-mon[29180]: pgmap v17543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31863.334832 secs 2024-04-27T03:26:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:21 smithi119 ceph-mon[32752]: pgmap v17543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31863.334832 secs 2024-04-27T03:26:23.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:23 smithi167 ceph-mon[32751]: pgmap v17544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:23.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:23 smithi090 ceph-mon[29180]: pgmap v17544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:24.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:23 smithi119 ceph-mon[32752]: pgmap v17544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:25 smithi167 ceph-mon[32751]: pgmap v17545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:25.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31867.477109 secs 2024-04-27T03:26:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:25 smithi090 ceph-mon[29180]: pgmap v17545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31867.477109 secs 2024-04-27T03:26:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:25 smithi119 ceph-mon[32752]: pgmap v17545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T03:26:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31867.477109 secs 2024-04-27T03:26:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31868.334943 secs 2024-04-27T03:26:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31868.334943 secs 2024-04-27T03:26:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31868.334943 secs 2024-04-27T03:26:27.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:27 smithi090 ceph-mon[29180]: pgmap v17546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:28.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:27 smithi119 ceph-mon[32752]: pgmap v17546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:28.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:27 smithi167 ceph-mon[32751]: pgmap v17546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:28.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:26:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:26:29.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:26:29.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:29 smithi090 ceph-mon[29180]: pgmap v17547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:29.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:26:29.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:26:29.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:26:30.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:29 smithi119 ceph-mon[32752]: pgmap v17547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:30.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:26:30.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:26:30.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:26:30.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:29 smithi167 ceph-mon[32751]: pgmap v17547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:30.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:26:30.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:26:30.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:26:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:30 smithi090 ceph-mon[29180]: pgmap v17548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:26:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:30 smithi090 ceph-mon[29180]: pgmap v17549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31872.477171 secs 2024-04-27T03:26:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:30 smithi119 ceph-mon[32752]: pgmap v17548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:26:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:30 smithi119 ceph-mon[32752]: pgmap v17549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31872.477171 secs 2024-04-27T03:26:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:30 smithi167 ceph-mon[32751]: pgmap v17548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:26:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:30 smithi167 ceph-mon[32751]: pgmap v17549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31872.477171 secs 2024-04-27T03:26:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31873.335060 secs 2024-04-27T03:26:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31873.335060 secs 2024-04-27T03:26:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31873.335060 secs 2024-04-27T03:26:32.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:32 smithi090 ceph-mon[29180]: pgmap v17550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:32 smithi119 ceph-mon[32752]: pgmap v17550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:32 smithi167 ceph-mon[32751]: pgmap v17550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:34.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:34 smithi090 ceph-mon[29180]: pgmap v17551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:26:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:34 smithi119 ceph-mon[32752]: pgmap v17551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:26:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:34 smithi167 ceph-mon[32751]: pgmap v17551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:26:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31877.477229 secs 2024-04-27T03:26:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31877.477229 secs 2024-04-27T03:26:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31877.477229 secs 2024-04-27T03:26:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:36 smithi090 ceph-mon[29180]: pgmap v17552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31878.335174 secs 2024-04-27T03:26:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:36 smithi119 ceph-mon[32752]: pgmap v17552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31878.335174 secs 2024-04-27T03:26:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:36 smithi167 ceph-mon[32751]: pgmap v17552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:26:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31878.335174 secs 2024-04-27T03:26:38.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:38 smithi090 ceph-mon[29180]: pgmap v17553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:26:39.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:38 smithi119 ceph-mon[32752]: pgmap v17553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:26:39.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:38 smithi167 ceph-mon[32751]: pgmap v17553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:26:39.881 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:26:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:26:39.611+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:26:39.882 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:26:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:26:39.611+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:26:39.882 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:26:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:26:39.611+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:26:39.882 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:26:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:26:39.611+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:26:39.882 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:26:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:26:39.611+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:26:40.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:40 smithi167 ceph-mon[32751]: pgmap v17554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:26:40.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31882.477312 secs 2024-04-27T03:26:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:40 smithi090 ceph-mon[29180]: pgmap v17554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:26:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31882.477312 secs 2024-04-27T03:26:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:40 smithi119 ceph-mon[32752]: pgmap v17554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:26:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31882.477312 secs 2024-04-27T03:26:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31883.335307 secs 2024-04-27T03:26:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31883.335307 secs 2024-04-27T03:26:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31883.335307 secs 2024-04-27T03:26:42.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:42 smithi090 ceph-mon[29180]: pgmap v17555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:42 smithi119 ceph-mon[32752]: pgmap v17555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:42 smithi167 ceph-mon[32751]: pgmap v17555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:44.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:44 smithi090 ceph-mon[29180]: pgmap v17556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:45.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:44 smithi119 ceph-mon[32752]: pgmap v17556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:45.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:44 smithi167 ceph-mon[32751]: pgmap v17556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31887.477370 secs 2024-04-27T03:26:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31887.477370 secs 2024-04-27T03:26:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31887.477370 secs 2024-04-27T03:26:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:46 smithi119 ceph-mon[32752]: pgmap v17557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:47.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31888.335464 secs 2024-04-27T03:26:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:46 smithi167 ceph-mon[32751]: pgmap v17557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31888.335464 secs 2024-04-27T03:26:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:46 smithi090 ceph-mon[29180]: pgmap v17557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:47.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31888.335464 secs 2024-04-27T03:26:49.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:48 smithi119 ceph-mon[32752]: pgmap v17558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:49.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:48 smithi167 ceph-mon[32751]: pgmap v17558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:49.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:48 smithi090 ceph-mon[29180]: pgmap v17558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:50.933 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:50 smithi090 ceph-mon[29180]: pgmap v17559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:50.933 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31892.477451 secs 2024-04-27T03:26:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:50 smithi119 ceph-mon[32752]: pgmap v17559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31892.477451 secs 2024-04-27T03:26:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:50 smithi167 ceph-mon[32751]: pgmap v17559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31892.477451 secs 2024-04-27T03:26:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31893.335579 secs 2024-04-27T03:26:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31893.335579 secs 2024-04-27T03:26:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31893.335579 secs 2024-04-27T03:26:53.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:52 smithi119 ceph-mon[32752]: pgmap v17560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:53.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:52 smithi167 ceph-mon[32751]: pgmap v17560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:52 smithi090 ceph-mon[29180]: pgmap v17560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:55.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:54 smithi119 ceph-mon[32752]: pgmap v17561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:55.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:54 smithi167 ceph-mon[32751]: pgmap v17561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:55.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:54 smithi090 ceph-mon[29180]: pgmap v17561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31897.477532 secs 2024-04-27T03:26:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31897.477532 secs 2024-04-27T03:26:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31897.477532 secs 2024-04-27T03:26:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:56 smithi119 ceph-mon[32752]: pgmap v17562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31898.335695 secs 2024-04-27T03:26:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:56 smithi167 ceph-mon[32751]: pgmap v17562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31898.335695 secs 2024-04-27T03:26:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:56 smithi090 ceph-mon[29180]: pgmap v17562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:26:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31898.335695 secs 2024-04-27T03:26:59.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:26:58 smithi119 ceph-mon[32752]: pgmap v17563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:59.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:26:58 smithi167 ceph-mon[32751]: pgmap v17563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:26:59.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:26:58 smithi090 ceph-mon[29180]: pgmap v17563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:00 smithi119 ceph-mon[32752]: pgmap v17564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31902.477618 secs 2024-04-27T03:27:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:00 smithi167 ceph-mon[32751]: pgmap v17564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31902.477618 secs 2024-04-27T03:27:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:00 smithi090 ceph-mon[29180]: pgmap v17564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31902.477618 secs 2024-04-27T03:27:01.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31903.335803 secs 2024-04-27T03:27:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31903.335803 secs 2024-04-27T03:27:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31903.335803 secs 2024-04-27T03:27:03.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:02 smithi119 ceph-mon[32752]: pgmap v17565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:03.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:02 smithi167 ceph-mon[32751]: pgmap v17565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:03.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:02 smithi090 ceph-mon[29180]: pgmap v17565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:05.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:04 smithi119 ceph-mon[32752]: pgmap v17566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:05.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:04 smithi167 ceph-mon[32751]: pgmap v17566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:05.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:04 smithi090 ceph-mon[29180]: pgmap v17566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31907.477699 secs 2024-04-27T03:27:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31907.477699 secs 2024-04-27T03:27:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31907.477699 secs 2024-04-27T03:27:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:06 smithi119 ceph-mon[32752]: pgmap v17567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31908.335903 secs 2024-04-27T03:27:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:06 smithi167 ceph-mon[32751]: pgmap v17567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31908.335903 secs 2024-04-27T03:27:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:06 smithi090 ceph-mon[29180]: pgmap v17567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31908.335903 secs 2024-04-27T03:27:09.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:08 smithi119 ceph-mon[32752]: pgmap v17568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:09.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:08 smithi167 ceph-mon[32751]: pgmap v17568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:09.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:08 smithi090 ceph-mon[29180]: pgmap v17568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:10 smithi119 ceph-mon[32752]: pgmap v17569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:10 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 31912.477814 secs 2024-04-27T03:27:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:10 smithi119 ceph-mon[32752]: slow request 11522.682850 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:27:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:10 smithi167 ceph-mon[32751]: pgmap v17569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:10 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 31912.477814 secs 2024-04-27T03:27:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:10 smithi167 ceph-mon[32751]: slow request 11522.682850 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:27:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:10 smithi090 ceph-mon[29180]: pgmap v17569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:10 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 31912.477814 secs 2024-04-27T03:27:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:10 smithi090 ceph-mon[29180]: slow request 11522.682850 seconds old, received at 2024-04-27T00:15:07.548368+0000: client_request(client.24733:267896 getattr AsXsFs #0x100000001fb 2024-04-27T00:15:07.547481+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:27:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31913.336036 secs 2024-04-27T03:27:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31913.336036 secs 2024-04-27T03:27:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31913.336036 secs 2024-04-27T03:27:13.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:12 smithi119 ceph-mon[32752]: pgmap v17570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:13.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:12 smithi167 ceph-mon[32751]: pgmap v17570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:12 smithi090 ceph-mon[29180]: pgmap v17570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:15.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:14 smithi119 ceph-mon[32752]: pgmap v17571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:15.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:14 smithi167 ceph-mon[32751]: pgmap v17571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:15.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:14 smithi090 ceph-mon[29180]: pgmap v17571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31917.477873 secs 2024-04-27T03:27:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31917.477873 secs 2024-04-27T03:27:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31917.477873 secs 2024-04-27T03:27:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:16 smithi090 ceph-mon[29180]: pgmap v17572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:27:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31918.336156 secs 2024-04-27T03:27:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:16 smithi119 ceph-mon[32752]: pgmap v17572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:27:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31918.336156 secs 2024-04-27T03:27:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:16 smithi167 ceph-mon[32751]: pgmap v17572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:27:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31918.336156 secs 2024-04-27T03:27:19.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:18 smithi119 ceph-mon[32752]: pgmap v17573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:19.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:18 smithi167 ceph-mon[32751]: pgmap v17573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:19.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:18 smithi090 ceph-mon[29180]: pgmap v17573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:20 smithi119 ceph-mon[32752]: pgmap v17574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31922.477933 secs 2024-04-27T03:27:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:20 smithi167 ceph-mon[32751]: pgmap v17574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31922.477933 secs 2024-04-27T03:27:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:20 smithi090 ceph-mon[29180]: pgmap v17574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31922.477933 secs 2024-04-27T03:27:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31923.336271 secs 2024-04-27T03:27:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31923.336271 secs 2024-04-27T03:27:22.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31923.336271 secs 2024-04-27T03:27:23.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:22 smithi119 ceph-mon[32752]: pgmap v17575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:23.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:22 smithi167 ceph-mon[32751]: pgmap v17575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:23.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:22 smithi090 ceph-mon[29180]: pgmap v17575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:25.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:24 smithi119 ceph-mon[32752]: pgmap v17576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:25.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:24 smithi167 ceph-mon[32751]: pgmap v17576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:25.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:24 smithi090 ceph-mon[29180]: pgmap v17576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31927.478013 secs 2024-04-27T03:27:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31927.478013 secs 2024-04-27T03:27:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31927.478013 secs 2024-04-27T03:27:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:26 smithi119 ceph-mon[32752]: pgmap v17577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:27:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31928.336396 secs 2024-04-27T03:27:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:26 smithi167 ceph-mon[32751]: pgmap v17577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:27:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31928.336396 secs 2024-04-27T03:27:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:26 smithi090 ceph-mon[29180]: pgmap v17577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:27:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31928.336396 secs 2024-04-27T03:27:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:28 smithi119 ceph-mon[32752]: pgmap v17578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:29.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:28 smithi167 ceph-mon[32751]: pgmap v17578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:29.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:28 smithi090 ceph-mon[29180]: pgmap v17578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:30.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:27:30.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:27:30.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:27:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:30 smithi119 ceph-mon[32752]: pgmap v17579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:27:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:27:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:27:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31932.478084 secs 2024-04-27T03:27:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:30 smithi167 ceph-mon[32751]: pgmap v17579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:27:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:27:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:27:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31932.478084 secs 2024-04-27T03:27:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:30 smithi090 ceph-mon[29180]: pgmap v17579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:27:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:27:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:27:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31932.478084 secs 2024-04-27T03:27:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:31 smithi119 ceph-mon[32752]: pgmap v17580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:27:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:31 smithi119 ceph-mon[32752]: pgmap v17581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31933.336483 secs 2024-04-27T03:27:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:31 smithi167 ceph-mon[32751]: pgmap v17580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:27:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:31 smithi167 ceph-mon[32751]: pgmap v17581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31933.336483 secs 2024-04-27T03:27:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:31 smithi090 ceph-mon[29180]: pgmap v17580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:27:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:31 smithi090 ceph-mon[29180]: pgmap v17581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31933.336483 secs 2024-04-27T03:27:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:33 smithi119 ceph-mon[32752]: pgmap v17582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:27:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:33 smithi167 ceph-mon[32751]: pgmap v17582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:27:34.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:33 smithi090 ceph-mon[29180]: pgmap v17582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:27:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:35 smithi119 ceph-mon[32752]: pgmap v17583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31937.478165 secs 2024-04-27T03:27:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:35 smithi167 ceph-mon[32751]: pgmap v17583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31937.478165 secs 2024-04-27T03:27:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:35 smithi090 ceph-mon[29180]: pgmap v17583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31937.478165 secs 2024-04-27T03:27:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31938.336553 secs 2024-04-27T03:27:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31938.336553 secs 2024-04-27T03:27:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31938.336553 secs 2024-04-27T03:27:38.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:37 smithi119 ceph-mon[32752]: pgmap v17584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:37 smithi167 ceph-mon[32751]: pgmap v17584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:37 smithi090 ceph-mon[29180]: pgmap v17584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:27:40.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:39 smithi119 ceph-mon[32752]: pgmap v17585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:27:40.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:39 smithi167 ceph-mon[32751]: pgmap v17585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:27:40.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:39 smithi090 ceph-mon[29180]: pgmap v17585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:27:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31942.478248 secs 2024-04-27T03:27:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31942.478248 secs 2024-04-27T03:27:41.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31942.478248 secs 2024-04-27T03:27:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:41 smithi119 ceph-mon[32752]: pgmap v17586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:27:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31943.336673 secs 2024-04-27T03:27:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:41 smithi167 ceph-mon[32751]: pgmap v17586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:27:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31943.336673 secs 2024-04-27T03:27:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:41 smithi090 ceph-mon[29180]: pgmap v17586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:27:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31943.336673 secs 2024-04-27T03:27:44.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:43 smithi119 ceph-mon[32752]: pgmap v17587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:44.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:43 smithi167 ceph-mon[32751]: pgmap v17587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:44.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:43 smithi090 ceph-mon[29180]: pgmap v17587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:45 smithi119 ceph-mon[32752]: pgmap v17588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31947.478334 secs 2024-04-27T03:27:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:45 smithi167 ceph-mon[32751]: pgmap v17588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31947.478334 secs 2024-04-27T03:27:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:45 smithi090 ceph-mon[29180]: pgmap v17588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31947.478334 secs 2024-04-27T03:27:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31948.336786 secs 2024-04-27T03:27:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31948.336786 secs 2024-04-27T03:27:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31948.336786 secs 2024-04-27T03:27:48.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:47 smithi090 ceph-mon[29180]: pgmap v17589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:48.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:47 smithi119 ceph-mon[32752]: pgmap v17589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:48.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:47 smithi167 ceph-mon[32751]: pgmap v17589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:50.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:49 smithi119 ceph-mon[32752]: pgmap v17590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:50.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:49 smithi167 ceph-mon[32751]: pgmap v17590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:50.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:49 smithi090 ceph-mon[29180]: pgmap v17590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31952.478418 secs 2024-04-27T03:27:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31952.478418 secs 2024-04-27T03:27:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31952.478418 secs 2024-04-27T03:27:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:51 smithi119 ceph-mon[32752]: pgmap v17591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31953.336852 secs 2024-04-27T03:27:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:51 smithi167 ceph-mon[32751]: pgmap v17591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31953.336852 secs 2024-04-27T03:27:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:51 smithi090 ceph-mon[29180]: pgmap v17591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31953.336852 secs 2024-04-27T03:27:54.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:53 smithi119 ceph-mon[32752]: pgmap v17592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:54.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:53 smithi167 ceph-mon[32751]: pgmap v17592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:54.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:53 smithi090 ceph-mon[29180]: pgmap v17592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:55 smithi119 ceph-mon[32752]: pgmap v17593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31957.478497 secs 2024-04-27T03:27:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:55 smithi167 ceph-mon[32751]: pgmap v17593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31957.478497 secs 2024-04-27T03:27:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:55 smithi090 ceph-mon[29180]: pgmap v17593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:27:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31957.478497 secs 2024-04-27T03:27:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31958.336969 secs 2024-04-27T03:27:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31958.336969 secs 2024-04-27T03:27:57.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31958.336969 secs 2024-04-27T03:27:58.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:57 smithi167 ceph-mon[32751]: pgmap v17594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:57 smithi090 ceph-mon[29180]: pgmap v17594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:58.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:57 smithi119 ceph-mon[32752]: pgmap v17594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:59.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:27:58 smithi090 ceph-mon[29180]: pgmap v17595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:59.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:27:58 smithi119 ceph-mon[32752]: pgmap v17595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:27:59.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:27:58 smithi167 ceph-mon[32751]: pgmap v17595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:01.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:01 smithi119 ceph-mon[32752]: pgmap v17596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:01.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:01 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31962.478582 secs 2024-04-27T03:28:01.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:01 smithi167 ceph-mon[32751]: pgmap v17596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:01.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:01 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31962.478582 secs 2024-04-27T03:28:01.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:01 smithi090 ceph-mon[29180]: pgmap v17596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:01.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:01 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31962.478582 secs 2024-04-27T03:28:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:02 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31963.337098 secs 2024-04-27T03:28:02.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:02 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31963.337098 secs 2024-04-27T03:28:02.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:02 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31963.337098 secs 2024-04-27T03:28:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:03 smithi167 ceph-mon[32751]: pgmap v17597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:03 smithi090 ceph-mon[29180]: pgmap v17597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:03.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:03 smithi119 ceph-mon[32752]: pgmap v17597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:05.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:05 smithi167 ceph-mon[32751]: pgmap v17598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:05.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:05 smithi090 ceph-mon[29180]: pgmap v17598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:05.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:05 smithi119 ceph-mon[32752]: pgmap v17598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:06.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:06 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31967.478658 secs 2024-04-27T03:28:06.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:06 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31967.478658 secs 2024-04-27T03:28:06.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:06 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31967.478658 secs 2024-04-27T03:28:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:07 smithi090 ceph-mon[29180]: pgmap v17599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:07 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31968.337233 secs 2024-04-27T03:28:07.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:07 smithi119 ceph-mon[32752]: pgmap v17599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:07.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:07 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31968.337233 secs 2024-04-27T03:28:07.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:07 smithi167 ceph-mon[32751]: pgmap v17599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:07.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:07 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31968.337233 secs 2024-04-27T03:28:09.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:09 smithi090 ceph-mon[29180]: pgmap v17600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:09.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:09 smithi119 ceph-mon[32752]: pgmap v17600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:09.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:09 smithi167 ceph-mon[32751]: pgmap v17600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:11.375 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:11 smithi167 ceph-mon[32751]: pgmap v17601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:11.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:11 smithi167 ceph-mon[32751]: 43 slow requests, 2 included below; oldest blocked for > 31972.478738 secs 2024-04-27T03:28:11.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:11 smithi167 ceph-mon[32751]: slow request 2881.406156 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:28:11.376 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:11 smithi167 ceph-mon[32751]: slow request 720.811147 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:28:11.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:11 smithi090 ceph-mon[29180]: pgmap v17601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:11.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:11 smithi090 ceph-mon[29180]: 43 slow requests, 2 included below; oldest blocked for > 31972.478738 secs 2024-04-27T03:28:11.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:11 smithi090 ceph-mon[29180]: slow request 2881.406156 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:28:11.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:11 smithi090 ceph-mon[29180]: slow request 720.811147 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:28:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:11 smithi119 ceph-mon[32752]: pgmap v17601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:11 smithi119 ceph-mon[32752]: 43 slow requests, 2 included below; oldest blocked for > 31972.478738 secs 2024-04-27T03:28:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:11 smithi119 ceph-mon[32752]: slow request 2881.406156 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:28:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:11 smithi119 ceph-mon[32752]: slow request 720.811147 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:28:12.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31973.337362 secs 2024-04-27T03:28:12.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31973.337362 secs 2024-04-27T03:28:12.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31973.337362 secs 2024-04-27T03:28:13.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:13 smithi090 ceph-mon[29180]: pgmap v17602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:13.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:13 smithi119 ceph-mon[32752]: pgmap v17602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:13.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:13 smithi167 ceph-mon[32751]: pgmap v17602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:15.361 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:15 smithi119 ceph-mon[32752]: pgmap v17603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:15.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:15 smithi090 ceph-mon[29180]: pgmap v17603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:15.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:15 smithi167 ceph-mon[32751]: pgmap v17603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:16.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:16 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31977.478821 secs 2024-04-27T03:28:16.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:16 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31977.478821 secs 2024-04-27T03:28:16.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:16 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31977.478821 secs 2024-04-27T03:28:17.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:17 smithi090 ceph-mon[29180]: pgmap v17604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:17.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31978.337473 secs 2024-04-27T03:28:17.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:17 smithi119 ceph-mon[32752]: pgmap v17604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:17.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31978.337473 secs 2024-04-27T03:28:17.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:17 smithi167 ceph-mon[32751]: pgmap v17604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:17.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31978.337473 secs 2024-04-27T03:28:19.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:19 smithi090 ceph-mon[29180]: pgmap v17605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:19.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:19 smithi119 ceph-mon[32752]: pgmap v17605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:19.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:19 smithi167 ceph-mon[32751]: pgmap v17605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:21.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:21 smithi090 ceph-mon[29180]: pgmap v17606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:21.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:21 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31982.478883 secs 2024-04-27T03:28:21.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:21 smithi119 ceph-mon[32752]: pgmap v17606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:21.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:21 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31982.478883 secs 2024-04-27T03:28:21.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:21 smithi167 ceph-mon[32751]: pgmap v17606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:21.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:21 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31982.478883 secs 2024-04-27T03:28:22.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:22 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31983.337568 secs 2024-04-27T03:28:22.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:22 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31983.337568 secs 2024-04-27T03:28:22.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:22 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31983.337568 secs 2024-04-27T03:28:23.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:23 smithi090 ceph-mon[29180]: pgmap v17607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:23.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:23 smithi119 ceph-mon[32752]: pgmap v17607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:23.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:23 smithi167 ceph-mon[32751]: pgmap v17607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:25.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:25 smithi090 ceph-mon[29180]: pgmap v17608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:25.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:25 smithi119 ceph-mon[32752]: pgmap v17608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:25.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:25 smithi167 ceph-mon[32751]: pgmap v17608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:26.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:26 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31987.478941 secs 2024-04-27T03:28:26.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31988.337651 secs 2024-04-27T03:28:26.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:26 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31987.478941 secs 2024-04-27T03:28:26.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31988.337651 secs 2024-04-27T03:28:26.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:26 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31987.478941 secs 2024-04-27T03:28:26.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31988.337651 secs 2024-04-27T03:28:27.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:27 smithi119 ceph-mon[32752]: pgmap v17609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:27.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:27 smithi167 ceph-mon[32751]: pgmap v17609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:27.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:27 smithi090 ceph-mon[29180]: pgmap v17609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:29.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:29 smithi119 ceph-mon[32752]: pgmap v17610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:29.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:29 smithi167 ceph-mon[32751]: pgmap v17610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:29.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:29 smithi090 ceph-mon[29180]: pgmap v17610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:30.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:28:30.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:28:30.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:28:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: pgmap v17611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31992.479015 secs 2024-04-27T03:28:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:28:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:28:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: pgmap v17612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:28:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: pgmap v17613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:28:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31993.337746 secs 2024-04-27T03:28:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: pgmap v17611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31992.479015 secs 2024-04-27T03:28:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:28:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:28:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: pgmap v17612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:28:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: pgmap v17613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:28:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31993.337746 secs 2024-04-27T03:28:31.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: pgmap v17611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:31.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31992.479015 secs 2024-04-27T03:28:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:28:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:28:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: pgmap v17612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:28:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: pgmap v17613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:28:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31993.337746 secs 2024-04-27T03:28:33.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:32 smithi090 ceph-mon[29180]: pgmap v17614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:28:33.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:32 smithi119 ceph-mon[32752]: pgmap v17614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:28:33.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:32 smithi167 ceph-mon[32751]: pgmap v17614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:28:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:34 smithi090 ceph-mon[29180]: pgmap v17615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:35.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:34 smithi119 ceph-mon[32752]: pgmap v17615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:35.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:34 smithi167 ceph-mon[32751]: pgmap v17615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 31997.479073 secs 2024-04-27T03:28:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 31997.479073 secs 2024-04-27T03:28:36.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 31997.479073 secs 2024-04-27T03:28:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 31998.337844 secs 2024-04-27T03:28:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:36 smithi090 ceph-mon[29180]: pgmap v17616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 31998.337844 secs 2024-04-27T03:28:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:36 smithi119 ceph-mon[32752]: pgmap v17616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:37.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 31998.337844 secs 2024-04-27T03:28:37.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:36 smithi167 ceph-mon[32751]: pgmap v17616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:39.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:38 smithi090 ceph-mon[29180]: pgmap v17617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:38 smithi119 ceph-mon[32752]: pgmap v17617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:38 smithi167 ceph-mon[32751]: pgmap v17617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:28:40.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32002.479134 secs 2024-04-27T03:28:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32002.479134 secs 2024-04-27T03:28:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32002.479134 secs 2024-04-27T03:28:41.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:41 smithi167 ceph-mon[32751]: pgmap v17618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:28:41.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32003.337965 secs 2024-04-27T03:28:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:41 smithi119 ceph-mon[32752]: pgmap v17618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:28:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32003.337965 secs 2024-04-27T03:28:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:41 smithi090 ceph-mon[29180]: pgmap v17618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:28:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32003.337965 secs 2024-04-27T03:28:43.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:42 smithi090 ceph-mon[29180]: pgmap v17619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:43.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:42 smithi119 ceph-mon[32752]: pgmap v17619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:43.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:42 smithi167 ceph-mon[32751]: pgmap v17619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:45.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:44 smithi090 ceph-mon[29180]: pgmap v17620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:45.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:44 smithi119 ceph-mon[32752]: pgmap v17620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:45.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:44 smithi167 ceph-mon[32751]: pgmap v17620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:46.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32007.479192 secs 2024-04-27T03:28:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32007.479192 secs 2024-04-27T03:28:46.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32007.479192 secs 2024-04-27T03:28:47.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32008.338083 secs 2024-04-27T03:28:47.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:46 smithi119 ceph-mon[32752]: pgmap v17621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:47.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32008.338083 secs 2024-04-27T03:28:47.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:46 smithi167 ceph-mon[32751]: pgmap v17621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:47.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32008.338083 secs 2024-04-27T03:28:47.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:46 smithi090 ceph-mon[29180]: pgmap v17621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:49.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:48 smithi090 ceph-mon[29180]: pgmap v17622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:49.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:48 smithi119 ceph-mon[32752]: pgmap v17622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:49.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:48 smithi167 ceph-mon[32751]: pgmap v17622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32012.479254 secs 2024-04-27T03:28:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32012.479254 secs 2024-04-27T03:28:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32012.479254 secs 2024-04-27T03:28:51.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:51 smithi090 ceph-mon[29180]: pgmap v17623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:51.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32013.338198 secs 2024-04-27T03:28:51.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:51 smithi119 ceph-mon[32752]: pgmap v17623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:51.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32013.338198 secs 2024-04-27T03:28:51.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:51 smithi167 ceph-mon[32751]: pgmap v17623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:51.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32013.338198 secs 2024-04-27T03:28:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:52 smithi090 ceph-mon[29180]: pgmap v17624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:52 smithi119 ceph-mon[32752]: pgmap v17624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:52 smithi167 ceph-mon[32751]: pgmap v17624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:55.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:54 smithi090 ceph-mon[29180]: pgmap v17625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:55.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:54 smithi119 ceph-mon[32752]: pgmap v17625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:55.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:54 smithi167 ceph-mon[32751]: pgmap v17625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32017.479311 secs 2024-04-27T03:28:56.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32017.479311 secs 2024-04-27T03:28:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32017.479311 secs 2024-04-27T03:28:57.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32018.338283 secs 2024-04-27T03:28:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:56 smithi119 ceph-mon[32752]: pgmap v17626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32018.338283 secs 2024-04-27T03:28:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:56 smithi167 ceph-mon[32751]: pgmap v17626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:57.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32018.338283 secs 2024-04-27T03:28:57.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:56 smithi090 ceph-mon[29180]: pgmap v17626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:28:59.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:28:58 smithi090 ceph-mon[29180]: pgmap v17627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:59.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:28:58 smithi119 ceph-mon[32752]: pgmap v17627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:28:59.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:28:58 smithi167 ceph-mon[32751]: pgmap v17627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:00.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32022.479396 secs 2024-04-27T03:29:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32022.479396 secs 2024-04-27T03:29:00.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32022.479396 secs 2024-04-27T03:29:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:01 smithi119 ceph-mon[32752]: pgmap v17628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32023.338401 secs 2024-04-27T03:29:01.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:01 smithi167 ceph-mon[32751]: pgmap v17628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:01.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32023.338401 secs 2024-04-27T03:29:01.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:01 smithi090 ceph-mon[29180]: pgmap v17628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:01.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32023.338401 secs 2024-04-27T03:29:03.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:02 smithi090 ceph-mon[29180]: pgmap v17629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:03.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:02 smithi119 ceph-mon[32752]: pgmap v17629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:02 smithi167 ceph-mon[32751]: pgmap v17629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:05.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:04 smithi090 ceph-mon[29180]: pgmap v17630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:05.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:04 smithi119 ceph-mon[32752]: pgmap v17630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:05.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:04 smithi167 ceph-mon[32751]: pgmap v17630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32027.479477 secs 2024-04-27T03:29:06.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32027.479477 secs 2024-04-27T03:29:06.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32027.479477 secs 2024-04-27T03:29:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32028.338514 secs 2024-04-27T03:29:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:06 smithi119 ceph-mon[32752]: pgmap v17631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:07.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32028.338514 secs 2024-04-27T03:29:07.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:06 smithi167 ceph-mon[32751]: pgmap v17631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32028.338514 secs 2024-04-27T03:29:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:06 smithi090 ceph-mon[29180]: pgmap v17631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:09.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:08 smithi090 ceph-mon[29180]: pgmap v17632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:09.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:08 smithi119 ceph-mon[32752]: pgmap v17632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:09.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:08 smithi167 ceph-mon[32751]: pgmap v17632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:10.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32032.479563 secs 2024-04-27T03:29:10.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32032.479563 secs 2024-04-27T03:29:10.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32032.479563 secs 2024-04-27T03:29:11.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:11 smithi167 ceph-mon[32751]: pgmap v17633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:11.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32033.338589 secs 2024-04-27T03:29:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:11 smithi119 ceph-mon[32752]: pgmap v17633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32033.338589 secs 2024-04-27T03:29:11.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:11 smithi090 ceph-mon[29180]: pgmap v17633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:11.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32033.338589 secs 2024-04-27T03:29:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:12 smithi090 ceph-mon[29180]: pgmap v17634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:13.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:12 smithi119 ceph-mon[32752]: pgmap v17634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:13.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:12 smithi167 ceph-mon[32751]: pgmap v17634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:15.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:14 smithi090 ceph-mon[29180]: pgmap v17635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:15.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:14 smithi119 ceph-mon[32752]: pgmap v17635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:15.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:14 smithi167 ceph-mon[32751]: pgmap v17635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32037.479643 secs 2024-04-27T03:29:16.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32037.479643 secs 2024-04-27T03:29:16.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32037.479643 secs 2024-04-27T03:29:17.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32038.338708 secs 2024-04-27T03:29:17.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:16 smithi119 ceph-mon[32752]: pgmap v17636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32038.338708 secs 2024-04-27T03:29:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:16 smithi167 ceph-mon[32751]: pgmap v17636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:17.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32038.338708 secs 2024-04-27T03:29:17.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:16 smithi090 ceph-mon[29180]: pgmap v17636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:19.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:18 smithi090 ceph-mon[29180]: pgmap v17637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:19.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:18 smithi119 ceph-mon[32752]: pgmap v17637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:19.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:18 smithi167 ceph-mon[32751]: pgmap v17637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32042.479726 secs 2024-04-27T03:29:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32042.479726 secs 2024-04-27T03:29:20.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32042.479726 secs 2024-04-27T03:29:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:21 smithi119 ceph-mon[32752]: pgmap v17638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32043.338825 secs 2024-04-27T03:29:21.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:21 smithi167 ceph-mon[32751]: pgmap v17638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:21.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32043.338825 secs 2024-04-27T03:29:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:21 smithi090 ceph-mon[29180]: pgmap v17638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32043.338825 secs 2024-04-27T03:29:23.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:22 smithi090 ceph-mon[29180]: pgmap v17639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:23.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:22 smithi119 ceph-mon[32752]: pgmap v17639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:23.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:22 smithi167 ceph-mon[32751]: pgmap v17639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:25.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:24 smithi090 ceph-mon[29180]: pgmap v17640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:25.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:24 smithi119 ceph-mon[32752]: pgmap v17640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:25.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:24 smithi167 ceph-mon[32751]: pgmap v17640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32047.479809 secs 2024-04-27T03:29:26.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32047.479809 secs 2024-04-27T03:29:26.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32047.479809 secs 2024-04-27T03:29:27.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32048.338924 secs 2024-04-27T03:29:27.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:26 smithi119 ceph-mon[32752]: pgmap v17641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:27.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32048.338924 secs 2024-04-27T03:29:27.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:26 smithi167 ceph-mon[32751]: pgmap v17641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:27.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32048.338924 secs 2024-04-27T03:29:27.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:26 smithi090 ceph-mon[29180]: pgmap v17641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:29.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:28 smithi090 ceph-mon[29180]: pgmap v17642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:29.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:28 smithi119 ceph-mon[32752]: pgmap v17642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:29.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:28 smithi167 ceph-mon[32751]: pgmap v17642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32052.479872 secs 2024-04-27T03:29:30.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32052.479872 secs 2024-04-27T03:29:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32052.479872 secs 2024-04-27T03:29:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:31 smithi119 ceph-mon[32752]: pgmap v17643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:29:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32053.338988 secs 2024-04-27T03:29:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:31 smithi167 ceph-mon[32751]: pgmap v17643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:29:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32053.338988 secs 2024-04-27T03:29:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:31 smithi090 ceph-mon[29180]: pgmap v17643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:29:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32053.338988 secs 2024-04-27T03:29:32.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:29:32.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:29:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:32 smithi119 ceph-mon[32752]: pgmap v17644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:29:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:32 smithi119 ceph-mon[32752]: pgmap v17645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:29:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:29:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:29:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:32 smithi167 ceph-mon[32751]: pgmap v17644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:29:32.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:32 smithi167 ceph-mon[32751]: pgmap v17645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:32.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:29:32.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:29:32.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:29:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:32 smithi090 ceph-mon[29180]: pgmap v17644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:29:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:32 smithi090 ceph-mon[29180]: pgmap v17645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:29:34.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:34 smithi090 ceph-mon[29180]: pgmap v17646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:34 smithi119 ceph-mon[32752]: pgmap v17646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:34 smithi167 ceph-mon[32751]: pgmap v17646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32057.479932 secs 2024-04-27T03:29:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32057.479932 secs 2024-04-27T03:29:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32057.479932 secs 2024-04-27T03:29:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:36 smithi090 ceph-mon[29180]: pgmap v17647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32058.339083 secs 2024-04-27T03:29:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:36 smithi119 ceph-mon[32752]: pgmap v17647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32058.339083 secs 2024-04-27T03:29:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:36 smithi167 ceph-mon[32751]: pgmap v17647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32058.339083 secs 2024-04-27T03:29:38.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:38 smithi090 ceph-mon[29180]: pgmap v17648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:29:39.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:38 smithi119 ceph-mon[32752]: pgmap v17648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:29:39.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:38 smithi167 ceph-mon[32751]: pgmap v17648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:29:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:40 smithi090 ceph-mon[29180]: pgmap v17649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:40 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32062.479991 secs 2024-04-27T03:29:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:40 smithi090 ceph-mon[29180]: slow request 1440.989513 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:29:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:40 smithi119 ceph-mon[32752]: pgmap v17649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:40 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32062.479991 secs 2024-04-27T03:29:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:40 smithi119 ceph-mon[32752]: slow request 1440.989513 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:29:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:40 smithi167 ceph-mon[32751]: pgmap v17649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:29:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:40 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32062.479991 secs 2024-04-27T03:29:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:40 smithi167 ceph-mon[32751]: slow request 1440.989513 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:29:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32063.339196 secs 2024-04-27T03:29:41.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32063.339196 secs 2024-04-27T03:29:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32063.339196 secs 2024-04-27T03:29:42.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:42 smithi090 ceph-mon[29180]: pgmap v17650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:29:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:42 smithi119 ceph-mon[32752]: pgmap v17650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:29:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:42 smithi167 ceph-mon[32751]: pgmap v17650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:29:45.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:44 smithi119 ceph-mon[32752]: pgmap v17651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:45.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:44 smithi167 ceph-mon[32751]: pgmap v17651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:45.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:44 smithi090 ceph-mon[29180]: pgmap v17651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:45.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32067.480049 secs 2024-04-27T03:29:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32067.480049 secs 2024-04-27T03:29:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32067.480049 secs 2024-04-27T03:29:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:46 smithi119 ceph-mon[32752]: pgmap v17652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32068.339292 secs 2024-04-27T03:29:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:46 smithi167 ceph-mon[32751]: pgmap v17652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32068.339292 secs 2024-04-27T03:29:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:46 smithi090 ceph-mon[29180]: pgmap v17652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:47.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32068.339292 secs 2024-04-27T03:29:49.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:48 smithi119 ceph-mon[32752]: pgmap v17653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:49.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:48 smithi167 ceph-mon[32751]: pgmap v17653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:49.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:48 smithi090 ceph-mon[29180]: pgmap v17653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:50 smithi119 ceph-mon[32752]: pgmap v17654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32072.480133 secs 2024-04-27T03:29:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:50 smithi167 ceph-mon[32751]: pgmap v17654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32072.480133 secs 2024-04-27T03:29:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:50 smithi090 ceph-mon[29180]: pgmap v17654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32072.480133 secs 2024-04-27T03:29:52.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32073.339410 secs 2024-04-27T03:29:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32073.339410 secs 2024-04-27T03:29:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32073.339410 secs 2024-04-27T03:29:52.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:52 smithi090 ceph-mon[29180]: pgmap v17655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:53.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:52 smithi119 ceph-mon[32752]: pgmap v17655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:53.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:52 smithi167 ceph-mon[32751]: pgmap v17655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:55.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:54 smithi119 ceph-mon[32752]: pgmap v17656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:55.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:54 smithi167 ceph-mon[32751]: pgmap v17656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:55.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:54 smithi090 ceph-mon[29180]: pgmap v17656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:29:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32077.480191 secs 2024-04-27T03:29:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32077.480191 secs 2024-04-27T03:29:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32077.480191 secs 2024-04-27T03:29:57.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:56 smithi119 ceph-mon[32752]: pgmap v17657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:57.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32078.339563 secs 2024-04-27T03:29:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:56 smithi167 ceph-mon[32751]: pgmap v17657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32078.339563 secs 2024-04-27T03:29:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:56 smithi090 ceph-mon[29180]: pgmap v17657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32078.339563 secs 2024-04-27T03:29:59.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:29:58 smithi119 ceph-mon[32752]: pgmap v17658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:59.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:29:58 smithi167 ceph-mon[32751]: pgmap v17658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:29:59.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:29:58 smithi090 ceph-mon[29180]: pgmap v17658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:00 smithi119 ceph-mon[32752]: pgmap v17659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:30:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32082.480275 secs 2024-04-27T03:30:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:00 smithi167 ceph-mon[32751]: pgmap v17659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:30:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32082.480275 secs 2024-04-27T03:30:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:00 smithi090 ceph-mon[29180]: pgmap v17659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:30:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32082.480275 secs 2024-04-27T03:30:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32083.339627 secs 2024-04-27T03:30:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32083.339627 secs 2024-04-27T03:30:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32083.339627 secs 2024-04-27T03:30:03.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:02 smithi119 ceph-mon[32752]: pgmap v17660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:03.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:02 smithi167 ceph-mon[32751]: pgmap v17660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:03.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:02 smithi090 ceph-mon[29180]: pgmap v17660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:05.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:04 smithi119 ceph-mon[32752]: pgmap v17661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:05.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:04 smithi167 ceph-mon[32751]: pgmap v17661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:05.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:04 smithi090 ceph-mon[29180]: pgmap v17661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32087.480360 secs 2024-04-27T03:30:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32087.480360 secs 2024-04-27T03:30:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32087.480360 secs 2024-04-27T03:30:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:06 smithi119 ceph-mon[32752]: pgmap v17662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32088.339712 secs 2024-04-27T03:30:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:06 smithi167 ceph-mon[32751]: pgmap v17662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32088.339712 secs 2024-04-27T03:30:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:06 smithi090 ceph-mon[29180]: pgmap v17662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32088.339712 secs 2024-04-27T03:30:09.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:08 smithi119 ceph-mon[32752]: pgmap v17663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:09.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:08 smithi167 ceph-mon[32751]: pgmap v17663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:09.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:08 smithi090 ceph-mon[29180]: pgmap v17663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:10 smithi119 ceph-mon[32752]: pgmap v17664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32092.480445 secs 2024-04-27T03:30:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:10 smithi167 ceph-mon[32751]: pgmap v17664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32092.480445 secs 2024-04-27T03:30:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:10 smithi090 ceph-mon[29180]: pgmap v17664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32092.480445 secs 2024-04-27T03:30:11.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32093.339826 secs 2024-04-27T03:30:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32093.339826 secs 2024-04-27T03:30:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32093.339826 secs 2024-04-27T03:30:13.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:12 smithi119 ceph-mon[32752]: pgmap v17665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:13.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:12 smithi167 ceph-mon[32751]: pgmap v17665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:12 smithi090 ceph-mon[29180]: pgmap v17665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:15.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:14 smithi119 ceph-mon[32752]: pgmap v17666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:15.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:14 smithi167 ceph-mon[32751]: pgmap v17666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:15.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:14 smithi090 ceph-mon[29180]: pgmap v17666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:15.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32097.480504 secs 2024-04-27T03:30:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32097.480504 secs 2024-04-27T03:30:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32097.480504 secs 2024-04-27T03:30:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:16 smithi119 ceph-mon[32752]: pgmap v17667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:17.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32098.339943 secs 2024-04-27T03:30:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:16 smithi167 ceph-mon[32751]: pgmap v17667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32098.339943 secs 2024-04-27T03:30:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:16 smithi090 ceph-mon[29180]: pgmap v17667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32098.339943 secs 2024-04-27T03:30:19.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:18 smithi119 ceph-mon[32752]: pgmap v17668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:19.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:18 smithi167 ceph-mon[32751]: pgmap v17668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:19.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:18 smithi090 ceph-mon[29180]: pgmap v17668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:20 smithi119 ceph-mon[32752]: pgmap v17669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32102.480588 secs 2024-04-27T03:30:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:20 smithi167 ceph-mon[32751]: pgmap v17669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32102.480588 secs 2024-04-27T03:30:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:20 smithi090 ceph-mon[29180]: pgmap v17669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32102.480588 secs 2024-04-27T03:30:22.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32103.340065 secs 2024-04-27T03:30:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32103.340065 secs 2024-04-27T03:30:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32103.340065 secs 2024-04-27T03:30:23.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:22 smithi119 ceph-mon[32752]: pgmap v17670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:23.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:22 smithi167 ceph-mon[32751]: pgmap v17670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:23.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:22 smithi090 ceph-mon[29180]: pgmap v17670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:25.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:24 smithi119 ceph-mon[32752]: pgmap v17671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:25.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:24 smithi167 ceph-mon[32751]: pgmap v17671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:25.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:24 smithi090 ceph-mon[29180]: pgmap v17671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32107.480645 secs 2024-04-27T03:30:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32107.480645 secs 2024-04-27T03:30:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32107.480645 secs 2024-04-27T03:30:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:26 smithi119 ceph-mon[32752]: pgmap v17672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32108.340206 secs 2024-04-27T03:30:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:26 smithi167 ceph-mon[32751]: pgmap v17672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32108.340206 secs 2024-04-27T03:30:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:26 smithi090 ceph-mon[29180]: pgmap v17672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32108.340206 secs 2024-04-27T03:30:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:28 smithi119 ceph-mon[32752]: pgmap v17673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:29.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:28 smithi167 ceph-mon[32751]: pgmap v17673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:29.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:28 smithi090 ceph-mon[29180]: pgmap v17673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:31.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:30 smithi119 ceph-mon[32752]: pgmap v17674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:31.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32112.480728 secs 2024-04-27T03:30:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:30 smithi167 ceph-mon[32751]: pgmap v17674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32112.480728 secs 2024-04-27T03:30:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:30 smithi090 ceph-mon[29180]: pgmap v17674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32112.480728 secs 2024-04-27T03:30:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32113.340337 secs 2024-04-27T03:30:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:30:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32113.340337 secs 2024-04-27T03:30:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:30:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32113.340337 secs 2024-04-27T03:30:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:30:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:32 smithi119 ceph-mon[32752]: pgmap v17675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:30:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:30:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:30:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:32 smithi167 ceph-mon[32751]: pgmap v17675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:30:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:30:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:30:33.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:32 smithi090 ceph-mon[29180]: pgmap v17675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:33.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:30:33.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:30:33.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:30:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:33 smithi119 ceph-mon[32752]: pgmap v17676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:30:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:33 smithi119 ceph-mon[32752]: pgmap v17677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:30:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:33 smithi167 ceph-mon[32751]: pgmap v17676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:30:34.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:33 smithi167 ceph-mon[32751]: pgmap v17677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:30:34.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:33 smithi090 ceph-mon[29180]: pgmap v17676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:30:34.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:33 smithi090 ceph-mon[29180]: pgmap v17677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:30:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:35 smithi119 ceph-mon[32752]: pgmap v17678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32117.480813 secs 2024-04-27T03:30:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:35 smithi167 ceph-mon[32751]: pgmap v17678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32117.480813 secs 2024-04-27T03:30:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:35 smithi090 ceph-mon[29180]: pgmap v17678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32117.480813 secs 2024-04-27T03:30:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32118.340454 secs 2024-04-27T03:30:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32118.340454 secs 2024-04-27T03:30:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32118.340454 secs 2024-04-27T03:30:38.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:37 smithi119 ceph-mon[32752]: pgmap v17679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:37 smithi167 ceph-mon[32751]: pgmap v17679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:37 smithi090 ceph-mon[29180]: pgmap v17679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:40.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:39 smithi119 ceph-mon[32752]: pgmap v17680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:40.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:39 smithi167 ceph-mon[32751]: pgmap v17680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:40.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:39 smithi090 ceph-mon[29180]: pgmap v17680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32122.480875 secs 2024-04-27T03:30:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32122.480875 secs 2024-04-27T03:30:41.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32122.480875 secs 2024-04-27T03:30:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:41 smithi119 ceph-mon[32752]: pgmap v17681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32123.340592 secs 2024-04-27T03:30:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:41 smithi167 ceph-mon[32751]: pgmap v17681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32123.340592 secs 2024-04-27T03:30:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:41 smithi090 ceph-mon[29180]: pgmap v17681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:30:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32123.340592 secs 2024-04-27T03:30:44.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:43 smithi119 ceph-mon[32752]: pgmap v17682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:30:44.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:43 smithi167 ceph-mon[32751]: pgmap v17682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:30:44.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:43 smithi090 ceph-mon[29180]: pgmap v17682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:30:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:45 smithi119 ceph-mon[32752]: pgmap v17683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32127.480934 secs 2024-04-27T03:30:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:45 smithi167 ceph-mon[32751]: pgmap v17683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32127.480934 secs 2024-04-27T03:30:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:45 smithi090 ceph-mon[29180]: pgmap v17683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32127.480934 secs 2024-04-27T03:30:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32128.340773 secs 2024-04-27T03:30:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32128.340773 secs 2024-04-27T03:30:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32128.340773 secs 2024-04-27T03:30:48.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:47 smithi119 ceph-mon[32752]: pgmap v17684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:48.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:47 smithi167 ceph-mon[32751]: pgmap v17684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:48.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:47 smithi090 ceph-mon[29180]: pgmap v17684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:50.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:49 smithi119 ceph-mon[32752]: pgmap v17685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:50.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:49 smithi167 ceph-mon[32751]: pgmap v17685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:50.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:49 smithi090 ceph-mon[29180]: pgmap v17685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32132.480995 secs 2024-04-27T03:30:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32132.480995 secs 2024-04-27T03:30:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32132.480995 secs 2024-04-27T03:30:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:51 smithi119 ceph-mon[32752]: pgmap v17686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32133.340886 secs 2024-04-27T03:30:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:51 smithi167 ceph-mon[32751]: pgmap v17686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32133.340886 secs 2024-04-27T03:30:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:51 smithi090 ceph-mon[29180]: pgmap v17686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32133.340886 secs 2024-04-27T03:30:54.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:53 smithi119 ceph-mon[32752]: pgmap v17687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:54.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:53 smithi167 ceph-mon[32751]: pgmap v17687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:54.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:53 smithi090 ceph-mon[29180]: pgmap v17687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:55 smithi119 ceph-mon[32752]: pgmap v17688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32137.481076 secs 2024-04-27T03:30:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:55 smithi167 ceph-mon[32751]: pgmap v17688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32137.481076 secs 2024-04-27T03:30:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:55 smithi090 ceph-mon[29180]: pgmap v17688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:30:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32137.481076 secs 2024-04-27T03:30:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32138.340973 secs 2024-04-27T03:30:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32138.340973 secs 2024-04-27T03:30:57.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32138.340973 secs 2024-04-27T03:30:58.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:57 smithi167 ceph-mon[32751]: pgmap v17689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:57 smithi090 ceph-mon[29180]: pgmap v17689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:30:58.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:57 smithi119 ceph-mon[32752]: pgmap v17689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:00.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:30:59 smithi167 ceph-mon[32751]: pgmap v17690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:00.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:30:59 smithi090 ceph-mon[29180]: pgmap v17690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:00.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:30:59 smithi119 ceph-mon[32752]: pgmap v17690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32142.481138 secs 2024-04-27T03:31:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32142.481138 secs 2024-04-27T03:31:01.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32142.481138 secs 2024-04-27T03:31:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:01 smithi090 ceph-mon[29180]: pgmap v17691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32143.341126 secs 2024-04-27T03:31:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:01 smithi119 ceph-mon[32752]: pgmap v17691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32143.341126 secs 2024-04-27T03:31:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:01 smithi167 ceph-mon[32751]: pgmap v17691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32143.341126 secs 2024-04-27T03:31:04.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:03 smithi090 ceph-mon[29180]: pgmap v17692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:04.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:03 smithi119 ceph-mon[32752]: pgmap v17692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:04.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:03 smithi167 ceph-mon[32751]: pgmap v17692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:05 smithi090 ceph-mon[29180]: pgmap v17693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32147.481219 secs 2024-04-27T03:31:06.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:05 smithi119 ceph-mon[32752]: pgmap v17693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:06.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32147.481219 secs 2024-04-27T03:31:06.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:05 smithi167 ceph-mon[32751]: pgmap v17693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:06.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32147.481219 secs 2024-04-27T03:31:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32148.341255 secs 2024-04-27T03:31:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32148.341255 secs 2024-04-27T03:31:07.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32148.341255 secs 2024-04-27T03:31:08.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:07 smithi090 ceph-mon[29180]: pgmap v17694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:08.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:07 smithi119 ceph-mon[32752]: pgmap v17694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:08.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:07 smithi167 ceph-mon[32751]: pgmap v17694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:10.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:09 smithi090 ceph-mon[29180]: pgmap v17695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:10.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:09 smithi119 ceph-mon[32752]: pgmap v17695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:10.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:09 smithi167 ceph-mon[32751]: pgmap v17695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32152.481278 secs 2024-04-27T03:31:11.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32152.481278 secs 2024-04-27T03:31:11.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32152.481278 secs 2024-04-27T03:31:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:11 smithi090 ceph-mon[29180]: pgmap v17696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32153.341356 secs 2024-04-27T03:31:12.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:11 smithi119 ceph-mon[32752]: pgmap v17696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:12.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32153.341356 secs 2024-04-27T03:31:12.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:11 smithi167 ceph-mon[32751]: pgmap v17696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:12.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32153.341356 secs 2024-04-27T03:31:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:12 smithi090 ceph-mon[29180]: pgmap v17697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:13.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:12 smithi119 ceph-mon[32752]: pgmap v17697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:13.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:12 smithi167 ceph-mon[32751]: pgmap v17697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:15.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:15 smithi119 ceph-mon[32752]: pgmap v17698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:15.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:15 smithi167 ceph-mon[32751]: pgmap v17698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:15.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:15 smithi090 ceph-mon[29180]: pgmap v17698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:16 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32157.481359 secs 2024-04-27T03:31:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:16 smithi119 ceph-mon[32752]: slow request 720.284904 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:31:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32158.341452 secs 2024-04-27T03:31:16.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:16 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32157.481359 secs 2024-04-27T03:31:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:16 smithi167 ceph-mon[32751]: slow request 720.284904 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:31:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32158.341452 secs 2024-04-27T03:31:16.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:16 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32157.481359 secs 2024-04-27T03:31:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:16 smithi090 ceph-mon[29180]: slow request 720.284904 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:31:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32158.341452 secs 2024-04-27T03:31:17.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:17 smithi119 ceph-mon[32752]: pgmap v17699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:17.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:17 smithi167 ceph-mon[32751]: pgmap v17699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:17.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:17 smithi090 ceph-mon[29180]: pgmap v17699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:19.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:19 smithi119 ceph-mon[32752]: pgmap v17700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:19.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:19 smithi167 ceph-mon[32751]: pgmap v17700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:19.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:19 smithi090 ceph-mon[29180]: pgmap v17700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32162.481443 secs 2024-04-27T03:31:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32162.481443 secs 2024-04-27T03:31:20.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32162.481443 secs 2024-04-27T03:31:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:21 smithi119 ceph-mon[32752]: pgmap v17701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32163.341551 secs 2024-04-27T03:31:21.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:21 smithi167 ceph-mon[32751]: pgmap v17701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:21.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32163.341551 secs 2024-04-27T03:31:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:21 smithi090 ceph-mon[29180]: pgmap v17701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32163.341551 secs 2024-04-27T03:31:23.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:23 smithi119 ceph-mon[32752]: pgmap v17702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:23.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:23 smithi167 ceph-mon[32751]: pgmap v17702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:23.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:23 smithi090 ceph-mon[29180]: pgmap v17702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:25.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:25 smithi119 ceph-mon[32752]: pgmap v17703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:25.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32167.481525 secs 2024-04-27T03:31:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:25 smithi167 ceph-mon[32751]: pgmap v17703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32167.481525 secs 2024-04-27T03:31:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:25 smithi090 ceph-mon[29180]: pgmap v17703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32167.481525 secs 2024-04-27T03:31:26.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32168.341631 secs 2024-04-27T03:31:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32168.341631 secs 2024-04-27T03:31:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32168.341631 secs 2024-04-27T03:31:27.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:27 smithi119 ceph-mon[32752]: pgmap v17704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:27.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:27 smithi167 ceph-mon[32751]: pgmap v17704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:27.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:27 smithi090 ceph-mon[29180]: pgmap v17704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:29.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:29 smithi119 ceph-mon[32752]: pgmap v17705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:29.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:29 smithi167 ceph-mon[32751]: pgmap v17705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:29.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:29 smithi090 ceph-mon[29180]: pgmap v17705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32172.481609 secs 2024-04-27T03:31:30.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32172.481609 secs 2024-04-27T03:31:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32172.481609 secs 2024-04-27T03:31:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:31 smithi119 ceph-mon[32752]: pgmap v17706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32173.341713 secs 2024-04-27T03:31:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:31 smithi167 ceph-mon[32751]: pgmap v17706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32173.341713 secs 2024-04-27T03:31:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:31 smithi090 ceph-mon[29180]: pgmap v17706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32173.341713 secs 2024-04-27T03:31:32.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:31:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:31:32.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:31:33.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:33 smithi119 ceph-mon[32752]: pgmap v17707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:33.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:31:33.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:31:33.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:31:33.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:33 smithi167 ceph-mon[32751]: pgmap v17707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:33.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:31:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:31:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:31:33.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:33 smithi090 ceph-mon[29180]: pgmap v17707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:33.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:31:33.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:31:33.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:31:34.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:34 smithi119 ceph-mon[32752]: pgmap v17708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:31:34.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:34 smithi119 ceph-mon[32752]: pgmap v17709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:31:34.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:34 smithi167 ceph-mon[32751]: pgmap v17708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:31:34.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:34 smithi167 ceph-mon[32751]: pgmap v17709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:31:34.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:34 smithi090 ceph-mon[29180]: pgmap v17708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:31:34.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:34 smithi090 ceph-mon[29180]: pgmap v17709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:31:35.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32177.481689 secs 2024-04-27T03:31:35.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32177.481689 secs 2024-04-27T03:31:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32177.481689 secs 2024-04-27T03:31:36.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:36 smithi119 ceph-mon[32752]: pgmap v17710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:36.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32178.341792 secs 2024-04-27T03:31:36.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:36 smithi167 ceph-mon[32751]: pgmap v17710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:36.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32178.341792 secs 2024-04-27T03:31:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:36 smithi090 ceph-mon[29180]: pgmap v17710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32178.341792 secs 2024-04-27T03:31:38.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:38 smithi119 ceph-mon[32752]: pgmap v17711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:31:38.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:38 smithi167 ceph-mon[32751]: pgmap v17711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:31:38.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:38 smithi090 ceph-mon[29180]: pgmap v17711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:31:40.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:40 smithi119 ceph-mon[32752]: pgmap v17712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:40.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32182.481770 secs 2024-04-27T03:31:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:40 smithi167 ceph-mon[32751]: pgmap v17712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:40.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32182.481770 secs 2024-04-27T03:31:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:40 smithi090 ceph-mon[29180]: pgmap v17712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32182.481770 secs 2024-04-27T03:31:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32183.341914 secs 2024-04-27T03:31:41.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32183.341914 secs 2024-04-27T03:31:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32183.341914 secs 2024-04-27T03:31:42.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:42 smithi119 ceph-mon[32752]: pgmap v17713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:42.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:42 smithi167 ceph-mon[32751]: pgmap v17713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:42.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:42 smithi090 ceph-mon[29180]: pgmap v17713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:31:44.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:44 smithi119 ceph-mon[32752]: pgmap v17714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:31:44.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:44 smithi167 ceph-mon[32751]: pgmap v17714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:31:44.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:44 smithi090 ceph-mon[29180]: pgmap v17714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:31:45.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32187.481849 secs 2024-04-27T03:31:45.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32187.481849 secs 2024-04-27T03:31:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32187.481849 secs 2024-04-27T03:31:46.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:46 smithi119 ceph-mon[32752]: pgmap v17715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:46.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32188.342025 secs 2024-04-27T03:31:46.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:46 smithi167 ceph-mon[32751]: pgmap v17715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:46.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32188.342025 secs 2024-04-27T03:31:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:46 smithi090 ceph-mon[29180]: pgmap v17715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32188.342025 secs 2024-04-27T03:31:48.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:48 smithi119 ceph-mon[32752]: pgmap v17716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:48.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:48 smithi167 ceph-mon[32751]: pgmap v17716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:48.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:48 smithi090 ceph-mon[29180]: pgmap v17716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:50 smithi119 ceph-mon[32752]: pgmap v17717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32192.481911 secs 2024-04-27T03:31:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:50 smithi167 ceph-mon[32751]: pgmap v17717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32192.481911 secs 2024-04-27T03:31:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:50 smithi090 ceph-mon[29180]: pgmap v17717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32192.481911 secs 2024-04-27T03:31:51.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32193.342136 secs 2024-04-27T03:31:51.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32193.342136 secs 2024-04-27T03:31:51.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32193.342136 secs 2024-04-27T03:31:52.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:52 smithi119 ceph-mon[32752]: pgmap v17718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:52.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:52 smithi167 ceph-mon[32751]: pgmap v17718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:52.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:52 smithi090 ceph-mon[29180]: pgmap v17718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:54.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:54 smithi119 ceph-mon[32752]: pgmap v17719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:54.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:54 smithi167 ceph-mon[32751]: pgmap v17719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:54.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:54 smithi090 ceph-mon[29180]: pgmap v17719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:55.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32197.481971 secs 2024-04-27T03:31:55.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32197.481971 secs 2024-04-27T03:31:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32197.481971 secs 2024-04-27T03:31:56.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:56 smithi090 ceph-mon[29180]: pgmap v17720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:56.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32198.342258 secs 2024-04-27T03:31:56.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:56 smithi167 ceph-mon[32751]: pgmap v17720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:56.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32198.342258 secs 2024-04-27T03:31:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:56 smithi119 ceph-mon[32752]: pgmap v17720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:31:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32198.342258 secs 2024-04-27T03:31:58.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:31:58 smithi167 ceph-mon[32751]: pgmap v17721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:58.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:31:58 smithi090 ceph-mon[29180]: pgmap v17721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:31:59.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:31:58 smithi119 ceph-mon[32752]: pgmap v17721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:00 smithi167 ceph-mon[32751]: pgmap v17722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32202.482052 secs 2024-04-27T03:32:00.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:00 smithi090 ceph-mon[29180]: pgmap v17722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:00.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32202.482052 secs 2024-04-27T03:32:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:00 smithi119 ceph-mon[32752]: pgmap v17722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32202.482052 secs 2024-04-27T03:32:01.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32203.342383 secs 2024-04-27T03:32:01.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32203.342383 secs 2024-04-27T03:32:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32203.342383 secs 2024-04-27T03:32:02.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:02 smithi167 ceph-mon[32751]: pgmap v17723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:02.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:02 smithi090 ceph-mon[29180]: pgmap v17723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:03.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:02 smithi119 ceph-mon[32752]: pgmap v17723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:04.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:04 smithi090 ceph-mon[29180]: pgmap v17724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:05.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:04 smithi119 ceph-mon[32752]: pgmap v17724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:05.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:04 smithi167 ceph-mon[32751]: pgmap v17724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:05.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32207.482112 secs 2024-04-27T03:32:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32207.482112 secs 2024-04-27T03:32:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32207.482112 secs 2024-04-27T03:32:06.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:06 smithi090 ceph-mon[29180]: pgmap v17725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:06.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32208.342499 secs 2024-04-27T03:32:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:06 smithi119 ceph-mon[32752]: pgmap v17725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32208.342499 secs 2024-04-27T03:32:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:06 smithi167 ceph-mon[32751]: pgmap v17725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32208.342499 secs 2024-04-27T03:32:08.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:08 smithi090 ceph-mon[29180]: pgmap v17726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:09.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:08 smithi119 ceph-mon[32752]: pgmap v17726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:09.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:08 smithi167 ceph-mon[32751]: pgmap v17726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:10.131 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:32:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:32:09.724+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:32:10.132 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:32:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:32:09.724+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:32:10.132 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:32:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:32:09.724+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:32:10.132 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:32:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:32:09.724+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:32:10.132 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:32:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:32:09.724+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:32:10.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:10 smithi090 ceph-mon[29180]: pgmap v17727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:10.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32212.482195 secs 2024-04-27T03:32:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:10 smithi119 ceph-mon[32752]: pgmap v17727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32212.482195 secs 2024-04-27T03:32:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:10 smithi167 ceph-mon[32751]: pgmap v17727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32212.482195 secs 2024-04-27T03:32:11.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32213.342617 secs 2024-04-27T03:32:11.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32213.342617 secs 2024-04-27T03:32:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32213.342617 secs 2024-04-27T03:32:12.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:12 smithi090 ceph-mon[29180]: pgmap v17728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:13.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:12 smithi119 ceph-mon[32752]: pgmap v17728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:13.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:12 smithi167 ceph-mon[32751]: pgmap v17728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:14.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:14 smithi090 ceph-mon[29180]: pgmap v17729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:15.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:14 smithi119 ceph-mon[32752]: pgmap v17729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:15.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:14 smithi167 ceph-mon[32751]: pgmap v17729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:15.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32217.482277 secs 2024-04-27T03:32:15.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32217.482277 secs 2024-04-27T03:32:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32217.482277 secs 2024-04-27T03:32:16.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:16 smithi090 ceph-mon[29180]: pgmap v17730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:16.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32218.342693 secs 2024-04-27T03:32:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:16 smithi119 ceph-mon[32752]: pgmap v17730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32218.342693 secs 2024-04-27T03:32:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:16 smithi167 ceph-mon[32751]: pgmap v17730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32218.342693 secs 2024-04-27T03:32:18.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:18 smithi090 ceph-mon[29180]: pgmap v17731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:19.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:18 smithi119 ceph-mon[32752]: pgmap v17731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:19.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:18 smithi167 ceph-mon[32751]: pgmap v17731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:20.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:20 smithi090 ceph-mon[29180]: pgmap v17732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:20.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32222.482359 secs 2024-04-27T03:32:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:20 smithi119 ceph-mon[32752]: pgmap v17732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32222.482359 secs 2024-04-27T03:32:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:20 smithi167 ceph-mon[32751]: pgmap v17732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32222.482359 secs 2024-04-27T03:32:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32223.342774 secs 2024-04-27T03:32:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32223.342774 secs 2024-04-27T03:32:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32223.342774 secs 2024-04-27T03:32:22.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:22 smithi090 ceph-mon[29180]: pgmap v17733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:23.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:22 smithi119 ceph-mon[32752]: pgmap v17733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:23.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:22 smithi167 ceph-mon[32751]: pgmap v17733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:24.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:24 smithi090 ceph-mon[29180]: pgmap v17734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:25.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:24 smithi119 ceph-mon[32752]: pgmap v17734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:25.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:24 smithi167 ceph-mon[32751]: pgmap v17734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T03:32:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32227.482442 secs 2024-04-27T03:32:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32227.482442 secs 2024-04-27T03:32:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32227.482442 secs 2024-04-27T03:32:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:26 smithi090 ceph-mon[29180]: pgmap v17735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32228.342850 secs 2024-04-27T03:32:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:26 smithi119 ceph-mon[32752]: pgmap v17735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32228.342850 secs 2024-04-27T03:32:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:26 smithi167 ceph-mon[32751]: pgmap v17735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T03:32:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32228.342850 secs 2024-04-27T03:32:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:28 smithi119 ceph-mon[32752]: pgmap v17736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:29.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:28 smithi167 ceph-mon[32751]: pgmap v17736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:29.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:28 smithi090 ceph-mon[29180]: pgmap v17736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:30 smithi119 ceph-mon[32752]: pgmap v17737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32232.482524 secs 2024-04-27T03:32:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:30 smithi167 ceph-mon[32751]: pgmap v17737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32232.482524 secs 2024-04-27T03:32:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:30 smithi090 ceph-mon[29180]: pgmap v17737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32232.482524 secs 2024-04-27T03:32:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32233.342984 secs 2024-04-27T03:32:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32233.342984 secs 2024-04-27T03:32:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32233.342984 secs 2024-04-27T03:32:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:32 smithi119 ceph-mon[32752]: pgmap v17738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:32:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:32 smithi167 ceph-mon[32751]: pgmap v17738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:32:33.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:32 smithi090 ceph-mon[29180]: pgmap v17738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:32:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:32:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:32:34.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:32:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:34 smithi119 ceph-mon[32752]: pgmap v17739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:32:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:32:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:32:35.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:32:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:34 smithi167 ceph-mon[32751]: pgmap v17739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:32:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:32:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:32:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:32:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:34 smithi090 ceph-mon[29180]: pgmap v17739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:32:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:32:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:32:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:32:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:35 smithi119 ceph-mon[32752]: pgmap v17740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:32:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:35 smithi119 ceph-mon[32752]: pgmap v17741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:32:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32237.482607 secs 2024-04-27T03:32:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:35 smithi167 ceph-mon[32751]: pgmap v17740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:32:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:35 smithi167 ceph-mon[32751]: pgmap v17741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:32:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32237.482607 secs 2024-04-27T03:32:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:35 smithi090 ceph-mon[29180]: pgmap v17740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:32:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:35 smithi090 ceph-mon[29180]: pgmap v17741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:32:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32237.482607 secs 2024-04-27T03:32:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32238.343123 secs 2024-04-27T03:32:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32238.343123 secs 2024-04-27T03:32:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32238.343123 secs 2024-04-27T03:32:38.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:37 smithi119 ceph-mon[32752]: pgmap v17742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:32:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:37 smithi167 ceph-mon[32751]: pgmap v17742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:32:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:37 smithi090 ceph-mon[29180]: pgmap v17742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T03:32:40.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:39 smithi119 ceph-mon[32752]: pgmap v17743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:32:40.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:39 smithi167 ceph-mon[32751]: pgmap v17743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:32:40.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:39 smithi090 ceph-mon[29180]: pgmap v17743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:32:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32242.482693 secs 2024-04-27T03:32:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32242.482693 secs 2024-04-27T03:32:41.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32242.482693 secs 2024-04-27T03:32:41.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:41 smithi167 ceph-mon[32751]: pgmap v17744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:32:41.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32243.343266 secs 2024-04-27T03:32:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:41 smithi119 ceph-mon[32752]: pgmap v17744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:32:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32243.343266 secs 2024-04-27T03:32:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:41 smithi090 ceph-mon[29180]: pgmap v17744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:32:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32243.343266 secs 2024-04-27T03:32:43.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:43 smithi090 ceph-mon[29180]: pgmap v17745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:32:44.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:43 smithi119 ceph-mon[32752]: pgmap v17745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:32:44.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:43 smithi167 ceph-mon[32751]: pgmap v17745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:32:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:45 smithi119 ceph-mon[32752]: pgmap v17746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:32:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32247.482775 secs 2024-04-27T03:32:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:45 smithi167 ceph-mon[32751]: pgmap v17746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:32:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32247.482775 secs 2024-04-27T03:32:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:45 smithi090 ceph-mon[29180]: pgmap v17746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:32:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32247.482775 secs 2024-04-27T03:32:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32248.343411 secs 2024-04-27T03:32:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32248.343411 secs 2024-04-27T03:32:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32248.343411 secs 2024-04-27T03:32:48.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:47 smithi119 ceph-mon[32752]: pgmap v17747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:48.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:47 smithi167 ceph-mon[32751]: pgmap v17747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:48.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:47 smithi090 ceph-mon[29180]: pgmap v17747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:50.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:49 smithi119 ceph-mon[32752]: pgmap v17748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:50.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:49 smithi167 ceph-mon[32751]: pgmap v17748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:50.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:49 smithi090 ceph-mon[29180]: pgmap v17748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32252.482819 secs 2024-04-27T03:32:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32252.482819 secs 2024-04-27T03:32:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32252.482819 secs 2024-04-27T03:32:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:51 smithi119 ceph-mon[32752]: pgmap v17749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32253.343529 secs 2024-04-27T03:32:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:51 smithi167 ceph-mon[32751]: pgmap v17749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32253.343529 secs 2024-04-27T03:32:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:51 smithi090 ceph-mon[29180]: pgmap v17749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32253.343529 secs 2024-04-27T03:32:54.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:53 smithi119 ceph-mon[32752]: pgmap v17750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:54.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:53 smithi167 ceph-mon[32751]: pgmap v17750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:54.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:53 smithi090 ceph-mon[29180]: pgmap v17750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:55 smithi119 ceph-mon[32752]: pgmap v17751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32257.482878 secs 2024-04-27T03:32:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:55 smithi167 ceph-mon[32751]: pgmap v17751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32257.482878 secs 2024-04-27T03:32:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:55 smithi090 ceph-mon[29180]: pgmap v17751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:32:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32257.482878 secs 2024-04-27T03:32:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32258.343656 secs 2024-04-27T03:32:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32258.343656 secs 2024-04-27T03:32:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32258.343656 secs 2024-04-27T03:32:58.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:57 smithi119 ceph-mon[32752]: pgmap v17752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:58.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:57 smithi167 ceph-mon[32751]: pgmap v17752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:32:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:57 smithi090 ceph-mon[29180]: pgmap v17752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:00.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:32:59 smithi119 ceph-mon[32752]: pgmap v17753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:00.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:32:59 smithi167 ceph-mon[32751]: pgmap v17753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:00.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:32:59 smithi090 ceph-mon[29180]: pgmap v17753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32262.482963 secs 2024-04-27T03:33:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32262.482963 secs 2024-04-27T03:33:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32262.482963 secs 2024-04-27T03:33:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:01 smithi119 ceph-mon[32752]: pgmap v17754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32263.343778 secs 2024-04-27T03:33:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:01 smithi167 ceph-mon[32751]: pgmap v17754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32263.343778 secs 2024-04-27T03:33:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:01 smithi090 ceph-mon[29180]: pgmap v17754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32263.343778 secs 2024-04-27T03:33:04.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:03 smithi119 ceph-mon[32752]: pgmap v17755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:04.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:03 smithi167 ceph-mon[32751]: pgmap v17755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:04.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:03 smithi090 ceph-mon[29180]: pgmap v17755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:05 smithi119 ceph-mon[32752]: pgmap v17756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32267.483023 secs 2024-04-27T03:33:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:05 smithi167 ceph-mon[32751]: pgmap v17756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32267.483023 secs 2024-04-27T03:33:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:05 smithi090 ceph-mon[29180]: pgmap v17756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32267.483023 secs 2024-04-27T03:33:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32268.343899 secs 2024-04-27T03:33:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32268.343899 secs 2024-04-27T03:33:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32268.343899 secs 2024-04-27T03:33:08.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:07 smithi119 ceph-mon[32752]: pgmap v17757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:08.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:07 smithi167 ceph-mon[32751]: pgmap v17757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:08.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:07 smithi090 ceph-mon[29180]: pgmap v17757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:10.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:09 smithi119 ceph-mon[32752]: pgmap v17758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:10.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:09 smithi167 ceph-mon[32751]: pgmap v17758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:10.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:09 smithi090 ceph-mon[29180]: pgmap v17758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32272.483082 secs 2024-04-27T03:33:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32272.483082 secs 2024-04-27T03:33:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32272.483082 secs 2024-04-27T03:33:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:11 smithi119 ceph-mon[32752]: pgmap v17759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:11 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 32273.344015 secs 2024-04-27T03:33:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:11 smithi119 ceph-mon[32752]: slow request 2882.202305 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:33:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:11 smithi167 ceph-mon[32751]: pgmap v17759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:11 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 32273.344015 secs 2024-04-27T03:33:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:11 smithi167 ceph-mon[32751]: slow request 2882.202305 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:33:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:11 smithi090 ceph-mon[29180]: pgmap v17759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:11 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 32273.344015 secs 2024-04-27T03:33:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:11 smithi090 ceph-mon[29180]: slow request 2882.202305 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:33:14.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:13 smithi119 ceph-mon[32752]: pgmap v17760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:14.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:13 smithi167 ceph-mon[32751]: pgmap v17760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:14.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:13 smithi090 ceph-mon[29180]: pgmap v17760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:33:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:15 smithi119 ceph-mon[32752]: pgmap v17761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32277.483160 secs 2024-04-27T03:33:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:15 smithi167 ceph-mon[32751]: pgmap v17761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32277.483160 secs 2024-04-27T03:33:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:15 smithi090 ceph-mon[29180]: pgmap v17761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:33:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32277.483160 secs 2024-04-27T03:33:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32278.344132 secs 2024-04-27T03:33:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32278.344132 secs 2024-04-27T03:33:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32278.344132 secs 2024-04-27T03:33:18.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:17 smithi119 ceph-mon[32752]: pgmap v17762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:18.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:17 smithi167 ceph-mon[32751]: pgmap v17762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:18.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:17 smithi090 ceph-mon[29180]: pgmap v17762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:20.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:19 smithi119 ceph-mon[32752]: pgmap v17763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:20.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:19 smithi167 ceph-mon[32751]: pgmap v17763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:20.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:19 smithi090 ceph-mon[29180]: pgmap v17763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32282.483246 secs 2024-04-27T03:33:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32282.483246 secs 2024-04-27T03:33:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32282.483246 secs 2024-04-27T03:33:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:21 smithi119 ceph-mon[32752]: pgmap v17764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32283.344251 secs 2024-04-27T03:33:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:21 smithi167 ceph-mon[32751]: pgmap v17764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32283.344251 secs 2024-04-27T03:33:22.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:21 smithi090 ceph-mon[29180]: pgmap v17764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:22.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32283.344251 secs 2024-04-27T03:33:24.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:23 smithi119 ceph-mon[32752]: pgmap v17765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:24.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:23 smithi167 ceph-mon[32751]: pgmap v17765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:24.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:23 smithi090 ceph-mon[29180]: pgmap v17765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:25 smithi119 ceph-mon[32752]: pgmap v17766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32287.483329 secs 2024-04-27T03:33:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:25 smithi167 ceph-mon[32751]: pgmap v17766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32287.483329 secs 2024-04-27T03:33:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:25 smithi090 ceph-mon[29180]: pgmap v17766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32287.483329 secs 2024-04-27T03:33:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32288.344373 secs 2024-04-27T03:33:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32288.344373 secs 2024-04-27T03:33:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32288.344373 secs 2024-04-27T03:33:28.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:27 smithi119 ceph-mon[32752]: pgmap v17767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:28.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:27 smithi167 ceph-mon[32751]: pgmap v17767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:28.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:27 smithi090 ceph-mon[29180]: pgmap v17767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:30.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:29 smithi119 ceph-mon[32752]: pgmap v17768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:30.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:29 smithi167 ceph-mon[32751]: pgmap v17768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:30.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:29 smithi090 ceph-mon[29180]: pgmap v17768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32292.483412 secs 2024-04-27T03:33:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32292.483412 secs 2024-04-27T03:33:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32292.483412 secs 2024-04-27T03:33:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:31 smithi119 ceph-mon[32752]: pgmap v17769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32293.344508 secs 2024-04-27T03:33:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:31 smithi167 ceph-mon[32751]: pgmap v17769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32293.344508 secs 2024-04-27T03:33:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:31 smithi090 ceph-mon[29180]: pgmap v17769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32293.344508 secs 2024-04-27T03:33:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:33 smithi119 ceph-mon[32752]: pgmap v17770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:33 smithi167 ceph-mon[32751]: pgmap v17770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:34.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:33 smithi090 ceph-mon[29180]: pgmap v17770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:33:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:33:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:33:35.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:33:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:33:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:33:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:33:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:33:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:33:35.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:33:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:33:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:33:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:35 smithi119 ceph-mon[32752]: pgmap v17771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:35 smithi119 ceph-mon[32752]: pgmap v17772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:33:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:35 smithi119 ceph-mon[32752]: pgmap v17773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32297.483496 secs 2024-04-27T03:33:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:35 smithi167 ceph-mon[32751]: pgmap v17771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:35 smithi167 ceph-mon[32751]: pgmap v17772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:33:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:35 smithi167 ceph-mon[32751]: pgmap v17773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32297.483496 secs 2024-04-27T03:33:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:35 smithi090 ceph-mon[29180]: pgmap v17771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:35 smithi090 ceph-mon[29180]: pgmap v17772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:33:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:35 smithi090 ceph-mon[29180]: pgmap v17773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:36.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32297.483496 secs 2024-04-27T03:33:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32298.344620 secs 2024-04-27T03:33:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32298.344620 secs 2024-04-27T03:33:37.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32298.344620 secs 2024-04-27T03:33:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:37 smithi167 ceph-mon[32751]: pgmap v17774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:37 smithi090 ceph-mon[29180]: pgmap v17774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:37 smithi119 ceph-mon[32752]: pgmap v17774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:40.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:39 smithi167 ceph-mon[32751]: pgmap v17775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:40.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:39 smithi090 ceph-mon[29180]: pgmap v17775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:40.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:39 smithi119 ceph-mon[32752]: pgmap v17775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:41.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32302.483577 secs 2024-04-27T03:33:41.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32302.483577 secs 2024-04-27T03:33:41.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32302.483577 secs 2024-04-27T03:33:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:41 smithi090 ceph-mon[29180]: pgmap v17776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:41 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 32303.344762 secs 2024-04-27T03:33:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:41 smithi090 ceph-mon[29180]: slow request 5762.950779 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:33:42.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:41 smithi167 ceph-mon[32751]: pgmap v17776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:42.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:41 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 32303.344762 secs 2024-04-27T03:33:42.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:41 smithi167 ceph-mon[32751]: slow request 5762.950779 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:33:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:41 smithi119 ceph-mon[32752]: pgmap v17776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:33:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:41 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 32303.344762 secs 2024-04-27T03:33:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:41 smithi119 ceph-mon[32752]: slow request 5762.950779 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:33:44.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:43 smithi090 ceph-mon[29180]: pgmap v17777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:33:44.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:43 smithi119 ceph-mon[32752]: pgmap v17777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:33:44.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:43 smithi167 ceph-mon[32751]: pgmap v17777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:33:46.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:45 smithi119 ceph-mon[32752]: pgmap v17778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:33:46.112 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32307.483656 secs 2024-04-27T03:33:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:45 smithi090 ceph-mon[29180]: pgmap v17778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:33:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32307.483656 secs 2024-04-27T03:33:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:45 smithi167 ceph-mon[32751]: pgmap v17778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:33:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32307.483656 secs 2024-04-27T03:33:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32308.344875 secs 2024-04-27T03:33:47.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32308.344875 secs 2024-04-27T03:33:47.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32308.344875 secs 2024-04-27T03:33:48.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:47 smithi090 ceph-mon[29180]: pgmap v17779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:48.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:47 smithi119 ceph-mon[32752]: pgmap v17779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:48.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:47 smithi167 ceph-mon[32751]: pgmap v17779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:50.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:49 smithi090 ceph-mon[29180]: pgmap v17780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:50.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:49 smithi119 ceph-mon[32752]: pgmap v17780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:50.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:49 smithi167 ceph-mon[32751]: pgmap v17780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32312.483726 secs 2024-04-27T03:33:51.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32312.483726 secs 2024-04-27T03:33:51.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32312.483726 secs 2024-04-27T03:33:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:51 smithi090 ceph-mon[29180]: pgmap v17781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32313.344954 secs 2024-04-27T03:33:52.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:51 smithi119 ceph-mon[32752]: pgmap v17781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:52.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32313.344954 secs 2024-04-27T03:33:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:51 smithi167 ceph-mon[32751]: pgmap v17781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32313.344954 secs 2024-04-27T03:33:54.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:53 smithi090 ceph-mon[29180]: pgmap v17782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:54.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:53 smithi119 ceph-mon[32752]: pgmap v17782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:54.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:53 smithi167 ceph-mon[32751]: pgmap v17782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:55 smithi090 ceph-mon[29180]: pgmap v17783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32317.483802 secs 2024-04-27T03:33:56.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:55 smithi119 ceph-mon[32752]: pgmap v17783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:56.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32317.483802 secs 2024-04-27T03:33:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:55 smithi167 ceph-mon[32751]: pgmap v17783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:33:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32317.483802 secs 2024-04-27T03:33:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32318.345088 secs 2024-04-27T03:33:57.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32318.345088 secs 2024-04-27T03:33:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32318.345088 secs 2024-04-27T03:33:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:57 smithi090 ceph-mon[29180]: pgmap v17784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:58.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:57 smithi119 ceph-mon[32752]: pgmap v17784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:33:58.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:57 smithi167 ceph-mon[32751]: pgmap v17784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:00.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:33:59 smithi090 ceph-mon[29180]: pgmap v17785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:00.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:33:59 smithi119 ceph-mon[32752]: pgmap v17785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:00.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:33:59 smithi167 ceph-mon[32751]: pgmap v17785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32322.483825 secs 2024-04-27T03:34:01.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32322.483825 secs 2024-04-27T03:34:01.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32322.483825 secs 2024-04-27T03:34:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:01 smithi090 ceph-mon[29180]: pgmap v17786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32323.345211 secs 2024-04-27T03:34:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:01 smithi119 ceph-mon[32752]: pgmap v17786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32323.345211 secs 2024-04-27T03:34:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:01 smithi167 ceph-mon[32751]: pgmap v17786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32323.345211 secs 2024-04-27T03:34:03.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:02 smithi090 ceph-mon[29180]: pgmap v17787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:03.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:02 smithi119 ceph-mon[32752]: pgmap v17787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:02 smithi167 ceph-mon[32751]: pgmap v17787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:05 smithi119 ceph-mon[32752]: pgmap v17788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32327.483910 secs 2024-04-27T03:34:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:05 smithi167 ceph-mon[32751]: pgmap v17788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32327.483910 secs 2024-04-27T03:34:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:05 smithi090 ceph-mon[29180]: pgmap v17788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32327.483910 secs 2024-04-27T03:34:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32328.345354 secs 2024-04-27T03:34:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32328.345354 secs 2024-04-27T03:34:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32328.345354 secs 2024-04-27T03:34:08.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:07 smithi119 ceph-mon[32752]: pgmap v17789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:08.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:07 smithi167 ceph-mon[32751]: pgmap v17789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:08.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:07 smithi090 ceph-mon[29180]: pgmap v17789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:10.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:09 smithi119 ceph-mon[32752]: pgmap v17790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:10.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:09 smithi167 ceph-mon[32751]: pgmap v17790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:10.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:09 smithi090 ceph-mon[29180]: pgmap v17790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32332.483972 secs 2024-04-27T03:34:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32332.483972 secs 2024-04-27T03:34:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32332.483972 secs 2024-04-27T03:34:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:11 smithi119 ceph-mon[32752]: pgmap v17791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32333.345429 secs 2024-04-27T03:34:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:11 smithi167 ceph-mon[32751]: pgmap v17791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32333.345429 secs 2024-04-27T03:34:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:11 smithi090 ceph-mon[29180]: pgmap v17791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32333.345429 secs 2024-04-27T03:34:14.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:13 smithi119 ceph-mon[32752]: pgmap v17792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:14.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:13 smithi167 ceph-mon[32751]: pgmap v17792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:14.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:13 smithi090 ceph-mon[29180]: pgmap v17792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:15 smithi119 ceph-mon[32752]: pgmap v17793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32337.484055 secs 2024-04-27T03:34:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:15 smithi167 ceph-mon[32751]: pgmap v17793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32337.484055 secs 2024-04-27T03:34:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:15 smithi090 ceph-mon[29180]: pgmap v17793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32337.484055 secs 2024-04-27T03:34:17.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32338.345509 secs 2024-04-27T03:34:17.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32338.345509 secs 2024-04-27T03:34:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32338.345509 secs 2024-04-27T03:34:18.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:17 smithi119 ceph-mon[32752]: pgmap v17794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:18.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:17 smithi167 ceph-mon[32751]: pgmap v17794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:18.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:17 smithi090 ceph-mon[29180]: pgmap v17794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:20.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:19 smithi119 ceph-mon[32752]: pgmap v17795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:20.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:19 smithi167 ceph-mon[32751]: pgmap v17795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:20.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:19 smithi090 ceph-mon[29180]: pgmap v17795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32342.484116 secs 2024-04-27T03:34:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32342.484116 secs 2024-04-27T03:34:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32342.484116 secs 2024-04-27T03:34:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:21 smithi119 ceph-mon[32752]: pgmap v17796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32343.345627 secs 2024-04-27T03:34:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:21 smithi167 ceph-mon[32751]: pgmap v17796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32343.345627 secs 2024-04-27T03:34:22.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:21 smithi090 ceph-mon[29180]: pgmap v17796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:22.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32343.345627 secs 2024-04-27T03:34:24.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:23 smithi119 ceph-mon[32752]: pgmap v17797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:24.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:23 smithi167 ceph-mon[32751]: pgmap v17797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:24.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:23 smithi090 ceph-mon[29180]: pgmap v17797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:25 smithi119 ceph-mon[32752]: pgmap v17798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32347.484196 secs 2024-04-27T03:34:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:25 smithi167 ceph-mon[32751]: pgmap v17798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32347.484196 secs 2024-04-27T03:34:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:25 smithi090 ceph-mon[29180]: pgmap v17798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32347.484196 secs 2024-04-27T03:34:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32348.345755 secs 2024-04-27T03:34:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32348.345755 secs 2024-04-27T03:34:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32348.345755 secs 2024-04-27T03:34:28.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:27 smithi119 ceph-mon[32752]: pgmap v17799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:28.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:27 smithi167 ceph-mon[32751]: pgmap v17799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:28.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:27 smithi090 ceph-mon[29180]: pgmap v17799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:30.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:29 smithi119 ceph-mon[32752]: pgmap v17800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:30.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:29 smithi167 ceph-mon[32751]: pgmap v17800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:30.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:29 smithi090 ceph-mon[29180]: pgmap v17800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32352.484279 secs 2024-04-27T03:34:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32352.484279 secs 2024-04-27T03:34:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32352.484279 secs 2024-04-27T03:34:32.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:31 smithi090 ceph-mon[29180]: pgmap v17801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:32.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32353.345870 secs 2024-04-27T03:34:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:31 smithi119 ceph-mon[32752]: pgmap v17801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32353.345870 secs 2024-04-27T03:34:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:31 smithi167 ceph-mon[32751]: pgmap v17801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32353.345870 secs 2024-04-27T03:34:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:33 smithi119 ceph-mon[32752]: pgmap v17802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:33 smithi167 ceph-mon[32751]: pgmap v17802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:34.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:33 smithi090 ceph-mon[29180]: pgmap v17802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:34:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:34:35.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:34:35.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:35 smithi167 ceph-mon[32751]: pgmap v17803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:35.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32357.484350 secs 2024-04-27T03:34:35.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:35.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:35 smithi090 ceph-mon[29180]: pgmap v17803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:36.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32357.484350 secs 2024-04-27T03:34:36.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.082 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.083 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.094 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:35 smithi119 ceph-mon[32752]: pgmap v17803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:36.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32357.484350 secs 2024-04-27T03:34:36.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:36.095 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32358.345936 secs 2024-04-27T03:34:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32358.345936 secs 2024-04-27T03:34:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32358.345936 secs 2024-04-27T03:34:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: pgmap v17804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: pgmap v17804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: pgmap v17804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:38.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:39.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:34:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:34:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:34:39.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:34:39.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:34:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:34:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:34:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:34:39.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:34:39.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:34:39.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:34:39.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:34:40.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:39 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:34:40.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:39 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:34:40.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:39 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:34:40.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:39 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:34:40.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:39 smithi090 ceph-mon[29180]: pgmap v17805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T03:34:40.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:39 smithi090 ceph-mon[29180]: pgmap v17806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T03:34:40.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:39 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:34:40.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:39 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:34:40.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:39 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:34:40.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:39 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:34:40.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:39 smithi119 ceph-mon[32752]: pgmap v17805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T03:34:40.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:39 smithi119 ceph-mon[32752]: pgmap v17806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T03:34:40.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:39 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:34:40.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:39 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:34:40.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:39 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:34:40.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:39 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:34:40.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:39 smithi167 ceph-mon[32751]: pgmap v17805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 891 B/s rd, 1 op/s 2024-04-27T03:34:40.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:39 smithi167 ceph-mon[32751]: pgmap v17806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T03:34:41.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32362.484436 secs 2024-04-27T03:34:41.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32362.484436 secs 2024-04-27T03:34:41.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32362.484436 secs 2024-04-27T03:34:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:41 smithi090 ceph-mon[29180]: pgmap v17807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32363.346075 secs 2024-04-27T03:34:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:41 smithi119 ceph-mon[32752]: pgmap v17807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32363.346075 secs 2024-04-27T03:34:42.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:41 smithi167 ceph-mon[32751]: pgmap v17807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:42.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32363.346075 secs 2024-04-27T03:34:44.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:43 smithi167 ceph-mon[32751]: pgmap v17808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:44.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:43 smithi090 ceph-mon[29180]: pgmap v17808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:44.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:43 smithi119 ceph-mon[32752]: pgmap v17808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:46.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:45 smithi119 ceph-mon[32752]: pgmap v17809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:46.111 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32367.484521 secs 2024-04-27T03:34:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:45 smithi090 ceph-mon[29180]: pgmap v17809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32367.484521 secs 2024-04-27T03:34:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:45 smithi167 ceph-mon[32751]: pgmap v17809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:46.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32367.484521 secs 2024-04-27T03:34:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32368.346233 secs 2024-04-27T03:34:47.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32368.346233 secs 2024-04-27T03:34:47.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32368.346233 secs 2024-04-27T03:34:48.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:47 smithi090 ceph-mon[29180]: pgmap v17810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:48.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:47 smithi119 ceph-mon[32752]: pgmap v17810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:48.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:47 smithi167 ceph-mon[32751]: pgmap v17810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:34:50.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:49 smithi090 ceph-mon[29180]: pgmap v17811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:34:50.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:49 smithi119 ceph-mon[32752]: pgmap v17811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:34:50.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:49 smithi167 ceph-mon[32751]: pgmap v17811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:34:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32372.484609 secs 2024-04-27T03:34:51.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32372.484609 secs 2024-04-27T03:34:51.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32372.484609 secs 2024-04-27T03:34:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:51 smithi090 ceph-mon[29180]: pgmap v17812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32373.346355 secs 2024-04-27T03:34:52.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:51 smithi119 ceph-mon[32752]: pgmap v17812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:52.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32373.346355 secs 2024-04-27T03:34:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:51 smithi167 ceph-mon[32751]: pgmap v17812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32373.346355 secs 2024-04-27T03:34:54.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:53 smithi090 ceph-mon[29180]: pgmap v17813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:54.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:53 smithi119 ceph-mon[32752]: pgmap v17813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:54.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:53 smithi167 ceph-mon[32751]: pgmap v17813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:55 smithi090 ceph-mon[29180]: pgmap v17814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32377.484693 secs 2024-04-27T03:34:56.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:55 smithi119 ceph-mon[32752]: pgmap v17814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:56.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32377.484693 secs 2024-04-27T03:34:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:55 smithi167 ceph-mon[32751]: pgmap v17814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:34:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32377.484693 secs 2024-04-27T03:34:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32378.346442 secs 2024-04-27T03:34:57.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32378.346442 secs 2024-04-27T03:34:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32378.346442 secs 2024-04-27T03:34:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:57 smithi090 ceph-mon[29180]: pgmap v17815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:58.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:57 smithi119 ceph-mon[32752]: pgmap v17815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:34:58.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:57 smithi167 ceph-mon[32751]: pgmap v17815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:00.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:34:59 smithi090 ceph-mon[29180]: pgmap v17816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:00.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:34:59 smithi119 ceph-mon[32752]: pgmap v17816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:00.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:34:59 smithi167 ceph-mon[32751]: pgmap v17816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:00 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32382.484777 secs 2024-04-27T03:35:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:00 smithi090 ceph-mon[29180]: slow request 23040.482775 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:35:01.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:00 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32382.484777 secs 2024-04-27T03:35:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:00 smithi119 ceph-mon[32752]: slow request 23040.482775 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:35:01.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:00 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32382.484777 secs 2024-04-27T03:35:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:00 smithi167 ceph-mon[32751]: slow request 23040.482775 seconds old, received at 2024-04-26T21:10:59.755407+0000: client_request(client.24733:267887 getattr AsXsFs #0x100000001fb 2024-04-26T21:10:59.754898+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:35:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:01 smithi090 ceph-mon[29180]: pgmap v17817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32383.346564 secs 2024-04-27T03:35:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:01 smithi119 ceph-mon[32752]: pgmap v17817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32383.346564 secs 2024-04-27T03:35:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:01 smithi167 ceph-mon[32751]: pgmap v17817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32383.346564 secs 2024-04-27T03:35:03.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:02 smithi119 ceph-mon[32752]: pgmap v17818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:02 smithi167 ceph-mon[32751]: pgmap v17818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:02 smithi090 ceph-mon[29180]: pgmap v17818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:05.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:05 smithi119 ceph-mon[32752]: pgmap v17819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:05.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:05 smithi167 ceph-mon[32751]: pgmap v17819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:05.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:05 smithi090 ceph-mon[29180]: pgmap v17819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:06.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:06 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32387.484837 secs 2024-04-27T03:35:06.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32388.346666 secs 2024-04-27T03:35:06.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:06 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32387.484837 secs 2024-04-27T03:35:06.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32388.346666 secs 2024-04-27T03:35:06.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:06 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32387.484837 secs 2024-04-27T03:35:06.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32388.346666 secs 2024-04-27T03:35:07.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:07 smithi119 ceph-mon[32752]: pgmap v17820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:07.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:07 smithi167 ceph-mon[32751]: pgmap v17820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:07.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:07 smithi090 ceph-mon[29180]: pgmap v17820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:09.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:09 smithi119 ceph-mon[32752]: pgmap v17821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:09.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:09 smithi167 ceph-mon[32751]: pgmap v17821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:09.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:09 smithi090 ceph-mon[29180]: pgmap v17821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:11 smithi119 ceph-mon[32752]: pgmap v17822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:11 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32392.484898 secs 2024-04-27T03:35:11.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32393.346785 secs 2024-04-27T03:35:11.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:11 smithi167 ceph-mon[32751]: pgmap v17822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:11.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:11 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32392.484898 secs 2024-04-27T03:35:11.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32393.346785 secs 2024-04-27T03:35:11.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:11 smithi090 ceph-mon[29180]: pgmap v17822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:11.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:11 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32392.484898 secs 2024-04-27T03:35:11.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32393.346785 secs 2024-04-27T03:35:13.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:13 smithi119 ceph-mon[32752]: pgmap v17823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:13.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:13 smithi167 ceph-mon[32751]: pgmap v17823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:13.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:13 smithi090 ceph-mon[29180]: pgmap v17823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:15.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:15 smithi119 ceph-mon[32752]: pgmap v17824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:15.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:15 smithi167 ceph-mon[32751]: pgmap v17824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:15.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:15 smithi090 ceph-mon[29180]: pgmap v17824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:16.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:16 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32397.484984 secs 2024-04-27T03:35:16.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32398.346897 secs 2024-04-27T03:35:16.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:16 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32397.484984 secs 2024-04-27T03:35:16.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32398.346897 secs 2024-04-27T03:35:16.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:16 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32397.484984 secs 2024-04-27T03:35:16.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32398.346897 secs 2024-04-27T03:35:17.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:17 smithi119 ceph-mon[32752]: pgmap v17825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:17.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:17 smithi167 ceph-mon[32751]: pgmap v17825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:17.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:17 smithi090 ceph-mon[29180]: pgmap v17825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:19.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:19 smithi119 ceph-mon[32752]: pgmap v17826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:19.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:19 smithi167 ceph-mon[32751]: pgmap v17826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:19.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:19 smithi090 ceph-mon[29180]: pgmap v17826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:21.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:21 smithi119 ceph-mon[32752]: pgmap v17827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:21.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:21 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32402.485044 secs 2024-04-27T03:35:21.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32403.347019 secs 2024-04-27T03:35:21.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:21 smithi167 ceph-mon[32751]: pgmap v17827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:21.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:21 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32402.485044 secs 2024-04-27T03:35:21.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32403.347019 secs 2024-04-27T03:35:21.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:21 smithi090 ceph-mon[29180]: pgmap v17827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:21.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:21 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32402.485044 secs 2024-04-27T03:35:21.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32403.347019 secs 2024-04-27T03:35:23.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:23 smithi119 ceph-mon[32752]: pgmap v17828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:23.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:23 smithi167 ceph-mon[32751]: pgmap v17828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:23.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:23 smithi090 ceph-mon[29180]: pgmap v17828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:25.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:25 smithi119 ceph-mon[32752]: pgmap v17829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:25.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:25 smithi167 ceph-mon[32751]: pgmap v17829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:25.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:25 smithi090 ceph-mon[29180]: pgmap v17829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:26.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:26 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32407.485134 secs 2024-04-27T03:35:26.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32408.347120 secs 2024-04-27T03:35:26.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:26 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32407.485134 secs 2024-04-27T03:35:26.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32408.347120 secs 2024-04-27T03:35:26.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:26 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32407.485134 secs 2024-04-27T03:35:26.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32408.347120 secs 2024-04-27T03:35:27.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:27 smithi119 ceph-mon[32752]: pgmap v17830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:27.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:27 smithi167 ceph-mon[32751]: pgmap v17830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:27.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:27 smithi090 ceph-mon[29180]: pgmap v17830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:29.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:29 smithi119 ceph-mon[32752]: pgmap v17831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:29.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:29 smithi167 ceph-mon[32751]: pgmap v17831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:29.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:29 smithi090 ceph-mon[29180]: pgmap v17831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:31 smithi119 ceph-mon[32752]: pgmap v17832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:31 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32412.485196 secs 2024-04-27T03:35:31.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32413.347216 secs 2024-04-27T03:35:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:31 smithi167 ceph-mon[32751]: pgmap v17832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:31 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32412.485196 secs 2024-04-27T03:35:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32413.347216 secs 2024-04-27T03:35:31.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:31 smithi090 ceph-mon[29180]: pgmap v17832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:31.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:31 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32412.485196 secs 2024-04-27T03:35:31.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32413.347216 secs 2024-04-27T03:35:33.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:33 smithi119 ceph-mon[32752]: pgmap v17833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:33.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:33 smithi167 ceph-mon[32751]: pgmap v17833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:33.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:33 smithi090 ceph-mon[29180]: pgmap v17833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:35.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:35 smithi119 ceph-mon[32752]: pgmap v17834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:35.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:35 smithi167 ceph-mon[32751]: pgmap v17834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:35.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:35 smithi090 ceph-mon[29180]: pgmap v17834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:36.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:36 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32417.485269 secs 2024-04-27T03:35:36.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32418.347325 secs 2024-04-27T03:35:36.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:36 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32417.485269 secs 2024-04-27T03:35:36.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32418.347325 secs 2024-04-27T03:35:36.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:36 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32417.485269 secs 2024-04-27T03:35:36.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32418.347325 secs 2024-04-27T03:35:37.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:37 smithi119 ceph-mon[32752]: pgmap v17835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:37.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:37 smithi167 ceph-mon[32751]: pgmap v17835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:37.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:37 smithi090 ceph-mon[29180]: pgmap v17835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:39.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:39 smithi119 ceph-mon[32752]: pgmap v17836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:39.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:35:39.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:39.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:39.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:39 smithi167 ceph-mon[32751]: pgmap v17836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:39.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:35:39.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:39.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:39.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:39 smithi090 ceph-mon[29180]: pgmap v17836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:39.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:35:39.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:39.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:40.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:35:40.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:35:40.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:40 smithi119 ceph-mon[32752]: pgmap v17837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:35:40.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:40 smithi119 ceph-mon[32752]: pgmap v17838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:40.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:40.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:35:40.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:35:40.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:40 smithi167 ceph-mon[32751]: pgmap v17837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:35:40.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:40 smithi167 ceph-mon[32751]: pgmap v17838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:40.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:40.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:35:40.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:35:40.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:40 smithi090 ceph-mon[29180]: pgmap v17837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:35:40.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:40 smithi090 ceph-mon[29180]: pgmap v17838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:40.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:35:41.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:41 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32422.485353 secs 2024-04-27T03:35:41.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32423.347448 secs 2024-04-27T03:35:41.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:41 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32422.485353 secs 2024-04-27T03:35:41.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32423.347448 secs 2024-04-27T03:35:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:41 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32422.485353 secs 2024-04-27T03:35:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32423.347448 secs 2024-04-27T03:35:42.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:42 smithi167 ceph-mon[32751]: pgmap v17839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:42.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:42 smithi090 ceph-mon[29180]: pgmap v17839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:42.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:42 smithi119 ceph-mon[32752]: pgmap v17839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:44.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:44 smithi119 ceph-mon[32752]: pgmap v17840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:44.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:44 smithi167 ceph-mon[32751]: pgmap v17840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:44.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:44 smithi090 ceph-mon[29180]: pgmap v17840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:45.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32427.485434 secs 2024-04-27T03:35:45.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32427.485434 secs 2024-04-27T03:35:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32427.485434 secs 2024-04-27T03:35:46.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:46 smithi119 ceph-mon[32752]: pgmap v17841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:46.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32428.347571 secs 2024-04-27T03:35:46.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:46 smithi167 ceph-mon[32751]: pgmap v17841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:46.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32428.347571 secs 2024-04-27T03:35:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:46 smithi090 ceph-mon[29180]: pgmap v17841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32428.347571 secs 2024-04-27T03:35:48.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:48 smithi119 ceph-mon[32752]: pgmap v17842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:48.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:48 smithi167 ceph-mon[32751]: pgmap v17842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:48.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:48 smithi090 ceph-mon[29180]: pgmap v17842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:35:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:50 smithi119 ceph-mon[32752]: pgmap v17843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:35:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32432.485517 secs 2024-04-27T03:35:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:50 smithi167 ceph-mon[32751]: pgmap v17843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:35:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32432.485517 secs 2024-04-27T03:35:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:50 smithi090 ceph-mon[29180]: pgmap v17843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:35:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32432.485517 secs 2024-04-27T03:35:51.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32433.347687 secs 2024-04-27T03:35:51.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32433.347687 secs 2024-04-27T03:35:51.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32433.347687 secs 2024-04-27T03:35:52.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:52 smithi119 ceph-mon[32752]: pgmap v17844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:52.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:52 smithi167 ceph-mon[32751]: pgmap v17844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:52.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:52 smithi090 ceph-mon[29180]: pgmap v17844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:54.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:54 smithi119 ceph-mon[32752]: pgmap v17845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:54.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:54 smithi167 ceph-mon[32751]: pgmap v17845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:54.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:54 smithi090 ceph-mon[29180]: pgmap v17845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:35:55.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32437.485599 secs 2024-04-27T03:35:55.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32437.485599 secs 2024-04-27T03:35:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32437.485599 secs 2024-04-27T03:35:56.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:56 smithi167 ceph-mon[32751]: pgmap v17846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:56.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32438.347808 secs 2024-04-27T03:35:56.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:56 smithi090 ceph-mon[29180]: pgmap v17846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32438.347808 secs 2024-04-27T03:35:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:56 smithi119 ceph-mon[32752]: pgmap v17846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32438.347808 secs 2024-04-27T03:35:58.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:35:58 smithi167 ceph-mon[32751]: pgmap v17847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:58.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:35:58 smithi090 ceph-mon[29180]: pgmap v17847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:35:59.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:35:58 smithi119 ceph-mon[32752]: pgmap v17847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:00 smithi167 ceph-mon[32751]: pgmap v17848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32442.485683 secs 2024-04-27T03:36:00.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:00 smithi090 ceph-mon[29180]: pgmap v17848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:00.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32442.485683 secs 2024-04-27T03:36:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:00 smithi119 ceph-mon[32752]: pgmap v17848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32442.485683 secs 2024-04-27T03:36:01.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32443.347899 secs 2024-04-27T03:36:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32443.347899 secs 2024-04-27T03:36:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32443.347899 secs 2024-04-27T03:36:02.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:02 smithi090 ceph-mon[29180]: pgmap v17849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:03.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:02 smithi119 ceph-mon[32752]: pgmap v17849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:03.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:02 smithi167 ceph-mon[32751]: pgmap v17849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:04.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:04 smithi090 ceph-mon[29180]: pgmap v17850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:05.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:04 smithi119 ceph-mon[32752]: pgmap v17850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:05.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:04 smithi167 ceph-mon[32751]: pgmap v17850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:05.867 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32447.485765 secs 2024-04-27T03:36:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32447.485765 secs 2024-04-27T03:36:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32447.485765 secs 2024-04-27T03:36:06.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:06 smithi090 ceph-mon[29180]: pgmap v17851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32448.348019 secs 2024-04-27T03:36:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:06 smithi119 ceph-mon[32752]: pgmap v17851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:07.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32448.348019 secs 2024-04-27T03:36:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:06 smithi167 ceph-mon[32751]: pgmap v17851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32448.348019 secs 2024-04-27T03:36:08.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:08 smithi090 ceph-mon[29180]: pgmap v17852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:09.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:08 smithi119 ceph-mon[32752]: pgmap v17852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:09.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:08 smithi167 ceph-mon[32751]: pgmap v17852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:10.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:10 smithi090 ceph-mon[29180]: pgmap v17853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32452.485853 secs 2024-04-27T03:36:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:10 smithi119 ceph-mon[32752]: pgmap v17853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32452.485853 secs 2024-04-27T03:36:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:10 smithi167 ceph-mon[32751]: pgmap v17853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32452.485853 secs 2024-04-27T03:36:11.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32453.348154 secs 2024-04-27T03:36:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32453.348154 secs 2024-04-27T03:36:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32453.348154 secs 2024-04-27T03:36:12.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:12 smithi090 ceph-mon[29180]: pgmap v17854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:13.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:12 smithi119 ceph-mon[32752]: pgmap v17854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:13.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:12 smithi167 ceph-mon[32751]: pgmap v17854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:14.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:14 smithi090 ceph-mon[29180]: pgmap v17855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:15.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:14 smithi119 ceph-mon[32752]: pgmap v17855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:15.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:14 smithi167 ceph-mon[32751]: pgmap v17855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:16.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:16 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32457.485942 secs 2024-04-27T03:36:16.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:16 smithi119 ceph-mon[32752]: pgmap v17856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:16.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:16 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32457.485942 secs 2024-04-27T03:36:16.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:16 smithi167 ceph-mon[32751]: pgmap v17856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:16.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:16 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32457.485942 secs 2024-04-27T03:36:16.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:16 smithi090 ceph-mon[29180]: pgmap v17856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:17.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:17 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32458.348245 secs 2024-04-27T03:36:17.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:17 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32458.348245 secs 2024-04-27T03:36:17.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:17 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32458.348245 secs 2024-04-27T03:36:18.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:18 smithi119 ceph-mon[32752]: pgmap v17857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:18.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:18 smithi167 ceph-mon[32751]: pgmap v17857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:18.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:18 smithi090 ceph-mon[29180]: pgmap v17857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:20 smithi119 ceph-mon[32752]: pgmap v17858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32462.486003 secs 2024-04-27T03:36:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:20 smithi167 ceph-mon[32751]: pgmap v17858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32462.486003 secs 2024-04-27T03:36:20.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:20 smithi090 ceph-mon[29180]: pgmap v17858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32462.486003 secs 2024-04-27T03:36:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32463.348394 secs 2024-04-27T03:36:21.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32463.348394 secs 2024-04-27T03:36:21.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32463.348394 secs 2024-04-27T03:36:22.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:22 smithi119 ceph-mon[32752]: pgmap v17859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:22.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:22 smithi167 ceph-mon[32751]: pgmap v17859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:22.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:22 smithi090 ceph-mon[29180]: pgmap v17859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:24.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:24 smithi119 ceph-mon[32752]: pgmap v17860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:24.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:24 smithi167 ceph-mon[32751]: pgmap v17860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:24.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:24 smithi090 ceph-mon[29180]: pgmap v17860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32467.486082 secs 2024-04-27T03:36:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32467.486082 secs 2024-04-27T03:36:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32467.486082 secs 2024-04-27T03:36:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:26 smithi167 ceph-mon[32751]: pgmap v17861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32468.348512 secs 2024-04-27T03:36:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:26 smithi090 ceph-mon[29180]: pgmap v17861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32468.348512 secs 2024-04-27T03:36:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:26 smithi119 ceph-mon[32752]: pgmap v17861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32468.348512 secs 2024-04-27T03:36:28.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:28 smithi167 ceph-mon[32751]: pgmap v17862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:28.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:28 smithi090 ceph-mon[29180]: pgmap v17862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:28 smithi119 ceph-mon[32752]: pgmap v17862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:30.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:30 smithi167 ceph-mon[32751]: pgmap v17863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32472.486164 secs 2024-04-27T03:36:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:30 smithi090 ceph-mon[29180]: pgmap v17863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32472.486164 secs 2024-04-27T03:36:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:30 smithi119 ceph-mon[32752]: pgmap v17863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32472.486164 secs 2024-04-27T03:36:31.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32473.348628 secs 2024-04-27T03:36:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32473.348628 secs 2024-04-27T03:36:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32473.348628 secs 2024-04-27T03:36:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:32 smithi167 ceph-mon[32751]: pgmap v17864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:32.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:32 smithi090 ceph-mon[29180]: pgmap v17864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:32 smithi119 ceph-mon[32752]: pgmap v17864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:34.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:34 smithi090 ceph-mon[29180]: pgmap v17865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:34 smithi119 ceph-mon[32752]: pgmap v17865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:34 smithi167 ceph-mon[32751]: pgmap v17865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:35 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32477.486246 secs 2024-04-27T03:36:35.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:35 smithi090 ceph-mon[29180]: slow request 2881.726587 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:36:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:35 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32477.486246 secs 2024-04-27T03:36:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:35 smithi119 ceph-mon[32752]: slow request 2881.726587 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:36:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:35 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32477.486246 secs 2024-04-27T03:36:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:35 smithi167 ceph-mon[32751]: slow request 2881.726587 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:36:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:36 smithi090 ceph-mon[29180]: pgmap v17866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32478.348745 secs 2024-04-27T03:36:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:36 smithi119 ceph-mon[32752]: pgmap v17866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32478.348745 secs 2024-04-27T03:36:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:36 smithi167 ceph-mon[32751]: pgmap v17866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32478.348745 secs 2024-04-27T03:36:38.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:38 smithi090 ceph-mon[29180]: pgmap v17867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:39.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:38 smithi119 ceph-mon[32752]: pgmap v17867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:39.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:38 smithi167 ceph-mon[32751]: pgmap v17867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:39.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:36:40.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:36:40.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:36:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:40 smithi090 ceph-mon[29180]: pgmap v17868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32482.486331 secs 2024-04-27T03:36:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:36:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:36:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:36:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:40 smithi119 ceph-mon[32752]: pgmap v17868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32482.486331 secs 2024-04-27T03:36:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:36:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:36:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:36:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:40 smithi167 ceph-mon[32751]: pgmap v17868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32482.486331 secs 2024-04-27T03:36:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:36:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:36:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:36:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:41 smithi090 ceph-mon[29180]: pgmap v17869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:36:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:41 smithi090 ceph-mon[29180]: pgmap v17870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32483.348870 secs 2024-04-27T03:36:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:41 smithi119 ceph-mon[32752]: pgmap v17869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:36:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:41 smithi119 ceph-mon[32752]: pgmap v17870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32483.348870 secs 2024-04-27T03:36:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:41 smithi167 ceph-mon[32751]: pgmap v17869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:36:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:41 smithi167 ceph-mon[32751]: pgmap v17870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32483.348870 secs 2024-04-27T03:36:43.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:43 smithi090 ceph-mon[29180]: pgmap v17871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:36:44.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:43 smithi119 ceph-mon[32752]: pgmap v17871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:36:44.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:43 smithi167 ceph-mon[32751]: pgmap v17871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:36:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:45 smithi090 ceph-mon[29180]: pgmap v17872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32487.486414 secs 2024-04-27T03:36:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:45 smithi119 ceph-mon[32752]: pgmap v17872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32487.486414 secs 2024-04-27T03:36:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:45 smithi167 ceph-mon[32751]: pgmap v17872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32487.486414 secs 2024-04-27T03:36:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32488.349022 secs 2024-04-27T03:36:46.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32488.349022 secs 2024-04-27T03:36:46.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32488.349022 secs 2024-04-27T03:36:47.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:47 smithi090 ceph-mon[29180]: pgmap v17873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:48.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:47 smithi119 ceph-mon[32752]: pgmap v17873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:48.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:47 smithi167 ceph-mon[32751]: pgmap v17873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:36:49.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:49 smithi090 ceph-mon[29180]: pgmap v17874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T03:36:50.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:49 smithi119 ceph-mon[32752]: pgmap v17874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T03:36:50.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:49 smithi167 ceph-mon[32751]: pgmap v17874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T03:36:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32492.486497 secs 2024-04-27T03:36:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32492.486497 secs 2024-04-27T03:36:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32492.486497 secs 2024-04-27T03:36:51.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:51 smithi090 ceph-mon[29180]: pgmap v17875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:36:51.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32493.349139 secs 2024-04-27T03:36:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:51 smithi119 ceph-mon[32752]: pgmap v17875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:36:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32493.349139 secs 2024-04-27T03:36:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:51 smithi167 ceph-mon[32751]: pgmap v17875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:36:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32493.349139 secs 2024-04-27T03:36:53.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:53 smithi090 ceph-mon[29180]: pgmap v17876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:54.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:53 smithi119 ceph-mon[32752]: pgmap v17876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:54.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:53 smithi167 ceph-mon[32751]: pgmap v17876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:55 smithi119 ceph-mon[32752]: pgmap v17877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32497.486579 secs 2024-04-27T03:36:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:55 smithi167 ceph-mon[32751]: pgmap v17877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32497.486579 secs 2024-04-27T03:36:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:55 smithi090 ceph-mon[29180]: pgmap v17877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:36:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32497.486579 secs 2024-04-27T03:36:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32498.349256 secs 2024-04-27T03:36:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32498.349256 secs 2024-04-27T03:36:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32498.349256 secs 2024-04-27T03:36:58.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:57 smithi119 ceph-mon[32752]: pgmap v17878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:58.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:57 smithi167 ceph-mon[32751]: pgmap v17878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:36:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:57 smithi090 ceph-mon[29180]: pgmap v17878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:00.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:36:59 smithi119 ceph-mon[32752]: pgmap v17879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:00.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:36:59 smithi167 ceph-mon[32751]: pgmap v17879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:00.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:36:59 smithi090 ceph-mon[29180]: pgmap v17879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32502.486669 secs 2024-04-27T03:37:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32502.486669 secs 2024-04-27T03:37:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32502.486669 secs 2024-04-27T03:37:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:01 smithi119 ceph-mon[32752]: pgmap v17880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32503.349372 secs 2024-04-27T03:37:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:01 smithi167 ceph-mon[32751]: pgmap v17880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32503.349372 secs 2024-04-27T03:37:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:01 smithi090 ceph-mon[29180]: pgmap v17880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32503.349372 secs 2024-04-27T03:37:04.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:03 smithi119 ceph-mon[32752]: pgmap v17881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:04.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:03 smithi167 ceph-mon[32751]: pgmap v17881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:04.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:03 smithi090 ceph-mon[29180]: pgmap v17881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:05 smithi119 ceph-mon[32752]: pgmap v17882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32507.486752 secs 2024-04-27T03:37:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:05 smithi167 ceph-mon[32751]: pgmap v17882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32507.486752 secs 2024-04-27T03:37:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:05 smithi090 ceph-mon[29180]: pgmap v17882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32507.486752 secs 2024-04-27T03:37:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32508.349511 secs 2024-04-27T03:37:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32508.349511 secs 2024-04-27T03:37:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32508.349511 secs 2024-04-27T03:37:08.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:07 smithi090 ceph-mon[29180]: pgmap v17883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:08.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:07 smithi119 ceph-mon[32752]: pgmap v17883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:08.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:07 smithi167 ceph-mon[32751]: pgmap v17883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:10.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:09 smithi119 ceph-mon[32752]: pgmap v17884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:10.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:09 smithi167 ceph-mon[32751]: pgmap v17884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:10.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:09 smithi090 ceph-mon[29180]: pgmap v17884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32512.486848 secs 2024-04-27T03:37:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32512.486848 secs 2024-04-27T03:37:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32512.486848 secs 2024-04-27T03:37:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:11 smithi119 ceph-mon[32752]: pgmap v17885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32513.349630 secs 2024-04-27T03:37:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:11 smithi167 ceph-mon[32751]: pgmap v17885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32513.349630 secs 2024-04-27T03:37:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:11 smithi090 ceph-mon[29180]: pgmap v17885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:12.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32513.349630 secs 2024-04-27T03:37:14.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:13 smithi119 ceph-mon[32752]: pgmap v17886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:14.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:13 smithi167 ceph-mon[32751]: pgmap v17886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:14.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:13 smithi090 ceph-mon[29180]: pgmap v17886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:15 smithi119 ceph-mon[32752]: pgmap v17887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32517.486935 secs 2024-04-27T03:37:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:15 smithi167 ceph-mon[32751]: pgmap v17887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32517.486935 secs 2024-04-27T03:37:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:15 smithi090 ceph-mon[29180]: pgmap v17887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32517.486935 secs 2024-04-27T03:37:16.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32518.349742 secs 2024-04-27T03:37:16.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32518.349742 secs 2024-04-27T03:37:17.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32518.349742 secs 2024-04-27T03:37:18.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:17 smithi119 ceph-mon[32752]: pgmap v17888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:18.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:17 smithi167 ceph-mon[32751]: pgmap v17888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:18.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:17 smithi090 ceph-mon[29180]: pgmap v17888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:20.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:19 smithi119 ceph-mon[32752]: pgmap v17889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:20.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:19 smithi167 ceph-mon[32751]: pgmap v17889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:20.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:19 smithi090 ceph-mon[29180]: pgmap v17889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32522.487022 secs 2024-04-27T03:37:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32522.487022 secs 2024-04-27T03:37:21.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32522.487022 secs 2024-04-27T03:37:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:21 smithi119 ceph-mon[32752]: pgmap v17890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32523.349860 secs 2024-04-27T03:37:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:21 smithi167 ceph-mon[32751]: pgmap v17890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32523.349860 secs 2024-04-27T03:37:22.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:21 smithi090 ceph-mon[29180]: pgmap v17890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:22.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32523.349860 secs 2024-04-27T03:37:24.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:23 smithi119 ceph-mon[32752]: pgmap v17891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:24.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:23 smithi167 ceph-mon[32751]: pgmap v17891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:24.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:23 smithi090 ceph-mon[29180]: pgmap v17891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:25 smithi119 ceph-mon[32752]: pgmap v17892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32527.487095 secs 2024-04-27T03:37:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:25 smithi167 ceph-mon[32751]: pgmap v17892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:26.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32527.487095 secs 2024-04-27T03:37:26.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:25 smithi090 ceph-mon[29180]: pgmap v17892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32527.487095 secs 2024-04-27T03:37:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32528.349939 secs 2024-04-27T03:37:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32528.349939 secs 2024-04-27T03:37:27.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32528.349939 secs 2024-04-27T03:37:28.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:27 smithi119 ceph-mon[32752]: pgmap v17893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:28.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:27 smithi167 ceph-mon[32751]: pgmap v17893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:28.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:27 smithi090 ceph-mon[29180]: pgmap v17893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:30.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:29 smithi119 ceph-mon[32752]: pgmap v17894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:30.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:29 smithi167 ceph-mon[32751]: pgmap v17894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:30.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:29 smithi090 ceph-mon[29180]: pgmap v17894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32532.487181 secs 2024-04-27T03:37:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32532.487181 secs 2024-04-27T03:37:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32532.487181 secs 2024-04-27T03:37:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:31 smithi119 ceph-mon[32752]: pgmap v17895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32533.350062 secs 2024-04-27T03:37:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:31 smithi167 ceph-mon[32751]: pgmap v17895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32533.350062 secs 2024-04-27T03:37:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:31 smithi090 ceph-mon[29180]: pgmap v17895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:32.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32533.350062 secs 2024-04-27T03:37:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:33 smithi119 ceph-mon[32752]: pgmap v17896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:33 smithi167 ceph-mon[32751]: pgmap v17896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:34.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:33 smithi090 ceph-mon[29180]: pgmap v17896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:35 smithi119 ceph-mon[32752]: pgmap v17897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32537.487263 secs 2024-04-27T03:37:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:35 smithi167 ceph-mon[32751]: pgmap v17897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32537.487263 secs 2024-04-27T03:37:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:35 smithi090 ceph-mon[29180]: pgmap v17897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32537.487263 secs 2024-04-27T03:37:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32538.350158 secs 2024-04-27T03:37:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32538.350158 secs 2024-04-27T03:37:37.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32538.350158 secs 2024-04-27T03:37:38.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:37 smithi119 ceph-mon[32752]: pgmap v17898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:37 smithi167 ceph-mon[32751]: pgmap v17898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:38.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:37 smithi090 ceph-mon[29180]: pgmap v17898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:39.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:39 smithi090 ceph-mon[29180]: pgmap v17899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:39.882 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:37:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:37:39.774+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:37:39.882 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:37:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:37:39.774+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:37:39.882 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:37:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:37:39.774+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:37:39.883 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:37:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:37:39.774+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:37:39.883 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:37:39 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:37:39.774+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:37:40.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:39 smithi119 ceph-mon[32752]: pgmap v17899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:40.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:39 smithi167 ceph-mon[32751]: pgmap v17899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32542.487348 secs 2024-04-27T03:37:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:40 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:37:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32542.487348 secs 2024-04-27T03:37:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:40 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:37:41.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32542.487348 secs 2024-04-27T03:37:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:40 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:37:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:41 smithi119 ceph-mon[32752]: pgmap v17900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:37:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:37:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:37:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32543.350221 secs 2024-04-27T03:37:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:41 smithi167 ceph-mon[32751]: pgmap v17900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:37:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:37:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:37:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32543.350221 secs 2024-04-27T03:37:42.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:41 smithi090 ceph-mon[29180]: pgmap v17900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:37:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:37:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:37:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:37:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32543.350221 secs 2024-04-27T03:37:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:42 smithi119 ceph-mon[32752]: pgmap v17901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:37:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:42 smithi119 ceph-mon[32752]: pgmap v17902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:42 smithi167 ceph-mon[32751]: pgmap v17901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:37:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:42 smithi167 ceph-mon[32751]: pgmap v17902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:43.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:42 smithi090 ceph-mon[29180]: pgmap v17901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:37:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:42 smithi090 ceph-mon[29180]: pgmap v17902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:45.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:44 smithi119 ceph-mon[32752]: pgmap v17903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:37:45.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:44 smithi167 ceph-mon[32751]: pgmap v17903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:37:45.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:44 smithi090 ceph-mon[29180]: pgmap v17903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:37:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32547.487429 secs 2024-04-27T03:37:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32547.487429 secs 2024-04-27T03:37:46.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32547.487429 secs 2024-04-27T03:37:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:46 smithi119 ceph-mon[32752]: pgmap v17904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:47.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32548.350333 secs 2024-04-27T03:37:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:46 smithi167 ceph-mon[32751]: pgmap v17904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:47.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32548.350333 secs 2024-04-27T03:37:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:46 smithi090 ceph-mon[29180]: pgmap v17904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:47.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32548.350333 secs 2024-04-27T03:37:49.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:48 smithi119 ceph-mon[32752]: pgmap v17905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:37:49.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:48 smithi167 ceph-mon[32751]: pgmap v17905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:37:49.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:48 smithi090 ceph-mon[29180]: pgmap v17905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:37:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:50 smithi119 ceph-mon[32752]: pgmap v17906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32552.487515 secs 2024-04-27T03:37:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:50 smithi167 ceph-mon[32751]: pgmap v17906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32552.487515 secs 2024-04-27T03:37:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:50 smithi090 ceph-mon[29180]: pgmap v17906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:37:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32552.487515 secs 2024-04-27T03:37:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32553.350459 secs 2024-04-27T03:37:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32553.350459 secs 2024-04-27T03:37:52.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32553.350459 secs 2024-04-27T03:37:53.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:52 smithi119 ceph-mon[32752]: pgmap v17907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:37:53.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:52 smithi167 ceph-mon[32751]: pgmap v17907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:37:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:52 smithi090 ceph-mon[29180]: pgmap v17907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:37:55.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:54 smithi119 ceph-mon[32752]: pgmap v17908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:55.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:54 smithi167 ceph-mon[32751]: pgmap v17908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:55.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:54 smithi090 ceph-mon[29180]: pgmap v17908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32557.487597 secs 2024-04-27T03:37:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32557.487597 secs 2024-04-27T03:37:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32557.487597 secs 2024-04-27T03:37:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:56 smithi119 ceph-mon[32752]: pgmap v17909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32558.350550 secs 2024-04-27T03:37:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:56 smithi167 ceph-mon[32751]: pgmap v17909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32558.350550 secs 2024-04-27T03:37:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:56 smithi090 ceph-mon[29180]: pgmap v17909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:37:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32558.350550 secs 2024-04-27T03:37:59.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:37:58 smithi119 ceph-mon[32752]: pgmap v17910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:59.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:37:58 smithi167 ceph-mon[32751]: pgmap v17910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:37:59.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:37:58 smithi090 ceph-mon[29180]: pgmap v17910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:00 smithi119 ceph-mon[32752]: pgmap v17911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32562.487682 secs 2024-04-27T03:38:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:00 smithi167 ceph-mon[32751]: pgmap v17911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32562.487682 secs 2024-04-27T03:38:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:00 smithi090 ceph-mon[29180]: pgmap v17911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32562.487682 secs 2024-04-27T03:38:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32563.350632 secs 2024-04-27T03:38:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32563.350632 secs 2024-04-27T03:38:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32563.350632 secs 2024-04-27T03:38:03.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:02 smithi119 ceph-mon[32752]: pgmap v17912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:03.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:02 smithi167 ceph-mon[32751]: pgmap v17912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:03.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:02 smithi090 ceph-mon[29180]: pgmap v17912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:05.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:04 smithi119 ceph-mon[32752]: pgmap v17913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:05.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:04 smithi167 ceph-mon[32751]: pgmap v17913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:05.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:04 smithi090 ceph-mon[29180]: pgmap v17913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32567.487763 secs 2024-04-27T03:38:06.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32567.487763 secs 2024-04-27T03:38:06.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32567.487763 secs 2024-04-27T03:38:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:06 smithi167 ceph-mon[32751]: pgmap v17914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32568.350721 secs 2024-04-27T03:38:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:06 smithi090 ceph-mon[29180]: pgmap v17914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32568.350721 secs 2024-04-27T03:38:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:06 smithi119 ceph-mon[32752]: pgmap v17914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32568.350721 secs 2024-04-27T03:38:09.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:08 smithi167 ceph-mon[32751]: pgmap v17915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:09.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:08 smithi090 ceph-mon[29180]: pgmap v17915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:09.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:08 smithi119 ceph-mon[32752]: pgmap v17915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:10 smithi167 ceph-mon[32751]: pgmap v17916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32572.487844 secs 2024-04-27T03:38:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:10 smithi090 ceph-mon[29180]: pgmap v17916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32572.487844 secs 2024-04-27T03:38:11.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:10 smithi119 ceph-mon[32752]: pgmap v17916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:11.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32572.487844 secs 2024-04-27T03:38:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:11 smithi090 ceph-mon[29180]: 30 slow requests, 1 included below; oldest blocked for > 32573.350839 secs 2024-04-27T03:38:12.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:11 smithi090 ceph-mon[29180]: slow request 2882.094269 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:38:12.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:11 smithi119 ceph-mon[32752]: 30 slow requests, 1 included below; oldest blocked for > 32573.350839 secs 2024-04-27T03:38:12.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:11 smithi119 ceph-mon[32752]: slow request 2882.094269 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:38:12.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:11 smithi167 ceph-mon[32751]: 30 slow requests, 1 included below; oldest blocked for > 32573.350839 secs 2024-04-27T03:38:12.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:11 smithi167 ceph-mon[32751]: slow request 2882.094269 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:38:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:12 smithi090 ceph-mon[29180]: pgmap v17917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:13.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:12 smithi119 ceph-mon[32752]: pgmap v17917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:13.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:12 smithi167 ceph-mon[32751]: pgmap v17917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:15.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:14 smithi090 ceph-mon[29180]: pgmap v17918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:15.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:14 smithi119 ceph-mon[32752]: pgmap v17918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:15.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:14 smithi167 ceph-mon[32751]: pgmap v17918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:16.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32577.487924 secs 2024-04-27T03:38:16.210 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32577.487924 secs 2024-04-27T03:38:16.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32577.487924 secs 2024-04-27T03:38:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:16 smithi090 ceph-mon[29180]: pgmap v17919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32578.350936 secs 2024-04-27T03:38:17.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:16 smithi119 ceph-mon[32752]: pgmap v17919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:17.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32578.350936 secs 2024-04-27T03:38:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:16 smithi167 ceph-mon[32751]: pgmap v17919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32578.350936 secs 2024-04-27T03:38:19.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:18 smithi090 ceph-mon[29180]: pgmap v17920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:19.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:18 smithi119 ceph-mon[32752]: pgmap v17920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:19.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:18 smithi167 ceph-mon[32751]: pgmap v17920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:21.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:20 smithi090 ceph-mon[29180]: pgmap v17921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:21.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32582.488008 secs 2024-04-27T03:38:21.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:20 smithi119 ceph-mon[32752]: pgmap v17921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:21.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32582.488008 secs 2024-04-27T03:38:21.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:20 smithi167 ceph-mon[32751]: pgmap v17921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:21.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32582.488008 secs 2024-04-27T03:38:22.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32583.351018 secs 2024-04-27T03:38:22.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32583.351018 secs 2024-04-27T03:38:22.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32583.351018 secs 2024-04-27T03:38:23.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:22 smithi090 ceph-mon[29180]: pgmap v17922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:23.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:22 smithi119 ceph-mon[32752]: pgmap v17922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:23.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:22 smithi167 ceph-mon[32751]: pgmap v17922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:25.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:24 smithi090 ceph-mon[29180]: pgmap v17923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:25.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:24 smithi119 ceph-mon[32752]: pgmap v17923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:25.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:24 smithi167 ceph-mon[32751]: pgmap v17923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:26.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32587.488066 secs 2024-04-27T03:38:26.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32587.488066 secs 2024-04-27T03:38:26.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32587.488066 secs 2024-04-27T03:38:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:26 smithi090 ceph-mon[29180]: pgmap v17924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32588.351174 secs 2024-04-27T03:38:27.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:26 smithi119 ceph-mon[32752]: pgmap v17924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:27.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32588.351174 secs 2024-04-27T03:38:27.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:26 smithi167 ceph-mon[32751]: pgmap v17924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:27.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32588.351174 secs 2024-04-27T03:38:29.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:28 smithi090 ceph-mon[29180]: pgmap v17925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:29.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:28 smithi119 ceph-mon[32752]: pgmap v17925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:29.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:28 smithi167 ceph-mon[32751]: pgmap v17925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:31.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:30 smithi090 ceph-mon[29180]: pgmap v17926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32592.488150 secs 2024-04-27T03:38:31.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:30 smithi119 ceph-mon[32752]: pgmap v17926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:31.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32592.488150 secs 2024-04-27T03:38:31.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:30 smithi167 ceph-mon[32751]: pgmap v17926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:31.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32592.488150 secs 2024-04-27T03:38:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32593.351235 secs 2024-04-27T03:38:32.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32593.351235 secs 2024-04-27T03:38:32.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32593.351235 secs 2024-04-27T03:38:33.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:32 smithi090 ceph-mon[29180]: pgmap v17927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:33.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:32 smithi119 ceph-mon[32752]: pgmap v17927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:33.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:32 smithi167 ceph-mon[32751]: pgmap v17927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:35.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:34 smithi090 ceph-mon[29180]: pgmap v17928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:35.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:34 smithi119 ceph-mon[32752]: pgmap v17928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:35.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:34 smithi167 ceph-mon[32751]: pgmap v17928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:36.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32597.488228 secs 2024-04-27T03:38:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32597.488228 secs 2024-04-27T03:38:36.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32597.488228 secs 2024-04-27T03:38:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:36 smithi119 ceph-mon[32752]: pgmap v17929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32598.351374 secs 2024-04-27T03:38:37.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:36 smithi167 ceph-mon[32751]: pgmap v17929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:37.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32598.351374 secs 2024-04-27T03:38:37.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:36 smithi090 ceph-mon[29180]: pgmap v17929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:37.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32598.351374 secs 2024-04-27T03:38:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:38 smithi119 ceph-mon[32752]: pgmap v17930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:38 smithi167 ceph-mon[32751]: pgmap v17930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:38 smithi090 ceph-mon[29180]: pgmap v17930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:41.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:40 smithi119 ceph-mon[32752]: pgmap v17931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:41.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32602.488315 secs 2024-04-27T03:38:41.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:40 smithi167 ceph-mon[32751]: pgmap v17931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:41.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32602.488315 secs 2024-04-27T03:38:41.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:40 smithi090 ceph-mon[29180]: pgmap v17931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:41.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32602.488315 secs 2024-04-27T03:38:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32603.351495 secs 2024-04-27T03:38:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:38:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:38:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:38:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:38:42.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32603.351495 secs 2024-04-27T03:38:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:38:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:38:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:38:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:38:42.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32603.351495 secs 2024-04-27T03:38:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:38:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:38:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:38:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:38:43.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:42 smithi119 ceph-mon[32752]: pgmap v17932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:43.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:42 smithi119 ceph-mon[32752]: pgmap v17933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:38:43.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:42 smithi119 ceph-mon[32752]: pgmap v17934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:43.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:42 smithi167 ceph-mon[32751]: pgmap v17932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:42 smithi167 ceph-mon[32751]: pgmap v17933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:38:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:42 smithi167 ceph-mon[32751]: pgmap v17934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:43.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:42 smithi090 ceph-mon[29180]: pgmap v17932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:42 smithi090 ceph-mon[29180]: pgmap v17933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:38:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:42 smithi090 ceph-mon[29180]: pgmap v17934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:45.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:44 smithi119 ceph-mon[32752]: pgmap v17935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:45.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:44 smithi167 ceph-mon[32751]: pgmap v17935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:45.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:44 smithi090 ceph-mon[29180]: pgmap v17935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:46.210 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32607.488373 secs 2024-04-27T03:38:46.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32607.488373 secs 2024-04-27T03:38:46.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32607.488373 secs 2024-04-27T03:38:47.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:46 smithi119 ceph-mon[32752]: pgmap v17936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:47.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32608.351618 secs 2024-04-27T03:38:47.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:46 smithi167 ceph-mon[32751]: pgmap v17936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:47.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32608.351618 secs 2024-04-27T03:38:47.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:46 smithi090 ceph-mon[29180]: pgmap v17936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:47.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32608.351618 secs 2024-04-27T03:38:48.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:47 smithi119 ceph-mon[32752]: pgmap v17937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:38:48.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:47 smithi167 ceph-mon[32751]: pgmap v17937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:38:48.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:47 smithi090 ceph-mon[29180]: pgmap v17937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:38:51.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:50 smithi090 ceph-mon[29180]: pgmap v17938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32612.488457 secs 2024-04-27T03:38:51.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:50 smithi119 ceph-mon[32752]: pgmap v17938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32612.488457 secs 2024-04-27T03:38:51.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:50 smithi167 ceph-mon[32751]: pgmap v17938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:38:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32612.488457 secs 2024-04-27T03:38:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32613.351765 secs 2024-04-27T03:38:52.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32613.351765 secs 2024-04-27T03:38:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32613.351765 secs 2024-04-27T03:38:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:52 smithi090 ceph-mon[29180]: pgmap v17939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:38:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:52 smithi119 ceph-mon[32752]: pgmap v17939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:38:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:52 smithi167 ceph-mon[32751]: pgmap v17939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:38:55.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:54 smithi119 ceph-mon[32752]: pgmap v17940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:55.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:54 smithi167 ceph-mon[32751]: pgmap v17940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:55.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:54 smithi090 ceph-mon[29180]: pgmap v17940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:38:56.283 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32617.488540 secs 2024-04-27T03:38:56.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32617.488540 secs 2024-04-27T03:38:56.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32617.488540 secs 2024-04-27T03:38:57.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:56 smithi090 ceph-mon[29180]: pgmap v17941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:57.135 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32618.351841 secs 2024-04-27T03:38:57.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:56 smithi119 ceph-mon[32752]: pgmap v17941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:57.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32618.351841 secs 2024-04-27T03:38:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:56 smithi167 ceph-mon[32751]: pgmap v17941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:57.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32618.351841 secs 2024-04-27T03:38:59.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:38:58 smithi119 ceph-mon[32752]: pgmap v17942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:59.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:38:58 smithi167 ceph-mon[32751]: pgmap v17942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:38:59.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:38:58 smithi090 ceph-mon[29180]: pgmap v17942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:01.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:00 smithi119 ceph-mon[32752]: pgmap v17943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32622.488619 secs 2024-04-27T03:39:01.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:00 smithi167 ceph-mon[32751]: pgmap v17943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:01.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32622.488619 secs 2024-04-27T03:39:01.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:00 smithi090 ceph-mon[29180]: pgmap v17943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:01.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32622.488619 secs 2024-04-27T03:39:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32623.351961 secs 2024-04-27T03:39:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32623.351961 secs 2024-04-27T03:39:02.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32623.351961 secs 2024-04-27T03:39:03.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:02 smithi119 ceph-mon[32752]: pgmap v17944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:02 smithi167 ceph-mon[32751]: pgmap v17944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:03.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:02 smithi090 ceph-mon[29180]: pgmap v17944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:05.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:04 smithi119 ceph-mon[32752]: pgmap v17945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:05.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:04 smithi167 ceph-mon[32751]: pgmap v17945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:05.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:04 smithi090 ceph-mon[29180]: pgmap v17945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:06.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32627.488701 secs 2024-04-27T03:39:06.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32627.488701 secs 2024-04-27T03:39:06.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32627.488701 secs 2024-04-27T03:39:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:06 smithi119 ceph-mon[32752]: pgmap v17946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32628.352102 secs 2024-04-27T03:39:07.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:06 smithi167 ceph-mon[32751]: pgmap v17946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:07.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32628.352102 secs 2024-04-27T03:39:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:06 smithi090 ceph-mon[29180]: pgmap v17946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:07.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32628.352102 secs 2024-04-27T03:39:08.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:07 smithi119 ceph-mon[32752]: pgmap v17947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:08.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:07 smithi167 ceph-mon[32751]: pgmap v17947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:08.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:07 smithi090 ceph-mon[29180]: pgmap v17947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:11.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:10 smithi090 ceph-mon[29180]: pgmap v17948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32632.488784 secs 2024-04-27T03:39:11.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:10 smithi119 ceph-mon[32752]: pgmap v17948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:11.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32632.488784 secs 2024-04-27T03:39:11.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:10 smithi167 ceph-mon[32751]: pgmap v17948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:11.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32632.488784 secs 2024-04-27T03:39:12.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32633.352221 secs 2024-04-27T03:39:12.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32633.352221 secs 2024-04-27T03:39:12.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32633.352221 secs 2024-04-27T03:39:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:12 smithi090 ceph-mon[29180]: pgmap v17949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:13.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:12 smithi119 ceph-mon[32752]: pgmap v17949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:13.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:12 smithi167 ceph-mon[32751]: pgmap v17949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:15.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:14 smithi119 ceph-mon[32752]: pgmap v17950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:15.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:14 smithi167 ceph-mon[32751]: pgmap v17950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:15.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:14 smithi090 ceph-mon[29180]: pgmap v17950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:16.210 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32637.488865 secs 2024-04-27T03:39:16.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32637.488865 secs 2024-04-27T03:39:16.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32637.488865 secs 2024-04-27T03:39:17.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:16 smithi119 ceph-mon[32752]: pgmap v17951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:17.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32638.352333 secs 2024-04-27T03:39:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:16 smithi167 ceph-mon[32751]: pgmap v17951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32638.352333 secs 2024-04-27T03:39:17.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:16 smithi090 ceph-mon[29180]: pgmap v17951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:17.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32638.352333 secs 2024-04-27T03:39:19.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:18 smithi119 ceph-mon[32752]: pgmap v17952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:19.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:18 smithi167 ceph-mon[32751]: pgmap v17952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:19.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:18 smithi090 ceph-mon[29180]: pgmap v17952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:21.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:20 smithi119 ceph-mon[32752]: pgmap v17953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:21.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32642.488927 secs 2024-04-27T03:39:21.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:20 smithi167 ceph-mon[32751]: pgmap v17953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:21.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32642.488927 secs 2024-04-27T03:39:21.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:20 smithi090 ceph-mon[29180]: pgmap v17953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:21.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32642.488927 secs 2024-04-27T03:39:22.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32643.352400 secs 2024-04-27T03:39:22.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32643.352400 secs 2024-04-27T03:39:22.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32643.352400 secs 2024-04-27T03:39:23.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:22 smithi119 ceph-mon[32752]: pgmap v17954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:23.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:22 smithi167 ceph-mon[32751]: pgmap v17954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:23.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:22 smithi090 ceph-mon[29180]: pgmap v17954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:24.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:23 smithi119 ceph-mon[32752]: pgmap v17955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:24.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:23 smithi167 ceph-mon[32751]: pgmap v17955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:24.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:23 smithi090 ceph-mon[29180]: pgmap v17955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:25.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32647.489009 secs 2024-04-27T03:39:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32647.489009 secs 2024-04-27T03:39:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32647.489009 secs 2024-04-27T03:39:26.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:26 smithi090 ceph-mon[29180]: pgmap v17956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:26.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32648.352495 secs 2024-04-27T03:39:26.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:26 smithi119 ceph-mon[32752]: pgmap v17956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:26.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32648.352495 secs 2024-04-27T03:39:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:26 smithi167 ceph-mon[32751]: pgmap v17956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32648.352495 secs 2024-04-27T03:39:29.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:28 smithi119 ceph-mon[32752]: pgmap v17957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:29.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:28 smithi167 ceph-mon[32751]: pgmap v17957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:29.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:28 smithi090 ceph-mon[29180]: pgmap v17957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:31.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:30 smithi119 ceph-mon[32752]: pgmap v17958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:31.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32652.489070 secs 2024-04-27T03:39:31.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:30 smithi167 ceph-mon[32751]: pgmap v17958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:31.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32652.489070 secs 2024-04-27T03:39:31.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:30 smithi090 ceph-mon[29180]: pgmap v17958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:31.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32652.489070 secs 2024-04-27T03:39:32.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32653.352618 secs 2024-04-27T03:39:32.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32653.352618 secs 2024-04-27T03:39:32.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32653.352618 secs 2024-04-27T03:39:33.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:32 smithi119 ceph-mon[32752]: pgmap v17959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:33.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:32 smithi167 ceph-mon[32751]: pgmap v17959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:33.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:32 smithi090 ceph-mon[29180]: pgmap v17959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:35.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:34 smithi119 ceph-mon[32752]: pgmap v17960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:35.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:34 smithi167 ceph-mon[32751]: pgmap v17960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:35.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:34 smithi090 ceph-mon[29180]: pgmap v17960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:36.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32657.489129 secs 2024-04-27T03:39:36.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32657.489129 secs 2024-04-27T03:39:36.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32657.489129 secs 2024-04-27T03:39:37.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:36 smithi119 ceph-mon[32752]: pgmap v17961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:37.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32658.352736 secs 2024-04-27T03:39:37.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:36 smithi167 ceph-mon[32751]: pgmap v17961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:37.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32658.352736 secs 2024-04-27T03:39:37.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:36 smithi090 ceph-mon[29180]: pgmap v17961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:37.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32658.352736 secs 2024-04-27T03:39:39.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:38 smithi119 ceph-mon[32752]: pgmap v17962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:39.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:38 smithi167 ceph-mon[32751]: pgmap v17962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:39.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:38 smithi090 ceph-mon[29180]: pgmap v17962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:41.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:40 smithi119 ceph-mon[32752]: pgmap v17963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:41.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32662.489214 secs 2024-04-27T03:39:41.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:40 smithi167 ceph-mon[32751]: pgmap v17963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:41.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32662.489214 secs 2024-04-27T03:39:41.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:40 smithi090 ceph-mon[29180]: pgmap v17963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:41.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32662.489214 secs 2024-04-27T03:39:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32663.352818 secs 2024-04-27T03:39:42.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:41 smithi119 ceph-mon[32752]: pgmap v17964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:42.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32663.352818 secs 2024-04-27T03:39:42.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:41 smithi167 ceph-mon[32751]: pgmap v17964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:42.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32663.352818 secs 2024-04-27T03:39:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:41 smithi090 ceph-mon[29180]: pgmap v17964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:43.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:39:43.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:39:43.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:39:43.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:42 smithi119 ceph-mon[32752]: pgmap v17965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:39:43.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:42 smithi119 ceph-mon[32752]: pgmap v17966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:39:43.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:39:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:39:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:39:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:39:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:42 smithi167 ceph-mon[32751]: pgmap v17965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:39:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:42 smithi167 ceph-mon[32751]: pgmap v17966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:39:43.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:39:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:39:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:39:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:39:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:42 smithi090 ceph-mon[29180]: pgmap v17965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:39:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:42 smithi090 ceph-mon[29180]: pgmap v17966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:39:43.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:39:46.012 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:45 smithi119 ceph-mon[32752]: pgmap v17967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:46.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32667.489296 secs 2024-04-27T03:39:46.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:45 smithi167 ceph-mon[32751]: pgmap v17967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:46.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32667.489296 secs 2024-04-27T03:39:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:45 smithi090 ceph-mon[29180]: pgmap v17967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32667.489296 secs 2024-04-27T03:39:47.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32668.352957 secs 2024-04-27T03:39:47.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32668.352957 secs 2024-04-27T03:39:47.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32668.352957 secs 2024-04-27T03:39:48.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:47 smithi119 ceph-mon[32752]: pgmap v17968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:48.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:47 smithi167 ceph-mon[32751]: pgmap v17968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:48.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:47 smithi090 ceph-mon[29180]: pgmap v17968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:50.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:49 smithi119 ceph-mon[32752]: pgmap v17969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:50.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:49 smithi167 ceph-mon[32751]: pgmap v17969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:50.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:49 smithi090 ceph-mon[29180]: pgmap v17969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32672.489355 secs 2024-04-27T03:39:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32672.489355 secs 2024-04-27T03:39:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32672.489355 secs 2024-04-27T03:39:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:51 smithi119 ceph-mon[32752]: pgmap v17970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32673.353086 secs 2024-04-27T03:39:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:51 smithi167 ceph-mon[32751]: pgmap v17970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32673.353086 secs 2024-04-27T03:39:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:51 smithi090 ceph-mon[29180]: pgmap v17970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:39:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32673.353086 secs 2024-04-27T03:39:54.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:53 smithi119 ceph-mon[32752]: pgmap v17971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:39:54.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:53 smithi167 ceph-mon[32751]: pgmap v17971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:39:54.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:53 smithi090 ceph-mon[29180]: pgmap v17971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:39:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:55 smithi119 ceph-mon[32752]: pgmap v17972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32677.489438 secs 2024-04-27T03:39:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:55 smithi167 ceph-mon[32751]: pgmap v17972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32677.489438 secs 2024-04-27T03:39:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:55 smithi090 ceph-mon[29180]: pgmap v17972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:39:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32677.489438 secs 2024-04-27T03:39:57.032 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32678.353230 secs 2024-04-27T03:39:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32678.353230 secs 2024-04-27T03:39:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32678.353230 secs 2024-04-27T03:39:58.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:57 smithi119 ceph-mon[32752]: pgmap v17973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:58.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:57 smithi167 ceph-mon[32751]: pgmap v17973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:39:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:57 smithi090 ceph-mon[29180]: pgmap v17973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:00.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:39:59 smithi119 ceph-mon[32752]: pgmap v17974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:00.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:39:59 smithi167 ceph-mon[32751]: pgmap v17974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:00.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:39:59 smithi090 ceph-mon[29180]: pgmap v17974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:40:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32682.489519 secs 2024-04-27T03:40:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:40:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32682.489519 secs 2024-04-27T03:40:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:40:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32682.489519 secs 2024-04-27T03:40:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:01 smithi119 ceph-mon[32752]: pgmap v17975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:02.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32683.353347 secs 2024-04-27T03:40:02.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:01 smithi167 ceph-mon[32751]: pgmap v17975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:02.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32683.353347 secs 2024-04-27T03:40:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:01 smithi090 ceph-mon[29180]: pgmap v17975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32683.353347 secs 2024-04-27T03:40:04.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:03 smithi119 ceph-mon[32752]: pgmap v17976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:04.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:03 smithi167 ceph-mon[32751]: pgmap v17976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:04.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:03 smithi090 ceph-mon[29180]: pgmap v17976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:06.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:05 smithi119 ceph-mon[32752]: pgmap v17977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:05 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32687.489598 secs 2024-04-27T03:40:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:05 smithi119 ceph-mon[32752]: slow request 23041.897842 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:40:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:05 smithi167 ceph-mon[32751]: pgmap v17977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:05 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32687.489598 secs 2024-04-27T03:40:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:05 smithi167 ceph-mon[32751]: slow request 23041.897842 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:40:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:05 smithi090 ceph-mon[29180]: pgmap v17977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:05 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32687.489598 secs 2024-04-27T03:40:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:05 smithi090 ceph-mon[29180]: slow request 23041.897842 seconds old, received at 2024-04-26T21:16:03.345160+0000: client_request(client.17247:1 getattr pAsLsXsFs #0x1 2024-04-26T21:16:03.341214+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:40:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32688.353482 secs 2024-04-27T03:40:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32688.353482 secs 2024-04-27T03:40:07.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32688.353482 secs 2024-04-27T03:40:08.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:07 smithi119 ceph-mon[32752]: pgmap v17978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:08.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:07 smithi167 ceph-mon[32751]: pgmap v17978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:08.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:07 smithi090 ceph-mon[29180]: pgmap v17978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:10.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:09 smithi119 ceph-mon[32752]: pgmap v17979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:10.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:09 smithi167 ceph-mon[32751]: pgmap v17979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:10.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:09 smithi090 ceph-mon[29180]: pgmap v17979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:11.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:10 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32692.489682 secs 2024-04-27T03:40:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:10 smithi119 ceph-mon[32752]: slow request 1440.822092 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:40:11.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:10 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32692.489682 secs 2024-04-27T03:40:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:10 smithi167 ceph-mon[32751]: slow request 1440.822092 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:40:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:10 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32692.489682 secs 2024-04-27T03:40:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:10 smithi090 ceph-mon[29180]: slow request 1440.822092 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:40:12.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:11 smithi119 ceph-mon[32752]: pgmap v17980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:12.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32693.353600 secs 2024-04-27T03:40:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:11 smithi167 ceph-mon[32751]: pgmap v17980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:12.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32693.353600 secs 2024-04-27T03:40:12.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:11 smithi090 ceph-mon[29180]: pgmap v17980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:12.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32693.353600 secs 2024-04-27T03:40:14.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:13 smithi119 ceph-mon[32752]: pgmap v17981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:14.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:13 smithi167 ceph-mon[32751]: pgmap v17981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:14.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:13 smithi090 ceph-mon[29180]: pgmap v17981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:16.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:15 smithi119 ceph-mon[32752]: pgmap v17982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32697.489765 secs 2024-04-27T03:40:16.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:15 smithi167 ceph-mon[32751]: pgmap v17982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32697.489765 secs 2024-04-27T03:40:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:15 smithi090 ceph-mon[29180]: pgmap v17982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32697.489765 secs 2024-04-27T03:40:16.991 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32698.353691 secs 2024-04-27T03:40:17.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32698.353691 secs 2024-04-27T03:40:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32698.353691 secs 2024-04-27T03:40:18.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:17 smithi119 ceph-mon[32752]: pgmap v17983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:18.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:17 smithi167 ceph-mon[32751]: pgmap v17983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:18.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:17 smithi090 ceph-mon[29180]: pgmap v17983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:20.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:19 smithi119 ceph-mon[32752]: pgmap v17984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:20.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:19 smithi167 ceph-mon[32751]: pgmap v17984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:20.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:19 smithi090 ceph-mon[29180]: pgmap v17984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:21.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32702.489825 secs 2024-04-27T03:40:21.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32702.489825 secs 2024-04-27T03:40:21.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32702.489825 secs 2024-04-27T03:40:22.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:21 smithi119 ceph-mon[32752]: pgmap v17985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:22.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32703.353831 secs 2024-04-27T03:40:22.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:21 smithi167 ceph-mon[32751]: pgmap v17985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32703.353831 secs 2024-04-27T03:40:22.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:21 smithi090 ceph-mon[29180]: pgmap v17985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:22.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32703.353831 secs 2024-04-27T03:40:24.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:23 smithi119 ceph-mon[32752]: pgmap v17986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:24.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:23 smithi167 ceph-mon[32751]: pgmap v17986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:24.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:23 smithi090 ceph-mon[29180]: pgmap v17986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:25 smithi119 ceph-mon[32752]: pgmap v17987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32707.489885 secs 2024-04-27T03:40:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:25 smithi167 ceph-mon[32751]: pgmap v17987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32707.489885 secs 2024-04-27T03:40:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:25 smithi090 ceph-mon[29180]: pgmap v17987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32707.489885 secs 2024-04-27T03:40:27.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32708.353947 secs 2024-04-27T03:40:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32708.353947 secs 2024-04-27T03:40:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32708.353947 secs 2024-04-27T03:40:28.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:27 smithi119 ceph-mon[32752]: pgmap v17988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:28.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:27 smithi167 ceph-mon[32751]: pgmap v17988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:28.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:27 smithi090 ceph-mon[29180]: pgmap v17988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:29.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:29 smithi090 ceph-mon[29180]: pgmap v17989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:30.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:29 smithi119 ceph-mon[32752]: pgmap v17989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:30.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:29 smithi167 ceph-mon[32751]: pgmap v17989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32712.489971 secs 2024-04-27T03:40:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32712.489971 secs 2024-04-27T03:40:31.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32712.489971 secs 2024-04-27T03:40:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:31 smithi119 ceph-mon[32752]: pgmap v17990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:32.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32713.354060 secs 2024-04-27T03:40:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:31 smithi167 ceph-mon[32751]: pgmap v17990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32713.354060 secs 2024-04-27T03:40:32.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:31 smithi090 ceph-mon[29180]: pgmap v17990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:32.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32713.354060 secs 2024-04-27T03:40:34.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:33 smithi119 ceph-mon[32752]: pgmap v17991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:34.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:33 smithi167 ceph-mon[32751]: pgmap v17991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:34.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:33 smithi090 ceph-mon[29180]: pgmap v17991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:35 smithi119 ceph-mon[32752]: pgmap v17992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32717.490032 secs 2024-04-27T03:40:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:35 smithi167 ceph-mon[32751]: pgmap v17992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32717.490032 secs 2024-04-27T03:40:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:35 smithi090 ceph-mon[29180]: pgmap v17992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32717.490032 secs 2024-04-27T03:40:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32718.354176 secs 2024-04-27T03:40:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32718.354176 secs 2024-04-27T03:40:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32718.354176 secs 2024-04-27T03:40:38.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:37 smithi119 ceph-mon[32752]: pgmap v17993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:38.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:37 smithi167 ceph-mon[32751]: pgmap v17993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:37 smithi090 ceph-mon[29180]: pgmap v17993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:40.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:39 smithi119 ceph-mon[32752]: pgmap v17994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:40.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:39 smithi167 ceph-mon[32751]: pgmap v17994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:40.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:39 smithi090 ceph-mon[29180]: pgmap v17994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:40:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32722.490118 secs 2024-04-27T03:40:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32722.490118 secs 2024-04-27T03:40:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32722.490118 secs 2024-04-27T03:40:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:41 smithi119 ceph-mon[32752]: pgmap v17995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32723.354313 secs 2024-04-27T03:40:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:41 smithi167 ceph-mon[32751]: pgmap v17995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32723.354313 secs 2024-04-27T03:40:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:41 smithi090 ceph-mon[29180]: pgmap v17995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32723.354313 secs 2024-04-27T03:40:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:40:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:40:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:40:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:43 smithi167 ceph-mon[32751]: pgmap v17996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:40:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:40:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:40:44.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:43 smithi090 ceph-mon[29180]: pgmap v17996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:44.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:40:44.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:40:44.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:40:44.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:43 smithi119 ceph-mon[32752]: pgmap v17996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:44.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:40:44.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:40:44.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:40:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:44 smithi167 ceph-mon[32751]: pgmap v17997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:40:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:44 smithi167 ceph-mon[32751]: pgmap v17998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:45.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:44 smithi090 ceph-mon[29180]: pgmap v17997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:40:45.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:44 smithi090 ceph-mon[29180]: pgmap v17998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:45.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:44 smithi119 ceph-mon[32752]: pgmap v17997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:40:45.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:44 smithi119 ceph-mon[32752]: pgmap v17998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:46.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:46 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32727.490194 secs 2024-04-27T03:40:46.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:46 smithi090 ceph-mon[29180]: pgmap v17999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:46.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:46 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32727.490194 secs 2024-04-27T03:40:46.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:46 smithi119 ceph-mon[32752]: pgmap v17999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:46.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:46 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32727.490194 secs 2024-04-27T03:40:46.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:46 smithi167 ceph-mon[32751]: pgmap v17999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:47.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:47 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32728.354469 secs 2024-04-27T03:40:47.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:47 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32728.354469 secs 2024-04-27T03:40:47.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:47 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32728.354469 secs 2024-04-27T03:40:48.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:48 smithi090 ceph-mon[29180]: pgmap v18000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T03:40:48.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:48 smithi119 ceph-mon[32752]: pgmap v18000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T03:40:48.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:48 smithi167 ceph-mon[32751]: pgmap v18000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T03:40:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:50 smithi119 ceph-mon[32752]: pgmap v18001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:50.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:50 smithi167 ceph-mon[32751]: pgmap v18001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:50 smithi090 ceph-mon[29180]: pgmap v18001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:51.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:51 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32732.490270 secs 2024-04-27T03:40:51.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32733.354593 secs 2024-04-27T03:40:51.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:51 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32732.490270 secs 2024-04-27T03:40:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32733.354593 secs 2024-04-27T03:40:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:51 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32732.490270 secs 2024-04-27T03:40:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32733.354593 secs 2024-04-27T03:40:52.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:52 smithi119 ceph-mon[32752]: pgmap v18002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:52.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:52 smithi167 ceph-mon[32751]: pgmap v18002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:52 smithi090 ceph-mon[29180]: pgmap v18002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:40:54.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:54 smithi119 ceph-mon[32752]: pgmap v18003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:40:54.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:54 smithi167 ceph-mon[32751]: pgmap v18003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:40:54.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:54 smithi090 ceph-mon[29180]: pgmap v18003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:40:55.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32737.490327 secs 2024-04-27T03:40:55.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32737.490327 secs 2024-04-27T03:40:55.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32737.490327 secs 2024-04-27T03:40:56.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:56 smithi119 ceph-mon[32752]: pgmap v18004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32738.354725 secs 2024-04-27T03:40:56.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:56 smithi167 ceph-mon[32751]: pgmap v18004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32738.354725 secs 2024-04-27T03:40:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:56 smithi090 ceph-mon[29180]: pgmap v18004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32738.354725 secs 2024-04-27T03:40:58.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:40:58 smithi119 ceph-mon[32752]: pgmap v18005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:58.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:40:58 smithi167 ceph-mon[32751]: pgmap v18005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:40:58.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:40:58 smithi090 ceph-mon[29180]: pgmap v18005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:00.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:00 smithi090 ceph-mon[29180]: pgmap v18006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:00.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32742.490410 secs 2024-04-27T03:41:00.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:00 smithi119 ceph-mon[32752]: pgmap v18006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:00.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32742.490410 secs 2024-04-27T03:41:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:00 smithi167 ceph-mon[32751]: pgmap v18006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32742.490410 secs 2024-04-27T03:41:01.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32743.354839 secs 2024-04-27T03:41:01.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32743.354839 secs 2024-04-27T03:41:01.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32743.354839 secs 2024-04-27T03:41:02.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:02 smithi119 ceph-mon[32752]: pgmap v18007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:02.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:02 smithi167 ceph-mon[32751]: pgmap v18007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:02.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:02 smithi090 ceph-mon[29180]: pgmap v18007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:04.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:04 smithi119 ceph-mon[32752]: pgmap v18008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:04.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:04 smithi167 ceph-mon[32751]: pgmap v18008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:04.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:04 smithi090 ceph-mon[29180]: pgmap v18008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:05.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32747.490491 secs 2024-04-27T03:41:05.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32747.490491 secs 2024-04-27T03:41:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32747.490491 secs 2024-04-27T03:41:06.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:06 smithi119 ceph-mon[32752]: pgmap v18009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:06.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32748.354993 secs 2024-04-27T03:41:06.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:06 smithi167 ceph-mon[32751]: pgmap v18009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:06.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32748.354993 secs 2024-04-27T03:41:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:06 smithi090 ceph-mon[29180]: pgmap v18009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32748.354993 secs 2024-04-27T03:41:08.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:08 smithi119 ceph-mon[32752]: pgmap v18010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:08.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:08 smithi167 ceph-mon[32751]: pgmap v18010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:08.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:08 smithi090 ceph-mon[29180]: pgmap v18010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:10.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:10 smithi119 ceph-mon[32752]: pgmap v18011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32752.490583 secs 2024-04-27T03:41:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:10 smithi167 ceph-mon[32751]: pgmap v18011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32752.490583 secs 2024-04-27T03:41:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:10 smithi090 ceph-mon[29180]: pgmap v18011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32752.490583 secs 2024-04-27T03:41:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:11 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32753.355134 secs 2024-04-27T03:41:11.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:11 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32753.355134 secs 2024-04-27T03:41:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:11 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32753.355134 secs 2024-04-27T03:41:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:12 smithi119 ceph-mon[32752]: pgmap v18012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:12.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:12 smithi167 ceph-mon[32751]: pgmap v18012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:12 smithi090 ceph-mon[29180]: pgmap v18012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:14.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:14 smithi119 ceph-mon[32752]: pgmap v18013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:14.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:14 smithi167 ceph-mon[32751]: pgmap v18013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:14.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:14 smithi090 ceph-mon[29180]: pgmap v18013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:15.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32757.490665 secs 2024-04-27T03:41:15.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32757.490665 secs 2024-04-27T03:41:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32757.490665 secs 2024-04-27T03:41:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:16 smithi119 ceph-mon[32752]: pgmap v18014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32758.355262 secs 2024-04-27T03:41:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:16 smithi167 ceph-mon[32751]: pgmap v18014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32758.355262 secs 2024-04-27T03:41:16.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:16 smithi090 ceph-mon[29180]: pgmap v18014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32758.355262 secs 2024-04-27T03:41:18.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:18 smithi119 ceph-mon[32752]: pgmap v18015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:18.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:18 smithi167 ceph-mon[32751]: pgmap v18015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:18.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:18 smithi090 ceph-mon[29180]: pgmap v18015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:20 smithi119 ceph-mon[32752]: pgmap v18016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:20.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32762.490747 secs 2024-04-27T03:41:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:20 smithi167 ceph-mon[32751]: pgmap v18016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32762.490747 secs 2024-04-27T03:41:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:20 smithi090 ceph-mon[29180]: pgmap v18016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32762.490747 secs 2024-04-27T03:41:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32763.355408 secs 2024-04-27T03:41:21.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32763.355408 secs 2024-04-27T03:41:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32763.355408 secs 2024-04-27T03:41:22.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:22 smithi119 ceph-mon[32752]: pgmap v18017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:22.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:22 smithi167 ceph-mon[32751]: pgmap v18017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:22 smithi090 ceph-mon[29180]: pgmap v18017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:24.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:24 smithi119 ceph-mon[32752]: pgmap v18018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:24.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:24 smithi167 ceph-mon[32751]: pgmap v18018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:24.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:24 smithi090 ceph-mon[29180]: pgmap v18018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:25.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32767.490825 secs 2024-04-27T03:41:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32767.490825 secs 2024-04-27T03:41:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32767.490825 secs 2024-04-27T03:41:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:26 smithi167 ceph-mon[32751]: pgmap v18019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32768.355447 secs 2024-04-27T03:41:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:26 smithi090 ceph-mon[29180]: pgmap v18019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32768.355447 secs 2024-04-27T03:41:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:26 smithi119 ceph-mon[32752]: pgmap v18019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32768.355447 secs 2024-04-27T03:41:28.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:28 smithi167 ceph-mon[32751]: pgmap v18020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:28.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:28 smithi090 ceph-mon[29180]: pgmap v18020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:28 smithi119 ceph-mon[32752]: pgmap v18020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:30 smithi167 ceph-mon[32751]: pgmap v18021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32772.490914 secs 2024-04-27T03:41:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:30 smithi090 ceph-mon[29180]: pgmap v18021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32772.490914 secs 2024-04-27T03:41:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:30 smithi119 ceph-mon[32752]: pgmap v18021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32772.490914 secs 2024-04-27T03:41:31.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32773.355568 secs 2024-04-27T03:41:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32773.355568 secs 2024-04-27T03:41:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32773.355568 secs 2024-04-27T03:41:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:32 smithi090 ceph-mon[29180]: pgmap v18022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:32 smithi119 ceph-mon[32752]: pgmap v18022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:32 smithi167 ceph-mon[32751]: pgmap v18022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:34.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:34 smithi090 ceph-mon[29180]: pgmap v18023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:34 smithi119 ceph-mon[32752]: pgmap v18023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:34 smithi167 ceph-mon[32751]: pgmap v18023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:35.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32777.490975 secs 2024-04-27T03:41:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32777.490975 secs 2024-04-27T03:41:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32777.490975 secs 2024-04-27T03:41:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:36 smithi090 ceph-mon[29180]: pgmap v18024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32778.355664 secs 2024-04-27T03:41:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:36 smithi119 ceph-mon[32752]: pgmap v18024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:37.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32778.355664 secs 2024-04-27T03:41:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:36 smithi167 ceph-mon[32751]: pgmap v18024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32778.355664 secs 2024-04-27T03:41:38.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:38 smithi090 ceph-mon[29180]: pgmap v18025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:39.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:38 smithi119 ceph-mon[32752]: pgmap v18025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:39.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:38 smithi167 ceph-mon[32751]: pgmap v18025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:40 smithi090 ceph-mon[29180]: pgmap v18026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32782.491037 secs 2024-04-27T03:41:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:40 smithi119 ceph-mon[32752]: pgmap v18026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32782.491037 secs 2024-04-27T03:41:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:40 smithi167 ceph-mon[32751]: pgmap v18026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32782.491037 secs 2024-04-27T03:41:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32783.355762 secs 2024-04-27T03:41:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32783.355762 secs 2024-04-27T03:41:42.117 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32783.355762 secs 2024-04-27T03:41:42.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:42 smithi090 ceph-mon[29180]: pgmap v18027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:42 smithi119 ceph-mon[32752]: pgmap v18027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:42 smithi167 ceph-mon[32751]: pgmap v18027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:43.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:41:44.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:41:44.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:41:44.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:44 smithi090 ceph-mon[29180]: pgmap v18028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:44.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:41:44.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:41:44.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:41:45.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:44 smithi119 ceph-mon[32752]: pgmap v18028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:45.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:41:45.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:41:45.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:41:45.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:44 smithi167 ceph-mon[32751]: pgmap v18028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:41:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:41:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:41:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:41:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:45 smithi090 ceph-mon[29180]: pgmap v18029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:41:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:45 smithi090 ceph-mon[29180]: pgmap v18030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32787.491119 secs 2024-04-27T03:41:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:45 smithi119 ceph-mon[32752]: pgmap v18029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:41:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:45 smithi119 ceph-mon[32752]: pgmap v18030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32787.491119 secs 2024-04-27T03:41:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:45 smithi167 ceph-mon[32751]: pgmap v18029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:41:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:45 smithi167 ceph-mon[32751]: pgmap v18030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32787.491119 secs 2024-04-27T03:41:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32788.355897 secs 2024-04-27T03:41:46.961 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32788.355897 secs 2024-04-27T03:41:46.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32788.355897 secs 2024-04-27T03:41:47.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:47 smithi090 ceph-mon[29180]: pgmap v18031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:48.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:47 smithi119 ceph-mon[32752]: pgmap v18031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:48.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:47 smithi167 ceph-mon[32751]: pgmap v18031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:49.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:49 smithi090 ceph-mon[29180]: pgmap v18032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:41:50.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:49 smithi119 ceph-mon[32752]: pgmap v18032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:41:50.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:49 smithi167 ceph-mon[32751]: pgmap v18032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:41:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32792.491207 secs 2024-04-27T03:41:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32792.491207 secs 2024-04-27T03:41:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32792.491207 secs 2024-04-27T03:41:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:51 smithi090 ceph-mon[29180]: pgmap v18033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32793.356043 secs 2024-04-27T03:41:52.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:51 smithi119 ceph-mon[32752]: pgmap v18033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32793.356043 secs 2024-04-27T03:41:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:51 smithi167 ceph-mon[32751]: pgmap v18033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:41:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32793.356043 secs 2024-04-27T03:41:54.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:53 smithi119 ceph-mon[32752]: pgmap v18034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:41:54.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:53 smithi167 ceph-mon[32751]: pgmap v18034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:41:54.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:53 smithi090 ceph-mon[29180]: pgmap v18034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:41:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:55 smithi119 ceph-mon[32752]: pgmap v18035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:41:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32797.491288 secs 2024-04-27T03:41:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:55 smithi167 ceph-mon[32751]: pgmap v18035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:41:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32797.491288 secs 2024-04-27T03:41:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:55 smithi090 ceph-mon[29180]: pgmap v18035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:41:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32797.491288 secs 2024-04-27T03:41:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32798.356157 secs 2024-04-27T03:41:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32798.356157 secs 2024-04-27T03:41:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32798.356157 secs 2024-04-27T03:41:58.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:57 smithi090 ceph-mon[29180]: pgmap v18036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:58.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:57 smithi119 ceph-mon[32752]: pgmap v18036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:41:58.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:57 smithi167 ceph-mon[32751]: pgmap v18036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:00.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:41:59 smithi119 ceph-mon[32752]: pgmap v18037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:00.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:41:59 smithi167 ceph-mon[32751]: pgmap v18037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:00.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:41:59 smithi090 ceph-mon[29180]: pgmap v18037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32802.491370 secs 2024-04-27T03:42:01.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32802.491370 secs 2024-04-27T03:42:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32802.491370 secs 2024-04-27T03:42:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:01 smithi119 ceph-mon[32752]: pgmap v18038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:02.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32803.356285 secs 2024-04-27T03:42:02.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:01 smithi167 ceph-mon[32751]: pgmap v18038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:02.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32803.356285 secs 2024-04-27T03:42:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:01 smithi090 ceph-mon[29180]: pgmap v18038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32803.356285 secs 2024-04-27T03:42:04.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:03 smithi119 ceph-mon[32752]: pgmap v18039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:04.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:03 smithi167 ceph-mon[32751]: pgmap v18039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:04.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:03 smithi090 ceph-mon[29180]: pgmap v18039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:05 smithi119 ceph-mon[32752]: pgmap v18040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32807.491452 secs 2024-04-27T03:42:06.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:05 smithi167 ceph-mon[32751]: pgmap v18040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32807.491452 secs 2024-04-27T03:42:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:05 smithi090 ceph-mon[29180]: pgmap v18040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32807.491452 secs 2024-04-27T03:42:07.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32808.356395 secs 2024-04-27T03:42:07.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32808.356395 secs 2024-04-27T03:42:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32808.356395 secs 2024-04-27T03:42:08.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:07 smithi119 ceph-mon[32752]: pgmap v18041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:08.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:07 smithi167 ceph-mon[32751]: pgmap v18041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:08.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:07 smithi090 ceph-mon[29180]: pgmap v18041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:10.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:09 smithi119 ceph-mon[32752]: pgmap v18042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:10.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:09 smithi167 ceph-mon[32751]: pgmap v18042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:10.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:09 smithi090 ceph-mon[29180]: pgmap v18042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:11.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:11 smithi090 ceph-mon[29180]: pgmap v18043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:11.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:11 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32812.491538 secs 2024-04-27T03:42:11.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:11 smithi119 ceph-mon[32752]: pgmap v18043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:11.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:11 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32812.491538 secs 2024-04-27T03:42:11.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:11 smithi167 ceph-mon[32751]: pgmap v18043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:11.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:11 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32812.491538 secs 2024-04-27T03:42:12.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:12 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32813.356513 secs 2024-04-27T03:42:12.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:12 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32813.356513 secs 2024-04-27T03:42:12.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:12 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32813.356513 secs 2024-04-27T03:42:13.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:13 smithi090 ceph-mon[29180]: pgmap v18044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:13.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:13 smithi119 ceph-mon[32752]: pgmap v18044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:13.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:13 smithi167 ceph-mon[32751]: pgmap v18044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:15.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:15 smithi119 ceph-mon[32752]: pgmap v18045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:15.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:15 smithi167 ceph-mon[32751]: pgmap v18045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:15.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:15 smithi090 ceph-mon[29180]: pgmap v18045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:16.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:16 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32817.491619 secs 2024-04-27T03:42:16.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:16 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32818.356577 secs 2024-04-27T03:42:16.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:16 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32817.491619 secs 2024-04-27T03:42:16.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:16 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32818.356577 secs 2024-04-27T03:42:16.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:16 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32817.491619 secs 2024-04-27T03:42:16.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:16 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32818.356577 secs 2024-04-27T03:42:17.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:17 smithi119 ceph-mon[32752]: pgmap v18046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:17.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:17 smithi167 ceph-mon[32751]: pgmap v18046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:17.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:17 smithi090 ceph-mon[29180]: pgmap v18046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:19.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:19 smithi119 ceph-mon[32752]: pgmap v18047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:19.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:19 smithi167 ceph-mon[32751]: pgmap v18047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:19.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:19 smithi090 ceph-mon[29180]: pgmap v18047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:21.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:21 smithi119 ceph-mon[32752]: pgmap v18048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:21.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:21 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32822.491695 secs 2024-04-27T03:42:21.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:21 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32823.356695 secs 2024-04-27T03:42:21.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:21 smithi167 ceph-mon[32751]: pgmap v18048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:21.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:21 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32822.491695 secs 2024-04-27T03:42:21.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:21 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32823.356695 secs 2024-04-27T03:42:21.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:21 smithi090 ceph-mon[29180]: pgmap v18048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:21.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:21 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32822.491695 secs 2024-04-27T03:42:21.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:21 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32823.356695 secs 2024-04-27T03:42:23.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:23 smithi119 ceph-mon[32752]: pgmap v18049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:23.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:23 smithi167 ceph-mon[32751]: pgmap v18049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:23.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:23 smithi090 ceph-mon[29180]: pgmap v18049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:25.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:25 smithi119 ceph-mon[32752]: pgmap v18050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:25.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:25 smithi167 ceph-mon[32751]: pgmap v18050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:25.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:25 smithi090 ceph-mon[29180]: pgmap v18050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:26.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:26 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32827.491777 secs 2024-04-27T03:42:26.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:26 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32828.356805 secs 2024-04-27T03:42:26.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:26 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32827.491777 secs 2024-04-27T03:42:26.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:26 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32828.356805 secs 2024-04-27T03:42:26.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:26 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32827.491777 secs 2024-04-27T03:42:26.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:26 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32828.356805 secs 2024-04-27T03:42:27.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:27 smithi119 ceph-mon[32752]: pgmap v18051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:27.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:27 smithi167 ceph-mon[32751]: pgmap v18051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:27.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:27 smithi090 ceph-mon[29180]: pgmap v18051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:29.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:29 smithi119 ceph-mon[32752]: pgmap v18052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:29.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:29 smithi167 ceph-mon[32751]: pgmap v18052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:29.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:29 smithi090 ceph-mon[29180]: pgmap v18052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:31 smithi119 ceph-mon[32752]: pgmap v18053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:31 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32832.491866 secs 2024-04-27T03:42:31.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:31 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32833.356918 secs 2024-04-27T03:42:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:31 smithi167 ceph-mon[32751]: pgmap v18053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:31 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32832.491866 secs 2024-04-27T03:42:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:31 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32833.356918 secs 2024-04-27T03:42:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:31 smithi090 ceph-mon[29180]: pgmap v18053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:31 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32832.491866 secs 2024-04-27T03:42:31.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:31 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32833.356918 secs 2024-04-27T03:42:33.562 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:33 smithi119 ceph-mon[32752]: pgmap v18054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:33.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:33 smithi167 ceph-mon[32751]: pgmap v18054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:33.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:33 smithi090 ceph-mon[29180]: pgmap v18054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:35.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:35 smithi119 ceph-mon[32752]: pgmap v18055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:35.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:35 smithi167 ceph-mon[32751]: pgmap v18055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:35.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:35 smithi090 ceph-mon[29180]: pgmap v18055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:36.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:36 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32837.491952 secs 2024-04-27T03:42:36.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:36 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32838.357048 secs 2024-04-27T03:42:36.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:36 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32837.491952 secs 2024-04-27T03:42:36.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:36 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32838.357048 secs 2024-04-27T03:42:36.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:36 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32837.491952 secs 2024-04-27T03:42:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:36 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32838.357048 secs 2024-04-27T03:42:37.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:37 smithi167 ceph-mon[32751]: pgmap v18056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:37.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:37 smithi090 ceph-mon[29180]: pgmap v18056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:37.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:37 smithi119 ceph-mon[32752]: pgmap v18056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:39.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:39 smithi167 ceph-mon[32751]: pgmap v18057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:39.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:39 smithi090 ceph-mon[29180]: pgmap v18057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:39.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:39 smithi119 ceph-mon[32752]: pgmap v18057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:41.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:41 smithi167 ceph-mon[32751]: pgmap v18058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:41.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:41 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32842.492042 secs 2024-04-27T03:42:41.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:41 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32843.357172 secs 2024-04-27T03:42:41.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:41 smithi090 ceph-mon[29180]: pgmap v18058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:41.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:41 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32842.492042 secs 2024-04-27T03:42:41.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:41 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32843.357172 secs 2024-04-27T03:42:41.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:41 smithi119 ceph-mon[32752]: pgmap v18058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:41.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:41 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32842.492042 secs 2024-04-27T03:42:41.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:41 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32843.357172 secs 2024-04-27T03:42:43.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:43 smithi167 ceph-mon[32751]: pgmap v18059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:43.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:43 smithi090 ceph-mon[29180]: pgmap v18059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:43.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:43 smithi119 ceph-mon[32752]: pgmap v18059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:44.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:42:44.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:42:44.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:42:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:45 smithi090 ceph-mon[29180]: pgmap v18060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:45.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:42:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:42:45.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:42:45.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:45 smithi119 ceph-mon[32752]: pgmap v18060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:45.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:42:45.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:42:45.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:42:45.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:45 smithi167 ceph-mon[32751]: pgmap v18060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:42:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:42:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:42:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:42:46.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:46 smithi090 ceph-mon[29180]: pgmap v18061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:42:46.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:46 smithi090 ceph-mon[29180]: pgmap v18062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:46.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:46 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32847.492122 secs 2024-04-27T03:42:46.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:46 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32848.357297 secs 2024-04-27T03:42:46.710 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:46 smithi119 ceph-mon[32752]: pgmap v18061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:42:46.711 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:46 smithi119 ceph-mon[32752]: pgmap v18062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:46.711 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:46 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32847.492122 secs 2024-04-27T03:42:46.711 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:46 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32848.357297 secs 2024-04-27T03:42:46.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:46 smithi167 ceph-mon[32751]: pgmap v18061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:42:46.725 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:46 smithi167 ceph-mon[32751]: pgmap v18062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:46.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:46 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32847.492122 secs 2024-04-27T03:42:46.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:46 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32848.357297 secs 2024-04-27T03:42:48.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:48 smithi090 ceph-mon[29180]: pgmap v18063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:48.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:48 smithi119 ceph-mon[32752]: pgmap v18063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:48.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:48 smithi167 ceph-mon[32751]: pgmap v18063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:50.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:50 smithi090 ceph-mon[29180]: pgmap v18064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:50.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:50 smithi119 ceph-mon[32752]: pgmap v18064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:50.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:50 smithi167 ceph-mon[32751]: pgmap v18064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:51.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:51 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32852.492207 secs 2024-04-27T03:42:51.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:51 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32853.357403 secs 2024-04-27T03:42:51.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:51 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32852.492207 secs 2024-04-27T03:42:51.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:51 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32853.357403 secs 2024-04-27T03:42:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:51 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32852.492207 secs 2024-04-27T03:42:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:51 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32853.357403 secs 2024-04-27T03:42:52.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:52 smithi090 ceph-mon[29180]: pgmap v18065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:52.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:52 smithi119 ceph-mon[32752]: pgmap v18065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:52.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:52 smithi167 ceph-mon[32751]: pgmap v18065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:42:54.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:54 smithi090 ceph-mon[29180]: pgmap v18066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:42:54.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:54 smithi119 ceph-mon[32752]: pgmap v18066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:42:54.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:54 smithi167 ceph-mon[32751]: pgmap v18066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:42:56.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:56 smithi090 ceph-mon[29180]: pgmap v18067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:42:56.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:56 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32857.492285 secs 2024-04-27T03:42:56.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:56 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32858.357539 secs 2024-04-27T03:42:56.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:56 smithi119 ceph-mon[32752]: pgmap v18067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:42:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:56 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32857.492285 secs 2024-04-27T03:42:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:56 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32858.357539 secs 2024-04-27T03:42:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:56 smithi167 ceph-mon[32751]: pgmap v18067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:42:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:56 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32857.492285 secs 2024-04-27T03:42:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:56 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32858.357539 secs 2024-04-27T03:42:58.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:42:58 smithi090 ceph-mon[29180]: pgmap v18068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:58.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:42:58 smithi119 ceph-mon[32752]: pgmap v18068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:42:58.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:42:58 smithi167 ceph-mon[32751]: pgmap v18068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:00.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:00 smithi090 ceph-mon[29180]: pgmap v18069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:00.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:00 smithi119 ceph-mon[32752]: pgmap v18069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:00.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:00 smithi167 ceph-mon[32751]: pgmap v18069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:01.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:01 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32862.492371 secs 2024-04-27T03:43:01.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:01 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32863.357670 secs 2024-04-27T03:43:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:01 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32862.492371 secs 2024-04-27T03:43:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:01 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32863.357670 secs 2024-04-27T03:43:01.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:01 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32862.492371 secs 2024-04-27T03:43:01.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:01 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32863.357670 secs 2024-04-27T03:43:02.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:02 smithi090 ceph-mon[29180]: pgmap v18070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:02.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:02 smithi119 ceph-mon[32752]: pgmap v18070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:02.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:02 smithi167 ceph-mon[32751]: pgmap v18070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:04.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:04 smithi090 ceph-mon[29180]: pgmap v18071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:04.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:04 smithi119 ceph-mon[32752]: pgmap v18071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:04.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:04 smithi167 ceph-mon[32751]: pgmap v18071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:06.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:06 smithi090 ceph-mon[29180]: pgmap v18072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:06.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:06 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32867.492454 secs 2024-04-27T03:43:06.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:06 smithi090 ceph-mon[29180]: 30 slow requests, 0 included below; oldest blocked for > 32868.357734 secs 2024-04-27T03:43:06.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:06 smithi119 ceph-mon[32752]: pgmap v18072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:06.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:06 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32867.492454 secs 2024-04-27T03:43:06.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:06 smithi119 ceph-mon[32752]: 30 slow requests, 0 included below; oldest blocked for > 32868.357734 secs 2024-04-27T03:43:06.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:06 smithi167 ceph-mon[32751]: pgmap v18072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:06.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:06 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32867.492454 secs 2024-04-27T03:43:06.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:06 smithi167 ceph-mon[32751]: 30 slow requests, 0 included below; oldest blocked for > 32868.357734 secs 2024-04-27T03:43:08.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:08 smithi090 ceph-mon[29180]: pgmap v18073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:08.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:08 smithi119 ceph-mon[32752]: pgmap v18073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:08.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:08 smithi167 ceph-mon[32751]: pgmap v18073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:10.382 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:43:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:43:09.888+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:43:10.382 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:43:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:43:09.888+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:43:10.382 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:43:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:43:09.888+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:43:10.382 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:43:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:43:09.888+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:43:10.383 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:43:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:43:09.888+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:43:10.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:10 smithi119 ceph-mon[32752]: pgmap v18074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:10 smithi167 ceph-mon[32751]: pgmap v18074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:10 smithi090 ceph-mon[29180]: pgmap v18074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:11 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32872.492538 secs 2024-04-27T03:43:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:11 smithi119 ceph-mon[32752]: slow request 2881.184142 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:43:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32873.357846 secs 2024-04-27T03:43:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:11 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32872.492538 secs 2024-04-27T03:43:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:11 smithi167 ceph-mon[32751]: slow request 2881.184142 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:43:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32873.357846 secs 2024-04-27T03:43:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:11 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32872.492538 secs 2024-04-27T03:43:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:11 smithi090 ceph-mon[29180]: slow request 2881.184142 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:43:11.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32873.357846 secs 2024-04-27T03:43:12.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:12 smithi119 ceph-mon[32752]: pgmap v18075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:12.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:12 smithi167 ceph-mon[32751]: pgmap v18075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:12.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:12 smithi090 ceph-mon[29180]: pgmap v18075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:13.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:13 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T03:43:13.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:13 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:43:13.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:13 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T03:43:13.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:13 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:43:13.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:13 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T03:43:13.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:13 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:43:14.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:14 smithi119 ceph-mon[32752]: pgmap v18076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:14.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:14 smithi167 ceph-mon[32751]: pgmap v18076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:14.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:14 smithi090 ceph-mon[29180]: pgmap v18076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:16 smithi119 ceph-mon[32752]: pgmap v18077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:16 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32877.492618 secs 2024-04-27T03:43:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:16 smithi119 ceph-mon[32752]: slow request 1440.296164 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:43:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32878.357926 secs 2024-04-27T03:43:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:16 smithi167 ceph-mon[32751]: pgmap v18077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:16 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32877.492618 secs 2024-04-27T03:43:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:16 smithi167 ceph-mon[32751]: slow request 1440.296164 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:43:16.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32878.357926 secs 2024-04-27T03:43:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:16 smithi090 ceph-mon[29180]: pgmap v18077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:16 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32877.492618 secs 2024-04-27T03:43:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:16 smithi090 ceph-mon[29180]: slow request 1440.296164 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:43:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32878.357926 secs 2024-04-27T03:43:18.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:18 smithi119 ceph-mon[32752]: pgmap v18078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:18.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:18 smithi167 ceph-mon[32751]: pgmap v18078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:18.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:18 smithi090 ceph-mon[29180]: pgmap v18078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:20.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:20 smithi119 ceph-mon[32752]: pgmap v18079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:20.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:20 smithi167 ceph-mon[32751]: pgmap v18079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:20 smithi090 ceph-mon[29180]: pgmap v18079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:21.782 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:21 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32882.492704 secs 2024-04-27T03:43:21.783 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32883.358026 secs 2024-04-27T03:43:21.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:21 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32882.492704 secs 2024-04-27T03:43:21.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32883.358026 secs 2024-04-27T03:43:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:21 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32882.492704 secs 2024-04-27T03:43:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32883.358026 secs 2024-04-27T03:43:22.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:22 smithi119 ceph-mon[32752]: pgmap v18080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:22.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:22 smithi167 ceph-mon[32751]: pgmap v18080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:22.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:22 smithi090 ceph-mon[29180]: pgmap v18080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:24.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:24 smithi119 ceph-mon[32752]: pgmap v18081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:24.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:24 smithi167 ceph-mon[32751]: pgmap v18081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:24.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:24 smithi090 ceph-mon[29180]: pgmap v18081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T03:43:26.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:26 smithi119 ceph-mon[32752]: pgmap v18082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:26.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:26 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32887.492784 secs 2024-04-27T03:43:26.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32888.358143 secs 2024-04-27T03:43:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:26 smithi167 ceph-mon[32751]: pgmap v18082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:26 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32887.492784 secs 2024-04-27T03:43:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32888.358143 secs 2024-04-27T03:43:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:26 smithi090 ceph-mon[29180]: pgmap v18082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T03:43:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:26 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32887.492784 secs 2024-04-27T03:43:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32888.358143 secs 2024-04-27T03:43:28.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:28 smithi119 ceph-mon[32752]: pgmap v18083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:28.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:28 smithi167 ceph-mon[32751]: pgmap v18083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:28.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:28 smithi090 ceph-mon[29180]: pgmap v18083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:30 smithi119 ceph-mon[32752]: pgmap v18084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:30.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:30 smithi167 ceph-mon[32751]: pgmap v18084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:30 smithi090 ceph-mon[29180]: pgmap v18084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:31 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32892.492874 secs 2024-04-27T03:43:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32893.358227 secs 2024-04-27T03:43:31.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:31 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32892.492874 secs 2024-04-27T03:43:31.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32893.358227 secs 2024-04-27T03:43:31.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:31 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32892.492874 secs 2024-04-27T03:43:31.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32893.358227 secs 2024-04-27T03:43:32.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:32 smithi119 ceph-mon[32752]: pgmap v18085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:32.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:32 smithi167 ceph-mon[32751]: pgmap v18085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:32.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:32 smithi090 ceph-mon[29180]: pgmap v18085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:34.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:34 smithi119 ceph-mon[32752]: pgmap v18086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:34.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:34 smithi167 ceph-mon[32751]: pgmap v18086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:34.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:34 smithi090 ceph-mon[29180]: pgmap v18086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:36 smithi119 ceph-mon[32752]: pgmap v18087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:36 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32897.492957 secs 2024-04-27T03:43:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32898.358324 secs 2024-04-27T03:43:36.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:36 smithi167 ceph-mon[32751]: pgmap v18087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:36.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:36 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32897.492957 secs 2024-04-27T03:43:36.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32898.358324 secs 2024-04-27T03:43:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:36 smithi090 ceph-mon[29180]: pgmap v18087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:36 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32897.492957 secs 2024-04-27T03:43:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32898.358324 secs 2024-04-27T03:43:38.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:38 smithi119 ceph-mon[32752]: pgmap v18088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:38.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:38 smithi167 ceph-mon[32751]: pgmap v18088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:38.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:38 smithi090 ceph-mon[29180]: pgmap v18088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:40.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:40 smithi119 ceph-mon[32752]: pgmap v18089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:40.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:40 smithi167 ceph-mon[32751]: pgmap v18089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:40 smithi090 ceph-mon[29180]: pgmap v18089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:41.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:41 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32902.493040 secs 2024-04-27T03:43:41.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32903.358443 secs 2024-04-27T03:43:41.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:41 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32902.493040 secs 2024-04-27T03:43:41.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32903.358443 secs 2024-04-27T03:43:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:41 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32902.493040 secs 2024-04-27T03:43:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32903.358443 secs 2024-04-27T03:43:42.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:42 smithi119 ceph-mon[32752]: pgmap v18090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:42.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:42 smithi167 ceph-mon[32751]: pgmap v18090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:42.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:42 smithi090 ceph-mon[29180]: pgmap v18090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:44.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:44 smithi119 ceph-mon[32752]: pgmap v18091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:44.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:44 smithi167 ceph-mon[32751]: pgmap v18091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:44.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:44 smithi090 ceph-mon[29180]: pgmap v18091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:45.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:43:45.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:43:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:43:46.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: pgmap v18092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:46.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32907.493104 secs 2024-04-27T03:43:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:43:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:43:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: pgmap v18093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:43:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: pgmap v18094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:43:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32908.358576 secs 2024-04-27T03:43:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: pgmap v18092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32907.493104 secs 2024-04-27T03:43:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:43:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:43:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: pgmap v18093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:43:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: pgmap v18094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:46.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:43:46.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32908.358576 secs 2024-04-27T03:43:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: pgmap v18092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:43:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32907.493104 secs 2024-04-27T03:43:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:43:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:43:46.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: pgmap v18093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:43:46.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: pgmap v18094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:46.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:43:46.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32908.358576 secs 2024-04-27T03:43:49.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:48 smithi167 ceph-mon[32751]: pgmap v18095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:43:49.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:48 smithi090 ceph-mon[29180]: pgmap v18095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:43:49.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:48 smithi119 ceph-mon[32752]: pgmap v18095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T03:43:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:50 smithi167 ceph-mon[32751]: pgmap v18096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32912.493189 secs 2024-04-27T03:43:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:50 smithi090 ceph-mon[29180]: pgmap v18096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32912.493189 secs 2024-04-27T03:43:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:50 smithi119 ceph-mon[32752]: pgmap v18096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32912.493189 secs 2024-04-27T03:43:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32913.358660 secs 2024-04-27T03:43:52.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32913.358660 secs 2024-04-27T03:43:52.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32913.358660 secs 2024-04-27T03:43:53.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:52 smithi090 ceph-mon[29180]: pgmap v18097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:53.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:52 smithi119 ceph-mon[32752]: pgmap v18097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:53.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:52 smithi167 ceph-mon[32751]: pgmap v18097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:55.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:54 smithi090 ceph-mon[29180]: pgmap v18098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:55.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:54 smithi119 ceph-mon[32752]: pgmap v18098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:55.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:54 smithi167 ceph-mon[32751]: pgmap v18098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:43:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32917.493270 secs 2024-04-27T03:43:56.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32917.493270 secs 2024-04-27T03:43:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32917.493270 secs 2024-04-27T03:43:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:56 smithi090 ceph-mon[29180]: pgmap v18099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:43:57.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32918.358747 secs 2024-04-27T03:43:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:56 smithi119 ceph-mon[32752]: pgmap v18099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:43:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32918.358747 secs 2024-04-27T03:43:57.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:56 smithi167 ceph-mon[32751]: pgmap v18099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:43:57.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32918.358747 secs 2024-04-27T03:43:59.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:43:58 smithi090 ceph-mon[29180]: pgmap v18100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:59.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:43:58 smithi119 ceph-mon[32752]: pgmap v18100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:43:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:43:58 smithi167 ceph-mon[32751]: pgmap v18100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:00 smithi090 ceph-mon[29180]: pgmap v18101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32922.493356 secs 2024-04-27T03:44:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:00 smithi119 ceph-mon[32752]: pgmap v18101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32922.493356 secs 2024-04-27T03:44:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:00 smithi167 ceph-mon[32751]: pgmap v18101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32922.493356 secs 2024-04-27T03:44:02.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32923.358874 secs 2024-04-27T03:44:02.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32923.358874 secs 2024-04-27T03:44:02.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32923.358874 secs 2024-04-27T03:44:03.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:02 smithi090 ceph-mon[29180]: pgmap v18102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:03.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:02 smithi119 ceph-mon[32752]: pgmap v18102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:03.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:02 smithi167 ceph-mon[32751]: pgmap v18102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:05.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:04 smithi090 ceph-mon[29180]: pgmap v18103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:05.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:04 smithi119 ceph-mon[32752]: pgmap v18103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:05.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:04 smithi167 ceph-mon[32751]: pgmap v18103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32927.493435 secs 2024-04-27T03:44:06.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32927.493435 secs 2024-04-27T03:44:06.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32927.493435 secs 2024-04-27T03:44:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:06 smithi090 ceph-mon[29180]: pgmap v18104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32928.358975 secs 2024-04-27T03:44:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:06 smithi119 ceph-mon[32752]: pgmap v18104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32928.358975 secs 2024-04-27T03:44:07.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:06 smithi167 ceph-mon[32751]: pgmap v18104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:07.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32928.358975 secs 2024-04-27T03:44:09.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:08 smithi090 ceph-mon[29180]: pgmap v18105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:09.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:08 smithi119 ceph-mon[32752]: pgmap v18105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:09.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:08 smithi167 ceph-mon[32751]: pgmap v18105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:10 smithi090 ceph-mon[29180]: pgmap v18106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32932.493520 secs 2024-04-27T03:44:11.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:10 smithi119 ceph-mon[32752]: pgmap v18106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:11.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32932.493520 secs 2024-04-27T03:44:11.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:10 smithi167 ceph-mon[32751]: pgmap v18106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:11.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32932.493520 secs 2024-04-27T03:44:12.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32933.359047 secs 2024-04-27T03:44:12.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32933.359047 secs 2024-04-27T03:44:12.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32933.359047 secs 2024-04-27T03:44:13.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:12 smithi090 ceph-mon[29180]: pgmap v18107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:13.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:12 smithi119 ceph-mon[32752]: pgmap v18107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:13.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:12 smithi167 ceph-mon[32751]: pgmap v18107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T03:44:15.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:14 smithi090 ceph-mon[29180]: pgmap v18108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:15.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:14 smithi119 ceph-mon[32752]: pgmap v18108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:15.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:14 smithi167 ceph-mon[32751]: pgmap v18108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:44:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32937.493602 secs 2024-04-27T03:44:16.211 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32937.493602 secs 2024-04-27T03:44:16.225 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32937.493602 secs 2024-04-27T03:44:17.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:16 smithi119 ceph-mon[32752]: pgmap v18109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:17.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32938.359170 secs 2024-04-27T03:44:17.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:16 smithi167 ceph-mon[32751]: pgmap v18109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:17.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32938.359170 secs 2024-04-27T03:44:17.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:16 smithi090 ceph-mon[29180]: pgmap v18109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:17.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32938.359170 secs 2024-04-27T03:44:19.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:18 smithi119 ceph-mon[32752]: pgmap v18110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:19.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:18 smithi167 ceph-mon[32751]: pgmap v18110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:19.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:18 smithi090 ceph-mon[29180]: pgmap v18110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:21.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:20 smithi119 ceph-mon[32752]: pgmap v18111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:21.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32942.493680 secs 2024-04-27T03:44:21.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:20 smithi167 ceph-mon[32751]: pgmap v18111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:21.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32942.493680 secs 2024-04-27T03:44:21.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:20 smithi090 ceph-mon[29180]: pgmap v18111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:21.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32942.493680 secs 2024-04-27T03:44:22.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32943.359309 secs 2024-04-27T03:44:22.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32943.359309 secs 2024-04-27T03:44:22.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32943.359309 secs 2024-04-27T03:44:23.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:22 smithi119 ceph-mon[32752]: pgmap v18112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:23.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:22 smithi167 ceph-mon[32751]: pgmap v18112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:23.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:22 smithi090 ceph-mon[29180]: pgmap v18112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:25.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:24 smithi119 ceph-mon[32752]: pgmap v18113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:25.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:24 smithi167 ceph-mon[32751]: pgmap v18113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:25.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:24 smithi090 ceph-mon[29180]: pgmap v18113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:26.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32947.493759 secs 2024-04-27T03:44:26.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32947.493759 secs 2024-04-27T03:44:26.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32947.493759 secs 2024-04-27T03:44:27.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:26 smithi119 ceph-mon[32752]: pgmap v18114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:27.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32948.359415 secs 2024-04-27T03:44:27.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:26 smithi167 ceph-mon[32751]: pgmap v18114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:27.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32948.359415 secs 2024-04-27T03:44:27.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:26 smithi090 ceph-mon[29180]: pgmap v18114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:27.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32948.359415 secs 2024-04-27T03:44:29.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:28 smithi119 ceph-mon[32752]: pgmap v18115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:29.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:28 smithi167 ceph-mon[32751]: pgmap v18115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:29.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:28 smithi090 ceph-mon[29180]: pgmap v18115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:31.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:30 smithi090 ceph-mon[29180]: pgmap v18116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:31.185 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32952.493844 secs 2024-04-27T03:44:31.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:30 smithi119 ceph-mon[32752]: pgmap v18116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:31.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32952.493844 secs 2024-04-27T03:44:31.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:30 smithi167 ceph-mon[32751]: pgmap v18116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:31.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32952.493844 secs 2024-04-27T03:44:32.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32953.359477 secs 2024-04-27T03:44:32.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32953.359477 secs 2024-04-27T03:44:32.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32953.359477 secs 2024-04-27T03:44:33.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:32 smithi119 ceph-mon[32752]: pgmap v18117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:33.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:32 smithi167 ceph-mon[32751]: pgmap v18117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:33.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:32 smithi090 ceph-mon[29180]: pgmap v18117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:35.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:34 smithi119 ceph-mon[32752]: pgmap v18118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:35.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:34 smithi167 ceph-mon[32751]: pgmap v18118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:35.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:34 smithi090 ceph-mon[29180]: pgmap v18118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32957.493927 secs 2024-04-27T03:44:36.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32957.493927 secs 2024-04-27T03:44:36.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32957.493927 secs 2024-04-27T03:44:37.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:36 smithi119 ceph-mon[32752]: pgmap v18119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:37.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32958.359597 secs 2024-04-27T03:44:37.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:36 smithi167 ceph-mon[32751]: pgmap v18119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:37.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32958.359597 secs 2024-04-27T03:44:37.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:36 smithi090 ceph-mon[29180]: pgmap v18119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:37.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32958.359597 secs 2024-04-27T03:44:39.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:38 smithi119 ceph-mon[32752]: pgmap v18120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:39.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:38 smithi167 ceph-mon[32751]: pgmap v18120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:39.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:38 smithi090 ceph-mon[29180]: pgmap v18120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:41.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:40 smithi119 ceph-mon[32752]: pgmap v18121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:41.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32962.494011 secs 2024-04-27T03:44:41.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:40 smithi167 ceph-mon[32751]: pgmap v18121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:41.326 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32962.494011 secs 2024-04-27T03:44:41.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:40 smithi090 ceph-mon[29180]: pgmap v18121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:41.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32962.494011 secs 2024-04-27T03:44:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32963.359658 secs 2024-04-27T03:44:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32963.359658 secs 2024-04-27T03:44:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32963.359658 secs 2024-04-27T03:44:43.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:42 smithi119 ceph-mon[32752]: pgmap v18122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:43.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:42 smithi167 ceph-mon[32751]: pgmap v18122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:43.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:42 smithi090 ceph-mon[29180]: pgmap v18122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:45.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:44 smithi119 ceph-mon[32752]: pgmap v18123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:45.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:44 smithi167 ceph-mon[32751]: pgmap v18123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:45.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:44 smithi090 ceph-mon[29180]: pgmap v18123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:44:46.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:46 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32967.494090 secs 2024-04-27T03:44:46.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:46 smithi119 ceph-mon[32752]: pgmap v18124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:46.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:44:46.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:46 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32967.494090 secs 2024-04-27T03:44:46.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:46 smithi167 ceph-mon[32751]: pgmap v18124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:46.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:44:46.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:46 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32967.494090 secs 2024-04-27T03:44:46.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:46 smithi090 ceph-mon[29180]: pgmap v18124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:46.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:44:47.398 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:47 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32968.359746 secs 2024-04-27T03:44:47.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.399 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.447 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:47 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32968.359746 secs 2024-04-27T03:44:47.448 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.448 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.448 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.448 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:47 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32968.359746 secs 2024-04-27T03:44:47.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:47.575 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:48.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:48 smithi119 ceph-mon[32752]: pgmap v18125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:48.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:48.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:48.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:48 smithi167 ceph-mon[32751]: pgmap v18125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:48.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:48.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:48.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:48 smithi090 ceph-mon[29180]: pgmap v18125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:44:48.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:48.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:49.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:49.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:44:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:44:49.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.884 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:49.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:44:49.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:44:49.885 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:44:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:44:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:44:50.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:44:50.063 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:44:50.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:44:50.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:44:50.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:44:50.077 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:44:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:50 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:44:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:50 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:44:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:50 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:44:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:50 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:44:51.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:50 smithi119 ceph-mon[32752]: pgmap v18126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:44:51.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:50 smithi119 ceph-mon[32752]: pgmap v18127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:51.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32972.494176 secs 2024-04-27T03:44:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:50 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:44:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:50 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:44:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:50 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:44:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:50 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:44:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:50 smithi167 ceph-mon[32751]: pgmap v18126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:44:51.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:50 smithi167 ceph-mon[32751]: pgmap v18127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:51.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32972.494176 secs 2024-04-27T03:44:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:50 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:44:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:50 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:44:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:50 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:44:51.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:50 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:44:51.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:50 smithi090 ceph-mon[29180]: pgmap v18126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T03:44:51.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:50 smithi090 ceph-mon[29180]: pgmap v18127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:51.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32972.494176 secs 2024-04-27T03:44:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32973.359856 secs 2024-04-27T03:44:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32973.359856 secs 2024-04-27T03:44:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32973.359856 secs 2024-04-27T03:44:53.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:52 smithi119 ceph-mon[32752]: pgmap v18128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:52 smithi167 ceph-mon[32751]: pgmap v18128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:53.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:52 smithi090 ceph-mon[29180]: pgmap v18128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:55.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:54 smithi119 ceph-mon[32752]: pgmap v18129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T03:44:55.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:54 smithi167 ceph-mon[32751]: pgmap v18129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T03:44:55.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:54 smithi090 ceph-mon[29180]: pgmap v18129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 B/s rd, 1 op/s 2024-04-27T03:44:55.954 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32977.494258 secs 2024-04-27T03:44:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32977.494258 secs 2024-04-27T03:44:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32977.494258 secs 2024-04-27T03:44:57.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:56 smithi119 ceph-mon[32752]: pgmap v18130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:57.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32978.359949 secs 2024-04-27T03:44:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:56 smithi167 ceph-mon[32751]: pgmap v18130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32978.359949 secs 2024-04-27T03:44:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:56 smithi090 ceph-mon[29180]: pgmap v18130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32978.359949 secs 2024-04-27T03:44:59.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:44:58 smithi119 ceph-mon[32752]: pgmap v18131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:59.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:44:58 smithi167 ceph-mon[32751]: pgmap v18131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:44:59.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:44:58 smithi090 ceph-mon[29180]: pgmap v18131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:00 smithi119 ceph-mon[32752]: pgmap v18132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:45:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32982.494343 secs 2024-04-27T03:45:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:00 smithi167 ceph-mon[32751]: pgmap v18132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:45:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32982.494343 secs 2024-04-27T03:45:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:00 smithi090 ceph-mon[29180]: pgmap v18132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:45:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32982.494343 secs 2024-04-27T03:45:02.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32983.360080 secs 2024-04-27T03:45:02.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32983.360080 secs 2024-04-27T03:45:02.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32983.360080 secs 2024-04-27T03:45:03.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:02 smithi119 ceph-mon[32752]: pgmap v18133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:03.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:02 smithi167 ceph-mon[32751]: pgmap v18133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:03.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:02 smithi090 ceph-mon[29180]: pgmap v18133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:04 smithi119 ceph-mon[32752]: pgmap v18134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:05.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:04 smithi167 ceph-mon[32751]: pgmap v18134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:05.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:04 smithi090 ceph-mon[29180]: pgmap v18134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:05 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32987.494426 secs 2024-04-27T03:45:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:05 smithi119 ceph-mon[32752]: slow request 23041.762843 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:45:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:05 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32987.494426 secs 2024-04-27T03:45:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:05 smithi167 ceph-mon[32751]: slow request 23041.762843 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:45:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:05 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32987.494426 secs 2024-04-27T03:45:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:05 smithi090 ceph-mon[29180]: slow request 23041.762843 seconds old, received at 2024-04-26T21:21:03.484987+0000: client_request(client.17253:1 getattr pAsLsXsFs #0x1 2024-04-26T21:21:03.481573+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:45:07.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:06 smithi119 ceph-mon[32752]: pgmap v18135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:07.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32988.360201 secs 2024-04-27T03:45:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:06 smithi167 ceph-mon[32751]: pgmap v18135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32988.360201 secs 2024-04-27T03:45:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:06 smithi090 ceph-mon[29180]: pgmap v18135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32988.360201 secs 2024-04-27T03:45:09.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:08 smithi119 ceph-mon[32752]: pgmap v18136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:09.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:08 smithi167 ceph-mon[32751]: pgmap v18136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:09.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:08 smithi090 ceph-mon[29180]: pgmap v18136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:10 smithi119 ceph-mon[32752]: pgmap v18137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 32992.494510 secs 2024-04-27T03:45:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:10 smithi167 ceph-mon[32751]: pgmap v18137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 32992.494510 secs 2024-04-27T03:45:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:10 smithi090 ceph-mon[29180]: pgmap v18137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 32992.494510 secs 2024-04-27T03:45:12.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32993.360319 secs 2024-04-27T03:45:12.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32993.360319 secs 2024-04-27T03:45:12.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32993.360319 secs 2024-04-27T03:45:13.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:12 smithi119 ceph-mon[32752]: pgmap v18138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:13.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:12 smithi167 ceph-mon[32751]: pgmap v18138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:13.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:12 smithi090 ceph-mon[29180]: pgmap v18138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:15.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:14 smithi119 ceph-mon[32752]: pgmap v18139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:15.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:14 smithi167 ceph-mon[32751]: pgmap v18139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:15.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:14 smithi090 ceph-mon[29180]: pgmap v18139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:15 smithi119 ceph-mon[32752]: 43 slow requests, 1 included below; oldest blocked for > 32997.494591 secs 2024-04-27T03:45:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:15 smithi119 ceph-mon[32752]: slow request 23041.726326 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-27T03:45:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:15 smithi167 ceph-mon[32751]: 43 slow requests, 1 included below; oldest blocked for > 32997.494591 secs 2024-04-27T03:45:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:15 smithi167 ceph-mon[32751]: slow request 23041.726326 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-27T03:45:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:15 smithi090 ceph-mon[29180]: 43 slow requests, 1 included below; oldest blocked for > 32997.494591 secs 2024-04-27T03:45:16.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:15 smithi090 ceph-mon[29180]: slow request 23041.726326 seconds old, received at 2024-04-26T21:21:13.521669+0000: client_request(client.24733:267889 getattr AsXsFs #0x100000001fb 2024-04-26T21:21:13.520482+0000 caller_uid=0, caller_gid=0{0,}) currently joining batch getattr 2024-04-27T03:45:17.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:16 smithi119 ceph-mon[32752]: pgmap v18140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:17.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 32998.360454 secs 2024-04-27T03:45:17.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:16 smithi167 ceph-mon[32751]: pgmap v18140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:17.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 32998.360454 secs 2024-04-27T03:45:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:16 smithi090 ceph-mon[29180]: pgmap v18140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 32998.360454 secs 2024-04-27T03:45:19.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:18 smithi119 ceph-mon[32752]: pgmap v18141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:19.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:18 smithi167 ceph-mon[32751]: pgmap v18141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:19.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:18 smithi090 ceph-mon[29180]: pgmap v18141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:20 smithi119 ceph-mon[32752]: pgmap v18142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33002.494672 secs 2024-04-27T03:45:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:20 smithi167 ceph-mon[32751]: pgmap v18142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33002.494672 secs 2024-04-27T03:45:21.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:20 smithi090 ceph-mon[29180]: pgmap v18142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:21.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33002.494672 secs 2024-04-27T03:45:22.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33003.360518 secs 2024-04-27T03:45:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33003.360518 secs 2024-04-27T03:45:22.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33003.360518 secs 2024-04-27T03:45:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:22 smithi119 ceph-mon[32752]: pgmap v18143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:22 smithi167 ceph-mon[32751]: pgmap v18143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:23.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:22 smithi090 ceph-mon[29180]: pgmap v18143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:24 smithi119 ceph-mon[32752]: pgmap v18144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:24 smithi167 ceph-mon[32751]: pgmap v18144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:25.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:24 smithi090 ceph-mon[29180]: pgmap v18144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33007.494754 secs 2024-04-27T03:45:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33007.494754 secs 2024-04-27T03:45:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33007.494754 secs 2024-04-27T03:45:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:26 smithi119 ceph-mon[32752]: pgmap v18145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T03:45:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33008.360585 secs 2024-04-27T03:45:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:26 smithi167 ceph-mon[32751]: pgmap v18145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T03:45:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33008.360585 secs 2024-04-27T03:45:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:26 smithi090 ceph-mon[29180]: pgmap v18145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T03:45:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33008.360585 secs 2024-04-27T03:45:29.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:28 smithi119 ceph-mon[32752]: pgmap v18146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:28 smithi167 ceph-mon[32751]: pgmap v18146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:28 smithi090 ceph-mon[29180]: pgmap v18146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:30 smithi119 ceph-mon[32752]: pgmap v18147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33012.494840 secs 2024-04-27T03:45:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:30 smithi167 ceph-mon[32751]: pgmap v18147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33012.494840 secs 2024-04-27T03:45:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:30 smithi090 ceph-mon[29180]: pgmap v18147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33012.494840 secs 2024-04-27T03:45:32.033 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33013.360703 secs 2024-04-27T03:45:32.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33013.360703 secs 2024-04-27T03:45:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33013.360703 secs 2024-04-27T03:45:33.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:32 smithi119 ceph-mon[32752]: pgmap v18148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:32 smithi167 ceph-mon[32751]: pgmap v18148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:33.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:32 smithi090 ceph-mon[29180]: pgmap v18148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:35.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:34 smithi119 ceph-mon[32752]: pgmap v18149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:34 smithi167 ceph-mon[32751]: pgmap v18149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:34 smithi090 ceph-mon[29180]: pgmap v18149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:45:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33017.494900 secs 2024-04-27T03:45:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33017.494900 secs 2024-04-27T03:45:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33017.494900 secs 2024-04-27T03:45:37.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:36 smithi119 ceph-mon[32752]: pgmap v18150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T03:45:37.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33018.360813 secs 2024-04-27T03:45:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:36 smithi167 ceph-mon[32751]: pgmap v18150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T03:45:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33018.360813 secs 2024-04-27T03:45:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:36 smithi090 ceph-mon[29180]: pgmap v18150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T03:45:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33018.360813 secs 2024-04-27T03:45:39.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:38 smithi119 ceph-mon[32752]: pgmap v18151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:39.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:38 smithi167 ceph-mon[32751]: pgmap v18151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:39.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:38 smithi090 ceph-mon[29180]: pgmap v18151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:40 smithi119 ceph-mon[32752]: pgmap v18152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33022.494981 secs 2024-04-27T03:45:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:40 smithi167 ceph-mon[32751]: pgmap v18152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33022.494981 secs 2024-04-27T03:45:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:40 smithi090 ceph-mon[29180]: pgmap v18152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33022.494981 secs 2024-04-27T03:45:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:41 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33023.360955 secs 2024-04-27T03:45:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:41 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33023.360955 secs 2024-04-27T03:45:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:41 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33023.360955 secs 2024-04-27T03:45:43.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:42 smithi119 ceph-mon[32752]: pgmap v18153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:43.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:42 smithi167 ceph-mon[32751]: pgmap v18153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:43.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:42 smithi090 ceph-mon[29180]: pgmap v18153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:45.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:44 smithi167 ceph-mon[32751]: pgmap v18154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:45.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:44 smithi090 ceph-mon[29180]: pgmap v18154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:45.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:44 smithi119 ceph-mon[32752]: pgmap v18154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:45:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33027.495067 secs 2024-04-27T03:45:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33027.495067 secs 2024-04-27T03:45:46.211 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33027.495067 secs 2024-04-27T03:45:47.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:46 smithi167 ceph-mon[32751]: pgmap v18155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:47.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:46 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33028.361020 secs 2024-04-27T03:45:47.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:46 smithi090 ceph-mon[29180]: pgmap v18155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:47.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:46 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33028.361020 secs 2024-04-27T03:45:47.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:46 smithi119 ceph-mon[32752]: pgmap v18155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:47.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:46 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33028.361020 secs 2024-04-27T03:45:49.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:48 smithi167 ceph-mon[32751]: pgmap v18156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:49.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:48 smithi090 ceph-mon[29180]: pgmap v18156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:49.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:48 smithi119 ceph-mon[32752]: pgmap v18156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:45:50.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:45:50.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:45:50.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:45:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:50 smithi119 ceph-mon[32752]: pgmap v18157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33032.495152 secs 2024-04-27T03:45:51.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:45:51.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:45:51.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:50 smithi167 ceph-mon[32751]: pgmap v18157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33032.495152 secs 2024-04-27T03:45:51.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:45:51.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:45:51.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:50 smithi090 ceph-mon[29180]: pgmap v18157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:45:51.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33032.495152 secs 2024-04-27T03:45:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:45:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:45:51.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:45:52.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:51 smithi119 ceph-mon[32752]: pgmap v18158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T03:45:52.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:51 smithi119 ceph-mon[32752]: pgmap v18159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:52.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:51 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33033.361093 secs 2024-04-27T03:45:52.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:51 smithi167 ceph-mon[32751]: pgmap v18158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T03:45:52.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:51 smithi167 ceph-mon[32751]: pgmap v18159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:52.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:51 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33033.361093 secs 2024-04-27T03:45:52.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:51 smithi090 ceph-mon[29180]: pgmap v18158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T03:45:52.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:51 smithi090 ceph-mon[29180]: pgmap v18159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:52.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:51 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33033.361093 secs 2024-04-27T03:45:54.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:53 smithi119 ceph-mon[32752]: pgmap v18160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T03:45:54.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:53 smithi167 ceph-mon[32751]: pgmap v18160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T03:45:54.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:53 smithi090 ceph-mon[29180]: pgmap v18160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T03:45:56.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:55 smithi119 ceph-mon[32752]: pgmap v18161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:56.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33037.495233 secs 2024-04-27T03:45:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:55 smithi167 ceph-mon[32751]: pgmap v18161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33037.495233 secs 2024-04-27T03:45:56.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:55 smithi090 ceph-mon[29180]: pgmap v18161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:56.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33037.495233 secs 2024-04-27T03:45:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:56 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33038.361209 secs 2024-04-27T03:45:57.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:56 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33038.361209 secs 2024-04-27T03:45:57.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:56 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33038.361209 secs 2024-04-27T03:45:58.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:57 smithi090 ceph-mon[29180]: pgmap v18162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:57 smithi119 ceph-mon[32752]: pgmap v18162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:45:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:57 smithi167 ceph-mon[32751]: pgmap v18162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:00.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:45:59 smithi119 ceph-mon[32752]: pgmap v18163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:00.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:45:59 smithi167 ceph-mon[32751]: pgmap v18163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:00.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:45:59 smithi090 ceph-mon[29180]: pgmap v18163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33042.495318 secs 2024-04-27T03:46:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33042.495318 secs 2024-04-27T03:46:01.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33042.495318 secs 2024-04-27T03:46:02.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:01 smithi167 ceph-mon[32751]: pgmap v18164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:46:02.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:01 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33043.361272 secs 2024-04-27T03:46:02.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:01 smithi090 ceph-mon[29180]: pgmap v18164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:46:02.284 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:01 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33043.361272 secs 2024-04-27T03:46:02.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:01 smithi119 ceph-mon[32752]: pgmap v18164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:46:02.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:01 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33043.361272 secs 2024-04-27T03:46:04.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:03 smithi119 ceph-mon[32752]: pgmap v18165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:04.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:03 smithi167 ceph-mon[32751]: pgmap v18165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:04.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:03 smithi090 ceph-mon[29180]: pgmap v18165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:06.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:05 smithi119 ceph-mon[32752]: pgmap v18166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:06.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33047.495399 secs 2024-04-27T03:46:06.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:05 smithi167 ceph-mon[32751]: pgmap v18166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:06.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33047.495399 secs 2024-04-27T03:46:06.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:05 smithi090 ceph-mon[29180]: pgmap v18166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:06.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33047.495399 secs 2024-04-27T03:46:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33048.361391 secs 2024-04-27T03:46:07.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33048.361391 secs 2024-04-27T03:46:07.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:06 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33048.361391 secs 2024-04-27T03:46:08.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:07 smithi119 ceph-mon[32752]: pgmap v18167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:08.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:07 smithi167 ceph-mon[32751]: pgmap v18167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:08.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:07 smithi090 ceph-mon[29180]: pgmap v18167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:10.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:09 smithi119 ceph-mon[32752]: pgmap v18168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:10.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:09 smithi167 ceph-mon[32751]: pgmap v18168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:10.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:09 smithi090 ceph-mon[29180]: pgmap v18168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:11.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33052.495482 secs 2024-04-27T03:46:11.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33052.495482 secs 2024-04-27T03:46:11.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33052.495482 secs 2024-04-27T03:46:12.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:11 smithi119 ceph-mon[32752]: pgmap v18169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:12.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:11 smithi119 ceph-mon[32752]: 31 slow requests, 2 included below; oldest blocked for > 33053.361544 secs 2024-04-27T03:46:12.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:11 smithi119 ceph-mon[32752]: slow request 181.214112 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:46:12.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:11 smithi119 ceph-mon[32752]: slow request 11614.404293 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:46:12.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:11 smithi167 ceph-mon[32751]: pgmap v18169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:12.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:11 smithi167 ceph-mon[32751]: 31 slow requests, 2 included below; oldest blocked for > 33053.361544 secs 2024-04-27T03:46:12.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:11 smithi167 ceph-mon[32751]: slow request 181.214112 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:46:12.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:11 smithi167 ceph-mon[32751]: slow request 11614.404293 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:46:12.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:11 smithi090 ceph-mon[29180]: pgmap v18169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:11 smithi090 ceph-mon[29180]: 31 slow requests, 2 included below; oldest blocked for > 33053.361544 secs 2024-04-27T03:46:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:11 smithi090 ceph-mon[29180]: slow request 181.214112 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:46:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:11 smithi090 ceph-mon[29180]: slow request 11614.404293 seconds old, received at 2024-04-27T00:32:36.707896+0000: client_request(client.17514:1 getattr pAsLsXsFs #0x1 2024-04-27T00:32:36.703844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:46:13.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:12 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T03:46:13.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:12 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:46:13.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:12 smithi119 ceph-mon[32752]: pgmap v18170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:13.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:12 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T03:46:13.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:12 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:46:13.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:12 smithi167 ceph-mon[32751]: pgmap v18170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:13.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:12 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T03:46:13.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:12 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:46:13.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:12 smithi090 ceph-mon[29180]: pgmap v18170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:15 smithi090 ceph-mon[29180]: pgmap v18171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33057.495560 secs 2024-04-27T03:46:16.012 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:15 smithi119 ceph-mon[32752]: pgmap v18171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:16.013 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33057.495560 secs 2024-04-27T03:46:16.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:15 smithi167 ceph-mon[32751]: pgmap v18171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:16.027 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33057.495560 secs 2024-04-27T03:46:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33058.361675 secs 2024-04-27T03:46:17.027 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33058.361675 secs 2024-04-27T03:46:17.044 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33058.361675 secs 2024-04-27T03:46:17.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:17 smithi090 ceph-mon[29180]: pgmap v18172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:18.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:17 smithi119 ceph-mon[32752]: pgmap v18172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:18.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:17 smithi167 ceph-mon[32751]: pgmap v18172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:19.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:19 smithi090 ceph-mon[29180]: pgmap v18173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:20.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:19 smithi119 ceph-mon[32752]: pgmap v18173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:20.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:19 smithi167 ceph-mon[32751]: pgmap v18173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33062.495647 secs 2024-04-27T03:46:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33062.495647 secs 2024-04-27T03:46:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33062.495647 secs 2024-04-27T03:46:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:21 smithi090 ceph-mon[29180]: pgmap v18174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33063.361816 secs 2024-04-27T03:46:22.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:21 smithi119 ceph-mon[32752]: pgmap v18174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:22.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33063.361816 secs 2024-04-27T03:46:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:21 smithi167 ceph-mon[32751]: pgmap v18174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33063.361816 secs 2024-04-27T03:46:23.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:23 smithi090 ceph-mon[29180]: pgmap v18175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:24.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:23 smithi119 ceph-mon[32752]: pgmap v18175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:24.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:23 smithi167 ceph-mon[32751]: pgmap v18175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:25 smithi090 ceph-mon[29180]: pgmap v18176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33067.495730 secs 2024-04-27T03:46:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:25 smithi119 ceph-mon[32752]: pgmap v18176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33067.495730 secs 2024-04-27T03:46:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:25 smithi167 ceph-mon[32751]: pgmap v18176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33067.495730 secs 2024-04-27T03:46:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33068.361934 secs 2024-04-27T03:46:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33068.361934 secs 2024-04-27T03:46:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33068.361934 secs 2024-04-27T03:46:27.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:27 smithi090 ceph-mon[29180]: pgmap v18177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:28.008 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:27 smithi119 ceph-mon[32752]: pgmap v18177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:28.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:27 smithi167 ceph-mon[32751]: pgmap v18177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:30.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:29 smithi119 ceph-mon[32752]: pgmap v18178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:30.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:29 smithi167 ceph-mon[32751]: pgmap v18178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:30.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:29 smithi090 ceph-mon[29180]: pgmap v18178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33072.495813 secs 2024-04-27T03:46:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33072.495813 secs 2024-04-27T03:46:31.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33072.495813 secs 2024-04-27T03:46:32.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:31 smithi090 ceph-mon[29180]: pgmap v18179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:32.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33073.362054 secs 2024-04-27T03:46:32.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:31 smithi119 ceph-mon[32752]: pgmap v18179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:32.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33073.362054 secs 2024-04-27T03:46:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:31 smithi167 ceph-mon[32751]: pgmap v18179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33073.362054 secs 2024-04-27T03:46:34.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:33 smithi119 ceph-mon[32752]: pgmap v18180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:34.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:33 smithi167 ceph-mon[32751]: pgmap v18180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:34.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:33 smithi090 ceph-mon[29180]: pgmap v18180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:35 smithi119 ceph-mon[32752]: pgmap v18181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:36.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:35 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33077.495893 secs 2024-04-27T03:46:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:35 smithi167 ceph-mon[32751]: pgmap v18181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:36.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:35 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33077.495893 secs 2024-04-27T03:46:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:35 smithi090 ceph-mon[29180]: pgmap v18181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:36.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:35 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33077.495893 secs 2024-04-27T03:46:37.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33078.362198 secs 2024-04-27T03:46:37.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33078.362198 secs 2024-04-27T03:46:37.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33078.362198 secs 2024-04-27T03:46:38.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:37 smithi119 ceph-mon[32752]: pgmap v18182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:38.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:37 smithi167 ceph-mon[32751]: pgmap v18182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:38.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:37 smithi090 ceph-mon[29180]: pgmap v18182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:40.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:39 smithi119 ceph-mon[32752]: pgmap v18183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:40.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:39 smithi167 ceph-mon[32751]: pgmap v18183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:40.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:39 smithi090 ceph-mon[29180]: pgmap v18183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:40 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33082.495968 secs 2024-04-27T03:46:41.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:40 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33082.495968 secs 2024-04-27T03:46:41.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:40 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33082.495968 secs 2024-04-27T03:46:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:41 smithi119 ceph-mon[32752]: pgmap v18184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33083.362333 secs 2024-04-27T03:46:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:41 smithi167 ceph-mon[32751]: pgmap v18184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:42.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33083.362333 secs 2024-04-27T03:46:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:41 smithi090 ceph-mon[29180]: pgmap v18184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:42.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33083.362333 secs 2024-04-27T03:46:44.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:43 smithi119 ceph-mon[32752]: pgmap v18185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:43 smithi167 ceph-mon[32751]: pgmap v18185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:44.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:43 smithi090 ceph-mon[29180]: pgmap v18185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:45 smithi119 ceph-mon[32752]: pgmap v18186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:45 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33087.496054 secs 2024-04-27T03:46:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:45 smithi167 ceph-mon[32751]: pgmap v18186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:45 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33087.496054 secs 2024-04-27T03:46:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:45 smithi090 ceph-mon[29180]: pgmap v18186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:46.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:45 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33087.496054 secs 2024-04-27T03:46:46.961 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33088.362459 secs 2024-04-27T03:46:46.975 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33088.362459 secs 2024-04-27T03:46:47.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33088.362459 secs 2024-04-27T03:46:48.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:47 smithi119 ceph-mon[32752]: pgmap v18187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:48.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:47 smithi167 ceph-mon[32751]: pgmap v18187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:48.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:47 smithi090 ceph-mon[29180]: pgmap v18187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:50.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:49 smithi119 ceph-mon[32752]: pgmap v18188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:50.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:49 smithi167 ceph-mon[32751]: pgmap v18188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:50.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:49 smithi090 ceph-mon[29180]: pgmap v18188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:46:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:50 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33092.496141 secs 2024-04-27T03:46:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:46:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:50 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33092.496141 secs 2024-04-27T03:46:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:46:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:50 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33092.496141 secs 2024-04-27T03:46:51.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:46:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:51 smithi119 ceph-mon[32752]: pgmap v18189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33093.362523 secs 2024-04-27T03:46:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:46:52.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:46:52.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:46:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:51 smithi167 ceph-mon[32751]: pgmap v18189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33093.362523 secs 2024-04-27T03:46:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:46:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:46:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:46:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:51 smithi090 ceph-mon[29180]: pgmap v18189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:46:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33093.362523 secs 2024-04-27T03:46:52.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:46:52.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:46:52.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:46:53.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:52 smithi119 ceph-mon[32752]: pgmap v18190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:46:53.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:52 smithi119 ceph-mon[32752]: pgmap v18191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:52 smithi167 ceph-mon[32751]: pgmap v18190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:46:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:52 smithi167 ceph-mon[32751]: pgmap v18191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:53.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:52 smithi090 ceph-mon[29180]: pgmap v18190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:46:53.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:52 smithi090 ceph-mon[29180]: pgmap v18191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:55.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:54 smithi119 ceph-mon[32752]: pgmap v18192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:55.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:54 smithi167 ceph-mon[32751]: pgmap v18192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:55.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:54 smithi090 ceph-mon[29180]: pgmap v18192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:55 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33097.496198 secs 2024-04-27T03:46:56.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:55 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33097.496198 secs 2024-04-27T03:46:56.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:55 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33097.496198 secs 2024-04-27T03:46:57.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:56 smithi119 ceph-mon[32752]: pgmap v18193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:57.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33098.362635 secs 2024-04-27T03:46:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:56 smithi167 ceph-mon[32751]: pgmap v18193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:57.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33098.362635 secs 2024-04-27T03:46:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:56 smithi090 ceph-mon[29180]: pgmap v18193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:46:57.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33098.362635 secs 2024-04-27T03:46:59.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:46:58 smithi119 ceph-mon[32752]: pgmap v18194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:46:59.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:46:58 smithi167 ceph-mon[32751]: pgmap v18194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:46:59.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:46:58 smithi090 ceph-mon[29180]: pgmap v18194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:47:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:00 smithi119 ceph-mon[32752]: pgmap v18195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:47:01.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:00 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33102.496282 secs 2024-04-27T03:47:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:00 smithi167 ceph-mon[32751]: pgmap v18195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:47:01.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:00 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33102.496282 secs 2024-04-27T03:47:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:00 smithi090 ceph-mon[29180]: pgmap v18195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:47:01.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:00 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33102.496282 secs 2024-04-27T03:47:02.034 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33103.362757 secs 2024-04-27T03:47:02.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33103.362757 secs 2024-04-27T03:47:02.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33103.362757 secs 2024-04-27T03:47:03.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:02 smithi119 ceph-mon[32752]: pgmap v18196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:47:03.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:02 smithi167 ceph-mon[32751]: pgmap v18196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:47:03.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:02 smithi090 ceph-mon[29180]: pgmap v18196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:47:05.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:04 smithi119 ceph-mon[32752]: pgmap v18197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:05.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:04 smithi167 ceph-mon[32751]: pgmap v18197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:05.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:04 smithi090 ceph-mon[29180]: pgmap v18197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:06.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:05 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33107.496360 secs 2024-04-27T03:47:06.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:05 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33107.496360 secs 2024-04-27T03:47:06.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:05 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33107.496360 secs 2024-04-27T03:47:07.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:06 smithi119 ceph-mon[32752]: pgmap v18198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:07.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33108.362834 secs 2024-04-27T03:47:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:06 smithi167 ceph-mon[32751]: pgmap v18198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:07.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33108.362834 secs 2024-04-27T03:47:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:06 smithi090 ceph-mon[29180]: pgmap v18198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:07.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33108.362834 secs 2024-04-27T03:47:09.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:08 smithi119 ceph-mon[32752]: pgmap v18199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:09.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:08 smithi167 ceph-mon[32751]: pgmap v18199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:09.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:08 smithi090 ceph-mon[29180]: pgmap v18199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:10 smithi119 ceph-mon[32752]: pgmap v18200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:11.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:10 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33112.496443 secs 2024-04-27T03:47:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:10 smithi167 ceph-mon[32751]: pgmap v18200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:11.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:10 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33112.496443 secs 2024-04-27T03:47:11.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:10 smithi090 ceph-mon[29180]: pgmap v18200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:11.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:10 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33112.496443 secs 2024-04-27T03:47:12.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33113.362952 secs 2024-04-27T03:47:12.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33113.362952 secs 2024-04-27T03:47:12.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33113.362952 secs 2024-04-27T03:47:13.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:12 smithi119 ceph-mon[32752]: pgmap v18201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:13.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:12 smithi167 ceph-mon[32751]: pgmap v18201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:13.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:12 smithi090 ceph-mon[29180]: pgmap v18201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:15.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:14 smithi119 ceph-mon[32752]: pgmap v18202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:15.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:14 smithi167 ceph-mon[32751]: pgmap v18202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:15.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:14 smithi090 ceph-mon[29180]: pgmap v18202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:15 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33117.496524 secs 2024-04-27T03:47:16.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:15 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33117.496524 secs 2024-04-27T03:47:16.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:15 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33117.496524 secs 2024-04-27T03:47:17.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:16 smithi119 ceph-mon[32752]: pgmap v18203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:17.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33118.363069 secs 2024-04-27T03:47:17.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:16 smithi167 ceph-mon[32751]: pgmap v18203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:17.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33118.363069 secs 2024-04-27T03:47:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:16 smithi090 ceph-mon[29180]: pgmap v18203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:17.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33118.363069 secs 2024-04-27T03:47:19.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:18 smithi119 ceph-mon[32752]: pgmap v18204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:19.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:18 smithi167 ceph-mon[32751]: pgmap v18204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:19.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:18 smithi090 ceph-mon[29180]: pgmap v18204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:20 smithi119 ceph-mon[32752]: pgmap v18205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:21.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:20 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33122.496614 secs 2024-04-27T03:47:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:20 smithi167 ceph-mon[32751]: pgmap v18205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:21.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:20 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33122.496614 secs 2024-04-27T03:47:21.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:20 smithi090 ceph-mon[29180]: pgmap v18205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:21.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:20 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33122.496614 secs 2024-04-27T03:47:22.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33123.363205 secs 2024-04-27T03:47:22.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33123.363205 secs 2024-04-27T03:47:22.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33123.363205 secs 2024-04-27T03:47:23.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:22 smithi119 ceph-mon[32752]: pgmap v18206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:23.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:22 smithi167 ceph-mon[32751]: pgmap v18206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:23.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:22 smithi090 ceph-mon[29180]: pgmap v18206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:25.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:24 smithi119 ceph-mon[32752]: pgmap v18207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:25.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:24 smithi167 ceph-mon[32751]: pgmap v18207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:25.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:24 smithi090 ceph-mon[29180]: pgmap v18207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:26.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:25 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33127.496692 secs 2024-04-27T03:47:26.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:25 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33127.496692 secs 2024-04-27T03:47:26.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:25 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33127.496692 secs 2024-04-27T03:47:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:26 smithi119 ceph-mon[32752]: pgmap v18208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:27.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33128.363320 secs 2024-04-27T03:47:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:26 smithi167 ceph-mon[32751]: pgmap v18208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:27.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33128.363320 secs 2024-04-27T03:47:27.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:26 smithi090 ceph-mon[29180]: pgmap v18208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:27.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33128.363320 secs 2024-04-27T03:47:29.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:28 smithi119 ceph-mon[32752]: pgmap v18209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:29.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:28 smithi167 ceph-mon[32751]: pgmap v18209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:29.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:28 smithi090 ceph-mon[29180]: pgmap v18209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:30 smithi119 ceph-mon[32752]: pgmap v18210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:31.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:30 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33132.496746 secs 2024-04-27T03:47:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:30 smithi167 ceph-mon[32751]: pgmap v18210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:31.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:30 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33132.496746 secs 2024-04-27T03:47:31.133 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:30 smithi090 ceph-mon[29180]: pgmap v18210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:31.134 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:30 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33132.496746 secs 2024-04-27T03:47:32.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33133.363440 secs 2024-04-27T03:47:32.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33133.363440 secs 2024-04-27T03:47:32.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33133.363440 secs 2024-04-27T03:47:33.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:32 smithi167 ceph-mon[32751]: pgmap v18211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:33.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:32 smithi090 ceph-mon[29180]: pgmap v18211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:33.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:32 smithi119 ceph-mon[32752]: pgmap v18211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:35.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:34 smithi167 ceph-mon[32751]: pgmap v18212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:35.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:34 smithi090 ceph-mon[29180]: pgmap v18212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:35.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:34 smithi119 ceph-mon[32752]: pgmap v18212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:36.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:36 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33137.496850 secs 2024-04-27T03:47:36.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:36 smithi119 ceph-mon[32752]: pgmap v18213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:36.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:36 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33137.496850 secs 2024-04-27T03:47:36.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:36 smithi167 ceph-mon[32751]: pgmap v18213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:36.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:36 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33137.496850 secs 2024-04-27T03:47:36.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:36 smithi090 ceph-mon[29180]: pgmap v18213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:37.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33138.363503 secs 2024-04-27T03:47:37.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33138.363503 secs 2024-04-27T03:47:37.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33138.363503 secs 2024-04-27T03:47:38.560 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:38 smithi119 ceph-mon[32752]: pgmap v18214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:38.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:38 smithi167 ceph-mon[32751]: pgmap v18214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:38.632 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:38 smithi090 ceph-mon[29180]: pgmap v18214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:40.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:40 smithi119 ceph-mon[32752]: pgmap v18215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:40.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:40 smithi167 ceph-mon[32751]: pgmap v18215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:40 smithi090 ceph-mon[29180]: pgmap v18215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:41.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:41 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33142.496910 secs 2024-04-27T03:47:41.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33143.363623 secs 2024-04-27T03:47:41.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:41 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33142.496910 secs 2024-04-27T03:47:41.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33143.363623 secs 2024-04-27T03:47:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:41 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33142.496910 secs 2024-04-27T03:47:41.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33143.363623 secs 2024-04-27T03:47:42.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:42 smithi119 ceph-mon[32752]: pgmap v18216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:42.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:42 smithi167 ceph-mon[32751]: pgmap v18216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:42.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:42 smithi090 ceph-mon[29180]: pgmap v18216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:44.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:44 smithi119 ceph-mon[32752]: pgmap v18217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:44.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:44 smithi167 ceph-mon[32751]: pgmap v18217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:44.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:44 smithi090 ceph-mon[29180]: pgmap v18217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:46.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:46 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33147.496991 secs 2024-04-27T03:47:46.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:46 smithi119 ceph-mon[32752]: pgmap v18218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:46.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33148.363740 secs 2024-04-27T03:47:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:46 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33147.496991 secs 2024-04-27T03:47:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:46 smithi167 ceph-mon[32751]: pgmap v18218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:46.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33148.363740 secs 2024-04-27T03:47:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:46 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33147.496991 secs 2024-04-27T03:47:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:46 smithi090 ceph-mon[29180]: pgmap v18218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:46.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33148.363740 secs 2024-04-27T03:47:48.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:48 smithi119 ceph-mon[32752]: pgmap v18219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:48.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:48 smithi167 ceph-mon[32751]: pgmap v18219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:48.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:48 smithi090 ceph-mon[29180]: pgmap v18219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:50.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:50 smithi119 ceph-mon[32752]: pgmap v18220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:50.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:50 smithi167 ceph-mon[32751]: pgmap v18220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:50 smithi090 ceph-mon[29180]: pgmap v18220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:47:51.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:51 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33152.497080 secs 2024-04-27T03:47:51.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33153.363803 secs 2024-04-27T03:47:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:51 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33152.497080 secs 2024-04-27T03:47:51.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33153.363803 secs 2024-04-27T03:47:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:51 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33152.497080 secs 2024-04-27T03:47:51.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33153.363803 secs 2024-04-27T03:47:52.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:52 smithi119 ceph-mon[32752]: pgmap v18221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:52.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:47:52.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:47:52.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:47:52.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:47:52.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:52 smithi167 ceph-mon[32751]: pgmap v18221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:52.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:47:52.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:47:52.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:47:52.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:47:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:52 smithi090 ceph-mon[29180]: pgmap v18221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:47:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:47:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:47:52.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:47:52.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:47:53.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:53 smithi119 ceph-mon[32752]: pgmap v18222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:53.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:53 smithi167 ceph-mon[32751]: pgmap v18222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:53.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:53 smithi090 ceph-mon[29180]: pgmap v18222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:55.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:55 smithi119 ceph-mon[32752]: pgmap v18223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:55.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:55 smithi167 ceph-mon[32751]: pgmap v18223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:55.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:55 smithi090 ceph-mon[29180]: pgmap v18223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:56 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33157.497140 secs 2024-04-27T03:47:56.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33158.363921 secs 2024-04-27T03:47:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:56 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33157.497140 secs 2024-04-27T03:47:56.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33158.363921 secs 2024-04-27T03:47:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:56 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33157.497140 secs 2024-04-27T03:47:56.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33158.363921 secs 2024-04-27T03:47:57.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:57 smithi119 ceph-mon[32752]: pgmap v18224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:57.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:57 smithi167 ceph-mon[32751]: pgmap v18224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:57.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:57 smithi090 ceph-mon[29180]: pgmap v18224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:59.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:47:59 smithi119 ceph-mon[32752]: pgmap v18225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:59.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:47:59 smithi167 ceph-mon[32751]: pgmap v18225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:47:59.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:47:59 smithi090 ceph-mon[29180]: pgmap v18225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:48:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:01 smithi119 ceph-mon[32752]: pgmap v18226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:48:01.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:01 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33162.497201 secs 2024-04-27T03:48:01.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33163.364063 secs 2024-04-27T03:48:01.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:01 smithi167 ceph-mon[32751]: pgmap v18226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:48:01.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:01 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33162.497201 secs 2024-04-27T03:48:01.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33163.364063 secs 2024-04-27T03:48:01.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:01 smithi090 ceph-mon[29180]: pgmap v18226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:48:01.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:01 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33162.497201 secs 2024-04-27T03:48:01.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33163.364063 secs 2024-04-27T03:48:03.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:03 smithi119 ceph-mon[32752]: pgmap v18227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:48:03.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:03 smithi167 ceph-mon[32751]: pgmap v18227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:48:03.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:03 smithi090 ceph-mon[29180]: pgmap v18227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:48:05.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:05 smithi119 ceph-mon[32752]: pgmap v18228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:05.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:05 smithi167 ceph-mon[32751]: pgmap v18228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:05.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:05 smithi090 ceph-mon[29180]: pgmap v18228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:06.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:06 smithi119 ceph-mon[32752]: 43 slow requests, 0 included below; oldest blocked for > 33167.497282 secs 2024-04-27T03:48:06.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33168.364197 secs 2024-04-27T03:48:06.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:06 smithi167 ceph-mon[32751]: 43 slow requests, 0 included below; oldest blocked for > 33167.497282 secs 2024-04-27T03:48:06.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33168.364197 secs 2024-04-27T03:48:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:06 smithi090 ceph-mon[29180]: 43 slow requests, 0 included below; oldest blocked for > 33167.497282 secs 2024-04-27T03:48:06.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33168.364197 secs 2024-04-27T03:48:07.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:07 smithi119 ceph-mon[32752]: pgmap v18229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:07.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:07 smithi167 ceph-mon[32751]: pgmap v18229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:07.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:07 smithi090 ceph-mon[29180]: pgmap v18229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:09.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:09 smithi119 ceph-mon[32752]: pgmap v18230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:09.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:09 smithi167 ceph-mon[32751]: pgmap v18230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:09.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:09 smithi090 ceph-mon[29180]: pgmap v18230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:10.383 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:48:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:48:09.957+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:48:10.383 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:48:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:48:09.957+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:48:10.383 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:48:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:48:09.957+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:48:10.383 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:48:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:48:09.957+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:48:10.383 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:48:09 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:48:09.957+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:48:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:11 smithi119 ceph-mon[32752]: pgmap v18231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33172.497367 secs 2024-04-27T03:48:11.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33173.364305 secs 2024-04-27T03:48:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:11 smithi167 ceph-mon[32751]: pgmap v18231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:11.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33172.497367 secs 2024-04-27T03:48:11.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33173.364305 secs 2024-04-27T03:48:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:11 smithi090 ceph-mon[29180]: pgmap v18231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:11.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33172.497367 secs 2024-04-27T03:48:11.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33173.364305 secs 2024-04-27T03:48:13.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:13 smithi119 ceph-mon[32752]: pgmap v18232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:13.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:13 smithi167 ceph-mon[32751]: pgmap v18232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:13.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:13 smithi090 ceph-mon[29180]: pgmap v18232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:15.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:15 smithi119 ceph-mon[32752]: pgmap v18233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:15.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:15 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:48:15.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:15 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:48:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:15 smithi167 ceph-mon[32751]: pgmap v18233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:15 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:48:15.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:15 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:48:15.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:15 smithi090 ceph-mon[29180]: pgmap v18233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:15.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:15 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:48:15.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:15 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:48:16.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:16 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33177.497451 secs 2024-04-27T03:48:16.726 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33178.364424 secs 2024-04-27T03:48:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:16 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33177.497451 secs 2024-04-27T03:48:16.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33178.364424 secs 2024-04-27T03:48:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:16 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33177.497451 secs 2024-04-27T03:48:16.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33178.364424 secs 2024-04-27T03:48:17.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:17 smithi119 ceph-mon[32752]: pgmap v18234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:17.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:17 smithi167 ceph-mon[32751]: pgmap v18234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:17.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:17 smithi090 ceph-mon[29180]: pgmap v18234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:19.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:19 smithi119 ceph-mon[32752]: pgmap v18235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:19.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:19 smithi167 ceph-mon[32751]: pgmap v18235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:19.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:19 smithi090 ceph-mon[29180]: pgmap v18235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:21.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:21 smithi119 ceph-mon[32752]: pgmap v18236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:21.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:21 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33182.497538 secs 2024-04-27T03:48:21.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33183.364503 secs 2024-04-27T03:48:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:21 smithi167 ceph-mon[32751]: pgmap v18236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:21 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33182.497538 secs 2024-04-27T03:48:21.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33183.364503 secs 2024-04-27T03:48:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:21 smithi090 ceph-mon[29180]: pgmap v18236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:21 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33182.497538 secs 2024-04-27T03:48:21.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33183.364503 secs 2024-04-27T03:48:23.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:23 smithi119 ceph-mon[32752]: pgmap v18237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:23.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:23 smithi167 ceph-mon[32751]: pgmap v18237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:23.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:23 smithi090 ceph-mon[29180]: pgmap v18237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 511 B/s wr, 2 op/s 2024-04-27T03:48:25.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:25 smithi119 ceph-mon[32752]: pgmap v18238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:25.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:25 smithi167 ceph-mon[32751]: pgmap v18238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:25.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:25 smithi090 ceph-mon[29180]: pgmap v18238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-27T03:48:26.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:26 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33187.497617 secs 2024-04-27T03:48:26.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33188.364645 secs 2024-04-27T03:48:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:26 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33187.497617 secs 2024-04-27T03:48:26.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33188.364645 secs 2024-04-27T03:48:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:26 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33187.497617 secs 2024-04-27T03:48:26.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33188.364645 secs 2024-04-27T03:48:27.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:27 smithi119 ceph-mon[32752]: pgmap v18239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:27.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:27 smithi167 ceph-mon[32751]: pgmap v18239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:27.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:27 smithi090 ceph-mon[29180]: pgmap v18239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:29.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:29 smithi119 ceph-mon[32752]: pgmap v18240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:29.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:29 smithi167 ceph-mon[32751]: pgmap v18240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:29.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:29 smithi090 ceph-mon[29180]: pgmap v18240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:31 smithi119 ceph-mon[32752]: pgmap v18241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:31 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33192.497700 secs 2024-04-27T03:48:31.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33193.364756 secs 2024-04-27T03:48:31.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:31 smithi167 ceph-mon[32751]: pgmap v18241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:31.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:31 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33192.497700 secs 2024-04-27T03:48:31.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33193.364756 secs 2024-04-27T03:48:31.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:31 smithi090 ceph-mon[29180]: pgmap v18241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:31.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:31 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33192.497700 secs 2024-04-27T03:48:31.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33193.364756 secs 2024-04-27T03:48:33.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:33 smithi119 ceph-mon[32752]: pgmap v18242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:33.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:33 smithi167 ceph-mon[32751]: pgmap v18242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:33.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:33 smithi090 ceph-mon[29180]: pgmap v18242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:35.811 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:35 smithi119 ceph-mon[32752]: pgmap v18243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:35.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:35 smithi167 ceph-mon[32751]: pgmap v18243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:35.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:35 smithi090 ceph-mon[29180]: pgmap v18243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:36 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33197.497781 secs 2024-04-27T03:48:36.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33198.364818 secs 2024-04-27T03:48:36.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:36 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33197.497781 secs 2024-04-27T03:48:36.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33198.364818 secs 2024-04-27T03:48:36.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:36 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33197.497781 secs 2024-04-27T03:48:36.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33198.364818 secs 2024-04-27T03:48:37.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:37 smithi119 ceph-mon[32752]: pgmap v18244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:37.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:37 smithi167 ceph-mon[32751]: pgmap v18244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:37.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:37 smithi090 ceph-mon[29180]: pgmap v18244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:39.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:39 smithi119 ceph-mon[32752]: pgmap v18245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:39.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:39 smithi167 ceph-mon[32751]: pgmap v18245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:39.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:39 smithi090 ceph-mon[29180]: pgmap v18245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:40.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33202.497871 secs 2024-04-27T03:48:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33202.497871 secs 2024-04-27T03:48:41.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33202.497871 secs 2024-04-27T03:48:41.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:41 smithi167 ceph-mon[32751]: pgmap v18246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:41.825 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33203.364898 secs 2024-04-27T03:48:41.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:41 smithi090 ceph-mon[29180]: pgmap v18246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:41.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33203.364898 secs 2024-04-27T03:48:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:41 smithi119 ceph-mon[32752]: pgmap v18246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:42.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33203.364898 secs 2024-04-27T03:48:43.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:43 smithi090 ceph-mon[29180]: pgmap v18247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:44.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:43 smithi119 ceph-mon[32752]: pgmap v18247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:44.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:43 smithi167 ceph-mon[32751]: pgmap v18247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:45.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:45 smithi090 ceph-mon[29180]: pgmap v18248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:45.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:45 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33207.497954 secs 2024-04-27T03:48:46.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:45 smithi119 ceph-mon[32752]: pgmap v18248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:46.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:45 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33207.497954 secs 2024-04-27T03:48:46.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:45 smithi167 ceph-mon[32751]: pgmap v18248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:46.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:45 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33207.497954 secs 2024-04-27T03:48:46.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33208.364960 secs 2024-04-27T03:48:46.961 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33208.364960 secs 2024-04-27T03:48:46.976 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33208.364960 secs 2024-04-27T03:48:47.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:47 smithi090 ceph-mon[29180]: pgmap v18249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:48.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:47 smithi119 ceph-mon[32752]: pgmap v18249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:48.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:47 smithi167 ceph-mon[32751]: pgmap v18249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:49.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:49 smithi090 ceph-mon[29180]: pgmap v18250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:50.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:49 smithi119 ceph-mon[32752]: pgmap v18250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:50.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:49 smithi167 ceph-mon[32751]: pgmap v18250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:50.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33212.498038 secs 2024-04-27T03:48:51.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33212.498038 secs 2024-04-27T03:48:51.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33212.498038 secs 2024-04-27T03:48:51.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:51 smithi090 ceph-mon[29180]: pgmap v18251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:51.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33213.365105 secs 2024-04-27T03:48:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:51 smithi119 ceph-mon[32752]: pgmap v18251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:52.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33213.365105 secs 2024-04-27T03:48:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:51 smithi167 ceph-mon[32751]: pgmap v18251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:48:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33213.365105 secs 2024-04-27T03:48:52.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:48:53.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:48:53.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:48:53.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:53 smithi090 ceph-mon[29180]: pgmap v18252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:53.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:48:53.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:48:53.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:53 smithi090 ceph-mon[29180]: pgmap v18253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:48:53.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:53 smithi090 ceph-mon[29180]: pgmap v18254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:48:53.883 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:48:54.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:53 smithi119 ceph-mon[32752]: pgmap v18252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:54.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:48:54.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:48:54.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:53 smithi119 ceph-mon[32752]: pgmap v18253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:48:54.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:53 smithi119 ceph-mon[32752]: pgmap v18254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:48:54.062 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:48:54.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:53 smithi167 ceph-mon[32751]: pgmap v18252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:48:54.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:48:54.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:48:54.075 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:53 smithi167 ceph-mon[32751]: pgmap v18253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:48:54.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:53 smithi167 ceph-mon[32751]: pgmap v18254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:48:54.076 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:48:56.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:55 smithi119 ceph-mon[32752]: pgmap v18255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:48:56.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:55 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33217.498117 secs 2024-04-27T03:48:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:55 smithi167 ceph-mon[32751]: pgmap v18255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:48:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:55 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33217.498117 secs 2024-04-27T03:48:56.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:55 smithi090 ceph-mon[29180]: pgmap v18255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:48:56.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:55 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33217.498117 secs 2024-04-27T03:48:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33218.365242 secs 2024-04-27T03:48:57.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33218.365242 secs 2024-04-27T03:48:57.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33218.365242 secs 2024-04-27T03:48:58.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:57 smithi119 ceph-mon[32752]: pgmap v18256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:48:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:57 smithi167 ceph-mon[32751]: pgmap v18256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:48:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:57 smithi090 ceph-mon[29180]: pgmap v18256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:49:00.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:48:59 smithi119 ceph-mon[32752]: pgmap v18257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:49:00.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:48:59 smithi167 ceph-mon[32751]: pgmap v18257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:49:00.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:48:59 smithi090 ceph-mon[29180]: pgmap v18257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:49:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33222.498203 secs 2024-04-27T03:49:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33222.498203 secs 2024-04-27T03:49:01.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33222.498203 secs 2024-04-27T03:49:02.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:01 smithi119 ceph-mon[32752]: pgmap v18258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:49:02.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33223.365356 secs 2024-04-27T03:49:02.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:01 smithi167 ceph-mon[32751]: pgmap v18258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:49:02.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33223.365356 secs 2024-04-27T03:49:02.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:01 smithi090 ceph-mon[29180]: pgmap v18258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:49:02.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33223.365356 secs 2024-04-27T03:49:04.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:03 smithi119 ceph-mon[32752]: pgmap v18259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:49:04.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:03 smithi167 ceph-mon[32751]: pgmap v18259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:49:04.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:03 smithi090 ceph-mon[29180]: pgmap v18259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:49:06.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:05 smithi119 ceph-mon[32752]: pgmap v18260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:06.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:05 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33227.498288 secs 2024-04-27T03:49:06.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:05 smithi167 ceph-mon[32751]: pgmap v18260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:06.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:05 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33227.498288 secs 2024-04-27T03:49:06.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:05 smithi090 ceph-mon[29180]: pgmap v18260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:06.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:05 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33227.498288 secs 2024-04-27T03:49:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33228.365468 secs 2024-04-27T03:49:07.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33228.365468 secs 2024-04-27T03:49:07.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33228.365468 secs 2024-04-27T03:49:08.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:07 smithi119 ceph-mon[32752]: pgmap v18261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:08.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:07 smithi167 ceph-mon[32751]: pgmap v18261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:08.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:07 smithi090 ceph-mon[29180]: pgmap v18261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:10.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:09 smithi119 ceph-mon[32752]: pgmap v18262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:10.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:09 smithi167 ceph-mon[32751]: pgmap v18262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:10.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:09 smithi090 ceph-mon[29180]: pgmap v18262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:11.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:10 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33232.498373 secs 2024-04-27T03:49:11.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:10 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33232.498373 secs 2024-04-27T03:49:11.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:10 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33232.498373 secs 2024-04-27T03:49:12.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:11 smithi119 ceph-mon[32752]: pgmap v18263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:12.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:11 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 33233.365591 secs 2024-04-27T03:49:12.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:11 smithi119 ceph-mon[32752]: slow request 361.218159 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:49:12.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:11 smithi167 ceph-mon[32751]: pgmap v18263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:12.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:11 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 33233.365591 secs 2024-04-27T03:49:12.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:11 smithi167 ceph-mon[32751]: slow request 361.218159 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:49:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:11 smithi090 ceph-mon[29180]: pgmap v18263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:11 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 33233.365591 secs 2024-04-27T03:49:12.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:11 smithi090 ceph-mon[29180]: slow request 361.218159 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:49:13.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:13 smithi119 ceph-mon[32752]: pgmap v18264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:13.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:13 smithi167 ceph-mon[32751]: pgmap v18264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:13.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:13 smithi090 ceph-mon[29180]: pgmap v18264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:16.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:15 smithi119 ceph-mon[32752]: pgmap v18265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:16.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:15 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33237.498445 secs 2024-04-27T03:49:16.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:15 smithi167 ceph-mon[32751]: pgmap v18265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:16.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:15 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33237.498445 secs 2024-04-27T03:49:16.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:15 smithi090 ceph-mon[29180]: pgmap v18265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:16.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:15 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33237.498445 secs 2024-04-27T03:49:17.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33238.365657 secs 2024-04-27T03:49:17.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33238.365657 secs 2024-04-27T03:49:17.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33238.365657 secs 2024-04-27T03:49:18.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:17 smithi119 ceph-mon[32752]: pgmap v18266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:18.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:17 smithi167 ceph-mon[32751]: pgmap v18266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:18.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:17 smithi090 ceph-mon[29180]: pgmap v18266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:20.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:19 smithi119 ceph-mon[32752]: pgmap v18267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:20.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:19 smithi167 ceph-mon[32751]: pgmap v18267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:20.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:19 smithi090 ceph-mon[29180]: pgmap v18267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:21.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:20 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33242.498531 secs 2024-04-27T03:49:21.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:20 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33242.498531 secs 2024-04-27T03:49:21.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:20 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33242.498531 secs 2024-04-27T03:49:22.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:21 smithi119 ceph-mon[32752]: pgmap v18268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:22.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33243.365782 secs 2024-04-27T03:49:22.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:21 smithi167 ceph-mon[32751]: pgmap v18268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:22.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33243.365782 secs 2024-04-27T03:49:22.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:21 smithi090 ceph-mon[29180]: pgmap v18268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:22.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33243.365782 secs 2024-04-27T03:49:24.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:23 smithi119 ceph-mon[32752]: pgmap v18269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:24.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:23 smithi167 ceph-mon[32751]: pgmap v18269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:24.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:23 smithi090 ceph-mon[29180]: pgmap v18269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:26.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:25 smithi119 ceph-mon[32752]: pgmap v18270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:26.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:25 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33247.498613 secs 2024-04-27T03:49:26.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:25 smithi167 ceph-mon[32751]: pgmap v18270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:26.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:25 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33247.498613 secs 2024-04-27T03:49:26.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:25 smithi090 ceph-mon[29180]: pgmap v18270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:26.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:25 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33247.498613 secs 2024-04-27T03:49:27.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33248.365897 secs 2024-04-27T03:49:27.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33248.365897 secs 2024-04-27T03:49:27.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33248.365897 secs 2024-04-27T03:49:28.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:27 smithi119 ceph-mon[32752]: pgmap v18271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:28.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:27 smithi167 ceph-mon[32751]: pgmap v18271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:28.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:27 smithi090 ceph-mon[29180]: pgmap v18271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:30.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:29 smithi119 ceph-mon[32752]: pgmap v18272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:30.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:29 smithi167 ceph-mon[32751]: pgmap v18272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:30.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:29 smithi090 ceph-mon[29180]: pgmap v18272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:31.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33252.498700 secs 2024-04-27T03:49:31.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33252.498700 secs 2024-04-27T03:49:31.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33252.498700 secs 2024-04-27T03:49:32.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:31 smithi119 ceph-mon[32752]: pgmap v18273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:32.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33253.366035 secs 2024-04-27T03:49:32.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:31 smithi167 ceph-mon[32751]: pgmap v18273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:32.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33253.366035 secs 2024-04-27T03:49:32.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:31 smithi090 ceph-mon[29180]: pgmap v18273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:32.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33253.366035 secs 2024-04-27T03:49:33.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:32 smithi119 ceph-mon[32752]: pgmap v18274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:33.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:32 smithi167 ceph-mon[32751]: pgmap v18274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:33.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:32 smithi090 ceph-mon[29180]: pgmap v18274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:35 smithi119 ceph-mon[32752]: pgmap v18275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:36.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:35 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33257.498782 secs 2024-04-27T03:49:36.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:35 smithi167 ceph-mon[32751]: pgmap v18275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:36.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:35 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33257.498782 secs 2024-04-27T03:49:36.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:35 smithi090 ceph-mon[29180]: pgmap v18275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:36.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:35 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33257.498782 secs 2024-04-27T03:49:37.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33258.366172 secs 2024-04-27T03:49:37.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33258.366172 secs 2024-04-27T03:49:37.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33258.366172 secs 2024-04-27T03:49:38.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:37 smithi119 ceph-mon[32752]: pgmap v18276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:38.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:37 smithi167 ceph-mon[32751]: pgmap v18276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:38.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:37 smithi090 ceph-mon[29180]: pgmap v18276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:40.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:39 smithi119 ceph-mon[32752]: pgmap v18277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:40.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:39 smithi167 ceph-mon[32751]: pgmap v18277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:40.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:39 smithi090 ceph-mon[29180]: pgmap v18277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:41.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33262.498866 secs 2024-04-27T03:49:41.325 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33262.498866 secs 2024-04-27T03:49:41.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33262.498866 secs 2024-04-27T03:49:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:41 smithi119 ceph-mon[32752]: pgmap v18278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:42.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33263.366271 secs 2024-04-27T03:49:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:41 smithi167 ceph-mon[32751]: pgmap v18278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:42.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33263.366271 secs 2024-04-27T03:49:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:41 smithi090 ceph-mon[29180]: pgmap v18278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:42.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33263.366271 secs 2024-04-27T03:49:44.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:43 smithi119 ceph-mon[32752]: pgmap v18279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:44.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:43 smithi167 ceph-mon[32751]: pgmap v18279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:44.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:43 smithi090 ceph-mon[29180]: pgmap v18279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:46.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:45 smithi119 ceph-mon[32752]: pgmap v18280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:46.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:45 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33267.498924 secs 2024-04-27T03:49:46.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:45 smithi167 ceph-mon[32751]: pgmap v18280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:46.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:45 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33267.498924 secs 2024-04-27T03:49:46.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:45 smithi090 ceph-mon[29180]: pgmap v18280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:46.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:45 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33267.498924 secs 2024-04-27T03:49:47.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33268.366409 secs 2024-04-27T03:49:47.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33268.366409 secs 2024-04-27T03:49:47.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33268.366409 secs 2024-04-27T03:49:48.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:47 smithi119 ceph-mon[32752]: pgmap v18281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:48.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:47 smithi167 ceph-mon[32751]: pgmap v18281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:48.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:47 smithi090 ceph-mon[29180]: pgmap v18281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:50.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:49 smithi119 ceph-mon[32752]: pgmap v18282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:50.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:49 smithi167 ceph-mon[32751]: pgmap v18282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:50.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:49 smithi090 ceph-mon[29180]: pgmap v18282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33272.498988 secs 2024-04-27T03:49:51.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:50 smithi119 ceph-mon[32752]: pgmap v18283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33272.498988 secs 2024-04-27T03:49:51.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:50 smithi167 ceph-mon[32751]: pgmap v18283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:51.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33272.498988 secs 2024-04-27T03:49:51.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:50 smithi090 ceph-mon[29180]: pgmap v18283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:52.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33273.366535 secs 2024-04-27T03:49:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33273.366535 secs 2024-04-27T03:49:52.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33273.366535 secs 2024-04-27T03:49:53.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:53 smithi119 ceph-mon[32752]: pgmap v18284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:53.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:49:53.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:53 smithi167 ceph-mon[32751]: pgmap v18284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:53.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:49:53.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:53 smithi090 ceph-mon[29180]: pgmap v18284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:53.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:49:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:55 smithi119 ceph-mon[32752]: pgmap v18285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:56.060 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:55 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33277.499050 secs 2024-04-27T03:49:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:55 smithi167 ceph-mon[32751]: pgmap v18285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:56.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:55 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33277.499050 secs 2024-04-27T03:49:56.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:55 smithi090 ceph-mon[29180]: pgmap v18285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:49:56.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:55 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33277.499050 secs 2024-04-27T03:49:57.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33278.366600 secs 2024-04-27T03:49:57.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33278.366600 secs 2024-04-27T03:49:57.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33278.366600 secs 2024-04-27T03:49:58.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: pgmap v18286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:49:58.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:49:58.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: pgmap v18286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:49:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:49:58.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: pgmap v18286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:49:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:58.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:49:58.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:49:58.383 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:49:59.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:58 smithi119 ceph-mon[32752]: pgmap v18287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 B/s rd, 1 op/s 2024-04-27T03:49:59.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:49:58 smithi119 ceph-mon[32752]: pgmap v18288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 598 B/s rd, 1 op/s 2024-04-27T03:49:59.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:58 smithi167 ceph-mon[32751]: pgmap v18287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 B/s rd, 1 op/s 2024-04-27T03:49:59.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:49:58 smithi167 ceph-mon[32751]: pgmap v18288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 598 B/s rd, 1 op/s 2024-04-27T03:49:59.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:58 smithi090 ceph-mon[29180]: pgmap v18287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 970 B/s rd, 1 op/s 2024-04-27T03:49:59.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:49:58 smithi090 ceph-mon[29180]: pgmap v18288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 598 B/s rd, 1 op/s 2024-04-27T03:50:01.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:00 smithi119 ceph-mon[32752]: pgmap v18289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:50:01.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33282.499111 secs 2024-04-27T03:50:01.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:00 smithi167 ceph-mon[32751]: pgmap v18289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:50:01.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33282.499111 secs 2024-04-27T03:50:01.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:00 smithi090 ceph-mon[29180]: pgmap v18289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:01.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T03:50:01.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33282.499111 secs 2024-04-27T03:50:02.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33283.366742 secs 2024-04-27T03:50:02.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33283.366742 secs 2024-04-27T03:50:02.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33283.366742 secs 2024-04-27T03:50:03.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:02 smithi119 ceph-mon[32752]: pgmap v18290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:02 smithi167 ceph-mon[32751]: pgmap v18290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:03.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:02 smithi090 ceph-mon[29180]: pgmap v18290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:05.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:04 smithi119 ceph-mon[32752]: pgmap v18291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:05.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:04 smithi167 ceph-mon[32751]: pgmap v18291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:05.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:04 smithi090 ceph-mon[29180]: pgmap v18291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:06.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:05 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33287.499193 secs 2024-04-27T03:50:06.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:05 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33287.499193 secs 2024-04-27T03:50:06.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:05 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33287.499193 secs 2024-04-27T03:50:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:06 smithi119 ceph-mon[32752]: pgmap v18292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33288.366852 secs 2024-04-27T03:50:07.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:06 smithi167 ceph-mon[32751]: pgmap v18292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:07.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33288.366852 secs 2024-04-27T03:50:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:06 smithi090 ceph-mon[29180]: pgmap v18292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:07.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33288.366852 secs 2024-04-27T03:50:08.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:08 smithi119 ceph-mon[32752]: pgmap v18293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:50:08.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:08 smithi167 ceph-mon[32751]: pgmap v18293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:50:08.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:08 smithi090 ceph-mon[29180]: pgmap v18293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:50:10.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:10 smithi119 ceph-mon[32752]: pgmap v18294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:10.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:10 smithi167 ceph-mon[32751]: pgmap v18294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:10.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:10 smithi090 ceph-mon[29180]: pgmap v18294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:11 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33292.499276 secs 2024-04-27T03:50:11.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33293.366983 secs 2024-04-27T03:50:11.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:11 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33292.499276 secs 2024-04-27T03:50:11.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33293.366983 secs 2024-04-27T03:50:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:11 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33292.499276 secs 2024-04-27T03:50:11.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33293.366983 secs 2024-04-27T03:50:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:12 smithi119 ceph-mon[32752]: pgmap v18295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:12.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:12 smithi167 ceph-mon[32751]: pgmap v18295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:12.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:12 smithi090 ceph-mon[29180]: pgmap v18295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:14.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:14 smithi119 ceph-mon[32752]: pgmap v18296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:14.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:14 smithi167 ceph-mon[32751]: pgmap v18296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:14.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:14 smithi090 ceph-mon[29180]: pgmap v18296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:15.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:15 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33297.499358 secs 2024-04-27T03:50:15.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:15 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33297.499358 secs 2024-04-27T03:50:15.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:15 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33297.499358 secs 2024-04-27T03:50:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:16 smithi119 ceph-mon[32752]: pgmap v18297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:16.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33298.367105 secs 2024-04-27T03:50:16.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:16 smithi167 ceph-mon[32751]: pgmap v18297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:16.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33298.367105 secs 2024-04-27T03:50:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:16 smithi090 ceph-mon[29180]: pgmap v18297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:16.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33298.367105 secs 2024-04-27T03:50:18.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:18 smithi119 ceph-mon[32752]: pgmap v18298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:18.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:18 smithi167 ceph-mon[32751]: pgmap v18298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:18.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:18 smithi090 ceph-mon[29180]: pgmap v18298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:20 smithi119 ceph-mon[32752]: pgmap v18299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:20.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:20 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33302.499440 secs 2024-04-27T03:50:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:20 smithi167 ceph-mon[32751]: pgmap v18299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:20.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:20 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33302.499440 secs 2024-04-27T03:50:20.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:20 smithi090 ceph-mon[29180]: pgmap v18299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:20.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:20 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33302.499440 secs 2024-04-27T03:50:21.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33303.367220 secs 2024-04-27T03:50:21.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33303.367220 secs 2024-04-27T03:50:21.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33303.367220 secs 2024-04-27T03:50:22.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:22 smithi119 ceph-mon[32752]: pgmap v18300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:22.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:22 smithi167 ceph-mon[32751]: pgmap v18300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:22.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:22 smithi090 ceph-mon[29180]: pgmap v18300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:24.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:24 smithi119 ceph-mon[32752]: pgmap v18301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:24.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:24 smithi167 ceph-mon[32751]: pgmap v18301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:24.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:24 smithi090 ceph-mon[29180]: pgmap v18301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:25.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:25 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33307.499495 secs 2024-04-27T03:50:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:25 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33307.499495 secs 2024-04-27T03:50:26.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:25 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33307.499495 secs 2024-04-27T03:50:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:26 smithi167 ceph-mon[32751]: pgmap v18302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:26.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33308.367358 secs 2024-04-27T03:50:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:26 smithi090 ceph-mon[29180]: pgmap v18302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33308.367358 secs 2024-04-27T03:50:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:26 smithi119 ceph-mon[32752]: pgmap v18302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33308.367358 secs 2024-04-27T03:50:28.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:28 smithi167 ceph-mon[32751]: pgmap v18303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:28.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:28 smithi090 ceph-mon[29180]: pgmap v18303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:29.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:28 smithi119 ceph-mon[32752]: pgmap v18303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:30.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:30 smithi167 ceph-mon[32751]: pgmap v18304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:30.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:30 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33312.499573 secs 2024-04-27T03:50:30.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:30 smithi090 ceph-mon[29180]: pgmap v18304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:30.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:30 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33312.499573 secs 2024-04-27T03:50:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:30 smithi119 ceph-mon[32752]: pgmap v18304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:31.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:30 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33312.499573 secs 2024-04-27T03:50:31.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33313.367461 secs 2024-04-27T03:50:32.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33313.367461 secs 2024-04-27T03:50:32.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33313.367461 secs 2024-04-27T03:50:32.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:32 smithi090 ceph-mon[29180]: pgmap v18305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:33.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:32 smithi119 ceph-mon[32752]: pgmap v18305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:33.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:32 smithi167 ceph-mon[32751]: pgmap v18305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:34.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:34 smithi090 ceph-mon[29180]: pgmap v18306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:35.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:34 smithi119 ceph-mon[32752]: pgmap v18306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:35.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:34 smithi167 ceph-mon[32751]: pgmap v18306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:35.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:35 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33317.499644 secs 2024-04-27T03:50:36.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:35 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33317.499644 secs 2024-04-27T03:50:36.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:35 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33317.499644 secs 2024-04-27T03:50:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:36 smithi090 ceph-mon[29180]: pgmap v18307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33318.367575 secs 2024-04-27T03:50:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:36 smithi119 ceph-mon[32752]: pgmap v18307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:37.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33318.367575 secs 2024-04-27T03:50:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:36 smithi167 ceph-mon[32751]: pgmap v18307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:37.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33318.367575 secs 2024-04-27T03:50:38.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:38 smithi090 ceph-mon[29180]: pgmap v18308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:39.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:38 smithi119 ceph-mon[32752]: pgmap v18308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:39.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:38 smithi167 ceph-mon[32751]: pgmap v18308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:40.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:40 smithi090 ceph-mon[29180]: pgmap v18309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:40.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:40 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33322.499731 secs 2024-04-27T03:50:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:40 smithi119 ceph-mon[32752]: pgmap v18309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:41.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:40 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33322.499731 secs 2024-04-27T03:50:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:40 smithi167 ceph-mon[32751]: pgmap v18309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:41.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:40 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33322.499731 secs 2024-04-27T03:50:41.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33323.367675 secs 2024-04-27T03:50:42.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33323.367675 secs 2024-04-27T03:50:42.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33323.367675 secs 2024-04-27T03:50:42.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:42 smithi090 ceph-mon[29180]: pgmap v18310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:43.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:42 smithi119 ceph-mon[32752]: pgmap v18310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:43.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:42 smithi167 ceph-mon[32751]: pgmap v18310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:44.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:44 smithi090 ceph-mon[29180]: pgmap v18311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:45.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:44 smithi119 ceph-mon[32752]: pgmap v18311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:45.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:44 smithi167 ceph-mon[32751]: pgmap v18311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:45.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:45 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33327.499810 secs 2024-04-27T03:50:46.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:45 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33327.499810 secs 2024-04-27T03:50:46.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:45 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33327.499810 secs 2024-04-27T03:50:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:46 smithi090 ceph-mon[29180]: pgmap v18312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:46.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33328.367791 secs 2024-04-27T03:50:46.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:46 smithi119 ceph-mon[32752]: pgmap v18312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:46.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33328.367791 secs 2024-04-27T03:50:46.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:46 smithi167 ceph-mon[32751]: pgmap v18312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:46.974 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33328.367791 secs 2024-04-27T03:50:48.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:48 smithi090 ceph-mon[29180]: pgmap v18313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:49.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:48 smithi119 ceph-mon[32752]: pgmap v18313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:49.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:48 smithi167 ceph-mon[32751]: pgmap v18313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:50 smithi090 ceph-mon[29180]: pgmap v18314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:50.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:50 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33332.499873 secs 2024-04-27T03:50:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:50 smithi119 ceph-mon[32752]: pgmap v18314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:51.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:50 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33332.499873 secs 2024-04-27T03:50:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:50 smithi167 ceph-mon[32751]: pgmap v18314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:51.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:50 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33332.499873 secs 2024-04-27T03:50:51.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33333.367853 secs 2024-04-27T03:50:52.061 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33333.367853 secs 2024-04-27T03:50:52.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33333.367853 secs 2024-04-27T03:50:53.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:52 smithi119 ceph-mon[32752]: pgmap v18315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:53.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:52 smithi167 ceph-mon[32751]: pgmap v18315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:53.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:52 smithi090 ceph-mon[29180]: pgmap v18315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:55.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:54 smithi119 ceph-mon[32752]: pgmap v18316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:55.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:54 smithi167 ceph-mon[32751]: pgmap v18316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:55.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:54 smithi090 ceph-mon[29180]: pgmap v18316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:50:56.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:55 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33337.499933 secs 2024-04-27T03:50:56.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:55 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33337.499933 secs 2024-04-27T03:50:56.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:55 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33337.499933 secs 2024-04-27T03:50:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:56 smithi119 ceph-mon[32752]: pgmap v18317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:57.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33338.367933 secs 2024-04-27T03:50:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:56 smithi167 ceph-mon[32751]: pgmap v18317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:57.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33338.367933 secs 2024-04-27T03:50:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:56 smithi090 ceph-mon[29180]: pgmap v18317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:57.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33338.367933 secs 2024-04-27T03:50:58.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:50:58.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:50:58.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:50:58.911 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:50:58 smithi167 ceph-mon[32751]: pgmap v18318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:59.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:50:58 smithi119 ceph-mon[32752]: pgmap v18318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:50:59.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:50:58 smithi090 ceph-mon[29180]: pgmap v18318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:00 smithi167 ceph-mon[32751]: pgmap v18319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:00.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:00 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33342.499997 secs 2024-04-27T03:51:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:00 smithi119 ceph-mon[32752]: pgmap v18319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:01.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:00 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33342.499997 secs 2024-04-27T03:51:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:00 smithi090 ceph-mon[29180]: pgmap v18319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:01.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:00 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33342.499997 secs 2024-04-27T03:51:01.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33343.367997 secs 2024-04-27T03:51:02.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33343.367997 secs 2024-04-27T03:51:02.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33343.367997 secs 2024-04-27T03:51:03.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: pgmap v18320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:03.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:51:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:51:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: pgmap v18321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 B/s rd, 1 op/s 2024-04-27T03:51:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: pgmap v18322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T03:51:03.311 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: pgmap v18320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:51:03.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:51:03.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: pgmap v18321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 B/s rd, 1 op/s 2024-04-27T03:51:03.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: pgmap v18322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T03:51:03.324 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: pgmap v18320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:03.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:51:03.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:51:03.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: pgmap v18321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 B/s rd, 1 op/s 2024-04-27T03:51:03.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: pgmap v18322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T03:51:03.382 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:51:05.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:04 smithi119 ceph-mon[32752]: pgmap v18323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:05.338 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:04 smithi167 ceph-mon[32751]: pgmap v18323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:05.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:04 smithi090 ceph-mon[29180]: pgmap v18323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:06.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:06 smithi119 ceph-mon[32752]: 13 slow requests, 0 included below; oldest blocked for > 33347.500078 secs 2024-04-27T03:51:06.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:06 smithi167 ceph-mon[32751]: 13 slow requests, 0 included below; oldest blocked for > 33347.500078 secs 2024-04-27T03:51:06.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:05 smithi090 ceph-mon[29180]: 13 slow requests, 0 included below; oldest blocked for > 33347.500078 secs 2024-04-27T03:51:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:07 smithi119 ceph-mon[32752]: pgmap v18324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:07.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33348.368129 secs 2024-04-27T03:51:07.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:07 smithi167 ceph-mon[32751]: pgmap v18324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:07.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33348.368129 secs 2024-04-27T03:51:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:07 smithi090 ceph-mon[29180]: pgmap v18324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:07.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33348.368129 secs 2024-04-27T03:51:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:08 smithi119 ceph-mon[32752]: pgmap v18325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T03:51:08.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:08 smithi167 ceph-mon[32751]: pgmap v18325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T03:51:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:08 smithi090 ceph-mon[29180]: pgmap v18325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T03:51:11.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:10 smithi119 ceph-mon[32752]: pgmap v18326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:11.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:10 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 33352.500139 secs 2024-04-27T03:51:11.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:10 smithi119 ceph-mon[32752]: slow request 180.286590 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:51:11.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:10 smithi167 ceph-mon[32751]: pgmap v18326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:11.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:10 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 33352.500139 secs 2024-04-27T03:51:11.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:10 smithi167 ceph-mon[32751]: slow request 180.286590 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:51:11.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:10 smithi090 ceph-mon[29180]: pgmap v18326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:11.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:10 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 33352.500139 secs 2024-04-27T03:51:11.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:10 smithi090 ceph-mon[29180]: slow request 180.286590 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T03:51:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33353.368273 secs 2024-04-27T03:51:12.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33353.368273 secs 2024-04-27T03:51:12.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33353.368273 secs 2024-04-27T03:51:13.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:13 smithi119 ceph-mon[32752]: pgmap v18327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:51:13.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:13 smithi167 ceph-mon[32751]: pgmap v18327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:51:13.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:13 smithi090 ceph-mon[29180]: pgmap v18327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:51:14.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:14 smithi119 ceph-mon[32752]: pgmap v18328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:14.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:14 smithi167 ceph-mon[32751]: pgmap v18328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:14.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:14 smithi090 ceph-mon[29180]: pgmap v18328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:16.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:16 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:51:16.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:16 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:51:16.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:16 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33357.500213 secs 2024-04-27T03:51:16.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:16 smithi167 ceph-mon[32751]: pgmap v18329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:16.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:16 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:51:16.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:16 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:51:16.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:16 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33357.500213 secs 2024-04-27T03:51:16.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:16 smithi090 ceph-mon[29180]: pgmap v18329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:16.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:16 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:51:16.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:16 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:51:16.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:16 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33357.500213 secs 2024-04-27T03:51:16.382 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:16 smithi119 ceph-mon[32752]: pgmap v18329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:17.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33358.368340 secs 2024-04-27T03:51:17.327 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33358.368340 secs 2024-04-27T03:51:17.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33358.368340 secs 2024-04-27T03:51:18.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:18 smithi090 ceph-mon[29180]: pgmap v18330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:18.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:18 smithi119 ceph-mon[32752]: pgmap v18330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:18.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:18 smithi167 ceph-mon[32751]: pgmap v18330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:20 smithi119 ceph-mon[32752]: pgmap v18331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:21.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33362.500301 secs 2024-04-27T03:51:21.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:20 smithi167 ceph-mon[32751]: pgmap v18331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:21.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33362.500301 secs 2024-04-27T03:51:21.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:20 smithi090 ceph-mon[29180]: pgmap v18331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:21.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33362.500301 secs 2024-04-27T03:51:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33363.368500 secs 2024-04-27T03:51:22.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33363.368500 secs 2024-04-27T03:51:22.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33363.368500 secs 2024-04-27T03:51:23.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:23 smithi119 ceph-mon[32752]: pgmap v18332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:23.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:23 smithi167 ceph-mon[32751]: pgmap v18332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:23.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:23 smithi090 ceph-mon[29180]: pgmap v18332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:24.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:24 smithi119 ceph-mon[32752]: pgmap v18333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:24.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:24 smithi167 ceph-mon[32751]: pgmap v18333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:24.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:24 smithi090 ceph-mon[29180]: pgmap v18333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:25.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:25 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33367.500361 secs 2024-04-27T03:51:26.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:25 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33367.500361 secs 2024-04-27T03:51:26.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:25 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33367.500361 secs 2024-04-27T03:51:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:26 smithi090 ceph-mon[29180]: pgmap v18334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:26.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33368.368642 secs 2024-04-27T03:51:27.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:26 smithi119 ceph-mon[32752]: pgmap v18334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:27.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33368.368642 secs 2024-04-27T03:51:27.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:26 smithi167 ceph-mon[32751]: pgmap v18334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:27.073 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33368.368642 secs 2024-04-27T03:51:29.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:28 smithi119 ceph-mon[32752]: pgmap v18335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:29.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:29 smithi167 ceph-mon[32751]: pgmap v18335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:29.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:28 smithi090 ceph-mon[29180]: pgmap v18335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:31.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:31 smithi119 ceph-mon[32752]: pgmap v18336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:31.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:31 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33372.500447 secs 2024-04-27T03:51:31.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:31 smithi167 ceph-mon[32751]: pgmap v18336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:31.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:31 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33372.500447 secs 2024-04-27T03:51:31.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:31 smithi090 ceph-mon[29180]: pgmap v18336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:31.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:31 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33372.500447 secs 2024-04-27T03:51:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33373.368759 secs 2024-04-27T03:51:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33373.368759 secs 2024-04-27T03:51:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33373.368759 secs 2024-04-27T03:51:33.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:33 smithi119 ceph-mon[32752]: pgmap v18337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:33.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:33 smithi167 ceph-mon[32751]: pgmap v18337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:33.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:33 smithi090 ceph-mon[29180]: pgmap v18337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:34.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:34 smithi119 ceph-mon[32752]: pgmap v18338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:34.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:34 smithi167 ceph-mon[32751]: pgmap v18338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:34.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:34 smithi090 ceph-mon[29180]: pgmap v18338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:35.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:35 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33377.500527 secs 2024-04-27T03:51:36.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:35 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33377.500527 secs 2024-04-27T03:51:36.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:35 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33377.500527 secs 2024-04-27T03:51:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:36 smithi090 ceph-mon[29180]: pgmap v18339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:36.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33378.368840 secs 2024-04-27T03:51:37.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:36 smithi119 ceph-mon[32752]: pgmap v18339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:37.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33378.368840 secs 2024-04-27T03:51:37.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:36 smithi167 ceph-mon[32751]: pgmap v18339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:37.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33378.368840 secs 2024-04-27T03:51:39.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:39 smithi119 ceph-mon[32752]: pgmap v18340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:39.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:39 smithi167 ceph-mon[32751]: pgmap v18340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:39.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:39 smithi090 ceph-mon[29180]: pgmap v18340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:41.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:41 smithi119 ceph-mon[32752]: pgmap v18341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:41.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33382.500614 secs 2024-04-27T03:51:41.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:41 smithi167 ceph-mon[32751]: pgmap v18341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:41.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33382.500614 secs 2024-04-27T03:51:41.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:41 smithi090 ceph-mon[29180]: pgmap v18341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:41.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33382.500614 secs 2024-04-27T03:51:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33383.368914 secs 2024-04-27T03:51:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:42 smithi119 ceph-mon[32752]: pgmap v18342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33383.368914 secs 2024-04-27T03:51:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:42 smithi167 ceph-mon[32751]: pgmap v18342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:42.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33383.368914 secs 2024-04-27T03:51:42.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:42 smithi090 ceph-mon[29180]: pgmap v18342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:45.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:45 smithi119 ceph-mon[32752]: pgmap v18343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:45.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:45 smithi167 ceph-mon[32751]: pgmap v18343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:45.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:45 smithi090 ceph-mon[29180]: pgmap v18343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:46.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33387.500666 secs 2024-04-27T03:51:46.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33387.500666 secs 2024-04-27T03:51:46.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33387.500666 secs 2024-04-27T03:51:47.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:47 smithi119 ceph-mon[32752]: pgmap v18344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:47.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33388.368992 secs 2024-04-27T03:51:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:47 smithi167 ceph-mon[32751]: pgmap v18344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:47.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33388.368992 secs 2024-04-27T03:51:47.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:47 smithi090 ceph-mon[29180]: pgmap v18344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:47.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33388.368992 secs 2024-04-27T03:51:48.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:48 smithi119 ceph-mon[32752]: pgmap v18345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:48.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:48 smithi167 ceph-mon[32751]: pgmap v18345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:48.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:48 smithi090 ceph-mon[29180]: pgmap v18345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:51.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:51 smithi119 ceph-mon[32752]: pgmap v18346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:51.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33392.500746 secs 2024-04-27T03:51:51.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:51 smithi167 ceph-mon[32751]: pgmap v18346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:51.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33392.500746 secs 2024-04-27T03:51:51.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:51 smithi090 ceph-mon[29180]: pgmap v18346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:51.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33392.500746 secs 2024-04-27T03:51:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33393.369130 secs 2024-04-27T03:51:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:52 smithi119 ceph-mon[32752]: pgmap v18347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33393.369130 secs 2024-04-27T03:51:52.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:52 smithi167 ceph-mon[32751]: pgmap v18347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:52.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33393.369130 secs 2024-04-27T03:51:52.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:52 smithi090 ceph-mon[29180]: pgmap v18347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:54.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:54 smithi167 ceph-mon[32751]: pgmap v18348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:54.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:54 smithi090 ceph-mon[29180]: pgmap v18348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:54.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:54 smithi119 ceph-mon[32752]: pgmap v18348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:51:55.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33397.500830 secs 2024-04-27T03:51:56.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33397.500830 secs 2024-04-27T03:51:56.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33397.500830 secs 2024-04-27T03:51:56.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:56 smithi090 ceph-mon[29180]: pgmap v18349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:56.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33398.369246 secs 2024-04-27T03:51:57.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:56 smithi119 ceph-mon[32752]: pgmap v18349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:57.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33398.369246 secs 2024-04-27T03:51:57.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:56 smithi167 ceph-mon[32751]: pgmap v18349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:57.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33398.369246 secs 2024-04-27T03:51:58.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:51:58 smithi167 ceph-mon[32751]: pgmap v18350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:58.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:51:58 smithi090 ceph-mon[29180]: pgmap v18350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:51:58.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:51:58 smithi119 ceph-mon[32752]: pgmap v18350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:00.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:00 smithi167 ceph-mon[32751]: pgmap v18351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:00.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:00 smithi090 ceph-mon[29180]: pgmap v18351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:00.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:00 smithi119 ceph-mon[32752]: pgmap v18351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:01.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:01 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33402.500918 secs 2024-04-27T03:52:01.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:01 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33402.500918 secs 2024-04-27T03:52:01.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:01 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33402.500918 secs 2024-04-27T03:52:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33403.369404 secs 2024-04-27T03:52:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:02 smithi090 ceph-mon[29180]: pgmap v18352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:02.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33403.369404 secs 2024-04-27T03:52:02.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:02 smithi119 ceph-mon[32752]: pgmap v18352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:02.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33403.369404 secs 2024-04-27T03:52:02.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:02 smithi167 ceph-mon[32751]: pgmap v18352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:03.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:52:03.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:52:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:52:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:03 smithi090 ceph-mon[29180]: pgmap v18353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-27T03:52:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:03 smithi090 ceph-mon[29180]: pgmap v18354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T03:52:03.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:52:03.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:52:03.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:52:03.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:52:03.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:03 smithi119 ceph-mon[32752]: pgmap v18353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-27T03:52:03.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:03 smithi119 ceph-mon[32752]: pgmap v18354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T03:52:03.559 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:52:03.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:52:03.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:52:03.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:52:03.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:03 smithi167 ceph-mon[32751]: pgmap v18353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-27T03:52:03.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:03 smithi167 ceph-mon[32751]: pgmap v18354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T03:52:03.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:52:06.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:05 smithi090 ceph-mon[29180]: pgmap v18355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:06.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:05 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33407.501002 secs 2024-04-27T03:52:06.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:05 smithi119 ceph-mon[32752]: pgmap v18355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:06.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:05 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33407.501002 secs 2024-04-27T03:52:06.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:05 smithi167 ceph-mon[32751]: pgmap v18355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:06.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:05 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33407.501002 secs 2024-04-27T03:52:07.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33408.369479 secs 2024-04-27T03:52:07.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33408.369479 secs 2024-04-27T03:52:07.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33408.369479 secs 2024-04-27T03:52:08.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:07 smithi090 ceph-mon[29180]: pgmap v18356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:07 smithi119 ceph-mon[32752]: pgmap v18356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:08.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:07 smithi167 ceph-mon[32751]: pgmap v18356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:10.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:09 smithi090 ceph-mon[29180]: pgmap v18357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:10.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:09 smithi119 ceph-mon[32752]: pgmap v18357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:10.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:09 smithi167 ceph-mon[32751]: pgmap v18357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:11.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:10 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33412.501061 secs 2024-04-27T03:52:11.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:10 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33412.501061 secs 2024-04-27T03:52:11.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:10 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33412.501061 secs 2024-04-27T03:52:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:11 smithi090 ceph-mon[29180]: pgmap v18358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33413.369605 secs 2024-04-27T03:52:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:11 smithi119 ceph-mon[32752]: pgmap v18358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:12.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33413.369605 secs 2024-04-27T03:52:12.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:11 smithi167 ceph-mon[32751]: pgmap v18358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:52:12.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33413.369605 secs 2024-04-27T03:52:14.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:13 smithi090 ceph-mon[29180]: pgmap v18359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:52:14.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:13 smithi119 ceph-mon[32752]: pgmap v18359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:52:14.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:13 smithi167 ceph-mon[32751]: pgmap v18359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:52:16.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:15 smithi167 ceph-mon[32751]: pgmap v18360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:16.125 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:15 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33417.501146 secs 2024-04-27T03:52:16.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:15 smithi090 ceph-mon[29180]: pgmap v18360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:16.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:15 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33417.501146 secs 2024-04-27T03:52:16.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:15 smithi119 ceph-mon[32752]: pgmap v18360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:16.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:15 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33417.501146 secs 2024-04-27T03:52:17.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33418.369718 secs 2024-04-27T03:52:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33418.369718 secs 2024-04-27T03:52:17.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33418.369718 secs 2024-04-27T03:52:18.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:17 smithi090 ceph-mon[29180]: pgmap v18361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:18.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:17 smithi119 ceph-mon[32752]: pgmap v18361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:18.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:17 smithi167 ceph-mon[32751]: pgmap v18361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:20.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:19 smithi119 ceph-mon[32752]: pgmap v18362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:20.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:19 smithi167 ceph-mon[32751]: pgmap v18362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:20.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:19 smithi090 ceph-mon[29180]: pgmap v18362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:21.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:20 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33422.501226 secs 2024-04-27T03:52:21.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:20 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33422.501226 secs 2024-04-27T03:52:21.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:20 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33422.501226 secs 2024-04-27T03:52:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:21 smithi119 ceph-mon[32752]: pgmap v18363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:22.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33423.369819 secs 2024-04-27T03:52:22.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:21 smithi167 ceph-mon[32751]: pgmap v18363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:22.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33423.369819 secs 2024-04-27T03:52:22.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:21 smithi090 ceph-mon[29180]: pgmap v18363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:22.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33423.369819 secs 2024-04-27T03:52:24.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:23 smithi119 ceph-mon[32752]: pgmap v18364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:24.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:23 smithi167 ceph-mon[32751]: pgmap v18364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:24.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:23 smithi090 ceph-mon[29180]: pgmap v18364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:26.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:25 smithi090 ceph-mon[29180]: pgmap v18365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:26.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:25 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33427.501306 secs 2024-04-27T03:52:26.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:25 smithi119 ceph-mon[32752]: pgmap v18365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:26.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:25 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33427.501306 secs 2024-04-27T03:52:26.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:25 smithi167 ceph-mon[32751]: pgmap v18365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:26.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:25 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33427.501306 secs 2024-04-27T03:52:27.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33428.369909 secs 2024-04-27T03:52:27.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33428.369909 secs 2024-04-27T03:52:27.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33428.369909 secs 2024-04-27T03:52:28.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:27 smithi119 ceph-mon[32752]: pgmap v18366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:28.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:27 smithi167 ceph-mon[32751]: pgmap v18366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:28.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:27 smithi090 ceph-mon[29180]: pgmap v18366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:30.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:29 smithi119 ceph-mon[32752]: pgmap v18367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:30.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:29 smithi167 ceph-mon[32751]: pgmap v18367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:30.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:29 smithi090 ceph-mon[29180]: pgmap v18367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:31.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:30 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33432.501390 secs 2024-04-27T03:52:31.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:30 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33432.501390 secs 2024-04-27T03:52:31.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:30 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33432.501390 secs 2024-04-27T03:52:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:31 smithi119 ceph-mon[32752]: pgmap v18368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:32.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33433.370078 secs 2024-04-27T03:52:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:31 smithi167 ceph-mon[32751]: pgmap v18368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:32.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33433.370078 secs 2024-04-27T03:52:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:31 smithi090 ceph-mon[29180]: pgmap v18368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:32.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33433.370078 secs 2024-04-27T03:52:34.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:33 smithi119 ceph-mon[32752]: pgmap v18369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:34.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:33 smithi167 ceph-mon[32751]: pgmap v18369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:34.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:33 smithi090 ceph-mon[29180]: pgmap v18369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:36.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:35 smithi090 ceph-mon[29180]: pgmap v18370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:36.181 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:35 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33437.501475 secs 2024-04-27T03:52:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:35 smithi119 ceph-mon[32752]: pgmap v18370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:36.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:35 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33437.501475 secs 2024-04-27T03:52:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:35 smithi167 ceph-mon[32751]: pgmap v18370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:36.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:35 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33437.501475 secs 2024-04-27T03:52:37.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33438.370183 secs 2024-04-27T03:52:37.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33438.370183 secs 2024-04-27T03:52:37.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33438.370183 secs 2024-04-27T03:52:38.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:37 smithi119 ceph-mon[32752]: pgmap v18371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:37 smithi167 ceph-mon[32751]: pgmap v18371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:38.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:37 smithi090 ceph-mon[29180]: pgmap v18371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:40.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:39 smithi119 ceph-mon[32752]: pgmap v18372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:40.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:39 smithi167 ceph-mon[32751]: pgmap v18372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:40.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:39 smithi090 ceph-mon[29180]: pgmap v18372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:41.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:40 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33442.501560 secs 2024-04-27T03:52:41.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:40 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33442.501560 secs 2024-04-27T03:52:41.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:40 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33442.501560 secs 2024-04-27T03:52:42.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:41 smithi090 ceph-mon[29180]: pgmap v18373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:42.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33443.370309 secs 2024-04-27T03:52:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:41 smithi119 ceph-mon[32752]: pgmap v18373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:42.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33443.370309 secs 2024-04-27T03:52:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:41 smithi167 ceph-mon[32751]: pgmap v18373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:42.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33443.370309 secs 2024-04-27T03:52:44.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:43 smithi119 ceph-mon[32752]: pgmap v18374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:43 smithi167 ceph-mon[32751]: pgmap v18374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:44.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:43 smithi090 ceph-mon[29180]: pgmap v18374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:46.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:45 smithi119 ceph-mon[32752]: pgmap v18375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:46.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:45 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33447.501620 secs 2024-04-27T03:52:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:45 smithi167 ceph-mon[32751]: pgmap v18375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:46.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:45 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33447.501620 secs 2024-04-27T03:52:46.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:45 smithi090 ceph-mon[29180]: pgmap v18375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:46.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:45 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33447.501620 secs 2024-04-27T03:52:47.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33448.370436 secs 2024-04-27T03:52:47.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33448.370436 secs 2024-04-27T03:52:47.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33448.370436 secs 2024-04-27T03:52:48.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:47 smithi119 ceph-mon[32752]: pgmap v18376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:48.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:47 smithi167 ceph-mon[32751]: pgmap v18376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:48.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:47 smithi090 ceph-mon[29180]: pgmap v18376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:50.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:49 smithi119 ceph-mon[32752]: pgmap v18377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:50.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:49 smithi167 ceph-mon[32751]: pgmap v18377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:50.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:49 smithi090 ceph-mon[29180]: pgmap v18377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:51.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:50 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33452.501710 secs 2024-04-27T03:52:51.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:50 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33452.501710 secs 2024-04-27T03:52:51.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:50 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33452.501710 secs 2024-04-27T03:52:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:51 smithi119 ceph-mon[32752]: pgmap v18378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:52.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33453.370560 secs 2024-04-27T03:52:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:51 smithi167 ceph-mon[32751]: pgmap v18378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:52.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33453.370560 secs 2024-04-27T03:52:52.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:51 smithi090 ceph-mon[29180]: pgmap v18378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:52.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33453.370560 secs 2024-04-27T03:52:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:53 smithi119 ceph-mon[32752]: pgmap v18379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:54.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:53 smithi167 ceph-mon[32751]: pgmap v18379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:54.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:53 smithi090 ceph-mon[29180]: pgmap v18379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:56.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:55 smithi119 ceph-mon[32752]: pgmap v18380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:56.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:55 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 33457.501814 secs 2024-04-27T03:52:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:55 smithi167 ceph-mon[32751]: pgmap v18380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:56.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:55 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 33457.501814 secs 2024-04-27T03:52:56.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:55 smithi090 ceph-mon[29180]: pgmap v18380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:52:56.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:55 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 33457.501814 secs 2024-04-27T03:52:57.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33458.370685 secs 2024-04-27T03:52:57.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33458.370685 secs 2024-04-27T03:52:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33458.370685 secs 2024-04-27T03:52:58.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:57 smithi119 ceph-mon[32752]: pgmap v18381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:58.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:57 smithi167 ceph-mon[32751]: pgmap v18381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:52:58.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:57 smithi090 ceph-mon[29180]: pgmap v18381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:00.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:52:59 smithi119 ceph-mon[32752]: pgmap v18382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:00.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:52:59 smithi167 ceph-mon[32751]: pgmap v18382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:00.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:52:59 smithi090 ceph-mon[29180]: pgmap v18382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:01.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:00 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 33462.501875 secs 2024-04-27T03:53:01.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:00 smithi119 ceph-mon[32752]: slow request 183.979037 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:53:01.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:00 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 33462.501875 secs 2024-04-27T03:53:01.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:00 smithi167 ceph-mon[32751]: slow request 183.979037 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:53:01.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:00 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 33462.501875 secs 2024-04-27T03:53:01.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:00 smithi090 ceph-mon[29180]: slow request 183.979037 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:53:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:01 smithi119 ceph-mon[32752]: pgmap v18383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:02.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33463.370850 secs 2024-04-27T03:53:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:01 smithi167 ceph-mon[32751]: pgmap v18383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:02.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33463.370850 secs 2024-04-27T03:53:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:01 smithi090 ceph-mon[29180]: pgmap v18383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:02.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33463.370850 secs 2024-04-27T03:53:03.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:53:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:53:03.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:53:04.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:03 smithi119 ceph-mon[32752]: pgmap v18384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:04.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:53:04.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:03 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:53:04.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:53:04.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:53:04.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:53:04.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:03 smithi167 ceph-mon[32751]: pgmap v18384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:04.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:53:04.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:03 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:53:04.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:53:04.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:53:04.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:53:04.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:03 smithi090 ceph-mon[29180]: pgmap v18384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:04.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T03:53:04.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:03 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T03:53:04.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:53:04.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:53:04.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:53:05.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:04 smithi119 ceph-mon[32752]: pgmap v18385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:53:05.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:04 smithi119 ceph-mon[32752]: pgmap v18386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:05.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:04 smithi167 ceph-mon[32751]: pgmap v18385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:53:05.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:04 smithi167 ceph-mon[32751]: pgmap v18386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:05.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:04 smithi090 ceph-mon[29180]: pgmap v18385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T03:53:05.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:04 smithi090 ceph-mon[29180]: pgmap v18386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:06.281 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33467.501962 secs 2024-04-27T03:53:06.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33467.501962 secs 2024-04-27T03:53:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33467.501962 secs 2024-04-27T03:53:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:06 smithi119 ceph-mon[32752]: pgmap v18387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33468.370988 secs 2024-04-27T03:53:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:06 smithi167 ceph-mon[32751]: pgmap v18387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33468.370988 secs 2024-04-27T03:53:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:06 smithi090 ceph-mon[29180]: pgmap v18387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:07.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33468.370988 secs 2024-04-27T03:53:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:09 smithi119 ceph-mon[32752]: pgmap v18388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:53:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:09 smithi167 ceph-mon[32751]: pgmap v18388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:53:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:09 smithi090 ceph-mon[29180]: pgmap v18388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:53:10.380 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:53:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:53:10.085+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:53:10.380 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:53:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:53:10.085+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:53:10.380 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:53:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:53:10.085+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:53:10.381 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:53:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:53:10.085+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:53:10.381 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:53:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:53:10.085+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:53:11.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:11 smithi119 ceph-mon[32752]: pgmap v18389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:11.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33472.502045 secs 2024-04-27T03:53:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:11 smithi167 ceph-mon[32751]: pgmap v18389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:11.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33472.502045 secs 2024-04-27T03:53:11.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:11 smithi090 ceph-mon[29180]: pgmap v18389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:11.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33472.502045 secs 2024-04-27T03:53:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33473.371113 secs 2024-04-27T03:53:12.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33473.371113 secs 2024-04-27T03:53:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33473.371113 secs 2024-04-27T03:53:13.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:13 smithi119 ceph-mon[32752]: pgmap v18390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:13.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:13 smithi167 ceph-mon[32751]: pgmap v18390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:13.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:13 smithi090 ceph-mon[29180]: pgmap v18390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:53:15.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:15 smithi119 ceph-mon[32752]: pgmap v18391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 511 B/s wr, 16 op/s 2024-04-27T03:53:15.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:15 smithi167 ceph-mon[32751]: pgmap v18391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 511 B/s wr, 16 op/s 2024-04-27T03:53:15.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:15 smithi090 ceph-mon[29180]: pgmap v18391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 511 B/s wr, 16 op/s 2024-04-27T03:53:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33477.502127 secs 2024-04-27T03:53:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33477.502127 secs 2024-04-27T03:53:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33477.502127 secs 2024-04-27T03:53:17.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:17 smithi119 ceph-mon[32752]: pgmap v18392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:17.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33478.371235 secs 2024-04-27T03:53:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:17 smithi167 ceph-mon[32751]: pgmap v18392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:17.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33478.371235 secs 2024-04-27T03:53:17.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:17 smithi090 ceph-mon[29180]: pgmap v18392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:17.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33478.371235 secs 2024-04-27T03:53:18.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:18 smithi119 ceph-mon[32752]: pgmap v18393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:18.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:18 smithi167 ceph-mon[32751]: pgmap v18393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:18.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:18 smithi090 ceph-mon[29180]: pgmap v18393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:21.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:20 smithi119 ceph-mon[32752]: pgmap v18394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:53:21.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33482.502188 secs 2024-04-27T03:53:21.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:20 smithi167 ceph-mon[32751]: pgmap v18394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:53:21.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33482.502188 secs 2024-04-27T03:53:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:20 smithi090 ceph-mon[29180]: pgmap v18394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:53:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33482.502188 secs 2024-04-27T03:53:22.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33483.371355 secs 2024-04-27T03:53:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33483.371355 secs 2024-04-27T03:53:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33483.371355 secs 2024-04-27T03:53:23.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:22 smithi119 ceph-mon[32752]: pgmap v18395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:23.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:22 smithi167 ceph-mon[32751]: pgmap v18395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:23.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:22 smithi090 ceph-mon[29180]: pgmap v18395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T03:53:25.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:24 smithi119 ceph-mon[32752]: pgmap v18396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:53:25.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:24 smithi167 ceph-mon[32751]: pgmap v18396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:53:25.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:24 smithi090 ceph-mon[29180]: pgmap v18396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T03:53:26.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33487.502271 secs 2024-04-27T03:53:26.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33487.502271 secs 2024-04-27T03:53:26.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33487.502271 secs 2024-04-27T03:53:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:26 smithi119 ceph-mon[32752]: pgmap v18397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:27.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33488.371476 secs 2024-04-27T03:53:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:26 smithi167 ceph-mon[32751]: pgmap v18397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:27.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33488.371476 secs 2024-04-27T03:53:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:26 smithi090 ceph-mon[29180]: pgmap v18397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:27.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33488.371476 secs 2024-04-27T03:53:29.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:28 smithi119 ceph-mon[32752]: pgmap v18398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:29.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:28 smithi167 ceph-mon[32751]: pgmap v18398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:29.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:28 smithi090 ceph-mon[29180]: pgmap v18398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:31.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:30 smithi119 ceph-mon[32752]: pgmap v18399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:31.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33492.502354 secs 2024-04-27T03:53:31.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:30 smithi167 ceph-mon[32751]: pgmap v18399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:31.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33492.502354 secs 2024-04-27T03:53:31.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:30 smithi090 ceph-mon[29180]: pgmap v18399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:31.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33492.502354 secs 2024-04-27T03:53:32.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33493.371586 secs 2024-04-27T03:53:32.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33493.371586 secs 2024-04-27T03:53:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33493.371586 secs 2024-04-27T03:53:33.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:32 smithi119 ceph-mon[32752]: pgmap v18400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:33.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:32 smithi167 ceph-mon[32751]: pgmap v18400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:33.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:32 smithi090 ceph-mon[29180]: pgmap v18400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:35.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:34 smithi119 ceph-mon[32752]: pgmap v18401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:34 smithi167 ceph-mon[32751]: pgmap v18401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:34 smithi090 ceph-mon[29180]: pgmap v18401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:36.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33497.502437 secs 2024-04-27T03:53:36.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33497.502437 secs 2024-04-27T03:53:36.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33497.502437 secs 2024-04-27T03:53:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:36 smithi119 ceph-mon[32752]: pgmap v18402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:37.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33498.371722 secs 2024-04-27T03:53:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:36 smithi167 ceph-mon[32751]: pgmap v18402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:37.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33498.371722 secs 2024-04-27T03:53:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:36 smithi090 ceph-mon[29180]: pgmap v18402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:37.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33498.371722 secs 2024-04-27T03:53:39.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:38 smithi119 ceph-mon[32752]: pgmap v18403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:39.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:38 smithi167 ceph-mon[32751]: pgmap v18403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:39.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:38 smithi090 ceph-mon[29180]: pgmap v18403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:40 smithi119 ceph-mon[32752]: pgmap v18404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:40 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 33502.502523 secs 2024-04-27T03:53:41.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:40 smithi119 ceph-mon[32752]: slow request 2881.012046 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:53:41.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:40 smithi167 ceph-mon[32751]: pgmap v18404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:41.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:40 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 33502.502523 secs 2024-04-27T03:53:41.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:40 smithi167 ceph-mon[32751]: slow request 2881.012046 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:53:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:40 smithi090 ceph-mon[29180]: pgmap v18404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:40 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 33502.502523 secs 2024-04-27T03:53:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:40 smithi090 ceph-mon[29180]: slow request 2881.012046 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:53:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33503.371834 secs 2024-04-27T03:53:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33503.371834 secs 2024-04-27T03:53:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33503.371834 secs 2024-04-27T03:53:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:42 smithi119 ceph-mon[32752]: pgmap v18405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:43.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:42 smithi167 ceph-mon[32751]: pgmap v18405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:43.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:42 smithi090 ceph-mon[29180]: pgmap v18405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:45.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:44 smithi119 ceph-mon[32752]: pgmap v18406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:45.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:44 smithi167 ceph-mon[32751]: pgmap v18406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:45.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:44 smithi090 ceph-mon[29180]: pgmap v18406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:46.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33507.502605 secs 2024-04-27T03:53:46.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33507.502605 secs 2024-04-27T03:53:46.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33507.502605 secs 2024-04-27T03:53:46.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:46 smithi119 ceph-mon[32752]: pgmap v18407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:46.958 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33508.371956 secs 2024-04-27T03:53:46.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:46 smithi167 ceph-mon[32751]: pgmap v18407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:46.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33508.371956 secs 2024-04-27T03:53:47.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:46 smithi090 ceph-mon[29180]: pgmap v18407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:47.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33508.371956 secs 2024-04-27T03:53:49.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:48 smithi119 ceph-mon[32752]: pgmap v18408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:49.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:48 smithi167 ceph-mon[32751]: pgmap v18408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:49.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:48 smithi090 ceph-mon[29180]: pgmap v18408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:51.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:50 smithi119 ceph-mon[32752]: pgmap v18409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:51.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33512.502691 secs 2024-04-27T03:53:51.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:50 smithi167 ceph-mon[32751]: pgmap v18409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:51.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33512.502691 secs 2024-04-27T03:53:51.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:50 smithi090 ceph-mon[29180]: pgmap v18409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:51.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33512.502691 secs 2024-04-27T03:53:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33513.372083 secs 2024-04-27T03:53:52.074 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33513.372083 secs 2024-04-27T03:53:52.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33513.372083 secs 2024-04-27T03:53:53.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:52 smithi119 ceph-mon[32752]: pgmap v18410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:53.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:52 smithi167 ceph-mon[32751]: pgmap v18410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:52 smithi090 ceph-mon[29180]: pgmap v18410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:55.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:54 smithi119 ceph-mon[32752]: pgmap v18411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:55.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:54 smithi167 ceph-mon[32751]: pgmap v18411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:55.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:54 smithi090 ceph-mon[29180]: pgmap v18411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:53:56.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:55 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 33517.502775 secs 2024-04-27T03:53:56.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:55 smithi119 ceph-mon[32752]: slow request 5763.273468 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:53:56.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:55 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 33517.502775 secs 2024-04-27T03:53:56.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:55 smithi167 ceph-mon[32751]: slow request 5763.273468 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:53:56.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:55 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 33517.502775 secs 2024-04-27T03:53:56.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:55 smithi090 ceph-mon[29180]: slow request 5763.273468 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:53:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:56 smithi119 ceph-mon[32752]: pgmap v18412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:57.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33518.372192 secs 2024-04-27T03:53:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:56 smithi167 ceph-mon[32751]: pgmap v18412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:57.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33518.372192 secs 2024-04-27T03:53:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:56 smithi090 ceph-mon[29180]: pgmap v18412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:57.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33518.372192 secs 2024-04-27T03:53:59.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:53:58 smithi119 ceph-mon[32752]: pgmap v18413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:59.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:53:58 smithi167 ceph-mon[32751]: pgmap v18413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:53:59.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:53:58 smithi090 ceph-mon[29180]: pgmap v18413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:01.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:00 smithi119 ceph-mon[32752]: pgmap v18414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:01.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33522.502859 secs 2024-04-27T03:54:01.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:00 smithi167 ceph-mon[32751]: pgmap v18414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:01.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33522.502859 secs 2024-04-27T03:54:01.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:00 smithi090 ceph-mon[29180]: pgmap v18414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:01.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33522.502859 secs 2024-04-27T03:54:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33523.372285 secs 2024-04-27T03:54:02.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33523.372285 secs 2024-04-27T03:54:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33523.372285 secs 2024-04-27T03:54:03.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:02 smithi119 ceph-mon[32752]: pgmap v18415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:03.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:02 smithi167 ceph-mon[32751]: pgmap v18415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:03.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:02 smithi090 ceph-mon[29180]: pgmap v18415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:04.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:54:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:54:04.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:54:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:04 smithi119 ceph-mon[32752]: pgmap v18416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:54:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:54:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:54:05.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:04 smithi167 ceph-mon[32751]: pgmap v18416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:05.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:54:05.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:54:05.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:54:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:04 smithi090 ceph-mon[29180]: pgmap v18416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:54:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:54:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:54:06.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:05 smithi090 ceph-mon[29180]: pgmap v18417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:06.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:05 smithi090 ceph-mon[29180]: pgmap v18418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:06.031 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33527.502941 secs 2024-04-27T03:54:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:05 smithi119 ceph-mon[32752]: pgmap v18417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:05 smithi119 ceph-mon[32752]: pgmap v18418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33527.502941 secs 2024-04-27T03:54:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:05 smithi167 ceph-mon[32751]: pgmap v18417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:05 smithi167 ceph-mon[32751]: pgmap v18418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:06.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33527.502941 secs 2024-04-27T03:54:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33528.372435 secs 2024-04-27T03:54:07.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33528.372435 secs 2024-04-27T03:54:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33528.372435 secs 2024-04-27T03:54:08.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:07 smithi119 ceph-mon[32752]: pgmap v18419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:08.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:07 smithi167 ceph-mon[32751]: pgmap v18419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:08.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:07 smithi090 ceph-mon[29180]: pgmap v18419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:10.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:09 smithi119 ceph-mon[32752]: pgmap v18420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:10.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:09 smithi167 ceph-mon[32751]: pgmap v18420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:10.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:09 smithi090 ceph-mon[29180]: pgmap v18420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:10 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 33532.503000 secs 2024-04-27T03:54:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:10 smithi119 ceph-mon[32752]: slow request 360.289451 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:54:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:10 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 33532.503000 secs 2024-04-27T03:54:11.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:10 smithi167 ceph-mon[32751]: slow request 360.289451 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:54:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:10 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 33532.503000 secs 2024-04-27T03:54:11.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:10 smithi090 ceph-mon[29180]: slow request 360.289451 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:54:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:11 smithi119 ceph-mon[32752]: pgmap v18421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33533.372516 secs 2024-04-27T03:54:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:11 smithi167 ceph-mon[32751]: pgmap v18421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33533.372516 secs 2024-04-27T03:54:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:11 smithi090 ceph-mon[29180]: pgmap v18421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T03:54:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33533.372516 secs 2024-04-27T03:54:14.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:13 smithi119 ceph-mon[32752]: pgmap v18422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:54:14.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:13 smithi167 ceph-mon[32751]: pgmap v18422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:54:14.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:13 smithi090 ceph-mon[29180]: pgmap v18422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:54:16.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:15 smithi119 ceph-mon[32752]: pgmap v18423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:54:16.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:15 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33537.503061 secs 2024-04-27T03:54:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:15 smithi167 ceph-mon[32751]: pgmap v18423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:54:16.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:15 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33537.503061 secs 2024-04-27T03:54:16.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:15 smithi090 ceph-mon[29180]: pgmap v18423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:54:16.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:15 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33537.503061 secs 2024-04-27T03:54:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33538.372656 secs 2024-04-27T03:54:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33538.372656 secs 2024-04-27T03:54:17.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33538.372656 secs 2024-04-27T03:54:18.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:17 smithi119 ceph-mon[32752]: pgmap v18424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:18.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:17 smithi167 ceph-mon[32751]: pgmap v18424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:18.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:17 smithi090 ceph-mon[29180]: pgmap v18424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:20.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:19 smithi119 ceph-mon[32752]: pgmap v18425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:19 smithi167 ceph-mon[32751]: pgmap v18425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:20.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:19 smithi090 ceph-mon[29180]: pgmap v18425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:21.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33542.503122 secs 2024-04-27T03:54:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33542.503122 secs 2024-04-27T03:54:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33542.503122 secs 2024-04-27T03:54:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:21 smithi167 ceph-mon[32751]: pgmap v18426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:22.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33543.372775 secs 2024-04-27T03:54:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:21 smithi090 ceph-mon[29180]: pgmap v18426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:22.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33543.372775 secs 2024-04-27T03:54:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:21 smithi119 ceph-mon[32752]: pgmap v18426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33543.372775 secs 2024-04-27T03:54:24.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:23 smithi167 ceph-mon[32751]: pgmap v18427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:24.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:23 smithi090 ceph-mon[29180]: pgmap v18427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:24.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:23 smithi119 ceph-mon[32752]: pgmap v18427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:26.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:25 smithi167 ceph-mon[32751]: pgmap v18428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:26.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33547.503201 secs 2024-04-27T03:54:26.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:25 smithi090 ceph-mon[29180]: pgmap v18428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:26.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33547.503201 secs 2024-04-27T03:54:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:25 smithi119 ceph-mon[32752]: pgmap v18428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:26.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33547.503201 secs 2024-04-27T03:54:27.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33548.372921 secs 2024-04-27T03:54:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33548.372921 secs 2024-04-27T03:54:27.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33548.372921 secs 2024-04-27T03:54:28.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:27 smithi090 ceph-mon[29180]: pgmap v18429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:28.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:27 smithi119 ceph-mon[32752]: pgmap v18429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:28.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:27 smithi167 ceph-mon[32751]: pgmap v18429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:30.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:29 smithi090 ceph-mon[29180]: pgmap v18430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:30.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:29 smithi119 ceph-mon[32752]: pgmap v18430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:30.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:29 smithi167 ceph-mon[32751]: pgmap v18430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:31.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33552.503287 secs 2024-04-27T03:54:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33552.503287 secs 2024-04-27T03:54:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33552.503287 secs 2024-04-27T03:54:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:31 smithi090 ceph-mon[29180]: pgmap v18431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:32.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33553.373049 secs 2024-04-27T03:54:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:31 smithi119 ceph-mon[32752]: pgmap v18431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:32.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33553.373049 secs 2024-04-27T03:54:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:31 smithi167 ceph-mon[32751]: pgmap v18431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:32.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33553.373049 secs 2024-04-27T03:54:34.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:33 smithi090 ceph-mon[29180]: pgmap v18432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:34.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:33 smithi119 ceph-mon[32752]: pgmap v18432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:34.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:33 smithi167 ceph-mon[32751]: pgmap v18432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:36.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:35 smithi090 ceph-mon[29180]: pgmap v18433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:36.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33557.503348 secs 2024-04-27T03:54:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:35 smithi119 ceph-mon[32752]: pgmap v18433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:36.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33557.503348 secs 2024-04-27T03:54:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:35 smithi167 ceph-mon[32751]: pgmap v18433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:36.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33557.503348 secs 2024-04-27T03:54:37.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33558.373178 secs 2024-04-27T03:54:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33558.373178 secs 2024-04-27T03:54:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33558.373178 secs 2024-04-27T03:54:38.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:37 smithi090 ceph-mon[29180]: pgmap v18434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:37 smithi119 ceph-mon[32752]: pgmap v18434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:37 smithi167 ceph-mon[32751]: pgmap v18434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:40.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:39 smithi090 ceph-mon[29180]: pgmap v18435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:40.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:39 smithi119 ceph-mon[32752]: pgmap v18435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:40.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:39 smithi167 ceph-mon[32751]: pgmap v18435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:41.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33562.503432 secs 2024-04-27T03:54:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33562.503432 secs 2024-04-27T03:54:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33562.503432 secs 2024-04-27T03:54:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:41 smithi090 ceph-mon[29180]: pgmap v18436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:42.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33563.373292 secs 2024-04-27T03:54:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:41 smithi119 ceph-mon[32752]: pgmap v18436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:42.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33563.373292 secs 2024-04-27T03:54:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:41 smithi167 ceph-mon[32751]: pgmap v18436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:42.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33563.373292 secs 2024-04-27T03:54:44.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:43 smithi090 ceph-mon[29180]: pgmap v18437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:43 smithi119 ceph-mon[32752]: pgmap v18437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:44.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:43 smithi167 ceph-mon[32751]: pgmap v18437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:46.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:45 smithi167 ceph-mon[32751]: pgmap v18438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:46.123 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33567.503509 secs 2024-04-27T03:54:46.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:45 smithi090 ceph-mon[29180]: pgmap v18438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:46.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33567.503509 secs 2024-04-27T03:54:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:45 smithi119 ceph-mon[32752]: pgmap v18438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:46.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33567.503509 secs 2024-04-27T03:54:47.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33568.373419 secs 2024-04-27T03:54:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33568.373419 secs 2024-04-27T03:54:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33568.373419 secs 2024-04-27T03:54:48.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:47 smithi090 ceph-mon[29180]: pgmap v18439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:47 smithi119 ceph-mon[32752]: pgmap v18439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:48.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:47 smithi167 ceph-mon[32751]: pgmap v18439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:49 smithi090 ceph-mon[29180]: pgmap v18440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:49 smithi119 ceph-mon[32752]: pgmap v18440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:49 smithi167 ceph-mon[32751]: pgmap v18440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:51.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33572.503592 secs 2024-04-27T03:54:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33572.503592 secs 2024-04-27T03:54:51.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33572.503592 secs 2024-04-27T03:54:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:51 smithi119 ceph-mon[32752]: pgmap v18441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:52.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33573.373479 secs 2024-04-27T03:54:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:51 smithi167 ceph-mon[32751]: pgmap v18441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:52.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33573.373479 secs 2024-04-27T03:54:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:51 smithi090 ceph-mon[29180]: pgmap v18441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:52.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33573.373479 secs 2024-04-27T03:54:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:53 smithi119 ceph-mon[32752]: pgmap v18442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:53 smithi167 ceph-mon[32751]: pgmap v18442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:53 smithi090 ceph-mon[29180]: pgmap v18442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:55 smithi119 ceph-mon[32752]: pgmap v18443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:56.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33577.503678 secs 2024-04-27T03:54:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:55 smithi167 ceph-mon[32751]: pgmap v18443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33577.503678 secs 2024-04-27T03:54:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:55 smithi090 ceph-mon[29180]: pgmap v18443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:54:56.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33577.503678 secs 2024-04-27T03:54:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33578.373604 secs 2024-04-27T03:54:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33578.373604 secs 2024-04-27T03:54:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33578.373604 secs 2024-04-27T03:54:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:57 smithi119 ceph-mon[32752]: pgmap v18444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:57 smithi167 ceph-mon[32751]: pgmap v18444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:54:58.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:57 smithi090 ceph-mon[29180]: pgmap v18444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:00.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:54:59 smithi119 ceph-mon[32752]: pgmap v18445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:00.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:54:59 smithi167 ceph-mon[32751]: pgmap v18445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:00.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:54:59 smithi090 ceph-mon[29180]: pgmap v18445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33582.503771 secs 2024-04-27T03:55:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33582.503771 secs 2024-04-27T03:55:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33582.503771 secs 2024-04-27T03:55:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:01 smithi119 ceph-mon[32752]: pgmap v18446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33583.373725 secs 2024-04-27T03:55:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:01 smithi167 ceph-mon[32751]: pgmap v18446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33583.373725 secs 2024-04-27T03:55:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:01 smithi090 ceph-mon[29180]: pgmap v18446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:02.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33583.373725 secs 2024-04-27T03:55:04.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:03 smithi119 ceph-mon[32752]: pgmap v18447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:04.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:03 smithi167 ceph-mon[32751]: pgmap v18447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:04.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:03 smithi090 ceph-mon[29180]: pgmap v18447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:55:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:55:05.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:55:06.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:05 smithi119 ceph-mon[32752]: pgmap v18448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:06.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33587.503843 secs 2024-04-27T03:55:06.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:05 smithi090 ceph-mon[29180]: pgmap v18448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33587.503843 secs 2024-04-27T03:55:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:05 smithi167 ceph-mon[32751]: pgmap v18448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:06.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33587.503843 secs 2024-04-27T03:55:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33588.373803 secs 2024-04-27T03:55:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:07.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33588.373803 secs 2024-04-27T03:55:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:07.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33588.373803 secs 2024-04-27T03:55:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: pgmap v18449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: pgmap v18449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: pgmap v18449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:08.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:55:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:55:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:55:09.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:55:09.309 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:55:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:55:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:55:09.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:55:09.323 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T03:55:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T03:55:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.380 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T03:55:09.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:55:09.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:55:09.381 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:55:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:55:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:09 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:55:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:09 smithi119 ceph-mon[32752]: pgmap v18450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:09 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:55:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:09 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:55:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:09 smithi119 ceph-mon[32752]: pgmap v18451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:55:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:09 smithi119 ceph-mon[32752]: pgmap v18452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:55:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:09 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:55:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:09 smithi167 ceph-mon[32751]: pgmap v18450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:09 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:55:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:09 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:55:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:09 smithi167 ceph-mon[32751]: pgmap v18451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:55:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:09 smithi167 ceph-mon[32751]: pgmap v18452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T03:55:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:09 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T03:55:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:09 smithi090 ceph-mon[29180]: pgmap v18450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:09 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T03:55:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:09 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T03:55:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:09 smithi090 ceph-mon[29180]: pgmap v18451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:55:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:09 smithi090 ceph-mon[29180]: pgmap v18452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33592.503926 secs 2024-04-27T03:55:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33592.503926 secs 2024-04-27T03:55:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33592.503926 secs 2024-04-27T03:55:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:12 smithi119 ceph-mon[32752]: pgmap v18453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:12 smithi119 ceph-mon[32752]: 31 slow requests, 3 included below; oldest blocked for > 33593.373945 secs 2024-04-27T03:55:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:12 smithi119 ceph-mon[32752]: slow request 11524.255078 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:12 smithi119 ceph-mon[32752]: slow request 5762.644529 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:12 smithi119 ceph-mon[32752]: slow request 721.226513 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:12 smithi167 ceph-mon[32751]: pgmap v18453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:12 smithi167 ceph-mon[32751]: 31 slow requests, 3 included below; oldest blocked for > 33593.373945 secs 2024-04-27T03:55:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:12 smithi167 ceph-mon[32751]: slow request 11524.255078 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:12 smithi167 ceph-mon[32751]: slow request 5762.644529 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:12 smithi167 ceph-mon[32751]: slow request 721.226513 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:12 smithi090 ceph-mon[29180]: pgmap v18453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:12 smithi090 ceph-mon[29180]: 31 slow requests, 3 included below; oldest blocked for > 33593.373945 secs 2024-04-27T03:55:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:12 smithi090 ceph-mon[29180]: slow request 11524.255078 seconds old, received at 2024-04-27T00:43:06.869512+0000: client_request(client.17526:1 getattr pAsLsXsFs #0x1 2024-04-27T00:43:06.865691+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:12 smithi090 ceph-mon[29180]: slow request 5762.644529 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:12 smithi090 ceph-mon[29180]: slow request 721.226513 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T03:55:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:14 smithi119 ceph-mon[32752]: pgmap v18454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:55:14.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:14 smithi167 ceph-mon[32751]: pgmap v18454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:55:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:14 smithi090 ceph-mon[29180]: pgmap v18454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:55:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:16 smithi119 ceph-mon[32752]: pgmap v18455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33597.503986 secs 2024-04-27T03:55:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:16 smithi167 ceph-mon[32751]: pgmap v18455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33597.503986 secs 2024-04-27T03:55:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:16 smithi090 ceph-mon[29180]: pgmap v18455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33597.503986 secs 2024-04-27T03:55:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33598.374107 secs 2024-04-27T03:55:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33598.374107 secs 2024-04-27T03:55:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33598.374107 secs 2024-04-27T03:55:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:18 smithi090 ceph-mon[29180]: pgmap v18456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:55:18.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:18 smithi119 ceph-mon[32752]: pgmap v18456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:55:18.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:18 smithi167 ceph-mon[32751]: pgmap v18456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T03:55:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:20 smithi119 ceph-mon[32752]: pgmap v18457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:55:20.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:20 smithi167 ceph-mon[32751]: pgmap v18457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:55:20.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:20 smithi090 ceph-mon[29180]: pgmap v18457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:55:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33602.504065 secs 2024-04-27T03:55:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33602.504065 secs 2024-04-27T03:55:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33602.504065 secs 2024-04-27T03:55:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:22 smithi119 ceph-mon[32752]: pgmap v18458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:22.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33603.374248 secs 2024-04-27T03:55:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:22 smithi167 ceph-mon[32751]: pgmap v18458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33603.374248 secs 2024-04-27T03:55:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:22 smithi090 ceph-mon[29180]: pgmap v18458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33603.374248 secs 2024-04-27T03:55:24.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:24 smithi119 ceph-mon[32752]: pgmap v18459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:24.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:24 smithi167 ceph-mon[32751]: pgmap v18459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:24.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:24 smithi090 ceph-mon[29180]: pgmap v18459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:26 smithi119 ceph-mon[32752]: pgmap v18460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:26 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33607.504147 secs 2024-04-27T03:55:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:26 smithi167 ceph-mon[32751]: pgmap v18460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:26 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33607.504147 secs 2024-04-27T03:55:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:26 smithi090 ceph-mon[29180]: pgmap v18460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:26.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:26 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33607.504147 secs 2024-04-27T03:55:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33608.374374 secs 2024-04-27T03:55:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33608.374374 secs 2024-04-27T03:55:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33608.374374 secs 2024-04-27T03:55:28.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:28 smithi119 ceph-mon[32752]: pgmap v18461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:28.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:28 smithi167 ceph-mon[32751]: pgmap v18461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:28.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:28 smithi090 ceph-mon[29180]: pgmap v18461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:29.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:29 smithi119 ceph-mon[32752]: pgmap v18462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:29.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:29 smithi167 ceph-mon[32751]: pgmap v18462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:29.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:29 smithi090 ceph-mon[29180]: pgmap v18462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:31 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33612.504222 secs 2024-04-27T03:55:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:31 smithi119 ceph-mon[32752]: pgmap v18463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33613.374511 secs 2024-04-27T03:55:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:31 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33612.504222 secs 2024-04-27T03:55:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:31 smithi167 ceph-mon[32751]: pgmap v18463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33613.374511 secs 2024-04-27T03:55:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:31 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33612.504222 secs 2024-04-27T03:55:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:31 smithi090 ceph-mon[29180]: pgmap v18463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33613.374511 secs 2024-04-27T03:55:33.736 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:33 smithi090 ceph-mon[29180]: pgmap v18464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:33 smithi119 ceph-mon[32752]: pgmap v18464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:33.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:33 smithi167 ceph-mon[32751]: pgmap v18464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:35 smithi119 ceph-mon[32752]: pgmap v18465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:35 smithi167 ceph-mon[32751]: pgmap v18465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:35 smithi090 ceph-mon[29180]: pgmap v18465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:36 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33617.504304 secs 2024-04-27T03:55:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33618.374612 secs 2024-04-27T03:55:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:36 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33617.504304 secs 2024-04-27T03:55:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33618.374612 secs 2024-04-27T03:55:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:36 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33617.504304 secs 2024-04-27T03:55:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33618.374612 secs 2024-04-27T03:55:37.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:37 smithi090 ceph-mon[29180]: pgmap v18466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:37 smithi119 ceph-mon[32752]: pgmap v18466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:37 smithi167 ceph-mon[32751]: pgmap v18466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:39 smithi119 ceph-mon[32752]: pgmap v18467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:39 smithi167 ceph-mon[32751]: pgmap v18467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:39 smithi090 ceph-mon[29180]: pgmap v18467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:41 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33622.504388 secs 2024-04-27T03:55:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:41 smithi119 ceph-mon[32752]: pgmap v18468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33623.374733 secs 2024-04-27T03:55:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:41 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33622.504388 secs 2024-04-27T03:55:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:41 smithi167 ceph-mon[32751]: pgmap v18468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33623.374733 secs 2024-04-27T03:55:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:41 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33622.504388 secs 2024-04-27T03:55:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:41 smithi090 ceph-mon[29180]: pgmap v18468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:41.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33623.374733 secs 2024-04-27T03:55:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:43 smithi119 ceph-mon[32752]: pgmap v18469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:43 smithi167 ceph-mon[32751]: pgmap v18469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:43 smithi090 ceph-mon[29180]: pgmap v18469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:45 smithi119 ceph-mon[32752]: pgmap v18470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:45 smithi167 ceph-mon[32751]: pgmap v18470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:45 smithi090 ceph-mon[29180]: pgmap v18470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:46 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33627.504469 secs 2024-04-27T03:55:46.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33628.374796 secs 2024-04-27T03:55:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:46 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33627.504469 secs 2024-04-27T03:55:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33628.374796 secs 2024-04-27T03:55:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:46 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33627.504469 secs 2024-04-27T03:55:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33628.374796 secs 2024-04-27T03:55:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:47 smithi119 ceph-mon[32752]: pgmap v18471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:47 smithi167 ceph-mon[32751]: pgmap v18471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:47 smithi090 ceph-mon[29180]: pgmap v18471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:49 smithi119 ceph-mon[32752]: pgmap v18472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:49 smithi167 ceph-mon[32751]: pgmap v18472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:49 smithi090 ceph-mon[29180]: pgmap v18472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:51 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33632.504555 secs 2024-04-27T03:55:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:51 smithi119 ceph-mon[32752]: pgmap v18473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33633.374862 secs 2024-04-27T03:55:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:51 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33632.504555 secs 2024-04-27T03:55:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:51 smithi167 ceph-mon[32751]: pgmap v18473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33633.374862 secs 2024-04-27T03:55:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:51 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33632.504555 secs 2024-04-27T03:55:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:51 smithi090 ceph-mon[29180]: pgmap v18473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33633.374862 secs 2024-04-27T03:55:53.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:53 smithi119 ceph-mon[32752]: pgmap v18474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:53.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:53 smithi167 ceph-mon[32751]: pgmap v18474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:53 smithi090 ceph-mon[29180]: pgmap v18474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:55.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:55 smithi119 ceph-mon[32752]: pgmap v18475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:55.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:55 smithi167 ceph-mon[32751]: pgmap v18475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:55 smithi090 ceph-mon[29180]: pgmap v18475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:56 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33637.504638 secs 2024-04-27T03:55:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33638.374990 secs 2024-04-27T03:55:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:56 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33637.504638 secs 2024-04-27T03:55:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33638.374990 secs 2024-04-27T03:55:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:56 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33637.504638 secs 2024-04-27T03:55:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33638.374990 secs 2024-04-27T03:55:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:57 smithi119 ceph-mon[32752]: pgmap v18476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:57 smithi167 ceph-mon[32751]: pgmap v18476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:57 smithi090 ceph-mon[29180]: pgmap v18476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:55:59.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:55:59 smithi119 ceph-mon[32752]: pgmap v18477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:59.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:55:59 smithi167 ceph-mon[32751]: pgmap v18477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:55:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:55:59 smithi090 ceph-mon[29180]: pgmap v18477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:01 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 33642.504721 secs 2024-04-27T03:56:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:01 smithi119 ceph-mon[32752]: slow request 363.981883 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:56:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:01 smithi119 ceph-mon[32752]: pgmap v18478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33643.375110 secs 2024-04-27T03:56:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:01 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 33642.504721 secs 2024-04-27T03:56:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:01 smithi167 ceph-mon[32751]: slow request 363.981883 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:56:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:01 smithi167 ceph-mon[32751]: pgmap v18478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33643.375110 secs 2024-04-27T03:56:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:01 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 33642.504721 secs 2024-04-27T03:56:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:01 smithi090 ceph-mon[29180]: slow request 363.981883 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:56:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:01 smithi090 ceph-mon[29180]: pgmap v18478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:01.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33643.375110 secs 2024-04-27T03:56:03.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:03 smithi119 ceph-mon[32752]: pgmap v18479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:03.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:03 smithi167 ceph-mon[32751]: pgmap v18479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:03 smithi090 ceph-mon[29180]: pgmap v18479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:05 smithi119 ceph-mon[32752]: pgmap v18480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:05 smithi167 ceph-mon[32751]: pgmap v18480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:05 smithi090 ceph-mon[29180]: pgmap v18480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:06 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33647.504801 secs 2024-04-27T03:56:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33648.375258 secs 2024-04-27T03:56:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:06 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33647.504801 secs 2024-04-27T03:56:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33648.375258 secs 2024-04-27T03:56:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:06 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33647.504801 secs 2024-04-27T03:56:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33648.375258 secs 2024-04-27T03:56:07.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:07 smithi090 ceph-mon[29180]: pgmap v18481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:07 smithi119 ceph-mon[32752]: pgmap v18481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:07 smithi167 ceph-mon[32751]: pgmap v18481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:56:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:56:08.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:56:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:10 smithi119 ceph-mon[32752]: pgmap v18482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:56:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:56:10.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:10 smithi167 ceph-mon[32751]: pgmap v18482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:56:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:56:10.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:10 smithi090 ceph-mon[29180]: pgmap v18482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:56:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:56:10.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:56:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:11 smithi119 ceph-mon[32752]: pgmap v18483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T03:56:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:11 smithi119 ceph-mon[32752]: pgmap v18484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33652.504863 secs 2024-04-27T03:56:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:11 smithi167 ceph-mon[32751]: pgmap v18483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T03:56:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:11 smithi167 ceph-mon[32751]: pgmap v18484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33652.504863 secs 2024-04-27T03:56:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:11 smithi090 ceph-mon[29180]: pgmap v18483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T03:56:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:11 smithi090 ceph-mon[29180]: pgmap v18484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33652.504863 secs 2024-04-27T03:56:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33653.375403 secs 2024-04-27T03:56:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33653.375403 secs 2024-04-27T03:56:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33653.375403 secs 2024-04-27T03:56:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:13 smithi119 ceph-mon[32752]: pgmap v18485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:13.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:13 smithi167 ceph-mon[32751]: pgmap v18485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:13 smithi090 ceph-mon[29180]: pgmap v18485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:15.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:15 smithi119 ceph-mon[32752]: pgmap v18486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:15.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:15 smithi167 ceph-mon[32751]: pgmap v18486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:15.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:15 smithi090 ceph-mon[29180]: pgmap v18486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33657.504945 secs 2024-04-27T03:56:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33657.504945 secs 2024-04-27T03:56:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33657.504945 secs 2024-04-27T03:56:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:17 smithi119 ceph-mon[32752]: pgmap v18487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33658.375531 secs 2024-04-27T03:56:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:17 smithi167 ceph-mon[32751]: pgmap v18487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33658.375531 secs 2024-04-27T03:56:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:17 smithi090 ceph-mon[29180]: pgmap v18487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:56:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33658.375531 secs 2024-04-27T03:56:18.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:18 smithi090 ceph-mon[29180]: pgmap v18488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 554 B/s rd, 1 op/s 2024-04-27T03:56:18.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:18 smithi119 ceph-mon[32752]: pgmap v18488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 554 B/s rd, 1 op/s 2024-04-27T03:56:18.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:18 smithi167 ceph-mon[32751]: pgmap v18488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 554 B/s rd, 1 op/s 2024-04-27T03:56:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:20 smithi119 ceph-mon[32752]: pgmap v18489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:56:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33662.505007 secs 2024-04-27T03:56:21.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:20 smithi167 ceph-mon[32751]: pgmap v18489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:56:21.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33662.505007 secs 2024-04-27T03:56:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:20 smithi090 ceph-mon[29180]: pgmap v18489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:56:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33662.505007 secs 2024-04-27T03:56:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33663.375671 secs 2024-04-27T03:56:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33663.375671 secs 2024-04-27T03:56:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33663.375671 secs 2024-04-27T03:56:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:22 smithi119 ceph-mon[32752]: pgmap v18490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:22 smithi167 ceph-mon[32751]: pgmap v18490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:22 smithi090 ceph-mon[29180]: pgmap v18490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:25.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:24 smithi119 ceph-mon[32752]: pgmap v18491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:25.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:24 smithi167 ceph-mon[32751]: pgmap v18491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:24 smithi090 ceph-mon[29180]: pgmap v18491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33667.505090 secs 2024-04-27T03:56:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33667.505090 secs 2024-04-27T03:56:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33667.505090 secs 2024-04-27T03:56:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:26 smithi119 ceph-mon[32752]: pgmap v18492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33668.375791 secs 2024-04-27T03:56:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:26 smithi167 ceph-mon[32751]: pgmap v18492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33668.375791 secs 2024-04-27T03:56:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:26 smithi090 ceph-mon[29180]: pgmap v18492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33668.375791 secs 2024-04-27T03:56:29.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:28 smithi119 ceph-mon[32752]: pgmap v18493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:28 smithi167 ceph-mon[32751]: pgmap v18493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:29.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:28 smithi090 ceph-mon[29180]: pgmap v18493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:31.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:30 smithi119 ceph-mon[32752]: pgmap v18494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:31.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33672.505174 secs 2024-04-27T03:56:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:30 smithi167 ceph-mon[32751]: pgmap v18494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33672.505174 secs 2024-04-27T03:56:31.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:30 smithi090 ceph-mon[29180]: pgmap v18494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:31.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33672.505174 secs 2024-04-27T03:56:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33673.375914 secs 2024-04-27T03:56:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33673.375914 secs 2024-04-27T03:56:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33673.375914 secs 2024-04-27T03:56:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:32 smithi119 ceph-mon[32752]: pgmap v18495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:32 smithi167 ceph-mon[32751]: pgmap v18495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:32 smithi090 ceph-mon[29180]: pgmap v18495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:34 smithi119 ceph-mon[32752]: pgmap v18496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:35.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:34 smithi167 ceph-mon[32751]: pgmap v18496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:35.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:34 smithi090 ceph-mon[29180]: pgmap v18496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33677.505255 secs 2024-04-27T03:56:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33677.505255 secs 2024-04-27T03:56:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33677.505255 secs 2024-04-27T03:56:37.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:36 smithi090 ceph-mon[29180]: pgmap v18497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:37.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33678.376039 secs 2024-04-27T03:56:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:36 smithi119 ceph-mon[32752]: pgmap v18497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33678.376039 secs 2024-04-27T03:56:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:36 smithi167 ceph-mon[32751]: pgmap v18497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33678.376039 secs 2024-04-27T03:56:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:38 smithi119 ceph-mon[32752]: pgmap v18498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:38 smithi167 ceph-mon[32751]: pgmap v18498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:39.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:38 smithi090 ceph-mon[29180]: pgmap v18498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:40 smithi119 ceph-mon[32752]: pgmap v18499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:41.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33682.505345 secs 2024-04-27T03:56:41.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:40 smithi167 ceph-mon[32751]: pgmap v18499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:41.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33682.505345 secs 2024-04-27T03:56:41.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:40 smithi090 ceph-mon[29180]: pgmap v18499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:41.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33682.505345 secs 2024-04-27T03:56:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33683.376119 secs 2024-04-27T03:56:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33683.376119 secs 2024-04-27T03:56:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33683.376119 secs 2024-04-27T03:56:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:42 smithi119 ceph-mon[32752]: pgmap v18500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:42 smithi167 ceph-mon[32751]: pgmap v18500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:42 smithi090 ceph-mon[29180]: pgmap v18500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:44 smithi119 ceph-mon[32752]: pgmap v18501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:44 smithi167 ceph-mon[32751]: pgmap v18501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:45.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:44 smithi090 ceph-mon[29180]: pgmap v18501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:46 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33687.505430 secs 2024-04-27T03:56:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:46 smithi119 ceph-mon[32752]: pgmap v18502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:46 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33687.505430 secs 2024-04-27T03:56:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:46 smithi167 ceph-mon[32751]: pgmap v18502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:46 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33687.505430 secs 2024-04-27T03:56:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:46 smithi090 ceph-mon[29180]: pgmap v18502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33688.376233 secs 2024-04-27T03:56:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33688.376233 secs 2024-04-27T03:56:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33688.376233 secs 2024-04-27T03:56:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:48 smithi167 ceph-mon[32751]: pgmap v18503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:48 smithi090 ceph-mon[29180]: pgmap v18503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:48 smithi119 ceph-mon[32752]: pgmap v18503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:50 smithi119 ceph-mon[32752]: pgmap v18504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33692.505516 secs 2024-04-27T03:56:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:50 smithi167 ceph-mon[32751]: pgmap v18504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33692.505516 secs 2024-04-27T03:56:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:50 smithi090 ceph-mon[29180]: pgmap v18504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33692.505516 secs 2024-04-27T03:56:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33693.376301 secs 2024-04-27T03:56:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33693.376301 secs 2024-04-27T03:56:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33693.376301 secs 2024-04-27T03:56:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:52 smithi119 ceph-mon[32752]: pgmap v18505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:52 smithi167 ceph-mon[32751]: pgmap v18505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:52 smithi090 ceph-mon[29180]: pgmap v18505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:54 smithi119 ceph-mon[32752]: pgmap v18506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:54 smithi167 ceph-mon[32751]: pgmap v18506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:54 smithi090 ceph-mon[29180]: pgmap v18506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:56:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33697.505600 secs 2024-04-27T03:56:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33697.505600 secs 2024-04-27T03:56:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33697.505600 secs 2024-04-27T03:56:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:56 smithi119 ceph-mon[32752]: pgmap v18507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33698.376369 secs 2024-04-27T03:56:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:56 smithi167 ceph-mon[32751]: pgmap v18507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33698.376369 secs 2024-04-27T03:56:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:56 smithi090 ceph-mon[29180]: pgmap v18507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33698.376369 secs 2024-04-27T03:56:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:56:58 smithi119 ceph-mon[32752]: pgmap v18508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:56:58 smithi167 ceph-mon[32751]: pgmap v18508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:56:59.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:56:58 smithi090 ceph-mon[29180]: pgmap v18508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:00 smithi119 ceph-mon[32752]: pgmap v18509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33702.505686 secs 2024-04-27T03:57:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:00 smithi167 ceph-mon[32751]: pgmap v18509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33702.505686 secs 2024-04-27T03:57:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:00 smithi090 ceph-mon[29180]: pgmap v18509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33702.505686 secs 2024-04-27T03:57:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33703.376409 secs 2024-04-27T03:57:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33703.376409 secs 2024-04-27T03:57:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33703.376409 secs 2024-04-27T03:57:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:02 smithi119 ceph-mon[32752]: pgmap v18510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:02 smithi167 ceph-mon[32751]: pgmap v18510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:02 smithi090 ceph-mon[29180]: pgmap v18510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:04 smithi119 ceph-mon[32752]: pgmap v18511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:04 smithi167 ceph-mon[32751]: pgmap v18511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:04 smithi090 ceph-mon[29180]: pgmap v18511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33707.505766 secs 2024-04-27T03:57:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33707.505766 secs 2024-04-27T03:57:06.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33707.505766 secs 2024-04-27T03:57:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:06 smithi119 ceph-mon[32752]: pgmap v18512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33708.376498 secs 2024-04-27T03:57:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:06 smithi167 ceph-mon[32751]: pgmap v18512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33708.376498 secs 2024-04-27T03:57:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:06 smithi090 ceph-mon[29180]: pgmap v18512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33708.376498 secs 2024-04-27T03:57:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:08 smithi119 ceph-mon[32752]: pgmap v18513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:08 smithi167 ceph-mon[32751]: pgmap v18513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:08 smithi090 ceph-mon[29180]: pgmap v18513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T03:57:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:10 smithi119 ceph-mon[32752]: pgmap v18514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:57:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:10 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33712.505836 secs 2024-04-27T03:57:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:57:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:57:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:57:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:10 smithi167 ceph-mon[32751]: pgmap v18514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:57:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:10 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33712.505836 secs 2024-04-27T03:57:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:57:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:57:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:57:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:10 smithi090 ceph-mon[29180]: pgmap v18514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:57:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:10 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33712.505836 secs 2024-04-27T03:57:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:57:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:57:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:57:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:11 smithi119 ceph-mon[32752]: pgmap v18515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T03:57:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:11 smithi119 ceph-mon[32752]: pgmap v18516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33713.376563 secs 2024-04-27T03:57:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:11 smithi167 ceph-mon[32751]: pgmap v18515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T03:57:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:11 smithi167 ceph-mon[32751]: pgmap v18516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33713.376563 secs 2024-04-27T03:57:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:11 smithi090 ceph-mon[29180]: pgmap v18515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T03:57:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:11 smithi090 ceph-mon[29180]: pgmap v18516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33713.376563 secs 2024-04-27T03:57:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:13 smithi119 ceph-mon[32752]: pgmap v18517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T03:57:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:13 smithi167 ceph-mon[32751]: pgmap v18517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T03:57:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:13 smithi090 ceph-mon[29180]: pgmap v18517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T03:57:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:15 smithi119 ceph-mon[32752]: pgmap v18518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:15 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33717.505893 secs 2024-04-27T03:57:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:15 smithi167 ceph-mon[32751]: pgmap v18518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:15 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33717.505893 secs 2024-04-27T03:57:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:15 smithi090 ceph-mon[29180]: pgmap v18518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:15 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33717.505893 secs 2024-04-27T03:57:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33718.376629 secs 2024-04-27T03:57:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33718.376629 secs 2024-04-27T03:57:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33718.376629 secs 2024-04-27T03:57:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:17 smithi119 ceph-mon[32752]: pgmap v18519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:17 smithi167 ceph-mon[32751]: pgmap v18519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:17 smithi090 ceph-mon[29180]: pgmap v18519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:19 smithi119 ceph-mon[32752]: pgmap v18520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:20.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:19 smithi167 ceph-mon[32751]: pgmap v18520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:19 smithi090 ceph-mon[29180]: pgmap v18520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:57:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33722.505978 secs 2024-04-27T03:57:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33722.505978 secs 2024-04-27T03:57:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33722.505978 secs 2024-04-27T03:57:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:21 smithi090 ceph-mon[29180]: pgmap v18521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:57:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33723.376733 secs 2024-04-27T03:57:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:21 smithi119 ceph-mon[32752]: pgmap v18521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:57:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33723.376733 secs 2024-04-27T03:57:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:21 smithi167 ceph-mon[32751]: pgmap v18521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:57:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33723.376733 secs 2024-04-27T03:57:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:23 smithi119 ceph-mon[32752]: pgmap v18522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:23 smithi167 ceph-mon[32751]: pgmap v18522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:23 smithi090 ceph-mon[29180]: pgmap v18522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:25 smithi119 ceph-mon[32752]: pgmap v18523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33727.506039 secs 2024-04-27T03:57:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:25 smithi167 ceph-mon[32751]: pgmap v18523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33727.506039 secs 2024-04-27T03:57:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:25 smithi090 ceph-mon[29180]: pgmap v18523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33727.506039 secs 2024-04-27T03:57:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33728.376801 secs 2024-04-27T03:57:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33728.376801 secs 2024-04-27T03:57:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33728.376801 secs 2024-04-27T03:57:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:27 smithi119 ceph-mon[32752]: pgmap v18524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:27 smithi167 ceph-mon[32751]: pgmap v18524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:27 smithi090 ceph-mon[29180]: pgmap v18524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:29 smithi119 ceph-mon[32752]: pgmap v18525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:29 smithi167 ceph-mon[32751]: pgmap v18525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:29 smithi090 ceph-mon[29180]: pgmap v18525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33732.506124 secs 2024-04-27T03:57:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33732.506124 secs 2024-04-27T03:57:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33732.506124 secs 2024-04-27T03:57:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:31 smithi167 ceph-mon[32751]: pgmap v18526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33733.376895 secs 2024-04-27T03:57:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:31 smithi090 ceph-mon[29180]: pgmap v18526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33733.376895 secs 2024-04-27T03:57:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:31 smithi119 ceph-mon[32752]: pgmap v18526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33733.376895 secs 2024-04-27T03:57:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:33 smithi167 ceph-mon[32751]: pgmap v18527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:34.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:33 smithi090 ceph-mon[29180]: pgmap v18527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:34.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:33 smithi119 ceph-mon[32752]: pgmap v18527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:35 smithi167 ceph-mon[32751]: pgmap v18528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33737.506181 secs 2024-04-27T03:57:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:35 smithi090 ceph-mon[29180]: pgmap v18528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33737.506181 secs 2024-04-27T03:57:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:35 smithi119 ceph-mon[32752]: pgmap v18528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33737.506181 secs 2024-04-27T03:57:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33738.376982 secs 2024-04-27T03:57:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33738.376982 secs 2024-04-27T03:57:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33738.376982 secs 2024-04-27T03:57:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:37 smithi090 ceph-mon[29180]: pgmap v18529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:37 smithi119 ceph-mon[32752]: pgmap v18529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:38.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:37 smithi167 ceph-mon[32751]: pgmap v18529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:39 smithi090 ceph-mon[29180]: pgmap v18530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:40.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:39 smithi119 ceph-mon[32752]: pgmap v18530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:40.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:39 smithi167 ceph-mon[32751]: pgmap v18530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33742.506268 secs 2024-04-27T03:57:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33742.506268 secs 2024-04-27T03:57:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33742.506268 secs 2024-04-27T03:57:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:41 smithi090 ceph-mon[29180]: pgmap v18531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33743.377098 secs 2024-04-27T03:57:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:41 smithi119 ceph-mon[32752]: pgmap v18531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33743.377098 secs 2024-04-27T03:57:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:41 smithi167 ceph-mon[32751]: pgmap v18531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33743.377098 secs 2024-04-27T03:57:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:43 smithi090 ceph-mon[29180]: pgmap v18532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:44.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:43 smithi119 ceph-mon[32752]: pgmap v18532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:44.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:43 smithi167 ceph-mon[32751]: pgmap v18532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:46.107 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:45 smithi119 ceph-mon[32752]: pgmap v18533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:46.107 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33747.506329 secs 2024-04-27T03:57:46.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:45 smithi167 ceph-mon[32751]: pgmap v18533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:46.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33747.506329 secs 2024-04-27T03:57:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:45 smithi090 ceph-mon[29180]: pgmap v18533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33747.506329 secs 2024-04-27T03:57:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33748.377230 secs 2024-04-27T03:57:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33748.377230 secs 2024-04-27T03:57:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33748.377230 secs 2024-04-27T03:57:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:47 smithi090 ceph-mon[29180]: pgmap v18534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:47 smithi119 ceph-mon[32752]: pgmap v18534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:47 smithi167 ceph-mon[32751]: pgmap v18534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:49 smithi090 ceph-mon[29180]: pgmap v18535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:49 smithi119 ceph-mon[32752]: pgmap v18535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:49 smithi167 ceph-mon[32751]: pgmap v18535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:50 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 33752.506411 secs 2024-04-27T03:57:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:50 smithi090 ceph-mon[29180]: slow request 11521.880499 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:57:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:50 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 33752.506411 secs 2024-04-27T03:57:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:50 smithi119 ceph-mon[32752]: slow request 11521.880499 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:57:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:50 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 33752.506411 secs 2024-04-27T03:57:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:50 smithi167 ceph-mon[32751]: slow request 11521.880499 seconds old, received at 2024-04-27T00:45:48.379316+0000: client_request(client.24733:267897 getattr AsXsFs #0x100000001fb 2024-04-27T00:45:48.378336+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T03:57:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:51 smithi090 ceph-mon[29180]: pgmap v18536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33753.377368 secs 2024-04-27T03:57:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:51 smithi119 ceph-mon[32752]: pgmap v18536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33753.377368 secs 2024-04-27T03:57:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:51 smithi167 ceph-mon[32751]: pgmap v18536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33753.377368 secs 2024-04-27T03:57:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:53 smithi090 ceph-mon[29180]: pgmap v18537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:53 smithi119 ceph-mon[32752]: pgmap v18537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:53 smithi167 ceph-mon[32751]: pgmap v18537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:55 smithi090 ceph-mon[29180]: pgmap v18538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33757.506478 secs 2024-04-27T03:57:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:55 smithi119 ceph-mon[32752]: pgmap v18538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33757.506478 secs 2024-04-27T03:57:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:55 smithi167 ceph-mon[32751]: pgmap v18538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:57:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33757.506478 secs 2024-04-27T03:57:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33758.377447 secs 2024-04-27T03:57:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33758.377447 secs 2024-04-27T03:57:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33758.377447 secs 2024-04-27T03:57:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:57 smithi090 ceph-mon[29180]: pgmap v18539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:57 smithi119 ceph-mon[32752]: pgmap v18539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:57:58.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:57 smithi167 ceph-mon[32751]: pgmap v18539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:57:59 smithi090 ceph-mon[29180]: pgmap v18540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:57:59 smithi119 ceph-mon[32752]: pgmap v18540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:57:59 smithi167 ceph-mon[32751]: pgmap v18540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33762.506582 secs 2024-04-27T03:58:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33762.506582 secs 2024-04-27T03:58:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33762.506582 secs 2024-04-27T03:58:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:01 smithi119 ceph-mon[32752]: pgmap v18541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33763.377513 secs 2024-04-27T03:58:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:01 smithi167 ceph-mon[32751]: pgmap v18541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33763.377513 secs 2024-04-27T03:58:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:01 smithi090 ceph-mon[29180]: pgmap v18541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33763.377513 secs 2024-04-27T03:58:04.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:03 smithi119 ceph-mon[32752]: pgmap v18542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:04.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:03 smithi167 ceph-mon[32751]: pgmap v18542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:03 smithi090 ceph-mon[29180]: pgmap v18542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:05 smithi119 ceph-mon[32752]: pgmap v18543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33767.506662 secs 2024-04-27T03:58:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:05 smithi167 ceph-mon[32751]: pgmap v18543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33767.506662 secs 2024-04-27T03:58:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:05 smithi090 ceph-mon[29180]: pgmap v18543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33767.506662 secs 2024-04-27T03:58:07.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33768.377581 secs 2024-04-27T03:58:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33768.377581 secs 2024-04-27T03:58:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33768.377581 secs 2024-04-27T03:58:08.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:07 smithi119 ceph-mon[32752]: pgmap v18544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:08.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:07 smithi167 ceph-mon[32751]: pgmap v18544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:08.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:07 smithi090 ceph-mon[29180]: pgmap v18544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:10.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:09 smithi119 ceph-mon[32752]: pgmap v18545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:10.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:09 smithi167 ceph-mon[32751]: pgmap v18545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:10.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:09 smithi090 ceph-mon[29180]: pgmap v18545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:10 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33772.506744 secs 2024-04-27T03:58:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:10 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:58:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:10 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33772.506744 secs 2024-04-27T03:58:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:10 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:58:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:10 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33772.506744 secs 2024-04-27T03:58:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:10 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:58:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:11 smithi119 ceph-mon[32752]: pgmap v18546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33773.377650 secs 2024-04-27T03:58:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:58:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:58:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:58:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:11 smithi167 ceph-mon[32751]: pgmap v18546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33773.377650 secs 2024-04-27T03:58:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:58:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:58:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:58:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:11 smithi090 ceph-mon[29180]: pgmap v18546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33773.377650 secs 2024-04-27T03:58:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:58:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:58:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:58:13.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:12 smithi119 ceph-mon[32752]: pgmap v18547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:58:13.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:12 smithi119 ceph-mon[32752]: pgmap v18548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:58:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:12 smithi167 ceph-mon[32751]: pgmap v18547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:58:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:12 smithi167 ceph-mon[32751]: pgmap v18548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:58:13.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:12 smithi090 ceph-mon[29180]: pgmap v18547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T03:58:13.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:12 smithi090 ceph-mon[29180]: pgmap v18548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:58:15.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:14 smithi119 ceph-mon[32752]: pgmap v18549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 116 B/s wr, 1 op/s 2024-04-27T03:58:15.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:14 smithi167 ceph-mon[32751]: pgmap v18549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 116 B/s wr, 1 op/s 2024-04-27T03:58:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:14 smithi090 ceph-mon[29180]: pgmap v18549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 116 B/s wr, 1 op/s 2024-04-27T03:58:16.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:15 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33777.506849 secs 2024-04-27T03:58:16.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:15 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33777.506849 secs 2024-04-27T03:58:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:15 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33777.506849 secs 2024-04-27T03:58:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:16 smithi119 ceph-mon[32752]: pgmap v18550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T03:58:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33778.377762 secs 2024-04-27T03:58:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:16 smithi167 ceph-mon[32751]: pgmap v18550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T03:58:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33778.377762 secs 2024-04-27T03:58:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:16 smithi090 ceph-mon[29180]: pgmap v18550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T03:58:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33778.377762 secs 2024-04-27T03:58:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:18 smithi119 ceph-mon[32752]: pgmap v18551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T03:58:19.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:18 smithi167 ceph-mon[32751]: pgmap v18551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T03:58:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:18 smithi090 ceph-mon[29180]: pgmap v18551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 233 B/s wr, 1 op/s 2024-04-27T03:58:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:20 smithi119 ceph-mon[32752]: pgmap v18552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T03:58:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33782.506935 secs 2024-04-27T03:58:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:20 smithi167 ceph-mon[32751]: pgmap v18552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T03:58:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33782.506935 secs 2024-04-27T03:58:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:20 smithi090 ceph-mon[29180]: pgmap v18552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 233 B/s wr, 2 op/s 2024-04-27T03:58:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33782.506935 secs 2024-04-27T03:58:22.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33783.377872 secs 2024-04-27T03:58:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33783.377872 secs 2024-04-27T03:58:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33783.377872 secs 2024-04-27T03:58:23.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:22 smithi119 ceph-mon[32752]: pgmap v18553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T03:58:23.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:22 smithi167 ceph-mon[32751]: pgmap v18553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T03:58:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:22 smithi090 ceph-mon[29180]: pgmap v18553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 204 B/s wr, 2 op/s 2024-04-27T03:58:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:24 smithi090 ceph-mon[29180]: pgmap v18554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T03:58:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:24 smithi119 ceph-mon[32752]: pgmap v18554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T03:58:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:24 smithi167 ceph-mon[32751]: pgmap v18554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T03:58:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33787.506997 secs 2024-04-27T03:58:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33787.506997 secs 2024-04-27T03:58:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33787.506997 secs 2024-04-27T03:58:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:26 smithi119 ceph-mon[32752]: pgmap v18555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-04-27T03:58:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33788.377987 secs 2024-04-27T03:58:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:26 smithi167 ceph-mon[32751]: pgmap v18555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-04-27T03:58:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33788.377987 secs 2024-04-27T03:58:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:26 smithi090 ceph-mon[29180]: pgmap v18555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 2 op/s 2024-04-27T03:58:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33788.377987 secs 2024-04-27T03:58:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:28 smithi119 ceph-mon[32752]: pgmap v18556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:28 smithi167 ceph-mon[32751]: pgmap v18556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:28 smithi090 ceph-mon[29180]: pgmap v18556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:30 smithi119 ceph-mon[32752]: pgmap v18557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33792.507060 secs 2024-04-27T03:58:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:30 smithi167 ceph-mon[32751]: pgmap v18557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33792.507060 secs 2024-04-27T03:58:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:30 smithi090 ceph-mon[29180]: pgmap v18557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33792.507060 secs 2024-04-27T03:58:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33793.378129 secs 2024-04-27T03:58:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33793.378129 secs 2024-04-27T03:58:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33793.378129 secs 2024-04-27T03:58:33.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:32 smithi119 ceph-mon[32752]: pgmap v18558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:33.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:32 smithi167 ceph-mon[32751]: pgmap v18558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:32 smithi090 ceph-mon[29180]: pgmap v18558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:35.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:34 smithi119 ceph-mon[32752]: pgmap v18559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:35.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:34 smithi167 ceph-mon[32751]: pgmap v18559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:35.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:34 smithi090 ceph-mon[29180]: pgmap v18559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33797.507142 secs 2024-04-27T03:58:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33797.507142 secs 2024-04-27T03:58:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33797.507142 secs 2024-04-27T03:58:37.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:36 smithi090 ceph-mon[29180]: pgmap v18560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:37.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33798.378265 secs 2024-04-27T03:58:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:36 smithi119 ceph-mon[32752]: pgmap v18560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33798.378265 secs 2024-04-27T03:58:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:36 smithi167 ceph-mon[32751]: pgmap v18560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33798.378265 secs 2024-04-27T03:58:39.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:38 smithi119 ceph-mon[32752]: pgmap v18561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:38 smithi167 ceph-mon[32751]: pgmap v18561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:38 smithi090 ceph-mon[29180]: pgmap v18561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:40.615 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:58:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:58:40.136+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:58:40.615 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:58:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:58:40.136+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:58:40.615 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:58:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:58:40.136+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:58:40.615 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:58:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:58:40.136+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:58:40.615 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 03:58:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T03:58:40.136+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T03:58:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:40 smithi119 ceph-mon[32752]: pgmap v18562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33802.507226 secs 2024-04-27T03:58:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:40 smithi167 ceph-mon[32751]: pgmap v18562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33802.507226 secs 2024-04-27T03:58:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:40 smithi090 ceph-mon[29180]: pgmap v18562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33802.507226 secs 2024-04-27T03:58:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33803.378352 secs 2024-04-27T03:58:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33803.378352 secs 2024-04-27T03:58:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33803.378352 secs 2024-04-27T03:58:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:42 smithi119 ceph-mon[32752]: pgmap v18563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:42 smithi167 ceph-mon[32751]: pgmap v18563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:42 smithi090 ceph-mon[29180]: pgmap v18563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:45.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:44 smithi119 ceph-mon[32752]: pgmap v18564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:44 smithi167 ceph-mon[32751]: pgmap v18564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:44 smithi090 ceph-mon[29180]: pgmap v18564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:46 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33807.507303 secs 2024-04-27T03:58:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:46 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33807.507303 secs 2024-04-27T03:58:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:46 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33807.507303 secs 2024-04-27T03:58:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:47 smithi119 ceph-mon[32752]: pgmap v18565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33808.378468 secs 2024-04-27T03:58:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:47 smithi167 ceph-mon[32751]: pgmap v18565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33808.378468 secs 2024-04-27T03:58:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:47 smithi090 ceph-mon[29180]: pgmap v18565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33808.378468 secs 2024-04-27T03:58:49.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:49 smithi119 ceph-mon[32752]: pgmap v18566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:49 smithi167 ceph-mon[32751]: pgmap v18566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:49.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:49 smithi090 ceph-mon[29180]: pgmap v18566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:51 smithi119 ceph-mon[32752]: pgmap v18567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:51 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33812.507379 secs 2024-04-27T03:58:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:51 smithi167 ceph-mon[32751]: pgmap v18567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:51 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33812.507379 secs 2024-04-27T03:58:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:51 smithi090 ceph-mon[29180]: pgmap v18567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:51 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33812.507379 secs 2024-04-27T03:58:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33813.378586 secs 2024-04-27T03:58:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33813.378586 secs 2024-04-27T03:58:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33813.378586 secs 2024-04-27T03:58:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:53 smithi119 ceph-mon[32752]: pgmap v18568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:53.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:53 smithi167 ceph-mon[32751]: pgmap v18568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:53 smithi090 ceph-mon[29180]: pgmap v18568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:55 smithi119 ceph-mon[32752]: pgmap v18569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:55 smithi167 ceph-mon[32751]: pgmap v18569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:55 smithi090 ceph-mon[29180]: pgmap v18569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:58:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:56 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33817.507462 secs 2024-04-27T03:58:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:56 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33817.507462 secs 2024-04-27T03:58:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:56 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33817.507462 secs 2024-04-27T03:58:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:57 smithi119 ceph-mon[32752]: pgmap v18570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33818.378696 secs 2024-04-27T03:58:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:57 smithi167 ceph-mon[32751]: pgmap v18570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33818.378696 secs 2024-04-27T03:58:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:57 smithi090 ceph-mon[29180]: pgmap v18570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:58:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33818.378696 secs 2024-04-27T03:58:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:58:59 smithi119 ceph-mon[32752]: pgmap v18571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:58:59 smithi167 ceph-mon[32751]: pgmap v18571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:58:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:58:59 smithi090 ceph-mon[29180]: pgmap v18571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:01 smithi119 ceph-mon[32752]: pgmap v18572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:01 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33822.507542 secs 2024-04-27T03:59:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:01 smithi167 ceph-mon[32751]: pgmap v18572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:01 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33822.507542 secs 2024-04-27T03:59:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:01 smithi090 ceph-mon[29180]: pgmap v18572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:01 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33822.507542 secs 2024-04-27T03:59:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33823.378846 secs 2024-04-27T03:59:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33823.378846 secs 2024-04-27T03:59:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33823.378846 secs 2024-04-27T03:59:03.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:03 smithi119 ceph-mon[32752]: pgmap v18573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:03 smithi167 ceph-mon[32751]: pgmap v18573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:03 smithi090 ceph-mon[29180]: pgmap v18573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:05.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:05 smithi119 ceph-mon[32752]: pgmap v18574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:05.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:05 smithi167 ceph-mon[32751]: pgmap v18574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:05 smithi090 ceph-mon[29180]: pgmap v18574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:06 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33827.507623 secs 2024-04-27T03:59:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:06 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33827.507623 secs 2024-04-27T03:59:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:06 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33827.507623 secs 2024-04-27T03:59:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:07 smithi119 ceph-mon[32752]: pgmap v18575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:59:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33828.378945 secs 2024-04-27T03:59:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:07 smithi167 ceph-mon[32751]: pgmap v18575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:59:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33828.378945 secs 2024-04-27T03:59:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:07 smithi090 ceph-mon[29180]: pgmap v18575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T03:59:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33828.378945 secs 2024-04-27T03:59:09.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:09 smithi119 ceph-mon[32752]: pgmap v18576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:09.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:09 smithi167 ceph-mon[32751]: pgmap v18576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:09.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:09 smithi090 ceph-mon[29180]: pgmap v18576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:11 smithi090 ceph-mon[29180]: pgmap v18577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33832.507707 secs 2024-04-27T03:59:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:11 smithi167 ceph-mon[32751]: pgmap v18577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33832.507707 secs 2024-04-27T03:59:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:11 smithi119 ceph-mon[32752]: pgmap v18577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33832.507707 secs 2024-04-27T03:59:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33833.379092 secs 2024-04-27T03:59:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:59:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:59:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:59:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:59:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33833.379092 secs 2024-04-27T03:59:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:59:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:59:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:59:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:59:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33833.379092 secs 2024-04-27T03:59:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T03:59:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T03:59:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T03:59:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T03:59:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:13 smithi167 ceph-mon[32751]: pgmap v18578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:13 smithi167 ceph-mon[32751]: pgmap v18579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:59:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:13 smithi167 ceph-mon[32751]: pgmap v18580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:59:13.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:13 smithi090 ceph-mon[29180]: pgmap v18578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:13.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:13 smithi090 ceph-mon[29180]: pgmap v18579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:59:13.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:13 smithi090 ceph-mon[29180]: pgmap v18580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:59:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:13 smithi119 ceph-mon[32752]: pgmap v18578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:13 smithi119 ceph-mon[32752]: pgmap v18579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T03:59:13.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:13 smithi119 ceph-mon[32752]: pgmap v18580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:59:15.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:15 smithi167 ceph-mon[32751]: pgmap v18581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:15 smithi090 ceph-mon[29180]: pgmap v18581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:15 smithi119 ceph-mon[32752]: pgmap v18581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33837.507793 secs 2024-04-27T03:59:16.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33837.507793 secs 2024-04-27T03:59:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33837.507793 secs 2024-04-27T03:59:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:17 smithi090 ceph-mon[29180]: pgmap v18582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33838.379234 secs 2024-04-27T03:59:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:17 smithi119 ceph-mon[32752]: pgmap v18582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33838.379234 secs 2024-04-27T03:59:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:17 smithi167 ceph-mon[32751]: pgmap v18582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33838.379234 secs 2024-04-27T03:59:18.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:18 smithi090 ceph-mon[29180]: pgmap v18583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:59:18.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:18 smithi119 ceph-mon[32752]: pgmap v18583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:59:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:18 smithi167 ceph-mon[32751]: pgmap v18583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T03:59:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:20 smithi090 ceph-mon[29180]: pgmap v18584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:20 smithi119 ceph-mon[32752]: pgmap v18584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:20.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:20 smithi167 ceph-mon[32751]: pgmap v18584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T03:59:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33842.507872 secs 2024-04-27T03:59:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33842.507872 secs 2024-04-27T03:59:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33842.507872 secs 2024-04-27T03:59:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33843.379410 secs 2024-04-27T03:59:22.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:22 smithi090 ceph-mon[29180]: pgmap v18585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:59:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33843.379410 secs 2024-04-27T03:59:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:22 smithi119 ceph-mon[32752]: pgmap v18585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:59:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33843.379410 secs 2024-04-27T03:59:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:22 smithi167 ceph-mon[32751]: pgmap v18585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T03:59:24.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:24 smithi090 ceph-mon[29180]: pgmap v18586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:24.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:24 smithi119 ceph-mon[32752]: pgmap v18586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:24.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:24 smithi167 ceph-mon[32751]: pgmap v18586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33847.507928 secs 2024-04-27T03:59:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33847.507928 secs 2024-04-27T03:59:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33847.507928 secs 2024-04-27T03:59:27.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:26 smithi090 ceph-mon[29180]: pgmap v18587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:27.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33848.379454 secs 2024-04-27T03:59:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:26 smithi119 ceph-mon[32752]: pgmap v18587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33848.379454 secs 2024-04-27T03:59:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:26 smithi167 ceph-mon[32751]: pgmap v18587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33848.379454 secs 2024-04-27T03:59:28.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:28 smithi090 ceph-mon[29180]: pgmap v18588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:28.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:28 smithi119 ceph-mon[32752]: pgmap v18588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:28.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:28 smithi167 ceph-mon[32751]: pgmap v18588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:30.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:30 smithi090 ceph-mon[29180]: pgmap v18589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:30.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:30 smithi119 ceph-mon[32752]: pgmap v18589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:30.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:30 smithi167 ceph-mon[32751]: pgmap v18589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:31 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33852.507984 secs 2024-04-27T03:59:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:31 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33852.507984 secs 2024-04-27T03:59:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:31 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33852.507984 secs 2024-04-27T03:59:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33853.379524 secs 2024-04-27T03:59:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:32 smithi119 ceph-mon[32752]: pgmap v18590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33853.379524 secs 2024-04-27T03:59:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:32 smithi167 ceph-mon[32751]: pgmap v18590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33853.379524 secs 2024-04-27T03:59:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:32 smithi090 ceph-mon[29180]: pgmap v18590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:34.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:34 smithi090 ceph-mon[29180]: pgmap v18591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:34 smithi119 ceph-mon[32752]: pgmap v18591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:34.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:34 smithi167 ceph-mon[32751]: pgmap v18591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33857.508058 secs 2024-04-27T03:59:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33857.508058 secs 2024-04-27T03:59:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33857.508058 secs 2024-04-27T03:59:37.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:36 smithi090 ceph-mon[29180]: pgmap v18592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:37.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33858.379610 secs 2024-04-27T03:59:37.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:36 smithi119 ceph-mon[32752]: pgmap v18592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33858.379610 secs 2024-04-27T03:59:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:36 smithi167 ceph-mon[32751]: pgmap v18592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33858.379610 secs 2024-04-27T03:59:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:38 smithi090 ceph-mon[29180]: pgmap v18593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:38.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:38 smithi119 ceph-mon[32752]: pgmap v18593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:38.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:38 smithi167 ceph-mon[32751]: pgmap v18593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:40 smithi090 ceph-mon[29180]: pgmap v18594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:40.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:40 smithi119 ceph-mon[32752]: pgmap v18594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:40 smithi167 ceph-mon[32751]: pgmap v18594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:41 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33862.508145 secs 2024-04-27T03:59:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:41 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33862.508145 secs 2024-04-27T03:59:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:41 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33862.508145 secs 2024-04-27T03:59:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33863.379697 secs 2024-04-27T03:59:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:42 smithi119 ceph-mon[32752]: pgmap v18595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:42.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33863.379697 secs 2024-04-27T03:59:42.566 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:42 smithi090 ceph-mon[29180]: pgmap v18595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33863.379697 secs 2024-04-27T03:59:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:42 smithi167 ceph-mon[32751]: pgmap v18595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:44.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:44 smithi090 ceph-mon[29180]: pgmap v18596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:44 smithi119 ceph-mon[32752]: pgmap v18596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:44 smithi167 ceph-mon[32751]: pgmap v18596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33867.508202 secs 2024-04-27T03:59:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33867.508202 secs 2024-04-27T03:59:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33867.508202 secs 2024-04-27T03:59:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:46 smithi119 ceph-mon[32752]: pgmap v18597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33868.379763 secs 2024-04-27T03:59:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:46 smithi167 ceph-mon[32751]: pgmap v18597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33868.379763 secs 2024-04-27T03:59:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:46 smithi090 ceph-mon[29180]: pgmap v18597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33868.379763 secs 2024-04-27T03:59:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:48 smithi090 ceph-mon[29180]: pgmap v18598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:48.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:48 smithi119 ceph-mon[32752]: pgmap v18598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:48.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:48 smithi167 ceph-mon[32751]: pgmap v18598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:50 smithi090 ceph-mon[29180]: pgmap v18599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:50.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:50 smithi119 ceph-mon[32752]: pgmap v18599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:50.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:50 smithi167 ceph-mon[32751]: pgmap v18599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:51 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33872.508264 secs 2024-04-27T03:59:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:51 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33872.508264 secs 2024-04-27T03:59:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:51 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33872.508264 secs 2024-04-27T03:59:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33873.379830 secs 2024-04-27T03:59:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:52 smithi119 ceph-mon[32752]: pgmap v18600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33873.379830 secs 2024-04-27T03:59:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:52 smithi167 ceph-mon[32751]: pgmap v18600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33873.379830 secs 2024-04-27T03:59:52.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:52 smithi090 ceph-mon[29180]: pgmap v18600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:54 smithi090 ceph-mon[29180]: pgmap v18601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:54 smithi119 ceph-mon[32752]: pgmap v18601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:54 smithi167 ceph-mon[32751]: pgmap v18601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T03:59:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33877.508343 secs 2024-04-27T03:59:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33877.508343 secs 2024-04-27T03:59:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33877.508343 secs 2024-04-27T03:59:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:56 smithi119 ceph-mon[32752]: pgmap v18602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33878.379906 secs 2024-04-27T03:59:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:56 smithi167 ceph-mon[32751]: pgmap v18602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33878.379906 secs 2024-04-27T03:59:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:56 smithi090 ceph-mon[29180]: pgmap v18602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33878.379906 secs 2024-04-27T03:59:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 03:59:58 smithi090 ceph-mon[29180]: pgmap v18603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 03:59:58 smithi119 ceph-mon[32752]: pgmap v18603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T03:59:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 03:59:58 smithi167 ceph-mon[32751]: pgmap v18603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:00:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:00:00.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:00:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:01 smithi119 ceph-mon[32752]: pgmap v18604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:01 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33882.508429 secs 2024-04-27T04:00:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:01 smithi167 ceph-mon[32751]: pgmap v18604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:01 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33882.508429 secs 2024-04-27T04:00:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:01 smithi090 ceph-mon[29180]: pgmap v18604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:01 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33882.508429 secs 2024-04-27T04:00:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33883.380020 secs 2024-04-27T04:00:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33883.380020 secs 2024-04-27T04:00:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33883.380020 secs 2024-04-27T04:00:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:03 smithi167 ceph-mon[32751]: pgmap v18605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:03 smithi090 ceph-mon[29180]: pgmap v18605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:03 smithi119 ceph-mon[32752]: pgmap v18605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:05 smithi167 ceph-mon[32751]: pgmap v18606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:05 smithi090 ceph-mon[29180]: pgmap v18606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:05 smithi119 ceph-mon[32752]: pgmap v18606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:06 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33887.508507 secs 2024-04-27T04:00:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:06 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33887.508507 secs 2024-04-27T04:00:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:06 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33887.508507 secs 2024-04-27T04:00:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:07 smithi090 ceph-mon[29180]: pgmap v18607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33888.380085 secs 2024-04-27T04:00:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:07 smithi119 ceph-mon[32752]: pgmap v18607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33888.380085 secs 2024-04-27T04:00:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:07 smithi167 ceph-mon[32751]: pgmap v18607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33888.380085 secs 2024-04-27T04:00:09.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:09 smithi090 ceph-mon[29180]: pgmap v18608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:09 smithi119 ceph-mon[32752]: pgmap v18608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:09 smithi167 ceph-mon[32751]: pgmap v18608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:10.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:10 smithi090 ceph-mon[29180]: pgmap v18609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:10 smithi119 ceph-mon[32752]: pgmap v18609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:10 smithi167 ceph-mon[32751]: pgmap v18609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:11 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 33892.508588 secs 2024-04-27T04:00:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:11 smithi090 ceph-mon[29180]: slow request 720.295039 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:00:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:11 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 33892.508588 secs 2024-04-27T04:00:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:11 smithi119 ceph-mon[32752]: slow request 720.295039 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:00:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:11 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 33892.508588 secs 2024-04-27T04:00:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:11 smithi167 ceph-mon[32751]: slow request 720.295039 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:00:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33893.380197 secs 2024-04-27T04:00:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:12 smithi090 ceph-mon[29180]: pgmap v18610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33893.380197 secs 2024-04-27T04:00:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:12 smithi119 ceph-mon[32752]: pgmap v18610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33893.380197 secs 2024-04-27T04:00:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:12 smithi167 ceph-mon[32751]: pgmap v18610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:00:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:00:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:00:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:13 smithi090 ceph-mon[29180]: pgmap v18611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:00:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:13 smithi090 ceph-mon[29180]: pgmap v18612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:00:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:00:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:00:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:13 smithi119 ceph-mon[32752]: pgmap v18611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:13 smithi119 ceph-mon[32752]: pgmap v18612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:00:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:00:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:00:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:13 smithi167 ceph-mon[32751]: pgmap v18611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:00:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:13 smithi167 ceph-mon[32751]: pgmap v18612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:00:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:00:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:15 smithi090 ceph-mon[29180]: pgmap v18613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:15 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33897.508673 secs 2024-04-27T04:00:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:15 smithi119 ceph-mon[32752]: pgmap v18613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:15 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33897.508673 secs 2024-04-27T04:00:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:15 smithi167 ceph-mon[32751]: pgmap v18613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:15 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33897.508673 secs 2024-04-27T04:00:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33898.380313 secs 2024-04-27T04:00:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33898.380313 secs 2024-04-27T04:00:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33898.380313 secs 2024-04-27T04:00:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:17 smithi090 ceph-mon[29180]: pgmap v18614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:17 smithi119 ceph-mon[32752]: pgmap v18614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:17 smithi167 ceph-mon[32751]: pgmap v18614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:19 smithi090 ceph-mon[29180]: pgmap v18615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:19 smithi119 ceph-mon[32752]: pgmap v18615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:19 smithi167 ceph-mon[32751]: pgmap v18615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33902.508752 secs 2024-04-27T04:00:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33902.508752 secs 2024-04-27T04:00:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33902.508752 secs 2024-04-27T04:00:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:21 smithi090 ceph-mon[29180]: pgmap v18616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33903.380462 secs 2024-04-27T04:00:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:21 smithi119 ceph-mon[32752]: pgmap v18616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33903.380462 secs 2024-04-27T04:00:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:21 smithi167 ceph-mon[32751]: pgmap v18616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:00:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33903.380462 secs 2024-04-27T04:00:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:23 smithi090 ceph-mon[29180]: pgmap v18617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:00:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:23 smithi119 ceph-mon[32752]: pgmap v18617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:00:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:23 smithi167 ceph-mon[32751]: pgmap v18617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:00:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:25 smithi090 ceph-mon[29180]: pgmap v18618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33907.508847 secs 2024-04-27T04:00:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:25 smithi119 ceph-mon[32752]: pgmap v18618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33907.508847 secs 2024-04-27T04:00:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:25 smithi167 ceph-mon[32751]: pgmap v18618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33907.508847 secs 2024-04-27T04:00:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33908.380532 secs 2024-04-27T04:00:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33908.380532 secs 2024-04-27T04:00:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33908.380532 secs 2024-04-27T04:00:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:27 smithi119 ceph-mon[32752]: pgmap v18619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:27 smithi167 ceph-mon[32751]: pgmap v18619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:27 smithi090 ceph-mon[29180]: pgmap v18619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:29 smithi119 ceph-mon[32752]: pgmap v18620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:29 smithi167 ceph-mon[32751]: pgmap v18620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:30.330 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:29 smithi090 ceph-mon[29180]: pgmap v18620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33912.508906 secs 2024-04-27T04:00:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33912.508906 secs 2024-04-27T04:00:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33912.508906 secs 2024-04-27T04:00:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:31 smithi119 ceph-mon[32752]: pgmap v18621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33913.380609 secs 2024-04-27T04:00:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:31 smithi167 ceph-mon[32751]: pgmap v18621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33913.380609 secs 2024-04-27T04:00:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:31 smithi090 ceph-mon[29180]: pgmap v18621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33913.380609 secs 2024-04-27T04:00:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:33 smithi119 ceph-mon[32752]: pgmap v18622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:33 smithi167 ceph-mon[32751]: pgmap v18622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:33 smithi090 ceph-mon[29180]: pgmap v18622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:35 smithi119 ceph-mon[32752]: pgmap v18623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33917.508981 secs 2024-04-27T04:00:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:35 smithi167 ceph-mon[32751]: pgmap v18623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33917.508981 secs 2024-04-27T04:00:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:35 smithi090 ceph-mon[29180]: pgmap v18623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33917.508981 secs 2024-04-27T04:00:37.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33918.380755 secs 2024-04-27T04:00:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33918.380755 secs 2024-04-27T04:00:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33918.380755 secs 2024-04-27T04:00:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:37 smithi119 ceph-mon[32752]: pgmap v18624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:37 smithi167 ceph-mon[32751]: pgmap v18624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:37 smithi090 ceph-mon[29180]: pgmap v18624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:39 smithi090 ceph-mon[29180]: pgmap v18625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:39 smithi119 ceph-mon[32752]: pgmap v18625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:39 smithi167 ceph-mon[32751]: pgmap v18625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33922.509085 secs 2024-04-27T04:00:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33922.509085 secs 2024-04-27T04:00:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33922.509085 secs 2024-04-27T04:00:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:41 smithi119 ceph-mon[32752]: pgmap v18626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33923.380864 secs 2024-04-27T04:00:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:41 smithi167 ceph-mon[32751]: pgmap v18626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33923.380864 secs 2024-04-27T04:00:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:41 smithi090 ceph-mon[29180]: pgmap v18626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33923.380864 secs 2024-04-27T04:00:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:43 smithi119 ceph-mon[32752]: pgmap v18627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:43 smithi167 ceph-mon[32751]: pgmap v18627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:43 smithi090 ceph-mon[29180]: pgmap v18627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:45 smithi119 ceph-mon[32752]: pgmap v18628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33927.509137 secs 2024-04-27T04:00:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:45 smithi167 ceph-mon[32751]: pgmap v18628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33927.509137 secs 2024-04-27T04:00:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:45 smithi090 ceph-mon[29180]: pgmap v18628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33927.509137 secs 2024-04-27T04:00:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33928.380998 secs 2024-04-27T04:00:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33928.380998 secs 2024-04-27T04:00:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33928.380998 secs 2024-04-27T04:00:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:47 smithi119 ceph-mon[32752]: pgmap v18629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:47 smithi167 ceph-mon[32751]: pgmap v18629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:47 smithi090 ceph-mon[29180]: pgmap v18629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:49 smithi119 ceph-mon[32752]: pgmap v18630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:49 smithi167 ceph-mon[32751]: pgmap v18630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:49 smithi090 ceph-mon[29180]: pgmap v18630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33932.509198 secs 2024-04-27T04:00:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33932.509198 secs 2024-04-27T04:00:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33932.509198 secs 2024-04-27T04:00:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:51 smithi119 ceph-mon[32752]: pgmap v18631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33933.381124 secs 2024-04-27T04:00:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:51 smithi167 ceph-mon[32751]: pgmap v18631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33933.381124 secs 2024-04-27T04:00:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:51 smithi090 ceph-mon[29180]: pgmap v18631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33933.381124 secs 2024-04-27T04:00:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:53 smithi119 ceph-mon[32752]: pgmap v18632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:54.327 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:53 smithi167 ceph-mon[32751]: pgmap v18632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:53 smithi090 ceph-mon[29180]: pgmap v18632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:55 smithi119 ceph-mon[32752]: pgmap v18633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33937.509278 secs 2024-04-27T04:00:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:55 smithi167 ceph-mon[32751]: pgmap v18633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33937.509278 secs 2024-04-27T04:00:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:55 smithi090 ceph-mon[29180]: pgmap v18633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:00:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33937.509278 secs 2024-04-27T04:00:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33938.381241 secs 2024-04-27T04:00:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33938.381241 secs 2024-04-27T04:00:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33938.381241 secs 2024-04-27T04:00:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:57 smithi119 ceph-mon[32752]: pgmap v18634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:57 smithi167 ceph-mon[32751]: pgmap v18634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:00:58.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:57 smithi090 ceph-mon[29180]: pgmap v18634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:00.179 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:00:59 smithi090 ceph-mon[29180]: pgmap v18635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:00:59 smithi119 ceph-mon[32752]: pgmap v18635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:00:59 smithi167 ceph-mon[32751]: pgmap v18635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33942.509359 secs 2024-04-27T04:01:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33942.509359 secs 2024-04-27T04:01:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33942.509359 secs 2024-04-27T04:01:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:01 smithi119 ceph-mon[32752]: pgmap v18636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33943.381362 secs 2024-04-27T04:01:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:01 smithi167 ceph-mon[32751]: pgmap v18636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33943.381362 secs 2024-04-27T04:01:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:01 smithi090 ceph-mon[29180]: pgmap v18636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33943.381362 secs 2024-04-27T04:01:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:03 smithi119 ceph-mon[32752]: pgmap v18637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:03 smithi167 ceph-mon[32751]: pgmap v18637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:04.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:03 smithi090 ceph-mon[29180]: pgmap v18637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:05 smithi119 ceph-mon[32752]: pgmap v18638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33947.509439 secs 2024-04-27T04:01:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:05 smithi167 ceph-mon[32751]: pgmap v18638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33947.509439 secs 2024-04-27T04:01:06.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:05 smithi090 ceph-mon[29180]: pgmap v18638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33947.509439 secs 2024-04-27T04:01:07.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33948.381477 secs 2024-04-27T04:01:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33948.381477 secs 2024-04-27T04:01:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33948.381477 secs 2024-04-27T04:01:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:07 smithi119 ceph-mon[32752]: pgmap v18639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:07 smithi167 ceph-mon[32751]: pgmap v18639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:08.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:07 smithi090 ceph-mon[29180]: pgmap v18639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:09 smithi119 ceph-mon[32752]: pgmap v18640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:09 smithi167 ceph-mon[32751]: pgmap v18640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:10.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:09 smithi090 ceph-mon[29180]: pgmap v18640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:10 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33952.509539 secs 2024-04-27T04:01:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:10 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33952.509539 secs 2024-04-27T04:01:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:10 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33952.509539 secs 2024-04-27T04:01:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:11 smithi119 ceph-mon[32752]: pgmap v18641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33953.381595 secs 2024-04-27T04:01:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:11 smithi167 ceph-mon[32751]: pgmap v18641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33953.381595 secs 2024-04-27T04:01:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:11 smithi090 ceph-mon[29180]: pgmap v18641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33953.381595 secs 2024-04-27T04:01:13.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:01:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:01:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:01:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:13 smithi119 ceph-mon[32752]: pgmap v18642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:01:14.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:01:14.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:01:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:13 smithi167 ceph-mon[32751]: pgmap v18642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:01:14.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:01:14.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:01:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:13 smithi090 ceph-mon[29180]: pgmap v18642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:14.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:01:14.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:01:14.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:01:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:14 smithi119 ceph-mon[32752]: pgmap v18643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:01:15.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:14 smithi119 ceph-mon[32752]: pgmap v18644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:14 smithi167 ceph-mon[32751]: pgmap v18643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:01:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:14 smithi167 ceph-mon[32751]: pgmap v18644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:14 smithi090 ceph-mon[29180]: pgmap v18643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:01:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:14 smithi090 ceph-mon[29180]: pgmap v18644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33957.509589 secs 2024-04-27T04:01:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33957.509589 secs 2024-04-27T04:01:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33957.509589 secs 2024-04-27T04:01:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:17 smithi119 ceph-mon[32752]: pgmap v18645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33958.381732 secs 2024-04-27T04:01:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:17 smithi167 ceph-mon[32751]: pgmap v18645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33958.381732 secs 2024-04-27T04:01:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:17 smithi090 ceph-mon[29180]: pgmap v18645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33958.381732 secs 2024-04-27T04:01:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:19 smithi119 ceph-mon[32752]: pgmap v18646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:01:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:19 smithi167 ceph-mon[32751]: pgmap v18646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:01:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:19 smithi090 ceph-mon[29180]: pgmap v18646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:01:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:21 smithi119 ceph-mon[32752]: pgmap v18647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33962.509653 secs 2024-04-27T04:01:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:21 smithi167 ceph-mon[32751]: pgmap v18647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33962.509653 secs 2024-04-27T04:01:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:21 smithi090 ceph-mon[29180]: pgmap v18647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33962.509653 secs 2024-04-27T04:01:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33963.381836 secs 2024-04-27T04:01:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33963.381836 secs 2024-04-27T04:01:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33963.381836 secs 2024-04-27T04:01:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:23 smithi119 ceph-mon[32752]: pgmap v18648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:23 smithi167 ceph-mon[32751]: pgmap v18648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:23 smithi090 ceph-mon[29180]: pgmap v18648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:01:25.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:25 smithi119 ceph-mon[32752]: pgmap v18649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:01:25.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:25 smithi167 ceph-mon[32751]: pgmap v18649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:01:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:25 smithi090 ceph-mon[29180]: pgmap v18649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:01:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:26 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33967.509705 secs 2024-04-27T04:01:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:26 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33967.509705 secs 2024-04-27T04:01:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:26 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33967.509705 secs 2024-04-27T04:01:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:27 smithi119 ceph-mon[32752]: pgmap v18650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33968.381922 secs 2024-04-27T04:01:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:27 smithi167 ceph-mon[32751]: pgmap v18650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33968.381922 secs 2024-04-27T04:01:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:27 smithi090 ceph-mon[29180]: pgmap v18650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33968.381922 secs 2024-04-27T04:01:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:29 smithi119 ceph-mon[32752]: pgmap v18651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:29 smithi167 ceph-mon[32751]: pgmap v18651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:29 smithi090 ceph-mon[29180]: pgmap v18651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:31 smithi119 ceph-mon[32752]: pgmap v18652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:31 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33972.509750 secs 2024-04-27T04:01:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:31 smithi167 ceph-mon[32751]: pgmap v18652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:31 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33972.509750 secs 2024-04-27T04:01:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:31 smithi090 ceph-mon[29180]: pgmap v18652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:31 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33972.509750 secs 2024-04-27T04:01:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33973.382033 secs 2024-04-27T04:01:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33973.382033 secs 2024-04-27T04:01:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33973.382033 secs 2024-04-27T04:01:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:33 smithi119 ceph-mon[32752]: pgmap v18653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:33 smithi167 ceph-mon[32751]: pgmap v18653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:33 smithi090 ceph-mon[29180]: pgmap v18653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:35 smithi119 ceph-mon[32752]: pgmap v18654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:35 smithi167 ceph-mon[32751]: pgmap v18654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:35 smithi090 ceph-mon[29180]: pgmap v18654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:36 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33977.509832 secs 2024-04-27T04:01:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:36 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33977.509832 secs 2024-04-27T04:01:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:36 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33977.509832 secs 2024-04-27T04:01:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:37 smithi167 ceph-mon[32751]: pgmap v18655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33978.382168 secs 2024-04-27T04:01:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:37 smithi090 ceph-mon[29180]: pgmap v18655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33978.382168 secs 2024-04-27T04:01:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:37 smithi119 ceph-mon[32752]: pgmap v18655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33978.382168 secs 2024-04-27T04:01:39.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:39 smithi167 ceph-mon[32751]: pgmap v18656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:39 smithi090 ceph-mon[29180]: pgmap v18656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:39 smithi119 ceph-mon[32752]: pgmap v18656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:41 smithi167 ceph-mon[32751]: pgmap v18657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:41 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33982.509872 secs 2024-04-27T04:01:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:41 smithi090 ceph-mon[29180]: pgmap v18657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:41 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33982.509872 secs 2024-04-27T04:01:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:41 smithi119 ceph-mon[32752]: pgmap v18657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:41 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33982.509872 secs 2024-04-27T04:01:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33983.382280 secs 2024-04-27T04:01:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33983.382280 secs 2024-04-27T04:01:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33983.382280 secs 2024-04-27T04:01:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:43 smithi090 ceph-mon[29180]: pgmap v18658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:43 smithi119 ceph-mon[32752]: pgmap v18658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:43 smithi167 ceph-mon[32751]: pgmap v18658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:45 smithi090 ceph-mon[29180]: pgmap v18659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:45 smithi119 ceph-mon[32752]: pgmap v18659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:45 smithi167 ceph-mon[32751]: pgmap v18659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:46 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33987.509924 secs 2024-04-27T04:01:46.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:46 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33987.509924 secs 2024-04-27T04:01:46.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:46 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33987.509924 secs 2024-04-27T04:01:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:47 smithi090 ceph-mon[29180]: pgmap v18660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33988.382430 secs 2024-04-27T04:01:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:47 smithi119 ceph-mon[32752]: pgmap v18660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33988.382430 secs 2024-04-27T04:01:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:47 smithi167 ceph-mon[32751]: pgmap v18660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33988.382430 secs 2024-04-27T04:01:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:49 smithi090 ceph-mon[29180]: pgmap v18661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:49 smithi119 ceph-mon[32752]: pgmap v18661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:49 smithi167 ceph-mon[32751]: pgmap v18661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:51 smithi090 ceph-mon[29180]: pgmap v18662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:51 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33992.509986 secs 2024-04-27T04:01:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:51 smithi119 ceph-mon[32752]: pgmap v18662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:51 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33992.509986 secs 2024-04-27T04:01:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:51 smithi167 ceph-mon[32751]: pgmap v18662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:51 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33992.509986 secs 2024-04-27T04:01:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33993.382551 secs 2024-04-27T04:01:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33993.382551 secs 2024-04-27T04:01:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33993.382551 secs 2024-04-27T04:01:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:53 smithi090 ceph-mon[29180]: pgmap v18663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:53 smithi119 ceph-mon[32752]: pgmap v18663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:53 smithi167 ceph-mon[32751]: pgmap v18663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:54 smithi090 ceph-mon[29180]: pgmap v18664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:54 smithi119 ceph-mon[32752]: pgmap v18664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:54 smithi167 ceph-mon[32751]: pgmap v18664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:01:56.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 33997.510046 secs 2024-04-27T04:01:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 33997.510046 secs 2024-04-27T04:01:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 33997.510046 secs 2024-04-27T04:01:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:56 smithi119 ceph-mon[32752]: pgmap v18665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 33998.382629 secs 2024-04-27T04:01:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:56 smithi167 ceph-mon[32751]: pgmap v18665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 33998.382629 secs 2024-04-27T04:01:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:56 smithi090 ceph-mon[29180]: pgmap v18665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 33998.382629 secs 2024-04-27T04:01:59.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:01:58 smithi119 ceph-mon[32752]: pgmap v18666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:59.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:01:58 smithi167 ceph-mon[32751]: pgmap v18666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:01:59.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:01:58 smithi090 ceph-mon[29180]: pgmap v18666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:00 smithi119 ceph-mon[32752]: pgmap v18667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:00 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 34002.510108 secs 2024-04-27T04:02:01.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:00 smithi119 ceph-mon[32752]: slow request 723.987270 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:02:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:00 smithi167 ceph-mon[32751]: pgmap v18667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:00 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 34002.510108 secs 2024-04-27T04:02:01.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:00 smithi167 ceph-mon[32751]: slow request 723.987270 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:00 smithi090 ceph-mon[29180]: pgmap v18667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:00 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 34002.510108 secs 2024-04-27T04:02:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:00 smithi090 ceph-mon[29180]: slow request 723.987270 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:02:02.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34003.382691 secs 2024-04-27T04:02:02.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34003.382691 secs 2024-04-27T04:02:02.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34003.382691 secs 2024-04-27T04:02:03.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:02 smithi119 ceph-mon[32752]: pgmap v18668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:03.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:02 smithi167 ceph-mon[32751]: pgmap v18668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:03.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:02 smithi090 ceph-mon[29180]: pgmap v18668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:05.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:04 smithi119 ceph-mon[32752]: pgmap v18669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:05.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:04 smithi167 ceph-mon[32751]: pgmap v18669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:05.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:04 smithi090 ceph-mon[29180]: pgmap v18669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:06.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34007.510163 secs 2024-04-27T04:02:06.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34007.510163 secs 2024-04-27T04:02:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34007.510163 secs 2024-04-27T04:02:07.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:06 smithi090 ceph-mon[29180]: pgmap v18670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:07.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34008.382768 secs 2024-04-27T04:02:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:06 smithi119 ceph-mon[32752]: pgmap v18670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:07.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34008.382768 secs 2024-04-27T04:02:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:06 smithi167 ceph-mon[32751]: pgmap v18670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:07.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34008.382768 secs 2024-04-27T04:02:09.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:08 smithi119 ceph-mon[32752]: pgmap v18671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:09.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:08 smithi167 ceph-mon[32751]: pgmap v18671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:09.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:08 smithi090 ceph-mon[29180]: pgmap v18671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:10 smithi119 ceph-mon[32752]: pgmap v18672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:11.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:10 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34012.510219 secs 2024-04-27T04:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:10 smithi167 ceph-mon[32751]: pgmap v18672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:11.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:10 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34012.510219 secs 2024-04-27T04:02:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:10 smithi090 ceph-mon[29180]: pgmap v18672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:10 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34012.510219 secs 2024-04-27T04:02:12.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34013.382832 secs 2024-04-27T04:02:12.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34013.382832 secs 2024-04-27T04:02:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34013.382832 secs 2024-04-27T04:02:13.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:12 smithi119 ceph-mon[32752]: pgmap v18673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:13.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:12 smithi167 ceph-mon[32751]: pgmap v18673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:13.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:12 smithi090 ceph-mon[29180]: pgmap v18673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:14.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:02:14.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:02:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:02:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:14 smithi119 ceph-mon[32752]: pgmap v18674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:02:15.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:02:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:14 smithi167 ceph-mon[32751]: pgmap v18674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:02:15.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:02:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:14 smithi090 ceph-mon[29180]: pgmap v18674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:02:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:02:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:02:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:15 smithi119 ceph-mon[32752]: pgmap v18675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:02:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:15 smithi119 ceph-mon[32752]: pgmap v18676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:16.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:15 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34017.510281 secs 2024-04-27T04:02:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:15 smithi167 ceph-mon[32751]: pgmap v18675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:02:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:15 smithi167 ceph-mon[32751]: pgmap v18676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:16.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:15 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34017.510281 secs 2024-04-27T04:02:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:15 smithi090 ceph-mon[29180]: pgmap v18675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:02:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:15 smithi090 ceph-mon[29180]: pgmap v18676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:15 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34017.510281 secs 2024-04-27T04:02:17.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34018.382966 secs 2024-04-27T04:02:17.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34018.382966 secs 2024-04-27T04:02:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34018.382966 secs 2024-04-27T04:02:18.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:17 smithi119 ceph-mon[32752]: pgmap v18677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:18.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:17 smithi167 ceph-mon[32751]: pgmap v18677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:18.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:17 smithi090 ceph-mon[29180]: pgmap v18677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:19 smithi119 ceph-mon[32752]: pgmap v18678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:20.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:19 smithi167 ceph-mon[32751]: pgmap v18678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:19 smithi090 ceph-mon[29180]: pgmap v18678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:21.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34022.510369 secs 2024-04-27T04:02:21.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34022.510369 secs 2024-04-27T04:02:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34022.510369 secs 2024-04-27T04:02:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:21 smithi119 ceph-mon[32752]: pgmap v18679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34023.383102 secs 2024-04-27T04:02:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:21 smithi167 ceph-mon[32751]: pgmap v18679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34023.383102 secs 2024-04-27T04:02:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:21 smithi090 ceph-mon[29180]: pgmap v18679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:02:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34023.383102 secs 2024-04-27T04:02:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:23 smithi119 ceph-mon[32752]: pgmap v18680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:02:24.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:23 smithi167 ceph-mon[32751]: pgmap v18680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:02:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:23 smithi090 ceph-mon[29180]: pgmap v18680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:02:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:25 smithi119 ceph-mon[32752]: pgmap v18681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:02:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34027.510454 secs 2024-04-27T04:02:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:25 smithi167 ceph-mon[32751]: pgmap v18681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:02:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34027.510454 secs 2024-04-27T04:02:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:25 smithi090 ceph-mon[29180]: pgmap v18681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:02:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34027.510454 secs 2024-04-27T04:02:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34028.383236 secs 2024-04-27T04:02:27.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34028.383236 secs 2024-04-27T04:02:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34028.383236 secs 2024-04-27T04:02:28.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:27 smithi119 ceph-mon[32752]: pgmap v18682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:27 smithi167 ceph-mon[32751]: pgmap v18682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:27 smithi090 ceph-mon[29180]: pgmap v18682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:30.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:29 smithi119 ceph-mon[32752]: pgmap v18683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:29 smithi167 ceph-mon[32751]: pgmap v18683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:29 smithi090 ceph-mon[29180]: pgmap v18683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:31.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34032.510539 secs 2024-04-27T04:02:31.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34032.510539 secs 2024-04-27T04:02:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34032.510539 secs 2024-04-27T04:02:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:31 smithi119 ceph-mon[32752]: pgmap v18684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:32.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34033.383348 secs 2024-04-27T04:02:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:31 smithi167 ceph-mon[32751]: pgmap v18684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34033.383348 secs 2024-04-27T04:02:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:31 smithi090 ceph-mon[29180]: pgmap v18684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34033.383348 secs 2024-04-27T04:02:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:33 smithi090 ceph-mon[29180]: pgmap v18685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:34.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:33 smithi119 ceph-mon[32752]: pgmap v18685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:34.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:33 smithi167 ceph-mon[32751]: pgmap v18685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:35 smithi119 ceph-mon[32752]: pgmap v18686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:36.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34037.510618 secs 2024-04-27T04:02:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:35 smithi167 ceph-mon[32751]: pgmap v18686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:36.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34037.510618 secs 2024-04-27T04:02:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:35 smithi090 ceph-mon[29180]: pgmap v18686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34037.510618 secs 2024-04-27T04:02:37.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34038.383430 secs 2024-04-27T04:02:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34038.383430 secs 2024-04-27T04:02:37.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34038.383430 secs 2024-04-27T04:02:38.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:37 smithi119 ceph-mon[32752]: pgmap v18687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:38.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:37 smithi167 ceph-mon[32751]: pgmap v18687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:37 smithi090 ceph-mon[29180]: pgmap v18687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:40.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:39 smithi119 ceph-mon[32752]: pgmap v18688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:40.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:39 smithi167 ceph-mon[32751]: pgmap v18688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:40.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:39 smithi090 ceph-mon[29180]: pgmap v18688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:41.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34042.510702 secs 2024-04-27T04:02:41.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34042.510702 secs 2024-04-27T04:02:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34042.510702 secs 2024-04-27T04:02:42.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:41 smithi119 ceph-mon[32752]: pgmap v18689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34043.383516 secs 2024-04-27T04:02:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:41 smithi167 ceph-mon[32751]: pgmap v18689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:42.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34043.383516 secs 2024-04-27T04:02:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:41 smithi090 ceph-mon[29180]: pgmap v18689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34043.383516 secs 2024-04-27T04:02:44.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:43 smithi119 ceph-mon[32752]: pgmap v18690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:44.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:43 smithi167 ceph-mon[32751]: pgmap v18690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:44.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:43 smithi090 ceph-mon[29180]: pgmap v18690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:45 smithi119 ceph-mon[32752]: pgmap v18691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:46.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34047.510812 secs 2024-04-27T04:02:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:45 smithi167 ceph-mon[32751]: pgmap v18691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34047.510812 secs 2024-04-27T04:02:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:45 smithi090 ceph-mon[29180]: pgmap v18691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:46.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34047.510812 secs 2024-04-27T04:02:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34048.383599 secs 2024-04-27T04:02:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34048.383599 secs 2024-04-27T04:02:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34048.383599 secs 2024-04-27T04:02:48.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:47 smithi119 ceph-mon[32752]: pgmap v18692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:48.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:47 smithi167 ceph-mon[32751]: pgmap v18692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:47 smithi090 ceph-mon[29180]: pgmap v18692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:50.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:49 smithi119 ceph-mon[32752]: pgmap v18693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:50.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:49 smithi167 ceph-mon[32751]: pgmap v18693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:50.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:49 smithi090 ceph-mon[29180]: pgmap v18693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34052.510902 secs 2024-04-27T04:02:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34052.510902 secs 2024-04-27T04:02:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34052.510902 secs 2024-04-27T04:02:52.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:51 smithi167 ceph-mon[32751]: pgmap v18694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:52.113 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34053.383665 secs 2024-04-27T04:02:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:51 smithi090 ceph-mon[29180]: pgmap v18694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34053.383665 secs 2024-04-27T04:02:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:51 smithi119 ceph-mon[32752]: pgmap v18694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34053.383665 secs 2024-04-27T04:02:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:53 smithi167 ceph-mon[32751]: pgmap v18695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:53 smithi090 ceph-mon[29180]: pgmap v18695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:53 smithi119 ceph-mon[32752]: pgmap v18695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:55 smithi167 ceph-mon[32751]: pgmap v18696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34057.510963 secs 2024-04-27T04:02:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:55 smithi090 ceph-mon[29180]: pgmap v18696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34057.510963 secs 2024-04-27T04:02:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:55 smithi119 ceph-mon[32752]: pgmap v18696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:02:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34057.510963 secs 2024-04-27T04:02:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34058.383746 secs 2024-04-27T04:02:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34058.383746 secs 2024-04-27T04:02:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34058.383746 secs 2024-04-27T04:02:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:57 smithi090 ceph-mon[29180]: pgmap v18697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:57 smithi119 ceph-mon[32752]: pgmap v18697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:02:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:57 smithi167 ceph-mon[32751]: pgmap v18697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:02:59 smithi090 ceph-mon[29180]: pgmap v18698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:00.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:02:59 smithi119 ceph-mon[32752]: pgmap v18698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:02:59 smithi167 ceph-mon[32751]: pgmap v18698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:01.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34062.511052 secs 2024-04-27T04:03:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34062.511052 secs 2024-04-27T04:03:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34062.511052 secs 2024-04-27T04:03:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:01 smithi090 ceph-mon[29180]: pgmap v18699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34063.383880 secs 2024-04-27T04:03:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:01 smithi119 ceph-mon[32752]: pgmap v18699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34063.383880 secs 2024-04-27T04:03:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:01 smithi167 ceph-mon[32751]: pgmap v18699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34063.383880 secs 2024-04-27T04:03:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:03 smithi090 ceph-mon[29180]: pgmap v18700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:03 smithi119 ceph-mon[32752]: pgmap v18700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:04.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:03 smithi167 ceph-mon[32751]: pgmap v18700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:05 smithi090 ceph-mon[29180]: pgmap v18701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34067.511114 secs 2024-04-27T04:03:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:05 smithi119 ceph-mon[32752]: pgmap v18701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34067.511114 secs 2024-04-27T04:03:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:05 smithi167 ceph-mon[32751]: pgmap v18701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34067.511114 secs 2024-04-27T04:03:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34068.384016 secs 2024-04-27T04:03:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34068.384016 secs 2024-04-27T04:03:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34068.384016 secs 2024-04-27T04:03:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:07 smithi090 ceph-mon[29180]: pgmap v18702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:07 smithi119 ceph-mon[32752]: pgmap v18702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:08.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:07 smithi167 ceph-mon[32751]: pgmap v18702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:09 smithi090 ceph-mon[29180]: pgmap v18703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:09 smithi119 ceph-mon[32752]: pgmap v18703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:09 smithi167 ceph-mon[32751]: pgmap v18703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:10 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34072.511199 secs 2024-04-27T04:03:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:10 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34072.511199 secs 2024-04-27T04:03:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:10 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34072.511199 secs 2024-04-27T04:03:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:11 smithi090 ceph-mon[29180]: pgmap v18704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34073.384182 secs 2024-04-27T04:03:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:11 smithi119 ceph-mon[32752]: pgmap v18704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34073.384182 secs 2024-04-27T04:03:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:11 smithi167 ceph-mon[32751]: pgmap v18704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34073.384182 secs 2024-04-27T04:03:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:13 smithi090 ceph-mon[29180]: pgmap v18705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:13 smithi119 ceph-mon[32752]: pgmap v18705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:13 smithi167 ceph-mon[32751]: pgmap v18705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:15.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:03:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:15 smithi090 ceph-mon[29180]: pgmap v18706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:03:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:03:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:15 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34077.511284 secs 2024-04-27T04:03:16.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:15 smithi119 ceph-mon[32752]: pgmap v18706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:16.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:16.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:03:16.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:03:16.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:15 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34077.511284 secs 2024-04-27T04:03:16.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:15 smithi167 ceph-mon[32751]: pgmap v18706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:16.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:03:16.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:03:16.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:03:16.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:15 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34077.511284 secs 2024-04-27T04:03:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:16 smithi119 ceph-mon[32752]: pgmap v18707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:03:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:16 smithi119 ceph-mon[32752]: pgmap v18708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34078.384302 secs 2024-04-27T04:03:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:16 smithi167 ceph-mon[32751]: pgmap v18707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:03:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:16 smithi167 ceph-mon[32751]: pgmap v18708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34078.384302 secs 2024-04-27T04:03:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:16 smithi090 ceph-mon[29180]: pgmap v18707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:03:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:16 smithi090 ceph-mon[29180]: pgmap v18708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34078.384302 secs 2024-04-27T04:03:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:18 smithi119 ceph-mon[32752]: pgmap v18709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:03:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:18 smithi167 ceph-mon[32751]: pgmap v18709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:03:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:18 smithi090 ceph-mon[29180]: pgmap v18709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:03:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:20 smithi119 ceph-mon[32752]: pgmap v18710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34082.511372 secs 2024-04-27T04:03:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:20 smithi167 ceph-mon[32751]: pgmap v18710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34082.511372 secs 2024-04-27T04:03:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:20 smithi090 ceph-mon[29180]: pgmap v18710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34082.511372 secs 2024-04-27T04:03:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34083.384419 secs 2024-04-27T04:03:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34083.384419 secs 2024-04-27T04:03:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34083.384419 secs 2024-04-27T04:03:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:22 smithi119 ceph-mon[32752]: pgmap v18711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:22 smithi167 ceph-mon[32751]: pgmap v18711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:22 smithi090 ceph-mon[29180]: pgmap v18711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:03:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:24 smithi119 ceph-mon[32752]: pgmap v18712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:03:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:24 smithi167 ceph-mon[32751]: pgmap v18712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:03:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:24 smithi090 ceph-mon[29180]: pgmap v18712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:03:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34087.511457 secs 2024-04-27T04:03:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34087.511457 secs 2024-04-27T04:03:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34087.511457 secs 2024-04-27T04:03:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:26 smithi119 ceph-mon[32752]: pgmap v18713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:03:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34088.384483 secs 2024-04-27T04:03:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:26 smithi167 ceph-mon[32751]: pgmap v18713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:03:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34088.384483 secs 2024-04-27T04:03:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:26 smithi090 ceph-mon[29180]: pgmap v18713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:03:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34088.384483 secs 2024-04-27T04:03:29.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:28 smithi119 ceph-mon[32752]: pgmap v18714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:28 smithi167 ceph-mon[32751]: pgmap v18714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:28 smithi090 ceph-mon[29180]: pgmap v18714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:30 smithi119 ceph-mon[32752]: pgmap v18715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34092.511560 secs 2024-04-27T04:03:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:30 smithi167 ceph-mon[32751]: pgmap v18715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34092.511560 secs 2024-04-27T04:03:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:30 smithi090 ceph-mon[29180]: pgmap v18715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34092.511560 secs 2024-04-27T04:03:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34093.384579 secs 2024-04-27T04:03:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34093.384579 secs 2024-04-27T04:03:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34093.384579 secs 2024-04-27T04:03:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:32 smithi119 ceph-mon[32752]: pgmap v18716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:33.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:32 smithi167 ceph-mon[32751]: pgmap v18716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:32 smithi090 ceph-mon[29180]: pgmap v18716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:35.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:34 smithi119 ceph-mon[32752]: pgmap v18717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:35.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:34 smithi167 ceph-mon[32751]: pgmap v18717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:34 smithi090 ceph-mon[29180]: pgmap v18717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34097.511646 secs 2024-04-27T04:03:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34097.511646 secs 2024-04-27T04:03:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34097.511646 secs 2024-04-27T04:03:37.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:36 smithi090 ceph-mon[29180]: pgmap v18718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:37.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34098.384649 secs 2024-04-27T04:03:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:36 smithi119 ceph-mon[32752]: pgmap v18718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34098.384649 secs 2024-04-27T04:03:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:36 smithi167 ceph-mon[32751]: pgmap v18718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34098.384649 secs 2024-04-27T04:03:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:38 smithi119 ceph-mon[32752]: pgmap v18719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:38 smithi167 ceph-mon[32751]: pgmap v18719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:38 smithi090 ceph-mon[29180]: pgmap v18719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:40 smithi119 ceph-mon[32752]: pgmap v18720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34102.511731 secs 2024-04-27T04:03:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:40 smithi167 ceph-mon[32751]: pgmap v18720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34102.511731 secs 2024-04-27T04:03:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:40 smithi090 ceph-mon[29180]: pgmap v18720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34102.511731 secs 2024-04-27T04:03:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34103.384715 secs 2024-04-27T04:03:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34103.384715 secs 2024-04-27T04:03:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34103.384715 secs 2024-04-27T04:03:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:42 smithi119 ceph-mon[32752]: pgmap v18721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:42 smithi167 ceph-mon[32751]: pgmap v18721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:42 smithi090 ceph-mon[29180]: pgmap v18721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:44 smithi119 ceph-mon[32752]: pgmap v18722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:44 smithi167 ceph-mon[32751]: pgmap v18722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:45.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:44 smithi090 ceph-mon[29180]: pgmap v18722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:46.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34107.511834 secs 2024-04-27T04:03:46.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34107.511834 secs 2024-04-27T04:03:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34107.511834 secs 2024-04-27T04:03:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:46 smithi119 ceph-mon[32752]: pgmap v18723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34108.384840 secs 2024-04-27T04:03:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:46 smithi167 ceph-mon[32751]: pgmap v18723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34108.384840 secs 2024-04-27T04:03:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:46 smithi090 ceph-mon[29180]: pgmap v18723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34108.384840 secs 2024-04-27T04:03:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:48 smithi119 ceph-mon[32752]: pgmap v18724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:49.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:48 smithi167 ceph-mon[32751]: pgmap v18724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:49.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:48 smithi090 ceph-mon[29180]: pgmap v18724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:50 smithi119 ceph-mon[32752]: pgmap v18725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34112.511922 secs 2024-04-27T04:03:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:50 smithi167 ceph-mon[32751]: pgmap v18725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34112.511922 secs 2024-04-27T04:03:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:50 smithi090 ceph-mon[29180]: pgmap v18725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34112.511922 secs 2024-04-27T04:03:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34113.384976 secs 2024-04-27T04:03:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34113.384976 secs 2024-04-27T04:03:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34113.384976 secs 2024-04-27T04:03:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:52 smithi119 ceph-mon[32752]: pgmap v18726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:52 smithi167 ceph-mon[32751]: pgmap v18726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:52 smithi090 ceph-mon[29180]: pgmap v18726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:54 smithi119 ceph-mon[32752]: pgmap v18727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:54 smithi167 ceph-mon[32751]: pgmap v18727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:54 smithi090 ceph-mon[29180]: pgmap v18727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34117.512004 secs 2024-04-27T04:03:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34117.512004 secs 2024-04-27T04:03:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34117.512004 secs 2024-04-27T04:03:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:56 smithi119 ceph-mon[32752]: pgmap v18728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34118.385101 secs 2024-04-27T04:03:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:56 smithi167 ceph-mon[32751]: pgmap v18728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34118.385101 secs 2024-04-27T04:03:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:56 smithi090 ceph-mon[29180]: pgmap v18728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:03:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34118.385101 secs 2024-04-27T04:03:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:03:58 smithi119 ceph-mon[32752]: pgmap v18729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:03:58 smithi167 ceph-mon[32751]: pgmap v18729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:03:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:03:58 smithi090 ceph-mon[29180]: pgmap v18729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:00 smithi119 ceph-mon[32752]: pgmap v18730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34122.512088 secs 2024-04-27T04:04:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:00 smithi167 ceph-mon[32751]: pgmap v18730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34122.512088 secs 2024-04-27T04:04:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:00 smithi090 ceph-mon[29180]: pgmap v18730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34122.512088 secs 2024-04-27T04:04:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34123.385247 secs 2024-04-27T04:04:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34123.385247 secs 2024-04-27T04:04:02.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34123.385247 secs 2024-04-27T04:04:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:02 smithi119 ceph-mon[32752]: pgmap v18731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:02 smithi167 ceph-mon[32751]: pgmap v18731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:02 smithi090 ceph-mon[29180]: pgmap v18731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:04 smithi119 ceph-mon[32752]: pgmap v18732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:04 smithi167 ceph-mon[32751]: pgmap v18732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:04 smithi090 ceph-mon[29180]: pgmap v18732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34127.512172 secs 2024-04-27T04:04:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34127.512172 secs 2024-04-27T04:04:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34127.512172 secs 2024-04-27T04:04:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:06 smithi090 ceph-mon[29180]: pgmap v18733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34128.385352 secs 2024-04-27T04:04:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:06 smithi119 ceph-mon[32752]: pgmap v18733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34128.385352 secs 2024-04-27T04:04:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:06 smithi167 ceph-mon[32751]: pgmap v18733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34128.385352 secs 2024-04-27T04:04:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:09 smithi119 ceph-mon[32752]: pgmap v18734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:09 smithi167 ceph-mon[32751]: pgmap v18734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:09.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:09 smithi090 ceph-mon[29180]: pgmap v18734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:10.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:04:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:04:10.249+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:04:10.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:04:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:04:10.249+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:04:10.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:04:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:04:10.249+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:04:10.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:04:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:04:10.249+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:04:10.627 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:04:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:04:10.249+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:04:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:11 smithi119 ceph-mon[32752]: pgmap v18735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:11 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 34132.512257 secs 2024-04-27T04:04:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:11 smithi119 ceph-mon[32752]: slow request 2880.844666 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:04:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:11 smithi167 ceph-mon[32751]: pgmap v18735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:11 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 34132.512257 secs 2024-04-27T04:04:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:11 smithi167 ceph-mon[32751]: slow request 2880.844666 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:04:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:11 smithi090 ceph-mon[29180]: pgmap v18735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:11 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 34132.512257 secs 2024-04-27T04:04:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:11 smithi090 ceph-mon[29180]: slow request 2880.844666 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:04:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34133.385490 secs 2024-04-27T04:04:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34133.385490 secs 2024-04-27T04:04:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34133.385490 secs 2024-04-27T04:04:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:13 smithi119 ceph-mon[32752]: pgmap v18736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:13 smithi167 ceph-mon[32751]: pgmap v18736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:13 smithi090 ceph-mon[29180]: pgmap v18736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:15 smithi119 ceph-mon[32752]: pgmap v18737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:15 smithi167 ceph-mon[32751]: pgmap v18737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:15 smithi090 ceph-mon[29180]: pgmap v18737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34137.512319 secs 2024-04-27T04:04:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:04:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34137.512319 secs 2024-04-27T04:04:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:04:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:04:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34137.512319 secs 2024-04-27T04:04:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:04:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:04:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:04:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:17 smithi119 ceph-mon[32752]: pgmap v18738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T04:04:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:17 smithi119 ceph-mon[32752]: pgmap v18739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T04:04:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:17 smithi119 ceph-mon[32752]: pgmap v18740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34138.385607 secs 2024-04-27T04:04:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:17 smithi167 ceph-mon[32751]: pgmap v18738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T04:04:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:17 smithi167 ceph-mon[32751]: pgmap v18739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T04:04:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:17 smithi167 ceph-mon[32751]: pgmap v18740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34138.385607 secs 2024-04-27T04:04:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:17 smithi090 ceph-mon[29180]: pgmap v18738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T04:04:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:17 smithi090 ceph-mon[29180]: pgmap v18739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T04:04:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:17 smithi090 ceph-mon[29180]: pgmap v18740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34138.385607 secs 2024-04-27T04:04:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:19 smithi119 ceph-mon[32752]: pgmap v18741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-27T04:04:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:19 smithi167 ceph-mon[32751]: pgmap v18741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-27T04:04:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:19 smithi090 ceph-mon[29180]: pgmap v18741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-27T04:04:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:21 smithi119 ceph-mon[32752]: pgmap v18742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34142.512403 secs 2024-04-27T04:04:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:21 smithi167 ceph-mon[32751]: pgmap v18742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34142.512403 secs 2024-04-27T04:04:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:21 smithi090 ceph-mon[29180]: pgmap v18742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34142.512403 secs 2024-04-27T04:04:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34143.385681 secs 2024-04-27T04:04:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34143.385681 secs 2024-04-27T04:04:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34143.385681 secs 2024-04-27T04:04:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:23 smithi119 ceph-mon[32752]: pgmap v18743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:23 smithi167 ceph-mon[32751]: pgmap v18743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:23 smithi090 ceph-mon[29180]: pgmap v18743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T04:04:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:25 smithi119 ceph-mon[32752]: pgmap v18744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:04:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:25 smithi167 ceph-mon[32751]: pgmap v18744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:04:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:25 smithi090 ceph-mon[29180]: pgmap v18744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:04:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:26 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34147.512487 secs 2024-04-27T04:04:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:26 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34147.512487 secs 2024-04-27T04:04:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:26 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34147.512487 secs 2024-04-27T04:04:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:27 smithi119 ceph-mon[32752]: pgmap v18745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:04:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34148.385747 secs 2024-04-27T04:04:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:27 smithi167 ceph-mon[32751]: pgmap v18745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:04:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34148.385747 secs 2024-04-27T04:04:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:27 smithi090 ceph-mon[29180]: pgmap v18745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:04:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34148.385747 secs 2024-04-27T04:04:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:29 smithi119 ceph-mon[32752]: pgmap v18746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:29 smithi167 ceph-mon[32751]: pgmap v18746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:29.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:29 smithi090 ceph-mon[29180]: pgmap v18746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:31 smithi119 ceph-mon[32752]: pgmap v18747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:31 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34152.512572 secs 2024-04-27T04:04:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:31 smithi167 ceph-mon[32751]: pgmap v18747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:31 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34152.512572 secs 2024-04-27T04:04:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:31 smithi090 ceph-mon[29180]: pgmap v18747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:31 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34152.512572 secs 2024-04-27T04:04:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34153.385862 secs 2024-04-27T04:04:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34153.385862 secs 2024-04-27T04:04:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34153.385862 secs 2024-04-27T04:04:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:33 smithi167 ceph-mon[32751]: pgmap v18748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:33.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:33 smithi090 ceph-mon[29180]: pgmap v18748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:33 smithi119 ceph-mon[32752]: pgmap v18748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:35 smithi167 ceph-mon[32751]: pgmap v18749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:35.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:35 smithi090 ceph-mon[29180]: pgmap v18749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:35 smithi119 ceph-mon[32752]: pgmap v18749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:36 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34157.512655 secs 2024-04-27T04:04:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:36 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34157.512655 secs 2024-04-27T04:04:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:36 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34157.512655 secs 2024-04-27T04:04:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:37 smithi090 ceph-mon[29180]: pgmap v18750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34158.385981 secs 2024-04-27T04:04:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:37 smithi119 ceph-mon[32752]: pgmap v18750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34158.385981 secs 2024-04-27T04:04:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:37 smithi167 ceph-mon[32751]: pgmap v18750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34158.385981 secs 2024-04-27T04:04:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:39 smithi090 ceph-mon[29180]: pgmap v18751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:39 smithi119 ceph-mon[32752]: pgmap v18751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:39 smithi167 ceph-mon[32751]: pgmap v18751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:41 smithi090 ceph-mon[29180]: pgmap v18752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:41 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34162.512741 secs 2024-04-27T04:04:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:41 smithi119 ceph-mon[32752]: pgmap v18752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:41 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34162.512741 secs 2024-04-27T04:04:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:41 smithi167 ceph-mon[32751]: pgmap v18752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:41 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34162.512741 secs 2024-04-27T04:04:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34163.386129 secs 2024-04-27T04:04:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34163.386129 secs 2024-04-27T04:04:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34163.386129 secs 2024-04-27T04:04:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:43 smithi090 ceph-mon[29180]: pgmap v18753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:43 smithi119 ceph-mon[32752]: pgmap v18753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:43 smithi167 ceph-mon[32751]: pgmap v18753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:45 smithi090 ceph-mon[29180]: pgmap v18754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:45 smithi119 ceph-mon[32752]: pgmap v18754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:45 smithi167 ceph-mon[32751]: pgmap v18754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:46 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34167.512828 secs 2024-04-27T04:04:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:46 smithi119 ceph-mon[32752]: pgmap v18755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:46 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34167.512828 secs 2024-04-27T04:04:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:46 smithi167 ceph-mon[32751]: pgmap v18755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:46 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34167.512828 secs 2024-04-27T04:04:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:46 smithi090 ceph-mon[29180]: pgmap v18755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:47.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34168.386244 secs 2024-04-27T04:04:47.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34168.386244 secs 2024-04-27T04:04:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34168.386244 secs 2024-04-27T04:04:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:48 smithi119 ceph-mon[32752]: pgmap v18756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:48 smithi167 ceph-mon[32751]: pgmap v18756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:48 smithi090 ceph-mon[29180]: pgmap v18756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:50 smithi119 ceph-mon[32752]: pgmap v18757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34172.512913 secs 2024-04-27T04:04:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:50 smithi167 ceph-mon[32751]: pgmap v18757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34172.512913 secs 2024-04-27T04:04:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:50 smithi090 ceph-mon[29180]: pgmap v18757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34172.512913 secs 2024-04-27T04:04:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34173.386360 secs 2024-04-27T04:04:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34173.386360 secs 2024-04-27T04:04:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34173.386360 secs 2024-04-27T04:04:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:52 smithi119 ceph-mon[32752]: pgmap v18758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:52 smithi167 ceph-mon[32751]: pgmap v18758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:52 smithi090 ceph-mon[29180]: pgmap v18758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:54 smithi119 ceph-mon[32752]: pgmap v18759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:54 smithi167 ceph-mon[32751]: pgmap v18759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:55.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:54 smithi090 ceph-mon[29180]: pgmap v18759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:04:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34177.513001 secs 2024-04-27T04:04:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34177.513001 secs 2024-04-27T04:04:56.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34177.513001 secs 2024-04-27T04:04:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:56 smithi119 ceph-mon[32752]: pgmap v18760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34178.386417 secs 2024-04-27T04:04:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:56 smithi167 ceph-mon[32751]: pgmap v18760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34178.386417 secs 2024-04-27T04:04:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:56 smithi090 ceph-mon[29180]: pgmap v18760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:57.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34178.386417 secs 2024-04-27T04:04:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:04:58 smithi119 ceph-mon[32752]: pgmap v18761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:04:59 smithi167 ceph-mon[32751]: pgmap v18761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:04:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:04:58 smithi090 ceph-mon[29180]: pgmap v18761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:01 smithi119 ceph-mon[32752]: pgmap v18762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:01 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34182.513087 secs 2024-04-27T04:05:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:01 smithi167 ceph-mon[32751]: pgmap v18762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:01 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34182.513087 secs 2024-04-27T04:05:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:01 smithi090 ceph-mon[29180]: pgmap v18762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:01 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34182.513087 secs 2024-04-27T04:05:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34183.386528 secs 2024-04-27T04:05:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34183.386528 secs 2024-04-27T04:05:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34183.386528 secs 2024-04-27T04:05:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:03 smithi119 ceph-mon[32752]: pgmap v18763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:03 smithi167 ceph-mon[32751]: pgmap v18763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:03 smithi090 ceph-mon[29180]: pgmap v18763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:05 smithi119 ceph-mon[32752]: pgmap v18764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:05 smithi167 ceph-mon[32751]: pgmap v18764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:05 smithi090 ceph-mon[29180]: pgmap v18764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:06 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34187.513147 secs 2024-04-27T04:05:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:06 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34187.513147 secs 2024-04-27T04:05:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:06 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34187.513147 secs 2024-04-27T04:05:07.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:07 smithi090 ceph-mon[29180]: pgmap v18765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:07.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34188.386643 secs 2024-04-27T04:05:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:07 smithi119 ceph-mon[32752]: pgmap v18765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34188.386643 secs 2024-04-27T04:05:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:07 smithi167 ceph-mon[32751]: pgmap v18765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34188.386643 secs 2024-04-27T04:05:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:09 smithi119 ceph-mon[32752]: pgmap v18766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:09 smithi167 ceph-mon[32751]: pgmap v18766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:09 smithi090 ceph-mon[29180]: pgmap v18766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:11 smithi119 ceph-mon[32752]: pgmap v18767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34192.513208 secs 2024-04-27T04:05:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:11 smithi167 ceph-mon[32751]: pgmap v18767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34192.513208 secs 2024-04-27T04:05:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:11 smithi090 ceph-mon[29180]: pgmap v18767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34192.513208 secs 2024-04-27T04:05:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34193.386716 secs 2024-04-27T04:05:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34193.386716 secs 2024-04-27T04:05:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34193.386716 secs 2024-04-27T04:05:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:13 smithi119 ceph-mon[32752]: pgmap v18768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:13 smithi167 ceph-mon[32751]: pgmap v18768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:13 smithi090 ceph-mon[29180]: pgmap v18768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:05:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:15 smithi119 ceph-mon[32752]: pgmap v18769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:15 smithi167 ceph-mon[32751]: pgmap v18769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:15 smithi090 ceph-mon[29180]: pgmap v18769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:05:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34197.513267 secs 2024-04-27T04:05:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34197.513267 secs 2024-04-27T04:05:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34197.513267 secs 2024-04-27T04:05:16.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:05:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:17 smithi119 ceph-mon[32752]: pgmap v18770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34198.386781 secs 2024-04-27T04:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:17 smithi167 ceph-mon[32751]: pgmap v18770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34198.386781 secs 2024-04-27T04:05:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:17 smithi090 ceph-mon[29180]: pgmap v18770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34198.386781 secs 2024-04-27T04:05:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: pgmap v18771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: pgmap v18771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: pgmap v18771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:19.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:05:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:05:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:05:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:05:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:20.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:05:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:05:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:05:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:05:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:21 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:21 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:21 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:21 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:21 smithi167 ceph-mon[32751]: pgmap v18772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T04:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:21 smithi167 ceph-mon[32751]: pgmap v18773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34202.513354 secs 2024-04-27T04:05:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:21 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:05:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:21 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:05:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:21 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:05:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:21 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:05:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:21 smithi090 ceph-mon[29180]: pgmap v18772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T04:05:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:21 smithi090 ceph-mon[29180]: pgmap v18773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34202.513354 secs 2024-04-27T04:05:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:21 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:21 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:21 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:21 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:21 smithi119 ceph-mon[32752]: pgmap v18772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T04:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:21 smithi119 ceph-mon[32752]: pgmap v18773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34202.513354 secs 2024-04-27T04:05:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34203.386899 secs 2024-04-27T04:05:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34203.386899 secs 2024-04-27T04:05:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34203.386899 secs 2024-04-27T04:05:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:23 smithi167 ceph-mon[32751]: pgmap v18774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:23 smithi090 ceph-mon[29180]: pgmap v18774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:23 smithi119 ceph-mon[32752]: pgmap v18774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:25 smithi090 ceph-mon[29180]: pgmap v18775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:25 smithi119 ceph-mon[32752]: pgmap v18775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:25 smithi167 ceph-mon[32751]: pgmap v18775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:26 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34207.513438 secs 2024-04-27T04:05:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:26 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34207.513438 secs 2024-04-27T04:05:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:26 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34207.513438 secs 2024-04-27T04:05:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:27 smithi090 ceph-mon[29180]: pgmap v18776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34208.387019 secs 2024-04-27T04:05:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:27 smithi119 ceph-mon[32752]: pgmap v18776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34208.387019 secs 2024-04-27T04:05:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:27 smithi167 ceph-mon[32751]: pgmap v18776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34208.387019 secs 2024-04-27T04:05:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:29 smithi090 ceph-mon[29180]: pgmap v18777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:29 smithi119 ceph-mon[32752]: pgmap v18777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:29 smithi167 ceph-mon[32751]: pgmap v18777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:05:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:31 smithi090 ceph-mon[29180]: pgmap v18778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:05:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:31 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34212.513524 secs 2024-04-27T04:05:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:31 smithi119 ceph-mon[32752]: pgmap v18778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:05:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:31 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34212.513524 secs 2024-04-27T04:05:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:31 smithi167 ceph-mon[32751]: pgmap v18778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:05:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:31 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34212.513524 secs 2024-04-27T04:05:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34213.387151 secs 2024-04-27T04:05:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34213.387151 secs 2024-04-27T04:05:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34213.387151 secs 2024-04-27T04:05:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:33 smithi090 ceph-mon[29180]: pgmap v18779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:33 smithi119 ceph-mon[32752]: pgmap v18779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:33 smithi167 ceph-mon[32751]: pgmap v18779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:35 smithi090 ceph-mon[29180]: pgmap v18780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:35 smithi119 ceph-mon[32752]: pgmap v18780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:35 smithi167 ceph-mon[32751]: pgmap v18780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:36 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34217.513606 secs 2024-04-27T04:05:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:36 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34217.513606 secs 2024-04-27T04:05:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:36 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34217.513606 secs 2024-04-27T04:05:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:37 smithi090 ceph-mon[29180]: pgmap v18781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:37.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34218.387228 secs 2024-04-27T04:05:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:37 smithi119 ceph-mon[32752]: pgmap v18781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34218.387228 secs 2024-04-27T04:05:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:37 smithi167 ceph-mon[32751]: pgmap v18781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34218.387228 secs 2024-04-27T04:05:39.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:39 smithi090 ceph-mon[29180]: pgmap v18782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:39 smithi119 ceph-mon[32752]: pgmap v18782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:39 smithi167 ceph-mon[32751]: pgmap v18782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:41 smithi090 ceph-mon[29180]: pgmap v18783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:41 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34222.513690 secs 2024-04-27T04:05:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:41 smithi119 ceph-mon[32752]: pgmap v18783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:41 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34222.513690 secs 2024-04-27T04:05:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:41 smithi167 ceph-mon[32751]: pgmap v18783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:41 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34222.513690 secs 2024-04-27T04:05:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:42 smithi090 ceph-mon[29180]: 31 slow requests, 2 included below; oldest blocked for > 34223.387345 secs 2024-04-27T04:05:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:42 smithi090 ceph-mon[29180]: slow request 11524.082632 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:05:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:42 smithi090 ceph-mon[29180]: slow request 5762.478469 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:05:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:42 smithi119 ceph-mon[32752]: 31 slow requests, 2 included below; oldest blocked for > 34223.387345 secs 2024-04-27T04:05:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:42 smithi119 ceph-mon[32752]: slow request 11524.082632 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:05:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:42 smithi119 ceph-mon[32752]: slow request 5762.478469 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:05:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:42 smithi167 ceph-mon[32751]: 31 slow requests, 2 included below; oldest blocked for > 34223.387345 secs 2024-04-27T04:05:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:42 smithi167 ceph-mon[32751]: slow request 11524.082632 seconds old, received at 2024-04-27T00:53:37.055357+0000: client_request(client.17538:1 getattr pAsLsXsFs #0x1 2024-04-27T00:53:37.048836+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:05:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:42 smithi167 ceph-mon[32751]: slow request 5762.478469 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:05:43.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:43 smithi090 ceph-mon[29180]: pgmap v18784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:43 smithi119 ceph-mon[32752]: pgmap v18784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:43 smithi167 ceph-mon[32751]: pgmap v18784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:45.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:45 smithi090 ceph-mon[29180]: pgmap v18785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:45 smithi119 ceph-mon[32752]: pgmap v18785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:45 smithi167 ceph-mon[32751]: pgmap v18785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:46.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:46 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 34227.513751 secs 2024-04-27T04:05:46.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:46 smithi119 ceph-mon[32752]: slow request 23043.812468 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:05:46.722 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:46 smithi119 ceph-mon[32752]: pgmap v18786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:46.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:46 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 34227.513751 secs 2024-04-27T04:05:46.744 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:46 smithi167 ceph-mon[32751]: slow request 23043.812468 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:05:46.745 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:46 smithi167 ceph-mon[32751]: pgmap v18786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:46 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 34227.513751 secs 2024-04-27T04:05:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:46 smithi090 ceph-mon[29180]: slow request 23043.812468 seconds old, received at 2024-04-26T21:41:41.454687+0000: client_request(client.24733:267890 getattr AsXsFs #0x100000001fb 2024-04-26T21:41:41.453714+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:05:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:46 smithi090 ceph-mon[29180]: pgmap v18786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:47.670 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34228.387463 secs 2024-04-27T04:05:47.679 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34228.387463 secs 2024-04-27T04:05:47.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34228.387463 secs 2024-04-27T04:05:48.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:48 smithi119 ceph-mon[32752]: pgmap v18787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:48.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:48 smithi167 ceph-mon[32751]: pgmap v18787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:48.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:48 smithi090 ceph-mon[29180]: pgmap v18787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:50.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:50 smithi119 ceph-mon[32752]: pgmap v18788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:50.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:50 smithi167 ceph-mon[32751]: pgmap v18788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:50.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:50 smithi090 ceph-mon[29180]: pgmap v18788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:51 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34232.513822 secs 2024-04-27T04:05:51.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34233.387586 secs 2024-04-27T04:05:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:51 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34232.513822 secs 2024-04-27T04:05:51.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34233.387586 secs 2024-04-27T04:05:51.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:51 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34232.513822 secs 2024-04-27T04:05:51.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34233.387586 secs 2024-04-27T04:05:52.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:52 smithi119 ceph-mon[32752]: pgmap v18789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:52.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:52 smithi167 ceph-mon[32751]: pgmap v18789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:52.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:52 smithi090 ceph-mon[29180]: pgmap v18789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:54.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:54 smithi119 ceph-mon[32752]: pgmap v18790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:54.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:54 smithi167 ceph-mon[32751]: pgmap v18790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:54.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:54 smithi090 ceph-mon[29180]: pgmap v18790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:05:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:56 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34237.513909 secs 2024-04-27T04:05:56.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:56 smithi119 ceph-mon[32752]: pgmap v18791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:56.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34238.387667 secs 2024-04-27T04:05:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:56 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34237.513909 secs 2024-04-27T04:05:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:56 smithi167 ceph-mon[32751]: pgmap v18791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:56.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34238.387667 secs 2024-04-27T04:05:56.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:56 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34237.513909 secs 2024-04-27T04:05:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:56 smithi090 ceph-mon[29180]: pgmap v18791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34238.387667 secs 2024-04-27T04:05:58.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:05:58 smithi119 ceph-mon[32752]: pgmap v18792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:58.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:05:58 smithi167 ceph-mon[32751]: pgmap v18792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:05:58.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:05:58 smithi090 ceph-mon[29180]: pgmap v18792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:00.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:00 smithi119 ceph-mon[32752]: pgmap v18793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:00.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:00 smithi167 ceph-mon[32751]: pgmap v18793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:00.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:00 smithi090 ceph-mon[29180]: pgmap v18793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:01 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34242.513990 secs 2024-04-27T04:06:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34243.387779 secs 2024-04-27T04:06:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:01 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34242.513990 secs 2024-04-27T04:06:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34243.387779 secs 2024-04-27T04:06:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:01 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34242.513990 secs 2024-04-27T04:06:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34243.387779 secs 2024-04-27T04:06:02.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:02 smithi119 ceph-mon[32752]: pgmap v18794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:02.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:02 smithi167 ceph-mon[32751]: pgmap v18794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:02.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:02 smithi090 ceph-mon[29180]: pgmap v18794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:04.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:04 smithi119 ceph-mon[32752]: pgmap v18795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:04.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:04 smithi167 ceph-mon[32751]: pgmap v18795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:04.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:04 smithi090 ceph-mon[29180]: pgmap v18795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:06 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34247.514071 secs 2024-04-27T04:06:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:06 smithi119 ceph-mon[32752]: pgmap v18796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:06 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34247.514071 secs 2024-04-27T04:06:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:06 smithi167 ceph-mon[32751]: pgmap v18796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:06 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34247.514071 secs 2024-04-27T04:06:06.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:06 smithi090 ceph-mon[29180]: pgmap v18796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:07.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34248.387896 secs 2024-04-27T04:06:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34248.387896 secs 2024-04-27T04:06:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34248.387896 secs 2024-04-27T04:06:08.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:08 smithi119 ceph-mon[32752]: pgmap v18797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:08.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:08 smithi167 ceph-mon[32751]: pgmap v18797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:08.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:08 smithi090 ceph-mon[29180]: pgmap v18797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:10.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:10 smithi119 ceph-mon[32752]: pgmap v18798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:10.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:10 smithi167 ceph-mon[32751]: pgmap v18798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:10.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:10 smithi090 ceph-mon[29180]: pgmap v18798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:11.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34252.514154 secs 2024-04-27T04:06:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34252.514154 secs 2024-04-27T04:06:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34252.514154 secs 2024-04-27T04:06:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34253.388012 secs 2024-04-27T04:06:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:12 smithi119 ceph-mon[32752]: pgmap v18799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34253.388012 secs 2024-04-27T04:06:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:12 smithi167 ceph-mon[32751]: pgmap v18799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34253.388012 secs 2024-04-27T04:06:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:12 smithi090 ceph-mon[29180]: pgmap v18799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:14.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:14 smithi119 ceph-mon[32752]: pgmap v18800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:14.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:14 smithi167 ceph-mon[32751]: pgmap v18800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:14.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:14 smithi090 ceph-mon[29180]: pgmap v18800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34257.514237 secs 2024-04-27T04:06:16.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:16 smithi119 ceph-mon[32752]: pgmap v18801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34258.388102 secs 2024-04-27T04:06:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34257.514237 secs 2024-04-27T04:06:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:16 smithi167 ceph-mon[32751]: pgmap v18801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:16.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34258.388102 secs 2024-04-27T04:06:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34257.514237 secs 2024-04-27T04:06:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:16 smithi090 ceph-mon[29180]: pgmap v18801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34258.388102 secs 2024-04-27T04:06:18.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:18 smithi119 ceph-mon[32752]: pgmap v18802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:18.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:18 smithi167 ceph-mon[32751]: pgmap v18802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:18.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:18 smithi090 ceph-mon[29180]: pgmap v18802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:20 smithi119 ceph-mon[32752]: pgmap v18803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:20.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:06:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:20 smithi167 ceph-mon[32751]: pgmap v18803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:06:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:20.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:20 smithi090 ceph-mon[29180]: pgmap v18803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:06:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:20.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34262.514324 secs 2024-04-27T04:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:21 smithi119 ceph-mon[32752]: pgmap v18804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:21 smithi119 ceph-mon[32752]: pgmap v18805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:21.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34263.388202 secs 2024-04-27T04:06:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34262.514324 secs 2024-04-27T04:06:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:06:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:21 smithi167 ceph-mon[32751]: pgmap v18804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:06:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:21 smithi167 ceph-mon[32751]: pgmap v18805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:21.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34263.388202 secs 2024-04-27T04:06:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34262.514324 secs 2024-04-27T04:06:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:06:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:06:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:21 smithi090 ceph-mon[29180]: pgmap v18804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:06:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:21 smithi090 ceph-mon[29180]: pgmap v18805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:06:21.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34263.388202 secs 2024-04-27T04:06:24.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:23 smithi119 ceph-mon[32752]: pgmap v18806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T04:06:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:23 smithi167 ceph-mon[32751]: pgmap v18806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T04:06:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:23 smithi090 ceph-mon[29180]: pgmap v18806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T04:06:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:25 smithi119 ceph-mon[32752]: pgmap v18807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34267.514405 secs 2024-04-27T04:06:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:25 smithi167 ceph-mon[32751]: pgmap v18807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:26.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34267.514405 secs 2024-04-27T04:06:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:25 smithi090 ceph-mon[29180]: pgmap v18807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:26.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34267.514405 secs 2024-04-27T04:06:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34268.388337 secs 2024-04-27T04:06:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34268.388337 secs 2024-04-27T04:06:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34268.388337 secs 2024-04-27T04:06:27.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:27 smithi090 ceph-mon[29180]: pgmap v18808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:28.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:27 smithi119 ceph-mon[32752]: pgmap v18808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:28.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:27 smithi167 ceph-mon[32751]: pgmap v18808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:30.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:29 smithi119 ceph-mon[32752]: pgmap v18809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:30.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:29 smithi167 ceph-mon[32751]: pgmap v18809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:30.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:29 smithi090 ceph-mon[29180]: pgmap v18809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:06:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34272.514486 secs 2024-04-27T04:06:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34272.514486 secs 2024-04-27T04:06:31.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34272.514486 secs 2024-04-27T04:06:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:31 smithi119 ceph-mon[32752]: pgmap v18810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:06:32.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34273.388463 secs 2024-04-27T04:06:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:31 smithi167 ceph-mon[32751]: pgmap v18810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:06:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34273.388463 secs 2024-04-27T04:06:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:31 smithi090 ceph-mon[29180]: pgmap v18810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:06:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34273.388463 secs 2024-04-27T04:06:34.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:33 smithi119 ceph-mon[32752]: pgmap v18811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:34.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:33 smithi167 ceph-mon[32751]: pgmap v18811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:34.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:33 smithi090 ceph-mon[29180]: pgmap v18811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:35 smithi119 ceph-mon[32752]: pgmap v18812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:36.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34277.514557 secs 2024-04-27T04:06:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:35 smithi167 ceph-mon[32751]: pgmap v18812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34277.514557 secs 2024-04-27T04:06:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:35 smithi090 ceph-mon[29180]: pgmap v18812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:36.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34277.514557 secs 2024-04-27T04:06:37.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34278.388609 secs 2024-04-27T04:06:37.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34278.388609 secs 2024-04-27T04:06:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34278.388609 secs 2024-04-27T04:06:38.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:37 smithi119 ceph-mon[32752]: pgmap v18813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:38.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:37 smithi167 ceph-mon[32751]: pgmap v18813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:38.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:37 smithi090 ceph-mon[29180]: pgmap v18813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:39 smithi119 ceph-mon[32752]: pgmap v18814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:40.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:39 smithi167 ceph-mon[32751]: pgmap v18814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:39 smithi090 ceph-mon[29180]: pgmap v18814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34282.514641 secs 2024-04-27T04:06:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34282.514641 secs 2024-04-27T04:06:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34282.514641 secs 2024-04-27T04:06:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:41 smithi167 ceph-mon[32751]: pgmap v18815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:42.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34283.388734 secs 2024-04-27T04:06:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:41 smithi090 ceph-mon[29180]: pgmap v18815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34283.388734 secs 2024-04-27T04:06:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:41 smithi119 ceph-mon[32752]: pgmap v18815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34283.388734 secs 2024-04-27T04:06:44.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:43 smithi167 ceph-mon[32751]: pgmap v18816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:43 smithi090 ceph-mon[29180]: pgmap v18816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:43 smithi119 ceph-mon[32752]: pgmap v18816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:46.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:45 smithi167 ceph-mon[32751]: pgmap v18817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:46.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34287.514721 secs 2024-04-27T04:06:46.104 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:45 smithi119 ceph-mon[32752]: pgmap v18817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:46.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34287.514721 secs 2024-04-27T04:06:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:45 smithi090 ceph-mon[29180]: pgmap v18817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34287.514721 secs 2024-04-27T04:06:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34288.388824 secs 2024-04-27T04:06:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34288.388824 secs 2024-04-27T04:06:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34288.388824 secs 2024-04-27T04:06:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:47 smithi090 ceph-mon[29180]: pgmap v18818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:47 smithi119 ceph-mon[32752]: pgmap v18818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:47 smithi167 ceph-mon[32751]: pgmap v18818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:49 smithi090 ceph-mon[29180]: pgmap v18819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:49 smithi119 ceph-mon[32752]: pgmap v18819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:49 smithi167 ceph-mon[32751]: pgmap v18819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34292.514807 secs 2024-04-27T04:06:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34292.514807 secs 2024-04-27T04:06:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34292.514807 secs 2024-04-27T04:06:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:51 smithi090 ceph-mon[29180]: pgmap v18820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34293.388946 secs 2024-04-27T04:06:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:51 smithi119 ceph-mon[32752]: pgmap v18820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34293.388946 secs 2024-04-27T04:06:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:51 smithi167 ceph-mon[32751]: pgmap v18820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34293.388946 secs 2024-04-27T04:06:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:53 smithi090 ceph-mon[29180]: pgmap v18821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:53 smithi119 ceph-mon[32752]: pgmap v18821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:53 smithi167 ceph-mon[32751]: pgmap v18821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:55 smithi090 ceph-mon[29180]: pgmap v18822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34297.514868 secs 2024-04-27T04:06:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:55 smithi119 ceph-mon[32752]: pgmap v18822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34297.514868 secs 2024-04-27T04:06:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:55 smithi167 ceph-mon[32751]: pgmap v18822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:06:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34297.514868 secs 2024-04-27T04:06:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34298.389082 secs 2024-04-27T04:06:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34298.389082 secs 2024-04-27T04:06:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34298.389082 secs 2024-04-27T04:06:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:57 smithi090 ceph-mon[29180]: pgmap v18823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:57 smithi119 ceph-mon[32752]: pgmap v18823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:06:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:57 smithi167 ceph-mon[32751]: pgmap v18823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:00.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:06:59 smithi090 ceph-mon[29180]: pgmap v18824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:06:59 smithi119 ceph-mon[32752]: pgmap v18824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:00.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:06:59 smithi167 ceph-mon[32751]: pgmap v18824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:00 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34302.514952 secs 2024-04-27T04:07:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:00 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34302.514952 secs 2024-04-27T04:07:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:00 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34302.514952 secs 2024-04-27T04:07:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:01 smithi090 ceph-mon[29180]: pgmap v18825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34303.389212 secs 2024-04-27T04:07:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:01 smithi119 ceph-mon[32752]: pgmap v18825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34303.389212 secs 2024-04-27T04:07:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:01 smithi167 ceph-mon[32751]: pgmap v18825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34303.389212 secs 2024-04-27T04:07:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:03 smithi090 ceph-mon[29180]: pgmap v18826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:04.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:03 smithi119 ceph-mon[32752]: pgmap v18826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:03 smithi167 ceph-mon[32751]: pgmap v18826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:05 smithi090 ceph-mon[29180]: pgmap v18827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:05 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34307.515031 secs 2024-04-27T04:07:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:05 smithi119 ceph-mon[32752]: pgmap v18827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:05 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34307.515031 secs 2024-04-27T04:07:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:05 smithi167 ceph-mon[32751]: pgmap v18827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:05 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34307.515031 secs 2024-04-27T04:07:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34308.389361 secs 2024-04-27T04:07:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34308.389361 secs 2024-04-27T04:07:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34308.389361 secs 2024-04-27T04:07:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:07 smithi090 ceph-mon[29180]: pgmap v18828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:07 smithi119 ceph-mon[32752]: pgmap v18828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:07 smithi167 ceph-mon[32751]: pgmap v18828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:10.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:09 smithi119 ceph-mon[32752]: pgmap v18829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:10.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:09 smithi167 ceph-mon[32751]: pgmap v18829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:10.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:09 smithi090 ceph-mon[29180]: pgmap v18829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:10 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34312.515114 secs 2024-04-27T04:07:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:10 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34312.515114 secs 2024-04-27T04:07:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:10 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34312.515114 secs 2024-04-27T04:07:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:11 smithi119 ceph-mon[32752]: pgmap v18830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:11 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 34313.389423 secs 2024-04-27T04:07:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:11 smithi119 ceph-mon[32752]: slow request 1441.241991 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:07:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:11 smithi167 ceph-mon[32751]: pgmap v18830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:11 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 34313.389423 secs 2024-04-27T04:07:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:11 smithi167 ceph-mon[32751]: slow request 1441.241991 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:07:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:11 smithi090 ceph-mon[29180]: pgmap v18830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:11 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 34313.389423 secs 2024-04-27T04:07:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:11 smithi090 ceph-mon[29180]: slow request 1441.241991 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:07:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:13 smithi090 ceph-mon[29180]: pgmap v18831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:14.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:13 smithi119 ceph-mon[32752]: pgmap v18831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:13 smithi167 ceph-mon[32751]: pgmap v18831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:15 smithi119 ceph-mon[32752]: pgmap v18832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:15 smithi119 ceph-mon[32752]: 45 slow requests, 1 included below; oldest blocked for > 34317.515173 secs 2024-04-27T04:07:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:15 smithi119 ceph-mon[32752]: slow request 2880.318719 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:07:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:15 smithi167 ceph-mon[32751]: pgmap v18832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:15 smithi167 ceph-mon[32751]: 45 slow requests, 1 included below; oldest blocked for > 34317.515173 secs 2024-04-27T04:07:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:15 smithi167 ceph-mon[32751]: slow request 2880.318719 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:07:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:15 smithi090 ceph-mon[29180]: pgmap v18832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:15 smithi090 ceph-mon[29180]: 45 slow requests, 1 included below; oldest blocked for > 34317.515173 secs 2024-04-27T04:07:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:15 smithi090 ceph-mon[29180]: slow request 2880.318719 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:07:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34318.389551 secs 2024-04-27T04:07:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34318.389551 secs 2024-04-27T04:07:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34318.389551 secs 2024-04-27T04:07:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:17 smithi119 ceph-mon[32752]: pgmap v18833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:17 smithi167 ceph-mon[32751]: pgmap v18833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:18.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:17 smithi090 ceph-mon[29180]: pgmap v18833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:19 smithi119 ceph-mon[32752]: pgmap v18834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:20.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:19 smithi167 ceph-mon[32751]: pgmap v18834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:20.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:19 smithi090 ceph-mon[29180]: pgmap v18834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:20 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34322.515260 secs 2024-04-27T04:07:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:07:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:20 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34322.515260 secs 2024-04-27T04:07:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:07:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:20 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34322.515260 secs 2024-04-27T04:07:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:07:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:21 smithi119 ceph-mon[32752]: pgmap v18835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34323.389605 secs 2024-04-27T04:07:22.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:07:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:07:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:21 smithi167 ceph-mon[32751]: pgmap v18835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34323.389605 secs 2024-04-27T04:07:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:07:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:21 smithi090 ceph-mon[29180]: pgmap v18835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34323.389605 secs 2024-04-27T04:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:07:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:07:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:22 smithi119 ceph-mon[32752]: pgmap v18836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-27T04:07:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:22 smithi119 ceph-mon[32752]: pgmap v18837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:22 smithi167 ceph-mon[32751]: pgmap v18836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-27T04:07:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:22 smithi167 ceph-mon[32751]: pgmap v18837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:22 smithi090 ceph-mon[29180]: pgmap v18836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 947 B/s rd, 1 op/s 2024-04-27T04:07:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:22 smithi090 ceph-mon[29180]: pgmap v18837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:24 smithi119 ceph-mon[32752]: pgmap v18838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:24 smithi167 ceph-mon[32751]: pgmap v18838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:24 smithi090 ceph-mon[29180]: pgmap v18838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:26.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:25 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34327.515320 secs 2024-04-27T04:07:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:25 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34327.515320 secs 2024-04-27T04:07:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:25 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34327.515320 secs 2024-04-27T04:07:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:26 smithi119 ceph-mon[32752]: pgmap v18839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34328.389668 secs 2024-04-27T04:07:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:26 smithi167 ceph-mon[32751]: pgmap v18839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34328.389668 secs 2024-04-27T04:07:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:26 smithi090 ceph-mon[29180]: pgmap v18839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34328.389668 secs 2024-04-27T04:07:29.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:28 smithi119 ceph-mon[32752]: pgmap v18840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T04:07:29.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:28 smithi167 ceph-mon[32751]: pgmap v18840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T04:07:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:28 smithi090 ceph-mon[29180]: pgmap v18840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T04:07:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:30 smithi119 ceph-mon[32752]: pgmap v18841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:30 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34332.515404 secs 2024-04-27T04:07:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:30 smithi167 ceph-mon[32751]: pgmap v18841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:30 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34332.515404 secs 2024-04-27T04:07:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:30 smithi090 ceph-mon[29180]: pgmap v18841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:07:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:30 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34332.515404 secs 2024-04-27T04:07:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34333.389738 secs 2024-04-27T04:07:32.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34333.389738 secs 2024-04-27T04:07:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34333.389738 secs 2024-04-27T04:07:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:32 smithi119 ceph-mon[32752]: pgmap v18842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:07:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:32 smithi167 ceph-mon[32751]: pgmap v18842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:07:33.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:32 smithi090 ceph-mon[29180]: pgmap v18842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:07:35.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:34 smithi119 ceph-mon[32752]: pgmap v18843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:35.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:34 smithi167 ceph-mon[32751]: pgmap v18843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:35.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:34 smithi090 ceph-mon[29180]: pgmap v18843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:35 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34337.515488 secs 2024-04-27T04:07:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:35 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34337.515488 secs 2024-04-27T04:07:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:35 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34337.515488 secs 2024-04-27T04:07:37.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:36 smithi090 ceph-mon[29180]: pgmap v18844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:37.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34338.389866 secs 2024-04-27T04:07:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:36 smithi119 ceph-mon[32752]: pgmap v18844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34338.389866 secs 2024-04-27T04:07:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:36 smithi167 ceph-mon[32751]: pgmap v18844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34338.389866 secs 2024-04-27T04:07:39.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:38 smithi119 ceph-mon[32752]: pgmap v18845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:38 smithi167 ceph-mon[32751]: pgmap v18845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:38 smithi090 ceph-mon[29180]: pgmap v18845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:40 smithi119 ceph-mon[32752]: pgmap v18846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:40 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34342.515574 secs 2024-04-27T04:07:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:40 smithi167 ceph-mon[32751]: pgmap v18846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:40 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34342.515574 secs 2024-04-27T04:07:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:40 smithi090 ceph-mon[29180]: pgmap v18846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:40 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34342.515574 secs 2024-04-27T04:07:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34343.389978 secs 2024-04-27T04:07:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34343.389978 secs 2024-04-27T04:07:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34343.389978 secs 2024-04-27T04:07:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:42 smithi119 ceph-mon[32752]: pgmap v18847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:42 smithi167 ceph-mon[32751]: pgmap v18847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:42 smithi090 ceph-mon[29180]: pgmap v18847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:45.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:44 smithi090 ceph-mon[29180]: pgmap v18848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:45.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:44 smithi119 ceph-mon[32752]: pgmap v18848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:45.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:44 smithi167 ceph-mon[32751]: pgmap v18848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:45 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34347.515676 secs 2024-04-27T04:07:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:45 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34347.515676 secs 2024-04-27T04:07:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:45 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34347.515676 secs 2024-04-27T04:07:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:46 smithi119 ceph-mon[32752]: pgmap v18849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34348.390040 secs 2024-04-27T04:07:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:46 smithi167 ceph-mon[32751]: pgmap v18849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34348.390040 secs 2024-04-27T04:07:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:46 smithi090 ceph-mon[29180]: pgmap v18849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34348.390040 secs 2024-04-27T04:07:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:48 smithi119 ceph-mon[32752]: pgmap v18850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:48 smithi167 ceph-mon[32751]: pgmap v18850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:48 smithi090 ceph-mon[29180]: pgmap v18850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:50 smithi119 ceph-mon[32752]: pgmap v18851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:50 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34352.515758 secs 2024-04-27T04:07:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:50 smithi167 ceph-mon[32751]: pgmap v18851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:50 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34352.515758 secs 2024-04-27T04:07:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:50 smithi090 ceph-mon[29180]: pgmap v18851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:50 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34352.515758 secs 2024-04-27T04:07:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34353.390101 secs 2024-04-27T04:07:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34353.390101 secs 2024-04-27T04:07:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34353.390101 secs 2024-04-27T04:07:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:52 smithi119 ceph-mon[32752]: pgmap v18852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:52 smithi167 ceph-mon[32751]: pgmap v18852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:52 smithi090 ceph-mon[29180]: pgmap v18852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:54 smithi119 ceph-mon[32752]: pgmap v18853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:54 smithi167 ceph-mon[32751]: pgmap v18853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:54 smithi090 ceph-mon[29180]: pgmap v18853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:07:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:55 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34357.515839 secs 2024-04-27T04:07:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:55 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34357.515839 secs 2024-04-27T04:07:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:55 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34357.515839 secs 2024-04-27T04:07:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:56 smithi119 ceph-mon[32752]: pgmap v18854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34358.390164 secs 2024-04-27T04:07:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:56 smithi167 ceph-mon[32751]: pgmap v18854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34358.390164 secs 2024-04-27T04:07:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:56 smithi090 ceph-mon[29180]: pgmap v18854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34358.390164 secs 2024-04-27T04:07:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:07:59 smithi119 ceph-mon[32752]: pgmap v18855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:07:59 smithi167 ceph-mon[32751]: pgmap v18855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:07:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:07:58 smithi090 ceph-mon[29180]: pgmap v18855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:01 smithi119 ceph-mon[32752]: pgmap v18856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:01 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34362.515924 secs 2024-04-27T04:08:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:01 smithi167 ceph-mon[32751]: pgmap v18856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:01 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34362.515924 secs 2024-04-27T04:08:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:01 smithi090 ceph-mon[29180]: pgmap v18856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:01 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34362.515924 secs 2024-04-27T04:08:02.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34363.390315 secs 2024-04-27T04:08:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34363.390315 secs 2024-04-27T04:08:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34363.390315 secs 2024-04-27T04:08:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:03 smithi119 ceph-mon[32752]: pgmap v18857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:03 smithi167 ceph-mon[32751]: pgmap v18857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:03 smithi090 ceph-mon[29180]: pgmap v18857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:05 smithi119 ceph-mon[32752]: pgmap v18858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:05 smithi167 ceph-mon[32751]: pgmap v18858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:05 smithi090 ceph-mon[29180]: pgmap v18858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:06 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34367.516008 secs 2024-04-27T04:08:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:06 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34367.516008 secs 2024-04-27T04:08:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:06 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34367.516008 secs 2024-04-27T04:08:07.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:07 smithi090 ceph-mon[29180]: pgmap v18859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:07.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34368.390434 secs 2024-04-27T04:08:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:07 smithi119 ceph-mon[32752]: pgmap v18859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34368.390434 secs 2024-04-27T04:08:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:07 smithi167 ceph-mon[32751]: pgmap v18859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34368.390434 secs 2024-04-27T04:08:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:09 smithi119 ceph-mon[32752]: pgmap v18860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:09.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:09 smithi167 ceph-mon[32751]: pgmap v18860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:09 smithi090 ceph-mon[29180]: pgmap v18860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:11 smithi119 ceph-mon[32752]: pgmap v18861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34372.516091 secs 2024-04-27T04:08:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:11 smithi167 ceph-mon[32751]: pgmap v18861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34372.516091 secs 2024-04-27T04:08:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:11 smithi090 ceph-mon[29180]: pgmap v18861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34372.516091 secs 2024-04-27T04:08:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34373.390548 secs 2024-04-27T04:08:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34373.390548 secs 2024-04-27T04:08:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34373.390548 secs 2024-04-27T04:08:13.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:13 smithi119 ceph-mon[32752]: pgmap v18862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:13.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:13 smithi167 ceph-mon[32751]: pgmap v18862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:13 smithi090 ceph-mon[29180]: pgmap v18862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:15.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:15 smithi119 ceph-mon[32752]: pgmap v18863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:15.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:15 smithi167 ceph-mon[32751]: pgmap v18863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:15 smithi090 ceph-mon[29180]: pgmap v18863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:16.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34377.516170 secs 2024-04-27T04:08:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34377.516170 secs 2024-04-27T04:08:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34377.516170 secs 2024-04-27T04:08:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:17 smithi119 ceph-mon[32752]: pgmap v18864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34378.390675 secs 2024-04-27T04:08:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:17 smithi167 ceph-mon[32751]: pgmap v18864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:17.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34378.390675 secs 2024-04-27T04:08:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:17 smithi090 ceph-mon[29180]: pgmap v18864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34378.390675 secs 2024-04-27T04:08:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:19 smithi119 ceph-mon[32752]: pgmap v18865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:19.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:19 smithi167 ceph-mon[32751]: pgmap v18865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:19 smithi090 ceph-mon[29180]: pgmap v18865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:21 smithi119 ceph-mon[32752]: pgmap v18866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34382.516255 secs 2024-04-27T04:08:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:21 smithi167 ceph-mon[32751]: pgmap v18866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34382.516255 secs 2024-04-27T04:08:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:21 smithi090 ceph-mon[29180]: pgmap v18866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34382.516255 secs 2024-04-27T04:08:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34383.390803 secs 2024-04-27T04:08:22.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34383.390803 secs 2024-04-27T04:08:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34383.390803 secs 2024-04-27T04:08:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:08:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:23 smithi167 ceph-mon[32751]: pgmap v18867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:08:23.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:08:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:08:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:23 smithi090 ceph-mon[29180]: pgmap v18867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:08:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:08:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:08:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:23 smithi119 ceph-mon[32752]: pgmap v18867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:08:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:08:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:08:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:24 smithi167 ceph-mon[32751]: pgmap v18868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:08:24.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:24 smithi167 ceph-mon[32751]: pgmap v18869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:08:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:24 smithi090 ceph-mon[29180]: pgmap v18868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:08:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:24 smithi090 ceph-mon[29180]: pgmap v18869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:08:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:24 smithi119 ceph-mon[32752]: pgmap v18868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:08:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:24 smithi119 ceph-mon[32752]: pgmap v18869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:08:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:26 smithi167 ceph-mon[32751]: pgmap v18870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:26 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34387.516340 secs 2024-04-27T04:08:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:26 smithi090 ceph-mon[29180]: pgmap v18870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:26 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34387.516340 secs 2024-04-27T04:08:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:26 smithi119 ceph-mon[32752]: pgmap v18870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:26 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34387.516340 secs 2024-04-27T04:08:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34388.390882 secs 2024-04-27T04:08:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34388.390882 secs 2024-04-27T04:08:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34388.390882 secs 2024-04-27T04:08:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:28 smithi090 ceph-mon[29180]: pgmap v18871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:28 smithi119 ceph-mon[32752]: pgmap v18871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:28 smithi167 ceph-mon[32751]: pgmap v18871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:30.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:30 smithi090 ceph-mon[29180]: pgmap v18872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:08:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:30 smithi119 ceph-mon[32752]: pgmap v18872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:08:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:30 smithi167 ceph-mon[32751]: pgmap v18872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:08:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:31 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34392.516419 secs 2024-04-27T04:08:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:31 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34392.516419 secs 2024-04-27T04:08:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:31 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34392.516419 secs 2024-04-27T04:08:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:32 smithi090 ceph-mon[29180]: pgmap v18873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34393.390958 secs 2024-04-27T04:08:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:32 smithi119 ceph-mon[32752]: pgmap v18873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34393.390958 secs 2024-04-27T04:08:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:32 smithi167 ceph-mon[32751]: pgmap v18873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:08:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34393.390958 secs 2024-04-27T04:08:34.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:34 smithi090 ceph-mon[29180]: pgmap v18874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:08:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:34 smithi119 ceph-mon[32752]: pgmap v18874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:08:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:34 smithi167 ceph-mon[32751]: pgmap v18874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:08:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:36 smithi090 ceph-mon[29180]: pgmap v18875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:36 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34397.516498 secs 2024-04-27T04:08:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:36 smithi119 ceph-mon[32752]: pgmap v18875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:36 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34397.516498 secs 2024-04-27T04:08:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:36 smithi167 ceph-mon[32751]: pgmap v18875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:36 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34397.516498 secs 2024-04-27T04:08:37.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34398.391099 secs 2024-04-27T04:08:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34398.391099 secs 2024-04-27T04:08:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34398.391099 secs 2024-04-27T04:08:38.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:38 smithi090 ceph-mon[29180]: pgmap v18876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:38 smithi119 ceph-mon[32752]: pgmap v18876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:38.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:38 smithi167 ceph-mon[32751]: pgmap v18876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:40.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:40 smithi090 ceph-mon[29180]: pgmap v18877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:40 smithi119 ceph-mon[32752]: pgmap v18877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:40 smithi167 ceph-mon[32751]: pgmap v18877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:41 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34402.516582 secs 2024-04-27T04:08:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:41 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34402.516582 secs 2024-04-27T04:08:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:41 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34402.516582 secs 2024-04-27T04:08:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:42 smithi090 ceph-mon[29180]: pgmap v18878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:08:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34403.391222 secs 2024-04-27T04:08:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:42 smithi119 ceph-mon[32752]: pgmap v18878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:08:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34403.391222 secs 2024-04-27T04:08:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:42 smithi167 ceph-mon[32751]: pgmap v18878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:08:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34403.391222 secs 2024-04-27T04:08:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:44 smithi090 ceph-mon[29180]: pgmap v18879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:44 smithi119 ceph-mon[32752]: pgmap v18879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:44 smithi167 ceph-mon[32751]: pgmap v18879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:46.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:46 smithi167 ceph-mon[32751]: pgmap v18880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:46.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:46 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34407.516665 secs 2024-04-27T04:08:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:46 smithi090 ceph-mon[29180]: pgmap v18880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:46 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34407.516665 secs 2024-04-27T04:08:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:46 smithi119 ceph-mon[32752]: pgmap v18880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:46 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34407.516665 secs 2024-04-27T04:08:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34408.391310 secs 2024-04-27T04:08:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34408.391310 secs 2024-04-27T04:08:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34408.391310 secs 2024-04-27T04:08:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:48 smithi090 ceph-mon[29180]: pgmap v18881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:48 smithi119 ceph-mon[32752]: pgmap v18881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:48 smithi167 ceph-mon[32751]: pgmap v18881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:50 smithi090 ceph-mon[29180]: pgmap v18882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:50 smithi119 ceph-mon[32752]: pgmap v18882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:50 smithi167 ceph-mon[32751]: pgmap v18882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:51.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:51 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34412.516750 secs 2024-04-27T04:08:51.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:51 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34412.516750 secs 2024-04-27T04:08:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:51 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34412.516750 secs 2024-04-27T04:08:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:52 smithi119 ceph-mon[32752]: pgmap v18883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:08:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34413.391438 secs 2024-04-27T04:08:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:52 smithi167 ceph-mon[32751]: pgmap v18883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:08:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34413.391438 secs 2024-04-27T04:08:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:52 smithi090 ceph-mon[29180]: pgmap v18883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:08:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34413.391438 secs 2024-04-27T04:08:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:54 smithi119 ceph-mon[32752]: pgmap v18884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:54 smithi167 ceph-mon[32751]: pgmap v18884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:54 smithi090 ceph-mon[29180]: pgmap v18884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:56 smithi119 ceph-mon[32752]: pgmap v18885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:56 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34417.516832 secs 2024-04-27T04:08:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:56 smithi167 ceph-mon[32751]: pgmap v18885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:56 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34417.516832 secs 2024-04-27T04:08:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:56 smithi090 ceph-mon[29180]: pgmap v18885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:08:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:56 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34417.516832 secs 2024-04-27T04:08:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34418.391505 secs 2024-04-27T04:08:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34418.391505 secs 2024-04-27T04:08:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34418.391505 secs 2024-04-27T04:08:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:08:58 smithi119 ceph-mon[32752]: pgmap v18886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:58.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:08:58 smithi167 ceph-mon[32751]: pgmap v18886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:08:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:08:58 smithi090 ceph-mon[29180]: pgmap v18886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:00 smithi119 ceph-mon[32752]: pgmap v18887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:00.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:00 smithi167 ceph-mon[32751]: pgmap v18887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:00 smithi090 ceph-mon[29180]: pgmap v18887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:01 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34422.516920 secs 2024-04-27T04:09:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:01 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34422.516920 secs 2024-04-27T04:09:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:01 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34422.516920 secs 2024-04-27T04:09:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:02 smithi119 ceph-mon[32752]: pgmap v18888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:09:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34423.391637 secs 2024-04-27T04:09:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:02 smithi167 ceph-mon[32751]: pgmap v18888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:09:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34423.391637 secs 2024-04-27T04:09:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:02 smithi090 ceph-mon[29180]: pgmap v18888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:09:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34423.391637 secs 2024-04-27T04:09:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:03 smithi119 ceph-mon[32752]: pgmap v18889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:03.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:03 smithi167 ceph-mon[32751]: pgmap v18889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:03 smithi090 ceph-mon[29180]: pgmap v18889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:05.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:05 smithi090 ceph-mon[29180]: pgmap v18890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:05.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:05 smithi119 ceph-mon[32752]: pgmap v18890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:05.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:05 smithi167 ceph-mon[32751]: pgmap v18890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:06 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34427.517001 secs 2024-04-27T04:09:06.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:06 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34427.517001 secs 2024-04-27T04:09:06.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:06 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34427.517001 secs 2024-04-27T04:09:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34428.391777 secs 2024-04-27T04:09:07.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:07 smithi090 ceph-mon[29180]: pgmap v18891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34428.391777 secs 2024-04-27T04:09:07.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:07 smithi119 ceph-mon[32752]: pgmap v18891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34428.391777 secs 2024-04-27T04:09:07.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:07 smithi167 ceph-mon[32751]: pgmap v18891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:09.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:09 smithi119 ceph-mon[32752]: pgmap v18892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:09:09.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:09 smithi167 ceph-mon[32751]: pgmap v18892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:09:09.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:09 smithi090 ceph-mon[29180]: pgmap v18892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:09:11.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:11 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34432.517110 secs 2024-04-27T04:09:11.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:11 smithi119 ceph-mon[32752]: pgmap v18893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:11 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34432.517110 secs 2024-04-27T04:09:11.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:11 smithi167 ceph-mon[32751]: pgmap v18893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:11 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34432.517110 secs 2024-04-27T04:09:11.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:11 smithi090 ceph-mon[29180]: pgmap v18893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:12.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34433.391931 secs 2024-04-27T04:09:12.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34433.391931 secs 2024-04-27T04:09:12.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34433.391931 secs 2024-04-27T04:09:13.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:13 smithi119 ceph-mon[32752]: pgmap v18894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:13.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:13 smithi167 ceph-mon[32751]: pgmap v18894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:13.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:13 smithi090 ceph-mon[29180]: pgmap v18894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:15.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:15 smithi119 ceph-mon[32752]: pgmap v18895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:15.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:15 smithi167 ceph-mon[32751]: pgmap v18895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:15.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:15 smithi090 ceph-mon[29180]: pgmap v18895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:16.704 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:16 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34437.517169 secs 2024-04-27T04:09:16.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:16 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34437.517169 secs 2024-04-27T04:09:16.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:16 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34437.517169 secs 2024-04-27T04:09:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34438.392062 secs 2024-04-27T04:09:17.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:17 smithi119 ceph-mon[32752]: pgmap v18896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34438.392062 secs 2024-04-27T04:09:17.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:17 smithi167 ceph-mon[32751]: pgmap v18896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34438.392062 secs 2024-04-27T04:09:17.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:17 smithi090 ceph-mon[29180]: pgmap v18896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:09:19.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:19 smithi119 ceph-mon[32752]: pgmap v18897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:09:19.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:19 smithi167 ceph-mon[32751]: pgmap v18897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:09:19.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:19 smithi090 ceph-mon[29180]: pgmap v18897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:09:21.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:21 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34442.517254 secs 2024-04-27T04:09:21.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:21 smithi119 ceph-mon[32752]: pgmap v18898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:09:21.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:21 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34442.517254 secs 2024-04-27T04:09:21.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:21 smithi167 ceph-mon[32751]: pgmap v18898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:09:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:21 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34442.517254 secs 2024-04-27T04:09:21.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:21 smithi090 ceph-mon[29180]: pgmap v18898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:09:22.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34443.392191 secs 2024-04-27T04:09:22.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34443.392191 secs 2024-04-27T04:09:22.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34443.392191 secs 2024-04-27T04:09:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:23 smithi119 ceph-mon[32752]: pgmap v18899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:09:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:09:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:09:23.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:23 smithi119 ceph-mon[32752]: pgmap v18900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:09:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:23 smithi119 ceph-mon[32752]: pgmap v18901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:09:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:09:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:23 smithi167 ceph-mon[32751]: pgmap v18899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:09:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:09:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:09:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:23 smithi167 ceph-mon[32751]: pgmap v18900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:09:23.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:23 smithi167 ceph-mon[32751]: pgmap v18901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:09:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:09:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:23 smithi090 ceph-mon[29180]: pgmap v18899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:09:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:09:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:09:23.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:23 smithi090 ceph-mon[29180]: pgmap v18900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:09:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:23 smithi090 ceph-mon[29180]: pgmap v18901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:09:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:09:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:26 smithi119 ceph-mon[32752]: pgmap v18902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:26 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34447.517337 secs 2024-04-27T04:09:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:26 smithi167 ceph-mon[32751]: pgmap v18902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:26 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34447.517337 secs 2024-04-27T04:09:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:26 smithi090 ceph-mon[29180]: pgmap v18902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:26 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34447.517337 secs 2024-04-27T04:09:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34448.392295 secs 2024-04-27T04:09:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34448.392295 secs 2024-04-27T04:09:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34448.392295 secs 2024-04-27T04:09:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:28 smithi119 ceph-mon[32752]: pgmap v18903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:09:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:28 smithi167 ceph-mon[32751]: pgmap v18903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:09:28.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:28 smithi090 ceph-mon[29180]: pgmap v18903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:09:30.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:30 smithi119 ceph-mon[32752]: pgmap v18904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:30 smithi167 ceph-mon[32751]: pgmap v18904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:30 smithi090 ceph-mon[29180]: pgmap v18904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:31 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34452.517419 secs 2024-04-27T04:09:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:31 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34452.517419 secs 2024-04-27T04:09:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:31 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34452.517419 secs 2024-04-27T04:09:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:32 smithi119 ceph-mon[32752]: pgmap v18905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34453.392420 secs 2024-04-27T04:09:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:32 smithi167 ceph-mon[32751]: pgmap v18905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34453.392420 secs 2024-04-27T04:09:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:32 smithi090 ceph-mon[29180]: pgmap v18905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:09:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34453.392420 secs 2024-04-27T04:09:34.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:34 smithi119 ceph-mon[32752]: pgmap v18906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:09:34.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:34 smithi167 ceph-mon[32751]: pgmap v18906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:09:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:34 smithi090 ceph-mon[29180]: pgmap v18906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:09:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:35 smithi119 ceph-mon[32752]: pgmap v18907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:35.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:35 smithi167 ceph-mon[32751]: pgmap v18907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:35 smithi090 ceph-mon[29180]: pgmap v18907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:36 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34457.517503 secs 2024-04-27T04:09:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:36 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34457.517503 secs 2024-04-27T04:09:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:36 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34457.517503 secs 2024-04-27T04:09:37.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34458.392537 secs 2024-04-27T04:09:37.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:37 smithi090 ceph-mon[29180]: pgmap v18908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34458.392537 secs 2024-04-27T04:09:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:37 smithi119 ceph-mon[32752]: pgmap v18908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34458.392537 secs 2024-04-27T04:09:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:37 smithi167 ceph-mon[32751]: pgmap v18908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:40 smithi119 ceph-mon[32752]: pgmap v18909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:40 smithi167 ceph-mon[32751]: pgmap v18909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:40 smithi090 ceph-mon[29180]: pgmap v18909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:09:40.625 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:09:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:09:40.301+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:09:40.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:09:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:09:40.301+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:09:40.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:09:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:09:40.301+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:09:40.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:09:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:09:40.301+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:09:40.626 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:09:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:09:40.301+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:09:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:41 smithi119 ceph-mon[32752]: 45 slow requests, 0 included below; oldest blocked for > 34462.517593 secs 2024-04-27T04:09:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:41 smithi167 ceph-mon[32751]: 45 slow requests, 0 included below; oldest blocked for > 34462.517593 secs 2024-04-27T04:09:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:41 smithi090 ceph-mon[29180]: 45 slow requests, 0 included below; oldest blocked for > 34462.517593 secs 2024-04-27T04:09:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:42 smithi119 ceph-mon[32752]: pgmap v18910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34463.392605 secs 2024-04-27T04:09:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:42 smithi167 ceph-mon[32751]: pgmap v18910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34463.392605 secs 2024-04-27T04:09:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:42 smithi090 ceph-mon[29180]: pgmap v18910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34463.392605 secs 2024-04-27T04:09:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:44 smithi119 ceph-mon[32752]: pgmap v18911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:44 smithi167 ceph-mon[32751]: pgmap v18911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:44 smithi090 ceph-mon[29180]: pgmap v18911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:46.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:46 smithi119 ceph-mon[32752]: pgmap v18912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:46 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34467.517677 secs 2024-04-27T04:09:46.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:46 smithi167 ceph-mon[32751]: pgmap v18912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:46 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34467.517677 secs 2024-04-27T04:09:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:46 smithi090 ceph-mon[29180]: pgmap v18912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:46 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34467.517677 secs 2024-04-27T04:09:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34468.392726 secs 2024-04-27T04:09:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34468.392726 secs 2024-04-27T04:09:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34468.392726 secs 2024-04-27T04:09:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:48 smithi119 ceph-mon[32752]: pgmap v18913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:48 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:09:48.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:48 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:09:48.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:48 smithi167 ceph-mon[32751]: pgmap v18913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:48 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:09:48.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:48 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:09:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:48 smithi090 ceph-mon[29180]: pgmap v18913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:48 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:09:48.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:48 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:09:49.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:49 smithi119 ceph-mon[32752]: pgmap v18914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:49 smithi167 ceph-mon[32751]: pgmap v18914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:49 smithi090 ceph-mon[29180]: pgmap v18914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:50 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34472.517762 secs 2024-04-27T04:09:51.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:50 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34472.517762 secs 2024-04-27T04:09:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:50 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34472.517762 secs 2024-04-27T04:09:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:51 smithi119 ceph-mon[32752]: pgmap v18915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:52.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34473.392805 secs 2024-04-27T04:09:52.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:51 smithi167 ceph-mon[32751]: pgmap v18915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34473.392805 secs 2024-04-27T04:09:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:51 smithi090 ceph-mon[29180]: pgmap v18915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34473.392805 secs 2024-04-27T04:09:54.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:54 smithi119 ceph-mon[32752]: pgmap v18916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:54.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:54 smithi167 ceph-mon[32751]: pgmap v18916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:54 smithi090 ceph-mon[29180]: pgmap v18916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:09:56.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:56 smithi119 ceph-mon[32752]: pgmap v18917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:56 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34477.517839 secs 2024-04-27T04:09:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:56 smithi167 ceph-mon[32751]: pgmap v18917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:56 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34477.517839 secs 2024-04-27T04:09:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:56 smithi090 ceph-mon[29180]: pgmap v18917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:09:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:56 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34477.517839 secs 2024-04-27T04:09:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34478.392901 secs 2024-04-27T04:09:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34478.392901 secs 2024-04-27T04:09:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34478.392901 secs 2024-04-27T04:09:58.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:09:58 smithi119 ceph-mon[32752]: pgmap v18918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:09:58 smithi167 ceph-mon[32751]: pgmap v18918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:09:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:09:58 smithi090 ceph-mon[29180]: pgmap v18918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:00 smithi119 ceph-mon[32752]: pgmap v18919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:10:00.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:00 smithi167 ceph-mon[32751]: pgmap v18919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:10:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:00 smithi090 ceph-mon[29180]: pgmap v18919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:10:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:01 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34482.517927 secs 2024-04-27T04:10:01.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:01 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34482.517927 secs 2024-04-27T04:10:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:01 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34482.517927 secs 2024-04-27T04:10:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:02 smithi119 ceph-mon[32752]: pgmap v18920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34483.393021 secs 2024-04-27T04:10:02.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:02 smithi167 ceph-mon[32751]: pgmap v18920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34483.393021 secs 2024-04-27T04:10:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:02 smithi090 ceph-mon[29180]: pgmap v18920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34483.393021 secs 2024-04-27T04:10:03.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:03 smithi119 ceph-mon[32752]: pgmap v18921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:03.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:03 smithi167 ceph-mon[32751]: pgmap v18921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:03.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:03 smithi090 ceph-mon[29180]: pgmap v18921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:06 smithi119 ceph-mon[32752]: pgmap v18922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:06 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34487.518013 secs 2024-04-27T04:10:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:06 smithi167 ceph-mon[32751]: pgmap v18922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:06 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34487.518013 secs 2024-04-27T04:10:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:06 smithi090 ceph-mon[29180]: pgmap v18922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:06 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34487.518013 secs 2024-04-27T04:10:07.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34488.393138 secs 2024-04-27T04:10:07.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34488.393138 secs 2024-04-27T04:10:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34488.393138 secs 2024-04-27T04:10:08.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:08 smithi119 ceph-mon[32752]: pgmap v18923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:08.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:08 smithi167 ceph-mon[32751]: pgmap v18923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:08 smithi090 ceph-mon[29180]: pgmap v18923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:10.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:10 smithi119 ceph-mon[32752]: pgmap v18924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:10.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:10 smithi167 ceph-mon[32751]: pgmap v18924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:10 smithi090 ceph-mon[29180]: pgmap v18924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:11.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:11 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34492.518107 secs 2024-04-27T04:10:11.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:11 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34492.518107 secs 2024-04-27T04:10:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:11 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34492.518107 secs 2024-04-27T04:10:12.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:12 smithi119 ceph-mon[32752]: pgmap v18925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34493.393249 secs 2024-04-27T04:10:12.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:12 smithi167 ceph-mon[32751]: pgmap v18925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34493.393249 secs 2024-04-27T04:10:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:12 smithi090 ceph-mon[29180]: pgmap v18925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34493.393249 secs 2024-04-27T04:10:13.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:13 smithi119 ceph-mon[32752]: pgmap v18926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:13.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:13 smithi167 ceph-mon[32751]: pgmap v18926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:13.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:13 smithi090 ceph-mon[29180]: pgmap v18926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:16.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:16 smithi119 ceph-mon[32752]: pgmap v18927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:16 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34497.518189 secs 2024-04-27T04:10:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:16 smithi167 ceph-mon[32751]: pgmap v18927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:16 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34497.518189 secs 2024-04-27T04:10:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:16 smithi090 ceph-mon[29180]: pgmap v18927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:16 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34497.518189 secs 2024-04-27T04:10:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34498.393366 secs 2024-04-27T04:10:17.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34498.393366 secs 2024-04-27T04:10:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34498.393366 secs 2024-04-27T04:10:18.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:18 smithi119 ceph-mon[32752]: pgmap v18928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:18.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:18 smithi167 ceph-mon[32751]: pgmap v18928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:18 smithi090 ceph-mon[29180]: pgmap v18928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:20.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:20 smithi090 ceph-mon[29180]: pgmap v18929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:20.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:20 smithi119 ceph-mon[32752]: pgmap v18929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:20.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:20 smithi167 ceph-mon[32751]: pgmap v18929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:21.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:21 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34502.518279 secs 2024-04-27T04:10:21.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:21 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34502.518279 secs 2024-04-27T04:10:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:21 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34502.518279 secs 2024-04-27T04:10:22.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:22 smithi119 ceph-mon[32752]: pgmap v18930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34503.393513 secs 2024-04-27T04:10:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:22 smithi167 ceph-mon[32751]: pgmap v18930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:22.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34503.393513 secs 2024-04-27T04:10:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:22 smithi090 ceph-mon[29180]: pgmap v18930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34503.393513 secs 2024-04-27T04:10:23.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:23 smithi119 ceph-mon[32752]: pgmap v18931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:23.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:23 smithi167 ceph-mon[32751]: pgmap v18931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:23 smithi090 ceph-mon[29180]: pgmap v18931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:24.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:10:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:10:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:10:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:24 smithi119 ceph-mon[32752]: pgmap v18932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:10:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:24 smithi119 ceph-mon[32752]: pgmap v18933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:10:24.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:10:24.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:10:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:10:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:24 smithi167 ceph-mon[32751]: pgmap v18932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:10:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:24 smithi167 ceph-mon[32751]: pgmap v18933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:10:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:10:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:10:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:10:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:24 smithi090 ceph-mon[29180]: pgmap v18932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:10:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:24 smithi090 ceph-mon[29180]: pgmap v18933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:24.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:10:26.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:25 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34507.518361 secs 2024-04-27T04:10:26.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:25 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34507.518361 secs 2024-04-27T04:10:26.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:25 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34507.518361 secs 2024-04-27T04:10:27.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:26 smithi119 ceph-mon[32752]: pgmap v18934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34508.393637 secs 2024-04-27T04:10:27.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:26 smithi167 ceph-mon[32751]: pgmap v18934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:27.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34508.393637 secs 2024-04-27T04:10:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:26 smithi090 ceph-mon[29180]: pgmap v18934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34508.393637 secs 2024-04-27T04:10:29.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:28 smithi119 ceph-mon[32752]: pgmap v18935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:10:29.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:28 smithi167 ceph-mon[32751]: pgmap v18935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:10:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:28 smithi090 ceph-mon[29180]: pgmap v18935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:10:31.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:30 smithi119 ceph-mon[32752]: pgmap v18936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:31.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:30 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34512.518420 secs 2024-04-27T04:10:31.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:30 smithi167 ceph-mon[32751]: pgmap v18936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:31.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:30 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34512.518420 secs 2024-04-27T04:10:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:30 smithi090 ceph-mon[29180]: pgmap v18936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:30 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34512.518420 secs 2024-04-27T04:10:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34513.393701 secs 2024-04-27T04:10:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34513.393701 secs 2024-04-27T04:10:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34513.393701 secs 2024-04-27T04:10:33.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:32 smithi119 ceph-mon[32752]: pgmap v18937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:32 smithi167 ceph-mon[32751]: pgmap v18937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:32 smithi090 ceph-mon[29180]: pgmap v18937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:10:35.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:34 smithi119 ceph-mon[32752]: pgmap v18938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:10:35.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:34 smithi167 ceph-mon[32751]: pgmap v18938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:10:35.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:34 smithi090 ceph-mon[29180]: pgmap v18938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:10:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:35 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34517.518503 secs 2024-04-27T04:10:36.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:35 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34517.518503 secs 2024-04-27T04:10:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:35 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34517.518503 secs 2024-04-27T04:10:37.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:36 smithi090 ceph-mon[29180]: pgmap v18939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:37.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34518.393796 secs 2024-04-27T04:10:37.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:36 smithi119 ceph-mon[32752]: pgmap v18939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34518.393796 secs 2024-04-27T04:10:37.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:36 smithi167 ceph-mon[32751]: pgmap v18939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34518.393796 secs 2024-04-27T04:10:39.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:38 smithi119 ceph-mon[32752]: pgmap v18940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:39.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:38 smithi167 ceph-mon[32751]: pgmap v18940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:39.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:38 smithi090 ceph-mon[29180]: pgmap v18940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:41.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:40 smithi119 ceph-mon[32752]: pgmap v18941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:41.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:40 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34522.518563 secs 2024-04-27T04:10:41.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:40 smithi167 ceph-mon[32751]: pgmap v18941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:41.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:40 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34522.518563 secs 2024-04-27T04:10:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:40 smithi090 ceph-mon[29180]: pgmap v18941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:40 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34522.518563 secs 2024-04-27T04:10:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34523.393915 secs 2024-04-27T04:10:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34523.393915 secs 2024-04-27T04:10:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34523.393915 secs 2024-04-27T04:10:43.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:42 smithi119 ceph-mon[32752]: pgmap v18942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:43.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:42 smithi167 ceph-mon[32751]: pgmap v18942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:43.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:42 smithi090 ceph-mon[29180]: pgmap v18942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:45.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:44 smithi119 ceph-mon[32752]: pgmap v18943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:45.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:44 smithi167 ceph-mon[32751]: pgmap v18943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:45.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:44 smithi090 ceph-mon[29180]: pgmap v18943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:46.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:45 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34527.518643 secs 2024-04-27T04:10:46.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:45 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34527.518643 secs 2024-04-27T04:10:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:45 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34527.518643 secs 2024-04-27T04:10:47.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:46 smithi119 ceph-mon[32752]: pgmap v18944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34528.394038 secs 2024-04-27T04:10:47.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:46 smithi167 ceph-mon[32751]: pgmap v18944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:47.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34528.394038 secs 2024-04-27T04:10:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:46 smithi090 ceph-mon[29180]: pgmap v18944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34528.394038 secs 2024-04-27T04:10:49.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:48 smithi119 ceph-mon[32752]: pgmap v18945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:49.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:48 smithi167 ceph-mon[32751]: pgmap v18945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:49.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:48 smithi090 ceph-mon[29180]: pgmap v18945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:50 smithi119 ceph-mon[32752]: pgmap v18946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:50 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34532.518726 secs 2024-04-27T04:10:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:50 smithi167 ceph-mon[32751]: pgmap v18946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:50 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34532.518726 secs 2024-04-27T04:10:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:50 smithi090 ceph-mon[29180]: pgmap v18946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:50 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34532.518726 secs 2024-04-27T04:10:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34533.394122 secs 2024-04-27T04:10:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34533.394122 secs 2024-04-27T04:10:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34533.394122 secs 2024-04-27T04:10:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:52 smithi119 ceph-mon[32752]: pgmap v18947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:53.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:52 smithi167 ceph-mon[32751]: pgmap v18947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:52 smithi090 ceph-mon[29180]: pgmap v18947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:55.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:54 smithi119 ceph-mon[32752]: pgmap v18948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:55.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:54 smithi167 ceph-mon[32751]: pgmap v18948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:54 smithi090 ceph-mon[29180]: pgmap v18948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:10:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:55 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34537.518810 secs 2024-04-27T04:10:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:55 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34537.518810 secs 2024-04-27T04:10:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:55 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34537.518810 secs 2024-04-27T04:10:57.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:56 smithi119 ceph-mon[32752]: pgmap v18949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34538.394244 secs 2024-04-27T04:10:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:56 smithi167 ceph-mon[32751]: pgmap v18949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34538.394244 secs 2024-04-27T04:10:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:56 smithi090 ceph-mon[29180]: pgmap v18949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34538.394244 secs 2024-04-27T04:10:59.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:10:58 smithi119 ceph-mon[32752]: pgmap v18950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:59.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:10:58 smithi167 ceph-mon[32751]: pgmap v18950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:10:59.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:10:58 smithi090 ceph-mon[29180]: pgmap v18950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:01.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:00 smithi119 ceph-mon[32752]: pgmap v18951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:00 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34542.518886 secs 2024-04-27T04:11:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:00 smithi167 ceph-mon[32751]: pgmap v18951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:00 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34542.518886 secs 2024-04-27T04:11:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:00 smithi090 ceph-mon[29180]: pgmap v18951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:00 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34542.518886 secs 2024-04-27T04:11:02.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34543.394357 secs 2024-04-27T04:11:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34543.394357 secs 2024-04-27T04:11:02.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34543.394357 secs 2024-04-27T04:11:03.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:02 smithi119 ceph-mon[32752]: pgmap v18952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:03.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:02 smithi167 ceph-mon[32751]: pgmap v18952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:03.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:02 smithi090 ceph-mon[29180]: pgmap v18952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:05.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:05 smithi119 ceph-mon[32752]: pgmap v18953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:05.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:05 smithi167 ceph-mon[32751]: pgmap v18953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:05.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:05 smithi090 ceph-mon[29180]: pgmap v18953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:06.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:06 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34547.518967 secs 2024-04-27T04:11:06.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:06 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34547.518967 secs 2024-04-27T04:11:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:06 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34547.518967 secs 2024-04-27T04:11:07.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:07 smithi090 ceph-mon[29180]: pgmap v18954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:07.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34548.394482 secs 2024-04-27T04:11:07.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:07 smithi119 ceph-mon[32752]: pgmap v18954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34548.394482 secs 2024-04-27T04:11:07.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:07 smithi167 ceph-mon[32751]: pgmap v18954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34548.394482 secs 2024-04-27T04:11:09.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:09 smithi119 ceph-mon[32752]: pgmap v18955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:09.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:09 smithi167 ceph-mon[32751]: pgmap v18955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:09.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:09 smithi090 ceph-mon[29180]: pgmap v18955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:11.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:11 smithi119 ceph-mon[32752]: pgmap v18956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:11 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34552.519041 secs 2024-04-27T04:11:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:11 smithi167 ceph-mon[32751]: pgmap v18956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:11 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34552.519041 secs 2024-04-27T04:11:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:11 smithi090 ceph-mon[29180]: pgmap v18956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:11 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34552.519041 secs 2024-04-27T04:11:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34553.394598 secs 2024-04-27T04:11:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34553.394598 secs 2024-04-27T04:11:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34553.394598 secs 2024-04-27T04:11:13.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:13 smithi119 ceph-mon[32752]: pgmap v18957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:13.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:13 smithi167 ceph-mon[32751]: pgmap v18957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:13 smithi090 ceph-mon[29180]: pgmap v18957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:15.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:15 smithi119 ceph-mon[32752]: pgmap v18958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:15.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:15 smithi167 ceph-mon[32751]: pgmap v18958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:15.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:15 smithi090 ceph-mon[29180]: pgmap v18958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:16 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34557.519124 secs 2024-04-27T04:11:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:16 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34557.519124 secs 2024-04-27T04:11:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:16 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34557.519124 secs 2024-04-27T04:11:17.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:17 smithi119 ceph-mon[32752]: pgmap v18959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:17.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34558.394738 secs 2024-04-27T04:11:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:17 smithi167 ceph-mon[32751]: pgmap v18959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34558.394738 secs 2024-04-27T04:11:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:17 smithi090 ceph-mon[29180]: pgmap v18959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34558.394738 secs 2024-04-27T04:11:19.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:19 smithi119 ceph-mon[32752]: pgmap v18960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:19.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:19 smithi167 ceph-mon[32751]: pgmap v18960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:19.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:19 smithi090 ceph-mon[29180]: pgmap v18960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:21.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:21 smithi119 ceph-mon[32752]: pgmap v18961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:21 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34562.519206 secs 2024-04-27T04:11:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:21 smithi167 ceph-mon[32751]: pgmap v18961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:21.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:21 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34562.519206 secs 2024-04-27T04:11:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:21 smithi090 ceph-mon[29180]: pgmap v18961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:21 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34562.519206 secs 2024-04-27T04:11:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34563.394859 secs 2024-04-27T04:11:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34563.394859 secs 2024-04-27T04:11:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34563.394859 secs 2024-04-27T04:11:23.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:23 smithi119 ceph-mon[32752]: pgmap v18962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:23.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:23 smithi167 ceph-mon[32751]: pgmap v18962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:23.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:23 smithi090 ceph-mon[29180]: pgmap v18962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:24.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:24.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:24.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:11:25.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:25 smithi119 ceph-mon[32752]: pgmap v18963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:11:25.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:11:25.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:25 smithi167 ceph-mon[32751]: pgmap v18963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:25.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:25.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:11:25.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:11:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:25 smithi090 ceph-mon[29180]: pgmap v18963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:11:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:11:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:11:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:26 smithi167 ceph-mon[32751]: pgmap v18964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:26 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34567.519290 secs 2024-04-27T04:11:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:26 smithi090 ceph-mon[29180]: pgmap v18964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:26 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34567.519290 secs 2024-04-27T04:11:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:26 smithi119 ceph-mon[32752]: pgmap v18964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:26 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34567.519290 secs 2024-04-27T04:11:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34568.394986 secs 2024-04-27T04:11:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34568.394986 secs 2024-04-27T04:11:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34568.394986 secs 2024-04-27T04:11:28.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:28 smithi167 ceph-mon[32751]: pgmap v18965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:11:28.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:28 smithi090 ceph-mon[29180]: pgmap v18965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:11:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:28 smithi119 ceph-mon[32752]: pgmap v18965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:11:30.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:30 smithi167 ceph-mon[32751]: pgmap v18966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:11:30.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:30 smithi090 ceph-mon[29180]: pgmap v18966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:11:30.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:30 smithi119 ceph-mon[32752]: pgmap v18966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:11:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:31 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34572.519373 secs 2024-04-27T04:11:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:31 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34572.519373 secs 2024-04-27T04:11:31.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:31 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34572.519373 secs 2024-04-27T04:11:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:32 smithi090 ceph-mon[29180]: pgmap v18967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:32.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34573.395098 secs 2024-04-27T04:11:32.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:32 smithi119 ceph-mon[32752]: pgmap v18967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34573.395098 secs 2024-04-27T04:11:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:32 smithi167 ceph-mon[32751]: pgmap v18967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34573.395098 secs 2024-04-27T04:11:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:34 smithi090 ceph-mon[29180]: pgmap v18968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:34.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:34 smithi119 ceph-mon[32752]: pgmap v18968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:34.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:34 smithi167 ceph-mon[32751]: pgmap v18968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:36 smithi090 ceph-mon[29180]: pgmap v18969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:36 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34577.519433 secs 2024-04-27T04:11:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:36 smithi119 ceph-mon[32752]: pgmap v18969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:36 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34577.519433 secs 2024-04-27T04:11:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:36 smithi167 ceph-mon[32751]: pgmap v18969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:11:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:36 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34577.519433 secs 2024-04-27T04:11:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34578.395189 secs 2024-04-27T04:11:37.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34578.395189 secs 2024-04-27T04:11:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34578.395189 secs 2024-04-27T04:11:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:38 smithi090 ceph-mon[29180]: pgmap v18970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:38.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:38 smithi119 ceph-mon[32752]: pgmap v18970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:38.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:38 smithi167 ceph-mon[32751]: pgmap v18970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:40 smithi090 ceph-mon[29180]: pgmap v18971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:40.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:40 smithi119 ceph-mon[32752]: pgmap v18971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:40.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:40 smithi167 ceph-mon[32751]: pgmap v18971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:41.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:41 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34582.519515 secs 2024-04-27T04:11:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:41 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34582.519515 secs 2024-04-27T04:11:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:41 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34582.519515 secs 2024-04-27T04:11:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:42 smithi090 ceph-mon[29180]: pgmap v18972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34583.395315 secs 2024-04-27T04:11:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:42 smithi119 ceph-mon[32752]: pgmap v18972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34583.395315 secs 2024-04-27T04:11:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:42 smithi167 ceph-mon[32751]: pgmap v18972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34583.395315 secs 2024-04-27T04:11:44.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:44 smithi090 ceph-mon[29180]: pgmap v18973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:44.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:44 smithi119 ceph-mon[32752]: pgmap v18973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:44 smithi167 ceph-mon[32751]: pgmap v18973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:46.354 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:46 smithi119 ceph-mon[32752]: pgmap v18974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:46.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:46 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34587.519597 secs 2024-04-27T04:11:46.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:46 smithi167 ceph-mon[32751]: pgmap v18974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:46.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:46 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34587.519597 secs 2024-04-27T04:11:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:46 smithi090 ceph-mon[29180]: pgmap v18974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:46 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34587.519597 secs 2024-04-27T04:11:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34588.395443 secs 2024-04-27T04:11:47.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34588.395443 secs 2024-04-27T04:11:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34588.395443 secs 2024-04-27T04:11:48.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:48 smithi090 ceph-mon[29180]: pgmap v18975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:48.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:48 smithi119 ceph-mon[32752]: pgmap v18975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:48.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:48 smithi167 ceph-mon[32751]: pgmap v18975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:50.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:50 smithi090 ceph-mon[29180]: pgmap v18976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:50.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:50 smithi119 ceph-mon[32752]: pgmap v18976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:50.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:50 smithi167 ceph-mon[32751]: pgmap v18976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:51 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34592.519682 secs 2024-04-27T04:11:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:51 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34592.519682 secs 2024-04-27T04:11:51.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:51 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34592.519682 secs 2024-04-27T04:11:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:52 smithi090 ceph-mon[29180]: pgmap v18977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34593.395540 secs 2024-04-27T04:11:52.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:52 smithi119 ceph-mon[32752]: pgmap v18977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34593.395540 secs 2024-04-27T04:11:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:52 smithi167 ceph-mon[32751]: pgmap v18977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34593.395540 secs 2024-04-27T04:11:54.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:54 smithi090 ceph-mon[29180]: pgmap v18978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:54.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:54 smithi119 ceph-mon[32752]: pgmap v18978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:54.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:54 smithi167 ceph-mon[32751]: pgmap v18978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:56.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:56 smithi119 ceph-mon[32752]: pgmap v18979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:56 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34597.519766 secs 2024-04-27T04:11:56.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:56 smithi167 ceph-mon[32751]: pgmap v18979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:56.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:56 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34597.519766 secs 2024-04-27T04:11:56.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:56 smithi090 ceph-mon[29180]: pgmap v18979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:11:56.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:56 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34597.519766 secs 2024-04-27T04:11:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34598.395652 secs 2024-04-27T04:11:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34598.395652 secs 2024-04-27T04:11:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34598.395652 secs 2024-04-27T04:11:58.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:11:58 smithi119 ceph-mon[32752]: pgmap v18980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:58.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:11:58 smithi167 ceph-mon[32751]: pgmap v18980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:11:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:11:58 smithi090 ceph-mon[29180]: pgmap v18980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:00.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:00 smithi119 ceph-mon[32752]: pgmap v18981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:00.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:00 smithi167 ceph-mon[32751]: pgmap v18981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:00 smithi090 ceph-mon[29180]: pgmap v18981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:01.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:01 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34602.519848 secs 2024-04-27T04:12:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:01 smithi119 ceph-mon[32752]: pgmap v18982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:01.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34603.395752 secs 2024-04-27T04:12:01.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:01 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34602.519848 secs 2024-04-27T04:12:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:01 smithi167 ceph-mon[32751]: pgmap v18982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:01.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34603.395752 secs 2024-04-27T04:12:01.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:01 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34602.519848 secs 2024-04-27T04:12:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:01 smithi090 ceph-mon[29180]: pgmap v18982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:01.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34603.395752 secs 2024-04-27T04:12:04.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:03 smithi119 ceph-mon[32752]: pgmap v18983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:04.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:03 smithi167 ceph-mon[32751]: pgmap v18983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:04.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:03 smithi090 ceph-mon[29180]: pgmap v18983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:06.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:05 smithi119 ceph-mon[32752]: pgmap v18984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:05 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34607.519931 secs 2024-04-27T04:12:06.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:05 smithi167 ceph-mon[32751]: pgmap v18984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:06.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:05 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34607.519931 secs 2024-04-27T04:12:06.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:05 smithi090 ceph-mon[29180]: pgmap v18984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:05 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34607.519931 secs 2024-04-27T04:12:07.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34608.395893 secs 2024-04-27T04:12:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34608.395893 secs 2024-04-27T04:12:07.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34608.395893 secs 2024-04-27T04:12:08.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:07 smithi119 ceph-mon[32752]: pgmap v18985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:08.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:07 smithi167 ceph-mon[32751]: pgmap v18985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:07 smithi090 ceph-mon[29180]: pgmap v18985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:09.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:09 smithi090 ceph-mon[29180]: pgmap v18986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:10.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:09 smithi119 ceph-mon[32752]: pgmap v18986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:10.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:09 smithi167 ceph-mon[32751]: pgmap v18986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:11.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:10 smithi119 ceph-mon[32752]: 17 slow requests, 1 included below; oldest blocked for > 34612.520016 secs 2024-04-27T04:12:11.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:10 smithi119 ceph-mon[32752]: slow request 1440.306466 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:12:11.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:10 smithi167 ceph-mon[32751]: 17 slow requests, 1 included below; oldest blocked for > 34612.520016 secs 2024-04-27T04:12:11.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:10 smithi167 ceph-mon[32751]: slow request 1440.306466 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:12:11.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:10 smithi090 ceph-mon[29180]: 17 slow requests, 1 included below; oldest blocked for > 34612.520016 secs 2024-04-27T04:12:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:10 smithi090 ceph-mon[29180]: slow request 1440.306466 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:12:12.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:11 smithi119 ceph-mon[32752]: pgmap v18987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34613.396013 secs 2024-04-27T04:12:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:11 smithi167 ceph-mon[32751]: pgmap v18987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34613.396013 secs 2024-04-27T04:12:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:11 smithi090 ceph-mon[29180]: pgmap v18987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34613.396013 secs 2024-04-27T04:12:14.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:13 smithi119 ceph-mon[32752]: pgmap v18988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:14.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:13 smithi167 ceph-mon[32751]: pgmap v18988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:14.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:13 smithi090 ceph-mon[29180]: pgmap v18988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:16.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:15 smithi119 ceph-mon[32752]: pgmap v18989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:16.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:15 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34617.520102 secs 2024-04-27T04:12:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:15 smithi167 ceph-mon[32751]: pgmap v18989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:16.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:15 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34617.520102 secs 2024-04-27T04:12:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:15 smithi090 ceph-mon[29180]: pgmap v18989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:16.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:15 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34617.520102 secs 2024-04-27T04:12:17.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34618.396087 secs 2024-04-27T04:12:17.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34618.396087 secs 2024-04-27T04:12:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34618.396087 secs 2024-04-27T04:12:18.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:17 smithi119 ceph-mon[32752]: pgmap v18990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:18.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:17 smithi167 ceph-mon[32751]: pgmap v18990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:18.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:17 smithi090 ceph-mon[29180]: pgmap v18990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:20.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:19 smithi119 ceph-mon[32752]: pgmap v18991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:20.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:19 smithi167 ceph-mon[32751]: pgmap v18991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:20.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:19 smithi090 ceph-mon[29180]: pgmap v18991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:21.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:20 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34622.520162 secs 2024-04-27T04:12:21.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:20 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34622.520162 secs 2024-04-27T04:12:21.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:20 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34622.520162 secs 2024-04-27T04:12:22.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:21 smithi119 ceph-mon[32752]: pgmap v18992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:22.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34623.396207 secs 2024-04-27T04:12:22.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:21 smithi167 ceph-mon[32751]: pgmap v18992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:22.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34623.396207 secs 2024-04-27T04:12:22.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:21 smithi090 ceph-mon[29180]: pgmap v18992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:22.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34623.396207 secs 2024-04-27T04:12:24.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:23 smithi119 ceph-mon[32752]: pgmap v18993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:24.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:23 smithi167 ceph-mon[32751]: pgmap v18993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:24.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:23 smithi090 ceph-mon[29180]: pgmap v18993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:25.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:25.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:25.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:12:26.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:25 smithi119 ceph-mon[32752]: pgmap v18994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:26.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:25 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34627.520228 secs 2024-04-27T04:12:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:26.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:12:26.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:25 smithi167 ceph-mon[32751]: pgmap v18994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:26.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:25 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34627.520228 secs 2024-04-27T04:12:26.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:12:26.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:25 smithi090 ceph-mon[29180]: pgmap v18994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:25 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34627.520228 secs 2024-04-27T04:12:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:12:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:12:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:12:27.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:26 smithi119 ceph-mon[32752]: pgmap v18995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:12:27.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:26 smithi119 ceph-mon[32752]: pgmap v18996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:27.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34628.396302 secs 2024-04-27T04:12:27.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:26 smithi167 ceph-mon[32751]: pgmap v18995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:12:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:26 smithi167 ceph-mon[32751]: pgmap v18996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34628.396302 secs 2024-04-27T04:12:27.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:26 smithi090 ceph-mon[29180]: pgmap v18995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:12:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:26 smithi090 ceph-mon[29180]: pgmap v18996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:27.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34628.396302 secs 2024-04-27T04:12:29.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:28 smithi119 ceph-mon[32752]: pgmap v18997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:12:29.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:28 smithi167 ceph-mon[32751]: pgmap v18997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:12:29.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:28 smithi090 ceph-mon[29180]: pgmap v18997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:12:31.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:30 smithi119 ceph-mon[32752]: pgmap v18998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:31.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:30 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34632.520311 secs 2024-04-27T04:12:31.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:30 smithi167 ceph-mon[32751]: pgmap v18998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:31.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:30 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34632.520311 secs 2024-04-27T04:12:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:30 smithi090 ceph-mon[29180]: pgmap v18998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:30 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34632.520311 secs 2024-04-27T04:12:32.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34633.396461 secs 2024-04-27T04:12:32.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34633.396461 secs 2024-04-27T04:12:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34633.396461 secs 2024-04-27T04:12:33.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:32 smithi119 ceph-mon[32752]: pgmap v18999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:33.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:32 smithi167 ceph-mon[32751]: pgmap v18999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:32 smithi090 ceph-mon[29180]: pgmap v18999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:35.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:34 smithi119 ceph-mon[32752]: pgmap v19000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:35.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:34 smithi167 ceph-mon[32751]: pgmap v19000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:35.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:34 smithi090 ceph-mon[29180]: pgmap v19000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:12:36.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:35 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34637.520395 secs 2024-04-27T04:12:36.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:35 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34637.520395 secs 2024-04-27T04:12:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:35 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34637.520395 secs 2024-04-27T04:12:37.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:36 smithi119 ceph-mon[32752]: pgmap v19001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:12:37.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34638.396587 secs 2024-04-27T04:12:37.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:36 smithi167 ceph-mon[32751]: pgmap v19001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:12:37.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34638.396587 secs 2024-04-27T04:12:37.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:36 smithi090 ceph-mon[29180]: pgmap v19001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:12:37.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34638.396587 secs 2024-04-27T04:12:39.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:38 smithi119 ceph-mon[32752]: pgmap v19002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:39.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:38 smithi167 ceph-mon[32751]: pgmap v19002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:39.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:38 smithi090 ceph-mon[29180]: pgmap v19002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:41.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:40 smithi119 ceph-mon[32752]: pgmap v19003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:41.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:40 smithi119 ceph-mon[32752]: 17 slow requests, 0 included below; oldest blocked for > 34642.520474 secs 2024-04-27T04:12:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:40 smithi167 ceph-mon[32751]: pgmap v19003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:41.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:40 smithi167 ceph-mon[32751]: 17 slow requests, 0 included below; oldest blocked for > 34642.520474 secs 2024-04-27T04:12:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:40 smithi090 ceph-mon[29180]: pgmap v19003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:41.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:40 smithi090 ceph-mon[29180]: 17 slow requests, 0 included below; oldest blocked for > 34642.520474 secs 2024-04-27T04:12:42.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34643.396694 secs 2024-04-27T04:12:42.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34643.396694 secs 2024-04-27T04:12:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34643.396694 secs 2024-04-27T04:12:43.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:42 smithi167 ceph-mon[32751]: pgmap v19004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:43.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:42 smithi090 ceph-mon[29180]: pgmap v19004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:43.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:42 smithi119 ceph-mon[32752]: pgmap v19004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:45.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:44 smithi167 ceph-mon[32751]: pgmap v19005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:45.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:44 smithi090 ceph-mon[29180]: pgmap v19005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:45.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:44 smithi119 ceph-mon[32752]: pgmap v19005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:46.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:46 smithi119 ceph-mon[32752]: 46 slow requests, 1 included below; oldest blocked for > 34647.520554 secs 2024-04-27T04:12:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:46 smithi119 ceph-mon[32752]: slow request 184.967157 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:12:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:46 smithi119 ceph-mon[32752]: pgmap v19006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:46.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34648.396813 secs 2024-04-27T04:12:46.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:46 smithi167 ceph-mon[32751]: 46 slow requests, 1 included below; oldest blocked for > 34647.520554 secs 2024-04-27T04:12:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:46 smithi167 ceph-mon[32751]: slow request 184.967157 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:12:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:46 smithi167 ceph-mon[32751]: pgmap v19006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:46.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34648.396813 secs 2024-04-27T04:12:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:46 smithi090 ceph-mon[29180]: 46 slow requests, 1 included below; oldest blocked for > 34647.520554 secs 2024-04-27T04:12:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:46 smithi090 ceph-mon[29180]: slow request 184.967157 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:12:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:46 smithi090 ceph-mon[29180]: pgmap v19006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34648.396813 secs 2024-04-27T04:12:48.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:12:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:12:48.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:12:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:12:48.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:12:48.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:12:49.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:48 smithi090 ceph-mon[29180]: pgmap v19007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:49.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:48 smithi119 ceph-mon[32752]: pgmap v19007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:49.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:48 smithi167 ceph-mon[32751]: pgmap v19007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:51.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:50 smithi090 ceph-mon[29180]: pgmap v19008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:51.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:50 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34652.520638 secs 2024-04-27T04:12:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:50 smithi119 ceph-mon[32752]: pgmap v19008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:50 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34652.520638 secs 2024-04-27T04:12:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:50 smithi167 ceph-mon[32751]: pgmap v19008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:50 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34652.520638 secs 2024-04-27T04:12:52.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34653.396908 secs 2024-04-27T04:12:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34653.396908 secs 2024-04-27T04:12:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34653.396908 secs 2024-04-27T04:12:53.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:52 smithi090 ceph-mon[29180]: pgmap v19009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:53.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:52 smithi119 ceph-mon[32752]: pgmap v19009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:53.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:52 smithi167 ceph-mon[32751]: pgmap v19009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:54 smithi090 ceph-mon[29180]: pgmap v19010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:55.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:54 smithi119 ceph-mon[32752]: pgmap v19010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:54 smithi167 ceph-mon[32751]: pgmap v19010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:12:56.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:55 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34657.520694 secs 2024-04-27T04:12:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:55 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34657.520694 secs 2024-04-27T04:12:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:55 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34657.520694 secs 2024-04-27T04:12:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:56 smithi090 ceph-mon[29180]: pgmap v19011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34658.397030 secs 2024-04-27T04:12:57.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:56 smithi119 ceph-mon[32752]: pgmap v19011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34658.397030 secs 2024-04-27T04:12:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:56 smithi167 ceph-mon[32751]: pgmap v19011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34658.397030 secs 2024-04-27T04:12:59.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:12:58 smithi090 ceph-mon[29180]: pgmap v19012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:59.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:12:58 smithi119 ceph-mon[32752]: pgmap v19012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:12:59.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:12:58 smithi167 ceph-mon[32751]: pgmap v19012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:01.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:00 smithi090 ceph-mon[29180]: pgmap v19013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:01.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:00 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34662.520779 secs 2024-04-27T04:13:01.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:00 smithi119 ceph-mon[32752]: pgmap v19013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:00 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34662.520779 secs 2024-04-27T04:13:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:00 smithi167 ceph-mon[32751]: pgmap v19013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:00 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34662.520779 secs 2024-04-27T04:13:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34663.397182 secs 2024-04-27T04:13:02.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34663.397182 secs 2024-04-27T04:13:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34663.397182 secs 2024-04-27T04:13:03.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:02 smithi090 ceph-mon[29180]: pgmap v19014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:03.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:02 smithi119 ceph-mon[32752]: pgmap v19014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:03.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:02 smithi167 ceph-mon[32751]: pgmap v19014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:05.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:04 smithi090 ceph-mon[29180]: pgmap v19015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:05.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:04 smithi119 ceph-mon[32752]: pgmap v19015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:05.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:04 smithi167 ceph-mon[32751]: pgmap v19015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:06.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:05 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34667.520860 secs 2024-04-27T04:13:06.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:05 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34667.520860 secs 2024-04-27T04:13:06.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:05 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34667.520860 secs 2024-04-27T04:13:07.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:06 smithi090 ceph-mon[29180]: pgmap v19016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:07.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34668.397336 secs 2024-04-27T04:13:07.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:06 smithi119 ceph-mon[32752]: pgmap v19016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:07.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34668.397336 secs 2024-04-27T04:13:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:06 smithi167 ceph-mon[32751]: pgmap v19016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34668.397336 secs 2024-04-27T04:13:09.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:08 smithi090 ceph-mon[29180]: pgmap v19017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:09.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:08 smithi119 ceph-mon[32752]: pgmap v19017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:09.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:08 smithi167 ceph-mon[32751]: pgmap v19017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:11.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:10 smithi090 ceph-mon[29180]: pgmap v19018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:11.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:10 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34672.520924 secs 2024-04-27T04:13:11.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:10 smithi119 ceph-mon[32752]: pgmap v19018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:11.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:10 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34672.520924 secs 2024-04-27T04:13:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:10 smithi167 ceph-mon[32751]: pgmap v19018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:10 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34672.520924 secs 2024-04-27T04:13:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34673.397466 secs 2024-04-27T04:13:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34673.397466 secs 2024-04-27T04:13:12.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34673.397466 secs 2024-04-27T04:13:13.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:12 smithi119 ceph-mon[32752]: pgmap v19019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:13.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:12 smithi167 ceph-mon[32751]: pgmap v19019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:12 smithi090 ceph-mon[29180]: pgmap v19019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:15.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:14 smithi119 ceph-mon[32752]: pgmap v19020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:15.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:14 smithi167 ceph-mon[32751]: pgmap v19020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:15.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:14 smithi090 ceph-mon[29180]: pgmap v19020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:16.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:15 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34677.521005 secs 2024-04-27T04:13:16.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:15 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34677.521005 secs 2024-04-27T04:13:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:15 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34677.521005 secs 2024-04-27T04:13:17.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:16 smithi119 ceph-mon[32752]: pgmap v19021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:17.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34678.397594 secs 2024-04-27T04:13:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:16 smithi167 ceph-mon[32751]: pgmap v19021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34678.397594 secs 2024-04-27T04:13:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:16 smithi090 ceph-mon[29180]: pgmap v19021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34678.397594 secs 2024-04-27T04:13:19.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:18 smithi119 ceph-mon[32752]: pgmap v19022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:19.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:18 smithi167 ceph-mon[32751]: pgmap v19022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:19.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:18 smithi090 ceph-mon[29180]: pgmap v19022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:21.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:20 smithi119 ceph-mon[32752]: pgmap v19023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:21.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:20 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34682.521109 secs 2024-04-27T04:13:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:20 smithi167 ceph-mon[32751]: pgmap v19023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:20 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34682.521109 secs 2024-04-27T04:13:21.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:20 smithi090 ceph-mon[29180]: pgmap v19023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:20 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34682.521109 secs 2024-04-27T04:13:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34683.397737 secs 2024-04-27T04:13:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34683.397737 secs 2024-04-27T04:13:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34683.397737 secs 2024-04-27T04:13:23.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:22 smithi119 ceph-mon[32752]: pgmap v19024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:23.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:22 smithi167 ceph-mon[32751]: pgmap v19024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:23.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:22 smithi090 ceph-mon[29180]: pgmap v19024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:25.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:24 smithi119 ceph-mon[32752]: pgmap v19025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:25.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:24 smithi167 ceph-mon[32751]: pgmap v19025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:24 smithi090 ceph-mon[29180]: pgmap v19025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:25 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34687.521189 secs 2024-04-27T04:13:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:25 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34687.521189 secs 2024-04-27T04:13:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:25 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34687.521189 secs 2024-04-27T04:13:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:13:27.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:26 smithi090 ceph-mon[29180]: pgmap v19026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:27.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34688.397819 secs 2024-04-27T04:13:27.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:27.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:27.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:13:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:26 smithi119 ceph-mon[32752]: pgmap v19026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34688.397819 secs 2024-04-27T04:13:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:27.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:13:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:26 smithi167 ceph-mon[32751]: pgmap v19026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34688.397819 secs 2024-04-27T04:13:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:13:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:13:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:13:28.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:27 smithi119 ceph-mon[32752]: pgmap v19027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:13:28.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:27 smithi119 ceph-mon[32752]: pgmap v19028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:28.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:27 smithi167 ceph-mon[32751]: pgmap v19027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:13:28.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:27 smithi167 ceph-mon[32751]: pgmap v19028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:28.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:27 smithi090 ceph-mon[29180]: pgmap v19027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:13:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:27 smithi090 ceph-mon[29180]: pgmap v19028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:30.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:29 smithi119 ceph-mon[32752]: pgmap v19029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:13:30.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:29 smithi167 ceph-mon[32751]: pgmap v19029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:13:30.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:29 smithi090 ceph-mon[29180]: pgmap v19029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:13:31.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:30 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34692.521272 secs 2024-04-27T04:13:31.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:30 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34692.521272 secs 2024-04-27T04:13:31.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:30 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34692.521272 secs 2024-04-27T04:13:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:31 smithi119 ceph-mon[32752]: pgmap v19030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34693.397938 secs 2024-04-27T04:13:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:31 smithi167 ceph-mon[32751]: pgmap v19030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34693.397938 secs 2024-04-27T04:13:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:31 smithi090 ceph-mon[29180]: pgmap v19030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34693.397938 secs 2024-04-27T04:13:34.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:33 smithi119 ceph-mon[32752]: pgmap v19031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:13:34.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:33 smithi167 ceph-mon[32751]: pgmap v19031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:13:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:33 smithi090 ceph-mon[29180]: pgmap v19031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:13:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:35 smithi119 ceph-mon[32752]: pgmap v19032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:35 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34697.521326 secs 2024-04-27T04:13:36.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:35 smithi167 ceph-mon[32751]: pgmap v19032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:36.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:35 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34697.521326 secs 2024-04-27T04:13:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:35 smithi090 ceph-mon[29180]: pgmap v19032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:13:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:35 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34697.521326 secs 2024-04-27T04:13:37.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34698.398035 secs 2024-04-27T04:13:37.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34698.398035 secs 2024-04-27T04:13:37.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34698.398035 secs 2024-04-27T04:13:38.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:37 smithi119 ceph-mon[32752]: pgmap v19033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:13:38.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:37 smithi167 ceph-mon[32751]: pgmap v19033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:13:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:37 smithi090 ceph-mon[29180]: pgmap v19033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:13:40.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:39 smithi119 ceph-mon[32752]: pgmap v19034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:40.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:39 smithi167 ceph-mon[32751]: pgmap v19034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:39 smithi090 ceph-mon[29180]: pgmap v19034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:41.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:40 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34702.521409 secs 2024-04-27T04:13:41.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:40 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34702.521409 secs 2024-04-27T04:13:41.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:40 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34702.521409 secs 2024-04-27T04:13:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:41 smithi119 ceph-mon[32752]: pgmap v19035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34703.398159 secs 2024-04-27T04:13:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:41 smithi167 ceph-mon[32751]: pgmap v19035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34703.398159 secs 2024-04-27T04:13:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:41 smithi090 ceph-mon[29180]: pgmap v19035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34703.398159 secs 2024-04-27T04:13:44.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:43 smithi119 ceph-mon[32752]: pgmap v19036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:44.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:43 smithi167 ceph-mon[32751]: pgmap v19036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:43 smithi090 ceph-mon[29180]: pgmap v19036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:46.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:45 smithi119 ceph-mon[32752]: pgmap v19037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:46.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:45 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34707.521487 secs 2024-04-27T04:13:46.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:45 smithi167 ceph-mon[32751]: pgmap v19037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:46.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:45 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34707.521487 secs 2024-04-27T04:13:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:45 smithi090 ceph-mon[29180]: pgmap v19037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:45 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34707.521487 secs 2024-04-27T04:13:47.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34708.398308 secs 2024-04-27T04:13:47.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34708.398308 secs 2024-04-27T04:13:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34708.398308 secs 2024-04-27T04:13:48.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:47 smithi119 ceph-mon[32752]: pgmap v19038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:48.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:47 smithi167 ceph-mon[32751]: pgmap v19038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:48.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:47 smithi090 ceph-mon[29180]: pgmap v19038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:50.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:49 smithi119 ceph-mon[32752]: pgmap v19039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:50.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:49 smithi167 ceph-mon[32751]: pgmap v19039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:50.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:49 smithi090 ceph-mon[29180]: pgmap v19039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:50 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34712.521567 secs 2024-04-27T04:13:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:50 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34712.521567 secs 2024-04-27T04:13:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:50 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34712.521567 secs 2024-04-27T04:13:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:51 smithi119 ceph-mon[32752]: pgmap v19040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34713.398466 secs 2024-04-27T04:13:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:51 smithi167 ceph-mon[32751]: pgmap v19040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34713.398466 secs 2024-04-27T04:13:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:51 smithi090 ceph-mon[29180]: pgmap v19040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34713.398466 secs 2024-04-27T04:13:54.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:53 smithi119 ceph-mon[32752]: pgmap v19041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:54.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:53 smithi167 ceph-mon[32751]: pgmap v19041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:54.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:53 smithi090 ceph-mon[29180]: pgmap v19041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:55 smithi119 ceph-mon[32752]: pgmap v19042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:55 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34717.521652 secs 2024-04-27T04:13:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:55 smithi167 ceph-mon[32751]: pgmap v19042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:55 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34717.521652 secs 2024-04-27T04:13:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:55 smithi090 ceph-mon[29180]: pgmap v19042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:13:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:55 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34717.521652 secs 2024-04-27T04:13:57.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34718.398541 secs 2024-04-27T04:13:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34718.398541 secs 2024-04-27T04:13:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34718.398541 secs 2024-04-27T04:13:58.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:57 smithi119 ceph-mon[32752]: pgmap v19043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:58.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:57 smithi167 ceph-mon[32751]: pgmap v19043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:13:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:57 smithi090 ceph-mon[29180]: pgmap v19043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:00.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:13:59 smithi119 ceph-mon[32752]: pgmap v19044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:00.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:13:59 smithi167 ceph-mon[32751]: pgmap v19044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:00.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:13:59 smithi090 ceph-mon[29180]: pgmap v19044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:00 smithi119 ceph-mon[32752]: 46 slow requests, 1 included below; oldest blocked for > 34722.521740 secs 2024-04-27T04:14:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:00 smithi119 ceph-mon[32752]: slow request 1443.998902 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:14:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:00 smithi167 ceph-mon[32751]: 46 slow requests, 1 included below; oldest blocked for > 34722.521740 secs 2024-04-27T04:14:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:00 smithi167 ceph-mon[32751]: slow request 1443.998902 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:14:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:00 smithi090 ceph-mon[29180]: 46 slow requests, 1 included below; oldest blocked for > 34722.521740 secs 2024-04-27T04:14:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:00 smithi090 ceph-mon[29180]: slow request 1443.998902 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:14:02.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:01 smithi119 ceph-mon[32752]: pgmap v19045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:02.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34723.398611 secs 2024-04-27T04:14:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:01 smithi167 ceph-mon[32751]: pgmap v19045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34723.398611 secs 2024-04-27T04:14:02.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:01 smithi090 ceph-mon[29180]: pgmap v19045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34723.398611 secs 2024-04-27T04:14:04.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:04 smithi119 ceph-mon[32752]: pgmap v19046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:04.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:04 smithi167 ceph-mon[32751]: pgmap v19046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:04.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:04 smithi090 ceph-mon[29180]: pgmap v19046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:06.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:06 smithi119 ceph-mon[32752]: pgmap v19047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:06 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34727.521846 secs 2024-04-27T04:14:06.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:06 smithi167 ceph-mon[32751]: pgmap v19047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:06 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34727.521846 secs 2024-04-27T04:14:06.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:06 smithi090 ceph-mon[29180]: pgmap v19047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:06 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34727.521846 secs 2024-04-27T04:14:07.276 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34728.398728 secs 2024-04-27T04:14:07.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34728.398728 secs 2024-04-27T04:14:07.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34728.398728 secs 2024-04-27T04:14:08.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:08 smithi119 ceph-mon[32752]: pgmap v19048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:08.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:08 smithi167 ceph-mon[32751]: pgmap v19048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:08.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:08 smithi090 ceph-mon[29180]: pgmap v19048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:10.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:10 smithi119 ceph-mon[32752]: pgmap v19049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:10 smithi167 ceph-mon[32751]: pgmap v19049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:10 smithi090 ceph-mon[29180]: pgmap v19049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:11.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:11 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34732.521934 secs 2024-04-27T04:14:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:11 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34732.521934 secs 2024-04-27T04:14:11.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:11 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34732.521934 secs 2024-04-27T04:14:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:12 smithi119 ceph-mon[32752]: pgmap v19050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34733.398848 secs 2024-04-27T04:14:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:12 smithi167 ceph-mon[32751]: pgmap v19050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34733.398848 secs 2024-04-27T04:14:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:12 smithi090 ceph-mon[29180]: pgmap v19050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:14:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34733.398848 secs 2024-04-27T04:14:14.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:14 smithi119 ceph-mon[32752]: pgmap v19051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:14.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:14 smithi167 ceph-mon[32751]: pgmap v19051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:14.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:14 smithi090 ceph-mon[29180]: pgmap v19051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:16 smithi119 ceph-mon[32752]: pgmap v19052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:16 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34737.522015 secs 2024-04-27T04:14:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:16 smithi167 ceph-mon[32751]: pgmap v19052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:16 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34737.522015 secs 2024-04-27T04:14:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:16 smithi090 ceph-mon[29180]: pgmap v19052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:16 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34737.522015 secs 2024-04-27T04:14:17.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34738.398967 secs 2024-04-27T04:14:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34738.398967 secs 2024-04-27T04:14:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34738.398967 secs 2024-04-27T04:14:18.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:18 smithi119 ceph-mon[32752]: pgmap v19053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:18 smithi167 ceph-mon[32751]: pgmap v19053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:18.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:18 smithi090 ceph-mon[29180]: pgmap v19053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:20.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:20 smithi119 ceph-mon[32752]: pgmap v19054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:20.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:20 smithi167 ceph-mon[32751]: pgmap v19054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:20.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:20 smithi090 ceph-mon[29180]: pgmap v19054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:21.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:21 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34742.522094 secs 2024-04-27T04:14:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:21 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34742.522094 secs 2024-04-27T04:14:21.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:21 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34742.522094 secs 2024-04-27T04:14:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:22 smithi119 ceph-mon[32752]: pgmap v19055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34743.399100 secs 2024-04-27T04:14:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:22 smithi167 ceph-mon[32751]: pgmap v19055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34743.399100 secs 2024-04-27T04:14:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:22 smithi090 ceph-mon[29180]: pgmap v19055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34743.399100 secs 2024-04-27T04:14:24.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:24 smithi119 ceph-mon[32752]: pgmap v19056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:24.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:24 smithi167 ceph-mon[32751]: pgmap v19056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:24.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:24 smithi090 ceph-mon[29180]: pgmap v19056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:26 smithi119 ceph-mon[32752]: pgmap v19057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:26 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34747.522176 secs 2024-04-27T04:14:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:26 smithi167 ceph-mon[32751]: pgmap v19057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:26 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34747.522176 secs 2024-04-27T04:14:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:26 smithi090 ceph-mon[29180]: pgmap v19057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:26 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34747.522176 secs 2024-04-27T04:14:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34748.399171 secs 2024-04-27T04:14:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:27.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:14:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34748.399171 secs 2024-04-27T04:14:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:14:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34748.399171 secs 2024-04-27T04:14:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:14:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:14:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:14:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:14:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:28 smithi167 ceph-mon[32751]: pgmap v19058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:28 smithi167 ceph-mon[32751]: pgmap v19059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:14:28.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:28 smithi167 ceph-mon[32751]: pgmap v19060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:14:28.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:28 smithi090 ceph-mon[29180]: pgmap v19058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:28 smithi090 ceph-mon[29180]: pgmap v19059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:14:28.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:28 smithi090 ceph-mon[29180]: pgmap v19060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:14:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:28 smithi119 ceph-mon[32752]: pgmap v19058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:28 smithi119 ceph-mon[32752]: pgmap v19059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:14:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:28 smithi119 ceph-mon[32752]: pgmap v19060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:14:30.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:30 smithi090 ceph-mon[29180]: pgmap v19061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:30.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:30 smithi167 ceph-mon[32751]: pgmap v19061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:30.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:30 smithi119 ceph-mon[32752]: pgmap v19061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:31.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:31 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34752.522263 secs 2024-04-27T04:14:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:31 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34752.522263 secs 2024-04-27T04:14:31.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:31 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34752.522263 secs 2024-04-27T04:14:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:32 smithi090 ceph-mon[29180]: pgmap v19062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34753.399268 secs 2024-04-27T04:14:32.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:32 smithi119 ceph-mon[32752]: pgmap v19062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:32.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34753.399268 secs 2024-04-27T04:14:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:32 smithi167 ceph-mon[32751]: pgmap v19062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34753.399268 secs 2024-04-27T04:14:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:34 smithi090 ceph-mon[29180]: pgmap v19063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:14:34.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:34 smithi119 ceph-mon[32752]: pgmap v19063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:14:34.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:34 smithi167 ceph-mon[32751]: pgmap v19063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:14:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:36 smithi090 ceph-mon[29180]: pgmap v19064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:36 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34757.522348 secs 2024-04-27T04:14:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:36 smithi119 ceph-mon[32752]: pgmap v19064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:36 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34757.522348 secs 2024-04-27T04:14:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:36 smithi167 ceph-mon[32751]: pgmap v19064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:14:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:36 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34757.522348 secs 2024-04-27T04:14:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34758.399408 secs 2024-04-27T04:14:37.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34758.399408 secs 2024-04-27T04:14:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34758.399408 secs 2024-04-27T04:14:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:38 smithi090 ceph-mon[29180]: pgmap v19065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:14:38.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:38 smithi119 ceph-mon[32752]: pgmap v19065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:14:38.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:38 smithi167 ceph-mon[32751]: pgmap v19065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:14:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:40 smithi090 ceph-mon[29180]: pgmap v19066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:40.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:40 smithi119 ceph-mon[32752]: pgmap v19066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:40.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:40 smithi167 ceph-mon[32751]: pgmap v19066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:14:40.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:14:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:14:40.433+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:14:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:14:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:14:40.433+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:14:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:14:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:14:40.433+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:14:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:14:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:14:40.433+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:14:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:14:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:14:40.433+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:14:41.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:41 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34762.522433 secs 2024-04-27T04:14:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:41 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34762.522433 secs 2024-04-27T04:14:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:41 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34762.522433 secs 2024-04-27T04:14:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:42 smithi090 ceph-mon[29180]: pgmap v19067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34763.399526 secs 2024-04-27T04:14:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:42 smithi119 ceph-mon[32752]: pgmap v19067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34763.399526 secs 2024-04-27T04:14:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:42 smithi167 ceph-mon[32751]: pgmap v19067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34763.399526 secs 2024-04-27T04:14:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:44 smithi090 ceph-mon[29180]: pgmap v19068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:44.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:44 smithi119 ceph-mon[32752]: pgmap v19068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:44 smithi167 ceph-mon[32751]: pgmap v19068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:46.354 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:46 smithi119 ceph-mon[32752]: pgmap v19069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:46.354 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:46 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34767.522516 secs 2024-04-27T04:14:46.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:46 smithi167 ceph-mon[32751]: pgmap v19069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:46.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:46 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34767.522516 secs 2024-04-27T04:14:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:46 smithi090 ceph-mon[29180]: pgmap v19069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:46 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34767.522516 secs 2024-04-27T04:14:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34768.399600 secs 2024-04-27T04:14:47.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34768.399600 secs 2024-04-27T04:14:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34768.399600 secs 2024-04-27T04:14:48.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:48 smithi090 ceph-mon[29180]: pgmap v19070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:48.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:48 smithi119 ceph-mon[32752]: pgmap v19070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:48.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:48 smithi167 ceph-mon[32751]: pgmap v19070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:50.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:50 smithi090 ceph-mon[29180]: pgmap v19071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:50.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:50 smithi119 ceph-mon[32752]: pgmap v19071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:50.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:50 smithi167 ceph-mon[32751]: pgmap v19071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:51 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34772.522598 secs 2024-04-27T04:14:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:51 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34772.522598 secs 2024-04-27T04:14:51.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:51 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34772.522598 secs 2024-04-27T04:14:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:52 smithi090 ceph-mon[29180]: pgmap v19072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34773.399712 secs 2024-04-27T04:14:52.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:52 smithi119 ceph-mon[32752]: pgmap v19072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:52.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34773.399712 secs 2024-04-27T04:14:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:52 smithi167 ceph-mon[32751]: pgmap v19072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34773.399712 secs 2024-04-27T04:14:54.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:54 smithi090 ceph-mon[29180]: pgmap v19073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:54.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:54 smithi119 ceph-mon[32752]: pgmap v19073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:54.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:54 smithi167 ceph-mon[32751]: pgmap v19073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:56 smithi090 ceph-mon[29180]: pgmap v19074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:56 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34777.522681 secs 2024-04-27T04:14:56.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:56 smithi119 ceph-mon[32752]: pgmap v19074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:56.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:56 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34777.522681 secs 2024-04-27T04:14:56.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:56 smithi167 ceph-mon[32751]: pgmap v19074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:14:56.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:56 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34777.522681 secs 2024-04-27T04:14:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34778.399852 secs 2024-04-27T04:14:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34778.399852 secs 2024-04-27T04:14:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34778.399852 secs 2024-04-27T04:14:58.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:14:58 smithi119 ceph-mon[32752]: pgmap v19075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:58.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:14:58 smithi167 ceph-mon[32751]: pgmap v19075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:14:58.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:14:58 smithi090 ceph-mon[29180]: pgmap v19075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:00.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:00 smithi119 ceph-mon[32752]: pgmap v19076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:00.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:00 smithi167 ceph-mon[32751]: pgmap v19076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:00.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:00 smithi090 ceph-mon[29180]: pgmap v19076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:01 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34782.522768 secs 2024-04-27T04:15:01.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:01 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34782.522768 secs 2024-04-27T04:15:01.603 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:01 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34782.522768 secs 2024-04-27T04:15:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:02 smithi119 ceph-mon[32752]: pgmap v19077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34783.399933 secs 2024-04-27T04:15:02.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:02 smithi167 ceph-mon[32751]: pgmap v19077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:02.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34783.399933 secs 2024-04-27T04:15:02.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:02 smithi090 ceph-mon[29180]: pgmap v19077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:02.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34783.399933 secs 2024-04-27T04:15:04.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:04 smithi119 ceph-mon[32752]: pgmap v19078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:04.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:04 smithi167 ceph-mon[32751]: pgmap v19078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:04.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:04 smithi090 ceph-mon[29180]: pgmap v19078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:06.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:06 smithi119 ceph-mon[32752]: pgmap v19079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:06.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:06 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34787.522852 secs 2024-04-27T04:15:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:06 smithi167 ceph-mon[32751]: pgmap v19079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:06 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34787.522852 secs 2024-04-27T04:15:06.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:06 smithi090 ceph-mon[29180]: pgmap v19079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:06.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:06 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34787.522852 secs 2024-04-27T04:15:07.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34788.399999 secs 2024-04-27T04:15:07.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34788.399999 secs 2024-04-27T04:15:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34788.399999 secs 2024-04-27T04:15:08.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:08 smithi119 ceph-mon[32752]: pgmap v19080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:08.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:08 smithi167 ceph-mon[32751]: pgmap v19080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:08.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:08 smithi090 ceph-mon[29180]: pgmap v19080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:10.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:10 smithi119 ceph-mon[32752]: pgmap v19081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:10.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:10 smithi167 ceph-mon[32751]: pgmap v19081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:10.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:10 smithi090 ceph-mon[29180]: pgmap v19081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:11.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:11 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34792.522935 secs 2024-04-27T04:15:11.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:11 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34792.522935 secs 2024-04-27T04:15:11.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:11 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34792.522935 secs 2024-04-27T04:15:12.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:12 smithi119 ceph-mon[32752]: pgmap v19082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:12.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34793.400112 secs 2024-04-27T04:15:12.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:12 smithi167 ceph-mon[32751]: pgmap v19082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:12.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34793.400112 secs 2024-04-27T04:15:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:12 smithi090 ceph-mon[29180]: pgmap v19082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34793.400112 secs 2024-04-27T04:15:14.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:14 smithi119 ceph-mon[32752]: pgmap v19083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:14.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:14 smithi167 ceph-mon[32751]: pgmap v19083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:14 smithi090 ceph-mon[29180]: pgmap v19083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:15:16.452 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:16 smithi090 ceph-mon[29180]: pgmap v19084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:16.452 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:16 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34797.523018 secs 2024-04-27T04:15:16.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:16 smithi119 ceph-mon[32752]: pgmap v19084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:16.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:16 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34797.523018 secs 2024-04-27T04:15:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:16 smithi167 ceph-mon[32751]: pgmap v19084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:15:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:16 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34797.523018 secs 2024-04-27T04:15:17.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34798.400246 secs 2024-04-27T04:15:17.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34798.400246 secs 2024-04-27T04:15:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34798.400246 secs 2024-04-27T04:15:18.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:18 smithi119 ceph-mon[32752]: pgmap v19085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:18.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:18 smithi167 ceph-mon[32751]: pgmap v19085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:18.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:18 smithi090 ceph-mon[29180]: pgmap v19085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:20.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:20 smithi119 ceph-mon[32752]: pgmap v19086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:20.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:20 smithi167 ceph-mon[32751]: pgmap v19086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:20.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:20 smithi090 ceph-mon[29180]: pgmap v19086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:21.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:21 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34802.523104 secs 2024-04-27T04:15:21.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:21 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34802.523104 secs 2024-04-27T04:15:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:21 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34802.523104 secs 2024-04-27T04:15:22.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:22 smithi119 ceph-mon[32752]: pgmap v19087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:22.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34803.400363 secs 2024-04-27T04:15:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:22 smithi167 ceph-mon[32751]: pgmap v19087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34803.400363 secs 2024-04-27T04:15:22.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:22 smithi090 ceph-mon[29180]: pgmap v19087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:22.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34803.400363 secs 2024-04-27T04:15:24.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:24 smithi119 ceph-mon[32752]: pgmap v19088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:24.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:24 smithi167 ceph-mon[32751]: pgmap v19088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:24.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:24 smithi090 ceph-mon[29180]: pgmap v19088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:26 smithi119 ceph-mon[32752]: pgmap v19089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:26 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34807.523188 secs 2024-04-27T04:15:26.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:26 smithi167 ceph-mon[32751]: pgmap v19089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:26.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:26 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34807.523188 secs 2024-04-27T04:15:26.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:26 smithi090 ceph-mon[29180]: pgmap v19089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:26 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34807.523188 secs 2024-04-27T04:15:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34808.400529 secs 2024-04-27T04:15:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34808.400529 secs 2024-04-27T04:15:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34808.400529 secs 2024-04-27T04:15:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:15:28.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:28 smithi167 ceph-mon[32751]: pgmap v19090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:28.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.204 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:28 smithi090 ceph-mon[29180]: pgmap v19090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:28.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.516 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.517 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.517 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:28 smithi119 ceph-mon[32752]: pgmap v19090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:28.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:29.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:29 smithi119 ceph-mon[32752]: pgmap v19091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:29.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:29 smithi167 ceph-mon[32751]: pgmap v19091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:29.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:29 smithi090 ceph-mon[29180]: pgmap v19091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:30.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:15:30.635 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:15:30.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.636 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:15:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:15:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.827 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:30.828 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34812.523251 secs 2024-04-27T04:15:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:15:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:15:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:15:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:15:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:15:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: pgmap v19092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T04:15:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: pgmap v19093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:31.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34812.523251 secs 2024-04-27T04:15:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:15:31.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:15:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:15:31.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:15:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:15:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: pgmap v19092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T04:15:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: pgmap v19093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:31.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34812.523251 secs 2024-04-27T04:15:31.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:15:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:15:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:15:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:15:31.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:15:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:15:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:15:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: pgmap v19092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T04:15:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: pgmap v19093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:15:32.393 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34813.400624 secs 2024-04-27T04:15:32.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34813.400624 secs 2024-04-27T04:15:32.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34813.400624 secs 2024-04-27T04:15:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:33 smithi090 ceph-mon[29180]: pgmap v19094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:33.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:33 smithi119 ceph-mon[32752]: pgmap v19094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:33.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:33 smithi167 ceph-mon[32751]: pgmap v19094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:35.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:35 smithi090 ceph-mon[29180]: pgmap v19095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:36.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:35 smithi119 ceph-mon[32752]: pgmap v19095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:36.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:35 smithi167 ceph-mon[32751]: pgmap v19095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:36.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:36 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34817.523331 secs 2024-04-27T04:15:36.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34818.400746 secs 2024-04-27T04:15:37.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:36 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34817.523331 secs 2024-04-27T04:15:37.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34818.400746 secs 2024-04-27T04:15:37.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:36 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34817.523331 secs 2024-04-27T04:15:37.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34818.400746 secs 2024-04-27T04:15:37.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:37 smithi090 ceph-mon[29180]: pgmap v19096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:38.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:37 smithi119 ceph-mon[32752]: pgmap v19096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:38.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:37 smithi167 ceph-mon[32751]: pgmap v19096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:39 smithi090 ceph-mon[29180]: pgmap v19097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:40.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:39 smithi119 ceph-mon[32752]: pgmap v19097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:40.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:39 smithi167 ceph-mon[32751]: pgmap v19097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:15:41.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:41 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34822.523414 secs 2024-04-27T04:15:41.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:41 smithi090 ceph-mon[29180]: pgmap v19098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:15:41.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34823.400900 secs 2024-04-27T04:15:42.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:41 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34822.523414 secs 2024-04-27T04:15:42.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:41 smithi119 ceph-mon[32752]: pgmap v19098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:15:42.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34823.400900 secs 2024-04-27T04:15:42.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:41 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34822.523414 secs 2024-04-27T04:15:42.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:41 smithi167 ceph-mon[32751]: pgmap v19098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:15:42.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34823.400900 secs 2024-04-27T04:15:43.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:43 smithi090 ceph-mon[29180]: pgmap v19099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:44.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:43 smithi119 ceph-mon[32752]: pgmap v19099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:44.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:43 smithi167 ceph-mon[32751]: pgmap v19099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:45.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:45 smithi090 ceph-mon[29180]: pgmap v19100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:46.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:45 smithi119 ceph-mon[32752]: pgmap v19100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:46.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:45 smithi167 ceph-mon[32751]: pgmap v19100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:46 smithi090 ceph-mon[29180]: 46 slow requests, 1 included below; oldest blocked for > 34827.523495 secs 2024-04-27T04:15:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:46 smithi090 ceph-mon[29180]: slow request 364.970098 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:15:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34828.401030 secs 2024-04-27T04:15:46.953 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:46 smithi119 ceph-mon[32752]: 46 slow requests, 1 included below; oldest blocked for > 34827.523495 secs 2024-04-27T04:15:46.953 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:46 smithi119 ceph-mon[32752]: slow request 364.970098 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:15:46.954 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34828.401030 secs 2024-04-27T04:15:46.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:46 smithi167 ceph-mon[32751]: 46 slow requests, 1 included below; oldest blocked for > 34827.523495 secs 2024-04-27T04:15:46.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:46 smithi167 ceph-mon[32751]: slow request 364.970098 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:15:46.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34828.401030 secs 2024-04-27T04:15:47.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:47 smithi090 ceph-mon[29180]: pgmap v19101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:47.985 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:47 smithi167 ceph-mon[32751]: pgmap v19101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:48.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:47 smithi119 ceph-mon[32752]: pgmap v19101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:49.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:49 smithi090 ceph-mon[29180]: pgmap v19102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:50.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:49 smithi119 ceph-mon[32752]: pgmap v19102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:50.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:49 smithi167 ceph-mon[32751]: pgmap v19102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:51.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:51 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34832.523585 secs 2024-04-27T04:15:51.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:51 smithi090 ceph-mon[29180]: pgmap v19103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:51.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34833.401139 secs 2024-04-27T04:15:52.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:51 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34832.523585 secs 2024-04-27T04:15:52.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:51 smithi119 ceph-mon[32752]: pgmap v19103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:52.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34833.401139 secs 2024-04-27T04:15:52.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:51 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34832.523585 secs 2024-04-27T04:15:52.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:51 smithi167 ceph-mon[32751]: pgmap v19103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:52.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34833.401139 secs 2024-04-27T04:15:53.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:53 smithi090 ceph-mon[29180]: pgmap v19104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:54.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:53 smithi119 ceph-mon[32752]: pgmap v19104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:54.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:53 smithi167 ceph-mon[32751]: pgmap v19104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:56.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:55 smithi119 ceph-mon[32752]: pgmap v19105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:56.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:55 smithi167 ceph-mon[32751]: pgmap v19105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:56.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:55 smithi090 ceph-mon[29180]: pgmap v19105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:15:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:56 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34837.523665 secs 2024-04-27T04:15:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34838.401291 secs 2024-04-27T04:15:57.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:56 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34837.523665 secs 2024-04-27T04:15:57.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34838.401291 secs 2024-04-27T04:15:57.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:56 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34837.523665 secs 2024-04-27T04:15:57.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34838.401291 secs 2024-04-27T04:15:58.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:57 smithi119 ceph-mon[32752]: pgmap v19106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:58.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:57 smithi167 ceph-mon[32751]: pgmap v19106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:15:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:57 smithi090 ceph-mon[29180]: pgmap v19106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:00.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:15:59 smithi119 ceph-mon[32752]: pgmap v19107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:00.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:15:59 smithi167 ceph-mon[32751]: pgmap v19107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:00.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:15:59 smithi090 ceph-mon[29180]: pgmap v19107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:01 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34842.523747 secs 2024-04-27T04:16:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:01 smithi119 ceph-mon[32752]: pgmap v19108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34843.401425 secs 2024-04-27T04:16:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:01 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34842.523747 secs 2024-04-27T04:16:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:01 smithi167 ceph-mon[32751]: pgmap v19108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34843.401425 secs 2024-04-27T04:16:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:01 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34842.523747 secs 2024-04-27T04:16:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:01 smithi090 ceph-mon[29180]: pgmap v19108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34843.401425 secs 2024-04-27T04:16:03.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:03 smithi090 ceph-mon[29180]: pgmap v19109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:04.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:03 smithi119 ceph-mon[32752]: pgmap v19109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:04.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:03 smithi167 ceph-mon[32751]: pgmap v19109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:06.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:05 smithi119 ceph-mon[32752]: pgmap v19110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:06.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:05 smithi167 ceph-mon[32751]: pgmap v19110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:06.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:05 smithi090 ceph-mon[29180]: pgmap v19110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:07.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:06 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34847.523847 secs 2024-04-27T04:16:07.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34848.401507 secs 2024-04-27T04:16:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:06 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34847.523847 secs 2024-04-27T04:16:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34848.401507 secs 2024-04-27T04:16:07.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:06 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34847.523847 secs 2024-04-27T04:16:07.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34848.401507 secs 2024-04-27T04:16:08.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:07 smithi119 ceph-mon[32752]: pgmap v19111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:08.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:07 smithi167 ceph-mon[32751]: pgmap v19111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:08.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:07 smithi090 ceph-mon[29180]: pgmap v19111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:10.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:09 smithi119 ceph-mon[32752]: pgmap v19112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:10.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:09 smithi167 ceph-mon[32751]: pgmap v19112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:10.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:09 smithi090 ceph-mon[29180]: pgmap v19112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:12.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:11 smithi119 ceph-mon[32752]: 46 slow requests, 1 included below; oldest blocked for > 34852.523937 secs 2024-04-27T04:16:12.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:11 smithi119 ceph-mon[32752]: slow request 5761.451356 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:16:12.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:11 smithi119 ceph-mon[32752]: pgmap v19113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:12.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34853.401620 secs 2024-04-27T04:16:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:11 smithi167 ceph-mon[32751]: 46 slow requests, 1 included below; oldest blocked for > 34852.523937 secs 2024-04-27T04:16:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:11 smithi167 ceph-mon[32751]: slow request 5761.451356 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:16:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:11 smithi167 ceph-mon[32751]: pgmap v19113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34853.401620 secs 2024-04-27T04:16:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:11 smithi090 ceph-mon[29180]: 46 slow requests, 1 included below; oldest blocked for > 34852.523937 secs 2024-04-27T04:16:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:11 smithi090 ceph-mon[29180]: slow request 5761.451356 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:16:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:11 smithi090 ceph-mon[29180]: pgmap v19113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34853.401620 secs 2024-04-27T04:16:14.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:13 smithi119 ceph-mon[32752]: pgmap v19114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:14.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:13 smithi167 ceph-mon[32751]: pgmap v19114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:14.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:13 smithi090 ceph-mon[29180]: pgmap v19114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:16.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:15 smithi119 ceph-mon[32752]: pgmap v19115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:16.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:15 smithi167 ceph-mon[32751]: pgmap v19115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:16.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:15 smithi090 ceph-mon[29180]: pgmap v19115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:16.953 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:16 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34857.524020 secs 2024-04-27T04:16:16.953 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34858.401736 secs 2024-04-27T04:16:16.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:16 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34857.524020 secs 2024-04-27T04:16:16.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34858.401736 secs 2024-04-27T04:16:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:16 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34857.524020 secs 2024-04-27T04:16:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34858.401736 secs 2024-04-27T04:16:18.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:17 smithi119 ceph-mon[32752]: pgmap v19116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:18.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:17 smithi167 ceph-mon[32751]: pgmap v19116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:18.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:17 smithi090 ceph-mon[29180]: pgmap v19116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:20.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:19 smithi119 ceph-mon[32752]: pgmap v19117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:20.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:19 smithi167 ceph-mon[32751]: pgmap v19117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:20.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:19 smithi090 ceph-mon[29180]: pgmap v19117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:21.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:20 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34862.524112 secs 2024-04-27T04:16:21.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:20 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34862.524112 secs 2024-04-27T04:16:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:20 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34862.524112 secs 2024-04-27T04:16:22.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:21 smithi119 ceph-mon[32752]: pgmap v19118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:22.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34863.401815 secs 2024-04-27T04:16:22.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:21 smithi167 ceph-mon[32751]: pgmap v19118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:22.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34863.401815 secs 2024-04-27T04:16:22.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:21 smithi090 ceph-mon[29180]: pgmap v19118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:22.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34863.401815 secs 2024-04-27T04:16:24.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:23 smithi119 ceph-mon[32752]: pgmap v19119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:24.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:23 smithi167 ceph-mon[32751]: pgmap v19119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:24.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:23 smithi090 ceph-mon[29180]: pgmap v19119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:26.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:25 smithi119 ceph-mon[32752]: pgmap v19120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:26.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:25 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34867.524172 secs 2024-04-27T04:16:26.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:25 smithi167 ceph-mon[32751]: pgmap v19120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:26.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:25 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34867.524172 secs 2024-04-27T04:16:26.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:25 smithi090 ceph-mon[29180]: pgmap v19120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:26.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:25 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34867.524172 secs 2024-04-27T04:16:27.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34868.401953 secs 2024-04-27T04:16:27.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34868.401953 secs 2024-04-27T04:16:27.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34868.401953 secs 2024-04-27T04:16:28.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:27 smithi119 ceph-mon[32752]: pgmap v19121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:28.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:27 smithi167 ceph-mon[32751]: pgmap v19121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:28.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:27 smithi090 ceph-mon[29180]: pgmap v19121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:30.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:29 smithi119 ceph-mon[32752]: pgmap v19122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:30.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:29 smithi167 ceph-mon[32751]: pgmap v19122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:30.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:29 smithi090 ceph-mon[29180]: pgmap v19122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:31.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:30 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34872.524256 secs 2024-04-27T04:16:31.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:31.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:30 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34872.524256 secs 2024-04-27T04:16:31.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:31.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:30 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34872.524256 secs 2024-04-27T04:16:31.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:32 smithi119 ceph-mon[32752]: pgmap v19123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34873.402042 secs 2024-04-27T04:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:16:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:32 smithi167 ceph-mon[32751]: pgmap v19123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34873.402042 secs 2024-04-27T04:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:16:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:32 smithi090 ceph-mon[29180]: pgmap v19123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34873.402042 secs 2024-04-27T04:16:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:16:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:16:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:16:33.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:33 smithi119 ceph-mon[32752]: pgmap v19124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:16:33.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:33 smithi119 ceph-mon[32752]: pgmap v19125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:33.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:33 smithi167 ceph-mon[32751]: pgmap v19124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:16:33.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:33 smithi167 ceph-mon[32751]: pgmap v19125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:33 smithi090 ceph-mon[29180]: pgmap v19124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:16:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:33 smithi090 ceph-mon[29180]: pgmap v19125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:35.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:35 smithi119 ceph-mon[32752]: pgmap v19126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:35.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:35 smithi167 ceph-mon[32751]: pgmap v19126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:35 smithi090 ceph-mon[29180]: pgmap v19126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:36 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34877.524342 secs 2024-04-27T04:16:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:36 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34877.524342 secs 2024-04-27T04:16:36.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:36 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34877.524342 secs 2024-04-27T04:16:37.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:37 smithi090 ceph-mon[29180]: pgmap v19127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:37.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34878.402168 secs 2024-04-27T04:16:37.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:37 smithi119 ceph-mon[32752]: pgmap v19127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34878.402168 secs 2024-04-27T04:16:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:37 smithi167 ceph-mon[32751]: pgmap v19127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34878.402168 secs 2024-04-27T04:16:39.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:39 smithi119 ceph-mon[32752]: pgmap v19128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T04:16:39.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:39 smithi167 ceph-mon[32751]: pgmap v19128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T04:16:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:39 smithi090 ceph-mon[29180]: pgmap v19128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T04:16:40.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:40 smithi119 ceph-mon[32752]: pgmap v19129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:40.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:40 smithi167 ceph-mon[32751]: pgmap v19129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:40.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:40 smithi090 ceph-mon[29180]: pgmap v19129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:16:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:41 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34882.524425 secs 2024-04-27T04:16:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:41 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34882.524425 secs 2024-04-27T04:16:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:41 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34882.524425 secs 2024-04-27T04:16:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34883.402327 secs 2024-04-27T04:16:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:42 smithi119 ceph-mon[32752]: pgmap v19130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:16:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34883.402327 secs 2024-04-27T04:16:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:42 smithi167 ceph-mon[32751]: pgmap v19130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:16:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34883.402327 secs 2024-04-27T04:16:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:42 smithi090 ceph-mon[29180]: pgmap v19130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:16:45.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:44 smithi090 ceph-mon[29180]: pgmap v19131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:45.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:44 smithi119 ceph-mon[32752]: pgmap v19131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:45.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:44 smithi167 ceph-mon[32751]: pgmap v19131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:46 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34887.524501 secs 2024-04-27T04:16:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:46 smithi090 ceph-mon[29180]: pgmap v19132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:46.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34888.402456 secs 2024-04-27T04:16:47.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:46 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34887.524501 secs 2024-04-27T04:16:47.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:46 smithi119 ceph-mon[32752]: pgmap v19132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:47.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34888.402456 secs 2024-04-27T04:16:47.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:46 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34887.524501 secs 2024-04-27T04:16:47.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:46 smithi167 ceph-mon[32751]: pgmap v19132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:47.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34888.402456 secs 2024-04-27T04:16:49.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:48 smithi090 ceph-mon[29180]: pgmap v19133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:49.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:48 smithi119 ceph-mon[32752]: pgmap v19133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:49.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:48 smithi167 ceph-mon[32751]: pgmap v19133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:50 smithi119 ceph-mon[32752]: pgmap v19134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:50 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34892.524582 secs 2024-04-27T04:16:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:50 smithi167 ceph-mon[32751]: pgmap v19134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:50 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34892.524582 secs 2024-04-27T04:16:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:50 smithi090 ceph-mon[29180]: pgmap v19134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:50 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34892.524582 secs 2024-04-27T04:16:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34893.402584 secs 2024-04-27T04:16:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34893.402584 secs 2024-04-27T04:16:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34893.402584 secs 2024-04-27T04:16:53.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:52 smithi119 ceph-mon[32752]: pgmap v19135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:53.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:52 smithi167 ceph-mon[32751]: pgmap v19135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:53.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:52 smithi090 ceph-mon[29180]: pgmap v19135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:55.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:54 smithi119 ceph-mon[32752]: pgmap v19136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:55.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:54 smithi167 ceph-mon[32751]: pgmap v19136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:55.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:54 smithi090 ceph-mon[29180]: pgmap v19136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:16:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:55 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34897.524664 secs 2024-04-27T04:16:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:55 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34897.524664 secs 2024-04-27T04:16:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:55 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34897.524664 secs 2024-04-27T04:16:57.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:56 smithi119 ceph-mon[32752]: pgmap v19137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:57.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34898.402732 secs 2024-04-27T04:16:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:56 smithi167 ceph-mon[32751]: pgmap v19137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34898.402732 secs 2024-04-27T04:16:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:56 smithi090 ceph-mon[29180]: pgmap v19137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34898.402732 secs 2024-04-27T04:16:59.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:16:58 smithi119 ceph-mon[32752]: pgmap v19138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:59.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:16:58 smithi167 ceph-mon[32751]: pgmap v19138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:16:59.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:16:58 smithi090 ceph-mon[29180]: pgmap v19138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:01.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:00 smithi119 ceph-mon[32752]: pgmap v19139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:01.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:00 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34902.524749 secs 2024-04-27T04:17:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:00 smithi167 ceph-mon[32751]: pgmap v19139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:00 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34902.524749 secs 2024-04-27T04:17:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:00 smithi090 ceph-mon[29180]: pgmap v19139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:00 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34902.524749 secs 2024-04-27T04:17:02.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34903.402858 secs 2024-04-27T04:17:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34903.402858 secs 2024-04-27T04:17:02.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34903.402858 secs 2024-04-27T04:17:03.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:02 smithi119 ceph-mon[32752]: pgmap v19140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:03.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:02 smithi167 ceph-mon[32751]: pgmap v19140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:03.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:02 smithi090 ceph-mon[29180]: pgmap v19140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:05.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:04 smithi119 ceph-mon[32752]: pgmap v19141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:05.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:04 smithi167 ceph-mon[32751]: pgmap v19141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:05.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:04 smithi090 ceph-mon[29180]: pgmap v19141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:06.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:05 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34907.524820 secs 2024-04-27T04:17:06.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:05 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34907.524820 secs 2024-04-27T04:17:06.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:05 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34907.524820 secs 2024-04-27T04:17:07.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:06 smithi090 ceph-mon[29180]: pgmap v19142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:07.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34908.403001 secs 2024-04-27T04:17:07.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:06 smithi119 ceph-mon[32752]: pgmap v19142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:07.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34908.403001 secs 2024-04-27T04:17:07.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:06 smithi167 ceph-mon[32751]: pgmap v19142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:07.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34908.403001 secs 2024-04-27T04:17:09.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:08 smithi119 ceph-mon[32752]: pgmap v19143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:09.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:08 smithi167 ceph-mon[32751]: pgmap v19143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:09.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:08 smithi090 ceph-mon[29180]: pgmap v19143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:11.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:10 smithi119 ceph-mon[32752]: pgmap v19144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:11.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:10 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34912.524905 secs 2024-04-27T04:17:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:10 smithi167 ceph-mon[32751]: pgmap v19144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:10 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34912.524905 secs 2024-04-27T04:17:11.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:10 smithi090 ceph-mon[29180]: pgmap v19144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:11.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:10 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34912.524905 secs 2024-04-27T04:17:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34913.403126 secs 2024-04-27T04:17:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34913.403126 secs 2024-04-27T04:17:12.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34913.403126 secs 2024-04-27T04:17:13.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:12 smithi119 ceph-mon[32752]: pgmap v19145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:13.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:12 smithi167 ceph-mon[32751]: pgmap v19145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:12 smithi090 ceph-mon[29180]: pgmap v19145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:15.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:14 smithi119 ceph-mon[32752]: pgmap v19146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:15.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:14 smithi167 ceph-mon[32751]: pgmap v19146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:15.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:14 smithi090 ceph-mon[29180]: pgmap v19146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:15 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34917.524963 secs 2024-04-27T04:17:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:15 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34917.524963 secs 2024-04-27T04:17:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:15 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34917.524963 secs 2024-04-27T04:17:17.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:16 smithi119 ceph-mon[32752]: pgmap v19147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:17.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34918.403188 secs 2024-04-27T04:17:17.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:16 smithi167 ceph-mon[32751]: pgmap v19147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:17.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34918.403188 secs 2024-04-27T04:17:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:16 smithi090 ceph-mon[29180]: pgmap v19147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34918.403188 secs 2024-04-27T04:17:19.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:18 smithi119 ceph-mon[32752]: pgmap v19148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:19.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:18 smithi167 ceph-mon[32751]: pgmap v19148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:19.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:18 smithi090 ceph-mon[29180]: pgmap v19148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:20 smithi090 ceph-mon[29180]: pgmap v19149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:20 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34922.525050 secs 2024-04-27T04:17:21.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:20 smithi119 ceph-mon[32752]: pgmap v19149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:21.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:20 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34922.525050 secs 2024-04-27T04:17:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:20 smithi167 ceph-mon[32751]: pgmap v19149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:20 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34922.525050 secs 2024-04-27T04:17:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34923.403333 secs 2024-04-27T04:17:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34923.403333 secs 2024-04-27T04:17:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34923.403333 secs 2024-04-27T04:17:23.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:22 smithi119 ceph-mon[32752]: pgmap v19150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:23.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:22 smithi167 ceph-mon[32751]: pgmap v19150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:23.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:22 smithi090 ceph-mon[29180]: pgmap v19150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:25.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:24 smithi119 ceph-mon[32752]: pgmap v19151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:25.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:24 smithi167 ceph-mon[32751]: pgmap v19151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:25.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:24 smithi090 ceph-mon[29180]: pgmap v19151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:25 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34927.525132 secs 2024-04-27T04:17:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:25 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34927.525132 secs 2024-04-27T04:17:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:25 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34927.525132 secs 2024-04-27T04:17:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:26 smithi119 ceph-mon[32752]: pgmap v19152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34928.403442 secs 2024-04-27T04:17:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:26 smithi167 ceph-mon[32751]: pgmap v19152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34928.403442 secs 2024-04-27T04:17:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:26 smithi090 ceph-mon[29180]: pgmap v19152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34928.403442 secs 2024-04-27T04:17:29.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:28 smithi119 ceph-mon[32752]: pgmap v19153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:29.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:28 smithi167 ceph-mon[32751]: pgmap v19153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:29.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:28 smithi090 ceph-mon[29180]: pgmap v19153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:31.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:30 smithi119 ceph-mon[32752]: pgmap v19154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:31.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:30 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34932.525211 secs 2024-04-27T04:17:31.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:30 smithi167 ceph-mon[32751]: pgmap v19154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:31.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:30 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34932.525211 secs 2024-04-27T04:17:31.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:30 smithi090 ceph-mon[29180]: pgmap v19154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:31.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:30 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34932.525211 secs 2024-04-27T04:17:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34933.403509 secs 2024-04-27T04:17:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34933.403509 secs 2024-04-27T04:17:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34933.403509 secs 2024-04-27T04:17:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:17:33.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:32 smithi119 ceph-mon[32752]: pgmap v19155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:33.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:33.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:17:33.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:17:33.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:32 smithi167 ceph-mon[32751]: pgmap v19155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:33.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:33.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:17:33.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:17:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:32 smithi090 ceph-mon[29180]: pgmap v19155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:17:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:17:33.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:17:34.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:34 smithi119 ceph-mon[32752]: pgmap v19156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:17:34.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:34 smithi119 ceph-mon[32752]: pgmap v19157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:17:34.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:34 smithi167 ceph-mon[32751]: pgmap v19156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:17:34.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:34 smithi167 ceph-mon[32751]: pgmap v19157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:17:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:33 smithi090 ceph-mon[29180]: pgmap v19156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:17:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:33 smithi090 ceph-mon[29180]: pgmap v19157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:17:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:36 smithi119 ceph-mon[32752]: pgmap v19158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:36 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34937.525293 secs 2024-04-27T04:17:36.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:36 smithi167 ceph-mon[32751]: pgmap v19158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:36.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:36 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34937.525293 secs 2024-04-27T04:17:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:36 smithi090 ceph-mon[29180]: pgmap v19158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:36 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34937.525293 secs 2024-04-27T04:17:37.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:37 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 34938.403641 secs 2024-04-27T04:17:37.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:37 smithi119 ceph-mon[32752]: slow request 23042.143772 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:17:37.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:37 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 34938.403641 secs 2024-04-27T04:17:37.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:37 smithi167 ceph-mon[32751]: slow request 23042.143772 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:17:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:37 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 34938.403641 secs 2024-04-27T04:17:37.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:37 smithi090 ceph-mon[29180]: slow request 23042.143772 seconds old, received at 2024-04-26T21:53:34.010513+0000: client_request(client.17331:1 getattr pAsLsXsFs #0x1 2024-04-26T21:53:34.003844+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:17:38.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:38 smithi119 ceph-mon[32752]: pgmap v19159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:38.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:38 smithi167 ceph-mon[32751]: pgmap v19159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:38 smithi090 ceph-mon[29180]: pgmap v19159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:40.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:40 smithi119 ceph-mon[32752]: pgmap v19160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:40.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:40 smithi167 ceph-mon[32751]: pgmap v19160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:40 smithi090 ceph-mon[29180]: pgmap v19160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:41.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:41 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34942.525379 secs 2024-04-27T04:17:41.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:41 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34942.525379 secs 2024-04-27T04:17:41.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:41 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34942.525379 secs 2024-04-27T04:17:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:42 smithi119 ceph-mon[32752]: pgmap v19161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34943.403768 secs 2024-04-27T04:17:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:42 smithi167 ceph-mon[32751]: pgmap v19161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34943.403768 secs 2024-04-27T04:17:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:42 smithi090 ceph-mon[29180]: pgmap v19161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:17:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34943.403768 secs 2024-04-27T04:17:44.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:44 smithi119 ceph-mon[32752]: pgmap v19162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:17:44.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:44 smithi167 ceph-mon[32751]: pgmap v19162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:17:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:44 smithi090 ceph-mon[29180]: pgmap v19162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:17:46.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:46 smithi119 ceph-mon[32752]: pgmap v19163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:46.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:46 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34947.525458 secs 2024-04-27T04:17:46.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:46 smithi167 ceph-mon[32751]: pgmap v19163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:46.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:46 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34947.525458 secs 2024-04-27T04:17:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:46 smithi090 ceph-mon[29180]: pgmap v19163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:46 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34947.525458 secs 2024-04-27T04:17:47.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34948.403912 secs 2024-04-27T04:17:47.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34948.403912 secs 2024-04-27T04:17:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34948.403912 secs 2024-04-27T04:17:48.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:48 smithi119 ceph-mon[32752]: pgmap v19164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:48.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:48 smithi167 ceph-mon[32751]: pgmap v19164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:48.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:48 smithi090 ceph-mon[29180]: pgmap v19164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:50.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:50 smithi119 ceph-mon[32752]: pgmap v19165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:50.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:50 smithi167 ceph-mon[32751]: pgmap v19165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:50.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:50 smithi090 ceph-mon[29180]: pgmap v19165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:51 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34952.525545 secs 2024-04-27T04:17:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:51 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34952.525545 secs 2024-04-27T04:17:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:51 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34952.525545 secs 2024-04-27T04:17:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:52 smithi119 ceph-mon[32752]: pgmap v19166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34953.404000 secs 2024-04-27T04:17:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:52 smithi167 ceph-mon[32751]: pgmap v19166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34953.404000 secs 2024-04-27T04:17:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:52 smithi090 ceph-mon[29180]: pgmap v19166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34953.404000 secs 2024-04-27T04:17:54.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:54 smithi119 ceph-mon[32752]: pgmap v19167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:54.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:54 smithi167 ceph-mon[32751]: pgmap v19167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:54.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:54 smithi090 ceph-mon[29180]: pgmap v19167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:56 smithi119 ceph-mon[32752]: pgmap v19168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:56 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34957.525627 secs 2024-04-27T04:17:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:56 smithi167 ceph-mon[32751]: pgmap v19168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:56 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34957.525627 secs 2024-04-27T04:17:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:56 smithi090 ceph-mon[29180]: pgmap v19168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:17:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:56 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34957.525627 secs 2024-04-27T04:17:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34958.404083 secs 2024-04-27T04:17:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34958.404083 secs 2024-04-27T04:17:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34958.404083 secs 2024-04-27T04:17:58.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:17:58 smithi167 ceph-mon[32751]: pgmap v19169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:17:58 smithi090 ceph-mon[29180]: pgmap v19169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:17:58.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:17:58 smithi119 ceph-mon[32752]: pgmap v19169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:00.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:00 smithi167 ceph-mon[32751]: pgmap v19170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:00.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:00 smithi090 ceph-mon[29180]: pgmap v19170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:00.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:00 smithi119 ceph-mon[32752]: pgmap v19170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:01 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34962.525712 secs 2024-04-27T04:18:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:01 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34962.525712 secs 2024-04-27T04:18:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:01 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34962.525712 secs 2024-04-27T04:18:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:02 smithi167 ceph-mon[32751]: pgmap v19171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34963.404185 secs 2024-04-27T04:18:02.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:02 smithi090 ceph-mon[29180]: pgmap v19171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:02.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34963.404185 secs 2024-04-27T04:18:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:02 smithi119 ceph-mon[32752]: pgmap v19171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34963.404185 secs 2024-04-27T04:18:04.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:04 smithi090 ceph-mon[29180]: pgmap v19172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:04.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:04 smithi119 ceph-mon[32752]: pgmap v19172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:04.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:04 smithi167 ceph-mon[32751]: pgmap v19172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:06.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:06 smithi090 ceph-mon[29180]: pgmap v19173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:06.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:06 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34967.525813 secs 2024-04-27T04:18:06.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:06 smithi119 ceph-mon[32752]: pgmap v19173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:06.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:06 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34967.525813 secs 2024-04-27T04:18:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:06 smithi167 ceph-mon[32751]: pgmap v19173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:06 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34967.525813 secs 2024-04-27T04:18:07.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34968.404311 secs 2024-04-27T04:18:07.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34968.404311 secs 2024-04-27T04:18:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34968.404311 secs 2024-04-27T04:18:08.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:08 smithi090 ceph-mon[29180]: pgmap v19174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:08.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:08 smithi119 ceph-mon[32752]: pgmap v19174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:08.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:08 smithi167 ceph-mon[32751]: pgmap v19174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:10 smithi090 ceph-mon[29180]: pgmap v19175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:10.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:10 smithi119 ceph-mon[32752]: pgmap v19175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:10.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:10 smithi167 ceph-mon[32751]: pgmap v19175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:11.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:11 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34972.525901 secs 2024-04-27T04:18:11.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:11 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34972.525901 secs 2024-04-27T04:18:11.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:11 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34972.525901 secs 2024-04-27T04:18:12.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:12 smithi090 ceph-mon[29180]: pgmap v19176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:12.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34973.404408 secs 2024-04-27T04:18:12.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:12 smithi119 ceph-mon[32752]: pgmap v19176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:12.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34973.404408 secs 2024-04-27T04:18:12.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:12 smithi167 ceph-mon[32751]: pgmap v19176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:12.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34973.404408 secs 2024-04-27T04:18:14.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:14 smithi090 ceph-mon[29180]: pgmap v19177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:14.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:14 smithi119 ceph-mon[32752]: pgmap v19177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:14.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:14 smithi167 ceph-mon[32751]: pgmap v19177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:16 smithi090 ceph-mon[29180]: pgmap v19178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:16 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34977.525982 secs 2024-04-27T04:18:16.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:16 smithi119 ceph-mon[32752]: pgmap v19178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:16.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:16 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34977.525982 secs 2024-04-27T04:18:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:16 smithi167 ceph-mon[32751]: pgmap v19178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:16 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34977.525982 secs 2024-04-27T04:18:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34978.404485 secs 2024-04-27T04:18:17.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34978.404485 secs 2024-04-27T04:18:17.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34978.404485 secs 2024-04-27T04:18:18.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:18 smithi090 ceph-mon[29180]: pgmap v19179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:18.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:18 smithi119 ceph-mon[32752]: pgmap v19179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:18.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:18 smithi167 ceph-mon[32751]: pgmap v19179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:20.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:20 smithi090 ceph-mon[29180]: pgmap v19180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:20.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:20 smithi119 ceph-mon[32752]: pgmap v19180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:20.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:20 smithi167 ceph-mon[32751]: pgmap v19180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:21.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:21 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34982.526087 secs 2024-04-27T04:18:21.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:21 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34982.526087 secs 2024-04-27T04:18:21.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:21 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34982.526087 secs 2024-04-27T04:18:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:22 smithi090 ceph-mon[29180]: pgmap v19181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34983.404610 secs 2024-04-27T04:18:22.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:22 smithi119 ceph-mon[32752]: pgmap v19181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:22.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34983.404610 secs 2024-04-27T04:18:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:22 smithi167 ceph-mon[32751]: pgmap v19181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34983.404610 secs 2024-04-27T04:18:24.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:24 smithi090 ceph-mon[29180]: pgmap v19182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:24.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:24 smithi119 ceph-mon[32752]: pgmap v19182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:24.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:24 smithi167 ceph-mon[32751]: pgmap v19182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:26 smithi090 ceph-mon[29180]: pgmap v19183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:26 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34987.526147 secs 2024-04-27T04:18:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:26 smithi119 ceph-mon[32752]: pgmap v19183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:26 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34987.526147 secs 2024-04-27T04:18:26.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:26 smithi167 ceph-mon[32751]: pgmap v19183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:26.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:26 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34987.526147 secs 2024-04-27T04:18:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34988.404740 secs 2024-04-27T04:18:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34988.404740 secs 2024-04-27T04:18:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34988.404740 secs 2024-04-27T04:18:28.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:28 smithi119 ceph-mon[32752]: pgmap v19184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:28.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:28 smithi167 ceph-mon[32751]: pgmap v19184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:28.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:28 smithi090 ceph-mon[29180]: pgmap v19184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:30.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:30 smithi119 ceph-mon[32752]: pgmap v19185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:30.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:30 smithi167 ceph-mon[32751]: pgmap v19185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:30.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:30 smithi090 ceph-mon[29180]: pgmap v19185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:31 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34992.526208 secs 2024-04-27T04:18:31.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:31 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34992.526208 secs 2024-04-27T04:18:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:31 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34992.526208 secs 2024-04-27T04:18:32.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:32 smithi119 ceph-mon[32752]: pgmap v19186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:32.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34993.404839 secs 2024-04-27T04:18:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:32 smithi167 ceph-mon[32751]: pgmap v19186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34993.404839 secs 2024-04-27T04:18:32.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:32 smithi090 ceph-mon[29180]: pgmap v19186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:32.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34993.404839 secs 2024-04-27T04:18:33.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:33.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:18:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:34 smithi119 ceph-mon[32752]: pgmap v19187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:34.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:18:34.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:34 smithi167 ceph-mon[32751]: pgmap v19187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:34.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:34.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:34.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:18:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:34 smithi090 ceph-mon[29180]: pgmap v19187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:18:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:18:34.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:18:35.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:35 smithi119 ceph-mon[32752]: pgmap v19188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:18:35.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:35 smithi119 ceph-mon[32752]: pgmap v19189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:35.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:35 smithi167 ceph-mon[32751]: pgmap v19188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:18:35.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:35 smithi167 ceph-mon[32751]: pgmap v19189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:35 smithi090 ceph-mon[29180]: pgmap v19188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:18:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:35 smithi090 ceph-mon[29180]: pgmap v19189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:36 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 34997.526267 secs 2024-04-27T04:18:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:36 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 34997.526267 secs 2024-04-27T04:18:36.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:36 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 34997.526267 secs 2024-04-27T04:18:37.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:37 smithi090 ceph-mon[29180]: pgmap v19190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:37.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 34998.404992 secs 2024-04-27T04:18:37.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:37 smithi119 ceph-mon[32752]: pgmap v19190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:37.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 34998.404992 secs 2024-04-27T04:18:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:37 smithi167 ceph-mon[32751]: pgmap v19190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 34998.404992 secs 2024-04-27T04:18:39.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:39 smithi119 ceph-mon[32752]: pgmap v19191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:18:39.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:39 smithi167 ceph-mon[32751]: pgmap v19191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:18:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:39 smithi090 ceph-mon[29180]: pgmap v19191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:18:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:41 smithi119 ceph-mon[32752]: pgmap v19192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:41 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35002.526328 secs 2024-04-27T04:18:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:41 smithi167 ceph-mon[32751]: pgmap v19192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:41 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35002.526328 secs 2024-04-27T04:18:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:41 smithi090 ceph-mon[29180]: pgmap v19192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:41 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35002.526328 secs 2024-04-27T04:18:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35003.405098 secs 2024-04-27T04:18:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35003.405098 secs 2024-04-27T04:18:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35003.405098 secs 2024-04-27T04:18:43.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:43 smithi119 ceph-mon[32752]: pgmap v19193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:43.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:43 smithi167 ceph-mon[32751]: pgmap v19193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:43.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:43 smithi090 ceph-mon[29180]: pgmap v19193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:18:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:45 smithi119 ceph-mon[32752]: pgmap v19194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:18:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:45 smithi167 ceph-mon[32751]: pgmap v19194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:18:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:45 smithi090 ceph-mon[29180]: pgmap v19194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:18:46.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:46 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35007.526412 secs 2024-04-27T04:18:46.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:46 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35007.526412 secs 2024-04-27T04:18:46.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:46 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35007.526412 secs 2024-04-27T04:18:47.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:47 smithi119 ceph-mon[32752]: pgmap v19195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:47.453 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35008.405174 secs 2024-04-27T04:18:47.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:47 smithi167 ceph-mon[32751]: pgmap v19195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:47.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35008.405174 secs 2024-04-27T04:18:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:47 smithi090 ceph-mon[29180]: pgmap v19195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35008.405174 secs 2024-04-27T04:18:49.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:49 smithi119 ceph-mon[32752]: pgmap v19196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:49.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:49 smithi167 ceph-mon[32751]: pgmap v19196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:49.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:49 smithi090 ceph-mon[29180]: pgmap v19196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:51 smithi119 ceph-mon[32752]: pgmap v19197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:51 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35012.526495 secs 2024-04-27T04:18:51.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:51 smithi167 ceph-mon[32751]: pgmap v19197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:51.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:51 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35012.526495 secs 2024-04-27T04:18:51.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:51 smithi090 ceph-mon[29180]: pgmap v19197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:51.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:51 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35012.526495 secs 2024-04-27T04:18:52.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35013.405313 secs 2024-04-27T04:18:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35013.405313 secs 2024-04-27T04:18:52.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35013.405313 secs 2024-04-27T04:18:53.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:53 smithi119 ceph-mon[32752]: pgmap v19198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:53.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:53 smithi167 ceph-mon[32751]: pgmap v19198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:53.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:53 smithi090 ceph-mon[29180]: pgmap v19198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:55.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:55 smithi119 ceph-mon[32752]: pgmap v19199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:55.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:55 smithi167 ceph-mon[32751]: pgmap v19199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:55 smithi090 ceph-mon[29180]: pgmap v19199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:18:56.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:56 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35017.526579 secs 2024-04-27T04:18:56.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:56 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35017.526579 secs 2024-04-27T04:18:56.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:56 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35017.526579 secs 2024-04-27T04:18:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:57 smithi119 ceph-mon[32752]: pgmap v19200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35018.405451 secs 2024-04-27T04:18:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:57 smithi167 ceph-mon[32751]: pgmap v19200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35018.405451 secs 2024-04-27T04:18:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:57 smithi090 ceph-mon[29180]: pgmap v19200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35018.405451 secs 2024-04-27T04:18:59.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:18:59 smithi119 ceph-mon[32752]: pgmap v19201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:59.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:18:59 smithi167 ceph-mon[32751]: pgmap v19201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:18:59.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:18:59 smithi090 ceph-mon[29180]: pgmap v19201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:01 smithi119 ceph-mon[32752]: pgmap v19202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:01 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35022.526663 secs 2024-04-27T04:19:01.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:01 smithi167 ceph-mon[32751]: pgmap v19202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:01.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:01 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35022.526663 secs 2024-04-27T04:19:01.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:01 smithi090 ceph-mon[29180]: pgmap v19202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:01.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:01 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35022.526663 secs 2024-04-27T04:19:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35023.405543 secs 2024-04-27T04:19:02.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35023.405543 secs 2024-04-27T04:19:02.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35023.405543 secs 2024-04-27T04:19:03.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:03 smithi119 ceph-mon[32752]: pgmap v19203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:03.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:03 smithi167 ceph-mon[32751]: pgmap v19203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:03.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:03 smithi090 ceph-mon[29180]: pgmap v19203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:04.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:04 smithi119 ceph-mon[32752]: pgmap v19204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:04.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:04 smithi167 ceph-mon[32751]: pgmap v19204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:04.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:04 smithi090 ceph-mon[29180]: pgmap v19204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:06.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:06 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35027.526720 secs 2024-04-27T04:19:06.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:06 smithi119 ceph-mon[32752]: pgmap v19205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:06.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:06 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35027.526720 secs 2024-04-27T04:19:06.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:06 smithi167 ceph-mon[32751]: pgmap v19205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:06 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35027.526720 secs 2024-04-27T04:19:06.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:06 smithi090 ceph-mon[29180]: pgmap v19205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:07.775 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35028.405656 secs 2024-04-27T04:19:07.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35028.405656 secs 2024-04-27T04:19:07.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35028.405656 secs 2024-04-27T04:19:08.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:08 smithi119 ceph-mon[32752]: pgmap v19206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:08.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:08 smithi167 ceph-mon[32751]: pgmap v19206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:08.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:08 smithi090 ceph-mon[29180]: pgmap v19206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:10.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:10 smithi119 ceph-mon[32752]: pgmap v19207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:10.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:10 smithi167 ceph-mon[32751]: pgmap v19207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:10.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:10 smithi090 ceph-mon[29180]: pgmap v19207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:11.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:11 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35032.526805 secs 2024-04-27T04:19:11.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:11 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35032.526805 secs 2024-04-27T04:19:11.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:11 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35032.526805 secs 2024-04-27T04:19:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35033.405770 secs 2024-04-27T04:19:12.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:12 smithi119 ceph-mon[32752]: pgmap v19208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35033.405770 secs 2024-04-27T04:19:12.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:12 smithi167 ceph-mon[32751]: pgmap v19208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:12.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35033.405770 secs 2024-04-27T04:19:12.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:12 smithi090 ceph-mon[29180]: pgmap v19208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:14.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:14 smithi119 ceph-mon[32752]: pgmap v19209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:14.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:14 smithi167 ceph-mon[32751]: pgmap v19209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:14.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:14 smithi090 ceph-mon[29180]: pgmap v19209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:16.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:16 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35037.526865 secs 2024-04-27T04:19:16.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:16 smithi119 ceph-mon[32752]: pgmap v19210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:16.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:16 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35037.526865 secs 2024-04-27T04:19:16.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:16 smithi167 ceph-mon[32751]: pgmap v19210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:16.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:16 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35037.526865 secs 2024-04-27T04:19:16.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:16 smithi090 ceph-mon[29180]: pgmap v19210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:17.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35038.405896 secs 2024-04-27T04:19:17.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35038.405896 secs 2024-04-27T04:19:17.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35038.405896 secs 2024-04-27T04:19:18.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:18 smithi119 ceph-mon[32752]: pgmap v19211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:18.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:18 smithi167 ceph-mon[32751]: pgmap v19211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:18.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:18 smithi090 ceph-mon[29180]: pgmap v19211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:20.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:20 smithi119 ceph-mon[32752]: pgmap v19212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:20.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:20 smithi167 ceph-mon[32751]: pgmap v19212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:20.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:20 smithi090 ceph-mon[29180]: pgmap v19212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:21.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:21 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35042.526955 secs 2024-04-27T04:19:21.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:21 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35042.526955 secs 2024-04-27T04:19:21.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:21 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35042.526955 secs 2024-04-27T04:19:22.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35043.406027 secs 2024-04-27T04:19:22.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:22 smithi119 ceph-mon[32752]: pgmap v19213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:22.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35043.406027 secs 2024-04-27T04:19:22.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:22 smithi167 ceph-mon[32751]: pgmap v19213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:22.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35043.406027 secs 2024-04-27T04:19:22.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:22 smithi090 ceph-mon[29180]: pgmap v19213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:24.771 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:24 smithi090 ceph-mon[29180]: pgmap v19214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:24.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:24 smithi119 ceph-mon[32752]: pgmap v19214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:24.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:24 smithi167 ceph-mon[32751]: pgmap v19214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:26.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:26 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35047.527015 secs 2024-04-27T04:19:26.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:26 smithi119 ceph-mon[32752]: pgmap v19215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:26.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:26 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35047.527015 secs 2024-04-27T04:19:26.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:26 smithi167 ceph-mon[32751]: pgmap v19215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:26.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:26 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35047.527015 secs 2024-04-27T04:19:26.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:26 smithi090 ceph-mon[29180]: pgmap v19215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:27.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35048.406163 secs 2024-04-27T04:19:27.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35048.406163 secs 2024-04-27T04:19:27.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35048.406163 secs 2024-04-27T04:19:28.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:28 smithi119 ceph-mon[32752]: pgmap v19216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:28.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:28 smithi167 ceph-mon[32751]: pgmap v19216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:28.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:28 smithi090 ceph-mon[29180]: pgmap v19216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:30.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:30 smithi119 ceph-mon[32752]: pgmap v19217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:30.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:30 smithi167 ceph-mon[32751]: pgmap v19217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:30.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:30 smithi090 ceph-mon[29180]: pgmap v19217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:31.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:31 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35052.527115 secs 2024-04-27T04:19:31.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:31 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35052.527115 secs 2024-04-27T04:19:31.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:31 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35052.527115 secs 2024-04-27T04:19:32.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35053.406317 secs 2024-04-27T04:19:32.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:32 smithi119 ceph-mon[32752]: pgmap v19218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:32.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35053.406317 secs 2024-04-27T04:19:32.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:32 smithi167 ceph-mon[32751]: pgmap v19218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35053.406317 secs 2024-04-27T04:19:32.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:32 smithi090 ceph-mon[29180]: pgmap v19218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:33.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:33.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:33.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:19:34.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:34 smithi119 ceph-mon[32752]: pgmap v19219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:34.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:34.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:34.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:34 smithi119 ceph-mon[32752]: pgmap v19220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:19:34.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:34 smithi119 ceph-mon[32752]: pgmap v19221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:34.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:19:34.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:34 smithi167 ceph-mon[32751]: pgmap v19219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:34.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:34.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:34.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:34 smithi167 ceph-mon[32751]: pgmap v19220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:19:34.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:34 smithi167 ceph-mon[32751]: pgmap v19221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:34.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:19:34.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:34 smithi090 ceph-mon[29180]: pgmap v19219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:34.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:19:34.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:19:34.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:34 smithi090 ceph-mon[29180]: pgmap v19220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:19:34.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:34 smithi090 ceph-mon[29180]: pgmap v19221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:34.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:19:36.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:35 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35057.527194 secs 2024-04-27T04:19:36.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:35 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35057.527194 secs 2024-04-27T04:19:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:35 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35057.527194 secs 2024-04-27T04:19:37.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35058.406428 secs 2024-04-27T04:19:37.026 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:36 smithi090 ceph-mon[29180]: pgmap v19222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:37.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35058.406428 secs 2024-04-27T04:19:37.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:36 smithi119 ceph-mon[32752]: pgmap v19222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:37.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35058.406428 secs 2024-04-27T04:19:37.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:36 smithi167 ceph-mon[32751]: pgmap v19222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:39.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:39 smithi119 ceph-mon[32752]: pgmap v19223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:19:39.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:39 smithi167 ceph-mon[32751]: pgmap v19223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:19:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:39 smithi090 ceph-mon[29180]: pgmap v19223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:19:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:41 smithi119 ceph-mon[32752]: pgmap v19224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:41 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35062.527279 secs 2024-04-27T04:19:41.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:41 smithi167 ceph-mon[32751]: pgmap v19224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:41 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35062.527279 secs 2024-04-27T04:19:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:41 smithi090 ceph-mon[29180]: pgmap v19224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:19:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:41 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35062.527279 secs 2024-04-27T04:19:42.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35063.406500 secs 2024-04-27T04:19:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35063.406500 secs 2024-04-27T04:19:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35063.406500 secs 2024-04-27T04:19:43.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:43 smithi119 ceph-mon[32752]: pgmap v19225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:19:43.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:43 smithi167 ceph-mon[32751]: pgmap v19225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:19:43.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:43 smithi090 ceph-mon[29180]: pgmap v19225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:19:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:45 smithi119 ceph-mon[32752]: pgmap v19226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:19:45.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:45 smithi167 ceph-mon[32751]: pgmap v19226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:19:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:45 smithi090 ceph-mon[29180]: pgmap v19226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:19:46.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:46 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35067.527365 secs 2024-04-27T04:19:46.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:46 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35067.527365 secs 2024-04-27T04:19:46.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:46 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35067.527365 secs 2024-04-27T04:19:47.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35068.406631 secs 2024-04-27T04:19:47.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:47 smithi119 ceph-mon[32752]: pgmap v19227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35068.406631 secs 2024-04-27T04:19:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:47 smithi167 ceph-mon[32751]: pgmap v19227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35068.406631 secs 2024-04-27T04:19:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:47 smithi090 ceph-mon[29180]: pgmap v19227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:48.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:48 smithi119 ceph-mon[32752]: pgmap v19228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:48.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:48 smithi167 ceph-mon[32751]: pgmap v19228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:48.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:48 smithi090 ceph-mon[29180]: pgmap v19228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:51 smithi119 ceph-mon[32752]: pgmap v19229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:51 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35072.527451 secs 2024-04-27T04:19:51.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:51 smithi167 ceph-mon[32751]: pgmap v19229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:51.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:51 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35072.527451 secs 2024-04-27T04:19:51.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:51 smithi090 ceph-mon[29180]: pgmap v19229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:51 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35072.527451 secs 2024-04-27T04:19:52.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35073.406731 secs 2024-04-27T04:19:52.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35073.406731 secs 2024-04-27T04:19:52.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35073.406731 secs 2024-04-27T04:19:53.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:53 smithi119 ceph-mon[32752]: pgmap v19230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:53.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:53 smithi167 ceph-mon[32751]: pgmap v19230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:53.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:53 smithi090 ceph-mon[29180]: pgmap v19230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:55.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:55 smithi119 ceph-mon[32752]: pgmap v19231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:55.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:55 smithi167 ceph-mon[32751]: pgmap v19231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:55 smithi090 ceph-mon[29180]: pgmap v19231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:19:56.515 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:56 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35077.527529 secs 2024-04-27T04:19:56.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:56 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35077.527529 secs 2024-04-27T04:19:56.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:56 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35077.527529 secs 2024-04-27T04:19:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35078.406798 secs 2024-04-27T04:19:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:57 smithi119 ceph-mon[32752]: pgmap v19232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35078.406798 secs 2024-04-27T04:19:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:57 smithi167 ceph-mon[32751]: pgmap v19232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35078.406798 secs 2024-04-27T04:19:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:57 smithi090 ceph-mon[29180]: pgmap v19232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:59.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:19:59 smithi119 ceph-mon[32752]: pgmap v19233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:59.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:19:59 smithi167 ceph-mon[32751]: pgmap v19233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:19:59.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:19:59 smithi090 ceph-mon[29180]: pgmap v19233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:00.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:20:00.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:20:00.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:20:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:01 smithi119 ceph-mon[32752]: pgmap v19234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:01 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35082.527613 secs 2024-04-27T04:20:01.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:01 smithi167 ceph-mon[32751]: pgmap v19234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:01.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:01 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35082.527613 secs 2024-04-27T04:20:01.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:01 smithi090 ceph-mon[29180]: pgmap v19234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:01.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:01 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35082.527613 secs 2024-04-27T04:20:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35083.406910 secs 2024-04-27T04:20:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:02 smithi119 ceph-mon[32752]: pgmap v19235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:02.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35083.406910 secs 2024-04-27T04:20:02.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:02 smithi167 ceph-mon[32751]: pgmap v19235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:02.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35083.406910 secs 2024-04-27T04:20:02.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:02 smithi090 ceph-mon[29180]: pgmap v19235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:05.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:05 smithi119 ceph-mon[32752]: pgmap v19236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:05.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:05 smithi167 ceph-mon[32751]: pgmap v19236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:05.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:05 smithi090 ceph-mon[29180]: pgmap v19236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:06.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:06 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35087.527697 secs 2024-04-27T04:20:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:06 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35087.527697 secs 2024-04-27T04:20:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:06 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35087.527697 secs 2024-04-27T04:20:07.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35088.407046 secs 2024-04-27T04:20:07.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:07 smithi090 ceph-mon[29180]: pgmap v19237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:07.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35088.407046 secs 2024-04-27T04:20:07.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:07 smithi119 ceph-mon[32752]: pgmap v19237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35088.407046 secs 2024-04-27T04:20:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:07 smithi167 ceph-mon[32751]: pgmap v19237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:09.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:09 smithi119 ceph-mon[32752]: pgmap v19238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:09.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:09 smithi167 ceph-mon[32751]: pgmap v19238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:09.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:09 smithi090 ceph-mon[29180]: pgmap v19238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:10.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:20:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:20:10.484+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:20:10.874 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:20:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:20:10.484+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:20:10.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:20:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:20:10.484+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:20:10.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:20:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:20:10.484+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:20:10.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:20:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:20:10.484+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:20:11.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:11 smithi119 ceph-mon[32752]: pgmap v19239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:11.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:11 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35092.527784 secs 2024-04-27T04:20:11.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:11 smithi167 ceph-mon[32751]: pgmap v19239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:11.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:11 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35092.527784 secs 2024-04-27T04:20:11.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:11 smithi090 ceph-mon[29180]: pgmap v19239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:11.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:11 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35092.527784 secs 2024-04-27T04:20:12.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35093.407164 secs 2024-04-27T04:20:12.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35093.407164 secs 2024-04-27T04:20:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35093.407164 secs 2024-04-27T04:20:13.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:13 smithi119 ceph-mon[32752]: pgmap v19240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:13.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:13 smithi167 ceph-mon[32751]: pgmap v19240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:13 smithi090 ceph-mon[29180]: pgmap v19240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:14.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:14 smithi119 ceph-mon[32752]: pgmap v19241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:14.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:14 smithi167 ceph-mon[32751]: pgmap v19241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:14.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:14 smithi090 ceph-mon[29180]: pgmap v19241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:16.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:15 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35097.527863 secs 2024-04-27T04:20:16.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:15 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35097.527863 secs 2024-04-27T04:20:16.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:15 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35097.527863 secs 2024-04-27T04:20:17.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35098.407285 secs 2024-04-27T04:20:17.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:16 smithi119 ceph-mon[32752]: pgmap v19242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:17.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35098.407285 secs 2024-04-27T04:20:17.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:16 smithi167 ceph-mon[32751]: pgmap v19242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35098.407285 secs 2024-04-27T04:20:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:16 smithi090 ceph-mon[29180]: pgmap v19242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:19.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:19 smithi119 ceph-mon[32752]: pgmap v19243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:19.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:19 smithi167 ceph-mon[32751]: pgmap v19243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:19 smithi090 ceph-mon[29180]: pgmap v19243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:21.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:21 smithi119 ceph-mon[32752]: pgmap v19244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:21.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:21 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35102.527935 secs 2024-04-27T04:20:21.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:21 smithi167 ceph-mon[32751]: pgmap v19244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:21.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:21 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35102.527935 secs 2024-04-27T04:20:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:21 smithi090 ceph-mon[29180]: pgmap v19244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:21 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35102.527935 secs 2024-04-27T04:20:22.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35103.407409 secs 2024-04-27T04:20:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35103.407409 secs 2024-04-27T04:20:22.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35103.407409 secs 2024-04-27T04:20:23.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:23 smithi119 ceph-mon[32752]: pgmap v19245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:23.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:23 smithi167 ceph-mon[32751]: pgmap v19245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:23 smithi090 ceph-mon[29180]: pgmap v19245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:20:25.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:25 smithi119 ceph-mon[32752]: pgmap v19246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:25.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:25 smithi167 ceph-mon[32751]: pgmap v19246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:25.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:25 smithi090 ceph-mon[29180]: pgmap v19246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:20:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:26 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35107.528018 secs 2024-04-27T04:20:26.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:26 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35107.528018 secs 2024-04-27T04:20:26.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:26 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35107.528018 secs 2024-04-27T04:20:27.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35108.407476 secs 2024-04-27T04:20:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:27 smithi119 ceph-mon[32752]: pgmap v19247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35108.407476 secs 2024-04-27T04:20:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:27 smithi167 ceph-mon[32751]: pgmap v19247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35108.407476 secs 2024-04-27T04:20:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:27 smithi090 ceph-mon[29180]: pgmap v19247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:28.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:28 smithi119 ceph-mon[32752]: pgmap v19248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:28.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:28 smithi167 ceph-mon[32751]: pgmap v19248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:28.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:28 smithi090 ceph-mon[29180]: pgmap v19248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:31 smithi119 ceph-mon[32752]: pgmap v19249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:31 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35112.528119 secs 2024-04-27T04:20:31.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:31 smithi167 ceph-mon[32751]: pgmap v19249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:31.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:31 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35112.528119 secs 2024-04-27T04:20:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:31 smithi090 ceph-mon[29180]: pgmap v19249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:31 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35112.528119 secs 2024-04-27T04:20:32.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35113.407598 secs 2024-04-27T04:20:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35113.407598 secs 2024-04-27T04:20:32.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35113.407598 secs 2024-04-27T04:20:33.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:33 smithi119 ceph-mon[32752]: pgmap v19250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:33.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:33 smithi167 ceph-mon[32751]: pgmap v19250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:33 smithi090 ceph-mon[29180]: pgmap v19250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:35.550 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:35 smithi090 ceph-mon[29180]: pgmap v19251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:35.551 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:35.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:35 smithi119 ceph-mon[32752]: pgmap v19251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:35.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:35.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:35 smithi167 ceph-mon[32751]: pgmap v19251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:35.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:20:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:36 smithi119 ceph-mon[32752]: 46 slow requests, 0 included below; oldest blocked for > 35117.528190 secs 2024-04-27T04:20:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:36 smithi167 ceph-mon[32751]: 46 slow requests, 0 included below; oldest blocked for > 35117.528190 secs 2024-04-27T04:20:36.569 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:36 smithi090 ceph-mon[29180]: 46 slow requests, 0 included below; oldest blocked for > 35117.528190 secs 2024-04-27T04:20:37.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35118.407714 secs 2024-04-27T04:20:37.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:37 smithi119 ceph-mon[32752]: pgmap v19252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35118.407714 secs 2024-04-27T04:20:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:37 smithi167 ceph-mon[32751]: pgmap v19252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:37.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35118.407714 secs 2024-04-27T04:20:37.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:37 smithi090 ceph-mon[29180]: pgmap v19252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:38 smithi090 ceph-mon[29180]: pgmap v19253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:38.487 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:38 smithi119 ceph-mon[32752]: pgmap v19253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:38.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:38 smithi167 ceph-mon[32751]: pgmap v19253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:39.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: pgmap v19254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:20:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: pgmap v19255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:39.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:39.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: pgmap v19254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:20:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: pgmap v19255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:39.875 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:40.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:40.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:40.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:40.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:40.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:20:40.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:20:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: pgmap v19254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:20:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: pgmap v19255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:40.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:20:41.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35122.528273 secs 2024-04-27T04:20:41.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35122.528273 secs 2024-04-27T04:20:41.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35122.528273 secs 2024-04-27T04:20:42.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:41 smithi119 ceph-mon[32752]: pgmap v19256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:42.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35123.407823 secs 2024-04-27T04:20:42.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:41 smithi167 ceph-mon[32751]: pgmap v19256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:42.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35123.407823 secs 2024-04-27T04:20:42.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:41 smithi090 ceph-mon[29180]: pgmap v19256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:42.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35123.407823 secs 2024-04-27T04:20:43.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:43 smithi090 ceph-mon[29180]: pgmap v19257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 B/s rd, 1 op/s 2024-04-27T04:20:43.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:20:43.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:20:44.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:43 smithi119 ceph-mon[32752]: pgmap v19257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 B/s rd, 1 op/s 2024-04-27T04:20:44.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:20:44.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:20:44.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:43 smithi167 ceph-mon[32751]: pgmap v19257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 595 B/s rd, 1 op/s 2024-04-27T04:20:44.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:20:44.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:20:46.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:45 smithi119 ceph-mon[32752]: pgmap v19258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:46.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35127.528350 secs 2024-04-27T04:20:46.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:45 smithi167 ceph-mon[32751]: pgmap v19258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:46.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35127.528350 secs 2024-04-27T04:20:46.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:45 smithi090 ceph-mon[29180]: pgmap v19258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:46.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35127.528350 secs 2024-04-27T04:20:47.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35128.407941 secs 2024-04-27T04:20:47.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35128.407941 secs 2024-04-27T04:20:47.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35128.407941 secs 2024-04-27T04:20:48.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:47 smithi119 ceph-mon[32752]: pgmap v19259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:48.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:47 smithi167 ceph-mon[32751]: pgmap v19259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:48.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:47 smithi090 ceph-mon[29180]: pgmap v19259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:49.817 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:49 smithi119 ceph-mon[32752]: pgmap v19260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:20:50.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:49 smithi167 ceph-mon[32751]: pgmap v19260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:20:50.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:49 smithi090 ceph-mon[29180]: pgmap v19260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:20:51.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35132.528442 secs 2024-04-27T04:20:51.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35132.528442 secs 2024-04-27T04:20:51.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35132.528442 secs 2024-04-27T04:20:52.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:51 smithi167 ceph-mon[32751]: pgmap v19261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:52.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35133.408058 secs 2024-04-27T04:20:52.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:51 smithi090 ceph-mon[29180]: pgmap v19261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:52.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35133.408058 secs 2024-04-27T04:20:52.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:51 smithi119 ceph-mon[32752]: pgmap v19261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35133.408058 secs 2024-04-27T04:20:54.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:53 smithi167 ceph-mon[32751]: pgmap v19262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:54.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:53 smithi090 ceph-mon[29180]: pgmap v19262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:54.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:53 smithi119 ceph-mon[32752]: pgmap v19262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:56.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:55 smithi167 ceph-mon[32751]: pgmap v19263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:56.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:55 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35137.528519 secs 2024-04-27T04:20:56.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:55 smithi090 ceph-mon[29180]: pgmap v19263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:56.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:55 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35137.528519 secs 2024-04-27T04:20:56.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:55 smithi119 ceph-mon[32752]: pgmap v19263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:20:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:55 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35137.528519 secs 2024-04-27T04:20:57.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35138.408172 secs 2024-04-27T04:20:57.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35138.408172 secs 2024-04-27T04:20:57.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35138.408172 secs 2024-04-27T04:20:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:57 smithi090 ceph-mon[29180]: pgmap v19264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:58.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:57 smithi119 ceph-mon[32752]: pgmap v19264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:20:58.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:57 smithi167 ceph-mon[32751]: pgmap v19264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:00.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:20:59 smithi090 ceph-mon[29180]: pgmap v19265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:00.240 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:20:59 smithi119 ceph-mon[32752]: pgmap v19265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:00.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:20:59 smithi167 ceph-mon[32751]: pgmap v19265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:01.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35142.528606 secs 2024-04-27T04:21:01.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35142.528606 secs 2024-04-27T04:21:01.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35142.528606 secs 2024-04-27T04:21:02.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:01 smithi090 ceph-mon[29180]: pgmap v19266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35143.408311 secs 2024-04-27T04:21:02.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:01 smithi119 ceph-mon[32752]: pgmap v19266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:02.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35143.408311 secs 2024-04-27T04:21:02.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:01 smithi167 ceph-mon[32751]: pgmap v19266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:02.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35143.408311 secs 2024-04-27T04:21:04.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:03 smithi090 ceph-mon[29180]: pgmap v19267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:04.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:03 smithi119 ceph-mon[32752]: pgmap v19267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:04.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:03 smithi167 ceph-mon[32751]: pgmap v19267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:06.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:05 smithi090 ceph-mon[29180]: pgmap v19268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:06.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:05 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35147.528684 secs 2024-04-27T04:21:06.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:05 smithi119 ceph-mon[32752]: pgmap v19268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:06.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:05 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35147.528684 secs 2024-04-27T04:21:06.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:05 smithi167 ceph-mon[32751]: pgmap v19268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:06.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:05 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35147.528684 secs 2024-04-27T04:21:07.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35148.408444 secs 2024-04-27T04:21:07.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35148.408444 secs 2024-04-27T04:21:07.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35148.408444 secs 2024-04-27T04:21:08.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:07 smithi090 ceph-mon[29180]: pgmap v19269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:08.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:07 smithi119 ceph-mon[32752]: pgmap v19269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:08.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:07 smithi167 ceph-mon[32751]: pgmap v19269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:10.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:09 smithi090 ceph-mon[29180]: pgmap v19270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:10.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:09 smithi119 ceph-mon[32752]: pgmap v19270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:10.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:09 smithi167 ceph-mon[32751]: pgmap v19270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:11.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35152.528771 secs 2024-04-27T04:21:11.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35152.528771 secs 2024-04-27T04:21:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35152.528771 secs 2024-04-27T04:21:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:11 smithi090 ceph-mon[29180]: pgmap v19271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:11 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 35153.408509 secs 2024-04-27T04:21:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:11 smithi090 ceph-mon[29180]: slow request 5762.266799 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:21:12.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:11 smithi119 ceph-mon[32752]: pgmap v19271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:11 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 35153.408509 secs 2024-04-27T04:21:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:11 smithi119 ceph-mon[32752]: slow request 5762.266799 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:21:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:11 smithi167 ceph-mon[32751]: pgmap v19271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:11 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 35153.408509 secs 2024-04-27T04:21:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:11 smithi167 ceph-mon[32751]: slow request 5762.266799 seconds old, received at 2024-04-27T02:45:08.892355+0000: client_request(client.17664:1 getattr pAsLsXsFs #0x1 2024-04-27T02:45:08.887916+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:21:14.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:13 smithi090 ceph-mon[29180]: pgmap v19272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:14.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:13 smithi119 ceph-mon[32752]: pgmap v19272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:14.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:13 smithi167 ceph-mon[32751]: pgmap v19272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:21:16.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:15 smithi090 ceph-mon[29180]: pgmap v19273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:16.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:15 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35157.528857 secs 2024-04-27T04:21:16.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:15 smithi119 ceph-mon[32752]: pgmap v19273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:15 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35157.528857 secs 2024-04-27T04:21:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:15 smithi167 ceph-mon[32751]: pgmap v19273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:15 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35157.528857 secs 2024-04-27T04:21:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35158.408631 secs 2024-04-27T04:21:17.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35158.408631 secs 2024-04-27T04:21:17.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35158.408631 secs 2024-04-27T04:21:18.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:17 smithi090 ceph-mon[29180]: pgmap v19274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:18.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:17 smithi119 ceph-mon[32752]: pgmap v19274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:17 smithi167 ceph-mon[32751]: pgmap v19274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:20.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:19 smithi090 ceph-mon[29180]: pgmap v19275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:20.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:19 smithi119 ceph-mon[32752]: pgmap v19275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:20.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:19 smithi167 ceph-mon[32751]: pgmap v19275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:21.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35162.528945 secs 2024-04-27T04:21:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35162.528945 secs 2024-04-27T04:21:21.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35162.528945 secs 2024-04-27T04:21:22.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:21 smithi119 ceph-mon[32752]: pgmap v19276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35163.408728 secs 2024-04-27T04:21:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:21 smithi167 ceph-mon[32751]: pgmap v19276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35163.408728 secs 2024-04-27T04:21:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:21 smithi090 ceph-mon[29180]: pgmap v19276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35163.408728 secs 2024-04-27T04:21:24.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:23 smithi119 ceph-mon[32752]: pgmap v19277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:24.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:23 smithi167 ceph-mon[32751]: pgmap v19277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:24.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:23 smithi090 ceph-mon[29180]: pgmap v19277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:25 smithi119 ceph-mon[32752]: pgmap v19278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:25 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35167.529025 secs 2024-04-27T04:21:26.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:25 smithi167 ceph-mon[32751]: pgmap v19278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:25 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35167.529025 secs 2024-04-27T04:21:26.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:25 smithi090 ceph-mon[29180]: pgmap v19278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:25 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35167.529025 secs 2024-04-27T04:21:27.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35168.408889 secs 2024-04-27T04:21:27.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35168.408889 secs 2024-04-27T04:21:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35168.408889 secs 2024-04-27T04:21:28.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:27 smithi119 ceph-mon[32752]: pgmap v19279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:28.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:27 smithi167 ceph-mon[32751]: pgmap v19279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:28.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:27 smithi090 ceph-mon[29180]: pgmap v19279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:30.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:29 smithi090 ceph-mon[29180]: pgmap v19280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:30.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:29 smithi119 ceph-mon[32752]: pgmap v19280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:30.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:29 smithi167 ceph-mon[32751]: pgmap v19280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:31.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35172.529114 secs 2024-04-27T04:21:31.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35172.529114 secs 2024-04-27T04:21:31.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35172.529114 secs 2024-04-27T04:21:32.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:31 smithi119 ceph-mon[32752]: pgmap v19281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35173.408967 secs 2024-04-27T04:21:32.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:31 smithi167 ceph-mon[32751]: pgmap v19281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35173.408967 secs 2024-04-27T04:21:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:31 smithi090 ceph-mon[29180]: pgmap v19281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35173.408967 secs 2024-04-27T04:21:34.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:33 smithi119 ceph-mon[32752]: pgmap v19282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:34.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:33 smithi167 ceph-mon[32751]: pgmap v19282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:33 smithi090 ceph-mon[29180]: pgmap v19282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:35 smithi119 ceph-mon[32752]: pgmap v19283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:35 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35177.529198 secs 2024-04-27T04:21:36.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:35 smithi167 ceph-mon[32751]: pgmap v19283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:36.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:35 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35177.529198 secs 2024-04-27T04:21:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:35 smithi090 ceph-mon[29180]: pgmap v19283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:35 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35177.529198 secs 2024-04-27T04:21:37.274 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35178.409106 secs 2024-04-27T04:21:37.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35178.409106 secs 2024-04-27T04:21:37.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35178.409106 secs 2024-04-27T04:21:38.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:37 smithi119 ceph-mon[32752]: pgmap v19284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:38.315 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:37 smithi167 ceph-mon[32751]: pgmap v19284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:37 smithi090 ceph-mon[29180]: pgmap v19284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:40.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:39 smithi167 ceph-mon[32751]: pgmap v19285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:40.247 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:21:40.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:39 smithi119 ceph-mon[32752]: pgmap v19285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:40.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:21:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:39 smithi090 ceph-mon[29180]: pgmap v19285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:21:41.278 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35182.529282 secs 2024-04-27T04:21:41.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35182.529282 secs 2024-04-27T04:21:41.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35182.529282 secs 2024-04-27T04:21:42.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:41 smithi119 ceph-mon[32752]: pgmap v19286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35183.409255 secs 2024-04-27T04:21:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:41 smithi167 ceph-mon[32751]: pgmap v19286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35183.409255 secs 2024-04-27T04:21:42.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:41 smithi090 ceph-mon[29180]: pgmap v19286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:42.373 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35183.409255 secs 2024-04-27T04:21:44.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: pgmap v19287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:44.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:21:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:21:44.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: pgmap v19288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 1 op/s 2024-04-27T04:21:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: pgmap v19289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T04:21:44.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: pgmap v19287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:21:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:21:44.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: pgmap v19288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 1 op/s 2024-04-27T04:21:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: pgmap v19289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T04:21:44.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: pgmap v19287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:21:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:21:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: pgmap v19288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 980 B/s rd, 1 op/s 2024-04-27T04:21:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: pgmap v19289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T04:21:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:21:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:45 smithi167 ceph-mon[32751]: pgmap v19290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:45.571 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:45 smithi090 ceph-mon[29180]: pgmap v19290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:45.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:45 smithi119 ceph-mon[32752]: pgmap v19290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:46.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:46 smithi167 ceph-mon[32751]: 19 slow requests, 1 included below; oldest blocked for > 35187.529360 secs 2024-04-27T04:21:46.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:46 smithi167 ceph-mon[32751]: slow request 724.975964 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:21:46.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:46 smithi090 ceph-mon[29180]: 19 slow requests, 1 included below; oldest blocked for > 35187.529360 secs 2024-04-27T04:21:46.584 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:46 smithi090 ceph-mon[29180]: slow request 724.975964 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:21:46.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:46 smithi119 ceph-mon[32752]: 19 slow requests, 1 included below; oldest blocked for > 35187.529360 secs 2024-04-27T04:21:46.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:46 smithi119 ceph-mon[32752]: slow request 724.975964 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:21:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35188.409421 secs 2024-04-27T04:21:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:47 smithi167 ceph-mon[32751]: pgmap v19291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T04:21:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35188.409421 secs 2024-04-27T04:21:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:47 smithi090 ceph-mon[29180]: pgmap v19291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T04:21:47.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35188.409421 secs 2024-04-27T04:21:47.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:47 smithi119 ceph-mon[32752]: pgmap v19291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 606 B/s rd, 1 op/s 2024-04-27T04:21:49.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:49 smithi167 ceph-mon[32751]: pgmap v19292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:49.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:49 smithi090 ceph-mon[29180]: pgmap v19292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:49.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:49 smithi119 ceph-mon[32752]: pgmap v19292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:51.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:50 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35192.529443 secs 2024-04-27T04:21:51.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:50 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35192.529443 secs 2024-04-27T04:21:51.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:50 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35192.529443 secs 2024-04-27T04:21:52.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35193.409551 secs 2024-04-27T04:21:52.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:51 smithi119 ceph-mon[32752]: pgmap v19293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:52.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35193.409551 secs 2024-04-27T04:21:52.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:51 smithi167 ceph-mon[32751]: pgmap v19293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:52.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35193.409551 secs 2024-04-27T04:21:52.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:51 smithi090 ceph-mon[29180]: pgmap v19293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:53.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:53 smithi167 ceph-mon[32751]: pgmap v19294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:21:53.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:53 smithi090 ceph-mon[29180]: pgmap v19294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:21:53.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:53 smithi119 ceph-mon[32752]: pgmap v19294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:21:55.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:55 smithi167 ceph-mon[32751]: pgmap v19295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:55.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:55 smithi090 ceph-mon[29180]: pgmap v19295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:55.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:55 smithi119 ceph-mon[32752]: pgmap v19295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:56.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35197.529525 secs 2024-04-27T04:21:56.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35197.529525 secs 2024-04-27T04:21:56.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35197.529525 secs 2024-04-27T04:21:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35198.409661 secs 2024-04-27T04:21:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:57 smithi167 ceph-mon[32751]: pgmap v19296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35198.409661 secs 2024-04-27T04:21:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:57 smithi090 ceph-mon[29180]: pgmap v19296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:57.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35198.409661 secs 2024-04-27T04:21:57.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:57 smithi119 ceph-mon[32752]: pgmap v19296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:21:59.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:21:59 smithi167 ceph-mon[32751]: pgmap v19297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:59.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:21:59 smithi090 ceph-mon[29180]: pgmap v19297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:21:59.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:21:59 smithi119 ceph-mon[32752]: pgmap v19297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:01.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:00 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35202.529609 secs 2024-04-27T04:22:01.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:00 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35202.529609 secs 2024-04-27T04:22:01.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:00 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35202.529609 secs 2024-04-27T04:22:02.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35203.409724 secs 2024-04-27T04:22:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:01 smithi119 ceph-mon[32752]: pgmap v19298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:02.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35203.409724 secs 2024-04-27T04:22:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:01 smithi167 ceph-mon[32751]: pgmap v19298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35203.409724 secs 2024-04-27T04:22:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:01 smithi090 ceph-mon[29180]: pgmap v19298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:03.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:03 smithi167 ceph-mon[32751]: pgmap v19299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:03 smithi090 ceph-mon[29180]: pgmap v19299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:03.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:03 smithi119 ceph-mon[32752]: pgmap v19299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:05.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:05 smithi167 ceph-mon[32751]: pgmap v19300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:05.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:05 smithi090 ceph-mon[29180]: pgmap v19300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:05.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:05 smithi119 ceph-mon[32752]: pgmap v19300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35207.529690 secs 2024-04-27T04:22:06.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35207.529690 secs 2024-04-27T04:22:06.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35207.529690 secs 2024-04-27T04:22:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35208.409851 secs 2024-04-27T04:22:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:07 smithi167 ceph-mon[32751]: pgmap v19301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:07.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35208.409851 secs 2024-04-27T04:22:07.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:07 smithi090 ceph-mon[29180]: pgmap v19301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:07.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35208.409851 secs 2024-04-27T04:22:07.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:07 smithi119 ceph-mon[32752]: pgmap v19301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:09.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:09 smithi167 ceph-mon[32751]: pgmap v19302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:09.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:09 smithi090 ceph-mon[29180]: pgmap v19302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:09.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:09 smithi119 ceph-mon[32752]: pgmap v19302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:11.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:10 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35212.529775 secs 2024-04-27T04:22:11.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:10 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35212.529775 secs 2024-04-27T04:22:11.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:10 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35212.529775 secs 2024-04-27T04:22:12.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35213.409951 secs 2024-04-27T04:22:12.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:11 smithi119 ceph-mon[32752]: pgmap v19303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35213.409951 secs 2024-04-27T04:22:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:11 smithi167 ceph-mon[32751]: pgmap v19303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:12.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35213.409951 secs 2024-04-27T04:22:12.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:11 smithi090 ceph-mon[29180]: pgmap v19303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:13.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:13 smithi167 ceph-mon[32751]: pgmap v19304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:13.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:13 smithi090 ceph-mon[29180]: pgmap v19304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:13.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:13 smithi119 ceph-mon[32752]: pgmap v19304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:15.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:15 smithi167 ceph-mon[32751]: pgmap v19305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:15 smithi090 ceph-mon[29180]: pgmap v19305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:15.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:15 smithi119 ceph-mon[32752]: pgmap v19305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35217.529830 secs 2024-04-27T04:22:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35217.529830 secs 2024-04-27T04:22:16.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35217.529830 secs 2024-04-27T04:22:17.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35218.410070 secs 2024-04-27T04:22:17.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:17 smithi167 ceph-mon[32751]: pgmap v19306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35218.410070 secs 2024-04-27T04:22:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:17 smithi090 ceph-mon[29180]: pgmap v19306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:17.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35218.410070 secs 2024-04-27T04:22:17.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:17 smithi119 ceph-mon[32752]: pgmap v19306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:19.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:19 smithi167 ceph-mon[32751]: pgmap v19307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:19 smithi090 ceph-mon[29180]: pgmap v19307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:19.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:19 smithi119 ceph-mon[32752]: pgmap v19307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:21.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:20 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35222.529891 secs 2024-04-27T04:22:21.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:20 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35222.529891 secs 2024-04-27T04:22:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:20 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35222.529891 secs 2024-04-27T04:22:22.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35223.410189 secs 2024-04-27T04:22:22.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:21 smithi119 ceph-mon[32752]: pgmap v19308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:22.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35223.410189 secs 2024-04-27T04:22:22.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:21 smithi167 ceph-mon[32751]: pgmap v19308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35223.410189 secs 2024-04-27T04:22:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:21 smithi090 ceph-mon[29180]: pgmap v19308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:23.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:23 smithi167 ceph-mon[32751]: pgmap v19309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:23 smithi090 ceph-mon[29180]: pgmap v19309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:23.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:23 smithi119 ceph-mon[32752]: pgmap v19309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:25.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:25 smithi167 ceph-mon[32751]: pgmap v19310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:25.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:25 smithi090 ceph-mon[29180]: pgmap v19310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:25.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:25 smithi119 ceph-mon[32752]: pgmap v19310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:26.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35227.529978 secs 2024-04-27T04:22:26.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35227.529978 secs 2024-04-27T04:22:26.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35227.529978 secs 2024-04-27T04:22:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35228.410309 secs 2024-04-27T04:22:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:27 smithi090 ceph-mon[29180]: pgmap v19311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:27.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35228.410309 secs 2024-04-27T04:22:27.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:27 smithi119 ceph-mon[32752]: pgmap v19311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:27.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35228.410309 secs 2024-04-27T04:22:27.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:27 smithi167 ceph-mon[32751]: pgmap v19311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:29.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:29 smithi167 ceph-mon[32751]: pgmap v19312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:29.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:29 smithi090 ceph-mon[29180]: pgmap v19312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:29.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:29 smithi119 ceph-mon[32752]: pgmap v19312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:31.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:30 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35232.530040 secs 2024-04-27T04:22:31.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:30 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35232.530040 secs 2024-04-27T04:22:31.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:30 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35232.530040 secs 2024-04-27T04:22:32.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35233.410427 secs 2024-04-27T04:22:32.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:31 smithi119 ceph-mon[32752]: pgmap v19313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:32.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35233.410427 secs 2024-04-27T04:22:32.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:31 smithi167 ceph-mon[32751]: pgmap v19313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35233.410427 secs 2024-04-27T04:22:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:31 smithi090 ceph-mon[29180]: pgmap v19313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:33.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:33 smithi167 ceph-mon[32751]: pgmap v19314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:33.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:33 smithi090 ceph-mon[29180]: pgmap v19314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:33.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:33 smithi119 ceph-mon[32752]: pgmap v19314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:35.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:35 smithi167 ceph-mon[32751]: pgmap v19315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:35.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:35 smithi090 ceph-mon[29180]: pgmap v19315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:35.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:35 smithi119 ceph-mon[32752]: pgmap v19315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:36.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35237.530096 secs 2024-04-27T04:22:36.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35237.530096 secs 2024-04-27T04:22:36.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35237.530096 secs 2024-04-27T04:22:37.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35238.410539 secs 2024-04-27T04:22:37.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:37 smithi090 ceph-mon[29180]: pgmap v19316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:37.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35238.410539 secs 2024-04-27T04:22:37.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:37 smithi119 ceph-mon[32752]: pgmap v19316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:37.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35238.410539 secs 2024-04-27T04:22:37.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:37 smithi167 ceph-mon[32751]: pgmap v19316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:39.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:39 smithi167 ceph-mon[32751]: pgmap v19317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:39.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:39 smithi090 ceph-mon[29180]: pgmap v19317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:39.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:39 smithi119 ceph-mon[32752]: pgmap v19317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:22:41.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:40 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35242.530181 secs 2024-04-27T04:22:41.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:40 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35242.530181 secs 2024-04-27T04:22:41.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:40 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35242.530181 secs 2024-04-27T04:22:42.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35243.410670 secs 2024-04-27T04:22:42.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:41 smithi119 ceph-mon[32752]: pgmap v19318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:42.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35243.410670 secs 2024-04-27T04:22:42.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:41 smithi167 ceph-mon[32751]: pgmap v19318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:42.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35243.410670 secs 2024-04-27T04:22:42.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:41 smithi090 ceph-mon[29180]: pgmap v19318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:43.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:43 smithi167 ceph-mon[32751]: pgmap v19319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:43 smithi090 ceph-mon[29180]: pgmap v19319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:43.801 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:43 smithi119 ceph-mon[32752]: pgmap v19319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:44.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:22:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:22:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:22:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:44 smithi090 ceph-mon[29180]: pgmap v19320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:22:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:44 smithi090 ceph-mon[29180]: pgmap v19321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:44.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:22:44.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:22:44.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:22:44.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:22:44.802 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:44 smithi119 ceph-mon[32752]: pgmap v19320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:22:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:44 smithi119 ceph-mon[32752]: pgmap v19321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:44.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:22:44.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:22:44.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:22:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:22:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:44 smithi167 ceph-mon[32751]: pgmap v19320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:22:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:44 smithi167 ceph-mon[32751]: pgmap v19321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:44.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:22:46.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:45 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35247.530260 secs 2024-04-27T04:22:46.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:45 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35247.530260 secs 2024-04-27T04:22:46.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:45 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35247.530260 secs 2024-04-27T04:22:47.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:46 smithi119 ceph-mon[32752]: pgmap v19322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:47.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35248.410781 secs 2024-04-27T04:22:47.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:46 smithi167 ceph-mon[32751]: pgmap v19322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:47.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35248.410781 secs 2024-04-27T04:22:47.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:46 smithi090 ceph-mon[29180]: pgmap v19322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:47.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35248.410781 secs 2024-04-27T04:22:49.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:49 smithi119 ceph-mon[32752]: pgmap v19323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:22:49.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:49 smithi167 ceph-mon[32751]: pgmap v19323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:22:49.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:49 smithi090 ceph-mon[29180]: pgmap v19323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:22:51.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:51 smithi119 ceph-mon[32752]: pgmap v19324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:51.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:51 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35252.530343 secs 2024-04-27T04:22:51.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:51 smithi167 ceph-mon[32751]: pgmap v19324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:51.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:51 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35252.530343 secs 2024-04-27T04:22:51.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:51 smithi090 ceph-mon[29180]: pgmap v19324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:51.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:51 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35252.530343 secs 2024-04-27T04:22:52.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35253.410901 secs 2024-04-27T04:22:52.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35253.410901 secs 2024-04-27T04:22:52.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35253.410901 secs 2024-04-27T04:22:53.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:53 smithi119 ceph-mon[32752]: pgmap v19325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:53.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:53 smithi167 ceph-mon[32751]: pgmap v19325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:53.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:53 smithi090 ceph-mon[29180]: pgmap v19325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:22:55.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:55 smithi119 ceph-mon[32752]: pgmap v19326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:22:55.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:55 smithi167 ceph-mon[32751]: pgmap v19326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:22:55.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:55 smithi090 ceph-mon[29180]: pgmap v19326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:22:56.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:56 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35257.530424 secs 2024-04-27T04:22:56.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:56 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35257.530424 secs 2024-04-27T04:22:56.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:56 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35257.530424 secs 2024-04-27T04:22:57.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:57 smithi119 ceph-mon[32752]: pgmap v19327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35258.410973 secs 2024-04-27T04:22:57.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:57 smithi167 ceph-mon[32751]: pgmap v19327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:57.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35258.410973 secs 2024-04-27T04:22:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:57 smithi090 ceph-mon[29180]: pgmap v19327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:57.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35258.410973 secs 2024-04-27T04:22:59.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:22:59 smithi119 ceph-mon[32752]: pgmap v19328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:59.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:22:59 smithi167 ceph-mon[32751]: pgmap v19328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:22:59.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:22:59 smithi090 ceph-mon[29180]: pgmap v19328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:01.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:01 smithi119 ceph-mon[32752]: pgmap v19329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:01 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35262.530484 secs 2024-04-27T04:23:01.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:01 smithi167 ceph-mon[32751]: pgmap v19329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:01.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:01 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35262.530484 secs 2024-04-27T04:23:01.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:01 smithi090 ceph-mon[29180]: pgmap v19329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:01.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:01 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35262.530484 secs 2024-04-27T04:23:02.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35263.411123 secs 2024-04-27T04:23:02.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35263.411123 secs 2024-04-27T04:23:02.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35263.411123 secs 2024-04-27T04:23:03.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:03 smithi119 ceph-mon[32752]: pgmap v19330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:03.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:03 smithi167 ceph-mon[32751]: pgmap v19330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:03.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:03 smithi090 ceph-mon[29180]: pgmap v19330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:05.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:05 smithi119 ceph-mon[32752]: pgmap v19331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:05.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:05 smithi167 ceph-mon[32751]: pgmap v19331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:05.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:05 smithi090 ceph-mon[29180]: pgmap v19331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:06.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:06 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35267.530568 secs 2024-04-27T04:23:06.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:06 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35267.530568 secs 2024-04-27T04:23:06.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:06 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35267.530568 secs 2024-04-27T04:23:07.524 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:07 smithi090 ceph-mon[29180]: pgmap v19332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:07.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35268.411197 secs 2024-04-27T04:23:07.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:07 smithi119 ceph-mon[32752]: pgmap v19332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:07.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35268.411197 secs 2024-04-27T04:23:07.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:07 smithi167 ceph-mon[32751]: pgmap v19332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35268.411197 secs 2024-04-27T04:23:09.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:09 smithi119 ceph-mon[32752]: pgmap v19333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:09.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:09 smithi167 ceph-mon[32751]: pgmap v19333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:09.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:09 smithi090 ceph-mon[29180]: pgmap v19333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:11.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:11 smithi119 ceph-mon[32752]: pgmap v19334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:11.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:11 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35272.530653 secs 2024-04-27T04:23:11.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:11 smithi167 ceph-mon[32751]: pgmap v19334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:11.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:11 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35272.530653 secs 2024-04-27T04:23:11.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:11 smithi090 ceph-mon[29180]: pgmap v19334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:11.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:11 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35272.530653 secs 2024-04-27T04:23:12.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35273.411322 secs 2024-04-27T04:23:12.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35273.411322 secs 2024-04-27T04:23:12.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35273.411322 secs 2024-04-27T04:23:13.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:13 smithi119 ceph-mon[32752]: pgmap v19335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:13.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:13 smithi167 ceph-mon[32751]: pgmap v19335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:13.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:13 smithi090 ceph-mon[29180]: pgmap v19335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:15.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:15 smithi119 ceph-mon[32752]: pgmap v19336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:15.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:15 smithi167 ceph-mon[32751]: pgmap v19336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:15.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:15 smithi090 ceph-mon[29180]: pgmap v19336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:16.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:16 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35277.530736 secs 2024-04-27T04:23:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:16 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35277.530736 secs 2024-04-27T04:23:16.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:16 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35277.530736 secs 2024-04-27T04:23:17.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:17 smithi119 ceph-mon[32752]: pgmap v19337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:17.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35278.411432 secs 2024-04-27T04:23:17.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:17 smithi167 ceph-mon[32751]: pgmap v19337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:17.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35278.411432 secs 2024-04-27T04:23:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:17 smithi090 ceph-mon[29180]: pgmap v19337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:17.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35278.411432 secs 2024-04-27T04:23:19.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:19 smithi119 ceph-mon[32752]: pgmap v19338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:19.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:19 smithi167 ceph-mon[32751]: pgmap v19338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:19.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:19 smithi090 ceph-mon[29180]: pgmap v19338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:21.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:21 smithi119 ceph-mon[32752]: pgmap v19339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:21.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:21 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35282.530841 secs 2024-04-27T04:23:21.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:21 smithi167 ceph-mon[32751]: pgmap v19339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:21.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:21 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35282.530841 secs 2024-04-27T04:23:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:21 smithi090 ceph-mon[29180]: pgmap v19339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:21.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:21 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35282.530841 secs 2024-04-27T04:23:22.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35283.411568 secs 2024-04-27T04:23:22.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35283.411568 secs 2024-04-27T04:23:22.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35283.411568 secs 2024-04-27T04:23:23.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:23 smithi119 ceph-mon[32752]: pgmap v19340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:23.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:23 smithi167 ceph-mon[32751]: pgmap v19340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:23.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:23 smithi090 ceph-mon[29180]: pgmap v19340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:25.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:25 smithi119 ceph-mon[32752]: pgmap v19341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:25.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:25 smithi167 ceph-mon[32751]: pgmap v19341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:25.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:25 smithi090 ceph-mon[29180]: pgmap v19341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:26.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:26 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35287.530931 secs 2024-04-27T04:23:26.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:26 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35287.530931 secs 2024-04-27T04:23:26.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:26 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35287.530931 secs 2024-04-27T04:23:27.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:27 smithi119 ceph-mon[32752]: pgmap v19342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35288.411686 secs 2024-04-27T04:23:27.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:27 smithi167 ceph-mon[32751]: pgmap v19342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35288.411686 secs 2024-04-27T04:23:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:27 smithi090 ceph-mon[29180]: pgmap v19342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:27.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35288.411686 secs 2024-04-27T04:23:29.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:29 smithi119 ceph-mon[32752]: pgmap v19343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:29.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:29 smithi167 ceph-mon[32751]: pgmap v19343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:29.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:29 smithi090 ceph-mon[29180]: pgmap v19343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:31.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:31 smithi119 ceph-mon[32752]: pgmap v19344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:31 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35292.530992 secs 2024-04-27T04:23:31.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:31 smithi167 ceph-mon[32751]: pgmap v19344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:31.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:31 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35292.530992 secs 2024-04-27T04:23:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:31 smithi090 ceph-mon[29180]: pgmap v19344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:31.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:31 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35292.530992 secs 2024-04-27T04:23:32.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35293.411815 secs 2024-04-27T04:23:32.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35293.411815 secs 2024-04-27T04:23:32.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35293.411815 secs 2024-04-27T04:23:33.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:33 smithi119 ceph-mon[32752]: pgmap v19345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:33.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:33 smithi167 ceph-mon[32751]: pgmap v19345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:33 smithi090 ceph-mon[29180]: pgmap v19345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:35.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:35 smithi119 ceph-mon[32752]: pgmap v19346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:35.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:35 smithi167 ceph-mon[32751]: pgmap v19346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:35 smithi090 ceph-mon[29180]: pgmap v19346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:36.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:36 smithi119 ceph-mon[32752]: 19 slow requests, 0 included below; oldest blocked for > 35297.531082 secs 2024-04-27T04:23:36.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:36 smithi167 ceph-mon[32751]: 19 slow requests, 0 included below; oldest blocked for > 35297.531082 secs 2024-04-27T04:23:36.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:36 smithi090 ceph-mon[29180]: 19 slow requests, 0 included below; oldest blocked for > 35297.531082 secs 2024-04-27T04:23:37.525 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:37 smithi090 ceph-mon[29180]: pgmap v19347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:37.592 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35298.411900 secs 2024-04-27T04:23:37.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:37 smithi167 ceph-mon[32751]: pgmap v19347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:37.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35298.411900 secs 2024-04-27T04:23:37.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:37 smithi119 ceph-mon[32752]: pgmap v19347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:37.593 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35298.411900 secs 2024-04-27T04:23:39.551 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:39 smithi119 ceph-mon[32752]: pgmap v19348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:39.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:39 smithi167 ceph-mon[32751]: pgmap v19348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:39.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:39 smithi090 ceph-mon[29180]: pgmap v19348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:41 smithi119 ceph-mon[32752]: pgmap v19349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:41 smithi119 ceph-mon[32752]: 47 slow requests, 2 included below; oldest blocked for > 35302.531145 secs 2024-04-27T04:23:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:41 smithi119 ceph-mon[32752]: slow request 182.643790 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:23:41.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:41 smithi119 ceph-mon[32752]: slow request 11642.997177 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:23:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:41 smithi167 ceph-mon[32751]: pgmap v19349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:41 smithi167 ceph-mon[32751]: 47 slow requests, 2 included below; oldest blocked for > 35302.531145 secs 2024-04-27T04:23:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:41 smithi167 ceph-mon[32751]: slow request 182.643790 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:23:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:41 smithi167 ceph-mon[32751]: slow request 11642.997177 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:23:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:41 smithi090 ceph-mon[29180]: pgmap v19349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:41 smithi090 ceph-mon[29180]: 47 slow requests, 2 included below; oldest blocked for > 35302.531145 secs 2024-04-27T04:23:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:41 smithi090 ceph-mon[29180]: slow request 182.643790 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:23:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:41 smithi090 ceph-mon[29180]: slow request 11642.997177 seconds old, received at 2024-04-27T01:09:37.287372+0000: client_request(client.26873:1 getattr pAsLsXsFs #0x1 2024-04-27T01:09:37.283512+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:23:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35303.412012 secs 2024-04-27T04:23:42.565 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35303.412012 secs 2024-04-27T04:23:42.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35303.412012 secs 2024-04-27T04:23:43.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:43 smithi119 ceph-mon[32752]: pgmap v19350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:43.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:43 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:23:43.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:43 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:23:43.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:43 smithi167 ceph-mon[32751]: pgmap v19350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:43.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:43 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:23:43.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:43 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:23:43.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:43 smithi090 ceph-mon[29180]: pgmap v19350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:43.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:43 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:23:43.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:43 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:23:44.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:44 smithi119 ceph-mon[32752]: pgmap v19351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:44.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:23:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:44 smithi167 ceph-mon[32751]: pgmap v19351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:44.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:23:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:44 smithi090 ceph-mon[29180]: pgmap v19351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:23:44.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:23:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:23:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:23:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:45 smithi119 ceph-mon[32752]: pgmap v19352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:23:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:45 smithi119 ceph-mon[32752]: pgmap v19353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:23:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:23:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:23:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:45 smithi167 ceph-mon[32751]: pgmap v19352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:23:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:45 smithi167 ceph-mon[32751]: pgmap v19353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:23:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:23:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:23:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:45 smithi090 ceph-mon[29180]: pgmap v19352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:23:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:45 smithi090 ceph-mon[29180]: pgmap v19353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:23:46.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35307.531204 secs 2024-04-27T04:23:46.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35307.531204 secs 2024-04-27T04:23:46.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35307.531204 secs 2024-04-27T04:23:47.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35308.412136 secs 2024-04-27T04:23:47.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:47 smithi119 ceph-mon[32752]: pgmap v19354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35308.412136 secs 2024-04-27T04:23:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:47 smithi167 ceph-mon[32751]: pgmap v19354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35308.412136 secs 2024-04-27T04:23:47.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:47 smithi090 ceph-mon[29180]: pgmap v19354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:50.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:49 smithi119 ceph-mon[32752]: pgmap v19355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:50.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:49 smithi167 ceph-mon[32751]: pgmap v19355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:50.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:49 smithi090 ceph-mon[29180]: pgmap v19355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:51.203 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:50 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35312.531296 secs 2024-04-27T04:23:51.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:50 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35312.531296 secs 2024-04-27T04:23:51.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:50 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35312.531296 secs 2024-04-27T04:23:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:51 smithi119 ceph-mon[32752]: pgmap v19356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35313.412214 secs 2024-04-27T04:23:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:51 smithi167 ceph-mon[32751]: pgmap v19356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35313.412214 secs 2024-04-27T04:23:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:51 smithi090 ceph-mon[29180]: pgmap v19356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:23:52.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35313.412214 secs 2024-04-27T04:23:54.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:53 smithi119 ceph-mon[32752]: pgmap v19357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:23:54.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:53 smithi167 ceph-mon[32751]: pgmap v19357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:23:54.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:53 smithi090 ceph-mon[29180]: pgmap v19357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:23:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:55 smithi119 ceph-mon[32752]: pgmap v19358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:23:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:55 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35317.531374 secs 2024-04-27T04:23:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:55 smithi167 ceph-mon[32751]: pgmap v19358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:23:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:55 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35317.531374 secs 2024-04-27T04:23:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:55 smithi090 ceph-mon[29180]: pgmap v19358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:23:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:55 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35317.531374 secs 2024-04-27T04:23:57.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35318.412336 secs 2024-04-27T04:23:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35318.412336 secs 2024-04-27T04:23:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35318.412336 secs 2024-04-27T04:23:58.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:57 smithi119 ceph-mon[32752]: pgmap v19359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:58.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:57 smithi167 ceph-mon[32751]: pgmap v19359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:23:58.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:57 smithi090 ceph-mon[29180]: pgmap v19359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:00.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:23:59 smithi119 ceph-mon[32752]: pgmap v19360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:00.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:23:59 smithi167 ceph-mon[32751]: pgmap v19360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:00.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:23:59 smithi090 ceph-mon[29180]: pgmap v19360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35322.531455 secs 2024-04-27T04:24:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:01 smithi119 ceph-mon[32752]: pgmap v19361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35323.412458 secs 2024-04-27T04:24:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35322.531455 secs 2024-04-27T04:24:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:01 smithi167 ceph-mon[32751]: pgmap v19361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35323.412458 secs 2024-04-27T04:24:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35322.531455 secs 2024-04-27T04:24:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:01 smithi090 ceph-mon[29180]: pgmap v19361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35323.412458 secs 2024-04-27T04:24:04.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:03 smithi119 ceph-mon[32752]: pgmap v19362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:04.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:03 smithi167 ceph-mon[32751]: pgmap v19362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:04.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:03 smithi090 ceph-mon[29180]: pgmap v19362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:06.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:05 smithi090 ceph-mon[29180]: pgmap v19363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:06.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:05 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35327.531534 secs 2024-04-27T04:24:06.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:05 smithi119 ceph-mon[32752]: pgmap v19363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:06.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:05 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35327.531534 secs 2024-04-27T04:24:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:05 smithi167 ceph-mon[32751]: pgmap v19363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:05 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35327.531534 secs 2024-04-27T04:24:07.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35328.412581 secs 2024-04-27T04:24:07.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35328.412581 secs 2024-04-27T04:24:07.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35328.412581 secs 2024-04-27T04:24:08.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:07 smithi119 ceph-mon[32752]: pgmap v19364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:08.333 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:07 smithi167 ceph-mon[32751]: pgmap v19364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:08.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:07 smithi090 ceph-mon[29180]: pgmap v19364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:10.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:09 smithi119 ceph-mon[32752]: pgmap v19365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:10.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:09 smithi167 ceph-mon[32751]: pgmap v19365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:10.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:09 smithi090 ceph-mon[29180]: pgmap v19365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:11.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:10 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35332.531614 secs 2024-04-27T04:24:11.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:10 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35332.531614 secs 2024-04-27T04:24:11.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:10 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35332.531614 secs 2024-04-27T04:24:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:11 smithi119 ceph-mon[32752]: pgmap v19366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35333.412703 secs 2024-04-27T04:24:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:11 smithi167 ceph-mon[32751]: pgmap v19366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:12.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35333.412703 secs 2024-04-27T04:24:12.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:11 smithi090 ceph-mon[29180]: pgmap v19366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:12.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35333.412703 secs 2024-04-27T04:24:14.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:13 smithi119 ceph-mon[32752]: pgmap v19367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:14.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:13 smithi167 ceph-mon[32751]: pgmap v19367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:14.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:13 smithi090 ceph-mon[29180]: pgmap v19367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:15 smithi119 ceph-mon[32752]: pgmap v19368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:15 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35337.531692 secs 2024-04-27T04:24:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:15 smithi167 ceph-mon[32751]: pgmap v19368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:15 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35337.531692 secs 2024-04-27T04:24:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:15 smithi090 ceph-mon[29180]: pgmap v19368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:15 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35337.531692 secs 2024-04-27T04:24:17.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35338.412819 secs 2024-04-27T04:24:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35338.412819 secs 2024-04-27T04:24:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35338.412819 secs 2024-04-27T04:24:18.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:17 smithi119 ceph-mon[32752]: pgmap v19369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:17 smithi167 ceph-mon[32751]: pgmap v19369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:18.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:17 smithi090 ceph-mon[29180]: pgmap v19369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:20.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:19 smithi119 ceph-mon[32752]: pgmap v19370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:20.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:19 smithi167 ceph-mon[32751]: pgmap v19370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:20.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:19 smithi090 ceph-mon[29180]: pgmap v19370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:21.217 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:20 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35342.531775 secs 2024-04-27T04:24:21.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:20 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35342.531775 secs 2024-04-27T04:24:21.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:20 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35342.531775 secs 2024-04-27T04:24:22.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:21 smithi090 ceph-mon[29180]: pgmap v19371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:22.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35343.412938 secs 2024-04-27T04:24:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:21 smithi119 ceph-mon[32752]: pgmap v19371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35343.412938 secs 2024-04-27T04:24:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:21 smithi167 ceph-mon[32751]: pgmap v19371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:22.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35343.412938 secs 2024-04-27T04:24:24.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:23 smithi119 ceph-mon[32752]: pgmap v19372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:24.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:23 smithi167 ceph-mon[32751]: pgmap v19372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:24.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:23 smithi090 ceph-mon[29180]: pgmap v19372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:25 smithi119 ceph-mon[32752]: pgmap v19373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:26.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:25 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35347.531861 secs 2024-04-27T04:24:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:25 smithi167 ceph-mon[32751]: pgmap v19373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:26.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:25 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35347.531861 secs 2024-04-27T04:24:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:25 smithi090 ceph-mon[29180]: pgmap v19373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:26.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:25 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35347.531861 secs 2024-04-27T04:24:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35348.413053 secs 2024-04-27T04:24:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35348.413053 secs 2024-04-27T04:24:27.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35348.413053 secs 2024-04-27T04:24:28.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:27 smithi119 ceph-mon[32752]: pgmap v19374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:28.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:27 smithi167 ceph-mon[32751]: pgmap v19374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:28.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:27 smithi090 ceph-mon[29180]: pgmap v19374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:30.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:29 smithi119 ceph-mon[32752]: pgmap v19375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:30.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:29 smithi167 ceph-mon[32751]: pgmap v19375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:30.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:29 smithi090 ceph-mon[29180]: pgmap v19375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:31.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:30 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35352.531946 secs 2024-04-27T04:24:31.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:30 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35352.531946 secs 2024-04-27T04:24:31.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:30 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35352.531946 secs 2024-04-27T04:24:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:31 smithi119 ceph-mon[32752]: pgmap v19376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35353.413166 secs 2024-04-27T04:24:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:31 smithi167 ceph-mon[32751]: pgmap v19376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35353.413166 secs 2024-04-27T04:24:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:31 smithi090 ceph-mon[29180]: pgmap v19376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:32.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35353.413166 secs 2024-04-27T04:24:34.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:33 smithi119 ceph-mon[32752]: pgmap v19377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:34.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:33 smithi167 ceph-mon[32751]: pgmap v19377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:34.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:33 smithi090 ceph-mon[29180]: pgmap v19377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:36 smithi119 ceph-mon[32752]: pgmap v19378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:36 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 35357.532027 secs 2024-04-27T04:24:36.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:36 smithi119 ceph-mon[32752]: slow request 5761.772368 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:24:36.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:36 smithi167 ceph-mon[32751]: pgmap v19378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:36 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 35357.532027 secs 2024-04-27T04:24:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:36 smithi167 ceph-mon[32751]: slow request 5761.772368 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:24:36.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:36 smithi090 ceph-mon[29180]: pgmap v19378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:36 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 35357.532027 secs 2024-04-27T04:24:36.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:36 smithi090 ceph-mon[29180]: slow request 5761.772368 seconds old, received at 2024-04-27T02:48:33.513063+0000: client_request(client.24733:267901 getattr AsXsFs #0x100000001fb 2024-04-27T02:48:33.511387+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:24:37.275 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35358.413323 secs 2024-04-27T04:24:37.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35358.413323 secs 2024-04-27T04:24:37.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35358.413323 secs 2024-04-27T04:24:38.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:38 smithi119 ceph-mon[32752]: pgmap v19379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:38.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:38 smithi167 ceph-mon[32751]: pgmap v19379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:38.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:38 smithi090 ceph-mon[29180]: pgmap v19379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:40.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:40 smithi119 ceph-mon[32752]: pgmap v19380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:40 smithi167 ceph-mon[32751]: pgmap v19380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:40.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:40 smithi090 ceph-mon[29180]: pgmap v19380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:41.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35362.532113 secs 2024-04-27T04:24:41.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35362.532113 secs 2024-04-27T04:24:41.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35362.532113 secs 2024-04-27T04:24:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:42 smithi119 ceph-mon[32752]: pgmap v19381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:42.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35363.413462 secs 2024-04-27T04:24:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:42 smithi167 ceph-mon[32751]: pgmap v19381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:42.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35363.413462 secs 2024-04-27T04:24:42.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:42 smithi090 ceph-mon[29180]: pgmap v19381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:42.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35363.413462 secs 2024-04-27T04:24:44.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:44 smithi119 ceph-mon[32752]: pgmap v19382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:44.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:44 smithi167 ceph-mon[32751]: pgmap v19382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:44.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:44 smithi090 ceph-mon[29180]: pgmap v19382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:45.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:24:45.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:24:45.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:24:46.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:46 smithi119 ceph-mon[32752]: pgmap v19383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:46.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35367.532202 secs 2024-04-27T04:24:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:24:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:24:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:24:46.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:46 smithi167 ceph-mon[32751]: pgmap v19383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35367.532202 secs 2024-04-27T04:24:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:24:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:24:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:24:46.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:46 smithi090 ceph-mon[29180]: pgmap v19383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:24:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35367.532202 secs 2024-04-27T04:24:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:24:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:24:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:24:47.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:47 smithi119 ceph-mon[32752]: pgmap v19384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:24:47.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:47 smithi119 ceph-mon[32752]: pgmap v19385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35368.413590 secs 2024-04-27T04:24:47.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:47 smithi167 ceph-mon[32751]: pgmap v19384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:24:47.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:47 smithi167 ceph-mon[32751]: pgmap v19385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35368.413590 secs 2024-04-27T04:24:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:47 smithi090 ceph-mon[29180]: pgmap v19384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:24:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:47 smithi090 ceph-mon[29180]: pgmap v19385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35368.413590 secs 2024-04-27T04:24:49.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:49 smithi119 ceph-mon[32752]: pgmap v19386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:24:49.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:49 smithi167 ceph-mon[32751]: pgmap v19386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:24:49.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:49 smithi090 ceph-mon[29180]: pgmap v19386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:24:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:51 smithi119 ceph-mon[32752]: pgmap v19387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35372.532286 secs 2024-04-27T04:24:51.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:51 smithi167 ceph-mon[32751]: pgmap v19387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35372.532286 secs 2024-04-27T04:24:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:51 smithi090 ceph-mon[29180]: pgmap v19387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35372.532286 secs 2024-04-27T04:24:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35373.413722 secs 2024-04-27T04:24:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35373.413722 secs 2024-04-27T04:24:52.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35373.413722 secs 2024-04-27T04:24:53.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:53 smithi119 ceph-mon[32752]: pgmap v19388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:53.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:53 smithi167 ceph-mon[32751]: pgmap v19388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:53.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:53 smithi090 ceph-mon[29180]: pgmap v19388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:55.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:55 smithi119 ceph-mon[32752]: pgmap v19389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:55.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:55 smithi167 ceph-mon[32751]: pgmap v19389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:55.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:55 smithi090 ceph-mon[29180]: pgmap v19389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:24:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35377.532367 secs 2024-04-27T04:24:56.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35377.532367 secs 2024-04-27T04:24:56.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35377.532367 secs 2024-04-27T04:24:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:57 smithi167 ceph-mon[32751]: pgmap v19390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:24:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35378.413854 secs 2024-04-27T04:24:57.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:57 smithi090 ceph-mon[29180]: pgmap v19390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:24:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35378.413854 secs 2024-04-27T04:24:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:57 smithi119 ceph-mon[32752]: pgmap v19390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:24:57.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35378.413854 secs 2024-04-27T04:24:59.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:24:59 smithi167 ceph-mon[32751]: pgmap v19391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:59.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:24:59 smithi090 ceph-mon[29180]: pgmap v19391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:24:59.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:24:59 smithi119 ceph-mon[32752]: pgmap v19391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:01 smithi167 ceph-mon[32751]: pgmap v19392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35382.532454 secs 2024-04-27T04:25:01.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:01 smithi090 ceph-mon[29180]: pgmap v19392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35382.532454 secs 2024-04-27T04:25:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:01 smithi119 ceph-mon[32752]: pgmap v19392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:01.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35382.532454 secs 2024-04-27T04:25:02.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35383.413969 secs 2024-04-27T04:25:02.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35383.413969 secs 2024-04-27T04:25:02.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35383.413969 secs 2024-04-27T04:25:03.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:03 smithi090 ceph-mon[29180]: pgmap v19393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:03.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:03 smithi119 ceph-mon[32752]: pgmap v19393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:03.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:03 smithi167 ceph-mon[32751]: pgmap v19393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:05.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:05 smithi090 ceph-mon[29180]: pgmap v19394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:05.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:05 smithi119 ceph-mon[32752]: pgmap v19394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:05.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:05 smithi167 ceph-mon[32751]: pgmap v19394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:06.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35387.532534 secs 2024-04-27T04:25:06.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35387.532534 secs 2024-04-27T04:25:06.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35387.532534 secs 2024-04-27T04:25:07.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:07 smithi090 ceph-mon[29180]: pgmap v19395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35388.414093 secs 2024-04-27T04:25:07.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:07 smithi119 ceph-mon[32752]: pgmap v19395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:07.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35388.414093 secs 2024-04-27T04:25:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:07 smithi167 ceph-mon[32751]: pgmap v19395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:07.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35388.414093 secs 2024-04-27T04:25:09.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:09 smithi090 ceph-mon[29180]: pgmap v19396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:09.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:09 smithi119 ceph-mon[32752]: pgmap v19396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:09.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:09 smithi167 ceph-mon[32751]: pgmap v19396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:11 smithi090 ceph-mon[29180]: pgmap v19397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35392.532621 secs 2024-04-27T04:25:11.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:11 smithi119 ceph-mon[32752]: pgmap v19397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35392.532621 secs 2024-04-27T04:25:11.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:11 smithi167 ceph-mon[32751]: pgmap v19397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35392.532621 secs 2024-04-27T04:25:12.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35393.414221 secs 2024-04-27T04:25:12.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35393.414221 secs 2024-04-27T04:25:12.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35393.414221 secs 2024-04-27T04:25:13.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:13 smithi090 ceph-mon[29180]: pgmap v19398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:13.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:13 smithi119 ceph-mon[32752]: pgmap v19398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:13.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:13 smithi167 ceph-mon[32751]: pgmap v19398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:15.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:15 smithi090 ceph-mon[29180]: pgmap v19399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:15.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:15 smithi119 ceph-mon[32752]: pgmap v19399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:15.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:15 smithi167 ceph-mon[32751]: pgmap v19399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:16.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35397.532700 secs 2024-04-27T04:25:16.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35397.532700 secs 2024-04-27T04:25:16.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35397.532700 secs 2024-04-27T04:25:17.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:17 smithi090 ceph-mon[29180]: pgmap v19400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35398.414364 secs 2024-04-27T04:25:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:17 smithi119 ceph-mon[32752]: pgmap v19400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35398.414364 secs 2024-04-27T04:25:17.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:17 smithi167 ceph-mon[32751]: pgmap v19400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:17.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35398.414364 secs 2024-04-27T04:25:19.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:19 smithi090 ceph-mon[29180]: pgmap v19401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:19.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:19 smithi119 ceph-mon[32752]: pgmap v19401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:19.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:19 smithi167 ceph-mon[32751]: pgmap v19401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:21.368 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:21 smithi167 ceph-mon[32751]: pgmap v19402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:21.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35402.532792 secs 2024-04-27T04:25:21.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:21 smithi090 ceph-mon[29180]: pgmap v19402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:21.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35402.532792 secs 2024-04-27T04:25:21.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:21 smithi119 ceph-mon[32752]: pgmap v19402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35402.532792 secs 2024-04-27T04:25:22.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35403.414417 secs 2024-04-27T04:25:22.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35403.414417 secs 2024-04-27T04:25:22.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35403.414417 secs 2024-04-27T04:25:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:23 smithi090 ceph-mon[29180]: pgmap v19403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:23.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:23 smithi119 ceph-mon[32752]: pgmap v19403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:23.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:23 smithi167 ceph-mon[32751]: pgmap v19403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:25:25.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:25 smithi119 ceph-mon[32752]: pgmap v19404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:25.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:25 smithi167 ceph-mon[32751]: pgmap v19404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:25.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:25 smithi090 ceph-mon[29180]: pgmap v19404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:25:26.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35407.532868 secs 2024-04-27T04:25:26.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35407.532868 secs 2024-04-27T04:25:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35407.532868 secs 2024-04-27T04:25:27.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:27 smithi119 ceph-mon[32752]: pgmap v19405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35408.414509 secs 2024-04-27T04:25:27.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:27 smithi167 ceph-mon[32751]: pgmap v19405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35408.414509 secs 2024-04-27T04:25:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:27 smithi090 ceph-mon[29180]: pgmap v19405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35408.414509 secs 2024-04-27T04:25:29.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:29 smithi119 ceph-mon[32752]: pgmap v19406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:29.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:29 smithi167 ceph-mon[32751]: pgmap v19406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:29.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:29 smithi090 ceph-mon[29180]: pgmap v19406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:31.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:31 smithi119 ceph-mon[32752]: pgmap v19407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35412.532947 secs 2024-04-27T04:25:31.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:31 smithi167 ceph-mon[32751]: pgmap v19407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35412.532947 secs 2024-04-27T04:25:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:31 smithi090 ceph-mon[29180]: pgmap v19407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35412.532947 secs 2024-04-27T04:25:32.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35413.414831 secs 2024-04-27T04:25:32.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35413.414831 secs 2024-04-27T04:25:32.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35413.414831 secs 2024-04-27T04:25:33.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:33 smithi119 ceph-mon[32752]: pgmap v19408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:33.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:33 smithi167 ceph-mon[32751]: pgmap v19408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:33.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:33 smithi090 ceph-mon[29180]: pgmap v19408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:35.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:35 smithi119 ceph-mon[32752]: pgmap v19409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:35.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:35 smithi167 ceph-mon[32751]: pgmap v19409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:35.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:35 smithi090 ceph-mon[29180]: pgmap v19409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:36.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35417.533004 secs 2024-04-27T04:25:36.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35417.533004 secs 2024-04-27T04:25:36.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35417.533004 secs 2024-04-27T04:25:37.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:37 smithi090 ceph-mon[29180]: pgmap v19410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:37.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35418.414914 secs 2024-04-27T04:25:37.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:37 smithi119 ceph-mon[32752]: pgmap v19410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35418.414914 secs 2024-04-27T04:25:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:37 smithi167 ceph-mon[32751]: pgmap v19410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:37.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35418.414914 secs 2024-04-27T04:25:39.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:39 smithi119 ceph-mon[32752]: pgmap v19411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:39.556 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:39 smithi090 ceph-mon[29180]: pgmap v19411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:39.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:39 smithi167 ceph-mon[32751]: pgmap v19411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:25:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:25:40.601+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:25:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:25:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:25:40.601+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:25:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:25:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:25:40.601+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:25:40.875 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:25:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:25:40.601+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:25:40.876 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:25:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:25:40.601+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:25:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:41 smithi119 ceph-mon[32752]: pgmap v19412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35422.533091 secs 2024-04-27T04:25:41.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:41 smithi167 ceph-mon[32751]: pgmap v19412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35422.533091 secs 2024-04-27T04:25:41.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:41 smithi090 ceph-mon[29180]: pgmap v19412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35422.533091 secs 2024-04-27T04:25:42.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35423.415008 secs 2024-04-27T04:25:42.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35423.415008 secs 2024-04-27T04:25:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35423.415008 secs 2024-04-27T04:25:43.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:43 smithi119 ceph-mon[32752]: pgmap v19413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:43.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:43 smithi167 ceph-mon[32751]: pgmap v19413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:43.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:43 smithi090 ceph-mon[29180]: pgmap v19413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:25:45.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:45 smithi119 ceph-mon[32752]: pgmap v19414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:45.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:45 smithi167 ceph-mon[32751]: pgmap v19414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:45.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:45 smithi090 ceph-mon[29180]: pgmap v19414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:46.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35427.533165 secs 2024-04-27T04:25:46.520 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:25:46.552 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35427.533165 secs 2024-04-27T04:25:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:25:46.624 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35427.533165 secs 2024-04-27T04:25:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:25:47.374 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:47 smithi090 ceph-mon[29180]: pgmap v19415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35428.415094 secs 2024-04-27T04:25:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:47 smithi119 ceph-mon[32752]: pgmap v19415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35428.415094 secs 2024-04-27T04:25:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.566 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:47 smithi167 ceph-mon[32751]: pgmap v19415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35428.415094 secs 2024-04-27T04:25:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:48 smithi119 ceph-mon[32752]: pgmap v19416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:48 smithi167 ceph-mon[32751]: pgmap v19416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:48 smithi090 ceph-mon[29180]: pgmap v19416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:49.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:25:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:25:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:25:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:25:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:25:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:25:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:25:51.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:25:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:25:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: pgmap v19417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:25:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:25:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: pgmap v19418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 204 B/s wr, 15 op/s 2024-04-27T04:25:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: pgmap v19419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 B/s wr, 19 op/s 2024-04-27T04:25:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35432.533251 secs 2024-04-27T04:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: pgmap v19417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: pgmap v19418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 204 B/s wr, 15 op/s 2024-04-27T04:25:51.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: pgmap v19419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 B/s wr, 19 op/s 2024-04-27T04:25:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35432.533251 secs 2024-04-27T04:25:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:25:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:25:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: pgmap v19417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:25:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:25:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:25:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: pgmap v19418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 204 B/s wr, 15 op/s 2024-04-27T04:25:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: pgmap v19419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 255 B/s wr, 19 op/s 2024-04-27T04:25:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35432.533251 secs 2024-04-27T04:25:52.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35433.415168 secs 2024-04-27T04:25:52.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35433.415168 secs 2024-04-27T04:25:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35433.415168 secs 2024-04-27T04:25:53.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:53 smithi119 ceph-mon[32752]: pgmap v19420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:25:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:53 smithi167 ceph-mon[32751]: pgmap v19420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:25:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:53 smithi090 ceph-mon[29180]: pgmap v19420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:25:55.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:55 smithi119 ceph-mon[32752]: pgmap v19421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:55 smithi167 ceph-mon[32751]: pgmap v19421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:55 smithi090 ceph-mon[29180]: pgmap v19421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:56.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35437.533308 secs 2024-04-27T04:25:56.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35437.533308 secs 2024-04-27T04:25:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35437.533308 secs 2024-04-27T04:25:57.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:57 smithi119 ceph-mon[32752]: pgmap v19422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35438.415264 secs 2024-04-27T04:25:57.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:57 smithi167 ceph-mon[32751]: pgmap v19422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35438.415264 secs 2024-04-27T04:25:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:57 smithi090 ceph-mon[29180]: pgmap v19422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:25:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35438.415264 secs 2024-04-27T04:25:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:25:59 smithi119 ceph-mon[32752]: pgmap v19423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:25:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:25:59 smithi167 ceph-mon[32751]: pgmap v19423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:25:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:25:59 smithi090 ceph-mon[29180]: pgmap v19423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 639 B/s rd, 1 op/s 2024-04-27T04:26:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:01 smithi119 ceph-mon[32752]: pgmap v19424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:26:01.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35442.533393 secs 2024-04-27T04:26:01.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:01 smithi167 ceph-mon[32751]: pgmap v19424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:26:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35442.533393 secs 2024-04-27T04:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:01 smithi090 ceph-mon[29180]: pgmap v19424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:26:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35442.533393 secs 2024-04-27T04:26:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35443.415361 secs 2024-04-27T04:26:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35443.415361 secs 2024-04-27T04:26:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35443.415361 secs 2024-04-27T04:26:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:03 smithi119 ceph-mon[32752]: pgmap v19425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:03 smithi167 ceph-mon[32751]: pgmap v19425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:03 smithi090 ceph-mon[29180]: pgmap v19425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:05.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:05 smithi119 ceph-mon[32752]: pgmap v19426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:05.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:05 smithi167 ceph-mon[32751]: pgmap v19426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:05.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:05 smithi090 ceph-mon[29180]: pgmap v19426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:06.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35447.533475 secs 2024-04-27T04:26:06.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35447.533475 secs 2024-04-27T04:26:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35447.533475 secs 2024-04-27T04:26:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:07 smithi119 ceph-mon[32752]: pgmap v19427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35448.415450 secs 2024-04-27T04:26:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:07 smithi167 ceph-mon[32751]: pgmap v19427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:07.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35448.415450 secs 2024-04-27T04:26:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:07 smithi090 ceph-mon[29180]: pgmap v19427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:07.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35448.415450 secs 2024-04-27T04:26:09.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:09 smithi119 ceph-mon[32752]: pgmap v19428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:09.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:09 smithi167 ceph-mon[32751]: pgmap v19428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:09 smithi090 ceph-mon[29180]: pgmap v19428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:11 smithi167 ceph-mon[32751]: pgmap v19429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:11.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35452.533554 secs 2024-04-27T04:26:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:11 smithi090 ceph-mon[29180]: pgmap v19429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35452.533554 secs 2024-04-27T04:26:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:11 smithi119 ceph-mon[32752]: pgmap v19429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35452.533554 secs 2024-04-27T04:26:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:12 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 35453.415567 secs 2024-04-27T04:26:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:12 smithi167 ceph-mon[32751]: slow request 5762.158998 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:26:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:12 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 35453.415567 secs 2024-04-27T04:26:12.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:12 smithi090 ceph-mon[29180]: slow request 5762.158998 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:26:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:12 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 35453.415567 secs 2024-04-27T04:26:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:12 smithi119 ceph-mon[32752]: slow request 5762.158998 seconds old, received at 2024-04-27T02:50:09.007214+0000: client_request(client.17670:1 getattr pAsLsXsFs #0x1 2024-04-27T02:50:09.003502+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:26:13.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:13 smithi090 ceph-mon[29180]: pgmap v19430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:13 smithi119 ceph-mon[32752]: pgmap v19430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:13 smithi167 ceph-mon[32751]: pgmap v19430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:26:15.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:15 smithi090 ceph-mon[29180]: pgmap v19431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:15 smithi119 ceph-mon[32752]: pgmap v19431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:15 smithi167 ceph-mon[32751]: pgmap v19431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:26:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35457.533641 secs 2024-04-27T04:26:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35457.533641 secs 2024-04-27T04:26:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35457.533641 secs 2024-04-27T04:26:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:17 smithi090 ceph-mon[29180]: pgmap v19432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:17.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35458.415690 secs 2024-04-27T04:26:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:17 smithi119 ceph-mon[32752]: pgmap v19432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35458.415690 secs 2024-04-27T04:26:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:17 smithi167 ceph-mon[32751]: pgmap v19432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35458.415690 secs 2024-04-27T04:26:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:19 smithi090 ceph-mon[29180]: pgmap v19433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:19.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:19 smithi119 ceph-mon[32752]: pgmap v19433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:19 smithi167 ceph-mon[32751]: pgmap v19433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:21.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:21 smithi119 ceph-mon[32752]: pgmap v19434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:21.355 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35462.533730 secs 2024-04-27T04:26:21.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:21 smithi167 ceph-mon[32751]: pgmap v19434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:21.369 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35462.533730 secs 2024-04-27T04:26:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:21 smithi090 ceph-mon[29180]: pgmap v19434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35462.533730 secs 2024-04-27T04:26:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35463.415812 secs 2024-04-27T04:26:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35463.415812 secs 2024-04-27T04:26:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35463.415812 secs 2024-04-27T04:26:23.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:23 smithi090 ceph-mon[29180]: pgmap v19435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:23 smithi119 ceph-mon[32752]: pgmap v19435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:23 smithi167 ceph-mon[32751]: pgmap v19435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:25.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:25 smithi090 ceph-mon[29180]: pgmap v19436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:25 smithi119 ceph-mon[32752]: pgmap v19436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:25 smithi167 ceph-mon[32751]: pgmap v19436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:26.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35467.533794 secs 2024-04-27T04:26:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35467.533794 secs 2024-04-27T04:26:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35467.533794 secs 2024-04-27T04:26:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:27 smithi090 ceph-mon[29180]: pgmap v19437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:27.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35468.415940 secs 2024-04-27T04:26:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:27 smithi119 ceph-mon[32752]: pgmap v19437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35468.415940 secs 2024-04-27T04:26:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:27 smithi167 ceph-mon[32751]: pgmap v19437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35468.415940 secs 2024-04-27T04:26:29.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:29 smithi090 ceph-mon[29180]: pgmap v19438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:29 smithi119 ceph-mon[32752]: pgmap v19438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:29 smithi167 ceph-mon[32751]: pgmap v19438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:31 smithi090 ceph-mon[29180]: pgmap v19439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:31.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35472.533826 secs 2024-04-27T04:26:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:31 smithi119 ceph-mon[32752]: pgmap v19439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35472.533826 secs 2024-04-27T04:26:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:31 smithi167 ceph-mon[32751]: pgmap v19439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35472.533826 secs 2024-04-27T04:26:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35473.416055 secs 2024-04-27T04:26:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35473.416055 secs 2024-04-27T04:26:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35473.416055 secs 2024-04-27T04:26:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:33 smithi119 ceph-mon[32752]: pgmap v19440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:33 smithi167 ceph-mon[32751]: pgmap v19440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:33.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:33 smithi090 ceph-mon[29180]: pgmap v19440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:35 smithi119 ceph-mon[32752]: pgmap v19441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:35 smithi167 ceph-mon[32751]: pgmap v19441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:35.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:35 smithi090 ceph-mon[29180]: pgmap v19441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35477.533885 secs 2024-04-27T04:26:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35477.533885 secs 2024-04-27T04:26:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35477.533885 secs 2024-04-27T04:26:37.526 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:37 smithi090 ceph-mon[29180]: pgmap v19442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:37.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35478.416184 secs 2024-04-27T04:26:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:37 smithi119 ceph-mon[32752]: pgmap v19442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35478.416184 secs 2024-04-27T04:26:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:37 smithi167 ceph-mon[32751]: pgmap v19442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35478.416184 secs 2024-04-27T04:26:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:39 smithi119 ceph-mon[32752]: pgmap v19443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:39 smithi167 ceph-mon[32751]: pgmap v19443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:39.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:39 smithi090 ceph-mon[29180]: pgmap v19443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:41.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:41 smithi090 ceph-mon[29180]: pgmap v19444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:41.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:41 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 35482.533973 secs 2024-04-27T04:26:41.509 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:41 smithi090 ceph-mon[29180]: slow request 362.646618 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:26:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:41 smithi119 ceph-mon[32752]: pgmap v19444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:41 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 35482.533973 secs 2024-04-27T04:26:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:41 smithi119 ceph-mon[32752]: slow request 362.646618 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:26:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:41 smithi167 ceph-mon[32751]: pgmap v19444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:41 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 35482.533973 secs 2024-04-27T04:26:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:41 smithi167 ceph-mon[32751]: slow request 362.646618 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:26:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35483.416322 secs 2024-04-27T04:26:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35483.416322 secs 2024-04-27T04:26:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35483.416322 secs 2024-04-27T04:26:43.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:43 smithi119 ceph-mon[32752]: pgmap v19445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:43 smithi167 ceph-mon[32751]: pgmap v19445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:43 smithi090 ceph-mon[29180]: pgmap v19445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:45 smithi119 ceph-mon[32752]: pgmap v19446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:45 smithi167 ceph-mon[32751]: pgmap v19446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:45.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:45 smithi090 ceph-mon[29180]: pgmap v19446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:46.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35487.534057 secs 2024-04-27T04:26:46.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35487.534057 secs 2024-04-27T04:26:46.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35487.534057 secs 2024-04-27T04:26:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:47 smithi119 ceph-mon[32752]: pgmap v19447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:47.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35488.416401 secs 2024-04-27T04:26:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:47 smithi167 ceph-mon[32751]: pgmap v19447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:47.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35488.416401 secs 2024-04-27T04:26:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:47 smithi090 ceph-mon[29180]: pgmap v19447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:47.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35488.416401 secs 2024-04-27T04:26:49.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:49 smithi119 ceph-mon[32752]: pgmap v19448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:49.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:49 smithi167 ceph-mon[32751]: pgmap v19448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:49.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:49 smithi090 ceph-mon[29180]: pgmap v19448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:26:50.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:26:50.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:26:50.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:26:51.454 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:51 smithi119 ceph-mon[32752]: pgmap v19449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35492.534146 secs 2024-04-27T04:26:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:26:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:26:51.455 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:51 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:51 smithi167 ceph-mon[32751]: pgmap v19449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:51.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35492.534146 secs 2024-04-27T04:26:51.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:26:51.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:26:51.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:51 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:51 smithi090 ceph-mon[29180]: pgmap v19449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:26:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35492.534146 secs 2024-04-27T04:26:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:26:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:26:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:51 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:26:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:52 smithi119 ceph-mon[32752]: pgmap v19450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:26:52.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:52 smithi119 ceph-mon[32752]: pgmap v19451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35493.416488 secs 2024-04-27T04:26:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:52 smithi167 ceph-mon[32751]: pgmap v19450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:26:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:52 smithi167 ceph-mon[32751]: pgmap v19451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:52.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35493.416488 secs 2024-04-27T04:26:52.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:52 smithi090 ceph-mon[29180]: pgmap v19450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T04:26:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:52 smithi090 ceph-mon[29180]: pgmap v19451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35493.416488 secs 2024-04-27T04:26:54.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:54 smithi119 ceph-mon[32752]: pgmap v19452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T04:26:54.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:54 smithi167 ceph-mon[32751]: pgmap v19452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T04:26:54.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:54 smithi090 ceph-mon[29180]: pgmap v19452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T04:26:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:56 smithi119 ceph-mon[32752]: pgmap v19453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:56.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35497.534228 secs 2024-04-27T04:26:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:56 smithi167 ceph-mon[32751]: pgmap v19453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:56.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35497.534228 secs 2024-04-27T04:26:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:56 smithi090 ceph-mon[29180]: pgmap v19453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:56.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35497.534228 secs 2024-04-27T04:26:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35498.416617 secs 2024-04-27T04:26:57.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35498.416617 secs 2024-04-27T04:26:57.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35498.416617 secs 2024-04-27T04:26:58.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:26:58 smithi119 ceph-mon[32752]: pgmap v19454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:58.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:26:58 smithi167 ceph-mon[32751]: pgmap v19454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:26:58.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:26:58 smithi090 ceph-mon[29180]: pgmap v19454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:00.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:00 smithi119 ceph-mon[32752]: pgmap v19455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:00.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:00 smithi167 ceph-mon[32751]: pgmap v19455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:00.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:00 smithi090 ceph-mon[29180]: pgmap v19455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35502.534315 secs 2024-04-27T04:27:01.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35502.534315 secs 2024-04-27T04:27:01.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35502.534315 secs 2024-04-27T04:27:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:02 smithi119 ceph-mon[32752]: pgmap v19456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:27:02.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35503.416733 secs 2024-04-27T04:27:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:02 smithi167 ceph-mon[32751]: pgmap v19456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:27:02.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35503.416733 secs 2024-04-27T04:27:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:02 smithi090 ceph-mon[29180]: pgmap v19456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:27:02.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35503.416733 secs 2024-04-27T04:27:04.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:04 smithi119 ceph-mon[32752]: pgmap v19457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:04.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:04 smithi167 ceph-mon[32751]: pgmap v19457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:04.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:04 smithi090 ceph-mon[29180]: pgmap v19457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:06 smithi119 ceph-mon[32752]: pgmap v19458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35507.534399 secs 2024-04-27T04:27:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:06 smithi167 ceph-mon[32751]: pgmap v19458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35507.534399 secs 2024-04-27T04:27:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:06 smithi090 ceph-mon[29180]: pgmap v19458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:06.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35507.534399 secs 2024-04-27T04:27:07.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35508.416830 secs 2024-04-27T04:27:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35508.416830 secs 2024-04-27T04:27:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35508.416830 secs 2024-04-27T04:27:08.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:08 smithi119 ceph-mon[32752]: pgmap v19459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:08.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:08 smithi167 ceph-mon[32751]: pgmap v19459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:08.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:08 smithi090 ceph-mon[29180]: pgmap v19459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:10.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:10 smithi119 ceph-mon[32752]: pgmap v19460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:10.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:10 smithi167 ceph-mon[32751]: pgmap v19460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:10.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:10 smithi090 ceph-mon[29180]: pgmap v19460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35512.534460 secs 2024-04-27T04:27:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35512.534460 secs 2024-04-27T04:27:11.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35512.534460 secs 2024-04-27T04:27:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:12 smithi119 ceph-mon[32752]: pgmap v19461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35513.416979 secs 2024-04-27T04:27:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:12 smithi167 ceph-mon[32751]: pgmap v19461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:12.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35513.416979 secs 2024-04-27T04:27:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:12 smithi090 ceph-mon[29180]: pgmap v19461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:12.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35513.416979 secs 2024-04-27T04:27:14.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:14 smithi119 ceph-mon[32752]: pgmap v19462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:14.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:14 smithi167 ceph-mon[32751]: pgmap v19462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:14.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:14 smithi090 ceph-mon[29180]: pgmap v19462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:16 smithi119 ceph-mon[32752]: pgmap v19463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35517.534544 secs 2024-04-27T04:27:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:16 smithi167 ceph-mon[32751]: pgmap v19463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35517.534544 secs 2024-04-27T04:27:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:16 smithi090 ceph-mon[29180]: pgmap v19463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:16.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35517.534544 secs 2024-04-27T04:27:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35518.417049 secs 2024-04-27T04:27:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35518.417049 secs 2024-04-27T04:27:17.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35518.417049 secs 2024-04-27T04:27:18.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:18 smithi119 ceph-mon[32752]: pgmap v19464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:18.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:18 smithi167 ceph-mon[32751]: pgmap v19464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:18.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:18 smithi090 ceph-mon[29180]: pgmap v19464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:20.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:20 smithi119 ceph-mon[32752]: pgmap v19465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:20.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:20 smithi167 ceph-mon[32751]: pgmap v19465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:20.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:20 smithi090 ceph-mon[29180]: pgmap v19465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35522.534628 secs 2024-04-27T04:27:21.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35522.534628 secs 2024-04-27T04:27:21.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35522.534628 secs 2024-04-27T04:27:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:22 smithi119 ceph-mon[32752]: pgmap v19466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35523.417170 secs 2024-04-27T04:27:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:22 smithi167 ceph-mon[32751]: pgmap v19466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35523.417170 secs 2024-04-27T04:27:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:22 smithi090 ceph-mon[29180]: pgmap v19466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:22.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35523.417170 secs 2024-04-27T04:27:24.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:24 smithi119 ceph-mon[32752]: pgmap v19467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:24.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:24 smithi167 ceph-mon[32751]: pgmap v19467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:24.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:24 smithi090 ceph-mon[29180]: pgmap v19467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:26 smithi119 ceph-mon[32752]: pgmap v19468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35527.534712 secs 2024-04-27T04:27:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:26 smithi167 ceph-mon[32751]: pgmap v19468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35527.534712 secs 2024-04-27T04:27:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:26 smithi090 ceph-mon[29180]: pgmap v19468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:26.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35527.534712 secs 2024-04-27T04:27:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35528.417292 secs 2024-04-27T04:27:27.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35528.417292 secs 2024-04-27T04:27:27.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35528.417292 secs 2024-04-27T04:27:28.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:28 smithi119 ceph-mon[32752]: pgmap v19469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:28.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:28 smithi167 ceph-mon[32751]: pgmap v19469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:28.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:28 smithi090 ceph-mon[29180]: pgmap v19469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:30.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:30 smithi119 ceph-mon[32752]: pgmap v19470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:30.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:30 smithi167 ceph-mon[32751]: pgmap v19470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:30.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:30 smithi090 ceph-mon[29180]: pgmap v19470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35532.534816 secs 2024-04-27T04:27:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35532.534816 secs 2024-04-27T04:27:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35532.534816 secs 2024-04-27T04:27:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:32 smithi119 ceph-mon[32752]: pgmap v19471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35533.417412 secs 2024-04-27T04:27:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:32 smithi167 ceph-mon[32751]: pgmap v19471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35533.417412 secs 2024-04-27T04:27:32.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:32 smithi090 ceph-mon[29180]: pgmap v19471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35533.417412 secs 2024-04-27T04:27:34.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:34 smithi119 ceph-mon[32752]: pgmap v19472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:34.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:34 smithi167 ceph-mon[32751]: pgmap v19472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:34.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:34 smithi090 ceph-mon[29180]: pgmap v19472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:36 smithi119 ceph-mon[32752]: pgmap v19473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35537.534879 secs 2024-04-27T04:27:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:36 smithi167 ceph-mon[32751]: pgmap v19473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35537.534879 secs 2024-04-27T04:27:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:36 smithi090 ceph-mon[29180]: pgmap v19473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:36.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35537.534879 secs 2024-04-27T04:27:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35538.417483 secs 2024-04-27T04:27:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35538.417483 secs 2024-04-27T04:27:37.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35538.417483 secs 2024-04-27T04:27:38.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:38 smithi119 ceph-mon[32752]: pgmap v19474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:38.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:38 smithi167 ceph-mon[32751]: pgmap v19474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:38.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:38 smithi090 ceph-mon[29180]: pgmap v19474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:40.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:40 smithi119 ceph-mon[32752]: pgmap v19475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:40.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:40 smithi167 ceph-mon[32751]: pgmap v19475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:40.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:40 smithi090 ceph-mon[29180]: pgmap v19475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:41.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35542.534940 secs 2024-04-27T04:27:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35542.534940 secs 2024-04-27T04:27:41.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35542.534940 secs 2024-04-27T04:27:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:42 smithi167 ceph-mon[32751]: pgmap v19476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35543.417581 secs 2024-04-27T04:27:42.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:42 smithi090 ceph-mon[29180]: pgmap v19476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35543.417581 secs 2024-04-27T04:27:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:42 smithi119 ceph-mon[32752]: pgmap v19476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:42.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35543.417581 secs 2024-04-27T04:27:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:43 smithi167 ceph-mon[32751]: pgmap v19477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:43.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:43 smithi090 ceph-mon[29180]: pgmap v19477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:43.803 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:43 smithi119 ceph-mon[32752]: pgmap v19477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:46 smithi119 ceph-mon[32752]: pgmap v19478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35547.535022 secs 2024-04-27T04:27:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:46 smithi167 ceph-mon[32751]: pgmap v19478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35547.535022 secs 2024-04-27T04:27:46.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:46 smithi090 ceph-mon[29180]: pgmap v19478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35547.535022 secs 2024-04-27T04:27:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35548.417735 secs 2024-04-27T04:27:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35548.417735 secs 2024-04-27T04:27:47.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35548.417735 secs 2024-04-27T04:27:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:48 smithi119 ceph-mon[32752]: pgmap v19479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:48 smithi167 ceph-mon[32751]: pgmap v19479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:48.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:48 smithi090 ceph-mon[29180]: pgmap v19479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:50 smithi119 ceph-mon[32752]: pgmap v19480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:50 smithi167 ceph-mon[32751]: pgmap v19480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:50.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:50 smithi090 ceph-mon[29180]: pgmap v19480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:27:51.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35552.535097 secs 2024-04-27T04:27:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35552.535097 secs 2024-04-27T04:27:51.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35552.535097 secs 2024-04-27T04:27:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:52 smithi119 ceph-mon[32752]: pgmap v19481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:27:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35553.417829 secs 2024-04-27T04:27:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:27:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:27:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:27:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:52 smithi167 ceph-mon[32751]: pgmap v19481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:52.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:27:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35553.417829 secs 2024-04-27T04:27:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:27:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:27:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:27:52.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:52 smithi090 ceph-mon[29180]: pgmap v19481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:27:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:27:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35553.417829 secs 2024-04-27T04:27:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:27:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:27:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:27:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:53 smithi119 ceph-mon[32752]: pgmap v19482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:27:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:53 smithi119 ceph-mon[32752]: pgmap v19483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:53 smithi167 ceph-mon[32751]: pgmap v19482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:27:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:53 smithi167 ceph-mon[32751]: pgmap v19483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:53 smithi090 ceph-mon[29180]: pgmap v19482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:27:53.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:53 smithi090 ceph-mon[29180]: pgmap v19483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:55.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:55 smithi119 ceph-mon[32752]: pgmap v19484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:55.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:55 smithi167 ceph-mon[32751]: pgmap v19484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:55.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:55 smithi090 ceph-mon[29180]: pgmap v19484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35557.535157 secs 2024-04-27T04:27:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35557.535157 secs 2024-04-27T04:27:56.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35557.535157 secs 2024-04-27T04:27:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:57 smithi167 ceph-mon[32751]: pgmap v19485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35558.417941 secs 2024-04-27T04:27:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:57 smithi090 ceph-mon[29180]: pgmap v19485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:57.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35558.417941 secs 2024-04-27T04:27:57.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:57 smithi119 ceph-mon[32752]: pgmap v19485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:27:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35558.417941 secs 2024-04-27T04:27:59.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:27:59 smithi090 ceph-mon[29180]: pgmap v19486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:27:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:27:59 smithi167 ceph-mon[32751]: pgmap v19486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:27:59.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:27:59 smithi119 ceph-mon[32752]: pgmap v19486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:28:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:01 smithi167 ceph-mon[32751]: pgmap v19487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35562.535241 secs 2024-04-27T04:28:01.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:01 smithi090 ceph-mon[29180]: pgmap v19487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:01.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35562.535241 secs 2024-04-27T04:28:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:01 smithi119 ceph-mon[32752]: pgmap v19487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:01.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35562.535241 secs 2024-04-27T04:28:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35563.418053 secs 2024-04-27T04:28:02.345 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35563.418053 secs 2024-04-27T04:28:02.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35563.418053 secs 2024-04-27T04:28:03.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:03 smithi119 ceph-mon[32752]: pgmap v19488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:28:03.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:03 smithi167 ceph-mon[32751]: pgmap v19488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:28:03.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:03 smithi090 ceph-mon[29180]: pgmap v19488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:28:05.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:05 smithi090 ceph-mon[29180]: pgmap v19489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:05.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:05 smithi119 ceph-mon[32752]: pgmap v19489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:05.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:05 smithi167 ceph-mon[32751]: pgmap v19489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:06.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35567.535300 secs 2024-04-27T04:28:06.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35567.535300 secs 2024-04-27T04:28:06.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35567.535300 secs 2024-04-27T04:28:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:07 smithi090 ceph-mon[29180]: pgmap v19490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:07.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35568.418187 secs 2024-04-27T04:28:07.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:07 smithi119 ceph-mon[32752]: pgmap v19490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35568.418187 secs 2024-04-27T04:28:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:07 smithi167 ceph-mon[32751]: pgmap v19490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:07.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35568.418187 secs 2024-04-27T04:28:09.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:09 smithi090 ceph-mon[29180]: pgmap v19491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:09.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:09 smithi119 ceph-mon[32752]: pgmap v19491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:09.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:09 smithi167 ceph-mon[32751]: pgmap v19491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:11.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:11 smithi090 ceph-mon[29180]: pgmap v19492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:11.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35572.535387 secs 2024-04-27T04:28:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:11 smithi119 ceph-mon[32752]: pgmap v19492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:11.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35572.535387 secs 2024-04-27T04:28:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:11 smithi167 ceph-mon[32751]: pgmap v19492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:11.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35572.535387 secs 2024-04-27T04:28:12.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35573.418327 secs 2024-04-27T04:28:12.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35573.418327 secs 2024-04-27T04:28:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35573.418327 secs 2024-04-27T04:28:13.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:13 smithi090 ceph-mon[29180]: pgmap v19493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:13.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:13 smithi119 ceph-mon[32752]: pgmap v19493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:13.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:13 smithi167 ceph-mon[32751]: pgmap v19493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:15.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:15 smithi090 ceph-mon[29180]: pgmap v19494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:15.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:15 smithi119 ceph-mon[32752]: pgmap v19494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:15.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:15 smithi167 ceph-mon[32751]: pgmap v19494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:16.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35577.535469 secs 2024-04-27T04:28:16.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35577.535469 secs 2024-04-27T04:28:16.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35577.535469 secs 2024-04-27T04:28:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:17 smithi090 ceph-mon[29180]: pgmap v19495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:17.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35578.418472 secs 2024-04-27T04:28:17.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:17 smithi119 ceph-mon[32752]: pgmap v19495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35578.418472 secs 2024-04-27T04:28:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:17 smithi167 ceph-mon[32751]: pgmap v19495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:17.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35578.418472 secs 2024-04-27T04:28:19.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:19 smithi090 ceph-mon[29180]: pgmap v19496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:19 smithi119 ceph-mon[32752]: pgmap v19496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:19.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:19 smithi167 ceph-mon[32751]: pgmap v19496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:21.370 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:21 smithi167 ceph-mon[32751]: pgmap v19497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:21.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35582.535555 secs 2024-04-27T04:28:21.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:21 smithi090 ceph-mon[29180]: pgmap v19497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35582.535555 secs 2024-04-27T04:28:21.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:21 smithi119 ceph-mon[32752]: pgmap v19497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35582.535555 secs 2024-04-27T04:28:22.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35583.418564 secs 2024-04-27T04:28:22.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35583.418564 secs 2024-04-27T04:28:22.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35583.418564 secs 2024-04-27T04:28:23.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:23 smithi090 ceph-mon[29180]: pgmap v19498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:23.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:23 smithi119 ceph-mon[32752]: pgmap v19498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:23.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:23 smithi167 ceph-mon[32751]: pgmap v19498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:25.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:25 smithi090 ceph-mon[29180]: pgmap v19499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:25.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:25 smithi119 ceph-mon[32752]: pgmap v19499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:25.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:25 smithi167 ceph-mon[32751]: pgmap v19499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35587.535638 secs 2024-04-27T04:28:26.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35587.535638 secs 2024-04-27T04:28:26.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35587.535638 secs 2024-04-27T04:28:27.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:27 smithi119 ceph-mon[32752]: pgmap v19500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35588.418652 secs 2024-04-27T04:28:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:27 smithi167 ceph-mon[32751]: pgmap v19500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35588.418652 secs 2024-04-27T04:28:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:27 smithi090 ceph-mon[29180]: pgmap v19500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35588.418652 secs 2024-04-27T04:28:29.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:29 smithi119 ceph-mon[32752]: pgmap v19501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:29.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:29 smithi167 ceph-mon[32751]: pgmap v19501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:29.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:29 smithi090 ceph-mon[29180]: pgmap v19501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:31.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:31 smithi119 ceph-mon[32752]: pgmap v19502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:31 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 35592.535722 secs 2024-04-27T04:28:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:31 smithi119 ceph-mon[32752]: slow request 11520.637407 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:28:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:31 smithi167 ceph-mon[32751]: pgmap v19502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:31 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 35592.535722 secs 2024-04-27T04:28:31.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:31 smithi167 ceph-mon[32751]: slow request 11520.637407 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:28:31.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:31 smithi090 ceph-mon[29180]: pgmap v19502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:31 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 35592.535722 secs 2024-04-27T04:28:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:31 smithi090 ceph-mon[29180]: slow request 11520.637407 seconds old, received at 2024-04-27T01:16:29.651719+0000: client_request(client.24733:267898 getattr AsXsFs #0x100000001fb 2024-04-27T01:16:29.650233+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:28:32.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35593.418773 secs 2024-04-27T04:28:32.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35593.418773 secs 2024-04-27T04:28:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35593.418773 secs 2024-04-27T04:28:33.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:33 smithi119 ceph-mon[32752]: pgmap v19503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:33.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:33 smithi167 ceph-mon[32751]: pgmap v19503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:33 smithi090 ceph-mon[29180]: pgmap v19503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:35.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:35 smithi119 ceph-mon[32752]: pgmap v19504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:35.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:35 smithi167 ceph-mon[32751]: pgmap v19504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:35 smithi090 ceph-mon[29180]: pgmap v19504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:36.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35597.535802 secs 2024-04-27T04:28:36.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35597.535802 secs 2024-04-27T04:28:36.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35597.535802 secs 2024-04-27T04:28:37.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:37 smithi090 ceph-mon[29180]: pgmap v19505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:37.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35598.418878 secs 2024-04-27T04:28:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:37 smithi119 ceph-mon[32752]: pgmap v19505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:37.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35598.418878 secs 2024-04-27T04:28:37.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:37 smithi167 ceph-mon[32751]: pgmap v19505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35598.418878 secs 2024-04-27T04:28:39.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:39 smithi119 ceph-mon[32752]: pgmap v19506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:39.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:39 smithi167 ceph-mon[32751]: pgmap v19506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:39 smithi090 ceph-mon[29180]: pgmap v19506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:41.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:41 smithi119 ceph-mon[32752]: pgmap v19507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35602.535894 secs 2024-04-27T04:28:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:41 smithi167 ceph-mon[32751]: pgmap v19507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35602.535894 secs 2024-04-27T04:28:41.625 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:41 smithi090 ceph-mon[29180]: pgmap v19507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35602.535894 secs 2024-04-27T04:28:42.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35603.419018 secs 2024-04-27T04:28:42.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35603.419018 secs 2024-04-27T04:28:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35603.419018 secs 2024-04-27T04:28:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:43 smithi119 ceph-mon[32752]: pgmap v19508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:43.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:43 smithi167 ceph-mon[32751]: pgmap v19508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:43 smithi090 ceph-mon[29180]: pgmap v19508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:45.553 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:45 smithi119 ceph-mon[32752]: pgmap v19509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:45.567 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:45 smithi167 ceph-mon[32751]: pgmap v19509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:45 smithi090 ceph-mon[29180]: pgmap v19509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35607.535952 secs 2024-04-27T04:28:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:46 smithi119 ceph-mon[32752]: pgmap v19510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:47.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35608.419085 secs 2024-04-27T04:28:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35607.535952 secs 2024-04-27T04:28:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:46 smithi167 ceph-mon[32751]: pgmap v19510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:47.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35608.419085 secs 2024-04-27T04:28:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35607.535952 secs 2024-04-27T04:28:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:46 smithi090 ceph-mon[29180]: pgmap v19510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:47.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35608.419085 secs 2024-04-27T04:28:49.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:48 smithi119 ceph-mon[32752]: pgmap v19511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:49.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:48 smithi167 ceph-mon[32751]: pgmap v19511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:49.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:48 smithi090 ceph-mon[29180]: pgmap v19511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:51.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:50 smithi119 ceph-mon[32752]: pgmap v19512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:51.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:50 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35612.536013 secs 2024-04-27T04:28:51.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:50 smithi167 ceph-mon[32751]: pgmap v19512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:51.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:50 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35612.536013 secs 2024-04-27T04:28:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:50 smithi090 ceph-mon[29180]: pgmap v19512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:28:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:50 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35612.536013 secs 2024-04-27T04:28:52.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35613.419226 secs 2024-04-27T04:28:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35613.419226 secs 2024-04-27T04:28:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35613.419226 secs 2024-04-27T04:28:53.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:52 smithi167 ceph-mon[32751]: pgmap v19513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:28:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:28:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:28:53.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:28:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:52 smithi090 ceph-mon[29180]: pgmap v19513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:28:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:28:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:28:53.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:28:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:52 smithi119 ceph-mon[32752]: pgmap v19513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:28:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:28:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:28:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:28:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:28:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:53 smithi167 ceph-mon[32751]: pgmap v19514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:28:54.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:53 smithi167 ceph-mon[32751]: pgmap v19515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:28:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:53 smithi090 ceph-mon[29180]: pgmap v19514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:28:54.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:53 smithi090 ceph-mon[29180]: pgmap v19515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:28:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:53 smithi119 ceph-mon[32752]: pgmap v19514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:28:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:53 smithi119 ceph-mon[32752]: pgmap v19515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:28:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:55 smithi167 ceph-mon[32751]: pgmap v19516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:56.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:55 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35617.536071 secs 2024-04-27T04:28:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:55 smithi090 ceph-mon[29180]: pgmap v19516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:55 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35617.536071 secs 2024-04-27T04:28:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:55 smithi119 ceph-mon[32752]: pgmap v19516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:55 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35617.536071 secs 2024-04-27T04:28:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35618.419290 secs 2024-04-27T04:28:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35618.419290 secs 2024-04-27T04:28:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35618.419290 secs 2024-04-27T04:28:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:57 smithi090 ceph-mon[29180]: pgmap v19517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:58.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:57 smithi119 ceph-mon[32752]: pgmap v19517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:28:58.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:57 smithi167 ceph-mon[32751]: pgmap v19517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:00.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:28:59 smithi090 ceph-mon[29180]: pgmap v19518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:00.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:28:59 smithi119 ceph-mon[32752]: pgmap v19518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:00.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:28:59 smithi167 ceph-mon[32751]: pgmap v19518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:01.027 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:00 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35622.536131 secs 2024-04-27T04:29:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:00 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35622.536131 secs 2024-04-27T04:29:01.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:00 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35622.536131 secs 2024-04-27T04:29:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:01 smithi090 ceph-mon[29180]: pgmap v19519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:02.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35623.419414 secs 2024-04-27T04:29:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:01 smithi119 ceph-mon[32752]: pgmap v19519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35623.419414 secs 2024-04-27T04:29:02.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:01 smithi167 ceph-mon[32751]: pgmap v19519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35623.419414 secs 2024-04-27T04:29:04.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:03 smithi090 ceph-mon[29180]: pgmap v19520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:29:04.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:03 smithi119 ceph-mon[32752]: pgmap v19520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:29:04.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:03 smithi167 ceph-mon[32751]: pgmap v19520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:29:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:05 smithi090 ceph-mon[29180]: pgmap v19521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:06.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:05 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35627.536187 secs 2024-04-27T04:29:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:05 smithi119 ceph-mon[32752]: pgmap v19521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:05 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35627.536187 secs 2024-04-27T04:29:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:05 smithi167 ceph-mon[32751]: pgmap v19521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:05 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35627.536187 secs 2024-04-27T04:29:07.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35628.419535 secs 2024-04-27T04:29:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35628.419535 secs 2024-04-27T04:29:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35628.419535 secs 2024-04-27T04:29:08.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:07 smithi090 ceph-mon[29180]: pgmap v19522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:08.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:07 smithi119 ceph-mon[32752]: pgmap v19522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:08.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:07 smithi167 ceph-mon[32751]: pgmap v19522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:09 smithi090 ceph-mon[29180]: pgmap v19523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:10.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:09 smithi119 ceph-mon[32752]: pgmap v19523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:10.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:09 smithi167 ceph-mon[32751]: pgmap v19523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:10 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35632.536251 secs 2024-04-27T04:29:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:10 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35632.536251 secs 2024-04-27T04:29:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:10 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35632.536251 secs 2024-04-27T04:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:11 smithi090 ceph-mon[29180]: pgmap v19524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35633.419658 secs 2024-04-27T04:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:11 smithi119 ceph-mon[32752]: pgmap v19524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35633.419658 secs 2024-04-27T04:29:12.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:11 smithi167 ceph-mon[32751]: pgmap v19524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35633.419658 secs 2024-04-27T04:29:14.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:13 smithi090 ceph-mon[29180]: pgmap v19525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:14.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:13 smithi119 ceph-mon[32752]: pgmap v19525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:14.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:13 smithi167 ceph-mon[32751]: pgmap v19525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:15 smithi090 ceph-mon[29180]: pgmap v19526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:16.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:15 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35637.536332 secs 2024-04-27T04:29:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:15 smithi119 ceph-mon[32752]: pgmap v19526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:15 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35637.536332 secs 2024-04-27T04:29:16.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:15 smithi167 ceph-mon[32751]: pgmap v19526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:15 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35637.536332 secs 2024-04-27T04:29:17.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:16 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35638.419799 secs 2024-04-27T04:29:17.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:16 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35638.419799 secs 2024-04-27T04:29:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:16 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35638.419799 secs 2024-04-27T04:29:18.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:17 smithi119 ceph-mon[32752]: pgmap v19527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:18.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:17 smithi167 ceph-mon[32751]: pgmap v19527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:18.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:17 smithi090 ceph-mon[29180]: pgmap v19527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:20.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:19 smithi119 ceph-mon[32752]: pgmap v19528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:20.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:19 smithi167 ceph-mon[32751]: pgmap v19528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:20.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:19 smithi090 ceph-mon[29180]: pgmap v19528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:21.205 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:20 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35642.536419 secs 2024-04-27T04:29:21.219 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:20 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35642.536419 secs 2024-04-27T04:29:21.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:20 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35642.536419 secs 2024-04-27T04:29:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:21 smithi119 ceph-mon[32752]: pgmap v19529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35643.419924 secs 2024-04-27T04:29:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:21 smithi167 ceph-mon[32751]: pgmap v19529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35643.419924 secs 2024-04-27T04:29:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:21 smithi090 ceph-mon[29180]: pgmap v19529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35643.419924 secs 2024-04-27T04:29:24.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:23 smithi119 ceph-mon[32752]: pgmap v19530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:24.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:23 smithi167 ceph-mon[32751]: pgmap v19530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:24.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:23 smithi090 ceph-mon[29180]: pgmap v19530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:26.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:25 smithi119 ceph-mon[32752]: pgmap v19531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:25 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35647.536500 secs 2024-04-27T04:29:26.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:25 smithi167 ceph-mon[32751]: pgmap v19531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:26.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:25 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35647.536500 secs 2024-04-27T04:29:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:25 smithi090 ceph-mon[29180]: pgmap v19531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:26.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:25 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35647.536500 secs 2024-04-27T04:29:27.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35648.420046 secs 2024-04-27T04:29:27.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35648.420046 secs 2024-04-27T04:29:27.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35648.420046 secs 2024-04-27T04:29:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:27 smithi119 ceph-mon[32752]: pgmap v19532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:28.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:27 smithi167 ceph-mon[32751]: pgmap v19532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:28.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:27 smithi090 ceph-mon[29180]: pgmap v19532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:29 smithi119 ceph-mon[32752]: pgmap v19533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:30.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:29 smithi167 ceph-mon[32751]: pgmap v19533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:30.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:29 smithi090 ceph-mon[29180]: pgmap v19533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:30 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35652.536585 secs 2024-04-27T04:29:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:30 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35652.536585 secs 2024-04-27T04:29:31.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:30 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35652.536585 secs 2024-04-27T04:29:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:31 smithi119 ceph-mon[32752]: pgmap v19534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35653.420168 secs 2024-04-27T04:29:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:31 smithi167 ceph-mon[32751]: pgmap v19534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35653.420168 secs 2024-04-27T04:29:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:31 smithi090 ceph-mon[29180]: pgmap v19534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:32.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35653.420168 secs 2024-04-27T04:29:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:33 smithi119 ceph-mon[32752]: pgmap v19535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:33 smithi167 ceph-mon[32751]: pgmap v19535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:34.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:33 smithi090 ceph-mon[29180]: pgmap v19535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:35 smithi119 ceph-mon[32752]: pgmap v19536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:35 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35657.536669 secs 2024-04-27T04:29:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:35 smithi167 ceph-mon[32751]: pgmap v19536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:35 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35657.536669 secs 2024-04-27T04:29:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:35 smithi090 ceph-mon[29180]: pgmap v19536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:36.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:35 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35657.536669 secs 2024-04-27T04:29:37.277 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35658.420301 secs 2024-04-27T04:29:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35658.420301 secs 2024-04-27T04:29:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35658.420301 secs 2024-04-27T04:29:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:37 smithi119 ceph-mon[32752]: pgmap v19537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:38.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:37 smithi167 ceph-mon[32751]: pgmap v19537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:38.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:37 smithi090 ceph-mon[29180]: pgmap v19537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:39 smithi119 ceph-mon[32752]: pgmap v19538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:40.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:39 smithi167 ceph-mon[32751]: pgmap v19538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:40.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:39 smithi090 ceph-mon[29180]: pgmap v19538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:40 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35662.536753 secs 2024-04-27T04:29:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:40 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35662.536753 secs 2024-04-27T04:29:41.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:40 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35662.536753 secs 2024-04-27T04:29:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:41 smithi119 ceph-mon[32752]: pgmap v19539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35663.420420 secs 2024-04-27T04:29:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:41 smithi167 ceph-mon[32751]: pgmap v19539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35663.420420 secs 2024-04-27T04:29:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:41 smithi090 ceph-mon[29180]: pgmap v19539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35663.420420 secs 2024-04-27T04:29:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:43 smithi119 ceph-mon[32752]: pgmap v19540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:43 smithi167 ceph-mon[32751]: pgmap v19540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:44.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:43 smithi090 ceph-mon[29180]: pgmap v19540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:45 smithi119 ceph-mon[32752]: pgmap v19541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:45 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35667.536796 secs 2024-04-27T04:29:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:45 smithi167 ceph-mon[32751]: pgmap v19541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:45 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35667.536796 secs 2024-04-27T04:29:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:45 smithi090 ceph-mon[29180]: pgmap v19541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:46.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:45 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35667.536796 secs 2024-04-27T04:29:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35668.420541 secs 2024-04-27T04:29:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35668.420541 secs 2024-04-27T04:29:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35668.420541 secs 2024-04-27T04:29:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:47 smithi119 ceph-mon[32752]: pgmap v19542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:48.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:47 smithi167 ceph-mon[32751]: pgmap v19542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:48.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:47 smithi090 ceph-mon[29180]: pgmap v19542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:49 smithi119 ceph-mon[32752]: pgmap v19543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:49 smithi167 ceph-mon[32751]: pgmap v19543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:50.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:49 smithi090 ceph-mon[29180]: pgmap v19543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:29:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:50 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35672.536844 secs 2024-04-27T04:29:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:50 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35672.536844 secs 2024-04-27T04:29:51.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:50 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35672.536844 secs 2024-04-27T04:29:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:51 smithi119 ceph-mon[32752]: pgmap v19544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35673.420664 secs 2024-04-27T04:29:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:51 smithi167 ceph-mon[32751]: pgmap v19544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35673.420664 secs 2024-04-27T04:29:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:51 smithi090 ceph-mon[29180]: pgmap v19544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:52.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35673.420664 secs 2024-04-27T04:29:53.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:29:53.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:29:53.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:29:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:53 smithi119 ceph-mon[32752]: pgmap v19545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:29:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:29:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:53 smithi119 ceph-mon[32752]: pgmap v19546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:29:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:53 smithi119 ceph-mon[32752]: pgmap v19547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:29:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:29:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:53 smithi167 ceph-mon[32751]: pgmap v19545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:29:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:29:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:53 smithi167 ceph-mon[32751]: pgmap v19546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:29:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:53 smithi167 ceph-mon[32751]: pgmap v19547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:29:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:29:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:53 smithi090 ceph-mon[29180]: pgmap v19545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:29:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:29:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:29:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:53 smithi090 ceph-mon[29180]: pgmap v19546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:29:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:53 smithi090 ceph-mon[29180]: pgmap v19547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:29:54.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:29:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:55 smithi090 ceph-mon[29180]: pgmap v19548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:55.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:55 smithi119 ceph-mon[32752]: pgmap v19548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:55 smithi167 ceph-mon[32751]: pgmap v19548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35677.536931 secs 2024-04-27T04:29:57.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35678.420774 secs 2024-04-27T04:29:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35677.536931 secs 2024-04-27T04:29:57.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35678.420774 secs 2024-04-27T04:29:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35677.536931 secs 2024-04-27T04:29:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35678.420774 secs 2024-04-27T04:29:58.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:57 smithi119 ceph-mon[32752]: pgmap v19549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:29:58.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:57 smithi167 ceph-mon[32751]: pgmap v19549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:29:58.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:57 smithi090 ceph-mon[29180]: pgmap v19549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:29:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:29:59 smithi090 ceph-mon[29180]: pgmap v19550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:59.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:29:59 smithi119 ceph-mon[32752]: pgmap v19550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:29:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:29:59 smithi167 ceph-mon[32751]: pgmap v19550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:00.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:30:00.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:30:00.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:30:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35682.536991 secs 2024-04-27T04:30:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:01 smithi119 ceph-mon[32752]: pgmap v19551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35682.536991 secs 2024-04-27T04:30:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:01 smithi167 ceph-mon[32751]: pgmap v19551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35682.536991 secs 2024-04-27T04:30:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:01 smithi090 ceph-mon[29180]: pgmap v19551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35683.420913 secs 2024-04-27T04:30:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35683.420913 secs 2024-04-27T04:30:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35683.420913 secs 2024-04-27T04:30:03.749 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:03 smithi090 ceph-mon[29180]: pgmap v19552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:30:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:03 smithi119 ceph-mon[32752]: pgmap v19552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:30:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:03 smithi167 ceph-mon[32751]: pgmap v19552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:30:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:05 smithi090 ceph-mon[29180]: pgmap v19553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:30:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:05 smithi119 ceph-mon[32752]: pgmap v19553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:30:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:05 smithi167 ceph-mon[32751]: pgmap v19553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:30:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35687.537048 secs 2024-04-27T04:30:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35687.537048 secs 2024-04-27T04:30:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35687.537048 secs 2024-04-27T04:30:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35688.421054 secs 2024-04-27T04:30:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:07 smithi119 ceph-mon[32752]: pgmap v19554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35688.421054 secs 2024-04-27T04:30:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:07 smithi167 ceph-mon[32751]: pgmap v19554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35688.421054 secs 2024-04-27T04:30:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:07 smithi090 ceph-mon[29180]: pgmap v19554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:09 smithi090 ceph-mon[29180]: pgmap v19555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:09 smithi119 ceph-mon[32752]: pgmap v19555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:09 smithi167 ceph-mon[32751]: pgmap v19555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:11.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:10 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35692.537111 secs 2024-04-27T04:30:11.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:10 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35692.537111 secs 2024-04-27T04:30:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:10 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35692.537111 secs 2024-04-27T04:30:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:11 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35693.421192 secs 2024-04-27T04:30:12.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:11 smithi090 ceph-mon[29180]: pgmap v19556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:11 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35693.421192 secs 2024-04-27T04:30:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:11 smithi119 ceph-mon[32752]: pgmap v19556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:11 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35693.421192 secs 2024-04-27T04:30:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:11 smithi167 ceph-mon[32751]: pgmap v19556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:13.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:13 smithi090 ceph-mon[29180]: pgmap v19557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:13 smithi119 ceph-mon[32752]: pgmap v19557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:13 smithi167 ceph-mon[32751]: pgmap v19557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:15 smithi119 ceph-mon[32752]: pgmap v19558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:30:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:15 smithi167 ceph-mon[32751]: pgmap v19558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:30:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:15 smithi090 ceph-mon[29180]: pgmap v19558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:30:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35697.537191 secs 2024-04-27T04:30:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35697.537191 secs 2024-04-27T04:30:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35697.537191 secs 2024-04-27T04:30:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35698.421302 secs 2024-04-27T04:30:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:17 smithi119 ceph-mon[32752]: pgmap v19559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35698.421302 secs 2024-04-27T04:30:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:17 smithi167 ceph-mon[32751]: pgmap v19559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35698.421302 secs 2024-04-27T04:30:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:17 smithi090 ceph-mon[29180]: pgmap v19559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:19.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:19 smithi119 ceph-mon[32752]: pgmap v19560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:19.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:19 smithi167 ceph-mon[32751]: pgmap v19560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:19.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:19 smithi090 ceph-mon[29180]: pgmap v19560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:21.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:20 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35702.537276 secs 2024-04-27T04:30:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:20 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35702.537276 secs 2024-04-27T04:30:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:20 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35702.537276 secs 2024-04-27T04:30:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:21 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35703.421415 secs 2024-04-27T04:30:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:21 smithi090 ceph-mon[29180]: pgmap v19561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:21 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35703.421415 secs 2024-04-27T04:30:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:21 smithi119 ceph-mon[32752]: pgmap v19561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:21 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35703.421415 secs 2024-04-27T04:30:22.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:21 smithi167 ceph-mon[32751]: pgmap v19561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:23.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:23 smithi119 ceph-mon[32752]: pgmap v19562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:23.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:23 smithi167 ceph-mon[32751]: pgmap v19562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:23.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:23 smithi090 ceph-mon[29180]: pgmap v19562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:25.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:25 smithi119 ceph-mon[32752]: pgmap v19563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:25.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:25 smithi167 ceph-mon[32751]: pgmap v19563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:25.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:25 smithi090 ceph-mon[29180]: pgmap v19563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35707.537346 secs 2024-04-27T04:30:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35707.537346 secs 2024-04-27T04:30:26.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35707.537346 secs 2024-04-27T04:30:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35708.421547 secs 2024-04-27T04:30:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:27 smithi119 ceph-mon[32752]: pgmap v19564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35708.421547 secs 2024-04-27T04:30:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:27 smithi167 ceph-mon[32751]: pgmap v19564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35708.421547 secs 2024-04-27T04:30:27.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:27 smithi090 ceph-mon[29180]: pgmap v19564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:29 smithi119 ceph-mon[32752]: pgmap v19565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:29 smithi167 ceph-mon[32751]: pgmap v19565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:29.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:29 smithi090 ceph-mon[29180]: pgmap v19565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:31.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:30 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35712.537419 secs 2024-04-27T04:30:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:30 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35712.537419 secs 2024-04-27T04:30:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:30 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35712.537419 secs 2024-04-27T04:30:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35713.421709 secs 2024-04-27T04:30:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:31 smithi090 ceph-mon[29180]: pgmap v19566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35713.421709 secs 2024-04-27T04:30:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:31 smithi119 ceph-mon[32752]: pgmap v19566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35713.421709 secs 2024-04-27T04:30:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:31 smithi167 ceph-mon[32751]: pgmap v19566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:33.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:33 smithi119 ceph-mon[32752]: pgmap v19567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:33.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:33 smithi167 ceph-mon[32751]: pgmap v19567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:33.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:33 smithi090 ceph-mon[29180]: pgmap v19567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:35.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:35 smithi119 ceph-mon[32752]: pgmap v19568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:35.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:35 smithi167 ceph-mon[32751]: pgmap v19568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:35.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:35 smithi090 ceph-mon[29180]: pgmap v19568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:36.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35717.537500 secs 2024-04-27T04:30:36.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35717.537500 secs 2024-04-27T04:30:36.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35717.537500 secs 2024-04-27T04:30:37.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35718.421822 secs 2024-04-27T04:30:37.777 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:37 smithi090 ceph-mon[29180]: pgmap v19569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35718.421822 secs 2024-04-27T04:30:37.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:37 smithi119 ceph-mon[32752]: pgmap v19569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35718.421822 secs 2024-04-27T04:30:37.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:37 smithi167 ceph-mon[32751]: pgmap v19569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:39.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:39 smithi119 ceph-mon[32752]: pgmap v19570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:39.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:39 smithi167 ceph-mon[32751]: pgmap v19570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:39.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:39 smithi090 ceph-mon[29180]: pgmap v19570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:30:41.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:40 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35722.537580 secs 2024-04-27T04:30:41.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:40 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35722.537580 secs 2024-04-27T04:30:41.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:40 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35722.537580 secs 2024-04-27T04:30:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35723.421974 secs 2024-04-27T04:30:42.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:41 smithi090 ceph-mon[29180]: pgmap v19571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35723.421974 secs 2024-04-27T04:30:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:41 smithi119 ceph-mon[32752]: pgmap v19571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35723.421974 secs 2024-04-27T04:30:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:41 smithi167 ceph-mon[32751]: pgmap v19571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:30:43.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:43 smithi119 ceph-mon[32752]: pgmap v19572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:43.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:43 smithi167 ceph-mon[32751]: pgmap v19572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:43.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:43 smithi090 ceph-mon[29180]: pgmap v19572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:45.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:45 smithi119 ceph-mon[32752]: pgmap v19573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:45.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:45 smithi167 ceph-mon[32751]: pgmap v19573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:45.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:45 smithi090 ceph-mon[29180]: pgmap v19573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:46.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35727.537657 secs 2024-04-27T04:30:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35727.537657 secs 2024-04-27T04:30:46.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35727.537657 secs 2024-04-27T04:30:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35728.422085 secs 2024-04-27T04:30:47.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:47 smithi119 ceph-mon[32752]: pgmap v19574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35728.422085 secs 2024-04-27T04:30:47.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:47 smithi167 ceph-mon[32751]: pgmap v19574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35728.422085 secs 2024-04-27T04:30:47.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:47 smithi090 ceph-mon[29180]: pgmap v19574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:49.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:49 smithi119 ceph-mon[32752]: pgmap v19575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:49 smithi167 ceph-mon[32751]: pgmap v19575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:49.829 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:49 smithi090 ceph-mon[29180]: pgmap v19575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:51.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:50 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35732.537739 secs 2024-04-27T04:30:51.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:50 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35732.537739 secs 2024-04-27T04:30:51.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:50 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35732.537739 secs 2024-04-27T04:30:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35733.422183 secs 2024-04-27T04:30:52.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:51 smithi090 ceph-mon[29180]: pgmap v19576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35733.422183 secs 2024-04-27T04:30:52.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:51 smithi119 ceph-mon[32752]: pgmap v19576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35733.422183 secs 2024-04-27T04:30:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:51 smithi167 ceph-mon[32751]: pgmap v19576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:30:53.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:53 smithi119 ceph-mon[32752]: pgmap v19577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:53 smithi167 ceph-mon[32751]: pgmap v19577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:53.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:53 smithi090 ceph-mon[29180]: pgmap v19577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:30:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:30:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:30:54.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:30:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:54 smithi119 ceph-mon[32752]: pgmap v19578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:30:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:54 smithi119 ceph-mon[32752]: pgmap v19579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:30:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:30:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:30:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:30:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:54 smithi167 ceph-mon[32751]: pgmap v19578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:30:54.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:54 smithi167 ceph-mon[32751]: pgmap v19579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:30:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:30:54.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:30:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:30:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:54 smithi090 ceph-mon[29180]: pgmap v19578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:30:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:54 smithi090 ceph-mon[29180]: pgmap v19579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:30:56.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:55 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35737.537819 secs 2024-04-27T04:30:56.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:55 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35737.537819 secs 2024-04-27T04:30:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:55 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35737.537819 secs 2024-04-27T04:30:57.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:56 smithi090 ceph-mon[29180]: pgmap v19580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35738.422327 secs 2024-04-27T04:30:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:56 smithi119 ceph-mon[32752]: pgmap v19580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35738.422327 secs 2024-04-27T04:30:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:56 smithi167 ceph-mon[32751]: pgmap v19580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:30:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35738.422327 secs 2024-04-27T04:30:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:30:59 smithi090 ceph-mon[29180]: pgmap v19581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:30:59.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:30:59 smithi119 ceph-mon[32752]: pgmap v19581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:30:59.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:30:59 smithi167 ceph-mon[32751]: pgmap v19581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:31:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:01 smithi119 ceph-mon[32752]: pgmap v19582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35742.537908 secs 2024-04-27T04:31:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:01 smithi167 ceph-mon[32751]: pgmap v19582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35742.537908 secs 2024-04-27T04:31:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:01 smithi090 ceph-mon[29180]: pgmap v19582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:01.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35742.537908 secs 2024-04-27T04:31:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35743.422416 secs 2024-04-27T04:31:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35743.422416 secs 2024-04-27T04:31:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35743.422416 secs 2024-04-27T04:31:03.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:03 smithi119 ceph-mon[32752]: pgmap v19583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:31:03.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:03 smithi167 ceph-mon[32751]: pgmap v19583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:31:03.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:03 smithi090 ceph-mon[29180]: pgmap v19583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:31:05.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:05 smithi119 ceph-mon[32752]: pgmap v19584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:31:05.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:05 smithi167 ceph-mon[32751]: pgmap v19584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:31:05.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:05 smithi090 ceph-mon[29180]: pgmap v19584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:31:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35747.537988 secs 2024-04-27T04:31:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35747.537988 secs 2024-04-27T04:31:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35747.537988 secs 2024-04-27T04:31:07.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:07 smithi090 ceph-mon[29180]: pgmap v19585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:07.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35748.422528 secs 2024-04-27T04:31:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:07 smithi119 ceph-mon[32752]: pgmap v19585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35748.422528 secs 2024-04-27T04:31:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:07 smithi167 ceph-mon[32751]: pgmap v19585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35748.422528 secs 2024-04-27T04:31:09.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:09 smithi119 ceph-mon[32752]: pgmap v19586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:09.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:09 smithi167 ceph-mon[32751]: pgmap v19586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:09.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:09 smithi090 ceph-mon[29180]: pgmap v19586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:11.126 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:31:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:31:10.667+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:31:11.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:31:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:31:10.667+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:31:11.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:31:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:31:10.667+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:31:11.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:31:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:31:10.667+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:31:11.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:31:10 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:31:10.667+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:31:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:11 smithi119 ceph-mon[32752]: pgmap v19587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:11.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:11 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 35752.538092 secs 2024-04-27T04:31:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:11 smithi119 ceph-mon[32752]: slow request 5761.229696 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:31:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:11 smithi167 ceph-mon[32751]: pgmap v19587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:11.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:11 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 35752.538092 secs 2024-04-27T04:31:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:11 smithi167 ceph-mon[32751]: slow request 5761.229696 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:31:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:11 smithi090 ceph-mon[29180]: pgmap v19587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:11 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 35752.538092 secs 2024-04-27T04:31:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:11 smithi090 ceph-mon[29180]: slow request 5761.229696 seconds old, received at 2024-04-27T02:55:09.061800+0000: client_request(client.26990:1 getattr pAsLsXsFs #0x1 2024-04-27T02:55:09.057458+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:31:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:12 smithi119 ceph-mon[32752]: 31 slow requests, 1 included below; oldest blocked for > 35753.422638 secs 2024-04-27T04:31:12.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:12 smithi119 ceph-mon[32752]: slow request 2881.275206 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:31:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:12 smithi167 ceph-mon[32751]: 31 slow requests, 1 included below; oldest blocked for > 35753.422638 secs 2024-04-27T04:31:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:12 smithi167 ceph-mon[32751]: slow request 2881.275206 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:31:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:12 smithi090 ceph-mon[29180]: 31 slow requests, 1 included below; oldest blocked for > 35753.422638 secs 2024-04-27T04:31:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:12 smithi090 ceph-mon[29180]: slow request 2881.275206 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:31:13.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:13 smithi119 ceph-mon[32752]: pgmap v19588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:13.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:13 smithi167 ceph-mon[32751]: pgmap v19588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:13 smithi090 ceph-mon[29180]: pgmap v19588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:15.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:15 smithi119 ceph-mon[32752]: pgmap v19589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:15.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:15 smithi167 ceph-mon[32751]: pgmap v19589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:15.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:15 smithi090 ceph-mon[29180]: pgmap v19589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:16.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35757.538173 secs 2024-04-27T04:31:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35757.538173 secs 2024-04-27T04:31:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35757.538173 secs 2024-04-27T04:31:17.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:17 smithi119 ceph-mon[32752]: pgmap v19590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35758.422704 secs 2024-04-27T04:31:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:17 smithi167 ceph-mon[32751]: pgmap v19590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:17.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35758.422704 secs 2024-04-27T04:31:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:17 smithi090 ceph-mon[29180]: pgmap v19590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35758.422704 secs 2024-04-27T04:31:19.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:19 smithi119 ceph-mon[32752]: pgmap v19591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:19.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:19 smithi167 ceph-mon[32751]: pgmap v19591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:19.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:19 smithi090 ceph-mon[29180]: pgmap v19591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:21 smithi119 ceph-mon[32752]: pgmap v19592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:21.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35762.538263 secs 2024-04-27T04:31:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:21 smithi167 ceph-mon[32751]: pgmap v19592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:21.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35762.538263 secs 2024-04-27T04:31:21.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:21 smithi090 ceph-mon[29180]: pgmap v19592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:21.574 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35762.538263 secs 2024-04-27T04:31:22.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35763.422819 secs 2024-04-27T04:31:22.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35763.422819 secs 2024-04-27T04:31:22.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35763.422819 secs 2024-04-27T04:31:23.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:23 smithi119 ceph-mon[32752]: pgmap v19593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:23.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:23 smithi167 ceph-mon[32751]: pgmap v19593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:23.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:23 smithi090 ceph-mon[29180]: pgmap v19593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:31:25.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:25 smithi119 ceph-mon[32752]: pgmap v19594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:25.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:25 smithi167 ceph-mon[32751]: pgmap v19594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:25.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:25 smithi090 ceph-mon[29180]: pgmap v19594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:31:26.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35767.538342 secs 2024-04-27T04:31:26.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35767.538342 secs 2024-04-27T04:31:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35767.538342 secs 2024-04-27T04:31:27.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:27 smithi119 ceph-mon[32752]: pgmap v19595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35768.422917 secs 2024-04-27T04:31:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:27 smithi167 ceph-mon[32751]: pgmap v19595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:27.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35768.422917 secs 2024-04-27T04:31:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:27 smithi090 ceph-mon[29180]: pgmap v19595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:27.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35768.422917 secs 2024-04-27T04:31:29.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:29 smithi119 ceph-mon[32752]: pgmap v19596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:29.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:29 smithi167 ceph-mon[32751]: pgmap v19596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:29.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:29 smithi090 ceph-mon[29180]: pgmap v19596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:31 smithi119 ceph-mon[32752]: pgmap v19597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:31.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35772.538430 secs 2024-04-27T04:31:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:31 smithi167 ceph-mon[32751]: pgmap v19597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35772.538430 secs 2024-04-27T04:31:31.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:31 smithi090 ceph-mon[29180]: pgmap v19597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35772.538430 secs 2024-04-27T04:31:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35773.423030 secs 2024-04-27T04:31:32.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35773.423030 secs 2024-04-27T04:31:32.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35773.423030 secs 2024-04-27T04:31:33.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:33 smithi119 ceph-mon[32752]: pgmap v19598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:33.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:33 smithi167 ceph-mon[32751]: pgmap v19598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:33.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:33 smithi090 ceph-mon[29180]: pgmap v19598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:35.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:35 smithi119 ceph-mon[32752]: pgmap v19599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:35.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:35 smithi167 ceph-mon[32751]: pgmap v19599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:35.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:35 smithi090 ceph-mon[29180]: pgmap v19599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:36.527 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35777.538513 secs 2024-04-27T04:31:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35777.538513 secs 2024-04-27T04:31:36.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35777.538513 secs 2024-04-27T04:31:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:37 smithi119 ceph-mon[32752]: pgmap v19600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35778.423153 secs 2024-04-27T04:31:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:37 smithi167 ceph-mon[32751]: pgmap v19600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:37.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35778.423153 secs 2024-04-27T04:31:37.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:37 smithi090 ceph-mon[29180]: pgmap v19600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35778.423153 secs 2024-04-27T04:31:39.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:39 smithi119 ceph-mon[32752]: pgmap v19601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:39.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:39 smithi167 ceph-mon[32751]: pgmap v19601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:39.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:39 smithi090 ceph-mon[29180]: pgmap v19601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:41.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:41 smithi119 ceph-mon[32752]: pgmap v19602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35782.538595 secs 2024-04-27T04:31:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:41 smithi167 ceph-mon[32751]: pgmap v19602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:41.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35782.538595 secs 2024-04-27T04:31:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:41 smithi090 ceph-mon[29180]: pgmap v19602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:41.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35782.538595 secs 2024-04-27T04:31:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35783.423270 secs 2024-04-27T04:31:42.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35783.423270 secs 2024-04-27T04:31:42.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35783.423270 secs 2024-04-27T04:31:43.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:43 smithi119 ceph-mon[32752]: pgmap v19603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:43 smithi167 ceph-mon[32751]: pgmap v19603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:43.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:43 smithi090 ceph-mon[29180]: pgmap v19603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:45.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:45 smithi119 ceph-mon[32752]: pgmap v19604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:45.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:45 smithi167 ceph-mon[32751]: pgmap v19604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:45.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:45 smithi090 ceph-mon[29180]: pgmap v19604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:46.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35787.538653 secs 2024-04-27T04:31:46.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35787.538653 secs 2024-04-27T04:31:46.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35787.538653 secs 2024-04-27T04:31:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:47 smithi119 ceph-mon[32752]: pgmap v19605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:47.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35788.423414 secs 2024-04-27T04:31:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:47 smithi167 ceph-mon[32751]: pgmap v19605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:47.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35788.423414 secs 2024-04-27T04:31:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:47 smithi090 ceph-mon[29180]: pgmap v19605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:47.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35788.423414 secs 2024-04-27T04:31:49.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:49 smithi119 ceph-mon[32752]: pgmap v19606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:49 smithi167 ceph-mon[32751]: pgmap v19606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:49 smithi090 ceph-mon[29180]: pgmap v19606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:51.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:51 smithi119 ceph-mon[32752]: pgmap v19607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35792.538714 secs 2024-04-27T04:31:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:51 smithi167 ceph-mon[32751]: pgmap v19607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:51.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35792.538714 secs 2024-04-27T04:31:51.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:51 smithi090 ceph-mon[29180]: pgmap v19607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35792.538714 secs 2024-04-27T04:31:52.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35793.423587 secs 2024-04-27T04:31:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35793.423587 secs 2024-04-27T04:31:52.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35793.423587 secs 2024-04-27T04:31:53.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:53 smithi119 ceph-mon[32752]: pgmap v19608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:53.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:53 smithi167 ceph-mon[32751]: pgmap v19608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:53.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:53 smithi090 ceph-mon[29180]: pgmap v19608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:31:54.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:31:54.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:31:54.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:31:55.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:55 smithi119 ceph-mon[32752]: pgmap v19609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:31:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:31:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:31:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:55 smithi167 ceph-mon[32751]: pgmap v19609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:31:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:31:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:31:55.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:55 smithi090 ceph-mon[29180]: pgmap v19609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:31:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:31:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:31:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:31:56.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:56 smithi119 ceph-mon[32752]: pgmap v19610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:31:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:56 smithi119 ceph-mon[32752]: pgmap v19611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35797.538775 secs 2024-04-27T04:31:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:56 smithi167 ceph-mon[32751]: pgmap v19610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:31:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:56 smithi167 ceph-mon[32751]: pgmap v19611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35797.538775 secs 2024-04-27T04:31:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:56 smithi090 ceph-mon[29180]: pgmap v19610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:31:56.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:56 smithi090 ceph-mon[29180]: pgmap v19611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35797.538775 secs 2024-04-27T04:31:57.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35798.423663 secs 2024-04-27T04:31:57.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35798.423663 secs 2024-04-27T04:31:57.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35798.423663 secs 2024-04-27T04:31:58.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:31:58 smithi119 ceph-mon[32752]: pgmap v19612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:58.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:31:58 smithi167 ceph-mon[32751]: pgmap v19612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:31:58.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:31:58 smithi090 ceph-mon[29180]: pgmap v19612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:32:00.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:00 smithi119 ceph-mon[32752]: pgmap v19613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:32:00.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:00 smithi167 ceph-mon[32751]: pgmap v19613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:32:00.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:00 smithi090 ceph-mon[29180]: pgmap v19613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:32:01.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35802.538837 secs 2024-04-27T04:32:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35802.538837 secs 2024-04-27T04:32:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35802.538837 secs 2024-04-27T04:32:02.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:02 smithi119 ceph-mon[32752]: pgmap v19614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:32:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35803.423818 secs 2024-04-27T04:32:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:02 smithi167 ceph-mon[32751]: pgmap v19614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:32:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35803.423818 secs 2024-04-27T04:32:02.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:02 smithi090 ceph-mon[29180]: pgmap v19614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:32:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35803.423818 secs 2024-04-27T04:32:04.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:04 smithi119 ceph-mon[32752]: pgmap v19615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:32:04.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:04 smithi167 ceph-mon[32751]: pgmap v19615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:32:04.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:04 smithi090 ceph-mon[29180]: pgmap v19615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:32:06.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:06 smithi119 ceph-mon[32752]: pgmap v19616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:32:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35807.538899 secs 2024-04-27T04:32:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:06 smithi167 ceph-mon[32751]: pgmap v19616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:32:06.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35807.538899 secs 2024-04-27T04:32:06.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:06 smithi090 ceph-mon[29180]: pgmap v19616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:32:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35807.538899 secs 2024-04-27T04:32:07.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35808.423892 secs 2024-04-27T04:32:07.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35808.423892 secs 2024-04-27T04:32:07.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35808.423892 secs 2024-04-27T04:32:08.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:08 smithi119 ceph-mon[32752]: pgmap v19617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:08 smithi167 ceph-mon[32751]: pgmap v19617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:08.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:08 smithi090 ceph-mon[29180]: pgmap v19617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:10.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:10 smithi119 ceph-mon[32752]: pgmap v19618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:10 smithi167 ceph-mon[32751]: pgmap v19618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:10.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:10 smithi090 ceph-mon[29180]: pgmap v19618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35812.538958 secs 2024-04-27T04:32:11.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35812.538958 secs 2024-04-27T04:32:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35812.538958 secs 2024-04-27T04:32:12.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:12 smithi167 ceph-mon[32751]: pgmap v19619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35813.423967 secs 2024-04-27T04:32:12.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:12 smithi090 ceph-mon[29180]: pgmap v19619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35813.423967 secs 2024-04-27T04:32:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:12 smithi119 ceph-mon[32752]: pgmap v19619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35813.423967 secs 2024-04-27T04:32:14.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:14 smithi167 ceph-mon[32751]: pgmap v19620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:14.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:14 smithi090 ceph-mon[29180]: pgmap v19620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:14.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:14 smithi119 ceph-mon[32752]: pgmap v19620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:16.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:16 smithi167 ceph-mon[32751]: pgmap v19621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35817.539017 secs 2024-04-27T04:32:16.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:16 smithi090 ceph-mon[29180]: pgmap v19621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35817.539017 secs 2024-04-27T04:32:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:16 smithi119 ceph-mon[32752]: pgmap v19621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35817.539017 secs 2024-04-27T04:32:17.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35818.424057 secs 2024-04-27T04:32:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35818.424057 secs 2024-04-27T04:32:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35818.424057 secs 2024-04-27T04:32:18.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:18 smithi090 ceph-mon[29180]: pgmap v19622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:18.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:18 smithi119 ceph-mon[32752]: pgmap v19622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:18.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:18 smithi167 ceph-mon[32751]: pgmap v19622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:20 smithi090 ceph-mon[29180]: pgmap v19623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:20.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:20 smithi119 ceph-mon[32752]: pgmap v19623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:20 smithi167 ceph-mon[32751]: pgmap v19623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35822.539078 secs 2024-04-27T04:32:21.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35822.539078 secs 2024-04-27T04:32:21.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35822.539078 secs 2024-04-27T04:32:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:22 smithi090 ceph-mon[29180]: pgmap v19624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35823.424177 secs 2024-04-27T04:32:22.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:22 smithi119 ceph-mon[32752]: pgmap v19624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35823.424177 secs 2024-04-27T04:32:22.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:22 smithi167 ceph-mon[32751]: pgmap v19624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35823.424177 secs 2024-04-27T04:32:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:24 smithi090 ceph-mon[29180]: pgmap v19625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:24.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:24 smithi119 ceph-mon[32752]: pgmap v19625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:24.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:24 smithi167 ceph-mon[32751]: pgmap v19625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:26.626 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:26 smithi090 ceph-mon[29180]: pgmap v19626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35827.539159 secs 2024-04-27T04:32:26.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:26 smithi119 ceph-mon[32752]: pgmap v19626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35827.539159 secs 2024-04-27T04:32:26.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:26 smithi167 ceph-mon[32751]: pgmap v19626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35827.539159 secs 2024-04-27T04:32:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35828.424299 secs 2024-04-27T04:32:27.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35828.424299 secs 2024-04-27T04:32:27.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35828.424299 secs 2024-04-27T04:32:28.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:28 smithi090 ceph-mon[29180]: pgmap v19627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:28.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:28 smithi119 ceph-mon[32752]: pgmap v19627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:28.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:28 smithi167 ceph-mon[32751]: pgmap v19627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:29.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:29 smithi090 ceph-mon[29180]: pgmap v19628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:29.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:29 smithi119 ceph-mon[32752]: pgmap v19628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:29.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:29 smithi167 ceph-mon[32751]: pgmap v19628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:30 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35832.539235 secs 2024-04-27T04:32:31.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:30 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35832.539235 secs 2024-04-27T04:32:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:30 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35832.539235 secs 2024-04-27T04:32:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:31 smithi090 ceph-mon[29180]: pgmap v19629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35833.424421 secs 2024-04-27T04:32:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:31 smithi119 ceph-mon[32752]: pgmap v19629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35833.424421 secs 2024-04-27T04:32:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:31 smithi167 ceph-mon[32751]: pgmap v19629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:32.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35833.424421 secs 2024-04-27T04:32:34.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:33 smithi119 ceph-mon[32752]: pgmap v19630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:34.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:33 smithi167 ceph-mon[32751]: pgmap v19630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:34.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:33 smithi090 ceph-mon[29180]: pgmap v19630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:36.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:35 smithi119 ceph-mon[32752]: pgmap v19631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:35 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35837.539312 secs 2024-04-27T04:32:36.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:35 smithi167 ceph-mon[32751]: pgmap v19631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:35 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35837.539312 secs 2024-04-27T04:32:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:35 smithi090 ceph-mon[29180]: pgmap v19631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:35 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35837.539312 secs 2024-04-27T04:32:37.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35838.424532 secs 2024-04-27T04:32:37.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35838.424532 secs 2024-04-27T04:32:37.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35838.424532 secs 2024-04-27T04:32:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:37 smithi119 ceph-mon[32752]: pgmap v19632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:38.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:37 smithi167 ceph-mon[32751]: pgmap v19632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:37 smithi090 ceph-mon[29180]: pgmap v19632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:40.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:39 smithi119 ceph-mon[32752]: pgmap v19633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:40.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:39 smithi167 ceph-mon[32751]: pgmap v19633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:40.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:39 smithi090 ceph-mon[29180]: pgmap v19633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:40 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 35842.539396 secs 2024-04-27T04:32:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:40 smithi119 ceph-mon[32752]: slow request 722.652041 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:32:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:40 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 35842.539396 secs 2024-04-27T04:32:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:40 smithi167 ceph-mon[32751]: slow request 722.652041 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:32:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:40 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 35842.539396 secs 2024-04-27T04:32:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:40 smithi090 ceph-mon[29180]: slow request 722.652041 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:32:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:41 smithi119 ceph-mon[32752]: pgmap v19634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:41 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35843.424644 secs 2024-04-27T04:32:42.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:41 smithi167 ceph-mon[32751]: pgmap v19634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:41 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35843.424644 secs 2024-04-27T04:32:42.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:41 smithi090 ceph-mon[29180]: pgmap v19634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:41 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35843.424644 secs 2024-04-27T04:32:44.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:43 smithi119 ceph-mon[32752]: pgmap v19635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:44.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:43 smithi167 ceph-mon[32751]: pgmap v19635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:44.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:43 smithi090 ceph-mon[29180]: pgmap v19635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:46.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:45 smithi119 ceph-mon[32752]: pgmap v19636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:45 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35847.539470 secs 2024-04-27T04:32:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:45 smithi167 ceph-mon[32751]: pgmap v19636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:46.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:45 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35847.539470 secs 2024-04-27T04:32:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:45 smithi090 ceph-mon[29180]: pgmap v19636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:45 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35847.539470 secs 2024-04-27T04:32:47.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:46 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35848.424771 secs 2024-04-27T04:32:47.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:46 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35848.424771 secs 2024-04-27T04:32:47.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:46 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35848.424771 secs 2024-04-27T04:32:48.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:47 smithi119 ceph-mon[32752]: pgmap v19637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:47 smithi167 ceph-mon[32751]: pgmap v19637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:47 smithi090 ceph-mon[29180]: pgmap v19637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:50.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:49 smithi119 ceph-mon[32752]: pgmap v19638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:50.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:49 smithi167 ceph-mon[32751]: pgmap v19638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:50.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:49 smithi090 ceph-mon[29180]: pgmap v19638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:51.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:50 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35852.539554 secs 2024-04-27T04:32:51.220 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:50 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35852.539554 secs 2024-04-27T04:32:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:50 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35852.539554 secs 2024-04-27T04:32:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:51 smithi119 ceph-mon[32752]: pgmap v19639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:51 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35853.424899 secs 2024-04-27T04:32:52.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:51 smithi167 ceph-mon[32751]: pgmap v19639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:51 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35853.424899 secs 2024-04-27T04:32:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:51 smithi090 ceph-mon[29180]: pgmap v19639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:51 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35853.424899 secs 2024-04-27T04:32:54.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:53 smithi119 ceph-mon[32752]: pgmap v19640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:54.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:53 smithi167 ceph-mon[32751]: pgmap v19640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:54.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:53 smithi090 ceph-mon[29180]: pgmap v19640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:32:55.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:32:55.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:32:55.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:32:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:55 smithi119 ceph-mon[32752]: pgmap v19641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:55 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35857.539644 secs 2024-04-27T04:32:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:32:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:32:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:32:56.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:55 smithi167 ceph-mon[32751]: pgmap v19641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:55 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35857.539644 secs 2024-04-27T04:32:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:32:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:32:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:32:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:55 smithi090 ceph-mon[29180]: pgmap v19641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:32:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:55 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35857.539644 secs 2024-04-27T04:32:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:32:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:32:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:32:57.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:56 smithi119 ceph-mon[32752]: pgmap v19642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T04:32:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:56 smithi119 ceph-mon[32752]: pgmap v19643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:32:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:56 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35858.424991 secs 2024-04-27T04:32:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:56 smithi167 ceph-mon[32751]: pgmap v19642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T04:32:57.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:56 smithi167 ceph-mon[32751]: pgmap v19643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:32:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:56 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35858.424991 secs 2024-04-27T04:32:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:56 smithi090 ceph-mon[29180]: pgmap v19642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T04:32:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:56 smithi090 ceph-mon[29180]: pgmap v19643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:32:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:56 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35858.424991 secs 2024-04-27T04:32:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:32:58 smithi119 ceph-mon[32752]: pgmap v19644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T04:32:59.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:32:58 smithi167 ceph-mon[32751]: pgmap v19644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T04:32:59.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:32:58 smithi090 ceph-mon[29180]: pgmap v19644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T04:33:01.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:00 smithi119 ceph-mon[32752]: pgmap v19645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:33:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:00 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35862.539728 secs 2024-04-27T04:33:01.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:00 smithi167 ceph-mon[32751]: pgmap v19645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:33:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:00 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35862.539728 secs 2024-04-27T04:33:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:00 smithi090 ceph-mon[29180]: pgmap v19645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:33:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:00 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35862.539728 secs 2024-04-27T04:33:02.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:01 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35863.425106 secs 2024-04-27T04:33:02.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:01 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35863.425106 secs 2024-04-27T04:33:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:01 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35863.425106 secs 2024-04-27T04:33:03.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:02 smithi119 ceph-mon[32752]: pgmap v19646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:33:03.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:02 smithi167 ceph-mon[32751]: pgmap v19646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:33:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:02 smithi090 ceph-mon[29180]: pgmap v19646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:33:05.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:04 smithi119 ceph-mon[32752]: pgmap v19647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:33:05.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:04 smithi167 ceph-mon[32751]: pgmap v19647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:33:05.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:04 smithi090 ceph-mon[29180]: pgmap v19647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:33:06.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:05 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35867.539827 secs 2024-04-27T04:33:06.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:05 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35867.539827 secs 2024-04-27T04:33:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:05 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35867.539827 secs 2024-04-27T04:33:07.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:06 smithi090 ceph-mon[29180]: pgmap v19648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:33:07.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:06 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35868.425207 secs 2024-04-27T04:33:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:06 smithi119 ceph-mon[32752]: pgmap v19648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:33:07.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:06 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35868.425207 secs 2024-04-27T04:33:07.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:06 smithi167 ceph-mon[32751]: pgmap v19648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:33:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:06 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35868.425207 secs 2024-04-27T04:33:09.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:09 smithi119 ceph-mon[32752]: pgmap v19649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:09.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:09 smithi167 ceph-mon[32751]: pgmap v19649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:09.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:09 smithi090 ceph-mon[29180]: pgmap v19649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:11 smithi119 ceph-mon[32752]: pgmap v19650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35872.539912 secs 2024-04-27T04:33:11.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:11 smithi167 ceph-mon[32751]: pgmap v19650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35872.539912 secs 2024-04-27T04:33:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:11 smithi090 ceph-mon[29180]: pgmap v19650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35872.539912 secs 2024-04-27T04:33:12.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35873.425353 secs 2024-04-27T04:33:12.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35873.425353 secs 2024-04-27T04:33:12.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35873.425353 secs 2024-04-27T04:33:13.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:13 smithi119 ceph-mon[32752]: pgmap v19651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:13.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:13 smithi167 ceph-mon[32751]: pgmap v19651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:13.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:13 smithi090 ceph-mon[29180]: pgmap v19651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:15.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:15 smithi119 ceph-mon[32752]: pgmap v19652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:15.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:15 smithi167 ceph-mon[32751]: pgmap v19652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:15.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:15 smithi090 ceph-mon[29180]: pgmap v19652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:16.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35877.539992 secs 2024-04-27T04:33:16.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35877.539992 secs 2024-04-27T04:33:16.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35877.539992 secs 2024-04-27T04:33:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:17 smithi119 ceph-mon[32752]: pgmap v19653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35878.425463 secs 2024-04-27T04:33:17.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:17 smithi167 ceph-mon[32751]: pgmap v19653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35878.425463 secs 2024-04-27T04:33:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:17 smithi090 ceph-mon[29180]: pgmap v19653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:17.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35878.425463 secs 2024-04-27T04:33:19.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:19 smithi119 ceph-mon[32752]: pgmap v19654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:19.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:19 smithi167 ceph-mon[32751]: pgmap v19654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:19.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:19 smithi090 ceph-mon[29180]: pgmap v19654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:21.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:21 smithi119 ceph-mon[32752]: pgmap v19655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35882.540037 secs 2024-04-27T04:33:21.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:21 smithi167 ceph-mon[32751]: pgmap v19655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35882.540037 secs 2024-04-27T04:33:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:21 smithi090 ceph-mon[29180]: pgmap v19655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:21.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35882.540037 secs 2024-04-27T04:33:22.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35883.425583 secs 2024-04-27T04:33:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35883.425583 secs 2024-04-27T04:33:22.376 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35883.425583 secs 2024-04-27T04:33:23.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:23 smithi119 ceph-mon[32752]: pgmap v19656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:23.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:23 smithi167 ceph-mon[32751]: pgmap v19656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:23.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:23 smithi090 ceph-mon[29180]: pgmap v19656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:25 smithi119 ceph-mon[32752]: pgmap v19657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:25 smithi167 ceph-mon[32751]: pgmap v19657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:25.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:25 smithi090 ceph-mon[29180]: pgmap v19657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35887.540111 secs 2024-04-27T04:33:26.348 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35887.540111 secs 2024-04-27T04:33:26.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35887.540111 secs 2024-04-27T04:33:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:27 smithi119 ceph-mon[32752]: pgmap v19658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35888.425649 secs 2024-04-27T04:33:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:27 smithi167 ceph-mon[32751]: pgmap v19658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35888.425649 secs 2024-04-27T04:33:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:27 smithi090 ceph-mon[29180]: pgmap v19658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35888.425649 secs 2024-04-27T04:33:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:29 smithi119 ceph-mon[32752]: pgmap v19659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:29.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:29 smithi167 ceph-mon[32751]: pgmap v19659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:29 smithi090 ceph-mon[29180]: pgmap v19659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:31 smithi119 ceph-mon[32752]: pgmap v19660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35892.540170 secs 2024-04-27T04:33:31.318 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:31 smithi167 ceph-mon[32751]: pgmap v19660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35892.540170 secs 2024-04-27T04:33:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:31 smithi090 ceph-mon[29180]: pgmap v19660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35892.540170 secs 2024-04-27T04:33:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35893.425792 secs 2024-04-27T04:33:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35893.425792 secs 2024-04-27T04:33:32.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35893.425792 secs 2024-04-27T04:33:33.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:33 smithi167 ceph-mon[32751]: pgmap v19661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:33 smithi090 ceph-mon[29180]: pgmap v19661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:33 smithi119 ceph-mon[32752]: pgmap v19661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:35.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:35 smithi167 ceph-mon[32751]: pgmap v19662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:35.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:35 smithi090 ceph-mon[29180]: pgmap v19662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:35 smithi119 ceph-mon[32752]: pgmap v19662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35897.540252 secs 2024-04-27T04:33:36.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35897.540252 secs 2024-04-27T04:33:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35897.540252 secs 2024-04-27T04:33:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:37 smithi090 ceph-mon[29180]: pgmap v19663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35898.425917 secs 2024-04-27T04:33:37.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:37 smithi119 ceph-mon[32752]: pgmap v19663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35898.425917 secs 2024-04-27T04:33:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:37 smithi167 ceph-mon[32751]: pgmap v19663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35898.425917 secs 2024-04-27T04:33:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:39 smithi090 ceph-mon[29180]: pgmap v19664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:39.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:39 smithi119 ceph-mon[32752]: pgmap v19664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:39 smithi167 ceph-mon[32751]: pgmap v19664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:41 smithi090 ceph-mon[29180]: pgmap v19665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35902.540355 secs 2024-04-27T04:33:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:41 smithi119 ceph-mon[32752]: pgmap v19665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35902.540355 secs 2024-04-27T04:33:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:41 smithi167 ceph-mon[32751]: pgmap v19665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35902.540355 secs 2024-04-27T04:33:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35903.426035 secs 2024-04-27T04:33:42.554 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35903.426035 secs 2024-04-27T04:33:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35903.426035 secs 2024-04-27T04:33:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:43 smithi090 ceph-mon[29180]: pgmap v19666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:43 smithi119 ceph-mon[32752]: pgmap v19666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:43.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:43 smithi167 ceph-mon[32751]: pgmap v19666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:45 smithi090 ceph-mon[29180]: pgmap v19667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:45.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:45 smithi119 ceph-mon[32752]: pgmap v19667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:45 smithi167 ceph-mon[32751]: pgmap v19667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:46 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 35907.540440 secs 2024-04-27T04:33:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:46 smithi090 ceph-mon[29180]: slow request 1444.987044 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:33:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:46 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 35907.540440 secs 2024-04-27T04:33:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:46 smithi119 ceph-mon[32752]: slow request 1444.987044 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:33:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:46 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 35907.540440 secs 2024-04-27T04:33:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:46 smithi167 ceph-mon[32751]: slow request 1444.987044 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:33:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:47 smithi090 ceph-mon[29180]: pgmap v19668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35908.426142 secs 2024-04-27T04:33:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:47 smithi119 ceph-mon[32752]: pgmap v19668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35908.426142 secs 2024-04-27T04:33:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:47 smithi167 ceph-mon[32751]: pgmap v19668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35908.426142 secs 2024-04-27T04:33:49.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:49 smithi090 ceph-mon[29180]: pgmap v19669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:49.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:49 smithi119 ceph-mon[32752]: pgmap v19669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:49.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:49 smithi167 ceph-mon[32751]: pgmap v19669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:51.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:51 smithi119 ceph-mon[32752]: pgmap v19670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:51.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35912.540529 secs 2024-04-27T04:33:51.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:51 smithi167 ceph-mon[32751]: pgmap v19670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:51.371 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35912.540529 secs 2024-04-27T04:33:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:51 smithi090 ceph-mon[29180]: pgmap v19670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35912.540529 secs 2024-04-27T04:33:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35913.426265 secs 2024-04-27T04:33:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35913.426265 secs 2024-04-27T04:33:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35913.426265 secs 2024-04-27T04:33:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:53 smithi090 ceph-mon[29180]: pgmap v19671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:53.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:53 smithi119 ceph-mon[32752]: pgmap v19671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:53.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:53 smithi167 ceph-mon[32751]: pgmap v19671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:55.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:55 smithi090 ceph-mon[29180]: pgmap v19672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:55 smithi119 ceph-mon[32752]: pgmap v19672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:55.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:55 smithi167 ceph-mon[32751]: pgmap v19672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:33:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35917.540616 secs 2024-04-27T04:33:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:33:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35917.540616 secs 2024-04-27T04:33:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:33:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35917.540616 secs 2024-04-27T04:33:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:33:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:57 smithi090 ceph-mon[29180]: pgmap v19673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35918.426352 secs 2024-04-27T04:33:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:33:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:33:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:33:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:57 smithi119 ceph-mon[32752]: pgmap v19673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35918.426352 secs 2024-04-27T04:33:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:33:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:33:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:33:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:57 smithi167 ceph-mon[32751]: pgmap v19673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:33:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35918.426352 secs 2024-04-27T04:33:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:33:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:33:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:33:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:58 smithi090 ceph-mon[29180]: pgmap v19674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T04:33:58.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:33:58 smithi090 ceph-mon[29180]: pgmap v19675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:33:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:58 smithi119 ceph-mon[32752]: pgmap v19674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T04:33:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:33:58 smithi119 ceph-mon[32752]: pgmap v19675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:33:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:58 smithi167 ceph-mon[32751]: pgmap v19674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T04:33:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:33:58 smithi167 ceph-mon[32751]: pgmap v19675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:00.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:00 smithi090 ceph-mon[29180]: pgmap v19676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:00 smithi119 ceph-mon[32752]: pgmap v19676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:00 smithi167 ceph-mon[32751]: pgmap v19676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35922.540671 secs 2024-04-27T04:34:01.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35922.540671 secs 2024-04-27T04:34:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35922.540671 secs 2024-04-27T04:34:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:02 smithi119 ceph-mon[32752]: pgmap v19677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35923.426457 secs 2024-04-27T04:34:02.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:02 smithi167 ceph-mon[32751]: pgmap v19677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35923.426457 secs 2024-04-27T04:34:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:02 smithi090 ceph-mon[29180]: pgmap v19677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35923.426457 secs 2024-04-27T04:34:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:04 smithi119 ceph-mon[32752]: pgmap v19678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T04:34:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:04 smithi167 ceph-mon[32751]: pgmap v19678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T04:34:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:04 smithi090 ceph-mon[29180]: pgmap v19678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T04:34:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:06 smithi119 ceph-mon[32752]: pgmap v19679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35927.540728 secs 2024-04-27T04:34:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:06 smithi167 ceph-mon[32751]: pgmap v19679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35927.540728 secs 2024-04-27T04:34:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:06 smithi090 ceph-mon[29180]: pgmap v19679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:34:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35927.540728 secs 2024-04-27T04:34:07.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35928.426521 secs 2024-04-27T04:34:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35928.426521 secs 2024-04-27T04:34:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35928.426521 secs 2024-04-27T04:34:08.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:08 smithi119 ceph-mon[32752]: pgmap v19680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:34:08.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:08 smithi167 ceph-mon[32751]: pgmap v19680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:34:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:08 smithi090 ceph-mon[29180]: pgmap v19680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:34:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:10 smithi119 ceph-mon[32752]: pgmap v19681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:10.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:10 smithi167 ceph-mon[32751]: pgmap v19681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:10 smithi090 ceph-mon[29180]: pgmap v19681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35932.540794 secs 2024-04-27T04:34:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35932.540794 secs 2024-04-27T04:34:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35932.540794 secs 2024-04-27T04:34:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:12 smithi119 ceph-mon[32752]: pgmap v19682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35933.426645 secs 2024-04-27T04:34:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:12 smithi167 ceph-mon[32751]: pgmap v19682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35933.426645 secs 2024-04-27T04:34:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:12 smithi090 ceph-mon[29180]: pgmap v19682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35933.426645 secs 2024-04-27T04:34:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:14 smithi119 ceph-mon[32752]: pgmap v19683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:14 smithi167 ceph-mon[32751]: pgmap v19683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:14 smithi090 ceph-mon[29180]: pgmap v19683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:16 smithi119 ceph-mon[32752]: pgmap v19684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35937.540852 secs 2024-04-27T04:34:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:16 smithi167 ceph-mon[32751]: pgmap v19684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35937.540852 secs 2024-04-27T04:34:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:16 smithi090 ceph-mon[29180]: pgmap v19684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35937.540852 secs 2024-04-27T04:34:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35938.426766 secs 2024-04-27T04:34:17.614 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35938.426766 secs 2024-04-27T04:34:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35938.426766 secs 2024-04-27T04:34:18.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:18 smithi119 ceph-mon[32752]: pgmap v19685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:18 smithi167 ceph-mon[32751]: pgmap v19685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:18.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:18 smithi090 ceph-mon[29180]: pgmap v19685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:20 smithi167 ceph-mon[32751]: pgmap v19686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:20 smithi090 ceph-mon[29180]: pgmap v19686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:21.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:20 smithi119 ceph-mon[32752]: pgmap v19686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:21.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35942.540911 secs 2024-04-27T04:34:21.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35942.540911 secs 2024-04-27T04:34:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35942.540911 secs 2024-04-27T04:34:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:22 smithi119 ceph-mon[32752]: pgmap v19687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35943.426862 secs 2024-04-27T04:34:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:22 smithi167 ceph-mon[32751]: pgmap v19687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35943.426862 secs 2024-04-27T04:34:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:22 smithi090 ceph-mon[29180]: pgmap v19687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35943.426862 secs 2024-04-27T04:34:24.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:24 smithi119 ceph-mon[32752]: pgmap v19688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:24.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:24 smithi167 ceph-mon[32751]: pgmap v19688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:24 smithi090 ceph-mon[29180]: pgmap v19688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:26 smithi119 ceph-mon[32752]: pgmap v19689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35947.540970 secs 2024-04-27T04:34:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:26 smithi167 ceph-mon[32751]: pgmap v19689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35947.540970 secs 2024-04-27T04:34:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:26 smithi090 ceph-mon[29180]: pgmap v19689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35947.540970 secs 2024-04-27T04:34:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35948.426953 secs 2024-04-27T04:34:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35948.426953 secs 2024-04-27T04:34:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35948.426953 secs 2024-04-27T04:34:28.454 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:28 smithi090 ceph-mon[29180]: pgmap v19690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:28.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:28 smithi119 ceph-mon[32752]: pgmap v19690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:28.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:28 smithi167 ceph-mon[32751]: pgmap v19690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:30 smithi119 ceph-mon[32752]: pgmap v19691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:30 smithi167 ceph-mon[32751]: pgmap v19691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:30 smithi090 ceph-mon[29180]: pgmap v19691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35952.541024 secs 2024-04-27T04:34:31.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35952.541024 secs 2024-04-27T04:34:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35952.541024 secs 2024-04-27T04:34:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:32 smithi119 ceph-mon[32752]: pgmap v19692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35953.427069 secs 2024-04-27T04:34:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:32 smithi167 ceph-mon[32751]: pgmap v19692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35953.427069 secs 2024-04-27T04:34:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:32 smithi090 ceph-mon[29180]: pgmap v19692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35953.427069 secs 2024-04-27T04:34:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:34 smithi119 ceph-mon[32752]: pgmap v19693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:34.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:34 smithi167 ceph-mon[32751]: pgmap v19693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:34 smithi090 ceph-mon[29180]: pgmap v19693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:36 smithi119 ceph-mon[32752]: pgmap v19694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35957.541074 secs 2024-04-27T04:34:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:36 smithi167 ceph-mon[32751]: pgmap v19694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35957.541074 secs 2024-04-27T04:34:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:36 smithi090 ceph-mon[29180]: pgmap v19694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35957.541074 secs 2024-04-27T04:34:37.528 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35958.427184 secs 2024-04-27T04:34:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35958.427184 secs 2024-04-27T04:34:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35958.427184 secs 2024-04-27T04:34:38.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:38 smithi119 ceph-mon[32752]: pgmap v19695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:38.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:38 smithi167 ceph-mon[32751]: pgmap v19695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:38.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:38 smithi090 ceph-mon[29180]: pgmap v19695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:40 smithi119 ceph-mon[32752]: pgmap v19696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:40 smithi167 ceph-mon[32751]: pgmap v19696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:40 smithi090 ceph-mon[29180]: pgmap v19696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35962.541133 secs 2024-04-27T04:34:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35962.541133 secs 2024-04-27T04:34:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35962.541133 secs 2024-04-27T04:34:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:42 smithi119 ceph-mon[32752]: pgmap v19697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35963.427293 secs 2024-04-27T04:34:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:42 smithi167 ceph-mon[32751]: pgmap v19697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35963.427293 secs 2024-04-27T04:34:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:42 smithi090 ceph-mon[29180]: pgmap v19697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35963.427293 secs 2024-04-27T04:34:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:44 smithi119 ceph-mon[32752]: pgmap v19698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:44.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:44 smithi167 ceph-mon[32751]: pgmap v19698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:44.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:44 smithi090 ceph-mon[29180]: pgmap v19698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:46 smithi119 ceph-mon[32752]: pgmap v19699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35967.541190 secs 2024-04-27T04:34:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:46 smithi167 ceph-mon[32751]: pgmap v19699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35967.541190 secs 2024-04-27T04:34:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:46 smithi090 ceph-mon[29180]: pgmap v19699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35967.541190 secs 2024-04-27T04:34:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35968.427413 secs 2024-04-27T04:34:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35968.427413 secs 2024-04-27T04:34:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35968.427413 secs 2024-04-27T04:34:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:48 smithi119 ceph-mon[32752]: pgmap v19700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:48 smithi167 ceph-mon[32751]: pgmap v19700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:48 smithi090 ceph-mon[29180]: pgmap v19700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:50 smithi119 ceph-mon[32752]: pgmap v19701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:50 smithi167 ceph-mon[32751]: pgmap v19701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:50 smithi090 ceph-mon[29180]: pgmap v19701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35972.541278 secs 2024-04-27T04:34:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35972.541278 secs 2024-04-27T04:34:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35972.541278 secs 2024-04-27T04:34:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:52 smithi119 ceph-mon[32752]: pgmap v19702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35973.427529 secs 2024-04-27T04:34:52.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:52 smithi167 ceph-mon[32751]: pgmap v19702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35973.427529 secs 2024-04-27T04:34:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:52 smithi090 ceph-mon[29180]: pgmap v19702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35973.427529 secs 2024-04-27T04:34:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:54 smithi119 ceph-mon[32752]: pgmap v19703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:54 smithi167 ceph-mon[32751]: pgmap v19703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:54 smithi090 ceph-mon[29180]: pgmap v19703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:56 smithi119 ceph-mon[32752]: pgmap v19704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35977.541360 secs 2024-04-27T04:34:56.568 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:56 smithi167 ceph-mon[32751]: pgmap v19704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35977.541360 secs 2024-04-27T04:34:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:56 smithi090 ceph-mon[29180]: pgmap v19704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:34:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35977.541360 secs 2024-04-27T04:34:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35978.427649 secs 2024-04-27T04:34:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:34:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:34:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:34:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:34:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35978.427649 secs 2024-04-27T04:34:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:34:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:34:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:34:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:34:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35978.427649 secs 2024-04-27T04:34:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:34:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:34:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:34:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:34:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:58 smithi119 ceph-mon[32752]: pgmap v19705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:58 smithi119 ceph-mon[32752]: pgmap v19706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:34:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:34:58 smithi119 ceph-mon[32752]: pgmap v19707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:34:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:58 smithi167 ceph-mon[32751]: pgmap v19705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:58 smithi167 ceph-mon[32751]: pgmap v19706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:34:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:34:58 smithi167 ceph-mon[32751]: pgmap v19707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:34:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:58 smithi090 ceph-mon[29180]: pgmap v19705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:34:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:58 smithi090 ceph-mon[29180]: pgmap v19706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:34:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:34:58 smithi090 ceph-mon[29180]: pgmap v19707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:35:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:00 smithi119 ceph-mon[32752]: pgmap v19708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:00 smithi167 ceph-mon[32751]: pgmap v19708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:00 smithi090 ceph-mon[29180]: pgmap v19708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35982.541449 secs 2024-04-27T04:35:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35982.541449 secs 2024-04-27T04:35:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35982.541449 secs 2024-04-27T04:35:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35983.427767 secs 2024-04-27T04:35:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:02 smithi119 ceph-mon[32752]: pgmap v19709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35983.427767 secs 2024-04-27T04:35:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:02 smithi167 ceph-mon[32751]: pgmap v19709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35983.427767 secs 2024-04-27T04:35:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:02 smithi090 ceph-mon[29180]: pgmap v19709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:04 smithi119 ceph-mon[32752]: pgmap v19710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:35:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:04 smithi167 ceph-mon[32751]: pgmap v19710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:35:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:04 smithi090 ceph-mon[29180]: pgmap v19710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:35:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:06 smithi119 ceph-mon[32752]: pgmap v19711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35987.541536 secs 2024-04-27T04:35:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:06 smithi167 ceph-mon[32751]: pgmap v19711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35987.541536 secs 2024-04-27T04:35:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:06 smithi090 ceph-mon[29180]: pgmap v19711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:35:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35987.541536 secs 2024-04-27T04:35:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35988.427876 secs 2024-04-27T04:35:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35988.427876 secs 2024-04-27T04:35:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35988.427876 secs 2024-04-27T04:35:08.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:08 smithi119 ceph-mon[32752]: pgmap v19712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:35:08.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:08 smithi167 ceph-mon[32751]: pgmap v19712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:35:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:08 smithi090 ceph-mon[29180]: pgmap v19712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:35:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:10 smithi119 ceph-mon[32752]: pgmap v19713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:10.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:10 smithi167 ceph-mon[32751]: pgmap v19713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:10 smithi090 ceph-mon[29180]: pgmap v19713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35992.541623 secs 2024-04-27T04:35:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35992.541623 secs 2024-04-27T04:35:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35992.541623 secs 2024-04-27T04:35:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35993.427986 secs 2024-04-27T04:35:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:12 smithi119 ceph-mon[32752]: pgmap v19714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35993.427986 secs 2024-04-27T04:35:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:12 smithi167 ceph-mon[32751]: pgmap v19714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35993.427986 secs 2024-04-27T04:35:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:12 smithi090 ceph-mon[29180]: pgmap v19714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:14 smithi119 ceph-mon[32752]: pgmap v19715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:14 smithi167 ceph-mon[32751]: pgmap v19715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:14 smithi090 ceph-mon[29180]: pgmap v19715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:16 smithi167 ceph-mon[32751]: pgmap v19716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 35997.541708 secs 2024-04-27T04:35:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:16 smithi090 ceph-mon[29180]: pgmap v19716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 35997.541708 secs 2024-04-27T04:35:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:16 smithi119 ceph-mon[32752]: pgmap v19716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 35997.541708 secs 2024-04-27T04:35:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 35998.428121 secs 2024-04-27T04:35:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 35998.428121 secs 2024-04-27T04:35:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 35998.428121 secs 2024-04-27T04:35:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:18 smithi167 ceph-mon[32751]: pgmap v19717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:18.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:18 smithi090 ceph-mon[29180]: pgmap v19717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:18 smithi119 ceph-mon[32752]: pgmap v19717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:20 smithi167 ceph-mon[32751]: pgmap v19718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:20 smithi090 ceph-mon[29180]: pgmap v19718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:20 smithi119 ceph-mon[32752]: pgmap v19718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36002.541784 secs 2024-04-27T04:35:21.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36002.541784 secs 2024-04-27T04:35:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36002.541784 secs 2024-04-27T04:35:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36003.428261 secs 2024-04-27T04:35:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:22 smithi090 ceph-mon[29180]: pgmap v19719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36003.428261 secs 2024-04-27T04:35:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:22 smithi119 ceph-mon[32752]: pgmap v19719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36003.428261 secs 2024-04-27T04:35:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:22 smithi167 ceph-mon[32751]: pgmap v19719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:24 smithi090 ceph-mon[29180]: pgmap v19720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:24.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:24 smithi119 ceph-mon[32752]: pgmap v19720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:24.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:24 smithi167 ceph-mon[32751]: pgmap v19720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:26 smithi090 ceph-mon[29180]: pgmap v19721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36007.541840 secs 2024-04-27T04:35:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:26 smithi119 ceph-mon[32752]: pgmap v19721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36007.541840 secs 2024-04-27T04:35:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:26 smithi167 ceph-mon[32751]: pgmap v19721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36007.541840 secs 2024-04-27T04:35:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:27 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36008.428387 secs 2024-04-27T04:35:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:27 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36008.428387 secs 2024-04-27T04:35:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:27 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36008.428387 secs 2024-04-27T04:35:28.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:28 smithi090 ceph-mon[29180]: pgmap v19722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:28.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:28 smithi119 ceph-mon[32752]: pgmap v19722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:28 smithi167 ceph-mon[32751]: pgmap v19722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:30 smithi090 ceph-mon[29180]: pgmap v19723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:30 smithi119 ceph-mon[32752]: pgmap v19723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:30 smithi167 ceph-mon[32751]: pgmap v19723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36012.541924 secs 2024-04-27T04:35:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36012.541924 secs 2024-04-27T04:35:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36012.541924 secs 2024-04-27T04:35:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:32 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36013.428504 secs 2024-04-27T04:35:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:32 smithi090 ceph-mon[29180]: pgmap v19724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:32 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36013.428504 secs 2024-04-27T04:35:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:32 smithi119 ceph-mon[32752]: pgmap v19724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:32 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36013.428504 secs 2024-04-27T04:35:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:32 smithi167 ceph-mon[32751]: pgmap v19724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:34.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:34 smithi090 ceph-mon[29180]: pgmap v19725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:34 smithi119 ceph-mon[32752]: pgmap v19725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:34 smithi167 ceph-mon[32751]: pgmap v19725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:36 smithi090 ceph-mon[29180]: pgmap v19726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36017.542030 secs 2024-04-27T04:35:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:36 smithi119 ceph-mon[32752]: pgmap v19726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36017.542030 secs 2024-04-27T04:35:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:36 smithi167 ceph-mon[32751]: pgmap v19726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36017.542030 secs 2024-04-27T04:35:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:37 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36018.428624 secs 2024-04-27T04:35:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:37 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36018.428624 secs 2024-04-27T04:35:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:37 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36018.428624 secs 2024-04-27T04:35:38.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:38 smithi090 ceph-mon[29180]: pgmap v19727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:38 smithi119 ceph-mon[32752]: pgmap v19727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:38 smithi167 ceph-mon[32751]: pgmap v19727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:40 smithi090 ceph-mon[29180]: pgmap v19728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:40 smithi119 ceph-mon[32752]: pgmap v19728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:40 smithi167 ceph-mon[32751]: pgmap v19728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36022.542109 secs 2024-04-27T04:35:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36022.542109 secs 2024-04-27T04:35:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36022.542109 secs 2024-04-27T04:35:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:42 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36023.428708 secs 2024-04-27T04:35:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:42 smithi090 ceph-mon[29180]: pgmap v19729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:42 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36023.428708 secs 2024-04-27T04:35:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:42 smithi119 ceph-mon[32752]: pgmap v19729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:42 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36023.428708 secs 2024-04-27T04:35:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:42 smithi167 ceph-mon[32751]: pgmap v19729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:43 smithi119 ceph-mon[32752]: pgmap v19730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:43 smithi167 ceph-mon[32751]: pgmap v19730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:43 smithi090 ceph-mon[29180]: pgmap v19730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:46 smithi119 ceph-mon[32752]: pgmap v19731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36027.542188 secs 2024-04-27T04:35:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:46 smithi167 ceph-mon[32751]: pgmap v19731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36027.542188 secs 2024-04-27T04:35:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:46 smithi090 ceph-mon[29180]: pgmap v19731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36027.542188 secs 2024-04-27T04:35:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:47 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36028.428833 secs 2024-04-27T04:35:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:47 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36028.428833 secs 2024-04-27T04:35:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:47 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36028.428833 secs 2024-04-27T04:35:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:48 smithi119 ceph-mon[32752]: pgmap v19732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:48 smithi167 ceph-mon[32751]: pgmap v19732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:48 smithi090 ceph-mon[29180]: pgmap v19732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:50 smithi119 ceph-mon[32752]: pgmap v19733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:50 smithi167 ceph-mon[32751]: pgmap v19733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:50 smithi090 ceph-mon[29180]: pgmap v19733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36032.542242 secs 2024-04-27T04:35:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36032.542242 secs 2024-04-27T04:35:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36032.542242 secs 2024-04-27T04:35:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:52 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36033.428959 secs 2024-04-27T04:35:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:52 smithi119 ceph-mon[32752]: pgmap v19734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:52 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36033.428959 secs 2024-04-27T04:35:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:52 smithi167 ceph-mon[32751]: pgmap v19734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:52 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36033.428959 secs 2024-04-27T04:35:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:52 smithi090 ceph-mon[29180]: pgmap v19734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:54 smithi167 ceph-mon[32751]: pgmap v19735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:54 smithi090 ceph-mon[29180]: pgmap v19735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:54 smithi119 ceph-mon[32752]: pgmap v19735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:56 smithi167 ceph-mon[32751]: pgmap v19736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36037.542320 secs 2024-04-27T04:35:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:56 smithi090 ceph-mon[29180]: pgmap v19736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36037.542320 secs 2024-04-27T04:35:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:56 smithi119 ceph-mon[32752]: pgmap v19736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:35:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36037.542320 secs 2024-04-27T04:35:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:57 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36038.429112 secs 2024-04-27T04:35:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:57 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36038.429112 secs 2024-04-27T04:35:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:57 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36038.429112 secs 2024-04-27T04:35:58.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:58 smithi167 ceph-mon[32751]: pgmap v19737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:58.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:35:58.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:35:58 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:58 smithi090 ceph-mon[29180]: pgmap v19737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:35:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:35:58 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.629 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:58 smithi119 ceph-mon[32752]: pgmap v19737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:35:58.629 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:35:58.629 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.629 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.630 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:35:58.630 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:35:58 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: pgmap v19738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: pgmap v19738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: pgmap v19738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:36:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:36:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36042.542413 secs 2024-04-27T04:36:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:36:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:36:01.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:36:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36042.542413 secs 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:36:01.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:36:01.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:36:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:36:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36042.542413 secs 2024-04-27T04:36:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:36:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:36:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:36:01.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:36:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:36:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:36:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:02 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:36:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:02 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:36:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:02 smithi090 ceph-mon[29180]: pgmap v19739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-27T04:36:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:02 smithi090 ceph-mon[29180]: pgmap v19740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:02 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36043.429256 secs 2024-04-27T04:36:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:36:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:36:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:02 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:36:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:02 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:36:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:02 smithi119 ceph-mon[32752]: pgmap v19739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-27T04:36:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:02 smithi119 ceph-mon[32752]: pgmap v19740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:02 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36043.429256 secs 2024-04-27T04:36:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:36:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:36:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:02 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:36:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:02 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:36:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:02 smithi167 ceph-mon[32751]: pgmap v19739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 892 B/s rd, 1 op/s 2024-04-27T04:36:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:02 smithi167 ceph-mon[32751]: pgmap v19740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:02 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36043.429256 secs 2024-04-27T04:36:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:04 smithi090 ceph-mon[29180]: pgmap v19741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:04 smithi119 ceph-mon[32752]: pgmap v19741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:04 smithi167 ceph-mon[32751]: pgmap v19741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:06 smithi090 ceph-mon[29180]: pgmap v19742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36047.542494 secs 2024-04-27T04:36:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:06 smithi119 ceph-mon[32752]: pgmap v19742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36047.542494 secs 2024-04-27T04:36:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:06 smithi167 ceph-mon[32751]: pgmap v19742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36047.542494 secs 2024-04-27T04:36:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:07 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36048.429406 secs 2024-04-27T04:36:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:07 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36048.429406 secs 2024-04-27T04:36:07.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:07 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36048.429406 secs 2024-04-27T04:36:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:08 smithi090 ceph-mon[29180]: pgmap v19743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:08 smithi119 ceph-mon[32752]: pgmap v19743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:08 smithi167 ceph-mon[32751]: pgmap v19743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:10 smithi090 ceph-mon[29180]: pgmap v19744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:10.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:10 smithi119 ceph-mon[32752]: pgmap v19744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:10 smithi167 ceph-mon[32751]: pgmap v19744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:36:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:11 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 36052.542577 secs 2024-04-27T04:36:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:11 smithi090 ceph-mon[29180]: slow request 2880.329028 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:36:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:11 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 36052.542577 secs 2024-04-27T04:36:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:11 smithi119 ceph-mon[32752]: slow request 2880.329028 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:36:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:11 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 36052.542577 secs 2024-04-27T04:36:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:11 smithi167 ceph-mon[32751]: slow request 2880.329028 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:36:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:12 smithi090 ceph-mon[29180]: pgmap v19745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:36:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:12 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36053.429496 secs 2024-04-27T04:36:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:12 smithi119 ceph-mon[32752]: pgmap v19745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:36:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:12 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36053.429496 secs 2024-04-27T04:36:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:12 smithi167 ceph-mon[32751]: pgmap v19745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:36:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:12 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36053.429496 secs 2024-04-27T04:36:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:14 smithi090 ceph-mon[29180]: pgmap v19746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:14 smithi119 ceph-mon[32752]: pgmap v19746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:14.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:14 smithi167 ceph-mon[32751]: pgmap v19746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:16 smithi090 ceph-mon[29180]: pgmap v19747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36057.542660 secs 2024-04-27T04:36:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:16 smithi119 ceph-mon[32752]: pgmap v19747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36057.542660 secs 2024-04-27T04:36:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:16 smithi167 ceph-mon[32751]: pgmap v19747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36057.542660 secs 2024-04-27T04:36:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:17 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36058.429575 secs 2024-04-27T04:36:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:17 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36058.429575 secs 2024-04-27T04:36:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:17 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36058.429575 secs 2024-04-27T04:36:18.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:18 smithi090 ceph-mon[29180]: pgmap v19748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:18 smithi119 ceph-mon[32752]: pgmap v19748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:18.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:18 smithi167 ceph-mon[32751]: pgmap v19748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:20 smithi090 ceph-mon[29180]: pgmap v19749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:20 smithi119 ceph-mon[32752]: pgmap v19749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:20 smithi167 ceph-mon[32751]: pgmap v19749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:21.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36062.542742 secs 2024-04-27T04:36:21.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:21 smithi119 ceph-mon[32752]: pgmap v19750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:21.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36062.542742 secs 2024-04-27T04:36:21.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:21 smithi167 ceph-mon[32751]: pgmap v19750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36062.542742 secs 2024-04-27T04:36:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:21 smithi090 ceph-mon[29180]: pgmap v19750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:22 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36063.429706 secs 2024-04-27T04:36:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:22 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36063.429706 secs 2024-04-27T04:36:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:22 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36063.429706 secs 2024-04-27T04:36:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:23 smithi119 ceph-mon[32752]: pgmap v19751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:23.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:23 smithi167 ceph-mon[32751]: pgmap v19751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:23 smithi090 ceph-mon[29180]: pgmap v19751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T04:36:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:25 smithi119 ceph-mon[32752]: pgmap v19752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:25 smithi167 ceph-mon[32751]: pgmap v19752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:25 smithi090 ceph-mon[29180]: pgmap v19752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T04:36:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:26 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 36067.542822 secs 2024-04-27T04:36:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:26 smithi119 ceph-mon[32752]: slow request 23042.508928 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:36:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:26 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36068.429781 secs 2024-04-27T04:36:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:26 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 36067.542822 secs 2024-04-27T04:36:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:26 smithi167 ceph-mon[32751]: slow request 23042.508928 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:36:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:26 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36068.429781 secs 2024-04-27T04:36:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:26 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 36067.542822 secs 2024-04-27T04:36:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:26 smithi090 ceph-mon[29180]: slow request 23042.508928 seconds old, received at 2024-04-26T22:12:22.787298+0000: client_request(client.24733:267891 getattr AsXsFs #0x100000001fb 2024-04-26T22:12:22.785819+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:36:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:26 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36068.429781 secs 2024-04-27T04:36:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:27 smithi119 ceph-mon[32752]: pgmap v19753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:27 smithi167 ceph-mon[32751]: pgmap v19753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:27 smithi090 ceph-mon[29180]: pgmap v19753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:29 smithi119 ceph-mon[32752]: pgmap v19754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:29 smithi167 ceph-mon[32751]: pgmap v19754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:29 smithi090 ceph-mon[29180]: pgmap v19754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36072.542881 secs 2024-04-27T04:36:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:31 smithi119 ceph-mon[32752]: pgmap v19755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:31 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36073.429905 secs 2024-04-27T04:36:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36072.542881 secs 2024-04-27T04:36:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:31 smithi167 ceph-mon[32751]: pgmap v19755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:31 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36073.429905 secs 2024-04-27T04:36:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36072.542881 secs 2024-04-27T04:36:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:31 smithi090 ceph-mon[29180]: pgmap v19755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:31 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36073.429905 secs 2024-04-27T04:36:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:33 smithi119 ceph-mon[32752]: pgmap v19756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:33 smithi167 ceph-mon[32751]: pgmap v19756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:34.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:33 smithi090 ceph-mon[29180]: pgmap v19756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:35 smithi119 ceph-mon[32752]: pgmap v19757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:35 smithi167 ceph-mon[32751]: pgmap v19757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:35 smithi090 ceph-mon[29180]: pgmap v19757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36077.542940 secs 2024-04-27T04:36:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:36 smithi119 ceph-mon[32752]: 31 slow requests, 0 included below; oldest blocked for > 36078.429970 secs 2024-04-27T04:36:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36077.542940 secs 2024-04-27T04:36:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:36 smithi167 ceph-mon[32751]: 31 slow requests, 0 included below; oldest blocked for > 36078.429970 secs 2024-04-27T04:36:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36077.542940 secs 2024-04-27T04:36:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:36 smithi090 ceph-mon[29180]: 31 slow requests, 0 included below; oldest blocked for > 36078.429970 secs 2024-04-27T04:36:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:37 smithi119 ceph-mon[32752]: pgmap v19758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:37 smithi167 ceph-mon[32751]: pgmap v19758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:37 smithi090 ceph-mon[29180]: pgmap v19758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:39 smithi119 ceph-mon[32752]: pgmap v19759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:39 smithi167 ceph-mon[32751]: pgmap v19759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:39 smithi090 ceph-mon[29180]: pgmap v19759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:36:41.127 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:36:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:36:40.786+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:36:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:36:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:36:40.786+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:36:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:36:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:36:40.786+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:36:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:36:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:36:40.786+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:36:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:36:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:36:40.786+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:36:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36082.543027 secs 2024-04-27T04:36:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:41 smithi119 ceph-mon[32752]: pgmap v19760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:41 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36083.430103 secs 2024-04-27T04:36:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36082.543027 secs 2024-04-27T04:36:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:41 smithi167 ceph-mon[32751]: pgmap v19760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:41 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36083.430103 secs 2024-04-27T04:36:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36082.543027 secs 2024-04-27T04:36:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:41 smithi090 ceph-mon[29180]: pgmap v19760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:41 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36083.430103 secs 2024-04-27T04:36:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:43 smithi167 ceph-mon[32751]: pgmap v19761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:43 smithi090 ceph-mon[29180]: pgmap v19761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:44.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:43 smithi119 ceph-mon[32752]: pgmap v19761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:45 smithi167 ceph-mon[32751]: pgmap v19762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:36:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:36:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:45 smithi090 ceph-mon[29180]: pgmap v19762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:36:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:36:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:45 smithi119 ceph-mon[32752]: pgmap v19762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:36:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:36:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36087.543093 secs 2024-04-27T04:36:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:46 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36088.430226 secs 2024-04-27T04:36:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36087.543093 secs 2024-04-27T04:36:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:46 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36088.430226 secs 2024-04-27T04:36:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36087.543093 secs 2024-04-27T04:36:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:46 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36088.430226 secs 2024-04-27T04:36:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:47 smithi090 ceph-mon[29180]: pgmap v19763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:47 smithi119 ceph-mon[32752]: pgmap v19763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:47 smithi167 ceph-mon[32751]: pgmap v19763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:49 smithi090 ceph-mon[29180]: pgmap v19764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:49 smithi119 ceph-mon[32752]: pgmap v19764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:49 smithi167 ceph-mon[32751]: pgmap v19764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:50 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36092.543146 secs 2024-04-27T04:36:51.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:50 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36092.543146 secs 2024-04-27T04:36:51.221 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:50 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36092.543146 secs 2024-04-27T04:36:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:51 smithi090 ceph-mon[29180]: pgmap v19765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:51 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36093.430369 secs 2024-04-27T04:36:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:51 smithi119 ceph-mon[32752]: pgmap v19765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:51 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36093.430369 secs 2024-04-27T04:36:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:51 smithi167 ceph-mon[32751]: pgmap v19765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:51 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36093.430369 secs 2024-04-27T04:36:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:53 smithi090 ceph-mon[29180]: pgmap v19766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:54.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:53 smithi119 ceph-mon[32752]: pgmap v19766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:53 smithi167 ceph-mon[32751]: pgmap v19766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:55 smithi090 ceph-mon[29180]: pgmap v19767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:55 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36097.543223 secs 2024-04-27T04:36:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:55 smithi119 ceph-mon[32752]: pgmap v19767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:55 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36097.543223 secs 2024-04-27T04:36:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:55 smithi167 ceph-mon[32751]: pgmap v19767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 170 B/s wr, 13 op/s 2024-04-27T04:36:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:55 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36097.543223 secs 2024-04-27T04:36:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:56 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36098.430462 secs 2024-04-27T04:36:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:56 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36098.430462 secs 2024-04-27T04:36:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:56 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36098.430462 secs 2024-04-27T04:36:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:57 smithi090 ceph-mon[29180]: pgmap v19768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:58.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:57 smithi119 ceph-mon[32752]: pgmap v19768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:36:58.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:57 smithi167 ceph-mon[32751]: pgmap v19768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:36:59 smithi090 ceph-mon[29180]: pgmap v19769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:00.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:36:59 smithi119 ceph-mon[32752]: pgmap v19769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:36:59 smithi167 ceph-mon[32751]: pgmap v19769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:00 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36102.543314 secs 2024-04-27T04:37:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:00 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:37:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:00 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36102.543314 secs 2024-04-27T04:37:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:00 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:37:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:00 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36102.543314 secs 2024-04-27T04:37:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:00 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:37:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:02 smithi090 ceph-mon[29180]: pgmap v19770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36103.430525 secs 2024-04-27T04:37:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:37:02.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:37:02.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:02 smithi119 ceph-mon[32752]: pgmap v19770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36103.430525 secs 2024-04-27T04:37:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:37:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:37:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:02 smithi167 ceph-mon[32751]: pgmap v19770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36103.430525 secs 2024-04-27T04:37:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:37:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:37:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:37:03.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:03 smithi090 ceph-mon[29180]: pgmap v19771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T04:37:03.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:03 smithi090 ceph-mon[29180]: pgmap v19772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:37:03.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:03 smithi119 ceph-mon[32752]: pgmap v19771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T04:37:03.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:03 smithi119 ceph-mon[32752]: pgmap v19772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:37:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:03 smithi167 ceph-mon[32751]: pgmap v19771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 B/s rd, 1 op/s 2024-04-27T04:37:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:03 smithi167 ceph-mon[32751]: pgmap v19772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:37:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:05 smithi090 ceph-mon[29180]: pgmap v19773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:05 smithi119 ceph-mon[32752]: pgmap v19773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:05 smithi167 ceph-mon[32751]: pgmap v19773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36107.543369 secs 2024-04-27T04:37:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36107.543369 secs 2024-04-27T04:37:06.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36107.543369 secs 2024-04-27T04:37:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:07 smithi090 ceph-mon[29180]: pgmap v19774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36108.430589 secs 2024-04-27T04:37:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:07 smithi119 ceph-mon[32752]: pgmap v19774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36108.430589 secs 2024-04-27T04:37:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:07 smithi167 ceph-mon[32751]: pgmap v19774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36108.430589 secs 2024-04-27T04:37:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:09 smithi090 ceph-mon[29180]: pgmap v19775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:37:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:09 smithi119 ceph-mon[32752]: pgmap v19775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:37:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:09 smithi167 ceph-mon[32751]: pgmap v19775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:37:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:11 smithi090 ceph-mon[29180]: pgmap v19776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36112.543452 secs 2024-04-27T04:37:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:11 smithi119 ceph-mon[32752]: pgmap v19776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36112.543452 secs 2024-04-27T04:37:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:11 smithi167 ceph-mon[32751]: pgmap v19776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36112.543452 secs 2024-04-27T04:37:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36113.430718 secs 2024-04-27T04:37:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36113.430718 secs 2024-04-27T04:37:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36113.430718 secs 2024-04-27T04:37:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:13 smithi090 ceph-mon[29180]: pgmap v19777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:13 smithi119 ceph-mon[32752]: pgmap v19777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:13 smithi167 ceph-mon[32751]: pgmap v19777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:15.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:15 smithi090 ceph-mon[29180]: pgmap v19778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:15 smithi119 ceph-mon[32752]: pgmap v19778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:15 smithi167 ceph-mon[32751]: pgmap v19778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:37:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36117.543534 secs 2024-04-27T04:37:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36117.543534 secs 2024-04-27T04:37:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36117.543534 secs 2024-04-27T04:37:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:17 smithi119 ceph-mon[32752]: pgmap v19779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36118.430863 secs 2024-04-27T04:37:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:17 smithi167 ceph-mon[32751]: pgmap v19779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36118.430863 secs 2024-04-27T04:37:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:17 smithi090 ceph-mon[29180]: pgmap v19779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36118.430863 secs 2024-04-27T04:37:19.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:19 smithi090 ceph-mon[29180]: pgmap v19780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:19 smithi119 ceph-mon[32752]: pgmap v19780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:19 smithi167 ceph-mon[32751]: pgmap v19780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:21 smithi119 ceph-mon[32752]: pgmap v19781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36122.543618 secs 2024-04-27T04:37:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:21 smithi167 ceph-mon[32751]: pgmap v19781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36122.543618 secs 2024-04-27T04:37:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:21 smithi090 ceph-mon[29180]: pgmap v19781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36122.543618 secs 2024-04-27T04:37:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36123.431013 secs 2024-04-27T04:37:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36123.431013 secs 2024-04-27T04:37:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36123.431013 secs 2024-04-27T04:37:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:23 smithi119 ceph-mon[32752]: pgmap v19782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:23 smithi167 ceph-mon[32751]: pgmap v19782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:23 smithi090 ceph-mon[29180]: pgmap v19782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:24.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:24 smithi119 ceph-mon[32752]: pgmap v19783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:24.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:24 smithi167 ceph-mon[32751]: pgmap v19783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:24.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:24 smithi090 ceph-mon[29180]: pgmap v19783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:25 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36127.543687 secs 2024-04-27T04:37:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:25 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36127.543687 secs 2024-04-27T04:37:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:25 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36127.543687 secs 2024-04-27T04:37:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:26 smithi119 ceph-mon[32752]: pgmap v19784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:26 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36128.431167 secs 2024-04-27T04:37:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:26 smithi167 ceph-mon[32751]: pgmap v19784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:26 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36128.431167 secs 2024-04-27T04:37:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:26 smithi090 ceph-mon[29180]: pgmap v19784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:26 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36128.431167 secs 2024-04-27T04:37:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:29 smithi119 ceph-mon[32752]: pgmap v19785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:29 smithi167 ceph-mon[32751]: pgmap v19785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:29.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:29 smithi090 ceph-mon[29180]: pgmap v19785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:31 smithi119 ceph-mon[32752]: pgmap v19786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36132.543772 secs 2024-04-27T04:37:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:31 smithi167 ceph-mon[32751]: pgmap v19786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36132.543772 secs 2024-04-27T04:37:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:31 smithi090 ceph-mon[29180]: pgmap v19786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36132.543772 secs 2024-04-27T04:37:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36133.431311 secs 2024-04-27T04:37:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36133.431311 secs 2024-04-27T04:37:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36133.431311 secs 2024-04-27T04:37:33.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:33 smithi119 ceph-mon[32752]: pgmap v19787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:33.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:33 smithi167 ceph-mon[32751]: pgmap v19787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:33.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:33 smithi090 ceph-mon[29180]: pgmap v19787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:35 smithi090 ceph-mon[29180]: pgmap v19788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:35.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:35 smithi119 ceph-mon[32752]: pgmap v19788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:35.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:35 smithi167 ceph-mon[32751]: pgmap v19788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36137.543831 secs 2024-04-27T04:37:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36137.543831 secs 2024-04-27T04:37:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36137.543831 secs 2024-04-27T04:37:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:37 smithi119 ceph-mon[32752]: pgmap v19789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36138.431412 secs 2024-04-27T04:37:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:37 smithi167 ceph-mon[32751]: pgmap v19789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36138.431412 secs 2024-04-27T04:37:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:37 smithi090 ceph-mon[29180]: pgmap v19789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36138.431412 secs 2024-04-27T04:37:39.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:39 smithi119 ceph-mon[32752]: pgmap v19790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:39 smithi167 ceph-mon[32751]: pgmap v19790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:39.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:39 smithi090 ceph-mon[29180]: pgmap v19790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:41 smithi119 ceph-mon[32752]: pgmap v19791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:41.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:41 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 36142.543917 secs 2024-04-27T04:37:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:41 smithi119 ceph-mon[32752]: slow request 11522.712528 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:37:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:41 smithi167 ceph-mon[32751]: pgmap v19791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:41 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 36142.543917 secs 2024-04-27T04:37:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:41 smithi167 ceph-mon[32751]: slow request 11522.712528 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:37:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:41 smithi090 ceph-mon[29180]: pgmap v19791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:41 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 36142.543917 secs 2024-04-27T04:37:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:41 smithi090 ceph-mon[29180]: slow request 11522.712528 seconds old, received at 2024-04-27T01:25:37.584793+0000: client_request(client.17574:1 getattr pAsLsXsFs #0x1 2024-04-27T01:25:37.578078+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:37:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36143.431486 secs 2024-04-27T04:37:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36143.431486 secs 2024-04-27T04:37:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36143.431486 secs 2024-04-27T04:37:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:43 smithi167 ceph-mon[32751]: pgmap v19792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:43.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:43 smithi090 ceph-mon[29180]: pgmap v19792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:43 smithi119 ceph-mon[32752]: pgmap v19792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:45 smithi167 ceph-mon[32751]: pgmap v19793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:45.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:45 smithi090 ceph-mon[29180]: pgmap v19793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:45.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:45 smithi119 ceph-mon[32752]: pgmap v19793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:46.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36147.544003 secs 2024-04-27T04:37:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36147.544003 secs 2024-04-27T04:37:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36147.544003 secs 2024-04-27T04:37:47.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:47 smithi090 ceph-mon[29180]: pgmap v19794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36148.431617 secs 2024-04-27T04:37:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:47 smithi119 ceph-mon[32752]: pgmap v19794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36148.431617 secs 2024-04-27T04:37:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:47 smithi167 ceph-mon[32751]: pgmap v19794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36148.431617 secs 2024-04-27T04:37:49.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:49 smithi090 ceph-mon[29180]: pgmap v19795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:49.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:49 smithi119 ceph-mon[32752]: pgmap v19795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:49 smithi167 ceph-mon[32751]: pgmap v19795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:51.357 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:51 smithi119 ceph-mon[32752]: pgmap v19796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:51.358 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36152.544065 secs 2024-04-27T04:37:51.372 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:51 smithi167 ceph-mon[32751]: pgmap v19796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:51.372 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36152.544065 secs 2024-04-27T04:37:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:51 smithi090 ceph-mon[29180]: pgmap v19796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:51.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36152.544065 secs 2024-04-27T04:37:52.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36153.431729 secs 2024-04-27T04:37:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36153.431729 secs 2024-04-27T04:37:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36153.431729 secs 2024-04-27T04:37:53.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:53 smithi090 ceph-mon[29180]: pgmap v19797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:53.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:53 smithi119 ceph-mon[32752]: pgmap v19797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:53.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:53 smithi167 ceph-mon[32751]: pgmap v19797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:55.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:55 smithi090 ceph-mon[29180]: pgmap v19798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:55 smithi119 ceph-mon[32752]: pgmap v19798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:55.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:55 smithi167 ceph-mon[32751]: pgmap v19798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:37:56.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36157.544145 secs 2024-04-27T04:37:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36157.544145 secs 2024-04-27T04:37:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36157.544145 secs 2024-04-27T04:37:57.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:57 smithi090 ceph-mon[29180]: pgmap v19799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36158.431856 secs 2024-04-27T04:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:57 smithi119 ceph-mon[32752]: pgmap v19799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36158.431856 secs 2024-04-27T04:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:57 smithi167 ceph-mon[32751]: pgmap v19799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36158.431856 secs 2024-04-27T04:37:59.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:37:59 smithi090 ceph-mon[29180]: pgmap v19800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:59.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:37:59 smithi119 ceph-mon[32752]: pgmap v19800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:37:59.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:37:59 smithi167 ceph-mon[32751]: pgmap v19800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:01.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:01 smithi090 ceph-mon[29180]: pgmap v19801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:01 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 36162.544229 secs 2024-04-27T04:38:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:01 smithi090 ceph-mon[29180]: slow request 2884.021391 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:38:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:01 smithi119 ceph-mon[32752]: pgmap v19801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:01 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 36162.544229 secs 2024-04-27T04:38:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:01 smithi119 ceph-mon[32752]: slow request 2884.021391 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:38:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:01 smithi167 ceph-mon[32751]: pgmap v19801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:01 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 36162.544229 secs 2024-04-27T04:38:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:01 smithi167 ceph-mon[32751]: slow request 2884.021391 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:38:02.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36163.431962 secs 2024-04-27T04:38:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:38:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36163.431962 secs 2024-04-27T04:38:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:38:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36163.431962 secs 2024-04-27T04:38:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:38:03.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:03 smithi090 ceph-mon[29180]: pgmap v19802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:38:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:38:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:38:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:03 smithi119 ceph-mon[32752]: pgmap v19802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:38:03.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:38:03.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:38:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:03 smithi167 ceph-mon[32751]: pgmap v19802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:38:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:38:03.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:38:04.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:04 smithi090 ceph-mon[29180]: pgmap v19803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T04:38:04.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:04 smithi090 ceph-mon[29180]: pgmap v19804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T04:38:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:04 smithi119 ceph-mon[32752]: pgmap v19803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T04:38:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:04 smithi119 ceph-mon[32752]: pgmap v19804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T04:38:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:04 smithi167 ceph-mon[32751]: pgmap v19803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T04:38:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:04 smithi167 ceph-mon[32751]: pgmap v19804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T04:38:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:06 smithi090 ceph-mon[29180]: pgmap v19805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36167.544288 secs 2024-04-27T04:38:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:06 smithi119 ceph-mon[32752]: pgmap v19805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36167.544288 secs 2024-04-27T04:38:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:06 smithi167 ceph-mon[32751]: pgmap v19805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36167.544288 secs 2024-04-27T04:38:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36168.432090 secs 2024-04-27T04:38:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36168.432090 secs 2024-04-27T04:38:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36168.432090 secs 2024-04-27T04:38:08.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:08 smithi119 ceph-mon[32752]: pgmap v19806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:08.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:08 smithi167 ceph-mon[32751]: pgmap v19806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:08 smithi090 ceph-mon[29180]: pgmap v19806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:10 smithi119 ceph-mon[32752]: pgmap v19807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:10.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:10 smithi167 ceph-mon[32751]: pgmap v19807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:10 smithi090 ceph-mon[29180]: pgmap v19807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36172.544371 secs 2024-04-27T04:38:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36172.544371 secs 2024-04-27T04:38:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36172.544371 secs 2024-04-27T04:38:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:12 smithi119 ceph-mon[32752]: pgmap v19808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36173.432238 secs 2024-04-27T04:38:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:12 smithi167 ceph-mon[32751]: pgmap v19808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36173.432238 secs 2024-04-27T04:38:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:12 smithi090 ceph-mon[29180]: pgmap v19808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:38:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36173.432238 secs 2024-04-27T04:38:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:14 smithi119 ceph-mon[32752]: pgmap v19809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:38:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:14 smithi167 ceph-mon[32751]: pgmap v19809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:38:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:14 smithi090 ceph-mon[29180]: pgmap v19809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:38:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:16 smithi119 ceph-mon[32752]: pgmap v19810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36177.544441 secs 2024-04-27T04:38:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:16 smithi167 ceph-mon[32751]: pgmap v19810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36177.544441 secs 2024-04-27T04:38:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:16 smithi090 ceph-mon[29180]: pgmap v19810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36177.544441 secs 2024-04-27T04:38:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36178.432367 secs 2024-04-27T04:38:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36178.432367 secs 2024-04-27T04:38:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36178.432367 secs 2024-04-27T04:38:18.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:18 smithi119 ceph-mon[32752]: pgmap v19811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:18 smithi167 ceph-mon[32751]: pgmap v19811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:18.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:18 smithi090 ceph-mon[29180]: pgmap v19811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:20 smithi119 ceph-mon[32752]: pgmap v19812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:20 smithi167 ceph-mon[32751]: pgmap v19812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:20.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:20 smithi090 ceph-mon[29180]: pgmap v19812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:21.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36182.544529 secs 2024-04-27T04:38:21.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36182.544529 secs 2024-04-27T04:38:21.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36182.544529 secs 2024-04-27T04:38:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:22 smithi119 ceph-mon[32752]: pgmap v19813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36183.432469 secs 2024-04-27T04:38:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:22 smithi167 ceph-mon[32751]: pgmap v19813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36183.432469 secs 2024-04-27T04:38:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:22 smithi090 ceph-mon[29180]: pgmap v19813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36183.432469 secs 2024-04-27T04:38:24.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:24 smithi119 ceph-mon[32752]: pgmap v19814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:24.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:24 smithi167 ceph-mon[32751]: pgmap v19814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:24 smithi090 ceph-mon[29180]: pgmap v19814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:26 smithi119 ceph-mon[32752]: pgmap v19815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36187.544614 secs 2024-04-27T04:38:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:26 smithi167 ceph-mon[32751]: pgmap v19815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36187.544614 secs 2024-04-27T04:38:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:26 smithi090 ceph-mon[29180]: pgmap v19815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:26.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36187.544614 secs 2024-04-27T04:38:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36188.432538 secs 2024-04-27T04:38:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36188.432538 secs 2024-04-27T04:38:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36188.432538 secs 2024-04-27T04:38:28.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:28 smithi119 ceph-mon[32752]: pgmap v19816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:28.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:28 smithi167 ceph-mon[32751]: pgmap v19816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:28 smithi090 ceph-mon[29180]: pgmap v19816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:30 smithi119 ceph-mon[32752]: pgmap v19817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:30 smithi167 ceph-mon[32751]: pgmap v19817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:30 smithi090 ceph-mon[29180]: pgmap v19817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36192.544694 secs 2024-04-27T04:38:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36192.544694 secs 2024-04-27T04:38:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36192.544694 secs 2024-04-27T04:38:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:32 smithi119 ceph-mon[32752]: pgmap v19818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36193.432605 secs 2024-04-27T04:38:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:32 smithi167 ceph-mon[32751]: pgmap v19818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36193.432605 secs 2024-04-27T04:38:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:32 smithi090 ceph-mon[29180]: pgmap v19818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36193.432605 secs 2024-04-27T04:38:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:34 smithi119 ceph-mon[32752]: pgmap v19819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:34.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:34 smithi167 ceph-mon[32751]: pgmap v19819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:34 smithi090 ceph-mon[29180]: pgmap v19819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:36 smithi119 ceph-mon[32752]: pgmap v19820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36197.544774 secs 2024-04-27T04:38:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:36 smithi167 ceph-mon[32751]: pgmap v19820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36197.544774 secs 2024-04-27T04:38:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:36 smithi090 ceph-mon[29180]: pgmap v19820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36197.544774 secs 2024-04-27T04:38:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36198.432688 secs 2024-04-27T04:38:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36198.432688 secs 2024-04-27T04:38:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36198.432688 secs 2024-04-27T04:38:38.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:38 smithi119 ceph-mon[32752]: pgmap v19821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:38.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:38 smithi167 ceph-mon[32751]: pgmap v19821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:38.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:38 smithi090 ceph-mon[29180]: pgmap v19821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:40 smithi119 ceph-mon[32752]: pgmap v19822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:40 smithi167 ceph-mon[32751]: pgmap v19822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:40 smithi090 ceph-mon[29180]: pgmap v19822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36202.544866 secs 2024-04-27T04:38:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36202.544866 secs 2024-04-27T04:38:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36202.544866 secs 2024-04-27T04:38:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:42 smithi119 ceph-mon[32752]: pgmap v19823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:42 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36203.432812 secs 2024-04-27T04:38:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:42 smithi167 ceph-mon[32751]: pgmap v19823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:42 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36203.432812 secs 2024-04-27T04:38:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:42 smithi090 ceph-mon[29180]: pgmap v19823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:42 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36203.432812 secs 2024-04-27T04:38:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:44 smithi119 ceph-mon[32752]: pgmap v19824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:44.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:44 smithi167 ceph-mon[32751]: pgmap v19824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:44.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:44 smithi090 ceph-mon[29180]: pgmap v19824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:46 smithi119 ceph-mon[32752]: pgmap v19825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36207.544934 secs 2024-04-27T04:38:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:46 smithi167 ceph-mon[32751]: pgmap v19825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36207.544934 secs 2024-04-27T04:38:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:46 smithi090 ceph-mon[29180]: pgmap v19825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36207.544934 secs 2024-04-27T04:38:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:47 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36208.432919 secs 2024-04-27T04:38:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:47 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36208.432919 secs 2024-04-27T04:38:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:47 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36208.432919 secs 2024-04-27T04:38:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:48 smithi119 ceph-mon[32752]: pgmap v19826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:48 smithi167 ceph-mon[32751]: pgmap v19826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:48 smithi090 ceph-mon[29180]: pgmap v19826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:50 smithi119 ceph-mon[32752]: pgmap v19827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:50 smithi167 ceph-mon[32751]: pgmap v19827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:50 smithi090 ceph-mon[29180]: pgmap v19827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36212.545016 secs 2024-04-27T04:38:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36212.545016 secs 2024-04-27T04:38:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36212.545016 secs 2024-04-27T04:38:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:52 smithi119 ceph-mon[32752]: pgmap v19828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:52 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36213.433042 secs 2024-04-27T04:38:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:52 smithi167 ceph-mon[32751]: pgmap v19828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:52 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36213.433042 secs 2024-04-27T04:38:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:52 smithi090 ceph-mon[29180]: pgmap v19828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:52 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36213.433042 secs 2024-04-27T04:38:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:54 smithi119 ceph-mon[32752]: pgmap v19829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:54 smithi167 ceph-mon[32751]: pgmap v19829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:54 smithi090 ceph-mon[29180]: pgmap v19829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:56 smithi119 ceph-mon[32752]: pgmap v19830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36217.545120 secs 2024-04-27T04:38:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:56 smithi167 ceph-mon[32751]: pgmap v19830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36217.545120 secs 2024-04-27T04:38:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:56 smithi090 ceph-mon[29180]: pgmap v19830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:38:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36217.545120 secs 2024-04-27T04:38:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:57 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36218.433183 secs 2024-04-27T04:38:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:57 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36218.433183 secs 2024-04-27T04:38:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:57 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36218.433183 secs 2024-04-27T04:38:58.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:38:58 smithi119 ceph-mon[32752]: pgmap v19831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:38:58 smithi167 ceph-mon[32751]: pgmap v19831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:38:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:38:58 smithi090 ceph-mon[29180]: pgmap v19831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:00 smithi119 ceph-mon[32752]: pgmap v19832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:00 smithi167 ceph-mon[32751]: pgmap v19832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:00 smithi090 ceph-mon[29180]: pgmap v19832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36222.545208 secs 2024-04-27T04:39:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36222.545208 secs 2024-04-27T04:39:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36222.545208 secs 2024-04-27T04:39:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:02 smithi119 ceph-mon[32752]: pgmap v19833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:02 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36223.433338 secs 2024-04-27T04:39:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:02 smithi167 ceph-mon[32751]: pgmap v19833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:02 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36223.433338 secs 2024-04-27T04:39:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:02 smithi090 ceph-mon[29180]: pgmap v19833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:02 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36223.433338 secs 2024-04-27T04:39:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:03 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:39:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:03 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:39:03.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:03 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:39:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:04 smithi119 ceph-mon[32752]: pgmap v19834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:39:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:39:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:39:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:04 smithi167 ceph-mon[32751]: pgmap v19834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:39:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:39:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:39:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:04 smithi090 ceph-mon[29180]: pgmap v19834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:39:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:39:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:39:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:05 smithi119 ceph-mon[32752]: pgmap v19835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:39:05.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:05 smithi119 ceph-mon[32752]: pgmap v19836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:05 smithi167 ceph-mon[32751]: pgmap v19835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:39:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:05 smithi167 ceph-mon[32751]: pgmap v19836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:05.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:05 smithi090 ceph-mon[29180]: pgmap v19835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:39:05.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:05 smithi090 ceph-mon[29180]: pgmap v19836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36227.545290 secs 2024-04-27T04:39:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36227.545290 secs 2024-04-27T04:39:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36227.545290 secs 2024-04-27T04:39:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:07 smithi167 ceph-mon[32751]: pgmap v19837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:07 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36228.433470 secs 2024-04-27T04:39:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:07 smithi090 ceph-mon[29180]: pgmap v19837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:07 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36228.433470 secs 2024-04-27T04:39:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:07 smithi119 ceph-mon[32752]: pgmap v19837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:07 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36228.433470 secs 2024-04-27T04:39:09.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:09 smithi167 ceph-mon[32751]: pgmap v19838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:39:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:09 smithi090 ceph-mon[29180]: pgmap v19838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:39:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:09 smithi119 ceph-mon[32752]: pgmap v19838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:39:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:11 smithi167 ceph-mon[32751]: pgmap v19839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36232.545372 secs 2024-04-27T04:39:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:11 smithi090 ceph-mon[29180]: pgmap v19839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36232.545372 secs 2024-04-27T04:39:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:11 smithi119 ceph-mon[32752]: pgmap v19839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36232.545372 secs 2024-04-27T04:39:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:12 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36233.433600 secs 2024-04-27T04:39:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:12 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36233.433600 secs 2024-04-27T04:39:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:12 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36233.433600 secs 2024-04-27T04:39:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:13 smithi090 ceph-mon[29180]: pgmap v19840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:13 smithi119 ceph-mon[32752]: pgmap v19840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:13 smithi167 ceph-mon[32751]: pgmap v19840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:39:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:15 smithi090 ceph-mon[29180]: pgmap v19841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:39:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:15 smithi119 ceph-mon[32752]: pgmap v19841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:39:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:15 smithi167 ceph-mon[32751]: pgmap v19841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:39:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36237.545442 secs 2024-04-27T04:39:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36237.545442 secs 2024-04-27T04:39:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36237.545442 secs 2024-04-27T04:39:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:17 smithi090 ceph-mon[29180]: pgmap v19842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:17 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36238.433726 secs 2024-04-27T04:39:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:17 smithi119 ceph-mon[32752]: pgmap v19842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:17 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36238.433726 secs 2024-04-27T04:39:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:17 smithi167 ceph-mon[32751]: pgmap v19842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:17 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36238.433726 secs 2024-04-27T04:39:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:19 smithi090 ceph-mon[29180]: pgmap v19843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:19 smithi119 ceph-mon[32752]: pgmap v19843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:19 smithi167 ceph-mon[32751]: pgmap v19843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:21.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:21 smithi119 ceph-mon[32752]: pgmap v19844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:21.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36242.545526 secs 2024-04-27T04:39:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:21 smithi167 ceph-mon[32751]: pgmap v19844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36242.545526 secs 2024-04-27T04:39:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:21 smithi090 ceph-mon[29180]: pgmap v19844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36242.545526 secs 2024-04-27T04:39:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:22 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36243.433792 secs 2024-04-27T04:39:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:22 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36243.433792 secs 2024-04-27T04:39:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:22 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36243.433792 secs 2024-04-27T04:39:23.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:23 smithi090 ceph-mon[29180]: pgmap v19845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:23 smithi119 ceph-mon[32752]: pgmap v19845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:23 smithi167 ceph-mon[32751]: pgmap v19845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:25 smithi090 ceph-mon[29180]: pgmap v19846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:25 smithi119 ceph-mon[32752]: pgmap v19846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:25 smithi167 ceph-mon[32751]: pgmap v19846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36247.545605 secs 2024-04-27T04:39:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36247.545605 secs 2024-04-27T04:39:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36247.545605 secs 2024-04-27T04:39:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:27 smithi090 ceph-mon[29180]: pgmap v19847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:27.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:27 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36248.433915 secs 2024-04-27T04:39:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:27 smithi119 ceph-mon[32752]: pgmap v19847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:27 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36248.433915 secs 2024-04-27T04:39:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:27 smithi167 ceph-mon[32751]: pgmap v19847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:27 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36248.433915 secs 2024-04-27T04:39:29.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:29 smithi090 ceph-mon[29180]: pgmap v19848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:29 smithi119 ceph-mon[32752]: pgmap v19848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:29 smithi167 ceph-mon[32751]: pgmap v19848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:31 smithi090 ceph-mon[29180]: pgmap v19849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36252.545690 secs 2024-04-27T04:39:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:31 smithi119 ceph-mon[32752]: pgmap v19849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36252.545690 secs 2024-04-27T04:39:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:31 smithi167 ceph-mon[32751]: pgmap v19849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36252.545690 secs 2024-04-27T04:39:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:32 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36253.434039 secs 2024-04-27T04:39:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:32 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36253.434039 secs 2024-04-27T04:39:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:32 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36253.434039 secs 2024-04-27T04:39:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:33 smithi090 ceph-mon[29180]: pgmap v19850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:33 smithi119 ceph-mon[32752]: pgmap v19850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:33 smithi167 ceph-mon[32751]: pgmap v19850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:35 smithi119 ceph-mon[32752]: pgmap v19851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:35 smithi167 ceph-mon[32751]: pgmap v19851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:35 smithi090 ceph-mon[29180]: pgmap v19851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36257.545773 secs 2024-04-27T04:39:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36257.545773 secs 2024-04-27T04:39:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36257.545773 secs 2024-04-27T04:39:37.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:37 smithi090 ceph-mon[29180]: pgmap v19852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:37.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:37 smithi090 ceph-mon[29180]: 14 slow requests, 0 included below; oldest blocked for > 36258.434163 secs 2024-04-27T04:39:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:37 smithi119 ceph-mon[32752]: pgmap v19852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:37 smithi119 ceph-mon[32752]: 14 slow requests, 0 included below; oldest blocked for > 36258.434163 secs 2024-04-27T04:39:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:37 smithi167 ceph-mon[32751]: pgmap v19852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:37 smithi167 ceph-mon[32751]: 14 slow requests, 0 included below; oldest blocked for > 36258.434163 secs 2024-04-27T04:39:39.751 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:39 smithi090 ceph-mon[29180]: pgmap v19853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:39 smithi119 ceph-mon[32752]: pgmap v19853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:39 smithi167 ceph-mon[32751]: pgmap v19853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:41 smithi119 ceph-mon[32752]: pgmap v19854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36262.545817 secs 2024-04-27T04:39:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:41 smithi167 ceph-mon[32751]: pgmap v19854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36262.545817 secs 2024-04-27T04:39:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:41 smithi090 ceph-mon[29180]: pgmap v19854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36262.545817 secs 2024-04-27T04:39:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:42 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 36263.434290 secs 2024-04-27T04:39:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:42 smithi119 ceph-mon[32752]: slow request 180.392939 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:39:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:42 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 36263.434290 secs 2024-04-27T04:39:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:42 smithi167 ceph-mon[32751]: slow request 180.392939 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:39:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:42 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 36263.434290 secs 2024-04-27T04:39:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:42 smithi090 ceph-mon[29180]: slow request 180.392939 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:39:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:43 smithi119 ceph-mon[32752]: pgmap v19855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:43 smithi167 ceph-mon[32751]: pgmap v19855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:43 smithi090 ceph-mon[29180]: pgmap v19855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:45 smithi119 ceph-mon[32752]: pgmap v19856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:39:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:39:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:45 smithi167 ceph-mon[32751]: pgmap v19856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:39:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:39:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:45 smithi090 ceph-mon[29180]: pgmap v19856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:39:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:39:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36267.545892 secs 2024-04-27T04:39:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:46 smithi119 ceph-mon[32752]: pgmap v19857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36267.545892 secs 2024-04-27T04:39:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:46 smithi167 ceph-mon[32751]: pgmap v19857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36267.545892 secs 2024-04-27T04:39:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:46 smithi090 ceph-mon[29180]: pgmap v19857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:47 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36268.434414 secs 2024-04-27T04:39:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:47 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36268.434414 secs 2024-04-27T04:39:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:47 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36268.434414 secs 2024-04-27T04:39:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:48 smithi119 ceph-mon[32752]: pgmap v19858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:48 smithi167 ceph-mon[32751]: pgmap v19858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:48 smithi090 ceph-mon[29180]: pgmap v19858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:50 smithi090 ceph-mon[29180]: pgmap v19859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:50 smithi119 ceph-mon[32752]: pgmap v19859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:50 smithi167 ceph-mon[32751]: pgmap v19859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36272.545974 secs 2024-04-27T04:39:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36272.545974 secs 2024-04-27T04:39:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36272.545974 secs 2024-04-27T04:39:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:52 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36273.434531 secs 2024-04-27T04:39:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:52 smithi119 ceph-mon[32752]: pgmap v19860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:52 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36273.434531 secs 2024-04-27T04:39:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:52 smithi167 ceph-mon[32751]: pgmap v19860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:52 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36273.434531 secs 2024-04-27T04:39:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:52 smithi090 ceph-mon[29180]: pgmap v19860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:54 smithi119 ceph-mon[32752]: pgmap v19861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:54 smithi167 ceph-mon[32751]: pgmap v19861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:54 smithi090 ceph-mon[29180]: pgmap v19861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:39:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36277.546056 secs 2024-04-27T04:39:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:56 smithi119 ceph-mon[32752]: pgmap v19862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36277.546056 secs 2024-04-27T04:39:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:56 smithi167 ceph-mon[32751]: pgmap v19862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36277.546056 secs 2024-04-27T04:39:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:56 smithi090 ceph-mon[29180]: pgmap v19862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:57 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36278.434648 secs 2024-04-27T04:39:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:57 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36278.434648 secs 2024-04-27T04:39:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:57 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36278.434648 secs 2024-04-27T04:39:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:39:58 smithi119 ceph-mon[32752]: pgmap v19863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:39:58 smithi167 ceph-mon[32751]: pgmap v19863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:39:59.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:39:58 smithi090 ceph-mon[29180]: pgmap v19863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:00 smithi119 ceph-mon[32752]: pgmap v19864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:40:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:00 smithi167 ceph-mon[32751]: pgmap v19864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:40:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:00 smithi090 ceph-mon[29180]: pgmap v19864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:40:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36282.546118 secs 2024-04-27T04:40:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36282.546118 secs 2024-04-27T04:40:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36282.546118 secs 2024-04-27T04:40:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:02 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36283.434772 secs 2024-04-27T04:40:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:02 smithi119 ceph-mon[32752]: pgmap v19865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:02 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36283.434772 secs 2024-04-27T04:40:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:02 smithi167 ceph-mon[32751]: pgmap v19865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:02 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36283.434772 secs 2024-04-27T04:40:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:02 smithi090 ceph-mon[29180]: pgmap v19865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:04 smithi119 ceph-mon[32752]: pgmap v19866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:40:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:40:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:40:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:04 smithi119 ceph-mon[32752]: pgmap v19867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:40:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:04 smithi119 ceph-mon[32752]: pgmap v19868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:05.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:40:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:04 smithi167 ceph-mon[32751]: pgmap v19866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:40:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:40:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:40:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:04 smithi167 ceph-mon[32751]: pgmap v19867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:40:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:04 smithi167 ceph-mon[32751]: pgmap v19868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:40:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:04 smithi090 ceph-mon[29180]: pgmap v19866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:40:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:40:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:40:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:04 smithi090 ceph-mon[29180]: pgmap v19867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:40:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:04 smithi090 ceph-mon[29180]: pgmap v19868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:05.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:40:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:05 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36287.546194 secs 2024-04-27T04:40:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:05 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36287.546194 secs 2024-04-27T04:40:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:05 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36287.546194 secs 2024-04-27T04:40:07.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:06 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36288.434927 secs 2024-04-27T04:40:07.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:06 smithi090 ceph-mon[29180]: pgmap v19869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:06 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36288.434927 secs 2024-04-27T04:40:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:06 smithi119 ceph-mon[32752]: pgmap v19869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:06 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36288.434927 secs 2024-04-27T04:40:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:06 smithi167 ceph-mon[32751]: pgmap v19869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:09 smithi119 ceph-mon[32752]: pgmap v19870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:40:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:09 smithi167 ceph-mon[32751]: pgmap v19870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:40:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:09 smithi090 ceph-mon[29180]: pgmap v19870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 1 op/s 2024-04-27T04:40:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36292.546278 secs 2024-04-27T04:40:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:11 smithi119 ceph-mon[32752]: pgmap v19871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36292.546278 secs 2024-04-27T04:40:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:11 smithi167 ceph-mon[32751]: pgmap v19871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36292.546278 secs 2024-04-27T04:40:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:11 smithi090 ceph-mon[29180]: pgmap v19871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:40:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:12 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36293.435033 secs 2024-04-27T04:40:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:12 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36293.435033 secs 2024-04-27T04:40:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:12 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36293.435033 secs 2024-04-27T04:40:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:13 smithi119 ceph-mon[32752]: pgmap v19872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:40:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:13 smithi167 ceph-mon[32751]: pgmap v19872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:40:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:13 smithi090 ceph-mon[29180]: pgmap v19872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:40:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:15 smithi119 ceph-mon[32752]: pgmap v19873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:40:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:15 smithi167 ceph-mon[32751]: pgmap v19873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:40:15.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:15 smithi090 ceph-mon[29180]: pgmap v19873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:40:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36297.546361 secs 2024-04-27T04:40:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36297.546361 secs 2024-04-27T04:40:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36297.546361 secs 2024-04-27T04:40:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:17 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36298.435159 secs 2024-04-27T04:40:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:17 smithi119 ceph-mon[32752]: pgmap v19874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:17 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36298.435159 secs 2024-04-27T04:40:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:17 smithi167 ceph-mon[32751]: pgmap v19874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:17 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36298.435159 secs 2024-04-27T04:40:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:17 smithi090 ceph-mon[29180]: pgmap v19874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:18 smithi119 ceph-mon[32752]: pgmap v19875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:18.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:18 smithi167 ceph-mon[32751]: pgmap v19875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:18 smithi090 ceph-mon[29180]: pgmap v19875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:21.786 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36302.546441 secs 2024-04-27T04:40:21.787 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:21 smithi119 ceph-mon[32752]: pgmap v19876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:40:21.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36302.546441 secs 2024-04-27T04:40:21.799 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:21 smithi167 ceph-mon[32751]: pgmap v19876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:40:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36302.546441 secs 2024-04-27T04:40:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:21 smithi090 ceph-mon[29180]: pgmap v19876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:40:22.738 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:22 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36303.435281 secs 2024-04-27T04:40:22.750 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:22 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36303.435281 secs 2024-04-27T04:40:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:22 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36303.435281 secs 2024-04-27T04:40:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:23 smithi119 ceph-mon[32752]: pgmap v19877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:23 smithi167 ceph-mon[32751]: pgmap v19877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:23 smithi090 ceph-mon[29180]: pgmap v19877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:25 smithi119 ceph-mon[32752]: pgmap v19878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:25 smithi167 ceph-mon[32751]: pgmap v19878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:25.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:25 smithi090 ceph-mon[29180]: pgmap v19878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36307.546519 secs 2024-04-27T04:40:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36307.546519 secs 2024-04-27T04:40:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36307.546519 secs 2024-04-27T04:40:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:27 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36308.435373 secs 2024-04-27T04:40:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:27 smithi119 ceph-mon[32752]: pgmap v19879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:27 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36308.435373 secs 2024-04-27T04:40:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:27 smithi167 ceph-mon[32751]: pgmap v19879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:27 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36308.435373 secs 2024-04-27T04:40:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:27 smithi090 ceph-mon[29180]: pgmap v19879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:28.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:28 smithi119 ceph-mon[32752]: pgmap v19880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:28 smithi167 ceph-mon[32751]: pgmap v19880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:28 smithi090 ceph-mon[29180]: pgmap v19880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36312.546588 secs 2024-04-27T04:40:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:31 smithi119 ceph-mon[32752]: pgmap v19881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:40:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36312.546588 secs 2024-04-27T04:40:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:31 smithi167 ceph-mon[32751]: pgmap v19881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:40:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36312.546588 secs 2024-04-27T04:40:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:31 smithi090 ceph-mon[29180]: pgmap v19881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:40:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:32 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36313.435491 secs 2024-04-27T04:40:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:32 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36313.435491 secs 2024-04-27T04:40:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:32 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36313.435491 secs 2024-04-27T04:40:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:33 smithi119 ceph-mon[32752]: pgmap v19882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:33 smithi167 ceph-mon[32751]: pgmap v19882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:33 smithi090 ceph-mon[29180]: pgmap v19882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:35 smithi119 ceph-mon[32752]: pgmap v19883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:35 smithi167 ceph-mon[32751]: pgmap v19883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:35 smithi090 ceph-mon[29180]: pgmap v19883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36317.546672 secs 2024-04-27T04:40:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36317.546672 secs 2024-04-27T04:40:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36317.546672 secs 2024-04-27T04:40:37.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:37 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36318.435614 secs 2024-04-27T04:40:37.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:37 smithi090 ceph-mon[29180]: pgmap v19884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:37 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36318.435614 secs 2024-04-27T04:40:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:37 smithi119 ceph-mon[32752]: pgmap v19884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:37 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36318.435614 secs 2024-04-27T04:40:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:37 smithi167 ceph-mon[32751]: pgmap v19884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:38 smithi119 ceph-mon[32752]: pgmap v19885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:38 smithi167 ceph-mon[32751]: pgmap v19885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:38 smithi090 ceph-mon[29180]: pgmap v19885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:41.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:41 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36322.546754 secs 2024-04-27T04:40:41.703 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:41 smithi090 ceph-mon[29180]: pgmap v19886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:41 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36322.546754 secs 2024-04-27T04:40:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:41 smithi119 ceph-mon[32752]: pgmap v19886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:41 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36322.546754 secs 2024-04-27T04:40:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:41 smithi167 ceph-mon[32751]: pgmap v19886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:42 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36323.435746 secs 2024-04-27T04:40:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:42 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36323.435746 secs 2024-04-27T04:40:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:42 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36323.435746 secs 2024-04-27T04:40:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:43 smithi119 ceph-mon[32752]: pgmap v19887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:43 smithi167 ceph-mon[32751]: pgmap v19887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:43 smithi090 ceph-mon[29180]: pgmap v19887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:45 smithi119 ceph-mon[32752]: pgmap v19888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:45 smithi167 ceph-mon[32751]: pgmap v19888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:45 smithi090 ceph-mon[29180]: pgmap v19888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:46 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36327.546844 secs 2024-04-27T04:40:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:46 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36327.546844 secs 2024-04-27T04:40:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:46 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36327.546844 secs 2024-04-27T04:40:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:47 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36328.435870 secs 2024-04-27T04:40:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:47 smithi119 ceph-mon[32752]: pgmap v19889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:47 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36328.435870 secs 2024-04-27T04:40:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:47 smithi167 ceph-mon[32751]: pgmap v19889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:47 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36328.435870 secs 2024-04-27T04:40:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:47 smithi090 ceph-mon[29180]: pgmap v19889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:48 smithi119 ceph-mon[32752]: pgmap v19890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:48 smithi167 ceph-mon[32751]: pgmap v19890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:48.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:48 smithi090 ceph-mon[29180]: pgmap v19890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:51 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36332.546933 secs 2024-04-27T04:40:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:51 smithi119 ceph-mon[32752]: pgmap v19891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:51 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36332.546933 secs 2024-04-27T04:40:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:51 smithi167 ceph-mon[32751]: pgmap v19891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:51 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36332.546933 secs 2024-04-27T04:40:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:51 smithi090 ceph-mon[29180]: pgmap v19891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:52 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36333.435978 secs 2024-04-27T04:40:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:52 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36333.435978 secs 2024-04-27T04:40:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:52 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36333.435978 secs 2024-04-27T04:40:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:53 smithi119 ceph-mon[32752]: pgmap v19892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:53 smithi167 ceph-mon[32751]: pgmap v19892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:53 smithi090 ceph-mon[29180]: pgmap v19892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:55 smithi119 ceph-mon[32752]: pgmap v19893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:55 smithi167 ceph-mon[32751]: pgmap v19893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:55 smithi090 ceph-mon[29180]: pgmap v19893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:56 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36337.546995 secs 2024-04-27T04:40:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:56 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36337.546995 secs 2024-04-27T04:40:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:56 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36337.546995 secs 2024-04-27T04:40:57.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:57 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36338.436105 secs 2024-04-27T04:40:57.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:57 smithi090 ceph-mon[29180]: pgmap v19894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:57 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36338.436105 secs 2024-04-27T04:40:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:57 smithi119 ceph-mon[32752]: pgmap v19894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:57 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36338.436105 secs 2024-04-27T04:40:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:57 smithi167 ceph-mon[32751]: pgmap v19894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:40:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:40:58 smithi119 ceph-mon[32752]: pgmap v19895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:40:58 smithi167 ceph-mon[32751]: pgmap v19895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:40:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:40:58 smithi090 ceph-mon[29180]: pgmap v19895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:01 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36342.547049 secs 2024-04-27T04:41:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:01 smithi119 ceph-mon[32752]: pgmap v19896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:01 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36342.547049 secs 2024-04-27T04:41:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:01 smithi167 ceph-mon[32751]: pgmap v19896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:01 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36342.547049 secs 2024-04-27T04:41:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:01 smithi090 ceph-mon[29180]: pgmap v19896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:02 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36343.436259 secs 2024-04-27T04:41:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:02 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36343.436259 secs 2024-04-27T04:41:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:02 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36343.436259 secs 2024-04-27T04:41:03.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:03 smithi119 ceph-mon[32752]: pgmap v19897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:03 smithi167 ceph-mon[32751]: pgmap v19897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:03 smithi090 ceph-mon[29180]: pgmap v19897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:04 smithi119 ceph-mon[32752]: pgmap v19898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:04 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:41:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:04 smithi167 ceph-mon[32751]: pgmap v19898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:04 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:41:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:04 smithi090 ceph-mon[29180]: pgmap v19898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:04 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:41:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:41:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:41:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:05 smithi119 ceph-mon[32752]: pgmap v19899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:41:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:05 smithi119 ceph-mon[32752]: pgmap v19900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:41:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:41:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:41:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:05 smithi167 ceph-mon[32751]: pgmap v19899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:41:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:05 smithi167 ceph-mon[32751]: pgmap v19900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:41:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:41:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:41:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:05 smithi090 ceph-mon[29180]: pgmap v19899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:41:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:05 smithi090 ceph-mon[29180]: pgmap v19900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:41:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:06 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36347.547131 secs 2024-04-27T04:41:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:06 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36347.547131 secs 2024-04-27T04:41:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:06 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36347.547131 secs 2024-04-27T04:41:07.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:07 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36348.436414 secs 2024-04-27T04:41:07.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:07 smithi090 ceph-mon[29180]: pgmap v19901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:41:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:07 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36348.436414 secs 2024-04-27T04:41:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:07 smithi119 ceph-mon[32752]: pgmap v19901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:41:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:07 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36348.436414 secs 2024-04-27T04:41:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:07 smithi167 ceph-mon[32751]: pgmap v19901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:41:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:10 smithi119 ceph-mon[32752]: pgmap v19902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:10.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:10 smithi167 ceph-mon[32751]: pgmap v19902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:10.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:10 smithi090 ceph-mon[29180]: pgmap v19902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:11 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36352.547194 secs 2024-04-27T04:41:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:11 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36352.547194 secs 2024-04-27T04:41:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:11 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36352.547194 secs 2024-04-27T04:41:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:12 smithi119 ceph-mon[32752]: pgmap v19903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:12 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36353.436509 secs 2024-04-27T04:41:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:12 smithi167 ceph-mon[32751]: pgmap v19903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:12 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36353.436509 secs 2024-04-27T04:41:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:12 smithi090 ceph-mon[29180]: pgmap v19903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:41:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:12 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36353.436509 secs 2024-04-27T04:41:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:14 smithi119 ceph-mon[32752]: pgmap v19904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:41:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:14 smithi167 ceph-mon[32751]: pgmap v19904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:41:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:14 smithi090 ceph-mon[29180]: pgmap v19904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:41:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:16 smithi119 ceph-mon[32752]: pgmap v19905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:41:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:16 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36357.547264 secs 2024-04-27T04:41:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:16 smithi167 ceph-mon[32751]: pgmap v19905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:41:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:16 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36357.547264 secs 2024-04-27T04:41:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:16 smithi090 ceph-mon[29180]: pgmap v19905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:41:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:16 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36357.547264 secs 2024-04-27T04:41:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:17 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36358.436578 secs 2024-04-27T04:41:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:17 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36358.436578 secs 2024-04-27T04:41:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:17 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36358.436578 secs 2024-04-27T04:41:18.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:18 smithi119 ceph-mon[32752]: pgmap v19906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:18 smithi167 ceph-mon[32751]: pgmap v19906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:18.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:18 smithi090 ceph-mon[29180]: pgmap v19906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:20 smithi119 ceph-mon[32752]: pgmap v19907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:20 smithi167 ceph-mon[32751]: pgmap v19907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:20.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:20 smithi090 ceph-mon[29180]: pgmap v19907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:21.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:21 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36362.547352 secs 2024-04-27T04:41:21.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:21 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36362.547352 secs 2024-04-27T04:41:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:21 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36362.547352 secs 2024-04-27T04:41:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:22 smithi119 ceph-mon[32752]: pgmap v19908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:22 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36363.436706 secs 2024-04-27T04:41:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:22 smithi167 ceph-mon[32751]: pgmap v19908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:22 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36363.436706 secs 2024-04-27T04:41:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:22 smithi090 ceph-mon[29180]: pgmap v19908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:22 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36363.436706 secs 2024-04-27T04:41:24.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:24 smithi119 ceph-mon[32752]: pgmap v19909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:24.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:24 smithi167 ceph-mon[32751]: pgmap v19909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:24 smithi090 ceph-mon[29180]: pgmap v19909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:26 smithi119 ceph-mon[32752]: pgmap v19910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:26 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36367.547428 secs 2024-04-27T04:41:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:26 smithi167 ceph-mon[32751]: pgmap v19910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:26 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36367.547428 secs 2024-04-27T04:41:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:26 smithi090 ceph-mon[29180]: pgmap v19910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:26 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36367.547428 secs 2024-04-27T04:41:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:27 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36368.436780 secs 2024-04-27T04:41:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:27 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36368.436780 secs 2024-04-27T04:41:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:27 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36368.436780 secs 2024-04-27T04:41:28.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:28 smithi119 ceph-mon[32752]: pgmap v19911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:28.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:28 smithi167 ceph-mon[32751]: pgmap v19911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:28 smithi090 ceph-mon[29180]: pgmap v19911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:30 smithi119 ceph-mon[32752]: pgmap v19912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:30 smithi167 ceph-mon[32751]: pgmap v19912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:30 smithi090 ceph-mon[29180]: pgmap v19912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:31 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36372.547513 secs 2024-04-27T04:41:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:31 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36372.547513 secs 2024-04-27T04:41:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:31 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36372.547513 secs 2024-04-27T04:41:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:32 smithi119 ceph-mon[32752]: pgmap v19913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:32 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36373.436914 secs 2024-04-27T04:41:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:32 smithi167 ceph-mon[32751]: pgmap v19913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:32 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36373.436914 secs 2024-04-27T04:41:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:32 smithi090 ceph-mon[29180]: pgmap v19913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:32 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36373.436914 secs 2024-04-27T04:41:34.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:34 smithi119 ceph-mon[32752]: pgmap v19914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:34.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:34 smithi167 ceph-mon[32751]: pgmap v19914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:34 smithi090 ceph-mon[29180]: pgmap v19914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:36 smithi119 ceph-mon[32752]: pgmap v19915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:36 smithi119 ceph-mon[32752]: 47 slow requests, 0 included below; oldest blocked for > 36377.547567 secs 2024-04-27T04:41:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:36 smithi167 ceph-mon[32751]: pgmap v19915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:36 smithi167 ceph-mon[32751]: 47 slow requests, 0 included below; oldest blocked for > 36377.547567 secs 2024-04-27T04:41:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:36 smithi090 ceph-mon[29180]: pgmap v19915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:36 smithi090 ceph-mon[29180]: 47 slow requests, 0 included below; oldest blocked for > 36377.547567 secs 2024-04-27T04:41:37.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:37 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36378.436992 secs 2024-04-27T04:41:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:37 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36378.436992 secs 2024-04-27T04:41:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:37 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36378.436992 secs 2024-04-27T04:41:38.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:38 smithi119 ceph-mon[32752]: pgmap v19916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:38.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:38 smithi167 ceph-mon[32751]: pgmap v19916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:38 smithi090 ceph-mon[29180]: pgmap v19916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:40 smithi119 ceph-mon[32752]: pgmap v19917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:40 smithi167 ceph-mon[32751]: pgmap v19917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:40.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:40 smithi090 ceph-mon[29180]: pgmap v19917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:41:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:41:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:41:40.840+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:41:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:41:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:41:40.840+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:41:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:41:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:41:40.840+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:41:41.128 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:41:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:41:40.840+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:41:41.129 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:41:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:41:40.840+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:41:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:41 smithi119 ceph-mon[32752]: 47 slow requests, 1 included below; oldest blocked for > 36382.547648 secs 2024-04-27T04:41:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:41 smithi119 ceph-mon[32752]: slow request 5761.057171 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:41:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:41 smithi167 ceph-mon[32751]: 47 slow requests, 1 included below; oldest blocked for > 36382.547648 secs 2024-04-27T04:41:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:41 smithi167 ceph-mon[32751]: slow request 5761.057171 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:41:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:41 smithi090 ceph-mon[29180]: 47 slow requests, 1 included below; oldest blocked for > 36382.547648 secs 2024-04-27T04:41:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:41 smithi090 ceph-mon[29180]: slow request 5761.057171 seconds old, received at 2024-04-27T03:05:39.243881+0000: client_request(client.27002:1 getattr pAsLsXsFs #0x1 2024-04-27T03:05:39.236239+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:41:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:42 smithi119 ceph-mon[32752]: pgmap v19918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:42 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36383.437074 secs 2024-04-27T04:41:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:42 smithi167 ceph-mon[32751]: pgmap v19918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:42 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36383.437074 secs 2024-04-27T04:41:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:42 smithi090 ceph-mon[29180]: pgmap v19918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:42 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36383.437074 secs 2024-04-27T04:41:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:44 smithi119 ceph-mon[32752]: pgmap v19919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:44.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:44 smithi167 ceph-mon[32751]: pgmap v19919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:44 smithi090 ceph-mon[29180]: pgmap v19919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:46 smithi119 ceph-mon[32752]: pgmap v19920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36387.547727 secs 2024-04-27T04:41:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:46 smithi167 ceph-mon[32751]: pgmap v19920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36387.547727 secs 2024-04-27T04:41:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:46 smithi090 ceph-mon[29180]: pgmap v19920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36387.547727 secs 2024-04-27T04:41:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:47 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36388.437209 secs 2024-04-27T04:41:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:41:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:41:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:47 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36388.437209 secs 2024-04-27T04:41:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:41:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:41:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:47 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36388.437209 secs 2024-04-27T04:41:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:41:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:41:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:48 smithi119 ceph-mon[32752]: pgmap v19921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:48.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:48 smithi167 ceph-mon[32751]: pgmap v19921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:48 smithi090 ceph-mon[29180]: pgmap v19921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:50 smithi119 ceph-mon[32752]: pgmap v19922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:50 smithi167 ceph-mon[32751]: pgmap v19922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:50 smithi090 ceph-mon[29180]: pgmap v19922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36392.547831 secs 2024-04-27T04:41:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36392.547831 secs 2024-04-27T04:41:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36392.547831 secs 2024-04-27T04:41:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:52 smithi119 ceph-mon[32752]: pgmap v19923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:52 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36393.437346 secs 2024-04-27T04:41:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:52 smithi167 ceph-mon[32751]: pgmap v19923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:52 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36393.437346 secs 2024-04-27T04:41:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:52 smithi090 ceph-mon[29180]: pgmap v19923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:52 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36393.437346 secs 2024-04-27T04:41:54.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:54 smithi119 ceph-mon[32752]: pgmap v19924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:54 smithi167 ceph-mon[32751]: pgmap v19924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:54.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:54 smithi090 ceph-mon[29180]: pgmap v19924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:56 smithi119 ceph-mon[32752]: pgmap v19925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36397.547900 secs 2024-04-27T04:41:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:56 smithi167 ceph-mon[32751]: pgmap v19925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36397.547900 secs 2024-04-27T04:41:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:56 smithi090 ceph-mon[29180]: pgmap v19925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 341 B/s wr, 2 op/s 2024-04-27T04:41:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36397.547900 secs 2024-04-27T04:41:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:57 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36398.437464 secs 2024-04-27T04:41:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:57 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36398.437464 secs 2024-04-27T04:41:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:57 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36398.437464 secs 2024-04-27T04:41:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:41:58 smithi119 ceph-mon[32752]: pgmap v19926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:41:58 smithi167 ceph-mon[32751]: pgmap v19926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:41:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:41:58 smithi090 ceph-mon[29180]: pgmap v19926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:00 smithi119 ceph-mon[32752]: pgmap v19927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:00 smithi167 ceph-mon[32751]: pgmap v19927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:00 smithi090 ceph-mon[29180]: pgmap v19927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36402.547989 secs 2024-04-27T04:42:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36402.547989 secs 2024-04-27T04:42:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36402.547989 secs 2024-04-27T04:42:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:02 smithi119 ceph-mon[32752]: pgmap v19928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:02 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36403.437582 secs 2024-04-27T04:42:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:02 smithi167 ceph-mon[32751]: pgmap v19928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:02 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36403.437582 secs 2024-04-27T04:42:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:02 smithi090 ceph-mon[29180]: pgmap v19928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:02 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36403.437582 secs 2024-04-27T04:42:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:04 smithi119 ceph-mon[32752]: pgmap v19929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:04 smithi167 ceph-mon[32751]: pgmap v19929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:04 smithi090 ceph-mon[29180]: pgmap v19929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:05 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:42:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:05 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:42:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:05 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:42:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:06 smithi167 ceph-mon[32751]: pgmap v19930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36407.548052 secs 2024-04-27T04:42:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:42:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:42:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:42:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:06 smithi090 ceph-mon[29180]: pgmap v19930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36407.548052 secs 2024-04-27T04:42:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:42:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:42:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:42:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:06 smithi119 ceph-mon[32752]: pgmap v19930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36407.548052 secs 2024-04-27T04:42:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:42:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:42:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:42:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:07 smithi167 ceph-mon[32751]: pgmap v19931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:42:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:07 smithi167 ceph-mon[32751]: pgmap v19932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:07 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36408.437680 secs 2024-04-27T04:42:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:07 smithi090 ceph-mon[29180]: pgmap v19931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:42:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:07 smithi090 ceph-mon[29180]: pgmap v19932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:07 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36408.437680 secs 2024-04-27T04:42:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:07 smithi119 ceph-mon[32752]: pgmap v19931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:42:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:07 smithi119 ceph-mon[32752]: pgmap v19932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:07 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36408.437680 secs 2024-04-27T04:42:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:09 smithi090 ceph-mon[29180]: pgmap v19933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:42:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:09 smithi119 ceph-mon[32752]: pgmap v19933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:42:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:09 smithi167 ceph-mon[32751]: pgmap v19933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:42:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:11 smithi090 ceph-mon[29180]: pgmap v19934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36412.548122 secs 2024-04-27T04:42:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:11 smithi119 ceph-mon[32752]: pgmap v19934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36412.548122 secs 2024-04-27T04:42:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:11 smithi167 ceph-mon[32751]: pgmap v19934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36412.548122 secs 2024-04-27T04:42:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:12 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36413.437817 secs 2024-04-27T04:42:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:12 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36413.437817 secs 2024-04-27T04:42:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:12 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36413.437817 secs 2024-04-27T04:42:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:13 smithi090 ceph-mon[29180]: pgmap v19935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:13 smithi119 ceph-mon[32752]: pgmap v19935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:13 smithi167 ceph-mon[32751]: pgmap v19935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:15.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:15 smithi090 ceph-mon[29180]: pgmap v19936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:15 smithi119 ceph-mon[32752]: pgmap v19936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:15 smithi167 ceph-mon[32751]: pgmap v19936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:42:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36417.548201 secs 2024-04-27T04:42:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36417.548201 secs 2024-04-27T04:42:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36417.548201 secs 2024-04-27T04:42:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:17 smithi090 ceph-mon[29180]: pgmap v19937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:42:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:17 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36418.437924 secs 2024-04-27T04:42:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:17 smithi119 ceph-mon[32752]: pgmap v19937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:42:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:17 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36418.437924 secs 2024-04-27T04:42:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:17 smithi167 ceph-mon[32751]: pgmap v19937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:42:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:17 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36418.437924 secs 2024-04-27T04:42:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:19 smithi090 ceph-mon[29180]: pgmap v19938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:19 smithi119 ceph-mon[32752]: pgmap v19938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:19.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:19 smithi167 ceph-mon[32751]: pgmap v19938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:21.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:21 smithi119 ceph-mon[32752]: pgmap v19939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:21.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36422.548281 secs 2024-04-27T04:42:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:21 smithi167 ceph-mon[32751]: pgmap v19939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36422.548281 secs 2024-04-27T04:42:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:21 smithi090 ceph-mon[29180]: pgmap v19939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36422.548281 secs 2024-04-27T04:42:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:22 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36423.438043 secs 2024-04-27T04:42:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:22 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36423.438043 secs 2024-04-27T04:42:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:22 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36423.438043 secs 2024-04-27T04:42:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:23 smithi090 ceph-mon[29180]: pgmap v19940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:23 smithi119 ceph-mon[32752]: pgmap v19940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:23 smithi167 ceph-mon[32751]: pgmap v19940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:25.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:25 smithi090 ceph-mon[29180]: pgmap v19941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:25 smithi119 ceph-mon[32752]: pgmap v19941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:25 smithi167 ceph-mon[32751]: pgmap v19941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36427.548364 secs 2024-04-27T04:42:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36427.548364 secs 2024-04-27T04:42:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36427.548364 secs 2024-04-27T04:42:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:27 smithi090 ceph-mon[29180]: pgmap v19942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:27 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36428.438165 secs 2024-04-27T04:42:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:27 smithi119 ceph-mon[32752]: pgmap v19942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:27 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36428.438165 secs 2024-04-27T04:42:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:27 smithi167 ceph-mon[32751]: pgmap v19942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:27 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36428.438165 secs 2024-04-27T04:42:29.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:29 smithi090 ceph-mon[29180]: pgmap v19943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:29 smithi119 ceph-mon[32752]: pgmap v19943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:29 smithi167 ceph-mon[32751]: pgmap v19943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:31 smithi090 ceph-mon[29180]: pgmap v19944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36432.548447 secs 2024-04-27T04:42:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:31 smithi119 ceph-mon[32752]: pgmap v19944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36432.548447 secs 2024-04-27T04:42:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:31 smithi167 ceph-mon[32751]: pgmap v19944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36432.548447 secs 2024-04-27T04:42:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:32 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36433.438286 secs 2024-04-27T04:42:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:32 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36433.438286 secs 2024-04-27T04:42:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:32 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36433.438286 secs 2024-04-27T04:42:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:33 smithi119 ceph-mon[32752]: pgmap v19945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:33 smithi167 ceph-mon[32751]: pgmap v19945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:33 smithi090 ceph-mon[29180]: pgmap v19945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:35 smithi119 ceph-mon[32752]: pgmap v19946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:35 smithi167 ceph-mon[32751]: pgmap v19946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:35 smithi090 ceph-mon[29180]: pgmap v19946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36437.548528 secs 2024-04-27T04:42:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36437.548528 secs 2024-04-27T04:42:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36437.548528 secs 2024-04-27T04:42:37.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:37 smithi090 ceph-mon[29180]: pgmap v19947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:37.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:37 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36438.438370 secs 2024-04-27T04:42:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:37 smithi119 ceph-mon[32752]: pgmap v19947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:37 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36438.438370 secs 2024-04-27T04:42:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:37 smithi167 ceph-mon[32751]: pgmap v19947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:37 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36438.438370 secs 2024-04-27T04:42:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:39 smithi119 ceph-mon[32752]: pgmap v19948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:39 smithi167 ceph-mon[32751]: pgmap v19948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:39 smithi090 ceph-mon[29180]: pgmap v19948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:41 smithi119 ceph-mon[32752]: pgmap v19949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36442.548614 secs 2024-04-27T04:42:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:41 smithi167 ceph-mon[32751]: pgmap v19949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36442.548614 secs 2024-04-27T04:42:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:41 smithi090 ceph-mon[29180]: pgmap v19949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36442.548614 secs 2024-04-27T04:42:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:42 smithi119 ceph-mon[32752]: 32 slow requests, 1 included below; oldest blocked for > 36443.438448 secs 2024-04-27T04:42:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:42 smithi119 ceph-mon[32752]: slow request 360.397097 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:42:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:42 smithi167 ceph-mon[32751]: 32 slow requests, 1 included below; oldest blocked for > 36443.438448 secs 2024-04-27T04:42:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:42 smithi167 ceph-mon[32751]: slow request 360.397097 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:42:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:42 smithi090 ceph-mon[29180]: 32 slow requests, 1 included below; oldest blocked for > 36443.438448 secs 2024-04-27T04:42:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:42 smithi090 ceph-mon[29180]: slow request 360.397097 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:42:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:43 smithi119 ceph-mon[32752]: pgmap v19950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:43 smithi167 ceph-mon[32751]: pgmap v19950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:43 smithi090 ceph-mon[29180]: pgmap v19950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:45 smithi119 ceph-mon[32752]: pgmap v19951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:45 smithi167 ceph-mon[32751]: pgmap v19951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:45 smithi090 ceph-mon[29180]: pgmap v19951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36447.548694 secs 2024-04-27T04:42:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36447.548694 secs 2024-04-27T04:42:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36447.548694 secs 2024-04-27T04:42:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:47 smithi119 ceph-mon[32752]: pgmap v19952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:47 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36448.438522 secs 2024-04-27T04:42:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:47 smithi167 ceph-mon[32751]: pgmap v19952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:47 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36448.438522 secs 2024-04-27T04:42:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:47 smithi090 ceph-mon[29180]: pgmap v19952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:47 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36448.438522 secs 2024-04-27T04:42:48.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:48 smithi119 ceph-mon[32752]: pgmap v19953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:48 smithi167 ceph-mon[32751]: pgmap v19953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:48.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:48 smithi090 ceph-mon[29180]: pgmap v19953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:51.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:50 smithi119 ceph-mon[32752]: pgmap v19954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:50 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36452.548789 secs 2024-04-27T04:42:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:50 smithi167 ceph-mon[32751]: pgmap v19954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:51.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:50 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36452.548789 secs 2024-04-27T04:42:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:50 smithi090 ceph-mon[29180]: pgmap v19954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:50 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36452.548789 secs 2024-04-27T04:42:52.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:51 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36453.438591 secs 2024-04-27T04:42:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:51 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36453.438591 secs 2024-04-27T04:42:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:51 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36453.438591 secs 2024-04-27T04:42:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:52 smithi119 ceph-mon[32752]: pgmap v19955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:53.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:52 smithi167 ceph-mon[32751]: pgmap v19955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:53.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:52 smithi090 ceph-mon[29180]: pgmap v19955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:55.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:54 smithi119 ceph-mon[32752]: pgmap v19956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:55.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:54 smithi167 ceph-mon[32751]: pgmap v19956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:55.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:54 smithi090 ceph-mon[29180]: pgmap v19956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:42:56.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:55 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36457.548867 secs 2024-04-27T04:42:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:55 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36457.548867 secs 2024-04-27T04:42:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:55 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36457.548867 secs 2024-04-27T04:42:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:56 smithi119 ceph-mon[32752]: pgmap v19957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:56 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36458.438662 secs 2024-04-27T04:42:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:56 smithi167 ceph-mon[32751]: pgmap v19957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:56 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36458.438662 secs 2024-04-27T04:42:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:56 smithi090 ceph-mon[29180]: pgmap v19957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:56 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36458.438662 secs 2024-04-27T04:42:59.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:42:58 smithi119 ceph-mon[32752]: pgmap v19958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:42:58 smithi167 ceph-mon[32751]: pgmap v19958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:42:59.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:42:58 smithi090 ceph-mon[29180]: pgmap v19958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:00 smithi119 ceph-mon[32752]: pgmap v19959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:00 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36462.548927 secs 2024-04-27T04:43:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:00 smithi167 ceph-mon[32751]: pgmap v19959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:00 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36462.548927 secs 2024-04-27T04:43:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:00 smithi090 ceph-mon[29180]: pgmap v19959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:00 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36462.548927 secs 2024-04-27T04:43:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:01 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36463.438784 secs 2024-04-27T04:43:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:01 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36463.438784 secs 2024-04-27T04:43:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:01 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36463.438784 secs 2024-04-27T04:43:03.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:02 smithi119 ceph-mon[32752]: pgmap v19960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:03.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:02 smithi167 ceph-mon[32751]: pgmap v19960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:02 smithi090 ceph-mon[29180]: pgmap v19960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:05.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:04 smithi119 ceph-mon[32752]: pgmap v19961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:05.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:04 smithi167 ceph-mon[32751]: pgmap v19961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:05.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:04 smithi090 ceph-mon[29180]: pgmap v19961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36467.549004 secs 2024-04-27T04:43:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36467.549004 secs 2024-04-27T04:43:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36467.549004 secs 2024-04-27T04:43:07.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:07 smithi090 ceph-mon[29180]: pgmap v19962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:07.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:43:07.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:07 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36468.438872 secs 2024-04-27T04:43:07.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:43:07.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:43:07.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:43:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:07 smithi119 ceph-mon[32752]: pgmap v19962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:43:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:07 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36468.438872 secs 2024-04-27T04:43:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:43:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:43:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:43:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:07 smithi167 ceph-mon[32751]: pgmap v19962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:43:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:07 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36468.438872 secs 2024-04-27T04:43:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:43:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:43:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:43:08.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:08 smithi119 ceph-mon[32752]: pgmap v19963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:43:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:08 smithi119 ceph-mon[32752]: pgmap v19964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:08.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:08 smithi167 ceph-mon[32751]: pgmap v19963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:43:08.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:08 smithi167 ceph-mon[32751]: pgmap v19964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:08.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:08 smithi090 ceph-mon[29180]: pgmap v19963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:43:08.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:08 smithi090 ceph-mon[29180]: pgmap v19964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:10.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:10 smithi119 ceph-mon[32752]: pgmap v19965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:10.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:10 smithi167 ceph-mon[32751]: pgmap v19965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:10 smithi090 ceph-mon[29180]: pgmap v19965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36472.549068 secs 2024-04-27T04:43:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36472.549068 secs 2024-04-27T04:43:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36472.549068 secs 2024-04-27T04:43:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:12 smithi119 ceph-mon[32752]: pgmap v19966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:12 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36473.438939 secs 2024-04-27T04:43:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:12 smithi167 ceph-mon[32751]: pgmap v19966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:12 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36473.438939 secs 2024-04-27T04:43:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:12 smithi090 ceph-mon[29180]: pgmap v19966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:12 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36473.438939 secs 2024-04-27T04:43:14.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:14 smithi119 ceph-mon[32752]: pgmap v19967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:43:14.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:14 smithi167 ceph-mon[32751]: pgmap v19967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:43:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:14 smithi090 ceph-mon[29180]: pgmap v19967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:43:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:16 smithi119 ceph-mon[32752]: pgmap v19968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36477.549147 secs 2024-04-27T04:43:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:16 smithi167 ceph-mon[32751]: pgmap v19968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36477.549147 secs 2024-04-27T04:43:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:16 smithi090 ceph-mon[29180]: pgmap v19968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:43:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36477.549147 secs 2024-04-27T04:43:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:17 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36478.439060 secs 2024-04-27T04:43:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:17 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36478.439060 secs 2024-04-27T04:43:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:17 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36478.439060 secs 2024-04-27T04:43:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:18 smithi090 ceph-mon[29180]: pgmap v19969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:43:18.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:18 smithi119 ceph-mon[32752]: pgmap v19969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:43:18.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:18 smithi167 ceph-mon[32751]: pgmap v19969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:43:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:20 smithi119 ceph-mon[32752]: pgmap v19970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:20 smithi167 ceph-mon[32751]: pgmap v19970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:20.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:20 smithi090 ceph-mon[29180]: pgmap v19970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36482.549252 secs 2024-04-27T04:43:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36482.549252 secs 2024-04-27T04:43:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36482.549252 secs 2024-04-27T04:43:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:22 smithi119 ceph-mon[32752]: pgmap v19971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:22 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36483.439210 secs 2024-04-27T04:43:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:22 smithi167 ceph-mon[32751]: pgmap v19971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:22 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36483.439210 secs 2024-04-27T04:43:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:22 smithi090 ceph-mon[29180]: pgmap v19971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:22 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36483.439210 secs 2024-04-27T04:43:24.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:24 smithi119 ceph-mon[32752]: pgmap v19972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:24.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:24 smithi167 ceph-mon[32751]: pgmap v19972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:24.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:24 smithi090 ceph-mon[29180]: pgmap v19972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:26 smithi119 ceph-mon[32752]: pgmap v19973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36487.549333 secs 2024-04-27T04:43:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:26 smithi167 ceph-mon[32751]: pgmap v19973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36487.549333 secs 2024-04-27T04:43:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:26 smithi090 ceph-mon[29180]: pgmap v19973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36487.549333 secs 2024-04-27T04:43:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:27 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36488.439313 secs 2024-04-27T04:43:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:27 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36488.439313 secs 2024-04-27T04:43:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:27 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36488.439313 secs 2024-04-27T04:43:28.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:28 smithi167 ceph-mon[32751]: pgmap v19974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:28.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:28 smithi090 ceph-mon[29180]: pgmap v19974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:28.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:28 smithi119 ceph-mon[32752]: pgmap v19974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:30.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:30 smithi167 ceph-mon[32751]: pgmap v19975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:30.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:30 smithi090 ceph-mon[29180]: pgmap v19975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:30.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:30 smithi119 ceph-mon[32752]: pgmap v19975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36492.549415 secs 2024-04-27T04:43:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36492.549415 secs 2024-04-27T04:43:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36492.549415 secs 2024-04-27T04:43:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:32 smithi167 ceph-mon[32751]: pgmap v19976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:32 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36493.439441 secs 2024-04-27T04:43:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:32 smithi090 ceph-mon[29180]: pgmap v19976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:32 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36493.439441 secs 2024-04-27T04:43:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:32 smithi119 ceph-mon[32752]: pgmap v19976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:32 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36493.439441 secs 2024-04-27T04:43:34.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:34 smithi090 ceph-mon[29180]: pgmap v19977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:34.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:34 smithi119 ceph-mon[32752]: pgmap v19977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:34.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:34 smithi167 ceph-mon[32751]: pgmap v19977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:36 smithi090 ceph-mon[29180]: pgmap v19978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36497.549498 secs 2024-04-27T04:43:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:36 smithi119 ceph-mon[32752]: pgmap v19978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36497.549498 secs 2024-04-27T04:43:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:36 smithi167 ceph-mon[32751]: pgmap v19978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36497.549498 secs 2024-04-27T04:43:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:37 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36498.439558 secs 2024-04-27T04:43:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:37 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36498.439558 secs 2024-04-27T04:43:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:37 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36498.439558 secs 2024-04-27T04:43:38.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:38 smithi090 ceph-mon[29180]: pgmap v19979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:38.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:38 smithi119 ceph-mon[32752]: pgmap v19979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:38.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:38 smithi167 ceph-mon[32751]: pgmap v19979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:40.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:40 smithi090 ceph-mon[29180]: pgmap v19980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:40.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:40 smithi119 ceph-mon[32752]: pgmap v19980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:40 smithi167 ceph-mon[32751]: pgmap v19980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:41 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36502.549587 secs 2024-04-27T04:43:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:41 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36502.549587 secs 2024-04-27T04:43:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:41 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36502.549587 secs 2024-04-27T04:43:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:42 smithi090 ceph-mon[29180]: pgmap v19981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:42 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36503.439622 secs 2024-04-27T04:43:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:42 smithi119 ceph-mon[32752]: pgmap v19981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:42 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36503.439622 secs 2024-04-27T04:43:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:42 smithi167 ceph-mon[32751]: pgmap v19981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:42 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36503.439622 secs 2024-04-27T04:43:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:44 smithi090 ceph-mon[29180]: pgmap v19982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:44.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:44 smithi119 ceph-mon[32752]: pgmap v19982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:44.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:44 smithi167 ceph-mon[32751]: pgmap v19982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:46 smithi090 ceph-mon[29180]: pgmap v19983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:46 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36507.549666 secs 2024-04-27T04:43:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:46 smithi119 ceph-mon[32752]: pgmap v19983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:46 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36507.549666 secs 2024-04-27T04:43:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:46 smithi167 ceph-mon[32751]: pgmap v19983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:46 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36507.549666 secs 2024-04-27T04:43:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:47 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36508.439745 secs 2024-04-27T04:43:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:47 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36508.439745 secs 2024-04-27T04:43:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:47 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36508.439745 secs 2024-04-27T04:43:48.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:48 smithi090 ceph-mon[29180]: pgmap v19984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:48 smithi119 ceph-mon[32752]: pgmap v19984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:48 smithi167 ceph-mon[32751]: pgmap v19984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:50 smithi090 ceph-mon[29180]: pgmap v19985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:50.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:50 smithi119 ceph-mon[32752]: pgmap v19985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:50.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:50 smithi167 ceph-mon[32751]: pgmap v19985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:51 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36512.549750 secs 2024-04-27T04:43:51.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:51 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36512.549750 secs 2024-04-27T04:43:51.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:51 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36512.549750 secs 2024-04-27T04:43:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:52 smithi090 ceph-mon[29180]: pgmap v19986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:52 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36513.439871 secs 2024-04-27T04:43:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:52 smithi119 ceph-mon[32752]: pgmap v19986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:52 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36513.439871 secs 2024-04-27T04:43:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:52 smithi167 ceph-mon[32751]: pgmap v19986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:52 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36513.439871 secs 2024-04-27T04:43:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:54 smithi090 ceph-mon[29180]: pgmap v19987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:54.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:54 smithi119 ceph-mon[32752]: pgmap v19987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:54 smithi167 ceph-mon[32751]: pgmap v19987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:56 smithi090 ceph-mon[29180]: pgmap v19988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:56 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36517.549844 secs 2024-04-27T04:43:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:56 smithi119 ceph-mon[32752]: pgmap v19988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:56 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36517.549844 secs 2024-04-27T04:43:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:56 smithi167 ceph-mon[32751]: pgmap v19988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:43:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:56 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36517.549844 secs 2024-04-27T04:43:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:57 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36518.439965 secs 2024-04-27T04:43:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:57 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36518.439965 secs 2024-04-27T04:43:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:57 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36518.439965 secs 2024-04-27T04:43:58.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:43:58 smithi119 ceph-mon[32752]: pgmap v19989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:43:58 smithi167 ceph-mon[32751]: pgmap v19989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:43:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:43:58 smithi090 ceph-mon[29180]: pgmap v19989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:00.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:00 smithi119 ceph-mon[32752]: pgmap v19990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:00 smithi167 ceph-mon[32751]: pgmap v19990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:00 smithi090 ceph-mon[29180]: pgmap v19990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:01 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36522.549900 secs 2024-04-27T04:44:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:01 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36522.549900 secs 2024-04-27T04:44:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:01 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36522.549900 secs 2024-04-27T04:44:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:02 smithi119 ceph-mon[32752]: pgmap v19991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:02 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36523.440032 secs 2024-04-27T04:44:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:02 smithi167 ceph-mon[32751]: pgmap v19991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:02 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36523.440032 secs 2024-04-27T04:44:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:02 smithi090 ceph-mon[29180]: pgmap v19991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:02 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36523.440032 secs 2024-04-27T04:44:04.455 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:04 smithi090 ceph-mon[29180]: pgmap v19992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:04.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:04 smithi119 ceph-mon[32752]: pgmap v19992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:04 smithi167 ceph-mon[32751]: pgmap v19992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:06 smithi119 ceph-mon[32752]: pgmap v19993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:06 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36527.549950 secs 2024-04-27T04:44:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:06 smithi167 ceph-mon[32751]: pgmap v19993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:06 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36527.549950 secs 2024-04-27T04:44:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:06 smithi090 ceph-mon[29180]: pgmap v19993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:06.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:06 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36527.549950 secs 2024-04-27T04:44:07.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:07 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36528.440155 secs 2024-04-27T04:44:07.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:44:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:07 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36528.440155 secs 2024-04-27T04:44:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:44:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:07 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36528.440155 secs 2024-04-27T04:44:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:44:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:08 smithi119 ceph-mon[32752]: pgmap v19994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:44:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:44:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:44:08.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:08 smithi167 ceph-mon[32751]: pgmap v19994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:08.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:44:08.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:44:08.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:44:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:08 smithi090 ceph-mon[29180]: pgmap v19994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:44:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:44:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:44:09.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:09 smithi119 ceph-mon[32752]: pgmap v19995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:44:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:09 smithi119 ceph-mon[32752]: pgmap v19996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:44:09.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:09 smithi167 ceph-mon[32751]: pgmap v19995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:44:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:09 smithi167 ceph-mon[32751]: pgmap v19996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:44:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:09 smithi090 ceph-mon[29180]: pgmap v19995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:44:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:09 smithi090 ceph-mon[29180]: pgmap v19996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:44:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:11 smithi119 ceph-mon[32752]: pgmap v19997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:11 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36532.550033 secs 2024-04-27T04:44:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:11 smithi167 ceph-mon[32751]: pgmap v19997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:11 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36532.550033 secs 2024-04-27T04:44:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:11 smithi090 ceph-mon[29180]: pgmap v19997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:11 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36532.550033 secs 2024-04-27T04:44:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:12 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36533.440303 secs 2024-04-27T04:44:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:12 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36533.440303 secs 2024-04-27T04:44:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:12 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36533.440303 secs 2024-04-27T04:44:13.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:13 smithi119 ceph-mon[32752]: pgmap v19998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:13 smithi167 ceph-mon[32751]: pgmap v19998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:13 smithi090 ceph-mon[29180]: pgmap v19998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:15 smithi119 ceph-mon[32752]: pgmap v19999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:15 smithi167 ceph-mon[32751]: pgmap v19999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:15 smithi090 ceph-mon[29180]: pgmap v19999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:16 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36537.550092 secs 2024-04-27T04:44:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:16 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36537.550092 secs 2024-04-27T04:44:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:16 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36537.550092 secs 2024-04-27T04:44:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:17 smithi119 ceph-mon[32752]: pgmap v20000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:17 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36538.440426 secs 2024-04-27T04:44:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:17 smithi167 ceph-mon[32751]: pgmap v20000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:17 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36538.440426 secs 2024-04-27T04:44:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:17 smithi090 ceph-mon[29180]: pgmap v20000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:44:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:17 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36538.440426 secs 2024-04-27T04:44:19.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:19 smithi119 ceph-mon[32752]: pgmap v20001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:44:19.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:19 smithi167 ceph-mon[32751]: pgmap v20001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:44:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:19 smithi090 ceph-mon[29180]: pgmap v20001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:44:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:21 smithi119 ceph-mon[32752]: pgmap v20002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:21 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36542.550179 secs 2024-04-27T04:44:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:21 smithi167 ceph-mon[32751]: pgmap v20002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:21 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36542.550179 secs 2024-04-27T04:44:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:21 smithi090 ceph-mon[29180]: pgmap v20002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:21 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36542.550179 secs 2024-04-27T04:44:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:22 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36543.440494 secs 2024-04-27T04:44:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:22 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36543.440494 secs 2024-04-27T04:44:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:22 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36543.440494 secs 2024-04-27T04:44:23.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:23 smithi119 ceph-mon[32752]: pgmap v20003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:23 smithi167 ceph-mon[32751]: pgmap v20003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:23 smithi090 ceph-mon[29180]: pgmap v20003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:25.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:25 smithi119 ceph-mon[32752]: pgmap v20004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:25 smithi167 ceph-mon[32751]: pgmap v20004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:25.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:25 smithi090 ceph-mon[29180]: pgmap v20004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:26 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36547.550257 secs 2024-04-27T04:44:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:26 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36547.550257 secs 2024-04-27T04:44:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:26 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36547.550257 secs 2024-04-27T04:44:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:27 smithi119 ceph-mon[32752]: pgmap v20005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:27.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:27 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36548.440616 secs 2024-04-27T04:44:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:27 smithi167 ceph-mon[32751]: pgmap v20005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:27 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36548.440616 secs 2024-04-27T04:44:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:27 smithi090 ceph-mon[29180]: pgmap v20005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:27 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36548.440616 secs 2024-04-27T04:44:29.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:29 smithi119 ceph-mon[32752]: pgmap v20006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:29 smithi167 ceph-mon[32751]: pgmap v20006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:29.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:29 smithi090 ceph-mon[29180]: pgmap v20006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:31 smithi119 ceph-mon[32752]: pgmap v20007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:31 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36552.550342 secs 2024-04-27T04:44:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:31 smithi167 ceph-mon[32751]: pgmap v20007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:31 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36552.550342 secs 2024-04-27T04:44:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:31 smithi090 ceph-mon[29180]: pgmap v20007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:31 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36552.550342 secs 2024-04-27T04:44:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:32 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36553.440743 secs 2024-04-27T04:44:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:32 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36553.440743 secs 2024-04-27T04:44:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:32 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36553.440743 secs 2024-04-27T04:44:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:33 smithi119 ceph-mon[32752]: pgmap v20008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:33.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:33 smithi167 ceph-mon[32751]: pgmap v20008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:33 smithi090 ceph-mon[29180]: pgmap v20008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:35.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:35 smithi119 ceph-mon[32752]: pgmap v20009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:35.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:35 smithi167 ceph-mon[32751]: pgmap v20009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:35 smithi090 ceph-mon[29180]: pgmap v20009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:36 smithi119 ceph-mon[32752]: 21 slow requests, 0 included below; oldest blocked for > 36557.550421 secs 2024-04-27T04:44:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:36 smithi167 ceph-mon[32751]: 21 slow requests, 0 included below; oldest blocked for > 36557.550421 secs 2024-04-27T04:44:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:36 smithi090 ceph-mon[29180]: 21 slow requests, 0 included below; oldest blocked for > 36557.550421 secs 2024-04-27T04:44:37.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:37 smithi090 ceph-mon[29180]: pgmap v20010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:37.531 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:37 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36558.440837 secs 2024-04-27T04:44:37.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:37 smithi119 ceph-mon[32752]: pgmap v20010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:37 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36558.440837 secs 2024-04-27T04:44:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:37 smithi167 ceph-mon[32751]: pgmap v20010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:37.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:37 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36558.440837 secs 2024-04-27T04:44:39.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:39 smithi119 ceph-mon[32752]: pgmap v20011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:39 smithi167 ceph-mon[32751]: pgmap v20011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:39.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:39 smithi090 ceph-mon[29180]: pgmap v20011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:41 smithi119 ceph-mon[32752]: pgmap v20012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:41 smithi119 ceph-mon[32752]: 21 slow requests, 1 included below; oldest blocked for > 36562.550508 secs 2024-04-27T04:44:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:41 smithi119 ceph-mon[32752]: slow request 1442.663153 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:44:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:41 smithi167 ceph-mon[32751]: pgmap v20012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:41 smithi167 ceph-mon[32751]: 21 slow requests, 1 included below; oldest blocked for > 36562.550508 secs 2024-04-27T04:44:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:41 smithi167 ceph-mon[32751]: slow request 1442.663153 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:44:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:41 smithi090 ceph-mon[29180]: pgmap v20012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:41 smithi090 ceph-mon[29180]: 21 slow requests, 1 included below; oldest blocked for > 36562.550508 secs 2024-04-27T04:44:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:41 smithi090 ceph-mon[29180]: slow request 1442.663153 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:44:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:42 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36563.440931 secs 2024-04-27T04:44:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:42 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36563.440931 secs 2024-04-27T04:44:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:42 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36563.440931 secs 2024-04-27T04:44:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:43 smithi119 ceph-mon[32752]: pgmap v20013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:43.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:43 smithi167 ceph-mon[32751]: pgmap v20013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:43.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:43 smithi090 ceph-mon[29180]: pgmap v20013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:45.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:45 smithi119 ceph-mon[32752]: pgmap v20014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:45.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:45 smithi167 ceph-mon[32751]: pgmap v20014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:45 smithi090 ceph-mon[29180]: pgmap v20014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:46 smithi119 ceph-mon[32752]: 48 slow requests, 2 included below; oldest blocked for > 36567.550598 secs 2024-04-27T04:44:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:46 smithi119 ceph-mon[32752]: slow request 184.456432 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:44:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:46 smithi119 ceph-mon[32752]: slow request 23050.884732 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:44:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:46 smithi167 ceph-mon[32751]: 48 slow requests, 2 included below; oldest blocked for > 36567.550598 secs 2024-04-27T04:44:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:46 smithi167 ceph-mon[32751]: slow request 184.456432 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:44:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:46 smithi167 ceph-mon[32751]: slow request 23050.884732 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:44:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:46 smithi090 ceph-mon[29180]: 48 slow requests, 2 included below; oldest blocked for > 36567.550598 secs 2024-04-27T04:44:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:46 smithi090 ceph-mon[29180]: slow request 184.456432 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:44:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:46 smithi090 ceph-mon[29180]: slow request 23050.884732 seconds old, received at 2024-04-26T22:20:34.419270+0000: client_request(client.17361:1 getattr pAsLsXsFs #0x1 2024-04-26T22:20:34.411571+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:44:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:47 smithi119 ceph-mon[32752]: pgmap v20015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:47 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36568.441031 secs 2024-04-27T04:44:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:44:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:44:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:47 smithi167 ceph-mon[32751]: pgmap v20015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:47 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36568.441031 secs 2024-04-27T04:44:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:44:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:44:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:47 smithi090 ceph-mon[29180]: pgmap v20015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:47 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36568.441031 secs 2024-04-27T04:44:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:44:47.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:44:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:49 smithi119 ceph-mon[32752]: pgmap v20016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:49 smithi167 ceph-mon[32751]: pgmap v20016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:49.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:49 smithi090 ceph-mon[29180]: pgmap v20016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:51 smithi119 ceph-mon[32752]: pgmap v20017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:51 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36572.550682 secs 2024-04-27T04:44:51.559 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:51 smithi090 ceph-mon[29180]: pgmap v20017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:51.560 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:51 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36572.550682 secs 2024-04-27T04:44:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:51 smithi167 ceph-mon[32751]: pgmap v20017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:51 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36572.550682 secs 2024-04-27T04:44:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:52 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36573.441156 secs 2024-04-27T04:44:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:52 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36573.441156 secs 2024-04-27T04:44:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:52 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36573.441156 secs 2024-04-27T04:44:53.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:53 smithi119 ceph-mon[32752]: pgmap v20018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:53.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:53 smithi167 ceph-mon[32751]: pgmap v20018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:53.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:53 smithi090 ceph-mon[29180]: pgmap v20018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:55 smithi119 ceph-mon[32752]: pgmap v20019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:55.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:55 smithi167 ceph-mon[32751]: pgmap v20019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:55.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:55 smithi090 ceph-mon[29180]: pgmap v20019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:44:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:56 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36577.550763 secs 2024-04-27T04:44:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:56 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36577.550763 secs 2024-04-27T04:44:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:56 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36577.550763 secs 2024-04-27T04:44:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:57 smithi119 ceph-mon[32752]: pgmap v20020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:57 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36578.441272 secs 2024-04-27T04:44:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:57 smithi167 ceph-mon[32751]: pgmap v20020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:57 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36578.441272 secs 2024-04-27T04:44:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:57 smithi090 ceph-mon[29180]: pgmap v20020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:57 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36578.441272 secs 2024-04-27T04:44:59.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:44:59 smithi119 ceph-mon[32752]: pgmap v20021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:59.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:44:59 smithi167 ceph-mon[32751]: pgmap v20021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:44:59.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:44:59 smithi090 ceph-mon[29180]: pgmap v20021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:01 smithi119 ceph-mon[32752]: pgmap v20022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:01 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36582.550853 secs 2024-04-27T04:45:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:01 smithi167 ceph-mon[32751]: pgmap v20022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:01 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36582.550853 secs 2024-04-27T04:45:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:01 smithi090 ceph-mon[29180]: pgmap v20022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:01 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36582.550853 secs 2024-04-27T04:45:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:02 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36583.441405 secs 2024-04-27T04:45:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:02 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36583.441405 secs 2024-04-27T04:45:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:02 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36583.441405 secs 2024-04-27T04:45:03.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:03 smithi119 ceph-mon[32752]: pgmap v20023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:03.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:03 smithi167 ceph-mon[32751]: pgmap v20023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:03.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:03 smithi090 ceph-mon[29180]: pgmap v20023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:05.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:05 smithi119 ceph-mon[32752]: pgmap v20024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:05 smithi167 ceph-mon[32751]: pgmap v20024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:05.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:05 smithi090 ceph-mon[29180]: pgmap v20024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:06 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36587.550942 secs 2024-04-27T04:45:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:06 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36587.550942 secs 2024-04-27T04:45:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:06 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36587.550942 secs 2024-04-27T04:45:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:07 smithi119 ceph-mon[32752]: pgmap v20025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:07 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36588.441518 secs 2024-04-27T04:45:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:07 smithi167 ceph-mon[32751]: pgmap v20025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:07 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36588.441518 secs 2024-04-27T04:45:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:07 smithi090 ceph-mon[29180]: pgmap v20025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:07 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36588.441518 secs 2024-04-27T04:45:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:45:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:45:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:45:08.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:45:08.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:45:08.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:45:08.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:45:08.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:45:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:45:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:45:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:45:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:45:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:09 smithi119 ceph-mon[32752]: pgmap v20026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:09 smithi119 ceph-mon[32752]: pgmap v20027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 2 op/s 2024-04-27T04:45:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:09 smithi119 ceph-mon[32752]: pgmap v20028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:45:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:09 smithi167 ceph-mon[32751]: pgmap v20026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:09 smithi167 ceph-mon[32751]: pgmap v20027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 2 op/s 2024-04-27T04:45:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:09 smithi167 ceph-mon[32751]: pgmap v20028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:45:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:09 smithi090 ceph-mon[29180]: pgmap v20026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:09 smithi090 ceph-mon[29180]: pgmap v20027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 2 op/s 2024-04-27T04:45:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:09 smithi090 ceph-mon[29180]: pgmap v20028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:45:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:11 smithi167 ceph-mon[32751]: pgmap v20029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:11 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36592.551008 secs 2024-04-27T04:45:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:11 smithi090 ceph-mon[29180]: pgmap v20029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:11.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:11 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36592.551008 secs 2024-04-27T04:45:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:11 smithi119 ceph-mon[32752]: pgmap v20029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:11 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36592.551008 secs 2024-04-27T04:45:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:12 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36593.441641 secs 2024-04-27T04:45:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:12 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36593.441641 secs 2024-04-27T04:45:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:12 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36593.441641 secs 2024-04-27T04:45:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:13 smithi167 ceph-mon[32751]: pgmap v20030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:45:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:13 smithi090 ceph-mon[29180]: pgmap v20030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:45:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:13 smithi119 ceph-mon[32752]: pgmap v20030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:45:15.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:15 smithi167 ceph-mon[32751]: pgmap v20031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:15 smithi090 ceph-mon[29180]: pgmap v20031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:15 smithi119 ceph-mon[32752]: pgmap v20031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:16 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36597.551087 secs 2024-04-27T04:45:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:16 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36597.551087 secs 2024-04-27T04:45:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:16 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36597.551087 secs 2024-04-27T04:45:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:17 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36598.441719 secs 2024-04-27T04:45:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:17 smithi090 ceph-mon[29180]: pgmap v20032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:17 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36598.441719 secs 2024-04-27T04:45:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:17 smithi119 ceph-mon[32752]: pgmap v20032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:17 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36598.441719 secs 2024-04-27T04:45:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:17 smithi167 ceph-mon[32751]: pgmap v20032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:45:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:19 smithi090 ceph-mon[29180]: pgmap v20033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:45:19.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:19 smithi119 ceph-mon[32752]: pgmap v20033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:45:19.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:19 smithi167 ceph-mon[32751]: pgmap v20033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:45:21.607 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:21 smithi119 ceph-mon[32752]: pgmap v20034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:21.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:21 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36602.551150 secs 2024-04-27T04:45:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:21 smithi167 ceph-mon[32751]: pgmap v20034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:21 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36602.551150 secs 2024-04-27T04:45:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:21 smithi090 ceph-mon[29180]: pgmap v20034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:21 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36602.551150 secs 2024-04-27T04:45:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:22 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36603.441830 secs 2024-04-27T04:45:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:22 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36603.441830 secs 2024-04-27T04:45:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:22 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36603.441830 secs 2024-04-27T04:45:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:23 smithi090 ceph-mon[29180]: pgmap v20035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:23 smithi119 ceph-mon[32752]: pgmap v20035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:23 smithi167 ceph-mon[32751]: pgmap v20035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:25.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:25 smithi090 ceph-mon[29180]: pgmap v20036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:25 smithi119 ceph-mon[32752]: pgmap v20036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:25 smithi167 ceph-mon[32751]: pgmap v20036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:26 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36607.551232 secs 2024-04-27T04:45:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:26 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36607.551232 secs 2024-04-27T04:45:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:26 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36607.551232 secs 2024-04-27T04:45:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:27 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36608.441965 secs 2024-04-27T04:45:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:27 smithi090 ceph-mon[29180]: pgmap v20037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:27 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36608.441965 secs 2024-04-27T04:45:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:27 smithi119 ceph-mon[32752]: pgmap v20037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:27 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36608.441965 secs 2024-04-27T04:45:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:27 smithi167 ceph-mon[32751]: pgmap v20037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:29.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:29 smithi090 ceph-mon[29180]: pgmap v20038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:29.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:29 smithi119 ceph-mon[32752]: pgmap v20038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:29 smithi167 ceph-mon[32751]: pgmap v20038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:31 smithi090 ceph-mon[29180]: pgmap v20039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:31 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36612.551317 secs 2024-04-27T04:45:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:31 smithi119 ceph-mon[32752]: pgmap v20039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:31 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36612.551317 secs 2024-04-27T04:45:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:31 smithi167 ceph-mon[32751]: pgmap v20039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:31 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36612.551317 secs 2024-04-27T04:45:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:32 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36613.442100 secs 2024-04-27T04:45:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:32 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36613.442100 secs 2024-04-27T04:45:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:32 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36613.442100 secs 2024-04-27T04:45:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:33 smithi090 ceph-mon[29180]: pgmap v20040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:33 smithi119 ceph-mon[32752]: pgmap v20040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:33 smithi167 ceph-mon[32751]: pgmap v20040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:35 smithi090 ceph-mon[29180]: pgmap v20041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:35 smithi119 ceph-mon[32752]: pgmap v20041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:35 smithi167 ceph-mon[32751]: pgmap v20041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:36 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36617.551376 secs 2024-04-27T04:45:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:36 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36617.551376 secs 2024-04-27T04:45:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:36 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36617.551376 secs 2024-04-27T04:45:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:37 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36618.442217 secs 2024-04-27T04:45:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:37 smithi090 ceph-mon[29180]: pgmap v20042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:37 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36618.442217 secs 2024-04-27T04:45:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:37 smithi119 ceph-mon[32752]: pgmap v20042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:37 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36618.442217 secs 2024-04-27T04:45:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:37 smithi167 ceph-mon[32751]: pgmap v20042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:39 smithi119 ceph-mon[32752]: pgmap v20043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:39 smithi167 ceph-mon[32751]: pgmap v20043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:39 smithi090 ceph-mon[29180]: pgmap v20043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:41 smithi119 ceph-mon[32752]: pgmap v20044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:45:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:41 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36622.551436 secs 2024-04-27T04:45:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:41 smithi167 ceph-mon[32751]: pgmap v20044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:45:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:41 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36622.551436 secs 2024-04-27T04:45:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:41 smithi090 ceph-mon[29180]: pgmap v20044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-04-27T04:45:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:41 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36622.551436 secs 2024-04-27T04:45:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:42 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36623.442338 secs 2024-04-27T04:45:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:42 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36623.442338 secs 2024-04-27T04:45:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:42 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36623.442338 secs 2024-04-27T04:45:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:43 smithi119 ceph-mon[32752]: pgmap v20045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:43 smithi167 ceph-mon[32751]: pgmap v20045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:43 smithi090 ceph-mon[29180]: pgmap v20045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:45:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:45 smithi119 ceph-mon[32752]: pgmap v20046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:45 smithi167 ceph-mon[32751]: pgmap v20046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:45 smithi090 ceph-mon[29180]: pgmap v20046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:46 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36627.551542 secs 2024-04-27T04:45:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:46 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36627.551542 secs 2024-04-27T04:45:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:46 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36627.551542 secs 2024-04-27T04:45:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:47 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36628.442458 secs 2024-04-27T04:45:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:47 smithi119 ceph-mon[32752]: pgmap v20047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:47 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36628.442458 secs 2024-04-27T04:45:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:47 smithi167 ceph-mon[32751]: pgmap v20047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:47 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36628.442458 secs 2024-04-27T04:45:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:47 smithi090 ceph-mon[29180]: pgmap v20047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:49 smithi119 ceph-mon[32752]: pgmap v20048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:49 smithi167 ceph-mon[32751]: pgmap v20048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:49 smithi090 ceph-mon[29180]: pgmap v20048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:51 smithi119 ceph-mon[32752]: pgmap v20049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:51 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36632.551627 secs 2024-04-27T04:45:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:51 smithi167 ceph-mon[32751]: pgmap v20049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:51 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36632.551627 secs 2024-04-27T04:45:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:51 smithi090 ceph-mon[29180]: pgmap v20049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:51 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36632.551627 secs 2024-04-27T04:45:52.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:52 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36633.442580 secs 2024-04-27T04:45:52.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:52 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36633.442580 secs 2024-04-27T04:45:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:52 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36633.442580 secs 2024-04-27T04:45:53.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:53 smithi119 ceph-mon[32752]: pgmap v20050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:53 smithi167 ceph-mon[32751]: pgmap v20050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:53 smithi090 ceph-mon[29180]: pgmap v20050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:55.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:55 smithi119 ceph-mon[32752]: pgmap v20051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:55 smithi167 ceph-mon[32751]: pgmap v20051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:55 smithi090 ceph-mon[29180]: pgmap v20051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:45:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:56 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36637.551708 secs 2024-04-27T04:45:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:56 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36637.551708 secs 2024-04-27T04:45:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:56 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36637.551708 secs 2024-04-27T04:45:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:57 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36638.442657 secs 2024-04-27T04:45:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:57 smithi119 ceph-mon[32752]: pgmap v20052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:57 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36638.442657 secs 2024-04-27T04:45:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:57 smithi167 ceph-mon[32751]: pgmap v20052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:57 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36638.442657 secs 2024-04-27T04:45:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:57 smithi090 ceph-mon[29180]: pgmap v20052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:45:59.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:45:59 smithi119 ceph-mon[32752]: pgmap v20053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:59.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:45:59 smithi167 ceph-mon[32751]: pgmap v20053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:45:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:45:59 smithi090 ceph-mon[29180]: pgmap v20053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:46:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:01 smithi119 ceph-mon[32752]: pgmap v20054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:01 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36642.551817 secs 2024-04-27T04:46:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:01 smithi167 ceph-mon[32751]: pgmap v20054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:01 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36642.551817 secs 2024-04-27T04:46:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:01 smithi090 ceph-mon[29180]: pgmap v20054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:01 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36642.551817 secs 2024-04-27T04:46:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:02 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36643.442769 secs 2024-04-27T04:46:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:02 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36643.442769 secs 2024-04-27T04:46:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:02 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36643.442769 secs 2024-04-27T04:46:03.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:03 smithi119 ceph-mon[32752]: pgmap v20055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:03 smithi167 ceph-mon[32751]: pgmap v20055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:03 smithi090 ceph-mon[29180]: pgmap v20055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:05 smithi119 ceph-mon[32752]: pgmap v20056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:05 smithi167 ceph-mon[32751]: pgmap v20056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:05 smithi090 ceph-mon[29180]: pgmap v20056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:06 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36647.551867 secs 2024-04-27T04:46:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:06 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36647.551867 secs 2024-04-27T04:46:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:06 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36647.551867 secs 2024-04-27T04:46:07.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:07 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36648.442868 secs 2024-04-27T04:46:07.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:07 smithi090 ceph-mon[29180]: pgmap v20057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:07 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36648.442868 secs 2024-04-27T04:46:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:07 smithi119 ceph-mon[32752]: pgmap v20057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:07 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36648.442868 secs 2024-04-27T04:46:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:07 smithi167 ceph-mon[32751]: pgmap v20057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:46:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:46:08.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:46:09.735 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:09 smithi119 ceph-mon[32752]: pgmap v20058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:46:09.735 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.735 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.735 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.735 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:09 smithi090 ceph-mon[29180]: pgmap v20058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:46:09.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.805 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:09 smithi167 ceph-mon[32751]: pgmap v20058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T04:46:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:10 smithi119 ceph-mon[32752]: pgmap v20059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:10 smithi167 ceph-mon[32751]: pgmap v20059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:10 smithi090 ceph-mon[29180]: pgmap v20059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36652.551933 secs 2024-04-27T04:46:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:46:12.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:11 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:46:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36652.551933 secs 2024-04-27T04:46:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:46:12.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:11 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:46:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36652.551933 secs 2024-04-27T04:46:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:12.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:46:12.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:11 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:46:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36653.442961 secs 2024-04-27T04:46:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:46:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:46:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:46:13.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:46:13.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:46:13.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:46:13.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: pgmap v20060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-27T04:46:13.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: pgmap v20061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:13.059 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36653.442961 secs 2024-04-27T04:46:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:46:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:46:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:46:13.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:46:13.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:46:13.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:46:13.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: pgmap v20060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-27T04:46:13.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: pgmap v20061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:13.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36653.442961 secs 2024-04-27T04:46:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:46:13.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:46:13.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:46:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:46:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:46:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:46:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:46:13.131 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: pgmap v20060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 884 B/s rd, 1 op/s 2024-04-27T04:46:13.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: pgmap v20061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:13.132 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:46:15.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:14 smithi119 ceph-mon[32752]: pgmap v20062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:15.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:14 smithi167 ceph-mon[32751]: pgmap v20062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:15.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:14 smithi090 ceph-mon[29180]: pgmap v20062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:15 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36657.551992 secs 2024-04-27T04:46:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:15 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36657.551992 secs 2024-04-27T04:46:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:15 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36657.551992 secs 2024-04-27T04:46:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:16 smithi119 ceph-mon[32752]: pgmap v20063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:16 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36658.443086 secs 2024-04-27T04:46:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:16 smithi167 ceph-mon[32751]: pgmap v20063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:16 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36658.443086 secs 2024-04-27T04:46:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:16 smithi090 ceph-mon[29180]: pgmap v20063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:16 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36658.443086 secs 2024-04-27T04:46:19.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:18 smithi119 ceph-mon[32752]: pgmap v20064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T04:46:19.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:18 smithi167 ceph-mon[32751]: pgmap v20064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T04:46:19.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:18 smithi090 ceph-mon[29180]: pgmap v20064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-04-27T04:46:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:20 smithi119 ceph-mon[32752]: pgmap v20065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:20 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36662.552075 secs 2024-04-27T04:46:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:20 smithi167 ceph-mon[32751]: pgmap v20065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:20 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36662.552075 secs 2024-04-27T04:46:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:20 smithi090 ceph-mon[29180]: pgmap v20065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:46:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:20 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36662.552075 secs 2024-04-27T04:46:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:21 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36663.443229 secs 2024-04-27T04:46:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:21 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36663.443229 secs 2024-04-27T04:46:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:21 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36663.443229 secs 2024-04-27T04:46:23.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:22 smithi119 ceph-mon[32752]: pgmap v20066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:46:23.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:22 smithi167 ceph-mon[32751]: pgmap v20066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:46:23.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:22 smithi090 ceph-mon[29180]: pgmap v20066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:46:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:24 smithi090 ceph-mon[29180]: pgmap v20067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:24 smithi119 ceph-mon[32752]: pgmap v20067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:25.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:24 smithi167 ceph-mon[32751]: pgmap v20067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:25 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36667.552136 secs 2024-04-27T04:46:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:25 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36667.552136 secs 2024-04-27T04:46:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:25 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36667.552136 secs 2024-04-27T04:46:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:26 smithi119 ceph-mon[32752]: pgmap v20068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:26 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36668.443304 secs 2024-04-27T04:46:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:26 smithi167 ceph-mon[32751]: pgmap v20068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:26 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36668.443304 secs 2024-04-27T04:46:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:26 smithi090 ceph-mon[29180]: pgmap v20068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:26 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36668.443304 secs 2024-04-27T04:46:29.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:28 smithi119 ceph-mon[32752]: pgmap v20069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:28 smithi167 ceph-mon[32751]: pgmap v20069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:29.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:28 smithi090 ceph-mon[29180]: pgmap v20069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:30 smithi119 ceph-mon[32752]: pgmap v20070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:31.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:30 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36672.552220 secs 2024-04-27T04:46:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:30 smithi167 ceph-mon[32751]: pgmap v20070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:30 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36672.552220 secs 2024-04-27T04:46:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:30 smithi090 ceph-mon[29180]: pgmap v20070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:30 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36672.552220 secs 2024-04-27T04:46:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:31 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36673.443414 secs 2024-04-27T04:46:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:31 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36673.443414 secs 2024-04-27T04:46:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:31 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36673.443414 secs 2024-04-27T04:46:33.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:32 smithi119 ceph-mon[32752]: pgmap v20071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:33.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:32 smithi167 ceph-mon[32751]: pgmap v20071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:33.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:32 smithi090 ceph-mon[29180]: pgmap v20071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:35.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:34 smithi119 ceph-mon[32752]: pgmap v20072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:35.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:34 smithi167 ceph-mon[32751]: pgmap v20072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:35.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:34 smithi090 ceph-mon[29180]: pgmap v20072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:35 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36677.552283 secs 2024-04-27T04:46:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:35 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36677.552283 secs 2024-04-27T04:46:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:35 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36677.552283 secs 2024-04-27T04:46:37.278 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:36 smithi090 ceph-mon[29180]: pgmap v20073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:37.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:36 smithi090 ceph-mon[29180]: 32 slow requests, 0 included below; oldest blocked for > 36678.443539 secs 2024-04-27T04:46:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:36 smithi119 ceph-mon[32752]: pgmap v20073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:36 smithi119 ceph-mon[32752]: 32 slow requests, 0 included below; oldest blocked for > 36678.443539 secs 2024-04-27T04:46:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:36 smithi167 ceph-mon[32751]: pgmap v20073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:36 smithi167 ceph-mon[32751]: 32 slow requests, 0 included below; oldest blocked for > 36678.443539 secs 2024-04-27T04:46:39.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:38 smithi119 ceph-mon[32752]: pgmap v20074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:39.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:38 smithi167 ceph-mon[32751]: pgmap v20074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:39.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:38 smithi090 ceph-mon[29180]: pgmap v20074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:40 smithi119 ceph-mon[32752]: pgmap v20075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:40 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36682.552372 secs 2024-04-27T04:46:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:40 smithi167 ceph-mon[32751]: pgmap v20075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:40 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36682.552372 secs 2024-04-27T04:46:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:40 smithi090 ceph-mon[29180]: pgmap v20075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:46:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:40 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36682.552372 secs 2024-04-27T04:46:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:46:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:46:40.972+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:46:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:46:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:46:40.972+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:46:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:46:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:46:40.972+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:46:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:46:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:46:40.972+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:46:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:46:40 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:46:40.972+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:46:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:41 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36683.443657 secs 2024-04-27T04:46:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:41 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36683.443657 secs 2024-04-27T04:46:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:41 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36683.443657 secs 2024-04-27T04:46:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:42 smithi119 ceph-mon[32752]: pgmap v20076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:42 smithi167 ceph-mon[32751]: pgmap v20076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:42 smithi090 ceph-mon[29180]: pgmap v20076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:44 smithi119 ceph-mon[32752]: pgmap v20077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:45.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:44 smithi167 ceph-mon[32751]: pgmap v20077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:44 smithi090 ceph-mon[29180]: pgmap v20077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:46:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:46:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:45 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36687.552435 secs 2024-04-27T04:46:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:46:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:46:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:45 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36687.552435 secs 2024-04-27T04:46:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:46:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:46:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:45 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36687.552435 secs 2024-04-27T04:46:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:46 smithi119 ceph-mon[32752]: pgmap v20078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:46 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36688.443779 secs 2024-04-27T04:46:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:46 smithi167 ceph-mon[32751]: pgmap v20078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:46 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36688.443779 secs 2024-04-27T04:46:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:46 smithi090 ceph-mon[29180]: pgmap v20078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:46 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36688.443779 secs 2024-04-27T04:46:49.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:48 smithi119 ceph-mon[32752]: pgmap v20079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:49.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:48 smithi167 ceph-mon[32751]: pgmap v20079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:49.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:48 smithi090 ceph-mon[29180]: pgmap v20079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:50 smithi119 ceph-mon[32752]: pgmap v20080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:50 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36692.552518 secs 2024-04-27T04:46:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:50 smithi167 ceph-mon[32751]: pgmap v20080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:51.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:50 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36692.552518 secs 2024-04-27T04:46:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:50 smithi090 ceph-mon[29180]: pgmap v20080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:50 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36692.552518 secs 2024-04-27T04:46:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:51 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36693.443882 secs 2024-04-27T04:46:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:51 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36693.443882 secs 2024-04-27T04:46:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:51 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36693.443882 secs 2024-04-27T04:46:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:52 smithi119 ceph-mon[32752]: pgmap v20081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:53.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:52 smithi167 ceph-mon[32751]: pgmap v20081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:53.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:52 smithi090 ceph-mon[29180]: pgmap v20081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 341 B/s wr, 13 op/s 2024-04-27T04:46:55.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:54 smithi119 ceph-mon[32752]: pgmap v20082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:55.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:54 smithi167 ceph-mon[32751]: pgmap v20082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:55.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:54 smithi090 ceph-mon[29180]: pgmap v20082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T04:46:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:55 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36697.552606 secs 2024-04-27T04:46:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:55 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36697.552606 secs 2024-04-27T04:46:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:55 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36697.552606 secs 2024-04-27T04:46:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:56 smithi119 ceph-mon[32752]: pgmap v20083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:56 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36698.443985 secs 2024-04-27T04:46:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:56 smithi167 ceph-mon[32751]: pgmap v20083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:56 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36698.443985 secs 2024-04-27T04:46:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:56 smithi090 ceph-mon[29180]: pgmap v20083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:56 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36698.443985 secs 2024-04-27T04:46:59.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:46:59 smithi119 ceph-mon[32752]: pgmap v20084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:59.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:46:59 smithi167 ceph-mon[32751]: pgmap v20084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:46:59.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:46:59 smithi090 ceph-mon[29180]: pgmap v20084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:01 smithi119 ceph-mon[32752]: pgmap v20085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:01 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36702.552663 secs 2024-04-27T04:47:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:01 smithi167 ceph-mon[32751]: pgmap v20085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:01 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36702.552663 secs 2024-04-27T04:47:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:01 smithi090 ceph-mon[29180]: pgmap v20085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:01 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36702.552663 secs 2024-04-27T04:47:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:02 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36703.444128 secs 2024-04-27T04:47:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:02 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36703.444128 secs 2024-04-27T04:47:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:02 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36703.444128 secs 2024-04-27T04:47:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:03 smithi119 ceph-mon[32752]: pgmap v20086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:03.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:03 smithi167 ceph-mon[32751]: pgmap v20086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:03 smithi090 ceph-mon[29180]: pgmap v20086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:05 smithi119 ceph-mon[32752]: pgmap v20087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:05.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:05 smithi167 ceph-mon[32751]: pgmap v20087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:05.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:05 smithi090 ceph-mon[29180]: pgmap v20087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:06 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36707.552751 secs 2024-04-27T04:47:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:06 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36707.552751 secs 2024-04-27T04:47:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:06 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36707.552751 secs 2024-04-27T04:47:07.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:07 smithi090 ceph-mon[29180]: pgmap v20088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:07.280 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:07 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36708.444244 secs 2024-04-27T04:47:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:07 smithi119 ceph-mon[32752]: pgmap v20088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:07.310 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:07 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36708.444244 secs 2024-04-27T04:47:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:07 smithi167 ceph-mon[32751]: pgmap v20088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:07 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36708.444244 secs 2024-04-27T04:47:09.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:09 smithi119 ceph-mon[32752]: pgmap v20089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:09.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:09 smithi167 ceph-mon[32751]: pgmap v20089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:09.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:09 smithi090 ceph-mon[29180]: pgmap v20089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:11 smithi090 ceph-mon[29180]: pgmap v20090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:11 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36712.552802 secs 2024-04-27T04:47:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:11 smithi119 ceph-mon[32752]: pgmap v20090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:11 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36712.552802 secs 2024-04-27T04:47:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:11 smithi167 ceph-mon[32751]: pgmap v20090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:11 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36712.552802 secs 2024-04-27T04:47:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:12 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36713.444360 secs 2024-04-27T04:47:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:47:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:12 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36713.444360 secs 2024-04-27T04:47:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:47:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:12 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36713.444360 secs 2024-04-27T04:47:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:47:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:13 smithi119 ceph-mon[32752]: pgmap v20091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:13.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:13 smithi167 ceph-mon[32751]: pgmap v20091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:13.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:13.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:13 smithi090 ceph-mon[29180]: pgmap v20091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:47:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:47:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:47:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:14 smithi119 ceph-mon[32752]: pgmap v20092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:14 smithi119 ceph-mon[32752]: pgmap v20093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T04:47:14.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:47:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:47:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:14 smithi167 ceph-mon[32751]: pgmap v20092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:14 smithi167 ceph-mon[32751]: pgmap v20093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T04:47:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:47:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:47:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:14 smithi090 ceph-mon[29180]: pgmap v20092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:47:14.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:14 smithi090 ceph-mon[29180]: pgmap v20093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 551 B/s rd, 1 op/s 2024-04-27T04:47:14.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:47:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:16 smithi119 ceph-mon[32752]: pgmap v20094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:16 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36717.552883 secs 2024-04-27T04:47:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:16 smithi167 ceph-mon[32751]: pgmap v20094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:16 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36717.552883 secs 2024-04-27T04:47:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:16 smithi090 ceph-mon[29180]: pgmap v20094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:16 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36717.552883 secs 2024-04-27T04:47:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:17 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36718.444424 secs 2024-04-27T04:47:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:17 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36718.444424 secs 2024-04-27T04:47:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:17 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36718.444424 secs 2024-04-27T04:47:18.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:18 smithi119 ceph-mon[32752]: pgmap v20095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:18.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:18 smithi167 ceph-mon[32751]: pgmap v20095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:18.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:18 smithi090 ceph-mon[29180]: pgmap v20095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:20.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:20 smithi119 ceph-mon[32752]: pgmap v20096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:20.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:20 smithi167 ceph-mon[32751]: pgmap v20096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:20.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:20 smithi090 ceph-mon[29180]: pgmap v20096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:21.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:21 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36722.552943 secs 2024-04-27T04:47:21.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:21 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36722.552943 secs 2024-04-27T04:47:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:21 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36722.552943 secs 2024-04-27T04:47:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:22 smithi119 ceph-mon[32752]: pgmap v20097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:22 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36723.444487 secs 2024-04-27T04:47:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:22 smithi167 ceph-mon[32751]: pgmap v20097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:22 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36723.444487 secs 2024-04-27T04:47:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:22 smithi090 ceph-mon[29180]: pgmap v20097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:47:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:22 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36723.444487 secs 2024-04-27T04:47:24.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:24 smithi119 ceph-mon[32752]: pgmap v20098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:47:24.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:24 smithi167 ceph-mon[32751]: pgmap v20098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:47:24.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:24 smithi090 ceph-mon[29180]: pgmap v20098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:47:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:26 smithi119 ceph-mon[32752]: pgmap v20099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:26 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36727.553029 secs 2024-04-27T04:47:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:26 smithi167 ceph-mon[32751]: pgmap v20099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:26 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36727.553029 secs 2024-04-27T04:47:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:26 smithi090 ceph-mon[29180]: pgmap v20099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:26 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36727.553029 secs 2024-04-27T04:47:27.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:27 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36728.444652 secs 2024-04-27T04:47:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:27 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36728.444652 secs 2024-04-27T04:47:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:27 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36728.444652 secs 2024-04-27T04:47:28.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:28 smithi119 ceph-mon[32752]: pgmap v20100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:28.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:28 smithi167 ceph-mon[32751]: pgmap v20100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:28 smithi090 ceph-mon[29180]: pgmap v20100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:30.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:30 smithi119 ceph-mon[32752]: pgmap v20101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:30.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:30 smithi167 ceph-mon[32751]: pgmap v20101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:30 smithi090 ceph-mon[29180]: pgmap v20101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:31 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36732.553113 secs 2024-04-27T04:47:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:31 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36732.553113 secs 2024-04-27T04:47:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:31 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36732.553113 secs 2024-04-27T04:47:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:32 smithi119 ceph-mon[32752]: pgmap v20102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:32 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36733.444764 secs 2024-04-27T04:47:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:32 smithi167 ceph-mon[32751]: pgmap v20102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:32 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36733.444764 secs 2024-04-27T04:47:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:32 smithi090 ceph-mon[29180]: pgmap v20102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:32 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36733.444764 secs 2024-04-27T04:47:34.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:34 smithi119 ceph-mon[32752]: pgmap v20103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:34.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:34 smithi167 ceph-mon[32751]: pgmap v20103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:34 smithi090 ceph-mon[29180]: pgmap v20103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:36 smithi119 ceph-mon[32752]: pgmap v20104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:36 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36737.553196 secs 2024-04-27T04:47:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:36 smithi167 ceph-mon[32751]: pgmap v20104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:36 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36737.553196 secs 2024-04-27T04:47:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:36 smithi090 ceph-mon[29180]: pgmap v20104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:36 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36737.553196 secs 2024-04-27T04:47:37.529 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:37 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36738.444904 secs 2024-04-27T04:47:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:37 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36738.444904 secs 2024-04-27T04:47:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:37 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36738.444904 secs 2024-04-27T04:47:38.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:38 smithi119 ceph-mon[32752]: pgmap v20105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:38.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:38 smithi167 ceph-mon[32751]: pgmap v20105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:38 smithi090 ceph-mon[29180]: pgmap v20105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:40.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:40 smithi119 ceph-mon[32752]: pgmap v20106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:40.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:40 smithi167 ceph-mon[32751]: pgmap v20106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:40.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:40 smithi090 ceph-mon[29180]: pgmap v20106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:41 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36742.553286 secs 2024-04-27T04:47:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:41 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36742.553286 secs 2024-04-27T04:47:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:41 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36742.553286 secs 2024-04-27T04:47:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:42 smithi119 ceph-mon[32752]: pgmap v20107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:42 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36743.445051 secs 2024-04-27T04:47:42.567 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:42 smithi090 ceph-mon[29180]: pgmap v20107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:42.568 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:42 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36743.445051 secs 2024-04-27T04:47:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:42 smithi167 ceph-mon[32751]: pgmap v20107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:42 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36743.445051 secs 2024-04-27T04:47:44.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:44 smithi119 ceph-mon[32752]: pgmap v20108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:44.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:44 smithi167 ceph-mon[32751]: pgmap v20108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:44 smithi090 ceph-mon[29180]: pgmap v20108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:46 smithi119 ceph-mon[32752]: pgmap v20109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:46 smithi119 ceph-mon[32752]: 48 slow requests, 1 included below; oldest blocked for > 36747.553368 secs 2024-04-27T04:47:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:46 smithi119 ceph-mon[32752]: slow request 364.459202 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:47:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:46 smithi167 ceph-mon[32751]: pgmap v20109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:46 smithi167 ceph-mon[32751]: 48 slow requests, 1 included below; oldest blocked for > 36747.553368 secs 2024-04-27T04:47:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:46 smithi167 ceph-mon[32751]: slow request 364.459202 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:47:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:46 smithi090 ceph-mon[29180]: pgmap v20109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:46 smithi090 ceph-mon[29180]: 48 slow requests, 1 included below; oldest blocked for > 36747.553368 secs 2024-04-27T04:47:46.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:46 smithi090 ceph-mon[29180]: slow request 364.459202 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:47:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:47 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36748.445180 secs 2024-04-27T04:47:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:47 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36748.445180 secs 2024-04-27T04:47:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:47 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36748.445180 secs 2024-04-27T04:47:48.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:48 smithi119 ceph-mon[32752]: pgmap v20110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:48.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:48 smithi167 ceph-mon[32751]: pgmap v20110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:48 smithi090 ceph-mon[29180]: pgmap v20110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:50.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:50 smithi119 ceph-mon[32752]: pgmap v20111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:50.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:50 smithi167 ceph-mon[32751]: pgmap v20111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:50 smithi090 ceph-mon[29180]: pgmap v20111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:51 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36752.553452 secs 2024-04-27T04:47:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:51 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36752.553452 secs 2024-04-27T04:47:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:51 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36752.553452 secs 2024-04-27T04:47:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:52 smithi119 ceph-mon[32752]: pgmap v20112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:52 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36753.445292 secs 2024-04-27T04:47:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:52 smithi167 ceph-mon[32751]: pgmap v20112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:52 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36753.445292 secs 2024-04-27T04:47:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:52 smithi090 ceph-mon[29180]: pgmap v20112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:52 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36753.445292 secs 2024-04-27T04:47:54.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:54 smithi119 ceph-mon[32752]: pgmap v20113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:54.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:54 smithi167 ceph-mon[32751]: pgmap v20113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:54.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:54 smithi090 ceph-mon[29180]: pgmap v20113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:56 smithi119 ceph-mon[32752]: pgmap v20114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:56 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36757.553536 secs 2024-04-27T04:47:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:56 smithi167 ceph-mon[32751]: pgmap v20114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:56 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36757.553536 secs 2024-04-27T04:47:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:56 smithi090 ceph-mon[29180]: pgmap v20114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:47:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:56 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36757.553536 secs 2024-04-27T04:47:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:57 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36758.445411 secs 2024-04-27T04:47:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:57 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36758.445411 secs 2024-04-27T04:47:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:57 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36758.445411 secs 2024-04-27T04:47:58.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:47:58 smithi119 ceph-mon[32752]: pgmap v20115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:58.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:47:58 smithi167 ceph-mon[32751]: pgmap v20115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:47:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:47:58 smithi090 ceph-mon[29180]: pgmap v20115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:00.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:00 smithi119 ceph-mon[32752]: pgmap v20116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:00.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:00 smithi167 ceph-mon[32751]: pgmap v20116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:00 smithi090 ceph-mon[29180]: pgmap v20116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:01 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36762.553623 secs 2024-04-27T04:48:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:01 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36762.553623 secs 2024-04-27T04:48:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:01 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36762.553623 secs 2024-04-27T04:48:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:02 smithi167 ceph-mon[32751]: pgmap v20117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:02 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36763.445530 secs 2024-04-27T04:48:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:02 smithi090 ceph-mon[29180]: pgmap v20117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:02 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36763.445530 secs 2024-04-27T04:48:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:02 smithi119 ceph-mon[32752]: pgmap v20117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:02 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36763.445530 secs 2024-04-27T04:48:04.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:04 smithi167 ceph-mon[32751]: pgmap v20118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:04 smithi090 ceph-mon[29180]: pgmap v20118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:04 smithi119 ceph-mon[32752]: pgmap v20118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:06 smithi167 ceph-mon[32751]: pgmap v20119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:06 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36767.553704 secs 2024-04-27T04:48:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:06 smithi090 ceph-mon[29180]: pgmap v20119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:06 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36767.553704 secs 2024-04-27T04:48:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:06 smithi119 ceph-mon[32752]: pgmap v20119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:06 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36767.553704 secs 2024-04-27T04:48:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:07 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36768.445611 secs 2024-04-27T04:48:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:07 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36768.445611 secs 2024-04-27T04:48:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:07 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36768.445611 secs 2024-04-27T04:48:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:08 smithi090 ceph-mon[29180]: pgmap v20120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:08 smithi119 ceph-mon[32752]: pgmap v20120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:08 smithi167 ceph-mon[32751]: pgmap v20120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:10.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:10 smithi090 ceph-mon[29180]: pgmap v20121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:10 smithi119 ceph-mon[32752]: pgmap v20121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:10 smithi167 ceph-mon[32751]: pgmap v20121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:11 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36772.553813 secs 2024-04-27T04:48:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:11 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36772.553813 secs 2024-04-27T04:48:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:11 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36772.553813 secs 2024-04-27T04:48:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:12 smithi090 ceph-mon[29180]: pgmap v20122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:12 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36773.445684 secs 2024-04-27T04:48:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:12 smithi119 ceph-mon[32752]: pgmap v20122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:12 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36773.445684 secs 2024-04-27T04:48:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:12 smithi167 ceph-mon[32751]: pgmap v20122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:12 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36773.445684 secs 2024-04-27T04:48:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:48:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:48:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:48:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:14 smithi090 ceph-mon[29180]: pgmap v20123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:48:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:48:14.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:48:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:14 smithi119 ceph-mon[32752]: pgmap v20123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:48:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:48:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:48:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:14 smithi167 ceph-mon[32751]: pgmap v20123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:48:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:48:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:48:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:15 smithi090 ceph-mon[29180]: pgmap v20124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:48:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:15 smithi090 ceph-mon[29180]: pgmap v20125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:15 smithi119 ceph-mon[32752]: pgmap v20124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:48:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:15 smithi119 ceph-mon[32752]: pgmap v20125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:15 smithi167 ceph-mon[32751]: pgmap v20124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:48:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:15 smithi167 ceph-mon[32751]: pgmap v20125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:16 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36777.553900 secs 2024-04-27T04:48:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:16 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36777.553900 secs 2024-04-27T04:48:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:16 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36777.553900 secs 2024-04-27T04:48:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:17 smithi090 ceph-mon[29180]: pgmap v20126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:17 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36778.445807 secs 2024-04-27T04:48:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:17 smithi119 ceph-mon[32752]: pgmap v20126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:17 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36778.445807 secs 2024-04-27T04:48:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:17 smithi167 ceph-mon[32751]: pgmap v20126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:17 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36778.445807 secs 2024-04-27T04:48:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:19 smithi090 ceph-mon[29180]: pgmap v20127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:48:19.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:19 smithi119 ceph-mon[32752]: pgmap v20127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:48:19.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:19 smithi167 ceph-mon[32751]: pgmap v20127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:48:21.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:21 smithi119 ceph-mon[32752]: pgmap v20128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:21.608 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:21 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36782.553989 secs 2024-04-27T04:48:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:21 smithi167 ceph-mon[32751]: pgmap v20128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:21.622 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:21 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36782.553989 secs 2024-04-27T04:48:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:21 smithi090 ceph-mon[29180]: pgmap v20128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:21 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36782.553989 secs 2024-04-27T04:48:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:22 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36783.445892 secs 2024-04-27T04:48:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:22 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36783.445892 secs 2024-04-27T04:48:22.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:22 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36783.445892 secs 2024-04-27T04:48:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:23 smithi090 ceph-mon[29180]: pgmap v20129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:23 smithi119 ceph-mon[32752]: pgmap v20129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:23 smithi167 ceph-mon[32751]: pgmap v20129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:48:25.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:25 smithi090 ceph-mon[29180]: pgmap v20130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:48:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:25 smithi119 ceph-mon[32752]: pgmap v20130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:48:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:25 smithi167 ceph-mon[32751]: pgmap v20130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:48:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:26 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36787.554088 secs 2024-04-27T04:48:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:26 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36787.554088 secs 2024-04-27T04:48:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:26 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36787.554088 secs 2024-04-27T04:48:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:27 smithi090 ceph-mon[29180]: pgmap v20131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:27 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36788.446008 secs 2024-04-27T04:48:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:27 smithi119 ceph-mon[32752]: pgmap v20131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:27 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36788.446008 secs 2024-04-27T04:48:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:27 smithi167 ceph-mon[32751]: pgmap v20131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:27 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36788.446008 secs 2024-04-27T04:48:29.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:29 smithi090 ceph-mon[29180]: pgmap v20132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:29.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:29 smithi119 ceph-mon[32752]: pgmap v20132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:29.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:29 smithi167 ceph-mon[32751]: pgmap v20132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:31 smithi119 ceph-mon[32752]: pgmap v20133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:31 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36792.554192 secs 2024-04-27T04:48:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:31 smithi167 ceph-mon[32751]: pgmap v20133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:31 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36792.554192 secs 2024-04-27T04:48:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:31 smithi090 ceph-mon[29180]: pgmap v20133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:31 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36792.554192 secs 2024-04-27T04:48:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:32 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36793.446145 secs 2024-04-27T04:48:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:32 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36793.446145 secs 2024-04-27T04:48:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:32 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36793.446145 secs 2024-04-27T04:48:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:33 smithi119 ceph-mon[32752]: pgmap v20134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:33.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:33 smithi167 ceph-mon[32751]: pgmap v20134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:33 smithi090 ceph-mon[29180]: pgmap v20134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:35 smithi119 ceph-mon[32752]: pgmap v20135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:35 smithi167 ceph-mon[32751]: pgmap v20135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:35 smithi090 ceph-mon[29180]: pgmap v20135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:36 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36797.554290 secs 2024-04-27T04:48:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:36 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36797.554290 secs 2024-04-27T04:48:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:36 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36797.554290 secs 2024-04-27T04:48:37.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:37 smithi090 ceph-mon[29180]: pgmap v20136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:37.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:37 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36798.446246 secs 2024-04-27T04:48:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:37 smithi119 ceph-mon[32752]: pgmap v20136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:37 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36798.446246 secs 2024-04-27T04:48:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:37 smithi167 ceph-mon[32751]: pgmap v20136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:37 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36798.446246 secs 2024-04-27T04:48:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:39 smithi119 ceph-mon[32752]: pgmap v20137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:39 smithi167 ceph-mon[32751]: pgmap v20137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:39 smithi090 ceph-mon[29180]: pgmap v20137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:41 smithi119 ceph-mon[32752]: pgmap v20138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:41 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36802.554393 secs 2024-04-27T04:48:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:41 smithi167 ceph-mon[32751]: pgmap v20138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:41 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36802.554393 secs 2024-04-27T04:48:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:41 smithi090 ceph-mon[29180]: pgmap v20138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:41 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36802.554393 secs 2024-04-27T04:48:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:42 smithi119 ceph-mon[32752]: 15 slow requests, 1 included below; oldest blocked for > 36803.446321 secs 2024-04-27T04:48:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:42 smithi119 ceph-mon[32752]: slow request 720.404970 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:48:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:42 smithi167 ceph-mon[32751]: 15 slow requests, 1 included below; oldest blocked for > 36803.446321 secs 2024-04-27T04:48:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:42 smithi167 ceph-mon[32751]: slow request 720.404970 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:48:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:42 smithi090 ceph-mon[29180]: 15 slow requests, 1 included below; oldest blocked for > 36803.446321 secs 2024-04-27T04:48:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:42 smithi090 ceph-mon[29180]: slow request 720.404970 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:48:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:43 smithi119 ceph-mon[32752]: pgmap v20139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:43 smithi167 ceph-mon[32751]: pgmap v20139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:43 smithi090 ceph-mon[29180]: pgmap v20139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:45 smithi119 ceph-mon[32752]: pgmap v20140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:45 smithi167 ceph-mon[32751]: pgmap v20140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:45 smithi090 ceph-mon[29180]: pgmap v20140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:46 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36807.554501 secs 2024-04-27T04:48:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:46 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36807.554501 secs 2024-04-27T04:48:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:46 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36807.554501 secs 2024-04-27T04:48:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:47 smithi119 ceph-mon[32752]: pgmap v20141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:47 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36808.446415 secs 2024-04-27T04:48:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:47 smithi167 ceph-mon[32751]: pgmap v20141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:47 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36808.446415 secs 2024-04-27T04:48:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:47 smithi090 ceph-mon[29180]: pgmap v20141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:47.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:47 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36808.446415 secs 2024-04-27T04:48:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:49 smithi119 ceph-mon[32752]: pgmap v20142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:49 smithi167 ceph-mon[32751]: pgmap v20142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:49 smithi090 ceph-mon[29180]: pgmap v20142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:51 smithi119 ceph-mon[32752]: pgmap v20143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:51 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36812.554586 secs 2024-04-27T04:48:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:51 smithi167 ceph-mon[32751]: pgmap v20143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:51 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36812.554586 secs 2024-04-27T04:48:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:51 smithi090 ceph-mon[29180]: pgmap v20143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:51 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36812.554586 secs 2024-04-27T04:48:52.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:52 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36813.446524 secs 2024-04-27T04:48:52.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:52 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36813.446524 secs 2024-04-27T04:48:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:52 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36813.446524 secs 2024-04-27T04:48:53.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:53 smithi119 ceph-mon[32752]: pgmap v20144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:53.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:53 smithi167 ceph-mon[32751]: pgmap v20144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:53 smithi090 ceph-mon[29180]: pgmap v20144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:55.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:55 smithi119 ceph-mon[32752]: pgmap v20145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:55.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:55 smithi167 ceph-mon[32751]: pgmap v20145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:55 smithi090 ceph-mon[29180]: pgmap v20145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:48:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:56 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36817.554669 secs 2024-04-27T04:48:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:56 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36817.554669 secs 2024-04-27T04:48:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:56 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36817.554669 secs 2024-04-27T04:48:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:57 smithi119 ceph-mon[32752]: pgmap v20146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:57 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36818.446589 secs 2024-04-27T04:48:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:57 smithi167 ceph-mon[32751]: pgmap v20146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:57 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36818.446589 secs 2024-04-27T04:48:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:57 smithi090 ceph-mon[29180]: pgmap v20146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:57.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:57 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36818.446589 secs 2024-04-27T04:48:59.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:48:59 smithi119 ceph-mon[32752]: pgmap v20147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:59.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:48:59 smithi167 ceph-mon[32751]: pgmap v20147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:48:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:48:59 smithi090 ceph-mon[29180]: pgmap v20147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:01 smithi119 ceph-mon[32752]: pgmap v20148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:01 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36822.554774 secs 2024-04-27T04:49:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:01 smithi167 ceph-mon[32751]: pgmap v20148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:01 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36822.554774 secs 2024-04-27T04:49:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:01 smithi090 ceph-mon[29180]: pgmap v20148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:01 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36822.554774 secs 2024-04-27T04:49:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:02 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36823.446678 secs 2024-04-27T04:49:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:02 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36823.446678 secs 2024-04-27T04:49:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:02 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36823.446678 secs 2024-04-27T04:49:03.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:03 smithi119 ceph-mon[32752]: pgmap v20149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:03.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:03 smithi167 ceph-mon[32751]: pgmap v20149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:03 smithi090 ceph-mon[29180]: pgmap v20149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:05 smithi119 ceph-mon[32752]: pgmap v20150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:05 smithi167 ceph-mon[32751]: pgmap v20150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:05 smithi090 ceph-mon[29180]: pgmap v20150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:06 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36827.554797 secs 2024-04-27T04:49:06.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:06 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36827.554797 secs 2024-04-27T04:49:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:06 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36827.554797 secs 2024-04-27T04:49:07.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:07 smithi090 ceph-mon[29180]: pgmap v20151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:07.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:07 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36828.446805 secs 2024-04-27T04:49:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:07 smithi119 ceph-mon[32752]: pgmap v20151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:07 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36828.446805 secs 2024-04-27T04:49:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:07 smithi167 ceph-mon[32751]: pgmap v20151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:07 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36828.446805 secs 2024-04-27T04:49:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:09 smithi119 ceph-mon[32752]: pgmap v20152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:09 smithi167 ceph-mon[32751]: pgmap v20152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:09 smithi090 ceph-mon[29180]: pgmap v20152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:11 smithi119 ceph-mon[32752]: pgmap v20153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:11.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:11 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36832.554866 secs 2024-04-27T04:49:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:11 smithi167 ceph-mon[32751]: pgmap v20153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:11 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36832.554866 secs 2024-04-27T04:49:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:11 smithi090 ceph-mon[29180]: pgmap v20153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:11 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36832.554866 secs 2024-04-27T04:49:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:12 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36833.446897 secs 2024-04-27T04:49:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:12 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36833.446897 secs 2024-04-27T04:49:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:12 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36833.446897 secs 2024-04-27T04:49:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:13 smithi119 ceph-mon[32752]: pgmap v20154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:13.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:13 smithi167 ceph-mon[32751]: pgmap v20154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:13 smithi090 ceph-mon[29180]: pgmap v20154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:49:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:49:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:49:15.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:15 smithi090 ceph-mon[29180]: pgmap v20155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:15.752 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:49:15.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:49:15.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:15 smithi090 ceph-mon[29180]: pgmap v20156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:49:15.753 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:49:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:15 smithi119 ceph-mon[32752]: pgmap v20155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:49:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:49:15.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:15 smithi119 ceph-mon[32752]: pgmap v20156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:49:15.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:49:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:15 smithi167 ceph-mon[32751]: pgmap v20155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:49:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:49:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:15 smithi167 ceph-mon[32751]: pgmap v20156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:49:15.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:49:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:16 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36837.554925 secs 2024-04-27T04:49:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:16 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36837.554925 secs 2024-04-27T04:49:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:16 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36837.554925 secs 2024-04-27T04:49:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:17 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36838.446986 secs 2024-04-27T04:49:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:17 smithi119 ceph-mon[32752]: pgmap v20157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:17 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36838.446986 secs 2024-04-27T04:49:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:17 smithi167 ceph-mon[32751]: pgmap v20157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:17 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36838.446986 secs 2024-04-27T04:49:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:17 smithi090 ceph-mon[29180]: pgmap v20157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:20.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:19 smithi119 ceph-mon[32752]: pgmap v20158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:49:20.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:19 smithi167 ceph-mon[32751]: pgmap v20158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:49:20.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:19 smithi090 ceph-mon[29180]: pgmap v20158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:49:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:21 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36842.554983 secs 2024-04-27T04:49:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:21 smithi119 ceph-mon[32752]: pgmap v20159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:49:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:21 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36842.554983 secs 2024-04-27T04:49:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:21 smithi167 ceph-mon[32751]: pgmap v20159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:49:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:21 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36842.554983 secs 2024-04-27T04:49:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:21 smithi090 ceph-mon[29180]: pgmap v20159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T04:49:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:22 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36843.447112 secs 2024-04-27T04:49:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:22 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36843.447112 secs 2024-04-27T04:49:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:22 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36843.447112 secs 2024-04-27T04:49:24.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:23 smithi119 ceph-mon[32752]: pgmap v20160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:24.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:23 smithi167 ceph-mon[32751]: pgmap v20160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:23 smithi090 ceph-mon[29180]: pgmap v20160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:25 smithi119 ceph-mon[32752]: pgmap v20161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:25 smithi167 ceph-mon[32751]: pgmap v20161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:26.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:25 smithi090 ceph-mon[29180]: pgmap v20161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:49:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:26 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36847.555061 secs 2024-04-27T04:49:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:26 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36847.555061 secs 2024-04-27T04:49:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:26 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36847.555061 secs 2024-04-27T04:49:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:27 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36848.447198 secs 2024-04-27T04:49:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:27 smithi119 ceph-mon[32752]: pgmap v20162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:27 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36848.447198 secs 2024-04-27T04:49:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:27 smithi167 ceph-mon[32751]: pgmap v20162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:27 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36848.447198 secs 2024-04-27T04:49:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:27 smithi090 ceph-mon[29180]: pgmap v20162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:29 smithi119 ceph-mon[32752]: pgmap v20163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:30.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:29 smithi167 ceph-mon[32751]: pgmap v20163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:29 smithi090 ceph-mon[29180]: pgmap v20163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:31 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36852.555149 secs 2024-04-27T04:49:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:31 smithi090 ceph-mon[29180]: pgmap v20164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:31 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36852.555149 secs 2024-04-27T04:49:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:31 smithi119 ceph-mon[32752]: pgmap v20164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:31 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36852.555149 secs 2024-04-27T04:49:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:31 smithi167 ceph-mon[32751]: pgmap v20164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:32 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36853.447280 secs 2024-04-27T04:49:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:32 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36853.447280 secs 2024-04-27T04:49:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:32 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36853.447280 secs 2024-04-27T04:49:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:33 smithi119 ceph-mon[32752]: pgmap v20165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:34.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:33 smithi167 ceph-mon[32751]: pgmap v20165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:33 smithi090 ceph-mon[29180]: pgmap v20165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:35 smithi119 ceph-mon[32752]: pgmap v20166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:36.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:35 smithi167 ceph-mon[32751]: pgmap v20166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:36.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:35 smithi090 ceph-mon[29180]: pgmap v20166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:36 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36857.555229 secs 2024-04-27T04:49:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:36 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36857.555229 secs 2024-04-27T04:49:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:36 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36857.555229 secs 2024-04-27T04:49:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:37 smithi119 ceph-mon[32752]: 15 slow requests, 0 included below; oldest blocked for > 36858.447394 secs 2024-04-27T04:49:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:37 smithi119 ceph-mon[32752]: pgmap v20167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:37 smithi167 ceph-mon[32751]: 15 slow requests, 0 included below; oldest blocked for > 36858.447394 secs 2024-04-27T04:49:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:37 smithi167 ceph-mon[32751]: pgmap v20167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:37 smithi090 ceph-mon[29180]: 15 slow requests, 0 included below; oldest blocked for > 36858.447394 secs 2024-04-27T04:49:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:37 smithi090 ceph-mon[29180]: pgmap v20167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:39 smithi167 ceph-mon[32751]: pgmap v20168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:40.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:39 smithi090 ceph-mon[29180]: pgmap v20168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:40.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:39 smithi119 ceph-mon[32752]: pgmap v20168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:41 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36862.555292 secs 2024-04-27T04:49:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:41 smithi167 ceph-mon[32751]: pgmap v20169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:41 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36862.555292 secs 2024-04-27T04:49:42.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:41 smithi090 ceph-mon[29180]: pgmap v20169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:41 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36862.555292 secs 2024-04-27T04:49:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:41 smithi119 ceph-mon[32752]: pgmap v20169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:42 smithi167 ceph-mon[32751]: 33 slow requests, 1 included below; oldest blocked for > 36863.447464 secs 2024-04-27T04:49:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:42 smithi167 ceph-mon[32751]: slow request 180.218764 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:49:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:42 smithi090 ceph-mon[29180]: 33 slow requests, 1 included below; oldest blocked for > 36863.447464 secs 2024-04-27T04:49:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:42 smithi090 ceph-mon[29180]: slow request 180.218764 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:49:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:42 smithi119 ceph-mon[32752]: 33 slow requests, 1 included below; oldest blocked for > 36863.447464 secs 2024-04-27T04:49:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:42 smithi119 ceph-mon[32752]: slow request 180.218764 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:49:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:43 smithi167 ceph-mon[32751]: pgmap v20170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:44.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:43 smithi090 ceph-mon[29180]: pgmap v20170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:43 smithi119 ceph-mon[32752]: pgmap v20170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:45 smithi090 ceph-mon[29180]: pgmap v20171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:46.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:49:46.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:49:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:45 smithi119 ceph-mon[32752]: pgmap v20171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:49:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:49:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:45 smithi167 ceph-mon[32751]: pgmap v20171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:49:46.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:49:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:46 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36867.555373 secs 2024-04-27T04:49:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:46 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36867.555373 secs 2024-04-27T04:49:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:46 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36867.555373 secs 2024-04-27T04:49:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:47 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36868.447575 secs 2024-04-27T04:49:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:47 smithi090 ceph-mon[29180]: pgmap v20172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:47 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36868.447575 secs 2024-04-27T04:49:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:47 smithi119 ceph-mon[32752]: pgmap v20172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:48.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:47 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36868.447575 secs 2024-04-27T04:49:48.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:47 smithi167 ceph-mon[32751]: pgmap v20172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:49 smithi090 ceph-mon[29180]: pgmap v20173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:49 smithi119 ceph-mon[32752]: pgmap v20173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:50.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:49 smithi167 ceph-mon[32751]: pgmap v20173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:52.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:51 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36872.555457 secs 2024-04-27T04:49:52.108 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:51 smithi119 ceph-mon[32752]: pgmap v20174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:52.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:51 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36872.555457 secs 2024-04-27T04:49:52.122 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:51 smithi167 ceph-mon[32751]: pgmap v20174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:51 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36872.555457 secs 2024-04-27T04:49:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:51 smithi090 ceph-mon[29180]: pgmap v20174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:52 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36873.447700 secs 2024-04-27T04:49:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:52 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36873.447700 secs 2024-04-27T04:49:53.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:52 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36873.447700 secs 2024-04-27T04:49:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:53 smithi090 ceph-mon[29180]: pgmap v20175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:53 smithi119 ceph-mon[32752]: pgmap v20175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:54.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:53 smithi167 ceph-mon[32751]: pgmap v20175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:55 smithi090 ceph-mon[29180]: pgmap v20176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:55 smithi119 ceph-mon[32752]: pgmap v20176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:55 smithi167 ceph-mon[32751]: pgmap v20176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:49:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:56 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36877.555556 secs 2024-04-27T04:49:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:56 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36878.447771 secs 2024-04-27T04:49:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:56 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36877.555556 secs 2024-04-27T04:49:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:56 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36878.447771 secs 2024-04-27T04:49:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:56 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36877.555556 secs 2024-04-27T04:49:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:56 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36878.447771 secs 2024-04-27T04:49:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:57 smithi090 ceph-mon[29180]: pgmap v20177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:57 smithi119 ceph-mon[32752]: pgmap v20177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:49:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:57 smithi167 ceph-mon[32751]: pgmap v20177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:00.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:49:59 smithi090 ceph-mon[29180]: pgmap v20178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:00.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:49:59 smithi119 ceph-mon[32752]: pgmap v20178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:00.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:49:59 smithi167 ceph-mon[32751]: pgmap v20178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:50:01.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:50:01.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T04:50:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:01 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36882.555640 secs 2024-04-27T04:50:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:01 smithi090 ceph-mon[29180]: pgmap v20179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:01 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36883.447892 secs 2024-04-27T04:50:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:01 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36882.555640 secs 2024-04-27T04:50:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:01 smithi119 ceph-mon[32752]: pgmap v20179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:02.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:01 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36883.447892 secs 2024-04-27T04:50:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:01 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36882.555640 secs 2024-04-27T04:50:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:01 smithi167 ceph-mon[32751]: pgmap v20179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:02.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:01 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36883.447892 secs 2024-04-27T04:50:04.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:03 smithi090 ceph-mon[29180]: pgmap v20180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:04.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:03 smithi119 ceph-mon[32752]: pgmap v20180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:04.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:03 smithi167 ceph-mon[32751]: pgmap v20180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:06.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:05 smithi090 ceph-mon[29180]: pgmap v20181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:05 smithi119 ceph-mon[32752]: pgmap v20181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:05 smithi167 ceph-mon[32751]: pgmap v20181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:06 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36887.555724 secs 2024-04-27T04:50:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:06 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36888.448014 secs 2024-04-27T04:50:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:06 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36887.555724 secs 2024-04-27T04:50:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:06 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36888.448014 secs 2024-04-27T04:50:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:06 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36887.555724 secs 2024-04-27T04:50:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:06 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36888.448014 secs 2024-04-27T04:50:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:07 smithi119 ceph-mon[32752]: pgmap v20182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:08.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:07 smithi167 ceph-mon[32751]: pgmap v20182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:08.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:07 smithi090 ceph-mon[29180]: pgmap v20182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:09 smithi119 ceph-mon[32752]: pgmap v20183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:10.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:09 smithi167 ceph-mon[32751]: pgmap v20183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:09 smithi090 ceph-mon[29180]: pgmap v20183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:11 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36892.555827 secs 2024-04-27T04:50:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:11 smithi119 ceph-mon[32752]: pgmap v20184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:12.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:11 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36893.448149 secs 2024-04-27T04:50:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:11 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36892.555827 secs 2024-04-27T04:50:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:11 smithi167 ceph-mon[32751]: pgmap v20184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:11 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36893.448149 secs 2024-04-27T04:50:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:11 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36892.555827 secs 2024-04-27T04:50:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:11 smithi090 ceph-mon[29180]: pgmap v20184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:12.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:11 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36893.448149 secs 2024-04-27T04:50:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:13 smithi119 ceph-mon[32752]: pgmap v20185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:14.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:13 smithi167 ceph-mon[32751]: pgmap v20185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:13 smithi090 ceph-mon[29180]: pgmap v20185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:15.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:50:15.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:50:15.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:50:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:15 smithi119 ceph-mon[32752]: pgmap v20186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:50:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:50:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:15 smithi119 ceph-mon[32752]: pgmap v20187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:50:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:15 smithi119 ceph-mon[32752]: pgmap v20188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:16.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:50:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:15 smithi167 ceph-mon[32751]: pgmap v20186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:50:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:50:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:15 smithi167 ceph-mon[32751]: pgmap v20187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:50:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:15 smithi167 ceph-mon[32751]: pgmap v20188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:16.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:50:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:15 smithi090 ceph-mon[29180]: pgmap v20186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:50:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:50:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:15 smithi090 ceph-mon[29180]: pgmap v20187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:50:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:15 smithi090 ceph-mon[29180]: pgmap v20188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:16.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:50:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:16 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36897.555892 secs 2024-04-27T04:50:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:16 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36898.448266 secs 2024-04-27T04:50:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:16 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36897.555892 secs 2024-04-27T04:50:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:16 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36898.448266 secs 2024-04-27T04:50:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:16 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36897.555892 secs 2024-04-27T04:50:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:16 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36898.448266 secs 2024-04-27T04:50:18.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:17 smithi090 ceph-mon[29180]: pgmap v20189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:50:18.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:17 smithi119 ceph-mon[32752]: pgmap v20189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:50:18.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:17 smithi167 ceph-mon[32751]: pgmap v20189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:50:19.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:19 smithi119 ceph-mon[32752]: pgmap v20190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:19.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:19 smithi167 ceph-mon[32751]: pgmap v20190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:19 smithi090 ceph-mon[29180]: pgmap v20190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:20 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36902.555972 secs 2024-04-27T04:50:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:20 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36902.555972 secs 2024-04-27T04:50:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:20 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36902.555972 secs 2024-04-27T04:50:22.221 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:21 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36903.448345 secs 2024-04-27T04:50:22.221 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:21 smithi119 ceph-mon[32752]: pgmap v20191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:22.221 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:21 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36903.448345 secs 2024-04-27T04:50:22.222 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:21 smithi167 ceph-mon[32751]: pgmap v20191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:21 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36903.448345 secs 2024-04-27T04:50:22.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:21 smithi090 ceph-mon[29180]: pgmap v20191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:23 smithi119 ceph-mon[32752]: pgmap v20192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:23 smithi167 ceph-mon[32751]: pgmap v20192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:23 smithi090 ceph-mon[29180]: pgmap v20192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:50:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:25 smithi119 ceph-mon[32752]: pgmap v20193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:50:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:25 smithi167 ceph-mon[32751]: pgmap v20193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:50:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:25 smithi090 ceph-mon[29180]: pgmap v20193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:50:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:26 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36907.556033 secs 2024-04-27T04:50:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:26 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36907.556033 secs 2024-04-27T04:50:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:26 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36907.556033 secs 2024-04-27T04:50:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:27 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36908.448421 secs 2024-04-27T04:50:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:27 smithi119 ceph-mon[32752]: pgmap v20194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:27 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36908.448421 secs 2024-04-27T04:50:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:27 smithi167 ceph-mon[32751]: pgmap v20194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:27 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36908.448421 secs 2024-04-27T04:50:27.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:27 smithi090 ceph-mon[29180]: pgmap v20194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:30 smithi119 ceph-mon[32752]: pgmap v20195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:30.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:30 smithi167 ceph-mon[32751]: pgmap v20195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:30 smithi090 ceph-mon[29180]: pgmap v20195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:31 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36912.556120 secs 2024-04-27T04:50:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:31 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36912.556120 secs 2024-04-27T04:50:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:31 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36912.556120 secs 2024-04-27T04:50:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:32 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36913.448521 secs 2024-04-27T04:50:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:32 smithi119 ceph-mon[32752]: pgmap v20196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:32 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36913.448521 secs 2024-04-27T04:50:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:32 smithi167 ceph-mon[32751]: pgmap v20196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:32 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36913.448521 secs 2024-04-27T04:50:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:32 smithi090 ceph-mon[29180]: pgmap v20196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:34 smithi167 ceph-mon[32751]: pgmap v20197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:34 smithi090 ceph-mon[29180]: pgmap v20197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:35.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:34 smithi119 ceph-mon[32752]: pgmap v20197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:36 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36917.556203 secs 2024-04-27T04:50:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:36 smithi167 ceph-mon[32751]: pgmap v20198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:36 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36917.556203 secs 2024-04-27T04:50:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:36 smithi090 ceph-mon[29180]: pgmap v20198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:36 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36917.556203 secs 2024-04-27T04:50:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:36 smithi119 ceph-mon[32752]: pgmap v20198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:37 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36918.448624 secs 2024-04-27T04:50:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:37 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36918.448624 secs 2024-04-27T04:50:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:37 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36918.448624 secs 2024-04-27T04:50:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:38 smithi167 ceph-mon[32751]: pgmap v20199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:38 smithi090 ceph-mon[29180]: pgmap v20199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:39.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:38 smithi119 ceph-mon[32752]: pgmap v20199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:40.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:40 smithi090 ceph-mon[29180]: pgmap v20200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:41.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:40 smithi119 ceph-mon[32752]: pgmap v20200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:41.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:40 smithi167 ceph-mon[32751]: pgmap v20200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:41 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36922.556266 secs 2024-04-27T04:50:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:41 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36922.556266 secs 2024-04-27T04:50:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:41 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36922.556266 secs 2024-04-27T04:50:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:42 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36923.448736 secs 2024-04-27T04:50:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:42 smithi090 ceph-mon[29180]: pgmap v20201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:42 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36923.448736 secs 2024-04-27T04:50:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:42 smithi119 ceph-mon[32752]: pgmap v20201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:42 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36923.448736 secs 2024-04-27T04:50:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:42 smithi167 ceph-mon[32751]: pgmap v20201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:44.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:44 smithi090 ceph-mon[29180]: pgmap v20202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:44 smithi119 ceph-mon[32752]: pgmap v20202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:45.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:44 smithi167 ceph-mon[32751]: pgmap v20202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:46 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36927.556347 secs 2024-04-27T04:50:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:46 smithi090 ceph-mon[29180]: pgmap v20203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:46 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36927.556347 secs 2024-04-27T04:50:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:46 smithi119 ceph-mon[32752]: pgmap v20203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:46 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36927.556347 secs 2024-04-27T04:50:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:46 smithi167 ceph-mon[32751]: pgmap v20203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:47 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36928.448823 secs 2024-04-27T04:50:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:47 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36928.448823 secs 2024-04-27T04:50:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:47 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36928.448823 secs 2024-04-27T04:50:48.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:48 smithi090 ceph-mon[29180]: pgmap v20204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:49.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:48 smithi119 ceph-mon[32752]: pgmap v20204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:49.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:48 smithi167 ceph-mon[32751]: pgmap v20204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:50 smithi090 ceph-mon[29180]: pgmap v20205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:50 smithi119 ceph-mon[32752]: pgmap v20205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:50 smithi167 ceph-mon[32751]: pgmap v20205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:51 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36932.556434 secs 2024-04-27T04:50:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:51 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36932.556434 secs 2024-04-27T04:50:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:51 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36932.556434 secs 2024-04-27T04:50:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:52 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36933.448947 secs 2024-04-27T04:50:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:52 smithi090 ceph-mon[29180]: pgmap v20206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:52 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36933.448947 secs 2024-04-27T04:50:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:52 smithi119 ceph-mon[32752]: pgmap v20206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:52.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:52 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36933.448947 secs 2024-04-27T04:50:52.972 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:52 smithi167 ceph-mon[32751]: pgmap v20206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:54 smithi090 ceph-mon[29180]: pgmap v20207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:55.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:54 smithi119 ceph-mon[32752]: pgmap v20207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:55.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:54 smithi167 ceph-mon[32751]: pgmap v20207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:50:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:56 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36937.556516 secs 2024-04-27T04:50:56.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:56 smithi090 ceph-mon[29180]: pgmap v20208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:56 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36937.556516 secs 2024-04-27T04:50:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:56 smithi119 ceph-mon[32752]: pgmap v20208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:56 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36937.556516 secs 2024-04-27T04:50:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:56 smithi167 ceph-mon[32751]: pgmap v20208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:57 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36938.449031 secs 2024-04-27T04:50:58.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:57 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36938.449031 secs 2024-04-27T04:50:58.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:57 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36938.449031 secs 2024-04-27T04:50:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:50:58 smithi090 ceph-mon[29180]: pgmap v20209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:59.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:50:58 smithi119 ceph-mon[32752]: pgmap v20209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:50:59.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:50:58 smithi167 ceph-mon[32751]: pgmap v20209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:00 smithi090 ceph-mon[29180]: pgmap v20210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:00 smithi119 ceph-mon[32752]: pgmap v20210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:01.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:00 smithi167 ceph-mon[32751]: pgmap v20210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:01 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36942.556609 secs 2024-04-27T04:51:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:01 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36942.556609 secs 2024-04-27T04:51:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:01 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36942.556609 secs 2024-04-27T04:51:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:02 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36943.449179 secs 2024-04-27T04:51:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:02 smithi090 ceph-mon[29180]: pgmap v20211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:02 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36943.449179 secs 2024-04-27T04:51:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:02 smithi119 ceph-mon[32752]: pgmap v20211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:02 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36943.449179 secs 2024-04-27T04:51:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:02 smithi167 ceph-mon[32751]: pgmap v20211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:04 smithi119 ceph-mon[32752]: pgmap v20212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:05.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:04 smithi167 ceph-mon[32751]: pgmap v20212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:04 smithi090 ceph-mon[29180]: pgmap v20212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:06 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36947.556691 secs 2024-04-27T04:51:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:06 smithi119 ceph-mon[32752]: pgmap v20213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:06 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36947.556691 secs 2024-04-27T04:51:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:06 smithi167 ceph-mon[32751]: pgmap v20213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:06 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36947.556691 secs 2024-04-27T04:51:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:06 smithi090 ceph-mon[29180]: pgmap v20213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:08.030 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:07 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36948.449284 secs 2024-04-27T04:51:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:07 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36948.449284 secs 2024-04-27T04:51:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:07 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36948.449284 secs 2024-04-27T04:51:09.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:08 smithi119 ceph-mon[32752]: pgmap v20214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:09.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:08 smithi167 ceph-mon[32751]: pgmap v20214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:08 smithi090 ceph-mon[29180]: pgmap v20214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:10 smithi119 ceph-mon[32752]: pgmap v20215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:11.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:10 smithi167 ceph-mon[32751]: pgmap v20215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:10 smithi090 ceph-mon[29180]: pgmap v20215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:11 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36952.556776 secs 2024-04-27T04:51:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:11 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36952.556776 secs 2024-04-27T04:51:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:11 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36952.556776 secs 2024-04-27T04:51:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:12 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36953.449429 secs 2024-04-27T04:51:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:12 smithi119 ceph-mon[32752]: pgmap v20216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:12 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36953.449429 secs 2024-04-27T04:51:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:12 smithi167 ceph-mon[32751]: pgmap v20216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:12 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36953.449429 secs 2024-04-27T04:51:13.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:12 smithi090 ceph-mon[29180]: pgmap v20216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:14 smithi119 ceph-mon[32752]: pgmap v20217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:15.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:14 smithi167 ceph-mon[32751]: pgmap v20217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:15.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:14 smithi090 ceph-mon[29180]: pgmap v20217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:51:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:51:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:51:16.914 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:16 smithi119 ceph-mon[32752]: 48 slow requests, 0 included below; oldest blocked for > 36957.556858 secs 2024-04-27T04:51:16.914 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:16 smithi119 ceph-mon[32752]: pgmap v20218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:16.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:16 smithi090 ceph-mon[29180]: 48 slow requests, 0 included below; oldest blocked for > 36957.556858 secs 2024-04-27T04:51:16.991 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:16 smithi090 ceph-mon[29180]: pgmap v20218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:16 smithi167 ceph-mon[32751]: 48 slow requests, 0 included below; oldest blocked for > 36957.556858 secs 2024-04-27T04:51:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:16 smithi167 ceph-mon[32751]: pgmap v20218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:17.960 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:17 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36958.449521 secs 2024-04-27T04:51:17.970 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:17 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36958.449521 secs 2024-04-27T04:51:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:17 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36958.449521 secs 2024-04-27T04:51:19.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:18 smithi119 ceph-mon[32752]: pgmap v20219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:19.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:18 smithi167 ceph-mon[32751]: pgmap v20219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:19.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:18 smithi090 ceph-mon[29180]: pgmap v20219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: pgmap v20220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:51:21.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: pgmap v20221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-27T04:51:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: pgmap v20222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:21.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: pgmap v20220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:51:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:51:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: pgmap v20221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-27T04:51:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: pgmap v20222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:21.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: pgmap v20220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:51:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:51:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: pgmap v20221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 971 B/s rd, 1 op/s 2024-04-27T04:51:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: pgmap v20222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:21.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:51:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:21 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36962.556918 secs 2024-04-27T04:51:22.208 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:21 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36962.556918 secs 2024-04-27T04:51:22.221 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:21 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36962.556918 secs 2024-04-27T04:51:23.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:22 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36963.449600 secs 2024-04-27T04:51:23.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:22 smithi090 ceph-mon[29180]: pgmap v20223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-27T04:51:23.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:22 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36963.449600 secs 2024-04-27T04:51:23.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:22 smithi119 ceph-mon[32752]: pgmap v20223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-27T04:51:23.321 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:22 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36963.449600 secs 2024-04-27T04:51:23.322 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:22 smithi167 ceph-mon[32751]: pgmap v20223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-27T04:51:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:51:24.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:23 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:51:24.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:51:24.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:23 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:51:24.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:51:24.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:23 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:51:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:24 smithi090 ceph-mon[29180]: pgmap v20224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:24 smithi119 ceph-mon[32752]: pgmap v20224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:25.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:24 smithi167 ceph-mon[32751]: pgmap v20224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:26.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:25 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36967.556976 secs 2024-04-27T04:51:26.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:25 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36967.556976 secs 2024-04-27T04:51:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:25 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36967.556976 secs 2024-04-27T04:51:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:26 smithi119 ceph-mon[32752]: pgmap v20225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:26 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36968.449683 secs 2024-04-27T04:51:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:26 smithi167 ceph-mon[32751]: pgmap v20225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:26 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36968.449683 secs 2024-04-27T04:51:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:26 smithi090 ceph-mon[29180]: pgmap v20225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:26 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36968.449683 secs 2024-04-27T04:51:29.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:29 smithi090 ceph-mon[29180]: pgmap v20226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-27T04:51:29.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:29 smithi119 ceph-mon[32752]: pgmap v20226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-27T04:51:29.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:29 smithi167 ceph-mon[32751]: pgmap v20226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 599 B/s rd, 1 op/s 2024-04-27T04:51:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:31 smithi090 ceph-mon[29180]: pgmap v20227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:51:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:31 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36972.557036 secs 2024-04-27T04:51:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:31 smithi119 ceph-mon[32752]: pgmap v20227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:51:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:31 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36972.557036 secs 2024-04-27T04:51:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:31 smithi167 ceph-mon[32751]: pgmap v20227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:51:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:31 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36972.557036 secs 2024-04-27T04:51:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:32 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36973.449803 secs 2024-04-27T04:51:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:32 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36973.449803 secs 2024-04-27T04:51:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:32 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36973.449803 secs 2024-04-27T04:51:33.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:33 smithi090 ceph-mon[29180]: pgmap v20228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:33 smithi119 ceph-mon[32752]: pgmap v20228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:33.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:33 smithi167 ceph-mon[32751]: pgmap v20228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:35.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:35 smithi090 ceph-mon[29180]: pgmap v20229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:35.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:35 smithi119 ceph-mon[32752]: pgmap v20229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:35 smithi167 ceph-mon[32751]: pgmap v20229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:36 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36977.557106 secs 2024-04-27T04:51:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:36 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36977.557106 secs 2024-04-27T04:51:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:36 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36977.557106 secs 2024-04-27T04:51:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:37 smithi090 ceph-mon[29180]: pgmap v20230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:37 smithi090 ceph-mon[29180]: 33 slow requests, 0 included below; oldest blocked for > 36978.449936 secs 2024-04-27T04:51:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:37 smithi119 ceph-mon[32752]: pgmap v20230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:37.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:37 smithi119 ceph-mon[32752]: 33 slow requests, 0 included below; oldest blocked for > 36978.449936 secs 2024-04-27T04:51:37.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:37 smithi167 ceph-mon[32751]: pgmap v20230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:37.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:37 smithi167 ceph-mon[32751]: 33 slow requests, 0 included below; oldest blocked for > 36978.449936 secs 2024-04-27T04:51:39.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:39 smithi090 ceph-mon[29180]: pgmap v20231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:39.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:39 smithi119 ceph-mon[32752]: pgmap v20231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:39.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:39 smithi167 ceph-mon[32751]: pgmap v20231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:41 smithi090 ceph-mon[29180]: pgmap v20232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:41.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:41 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36982.557184 secs 2024-04-27T04:51:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:51:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:51:41.024+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:51:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:51:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:51:41.024+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:51:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:51:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:51:41.024+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:51:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:51:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:51:41.024+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:51:41.379 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:51:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:51:41.024+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:51:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:41 smithi119 ceph-mon[32752]: pgmap v20232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:41 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36982.557184 secs 2024-04-27T04:51:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:41 smithi167 ceph-mon[32751]: pgmap v20232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:51:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:41 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36982.557184 secs 2024-04-27T04:51:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 36983.450001 secs 2024-04-27T04:51:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 36983.450001 secs 2024-04-27T04:51:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 36983.450001 secs 2024-04-27T04:51:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:43 smithi119 ceph-mon[32752]: pgmap v20233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:43.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:43 smithi167 ceph-mon[32751]: pgmap v20233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:43.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:43 smithi090 ceph-mon[29180]: pgmap v20233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:45.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:45 smithi119 ceph-mon[32752]: pgmap v20234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:45.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:51:45.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:51:45.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:45 smithi167 ceph-mon[32751]: pgmap v20234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:45.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:51:45.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:51:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:45 smithi090 ceph-mon[29180]: pgmap v20234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:51:45.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:51:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:46 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36987.557261 secs 2024-04-27T04:51:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:46 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36987.557261 secs 2024-04-27T04:51:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:46 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36987.557261 secs 2024-04-27T04:51:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:47 smithi119 ceph-mon[32752]: pgmap v20235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 36988.450127 secs 2024-04-27T04:51:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:47 smithi167 ceph-mon[32751]: pgmap v20235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 36988.450127 secs 2024-04-27T04:51:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:47 smithi090 ceph-mon[29180]: pgmap v20235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 36988.450127 secs 2024-04-27T04:51:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:49 smithi119 ceph-mon[32752]: pgmap v20236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:49 smithi167 ceph-mon[32751]: pgmap v20236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:49.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:49 smithi090 ceph-mon[29180]: pgmap v20236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:51 smithi119 ceph-mon[32752]: pgmap v20237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:51 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36992.557347 secs 2024-04-27T04:51:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:51 smithi167 ceph-mon[32751]: pgmap v20237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:51 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36992.557347 secs 2024-04-27T04:51:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:51 smithi090 ceph-mon[29180]: pgmap v20237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:51 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36992.557347 secs 2024-04-27T04:51:52.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 36993.450280 secs 2024-04-27T04:51:52.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 36993.450280 secs 2024-04-27T04:51:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 36993.450280 secs 2024-04-27T04:51:53.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:53 smithi119 ceph-mon[32752]: pgmap v20238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:53.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:53 smithi167 ceph-mon[32751]: pgmap v20238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:53.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:53 smithi090 ceph-mon[29180]: pgmap v20238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:55.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:55 smithi119 ceph-mon[32752]: pgmap v20239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:55.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:55 smithi167 ceph-mon[32751]: pgmap v20239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:55.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:55 smithi090 ceph-mon[29180]: pgmap v20239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T04:51:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:56 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 36997.557427 secs 2024-04-27T04:51:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:56 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 36997.557427 secs 2024-04-27T04:51:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:56 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 36997.557427 secs 2024-04-27T04:51:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:57 smithi119 ceph-mon[32752]: pgmap v20240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 36998.450426 secs 2024-04-27T04:51:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:57 smithi167 ceph-mon[32751]: pgmap v20240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 36998.450426 secs 2024-04-27T04:51:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:57 smithi090 ceph-mon[29180]: pgmap v20240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 36998.450426 secs 2024-04-27T04:51:59.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:51:59 smithi119 ceph-mon[32752]: pgmap v20241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:59.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:51:59 smithi167 ceph-mon[32751]: pgmap v20241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:51:59.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:51:59 smithi090 ceph-mon[29180]: pgmap v20241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:01 smithi119 ceph-mon[32752]: pgmap v20242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:01.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:01 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37002.557513 secs 2024-04-27T04:52:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:01 smithi167 ceph-mon[32751]: pgmap v20242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:01 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37002.557513 secs 2024-04-27T04:52:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:01 smithi090 ceph-mon[29180]: pgmap v20242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:01.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:01 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37002.557513 secs 2024-04-27T04:52:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37003.450555 secs 2024-04-27T04:52:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37003.450555 secs 2024-04-27T04:52:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37003.450555 secs 2024-04-27T04:52:03.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:03 smithi119 ceph-mon[32752]: pgmap v20243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:03.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:03 smithi167 ceph-mon[32751]: pgmap v20243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:03.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:03 smithi090 ceph-mon[29180]: pgmap v20243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:05.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:05 smithi119 ceph-mon[32752]: pgmap v20244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:05.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:05 smithi167 ceph-mon[32751]: pgmap v20244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:05.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:05 smithi090 ceph-mon[29180]: pgmap v20244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:06 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37007.557591 secs 2024-04-27T04:52:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:06 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37007.557591 secs 2024-04-27T04:52:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:06 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37007.557591 secs 2024-04-27T04:52:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:07 smithi119 ceph-mon[32752]: pgmap v20245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37008.450621 secs 2024-04-27T04:52:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:07 smithi167 ceph-mon[32751]: pgmap v20245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37008.450621 secs 2024-04-27T04:52:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:07 smithi090 ceph-mon[29180]: pgmap v20245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37008.450621 secs 2024-04-27T04:52:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:09 smithi119 ceph-mon[32752]: pgmap v20246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:09 smithi167 ceph-mon[32751]: pgmap v20246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:09 smithi090 ceph-mon[29180]: pgmap v20246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:11 smithi119 ceph-mon[32752]: pgmap v20247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:11 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 37012.557676 secs 2024-04-27T04:52:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:11 smithi119 ceph-mon[32752]: slow request 5760.890085 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:52:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:11 smithi167 ceph-mon[32751]: pgmap v20247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:11 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 37012.557676 secs 2024-04-27T04:52:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:11 smithi167 ceph-mon[32751]: slow request 5760.890085 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:52:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:11 smithi090 ceph-mon[29180]: pgmap v20247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:11 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 37012.557676 secs 2024-04-27T04:52:11.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:11 smithi090 ceph-mon[29180]: slow request 5760.890085 seconds old, received at 2024-04-27T03:16:09.420995+0000: client_request(client.27014:1 getattr pAsLsXsFs #0x1 2024-04-27T03:16:09.416642+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:52:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37013.450737 secs 2024-04-27T04:52:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37013.450737 secs 2024-04-27T04:52:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37013.450737 secs 2024-04-27T04:52:13.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:13 smithi119 ceph-mon[32752]: pgmap v20248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:13 smithi167 ceph-mon[32751]: pgmap v20248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:13 smithi090 ceph-mon[29180]: pgmap v20248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:15.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:15 smithi119 ceph-mon[32752]: pgmap v20249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:15.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:15 smithi167 ceph-mon[32751]: pgmap v20249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:15 smithi090 ceph-mon[29180]: pgmap v20249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:16 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37017.557757 secs 2024-04-27T04:52:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:16 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37017.557757 secs 2024-04-27T04:52:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:16 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37017.557757 secs 2024-04-27T04:52:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:17 smithi119 ceph-mon[32752]: pgmap v20250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37018.450862 secs 2024-04-27T04:52:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:17 smithi167 ceph-mon[32751]: pgmap v20250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37018.450862 secs 2024-04-27T04:52:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:17 smithi090 ceph-mon[29180]: pgmap v20250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37018.450862 secs 2024-04-27T04:52:19.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:19 smithi119 ceph-mon[32752]: pgmap v20251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:19.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:19 smithi167 ceph-mon[32751]: pgmap v20251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:19 smithi090 ceph-mon[29180]: pgmap v20251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:20.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:52:20.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:52:20.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:52:21.502 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:21 smithi167 ceph-mon[32751]: pgmap v20252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:21.503 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:21 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37022.557846 secs 2024-04-27T04:52:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:21 smithi119 ceph-mon[32752]: pgmap v20252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:21 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37022.557846 secs 2024-04-27T04:52:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:21 smithi090 ceph-mon[29180]: pgmap v20252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:21.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:21 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37022.557846 secs 2024-04-27T04:52:22.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37023.451003 secs 2024-04-27T04:52:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37023.451003 secs 2024-04-27T04:52:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37023.451003 secs 2024-04-27T04:52:23.551 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:23 smithi167 ceph-mon[32751]: pgmap v20253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:23.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:23 smithi119 ceph-mon[32752]: pgmap v20253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:23 smithi090 ceph-mon[29180]: pgmap v20253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: pgmap v20254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:52:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:52:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: pgmap v20255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-27T04:52:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: pgmap v20256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:25.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: pgmap v20254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:52:25.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:52:25.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: pgmap v20255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-27T04:52:25.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: pgmap v20256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:25.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: pgmap v20254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:52:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:52:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: pgmap v20255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 983 B/s rd, 1 op/s 2024-04-27T04:52:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: pgmap v20256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:52:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:26 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37027.557923 secs 2024-04-27T04:52:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:26 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37027.557923 secs 2024-04-27T04:52:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:26 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37027.557923 secs 2024-04-27T04:52:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37028.451135 secs 2024-04-27T04:52:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:27 smithi119 ceph-mon[32752]: pgmap v20257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37028.451135 secs 2024-04-27T04:52:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:27 smithi167 ceph-mon[32751]: pgmap v20257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37028.451135 secs 2024-04-27T04:52:27.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:27 smithi090 ceph-mon[29180]: pgmap v20257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:29.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:29 smithi119 ceph-mon[32752]: pgmap v20258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:29.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:29 smithi167 ceph-mon[32751]: pgmap v20258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:29 smithi090 ceph-mon[29180]: pgmap v20258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:31 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37032.557998 secs 2024-04-27T04:52:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:31 smithi119 ceph-mon[32752]: pgmap v20259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:31 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37032.557998 secs 2024-04-27T04:52:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:31 smithi167 ceph-mon[32751]: pgmap v20259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:31 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37032.557998 secs 2024-04-27T04:52:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:31 smithi090 ceph-mon[29180]: pgmap v20259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37033.451262 secs 2024-04-27T04:52:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37033.451262 secs 2024-04-27T04:52:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37033.451262 secs 2024-04-27T04:52:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:33 smithi119 ceph-mon[32752]: pgmap v20260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-27T04:52:33.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:33 smithi167 ceph-mon[32751]: pgmap v20260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-27T04:52:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:33 smithi090 ceph-mon[29180]: pgmap v20260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 B/s rd, 1 op/s 2024-04-27T04:52:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:35 smithi119 ceph-mon[32752]: pgmap v20261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:52:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:35 smithi167 ceph-mon[32751]: pgmap v20261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:52:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:35 smithi090 ceph-mon[29180]: pgmap v20261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:52:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:36 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37037.558077 secs 2024-04-27T04:52:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:36 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37037.558077 secs 2024-04-27T04:52:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:36 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37037.558077 secs 2024-04-27T04:52:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37038.451411 secs 2024-04-27T04:52:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:37 smithi119 ceph-mon[32752]: pgmap v20262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37038.451411 secs 2024-04-27T04:52:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:37 smithi167 ceph-mon[32751]: pgmap v20262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37038.451411 secs 2024-04-27T04:52:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:37 smithi090 ceph-mon[29180]: pgmap v20262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:38 smithi119 ceph-mon[32752]: pgmap v20263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:38 smithi167 ceph-mon[32751]: pgmap v20263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:38 smithi090 ceph-mon[29180]: pgmap v20263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:40.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:40 smithi167 ceph-mon[32751]: pgmap v20264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:40.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:40 smithi090 ceph-mon[29180]: pgmap v20264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:41.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:40 smithi119 ceph-mon[32752]: pgmap v20264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:41 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37042.558160 secs 2024-04-27T04:52:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:41 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37042.558160 secs 2024-04-27T04:52:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:41 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37042.558160 secs 2024-04-27T04:52:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:42 smithi167 ceph-mon[32751]: 16 slow requests, 1 included below; oldest blocked for > 37043.451543 secs 2024-04-27T04:52:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:42 smithi167 ceph-mon[32751]: slow request 360.222843 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:52:42.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:42 smithi167 ceph-mon[32751]: pgmap v20265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:42 smithi090 ceph-mon[29180]: 16 slow requests, 1 included below; oldest blocked for > 37043.451543 secs 2024-04-27T04:52:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:42 smithi090 ceph-mon[29180]: slow request 360.222843 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:52:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:42 smithi090 ceph-mon[29180]: pgmap v20265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:42 smithi119 ceph-mon[32752]: 16 slow requests, 1 included below; oldest blocked for > 37043.451543 secs 2024-04-27T04:52:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:42 smithi119 ceph-mon[32752]: slow request 360.222843 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:52:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:42 smithi119 ceph-mon[32752]: pgmap v20265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:44.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:44 smithi167 ceph-mon[32751]: pgmap v20266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:44.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:44 smithi090 ceph-mon[29180]: pgmap v20266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:45.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:44 smithi119 ceph-mon[32752]: pgmap v20266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:45 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37047.558239 secs 2024-04-27T04:52:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:45 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37047.558239 secs 2024-04-27T04:52:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:45 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37047.558239 secs 2024-04-27T04:52:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37048.451657 secs 2024-04-27T04:52:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:47 smithi167 ceph-mon[32751]: pgmap v20267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37048.451657 secs 2024-04-27T04:52:47.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:47 smithi090 ceph-mon[29180]: pgmap v20267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37048.451657 secs 2024-04-27T04:52:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:47 smithi119 ceph-mon[32752]: pgmap v20267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:48.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:48 smithi167 ceph-mon[32751]: pgmap v20268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:48.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:48 smithi090 ceph-mon[29180]: pgmap v20268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:49.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:48 smithi119 ceph-mon[32752]: pgmap v20268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:50.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:50 smithi167 ceph-mon[32751]: pgmap v20269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:50 smithi090 ceph-mon[29180]: pgmap v20269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:50 smithi119 ceph-mon[32752]: pgmap v20269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:51 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37052.558321 secs 2024-04-27T04:52:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:51 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37052.558321 secs 2024-04-27T04:52:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:51 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37052.558321 secs 2024-04-27T04:52:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37053.451734 secs 2024-04-27T04:52:52.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:52 smithi090 ceph-mon[29180]: pgmap v20270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37053.451734 secs 2024-04-27T04:52:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:52 smithi119 ceph-mon[32752]: pgmap v20270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:53.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37053.451734 secs 2024-04-27T04:52:53.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:52 smithi167 ceph-mon[32751]: pgmap v20270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:54 smithi167 ceph-mon[32751]: pgmap v20271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:54 smithi090 ceph-mon[29180]: pgmap v20271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:55.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:54 smithi119 ceph-mon[32752]: pgmap v20271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:55 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37057.558405 secs 2024-04-27T04:52:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:55 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37057.558405 secs 2024-04-27T04:52:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:55 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37057.558405 secs 2024-04-27T04:52:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:56 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37058.451880 secs 2024-04-27T04:52:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:56 smithi119 ceph-mon[32752]: pgmap v20272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:56 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37058.451880 secs 2024-04-27T04:52:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:56 smithi167 ceph-mon[32751]: pgmap v20272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:56 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37058.451880 secs 2024-04-27T04:52:57.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:56 smithi090 ceph-mon[29180]: pgmap v20272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:52:58.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:52:58 smithi167 ceph-mon[32751]: pgmap v20273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:52:58 smithi090 ceph-mon[29180]: pgmap v20273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:52:59.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:52:58 smithi119 ceph-mon[32752]: pgmap v20273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:00 smithi167 ceph-mon[32751]: pgmap v20274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:00 smithi090 ceph-mon[29180]: pgmap v20274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:00 smithi119 ceph-mon[32752]: pgmap v20274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:01 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37062.558487 secs 2024-04-27T04:53:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:01 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37062.558487 secs 2024-04-27T04:53:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:01 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37062.558487 secs 2024-04-27T04:53:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37063.451997 secs 2024-04-27T04:53:02.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:02 smithi090 ceph-mon[29180]: pgmap v20275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37063.451997 secs 2024-04-27T04:53:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:02 smithi119 ceph-mon[32752]: pgmap v20275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37063.451997 secs 2024-04-27T04:53:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:02 smithi167 ceph-mon[32751]: pgmap v20275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:04 smithi167 ceph-mon[32751]: pgmap v20276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:04 smithi090 ceph-mon[29180]: pgmap v20276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:05.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:04 smithi119 ceph-mon[32752]: pgmap v20276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:05 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37067.558568 secs 2024-04-27T04:53:06.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:05 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37067.558568 secs 2024-04-27T04:53:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:05 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37067.558568 secs 2024-04-27T04:53:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:06 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37068.452073 secs 2024-04-27T04:53:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:06 smithi119 ceph-mon[32752]: pgmap v20277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:06 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37068.452073 secs 2024-04-27T04:53:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:06 smithi167 ceph-mon[32751]: pgmap v20277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:06 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37068.452073 secs 2024-04-27T04:53:07.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:06 smithi090 ceph-mon[29180]: pgmap v20277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:08 smithi167 ceph-mon[32751]: pgmap v20278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:08.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:08 smithi090 ceph-mon[29180]: pgmap v20278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:09.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:08 smithi119 ceph-mon[32752]: pgmap v20278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:10 smithi167 ceph-mon[32751]: pgmap v20279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:10 smithi090 ceph-mon[29180]: pgmap v20279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:11.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:10 smithi119 ceph-mon[32752]: pgmap v20279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:11 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37072.558654 secs 2024-04-27T04:53:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:11 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37072.558654 secs 2024-04-27T04:53:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:11 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37072.558654 secs 2024-04-27T04:53:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37073.452203 secs 2024-04-27T04:53:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:12 smithi090 ceph-mon[29180]: pgmap v20280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37073.452203 secs 2024-04-27T04:53:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:12 smithi119 ceph-mon[32752]: pgmap v20280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37073.452203 secs 2024-04-27T04:53:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:12 smithi167 ceph-mon[32751]: pgmap v20280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:14 smithi167 ceph-mon[32751]: pgmap v20281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:14 smithi090 ceph-mon[29180]: pgmap v20281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:14 smithi119 ceph-mon[32752]: pgmap v20281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:15 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37077.558737 secs 2024-04-27T04:53:16.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:15 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37077.558737 secs 2024-04-27T04:53:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:15 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37077.558737 secs 2024-04-27T04:53:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:16 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37078.452339 secs 2024-04-27T04:53:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:16 smithi119 ceph-mon[32752]: pgmap v20282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:16 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37078.452339 secs 2024-04-27T04:53:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:16 smithi167 ceph-mon[32751]: pgmap v20282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:16 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37078.452339 secs 2024-04-27T04:53:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:16 smithi090 ceph-mon[29180]: pgmap v20282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:18.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:18 smithi167 ceph-mon[32751]: pgmap v20283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:18 smithi090 ceph-mon[29180]: pgmap v20283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:19.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:18 smithi119 ceph-mon[32752]: pgmap v20283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:20 smithi167 ceph-mon[32751]: pgmap v20284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:20 smithi090 ceph-mon[29180]: pgmap v20284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:20 smithi119 ceph-mon[32752]: pgmap v20284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:21 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37082.558825 secs 2024-04-27T04:53:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:21 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37082.558825 secs 2024-04-27T04:53:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:21 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37082.558825 secs 2024-04-27T04:53:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37083.452474 secs 2024-04-27T04:53:22.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:22 smithi090 ceph-mon[29180]: pgmap v20285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:22.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37083.452474 secs 2024-04-27T04:53:22.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:22 smithi119 ceph-mon[32752]: pgmap v20285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37083.452474 secs 2024-04-27T04:53:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:22 smithi167 ceph-mon[32751]: pgmap v20285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:24 smithi167 ceph-mon[32751]: pgmap v20286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:24 smithi090 ceph-mon[29180]: pgmap v20286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:25.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:24 smithi119 ceph-mon[32752]: pgmap v20286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:53:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:53:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:53:25.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:25 smithi090 ceph-mon[29180]: pgmap v20287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-27T04:53:25.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:25 smithi090 ceph-mon[29180]: pgmap v20288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:25.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:53:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:53:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:53:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:53:26.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:25 smithi119 ceph-mon[32752]: pgmap v20287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-27T04:53:26.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:25 smithi119 ceph-mon[32752]: pgmap v20288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:26.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:53:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:53:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:53:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:53:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:25 smithi167 ceph-mon[32751]: pgmap v20287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 944 B/s rd, 1 op/s 2024-04-27T04:53:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:25 smithi167 ceph-mon[32751]: pgmap v20288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:26.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:53:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:26 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37087.558909 secs 2024-04-27T04:53:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:26 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37087.558909 secs 2024-04-27T04:53:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:26 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37087.558909 secs 2024-04-27T04:53:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37088.452570 secs 2024-04-27T04:53:27.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:27 smithi090 ceph-mon[29180]: pgmap v20289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:53:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37088.452570 secs 2024-04-27T04:53:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:27 smithi119 ceph-mon[32752]: pgmap v20289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:53:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37088.452570 secs 2024-04-27T04:53:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:27 smithi167 ceph-mon[32751]: pgmap v20289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:53:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:30 smithi119 ceph-mon[32752]: pgmap v20290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:30.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:30 smithi167 ceph-mon[32751]: pgmap v20290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:30 smithi090 ceph-mon[29180]: pgmap v20290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:31 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37092.558968 secs 2024-04-27T04:53:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:31 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37092.558968 secs 2024-04-27T04:53:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:31 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37092.558968 secs 2024-04-27T04:53:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37093.452676 secs 2024-04-27T04:53:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:32 smithi119 ceph-mon[32752]: pgmap v20291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37093.452676 secs 2024-04-27T04:53:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:32 smithi167 ceph-mon[32751]: pgmap v20291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37093.452676 secs 2024-04-27T04:53:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:32 smithi090 ceph-mon[29180]: pgmap v20291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:53:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:34 smithi119 ceph-mon[32752]: pgmap v20292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:53:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:34 smithi167 ceph-mon[32751]: pgmap v20292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:53:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:34 smithi090 ceph-mon[29180]: pgmap v20292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T04:53:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:36 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37097.559057 secs 2024-04-27T04:53:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:36 smithi119 ceph-mon[32752]: pgmap v20293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:53:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:36 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37097.559057 secs 2024-04-27T04:53:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:36 smithi167 ceph-mon[32751]: pgmap v20293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:53:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:36 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37097.559057 secs 2024-04-27T04:53:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:36 smithi090 ceph-mon[29180]: pgmap v20293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:53:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37098.452801 secs 2024-04-27T04:53:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37098.452801 secs 2024-04-27T04:53:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37098.452801 secs 2024-04-27T04:53:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:38 smithi119 ceph-mon[32752]: pgmap v20294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:38 smithi167 ceph-mon[32751]: pgmap v20294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:38 smithi090 ceph-mon[29180]: pgmap v20294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:40.681 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:40 smithi090 ceph-mon[29180]: pgmap v20295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:40.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:40 smithi119 ceph-mon[32752]: pgmap v20295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:40.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:40 smithi167 ceph-mon[32751]: pgmap v20295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:41 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37102.559140 secs 2024-04-27T04:53:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:41 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37102.559140 secs 2024-04-27T04:53:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:41 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37102.559140 secs 2024-04-27T04:53:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:42 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37103.452882 secs 2024-04-27T04:53:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:42 smithi119 ceph-mon[32752]: pgmap v20296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:42 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37103.452882 secs 2024-04-27T04:53:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:42 smithi167 ceph-mon[32751]: pgmap v20296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:42 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37103.452882 secs 2024-04-27T04:53:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:42 smithi090 ceph-mon[29180]: pgmap v20296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:44 smithi119 ceph-mon[32752]: pgmap v20297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:44.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:44 smithi167 ceph-mon[32751]: pgmap v20297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:44.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:44 smithi090 ceph-mon[29180]: pgmap v20297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:46 smithi119 ceph-mon[32752]: 23 slow requests, 1 included below; oldest blocked for > 37107.559226 secs 2024-04-27T04:53:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:46 smithi119 ceph-mon[32752]: slow request 724.465060 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:53:46.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:46 smithi119 ceph-mon[32752]: pgmap v20298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:46 smithi167 ceph-mon[32751]: 23 slow requests, 1 included below; oldest blocked for > 37107.559226 secs 2024-04-27T04:53:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:46 smithi167 ceph-mon[32751]: slow request 724.465060 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:53:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:46 smithi167 ceph-mon[32751]: pgmap v20298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:46 smithi090 ceph-mon[29180]: 23 slow requests, 1 included below; oldest blocked for > 37107.559226 secs 2024-04-27T04:53:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:46 smithi090 ceph-mon[29180]: slow request 724.465060 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:53:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:46 smithi090 ceph-mon[29180]: pgmap v20298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:47 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37108.453025 secs 2024-04-27T04:53:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:47 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37108.453025 secs 2024-04-27T04:53:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:47 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37108.453025 secs 2024-04-27T04:53:48.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:48 smithi119 ceph-mon[32752]: pgmap v20299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:48.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:48 smithi167 ceph-mon[32751]: pgmap v20299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:48.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:48 smithi090 ceph-mon[29180]: pgmap v20299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:50.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:50 smithi119 ceph-mon[32752]: pgmap v20300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:50.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:50 smithi167 ceph-mon[32751]: pgmap v20300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:50 smithi090 ceph-mon[29180]: pgmap v20300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:51 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37112.559310 secs 2024-04-27T04:53:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:51 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37112.559310 secs 2024-04-27T04:53:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:51 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37112.559310 secs 2024-04-27T04:53:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:52 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37113.453158 secs 2024-04-27T04:53:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:52 smithi119 ceph-mon[32752]: pgmap v20301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:52 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37113.453158 secs 2024-04-27T04:53:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:52 smithi167 ceph-mon[32751]: pgmap v20301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:52 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37113.453158 secs 2024-04-27T04:53:52.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:52 smithi090 ceph-mon[29180]: pgmap v20301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:54 smithi119 ceph-mon[32752]: pgmap v20302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:54.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:54 smithi167 ceph-mon[32751]: pgmap v20302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:54 smithi090 ceph-mon[29180]: pgmap v20302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:53:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:56 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37117.559394 secs 2024-04-27T04:53:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:56 smithi119 ceph-mon[32752]: pgmap v20303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:56 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37117.559394 secs 2024-04-27T04:53:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:56 smithi167 ceph-mon[32751]: pgmap v20303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:56 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37117.559394 secs 2024-04-27T04:53:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:56 smithi090 ceph-mon[29180]: pgmap v20303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:57 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37118.453252 secs 2024-04-27T04:53:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:57 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37118.453252 secs 2024-04-27T04:53:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:57 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37118.453252 secs 2024-04-27T04:53:58.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:53:58 smithi119 ceph-mon[32752]: pgmap v20304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:58.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:53:58 smithi167 ceph-mon[32751]: pgmap v20304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:53:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:53:58 smithi090 ceph-mon[29180]: pgmap v20304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:00 smithi119 ceph-mon[32752]: pgmap v20305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:00 smithi167 ceph-mon[32751]: pgmap v20305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:00 smithi090 ceph-mon[29180]: pgmap v20305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:01 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37122.559476 secs 2024-04-27T04:54:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:01 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37122.559476 secs 2024-04-27T04:54:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:01 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37122.559476 secs 2024-04-27T04:54:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:02 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37123.453393 secs 2024-04-27T04:54:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:02 smithi119 ceph-mon[32752]: pgmap v20306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:02 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37123.453393 secs 2024-04-27T04:54:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:02 smithi167 ceph-mon[32751]: pgmap v20306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:02 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37123.453393 secs 2024-04-27T04:54:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:02 smithi090 ceph-mon[29180]: pgmap v20306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:04 smithi119 ceph-mon[32752]: pgmap v20307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:04 smithi167 ceph-mon[32751]: pgmap v20307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:04 smithi090 ceph-mon[29180]: pgmap v20307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:06 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37127.559564 secs 2024-04-27T04:54:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:06 smithi119 ceph-mon[32752]: pgmap v20308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:06 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37127.559564 secs 2024-04-27T04:54:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:06 smithi167 ceph-mon[32751]: pgmap v20308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:06 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37127.559564 secs 2024-04-27T04:54:06.882 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:06 smithi090 ceph-mon[29180]: pgmap v20308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:07 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37128.453514 secs 2024-04-27T04:54:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:07 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37128.453514 secs 2024-04-27T04:54:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:07 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37128.453514 secs 2024-04-27T04:54:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:08 smithi119 ceph-mon[32752]: pgmap v20309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:08 smithi167 ceph-mon[32751]: pgmap v20309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:08.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:08 smithi090 ceph-mon[29180]: pgmap v20309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:10 smithi119 ceph-mon[32752]: pgmap v20310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:10 smithi167 ceph-mon[32751]: pgmap v20310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:10 smithi090 ceph-mon[29180]: pgmap v20310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:11.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:11 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37132.559651 secs 2024-04-27T04:54:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:11 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37132.559651 secs 2024-04-27T04:54:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:11 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37132.559651 secs 2024-04-27T04:54:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:12 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37133.453613 secs 2024-04-27T04:54:12.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:12 smithi119 ceph-mon[32752]: pgmap v20311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:12 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37133.453613 secs 2024-04-27T04:54:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:12 smithi167 ceph-mon[32751]: pgmap v20311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:12 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37133.453613 secs 2024-04-27T04:54:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:12 smithi090 ceph-mon[29180]: pgmap v20311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:14 smithi119 ceph-mon[32752]: pgmap v20312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:14 smithi167 ceph-mon[32751]: pgmap v20312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:14 smithi090 ceph-mon[29180]: pgmap v20312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:16 smithi119 ceph-mon[32752]: 23 slow requests, 0 included below; oldest blocked for > 37137.559735 secs 2024-04-27T04:54:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:16 smithi119 ceph-mon[32752]: pgmap v20313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:16 smithi167 ceph-mon[32751]: 23 slow requests, 0 included below; oldest blocked for > 37137.559735 secs 2024-04-27T04:54:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:16 smithi167 ceph-mon[32751]: pgmap v20313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:16 smithi090 ceph-mon[29180]: 23 slow requests, 0 included below; oldest blocked for > 37137.559735 secs 2024-04-27T04:54:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:16 smithi090 ceph-mon[29180]: pgmap v20313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:17 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37138.453728 secs 2024-04-27T04:54:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:17 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37138.453728 secs 2024-04-27T04:54:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:17 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37138.453728 secs 2024-04-27T04:54:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:18 smithi119 ceph-mon[32752]: pgmap v20314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:18.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:18 smithi167 ceph-mon[32751]: pgmap v20314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:18 smithi090 ceph-mon[29180]: pgmap v20314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:20 smithi119 ceph-mon[32752]: pgmap v20315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:20 smithi167 ceph-mon[32751]: pgmap v20315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:20 smithi090 ceph-mon[29180]: pgmap v20315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:21 smithi119 ceph-mon[32752]: 49 slow requests, 1 included below; oldest blocked for > 37142.559839 secs 2024-04-27T04:54:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:21 smithi119 ceph-mon[32752]: slow request 181.440877 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:54:21.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:21 smithi167 ceph-mon[32751]: 49 slow requests, 1 included below; oldest blocked for > 37142.559839 secs 2024-04-27T04:54:21.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:21 smithi167 ceph-mon[32751]: slow request 181.440877 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:54:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:21 smithi090 ceph-mon[29180]: 49 slow requests, 1 included below; oldest blocked for > 37142.559839 secs 2024-04-27T04:54:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:21 smithi090 ceph-mon[29180]: slow request 181.440877 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:54:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:22 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37143.453844 secs 2024-04-27T04:54:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:22 smithi119 ceph-mon[32752]: pgmap v20316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:22.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:22 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37143.453844 secs 2024-04-27T04:54:22.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:22 smithi167 ceph-mon[32751]: pgmap v20316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:22 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37143.453844 secs 2024-04-27T04:54:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:22 smithi090 ceph-mon[29180]: pgmap v20316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:23 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:54:23.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:23 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:54:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:23 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:54:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:23 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:54:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:23 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:54:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:23 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:54:24.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:24 smithi119 ceph-mon[32752]: pgmap v20317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:24 smithi167 ceph-mon[32751]: pgmap v20317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:24 smithi090 ceph-mon[29180]: pgmap v20317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:25 smithi119 ceph-mon[32752]: pgmap v20318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:54:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:25 smithi167 ceph-mon[32751]: pgmap v20318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:54:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:25 smithi090 ceph-mon[29180]: pgmap v20318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:54:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:26 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37147.559923 secs 2024-04-27T04:54:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:54:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:54:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:26 smithi119 ceph-mon[32752]: pgmap v20319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:54:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:26 smithi119 ceph-mon[32752]: pgmap v20320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:26.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:54:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:26 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37147.559923 secs 2024-04-27T04:54:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:54:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:54:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:26 smithi167 ceph-mon[32751]: pgmap v20319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:54:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:26 smithi167 ceph-mon[32751]: pgmap v20320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:26.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:54:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:26 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37147.559923 secs 2024-04-27T04:54:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:54:26.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:54:26.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:26 smithi090 ceph-mon[29180]: pgmap v20319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:54:26.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:26 smithi090 ceph-mon[29180]: pgmap v20320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:26.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:54:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:27 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37148.453956 secs 2024-04-27T04:54:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:27 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37148.453956 secs 2024-04-27T04:54:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:27 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37148.453956 secs 2024-04-27T04:54:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:28 smithi119 ceph-mon[32752]: pgmap v20321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:54:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:28 smithi167 ceph-mon[32751]: pgmap v20321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:54:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:28 smithi090 ceph-mon[29180]: pgmap v20321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:54:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:31 smithi119 ceph-mon[32752]: pgmap v20322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:31 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37152.560027 secs 2024-04-27T04:54:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:31 smithi167 ceph-mon[32751]: pgmap v20322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:31 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37152.560027 secs 2024-04-27T04:54:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:31 smithi090 ceph-mon[29180]: pgmap v20322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:31 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37152.560027 secs 2024-04-27T04:54:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:32 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37153.454048 secs 2024-04-27T04:54:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:32 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37153.454048 secs 2024-04-27T04:54:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:32 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37153.454048 secs 2024-04-27T04:54:33.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:33 smithi119 ceph-mon[32752]: pgmap v20323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:54:33.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:33 smithi167 ceph-mon[32751]: pgmap v20323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:54:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:33 smithi090 ceph-mon[29180]: pgmap v20323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:54:35.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:35 smithi119 ceph-mon[32752]: pgmap v20324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:35.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:35 smithi167 ceph-mon[32751]: pgmap v20324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:35 smithi090 ceph-mon[29180]: pgmap v20324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:54:36.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:36 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37157.560112 secs 2024-04-27T04:54:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:36 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37157.560112 secs 2024-04-27T04:54:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:36 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37157.560112 secs 2024-04-27T04:54:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:37 smithi119 ceph-mon[32752]: pgmap v20325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:54:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:37 smithi119 ceph-mon[32752]: 16 slow requests, 0 included below; oldest blocked for > 37158.454170 secs 2024-04-27T04:54:37.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:37 smithi167 ceph-mon[32751]: pgmap v20325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:54:37.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:37 smithi167 ceph-mon[32751]: 16 slow requests, 0 included below; oldest blocked for > 37158.454170 secs 2024-04-27T04:54:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:37 smithi090 ceph-mon[29180]: pgmap v20325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:54:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:37 smithi090 ceph-mon[29180]: 16 slow requests, 0 included below; oldest blocked for > 37158.454170 secs 2024-04-27T04:54:39.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:39 smithi119 ceph-mon[32752]: pgmap v20326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:39.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:39 smithi167 ceph-mon[32751]: pgmap v20326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:39.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:39 smithi090 ceph-mon[29180]: pgmap v20326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:41 smithi119 ceph-mon[32752]: pgmap v20327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:41 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37162.560203 secs 2024-04-27T04:54:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:41 smithi167 ceph-mon[32751]: pgmap v20327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:41.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:41 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37162.560203 secs 2024-04-27T04:54:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:41 smithi090 ceph-mon[29180]: pgmap v20327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:41 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37162.560203 secs 2024-04-27T04:54:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:42 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 37163.454290 secs 2024-04-27T04:54:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:42 smithi119 ceph-mon[32752]: slow request 180.170368 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:54:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:42 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 37163.454290 secs 2024-04-27T04:54:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:42 smithi167 ceph-mon[32751]: slow request 180.170368 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:54:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:42 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 37163.454290 secs 2024-04-27T04:54:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:42 smithi090 ceph-mon[29180]: slow request 180.170368 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:54:43.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:43 smithi119 ceph-mon[32752]: pgmap v20328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:43.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:43 smithi167 ceph-mon[32751]: pgmap v20328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:43.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:43 smithi090 ceph-mon[29180]: pgmap v20328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:45.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:45 smithi119 ceph-mon[32752]: pgmap v20329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:45.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:45 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:54:45.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:45 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:54:45.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:45 smithi167 ceph-mon[32751]: pgmap v20329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:45.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:45 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:54:45.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:45 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:54:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:45 smithi090 ceph-mon[29180]: pgmap v20329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:45 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T04:54:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:45 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:54:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:46 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37167.560282 secs 2024-04-27T04:54:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:46 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37167.560282 secs 2024-04-27T04:54:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:46 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37167.560282 secs 2024-04-27T04:54:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:47 smithi119 ceph-mon[32752]: pgmap v20330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:47 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37168.454396 secs 2024-04-27T04:54:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:47 smithi167 ceph-mon[32751]: pgmap v20330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:47 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37168.454396 secs 2024-04-27T04:54:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:47 smithi090 ceph-mon[29180]: pgmap v20330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:47 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37168.454396 secs 2024-04-27T04:54:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:49 smithi119 ceph-mon[32752]: pgmap v20331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:49 smithi167 ceph-mon[32751]: pgmap v20331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:49.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:49 smithi090 ceph-mon[29180]: pgmap v20331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:51 smithi119 ceph-mon[32752]: pgmap v20332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:51 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37172.560343 secs 2024-04-27T04:54:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:51 smithi167 ceph-mon[32751]: pgmap v20332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:51 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37172.560343 secs 2024-04-27T04:54:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:51 smithi090 ceph-mon[29180]: pgmap v20332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:51 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37172.560343 secs 2024-04-27T04:54:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:52 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37173.454521 secs 2024-04-27T04:54:52.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:52 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37173.454521 secs 2024-04-27T04:54:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:52 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37173.454521 secs 2024-04-27T04:54:53.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:53 smithi119 ceph-mon[32752]: pgmap v20333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:53.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:53 smithi167 ceph-mon[32751]: pgmap v20333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:53.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:53 smithi090 ceph-mon[29180]: pgmap v20333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:55.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:55 smithi119 ceph-mon[32752]: pgmap v20334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:55.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:55 smithi167 ceph-mon[32751]: pgmap v20334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:55.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:55 smithi090 ceph-mon[29180]: pgmap v20334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:54:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:56 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37177.560430 secs 2024-04-27T04:54:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:56 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37177.560430 secs 2024-04-27T04:54:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:56 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37177.560430 secs 2024-04-27T04:54:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:57 smithi119 ceph-mon[32752]: pgmap v20335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:57 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37178.454614 secs 2024-04-27T04:54:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:57 smithi167 ceph-mon[32751]: pgmap v20335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:57 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37178.454614 secs 2024-04-27T04:54:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:57 smithi090 ceph-mon[29180]: pgmap v20335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:57 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37178.454614 secs 2024-04-27T04:54:59.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:54:59 smithi119 ceph-mon[32752]: pgmap v20336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:59.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:54:59 smithi167 ceph-mon[32751]: pgmap v20336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:54:59.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:54:59 smithi090 ceph-mon[29180]: pgmap v20336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:01 smithi119 ceph-mon[32752]: pgmap v20337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:01 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37182.560515 secs 2024-04-27T04:55:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:01 smithi167 ceph-mon[32751]: pgmap v20337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:01 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37182.560515 secs 2024-04-27T04:55:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:01 smithi090 ceph-mon[29180]: pgmap v20337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:01 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37182.560515 secs 2024-04-27T04:55:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:02 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37183.454783 secs 2024-04-27T04:55:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:02 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37183.454783 secs 2024-04-27T04:55:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:02 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37183.454783 secs 2024-04-27T04:55:03.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:03 smithi119 ceph-mon[32752]: pgmap v20338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:03.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:03 smithi167 ceph-mon[32751]: pgmap v20338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:03.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:03 smithi090 ceph-mon[29180]: pgmap v20338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:05.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:05 smithi119 ceph-mon[32752]: pgmap v20339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:05.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:05 smithi167 ceph-mon[32751]: pgmap v20339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:05.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:05 smithi090 ceph-mon[29180]: pgmap v20339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:06 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37187.560602 secs 2024-04-27T04:55:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:06 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37187.560602 secs 2024-04-27T04:55:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:06 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37187.560602 secs 2024-04-27T04:55:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:07 smithi119 ceph-mon[32752]: pgmap v20340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:07 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 37188.454924 secs 2024-04-27T04:55:07.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:07 smithi119 ceph-mon[32752]: slow request 23041.612784 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:55:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:07 smithi167 ceph-mon[32751]: pgmap v20340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:07 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 37188.454924 secs 2024-04-27T04:55:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:07 smithi167 ceph-mon[32751]: slow request 23041.612784 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:55:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:07 smithi090 ceph-mon[29180]: pgmap v20340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:07 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 37188.454924 secs 2024-04-27T04:55:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:07 smithi090 ceph-mon[29180]: slow request 23041.612784 seconds old, received at 2024-04-26T22:31:04.592784+0000: client_request(client.17373:1 getattr pAsLsXsFs #0x1 2024-04-26T22:31:04.585331+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:55:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:09 smithi119 ceph-mon[32752]: pgmap v20341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:09 smithi167 ceph-mon[32751]: pgmap v20341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:09 smithi090 ceph-mon[29180]: pgmap v20341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:11 smithi119 ceph-mon[32752]: pgmap v20342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:11 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37192.560687 secs 2024-04-27T04:55:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:11 smithi167 ceph-mon[32751]: pgmap v20342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:11 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37192.560687 secs 2024-04-27T04:55:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:11 smithi090 ceph-mon[29180]: pgmap v20342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:11 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37192.560687 secs 2024-04-27T04:55:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:12 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37193.455003 secs 2024-04-27T04:55:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:12 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37193.455003 secs 2024-04-27T04:55:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:12 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37193.455003 secs 2024-04-27T04:55:13.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:13 smithi119 ceph-mon[32752]: pgmap v20343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:13 smithi167 ceph-mon[32751]: pgmap v20343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:13 smithi090 ceph-mon[29180]: pgmap v20343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:15.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:15 smithi119 ceph-mon[32752]: pgmap v20344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:15.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:15 smithi167 ceph-mon[32751]: pgmap v20344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:15 smithi090 ceph-mon[29180]: pgmap v20344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:16 smithi119 ceph-mon[32752]: 49 slow requests, 1 included below; oldest blocked for > 37197.560770 secs 2024-04-27T04:55:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:16 smithi119 ceph-mon[32752]: slow request 5760.364316 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:55:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:16 smithi167 ceph-mon[32751]: 49 slow requests, 1 included below; oldest blocked for > 37197.560770 secs 2024-04-27T04:55:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:16 smithi167 ceph-mon[32751]: slow request 5760.364316 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:55:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:16 smithi090 ceph-mon[29180]: 49 slow requests, 1 included below; oldest blocked for > 37197.560770 secs 2024-04-27T04:55:16.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:16 smithi090 ceph-mon[29180]: slow request 5760.364316 seconds old, received at 2024-04-27T03:19:14.949858+0000: client_request(client.24733:267902 getattr AsXsFs #0x100000001fb 2024-04-27T03:19:14.948276+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:55:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:17 smithi119 ceph-mon[32752]: pgmap v20345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:17 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37198.455147 secs 2024-04-27T04:55:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:17 smithi167 ceph-mon[32751]: pgmap v20345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:17 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37198.455147 secs 2024-04-27T04:55:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:17 smithi090 ceph-mon[29180]: pgmap v20345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:17 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37198.455147 secs 2024-04-27T04:55:19.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:19 smithi119 ceph-mon[32752]: pgmap v20346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:19.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:19 smithi167 ceph-mon[32751]: pgmap v20346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:19 smithi090 ceph-mon[29180]: pgmap v20346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:21 smithi119 ceph-mon[32752]: pgmap v20347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:21 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37202.560859 secs 2024-04-27T04:55:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:21 smithi167 ceph-mon[32751]: pgmap v20347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:21 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37202.560859 secs 2024-04-27T04:55:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:21 smithi090 ceph-mon[29180]: pgmap v20347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:21.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:21 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37202.560859 secs 2024-04-27T04:55:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:22 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37203.455264 secs 2024-04-27T04:55:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:22 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37203.455264 secs 2024-04-27T04:55:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:22 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37203.455264 secs 2024-04-27T04:55:23.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:23 smithi119 ceph-mon[32752]: pgmap v20348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:23.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:23 smithi167 ceph-mon[32751]: pgmap v20348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:23 smithi090 ceph-mon[29180]: pgmap v20348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:25.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:25 smithi119 ceph-mon[32752]: pgmap v20349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:25.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:25 smithi167 ceph-mon[32751]: pgmap v20349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:25.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:25 smithi090 ceph-mon[29180]: pgmap v20349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:26 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37207.560948 secs 2024-04-27T04:55:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:55:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:26 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37207.560948 secs 2024-04-27T04:55:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:55:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:26 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37207.560948 secs 2024-04-27T04:55:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:55:27.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:27 smithi119 ceph-mon[32752]: pgmap v20350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:27.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:27 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37208.455414 secs 2024-04-27T04:55:27.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:55:27.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:55:27.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:55:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:27 smithi167 ceph-mon[32751]: pgmap v20350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:27 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37208.455414 secs 2024-04-27T04:55:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:55:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:55:27.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:55:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:27 smithi090 ceph-mon[29180]: pgmap v20350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:27 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37208.455414 secs 2024-04-27T04:55:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:55:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:55:27.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:55:28.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:28 smithi167 ceph-mon[32751]: pgmap v20351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:55:28.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:28 smithi167 ceph-mon[32751]: pgmap v20352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:28 smithi090 ceph-mon[29180]: pgmap v20351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:55:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:28 smithi090 ceph-mon[29180]: pgmap v20352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:28 smithi119 ceph-mon[32752]: pgmap v20351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:55:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:28 smithi119 ceph-mon[32752]: pgmap v20352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:30 smithi167 ceph-mon[32751]: pgmap v20353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:30 smithi090 ceph-mon[29180]: pgmap v20353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:30 smithi119 ceph-mon[32752]: pgmap v20353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:31 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37212.561010 secs 2024-04-27T04:55:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:31 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37212.561010 secs 2024-04-27T04:55:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:31 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37212.561010 secs 2024-04-27T04:55:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:32 smithi167 ceph-mon[32751]: pgmap v20354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:32 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37213.455542 secs 2024-04-27T04:55:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:32 smithi090 ceph-mon[29180]: pgmap v20354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:32 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37213.455542 secs 2024-04-27T04:55:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:32 smithi119 ceph-mon[32752]: pgmap v20354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:32 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37213.455542 secs 2024-04-27T04:55:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:34 smithi090 ceph-mon[29180]: pgmap v20355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:55:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:34 smithi119 ceph-mon[32752]: pgmap v20355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:55:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:34 smithi167 ceph-mon[32751]: pgmap v20355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T04:55:36.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:36 smithi090 ceph-mon[29180]: pgmap v20356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:36.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:36 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37217.561067 secs 2024-04-27T04:55:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:36 smithi119 ceph-mon[32752]: pgmap v20356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:36 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37217.561067 secs 2024-04-27T04:55:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:36 smithi167 ceph-mon[32751]: pgmap v20356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:55:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:36 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37217.561067 secs 2024-04-27T04:55:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:37 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37218.455628 secs 2024-04-27T04:55:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:37 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37218.455628 secs 2024-04-27T04:55:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:37 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37218.455628 secs 2024-04-27T04:55:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:38 smithi090 ceph-mon[29180]: pgmap v20357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:55:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:38 smithi119 ceph-mon[32752]: pgmap v20357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:55:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:38 smithi167 ceph-mon[32751]: pgmap v20357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:55:40.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:40 smithi090 ceph-mon[29180]: pgmap v20358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:40.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:40 smithi119 ceph-mon[32752]: pgmap v20358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:40.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:40 smithi167 ceph-mon[32751]: pgmap v20358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:41 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37222.561130 secs 2024-04-27T04:55:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:41 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37222.561130 secs 2024-04-27T04:55:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:41 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37222.561130 secs 2024-04-27T04:55:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:42 smithi090 ceph-mon[29180]: pgmap v20359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:42 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37223.455710 secs 2024-04-27T04:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:42 smithi119 ceph-mon[32752]: pgmap v20359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:42 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37223.455710 secs 2024-04-27T04:55:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:42 smithi167 ceph-mon[32751]: pgmap v20359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:42 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37223.455710 secs 2024-04-27T04:55:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:44 smithi090 ceph-mon[29180]: pgmap v20360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:44 smithi119 ceph-mon[32752]: pgmap v20360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:44.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:44 smithi167 ceph-mon[32751]: pgmap v20360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:46 smithi090 ceph-mon[29180]: pgmap v20361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:46.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:46 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37227.561209 secs 2024-04-27T04:55:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:46 smithi119 ceph-mon[32752]: pgmap v20361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:46 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37227.561209 secs 2024-04-27T04:55:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:46 smithi167 ceph-mon[32751]: pgmap v20361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:46 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37227.561209 secs 2024-04-27T04:55:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:47 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37228.455837 secs 2024-04-27T04:55:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:47 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37228.455837 secs 2024-04-27T04:55:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:47 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37228.455837 secs 2024-04-27T04:55:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:48 smithi090 ceph-mon[29180]: pgmap v20362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:48.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:48 smithi119 ceph-mon[32752]: pgmap v20362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:48.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:48 smithi167 ceph-mon[32751]: pgmap v20362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:50 smithi090 ceph-mon[29180]: pgmap v20363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:50.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:50 smithi119 ceph-mon[32752]: pgmap v20363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:50.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:50 smithi167 ceph-mon[32751]: pgmap v20363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:51 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37232.561294 secs 2024-04-27T04:55:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:51 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37232.561294 secs 2024-04-27T04:55:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:51 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37232.561294 secs 2024-04-27T04:55:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:52 smithi090 ceph-mon[29180]: pgmap v20364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:52 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37233.455923 secs 2024-04-27T04:55:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:52 smithi119 ceph-mon[32752]: pgmap v20364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:52.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:52 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37233.455923 secs 2024-04-27T04:55:52.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:52 smithi167 ceph-mon[32751]: pgmap v20364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:52.722 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:52 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37233.455923 secs 2024-04-27T04:55:54.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:54 smithi090 ceph-mon[29180]: pgmap v20365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:54 smithi119 ceph-mon[32752]: pgmap v20365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:54 smithi167 ceph-mon[32751]: pgmap v20365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:56 smithi090 ceph-mon[29180]: pgmap v20366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:56 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37237.561376 secs 2024-04-27T04:55:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:56 smithi119 ceph-mon[32752]: pgmap v20366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:56 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37237.561376 secs 2024-04-27T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:56 smithi167 ceph-mon[32751]: pgmap v20366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:55:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:56 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37237.561376 secs 2024-04-27T04:55:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:57 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37238.456045 secs 2024-04-27T04:55:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:57 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37238.456045 secs 2024-04-27T04:55:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:57 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37238.456045 secs 2024-04-27T04:55:58.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:55:58 smithi119 ceph-mon[32752]: pgmap v20367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:58.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:55:58 smithi167 ceph-mon[32751]: pgmap v20367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:55:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:55:58 smithi090 ceph-mon[29180]: pgmap v20367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:00.744 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:00 smithi090 ceph-mon[29180]: pgmap v20368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:00 smithi119 ceph-mon[32752]: pgmap v20368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:00 smithi167 ceph-mon[32751]: pgmap v20368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:01 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37242.561455 secs 2024-04-27T04:56:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:01 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37242.561455 secs 2024-04-27T04:56:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:01 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37242.561455 secs 2024-04-27T04:56:02.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:02 smithi119 ceph-mon[32752]: pgmap v20369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:02.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:02 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37243.456189 secs 2024-04-27T04:56:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:02 smithi167 ceph-mon[32751]: pgmap v20369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:02 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37243.456189 secs 2024-04-27T04:56:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:02 smithi090 ceph-mon[29180]: pgmap v20369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:02 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37243.456189 secs 2024-04-27T04:56:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:04 smithi119 ceph-mon[32752]: pgmap v20370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:04 smithi167 ceph-mon[32751]: pgmap v20370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:04 smithi090 ceph-mon[29180]: pgmap v20370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:06.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:06 smithi119 ceph-mon[32752]: pgmap v20371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:06.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:06 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37247.561538 secs 2024-04-27T04:56:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:06 smithi167 ceph-mon[32751]: pgmap v20371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:06 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37247.561538 secs 2024-04-27T04:56:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:06 smithi090 ceph-mon[29180]: pgmap v20371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:06 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37247.561538 secs 2024-04-27T04:56:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:07 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37248.456315 secs 2024-04-27T04:56:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:07 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37248.456315 secs 2024-04-27T04:56:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:07 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37248.456315 secs 2024-04-27T04:56:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:08 smithi119 ceph-mon[32752]: pgmap v20372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:08 smithi167 ceph-mon[32751]: pgmap v20372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:08.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:08 smithi090 ceph-mon[29180]: pgmap v20372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:10 smithi119 ceph-mon[32752]: pgmap v20373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:10 smithi167 ceph-mon[32751]: pgmap v20373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:10 smithi090 ceph-mon[29180]: pgmap v20373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:11.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:11 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37252.561618 secs 2024-04-27T04:56:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:11 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37252.561618 secs 2024-04-27T04:56:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:11 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37252.561618 secs 2024-04-27T04:56:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:12 smithi119 ceph-mon[32752]: pgmap v20374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:12 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37253.456435 secs 2024-04-27T04:56:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:12 smithi167 ceph-mon[32751]: pgmap v20374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:12 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37253.456435 secs 2024-04-27T04:56:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:12 smithi090 ceph-mon[29180]: pgmap v20374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:12 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37253.456435 secs 2024-04-27T04:56:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:14 smithi119 ceph-mon[32752]: pgmap v20375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:14 smithi167 ceph-mon[32751]: pgmap v20375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:14.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:14 smithi090 ceph-mon[29180]: pgmap v20375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:16 smithi119 ceph-mon[32752]: pgmap v20376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:16 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37257.561699 secs 2024-04-27T04:56:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:16 smithi167 ceph-mon[32751]: pgmap v20376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:16 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37257.561699 secs 2024-04-27T04:56:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:16 smithi090 ceph-mon[29180]: pgmap v20376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:16 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37257.561699 secs 2024-04-27T04:56:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:17 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37258.456559 secs 2024-04-27T04:56:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:17 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37258.456559 secs 2024-04-27T04:56:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:17 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37258.456559 secs 2024-04-27T04:56:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:18 smithi119 ceph-mon[32752]: pgmap v20377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:18.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:18 smithi167 ceph-mon[32751]: pgmap v20377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:18 smithi090 ceph-mon[29180]: pgmap v20377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:20 smithi119 ceph-mon[32752]: pgmap v20378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:20 smithi167 ceph-mon[32751]: pgmap v20378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:20 smithi090 ceph-mon[29180]: pgmap v20378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:21 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37262.561790 secs 2024-04-27T04:56:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:21 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37262.561790 secs 2024-04-27T04:56:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:21 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37262.561790 secs 2024-04-27T04:56:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:22 smithi119 ceph-mon[32752]: pgmap v20379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:22.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:22 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37263.456676 secs 2024-04-27T04:56:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:22 smithi167 ceph-mon[32751]: pgmap v20379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:22 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37263.456676 secs 2024-04-27T04:56:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:22 smithi090 ceph-mon[29180]: pgmap v20379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:22.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:22 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37263.456676 secs 2024-04-27T04:56:24.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:24 smithi119 ceph-mon[32752]: pgmap v20380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:24 smithi167 ceph-mon[32751]: pgmap v20380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:24 smithi090 ceph-mon[29180]: pgmap v20380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:26 smithi119 ceph-mon[32752]: pgmap v20381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:26 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37267.561861 secs 2024-04-27T04:56:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:26 smithi167 ceph-mon[32751]: pgmap v20381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:26 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37267.561861 secs 2024-04-27T04:56:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:26 smithi090 ceph-mon[29180]: pgmap v20381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:26 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37267.561861 secs 2024-04-27T04:56:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:27 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37268.456798 secs 2024-04-27T04:56:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:56:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:27 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37268.456798 secs 2024-04-27T04:56:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:56:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:27 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37268.456798 secs 2024-04-27T04:56:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:56:28.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:28 smithi090 ceph-mon[29180]: pgmap v20382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:28.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:28 smithi119 ceph-mon[32752]: pgmap v20382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:28.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:28 smithi167 ceph-mon[32751]: pgmap v20382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:30 smithi119 ceph-mon[32752]: pgmap v20383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:30.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:30 smithi167 ceph-mon[32751]: pgmap v20383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:30 smithi090 ceph-mon[29180]: pgmap v20383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:56:30.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:30.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37272.561930 secs 2024-04-27T04:56:31.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:56:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:56:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.572 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.573 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:56:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:56:31.574 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37272.561930 secs 2024-04-27T04:56:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.631 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.633 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:56:31.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:56:31.634 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37272.561930 secs 2024-04-27T04:56:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T04:56:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T04:56:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T04:56:31.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:56:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:56:31.810 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:56:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:56:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:56:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:32 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:56:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:32 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:56:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:32 smithi167 ceph-mon[32751]: pgmap v20384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 B/s rd, 1 op/s 2024-04-27T04:56:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:32 smithi167 ceph-mon[32751]: pgmap v20385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:32.571 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:32 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37273.456912 secs 2024-04-27T04:56:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:56:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:56:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:32 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:56:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:32 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:56:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:32 smithi090 ceph-mon[29180]: pgmap v20384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 B/s rd, 1 op/s 2024-04-27T04:56:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:32 smithi090 ceph-mon[29180]: pgmap v20385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:32.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:32 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37273.456912 secs 2024-04-27T04:56:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T04:56:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T04:56:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:32 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T04:56:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:32 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T04:56:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:32 smithi119 ceph-mon[32752]: pgmap v20384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 855 B/s rd, 1 op/s 2024-04-27T04:56:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:32 smithi119 ceph-mon[32752]: pgmap v20385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:32 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37273.456912 secs 2024-04-27T04:56:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:34 smithi090 ceph-mon[29180]: pgmap v20386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:34 smithi119 ceph-mon[32752]: pgmap v20386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:34 smithi167 ceph-mon[32751]: pgmap v20386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:36 smithi090 ceph-mon[29180]: pgmap v20387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:36 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37277.562010 secs 2024-04-27T04:56:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:36 smithi119 ceph-mon[32752]: pgmap v20387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:36 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37277.562010 secs 2024-04-27T04:56:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:36 smithi167 ceph-mon[32751]: pgmap v20387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:36 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37277.562010 secs 2024-04-27T04:56:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:37 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37278.457040 secs 2024-04-27T04:56:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:37 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37278.457040 secs 2024-04-27T04:56:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:37 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37278.457040 secs 2024-04-27T04:56:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:38 smithi090 ceph-mon[29180]: pgmap v20388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:38 smithi119 ceph-mon[32752]: pgmap v20388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:38 smithi167 ceph-mon[32751]: pgmap v20388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:40.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:40 smithi090 ceph-mon[29180]: pgmap v20389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:40.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:40 smithi119 ceph-mon[32752]: pgmap v20389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:40.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:40 smithi167 ceph-mon[32751]: pgmap v20389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T04:56:41.530 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:41 smithi090 ceph-mon[29180]: 49 slow requests, 0 included below; oldest blocked for > 37282.562070 secs 2024-04-27T04:56:41.530 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:56:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:56:41.153+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:56:41.530 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:56:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:56:41.153+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:56:41.531 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:56:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:56:41.153+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:56:41.531 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:56:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:56:41.153+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:56:41.531 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 04:56:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T04:56:41.153+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T04:56:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:41 smithi119 ceph-mon[32752]: 49 slow requests, 0 included below; oldest blocked for > 37282.562070 secs 2024-04-27T04:56:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:41 smithi167 ceph-mon[32751]: 49 slow requests, 0 included below; oldest blocked for > 37282.562070 secs 2024-04-27T04:56:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:42 smithi090 ceph-mon[29180]: pgmap v20390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:56:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:42 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37283.457194 secs 2024-04-27T04:56:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:42 smithi119 ceph-mon[32752]: pgmap v20390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:56:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:42 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37283.457194 secs 2024-04-27T04:56:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:42 smithi167 ceph-mon[32751]: pgmap v20390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:56:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:42 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37283.457194 secs 2024-04-27T04:56:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:44 smithi090 ceph-mon[29180]: pgmap v20391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:44 smithi119 ceph-mon[32752]: pgmap v20391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:44.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:44 smithi167 ceph-mon[32751]: pgmap v20391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:46 smithi090 ceph-mon[29180]: pgmap v20392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:46.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37287.562171 secs 2024-04-27T04:56:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:46 smithi119 ceph-mon[32752]: pgmap v20392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37287.562171 secs 2024-04-27T04:56:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:46 smithi167 ceph-mon[32751]: pgmap v20392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37287.562171 secs 2024-04-27T04:56:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:47 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37288.457367 secs 2024-04-27T04:56:47.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:56:47.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:56:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:47 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37288.457367 secs 2024-04-27T04:56:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:56:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:56:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:47 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37288.457367 secs 2024-04-27T04:56:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:56:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:56:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:48 smithi090 ceph-mon[29180]: pgmap v20393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:48.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:48 smithi119 ceph-mon[32752]: pgmap v20393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:48.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:48 smithi167 ceph-mon[32751]: pgmap v20393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:50 smithi090 ceph-mon[29180]: pgmap v20394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:50.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:50 smithi119 ceph-mon[32752]: pgmap v20394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:50.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:50 smithi167 ceph-mon[32751]: pgmap v20394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37292.562248 secs 2024-04-27T04:56:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37292.562248 secs 2024-04-27T04:56:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37292.562248 secs 2024-04-27T04:56:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:52 smithi090 ceph-mon[29180]: pgmap v20395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:52 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37293.457437 secs 2024-04-27T04:56:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:52 smithi119 ceph-mon[32752]: pgmap v20395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:52.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:52 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37293.457437 secs 2024-04-27T04:56:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:52 smithi167 ceph-mon[32751]: pgmap v20395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:52 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37293.457437 secs 2024-04-27T04:56:54.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:54 smithi090 ceph-mon[29180]: pgmap v20396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:54 smithi119 ceph-mon[32752]: pgmap v20396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:54 smithi167 ceph-mon[32751]: pgmap v20396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 426 B/s wr, 13 op/s 2024-04-27T04:56:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:56 smithi090 ceph-mon[29180]: pgmap v20397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37297.562326 secs 2024-04-27T04:56:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:56 smithi119 ceph-mon[32752]: pgmap v20397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37297.562326 secs 2024-04-27T04:56:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:56 smithi167 ceph-mon[32751]: pgmap v20397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 426 B/s wr, 14 op/s 2024-04-27T04:56:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37297.562326 secs 2024-04-27T04:56:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:57 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37298.457548 secs 2024-04-27T04:56:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:57 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37298.457548 secs 2024-04-27T04:56:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:57 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37298.457548 secs 2024-04-27T04:56:58.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:56:58 smithi119 ceph-mon[32752]: pgmap v20398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:58.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:56:58 smithi167 ceph-mon[32751]: pgmap v20398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:56:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:56:58 smithi090 ceph-mon[29180]: pgmap v20398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:00 smithi119 ceph-mon[32752]: pgmap v20399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:00 smithi167 ceph-mon[32751]: pgmap v20399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:00 smithi090 ceph-mon[29180]: pgmap v20399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:01 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37302.562408 secs 2024-04-27T04:57:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:01 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37302.562408 secs 2024-04-27T04:57:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:01 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37302.562408 secs 2024-04-27T04:57:02.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:02 smithi090 ceph-mon[29180]: pgmap v20400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:57:02.696 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:02 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37303.457632 secs 2024-04-27T04:57:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:02 smithi119 ceph-mon[32752]: pgmap v20400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:57:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:02 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37303.457632 secs 2024-04-27T04:57:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:02 smithi167 ceph-mon[32751]: pgmap v20400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:57:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:02 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37303.457632 secs 2024-04-27T04:57:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:04 smithi119 ceph-mon[32752]: pgmap v20401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:57:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:04 smithi167 ceph-mon[32751]: pgmap v20401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:57:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:04 smithi090 ceph-mon[29180]: pgmap v20401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T04:57:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:06 smithi119 ceph-mon[32752]: pgmap v20402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37307.562490 secs 2024-04-27T04:57:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:06 smithi167 ceph-mon[32751]: pgmap v20402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37307.562490 secs 2024-04-27T04:57:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:06 smithi090 ceph-mon[29180]: pgmap v20402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37307.562490 secs 2024-04-27T04:57:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:07 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37308.457711 secs 2024-04-27T04:57:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:07 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37308.457711 secs 2024-04-27T04:57:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:07 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37308.457711 secs 2024-04-27T04:57:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:08 smithi119 ceph-mon[32752]: pgmap v20403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:08 smithi167 ceph-mon[32751]: pgmap v20403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:08.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:08 smithi090 ceph-mon[29180]: pgmap v20403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:10 smithi119 ceph-mon[32752]: pgmap v20404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:10 smithi167 ceph-mon[32751]: pgmap v20404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:10 smithi090 ceph-mon[29180]: pgmap v20404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:11.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37312.562574 secs 2024-04-27T04:57:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37312.562574 secs 2024-04-27T04:57:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37312.562574 secs 2024-04-27T04:57:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:12 smithi119 ceph-mon[32752]: pgmap v20405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:12 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37313.457847 secs 2024-04-27T04:57:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:12 smithi167 ceph-mon[32751]: pgmap v20405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:12 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37313.457847 secs 2024-04-27T04:57:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:12 smithi090 ceph-mon[29180]: pgmap v20405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:12 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37313.457847 secs 2024-04-27T04:57:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:14 smithi119 ceph-mon[32752]: pgmap v20406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:14 smithi167 ceph-mon[32751]: pgmap v20406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:14 smithi090 ceph-mon[29180]: pgmap v20406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T04:57:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:16 smithi119 ceph-mon[32752]: pgmap v20407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37317.562650 secs 2024-04-27T04:57:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:16 smithi167 ceph-mon[32751]: pgmap v20407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37317.562650 secs 2024-04-27T04:57:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:16 smithi090 ceph-mon[29180]: pgmap v20407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T04:57:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37317.562650 secs 2024-04-27T04:57:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:17 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37318.457933 secs 2024-04-27T04:57:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:17 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37318.457933 secs 2024-04-27T04:57:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:17 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37318.457933 secs 2024-04-27T04:57:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:18 smithi119 ceph-mon[32752]: pgmap v20408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:18.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:18 smithi167 ceph-mon[32751]: pgmap v20408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:18 smithi090 ceph-mon[29180]: pgmap v20408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:20 smithi119 ceph-mon[32752]: pgmap v20409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:20 smithi167 ceph-mon[32751]: pgmap v20409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:20 smithi090 ceph-mon[29180]: pgmap v20409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:21 smithi119 ceph-mon[32752]: 25 slow requests, 1 included below; oldest blocked for > 37322.562731 secs 2024-04-27T04:57:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:21 smithi119 ceph-mon[32752]: slow request 361.443770 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:57:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:21 smithi167 ceph-mon[32751]: 25 slow requests, 1 included below; oldest blocked for > 37322.562731 secs 2024-04-27T04:57:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:21 smithi167 ceph-mon[32751]: slow request 361.443770 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:57:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:21 smithi090 ceph-mon[29180]: 25 slow requests, 1 included below; oldest blocked for > 37322.562731 secs 2024-04-27T04:57:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:21 smithi090 ceph-mon[29180]: slow request 361.443770 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:57:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:22 smithi119 ceph-mon[32752]: pgmap v20410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:22.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:22 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37323.458053 secs 2024-04-27T04:57:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:22 smithi167 ceph-mon[32751]: pgmap v20410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:22 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37323.458053 secs 2024-04-27T04:57:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:22 smithi090 ceph-mon[29180]: pgmap v20410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:22.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:22 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37323.458053 secs 2024-04-27T04:57:24.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:24 smithi119 ceph-mon[32752]: pgmap v20411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:24 smithi167 ceph-mon[32751]: pgmap v20411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:24 smithi090 ceph-mon[29180]: pgmap v20411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:26 smithi119 ceph-mon[32752]: pgmap v20412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37327.562810 secs 2024-04-27T04:57:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:26 smithi167 ceph-mon[32751]: pgmap v20412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37327.562810 secs 2024-04-27T04:57:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:26 smithi090 ceph-mon[29180]: pgmap v20412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37327.562810 secs 2024-04-27T04:57:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:27 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37328.458181 secs 2024-04-27T04:57:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:27 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37328.458181 secs 2024-04-27T04:57:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:27 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37328.458181 secs 2024-04-27T04:57:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:28 smithi119 ceph-mon[32752]: pgmap v20413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:28 smithi167 ceph-mon[32751]: pgmap v20413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:28 smithi090 ceph-mon[29180]: pgmap v20413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:30 smithi119 ceph-mon[32752]: pgmap v20414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:30.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:30 smithi167 ceph-mon[32751]: pgmap v20414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:30 smithi090 ceph-mon[29180]: pgmap v20414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37332.562870 secs 2024-04-27T04:57:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:57:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37332.562870 secs 2024-04-27T04:57:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:57:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37332.562870 secs 2024-04-27T04:57:31.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:57:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:32 smithi119 ceph-mon[32752]: pgmap v20415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:32 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37333.458275 secs 2024-04-27T04:57:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:57:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:57:32.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:32 smithi167 ceph-mon[32751]: pgmap v20415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:32 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37333.458275 secs 2024-04-27T04:57:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:57:32.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:57:32.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:32 smithi090 ceph-mon[29180]: pgmap v20415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:32 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37333.458275 secs 2024-04-27T04:57:32.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:32.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:57:32.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:57:32.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:57:33.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:33 smithi090 ceph-mon[29180]: pgmap v20416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T04:57:33.780 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:33 smithi090 ceph-mon[29180]: pgmap v20417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:33 smithi119 ceph-mon[32752]: pgmap v20416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T04:57:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:33 smithi119 ceph-mon[32752]: pgmap v20417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:33.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:33 smithi167 ceph-mon[32751]: pgmap v20416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 910 B/s rd, 1 op/s 2024-04-27T04:57:33.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:33 smithi167 ceph-mon[32751]: pgmap v20417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:35 smithi119 ceph-mon[32752]: pgmap v20418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:35 smithi167 ceph-mon[32751]: pgmap v20418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:35 smithi090 ceph-mon[29180]: pgmap v20418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37337.562955 secs 2024-04-27T04:57:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37337.562955 secs 2024-04-27T04:57:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37337.562955 secs 2024-04-27T04:57:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:37 smithi119 ceph-mon[32752]: pgmap v20419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:37.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:37 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37338.458433 secs 2024-04-27T04:57:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:37 smithi167 ceph-mon[32751]: pgmap v20419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:37 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37338.458433 secs 2024-04-27T04:57:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:37 smithi090 ceph-mon[29180]: pgmap v20419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:37 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37338.458433 secs 2024-04-27T04:57:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:39 smithi119 ceph-mon[32752]: pgmap v20420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T04:57:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:39 smithi167 ceph-mon[32751]: pgmap v20420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T04:57:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:39 smithi090 ceph-mon[29180]: pgmap v20420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 B/s rd, 1 op/s 2024-04-27T04:57:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:41 smithi119 ceph-mon[32752]: pgmap v20421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:41 smithi119 ceph-mon[32752]: 25 slow requests, 1 included below; oldest blocked for > 37342.563042 secs 2024-04-27T04:57:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:41 smithi119 ceph-mon[32752]: slow request 2880.009646 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:57:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:41 smithi167 ceph-mon[32751]: pgmap v20421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:41 smithi167 ceph-mon[32751]: 25 slow requests, 1 included below; oldest blocked for > 37342.563042 secs 2024-04-27T04:57:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:41 smithi167 ceph-mon[32751]: slow request 2880.009646 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:57:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:41 smithi090 ceph-mon[29180]: pgmap v20421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:57:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:41 smithi090 ceph-mon[29180]: 25 slow requests, 1 included below; oldest blocked for > 37342.563042 secs 2024-04-27T04:57:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:41 smithi090 ceph-mon[29180]: slow request 2880.009646 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:57:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:42 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 37343.458559 secs 2024-04-27T04:57:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:42 smithi119 ceph-mon[32752]: slow request 360.174637 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:57:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:42 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 37343.458559 secs 2024-04-27T04:57:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:42 smithi167 ceph-mon[32751]: slow request 360.174637 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:57:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:42 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 37343.458559 secs 2024-04-27T04:57:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:42 smithi090 ceph-mon[29180]: slow request 360.174637 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:57:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:43 smithi119 ceph-mon[32752]: pgmap v20422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:57:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:43 smithi167 ceph-mon[32751]: pgmap v20422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:57:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:43 smithi090 ceph-mon[29180]: pgmap v20422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:57:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:45 smithi119 ceph-mon[32752]: pgmap v20423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:45 smithi167 ceph-mon[32751]: pgmap v20423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:45 smithi090 ceph-mon[29180]: pgmap v20423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37347.563123 secs 2024-04-27T04:57:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37347.563123 secs 2024-04-27T04:57:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37347.563123 secs 2024-04-27T04:57:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:47 smithi119 ceph-mon[32752]: pgmap v20424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:47 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37348.458658 secs 2024-04-27T04:57:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:47 smithi167 ceph-mon[32751]: pgmap v20424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:47 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37348.458658 secs 2024-04-27T04:57:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:47 smithi090 ceph-mon[29180]: pgmap v20424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:47 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37348.458658 secs 2024-04-27T04:57:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:49 smithi119 ceph-mon[32752]: pgmap v20425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:49 smithi167 ceph-mon[32751]: pgmap v20425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:49 smithi090 ceph-mon[29180]: pgmap v20425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:51 smithi119 ceph-mon[32752]: pgmap v20426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37352.563183 secs 2024-04-27T04:57:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:51 smithi167 ceph-mon[32751]: pgmap v20426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37352.563183 secs 2024-04-27T04:57:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:51 smithi090 ceph-mon[29180]: pgmap v20426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37352.563183 secs 2024-04-27T04:57:52.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:52 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37353.458778 secs 2024-04-27T04:57:52.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:52 smithi167 ceph-mon[32751]: pgmap v20427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:52 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37353.458778 secs 2024-04-27T04:57:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:52 smithi090 ceph-mon[29180]: pgmap v20427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:52 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37353.458778 secs 2024-04-27T04:57:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:52 smithi119 ceph-mon[32752]: pgmap v20427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:55.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:55 smithi119 ceph-mon[32752]: pgmap v20428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:55.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:55 smithi167 ceph-mon[32751]: pgmap v20428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:55.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:55 smithi090 ceph-mon[29180]: pgmap v20428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:57:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37357.563263 secs 2024-04-27T04:57:56.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37357.563263 secs 2024-04-27T04:57:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37357.563263 secs 2024-04-27T04:57:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:57 smithi119 ceph-mon[32752]: pgmap v20429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:57.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:57 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37358.458896 secs 2024-04-27T04:57:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:57 smithi167 ceph-mon[32751]: pgmap v20429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:57 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37358.458896 secs 2024-04-27T04:57:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:57 smithi090 ceph-mon[29180]: pgmap v20429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:57.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:57 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37358.458896 secs 2024-04-27T04:57:59.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:57:59 smithi119 ceph-mon[32752]: pgmap v20430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:59.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:57:59 smithi167 ceph-mon[32751]: pgmap v20430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:57:59.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:57:59 smithi090 ceph-mon[29180]: pgmap v20430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:01 smithi119 ceph-mon[32752]: pgmap v20431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:01 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37362.563349 secs 2024-04-27T04:58:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:01 smithi167 ceph-mon[32751]: pgmap v20431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:01 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37362.563349 secs 2024-04-27T04:58:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:01 smithi090 ceph-mon[29180]: pgmap v20431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:01 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37362.563349 secs 2024-04-27T04:58:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:02 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37363.458983 secs 2024-04-27T04:58:02.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:02 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37363.458983 secs 2024-04-27T04:58:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:02 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37363.458983 secs 2024-04-27T04:58:03.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:03 smithi119 ceph-mon[32752]: pgmap v20432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:03.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:03 smithi167 ceph-mon[32751]: pgmap v20432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:03.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:03 smithi090 ceph-mon[29180]: pgmap v20432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:05.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:05 smithi119 ceph-mon[32752]: pgmap v20433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:05.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:05 smithi167 ceph-mon[32751]: pgmap v20433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:05.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:05 smithi090 ceph-mon[29180]: pgmap v20433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37367.563417 secs 2024-04-27T04:58:06.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37367.563417 secs 2024-04-27T04:58:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37367.563417 secs 2024-04-27T04:58:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:07 smithi119 ceph-mon[32752]: pgmap v20434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:07 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37368.459108 secs 2024-04-27T04:58:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:07 smithi167 ceph-mon[32751]: pgmap v20434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:07.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:07 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37368.459108 secs 2024-04-27T04:58:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:07 smithi090 ceph-mon[29180]: pgmap v20434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:07.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:07 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37368.459108 secs 2024-04-27T04:58:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:09 smithi119 ceph-mon[32752]: pgmap v20435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:09.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:09 smithi167 ceph-mon[32751]: pgmap v20435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:09 smithi090 ceph-mon[29180]: pgmap v20435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:11 smithi119 ceph-mon[32752]: pgmap v20436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:11.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37372.563502 secs 2024-04-27T04:58:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:11 smithi167 ceph-mon[32751]: pgmap v20436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37372.563502 secs 2024-04-27T04:58:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:11 smithi090 ceph-mon[29180]: pgmap v20436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37372.563502 secs 2024-04-27T04:58:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:12 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37373.459230 secs 2024-04-27T04:58:12.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:12 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37373.459230 secs 2024-04-27T04:58:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:12 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37373.459230 secs 2024-04-27T04:58:13.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:13 smithi119 ceph-mon[32752]: pgmap v20437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:13 smithi167 ceph-mon[32751]: pgmap v20437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:13 smithi090 ceph-mon[29180]: pgmap v20437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:15.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:15 smithi119 ceph-mon[32752]: pgmap v20438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:15.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:15 smithi167 ceph-mon[32751]: pgmap v20438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:15 smithi090 ceph-mon[29180]: pgmap v20438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37377.563586 secs 2024-04-27T04:58:16.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37377.563586 secs 2024-04-27T04:58:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37377.563586 secs 2024-04-27T04:58:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:17 smithi119 ceph-mon[32752]: pgmap v20439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:17 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37378.459312 secs 2024-04-27T04:58:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:17 smithi167 ceph-mon[32751]: pgmap v20439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:17 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37378.459312 secs 2024-04-27T04:58:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:17 smithi090 ceph-mon[29180]: pgmap v20439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:17 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37378.459312 secs 2024-04-27T04:58:19.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:19 smithi119 ceph-mon[32752]: pgmap v20440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:19.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:19 smithi167 ceph-mon[32751]: pgmap v20440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:19 smithi090 ceph-mon[29180]: pgmap v20440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:21 smithi119 ceph-mon[32752]: pgmap v20441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37382.563672 secs 2024-04-27T04:58:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:21 smithi167 ceph-mon[32751]: pgmap v20441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37382.563672 secs 2024-04-27T04:58:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:21 smithi090 ceph-mon[29180]: pgmap v20441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37382.563672 secs 2024-04-27T04:58:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:22 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37383.459432 secs 2024-04-27T04:58:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:22 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37383.459432 secs 2024-04-27T04:58:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:22 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37383.459432 secs 2024-04-27T04:58:23.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:23 smithi119 ceph-mon[32752]: pgmap v20442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:23.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:23 smithi167 ceph-mon[32751]: pgmap v20442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:23 smithi090 ceph-mon[29180]: pgmap v20442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:25.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:25 smithi119 ceph-mon[32752]: pgmap v20443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:25.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:25 smithi167 ceph-mon[32751]: pgmap v20443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:25.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:25 smithi090 ceph-mon[29180]: pgmap v20443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:26.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37387.563755 secs 2024-04-27T04:58:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37387.563755 secs 2024-04-27T04:58:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37387.563755 secs 2024-04-27T04:58:27.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:27 smithi119 ceph-mon[32752]: pgmap v20444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:27.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:27 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37388.459560 secs 2024-04-27T04:58:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:27 smithi167 ceph-mon[32751]: pgmap v20444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:27 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37388.459560 secs 2024-04-27T04:58:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:27 smithi090 ceph-mon[29180]: pgmap v20444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:27 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37388.459560 secs 2024-04-27T04:58:29.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:29 smithi119 ceph-mon[32752]: pgmap v20445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:29.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:29 smithi167 ceph-mon[32751]: pgmap v20445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:29.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:29 smithi090 ceph-mon[29180]: pgmap v20445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:31 smithi119 ceph-mon[32752]: pgmap v20446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37392.563843 secs 2024-04-27T04:58:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:31 smithi167 ceph-mon[32751]: pgmap v20446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:31.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37392.563843 secs 2024-04-27T04:58:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:31 smithi090 ceph-mon[29180]: pgmap v20446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37392.563843 secs 2024-04-27T04:58:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:32 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37393.459671 secs 2024-04-27T04:58:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:32 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:58:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:32 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37393.459671 secs 2024-04-27T04:58:32.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:32 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:58:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:32 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37393.459671 secs 2024-04-27T04:58:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:32 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:58:33.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:33 smithi167 ceph-mon[32751]: pgmap v20447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:33.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:58:33.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:58:33.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:58:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:33 smithi090 ceph-mon[29180]: pgmap v20447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:58:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:58:33.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:58:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:33 smithi119 ceph-mon[32752]: pgmap v20447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:58:33.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:58:33.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:58:34.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:34 smithi167 ceph-mon[32751]: pgmap v20448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:58:34.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:34 smithi167 ceph-mon[32751]: pgmap v20449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:58:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:34 smithi090 ceph-mon[29180]: pgmap v20448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:58:34.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:34 smithi090 ceph-mon[29180]: pgmap v20449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:58:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:34 smithi119 ceph-mon[32752]: pgmap v20448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T04:58:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:34 smithi119 ceph-mon[32752]: pgmap v20449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:58:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:36 smithi167 ceph-mon[32751]: pgmap v20450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:36.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37397.563917 secs 2024-04-27T04:58:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:36 smithi090 ceph-mon[29180]: pgmap v20450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37397.563917 secs 2024-04-27T04:58:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:36 smithi119 ceph-mon[32752]: pgmap v20450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37397.563917 secs 2024-04-27T04:58:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:37 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37398.459749 secs 2024-04-27T04:58:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:37 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37398.459749 secs 2024-04-27T04:58:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:37 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37398.459749 secs 2024-04-27T04:58:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:38 smithi090 ceph-mon[29180]: pgmap v20451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:38 smithi119 ceph-mon[32752]: pgmap v20451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:38.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:38 smithi167 ceph-mon[32751]: pgmap v20451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:40.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:40 smithi090 ceph-mon[29180]: pgmap v20452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:40.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:40 smithi119 ceph-mon[32752]: pgmap v20452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:40.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:40 smithi167 ceph-mon[32751]: pgmap v20452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37402.564002 secs 2024-04-27T04:58:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37402.564002 secs 2024-04-27T04:58:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37402.564002 secs 2024-04-27T04:58:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:42 smithi090 ceph-mon[29180]: pgmap v20453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:42 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 37403.459877 secs 2024-04-27T04:58:42.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:42 smithi090 ceph-mon[29180]: slow request 720.231176 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:58:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:42 smithi119 ceph-mon[32752]: pgmap v20453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:42 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 37403.459877 secs 2024-04-27T04:58:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:42 smithi119 ceph-mon[32752]: slow request 720.231176 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:58:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:42 smithi167 ceph-mon[32751]: pgmap v20453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:58:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:42 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 37403.459877 secs 2024-04-27T04:58:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:42 smithi167 ceph-mon[32751]: slow request 720.231176 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:58:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:44 smithi090 ceph-mon[29180]: pgmap v20454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:58:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:44 smithi119 ceph-mon[32752]: pgmap v20454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:58:44.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:44 smithi167 ceph-mon[32751]: pgmap v20454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:58:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:46 smithi090 ceph-mon[29180]: pgmap v20455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:46.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:46 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37407.564106 secs 2024-04-27T04:58:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:46 smithi119 ceph-mon[32752]: pgmap v20455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:46 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37407.564106 secs 2024-04-27T04:58:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:46 smithi167 ceph-mon[32751]: pgmap v20455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:46 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37407.564106 secs 2024-04-27T04:58:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:47 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37408.459937 secs 2024-04-27T04:58:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:47 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37408.459937 secs 2024-04-27T04:58:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:47 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37408.459937 secs 2024-04-27T04:58:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:48 smithi090 ceph-mon[29180]: pgmap v20456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:48.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:48 smithi119 ceph-mon[32752]: pgmap v20456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:48.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:48 smithi167 ceph-mon[32751]: pgmap v20456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:50.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:50 smithi090 ceph-mon[29180]: pgmap v20457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:50.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:50 smithi119 ceph-mon[32752]: pgmap v20457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:50.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:50 smithi167 ceph-mon[32751]: pgmap v20457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:51 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37412.564194 secs 2024-04-27T04:58:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:51 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37412.564194 secs 2024-04-27T04:58:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:51 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37412.564194 secs 2024-04-27T04:58:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:52 smithi090 ceph-mon[29180]: pgmap v20458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:52.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:52 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37413.460063 secs 2024-04-27T04:58:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:52 smithi119 ceph-mon[32752]: pgmap v20458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:52.708 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:52 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37413.460063 secs 2024-04-27T04:58:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:52 smithi167 ceph-mon[32751]: pgmap v20458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:52 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37413.460063 secs 2024-04-27T04:58:54.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:54 smithi090 ceph-mon[29180]: pgmap v20459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:54 smithi119 ceph-mon[32752]: pgmap v20459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:54 smithi167 ceph-mon[32751]: pgmap v20459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:56 smithi090 ceph-mon[29180]: pgmap v20460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:56.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:56 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37417.564280 secs 2024-04-27T04:58:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:56 smithi119 ceph-mon[32752]: pgmap v20460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:56 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37417.564280 secs 2024-04-27T04:58:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:56 smithi167 ceph-mon[32751]: pgmap v20460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:58:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:56 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37417.564280 secs 2024-04-27T04:58:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:57 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37418.460184 secs 2024-04-27T04:58:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:57 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37418.460184 secs 2024-04-27T04:58:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:57 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37418.460184 secs 2024-04-27T04:58:58.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:58:58 smithi090 ceph-mon[29180]: pgmap v20461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:58.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:58:58 smithi119 ceph-mon[32752]: pgmap v20461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:58:58.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:58:58 smithi167 ceph-mon[32751]: pgmap v20461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:00 smithi119 ceph-mon[32752]: pgmap v20462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:00 smithi167 ceph-mon[32751]: pgmap v20462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:00 smithi090 ceph-mon[29180]: pgmap v20462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:01 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37422.564371 secs 2024-04-27T04:59:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:01 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37422.564371 secs 2024-04-27T04:59:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:01 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37422.564371 secs 2024-04-27T04:59:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:02 smithi119 ceph-mon[32752]: pgmap v20463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:02 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37423.460317 secs 2024-04-27T04:59:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:02 smithi167 ceph-mon[32751]: pgmap v20463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:02 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37423.460317 secs 2024-04-27T04:59:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:02 smithi090 ceph-mon[29180]: pgmap v20463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:02 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37423.460317 secs 2024-04-27T04:59:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:04 smithi119 ceph-mon[32752]: pgmap v20464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:04 smithi167 ceph-mon[32751]: pgmap v20464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:04 smithi090 ceph-mon[29180]: pgmap v20464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:06 smithi119 ceph-mon[32752]: pgmap v20465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:06 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37427.564456 secs 2024-04-27T04:59:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:06 smithi167 ceph-mon[32751]: pgmap v20465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:06 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37427.564456 secs 2024-04-27T04:59:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:06 smithi090 ceph-mon[29180]: pgmap v20465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:06 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37427.564456 secs 2024-04-27T04:59:07.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:07 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37428.460468 secs 2024-04-27T04:59:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:07 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37428.460468 secs 2024-04-27T04:59:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:07 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37428.460468 secs 2024-04-27T04:59:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:08 smithi119 ceph-mon[32752]: pgmap v20466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:08.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:08 smithi167 ceph-mon[32751]: pgmap v20466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:08.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:08 smithi090 ceph-mon[29180]: pgmap v20466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:10 smithi119 ceph-mon[32752]: pgmap v20467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:10.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:10 smithi167 ceph-mon[32751]: pgmap v20467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:10 smithi090 ceph-mon[29180]: pgmap v20467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:11.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:11 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37432.564536 secs 2024-04-27T04:59:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:11 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37432.564536 secs 2024-04-27T04:59:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:11 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37432.564536 secs 2024-04-27T04:59:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:12 smithi119 ceph-mon[32752]: pgmap v20468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:12 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 37433.460600 secs 2024-04-27T04:59:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:12 smithi119 ceph-mon[32752]: slow request 11523.242649 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:59:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:12 smithi167 ceph-mon[32751]: pgmap v20468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:12 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 37433.460600 secs 2024-04-27T04:59:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:12 smithi167 ceph-mon[32751]: slow request 11523.242649 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:59:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:12 smithi090 ceph-mon[29180]: pgmap v20468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:12 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 37433.460600 secs 2024-04-27T04:59:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:12 smithi090 ceph-mon[29180]: slow request 11523.242649 seconds old, received at 2024-04-27T01:47:07.968595+0000: client_request(client.17598:1 getattr pAsLsXsFs #0x1 2024-04-27T01:47:07.964340+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T04:59:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:14 smithi119 ceph-mon[32752]: pgmap v20469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:14 smithi167 ceph-mon[32751]: pgmap v20469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:14 smithi090 ceph-mon[29180]: pgmap v20469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:16 smithi119 ceph-mon[32752]: pgmap v20470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:16 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37437.564626 secs 2024-04-27T04:59:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:16 smithi167 ceph-mon[32751]: pgmap v20470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:16 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37437.564626 secs 2024-04-27T04:59:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:16 smithi090 ceph-mon[29180]: pgmap v20470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:16 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37437.564626 secs 2024-04-27T04:59:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:17 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37438.460677 secs 2024-04-27T04:59:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:17 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37438.460677 secs 2024-04-27T04:59:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:17 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37438.460677 secs 2024-04-27T04:59:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:18 smithi119 ceph-mon[32752]: pgmap v20471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:18.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:18 smithi167 ceph-mon[32751]: pgmap v20471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:18 smithi090 ceph-mon[29180]: pgmap v20471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:20 smithi119 ceph-mon[32752]: pgmap v20472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:20.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:20 smithi167 ceph-mon[32751]: pgmap v20472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:20 smithi090 ceph-mon[29180]: pgmap v20472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:21 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37442.564713 secs 2024-04-27T04:59:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:21 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37442.564713 secs 2024-04-27T04:59:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:21 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37442.564713 secs 2024-04-27T04:59:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:22 smithi119 ceph-mon[32752]: pgmap v20473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:22 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37443.460801 secs 2024-04-27T04:59:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:22 smithi167 ceph-mon[32751]: pgmap v20473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:22 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37443.460801 secs 2024-04-27T04:59:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:22 smithi090 ceph-mon[29180]: pgmap v20473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:22.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:22 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37443.460801 secs 2024-04-27T04:59:24.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:24 smithi119 ceph-mon[32752]: pgmap v20474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:24 smithi167 ceph-mon[32751]: pgmap v20474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:24 smithi090 ceph-mon[29180]: pgmap v20474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:26 smithi119 ceph-mon[32752]: pgmap v20475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:26 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37447.564829 secs 2024-04-27T04:59:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:26 smithi167 ceph-mon[32751]: pgmap v20475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:26 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37447.564829 secs 2024-04-27T04:59:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:26 smithi090 ceph-mon[29180]: pgmap v20475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:26 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37447.564829 secs 2024-04-27T04:59:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:27 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37448.460900 secs 2024-04-27T04:59:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:27 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37448.460900 secs 2024-04-27T04:59:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:27 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37448.460900 secs 2024-04-27T04:59:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:28 smithi119 ceph-mon[32752]: pgmap v20476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:28.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:28 smithi167 ceph-mon[32751]: pgmap v20476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:28 smithi090 ceph-mon[29180]: pgmap v20476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:30 smithi119 ceph-mon[32752]: pgmap v20477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:30.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:30 smithi167 ceph-mon[32751]: pgmap v20477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:30 smithi090 ceph-mon[29180]: pgmap v20477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:31 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37452.564927 secs 2024-04-27T04:59:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:31 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37452.564927 secs 2024-04-27T04:59:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:31 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37452.564927 secs 2024-04-27T04:59:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:32 smithi119 ceph-mon[32752]: pgmap v20478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:32 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37453.461031 secs 2024-04-27T04:59:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:32 smithi167 ceph-mon[32751]: pgmap v20478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:32 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37453.461031 secs 2024-04-27T04:59:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:32 smithi090 ceph-mon[29180]: pgmap v20478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:32 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37453.461031 secs 2024-04-27T04:59:33.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:59:33.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:59:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T04:59:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:34 smithi119 ceph-mon[32752]: pgmap v20479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:59:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:59:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:59:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:34 smithi167 ceph-mon[32751]: pgmap v20479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:59:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:59:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:59:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:34 smithi090 ceph-mon[29180]: pgmap v20479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T04:59:34.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T04:59:34.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T04:59:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:35 smithi119 ceph-mon[32752]: pgmap v20480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:59:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:35 smithi119 ceph-mon[32752]: pgmap v20481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:35 smithi167 ceph-mon[32751]: pgmap v20480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:59:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:35 smithi167 ceph-mon[32751]: pgmap v20481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:35 smithi090 ceph-mon[29180]: pgmap v20480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T04:59:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:35 smithi090 ceph-mon[29180]: pgmap v20481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:36 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37457.565010 secs 2024-04-27T04:59:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:36 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37457.565010 secs 2024-04-27T04:59:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:36 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37457.565010 secs 2024-04-27T04:59:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:37 smithi119 ceph-mon[32752]: pgmap v20482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:37 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37458.461126 secs 2024-04-27T04:59:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:37 smithi167 ceph-mon[32751]: pgmap v20482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:37 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37458.461126 secs 2024-04-27T04:59:37.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:37 smithi090 ceph-mon[29180]: pgmap v20482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:37.832 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:37 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37458.461126 secs 2024-04-27T04:59:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:39 smithi119 ceph-mon[32752]: pgmap v20483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:59:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:39 smithi167 ceph-mon[32751]: pgmap v20483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:59:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:39 smithi090 ceph-mon[29180]: pgmap v20483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T04:59:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:41 smithi119 ceph-mon[32752]: pgmap v20484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:41 smithi119 ceph-mon[32752]: 25 slow requests, 0 included below; oldest blocked for > 37462.565070 secs 2024-04-27T04:59:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:41 smithi167 ceph-mon[32751]: pgmap v20484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:41.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:41 smithi167 ceph-mon[32751]: 25 slow requests, 0 included below; oldest blocked for > 37462.565070 secs 2024-04-27T04:59:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:41 smithi090 ceph-mon[29180]: pgmap v20484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:41 smithi090 ceph-mon[29180]: 25 slow requests, 0 included below; oldest blocked for > 37462.565070 secs 2024-04-27T04:59:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:42 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37463.461244 secs 2024-04-27T04:59:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:42 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37463.461244 secs 2024-04-27T04:59:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:42 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37463.461244 secs 2024-04-27T04:59:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:43 smithi119 ceph-mon[32752]: pgmap v20485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:43 smithi167 ceph-mon[32751]: pgmap v20485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:43 smithi090 ceph-mon[29180]: pgmap v20485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T04:59:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:45 smithi119 ceph-mon[32752]: pgmap v20486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:59:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:45 smithi167 ceph-mon[32751]: pgmap v20486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:59:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:45 smithi090 ceph-mon[29180]: pgmap v20486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T04:59:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:46 smithi119 ceph-mon[32752]: 50 slow requests, 2 included below; oldest blocked for > 37467.565127 secs 2024-04-27T04:59:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:46 smithi119 ceph-mon[32752]: slow request 184.159805 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:59:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:46 smithi119 ceph-mon[32752]: slow request 11554.441908 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:59:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:46 smithi167 ceph-mon[32751]: 50 slow requests, 2 included below; oldest blocked for > 37467.565127 secs 2024-04-27T04:59:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:46 smithi167 ceph-mon[32751]: slow request 184.159805 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:59:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:46 smithi167 ceph-mon[32751]: slow request 11554.441908 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:59:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:46 smithi090 ceph-mon[29180]: 50 slow requests, 2 included below; oldest blocked for > 37467.565127 secs 2024-04-27T04:59:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:46 smithi090 ceph-mon[29180]: slow request 184.159805 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T04:59:46.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:46 smithi090 ceph-mon[29180]: slow request 11554.441908 seconds old, received at 2024-04-27T01:47:10.876623+0000: client_request(client.24733:267899 getattr AsXsFs #0x100000001fb 2024-04-27T01:47:10.875666+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T04:59:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:47 smithi119 ceph-mon[32752]: pgmap v20487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:47 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37468.461329 secs 2024-04-27T04:59:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:59:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:59:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:47 smithi167 ceph-mon[32751]: pgmap v20487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:47 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37468.461329 secs 2024-04-27T04:59:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:59:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:59:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:47 smithi090 ceph-mon[29180]: pgmap v20487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:47 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37468.461329 secs 2024-04-27T04:59:47.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T04:59:47.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T04:59:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:49 smithi119 ceph-mon[32752]: pgmap v20488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:49 smithi167 ceph-mon[32751]: pgmap v20488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:49 smithi090 ceph-mon[29180]: pgmap v20488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:51 smithi119 ceph-mon[32752]: pgmap v20489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37472.565209 secs 2024-04-27T04:59:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:51 smithi167 ceph-mon[32751]: pgmap v20489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37472.565209 secs 2024-04-27T04:59:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:51 smithi090 ceph-mon[29180]: pgmap v20489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37472.565209 secs 2024-04-27T04:59:52.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:52 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37473.461472 secs 2024-04-27T04:59:52.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:52 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37473.461472 secs 2024-04-27T04:59:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:52 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37473.461472 secs 2024-04-27T04:59:53.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:53 smithi119 ceph-mon[32752]: pgmap v20490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:53.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:53 smithi167 ceph-mon[32751]: pgmap v20490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:53 smithi090 ceph-mon[29180]: pgmap v20490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T04:59:55.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:55 smithi119 ceph-mon[32752]: pgmap v20491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:55.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:55 smithi167 ceph-mon[32751]: pgmap v20491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:55 smithi090 ceph-mon[29180]: pgmap v20491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T04:59:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37477.565287 secs 2024-04-27T04:59:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37477.565287 secs 2024-04-27T04:59:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37477.565287 secs 2024-04-27T04:59:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:57 smithi119 ceph-mon[32752]: pgmap v20492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T04:59:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:57 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37478.461623 secs 2024-04-27T04:59:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:57 smithi167 ceph-mon[32751]: pgmap v20492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T04:59:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:57 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37478.461623 secs 2024-04-27T04:59:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:57 smithi090 ceph-mon[29180]: pgmap v20492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T04:59:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:57 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37478.461623 secs 2024-04-27T04:59:59.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 04:59:59 smithi119 ceph-mon[32752]: pgmap v20493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T04:59:59.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 04:59:59 smithi167 ceph-mon[32751]: pgmap v20493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T04:59:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 04:59:59 smithi090 ceph-mon[29180]: pgmap v20493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T05:00:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:00:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:00:00.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:00:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:01 smithi119 ceph-mon[32752]: pgmap v20494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37482.565375 secs 2024-04-27T05:00:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:01 smithi167 ceph-mon[32751]: pgmap v20494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37482.565375 secs 2024-04-27T05:00:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:01 smithi090 ceph-mon[29180]: pgmap v20494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37482.565375 secs 2024-04-27T05:00:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:02 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37483.461744 secs 2024-04-27T05:00:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:02 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37483.461744 secs 2024-04-27T05:00:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:02 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37483.461744 secs 2024-04-27T05:00:03.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:03 smithi119 ceph-mon[32752]: pgmap v20495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T05:00:03.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:03 smithi167 ceph-mon[32751]: pgmap v20495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T05:00:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:03 smithi090 ceph-mon[29180]: pgmap v20495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-27T05:00:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:05 smithi119 ceph-mon[32752]: pgmap v20496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:05 smithi167 ceph-mon[32751]: pgmap v20496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:05 smithi090 ceph-mon[29180]: pgmap v20496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37487.565456 secs 2024-04-27T05:00:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37487.565456 secs 2024-04-27T05:00:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37487.565456 secs 2024-04-27T05:00:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:07 smithi119 ceph-mon[32752]: pgmap v20497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:07 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37488.461806 secs 2024-04-27T05:00:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:07 smithi167 ceph-mon[32751]: pgmap v20497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:07 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37488.461806 secs 2024-04-27T05:00:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:07 smithi090 ceph-mon[29180]: pgmap v20497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:07 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37488.461806 secs 2024-04-27T05:00:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:09 smithi167 ceph-mon[32751]: pgmap v20498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:09 smithi090 ceph-mon[29180]: pgmap v20498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:10.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:09 smithi119 ceph-mon[32752]: pgmap v20498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:11 smithi167 ceph-mon[32751]: pgmap v20499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37492.565533 secs 2024-04-27T05:00:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:11 smithi090 ceph-mon[29180]: pgmap v20499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37492.565533 secs 2024-04-27T05:00:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:11 smithi119 ceph-mon[32752]: pgmap v20499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37492.565533 secs 2024-04-27T05:00:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:12 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37493.461928 secs 2024-04-27T05:00:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:12 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37493.461928 secs 2024-04-27T05:00:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:12 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37493.461928 secs 2024-04-27T05:00:13.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:13 smithi167 ceph-mon[32751]: pgmap v20500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:13 smithi090 ceph-mon[29180]: pgmap v20500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:13 smithi119 ceph-mon[32752]: pgmap v20500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:15.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:15 smithi090 ceph-mon[29180]: pgmap v20501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:15 smithi119 ceph-mon[32752]: pgmap v20501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:16.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:15 smithi167 ceph-mon[32751]: pgmap v20501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37497.565591 secs 2024-04-27T05:00:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37497.565591 secs 2024-04-27T05:00:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37497.565591 secs 2024-04-27T05:00:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:17 smithi090 ceph-mon[29180]: pgmap v20502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:17.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:17 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37498.462057 secs 2024-04-27T05:00:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:17 smithi119 ceph-mon[32752]: pgmap v20502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:17 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37498.462057 secs 2024-04-27T05:00:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:17 smithi167 ceph-mon[32751]: pgmap v20502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:18.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:17 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37498.462057 secs 2024-04-27T05:00:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:18 smithi090 ceph-mon[29180]: pgmap v20503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:19.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:18 smithi119 ceph-mon[32752]: pgmap v20503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:19.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:18 smithi167 ceph-mon[32751]: pgmap v20503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:20 smithi119 ceph-mon[32752]: pgmap v20504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:21.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:20 smithi167 ceph-mon[32751]: pgmap v20504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:20 smithi090 ceph-mon[29180]: pgmap v20504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37502.565651 secs 2024-04-27T05:00:22.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37502.565651 secs 2024-04-27T05:00:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37502.565651 secs 2024-04-27T05:00:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:22 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37503.462187 secs 2024-04-27T05:00:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:22 smithi119 ceph-mon[32752]: pgmap v20505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:22 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37503.462187 secs 2024-04-27T05:00:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:22 smithi167 ceph-mon[32751]: pgmap v20505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:22 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37503.462187 secs 2024-04-27T05:00:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:22 smithi090 ceph-mon[29180]: pgmap v20505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:25.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:24 smithi119 ceph-mon[32752]: pgmap v20506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:25.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:24 smithi167 ceph-mon[32751]: pgmap v20506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:24 smithi090 ceph-mon[29180]: pgmap v20506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37507.565726 secs 2024-04-27T05:00:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:26 smithi119 ceph-mon[32752]: pgmap v20507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37507.565726 secs 2024-04-27T05:00:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:26 smithi167 ceph-mon[32751]: pgmap v20507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37507.565726 secs 2024-04-27T05:00:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:26 smithi090 ceph-mon[29180]: pgmap v20507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:27 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37508.462307 secs 2024-04-27T05:00:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:27 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37508.462307 secs 2024-04-27T05:00:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:27 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37508.462307 secs 2024-04-27T05:00:29.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:28 smithi119 ceph-mon[32752]: pgmap v20508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:29.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:28 smithi167 ceph-mon[32751]: pgmap v20508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:29.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:28 smithi090 ceph-mon[29180]: pgmap v20508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:31.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:30 smithi119 ceph-mon[32752]: pgmap v20509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:30 smithi167 ceph-mon[32751]: pgmap v20509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:31.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:30 smithi090 ceph-mon[29180]: pgmap v20509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37512.565829 secs 2024-04-27T05:00:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37512.565829 secs 2024-04-27T05:00:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37512.565829 secs 2024-04-27T05:00:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:32 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37513.462452 secs 2024-04-27T05:00:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:32 smithi119 ceph-mon[32752]: pgmap v20510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:32 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37513.462452 secs 2024-04-27T05:00:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:32 smithi167 ceph-mon[32751]: pgmap v20510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:32 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37513.462452 secs 2024-04-27T05:00:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:32 smithi090 ceph-mon[29180]: pgmap v20510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:35.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:34 smithi167 ceph-mon[32751]: pgmap v20511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:35.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:00:35.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:00:35.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:00:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:34 smithi167 ceph-mon[32751]: pgmap v20512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:00:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:34 smithi167 ceph-mon[32751]: pgmap v20513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:35.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:00:35.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:34 smithi090 ceph-mon[29180]: pgmap v20511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:35.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:00:35.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:00:35.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:00:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:34 smithi090 ceph-mon[29180]: pgmap v20512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:00:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:34 smithi090 ceph-mon[29180]: pgmap v20513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:35.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:00:35.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:34 smithi119 ceph-mon[32752]: pgmap v20511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:35.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:00:35.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:00:35.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:00:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:34 smithi119 ceph-mon[32752]: pgmap v20512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:00:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:34 smithi119 ceph-mon[32752]: pgmap v20513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:35.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:00:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37517.565890 secs 2024-04-27T05:00:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37517.565890 secs 2024-04-27T05:00:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37517.565890 secs 2024-04-27T05:00:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:37 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37518.462578 secs 2024-04-27T05:00:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:37 smithi119 ceph-mon[32752]: pgmap v20514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:37 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37518.462578 secs 2024-04-27T05:00:37.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:37 smithi167 ceph-mon[32751]: pgmap v20514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:37 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37518.462578 secs 2024-04-27T05:00:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:37 smithi090 ceph-mon[29180]: pgmap v20514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:39 smithi119 ceph-mon[32752]: pgmap v20515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:39 smithi167 ceph-mon[32751]: pgmap v20515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:39 smithi090 ceph-mon[29180]: pgmap v20515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37522.565976 secs 2024-04-27T05:00:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:41 smithi119 ceph-mon[32752]: pgmap v20516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37522.565976 secs 2024-04-27T05:00:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:41 smithi167 ceph-mon[32751]: pgmap v20516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37522.565976 secs 2024-04-27T05:00:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:41 smithi090 ceph-mon[29180]: pgmap v20516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:00:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:42 smithi119 ceph-mon[32752]: 34 slow requests, 1 included below; oldest blocked for > 37523.462663 secs 2024-04-27T05:00:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:42 smithi119 ceph-mon[32752]: slow request 1440.421311 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:00:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:42 smithi167 ceph-mon[32751]: 34 slow requests, 1 included below; oldest blocked for > 37523.462663 secs 2024-04-27T05:00:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:42 smithi167 ceph-mon[32751]: slow request 1440.421311 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:00:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:42 smithi090 ceph-mon[29180]: 34 slow requests, 1 included below; oldest blocked for > 37523.462663 secs 2024-04-27T05:00:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:42 smithi090 ceph-mon[29180]: slow request 1440.421311 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:00:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:43 smithi167 ceph-mon[32751]: pgmap v20517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:00:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:43 smithi090 ceph-mon[29180]: pgmap v20517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:00:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:43 smithi119 ceph-mon[32752]: pgmap v20517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:00:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:45 smithi167 ceph-mon[32751]: pgmap v20518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:00:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:45 smithi090 ceph-mon[29180]: pgmap v20518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:00:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:45 smithi119 ceph-mon[32752]: pgmap v20518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:00:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37527.566035 secs 2024-04-27T05:00:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37527.566035 secs 2024-04-27T05:00:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37527.566035 secs 2024-04-27T05:00:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:47 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37528.462738 secs 2024-04-27T05:00:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:47 smithi167 ceph-mon[32751]: pgmap v20519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:47 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37528.462738 secs 2024-04-27T05:00:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:47 smithi090 ceph-mon[29180]: pgmap v20519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:47 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37528.462738 secs 2024-04-27T05:00:48.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:47 smithi119 ceph-mon[32752]: pgmap v20519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:49 smithi167 ceph-mon[32751]: pgmap v20520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:49 smithi090 ceph-mon[29180]: pgmap v20520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:49 smithi119 ceph-mon[32752]: pgmap v20520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37532.566121 secs 2024-04-27T05:00:51.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:51 smithi167 ceph-mon[32751]: pgmap v20521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37532.566121 secs 2024-04-27T05:00:51.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:51 smithi090 ceph-mon[29180]: pgmap v20521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37532.566121 secs 2024-04-27T05:00:52.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:51 smithi119 ceph-mon[32752]: pgmap v20521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:52 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37533.462871 secs 2024-04-27T05:00:52.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:52 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37533.462871 secs 2024-04-27T05:00:53.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:52 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37533.462871 secs 2024-04-27T05:00:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:53 smithi090 ceph-mon[29180]: pgmap v20522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:53.993 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:53 smithi167 ceph-mon[32751]: pgmap v20522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:53 smithi119 ceph-mon[32752]: pgmap v20522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:54 smithi090 ceph-mon[29180]: pgmap v20523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:54.928 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:54 smithi167 ceph-mon[32751]: pgmap v20523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:55.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:54 smithi119 ceph-mon[32752]: pgmap v20523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37537.566204 secs 2024-04-27T05:00:56.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37537.566204 secs 2024-04-27T05:00:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37537.566204 secs 2024-04-27T05:00:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:57 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37538.462982 secs 2024-04-27T05:00:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:57 smithi119 ceph-mon[32752]: pgmap v20524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:57 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37538.462982 secs 2024-04-27T05:00:57.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:57 smithi167 ceph-mon[32751]: pgmap v20524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:57 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37538.462982 secs 2024-04-27T05:00:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:57 smithi090 ceph-mon[29180]: pgmap v20524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:00:59.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:00:59 smithi119 ceph-mon[32752]: pgmap v20525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:59.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:00:59 smithi167 ceph-mon[32751]: pgmap v20525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:00:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:00:59 smithi090 ceph-mon[29180]: pgmap v20525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37542.566288 secs 2024-04-27T05:01:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:01 smithi119 ceph-mon[32752]: pgmap v20526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37542.566288 secs 2024-04-27T05:01:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:01 smithi167 ceph-mon[32751]: pgmap v20526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37542.566288 secs 2024-04-27T05:01:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:01 smithi090 ceph-mon[29180]: pgmap v20526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:02 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37543.463125 secs 2024-04-27T05:01:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:02 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37543.463125 secs 2024-04-27T05:01:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:02 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37543.463125 secs 2024-04-27T05:01:03.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:03 smithi167 ceph-mon[32751]: pgmap v20527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:03 smithi090 ceph-mon[29180]: pgmap v20527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:03 smithi119 ceph-mon[32752]: pgmap v20527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:05.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:05 smithi167 ceph-mon[32751]: pgmap v20528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:05 smithi090 ceph-mon[29180]: pgmap v20528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:06.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:05 smithi119 ceph-mon[32752]: pgmap v20528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37547.566367 secs 2024-04-27T05:01:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37547.566367 secs 2024-04-27T05:01:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37547.566367 secs 2024-04-27T05:01:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:07 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37548.463251 secs 2024-04-27T05:01:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:07 smithi167 ceph-mon[32751]: pgmap v20529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:07 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37548.463251 secs 2024-04-27T05:01:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:07 smithi090 ceph-mon[29180]: pgmap v20529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:07 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37548.463251 secs 2024-04-27T05:01:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:07 smithi119 ceph-mon[32752]: pgmap v20529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:09 smithi090 ceph-mon[29180]: pgmap v20530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:10.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:09 smithi119 ceph-mon[32752]: pgmap v20530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:09 smithi167 ceph-mon[32751]: pgmap v20530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37552.566453 secs 2024-04-27T05:01:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:11 smithi090 ceph-mon[29180]: pgmap v20531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37552.566453 secs 2024-04-27T05:01:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:11 smithi119 ceph-mon[32752]: pgmap v20531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37552.566453 secs 2024-04-27T05:01:12.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:11 smithi167 ceph-mon[32751]: pgmap v20531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:12 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37553.463395 secs 2024-04-27T05:01:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:12 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37553.463395 secs 2024-04-27T05:01:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:12 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37553.463395 secs 2024-04-27T05:01:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:13 smithi090 ceph-mon[29180]: pgmap v20532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:13 smithi119 ceph-mon[32752]: pgmap v20532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:13 smithi167 ceph-mon[32751]: pgmap v20532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:14 smithi090 ceph-mon[29180]: pgmap v20533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:14 smithi119 ceph-mon[32752]: pgmap v20533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:14 smithi167 ceph-mon[32751]: pgmap v20533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37557.566510 secs 2024-04-27T05:01:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37557.566510 secs 2024-04-27T05:01:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37557.566510 secs 2024-04-27T05:01:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:17 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37558.463473 secs 2024-04-27T05:01:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:17 smithi119 ceph-mon[32752]: pgmap v20534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:17 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37558.463473 secs 2024-04-27T05:01:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:17 smithi167 ceph-mon[32751]: pgmap v20534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:17 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37558.463473 secs 2024-04-27T05:01:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:17 smithi090 ceph-mon[29180]: pgmap v20534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:19.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:19 smithi119 ceph-mon[32752]: pgmap v20535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:19.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:19 smithi167 ceph-mon[32751]: pgmap v20535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:19 smithi090 ceph-mon[29180]: pgmap v20535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37562.566593 secs 2024-04-27T05:01:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:21 smithi167 ceph-mon[32751]: pgmap v20536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37562.566593 secs 2024-04-27T05:01:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:21 smithi090 ceph-mon[29180]: pgmap v20536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37562.566593 secs 2024-04-27T05:01:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:21 smithi119 ceph-mon[32752]: pgmap v20536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:22.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:22 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37563.463596 secs 2024-04-27T05:01:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:22 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37563.463596 secs 2024-04-27T05:01:22.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:22 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37563.463596 secs 2024-04-27T05:01:23.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:23 smithi167 ceph-mon[32751]: pgmap v20537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:23 smithi090 ceph-mon[29180]: pgmap v20537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:24.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:23 smithi119 ceph-mon[32752]: pgmap v20537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:25.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:25 smithi167 ceph-mon[32751]: pgmap v20538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:25 smithi090 ceph-mon[29180]: pgmap v20538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:25 smithi119 ceph-mon[32752]: pgmap v20538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37567.566672 secs 2024-04-27T05:01:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37567.566672 secs 2024-04-27T05:01:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37567.566672 secs 2024-04-27T05:01:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:27 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37568.463691 secs 2024-04-27T05:01:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:27 smithi090 ceph-mon[29180]: pgmap v20539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:27 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37568.463691 secs 2024-04-27T05:01:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:27 smithi119 ceph-mon[32752]: pgmap v20539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:27 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37568.463691 secs 2024-04-27T05:01:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:27 smithi167 ceph-mon[32751]: pgmap v20539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:29 smithi090 ceph-mon[29180]: pgmap v20540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:29 smithi119 ceph-mon[32752]: pgmap v20540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:30.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:29 smithi167 ceph-mon[32751]: pgmap v20540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:31.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:30 smithi119 ceph-mon[32752]: pgmap v20541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:31.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:30 smithi167 ceph-mon[32751]: pgmap v20541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:31.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:30 smithi090 ceph-mon[29180]: pgmap v20541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37572.566759 secs 2024-04-27T05:01:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37572.566759 secs 2024-04-27T05:01:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37572.566759 secs 2024-04-27T05:01:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:32 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37573.463765 secs 2024-04-27T05:01:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:32 smithi119 ceph-mon[32752]: pgmap v20542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:32 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37573.463765 secs 2024-04-27T05:01:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:32 smithi167 ceph-mon[32751]: pgmap v20542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:32 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37573.463765 secs 2024-04-27T05:01:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:32 smithi090 ceph-mon[29180]: pgmap v20542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:35 smithi167 ceph-mon[32751]: pgmap v20543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:01:35.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:01:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:01:35.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:01:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:35 smithi090 ceph-mon[29180]: pgmap v20543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:01:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:01:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:01:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:01:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:35 smithi119 ceph-mon[32752]: pgmap v20543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:01:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:01:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:01:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:01:36.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:01:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:36 smithi167 ceph-mon[32751]: pgmap v20544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:01:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:36 smithi167 ceph-mon[32751]: pgmap v20545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37577.566808 secs 2024-04-27T05:01:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:36 smithi090 ceph-mon[29180]: pgmap v20544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:01:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:36 smithi090 ceph-mon[29180]: pgmap v20545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37577.566808 secs 2024-04-27T05:01:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:36 smithi119 ceph-mon[32752]: pgmap v20544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:01:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:36 smithi119 ceph-mon[32752]: pgmap v20545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37577.566808 secs 2024-04-27T05:01:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:37 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37578.463874 secs 2024-04-27T05:01:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:37 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37578.463874 secs 2024-04-27T05:01:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:37 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37578.463874 secs 2024-04-27T05:01:38.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:38 smithi090 ceph-mon[29180]: pgmap v20546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:01:39.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:38 smithi119 ceph-mon[32752]: pgmap v20546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:01:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:38 smithi167 ceph-mon[32751]: pgmap v20546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:01:40.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:40 smithi090 ceph-mon[29180]: pgmap v20547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:41.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:40 smithi119 ceph-mon[32752]: pgmap v20547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:41.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:40 smithi167 ceph-mon[32751]: pgmap v20547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:41.577 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:01:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:01:41.220+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:01:41.578 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:01:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:01:41.220+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:01:41.578 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:01:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:01:41.220+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:01:41.578 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:01:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:01:41.220+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:01:41.578 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:01:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:01:41.220+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:01:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37582.566889 secs 2024-04-27T05:01:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37582.566889 secs 2024-04-27T05:01:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37582.566889 secs 2024-04-27T05:01:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:42 smithi090 ceph-mon[29180]: 34 slow requests, 0 included below; oldest blocked for > 37583.463943 secs 2024-04-27T05:01:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:42 smithi090 ceph-mon[29180]: pgmap v20548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:42 smithi119 ceph-mon[32752]: 34 slow requests, 0 included below; oldest blocked for > 37583.463943 secs 2024-04-27T05:01:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:42 smithi119 ceph-mon[32752]: pgmap v20548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:42 smithi167 ceph-mon[32751]: 34 slow requests, 0 included below; oldest blocked for > 37583.463943 secs 2024-04-27T05:01:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:42 smithi167 ceph-mon[32751]: pgmap v20548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:01:44.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:44 smithi090 ceph-mon[29180]: pgmap v20549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T05:01:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:44 smithi119 ceph-mon[32752]: pgmap v20549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T05:01:45.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:44 smithi167 ceph-mon[32751]: pgmap v20549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 467 B/s wr, 2 op/s 2024-04-27T05:01:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:45 smithi090 ceph-mon[29180]: pgmap v20550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T05:01:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:45 smithi119 ceph-mon[32752]: pgmap v20550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T05:01:46.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:45 smithi167 ceph-mon[32751]: pgmap v20550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 511 B/s wr, 2 op/s 2024-04-27T05:01:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37587.566948 secs 2024-04-27T05:01:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37587.566948 secs 2024-04-27T05:01:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37587.566948 secs 2024-04-27T05:01:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37588.464057 secs 2024-04-27T05:01:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:47 smithi090 ceph-mon[29180]: pgmap v20551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37588.464057 secs 2024-04-27T05:01:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:47 smithi119 ceph-mon[32752]: pgmap v20551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37588.464057 secs 2024-04-27T05:01:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:47 smithi167 ceph-mon[32751]: pgmap v20551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:49 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T05:01:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:49 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:01:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:49 smithi090 ceph-mon[29180]: pgmap v20552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:49 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T05:01:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:49 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:01:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:49 smithi119 ceph-mon[32752]: pgmap v20552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:49 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T05:01:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:49 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:01:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:49 smithi167 ceph-mon[32751]: pgmap v20552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:51.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37592.567038 secs 2024-04-27T05:01:51.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37592.567038 secs 2024-04-27T05:01:51.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37592.567038 secs 2024-04-27T05:01:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37593.464141 secs 2024-04-27T05:01:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:52 smithi119 ceph-mon[32752]: pgmap v20553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37593.464141 secs 2024-04-27T05:01:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:52 smithi167 ceph-mon[32751]: pgmap v20553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37593.464141 secs 2024-04-27T05:01:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:52 smithi090 ceph-mon[29180]: pgmap v20553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:54.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:54 smithi090 ceph-mon[29180]: pgmap v20554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:54 smithi119 ceph-mon[32752]: pgmap v20554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:54 smithi167 ceph-mon[32751]: pgmap v20554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-27T05:01:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:56 smithi090 ceph-mon[29180]: pgmap v20555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-27T05:01:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37597.567118 secs 2024-04-27T05:01:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:56 smithi119 ceph-mon[32752]: pgmap v20555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-27T05:01:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37597.567118 secs 2024-04-27T05:01:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:56 smithi167 ceph-mon[32751]: pgmap v20555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-27T05:01:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37597.567118 secs 2024-04-27T05:01:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37598.464253 secs 2024-04-27T05:01:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37598.464253 secs 2024-04-27T05:01:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37598.464253 secs 2024-04-27T05:01:58.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:01:58 smithi090 ceph-mon[29180]: pgmap v20556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:58.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:01:58 smithi119 ceph-mon[32752]: pgmap v20556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:01:58.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:01:58 smithi167 ceph-mon[32751]: pgmap v20556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:00 smithi090 ceph-mon[29180]: pgmap v20557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:00.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:00 smithi119 ceph-mon[32752]: pgmap v20557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:00.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:00 smithi167 ceph-mon[32751]: pgmap v20557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37602.567202 secs 2024-04-27T05:02:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37602.567202 secs 2024-04-27T05:02:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37602.567202 secs 2024-04-27T05:02:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37603.464339 secs 2024-04-27T05:02:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:02 smithi090 ceph-mon[29180]: pgmap v20558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37603.464339 secs 2024-04-27T05:02:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:02 smithi119 ceph-mon[32752]: pgmap v20558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37603.464339 secs 2024-04-27T05:02:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:02 smithi167 ceph-mon[32751]: pgmap v20558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:04.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:04 smithi090 ceph-mon[29180]: pgmap v20559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:04.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:04 smithi119 ceph-mon[32752]: pgmap v20559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:04 smithi167 ceph-mon[32751]: pgmap v20559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:06 smithi090 ceph-mon[29180]: pgmap v20560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:06.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37607.567280 secs 2024-04-27T05:02:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:06 smithi119 ceph-mon[32752]: pgmap v20560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37607.567280 secs 2024-04-27T05:02:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:06 smithi167 ceph-mon[32751]: pgmap v20560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:06.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37607.567280 secs 2024-04-27T05:02:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37608.464459 secs 2024-04-27T05:02:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37608.464459 secs 2024-04-27T05:02:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37608.464459 secs 2024-04-27T05:02:08.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:08 smithi090 ceph-mon[29180]: pgmap v20561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:08 smithi119 ceph-mon[32752]: pgmap v20561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:08 smithi167 ceph-mon[32751]: pgmap v20561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:10.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:10 smithi090 ceph-mon[29180]: pgmap v20562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:10.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:10 smithi119 ceph-mon[32752]: pgmap v20562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:10 smithi167 ceph-mon[32751]: pgmap v20562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37612.567364 secs 2024-04-27T05:02:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37612.567364 secs 2024-04-27T05:02:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37612.567364 secs 2024-04-27T05:02:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37613.464585 secs 2024-04-27T05:02:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:12 smithi090 ceph-mon[29180]: pgmap v20563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37613.464585 secs 2024-04-27T05:02:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:12 smithi119 ceph-mon[32752]: pgmap v20563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37613.464585 secs 2024-04-27T05:02:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:12 smithi167 ceph-mon[32751]: pgmap v20563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:14 smithi090 ceph-mon[29180]: pgmap v20564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:14.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:14 smithi119 ceph-mon[32752]: pgmap v20564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:14.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:14 smithi167 ceph-mon[32751]: pgmap v20564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:16 smithi119 ceph-mon[32752]: pgmap v20565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37617.567445 secs 2024-04-27T05:02:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:16 smithi167 ceph-mon[32751]: pgmap v20565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37617.567445 secs 2024-04-27T05:02:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:16 smithi090 ceph-mon[29180]: pgmap v20565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37617.567445 secs 2024-04-27T05:02:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37618.464681 secs 2024-04-27T05:02:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37618.464681 secs 2024-04-27T05:02:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37618.464681 secs 2024-04-27T05:02:18.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:18 smithi119 ceph-mon[32752]: pgmap v20566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:18.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:18 smithi167 ceph-mon[32751]: pgmap v20566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:18 smithi090 ceph-mon[29180]: pgmap v20566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:20.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:20 smithi119 ceph-mon[32752]: pgmap v20567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:20 smithi167 ceph-mon[32751]: pgmap v20567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:20 smithi090 ceph-mon[29180]: pgmap v20567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37622.567527 secs 2024-04-27T05:02:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37622.567527 secs 2024-04-27T05:02:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37622.567527 secs 2024-04-27T05:02:22.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37623.464804 secs 2024-04-27T05:02:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:22 smithi119 ceph-mon[32752]: pgmap v20568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37623.464804 secs 2024-04-27T05:02:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:22 smithi167 ceph-mon[32751]: pgmap v20568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37623.464804 secs 2024-04-27T05:02:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:22 smithi090 ceph-mon[29180]: pgmap v20568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:24.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:24 smithi119 ceph-mon[32752]: pgmap v20569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:24.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:24 smithi167 ceph-mon[32751]: pgmap v20569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:24 smithi090 ceph-mon[29180]: pgmap v20569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:26 smithi119 ceph-mon[32752]: pgmap v20570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37627.567605 secs 2024-04-27T05:02:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:26 smithi167 ceph-mon[32751]: pgmap v20570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:26.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37627.567605 secs 2024-04-27T05:02:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:26 smithi090 ceph-mon[29180]: pgmap v20570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37627.567605 secs 2024-04-27T05:02:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37628.464898 secs 2024-04-27T05:02:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37628.464898 secs 2024-04-27T05:02:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37628.464898 secs 2024-04-27T05:02:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:28 smithi119 ceph-mon[32752]: pgmap v20571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:28 smithi167 ceph-mon[32751]: pgmap v20571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:28.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:28 smithi090 ceph-mon[29180]: pgmap v20571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:30.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:30 smithi119 ceph-mon[32752]: pgmap v20572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:30 smithi167 ceph-mon[32751]: pgmap v20572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:30.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:30 smithi090 ceph-mon[29180]: pgmap v20572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37632.567686 secs 2024-04-27T05:02:31.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37632.567686 secs 2024-04-27T05:02:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37632.567686 secs 2024-04-27T05:02:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37633.465018 secs 2024-04-27T05:02:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:32 smithi119 ceph-mon[32752]: pgmap v20573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37633.465018 secs 2024-04-27T05:02:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:32 smithi167 ceph-mon[32751]: pgmap v20573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37633.465018 secs 2024-04-27T05:02:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:32 smithi090 ceph-mon[29180]: pgmap v20573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:34.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:34 smithi119 ceph-mon[32752]: pgmap v20574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:34.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:34 smithi167 ceph-mon[32751]: pgmap v20574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:34.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:34 smithi090 ceph-mon[29180]: pgmap v20574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:35.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:02:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:02:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:02:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:36 smithi119 ceph-mon[32752]: pgmap v20575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37637.567771 secs 2024-04-27T05:02:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:02:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:02:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:02:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:36 smithi167 ceph-mon[32751]: pgmap v20575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37637.567771 secs 2024-04-27T05:02:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:02:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:02:36.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:02:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:36 smithi090 ceph-mon[29180]: pgmap v20575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37637.567771 secs 2024-04-27T05:02:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:02:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:02:36.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:02:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:37 smithi119 ceph-mon[32752]: pgmap v20576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:02:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:37 smithi119 ceph-mon[32752]: pgmap v20577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37638.465154 secs 2024-04-27T05:02:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:37 smithi167 ceph-mon[32751]: pgmap v20576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:02:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:37 smithi167 ceph-mon[32751]: pgmap v20577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:37.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37638.465154 secs 2024-04-27T05:02:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:37 smithi090 ceph-mon[29180]: pgmap v20576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:02:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:37 smithi090 ceph-mon[29180]: pgmap v20577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37638.465154 secs 2024-04-27T05:02:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:39 smithi119 ceph-mon[32752]: pgmap v20578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:02:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:39 smithi167 ceph-mon[32751]: pgmap v20578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:02:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:39 smithi090 ceph-mon[29180]: pgmap v20578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:02:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:41 smithi119 ceph-mon[32752]: pgmap v20579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37642.567856 secs 2024-04-27T05:02:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:41 smithi167 ceph-mon[32751]: pgmap v20579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37642.567856 secs 2024-04-27T05:02:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:41 smithi090 ceph-mon[29180]: pgmap v20579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37642.567856 secs 2024-04-27T05:02:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37643.465270 secs 2024-04-27T05:02:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37643.465270 secs 2024-04-27T05:02:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37643.465270 secs 2024-04-27T05:02:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:43 smithi119 ceph-mon[32752]: pgmap v20580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:02:43.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:43 smithi167 ceph-mon[32751]: pgmap v20580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:02:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:43 smithi090 ceph-mon[29180]: pgmap v20580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:02:45.735 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:45 smithi090 ceph-mon[29180]: pgmap v20581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:45 smithi119 ceph-mon[32752]: pgmap v20581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:45 smithi167 ceph-mon[32751]: pgmap v20581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:02:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:46 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 37647.567944 secs 2024-04-27T05:02:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:46 smithi119 ceph-mon[32752]: slow request 364.162622 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:02:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:46 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 37647.567944 secs 2024-04-27T05:02:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:46 smithi167 ceph-mon[32751]: slow request 364.162622 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:02:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:46 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 37647.567944 secs 2024-04-27T05:02:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:46 smithi090 ceph-mon[29180]: slow request 364.162622 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:02:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:47 smithi119 ceph-mon[32752]: pgmap v20582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:02:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37648.465409 secs 2024-04-27T05:02:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:47 smithi167 ceph-mon[32751]: pgmap v20582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:02:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37648.465409 secs 2024-04-27T05:02:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:47 smithi090 ceph-mon[29180]: pgmap v20582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:02:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37648.465409 secs 2024-04-27T05:02:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:49 smithi119 ceph-mon[32752]: pgmap v20583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:49 smithi167 ceph-mon[32751]: pgmap v20583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:49 smithi090 ceph-mon[29180]: pgmap v20583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:51 smithi119 ceph-mon[32752]: pgmap v20584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37652.568032 secs 2024-04-27T05:02:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:51 smithi167 ceph-mon[32751]: pgmap v20584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:51.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37652.568032 secs 2024-04-27T05:02:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:51 smithi090 ceph-mon[29180]: pgmap v20584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37652.568032 secs 2024-04-27T05:02:52.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37653.465554 secs 2024-04-27T05:02:52.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37653.465554 secs 2024-04-27T05:02:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37653.465554 secs 2024-04-27T05:02:53.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:53 smithi119 ceph-mon[32752]: pgmap v20585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:53 smithi167 ceph-mon[32751]: pgmap v20585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:53 smithi090 ceph-mon[29180]: pgmap v20585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:55.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:55 smithi119 ceph-mon[32752]: pgmap v20586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:55 smithi167 ceph-mon[32751]: pgmap v20586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:55 smithi090 ceph-mon[29180]: pgmap v20586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:02:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37657.568111 secs 2024-04-27T05:02:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37657.568111 secs 2024-04-27T05:02:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37657.568111 secs 2024-04-27T05:02:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:57 smithi119 ceph-mon[32752]: pgmap v20587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37658.465665 secs 2024-04-27T05:02:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:57 smithi167 ceph-mon[32751]: pgmap v20587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37658.465665 secs 2024-04-27T05:02:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:57 smithi090 ceph-mon[29180]: pgmap v20587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37658.465665 secs 2024-04-27T05:02:59.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:02:59 smithi119 ceph-mon[32752]: pgmap v20588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:59.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:02:59 smithi167 ceph-mon[32751]: pgmap v20588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:02:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:02:59 smithi090 ceph-mon[29180]: pgmap v20588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:01 smithi119 ceph-mon[32752]: pgmap v20589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37662.568195 secs 2024-04-27T05:03:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:01 smithi167 ceph-mon[32751]: pgmap v20589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:01.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37662.568195 secs 2024-04-27T05:03:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:01 smithi090 ceph-mon[29180]: pgmap v20589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37662.568195 secs 2024-04-27T05:03:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37663.465781 secs 2024-04-27T05:03:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37663.465781 secs 2024-04-27T05:03:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37663.465781 secs 2024-04-27T05:03:03.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:03 smithi119 ceph-mon[32752]: pgmap v20590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:03 smithi167 ceph-mon[32751]: pgmap v20590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:03 smithi090 ceph-mon[29180]: pgmap v20590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:05.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:05 smithi119 ceph-mon[32752]: pgmap v20591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:05 smithi167 ceph-mon[32751]: pgmap v20591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:05 smithi090 ceph-mon[29180]: pgmap v20591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37667.568278 secs 2024-04-27T05:03:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37667.568278 secs 2024-04-27T05:03:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37667.568278 secs 2024-04-27T05:03:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:07 smithi119 ceph-mon[32752]: pgmap v20592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37668.465894 secs 2024-04-27T05:03:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:07 smithi167 ceph-mon[32751]: pgmap v20592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37668.465894 secs 2024-04-27T05:03:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:07 smithi090 ceph-mon[29180]: pgmap v20592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37668.465894 secs 2024-04-27T05:03:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:09 smithi119 ceph-mon[32752]: pgmap v20593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:09.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:09 smithi167 ceph-mon[32751]: pgmap v20593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:09 smithi090 ceph-mon[29180]: pgmap v20593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:11 smithi119 ceph-mon[32752]: pgmap v20594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37672.568361 secs 2024-04-27T05:03:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:11 smithi167 ceph-mon[32751]: pgmap v20594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37672.568361 secs 2024-04-27T05:03:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:11 smithi090 ceph-mon[29180]: pgmap v20594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:11.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37672.568361 secs 2024-04-27T05:03:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37673.466011 secs 2024-04-27T05:03:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37673.466011 secs 2024-04-27T05:03:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37673.466011 secs 2024-04-27T05:03:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:13 smithi119 ceph-mon[32752]: pgmap v20595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:13 smithi167 ceph-mon[32751]: pgmap v20595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:13 smithi090 ceph-mon[29180]: pgmap v20595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:15.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:15 smithi119 ceph-mon[32752]: pgmap v20596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:15 smithi167 ceph-mon[32751]: pgmap v20596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:15.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:15 smithi090 ceph-mon[29180]: pgmap v20596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:16.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37677.568441 secs 2024-04-27T05:03:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37677.568441 secs 2024-04-27T05:03:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37677.568441 secs 2024-04-27T05:03:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:17 smithi119 ceph-mon[32752]: pgmap v20597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:17 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37678.466133 secs 2024-04-27T05:03:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:17 smithi167 ceph-mon[32751]: pgmap v20597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:17 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37678.466133 secs 2024-04-27T05:03:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:17 smithi090 ceph-mon[29180]: pgmap v20597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:17 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37678.466133 secs 2024-04-27T05:03:19.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:19 smithi119 ceph-mon[32752]: pgmap v20598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:19 smithi167 ceph-mon[32751]: pgmap v20598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:19 smithi090 ceph-mon[29180]: pgmap v20598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:21 smithi119 ceph-mon[32752]: pgmap v20599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:21 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 37682.568524 secs 2024-04-27T05:03:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:21 smithi119 ceph-mon[32752]: slow request 721.449563 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:03:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:21 smithi167 ceph-mon[32751]: pgmap v20599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:21 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 37682.568524 secs 2024-04-27T05:03:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:21 smithi167 ceph-mon[32751]: slow request 721.449563 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:03:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:21 smithi090 ceph-mon[29180]: pgmap v20599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:21 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 37682.568524 secs 2024-04-27T05:03:21.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:21 smithi090 ceph-mon[29180]: slow request 721.449563 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:03:22.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:22 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37683.466250 secs 2024-04-27T05:03:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:22 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37683.466250 secs 2024-04-27T05:03:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:22 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37683.466250 secs 2024-04-27T05:03:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:23 smithi167 ceph-mon[32751]: pgmap v20600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:23 smithi090 ceph-mon[29180]: pgmap v20600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:24.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:23 smithi119 ceph-mon[32752]: pgmap v20600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:25 smithi167 ceph-mon[32751]: pgmap v20601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:25 smithi090 ceph-mon[29180]: pgmap v20601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:26.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:25 smithi119 ceph-mon[32752]: pgmap v20601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37687.568605 secs 2024-04-27T05:03:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37687.568605 secs 2024-04-27T05:03:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37687.568605 secs 2024-04-27T05:03:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:27 smithi090 ceph-mon[29180]: pgmap v20602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:27 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37688.466396 secs 2024-04-27T05:03:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:27 smithi119 ceph-mon[32752]: pgmap v20602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:27 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37688.466396 secs 2024-04-27T05:03:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:27 smithi167 ceph-mon[32751]: pgmap v20602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:28.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:27 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37688.466396 secs 2024-04-27T05:03:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:29 smithi090 ceph-mon[29180]: pgmap v20603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:30.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:29 smithi119 ceph-mon[32752]: pgmap v20603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:29 smithi167 ceph-mon[32751]: pgmap v20603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:31 smithi090 ceph-mon[29180]: pgmap v20604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37692.568690 secs 2024-04-27T05:03:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:31 smithi119 ceph-mon[32752]: pgmap v20604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37692.568690 secs 2024-04-27T05:03:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:31 smithi167 ceph-mon[32751]: pgmap v20604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:32.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37692.568690 secs 2024-04-27T05:03:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:32 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37693.466491 secs 2024-04-27T05:03:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:32 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37693.466491 secs 2024-04-27T05:03:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37693.466491 secs 2024-04-27T05:03:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:33 smithi090 ceph-mon[29180]: pgmap v20605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:33 smithi119 ceph-mon[32752]: pgmap v20605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:33 smithi167 ceph-mon[32751]: pgmap v20605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:35 smithi090 ceph-mon[29180]: pgmap v20606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:35 smithi119 ceph-mon[32752]: pgmap v20606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:35 smithi167 ceph-mon[32751]: pgmap v20606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37697.568773 secs 2024-04-27T05:03:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:03:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37697.568773 secs 2024-04-27T05:03:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:03:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37697.568773 secs 2024-04-27T05:03:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:03:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:37 smithi090 ceph-mon[29180]: pgmap v20607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37698.466560 secs 2024-04-27T05:03:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:03:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:03:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:37 smithi090 ceph-mon[29180]: pgmap v20608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:03:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:37 smithi090 ceph-mon[29180]: pgmap v20609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:37.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:03:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:37 smithi119 ceph-mon[32752]: pgmap v20607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37698.466560 secs 2024-04-27T05:03:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:03:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:03:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:37 smithi119 ceph-mon[32752]: pgmap v20608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:03:38.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:37 smithi119 ceph-mon[32752]: pgmap v20609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:38.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:03:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:37 smithi167 ceph-mon[32751]: pgmap v20607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37698.466560 secs 2024-04-27T05:03:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:03:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:03:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:37 smithi167 ceph-mon[32751]: pgmap v20608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:03:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:37 smithi167 ceph-mon[32751]: pgmap v20609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:03:40.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:39 smithi119 ceph-mon[32752]: pgmap v20610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:40.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:39 smithi167 ceph-mon[32751]: pgmap v20610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:40.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:39 smithi090 ceph-mon[29180]: pgmap v20610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:42.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37702.568857 secs 2024-04-27T05:03:42.180 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:41 smithi090 ceph-mon[29180]: pgmap v20611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37702.568857 secs 2024-04-27T05:03:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:41 smithi119 ceph-mon[32752]: pgmap v20611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37702.568857 secs 2024-04-27T05:03:42.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:41 smithi167 ceph-mon[32751]: pgmap v20611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:42 smithi119 ceph-mon[32752]: 18 slow requests, 1 included below; oldest blocked for > 37703.466678 secs 2024-04-27T05:03:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:42 smithi119 ceph-mon[32752]: slow request 720.182755 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:03:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:42 smithi167 ceph-mon[32751]: 18 slow requests, 1 included below; oldest blocked for > 37703.466678 secs 2024-04-27T05:03:43.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:42 smithi167 ceph-mon[32751]: slow request 720.182755 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:03:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:42 smithi090 ceph-mon[29180]: 18 slow requests, 1 included below; oldest blocked for > 37703.466678 secs 2024-04-27T05:03:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:42 smithi090 ceph-mon[29180]: slow request 720.182755 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:03:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:43 smithi119 ceph-mon[32752]: pgmap v20612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:03:44.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:43 smithi167 ceph-mon[32751]: pgmap v20612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:03:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:43 smithi090 ceph-mon[29180]: pgmap v20612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:03:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:45 smithi119 ceph-mon[32752]: pgmap v20613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:45 smithi167 ceph-mon[32751]: pgmap v20613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:45 smithi090 ceph-mon[29180]: pgmap v20613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:03:47.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37707.568947 secs 2024-04-27T05:03:47.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37707.568947 secs 2024-04-27T05:03:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37707.568947 secs 2024-04-27T05:03:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:47 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37708.466775 secs 2024-04-27T05:03:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:47 smithi090 ceph-mon[29180]: pgmap v20614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:03:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:47 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37708.466775 secs 2024-04-27T05:03:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:47 smithi119 ceph-mon[32752]: pgmap v20614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:03:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:47 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37708.466775 secs 2024-04-27T05:03:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:47 smithi167 ceph-mon[32751]: pgmap v20614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:03:50.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:49 smithi119 ceph-mon[32752]: pgmap v20615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:49 smithi167 ceph-mon[32751]: pgmap v20615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:50.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:49 smithi090 ceph-mon[29180]: pgmap v20615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37712.569031 secs 2024-04-27T05:03:52.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:51 smithi119 ceph-mon[32752]: pgmap v20616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:52.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37712.569031 secs 2024-04-27T05:03:52.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:51 smithi167 ceph-mon[32751]: pgmap v20616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37712.569031 secs 2024-04-27T05:03:52.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:51 smithi090 ceph-mon[29180]: pgmap v20616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:52 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37713.466919 secs 2024-04-27T05:03:53.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:52 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37713.466919 secs 2024-04-27T05:03:53.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:52 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37713.466919 secs 2024-04-27T05:03:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:53 smithi119 ceph-mon[32752]: pgmap v20617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:54.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:53 smithi167 ceph-mon[32751]: pgmap v20617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:54.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:53 smithi090 ceph-mon[29180]: pgmap v20617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:56.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:55 smithi119 ceph-mon[32752]: pgmap v20618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:56.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:55 smithi167 ceph-mon[32751]: pgmap v20618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:56.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:55 smithi090 ceph-mon[29180]: pgmap v20618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:03:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37717.569113 secs 2024-04-27T05:03:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37717.569113 secs 2024-04-27T05:03:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37717.569113 secs 2024-04-27T05:03:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:57 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37718.467044 secs 2024-04-27T05:03:58.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:57 smithi119 ceph-mon[32752]: pgmap v20619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:57 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37718.467044 secs 2024-04-27T05:03:58.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:57 smithi167 ceph-mon[32751]: pgmap v20619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:03:58.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:57 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37718.467044 secs 2024-04-27T05:03:58.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:57 smithi090 ceph-mon[29180]: pgmap v20619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:00.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:03:59 smithi119 ceph-mon[32752]: pgmap v20620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:00.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:03:59 smithi167 ceph-mon[32751]: pgmap v20620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:00.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:03:59 smithi090 ceph-mon[29180]: pgmap v20620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37722.569197 secs 2024-04-27T05:04:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:01 smithi119 ceph-mon[32752]: pgmap v20621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37722.569197 secs 2024-04-27T05:04:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:01 smithi167 ceph-mon[32751]: pgmap v20621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37722.569197 secs 2024-04-27T05:04:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:01 smithi090 ceph-mon[29180]: pgmap v20621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:02 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37723.467152 secs 2024-04-27T05:04:03.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:02 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37723.467152 secs 2024-04-27T05:04:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:02 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37723.467152 secs 2024-04-27T05:04:04.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:03 smithi119 ceph-mon[32752]: pgmap v20622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:04.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:03 smithi167 ceph-mon[32751]: pgmap v20622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:04.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:03 smithi090 ceph-mon[29180]: pgmap v20622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:06.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:05 smithi119 ceph-mon[32752]: pgmap v20623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:05 smithi167 ceph-mon[32751]: pgmap v20623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:06.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:05 smithi090 ceph-mon[29180]: pgmap v20623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37727.569253 secs 2024-04-27T05:04:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37727.569253 secs 2024-04-27T05:04:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37727.569253 secs 2024-04-27T05:04:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:07 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37728.467263 secs 2024-04-27T05:04:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:07 smithi119 ceph-mon[32752]: pgmap v20624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:08.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:07 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37728.467263 secs 2024-04-27T05:04:08.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:07 smithi167 ceph-mon[32751]: pgmap v20624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:08.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:07 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37728.467263 secs 2024-04-27T05:04:08.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:07 smithi090 ceph-mon[29180]: pgmap v20624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:10.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:09 smithi119 ceph-mon[32752]: pgmap v20625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:10.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:09 smithi167 ceph-mon[32751]: pgmap v20625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:10.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:09 smithi090 ceph-mon[29180]: pgmap v20625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37732.569342 secs 2024-04-27T05:04:12.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:11 smithi119 ceph-mon[32752]: pgmap v20626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37732.569342 secs 2024-04-27T05:04:12.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:11 smithi167 ceph-mon[32751]: pgmap v20626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37732.569342 secs 2024-04-27T05:04:12.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:11 smithi090 ceph-mon[29180]: pgmap v20626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:13.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:12 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37733.467399 secs 2024-04-27T05:04:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:12 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37733.467399 secs 2024-04-27T05:04:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:12 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37733.467399 secs 2024-04-27T05:04:14.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:13 smithi119 ceph-mon[32752]: pgmap v20627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:14.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:13 smithi167 ceph-mon[32751]: pgmap v20627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:14.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:13 smithi090 ceph-mon[29180]: pgmap v20627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:16.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:15 smithi119 ceph-mon[32752]: pgmap v20628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:15 smithi167 ceph-mon[32751]: pgmap v20628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:15 smithi090 ceph-mon[29180]: pgmap v20628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37737.569429 secs 2024-04-27T05:04:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:16 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37738.467459 secs 2024-04-27T05:04:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37737.569429 secs 2024-04-27T05:04:17.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:16 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37738.467459 secs 2024-04-27T05:04:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37737.569429 secs 2024-04-27T05:04:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:16 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37738.467459 secs 2024-04-27T05:04:18.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:17 smithi119 ceph-mon[32752]: pgmap v20629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:18.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:17 smithi167 ceph-mon[32751]: pgmap v20629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:18.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:17 smithi090 ceph-mon[29180]: pgmap v20629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:20.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:19 smithi119 ceph-mon[32752]: pgmap v20630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:20.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:19 smithi167 ceph-mon[32751]: pgmap v20630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:20.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:19 smithi090 ceph-mon[29180]: pgmap v20630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37742.569513 secs 2024-04-27T05:04:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:21 smithi119 ceph-mon[32752]: pgmap v20631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:21 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37743.467585 secs 2024-04-27T05:04:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37742.569513 secs 2024-04-27T05:04:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:21 smithi167 ceph-mon[32751]: pgmap v20631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:22.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:21 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37743.467585 secs 2024-04-27T05:04:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37742.569513 secs 2024-04-27T05:04:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:21 smithi090 ceph-mon[29180]: pgmap v20631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:21 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37743.467585 secs 2024-04-27T05:04:24.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:23 smithi119 ceph-mon[32752]: pgmap v20632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:24.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:23 smithi167 ceph-mon[32751]: pgmap v20632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:24.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:23 smithi090 ceph-mon[29180]: pgmap v20632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:25 smithi119 ceph-mon[32752]: pgmap v20633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:25 smithi167 ceph-mon[32751]: pgmap v20633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:25 smithi090 ceph-mon[29180]: pgmap v20633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37747.569592 secs 2024-04-27T05:04:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:26 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37748.467701 secs 2024-04-27T05:04:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37747.569592 secs 2024-04-27T05:04:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:26 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37748.467701 secs 2024-04-27T05:04:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37747.569592 secs 2024-04-27T05:04:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:26 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37748.467701 secs 2024-04-27T05:04:28.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:27 smithi119 ceph-mon[32752]: pgmap v20634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:28.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:27 smithi167 ceph-mon[32751]: pgmap v20634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:28.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:27 smithi090 ceph-mon[29180]: pgmap v20634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:30.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:29 smithi119 ceph-mon[32752]: pgmap v20635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:30.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:29 smithi167 ceph-mon[32751]: pgmap v20635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:30.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:29 smithi090 ceph-mon[29180]: pgmap v20635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37752.569673 secs 2024-04-27T05:04:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:31 smithi119 ceph-mon[32752]: pgmap v20636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:32.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:31 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37753.467802 secs 2024-04-27T05:04:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:32 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37752.569673 secs 2024-04-27T05:04:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:32 smithi167 ceph-mon[32751]: pgmap v20636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:32.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:32 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37753.467802 secs 2024-04-27T05:04:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37752.569673 secs 2024-04-27T05:04:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:31 smithi090 ceph-mon[29180]: pgmap v20636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:31 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37753.467802 secs 2024-04-27T05:04:34.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:34 smithi090 ceph-mon[29180]: pgmap v20637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:34.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:34 smithi119 ceph-mon[32752]: pgmap v20637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:34.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:34 smithi167 ceph-mon[32751]: pgmap v20637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:36 smithi119 ceph-mon[32752]: pgmap v20638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:36.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37757.569752 secs 2024-04-27T05:04:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:36 smithi167 ceph-mon[32751]: pgmap v20638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:36.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37757.569752 secs 2024-04-27T05:04:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:36 smithi090 ceph-mon[29180]: pgmap v20638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37757.569752 secs 2024-04-27T05:04:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:37 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37758.467876 secs 2024-04-27T05:04:37.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:04:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:37 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37758.467876 secs 2024-04-27T05:04:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:04:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:37 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37758.467876 secs 2024-04-27T05:04:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:04:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:38 smithi119 ceph-mon[32752]: pgmap v20639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:04:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:04:38.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:04:38.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:38 smithi167 ceph-mon[32751]: pgmap v20639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:38.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:04:38.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:04:38.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:04:38.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:38 smithi090 ceph-mon[29180]: pgmap v20639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:38.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:04:38.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:04:38.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:04:39.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:39 smithi119 ceph-mon[32752]: pgmap v20640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:04:39.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:39 smithi119 ceph-mon[32752]: pgmap v20641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:04:39.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:39 smithi167 ceph-mon[32751]: pgmap v20640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:04:39.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:39 smithi167 ceph-mon[32751]: pgmap v20641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:04:39.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:39 smithi090 ceph-mon[29180]: pgmap v20640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:04:39.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:39 smithi090 ceph-mon[29180]: pgmap v20641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:04:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:41 smithi119 ceph-mon[32752]: pgmap v20642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:41.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37762.569823 secs 2024-04-27T05:04:41.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:41 smithi167 ceph-mon[32751]: pgmap v20642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:41.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37762.569823 secs 2024-04-27T05:04:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:41 smithi090 ceph-mon[29180]: pgmap v20642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37762.569823 secs 2024-04-27T05:04:42.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:42 smithi090 ceph-mon[29180]: 18 slow requests, 0 included below; oldest blocked for > 37763.467998 secs 2024-04-27T05:04:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:42 smithi119 ceph-mon[32752]: 18 slow requests, 0 included below; oldest blocked for > 37763.467998 secs 2024-04-27T05:04:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:42 smithi167 ceph-mon[32751]: 18 slow requests, 0 included below; oldest blocked for > 37763.467998 secs 2024-04-27T05:04:43.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:43 smithi119 ceph-mon[32752]: pgmap v20643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:43 smithi167 ceph-mon[32751]: pgmap v20643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:43.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:43 smithi090 ceph-mon[29180]: pgmap v20643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:45.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:45 smithi119 ceph-mon[32752]: pgmap v20644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:45.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:45 smithi167 ceph-mon[32751]: pgmap v20644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:45.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:45 smithi090 ceph-mon[29180]: pgmap v20644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37767.569909 secs 2024-04-27T05:04:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37767.569909 secs 2024-04-27T05:04:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37767.569909 secs 2024-04-27T05:04:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:47 smithi119 ceph-mon[32752]: pgmap v20645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:47 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 37768.468118 secs 2024-04-27T05:04:47.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:47 smithi119 ceph-mon[32752]: slow request 184.989175 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T05:04:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:47 smithi167 ceph-mon[32751]: pgmap v20645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:47 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 37768.468118 secs 2024-04-27T05:04:47.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:47 smithi167 ceph-mon[32751]: slow request 184.989175 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T05:04:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:47 smithi090 ceph-mon[29180]: pgmap v20645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:04:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:47 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 37768.468118 secs 2024-04-27T05:04:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:47 smithi090 ceph-mon[29180]: slow request 184.989175 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently joining batch getattr 2024-04-27T05:04:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:49 smithi119 ceph-mon[32752]: pgmap v20646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:04:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:49 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T05:04:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:49 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:04:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:49 smithi167 ceph-mon[32751]: pgmap v20646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:04:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:49 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T05:04:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:49 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:04:49.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:49 smithi090 ceph-mon[29180]: pgmap v20646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:04:49.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:49 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.90:6834/439129317,v1:172.21.15.90:6835/439129317] up:active 2024-04-27T05:04:49.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:49 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:04:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:51 smithi119 ceph-mon[32752]: pgmap v20647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37772.570004 secs 2024-04-27T05:04:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:51 smithi167 ceph-mon[32751]: pgmap v20647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:51.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37772.570004 secs 2024-04-27T05:04:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:51 smithi090 ceph-mon[29180]: pgmap v20647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37772.570004 secs 2024-04-27T05:04:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37773.468242 secs 2024-04-27T05:04:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37773.468242 secs 2024-04-27T05:04:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37773.468242 secs 2024-04-27T05:04:53.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:53 smithi119 ceph-mon[32752]: pgmap v20648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:53.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:53 smithi167 ceph-mon[32751]: pgmap v20648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:53.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:53 smithi090 ceph-mon[29180]: pgmap v20648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:55.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:55 smithi119 ceph-mon[32752]: pgmap v20649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:55.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:55 smithi167 ceph-mon[32751]: pgmap v20649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:55.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:55 smithi090 ceph-mon[29180]: pgmap v20649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:04:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37777.570086 secs 2024-04-27T05:04:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37777.570086 secs 2024-04-27T05:04:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37777.570086 secs 2024-04-27T05:04:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:57 smithi119 ceph-mon[32752]: pgmap v20650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37778.468398 secs 2024-04-27T05:04:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:57 smithi167 ceph-mon[32751]: pgmap v20650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:57.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37778.468398 secs 2024-04-27T05:04:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:57 smithi090 ceph-mon[29180]: pgmap v20650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37778.468398 secs 2024-04-27T05:04:59.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:04:59 smithi119 ceph-mon[32752]: pgmap v20651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:59.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:04:59 smithi167 ceph-mon[32751]: pgmap v20651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:04:59.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:04:59 smithi090 ceph-mon[29180]: pgmap v20651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:01 smithi119 ceph-mon[32752]: pgmap v20652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37782.570169 secs 2024-04-27T05:05:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:01 smithi167 ceph-mon[32751]: pgmap v20652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:01.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37782.570169 secs 2024-04-27T05:05:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:01 smithi090 ceph-mon[29180]: pgmap v20652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37782.570169 secs 2024-04-27T05:05:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37783.468449 secs 2024-04-27T05:05:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37783.468449 secs 2024-04-27T05:05:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37783.468449 secs 2024-04-27T05:05:03.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:03 smithi119 ceph-mon[32752]: pgmap v20653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:03 smithi167 ceph-mon[32751]: pgmap v20653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:03.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:03 smithi090 ceph-mon[29180]: pgmap v20653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:05.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:05 smithi119 ceph-mon[32752]: pgmap v20654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:05 smithi167 ceph-mon[32751]: pgmap v20654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:05.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:05 smithi090 ceph-mon[29180]: pgmap v20654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37787.570248 secs 2024-04-27T05:05:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37787.570248 secs 2024-04-27T05:05:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37787.570248 secs 2024-04-27T05:05:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:07 smithi119 ceph-mon[32752]: pgmap v20655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37788.468515 secs 2024-04-27T05:05:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:07 smithi167 ceph-mon[32751]: pgmap v20655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37788.468515 secs 2024-04-27T05:05:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:07 smithi090 ceph-mon[29180]: pgmap v20655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37788.468515 secs 2024-04-27T05:05:09.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:09 smithi119 ceph-mon[32752]: pgmap v20656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:09.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:09 smithi167 ceph-mon[32751]: pgmap v20656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:09.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:09 smithi090 ceph-mon[29180]: pgmap v20656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:11 smithi119 ceph-mon[32752]: pgmap v20657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:11.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37792.570334 secs 2024-04-27T05:05:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:11 smithi167 ceph-mon[32751]: pgmap v20657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37792.570334 secs 2024-04-27T05:05:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:11 smithi090 ceph-mon[29180]: pgmap v20657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37792.570334 secs 2024-04-27T05:05:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37793.468600 secs 2024-04-27T05:05:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37793.468600 secs 2024-04-27T05:05:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37793.468600 secs 2024-04-27T05:05:13.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:13 smithi119 ceph-mon[32752]: pgmap v20658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:13.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:13 smithi167 ceph-mon[32751]: pgmap v20658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:13.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:13 smithi090 ceph-mon[29180]: pgmap v20658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:15 smithi119 ceph-mon[32752]: pgmap v20659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:15 smithi167 ceph-mon[32751]: pgmap v20659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:15.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:15 smithi090 ceph-mon[29180]: pgmap v20659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37797.570418 secs 2024-04-27T05:05:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37797.570418 secs 2024-04-27T05:05:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37797.570418 secs 2024-04-27T05:05:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:17 smithi119 ceph-mon[32752]: pgmap v20660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37798.468666 secs 2024-04-27T05:05:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:17 smithi167 ceph-mon[32751]: pgmap v20660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:17.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37798.468666 secs 2024-04-27T05:05:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:17 smithi090 ceph-mon[29180]: pgmap v20660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37798.468666 secs 2024-04-27T05:05:19.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:19 smithi119 ceph-mon[32752]: pgmap v20661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:19.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:19 smithi167 ceph-mon[32751]: pgmap v20661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:19.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:19 smithi090 ceph-mon[29180]: pgmap v20661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:21 smithi119 ceph-mon[32752]: pgmap v20662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37802.570507 secs 2024-04-27T05:05:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:21 smithi167 ceph-mon[32751]: pgmap v20662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:21.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37802.570507 secs 2024-04-27T05:05:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:21 smithi090 ceph-mon[29180]: pgmap v20662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37802.570507 secs 2024-04-27T05:05:22.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37803.468756 secs 2024-04-27T05:05:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37803.468756 secs 2024-04-27T05:05:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37803.468756 secs 2024-04-27T05:05:23.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:23 smithi119 ceph-mon[32752]: pgmap v20663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:23 smithi167 ceph-mon[32751]: pgmap v20663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:23.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:23 smithi090 ceph-mon[29180]: pgmap v20663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:25.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:25 smithi167 ceph-mon[32751]: pgmap v20664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:25.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:25 smithi090 ceph-mon[29180]: pgmap v20664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:25 smithi119 ceph-mon[32752]: pgmap v20664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37807.570608 secs 2024-04-27T05:05:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37807.570608 secs 2024-04-27T05:05:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37807.570608 secs 2024-04-27T05:05:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:27 smithi167 ceph-mon[32751]: pgmap v20665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:27.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37808.468851 secs 2024-04-27T05:05:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:27 smithi090 ceph-mon[29180]: pgmap v20665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37808.468851 secs 2024-04-27T05:05:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:27 smithi119 ceph-mon[32752]: pgmap v20665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37808.468851 secs 2024-04-27T05:05:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:29 smithi167 ceph-mon[32751]: pgmap v20666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:29.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:29 smithi090 ceph-mon[29180]: pgmap v20666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:29.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:29 smithi119 ceph-mon[32752]: pgmap v20666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:31 smithi090 ceph-mon[29180]: pgmap v20667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37812.570730 secs 2024-04-27T05:05:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:31 smithi119 ceph-mon[32752]: pgmap v20667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37812.570730 secs 2024-04-27T05:05:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:31 smithi167 ceph-mon[32751]: pgmap v20667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37812.570730 secs 2024-04-27T05:05:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37813.468990 secs 2024-04-27T05:05:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37813.468990 secs 2024-04-27T05:05:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37813.468990 secs 2024-04-27T05:05:33.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:33 smithi090 ceph-mon[29180]: pgmap v20668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:33 smithi119 ceph-mon[32752]: pgmap v20668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:33 smithi167 ceph-mon[32751]: pgmap v20668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:35.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:35 smithi090 ceph-mon[29180]: pgmap v20669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:35 smithi119 ceph-mon[32752]: pgmap v20669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:35 smithi167 ceph-mon[32751]: pgmap v20669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37817.570829 secs 2024-04-27T05:05:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37817.570829 secs 2024-04-27T05:05:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37817.570829 secs 2024-04-27T05:05:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:37 smithi090 ceph-mon[29180]: pgmap v20670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37818.469105 secs 2024-04-27T05:05:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:37 smithi119 ceph-mon[32752]: pgmap v20670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37818.469105 secs 2024-04-27T05:05:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:37 smithi167 ceph-mon[32751]: pgmap v20670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37818.469105 secs 2024-04-27T05:05:38.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:05:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:05:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:05:39.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:39 smithi090 ceph-mon[29180]: pgmap v20671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:39.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:05:39.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:05:39.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:39 smithi090 ceph-mon[29180]: pgmap v20672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:05:39.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:39 smithi090 ceph-mon[29180]: pgmap v20673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:39.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:05:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:39 smithi119 ceph-mon[32752]: pgmap v20671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:05:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:05:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:39 smithi119 ceph-mon[32752]: pgmap v20672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:05:39.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:39 smithi119 ceph-mon[32752]: pgmap v20673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:39.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:05:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:39 smithi167 ceph-mon[32751]: pgmap v20671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:05:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:05:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:39 smithi167 ceph-mon[32751]: pgmap v20672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:05:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:39 smithi167 ceph-mon[32751]: pgmap v20673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:39.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:05:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37822.570917 secs 2024-04-27T05:05:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:41 smithi119 ceph-mon[32752]: pgmap v20674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37822.570917 secs 2024-04-27T05:05:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:41 smithi167 ceph-mon[32751]: pgmap v20674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37822.570917 secs 2024-04-27T05:05:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:41 smithi090 ceph-mon[29180]: pgmap v20674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:42.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37823.469224 secs 2024-04-27T05:05:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37823.469224 secs 2024-04-27T05:05:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37823.469224 secs 2024-04-27T05:05:43.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:43 smithi119 ceph-mon[32752]: pgmap v20675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:05:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:43 smithi167 ceph-mon[32751]: pgmap v20675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:05:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:43 smithi090 ceph-mon[29180]: pgmap v20675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:05:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:45 smithi119 ceph-mon[32752]: pgmap v20676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:45 smithi167 ceph-mon[32751]: pgmap v20676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:45 smithi090 ceph-mon[29180]: pgmap v20676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:46 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 37827.571003 secs 2024-04-27T05:05:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:46 smithi119 ceph-mon[32752]: slow request 1444.476838 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:05:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:46 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 37827.571003 secs 2024-04-27T05:05:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:46 smithi167 ceph-mon[32751]: slow request 1444.476838 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:05:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:46 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 37827.571003 secs 2024-04-27T05:05:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:46 smithi090 ceph-mon[29180]: slow request 1444.476838 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:05:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37828.469366 secs 2024-04-27T05:05:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:47 smithi119 ceph-mon[32752]: pgmap v20677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37828.469366 secs 2024-04-27T05:05:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:47 smithi167 ceph-mon[32751]: pgmap v20677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37828.469366 secs 2024-04-27T05:05:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:47 smithi090 ceph-mon[29180]: pgmap v20677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:05:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:49 smithi119 ceph-mon[32752]: pgmap v20678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:05:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:49 smithi167 ceph-mon[32751]: pgmap v20678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:05:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:49 smithi090 ceph-mon[29180]: pgmap v20678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:05:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37832.571085 secs 2024-04-27T05:05:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:51 smithi119 ceph-mon[32752]: pgmap v20679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37832.571085 secs 2024-04-27T05:05:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:51 smithi167 ceph-mon[32751]: pgmap v20679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37832.571085 secs 2024-04-27T05:05:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:51 smithi090 ceph-mon[29180]: pgmap v20679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:52.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37833.469488 secs 2024-04-27T05:05:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37833.469488 secs 2024-04-27T05:05:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37833.469488 secs 2024-04-27T05:05:53.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:53 smithi119 ceph-mon[32752]: pgmap v20680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:53 smithi167 ceph-mon[32751]: pgmap v20680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:53.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:53 smithi090 ceph-mon[29180]: pgmap v20680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:55.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:55 smithi119 ceph-mon[32752]: pgmap v20681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:55 smithi167 ceph-mon[32751]: pgmap v20681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:55 smithi090 ceph-mon[29180]: pgmap v20681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37837.571168 secs 2024-04-27T05:05:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37837.571168 secs 2024-04-27T05:05:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37837.571168 secs 2024-04-27T05:05:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37838.469552 secs 2024-04-27T05:05:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:57 smithi119 ceph-mon[32752]: pgmap v20682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37838.469552 secs 2024-04-27T05:05:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:57 smithi167 ceph-mon[32751]: pgmap v20682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37838.469552 secs 2024-04-27T05:05:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:57 smithi090 ceph-mon[29180]: pgmap v20682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:05:59.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:05:59 smithi119 ceph-mon[32752]: pgmap v20683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:59.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:05:59 smithi167 ceph-mon[32751]: pgmap v20683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:05:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:05:59 smithi090 ceph-mon[29180]: pgmap v20683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37842.571255 secs 2024-04-27T05:06:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:01 smithi119 ceph-mon[32752]: pgmap v20684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37842.571255 secs 2024-04-27T05:06:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:01 smithi167 ceph-mon[32751]: pgmap v20684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37842.571255 secs 2024-04-27T05:06:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:01 smithi090 ceph-mon[29180]: pgmap v20684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37843.469626 secs 2024-04-27T05:06:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37843.469626 secs 2024-04-27T05:06:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37843.469626 secs 2024-04-27T05:06:03.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:03 smithi119 ceph-mon[32752]: pgmap v20685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:03 smithi167 ceph-mon[32751]: pgmap v20685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:03.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:03 smithi090 ceph-mon[29180]: pgmap v20685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:05 smithi119 ceph-mon[32752]: pgmap v20686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:05 smithi167 ceph-mon[32751]: pgmap v20686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:05.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:05 smithi090 ceph-mon[29180]: pgmap v20686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37847.571341 secs 2024-04-27T05:06:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37847.571341 secs 2024-04-27T05:06:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37847.571341 secs 2024-04-27T05:06:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37848.469742 secs 2024-04-27T05:06:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:07 smithi119 ceph-mon[32752]: pgmap v20687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37848.469742 secs 2024-04-27T05:06:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:07 smithi167 ceph-mon[32751]: pgmap v20687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37848.469742 secs 2024-04-27T05:06:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:07 smithi090 ceph-mon[29180]: pgmap v20687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:09 smithi119 ceph-mon[32752]: pgmap v20688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:09 smithi167 ceph-mon[32751]: pgmap v20688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:09 smithi090 ceph-mon[29180]: pgmap v20688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37852.571425 secs 2024-04-27T05:06:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:11 smithi119 ceph-mon[32752]: pgmap v20689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37852.571425 secs 2024-04-27T05:06:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:11 smithi167 ceph-mon[32751]: pgmap v20689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37852.571425 secs 2024-04-27T05:06:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:11 smithi090 ceph-mon[29180]: pgmap v20689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:12.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37853.469858 secs 2024-04-27T05:06:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37853.469858 secs 2024-04-27T05:06:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37853.469858 secs 2024-04-27T05:06:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:13 smithi119 ceph-mon[32752]: pgmap v20690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:13 smithi167 ceph-mon[32751]: pgmap v20690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:13 smithi090 ceph-mon[29180]: pgmap v20690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:15 smithi119 ceph-mon[32752]: pgmap v20691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:15 smithi167 ceph-mon[32751]: pgmap v20691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:15.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:15 smithi090 ceph-mon[29180]: pgmap v20691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37857.571506 secs 2024-04-27T05:06:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37857.571506 secs 2024-04-27T05:06:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37857.571506 secs 2024-04-27T05:06:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37858.469921 secs 2024-04-27T05:06:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:17 smithi119 ceph-mon[32752]: pgmap v20692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37858.469921 secs 2024-04-27T05:06:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:17 smithi167 ceph-mon[32751]: pgmap v20692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37858.469921 secs 2024-04-27T05:06:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:17 smithi090 ceph-mon[29180]: pgmap v20692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:19 smithi119 ceph-mon[32752]: pgmap v20693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:19 smithi167 ceph-mon[32751]: pgmap v20693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:19 smithi090 ceph-mon[29180]: pgmap v20693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37862.571593 secs 2024-04-27T05:06:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:21 smithi119 ceph-mon[32752]: pgmap v20694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37862.571593 secs 2024-04-27T05:06:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:21 smithi167 ceph-mon[32751]: pgmap v20694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37862.571593 secs 2024-04-27T05:06:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:21 smithi090 ceph-mon[29180]: pgmap v20694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37863.470043 secs 2024-04-27T05:06:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37863.470043 secs 2024-04-27T05:06:22.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37863.470043 secs 2024-04-27T05:06:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:23 smithi119 ceph-mon[32752]: pgmap v20695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:23 smithi167 ceph-mon[32751]: pgmap v20695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:23 smithi090 ceph-mon[29180]: pgmap v20695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:25.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:25 smithi119 ceph-mon[32752]: pgmap v20696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:25 smithi167 ceph-mon[32751]: pgmap v20696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:25 smithi090 ceph-mon[29180]: pgmap v20696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37867.571674 secs 2024-04-27T05:06:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37867.571674 secs 2024-04-27T05:06:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37867.571674 secs 2024-04-27T05:06:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37868.470115 secs 2024-04-27T05:06:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:27 smithi167 ceph-mon[32751]: pgmap v20697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37868.470115 secs 2024-04-27T05:06:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:27 smithi090 ceph-mon[29180]: pgmap v20697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37868.470115 secs 2024-04-27T05:06:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:27 smithi119 ceph-mon[32752]: pgmap v20697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:29.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:29 smithi167 ceph-mon[32751]: pgmap v20698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:29 smithi090 ceph-mon[29180]: pgmap v20698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:29 smithi119 ceph-mon[32752]: pgmap v20698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37872.571758 secs 2024-04-27T05:06:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:31 smithi090 ceph-mon[29180]: pgmap v20699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37872.571758 secs 2024-04-27T05:06:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:31 smithi119 ceph-mon[32752]: pgmap v20699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37872.571758 secs 2024-04-27T05:06:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:31 smithi167 ceph-mon[32751]: pgmap v20699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37873.470237 secs 2024-04-27T05:06:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37873.470237 secs 2024-04-27T05:06:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37873.470237 secs 2024-04-27T05:06:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:33 smithi090 ceph-mon[29180]: pgmap v20700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:34.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:33 smithi119 ceph-mon[32752]: pgmap v20700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:33 smithi167 ceph-mon[32751]: pgmap v20700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:35 smithi090 ceph-mon[29180]: pgmap v20701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:36.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:35 smithi119 ceph-mon[32752]: pgmap v20701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:35 smithi167 ceph-mon[32751]: pgmap v20701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37877.571845 secs 2024-04-27T05:06:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37877.571845 secs 2024-04-27T05:06:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37877.571845 secs 2024-04-27T05:06:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37878.470337 secs 2024-04-27T05:06:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:37 smithi090 ceph-mon[29180]: pgmap v20702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37878.470337 secs 2024-04-27T05:06:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:37 smithi119 ceph-mon[32752]: pgmap v20702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37878.470337 secs 2024-04-27T05:06:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:37 smithi167 ceph-mon[32751]: pgmap v20702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:06:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:06:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:06:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:39 smithi090 ceph-mon[29180]: pgmap v20703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:39.959 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:39 smithi119 ceph-mon[32752]: pgmap v20703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:39.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:39.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:39.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:39.960 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:39 smithi167 ceph-mon[32751]: pgmap v20703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:06:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:40.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:41.628 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:06:41.355+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:06:41.628 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:06:41.355+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:06:41.628 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:06:41.355+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:06:41.628 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:06:41.355+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:06:41.628 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:06:41.355+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37882.571912 secs 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: pgmap v20704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:41.629 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:41.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:41.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:41.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:41.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:41 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37882.571912 secs 2024-04-27T05:06:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: pgmap v20704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:41 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37882.571912 secs 2024-04-27T05:06:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: pgmap v20704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:41 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37883.470428 secs 2024-04-27T05:06:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:06:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:06:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:42.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:06:42.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:06:42.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37883.470428 secs 2024-04-27T05:06:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:06:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:06:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:06:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37883.470428 secs 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:06:43.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:06:43.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:06:43.072 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:06:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:43 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:06:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:43 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:06:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:43 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:06:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:43 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:06:43.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:43 smithi090 ceph-mon[29180]: pgmap v20705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T05:06:43.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:43 smithi090 ceph-mon[29180]: pgmap v20706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:06:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:43 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:06:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:43 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:06:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:43 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:06:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:43 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:06:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:43 smithi119 ceph-mon[32752]: pgmap v20705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T05:06:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:43 smithi119 ceph-mon[32752]: pgmap v20706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:06:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:43 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:06:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:43 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:06:44.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:43 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:06:44.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:43 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:06:44.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:43 smithi167 ceph-mon[32751]: pgmap v20705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 888 B/s rd, 1 op/s 2024-04-27T05:06:44.071 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:43 smithi167 ceph-mon[32751]: pgmap v20706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:06:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:45 smithi090 ceph-mon[29180]: pgmap v20707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:46.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:45 smithi119 ceph-mon[32752]: pgmap v20707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:45 smithi167 ceph-mon[32751]: pgmap v20707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37887.571997 secs 2024-04-27T05:06:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37887.571997 secs 2024-04-27T05:06:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37887.571997 secs 2024-04-27T05:06:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:47 smithi090 ceph-mon[29180]: pgmap v20708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37888.470494 secs 2024-04-27T05:06:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:47 smithi119 ceph-mon[32752]: pgmap v20708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37888.470494 secs 2024-04-27T05:06:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:47 smithi167 ceph-mon[32751]: pgmap v20708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37888.470494 secs 2024-04-27T05:06:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:49 smithi090 ceph-mon[29180]: pgmap v20709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 429 B/s wr, 15 op/s 2024-04-27T05:06:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:49 smithi119 ceph-mon[32752]: pgmap v20709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 429 B/s wr, 15 op/s 2024-04-27T05:06:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:49 smithi167 ceph-mon[32751]: pgmap v20709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 429 B/s wr, 15 op/s 2024-04-27T05:06:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:51 smithi090 ceph-mon[29180]: pgmap v20710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37892.572084 secs 2024-04-27T05:06:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:51 smithi119 ceph-mon[32752]: pgmap v20710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37892.572084 secs 2024-04-27T05:06:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:51 smithi167 ceph-mon[32751]: pgmap v20710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 429 B/s wr, 16 op/s 2024-04-27T05:06:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37892.572084 secs 2024-04-27T05:06:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37893.470608 secs 2024-04-27T05:06:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37893.470608 secs 2024-04-27T05:06:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37893.470608 secs 2024-04-27T05:06:54.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:53 smithi119 ceph-mon[32752]: pgmap v20711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-27T05:06:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:53 smithi167 ceph-mon[32751]: pgmap v20711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-27T05:06:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:53 smithi090 ceph-mon[29180]: pgmap v20711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 409 B/s wr, 15 op/s 2024-04-27T05:06:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:54 smithi090 ceph-mon[29180]: pgmap v20712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T05:06:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:54 smithi119 ceph-mon[32752]: pgmap v20712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T05:06:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:54 smithi167 ceph-mon[32751]: pgmap v20712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 341 B/s wr, 14 op/s 2024-04-27T05:06:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37897.572134 secs 2024-04-27T05:06:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:56 smithi119 ceph-mon[32752]: pgmap v20713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37897.572134 secs 2024-04-27T05:06:57.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:56 smithi167 ceph-mon[32751]: pgmap v20713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37897.572134 secs 2024-04-27T05:06:57.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:56 smithi090 ceph-mon[29180]: pgmap v20713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:58.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37898.470726 secs 2024-04-27T05:06:58.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37898.470726 secs 2024-04-27T05:06:58.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37898.470726 secs 2024-04-27T05:06:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:06:58 smithi119 ceph-mon[32752]: pgmap v20714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:59.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:06:58 smithi167 ceph-mon[32751]: pgmap v20714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:06:59.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:06:58 smithi090 ceph-mon[29180]: pgmap v20714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:00 smithi119 ceph-mon[32752]: pgmap v20715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:01.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:00 smithi167 ceph-mon[32751]: pgmap v20715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:01.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:00 smithi090 ceph-mon[29180]: pgmap v20715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:02 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37902.572193 secs 2024-04-27T05:07:02.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37903.470807 secs 2024-04-27T05:07:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:02 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37902.572193 secs 2024-04-27T05:07:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37903.470807 secs 2024-04-27T05:07:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:02 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37902.572193 secs 2024-04-27T05:07:02.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37903.470807 secs 2024-04-27T05:07:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:03 smithi119 ceph-mon[32752]: pgmap v20716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:03.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:03 smithi167 ceph-mon[32751]: pgmap v20716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:03.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:03 smithi090 ceph-mon[29180]: pgmap v20716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:05.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:05 smithi119 ceph-mon[32752]: pgmap v20717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:05.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:05 smithi167 ceph-mon[32751]: pgmap v20717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:05.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:05 smithi090 ceph-mon[29180]: pgmap v20717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:07 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 37907.572250 secs 2024-04-27T05:07:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:07 smithi119 ceph-mon[32752]: slow request 23041.450873 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:07:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:07 smithi119 ceph-mon[32752]: pgmap v20718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:07.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37908.470916 secs 2024-04-27T05:07:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:07 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 37907.572250 secs 2024-04-27T05:07:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:07 smithi167 ceph-mon[32751]: slow request 23041.450873 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:07:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:07 smithi167 ceph-mon[32751]: pgmap v20718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:07.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37908.470916 secs 2024-04-27T05:07:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:07 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 37907.572250 secs 2024-04-27T05:07:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:07 smithi090 ceph-mon[29180]: slow request 23041.450873 seconds old, received at 2024-04-26T22:43:03.874781+0000: client_request(client.24733:267892 getattr AsXsFs #0x100000001fb 2024-04-26T22:43:03.873920+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:07:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:07 smithi090 ceph-mon[29180]: pgmap v20718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:07.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37908.470916 secs 2024-04-27T05:07:09.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:09 smithi119 ceph-mon[32752]: pgmap v20719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:09.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:09 smithi167 ceph-mon[32751]: pgmap v20719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:09.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:09 smithi090 ceph-mon[29180]: pgmap v20719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:11.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:11 smithi119 ceph-mon[32752]: pgmap v20720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:11.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37912.572336 secs 2024-04-27T05:07:11.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:11 smithi167 ceph-mon[32751]: pgmap v20720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:11.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37912.572336 secs 2024-04-27T05:07:11.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:11 smithi090 ceph-mon[29180]: pgmap v20720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:11.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37912.572336 secs 2024-04-27T05:07:12.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37913.471099 secs 2024-04-27T05:07:12.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37913.471099 secs 2024-04-27T05:07:12.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37913.471099 secs 2024-04-27T05:07:13.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:13 smithi119 ceph-mon[32752]: pgmap v20721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:13.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:13 smithi167 ceph-mon[32751]: pgmap v20721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:13.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:13 smithi090 ceph-mon[29180]: pgmap v20721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:07:15.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:15 smithi119 ceph-mon[32752]: pgmap v20722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:15.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:15 smithi167 ceph-mon[32751]: pgmap v20722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:15.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:15 smithi090 ceph-mon[29180]: pgmap v20722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:07:16.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37917.572420 secs 2024-04-27T05:07:16.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37917.572420 secs 2024-04-27T05:07:16.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37917.572420 secs 2024-04-27T05:07:17.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:17 smithi119 ceph-mon[32752]: pgmap v20723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:17.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37918.471233 secs 2024-04-27T05:07:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:17 smithi167 ceph-mon[32751]: pgmap v20723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:17.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37918.471233 secs 2024-04-27T05:07:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:17 smithi090 ceph-mon[29180]: pgmap v20723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:17.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37918.471233 secs 2024-04-27T05:07:19.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:19 smithi119 ceph-mon[32752]: pgmap v20724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:19.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:19 smithi167 ceph-mon[32751]: pgmap v20724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:19.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:19 smithi090 ceph-mon[29180]: pgmap v20724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:21.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:21 smithi119 ceph-mon[32752]: pgmap v20725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:21.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37922.572508 secs 2024-04-27T05:07:21.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:21 smithi167 ceph-mon[32751]: pgmap v20725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:21.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37922.572508 secs 2024-04-27T05:07:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:21 smithi090 ceph-mon[29180]: pgmap v20725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:21.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37922.572508 secs 2024-04-27T05:07:22.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37923.471373 secs 2024-04-27T05:07:22.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37923.471373 secs 2024-04-27T05:07:22.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37923.471373 secs 2024-04-27T05:07:23.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:23 smithi119 ceph-mon[32752]: pgmap v20726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:23.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:23 smithi167 ceph-mon[32751]: pgmap v20726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:23.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:23 smithi090 ceph-mon[29180]: pgmap v20726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:25.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:25 smithi119 ceph-mon[32752]: pgmap v20727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:25.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:25 smithi167 ceph-mon[32751]: pgmap v20727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:25.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:25 smithi090 ceph-mon[29180]: pgmap v20727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37927.572591 secs 2024-04-27T05:07:26.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37927.572591 secs 2024-04-27T05:07:26.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37927.572591 secs 2024-04-27T05:07:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:27 smithi167 ceph-mon[32751]: pgmap v20728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:27.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37928.471485 secs 2024-04-27T05:07:27.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:27 smithi090 ceph-mon[29180]: pgmap v20728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:27.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37928.471485 secs 2024-04-27T05:07:27.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:27 smithi119 ceph-mon[32752]: pgmap v20728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:27.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37928.471485 secs 2024-04-27T05:07:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:29 smithi167 ceph-mon[32751]: pgmap v20729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:29.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:29 smithi090 ceph-mon[29180]: pgmap v20729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:29.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:29 smithi119 ceph-mon[32752]: pgmap v20729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:31 smithi167 ceph-mon[32751]: pgmap v20730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:31.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37932.572677 secs 2024-04-27T05:07:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:31 smithi090 ceph-mon[29180]: pgmap v20730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:31.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37932.572677 secs 2024-04-27T05:07:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:31 smithi119 ceph-mon[32752]: pgmap v20730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:31.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37932.572677 secs 2024-04-27T05:07:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37933.471607 secs 2024-04-27T05:07:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37933.471607 secs 2024-04-27T05:07:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37933.471607 secs 2024-04-27T05:07:33.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:33 smithi090 ceph-mon[29180]: pgmap v20731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:33.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:33 smithi119 ceph-mon[32752]: pgmap v20731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:33.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:33 smithi167 ceph-mon[32751]: pgmap v20731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:35.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:35 smithi090 ceph-mon[29180]: pgmap v20732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:35.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:35 smithi119 ceph-mon[32752]: pgmap v20732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:35.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:35 smithi167 ceph-mon[32751]: pgmap v20732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:36.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37937.572756 secs 2024-04-27T05:07:36.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37937.572756 secs 2024-04-27T05:07:36.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37937.572756 secs 2024-04-27T05:07:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:37 smithi090 ceph-mon[29180]: pgmap v20733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:37.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37938.471725 secs 2024-04-27T05:07:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:37 smithi119 ceph-mon[32752]: pgmap v20733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:37.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37938.471725 secs 2024-04-27T05:07:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:37 smithi167 ceph-mon[32751]: pgmap v20733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:37.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37938.471725 secs 2024-04-27T05:07:39.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:39 smithi090 ceph-mon[29180]: pgmap v20734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:39.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:39 smithi119 ceph-mon[32752]: pgmap v20734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:39.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:39 smithi167 ceph-mon[32751]: pgmap v20734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:41.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:41 smithi090 ceph-mon[29180]: pgmap v20735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:41.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37942.572838 secs 2024-04-27T05:07:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:41 smithi119 ceph-mon[32752]: pgmap v20735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:41.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37942.572838 secs 2024-04-27T05:07:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:41 smithi167 ceph-mon[32751]: pgmap v20735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37942.572838 secs 2024-04-27T05:07:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37943.471846 secs 2024-04-27T05:07:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:07:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37943.471846 secs 2024-04-27T05:07:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:07:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37943.471846 secs 2024-04-27T05:07:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:07:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:43 smithi090 ceph-mon[29180]: pgmap v20736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:07:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:07:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:43 smithi119 ceph-mon[32752]: pgmap v20736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:07:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:07:44.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:43 smithi167 ceph-mon[32751]: pgmap v20736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:07:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:07:44.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:07:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:44 smithi090 ceph-mon[29180]: pgmap v20737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T05:07:44.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:44 smithi090 ceph-mon[29180]: pgmap v20738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T05:07:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:44 smithi119 ceph-mon[32752]: pgmap v20737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T05:07:45.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:44 smithi119 ceph-mon[32752]: pgmap v20738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T05:07:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:44 smithi167 ceph-mon[32751]: pgmap v20737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T05:07:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:44 smithi167 ceph-mon[32751]: pgmap v20738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T05:07:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:46 smithi090 ceph-mon[29180]: pgmap v20739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37947.572917 secs 2024-04-27T05:07:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:46 smithi119 ceph-mon[32752]: pgmap v20739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37947.572917 secs 2024-04-27T05:07:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:46 smithi167 ceph-mon[32751]: pgmap v20739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37947.572917 secs 2024-04-27T05:07:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:47 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 37948.471967 secs 2024-04-27T05:07:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:47 smithi090 ceph-mon[29180]: slow request 364.993023 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:07:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:47 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 37948.471967 secs 2024-04-27T05:07:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:47 smithi119 ceph-mon[32752]: slow request 364.993023 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:07:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:47 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 37948.471967 secs 2024-04-27T05:07:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:47 smithi167 ceph-mon[32751]: slow request 364.993023 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:07:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:48 smithi090 ceph-mon[29180]: pgmap v20740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:48 smithi119 ceph-mon[32752]: pgmap v20740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:48 smithi167 ceph-mon[32751]: pgmap v20740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:50 smithi090 ceph-mon[29180]: pgmap v20741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:50 smithi119 ceph-mon[32752]: pgmap v20741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:50 smithi167 ceph-mon[32751]: pgmap v20741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37952.573002 secs 2024-04-27T05:07:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37952.573002 secs 2024-04-27T05:07:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37952.573002 secs 2024-04-27T05:07:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:52 smithi090 ceph-mon[29180]: pgmap v20742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37953.472083 secs 2024-04-27T05:07:52.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:52 smithi119 ceph-mon[32752]: pgmap v20742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:52.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37953.472083 secs 2024-04-27T05:07:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:52 smithi167 ceph-mon[32751]: pgmap v20742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:07:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37953.472083 secs 2024-04-27T05:07:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:54 smithi090 ceph-mon[29180]: pgmap v20743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:07:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:54 smithi119 ceph-mon[32752]: pgmap v20743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:07:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:54 smithi167 ceph-mon[32751]: pgmap v20743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:07:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:56 smithi090 ceph-mon[29180]: pgmap v20744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37957.573055 secs 2024-04-27T05:07:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:56 smithi119 ceph-mon[32752]: pgmap v20744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37957.573055 secs 2024-04-27T05:07:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:56 smithi167 ceph-mon[32751]: pgmap v20744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:07:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37957.573055 secs 2024-04-27T05:07:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37958.472172 secs 2024-04-27T05:07:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37958.472172 secs 2024-04-27T05:07:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37958.472172 secs 2024-04-27T05:07:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:07:58 smithi090 ceph-mon[29180]: pgmap v20745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:59.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:07:58 smithi119 ceph-mon[32752]: pgmap v20745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:07:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:07:58 smithi167 ceph-mon[32751]: pgmap v20745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:00 smithi119 ceph-mon[32752]: pgmap v20746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:00 smithi167 ceph-mon[32751]: pgmap v20746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:00 smithi090 ceph-mon[29180]: pgmap v20746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37962.573128 secs 2024-04-27T05:08:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37962.573128 secs 2024-04-27T05:08:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37962.573128 secs 2024-04-27T05:08:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:02 smithi119 ceph-mon[32752]: pgmap v20747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37963.472238 secs 2024-04-27T05:08:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:02 smithi167 ceph-mon[32751]: pgmap v20747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37963.472238 secs 2024-04-27T05:08:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:02 smithi090 ceph-mon[29180]: pgmap v20747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37963.472238 secs 2024-04-27T05:08:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:04 smithi119 ceph-mon[32752]: pgmap v20748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:04 smithi167 ceph-mon[32751]: pgmap v20748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:04 smithi090 ceph-mon[29180]: pgmap v20748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:05 smithi119 ceph-mon[32752]: pgmap v20749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:05 smithi167 ceph-mon[32751]: pgmap v20749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:06.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:05 smithi090 ceph-mon[29180]: pgmap v20749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37967.573188 secs 2024-04-27T05:08:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37967.573188 secs 2024-04-27T05:08:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37967.573188 secs 2024-04-27T05:08:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37968.472363 secs 2024-04-27T05:08:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:07 smithi119 ceph-mon[32752]: pgmap v20750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37968.472363 secs 2024-04-27T05:08:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:07 smithi167 ceph-mon[32751]: pgmap v20750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37968.472363 secs 2024-04-27T05:08:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:07 smithi090 ceph-mon[29180]: pgmap v20750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:10.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:10 smithi119 ceph-mon[32752]: pgmap v20751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:10.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:10 smithi167 ceph-mon[32751]: pgmap v20751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:10 smithi090 ceph-mon[29180]: pgmap v20751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37972.573270 secs 2024-04-27T05:08:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37972.573270 secs 2024-04-27T05:08:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37972.573270 secs 2024-04-27T05:08:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37973.472481 secs 2024-04-27T05:08:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:12 smithi119 ceph-mon[32752]: pgmap v20752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37973.472481 secs 2024-04-27T05:08:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:12 smithi167 ceph-mon[32751]: pgmap v20752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37973.472481 secs 2024-04-27T05:08:12.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:12 smithi090 ceph-mon[29180]: pgmap v20752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:14 smithi119 ceph-mon[32752]: pgmap v20753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:14 smithi167 ceph-mon[32751]: pgmap v20753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:14.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:14 smithi090 ceph-mon[29180]: pgmap v20753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:16 smithi119 ceph-mon[32752]: pgmap v20754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37977.573353 secs 2024-04-27T05:08:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:16 smithi167 ceph-mon[32751]: pgmap v20754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37977.573353 secs 2024-04-27T05:08:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:16 smithi090 ceph-mon[29180]: pgmap v20754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37977.573353 secs 2024-04-27T05:08:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37978.472618 secs 2024-04-27T05:08:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37978.472618 secs 2024-04-27T05:08:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37978.472618 secs 2024-04-27T05:08:18.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:18 smithi119 ceph-mon[32752]: pgmap v20755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:18.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:18 smithi167 ceph-mon[32751]: pgmap v20755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:18.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:18 smithi090 ceph-mon[29180]: pgmap v20755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:20.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:20 smithi119 ceph-mon[32752]: pgmap v20756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:20.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:20 smithi167 ceph-mon[32751]: pgmap v20756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:20.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:20 smithi090 ceph-mon[29180]: pgmap v20756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37982.573412 secs 2024-04-27T05:08:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37982.573412 secs 2024-04-27T05:08:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37982.573412 secs 2024-04-27T05:08:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37983.472702 secs 2024-04-27T05:08:22.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:22 smithi167 ceph-mon[32751]: pgmap v20757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37983.472702 secs 2024-04-27T05:08:22.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:22 smithi090 ceph-mon[29180]: pgmap v20757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37983.472702 secs 2024-04-27T05:08:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:22 smithi119 ceph-mon[32752]: pgmap v20757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:24.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:24 smithi167 ceph-mon[32751]: pgmap v20758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:24.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:24 smithi090 ceph-mon[29180]: pgmap v20758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:24.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:24 smithi119 ceph-mon[32752]: pgmap v20758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:26 smithi167 ceph-mon[32751]: pgmap v20759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:26.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37987.573500 secs 2024-04-27T05:08:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:26 smithi090 ceph-mon[29180]: pgmap v20759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:26.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37987.573500 secs 2024-04-27T05:08:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:26 smithi119 ceph-mon[32752]: pgmap v20759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37987.573500 secs 2024-04-27T05:08:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37988.472815 secs 2024-04-27T05:08:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37988.472815 secs 2024-04-27T05:08:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37988.472815 secs 2024-04-27T05:08:28.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:28 smithi090 ceph-mon[29180]: pgmap v20760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:28.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:28 smithi119 ceph-mon[32752]: pgmap v20760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:28 smithi167 ceph-mon[32751]: pgmap v20760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:30.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:30 smithi090 ceph-mon[29180]: pgmap v20761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:30 smithi119 ceph-mon[32752]: pgmap v20761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:30 smithi167 ceph-mon[32751]: pgmap v20761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37992.573563 secs 2024-04-27T05:08:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37992.573563 secs 2024-04-27T05:08:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37992.573563 secs 2024-04-27T05:08:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37993.472960 secs 2024-04-27T05:08:32.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:32 smithi090 ceph-mon[29180]: pgmap v20762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37993.472960 secs 2024-04-27T05:08:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:32 smithi119 ceph-mon[32752]: pgmap v20762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37993.472960 secs 2024-04-27T05:08:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:32 smithi167 ceph-mon[32751]: pgmap v20762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:34.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:34 smithi090 ceph-mon[29180]: pgmap v20763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:34 smithi119 ceph-mon[32752]: pgmap v20763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:34 smithi167 ceph-mon[32751]: pgmap v20763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:36 smithi090 ceph-mon[29180]: pgmap v20764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:36.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 37997.573611 secs 2024-04-27T05:08:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:36 smithi119 ceph-mon[32752]: pgmap v20764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 37997.573611 secs 2024-04-27T05:08:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:36 smithi167 ceph-mon[32751]: pgmap v20764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 37997.573611 secs 2024-04-27T05:08:37.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 37998.473047 secs 2024-04-27T05:08:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 37998.473047 secs 2024-04-27T05:08:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 37998.473047 secs 2024-04-27T05:08:38.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:38 smithi090 ceph-mon[29180]: pgmap v20765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:38 smithi119 ceph-mon[32752]: pgmap v20765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:38 smithi167 ceph-mon[32751]: pgmap v20765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:40 smithi090 ceph-mon[29180]: pgmap v20766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:40 smithi119 ceph-mon[32752]: pgmap v20766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:40 smithi167 ceph-mon[32751]: pgmap v20766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:08:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:41 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 38002.573672 secs 2024-04-27T05:08:41.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:41 smithi090 ceph-mon[29180]: slow request 2882.686317 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:08:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:41 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 38002.573672 secs 2024-04-27T05:08:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:41 smithi119 ceph-mon[32752]: slow request 2882.686317 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:08:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:41 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 38002.573672 secs 2024-04-27T05:08:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:41 smithi167 ceph-mon[32751]: slow request 2882.686317 seconds old, received at 2024-04-27T04:20:37.640759+0000: client_request(client.24733:267904 getattr AsXsFs #0x100000001fb 2024-04-27T04:20:37.639109+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:08:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38003.473162 secs 2024-04-27T05:08:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:42 smithi090 ceph-mon[29180]: pgmap v20767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38003.473162 secs 2024-04-27T05:08:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:42 smithi119 ceph-mon[32752]: pgmap v20767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38003.473162 secs 2024-04-27T05:08:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:42 smithi167 ceph-mon[32751]: pgmap v20767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:43.623 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:08:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:08:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:08:44.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:44 smithi090 ceph-mon[29180]: pgmap v20768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:08:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:08:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:08:44.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:44 smithi119 ceph-mon[32752]: pgmap v20768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:08:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:08:44.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:08:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:44 smithi167 ceph-mon[32751]: pgmap v20768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:08:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:08:44.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:08:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:45 smithi090 ceph-mon[29180]: pgmap v20769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:08:45.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:45 smithi090 ceph-mon[29180]: pgmap v20770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:45 smithi119 ceph-mon[32752]: pgmap v20769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:08:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:45 smithi119 ceph-mon[32752]: pgmap v20770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:45 smithi167 ceph-mon[32751]: pgmap v20769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:08:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:45 smithi167 ceph-mon[32751]: pgmap v20770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:46 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 38007.573754 secs 2024-04-27T05:08:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:46 smithi119 ceph-mon[32752]: slow request 724.168432 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:08:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:46 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 38007.573754 secs 2024-04-27T05:08:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:46 smithi167 ceph-mon[32751]: slow request 724.168432 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:08:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:46 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 38007.573754 secs 2024-04-27T05:08:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:46 smithi090 ceph-mon[29180]: slow request 724.168432 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:08:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:47 smithi119 ceph-mon[32752]: pgmap v20771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38008.473296 secs 2024-04-27T05:08:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:47 smithi167 ceph-mon[32751]: pgmap v20771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38008.473296 secs 2024-04-27T05:08:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:47 smithi090 ceph-mon[29180]: pgmap v20771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38008.473296 secs 2024-04-27T05:08:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:49 smithi119 ceph-mon[32752]: pgmap v20772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T05:08:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:49 smithi167 ceph-mon[32751]: pgmap v20772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T05:08:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:49 smithi090 ceph-mon[29180]: pgmap v20772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T05:08:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:51 smithi119 ceph-mon[32752]: pgmap v20773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38012.573848 secs 2024-04-27T05:08:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:51 smithi167 ceph-mon[32751]: pgmap v20773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38012.573848 secs 2024-04-27T05:08:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:51 smithi090 ceph-mon[29180]: pgmap v20773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38012.573848 secs 2024-04-27T05:08:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38013.473398 secs 2024-04-27T05:08:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38013.473398 secs 2024-04-27T05:08:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38013.473398 secs 2024-04-27T05:08:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:53 smithi119 ceph-mon[32752]: pgmap v20774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:53 smithi167 ceph-mon[32751]: pgmap v20774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:53 smithi090 ceph-mon[29180]: pgmap v20774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:08:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:55 smithi119 ceph-mon[32752]: pgmap v20775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:08:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:55 smithi167 ceph-mon[32751]: pgmap v20775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:08:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:55 smithi090 ceph-mon[29180]: pgmap v20775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:08:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38017.573928 secs 2024-04-27T05:08:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38017.573928 secs 2024-04-27T05:08:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38017.573928 secs 2024-04-27T05:08:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:57 smithi119 ceph-mon[32752]: pgmap v20776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38018.473442 secs 2024-04-27T05:08:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:57 smithi167 ceph-mon[32751]: pgmap v20776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38018.473442 secs 2024-04-27T05:08:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:57 smithi090 ceph-mon[29180]: pgmap v20776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38018.473442 secs 2024-04-27T05:08:59.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:08:59 smithi119 ceph-mon[32752]: pgmap v20777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:59.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:08:59 smithi167 ceph-mon[32751]: pgmap v20777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:08:59.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:08:59 smithi090 ceph-mon[29180]: pgmap v20777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:01 smithi119 ceph-mon[32752]: pgmap v20778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38022.573986 secs 2024-04-27T05:09:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:01 smithi167 ceph-mon[32751]: pgmap v20778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38022.573986 secs 2024-04-27T05:09:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:01 smithi090 ceph-mon[29180]: pgmap v20778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38022.573986 secs 2024-04-27T05:09:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38023.473543 secs 2024-04-27T05:09:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38023.473543 secs 2024-04-27T05:09:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38023.473543 secs 2024-04-27T05:09:03.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:03 smithi119 ceph-mon[32752]: pgmap v20779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:03 smithi167 ceph-mon[32751]: pgmap v20779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:03 smithi090 ceph-mon[29180]: pgmap v20779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:05 smithi119 ceph-mon[32752]: pgmap v20780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:05 smithi167 ceph-mon[32751]: pgmap v20780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:05 smithi090 ceph-mon[29180]: pgmap v20780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38027.574063 secs 2024-04-27T05:09:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38027.574063 secs 2024-04-27T05:09:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38027.574063 secs 2024-04-27T05:09:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:07 smithi119 ceph-mon[32752]: pgmap v20781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38028.473669 secs 2024-04-27T05:09:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:07 smithi167 ceph-mon[32751]: pgmap v20781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:07.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38028.473669 secs 2024-04-27T05:09:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:07 smithi090 ceph-mon[29180]: pgmap v20781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38028.473669 secs 2024-04-27T05:09:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:09 smithi119 ceph-mon[32752]: pgmap v20782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:09 smithi167 ceph-mon[32751]: pgmap v20782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:09 smithi090 ceph-mon[29180]: pgmap v20782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:11 smithi119 ceph-mon[32752]: pgmap v20783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38032.574163 secs 2024-04-27T05:09:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:11 smithi167 ceph-mon[32751]: pgmap v20783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38032.574163 secs 2024-04-27T05:09:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:11 smithi090 ceph-mon[29180]: pgmap v20783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38032.574163 secs 2024-04-27T05:09:12.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38033.473784 secs 2024-04-27T05:09:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38033.473784 secs 2024-04-27T05:09:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38033.473784 secs 2024-04-27T05:09:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:13 smithi119 ceph-mon[32752]: pgmap v20784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:13 smithi167 ceph-mon[32751]: pgmap v20784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:13.831 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:13 smithi090 ceph-mon[29180]: pgmap v20784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:15 smithi119 ceph-mon[32752]: pgmap v20785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:15 smithi167 ceph-mon[32751]: pgmap v20785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:15 smithi090 ceph-mon[29180]: pgmap v20785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38037.574243 secs 2024-04-27T05:09:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38037.574243 secs 2024-04-27T05:09:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38037.574243 secs 2024-04-27T05:09:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:17 smithi119 ceph-mon[32752]: pgmap v20786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38038.473860 secs 2024-04-27T05:09:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:17 smithi167 ceph-mon[32751]: pgmap v20786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38038.473860 secs 2024-04-27T05:09:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:17 smithi090 ceph-mon[29180]: pgmap v20786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38038.473860 secs 2024-04-27T05:09:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:19 smithi119 ceph-mon[32752]: pgmap v20787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:19 smithi167 ceph-mon[32751]: pgmap v20787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:19.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:19 smithi090 ceph-mon[29180]: pgmap v20787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:21 smithi119 ceph-mon[32752]: pgmap v20788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38042.574326 secs 2024-04-27T05:09:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:21 smithi167 ceph-mon[32751]: pgmap v20788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38042.574326 secs 2024-04-27T05:09:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:21 smithi090 ceph-mon[29180]: pgmap v20788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38042.574326 secs 2024-04-27T05:09:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38043.473975 secs 2024-04-27T05:09:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38043.473975 secs 2024-04-27T05:09:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38043.473975 secs 2024-04-27T05:09:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:23 smithi119 ceph-mon[32752]: pgmap v20789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:23 smithi167 ceph-mon[32751]: pgmap v20789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:23.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:23 smithi090 ceph-mon[29180]: pgmap v20789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:25 smithi119 ceph-mon[32752]: pgmap v20790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:25 smithi167 ceph-mon[32751]: pgmap v20790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:25.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:25 smithi090 ceph-mon[29180]: pgmap v20790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38047.574404 secs 2024-04-27T05:09:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38047.574404 secs 2024-04-27T05:09:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38047.574404 secs 2024-04-27T05:09:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:27 smithi119 ceph-mon[32752]: pgmap v20791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38048.474118 secs 2024-04-27T05:09:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:27 smithi167 ceph-mon[32751]: pgmap v20791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38048.474118 secs 2024-04-27T05:09:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:27 smithi090 ceph-mon[29180]: pgmap v20791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38048.474118 secs 2024-04-27T05:09:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:29 smithi119 ceph-mon[32752]: pgmap v20792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:29 smithi167 ceph-mon[32751]: pgmap v20792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:29 smithi090 ceph-mon[29180]: pgmap v20792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:09:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:31 smithi119 ceph-mon[32752]: pgmap v20793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38052.574489 secs 2024-04-27T05:09:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:31 smithi167 ceph-mon[32751]: pgmap v20793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38052.574489 secs 2024-04-27T05:09:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:31 smithi090 ceph-mon[29180]: pgmap v20793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38052.574489 secs 2024-04-27T05:09:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38053.474238 secs 2024-04-27T05:09:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38053.474238 secs 2024-04-27T05:09:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38053.474238 secs 2024-04-27T05:09:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:33 smithi119 ceph-mon[32752]: pgmap v20794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:33 smithi167 ceph-mon[32751]: pgmap v20794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:33 smithi090 ceph-mon[29180]: pgmap v20794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:35 smithi119 ceph-mon[32752]: pgmap v20795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:35 smithi167 ceph-mon[32751]: pgmap v20795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:35 smithi090 ceph-mon[29180]: pgmap v20795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38057.574571 secs 2024-04-27T05:09:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38057.574571 secs 2024-04-27T05:09:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38057.574571 secs 2024-04-27T05:09:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:37 smithi119 ceph-mon[32752]: pgmap v20796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38058.474328 secs 2024-04-27T05:09:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:37 smithi167 ceph-mon[32751]: pgmap v20796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38058.474328 secs 2024-04-27T05:09:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:37 smithi090 ceph-mon[29180]: pgmap v20796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38058.474328 secs 2024-04-27T05:09:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:39 smithi119 ceph-mon[32752]: pgmap v20797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:39 smithi167 ceph-mon[32751]: pgmap v20797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:39 smithi090 ceph-mon[29180]: pgmap v20797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:41 smithi119 ceph-mon[32752]: pgmap v20798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38062.574653 secs 2024-04-27T05:09:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:41 smithi167 ceph-mon[32751]: pgmap v20798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38062.574653 secs 2024-04-27T05:09:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:41 smithi090 ceph-mon[29180]: pgmap v20798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38062.574653 secs 2024-04-27T05:09:42.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 38063.474461 secs 2024-04-27T05:09:42.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:42 smithi090 ceph-mon[29180]: slow request 11523.080478 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:09:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 38063.474461 secs 2024-04-27T05:09:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:42 smithi119 ceph-mon[32752]: slow request 11523.080478 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:09:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 38063.474461 secs 2024-04-27T05:09:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:42 smithi167 ceph-mon[32751]: slow request 11523.080478 seconds old, received at 2024-04-27T01:57:38.144628+0000: client_request(client.17610:1 getattr pAsLsXsFs #0x1 2024-04-27T01:57:38.141018+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:09:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:43 smithi119 ceph-mon[32752]: pgmap v20799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:43 smithi167 ceph-mon[32751]: pgmap v20799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:43.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:43 smithi090 ceph-mon[29180]: pgmap v20799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:44.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:44 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:09:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:44 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:09:44.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:44 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:09:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:45 smithi119 ceph-mon[32752]: pgmap v20800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:09:45.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:09:45.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:09:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:45 smithi167 ceph-mon[32751]: pgmap v20800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:09:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:09:45.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:09:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:45 smithi090 ceph-mon[29180]: pgmap v20800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:09:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:09:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:09:45.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:09:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:46 smithi119 ceph-mon[32752]: pgmap v20801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:09:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:46 smithi119 ceph-mon[32752]: pgmap v20802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38067.574736 secs 2024-04-27T05:09:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:46 smithi167 ceph-mon[32751]: pgmap v20801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:09:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:46 smithi167 ceph-mon[32751]: pgmap v20802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38067.574736 secs 2024-04-27T05:09:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:46 smithi090 ceph-mon[29180]: pgmap v20801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:09:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:46 smithi090 ceph-mon[29180]: pgmap v20802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38067.574736 secs 2024-04-27T05:09:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38068.474537 secs 2024-04-27T05:09:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38068.474537 secs 2024-04-27T05:09:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38068.474537 secs 2024-04-27T05:09:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:48 smithi119 ceph-mon[32752]: pgmap v20803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:48 smithi167 ceph-mon[32751]: pgmap v20803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:48 smithi090 ceph-mon[29180]: pgmap v20803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:50 smithi119 ceph-mon[32752]: pgmap v20804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:50 smithi167 ceph-mon[32751]: pgmap v20804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:50 smithi090 ceph-mon[29180]: pgmap v20804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38072.574836 secs 2024-04-27T05:09:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38072.574836 secs 2024-04-27T05:09:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38072.574836 secs 2024-04-27T05:09:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:52 smithi167 ceph-mon[32751]: pgmap v20805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:52.847 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38073.474638 secs 2024-04-27T05:09:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:52 smithi090 ceph-mon[29180]: pgmap v20805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38073.474638 secs 2024-04-27T05:09:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:52 smithi119 ceph-mon[32752]: pgmap v20805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:09:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38073.474638 secs 2024-04-27T05:09:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:54 smithi167 ceph-mon[32751]: pgmap v20806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:09:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:54 smithi090 ceph-mon[29180]: pgmap v20806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:09:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:54 smithi119 ceph-mon[32752]: pgmap v20806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:09:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:56 smithi167 ceph-mon[32751]: pgmap v20807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:09:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38077.574923 secs 2024-04-27T05:09:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:56 smithi090 ceph-mon[29180]: pgmap v20807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:09:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38077.574923 secs 2024-04-27T05:09:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:56 smithi119 ceph-mon[32752]: pgmap v20807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:09:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38077.574923 secs 2024-04-27T05:09:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38078.474701 secs 2024-04-27T05:09:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38078.474701 secs 2024-04-27T05:09:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38078.474701 secs 2024-04-27T05:09:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:09:58 smithi090 ceph-mon[29180]: pgmap v20808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:09:58 smithi119 ceph-mon[32752]: pgmap v20808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:09:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:09:58 smithi167 ceph-mon[32751]: pgmap v20808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:00.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:00 smithi090 ceph-mon[29180]: pgmap v20809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:00.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:10:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:00 smithi119 ceph-mon[32752]: pgmap v20809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:01.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:10:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:00 smithi167 ceph-mon[32751]: pgmap v20809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:10:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38082.575015 secs 2024-04-27T05:10:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38082.575015 secs 2024-04-27T05:10:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38082.575015 secs 2024-04-27T05:10:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:02 smithi090 ceph-mon[29180]: pgmap v20810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38083.474775 secs 2024-04-27T05:10:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:02 smithi119 ceph-mon[32752]: pgmap v20810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38083.474775 secs 2024-04-27T05:10:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:02 smithi167 ceph-mon[32751]: pgmap v20810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38083.474775 secs 2024-04-27T05:10:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:04 smithi119 ceph-mon[32752]: pgmap v20811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:04.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:04 smithi090 ceph-mon[29180]: pgmap v20811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:04 smithi167 ceph-mon[32751]: pgmap v20811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:06 smithi090 ceph-mon[29180]: pgmap v20812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38087.575098 secs 2024-04-27T05:10:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:06 smithi119 ceph-mon[32752]: pgmap v20812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38087.575098 secs 2024-04-27T05:10:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:06 smithi167 ceph-mon[32751]: pgmap v20812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38087.575098 secs 2024-04-27T05:10:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38088.474868 secs 2024-04-27T05:10:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38088.474868 secs 2024-04-27T05:10:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38088.474868 secs 2024-04-27T05:10:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:08 smithi090 ceph-mon[29180]: pgmap v20813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:08 smithi119 ceph-mon[32752]: pgmap v20813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:08 smithi167 ceph-mon[32751]: pgmap v20813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:10 smithi090 ceph-mon[29180]: pgmap v20814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:10 smithi119 ceph-mon[32752]: pgmap v20814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:10 smithi167 ceph-mon[32751]: pgmap v20814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38092.575159 secs 2024-04-27T05:10:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38092.575159 secs 2024-04-27T05:10:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38092.575159 secs 2024-04-27T05:10:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:12 smithi090 ceph-mon[29180]: pgmap v20815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38093.474994 secs 2024-04-27T05:10:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:12 smithi119 ceph-mon[32752]: pgmap v20815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38093.474994 secs 2024-04-27T05:10:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:12 smithi167 ceph-mon[32751]: pgmap v20815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38093.474994 secs 2024-04-27T05:10:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:14 smithi090 ceph-mon[29180]: pgmap v20816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:14 smithi119 ceph-mon[32752]: pgmap v20816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:14 smithi167 ceph-mon[32751]: pgmap v20816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:16 smithi090 ceph-mon[29180]: pgmap v20817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38097.575217 secs 2024-04-27T05:10:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:16 smithi119 ceph-mon[32752]: pgmap v20817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38097.575217 secs 2024-04-27T05:10:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:16 smithi167 ceph-mon[32751]: pgmap v20817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38097.575217 secs 2024-04-27T05:10:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38098.475111 secs 2024-04-27T05:10:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38098.475111 secs 2024-04-27T05:10:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38098.475111 secs 2024-04-27T05:10:18.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:18 smithi090 ceph-mon[29180]: pgmap v20818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:18 smithi119 ceph-mon[32752]: pgmap v20818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:18 smithi167 ceph-mon[32751]: pgmap v20818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:20.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:20 smithi090 ceph-mon[29180]: pgmap v20819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:20 smithi119 ceph-mon[32752]: pgmap v20819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:20 smithi167 ceph-mon[32751]: pgmap v20819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38102.575304 secs 2024-04-27T05:10:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38102.575304 secs 2024-04-27T05:10:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38102.575304 secs 2024-04-27T05:10:22.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:22 smithi119 ceph-mon[32752]: pgmap v20820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:22.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38103.475247 secs 2024-04-27T05:10:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:22 smithi167 ceph-mon[32751]: pgmap v20820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38103.475247 secs 2024-04-27T05:10:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:22 smithi090 ceph-mon[29180]: pgmap v20820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38103.475247 secs 2024-04-27T05:10:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:24 smithi119 ceph-mon[32752]: pgmap v20821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:24 smithi167 ceph-mon[32751]: pgmap v20821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:24 smithi090 ceph-mon[29180]: pgmap v20821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:26 smithi119 ceph-mon[32752]: pgmap v20822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38107.575388 secs 2024-04-27T05:10:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:26 smithi167 ceph-mon[32751]: pgmap v20822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38107.575388 secs 2024-04-27T05:10:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:26 smithi090 ceph-mon[29180]: pgmap v20822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38107.575388 secs 2024-04-27T05:10:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38108.475404 secs 2024-04-27T05:10:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38108.475404 secs 2024-04-27T05:10:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38108.475404 secs 2024-04-27T05:10:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:28 smithi119 ceph-mon[32752]: pgmap v20823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:28 smithi167 ceph-mon[32751]: pgmap v20823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:29.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:28 smithi090 ceph-mon[29180]: pgmap v20823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:30 smithi119 ceph-mon[32752]: pgmap v20824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:30 smithi167 ceph-mon[32751]: pgmap v20824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:31.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:30 smithi090 ceph-mon[29180]: pgmap v20824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:32.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38112.575475 secs 2024-04-27T05:10:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38112.575475 secs 2024-04-27T05:10:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38112.575475 secs 2024-04-27T05:10:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:32 smithi119 ceph-mon[32752]: pgmap v20825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38113.475472 secs 2024-04-27T05:10:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:32 smithi167 ceph-mon[32751]: pgmap v20825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38113.475472 secs 2024-04-27T05:10:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:32 smithi090 ceph-mon[29180]: pgmap v20825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38113.475472 secs 2024-04-27T05:10:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:34 smithi119 ceph-mon[32752]: pgmap v20826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:34 smithi167 ceph-mon[32751]: pgmap v20826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:34 smithi090 ceph-mon[29180]: pgmap v20826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:35 smithi119 ceph-mon[32752]: pgmap v20827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:35 smithi167 ceph-mon[32751]: pgmap v20827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:35 smithi090 ceph-mon[29180]: pgmap v20827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38117.575555 secs 2024-04-27T05:10:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38117.575555 secs 2024-04-27T05:10:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38117.575555 secs 2024-04-27T05:10:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38118.475594 secs 2024-04-27T05:10:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:37 smithi119 ceph-mon[32752]: pgmap v20828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38118.475594 secs 2024-04-27T05:10:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:37 smithi167 ceph-mon[32751]: pgmap v20828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38118.475594 secs 2024-04-27T05:10:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:37 smithi090 ceph-mon[29180]: pgmap v20828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:39 smithi090 ceph-mon[29180]: pgmap v20829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:40.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:39 smithi119 ceph-mon[32752]: pgmap v20829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:40.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:39 smithi167 ceph-mon[32751]: pgmap v20829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:42.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38122.575642 secs 2024-04-27T05:10:42.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:41 smithi119 ceph-mon[32752]: pgmap v20830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38122.575642 secs 2024-04-27T05:10:42.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:41 smithi167 ceph-mon[32751]: pgmap v20830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:42.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38122.575642 secs 2024-04-27T05:10:42.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:41 smithi090 ceph-mon[29180]: pgmap v20830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:43.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 38123.475693 secs 2024-04-27T05:10:43.279 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:42 smithi090 ceph-mon[29180]: slow request 1440.246993 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:10:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 38123.475693 secs 2024-04-27T05:10:43.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:42 smithi119 ceph-mon[32752]: slow request 1440.246993 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:10:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 38123.475693 secs 2024-04-27T05:10:43.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:42 smithi167 ceph-mon[32751]: slow request 1440.246993 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:10:44.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:43 smithi119 ceph-mon[32752]: pgmap v20831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:44.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:43 smithi167 ceph-mon[32751]: pgmap v20831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:44.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:43 smithi090 ceph-mon[29180]: pgmap v20831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:46.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:45 smithi119 ceph-mon[32752]: pgmap v20832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:10:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:10:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:10:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:45 smithi119 ceph-mon[32752]: pgmap v20833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:10:46.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:45 smithi119 ceph-mon[32752]: pgmap v20834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:46.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:45 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:10:46.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:45 smithi167 ceph-mon[32751]: pgmap v20832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:10:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:10:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:10:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:45 smithi167 ceph-mon[32751]: pgmap v20833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:10:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:45 smithi167 ceph-mon[32751]: pgmap v20834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:46.320 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:45 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:10:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:45 smithi090 ceph-mon[29180]: pgmap v20832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:10:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:10:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:10:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:10:46.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:45 smithi090 ceph-mon[29180]: pgmap v20833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:10:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:45 smithi090 ceph-mon[29180]: pgmap v20834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:46.379 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:45 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:10:47.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38127.575707 secs 2024-04-27T05:10:47.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38127.575707 secs 2024-04-27T05:10:47.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38127.575707 secs 2024-04-27T05:10:48.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38128.475813 secs 2024-04-27T05:10:48.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:47 smithi119 ceph-mon[32752]: pgmap v20835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:10:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38128.475813 secs 2024-04-27T05:10:48.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:47 smithi167 ceph-mon[32751]: pgmap v20835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:10:48.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38128.475813 secs 2024-04-27T05:10:48.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:47 smithi090 ceph-mon[29180]: pgmap v20835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:10:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:50 smithi119 ceph-mon[32752]: pgmap v20836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:50 smithi167 ceph-mon[32751]: pgmap v20836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:51.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:50 smithi090 ceph-mon[29180]: pgmap v20836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38132.575764 secs 2024-04-27T05:10:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38132.575764 secs 2024-04-27T05:10:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38132.575764 secs 2024-04-27T05:10:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38133.475915 secs 2024-04-27T05:10:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:52 smithi119 ceph-mon[32752]: pgmap v20837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:52.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38133.475915 secs 2024-04-27T05:10:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:52 smithi167 ceph-mon[32751]: pgmap v20837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38133.475915 secs 2024-04-27T05:10:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:52 smithi090 ceph-mon[29180]: pgmap v20837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:54 smithi119 ceph-mon[32752]: pgmap v20838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:54 smithi167 ceph-mon[32751]: pgmap v20838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:54 smithi090 ceph-mon[29180]: pgmap v20838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:10:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38137.575797 secs 2024-04-27T05:10:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:56 smithi119 ceph-mon[32752]: pgmap v20839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:10:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38137.575797 secs 2024-04-27T05:10:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:56 smithi167 ceph-mon[32751]: pgmap v20839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:10:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38137.575797 secs 2024-04-27T05:10:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:56 smithi090 ceph-mon[29180]: pgmap v20839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:10:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38138.476066 secs 2024-04-27T05:10:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38138.476066 secs 2024-04-27T05:10:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38138.476066 secs 2024-04-27T05:10:59.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:10:58 smithi119 ceph-mon[32752]: pgmap v20840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:10:58 smithi167 ceph-mon[32751]: pgmap v20840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:10:59.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:10:58 smithi090 ceph-mon[29180]: pgmap v20840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:00 smithi119 ceph-mon[32752]: pgmap v20841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:00 smithi167 ceph-mon[32751]: pgmap v20841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:01.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:00 smithi090 ceph-mon[29180]: pgmap v20841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38142.575858 secs 2024-04-27T05:11:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38142.575858 secs 2024-04-27T05:11:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38142.575858 secs 2024-04-27T05:11:03.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38143.476180 secs 2024-04-27T05:11:03.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:02 smithi090 ceph-mon[29180]: pgmap v20842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38143.476180 secs 2024-04-27T05:11:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:02 smithi119 ceph-mon[32752]: pgmap v20842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38143.476180 secs 2024-04-27T05:11:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:02 smithi167 ceph-mon[32751]: pgmap v20842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:03 smithi119 ceph-mon[32752]: pgmap v20843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:03 smithi167 ceph-mon[32751]: pgmap v20843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:03 smithi090 ceph-mon[29180]: pgmap v20843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38147.575909 secs 2024-04-27T05:11:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:06 smithi119 ceph-mon[32752]: pgmap v20844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38147.575909 secs 2024-04-27T05:11:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:06 smithi167 ceph-mon[32751]: pgmap v20844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38147.575909 secs 2024-04-27T05:11:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:06 smithi090 ceph-mon[29180]: pgmap v20844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:07 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 38148.476297 secs 2024-04-27T05:11:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:07 smithi119 ceph-mon[32752]: slow request 23041.344685 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:11:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:07 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 38148.476297 secs 2024-04-27T05:11:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:07 smithi167 ceph-mon[32751]: slow request 23041.344685 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:11:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:07 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 38148.476297 secs 2024-04-27T05:11:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:07 smithi090 ceph-mon[29180]: slow request 23041.344685 seconds old, received at 2024-04-26T22:47:04.882256+0000: client_request(client.17391:1 getattr pAsLsXsFs #0x1 2024-04-26T22:47:04.878605+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:11:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:08 smithi119 ceph-mon[32752]: pgmap v20845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:08 smithi167 ceph-mon[32751]: pgmap v20845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:08 smithi090 ceph-mon[29180]: pgmap v20845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:10 smithi119 ceph-mon[32752]: pgmap v20846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:10 smithi167 ceph-mon[32751]: pgmap v20846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:11.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:10 smithi090 ceph-mon[29180]: pgmap v20846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38152.575970 secs 2024-04-27T05:11:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38152.575970 secs 2024-04-27T05:11:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38152.575970 secs 2024-04-27T05:11:13.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38153.476456 secs 2024-04-27T05:11:13.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:12 smithi090 ceph-mon[29180]: pgmap v20847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38153.476456 secs 2024-04-27T05:11:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:12 smithi119 ceph-mon[32752]: pgmap v20847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38153.476456 secs 2024-04-27T05:11:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:12 smithi167 ceph-mon[32751]: pgmap v20847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:14 smithi119 ceph-mon[32752]: pgmap v20848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:14 smithi167 ceph-mon[32751]: pgmap v20848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:15.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:14 smithi090 ceph-mon[29180]: pgmap v20848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:15 smithi119 ceph-mon[32752]: pgmap v20849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:15 smithi167 ceph-mon[32751]: pgmap v20849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:15 smithi090 ceph-mon[29180]: pgmap v20849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38157.576020 secs 2024-04-27T05:11:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38157.576020 secs 2024-04-27T05:11:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38157.576020 secs 2024-04-27T05:11:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38158.476558 secs 2024-04-27T05:11:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:17 smithi119 ceph-mon[32752]: pgmap v20850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38158.476558 secs 2024-04-27T05:11:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:17 smithi167 ceph-mon[32751]: pgmap v20850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38158.476558 secs 2024-04-27T05:11:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:17 smithi090 ceph-mon[29180]: pgmap v20850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:20 smithi119 ceph-mon[32752]: pgmap v20851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:20 smithi167 ceph-mon[32751]: pgmap v20851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:20 smithi090 ceph-mon[29180]: pgmap v20851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38162.576078 secs 2024-04-27T05:11:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38162.576078 secs 2024-04-27T05:11:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38162.576078 secs 2024-04-27T05:11:22.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38163.476695 secs 2024-04-27T05:11:22.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:22 smithi119 ceph-mon[32752]: pgmap v20852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38163.476695 secs 2024-04-27T05:11:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:22 smithi167 ceph-mon[32751]: pgmap v20852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38163.476695 secs 2024-04-27T05:11:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:22 smithi090 ceph-mon[29180]: pgmap v20852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:24 smithi119 ceph-mon[32752]: pgmap v20853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:24 smithi167 ceph-mon[32751]: pgmap v20853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:24 smithi090 ceph-mon[29180]: pgmap v20853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38167.576132 secs 2024-04-27T05:11:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:26 smithi119 ceph-mon[32752]: pgmap v20854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38167.576132 secs 2024-04-27T05:11:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:26 smithi167 ceph-mon[32751]: pgmap v20854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38167.576132 secs 2024-04-27T05:11:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:26 smithi090 ceph-mon[29180]: pgmap v20854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38168.476807 secs 2024-04-27T05:11:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38168.476807 secs 2024-04-27T05:11:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38168.476807 secs 2024-04-27T05:11:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:28 smithi119 ceph-mon[32752]: pgmap v20855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:28 smithi167 ceph-mon[32751]: pgmap v20855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:28 smithi090 ceph-mon[29180]: pgmap v20855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:29 smithi119 ceph-mon[32752]: pgmap v20856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:29 smithi167 ceph-mon[32751]: pgmap v20856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:30.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:29 smithi090 ceph-mon[29180]: pgmap v20856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:31.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38172.576194 secs 2024-04-27T05:11:31.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38172.576194 secs 2024-04-27T05:11:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38172.576194 secs 2024-04-27T05:11:32.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38173.476891 secs 2024-04-27T05:11:32.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:32 smithi090 ceph-mon[29180]: pgmap v20857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:32.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38173.476891 secs 2024-04-27T05:11:32.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:32 smithi119 ceph-mon[32752]: pgmap v20857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38173.476891 secs 2024-04-27T05:11:32.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:32 smithi167 ceph-mon[32751]: pgmap v20857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:34 smithi119 ceph-mon[32752]: pgmap v20858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:34 smithi167 ceph-mon[32751]: pgmap v20858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:35.080 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:34 smithi090 ceph-mon[29180]: pgmap v20858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38177.576252 secs 2024-04-27T05:11:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:36 smithi119 ceph-mon[32752]: pgmap v20859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38177.576252 secs 2024-04-27T05:11:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:36 smithi167 ceph-mon[32751]: pgmap v20859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38177.576252 secs 2024-04-27T05:11:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:36 smithi090 ceph-mon[29180]: pgmap v20859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38178.476962 secs 2024-04-27T05:11:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38178.476962 secs 2024-04-27T05:11:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38178.476962 secs 2024-04-27T05:11:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:38 smithi119 ceph-mon[32752]: pgmap v20860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:38 smithi167 ceph-mon[32751]: pgmap v20860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:38 smithi090 ceph-mon[29180]: pgmap v20860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:40 smithi119 ceph-mon[32752]: pgmap v20861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:40 smithi167 ceph-mon[32751]: pgmap v20861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:41.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:40 smithi090 ceph-mon[29180]: pgmap v20861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38182.576311 secs 2024-04-27T05:11:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38182.576311 secs 2024-04-27T05:11:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38182.576311 secs 2024-04-27T05:11:43.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38183.477103 secs 2024-04-27T05:11:43.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:42 smithi090 ceph-mon[29180]: pgmap v20862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:43.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38183.477103 secs 2024-04-27T05:11:43.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:42 smithi119 ceph-mon[32752]: pgmap v20862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38183.477103 secs 2024-04-27T05:11:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:42 smithi167 ceph-mon[32751]: pgmap v20862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:43 smithi119 ceph-mon[32752]: pgmap v20863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:43 smithi167 ceph-mon[32751]: pgmap v20863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:43 smithi090 ceph-mon[29180]: pgmap v20863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38187.576366 secs 2024-04-27T05:11:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:46 smithi119 ceph-mon[32752]: pgmap v20864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:11:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:11:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:11:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:11:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38187.576366 secs 2024-04-27T05:11:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:46 smithi167 ceph-mon[32751]: pgmap v20864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:11:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:11:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:11:47.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:11:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38187.576366 secs 2024-04-27T05:11:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:46 smithi090 ceph-mon[29180]: pgmap v20864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:11:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:11:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:11:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:11:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:11:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38188.477193 secs 2024-04-27T05:11:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:47 smithi119 ceph-mon[32752]: pgmap v20865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:11:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:47 smithi119 ceph-mon[32752]: pgmap v20866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38188.477193 secs 2024-04-27T05:11:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:47 smithi167 ceph-mon[32751]: pgmap v20865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:11:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:47 smithi167 ceph-mon[32751]: pgmap v20866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38188.477193 secs 2024-04-27T05:11:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:47 smithi090 ceph-mon[29180]: pgmap v20865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:11:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:47 smithi090 ceph-mon[29180]: pgmap v20866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:49 smithi119 ceph-mon[32752]: pgmap v20867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:49 smithi167 ceph-mon[32751]: pgmap v20867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:49 smithi090 ceph-mon[29180]: pgmap v20867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38192.576427 secs 2024-04-27T05:11:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:51 smithi119 ceph-mon[32752]: pgmap v20868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38192.576427 secs 2024-04-27T05:11:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:51 smithi167 ceph-mon[32751]: pgmap v20868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38192.576427 secs 2024-04-27T05:11:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:51 smithi090 ceph-mon[29180]: pgmap v20868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38193.477267 secs 2024-04-27T05:11:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38193.477267 secs 2024-04-27T05:11:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38193.477267 secs 2024-04-27T05:11:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:53 smithi119 ceph-mon[32752]: pgmap v20869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T05:11:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:53 smithi167 ceph-mon[32751]: pgmap v20869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T05:11:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:53 smithi090 ceph-mon[29180]: pgmap v20869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 582 B/s rd, 1 op/s 2024-04-27T05:11:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:55 smithi119 ceph-mon[32752]: pgmap v20870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:55 smithi167 ceph-mon[32751]: pgmap v20870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:55 smithi090 ceph-mon[29180]: pgmap v20870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:11:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38197.576482 secs 2024-04-27T05:11:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:56 smithi119 ceph-mon[32752]: pgmap v20871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:11:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38197.576482 secs 2024-04-27T05:11:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:56 smithi167 ceph-mon[32751]: pgmap v20871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:11:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38197.576482 secs 2024-04-27T05:11:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:56 smithi090 ceph-mon[29180]: pgmap v20871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:11:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38198.477399 secs 2024-04-27T05:11:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38198.477399 secs 2024-04-27T05:11:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38198.477399 secs 2024-04-27T05:11:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:11:58 smithi119 ceph-mon[32752]: pgmap v20872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:11:58 smithi167 ceph-mon[32751]: pgmap v20872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:11:59.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:11:58 smithi090 ceph-mon[29180]: pgmap v20872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38202.576567 secs 2024-04-27T05:12:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:01 smithi119 ceph-mon[32752]: pgmap v20873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38202.576567 secs 2024-04-27T05:12:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:01 smithi167 ceph-mon[32751]: pgmap v20873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38202.576567 secs 2024-04-27T05:12:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:01 smithi090 ceph-mon[29180]: pgmap v20873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38203.477489 secs 2024-04-27T05:12:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38203.477489 secs 2024-04-27T05:12:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38203.477489 secs 2024-04-27T05:12:03.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:03 smithi119 ceph-mon[32752]: pgmap v20874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:03 smithi167 ceph-mon[32751]: pgmap v20874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:03 smithi090 ceph-mon[29180]: pgmap v20874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:05 smithi119 ceph-mon[32752]: pgmap v20875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:05 smithi167 ceph-mon[32751]: pgmap v20875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:05.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:05 smithi090 ceph-mon[29180]: pgmap v20875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38207.576645 secs 2024-04-27T05:12:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38207.576645 secs 2024-04-27T05:12:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38207.576645 secs 2024-04-27T05:12:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38208.477638 secs 2024-04-27T05:12:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:07 smithi119 ceph-mon[32752]: pgmap v20876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38208.477638 secs 2024-04-27T05:12:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:07 smithi167 ceph-mon[32751]: pgmap v20876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38208.477638 secs 2024-04-27T05:12:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:07 smithi090 ceph-mon[29180]: pgmap v20876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:09 smithi119 ceph-mon[32752]: pgmap v20877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:09 smithi167 ceph-mon[32751]: pgmap v20877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:09 smithi090 ceph-mon[29180]: pgmap v20877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38212.576729 secs 2024-04-27T05:12:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:11 smithi119 ceph-mon[32752]: pgmap v20878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38212.576729 secs 2024-04-27T05:12:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:11 smithi167 ceph-mon[32751]: pgmap v20878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38212.576729 secs 2024-04-27T05:12:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:11 smithi090 ceph-mon[29180]: pgmap v20878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:11.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:12:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:12:11.404+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:12:11.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:12:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:12:11.404+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:12:11.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:12:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:12:11.404+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:12:11.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:12:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:12:11.404+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:12:11.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:12:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:12:11.404+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:12:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38213.477771 secs 2024-04-27T05:12:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38213.477771 secs 2024-04-27T05:12:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38213.477771 secs 2024-04-27T05:12:13.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:13 smithi090 ceph-mon[29180]: pgmap v20879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:13 smithi119 ceph-mon[32752]: pgmap v20879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:13 smithi167 ceph-mon[32751]: pgmap v20879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:15 smithi119 ceph-mon[32752]: pgmap v20880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:15 smithi167 ceph-mon[32751]: pgmap v20880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:15 smithi090 ceph-mon[29180]: pgmap v20880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38217.576818 secs 2024-04-27T05:12:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38217.576818 secs 2024-04-27T05:12:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38217.576818 secs 2024-04-27T05:12:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38218.477883 secs 2024-04-27T05:12:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:17 smithi119 ceph-mon[32752]: pgmap v20881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38218.477883 secs 2024-04-27T05:12:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:17 smithi167 ceph-mon[32751]: pgmap v20881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38218.477883 secs 2024-04-27T05:12:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:17 smithi090 ceph-mon[29180]: pgmap v20881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:19 smithi119 ceph-mon[32752]: pgmap v20882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:19 smithi167 ceph-mon[32751]: pgmap v20882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:19 smithi090 ceph-mon[29180]: pgmap v20882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38222.576902 secs 2024-04-27T05:12:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:21 smithi119 ceph-mon[32752]: pgmap v20883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38222.576902 secs 2024-04-27T05:12:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:21 smithi167 ceph-mon[32751]: pgmap v20883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38222.576902 secs 2024-04-27T05:12:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:21 smithi090 ceph-mon[29180]: pgmap v20883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38223.477993 secs 2024-04-27T05:12:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38223.477993 secs 2024-04-27T05:12:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38223.477993 secs 2024-04-27T05:12:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:23 smithi119 ceph-mon[32752]: pgmap v20884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:23 smithi167 ceph-mon[32751]: pgmap v20884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:23 smithi090 ceph-mon[29180]: pgmap v20884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:12:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:25 smithi119 ceph-mon[32752]: pgmap v20885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:25 smithi167 ceph-mon[32751]: pgmap v20885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:25 smithi090 ceph-mon[29180]: pgmap v20885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:12:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38227.576988 secs 2024-04-27T05:12:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38227.576988 secs 2024-04-27T05:12:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38227.576988 secs 2024-04-27T05:12:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38228.478109 secs 2024-04-27T05:12:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:27 smithi119 ceph-mon[32752]: pgmap v20886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38228.478109 secs 2024-04-27T05:12:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:27 smithi167 ceph-mon[32751]: pgmap v20886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38228.478109 secs 2024-04-27T05:12:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:27 smithi090 ceph-mon[29180]: pgmap v20886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:29 smithi119 ceph-mon[32752]: pgmap v20887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:29 smithi167 ceph-mon[32751]: pgmap v20887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:29.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:29 smithi090 ceph-mon[29180]: pgmap v20887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38232.577093 secs 2024-04-27T05:12:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:31 smithi119 ceph-mon[32752]: pgmap v20888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38232.577093 secs 2024-04-27T05:12:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:31 smithi167 ceph-mon[32751]: pgmap v20888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38232.577093 secs 2024-04-27T05:12:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:31 smithi090 ceph-mon[29180]: pgmap v20888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38233.478228 secs 2024-04-27T05:12:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38233.478228 secs 2024-04-27T05:12:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38233.478228 secs 2024-04-27T05:12:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:33 smithi119 ceph-mon[32752]: pgmap v20889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:33 smithi167 ceph-mon[32751]: pgmap v20889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:33.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:33 smithi090 ceph-mon[29180]: pgmap v20889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:35 smithi119 ceph-mon[32752]: pgmap v20890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:35 smithi167 ceph-mon[32751]: pgmap v20890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:35.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:35 smithi090 ceph-mon[29180]: pgmap v20890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38237.577177 secs 2024-04-27T05:12:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38237.577177 secs 2024-04-27T05:12:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38237.577177 secs 2024-04-27T05:12:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38238.478334 secs 2024-04-27T05:12:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:37 smithi119 ceph-mon[32752]: pgmap v20891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38238.478334 secs 2024-04-27T05:12:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:37 smithi167 ceph-mon[32751]: pgmap v20891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38238.478334 secs 2024-04-27T05:12:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:37 smithi090 ceph-mon[29180]: pgmap v20891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:39 smithi119 ceph-mon[32752]: pgmap v20892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:39 smithi167 ceph-mon[32751]: pgmap v20892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:39.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:39 smithi090 ceph-mon[29180]: pgmap v20892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38242.577238 secs 2024-04-27T05:12:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:41 smithi119 ceph-mon[32752]: pgmap v20893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38242.577238 secs 2024-04-27T05:12:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:41 smithi167 ceph-mon[32751]: pgmap v20893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38242.577238 secs 2024-04-27T05:12:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:41 smithi090 ceph-mon[29180]: pgmap v20893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38243.478450 secs 2024-04-27T05:12:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38243.478450 secs 2024-04-27T05:12:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38243.478450 secs 2024-04-27T05:12:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:43 smithi119 ceph-mon[32752]: pgmap v20894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:43 smithi167 ceph-mon[32751]: pgmap v20894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:43 smithi090 ceph-mon[29180]: pgmap v20894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:45 smithi167 ceph-mon[32751]: pgmap v20895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:45 smithi090 ceph-mon[29180]: pgmap v20895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:45 smithi119 ceph-mon[32752]: pgmap v20895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:12:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38247.577298 secs 2024-04-27T05:12:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:46 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:12:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38247.577298 secs 2024-04-27T05:12:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:46 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:12:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38247.577298 secs 2024-04-27T05:12:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:46 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:12:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38248.478510 secs 2024-04-27T05:12:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:47 smithi167 ceph-mon[32751]: pgmap v20896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:12:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:12:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:47 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:12:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38248.478510 secs 2024-04-27T05:12:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:47 smithi090 ceph-mon[29180]: pgmap v20896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:12:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:12:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:47 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:12:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38248.478510 secs 2024-04-27T05:12:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:47 smithi119 ceph-mon[32752]: pgmap v20896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:12:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:12:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:12:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:47 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:12:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:48 smithi167 ceph-mon[32751]: pgmap v20897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:12:48.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:48 smithi167 ceph-mon[32751]: pgmap v20898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:12:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:48 smithi090 ceph-mon[29180]: pgmap v20897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:12:48.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:48 smithi090 ceph-mon[29180]: pgmap v20898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:12:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:48 smithi119 ceph-mon[32752]: pgmap v20897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:12:49.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:48 smithi119 ceph-mon[32752]: pgmap v20898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:12:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:50 smithi167 ceph-mon[32751]: pgmap v20899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:50 smithi090 ceph-mon[29180]: pgmap v20899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:50 smithi119 ceph-mon[32752]: pgmap v20899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38252.577384 secs 2024-04-27T05:12:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38252.577384 secs 2024-04-27T05:12:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38252.577384 secs 2024-04-27T05:12:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:52 smithi090 ceph-mon[29180]: pgmap v20900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38253.478621 secs 2024-04-27T05:12:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:52 smithi119 ceph-mon[32752]: pgmap v20900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38253.478621 secs 2024-04-27T05:12:52.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:52 smithi167 ceph-mon[32751]: pgmap v20900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38253.478621 secs 2024-04-27T05:12:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:54 smithi090 ceph-mon[29180]: pgmap v20901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:12:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:54 smithi119 ceph-mon[32752]: pgmap v20901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:12:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:54 smithi167 ceph-mon[32751]: pgmap v20901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:12:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:56 smithi090 ceph-mon[29180]: pgmap v20902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38257.577442 secs 2024-04-27T05:12:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:56 smithi119 ceph-mon[32752]: pgmap v20902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38257.577442 secs 2024-04-27T05:12:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:56 smithi167 ceph-mon[32751]: pgmap v20902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:12:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38257.577442 secs 2024-04-27T05:12:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38258.478769 secs 2024-04-27T05:12:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38258.478769 secs 2024-04-27T05:12:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38258.478769 secs 2024-04-27T05:12:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:12:58 smithi090 ceph-mon[29180]: pgmap v20903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:12:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:12:58 smithi119 ceph-mon[32752]: pgmap v20903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:12:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:12:58 smithi167 ceph-mon[32751]: pgmap v20903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:13:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:00 smithi090 ceph-mon[29180]: pgmap v20904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:00 smithi119 ceph-mon[32752]: pgmap v20904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:00 smithi167 ceph-mon[32751]: pgmap v20904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38262.577531 secs 2024-04-27T05:13:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38262.577531 secs 2024-04-27T05:13:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38262.577531 secs 2024-04-27T05:13:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:02 smithi090 ceph-mon[29180]: pgmap v20905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38263.478872 secs 2024-04-27T05:13:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:02 smithi119 ceph-mon[32752]: pgmap v20905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38263.478872 secs 2024-04-27T05:13:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:02 smithi167 ceph-mon[32751]: pgmap v20905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38263.478872 secs 2024-04-27T05:13:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:04 smithi090 ceph-mon[29180]: pgmap v20906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:04 smithi119 ceph-mon[32752]: pgmap v20906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:04 smithi167 ceph-mon[32751]: pgmap v20906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:06 smithi090 ceph-mon[29180]: pgmap v20907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38267.577613 secs 2024-04-27T05:13:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:06 smithi119 ceph-mon[32752]: pgmap v20907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38267.577613 secs 2024-04-27T05:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:06 smithi167 ceph-mon[32751]: pgmap v20907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38267.577613 secs 2024-04-27T05:13:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38268.479007 secs 2024-04-27T05:13:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38268.479007 secs 2024-04-27T05:13:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38268.479007 secs 2024-04-27T05:13:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:08 smithi090 ceph-mon[29180]: pgmap v20908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:08 smithi119 ceph-mon[32752]: pgmap v20908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:08 smithi167 ceph-mon[32751]: pgmap v20908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:10.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:10 smithi090 ceph-mon[29180]: pgmap v20909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:11.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:10 smithi119 ceph-mon[32752]: pgmap v20909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:10 smithi167 ceph-mon[32751]: pgmap v20909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38272.577698 secs 2024-04-27T05:13:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38272.577698 secs 2024-04-27T05:13:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38272.577698 secs 2024-04-27T05:13:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:12 smithi090 ceph-mon[29180]: pgmap v20910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38273.479123 secs 2024-04-27T05:13:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:12 smithi119 ceph-mon[32752]: pgmap v20910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38273.479123 secs 2024-04-27T05:13:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:12 smithi167 ceph-mon[32751]: pgmap v20910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38273.479123 secs 2024-04-27T05:13:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:14 smithi090 ceph-mon[29180]: pgmap v20911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:14 smithi119 ceph-mon[32752]: pgmap v20911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:14 smithi167 ceph-mon[32751]: pgmap v20911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:16 smithi119 ceph-mon[32752]: pgmap v20912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38277.577779 secs 2024-04-27T05:13:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:16 smithi167 ceph-mon[32751]: pgmap v20912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38277.577779 secs 2024-04-27T05:13:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:16 smithi090 ceph-mon[29180]: pgmap v20912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38277.577779 secs 2024-04-27T05:13:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38278.479230 secs 2024-04-27T05:13:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38278.479230 secs 2024-04-27T05:13:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38278.479230 secs 2024-04-27T05:13:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:18 smithi119 ceph-mon[32752]: pgmap v20913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:18 smithi167 ceph-mon[32751]: pgmap v20913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:18 smithi090 ceph-mon[29180]: pgmap v20913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:20 smithi119 ceph-mon[32752]: pgmap v20914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:20 smithi167 ceph-mon[32751]: pgmap v20914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:21.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:20 smithi090 ceph-mon[29180]: pgmap v20914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38282.577824 secs 2024-04-27T05:13:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38282.577824 secs 2024-04-27T05:13:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38282.577824 secs 2024-04-27T05:13:22.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:22 smithi119 ceph-mon[32752]: pgmap v20915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:22.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38283.479340 secs 2024-04-27T05:13:22.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:22 smithi167 ceph-mon[32751]: pgmap v20915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38283.479340 secs 2024-04-27T05:13:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:22 smithi090 ceph-mon[29180]: pgmap v20915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38283.479340 secs 2024-04-27T05:13:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:24 smithi119 ceph-mon[32752]: pgmap v20916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:24 smithi167 ceph-mon[32751]: pgmap v20916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:25.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:24 smithi090 ceph-mon[29180]: pgmap v20916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:26 smithi119 ceph-mon[32752]: pgmap v20917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38287.577905 secs 2024-04-27T05:13:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:26 smithi167 ceph-mon[32751]: pgmap v20917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38287.577905 secs 2024-04-27T05:13:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:26 smithi090 ceph-mon[29180]: pgmap v20917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38287.577905 secs 2024-04-27T05:13:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38288.479481 secs 2024-04-27T05:13:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38288.479481 secs 2024-04-27T05:13:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38288.479481 secs 2024-04-27T05:13:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:28 smithi119 ceph-mon[32752]: pgmap v20918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:29.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:28 smithi167 ceph-mon[32751]: pgmap v20918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:29.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:28 smithi090 ceph-mon[29180]: pgmap v20918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:30 smithi119 ceph-mon[32752]: pgmap v20919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:30 smithi167 ceph-mon[32751]: pgmap v20919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:30 smithi090 ceph-mon[29180]: pgmap v20919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38292.577991 secs 2024-04-27T05:13:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38292.577991 secs 2024-04-27T05:13:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38292.577991 secs 2024-04-27T05:13:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:32 smithi119 ceph-mon[32752]: pgmap v20920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38293.479578 secs 2024-04-27T05:13:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:32 smithi167 ceph-mon[32751]: pgmap v20920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38293.479578 secs 2024-04-27T05:13:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:32 smithi090 ceph-mon[29180]: pgmap v20920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38293.479578 secs 2024-04-27T05:13:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:34 smithi119 ceph-mon[32752]: pgmap v20921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:34 smithi167 ceph-mon[32751]: pgmap v20921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:35.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:34 smithi090 ceph-mon[29180]: pgmap v20921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:36 smithi119 ceph-mon[32752]: pgmap v20922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38297.578071 secs 2024-04-27T05:13:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:36 smithi167 ceph-mon[32751]: pgmap v20922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38297.578071 secs 2024-04-27T05:13:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:36 smithi090 ceph-mon[29180]: pgmap v20922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38297.578071 secs 2024-04-27T05:13:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38298.479644 secs 2024-04-27T05:13:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38298.479644 secs 2024-04-27T05:13:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38298.479644 secs 2024-04-27T05:13:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:38 smithi119 ceph-mon[32752]: pgmap v20923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:38 smithi167 ceph-mon[32751]: pgmap v20923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:39.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:38 smithi090 ceph-mon[29180]: pgmap v20923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:40 smithi119 ceph-mon[32752]: pgmap v20924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:40 smithi167 ceph-mon[32751]: pgmap v20924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:41.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:40 smithi090 ceph-mon[29180]: pgmap v20924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38302.578151 secs 2024-04-27T05:13:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38302.578151 secs 2024-04-27T05:13:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38302.578151 secs 2024-04-27T05:13:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:42 smithi119 ceph-mon[32752]: pgmap v20925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 38303.479762 secs 2024-04-27T05:13:43.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:42 smithi119 ceph-mon[32752]: slow request 720.000818 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:13:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:42 smithi167 ceph-mon[32751]: pgmap v20925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 38303.479762 secs 2024-04-27T05:13:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:42 smithi167 ceph-mon[32751]: slow request 720.000818 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:13:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:42 smithi090 ceph-mon[29180]: pgmap v20925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 38303.479762 secs 2024-04-27T05:13:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:42 smithi090 ceph-mon[29180]: slow request 720.000818 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:13:44.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:43 smithi090 ceph-mon[29180]: pgmap v20926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:43 smithi119 ceph-mon[32752]: pgmap v20926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:43 smithi167 ceph-mon[32751]: pgmap v20926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:46 smithi119 ceph-mon[32752]: pgmap v20927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38307.578237 secs 2024-04-27T05:13:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:46 smithi167 ceph-mon[32751]: pgmap v20927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38307.578237 secs 2024-04-27T05:13:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:46 smithi090 ceph-mon[29180]: pgmap v20927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:13:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38307.578237 secs 2024-04-27T05:13:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38308.479870 secs 2024-04-27T05:13:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38308.479870 secs 2024-04-27T05:13:47.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38308.479870 secs 2024-04-27T05:13:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:48 smithi119 ceph-mon[32752]: pgmap v20928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:48.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:13:48.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:13:48.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:13:48.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:13:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:48 smithi167 ceph-mon[32751]: pgmap v20928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:13:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:13:48.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:13:48.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:13:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:48 smithi090 ceph-mon[29180]: pgmap v20928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:13:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:13:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:13:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:13:48.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:13:49.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:49 smithi119 ceph-mon[32752]: pgmap v20929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:13:49.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:49 smithi119 ceph-mon[32752]: pgmap v20930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:13:49.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:49 smithi167 ceph-mon[32751]: pgmap v20929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:13:49.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:49 smithi167 ceph-mon[32751]: pgmap v20930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:13:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:49 smithi090 ceph-mon[29180]: pgmap v20929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:13:49.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:49 smithi090 ceph-mon[29180]: pgmap v20930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:13:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:51 smithi119 ceph-mon[32752]: pgmap v20931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:51.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38312.578299 secs 2024-04-27T05:13:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:51 smithi167 ceph-mon[32751]: pgmap v20931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38312.578299 secs 2024-04-27T05:13:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:51 smithi090 ceph-mon[29180]: pgmap v20931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38312.578299 secs 2024-04-27T05:13:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38313.480011 secs 2024-04-27T05:13:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38313.480011 secs 2024-04-27T05:13:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38313.480011 secs 2024-04-27T05:13:53.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:53 smithi119 ceph-mon[32752]: pgmap v20932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:53.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:53 smithi167 ceph-mon[32751]: pgmap v20932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:53.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:53 smithi090 ceph-mon[29180]: pgmap v20932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:55.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:55 smithi119 ceph-mon[32752]: pgmap v20933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:55.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:55 smithi167 ceph-mon[32751]: pgmap v20933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:55.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:55 smithi090 ceph-mon[29180]: pgmap v20933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38317.578381 secs 2024-04-27T05:13:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38317.578381 secs 2024-04-27T05:13:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38317.578381 secs 2024-04-27T05:13:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:57 smithi119 ceph-mon[32752]: pgmap v20934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:57.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38318.480125 secs 2024-04-27T05:13:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:57 smithi167 ceph-mon[32751]: pgmap v20934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38318.480125 secs 2024-04-27T05:13:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:57 smithi090 ceph-mon[29180]: pgmap v20934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:13:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38318.480125 secs 2024-04-27T05:13:59.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:13:59 smithi119 ceph-mon[32752]: pgmap v20935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:13:59.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:13:59 smithi167 ceph-mon[32751]: pgmap v20935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:13:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:13:59 smithi090 ceph-mon[29180]: pgmap v20935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:14:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:01 smithi119 ceph-mon[32752]: pgmap v20936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:01.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38322.578465 secs 2024-04-27T05:14:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:01 smithi167 ceph-mon[32751]: pgmap v20936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38322.578465 secs 2024-04-27T05:14:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:01 smithi090 ceph-mon[29180]: pgmap v20936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38322.578465 secs 2024-04-27T05:14:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38323.480206 secs 2024-04-27T05:14:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38323.480206 secs 2024-04-27T05:14:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38323.480206 secs 2024-04-27T05:14:03.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:03 smithi119 ceph-mon[32752]: pgmap v20937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:03.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:03 smithi167 ceph-mon[32751]: pgmap v20937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:03.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:03 smithi090 ceph-mon[29180]: pgmap v20937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:05.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:05 smithi119 ceph-mon[32752]: pgmap v20938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:05.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:05 smithi167 ceph-mon[32751]: pgmap v20938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:05 smithi090 ceph-mon[29180]: pgmap v20938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38327.578542 secs 2024-04-27T05:14:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38327.578542 secs 2024-04-27T05:14:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38327.578542 secs 2024-04-27T05:14:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:07 smithi119 ceph-mon[32752]: pgmap v20939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:07.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38328.480322 secs 2024-04-27T05:14:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:07 smithi167 ceph-mon[32751]: pgmap v20939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38328.480322 secs 2024-04-27T05:14:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:07 smithi090 ceph-mon[29180]: pgmap v20939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38328.480322 secs 2024-04-27T05:14:09.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:09 smithi167 ceph-mon[32751]: pgmap v20940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:09 smithi090 ceph-mon[29180]: pgmap v20940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:09 smithi119 ceph-mon[32752]: pgmap v20940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:11 smithi167 ceph-mon[32751]: pgmap v20941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:11.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38332.578622 secs 2024-04-27T05:14:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:11 smithi090 ceph-mon[29180]: pgmap v20941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38332.578622 secs 2024-04-27T05:14:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:11 smithi119 ceph-mon[32752]: pgmap v20941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38332.578622 secs 2024-04-27T05:14:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38333.480438 secs 2024-04-27T05:14:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38333.480438 secs 2024-04-27T05:14:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38333.480438 secs 2024-04-27T05:14:13.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:13 smithi167 ceph-mon[32751]: pgmap v20942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:13 smithi090 ceph-mon[29180]: pgmap v20942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:13 smithi119 ceph-mon[32752]: pgmap v20942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:15 smithi090 ceph-mon[29180]: pgmap v20943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:15 smithi119 ceph-mon[32752]: pgmap v20943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:15 smithi167 ceph-mon[32751]: pgmap v20943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38337.578708 secs 2024-04-27T05:14:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38337.578708 secs 2024-04-27T05:14:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38337.578708 secs 2024-04-27T05:14:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:17 smithi090 ceph-mon[29180]: pgmap v20944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38338.480560 secs 2024-04-27T05:14:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:17 smithi119 ceph-mon[32752]: pgmap v20944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38338.480560 secs 2024-04-27T05:14:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:17 smithi167 ceph-mon[32751]: pgmap v20944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38338.480560 secs 2024-04-27T05:14:19.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:19 smithi090 ceph-mon[29180]: pgmap v20945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:19 smithi119 ceph-mon[32752]: pgmap v20945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:19 smithi167 ceph-mon[32751]: pgmap v20945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:21 smithi090 ceph-mon[29180]: pgmap v20946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38342.578816 secs 2024-04-27T05:14:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:21 smithi119 ceph-mon[32752]: pgmap v20946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38342.578816 secs 2024-04-27T05:14:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:21 smithi167 ceph-mon[32751]: pgmap v20946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38342.578816 secs 2024-04-27T05:14:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38343.480636 secs 2024-04-27T05:14:22.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38343.480636 secs 2024-04-27T05:14:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38343.480636 secs 2024-04-27T05:14:23.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:23 smithi090 ceph-mon[29180]: pgmap v20947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:23 smithi119 ceph-mon[32752]: pgmap v20947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:23 smithi167 ceph-mon[32751]: pgmap v20947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:25 smithi090 ceph-mon[29180]: pgmap v20948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:25 smithi119 ceph-mon[32752]: pgmap v20948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:25 smithi167 ceph-mon[32751]: pgmap v20948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38347.578901 secs 2024-04-27T05:14:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38347.578901 secs 2024-04-27T05:14:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38347.578901 secs 2024-04-27T05:14:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:27 smithi090 ceph-mon[29180]: pgmap v20949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:27.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38348.480733 secs 2024-04-27T05:14:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:27 smithi119 ceph-mon[32752]: pgmap v20949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38348.480733 secs 2024-04-27T05:14:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:27 smithi167 ceph-mon[32751]: pgmap v20949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38348.480733 secs 2024-04-27T05:14:29.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:29 smithi090 ceph-mon[29180]: pgmap v20950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:29 smithi119 ceph-mon[32752]: pgmap v20950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:29 smithi167 ceph-mon[32751]: pgmap v20950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:31 smithi090 ceph-mon[29180]: pgmap v20951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:31.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38352.578964 secs 2024-04-27T05:14:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:31 smithi119 ceph-mon[32752]: pgmap v20951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38352.578964 secs 2024-04-27T05:14:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:31 smithi167 ceph-mon[32751]: pgmap v20951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38352.578964 secs 2024-04-27T05:14:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38353.480851 secs 2024-04-27T05:14:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38353.480851 secs 2024-04-27T05:14:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38353.480851 secs 2024-04-27T05:14:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:33 smithi090 ceph-mon[29180]: pgmap v20952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:33 smithi119 ceph-mon[32752]: pgmap v20952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:33 smithi167 ceph-mon[32751]: pgmap v20952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:35 smithi090 ceph-mon[29180]: pgmap v20953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:35 smithi119 ceph-mon[32752]: pgmap v20953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:35 smithi167 ceph-mon[32751]: pgmap v20953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38357.579047 secs 2024-04-27T05:14:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38357.579047 secs 2024-04-27T05:14:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38357.579047 secs 2024-04-27T05:14:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:37 smithi119 ceph-mon[32752]: pgmap v20954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38358.480916 secs 2024-04-27T05:14:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:37 smithi167 ceph-mon[32751]: pgmap v20954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38358.480916 secs 2024-04-27T05:14:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:37 smithi090 ceph-mon[29180]: pgmap v20954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38358.480916 secs 2024-04-27T05:14:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:39 smithi119 ceph-mon[32752]: pgmap v20955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:39 smithi167 ceph-mon[32751]: pgmap v20955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:39 smithi090 ceph-mon[29180]: pgmap v20955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:41.679 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:41 smithi090 ceph-mon[29180]: pgmap v20956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:41.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38362.579131 secs 2024-04-27T05:14:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:41 smithi119 ceph-mon[32752]: pgmap v20956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38362.579131 secs 2024-04-27T05:14:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:41 smithi167 ceph-mon[32751]: pgmap v20956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38362.579131 secs 2024-04-27T05:14:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38363.481049 secs 2024-04-27T05:14:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38363.481049 secs 2024-04-27T05:14:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38363.481049 secs 2024-04-27T05:14:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:43 smithi119 ceph-mon[32752]: pgmap v20957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:43 smithi167 ceph-mon[32751]: pgmap v20957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:43 smithi090 ceph-mon[29180]: pgmap v20957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:45.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:45 smithi090 ceph-mon[29180]: pgmap v20958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:45 smithi119 ceph-mon[32752]: pgmap v20958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:45 smithi167 ceph-mon[32751]: pgmap v20958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:14:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38367.579209 secs 2024-04-27T05:14:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38367.579209 secs 2024-04-27T05:14:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38367.579209 secs 2024-04-27T05:14:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:47 smithi119 ceph-mon[32752]: pgmap v20959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38368.481168 secs 2024-04-27T05:14:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:47 smithi167 ceph-mon[32751]: pgmap v20959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38368.481168 secs 2024-04-27T05:14:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:47 smithi090 ceph-mon[29180]: pgmap v20959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38368.481168 secs 2024-04-27T05:14:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:14:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:14:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:14:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:49 smithi119 ceph-mon[32752]: pgmap v20960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:14:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:14:49.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:14:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:49 smithi167 ceph-mon[32751]: pgmap v20960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:14:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:14:49.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:14:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:49 smithi090 ceph-mon[29180]: pgmap v20960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:14:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:14:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:14:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:14:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:50 smithi119 ceph-mon[32752]: pgmap v20961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:14:50.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:50 smithi119 ceph-mon[32752]: pgmap v20962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:50 smithi167 ceph-mon[32751]: pgmap v20961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:14:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:50 smithi167 ceph-mon[32751]: pgmap v20962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:50 smithi090 ceph-mon[29180]: pgmap v20961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:14:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:50 smithi090 ceph-mon[29180]: pgmap v20962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38372.579290 secs 2024-04-27T05:14:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38372.579290 secs 2024-04-27T05:14:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38372.579290 secs 2024-04-27T05:14:52.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:52 smithi119 ceph-mon[32752]: pgmap v20963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38373.481301 secs 2024-04-27T05:14:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:52 smithi167 ceph-mon[32751]: pgmap v20963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38373.481301 secs 2024-04-27T05:14:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:52 smithi090 ceph-mon[29180]: pgmap v20963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38373.481301 secs 2024-04-27T05:14:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:54 smithi119 ceph-mon[32752]: pgmap v20964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:14:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:54 smithi167 ceph-mon[32751]: pgmap v20964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:14:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:54 smithi090 ceph-mon[29180]: pgmap v20964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:14:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:56 smithi119 ceph-mon[32752]: pgmap v20965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38377.579371 secs 2024-04-27T05:14:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:56 smithi167 ceph-mon[32751]: pgmap v20965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38377.579371 secs 2024-04-27T05:14:56.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:56 smithi090 ceph-mon[29180]: pgmap v20965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:56.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38377.579371 secs 2024-04-27T05:14:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38378.481408 secs 2024-04-27T05:14:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38378.481408 secs 2024-04-27T05:14:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38378.481408 secs 2024-04-27T05:14:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:14:58 smithi119 ceph-mon[32752]: pgmap v20966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:14:58 smithi167 ceph-mon[32751]: pgmap v20966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:14:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:14:58 smithi090 ceph-mon[29180]: pgmap v20966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:00 smithi119 ceph-mon[32752]: pgmap v20967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:15:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:00 smithi167 ceph-mon[32751]: pgmap v20967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:15:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:00 smithi090 ceph-mon[29180]: pgmap v20967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:15:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38382.579456 secs 2024-04-27T05:15:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38382.579456 secs 2024-04-27T05:15:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38382.579456 secs 2024-04-27T05:15:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:02 smithi119 ceph-mon[32752]: pgmap v20968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38383.481470 secs 2024-04-27T05:15:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:02 smithi167 ceph-mon[32751]: pgmap v20968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:02.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38383.481470 secs 2024-04-27T05:15:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:02 smithi090 ceph-mon[29180]: pgmap v20968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38383.481470 secs 2024-04-27T05:15:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:04 smithi119 ceph-mon[32752]: pgmap v20969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:04 smithi167 ceph-mon[32751]: pgmap v20969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:04 smithi090 ceph-mon[29180]: pgmap v20969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:06 smithi119 ceph-mon[32752]: pgmap v20970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38387.579535 secs 2024-04-27T05:15:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:06 smithi167 ceph-mon[32751]: pgmap v20970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38387.579535 secs 2024-04-27T05:15:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:06 smithi090 ceph-mon[29180]: pgmap v20970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38387.579535 secs 2024-04-27T05:15:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38388.481558 secs 2024-04-27T05:15:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38388.481558 secs 2024-04-27T05:15:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38388.481558 secs 2024-04-27T05:15:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:08 smithi119 ceph-mon[32752]: pgmap v20971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:08 smithi167 ceph-mon[32751]: pgmap v20971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:08 smithi090 ceph-mon[29180]: pgmap v20971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:10.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:10 smithi119 ceph-mon[32752]: pgmap v20972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:10 smithi167 ceph-mon[32751]: pgmap v20972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:10 smithi090 ceph-mon[29180]: pgmap v20972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38392.579619 secs 2024-04-27T05:15:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38392.579619 secs 2024-04-27T05:15:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38392.579619 secs 2024-04-27T05:15:12.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:12 smithi090 ceph-mon[29180]: pgmap v20973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:12.743 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38393.481678 secs 2024-04-27T05:15:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:12 smithi119 ceph-mon[32752]: pgmap v20973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38393.481678 secs 2024-04-27T05:15:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:12 smithi167 ceph-mon[32751]: pgmap v20973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38393.481678 secs 2024-04-27T05:15:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:14 smithi119 ceph-mon[32752]: pgmap v20974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:14.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:14 smithi167 ceph-mon[32751]: pgmap v20974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:14 smithi090 ceph-mon[29180]: pgmap v20974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:16 smithi119 ceph-mon[32752]: pgmap v20975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38397.579701 secs 2024-04-27T05:15:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:16 smithi167 ceph-mon[32751]: pgmap v20975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38397.579701 secs 2024-04-27T05:15:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:16 smithi090 ceph-mon[29180]: pgmap v20975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:16.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38397.579701 secs 2024-04-27T05:15:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38398.481791 secs 2024-04-27T05:15:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38398.481791 secs 2024-04-27T05:15:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38398.481791 secs 2024-04-27T05:15:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:18 smithi119 ceph-mon[32752]: pgmap v20976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:18.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:18 smithi167 ceph-mon[32751]: pgmap v20976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:18.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:18 smithi090 ceph-mon[29180]: pgmap v20976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:20 smithi119 ceph-mon[32752]: pgmap v20977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:20 smithi167 ceph-mon[32751]: pgmap v20977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:20 smithi090 ceph-mon[29180]: pgmap v20977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:21 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 38402.579782 secs 2024-04-27T05:15:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:21 smithi119 ceph-mon[32752]: slow request 1441.460821 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:15:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:21 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 38402.579782 secs 2024-04-27T05:15:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:21 smithi167 ceph-mon[32751]: slow request 1441.460821 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:15:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:21 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 38402.579782 secs 2024-04-27T05:15:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:21 smithi090 ceph-mon[29180]: slow request 1441.460821 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:15:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:22 smithi119 ceph-mon[32752]: pgmap v20978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38403.481882 secs 2024-04-27T05:15:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:22 smithi167 ceph-mon[32751]: pgmap v20978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38403.481882 secs 2024-04-27T05:15:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:22 smithi090 ceph-mon[29180]: pgmap v20978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38403.481882 secs 2024-04-27T05:15:24.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:24 smithi119 ceph-mon[32752]: pgmap v20979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:24.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:24 smithi167 ceph-mon[32751]: pgmap v20979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:24.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:24 smithi090 ceph-mon[29180]: pgmap v20979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:26 smithi119 ceph-mon[32752]: pgmap v20980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38407.579860 secs 2024-04-27T05:15:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:26 smithi167 ceph-mon[32751]: pgmap v20980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38407.579860 secs 2024-04-27T05:15:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:26 smithi090 ceph-mon[29180]: pgmap v20980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38407.579860 secs 2024-04-27T05:15:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38408.482000 secs 2024-04-27T05:15:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38408.482000 secs 2024-04-27T05:15:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38408.482000 secs 2024-04-27T05:15:28.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:28 smithi119 ceph-mon[32752]: pgmap v20981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:28 smithi167 ceph-mon[32751]: pgmap v20981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:28 smithi090 ceph-mon[29180]: pgmap v20981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:30 smithi119 ceph-mon[32752]: pgmap v20982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:30.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:30 smithi167 ceph-mon[32751]: pgmap v20982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:30 smithi090 ceph-mon[29180]: pgmap v20982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38412.579950 secs 2024-04-27T05:15:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38412.579950 secs 2024-04-27T05:15:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38412.579950 secs 2024-04-27T05:15:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:32 smithi119 ceph-mon[32752]: pgmap v20983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38413.482075 secs 2024-04-27T05:15:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:32 smithi167 ceph-mon[32751]: pgmap v20983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38413.482075 secs 2024-04-27T05:15:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:32 smithi090 ceph-mon[29180]: pgmap v20983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38413.482075 secs 2024-04-27T05:15:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:34 smithi119 ceph-mon[32752]: pgmap v20984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:34 smithi167 ceph-mon[32751]: pgmap v20984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:34.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:34 smithi090 ceph-mon[29180]: pgmap v20984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:36 smithi119 ceph-mon[32752]: pgmap v20985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38417.580009 secs 2024-04-27T05:15:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:36 smithi167 ceph-mon[32751]: pgmap v20985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38417.580009 secs 2024-04-27T05:15:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:36 smithi090 ceph-mon[29180]: pgmap v20985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38417.580009 secs 2024-04-27T05:15:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38418.482195 secs 2024-04-27T05:15:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38418.482195 secs 2024-04-27T05:15:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38418.482195 secs 2024-04-27T05:15:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:38 smithi119 ceph-mon[32752]: pgmap v20986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:38 smithi167 ceph-mon[32751]: pgmap v20986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:38 smithi090 ceph-mon[29180]: pgmap v20986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:40 smithi119 ceph-mon[32752]: pgmap v20987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:40 smithi167 ceph-mon[32751]: pgmap v20987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:40 smithi090 ceph-mon[29180]: pgmap v20987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38422.580085 secs 2024-04-27T05:15:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38422.580085 secs 2024-04-27T05:15:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38422.580085 secs 2024-04-27T05:15:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:42 smithi119 ceph-mon[32752]: pgmap v20988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 38423.482317 secs 2024-04-27T05:15:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:42 smithi119 ceph-mon[32752]: slow request 1440.198395 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:15:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:42 smithi167 ceph-mon[32751]: pgmap v20988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 38423.482317 secs 2024-04-27T05:15:42.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:42 smithi167 ceph-mon[32751]: slow request 1440.198395 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:15:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:42 smithi090 ceph-mon[29180]: pgmap v20988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 38423.482317 secs 2024-04-27T05:15:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:42 smithi090 ceph-mon[29180]: slow request 1440.198395 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:15:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:44 smithi167 ceph-mon[32751]: pgmap v20989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:44 smithi090 ceph-mon[29180]: pgmap v20989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:44 smithi119 ceph-mon[32752]: pgmap v20989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:46 smithi167 ceph-mon[32751]: pgmap v20990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:46.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38427.580145 secs 2024-04-27T05:15:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:46 smithi090 ceph-mon[29180]: pgmap v20990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38427.580145 secs 2024-04-27T05:15:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:46 smithi119 ceph-mon[32752]: pgmap v20990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38427.580145 secs 2024-04-27T05:15:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38428.482462 secs 2024-04-27T05:15:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38428.482462 secs 2024-04-27T05:15:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38428.482462 secs 2024-04-27T05:15:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:48 smithi167 ceph-mon[32751]: pgmap v20991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:48 smithi090 ceph-mon[29180]: pgmap v20991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:48 smithi119 ceph-mon[32752]: pgmap v20991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:15:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:15:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:15:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:15:49.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:15:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:15:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:15:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:15:50.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:15:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:15:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:15:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:15:50.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:15:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:50 smithi090 ceph-mon[29180]: pgmap v20992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:50 smithi090 ceph-mon[29180]: pgmap v20993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:15:50.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:50 smithi090 ceph-mon[29180]: pgmap v20994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:50 smithi119 ceph-mon[32752]: pgmap v20992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:50 smithi119 ceph-mon[32752]: pgmap v20993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:15:51.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:50 smithi119 ceph-mon[32752]: pgmap v20994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:50 smithi167 ceph-mon[32751]: pgmap v20992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:15:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:50 smithi167 ceph-mon[32751]: pgmap v20993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:15:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:50 smithi167 ceph-mon[32751]: pgmap v20994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38432.580230 secs 2024-04-27T05:15:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38432.580230 secs 2024-04-27T05:15:52.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38432.580230 secs 2024-04-27T05:15:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38433.482598 secs 2024-04-27T05:15:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:52 smithi090 ceph-mon[29180]: pgmap v20995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38433.482598 secs 2024-04-27T05:15:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:52 smithi119 ceph-mon[32752]: pgmap v20995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38433.482598 secs 2024-04-27T05:15:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:52 smithi167 ceph-mon[32751]: pgmap v20995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:54 smithi090 ceph-mon[29180]: pgmap v20996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:54 smithi119 ceph-mon[32752]: pgmap v20996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:54 smithi167 ceph-mon[32751]: pgmap v20996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38437.580309 secs 2024-04-27T05:15:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:56 smithi090 ceph-mon[29180]: pgmap v20997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38437.580309 secs 2024-04-27T05:15:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:56 smithi119 ceph-mon[32752]: pgmap v20997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38437.580309 secs 2024-04-27T05:15:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:56 smithi167 ceph-mon[32751]: pgmap v20997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:15:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38438.482713 secs 2024-04-27T05:15:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38438.482713 secs 2024-04-27T05:15:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38438.482713 secs 2024-04-27T05:15:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:15:58 smithi090 ceph-mon[29180]: pgmap v20998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:15:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:15:58 smithi119 ceph-mon[32752]: pgmap v20998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:15:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:15:58 smithi167 ceph-mon[32751]: pgmap v20998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:16:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:00 smithi090 ceph-mon[29180]: pgmap v20999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:16:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:00 smithi119 ceph-mon[32752]: pgmap v20999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:16:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:00 smithi167 ceph-mon[32751]: pgmap v20999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:16:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38442.580394 secs 2024-04-27T05:16:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38442.580394 secs 2024-04-27T05:16:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38442.580394 secs 2024-04-27T05:16:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38443.482819 secs 2024-04-27T05:16:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:02 smithi090 ceph-mon[29180]: pgmap v21000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38443.482819 secs 2024-04-27T05:16:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:02 smithi119 ceph-mon[32752]: pgmap v21000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38443.482819 secs 2024-04-27T05:16:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:02 smithi167 ceph-mon[32751]: pgmap v21000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:04 smithi090 ceph-mon[29180]: pgmap v21001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:04 smithi119 ceph-mon[32752]: pgmap v21001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:04 smithi167 ceph-mon[32751]: pgmap v21001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38447.580477 secs 2024-04-27T05:16:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:06 smithi090 ceph-mon[29180]: pgmap v21002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38447.580477 secs 2024-04-27T05:16:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:06 smithi119 ceph-mon[32752]: pgmap v21002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38447.580477 secs 2024-04-27T05:16:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:06 smithi167 ceph-mon[32751]: pgmap v21002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38448.482933 secs 2024-04-27T05:16:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38448.482933 secs 2024-04-27T05:16:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38448.482933 secs 2024-04-27T05:16:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:08 smithi090 ceph-mon[29180]: pgmap v21003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:08 smithi119 ceph-mon[32752]: pgmap v21003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:08 smithi167 ceph-mon[32751]: pgmap v21003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:10 smithi090 ceph-mon[29180]: pgmap v21004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:10 smithi119 ceph-mon[32752]: pgmap v21004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:10 smithi167 ceph-mon[32751]: pgmap v21004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38452.580561 secs 2024-04-27T05:16:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38452.580561 secs 2024-04-27T05:16:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38452.580561 secs 2024-04-27T05:16:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38453.483066 secs 2024-04-27T05:16:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:12 smithi119 ceph-mon[32752]: pgmap v21005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38453.483066 secs 2024-04-27T05:16:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:12 smithi167 ceph-mon[32751]: pgmap v21005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38453.483066 secs 2024-04-27T05:16:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:12 smithi090 ceph-mon[29180]: pgmap v21005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:14.929 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:14 smithi090 ceph-mon[29180]: pgmap v21006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:14 smithi119 ceph-mon[32752]: pgmap v21006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:14 smithi167 ceph-mon[32751]: pgmap v21006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38457.580654 secs 2024-04-27T05:16:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:16 smithi119 ceph-mon[32752]: pgmap v21007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38457.580654 secs 2024-04-27T05:16:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:16 smithi167 ceph-mon[32751]: pgmap v21007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38457.580654 secs 2024-04-27T05:16:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:16 smithi090 ceph-mon[29180]: pgmap v21007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38458.483166 secs 2024-04-27T05:16:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38458.483166 secs 2024-04-27T05:16:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38458.483166 secs 2024-04-27T05:16:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:18 smithi119 ceph-mon[32752]: pgmap v21008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:18 smithi167 ceph-mon[32751]: pgmap v21008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:19.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:18 smithi090 ceph-mon[29180]: pgmap v21008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:20 smithi119 ceph-mon[32752]: pgmap v21009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:20 smithi167 ceph-mon[32751]: pgmap v21009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:20 smithi090 ceph-mon[29180]: pgmap v21009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38462.580738 secs 2024-04-27T05:16:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38462.580738 secs 2024-04-27T05:16:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38462.580738 secs 2024-04-27T05:16:22.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38463.483248 secs 2024-04-27T05:16:22.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:22 smithi119 ceph-mon[32752]: pgmap v21010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38463.483248 secs 2024-04-27T05:16:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:22 smithi167 ceph-mon[32751]: pgmap v21010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38463.483248 secs 2024-04-27T05:16:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:22 smithi090 ceph-mon[29180]: pgmap v21010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:25.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:24 smithi119 ceph-mon[32752]: pgmap v21011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:24 smithi167 ceph-mon[32751]: pgmap v21011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:24 smithi090 ceph-mon[29180]: pgmap v21011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38467.580820 secs 2024-04-27T05:16:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:26 smithi119 ceph-mon[32752]: pgmap v21012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38467.580820 secs 2024-04-27T05:16:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:26 smithi167 ceph-mon[32751]: pgmap v21012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38467.580820 secs 2024-04-27T05:16:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:26 smithi090 ceph-mon[29180]: pgmap v21012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38468.483394 secs 2024-04-27T05:16:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38468.483394 secs 2024-04-27T05:16:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38468.483394 secs 2024-04-27T05:16:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:28 smithi119 ceph-mon[32752]: pgmap v21013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:28 smithi167 ceph-mon[32751]: pgmap v21013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:28 smithi090 ceph-mon[29180]: pgmap v21013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:30 smithi090 ceph-mon[29180]: pgmap v21014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:30 smithi119 ceph-mon[32752]: pgmap v21014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:30 smithi167 ceph-mon[32751]: pgmap v21014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38472.580882 secs 2024-04-27T05:16:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38472.580882 secs 2024-04-27T05:16:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38472.580882 secs 2024-04-27T05:16:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38473.483471 secs 2024-04-27T05:16:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:32 smithi119 ceph-mon[32752]: pgmap v21015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38473.483471 secs 2024-04-27T05:16:33.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:32 smithi167 ceph-mon[32751]: pgmap v21015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38473.483471 secs 2024-04-27T05:16:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:32 smithi090 ceph-mon[29180]: pgmap v21015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:34 smithi119 ceph-mon[32752]: pgmap v21016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:34 smithi167 ceph-mon[32751]: pgmap v21016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:34 smithi090 ceph-mon[29180]: pgmap v21016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38477.580970 secs 2024-04-27T05:16:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:36 smithi119 ceph-mon[32752]: pgmap v21017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38477.580970 secs 2024-04-27T05:16:37.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:36 smithi167 ceph-mon[32751]: pgmap v21017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38477.580970 secs 2024-04-27T05:16:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:36 smithi090 ceph-mon[29180]: pgmap v21017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38478.483580 secs 2024-04-27T05:16:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38478.483580 secs 2024-04-27T05:16:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38478.483580 secs 2024-04-27T05:16:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:38 smithi119 ceph-mon[32752]: pgmap v21018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:38 smithi167 ceph-mon[32751]: pgmap v21018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:38 smithi090 ceph-mon[29180]: pgmap v21018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:40 smithi119 ceph-mon[32752]: pgmap v21019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:40 smithi167 ceph-mon[32751]: pgmap v21019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:40 smithi090 ceph-mon[29180]: pgmap v21019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38482.581030 secs 2024-04-27T05:16:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38482.581030 secs 2024-04-27T05:16:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38482.581030 secs 2024-04-27T05:16:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38483.483697 secs 2024-04-27T05:16:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:42 smithi119 ceph-mon[32752]: pgmap v21020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38483.483697 secs 2024-04-27T05:16:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:42 smithi167 ceph-mon[32751]: pgmap v21020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38483.483697 secs 2024-04-27T05:16:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:42 smithi090 ceph-mon[29180]: pgmap v21020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:44 smithi119 ceph-mon[32752]: pgmap v21021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:44 smithi167 ceph-mon[32751]: pgmap v21021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:45.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:44 smithi090 ceph-mon[29180]: pgmap v21021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38487.581098 secs 2024-04-27T05:16:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:46 smithi119 ceph-mon[32752]: pgmap v21022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38487.581098 secs 2024-04-27T05:16:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:46 smithi167 ceph-mon[32751]: pgmap v21022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38487.581098 secs 2024-04-27T05:16:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:46 smithi090 ceph-mon[29180]: pgmap v21022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38488.483788 secs 2024-04-27T05:16:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38488.483788 secs 2024-04-27T05:16:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38488.483788 secs 2024-04-27T05:16:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:48 smithi119 ceph-mon[32752]: pgmap v21023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:48 smithi167 ceph-mon[32751]: pgmap v21023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:49.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:48 smithi090 ceph-mon[29180]: pgmap v21023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:49 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:16:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:49 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:16:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:49 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:16:50.994 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:50 smithi119 ceph-mon[32752]: pgmap v21024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:50.995 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:50.995 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:50.995 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:50.995 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:50 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:50 smithi167 ceph-mon[32751]: pgmap v21024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:50 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.078 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:50 smithi090 ceph-mon[29180]: pgmap v21024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:16:51.078 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.078 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.079 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:51.079 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:50 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38492.581161 secs 2024-04-27T05:16:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:51 smithi119 ceph-mon[32752]: pgmap v21025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38492.581161 secs 2024-04-27T05:16:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:51 smithi167 ceph-mon[32751]: pgmap v21025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38492.581161 secs 2024-04-27T05:16:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:51 smithi090 ceph-mon[29180]: pgmap v21025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:16:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38493.483939 secs 2024-04-27T05:16:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:16:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:16:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:52 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.255 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38493.483939 secs 2024-04-27T05:16:53.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.256 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:16:53.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:16:53.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.257 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:52 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38493.483939 secs 2024-04-27T05:16:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:16:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:16:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:53.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:52 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:16:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:16:54.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:16:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:16:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:16:54.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:16:54.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: pgmap v21026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 B/s rd, 1 op/s 2024-04-27T05:16:54.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: pgmap v21027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 B/s rd, 1 op/s 2024-04-27T05:16:54.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:54.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.191 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.192 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:16:54.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:16:54.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:16:54.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:16:54.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: pgmap v21026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 B/s rd, 1 op/s 2024-04-27T05:16:54.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: pgmap v21027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 B/s rd, 1 op/s 2024-04-27T05:16:54.193 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:16:54.307 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:16:54.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:16:54.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:16:54.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: pgmap v21026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 881 B/s rd, 1 op/s 2024-04-27T05:16:54.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: pgmap v21027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 532 B/s rd, 1 op/s 2024-04-27T05:16:54.308 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:16:56.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:56 smithi119 ceph-mon[32752]: pgmap v21028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:16:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38497.581218 secs 2024-04-27T05:16:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:56 smithi167 ceph-mon[32751]: pgmap v21028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:16:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38497.581218 secs 2024-04-27T05:16:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:56 smithi090 ceph-mon[29180]: pgmap v21028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:16:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38497.581218 secs 2024-04-27T05:16:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38498.484057 secs 2024-04-27T05:16:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38498.484057 secs 2024-04-27T05:16:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38498.484057 secs 2024-04-27T05:16:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:16:58 smithi119 ceph-mon[32752]: pgmap v21029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:16:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:16:58 smithi167 ceph-mon[32751]: pgmap v21029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:16:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:16:58 smithi090 ceph-mon[29180]: pgmap v21029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:17:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:00 smithi119 ceph-mon[32752]: pgmap v21030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:17:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:00 smithi167 ceph-mon[32751]: pgmap v21030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:17:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:00 smithi090 ceph-mon[29180]: pgmap v21030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:17:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38502.581307 secs 2024-04-27T05:17:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38502.581307 secs 2024-04-27T05:17:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38502.581307 secs 2024-04-27T05:17:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:02 smithi119 ceph-mon[32752]: pgmap v21031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:17:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38503.484178 secs 2024-04-27T05:17:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:02 smithi167 ceph-mon[32751]: pgmap v21031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:17:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38503.484178 secs 2024-04-27T05:17:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:02 smithi090 ceph-mon[29180]: pgmap v21031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:17:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38503.484178 secs 2024-04-27T05:17:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:04 smithi119 ceph-mon[32752]: pgmap v21032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:17:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:04 smithi167 ceph-mon[32751]: pgmap v21032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:17:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:04 smithi090 ceph-mon[29180]: pgmap v21032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:17:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:06 smithi119 ceph-mon[32752]: pgmap v21033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38507.581388 secs 2024-04-27T05:17:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:06 smithi167 ceph-mon[32751]: pgmap v21033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38507.581388 secs 2024-04-27T05:17:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:06 smithi090 ceph-mon[29180]: pgmap v21033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38507.581388 secs 2024-04-27T05:17:07.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38508.484317 secs 2024-04-27T05:17:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38508.484317 secs 2024-04-27T05:17:07.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38508.484317 secs 2024-04-27T05:17:08.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:08 smithi119 ceph-mon[32752]: pgmap v21034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:08.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:08 smithi167 ceph-mon[32751]: pgmap v21034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:08 smithi090 ceph-mon[29180]: pgmap v21034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:10.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:10 smithi119 ceph-mon[32752]: pgmap v21035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:10.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:10 smithi167 ceph-mon[32751]: pgmap v21035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:10 smithi090 ceph-mon[29180]: pgmap v21035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38512.581478 secs 2024-04-27T05:17:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38512.581478 secs 2024-04-27T05:17:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38512.581478 secs 2024-04-27T05:17:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:12 smithi119 ceph-mon[32752]: pgmap v21036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38513.484416 secs 2024-04-27T05:17:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:12 smithi167 ceph-mon[32751]: pgmap v21036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38513.484416 secs 2024-04-27T05:17:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:12 smithi090 ceph-mon[29180]: pgmap v21036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38513.484416 secs 2024-04-27T05:17:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:14 smithi119 ceph-mon[32752]: pgmap v21037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:14.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:14 smithi167 ceph-mon[32751]: pgmap v21037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:14.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:14 smithi090 ceph-mon[29180]: pgmap v21037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:16.647 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:16 smithi090 ceph-mon[29180]: pgmap v21038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:16.648 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38517.581533 secs 2024-04-27T05:17:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:16 smithi119 ceph-mon[32752]: pgmap v21038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:16.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38517.581533 secs 2024-04-27T05:17:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:16 smithi167 ceph-mon[32751]: pgmap v21038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38517.581533 secs 2024-04-27T05:17:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38518.484533 secs 2024-04-27T05:17:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38518.484533 secs 2024-04-27T05:17:17.852 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38518.484533 secs 2024-04-27T05:17:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:18 smithi119 ceph-mon[32752]: pgmap v21039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:18.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:18 smithi167 ceph-mon[32751]: pgmap v21039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:18.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:18 smithi090 ceph-mon[29180]: pgmap v21039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:20 smithi119 ceph-mon[32752]: pgmap v21040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:20 smithi167 ceph-mon[32751]: pgmap v21040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:20 smithi090 ceph-mon[29180]: pgmap v21040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38522.581593 secs 2024-04-27T05:17:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38522.581593 secs 2024-04-27T05:17:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38522.581593 secs 2024-04-27T05:17:22.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:22 smithi119 ceph-mon[32752]: pgmap v21041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:22.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38523.484650 secs 2024-04-27T05:17:22.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:22 smithi167 ceph-mon[32751]: pgmap v21041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:22.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38523.484650 secs 2024-04-27T05:17:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:22 smithi090 ceph-mon[29180]: pgmap v21041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38523.484650 secs 2024-04-27T05:17:24.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:24 smithi119 ceph-mon[32752]: pgmap v21042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:24.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:24 smithi167 ceph-mon[32751]: pgmap v21042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:24.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:24 smithi090 ceph-mon[29180]: pgmap v21042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-27T05:17:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:26 smithi119 ceph-mon[32752]: pgmap v21043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38527.581650 secs 2024-04-27T05:17:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:26 smithi167 ceph-mon[32751]: pgmap v21043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38527.581650 secs 2024-04-27T05:17:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:26 smithi090 ceph-mon[29180]: pgmap v21043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-27T05:17:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38527.581650 secs 2024-04-27T05:17:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38528.484773 secs 2024-04-27T05:17:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38528.484773 secs 2024-04-27T05:17:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38528.484773 secs 2024-04-27T05:17:28.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:28 smithi119 ceph-mon[32752]: pgmap v21044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:28 smithi167 ceph-mon[32751]: pgmap v21044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:28 smithi090 ceph-mon[29180]: pgmap v21044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:30 smithi119 ceph-mon[32752]: pgmap v21045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:30 smithi167 ceph-mon[32751]: pgmap v21045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:30 smithi090 ceph-mon[29180]: pgmap v21045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38532.581709 secs 2024-04-27T05:17:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38532.581709 secs 2024-04-27T05:17:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38532.581709 secs 2024-04-27T05:17:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:32 smithi119 ceph-mon[32752]: pgmap v21046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38533.484861 secs 2024-04-27T05:17:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:32 smithi167 ceph-mon[32751]: pgmap v21046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38533.484861 secs 2024-04-27T05:17:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:32 smithi090 ceph-mon[29180]: pgmap v21046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38533.484861 secs 2024-04-27T05:17:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:34 smithi119 ceph-mon[32752]: pgmap v21047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:34 smithi167 ceph-mon[32751]: pgmap v21047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:34.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:34 smithi090 ceph-mon[29180]: pgmap v21047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:36 smithi119 ceph-mon[32752]: pgmap v21048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:36.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38537.581767 secs 2024-04-27T05:17:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:36 smithi167 ceph-mon[32751]: pgmap v21048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38537.581767 secs 2024-04-27T05:17:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:36 smithi090 ceph-mon[29180]: pgmap v21048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38537.581767 secs 2024-04-27T05:17:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38538.484921 secs 2024-04-27T05:17:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38538.484921 secs 2024-04-27T05:17:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38538.484921 secs 2024-04-27T05:17:38.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:38 smithi119 ceph-mon[32752]: pgmap v21049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:38 smithi167 ceph-mon[32751]: pgmap v21049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:38 smithi090 ceph-mon[29180]: pgmap v21049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:40 smithi119 ceph-mon[32752]: pgmap v21050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:40 smithi167 ceph-mon[32751]: pgmap v21050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:40 smithi090 ceph-mon[29180]: pgmap v21050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:17:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38542.581828 secs 2024-04-27T05:17:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38542.581828 secs 2024-04-27T05:17:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38542.581828 secs 2024-04-27T05:17:41.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:17:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:17:41.534+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:17:41.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:17:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:17:41.534+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:17:41.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:17:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:17:41.534+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:17:41.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:17:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:17:41.534+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:17:41.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:17:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:17:41.534+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:17:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:42 smithi119 ceph-mon[32752]: pgmap v21051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38543.485020 secs 2024-04-27T05:17:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:42 smithi167 ceph-mon[32751]: pgmap v21051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38543.485020 secs 2024-04-27T05:17:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:42 smithi090 ceph-mon[29180]: pgmap v21051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38543.485020 secs 2024-04-27T05:17:44.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:44 smithi119 ceph-mon[32752]: pgmap v21052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:44 smithi167 ceph-mon[32751]: pgmap v21052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:44 smithi090 ceph-mon[29180]: pgmap v21052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:17:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:46 smithi119 ceph-mon[32752]: pgmap v21053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:17:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38547.581889 secs 2024-04-27T05:17:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:46 smithi167 ceph-mon[32751]: pgmap v21053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:17:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38547.581889 secs 2024-04-27T05:17:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:46 smithi090 ceph-mon[29180]: pgmap v21053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:17:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38547.581889 secs 2024-04-27T05:17:47.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38548.485155 secs 2024-04-27T05:17:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38548.485155 secs 2024-04-27T05:17:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38548.485155 secs 2024-04-27T05:17:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:48 smithi119 ceph-mon[32752]: pgmap v21054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:48 smithi167 ceph-mon[32751]: pgmap v21054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:48 smithi090 ceph-mon[29180]: pgmap v21054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:50 smithi119 ceph-mon[32752]: pgmap v21055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:17:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:50 smithi167 ceph-mon[32751]: pgmap v21055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:17:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:50 smithi090 ceph-mon[29180]: pgmap v21055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:17:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38552.581967 secs 2024-04-27T05:17:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38552.581967 secs 2024-04-27T05:17:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38552.581967 secs 2024-04-27T05:17:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:52 smithi119 ceph-mon[32752]: pgmap v21056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:52.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38553.485276 secs 2024-04-27T05:17:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:52 smithi167 ceph-mon[32751]: pgmap v21056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:52.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38553.485276 secs 2024-04-27T05:17:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:52 smithi090 ceph-mon[29180]: pgmap v21056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38553.485276 secs 2024-04-27T05:17:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:53 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:17:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:53 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:17:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:53 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:17:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:54 smithi119 ceph-mon[32752]: pgmap v21057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:17:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:17:54.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:54 smithi167 ceph-mon[32751]: pgmap v21057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:17:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:17:54.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:54 smithi090 ceph-mon[29180]: pgmap v21057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:17:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:17:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:17:54.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:17:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:55 smithi119 ceph-mon[32752]: pgmap v21058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T05:17:55.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:55 smithi119 ceph-mon[32752]: pgmap v21059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:55 smithi167 ceph-mon[32751]: pgmap v21058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T05:17:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:55 smithi167 ceph-mon[32751]: pgmap v21059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:55 smithi090 ceph-mon[29180]: pgmap v21058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-27T05:17:55.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:55 smithi090 ceph-mon[29180]: pgmap v21059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38557.582026 secs 2024-04-27T05:17:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38557.582026 secs 2024-04-27T05:17:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38557.582026 secs 2024-04-27T05:17:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38558.485408 secs 2024-04-27T05:17:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:57 smithi119 ceph-mon[32752]: pgmap v21060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38558.485408 secs 2024-04-27T05:17:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:57 smithi167 ceph-mon[32751]: pgmap v21060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38558.485408 secs 2024-04-27T05:17:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:57 smithi090 ceph-mon[29180]: pgmap v21060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:59.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:17:59 smithi119 ceph-mon[32752]: pgmap v21061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:59.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:17:59 smithi167 ceph-mon[32751]: pgmap v21061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:17:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:17:59 smithi090 ceph-mon[29180]: pgmap v21061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38562.582126 secs 2024-04-27T05:18:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:01 smithi119 ceph-mon[32752]: pgmap v21062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38562.582126 secs 2024-04-27T05:18:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:01 smithi167 ceph-mon[32751]: pgmap v21062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38562.582126 secs 2024-04-27T05:18:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:01 smithi090 ceph-mon[29180]: pgmap v21062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38563.485473 secs 2024-04-27T05:18:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38563.485473 secs 2024-04-27T05:18:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38563.485473 secs 2024-04-27T05:18:03.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:03 smithi119 ceph-mon[32752]: pgmap v21063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:03 smithi167 ceph-mon[32751]: pgmap v21063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:03 smithi090 ceph-mon[29180]: pgmap v21063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:05 smithi119 ceph-mon[32752]: pgmap v21064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:05 smithi167 ceph-mon[32751]: pgmap v21064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:05 smithi090 ceph-mon[29180]: pgmap v21064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38567.582210 secs 2024-04-27T05:18:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38567.582210 secs 2024-04-27T05:18:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38567.582210 secs 2024-04-27T05:18:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38568.485551 secs 2024-04-27T05:18:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:07 smithi119 ceph-mon[32752]: pgmap v21065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38568.485551 secs 2024-04-27T05:18:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:07 smithi167 ceph-mon[32751]: pgmap v21065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38568.485551 secs 2024-04-27T05:18:07.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:07 smithi090 ceph-mon[29180]: pgmap v21065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:09 smithi119 ceph-mon[32752]: pgmap v21066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:09 smithi167 ceph-mon[32751]: pgmap v21066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:09 smithi090 ceph-mon[29180]: pgmap v21066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38572.582295 secs 2024-04-27T05:18:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:11 smithi119 ceph-mon[32752]: pgmap v21067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38572.582295 secs 2024-04-27T05:18:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:11 smithi167 ceph-mon[32751]: pgmap v21067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38572.582295 secs 2024-04-27T05:18:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:11 smithi090 ceph-mon[29180]: pgmap v21067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38573.485668 secs 2024-04-27T05:18:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38573.485668 secs 2024-04-27T05:18:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38573.485668 secs 2024-04-27T05:18:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:13 smithi119 ceph-mon[32752]: pgmap v21068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:13 smithi167 ceph-mon[32751]: pgmap v21068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:13 smithi090 ceph-mon[29180]: pgmap v21068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:18:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:15 smithi119 ceph-mon[32752]: pgmap v21069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:15 smithi167 ceph-mon[32751]: pgmap v21069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:15 smithi090 ceph-mon[29180]: pgmap v21069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:18:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38577.582384 secs 2024-04-27T05:18:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38577.582384 secs 2024-04-27T05:18:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38577.582384 secs 2024-04-27T05:18:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38578.485748 secs 2024-04-27T05:18:17.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:17 smithi119 ceph-mon[32752]: pgmap v21070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38578.485748 secs 2024-04-27T05:18:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:17 smithi167 ceph-mon[32751]: pgmap v21070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38578.485748 secs 2024-04-27T05:18:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:17 smithi090 ceph-mon[29180]: pgmap v21070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:19 smithi119 ceph-mon[32752]: pgmap v21071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:19 smithi167 ceph-mon[32751]: pgmap v21071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:19 smithi090 ceph-mon[29180]: pgmap v21071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38582.582472 secs 2024-04-27T05:18:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:21 smithi119 ceph-mon[32752]: pgmap v21072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38582.582472 secs 2024-04-27T05:18:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:21 smithi167 ceph-mon[32751]: pgmap v21072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38582.582472 secs 2024-04-27T05:18:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:21 smithi090 ceph-mon[29180]: pgmap v21072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38583.485869 secs 2024-04-27T05:18:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38583.485869 secs 2024-04-27T05:18:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38583.485869 secs 2024-04-27T05:18:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:23 smithi167 ceph-mon[32751]: pgmap v21073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:23 smithi090 ceph-mon[29180]: pgmap v21073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:23 smithi119 ceph-mon[32752]: pgmap v21073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:25 smithi167 ceph-mon[32751]: pgmap v21074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:25 smithi090 ceph-mon[29180]: pgmap v21074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:25 smithi119 ceph-mon[32752]: pgmap v21074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38587.582556 secs 2024-04-27T05:18:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38587.582556 secs 2024-04-27T05:18:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38587.582556 secs 2024-04-27T05:18:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38588.485989 secs 2024-04-27T05:18:27.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:27 smithi167 ceph-mon[32751]: pgmap v21075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38588.485989 secs 2024-04-27T05:18:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:27 smithi090 ceph-mon[29180]: pgmap v21075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38588.485989 secs 2024-04-27T05:18:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:27 smithi119 ceph-mon[32752]: pgmap v21075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:29 smithi090 ceph-mon[29180]: pgmap v21076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:29 smithi119 ceph-mon[32752]: pgmap v21076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:29 smithi167 ceph-mon[32751]: pgmap v21076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38592.582646 secs 2024-04-27T05:18:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:31 smithi090 ceph-mon[29180]: pgmap v21077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38592.582646 secs 2024-04-27T05:18:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:31 smithi119 ceph-mon[32752]: pgmap v21077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38592.582646 secs 2024-04-27T05:18:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:31 smithi167 ceph-mon[32751]: pgmap v21077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38593.486104 secs 2024-04-27T05:18:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38593.486104 secs 2024-04-27T05:18:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38593.486104 secs 2024-04-27T05:18:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:33 smithi090 ceph-mon[29180]: pgmap v21078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:33 smithi119 ceph-mon[32752]: pgmap v21078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:33 smithi167 ceph-mon[32751]: pgmap v21078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:35 smithi090 ceph-mon[29180]: pgmap v21079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:35 smithi119 ceph-mon[32752]: pgmap v21079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:35 smithi167 ceph-mon[32751]: pgmap v21079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38597.582729 secs 2024-04-27T05:18:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38597.582729 secs 2024-04-27T05:18:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38597.582729 secs 2024-04-27T05:18:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38598.486253 secs 2024-04-27T05:18:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:37 smithi090 ceph-mon[29180]: pgmap v21080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38598.486253 secs 2024-04-27T05:18:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:37 smithi119 ceph-mon[32752]: pgmap v21080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38598.486253 secs 2024-04-27T05:18:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:37 smithi167 ceph-mon[32751]: pgmap v21080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:39 smithi090 ceph-mon[29180]: pgmap v21081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:39 smithi119 ceph-mon[32752]: pgmap v21081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:39 smithi167 ceph-mon[32751]: pgmap v21081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38602.582815 secs 2024-04-27T05:18:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:41 smithi090 ceph-mon[29180]: pgmap v21082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38602.582815 secs 2024-04-27T05:18:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:41 smithi119 ceph-mon[32752]: pgmap v21082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38602.582815 secs 2024-04-27T05:18:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:41 smithi167 ceph-mon[32751]: pgmap v21082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38603.486397 secs 2024-04-27T05:18:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38603.486397 secs 2024-04-27T05:18:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38603.486397 secs 2024-04-27T05:18:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:43 smithi090 ceph-mon[29180]: pgmap v21083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:43 smithi119 ceph-mon[32752]: pgmap v21083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:43 smithi167 ceph-mon[32751]: pgmap v21083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:45 smithi090 ceph-mon[29180]: pgmap v21084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:45 smithi119 ceph-mon[32752]: pgmap v21084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:45 smithi167 ceph-mon[32751]: pgmap v21084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38607.582903 secs 2024-04-27T05:18:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38607.582903 secs 2024-04-27T05:18:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38607.582903 secs 2024-04-27T05:18:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38608.486520 secs 2024-04-27T05:18:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:47 smithi090 ceph-mon[29180]: pgmap v21085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38608.486520 secs 2024-04-27T05:18:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:47 smithi119 ceph-mon[32752]: pgmap v21085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38608.486520 secs 2024-04-27T05:18:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:47 smithi167 ceph-mon[32751]: pgmap v21085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:49 smithi090 ceph-mon[29180]: pgmap v21086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:49 smithi119 ceph-mon[32752]: pgmap v21086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:49 smithi167 ceph-mon[32751]: pgmap v21086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38612.582994 secs 2024-04-27T05:18:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:51 smithi090 ceph-mon[29180]: pgmap v21087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38612.582994 secs 2024-04-27T05:18:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:51 smithi119 ceph-mon[32752]: pgmap v21087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38612.582994 secs 2024-04-27T05:18:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:51 smithi167 ceph-mon[32751]: pgmap v21087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:52.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38613.486639 secs 2024-04-27T05:18:52.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38613.486639 secs 2024-04-27T05:18:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38613.486639 secs 2024-04-27T05:18:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:53 smithi119 ceph-mon[32752]: pgmap v21088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:53 smithi167 ceph-mon[32751]: pgmap v21088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:53 smithi090 ceph-mon[29180]: pgmap v21088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:18:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:54 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:18:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:54 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:18:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:54 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:18:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:55 smithi119 ceph-mon[32752]: pgmap v21089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:56.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:18:56.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:18:56.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:55 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:18:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:55 smithi167 ceph-mon[32751]: pgmap v21089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:18:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:18:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:55 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:18:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:55 smithi090 ceph-mon[29180]: pgmap v21089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:18:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:18:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:18:56.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:55 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:18:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:56 smithi119 ceph-mon[32752]: pgmap v21090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:18:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:56 smithi119 ceph-mon[32752]: pgmap v21091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38617.583100 secs 2024-04-27T05:18:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:56 smithi167 ceph-mon[32751]: pgmap v21090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:18:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:56 smithi167 ceph-mon[32751]: pgmap v21091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38617.583100 secs 2024-04-27T05:18:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:56 smithi090 ceph-mon[29180]: pgmap v21090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:18:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:56 smithi090 ceph-mon[29180]: pgmap v21091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:18:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38617.583100 secs 2024-04-27T05:18:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38618.486756 secs 2024-04-27T05:18:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38618.486756 secs 2024-04-27T05:18:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38618.486756 secs 2024-04-27T05:18:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:18:58 smithi119 ceph-mon[32752]: pgmap v21092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:18:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:18:58 smithi167 ceph-mon[32751]: pgmap v21092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:18:59.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:18:58 smithi090 ceph-mon[29180]: pgmap v21092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:19:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:00 smithi119 ceph-mon[32752]: pgmap v21093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:00 smithi167 ceph-mon[32751]: pgmap v21093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:00 smithi090 ceph-mon[29180]: pgmap v21093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38622.583188 secs 2024-04-27T05:19:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38622.583188 secs 2024-04-27T05:19:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38622.583188 secs 2024-04-27T05:19:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:02 smithi119 ceph-mon[32752]: pgmap v21094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38623.486818 secs 2024-04-27T05:19:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:02 smithi167 ceph-mon[32751]: pgmap v21094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38623.486818 secs 2024-04-27T05:19:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:02 smithi090 ceph-mon[29180]: pgmap v21094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38623.486818 secs 2024-04-27T05:19:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:04 smithi119 ceph-mon[32752]: pgmap v21095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:19:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:04 smithi167 ceph-mon[32751]: pgmap v21095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:19:05.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:04 smithi090 ceph-mon[29180]: pgmap v21095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:19:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:06 smithi119 ceph-mon[32752]: pgmap v21096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:19:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38627.583246 secs 2024-04-27T05:19:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:06 smithi167 ceph-mon[32751]: pgmap v21096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:19:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38627.583246 secs 2024-04-27T05:19:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:06 smithi090 ceph-mon[29180]: pgmap v21096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:19:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38627.583246 secs 2024-04-27T05:19:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38628.486893 secs 2024-04-27T05:19:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38628.486893 secs 2024-04-27T05:19:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38628.486893 secs 2024-04-27T05:19:09.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:08 smithi119 ceph-mon[32752]: pgmap v21097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:08 smithi167 ceph-mon[32751]: pgmap v21097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:08 smithi090 ceph-mon[29180]: pgmap v21097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:10 smithi119 ceph-mon[32752]: pgmap v21098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:10 smithi167 ceph-mon[32751]: pgmap v21098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:10 smithi090 ceph-mon[29180]: pgmap v21098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38632.583334 secs 2024-04-27T05:19:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38632.583334 secs 2024-04-27T05:19:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38632.583334 secs 2024-04-27T05:19:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:12 smithi119 ceph-mon[32752]: pgmap v21099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:12 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 38633.487011 secs 2024-04-27T05:19:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:12 smithi119 ceph-mon[32752]: slow request 5761.339580 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:19:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:12 smithi167 ceph-mon[32751]: pgmap v21099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:12 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 38633.487011 secs 2024-04-27T05:19:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:12 smithi167 ceph-mon[32751]: slow request 5761.339580 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:19:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:12 smithi090 ceph-mon[29180]: pgmap v21099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:12 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 38633.487011 secs 2024-04-27T05:19:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:12 smithi090 ceph-mon[29180]: slow request 5761.339580 seconds old, received at 2024-04-27T03:43:09.898076+0000: client_request(client.27044:1 getattr pAsLsXsFs #0x1 2024-04-27T03:43:09.891053+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:19:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:14 smithi119 ceph-mon[32752]: pgmap v21100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:14 smithi167 ceph-mon[32751]: pgmap v21100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:14 smithi090 ceph-mon[29180]: pgmap v21100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:16 smithi119 ceph-mon[32752]: pgmap v21101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38637.583416 secs 2024-04-27T05:19:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:16 smithi167 ceph-mon[32751]: pgmap v21101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38637.583416 secs 2024-04-27T05:19:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:16 smithi090 ceph-mon[29180]: pgmap v21101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38637.583416 secs 2024-04-27T05:19:18.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38638.487126 secs 2024-04-27T05:19:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38638.487126 secs 2024-04-27T05:19:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38638.487126 secs 2024-04-27T05:19:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:18 smithi119 ceph-mon[32752]: pgmap v21102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:18 smithi167 ceph-mon[32751]: pgmap v21102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:18 smithi090 ceph-mon[29180]: pgmap v21102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:21.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:20 smithi119 ceph-mon[32752]: pgmap v21103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:20 smithi167 ceph-mon[32751]: pgmap v21103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:20 smithi090 ceph-mon[29180]: pgmap v21103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38642.583504 secs 2024-04-27T05:19:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38642.583504 secs 2024-04-27T05:19:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38642.583504 secs 2024-04-27T05:19:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:22 smithi167 ceph-mon[32751]: pgmap v21104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38643.487206 secs 2024-04-27T05:19:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:22 smithi119 ceph-mon[32752]: pgmap v21104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38643.487206 secs 2024-04-27T05:19:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:22 smithi090 ceph-mon[29180]: pgmap v21104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38643.487206 secs 2024-04-27T05:19:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:24 smithi119 ceph-mon[32752]: pgmap v21105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:24 smithi167 ceph-mon[32751]: pgmap v21105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:24 smithi090 ceph-mon[29180]: pgmap v21105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:26 smithi119 ceph-mon[32752]: pgmap v21106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38647.583587 secs 2024-04-27T05:19:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:26 smithi167 ceph-mon[32751]: pgmap v21106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38647.583587 secs 2024-04-27T05:19:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:26 smithi090 ceph-mon[29180]: pgmap v21106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38647.583587 secs 2024-04-27T05:19:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38648.487279 secs 2024-04-27T05:19:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38648.487279 secs 2024-04-27T05:19:28.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38648.487279 secs 2024-04-27T05:19:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:28 smithi119 ceph-mon[32752]: pgmap v21107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:28 smithi167 ceph-mon[32751]: pgmap v21107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:28 smithi090 ceph-mon[29180]: pgmap v21107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:30 smithi119 ceph-mon[32752]: pgmap v21108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:30 smithi167 ceph-mon[32751]: pgmap v21108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:30 smithi090 ceph-mon[29180]: pgmap v21108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38652.583647 secs 2024-04-27T05:19:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38652.583647 secs 2024-04-27T05:19:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38652.583647 secs 2024-04-27T05:19:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:32 smithi119 ceph-mon[32752]: pgmap v21109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38653.487375 secs 2024-04-27T05:19:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:32 smithi167 ceph-mon[32751]: pgmap v21109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38653.487375 secs 2024-04-27T05:19:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:32 smithi090 ceph-mon[29180]: pgmap v21109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38653.487375 secs 2024-04-27T05:19:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:34 smithi119 ceph-mon[32752]: pgmap v21110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:34 smithi167 ceph-mon[32751]: pgmap v21110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:34 smithi090 ceph-mon[29180]: pgmap v21110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:36 smithi119 ceph-mon[32752]: pgmap v21111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38657.583703 secs 2024-04-27T05:19:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:36 smithi167 ceph-mon[32751]: pgmap v21111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38657.583703 secs 2024-04-27T05:19:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:36 smithi090 ceph-mon[29180]: pgmap v21111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38657.583703 secs 2024-04-27T05:19:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38658.487494 secs 2024-04-27T05:19:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38658.487494 secs 2024-04-27T05:19:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38658.487494 secs 2024-04-27T05:19:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:38 smithi119 ceph-mon[32752]: pgmap v21112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:38 smithi167 ceph-mon[32751]: pgmap v21112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:38 smithi090 ceph-mon[29180]: pgmap v21112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:39 smithi119 ceph-mon[32752]: pgmap v21113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:39 smithi167 ceph-mon[32751]: pgmap v21113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:39 smithi090 ceph-mon[29180]: pgmap v21113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:41.557 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38662.583816 secs 2024-04-27T05:19:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38662.583816 secs 2024-04-27T05:19:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38662.583816 secs 2024-04-27T05:19:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:42 smithi119 ceph-mon[32752]: pgmap v21114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:42.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38663.487616 secs 2024-04-27T05:19:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:42 smithi167 ceph-mon[32751]: pgmap v21114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38663.487616 secs 2024-04-27T05:19:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:42 smithi090 ceph-mon[29180]: pgmap v21114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:42.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38663.487616 secs 2024-04-27T05:19:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:44 smithi119 ceph-mon[32752]: pgmap v21115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:44.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:44 smithi167 ceph-mon[32751]: pgmap v21115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:44.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:44 smithi090 ceph-mon[29180]: pgmap v21115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:46 smithi119 ceph-mon[32752]: pgmap v21116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38667.583898 secs 2024-04-27T05:19:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:46 smithi167 ceph-mon[32751]: pgmap v21116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38667.583898 secs 2024-04-27T05:19:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:46 smithi090 ceph-mon[29180]: pgmap v21116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38667.583898 secs 2024-04-27T05:19:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38668.487697 secs 2024-04-27T05:19:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38668.487697 secs 2024-04-27T05:19:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38668.487697 secs 2024-04-27T05:19:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:48 smithi119 ceph-mon[32752]: pgmap v21117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:48 smithi167 ceph-mon[32751]: pgmap v21117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:48 smithi090 ceph-mon[29180]: pgmap v21117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:50 smithi119 ceph-mon[32752]: pgmap v21118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:50 smithi167 ceph-mon[32751]: pgmap v21118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:50 smithi090 ceph-mon[29180]: pgmap v21118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38672.583992 secs 2024-04-27T05:19:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38672.583992 secs 2024-04-27T05:19:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38672.583992 secs 2024-04-27T05:19:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:52 smithi167 ceph-mon[32751]: pgmap v21119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38673.487814 secs 2024-04-27T05:19:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:52 smithi090 ceph-mon[29180]: pgmap v21119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38673.487814 secs 2024-04-27T05:19:52.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:52 smithi119 ceph-mon[32752]: pgmap v21119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38673.487814 secs 2024-04-27T05:19:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:54 smithi167 ceph-mon[32751]: pgmap v21120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:54 smithi090 ceph-mon[29180]: pgmap v21120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:54 smithi119 ceph-mon[32752]: pgmap v21120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:19:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:56 smithi090 ceph-mon[29180]: pgmap v21121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:19:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38677.584089 secs 2024-04-27T05:19:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:19:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:19:56.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:19:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:56 smithi119 ceph-mon[32752]: pgmap v21121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:19:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38677.584089 secs 2024-04-27T05:19:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:19:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:19:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:19:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:56 smithi167 ceph-mon[32751]: pgmap v21121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:19:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:19:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38677.584089 secs 2024-04-27T05:19:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:19:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:19:56.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:19:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:57 smithi090 ceph-mon[29180]: pgmap v21122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:19:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:57 smithi090 ceph-mon[29180]: pgmap v21123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:57.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38678.487872 secs 2024-04-27T05:19:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:57 smithi119 ceph-mon[32752]: pgmap v21122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:19:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:57 smithi119 ceph-mon[32752]: pgmap v21123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38678.487872 secs 2024-04-27T05:19:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:57 smithi167 ceph-mon[32751]: pgmap v21122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:19:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:57 smithi167 ceph-mon[32751]: pgmap v21123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:19:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38678.487872 secs 2024-04-27T05:19:59.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:19:59 smithi090 ceph-mon[29180]: pgmap v21124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:19:59.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:19:59 smithi119 ceph-mon[32752]: pgmap v21124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:19:59.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:19:59 smithi167 ceph-mon[32751]: pgmap v21124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:20:00.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:20:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:20:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:20:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:01 smithi090 ceph-mon[29180]: pgmap v21125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:01.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38682.584157 secs 2024-04-27T05:20:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:01 smithi119 ceph-mon[32752]: pgmap v21125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38682.584157 secs 2024-04-27T05:20:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:01 smithi167 ceph-mon[32751]: pgmap v21125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38682.584157 secs 2024-04-27T05:20:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38683.488010 secs 2024-04-27T05:20:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38683.488010 secs 2024-04-27T05:20:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38683.488010 secs 2024-04-27T05:20:03.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:03 smithi090 ceph-mon[29180]: pgmap v21126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:03.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:03 smithi119 ceph-mon[32752]: pgmap v21126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:03.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:03 smithi167 ceph-mon[32751]: pgmap v21126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:05.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:05 smithi090 ceph-mon[29180]: pgmap v21127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:05.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:05 smithi119 ceph-mon[32752]: pgmap v21127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:05.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:05 smithi167 ceph-mon[32751]: pgmap v21127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38687.584255 secs 2024-04-27T05:20:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38687.584255 secs 2024-04-27T05:20:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38687.584255 secs 2024-04-27T05:20:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:07 smithi090 ceph-mon[29180]: pgmap v21128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:20:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38688.488076 secs 2024-04-27T05:20:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:07 smithi119 ceph-mon[32752]: pgmap v21128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:20:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38688.488076 secs 2024-04-27T05:20:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:07 smithi167 ceph-mon[32751]: pgmap v21128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:20:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38688.488076 secs 2024-04-27T05:20:09.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:09 smithi090 ceph-mon[29180]: pgmap v21129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:09 smithi119 ceph-mon[32752]: pgmap v21129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:09 smithi167 ceph-mon[32751]: pgmap v21129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:11 smithi090 ceph-mon[29180]: pgmap v21130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:11.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38692.584359 secs 2024-04-27T05:20:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:11 smithi119 ceph-mon[32752]: pgmap v21130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38692.584359 secs 2024-04-27T05:20:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:11 smithi167 ceph-mon[32751]: pgmap v21130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38692.584359 secs 2024-04-27T05:20:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38693.488219 secs 2024-04-27T05:20:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38693.488219 secs 2024-04-27T05:20:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38693.488219 secs 2024-04-27T05:20:13.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:13 smithi090 ceph-mon[29180]: pgmap v21131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:13 smithi119 ceph-mon[32752]: pgmap v21131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:13 smithi167 ceph-mon[32751]: pgmap v21131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:15 smithi090 ceph-mon[29180]: pgmap v21132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:15 smithi119 ceph-mon[32752]: pgmap v21132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:15 smithi167 ceph-mon[32751]: pgmap v21132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38697.584455 secs 2024-04-27T05:20:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38697.584455 secs 2024-04-27T05:20:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38697.584455 secs 2024-04-27T05:20:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:17 smithi090 ceph-mon[29180]: pgmap v21133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:17.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38698.488354 secs 2024-04-27T05:20:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:17 smithi119 ceph-mon[32752]: pgmap v21133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38698.488354 secs 2024-04-27T05:20:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:17 smithi167 ceph-mon[32751]: pgmap v21133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38698.488354 secs 2024-04-27T05:20:19.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:19 smithi090 ceph-mon[29180]: pgmap v21134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:19 smithi119 ceph-mon[32752]: pgmap v21134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:19 smithi167 ceph-mon[32751]: pgmap v21134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:21 smithi119 ceph-mon[32752]: pgmap v21135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:21.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38702.584556 secs 2024-04-27T05:20:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:21 smithi167 ceph-mon[32751]: pgmap v21135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38702.584556 secs 2024-04-27T05:20:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:21 smithi090 ceph-mon[29180]: pgmap v21135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38702.584556 secs 2024-04-27T05:20:22.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38703.488468 secs 2024-04-27T05:20:22.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38703.488468 secs 2024-04-27T05:20:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38703.488468 secs 2024-04-27T05:20:23.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:23 smithi090 ceph-mon[29180]: pgmap v21136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:23 smithi119 ceph-mon[32752]: pgmap v21136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:23 smithi167 ceph-mon[32751]: pgmap v21136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:25 smithi119 ceph-mon[32752]: pgmap v21137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:25 smithi167 ceph-mon[32751]: pgmap v21137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:25 smithi090 ceph-mon[29180]: pgmap v21137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38707.584652 secs 2024-04-27T05:20:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38707.584652 secs 2024-04-27T05:20:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38707.584652 secs 2024-04-27T05:20:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:27 smithi119 ceph-mon[32752]: pgmap v21138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38708.488599 secs 2024-04-27T05:20:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:27 smithi167 ceph-mon[32751]: pgmap v21138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38708.488599 secs 2024-04-27T05:20:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:27 smithi090 ceph-mon[29180]: pgmap v21138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38708.488599 secs 2024-04-27T05:20:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:29 smithi119 ceph-mon[32752]: pgmap v21139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:29 smithi167 ceph-mon[32751]: pgmap v21139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:29 smithi090 ceph-mon[29180]: pgmap v21139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:31 smithi119 ceph-mon[32752]: pgmap v21140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:31.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38712.584756 secs 2024-04-27T05:20:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:31 smithi167 ceph-mon[32751]: pgmap v21140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38712.584756 secs 2024-04-27T05:20:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:31 smithi090 ceph-mon[29180]: pgmap v21140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38712.584756 secs 2024-04-27T05:20:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38713.488743 secs 2024-04-27T05:20:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38713.488743 secs 2024-04-27T05:20:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38713.488743 secs 2024-04-27T05:20:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:33 smithi119 ceph-mon[32752]: pgmap v21141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:33 smithi167 ceph-mon[32751]: pgmap v21141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:33 smithi090 ceph-mon[29180]: pgmap v21141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:35 smithi119 ceph-mon[32752]: pgmap v21142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:35 smithi167 ceph-mon[32751]: pgmap v21142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:35 smithi090 ceph-mon[29180]: pgmap v21142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38717.584852 secs 2024-04-27T05:20:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38717.584852 secs 2024-04-27T05:20:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38717.584852 secs 2024-04-27T05:20:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:37 smithi119 ceph-mon[32752]: pgmap v21143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:37.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38718.488861 secs 2024-04-27T05:20:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:37 smithi167 ceph-mon[32751]: pgmap v21143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38718.488861 secs 2024-04-27T05:20:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:37 smithi090 ceph-mon[29180]: pgmap v21143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38718.488861 secs 2024-04-27T05:20:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:39 smithi119 ceph-mon[32752]: pgmap v21144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:39 smithi167 ceph-mon[32751]: pgmap v21144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:39 smithi090 ceph-mon[29180]: pgmap v21144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:41 smithi119 ceph-mon[32752]: pgmap v21145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:41.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38722.584935 secs 2024-04-27T05:20:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:41 smithi167 ceph-mon[32751]: pgmap v21145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38722.584935 secs 2024-04-27T05:20:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:41 smithi090 ceph-mon[29180]: pgmap v21145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38722.584935 secs 2024-04-27T05:20:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38723.488987 secs 2024-04-27T05:20:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38723.488987 secs 2024-04-27T05:20:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38723.488987 secs 2024-04-27T05:20:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:43 smithi119 ceph-mon[32752]: pgmap v21146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:43 smithi167 ceph-mon[32751]: pgmap v21146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:43 smithi090 ceph-mon[29180]: pgmap v21146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:45 smithi119 ceph-mon[32752]: pgmap v21147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:45 smithi167 ceph-mon[32751]: pgmap v21147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:45 smithi090 ceph-mon[29180]: pgmap v21147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:46 smithi119 ceph-mon[32752]: 50 slow requests, 1 included below; oldest blocked for > 38727.585020 secs 2024-04-27T05:20:46.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:46 smithi119 ceph-mon[32752]: slow request 1444.179698 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:20:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:46 smithi167 ceph-mon[32751]: 50 slow requests, 1 included below; oldest blocked for > 38727.585020 secs 2024-04-27T05:20:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:46 smithi167 ceph-mon[32751]: slow request 1444.179698 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:20:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:46 smithi090 ceph-mon[29180]: 50 slow requests, 1 included below; oldest blocked for > 38727.585020 secs 2024-04-27T05:20:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:46 smithi090 ceph-mon[29180]: slow request 1444.179698 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:20:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:47 smithi119 ceph-mon[32752]: pgmap v21148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:47.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38728.489121 secs 2024-04-27T05:20:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:47 smithi167 ceph-mon[32751]: pgmap v21148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:47.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38728.489121 secs 2024-04-27T05:20:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:47 smithi090 ceph-mon[29180]: pgmap v21148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38728.489121 secs 2024-04-27T05:20:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:49 smithi119 ceph-mon[32752]: pgmap v21149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:49.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:49 smithi167 ceph-mon[32751]: pgmap v21149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:49 smithi090 ceph-mon[29180]: pgmap v21149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:51 smithi119 ceph-mon[32752]: pgmap v21150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38732.585124 secs 2024-04-27T05:20:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:51 smithi167 ceph-mon[32751]: pgmap v21150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38732.585124 secs 2024-04-27T05:20:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:51 smithi090 ceph-mon[29180]: pgmap v21150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38732.585124 secs 2024-04-27T05:20:52.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38733.489260 secs 2024-04-27T05:20:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38733.489260 secs 2024-04-27T05:20:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38733.489260 secs 2024-04-27T05:20:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:53 smithi119 ceph-mon[32752]: pgmap v21151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:53.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:53 smithi167 ceph-mon[32751]: pgmap v21151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:53 smithi090 ceph-mon[29180]: pgmap v21151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:20:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:55 smithi119 ceph-mon[32752]: pgmap v21152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:55.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:55 smithi167 ceph-mon[32751]: pgmap v21152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:55 smithi090 ceph-mon[29180]: pgmap v21152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:20:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38737.585211 secs 2024-04-27T05:20:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:56 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:20:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38737.585211 secs 2024-04-27T05:20:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:56 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:20:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38737.585211 secs 2024-04-27T05:20:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:56 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:20:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:57 smithi119 ceph-mon[32752]: pgmap v21153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38738.489365 secs 2024-04-27T05:20:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:20:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:20:57.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:20:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:57 smithi167 ceph-mon[32751]: pgmap v21153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38738.489365 secs 2024-04-27T05:20:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:20:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:20:57.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:20:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:57 smithi090 ceph-mon[29180]: pgmap v21153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:20:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38738.489365 secs 2024-04-27T05:20:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:20:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:20:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:20:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:58 smithi119 ceph-mon[32752]: pgmap v21154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:20:58.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:20:58 smithi119 ceph-mon[32752]: pgmap v21155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:58 smithi167 ceph-mon[32751]: pgmap v21154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:20:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:20:58 smithi167 ceph-mon[32751]: pgmap v21155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:20:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:58 smithi090 ceph-mon[29180]: pgmap v21154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:20:58.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:20:58 smithi090 ceph-mon[29180]: pgmap v21155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:00 smithi119 ceph-mon[32752]: pgmap v21156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:00 smithi167 ceph-mon[32751]: pgmap v21156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:00 smithi090 ceph-mon[29180]: pgmap v21156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:01 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38742.585312 secs 2024-04-27T05:21:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:01 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38742.585312 secs 2024-04-27T05:21:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:01 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38742.585312 secs 2024-04-27T05:21:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:02 smithi119 ceph-mon[32752]: pgmap v21157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38743.489490 secs 2024-04-27T05:21:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:02 smithi167 ceph-mon[32751]: pgmap v21157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38743.489490 secs 2024-04-27T05:21:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:02 smithi090 ceph-mon[29180]: pgmap v21157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38743.489490 secs 2024-04-27T05:21:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:04 smithi119 ceph-mon[32752]: pgmap v21158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:21:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:04 smithi167 ceph-mon[32751]: pgmap v21158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:21:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:04 smithi090 ceph-mon[29180]: pgmap v21158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:21:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:06 smithi119 ceph-mon[32752]: pgmap v21159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:06 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38747.585410 secs 2024-04-27T05:21:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:06 smithi167 ceph-mon[32751]: pgmap v21159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:06 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38747.585410 secs 2024-04-27T05:21:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:06 smithi090 ceph-mon[29180]: pgmap v21159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:21:06.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:06 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38747.585410 secs 2024-04-27T05:21:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38748.489565 secs 2024-04-27T05:21:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38748.489565 secs 2024-04-27T05:21:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38748.489565 secs 2024-04-27T05:21:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:08 smithi119 ceph-mon[32752]: pgmap v21160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:21:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:08 smithi167 ceph-mon[32751]: pgmap v21160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:21:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:08 smithi090 ceph-mon[29180]: pgmap v21160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:21:10.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:10 smithi119 ceph-mon[32752]: pgmap v21161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:10.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:10 smithi167 ceph-mon[32751]: pgmap v21161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:10 smithi090 ceph-mon[29180]: pgmap v21161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:11 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38752.585512 secs 2024-04-27T05:21:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:11 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38752.585512 secs 2024-04-27T05:21:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:11 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38752.585512 secs 2024-04-27T05:21:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:12 smithi119 ceph-mon[32752]: pgmap v21162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38753.489681 secs 2024-04-27T05:21:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:12 smithi167 ceph-mon[32751]: pgmap v21162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38753.489681 secs 2024-04-27T05:21:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:12 smithi090 ceph-mon[29180]: pgmap v21162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38753.489681 secs 2024-04-27T05:21:14.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:14 smithi119 ceph-mon[32752]: pgmap v21163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:14.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:14 smithi167 ceph-mon[32751]: pgmap v21163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:14.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:14 smithi090 ceph-mon[29180]: pgmap v21163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:16 smithi119 ceph-mon[32752]: pgmap v21164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:16 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38757.585612 secs 2024-04-27T05:21:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:16 smithi167 ceph-mon[32751]: pgmap v21164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:16.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:16 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38757.585612 secs 2024-04-27T05:21:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:16 smithi090 ceph-mon[29180]: pgmap v21164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:16 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38757.585612 secs 2024-04-27T05:21:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38758.489792 secs 2024-04-27T05:21:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38758.489792 secs 2024-04-27T05:21:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38758.489792 secs 2024-04-27T05:21:18.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:18 smithi119 ceph-mon[32752]: pgmap v21165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:18.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:18 smithi167 ceph-mon[32751]: pgmap v21165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:18.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:18 smithi090 ceph-mon[29180]: pgmap v21165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:20.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:20 smithi119 ceph-mon[32752]: pgmap v21166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:20.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:20 smithi167 ceph-mon[32751]: pgmap v21166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:20.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:20 smithi090 ceph-mon[29180]: pgmap v21166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:21 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38762.585713 secs 2024-04-27T05:21:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:21 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38762.585713 secs 2024-04-27T05:21:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:21 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38762.585713 secs 2024-04-27T05:21:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:22 smithi119 ceph-mon[32752]: pgmap v21167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:22.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38763.489911 secs 2024-04-27T05:21:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:22 smithi167 ceph-mon[32751]: pgmap v21167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38763.489911 secs 2024-04-27T05:21:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:22 smithi090 ceph-mon[29180]: pgmap v21167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:22.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38763.489911 secs 2024-04-27T05:21:24.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:24 smithi119 ceph-mon[32752]: pgmap v21168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:24.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:24 smithi167 ceph-mon[32751]: pgmap v21168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:24.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:24 smithi090 ceph-mon[29180]: pgmap v21168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:26 smithi119 ceph-mon[32752]: pgmap v21169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:26.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:26 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38767.585812 secs 2024-04-27T05:21:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:26 smithi167 ceph-mon[32751]: pgmap v21169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:26 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38767.585812 secs 2024-04-27T05:21:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:26 smithi090 ceph-mon[29180]: pgmap v21169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:26.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:26 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38767.585812 secs 2024-04-27T05:21:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38768.489989 secs 2024-04-27T05:21:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38768.489989 secs 2024-04-27T05:21:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38768.489989 secs 2024-04-27T05:21:28.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:28 smithi167 ceph-mon[32751]: pgmap v21170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:28.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:28 smithi090 ceph-mon[29180]: pgmap v21170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:28 smithi119 ceph-mon[32752]: pgmap v21170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:30 smithi167 ceph-mon[32751]: pgmap v21171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:30 smithi090 ceph-mon[29180]: pgmap v21171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:30 smithi119 ceph-mon[32752]: pgmap v21171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:31 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38772.585902 secs 2024-04-27T05:21:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:31 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38772.585902 secs 2024-04-27T05:21:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:31 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38772.585902 secs 2024-04-27T05:21:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:32 smithi167 ceph-mon[32751]: pgmap v21172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:32.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38773.490123 secs 2024-04-27T05:21:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:32 smithi090 ceph-mon[29180]: pgmap v21172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38773.490123 secs 2024-04-27T05:21:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:32 smithi119 ceph-mon[32752]: pgmap v21172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38773.490123 secs 2024-04-27T05:21:34.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:34 smithi090 ceph-mon[29180]: pgmap v21173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:34 smithi119 ceph-mon[32752]: pgmap v21173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:34 smithi167 ceph-mon[32751]: pgmap v21173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:36 smithi090 ceph-mon[29180]: pgmap v21174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:36 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38777.586005 secs 2024-04-27T05:21:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:36 smithi119 ceph-mon[32752]: pgmap v21174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:36 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38777.586005 secs 2024-04-27T05:21:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:36 smithi167 ceph-mon[32751]: pgmap v21174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:36 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38777.586005 secs 2024-04-27T05:21:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38778.490269 secs 2024-04-27T05:21:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38778.490269 secs 2024-04-27T05:21:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38778.490269 secs 2024-04-27T05:21:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:38 smithi090 ceph-mon[29180]: pgmap v21175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:38 smithi119 ceph-mon[32752]: pgmap v21175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:38 smithi167 ceph-mon[32751]: pgmap v21175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:40 smithi090 ceph-mon[29180]: pgmap v21176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:40 smithi119 ceph-mon[32752]: pgmap v21176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:40 smithi167 ceph-mon[32751]: pgmap v21176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:41.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:41 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38782.586095 secs 2024-04-27T05:21:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:41 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38782.586095 secs 2024-04-27T05:21:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:41 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38782.586095 secs 2024-04-27T05:21:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:42 smithi090 ceph-mon[29180]: pgmap v21177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38783.490332 secs 2024-04-27T05:21:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:42 smithi119 ceph-mon[32752]: pgmap v21177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38783.490332 secs 2024-04-27T05:21:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:42 smithi167 ceph-mon[32751]: pgmap v21177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38783.490332 secs 2024-04-27T05:21:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:44 smithi090 ceph-mon[29180]: pgmap v21178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:44 smithi119 ceph-mon[32752]: pgmap v21178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:44 smithi167 ceph-mon[32751]: pgmap v21178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:46 smithi090 ceph-mon[29180]: pgmap v21179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:46.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:46 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38787.586153 secs 2024-04-27T05:21:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:46 smithi119 ceph-mon[32752]: pgmap v21179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:46 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38787.586153 secs 2024-04-27T05:21:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:46 smithi167 ceph-mon[32751]: pgmap v21179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:46 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38787.586153 secs 2024-04-27T05:21:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38788.490478 secs 2024-04-27T05:21:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38788.490478 secs 2024-04-27T05:21:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38788.490478 secs 2024-04-27T05:21:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:48 smithi090 ceph-mon[29180]: pgmap v21180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:48 smithi119 ceph-mon[32752]: pgmap v21180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:48 smithi167 ceph-mon[32751]: pgmap v21180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:50.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:50 smithi090 ceph-mon[29180]: pgmap v21181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:50 smithi119 ceph-mon[32752]: pgmap v21181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:50 smithi167 ceph-mon[32751]: pgmap v21181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:51 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38792.586235 secs 2024-04-27T05:21:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:51 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38792.586235 secs 2024-04-27T05:21:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:51 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38792.586235 secs 2024-04-27T05:21:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:52 smithi090 ceph-mon[29180]: pgmap v21182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38793.490574 secs 2024-04-27T05:21:52.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:52 smithi119 ceph-mon[32752]: pgmap v21182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38793.490574 secs 2024-04-27T05:21:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:52 smithi167 ceph-mon[32751]: pgmap v21182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:52.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38793.490574 secs 2024-04-27T05:21:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:54 smithi090 ceph-mon[29180]: pgmap v21183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:54 smithi119 ceph-mon[32752]: pgmap v21183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:54 smithi167 ceph-mon[32751]: pgmap v21183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:56 smithi119 ceph-mon[32752]: pgmap v21184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:56 smithi119 ceph-mon[32752]: 50 slow requests, 0 included below; oldest blocked for > 38797.586316 secs 2024-04-27T05:21:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:56 smithi167 ceph-mon[32751]: pgmap v21184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:56 smithi167 ceph-mon[32751]: 50 slow requests, 0 included below; oldest blocked for > 38797.586316 secs 2024-04-27T05:21:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:56 smithi090 ceph-mon[29180]: pgmap v21184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:21:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:56 smithi090 ceph-mon[29180]: 50 slow requests, 0 included below; oldest blocked for > 38797.586316 secs 2024-04-27T05:21:58.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38798.490687 secs 2024-04-27T05:21:58.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:57 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:21:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38798.490687 secs 2024-04-27T05:21:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:57 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:21:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38798.490687 secs 2024-04-27T05:21:58.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:57 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:21:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:21:58 smithi119 ceph-mon[32752]: pgmap v21185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:21:58 smithi167 ceph-mon[32751]: pgmap v21185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:21:59.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:21:58 smithi090 ceph-mon[29180]: pgmap v21185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:00 smithi119 ceph-mon[32752]: pgmap v21186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:00 smithi090 ceph-mon[29180]: pgmap v21186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:00 smithi167 ceph-mon[32751]: pgmap v21186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38802.586382 secs 2024-04-27T05:22:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:22:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:22:02.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:01 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38802.586382 secs 2024-04-27T05:22:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:22:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:22:02.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:01 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38802.586382 secs 2024-04-27T05:22:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:22:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:22:02.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:01 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:22:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:02 smithi119 ceph-mon[32752]: pgmap v21187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38803.490758 secs 2024-04-27T05:22:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:02 smithi119 ceph-mon[32752]: pgmap v21188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 B/s rd, 1 op/s 2024-04-27T05:22:03.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:02 smithi119 ceph-mon[32752]: pgmap v21189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:02 smithi167 ceph-mon[32751]: pgmap v21187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38803.490758 secs 2024-04-27T05:22:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:02 smithi167 ceph-mon[32751]: pgmap v21188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 B/s rd, 1 op/s 2024-04-27T05:22:03.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:02 smithi167 ceph-mon[32751]: pgmap v21189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:02 smithi090 ceph-mon[29180]: pgmap v21187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38803.490758 secs 2024-04-27T05:22:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:02 smithi090 ceph-mon[29180]: pgmap v21188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 B/s rd, 1 op/s 2024-04-27T05:22:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:02 smithi090 ceph-mon[29180]: pgmap v21189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:22:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:03 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:22:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:22:04.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:03 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:22:04.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:22:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:03 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:22:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:04 smithi119 ceph-mon[32752]: pgmap v21190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:22:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:04 smithi167 ceph-mon[32751]: pgmap v21190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:22:05.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:04 smithi090 ceph-mon[29180]: pgmap v21190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:22:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:06 smithi119 ceph-mon[32752]: pgmap v21191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:06 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38807.586462 secs 2024-04-27T05:22:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:06 smithi167 ceph-mon[32751]: pgmap v21191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:06 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38807.586462 secs 2024-04-27T05:22:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:06 smithi090 ceph-mon[29180]: pgmap v21191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:06 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38807.586462 secs 2024-04-27T05:22:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38808.490854 secs 2024-04-27T05:22:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38808.490854 secs 2024-04-27T05:22:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38808.490854 secs 2024-04-27T05:22:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:08 smithi119 ceph-mon[32752]: pgmap v21192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:22:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:08 smithi167 ceph-mon[32751]: pgmap v21192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:22:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:08 smithi090 ceph-mon[29180]: pgmap v21192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:22:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:10 smithi119 ceph-mon[32752]: pgmap v21193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:10 smithi167 ceph-mon[32751]: pgmap v21193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:10 smithi090 ceph-mon[29180]: pgmap v21193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:11 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38812.586547 secs 2024-04-27T05:22:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:11 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38812.586547 secs 2024-04-27T05:22:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:11 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38812.586547 secs 2024-04-27T05:22:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38813.490991 secs 2024-04-27T05:22:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:12 smithi119 ceph-mon[32752]: pgmap v21194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:22:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38813.490991 secs 2024-04-27T05:22:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:12 smithi167 ceph-mon[32751]: pgmap v21194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:22:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38813.490991 secs 2024-04-27T05:22:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:12 smithi090 ceph-mon[29180]: pgmap v21194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:22:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:14 smithi119 ceph-mon[32752]: pgmap v21195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:14 smithi167 ceph-mon[32751]: pgmap v21195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:14 smithi090 ceph-mon[29180]: pgmap v21195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:16 smithi119 ceph-mon[32752]: pgmap v21196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:16 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38817.586628 secs 2024-04-27T05:22:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:16 smithi167 ceph-mon[32751]: pgmap v21196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:16 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38817.586628 secs 2024-04-27T05:22:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:16 smithi090 ceph-mon[29180]: pgmap v21196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:16 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38817.586628 secs 2024-04-27T05:22:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38818.491122 secs 2024-04-27T05:22:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38818.491122 secs 2024-04-27T05:22:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38818.491122 secs 2024-04-27T05:22:19.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:18 smithi119 ceph-mon[32752]: pgmap v21197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:18 smithi167 ceph-mon[32751]: pgmap v21197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:18 smithi090 ceph-mon[29180]: pgmap v21197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:20 smithi119 ceph-mon[32752]: pgmap v21198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:20 smithi167 ceph-mon[32751]: pgmap v21198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:20 smithi090 ceph-mon[29180]: pgmap v21198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:21 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38822.586692 secs 2024-04-27T05:22:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:21 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38822.586692 secs 2024-04-27T05:22:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:21 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38822.586692 secs 2024-04-27T05:22:22.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38823.491210 secs 2024-04-27T05:22:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:22 smithi167 ceph-mon[32751]: pgmap v21199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38823.491210 secs 2024-04-27T05:22:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:22 smithi119 ceph-mon[32752]: pgmap v21199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38823.491210 secs 2024-04-27T05:22:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:22 smithi090 ceph-mon[29180]: pgmap v21199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:24 smithi119 ceph-mon[32752]: pgmap v21200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:24 smithi167 ceph-mon[32751]: pgmap v21200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:24 smithi090 ceph-mon[29180]: pgmap v21200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:26 smithi119 ceph-mon[32752]: pgmap v21201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:26 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38827.586774 secs 2024-04-27T05:22:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:26 smithi167 ceph-mon[32751]: pgmap v21201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:26 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38827.586774 secs 2024-04-27T05:22:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:26 smithi090 ceph-mon[29180]: pgmap v21201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:26 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38827.586774 secs 2024-04-27T05:22:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38828.491336 secs 2024-04-27T05:22:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38828.491336 secs 2024-04-27T05:22:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38828.491336 secs 2024-04-27T05:22:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:28 smithi119 ceph-mon[32752]: pgmap v21202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:28 smithi167 ceph-mon[32751]: pgmap v21202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:28 smithi090 ceph-mon[29180]: pgmap v21202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:30 smithi119 ceph-mon[32752]: pgmap v21203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:30 smithi167 ceph-mon[32751]: pgmap v21203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:30 smithi090 ceph-mon[29180]: pgmap v21203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38832.586859 secs 2024-04-27T05:22:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38832.586859 secs 2024-04-27T05:22:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38832.586859 secs 2024-04-27T05:22:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38833.491490 secs 2024-04-27T05:22:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:32 smithi119 ceph-mon[32752]: pgmap v21204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38833.491490 secs 2024-04-27T05:22:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:32 smithi167 ceph-mon[32751]: pgmap v21204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38833.491490 secs 2024-04-27T05:22:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:32 smithi090 ceph-mon[29180]: pgmap v21204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:34 smithi119 ceph-mon[32752]: pgmap v21205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:34 smithi167 ceph-mon[32751]: pgmap v21205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:34 smithi090 ceph-mon[29180]: pgmap v21205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:36 smithi119 ceph-mon[32752]: pgmap v21206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38837.586941 secs 2024-04-27T05:22:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:36 smithi167 ceph-mon[32751]: pgmap v21206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38837.586941 secs 2024-04-27T05:22:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:36 smithi090 ceph-mon[29180]: pgmap v21206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38837.586941 secs 2024-04-27T05:22:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38838.491599 secs 2024-04-27T05:22:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38838.491599 secs 2024-04-27T05:22:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38838.491599 secs 2024-04-27T05:22:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:38 smithi119 ceph-mon[32752]: pgmap v21207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:38 smithi167 ceph-mon[32751]: pgmap v21207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:38 smithi090 ceph-mon[29180]: pgmap v21207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:40 smithi119 ceph-mon[32752]: pgmap v21208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:40 smithi167 ceph-mon[32751]: pgmap v21208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:40 smithi090 ceph-mon[29180]: pgmap v21208: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38842.587031 secs 2024-04-27T05:22:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38842.587031 secs 2024-04-27T05:22:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38842.587031 secs 2024-04-27T05:22:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38843.491719 secs 2024-04-27T05:22:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:42 smithi119 ceph-mon[32752]: pgmap v21209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38843.491719 secs 2024-04-27T05:22:43.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:42 smithi167 ceph-mon[32751]: pgmap v21209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38843.491719 secs 2024-04-27T05:22:43.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:42 smithi090 ceph-mon[29180]: pgmap v21209: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:44 smithi119 ceph-mon[32752]: pgmap v21210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:44 smithi167 ceph-mon[32751]: pgmap v21210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:45.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:44 smithi090 ceph-mon[29180]: pgmap v21210: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:46 smithi119 ceph-mon[32752]: pgmap v21211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:46 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38847.587111 secs 2024-04-27T05:22:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:46 smithi167 ceph-mon[32751]: pgmap v21211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:46 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38847.587111 secs 2024-04-27T05:22:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:46 smithi090 ceph-mon[29180]: pgmap v21211: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:46 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38847.587111 secs 2024-04-27T05:22:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38848.491834 secs 2024-04-27T05:22:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38848.491834 secs 2024-04-27T05:22:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38848.491834 secs 2024-04-27T05:22:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:48 smithi119 ceph-mon[32752]: pgmap v21212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:48 smithi167 ceph-mon[32751]: pgmap v21212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:48 smithi090 ceph-mon[29180]: pgmap v21212: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:50.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:49 smithi090 ceph-mon[29180]: pgmap v21213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:50.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:49 smithi119 ceph-mon[32752]: pgmap v21213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:50.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:49 smithi167 ceph-mon[32751]: pgmap v21213: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:51 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38852.587170 secs 2024-04-27T05:22:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:51 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38852.587170 secs 2024-04-27T05:22:51.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:51 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38852.587170 secs 2024-04-27T05:22:52.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38853.491993 secs 2024-04-27T05:22:52.457 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:52 smithi119 ceph-mon[32752]: pgmap v21214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:52.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38853.491993 secs 2024-04-27T05:22:52.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:52 smithi167 ceph-mon[32751]: pgmap v21214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38853.491993 secs 2024-04-27T05:22:52.630 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:52 smithi090 ceph-mon[29180]: pgmap v21214: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:54 smithi090 ceph-mon[29180]: pgmap v21215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:54 smithi119 ceph-mon[32752]: pgmap v21215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:54 smithi167 ceph-mon[32751]: pgmap v21215: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:56 smithi090 ceph-mon[29180]: pgmap v21216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:56 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38857.587244 secs 2024-04-27T05:22:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:56 smithi119 ceph-mon[32752]: pgmap v21216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:56.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:56 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38857.587244 secs 2024-04-27T05:22:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:56 smithi167 ceph-mon[32751]: pgmap v21216: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:22:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:56 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38857.587244 secs 2024-04-27T05:22:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38858.492103 secs 2024-04-27T05:22:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38858.492103 secs 2024-04-27T05:22:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38858.492103 secs 2024-04-27T05:22:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:22:58 smithi090 ceph-mon[29180]: pgmap v21217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:22:58 smithi119 ceph-mon[32752]: pgmap v21217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:22:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:22:58 smithi167 ceph-mon[32751]: pgmap v21217: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:00 smithi090 ceph-mon[29180]: pgmap v21218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:00 smithi119 ceph-mon[32752]: pgmap v21218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:00 smithi167 ceph-mon[32751]: pgmap v21218: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:01 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38862.587324 secs 2024-04-27T05:23:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:01 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38862.587324 secs 2024-04-27T05:23:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:01 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38862.587324 secs 2024-04-27T05:23:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38863.492218 secs 2024-04-27T05:23:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:02 smithi090 ceph-mon[29180]: pgmap v21219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:02 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:23:02.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38863.492218 secs 2024-04-27T05:23:02.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:02 smithi167 ceph-mon[32751]: pgmap v21219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:02.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:02 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:23:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38863.492218 secs 2024-04-27T05:23:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:02 smithi119 ceph-mon[32752]: pgmap v21219: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:02.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:02 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:23:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:04 smithi167 ceph-mon[32751]: pgmap v21220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:04 smithi090 ceph-mon[29180]: pgmap v21220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:04 smithi119 ceph-mon[32752]: pgmap v21220: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: pgmap v21221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38867.587380 secs 2024-04-27T05:23:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:23:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:23:07.057 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: pgmap v21222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-27T05:23:07.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: pgmap v21223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:07.058 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: pgmap v21221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38867.587380 secs 2024-04-27T05:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: pgmap v21222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-27T05:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: pgmap v21223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: pgmap v21221: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38867.587380 secs 2024-04-27T05:23:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:23:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:23:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: pgmap v21222: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 B/s rd, 1 op/s 2024-04-27T05:23:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: pgmap v21223: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:07.129 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:23:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38868.492354 secs 2024-04-27T05:23:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38868.492354 secs 2024-04-27T05:23:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38868.492354 secs 2024-04-27T05:23:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:08 smithi119 ceph-mon[32752]: pgmap v21224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T05:23:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:08 smithi167 ceph-mon[32751]: pgmap v21224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T05:23:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:08 smithi090 ceph-mon[29180]: pgmap v21224: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 607 B/s rd, 1 op/s 2024-04-27T05:23:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:10 smithi119 ceph-mon[32752]: pgmap v21225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:10 smithi167 ceph-mon[32751]: pgmap v21225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:10 smithi090 ceph-mon[29180]: pgmap v21225: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:11 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38872.587441 secs 2024-04-27T05:23:11.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:23:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:23:11.583+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:23:11.878 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:23:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:23:11.583+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:23:11.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:23:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:23:11.583+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:23:11.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:23:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:23:11.583+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:23:11.879 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:23:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:23:11.583+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:23:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:11 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38872.587441 secs 2024-04-27T05:23:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:11 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38872.587441 secs 2024-04-27T05:23:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38873.492401 secs 2024-04-27T05:23:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:12 smithi119 ceph-mon[32752]: pgmap v21226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38873.492401 secs 2024-04-27T05:23:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:12 smithi167 ceph-mon[32751]: pgmap v21226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38873.492401 secs 2024-04-27T05:23:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:12 smithi090 ceph-mon[29180]: pgmap v21226: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:14 smithi119 ceph-mon[32752]: pgmap v21227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:14 smithi167 ceph-mon[32751]: pgmap v21227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:14 smithi090 ceph-mon[29180]: pgmap v21227: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:16 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38877.587499 secs 2024-04-27T05:23:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:16 smithi119 ceph-mon[32752]: pgmap v21228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:23:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:16 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38877.587499 secs 2024-04-27T05:23:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:16 smithi167 ceph-mon[32751]: pgmap v21228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:23:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:16 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38877.587499 secs 2024-04-27T05:23:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:16 smithi090 ceph-mon[29180]: pgmap v21228: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:23:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38878.492471 secs 2024-04-27T05:23:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38878.492471 secs 2024-04-27T05:23:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38878.492471 secs 2024-04-27T05:23:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:18 smithi119 ceph-mon[32752]: pgmap v21229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:18 smithi167 ceph-mon[32751]: pgmap v21229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:18 smithi090 ceph-mon[29180]: pgmap v21229: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:20 smithi119 ceph-mon[32752]: pgmap v21230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:20 smithi167 ceph-mon[32751]: pgmap v21230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:20 smithi090 ceph-mon[29180]: pgmap v21230: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:21 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38882.587557 secs 2024-04-27T05:23:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:21 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38882.587557 secs 2024-04-27T05:23:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:21 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38882.587557 secs 2024-04-27T05:23:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38883.492551 secs 2024-04-27T05:23:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:22 smithi119 ceph-mon[32752]: pgmap v21231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38883.492551 secs 2024-04-27T05:23:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:22 smithi167 ceph-mon[32751]: pgmap v21231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38883.492551 secs 2024-04-27T05:23:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:22 smithi090 ceph-mon[29180]: pgmap v21231: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:24 smithi119 ceph-mon[32752]: pgmap v21232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:24 smithi167 ceph-mon[32751]: pgmap v21232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:24 smithi090 ceph-mon[29180]: pgmap v21232: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:26 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38887.587613 secs 2024-04-27T05:23:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:26 smithi119 ceph-mon[32752]: pgmap v21233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:26 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38887.587613 secs 2024-04-27T05:23:27.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:26 smithi167 ceph-mon[32751]: pgmap v21233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:26 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38887.587613 secs 2024-04-27T05:23:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:26 smithi090 ceph-mon[29180]: pgmap v21233: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38888.492612 secs 2024-04-27T05:23:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38888.492612 secs 2024-04-27T05:23:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38888.492612 secs 2024-04-27T05:23:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:28 smithi119 ceph-mon[32752]: pgmap v21234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:28 smithi167 ceph-mon[32751]: pgmap v21234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:28 smithi090 ceph-mon[29180]: pgmap v21234: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:30 smithi119 ceph-mon[32752]: pgmap v21235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:30 smithi167 ceph-mon[32751]: pgmap v21235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:30 smithi090 ceph-mon[29180]: pgmap v21235: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38892.587673 secs 2024-04-27T05:23:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38892.587673 secs 2024-04-27T05:23:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38892.587673 secs 2024-04-27T05:23:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38893.492685 secs 2024-04-27T05:23:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:32 smithi119 ceph-mon[32752]: pgmap v21236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38893.492685 secs 2024-04-27T05:23:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:32 smithi167 ceph-mon[32751]: pgmap v21236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38893.492685 secs 2024-04-27T05:23:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:32 smithi090 ceph-mon[29180]: pgmap v21236: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:34 smithi119 ceph-mon[32752]: pgmap v21237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:34 smithi167 ceph-mon[32751]: pgmap v21237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:34 smithi090 ceph-mon[29180]: pgmap v21237: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38897.587753 secs 2024-04-27T05:23:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:36 smithi119 ceph-mon[32752]: pgmap v21238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38897.587753 secs 2024-04-27T05:23:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:36 smithi167 ceph-mon[32751]: pgmap v21238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38897.587753 secs 2024-04-27T05:23:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:36 smithi090 ceph-mon[29180]: pgmap v21238: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38898.492786 secs 2024-04-27T05:23:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38898.492786 secs 2024-04-27T05:23:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38898.492786 secs 2024-04-27T05:23:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:38 smithi119 ceph-mon[32752]: pgmap v21239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:38 smithi167 ceph-mon[32751]: pgmap v21239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:38 smithi090 ceph-mon[29180]: pgmap v21239: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:39 smithi119 ceph-mon[32752]: pgmap v21240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:39 smithi167 ceph-mon[32751]: pgmap v21240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:39 smithi090 ceph-mon[29180]: pgmap v21240: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38902.587840 secs 2024-04-27T05:23:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38902.587840 secs 2024-04-27T05:23:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38902.587840 secs 2024-04-27T05:23:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38903.492928 secs 2024-04-27T05:23:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:42 smithi119 ceph-mon[32752]: pgmap v21241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38903.492928 secs 2024-04-27T05:23:42.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:42 smithi167 ceph-mon[32751]: pgmap v21241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38903.492928 secs 2024-04-27T05:23:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:42 smithi090 ceph-mon[29180]: pgmap v21241: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:44 smithi119 ceph-mon[32752]: pgmap v21242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:44 smithi167 ceph-mon[32751]: pgmap v21242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:45.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:44 smithi090 ceph-mon[29180]: pgmap v21242: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:46 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38907.587921 secs 2024-04-27T05:23:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:46 smithi119 ceph-mon[32752]: pgmap v21243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:46 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38907.587921 secs 2024-04-27T05:23:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:46 smithi167 ceph-mon[32751]: pgmap v21243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:46 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38907.587921 secs 2024-04-27T05:23:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:46 smithi090 ceph-mon[29180]: pgmap v21243: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38908.493030 secs 2024-04-27T05:23:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38908.493030 secs 2024-04-27T05:23:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38908.493030 secs 2024-04-27T05:23:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:48 smithi119 ceph-mon[32752]: pgmap v21244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:48 smithi167 ceph-mon[32751]: pgmap v21244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:48 smithi090 ceph-mon[29180]: pgmap v21244: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:49 smithi119 ceph-mon[32752]: pgmap v21245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:49 smithi167 ceph-mon[32751]: pgmap v21245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:49 smithi090 ceph-mon[29180]: pgmap v21245: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:51.558 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:51 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38912.588006 secs 2024-04-27T05:23:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:51 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38912.588006 secs 2024-04-27T05:23:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:51 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38912.588006 secs 2024-04-27T05:23:52.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38913.493184 secs 2024-04-27T05:23:52.456 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:52 smithi119 ceph-mon[32752]: pgmap v21246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:52.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38913.493184 secs 2024-04-27T05:23:52.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:52 smithi167 ceph-mon[32751]: pgmap v21246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38913.493184 secs 2024-04-27T05:23:52.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:52 smithi090 ceph-mon[29180]: pgmap v21246: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:54 smithi119 ceph-mon[32752]: pgmap v21247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:54 smithi167 ceph-mon[32751]: pgmap v21247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:54 smithi090 ceph-mon[29180]: pgmap v21247: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:23:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:56 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38917.588065 secs 2024-04-27T05:23:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:56 smithi119 ceph-mon[32752]: pgmap v21248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:56 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38917.588065 secs 2024-04-27T05:23:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:56 smithi167 ceph-mon[32751]: pgmap v21248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:56 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38917.588065 secs 2024-04-27T05:23:57.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:56 smithi090 ceph-mon[29180]: pgmap v21248: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38918.493298 secs 2024-04-27T05:23:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38918.493298 secs 2024-04-27T05:23:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38918.493298 secs 2024-04-27T05:23:59.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:58 smithi119 ceph-mon[32752]: pgmap v21249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:58 smithi167 ceph-mon[32751]: pgmap v21249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:23:59.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:58 smithi090 ceph-mon[29180]: pgmap v21249: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:23:59 smithi119 ceph-mon[32752]: pgmap v21250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:23:59 smithi167 ceph-mon[32751]: pgmap v21250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:23:59 smithi090 ceph-mon[29180]: pgmap v21250: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:01 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38922.588150 secs 2024-04-27T05:24:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:01 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38922.588150 secs 2024-04-27T05:24:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:01 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38922.588150 secs 2024-04-27T05:24:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38923.493440 secs 2024-04-27T05:24:02.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:02 smithi119 ceph-mon[32752]: pgmap v21251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38923.493440 secs 2024-04-27T05:24:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:02 smithi167 ceph-mon[32751]: pgmap v21251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:02.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38923.493440 secs 2024-04-27T05:24:02.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:02 smithi090 ceph-mon[29180]: pgmap v21251: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:05.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:04 smithi119 ceph-mon[32752]: pgmap v21252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:05.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:04 smithi167 ceph-mon[32751]: pgmap v21252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:05.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:04 smithi090 ceph-mon[29180]: pgmap v21252: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:06 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38927.588232 secs 2024-04-27T05:24:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:06 smithi119 ceph-mon[32752]: pgmap v21253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:24:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:24:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:24:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:24:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:06 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38927.588232 secs 2024-04-27T05:24:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:06 smithi167 ceph-mon[32751]: pgmap v21253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:24:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:24:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:24:07.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:24:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:06 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38927.588232 secs 2024-04-27T05:24:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:06 smithi090 ceph-mon[29180]: pgmap v21253: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:24:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:24:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:24:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:24:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38928.493526 secs 2024-04-27T05:24:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:07 smithi119 ceph-mon[32752]: pgmap v21254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T05:24:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:07 smithi119 ceph-mon[32752]: pgmap v21255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38928.493526 secs 2024-04-27T05:24:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:07 smithi167 ceph-mon[32751]: pgmap v21254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T05:24:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:07 smithi167 ceph-mon[32751]: pgmap v21255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38928.493526 secs 2024-04-27T05:24:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:07 smithi090 ceph-mon[29180]: pgmap v21254: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 B/s rd, 1 op/s 2024-04-27T05:24:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:07 smithi090 ceph-mon[29180]: pgmap v21255: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:09 smithi119 ceph-mon[32752]: pgmap v21256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:09 smithi167 ceph-mon[32751]: pgmap v21256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:09 smithi090 ceph-mon[29180]: pgmap v21256: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:10 smithi119 ceph-mon[32752]: pgmap v21257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:10 smithi167 ceph-mon[32751]: pgmap v21257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:10 smithi090 ceph-mon[29180]: pgmap v21257: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:11 smithi119 ceph-mon[32752]: 27 slow requests, 1 included below; oldest blocked for > 38932.588316 secs 2024-04-27T05:24:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:11 smithi119 ceph-mon[32752]: slow request 5760.374767 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:24:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:11 smithi167 ceph-mon[32751]: 27 slow requests, 1 included below; oldest blocked for > 38932.588316 secs 2024-04-27T05:24:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:11 smithi167 ceph-mon[32751]: slow request 5760.374767 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:24:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:11 smithi090 ceph-mon[29180]: 27 slow requests, 1 included below; oldest blocked for > 38932.588316 secs 2024-04-27T05:24:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:11 smithi090 ceph-mon[29180]: slow request 5760.374767 seconds old, received at 2024-04-27T03:48:09.966954+0000: client_request(client.27050:1 getattr pAsLsXsFs #0x1 2024-04-27T03:48:09.959889+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:24:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38933.493646 secs 2024-04-27T05:24:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:12 smithi119 ceph-mon[32752]: pgmap v21258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T05:24:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38933.493646 secs 2024-04-27T05:24:13.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:12 smithi167 ceph-mon[32751]: pgmap v21258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T05:24:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38933.493646 secs 2024-04-27T05:24:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:12 smithi090 ceph-mon[29180]: pgmap v21258: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 579 B/s rd, 1 op/s 2024-04-27T05:24:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:15 smithi090 ceph-mon[29180]: pgmap v21259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:15 smithi119 ceph-mon[32752]: pgmap v21259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:15 smithi167 ceph-mon[32751]: pgmap v21259: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:24:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:16 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38937.588400 secs 2024-04-27T05:24:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:16 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38937.588400 secs 2024-04-27T05:24:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:16 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38937.588400 secs 2024-04-27T05:24:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38938.493771 secs 2024-04-27T05:24:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:17 smithi090 ceph-mon[29180]: pgmap v21260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:24:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38938.493771 secs 2024-04-27T05:24:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:17 smithi119 ceph-mon[32752]: pgmap v21260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:24:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38938.493771 secs 2024-04-27T05:24:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:17 smithi167 ceph-mon[32751]: pgmap v21260: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:24:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:19 smithi090 ceph-mon[29180]: pgmap v21261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:19 smithi119 ceph-mon[32752]: pgmap v21261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:20.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:19 smithi167 ceph-mon[32751]: pgmap v21261: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:21 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38942.588486 secs 2024-04-27T05:24:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:21 smithi090 ceph-mon[29180]: pgmap v21262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:21 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38942.588486 secs 2024-04-27T05:24:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:21 smithi119 ceph-mon[32752]: pgmap v21262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:21 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38942.588486 secs 2024-04-27T05:24:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:21 smithi167 ceph-mon[32751]: pgmap v21262: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:22.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38943.493898 secs 2024-04-27T05:24:22.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38943.493898 secs 2024-04-27T05:24:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38943.493898 secs 2024-04-27T05:24:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:23 smithi119 ceph-mon[32752]: pgmap v21263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:23 smithi167 ceph-mon[32751]: pgmap v21263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:23 smithi090 ceph-mon[29180]: pgmap v21263: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:25 smithi119 ceph-mon[32752]: pgmap v21264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:25 smithi167 ceph-mon[32751]: pgmap v21264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:25 smithi090 ceph-mon[29180]: pgmap v21264: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:26 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38947.588567 secs 2024-04-27T05:24:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:26 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38947.588567 secs 2024-04-27T05:24:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:26 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38947.588567 secs 2024-04-27T05:24:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38948.493960 secs 2024-04-27T05:24:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:27 smithi119 ceph-mon[32752]: pgmap v21265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38948.493960 secs 2024-04-27T05:24:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:27 smithi167 ceph-mon[32751]: pgmap v21265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38948.493960 secs 2024-04-27T05:24:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:27 smithi090 ceph-mon[29180]: pgmap v21265: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:29 smithi119 ceph-mon[32752]: pgmap v21266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:29 smithi167 ceph-mon[32751]: pgmap v21266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:29 smithi090 ceph-mon[29180]: pgmap v21266: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:31 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38952.588646 secs 2024-04-27T05:24:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:31 smithi119 ceph-mon[32752]: pgmap v21267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:31 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38952.588646 secs 2024-04-27T05:24:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:31 smithi167 ceph-mon[32751]: pgmap v21267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:31 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38952.588646 secs 2024-04-27T05:24:32.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:31 smithi090 ceph-mon[29180]: pgmap v21267: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:33.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38953.494082 secs 2024-04-27T05:24:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38953.494082 secs 2024-04-27T05:24:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38953.494082 secs 2024-04-27T05:24:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:33 smithi119 ceph-mon[32752]: pgmap v21268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:33 smithi167 ceph-mon[32751]: pgmap v21268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:34.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:33 smithi090 ceph-mon[29180]: pgmap v21268: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:35 smithi119 ceph-mon[32752]: pgmap v21269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:35 smithi167 ceph-mon[32751]: pgmap v21269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:35 smithi090 ceph-mon[29180]: pgmap v21269: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:36 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38957.588728 secs 2024-04-27T05:24:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:36 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38957.588728 secs 2024-04-27T05:24:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:36 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38957.588728 secs 2024-04-27T05:24:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38958.494197 secs 2024-04-27T05:24:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:37 smithi119 ceph-mon[32752]: pgmap v21270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38958.494197 secs 2024-04-27T05:24:38.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:37 smithi167 ceph-mon[32751]: pgmap v21270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38958.494197 secs 2024-04-27T05:24:38.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:37 smithi090 ceph-mon[29180]: pgmap v21270: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:39 smithi119 ceph-mon[32752]: pgmap v21271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:39 smithi167 ceph-mon[32751]: pgmap v21271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:39 smithi090 ceph-mon[29180]: pgmap v21271: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:41 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38962.588831 secs 2024-04-27T05:24:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:41 smithi119 ceph-mon[32752]: pgmap v21272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:41 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38962.588831 secs 2024-04-27T05:24:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:41 smithi167 ceph-mon[32751]: pgmap v21272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:41 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38962.588831 secs 2024-04-27T05:24:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:41 smithi090 ceph-mon[29180]: pgmap v21272: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 38963.494317 secs 2024-04-27T05:24:43.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:42 smithi119 ceph-mon[32752]: slow request 2880.452966 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:24:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 38963.494317 secs 2024-04-27T05:24:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:42 smithi167 ceph-mon[32751]: slow request 2880.452966 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:24:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 38963.494317 secs 2024-04-27T05:24:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:42 smithi090 ceph-mon[29180]: slow request 2880.452966 seconds old, received at 2024-04-27T04:36:40.791995+0000: client_request(client.17790:1 getattr pAsLsXsFs #0x1 2024-04-27T04:36:40.788383+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:24:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:43 smithi119 ceph-mon[32752]: pgmap v21273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:43 smithi167 ceph-mon[32751]: pgmap v21273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:44.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:43 smithi090 ceph-mon[29180]: pgmap v21273: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:45 smithi119 ceph-mon[32752]: pgmap v21274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:45 smithi167 ceph-mon[32751]: pgmap v21274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:45 smithi090 ceph-mon[29180]: pgmap v21274: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:46 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38967.588917 secs 2024-04-27T05:24:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:46 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38967.588917 secs 2024-04-27T05:24:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:46 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38967.588917 secs 2024-04-27T05:24:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38968.494435 secs 2024-04-27T05:24:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:47 smithi119 ceph-mon[32752]: pgmap v21275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38968.494435 secs 2024-04-27T05:24:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:47 smithi167 ceph-mon[32751]: pgmap v21275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38968.494435 secs 2024-04-27T05:24:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:47 smithi090 ceph-mon[29180]: pgmap v21275: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:49 smithi119 ceph-mon[32752]: pgmap v21276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:49 smithi167 ceph-mon[32751]: pgmap v21276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:49 smithi090 ceph-mon[29180]: pgmap v21276: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:51 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38972.589006 secs 2024-04-27T05:24:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:51 smithi119 ceph-mon[32752]: pgmap v21277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:51 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38972.589006 secs 2024-04-27T05:24:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:51 smithi167 ceph-mon[32751]: pgmap v21277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:51 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38972.589006 secs 2024-04-27T05:24:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:51 smithi090 ceph-mon[29180]: pgmap v21277: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:52.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38973.494566 secs 2024-04-27T05:24:52.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38973.494566 secs 2024-04-27T05:24:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38973.494566 secs 2024-04-27T05:24:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:53 smithi119 ceph-mon[32752]: pgmap v21278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:53 smithi167 ceph-mon[32751]: pgmap v21278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:53 smithi090 ceph-mon[29180]: pgmap v21278: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:55 smithi119 ceph-mon[32752]: pgmap v21279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:55 smithi167 ceph-mon[32751]: pgmap v21279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:55 smithi090 ceph-mon[29180]: pgmap v21279: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:24:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:56 smithi119 ceph-mon[32752]: 27 slow requests, 0 included below; oldest blocked for > 38977.589064 secs 2024-04-27T05:24:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:56 smithi167 ceph-mon[32751]: 27 slow requests, 0 included below; oldest blocked for > 38977.589064 secs 2024-04-27T05:24:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:56 smithi090 ceph-mon[29180]: 27 slow requests, 0 included below; oldest blocked for > 38977.589064 secs 2024-04-27T05:24:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38978.494668 secs 2024-04-27T05:24:58.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:57 smithi119 ceph-mon[32752]: pgmap v21280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38978.494668 secs 2024-04-27T05:24:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:57 smithi167 ceph-mon[32751]: pgmap v21280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:24:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38978.494668 secs 2024-04-27T05:24:58.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:57 smithi090 ceph-mon[29180]: pgmap v21280: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:24:59 smithi119 ceph-mon[32752]: pgmap v21281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:24:59 smithi167 ceph-mon[32751]: pgmap v21281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:24:59 smithi090 ceph-mon[29180]: pgmap v21281: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:01 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 38982.589133 secs 2024-04-27T05:25:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:01 smithi119 ceph-mon[32752]: slow request 180.237432 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:25:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:01 smithi119 ceph-mon[32752]: pgmap v21282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:01 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 38982.589133 secs 2024-04-27T05:25:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:01 smithi167 ceph-mon[32751]: slow request 180.237432 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:25:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:01 smithi167 ceph-mon[32751]: pgmap v21282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:01 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 38982.589133 secs 2024-04-27T05:25:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:01 smithi090 ceph-mon[29180]: slow request 180.237432 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:25:02.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:01 smithi090 ceph-mon[29180]: pgmap v21282: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38983.494795 secs 2024-04-27T05:25:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38983.494795 secs 2024-04-27T05:25:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38983.494795 secs 2024-04-27T05:25:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:03 smithi119 ceph-mon[32752]: pgmap v21283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:03 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:25:04.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:03 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:25:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:03 smithi167 ceph-mon[32751]: pgmap v21283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:03 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:25:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:03 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:25:04.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:03 smithi090 ceph-mon[29180]: pgmap v21283: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:03 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:25:04.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:03 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:25:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:05 smithi119 ceph-mon[32752]: pgmap v21284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:05 smithi167 ceph-mon[32751]: pgmap v21284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:05 smithi090 ceph-mon[29180]: pgmap v21284: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 38987.589214 secs 2024-04-27T05:25:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:06 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:25:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 38987.589214 secs 2024-04-27T05:25:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:06 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:25:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 38987.589214 secs 2024-04-27T05:25:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:06 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:25:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38988.494930 secs 2024-04-27T05:25:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:07 smithi119 ceph-mon[32752]: pgmap v21285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:25:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:25:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:25:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38988.494930 secs 2024-04-27T05:25:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:07 smithi167 ceph-mon[32751]: pgmap v21285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:25:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:25:08.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:25:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38988.494930 secs 2024-04-27T05:25:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:07 smithi090 ceph-mon[29180]: pgmap v21285: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:25:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:25:08.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:25:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:08 smithi119 ceph-mon[32752]: pgmap v21286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:25:09.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:08 smithi119 ceph-mon[32752]: pgmap v21287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:25:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:08 smithi167 ceph-mon[32751]: pgmap v21286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:25:09.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:08 smithi167 ceph-mon[32751]: pgmap v21287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:25:09.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:08 smithi090 ceph-mon[29180]: pgmap v21286: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:25:09.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:08 smithi090 ceph-mon[29180]: pgmap v21287: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:25:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:10 smithi119 ceph-mon[32752]: pgmap v21288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:10 smithi167 ceph-mon[32751]: pgmap v21288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:11.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:10 smithi090 ceph-mon[29180]: pgmap v21288: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 38992.589291 secs 2024-04-27T05:25:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 38992.589291 secs 2024-04-27T05:25:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 38992.589291 secs 2024-04-27T05:25:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38993.495047 secs 2024-04-27T05:25:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:12 smithi119 ceph-mon[32752]: pgmap v21289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38993.495047 secs 2024-04-27T05:25:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:12 smithi167 ceph-mon[32751]: pgmap v21289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38993.495047 secs 2024-04-27T05:25:13.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:12 smithi090 ceph-mon[29180]: pgmap v21289: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:14 smithi119 ceph-mon[32752]: pgmap v21290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:25:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:14 smithi167 ceph-mon[32751]: pgmap v21290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:25:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:14 smithi090 ceph-mon[29180]: pgmap v21290: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:25:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:16 smithi119 ceph-mon[32752]: pgmap v21291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 38997.589373 secs 2024-04-27T05:25:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:16 smithi167 ceph-mon[32751]: pgmap v21291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 38997.589373 secs 2024-04-27T05:25:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:16 smithi090 ceph-mon[29180]: pgmap v21291: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:25:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 38997.589373 secs 2024-04-27T05:25:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 38998.495167 secs 2024-04-27T05:25:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 38998.495167 secs 2024-04-27T05:25:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 38998.495167 secs 2024-04-27T05:25:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:18 smithi119 ceph-mon[32752]: pgmap v21292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:25:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:18 smithi167 ceph-mon[32751]: pgmap v21292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:25:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:18 smithi090 ceph-mon[29180]: pgmap v21292: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:25:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:20 smithi119 ceph-mon[32752]: pgmap v21293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:20 smithi167 ceph-mon[32751]: pgmap v21293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:20 smithi090 ceph-mon[29180]: pgmap v21293: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39002.589457 secs 2024-04-27T05:25:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39002.589457 secs 2024-04-27T05:25:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39002.589457 secs 2024-04-27T05:25:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39003.495294 secs 2024-04-27T05:25:23.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:22 smithi119 ceph-mon[32752]: pgmap v21294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39003.495294 secs 2024-04-27T05:25:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:22 smithi167 ceph-mon[32751]: pgmap v21294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39003.495294 secs 2024-04-27T05:25:23.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:22 smithi090 ceph-mon[29180]: pgmap v21294: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:24 smithi119 ceph-mon[32752]: pgmap v21295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:24 smithi167 ceph-mon[32751]: pgmap v21295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:24 smithi090 ceph-mon[29180]: pgmap v21295: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:25 smithi090 ceph-mon[29180]: pgmap v21296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:26.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:25 smithi119 ceph-mon[32752]: pgmap v21296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:26.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:25 smithi167 ceph-mon[32751]: pgmap v21296: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39007.589537 secs 2024-04-27T05:25:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39007.589537 secs 2024-04-27T05:25:27.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39007.589537 secs 2024-04-27T05:25:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39008.495452 secs 2024-04-27T05:25:28.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:27 smithi090 ceph-mon[29180]: pgmap v21297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:28.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39008.495452 secs 2024-04-27T05:25:28.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:27 smithi119 ceph-mon[32752]: pgmap v21297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:28.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39008.495452 secs 2024-04-27T05:25:28.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:27 smithi167 ceph-mon[32751]: pgmap v21297: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:30.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:30 smithi119 ceph-mon[32752]: pgmap v21298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:30.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:30 smithi167 ceph-mon[32751]: pgmap v21298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:30.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:30 smithi090 ceph-mon[29180]: pgmap v21298: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39012.589624 secs 2024-04-27T05:25:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39012.589624 secs 2024-04-27T05:25:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39012.589624 secs 2024-04-27T05:25:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39013.495571 secs 2024-04-27T05:25:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:32 smithi119 ceph-mon[32752]: pgmap v21299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39013.495571 secs 2024-04-27T05:25:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:32 smithi167 ceph-mon[32751]: pgmap v21299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39013.495571 secs 2024-04-27T05:25:32.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:32 smithi090 ceph-mon[29180]: pgmap v21299: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:34.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:34 smithi119 ceph-mon[32752]: pgmap v21300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:34.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:34 smithi167 ceph-mon[32751]: pgmap v21300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:34.823 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:34 smithi090 ceph-mon[29180]: pgmap v21300: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:36 smithi119 ceph-mon[32752]: pgmap v21301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:36.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39017.589705 secs 2024-04-27T05:25:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:36 smithi167 ceph-mon[32751]: pgmap v21301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39017.589705 secs 2024-04-27T05:25:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:36 smithi090 ceph-mon[29180]: pgmap v21301: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:36.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39017.589705 secs 2024-04-27T05:25:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39018.495692 secs 2024-04-27T05:25:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39018.495692 secs 2024-04-27T05:25:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39018.495692 secs 2024-04-27T05:25:38.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:38 smithi119 ceph-mon[32752]: pgmap v21302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:38.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:38 smithi167 ceph-mon[32751]: pgmap v21302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:38.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:38 smithi090 ceph-mon[29180]: pgmap v21302: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:40.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:40 smithi119 ceph-mon[32752]: pgmap v21303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:40.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:40 smithi167 ceph-mon[32751]: pgmap v21303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:40.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:40 smithi090 ceph-mon[29180]: pgmap v21303: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39022.589818 secs 2024-04-27T05:25:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39022.589818 secs 2024-04-27T05:25:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39022.589818 secs 2024-04-27T05:25:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 39023.495780 secs 2024-04-27T05:25:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:42 smithi119 ceph-mon[32752]: slow request 1440.016836 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:25:42.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:42 smithi119 ceph-mon[32752]: pgmap v21304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 39023.495780 secs 2024-04-27T05:25:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:42 smithi167 ceph-mon[32751]: slow request 1440.016836 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:25:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:42 smithi167 ceph-mon[32751]: pgmap v21304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 39023.495780 secs 2024-04-27T05:25:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:42 smithi090 ceph-mon[29180]: slow request 1440.016836 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:25:42.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:42 smithi090 ceph-mon[29180]: pgmap v21304: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:44.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:44 smithi119 ceph-mon[32752]: pgmap v21305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:44.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:44 smithi167 ceph-mon[32751]: pgmap v21305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:44.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:44 smithi090 ceph-mon[29180]: pgmap v21305: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39027.589903 secs 2024-04-27T05:25:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:46 smithi119 ceph-mon[32752]: pgmap v21306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39027.589903 secs 2024-04-27T05:25:46.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:46 smithi167 ceph-mon[32751]: pgmap v21306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39027.589903 secs 2024-04-27T05:25:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:46 smithi090 ceph-mon[29180]: pgmap v21306: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39028.495898 secs 2024-04-27T05:25:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39028.495898 secs 2024-04-27T05:25:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39028.495898 secs 2024-04-27T05:25:48.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:48 smithi119 ceph-mon[32752]: pgmap v21307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:48.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:48 smithi167 ceph-mon[32751]: pgmap v21307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:48.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:48 smithi090 ceph-mon[29180]: pgmap v21307: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:50.695 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:50 smithi090 ceph-mon[29180]: pgmap v21308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:50.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:50 smithi119 ceph-mon[32752]: pgmap v21308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:50.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:50 smithi167 ceph-mon[32751]: pgmap v21308: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:51.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39032.589967 secs 2024-04-27T05:25:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39032.589967 secs 2024-04-27T05:25:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39032.589967 secs 2024-04-27T05:25:52.706 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39033.495972 secs 2024-04-27T05:25:52.707 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:52 smithi119 ceph-mon[32752]: pgmap v21309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39033.495972 secs 2024-04-27T05:25:52.721 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:52 smithi167 ceph-mon[32751]: pgmap v21309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39033.495972 secs 2024-04-27T05:25:52.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:52 smithi090 ceph-mon[29180]: pgmap v21309: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:54.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:54 smithi119 ceph-mon[32752]: pgmap v21310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:54.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:54 smithi167 ceph-mon[32751]: pgmap v21310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:54.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:54 smithi090 ceph-mon[29180]: pgmap v21310: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:25:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39037.590049 secs 2024-04-27T05:25:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:56 smithi119 ceph-mon[32752]: pgmap v21311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39037.590049 secs 2024-04-27T05:25:56.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:56 smithi167 ceph-mon[32751]: pgmap v21311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39037.590049 secs 2024-04-27T05:25:56.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:56 smithi090 ceph-mon[29180]: pgmap v21311: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39038.496102 secs 2024-04-27T05:25:57.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39038.496102 secs 2024-04-27T05:25:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39038.496102 secs 2024-04-27T05:25:58.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:25:58 smithi119 ceph-mon[32752]: pgmap v21312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:58.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:25:58 smithi167 ceph-mon[32751]: pgmap v21312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:25:58.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:25:58 smithi090 ceph-mon[29180]: pgmap v21312: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:00.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:00 smithi119 ceph-mon[32752]: pgmap v21313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:00.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:00 smithi167 ceph-mon[32751]: pgmap v21313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:00.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:00 smithi090 ceph-mon[29180]: pgmap v21313: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:01 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39042.590133 secs 2024-04-27T05:26:01.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:01 smithi119 ceph-mon[32752]: slow request 5764.067295 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:26:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:01 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39042.590133 secs 2024-04-27T05:26:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:01 smithi167 ceph-mon[32751]: slow request 5764.067295 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:26:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:01 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39042.590133 secs 2024-04-27T05:26:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:01 smithi090 ceph-mon[29180]: slow request 5764.067295 seconds old, received at 2024-04-27T03:49:56.276242+0000: client_request(client.24733:267903 getattr AsXsFs #0x100000001fb 2024-04-27T03:49:56.274054+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:26:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39043.496162 secs 2024-04-27T05:26:02.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:02 smithi119 ceph-mon[32752]: pgmap v21314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39043.496162 secs 2024-04-27T05:26:02.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:02 smithi167 ceph-mon[32751]: pgmap v21314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39043.496162 secs 2024-04-27T05:26:02.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:02 smithi090 ceph-mon[29180]: pgmap v21314: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:04.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:04 smithi119 ceph-mon[32752]: pgmap v21315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:04.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:04 smithi167 ceph-mon[32751]: pgmap v21315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:04.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:04 smithi090 ceph-mon[29180]: pgmap v21315: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39047.590217 secs 2024-04-27T05:26:06.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:06 smithi119 ceph-mon[32752]: pgmap v21316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39047.590217 secs 2024-04-27T05:26:06.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:06 smithi167 ceph-mon[32751]: pgmap v21316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39047.590217 secs 2024-04-27T05:26:06.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:06 smithi090 ceph-mon[29180]: pgmap v21316: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:07.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39048.496292 secs 2024-04-27T05:26:07.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:07 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:26:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39048.496292 secs 2024-04-27T05:26:07.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:07 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:26:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39048.496292 secs 2024-04-27T05:26:07.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:07 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:26:08.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:08 smithi119 ceph-mon[32752]: pgmap v21317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:26:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:26:08.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:26:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:08 smithi167 ceph-mon[32751]: pgmap v21317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:26:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:26:08.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:26:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:08 smithi090 ceph-mon[29180]: pgmap v21317: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:26:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:26:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:26:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:09 smithi119 ceph-mon[32752]: pgmap v21318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:26:09.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:09 smithi119 ceph-mon[32752]: pgmap v21319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:26:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:09 smithi167 ceph-mon[32751]: pgmap v21318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:26:09.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:09 smithi167 ceph-mon[32751]: pgmap v21319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:26:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:09 smithi090 ceph-mon[29180]: pgmap v21318: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:26:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:09 smithi090 ceph-mon[29180]: pgmap v21319: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:26:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:11 smithi119 ceph-mon[32752]: pgmap v21320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39052.590302 secs 2024-04-27T05:26:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:11 smithi167 ceph-mon[32751]: pgmap v21320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39052.590302 secs 2024-04-27T05:26:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:11 smithi090 ceph-mon[29180]: pgmap v21320: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39052.590302 secs 2024-04-27T05:26:12.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39053.496375 secs 2024-04-27T05:26:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39053.496375 secs 2024-04-27T05:26:12.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39053.496375 secs 2024-04-27T05:26:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:13 smithi119 ceph-mon[32752]: pgmap v21321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:26:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:13 smithi167 ceph-mon[32751]: pgmap v21321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:26:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:13 smithi090 ceph-mon[29180]: pgmap v21321: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:26:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:15 smithi119 ceph-mon[32752]: pgmap v21322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:15 smithi167 ceph-mon[32751]: pgmap v21322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:15 smithi090 ceph-mon[29180]: pgmap v21322: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:16.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39057.590384 secs 2024-04-27T05:26:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39057.590384 secs 2024-04-27T05:26:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39057.590384 secs 2024-04-27T05:26:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:17 smithi119 ceph-mon[32752]: pgmap v21323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39058.496489 secs 2024-04-27T05:26:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:17 smithi167 ceph-mon[32751]: pgmap v21323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39058.496489 secs 2024-04-27T05:26:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:17 smithi090 ceph-mon[29180]: pgmap v21323: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:26:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39058.496489 secs 2024-04-27T05:26:19.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:19 smithi119 ceph-mon[32752]: pgmap v21324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:26:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:19 smithi167 ceph-mon[32751]: pgmap v21324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:26:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:19 smithi090 ceph-mon[29180]: pgmap v21324: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:26:21.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:21 smithi090 ceph-mon[29180]: pgmap v21325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:21.680 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39062.590470 secs 2024-04-27T05:26:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:21 smithi119 ceph-mon[32752]: pgmap v21325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:21.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39062.590470 secs 2024-04-27T05:26:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:21 smithi167 ceph-mon[32751]: pgmap v21325: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39062.590470 secs 2024-04-27T05:26:22.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39063.496571 secs 2024-04-27T05:26:22.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39063.496571 secs 2024-04-27T05:26:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39063.496571 secs 2024-04-27T05:26:23.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:23 smithi119 ceph-mon[32752]: pgmap v21326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:23.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:23 smithi167 ceph-mon[32751]: pgmap v21326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:23 smithi090 ceph-mon[29180]: pgmap v21326: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:25.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:25 smithi119 ceph-mon[32752]: pgmap v21327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:25.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:25 smithi167 ceph-mon[32751]: pgmap v21327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:25 smithi090 ceph-mon[29180]: pgmap v21327: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39067.590554 secs 2024-04-27T05:26:26.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39067.590554 secs 2024-04-27T05:26:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39067.590554 secs 2024-04-27T05:26:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:27 smithi119 ceph-mon[32752]: pgmap v21328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:27.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39068.496685 secs 2024-04-27T05:26:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:27 smithi167 ceph-mon[32751]: pgmap v21328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:27.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39068.496685 secs 2024-04-27T05:26:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:27 smithi090 ceph-mon[29180]: pgmap v21328: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39068.496685 secs 2024-04-27T05:26:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:29 smithi119 ceph-mon[32752]: pgmap v21329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:29.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:29 smithi167 ceph-mon[32751]: pgmap v21329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:29 smithi090 ceph-mon[29180]: pgmap v21329: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:31 smithi119 ceph-mon[32752]: pgmap v21330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:31.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39072.590640 secs 2024-04-27T05:26:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:31 smithi167 ceph-mon[32751]: pgmap v21330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:31.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39072.590640 secs 2024-04-27T05:26:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:31 smithi090 ceph-mon[29180]: pgmap v21330: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39072.590640 secs 2024-04-27T05:26:32.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39073.496803 secs 2024-04-27T05:26:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39073.496803 secs 2024-04-27T05:26:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39073.496803 secs 2024-04-27T05:26:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:33 smithi119 ceph-mon[32752]: pgmap v21331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:33 smithi167 ceph-mon[32751]: pgmap v21331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:33 smithi090 ceph-mon[29180]: pgmap v21331: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:35 smithi119 ceph-mon[32752]: pgmap v21332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:35 smithi167 ceph-mon[32751]: pgmap v21332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:35 smithi090 ceph-mon[29180]: pgmap v21332: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39077.590723 secs 2024-04-27T05:26:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39077.590723 secs 2024-04-27T05:26:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39077.590723 secs 2024-04-27T05:26:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:37 smithi167 ceph-mon[32751]: pgmap v21333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39078.496922 secs 2024-04-27T05:26:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:37 smithi090 ceph-mon[29180]: pgmap v21333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:37.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39078.496922 secs 2024-04-27T05:26:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:37 smithi119 ceph-mon[32752]: pgmap v21333: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39078.496922 secs 2024-04-27T05:26:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:39 smithi167 ceph-mon[32751]: pgmap v21334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:39 smithi090 ceph-mon[29180]: pgmap v21334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:39 smithi119 ceph-mon[32752]: pgmap v21334: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:41 smithi167 ceph-mon[32751]: pgmap v21335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:41.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39082.590829 secs 2024-04-27T05:26:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:41 smithi090 ceph-mon[29180]: pgmap v21335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39082.590829 secs 2024-04-27T05:26:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:41 smithi119 ceph-mon[32752]: pgmap v21335: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39082.590829 secs 2024-04-27T05:26:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39083.497038 secs 2024-04-27T05:26:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39083.497038 secs 2024-04-27T05:26:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39083.497038 secs 2024-04-27T05:26:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:43 smithi090 ceph-mon[29180]: pgmap v21336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:43 smithi119 ceph-mon[32752]: pgmap v21336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:43 smithi167 ceph-mon[32751]: pgmap v21336: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:45 smithi090 ceph-mon[29180]: pgmap v21337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:45 smithi119 ceph-mon[32752]: pgmap v21337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:45 smithi167 ceph-mon[32751]: pgmap v21337: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:46.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39087.590909 secs 2024-04-27T05:26:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39087.590909 secs 2024-04-27T05:26:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39087.590909 secs 2024-04-27T05:26:47.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:47 smithi090 ceph-mon[29180]: pgmap v21338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:47.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39088.497135 secs 2024-04-27T05:26:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:47 smithi119 ceph-mon[32752]: pgmap v21338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39088.497135 secs 2024-04-27T05:26:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:47 smithi167 ceph-mon[32751]: pgmap v21338: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:48.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39088.497135 secs 2024-04-27T05:26:49.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:49 smithi090 ceph-mon[29180]: pgmap v21339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:49 smithi119 ceph-mon[32752]: pgmap v21339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:49 smithi167 ceph-mon[32751]: pgmap v21339: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:51.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:51 smithi090 ceph-mon[29180]: pgmap v21340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:51.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39092.591000 secs 2024-04-27T05:26:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:51 smithi119 ceph-mon[32752]: pgmap v21340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39092.591000 secs 2024-04-27T05:26:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:51 smithi167 ceph-mon[32751]: pgmap v21340: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39092.591000 secs 2024-04-27T05:26:52.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39093.497276 secs 2024-04-27T05:26:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39093.497276 secs 2024-04-27T05:26:52.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39093.497276 secs 2024-04-27T05:26:53.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:53 smithi090 ceph-mon[29180]: pgmap v21341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:53 smithi119 ceph-mon[32752]: pgmap v21341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:53 smithi167 ceph-mon[32751]: pgmap v21341: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:55 smithi090 ceph-mon[29180]: pgmap v21342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:55 smithi119 ceph-mon[32752]: pgmap v21342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:55 smithi167 ceph-mon[32751]: pgmap v21342: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:26:56.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39097.591083 secs 2024-04-27T05:26:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39097.591083 secs 2024-04-27T05:26:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39097.591083 secs 2024-04-27T05:26:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:57 smithi090 ceph-mon[29180]: pgmap v21343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:57.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39098.497418 secs 2024-04-27T05:26:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:57 smithi119 ceph-mon[32752]: pgmap v21343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:58.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39098.497418 secs 2024-04-27T05:26:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:57 smithi167 ceph-mon[32751]: pgmap v21343: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:26:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39098.497418 secs 2024-04-27T05:26:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:26:59 smithi090 ceph-mon[29180]: pgmap v21344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:26:59 smithi119 ceph-mon[32752]: pgmap v21344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:26:59 smithi167 ceph-mon[32751]: pgmap v21344: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:01 smithi090 ceph-mon[29180]: pgmap v21345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:01.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39102.591170 secs 2024-04-27T05:27:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:01 smithi119 ceph-mon[32752]: pgmap v21345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39102.591170 secs 2024-04-27T05:27:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:01 smithi167 ceph-mon[32751]: pgmap v21345: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39102.591170 secs 2024-04-27T05:27:02.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39103.497562 secs 2024-04-27T05:27:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39103.497562 secs 2024-04-27T05:27:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39103.497562 secs 2024-04-27T05:27:03.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:03 smithi090 ceph-mon[29180]: pgmap v21346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:03 smithi119 ceph-mon[32752]: pgmap v21346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:03 smithi167 ceph-mon[32751]: pgmap v21346: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:05.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:05 smithi090 ceph-mon[29180]: pgmap v21347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:05 smithi119 ceph-mon[32752]: pgmap v21347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:05 smithi167 ceph-mon[32751]: pgmap v21347: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:06 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39107.591253 secs 2024-04-27T05:27:07.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:06 smithi119 ceph-mon[32752]: slow request 23040.219086 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:27:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:06 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39107.591253 secs 2024-04-27T05:27:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:06 smithi167 ceph-mon[32751]: slow request 23040.219086 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:27:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:06 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39107.591253 secs 2024-04-27T05:27:07.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:06 smithi090 ceph-mon[29180]: slow request 23040.219086 seconds old, received at 2024-04-26T23:03:05.125571+0000: client_request(client.17409:1 getattr pAsLsXsFs #0x1 2024-04-26T23:03:05.117477+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:27:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:07 smithi119 ceph-mon[32752]: pgmap v21348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39108.497700 secs 2024-04-27T05:27:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:07 smithi167 ceph-mon[32751]: pgmap v21348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39108.497700 secs 2024-04-27T05:27:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:07 smithi090 ceph-mon[29180]: pgmap v21348: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39108.497700 secs 2024-04-27T05:27:08.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:08 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:27:09.040 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:08 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:27:09.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:08 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:27:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:09 smithi119 ceph-mon[32752]: pgmap v21349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:09.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:09 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:09 smithi090 ceph-mon[29180]: pgmap v21349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:09.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:09.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:09 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:09 smithi167 ceph-mon[32751]: pgmap v21349: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:10.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:09 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: pgmap v21350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39112.591346 secs 2024-04-27T05:27:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:11 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: pgmap v21350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:11.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39112.591346 secs 2024-04-27T05:27:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:11 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: pgmap v21350: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39112.591346 secs 2024-04-27T05:27:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:11.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:11 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39113.497761 secs 2024-04-27T05:27:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.807 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.808 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:27:12.809 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39113.497761 secs 2024-04-27T05:27:12.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:27:12.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:27:12.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:27:12.822 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39113.497761 secs 2024-04-27T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:27:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:27:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.879 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.880 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:27:12.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:27:12.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:27:12.881 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:27:13.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:13 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:27:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:13 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:27:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:13 smithi119 ceph-mon[32752]: pgmap v21351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:13 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:27:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:13 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:27:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:13 smithi119 ceph-mon[32752]: pgmap v21352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-27T05:27:13.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:13 smithi119 ceph-mon[32752]: pgmap v21353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T05:27:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:13 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:27:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:13 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:27:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:13 smithi167 ceph-mon[32751]: pgmap v21351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:13.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:13 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:27:13.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:13 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:27:13.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:13 smithi167 ceph-mon[32751]: pgmap v21352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-27T05:27:13.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:13 smithi167 ceph-mon[32751]: pgmap v21353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T05:27:13.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:13 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:27:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:13 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:27:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:13 smithi090 ceph-mon[29180]: pgmap v21351: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:13 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:27:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:13 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:27:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:13 smithi090 ceph-mon[29180]: pgmap v21352: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 1 op/s 2024-04-27T05:27:13.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:13 smithi090 ceph-mon[29180]: pgmap v21353: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T05:27:15.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:15 smithi119 ceph-mon[32752]: pgmap v21354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:15.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:15 smithi167 ceph-mon[32751]: pgmap v21354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:15 smithi090 ceph-mon[29180]: pgmap v21354: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:16.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39117.591429 secs 2024-04-27T05:27:16.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39117.591429 secs 2024-04-27T05:27:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39117.591429 secs 2024-04-27T05:27:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:17 smithi167 ceph-mon[32751]: pgmap v21355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:17.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39118.497880 secs 2024-04-27T05:27:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:17 smithi090 ceph-mon[29180]: pgmap v21355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:17.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39118.497880 secs 2024-04-27T05:27:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:17 smithi119 ceph-mon[32752]: pgmap v21355: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39118.497880 secs 2024-04-27T05:27:19.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:19 smithi167 ceph-mon[32751]: pgmap v21356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T05:27:19.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:19 smithi090 ceph-mon[29180]: pgmap v21356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T05:27:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:19 smithi119 ceph-mon[32752]: pgmap v21356: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 635 B/s rd, 1 op/s 2024-04-27T05:27:21.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:21 smithi167 ceph-mon[32751]: pgmap v21357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:21.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39122.591514 secs 2024-04-27T05:27:21.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:21 smithi090 ceph-mon[29180]: pgmap v21357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:21.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39122.591514 secs 2024-04-27T05:27:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:21 smithi119 ceph-mon[32752]: pgmap v21357: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39122.591514 secs 2024-04-27T05:27:22.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39123.498018 secs 2024-04-27T05:27:22.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39123.498018 secs 2024-04-27T05:27:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39123.498018 secs 2024-04-27T05:27:23.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:23 smithi090 ceph-mon[29180]: pgmap v21358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:27:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:23 smithi119 ceph-mon[32752]: pgmap v21358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:27:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:23 smithi167 ceph-mon[32751]: pgmap v21358: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:27:25.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:25 smithi090 ceph-mon[29180]: pgmap v21359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:25 smithi119 ceph-mon[32752]: pgmap v21359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:25 smithi167 ceph-mon[32751]: pgmap v21359: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:26.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39127.591593 secs 2024-04-27T05:27:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39127.591593 secs 2024-04-27T05:27:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39127.591593 secs 2024-04-27T05:27:27.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:27 smithi090 ceph-mon[29180]: pgmap v21360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:27.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39128.498098 secs 2024-04-27T05:27:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:27 smithi119 ceph-mon[32752]: pgmap v21360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39128.498098 secs 2024-04-27T05:27:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:27 smithi167 ceph-mon[32751]: pgmap v21360: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39128.498098 secs 2024-04-27T05:27:29.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:29 smithi090 ceph-mon[29180]: pgmap v21361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:29 smithi119 ceph-mon[32752]: pgmap v21361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:29 smithi167 ceph-mon[32751]: pgmap v21361: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:31.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:31 smithi090 ceph-mon[29180]: pgmap v21362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:31.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39132.591650 secs 2024-04-27T05:27:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:31 smithi119 ceph-mon[32752]: pgmap v21362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39132.591650 secs 2024-04-27T05:27:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:31 smithi167 ceph-mon[32751]: pgmap v21362: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39132.591650 secs 2024-04-27T05:27:32.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39133.498215 secs 2024-04-27T05:27:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39133.498215 secs 2024-04-27T05:27:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39133.498215 secs 2024-04-27T05:27:33.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:33 smithi090 ceph-mon[29180]: pgmap v21363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:33 smithi119 ceph-mon[32752]: pgmap v21363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:33 smithi167 ceph-mon[32751]: pgmap v21363: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:35.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:35 smithi090 ceph-mon[29180]: pgmap v21364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:35 smithi119 ceph-mon[32752]: pgmap v21364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:35 smithi167 ceph-mon[32751]: pgmap v21364: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:36.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39137.591705 secs 2024-04-27T05:27:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39137.591705 secs 2024-04-27T05:27:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39137.591705 secs 2024-04-27T05:27:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:37 smithi090 ceph-mon[29180]: pgmap v21365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:37.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39138.498328 secs 2024-04-27T05:27:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:37 smithi119 ceph-mon[32752]: pgmap v21365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:38.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39138.498328 secs 2024-04-27T05:27:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:37 smithi167 ceph-mon[32751]: pgmap v21365: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39138.498328 secs 2024-04-27T05:27:39.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:39 smithi090 ceph-mon[29180]: pgmap v21366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:39 smithi119 ceph-mon[32752]: pgmap v21366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:39 smithi167 ceph-mon[32751]: pgmap v21366: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:41.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:41 smithi090 ceph-mon[29180]: pgmap v21367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:41.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39142.591767 secs 2024-04-27T05:27:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:41 smithi119 ceph-mon[32752]: pgmap v21367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39142.591767 secs 2024-04-27T05:27:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:41 smithi167 ceph-mon[32751]: pgmap v21367: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39142.591767 secs 2024-04-27T05:27:42.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39143.498417 secs 2024-04-27T05:27:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39143.498417 secs 2024-04-27T05:27:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39143.498417 secs 2024-04-27T05:27:43.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:43 smithi090 ceph-mon[29180]: pgmap v21368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:43 smithi119 ceph-mon[32752]: pgmap v21368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:43 smithi167 ceph-mon[32751]: pgmap v21368: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:45.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:45 smithi090 ceph-mon[29180]: pgmap v21369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:45 smithi119 ceph-mon[32752]: pgmap v21369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:45 smithi167 ceph-mon[32751]: pgmap v21369: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39147.591821 secs 2024-04-27T05:27:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39147.591821 secs 2024-04-27T05:27:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39147.591821 secs 2024-04-27T05:27:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:47 smithi119 ceph-mon[32752]: pgmap v21370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39148.498528 secs 2024-04-27T05:27:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:47 smithi167 ceph-mon[32751]: pgmap v21370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39148.498528 secs 2024-04-27T05:27:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:47 smithi090 ceph-mon[29180]: pgmap v21370: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:48.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39148.498528 secs 2024-04-27T05:27:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:49 smithi119 ceph-mon[32752]: pgmap v21371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:49 smithi167 ceph-mon[32751]: pgmap v21371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:49 smithi090 ceph-mon[29180]: pgmap v21371: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:51.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:51 smithi090 ceph-mon[29180]: pgmap v21372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:51.930 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39152.591883 secs 2024-04-27T05:27:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:51 smithi119 ceph-mon[32752]: pgmap v21372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39152.591883 secs 2024-04-27T05:27:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:51 smithi167 ceph-mon[32751]: pgmap v21372: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39152.591883 secs 2024-04-27T05:27:52.957 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39153.498645 secs 2024-04-27T05:27:52.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39153.498645 secs 2024-04-27T05:27:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39153.498645 secs 2024-04-27T05:27:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:53 smithi119 ceph-mon[32752]: pgmap v21373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:53 smithi167 ceph-mon[32751]: pgmap v21373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:53 smithi090 ceph-mon[29180]: pgmap v21373: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:55 smithi090 ceph-mon[29180]: pgmap v21374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:55 smithi119 ceph-mon[32752]: pgmap v21374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:55 smithi167 ceph-mon[32751]: pgmap v21374: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:27:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39157.591939 secs 2024-04-27T05:27:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39157.591939 secs 2024-04-27T05:27:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39157.591939 secs 2024-04-27T05:27:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:57 smithi119 ceph-mon[32752]: pgmap v21375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39158.498780 secs 2024-04-27T05:27:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:57 smithi167 ceph-mon[32751]: pgmap v21375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39158.498780 secs 2024-04-27T05:27:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:57 smithi090 ceph-mon[29180]: pgmap v21375: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:27:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39158.498780 secs 2024-04-27T05:28:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:27:59 smithi119 ceph-mon[32752]: pgmap v21376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:27:59 smithi167 ceph-mon[32751]: pgmap v21376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:00.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:27:59 smithi090 ceph-mon[29180]: pgmap v21376: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:01 smithi119 ceph-mon[32752]: pgmap v21377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:01 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39162.592001 secs 2024-04-27T05:28:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:01 smithi119 ceph-mon[32752]: slow request 360.240299 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:28:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:01 smithi167 ceph-mon[32751]: pgmap v21377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:01 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39162.592001 secs 2024-04-27T05:28:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:01 smithi167 ceph-mon[32751]: slow request 360.240299 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:28:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:01 smithi090 ceph-mon[29180]: pgmap v21377: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:01 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39162.592001 secs 2024-04-27T05:28:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:01 smithi090 ceph-mon[29180]: slow request 360.240299 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:28:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39163.498899 secs 2024-04-27T05:28:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39163.498899 secs 2024-04-27T05:28:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39163.498899 secs 2024-04-27T05:28:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:03 smithi119 ceph-mon[32752]: pgmap v21378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:03 smithi167 ceph-mon[32751]: pgmap v21378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:04.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:03 smithi090 ceph-mon[29180]: pgmap v21378: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:05 smithi119 ceph-mon[32752]: pgmap v21379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:05 smithi167 ceph-mon[32751]: pgmap v21379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:05 smithi090 ceph-mon[29180]: pgmap v21379: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39167.592054 secs 2024-04-27T05:28:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39167.592054 secs 2024-04-27T05:28:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39167.592054 secs 2024-04-27T05:28:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:07 smithi119 ceph-mon[32752]: pgmap v21380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39168.499018 secs 2024-04-27T05:28:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:07 smithi167 ceph-mon[32751]: pgmap v21380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39168.499018 secs 2024-04-27T05:28:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:07 smithi090 ceph-mon[29180]: pgmap v21380: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39168.499018 secs 2024-04-27T05:28:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:09 smithi119 ceph-mon[32752]: pgmap v21381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:09 smithi167 ceph-mon[32751]: pgmap v21381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:09 smithi090 ceph-mon[29180]: pgmap v21381: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:11 smithi119 ceph-mon[32752]: pgmap v21382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39172.592112 secs 2024-04-27T05:28:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:11 smithi167 ceph-mon[32751]: pgmap v21382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39172.592112 secs 2024-04-27T05:28:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:11 smithi090 ceph-mon[29180]: pgmap v21382: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:12.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39172.592112 secs 2024-04-27T05:28:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39173.499135 secs 2024-04-27T05:28:13.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:12 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:28:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39173.499135 secs 2024-04-27T05:28:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:12 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:28:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39173.499135 secs 2024-04-27T05:28:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:12 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:28:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:13 smithi119 ceph-mon[32752]: pgmap v21383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:28:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:28:14.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:13 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:13 smithi167 ceph-mon[32751]: pgmap v21383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:28:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:28:14.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:13 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:13 smithi090 ceph-mon[29180]: pgmap v21383: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:14.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.189 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:14.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:28:14.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:28:14.190 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:13 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:28:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:14 smithi119 ceph-mon[32752]: pgmap v21384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:28:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:14 smithi119 ceph-mon[32752]: pgmap v21385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:14 smithi167 ceph-mon[32751]: pgmap v21384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:28:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:14 smithi167 ceph-mon[32751]: pgmap v21385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:15.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:14 smithi090 ceph-mon[29180]: pgmap v21384: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:28:15.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:14 smithi090 ceph-mon[29180]: pgmap v21385: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39177.592171 secs 2024-04-27T05:28:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:16 smithi119 ceph-mon[32752]: pgmap v21386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39177.592171 secs 2024-04-27T05:28:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:16 smithi167 ceph-mon[32751]: pgmap v21386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39177.592171 secs 2024-04-27T05:28:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:16 smithi090 ceph-mon[29180]: pgmap v21386: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39178.499270 secs 2024-04-27T05:28:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39178.499270 secs 2024-04-27T05:28:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39178.499270 secs 2024-04-27T05:28:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:18 smithi119 ceph-mon[32752]: pgmap v21387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-27T05:28:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:18 smithi167 ceph-mon[32751]: pgmap v21387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-27T05:28:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:18 smithi090 ceph-mon[29180]: pgmap v21387: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 549 B/s rd, 1 op/s 2024-04-27T05:28:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:20 smithi119 ceph-mon[32752]: pgmap v21388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:20 smithi167 ceph-mon[32751]: pgmap v21388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:20 smithi090 ceph-mon[29180]: pgmap v21388: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:22.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39182.592255 secs 2024-04-27T05:28:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39182.592255 secs 2024-04-27T05:28:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39182.592255 secs 2024-04-27T05:28:22.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39183.499392 secs 2024-04-27T05:28:22.971 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:22 smithi167 ceph-mon[32751]: pgmap v21389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39183.499392 secs 2024-04-27T05:28:23.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:22 smithi119 ceph-mon[32752]: pgmap v21389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39183.499392 secs 2024-04-27T05:28:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:22 smithi090 ceph-mon[29180]: pgmap v21389: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:28:25.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:24 smithi119 ceph-mon[32752]: pgmap v21390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:28:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:24 smithi167 ceph-mon[32751]: pgmap v21390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:28:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:24 smithi090 ceph-mon[29180]: pgmap v21390: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:28:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39187.592337 secs 2024-04-27T05:28:27.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:26 smithi119 ceph-mon[32752]: pgmap v21391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39187.592337 secs 2024-04-27T05:28:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:26 smithi167 ceph-mon[32751]: pgmap v21391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39187.592337 secs 2024-04-27T05:28:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:26 smithi090 ceph-mon[29180]: pgmap v21391: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39188.499498 secs 2024-04-27T05:28:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39188.499498 secs 2024-04-27T05:28:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39188.499498 secs 2024-04-27T05:28:29.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:28 smithi119 ceph-mon[32752]: pgmap v21392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:29.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:28 smithi167 ceph-mon[32751]: pgmap v21392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:28 smithi090 ceph-mon[29180]: pgmap v21392: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:31.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:30 smithi119 ceph-mon[32752]: pgmap v21393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:31.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:30 smithi167 ceph-mon[32751]: pgmap v21393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:30 smithi090 ceph-mon[29180]: pgmap v21393: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39192.592423 secs 2024-04-27T05:28:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39192.592423 secs 2024-04-27T05:28:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39192.592423 secs 2024-04-27T05:28:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39193.499570 secs 2024-04-27T05:28:33.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:32 smithi119 ceph-mon[32752]: pgmap v21394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39193.499570 secs 2024-04-27T05:28:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:32 smithi167 ceph-mon[32751]: pgmap v21394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39193.499570 secs 2024-04-27T05:28:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:32 smithi090 ceph-mon[29180]: pgmap v21394: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:35.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:34 smithi119 ceph-mon[32752]: pgmap v21395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:35.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:34 smithi167 ceph-mon[32751]: pgmap v21395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:35.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:34 smithi090 ceph-mon[29180]: pgmap v21395: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39197.592505 secs 2024-04-27T05:28:37.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:36 smithi119 ceph-mon[32752]: pgmap v21396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39197.592505 secs 2024-04-27T05:28:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:36 smithi167 ceph-mon[32751]: pgmap v21396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39197.592505 secs 2024-04-27T05:28:37.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:36 smithi090 ceph-mon[29180]: pgmap v21396: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39198.499659 secs 2024-04-27T05:28:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39198.499659 secs 2024-04-27T05:28:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39198.499659 secs 2024-04-27T05:28:39.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:38 smithi119 ceph-mon[32752]: pgmap v21397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:39.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:38 smithi167 ceph-mon[32751]: pgmap v21397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:39.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:38 smithi090 ceph-mon[29180]: pgmap v21397: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:41.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:40 smithi119 ceph-mon[32752]: pgmap v21398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:41.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:40 smithi167 ceph-mon[32751]: pgmap v21398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:41.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:40 smithi090 ceph-mon[29180]: pgmap v21398: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:28:42.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39202.592588 secs 2024-04-27T05:28:42.030 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:28:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:28:41.698+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:28:42.030 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:28:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:28:41.698+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:28:42.030 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:28:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:28:41.698+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:28:42.030 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:28:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:28:41.698+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:28:42.031 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:28:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:28:41.698+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:28:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39202.592588 secs 2024-04-27T05:28:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39202.592588 secs 2024-04-27T05:28:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39203.499796 secs 2024-04-27T05:28:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:42 smithi119 ceph-mon[32752]: pgmap v21399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39203.499796 secs 2024-04-27T05:28:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:42 smithi167 ceph-mon[32751]: pgmap v21399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39203.499796 secs 2024-04-27T05:28:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:42 smithi090 ceph-mon[29180]: pgmap v21399: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:45.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:44 smithi119 ceph-mon[32752]: pgmap v21400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:45.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:44 smithi167 ceph-mon[32751]: pgmap v21400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:45.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:44 smithi090 ceph-mon[29180]: pgmap v21400: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39207.592669 secs 2024-04-27T05:28:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:46 smithi119 ceph-mon[32752]: pgmap v21401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39207.592669 secs 2024-04-27T05:28:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:46 smithi167 ceph-mon[32751]: pgmap v21401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39207.592669 secs 2024-04-27T05:28:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:46 smithi090 ceph-mon[29180]: pgmap v21401: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39208.499874 secs 2024-04-27T05:28:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39208.499874 secs 2024-04-27T05:28:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39208.499874 secs 2024-04-27T05:28:49.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:48 smithi119 ceph-mon[32752]: pgmap v21402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:49.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:48 smithi167 ceph-mon[32751]: pgmap v21402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:49.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:48 smithi090 ceph-mon[29180]: pgmap v21402: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:51.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:50 smithi119 ceph-mon[32752]: pgmap v21403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:51.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:50 smithi167 ceph-mon[32751]: pgmap v21403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:51.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:50 smithi090 ceph-mon[29180]: pgmap v21403: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39212.592754 secs 2024-04-27T05:28:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39212.592754 secs 2024-04-27T05:28:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39212.592754 secs 2024-04-27T05:28:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39213.499990 secs 2024-04-27T05:28:53.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:52 smithi119 ceph-mon[32752]: pgmap v21404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39213.499990 secs 2024-04-27T05:28:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:52 smithi167 ceph-mon[32751]: pgmap v21404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:53.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39213.499990 secs 2024-04-27T05:28:53.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:52 smithi090 ceph-mon[29180]: pgmap v21404: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:28:55.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:54 smithi119 ceph-mon[32752]: pgmap v21405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:55.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:54 smithi167 ceph-mon[32751]: pgmap v21405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:55.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:54 smithi090 ceph-mon[29180]: pgmap v21405: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:28:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39217.592836 secs 2024-04-27T05:28:57.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:56 smithi167 ceph-mon[32751]: pgmap v21406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39217.592836 secs 2024-04-27T05:28:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:56 smithi090 ceph-mon[29180]: pgmap v21406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:57.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39217.592836 secs 2024-04-27T05:28:57.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:56 smithi119 ceph-mon[32752]: pgmap v21406: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39218.500087 secs 2024-04-27T05:28:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39218.500087 secs 2024-04-27T05:28:58.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39218.500087 secs 2024-04-27T05:28:59.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:28:58 smithi167 ceph-mon[32751]: pgmap v21407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:59.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:28:58 smithi090 ceph-mon[29180]: pgmap v21407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:28:59.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:28:58 smithi119 ceph-mon[32752]: pgmap v21407: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:00 smithi167 ceph-mon[32751]: pgmap v21408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:00 smithi090 ceph-mon[29180]: pgmap v21408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:01.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:00 smithi119 ceph-mon[32752]: pgmap v21408: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39222.592924 secs 2024-04-27T05:29:02.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39222.592924 secs 2024-04-27T05:29:02.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39222.592924 secs 2024-04-27T05:29:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39223.500217 secs 2024-04-27T05:29:03.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:02 smithi090 ceph-mon[29180]: pgmap v21409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:03.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39223.500217 secs 2024-04-27T05:29:03.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:02 smithi119 ceph-mon[32752]: pgmap v21409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:03.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39223.500217 secs 2024-04-27T05:29:03.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:02 smithi167 ceph-mon[32751]: pgmap v21409: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:05.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:04 smithi090 ceph-mon[29180]: pgmap v21410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:05.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:04 smithi119 ceph-mon[32752]: pgmap v21410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:05.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:04 smithi167 ceph-mon[32751]: pgmap v21410: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:06.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:05 smithi119 ceph-mon[32752]: pgmap v21411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:06.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:05 smithi167 ceph-mon[32751]: pgmap v21411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:06.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:05 smithi090 ceph-mon[29180]: pgmap v21411: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:07.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39227.592983 secs 2024-04-27T05:29:07.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39227.592983 secs 2024-04-27T05:29:07.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39227.592983 secs 2024-04-27T05:29:08.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39228.500334 secs 2024-04-27T05:29:08.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:07 smithi119 ceph-mon[32752]: pgmap v21412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:08.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39228.500334 secs 2024-04-27T05:29:08.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:07 smithi167 ceph-mon[32751]: pgmap v21412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:08.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39228.500334 secs 2024-04-27T05:29:08.378 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:07 smithi090 ceph-mon[29180]: pgmap v21412: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:10.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:10 smithi090 ceph-mon[29180]: pgmap v21413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:11.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:10 smithi119 ceph-mon[32752]: pgmap v21413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:11.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:10 smithi167 ceph-mon[32751]: pgmap v21413: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:11.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39232.593044 secs 2024-04-27T05:29:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39232.593044 secs 2024-04-27T05:29:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39232.593044 secs 2024-04-27T05:29:12.778 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39233.500450 secs 2024-04-27T05:29:12.779 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:12 smithi090 ceph-mon[29180]: pgmap v21414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39233.500450 secs 2024-04-27T05:29:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:12 smithi119 ceph-mon[32752]: pgmap v21414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39233.500450 secs 2024-04-27T05:29:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:12 smithi167 ceph-mon[32751]: pgmap v21414: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:29:14.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:14 smithi090 ceph-mon[29180]: pgmap v21415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:29:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:29:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:29:14.878 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:29:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:14 smithi119 ceph-mon[32752]: pgmap v21415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:29:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:29:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:29:15.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:29:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:14 smithi167 ceph-mon[32751]: pgmap v21415: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:29:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:29:15.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:29:15.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:29:15.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:29:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:15 smithi090 ceph-mon[29180]: pgmap v21416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:29:15.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:15 smithi090 ceph-mon[29180]: pgmap v21417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:15 smithi119 ceph-mon[32752]: pgmap v21416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:29:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:15 smithi119 ceph-mon[32752]: pgmap v21417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:15 smithi167 ceph-mon[32751]: pgmap v21416: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:29:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:15 smithi167 ceph-mon[32751]: pgmap v21417: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39237.593106 secs 2024-04-27T05:29:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39237.593106 secs 2024-04-27T05:29:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39237.593106 secs 2024-04-27T05:29:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39238.500510 secs 2024-04-27T05:29:18.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:17 smithi119 ceph-mon[32752]: pgmap v21418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39238.500510 secs 2024-04-27T05:29:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:17 smithi167 ceph-mon[32751]: pgmap v21418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39238.500510 secs 2024-04-27T05:29:18.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:17 smithi090 ceph-mon[29180]: pgmap v21418: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:20.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:19 smithi119 ceph-mon[32752]: pgmap v21419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:20.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:19 smithi167 ceph-mon[32751]: pgmap v21419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:20.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:19 smithi090 ceph-mon[29180]: pgmap v21419: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:22.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39242.593189 secs 2024-04-27T05:29:22.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:21 smithi119 ceph-mon[32752]: pgmap v21420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39242.593189 secs 2024-04-27T05:29:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:21 smithi167 ceph-mon[32751]: pgmap v21420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39242.593189 secs 2024-04-27T05:29:22.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:21 smithi090 ceph-mon[29180]: pgmap v21420: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:29:22.956 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39243.500598 secs 2024-04-27T05:29:22.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39243.500598 secs 2024-04-27T05:29:23.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39243.500598 secs 2024-04-27T05:29:24.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:23 smithi119 ceph-mon[32752]: pgmap v21421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:29:24.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:23 smithi167 ceph-mon[32751]: pgmap v21421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:29:24.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:23 smithi090 ceph-mon[29180]: pgmap v21421: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:29:26.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:25 smithi119 ceph-mon[32752]: pgmap v21422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:29:26.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:25 smithi167 ceph-mon[32751]: pgmap v21422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:29:26.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:25 smithi090 ceph-mon[29180]: pgmap v21422: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:29:27.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39247.593269 secs 2024-04-27T05:29:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39247.593269 secs 2024-04-27T05:29:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39247.593269 secs 2024-04-27T05:29:28.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39248.500713 secs 2024-04-27T05:29:28.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:27 smithi119 ceph-mon[32752]: pgmap v21423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39248.500713 secs 2024-04-27T05:29:28.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:27 smithi167 ceph-mon[32751]: pgmap v21423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39248.500713 secs 2024-04-27T05:29:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:27 smithi090 ceph-mon[29180]: pgmap v21423: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:30.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:29 smithi119 ceph-mon[32752]: pgmap v21424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:30.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:29 smithi167 ceph-mon[32751]: pgmap v21424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:30.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:29 smithi090 ceph-mon[29180]: pgmap v21424: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:32.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39252.593353 secs 2024-04-27T05:29:32.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:31 smithi119 ceph-mon[32752]: pgmap v21425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39252.593353 secs 2024-04-27T05:29:32.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:31 smithi167 ceph-mon[32751]: pgmap v21425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39252.593353 secs 2024-04-27T05:29:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:31 smithi090 ceph-mon[29180]: pgmap v21425: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:33.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39253.500779 secs 2024-04-27T05:29:33.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39253.500779 secs 2024-04-27T05:29:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39253.500779 secs 2024-04-27T05:29:34.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:33 smithi119 ceph-mon[32752]: pgmap v21426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:34.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:33 smithi167 ceph-mon[32751]: pgmap v21426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:34.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:33 smithi090 ceph-mon[29180]: pgmap v21426: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:36.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:35 smithi119 ceph-mon[32752]: pgmap v21427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:36.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:35 smithi167 ceph-mon[32751]: pgmap v21427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:36.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:35 smithi090 ceph-mon[29180]: pgmap v21427: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:37.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39257.593433 secs 2024-04-27T05:29:37.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39257.593433 secs 2024-04-27T05:29:37.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39257.593433 secs 2024-04-27T05:29:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39258.500897 secs 2024-04-27T05:29:38.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:37 smithi119 ceph-mon[32752]: pgmap v21428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39258.500897 secs 2024-04-27T05:29:38.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:37 smithi167 ceph-mon[32751]: pgmap v21428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39258.500897 secs 2024-04-27T05:29:38.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:37 smithi090 ceph-mon[29180]: pgmap v21428: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:40.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:39 smithi119 ceph-mon[32752]: pgmap v21429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:40.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:39 smithi167 ceph-mon[32751]: pgmap v21429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:40.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:39 smithi090 ceph-mon[29180]: pgmap v21429: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:42.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39262.593515 secs 2024-04-27T05:29:42.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:41 smithi119 ceph-mon[32752]: pgmap v21430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39262.593515 secs 2024-04-27T05:29:42.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:41 smithi167 ceph-mon[32751]: pgmap v21430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39262.593515 secs 2024-04-27T05:29:42.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:41 smithi090 ceph-mon[29180]: pgmap v21430: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:43.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39263.501014 secs 2024-04-27T05:29:43.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39263.501014 secs 2024-04-27T05:29:43.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39263.501014 secs 2024-04-27T05:29:44.028 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:43 smithi090 ceph-mon[29180]: pgmap v21431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:44.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:43 smithi119 ceph-mon[32752]: pgmap v21431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:44.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:43 smithi167 ceph-mon[32751]: pgmap v21431: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:46.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:45 smithi119 ceph-mon[32752]: pgmap v21432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:46.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:45 smithi167 ceph-mon[32751]: pgmap v21432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:46.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:45 smithi090 ceph-mon[29180]: pgmap v21432: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:47.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:46 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39267.593597 secs 2024-04-27T05:29:47.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:46 smithi119 ceph-mon[32752]: slow request 2884.499432 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:29:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:46 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39267.593597 secs 2024-04-27T05:29:47.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:46 smithi167 ceph-mon[32751]: slow request 2884.499432 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:29:47.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:46 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39267.593597 secs 2024-04-27T05:29:47.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:46 smithi090 ceph-mon[29180]: slow request 2884.499432 seconds old, received at 2024-04-27T04:41:40.847569+0000: client_request(client.17796:1 getattr pAsLsXsFs #0x1 2024-04-27T04:41:40.843202+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:29:48.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39268.501131 secs 2024-04-27T05:29:48.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:47 smithi119 ceph-mon[32752]: pgmap v21433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39268.501131 secs 2024-04-27T05:29:48.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:47 smithi167 ceph-mon[32751]: pgmap v21433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39268.501131 secs 2024-04-27T05:29:48.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:47 smithi090 ceph-mon[29180]: pgmap v21433: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:50.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:49 smithi119 ceph-mon[32752]: pgmap v21434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:50.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:49 smithi167 ceph-mon[32751]: pgmap v21434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:50.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:49 smithi090 ceph-mon[29180]: pgmap v21434: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:52.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39272.593681 secs 2024-04-27T05:29:52.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:51 smithi119 ceph-mon[32752]: pgmap v21435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39272.593681 secs 2024-04-27T05:29:52.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:51 smithi167 ceph-mon[32751]: pgmap v21435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:52.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39272.593681 secs 2024-04-27T05:29:52.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:51 smithi090 ceph-mon[29180]: pgmap v21435: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:53.029 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39273.501244 secs 2024-04-27T05:29:53.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39273.501244 secs 2024-04-27T05:29:53.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39273.501244 secs 2024-04-27T05:29:54.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:53 smithi119 ceph-mon[32752]: pgmap v21436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:54.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:53 smithi167 ceph-mon[32751]: pgmap v21436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:54.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:53 smithi090 ceph-mon[29180]: pgmap v21436: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:56.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:55 smithi119 ceph-mon[32752]: pgmap v21437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:56.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:55 smithi167 ceph-mon[32751]: pgmap v21437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:56.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:55 smithi090 ceph-mon[29180]: pgmap v21437: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:29:57.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:56 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39277.593762 secs 2024-04-27T05:29:57.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:56 smithi119 ceph-mon[32752]: slow request 11523.364455 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:29:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:56 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39277.593762 secs 2024-04-27T05:29:57.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:56 smithi167 ceph-mon[32751]: slow request 11523.364455 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:29:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:56 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39277.593762 secs 2024-04-27T05:29:57.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:56 smithi090 ceph-mon[29180]: slow request 11523.364455 seconds old, received at 2024-04-27T02:17:51.982711+0000: client_request(client.24733:267900 getattr AsXsFs #0x100000001fb 2024-04-27T02:17:51.981940+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:29:58.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39278.501410 secs 2024-04-27T05:29:58.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:57 smithi119 ceph-mon[32752]: pgmap v21438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39278.501410 secs 2024-04-27T05:29:58.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:57 smithi167 ceph-mon[32751]: pgmap v21438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39278.501410 secs 2024-04-27T05:29:58.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:57 smithi090 ceph-mon[29180]: pgmap v21438: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:29:59.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:29:59 smithi090 ceph-mon[29180]: pgmap v21439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:00.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:29:59 smithi119 ceph-mon[32752]: pgmap v21439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:00.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:29:59 smithi167 ceph-mon[32751]: pgmap v21439: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:01.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:30:01.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:30:01.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:30:02.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39282.593825 secs 2024-04-27T05:30:02.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:01 smithi119 ceph-mon[32752]: pgmap v21440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39282.593825 secs 2024-04-27T05:30:02.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:01 smithi167 ceph-mon[32751]: pgmap v21440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39282.593825 secs 2024-04-27T05:30:02.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:01 smithi090 ceph-mon[29180]: pgmap v21440: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:03.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39283.501529 secs 2024-04-27T05:30:03.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39283.501529 secs 2024-04-27T05:30:03.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39283.501529 secs 2024-04-27T05:30:04.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:03 smithi119 ceph-mon[32752]: pgmap v21441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:04.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:03 smithi167 ceph-mon[32751]: pgmap v21441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:04.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:03 smithi090 ceph-mon[29180]: pgmap v21441: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:06.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:05 smithi119 ceph-mon[32752]: pgmap v21442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:06.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:05 smithi167 ceph-mon[32751]: pgmap v21442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:06.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:05 smithi090 ceph-mon[29180]: pgmap v21442: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:07.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39287.593886 secs 2024-04-27T05:30:07.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39287.593886 secs 2024-04-27T05:30:07.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39287.593886 secs 2024-04-27T05:30:08.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39288.501641 secs 2024-04-27T05:30:08.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:07 smithi119 ceph-mon[32752]: pgmap v21443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39288.501641 secs 2024-04-27T05:30:08.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:07 smithi167 ceph-mon[32751]: pgmap v21443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39288.501641 secs 2024-04-27T05:30:08.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:07 smithi090 ceph-mon[29180]: pgmap v21443: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:10.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:09 smithi119 ceph-mon[32752]: pgmap v21444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:10.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:09 smithi167 ceph-mon[32751]: pgmap v21444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:10.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:09 smithi090 ceph-mon[29180]: pgmap v21444: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:12.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39292.593968 secs 2024-04-27T05:30:12.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:11 smithi119 ceph-mon[32752]: pgmap v21445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39292.593968 secs 2024-04-27T05:30:12.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:11 smithi167 ceph-mon[32751]: pgmap v21445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39292.593968 secs 2024-04-27T05:30:12.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:11 smithi090 ceph-mon[29180]: pgmap v21445: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:13.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39293.501763 secs 2024-04-27T05:30:13.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39293.501763 secs 2024-04-27T05:30:13.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39293.501763 secs 2024-04-27T05:30:14.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:13 smithi119 ceph-mon[32752]: pgmap v21446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:14.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:13 smithi167 ceph-mon[32751]: pgmap v21446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:14.130 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:13 smithi090 ceph-mon[29180]: pgmap v21446: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:15.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:14 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:30:15.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:14 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:30:15.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:14 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:30:16.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:15 smithi119 ceph-mon[32752]: pgmap v21447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:30:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:30:16.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:30:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:15 smithi167 ceph-mon[32751]: pgmap v21447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:30:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:30:16.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:30:16.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:15 smithi090 ceph-mon[29180]: pgmap v21447: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:30:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:30:16.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:30:17.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:16 smithi119 ceph-mon[32752]: pgmap v21448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:30:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:16 smithi119 ceph-mon[32752]: pgmap v21449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:17.056 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39297.594028 secs 2024-04-27T05:30:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:16 smithi167 ceph-mon[32751]: pgmap v21448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:30:17.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:16 smithi167 ceph-mon[32751]: pgmap v21449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:17.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39297.594028 secs 2024-04-27T05:30:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:16 smithi090 ceph-mon[29180]: pgmap v21448: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:30:17.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:16 smithi090 ceph-mon[29180]: pgmap v21449: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:17.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39297.594028 secs 2024-04-27T05:30:18.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39298.501845 secs 2024-04-27T05:30:18.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39298.501845 secs 2024-04-27T05:30:18.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39298.501845 secs 2024-04-27T05:30:19.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:18 smithi119 ceph-mon[32752]: pgmap v21450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:30:19.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:18 smithi167 ceph-mon[32751]: pgmap v21450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:30:19.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:18 smithi090 ceph-mon[29180]: pgmap v21450: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:30:21.055 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:20 smithi119 ceph-mon[32752]: pgmap v21451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:21.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:20 smithi167 ceph-mon[32751]: pgmap v21451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:21.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:20 smithi090 ceph-mon[29180]: pgmap v21451: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:22.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39302.594092 secs 2024-04-27T05:30:22.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39302.594092 secs 2024-04-27T05:30:22.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39302.594092 secs 2024-04-27T05:30:23.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:22 smithi167 ceph-mon[32751]: pgmap v21452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:23.070 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39303.501969 secs 2024-04-27T05:30:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:22 smithi090 ceph-mon[29180]: pgmap v21452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:23.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39303.501969 secs 2024-04-27T05:30:23.206 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:22 smithi119 ceph-mon[32752]: pgmap v21452: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:30:23.207 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39303.501969 secs 2024-04-27T05:30:25.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:24 smithi167 ceph-mon[32751]: pgmap v21453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:30:25.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:24 smithi090 ceph-mon[29180]: pgmap v21453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:30:25.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:24 smithi119 ceph-mon[32752]: pgmap v21453: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:30:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:26 smithi167 ceph-mon[32751]: pgmap v21454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:30:27.069 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39307.594148 secs 2024-04-27T05:30:27.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:26 smithi090 ceph-mon[29180]: pgmap v21454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:30:27.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39307.594148 secs 2024-04-27T05:30:27.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:26 smithi119 ceph-mon[32752]: pgmap v21454: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:30:27.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39307.594148 secs 2024-04-27T05:30:28.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39308.502093 secs 2024-04-27T05:30:28.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39308.502093 secs 2024-04-27T05:30:28.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39308.502093 secs 2024-04-27T05:30:29.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:28 smithi090 ceph-mon[29180]: pgmap v21455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:29.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:28 smithi119 ceph-mon[32752]: pgmap v21455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:29.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:28 smithi167 ceph-mon[32751]: pgmap v21455: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:31.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:30 smithi090 ceph-mon[29180]: pgmap v21456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:31.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:30 smithi119 ceph-mon[32752]: pgmap v21456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:31.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:30 smithi167 ceph-mon[32751]: pgmap v21456: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:32.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39312.594207 secs 2024-04-27T05:30:32.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39312.594207 secs 2024-04-27T05:30:32.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39312.594207 secs 2024-04-27T05:30:33.127 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:32 smithi090 ceph-mon[29180]: pgmap v21457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:33.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39313.502187 secs 2024-04-27T05:30:33.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:32 smithi119 ceph-mon[32752]: pgmap v21457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:33.306 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39313.502187 secs 2024-04-27T05:30:33.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:32 smithi167 ceph-mon[32751]: pgmap v21457: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:33.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39313.502187 secs 2024-04-27T05:30:35.128 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:34 smithi090 ceph-mon[29180]: pgmap v21458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:35.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:34 smithi119 ceph-mon[32752]: pgmap v21458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:35.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:34 smithi167 ceph-mon[32751]: pgmap v21458: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:36.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:35 smithi119 ceph-mon[32752]: pgmap v21459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:36.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:35 smithi167 ceph-mon[32751]: pgmap v21459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:36.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:35 smithi090 ceph-mon[29180]: pgmap v21459: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:37.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39317.594262 secs 2024-04-27T05:30:37.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39317.594262 secs 2024-04-27T05:30:37.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39317.594262 secs 2024-04-27T05:30:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39318.502310 secs 2024-04-27T05:30:38.305 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:37 smithi119 ceph-mon[32752]: pgmap v21460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:38.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39318.502310 secs 2024-04-27T05:30:38.319 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:37 smithi167 ceph-mon[32751]: pgmap v21460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39318.502310 secs 2024-04-27T05:30:38.377 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:37 smithi090 ceph-mon[29180]: pgmap v21460: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:30:40.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:40 smithi119 ceph-mon[32752]: pgmap v21461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:40.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:40 smithi167 ceph-mon[32751]: pgmap v21461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:40.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:40 smithi090 ceph-mon[29180]: pgmap v21461: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:41.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39322.594317 secs 2024-04-27T05:30:41.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39322.594317 secs 2024-04-27T05:30:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39322.594317 secs 2024-04-27T05:30:42.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:42 smithi119 ceph-mon[32752]: pgmap v21462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:42.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:42 smithi167 ceph-mon[32751]: pgmap v21462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:42 smithi090 ceph-mon[29180]: pgmap v21462: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:43.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:43 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39323.502421 secs 2024-04-27T05:30:43.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:43 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39323.502421 secs 2024-04-27T05:30:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:43 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39323.502421 secs 2024-04-27T05:30:44.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:44 smithi119 ceph-mon[32752]: pgmap v21463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:44.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:44 smithi167 ceph-mon[32751]: pgmap v21463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:44.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:44 smithi090 ceph-mon[29180]: pgmap v21463: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:46.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:46 smithi119 ceph-mon[32752]: pgmap v21464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:46.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39327.594398 secs 2024-04-27T05:30:46.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:46 smithi167 ceph-mon[32751]: pgmap v21464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:46.570 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39327.594398 secs 2024-04-27T05:30:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:46 smithi090 ceph-mon[29180]: pgmap v21464: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:46.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39327.594398 secs 2024-04-27T05:30:47.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39328.502533 secs 2024-04-27T05:30:47.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39328.502533 secs 2024-04-27T05:30:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39328.502533 secs 2024-04-27T05:30:48.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:48 smithi119 ceph-mon[32752]: pgmap v21465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:48.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:48 smithi167 ceph-mon[32751]: pgmap v21465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:48.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:48 smithi090 ceph-mon[29180]: pgmap v21465: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:50.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:50 smithi119 ceph-mon[32752]: pgmap v21466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:50.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:50 smithi167 ceph-mon[32751]: pgmap v21466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:50.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:50 smithi090 ceph-mon[29180]: pgmap v21466: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:51.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39332.594485 secs 2024-04-27T05:30:51.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39332.594485 secs 2024-04-27T05:30:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39332.594485 secs 2024-04-27T05:30:52.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:52 smithi119 ceph-mon[32752]: pgmap v21467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:52.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39333.502620 secs 2024-04-27T05:30:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:52 smithi167 ceph-mon[32751]: pgmap v21467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:52.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39333.502620 secs 2024-04-27T05:30:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:52 smithi090 ceph-mon[29180]: pgmap v21467: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39333.502620 secs 2024-04-27T05:30:54.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:54 smithi119 ceph-mon[32752]: pgmap v21468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:54.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:54 smithi167 ceph-mon[32751]: pgmap v21468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:54.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:54 smithi090 ceph-mon[29180]: pgmap v21468: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:56 smithi119 ceph-mon[32752]: pgmap v21469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:56.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39337.594571 secs 2024-04-27T05:30:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:56 smithi167 ceph-mon[32751]: pgmap v21469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:56.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39337.594571 secs 2024-04-27T05:30:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:56 smithi090 ceph-mon[29180]: pgmap v21469: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:30:56.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39337.594571 secs 2024-04-27T05:30:57.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39338.502684 secs 2024-04-27T05:30:57.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39338.502684 secs 2024-04-27T05:30:57.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39338.502684 secs 2024-04-27T05:30:58.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:30:58 smithi119 ceph-mon[32752]: pgmap v21470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:58.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:30:58 smithi167 ceph-mon[32751]: pgmap v21470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:30:58.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:30:58 smithi090 ceph-mon[29180]: pgmap v21470: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:00.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:00 smithi119 ceph-mon[32752]: pgmap v21471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:00.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:00 smithi167 ceph-mon[32751]: pgmap v21471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:00.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:00 smithi090 ceph-mon[29180]: pgmap v21471: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:01.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39342.594658 secs 2024-04-27T05:31:01.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39342.594658 secs 2024-04-27T05:31:01.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39342.594658 secs 2024-04-27T05:31:02.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:02 smithi090 ceph-mon[29180]: pgmap v21472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:02.503 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39343.502809 secs 2024-04-27T05:31:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:02 smithi119 ceph-mon[32752]: pgmap v21472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:02.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39343.502809 secs 2024-04-27T05:31:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:02 smithi167 ceph-mon[32751]: pgmap v21472: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:02.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39343.502809 secs 2024-04-27T05:31:04.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:04 smithi119 ceph-mon[32752]: pgmap v21473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:04.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:04 smithi167 ceph-mon[32751]: pgmap v21473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:04.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:04 smithi090 ceph-mon[29180]: pgmap v21473: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:06.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:06 smithi119 ceph-mon[32752]: pgmap v21474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:06.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39347.594741 secs 2024-04-27T05:31:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:06 smithi167 ceph-mon[32751]: pgmap v21474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:06.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39347.594741 secs 2024-04-27T05:31:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:06 smithi090 ceph-mon[29180]: pgmap v21474: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:06.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39347.594741 secs 2024-04-27T05:31:07.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39348.502878 secs 2024-04-27T05:31:07.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39348.502878 secs 2024-04-27T05:31:07.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39348.502878 secs 2024-04-27T05:31:08.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:08 smithi119 ceph-mon[32752]: pgmap v21475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:08.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:08 smithi167 ceph-mon[32751]: pgmap v21475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:08.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:08 smithi090 ceph-mon[29180]: pgmap v21475: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:10.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:10 smithi119 ceph-mon[32752]: pgmap v21476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:10.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:10 smithi167 ceph-mon[32751]: pgmap v21476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:10.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:10 smithi090 ceph-mon[29180]: pgmap v21476: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:11.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39352.594821 secs 2024-04-27T05:31:11.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39352.594821 secs 2024-04-27T05:31:11.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39352.594821 secs 2024-04-27T05:31:12.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:12 smithi119 ceph-mon[32752]: pgmap v21477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:12 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 39353.502996 secs 2024-04-27T05:31:12.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:12 smithi119 ceph-mon[32752]: slow request 11522.773580 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:31:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:12 smithi167 ceph-mon[32751]: pgmap v21477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:12 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 39353.502996 secs 2024-04-27T05:31:12.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:12 smithi167 ceph-mon[32751]: slow request 11522.773580 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:31:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:12 smithi090 ceph-mon[29180]: pgmap v21477: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:12 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 39353.502996 secs 2024-04-27T05:31:12.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:12 smithi090 ceph-mon[29180]: slow request 11522.773580 seconds old, received at 2024-04-27T02:19:08.480061+0000: client_request(client.17634:1 getattr pAsLsXsFs #0x1 2024-04-27T02:19:08.475874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:31:14.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:14 smithi119 ceph-mon[32752]: pgmap v21478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:14.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:14 smithi167 ceph-mon[32751]: pgmap v21478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:14.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:14 smithi090 ceph-mon[29180]: pgmap v21478: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:15.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:15 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:31:15.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:15 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:31:15.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:15 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:31:16.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:16 smithi119 ceph-mon[32752]: pgmap v21479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39357.594881 secs 2024-04-27T05:31:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:31:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:31:16.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:31:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:16 smithi167 ceph-mon[32751]: pgmap v21479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39357.594881 secs 2024-04-27T05:31:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:31:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:31:16.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:31:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:16 smithi090 ceph-mon[29180]: pgmap v21479: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:16.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39357.594881 secs 2024-04-27T05:31:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:31:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:31:16.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:31:17.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:17 smithi119 ceph-mon[32752]: pgmap v21480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:31:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:17 smithi119 ceph-mon[32752]: pgmap v21481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:17.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39358.503053 secs 2024-04-27T05:31:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:17 smithi167 ceph-mon[32751]: pgmap v21480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:31:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:17 smithi167 ceph-mon[32751]: pgmap v21481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:17.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39358.503053 secs 2024-04-27T05:31:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:17 smithi090 ceph-mon[29180]: pgmap v21480: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:31:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:17 smithi090 ceph-mon[29180]: pgmap v21481: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:17.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39358.503053 secs 2024-04-27T05:31:19.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:19 smithi119 ceph-mon[32752]: pgmap v21482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:31:19.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:19 smithi167 ceph-mon[32751]: pgmap v21482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:31:19.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:19 smithi090 ceph-mon[29180]: pgmap v21482: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:31:21.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:21 smithi119 ceph-mon[32752]: pgmap v21483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:21.556 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39362.594968 secs 2024-04-27T05:31:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:21 smithi167 ceph-mon[32751]: pgmap v21483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:21.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39362.594968 secs 2024-04-27T05:31:21.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:21 smithi090 ceph-mon[29180]: pgmap v21483: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:21.628 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39362.594968 secs 2024-04-27T05:31:22.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39363.503198 secs 2024-04-27T05:31:22.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39363.503198 secs 2024-04-27T05:31:22.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39363.503198 secs 2024-04-27T05:31:23.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:23 smithi119 ceph-mon[32752]: pgmap v21484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:23.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:23 smithi167 ceph-mon[32751]: pgmap v21484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:23.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:23 smithi090 ceph-mon[29180]: pgmap v21484: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:25.555 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:25 smithi119 ceph-mon[32752]: pgmap v21485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:25.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:25 smithi167 ceph-mon[32751]: pgmap v21485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:25.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:25 smithi090 ceph-mon[29180]: pgmap v21485: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:31:26.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39367.595026 secs 2024-04-27T05:31:26.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39367.595026 secs 2024-04-27T05:31:26.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39367.595026 secs 2024-04-27T05:31:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:27 smithi167 ceph-mon[32751]: pgmap v21486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:31:27.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39368.503316 secs 2024-04-27T05:31:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:27 smithi090 ceph-mon[29180]: pgmap v21486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:31:27.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39368.503316 secs 2024-04-27T05:31:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:27 smithi119 ceph-mon[32752]: pgmap v21486: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:31:27.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39368.503316 secs 2024-04-27T05:31:29.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:29 smithi167 ceph-mon[32751]: pgmap v21487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:29.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:29 smithi090 ceph-mon[29180]: pgmap v21487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:29.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:29 smithi119 ceph-mon[32752]: pgmap v21487: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:31 smithi167 ceph-mon[32751]: pgmap v21488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:31.569 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39372.595089 secs 2024-04-27T05:31:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:31 smithi090 ceph-mon[29180]: pgmap v21488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:31.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39372.595089 secs 2024-04-27T05:31:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:31 smithi119 ceph-mon[32752]: pgmap v21488: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:31.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39372.595089 secs 2024-04-27T05:31:32.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39373.503439 secs 2024-04-27T05:31:32.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39373.503439 secs 2024-04-27T05:31:32.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39373.503439 secs 2024-04-27T05:31:33.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:33 smithi090 ceph-mon[29180]: pgmap v21489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:33.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:33 smithi119 ceph-mon[32752]: pgmap v21489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:33.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:33 smithi167 ceph-mon[32751]: pgmap v21489: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:35.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:35 smithi090 ceph-mon[29180]: pgmap v21490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:35.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:35 smithi119 ceph-mon[32752]: pgmap v21490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:35.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:35 smithi167 ceph-mon[32751]: pgmap v21490: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:36.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39377.595144 secs 2024-04-27T05:31:36.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39377.595144 secs 2024-04-27T05:31:36.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39377.595144 secs 2024-04-27T05:31:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:37 smithi090 ceph-mon[29180]: pgmap v21491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:37.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39378.503496 secs 2024-04-27T05:31:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:37 smithi119 ceph-mon[32752]: pgmap v21491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:37.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39378.503496 secs 2024-04-27T05:31:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:37 smithi167 ceph-mon[32751]: pgmap v21491: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:37.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39378.503496 secs 2024-04-27T05:31:39.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:39 smithi090 ceph-mon[29180]: pgmap v21492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:39.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:39 smithi119 ceph-mon[32752]: pgmap v21492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:39.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:39 smithi167 ceph-mon[32751]: pgmap v21492: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:41 smithi090 ceph-mon[29180]: pgmap v21493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:41.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39382.595204 secs 2024-04-27T05:31:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:41 smithi119 ceph-mon[32752]: pgmap v21493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:41.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39382.595204 secs 2024-04-27T05:31:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:41 smithi167 ceph-mon[32751]: pgmap v21493: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:41.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39382.595204 secs 2024-04-27T05:31:42.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39383.503621 secs 2024-04-27T05:31:42.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39383.503621 secs 2024-04-27T05:31:42.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39383.503621 secs 2024-04-27T05:31:43.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:43 smithi090 ceph-mon[29180]: pgmap v21494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:43.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:43 smithi119 ceph-mon[32752]: pgmap v21494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:43.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:43 smithi167 ceph-mon[32751]: pgmap v21494: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:45.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:45 smithi090 ceph-mon[29180]: pgmap v21495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:45.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:45 smithi119 ceph-mon[32752]: pgmap v21495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:45.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:45 smithi167 ceph-mon[32751]: pgmap v21495: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:46.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39387.595274 secs 2024-04-27T05:31:46.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39387.595274 secs 2024-04-27T05:31:46.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39387.595274 secs 2024-04-27T05:31:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:47 smithi090 ceph-mon[29180]: pgmap v21496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:47.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39388.503734 secs 2024-04-27T05:31:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:47 smithi119 ceph-mon[32752]: pgmap v21496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:47.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39388.503734 secs 2024-04-27T05:31:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:47 smithi167 ceph-mon[32751]: pgmap v21496: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:47.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39388.503734 secs 2024-04-27T05:31:49.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:49 smithi090 ceph-mon[29180]: pgmap v21497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:49.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:49 smithi119 ceph-mon[32752]: pgmap v21497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:49.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:49 smithi167 ceph-mon[32751]: pgmap v21497: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:51 smithi090 ceph-mon[29180]: pgmap v21498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:51.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39392.595359 secs 2024-04-27T05:31:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:51 smithi119 ceph-mon[32752]: pgmap v21498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:51.806 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39392.595359 secs 2024-04-27T05:31:51.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:51 smithi167 ceph-mon[32751]: pgmap v21498: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:51.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39392.595359 secs 2024-04-27T05:31:52.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39393.503838 secs 2024-04-27T05:31:52.720 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39393.503838 secs 2024-04-27T05:31:52.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39393.503838 secs 2024-04-27T05:31:53.627 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:53 smithi090 ceph-mon[29180]: pgmap v21499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:53.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:53 smithi119 ceph-mon[32752]: pgmap v21499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:53.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:53 smithi167 ceph-mon[32751]: pgmap v21499: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:55.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:55 smithi119 ceph-mon[32752]: pgmap v21500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:55.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:55 smithi167 ceph-mon[32751]: pgmap v21500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:55.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:55 smithi090 ceph-mon[29180]: pgmap v21500: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:31:56.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39397.595445 secs 2024-04-27T05:31:56.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39397.595445 secs 2024-04-27T05:31:56.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39397.595445 secs 2024-04-27T05:31:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:57 smithi119 ceph-mon[32752]: pgmap v21501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:57.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39398.503911 secs 2024-04-27T05:31:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:57 smithi167 ceph-mon[32751]: pgmap v21501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:57.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39398.503911 secs 2024-04-27T05:31:57.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:57 smithi090 ceph-mon[29180]: pgmap v21501: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:57.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39398.503911 secs 2024-04-27T05:31:59.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:31:59 smithi119 ceph-mon[32752]: pgmap v21502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:59.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:31:59 smithi167 ceph-mon[32751]: pgmap v21502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:31:59.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:31:59 smithi090 ceph-mon[29180]: pgmap v21502: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:01.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:01 smithi119 ceph-mon[32752]: pgmap v21503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:01.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39402.595516 secs 2024-04-27T05:32:01.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:01 smithi167 ceph-mon[32751]: pgmap v21503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:01.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39402.595516 secs 2024-04-27T05:32:01.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:01 smithi090 ceph-mon[29180]: pgmap v21503: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:01.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39402.595516 secs 2024-04-27T05:32:02.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39403.504032 secs 2024-04-27T05:32:02.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39403.504032 secs 2024-04-27T05:32:02.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39403.504032 secs 2024-04-27T05:32:03.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:03 smithi119 ceph-mon[32752]: pgmap v21504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:03.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:03 smithi167 ceph-mon[32751]: pgmap v21504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:03.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:03 smithi090 ceph-mon[29180]: pgmap v21504: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:05.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:05 smithi119 ceph-mon[32752]: pgmap v21505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:05.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:05 smithi167 ceph-mon[32751]: pgmap v21505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:05.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:05 smithi090 ceph-mon[29180]: pgmap v21505: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:06.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:06 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39407.595601 secs 2024-04-27T05:32:06.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:06 smithi119 ceph-mon[32752]: slow request 23040.101613 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:32:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:06 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39407.595601 secs 2024-04-27T05:32:06.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:06 smithi167 ceph-mon[32751]: slow request 23040.101613 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:32:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:06 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39407.595601 secs 2024-04-27T05:32:06.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:06 smithi090 ceph-mon[29180]: slow request 23040.101613 seconds old, received at 2024-04-26T23:08:05.247392+0000: client_request(client.17415:1 getattr pAsLsXsFs #0x1 2024-04-26T23:08:05.241320+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:07 smithi119 ceph-mon[32752]: pgmap v21506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:07.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39408.504152 secs 2024-04-27T05:32:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:07 smithi167 ceph-mon[32751]: pgmap v21506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:07.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39408.504152 secs 2024-04-27T05:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:07 smithi090 ceph-mon[29180]: pgmap v21506: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:07.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39408.504152 secs 2024-04-27T05:32:09.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:09 smithi119 ceph-mon[32752]: pgmap v21507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:09.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:09 smithi167 ceph-mon[32751]: pgmap v21507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:09.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:09 smithi090 ceph-mon[29180]: pgmap v21507: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:11 smithi119 ceph-mon[32752]: pgmap v21508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:11.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39412.595684 secs 2024-04-27T05:32:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:11 smithi167 ceph-mon[32751]: pgmap v21508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:11.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39412.595684 secs 2024-04-27T05:32:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:11 smithi090 ceph-mon[29180]: pgmap v21508: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:11.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39412.595684 secs 2024-04-27T05:32:12.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39413.504281 secs 2024-04-27T05:32:12.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39413.504281 secs 2024-04-27T05:32:12.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39413.504281 secs 2024-04-27T05:32:13.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:13 smithi119 ceph-mon[32752]: pgmap v21509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:13.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:13 smithi167 ceph-mon[32751]: pgmap v21509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:13.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:13 smithi090 ceph-mon[29180]: pgmap v21509: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:15.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:15 smithi119 ceph-mon[32752]: pgmap v21510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:15.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:15 smithi167 ceph-mon[32751]: pgmap v21510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:15.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:15 smithi090 ceph-mon[29180]: pgmap v21510: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39417.595772 secs 2024-04-27T05:32:16.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:16 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:32:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39417.595772 secs 2024-04-27T05:32:16.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:16 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:32:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39417.595772 secs 2024-04-27T05:32:16.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:16 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:32:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:17 smithi119 ceph-mon[32752]: pgmap v21511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:17.804 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39418.504367 secs 2024-04-27T05:32:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:32:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:32:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:17 smithi119 ceph-mon[32752]: pgmap v21512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:32:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:17 smithi119 ceph-mon[32752]: pgmap v21513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:17.805 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:32:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:17 smithi167 ceph-mon[32751]: pgmap v21511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:17.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39418.504367 secs 2024-04-27T05:32:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:32:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:32:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:17 smithi167 ceph-mon[32751]: pgmap v21512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:32:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:17 smithi167 ceph-mon[32751]: pgmap v21513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:17.819 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:32:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:17 smithi090 ceph-mon[29180]: pgmap v21511: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:17.876 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39418.504367 secs 2024-04-27T05:32:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:32:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:32:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:17 smithi090 ceph-mon[29180]: pgmap v21512: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:32:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:17 smithi090 ceph-mon[29180]: pgmap v21513: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:17.877 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:32:20.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:19 smithi119 ceph-mon[32752]: pgmap v21514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:20.120 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:19 smithi167 ceph-mon[32751]: pgmap v21514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:20.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:19 smithi090 ceph-mon[29180]: pgmap v21514: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39422.595849 secs 2024-04-27T05:32:22.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:21 smithi119 ceph-mon[32752]: pgmap v21515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39422.595849 secs 2024-04-27T05:32:22.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:21 smithi167 ceph-mon[32751]: pgmap v21515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39422.595849 secs 2024-04-27T05:32:22.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:21 smithi090 ceph-mon[29180]: pgmap v21515: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:23.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39423.504492 secs 2024-04-27T05:32:23.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39423.504492 secs 2024-04-27T05:32:23.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39423.504492 secs 2024-04-27T05:32:24.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:23 smithi119 ceph-mon[32752]: pgmap v21516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:32:24.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:23 smithi167 ceph-mon[32751]: pgmap v21516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:32:24.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:23 smithi090 ceph-mon[29180]: pgmap v21516: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:32:26.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:25 smithi119 ceph-mon[32752]: pgmap v21517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:26.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:25 smithi167 ceph-mon[32751]: pgmap v21517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:26.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:25 smithi090 ceph-mon[29180]: pgmap v21517: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:32:27.054 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39427.595910 secs 2024-04-27T05:32:27.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39427.595910 secs 2024-04-27T05:32:27.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39427.595910 secs 2024-04-27T05:32:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39428.504605 secs 2024-04-27T05:32:28.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:27 smithi167 ceph-mon[32751]: pgmap v21518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:32:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39428.504605 secs 2024-04-27T05:32:28.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:27 smithi090 ceph-mon[29180]: pgmap v21518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:32:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39428.504605 secs 2024-04-27T05:32:28.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:27 smithi119 ceph-mon[32752]: pgmap v21518: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:32:30.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:29 smithi167 ceph-mon[32751]: pgmap v21519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:30.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:29 smithi090 ceph-mon[29180]: pgmap v21519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:30.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:29 smithi119 ceph-mon[32752]: pgmap v21519: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:32.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39432.595971 secs 2024-04-27T05:32:32.068 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:31 smithi167 ceph-mon[32751]: pgmap v21520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:32.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39432.595971 secs 2024-04-27T05:32:32.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:31 smithi090 ceph-mon[29180]: pgmap v21520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:32.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39432.595971 secs 2024-04-27T05:32:32.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:31 smithi119 ceph-mon[32752]: pgmap v21520: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:33.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39433.504728 secs 2024-04-27T05:32:33.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39433.504728 secs 2024-04-27T05:32:33.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39433.504728 secs 2024-04-27T05:32:34.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:33 smithi090 ceph-mon[29180]: pgmap v21521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:34.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:33 smithi119 ceph-mon[32752]: pgmap v21521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:34.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:33 smithi167 ceph-mon[32751]: pgmap v21521: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:36.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:35 smithi090 ceph-mon[29180]: pgmap v21522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:36.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:35 smithi119 ceph-mon[32752]: pgmap v21522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:36.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:35 smithi167 ceph-mon[32751]: pgmap v21522: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:37.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39437.596055 secs 2024-04-27T05:32:37.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39437.596055 secs 2024-04-27T05:32:37.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39437.596055 secs 2024-04-27T05:32:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39438.504849 secs 2024-04-27T05:32:38.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:37 smithi090 ceph-mon[29180]: pgmap v21523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:38.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39438.504849 secs 2024-04-27T05:32:38.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:37 smithi119 ceph-mon[32752]: pgmap v21523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39438.504849 secs 2024-04-27T05:32:38.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:37 smithi167 ceph-mon[32751]: pgmap v21523: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:40.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:39 smithi090 ceph-mon[29180]: pgmap v21524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:40.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:39 smithi119 ceph-mon[32752]: pgmap v21524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:40.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:39 smithi167 ceph-mon[32751]: pgmap v21524: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39442.596141 secs 2024-04-27T05:32:42.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:41 smithi090 ceph-mon[29180]: pgmap v21525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:42.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39442.596141 secs 2024-04-27T05:32:42.304 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:41 smithi119 ceph-mon[32752]: pgmap v21525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39442.596141 secs 2024-04-27T05:32:42.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:41 smithi167 ceph-mon[32751]: pgmap v21525: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:43.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39443.504965 secs 2024-04-27T05:32:43.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39443.504965 secs 2024-04-27T05:32:43.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39443.504965 secs 2024-04-27T05:32:44.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:43 smithi090 ceph-mon[29180]: pgmap v21526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:44.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:43 smithi119 ceph-mon[32752]: pgmap v21526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:44.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:43 smithi167 ceph-mon[32751]: pgmap v21526: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:46.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:45 smithi090 ceph-mon[29180]: pgmap v21527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:46.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:45 smithi119 ceph-mon[32752]: pgmap v21527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:46.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:45 smithi167 ceph-mon[32751]: pgmap v21527: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:47.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39447.596222 secs 2024-04-27T05:32:47.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39447.596222 secs 2024-04-27T05:32:47.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39447.596222 secs 2024-04-27T05:32:48.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39448.505106 secs 2024-04-27T05:32:48.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:47 smithi090 ceph-mon[29180]: pgmap v21528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39448.505106 secs 2024-04-27T05:32:48.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:47 smithi119 ceph-mon[32752]: pgmap v21528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39448.505106 secs 2024-04-27T05:32:48.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:47 smithi167 ceph-mon[32751]: pgmap v21528: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:50.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:49 smithi090 ceph-mon[29180]: pgmap v21529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:50.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:49 smithi119 ceph-mon[32752]: pgmap v21529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:50.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:49 smithi167 ceph-mon[32751]: pgmap v21529: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:52.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39452.596283 secs 2024-04-27T05:32:52.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:51 smithi167 ceph-mon[32751]: pgmap v21530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39452.596283 secs 2024-04-27T05:32:52.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:51 smithi090 ceph-mon[29180]: pgmap v21530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39452.596283 secs 2024-04-27T05:32:52.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:51 smithi119 ceph-mon[32752]: pgmap v21530: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:53.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39453.505206 secs 2024-04-27T05:32:53.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39453.505206 secs 2024-04-27T05:32:53.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39453.505206 secs 2024-04-27T05:32:54.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:53 smithi090 ceph-mon[29180]: pgmap v21531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:54.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:53 smithi119 ceph-mon[32752]: pgmap v21531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:54.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:53 smithi167 ceph-mon[32751]: pgmap v21531: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:56.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:55 smithi090 ceph-mon[29180]: pgmap v21532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:56.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:55 smithi119 ceph-mon[32752]: pgmap v21532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:56.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:55 smithi167 ceph-mon[32751]: pgmap v21532: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:57.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39457.596366 secs 2024-04-27T05:32:57.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39457.596366 secs 2024-04-27T05:32:57.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39457.596366 secs 2024-04-27T05:32:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39458.505328 secs 2024-04-27T05:32:58.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:57 smithi090 ceph-mon[29180]: pgmap v21533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39458.505328 secs 2024-04-27T05:32:58.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:57 smithi119 ceph-mon[32752]: pgmap v21533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:58.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39458.505328 secs 2024-04-27T05:32:58.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:57 smithi167 ceph-mon[32751]: pgmap v21533: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:32:59.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:32:58 smithi119 ceph-mon[32752]: pgmap v21534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:59.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:32:58 smithi167 ceph-mon[32751]: pgmap v21534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:32:59.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:32:58 smithi090 ceph-mon[29180]: pgmap v21534: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39462.596443 secs 2024-04-27T05:33:02.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:01 smithi119 ceph-mon[32752]: pgmap v21535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:02.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39462.596443 secs 2024-04-27T05:33:02.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:01 smithi167 ceph-mon[32751]: pgmap v21535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39462.596443 secs 2024-04-27T05:33:02.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:01 smithi090 ceph-mon[29180]: pgmap v21535: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:03.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39463.505463 secs 2024-04-27T05:33:03.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39463.505463 secs 2024-04-27T05:33:03.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39463.505463 secs 2024-04-27T05:33:04.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:03 smithi119 ceph-mon[32752]: pgmap v21536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:04.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:03 smithi167 ceph-mon[32751]: pgmap v21536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:04.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:03 smithi090 ceph-mon[29180]: pgmap v21536: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:06.053 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:05 smithi119 ceph-mon[32752]: pgmap v21537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:06.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:05 smithi167 ceph-mon[32751]: pgmap v21537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:06.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:05 smithi090 ceph-mon[29180]: pgmap v21537: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:07.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39467.596520 secs 2024-04-27T05:33:07.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39467.596520 secs 2024-04-27T05:33:07.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39467.596520 secs 2024-04-27T05:33:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39468.505508 secs 2024-04-27T05:33:08.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:07 smithi167 ceph-mon[32751]: pgmap v21538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39468.505508 secs 2024-04-27T05:33:08.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:07 smithi090 ceph-mon[29180]: pgmap v21538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39468.505508 secs 2024-04-27T05:33:08.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:07 smithi119 ceph-mon[32752]: pgmap v21538: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:10.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:09 smithi167 ceph-mon[32751]: pgmap v21539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:10.126 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:09 smithi090 ceph-mon[29180]: pgmap v21539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:10.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:09 smithi119 ceph-mon[32752]: pgmap v21539: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:12.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39472.596603 secs 2024-04-27T05:33:12.067 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:11 smithi167 ceph-mon[32751]: pgmap v21540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39472.596603 secs 2024-04-27T05:33:12.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:11 smithi090 ceph-mon[29180]: pgmap v21540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:12.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39472.596603 secs 2024-04-27T05:33:12.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:11 smithi119 ceph-mon[32752]: pgmap v21540: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:13.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39473.505631 secs 2024-04-27T05:33:13.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39473.505631 secs 2024-04-27T05:33:13.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39473.505631 secs 2024-04-27T05:33:14.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:13 smithi090 ceph-mon[29180]: pgmap v21541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:14.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:13 smithi119 ceph-mon[32752]: pgmap v21541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:14.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:13 smithi167 ceph-mon[32751]: pgmap v21541: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:16.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:15 smithi090 ceph-mon[29180]: pgmap v21542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:16.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:15 smithi119 ceph-mon[32752]: pgmap v21542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:16.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:15 smithi167 ceph-mon[32751]: pgmap v21542: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:17.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39477.596687 secs 2024-04-27T05:33:17.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39477.596687 secs 2024-04-27T05:33:17.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39477.596687 secs 2024-04-27T05:33:18.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39478.505718 secs 2024-04-27T05:33:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:17 smithi090 ceph-mon[29180]: pgmap v21543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:33:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:33:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:33:18.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:33:18.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39478.505718 secs 2024-04-27T05:33:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:17 smithi119 ceph-mon[32752]: pgmap v21543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:33:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:33:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:33:18.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:33:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39478.505718 secs 2024-04-27T05:33:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:17 smithi167 ceph-mon[32751]: pgmap v21543: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:33:18.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:33:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:33:18.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:33:19.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:18 smithi090 ceph-mon[29180]: pgmap v21544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:33:19.125 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:18 smithi090 ceph-mon[29180]: pgmap v21545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:33:19.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:18 smithi119 ceph-mon[32752]: pgmap v21544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:33:19.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:18 smithi119 ceph-mon[32752]: pgmap v21545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:33:19.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:18 smithi167 ceph-mon[32751]: pgmap v21544: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:33:19.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:18 smithi167 ceph-mon[32751]: pgmap v21545: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:33:21.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:20 smithi090 ceph-mon[29180]: pgmap v21546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:21.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:20 smithi119 ceph-mon[32752]: pgmap v21546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:21.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:20 smithi167 ceph-mon[32751]: pgmap v21546: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:22.025 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39482.596770 secs 2024-04-27T05:33:22.218 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39482.596770 secs 2024-04-27T05:33:22.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39482.596770 secs 2024-04-27T05:33:23.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39483.505803 secs 2024-04-27T05:33:23.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:22 smithi090 ceph-mon[29180]: pgmap v21547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:23.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39483.505803 secs 2024-04-27T05:33:23.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:22 smithi119 ceph-mon[32752]: pgmap v21547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:23.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39483.505803 secs 2024-04-27T05:33:23.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:22 smithi167 ceph-mon[32751]: pgmap v21547: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:25.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:24 smithi090 ceph-mon[29180]: pgmap v21548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:25.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:24 smithi119 ceph-mon[32752]: pgmap v21548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:25.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:24 smithi167 ceph-mon[32751]: pgmap v21548: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:27.104 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39487.596834 secs 2024-04-27T05:33:27.105 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:26 smithi119 ceph-mon[32752]: pgmap v21549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:27.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39487.596834 secs 2024-04-27T05:33:27.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:26 smithi090 ceph-mon[29180]: pgmap v21549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39487.596834 secs 2024-04-27T05:33:27.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:26 smithi167 ceph-mon[32751]: pgmap v21549: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:33:28.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39488.505928 secs 2024-04-27T05:33:28.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39488.505928 secs 2024-04-27T05:33:28.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39488.505928 secs 2024-04-27T05:33:29.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:28 smithi090 ceph-mon[29180]: pgmap v21550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:33:29.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:28 smithi119 ceph-mon[32752]: pgmap v21550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:33:29.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:28 smithi167 ceph-mon[32751]: pgmap v21550: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:33:31.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:30 smithi090 ceph-mon[29180]: pgmap v21551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:31.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:30 smithi119 ceph-mon[32752]: pgmap v21551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:31.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:30 smithi167 ceph-mon[32751]: pgmap v21551: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:32.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39492.596887 secs 2024-04-27T05:33:32.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39492.596887 secs 2024-04-27T05:33:32.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39492.596887 secs 2024-04-27T05:33:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39493.506048 secs 2024-04-27T05:33:33.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:32 smithi090 ceph-mon[29180]: pgmap v21552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:33.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39493.506048 secs 2024-04-27T05:33:33.303 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:32 smithi119 ceph-mon[32752]: pgmap v21552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39493.506048 secs 2024-04-27T05:33:33.317 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:32 smithi167 ceph-mon[32751]: pgmap v21552: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:35.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:34 smithi090 ceph-mon[29180]: pgmap v21553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:35.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:34 smithi119 ceph-mon[32752]: pgmap v21553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:35.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:34 smithi167 ceph-mon[32751]: pgmap v21553: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:37.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39497.596972 secs 2024-04-27T05:33:37.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:36 smithi090 ceph-mon[29180]: pgmap v21554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:37.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39497.596972 secs 2024-04-27T05:33:37.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:36 smithi119 ceph-mon[32752]: pgmap v21554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:37.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39497.596972 secs 2024-04-27T05:33:37.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:36 smithi167 ceph-mon[32751]: pgmap v21554: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:38.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39498.506141 secs 2024-04-27T05:33:38.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:37 smithi090 ceph-mon[29180]: pgmap v21555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:38.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39498.506141 secs 2024-04-27T05:33:38.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:37 smithi119 ceph-mon[32752]: pgmap v21555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:38.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39498.506141 secs 2024-04-27T05:33:38.316 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:37 smithi167 ceph-mon[32751]: pgmap v21555: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:40.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:40 smithi090 ceph-mon[29180]: pgmap v21556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:41.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:40 smithi119 ceph-mon[32752]: pgmap v21556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:41.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:40 smithi167 ceph-mon[32751]: pgmap v21556: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:41.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39502.597030 secs 2024-04-27T05:33:42.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39502.597030 secs 2024-04-27T05:33:42.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39502.597030 secs 2024-04-27T05:33:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39503.506278 secs 2024-04-27T05:33:42.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:42 smithi090 ceph-mon[29180]: pgmap v21557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:43.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39503.506278 secs 2024-04-27T05:33:43.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:42 smithi119 ceph-mon[32752]: pgmap v21557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:43.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39503.506278 secs 2024-04-27T05:33:43.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:42 smithi167 ceph-mon[32751]: pgmap v21557: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:44.873 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:44 smithi090 ceph-mon[29180]: pgmap v21558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:45.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:44 smithi119 ceph-mon[32752]: pgmap v21558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:45.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:44 smithi167 ceph-mon[32751]: pgmap v21558: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39507.597090 secs 2024-04-27T05:33:46.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:46 smithi090 ceph-mon[29180]: pgmap v21559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:47.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39507.597090 secs 2024-04-27T05:33:47.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:46 smithi119 ceph-mon[32752]: pgmap v21559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:47.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39507.597090 secs 2024-04-27T05:33:47.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:46 smithi167 ceph-mon[32751]: pgmap v21559: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:47.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39508.506426 secs 2024-04-27T05:33:48.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39508.506426 secs 2024-04-27T05:33:48.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39508.506426 secs 2024-04-27T05:33:48.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:48 smithi090 ceph-mon[29180]: pgmap v21560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:49.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:48 smithi119 ceph-mon[32752]: pgmap v21560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:49.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:48 smithi167 ceph-mon[32751]: pgmap v21560: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:50.874 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:50 smithi090 ceph-mon[29180]: pgmap v21561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:51.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:50 smithi119 ceph-mon[32752]: pgmap v21561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:51.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:50 smithi167 ceph-mon[32751]: pgmap v21561: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:52.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39512.597151 secs 2024-04-27T05:33:52.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39512.597151 secs 2024-04-27T05:33:52.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39512.597151 secs 2024-04-27T05:33:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39513.506497 secs 2024-04-27T05:33:52.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:52 smithi167 ceph-mon[32751]: pgmap v21562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:53.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39513.506497 secs 2024-04-27T05:33:53.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:52 smithi119 ceph-mon[32752]: pgmap v21562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:53.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39513.506497 secs 2024-04-27T05:33:53.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:52 smithi090 ceph-mon[29180]: pgmap v21562: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:55.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:54 smithi119 ceph-mon[32752]: pgmap v21563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:55.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:54 smithi167 ceph-mon[32751]: pgmap v21563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:55.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:54 smithi090 ceph-mon[29180]: pgmap v21563: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:33:56.926 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39517.597226 secs 2024-04-27T05:33:56.926 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:56 smithi090 ceph-mon[29180]: pgmap v21564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:57.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39517.597226 secs 2024-04-27T05:33:57.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:56 smithi119 ceph-mon[32752]: pgmap v21564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:57.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39517.597226 secs 2024-04-27T05:33:57.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:56 smithi167 ceph-mon[32751]: pgmap v21564: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:57.952 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39518.506569 secs 2024-04-27T05:33:58.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39518.506569 secs 2024-04-27T05:33:58.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39518.506569 secs 2024-04-27T05:33:59.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:33:58 smithi119 ceph-mon[32752]: pgmap v21565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:59.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:33:58 smithi167 ceph-mon[32751]: pgmap v21565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:33:59.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:33:58 smithi090 ceph-mon[29180]: pgmap v21565: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:01.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:00 smithi119 ceph-mon[32752]: pgmap v21566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:01.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:00 smithi167 ceph-mon[32751]: pgmap v21566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:01.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:00 smithi090 ceph-mon[29180]: pgmap v21566: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:02.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:01 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39522.597306 secs 2024-04-27T05:34:02.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:01 smithi119 ceph-mon[32752]: slow request 720.245605 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:34:02.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:01 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39522.597306 secs 2024-04-27T05:34:02.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:01 smithi167 ceph-mon[32751]: slow request 720.245605 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:34:02.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:01 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39522.597306 secs 2024-04-27T05:34:02.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:01 smithi090 ceph-mon[29180]: slow request 720.245605 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:34:03.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39523.506640 secs 2024-04-27T05:34:03.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:02 smithi119 ceph-mon[32752]: pgmap v21567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:03.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39523.506640 secs 2024-04-27T05:34:03.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:02 smithi167 ceph-mon[32751]: pgmap v21567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:03.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39523.506640 secs 2024-04-27T05:34:03.124 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:02 smithi090 ceph-mon[29180]: pgmap v21567: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:05.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:04 smithi119 ceph-mon[32752]: pgmap v21568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:05.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:04 smithi167 ceph-mon[32751]: pgmap v21568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:05.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:04 smithi090 ceph-mon[29180]: pgmap v21568: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:07.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39527.597385 secs 2024-04-27T05:34:07.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:06 smithi119 ceph-mon[32752]: pgmap v21569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:07.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39527.597385 secs 2024-04-27T05:34:07.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:06 smithi167 ceph-mon[32751]: pgmap v21569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:07.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39527.597385 secs 2024-04-27T05:34:07.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:06 smithi090 ceph-mon[29180]: pgmap v21569: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:08.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39528.506718 secs 2024-04-27T05:34:08.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39528.506718 secs 2024-04-27T05:34:08.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39528.506718 secs 2024-04-27T05:34:09.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:08 smithi119 ceph-mon[32752]: pgmap v21570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:09.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:08 smithi167 ceph-mon[32751]: pgmap v21570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:09.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:08 smithi090 ceph-mon[29180]: pgmap v21570: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:11.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:10 smithi119 ceph-mon[32752]: pgmap v21571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:11.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:10 smithi167 ceph-mon[32751]: pgmap v21571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:11.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:10 smithi090 ceph-mon[29180]: pgmap v21571: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:12.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39532.597466 secs 2024-04-27T05:34:12.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39532.597466 secs 2024-04-27T05:34:12.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39532.597466 secs 2024-04-27T05:34:12.124 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:34:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:34:11.747+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:34:12.124 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:34:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:34:11.747+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:34:12.124 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:34:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:34:11.747+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:34:12.124 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:34:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:34:11.747+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:34:12.124 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:34:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:34:11.747+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:34:13.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39533.506810 secs 2024-04-27T05:34:13.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:12 smithi119 ceph-mon[32752]: pgmap v21572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:13.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39533.506810 secs 2024-04-27T05:34:13.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:12 smithi167 ceph-mon[32751]: pgmap v21572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:13.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39533.506810 secs 2024-04-27T05:34:13.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:12 smithi090 ceph-mon[29180]: pgmap v21572: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:15.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:14 smithi119 ceph-mon[32752]: pgmap v21573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:15.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:14 smithi167 ceph-mon[32751]: pgmap v21573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:15.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:14 smithi090 ceph-mon[29180]: pgmap v21573: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:17.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39537.597550 secs 2024-04-27T05:34:17.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:16 smithi119 ceph-mon[32752]: pgmap v21574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:17.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39537.597550 secs 2024-04-27T05:34:17.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:16 smithi167 ceph-mon[32751]: pgmap v21574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:17.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39537.597550 secs 2024-04-27T05:34:17.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:16 smithi090 ceph-mon[29180]: pgmap v21574: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:18.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39538.506924 secs 2024-04-27T05:34:18.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:17 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:34:18.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39538.506924 secs 2024-04-27T05:34:18.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:17 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:34:18.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39538.506924 secs 2024-04-27T05:34:18.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:17 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:34:19.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:18 smithi119 ceph-mon[32752]: pgmap v21575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:19.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:34:19.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:34:19.052 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:34:19.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:18 smithi167 ceph-mon[32751]: pgmap v21575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:19.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:34:19.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:34:19.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:34:19.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:18 smithi090 ceph-mon[29180]: pgmap v21575: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:19.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:34:19.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:34:19.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:34:20.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:19 smithi119 ceph-mon[32752]: pgmap v21576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:34:20.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:19 smithi119 ceph-mon[32752]: pgmap v21577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:20.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:19 smithi167 ceph-mon[32751]: pgmap v21576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:34:20.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:19 smithi167 ceph-mon[32751]: pgmap v21577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:20.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:19 smithi090 ceph-mon[29180]: pgmap v21576: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:34:20.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:19 smithi090 ceph-mon[29180]: pgmap v21577: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:22.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:21 smithi119 ceph-mon[32752]: pgmap v21578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:22.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39542.597634 secs 2024-04-27T05:34:22.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:21 smithi167 ceph-mon[32751]: pgmap v21578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:22.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39542.597634 secs 2024-04-27T05:34:22.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:21 smithi090 ceph-mon[29180]: pgmap v21578: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:22.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39542.597634 secs 2024-04-27T05:34:22.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39543.507064 secs 2024-04-27T05:34:23.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39543.507064 secs 2024-04-27T05:34:23.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39543.507064 secs 2024-04-27T05:34:24.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:23 smithi119 ceph-mon[32752]: pgmap v21579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:34:24.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:23 smithi167 ceph-mon[32751]: pgmap v21579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:34:24.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:23 smithi090 ceph-mon[29180]: pgmap v21579: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:34:26.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:25 smithi119 ceph-mon[32752]: pgmap v21580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:26.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:25 smithi167 ceph-mon[32751]: pgmap v21580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:26.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:25 smithi090 ceph-mon[29180]: pgmap v21580: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:27.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39547.597714 secs 2024-04-27T05:34:27.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39547.597714 secs 2024-04-27T05:34:27.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39547.597714 secs 2024-04-27T05:34:28.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39548.507203 secs 2024-04-27T05:34:28.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:27 smithi119 ceph-mon[32752]: pgmap v21581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:28.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39548.507203 secs 2024-04-27T05:34:28.066 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:27 smithi167 ceph-mon[32751]: pgmap v21581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:28.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39548.507203 secs 2024-04-27T05:34:28.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:27 smithi090 ceph-mon[29180]: pgmap v21581: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:34:30.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:29 smithi119 ceph-mon[32752]: pgmap v21582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:34:30.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:29 smithi167 ceph-mon[32751]: pgmap v21582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:34:30.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:29 smithi090 ceph-mon[29180]: pgmap v21582: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:34:32.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:31 smithi119 ceph-mon[32752]: pgmap v21583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:32.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39552.597800 secs 2024-04-27T05:34:32.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:31 smithi167 ceph-mon[32751]: pgmap v21583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:32.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39552.597800 secs 2024-04-27T05:34:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:31 smithi090 ceph-mon[29180]: pgmap v21583: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:32.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39552.597800 secs 2024-04-27T05:34:33.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39553.507324 secs 2024-04-27T05:34:33.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39553.507324 secs 2024-04-27T05:34:33.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39553.507324 secs 2024-04-27T05:34:34.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:33 smithi119 ceph-mon[32752]: pgmap v21584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:34.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:33 smithi167 ceph-mon[32751]: pgmap v21584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:34.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:33 smithi090 ceph-mon[29180]: pgmap v21584: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:36.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:35 smithi119 ceph-mon[32752]: pgmap v21585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:36.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:35 smithi167 ceph-mon[32751]: pgmap v21585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:36.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:35 smithi090 ceph-mon[29180]: pgmap v21585: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:37.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39557.597860 secs 2024-04-27T05:34:37.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39557.597860 secs 2024-04-27T05:34:37.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39557.597860 secs 2024-04-27T05:34:38.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39558.507468 secs 2024-04-27T05:34:38.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:37 smithi119 ceph-mon[32752]: pgmap v21586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:38.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39558.507468 secs 2024-04-27T05:34:38.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:37 smithi167 ceph-mon[32751]: pgmap v21586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:38.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39558.507468 secs 2024-04-27T05:34:38.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:37 smithi090 ceph-mon[29180]: pgmap v21586: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:40.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:39 smithi119 ceph-mon[32752]: pgmap v21587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:40.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:39 smithi167 ceph-mon[32751]: pgmap v21587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:40.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:39 smithi090 ceph-mon[29180]: pgmap v21587: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:42.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:41 smithi119 ceph-mon[32752]: pgmap v21588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:42.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39562.597922 secs 2024-04-27T05:34:42.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:41 smithi167 ceph-mon[32751]: pgmap v21588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:42.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39562.597922 secs 2024-04-27T05:34:42.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:41 smithi090 ceph-mon[29180]: pgmap v21588: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:42.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39562.597922 secs 2024-04-27T05:34:43.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 39563.507591 secs 2024-04-27T05:34:43.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:42 smithi119 ceph-mon[32752]: slow request 2880.278891 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:34:43.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 39563.507591 secs 2024-04-27T05:34:43.065 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:42 smithi167 ceph-mon[32751]: slow request 2880.278891 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:34:43.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 39563.507591 secs 2024-04-27T05:34:43.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:42 smithi090 ceph-mon[29180]: slow request 2880.278891 seconds old, received at 2024-04-27T04:46:40.979345+0000: client_request(client.17802:1 getattr pAsLsXsFs #0x1 2024-04-27T04:46:40.975187+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:34:44.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:43 smithi119 ceph-mon[32752]: pgmap v21589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:44.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:43 smithi167 ceph-mon[32751]: pgmap v21589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:44.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:43 smithi090 ceph-mon[29180]: pgmap v21589: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:46.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:45 smithi119 ceph-mon[32752]: pgmap v21590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:46.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:45 smithi167 ceph-mon[32751]: pgmap v21590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:46.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:45 smithi090 ceph-mon[29180]: pgmap v21590: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:47.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39567.597984 secs 2024-04-27T05:34:47.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39567.597984 secs 2024-04-27T05:34:47.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39567.597984 secs 2024-04-27T05:34:48.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39568.507715 secs 2024-04-27T05:34:48.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:47 smithi119 ceph-mon[32752]: pgmap v21591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:48.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39568.507715 secs 2024-04-27T05:34:48.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:47 smithi167 ceph-mon[32751]: pgmap v21591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:48.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39568.507715 secs 2024-04-27T05:34:48.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:47 smithi090 ceph-mon[29180]: pgmap v21591: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:50.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:49 smithi119 ceph-mon[32752]: pgmap v21592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:50.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:49 smithi167 ceph-mon[32751]: pgmap v21592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:50.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:49 smithi090 ceph-mon[29180]: pgmap v21592: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:52.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:51 smithi119 ceph-mon[32752]: pgmap v21593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:52.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39572.598065 secs 2024-04-27T05:34:52.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:51 smithi167 ceph-mon[32751]: pgmap v21593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:52.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39572.598065 secs 2024-04-27T05:34:52.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:51 smithi090 ceph-mon[29180]: pgmap v21593: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:52.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39572.598065 secs 2024-04-27T05:34:53.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39573.507792 secs 2024-04-27T05:34:53.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39573.507792 secs 2024-04-27T05:34:53.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39573.507792 secs 2024-04-27T05:34:54.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:53 smithi119 ceph-mon[32752]: pgmap v21594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:54.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:53 smithi167 ceph-mon[32751]: pgmap v21594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:54.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:53 smithi090 ceph-mon[29180]: pgmap v21594: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:56.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:55 smithi119 ceph-mon[32752]: pgmap v21595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:56.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:55 smithi167 ceph-mon[32751]: pgmap v21595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:56.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:55 smithi090 ceph-mon[29180]: pgmap v21595: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:34:57.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39577.598123 secs 2024-04-27T05:34:57.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39577.598123 secs 2024-04-27T05:34:57.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39577.598123 secs 2024-04-27T05:34:58.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39578.507908 secs 2024-04-27T05:34:58.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:57 smithi119 ceph-mon[32752]: pgmap v21596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:58.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39578.507908 secs 2024-04-27T05:34:58.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:57 smithi167 ceph-mon[32751]: pgmap v21596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:34:58.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39578.507908 secs 2024-04-27T05:34:58.123 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:57 smithi090 ceph-mon[29180]: pgmap v21596: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:00.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:34:59 smithi119 ceph-mon[32752]: pgmap v21597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:00.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:34:59 smithi167 ceph-mon[32751]: pgmap v21597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:00.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:34:59 smithi090 ceph-mon[29180]: pgmap v21597: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:02.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:01 smithi119 ceph-mon[32752]: pgmap v21598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:02.051 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39582.598179 secs 2024-04-27T05:35:02.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:01 smithi167 ceph-mon[32751]: pgmap v21598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:02.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39582.598179 secs 2024-04-27T05:35:02.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:01 smithi090 ceph-mon[29180]: pgmap v21598: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:02.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39582.598179 secs 2024-04-27T05:35:03.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39583.508031 secs 2024-04-27T05:35:03.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39583.508031 secs 2024-04-27T05:35:03.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39583.508031 secs 2024-04-27T05:35:04.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:03 smithi119 ceph-mon[32752]: pgmap v21599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:04.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:03 smithi167 ceph-mon[32751]: pgmap v21599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:04.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:03 smithi090 ceph-mon[29180]: pgmap v21599: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:06.050 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:05 smithi119 ceph-mon[32752]: pgmap v21600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:06.064 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:05 smithi167 ceph-mon[32751]: pgmap v21600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:06.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:05 smithi090 ceph-mon[29180]: pgmap v21600: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:07.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39587.598239 secs 2024-04-27T05:35:07.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39587.598239 secs 2024-04-27T05:35:07.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39587.598239 secs 2024-04-27T05:35:08.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39588.508183 secs 2024-04-27T05:35:08.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:07 smithi090 ceph-mon[29180]: pgmap v21601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:08.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39588.508183 secs 2024-04-27T05:35:08.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:07 smithi119 ceph-mon[32752]: pgmap v21601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:08.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39588.508183 secs 2024-04-27T05:35:08.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:07 smithi167 ceph-mon[32751]: pgmap v21601: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:10.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:09 smithi090 ceph-mon[29180]: pgmap v21602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:10.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:09 smithi119 ceph-mon[32752]: pgmap v21602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:10.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:09 smithi167 ceph-mon[32751]: pgmap v21602: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:12.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:11 smithi090 ceph-mon[29180]: pgmap v21603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:12.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39592.598324 secs 2024-04-27T05:35:12.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:11 smithi119 ceph-mon[32752]: pgmap v21603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:12.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39592.598324 secs 2024-04-27T05:35:12.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:11 smithi167 ceph-mon[32751]: pgmap v21603: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:12.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39592.598324 secs 2024-04-27T05:35:13.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39593.508315 secs 2024-04-27T05:35:13.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39593.508315 secs 2024-04-27T05:35:13.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39593.508315 secs 2024-04-27T05:35:14.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:13 smithi090 ceph-mon[29180]: pgmap v21604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:14.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:13 smithi119 ceph-mon[32752]: pgmap v21604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:14.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:13 smithi167 ceph-mon[32751]: pgmap v21604: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:16.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:15 smithi090 ceph-mon[29180]: pgmap v21605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:16.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:15 smithi119 ceph-mon[32752]: pgmap v21605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:16.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:15 smithi167 ceph-mon[32751]: pgmap v21605: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:17.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39597.598409 secs 2024-04-27T05:35:17.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39597.598409 secs 2024-04-27T05:35:17.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39597.598409 secs 2024-04-27T05:35:18.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39598.508410 secs 2024-04-27T05:35:18.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:17 smithi090 ceph-mon[29180]: pgmap v21606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:18.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39598.508410 secs 2024-04-27T05:35:18.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:17 smithi119 ceph-mon[32752]: pgmap v21606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:18.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39598.508410 secs 2024-04-27T05:35:18.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:17 smithi167 ceph-mon[32751]: pgmap v21606: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:19.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:18 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:35:19.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:18 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:35:19.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:18 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:35:20.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:19 smithi090 ceph-mon[29180]: pgmap v21607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:20.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:35:20.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:35:20.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:35:20.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:19 smithi119 ceph-mon[32752]: pgmap v21607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:20.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:35:20.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:35:20.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:35:20.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:19 smithi167 ceph-mon[32751]: pgmap v21607: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:20.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:35:20.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:35:20.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:35:21.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:20 smithi090 ceph-mon[29180]: pgmap v21608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:35:21.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:20 smithi090 ceph-mon[29180]: pgmap v21609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:21.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:20 smithi119 ceph-mon[32752]: pgmap v21608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:35:21.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:20 smithi119 ceph-mon[32752]: pgmap v21609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:21.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:20 smithi167 ceph-mon[32751]: pgmap v21608: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:35:21.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:20 smithi167 ceph-mon[32751]: pgmap v21609: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:22.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39602.598488 secs 2024-04-27T05:35:22.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39602.598488 secs 2024-04-27T05:35:22.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39602.598488 secs 2024-04-27T05:35:23.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:22 smithi090 ceph-mon[29180]: pgmap v21610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:23.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39603.508530 secs 2024-04-27T05:35:23.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:22 smithi119 ceph-mon[32752]: pgmap v21610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:23.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39603.508530 secs 2024-04-27T05:35:23.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:22 smithi167 ceph-mon[32751]: pgmap v21610: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:23.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39603.508530 secs 2024-04-27T05:35:25.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:24 smithi090 ceph-mon[29180]: pgmap v21611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:35:25.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:24 smithi119 ceph-mon[32752]: pgmap v21611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:35:25.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:24 smithi167 ceph-mon[32751]: pgmap v21611: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:35:27.121 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:26 smithi090 ceph-mon[29180]: pgmap v21612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:27.122 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39607.598572 secs 2024-04-27T05:35:27.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:26 smithi119 ceph-mon[32752]: pgmap v21612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:27.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39607.598572 secs 2024-04-27T05:35:27.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:26 smithi167 ceph-mon[32751]: pgmap v21612: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:35:27.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39607.598572 secs 2024-04-27T05:35:28.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39608.508658 secs 2024-04-27T05:35:28.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39608.508658 secs 2024-04-27T05:35:28.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39608.508658 secs 2024-04-27T05:35:29.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:28 smithi119 ceph-mon[32752]: pgmap v21613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:35:29.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:28 smithi167 ceph-mon[32751]: pgmap v21613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:35:29.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:28 smithi090 ceph-mon[29180]: pgmap v21613: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:35:31.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:30 smithi119 ceph-mon[32752]: pgmap v21614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:35:31.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:30 smithi167 ceph-mon[32751]: pgmap v21614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:35:31.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:30 smithi090 ceph-mon[29180]: pgmap v21614: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:35:32.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39612.598655 secs 2024-04-27T05:35:32.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39612.598655 secs 2024-04-27T05:35:32.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39612.598655 secs 2024-04-27T05:35:33.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:32 smithi119 ceph-mon[32752]: pgmap v21615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:33.300 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39613.508756 secs 2024-04-27T05:35:33.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:32 smithi167 ceph-mon[32751]: pgmap v21615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:33.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39613.508756 secs 2024-04-27T05:35:33.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:32 smithi090 ceph-mon[29180]: pgmap v21615: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:33.372 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39613.508756 secs 2024-04-27T05:35:34.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:33 smithi119 ceph-mon[32752]: pgmap v21616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:34.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:33 smithi167 ceph-mon[32751]: pgmap v21616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:34.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:33 smithi090 ceph-mon[29180]: pgmap v21616: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:36.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:36 smithi119 ceph-mon[32752]: pgmap v21617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:36.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:36 smithi167 ceph-mon[32751]: pgmap v21617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:36.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:36 smithi090 ceph-mon[29180]: pgmap v21617: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:37.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:37 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39617.598744 secs 2024-04-27T05:35:37.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:37 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39617.598744 secs 2024-04-27T05:35:37.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:37 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39617.598744 secs 2024-04-27T05:35:38.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:38 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39618.508890 secs 2024-04-27T05:35:38.550 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:38 smithi119 ceph-mon[32752]: pgmap v21618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:38.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:38 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39618.508890 secs 2024-04-27T05:35:38.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:38 smithi167 ceph-mon[32751]: pgmap v21618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:38.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:38 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39618.508890 secs 2024-04-27T05:35:38.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:38 smithi090 ceph-mon[29180]: pgmap v21618: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:40.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:40 smithi119 ceph-mon[32752]: pgmap v21619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:40.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:40 smithi167 ceph-mon[32751]: pgmap v21619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:40.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:40 smithi090 ceph-mon[29180]: pgmap v21619: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:42.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:42 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39622.598828 secs 2024-04-27T05:35:42.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:42 smithi119 ceph-mon[32752]: pgmap v21620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:42.560 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:42 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39622.598828 secs 2024-04-27T05:35:42.561 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:42 smithi090 ceph-mon[29180]: pgmap v21620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:42.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:42 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39622.598828 secs 2024-04-27T05:35:42.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:42 smithi167 ceph-mon[32751]: pgmap v21620: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:43.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:43 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39623.509018 secs 2024-04-27T05:35:43.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:43 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39623.509018 secs 2024-04-27T05:35:43.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:43 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39623.509018 secs 2024-04-27T05:35:44.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:44 smithi119 ceph-mon[32752]: pgmap v21621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:44.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:44 smithi167 ceph-mon[32751]: pgmap v21621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:44.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:44 smithi090 ceph-mon[29180]: pgmap v21621: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:46.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:46 smithi119 ceph-mon[32752]: pgmap v21622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:46.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:46 smithi167 ceph-mon[32751]: pgmap v21622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:46.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:46 smithi090 ceph-mon[29180]: pgmap v21622: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:47.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:47 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39627.598917 secs 2024-04-27T05:35:47.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:47 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39627.598917 secs 2024-04-27T05:35:47.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:47 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39627.598917 secs 2024-04-27T05:35:48.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:48 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39628.509143 secs 2024-04-27T05:35:48.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:48 smithi119 ceph-mon[32752]: pgmap v21623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:48.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:48 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39628.509143 secs 2024-04-27T05:35:48.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:48 smithi167 ceph-mon[32751]: pgmap v21623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:48.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:48 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39628.509143 secs 2024-04-27T05:35:48.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:48 smithi090 ceph-mon[29180]: pgmap v21623: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:50.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:50 smithi119 ceph-mon[32752]: pgmap v21624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:50.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:50 smithi167 ceph-mon[32751]: pgmap v21624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:50.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:50 smithi090 ceph-mon[29180]: pgmap v21624: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:52.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:52 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39632.598980 secs 2024-04-27T05:35:52.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:52 smithi119 ceph-mon[32752]: pgmap v21625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:52.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:52 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39632.598980 secs 2024-04-27T05:35:52.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:52 smithi167 ceph-mon[32751]: pgmap v21625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:52.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:52 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39632.598980 secs 2024-04-27T05:35:52.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:52 smithi090 ceph-mon[29180]: pgmap v21625: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:53.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:53 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39633.509302 secs 2024-04-27T05:35:53.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:53 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39633.509302 secs 2024-04-27T05:35:53.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:53 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39633.509302 secs 2024-04-27T05:35:54.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:54 smithi119 ceph-mon[32752]: pgmap v21626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:54.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:54 smithi167 ceph-mon[32751]: pgmap v21626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:54.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:54 smithi090 ceph-mon[29180]: pgmap v21626: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:56.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:56 smithi119 ceph-mon[32752]: pgmap v21627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:56.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:56 smithi167 ceph-mon[32751]: pgmap v21627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:56.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:56 smithi090 ceph-mon[29180]: pgmap v21627: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:35:57.449 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:57 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39637.599086 secs 2024-04-27T05:35:57.523 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:57 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39637.599086 secs 2024-04-27T05:35:57.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:57 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39637.599086 secs 2024-04-27T05:35:58.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:58 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39638.509415 secs 2024-04-27T05:35:58.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:35:58 smithi119 ceph-mon[32752]: pgmap v21628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:58.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:58 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39638.509415 secs 2024-04-27T05:35:58.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:35:58 smithi167 ceph-mon[32751]: pgmap v21628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:35:58.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:58 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39638.509415 secs 2024-04-27T05:35:58.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:35:58 smithi090 ceph-mon[29180]: pgmap v21628: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:00.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:00 smithi119 ceph-mon[32752]: pgmap v21629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:00.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:00 smithi167 ceph-mon[32751]: pgmap v21629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:00.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:00 smithi090 ceph-mon[29180]: pgmap v21629: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:02.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:02 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39642.599175 secs 2024-04-27T05:36:02.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:02 smithi119 ceph-mon[32752]: pgmap v21630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:02.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:02 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39642.599175 secs 2024-04-27T05:36:02.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:02 smithi167 ceph-mon[32751]: pgmap v21630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:02.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:02 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39642.599175 secs 2024-04-27T05:36:02.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:02 smithi090 ceph-mon[29180]: pgmap v21630: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:03.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:03 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39643.509571 secs 2024-04-27T05:36:03.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:03 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39643.509571 secs 2024-04-27T05:36:03.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:03 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39643.509571 secs 2024-04-27T05:36:04.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:04 smithi119 ceph-mon[32752]: pgmap v21631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:04.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:04 smithi167 ceph-mon[32751]: pgmap v21631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:04.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:04 smithi090 ceph-mon[29180]: pgmap v21631: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:06.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:06 smithi119 ceph-mon[32752]: pgmap v21632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:06.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39647.599275 secs 2024-04-27T05:36:06.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:06 smithi167 ceph-mon[32751]: pgmap v21632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:06.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39647.599275 secs 2024-04-27T05:36:06.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:06 smithi090 ceph-mon[29180]: pgmap v21632: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:06.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39647.599275 secs 2024-04-27T05:36:08.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:08 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39648.509635 secs 2024-04-27T05:36:08.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:08 smithi119 ceph-mon[32752]: pgmap v21633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:08.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:08 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39648.509635 secs 2024-04-27T05:36:08.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:08 smithi167 ceph-mon[32751]: pgmap v21633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:08.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:08 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39648.509635 secs 2024-04-27T05:36:08.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:08 smithi090 ceph-mon[29180]: pgmap v21633: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:10.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:10 smithi119 ceph-mon[32752]: pgmap v21634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:10.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:10 smithi167 ceph-mon[32751]: pgmap v21634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:10.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:10 smithi090 ceph-mon[29180]: pgmap v21634: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:11.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39652.599364 secs 2024-04-27T05:36:11.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39652.599364 secs 2024-04-27T05:36:11.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39652.599364 secs 2024-04-27T05:36:12.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:12 smithi119 ceph-mon[32752]: pgmap v21635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:12.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:12 smithi167 ceph-mon[32751]: pgmap v21635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:12.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:12 smithi090 ceph-mon[29180]: pgmap v21635: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:13.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:13 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39653.509763 secs 2024-04-27T05:36:13.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:13 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39653.509763 secs 2024-04-27T05:36:13.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:13 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39653.509763 secs 2024-04-27T05:36:14.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:14 smithi119 ceph-mon[32752]: pgmap v21636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:14.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:14 smithi167 ceph-mon[32751]: pgmap v21636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:14.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:14 smithi090 ceph-mon[29180]: pgmap v21636: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:16.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:16 smithi119 ceph-mon[32752]: pgmap v21637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:16.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39657.599452 secs 2024-04-27T05:36:16.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:16 smithi167 ceph-mon[32751]: pgmap v21637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:16.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39657.599452 secs 2024-04-27T05:36:16.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:16 smithi090 ceph-mon[29180]: pgmap v21637: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:16.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39657.599452 secs 2024-04-27T05:36:18.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:18 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39658.509835 secs 2024-04-27T05:36:18.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:18 smithi119 ceph-mon[32752]: pgmap v21638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:18.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:18 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39658.509835 secs 2024-04-27T05:36:18.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:18 smithi167 ceph-mon[32751]: pgmap v21638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:18.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:18 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39658.509835 secs 2024-04-27T05:36:18.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:18 smithi090 ceph-mon[29180]: pgmap v21638: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:19.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:36:19.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:36:19.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:36:20.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:20 smithi119 ceph-mon[32752]: pgmap v21639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:20.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:36:20.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:36:20.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:20 smithi119 ceph-mon[32752]: pgmap v21640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:36:20.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:20 smithi119 ceph-mon[32752]: pgmap v21641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:20.549 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:20 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:36:20.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:20 smithi167 ceph-mon[32751]: pgmap v21639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:20.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:36:20.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:36:20.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:20 smithi167 ceph-mon[32751]: pgmap v21640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:36:20.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:20 smithi167 ceph-mon[32751]: pgmap v21641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:20.563 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:20 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:36:20.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:20 smithi090 ceph-mon[29180]: pgmap v21639: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:20.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:36:20.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:36:20.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:20 smithi090 ceph-mon[29180]: pgmap v21640: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:36:20.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:20 smithi090 ceph-mon[29180]: pgmap v21641: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:20.622 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:20 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:36:21.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39662.599541 secs 2024-04-27T05:36:21.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39662.599541 secs 2024-04-27T05:36:21.621 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39662.599541 secs 2024-04-27T05:36:22.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:22 smithi119 ceph-mon[32752]: pgmap v21642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:22.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:22 smithi167 ceph-mon[32751]: pgmap v21642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:22.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:22 smithi090 ceph-mon[29180]: pgmap v21642: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:23.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:23 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39663.509909 secs 2024-04-27T05:36:23.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:23 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39663.509909 secs 2024-04-27T05:36:23.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:23 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39663.509909 secs 2024-04-27T05:36:24.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:24 smithi119 ceph-mon[32752]: pgmap v21643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:24.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:24 smithi167 ceph-mon[32751]: pgmap v21643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:24.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:24 smithi090 ceph-mon[29180]: pgmap v21643: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:26.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:25 smithi119 ceph-mon[32752]: pgmap v21644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:26.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:25 smithi167 ceph-mon[32751]: pgmap v21644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:26.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:25 smithi090 ceph-mon[29180]: pgmap v21644: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:36:27.200 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39667.599628 secs 2024-04-27T05:36:27.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39667.599628 secs 2024-04-27T05:36:27.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39667.599628 secs 2024-04-27T05:36:28.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:27 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39668.510026 secs 2024-04-27T05:36:28.299 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:27 smithi119 ceph-mon[32752]: pgmap v21645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:36:28.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:27 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39668.510026 secs 2024-04-27T05:36:28.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:27 smithi167 ceph-mon[32751]: pgmap v21645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:36:28.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:27 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39668.510026 secs 2024-04-27T05:36:28.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:27 smithi090 ceph-mon[29180]: pgmap v21645: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:36:30.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:29 smithi119 ceph-mon[32752]: pgmap v21646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:36:30.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:29 smithi167 ceph-mon[32751]: pgmap v21646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:36:30.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:29 smithi090 ceph-mon[29180]: pgmap v21646: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:36:31.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39672.599717 secs 2024-04-27T05:36:31.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39672.599717 secs 2024-04-27T05:36:31.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39672.599717 secs 2024-04-27T05:36:32.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39673.510121 secs 2024-04-27T05:36:32.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:32 smithi119 ceph-mon[32752]: pgmap v21647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:32.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39673.510121 secs 2024-04-27T05:36:32.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:32 smithi167 ceph-mon[32751]: pgmap v21647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:32.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39673.510121 secs 2024-04-27T05:36:32.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:32 smithi090 ceph-mon[29180]: pgmap v21647: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:34.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:33 smithi119 ceph-mon[32752]: pgmap v21648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:34.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:33 smithi167 ceph-mon[32751]: pgmap v21648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:34.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:33 smithi090 ceph-mon[29180]: pgmap v21648: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:36.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:35 smithi119 ceph-mon[32752]: pgmap v21649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:36.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:35 smithi167 ceph-mon[32751]: pgmap v21649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:36.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:35 smithi090 ceph-mon[29180]: pgmap v21649: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:37.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39677.599799 secs 2024-04-27T05:36:37.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39677.599799 secs 2024-04-27T05:36:37.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39677.599799 secs 2024-04-27T05:36:38.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39678.510239 secs 2024-04-27T05:36:38.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:37 smithi119 ceph-mon[32752]: pgmap v21650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:38.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39678.510239 secs 2024-04-27T05:36:38.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:37 smithi167 ceph-mon[32751]: pgmap v21650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:38.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39678.510239 secs 2024-04-27T05:36:38.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:37 smithi090 ceph-mon[29180]: pgmap v21650: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:40.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:39 smithi119 ceph-mon[32752]: pgmap v21651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:40.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:39 smithi167 ceph-mon[32751]: pgmap v21651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:40.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:39 smithi090 ceph-mon[29180]: pgmap v21651: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:41.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39682.599852 secs 2024-04-27T05:36:41.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39682.599852 secs 2024-04-27T05:36:41.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39682.599852 secs 2024-04-27T05:36:42.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39683.510304 secs 2024-04-27T05:36:42.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:42 smithi119 ceph-mon[32752]: pgmap v21652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:42.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39683.510304 secs 2024-04-27T05:36:42.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:42 smithi167 ceph-mon[32751]: pgmap v21652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:42.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39683.510304 secs 2024-04-27T05:36:42.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:42 smithi090 ceph-mon[29180]: pgmap v21652: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:44.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:43 smithi119 ceph-mon[32752]: pgmap v21653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:44.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:43 smithi167 ceph-mon[32751]: pgmap v21653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:44.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:43 smithi090 ceph-mon[29180]: pgmap v21653: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:46.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:45 smithi119 ceph-mon[32752]: pgmap v21654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:46.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:45 smithi167 ceph-mon[32751]: pgmap v21654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:46.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:45 smithi090 ceph-mon[29180]: pgmap v21654: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:47.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39687.599917 secs 2024-04-27T05:36:47.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39687.599917 secs 2024-04-27T05:36:47.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39687.599917 secs 2024-04-27T05:36:48.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39688.510404 secs 2024-04-27T05:36:48.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:47 smithi119 ceph-mon[32752]: pgmap v21655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:48.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39688.510404 secs 2024-04-27T05:36:48.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:47 smithi167 ceph-mon[32751]: pgmap v21655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:48.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39688.510404 secs 2024-04-27T05:36:48.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:47 smithi090 ceph-mon[29180]: pgmap v21655: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:50.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:49 smithi119 ceph-mon[32752]: pgmap v21656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:50.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:49 smithi167 ceph-mon[32751]: pgmap v21656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:50.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:49 smithi090 ceph-mon[29180]: pgmap v21656: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:51.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39692.600004 secs 2024-04-27T05:36:51.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39692.600004 secs 2024-04-27T05:36:51.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39692.600004 secs 2024-04-27T05:36:52.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39693.510469 secs 2024-04-27T05:36:52.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:52 smithi119 ceph-mon[32752]: pgmap v21657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:52.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39693.510469 secs 2024-04-27T05:36:52.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:52 smithi167 ceph-mon[32751]: pgmap v21657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:52.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39693.510469 secs 2024-04-27T05:36:52.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:52 smithi090 ceph-mon[29180]: pgmap v21657: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:54.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:53 smithi119 ceph-mon[32752]: pgmap v21658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:54.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:53 smithi167 ceph-mon[32751]: pgmap v21658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:54.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:53 smithi090 ceph-mon[29180]: pgmap v21658: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:36:56.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:55 smithi119 ceph-mon[32752]: pgmap v21659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:56.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:55 smithi167 ceph-mon[32751]: pgmap v21659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:56.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:55 smithi090 ceph-mon[29180]: pgmap v21659: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:57.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39697.600066 secs 2024-04-27T05:36:57.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39697.600066 secs 2024-04-27T05:36:57.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39697.600066 secs 2024-04-27T05:36:58.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39698.510557 secs 2024-04-27T05:36:58.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:57 smithi119 ceph-mon[32752]: pgmap v21660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:58.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39698.510557 secs 2024-04-27T05:36:58.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:57 smithi167 ceph-mon[32751]: pgmap v21660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:36:58.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39698.510557 secs 2024-04-27T05:36:58.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:57 smithi090 ceph-mon[29180]: pgmap v21660: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:00.172 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:36:59 smithi090 ceph-mon[29180]: pgmap v21661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:00.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:36:59 smithi119 ceph-mon[32752]: pgmap v21661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:00.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:36:59 smithi167 ceph-mon[32751]: pgmap v21661: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:01.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39702.600127 secs 2024-04-27T05:37:01.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39702.600127 secs 2024-04-27T05:37:01.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39702.600127 secs 2024-04-27T05:37:02.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39703.510620 secs 2024-04-27T05:37:02.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:02 smithi119 ceph-mon[32752]: pgmap v21662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:02.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39703.510620 secs 2024-04-27T05:37:02.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:02 smithi167 ceph-mon[32751]: pgmap v21662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:02.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39703.510620 secs 2024-04-27T05:37:02.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:02 smithi090 ceph-mon[29180]: pgmap v21662: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:04.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:03 smithi119 ceph-mon[32752]: pgmap v21663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:04.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:03 smithi167 ceph-mon[32751]: pgmap v21663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:04.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:03 smithi090 ceph-mon[29180]: pgmap v21663: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:06.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:05 smithi119 ceph-mon[32752]: pgmap v21664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:06.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:05 smithi167 ceph-mon[32751]: pgmap v21664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:06.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:05 smithi090 ceph-mon[29180]: pgmap v21664: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:07.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:06 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39707.600186 secs 2024-04-27T05:37:07.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:06 smithi119 ceph-mon[32752]: slow request 23040.038814 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:37:07.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:06 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39707.600186 secs 2024-04-27T05:37:07.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:06 smithi167 ceph-mon[32751]: slow request 23040.038814 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:37:07.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:06 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39707.600186 secs 2024-04-27T05:37:07.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:06 smithi090 ceph-mon[29180]: slow request 23040.038814 seconds old, received at 2024-04-26T23:13:05.314776+0000: client_request(client.26738:1 getattr pAsLsXsFs #0x1 2024-04-26T23:13:05.307716+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:37:08.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39708.510736 secs 2024-04-27T05:37:08.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:07 smithi119 ceph-mon[32752]: pgmap v21665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:08.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39708.510736 secs 2024-04-27T05:37:08.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:07 smithi167 ceph-mon[32751]: pgmap v21665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:08.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39708.510736 secs 2024-04-27T05:37:08.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:07 smithi090 ceph-mon[29180]: pgmap v21665: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:10.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:09 smithi119 ceph-mon[32752]: pgmap v21666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:10.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:09 smithi167 ceph-mon[32751]: pgmap v21666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:10.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:09 smithi090 ceph-mon[29180]: pgmap v21666: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:11.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39712.600269 secs 2024-04-27T05:37:11.561 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39712.600269 secs 2024-04-27T05:37:11.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39712.600269 secs 2024-04-27T05:37:12.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39713.510834 secs 2024-04-27T05:37:12.548 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:12 smithi119 ceph-mon[32752]: pgmap v21667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:12.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39713.510834 secs 2024-04-27T05:37:12.562 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:12 smithi167 ceph-mon[32751]: pgmap v21667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:12.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39713.510834 secs 2024-04-27T05:37:12.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:12 smithi090 ceph-mon[29180]: pgmap v21667: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:14.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:13 smithi119 ceph-mon[32752]: pgmap v21668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:14.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:13 smithi167 ceph-mon[32751]: pgmap v21668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:14.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:13 smithi090 ceph-mon[29180]: pgmap v21668: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:16.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:15 smithi090 ceph-mon[29180]: pgmap v21669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:16.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:15 smithi119 ceph-mon[32752]: pgmap v21669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:16.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:15 smithi167 ceph-mon[32751]: pgmap v21669: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:17.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39717.600322 secs 2024-04-27T05:37:17.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39717.600322 secs 2024-04-27T05:37:17.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39717.600322 secs 2024-04-27T05:37:18.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39718.510894 secs 2024-04-27T05:37:18.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:17 smithi119 ceph-mon[32752]: pgmap v21670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:18.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39718.510894 secs 2024-04-27T05:37:18.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:17 smithi167 ceph-mon[32751]: pgmap v21670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:18.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39718.510894 secs 2024-04-27T05:37:18.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:17 smithi090 ceph-mon[29180]: pgmap v21670: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:20.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:19 smithi119 ceph-mon[32752]: pgmap v21671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:20.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:19 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:37:20.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:19 smithi167 ceph-mon[32751]: pgmap v21671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:20.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:19 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:37:20.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:19 smithi090 ceph-mon[29180]: pgmap v21671: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:20.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:19 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:37:21.619 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39722.600416 secs 2024-04-27T05:37:21.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:21 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.620 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39723.510962 secs 2024-04-27T05:37:21.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39722.600416 secs 2024-04-27T05:37:21.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:21 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39723.510962 secs 2024-04-27T05:37:21.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39722.600416 secs 2024-04-27T05:37:21.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:21 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:21.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39723.510962 secs 2024-04-27T05:37:22.797 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:22 smithi119 ceph-mon[32752]: pgmap v21672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:22.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:22.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:22 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:22.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:22 smithi167 ceph-mon[32751]: pgmap v21672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:22.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:22.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:22 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:22.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:22 smithi090 ceph-mon[29180]: pgmap v21672: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:22.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:22.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:22 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.799 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:37:23.800 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.811 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:37:23.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:37:23.812 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.813 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:37:23.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:37:23.814 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:37:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:37:23.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.871 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:37:23.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:37:23.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:37:23.872 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:37:25.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:24 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:37:25.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:24 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:37:25.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:24 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:37:25.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:24 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:37:25.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:24 smithi119 ceph-mon[32752]: pgmap v21673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:37:25.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:24 smithi119 ceph-mon[32752]: pgmap v21674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:25.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:24 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:37:25.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:24 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:37:25.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:24 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:37:25.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:24 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:37:25.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:24 smithi167 ceph-mon[32751]: pgmap v21673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:37:25.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:24 smithi167 ceph-mon[32751]: pgmap v21674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:25.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:24 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:37:25.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:24 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:37:25.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:24 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:37:25.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:24 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:37:25.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:24 smithi090 ceph-mon[29180]: pgmap v21673: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:37:25.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:24 smithi090 ceph-mon[29180]: pgmap v21674: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:27.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:26 smithi119 ceph-mon[32752]: pgmap v21675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:27.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39727.600501 secs 2024-04-27T05:37:27.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39728.511106 secs 2024-04-27T05:37:27.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:26 smithi167 ceph-mon[32751]: pgmap v21675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:27.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39727.600501 secs 2024-04-27T05:37:27.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39728.511106 secs 2024-04-27T05:37:27.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:26 smithi090 ceph-mon[29180]: pgmap v21675: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:27.147 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39727.600501 secs 2024-04-27T05:37:27.148 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39728.511106 secs 2024-04-27T05:37:29.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:28 smithi119 ceph-mon[32752]: pgmap v21676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:29.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:28 smithi167 ceph-mon[32751]: pgmap v21676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:29.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:28 smithi090 ceph-mon[29180]: pgmap v21676: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:31.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:30 smithi119 ceph-mon[32752]: pgmap v21677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:31.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:30 smithi167 ceph-mon[32751]: pgmap v21677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:31.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:30 smithi090 ceph-mon[29180]: pgmap v21677: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:31.869 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39732.600584 secs 2024-04-27T05:37:31.870 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39733.511222 secs 2024-04-27T05:37:32.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39732.600584 secs 2024-04-27T05:37:32.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39733.511222 secs 2024-04-27T05:37:32.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39732.600584 secs 2024-04-27T05:37:32.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39733.511222 secs 2024-04-27T05:37:33.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:32 smithi119 ceph-mon[32752]: pgmap v21678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:33.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:32 smithi167 ceph-mon[32751]: pgmap v21678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:33.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:32 smithi090 ceph-mon[29180]: pgmap v21678: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:37:35.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:34 smithi119 ceph-mon[32752]: pgmap v21679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:37:35.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:34 smithi167 ceph-mon[32751]: pgmap v21679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:37:35.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:34 smithi090 ceph-mon[29180]: pgmap v21679: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:37:37.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39737.600672 secs 2024-04-27T05:37:37.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:36 smithi119 ceph-mon[32752]: pgmap v21680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:37.048 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39738.511340 secs 2024-04-27T05:37:37.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39737.600672 secs 2024-04-27T05:37:37.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:36 smithi167 ceph-mon[32751]: pgmap v21680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:37.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39738.511340 secs 2024-04-27T05:37:37.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39737.600672 secs 2024-04-27T05:37:37.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:36 smithi090 ceph-mon[29180]: pgmap v21680: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:37.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39738.511340 secs 2024-04-27T05:37:39.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:38 smithi119 ceph-mon[32752]: pgmap v21681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:39.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:38 smithi167 ceph-mon[32751]: pgmap v21681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:39.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:38 smithi090 ceph-mon[29180]: pgmap v21681: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:41.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:40 smithi119 ceph-mon[32752]: pgmap v21682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:41.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:40 smithi167 ceph-mon[32751]: pgmap v21682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:41.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:40 smithi090 ceph-mon[29180]: pgmap v21682: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:42.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39742.600759 secs 2024-04-27T05:37:42.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39743.511458 secs 2024-04-27T05:37:42.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39742.600759 secs 2024-04-27T05:37:42.120 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39743.511458 secs 2024-04-27T05:37:42.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39742.600759 secs 2024-04-27T05:37:42.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39743.511458 secs 2024-04-27T05:37:43.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:42 smithi167 ceph-mon[32751]: pgmap v21683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:43.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:42 smithi090 ceph-mon[29180]: pgmap v21683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:43.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:42 smithi119 ceph-mon[32752]: pgmap v21683: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:45.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:44 smithi167 ceph-mon[32751]: pgmap v21684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:45.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:44 smithi090 ceph-mon[29180]: pgmap v21684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:45.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:44 smithi119 ceph-mon[32752]: pgmap v21684: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:47.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:46 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39747.600847 secs 2024-04-27T05:37:47.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:46 smithi167 ceph-mon[32751]: slow request 23040.370179 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:37:47.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:46 smithi167 ceph-mon[32751]: pgmap v21685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:47.062 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39748.511576 secs 2024-04-27T05:37:47.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:46 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39747.600847 secs 2024-04-27T05:37:47.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:46 smithi090 ceph-mon[29180]: slow request 23040.370179 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:37:47.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:46 smithi090 ceph-mon[29180]: pgmap v21685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:47.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39748.511576 secs 2024-04-27T05:37:47.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:46 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39747.600847 secs 2024-04-27T05:37:47.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:46 smithi119 ceph-mon[32752]: slow request 23040.370179 seconds old, received at 2024-04-26T23:13:44.984072+0000: client_request(client.24733:267893 getattr AsXsFs #0x100000001fb 2024-04-26T23:13:44.982673+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:37:47.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:46 smithi119 ceph-mon[32752]: pgmap v21685: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:47.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39748.511576 secs 2024-04-27T05:37:49.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:48 smithi090 ceph-mon[29180]: pgmap v21686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:49.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:48 smithi119 ceph-mon[32752]: pgmap v21686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:49.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:48 smithi167 ceph-mon[32751]: pgmap v21686: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:51.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:50 smithi090 ceph-mon[29180]: pgmap v21687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:51.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:50 smithi119 ceph-mon[32752]: pgmap v21687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:51.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:50 smithi167 ceph-mon[32751]: pgmap v21687: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:52.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39752.600905 secs 2024-04-27T05:37:52.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39753.511696 secs 2024-04-27T05:37:52.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39752.600905 secs 2024-04-27T05:37:52.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39753.511696 secs 2024-04-27T05:37:52.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39752.600905 secs 2024-04-27T05:37:52.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39753.511696 secs 2024-04-27T05:37:53.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:52 smithi090 ceph-mon[29180]: pgmap v21688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:53.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:52 smithi119 ceph-mon[32752]: pgmap v21688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:53.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:52 smithi167 ceph-mon[32751]: pgmap v21688: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:55.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:54 smithi090 ceph-mon[29180]: pgmap v21689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:55.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:54 smithi119 ceph-mon[32752]: pgmap v21689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:55.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:54 smithi167 ceph-mon[32751]: pgmap v21689: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:37:57.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39757.600989 secs 2024-04-27T05:37:57.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:56 smithi090 ceph-mon[29180]: pgmap v21690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:57.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39758.511775 secs 2024-04-27T05:37:57.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39757.600989 secs 2024-04-27T05:37:57.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:56 smithi119 ceph-mon[32752]: pgmap v21690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:57.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39758.511775 secs 2024-04-27T05:37:57.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39757.600989 secs 2024-04-27T05:37:57.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:56 smithi167 ceph-mon[32751]: pgmap v21690: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:57.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39758.511775 secs 2024-04-27T05:37:59.099 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:37:58 smithi119 ceph-mon[32752]: pgmap v21691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:59.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:37:58 smithi090 ceph-mon[29180]: pgmap v21691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:37:59.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:37:58 smithi167 ceph-mon[32751]: pgmap v21691: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:01.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:00 smithi090 ceph-mon[29180]: pgmap v21692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:01.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:00 smithi119 ceph-mon[32752]: pgmap v21692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:01.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:00 smithi167 ceph-mon[32751]: pgmap v21692: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:02.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39762.601072 secs 2024-04-27T05:38:02.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39763.511884 secs 2024-04-27T05:38:02.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39762.601072 secs 2024-04-27T05:38:02.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39763.511884 secs 2024-04-27T05:38:02.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39762.601072 secs 2024-04-27T05:38:02.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39763.511884 secs 2024-04-27T05:38:03.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:02 smithi090 ceph-mon[29180]: pgmap v21693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:03.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:02 smithi119 ceph-mon[32752]: pgmap v21693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:03.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:02 smithi167 ceph-mon[32751]: pgmap v21693: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:05.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:04 smithi090 ceph-mon[29180]: pgmap v21694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:05.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:04 smithi119 ceph-mon[32752]: pgmap v21694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:05.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:04 smithi167 ceph-mon[32751]: pgmap v21694: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:07.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39767.601160 secs 2024-04-27T05:38:07.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:06 smithi090 ceph-mon[29180]: pgmap v21695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:07.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39768.512019 secs 2024-04-27T05:38:07.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39767.601160 secs 2024-04-27T05:38:07.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:06 smithi119 ceph-mon[32752]: pgmap v21695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:07.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39768.512019 secs 2024-04-27T05:38:07.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39767.601160 secs 2024-04-27T05:38:07.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:06 smithi167 ceph-mon[32751]: pgmap v21695: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:07.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39768.512019 secs 2024-04-27T05:38:09.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:08 smithi090 ceph-mon[29180]: pgmap v21696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:09.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:08 smithi119 ceph-mon[32752]: pgmap v21696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:09.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:08 smithi167 ceph-mon[32751]: pgmap v21696: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:11.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:10 smithi090 ceph-mon[29180]: pgmap v21697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:11.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:10 smithi119 ceph-mon[32752]: pgmap v21697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:11.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:10 smithi167 ceph-mon[32751]: pgmap v21697: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:12.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39772.601245 secs 2024-04-27T05:38:12.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39773.512097 secs 2024-04-27T05:38:12.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39772.601245 secs 2024-04-27T05:38:12.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39773.512097 secs 2024-04-27T05:38:12.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39772.601245 secs 2024-04-27T05:38:12.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39773.512097 secs 2024-04-27T05:38:13.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:12 smithi090 ceph-mon[29180]: pgmap v21698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:13.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:12 smithi119 ceph-mon[32752]: pgmap v21698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:13.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:12 smithi167 ceph-mon[32751]: pgmap v21698: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:15.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:14 smithi090 ceph-mon[29180]: pgmap v21699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:15.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:14 smithi119 ceph-mon[32752]: pgmap v21699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:15.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:14 smithi167 ceph-mon[32751]: pgmap v21699: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:17.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39777.601336 secs 2024-04-27T05:38:17.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:16 smithi090 ceph-mon[29180]: pgmap v21700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:17.119 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39778.512229 secs 2024-04-27T05:38:17.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39777.601336 secs 2024-04-27T05:38:17.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:16 smithi119 ceph-mon[32752]: pgmap v21700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:17.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39778.512229 secs 2024-04-27T05:38:17.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39777.601336 secs 2024-04-27T05:38:17.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:16 smithi167 ceph-mon[32751]: pgmap v21700: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:17.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39778.512229 secs 2024-04-27T05:38:19.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:18 smithi119 ceph-mon[32752]: pgmap v21701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:19.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:18 smithi167 ceph-mon[32751]: pgmap v21701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:19.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:18 smithi090 ceph-mon[29180]: pgmap v21701: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:21.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:20 smithi119 ceph-mon[32752]: pgmap v21702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:21.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:20 smithi167 ceph-mon[32751]: pgmap v21702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:21.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:20 smithi090 ceph-mon[29180]: pgmap v21702: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:22.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39782.601419 secs 2024-04-27T05:38:22.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39783.512315 secs 2024-04-27T05:38:22.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39782.601419 secs 2024-04-27T05:38:22.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39783.512315 secs 2024-04-27T05:38:22.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39782.601419 secs 2024-04-27T05:38:22.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39783.512315 secs 2024-04-27T05:38:23.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:22 smithi119 ceph-mon[32752]: pgmap v21703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:23.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:22 smithi167 ceph-mon[32751]: pgmap v21703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:23.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:22 smithi090 ceph-mon[29180]: pgmap v21703: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:24.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:23 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:38:24.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:23 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:38:24.375 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:23 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:38:25.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:24 smithi119 ceph-mon[32752]: pgmap v21704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:25.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:38:25.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:38:25.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.312 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:24 smithi167 ceph-mon[32751]: pgmap v21704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:25.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:38:25.313 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:38:25.314 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:24 smithi090 ceph-mon[29180]: pgmap v21704: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:25.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:25.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:38:25.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:38:25.370 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:38:26.270 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:25 smithi090 ceph-mon[29180]: pgmap v21705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T05:38:26.270 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:25 smithi090 ceph-mon[29180]: pgmap v21706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:26.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:25 smithi119 ceph-mon[32752]: pgmap v21705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T05:38:26.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:25 smithi119 ceph-mon[32752]: pgmap v21706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:26.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:25 smithi167 ceph-mon[32751]: pgmap v21705: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 908 B/s rd, 1 op/s 2024-04-27T05:38:26.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:25 smithi167 ceph-mon[32751]: pgmap v21706: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:27.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39787.601504 secs 2024-04-27T05:38:27.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39788.512465 secs 2024-04-27T05:38:27.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39787.601504 secs 2024-04-27T05:38:27.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39788.512465 secs 2024-04-27T05:38:27.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39787.601504 secs 2024-04-27T05:38:27.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39788.512465 secs 2024-04-27T05:38:28.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:27 smithi119 ceph-mon[32752]: pgmap v21707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:28.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:27 smithi167 ceph-mon[32751]: pgmap v21707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:28.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:27 smithi090 ceph-mon[29180]: pgmap v21707: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:30.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:29 smithi119 ceph-mon[32752]: pgmap v21708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:30.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:29 smithi167 ceph-mon[32751]: pgmap v21708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:30.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:29 smithi090 ceph-mon[29180]: pgmap v21708: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:32.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39792.601591 secs 2024-04-27T05:38:32.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:31 smithi119 ceph-mon[32752]: pgmap v21709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:32.297 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39793.512534 secs 2024-04-27T05:38:32.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39792.601591 secs 2024-04-27T05:38:32.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:31 smithi167 ceph-mon[32751]: pgmap v21709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:32.311 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39793.512534 secs 2024-04-27T05:38:32.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39792.601591 secs 2024-04-27T05:38:32.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:31 smithi090 ceph-mon[29180]: pgmap v21709: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:38:32.369 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39793.512534 secs 2024-04-27T05:38:33.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:32 smithi119 ceph-mon[32752]: pgmap v21710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T05:38:33.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:32 smithi167 ceph-mon[32751]: pgmap v21710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T05:38:33.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:32 smithi090 ceph-mon[29180]: pgmap v21710: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 552 B/s rd, 1 op/s 2024-04-27T05:38:36.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:35 smithi119 ceph-mon[32752]: pgmap v21711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:38:36.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:35 smithi167 ceph-mon[32751]: pgmap v21711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:38:36.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:35 smithi090 ceph-mon[29180]: pgmap v21711: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:38:37.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39797.601676 secs 2024-04-27T05:38:37.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39798.512598 secs 2024-04-27T05:38:37.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39797.601676 secs 2024-04-27T05:38:37.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39798.512598 secs 2024-04-27T05:38:37.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39797.601676 secs 2024-04-27T05:38:37.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39798.512598 secs 2024-04-27T05:38:38.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:37 smithi119 ceph-mon[32752]: pgmap v21712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:38.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:37 smithi167 ceph-mon[32751]: pgmap v21712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:38.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:37 smithi090 ceph-mon[29180]: pgmap v21712: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:40.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:39 smithi119 ceph-mon[32752]: pgmap v21713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:40.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:39 smithi167 ceph-mon[32751]: pgmap v21713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:40.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:39 smithi090 ceph-mon[29180]: pgmap v21713: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:42.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39802.601761 secs 2024-04-27T05:38:42.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:41 smithi119 ceph-mon[32752]: pgmap v21714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:42.047 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39803.512721 secs 2024-04-27T05:38:42.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39802.601761 secs 2024-04-27T05:38:42.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:41 smithi167 ceph-mon[32751]: pgmap v21714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:42.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39803.512721 secs 2024-04-27T05:38:42.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39802.601761 secs 2024-04-27T05:38:42.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:41 smithi090 ceph-mon[29180]: pgmap v21714: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:42.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39803.512721 secs 2024-04-27T05:38:44.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:43 smithi119 ceph-mon[32752]: pgmap v21715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:44.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:43 smithi167 ceph-mon[32751]: pgmap v21715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:44.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:43 smithi090 ceph-mon[29180]: pgmap v21715: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:46.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:45 smithi119 ceph-mon[32752]: pgmap v21716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:46.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:45 smithi167 ceph-mon[32751]: pgmap v21716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:46.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:45 smithi090 ceph-mon[29180]: pgmap v21716: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:47.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39807.601843 secs 2024-04-27T05:38:47.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39808.512789 secs 2024-04-27T05:38:47.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39807.601843 secs 2024-04-27T05:38:47.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39808.512789 secs 2024-04-27T05:38:47.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39807.601843 secs 2024-04-27T05:38:47.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39808.512789 secs 2024-04-27T05:38:48.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:47 smithi119 ceph-mon[32752]: pgmap v21717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:48.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:47 smithi167 ceph-mon[32751]: pgmap v21717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:48.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:47 smithi090 ceph-mon[29180]: pgmap v21717: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:50.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:49 smithi119 ceph-mon[32752]: pgmap v21718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:50.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:49 smithi167 ceph-mon[32751]: pgmap v21718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:50.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:49 smithi090 ceph-mon[29180]: pgmap v21718: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:52.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39812.601904 secs 2024-04-27T05:38:52.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:51 smithi119 ceph-mon[32752]: pgmap v21719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:52.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39813.512872 secs 2024-04-27T05:38:52.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39812.601904 secs 2024-04-27T05:38:52.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:51 smithi167 ceph-mon[32751]: pgmap v21719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:52.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39813.512872 secs 2024-04-27T05:38:52.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39812.601904 secs 2024-04-27T05:38:52.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:51 smithi090 ceph-mon[29180]: pgmap v21719: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:52.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39813.512872 secs 2024-04-27T05:38:54.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:53 smithi119 ceph-mon[32752]: pgmap v21720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:54.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:53 smithi167 ceph-mon[32751]: pgmap v21720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:54.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:53 smithi090 ceph-mon[29180]: pgmap v21720: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:56.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:55 smithi119 ceph-mon[32752]: pgmap v21721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:56.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:55 smithi167 ceph-mon[32751]: pgmap v21721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:56.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:55 smithi090 ceph-mon[29180]: pgmap v21721: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:38:57.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39817.601966 secs 2024-04-27T05:38:57.019 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39818.512956 secs 2024-04-27T05:38:57.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39817.601966 secs 2024-04-27T05:38:57.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39818.512956 secs 2024-04-27T05:38:57.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39817.601966 secs 2024-04-27T05:38:57.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39818.512956 secs 2024-04-27T05:38:58.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:57 smithi119 ceph-mon[32752]: pgmap v21722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:58.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:57 smithi167 ceph-mon[32751]: pgmap v21722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:38:58.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:57 smithi090 ceph-mon[29180]: pgmap v21722: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:00.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:38:59 smithi119 ceph-mon[32752]: pgmap v21723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:00.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:38:59 smithi167 ceph-mon[32751]: pgmap v21723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:00.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:38:59 smithi090 ceph-mon[29180]: pgmap v21723: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:02.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39822.602029 secs 2024-04-27T05:39:02.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:01 smithi119 ceph-mon[32752]: pgmap v21724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:02.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39823.513077 secs 2024-04-27T05:39:02.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39822.602029 secs 2024-04-27T05:39:02.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:01 smithi167 ceph-mon[32751]: pgmap v21724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:02.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39823.513077 secs 2024-04-27T05:39:02.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39822.602029 secs 2024-04-27T05:39:02.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:01 smithi090 ceph-mon[29180]: pgmap v21724: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:02.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39823.513077 secs 2024-04-27T05:39:04.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:03 smithi119 ceph-mon[32752]: pgmap v21725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:04.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:03 smithi167 ceph-mon[32751]: pgmap v21725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:04.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:03 smithi090 ceph-mon[29180]: pgmap v21725: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:06.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:05 smithi119 ceph-mon[32752]: pgmap v21726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:06.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:05 smithi167 ceph-mon[32751]: pgmap v21726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:06.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:05 smithi090 ceph-mon[29180]: pgmap v21726: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:07.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39827.602090 secs 2024-04-27T05:39:07.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39828.513197 secs 2024-04-27T05:39:07.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39827.602090 secs 2024-04-27T05:39:07.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39828.513197 secs 2024-04-27T05:39:07.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39827.602090 secs 2024-04-27T05:39:07.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39828.513197 secs 2024-04-27T05:39:08.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:07 smithi119 ceph-mon[32752]: pgmap v21727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:08.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:07 smithi167 ceph-mon[32751]: pgmap v21727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:08.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:07 smithi090 ceph-mon[29180]: pgmap v21727: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:10.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:09 smithi119 ceph-mon[32752]: pgmap v21728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:10.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:09 smithi167 ceph-mon[32751]: pgmap v21728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:10.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:09 smithi090 ceph-mon[29180]: pgmap v21728: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:12.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39832.602173 secs 2024-04-27T05:39:12.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:11 smithi119 ceph-mon[32752]: pgmap v21729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:12.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39833.513273 secs 2024-04-27T05:39:12.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39832.602173 secs 2024-04-27T05:39:12.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:11 smithi167 ceph-mon[32751]: pgmap v21729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:12.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39833.513273 secs 2024-04-27T05:39:12.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39832.602173 secs 2024-04-27T05:39:12.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:11 smithi090 ceph-mon[29180]: pgmap v21729: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:12.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39833.513273 secs 2024-04-27T05:39:14.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:13 smithi119 ceph-mon[32752]: pgmap v21730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:14.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:13 smithi167 ceph-mon[32751]: pgmap v21730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:14.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:13 smithi090 ceph-mon[29180]: pgmap v21730: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:16.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:15 smithi119 ceph-mon[32752]: pgmap v21731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:16.061 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:15 smithi167 ceph-mon[32751]: pgmap v21731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:16.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:15 smithi090 ceph-mon[29180]: pgmap v21731: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:17.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39837.602257 secs 2024-04-27T05:39:17.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39838.513412 secs 2024-04-27T05:39:17.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39837.602257 secs 2024-04-27T05:39:17.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39838.513412 secs 2024-04-27T05:39:17.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39837.602257 secs 2024-04-27T05:39:17.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39838.513412 secs 2024-04-27T05:39:18.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:17 smithi119 ceph-mon[32752]: pgmap v21732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:18.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:17 smithi167 ceph-mon[32751]: pgmap v21732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:18.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:17 smithi090 ceph-mon[29180]: pgmap v21732: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:20.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:19 smithi119 ceph-mon[32752]: pgmap v21733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:20.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:19 smithi167 ceph-mon[32751]: pgmap v21733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:20.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:19 smithi090 ceph-mon[29180]: pgmap v21733: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:22.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:21 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 39842.602342 secs 2024-04-27T05:39:22.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:21 smithi119 ceph-mon[32752]: slow request 2881.483381 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:39:22.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:21 smithi119 ceph-mon[32752]: pgmap v21734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:22.046 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39843.513511 secs 2024-04-27T05:39:22.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:21 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 39842.602342 secs 2024-04-27T05:39:22.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:21 smithi167 ceph-mon[32751]: slow request 2881.483381 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:39:22.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:21 smithi167 ceph-mon[32751]: pgmap v21734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:22.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39843.513511 secs 2024-04-27T05:39:22.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:21 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 39842.602342 secs 2024-04-27T05:39:22.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:21 smithi090 ceph-mon[29180]: slow request 2881.483381 seconds old, received at 2024-04-27T04:51:18.872365+0000: client_request(client.24733:267905 getattr AsXsFs #0x100000001fb 2024-04-27T04:51:18.871212+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:39:22.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:21 smithi090 ceph-mon[29180]: pgmap v21734: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:22.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39843.513511 secs 2024-04-27T05:39:24.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:23 smithi119 ceph-mon[32752]: pgmap v21735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:24.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:23 smithi167 ceph-mon[32751]: pgmap v21735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:24.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:23 smithi090 ceph-mon[29180]: pgmap v21735: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:25.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:24 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:39:25.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:24 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:39:25.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:24 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:39:26.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:25 smithi167 ceph-mon[32751]: pgmap v21736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:26.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:39:26.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:39:26.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:39:26.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:25 smithi090 ceph-mon[29180]: pgmap v21736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:26.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:39:26.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:39:26.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:39:26.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:25 smithi119 ceph-mon[32752]: pgmap v21736: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:26.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:39:26.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:39:26.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:39:27.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39847.602430 secs 2024-04-27T05:39:27.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:26 smithi167 ceph-mon[32751]: pgmap v21737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:39:27.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:26 smithi167 ceph-mon[32751]: pgmap v21738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:27.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39848.513595 secs 2024-04-27T05:39:27.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39847.602430 secs 2024-04-27T05:39:27.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:26 smithi090 ceph-mon[29180]: pgmap v21737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:39:27.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:26 smithi090 ceph-mon[29180]: pgmap v21738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:27.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39848.513595 secs 2024-04-27T05:39:27.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39847.602430 secs 2024-04-27T05:39:27.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:26 smithi119 ceph-mon[32752]: pgmap v21737: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:39:27.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:26 smithi119 ceph-mon[32752]: pgmap v21738: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:27.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39848.513595 secs 2024-04-27T05:39:29.059 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:28 smithi167 ceph-mon[32751]: pgmap v21739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:39:29.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:28 smithi090 ceph-mon[29180]: pgmap v21739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:39:29.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:28 smithi119 ceph-mon[32752]: pgmap v21739: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:39:31.060 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:30 smithi167 ceph-mon[32751]: pgmap v21740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:31.098 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:30 smithi119 ceph-mon[32752]: pgmap v21740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:31.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:30 smithi090 ceph-mon[29180]: pgmap v21740: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:32.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39852.602515 secs 2024-04-27T05:39:32.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39853.513696 secs 2024-04-27T05:39:32.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39852.602515 secs 2024-04-27T05:39:32.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39853.513696 secs 2024-04-27T05:39:32.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39852.602515 secs 2024-04-27T05:39:32.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39853.513696 secs 2024-04-27T05:39:33.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:32 smithi090 ceph-mon[29180]: pgmap v21741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:33.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:32 smithi119 ceph-mon[32752]: pgmap v21741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:33.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:32 smithi167 ceph-mon[32751]: pgmap v21741: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:35.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:34 smithi090 ceph-mon[29180]: pgmap v21742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:35.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:34 smithi119 ceph-mon[32752]: pgmap v21742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:35.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:34 smithi167 ceph-mon[32751]: pgmap v21742: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:39:37.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39857.602602 secs 2024-04-27T05:39:37.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:36 smithi090 ceph-mon[29180]: pgmap v21743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:39:37.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39858.513770 secs 2024-04-27T05:39:37.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39857.602602 secs 2024-04-27T05:39:37.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:36 smithi119 ceph-mon[32752]: pgmap v21743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:39:37.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39858.513770 secs 2024-04-27T05:39:37.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39857.602602 secs 2024-04-27T05:39:37.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:36 smithi167 ceph-mon[32751]: pgmap v21743: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:39:37.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39858.513770 secs 2024-04-27T05:39:39.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:38 smithi090 ceph-mon[29180]: pgmap v21744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:39.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:38 smithi119 ceph-mon[32752]: pgmap v21744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:39.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:38 smithi167 ceph-mon[32751]: pgmap v21744: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:41.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:40 smithi090 ceph-mon[29180]: pgmap v21745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:41.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:40 smithi119 ceph-mon[32752]: pgmap v21745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:41.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:40 smithi167 ceph-mon[32751]: pgmap v21745: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:39:42.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39862.602688 secs 2024-04-27T05:39:42.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 39863.513840 secs 2024-04-27T05:39:42.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-mon[29180]: slow request 2880.229918 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:39:42.118 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:39:41.864+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:39:42.118 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:39:41.864+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:39:42.119 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:39:41.864+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:39:42.119 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:39:41.864+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:39:42.119 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:39:41 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:39:41.864+0000 7f823cfa8700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:39:42.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39862.602688 secs 2024-04-27T05:39:42.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:41 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 39863.513840 secs 2024-04-27T05:39:42.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:41 smithi119 ceph-mon[32752]: slow request 2880.229918 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:39:42.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39862.602688 secs 2024-04-27T05:39:42.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:41 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 39863.513840 secs 2024-04-27T05:39:42.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:41 smithi167 ceph-mon[32751]: slow request 2880.229918 seconds old, received at 2024-04-27T04:51:41.034567+0000: client_request(client.27122:1 getattr pAsLsXsFs #0x1 2024-04-27T04:51:41.027059+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:39:43.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:42 smithi090 ceph-mon[29180]: pgmap v21746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:43.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:42 smithi119 ceph-mon[32752]: pgmap v21746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:43.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:42 smithi167 ceph-mon[32751]: pgmap v21746: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:45.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:44 smithi090 ceph-mon[29180]: pgmap v21747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:45.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:44 smithi119 ceph-mon[32752]: pgmap v21747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:45.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:44 smithi167 ceph-mon[32751]: pgmap v21747: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:47.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39867.602775 secs 2024-04-27T05:39:47.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:46 smithi090 ceph-mon[29180]: pgmap v21748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:47.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39868.513951 secs 2024-04-27T05:39:47.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39867.602775 secs 2024-04-27T05:39:47.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:46 smithi119 ceph-mon[32752]: pgmap v21748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:47.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39868.513951 secs 2024-04-27T05:39:47.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39867.602775 secs 2024-04-27T05:39:47.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:46 smithi167 ceph-mon[32751]: pgmap v21748: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:47.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39868.513951 secs 2024-04-27T05:39:49.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:48 smithi090 ceph-mon[29180]: pgmap v21749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:49.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:48 smithi119 ceph-mon[32752]: pgmap v21749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:49.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:48 smithi167 ceph-mon[32751]: pgmap v21749: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:51.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:50 smithi090 ceph-mon[29180]: pgmap v21750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:39:51.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:50 smithi119 ceph-mon[32752]: pgmap v21750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:39:51.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:50 smithi167 ceph-mon[32751]: pgmap v21750: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:39:52.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39872.602857 secs 2024-04-27T05:39:52.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39873.514099 secs 2024-04-27T05:39:52.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39872.602857 secs 2024-04-27T05:39:52.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39873.514099 secs 2024-04-27T05:39:52.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39872.602857 secs 2024-04-27T05:39:52.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39873.514099 secs 2024-04-27T05:39:53.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:52 smithi090 ceph-mon[29180]: pgmap v21751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:53.211 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:52 smithi167 ceph-mon[32751]: pgmap v21751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:53.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:52 smithi119 ceph-mon[32752]: pgmap v21751: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:39:55.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:54 smithi090 ceph-mon[29180]: pgmap v21752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:39:55.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:54 smithi119 ceph-mon[32752]: pgmap v21752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:39:55.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:54 smithi167 ceph-mon[32751]: pgmap v21752: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:39:57.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39877.602946 secs 2024-04-27T05:39:57.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:56 smithi090 ceph-mon[29180]: pgmap v21753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-27T05:39:57.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39878.514221 secs 2024-04-27T05:39:57.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39877.602946 secs 2024-04-27T05:39:57.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:56 smithi119 ceph-mon[32752]: pgmap v21753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-27T05:39:57.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39878.514221 secs 2024-04-27T05:39:57.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39877.602946 secs 2024-04-27T05:39:57.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:56 smithi167 ceph-mon[32751]: pgmap v21753: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2 op/s 2024-04-27T05:39:57.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39878.514221 secs 2024-04-27T05:39:59.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:39:58 smithi090 ceph-mon[29180]: pgmap v21754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:59.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:39:58 smithi119 ceph-mon[32752]: pgmap v21754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:39:59.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:39:58 smithi167 ceph-mon[32751]: pgmap v21754: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:01.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:00 smithi090 ceph-mon[29180]: pgmap v21755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:01.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:40:01.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:00 smithi119 ceph-mon[32752]: pgmap v21755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:01.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:40:01.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:00 smithi167 ceph-mon[32751]: pgmap v21755: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:01.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:40:02.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39882.603010 secs 2024-04-27T05:40:02.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39883.514332 secs 2024-04-27T05:40:02.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39882.603010 secs 2024-04-27T05:40:02.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39883.514332 secs 2024-04-27T05:40:02.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39882.603010 secs 2024-04-27T05:40:02.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39883.514332 secs 2024-04-27T05:40:03.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:02 smithi090 ceph-mon[29180]: pgmap v21756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:03.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:02 smithi119 ceph-mon[32752]: pgmap v21756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:03.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:02 smithi167 ceph-mon[32751]: pgmap v21756: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:05.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:04 smithi119 ceph-mon[32752]: pgmap v21757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:05.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:04 smithi167 ceph-mon[32751]: pgmap v21757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:05.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:04 smithi090 ceph-mon[29180]: pgmap v21757: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:07.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39887.603074 secs 2024-04-27T05:40:07.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:06 smithi090 ceph-mon[29180]: pgmap v21758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:07.118 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39888.514485 secs 2024-04-27T05:40:07.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39887.603074 secs 2024-04-27T05:40:07.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:06 smithi119 ceph-mon[32752]: pgmap v21758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:07.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39888.514485 secs 2024-04-27T05:40:07.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39887.603074 secs 2024-04-27T05:40:07.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:06 smithi167 ceph-mon[32751]: pgmap v21758: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:07.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39888.514485 secs 2024-04-27T05:40:09.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:08 smithi119 ceph-mon[32752]: pgmap v21759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:09.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:08 smithi167 ceph-mon[32751]: pgmap v21759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:09.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:08 smithi090 ceph-mon[29180]: pgmap v21759: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:11.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:10 smithi119 ceph-mon[32752]: pgmap v21760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:11.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:10 smithi167 ceph-mon[32751]: pgmap v21760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:11.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:10 smithi090 ceph-mon[29180]: pgmap v21760: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:12.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39892.603159 secs 2024-04-27T05:40:12.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39893.514599 secs 2024-04-27T05:40:12.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39892.603159 secs 2024-04-27T05:40:12.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39893.514599 secs 2024-04-27T05:40:12.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39892.603159 secs 2024-04-27T05:40:12.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39893.514599 secs 2024-04-27T05:40:13.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:12 smithi119 ceph-mon[32752]: pgmap v21761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:13.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:12 smithi167 ceph-mon[32751]: pgmap v21761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:13.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:12 smithi090 ceph-mon[29180]: pgmap v21761: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:40:15.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:14 smithi119 ceph-mon[32752]: pgmap v21762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:15.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:14 smithi167 ceph-mon[32751]: pgmap v21762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:15.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:14 smithi090 ceph-mon[29180]: pgmap v21762: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:40:17.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39897.603247 secs 2024-04-27T05:40:17.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:16 smithi119 ceph-mon[32752]: pgmap v21763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:17.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39898.514699 secs 2024-04-27T05:40:17.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39897.603247 secs 2024-04-27T05:40:17.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:16 smithi167 ceph-mon[32751]: pgmap v21763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:17.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39898.514699 secs 2024-04-27T05:40:17.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39897.603247 secs 2024-04-27T05:40:17.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:16 smithi090 ceph-mon[29180]: pgmap v21763: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:17.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39898.514699 secs 2024-04-27T05:40:19.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:18 smithi119 ceph-mon[32752]: pgmap v21764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:19.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:18 smithi167 ceph-mon[32751]: pgmap v21764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:19.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:18 smithi090 ceph-mon[29180]: pgmap v21764: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:21.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:20 smithi119 ceph-mon[32752]: pgmap v21765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:21.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:20 smithi167 ceph-mon[32751]: pgmap v21765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:21.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:20 smithi090 ceph-mon[29180]: pgmap v21765: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:22.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39902.603333 secs 2024-04-27T05:40:22.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39903.514815 secs 2024-04-27T05:40:22.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39902.603333 secs 2024-04-27T05:40:22.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39903.514815 secs 2024-04-27T05:40:22.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39902.603333 secs 2024-04-27T05:40:22.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39903.514815 secs 2024-04-27T05:40:23.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:22 smithi167 ceph-mon[32751]: pgmap v21766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:23.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:22 smithi119 ceph-mon[32752]: pgmap v21766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:23.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:22 smithi090 ceph-mon[29180]: pgmap v21766: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:25.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:24 smithi119 ceph-mon[32752]: pgmap v21767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:25.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:24 smithi167 ceph-mon[32751]: pgmap v21767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:25.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:24 smithi090 ceph-mon[29180]: pgmap v21767: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:26.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:25 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:40:26.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:25 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:40:26.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:25 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:40:27.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39907.603422 secs 2024-04-27T05:40:27.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:26 smithi119 ceph-mon[32752]: pgmap v21768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:27.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:40:27.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:40:27.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:40:27.296 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39908.514906 secs 2024-04-27T05:40:27.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39907.603422 secs 2024-04-27T05:40:27.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:26 smithi167 ceph-mon[32751]: pgmap v21768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:27.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:40:27.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:40:27.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:40:27.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39908.514906 secs 2024-04-27T05:40:27.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39907.603422 secs 2024-04-27T05:40:27.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:26 smithi090 ceph-mon[29180]: pgmap v21768: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:27.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:40:27.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:40:27.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:40:27.368 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39908.514906 secs 2024-04-27T05:40:28.268 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:27 smithi090 ceph-mon[29180]: pgmap v21769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:40:28.268 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:27 smithi090 ceph-mon[29180]: pgmap v21770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:28.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:27 smithi119 ceph-mon[32752]: pgmap v21769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:40:28.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:27 smithi119 ceph-mon[32752]: pgmap v21770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:28.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:27 smithi167 ceph-mon[32751]: pgmap v21769: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:40:28.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:27 smithi167 ceph-mon[32751]: pgmap v21770: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:30.298 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:29 smithi119 ceph-mon[32752]: pgmap v21771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:40:30.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:29 smithi167 ceph-mon[32751]: pgmap v21771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:40:30.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:29 smithi090 ceph-mon[29180]: pgmap v21771: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:40:32.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:31 smithi119 ceph-mon[32752]: pgmap v21772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:32.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39912.603505 secs 2024-04-27T05:40:32.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39913.515020 secs 2024-04-27T05:40:32.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:31 smithi167 ceph-mon[32751]: pgmap v21772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:32.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39912.603505 secs 2024-04-27T05:40:32.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39913.515020 secs 2024-04-27T05:40:32.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:31 smithi090 ceph-mon[29180]: pgmap v21772: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:32.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39912.603505 secs 2024-04-27T05:40:32.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39913.515020 secs 2024-04-27T05:40:34.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:33 smithi119 ceph-mon[32752]: pgmap v21773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:40:34.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:33 smithi167 ceph-mon[32751]: pgmap v21773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:40:34.367 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:33 smithi090 ceph-mon[29180]: pgmap v21773: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:40:35.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:34 smithi119 ceph-mon[32752]: pgmap v21774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:35.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:34 smithi167 ceph-mon[32751]: pgmap v21774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:35.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:34 smithi090 ceph-mon[29180]: pgmap v21774: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:40:36.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39917.603608 secs 2024-04-27T05:40:36.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39917.603608 secs 2024-04-27T05:40:36.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39917.603608 secs 2024-04-27T05:40:37.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:37 smithi119 ceph-mon[32752]: pgmap v21775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:40:37.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39918.515167 secs 2024-04-27T05:40:37.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:37 smithi167 ceph-mon[32751]: pgmap v21775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:40:37.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39918.515167 secs 2024-04-27T05:40:37.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:37 smithi090 ceph-mon[29180]: pgmap v21775: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:40:37.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39918.515167 secs 2024-04-27T05:40:39.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:39 smithi119 ceph-mon[32752]: pgmap v21776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:39.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:39 smithi167 ceph-mon[32751]: pgmap v21776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:39.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:39 smithi090 ceph-mon[29180]: pgmap v21776: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:41.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:41 smithi119 ceph-mon[32752]: pgmap v21777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:41.545 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39922.603717 secs 2024-04-27T05:40:41.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:41 smithi167 ceph-mon[32751]: pgmap v21777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:41.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39922.603717 secs 2024-04-27T05:40:41.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:41 smithi090 ceph-mon[29180]: pgmap v21777: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:41.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39922.603717 secs 2024-04-27T05:40:42.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39923.515268 secs 2024-04-27T05:40:42.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39923.515268 secs 2024-04-27T05:40:42.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39923.515268 secs 2024-04-27T05:40:43.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:43 smithi119 ceph-mon[32752]: pgmap v21778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:43.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:43 smithi167 ceph-mon[32751]: pgmap v21778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:43.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:43 smithi090 ceph-mon[29180]: pgmap v21778: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:45.544 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:45 smithi119 ceph-mon[32752]: pgmap v21779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:45.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:45 smithi167 ceph-mon[32751]: pgmap v21779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:45.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:45 smithi090 ceph-mon[29180]: pgmap v21779: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:46.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39927.603778 secs 2024-04-27T05:40:46.559 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39928.515401 secs 2024-04-27T05:40:46.616 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39927.603778 secs 2024-04-27T05:40:46.617 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39928.515401 secs 2024-04-27T05:40:46.794 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39927.603778 secs 2024-04-27T05:40:46.795 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39928.515401 secs 2024-04-27T05:40:48.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:47 smithi119 ceph-mon[32752]: pgmap v21780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:48.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:47 smithi167 ceph-mon[32751]: pgmap v21780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:48.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:47 smithi090 ceph-mon[29180]: pgmap v21780: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:50.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:49 smithi119 ceph-mon[32752]: pgmap v21781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:50.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:49 smithi167 ceph-mon[32751]: pgmap v21781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:50.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:49 smithi090 ceph-mon[29180]: pgmap v21781: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:52.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:51 smithi119 ceph-mon[32752]: pgmap v21782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:52.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39932.603833 secs 2024-04-27T05:40:52.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39933.515520 secs 2024-04-27T05:40:52.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:51 smithi167 ceph-mon[32751]: pgmap v21782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:52.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39932.603833 secs 2024-04-27T05:40:52.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39933.515520 secs 2024-04-27T05:40:52.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:51 smithi090 ceph-mon[29180]: pgmap v21782: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:52.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39932.603833 secs 2024-04-27T05:40:52.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39933.515520 secs 2024-04-27T05:40:54.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:53 smithi119 ceph-mon[32752]: pgmap v21783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:54.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:53 smithi167 ceph-mon[32751]: pgmap v21783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:54.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:53 smithi090 ceph-mon[29180]: pgmap v21783: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:56.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:55 smithi119 ceph-mon[32752]: pgmap v21784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:56.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:55 smithi167 ceph-mon[32751]: pgmap v21784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:56.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:55 smithi090 ceph-mon[29180]: pgmap v21784: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:40:57.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39937.603916 secs 2024-04-27T05:40:57.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39938.515615 secs 2024-04-27T05:40:57.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39937.603916 secs 2024-04-27T05:40:57.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39938.515615 secs 2024-04-27T05:40:57.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39937.603916 secs 2024-04-27T05:40:57.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39938.515615 secs 2024-04-27T05:40:58.018 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:57 smithi090 ceph-mon[29180]: pgmap v21785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:58.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:57 smithi119 ceph-mon[32752]: pgmap v21785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:40:58.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:57 smithi167 ceph-mon[32751]: pgmap v21785: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:00.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:40:59 smithi119 ceph-mon[32752]: pgmap v21786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:00.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:40:59 smithi167 ceph-mon[32751]: pgmap v21786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:00.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:40:59 smithi090 ceph-mon[29180]: pgmap v21786: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:02.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:01 smithi119 ceph-mon[32752]: pgmap v21787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:02.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39942.603971 secs 2024-04-27T05:41:02.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39943.515772 secs 2024-04-27T05:41:02.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:01 smithi167 ceph-mon[32751]: pgmap v21787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:02.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39942.603971 secs 2024-04-27T05:41:02.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39943.515772 secs 2024-04-27T05:41:02.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:01 smithi090 ceph-mon[29180]: pgmap v21787: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:02.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39942.603971 secs 2024-04-27T05:41:02.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39943.515772 secs 2024-04-27T05:41:04.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:03 smithi119 ceph-mon[32752]: pgmap v21788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:04.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:03 smithi167 ceph-mon[32751]: pgmap v21788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:04.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:03 smithi090 ceph-mon[29180]: pgmap v21788: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:06.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:05 smithi119 ceph-mon[32752]: pgmap v21789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:06.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:05 smithi167 ceph-mon[32751]: pgmap v21789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:06.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:05 smithi090 ceph-mon[29180]: pgmap v21789: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:07.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39947.604029 secs 2024-04-27T05:41:07.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39948.515898 secs 2024-04-27T05:41:07.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39947.604029 secs 2024-04-27T05:41:07.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39948.515898 secs 2024-04-27T05:41:07.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39947.604029 secs 2024-04-27T05:41:07.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39948.515898 secs 2024-04-27T05:41:08.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:07 smithi119 ceph-mon[32752]: pgmap v21790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:08.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:07 smithi167 ceph-mon[32751]: pgmap v21790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:08.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:07 smithi090 ceph-mon[29180]: pgmap v21790: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:10.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:09 smithi119 ceph-mon[32752]: pgmap v21791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:10.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:09 smithi167 ceph-mon[32751]: pgmap v21791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:10.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:09 smithi090 ceph-mon[29180]: pgmap v21791: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:12.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:11 smithi119 ceph-mon[32752]: pgmap v21792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:12.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39952.604112 secs 2024-04-27T05:41:12.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39953.516035 secs 2024-04-27T05:41:12.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:11 smithi167 ceph-mon[32751]: pgmap v21792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:12.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39952.604112 secs 2024-04-27T05:41:12.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39953.516035 secs 2024-04-27T05:41:12.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:11 smithi090 ceph-mon[29180]: pgmap v21792: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:12.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39952.604112 secs 2024-04-27T05:41:12.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39953.516035 secs 2024-04-27T05:41:14.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:13 smithi119 ceph-mon[32752]: pgmap v21793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:14.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:13 smithi167 ceph-mon[32751]: pgmap v21793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:14.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:13 smithi090 ceph-mon[29180]: pgmap v21793: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:16.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:15 smithi119 ceph-mon[32752]: pgmap v21794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:16.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:15 smithi167 ceph-mon[32751]: pgmap v21794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:16.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:15 smithi090 ceph-mon[29180]: pgmap v21794: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:17.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39957.604197 secs 2024-04-27T05:41:17.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39958.516168 secs 2024-04-27T05:41:17.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39957.604197 secs 2024-04-27T05:41:17.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39958.516168 secs 2024-04-27T05:41:17.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39957.604197 secs 2024-04-27T05:41:17.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39958.516168 secs 2024-04-27T05:41:18.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:17 smithi119 ceph-mon[32752]: pgmap v21795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:18.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:17 smithi167 ceph-mon[32751]: pgmap v21795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:18.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:17 smithi090 ceph-mon[29180]: pgmap v21795: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:20.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:19 smithi119 ceph-mon[32752]: pgmap v21796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:20.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:19 smithi167 ceph-mon[32751]: pgmap v21796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:20.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:19 smithi090 ceph-mon[29180]: pgmap v21796: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:22.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:21 smithi119 ceph-mon[32752]: pgmap v21797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:22.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39962.604277 secs 2024-04-27T05:41:22.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39963.516294 secs 2024-04-27T05:41:22.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:21 smithi167 ceph-mon[32751]: pgmap v21797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:22.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39962.604277 secs 2024-04-27T05:41:22.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39963.516294 secs 2024-04-27T05:41:22.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:21 smithi090 ceph-mon[29180]: pgmap v21797: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:22.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39962.604277 secs 2024-04-27T05:41:22.117 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39963.516294 secs 2024-04-27T05:41:24.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:23 smithi119 ceph-mon[32752]: pgmap v21798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:24.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:23 smithi167 ceph-mon[32751]: pgmap v21798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:24.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:23 smithi090 ceph-mon[29180]: pgmap v21798: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:25.866 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:25 smithi090 ceph-mon[29180]: pgmap v21799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:26.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:25 smithi119 ceph-mon[32752]: pgmap v21799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:26.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:25 smithi167 ceph-mon[32751]: pgmap v21799: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:27.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39967.604358 secs 2024-04-27T05:41:27.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39968.516366 secs 2024-04-27T05:41:27.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:26 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:41:27.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39967.604358 secs 2024-04-27T05:41:27.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39968.516366 secs 2024-04-27T05:41:27.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:26 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:41:27.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39967.604358 secs 2024-04-27T05:41:27.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39968.516366 secs 2024-04-27T05:41:27.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:26 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:41:28.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:27 smithi119 ceph-mon[32752]: pgmap v21800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:28.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:41:28.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:41:28.045 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:41:28.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:27 smithi167 ceph-mon[32751]: pgmap v21800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:28.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:41:28.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:41:28.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:41:28.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:27 smithi090 ceph-mon[29180]: pgmap v21800: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:28.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:41:28.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:41:28.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:41:29.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:28 smithi119 ceph-mon[32752]: pgmap v21801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:41:29.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:28 smithi119 ceph-mon[32752]: pgmap v21802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:29.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:28 smithi167 ceph-mon[32751]: pgmap v21801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:41:29.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:28 smithi167 ceph-mon[32751]: pgmap v21802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:29.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:28 smithi090 ceph-mon[29180]: pgmap v21801: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:41:29.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:28 smithi090 ceph-mon[29180]: pgmap v21802: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:31.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:30 smithi119 ceph-mon[32752]: pgmap v21803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:31.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:30 smithi167 ceph-mon[32751]: pgmap v21803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:31.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:30 smithi090 ceph-mon[29180]: pgmap v21803: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:32.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39972.604441 secs 2024-04-27T05:41:32.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39973.516498 secs 2024-04-27T05:41:32.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39972.604441 secs 2024-04-27T05:41:32.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39973.516498 secs 2024-04-27T05:41:32.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39972.604441 secs 2024-04-27T05:41:32.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39973.516498 secs 2024-04-27T05:41:33.044 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:32 smithi119 ceph-mon[32752]: pgmap v21804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:33.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:32 smithi167 ceph-mon[32751]: pgmap v21804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:33.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:32 smithi090 ceph-mon[29180]: pgmap v21804: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:35.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:34 smithi167 ceph-mon[32751]: pgmap v21805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:41:35.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:34 smithi090 ceph-mon[29180]: pgmap v21805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:41:35.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:34 smithi119 ceph-mon[32752]: pgmap v21805: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:41:37.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:36 smithi167 ceph-mon[32751]: pgmap v21806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:37.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39977.604525 secs 2024-04-27T05:41:37.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39978.516630 secs 2024-04-27T05:41:37.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:36 smithi090 ceph-mon[29180]: pgmap v21806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:37.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39977.604525 secs 2024-04-27T05:41:37.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39978.516630 secs 2024-04-27T05:41:37.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:36 smithi119 ceph-mon[32752]: pgmap v21806: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:41:37.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39977.604525 secs 2024-04-27T05:41:37.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39978.516630 secs 2024-04-27T05:41:39.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:38 smithi167 ceph-mon[32751]: pgmap v21807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:41:39.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:38 smithi090 ceph-mon[29180]: pgmap v21807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:41:39.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:38 smithi119 ceph-mon[32752]: pgmap v21807: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:41:41.058 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:40 smithi167 ceph-mon[32751]: pgmap v21808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:41.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:40 smithi090 ceph-mon[29180]: pgmap v21808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:41.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:40 smithi119 ceph-mon[32752]: pgmap v21808: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:42.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39982.604617 secs 2024-04-27T05:41:42.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:41 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 39983.516725 secs 2024-04-27T05:41:42.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:41 smithi090 ceph-mon[29180]: slow request 11522.607848 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:41:42.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39982.604617 secs 2024-04-27T05:41:42.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:41 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 39983.516725 secs 2024-04-27T05:41:42.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:41 smithi119 ceph-mon[32752]: slow request 11522.607848 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:41:42.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39982.604617 secs 2024-04-27T05:41:42.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:41 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 39983.516725 secs 2024-04-27T05:41:42.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:41 smithi167 ceph-mon[32751]: slow request 11522.607848 seconds old, received at 2024-04-27T02:29:38.659521+0000: client_request(client.17646:1 getattr pAsLsXsFs #0x1 2024-04-27T02:29:38.655870+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:41:43.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:42 smithi090 ceph-mon[29180]: pgmap v21809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:43.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:42 smithi119 ceph-mon[32752]: pgmap v21809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:43.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:42 smithi167 ceph-mon[32751]: pgmap v21809: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:45.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:44 smithi090 ceph-mon[29180]: pgmap v21810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:45.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:44 smithi119 ceph-mon[32752]: pgmap v21810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:45.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:44 smithi167 ceph-mon[32751]: pgmap v21810: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:47.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:46 smithi090 ceph-mon[29180]: pgmap v21811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:47.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39987.604702 secs 2024-04-27T05:41:47.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39988.516840 secs 2024-04-27T05:41:47.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:46 smithi119 ceph-mon[32752]: pgmap v21811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:47.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39987.604702 secs 2024-04-27T05:41:47.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39988.516840 secs 2024-04-27T05:41:47.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:46 smithi167 ceph-mon[32751]: pgmap v21811: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:47.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39987.604702 secs 2024-04-27T05:41:47.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39988.516840 secs 2024-04-27T05:41:49.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:48 smithi090 ceph-mon[29180]: pgmap v21812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:49.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:48 smithi119 ceph-mon[32752]: pgmap v21812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:49.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:48 smithi167 ceph-mon[32751]: pgmap v21812: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:51.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:50 smithi090 ceph-mon[29180]: pgmap v21813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:51.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:50 smithi119 ceph-mon[32752]: pgmap v21813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:51.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:50 smithi167 ceph-mon[32751]: pgmap v21813: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:52.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39992.604763 secs 2024-04-27T05:41:52.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39993.516966 secs 2024-04-27T05:41:52.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39992.604763 secs 2024-04-27T05:41:52.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39993.516966 secs 2024-04-27T05:41:52.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39992.604763 secs 2024-04-27T05:41:52.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39993.516966 secs 2024-04-27T05:41:53.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:52 smithi090 ceph-mon[29180]: pgmap v21814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:53.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:52 smithi167 ceph-mon[32751]: pgmap v21814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:53.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:52 smithi119 ceph-mon[32752]: pgmap v21814: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:55.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:54 smithi090 ceph-mon[29180]: pgmap v21815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:55.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:54 smithi119 ceph-mon[32752]: pgmap v21815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:55.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:54 smithi167 ceph-mon[32751]: pgmap v21815: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:57.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:56 smithi090 ceph-mon[29180]: pgmap v21816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:57.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 39997.604846 secs 2024-04-27T05:41:57.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 39998.517096 secs 2024-04-27T05:41:57.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:56 smithi119 ceph-mon[32752]: pgmap v21816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:57.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 39997.604846 secs 2024-04-27T05:41:57.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 39998.517096 secs 2024-04-27T05:41:57.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:56 smithi167 ceph-mon[32751]: pgmap v21816: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:41:57.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 39997.604846 secs 2024-04-27T05:41:57.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 39998.517096 secs 2024-04-27T05:41:59.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:41:58 smithi090 ceph-mon[29180]: pgmap v21817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:59.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:41:58 smithi119 ceph-mon[32752]: pgmap v21817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:41:59.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:41:58 smithi167 ceph-mon[32751]: pgmap v21817: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:01.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:00 smithi090 ceph-mon[29180]: pgmap v21818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:01.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:00 smithi119 ceph-mon[32752]: pgmap v21818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:01.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:00 smithi167 ceph-mon[32751]: pgmap v21818: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:02.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40002.604932 secs 2024-04-27T05:42:02.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40003.517223 secs 2024-04-27T05:42:02.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40002.604932 secs 2024-04-27T05:42:02.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40003.517223 secs 2024-04-27T05:42:02.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40002.604932 secs 2024-04-27T05:42:02.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40003.517223 secs 2024-04-27T05:42:03.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:02 smithi090 ceph-mon[29180]: pgmap v21819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:03.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:02 smithi119 ceph-mon[32752]: pgmap v21819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:03.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:02 smithi167 ceph-mon[32751]: pgmap v21819: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:05.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:04 smithi090 ceph-mon[29180]: pgmap v21820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:05.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:04 smithi119 ceph-mon[32752]: pgmap v21820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:05.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:04 smithi167 ceph-mon[32751]: pgmap v21820: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:07.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:06 smithi090 ceph-mon[29180]: pgmap v21821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:07.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40007.604993 secs 2024-04-27T05:42:07.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40008.517288 secs 2024-04-27T05:42:07.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:06 smithi119 ceph-mon[32752]: pgmap v21821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:07.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40007.604993 secs 2024-04-27T05:42:07.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40008.517288 secs 2024-04-27T05:42:07.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:06 smithi167 ceph-mon[32751]: pgmap v21821: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:07.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40007.604993 secs 2024-04-27T05:42:07.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40008.517288 secs 2024-04-27T05:42:09.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:08 smithi119 ceph-mon[32752]: pgmap v21822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:09.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:08 smithi167 ceph-mon[32751]: pgmap v21822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:09.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:08 smithi090 ceph-mon[29180]: pgmap v21822: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:11.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:10 smithi119 ceph-mon[32752]: pgmap v21823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:11.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:10 smithi167 ceph-mon[32751]: pgmap v21823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:11.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:10 smithi090 ceph-mon[29180]: pgmap v21823: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:12.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40012.605054 secs 2024-04-27T05:42:12.116 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40013.517416 secs 2024-04-27T05:42:12.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40012.605054 secs 2024-04-27T05:42:12.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40013.517416 secs 2024-04-27T05:42:12.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40012.605054 secs 2024-04-27T05:42:12.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40013.517416 secs 2024-04-27T05:42:13.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:12 smithi119 ceph-mon[32752]: pgmap v21824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:13.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:12 smithi167 ceph-mon[32751]: pgmap v21824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:13.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:12 smithi090 ceph-mon[29180]: pgmap v21824: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:15.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:14 smithi119 ceph-mon[32752]: pgmap v21825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:15.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:14 smithi167 ceph-mon[32751]: pgmap v21825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:15.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:14 smithi090 ceph-mon[29180]: pgmap v21825: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:17.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:16 smithi119 ceph-mon[32752]: pgmap v21826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:17.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40017.605109 secs 2024-04-27T05:42:17.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40018.517500 secs 2024-04-27T05:42:17.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:16 smithi167 ceph-mon[32751]: pgmap v21826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:17.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40017.605109 secs 2024-04-27T05:42:17.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40018.517500 secs 2024-04-27T05:42:17.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:16 smithi090 ceph-mon[29180]: pgmap v21826: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:17.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40017.605109 secs 2024-04-27T05:42:17.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40018.517500 secs 2024-04-27T05:42:19.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:18 smithi119 ceph-mon[32752]: pgmap v21827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:19.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:18 smithi167 ceph-mon[32751]: pgmap v21827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:19.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:18 smithi090 ceph-mon[29180]: pgmap v21827: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:21.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:20 smithi119 ceph-mon[32752]: pgmap v21828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:21.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:20 smithi167 ceph-mon[32751]: pgmap v21828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:21.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:20 smithi090 ceph-mon[29180]: pgmap v21828: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:22.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40022.605192 secs 2024-04-27T05:42:22.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40023.517614 secs 2024-04-27T05:42:22.309 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40022.605192 secs 2024-04-27T05:42:22.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40023.517614 secs 2024-04-27T05:42:22.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40022.605192 secs 2024-04-27T05:42:22.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40023.517614 secs 2024-04-27T05:42:23.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:22 smithi167 ceph-mon[32751]: pgmap v21829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:23.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:22 smithi119 ceph-mon[32752]: pgmap v21829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:23.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:22 smithi090 ceph-mon[29180]: pgmap v21829: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:25.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:24 smithi119 ceph-mon[32752]: pgmap v21830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:25.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:24 smithi167 ceph-mon[32751]: pgmap v21830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:25.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:24 smithi090 ceph-mon[29180]: pgmap v21830: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:27.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:26 smithi119 ceph-mon[32752]: pgmap v21831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:27.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40027.605275 secs 2024-04-27T05:42:27.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40028.517737 secs 2024-04-27T05:42:27.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:26 smithi167 ceph-mon[32751]: pgmap v21831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:27.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40027.605275 secs 2024-04-27T05:42:27.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40028.517737 secs 2024-04-27T05:42:27.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:26 smithi090 ceph-mon[29180]: pgmap v21831: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:27.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40027.605275 secs 2024-04-27T05:42:27.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40028.517737 secs 2024-04-27T05:42:28.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:42:28.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:42:28.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:42:28.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:42:28.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:42:28.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:42:28.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:42:28.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:42:28.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:42:28.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:42:28.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:42:28.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:42:29.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:28 smithi119 ceph-mon[32752]: pgmap v21832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:42:29.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:28 smithi119 ceph-mon[32752]: pgmap v21833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T05:42:29.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:28 smithi119 ceph-mon[32752]: pgmap v21834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T05:42:29.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:28 smithi167 ceph-mon[32751]: pgmap v21832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:42:29.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:28 smithi167 ceph-mon[32751]: pgmap v21833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T05:42:29.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:28 smithi167 ceph-mon[32751]: pgmap v21834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T05:42:29.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:28 smithi090 ceph-mon[29180]: pgmap v21832: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-27T05:42:29.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:28 smithi090 ceph-mon[29180]: pgmap v21833: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 856 B/s rd, 1 op/s 2024-04-27T05:42:29.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:28 smithi090 ceph-mon[29180]: pgmap v21834: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 467 B/s rd, 1 op/s 2024-04-27T05:42:31.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:30 smithi119 ceph-mon[32752]: pgmap v21835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:42:31.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:30 smithi167 ceph-mon[32751]: pgmap v21835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:42:31.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:30 smithi090 ceph-mon[29180]: pgmap v21835: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:42:32.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40032.605361 secs 2024-04-27T05:42:32.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40033.517823 secs 2024-04-27T05:42:32.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40032.605361 secs 2024-04-27T05:42:32.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40033.517823 secs 2024-04-27T05:42:32.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40032.605361 secs 2024-04-27T05:42:32.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40033.517823 secs 2024-04-27T05:42:33.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:32 smithi119 ceph-mon[32752]: pgmap v21836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:42:33.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:32 smithi167 ceph-mon[32751]: pgmap v21836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:42:33.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:32 smithi090 ceph-mon[29180]: pgmap v21836: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-27T05:42:35.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:34 smithi119 ceph-mon[32752]: pgmap v21837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:42:35.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:34 smithi167 ceph-mon[32751]: pgmap v21837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:42:35.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:34 smithi090 ceph-mon[29180]: pgmap v21837: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:42:37.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40037.605445 secs 2024-04-27T05:42:37.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:36 smithi119 ceph-mon[32752]: pgmap v21838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:42:37.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40038.517945 secs 2024-04-27T05:42:37.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40037.605445 secs 2024-04-27T05:42:37.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:36 smithi167 ceph-mon[32751]: pgmap v21838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:42:37.308 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40038.517945 secs 2024-04-27T05:42:37.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40037.605445 secs 2024-04-27T05:42:37.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:36 smithi090 ceph-mon[29180]: pgmap v21838: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:42:37.366 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40038.517945 secs 2024-04-27T05:42:39.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:38 smithi119 ceph-mon[32752]: pgmap v21839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:42:39.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:38 smithi167 ceph-mon[32751]: pgmap v21839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:42:39.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:38 smithi090 ceph-mon[29180]: pgmap v21839: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:42:41.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:40 smithi119 ceph-mon[32752]: pgmap v21840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:41.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:40 smithi167 ceph-mon[32751]: pgmap v21840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:41.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:40 smithi090 ceph-mon[29180]: pgmap v21840: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:42.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40042.605506 secs 2024-04-27T05:42:42.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40043.518033 secs 2024-04-27T05:42:42.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40042.605506 secs 2024-04-27T05:42:42.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40043.518033 secs 2024-04-27T05:42:42.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40042.605506 secs 2024-04-27T05:42:42.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40043.518033 secs 2024-04-27T05:42:43.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:42 smithi119 ceph-mon[32752]: pgmap v21841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:43.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:42 smithi167 ceph-mon[32751]: pgmap v21841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:43.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:42 smithi090 ceph-mon[29180]: pgmap v21841: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:45.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:44 smithi119 ceph-mon[32752]: pgmap v21842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:45.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:44 smithi167 ceph-mon[32751]: pgmap v21842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:45.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:44 smithi090 ceph-mon[29180]: pgmap v21842: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:46.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:46 smithi119 ceph-mon[32752]: pgmap v21843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:46.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:46 smithi167 ceph-mon[32751]: pgmap v21843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:46.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:46 smithi090 ceph-mon[29180]: pgmap v21843: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:47.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:47 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40047.605593 secs 2024-04-27T05:42:47.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40048.518099 secs 2024-04-27T05:42:47.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:47 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40047.605593 secs 2024-04-27T05:42:47.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40048.518099 secs 2024-04-27T05:42:47.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:47 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40047.605593 secs 2024-04-27T05:42:47.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40048.518099 secs 2024-04-27T05:42:48.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:48 smithi119 ceph-mon[32752]: pgmap v21844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:48.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:48 smithi167 ceph-mon[32751]: pgmap v21844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:48.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:48 smithi090 ceph-mon[29180]: pgmap v21844: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:51.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:50 smithi167 ceph-mon[32751]: pgmap v21845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:51.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:50 smithi090 ceph-mon[29180]: pgmap v21845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:51.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:50 smithi119 ceph-mon[32752]: pgmap v21845: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:52.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40052.605676 secs 2024-04-27T05:42:52.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40053.518224 secs 2024-04-27T05:42:52.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40052.605676 secs 2024-04-27T05:42:52.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40053.518224 secs 2024-04-27T05:42:52.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40052.605676 secs 2024-04-27T05:42:52.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40053.518224 secs 2024-04-27T05:42:53.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:52 smithi090 ceph-mon[29180]: pgmap v21846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:53.209 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:52 smithi167 ceph-mon[32751]: pgmap v21846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:53.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:52 smithi119 ceph-mon[32752]: pgmap v21846: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:55.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:54 smithi090 ceph-mon[29180]: pgmap v21847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:55.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:54 smithi119 ceph-mon[32752]: pgmap v21847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:55.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:54 smithi167 ceph-mon[32751]: pgmap v21847: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:42:57.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40057.605762 secs 2024-04-27T05:42:57.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:56 smithi090 ceph-mon[29180]: pgmap v21848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:57.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40058.518318 secs 2024-04-27T05:42:57.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40057.605762 secs 2024-04-27T05:42:57.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:56 smithi119 ceph-mon[32752]: pgmap v21848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:57.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40058.518318 secs 2024-04-27T05:42:57.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40057.605762 secs 2024-04-27T05:42:57.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:56 smithi167 ceph-mon[32751]: pgmap v21848: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:57.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40058.518318 secs 2024-04-27T05:42:59.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:42:58 smithi090 ceph-mon[29180]: pgmap v21849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:59.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:42:58 smithi119 ceph-mon[32752]: pgmap v21849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:42:59.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:42:58 smithi167 ceph-mon[32751]: pgmap v21849: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:01.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:00 smithi090 ceph-mon[29180]: pgmap v21850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:01.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:00 smithi119 ceph-mon[32752]: pgmap v21850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:01.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:00 smithi167 ceph-mon[32751]: pgmap v21850: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:02.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40062.605831 secs 2024-04-27T05:43:02.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40063.518457 secs 2024-04-27T05:43:02.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40062.605831 secs 2024-04-27T05:43:02.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40063.518457 secs 2024-04-27T05:43:02.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40062.605831 secs 2024-04-27T05:43:02.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40063.518457 secs 2024-04-27T05:43:03.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:02 smithi090 ceph-mon[29180]: pgmap v21851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:03.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:02 smithi119 ceph-mon[32752]: pgmap v21851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:03.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:02 smithi167 ceph-mon[32751]: pgmap v21851: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:05.093 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:04 smithi119 ceph-mon[32752]: pgmap v21852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:05.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:04 smithi090 ceph-mon[29180]: pgmap v21852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:05.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:04 smithi167 ceph-mon[32751]: pgmap v21852: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:07.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40067.605888 secs 2024-04-27T05:43:07.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:06 smithi090 ceph-mon[29180]: pgmap v21853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:07.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40068.518558 secs 2024-04-27T05:43:07.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40067.605888 secs 2024-04-27T05:43:07.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:06 smithi119 ceph-mon[32752]: pgmap v21853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:07.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40068.518558 secs 2024-04-27T05:43:07.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40067.605888 secs 2024-04-27T05:43:07.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:06 smithi167 ceph-mon[32751]: pgmap v21853: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:07.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40068.518558 secs 2024-04-27T05:43:09.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:08 smithi090 ceph-mon[29180]: pgmap v21854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:09.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:08 smithi119 ceph-mon[32752]: pgmap v21854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:09.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:08 smithi167 ceph-mon[32751]: pgmap v21854: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:11.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:10 smithi090 ceph-mon[29180]: pgmap v21855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:11.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:10 smithi119 ceph-mon[32752]: pgmap v21855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:11.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:10 smithi167 ceph-mon[32751]: pgmap v21855: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:12.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40072.605971 secs 2024-04-27T05:43:12.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40073.518646 secs 2024-04-27T05:43:12.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40072.605971 secs 2024-04-27T05:43:12.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40073.518646 secs 2024-04-27T05:43:12.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40072.605971 secs 2024-04-27T05:43:12.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40073.518646 secs 2024-04-27T05:43:13.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:12 smithi090 ceph-mon[29180]: pgmap v21856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:13.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:12 smithi119 ceph-mon[32752]: pgmap v21856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:13.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:12 smithi167 ceph-mon[32751]: pgmap v21856: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:15.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:14 smithi090 ceph-mon[29180]: pgmap v21857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:15.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:14 smithi119 ceph-mon[32752]: pgmap v21857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:15.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:14 smithi167 ceph-mon[32751]: pgmap v21857: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:17.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40077.606032 secs 2024-04-27T05:43:17.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:16 smithi090 ceph-mon[29180]: pgmap v21858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:17.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40078.518728 secs 2024-04-27T05:43:17.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40077.606032 secs 2024-04-27T05:43:17.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:16 smithi119 ceph-mon[32752]: pgmap v21858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:17.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40078.518728 secs 2024-04-27T05:43:17.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40077.606032 secs 2024-04-27T05:43:17.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:16 smithi167 ceph-mon[32751]: pgmap v21858: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:17.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40078.518728 secs 2024-04-27T05:43:19.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:18 smithi119 ceph-mon[32752]: pgmap v21859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:19.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:18 smithi167 ceph-mon[32751]: pgmap v21859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:19.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:18 smithi090 ceph-mon[29180]: pgmap v21859: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:21.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:20 smithi119 ceph-mon[32752]: pgmap v21860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:21.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:20 smithi167 ceph-mon[32751]: pgmap v21860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:21.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:20 smithi090 ceph-mon[29180]: pgmap v21860: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:22.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40082.606094 secs 2024-04-27T05:43:22.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40083.518852 secs 2024-04-27T05:43:22.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40082.606094 secs 2024-04-27T05:43:22.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40083.518852 secs 2024-04-27T05:43:22.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40082.606094 secs 2024-04-27T05:43:22.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40083.518852 secs 2024-04-27T05:43:23.208 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:22 smithi167 ceph-mon[32751]: pgmap v21861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:23.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:22 smithi119 ceph-mon[32752]: pgmap v21861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:23.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:22 smithi090 ceph-mon[29180]: pgmap v21861: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:25.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:24 smithi119 ceph-mon[32752]: pgmap v21862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:25.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:24 smithi167 ceph-mon[32751]: pgmap v21862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:25.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:24 smithi090 ceph-mon[29180]: pgmap v21862: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:27.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40087.606179 secs 2024-04-27T05:43:27.301 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:26 smithi119 ceph-mon[32752]: pgmap v21863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:27.302 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40088.518923 secs 2024-04-27T05:43:27.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40087.606179 secs 2024-04-27T05:43:27.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:26 smithi167 ceph-mon[32751]: pgmap v21863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:27.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40088.518923 secs 2024-04-27T05:43:27.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40087.606179 secs 2024-04-27T05:43:27.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:26 smithi090 ceph-mon[29180]: pgmap v21863: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:27.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40088.518923 secs 2024-04-27T05:43:28.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:27 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:43:28.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:27 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:43:28.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:27 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:43:29.266 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:28 smithi090 ceph-mon[29180]: pgmap v21864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:29.266 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:43:29.266 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:43:29.267 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:43:29.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:28 smithi119 ceph-mon[32752]: pgmap v21864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:29.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:43:29.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:43:29.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:43:29.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:28 smithi167 ceph-mon[32751]: pgmap v21864: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:29.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:43:29.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:43:29.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:43:30.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:29 smithi119 ceph-mon[32752]: pgmap v21865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:43:30.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:29 smithi119 ceph-mon[32752]: pgmap v21866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:30.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:29 smithi167 ceph-mon[32751]: pgmap v21865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:43:30.310 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:29 smithi167 ceph-mon[32751]: pgmap v21866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:30.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:29 smithi090 ceph-mon[29180]: pgmap v21865: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-04-27T05:43:30.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:29 smithi090 ceph-mon[29180]: pgmap v21866: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:32.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40092.606263 secs 2024-04-27T05:43:32.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:31 smithi119 ceph-mon[32752]: pgmap v21867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:32.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40093.519028 secs 2024-04-27T05:43:32.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40092.606263 secs 2024-04-27T05:43:32.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:31 smithi167 ceph-mon[32751]: pgmap v21867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:32.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40093.519028 secs 2024-04-27T05:43:32.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40092.606263 secs 2024-04-27T05:43:32.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:31 smithi090 ceph-mon[29180]: pgmap v21867: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:32.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40093.519028 secs 2024-04-27T05:43:34.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:33 smithi119 ceph-mon[32752]: pgmap v21868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:43:34.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:33 smithi167 ceph-mon[32751]: pgmap v21868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:43:34.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:33 smithi090 ceph-mon[29180]: pgmap v21868: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:43:36.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:35 smithi119 ceph-mon[32752]: pgmap v21869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:36.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:35 smithi167 ceph-mon[32751]: pgmap v21869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:36.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:35 smithi090 ceph-mon[29180]: pgmap v21869: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:37.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40097.606312 secs 2024-04-27T05:43:37.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40098.519167 secs 2024-04-27T05:43:37.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40097.606312 secs 2024-04-27T05:43:37.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40098.519167 secs 2024-04-27T05:43:37.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40097.606312 secs 2024-04-27T05:43:37.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40098.519167 secs 2024-04-27T05:43:38.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:37 smithi119 ceph-mon[32752]: pgmap v21870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:38.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:37 smithi167 ceph-mon[32751]: pgmap v21870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:38.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:37 smithi090 ceph-mon[29180]: pgmap v21870: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:43:40.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:39 smithi119 ceph-mon[32752]: pgmap v21871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:43:40.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:39 smithi167 ceph-mon[32751]: pgmap v21871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:43:40.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:39 smithi090 ceph-mon[29180]: pgmap v21871: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:43:42.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40102.606372 secs 2024-04-27T05:43:42.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:41 smithi119 ceph-mon[32752]: pgmap v21872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:42.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40103.519325 secs 2024-04-27T05:43:42.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40102.606372 secs 2024-04-27T05:43:42.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:41 smithi167 ceph-mon[32751]: pgmap v21872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:42.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40103.519325 secs 2024-04-27T05:43:42.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40102.606372 secs 2024-04-27T05:43:42.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:41 smithi090 ceph-mon[29180]: pgmap v21872: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:42.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40103.519325 secs 2024-04-27T05:43:44.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:43 smithi119 ceph-mon[32752]: pgmap v21873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:44.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:43 smithi167 ceph-mon[32751]: pgmap v21873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:44.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:43 smithi090 ceph-mon[29180]: pgmap v21873: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:46.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:45 smithi119 ceph-mon[32752]: pgmap v21874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:46.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:45 smithi167 ceph-mon[32751]: pgmap v21874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:46.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:45 smithi090 ceph-mon[29180]: pgmap v21874: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:47.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40107.606435 secs 2024-04-27T05:43:47.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40108.519432 secs 2024-04-27T05:43:47.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40107.606435 secs 2024-04-27T05:43:47.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40108.519432 secs 2024-04-27T05:43:47.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40107.606435 secs 2024-04-27T05:43:47.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40108.519432 secs 2024-04-27T05:43:48.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:47 smithi119 ceph-mon[32752]: pgmap v21875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:48.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:47 smithi167 ceph-mon[32751]: pgmap v21875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:48.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:47 smithi090 ceph-mon[29180]: pgmap v21875: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:50.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:49 smithi119 ceph-mon[32752]: pgmap v21876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:50.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:49 smithi167 ceph-mon[32751]: pgmap v21876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:50.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:49 smithi090 ceph-mon[29180]: pgmap v21876: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:52.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40112.606492 secs 2024-04-27T05:43:52.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:51 smithi119 ceph-mon[32752]: pgmap v21877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:52.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40113.519549 secs 2024-04-27T05:43:52.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40112.606492 secs 2024-04-27T05:43:52.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:51 smithi167 ceph-mon[32751]: pgmap v21877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:52.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40113.519549 secs 2024-04-27T05:43:52.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40112.606492 secs 2024-04-27T05:43:52.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:51 smithi090 ceph-mon[29180]: pgmap v21877: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:52.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40113.519549 secs 2024-04-27T05:43:54.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:53 smithi119 ceph-mon[32752]: pgmap v21878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:54.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:53 smithi167 ceph-mon[32751]: pgmap v21878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:54.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:53 smithi090 ceph-mon[29180]: pgmap v21878: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:56.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:55 smithi119 ceph-mon[32752]: pgmap v21879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:56.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:55 smithi167 ceph-mon[32751]: pgmap v21879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:56.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:55 smithi090 ceph-mon[29180]: pgmap v21879: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:43:57.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40117.606552 secs 2024-04-27T05:43:57.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40118.519686 secs 2024-04-27T05:43:57.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40117.606552 secs 2024-04-27T05:43:57.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40118.519686 secs 2024-04-27T05:43:57.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40117.606552 secs 2024-04-27T05:43:57.365 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40118.519686 secs 2024-04-27T05:43:58.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:57 smithi119 ceph-mon[32752]: pgmap v21880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:58.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:57 smithi167 ceph-mon[32751]: pgmap v21880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:43:58.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:57 smithi090 ceph-mon[29180]: pgmap v21880: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:00.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:43:59 smithi119 ceph-mon[32752]: pgmap v21881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:00.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:43:59 smithi167 ceph-mon[32751]: pgmap v21881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:00.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:43:59 smithi090 ceph-mon[29180]: pgmap v21881: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:01.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:01 smithi119 ceph-mon[32752]: pgmap v21882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:01.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:01 smithi167 ceph-mon[32751]: pgmap v21882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:01.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:01 smithi090 ceph-mon[29180]: pgmap v21882: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:02.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:02 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40122.606614 secs 2024-04-27T05:44:02.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40123.519798 secs 2024-04-27T05:44:02.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:02 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40122.606614 secs 2024-04-27T05:44:02.307 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40123.519798 secs 2024-04-27T05:44:02.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:02 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40122.606614 secs 2024-04-27T05:44:02.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40123.519798 secs 2024-04-27T05:44:03.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:03 smithi119 ceph-mon[32752]: pgmap v21883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:03.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:03 smithi167 ceph-mon[32751]: pgmap v21883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:03.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:03 smithi090 ceph-mon[29180]: pgmap v21883: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:05.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:05 smithi090 ceph-mon[29180]: pgmap v21884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:05.994 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:05 smithi119 ceph-mon[32752]: pgmap v21884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:06.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:05 smithi167 ceph-mon[32751]: pgmap v21884: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:06.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40127.606676 secs 2024-04-27T05:44:06.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40128.519875 secs 2024-04-27T05:44:07.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40127.606676 secs 2024-04-27T05:44:07.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40128.519875 secs 2024-04-27T05:44:07.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40127.606676 secs 2024-04-27T05:44:07.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40128.519875 secs 2024-04-27T05:44:07.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:07 smithi090 ceph-mon[29180]: pgmap v21885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:08.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:07 smithi119 ceph-mon[32752]: pgmap v21885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:08.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:07 smithi167 ceph-mon[32751]: pgmap v21885: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:09.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:09 smithi090 ceph-mon[29180]: pgmap v21886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:10.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:09 smithi119 ceph-mon[32752]: pgmap v21886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:10.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:09 smithi167 ceph-mon[32751]: pgmap v21886: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:11.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40132.606735 secs 2024-04-27T05:44:11.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:11 smithi090 ceph-mon[29180]: pgmap v21887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:11.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40133.519937 secs 2024-04-27T05:44:12.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40132.606735 secs 2024-04-27T05:44:12.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:11 smithi119 ceph-mon[32752]: pgmap v21887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:12.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40133.519937 secs 2024-04-27T05:44:12.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40132.606735 secs 2024-04-27T05:44:12.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:11 smithi167 ceph-mon[32751]: pgmap v21887: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:12.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40133.519937 secs 2024-04-27T05:44:13.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:13 smithi090 ceph-mon[29180]: pgmap v21888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:14.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:13 smithi119 ceph-mon[32752]: pgmap v21888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:14.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:13 smithi167 ceph-mon[32751]: pgmap v21888: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:15.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:15 smithi090 ceph-mon[29180]: pgmap v21889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:16.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:15 smithi119 ceph-mon[32752]: pgmap v21889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:16.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:15 smithi167 ceph-mon[32751]: pgmap v21889: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:16.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40137.606799 secs 2024-04-27T05:44:16.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40138.520054 secs 2024-04-27T05:44:17.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40137.606799 secs 2024-04-27T05:44:17.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40138.520054 secs 2024-04-27T05:44:17.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40137.606799 secs 2024-04-27T05:44:17.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40138.520054 secs 2024-04-27T05:44:18.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:17 smithi119 ceph-mon[32752]: pgmap v21890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:18.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:17 smithi167 ceph-mon[32751]: pgmap v21890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:18.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:17 smithi090 ceph-mon[29180]: pgmap v21890: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:20.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:19 smithi119 ceph-mon[32752]: pgmap v21891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:20.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:19 smithi167 ceph-mon[32751]: pgmap v21891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:20.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:19 smithi090 ceph-mon[29180]: pgmap v21891: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:22.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40142.606860 secs 2024-04-27T05:44:22.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:21 smithi119 ceph-mon[32752]: pgmap v21892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:22.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40143.520118 secs 2024-04-27T05:44:22.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40142.606860 secs 2024-04-27T05:44:22.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:21 smithi167 ceph-mon[32751]: pgmap v21892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:22.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40143.520118 secs 2024-04-27T05:44:22.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40142.606860 secs 2024-04-27T05:44:22.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:21 smithi090 ceph-mon[29180]: pgmap v21892: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:22.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40143.520118 secs 2024-04-27T05:44:24.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:23 smithi119 ceph-mon[32752]: pgmap v21893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:24.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:23 smithi167 ceph-mon[32751]: pgmap v21893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:24.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:23 smithi090 ceph-mon[29180]: pgmap v21893: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:26.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:25 smithi119 ceph-mon[32752]: pgmap v21894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:26.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:25 smithi167 ceph-mon[32751]: pgmap v21894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:26.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:25 smithi090 ceph-mon[29180]: pgmap v21894: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:27.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40147.606914 secs 2024-04-27T05:44:27.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40148.520235 secs 2024-04-27T05:44:27.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40147.606914 secs 2024-04-27T05:44:27.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40148.520235 secs 2024-04-27T05:44:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40147.606914 secs 2024-04-27T05:44:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40148.520235 secs 2024-04-27T05:44:28.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:27 smithi119 ceph-mon[32752]: pgmap v21895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:28.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:27 smithi167 ceph-mon[32751]: pgmap v21895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:28.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:27 smithi090 ceph-mon[29180]: pgmap v21895: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:29.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:28 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:44:29.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:28 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:44:29.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:28 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:44:30.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:29 smithi090 ceph-mon[29180]: pgmap v21896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:30.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:44:30.016 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:44:30.017 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:44:30.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:29 smithi119 ceph-mon[32752]: pgmap v21896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:30.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:44:30.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:44:30.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:44:30.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:29 smithi167 ceph-mon[32751]: pgmap v21896: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:30.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:44:30.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:44:30.057 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:44:31.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:30 smithi119 ceph-mon[32752]: pgmap v21897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:44:31.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:30 smithi119 ceph-mon[32752]: pgmap v21898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:31.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:30 smithi167 ceph-mon[32751]: pgmap v21897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:44:31.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:30 smithi167 ceph-mon[32751]: pgmap v21898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:31.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:30 smithi090 ceph-mon[29180]: pgmap v21897: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:44:31.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:30 smithi090 ceph-mon[29180]: pgmap v21898: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:31.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40152.606999 secs 2024-04-27T05:44:31.864 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40153.520412 secs 2024-04-27T05:44:32.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40152.606999 secs 2024-04-27T05:44:32.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40153.520412 secs 2024-04-27T05:44:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40152.606999 secs 2024-04-27T05:44:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40153.520412 secs 2024-04-27T05:44:33.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:32 smithi119 ceph-mon[32752]: pgmap v21899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:33.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:32 smithi167 ceph-mon[32751]: pgmap v21899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:33.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:32 smithi090 ceph-mon[29180]: pgmap v21899: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:35.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:34 smithi119 ceph-mon[32752]: pgmap v21900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:35.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:34 smithi167 ceph-mon[32751]: pgmap v21900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:35.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:34 smithi090 ceph-mon[29180]: pgmap v21900: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:37.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:36 smithi119 ceph-mon[32752]: pgmap v21901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:37.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40157.607084 secs 2024-04-27T05:44:37.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40158.520565 secs 2024-04-27T05:44:37.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:36 smithi167 ceph-mon[32751]: pgmap v21901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:37.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40157.607084 secs 2024-04-27T05:44:37.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40158.520565 secs 2024-04-27T05:44:37.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:36 smithi090 ceph-mon[29180]: pgmap v21901: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:44:37.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40157.607084 secs 2024-04-27T05:44:37.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40158.520565 secs 2024-04-27T05:44:39.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:38 smithi119 ceph-mon[32752]: pgmap v21902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:44:39.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:38 smithi167 ceph-mon[32751]: pgmap v21902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:44:39.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:38 smithi090 ceph-mon[29180]: pgmap v21902: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:44:41.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:40 smithi119 ceph-mon[32752]: pgmap v21903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:44:41.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:40 smithi167 ceph-mon[32751]: pgmap v21903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:44:41.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:40 smithi090 ceph-mon[29180]: pgmap v21903: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:44:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40162.607164 secs 2024-04-27T05:44:42.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40163.520683 secs 2024-04-27T05:44:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40162.607164 secs 2024-04-27T05:44:42.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40163.520683 secs 2024-04-27T05:44:42.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40162.607164 secs 2024-04-27T05:44:42.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40163.520683 secs 2024-04-27T05:44:43.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:42 smithi119 ceph-mon[32752]: pgmap v21904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:43.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:42 smithi167 ceph-mon[32751]: pgmap v21904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:43.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:42 smithi090 ceph-mon[29180]: pgmap v21904: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:45.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:44 smithi119 ceph-mon[32752]: pgmap v21905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:45.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:44 smithi167 ceph-mon[32751]: pgmap v21905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:45.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:44 smithi090 ceph-mon[29180]: pgmap v21905: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:46 smithi119 ceph-mon[32752]: pgmap v21906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:46 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 40167.607247 secs 2024-04-27T05:44:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:46 smithi119 ceph-mon[32752]: slow request 2884.201925 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:44:47.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40168.520778 secs 2024-04-27T05:44:47.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:46 smithi167 ceph-mon[32751]: pgmap v21906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:47.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:46 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 40167.607247 secs 2024-04-27T05:44:47.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:46 smithi167 ceph-mon[32751]: slow request 2884.201925 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:44:47.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40168.520778 secs 2024-04-27T05:44:47.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:46 smithi090 ceph-mon[29180]: pgmap v21906: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:47.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:46 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 40167.607247 secs 2024-04-27T05:44:47.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:46 smithi090 ceph-mon[29180]: slow request 2884.201925 seconds old, received at 2024-04-27T04:56:41.158726+0000: client_request(client.17814:1 getattr pAsLsXsFs #0x1 2024-04-27T04:56:41.155282+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:44:47.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40168.520778 secs 2024-04-27T05:44:49.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:48 smithi119 ceph-mon[32752]: pgmap v21907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:49.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:48 smithi167 ceph-mon[32751]: pgmap v21907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:49.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:48 smithi090 ceph-mon[29180]: pgmap v21907: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:51.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:50 smithi119 ceph-mon[32752]: pgmap v21908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:51.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:50 smithi167 ceph-mon[32751]: pgmap v21908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:51.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:50 smithi090 ceph-mon[29180]: pgmap v21908: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:52.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40172.607330 secs 2024-04-27T05:44:52.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40173.520894 secs 2024-04-27T05:44:52.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40172.607330 secs 2024-04-27T05:44:52.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40173.520894 secs 2024-04-27T05:44:52.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40172.607330 secs 2024-04-27T05:44:52.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40173.520894 secs 2024-04-27T05:44:53.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:52 smithi119 ceph-mon[32752]: pgmap v21909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:53.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:52 smithi167 ceph-mon[32751]: pgmap v21909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:53.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:52 smithi090 ceph-mon[29180]: pgmap v21909: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:55.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:54 smithi119 ceph-mon[32752]: pgmap v21910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:55.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:54 smithi167 ceph-mon[32751]: pgmap v21910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:55.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:54 smithi090 ceph-mon[29180]: pgmap v21910: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:44:57.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:56 smithi119 ceph-mon[32752]: pgmap v21911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:57.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40177.607416 secs 2024-04-27T05:44:57.043 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40178.521017 secs 2024-04-27T05:44:57.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:56 smithi167 ceph-mon[32751]: pgmap v21911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:57.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40177.607416 secs 2024-04-27T05:44:57.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40178.521017 secs 2024-04-27T05:44:57.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:56 smithi090 ceph-mon[29180]: pgmap v21911: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:57.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40177.607416 secs 2024-04-27T05:44:57.115 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40178.521017 secs 2024-04-27T05:44:59.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:44:58 smithi119 ceph-mon[32752]: pgmap v21912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:59.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:44:58 smithi167 ceph-mon[32751]: pgmap v21912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:44:59.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:44:58 smithi090 ceph-mon[29180]: pgmap v21912: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:01.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:00 smithi119 ceph-mon[32752]: pgmap v21913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:01.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:00 smithi167 ceph-mon[32751]: pgmap v21913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:01.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:00 smithi090 ceph-mon[29180]: pgmap v21913: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:02.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40182.607501 secs 2024-04-27T05:45:02.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40183.521117 secs 2024-04-27T05:45:02.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40182.607501 secs 2024-04-27T05:45:02.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40183.521117 secs 2024-04-27T05:45:02.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40182.607501 secs 2024-04-27T05:45:02.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40183.521117 secs 2024-04-27T05:45:03.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:02 smithi119 ceph-mon[32752]: pgmap v21914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:03.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:02 smithi167 ceph-mon[32751]: pgmap v21914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:03.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:02 smithi090 ceph-mon[29180]: pgmap v21914: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:05.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:04 smithi119 ceph-mon[32752]: pgmap v21915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:05.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:04 smithi167 ceph-mon[32751]: pgmap v21915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:05.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:04 smithi090 ceph-mon[29180]: pgmap v21915: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:07.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:06 smithi167 ceph-mon[32751]: pgmap v21916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:07.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40187.607580 secs 2024-04-27T05:45:07.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40188.521256 secs 2024-04-27T05:45:07.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:06 smithi090 ceph-mon[29180]: pgmap v21916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:07.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40187.607580 secs 2024-04-27T05:45:07.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40188.521256 secs 2024-04-27T05:45:07.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:06 smithi119 ceph-mon[32752]: pgmap v21916: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:07.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40187.607580 secs 2024-04-27T05:45:07.295 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40188.521256 secs 2024-04-27T05:45:09.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:08 smithi167 ceph-mon[32751]: pgmap v21917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:09.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:08 smithi090 ceph-mon[29180]: pgmap v21917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:09.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:08 smithi119 ceph-mon[32752]: pgmap v21917: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:11.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:10 smithi167 ceph-mon[32751]: pgmap v21918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:11.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:10 smithi090 ceph-mon[29180]: pgmap v21918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:11.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:10 smithi119 ceph-mon[32752]: pgmap v21918: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:12.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40192.607660 secs 2024-04-27T05:45:12.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40193.521354 secs 2024-04-27T05:45:12.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40192.607660 secs 2024-04-27T05:45:12.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40193.521354 secs 2024-04-27T05:45:12.114 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:45:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:45:11.914+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:45:12.114 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:45:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:45:11.914+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:45:12.115 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:45:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:45:11.914+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:45:12.115 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:45:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:45:11.914+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:45:12.115 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:45:11 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:45:11.914+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:45:12.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40192.607660 secs 2024-04-27T05:45:12.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40193.521354 secs 2024-04-27T05:45:13.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:12 smithi090 ceph-mon[29180]: pgmap v21919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:13.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:12 smithi119 ceph-mon[32752]: pgmap v21919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:13.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:12 smithi167 ceph-mon[32751]: pgmap v21919: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:15.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:14 smithi090 ceph-mon[29180]: pgmap v21920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:15.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:14 smithi119 ceph-mon[32752]: pgmap v21920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:15.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:14 smithi167 ceph-mon[32751]: pgmap v21920: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:17.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:16 smithi090 ceph-mon[29180]: pgmap v21921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:17.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40197.607744 secs 2024-04-27T05:45:17.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40198.521426 secs 2024-04-27T05:45:17.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:16 smithi119 ceph-mon[32752]: pgmap v21921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:17.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40197.607744 secs 2024-04-27T05:45:17.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40198.521426 secs 2024-04-27T05:45:17.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:16 smithi167 ceph-mon[32751]: pgmap v21921: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:17.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40197.607744 secs 2024-04-27T05:45:17.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40198.521426 secs 2024-04-27T05:45:19.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:18 smithi090 ceph-mon[29180]: pgmap v21922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:19.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:18 smithi119 ceph-mon[32752]: pgmap v21922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:19.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:18 smithi167 ceph-mon[32751]: pgmap v21922: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:21.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:20 smithi090 ceph-mon[29180]: pgmap v21923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:21.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:20 smithi119 ceph-mon[32752]: pgmap v21923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:21.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:20 smithi167 ceph-mon[32751]: pgmap v21923: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:22.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40202.607821 secs 2024-04-27T05:45:22.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40203.521544 secs 2024-04-27T05:45:22.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40202.607821 secs 2024-04-27T05:45:22.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40203.521544 secs 2024-04-27T05:45:22.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40202.607821 secs 2024-04-27T05:45:22.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40203.521544 secs 2024-04-27T05:45:23.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:22 smithi090 ceph-mon[29180]: pgmap v21924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:23.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:22 smithi119 ceph-mon[32752]: pgmap v21924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:23.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:22 smithi167 ceph-mon[32751]: pgmap v21924: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:25.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:24 smithi090 ceph-mon[29180]: pgmap v21925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:25.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:24 smithi119 ceph-mon[32752]: pgmap v21925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:25.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:24 smithi167 ceph-mon[32751]: pgmap v21925: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:27.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:26 smithi167 ceph-mon[32751]: pgmap v21926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:27.108 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40207.607882 secs 2024-04-27T05:45:27.109 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40208.521609 secs 2024-04-27T05:45:27.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:26 smithi090 ceph-mon[29180]: pgmap v21926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40207.607882 secs 2024-04-27T05:45:27.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40208.521609 secs 2024-04-27T05:45:27.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:26 smithi119 ceph-mon[32752]: pgmap v21926: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:27.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40207.607882 secs 2024-04-27T05:45:27.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40208.521609 secs 2024-04-27T05:45:29.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:28 smithi090 ceph-mon[29180]: pgmap v21927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:29.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:28 smithi119 ceph-mon[32752]: pgmap v21927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:29.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:28 smithi167 ceph-mon[32751]: pgmap v21927: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:30.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:29 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:45:30.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:29 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:45:30.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:29 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:45:31.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:30 smithi090 ceph-mon[29180]: pgmap v21928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:31.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:45:31.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:45:31.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:30 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:45:31.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:30 smithi119 ceph-mon[32752]: pgmap v21928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:31.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:45:31.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:45:31.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:30 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:45:31.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:30 smithi167 ceph-mon[32751]: pgmap v21928: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:31.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:45:31.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:45:31.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:30 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:45:32.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:31 smithi090 ceph-mon[29180]: pgmap v21929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:45:32.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:31 smithi090 ceph-mon[29180]: pgmap v21930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:32.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40212.607969 secs 2024-04-27T05:45:32.114 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40213.521685 secs 2024-04-27T05:45:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:31 smithi119 ceph-mon[32752]: pgmap v21929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:45:32.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:31 smithi119 ceph-mon[32752]: pgmap v21930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:32.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40212.607969 secs 2024-04-27T05:45:32.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40213.521685 secs 2024-04-27T05:45:32.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:31 smithi167 ceph-mon[32751]: pgmap v21929: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:45:32.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:31 smithi167 ceph-mon[32751]: pgmap v21930: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:32.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40212.607969 secs 2024-04-27T05:45:32.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40213.521685 secs 2024-04-27T05:45:34.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:33 smithi090 ceph-mon[29180]: pgmap v21931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:45:34.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:33 smithi119 ceph-mon[32752]: pgmap v21931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:45:34.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:33 smithi167 ceph-mon[32751]: pgmap v21931: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:45:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:35 smithi119 ceph-mon[32752]: pgmap v21932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:35 smithi167 ceph-mon[32751]: pgmap v21932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:35 smithi090 ceph-mon[29180]: pgmap v21932: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40217.608028 secs 2024-04-27T05:45:37.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40218.521823 secs 2024-04-27T05:45:37.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40217.608028 secs 2024-04-27T05:45:37.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40218.521823 secs 2024-04-27T05:45:37.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40217.608028 secs 2024-04-27T05:45:37.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40218.521823 secs 2024-04-27T05:45:38.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:37 smithi119 ceph-mon[32752]: pgmap v21933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:38.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:37 smithi167 ceph-mon[32751]: pgmap v21933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:38.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:37 smithi090 ceph-mon[29180]: pgmap v21933: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:45:40.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:39 smithi119 ceph-mon[32752]: pgmap v21934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:45:40.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:39 smithi167 ceph-mon[32751]: pgmap v21934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:45:40.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:39 smithi090 ceph-mon[29180]: pgmap v21934: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:45:42.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:41 smithi119 ceph-mon[32752]: pgmap v21935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:45:42.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:41 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 40222.608090 secs 2024-04-27T05:45:42.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:41 smithi119 ceph-mon[32752]: slow request 5760.054693 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:45:42.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40223.521916 secs 2024-04-27T05:45:42.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:41 smithi167 ceph-mon[32751]: pgmap v21935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:45:42.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:41 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 40222.608090 secs 2024-04-27T05:45:42.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:41 smithi167 ceph-mon[32751]: slow request 5760.054693 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:45:42.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40223.521916 secs 2024-04-27T05:45:42.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:41 smithi090 ceph-mon[29180]: pgmap v21935: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:45:42.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:41 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 40222.608090 secs 2024-04-27T05:45:42.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:41 smithi090 ceph-mon[29180]: slow request 5760.054693 seconds old, received at 2024-04-27T04:09:40.306800+0000: client_request(client.17760:1 getattr pAsLsXsFs #0x1 2024-04-27T04:09:40.302539+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:45:42.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40223.521916 secs 2024-04-27T05:45:44.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:43 smithi119 ceph-mon[32752]: pgmap v21936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:44.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:43 smithi167 ceph-mon[32751]: pgmap v21936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:44.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:43 smithi090 ceph-mon[29180]: pgmap v21936: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:46.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:45 smithi119 ceph-mon[32752]: pgmap v21937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:46.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:45 smithi167 ceph-mon[32751]: pgmap v21937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:46.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:45 smithi090 ceph-mon[29180]: pgmap v21937: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:47.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40227.608179 secs 2024-04-27T05:45:47.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40228.522069 secs 2024-04-27T05:45:47.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40227.608179 secs 2024-04-27T05:45:47.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40228.522069 secs 2024-04-27T05:45:47.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40227.608179 secs 2024-04-27T05:45:47.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40228.522069 secs 2024-04-27T05:45:48.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:47 smithi119 ceph-mon[32752]: pgmap v21938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:48.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:47 smithi167 ceph-mon[32751]: pgmap v21938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:48.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:47 smithi090 ceph-mon[29180]: pgmap v21938: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:50.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:49 smithi119 ceph-mon[32752]: pgmap v21939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:50.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:49 smithi167 ceph-mon[32751]: pgmap v21939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:50.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:49 smithi090 ceph-mon[29180]: pgmap v21939: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:52.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:51 smithi119 ceph-mon[32752]: pgmap v21940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:52.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40232.608263 secs 2024-04-27T05:45:52.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40233.522192 secs 2024-04-27T05:45:52.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:51 smithi167 ceph-mon[32751]: pgmap v21940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:52.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40232.608263 secs 2024-04-27T05:45:52.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40233.522192 secs 2024-04-27T05:45:52.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:51 smithi090 ceph-mon[29180]: pgmap v21940: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:52.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40232.608263 secs 2024-04-27T05:45:52.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40233.522192 secs 2024-04-27T05:45:54.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:53 smithi119 ceph-mon[32752]: pgmap v21941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:54.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:53 smithi167 ceph-mon[32751]: pgmap v21941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:54.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:53 smithi090 ceph-mon[29180]: pgmap v21941: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:56.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:55 smithi119 ceph-mon[32752]: pgmap v21942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:56.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:55 smithi167 ceph-mon[32751]: pgmap v21942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:56.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:55 smithi090 ceph-mon[29180]: pgmap v21942: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:45:57.207 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40237.608345 secs 2024-04-27T05:45:57.207 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40238.522307 secs 2024-04-27T05:45:57.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40237.608345 secs 2024-04-27T05:45:57.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40238.522307 secs 2024-04-27T05:45:57.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40237.608345 secs 2024-04-27T05:45:57.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40238.522307 secs 2024-04-27T05:45:58.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:57 smithi119 ceph-mon[32752]: pgmap v21943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:58.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:57 smithi167 ceph-mon[32751]: pgmap v21943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:45:58.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:57 smithi090 ceph-mon[29180]: pgmap v21943: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:00.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:45:59 smithi119 ceph-mon[32752]: pgmap v21944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:00.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:45:59 smithi167 ceph-mon[32751]: pgmap v21944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:00.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:45:59 smithi090 ceph-mon[29180]: pgmap v21944: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:02.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:01 smithi119 ceph-mon[32752]: pgmap v21945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:02.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:01 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 40242.608428 secs 2024-04-27T05:46:02.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:01 smithi119 ceph-mon[32752]: slow request 1440.256726 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:46:02.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40243.522447 secs 2024-04-27T05:46:02.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:01 smithi167 ceph-mon[32751]: pgmap v21945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:02.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:01 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 40242.608428 secs 2024-04-27T05:46:02.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:01 smithi167 ceph-mon[32751]: slow request 1440.256726 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:46:02.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40243.522447 secs 2024-04-27T05:46:02.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:01 smithi090 ceph-mon[29180]: pgmap v21945: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:02.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:01 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 40242.608428 secs 2024-04-27T05:46:02.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:01 smithi090 ceph-mon[29180]: slow request 1440.256726 seconds old, received at 2024-04-27T05:22:00.105106+0000: client_request(client.24733:267906 getattr AsXsFs #0x100000001fb 2024-04-27T05:22:00.103554+0000 caller_uid=0, caller_gid=0{6,}) currently joining batch getattr 2024-04-27T05:46:02.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40243.522447 secs 2024-04-27T05:46:04.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:03 smithi119 ceph-mon[32752]: pgmap v21946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:04.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:03 smithi167 ceph-mon[32751]: pgmap v21946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:04.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:03 smithi090 ceph-mon[29180]: pgmap v21946: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:06.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:05 smithi119 ceph-mon[32752]: pgmap v21947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:06.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:05 smithi167 ceph-mon[32751]: pgmap v21947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:06.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:05 smithi090 ceph-mon[29180]: pgmap v21947: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:07.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40247.608511 secs 2024-04-27T05:46:07.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40248.522568 secs 2024-04-27T05:46:07.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40247.608511 secs 2024-04-27T05:46:07.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40248.522568 secs 2024-04-27T05:46:07.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40247.608511 secs 2024-04-27T05:46:07.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40248.522568 secs 2024-04-27T05:46:08.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:07 smithi119 ceph-mon[32752]: pgmap v21948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:08.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:07 smithi167 ceph-mon[32751]: pgmap v21948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:08.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:07 smithi090 ceph-mon[29180]: pgmap v21948: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:10.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:09 smithi119 ceph-mon[32752]: pgmap v21949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:10.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:09 smithi167 ceph-mon[32751]: pgmap v21949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:10.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:09 smithi090 ceph-mon[29180]: pgmap v21949: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:12.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:11 smithi119 ceph-mon[32752]: pgmap v21950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:12.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40252.608594 secs 2024-04-27T05:46:12.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40253.522687 secs 2024-04-27T05:46:12.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:11 smithi167 ceph-mon[32751]: pgmap v21950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:12.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40252.608594 secs 2024-04-27T05:46:12.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40253.522687 secs 2024-04-27T05:46:12.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:11 smithi090 ceph-mon[29180]: pgmap v21950: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:12.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40252.608594 secs 2024-04-27T05:46:12.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40253.522687 secs 2024-04-27T05:46:14.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:13 smithi119 ceph-mon[32752]: pgmap v21951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:14.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:13 smithi167 ceph-mon[32751]: pgmap v21951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:14.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:13 smithi090 ceph-mon[29180]: pgmap v21951: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:16.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:15 smithi119 ceph-mon[32752]: pgmap v21952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:16.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:15 smithi167 ceph-mon[32751]: pgmap v21952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:16.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:15 smithi090 ceph-mon[29180]: pgmap v21952: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:17.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40257.608655 secs 2024-04-27T05:46:17.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40258.522809 secs 2024-04-27T05:46:17.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40257.608655 secs 2024-04-27T05:46:17.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40258.522809 secs 2024-04-27T05:46:17.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40257.608655 secs 2024-04-27T05:46:17.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40258.522809 secs 2024-04-27T05:46:18.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:17 smithi119 ceph-mon[32752]: pgmap v21953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:18.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:17 smithi167 ceph-mon[32751]: pgmap v21953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:18.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:17 smithi090 ceph-mon[29180]: pgmap v21953: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:20.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:19 smithi119 ceph-mon[32752]: pgmap v21954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:20.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:19 smithi167 ceph-mon[32751]: pgmap v21954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:20.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:19 smithi090 ceph-mon[29180]: pgmap v21954: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:21 smithi119 ceph-mon[32752]: pgmap v21955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40262.608739 secs 2024-04-27T05:46:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40263.522930 secs 2024-04-27T05:46:22.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:21 smithi167 ceph-mon[32751]: pgmap v21955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:22.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40262.608739 secs 2024-04-27T05:46:22.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40263.522930 secs 2024-04-27T05:46:22.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:21 smithi090 ceph-mon[29180]: pgmap v21955: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:22.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40262.608739 secs 2024-04-27T05:46:22.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40263.522930 secs 2024-04-27T05:46:24.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:23 smithi119 ceph-mon[32752]: pgmap v21956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:24.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:23 smithi167 ceph-mon[32751]: pgmap v21956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:24.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:23 smithi090 ceph-mon[29180]: pgmap v21956: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:26.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:25 smithi119 ceph-mon[32752]: pgmap v21957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:26.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:25 smithi167 ceph-mon[32751]: pgmap v21957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:26.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:25 smithi090 ceph-mon[29180]: pgmap v21957: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:27.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40267.608827 secs 2024-04-27T05:46:27.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40268.523058 secs 2024-04-27T05:46:27.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40267.608827 secs 2024-04-27T05:46:27.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40268.523058 secs 2024-04-27T05:46:27.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40267.608827 secs 2024-04-27T05:46:27.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40268.523058 secs 2024-04-27T05:46:28.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:27 smithi119 ceph-mon[32752]: pgmap v21958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:28.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:27 smithi167 ceph-mon[32751]: pgmap v21958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:28.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:27 smithi090 ceph-mon[29180]: pgmap v21958: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:29.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:29 smithi119 ceph-mon[32752]: pgmap v21959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:29.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:29 smithi167 ceph-mon[32751]: pgmap v21959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:29.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:28 smithi090 ceph-mon[29180]: pgmap v21959: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:31.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:31 smithi119 ceph-mon[32752]: pgmap v21960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:31.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:46:31.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:46:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:46:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:31 smithi119 ceph-mon[32752]: pgmap v21961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:46:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:31 smithi119 ceph-mon[32752]: pgmap v21962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:31.542 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:46:31.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:31 smithi167 ceph-mon[32751]: pgmap v21960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:31.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:46:31.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:46:31.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:46:31.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:31 smithi167 ceph-mon[32751]: pgmap v21961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:46:31.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:31 smithi167 ceph-mon[32751]: pgmap v21962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:31.556 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:46:31.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:31 smithi090 ceph-mon[29180]: pgmap v21960: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:31.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:46:31.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:46:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:46:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:31 smithi090 ceph-mon[29180]: pgmap v21961: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 950 B/s rd, 1 op/s 2024-04-27T05:46:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:31 smithi090 ceph-mon[29180]: pgmap v21962: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:31.614 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:46:32.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:32 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40272.608900 secs 2024-04-27T05:46:32.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40273.523117 secs 2024-04-27T05:46:32.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:32 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40272.608900 secs 2024-04-27T05:46:32.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40273.523117 secs 2024-04-27T05:46:32.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:32 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40272.608900 secs 2024-04-27T05:46:32.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40273.523117 secs 2024-04-27T05:46:33.541 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:33 smithi119 ceph-mon[32752]: pgmap v21963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:46:33.555 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:33 smithi167 ceph-mon[32751]: pgmap v21963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:46:33.613 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:33 smithi090 ceph-mon[29180]: pgmap v21963: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 583 B/s rd, 1 op/s 2024-04-27T05:46:36.245 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:35 smithi119 ceph-mon[32752]: pgmap v21964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:35 smithi167 ceph-mon[32751]: pgmap v21964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:35 smithi090 ceph-mon[29180]: pgmap v21964: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40277.608958 secs 2024-04-27T05:46:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40278.523228 secs 2024-04-27T05:46:37.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40277.608958 secs 2024-04-27T05:46:37.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40278.523228 secs 2024-04-27T05:46:37.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40277.608958 secs 2024-04-27T05:46:37.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40278.523228 secs 2024-04-27T05:46:38.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:37 smithi119 ceph-mon[32752]: pgmap v21965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:38.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:37 smithi167 ceph-mon[32751]: pgmap v21965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:38.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:37 smithi090 ceph-mon[29180]: pgmap v21965: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:40.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:39 smithi119 ceph-mon[32752]: pgmap v21966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:40.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:39 smithi167 ceph-mon[32751]: pgmap v21966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:40.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:39 smithi090 ceph-mon[29180]: pgmap v21966: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:46:42.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40282.609042 secs 2024-04-27T05:46:42.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:41 smithi119 ceph-mon[32752]: pgmap v21967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:46:42.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40283.523348 secs 2024-04-27T05:46:42.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40282.609042 secs 2024-04-27T05:46:42.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:41 smithi167 ceph-mon[32751]: pgmap v21967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:46:42.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40283.523348 secs 2024-04-27T05:46:42.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40282.609042 secs 2024-04-27T05:46:42.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:41 smithi090 ceph-mon[29180]: pgmap v21967: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:46:42.371 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40283.523348 secs 2024-04-27T05:46:44.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:43 smithi119 ceph-mon[32752]: pgmap v21968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:44.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:43 smithi167 ceph-mon[32751]: pgmap v21968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:44.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:43 smithi090 ceph-mon[29180]: pgmap v21968: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:46.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:45 smithi119 ceph-mon[32752]: pgmap v21969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:46.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:45 smithi167 ceph-mon[32751]: pgmap v21969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:46.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:45 smithi090 ceph-mon[29180]: pgmap v21969: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:47.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40287.609126 secs 2024-04-27T05:46:47.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40288.523488 secs 2024-04-27T05:46:47.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40287.609126 secs 2024-04-27T05:46:47.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40288.523488 secs 2024-04-27T05:46:47.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40287.609126 secs 2024-04-27T05:46:47.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40288.523488 secs 2024-04-27T05:46:48.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:47 smithi119 ceph-mon[32752]: pgmap v21970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:48.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:47 smithi167 ceph-mon[32751]: pgmap v21970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:48.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:47 smithi090 ceph-mon[29180]: pgmap v21970: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:50.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:49 smithi119 ceph-mon[32752]: pgmap v21971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:50.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:49 smithi167 ceph-mon[32751]: pgmap v21971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:50.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:49 smithi090 ceph-mon[29180]: pgmap v21971: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:52.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40292.609211 secs 2024-04-27T05:46:52.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:51 smithi090 ceph-mon[29180]: pgmap v21972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:52.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40293.523570 secs 2024-04-27T05:46:52.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40292.609211 secs 2024-04-27T05:46:52.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:51 smithi119 ceph-mon[32752]: pgmap v21972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:52.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40293.523570 secs 2024-04-27T05:46:52.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40292.609211 secs 2024-04-27T05:46:52.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:51 smithi167 ceph-mon[32751]: pgmap v21972: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:52.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40293.523570 secs 2024-04-27T05:46:54.294 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:53 smithi119 ceph-mon[32752]: pgmap v21973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:54.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:53 smithi167 ceph-mon[32751]: pgmap v21973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:54.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:53 smithi090 ceph-mon[29180]: pgmap v21973: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:56.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:55 smithi119 ceph-mon[32752]: pgmap v21974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:56.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:55 smithi167 ceph-mon[32751]: pgmap v21974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:56.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:55 smithi090 ceph-mon[29180]: pgmap v21974: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:57.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40297.609290 secs 2024-04-27T05:46:57.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40298.523694 secs 2024-04-27T05:46:57.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40297.609290 secs 2024-04-27T05:46:57.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40298.523694 secs 2024-04-27T05:46:57.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40297.609290 secs 2024-04-27T05:46:57.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40298.523694 secs 2024-04-27T05:46:58.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:57 smithi119 ceph-mon[32752]: pgmap v21975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:58.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:57 smithi167 ceph-mon[32751]: pgmap v21975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:58.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:57 smithi090 ceph-mon[29180]: pgmap v21975: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:46:59.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:46:59 smithi119 ceph-mon[32752]: pgmap v21976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:59.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:46:59 smithi167 ceph-mon[32751]: pgmap v21976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:46:59.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:46:59 smithi090 ceph-mon[29180]: pgmap v21976: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:02.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40302.609349 secs 2024-04-27T05:47:02.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:01 smithi119 ceph-mon[32752]: pgmap v21977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:02.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40303.523776 secs 2024-04-27T05:47:02.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40302.609349 secs 2024-04-27T05:47:02.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:01 smithi167 ceph-mon[32751]: pgmap v21977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:02.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40303.523776 secs 2024-04-27T05:47:02.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40302.609349 secs 2024-04-27T05:47:02.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:01 smithi090 ceph-mon[29180]: pgmap v21977: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:02.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40303.523776 secs 2024-04-27T05:47:04.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:03 smithi119 ceph-mon[32752]: pgmap v21978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:04.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:03 smithi167 ceph-mon[32751]: pgmap v21978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:04.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:03 smithi090 ceph-mon[29180]: pgmap v21978: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:06.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:05 smithi119 ceph-mon[32752]: pgmap v21979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:06.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:05 smithi167 ceph-mon[32751]: pgmap v21979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:06.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:05 smithi090 ceph-mon[29180]: pgmap v21979: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:07.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40307.609432 secs 2024-04-27T05:47:07.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40308.523843 secs 2024-04-27T05:47:07.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40307.609432 secs 2024-04-27T05:47:07.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40308.523843 secs 2024-04-27T05:47:07.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40307.609432 secs 2024-04-27T05:47:07.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40308.523843 secs 2024-04-27T05:47:08.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:07 smithi119 ceph-mon[32752]: pgmap v21980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:08.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:07 smithi167 ceph-mon[32751]: pgmap v21980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:08.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:07 smithi090 ceph-mon[29180]: pgmap v21980: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:10.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:09 smithi119 ceph-mon[32752]: pgmap v21981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:10.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:09 smithi167 ceph-mon[32751]: pgmap v21981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:10.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:09 smithi090 ceph-mon[29180]: pgmap v21981: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:12.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40312.609516 secs 2024-04-27T05:47:12.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:11 smithi119 ceph-mon[32752]: pgmap v21982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:12.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40313.523937 secs 2024-04-27T05:47:12.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40312.609516 secs 2024-04-27T05:47:12.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:11 smithi167 ceph-mon[32751]: pgmap v21982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:12.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40313.523937 secs 2024-04-27T05:47:12.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40312.609516 secs 2024-04-27T05:47:12.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:11 smithi090 ceph-mon[29180]: pgmap v21982: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:12.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40313.523937 secs 2024-04-27T05:47:14.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:13 smithi119 ceph-mon[32752]: pgmap v21983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:14.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:13 smithi167 ceph-mon[32751]: pgmap v21983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:14.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:13 smithi090 ceph-mon[29180]: pgmap v21983: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:16.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:15 smithi119 ceph-mon[32752]: pgmap v21984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:16.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:15 smithi167 ceph-mon[32751]: pgmap v21984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:16.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:15 smithi090 ceph-mon[29180]: pgmap v21984: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:17.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40317.609604 secs 2024-04-27T05:47:17.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40318.524067 secs 2024-04-27T05:47:17.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40317.609604 secs 2024-04-27T05:47:17.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40318.524067 secs 2024-04-27T05:47:17.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40317.609604 secs 2024-04-27T05:47:17.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40318.524067 secs 2024-04-27T05:47:18.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:17 smithi119 ceph-mon[32752]: pgmap v21985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:18.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:17 smithi167 ceph-mon[32751]: pgmap v21985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:18.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:17 smithi090 ceph-mon[29180]: pgmap v21985: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:20.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:19 smithi119 ceph-mon[32752]: pgmap v21986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:20.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:19 smithi167 ceph-mon[32751]: pgmap v21986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:20.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:19 smithi090 ceph-mon[29180]: pgmap v21986: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40322.609690 secs 2024-04-27T05:47:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:21 smithi119 ceph-mon[32752]: pgmap v21987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40323.524196 secs 2024-04-27T05:47:22.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40322.609690 secs 2024-04-27T05:47:22.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:21 smithi167 ceph-mon[32751]: pgmap v21987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:22.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40323.524196 secs 2024-04-27T05:47:22.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40322.609690 secs 2024-04-27T05:47:22.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:21 smithi090 ceph-mon[29180]: pgmap v21987: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:22.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40323.524196 secs 2024-04-27T05:47:23.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:23 smithi119 ceph-mon[32752]: pgmap v21988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:23.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:23 smithi167 ceph-mon[32751]: pgmap v21988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:23.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:23 smithi090 ceph-mon[29180]: pgmap v21988: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:26.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:25 smithi119 ceph-mon[32752]: pgmap v21989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:26.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:25 smithi167 ceph-mon[32751]: pgmap v21989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:26.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:25 smithi090 ceph-mon[29180]: pgmap v21989: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:27.206 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40327.609753 secs 2024-04-27T05:47:27.206 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40328.524335 secs 2024-04-27T05:47:27.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40327.609753 secs 2024-04-27T05:47:27.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40328.524335 secs 2024-04-27T05:47:27.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40327.609753 secs 2024-04-27T05:47:27.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40328.524335 secs 2024-04-27T05:47:28.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:27 smithi119 ceph-mon[32752]: pgmap v21990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:28.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:27 smithi167 ceph-mon[32751]: pgmap v21990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:28.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:27 smithi090 ceph-mon[29180]: pgmap v21990: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:30.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:29 smithi119 ceph-mon[32752]: pgmap v21991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:30.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:29 smithi167 ceph-mon[32751]: pgmap v21991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:30.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:29 smithi090 ceph-mon[29180]: pgmap v21991: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:32.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40332.609804 secs 2024-04-27T05:47:32.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: pgmap v21992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:32.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:47:32.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40333.524442 secs 2024-04-27T05:47:32.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.056 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:31 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40332.609804 secs 2024-04-27T05:47:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: pgmap v21992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:47:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40333.524442 secs 2024-04-27T05:47:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:31 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40332.609804 secs 2024-04-27T05:47:32.305 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: pgmap v21992: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:32.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:47:32.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40333.524442 secs 2024-04-27T05:47:32.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:32.306 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:31 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: pgmap v21993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:33.967 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:33.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:33.968 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:33.969 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:33.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:33.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:33.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:33.970 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:33 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: pgmap v21993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:34.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:33 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: pgmap v21993: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:34.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:34.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:34.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:33 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:35.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:47:35.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:47:35.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:47:35.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:47:35.293 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:35.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:47:35.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:47:35.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:47:35.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:47:35.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:35.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi167 to 176.2M 2024-04-27T05:47:35.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi167 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-27T05:47:35.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-27T05:47:35.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:47:35.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:47:35.364 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:47:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:47:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:47:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:35 smithi119 ceph-mon[32752]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:47:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:35 smithi119 ceph-mon[32752]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:47:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:35 smithi119 ceph-mon[32752]: pgmap v21994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:47:36.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:35 smithi119 ceph-mon[32752]: pgmap v21995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:36.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:47:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:47:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:35 smithi167 ceph-mon[32751]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:47:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:35 smithi167 ceph-mon[32751]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:47:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:35 smithi167 ceph-mon[32751]: pgmap v21994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:47:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:35 smithi167 ceph-mon[32751]: pgmap v21995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi090 to 176.2M 2024-04-27T05:47:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi090 to 184764518: error parsing value: Value '184764518' is below minimum 939524096 2024-04-27T05:47:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:35 smithi090 ceph-mon[29180]: Adjusting osd_memory_target on smithi119 to 176.2M 2024-04-27T05:47:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:35 smithi090 ceph-mon[29180]: Unable to set osd_memory_target on smithi119 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-27T05:47:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:35 smithi090 ceph-mon[29180]: pgmap v21994: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:47:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:35 smithi090 ceph-mon[29180]: pgmap v21995: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:37.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40337.609889 secs 2024-04-27T05:47:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40338.524519 secs 2024-04-27T05:47:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40337.609889 secs 2024-04-27T05:47:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40338.524519 secs 2024-04-27T05:47:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40337.609889 secs 2024-04-27T05:47:37.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40338.524519 secs 2024-04-27T05:47:38.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:37 smithi090 ceph-mon[29180]: pgmap v21996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:38.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:37 smithi119 ceph-mon[32752]: pgmap v21996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:38.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:37 smithi167 ceph-mon[32751]: pgmap v21996: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:40.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:40 smithi119 ceph-mon[32752]: pgmap v21997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:40.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:40 smithi167 ceph-mon[32751]: pgmap v21997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:40.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:39 smithi090 ceph-mon[29180]: pgmap v21997: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:41.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:41 smithi119 ceph-mon[32752]: pgmap v21998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:41.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:41 smithi167 ceph-mon[32751]: pgmap v21998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:41.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:41 smithi090 ceph-mon[29180]: pgmap v21998: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:42.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:42 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40342.609950 secs 2024-04-27T05:47:42.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40343.524662 secs 2024-04-27T05:47:42.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:42 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40342.609950 secs 2024-04-27T05:47:42.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40343.524662 secs 2024-04-27T05:47:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:42 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40342.609950 secs 2024-04-27T05:47:42.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40343.524662 secs 2024-04-27T05:47:43.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:43 smithi167 ceph-mon[32751]: pgmap v21999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:43.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:43 smithi090 ceph-mon[29180]: pgmap v21999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:43.540 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:43 smithi119 ceph-mon[32752]: pgmap v21999: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:47:45.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:45 smithi119 ceph-mon[32752]: pgmap v22000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:47:45.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:45 smithi167 ceph-mon[32751]: pgmap v22000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:47:45.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:45 smithi090 ceph-mon[29180]: pgmap v22000: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:47:46.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40347.610013 secs 2024-04-27T05:47:46.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40348.524796 secs 2024-04-27T05:47:46.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40347.610013 secs 2024-04-27T05:47:46.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40348.524796 secs 2024-04-27T05:47:46.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40347.610013 secs 2024-04-27T05:47:46.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40348.524796 secs 2024-04-27T05:47:47.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:47 smithi119 ceph-mon[32752]: pgmap v22001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:47.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:47 smithi167 ceph-mon[32751]: pgmap v22001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:47.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:47 smithi090 ceph-mon[29180]: pgmap v22001: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:49.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:49 smithi119 ceph-mon[32752]: pgmap v22002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:49.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:49 smithi167 ceph-mon[32751]: pgmap v22002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:49.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:49 smithi090 ceph-mon[29180]: pgmap v22002: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:51.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40352.610097 secs 2024-04-27T05:47:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:51 smithi119 ceph-mon[32752]: pgmap v22003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:51.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40353.524923 secs 2024-04-27T05:47:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40352.610097 secs 2024-04-27T05:47:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:51 smithi167 ceph-mon[32751]: pgmap v22003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:51.805 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40353.524923 secs 2024-04-27T05:47:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40352.610097 secs 2024-04-27T05:47:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:51 smithi090 ceph-mon[29180]: pgmap v22003: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:51.863 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40353.524923 secs 2024-04-27T05:47:53.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:53 smithi119 ceph-mon[32752]: pgmap v22004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:53.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:53 smithi167 ceph-mon[32751]: pgmap v22004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:53.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:53 smithi090 ceph-mon[29180]: pgmap v22004: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:55.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:55 smithi119 ceph-mon[32752]: pgmap v22005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:55.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:55 smithi167 ceph-mon[32751]: pgmap v22005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:55.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:55 smithi090 ceph-mon[29180]: pgmap v22005: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:47:56.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40357.610155 secs 2024-04-27T05:47:56.791 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40358.525060 secs 2024-04-27T05:47:56.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40357.610155 secs 2024-04-27T05:47:56.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40358.525060 secs 2024-04-27T05:47:56.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40357.610155 secs 2024-04-27T05:47:56.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40358.525060 secs 2024-04-27T05:47:58.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:57 smithi167 ceph-mon[32751]: pgmap v22006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:58.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:57 smithi090 ceph-mon[29180]: pgmap v22006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:47:58.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:57 smithi119 ceph-mon[32752]: pgmap v22006: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:00.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:47:59 smithi167 ceph-mon[32751]: pgmap v22007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:00.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:47:59 smithi090 ceph-mon[29180]: pgmap v22007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:00.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:47:59 smithi119 ceph-mon[32752]: pgmap v22007: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:02.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40362.610239 secs 2024-04-27T05:48:02.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:01 smithi167 ceph-mon[32751]: pgmap v22008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:02.055 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40363.525127 secs 2024-04-27T05:48:02.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40362.610239 secs 2024-04-27T05:48:02.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:01 smithi090 ceph-mon[29180]: pgmap v22008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:02.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40363.525127 secs 2024-04-27T05:48:02.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40362.610239 secs 2024-04-27T05:48:02.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:01 smithi119 ceph-mon[32752]: pgmap v22008: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:02.292 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40363.525127 secs 2024-04-27T05:48:04.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:03 smithi167 ceph-mon[32751]: pgmap v22009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:04.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:03 smithi090 ceph-mon[29180]: pgmap v22009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:04.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:03 smithi119 ceph-mon[32752]: pgmap v22009: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:06.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:05 smithi167 ceph-mon[32751]: pgmap v22010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:06.092 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:05 smithi119 ceph-mon[32752]: pgmap v22010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:06.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:05 smithi090 ceph-mon[29180]: pgmap v22010: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40367.610324 secs 2024-04-27T05:48:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40368.525239 secs 2024-04-27T05:48:07.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40367.610324 secs 2024-04-27T05:48:07.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40368.525239 secs 2024-04-27T05:48:07.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40367.610324 secs 2024-04-27T05:48:07.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40368.525239 secs 2024-04-27T05:48:08.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:07 smithi090 ceph-mon[29180]: pgmap v22011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:08.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:07 smithi119 ceph-mon[32752]: pgmap v22011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:08.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:07 smithi167 ceph-mon[32751]: pgmap v22011: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:10.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:09 smithi090 ceph-mon[29180]: pgmap v22012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:10.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:09 smithi119 ceph-mon[32752]: pgmap v22012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:10.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:09 smithi167 ceph-mon[32751]: pgmap v22012: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:12.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40372.610410 secs 2024-04-27T05:48:12.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:11 smithi090 ceph-mon[29180]: pgmap v22013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:12.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40373.525360 secs 2024-04-27T05:48:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40372.610410 secs 2024-04-27T05:48:12.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:11 smithi119 ceph-mon[32752]: pgmap v22013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:12.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40373.525360 secs 2024-04-27T05:48:12.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40372.610410 secs 2024-04-27T05:48:12.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:11 smithi167 ceph-mon[32751]: pgmap v22013: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:12.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40373.525360 secs 2024-04-27T05:48:14.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:13 smithi090 ceph-mon[29180]: pgmap v22014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:14.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:13 smithi119 ceph-mon[32752]: pgmap v22014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:14.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:13 smithi167 ceph-mon[32751]: pgmap v22014: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:16.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:15 smithi090 ceph-mon[29180]: pgmap v22015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:16.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:15 smithi119 ceph-mon[32752]: pgmap v22015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:16.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:15 smithi167 ceph-mon[32751]: pgmap v22015: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40377.610488 secs 2024-04-27T05:48:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40378.525465 secs 2024-04-27T05:48:17.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40377.610488 secs 2024-04-27T05:48:17.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40378.525465 secs 2024-04-27T05:48:17.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40377.610488 secs 2024-04-27T05:48:17.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40378.525465 secs 2024-04-27T05:48:18.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:17 smithi090 ceph-mon[29180]: pgmap v22016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:18.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:17 smithi119 ceph-mon[32752]: pgmap v22016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:18.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:17 smithi167 ceph-mon[32751]: pgmap v22016: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:20.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:19 smithi090 ceph-mon[29180]: pgmap v22017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:20.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:19 smithi119 ceph-mon[32752]: pgmap v22017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:20.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:19 smithi167 ceph-mon[32751]: pgmap v22017: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:22.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40382.610574 secs 2024-04-27T05:48:22.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:21 smithi090 ceph-mon[29180]: pgmap v22018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:22.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40383.525567 secs 2024-04-27T05:48:22.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40382.610574 secs 2024-04-27T05:48:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:21 smithi119 ceph-mon[32752]: pgmap v22018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:22.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40383.525567 secs 2024-04-27T05:48:22.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40382.610574 secs 2024-04-27T05:48:22.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:21 smithi167 ceph-mon[32751]: pgmap v22018: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:22.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40383.525567 secs 2024-04-27T05:48:24.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:23 smithi090 ceph-mon[29180]: pgmap v22019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:24.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:23 smithi119 ceph-mon[32752]: pgmap v22019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:24.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:23 smithi167 ceph-mon[32751]: pgmap v22019: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:26.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:25 smithi090 ceph-mon[29180]: pgmap v22020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:26.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:25 smithi119 ceph-mon[32752]: pgmap v22020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:26.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:25 smithi167 ceph-mon[32751]: pgmap v22020: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40387.610659 secs 2024-04-27T05:48:27.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40388.525662 secs 2024-04-27T05:48:27.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40387.610659 secs 2024-04-27T05:48:27.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40388.525662 secs 2024-04-27T05:48:27.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40387.610659 secs 2024-04-27T05:48:27.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40388.525662 secs 2024-04-27T05:48:28.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:27 smithi090 ceph-mon[29180]: pgmap v22021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:28.205 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:27 smithi167 ceph-mon[32751]: pgmap v22021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:28.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:27 smithi119 ceph-mon[32752]: pgmap v22021: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:30.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:29 smithi090 ceph-mon[29180]: pgmap v22022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:30.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:29 smithi119 ceph-mon[32752]: pgmap v22022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:30.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:29 smithi167 ceph-mon[32751]: pgmap v22022: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:32.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40392.610743 secs 2024-04-27T05:48:32.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:31 smithi090 ceph-mon[29180]: pgmap v22023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:32.113 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40393.525739 secs 2024-04-27T05:48:32.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40392.610743 secs 2024-04-27T05:48:32.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:31 smithi119 ceph-mon[32752]: pgmap v22023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:32.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40393.525739 secs 2024-04-27T05:48:32.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40392.610743 secs 2024-04-27T05:48:32.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:31 smithi167 ceph-mon[32751]: pgmap v22023: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:32.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40393.525739 secs 2024-04-27T05:48:34.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:33 smithi090 ceph-mon[29180]: pgmap v22024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:34.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:33 smithi119 ceph-mon[32752]: pgmap v22024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:34.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:33 smithi167 ceph-mon[32751]: pgmap v22024: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:35.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:34 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:48:35.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:34 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:48:35.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:34 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:48:36.191 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:35 smithi119 ceph-mon[32752]: pgmap v22025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:36.192 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.192 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.192 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:48:36.192 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:48:36.192 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:35 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:35 smithi167 ceph-mon[32751]: pgmap v22025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:36.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:48:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:48:36.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:35 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:35 smithi090 ceph-mon[29180]: pgmap v22025: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:36.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:48:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:48:36.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:35 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:48:37.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40397.610849 secs 2024-04-27T05:48:37.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:36 smithi119 ceph-mon[32752]: pgmap v22026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T05:48:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:36 smithi119 ceph-mon[32752]: pgmap v22027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40398.525803 secs 2024-04-27T05:48:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40397.610849 secs 2024-04-27T05:48:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:36 smithi167 ceph-mon[32751]: pgmap v22026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T05:48:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:36 smithi167 ceph-mon[32751]: pgmap v22027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:37.305 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40398.525803 secs 2024-04-27T05:48:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40397.610849 secs 2024-04-27T05:48:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:36 smithi090 ceph-mon[29180]: pgmap v22026: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 906 B/s rd, 1 op/s 2024-04-27T05:48:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:36 smithi090 ceph-mon[29180]: pgmap v22027: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:37.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40398.525803 secs 2024-04-27T05:48:39.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:38 smithi119 ceph-mon[32752]: pgmap v22028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s rd, 1 op/s 2024-04-27T05:48:39.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:38 smithi167 ceph-mon[32751]: pgmap v22028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s rd, 1 op/s 2024-04-27T05:48:39.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:38 smithi090 ceph-mon[29180]: pgmap v22028: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 550 B/s rd, 1 op/s 2024-04-27T05:48:41.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:40 smithi119 ceph-mon[32752]: pgmap v22029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:41.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:40 smithi167 ceph-mon[32751]: pgmap v22029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:41.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:40 smithi090 ceph-mon[29180]: pgmap v22029: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40402.610932 secs 2024-04-27T05:48:42.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40403.525930 secs 2024-04-27T05:48:42.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40402.610932 secs 2024-04-27T05:48:42.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40403.525930 secs 2024-04-27T05:48:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40402.610932 secs 2024-04-27T05:48:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40403.525930 secs 2024-04-27T05:48:43.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:42 smithi119 ceph-mon[32752]: pgmap v22030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:43.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:42 smithi167 ceph-mon[32751]: pgmap v22030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:43.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:42 smithi090 ceph-mon[29180]: pgmap v22030: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:45.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:44 smithi119 ceph-mon[32752]: pgmap v22031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:45.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:44 smithi167 ceph-mon[32751]: pgmap v22031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:45.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:44 smithi090 ceph-mon[29180]: pgmap v22031: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:48:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40407.611016 secs 2024-04-27T05:48:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:46 smithi119 ceph-mon[32752]: pgmap v22032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:48:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40408.526001 secs 2024-04-27T05:48:47.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40407.611016 secs 2024-04-27T05:48:47.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:46 smithi167 ceph-mon[32751]: pgmap v22032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:48:47.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40408.526001 secs 2024-04-27T05:48:47.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40407.611016 secs 2024-04-27T05:48:47.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:46 smithi090 ceph-mon[29180]: pgmap v22032: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:48:47.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40408.526001 secs 2024-04-27T05:48:49.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:48 smithi119 ceph-mon[32752]: pgmap v22033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:49.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:48 smithi167 ceph-mon[32751]: pgmap v22033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:49.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:48 smithi090 ceph-mon[29180]: pgmap v22033: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:51.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:50 smithi119 ceph-mon[32752]: pgmap v22034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:51.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:50 smithi167 ceph-mon[32751]: pgmap v22034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:51.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:50 smithi090 ceph-mon[29180]: pgmap v22034: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:52.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40412.611077 secs 2024-04-27T05:48:52.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40413.526141 secs 2024-04-27T05:48:52.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40412.611077 secs 2024-04-27T05:48:52.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40413.526141 secs 2024-04-27T05:48:52.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40412.611077 secs 2024-04-27T05:48:52.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40413.526141 secs 2024-04-27T05:48:53.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:52 smithi119 ceph-mon[32752]: pgmap v22035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:53.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:52 smithi167 ceph-mon[32751]: pgmap v22035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:53.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:52 smithi090 ceph-mon[29180]: pgmap v22035: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:55.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:54 smithi119 ceph-mon[32752]: pgmap v22036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:55.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:54 smithi167 ceph-mon[32751]: pgmap v22036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:55.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:54 smithi090 ceph-mon[29180]: pgmap v22036: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:48:57.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40417.611160 secs 2024-04-27T05:48:57.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:56 smithi119 ceph-mon[32752]: pgmap v22037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:57.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40418.526235 secs 2024-04-27T05:48:57.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40417.611160 secs 2024-04-27T05:48:57.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:56 smithi167 ceph-mon[32751]: pgmap v22037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:57.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40418.526235 secs 2024-04-27T05:48:57.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40417.611160 secs 2024-04-27T05:48:57.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:56 smithi090 ceph-mon[29180]: pgmap v22037: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:57.321 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40418.526235 secs 2024-04-27T05:48:59.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:48:58 smithi119 ceph-mon[32752]: pgmap v22038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:59.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:48:58 smithi167 ceph-mon[32751]: pgmap v22038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:48:59.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:48:58 smithi090 ceph-mon[29180]: pgmap v22038: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:01.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:00 smithi119 ceph-mon[32752]: pgmap v22039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:01.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:00 smithi167 ceph-mon[32751]: pgmap v22039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:01.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:00 smithi090 ceph-mon[29180]: pgmap v22039: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:02.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40422.611242 secs 2024-04-27T05:49:02.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40423.526340 secs 2024-04-27T05:49:02.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40422.611242 secs 2024-04-27T05:49:02.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40423.526340 secs 2024-04-27T05:49:02.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40422.611242 secs 2024-04-27T05:49:02.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40423.526340 secs 2024-04-27T05:49:03.263 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:02 smithi090 ceph-mon[29180]: pgmap v22040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:03.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:02 smithi119 ceph-mon[32752]: pgmap v22040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:03.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:02 smithi167 ceph-mon[32751]: pgmap v22040: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:05.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:04 smithi119 ceph-mon[32752]: pgmap v22041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:05.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:04 smithi167 ceph-mon[32751]: pgmap v22041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:05.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:04 smithi090 ceph-mon[29180]: pgmap v22041: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40427.611327 secs 2024-04-27T05:49:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:06 smithi090 ceph-mon[29180]: pgmap v22042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:07.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40428.526467 secs 2024-04-27T05:49:07.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40427.611327 secs 2024-04-27T05:49:07.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:06 smithi119 ceph-mon[32752]: pgmap v22042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:07.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40428.526467 secs 2024-04-27T05:49:07.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40427.611327 secs 2024-04-27T05:49:07.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:06 smithi167 ceph-mon[32751]: pgmap v22042: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:07.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40428.526467 secs 2024-04-27T05:49:09.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:08 smithi119 ceph-mon[32752]: pgmap v22043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:09.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:08 smithi167 ceph-mon[32751]: pgmap v22043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:09.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:08 smithi090 ceph-mon[29180]: pgmap v22043: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:11.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:10 smithi119 ceph-mon[32752]: pgmap v22044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:11.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:10 smithi167 ceph-mon[32751]: pgmap v22044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:11.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:10 smithi090 ceph-mon[29180]: pgmap v22044: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40432.611412 secs 2024-04-27T05:49:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40433.526581 secs 2024-04-27T05:49:12.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40432.611412 secs 2024-04-27T05:49:12.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40433.526581 secs 2024-04-27T05:49:12.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40432.611412 secs 2024-04-27T05:49:12.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40433.526581 secs 2024-04-27T05:49:13.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:12 smithi119 ceph-mon[32752]: pgmap v22045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:13.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:12 smithi167 ceph-mon[32751]: pgmap v22045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:13.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:12 smithi090 ceph-mon[29180]: pgmap v22045: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:15.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:14 smithi119 ceph-mon[32752]: pgmap v22046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:15.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:14 smithi167 ceph-mon[32751]: pgmap v22046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:15.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:14 smithi090 ceph-mon[29180]: pgmap v22046: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:17.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40437.611515 secs 2024-04-27T05:49:17.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:16 smithi119 ceph-mon[32752]: pgmap v22047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:17.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40438.526662 secs 2024-04-27T05:49:17.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40437.611515 secs 2024-04-27T05:49:17.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:16 smithi167 ceph-mon[32751]: pgmap v22047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:17.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40438.526662 secs 2024-04-27T05:49:17.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40437.611515 secs 2024-04-27T05:49:17.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:16 smithi090 ceph-mon[29180]: pgmap v22047: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:17.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40438.526662 secs 2024-04-27T05:49:19.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:18 smithi119 ceph-mon[32752]: pgmap v22048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:19.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:18 smithi167 ceph-mon[32751]: pgmap v22048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:19.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:18 smithi090 ceph-mon[29180]: pgmap v22048: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:21.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:20 smithi119 ceph-mon[32752]: pgmap v22049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:21.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:20 smithi167 ceph-mon[32751]: pgmap v22049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:21.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:20 smithi090 ceph-mon[29180]: pgmap v22049: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:22.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40442.611574 secs 2024-04-27T05:49:22.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40443.526744 secs 2024-04-27T05:49:22.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40442.611574 secs 2024-04-27T05:49:22.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40443.526744 secs 2024-04-27T05:49:22.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40442.611574 secs 2024-04-27T05:49:22.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40443.526744 secs 2024-04-27T05:49:23.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:22 smithi119 ceph-mon[32752]: pgmap v22050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:23.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:22 smithi167 ceph-mon[32751]: pgmap v22050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:23.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:22 smithi090 ceph-mon[29180]: pgmap v22050: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:25.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:24 smithi119 ceph-mon[32752]: pgmap v22051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:25.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:24 smithi167 ceph-mon[32751]: pgmap v22051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:25.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:24 smithi090 ceph-mon[29180]: pgmap v22051: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:27.163 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40447.611674 secs 2024-04-27T05:49:27.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:26 smithi090 ceph-mon[29180]: pgmap v22052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:27.164 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40448.526893 secs 2024-04-27T05:49:27.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40447.611674 secs 2024-04-27T05:49:27.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:26 smithi119 ceph-mon[32752]: pgmap v22052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:27.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40448.526893 secs 2024-04-27T05:49:27.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40447.611674 secs 2024-04-27T05:49:27.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:26 smithi167 ceph-mon[32751]: pgmap v22052: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:27.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40448.526893 secs 2024-04-27T05:49:29.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:28 smithi119 ceph-mon[32752]: pgmap v22053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:29.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:28 smithi167 ceph-mon[32751]: pgmap v22053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:29.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:28 smithi090 ceph-mon[29180]: pgmap v22053: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:31.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:30 smithi119 ceph-mon[32752]: pgmap v22054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:31.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:30 smithi167 ceph-mon[32751]: pgmap v22054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:31.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:30 smithi090 ceph-mon[29180]: pgmap v22054: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:32.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40452.611759 secs 2024-04-27T05:49:32.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40453.526972 secs 2024-04-27T05:49:32.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40452.611759 secs 2024-04-27T05:49:32.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40453.526972 secs 2024-04-27T05:49:32.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40452.611759 secs 2024-04-27T05:49:32.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40453.526972 secs 2024-04-27T05:49:33.263 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:32 smithi090 ceph-mon[29180]: pgmap v22055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:33.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:32 smithi119 ceph-mon[32752]: pgmap v22055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:33.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:32 smithi167 ceph-mon[32751]: pgmap v22055: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:35.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:34 smithi119 ceph-mon[32752]: pgmap v22056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:35.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:34 smithi167 ceph-mon[32751]: pgmap v22056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:35.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:34 smithi090 ceph-mon[29180]: pgmap v22056: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:36.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:49:36.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:49:36.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:49:37.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:37 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40457.611823 secs 2024-04-27T05:49:37.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:37 smithi119 ceph-mon[32752]: pgmap v22057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40458.527059 secs 2024-04-27T05:49:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:49:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:49:37.291 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:49:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:37 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40457.611823 secs 2024-04-27T05:49:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:37 smithi167 ceph-mon[32751]: pgmap v22057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40458.527059 secs 2024-04-27T05:49:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:49:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:49:37.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:49:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:37 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40457.611823 secs 2024-04-27T05:49:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:37 smithi090 ceph-mon[29180]: pgmap v22057: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40458.527059 secs 2024-04-27T05:49:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:49:37.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:49:37.363 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:49:38.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:38 smithi119 ceph-mon[32752]: pgmap v22058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:49:38.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:38 smithi119 ceph-mon[32752]: pgmap v22059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:38.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:38 smithi167 ceph-mon[32751]: pgmap v22058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:49:38.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:38 smithi167 ceph-mon[32751]: pgmap v22059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:38.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:38 smithi090 ceph-mon[29180]: pgmap v22058: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:49:38.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:38 smithi090 ceph-mon[29180]: pgmap v22059: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:40.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:40 smithi119 ceph-mon[32752]: pgmap v22060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:40.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:40 smithi167 ceph-mon[32751]: pgmap v22060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:40.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:40 smithi090 ceph-mon[29180]: pgmap v22060: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:42 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40462.611921 secs 2024-04-27T05:49:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:42 smithi119 ceph-mon[32752]: pgmap v22061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:42 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 40463.527207 secs 2024-04-27T05:49:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:42 smithi119 ceph-mon[32752]: slow request 2880.048263 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:49:42.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:42 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40462.611921 secs 2024-04-27T05:49:42.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:42 smithi167 ceph-mon[32751]: pgmap v22061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:42.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:42 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 40463.527207 secs 2024-04-27T05:49:42.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:42 smithi167 ceph-mon[32751]: slow request 2880.048263 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:49:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:42 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40462.611921 secs 2024-04-27T05:49:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:42 smithi090 ceph-mon[29180]: pgmap v22061: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:42 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 40463.527207 secs 2024-04-27T05:49:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:42 smithi090 ceph-mon[29180]: slow request 2880.048263 seconds old, received at 2024-04-27T05:01:41.229589+0000: client_request(client.17820:1 getattr pAsLsXsFs #0x1 2024-04-27T05:01:41.222799+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:49:44.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:44 smithi119 ceph-mon[32752]: pgmap v22062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:49:44.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:44 smithi167 ceph-mon[32751]: pgmap v22062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:49:44.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:44 smithi090 ceph-mon[29180]: pgmap v22062: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:49:46.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:46 smithi119 ceph-mon[32752]: pgmap v22063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:46.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:46 smithi167 ceph-mon[32751]: pgmap v22063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:46.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:46 smithi090 ceph-mon[29180]: pgmap v22063: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:49:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:47 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40467.611982 secs 2024-04-27T05:49:47.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:47 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40468.527323 secs 2024-04-27T05:49:47.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:47 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40467.611982 secs 2024-04-27T05:49:47.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:47 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40468.527323 secs 2024-04-27T05:49:47.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:47 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40467.611982 secs 2024-04-27T05:49:47.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:47 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40468.527323 secs 2024-04-27T05:49:48.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:48 smithi119 ceph-mon[32752]: pgmap v22064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:49:48.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:48 smithi167 ceph-mon[32751]: pgmap v22064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:49:48.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:48 smithi090 ceph-mon[29180]: pgmap v22064: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:49:49.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:49 smithi119 ceph-mon[32752]: pgmap v22065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:49.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:49 smithi167 ceph-mon[32751]: pgmap v22065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:49.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:49 smithi090 ceph-mon[29180]: pgmap v22065: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:51.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40472.612041 secs 2024-04-27T05:49:51.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:51 smithi119 ceph-mon[32752]: pgmap v22066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:51.798 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40473.527411 secs 2024-04-27T05:49:51.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40472.612041 secs 2024-04-27T05:49:51.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:51 smithi167 ceph-mon[32751]: pgmap v22066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:51.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40473.527411 secs 2024-04-27T05:49:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40472.612041 secs 2024-04-27T05:49:51.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:51 smithi090 ceph-mon[29180]: pgmap v22066: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:51.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40473.527411 secs 2024-04-27T05:49:53.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:53 smithi119 ceph-mon[32752]: pgmap v22067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:53.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:53 smithi167 ceph-mon[32751]: pgmap v22067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:53.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:53 smithi090 ceph-mon[29180]: pgmap v22067: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:55.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:55 smithi119 ceph-mon[32752]: pgmap v22068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:55.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:55 smithi167 ceph-mon[32751]: pgmap v22068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:55.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:55 smithi090 ceph-mon[29180]: pgmap v22068: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:56.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40477.612120 secs 2024-04-27T05:49:56.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40478.527518 secs 2024-04-27T05:49:56.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40477.612120 secs 2024-04-27T05:49:56.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40478.527518 secs 2024-04-27T05:49:56.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40477.612120 secs 2024-04-27T05:49:56.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40478.527518 secs 2024-04-27T05:49:57.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:57 smithi119 ceph-mon[32752]: pgmap v22069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:57.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:57 smithi167 ceph-mon[32751]: pgmap v22069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:57.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:57 smithi090 ceph-mon[29180]: pgmap v22069: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:49:59.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:49:59 smithi119 ceph-mon[32752]: pgmap v22070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:59.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:49:59 smithi167 ceph-mon[32751]: pgmap v22070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:49:59.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:49:59 smithi090 ceph-mon[29180]: pgmap v22070: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:00.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:00 smithi167 ceph-mon[32751]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:50:00.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:00 smithi090 ceph-mon[29180]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:50:01.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:00 smithi119 ceph-mon[32752]: overall HEALTH_WARN 9 failed cephadm daemon(s); 2 MDSs report slow requests 2024-04-27T05:50:01.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40482.612184 secs 2024-04-27T05:50:01.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:01 smithi167 ceph-mon[32751]: pgmap v22071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:01.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40483.527585 secs 2024-04-27T05:50:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40482.612184 secs 2024-04-27T05:50:01.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:01 smithi090 ceph-mon[29180]: pgmap v22071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:01.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40483.527585 secs 2024-04-27T05:50:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40482.612184 secs 2024-04-27T05:50:02.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:01 smithi119 ceph-mon[32752]: pgmap v22071: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:02.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40483.527585 secs 2024-04-27T05:50:03.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:03 smithi167 ceph-mon[32751]: pgmap v22072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:03.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:03 smithi090 ceph-mon[29180]: pgmap v22072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:04.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:03 smithi119 ceph-mon[32752]: pgmap v22072: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:05.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:05 smithi167 ceph-mon[32751]: pgmap v22073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:05.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:05 smithi090 ceph-mon[29180]: pgmap v22073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:06.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:05 smithi119 ceph-mon[32752]: pgmap v22073: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:06.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40487.612268 secs 2024-04-27T05:50:06.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40488.527669 secs 2024-04-27T05:50:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40487.612268 secs 2024-04-27T05:50:07.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40488.527669 secs 2024-04-27T05:50:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40487.612268 secs 2024-04-27T05:50:07.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40488.527669 secs 2024-04-27T05:50:07.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:07 smithi090 ceph-mon[29180]: pgmap v22074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:07.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:07 smithi119 ceph-mon[32752]: pgmap v22074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:08.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:07 smithi167 ceph-mon[32751]: pgmap v22074: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:09.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:09 smithi090 ceph-mon[29180]: pgmap v22075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:10.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:09 smithi119 ceph-mon[32752]: pgmap v22075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:10.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:09 smithi167 ceph-mon[32751]: pgmap v22075: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:11.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40492.612356 secs 2024-04-27T05:50:11.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:11 smithi090 ceph-mon[29180]: pgmap v22076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:11.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40493.527788 secs 2024-04-27T05:50:12.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40492.612356 secs 2024-04-27T05:50:12.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:11 smithi119 ceph-mon[32752]: pgmap v22076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:12.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40493.527788 secs 2024-04-27T05:50:12.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40492.612356 secs 2024-04-27T05:50:12.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:11 smithi167 ceph-mon[32751]: pgmap v22076: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:12.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40493.527788 secs 2024-04-27T05:50:13.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:13 smithi090 ceph-mon[29180]: pgmap v22077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:14.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:13 smithi119 ceph-mon[32752]: pgmap v22077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:14.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:13 smithi167 ceph-mon[32751]: pgmap v22077: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:15.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:15 smithi090 ceph-mon[29180]: pgmap v22078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:16.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:15 smithi119 ceph-mon[32752]: pgmap v22078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:16.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:15 smithi167 ceph-mon[32751]: pgmap v22078: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:16.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40497.612441 secs 2024-04-27T05:50:16.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40498.527900 secs 2024-04-27T05:50:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40497.612441 secs 2024-04-27T05:50:17.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40498.527900 secs 2024-04-27T05:50:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40497.612441 secs 2024-04-27T05:50:17.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40498.527900 secs 2024-04-27T05:50:17.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:17 smithi090 ceph-mon[29180]: pgmap v22079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:18.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:17 smithi119 ceph-mon[32752]: pgmap v22079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:18.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:17 smithi167 ceph-mon[32751]: pgmap v22079: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:19.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:19 smithi090 ceph-mon[29180]: pgmap v22080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:20.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:19 smithi119 ceph-mon[32752]: pgmap v22080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:20.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:19 smithi167 ceph-mon[32751]: pgmap v22080: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:21.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40502.612526 secs 2024-04-27T05:50:21.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:21 smithi090 ceph-mon[29180]: pgmap v22081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:21.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40503.527987 secs 2024-04-27T05:50:22.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40502.612526 secs 2024-04-27T05:50:22.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:21 smithi119 ceph-mon[32752]: pgmap v22081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:22.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40503.527987 secs 2024-04-27T05:50:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40502.612526 secs 2024-04-27T05:50:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:21 smithi167 ceph-mon[32751]: pgmap v22081: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:22.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40503.527987 secs 2024-04-27T05:50:23.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:23 smithi090 ceph-mon[29180]: pgmap v22082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:24.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:23 smithi119 ceph-mon[32752]: pgmap v22082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:24.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:23 smithi167 ceph-mon[32751]: pgmap v22082: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:25.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:25 smithi090 ceph-mon[29180]: pgmap v22083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:26.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:25 smithi119 ceph-mon[32752]: pgmap v22083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:26.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:25 smithi167 ceph-mon[32751]: pgmap v22083: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40507.612608 secs 2024-04-27T05:50:26.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40508.528136 secs 2024-04-27T05:50:27.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40507.612608 secs 2024-04-27T05:50:27.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40508.528136 secs 2024-04-27T05:50:27.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40507.612608 secs 2024-04-27T05:50:27.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40508.528136 secs 2024-04-27T05:50:27.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:27 smithi090 ceph-mon[29180]: pgmap v22084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:28.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:27 smithi119 ceph-mon[32752]: pgmap v22084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:28.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:27 smithi167 ceph-mon[32751]: pgmap v22084: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:29.847 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:29 smithi090 ceph-mon[29180]: pgmap v22085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:29.856 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:29 smithi167 ceph-mon[32751]: pgmap v22085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:30.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:29 smithi119 ceph-mon[32752]: pgmap v22085: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:31.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40512.612691 secs 2024-04-27T05:50:31.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:31 smithi090 ceph-mon[29180]: pgmap v22086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:31.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40513.528292 secs 2024-04-27T05:50:32.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40512.612691 secs 2024-04-27T05:50:32.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:31 smithi119 ceph-mon[32752]: pgmap v22086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:32.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40513.528292 secs 2024-04-27T05:50:32.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40512.612691 secs 2024-04-27T05:50:32.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:31 smithi167 ceph-mon[32751]: pgmap v22086: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:32.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40513.528292 secs 2024-04-27T05:50:33.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:33 smithi090 ceph-mon[29180]: pgmap v22087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:34.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:33 smithi119 ceph-mon[32752]: pgmap v22087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:34.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:33 smithi167 ceph-mon[32751]: pgmap v22087: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:35.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:35 smithi090 ceph-mon[29180]: pgmap v22088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:36.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:35 smithi119 ceph-mon[32752]: pgmap v22088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:36.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:35 smithi167 ceph-mon[32751]: pgmap v22088: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:50:37.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40517.612777 secs 2024-04-27T05:50:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40518.528450 secs 2024-04-27T05:50:37.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:36 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:50:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40517.612777 secs 2024-04-27T05:50:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40518.528450 secs 2024-04-27T05:50:37.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:36 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:50:37.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40517.612777 secs 2024-04-27T05:50:37.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40518.528450 secs 2024-04-27T05:50:37.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:36 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:50:37.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:37 smithi119 ceph-mon[32752]: pgmap v22089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:37.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:50:37.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:50:37.941 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:50:38.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:37 smithi167 ceph-mon[32751]: pgmap v22089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:50:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:50:38.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:50:38.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:37 smithi090 ceph-mon[29180]: pgmap v22089: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:38.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:50:38.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:50:38.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:50:39.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:38 smithi119 ceph-mon[32752]: pgmap v22090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:50:39.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:38 smithi119 ceph-mon[32752]: pgmap v22091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:50:39.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:38 smithi167 ceph-mon[32751]: pgmap v22090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:50:39.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:38 smithi167 ceph-mon[32751]: pgmap v22091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:50:39.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:38 smithi090 ceph-mon[29180]: pgmap v22090: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 948 B/s rd, 1 op/s 2024-04-27T05:50:39.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:38 smithi090 ceph-mon[29180]: pgmap v22091: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 581 B/s rd, 1 op/s 2024-04-27T05:50:41.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:40 smithi119 ceph-mon[32752]: pgmap v22092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:50:41.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:40 smithi167 ceph-mon[32751]: pgmap v22092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:50:41.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:40 smithi090 ceph-mon[29180]: pgmap v22092: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:50:42.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40522.612864 secs 2024-04-27T05:50:42.015 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40523.528590 secs 2024-04-27T05:50:42.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40522.612864 secs 2024-04-27T05:50:42.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40523.528590 secs 2024-04-27T05:50:42.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40522.612864 secs 2024-04-27T05:50:42.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40523.528590 secs 2024-04-27T05:50:42.361 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:50:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:50:42.027+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:50:42.362 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:50:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:50:42.027+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:50:42.362 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:50:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:50:42.027+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:50:42.362 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:50:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:50:42.027+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:50:42.362 INFO:journalctl@ceph.mgr.x.smithi090.stdout:Apr 27 05:50:42 smithi090 ceph-b8878a18-03f6-11ef-bc93-c7b262605968-mgr-x[29406]: 2024-04-27T05:50:42.027+0000 7f8240fb0700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-27T05:50:43.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:42 smithi119 ceph-mon[32752]: pgmap v22093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:50:43.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:42 smithi167 ceph-mon[32751]: pgmap v22093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:50:43.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:42 smithi090 ceph-mon[29180]: pgmap v22093: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:50:45.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:44 smithi119 ceph-mon[32752]: pgmap v22094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s 2024-04-27T05:50:45.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:44 smithi167 ceph-mon[32751]: pgmap v22094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s 2024-04-27T05:50:45.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:44 smithi090 ceph-mon[29180]: pgmap v22094: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2 op/s 2024-04-27T05:50:47.041 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:46 smithi119 ceph-mon[32752]: pgmap v22095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T05:50:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40527.612924 secs 2024-04-27T05:50:47.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40528.528733 secs 2024-04-27T05:50:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:46 smithi167 ceph-mon[32751]: pgmap v22095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T05:50:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40527.612924 secs 2024-04-27T05:50:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40528.528733 secs 2024-04-27T05:50:47.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:46 smithi090 ceph-mon[29180]: pgmap v22095: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 0 B/s wr, 17 op/s 2024-04-27T05:50:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40527.612924 secs 2024-04-27T05:50:47.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40528.528733 secs 2024-04-27T05:50:49.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:48 smithi119 ceph-mon[32752]: pgmap v22096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-27T05:50:49.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:48 smithi167 ceph-mon[32751]: pgmap v22096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-27T05:50:49.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:48 smithi090 ceph-mon[29180]: pgmap v22096: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-27T05:50:51.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:50 smithi119 ceph-mon[32752]: pgmap v22097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:50:51.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:50 smithi167 ceph-mon[32751]: pgmap v22097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:50:51.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:50 smithi090 ceph-mon[29180]: pgmap v22097: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:50:52.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40532.613009 secs 2024-04-27T05:50:52.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40533.528871 secs 2024-04-27T05:50:52.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40532.613009 secs 2024-04-27T05:50:52.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40533.528871 secs 2024-04-27T05:50:52.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40532.613009 secs 2024-04-27T05:50:52.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40533.528871 secs 2024-04-27T05:50:53.042 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:52 smithi119 ceph-mon[32752]: pgmap v22098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:50:53.054 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:52 smithi167 ceph-mon[32751]: pgmap v22098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:50:53.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:52 smithi090 ceph-mon[29180]: pgmap v22098: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:50:55.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:54 smithi119 ceph-mon[32752]: pgmap v22099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:50:55.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:54 smithi167 ceph-mon[32751]: pgmap v22099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:50:55.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:54 smithi090 ceph-mon[29180]: pgmap v22099: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-27T05:50:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:56 smithi119 ceph-mon[32752]: pgmap v22100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:50:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40537.613093 secs 2024-04-27T05:50:57.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40538.528949 secs 2024-04-27T05:50:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:56 smithi167 ceph-mon[32751]: pgmap v22100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:50:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40537.613093 secs 2024-04-27T05:50:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40538.528949 secs 2024-04-27T05:50:57.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:56 smithi090 ceph-mon[29180]: pgmap v22100: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-27T05:50:57.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40537.613093 secs 2024-04-27T05:50:57.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40538.528949 secs 2024-04-27T05:50:59.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:50:58 smithi119 ceph-mon[32752]: pgmap v22101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:59.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:50:58 smithi167 ceph-mon[32751]: pgmap v22101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:50:59.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:50:58 smithi090 ceph-mon[29180]: pgmap v22101: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:01.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:00 smithi119 ceph-mon[32752]: pgmap v22102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:01.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:00 smithi167 ceph-mon[32751]: pgmap v22102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:01.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:00 smithi090 ceph-mon[29180]: pgmap v22102: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40542.613180 secs 2024-04-27T05:51:01.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40543.529097 secs 2024-04-27T05:51:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40542.613180 secs 2024-04-27T05:51:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40543.529097 secs 2024-04-27T05:51:02.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40542.613180 secs 2024-04-27T05:51:02.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40543.529097 secs 2024-04-27T05:51:03.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:02 smithi119 ceph-mon[32752]: pgmap v22103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:03.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:02 smithi167 ceph-mon[32751]: pgmap v22103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:03.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:02 smithi090 ceph-mon[29180]: pgmap v22103: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:05.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:04 smithi119 ceph-mon[32752]: pgmap v22104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:51:05.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:04 smithi167 ceph-mon[32751]: pgmap v22104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:51:05.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:04 smithi090 ceph-mon[29180]: pgmap v22104: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-04-27T05:51:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:06 smithi119 ceph-mon[32752]: pgmap v22105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40547.613269 secs 2024-04-27T05:51:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40548.529192 secs 2024-04-27T05:51:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:06 smithi167 ceph-mon[32751]: pgmap v22105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40547.613269 secs 2024-04-27T05:51:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40548.529192 secs 2024-04-27T05:51:07.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:06 smithi090 ceph-mon[29180]: pgmap v22105: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:07.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40547.613269 secs 2024-04-27T05:51:07.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40548.529192 secs 2024-04-27T05:51:09.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:08 smithi119 ceph-mon[32752]: pgmap v22106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:51:09.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:08 smithi167 ceph-mon[32751]: pgmap v22106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:51:09.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:08 smithi090 ceph-mon[29180]: pgmap v22106: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:51:11.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:10 smithi119 ceph-mon[32752]: pgmap v22107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:11.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:10 smithi167 ceph-mon[32751]: pgmap v22107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:11.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:10 smithi090 ceph-mon[29180]: pgmap v22107: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:12.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:11 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40552.613377 secs 2024-04-27T05:51:12.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40553.529289 secs 2024-04-27T05:51:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:11 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40552.613377 secs 2024-04-27T05:51:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40553.529289 secs 2024-04-27T05:51:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:11 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40552.613377 secs 2024-04-27T05:51:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40553.529289 secs 2024-04-27T05:51:13.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:12 smithi090 ceph-mon[29180]: pgmap v22108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:51:13.124 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:12 smithi167 ceph-mon[32751]: pgmap v22108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:51:13.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:12 smithi119 ceph-mon[32752]: pgmap v22108: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-04-27T05:51:15.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:14 smithi090 ceph-mon[29180]: pgmap v22109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:15.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:14 smithi119 ceph-mon[32752]: pgmap v22109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:15.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:14 smithi167 ceph-mon[32751]: pgmap v22109: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:17.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:16 smithi090 ceph-mon[29180]: pgmap v22110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40557.613486 secs 2024-04-27T05:51:17.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40558.529414 secs 2024-04-27T05:51:17.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:16 smithi119 ceph-mon[32752]: pgmap v22110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:17.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40557.613486 secs 2024-04-27T05:51:17.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40558.529414 secs 2024-04-27T05:51:17.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:16 smithi167 ceph-mon[32751]: pgmap v22110: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-27T05:51:17.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40557.613486 secs 2024-04-27T05:51:17.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40558.529414 secs 2024-04-27T05:51:19.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:18 smithi090 ceph-mon[29180]: pgmap v22111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:19.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:18 smithi119 ceph-mon[32752]: pgmap v22111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:19.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:18 smithi167 ceph-mon[32751]: pgmap v22111: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:21.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:20 smithi090 ceph-mon[29180]: pgmap v22112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:21.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:20 smithi119 ceph-mon[32752]: pgmap v22112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:21.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:20 smithi167 ceph-mon[32751]: pgmap v22112: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40562.613591 secs 2024-04-27T05:51:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40563.529484 secs 2024-04-27T05:51:22.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40562.613591 secs 2024-04-27T05:51:22.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40563.529484 secs 2024-04-27T05:51:22.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40562.613591 secs 2024-04-27T05:51:22.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40563.529484 secs 2024-04-27T05:51:23.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:22 smithi090 ceph-mon[29180]: pgmap v22113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:23.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:22 smithi119 ceph-mon[32752]: pgmap v22113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:23.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:22 smithi167 ceph-mon[32751]: pgmap v22113: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:25.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:24 smithi090 ceph-mon[29180]: pgmap v22114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:25.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:24 smithi119 ceph-mon[32752]: pgmap v22114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:25.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:24 smithi167 ceph-mon[32751]: pgmap v22114: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:26 smithi090 ceph-mon[29180]: pgmap v22115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T05:51:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40567.613702 secs 2024-04-27T05:51:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40568.529580 secs 2024-04-27T05:51:27.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:26 smithi119 ceph-mon[32752]: pgmap v22115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T05:51:27.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40567.613702 secs 2024-04-27T05:51:27.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40568.529580 secs 2024-04-27T05:51:27.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:26 smithi167 ceph-mon[32751]: pgmap v22115: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T05:51:27.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40567.613702 secs 2024-04-27T05:51:27.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40568.529580 secs 2024-04-27T05:51:29.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:28 smithi119 ceph-mon[32752]: pgmap v22116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:29.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:28 smithi167 ceph-mon[32751]: pgmap v22116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:29.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:28 smithi090 ceph-mon[29180]: pgmap v22116: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:31.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:30 smithi119 ceph-mon[32752]: pgmap v22117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:31.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:30 smithi167 ceph-mon[32751]: pgmap v22117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:31.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:30 smithi090 ceph-mon[29180]: pgmap v22117: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:32.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:31 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40572.613806 secs 2024-04-27T05:51:32.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40573.529679 secs 2024-04-27T05:51:32.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:31 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40572.613806 secs 2024-04-27T05:51:32.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40573.529679 secs 2024-04-27T05:51:32.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:31 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40572.613806 secs 2024-04-27T05:51:32.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40573.529679 secs 2024-04-27T05:51:33.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:32 smithi119 ceph-mon[32752]: pgmap v22118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:33.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:32 smithi167 ceph-mon[32751]: pgmap v22118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:33.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:32 smithi090 ceph-mon[29180]: pgmap v22118: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:35.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:34 smithi119 ceph-mon[32752]: pgmap v22119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:35.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:34 smithi167 ceph-mon[32751]: pgmap v22119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:35.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:34 smithi090 ceph-mon[29180]: pgmap v22119: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:37.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:36 smithi119 ceph-mon[32752]: pgmap v22120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T05:51:37.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:36 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40577.613907 secs 2024-04-27T05:51:37.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40578.529804 secs 2024-04-27T05:51:37.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:36 smithi167 ceph-mon[32751]: pgmap v22120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T05:51:37.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:36 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40577.613907 secs 2024-04-27T05:51:37.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40578.529804 secs 2024-04-27T05:51:37.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:36 smithi090 ceph-mon[29180]: pgmap v22120: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-04-27T05:51:37.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:36 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40577.613907 secs 2024-04-27T05:51:37.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40578.529804 secs 2024-04-27T05:51:38.190 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:37 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:51:38.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:37 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:51:38.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:37 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:51:39.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:38 smithi119 ceph-mon[32752]: pgmap v22121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:39.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:51:39.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:51:39.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:38 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:51:39.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:38 smithi167 ceph-mon[32751]: pgmap v22121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:39.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:51:39.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:51:39.340 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:38 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:51:39.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:38 smithi090 ceph-mon[29180]: pgmap v22121: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:39.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:51:39.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:51:39.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:38 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:51:40.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:39 smithi119 ceph-mon[32752]: pgmap v22122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:51:40.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:39 smithi119 ceph-mon[32752]: pgmap v22123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:51:40.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:39 smithi167 ceph-mon[32751]: pgmap v22122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:51:40.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:39 smithi167 ceph-mon[32751]: pgmap v22123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:51:40.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:39 smithi090 ceph-mon[29180]: pgmap v22122: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 951 B/s rd, 1 op/s 2024-04-27T05:51:40.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:39 smithi090 ceph-mon[29180]: pgmap v22123: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:51:42.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:41 smithi119 ceph-mon[32752]: pgmap v22124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:42.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:41 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40582.614006 secs 2024-04-27T05:51:42.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40583.529935 secs 2024-04-27T05:51:42.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:41 smithi167 ceph-mon[32751]: pgmap v22124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:42.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:41 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40582.614006 secs 2024-04-27T05:51:42.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40583.529935 secs 2024-04-27T05:51:42.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:41 smithi090 ceph-mon[29180]: pgmap v22124: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:42.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:41 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40582.614006 secs 2024-04-27T05:51:42.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40583.529935 secs 2024-04-27T05:51:44.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:43 smithi119 ceph-mon[32752]: pgmap v22125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:51:44.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:43 smithi167 ceph-mon[32751]: pgmap v22125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:51:44.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:43 smithi090 ceph-mon[29180]: pgmap v22125: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 584 B/s rd, 1 op/s 2024-04-27T05:51:46.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:45 smithi119 ceph-mon[32752]: pgmap v22126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:46.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:45 smithi167 ceph-mon[32751]: pgmap v22126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:46.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:45 smithi090 ceph-mon[29180]: pgmap v22126: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:47.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:46 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40587.614103 secs 2024-04-27T05:51:47.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40588.530071 secs 2024-04-27T05:51:47.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:46 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40587.614103 secs 2024-04-27T05:51:47.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40588.530071 secs 2024-04-27T05:51:47.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:46 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40587.614103 secs 2024-04-27T05:51:47.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40588.530071 secs 2024-04-27T05:51:48.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:47 smithi090 ceph-mon[29180]: pgmap v22127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:48.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:47 smithi119 ceph-mon[32752]: pgmap v22127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:48.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:47 smithi167 ceph-mon[32751]: pgmap v22127: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-27T05:51:50.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:49 smithi119 ceph-mon[32752]: pgmap v22128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:51:50.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:49 smithi167 ceph-mon[32751]: pgmap v22128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:51:50.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:49 smithi090 ceph-mon[29180]: pgmap v22128: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:51:52.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:51 smithi119 ceph-mon[32752]: pgmap v22129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:52.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:51 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40592.614187 secs 2024-04-27T05:51:52.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40593.530210 secs 2024-04-27T05:51:52.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:51 smithi167 ceph-mon[32751]: pgmap v22129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:52.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:51 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40592.614187 secs 2024-04-27T05:51:52.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40593.530210 secs 2024-04-27T05:51:52.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:51 smithi090 ceph-mon[29180]: pgmap v22129: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:52.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:51 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40592.614187 secs 2024-04-27T05:51:52.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40593.530210 secs 2024-04-27T05:51:54.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:53 smithi119 ceph-mon[32752]: pgmap v22130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:54.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:53 smithi167 ceph-mon[32751]: pgmap v22130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:54.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:53 smithi090 ceph-mon[29180]: pgmap v22130: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:56.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:55 smithi119 ceph-mon[32752]: pgmap v22131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:56.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:55 smithi167 ceph-mon[32751]: pgmap v22131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:56.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:55 smithi090 ceph-mon[29180]: pgmap v22131: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:51:57.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:56 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40597.614298 secs 2024-04-27T05:51:57.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40598.530342 secs 2024-04-27T05:51:57.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:56 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40597.614298 secs 2024-04-27T05:51:57.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40598.530342 secs 2024-04-27T05:51:57.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:56 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40597.614298 secs 2024-04-27T05:51:57.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40598.530342 secs 2024-04-27T05:51:58.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:57 smithi119 ceph-mon[32752]: pgmap v22132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:58.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:57 smithi167 ceph-mon[32751]: pgmap v22132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:51:58.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:57 smithi090 ceph-mon[29180]: pgmap v22132: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:00.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:51:59 smithi119 ceph-mon[32752]: pgmap v22133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:00.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:51:59 smithi167 ceph-mon[32751]: pgmap v22133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:00.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:51:59 smithi090 ceph-mon[29180]: pgmap v22133: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:02.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:01 smithi119 ceph-mon[32752]: pgmap v22134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:02.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:01 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40602.614395 secs 2024-04-27T05:52:02.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40603.530469 secs 2024-04-27T05:52:02.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:01 smithi167 ceph-mon[32751]: pgmap v22134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:02.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:01 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40602.614395 secs 2024-04-27T05:52:02.304 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40603.530469 secs 2024-04-27T05:52:02.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:01 smithi090 ceph-mon[29180]: pgmap v22134: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:02.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:01 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40602.614395 secs 2024-04-27T05:52:02.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40603.530469 secs 2024-04-27T05:52:04.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:03 smithi119 ceph-mon[32752]: pgmap v22135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:04.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:03 smithi167 ceph-mon[32751]: pgmap v22135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:04.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:03 smithi090 ceph-mon[29180]: pgmap v22135: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:06.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:05 smithi119 ceph-mon[32752]: pgmap v22136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:06.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:05 smithi167 ceph-mon[32751]: pgmap v22136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:06.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:05 smithi090 ceph-mon[29180]: pgmap v22136: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:07.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:06 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40607.614495 secs 2024-04-27T05:52:07.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:06 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40608.530590 secs 2024-04-27T05:52:07.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:06 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40607.614495 secs 2024-04-27T05:52:07.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:06 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40608.530590 secs 2024-04-27T05:52:07.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:06 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40607.614495 secs 2024-04-27T05:52:07.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:06 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40608.530590 secs 2024-04-27T05:52:08.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:07 smithi119 ceph-mon[32752]: pgmap v22137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:08.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:07 smithi167 ceph-mon[32751]: pgmap v22137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:08.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:07 smithi090 ceph-mon[29180]: pgmap v22137: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:10.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:09 smithi119 ceph-mon[32752]: pgmap v22138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:10.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:09 smithi167 ceph-mon[32751]: pgmap v22138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:10.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:09 smithi090 ceph-mon[29180]: pgmap v22138: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:12.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:11 smithi119 ceph-mon[32752]: pgmap v22139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:11 smithi119 ceph-mon[32752]: 51 slow requests, 1 included below; oldest blocked for > 40612.614598 secs 2024-04-27T05:52:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:11 smithi119 ceph-mon[32752]: slow request 11521.542016 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:52:12.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40613.530704 secs 2024-04-27T05:52:12.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:11 smithi167 ceph-mon[32751]: pgmap v22139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:12.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:11 smithi167 ceph-mon[32751]: 51 slow requests, 1 included below; oldest blocked for > 40612.614598 secs 2024-04-27T05:52:12.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:11 smithi167 ceph-mon[32751]: slow request 11521.542016 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:52:12.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40613.530704 secs 2024-04-27T05:52:12.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:11 smithi090 ceph-mon[29180]: pgmap v22139: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:12.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:11 smithi090 ceph-mon[29180]: 51 slow requests, 1 included below; oldest blocked for > 40612.614598 secs 2024-04-27T05:52:12.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:11 smithi090 ceph-mon[29180]: slow request 11521.542016 seconds old, received at 2024-04-27T02:40:08.825985+0000: client_request(client.26972:1 getattr pAsLsXsFs #0x1 2024-04-27T02:40:08.819526+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:52:12.362 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40613.530704 secs 2024-04-27T05:52:14.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:13 smithi119 ceph-mon[32752]: pgmap v22140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:14.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:13 smithi167 ceph-mon[32751]: pgmap v22140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:14.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:13 smithi090 ceph-mon[29180]: pgmap v22140: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:16.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:15 smithi119 ceph-mon[32752]: pgmap v22141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:16.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:15 smithi167 ceph-mon[32751]: pgmap v22141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:16.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:15 smithi090 ceph-mon[29180]: pgmap v22141: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:17.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:16 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40617.614699 secs 2024-04-27T05:52:17.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40618.530786 secs 2024-04-27T05:52:17.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:16 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40617.614699 secs 2024-04-27T05:52:17.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40618.530786 secs 2024-04-27T05:52:17.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:16 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40617.614699 secs 2024-04-27T05:52:17.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40618.530786 secs 2024-04-27T05:52:18.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:17 smithi119 ceph-mon[32752]: pgmap v22142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:18.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:17 smithi167 ceph-mon[32751]: pgmap v22142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:18.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:17 smithi090 ceph-mon[29180]: pgmap v22142: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:20.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:19 smithi119 ceph-mon[32752]: pgmap v22143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:20.306 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:19 smithi167 ceph-mon[32751]: pgmap v22143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:20.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:19 smithi090 ceph-mon[29180]: pgmap v22143: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:22.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:21 smithi119 ceph-mon[32752]: pgmap v22144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:22.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:21 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40622.614804 secs 2024-04-27T05:52:22.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40623.530915 secs 2024-04-27T05:52:22.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:21 smithi167 ceph-mon[32751]: pgmap v22144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:22.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:21 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40622.614804 secs 2024-04-27T05:52:22.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40623.530915 secs 2024-04-27T05:52:22.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:21 smithi090 ceph-mon[29180]: pgmap v22144: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:22.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:21 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40622.614804 secs 2024-04-27T05:52:22.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40623.530915 secs 2024-04-27T05:52:24.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:23 smithi119 ceph-mon[32752]: pgmap v22145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:24.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:23 smithi167 ceph-mon[32751]: pgmap v22145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:24.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:23 smithi090 ceph-mon[29180]: pgmap v22145: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:26.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:25 smithi119 ceph-mon[32752]: pgmap v22146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:26.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:25 smithi167 ceph-mon[32751]: pgmap v22146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:26.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:25 smithi090 ceph-mon[29180]: pgmap v22146: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:27.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:26 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40627.614892 secs 2024-04-27T05:52:27.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40628.531053 secs 2024-04-27T05:52:27.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:26 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40627.614892 secs 2024-04-27T05:52:27.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40628.531053 secs 2024-04-27T05:52:27.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:26 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40627.614892 secs 2024-04-27T05:52:27.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40628.531053 secs 2024-04-27T05:52:28.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:27 smithi119 ceph-mon[32752]: pgmap v22147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:28.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:27 smithi167 ceph-mon[32751]: pgmap v22147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:28.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:27 smithi090 ceph-mon[29180]: pgmap v22147: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:30.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:30 smithi119 ceph-mon[32752]: pgmap v22148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:30.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:30 smithi167 ceph-mon[32751]: pgmap v22148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:30.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:30 smithi090 ceph-mon[29180]: pgmap v22148: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:32.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:32 smithi119 ceph-mon[32752]: pgmap v22149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:32.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:32 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40632.614978 secs 2024-04-27T05:52:32.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:32 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40633.531181 secs 2024-04-27T05:52:32.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:32 smithi167 ceph-mon[32751]: pgmap v22149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:32.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:32 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40632.614978 secs 2024-04-27T05:52:32.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:32 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40633.531181 secs 2024-04-27T05:52:32.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:32 smithi090 ceph-mon[29180]: pgmap v22149: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:32.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:32 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40632.614978 secs 2024-04-27T05:52:32.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:32 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40633.531181 secs 2024-04-27T05:52:34.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:34 smithi090 ceph-mon[29180]: pgmap v22150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:34.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:34 smithi119 ceph-mon[32752]: pgmap v22150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:34.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:34 smithi167 ceph-mon[32751]: pgmap v22150: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:36.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:36 smithi119 ceph-mon[32752]: pgmap v22151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:36.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:36 smithi167 ceph-mon[32751]: pgmap v22151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:36.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:36 smithi090 ceph-mon[29180]: pgmap v22151: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:37.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:37 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40637.615076 secs 2024-04-27T05:52:37.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:37 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40638.531299 secs 2024-04-27T05:52:37.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:37 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40637.615076 secs 2024-04-27T05:52:37.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:37 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40638.531299 secs 2024-04-27T05:52:37.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:37 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40637.615076 secs 2024-04-27T05:52:37.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:37 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40638.531299 secs 2024-04-27T05:52:38.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:38 smithi119 ceph-mon[32752]: pgmap v22152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:38.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:38 smithi167 ceph-mon[32751]: pgmap v22152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:38.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:38 smithi090 ceph-mon[29180]: pgmap v22152: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:39.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:39 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:52:39.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:39 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:52:39.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:39 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:52:40.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:40 smithi119 ceph-mon[32752]: pgmap v22153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:40.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:40 smithi167 ceph-mon[32751]: pgmap v22153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:40.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:40 smithi090 ceph-mon[29180]: pgmap v22153: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:42 smithi090 ceph-mon[29180]: pgmap v22154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:42 smithi090 ceph-mon[29180]: 51 slow requests, 0 included below; oldest blocked for > 40642.615139 secs 2024-04-27T05:52:42.112 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:42 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40643.531409 secs 2024-04-27T05:52:42.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:42 smithi119 ceph-mon[32752]: pgmap v22154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:42 smithi119 ceph-mon[32752]: 51 slow requests, 0 included below; oldest blocked for > 40642.615139 secs 2024-04-27T05:52:42.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:42 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40643.531409 secs 2024-04-27T05:52:42.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:42 smithi167 ceph-mon[32751]: pgmap v22154: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:42.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:42 smithi167 ceph-mon[32751]: 51 slow requests, 0 included below; oldest blocked for > 40642.615139 secs 2024-04-27T05:52:42.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:42 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40643.531409 secs 2024-04-27T05:52:43.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: pgmap v22155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:43.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.789 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:52:43.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:52:43.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: pgmap v22156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 979 B/s rd, 1 op/s 2024-04-27T05:52:43.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: pgmap v22157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:52:43.790 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:43 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: pgmap v22155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:43.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:52:43.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:52:43.803 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: pgmap v22156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 979 B/s rd, 1 op/s 2024-04-27T05:52:43.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: pgmap v22157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:52:43.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:43 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: pgmap v22155: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:43.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:43.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:52:43.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:52:43.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: pgmap v22156: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 979 B/s rd, 1 op/s 2024-04-27T05:52:43.862 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: pgmap v22157: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 605 B/s rd, 1 op/s 2024-04-27T05:52:43.865 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:43 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:52:45.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:45 smithi090 ceph-mon[29180]: pgmap v22158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:46.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:45 smithi119 ceph-mon[32752]: pgmap v22158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:46.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:45 smithi167 ceph-mon[32751]: pgmap v22158: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:46.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:46 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40647.615201 secs 2024-04-27T05:52:46.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40648.531525 secs 2024-04-27T05:52:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:46 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40647.615201 secs 2024-04-27T05:52:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40648.531525 secs 2024-04-27T05:52:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:46 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40647.615201 secs 2024-04-27T05:52:47.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40648.531525 secs 2024-04-27T05:52:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:47 smithi090 ceph-mon[29180]: pgmap v22159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:47 smithi090 ceph-mon[29180]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:52:47.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:47 smithi090 ceph-mon[29180]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:52:48.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:47 smithi119 ceph-mon[32752]: pgmap v22159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:48.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:47 smithi119 ceph-mon[32752]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:52:48.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:47 smithi119 ceph-mon[32752]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:52:48.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:47 smithi167 ceph-mon[32751]: pgmap v22159: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:48.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:47 smithi167 ceph-mon[32751]: mds.? [v2:172.21.15.119:6838/1002360318,v1:172.21.15.119:6839/1002360318] up:active 2024-04-27T05:52:48.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:47 smithi167 ceph-mon[32751]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=j=up:active(laggy or crashed)} 1 up:standby-replay 2024-04-27T05:52:49.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:49 smithi090 ceph-mon[29180]: pgmap v22160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:50.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:49 smithi119 ceph-mon[32752]: pgmap v22160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:50.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:49 smithi167 ceph-mon[32751]: pgmap v22160: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:51 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40652.615263 secs 2024-04-27T05:52:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:51 smithi090 ceph-mon[29180]: pgmap v22161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:51.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:51 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40653.531614 secs 2024-04-27T05:52:52.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:51 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40652.615263 secs 2024-04-27T05:52:52.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:51 smithi119 ceph-mon[32752]: pgmap v22161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:52.040 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:51 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40653.531614 secs 2024-04-27T05:52:52.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:51 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40652.615263 secs 2024-04-27T05:52:52.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:51 smithi167 ceph-mon[32751]: pgmap v22161: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:52.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:51 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40653.531614 secs 2024-04-27T05:52:53.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:53 smithi090 ceph-mon[29180]: pgmap v22162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:52:54.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:53 smithi119 ceph-mon[32752]: pgmap v22162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:52:54.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:53 smithi167 ceph-mon[32751]: pgmap v22162: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-27T05:52:55.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:55 smithi090 ceph-mon[29180]: pgmap v22163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:56.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:55 smithi119 ceph-mon[32752]: pgmap v22163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:56.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:55 smithi167 ceph-mon[32751]: pgmap v22163: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:52:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:56 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40657.615325 secs 2024-04-27T05:52:56.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:56 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40658.531787 secs 2024-04-27T05:52:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:56 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40657.615325 secs 2024-04-27T05:52:57.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:56 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40658.531787 secs 2024-04-27T05:52:57.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:56 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40657.615325 secs 2024-04-27T05:52:57.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:56 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40658.531787 secs 2024-04-27T05:52:57.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:57 smithi090 ceph-mon[29180]: pgmap v22164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:58.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:57 smithi119 ceph-mon[32752]: pgmap v22164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:52:58.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:57 smithi167 ceph-mon[32751]: pgmap v22164: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:00.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:52:59 smithi119 ceph-mon[32752]: pgmap v22165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:00.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:52:59 smithi167 ceph-mon[32751]: pgmap v22165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:00.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:52:59 smithi090 ceph-mon[29180]: pgmap v22165: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:02.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:01 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40662.615408 secs 2024-04-27T05:53:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:01 smithi119 ceph-mon[32752]: pgmap v22166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:02.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:01 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40663.531915 secs 2024-04-27T05:53:02.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:01 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40662.615408 secs 2024-04-27T05:53:02.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:01 smithi167 ceph-mon[32751]: pgmap v22166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:02.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:01 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40663.531915 secs 2024-04-27T05:53:02.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:01 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40662.615408 secs 2024-04-27T05:53:02.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:01 smithi090 ceph-mon[29180]: pgmap v22166: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:02.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:01 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40663.531915 secs 2024-04-27T05:53:04.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:03 smithi119 ceph-mon[32752]: pgmap v22167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:04.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:03 smithi167 ceph-mon[32751]: pgmap v22167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:04.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:03 smithi090 ceph-mon[29180]: pgmap v22167: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:05.860 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:05 smithi090 ceph-mon[29180]: pgmap v22168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:06.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:05 smithi119 ceph-mon[32752]: pgmap v22168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:06.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:05 smithi167 ceph-mon[32751]: pgmap v22168: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:07.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:06 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40667.615493 secs 2024-04-27T05:53:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:06 smithi119 ceph-mon[32752]: 35 slow requests, 1 included below; oldest blocked for > 40668.532002 secs 2024-04-27T05:53:07.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:06 smithi119 ceph-mon[32752]: slow request 23040.672114 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:53:07.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:06 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40667.615493 secs 2024-04-27T05:53:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:06 smithi167 ceph-mon[32751]: 35 slow requests, 1 included below; oldest blocked for > 40668.532002 secs 2024-04-27T05:53:07.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:06 smithi167 ceph-mon[32751]: slow request 23040.672114 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:53:07.062 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:06 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40667.615493 secs 2024-04-27T05:53:07.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:06 smithi090 ceph-mon[29180]: 35 slow requests, 1 included below; oldest blocked for > 40668.532002 secs 2024-04-27T05:53:07.063 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:06 smithi090 ceph-mon[29180]: slow request 23040.672114 seconds old, received at 2024-04-26T23:29:05.610533+0000: client_request(client.17439:1 getattr pAsLsXsFs #0x1 2024-04-26T23:29:05.603874+0000 caller_uid=0, caller_gid=0{}) currently cleaned up request 2024-04-27T05:53:07.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:07 smithi090 ceph-mon[29180]: pgmap v22169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:08.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:07 smithi119 ceph-mon[32752]: pgmap v22169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:08.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:07 smithi167 ceph-mon[32751]: pgmap v22169: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:10.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:09 smithi119 ceph-mon[32752]: pgmap v22170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:10.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:09 smithi167 ceph-mon[32751]: pgmap v22170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:10.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:09 smithi090 ceph-mon[29180]: pgmap v22170: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:12.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:11 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40672.615587 secs 2024-04-27T05:53:12.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:11 smithi119 ceph-mon[32752]: pgmap v22171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:12.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:11 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40673.532079 secs 2024-04-27T05:53:12.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:11 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40672.615587 secs 2024-04-27T05:53:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:11 smithi167 ceph-mon[32751]: pgmap v22171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:12.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:11 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40673.532079 secs 2024-04-27T05:53:12.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:11 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40672.615587 secs 2024-04-27T05:53:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:11 smithi090 ceph-mon[29180]: pgmap v22171: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:12.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:11 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40673.532079 secs 2024-04-27T05:53:14.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:13 smithi119 ceph-mon[32752]: pgmap v22172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:14.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:13 smithi167 ceph-mon[32751]: pgmap v22172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:14.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:13 smithi090 ceph-mon[29180]: pgmap v22172: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:16.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:15 smithi119 ceph-mon[32752]: pgmap v22173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:16.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:15 smithi167 ceph-mon[32751]: pgmap v22173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:16.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:15 smithi090 ceph-mon[29180]: pgmap v22173: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:16.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:16 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40677.615664 secs 2024-04-27T05:53:16.861 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:16 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40678.532145 secs 2024-04-27T05:53:17.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:16 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40677.615664 secs 2024-04-27T05:53:17.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:16 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40678.532145 secs 2024-04-27T05:53:17.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:16 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40677.615664 secs 2024-04-27T05:53:17.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:16 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40678.532145 secs 2024-04-27T05:53:18.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:17 smithi119 ceph-mon[32752]: pgmap v22174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:18.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:17 smithi167 ceph-mon[32751]: pgmap v22174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:18.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:17 smithi090 ceph-mon[29180]: pgmap v22174: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:20.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:19 smithi119 ceph-mon[32752]: pgmap v22175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:20.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:19 smithi167 ceph-mon[32751]: pgmap v22175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:20.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:19 smithi090 ceph-mon[29180]: pgmap v22175: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:22.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:21 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40682.615749 secs 2024-04-27T05:53:22.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:21 smithi119 ceph-mon[32752]: pgmap v22176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:22.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:21 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40683.532255 secs 2024-04-27T05:53:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:21 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40682.615749 secs 2024-04-27T05:53:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:21 smithi167 ceph-mon[32751]: pgmap v22176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:22.053 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:21 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40683.532255 secs 2024-04-27T05:53:22.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:21 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40682.615749 secs 2024-04-27T05:53:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:21 smithi090 ceph-mon[29180]: pgmap v22176: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:22.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:21 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40683.532255 secs 2024-04-27T05:53:24.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:23 smithi119 ceph-mon[32752]: pgmap v22177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:24.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:23 smithi167 ceph-mon[32751]: pgmap v22177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:24.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:23 smithi090 ceph-mon[29180]: pgmap v22177: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:26.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:25 smithi119 ceph-mon[32752]: pgmap v22178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:26.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:25 smithi167 ceph-mon[32751]: pgmap v22178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:26.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:25 smithi090 ceph-mon[29180]: pgmap v22178: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:27.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:26 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40687.615836 secs 2024-04-27T05:53:27.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:26 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40688.532359 secs 2024-04-27T05:53:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:26 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40687.615836 secs 2024-04-27T05:53:27.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:26 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40688.532359 secs 2024-04-27T05:53:27.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:26 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40687.615836 secs 2024-04-27T05:53:27.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:26 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40688.532359 secs 2024-04-27T05:53:28.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:27 smithi119 ceph-mon[32752]: pgmap v22179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:28.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:27 smithi167 ceph-mon[32751]: pgmap v22179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:28.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:27 smithi090 ceph-mon[29180]: pgmap v22179: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:30.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:29 smithi119 ceph-mon[32752]: pgmap v22180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:30.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:29 smithi167 ceph-mon[32751]: pgmap v22180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:30.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:29 smithi090 ceph-mon[29180]: pgmap v22180: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:32.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:31 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40692.615920 secs 2024-04-27T05:53:32.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:31 smithi119 ceph-mon[32752]: pgmap v22181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:32.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:31 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40693.532501 secs 2024-04-27T05:53:32.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:31 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40692.615920 secs 2024-04-27T05:53:32.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:31 smithi167 ceph-mon[32751]: pgmap v22181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:32.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:31 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40693.532501 secs 2024-04-27T05:53:32.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:31 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40692.615920 secs 2024-04-27T05:53:32.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:31 smithi090 ceph-mon[29180]: pgmap v22181: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:32.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:31 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40693.532501 secs 2024-04-27T05:53:34.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:33 smithi119 ceph-mon[32752]: pgmap v22182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:34.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:33 smithi167 ceph-mon[32751]: pgmap v22182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:34.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:33 smithi090 ceph-mon[29180]: pgmap v22182: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:36.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:35 smithi119 ceph-mon[32752]: pgmap v22183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:36.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:35 smithi167 ceph-mon[32751]: pgmap v22183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:36.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:35 smithi090 ceph-mon[29180]: pgmap v22183: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:37.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:36 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40697.615983 secs 2024-04-27T05:53:37.013 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:36 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40698.532573 secs 2024-04-27T05:53:37.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:36 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40697.615983 secs 2024-04-27T05:53:37.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:36 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40698.532573 secs 2024-04-27T05:53:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:36 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40697.615983 secs 2024-04-27T05:53:37.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:36 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40698.532573 secs 2024-04-27T05:53:38.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:37 smithi119 ceph-mon[32752]: pgmap v22184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:38.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:37 smithi167 ceph-mon[32751]: pgmap v22184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:38.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:37 smithi090 ceph-mon[29180]: pgmap v22184: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:40.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:39 smithi119 ceph-mon[32752]: pgmap v22185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:40.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:39 smithi167 ceph-mon[32751]: pgmap v22185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:40.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:39 smithi090 ceph-mon[29180]: pgmap v22185: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:42.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:41 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40702.616046 secs 2024-04-27T05:53:42.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:41 smithi119 ceph-mon[32752]: pgmap v22186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:42.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:41 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40703.532686 secs 2024-04-27T05:53:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:41 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40702.616046 secs 2024-04-27T05:53:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:41 smithi167 ceph-mon[32751]: pgmap v22186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:42.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:41 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40703.532686 secs 2024-04-27T05:53:42.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:41 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40702.616046 secs 2024-04-27T05:53:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:41 smithi090 ceph-mon[29180]: pgmap v22186: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:42.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:41 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40703.532686 secs 2024-04-27T05:53:43.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:42 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:53:43.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:42 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:53:43.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:42 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-27T05:53:44.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:43 smithi119 ceph-mon[32752]: pgmap v22187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:44.052 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:43 smithi167 ceph-mon[32751]: pgmap v22187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:44.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:43 smithi090 ceph-mon[29180]: pgmap v22187: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:45.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:45 smithi167 ceph-mon[32751]: pgmap v22188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:46.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:45 smithi119 ceph-mon[32752]: pgmap v22188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:46.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:45 smithi090 ceph-mon[29180]: pgmap v22188: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:46.940 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:46 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40707.616107 secs 2024-04-27T05:53:46.940 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:46 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40708.532784 secs 2024-04-27T05:53:47.038 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:46 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40707.616107 secs 2024-04-27T05:53:47.039 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:46 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40708.532784 secs 2024-04-27T05:53:47.110 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:46 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40707.616107 secs 2024-04-27T05:53:47.111 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:46 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40708.532784 secs 2024-04-27T05:53:48.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:48 smithi119 ceph-mon[32752]: pgmap v22189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:48.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:53:48.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:53:48.289 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:48 smithi119 ceph-mon[32752]: pgmap v22190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:48.290 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:48 smithi119 ceph-mon[32752]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:48 smithi167 ceph-mon[32751]: pgmap v22189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:48.302 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:53:48.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:53:48.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:48 smithi167 ceph-mon[32751]: pgmap v22190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:48.303 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:48 smithi167 ceph-mon[32751]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:48 smithi090 ceph-mon[29180]: pgmap v22189: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:53:48.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:48.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-27T05:53:48.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-27T05:53:48.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:48 smithi090 ceph-mon[29180]: pgmap v22190: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:48.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:48 smithi090 ceph-mon[29180]: from='mgr.24752 172.21.15.90:0/1777062471' entity='mgr.x' 2024-04-27T05:53:49.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:49 smithi090 ceph-mon[29180]: pgmap v22191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:49.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:49 smithi119 ceph-mon[32752]: pgmap v22191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:49.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:49 smithi167 ceph-mon[32751]: pgmap v22191: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:51.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:51 smithi090 ceph-mon[29180]: pgmap v22192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:51.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:51 smithi119 ceph-mon[32752]: pgmap v22192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:51.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:51 smithi167 ceph-mon[32751]: pgmap v22192: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:52.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:52 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40712.616195 secs 2024-04-27T05:53:52.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:52 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40713.532891 secs 2024-04-27T05:53:52.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:52 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40712.616195 secs 2024-04-27T05:53:52.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:52 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40713.532891 secs 2024-04-27T05:53:52.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:52 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40712.616195 secs 2024-04-27T05:53:52.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:52 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40713.532891 secs 2024-04-27T05:53:53.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:53 smithi090 ceph-mon[29180]: pgmap v22193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:53.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:53 smithi119 ceph-mon[32752]: pgmap v22193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:53.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:53 smithi167 ceph-mon[32751]: pgmap v22193: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:55.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:55 smithi090 ceph-mon[29180]: pgmap v22194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:55.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:55 smithi119 ceph-mon[32752]: pgmap v22194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:55.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:55 smithi167 ceph-mon[32751]: pgmap v22194: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:56.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:56 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40717.616255 secs 2024-04-27T05:53:56.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:56 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40717.616255 secs 2024-04-27T05:53:56.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:56 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40717.616255 secs 2024-04-27T05:53:57.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:57 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40718.533042 secs 2024-04-27T05:53:57.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:57 smithi090 ceph-mon[29180]: pgmap v22195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:57.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:57 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40718.533042 secs 2024-04-27T05:53:57.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:57 smithi119 ceph-mon[32752]: pgmap v22195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:57.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:57 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40718.533042 secs 2024-04-27T05:53:57.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:57 smithi167 ceph-mon[32751]: pgmap v22195: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 976 B/s rd, 1 op/s 2024-04-27T05:53:59.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:53:59 smithi090 ceph-mon[29180]: pgmap v22196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:59.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:53:59 smithi119 ceph-mon[32752]: pgmap v22196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:53:59.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:53:59 smithi167 ceph-mon[32751]: pgmap v22196: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:01.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:01 smithi090 ceph-mon[29180]: pgmap v22197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:01.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:01 smithi119 ceph-mon[32752]: pgmap v22197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:01.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:01 smithi167 ceph-mon[32751]: pgmap v22197: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:02.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:02 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40722.616317 secs 2024-04-27T05:54:02.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:02 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40723.533166 secs 2024-04-27T05:54:02.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:02 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40722.616317 secs 2024-04-27T05:54:02.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:02 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40723.533166 secs 2024-04-27T05:54:02.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:02 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40722.616317 secs 2024-04-27T05:54:02.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:02 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40723.533166 secs 2024-04-27T05:54:03.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:03 smithi090 ceph-mon[29180]: pgmap v22198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:03.453 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:03 smithi167 ceph-mon[32751]: pgmap v22198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:03.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:03 smithi119 ceph-mon[32752]: pgmap v22198: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:05.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:05 smithi090 ceph-mon[29180]: pgmap v22199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:05.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:05 smithi119 ceph-mon[32752]: pgmap v22199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:05 smithi167 ceph-mon[32751]: pgmap v22199: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:06.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:06 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40727.616402 secs 2024-04-27T05:54:06.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:06 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40727.616402 secs 2024-04-27T05:54:06.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:06 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40727.616402 secs 2024-04-27T05:54:07.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:07 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40728.533287 secs 2024-04-27T05:54:07.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:07 smithi090 ceph-mon[29180]: pgmap v22200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:07.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:07 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40728.533287 secs 2024-04-27T05:54:07.788 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:07 smithi119 ceph-mon[32752]: pgmap v22200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:07.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:07 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40728.533287 secs 2024-04-27T05:54:07.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:07 smithi167 ceph-mon[32751]: pgmap v22200: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:09.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:09 smithi090 ceph-mon[29180]: pgmap v22201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:09.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:09 smithi119 ceph-mon[32752]: pgmap v22201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:09.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:09 smithi167 ceph-mon[32751]: pgmap v22201: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:11.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:11 smithi090 ceph-mon[29180]: pgmap v22202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:11.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:11 smithi119 ceph-mon[32752]: pgmap v22202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:11.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:11 smithi167 ceph-mon[32751]: pgmap v22202: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:12.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:12 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40732.616486 secs 2024-04-27T05:54:12.361 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:12 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40733.533410 secs 2024-04-27T05:54:12.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:12 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40732.616486 secs 2024-04-27T05:54:12.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:12 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40733.533410 secs 2024-04-27T05:54:12.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:12 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40732.616486 secs 2024-04-27T05:54:12.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:12 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40733.533410 secs 2024-04-27T05:54:13.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:13 smithi090 ceph-mon[29180]: pgmap v22203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:13.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:13 smithi119 ceph-mon[32752]: pgmap v22203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:13.554 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:13 smithi167 ceph-mon[32751]: pgmap v22203: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:16.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:16 smithi090 ceph-mon[29180]: pgmap v22204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:16.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:16 smithi119 ceph-mon[32752]: pgmap v22204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:16.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:16 smithi167 ceph-mon[32751]: pgmap v22204: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:17.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:17 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40737.616572 secs 2024-04-27T05:54:17.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:17 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40738.533524 secs 2024-04-27T05:54:17.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:17 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40737.616572 secs 2024-04-27T05:54:17.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:17 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40738.533524 secs 2024-04-27T05:54:17.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:17 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40737.616572 secs 2024-04-27T05:54:17.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:17 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40738.533524 secs 2024-04-27T05:54:18.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:18 smithi090 ceph-mon[29180]: pgmap v22205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:18.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:18 smithi119 ceph-mon[32752]: pgmap v22205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:18.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:18 smithi167 ceph-mon[32751]: pgmap v22205: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:20.360 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:20 smithi090 ceph-mon[29180]: pgmap v22206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:20.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:20 smithi119 ceph-mon[32752]: pgmap v22206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:20.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:20 smithi167 ceph-mon[32751]: pgmap v22206: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-27T05:54:22.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:22 smithi119 ceph-mon[32752]: 29 slow requests, 0 included below; oldest blocked for > 40742.616658 secs 2024-04-27T05:54:22.538 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:22 smithi119 ceph-mon[32752]: pgmap v22207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:22.539 INFO:journalctl@ceph.mon.b.smithi119.stdout:Apr 27 05:54:22 smithi119 ceph-mon[32752]: 35 slow requests, 0 included below; oldest blocked for > 40743.533622 secs 2024-04-27T05:54:22.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:22 smithi167 ceph-mon[32751]: 29 slow requests, 0 included below; oldest blocked for > 40742.616658 secs 2024-04-27T05:54:22.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:22 smithi167 ceph-mon[32751]: pgmap v22207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:22.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Apr 27 05:54:22 smithi167 ceph-mon[32751]: 35 slow requests, 0 included below; oldest blocked for > 40743.533622 secs 2024-04-27T05:54:22.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:22 smithi090 ceph-mon[29180]: 29 slow requests, 0 included below; oldest blocked for > 40742.616658 secs 2024-04-27T05:54:22.610 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:22 smithi090 ceph-mon[29180]: pgmap v22207: 97 pgs: 97 active+clean; 847 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-27T05:54:22.611 INFO:journalctl@ceph.mon.a.smithi090.stdout:Apr 27 05:54:22 smithi090 ceph-mon[29180]: 35 slow requests, 0 included below; oldest blocked for > 40743.533622 secs 2024-04-27T05:54:23.016 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-04-27T05:54:23.016 DEBUG:teuthology.task.console_log:Killing console logger for smithi090 2024-04-27T05:54:23.017 DEBUG:teuthology.task.console_log:Killing console logger for smithi119 2024-04-27T05:54:23.017 DEBUG:teuthology.task.console_log:Killing console logger for smithi167 2024-04-27T05:54:23.017 DEBUG:teuthology.exit:Finished running handlers